2024-09-19T08:24:13.067 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-19T08:24:13.091 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T08:24:13.178 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911853 branch: squid description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7911853' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1747 sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 sleep_before_teardown: 0 subset: 176/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi098.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI0s7Pcn1Or8WA8LA8yVHACWdXDnufadAAvOyqzmmzcAXIN5k0qRndsH5KzqnNT+h8/niDbqVLVOD3deu3EQXOM= smithi139.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNmn9o8aIuWzejSIIKW4rGU49scBUrXd9kUOp0BdeAut7acetTbX5heNkmsXfCAkBsmuw3wcFw/c9hxU3+ddSo4= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestMisc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-18_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-19T08:24:13.178 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa; will attempt to use it 2024-09-19T08:24:13.179 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks 2024-09-19T08:24:13.179 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-19T08:24:13.181 INFO:teuthology.task.internal:Checking packages... 2024-09-19T08:24:13.204 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '2f599a46ad10e37c929e5c43da2f6907ddcd1961' 2024-09-19T08:24:13.204 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-19T08:24:13.204 INFO:teuthology.packaging:ref: None 2024-09-19T08:24:13.204 INFO:teuthology.packaging:tag: None 2024-09-19T08:24:13.205 INFO:teuthology.packaging:branch: squid 2024-09-19T08:24:13.205 INFO:teuthology.packaging:sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:24:13.205 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=squid 2024-09-19T08:24:13.450 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-363-g6fad5dd7-1jammy 2024-09-19T08:24:13.453 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-19T08:24:13.460 INFO:teuthology.task.internal:no buildpackages task found 2024-09-19T08:24:13.460 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-19T08:24:13.477 INFO:teuthology.task.internal:Saving configuration 2024-09-19T08:24:13.494 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-19T08:24:13.501 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-19T08:24:13.524 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi098.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911853', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 08:18:43.201279', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDyQruBftYJvE3BYyi87jmQ9naDt0IftsJ27MMYJdpJj'} 2024-09-19T08:24:13.545 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi139.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911853', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 08:18:43.203233', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBS9vaQQsvwzA3vj8qzAgOutRlBIsF6Rx/bz34U2hFMM'} 2024-09-19T08:24:13.546 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-19T08:24:13.552 INFO:teuthology.task.internal:roles: ubuntu@smithi098.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-19T08:24:13.552 INFO:teuthology.task.internal:roles: ubuntu@smithi139.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-19T08:24:13.552 INFO:teuthology.run_tasks:Running task console_log... 2024-09-19T08:24:13.615 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f04c1b770a0>, signals=[15]) 2024-09-19T08:24:13.616 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-19T08:24:13.622 INFO:teuthology.task.internal:Opening connections... 2024-09-19T08:24:13.622 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:24:13.623 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T08:24:13.704 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:24:13.704 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T08:24:13.781 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-19T08:24:13.788 DEBUG:teuthology.orchestra.run.smithi098:> uname -m 2024-09-19T08:24:13.793 INFO:teuthology.orchestra.run.smithi098.stdout:x86_64 2024-09-19T08:24:13.793 DEBUG:teuthology.orchestra.run.smithi098:> cat /etc/os-release 2024-09-19T08:24:13.845 INFO:teuthology.orchestra.run.smithi098.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-19T08:24:13.845 INFO:teuthology.orchestra.run.smithi098.stdout:NAME="Ubuntu" 2024-09-19T08:24:13.845 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION_ID="22.04" 2024-09-19T08:24:13.845 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-19T08:24:13.845 INFO:teuthology.orchestra.run.smithi098.stdout:VERSION_CODENAME=jammy 2024-09-19T08:24:13.845 INFO:teuthology.orchestra.run.smithi098.stdout:ID=ubuntu 2024-09-19T08:24:13.845 INFO:teuthology.orchestra.run.smithi098.stdout:ID_LIKE=debian 2024-09-19T08:24:13.845 INFO:teuthology.orchestra.run.smithi098.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-19T08:24:13.846 INFO:teuthology.orchestra.run.smithi098.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-19T08:24:13.846 INFO:teuthology.orchestra.run.smithi098.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-19T08:24:13.846 INFO:teuthology.orchestra.run.smithi098.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-19T08:24:13.846 INFO:teuthology.orchestra.run.smithi098.stdout:UBUNTU_CODENAME=jammy 2024-09-19T08:24:13.846 INFO:teuthology.lock.ops:Updating smithi098.front.sepia.ceph.com on lock server 2024-09-19T08:24:13.867 DEBUG:teuthology.orchestra.run.smithi139:> uname -m 2024-09-19T08:24:13.872 INFO:teuthology.orchestra.run.smithi139.stdout:x86_64 2024-09-19T08:24:13.873 DEBUG:teuthology.orchestra.run.smithi139:> cat /etc/os-release 2024-09-19T08:24:13.917 INFO:teuthology.orchestra.run.smithi139.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-19T08:24:13.918 INFO:teuthology.orchestra.run.smithi139.stdout:NAME="Ubuntu" 2024-09-19T08:24:13.918 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION_ID="22.04" 2024-09-19T08:24:13.918 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-19T08:24:13.918 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION_CODENAME=jammy 2024-09-19T08:24:13.918 INFO:teuthology.orchestra.run.smithi139.stdout:ID=ubuntu 2024-09-19T08:24:13.918 INFO:teuthology.orchestra.run.smithi139.stdout:ID_LIKE=debian 2024-09-19T08:24:13.918 INFO:teuthology.orchestra.run.smithi139.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-19T08:24:13.918 INFO:teuthology.orchestra.run.smithi139.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-19T08:24:13.918 INFO:teuthology.orchestra.run.smithi139.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-19T08:24:13.918 INFO:teuthology.orchestra.run.smithi139.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-19T08:24:13.918 INFO:teuthology.orchestra.run.smithi139.stdout:UBUNTU_CODENAME=jammy 2024-09-19T08:24:13.919 INFO:teuthology.lock.ops:Updating smithi139.front.sepia.ceph.com on lock server 2024-09-19T08:24:13.942 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-19T08:24:13.951 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-19T08:24:13.957 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-19T08:24:13.958 DEBUG:teuthology.orchestra.run.smithi098:> test '!' -e /home/ubuntu/cephtest 2024-09-19T08:24:13.960 DEBUG:teuthology.orchestra.run.smithi139:> test '!' -e /home/ubuntu/cephtest 2024-09-19T08:24:13.965 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-19T08:24:13.975 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-19T08:24:13.976 DEBUG:teuthology.orchestra.run.smithi098:> test -z $(ls -A /var/lib/ceph) 2024-09-19T08:24:14.006 DEBUG:teuthology.orchestra.run.smithi139:> test -z $(ls -A /var/lib/ceph) 2024-09-19T08:24:14.019 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-19T08:24:14.078 INFO:teuthology.run_tasks:Running task kernel... 2024-09-19T08:24:14.095 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-19T08:24:14.095 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-19T08:24:14.095 DEBUG:teuthology.orchestra.run.smithi098:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T08:24:14.096 DEBUG:teuthology.orchestra.run.smithi139:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T08:24:14.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:24:14.100 DEBUG:teuthology.orchestra.run.smithi098:> uname -r 2024-09-19T08:24:14.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:24:14.102 DEBUG:teuthology.orchestra.run.smithi139:> uname -r 2024-09-19T08:24:14.149 INFO:teuthology.orchestra.run.smithi098.stdout:5.15.0-56-generic 2024-09-19T08:24:14.149 INFO:teuthology.task.kernel:Running kernel on smithi098: 5.15.0-56-generic 2024-09-19T08:24:14.149 DEBUG:teuthology.orchestra.run.smithi098:> sudo apt-get clean 2024-09-19T08:24:14.151 INFO:teuthology.orchestra.run.smithi139.stdout:5.15.0-56-generic 2024-09-19T08:24:14.151 INFO:teuthology.task.kernel:Running kernel on smithi139: 5.15.0-56-generic 2024-09-19T08:24:14.151 DEBUG:teuthology.orchestra.run.smithi139:> sudo apt-get clean 2024-09-19T08:24:14.273 DEBUG:teuthology.orchestra.run.smithi098:> sudo apt-get update 2024-09-19T08:24:14.297 DEBUG:teuthology.orchestra.run.smithi139:> sudo apt-get update 2024-09-19T08:24:14.438 INFO:teuthology.orchestra.run.smithi139.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-19T08:24:14.439 INFO:teuthology.orchestra.run.smithi139.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-19T08:24:14.439 INFO:teuthology.orchestra.run.smithi139.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-19T08:24:14.512 INFO:teuthology.orchestra.run.smithi139.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-19T08:24:14.540 INFO:teuthology.orchestra.run.smithi098.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-19T08:24:14.540 INFO:teuthology.orchestra.run.smithi098.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-19T08:24:14.601 INFO:teuthology.orchestra.run.smithi098.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-19T08:24:14.687 INFO:teuthology.orchestra.run.smithi139.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,839 kB] 2024-09-19T08:24:14.756 INFO:teuthology.orchestra.run.smithi139.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,058 kB] 2024-09-19T08:24:14.786 INFO:teuthology.orchestra.run.smithi139.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [544 kB] 2024-09-19T08:24:14.789 INFO:teuthology.orchestra.run.smithi139.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [298 kB] 2024-09-19T08:24:14.792 INFO:teuthology.orchestra.run.smithi139.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-19T08:24:14.792 INFO:teuthology.orchestra.run.smithi139.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-19T08:24:14.792 INFO:teuthology.orchestra.run.smithi139.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,431 kB] 2024-09-19T08:24:14.818 INFO:teuthology.orchestra.run.smithi139.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [418 kB] 2024-09-19T08:24:14.822 INFO:teuthology.orchestra.run.smithi139.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-19T08:24:14.822 INFO:teuthology.orchestra.run.smithi139.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [628 kB] 2024-09-19T08:24:14.826 INFO:teuthology.orchestra.run.smithi139.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [903 kB] 2024-09-19T08:24:14.832 INFO:teuthology.orchestra.run.smithi139.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [177 kB] 2024-09-19T08:24:14.833 INFO:teuthology.orchestra.run.smithi139.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.3 kB] 2024-09-19T08:24:14.833 INFO:teuthology.orchestra.run.smithi139.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-19T08:24:14.844 INFO:teuthology.orchestra.run.smithi139.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [701 kB] 2024-09-19T08:24:14.844 INFO:teuthology.orchestra.run.smithi139.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-19T08:24:14.844 INFO:teuthology.orchestra.run.smithi139.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-19T08:24:14.845 INFO:teuthology.orchestra.run.smithi139.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-19T08:24:14.849 INFO:teuthology.orchestra.run.smithi139.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [355 kB] 2024-09-19T08:24:14.851 INFO:teuthology.orchestra.run.smithi139.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-19T08:24:14.851 INFO:teuthology.orchestra.run.smithi139.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-19T08:24:14.852 INFO:teuthology.orchestra.run.smithi139.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,495 kB] 2024-09-19T08:24:14.871 INFO:teuthology.orchestra.run.smithi139.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [429 kB] 2024-09-19T08:24:14.875 INFO:teuthology.orchestra.run.smithi139.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-19T08:24:14.875 INFO:teuthology.orchestra.run.smithi139.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-19T08:24:14.882 INFO:teuthology.orchestra.run.smithi139.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-19T08:24:14.891 INFO:teuthology.orchestra.run.smithi139.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-19T08:24:14.894 INFO:teuthology.orchestra.run.smithi139.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-19T08:24:14.894 INFO:teuthology.orchestra.run.smithi139.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-19T08:24:14.895 INFO:teuthology.orchestra.run.smithi139.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-19T08:24:14.895 INFO:teuthology.orchestra.run.smithi139.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-19T08:24:14.904 INFO:teuthology.orchestra.run.smithi139.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-19T08:24:14.904 INFO:teuthology.orchestra.run.smithi139.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-19T08:24:14.905 INFO:teuthology.orchestra.run.smithi139.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-19T08:24:14.905 INFO:teuthology.orchestra.run.smithi139.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-19T08:24:14.922 INFO:teuthology.orchestra.run.smithi139.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-19T08:24:14.922 INFO:teuthology.orchestra.run.smithi139.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-19T08:24:14.922 INFO:teuthology.orchestra.run.smithi139.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-19T08:24:14.923 INFO:teuthology.orchestra.run.smithi139.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-19T08:24:14.923 INFO:teuthology.orchestra.run.smithi139.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-19T08:24:14.957 INFO:teuthology.orchestra.run.smithi098.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-19T08:24:14.980 INFO:teuthology.orchestra.run.smithi098.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,839 kB] 2024-09-19T08:24:15.095 INFO:teuthology.orchestra.run.smithi098.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [701 kB] 2024-09-19T08:24:15.301 INFO:teuthology.orchestra.run.smithi098.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,058 kB] 2024-09-19T08:24:15.403 INFO:teuthology.orchestra.run.smithi098.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [544 kB] 2024-09-19T08:24:15.419 INFO:teuthology.orchestra.run.smithi098.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [298 kB] 2024-09-19T08:24:15.428 INFO:teuthology.orchestra.run.smithi098.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-19T08:24:15.429 INFO:teuthology.orchestra.run.smithi098.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-19T08:24:15.430 INFO:teuthology.orchestra.run.smithi098.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,431 kB] 2024-09-19T08:24:15.450 INFO:teuthology.orchestra.run.smithi098.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [355 kB] 2024-09-19T08:24:15.464 INFO:teuthology.orchestra.run.smithi098.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-19T08:24:15.464 INFO:teuthology.orchestra.run.smithi098.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,495 kB] 2024-09-19T08:24:15.526 INFO:teuthology.orchestra.run.smithi098.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [418 kB] 2024-09-19T08:24:15.534 INFO:teuthology.orchestra.run.smithi098.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-19T08:24:15.575 INFO:teuthology.orchestra.run.smithi098.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [429 kB] 2024-09-19T08:24:15.576 INFO:teuthology.orchestra.run.smithi098.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-19T08:24:15.576 INFO:teuthology.orchestra.run.smithi098.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [903 kB] 2024-09-19T08:24:15.583 INFO:teuthology.orchestra.run.smithi098.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-19T08:24:15.584 INFO:teuthology.orchestra.run.smithi098.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-19T08:24:15.592 INFO:teuthology.orchestra.run.smithi098.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [628 kB] 2024-09-19T08:24:15.597 INFO:teuthology.orchestra.run.smithi098.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-19T08:24:15.602 INFO:teuthology.orchestra.run.smithi098.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [177 kB] 2024-09-19T08:24:15.606 INFO:teuthology.orchestra.run.smithi098.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.3 kB] 2024-09-19T08:24:15.606 INFO:teuthology.orchestra.run.smithi098.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-19T08:24:15.608 INFO:teuthology.orchestra.run.smithi098.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-19T08:24:15.608 INFO:teuthology.orchestra.run.smithi098.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-19T08:24:15.615 INFO:teuthology.orchestra.run.smithi098.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-19T08:24:15.618 INFO:teuthology.orchestra.run.smithi098.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-19T08:24:15.619 INFO:teuthology.orchestra.run.smithi098.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-19T08:24:15.619 INFO:teuthology.orchestra.run.smithi098.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-19T08:24:15.665 INFO:teuthology.orchestra.run.smithi098.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-19T08:24:15.695 INFO:teuthology.orchestra.run.smithi098.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-19T08:24:15.696 INFO:teuthology.orchestra.run.smithi098.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-19T08:24:15.696 INFO:teuthology.orchestra.run.smithi098.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-19T08:24:15.697 INFO:teuthology.orchestra.run.smithi098.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-19T08:24:15.698 INFO:teuthology.orchestra.run.smithi098.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-19T08:24:15.776 INFO:teuthology.orchestra.run.smithi098.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-19T08:24:15.777 INFO:teuthology.orchestra.run.smithi098.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-19T08:24:15.777 INFO:teuthology.orchestra.run.smithi098.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-19T08:24:15.778 INFO:teuthology.orchestra.run.smithi098.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-19T08:24:15.778 INFO:teuthology.orchestra.run.smithi098.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-19T08:24:21.205 INFO:teuthology.orchestra.run.smithi139.stdout:Fetched 16.2 MB in 2s (7,389 kB/s) 2024-09-19T08:24:21.598 INFO:teuthology.orchestra.run.smithi098.stdout:Fetched 16.2 MB in 3s (5,910 kB/s) 2024-09-19T08:24:22.644 INFO:teuthology.orchestra.run.smithi139.stdout:Reading package lists... 2024-09-19T08:24:22.661 DEBUG:teuthology.orchestra.run.smithi139:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-19T08:24:22.713 INFO:teuthology.orchestra.run.smithi139.stdout:Reading package lists... 2024-09-19T08:24:22.903 INFO:teuthology.orchestra.run.smithi139.stdout:Building dependency tree... 2024-09-19T08:24:22.903 INFO:teuthology.orchestra.run.smithi139.stdout:Reading state information... 2024-09-19T08:24:23.037 INFO:teuthology.orchestra.run.smithi098.stdout:Reading package lists... 2024-09-19T08:24:23.055 DEBUG:teuthology.orchestra.run.smithi098:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-19T08:24:23.070 INFO:teuthology.orchestra.run.smithi139.stdout:The following packages were automatically installed and are no longer required: 2024-09-19T08:24:23.070 INFO:teuthology.orchestra.run.smithi139.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-19T08:24:23.070 INFO:teuthology.orchestra.run.smithi139.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-19T08:24:23.071 INFO:teuthology.orchestra.run.smithi139.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-19T08:24:23.071 INFO:teuthology.orchestra.run.smithi139.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-19T08:24:23.071 INFO:teuthology.orchestra.run.smithi139.stdout: python2.7-minimal 2024-09-19T08:24:23.071 INFO:teuthology.orchestra.run.smithi139.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-19T08:24:23.072 INFO:teuthology.orchestra.run.smithi139.stdout:The following additional packages will be installed: 2024-09-19T08:24:23.072 INFO:teuthology.orchestra.run.smithi139.stdout: linux-generic linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-19T08:24:23.072 INFO:teuthology.orchestra.run.smithi139.stdout: linux-headers-generic linux-image-5.15.0-122-generic 2024-09-19T08:24:23.073 INFO:teuthology.orchestra.run.smithi139.stdout: linux-modules-5.15.0-122-generic linux-modules-extra-5.15.0-122-generic 2024-09-19T08:24:23.074 INFO:teuthology.orchestra.run.smithi139.stdout:Suggested packages: 2024-09-19T08:24:23.074 INFO:teuthology.orchestra.run.smithi139.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-19T08:24:23.074 INFO:teuthology.orchestra.run.smithi139.stdout:Recommended packages: 2024-09-19T08:24:23.074 INFO:teuthology.orchestra.run.smithi139.stdout: thermald 2024-09-19T08:24:23.116 INFO:teuthology.orchestra.run.smithi139.stdout:The following NEW packages will be installed: 2024-09-19T08:24:23.116 INFO:teuthology.orchestra.run.smithi139.stdout: linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-19T08:24:23.116 INFO:teuthology.orchestra.run.smithi139.stdout: linux-image-5.15.0-122-generic linux-modules-5.15.0-122-generic 2024-09-19T08:24:23.117 INFO:teuthology.orchestra.run.smithi098.stdout:Reading package lists... 2024-09-19T08:24:23.117 INFO:teuthology.orchestra.run.smithi139.stdout: linux-modules-extra-5.15.0-122-generic 2024-09-19T08:24:23.118 INFO:teuthology.orchestra.run.smithi139.stdout:The following packages will be upgraded: 2024-09-19T08:24:23.119 INFO:teuthology.orchestra.run.smithi139.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-19T08:24:23.297 INFO:teuthology.orchestra.run.smithi139.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-19T08:24:23.297 INFO:teuthology.orchestra.run.smithi139.stdout:Need to get 113 MB of archives. 2024-09-19T08:24:23.298 INFO:teuthology.orchestra.run.smithi139.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-19T08:24:23.298 INFO:teuthology.orchestra.run.smithi139.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-122-generic amd64 5.15.0-122.132 [22.7 MB] 2024-09-19T08:24:23.327 INFO:teuthology.orchestra.run.smithi098.stdout:Building dependency tree... 2024-09-19T08:24:23.328 INFO:teuthology.orchestra.run.smithi098.stdout:Reading state information... 2024-09-19T08:24:23.508 INFO:teuthology.orchestra.run.smithi098.stdout:The following packages were automatically installed and are no longer required: 2024-09-19T08:24:23.508 INFO:teuthology.orchestra.run.smithi098.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-19T08:24:23.509 INFO:teuthology.orchestra.run.smithi098.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-19T08:24:23.509 INFO:teuthology.orchestra.run.smithi098.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-19T08:24:23.509 INFO:teuthology.orchestra.run.smithi098.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-19T08:24:23.509 INFO:teuthology.orchestra.run.smithi098.stdout: python2.7-minimal 2024-09-19T08:24:23.509 INFO:teuthology.orchestra.run.smithi098.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-19T08:24:23.510 INFO:teuthology.orchestra.run.smithi098.stdout:The following additional packages will be installed: 2024-09-19T08:24:23.510 INFO:teuthology.orchestra.run.smithi098.stdout: linux-generic linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-19T08:24:23.510 INFO:teuthology.orchestra.run.smithi098.stdout: linux-headers-generic linux-image-5.15.0-122-generic 2024-09-19T08:24:23.510 INFO:teuthology.orchestra.run.smithi098.stdout: linux-modules-5.15.0-122-generic linux-modules-extra-5.15.0-122-generic 2024-09-19T08:24:23.512 INFO:teuthology.orchestra.run.smithi098.stdout:Suggested packages: 2024-09-19T08:24:23.512 INFO:teuthology.orchestra.run.smithi098.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-19T08:24:23.512 INFO:teuthology.orchestra.run.smithi098.stdout:Recommended packages: 2024-09-19T08:24:23.512 INFO:teuthology.orchestra.run.smithi098.stdout: thermald 2024-09-19T08:24:23.551 INFO:teuthology.orchestra.run.smithi098.stdout:The following NEW packages will be installed: 2024-09-19T08:24:23.552 INFO:teuthology.orchestra.run.smithi098.stdout: linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-19T08:24:23.552 INFO:teuthology.orchestra.run.smithi098.stdout: linux-image-5.15.0-122-generic linux-modules-5.15.0-122-generic 2024-09-19T08:24:23.552 INFO:teuthology.orchestra.run.smithi098.stdout: linux-modules-extra-5.15.0-122-generic 2024-09-19T08:24:23.553 INFO:teuthology.orchestra.run.smithi098.stdout:The following packages will be upgraded: 2024-09-19T08:24:23.554 INFO:teuthology.orchestra.run.smithi098.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-19T08:24:23.605 INFO:teuthology.orchestra.run.smithi098.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-19T08:24:23.605 INFO:teuthology.orchestra.run.smithi098.stdout:Need to get 113 MB of archives. 2024-09-19T08:24:23.605 INFO:teuthology.orchestra.run.smithi098.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-19T08:24:23.605 INFO:teuthology.orchestra.run.smithi098.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-122-generic amd64 5.15.0-122.132 [22.7 MB] 2024-09-19T08:24:24.077 INFO:teuthology.orchestra.run.smithi098.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-122-generic amd64 5.15.0-122.132 [11.5 MB] 2024-09-19T08:24:24.179 INFO:teuthology.orchestra.run.smithi098.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-122-generic amd64 5.15.0-122.132 [63.9 MB] 2024-09-19T08:24:24.716 INFO:teuthology.orchestra.run.smithi139.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-122-generic amd64 5.15.0-122.132 [11.5 MB] 2024-09-19T08:24:25.011 INFO:teuthology.orchestra.run.smithi098.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.122.122 [1,692 B] 2024-09-19T08:24:25.011 INFO:teuthology.orchestra.run.smithi098.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.122.122 [2,436 B] 2024-09-19T08:24:25.011 INFO:teuthology.orchestra.run.smithi098.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122 all 5.15.0-122.132 [12.3 MB] 2024-09-19T08:24:25.066 INFO:teuthology.orchestra.run.smithi139.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-122-generic amd64 5.15.0-122.132 [63.9 MB] 2024-09-19T08:24:25.133 INFO:teuthology.orchestra.run.smithi098.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122-generic amd64 5.15.0-122.132 [2,827 kB] 2024-09-19T08:24:25.182 INFO:teuthology.orchestra.run.smithi098.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.122.122 [2,294 B] 2024-09-19T08:24:25.584 INFO:teuthology.orchestra.run.smithi098.stdout:Fetched 113 MB in 2s (69.9 MB/s) 2024-09-19T08:24:25.756 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package linux-modules-5.15.0-122-generic. 2024-09-19T08:24:27.360 INFO:teuthology.orchestra.run.smithi139.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.122.122 [1,692 B] 2024-09-19T08:24:27.361 INFO:teuthology.orchestra.run.smithi139.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.122.122 [2,436 B] 2024-09-19T08:24:27.361 INFO:teuthology.orchestra.run.smithi139.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122 all 5.15.0-122.132 [12.3 MB] 2024-09-19T08:24:27.775 INFO:teuthology.orchestra.run.smithi098.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-19T08:24:27.779 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../0-linux-modules-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-19T08:24:27.823 INFO:teuthology.orchestra.run.smithi139.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122-generic amd64 5.15.0-122.132 [2,827 kB] 2024-09-19T08:24:27.863 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-19T08:24:27.944 INFO:teuthology.orchestra.run.smithi139.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.122.122 [2,294 B] 2024-09-19T08:24:28.339 INFO:teuthology.orchestra.run.smithi139.stdout:Fetched 113 MB in 5s (23.5 MB/s) 2024-09-19T08:24:28.500 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package linux-modules-5.15.0-122-generic. 2024-09-19T08:24:30.065 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package linux-image-5.15.0-122-generic. 2024-09-19T08:24:30.085 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../1-linux-image-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-19T08:24:30.173 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-19T08:24:30.429 INFO:teuthology.orchestra.run.smithi139.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-19T08:24:30.433 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../0-linux-modules-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-19T08:24:30.547 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-19T08:24:30.634 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-122-generic. 2024-09-19T08:24:30.655 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-19T08:24:30.693 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-19T08:24:32.673 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package linux-image-5.15.0-122-generic. 2024-09-19T08:24:32.695 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../1-linux-image-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-19T08:24:32.783 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-19T08:24:33.293 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-122-generic. 2024-09-19T08:24:33.313 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-19T08:24:33.351 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-19T08:24:37.837 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../3-linux-generic_5.15.0.122.122_amd64.deb ... 2024-09-19T08:24:37.962 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking linux-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-19T08:24:38.262 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.122.122_amd64.deb ... 2024-09-19T08:24:38.391 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking linux-image-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-19T08:24:38.727 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package linux-headers-5.15.0-122. 2024-09-19T08:24:38.753 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../5-linux-headers-5.15.0-122_5.15.0-122.132_all.deb ... 2024-09-19T08:24:38.794 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-19T08:24:40.533 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../3-linux-generic_5.15.0.122.122_amd64.deb ... 2024-09-19T08:24:40.686 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking linux-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-19T08:24:41.028 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.122.122_amd64.deb ... 2024-09-19T08:24:41.148 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking linux-image-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-19T08:24:41.467 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package linux-headers-5.15.0-122. 2024-09-19T08:24:41.490 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../5-linux-headers-5.15.0-122_5.15.0-122.132_all.deb ... 2024-09-19T08:24:41.533 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-19T08:24:44.816 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package linux-headers-5.15.0-122-generic. 2024-09-19T08:24:44.853 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../6-linux-headers-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-19T08:24:44.891 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-19T08:24:47.573 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.122.122_amd64.deb ... 2024-09-19T08:24:47.695 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking linux-headers-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-19T08:24:47.771 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package linux-headers-5.15.0-122-generic. 2024-09-19T08:24:47.808 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../6-linux-headers-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-19T08:24:47.846 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-19T08:24:48.073 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-19T08:24:48.192 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-19T08:24:48.318 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up linux-headers-generic (5.15.0.122.122) ... 2024-09-19T08:24:48.447 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-19T08:24:49.773 INFO:teuthology.orchestra.run.smithi098.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-19T08:24:49.773 INFO:teuthology.orchestra.run.smithi098.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-19T08:24:49.773 INFO:teuthology.orchestra.run.smithi098.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-122-generic 2024-09-19T08:24:49.773 INFO:teuthology.orchestra.run.smithi098.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-122-generic 2024-09-19T08:24:50.351 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-19T08:24:51.606 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.122.122_amd64.deb ... 2024-09-19T08:24:51.747 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking linux-headers-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-19T08:24:52.097 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-19T08:24:52.244 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-19T08:24:52.310 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up linux-image-generic (5.15.0.122.122) ... 2024-09-19T08:24:52.372 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up linux-headers-generic (5.15.0.122.122) ... 2024-09-19T08:24:52.444 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up linux-generic (5.15.0.122.122) ... 2024-09-19T08:24:52.515 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-19T08:24:52.580 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-19T08:24:53.859 INFO:teuthology.orchestra.run.smithi139.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-19T08:24:53.859 INFO:teuthology.orchestra.run.smithi139.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-19T08:24:53.859 INFO:teuthology.orchestra.run.smithi139.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-122-generic 2024-09-19T08:24:53.859 INFO:teuthology.orchestra.run.smithi139.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-122-generic 2024-09-19T08:24:53.939 INFO:teuthology.orchestra.run.smithi098.stdout:Processing triggers for linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-19T08:24:54.006 INFO:teuthology.orchestra.run.smithi098.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-19T08:24:54.006 INFO:teuthology.orchestra.run.smithi098.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-122-generic 2024-09-19T08:24:54.010 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-19T08:24:55.401 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up linux-image-generic (5.15.0.122.122) ... 2024-09-19T08:24:55.535 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up linux-generic (5.15.0.122.122) ... 2024-09-19T08:24:55.661 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-19T08:24:57.221 INFO:teuthology.orchestra.run.smithi139.stdout:Processing triggers for linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-19T08:24:57.272 INFO:teuthology.orchestra.run.smithi139.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-19T08:24:57.272 INFO:teuthology.orchestra.run.smithi139.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-122-generic 2024-09-19T08:25:08.372 INFO:teuthology.orchestra.run.smithi098.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-19T08:25:08.372 INFO:teuthology.orchestra.run.smithi098.stdout:Sourcing file `/etc/default/grub' 2024-09-19T08:25:08.406 INFO:teuthology.orchestra.run.smithi098.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-19T08:25:08.426 INFO:teuthology.orchestra.run.smithi098.stdout:Generating grub configuration file ... 2024-09-19T08:25:08.710 INFO:teuthology.orchestra.run.smithi098.stdout:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-19T08:25:08.730 INFO:teuthology.orchestra.run.smithi098.stdout:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-19T08:25:09.092 INFO:teuthology.orchestra.run.smithi098.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-19T08:25:09.097 INFO:teuthology.orchestra.run.smithi098.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-19T08:25:09.162 INFO:teuthology.orchestra.run.smithi098.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-19T08:25:09.168 INFO:teuthology.orchestra.run.smithi098.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-19T08:25:09.321 INFO:teuthology.orchestra.run.smithi098.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-19T08:25:09.321 INFO:teuthology.orchestra.run.smithi098.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-19T08:25:09.321 INFO:teuthology.orchestra.run.smithi098.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-19T08:25:09.361 INFO:teuthology.orchestra.run.smithi098.stdout:done 2024-09-19T08:25:09.852 DEBUG:teuthology.orchestra.run.smithi098:> dpkg -s linux-image-generic 2024-09-19T08:25:09.879 INFO:teuthology.orchestra.run.smithi098.stdout:Package: linux-image-generic 2024-09-19T08:25:09.879 INFO:teuthology.orchestra.run.smithi098.stdout:Status: install ok installed 2024-09-19T08:25:09.879 INFO:teuthology.orchestra.run.smithi098.stdout:Priority: optional 2024-09-19T08:25:09.879 INFO:teuthology.orchestra.run.smithi098.stdout:Section: kernel 2024-09-19T08:25:09.879 INFO:teuthology.orchestra.run.smithi098.stdout:Installed-Size: 21 2024-09-19T08:25:09.879 INFO:teuthology.orchestra.run.smithi098.stdout:Maintainer: Ubuntu Kernel Team 2024-09-19T08:25:09.879 INFO:teuthology.orchestra.run.smithi098.stdout:Architecture: amd64 2024-09-19T08:25:09.879 INFO:teuthology.orchestra.run.smithi098.stdout:Source: linux-meta 2024-09-19T08:25:09.879 INFO:teuthology.orchestra.run.smithi098.stdout:Version: 5.15.0.122.122 2024-09-19T08:25:09.880 INFO:teuthology.orchestra.run.smithi098.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-19T08:25:09.880 INFO:teuthology.orchestra.run.smithi098.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-19T08:25:09.880 INFO:teuthology.orchestra.run.smithi098.stdout:Recommends: thermald 2024-09-19T08:25:09.880 INFO:teuthology.orchestra.run.smithi098.stdout:Description: Generic Linux kernel image 2024-09-19T08:25:09.880 INFO:teuthology.orchestra.run.smithi098.stdout: This package will always depend on the latest generic kernel image 2024-09-19T08:25:09.880 INFO:teuthology.orchestra.run.smithi098.stdout: available. 2024-09-19T08:25:09.880 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-122-generic 2024-09-19T08:25:09.880 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-19T08:25:09.880 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-19T08:25:09.881 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-19T08:25:09.881 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi098.front.sepia.ceph.com, path=None, version=distro) 2024-09-19T08:25:09.881 DEBUG:teuthology.orchestra.run.smithi098:> sudo apt-get clean 2024-09-19T08:25:10.043 DEBUG:teuthology.orchestra.run.smithi098:> sudo apt-get update 2024-09-19T08:25:10.174 INFO:teuthology.orchestra.run.smithi098.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-19T08:25:10.277 INFO:teuthology.orchestra.run.smithi098.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-19T08:25:10.359 INFO:teuthology.orchestra.run.smithi098.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-19T08:25:10.439 INFO:teuthology.orchestra.run.smithi098.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-19T08:25:11.200 INFO:teuthology.orchestra.run.smithi139.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-19T08:25:11.201 INFO:teuthology.orchestra.run.smithi139.stdout:Sourcing file `/etc/default/grub' 2024-09-19T08:25:11.225 INFO:teuthology.orchestra.run.smithi139.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-19T08:25:11.244 INFO:teuthology.orchestra.run.smithi139.stdout:Generating grub configuration file ... 2024-09-19T08:25:11.544 INFO:teuthology.orchestra.run.smithi139.stdout:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-19T08:25:11.560 INFO:teuthology.orchestra.run.smithi139.stdout:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-19T08:25:11.597 INFO:teuthology.orchestra.run.smithi098.stdout:Reading package lists... 2024-09-19T08:25:11.615 DEBUG:teuthology.orchestra.run.smithi098:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-19T08:25:11.676 INFO:teuthology.orchestra.run.smithi098.stdout:Reading package lists... 2024-09-19T08:25:11.862 INFO:teuthology.orchestra.run.smithi139.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-19T08:25:11.867 INFO:teuthology.orchestra.run.smithi139.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-19T08:25:11.882 INFO:teuthology.orchestra.run.smithi098.stdout:Building dependency tree... 2024-09-19T08:25:11.882 INFO:teuthology.orchestra.run.smithi098.stdout:Reading state information... 2024-09-19T08:25:11.931 INFO:teuthology.orchestra.run.smithi139.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-19T08:25:11.936 INFO:teuthology.orchestra.run.smithi139.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-19T08:25:12.062 INFO:teuthology.orchestra.run.smithi098.stdout:linux-image-generic is already the newest version (5.15.0.122.122). 2024-09-19T08:25:12.062 INFO:teuthology.orchestra.run.smithi098.stdout:The following packages were automatically installed and are no longer required: 2024-09-19T08:25:12.063 INFO:teuthology.orchestra.run.smithi098.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-19T08:25:12.063 INFO:teuthology.orchestra.run.smithi098.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-19T08:25:12.063 INFO:teuthology.orchestra.run.smithi098.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-19T08:25:12.063 INFO:teuthology.orchestra.run.smithi098.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-19T08:25:12.063 INFO:teuthology.orchestra.run.smithi098.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-19T08:25:12.064 INFO:teuthology.orchestra.run.smithi098.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-19T08:25:12.093 INFO:teuthology.orchestra.run.smithi139.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-19T08:25:12.093 INFO:teuthology.orchestra.run.smithi139.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-19T08:25:12.093 INFO:teuthology.orchestra.run.smithi139.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-19T08:25:12.103 INFO:teuthology.orchestra.run.smithi098.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-19T08:25:12.105 DEBUG:teuthology.orchestra.run.smithi098:> dpkg -s linux-image-generic 2024-09-19T08:25:12.117 INFO:teuthology.orchestra.run.smithi098.stdout:Package: linux-image-generic 2024-09-19T08:25:12.117 INFO:teuthology.orchestra.run.smithi098.stdout:Status: install ok installed 2024-09-19T08:25:12.117 INFO:teuthology.orchestra.run.smithi098.stdout:Priority: optional 2024-09-19T08:25:12.117 INFO:teuthology.orchestra.run.smithi098.stdout:Section: kernel 2024-09-19T08:25:12.117 INFO:teuthology.orchestra.run.smithi098.stdout:Installed-Size: 21 2024-09-19T08:25:12.117 INFO:teuthology.orchestra.run.smithi098.stdout:Maintainer: Ubuntu Kernel Team 2024-09-19T08:25:12.117 INFO:teuthology.orchestra.run.smithi098.stdout:Architecture: amd64 2024-09-19T08:25:12.117 INFO:teuthology.orchestra.run.smithi098.stdout:Source: linux-meta 2024-09-19T08:25:12.117 INFO:teuthology.orchestra.run.smithi098.stdout:Version: 5.15.0.122.122 2024-09-19T08:25:12.118 INFO:teuthology.orchestra.run.smithi098.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-19T08:25:12.118 INFO:teuthology.orchestra.run.smithi098.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-19T08:25:12.118 INFO:teuthology.orchestra.run.smithi098.stdout:Recommends: thermald 2024-09-19T08:25:12.118 INFO:teuthology.orchestra.run.smithi098.stdout:Description: Generic Linux kernel image 2024-09-19T08:25:12.118 INFO:teuthology.orchestra.run.smithi098.stdout: This package will always depend on the latest generic kernel image 2024-09-19T08:25:12.118 INFO:teuthology.orchestra.run.smithi098.stdout: available. 2024-09-19T08:25:12.119 DEBUG:teuthology.orchestra.run.smithi098:> mktemp 2024-09-19T08:25:12.144 INFO:teuthology.orchestra.run.smithi139.stdout:done 2024-09-19T08:25:12.169 INFO:teuthology.orchestra.run.smithi098.stdout:/tmp/tmp.cEmQZ79aWD 2024-09-19T08:25:12.169 DEBUG:teuthology.orchestra.run.smithi098:> sudo cp /boot/grub/grub.cfg /tmp/tmp.cEmQZ79aWD 2024-09-19T08:25:12.225 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 0666 /tmp/tmp.cEmQZ79aWD 2024-09-19T08:25:12.337 DEBUG:teuthology.orchestra.remote:smithi098:/tmp/tmp.cEmQZ79aWD is 10KB 2024-09-19T08:25:12.390 DEBUG:teuthology.orchestra.run.smithi098:> rm -fr /tmp/tmp.cEmQZ79aWD 2024-09-19T08:25:12.395 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-19T08:25:12.448 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:25:12.448 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-19T08:25:12.448 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-19T08:25:12.517 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-122-generic 2024-09-19T08:25:12.517 DEBUG:teuthology.orchestra.run.smithi098:> sudo update-grub 2024-09-19T08:25:12.635 DEBUG:teuthology.orchestra.run.smithi139:> dpkg -s linux-image-generic 2024-09-19T08:25:12.663 INFO:teuthology.orchestra.run.smithi139.stdout:Package: linux-image-generic 2024-09-19T08:25:12.663 INFO:teuthology.orchestra.run.smithi139.stdout:Status: install ok installed 2024-09-19T08:25:12.663 INFO:teuthology.orchestra.run.smithi139.stdout:Priority: optional 2024-09-19T08:25:12.663 INFO:teuthology.orchestra.run.smithi139.stdout:Section: kernel 2024-09-19T08:25:12.664 INFO:teuthology.orchestra.run.smithi139.stdout:Installed-Size: 21 2024-09-19T08:25:12.664 INFO:teuthology.orchestra.run.smithi139.stdout:Maintainer: Ubuntu Kernel Team 2024-09-19T08:25:12.664 INFO:teuthology.orchestra.run.smithi139.stdout:Architecture: amd64 2024-09-19T08:25:12.664 INFO:teuthology.orchestra.run.smithi139.stdout:Source: linux-meta 2024-09-19T08:25:12.664 INFO:teuthology.orchestra.run.smithi139.stdout:Version: 5.15.0.122.122 2024-09-19T08:25:12.664 INFO:teuthology.orchestra.run.smithi139.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-19T08:25:12.664 INFO:teuthology.orchestra.run.smithi139.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-19T08:25:12.664 INFO:teuthology.orchestra.run.smithi139.stdout:Recommends: thermald 2024-09-19T08:25:12.664 INFO:teuthology.orchestra.run.smithi139.stdout:Description: Generic Linux kernel image 2024-09-19T08:25:12.664 INFO:teuthology.orchestra.run.smithi139.stdout: This package will always depend on the latest generic kernel image 2024-09-19T08:25:12.664 INFO:teuthology.orchestra.run.smithi139.stdout: available. 2024-09-19T08:25:12.668 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-122-generic 2024-09-19T08:25:12.668 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-19T08:25:12.668 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-19T08:25:12.668 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-19T08:25:12.669 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi139.front.sepia.ceph.com, path=None, version=distro) 2024-09-19T08:25:12.669 DEBUG:teuthology.orchestra.run.smithi139:> sudo apt-get clean 2024-09-19T08:25:12.830 DEBUG:teuthology.orchestra.run.smithi139:> sudo apt-get update 2024-09-19T08:25:12.940 INFO:teuthology.orchestra.run.smithi139.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-19T08:25:12.940 INFO:teuthology.orchestra.run.smithi139.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-19T08:25:12.950 INFO:teuthology.orchestra.run.smithi139.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-19T08:25:12.967 INFO:teuthology.orchestra.run.smithi139.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-19T08:25:13.804 INFO:teuthology.orchestra.run.smithi098.stderr:Sourcing file `/etc/default/grub' 2024-09-19T08:25:13.805 INFO:teuthology.orchestra.run.smithi098.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-19T08:25:13.807 INFO:teuthology.orchestra.run.smithi098.stderr:Generating grub configuration file ... 2024-09-19T08:25:13.996 INFO:teuthology.orchestra.run.smithi098.stderr:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-19T08:25:14.003 INFO:teuthology.orchestra.run.smithi098.stderr:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-19T08:25:14.290 INFO:teuthology.orchestra.run.smithi098.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-19T08:25:14.295 INFO:teuthology.orchestra.run.smithi098.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-19T08:25:14.338 INFO:teuthology.orchestra.run.smithi098.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-19T08:25:14.343 INFO:teuthology.orchestra.run.smithi098.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-19T08:25:14.430 INFO:teuthology.orchestra.run.smithi139.stdout:Reading package lists... 2024-09-19T08:25:14.449 DEBUG:teuthology.orchestra.run.smithi139:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-19T08:25:14.462 INFO:teuthology.orchestra.run.smithi098.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-19T08:25:14.462 INFO:teuthology.orchestra.run.smithi098.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-19T08:25:14.462 INFO:teuthology.orchestra.run.smithi098.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-19T08:25:14.478 INFO:teuthology.orchestra.run.smithi098.stderr:done 2024-09-19T08:25:14.480 DEBUG:teuthology.orchestra.run.smithi098:> sudo shutdown -r now 2024-09-19T08:25:14.514 INFO:teuthology.orchestra.run.smithi139.stdout:Reading package lists... 2024-09-19T08:25:14.728 INFO:teuthology.orchestra.run.smithi139.stdout:Building dependency tree... 2024-09-19T08:25:14.729 INFO:teuthology.orchestra.run.smithi139.stdout:Reading state information... 2024-09-19T08:25:14.916 INFO:teuthology.orchestra.run.smithi139.stdout:linux-image-generic is already the newest version (5.15.0.122.122). 2024-09-19T08:25:14.916 INFO:teuthology.orchestra.run.smithi139.stdout:The following packages were automatically installed and are no longer required: 2024-09-19T08:25:14.916 INFO:teuthology.orchestra.run.smithi139.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-19T08:25:14.916 INFO:teuthology.orchestra.run.smithi139.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-19T08:25:14.916 INFO:teuthology.orchestra.run.smithi139.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-19T08:25:14.916 INFO:teuthology.orchestra.run.smithi139.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-19T08:25:14.917 INFO:teuthology.orchestra.run.smithi139.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-19T08:25:14.917 INFO:teuthology.orchestra.run.smithi139.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-19T08:25:14.960 INFO:teuthology.orchestra.run.smithi139.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-19T08:25:14.961 DEBUG:teuthology.orchestra.run.smithi139:> dpkg -s linux-image-generic 2024-09-19T08:25:14.977 INFO:teuthology.orchestra.run.smithi139.stdout:Package: linux-image-generic 2024-09-19T08:25:14.977 INFO:teuthology.orchestra.run.smithi139.stdout:Status: install ok installed 2024-09-19T08:25:14.977 INFO:teuthology.orchestra.run.smithi139.stdout:Priority: optional 2024-09-19T08:25:14.977 INFO:teuthology.orchestra.run.smithi139.stdout:Section: kernel 2024-09-19T08:25:14.977 INFO:teuthology.orchestra.run.smithi139.stdout:Installed-Size: 21 2024-09-19T08:25:14.977 INFO:teuthology.orchestra.run.smithi139.stdout:Maintainer: Ubuntu Kernel Team 2024-09-19T08:25:14.978 INFO:teuthology.orchestra.run.smithi139.stdout:Architecture: amd64 2024-09-19T08:25:14.978 INFO:teuthology.orchestra.run.smithi139.stdout:Source: linux-meta 2024-09-19T08:25:14.978 INFO:teuthology.orchestra.run.smithi139.stdout:Version: 5.15.0.122.122 2024-09-19T08:25:14.978 INFO:teuthology.orchestra.run.smithi139.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-19T08:25:14.978 INFO:teuthology.orchestra.run.smithi139.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-19T08:25:14.978 INFO:teuthology.orchestra.run.smithi139.stdout:Recommends: thermald 2024-09-19T08:25:14.978 INFO:teuthology.orchestra.run.smithi139.stdout:Description: Generic Linux kernel image 2024-09-19T08:25:14.978 INFO:teuthology.orchestra.run.smithi139.stdout: This package will always depend on the latest generic kernel image 2024-09-19T08:25:14.978 INFO:teuthology.orchestra.run.smithi139.stdout: available. 2024-09-19T08:25:14.978 DEBUG:teuthology.orchestra.run.smithi139:> mktemp 2024-09-19T08:25:15.025 INFO:teuthology.orchestra.run.smithi139.stdout:/tmp/tmp.ip2WQFt8mU 2024-09-19T08:25:15.025 DEBUG:teuthology.orchestra.run.smithi139:> sudo cp /boot/grub/grub.cfg /tmp/tmp.ip2WQFt8mU 2024-09-19T08:25:15.085 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 0666 /tmp/tmp.ip2WQFt8mU 2024-09-19T08:25:15.201 DEBUG:teuthology.orchestra.remote:smithi139:/tmp/tmp.ip2WQFt8mU is 10KB 2024-09-19T08:25:15.213 DEBUG:teuthology.orchestra.run.smithi139:> rm -fr /tmp/tmp.ip2WQFt8mU 2024-09-19T08:25:15.218 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-19T08:25:15.272 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:25:15.272 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-19T08:25:15.273 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-19T08:25:15.363 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-122-generic 2024-09-19T08:25:15.363 DEBUG:teuthology.orchestra.run.smithi139:> sudo update-grub 2024-09-19T08:25:16.627 INFO:teuthology.orchestra.run.smithi139.stderr:Sourcing file `/etc/default/grub' 2024-09-19T08:25:16.628 INFO:teuthology.orchestra.run.smithi139.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-19T08:25:16.631 INFO:teuthology.orchestra.run.smithi139.stderr:Generating grub configuration file ... 2024-09-19T08:25:16.849 INFO:teuthology.orchestra.run.smithi139.stderr:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-19T08:25:16.855 INFO:teuthology.orchestra.run.smithi139.stderr:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-19T08:25:17.091 INFO:teuthology.orchestra.run.smithi139.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-19T08:25:17.097 INFO:teuthology.orchestra.run.smithi139.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-19T08:25:17.139 INFO:teuthology.orchestra.run.smithi139.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-19T08:25:17.144 INFO:teuthology.orchestra.run.smithi139.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-19T08:25:17.284 INFO:teuthology.orchestra.run.smithi139.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-19T08:25:17.284 INFO:teuthology.orchestra.run.smithi139.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-19T08:25:17.284 INFO:teuthology.orchestra.run.smithi139.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-19T08:25:17.302 INFO:teuthology.orchestra.run.smithi139.stderr:done 2024-09-19T08:25:17.309 DEBUG:teuthology.orchestra.run.smithi139:> sudo shutdown -r now 2024-09-19T08:25:44.509 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-19T08:25:44.510 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-09-19T08:25:44.512 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T08:25:47.313 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-19T08:25:47.313 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-19T08:25:47.314 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T08:26:19.070 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.98 2024-09-19T08:26:21.886 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.139 2024-09-19T08:26:28.073 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-09-19T08:26:28.074 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T08:26:30.889 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-19T08:26:30.889 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T08:26:46.461 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.98 2024-09-19T08:26:49.277 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.139 2024-09-19T08:26:58.470 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-09-19T08:26:58.471 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T08:26:58.758 DEBUG:teuthology.orchestra.run.smithi098:> true 2024-09-19T08:26:59.528 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-09-19T08:26:59.528 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-122-generic"... 2024-09-19T08:26:59.529 DEBUG:teuthology.orchestra.run.smithi098:> uname -r 2024-09-19T08:26:59.577 INFO:teuthology.orchestra.run.smithi098.stdout:5.15.0-122-generic 2024-09-19T08:26:59.577 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-122-generic vs 5.15.0-122-generic 2024-09-19T08:26:59.577 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T08:26:59.577 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-19T08:27:00.578 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-19T08:27:00.578 DEBUG:teuthology.orchestra.run.smithi098:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-19T08:27:00.667 INFO:teuthology.orchestra.run.smithi098.stdout:ttyS1 2024-09-19T08:27:00.699 DEBUG:teuthology.parallel:result is None 2024-09-19T08:27:01.278 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-19T08:27:01.279 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T08:27:01.750 DEBUG:teuthology.orchestra.run.smithi139:> true 2024-09-19T08:27:02.648 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-19T08:27:02.648 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-122-generic"... 2024-09-19T08:27:02.649 DEBUG:teuthology.orchestra.run.smithi139:> uname -r 2024-09-19T08:27:02.696 INFO:teuthology.orchestra.run.smithi139.stdout:5.15.0-122-generic 2024-09-19T08:27:02.697 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-122-generic vs 5.15.0-122-generic 2024-09-19T08:27:02.697 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T08:27:02.697 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-19T08:27:03.697 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-19T08:27:03.698 DEBUG:teuthology.orchestra.run.smithi139:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-19T08:27:04.026 INFO:teuthology.orchestra.run.smithi139.stdout:ttyS1 2024-09-19T08:27:04.045 DEBUG:teuthology.parallel:result is None 2024-09-19T08:27:04.045 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-19T08:27:04.055 INFO:teuthology.task.internal:Creating test directory... 2024-09-19T08:27:04.056 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T08:27:04.058 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T08:27:04.064 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-19T08:27:04.070 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-19T08:27:04.080 INFO:teuthology.task.internal:Creating archive directory... 2024-09-19T08:27:04.080 DEBUG:teuthology.orchestra.run.smithi098:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T08:27:04.106 DEBUG:teuthology.orchestra.run.smithi139:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T08:27:04.196 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-19T08:27:04.204 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-19T08:27:04.204 DEBUG:teuthology.orchestra.run.smithi098:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T08:27:04.207 DEBUG:teuthology.orchestra.run.smithi139:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T08:27:04.227 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T08:27:04.236 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T08:27:04.241 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T08:27:04.249 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T08:27:04.251 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-19T08:27:04.258 INFO:teuthology.task.internal:Configuring sudo... 2024-09-19T08:27:04.258 DEBUG:teuthology.orchestra.run.smithi098:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T08:27:04.287 DEBUG:teuthology.orchestra.run.smithi139:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T08:27:04.307 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-19T08:27:04.316 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-19T08:27:04.316 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T08:27:04.350 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T08:27:04.355 DEBUG:teuthology.orchestra.run.smithi098:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T08:27:04.401 DEBUG:teuthology.orchestra.run.smithi098:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T08:27:04.449 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:27:04.450 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T08:27:04.512 DEBUG:teuthology.orchestra.run.smithi139:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T08:27:04.518 DEBUG:teuthology.orchestra.run.smithi139:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T08:27:04.572 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:27:04.572 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T08:27:04.635 DEBUG:teuthology.orchestra.run.smithi098:> sudo service rsyslog restart 2024-09-19T08:27:04.637 DEBUG:teuthology.orchestra.run.smithi139:> sudo service rsyslog restart 2024-09-19T08:27:04.769 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-19T08:27:04.777 INFO:teuthology.task.internal:Starting timer... 2024-09-19T08:27:04.777 INFO:teuthology.run_tasks:Running task pcp... 2024-09-19T08:27:04.787 INFO:teuthology.run_tasks:Running task selinux... 2024-09-19T08:27:04.797 DEBUG:teuthology.task.selinux:Excluding smithi098: OS 'ubuntu' does not support SELinux 2024-09-19T08:27:04.797 DEBUG:teuthology.task.selinux:Excluding smithi139: OS 'ubuntu' does not support SELinux 2024-09-19T08:27:04.797 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-19T08:27:04.798 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-19T08:27:04.798 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-19T08:27:04.798 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-19T08:27:04.927 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-19T08:27:04.938 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-19T08:27:04.940 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi098.front.sepia.ceph.com,smithi139.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-19T08:34:34.538 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi098.front.sepia.ceph.com'), Remote(name='ubuntu@smithi139.front.sepia.ceph.com')] 2024-09-19T08:34:34.540 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-09-19T08:34:34.541 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi098.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T08:34:34.622 DEBUG:teuthology.orchestra.run.smithi098:> true 2024-09-19T08:34:34.697 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi098.front.sepia.ceph.com' 2024-09-19T08:34:34.697 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-19T08:34:34.698 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T08:34:34.775 DEBUG:teuthology.orchestra.run.smithi139:> true 2024-09-19T08:34:34.849 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-09-19T08:34:34.849 INFO:teuthology.run_tasks:Running task clock... 2024-09-19T08:34:34.859 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-19T08:34:34.859 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T08:34:34.860 DEBUG:teuthology.orchestra.run.smithi098:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T08:34:34.862 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T08:34:34.862 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T08:34:34.888 INFO:teuthology.orchestra.run.smithi098.stdout:19 Sep 08:34:34 ntpd[18489]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-19T08:34:34.888 INFO:teuthology.orchestra.run.smithi098.stdout:19 Sep 08:34:34 ntpd[18489]: Command line: ntpd -gq 2024-09-19T08:34:34.889 INFO:teuthology.orchestra.run.smithi098.stdout:19 Sep 08:34:34 ntpd[18489]: ---------------------------------------------------- 2024-09-19T08:34:34.889 INFO:teuthology.orchestra.run.smithi098.stdout:19 Sep 08:34:34 ntpd[18489]: ntp-4 is maintained by Network Time Foundation, 2024-09-19T08:34:34.889 INFO:teuthology.orchestra.run.smithi098.stdout:19 Sep 08:34:34 ntpd[18489]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-19T08:34:34.889 INFO:teuthology.orchestra.run.smithi098.stdout:19 Sep 08:34:34 ntpd[18489]: corporation. Support and training for ntp-4 are 2024-09-19T08:34:34.889 INFO:teuthology.orchestra.run.smithi098.stdout:19 Sep 08:34:34 ntpd[18489]: available at https://www.nwtime.org/support 2024-09-19T08:34:34.889 INFO:teuthology.orchestra.run.smithi098.stdout:19 Sep 08:34:34 ntpd[18489]: ---------------------------------------------------- 2024-09-19T08:34:34.890 INFO:teuthology.orchestra.run.smithi098.stdout:19 Sep 08:34:34 ntpd[18489]: proto: precision = 0.060 usec (-24) 2024-09-19T08:34:34.890 INFO:teuthology.orchestra.run.smithi098.stdout:19 Sep 08:34:34 ntpd[18489]: basedate set to 2022-02-04 2024-09-19T08:34:34.890 INFO:teuthology.orchestra.run.smithi098.stdout:19 Sep 08:34:34 ntpd[18489]: gps base set to 2022-02-06 (week 2196) 2024-09-19T08:34:34.890 INFO:teuthology.orchestra.run.smithi098.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-19T08:34:34.890 INFO:teuthology.orchestra.run.smithi098.stdout:19 Sep 08:34:34 ntpd[18489]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-19T08:34:34.890 INFO:teuthology.orchestra.run.smithi098.stdout:19 Sep 08:34:34 ntpd[18489]: restrict ::: KOD does nothing without LIMITED. 2024-09-19T08:34:34.891 INFO:teuthology.orchestra.run.smithi098.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-19T08:34:34.892 INFO:teuthology.orchestra.run.smithi098.stdout:19 Sep 08:34:34 ntpd[18489]: Listen and drop on 0 v6wildcard [::]:123 2024-09-19T08:34:34.892 INFO:teuthology.orchestra.run.smithi098.stdout:19 Sep 08:34:34 ntpd[18489]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-19T08:34:34.892 INFO:teuthology.orchestra.run.smithi098.stdout:19 Sep 08:34:34 ntpd[18489]: Listen normally on 2 lo 127.0.0.1:123 2024-09-19T08:34:34.892 INFO:teuthology.orchestra.run.smithi098.stdout:19 Sep 08:34:34 ntpd[18489]: Listen normally on 3 enp3s0f1 172.21.15.98:123 2024-09-19T08:34:34.892 INFO:teuthology.orchestra.run.smithi098.stdout:19 Sep 08:34:34 ntpd[18489]: Listen normally on 4 lo [::1]:123 2024-09-19T08:34:34.892 INFO:teuthology.orchestra.run.smithi098.stdout:19 Sep 08:34:34 ntpd[18489]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d26b%5]:123 2024-09-19T08:34:34.892 INFO:teuthology.orchestra.run.smithi098.stdout:19 Sep 08:34:34 ntpd[18489]: Listening on routing socket on fd #22 for interface updates 2024-09-19T08:34:34.922 INFO:teuthology.orchestra.run.smithi139.stdout:19 Sep 08:34:34 ntpd[18492]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-19T08:34:34.922 INFO:teuthology.orchestra.run.smithi139.stdout:19 Sep 08:34:34 ntpd[18492]: Command line: ntpd -gq 2024-09-19T08:34:34.922 INFO:teuthology.orchestra.run.smithi139.stdout:19 Sep 08:34:34 ntpd[18492]: ---------------------------------------------------- 2024-09-19T08:34:34.922 INFO:teuthology.orchestra.run.smithi139.stdout:19 Sep 08:34:34 ntpd[18492]: ntp-4 is maintained by Network Time Foundation, 2024-09-19T08:34:34.922 INFO:teuthology.orchestra.run.smithi139.stdout:19 Sep 08:34:34 ntpd[18492]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-19T08:34:34.922 INFO:teuthology.orchestra.run.smithi139.stdout:19 Sep 08:34:34 ntpd[18492]: corporation. Support and training for ntp-4 are 2024-09-19T08:34:34.922 INFO:teuthology.orchestra.run.smithi139.stdout:19 Sep 08:34:34 ntpd[18492]: available at https://www.nwtime.org/support 2024-09-19T08:34:34.923 INFO:teuthology.orchestra.run.smithi139.stdout:19 Sep 08:34:34 ntpd[18492]: ---------------------------------------------------- 2024-09-19T08:34:34.923 INFO:teuthology.orchestra.run.smithi139.stdout:19 Sep 08:34:34 ntpd[18492]: proto: precision = 0.044 usec (-24) 2024-09-19T08:34:34.923 INFO:teuthology.orchestra.run.smithi139.stdout:19 Sep 08:34:34 ntpd[18492]: basedate set to 2022-02-04 2024-09-19T08:34:34.923 INFO:teuthology.orchestra.run.smithi139.stdout:19 Sep 08:34:34 ntpd[18492]: gps base set to 2022-02-06 (week 2196) 2024-09-19T08:34:34.924 INFO:teuthology.orchestra.run.smithi139.stdout:19 Sep 08:34:34 ntpd[18492]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-19T08:34:34.924 INFO:teuthology.orchestra.run.smithi139.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-19T08:34:34.924 INFO:teuthology.orchestra.run.smithi139.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-19T08:34:34.925 INFO:teuthology.orchestra.run.smithi139.stdout:19 Sep 08:34:34 ntpd[18492]: restrict ::: KOD does nothing without LIMITED. 2024-09-19T08:34:34.925 INFO:teuthology.orchestra.run.smithi139.stdout:19 Sep 08:34:34 ntpd[18492]: Listen and drop on 0 v6wildcard [::]:123 2024-09-19T08:34:34.925 INFO:teuthology.orchestra.run.smithi139.stdout:19 Sep 08:34:34 ntpd[18492]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-19T08:34:34.925 INFO:teuthology.orchestra.run.smithi139.stdout:19 Sep 08:34:34 ntpd[18492]: Listen normally on 2 lo 127.0.0.1:123 2024-09-19T08:34:34.925 INFO:teuthology.orchestra.run.smithi139.stdout:19 Sep 08:34:34 ntpd[18492]: Listen normally on 3 enp3s0f1 172.21.15.139:123 2024-09-19T08:34:34.925 INFO:teuthology.orchestra.run.smithi139.stdout:19 Sep 08:34:34 ntpd[18492]: Listen normally on 4 lo [::1]:123 2024-09-19T08:34:34.925 INFO:teuthology.orchestra.run.smithi139.stdout:19 Sep 08:34:34 ntpd[18492]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7f59%5]:123 2024-09-19T08:34:34.926 INFO:teuthology.orchestra.run.smithi139.stdout:19 Sep 08:34:34 ntpd[18492]: Listening on routing socket on fd #22 for interface updates 2024-09-19T08:34:35.889 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:35 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:35.889 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:35 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:35.889 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:35 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:35.889 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:35 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:35.889 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:35 ntpd[18489]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T08:34:35.890 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:35 ntpd[18489]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T08:34:35.923 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:35 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:35.923 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:35 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:37.890 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:37 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:37.891 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:37 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:37.891 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:37 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:37.891 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:37 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:37.891 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:37 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:37.891 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:37 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:37.891 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:37 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:37.891 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:37 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:37.891 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:37 ntpd[18489]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T08:34:37.891 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:37 ntpd[18489]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T08:34:37.891 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:37 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:37.891 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:37 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:37.891 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:37 ntpd[18489]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T08:34:37.891 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:37 ntpd[18489]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T08:34:37.891 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:37 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:37.892 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:37 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:37.892 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:37 ntpd[18489]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T08:34:37.892 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:37 ntpd[18489]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T08:34:37.923 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:37 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:37.923 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:37 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:37.923 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:37 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:37.923 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:37 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:37.923 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:37 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:37.923 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:37 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:37.924 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:37 ntpd[18492]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T08:34:37.924 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:37 ntpd[18492]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T08:34:37.924 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:37 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:37.924 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:37 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:37.924 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:37 ntpd[18492]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T08:34:37.924 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:37 ntpd[18492]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T08:34:39.890 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:39 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:39.891 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:39 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:39.891 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:39 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:39.891 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:39 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:39.891 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:39 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:39.891 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:39 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:39.891 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:39 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:39.891 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:39 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:39.891 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:39 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:39.891 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:39 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:39.891 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:39 ntpd[18489]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T08:34:39.891 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:39 ntpd[18489]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T08:34:39.891 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:39 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:39.891 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:39 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:39.891 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:39 ntpd[18489]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T08:34:39.892 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:39 ntpd[18489]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T08:34:39.892 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:39 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:39.892 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:39 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:39.892 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:39 ntpd[18489]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T08:34:39.892 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:39 ntpd[18489]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T08:34:39.924 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:39 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:39.924 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:39 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:39.924 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:39 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:39.924 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:39 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:39.924 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:39 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:39.924 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:39 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:39.925 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:39 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:39.925 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:39 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:39.925 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:39 ntpd[18492]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T08:34:39.925 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:39 ntpd[18492]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T08:34:39.925 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:39 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:39.925 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:39 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:39.925 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:39 ntpd[18492]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T08:34:39.925 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:39 ntpd[18492]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T08:34:39.925 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:39 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:39.925 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:39 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:39.925 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:39 ntpd[18492]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T08:34:39.925 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:39 ntpd[18492]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T08:34:41.890 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:41 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:41.890 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:41 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:41.890 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:41 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:41.890 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:41 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:41.890 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:41 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:41.890 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:41 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:41.890 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:41 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:41.890 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:41 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:41.890 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:41 ntpd[18489]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T08:34:41.890 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:41 ntpd[18489]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T08:34:41.891 INFO:teuthology.orchestra.run.smithi098.stdout:19 Sep 08:34:41 ntpd[18489]: ntpd: time slew -0.000979 s 2024-09-19T08:34:41.891 INFO:teuthology.orchestra.run.smithi098.stdout:ntpd: time slew -0.000979s 2024-09-19T08:34:41.891 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:41 ntpd[18489]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-19T08:34:41.891 INFO:teuthology.orchestra.run.smithi098.stderr:19 Sep 08:34:41 ntpd[18489]: can't open /var/log/ntpstats/loopstats.20240919: Permission denied 2024-09-19T08:34:41.924 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:41 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:41.924 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:41 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:41.924 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:41 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:41.925 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:41 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:41.925 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:41 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:41.925 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:41 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:41.925 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:41 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:41.925 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:41 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:41.925 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:41 ntpd[18492]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T08:34:41.925 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:41 ntpd[18492]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T08:34:41.925 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:41 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:41.925 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:41 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:41.925 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:41 ntpd[18492]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T08:34:41.925 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:41 ntpd[18492]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T08:34:41.925 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:41 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:41.925 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:41 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:41.925 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:41 ntpd[18492]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T08:34:41.926 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:41 ntpd[18492]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T08:34:42.036 INFO:teuthology.orchestra.run.smithi098.stdout: remote refid st t when poll reach delay offset jitter 2024-09-19T08:34:42.036 INFO:teuthology.orchestra.run.smithi098.stdout:============================================================================== 2024-09-19T08:34:42.036 INFO:teuthology.orchestra.run.smithi098.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-19T08:34:42.036 INFO:teuthology.orchestra.run.smithi098.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-19T08:34:42.036 INFO:teuthology.orchestra.run.smithi098.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-19T08:34:42.036 INFO:teuthology.orchestra.run.smithi098.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-19T08:34:43.923 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:43 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:43.923 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:43 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:43.924 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:43 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:43.924 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:43 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:43.924 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:43 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:43.924 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:43 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:43.924 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:43 ntpd[18492]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T08:34:43.924 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:43 ntpd[18492]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T08:34:43.924 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:43 ntpd[18492]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T08:34:43.924 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:43 ntpd[18492]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T08:34:43.925 INFO:teuthology.orchestra.run.smithi139.stdout:19 Sep 08:34:43 ntpd[18492]: ntpd: time slew -0.001545 s 2024-09-19T08:34:43.925 INFO:teuthology.orchestra.run.smithi139.stdout:ntpd: time slew -0.001545s 2024-09-19T08:34:43.925 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:43 ntpd[18492]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-19T08:34:43.925 INFO:teuthology.orchestra.run.smithi139.stderr:19 Sep 08:34:43 ntpd[18492]: can't open /var/log/ntpstats/loopstats.20240919: Permission denied 2024-09-19T08:34:43.975 INFO:teuthology.orchestra.run.smithi139.stdout: remote refid st t when poll reach delay offset jitter 2024-09-19T08:34:43.975 INFO:teuthology.orchestra.run.smithi139.stdout:============================================================================== 2024-09-19T08:34:43.975 INFO:teuthology.orchestra.run.smithi139.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-19T08:34:43.976 INFO:teuthology.orchestra.run.smithi139.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-19T08:34:43.976 INFO:teuthology.orchestra.run.smithi139.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-19T08:34:43.976 INFO:teuthology.orchestra.run.smithi139.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-19T08:34:43.977 INFO:teuthology.run_tasks:Running task install... 2024-09-19T08:34:43.985 DEBUG:teuthology.task.install:project ceph 2024-09-19T08:34:43.986 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'}} 2024-09-19T08:34:43.986 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'} 2024-09-19T08:34:43.986 INFO:teuthology.task.install:Using flavor: default 2024-09-19T08:34:43.992 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-19T08:34:43.992 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-19T08:34:43.992 DEBUG:teuthology.orchestra.run.smithi098:> sudo apt-key list | grep Ceph 2024-09-19T08:34:43.993 DEBUG:teuthology.orchestra.run.smithi139:> sudo apt-key list | grep Ceph 2024-09-19T08:34:44.057 INFO:teuthology.orchestra.run.smithi098.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-19T08:34:44.077 INFO:teuthology.orchestra.run.smithi139.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-19T08:34:44.084 INFO:teuthology.orchestra.run.smithi098.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-19T08:34:44.085 INFO:teuthology.orchestra.run.smithi098.stdout:uid [ unknown] Ceph.com (release key) 2024-09-19T08:34:44.086 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-19T08:34:44.086 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-19T08:34:44.086 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:34:44.106 INFO:teuthology.orchestra.run.smithi139.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-19T08:34:44.106 INFO:teuthology.orchestra.run.smithi139.stdout:uid [ unknown] Ceph.com (release key) 2024-09-19T08:34:44.106 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-19T08:34:44.106 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-19T08:34:44.106 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:34:44.245 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default/ 2024-09-19T08:34:44.245 INFO:teuthology.task.install.deb:Package version is 19.1.1-361-g2f599a46-1jammy 2024-09-19T08:34:44.265 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default/ 2024-09-19T08:34:44.265 INFO:teuthology.task.install.deb:Package version is 19.1.1-361-g2f599a46-1jammy 2024-09-19T08:34:44.366 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:34:44.366 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-19T08:34:44.378 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:34:44.378 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-19T08:34:44.380 DEBUG:teuthology.orchestra.run.smithi098:> sudo apt-get update 2024-09-19T08:34:44.392 DEBUG:teuthology.orchestra.run.smithi139:> sudo apt-get update 2024-09-19T08:34:44.569 INFO:teuthology.orchestra.run.smithi098.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-19T08:34:44.569 INFO:teuthology.orchestra.run.smithi098.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-19T08:34:44.578 INFO:teuthology.orchestra.run.smithi098.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-19T08:34:44.596 INFO:teuthology.orchestra.run.smithi098.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-19T08:34:44.606 INFO:teuthology.orchestra.run.smithi098.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy InRelease 2024-09-19T08:34:44.615 INFO:teuthology.orchestra.run.smithi139.stdout:Ign:1 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy InRelease 2024-09-19T08:34:44.627 INFO:teuthology.orchestra.run.smithi098.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-19T08:34:44.637 INFO:teuthology.orchestra.run.smithi139.stdout:Get:2 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-19T08:34:44.649 INFO:teuthology.orchestra.run.smithi098.stdout:Ign:7 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-19T08:34:44.660 INFO:teuthology.orchestra.run.smithi139.stdout:Ign:3 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-19T08:34:44.671 INFO:teuthology.orchestra.run.smithi098.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,419 B] 2024-09-19T08:34:44.682 INFO:teuthology.orchestra.run.smithi139.stdout:Get:4 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,419 B] 2024-09-19T08:34:44.697 INFO:teuthology.orchestra.run.smithi098.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-09-19T08:34:44.700 INFO:teuthology.orchestra.run.smithi139.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-19T08:34:44.701 INFO:teuthology.orchestra.run.smithi139.stdout:Hit:6 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-19T08:34:44.722 INFO:teuthology.orchestra.run.smithi139.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-09-19T08:34:44.772 INFO:teuthology.orchestra.run.smithi139.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-19T08:34:44.855 INFO:teuthology.orchestra.run.smithi139.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-19T08:34:44.947 INFO:teuthology.orchestra.run.smithi098.stdout:Fetched 29.0 kB in 0s (77.6 kB/s) 2024-09-19T08:34:45.048 INFO:teuthology.orchestra.run.smithi139.stdout:Fetched 29.0 kB in 0s (61.1 kB/s) 2024-09-19T08:34:45.978 INFO:teuthology.orchestra.run.smithi098.stdout:Reading package lists... 2024-09-19T08:34:45.993 DEBUG:teuthology.orchestra.run.smithi098:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.1.1-361-g2f599a46-1jammy cephadm=19.1.1-361-g2f599a46-1jammy ceph-mds=19.1.1-361-g2f599a46-1jammy ceph-mgr=19.1.1-361-g2f599a46-1jammy ceph-common=19.1.1-361-g2f599a46-1jammy ceph-fuse=19.1.1-361-g2f599a46-1jammy ceph-test=19.1.1-361-g2f599a46-1jammy ceph-volume=19.1.1-361-g2f599a46-1jammy radosgw=19.1.1-361-g2f599a46-1jammy python3-rados=19.1.1-361-g2f599a46-1jammy python3-rgw=19.1.1-361-g2f599a46-1jammy python3-cephfs=19.1.1-361-g2f599a46-1jammy python3-rbd=19.1.1-361-g2f599a46-1jammy libcephfs2=19.1.1-361-g2f599a46-1jammy libcephfs-dev=19.1.1-361-g2f599a46-1jammy librados2=19.1.1-361-g2f599a46-1jammy librbd1=19.1.1-361-g2f599a46-1jammy rbd-fuse=19.1.1-361-g2f599a46-1jammy python3-cephfs=19.1.1-361-g2f599a46-1jammy cephfs-shell=19.1.1-361-g2f599a46-1jammy cephfs-top=19.1.1-361-g2f599a46-1jammy cephfs-mirror=19.1.1-361-g2f599a46-1jammy 2024-09-19T08:34:46.049 INFO:teuthology.orchestra.run.smithi098.stdout:Reading package lists... 2024-09-19T08:34:46.066 INFO:teuthology.orchestra.run.smithi139.stdout:Reading package lists... 2024-09-19T08:34:46.083 DEBUG:teuthology.orchestra.run.smithi139:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.1.1-361-g2f599a46-1jammy cephadm=19.1.1-361-g2f599a46-1jammy ceph-mds=19.1.1-361-g2f599a46-1jammy ceph-mgr=19.1.1-361-g2f599a46-1jammy ceph-common=19.1.1-361-g2f599a46-1jammy ceph-fuse=19.1.1-361-g2f599a46-1jammy ceph-test=19.1.1-361-g2f599a46-1jammy ceph-volume=19.1.1-361-g2f599a46-1jammy radosgw=19.1.1-361-g2f599a46-1jammy python3-rados=19.1.1-361-g2f599a46-1jammy python3-rgw=19.1.1-361-g2f599a46-1jammy python3-cephfs=19.1.1-361-g2f599a46-1jammy python3-rbd=19.1.1-361-g2f599a46-1jammy libcephfs2=19.1.1-361-g2f599a46-1jammy libcephfs-dev=19.1.1-361-g2f599a46-1jammy librados2=19.1.1-361-g2f599a46-1jammy librbd1=19.1.1-361-g2f599a46-1jammy rbd-fuse=19.1.1-361-g2f599a46-1jammy python3-cephfs=19.1.1-361-g2f599a46-1jammy cephfs-shell=19.1.1-361-g2f599a46-1jammy cephfs-top=19.1.1-361-g2f599a46-1jammy cephfs-mirror=19.1.1-361-g2f599a46-1jammy 2024-09-19T08:34:46.139 INFO:teuthology.orchestra.run.smithi139.stdout:Reading package lists... 2024-09-19T08:34:46.234 INFO:teuthology.orchestra.run.smithi098.stdout:Building dependency tree... 2024-09-19T08:34:46.234 INFO:teuthology.orchestra.run.smithi098.stdout:Reading state information... 2024-09-19T08:34:46.333 INFO:teuthology.orchestra.run.smithi139.stdout:Building dependency tree... 2024-09-19T08:34:46.333 INFO:teuthology.orchestra.run.smithi139.stdout:Reading state information... 2024-09-19T08:34:46.420 INFO:teuthology.orchestra.run.smithi098.stdout:The following packages were automatically installed and are no longer required: 2024-09-19T08:34:46.420 INFO:teuthology.orchestra.run.smithi098.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-19T08:34:46.420 INFO:teuthology.orchestra.run.smithi098.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-19T08:34:46.421 INFO:teuthology.orchestra.run.smithi098.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-19T08:34:46.422 INFO:teuthology.orchestra.run.smithi098.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-19T08:34:46.422 INFO:teuthology.orchestra.run.smithi098.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-19T08:34:46.422 INFO:teuthology.orchestra.run.smithi098.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-19T08:34:46.422 INFO:teuthology.orchestra.run.smithi098.stdout:The following additional packages will be installed: 2024-09-19T08:34:46.422 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-19T08:34:46.422 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-19T08:34:46.422 INFO:teuthology.orchestra.run.smithi098.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-19T08:34:46.423 INFO:teuthology.orchestra.run.smithi098.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-19T08:34:46.424 INFO:teuthology.orchestra.run.smithi098.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-19T08:34:46.424 INFO:teuthology.orchestra.run.smithi098.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-19T08:34:46.424 INFO:teuthology.orchestra.run.smithi098.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-19T08:34:46.424 INFO:teuthology.orchestra.run.smithi098.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-19T08:34:46.424 INFO:teuthology.orchestra.run.smithi098.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-19T08:34:46.424 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-09-19T08:34:46.425 INFO:teuthology.orchestra.run.smithi098.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-09-19T08:34:46.425 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-19T08:34:46.425 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-19T08:34:46.425 INFO:teuthology.orchestra.run.smithi098.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-19T08:34:46.425 INFO:teuthology.orchestra.run.smithi098.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-19T08:34:46.425 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-19T08:34:46.425 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-19T08:34:46.425 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-19T08:34:46.426 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-19T08:34:46.426 INFO:teuthology.orchestra.run.smithi098.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-19T08:34:46.426 INFO:teuthology.orchestra.run.smithi098.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-19T08:34:46.426 INFO:teuthology.orchestra.run.smithi098.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-19T08:34:46.426 INFO:teuthology.orchestra.run.smithi098.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-19T08:34:46.426 INFO:teuthology.orchestra.run.smithi098.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-19T08:34:46.426 INFO:teuthology.orchestra.run.smithi098.stdout:Suggested packages: 2024-09-19T08:34:46.426 INFO:teuthology.orchestra.run.smithi098.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-19T08:34:46.426 INFO:teuthology.orchestra.run.smithi098.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-19T08:34:46.427 INFO:teuthology.orchestra.run.smithi098.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-19T08:34:46.427 INFO:teuthology.orchestra.run.smithi098.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-19T08:34:46.427 INFO:teuthology.orchestra.run.smithi098.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-19T08:34:46.427 INFO:teuthology.orchestra.run.smithi098.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-19T08:34:46.427 INFO:teuthology.orchestra.run.smithi098.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-19T08:34:46.427 INFO:teuthology.orchestra.run.smithi098.stdout:Recommended packages: 2024-09-19T08:34:46.427 INFO:teuthology.orchestra.run.smithi098.stdout: btrfs-tools 2024-09-19T08:34:46.523 INFO:teuthology.orchestra.run.smithi139.stdout:The following packages were automatically installed and are no longer required: 2024-09-19T08:34:46.524 INFO:teuthology.orchestra.run.smithi139.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-19T08:34:46.524 INFO:teuthology.orchestra.run.smithi139.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-19T08:34:46.524 INFO:teuthology.orchestra.run.smithi139.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-19T08:34:46.524 INFO:teuthology.orchestra.run.smithi139.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-19T08:34:46.525 INFO:teuthology.orchestra.run.smithi139.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-19T08:34:46.525 INFO:teuthology.orchestra.run.smithi139.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-19T08:34:46.526 INFO:teuthology.orchestra.run.smithi139.stdout:The following additional packages will be installed: 2024-09-19T08:34:46.527 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-19T08:34:46.527 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-19T08:34:46.527 INFO:teuthology.orchestra.run.smithi139.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-19T08:34:46.527 INFO:teuthology.orchestra.run.smithi139.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-19T08:34:46.527 INFO:teuthology.orchestra.run.smithi139.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-19T08:34:46.527 INFO:teuthology.orchestra.run.smithi139.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-19T08:34:46.527 INFO:teuthology.orchestra.run.smithi139.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-19T08:34:46.527 INFO:teuthology.orchestra.run.smithi139.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-19T08:34:46.527 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-19T08:34:46.527 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-09-19T08:34:46.527 INFO:teuthology.orchestra.run.smithi139.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-09-19T08:34:46.527 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-19T08:34:46.528 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-19T08:34:46.528 INFO:teuthology.orchestra.run.smithi139.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-19T08:34:46.528 INFO:teuthology.orchestra.run.smithi139.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-19T08:34:46.528 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-19T08:34:46.528 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-19T08:34:46.528 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-19T08:34:46.528 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-19T08:34:46.528 INFO:teuthology.orchestra.run.smithi139.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-19T08:34:46.528 INFO:teuthology.orchestra.run.smithi139.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-19T08:34:46.528 INFO:teuthology.orchestra.run.smithi139.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-19T08:34:46.529 INFO:teuthology.orchestra.run.smithi139.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-19T08:34:46.529 INFO:teuthology.orchestra.run.smithi139.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-19T08:34:46.529 INFO:teuthology.orchestra.run.smithi139.stdout:Suggested packages: 2024-09-19T08:34:46.529 INFO:teuthology.orchestra.run.smithi139.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-19T08:34:46.529 INFO:teuthology.orchestra.run.smithi139.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-19T08:34:46.529 INFO:teuthology.orchestra.run.smithi139.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-19T08:34:46.529 INFO:teuthology.orchestra.run.smithi139.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-19T08:34:46.529 INFO:teuthology.orchestra.run.smithi139.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-19T08:34:46.530 INFO:teuthology.orchestra.run.smithi139.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-19T08:34:46.530 INFO:teuthology.orchestra.run.smithi139.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-19T08:34:46.530 INFO:teuthology.orchestra.run.smithi139.stdout:Recommended packages: 2024-09-19T08:34:46.530 INFO:teuthology.orchestra.run.smithi139.stdout: btrfs-tools 2024-09-19T08:34:46.532 INFO:teuthology.orchestra.run.smithi098.stdout:The following NEW packages will be installed: 2024-09-19T08:34:46.532 INFO:teuthology.orchestra.run.smithi098.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-19T08:34:46.532 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-19T08:34:46.533 INFO:teuthology.orchestra.run.smithi098.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-19T08:34:46.533 INFO:teuthology.orchestra.run.smithi098.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-19T08:34:46.533 INFO:teuthology.orchestra.run.smithi098.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-19T08:34:46.533 INFO:teuthology.orchestra.run.smithi098.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-19T08:34:46.533 INFO:teuthology.orchestra.run.smithi098.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-19T08:34:46.534 INFO:teuthology.orchestra.run.smithi098.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-19T08:34:46.534 INFO:teuthology.orchestra.run.smithi098.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-19T08:34:46.535 INFO:teuthology.orchestra.run.smithi098.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-19T08:34:46.535 INFO:teuthology.orchestra.run.smithi098.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-19T08:34:46.535 INFO:teuthology.orchestra.run.smithi098.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-19T08:34:46.535 INFO:teuthology.orchestra.run.smithi098.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-19T08:34:46.535 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-19T08:34:46.535 INFO:teuthology.orchestra.run.smithi098.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-19T08:34:46.535 INFO:teuthology.orchestra.run.smithi098.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-19T08:34:46.535 INFO:teuthology.orchestra.run.smithi098.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-19T08:34:46.535 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-19T08:34:46.535 INFO:teuthology.orchestra.run.smithi098.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-19T08:34:46.536 INFO:teuthology.orchestra.run.smithi098.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-19T08:34:46.536 INFO:teuthology.orchestra.run.smithi098.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-19T08:34:46.536 INFO:teuthology.orchestra.run.smithi098.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-19T08:34:46.536 INFO:teuthology.orchestra.run.smithi098.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-19T08:34:46.536 INFO:teuthology.orchestra.run.smithi098.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-19T08:34:46.536 INFO:teuthology.orchestra.run.smithi098.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-19T08:34:46.536 INFO:teuthology.orchestra.run.smithi098.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-19T08:34:46.536 INFO:teuthology.orchestra.run.smithi098.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-19T08:34:46.536 INFO:teuthology.orchestra.run.smithi098.stdout: socat unzip xmlstarlet zip 2024-09-19T08:34:46.593 INFO:teuthology.orchestra.run.smithi098.stdout:0 upgraded, 124 newly installed, 0 to remove and 331 not upgraded. 2024-09-19T08:34:46.593 INFO:teuthology.orchestra.run.smithi098.stdout:Need to get 181 MB of archives. 2024-09-19T08:34:46.593 INFO:teuthology.orchestra.run.smithi098.stdout:After this operation, 831 MB of additional disk space will be used. 2024-09-19T08:34:46.593 INFO:teuthology.orchestra.run.smithi098.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-19T08:34:46.611 INFO:teuthology.orchestra.run.smithi098.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-19T08:34:46.620 INFO:teuthology.orchestra.run.smithi098.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-19T08:34:46.637 INFO:teuthology.orchestra.run.smithi139.stdout:The following NEW packages will be installed: 2024-09-19T08:34:46.637 INFO:teuthology.orchestra.run.smithi139.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-19T08:34:46.637 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-19T08:34:46.637 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-19T08:34:46.639 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-19T08:34:46.639 INFO:teuthology.orchestra.run.smithi139.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-19T08:34:46.639 INFO:teuthology.orchestra.run.smithi139.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-19T08:34:46.639 INFO:teuthology.orchestra.run.smithi139.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-19T08:34:46.639 INFO:teuthology.orchestra.run.smithi139.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-19T08:34:46.639 INFO:teuthology.orchestra.run.smithi139.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-19T08:34:46.639 INFO:teuthology.orchestra.run.smithi139.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-19T08:34:46.639 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-19T08:34:46.639 INFO:teuthology.orchestra.run.smithi139.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-19T08:34:46.640 INFO:teuthology.orchestra.run.smithi139.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-19T08:34:46.640 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-19T08:34:46.640 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-19T08:34:46.640 INFO:teuthology.orchestra.run.smithi139.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-19T08:34:46.640 INFO:teuthology.orchestra.run.smithi139.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-19T08:34:46.640 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-19T08:34:46.640 INFO:teuthology.orchestra.run.smithi139.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-19T08:34:46.640 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-19T08:34:46.640 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-19T08:34:46.640 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-19T08:34:46.641 INFO:teuthology.orchestra.run.smithi139.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-19T08:34:46.641 INFO:teuthology.orchestra.run.smithi139.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-19T08:34:46.641 INFO:teuthology.orchestra.run.smithi139.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-19T08:34:46.641 INFO:teuthology.orchestra.run.smithi139.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-19T08:34:46.641 INFO:teuthology.orchestra.run.smithi139.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-19T08:34:46.641 INFO:teuthology.orchestra.run.smithi139.stdout: socat unzip xmlstarlet zip 2024-09-19T08:34:46.662 INFO:teuthology.orchestra.run.smithi098.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-19T08:34:46.675 INFO:teuthology.orchestra.run.smithi098.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.1.1-361-g2f599a46-1jammy [3,590 kB] 2024-09-19T08:34:46.683 INFO:teuthology.orchestra.run.smithi098.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-19T08:34:46.686 INFO:teuthology.orchestra.run.smithi098.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-19T08:34:46.698 INFO:teuthology.orchestra.run.smithi098.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-19T08:34:46.746 INFO:teuthology.orchestra.run.smithi098.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-19T08:34:46.750 INFO:teuthology.orchestra.run.smithi098.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-19T08:34:46.761 INFO:teuthology.orchestra.run.smithi098.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-19T08:34:46.765 INFO:teuthology.orchestra.run.smithi098.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-19T08:34:46.765 INFO:teuthology.orchestra.run.smithi098.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-19T08:34:46.773 INFO:teuthology.orchestra.run.smithi098.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-19T08:34:46.775 INFO:teuthology.orchestra.run.smithi098.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-19T08:34:46.777 INFO:teuthology.orchestra.run.smithi098.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-19T08:34:46.779 INFO:teuthology.orchestra.run.smithi098.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-19T08:34:46.779 INFO:teuthology.orchestra.run.smithi098.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-19T08:34:46.781 INFO:teuthology.orchestra.run.smithi139.stdout:0 upgraded, 124 newly installed, 0 to remove and 331 not upgraded. 2024-09-19T08:34:46.781 INFO:teuthology.orchestra.run.smithi139.stdout:Need to get 181 MB of archives. 2024-09-19T08:34:46.781 INFO:teuthology.orchestra.run.smithi139.stdout:After this operation, 831 MB of additional disk space will be used. 2024-09-19T08:34:46.781 INFO:teuthology.orchestra.run.smithi139.stdout:Get:1 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.1.1-361-g2f599a46-1jammy [3,590 kB] 2024-09-19T08:34:46.783 INFO:teuthology.orchestra.run.smithi098.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-19T08:34:46.785 INFO:teuthology.orchestra.run.smithi098.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-19T08:34:46.799 INFO:teuthology.orchestra.run.smithi098.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-19T08:34:46.800 INFO:teuthology.orchestra.run.smithi098.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-19T08:34:46.800 INFO:teuthology.orchestra.run.smithi098.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-19T08:34:46.801 INFO:teuthology.orchestra.run.smithi098.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-19T08:34:46.801 INFO:teuthology.orchestra.run.smithi098.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-19T08:34:46.802 INFO:teuthology.orchestra.run.smithi098.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-19T08:34:46.802 INFO:teuthology.orchestra.run.smithi098.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-19T08:34:46.802 INFO:teuthology.orchestra.run.smithi098.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-19T08:34:46.819 INFO:teuthology.orchestra.run.smithi098.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-19T08:34:46.819 INFO:teuthology.orchestra.run.smithi098.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-19T08:34:46.828 INFO:teuthology.orchestra.run.smithi139.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-19T08:34:46.836 INFO:teuthology.orchestra.run.smithi098.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-19T08:34:46.837 INFO:teuthology.orchestra.run.smithi098.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-19T08:34:46.837 INFO:teuthology.orchestra.run.smithi098.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-19T08:34:46.838 INFO:teuthology.orchestra.run.smithi098.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-19T08:34:46.839 INFO:teuthology.orchestra.run.smithi098.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-19T08:34:46.839 INFO:teuthology.orchestra.run.smithi098.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-19T08:34:46.839 INFO:teuthology.orchestra.run.smithi098.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-19T08:34:46.841 INFO:teuthology.orchestra.run.smithi098.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-19T08:34:46.853 INFO:teuthology.orchestra.run.smithi098.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-19T08:34:46.854 INFO:teuthology.orchestra.run.smithi098.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-19T08:34:46.857 INFO:teuthology.orchestra.run.smithi098.stdout:Get:41 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.1.1-361-g2f599a46-1jammy [3,241 kB] 2024-09-19T08:34:46.872 INFO:teuthology.orchestra.run.smithi098.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-19T08:34:46.872 INFO:teuthology.orchestra.run.smithi098.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-19T08:34:46.872 INFO:teuthology.orchestra.run.smithi098.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-19T08:34:46.873 INFO:teuthology.orchestra.run.smithi098.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-19T08:34:46.874 INFO:teuthology.orchestra.run.smithi098.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-19T08:34:46.876 INFO:teuthology.orchestra.run.smithi098.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-19T08:34:46.877 INFO:teuthology.orchestra.run.smithi098.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-19T08:34:46.878 INFO:teuthology.orchestra.run.smithi098.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-19T08:34:46.891 INFO:teuthology.orchestra.run.smithi098.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-19T08:34:46.893 INFO:teuthology.orchestra.run.smithi098.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-19T08:34:46.906 INFO:teuthology.orchestra.run.smithi098.stdout:Get:52 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.1.1-361-g2f599a46-1jammy [719 kB] 2024-09-19T08:34:46.911 INFO:teuthology.orchestra.run.smithi098.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-19T08:34:46.912 INFO:teuthology.orchestra.run.smithi098.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-19T08:34:46.912 INFO:teuthology.orchestra.run.smithi139.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-19T08:34:46.914 INFO:teuthology.orchestra.run.smithi098.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-19T08:34:46.919 INFO:teuthology.orchestra.run.smithi098.stdout:Get:56 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.1.1-361-g2f599a46-1jammy [357 kB] 2024-09-19T08:34:46.924 INFO:teuthology.orchestra.run.smithi098.stdout:Get:57 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.1.1-361-g2f599a46-1jammy [32.8 kB] 2024-09-19T08:34:46.925 INFO:teuthology.orchestra.run.smithi098.stdout:Get:58 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.1.1-361-g2f599a46-1jammy [180 kB] 2024-09-19T08:34:46.928 INFO:teuthology.orchestra.run.smithi098.stdout:Get:59 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.1.1-361-g2f599a46-1jammy [67.8 kB] 2024-09-19T08:34:46.929 INFO:teuthology.orchestra.run.smithi098.stdout:Get:60 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.1.1-361-g2f599a46-1jammy [333 kB] 2024-09-19T08:34:46.935 INFO:teuthology.orchestra.run.smithi098.stdout:Get:61 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.1.1-361-g2f599a46-1jammy [6,875 kB] 2024-09-19T08:34:46.959 INFO:teuthology.orchestra.run.smithi139.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-19T08:34:46.965 INFO:teuthology.orchestra.run.smithi098.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-19T08:34:46.966 INFO:teuthology.orchestra.run.smithi098.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-19T08:34:46.967 INFO:teuthology.orchestra.run.smithi098.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-19T08:34:46.967 INFO:teuthology.orchestra.run.smithi098.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-19T08:34:46.968 INFO:teuthology.orchestra.run.smithi098.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-19T08:34:46.985 INFO:teuthology.orchestra.run.smithi098.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-19T08:34:46.987 INFO:teuthology.orchestra.run.smithi098.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-19T08:34:46.988 INFO:teuthology.orchestra.run.smithi098.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-19T08:34:47.002 INFO:teuthology.orchestra.run.smithi098.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-19T08:34:47.003 INFO:teuthology.orchestra.run.smithi098.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-19T08:34:47.006 INFO:teuthology.orchestra.run.smithi098.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-19T08:34:47.006 INFO:teuthology.orchestra.run.smithi098.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-19T08:34:47.007 INFO:teuthology.orchestra.run.smithi098.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-19T08:34:47.007 INFO:teuthology.orchestra.run.smithi098.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-19T08:34:47.026 INFO:teuthology.orchestra.run.smithi098.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-19T08:34:47.026 INFO:teuthology.orchestra.run.smithi098.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-19T08:34:47.029 INFO:teuthology.orchestra.run.smithi098.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-19T08:34:47.031 INFO:teuthology.orchestra.run.smithi098.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-19T08:34:47.032 INFO:teuthology.orchestra.run.smithi098.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-19T08:34:47.032 INFO:teuthology.orchestra.run.smithi098.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-19T08:34:47.045 INFO:teuthology.orchestra.run.smithi098.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-19T08:34:47.048 INFO:teuthology.orchestra.run.smithi098.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-19T08:34:47.048 INFO:teuthology.orchestra.run.smithi098.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-19T08:34:47.049 INFO:teuthology.orchestra.run.smithi098.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-19T08:34:47.052 INFO:teuthology.orchestra.run.smithi098.stdout:Get:86 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.1.1-361-g2f599a46-1jammy [112 kB] 2024-09-19T08:34:47.053 INFO:teuthology.orchestra.run.smithi098.stdout:Get:87 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.1.1-361-g2f599a46-1jammy [472 kB] 2024-09-19T08:34:47.060 INFO:teuthology.orchestra.run.smithi098.stdout:Get:88 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.1.1-361-g2f599a46-1jammy [25.9 MB] 2024-09-19T08:34:47.065 INFO:teuthology.orchestra.run.smithi098.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-19T08:34:47.069 INFO:teuthology.orchestra.run.smithi098.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-19T08:34:47.069 INFO:teuthology.orchestra.run.smithi098.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-19T08:34:47.072 INFO:teuthology.orchestra.run.smithi098.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-19T08:34:47.073 INFO:teuthology.orchestra.run.smithi098.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-19T08:34:47.073 INFO:teuthology.orchestra.run.smithi098.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-19T08:34:47.084 INFO:teuthology.orchestra.run.smithi098.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-19T08:34:47.086 INFO:teuthology.orchestra.run.smithi098.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-19T08:34:47.086 INFO:teuthology.orchestra.run.smithi098.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-19T08:34:47.093 INFO:teuthology.orchestra.run.smithi098.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-19T08:34:47.103 INFO:teuthology.orchestra.run.smithi098.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-19T08:34:47.103 INFO:teuthology.orchestra.run.smithi098.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-19T08:34:47.105 INFO:teuthology.orchestra.run.smithi098.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-19T08:34:47.105 INFO:teuthology.orchestra.run.smithi098.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-19T08:34:47.163 INFO:teuthology.orchestra.run.smithi139.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-19T08:34:47.257 INFO:teuthology.orchestra.run.smithi139.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-19T08:34:47.270 INFO:teuthology.orchestra.run.smithi139.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-19T08:34:47.321 INFO:teuthology.orchestra.run.smithi139.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-19T08:34:47.501 INFO:teuthology.orchestra.run.smithi139.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-19T08:34:47.512 INFO:teuthology.orchestra.run.smithi139.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-19T08:34:47.542 INFO:teuthology.orchestra.run.smithi139.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-19T08:34:47.551 INFO:teuthology.orchestra.run.smithi139.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-19T08:34:47.553 INFO:teuthology.orchestra.run.smithi139.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-19T08:34:47.570 INFO:teuthology.orchestra.run.smithi139.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-19T08:34:47.575 INFO:teuthology.orchestra.run.smithi139.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-19T08:34:47.580 INFO:teuthology.orchestra.run.smithi139.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-19T08:34:47.585 INFO:teuthology.orchestra.run.smithi139.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-19T08:34:47.587 INFO:teuthology.orchestra.run.smithi139.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-19T08:34:47.623 INFO:teuthology.orchestra.run.smithi098.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.1.1-361-g2f599a46-1jammy [5,145 kB] 2024-09-19T08:34:47.668 INFO:teuthology.orchestra.run.smithi139.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-19T08:34:47.672 INFO:teuthology.orchestra.run.smithi139.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-19T08:34:47.676 INFO:teuthology.orchestra.run.smithi139.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-19T08:34:47.677 INFO:teuthology.orchestra.run.smithi139.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-19T08:34:47.678 INFO:teuthology.orchestra.run.smithi139.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-19T08:34:47.680 INFO:teuthology.orchestra.run.smithi139.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-19T08:34:47.680 INFO:teuthology.orchestra.run.smithi139.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-19T08:34:47.681 INFO:teuthology.orchestra.run.smithi139.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-19T08:34:47.681 INFO:teuthology.orchestra.run.smithi139.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-19T08:34:47.688 INFO:teuthology.orchestra.run.smithi098.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.1.1-361-g2f599a46-1jammy [245 kB] 2024-09-19T08:34:47.690 INFO:teuthology.orchestra.run.smithi098.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.1.1-361-g2f599a46-1jammy [124 kB] 2024-09-19T08:34:47.692 INFO:teuthology.orchestra.run.smithi098.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.1.1-361-g2f599a46-1jammy [1,555 kB] 2024-09-19T08:34:47.705 INFO:teuthology.orchestra.run.smithi098.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.1.1-361-g2f599a46-1jammy [6,142 kB] 2024-09-19T08:34:47.712 INFO:teuthology.orchestra.run.smithi139.stdout:Get:28 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.1.1-361-g2f599a46-1jammy [3,241 kB] 2024-09-19T08:34:47.758 INFO:teuthology.orchestra.run.smithi139.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-19T08:34:47.768 INFO:teuthology.orchestra.run.smithi139.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-19T08:34:47.769 INFO:teuthology.orchestra.run.smithi139.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-19T08:34:47.774 INFO:teuthology.orchestra.run.smithi139.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-19T08:34:47.776 INFO:teuthology.orchestra.run.smithi139.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-19T08:34:47.776 INFO:teuthology.orchestra.run.smithi139.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-19T08:34:47.777 INFO:teuthology.orchestra.run.smithi139.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-19T08:34:47.779 INFO:teuthology.orchestra.run.smithi139.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-19T08:34:47.780 INFO:teuthology.orchestra.run.smithi139.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-19T08:34:47.812 INFO:teuthology.orchestra.run.smithi098.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.1.1-361-g2f599a46-1jammy [22.8 MB] 2024-09-19T08:34:47.843 INFO:teuthology.orchestra.run.smithi139.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-19T08:34:47.866 INFO:teuthology.orchestra.run.smithi139.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-19T08:34:47.867 INFO:teuthology.orchestra.run.smithi139.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-19T08:34:47.868 INFO:teuthology.orchestra.run.smithi139.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-19T08:34:47.879 INFO:teuthology.orchestra.run.smithi139.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-19T08:34:47.879 INFO:teuthology.orchestra.run.smithi139.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-19T08:34:47.929 INFO:teuthology.orchestra.run.smithi139.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-19T08:34:47.930 INFO:teuthology.orchestra.run.smithi139.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-19T08:34:47.932 INFO:teuthology.orchestra.run.smithi139.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-19T08:34:47.937 INFO:teuthology.orchestra.run.smithi139.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-19T08:34:48.014 INFO:teuthology.orchestra.run.smithi139.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-19T08:34:48.017 INFO:teuthology.orchestra.run.smithi139.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-19T08:34:48.018 INFO:teuthology.orchestra.run.smithi139.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-19T08:34:48.023 INFO:teuthology.orchestra.run.smithi139.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-19T08:34:48.024 INFO:teuthology.orchestra.run.smithi139.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-19T08:34:48.101 INFO:teuthology.orchestra.run.smithi139.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-19T08:34:48.106 INFO:teuthology.orchestra.run.smithi139.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-19T08:34:48.220 INFO:teuthology.orchestra.run.smithi139.stdout:Get:55 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.1.1-361-g2f599a46-1jammy [719 kB] 2024-09-19T08:34:48.281 INFO:teuthology.orchestra.run.smithi139.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-19T08:34:48.283 INFO:teuthology.orchestra.run.smithi139.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-19T08:34:48.286 INFO:teuthology.orchestra.run.smithi139.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-19T08:34:48.286 INFO:teuthology.orchestra.run.smithi139.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-19T08:34:48.289 INFO:teuthology.orchestra.run.smithi139.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-19T08:34:48.302 INFO:teuthology.orchestra.run.smithi139.stdout:Get:61 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.1.1-361-g2f599a46-1jammy [357 kB] 2024-09-19T08:34:48.310 INFO:teuthology.orchestra.run.smithi139.stdout:Get:62 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.1.1-361-g2f599a46-1jammy [32.8 kB] 2024-09-19T08:34:48.324 INFO:teuthology.orchestra.run.smithi139.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-19T08:34:48.326 INFO:teuthology.orchestra.run.smithi139.stdout:Get:64 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.1.1-361-g2f599a46-1jammy [180 kB] 2024-09-19T08:34:48.326 INFO:teuthology.orchestra.run.smithi139.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-19T08:34:48.327 INFO:teuthology.orchestra.run.smithi139.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-19T08:34:48.328 INFO:teuthology.orchestra.run.smithi098.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.1.1-361-g2f599a46-1jammy [14.1 kB] 2024-09-19T08:34:48.328 INFO:teuthology.orchestra.run.smithi098.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.1.1-361-g2f599a46-1jammy [877 kB] 2024-09-19T08:34:48.330 INFO:teuthology.orchestra.run.smithi139.stdout:Get:67 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.1.1-361-g2f599a46-1jammy [67.8 kB] 2024-09-19T08:34:48.332 INFO:teuthology.orchestra.run.smithi139.stdout:Get:68 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.1.1-361-g2f599a46-1jammy [333 kB] 2024-09-19T08:34:48.341 INFO:teuthology.orchestra.run.smithi098.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.1.1-361-g2f599a46-1jammy [2,462 kB] 2024-09-19T08:34:48.353 INFO:teuthology.orchestra.run.smithi139.stdout:Get:69 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.1.1-361-g2f599a46-1jammy [6,875 kB] 2024-09-19T08:34:48.373 INFO:teuthology.orchestra.run.smithi098.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.1.1-361-g2f599a46-1jammy [794 kB] 2024-09-19T08:34:48.383 INFO:teuthology.orchestra.run.smithi098.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.1.1-361-g2f599a46-1jammy [152 kB] 2024-09-19T08:34:48.385 INFO:teuthology.orchestra.run.smithi098.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.1.1-361-g2f599a46-1jammy [2,347 kB] 2024-09-19T08:34:48.391 INFO:teuthology.orchestra.run.smithi139.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-19T08:34:48.391 INFO:teuthology.orchestra.run.smithi139.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-19T08:34:48.406 INFO:teuthology.orchestra.run.smithi098.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.1.1-361-g2f599a46-1jammy [8,625 kB] 2024-09-19T08:34:48.461 INFO:teuthology.orchestra.run.smithi139.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-19T08:34:48.462 INFO:teuthology.orchestra.run.smithi139.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-19T08:34:48.463 INFO:teuthology.orchestra.run.smithi139.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-19T08:34:48.464 INFO:teuthology.orchestra.run.smithi139.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-19T08:34:48.465 INFO:teuthology.orchestra.run.smithi139.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-19T08:34:48.466 INFO:teuthology.orchestra.run.smithi139.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-19T08:34:48.472 INFO:teuthology.orchestra.run.smithi139.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-19T08:34:48.548 INFO:teuthology.orchestra.run.smithi139.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-19T08:34:48.551 INFO:teuthology.orchestra.run.smithi139.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-19T08:34:48.553 INFO:teuthology.orchestra.run.smithi139.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-19T08:34:48.577 INFO:teuthology.orchestra.run.smithi098.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.1.1-361-g2f599a46-1jammy [14.2 kB] 2024-09-19T08:34:48.578 INFO:teuthology.orchestra.run.smithi098.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.1.1-361-g2f599a46-1jammy [51.4 MB] 2024-09-19T08:34:48.635 INFO:teuthology.orchestra.run.smithi139.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-19T08:34:48.642 INFO:teuthology.orchestra.run.smithi139.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-19T08:34:48.643 INFO:teuthology.orchestra.run.smithi139.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-19T08:34:48.646 INFO:teuthology.orchestra.run.smithi139.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-19T08:34:48.647 INFO:teuthology.orchestra.run.smithi139.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-19T08:34:48.658 INFO:teuthology.orchestra.run.smithi139.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-19T08:34:48.659 INFO:teuthology.orchestra.run.smithi139.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-19T08:34:48.721 INFO:teuthology.orchestra.run.smithi139.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-19T08:34:48.721 INFO:teuthology.orchestra.run.smithi139.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-19T08:34:48.723 INFO:teuthology.orchestra.run.smithi139.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-19T08:34:48.804 INFO:teuthology.orchestra.run.smithi139.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-19T08:34:48.808 INFO:teuthology.orchestra.run.smithi139.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-19T08:34:48.810 INFO:teuthology.orchestra.run.smithi139.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-19T08:34:48.828 INFO:teuthology.orchestra.run.smithi139.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-19T08:34:48.828 INFO:teuthology.orchestra.run.smithi139.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-19T08:34:48.829 INFO:teuthology.orchestra.run.smithi139.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-19T08:34:48.834 INFO:teuthology.orchestra.run.smithi139.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-19T08:34:48.887 INFO:teuthology.orchestra.run.smithi139.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-19T08:34:48.922 INFO:teuthology.orchestra.run.smithi139.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.1.1-361-g2f599a46-1jammy [112 kB] 2024-09-19T08:34:48.925 INFO:teuthology.orchestra.run.smithi139.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.1.1-361-g2f599a46-1jammy [472 kB] 2024-09-19T08:34:48.967 INFO:teuthology.orchestra.run.smithi139.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.1.1-361-g2f599a46-1jammy [25.9 MB] 2024-09-19T08:34:49.960 INFO:teuthology.orchestra.run.smithi098.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.1.1-361-g2f599a46-1jammy [130 kB] 2024-09-19T08:34:49.962 INFO:teuthology.orchestra.run.smithi098.stdout:Get:119 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.1.1-361-g2f599a46-1jammy [236 kB] 2024-09-19T08:34:49.967 INFO:teuthology.orchestra.run.smithi098.stdout:Get:120 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.1.1-361-g2f599a46-1jammy [34.0 kB] 2024-09-19T08:34:49.968 INFO:teuthology.orchestra.run.smithi098.stdout:Get:121 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.1.1-361-g2f599a46-1jammy [23.5 kB] 2024-09-19T08:34:49.969 INFO:teuthology.orchestra.run.smithi098.stdout:Get:122 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.1.1-361-g2f599a46-1jammy [39.0 kB] 2024-09-19T08:34:49.970 INFO:teuthology.orchestra.run.smithi098.stdout:Get:123 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.1.1-361-g2f599a46-1jammy [13.6 MB] 2024-09-19T08:34:50.221 INFO:teuthology.orchestra.run.smithi098.stdout:Get:124 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.1.1-361-g2f599a46-1jammy [91.5 kB] 2024-09-19T08:34:50.271 INFO:teuthology.orchestra.run.smithi139.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.1.1-361-g2f599a46-1jammy [5,145 kB] 2024-09-19T08:34:50.361 INFO:teuthology.orchestra.run.smithi139.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.1.1-361-g2f599a46-1jammy [245 kB] 2024-09-19T08:34:50.365 INFO:teuthology.orchestra.run.smithi139.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.1.1-361-g2f599a46-1jammy [124 kB] 2024-09-19T08:34:50.367 INFO:teuthology.orchestra.run.smithi139.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.1.1-361-g2f599a46-1jammy [1,555 kB] 2024-09-19T08:34:50.409 INFO:teuthology.orchestra.run.smithi139.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.1.1-361-g2f599a46-1jammy [6,142 kB] 2024-09-19T08:34:50.549 INFO:teuthology.orchestra.run.smithi139.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.1.1-361-g2f599a46-1jammy [22.8 MB] 2024-09-19T08:34:50.853 INFO:teuthology.orchestra.run.smithi098.stdout:Fetched 181 MB in 4s (49.3 MB/s) 2024-09-19T08:34:51.001 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-19T08:34:51.100 INFO:teuthology.orchestra.run.smithi098.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-19T08:34:51.105 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-19T08:34:51.149 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-19T08:34:51.166 INFO:teuthology.orchestra.run.smithi139.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.1.1-361-g2f599a46-1jammy [14.1 kB] 2024-09-19T08:34:51.166 INFO:teuthology.orchestra.run.smithi139.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.1.1-361-g2f599a46-1jammy [877 kB] 2024-09-19T08:34:51.180 INFO:teuthology.orchestra.run.smithi139.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.1.1-361-g2f599a46-1jammy [2,462 kB] 2024-09-19T08:34:51.216 INFO:teuthology.orchestra.run.smithi139.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.1.1-361-g2f599a46-1jammy [794 kB] 2024-09-19T08:34:51.226 INFO:teuthology.orchestra.run.smithi139.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.1.1-361-g2f599a46-1jammy [152 kB] 2024-09-19T08:34:51.228 INFO:teuthology.orchestra.run.smithi139.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.1.1-361-g2f599a46-1jammy [2,347 kB] 2024-09-19T08:34:51.285 INFO:teuthology.orchestra.run.smithi139.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.1.1-361-g2f599a46-1jammy [8,625 kB] 2024-09-19T08:34:51.418 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-19T08:34:51.434 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-19T08:34:51.469 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-19T08:34:51.484 INFO:teuthology.orchestra.run.smithi139.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.1.1-361-g2f599a46-1jammy [14.2 kB] 2024-09-19T08:34:51.484 INFO:teuthology.orchestra.run.smithi139.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.1.1-361-g2f599a46-1jammy [51.4 MB] 2024-09-19T08:34:51.805 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-cmd2. 2024-09-19T08:34:51.831 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-19T08:34:51.871 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-19T08:34:52.225 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-19T08:34:52.251 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-19T08:34:52.333 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-19T08:34:52.695 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-19T08:34:52.721 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-19T08:34:52.762 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-19T08:34:52.822 INFO:teuthology.orchestra.run.smithi139.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.1.1-361-g2f599a46-1jammy [130 kB] 2024-09-19T08:34:52.825 INFO:teuthology.orchestra.run.smithi139.stdout:Get:119 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.1.1-361-g2f599a46-1jammy [236 kB] 2024-09-19T08:34:52.829 INFO:teuthology.orchestra.run.smithi139.stdout:Get:120 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.1.1-361-g2f599a46-1jammy [34.0 kB] 2024-09-19T08:34:52.831 INFO:teuthology.orchestra.run.smithi139.stdout:Get:121 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.1.1-361-g2f599a46-1jammy [23.5 kB] 2024-09-19T08:34:52.831 INFO:teuthology.orchestra.run.smithi139.stdout:Get:122 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.1.1-361-g2f599a46-1jammy [39.0 kB] 2024-09-19T08:34:52.832 INFO:teuthology.orchestra.run.smithi139.stdout:Get:123 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.1.1-361-g2f599a46-1jammy [13.6 MB] 2024-09-19T08:34:53.068 INFO:teuthology.orchestra.run.smithi139.stdout:Get:124 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.1.1-361-g2f599a46-1jammy [91.5 kB] 2024-09-19T08:34:53.098 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-19T08:34:53.124 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-19T08:34:53.165 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-19T08:34:53.661 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-19T08:34:53.686 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-19T08:34:53.702 INFO:teuthology.orchestra.run.smithi139.stdout:Fetched 181 MB in 6s (28.3 MB/s) 2024-09-19T08:34:53.744 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-19T08:34:53.839 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-19T08:34:53.924 INFO:teuthology.orchestra.run.smithi139.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-19T08:34:53.929 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-19T08:34:53.971 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-19T08:34:54.130 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-19T08:34:54.156 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-19T08:34:54.197 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-19T08:34:54.391 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-19T08:34:54.417 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-19T08:34:54.449 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-19T08:34:54.550 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-19T08:34:54.577 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-19T08:34:54.617 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-19T08:34:54.760 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-cmd2. 2024-09-19T08:34:54.786 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-19T08:34:54.827 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-19T08:34:55.029 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-19T08:34:55.055 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-19T08:34:55.103 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-19T08:34:55.171 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-19T08:34:55.197 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-19T08:34:55.280 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-19T08:34:55.431 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package librados2. 2024-09-19T08:34:55.458 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../010-librados2_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:34:55.498 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking librados2 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:34:55.625 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-19T08:34:55.651 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-19T08:34:55.692 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-19T08:34:55.986 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-19T08:34:55.993 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package librbd1. 2024-09-19T08:34:56.012 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-19T08:34:56.019 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../011-librbd1_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:34:56.044 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-19T08:34:56.059 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking librbd1 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:34:56.464 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-19T08:34:56.491 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-19T08:34:56.529 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package libcephfs2. 2024-09-19T08:34:56.539 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-19T08:34:56.555 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../012-libcephfs2_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:34:56.604 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking libcephfs2 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:34:56.899 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-rados. 2024-09-19T08:34:56.913 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../013-python3-rados_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:34:56.957 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-rados (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:34:56.976 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-19T08:34:57.003 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-19T08:34:57.043 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-19T08:34:57.268 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-19T08:34:57.294 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T08:34:57.334 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-ceph-argparse (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:34:57.337 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-19T08:34:57.363 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-19T08:34:57.395 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-19T08:34:57.587 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-cephfs. 2024-09-19T08:34:57.602 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../015-python3-cephfs_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:34:57.645 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-cephfs (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:34:57.732 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-19T08:34:57.758 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-19T08:34:57.798 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-19T08:34:58.240 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-19T08:34:58.266 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../016-python3-ceph-common_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T08:34:58.315 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-ceph-common (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:34:58.326 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package librados2. 2024-09-19T08:34:58.352 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../010-librados2_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:34:58.393 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking librados2 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:34:58.660 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-prettytable. 2024-09-19T08:34:58.686 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-09-19T08:34:58.729 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-19T08:34:58.812 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package librbd1. 2024-09-19T08:34:58.827 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../011-librbd1_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:34:58.862 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking librbd1 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:34:58.988 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-rbd. 2024-09-19T08:34:59.003 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../018-python3-rbd_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:34:59.046 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-rbd (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:34:59.240 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package libcephfs2. 2024-09-19T08:34:59.267 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../012-libcephfs2_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:34:59.298 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking libcephfs2 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:34:59.400 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-19T08:34:59.425 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-19T08:34:59.466 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-19T08:34:59.626 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-rados. 2024-09-19T08:34:59.652 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../013-python3-rados_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:34:59.709 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-rados (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:34:59.777 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-19T08:34:59.792 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-19T08:34:59.844 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-19T08:35:00.028 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-19T08:35:00.043 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T08:35:00.086 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-ceph-argparse (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:00.188 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-19T08:35:00.214 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-19T08:35:00.255 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-19T08:35:00.314 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-cephfs. 2024-09-19T08:35:00.329 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../015-python3-cephfs_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:00.364 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-cephfs (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:00.567 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package lua5.1. 2024-09-19T08:35:00.593 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-19T08:35:00.633 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-19T08:35:00.650 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-19T08:35:00.676 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../016-python3-ceph-common_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T08:35:00.717 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-ceph-common (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:00.928 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package lua-any. 2024-09-19T08:35:00.955 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-09-19T08:35:00.995 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-19T08:35:01.003 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-prettytable. 2024-09-19T08:35:01.029 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-09-19T08:35:01.061 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-19T08:35:01.239 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package zip. 2024-09-19T08:35:01.265 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-09-19T08:35:01.306 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking zip (3.0-12build2) ... 2024-09-19T08:35:01.330 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-rbd. 2024-09-19T08:35:01.357 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../018-python3-rbd_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:01.397 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-rbd (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:01.618 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package unzip. 2024-09-19T08:35:01.644 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-19T08:35:01.684 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-19T08:35:01.742 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-19T08:35:01.768 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-19T08:35:01.825 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-19T08:35:02.022 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package luarocks. 2024-09-19T08:35:02.048 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-19T08:35:02.061 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-19T08:35:02.076 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-19T08:35:02.088 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-19T08:35:02.119 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-19T08:35:02.422 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-19T08:35:02.437 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-19T08:35:02.471 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package librgw2. 2024-09-19T08:35:02.480 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-19T08:35:02.498 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../027-librgw2_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:02.538 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking librgw2 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:02.817 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package lua5.1. 2024-09-19T08:35:02.843 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-19T08:35:02.884 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-19T08:35:03.108 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-rgw. 2024-09-19T08:35:03.134 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../028-python3-rgw_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:03.166 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-rgw (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:03.195 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package lua-any. 2024-09-19T08:35:03.221 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-09-19T08:35:03.253 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-19T08:35:03.627 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-19T08:35:03.647 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package zip. 2024-09-19T08:35:03.654 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-19T08:35:03.674 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-09-19T08:35:03.694 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-19T08:35:03.714 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking zip (3.0-12build2) ... 2024-09-19T08:35:04.005 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package libradosstriper1. 2024-09-19T08:35:04.026 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package unzip. 2024-09-19T08:35:04.032 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../030-libradosstriper1_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:04.053 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-19T08:35:04.072 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking libradosstriper1 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:04.092 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-19T08:35:04.446 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package luarocks. 2024-09-19T08:35:04.472 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-19T08:35:04.483 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package ceph-common. 2024-09-19T08:35:04.504 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-19T08:35:04.510 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../031-ceph-common_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:04.550 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking ceph-common (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:04.870 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package librgw2. 2024-09-19T08:35:04.896 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../027-librgw2_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:04.937 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking librgw2 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:05.414 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-rgw. 2024-09-19T08:35:05.429 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../028-python3-rgw_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:05.493 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-rgw (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:05.817 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-19T08:35:05.843 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-19T08:35:05.864 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package ceph-base. 2024-09-19T08:35:05.883 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-19T08:35:05.890 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../032-ceph-base_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:05.931 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking ceph-base (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:06.136 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package libradosstriper1. 2024-09-19T08:35:06.162 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../030-libradosstriper1_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:06.203 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking libradosstriper1 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:06.494 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-19T08:35:06.520 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-19T08:35:06.560 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-19T08:35:06.623 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package ceph-common. 2024-09-19T08:35:06.649 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../031-ceph-common_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:06.689 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking ceph-common (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:06.830 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-cheroot. 2024-09-19T08:35:06.856 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-19T08:35:06.888 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-19T08:35:07.217 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-19T08:35:07.248 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-19T08:35:07.283 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-19T08:35:07.539 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-19T08:35:07.562 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-19T08:35:07.594 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-19T08:35:07.897 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-19T08:35:07.923 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-19T08:35:07.964 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-19T08:35:07.996 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package ceph-base. 2024-09-19T08:35:08.022 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../032-ceph-base_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:08.064 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking ceph-base (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:08.250 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-tempora. 2024-09-19T08:35:08.276 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-09-19T08:35:08.317 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-19T08:35:08.587 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-portend. 2024-09-19T08:35:08.613 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-09-19T08:35:08.645 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-19T08:35:08.767 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-19T08:35:08.794 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-19T08:35:08.834 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-19T08:35:08.956 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-19T08:35:08.982 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-19T08:35:09.022 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-19T08:35:09.136 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-cheroot. 2024-09-19T08:35:09.163 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-19T08:35:09.203 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-19T08:35:09.334 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-19T08:35:09.360 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-19T08:35:09.409 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-19T08:35:09.522 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-19T08:35:09.549 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-19T08:35:09.589 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-19T08:35:09.729 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-natsort. 2024-09-19T08:35:09.756 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-09-19T08:35:09.796 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-19T08:35:09.917 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-19T08:35:09.944 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-19T08:35:09.992 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-19T08:35:10.116 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-logutils. 2024-09-19T08:35:10.142 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-09-19T08:35:10.182 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-19T08:35:10.261 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-19T08:35:10.288 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-19T08:35:10.319 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-19T08:35:10.494 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-19T08:35:10.520 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-19T08:35:10.560 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-19T08:35:10.605 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-tempora. 2024-09-19T08:35:10.632 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-09-19T08:35:10.672 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-19T08:35:10.805 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-mako. 2024-09-19T08:35:10.820 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-19T08:35:10.855 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-19T08:35:10.974 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-portend. 2024-09-19T08:35:11.001 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-09-19T08:35:11.041 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-19T08:35:11.142 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-19T08:35:11.168 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-19T08:35:11.208 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-19T08:35:11.327 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-19T08:35:11.354 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-19T08:35:11.394 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-19T08:35:11.511 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-19T08:35:11.538 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-19T08:35:11.578 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-19T08:35:11.613 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-19T08:35:11.629 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-19T08:35:11.663 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-19T08:35:11.872 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-webob. 2024-09-19T08:35:11.899 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-19T08:35:11.939 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-19T08:35:11.983 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-natsort. 2024-09-19T08:35:12.010 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-09-19T08:35:12.050 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-19T08:35:12.267 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-waitress. 2024-09-19T08:35:12.293 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-19T08:35:12.319 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-logutils. 2024-09-19T08:35:12.336 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-19T08:35:12.346 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-09-19T08:35:12.386 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-19T08:35:12.587 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-tempita. 2024-09-19T08:35:12.602 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-19T08:35:12.637 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-19T08:35:12.689 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-19T08:35:12.715 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-19T08:35:12.755 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-19T08:35:12.940 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-paste. 2024-09-19T08:35:12.966 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-19T08:35:13.006 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-19T08:35:13.058 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-mako. 2024-09-19T08:35:13.084 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-19T08:35:13.124 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-19T08:35:13.343 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-19T08:35:13.358 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-19T08:35:13.393 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-19T08:35:13.461 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-19T08:35:13.487 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-19T08:35:13.527 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-19T08:35:13.688 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-19T08:35:13.714 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-19T08:35:13.754 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-19T08:35:13.797 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-19T08:35:13.823 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-19T08:35:13.855 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-19T08:35:14.057 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-webtest. 2024-09-19T08:35:14.072 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-09-19T08:35:14.115 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-19T08:35:14.141 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-webob. 2024-09-19T08:35:14.167 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-19T08:35:14.207 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-19T08:35:14.427 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-pecan. 2024-09-19T08:35:14.453 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-19T08:35:14.493 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-19T08:35:14.561 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-waitress. 2024-09-19T08:35:14.587 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-19T08:35:14.631 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-19T08:35:14.930 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-tempita. 2024-09-19T08:35:14.956 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-19T08:35:14.957 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-certifi. 2024-09-19T08:35:14.983 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-09-19T08:35:14.996 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-19T08:35:15.023 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-19T08:35:15.299 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-paste. 2024-09-19T08:35:15.326 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-19T08:35:15.354 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-idna. 2024-09-19T08:35:15.366 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-19T08:35:15.380 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../057-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-19T08:35:15.412 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-19T08:35:15.685 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-19T08:35:15.701 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-19T08:35:15.715 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-urllib3. 2024-09-19T08:35:15.741 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-19T08:35:15.744 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-19T08:35:15.781 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-19T08:35:16.038 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-19T08:35:16.064 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-19T08:35:16.093 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-requests. 2024-09-19T08:35:16.105 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-19T08:35:16.119 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-19T08:35:16.159 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-19T08:35:16.349 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-webtest. 2024-09-19T08:35:16.376 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-09-19T08:35:16.407 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-19T08:35:16.513 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-19T08:35:16.539 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-19T08:35:16.588 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-19T08:35:16.710 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-pecan. 2024-09-19T08:35:16.737 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-19T08:35:16.777 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-19T08:35:16.882 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-19T08:35:16.909 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T08:35:16.957 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking ceph-mgr-modules-core (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:17.140 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-certifi. 2024-09-19T08:35:17.166 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-09-19T08:35:17.206 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-19T08:35:17.354 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-19T08:35:17.380 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:17.437 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking libsqlite3-mod-ceph (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:17.526 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-idna. 2024-09-19T08:35:17.552 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../057-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-19T08:35:17.592 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-19T08:35:17.698 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-19T08:35:17.725 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-19T08:35:17.765 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-19T08:35:17.912 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-urllib3. 2024-09-19T08:35:17.938 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-19T08:35:17.978 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-19T08:35:18.281 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-requests. 2024-09-19T08:35:18.308 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-19T08:35:18.348 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-19T08:35:18.650 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-19T08:35:18.677 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-19T08:35:18.709 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-19T08:35:19.012 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-19T08:35:19.032 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T08:35:19.119 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-openssl. 2024-09-19T08:35:19.146 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-09-19T08:35:19.185 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-19T08:35:19.195 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking ceph-mgr-modules-core (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:19.530 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package ceph-mgr. 2024-09-19T08:35:19.557 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../065-ceph-mgr_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:19.591 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-19T08:35:19.597 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking ceph-mgr (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:19.618 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:19.658 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking libsqlite3-mod-ceph (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:19.958 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package ceph-mon. 2024-09-19T08:35:19.985 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../066-ceph-mon_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:20.016 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking ceph-mon (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:20.629 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package ceph-osd. 2024-09-19T08:35:20.655 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../067-ceph-osd_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:20.704 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking ceph-osd (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:20.745 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-19T08:35:20.772 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-19T08:35:20.803 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-19T08:35:21.080 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-openssl. 2024-09-19T08:35:21.107 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-09-19T08:35:21.147 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-19T08:35:21.492 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package ceph-mgr. 2024-09-19T08:35:21.518 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../065-ceph-mgr_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:21.558 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking ceph-mgr (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:21.773 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package ceph. 2024-09-19T08:35:21.800 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../068-ceph_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:21.840 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking ceph (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:21.953 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package ceph-mon. 2024-09-19T08:35:21.979 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../066-ceph-mon_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:22.019 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking ceph-mon (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:22.118 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package ceph-fuse. 2024-09-19T08:35:22.144 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../069-ceph-fuse_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:22.176 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking ceph-fuse (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:22.556 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package ceph-osd. 2024-09-19T08:35:22.562 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package ceph-mds. 2024-09-19T08:35:22.583 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../067-ceph-osd_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:22.589 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../070-ceph-mds_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:22.614 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking ceph-osd (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:22.632 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking ceph-mds (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:23.032 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package cephadm. 2024-09-19T08:35:23.059 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../071-cephadm_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:23.099 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking cephadm (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:23.384 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-19T08:35:23.400 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-19T08:35:23.434 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-19T08:35:23.634 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package ceph. 2024-09-19T08:35:23.649 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../068-ceph_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:23.692 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking ceph (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:23.762 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-19T08:35:23.778 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-19T08:35:23.820 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-19T08:35:24.020 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package ceph-fuse. 2024-09-19T08:35:24.046 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../069-ceph-fuse_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:24.086 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking ceph-fuse (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:24.430 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package ceph-mds. 2024-09-19T08:35:24.446 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../070-ceph-mds_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:24.480 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking ceph-mds (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:24.850 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package cephadm. 2024-09-19T08:35:24.859 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-babel. 2024-09-19T08:35:24.876 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../071-cephadm_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:24.886 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-19T08:35:24.916 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking cephadm (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:24.925 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-19T08:35:25.236 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-19T08:35:25.253 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-jinja2. 2024-09-19T08:35:25.262 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-19T08:35:25.280 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-19T08:35:25.302 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-19T08:35:25.329 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-19T08:35:25.596 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-19T08:35:25.612 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-19T08:35:25.654 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-19T08:35:25.665 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-19T08:35:25.692 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T08:35:25.732 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking ceph-mgr-cephadm (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:26.068 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-19T08:35:26.095 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-09-19T08:35:26.160 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-19T08:35:26.471 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-routes. 2024-09-19T08:35:26.498 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-19T08:35:26.538 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-19T08:35:26.659 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-babel. 2024-09-19T08:35:26.686 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-19T08:35:26.726 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-19T08:35:26.858 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-19T08:35:26.885 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T08:35:26.924 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking ceph-mgr-dashboard (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:27.020 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-jinja2. 2024-09-19T08:35:27.047 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-19T08:35:27.078 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-19T08:35:27.431 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-19T08:35:27.458 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T08:35:27.498 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking ceph-mgr-cephadm (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:27.860 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-19T08:35:27.886 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-09-19T08:35:27.926 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-19T08:35:28.138 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-19T08:35:28.153 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-19T08:35:28.170 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-routes. 2024-09-19T08:35:28.196 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-19T08:35:28.197 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-19T08:35:28.228 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-19T08:35:28.565 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-19T08:35:28.591 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T08:35:28.631 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking ceph-mgr-dashboard (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:28.641 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-joblib. 2024-09-19T08:35:28.668 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-19T08:35:28.700 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-19T08:35:29.053 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-19T08:35:29.080 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-19T08:35:29.205 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-19T08:35:29.467 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-sklearn. 2024-09-19T08:35:29.493 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-19T08:35:29.533 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-19T08:35:30.120 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-19T08:35:30.135 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-19T08:35:30.178 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-19T08:35:30.209 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-19T08:35:30.237 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T08:35:30.276 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking ceph-mgr-diskprediction-local (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:30.681 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-joblib. 2024-09-19T08:35:30.708 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-19T08:35:30.747 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-19T08:35:31.084 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-19T08:35:31.111 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-19T08:35:31.220 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-cachetools. 2024-09-19T08:35:31.228 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-19T08:35:31.247 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-09-19T08:35:31.279 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-19T08:35:31.514 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-sklearn. 2024-09-19T08:35:31.541 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-19T08:35:31.574 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-19T08:35:31.597 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-19T08:35:31.600 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-19T08:35:31.640 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-19T08:35:31.993 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-19T08:35:32.020 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-19T08:35:32.060 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-19T08:35:32.348 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-19T08:35:32.375 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T08:35:32.380 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-rsa. 2024-09-19T08:35:32.405 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking ceph-mgr-diskprediction-local (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:32.406 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-09-19T08:35:32.446 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-19T08:35:32.767 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-google-auth. 2024-09-19T08:35:32.794 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-09-19T08:35:32.833 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-19T08:35:33.128 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-19T08:35:33.143 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-19T08:35:33.178 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-19T08:35:33.359 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-cachetools. 2024-09-19T08:35:33.386 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-09-19T08:35:33.425 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-19T08:35:33.456 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-websocket. 2024-09-19T08:35:33.483 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-09-19T08:35:33.523 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-19T08:35:33.703 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-19T08:35:33.730 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-19T08:35:33.769 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-19T08:35:33.851 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-19T08:35:33.878 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-19T08:35:33.918 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-19T08:35:34.064 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-19T08:35:34.080 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-19T08:35:34.114 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-19T08:35:34.375 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-rsa. 2024-09-19T08:35:34.402 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-09-19T08:35:34.433 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-19T08:35:34.754 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-google-auth. 2024-09-19T08:35:34.782 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-09-19T08:35:34.820 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-19T08:35:34.832 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-19T08:35:34.861 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T08:35:34.899 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking ceph-mgr-k8sevents (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:35.157 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-19T08:35:35.185 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-19T08:35:35.218 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-19T08:35:35.223 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-19T08:35:35.245 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-19T08:35:35.284 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-19T08:35:35.526 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-websocket. 2024-09-19T08:35:35.554 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-09-19T08:35:35.593 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-19T08:35:35.663 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-19T08:35:35.690 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-19T08:35:35.729 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-19T08:35:35.871 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-19T08:35:35.899 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-19T08:35:35.962 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-19T08:35:36.016 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package jq. 2024-09-19T08:35:36.044 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-19T08:35:36.082 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-19T08:35:36.335 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package socat. 2024-09-19T08:35:36.363 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-19T08:35:36.401 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-19T08:35:36.584 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-19T08:35:36.598 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T08:35:36.626 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking ceph-mgr-k8sevents (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:36.705 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package xmlstarlet. 2024-09-19T08:35:36.733 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-19T08:35:36.780 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-19T08:35:36.937 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-19T08:35:36.964 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-19T08:35:37.003 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-19T08:35:37.159 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package ceph-test. 2024-09-19T08:35:37.187 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../099-ceph-test_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:37.225 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking ceph-test (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:37.323 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-19T08:35:37.350 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-19T08:35:37.389 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-19T08:35:37.650 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package jq. 2024-09-19T08:35:37.666 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-19T08:35:37.708 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-19T08:35:37.919 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package socat. 2024-09-19T08:35:37.935 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-19T08:35:37.969 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-19T08:35:38.264 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package xmlstarlet. 2024-09-19T08:35:38.281 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-19T08:35:38.322 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-19T08:35:38.685 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package ceph-test. 2024-09-19T08:35:38.701 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../099-ceph-test_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:38.743 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking ceph-test (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:39.530 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package ceph-volume. 2024-09-19T08:35:39.557 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../100-ceph-volume_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T08:35:39.596 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking ceph-volume (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:39.959 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-19T08:35:39.986 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../101-cephfs-mirror_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:40.025 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking cephfs-mirror (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:40.337 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-colorama. 2024-09-19T08:35:40.364 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-09-19T08:35:40.403 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-19T08:35:40.656 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package cephfs-shell. 2024-09-19T08:35:40.672 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../103-cephfs-shell_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T08:35:40.705 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking cephfs-shell (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:40.966 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package cephfs-top. 2024-09-19T08:35:40.982 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../104-cephfs-top_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T08:35:41.025 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking cephfs-top (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:41.055 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package ceph-volume. 2024-09-19T08:35:41.083 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../100-ceph-volume_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T08:35:41.122 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking ceph-volume (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:41.269 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-19T08:35:41.285 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../105-libcephfs-dev_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:41.319 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking libcephfs-dev (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:41.509 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-19T08:35:41.537 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../101-cephfs-mirror_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:41.584 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking cephfs-mirror (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:41.597 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-19T08:35:41.625 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-19T08:35:41.664 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-19T08:35:41.854 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-colorama. 2024-09-19T08:35:41.869 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-09-19T08:35:41.903 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-19T08:35:41.976 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-19T08:35:42.000 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-09-19T08:35:42.043 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-19T08:35:42.164 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package cephfs-shell. 2024-09-19T08:35:42.192 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../103-cephfs-shell_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T08:35:42.231 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking cephfs-shell (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:42.414 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package nvme-cli. 2024-09-19T08:35:42.441 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-19T08:35:42.483 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package cephfs-top. 2024-09-19T08:35:42.488 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-19T08:35:42.510 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../104-cephfs-top_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T08:35:42.550 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking cephfs-top (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:42.810 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-19T08:35:42.826 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../105-libcephfs-dev_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:42.869 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking libcephfs-dev (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:42.869 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package pkg-config. 2024-09-19T08:35:42.893 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-19T08:35:42.935 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-19T08:35:43.146 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-19T08:35:43.162 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-19T08:35:43.171 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-19T08:35:43.187 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-19T08:35:43.205 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-19T08:35:43.229 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-19T08:35:43.517 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-19T08:35:43.532 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-09-19T08:35:43.575 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-19T08:35:43.625 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-19T08:35:43.652 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-19T08:35:43.683 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-19T08:35:43.939 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package nvme-cli. 2024-09-19T08:35:43.967 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-19T08:35:43.986 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-pastescript. 2024-09-19T08:35:44.006 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-19T08:35:44.013 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-09-19T08:35:44.052 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-19T08:35:44.381 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-pluggy. 2024-09-19T08:35:44.409 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-19T08:35:44.419 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package pkg-config. 2024-09-19T08:35:44.447 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-19T08:35:44.448 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-19T08:35:44.485 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-19T08:35:44.730 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-19T08:35:44.746 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-19T08:35:44.759 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-psutil. 2024-09-19T08:35:44.780 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-19T08:35:44.786 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-19T08:35:44.825 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-19T08:35:45.162 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-py. 2024-09-19T08:35:45.189 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-09-19T08:35:45.201 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-19T08:35:45.227 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-19T08:35:45.236 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-19T08:35:45.258 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-19T08:35:45.544 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-pastescript. 2024-09-19T08:35:45.571 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-09-19T08:35:45.573 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-pygments. 2024-09-19T08:35:45.601 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-19T08:35:45.610 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-19T08:35:45.640 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-19T08:35:45.930 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-pluggy. 2024-09-19T08:35:45.957 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-19T08:35:45.996 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-19T08:35:46.061 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-19T08:35:46.088 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-19T08:35:46.127 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-19T08:35:46.308 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-psutil. 2024-09-19T08:35:46.335 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-19T08:35:46.374 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-19T08:35:46.405 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-toml. 2024-09-19T08:35:46.432 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-09-19T08:35:46.471 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-19T08:35:46.702 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-py. 2024-09-19T08:35:46.729 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-09-19T08:35:46.768 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-19T08:35:46.774 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-pytest. 2024-09-19T08:35:46.802 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-19T08:35:46.841 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-19T08:35:47.097 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-pygments. 2024-09-19T08:35:47.124 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-19T08:35:47.153 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package python3-simplejson. 2024-09-19T08:35:47.163 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-19T08:35:47.180 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-19T08:35:47.219 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-19T08:35:47.473 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-19T08:35:47.500 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-19T08:35:47.547 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-19T08:35:47.625 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-19T08:35:47.653 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-19T08:35:47.692 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-19T08:35:47.969 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-toml. 2024-09-19T08:35:47.997 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-09-19T08:35:48.027 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-19T08:35:48.310 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package radosgw. 2024-09-19T08:35:48.322 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-pytest. 2024-09-19T08:35:48.338 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../122-radosgw_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:48.350 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-19T08:35:48.376 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking radosgw (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:48.388 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-19T08:35:48.725 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package python3-simplejson. 2024-09-19T08:35:48.763 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-19T08:35:48.784 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-19T08:35:49.094 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-19T08:35:49.111 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-19T08:35:49.152 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-19T08:35:49.154 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package rbd-fuse. 2024-09-19T08:35:49.170 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../123-rbd-fuse_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:49.212 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking rbd-fuse (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:49.526 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-19T08:35:49.759 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-19T08:35:49.889 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-19T08:35:49.889 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package radosgw. 2024-09-19T08:35:49.918 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../122-radosgw_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:49.956 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking radosgw (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:50.413 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-19T08:35:50.759 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package rbd-fuse. 2024-09-19T08:35:50.775 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../123-rbd-fuse_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T08:35:50.778 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-19T08:35:50.817 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking rbd-fuse (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:51.221 INFO:teuthology.orchestra.run.smithi098.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-19T08:35:51.296 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up cephadm (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:51.356 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-19T08:35:51.605 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-19T08:35:51.731 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-19T08:35:51.788 INFO:teuthology.orchestra.run.smithi098.stdout:Adding system user cephadm....done 2024-09-19T08:35:51.945 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-19T08:35:51.989 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-19T08:35:52.245 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-19T08:35:52.301 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-19T08:35:52.497 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-19T08:35:52.623 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-19T08:35:52.713 INFO:teuthology.orchestra.run.smithi139.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-19T08:35:52.783 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up cephadm (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:52.891 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-19T08:35:53.134 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-19T08:35:53.235 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-19T08:35:53.266 INFO:teuthology.orchestra.run.smithi139.stdout:Adding system user cephadm....done 2024-09-19T08:35:53.422 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-19T08:35:53.544 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-19T08:35:53.707 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-19T08:35:53.905 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-19T08:35:53.958 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-19T08:35:54.084 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-19T08:35:54.173 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-19T08:35:54.319 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-19T08:35:54.424 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-19T08:35:54.578 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-19T08:35:54.684 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-19T08:35:54.704 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-19T08:35:54.927 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-19T08:35:55.014 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-19T08:35:55.178 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-19T08:35:55.374 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-19T08:35:55.438 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-19T08:35:55.634 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-19T08:35:55.698 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-ceph-argparse (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:55.877 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-19T08:35:55.940 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-19T08:35:56.075 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-19T08:35:56.150 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-19T08:35:56.200 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-19T08:35:56.421 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-19T08:35:56.452 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-19T08:35:56.611 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-19T08:35:56.681 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-19T08:35:56.870 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-19T08:35:56.940 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-19T08:35:57.199 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-ceph-argparse (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:35:57.267 INFO:teuthology.orchestra.run.smithi098.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-19T08:35:57.296 INFO:teuthology.orchestra.run.smithi098.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-19T08:35:57.359 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-19T08:35:57.450 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-19T08:35:57.485 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-19T08:35:57.559 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-19T08:35:57.686 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-19T08:35:57.853 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-19T08:35:57.945 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-19T08:35:57.979 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-19T08:35:58.071 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-19T08:35:58.255 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-19T08:35:58.330 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-19T08:35:58.506 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-19T08:35:58.732 INFO:teuthology.orchestra.run.smithi139.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-19T08:35:58.761 INFO:teuthology.orchestra.run.smithi139.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-19T08:35:58.807 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up zip (3.0-12build2) ... 2024-09-19T08:35:58.825 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-19T08:35:58.933 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-19T08:35:58.943 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-19T08:35:59.294 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-19T08:35:59.420 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-19T08:35:59.669 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-19T08:35:59.687 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-19T08:35:59.928 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-19T08:35:59.955 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-19T08:36:00.054 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-19T08:36:00.180 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-19T08:36:00.256 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up zip (3.0-12build2) ... 2024-09-19T08:36:00.382 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-19T08:36:00.464 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-19T08:36:00.841 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-19T08:36:01.084 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-19T08:36:01.100 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-19T08:36:01.335 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-19T08:36:01.460 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-19T08:36:01.461 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-19T08:36:01.604 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-19T08:36:01.762 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-19T08:36:01.905 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-19T08:36:02.071 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-19T08:36:02.289 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-19T08:36:02.431 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-19T08:36:02.540 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-19T08:36:02.682 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-19T08:36:02.800 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-ceph-common (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:02.908 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-19T08:36:03.101 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-19T08:36:03.184 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-19T08:36:03.413 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-19T08:36:03.477 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-19T08:36:03.837 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-19T08:36:04.088 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-19T08:36:04.214 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-ceph-common (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:04.557 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-19T08:36:04.616 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-19T08:36:04.860 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-19T08:36:04.994 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-19T08:36:05.048 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-19T08:36:05.281 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-19T08:36:05.413 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-19T08:36:05.539 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-19T08:36:05.824 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-19T08:36:06.133 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-19T08:36:06.271 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-19T08:36:06.376 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-19T08:36:06.502 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-19T08:36:06.540 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-19T08:36:06.691 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-19T08:36:06.803 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-19T08:36:06.958 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-19T08:36:07.059 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-19T08:36:07.062 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-19T08:36:07.177 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-19T08:36:07.314 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-19T08:36:07.478 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-19T08:36:07.565 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-19T08:36:07.738 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-19T08:36:07.816 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-19T08:36:07.997 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-19T08:36:08.076 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-19T08:36:08.123 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-19T08:36:08.185 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-19T08:36:08.449 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-19T08:36:08.469 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-19T08:36:08.595 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-19T08:36:08.700 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-19T08:36:08.888 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-19T08:36:08.943 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-19T08:36:09.181 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-19T08:36:09.194 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-19T08:36:09.416 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-19T08:36:09.445 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-19T08:36:09.693 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-19T08:36:09.701 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-19T08:36:09.713 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-19T08:36:09.790 INFO:teuthology.orchestra.run.smithi098.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-19T08:36:09.839 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-19T08:36:10.011 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-19T08:36:10.137 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-19T08:36:10.139 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-19T08:36:10.282 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-19T08:36:10.530 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-19T08:36:10.567 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-19T08:36:10.860 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-19T08:36:10.948 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-19T08:36:11.049 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-19T08:36:11.111 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-19T08:36:11.237 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-19T08:36:11.300 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-19T08:36:11.362 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-19T08:36:11.404 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-19T08:36:11.452 INFO:teuthology.orchestra.run.smithi139.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-19T08:36:11.697 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-19T08:36:11.719 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-19T08:36:12.045 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-19T08:36:12.102 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-19T08:36:12.330 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-19T08:36:12.475 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-19T08:36:12.541 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-19T08:36:12.726 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-19T08:36:12.793 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-19T08:36:12.852 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-19T08:36:13.044 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-19T08:36:13.120 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-19T08:36:13.245 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-19T08:36:13.303 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-19T08:36:13.563 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-19T08:36:13.680 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-19T08:36:13.864 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-19T08:36:13.948 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-19T08:36:14.165 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-19T08:36:14.309 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-19T08:36:14.376 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-19T08:36:14.435 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up librados2 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:14.564 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up libsqlite3-mod-ceph (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:14.661 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-19T08:36:14.687 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-19T08:36:14.929 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-19T08:36:14.999 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up libcephfs2 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:15.116 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up libradosstriper1 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:15.188 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-19T08:36:15.242 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-19T08:36:15.519 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up librbd1 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:15.565 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-19T08:36:15.645 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up ceph-mgr-modules-core (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:15.771 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up ceph-fuse (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:15.850 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-19T08:36:15.985 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-19T08:36:15.985 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-19T08:36:16.161 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-19T08:36:16.288 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up librados2 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:16.413 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up libsqlite3-mod-ceph (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:16.431 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up libcephfs-dev (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:16.538 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-19T08:36:16.557 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-rados (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:16.683 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up librgw2 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:16.792 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up cephfs-top (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:16.866 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up libcephfs2 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:16.902 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-19T08:36:16.967 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up libradosstriper1 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:17.093 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-19T08:36:17.403 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up librbd1 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:17.529 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up ceph-mgr-modules-core (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:17.655 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up ceph-fuse (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:17.869 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-19T08:36:17.869 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-19T08:36:18.055 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-rbd (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:18.181 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up rbd-fuse (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:18.307 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-rgw (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:18.348 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up libcephfs-dev (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:18.442 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up python3-cephfs (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:18.474 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-rados (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:18.543 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up cephfs-shell (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:18.583 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up librgw2 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:18.660 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up ceph-common (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:18.692 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up cephfs-top (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:18.818 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-19T08:36:19.211 INFO:teuthology.orchestra.run.smithi098.stdout:Adding group ceph....done 2024-09-19T08:36:19.589 INFO:teuthology.orchestra.run.smithi098.stdout:Adding system user ceph....done 2024-09-19T08:36:19.715 INFO:teuthology.orchestra.run.smithi098.stdout:Setting system user ceph properties....done 2024-09-19T08:36:19.733 INFO:teuthology.orchestra.run.smithi098.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-19T08:36:19.873 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-19T08:36:19.972 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-rbd (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:20.098 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up rbd-fuse (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:20.180 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-19T08:36:20.224 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-rgw (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:20.333 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up python3-cephfs (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:20.442 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up cephfs-shell (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:20.568 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up ceph-common (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:20.637 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up cephfs-mirror (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:20.842 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-19T08:36:20.842 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-19T08:36:21.077 INFO:teuthology.orchestra.run.smithi139.stdout:Adding group ceph....done 2024-09-19T08:36:21.280 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up ceph-test (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:21.438 INFO:teuthology.orchestra.run.smithi139.stdout:Adding system user ceph....done 2024-09-19T08:36:21.459 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up radosgw (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:21.606 INFO:teuthology.orchestra.run.smithi139.stdout:Setting system user ceph properties....done 2024-09-19T08:36:21.623 INFO:teuthology.orchestra.run.smithi139.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-19T08:36:21.755 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-19T08:36:22.033 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-19T08:36:22.033 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-19T08:36:22.069 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-19T08:36:22.475 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up ceph-base (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:22.511 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up cephfs-mirror (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:22.718 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-19T08:36:22.718 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-19T08:36:22.938 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-19T08:36:23.187 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up ceph-test (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:23.338 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up radosgw (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:23.435 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up ceph-mds (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:23.727 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-19T08:36:23.727 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-19T08:36:23.804 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-19T08:36:23.804 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-19T08:36:24.171 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up ceph-mgr (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:24.248 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up ceph-base (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:24.389 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-19T08:36:24.389 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-19T08:36:24.603 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-19T08:36:24.815 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up ceph-osd (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:25.058 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up ceph-mds (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:25.083 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-19T08:36:25.083 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-19T08:36:25.275 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-19T08:36:25.275 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-19T08:36:25.549 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up ceph-mgr-k8sevents (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:25.667 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up ceph-mgr-diskprediction-local (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:25.726 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up ceph-mgr (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:25.793 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up ceph-mon (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:25.943 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-19T08:36:25.944 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-19T08:36:25.995 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-19T08:36:25.995 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-19T08:36:26.378 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up ceph-osd (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:26.403 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up ceph-mgr-cephadm (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:26.538 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up ceph (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:26.639 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up ceph-mgr-dashboard (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:26.645 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-19T08:36:26.646 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-19T08:36:26.774 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up ceph-volume (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:27.096 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up ceph-mgr-k8sevents (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:27.126 INFO:teuthology.orchestra.run.smithi098.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-19T08:36:27.222 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up ceph-mgr-diskprediction-local (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:27.236 INFO:teuthology.orchestra.run.smithi098.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-19T08:36:27.388 INFO:teuthology.orchestra.run.smithi098.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-19T08:36:27.400 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up ceph-mon (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:27.615 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-19T08:36:27.615 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-19T08:36:28.091 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up ceph-mgr-cephadm (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:28.163 INFO:teuthology.orchestra.run.smithi098.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-19T08:36:28.251 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up ceph (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:28.369 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up ceph-mgr-dashboard (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:28.520 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up ceph-volume (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T08:36:28.805 INFO:teuthology.orchestra.run.smithi098.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-19T08:36:28.808 DEBUG:teuthology.orchestra.run.smithi098:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-19T08:36:28.873 INFO:teuthology.orchestra.run.smithi139.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-19T08:36:28.912 INFO:teuthology.orchestra.run.smithi098.stdout:Reading package lists... 2024-09-19T08:36:28.973 INFO:teuthology.orchestra.run.smithi139.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-19T08:36:29.123 INFO:teuthology.orchestra.run.smithi098.stdout:Building dependency tree... 2024-09-19T08:36:29.124 INFO:teuthology.orchestra.run.smithi098.stdout:Reading state information... 2024-09-19T08:36:29.125 INFO:teuthology.orchestra.run.smithi139.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-19T08:36:29.316 INFO:teuthology.orchestra.run.smithi098.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-19T08:36:29.316 INFO:teuthology.orchestra.run.smithi098.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-19T08:36:29.316 INFO:teuthology.orchestra.run.smithi098.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-19T08:36:29.316 INFO:teuthology.orchestra.run.smithi098.stdout:The following packages were automatically installed and are no longer required: 2024-09-19T08:36:29.316 INFO:teuthology.orchestra.run.smithi098.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-19T08:36:29.316 INFO:teuthology.orchestra.run.smithi098.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-19T08:36:29.316 INFO:teuthology.orchestra.run.smithi098.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-19T08:36:29.317 INFO:teuthology.orchestra.run.smithi098.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-19T08:36:29.317 INFO:teuthology.orchestra.run.smithi098.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-19T08:36:29.317 INFO:teuthology.orchestra.run.smithi098.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-19T08:36:29.317 INFO:teuthology.orchestra.run.smithi098.stdout:The following additional packages will be installed: 2024-09-19T08:36:29.317 INFO:teuthology.orchestra.run.smithi098.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-19T08:36:29.318 INFO:teuthology.orchestra.run.smithi098.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-19T08:36:29.318 INFO:teuthology.orchestra.run.smithi098.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-19T08:36:29.318 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-19T08:36:29.318 INFO:teuthology.orchestra.run.smithi098.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-19T08:36:29.319 INFO:teuthology.orchestra.run.smithi098.stdout:Suggested packages: 2024-09-19T08:36:29.319 INFO:teuthology.orchestra.run.smithi098.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-19T08:36:29.319 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-19T08:36:29.320 INFO:teuthology.orchestra.run.smithi098.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-19T08:36:29.360 INFO:teuthology.orchestra.run.smithi098.stdout:The following NEW packages will be installed: 2024-09-19T08:36:29.361 INFO:teuthology.orchestra.run.smithi098.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-19T08:36:29.361 INFO:teuthology.orchestra.run.smithi098.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-19T08:36:29.361 INFO:teuthology.orchestra.run.smithi098.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-19T08:36:29.361 INFO:teuthology.orchestra.run.smithi098.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-19T08:36:29.361 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-19T08:36:29.362 INFO:teuthology.orchestra.run.smithi098.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-19T08:36:29.362 INFO:teuthology.orchestra.run.smithi098.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-19T08:36:29.363 INFO:teuthology.orchestra.run.smithi098.stdout:The following packages will be upgraded: 2024-09-19T08:36:29.363 INFO:teuthology.orchestra.run.smithi098.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-19T08:36:29.415 INFO:teuthology.orchestra.run.smithi098.stdout:4 upgraded, 39 newly installed, 0 to remove and 327 not upgraded. 2024-09-19T08:36:29.416 INFO:teuthology.orchestra.run.smithi098.stdout:Need to get 55.8 MB of archives. 2024-09-19T08:36:29.416 INFO:teuthology.orchestra.run.smithi098.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-19T08:36:29.416 INFO:teuthology.orchestra.run.smithi098.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-19T08:36:29.560 INFO:teuthology.orchestra.run.smithi098.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-19T08:36:29.568 INFO:teuthology.orchestra.run.smithi098.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-19T08:36:29.609 INFO:teuthology.orchestra.run.smithi098.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-19T08:36:29.609 INFO:teuthology.orchestra.run.smithi098.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-19T08:36:29.612 INFO:teuthology.orchestra.run.smithi098.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-19T08:36:29.621 INFO:teuthology.orchestra.run.smithi098.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-19T08:36:29.621 INFO:teuthology.orchestra.run.smithi098.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-19T08:36:29.623 INFO:teuthology.orchestra.run.smithi098.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-19T08:36:29.625 INFO:teuthology.orchestra.run.smithi098.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-19T08:36:29.625 INFO:teuthology.orchestra.run.smithi098.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-19T08:36:29.627 INFO:teuthology.orchestra.run.smithi098.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-19T08:36:29.628 INFO:teuthology.orchestra.run.smithi098.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-19T08:36:29.645 INFO:teuthology.orchestra.run.smithi098.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-19T08:36:29.645 INFO:teuthology.orchestra.run.smithi098.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-19T08:36:29.646 INFO:teuthology.orchestra.run.smithi098.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-19T08:36:29.962 INFO:teuthology.orchestra.run.smithi098.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-19T08:36:29.964 INFO:teuthology.orchestra.run.smithi098.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-19T08:36:29.964 INFO:teuthology.orchestra.run.smithi098.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-09-19T08:36:29.966 INFO:teuthology.orchestra.run.smithi098.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-19T08:36:29.967 INFO:teuthology.orchestra.run.smithi098.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-19T08:36:29.971 INFO:teuthology.orchestra.run.smithi098.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-19T08:36:29.972 INFO:teuthology.orchestra.run.smithi098.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-19T08:36:29.979 INFO:teuthology.orchestra.run.smithi098.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-19T08:36:29.979 INFO:teuthology.orchestra.run.smithi098.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-19T08:36:29.999 INFO:teuthology.orchestra.run.smithi098.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-19T08:36:29.999 INFO:teuthology.orchestra.run.smithi098.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-09-19T08:36:30.009 INFO:teuthology.orchestra.run.smithi098.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-19T08:36:30.032 INFO:teuthology.orchestra.run.smithi098.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-19T08:36:30.034 INFO:teuthology.orchestra.run.smithi098.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-19T08:36:30.035 INFO:teuthology.orchestra.run.smithi098.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-19T08:36:30.035 INFO:teuthology.orchestra.run.smithi098.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-19T08:36:30.035 INFO:teuthology.orchestra.run.smithi098.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-19T08:36:30.045 INFO:teuthology.orchestra.run.smithi098.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-19T08:36:30.047 INFO:teuthology.orchestra.run.smithi098.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-19T08:36:30.242 INFO:teuthology.orchestra.run.smithi139.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-19T08:36:30.265 INFO:teuthology.orchestra.run.smithi098.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-19T08:36:30.265 INFO:teuthology.orchestra.run.smithi098.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-19T08:36:30.265 INFO:teuthology.orchestra.run.smithi098.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-19T08:36:30.265 INFO:teuthology.orchestra.run.smithi098.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-19T08:36:30.265 INFO:teuthology.orchestra.run.smithi098.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-19T08:36:30.265 INFO:teuthology.orchestra.run.smithi098.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-19T08:36:30.277 INFO:teuthology.orchestra.run.smithi098.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-19T08:36:30.277 INFO:teuthology.orchestra.run.smithi098.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-19T08:36:30.683 INFO:teuthology.orchestra.run.smithi098.stdout:Preconfiguring packages ... 2024-09-19T08:36:30.840 INFO:teuthology.orchestra.run.smithi098.stdout:Fetched 55.8 MB in 1s (61.5 MB/s) 2024-09-19T08:36:30.905 INFO:teuthology.orchestra.run.smithi139.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-19T08:36:30.908 DEBUG:teuthology.orchestra.run.smithi139:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-19T08:36:31.012 INFO:teuthology.orchestra.run.smithi139.stdout:Reading package lists... 2024-09-19T08:36:31.151 INFO:teuthology.orchestra.run.smithi098.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173987 files and directories currently installed.) 2024-09-19T08:36:31.157 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-19T08:36:31.223 INFO:teuthology.orchestra.run.smithi139.stdout:Building dependency tree... 2024-09-19T08:36:31.224 INFO:teuthology.orchestra.run.smithi139.stdout:Reading state information... 2024-09-19T08:36:31.302 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-19T08:36:31.428 INFO:teuthology.orchestra.run.smithi139.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-19T08:36:31.428 INFO:teuthology.orchestra.run.smithi139.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-19T08:36:31.428 INFO:teuthology.orchestra.run.smithi139.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-19T08:36:31.429 INFO:teuthology.orchestra.run.smithi139.stdout:The following packages were automatically installed and are no longer required: 2024-09-19T08:36:31.429 INFO:teuthology.orchestra.run.smithi139.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-19T08:36:31.429 INFO:teuthology.orchestra.run.smithi139.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-19T08:36:31.429 INFO:teuthology.orchestra.run.smithi139.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-19T08:36:31.429 INFO:teuthology.orchestra.run.smithi139.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-19T08:36:31.430 INFO:teuthology.orchestra.run.smithi139.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-19T08:36:31.430 INFO:teuthology.orchestra.run.smithi139.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-19T08:36:31.430 INFO:teuthology.orchestra.run.smithi139.stdout:The following additional packages will be installed: 2024-09-19T08:36:31.431 INFO:teuthology.orchestra.run.smithi139.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-19T08:36:31.431 INFO:teuthology.orchestra.run.smithi139.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-19T08:36:31.431 INFO:teuthology.orchestra.run.smithi139.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-19T08:36:31.431 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-19T08:36:31.431 INFO:teuthology.orchestra.run.smithi139.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-19T08:36:31.432 INFO:teuthology.orchestra.run.smithi139.stdout:Suggested packages: 2024-09-19T08:36:31.432 INFO:teuthology.orchestra.run.smithi139.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-19T08:36:31.432 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-19T08:36:31.432 INFO:teuthology.orchestra.run.smithi139.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-19T08:36:31.473 INFO:teuthology.orchestra.run.smithi139.stdout:The following NEW packages will be installed: 2024-09-19T08:36:31.473 INFO:teuthology.orchestra.run.smithi139.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-19T08:36:31.474 INFO:teuthology.orchestra.run.smithi139.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-19T08:36:31.474 INFO:teuthology.orchestra.run.smithi139.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-19T08:36:31.474 INFO:teuthology.orchestra.run.smithi139.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-19T08:36:31.474 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-19T08:36:31.474 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-19T08:36:31.474 INFO:teuthology.orchestra.run.smithi139.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-19T08:36:31.475 INFO:teuthology.orchestra.run.smithi139.stdout:The following packages will be upgraded: 2024-09-19T08:36:31.476 INFO:teuthology.orchestra.run.smithi139.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-19T08:36:31.528 INFO:teuthology.orchestra.run.smithi139.stdout:4 upgraded, 39 newly installed, 0 to remove and 327 not upgraded. 2024-09-19T08:36:31.528 INFO:teuthology.orchestra.run.smithi139.stdout:Need to get 55.8 MB of archives. 2024-09-19T08:36:31.528 INFO:teuthology.orchestra.run.smithi139.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-19T08:36:31.528 INFO:teuthology.orchestra.run.smithi139.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-19T08:36:31.662 INFO:teuthology.orchestra.run.smithi139.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-19T08:36:31.670 INFO:teuthology.orchestra.run.smithi139.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-19T08:36:31.711 INFO:teuthology.orchestra.run.smithi139.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-19T08:36:31.711 INFO:teuthology.orchestra.run.smithi139.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-19T08:36:31.712 INFO:teuthology.orchestra.run.smithi139.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-19T08:36:31.723 INFO:teuthology.orchestra.run.smithi139.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-19T08:36:31.723 INFO:teuthology.orchestra.run.smithi139.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-19T08:36:31.725 INFO:teuthology.orchestra.run.smithi139.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-19T08:36:31.726 INFO:teuthology.orchestra.run.smithi139.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-19T08:36:31.727 INFO:teuthology.orchestra.run.smithi139.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-19T08:36:31.728 INFO:teuthology.orchestra.run.smithi139.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-19T08:36:31.729 INFO:teuthology.orchestra.run.smithi139.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-19T08:36:31.746 INFO:teuthology.orchestra.run.smithi139.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-19T08:36:31.747 INFO:teuthology.orchestra.run.smithi139.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-19T08:36:31.748 INFO:teuthology.orchestra.run.smithi139.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-19T08:36:31.993 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-19T08:36:32.038 INFO:teuthology.orchestra.run.smithi139.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-19T08:36:32.094 INFO:teuthology.orchestra.run.smithi139.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-19T08:36:32.094 INFO:teuthology.orchestra.run.smithi139.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-09-19T08:36:32.097 INFO:teuthology.orchestra.run.smithi139.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-19T08:36:32.097 INFO:teuthology.orchestra.run.smithi139.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-19T08:36:32.101 INFO:teuthology.orchestra.run.smithi139.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-19T08:36:32.103 INFO:teuthology.orchestra.run.smithi139.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-19T08:36:32.110 INFO:teuthology.orchestra.run.smithi139.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-19T08:36:32.110 INFO:teuthology.orchestra.run.smithi139.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-19T08:36:32.131 INFO:teuthology.orchestra.run.smithi139.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-19T08:36:32.131 INFO:teuthology.orchestra.run.smithi139.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-09-19T08:36:32.141 INFO:teuthology.orchestra.run.smithi139.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-19T08:36:32.161 INFO:teuthology.orchestra.run.smithi139.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-19T08:36:32.164 INFO:teuthology.orchestra.run.smithi139.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-19T08:36:32.164 INFO:teuthology.orchestra.run.smithi139.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-19T08:36:32.165 INFO:teuthology.orchestra.run.smithi139.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-19T08:36:32.165 INFO:teuthology.orchestra.run.smithi139.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-19T08:36:32.175 INFO:teuthology.orchestra.run.smithi139.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-19T08:36:32.177 INFO:teuthology.orchestra.run.smithi139.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-19T08:36:32.393 INFO:teuthology.orchestra.run.smithi139.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-19T08:36:32.394 INFO:teuthology.orchestra.run.smithi139.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-19T08:36:32.394 INFO:teuthology.orchestra.run.smithi139.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-19T08:36:32.394 INFO:teuthology.orchestra.run.smithi139.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-19T08:36:32.394 INFO:teuthology.orchestra.run.smithi139.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-19T08:36:32.394 INFO:teuthology.orchestra.run.smithi139.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-19T08:36:32.405 INFO:teuthology.orchestra.run.smithi139.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-19T08:36:32.406 INFO:teuthology.orchestra.run.smithi139.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-19T08:36:32.821 INFO:teuthology.orchestra.run.smithi139.stdout:Preconfiguring packages ... 2024-09-19T08:36:32.974 INFO:teuthology.orchestra.run.smithi098.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-19T08:36:32.987 INFO:teuthology.orchestra.run.smithi139.stdout:Fetched 55.8 MB in 1s (60.5 MB/s) 2024-09-19T08:36:33.242 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package flex. 2024-09-19T08:36:33.305 INFO:teuthology.orchestra.run.smithi139.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173987 files and directories currently installed.) 2024-09-19T08:36:33.310 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-19T08:36:33.345 INFO:teuthology.orchestra.run.smithi098.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173987 files and directories currently installed.) 2024-09-19T08:36:33.350 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-19T08:36:33.390 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-19T08:36:33.440 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-19T08:36:33.907 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-19T08:36:34.080 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-19T08:36:34.088 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-19T08:36:34.490 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-19T08:36:34.993 INFO:teuthology.orchestra.run.smithi139.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-19T08:36:34.998 INFO:teuthology.orchestra.run.smithi098.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174060 files and directories currently installed.) 2024-09-19T08:36:35.004 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-19T08:36:35.121 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-19T08:36:35.239 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package flex. 2024-09-19T08:36:35.341 INFO:teuthology.orchestra.run.smithi139.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173987 files and directories currently installed.) 2024-09-19T08:36:35.346 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-19T08:36:35.395 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-19T08:36:35.530 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-19T08:36:35.644 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-19T08:36:35.861 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-19T08:36:35.971 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-19T08:36:36.017 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-19T08:36:36.420 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-19T08:36:36.497 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package bison. 2024-09-19T08:36:36.603 INFO:teuthology.orchestra.run.smithi098.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174060 files and directories currently installed.) 2024-09-19T08:36:36.608 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-19T08:36:36.657 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-19T08:36:36.986 INFO:teuthology.orchestra.run.smithi139.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174060 files and directories currently installed.) 2024-09-19T08:36:36.992 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-19T08:36:37.025 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-19T08:36:37.041 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-19T08:36:37.074 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-19T08:36:37.100 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-19T08:36:37.420 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package dump. 2024-09-19T08:36:37.447 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-19T08:36:37.486 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking dump (0.4b47-3) ... 2024-09-19T08:36:37.526 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-19T08:36:37.765 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package indent. 2024-09-19T08:36:37.774 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-19T08:36:37.792 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-19T08:36:37.832 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-19T08:36:38.084 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-19T08:36:38.101 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-19T08:36:38.112 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-19T08:36:38.142 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-19T08:36:38.472 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-19T08:36:38.499 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-19T08:36:38.500 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package bison. 2024-09-19T08:36:38.538 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-19T08:36:38.605 INFO:teuthology.orchestra.run.smithi139.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174060 files and directories currently installed.) 2024-09-19T08:36:38.613 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-19T08:36:38.661 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-19T08:36:38.799 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-19T08:36:38.827 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-19T08:36:38.866 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-19T08:36:39.045 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-19T08:36:39.072 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-19T08:36:39.111 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-19T08:36:39.169 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-19T08:36:39.197 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-19T08:36:39.229 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-19T08:36:39.448 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package dump. 2024-09-19T08:36:39.464 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-19T08:36:39.476 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-19T08:36:39.481 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-19T08:36:39.514 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking dump (0.4b47-3) ... 2024-09-19T08:36:39.522 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-19T08:36:39.775 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package libjson-perl. 2024-09-19T08:36:39.790 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-19T08:36:39.801 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package indent. 2024-09-19T08:36:39.817 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-19T08:36:39.833 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-19T08:36:39.859 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-19T08:36:40.120 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-19T08:36:40.136 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-19T08:36:40.179 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-19T08:36:40.187 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-19T08:36:40.214 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-19T08:36:40.356 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-19T08:36:40.508 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-19T08:36:40.536 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-19T08:36:40.575 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-19T08:36:40.785 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-19T08:36:40.799 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-19T08:36:40.843 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-19T08:36:41.173 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-19T08:36:41.200 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-19T08:36:41.255 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-19T08:36:41.491 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-19T08:36:41.508 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-19T08:36:41.549 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-19T08:36:41.802 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package libjson-perl. 2024-09-19T08:36:41.818 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-19T08:36:41.860 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-19T08:36:42.163 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-19T08:36:42.191 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-19T08:36:42.230 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-19T08:36:42.456 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-19T08:36:42.484 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-19T08:36:42.523 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-19T08:36:42.784 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-19T08:36:42.811 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-19T08:36:42.850 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-19T08:36:43.170 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-19T08:36:43.198 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-19T08:36:43.237 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-19T08:36:43.532 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-19T08:36:43.559 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-19T08:36:43.598 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-19T08:36:43.918 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-19T08:36:43.945 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-19T08:36:43.984 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-19T08:36:44.058 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-19T08:36:44.086 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-19T08:36:44.124 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-19T08:36:44.329 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-19T08:36:44.357 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-19T08:36:44.385 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-19T08:36:44.395 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-19T08:36:44.401 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-19T08:36:44.443 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-19T08:36:44.690 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-19T08:36:44.718 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-19T08:36:44.757 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-19T08:36:44.772 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-19T08:36:44.799 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-19T08:36:44.838 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-19T08:36:45.060 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-19T08:36:45.076 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-19T08:36:45.118 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-19T08:36:45.183 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-19T08:36:45.210 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-19T08:36:45.249 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-19T08:36:45.365 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-19T08:36:45.390 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-19T08:36:45.429 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-19T08:36:45.594 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-19T08:36:45.622 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-19T08:36:45.660 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-19T08:36:45.972 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-19T08:36:45.982 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-19T08:36:46.000 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-19T08:36:46.010 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-19T08:36:46.038 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-19T08:36:46.049 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-19T08:36:46.375 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-19T08:36:46.403 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-19T08:36:46.404 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package modemmanager. 2024-09-19T08:36:46.432 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-19T08:36:46.441 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-19T08:36:46.471 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-19T08:36:46.736 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-19T08:36:46.764 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-19T08:36:46.802 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-19T08:36:47.038 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-19T08:36:47.054 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-19T08:36:47.077 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package network-manager. 2024-09-19T08:36:47.088 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-19T08:36:47.105 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-19T08:36:47.152 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-19T08:36:47.600 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-19T08:36:47.628 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-19T08:36:47.666 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-19T08:36:47.854 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package ppp. 2024-09-19T08:36:47.881 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-19T08:36:47.961 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package modemmanager. 2024-09-19T08:36:47.989 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-19T08:36:48.027 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-19T08:36:48.131 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-19T08:36:48.408 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package pptp-linux. 2024-09-19T08:36:48.436 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-19T08:36:48.467 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-19T08:36:48.694 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-19T08:36:48.710 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-19T08:36:48.711 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package network-manager. 2024-09-19T08:36:48.738 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-19T08:36:48.752 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-19T08:36:48.785 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-19T08:36:49.089 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-19T08:36:49.117 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-19T08:36:49.156 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking postgresql-client-common (238) ... 2024-09-19T08:36:49.393 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package ppp. 2024-09-19T08:36:49.420 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-19T08:36:49.460 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-19T08:36:49.487 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-19T08:36:49.526 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-19T08:36:49.684 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-19T08:36:50.023 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package pptp-linux. 2024-09-19T08:36:50.051 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-19T08:36:50.089 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-19T08:36:50.139 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package postgresql-common. 2024-09-19T08:36:50.167 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-19T08:36:50.224 INFO:teuthology.orchestra.run.smithi098.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-19T08:36:50.254 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking postgresql-common (238) ... 2024-09-19T08:36:50.342 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-19T08:36:50.358 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-19T08:36:50.400 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-19T08:36:50.673 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package postgresql-14. 2024-09-19T08:36:50.686 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-19T08:36:50.699 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-19T08:36:50.714 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-19T08:36:50.733 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-19T08:36:50.753 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking postgresql-client-common (238) ... 2024-09-19T08:36:51.048 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-19T08:36:51.076 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-19T08:36:51.115 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-19T08:36:51.661 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package postgresql-common. 2024-09-19T08:36:51.690 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-19T08:36:51.745 INFO:teuthology.orchestra.run.smithi139.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-19T08:36:51.775 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking postgresql-common (238) ... 2024-09-19T08:36:51.966 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package postgresql. 2024-09-19T08:36:51.984 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-19T08:36:52.024 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking postgresql (14+238) ... 2024-09-19T08:36:52.235 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package postgresql-14. 2024-09-19T08:36:52.252 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package postgresql-client. 2024-09-19T08:36:52.263 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-19T08:36:52.268 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-19T08:36:52.304 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-19T08:36:52.315 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking postgresql-client (14+238) ... 2024-09-19T08:36:52.906 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-19T08:36:52.924 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-19T08:36:52.956 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-19T08:36:53.283 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package tcl8.6. 2024-09-19T08:36:53.311 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-19T08:36:53.350 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-19T08:36:53.636 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package tcl. 2024-09-19T08:36:53.664 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-19T08:36:53.689 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package postgresql. 2024-09-19T08:36:53.708 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-19T08:36:53.718 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-19T08:36:53.764 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking postgresql (14+238) ... 2024-09-19T08:36:54.066 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package postgresql-client. 2024-09-19T08:36:54.072 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package wpasupplicant. 2024-09-19T08:36:54.095 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-19T08:36:54.100 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-19T08:36:54.139 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-19T08:36:54.191 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking postgresql-client (14+238) ... 2024-09-19T08:36:54.460 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-19T08:36:54.460 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-19T08:36:54.476 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-19T08:36:54.476 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-19T08:36:54.518 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-19T08:36:54.518 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-19T08:36:54.787 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package tcl8.6. 2024-09-19T08:36:54.815 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-19T08:36:54.854 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-19T08:36:54.905 INFO:teuthology.orchestra.run.smithi098.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-19T08:36:54.932 INFO:teuthology.orchestra.run.smithi098.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-19T08:36:54.980 INFO:teuthology.orchestra.run.smithi098.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-19T08:36:55.132 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package tcl. 2024-09-19T08:36:55.160 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-19T08:36:55.203 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-19T08:36:55.258 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up postgresql-client-common (238) ... 2024-09-19T08:36:55.469 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-19T08:36:55.551 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package wpasupplicant. 2024-09-19T08:36:55.581 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-19T08:36:55.595 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-19T08:36:55.618 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-19T08:36:55.721 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-19T08:36:55.823 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-19T08:36:55.923 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-19T08:36:55.964 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-19T08:36:55.980 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-19T08:36:56.030 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-19T08:36:56.041 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-19T08:36:56.383 INFO:teuthology.orchestra.run.smithi139.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-19T08:36:56.411 INFO:teuthology.orchestra.run.smithi139.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-19T08:36:56.458 INFO:teuthology.orchestra.run.smithi139.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-19T08:36:56.544 INFO:teuthology.orchestra.run.smithi098.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-19T08:36:56.630 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-19T08:36:56.746 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up postgresql-client-common (238) ... 2024-09-19T08:36:56.956 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-19T08:36:57.023 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-19T08:36:57.023 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-19T08:36:57.082 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-19T08:36:57.199 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-19T08:36:57.309 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-19T08:36:57.435 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-19T08:36:57.499 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-19T08:36:57.561 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-19T08:36:57.626 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-19T08:36:57.743 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-19T08:36:57.844 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-19T08:36:57.945 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-19T08:36:58.065 INFO:teuthology.orchestra.run.smithi139.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-19T08:36:58.088 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-19T08:36:58.149 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-19T08:36:58.214 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-19T08:36:58.340 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-19T08:36:58.527 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-19T08:36:58.527 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-19T08:36:58.937 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-19T08:36:59.002 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-19T08:36:59.055 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up dump (0.4b47-3) ... 2024-09-19T08:36:59.128 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-19T08:36:59.157 INFO:teuthology.orchestra.run.smithi098.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-19T08:36:59.223 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-19T08:36:59.254 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-19T08:36:59.349 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-19T08:36:59.389 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-19T08:36:59.479 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-19T08:36:59.515 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-19T08:36:59.601 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-19T08:36:59.632 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-19T08:36:59.719 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-19T08:36:59.741 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-19T08:36:59.793 INFO:teuthology.orchestra.run.smithi098.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-19T08:36:59.859 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-19T08:36:59.862 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-19T08:36:59.971 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-19T08:37:00.140 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-19T08:37:00.266 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-19T08:37:00.392 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-19T08:37:00.506 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-19T08:37:00.560 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up postgresql-client (14+238) ... 2024-09-19T08:37:00.624 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up dump (0.4b47-3) ... 2024-09-19T08:37:00.686 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-19T08:37:00.717 INFO:teuthology.orchestra.run.smithi139.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-19T08:37:00.767 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-19T08:37:00.885 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-19T08:37:01.011 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-19T08:37:01.117 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:37:01.117 INFO:teuthology.orchestra.run.smithi098.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-19T08:37:01.117 INFO:teuthology.orchestra.run.smithi098.stdout:which means they are currently configured by ifupdown: 2024-09-19T08:37:01.117 INFO:teuthology.orchestra.run.smithi098.stdout:- enp3s0f1 2024-09-19T08:37:01.117 INFO:teuthology.orchestra.run.smithi098.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-19T08:37:01.117 INFO:teuthology.orchestra.run.smithi098.stdout:remove their configuration from /etc/network/interfaces. 2024-09-19T08:37:01.118 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:37:01.137 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-19T08:37:01.241 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-19T08:37:01.262 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-19T08:37:01.353 INFO:teuthology.orchestra.run.smithi139.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-19T08:37:01.414 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-19T08:37:01.540 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-19T08:37:01.557 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-19T08:37:01.716 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-19T08:37:01.842 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-19T08:37:01.871 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-19T08:37:01.968 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-19T08:37:02.136 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up postgresql-client (14+238) ... 2024-09-19T08:37:02.246 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-19T08:37:02.338 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-19T08:37:02.473 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-19T08:37:02.574 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-19T08:37:02.673 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-19T08:37:02.673 INFO:teuthology.orchestra.run.smithi139.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-19T08:37:02.673 INFO:teuthology.orchestra.run.smithi139.stdout:which means they are currently configured by ifupdown: 2024-09-19T08:37:02.673 INFO:teuthology.orchestra.run.smithi139.stdout:- enp3s0f1 2024-09-19T08:37:02.673 INFO:teuthology.orchestra.run.smithi139.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-19T08:37:02.673 INFO:teuthology.orchestra.run.smithi139.stdout:remove their configuration from /etc/network/interfaces. 2024-09-19T08:37:02.674 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-19T08:37:02.675 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up postgresql-common (238) ... 2024-09-19T08:37:02.797 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-19T08:37:03.105 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-19T08:37:03.446 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-19T08:37:03.950 INFO:teuthology.orchestra.run.smithi098.stdout:Adding user postgres to group ssl-cert 2024-09-19T08:37:04.048 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-19T08:37:04.177 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:37:04.178 INFO:teuthology.orchestra.run.smithi098.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-19T08:37:04.183 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-19T08:37:04.355 INFO:teuthology.orchestra.run.smithi098.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-19T08:37:04.355 INFO:teuthology.orchestra.run.smithi098.stdout:Removing obsolete dictionary files: 2024-09-19T08:37:04.359 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-19T08:37:04.477 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up postgresql-common (238) ... 2024-09-19T08:37:04.880 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-19T08:37:05.261 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-19T08:37:05.336 INFO:teuthology.orchestra.run.smithi139.stdout:Adding user postgres to group ssl-cert 2024-09-19T08:37:05.553 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-19T08:37:05.553 INFO:teuthology.orchestra.run.smithi139.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-19T08:37:05.569 INFO:teuthology.orchestra.run.smithi098.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-19T08:37:05.581 INFO:teuthology.orchestra.run.smithi098.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-19T08:37:05.619 INFO:teuthology.orchestra.run.smithi098.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-19T08:37:05.619 INFO:teuthology.orchestra.run.smithi098.stdout:This user must also own the server process. 2024-09-19T08:37:05.619 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:37:05.619 INFO:teuthology.orchestra.run.smithi098.stdout:The database cluster will be initialized with locale "en_US". 2024-09-19T08:37:05.619 INFO:teuthology.orchestra.run.smithi098.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-19T08:37:05.619 INFO:teuthology.orchestra.run.smithi098.stdout:The default text search configuration will be set to "english". 2024-09-19T08:37:05.619 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:37:05.620 INFO:teuthology.orchestra.run.smithi098.stdout:Data page checksums are disabled. 2024-09-19T08:37:05.620 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:37:05.620 INFO:teuthology.orchestra.run.smithi098.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-19T08:37:05.620 INFO:teuthology.orchestra.run.smithi098.stdout:creating subdirectories ... ok 2024-09-19T08:37:05.620 INFO:teuthology.orchestra.run.smithi098.stdout:selecting dynamic shared memory implementation ... posix 2024-09-19T08:37:05.634 INFO:teuthology.orchestra.run.smithi098.stdout:selecting default max_connections ... 100 2024-09-19T08:37:05.655 INFO:teuthology.orchestra.run.smithi098.stdout:selecting default shared_buffers ... 128MB 2024-09-19T08:37:05.666 INFO:teuthology.orchestra.run.smithi098.stdout:selecting default time zone ... Etc/UTC 2024-09-19T08:37:05.667 INFO:teuthology.orchestra.run.smithi098.stdout:creating configuration files ... ok 2024-09-19T08:37:05.782 INFO:teuthology.orchestra.run.smithi098.stdout:running bootstrap script ... ok 2024-09-19T08:37:05.806 INFO:teuthology.orchestra.run.smithi139.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-19T08:37:05.806 INFO:teuthology.orchestra.run.smithi139.stdout:Removing obsolete dictionary files: 2024-09-19T08:37:06.356 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-19T08:37:06.438 INFO:teuthology.orchestra.run.smithi098.stdout:performing post-bootstrap initialization ... ok 2024-09-19T08:37:06.728 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-19T08:37:07.039 INFO:teuthology.orchestra.run.smithi139.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-19T08:37:07.060 INFO:teuthology.orchestra.run.smithi139.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-19T08:37:07.089 INFO:teuthology.orchestra.run.smithi098.stdout:syncing data to disk ... ok 2024-09-19T08:37:07.115 INFO:teuthology.orchestra.run.smithi139.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-19T08:37:07.115 INFO:teuthology.orchestra.run.smithi139.stdout:This user must also own the server process. 2024-09-19T08:37:07.115 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-19T08:37:07.115 INFO:teuthology.orchestra.run.smithi139.stdout:The database cluster will be initialized with locale "en_US". 2024-09-19T08:37:07.115 INFO:teuthology.orchestra.run.smithi139.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-19T08:37:07.115 INFO:teuthology.orchestra.run.smithi139.stdout:The default text search configuration will be set to "english". 2024-09-19T08:37:07.116 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-19T08:37:07.116 INFO:teuthology.orchestra.run.smithi139.stdout:Data page checksums are disabled. 2024-09-19T08:37:07.116 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-19T08:37:07.116 INFO:teuthology.orchestra.run.smithi139.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-19T08:37:07.116 INFO:teuthology.orchestra.run.smithi139.stdout:creating subdirectories ... ok 2024-09-19T08:37:07.116 INFO:teuthology.orchestra.run.smithi139.stdout:selecting dynamic shared memory implementation ... posix 2024-09-19T08:37:07.131 INFO:teuthology.orchestra.run.smithi139.stdout:selecting default max_connections ... 100 2024-09-19T08:37:07.151 INFO:teuthology.orchestra.run.smithi139.stdout:selecting default shared_buffers ... 128MB 2024-09-19T08:37:07.161 INFO:teuthology.orchestra.run.smithi139.stdout:selecting default time zone ... Etc/UTC 2024-09-19T08:37:07.162 INFO:teuthology.orchestra.run.smithi139.stdout:creating configuration files ... ok 2024-09-19T08:37:07.275 INFO:teuthology.orchestra.run.smithi139.stdout:running bootstrap script ... ok 2024-09-19T08:37:07.474 INFO:teuthology.orchestra.run.smithi098.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-19T08:37:07.908 INFO:teuthology.orchestra.run.smithi139.stdout:performing post-bootstrap initialization ... ok 2024-09-19T08:37:08.548 INFO:teuthology.orchestra.run.smithi139.stdout:syncing data to disk ... ok 2024-09-19T08:37:08.934 INFO:teuthology.orchestra.run.smithi139.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-19T08:37:10.217 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-19T08:37:10.343 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-19T08:37:10.469 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up postgresql (14+238) ... 2024-09-19T08:37:10.612 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-19T08:37:10.729 INFO:teuthology.orchestra.run.smithi098.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-19T08:37:10.958 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-19T08:37:10.959 INFO:teuthology.orchestra.run.smithi098.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-19T08:37:11.474 INFO:teuthology.orchestra.run.smithi098.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-19T08:37:11.549 INFO:teuthology.orchestra.run.smithi098.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-19T08:37:11.709 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-19T08:37:11.810 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-19T08:37:11.860 INFO:teuthology.orchestra.run.smithi098.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-19T08:37:11.936 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up postgresql (14+238) ... 2024-09-19T08:37:12.012 INFO:teuthology.orchestra.run.smithi098.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-19T08:37:12.062 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-19T08:37:12.188 INFO:teuthology.orchestra.run.smithi139.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-19T08:37:12.445 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-19T08:37:12.445 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-19T08:37:12.974 INFO:teuthology.orchestra.run.smithi139.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-19T08:37:13.067 INFO:teuthology.orchestra.run.smithi139.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-19T08:37:13.360 INFO:teuthology.orchestra.run.smithi139.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-19T08:37:13.449 INFO:teuthology.orchestra.run.smithi098.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-19T08:37:13.512 INFO:teuthology.orchestra.run.smithi139.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-19T08:37:13.549 INFO:teuthology.orchestra.run.smithi098.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-19T08:37:13.982 INFO:teuthology.orchestra.run.smithi098.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-19T08:37:13.988 DEBUG:teuthology.parallel:result is None 2024-09-19T08:37:15.017 INFO:teuthology.orchestra.run.smithi139.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-19T08:37:15.117 INFO:teuthology.orchestra.run.smithi139.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-19T08:37:15.618 INFO:teuthology.orchestra.run.smithi139.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-19T08:37:15.622 DEBUG:teuthology.parallel:result is None 2024-09-19T08:37:15.623 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:37:15.761 DEBUG:teuthology.orchestra.run.smithi098:> dpkg-query -W -f '${Version}' ceph 2024-09-19T08:37:15.781 INFO:teuthology.orchestra.run.smithi098.stdout:19.1.1-361-g2f599a46-1jammy 2024-09-19T08:37:15.782 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361-g2f599a46-1jammy 2024-09-19T08:37:15.782 INFO:teuthology.task.install:The correct ceph version 19.1.1-361-g2f599a46-1jammy is installed. 2024-09-19T08:37:15.784 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T08:37:15.924 DEBUG:teuthology.orchestra.run.smithi139:> dpkg-query -W -f '${Version}' ceph 2024-09-19T08:37:15.943 INFO:teuthology.orchestra.run.smithi139.stdout:19.1.1-361-g2f599a46-1jammy 2024-09-19T08:37:15.943 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361-g2f599a46-1jammy 2024-09-19T08:37:15.943 INFO:teuthology.task.install:The correct ceph version 19.1.1-361-g2f599a46-1jammy is installed. 2024-09-19T08:37:15.945 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-19T08:37:15.945 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:37:15.945 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T08:37:15.959 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:37:15.959 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T08:37:15.999 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-19T08:37:16.000 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:37:16.000 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T08:37:16.015 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T08:37:16.070 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:37:16.071 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T08:37:16.084 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T08:37:16.138 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-19T08:37:16.139 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:37:16.139 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T08:37:16.151 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T08:37:16.207 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:37:16.207 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T08:37:16.220 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T08:37:16.274 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-19T08:37:16.274 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:37:16.274 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T08:37:16.287 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T08:37:16.342 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:37:16.342 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T08:37:16.355 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T08:37:16.410 INFO:teuthology.run_tasks:Running task ceph... 2024-09-19T08:37:16.496 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-19T08:37:16.496 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 777 /var/log/ceph 2024-09-19T08:37:16.499 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 777 /var/log/ceph 2024-09-19T08:37:16.512 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-19T08:37:16.512 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-19T08:37:16.553 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-19T08:37:16.565 INFO:tasks.ceph:Creating extra log directories... 2024-09-19T08:37:16.566 DEBUG:teuthology.orchestra.run.smithi098:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-19T08:37:16.609 DEBUG:teuthology.orchestra.run.smithi139:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-19T08:37:16.627 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-19T08:37:16.627 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-19T08:37:16.627 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-19T08:37:16.627 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:37:16.627 DEBUG:teuthology.orchestra.run.smithi098:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-19T08:37:16.668 DEBUG:teuthology.orchestra.run.smithi098:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-19T08:37:16.743 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:37:16.743 DEBUG:teuthology.orchestra.run.smithi139:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-19T08:37:16.749 DEBUG:teuthology.orchestra.run.smithi139:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-19T08:37:16.826 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-19T08:37:16.826 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-19T08:37:16.826 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911853', 'branch': 'squid', 'description': 'fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7911853', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': '6fad5dd73c93545e52f57f681e140a9c3529c11e'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1747, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'sleep_before_teardown': 0, 'subset': '176/512', 'suite': 'fs', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '6fad5dd73c93545e52f57f681e140a9c3529c11e', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi098.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI0s7Pcn1Or8WA8LA8yVHACWdXDnufadAAvOyqzmmzcAXIN5k0qRndsH5KzqnNT+h8/niDbqVLVOD3deu3EQXOM=', 'smithi139.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNmn9o8aIuWzejSIIKW4rGU49scBUrXd9kUOp0BdeAut7acetTbX5heNkmsXfCAkBsmuw3wcFw/c9hxU3+ddSo4='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestMisc']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-18_21:24:17', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-19T08:37:16.827 DEBUG:teuthology.orchestra.run.smithi098:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-19T08:37:16.829 DEBUG:teuthology.orchestra.run.smithi139:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-19T08:37:16.869 DEBUG:teuthology.orchestra.run.smithi098:> sudo install -d -m0777 -- /var/run/ceph 2024-09-19T08:37:16.876 DEBUG:teuthology.orchestra.run.smithi139:> sudo install -d -m0777 -- /var/run/ceph 2024-09-19T08:37:16.921 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:37:16.921 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T08:37:16.936 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T08:37:16.936 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_1 2024-09-19T08:37:16.984 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T08:37:16.984 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:37:16.984 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 5h/5d Inode: 870 Links: 1 2024-09-19T08:37:16.984 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:37:16.985 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-09-19 08:31:03.755181226 +0000 2024-09-19T08:37:16.985 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-09-19 08:31:03.503183477 +0000 2024-09-19T08:37:16.985 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-09-19 08:31:03.503183477 +0000 2024-09-19T08:37:16.985 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: - 2024-09-19T08:37:16.985 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T08:37:17.039 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-09-19T08:37:17.039 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-09-19T08:37:17.039 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000391208 s, 1.3 MB/s 2024-09-19T08:37:17.040 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T08:37:17.089 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_2 2024-09-19T08:37:17.135 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T08:37:17.136 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:37:17.136 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 5h/5d Inode: 885 Links: 1 2024-09-19T08:37:17.136 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:37:17.136 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-09-19 08:31:04.231176973 +0000 2024-09-19T08:37:17.136 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-09-19 08:31:03.987179153 +0000 2024-09-19T08:37:17.136 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-09-19 08:31:03.987179153 +0000 2024-09-19T08:37:17.136 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: - 2024-09-19T08:37:17.136 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T08:37:17.194 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-09-19T08:37:17.194 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-09-19T08:37:17.194 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.00032143 s, 1.6 MB/s 2024-09-19T08:37:17.195 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T08:37:17.245 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_3 2024-09-19T08:37:17.292 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T08:37:17.292 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:37:17.292 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 5h/5d Inode: 909 Links: 1 2024-09-19T08:37:17.292 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:37:17.292 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-09-19 08:31:04.707172722 +0000 2024-09-19T08:37:17.293 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-09-19 08:31:04.447175044 +0000 2024-09-19T08:37:17.293 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-09-19 08:31:04.447175044 +0000 2024-09-19T08:37:17.293 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: - 2024-09-19T08:37:17.293 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T08:37:17.346 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-09-19T08:37:17.346 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-09-19T08:37:17.346 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000446123 s, 1.1 MB/s 2024-09-19T08:37:17.347 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T08:37:17.397 DEBUG:teuthology.orchestra.run.smithi098:> stat /dev/vg_nvme/lv_4 2024-09-19T08:37:17.444 INFO:teuthology.orchestra.run.smithi098.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T08:37:17.444 INFO:teuthology.orchestra.run.smithi098.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:37:17.444 INFO:teuthology.orchestra.run.smithi098.stdout:Device: 5h/5d Inode: 922 Links: 1 2024-09-19T08:37:17.444 INFO:teuthology.orchestra.run.smithi098.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:37:17.444 INFO:teuthology.orchestra.run.smithi098.stdout:Access: 2024-09-19 08:31:05.163168649 +0000 2024-09-19T08:37:17.444 INFO:teuthology.orchestra.run.smithi098.stdout:Modify: 2024-09-19 08:31:04.927170757 +0000 2024-09-19T08:37:17.444 INFO:teuthology.orchestra.run.smithi098.stdout:Change: 2024-09-19 08:31:04.927170757 +0000 2024-09-19T08:37:17.444 INFO:teuthology.orchestra.run.smithi098.stdout: Birth: - 2024-09-19T08:37:17.445 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T08:37:17.499 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-09-19T08:37:17.499 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-09-19T08:37:17.499 INFO:teuthology.orchestra.run.smithi098.stderr:512 bytes copied, 0.000287142 s, 1.8 MB/s 2024-09-19T08:37:17.500 DEBUG:teuthology.orchestra.run.smithi098:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T08:37:17.549 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-19T08:37:17.549 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:37:17.549 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T08:37:17.555 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T08:37:17.555 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_1 2024-09-19T08:37:17.605 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T08:37:17.605 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:37:17.605 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 881 Links: 1 2024-09-19T08:37:17.605 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:37:17.605 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-09-19 08:31:05.972584594 +0000 2024-09-19T08:37:17.605 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-09-19 08:31:05.716586879 +0000 2024-09-19T08:37:17.605 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-09-19 08:31:05.716586879 +0000 2024-09-19T08:37:17.605 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: - 2024-09-19T08:37:17.605 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T08:37:17.658 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-09-19T08:37:17.658 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-09-19T08:37:17.658 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000389407 s, 1.3 MB/s 2024-09-19T08:37:17.660 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T08:37:17.708 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_2 2024-09-19T08:37:17.755 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T08:37:17.755 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:37:17.755 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 896 Links: 1 2024-09-19T08:37:17.755 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:37:17.755 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-09-19 08:31:06.484580020 +0000 2024-09-19T08:37:17.755 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-09-19 08:31:06.220582379 +0000 2024-09-19T08:37:17.755 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-09-19 08:31:06.220582379 +0000 2024-09-19T08:37:17.755 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: - 2024-09-19T08:37:17.756 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T08:37:17.809 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-09-19T08:37:17.809 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-09-19T08:37:17.810 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000416699 s, 1.2 MB/s 2024-09-19T08:37:17.811 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T08:37:17.860 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_3 2024-09-19T08:37:17.907 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T08:37:17.907 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:37:17.907 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 908 Links: 1 2024-09-19T08:37:17.908 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:37:17.908 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-09-19 08:31:06.976575626 +0000 2024-09-19T08:37:17.908 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-09-19 08:31:06.728577841 +0000 2024-09-19T08:37:17.908 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-09-19 08:31:06.728577841 +0000 2024-09-19T08:37:17.908 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: - 2024-09-19T08:37:17.908 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T08:37:17.960 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-09-19T08:37:17.960 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-09-19T08:37:17.961 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000304855 s, 1.7 MB/s 2024-09-19T08:37:17.962 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T08:37:18.008 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_4 2024-09-19T08:37:18.055 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T08:37:18.055 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T08:37:18.055 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 926 Links: 1 2024-09-19T08:37:18.055 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T08:37:18.055 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-09-19 08:31:07.440571480 +0000 2024-09-19T08:37:18.056 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-09-19 08:31:07.204573588 +0000 2024-09-19T08:37:18.056 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-09-19 08:31:07.204573588 +0000 2024-09-19T08:37:18.056 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: - 2024-09-19T08:37:18.056 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T08:37:18.109 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-09-19T08:37:18.110 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-09-19T08:37:18.110 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.00039091 s, 1.3 MB/s 2024-09-19T08:37:18.111 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T08:37:18.160 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-19T08:37:18.160 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi098.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi139.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-19T08:37:18.160 INFO:tasks.ceph:Generating config... 2024-09-19T08:37:18.162 INFO:tasks.ceph:[client] client mount timeout = 600 2024-09-19T08:37:18.162 INFO:tasks.ceph:[client] debug client = 20 2024-09-19T08:37:18.162 INFO:tasks.ceph:[client] debug ms = 1 2024-09-19T08:37:18.162 INFO:tasks.ceph:[client] pid file = 2024-09-19T08:37:18.162 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-09-19T08:37:18.162 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-09-19T08:37:18.162 INFO:tasks.ceph:[mds] debug mds = 20 2024-09-19T08:37:18.162 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-09-19T08:37:18.162 INFO:tasks.ceph:[mds] debug ms = 1 2024-09-19T08:37:18.162 INFO:tasks.ceph:[mds] mds debug frag = True 2024-09-19T08:37:18.162 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-09-19T08:37:18.163 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-09-19T08:37:18.163 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-09-19T08:37:18.163 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-09-19T08:37:18.163 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-09-19T08:37:18.163 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-09-19T08:37:18.163 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-19T08:37:18.163 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-09-19T08:37:18.163 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-19T08:37:18.163 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-19T08:37:18.163 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-19T08:37:18.163 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-19T08:37:18.163 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-19T08:37:18.163 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-19T08:37:18.163 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-09-19T08:37:18.164 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-09-19T08:37:18.164 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-19T08:37:18.164 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-19T08:37:18.164 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-19T08:37:18.164 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-19T08:37:18.164 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-19T08:37:18.164 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-19T08:37:18.164 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-19T08:37:18.164 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-19T08:37:18.164 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-19T08:37:18.164 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-19T08:37:18.164 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-19T08:37:18.164 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-19T08:37:18.165 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-19T08:37:18.165 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-19T08:37:18.165 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-19T08:37:18.165 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-09-19T08:37:18.165 INFO:tasks.ceph:Setting up mon.a... 2024-09-19T08:37:18.165 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-19T08:37:18.198 INFO:teuthology.orchestra.run.smithi098.stdout:creating /etc/ceph/ceph.keyring 2024-09-19T08:37:18.212 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-19T08:37:18.291 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-19T08:37:18.347 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.98'), ('mon.b', '172.21.15.139'), ('mon.c', '[v2:172.21.15.139:3301,v1:172.21.15.139:6790]')] 2024-09-19T08:37:18.347 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.98,172.21.15.139,[v2:172.21.15.139:3301,v1:172.21.15.139:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-09-19T08:37:18.348 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:37:18.348 DEBUG:teuthology.orchestra.run.smithi098:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-19T08:37:18.396 DEBUG:teuthology.orchestra.run.smithi098:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.98 --add b 172.21.15.139 --addv c '[v2:172.21.15.139:3301,v1:172.21.15.139:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-19T08:37:18.480 INFO:teuthology.orchestra.run.smithi098.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-19T08:37:18.481 INFO:teuthology.orchestra.run.smithi098.stdout:monmaptool: generated fsid 552bf0ee-1f4f-4644-ba7c-d0931e55a451 2024-09-19T08:37:18.481 INFO:teuthology.orchestra.run.smithi098.stdout:setting min_mon_release = quincy 2024-09-19T08:37:18.481 INFO:teuthology.orchestra.run.smithi098.stdout:epoch 0 2024-09-19T08:37:18.481 INFO:teuthology.orchestra.run.smithi098.stdout:fsid 552bf0ee-1f4f-4644-ba7c-d0931e55a451 2024-09-19T08:37:18.481 INFO:teuthology.orchestra.run.smithi098.stdout:last_changed 2024-09-19T08:37:18.480688+0000 2024-09-19T08:37:18.481 INFO:teuthology.orchestra.run.smithi098.stdout:created 2024-09-19T08:37:18.480688+0000 2024-09-19T08:37:18.481 INFO:teuthology.orchestra.run.smithi098.stdout:min_mon_release 17 (quincy) 2024-09-19T08:37:18.481 INFO:teuthology.orchestra.run.smithi098.stdout:election_strategy: 1 2024-09-19T08:37:18.481 INFO:teuthology.orchestra.run.smithi098.stdout:0: [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] mon.a 2024-09-19T08:37:18.481 INFO:teuthology.orchestra.run.smithi098.stdout:1: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.b 2024-09-19T08:37:18.481 INFO:teuthology.orchestra.run.smithi098.stdout:2: [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] mon.c 2024-09-19T08:37:18.481 INFO:teuthology.orchestra.run.smithi098.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-19T08:37:18.482 DEBUG:teuthology.orchestra.run.smithi098:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-19T08:37:18.528 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 552bf0ee-1f4f-4644-ba7c-d0931e55a451... 2024-09-19T08:37:18.529 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-19T08:37:18.573 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-19T08:37:18.598 INFO:teuthology.orchestra.run.smithi098.stdout:[global] 2024-09-19T08:37:18.598 INFO:teuthology.orchestra.run.smithi098.stdout: chdir = "" 2024-09-19T08:37:18.598 INFO:teuthology.orchestra.run.smithi098.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-19T08:37:18.598 INFO:teuthology.orchestra.run.smithi098.stdout: auth supported = cephx 2024-09-19T08:37:18.598 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:37:18.598 INFO:teuthology.orchestra.run.smithi098.stdout: filestore xattr use omap = true 2024-09-19T08:37:18.598 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:37:18.599 INFO:teuthology.orchestra.run.smithi098.stdout: mon clock drift allowed = 1.000 2024-09-19T08:37:18.599 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:37:18.599 INFO:teuthology.orchestra.run.smithi098.stdout: osd crush chooseleaf type = 0 2024-09-19T08:37:18.599 INFO:teuthology.orchestra.run.smithi098.stdout: auth debug = true 2024-09-19T08:37:18.599 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:37:18.599 INFO:teuthology.orchestra.run.smithi098.stdout: ms die on old message = true 2024-09-19T08:37:18.599 INFO:teuthology.orchestra.run.smithi098.stdout: ms die on bug = true 2024-09-19T08:37:18.599 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:37:18.599 INFO:teuthology.orchestra.run.smithi098.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-19T08:37:18.599 INFO:teuthology.orchestra.run.smithi098.stdout: mon pg warn max object skew = 0 2024-09-19T08:37:18.599 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:37:18.599 INFO:teuthology.orchestra.run.smithi098.stdout: # disable pg_autoscaler by default for new pools 2024-09-19T08:37:18.599 INFO:teuthology.orchestra.run.smithi098.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-19T08:37:18.599 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:37:18.600 INFO:teuthology.orchestra.run.smithi098.stdout: osd pool default size = 2 2024-09-19T08:37:18.600 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:37:18.600 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd allow primary affinity = true 2024-09-19T08:37:18.600 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd allow pg remap = true 2024-09-19T08:37:18.600 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on legacy crush tunables = false 2024-09-19T08:37:18.600 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on crush straw calc version zero = false 2024-09-19T08:37:18.600 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on no sortbitwise = false 2024-09-19T08:37:18.600 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on osd down out interval zero = false 2024-09-19T08:37:18.600 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on too few osds = false 2024-09-19T08:37:18.600 INFO:teuthology.orchestra.run.smithi098.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-19T08:37:18.600 INFO:teuthology.orchestra.run.smithi098.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-19T08:37:18.600 INFO:teuthology.orchestra.run.smithi098.stdout: mon_allow_pool_size_one = true 2024-09-19T08:37:18.600 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:37:18.601 INFO:teuthology.orchestra.run.smithi098.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-19T08:37:18.601 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:37:18.601 INFO:teuthology.orchestra.run.smithi098.stdout: osd default data pool replay window = 5 2024-09-19T08:37:18.601 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:37:18.601 INFO:teuthology.orchestra.run.smithi098.stdout: mon allow pool delete = true 2024-09-19T08:37:18.601 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:37:18.601 INFO:teuthology.orchestra.run.smithi098.stdout: mon cluster log file level = debug 2024-09-19T08:37:18.601 INFO:teuthology.orchestra.run.smithi098.stdout: debug asserts on shutdown = true 2024-09-19T08:37:18.601 INFO:teuthology.orchestra.run.smithi098.stdout: mon health detail to clog = false 2024-09-19T08:37:18.602 INFO:teuthology.orchestra.run.smithi098.stdout: mon host = "172.21.15.98,172.21.15.139,[v2:172.21.15.139:3301,v1:172.21.15.139:6790]" 2024-09-19T08:37:18.602 INFO:teuthology.orchestra.run.smithi098.stdout: fsid = 552bf0ee-1f4f-4644-ba7c-d0931e55a451 2024-09-19T08:37:18.602 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:37:18.602 INFO:teuthology.orchestra.run.smithi098.stdout:[osd] 2024-09-19T08:37:18.602 INFO:teuthology.orchestra.run.smithi098.stdout: osd journal size = 100 2024-09-19T08:37:18.602 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:37:18.602 INFO:teuthology.orchestra.run.smithi098.stdout: osd scrub load threshold = 5.0 2024-09-19T08:37:18.602 INFO:teuthology.orchestra.run.smithi098.stdout: osd scrub max interval = 600 2024-09-19T08:37:18.602 INFO:teuthology.orchestra.run.smithi098.stdout: osd mclock profile = high_recovery_ops 2024-09-19T08:37:18.602 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:37:18.602 INFO:teuthology.orchestra.run.smithi098.stdout: osd recover clone overlap = true 2024-09-19T08:37:18.602 INFO:teuthology.orchestra.run.smithi098.stdout: osd recovery max chunk = 1048576 2024-09-19T08:37:18.603 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:37:18.603 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug shutdown = true 2024-09-19T08:37:18.603 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug op order = true 2024-09-19T08:37:18.603 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug verify stray on activate = true 2024-09-19T08:37:18.603 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug trim objects = true 2024-09-19T08:37:18.603 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:37:18.603 INFO:teuthology.orchestra.run.smithi098.stdout: osd open classes on start = true 2024-09-19T08:37:18.603 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug pg log writeout = true 2024-09-19T08:37:18.603 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:37:18.603 INFO:teuthology.orchestra.run.smithi098.stdout: osd deep scrub update digest min age = 30 2024-09-19T08:37:18.603 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:37:18.603 INFO:teuthology.orchestra.run.smithi098.stdout: osd map max advance = 10 2024-09-19T08:37:18.604 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:37:18.604 INFO:teuthology.orchestra.run.smithi098.stdout: journal zero on create = true 2024-09-19T08:37:18.604 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:37:18.604 INFO:teuthology.orchestra.run.smithi098.stdout: filestore ondisk finisher threads = 3 2024-09-19T08:37:18.604 INFO:teuthology.orchestra.run.smithi098.stdout: filestore apply finisher threads = 3 2024-09-19T08:37:18.604 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:37:18.604 INFO:teuthology.orchestra.run.smithi098.stdout: bdev debug aio = true 2024-09-19T08:37:18.604 INFO:teuthology.orchestra.run.smithi098.stdout: osd debug misdirected ops = true 2024-09-19T08:37:18.604 INFO:teuthology.orchestra.run.smithi098.stdout: bdev async discard = True 2024-09-19T08:37:18.604 INFO:teuthology.orchestra.run.smithi098.stdout: bdev enable discard = True 2024-09-19T08:37:18.604 INFO:teuthology.orchestra.run.smithi098.stdout: bluestore allocator = bitmap 2024-09-19T08:37:18.604 INFO:teuthology.orchestra.run.smithi098.stdout: bluestore block size = 96636764160 2024-09-19T08:37:18.604 INFO:teuthology.orchestra.run.smithi098.stdout: bluestore fsck on mount = True 2024-09-19T08:37:18.605 INFO:teuthology.orchestra.run.smithi098.stdout: debug bluefs = 1/20 2024-09-19T08:37:18.605 INFO:teuthology.orchestra.run.smithi098.stdout: debug bluestore = 1/20 2024-09-19T08:37:18.605 INFO:teuthology.orchestra.run.smithi098.stdout: debug ms = 1 2024-09-19T08:37:18.605 INFO:teuthology.orchestra.run.smithi098.stdout: debug osd = 20 2024-09-19T08:37:18.605 INFO:teuthology.orchestra.run.smithi098.stdout: debug rocksdb = 4/10 2024-09-19T08:37:18.605 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd backfillfull_ratio = 0.85 2024-09-19T08:37:18.605 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd full ratio = 0.9 2024-09-19T08:37:18.605 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd nearfull ratio = 0.8 2024-09-19T08:37:18.605 INFO:teuthology.orchestra.run.smithi098.stdout: osd failsafe full ratio = 0.95 2024-09-19T08:37:18.605 INFO:teuthology.orchestra.run.smithi098.stdout: osd objectstore = bluestore 2024-09-19T08:37:18.605 INFO:teuthology.orchestra.run.smithi098.stdout: osd op complaint time = 180 2024-09-19T08:37:18.605 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:37:18.605 INFO:teuthology.orchestra.run.smithi098.stdout:[mgr] 2024-09-19T08:37:18.606 INFO:teuthology.orchestra.run.smithi098.stdout: debug ms = 1 2024-09-19T08:37:18.606 INFO:teuthology.orchestra.run.smithi098.stdout: debug mgr = 20 2024-09-19T08:37:18.606 INFO:teuthology.orchestra.run.smithi098.stdout: debug mon = 20 2024-09-19T08:37:18.606 INFO:teuthology.orchestra.run.smithi098.stdout: debug auth = 20 2024-09-19T08:37:18.606 INFO:teuthology.orchestra.run.smithi098.stdout: mon reweight min pgs per osd = 4 2024-09-19T08:37:18.606 INFO:teuthology.orchestra.run.smithi098.stdout: mon reweight min bytes per osd = 10 2024-09-19T08:37:18.606 INFO:teuthology.orchestra.run.smithi098.stdout: mgr/telemetry/nag = false 2024-09-19T08:37:18.606 INFO:teuthology.orchestra.run.smithi098.stdout: debug client = 20 2024-09-19T08:37:18.606 INFO:teuthology.orchestra.run.smithi098.stdout: debug finisher = 20 2024-09-19T08:37:18.606 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:37:18.606 INFO:teuthology.orchestra.run.smithi098.stdout:[mon] 2024-09-19T08:37:18.606 INFO:teuthology.orchestra.run.smithi098.stdout: debug ms = 1 2024-09-19T08:37:18.606 INFO:teuthology.orchestra.run.smithi098.stdout: debug mon = 20 2024-09-19T08:37:18.607 INFO:teuthology.orchestra.run.smithi098.stdout: debug paxos = 20 2024-09-19T08:37:18.607 INFO:teuthology.orchestra.run.smithi098.stdout: debug auth = 20 2024-09-19T08:37:18.607 INFO:teuthology.orchestra.run.smithi098.stdout: mon data avail warn = 5 2024-09-19T08:37:18.607 INFO:teuthology.orchestra.run.smithi098.stdout: mon mgr mkfs grace = 240 2024-09-19T08:37:18.607 INFO:teuthology.orchestra.run.smithi098.stdout: mon reweight min pgs per osd = 4 2024-09-19T08:37:18.607 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd reporter subtree level = osd 2024-09-19T08:37:18.607 INFO:teuthology.orchestra.run.smithi098.stdout: mon osd prime pg temp = true 2024-09-19T08:37:18.607 INFO:teuthology.orchestra.run.smithi098.stdout: mon reweight min bytes per osd = 10 2024-09-19T08:37:18.607 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:37:18.607 INFO:teuthology.orchestra.run.smithi098.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-19T08:37:18.607 INFO:teuthology.orchestra.run.smithi098.stdout: auth mon ticket ttl = 660 # 11m 2024-09-19T08:37:18.607 INFO:teuthology.orchestra.run.smithi098.stdout: auth service ticket ttl = 240 # 4m 2024-09-19T08:37:18.607 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:37:18.607 INFO:teuthology.orchestra.run.smithi098.stdout: # don't complain about insecure global_id in the test suite 2024-09-19T08:37:18.608 INFO:teuthology.orchestra.run.smithi098.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-19T08:37:18.608 INFO:teuthology.orchestra.run.smithi098.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-19T08:37:18.608 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:37:18.608 INFO:teuthology.orchestra.run.smithi098.stdout: # 1m isn't quite enough 2024-09-19T08:37:18.608 INFO:teuthology.orchestra.run.smithi098.stdout: mon_down_mkfs_grace = 2m 2024-09-19T08:37:18.608 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:37:18.608 INFO:teuthology.orchestra.run.smithi098.stdout: mon_warn_on_filestore_osds = false 2024-09-19T08:37:18.608 INFO:teuthology.orchestra.run.smithi098.stdout: mon down mkfs grace = 300 2024-09-19T08:37:18.608 INFO:teuthology.orchestra.run.smithi098.stdout: mon op complaint time = 120 2024-09-19T08:37:18.608 INFO:teuthology.orchestra.run.smithi098.stdout: mon warn on pool no app = False 2024-09-19T08:37:18.608 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:37:18.608 INFO:teuthology.orchestra.run.smithi098.stdout:[client] 2024-09-19T08:37:18.608 INFO:teuthology.orchestra.run.smithi098.stdout: rgw cache enabled = true 2024-09-19T08:37:18.609 INFO:teuthology.orchestra.run.smithi098.stdout: rgw enable ops log = true 2024-09-19T08:37:18.609 INFO:teuthology.orchestra.run.smithi098.stdout: rgw enable usage log = true 2024-09-19T08:37:18.609 INFO:teuthology.orchestra.run.smithi098.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-19T08:37:18.609 INFO:teuthology.orchestra.run.smithi098.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-19T08:37:18.609 INFO:teuthology.orchestra.run.smithi098.stdout: client mount timeout = 600 2024-09-19T08:37:18.609 INFO:teuthology.orchestra.run.smithi098.stdout: debug client = 20 2024-09-19T08:37:18.609 INFO:teuthology.orchestra.run.smithi098.stdout: debug ms = 1 2024-09-19T08:37:18.609 INFO:teuthology.orchestra.run.smithi098.stdout: pid file = "" 2024-09-19T08:37:18.609 INFO:teuthology.orchestra.run.smithi098.stdout: rados mon op timeout = 900 2024-09-19T08:37:18.609 INFO:teuthology.orchestra.run.smithi098.stdout: rados osd op timeout = 900 2024-09-19T08:37:18.609 INFO:teuthology.orchestra.run.smithi098.stdout:[mon.a] 2024-09-19T08:37:18.609 INFO:teuthology.orchestra.run.smithi098.stdout:[mon.b] 2024-09-19T08:37:18.609 INFO:teuthology.orchestra.run.smithi098.stdout:[mon.c] 2024-09-19T08:37:18.610 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.a] 2024-09-19T08:37:18.610 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.c] 2024-09-19T08:37:18.610 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.b] 2024-09-19T08:37:18.610 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.d] 2024-09-19T08:37:18.610 INFO:teuthology.orchestra.run.smithi098.stdout:[mds] 2024-09-19T08:37:18.610 INFO:teuthology.orchestra.run.smithi098.stdout: debug mds = 20 2024-09-19T08:37:18.610 INFO:teuthology.orchestra.run.smithi098.stdout: debug mds balancer = 20 2024-09-19T08:37:18.610 INFO:teuthology.orchestra.run.smithi098.stdout: debug ms = 1 2024-09-19T08:37:18.610 INFO:teuthology.orchestra.run.smithi098.stdout: mds debug frag = True 2024-09-19T08:37:18.610 INFO:teuthology.orchestra.run.smithi098.stdout: mds debug scatterstat = True 2024-09-19T08:37:18.610 INFO:teuthology.orchestra.run.smithi098.stdout: mds op complaint time = 180 2024-09-19T08:37:18.610 INFO:teuthology.orchestra.run.smithi098.stdout: mds verify scatter = True 2024-09-19T08:37:18.610 INFO:teuthology.orchestra.run.smithi098.stdout: osd op complaint time = 180 2024-09-19T08:37:18.611 INFO:teuthology.orchestra.run.smithi098.stdout: rados mon op timeout = 900 2024-09-19T08:37:18.611 INFO:teuthology.orchestra.run.smithi098.stdout: rados osd op timeout = 900 2024-09-19T08:37:18.611 INFO:teuthology.orchestra.run.smithi139.stdout:[global] 2024-09-19T08:37:18.611 INFO:teuthology.orchestra.run.smithi139.stdout: chdir = "" 2024-09-19T08:37:18.611 INFO:teuthology.orchestra.run.smithi139.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-19T08:37:18.611 INFO:teuthology.orchestra.run.smithi139.stdout: auth supported = cephx 2024-09-19T08:37:18.611 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-19T08:37:18.612 INFO:teuthology.orchestra.run.smithi139.stdout: filestore xattr use omap = true 2024-09-19T08:37:18.612 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-19T08:37:18.612 INFO:teuthology.orchestra.run.smithi139.stdout: mon clock drift allowed = 1.000 2024-09-19T08:37:18.612 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-19T08:37:18.612 INFO:teuthology.orchestra.run.smithi139.stdout: osd crush chooseleaf type = 0 2024-09-19T08:37:18.612 INFO:teuthology.orchestra.run.smithi139.stdout: auth debug = true 2024-09-19T08:37:18.612 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-19T08:37:18.612 INFO:teuthology.orchestra.run.smithi139.stdout: ms die on old message = true 2024-09-19T08:37:18.612 INFO:teuthology.orchestra.run.smithi139.stdout: ms die on bug = true 2024-09-19T08:37:18.612 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-19T08:37:18.612 INFO:teuthology.orchestra.run.smithi139.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-19T08:37:18.612 INFO:teuthology.orchestra.run.smithi139.stdout: mon pg warn max object skew = 0 2024-09-19T08:37:18.612 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-19T08:37:18.613 INFO:teuthology.orchestra.run.smithi139.stdout: # disable pg_autoscaler by default for new pools 2024-09-19T08:37:18.613 INFO:teuthology.orchestra.run.smithi139.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-19T08:37:18.613 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-19T08:37:18.613 INFO:teuthology.orchestra.run.smithi139.stdout: osd pool default size = 2 2024-09-19T08:37:18.613 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-19T08:37:18.613 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd allow primary affinity = true 2024-09-19T08:37:18.613 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd allow pg remap = true 2024-09-19T08:37:18.613 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on legacy crush tunables = false 2024-09-19T08:37:18.613 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on crush straw calc version zero = false 2024-09-19T08:37:18.613 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on no sortbitwise = false 2024-09-19T08:37:18.613 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on osd down out interval zero = false 2024-09-19T08:37:18.613 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on too few osds = false 2024-09-19T08:37:18.614 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-19T08:37:18.614 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-19T08:37:18.614 INFO:teuthology.orchestra.run.smithi139.stdout: mon_allow_pool_size_one = true 2024-09-19T08:37:18.614 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-19T08:37:18.614 INFO:teuthology.orchestra.run.smithi139.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-19T08:37:18.614 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-19T08:37:18.614 INFO:teuthology.orchestra.run.smithi139.stdout: osd default data pool replay window = 5 2024-09-19T08:37:18.614 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-19T08:37:18.614 INFO:teuthology.orchestra.run.smithi139.stdout: mon allow pool delete = true 2024-09-19T08:37:18.614 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-19T08:37:18.614 INFO:teuthology.orchestra.run.smithi139.stdout: mon cluster log file level = debug 2024-09-19T08:37:18.614 INFO:teuthology.orchestra.run.smithi139.stdout: debug asserts on shutdown = true 2024-09-19T08:37:18.614 INFO:teuthology.orchestra.run.smithi139.stdout: mon health detail to clog = false 2024-09-19T08:37:18.614 INFO:teuthology.orchestra.run.smithi139.stdout: mon host = "172.21.15.98,172.21.15.139,[v2:172.21.15.139:3301,v1:172.21.15.139:6790]" 2024-09-19T08:37:18.615 INFO:teuthology.orchestra.run.smithi139.stdout: fsid = 552bf0ee-1f4f-4644-ba7c-d0931e55a451 2024-09-19T08:37:18.615 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-19T08:37:18.615 INFO:teuthology.orchestra.run.smithi139.stdout:[osd] 2024-09-19T08:37:18.615 INFO:teuthology.orchestra.run.smithi139.stdout: osd journal size = 100 2024-09-19T08:37:18.615 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-19T08:37:18.615 INFO:teuthology.orchestra.run.smithi139.stdout: osd scrub load threshold = 5.0 2024-09-19T08:37:18.615 INFO:teuthology.orchestra.run.smithi139.stdout: osd scrub max interval = 600 2024-09-19T08:37:18.615 INFO:teuthology.orchestra.run.smithi139.stdout: osd mclock profile = high_recovery_ops 2024-09-19T08:37:18.615 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-19T08:37:18.615 INFO:teuthology.orchestra.run.smithi139.stdout: osd recover clone overlap = true 2024-09-19T08:37:18.615 INFO:teuthology.orchestra.run.smithi139.stdout: osd recovery max chunk = 1048576 2024-09-19T08:37:18.615 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-19T08:37:18.615 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug shutdown = true 2024-09-19T08:37:18.616 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug op order = true 2024-09-19T08:37:18.616 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug verify stray on activate = true 2024-09-19T08:37:18.616 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug trim objects = true 2024-09-19T08:37:18.616 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-19T08:37:18.616 INFO:teuthology.orchestra.run.smithi139.stdout: osd open classes on start = true 2024-09-19T08:37:18.616 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug pg log writeout = true 2024-09-19T08:37:18.616 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-19T08:37:18.616 INFO:teuthology.orchestra.run.smithi139.stdout: osd deep scrub update digest min age = 30 2024-09-19T08:37:18.616 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-19T08:37:18.616 INFO:teuthology.orchestra.run.smithi139.stdout: osd map max advance = 10 2024-09-19T08:37:18.616 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-19T08:37:18.616 INFO:teuthology.orchestra.run.smithi139.stdout: journal zero on create = true 2024-09-19T08:37:18.616 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-19T08:37:18.616 INFO:teuthology.orchestra.run.smithi139.stdout: filestore ondisk finisher threads = 3 2024-09-19T08:37:18.617 INFO:teuthology.orchestra.run.smithi139.stdout: filestore apply finisher threads = 3 2024-09-19T08:37:18.617 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-19T08:37:18.617 INFO:teuthology.orchestra.run.smithi139.stdout: bdev debug aio = true 2024-09-19T08:37:18.617 INFO:teuthology.orchestra.run.smithi139.stdout: osd debug misdirected ops = true 2024-09-19T08:37:18.617 INFO:teuthology.orchestra.run.smithi139.stdout: bdev async discard = True 2024-09-19T08:37:18.617 INFO:teuthology.orchestra.run.smithi139.stdout: bdev enable discard = True 2024-09-19T08:37:18.617 INFO:teuthology.orchestra.run.smithi139.stdout: bluestore allocator = bitmap 2024-09-19T08:37:18.617 INFO:teuthology.orchestra.run.smithi139.stdout: bluestore block size = 96636764160 2024-09-19T08:37:18.617 INFO:teuthology.orchestra.run.smithi139.stdout: bluestore fsck on mount = True 2024-09-19T08:37:18.617 INFO:teuthology.orchestra.run.smithi139.stdout: debug bluefs = 1/20 2024-09-19T08:37:18.617 INFO:teuthology.orchestra.run.smithi139.stdout: debug bluestore = 1/20 2024-09-19T08:37:18.617 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-09-19T08:37:18.617 INFO:teuthology.orchestra.run.smithi139.stdout: debug osd = 20 2024-09-19T08:37:18.617 INFO:teuthology.orchestra.run.smithi139.stdout: debug rocksdb = 4/10 2024-09-19T08:37:18.618 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd backfillfull_ratio = 0.85 2024-09-19T08:37:18.618 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd full ratio = 0.9 2024-09-19T08:37:18.618 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd nearfull ratio = 0.8 2024-09-19T08:37:18.618 INFO:teuthology.orchestra.run.smithi139.stdout: osd failsafe full ratio = 0.95 2024-09-19T08:37:18.618 INFO:teuthology.orchestra.run.smithi139.stdout: osd objectstore = bluestore 2024-09-19T08:37:18.618 INFO:teuthology.orchestra.run.smithi139.stdout: osd op complaint time = 180 2024-09-19T08:37:18.618 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-19T08:37:18.618 INFO:teuthology.orchestra.run.smithi139.stdout:[mgr] 2024-09-19T08:37:18.618 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-09-19T08:37:18.618 INFO:teuthology.orchestra.run.smithi139.stdout: debug mgr = 20 2024-09-19T08:37:18.618 INFO:teuthology.orchestra.run.smithi139.stdout: debug mon = 20 2024-09-19T08:37:18.618 INFO:teuthology.orchestra.run.smithi139.stdout: debug auth = 20 2024-09-19T08:37:18.618 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min pgs per osd = 4 2024-09-19T08:37:18.618 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min bytes per osd = 10 2024-09-19T08:37:18.619 INFO:teuthology.orchestra.run.smithi139.stdout: mgr/telemetry/nag = false 2024-09-19T08:37:18.619 INFO:teuthology.orchestra.run.smithi139.stdout: debug client = 20 2024-09-19T08:37:18.619 INFO:teuthology.orchestra.run.smithi139.stdout: debug finisher = 20 2024-09-19T08:37:18.619 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-19T08:37:18.619 INFO:teuthology.orchestra.run.smithi139.stdout:[mon] 2024-09-19T08:37:18.619 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-09-19T08:37:18.619 INFO:teuthology.orchestra.run.smithi139.stdout: debug mon = 20 2024-09-19T08:37:18.619 INFO:teuthology.orchestra.run.smithi139.stdout: debug paxos = 20 2024-09-19T08:37:18.619 INFO:teuthology.orchestra.run.smithi139.stdout: debug auth = 20 2024-09-19T08:37:18.619 INFO:teuthology.orchestra.run.smithi139.stdout: mon data avail warn = 5 2024-09-19T08:37:18.619 INFO:teuthology.orchestra.run.smithi139.stdout: mon mgr mkfs grace = 240 2024-09-19T08:37:18.619 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min pgs per osd = 4 2024-09-19T08:37:18.619 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd reporter subtree level = osd 2024-09-19T08:37:18.620 INFO:teuthology.orchestra.run.smithi139.stdout: mon osd prime pg temp = true 2024-09-19T08:37:18.620 INFO:teuthology.orchestra.run.smithi139.stdout: mon reweight min bytes per osd = 10 2024-09-19T08:37:18.620 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-19T08:37:18.620 INFO:teuthology.orchestra.run.smithi139.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-19T08:37:18.620 INFO:teuthology.orchestra.run.smithi139.stdout: auth mon ticket ttl = 660 # 11m 2024-09-19T08:37:18.620 INFO:teuthology.orchestra.run.smithi139.stdout: auth service ticket ttl = 240 # 4m 2024-09-19T08:37:18.620 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-19T08:37:18.620 INFO:teuthology.orchestra.run.smithi139.stdout: # don't complain about insecure global_id in the test suite 2024-09-19T08:37:18.620 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-19T08:37:18.620 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-19T08:37:18.620 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-19T08:37:18.620 INFO:teuthology.orchestra.run.smithi139.stdout: # 1m isn't quite enough 2024-09-19T08:37:18.620 INFO:teuthology.orchestra.run.smithi139.stdout: mon_down_mkfs_grace = 2m 2024-09-19T08:37:18.621 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-19T08:37:18.621 INFO:teuthology.orchestra.run.smithi139.stdout: mon_warn_on_filestore_osds = false 2024-09-19T08:37:18.621 INFO:teuthology.orchestra.run.smithi139.stdout: mon down mkfs grace = 300 2024-09-19T08:37:18.621 INFO:teuthology.orchestra.run.smithi139.stdout: mon op complaint time = 120 2024-09-19T08:37:18.621 INFO:teuthology.orchestra.run.smithi139.stdout: mon warn on pool no app = False 2024-09-19T08:37:18.621 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-09-19T08:37:18.621 INFO:teuthology.orchestra.run.smithi139.stdout:[client] 2024-09-19T08:37:18.621 INFO:teuthology.orchestra.run.smithi139.stdout: rgw cache enabled = true 2024-09-19T08:37:18.621 INFO:teuthology.orchestra.run.smithi139.stdout: rgw enable ops log = true 2024-09-19T08:37:18.621 INFO:teuthology.orchestra.run.smithi139.stdout: rgw enable usage log = true 2024-09-19T08:37:18.621 INFO:teuthology.orchestra.run.smithi139.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-19T08:37:18.621 INFO:teuthology.orchestra.run.smithi139.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-19T08:37:18.621 INFO:teuthology.orchestra.run.smithi139.stdout: client mount timeout = 600 2024-09-19T08:37:18.621 INFO:teuthology.orchestra.run.smithi139.stdout: debug client = 20 2024-09-19T08:37:18.622 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-09-19T08:37:18.622 INFO:teuthology.orchestra.run.smithi139.stdout: pid file = "" 2024-09-19T08:37:18.622 INFO:teuthology.orchestra.run.smithi139.stdout: rados mon op timeout = 900 2024-09-19T08:37:18.622 INFO:teuthology.orchestra.run.smithi139.stdout: rados osd op timeout = 900 2024-09-19T08:37:18.622 INFO:teuthology.orchestra.run.smithi139.stdout:[mon.a] 2024-09-19T08:37:18.622 INFO:teuthology.orchestra.run.smithi139.stdout:[mon.b] 2024-09-19T08:37:18.622 INFO:teuthology.orchestra.run.smithi139.stdout:[mon.c] 2024-09-19T08:37:18.622 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.a] 2024-09-19T08:37:18.622 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.c] 2024-09-19T08:37:18.622 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.b] 2024-09-19T08:37:18.622 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.d] 2024-09-19T08:37:18.622 INFO:teuthology.orchestra.run.smithi139.stdout:[mds] 2024-09-19T08:37:18.622 INFO:teuthology.orchestra.run.smithi139.stdout: debug mds = 20 2024-09-19T08:37:18.623 INFO:teuthology.orchestra.run.smithi139.stdout: debug mds balancer = 20 2024-09-19T08:37:18.623 INFO:teuthology.orchestra.run.smithi139.stdout: debug ms = 1 2024-09-19T08:37:18.623 INFO:teuthology.orchestra.run.smithi139.stdout: mds debug frag = True 2024-09-19T08:37:18.623 INFO:teuthology.orchestra.run.smithi139.stdout: mds debug scatterstat = True 2024-09-19T08:37:18.623 INFO:teuthology.orchestra.run.smithi139.stdout: mds op complaint time = 180 2024-09-19T08:37:18.623 INFO:teuthology.orchestra.run.smithi139.stdout: mds verify scatter = True 2024-09-19T08:37:18.623 INFO:teuthology.orchestra.run.smithi139.stdout: osd op complaint time = 180 2024-09-19T08:37:18.623 INFO:teuthology.orchestra.run.smithi139.stdout: rados mon op timeout = 900 2024-09-19T08:37:18.623 INFO:teuthology.orchestra.run.smithi139.stdout: rados osd op timeout = 900 2024-09-19T08:37:18.625 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-19T08:37:18.625 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-19T08:37:18.715 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-19T08:37:18.716 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:37:18.716 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-19T08:37:18.721 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:37:18.721 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-19T08:37:18.768 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:37:18.768 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:37:18.768 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-19T08:37:18.768 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-19T08:37:18.832 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:37:18.832 DEBUG:teuthology.orchestra.run.smithi098:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-19T08:37:18.881 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:37:18.881 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:37:18.881 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-19T08:37:18.882 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-19T08:37:18.906 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:37:18.906 DEBUG:teuthology.orchestra.run.smithi139:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-19T08:37:18.951 INFO:tasks.ceph:Setting up mon nodes... 2024-09-19T08:37:18.951 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-19T08:37:18.952 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-19T08:37:18.997 INFO:teuthology.orchestra.run.smithi098.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-19T08:37:18.999 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-19T08:37:19.043 INFO:teuthology.orchestra.run.smithi139.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-19T08:37:19.055 INFO:tasks.ceph:Setting up mds nodes... 2024-09-19T08:37:19.056 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-19T08:37:19.096 INFO:teuthology.orchestra.run.smithi098.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-19T08:37:19.100 DEBUG:teuthology.orchestra.run.smithi098:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-19T08:37:19.154 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-19T08:37:19.235 INFO:teuthology.orchestra.run.smithi098.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-19T08:37:19.237 DEBUG:teuthology.orchestra.run.smithi098:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-19T08:37:19.290 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-19T08:37:19.336 INFO:teuthology.orchestra.run.smithi139.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-19T08:37:19.339 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-19T08:37:19.395 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-19T08:37:19.479 INFO:teuthology.orchestra.run.smithi139.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-19T08:37:19.482 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-19T08:37:19.539 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-19T08:37:19.539 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-19T08:37:19.577 INFO:teuthology.orchestra.run.smithi098.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-19T08:37:19.588 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-19T08:37:19.663 INFO:teuthology.orchestra.run.smithi098.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-19T08:37:19.674 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-19T08:37:19.710 INFO:teuthology.orchestra.run.smithi139.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-19T08:37:19.721 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-19T08:37:19.794 INFO:teuthology.orchestra.run.smithi139.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-19T08:37:19.805 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-19T08:37:19.805 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi098.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi139.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-19T08:37:19.805 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-19T08:37:19.819 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-19T08:37:19.819 INFO:tasks.ceph:role: osd.0 2024-09-19T08:37:19.819 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:37:19.819 DEBUG:teuthology.orchestra.run.smithi098:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-19T08:37:19.884 INFO:teuthology.orchestra.run.smithi098.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T08:37:19.884 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T08:37:19.884 INFO:teuthology.orchestra.run.smithi098.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T08:37:19.885 INFO:teuthology.orchestra.run.smithi098.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-19T08:37:19.885 INFO:teuthology.orchestra.run.smithi098.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T08:37:19.885 INFO:teuthology.orchestra.run.smithi098.stdout: = sunit=0 swidth=0 blks 2024-09-19T08:37:19.885 INFO:teuthology.orchestra.run.smithi098.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T08:37:19.885 INFO:teuthology.orchestra.run.smithi098.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-19T08:37:19.885 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T08:37:19.886 INFO:teuthology.orchestra.run.smithi098.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T08:37:20.340 INFO:teuthology.orchestra.run.smithi098.stdout:Discarding blocks...Done. 2024-09-19T08:37:20.341 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi098.front.sepia.ceph.com -o noatime 2024-09-19T08:37:20.341 DEBUG:teuthology.orchestra.run.smithi098:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-19T08:37:20.453 DEBUG:teuthology.orchestra.run.smithi098:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-19T08:37:20.463 INFO:teuthology.orchestra.run.smithi098.stderr:sudo: /sbin/restorecon: command not found 2024-09-19T08:37:20.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:37:20.464 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-19T08:37:20.518 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-19T08:37:20.519 INFO:tasks.ceph:role: osd.1 2024-09-19T08:37:20.519 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:37:20.519 DEBUG:teuthology.orchestra.run.smithi098:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-19T08:37:20.582 INFO:teuthology.orchestra.run.smithi098.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T08:37:20.582 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T08:37:20.582 INFO:teuthology.orchestra.run.smithi098.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T08:37:20.582 INFO:teuthology.orchestra.run.smithi098.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-19T08:37:20.582 INFO:teuthology.orchestra.run.smithi098.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T08:37:20.582 INFO:teuthology.orchestra.run.smithi098.stdout: = sunit=0 swidth=0 blks 2024-09-19T08:37:20.582 INFO:teuthology.orchestra.run.smithi098.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T08:37:20.582 INFO:teuthology.orchestra.run.smithi098.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-19T08:37:20.582 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T08:37:20.582 INFO:teuthology.orchestra.run.smithi098.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T08:37:21.050 INFO:teuthology.orchestra.run.smithi098.stdout:Discarding blocks...Done. 2024-09-19T08:37:21.052 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi098.front.sepia.ceph.com -o noatime 2024-09-19T08:37:21.052 DEBUG:teuthology.orchestra.run.smithi098:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-19T08:37:21.197 DEBUG:teuthology.orchestra.run.smithi098:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-19T08:37:21.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:37:21.209 INFO:teuthology.orchestra.run.smithi098.stderr:sudo: /sbin/restorecon: command not found 2024-09-19T08:37:21.209 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-19T08:37:21.264 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-19T08:37:21.264 INFO:tasks.ceph:role: osd.2 2024-09-19T08:37:21.264 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:37:21.264 DEBUG:teuthology.orchestra.run.smithi098:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-19T08:37:21.328 INFO:teuthology.orchestra.run.smithi098.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T08:37:21.328 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T08:37:21.328 INFO:teuthology.orchestra.run.smithi098.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T08:37:21.328 INFO:teuthology.orchestra.run.smithi098.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-19T08:37:21.328 INFO:teuthology.orchestra.run.smithi098.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T08:37:21.328 INFO:teuthology.orchestra.run.smithi098.stdout: = sunit=0 swidth=0 blks 2024-09-19T08:37:21.328 INFO:teuthology.orchestra.run.smithi098.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T08:37:21.328 INFO:teuthology.orchestra.run.smithi098.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-19T08:37:21.328 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T08:37:21.328 INFO:teuthology.orchestra.run.smithi098.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T08:37:21.812 INFO:teuthology.orchestra.run.smithi098.stdout:Discarding blocks...Done. 2024-09-19T08:37:21.814 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi098.front.sepia.ceph.com -o noatime 2024-09-19T08:37:21.814 DEBUG:teuthology.orchestra.run.smithi098:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-19T08:37:21.961 DEBUG:teuthology.orchestra.run.smithi098:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-19T08:37:21.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:37:21.973 INFO:teuthology.orchestra.run.smithi098.stderr:sudo: /sbin/restorecon: command not found 2024-09-19T08:37:21.973 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-19T08:37:22.028 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-19T08:37:22.028 INFO:tasks.ceph:role: osd.3 2024-09-19T08:37:22.028 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:37:22.028 DEBUG:teuthology.orchestra.run.smithi098:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-19T08:37:22.092 INFO:teuthology.orchestra.run.smithi098.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T08:37:22.092 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T08:37:22.092 INFO:teuthology.orchestra.run.smithi098.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T08:37:22.092 INFO:teuthology.orchestra.run.smithi098.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-19T08:37:22.092 INFO:teuthology.orchestra.run.smithi098.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T08:37:22.092 INFO:teuthology.orchestra.run.smithi098.stdout: = sunit=0 swidth=0 blks 2024-09-19T08:37:22.092 INFO:teuthology.orchestra.run.smithi098.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T08:37:22.092 INFO:teuthology.orchestra.run.smithi098.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-19T08:37:22.092 INFO:teuthology.orchestra.run.smithi098.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T08:37:22.093 INFO:teuthology.orchestra.run.smithi098.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T08:37:22.512 INFO:teuthology.orchestra.run.smithi098.stdout:Discarding blocks...Done. 2024-09-19T08:37:22.514 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi098.front.sepia.ceph.com -o noatime 2024-09-19T08:37:22.514 DEBUG:teuthology.orchestra.run.smithi098:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-19T08:37:22.612 DEBUG:teuthology.orchestra.run.smithi098:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-19T08:37:22.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:37:22.625 INFO:teuthology.orchestra.run.smithi098.stderr:sudo: /sbin/restorecon: command not found 2024-09-19T08:37:22.625 DEBUG:teuthology.orchestra.run.smithi098:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T08:37:22.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:22.730+0000 7fd3f5e437c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-19T08:37:22.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:22.734+0000 7fd3f5e437c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-19T08:37:22.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:22.734+0000 7fd3f5e437c0 -1 bdev(0x55878a2ef800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-19T08:37:22.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:22.734+0000 7fd3f5e437c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-19T08:37:26.099 DEBUG:teuthology.orchestra.run.smithi098:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-19T08:37:26.112 DEBUG:teuthology.orchestra.run.smithi098:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T08:37:26.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:26.190+0000 7fc84825e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-19T08:37:26.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:26.190+0000 7fc84825e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-19T08:37:26.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:26.190+0000 7fc84825e7c0 -1 bdev(0x5649a5691800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-19T08:37:26.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:26.190+0000 7fc84825e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-19T08:37:29.318 DEBUG:teuthology.orchestra.run.smithi098:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-19T08:37:29.372 DEBUG:teuthology.orchestra.run.smithi098:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T08:37:29.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:29.450+0000 7f7bc05997c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-19T08:37:29.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:29.450+0000 7f7bc05997c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-19T08:37:29.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:29.450+0000 7f7bc05997c0 -1 bdev(0x55e620e7f800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-19T08:37:29.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:29.450+0000 7f7bc05997c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-19T08:37:32.619 DEBUG:teuthology.orchestra.run.smithi098:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-19T08:37:32.676 DEBUG:teuthology.orchestra.run.smithi098:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T08:37:32.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:32.754+0000 7f929b6847c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-19T08:37:32.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:32.754+0000 7f929b6847c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-19T08:37:32.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:32.754+0000 7f929b6847c0 -1 bdev(0x55748b4d3800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-19T08:37:32.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:32.754+0000 7f929b6847c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-19T08:37:35.892 DEBUG:teuthology.orchestra.run.smithi098:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-19T08:37:35.948 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-19T08:37:35.963 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-19T08:37:35.964 INFO:tasks.ceph:role: osd.4 2024-09-19T08:37:35.964 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:37:35.964 DEBUG:teuthology.orchestra.run.smithi139:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-19T08:37:36.027 INFO:teuthology.orchestra.run.smithi139.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T08:37:36.027 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T08:37:36.027 INFO:teuthology.orchestra.run.smithi139.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T08:37:36.027 INFO:teuthology.orchestra.run.smithi139.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-19T08:37:36.027 INFO:teuthology.orchestra.run.smithi139.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T08:37:36.028 INFO:teuthology.orchestra.run.smithi139.stdout: = sunit=0 swidth=0 blks 2024-09-19T08:37:36.028 INFO:teuthology.orchestra.run.smithi139.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T08:37:36.028 INFO:teuthology.orchestra.run.smithi139.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-19T08:37:36.028 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T08:37:36.028 INFO:teuthology.orchestra.run.smithi139.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T08:37:36.475 INFO:teuthology.orchestra.run.smithi139.stdout:Discarding blocks...Done. 2024-09-19T08:37:36.476 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi139.front.sepia.ceph.com -o noatime 2024-09-19T08:37:36.477 DEBUG:teuthology.orchestra.run.smithi139:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-19T08:37:36.565 DEBUG:teuthology.orchestra.run.smithi139:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-19T08:37:36.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:37:36.575 INFO:teuthology.orchestra.run.smithi139.stderr:sudo: /sbin/restorecon: command not found 2024-09-19T08:37:36.576 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-19T08:37:36.628 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-19T08:37:36.628 INFO:tasks.ceph:role: osd.5 2024-09-19T08:37:36.628 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:37:36.628 DEBUG:teuthology.orchestra.run.smithi139:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-19T08:37:36.689 INFO:teuthology.orchestra.run.smithi139.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T08:37:36.690 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T08:37:36.690 INFO:teuthology.orchestra.run.smithi139.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T08:37:36.690 INFO:teuthology.orchestra.run.smithi139.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-19T08:37:36.690 INFO:teuthology.orchestra.run.smithi139.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T08:37:36.690 INFO:teuthology.orchestra.run.smithi139.stdout: = sunit=0 swidth=0 blks 2024-09-19T08:37:36.690 INFO:teuthology.orchestra.run.smithi139.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T08:37:36.690 INFO:teuthology.orchestra.run.smithi139.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-19T08:37:36.690 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T08:37:36.691 INFO:teuthology.orchestra.run.smithi139.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T08:37:37.170 INFO:teuthology.orchestra.run.smithi139.stdout:Discarding blocks...Done. 2024-09-19T08:37:37.172 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi139.front.sepia.ceph.com -o noatime 2024-09-19T08:37:37.173 DEBUG:teuthology.orchestra.run.smithi139:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-19T08:37:37.317 DEBUG:teuthology.orchestra.run.smithi139:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-19T08:37:37.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:37:37.329 INFO:teuthology.orchestra.run.smithi139.stderr:sudo: /sbin/restorecon: command not found 2024-09-19T08:37:37.330 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-19T08:37:37.383 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-19T08:37:37.383 INFO:tasks.ceph:role: osd.6 2024-09-19T08:37:37.383 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:37:37.384 DEBUG:teuthology.orchestra.run.smithi139:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-19T08:37:37.446 INFO:teuthology.orchestra.run.smithi139.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T08:37:37.446 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T08:37:37.446 INFO:teuthology.orchestra.run.smithi139.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T08:37:37.446 INFO:teuthology.orchestra.run.smithi139.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-19T08:37:37.446 INFO:teuthology.orchestra.run.smithi139.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T08:37:37.446 INFO:teuthology.orchestra.run.smithi139.stdout: = sunit=0 swidth=0 blks 2024-09-19T08:37:37.446 INFO:teuthology.orchestra.run.smithi139.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T08:37:37.446 INFO:teuthology.orchestra.run.smithi139.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-19T08:37:37.446 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T08:37:37.447 INFO:teuthology.orchestra.run.smithi139.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T08:37:37.921 INFO:teuthology.orchestra.run.smithi139.stdout:Discarding blocks...Done. 2024-09-19T08:37:37.923 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi139.front.sepia.ceph.com -o noatime 2024-09-19T08:37:37.923 DEBUG:teuthology.orchestra.run.smithi139:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-19T08:37:38.072 DEBUG:teuthology.orchestra.run.smithi139:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-19T08:37:38.083 INFO:teuthology.orchestra.run.smithi139.stderr:sudo: /sbin/restorecon: command not found 2024-09-19T08:37:38.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:37:38.085 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-19T08:37:38.137 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-19T08:37:38.138 INFO:tasks.ceph:role: osd.7 2024-09-19T08:37:38.138 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:37:38.138 DEBUG:teuthology.orchestra.run.smithi139:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-19T08:37:38.199 INFO:teuthology.orchestra.run.smithi139.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T08:37:38.199 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T08:37:38.199 INFO:teuthology.orchestra.run.smithi139.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T08:37:38.199 INFO:teuthology.orchestra.run.smithi139.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-19T08:37:38.200 INFO:teuthology.orchestra.run.smithi139.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T08:37:38.200 INFO:teuthology.orchestra.run.smithi139.stdout: = sunit=0 swidth=0 blks 2024-09-19T08:37:38.200 INFO:teuthology.orchestra.run.smithi139.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T08:37:38.200 INFO:teuthology.orchestra.run.smithi139.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-19T08:37:38.200 INFO:teuthology.orchestra.run.smithi139.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T08:37:38.200 INFO:teuthology.orchestra.run.smithi139.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T08:37:38.625 INFO:teuthology.orchestra.run.smithi139.stdout:Discarding blocks...Done. 2024-09-19T08:37:38.627 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi139.front.sepia.ceph.com -o noatime 2024-09-19T08:37:38.627 DEBUG:teuthology.orchestra.run.smithi139:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-19T08:37:38.725 DEBUG:teuthology.orchestra.run.smithi139:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-19T08:37:38.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:37:38.735 INFO:teuthology.orchestra.run.smithi139.stderr:sudo: /sbin/restorecon: command not found 2024-09-19T08:37:38.735 DEBUG:teuthology.orchestra.run.smithi139:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T08:37:38.847 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:38.845+0000 7fc64e01c7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-19T08:37:38.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:38.845+0000 7fc64e01c7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-19T08:37:38.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:38.845+0000 7fc64e01c7c0 -1 bdev(0x55a684b0f800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-19T08:37:38.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:38.849+0000 7fc64e01c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-19T08:37:42.211 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-19T08:37:42.224 DEBUG:teuthology.orchestra.run.smithi139:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T08:37:42.307 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:42.305+0000 7f9d22c747c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-19T08:37:42.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:42.305+0000 7f9d22c747c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-19T08:37:42.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:42.305+0000 7f9d22c747c0 -1 bdev(0x55c6a267f800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-19T08:37:42.308 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:42.305+0000 7f9d22c747c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-19T08:37:45.427 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-19T08:37:45.483 DEBUG:teuthology.orchestra.run.smithi139:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T08:37:45.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:45.561+0000 7fbd30eaf7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-19T08:37:45.566 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:45.565+0000 7fbd30eaf7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-19T08:37:45.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:45.565+0000 7fbd30eaf7c0 -1 bdev(0x562d83785800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-19T08:37:45.567 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:45.565+0000 7fbd30eaf7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-19T08:37:46.828 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:37:46.831 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:37:48.765 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-19T08:37:48.779 DEBUG:teuthology.orchestra.run.smithi139:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T08:37:48.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:48.857+0000 7fe492d857c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-19T08:37:48.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:48.857+0000 7fe492d857c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-19T08:37:48.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:48.857+0000 7fe492d857c0 -1 bdev(0x55bb0e75b800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-19T08:37:48.862 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:48.857+0000 7fe492d857c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-19T08:37:52.235 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-19T08:37:52.290 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-19T08:37:52.290 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:37:52.291 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-19T08:37:52.305 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:37:52.305 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-19T08:37:52.358 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:37:52.359 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-19T08:37:52.415 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:37:52.415 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-19T08:37:52.470 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:37:52.471 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-19T08:37:52.522 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:37:52.523 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-19T08:37:52.579 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:37:52.579 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-19T08:37:52.634 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:37:52.634 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-19T08:37:52.648 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:37:52.648 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-19T08:37:52.702 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:37:52.702 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-19T08:37:52.757 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:37:52.758 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-19T08:37:52.813 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:37:52.813 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-19T08:37:52.866 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:37:52.866 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-19T08:37:52.921 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:37:52.921 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-19T08:37:52.973 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:37:52.973 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-19T08:37:52.979 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:37:52.979 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-19T08:37:53.028 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:37:53.028 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-19T08:37:53.033 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:37:53.033 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-19T08:37:53.079 INFO:tasks.ceph:Adding keys to all mons... 2024-09-19T08:37:53.079 DEBUG:teuthology.orchestra.run.smithi098:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-19T08:37:53.081 DEBUG:teuthology.orchestra.run.smithi139:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-19T08:37:53.125 INFO:teuthology.orchestra.run.smithi098.stdout:[mgr.y] 2024-09-19T08:37:53.125 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQC+4utmU+ZrOxAAsvcwINH6qyxfN7N+rDrkqg== 2024-09-19T08:37:53.125 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.a] 2024-09-19T08:37:53.125 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQC/4utmuujGBRAAa2k/kW68NJI6nUKx+W+G2Q== 2024-09-19T08:37:53.125 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.c] 2024-09-19T08:37:53.125 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQC/4utmcQcDDhAAF0QJMlljMH+DiHfWW+/Sog== 2024-09-19T08:37:53.125 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.0] 2024-09-19T08:37:53.125 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQDC4utm9PjNKxAAa6SG9AE4smPiu4azRAFKvg== 2024-09-19T08:37:53.125 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.1] 2024-09-19T08:37:53.126 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQDG4utmHy6UCxAAN6hpG4DJEp81iG/KoloTvQ== 2024-09-19T08:37:53.126 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.2] 2024-09-19T08:37:53.126 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQDJ4utmUVP4GhAAJNAL30sEAseoDG2fE6YbZQ== 2024-09-19T08:37:53.126 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.3] 2024-09-19T08:37:53.126 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQDM4utmTKgoLRAAiCrPRLML+wvjOt6RhKay3w== 2024-09-19T08:37:53.126 INFO:teuthology.orchestra.run.smithi098.stdout:[mgr.x] 2024-09-19T08:37:53.126 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQC/4utmbKuyAhAA0/XjHUIPOIV/YvNEhB8rEQ== 2024-09-19T08:37:53.126 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.b] 2024-09-19T08:37:53.126 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQC/4utmFoojFBAAaOZPpa41fdjBPEn3CGsJnA== 2024-09-19T08:37:53.126 INFO:teuthology.orchestra.run.smithi098.stdout:[mds.d] 2024-09-19T08:37:53.126 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQC/4utmwWmqHBAAf15VcrdeHf8AzQjGsN9LoA== 2024-09-19T08:37:53.126 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.4] 2024-09-19T08:37:53.126 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQDS4utmAf6bMhAAI3uENuFoXLicUeuspgFjYA== 2024-09-19T08:37:53.127 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.5] 2024-09-19T08:37:53.127 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQDW4utmRjxwEhAAF1A2GtrZXVx9BlmjHjhUDQ== 2024-09-19T08:37:53.127 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.6] 2024-09-19T08:37:53.127 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQDZ4utmSk/XIRAAROSfzGL7smR88ZkDSx5jAg== 2024-09-19T08:37:53.127 INFO:teuthology.orchestra.run.smithi098.stdout:[osd.7] 2024-09-19T08:37:53.127 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQDc4utmXfpbMxAAqEM4I/ACFvfu/sMA8bMcrQ== 2024-09-19T08:37:53.127 INFO:teuthology.orchestra.run.smithi098.stdout:[client.0] 2024-09-19T08:37:53.127 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQC/4utm0vNpIhAA4S1E8dFZ0+ibmSKhmZ+bIA== 2024-09-19T08:37:53.127 INFO:teuthology.orchestra.run.smithi098.stdout:[client.2] 2024-09-19T08:37:53.127 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQC/4utmteOQJxAAtTF2Py21ZE2WRQjeQSrkKQ== 2024-09-19T08:37:53.127 INFO:teuthology.orchestra.run.smithi098.stdout:[client.1] 2024-09-19T08:37:53.127 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQC/4utmx7xyKhAAC84lykPEb01MiypXKQVREQ== 2024-09-19T08:37:53.127 INFO:teuthology.orchestra.run.smithi098.stdout:[client.3] 2024-09-19T08:37:53.127 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQC/4utmKN92LxAAMqraZOWRT6IIfx5AuDWS2g== 2024-09-19T08:37:53.133 INFO:teuthology.orchestra.run.smithi139.stdout:[mgr.y] 2024-09-19T08:37:53.133 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQC+4utmU+ZrOxAAsvcwINH6qyxfN7N+rDrkqg== 2024-09-19T08:37:53.133 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.a] 2024-09-19T08:37:53.133 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQC/4utmuujGBRAAa2k/kW68NJI6nUKx+W+G2Q== 2024-09-19T08:37:53.133 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.c] 2024-09-19T08:37:53.133 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQC/4utmcQcDDhAAF0QJMlljMH+DiHfWW+/Sog== 2024-09-19T08:37:53.133 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.0] 2024-09-19T08:37:53.133 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQDC4utm9PjNKxAAa6SG9AE4smPiu4azRAFKvg== 2024-09-19T08:37:53.134 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.1] 2024-09-19T08:37:53.134 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQDG4utmHy6UCxAAN6hpG4DJEp81iG/KoloTvQ== 2024-09-19T08:37:53.134 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.2] 2024-09-19T08:37:53.134 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQDJ4utmUVP4GhAAJNAL30sEAseoDG2fE6YbZQ== 2024-09-19T08:37:53.134 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.3] 2024-09-19T08:37:53.134 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQDM4utmTKgoLRAAiCrPRLML+wvjOt6RhKay3w== 2024-09-19T08:37:53.134 INFO:teuthology.orchestra.run.smithi139.stdout:[mgr.x] 2024-09-19T08:37:53.134 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQC/4utmbKuyAhAA0/XjHUIPOIV/YvNEhB8rEQ== 2024-09-19T08:37:53.134 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.b] 2024-09-19T08:37:53.134 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQC/4utmFoojFBAAaOZPpa41fdjBPEn3CGsJnA== 2024-09-19T08:37:53.134 INFO:teuthology.orchestra.run.smithi139.stdout:[mds.d] 2024-09-19T08:37:53.134 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQC/4utmwWmqHBAAf15VcrdeHf8AzQjGsN9LoA== 2024-09-19T08:37:53.134 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.4] 2024-09-19T08:37:53.134 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQDS4utmAf6bMhAAI3uENuFoXLicUeuspgFjYA== 2024-09-19T08:37:53.135 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.5] 2024-09-19T08:37:53.135 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQDW4utmRjxwEhAAF1A2GtrZXVx9BlmjHjhUDQ== 2024-09-19T08:37:53.135 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.6] 2024-09-19T08:37:53.135 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQDZ4utmSk/XIRAAROSfzGL7smR88ZkDSx5jAg== 2024-09-19T08:37:53.135 INFO:teuthology.orchestra.run.smithi139.stdout:[osd.7] 2024-09-19T08:37:53.135 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQDc4utmXfpbMxAAqEM4I/ACFvfu/sMA8bMcrQ== 2024-09-19T08:37:53.135 INFO:teuthology.orchestra.run.smithi139.stdout:[client.0] 2024-09-19T08:37:53.135 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQC/4utm0vNpIhAA4S1E8dFZ0+ibmSKhmZ+bIA== 2024-09-19T08:37:53.135 INFO:teuthology.orchestra.run.smithi139.stdout:[client.2] 2024-09-19T08:37:53.135 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQC/4utmteOQJxAAtTF2Py21ZE2WRQjeQSrkKQ== 2024-09-19T08:37:53.135 INFO:teuthology.orchestra.run.smithi139.stdout:[client.1] 2024-09-19T08:37:53.135 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQC/4utmx7xyKhAAC84lykPEb01MiypXKQVREQ== 2024-09-19T08:37:53.135 INFO:teuthology.orchestra.run.smithi139.stdout:[client.3] 2024-09-19T08:37:53.135 INFO:teuthology.orchestra.run.smithi139.stdout: key = AQC/4utmKN92LxAAMqraZOWRT6IIfx5AuDWS2g== 2024-09-19T08:37:53.136 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-19T08:37:53.173 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-19T08:37:53.222 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-19T08:37:53.224 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-19T08:37:53.317 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-19T08:37:53.319 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-19T08:37:53.402 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T08:37:53.405 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T08:37:53.485 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T08:37:53.488 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T08:37:53.568 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T08:37:53.572 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T08:37:53.654 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T08:37:53.656 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T08:37:53.738 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-19T08:37:53.745 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-19T08:37:53.823 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-19T08:37:53.829 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-19T08:37:53.914 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-19T08:37:53.921 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-19T08:37:53.964 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T08:37:53.967 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T08:37:54.050 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T08:37:54.052 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T08:37:54.140 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T08:37:54.143 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T08:37:54.186 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T08:37:54.229 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T08:37:54.271 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-19T08:37:54.317 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-19T08:37:54.360 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-19T08:37:54.401 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-19T08:37:54.444 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-19T08:37:54.484 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-19T08:37:54.529 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-19T08:37:54.531 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-19T08:37:54.611 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-19T08:37:54.611 DEBUG:teuthology.orchestra.run.smithi098:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-19T08:37:54.626 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-19T08:37:54.716 DEBUG:teuthology.orchestra.run.smithi098:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-19T08:37:54.726 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-19T08:37:54.740 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-19T08:37:54.835 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-19T08:37:54.847 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-19T08:37:54.901 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-19T08:37:54.994 DEBUG:teuthology.orchestra.run.smithi139:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-19T08:37:55.004 DEBUG:teuthology.orchestra.run.smithi098:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-19T08:37:55.006 DEBUG:teuthology.orchestra.run.smithi139:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-19T08:37:55.054 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-19T08:37:55.054 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-19T08:37:55.054 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-19T08:37:55.057 INFO:tasks.ceph.mon.a:Started 2024-09-19T08:37:55.057 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-19T08:37:55.057 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-19T08:37:55.100 INFO:tasks.ceph.mon.b:Started 2024-09-19T08:37:55.100 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-19T08:37:55.100 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-19T08:37:55.102 INFO:tasks.ceph.mon.c:Started 2024-09-19T08:37:55.103 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-19T08:37:55.103 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-19T08:37:55.103 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-19T08:37:55.105 INFO:tasks.ceph.mgr.y:Started 2024-09-19T08:37:55.106 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-19T08:37:55.106 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-19T08:37:55.109 INFO:tasks.ceph.mgr.x:Started 2024-09-19T08:37:55.109 INFO:tasks.ceph:Setting crush tunables to default 2024-09-19T08:37:55.109 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --cluster ceph osd crush tunables default 2024-09-19T08:37:55.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.222+0000 7f331dedf640 1 Processor -- start 2024-09-19T08:37:55.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.222+0000 7f331dedf640 1 -- start start 2024-09-19T08:37:55.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.226+0000 7f331dedf640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f331806f9f0 0x7f33180b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:55.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.226+0000 7f331dedf640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f331815c9f0 0x7f331815ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:55.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.226+0000 7f331dedf640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f331815d760 con 0x7f3318159f60 2024-09-19T08:37:55.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.226+0000 7f331dedf640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f331815a340 con 0x7f331806f9f0 2024-09-19T08:37:55.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.226+0000 7f331dedf640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f33180b5410 con 0x7f331815c9f0 2024-09-19T08:37:55.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.226+0000 7f3316ffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f331806f9f0 0x7f33180b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:55.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.226+0000 7f3316ffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f331806f9f0 0x7f33180b61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59408/0 (socket says 172.21.15.98:59408) 2024-09-19T08:37:55.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.226+0000 7f3316ffd640 1 -- 172.21.15.98:0/1876463523 learned_addr learned my addr 172.21.15.98:0/1876463523 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:37:55.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.226+0000 7f3316ffd640 1 -- 172.21.15.98:0/1876463523 >> v2:172.21.15.98:3300/0 conn(0x7f331806f9f0 msgr2=0x7f33180b61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-19T08:37:55.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.226+0000 7f3316ffd640 1 -- 172.21.15.98:0/1876463523 >> v2:172.21.15.98:3300/0 conn(0x7f331806f9f0 msgr2=0x7f33180b61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-19T08:37:55.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.226+0000 7f3316ffd640 1 --2- 172.21.15.98:0/1876463523 >> v2:172.21.15.98:3300/0 conn(0x7f331806f9f0 0x7f33180b61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T08:37:55.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.226+0000 7f3316ffd640 1 --2- 172.21.15.98:0/1876463523 >> v2:172.21.15.98:3300/0 conn(0x7f331806f9f0 0x7f33180b61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T08:37:55.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.226+0000 7f3317fff640 1 --2- 172.21.15.98:0/1876463523 >> v2:172.21.15.139:3300/0 conn(0x7f331815c9f0 0x7f331815ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:55.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.226+0000 7f3317fff640 1 -- 172.21.15.98:0/1876463523 >> v2:172.21.15.139:3300/0 conn(0x7f331815c9f0 msgr2=0x7f331815ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-19T08:37:55.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.226+0000 7f3317fff640 1 -- 172.21.15.98:0/1876463523 >> v2:172.21.15.139:3300/0 conn(0x7f331815c9f0 msgr2=0x7f331815ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-19T08:37:55.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.226+0000 7f3317fff640 1 --2- 172.21.15.98:0/1876463523 >> v2:172.21.15.139:3300/0 conn(0x7f331815c9f0 0x7f331815ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T08:37:55.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.226+0000 7f3317fff640 1 --2- 172.21.15.98:0/1876463523 >> v2:172.21.15.139:3300/0 conn(0x7f331815c9f0 0x7f331815ce20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T08:37:55.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.430+0000 7f3316ffd640 1 --2- 172.21.15.98:0/1876463523 >> v2:172.21.15.98:3300/0 conn(0x7f331806f9f0 0x7f33180b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:55.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.430+0000 7f3317fff640 1 --2- 172.21.15.98:0/1876463523 >> v2:172.21.15.139:3300/0 conn(0x7f331815c9f0 0x7f331815ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:55.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.430+0000 7f3316ffd640 1 -- 172.21.15.98:0/1876463523 >> v2:172.21.15.98:3300/0 conn(0x7f331806f9f0 msgr2=0x7f33180b61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-19T08:37:55.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.430+0000 7f3316ffd640 1 -- 172.21.15.98:0/1876463523 >> v2:172.21.15.98:3300/0 conn(0x7f331806f9f0 msgr2=0x7f33180b61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-19T08:37:55.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.430+0000 7f3316ffd640 1 --2- 172.21.15.98:0/1876463523 >> v2:172.21.15.98:3300/0 conn(0x7f331806f9f0 0x7f33180b61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T08:37:55.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.430+0000 7f3316ffd640 1 --2- 172.21.15.98:0/1876463523 >> v2:172.21.15.98:3300/0 conn(0x7f331806f9f0 0x7f33180b61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T08:37:55.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.430+0000 7f3317fff640 1 -- 172.21.15.98:0/1876463523 >> v2:172.21.15.139:3300/0 conn(0x7f331815c9f0 msgr2=0x7f331815ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-19T08:37:55.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.430+0000 7f3317fff640 1 -- 172.21.15.98:0/1876463523 >> v2:172.21.15.139:3300/0 conn(0x7f331815c9f0 msgr2=0x7f331815ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-19T08:37:55.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.430+0000 7f3317fff640 1 --2- 172.21.15.98:0/1876463523 >> v2:172.21.15.139:3300/0 conn(0x7f331815c9f0 0x7f331815ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T08:37:55.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.430+0000 7f3317fff640 1 --2- 172.21.15.98:0/1876463523 >> v2:172.21.15.139:3300/0 conn(0x7f331815c9f0 0x7f331815ce20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T08:37:55.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.470+0000 7f33167fc640 1 -- 172.21.15.98:0/1876463523 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 203520791 0 0) 0x7f331815d760 con 0x7f3318159f60 2024-09-19T08:37:55.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.470+0000 7f33167fc640 1 -- 172.21.15.98:0/1876463523 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3304003530 con 0x7f3318159f60 2024-09-19T08:37:55.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.470+0000 7f33167fc640 1 -- 172.21.15.98:0/1876463523 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 133246139 0 0) 0x7f3304003530 con 0x7f3318159f60 2024-09-19T08:37:55.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.470+0000 7f33167fc640 1 -- 172.21.15.98:0/1876463523 >> v2:172.21.15.139:3300/0 conn(0x7f331815c9f0 msgr2=0x7f331815ce20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T08:37:55.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.470+0000 7f33167fc640 1 --2- 172.21.15.98:0/1876463523 >> v2:172.21.15.139:3300/0 conn(0x7f331815c9f0 0x7f331815ce20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:55.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.470+0000 7f33167fc640 1 -- 172.21.15.98:0/1876463523 >> v2:172.21.15.98:3300/0 conn(0x7f331806f9f0 msgr2=0x7f33180b61d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T08:37:55.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.470+0000 7f33167fc640 1 --2- 172.21.15.98:0/1876463523 >> v2:172.21.15.98:3300/0 conn(0x7f331806f9f0 0x7f33180b61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:55.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.470+0000 7f33167fc640 1 -- 172.21.15.98:0/1876463523 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33180b56f0 con 0x7f3318159f60 2024-09-19T08:37:55.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.470+0000 7f33167fc640 1 -- 172.21.15.98:0/1876463523 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f3300001680 con 0x7f3318159f60 2024-09-19T08:37:55.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.470+0000 7f33167fc640 1 -- 172.21.15.98:0/1876463523 >> v1:172.21.15.98:6789/0 conn(0x7f3318159f60 legacy=0x7f331806f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:55.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.470+0000 7f33167fc640 1 --2- 172.21.15.98:0/1876463523 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3304003a10 0x7f3304003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:55.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.470+0000 7f33167fc640 1 --2- 172.21.15.98:0/1876463523 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3304004810 0x7f3304004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:55.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.470+0000 7f33167fc640 1 --2- 172.21.15.98:0/1876463523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33040051d0 0x7f330400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:55.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.470+0000 7f33167fc640 1 -- 172.21.15.98:0/1876463523 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f331815a340 con 0x7f3304004810 2024-09-19T08:37:55.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.470+0000 7f33167fc640 1 -- 172.21.15.98:0/1876463523 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f33180b5410 con 0x7f33040051d0 2024-09-19T08:37:55.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.470+0000 7f33167fc640 1 -- 172.21.15.98:0/1876463523 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f330400c120 con 0x7f3304003a10 2024-09-19T08:37:55.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.474+0000 7f3316ffd640 1 --2- 172.21.15.98:0/1876463523 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3304004810 0x7f3304004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:55.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.474+0000 7f33177fe640 1 --2- 172.21.15.98:0/1876463523 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3304003a10 0x7f3304003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:55.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.474+0000 7f3317fff640 1 --2- 172.21.15.98:0/1876463523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33040051d0 0x7f330400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:55.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.474+0000 7f3317fff640 1 -- 172.21.15.98:0/1876463523 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3304003a10 msgr2=0x7f3304003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:37:55.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.474+0000 7f3317fff640 1 --2- 172.21.15.98:0/1876463523 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3304003a10 0x7f3304003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:55.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.474+0000 7f3317fff640 1 -- 172.21.15.98:0/1876463523 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3304004810 msgr2=0x7f3304004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:37:55.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.474+0000 7f3317fff640 1 --2- 172.21.15.98:0/1876463523 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3304004810 0x7f3304004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:55.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.474+0000 7f3317fff640 1 -- 172.21.15.98:0/1876463523 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3304003530 con 0x7f33040051d0 2024-09-19T08:37:55.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.478+0000 7f3317fff640 1 -- 172.21.15.98:0/1876463523 reap_dead start 2024-09-19T08:37:55.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.478+0000 7f3317fff640 1 --2- 172.21.15.98:0/1876463523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33040051d0 0x7f330400b9d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3310009a70 tx=0x7f331001d010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdb8e0fc72950e49 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:55.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.478+0000 7f33167fc640 1 -- 172.21.15.98:0/1876463523 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3310009640 con 0x7f33040051d0 2024-09-19T08:37:55.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.478+0000 7f33167fc640 1 -- 172.21.15.98:0/1876463523 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33100ac680 con 0x7f33040051d0 2024-09-19T08:37:55.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.478+0000 7f33167fc640 1 -- 172.21.15.98:0/1876463523 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33100aca70 con 0x7f33040051d0 2024-09-19T08:37:55.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.478+0000 7f331dedf640 1 -- 172.21.15.98:0/1876463523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33040051d0 msgr2=0x7f330400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:55.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.478+0000 7f331dedf640 1 --2- 172.21.15.98:0/1876463523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33040051d0 0x7f330400b9d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3310009a70 tx=0x7f331001d010 comp rx=0 tx=0).stop 2024-09-19T08:37:55.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.478+0000 7f331dedf640 1 -- 172.21.15.98:0/1876463523 shutdown_connections 2024-09-19T08:37:55.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.478+0000 7f331dedf640 1 --2- 172.21.15.98:0/1876463523 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33040051d0 0x7f330400b9d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:55.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.478+0000 7f331dedf640 1 -- 172.21.15.98:0/1876463523 >> 172.21.15.98:0/1876463523 conn(0x7f331806dea0 msgr2=0x7f331806e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:55.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.478+0000 7f331dedf640 1 -- 172.21.15.98:0/1876463523 shutdown_connections 2024-09-19T08:37:55.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.478+0000 7f331dedf640 1 -- 172.21.15.98:0/1876463523 wait complete. 2024-09-19T08:37:55.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.482+0000 7f331dedf640 1 Processor -- start 2024-09-19T08:37:55.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.482+0000 7f331dedf640 1 -- start start 2024-09-19T08:37:55.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.482+0000 7f331dedf640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33040051d0 0x7f3318142e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:55.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.482+0000 7f331dedf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3318150f40 0x7f33181538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:55.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.482+0000 7f331dedf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33181513c0 0x7f3318153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:55.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.482+0000 7f331dedf640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f33180b61d0 con 0x7f33181513c0 2024-09-19T08:37:55.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.482+0000 7f331dedf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f33180b5cb0 con 0x7f3318150f40 2024-09-19T08:37:55.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.482+0000 7f331dedf640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f33180b6350 con 0x7f33040051d0 2024-09-19T08:37:55.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.482+0000 7f3317fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33181513c0 0x7f3318153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:55.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.482+0000 7f3317fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33181513c0 0x7f3318153e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59458/0 (socket says 172.21.15.98:59458) 2024-09-19T08:37:55.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.482+0000 7f3317fff640 1 -- 172.21.15.98:0/2947274201 learned_addr learned my addr 172.21.15.98:0/2947274201 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:37:55.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.482+0000 7f33177fe640 1 --2- 172.21.15.98:0/2947274201 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33040051d0 0x7f3318142e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:55.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.482+0000 7f3316ffd640 1 --2- 172.21.15.98:0/2947274201 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3318150f40 0x7f33181538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:55.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.482+0000 7f3317fff640 1 -- 172.21.15.98:0/2947274201 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33040051d0 msgr2=0x7f3318142e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:55.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.482+0000 7f3317fff640 1 --2- 172.21.15.98:0/2947274201 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33040051d0 0x7f3318142e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:55.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.482+0000 7f3317fff640 1 -- 172.21.15.98:0/2947274201 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3318150f40 msgr2=0x7f33181538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:55.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.482+0000 7f3317fff640 1 --2- 172.21.15.98:0/2947274201 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3318150f40 0x7f33181538f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:55.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.482+0000 7f3317fff640 1 -- 172.21.15.98:0/2947274201 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33181523f0 con 0x7f33181513c0 2024-09-19T08:37:55.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.486+0000 7f3317fff640 1 --2- 172.21.15.98:0/2947274201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33181513c0 0x7f3318153e30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f331001d4f0 tx=0x7f3310019db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:55.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.486+0000 7f330ffff640 1 -- 172.21.15.98:0/2947274201 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33100ace40 con 0x7f33181513c0 2024-09-19T08:37:55.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.486+0000 7f330ffff640 1 -- 172.21.15.98:0/2947274201 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33100bd070 con 0x7f33181513c0 2024-09-19T08:37:55.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.486+0000 7f331dedf640 1 -- 172.21.15.98:0/2947274201 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33181526d0 con 0x7f33181513c0 2024-09-19T08:37:55.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.486+0000 7f330ffff640 1 -- 172.21.15.98:0/2947274201 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3310009610 con 0x7f33181513c0 2024-09-19T08:37:55.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.486+0000 7f331dedf640 1 -- 172.21.15.98:0/2947274201 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3318152c90 con 0x7f33181513c0 2024-09-19T08:37:55.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.486+0000 7f330ffff640 1 -- 172.21.15.98:0/2947274201 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f33100b6660 con 0x7f33181513c0 2024-09-19T08:37:55.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.486+0000 7f330ffff640 1 -- 172.21.15.98:0/2947274201 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f33100c26e0 con 0x7f33181513c0 2024-09-19T08:37:55.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.486+0000 7f331dedf640 1 -- 172.21.15.98:0/2947274201 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f331806f2e0 con 0x7f33181513c0 2024-09-19T08:37:55.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.498+0000 7f330ffff640 1 -- 172.21.15.98:0/2947274201 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f3310016da0 con 0x7f33181513c0 2024-09-19T08:37:55.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.562+0000 7f331dedf640 1 -- 172.21.15.98:0/2947274201 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f331815ae90 con 0x7f33181513c0 2024-09-19T08:37:55.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.566+0000 7f330ffff640 1 -- 172.21.15.98:0/2947274201 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f33100b6b80 con 0x7f33181513c0 2024-09-19T08:37:55.568 INFO:teuthology.orchestra.run.smithi098.stderr:adjusted tunables profile to default 2024-09-19T08:37:55.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.566+0000 7f331dedf640 1 -- 172.21.15.98:0/2947274201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33181513c0 msgr2=0x7f3318153e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:55.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.566+0000 7f331dedf640 1 --2- 172.21.15.98:0/2947274201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33181513c0 0x7f3318153e30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f331001d4f0 tx=0x7f3310019db0 comp rx=0 tx=0).stop 2024-09-19T08:37:55.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.566+0000 7f331dedf640 1 -- 172.21.15.98:0/2947274201 shutdown_connections 2024-09-19T08:37:55.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.566+0000 7f331dedf640 1 --2- 172.21.15.98:0/2947274201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33181513c0 0x7f3318153e30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:55.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.566+0000 7f331dedf640 1 --2- 172.21.15.98:0/2947274201 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3318150f40 0x7f33181538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:55.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.566+0000 7f331dedf640 1 --2- 172.21.15.98:0/2947274201 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33040051d0 0x7f3318142e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:55.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.566+0000 7f331dedf640 1 -- 172.21.15.98:0/2947274201 >> 172.21.15.98:0/2947274201 conn(0x7f331806dea0 msgr2=0x7f331815b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:55.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.566+0000 7f331dedf640 1 -- 172.21.15.98:0/2947274201 shutdown_connections 2024-09-19T08:37:55.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:55.566+0000 7f331dedf640 1 -- 172.21.15.98:0/2947274201 wait complete. 2024-09-19T08:37:55.587 INFO:tasks.ceph:check_enable_crimson: False 2024-09-19T08:37:55.587 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-19T08:37:55.588 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:37:55.588 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-19T08:37:55.600 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:37:55.600 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-19T08:37:55.654 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:37:55.654 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-19T08:37:55.670 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:37:55.666+0000 7f3765211180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T08:37:55.680 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:37:55.680 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-19T08:37:55.682 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:37:55.681+0000 7ff7b5cad180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T08:37:55.733 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:37:55.733 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-19T08:37:55.747 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:37:55.747 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-19T08:37:55.802 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:37:55.802 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-19T08:37:55.857 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:37:55.857 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-19T08:37:55.910 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --cluster ceph osd new ac912a7b-9671-49b2-bc86-b3eae286c89f 0 2024-09-19T08:37:56.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.073+0000 7f968c9d2640 1 Processor -- start 2024-09-19T08:37:56.076 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.073+0000 7f968c9d2640 1 -- start start 2024-09-19T08:37:56.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.073+0000 7f968c9d2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96880cc4e0 0x7f9688068220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:56.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.073+0000 7f968c9d2640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f96880ce620 0x7f96880cead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:56.078 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.073+0000 7f968c9d2640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f968806fd20 con 0x7f968815b230 2024-09-19T08:37:56.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.073+0000 7f968c9d2640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f9688072ca0 con 0x7f96880ce620 2024-09-19T08:37:56.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.073+0000 7f968c9d2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f968815b610 con 0x7f96880cc4e0 2024-09-19T08:37:56.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.073+0000 7f968659c640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f968815b230 0x7f9688073130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:52062/0 (socket says 172.21.15.139:52062) 2024-09-19T08:37:56.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.073+0000 7f9685d9b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96880cc4e0 0x7f9688068220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:56.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.073+0000 7f968659c640 1 -- 172.21.15.139:0/85070404 learned_addr learned my addr 172.21.15.139:0/85070404 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-19T08:37:56.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.073+0000 7f9686d9d640 1 --2- 172.21.15.139:0/85070404 >> v2:172.21.15.98:3300/0 conn(0x7f96880ce620 0x7f96880cead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:56.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.073+0000 7f968559a640 1 -- 172.21.15.139:0/85070404 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3179641924 0 0) 0x7f968806fd20 con 0x7f968815b230 2024-09-19T08:37:56.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.073+0000 7f9685d9b640 1 -- 172.21.15.139:0/85070404 >> v2:172.21.15.98:3300/0 conn(0x7f96880ce620 msgr2=0x7f96880cead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:37:56.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.073+0000 7f9685d9b640 1 --2- 172.21.15.139:0/85070404 >> v2:172.21.15.98:3300/0 conn(0x7f96880ce620 0x7f96880cead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.073+0000 7f9685d9b640 1 -- 172.21.15.139:0/85070404 >> v1:172.21.15.139:6789/0 conn(0x7f968815b230 legacy=0x7f9688073130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:56.079 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.073+0000 7f9685d9b640 1 -- 172.21.15.139:0/85070404 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9688072370 con 0x7f96880cc4e0 2024-09-19T08:37:56.080 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.073+0000 7f9686d9d640 1 --2- 172.21.15.139:0/85070404 >> v2:172.21.15.98:3300/0 conn(0x7f96880ce620 0x7f96880cead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:37:56.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.077+0000 7f9685d9b640 1 --2- 172.21.15.139:0/85070404 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96880cc4e0 0x7f9688068220 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9688072e20 tx=0x7f967001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=307f814844042d81 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:56.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.077+0000 7f968559a640 1 -- 172.21.15.139:0/85070404 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96700a7070 con 0x7f96880cc4e0 2024-09-19T08:37:56.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.077+0000 7f968559a640 1 -- 172.21.15.139:0/85070404 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96700b0070 con 0x7f96880cc4e0 2024-09-19T08:37:56.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.077+0000 7f968559a640 1 -- 172.21.15.139:0/85070404 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96700ab890 con 0x7f96880cc4e0 2024-09-19T08:37:56.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.077+0000 7f968c9d2640 1 -- 172.21.15.139:0/85070404 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96880cc4e0 msgr2=0x7f9688068220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:56.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.077+0000 7f968c9d2640 1 --2- 172.21.15.139:0/85070404 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96880cc4e0 0x7f9688068220 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9688072e20 tx=0x7f967001c470 comp rx=0 tx=0).stop 2024-09-19T08:37:56.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.077+0000 7f968c9d2640 1 -- 172.21.15.139:0/85070404 shutdown_connections 2024-09-19T08:37:56.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.077+0000 7f968c9d2640 1 --2- 172.21.15.139:0/85070404 >> v2:172.21.15.98:3300/0 conn(0x7f96880ce620 0x7f96880cead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.081 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.077+0000 7f968c9d2640 1 --2- 172.21.15.139:0/85070404 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96880cc4e0 0x7f9688068220 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.077+0000 7f968c9d2640 1 -- 172.21.15.139:0/85070404 >> 172.21.15.139:0/85070404 conn(0x7f96880d0a50 msgr2=0x7f96880d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:56.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.077+0000 7f968c9d2640 1 -- 172.21.15.139:0/85070404 shutdown_connections 2024-09-19T08:37:56.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.077+0000 7f968c9d2640 1 -- 172.21.15.139:0/85070404 wait complete. 2024-09-19T08:37:56.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.081+0000 7f968c9d2640 1 Processor -- start 2024-09-19T08:37:56.082 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.081+0000 7f968c9d2640 1 -- start start 2024-09-19T08:37:56.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.081+0000 7f968c9d2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96880cc4e0 0x7f96880b4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:56.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.081+0000 7f968c9d2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96880ce620 0x7f96881479b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:56.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.081+0000 7f968c9d2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f968815b230 0x7f968815d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:56.084 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.081+0000 7f968c9d2640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f96880739c0 con 0x7f96880cc4e0 2024-09-19T08:37:56.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.081+0000 7f968c9d2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9688073840 con 0x7f96880ce620 2024-09-19T08:37:56.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.081+0000 7f968c9d2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9688167f40 con 0x7f968815b230 2024-09-19T08:37:56.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.081+0000 7f9685d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96880ce620 0x7f96881479b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:56.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.081+0000 7f9686d9d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f968815b230 0x7f968815d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:56.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.081+0000 7f9685d9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96880ce620 0x7f96881479b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60844/0 (socket says 172.21.15.139:60844) 2024-09-19T08:37:56.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.081+0000 7f9686d9d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f968815b230 0x7f968815d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.139:57622/0 (socket says 172.21.15.139:57622) 2024-09-19T08:37:56.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.081+0000 7f9685d9b640 1 -- 172.21.15.139:0/3199857398 learned_addr learned my addr 172.21.15.139:0/3199857398 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-19T08:37:56.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.081+0000 7f968659c640 1 --2- 172.21.15.139:0/3199857398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96880cc4e0 0x7f96880b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:56.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.081+0000 7f9686d9d640 1 -- 172.21.15.139:0/3199857398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96880ce620 msgr2=0x7f96881479b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:56.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.081+0000 7f9686d9d640 1 --2- 172.21.15.139:0/3199857398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96880ce620 0x7f96881479b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.081+0000 7f9686d9d640 1 -- 172.21.15.139:0/3199857398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96880cc4e0 msgr2=0x7f96880b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:56.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.081+0000 7f9686d9d640 1 --2- 172.21.15.139:0/3199857398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96880cc4e0 0x7f96880b4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.085 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.081+0000 7f9686d9d640 1 -- 172.21.15.139:0/3199857398 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f968815b950 con 0x7f968815b230 2024-09-19T08:37:56.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.081+0000 7f9686d9d640 1 --2- 172.21.15.139:0/3199857398 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f968815b230 0x7f968815d180 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f967800cdd0 tx=0x7f967800a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:56.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.081+0000 7f967effd640 1 -- 172.21.15.139:0/3199857398 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967801a070 con 0x7f968815b230 2024-09-19T08:37:56.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.081+0000 7f967effd640 1 -- 172.21.15.139:0/3199857398 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9678015680 con 0x7f968815b230 2024-09-19T08:37:56.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.081+0000 7f967effd640 1 -- 172.21.15.139:0/3199857398 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9678015a70 con 0x7f968815b230 2024-09-19T08:37:56.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.081+0000 7f968c9d2640 1 -- 172.21.15.139:0/3199857398 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96880ca760 con 0x7f968815b230 2024-09-19T08:37:56.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.081+0000 7f968c9d2640 1 -- 172.21.15.139:0/3199857398 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f968815bc30 con 0x7f968815b230 2024-09-19T08:37:56.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.081+0000 7f967effd640 1 -- 172.21.15.139:0/3199857398 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f967801d070 con 0x7f968815b230 2024-09-19T08:37:56.086 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.081+0000 7f967effd640 1 -- 172.21.15.139:0/3199857398 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f967800d770 con 0x7f968815b230 2024-09-19T08:37:56.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.081+0000 7f968c9d2640 1 -- 172.21.15.139:0/3199857398 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9688068220 con 0x7f968815b230 2024-09-19T08:37:56.087 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.085+0000 7f967effd640 1 -- 172.21.15.139:0/3199857398 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f9678015cc0 con 0x7f968815b230 2024-09-19T08:37:56.155 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:37:56.153+0000 7ff7b5cad180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T08:37:56.156 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.153+0000 7f968c9d2640 1 -- 172.21.15.139:0/3199857398 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ac912a7b-9671-49b2-bc86-b3eae286c89f", "id": 0} v 0) -- 0x7f9688075f20 con 0x7f968815b230 2024-09-19T08:37:56.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.161+0000 7f967effd640 1 -- 172.21.15.139:0/3199857398 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ac912a7b-9671-49b2-bc86-b3eae286c89f", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f967800d9c0 con 0x7f968815b230 2024-09-19T08:37:56.162 INFO:teuthology.orchestra.run.smithi139.stdout:0 2024-09-19T08:37:56.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.161+0000 7f968c9d2640 1 -- 172.21.15.139:0/3199857398 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f968815b230 msgr2=0x7f968815d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:56.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.161+0000 7f968c9d2640 1 --2- 172.21.15.139:0/3199857398 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f968815b230 0x7f968815d180 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f967800cdd0 tx=0x7f967800a710 comp rx=0 tx=0).stop 2024-09-19T08:37:56.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.161+0000 7f968c9d2640 1 -- 172.21.15.139:0/3199857398 shutdown_connections 2024-09-19T08:37:56.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.161+0000 7f968c9d2640 1 --2- 172.21.15.139:0/3199857398 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f968815b230 0x7f968815d180 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.161+0000 7f968c9d2640 1 --2- 172.21.15.139:0/3199857398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96880ce620 0x7f96881479b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.161+0000 7f968c9d2640 1 --2- 172.21.15.139:0/3199857398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96880cc4e0 0x7f96880b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.161+0000 7f968c9d2640 1 -- 172.21.15.139:0/3199857398 >> 172.21.15.139:0/3199857398 conn(0x7f96880d0a50 msgr2=0x7f9688073130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:56.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.161+0000 7f968c9d2640 1 -- 172.21.15.139:0/3199857398 shutdown_connections 2024-09-19T08:37:56.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.161+0000 7f968c9d2640 1 -- 172.21.15.139:0/3199857398 wait complete. 2024-09-19T08:37:56.179 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --cluster ceph osd new ed3494e5-e909-4d1a-a4f2-9080dbed7574 1 2024-09-19T08:37:56.183 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:37:56.182+0000 7f3765211180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T08:37:56.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.289+0000 7f1b44926640 1 Processor -- start 2024-09-19T08:37:56.293 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.289+0000 7f1b44926640 1 -- start start 2024-09-19T08:37:56.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.289+0000 7f1b44926640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f1b40074c70 0x7f1b4015c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:56.295 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.289+0000 7f1b44926640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b400b0cd0 con 0x7f1b401569c0 2024-09-19T08:37:56.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.289+0000 7f1b44926640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b400b1090 con 0x7f1b4006c3a0 2024-09-19T08:37:56.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.289+0000 7f1b44926640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f1b4015cd80 con 0x7f1b40074c70 2024-09-19T08:37:56.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.293+0000 7f1b3dd9b640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f1b4006c3a0 0x7f1b400b1850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:52066/0 (socket says 172.21.15.139:52066) 2024-09-19T08:37:56.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.293+0000 7f1b3dd9b640 1 -- 172.21.15.139:0/1510967805 learned_addr learned my addr 172.21.15.139:0/1510967805 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-19T08:37:56.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.293+0000 7f1b3ed9d640 1 --2- 172.21.15.139:0/1510967805 >> v2:172.21.15.98:3300/0 conn(0x7f1b40074c70 0x7f1b4015c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:56.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.293+0000 7f1b3d59a640 1 -- 172.21.15.139:0/1510967805 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2188937268 0 0) 0x7f1b400b1090 con 0x7f1b4006c3a0 2024-09-19T08:37:56.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.293+0000 7f1b3d59a640 1 -- 172.21.15.139:0/1510967805 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b28003530 con 0x7f1b4006c3a0 2024-09-19T08:37:56.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.293+0000 7f1b3d59a640 1 -- 172.21.15.139:0/1510967805 <== mon.1 v1:172.21.15.139:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1952493614 0 0) 0x7f1b28003530 con 0x7f1b4006c3a0 2024-09-19T08:37:56.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.293+0000 7f1b3d59a640 1 -- 172.21.15.139:0/1510967805 >> v2:172.21.15.98:3300/0 conn(0x7f1b40074c70 msgr2=0x7f1b4015c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:37:56.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.293+0000 7f1b3d59a640 1 --2- 172.21.15.139:0/1510967805 >> v2:172.21.15.98:3300/0 conn(0x7f1b40074c70 0x7f1b4015c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.293+0000 7f1b3d59a640 1 -- 172.21.15.139:0/1510967805 >> v1:172.21.15.98:6789/0 conn(0x7f1b401569c0 legacy=0x7f1b4006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:56.296 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.293+0000 7f1b3d59a640 1 -- 172.21.15.139:0/1510967805 --> v1:172.21.15.139:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b4015cfb0 con 0x7f1b4006c3a0 2024-09-19T08:37:56.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.293+0000 7f1b3d59a640 1 -- 172.21.15.139:0/1510967805 <== mon.1 v1:172.21.15.139:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f1b34001680 con 0x7f1b4006c3a0 2024-09-19T08:37:56.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.293+0000 7f1b3d59a640 1 -- 172.21.15.139:0/1510967805 >> v1:172.21.15.139:6789/0 conn(0x7f1b4006c3a0 legacy=0x7f1b400b1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:56.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.293+0000 7f1b3d59a640 1 --2- 172.21.15.139:0/1510967805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b280039c0 0x7f1b28003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:56.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.293+0000 7f1b3d59a640 1 --2- 172.21.15.139:0/1510967805 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b28004810 0x7f1b28004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:56.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.293+0000 7f1b3d59a640 1 --2- 172.21.15.139:0/1510967805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b280051d0 0x7f1b2800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:56.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.293+0000 7f1b3d59a640 1 -- 172.21.15.139:0/1510967805 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2800c0a0 con 0x7f1b280039c0 2024-09-19T08:37:56.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.293+0000 7f1b3d59a640 1 -- 172.21.15.139:0/1510967805 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1b28010630 con 0x7f1b280051d0 2024-09-19T08:37:56.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.293+0000 7f1b3d59a640 1 -- 172.21.15.139:0/1510967805 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1b28010830 con 0x7f1b28004810 2024-09-19T08:37:56.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.293+0000 7f1b3dd9b640 1 --2- 172.21.15.139:0/1510967805 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b28004810 0x7f1b28004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:56.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.293+0000 7f1b3ed9d640 1 --2- 172.21.15.139:0/1510967805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b280051d0 0x7f1b2800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:56.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.293+0000 7f1b3e59c640 1 --2- 172.21.15.139:0/1510967805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b280039c0 0x7f1b28003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:56.297 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.293+0000 7f1b3ed9d640 1 -- 172.21.15.139:0/1510967805 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b28004810 msgr2=0x7f1b28004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:37:56.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.293+0000 7f1b3ed9d640 1 --2- 172.21.15.139:0/1510967805 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b28004810 0x7f1b28004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.293+0000 7f1b3ed9d640 1 -- 172.21.15.139:0/1510967805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b280039c0 msgr2=0x7f1b28003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:37:56.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.293+0000 7f1b3ed9d640 1 --2- 172.21.15.139:0/1510967805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b280039c0 0x7f1b28003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.298 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.293+0000 7f1b3ed9d640 1 -- 172.21.15.139:0/1510967805 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b28003530 con 0x7f1b280051d0 2024-09-19T08:37:56.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.293+0000 7f1b3ed9d640 1 -- 172.21.15.139:0/1510967805 reap_dead start 2024-09-19T08:37:56.299 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.293+0000 7f1b3ed9d640 1 --2- 172.21.15.139:0/1510967805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b280051d0 0x7f1b2800b9d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1b38009fd0 tx=0x7f1b3801cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c392e047313c829f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:56.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.293+0000 7f1b3d59a640 1 -- 172.21.15.139:0/1510967805 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b380b2070 con 0x7f1b280051d0 2024-09-19T08:37:56.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.293+0000 7f1b3d59a640 1 -- 172.21.15.139:0/1510967805 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b380a5930 con 0x7f1b280051d0 2024-09-19T08:37:56.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.293+0000 7f1b3d59a640 1 -- 172.21.15.139:0/1510967805 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b38009bf0 con 0x7f1b280051d0 2024-09-19T08:37:56.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.297+0000 7f1b44926640 1 -- 172.21.15.139:0/1510967805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b280051d0 msgr2=0x7f1b2800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:56.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.297+0000 7f1b44926640 1 --2- 172.21.15.139:0/1510967805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b280051d0 0x7f1b2800b9d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1b38009fd0 tx=0x7f1b3801cc80 comp rx=0 tx=0).stop 2024-09-19T08:37:56.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.297+0000 7f1b44926640 1 -- 172.21.15.139:0/1510967805 shutdown_connections 2024-09-19T08:37:56.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.297+0000 7f1b44926640 1 --2- 172.21.15.139:0/1510967805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b280051d0 0x7f1b2800b9d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.297+0000 7f1b44926640 1 -- 172.21.15.139:0/1510967805 >> 172.21.15.139:0/1510967805 conn(0x7f1b400b4d60 msgr2=0x7f1b400b5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:56.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.297+0000 7f1b44926640 1 -- 172.21.15.139:0/1510967805 shutdown_connections 2024-09-19T08:37:56.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.297+0000 7f1b44926640 1 -- 172.21.15.139:0/1510967805 wait complete. 2024-09-19T08:37:56.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.297+0000 7f1b44926640 1 Processor -- start 2024-09-19T08:37:56.300 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.297+0000 7f1b44926640 1 -- start start 2024-09-19T08:37:56.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.297+0000 7f1b44926640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b280051d0 0x7f1b401507f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:56.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.297+0000 7f1b44926640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b4013c1b0 0x7f1b4013d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:56.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.297+0000 7f1b44926640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b4013dc90 0x7f1b4009c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:56.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.297+0000 7f1b44926640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4015cbd0 con 0x7f1b4013dc90 2024-09-19T08:37:56.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.297+0000 7f1b44926640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1b4015ca50 con 0x7f1b280051d0 2024-09-19T08:37:56.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.297+0000 7f1b44926640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1b400b13c0 con 0x7f1b4013c1b0 2024-09-19T08:37:56.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.297+0000 7f1b3dd9b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b4013c1b0 0x7f1b4013d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:56.303 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.297+0000 7f1b3e59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b280051d0 0x7f1b401507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:56.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.297+0000 7f1b3dd9b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b4013c1b0 0x7f1b4013d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.139:57644/0 (socket says 172.21.15.139:57644) 2024-09-19T08:37:56.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.297+0000 7f1b3dd9b640 1 -- 172.21.15.139:0/4254548934 learned_addr learned my addr 172.21.15.139:0/4254548934 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-19T08:37:56.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.297+0000 7f1b3e59c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b280051d0 0x7f1b401507f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60856/0 (socket says 172.21.15.139:60856) 2024-09-19T08:37:56.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.297+0000 7f1b3ed9d640 1 --2- 172.21.15.139:0/4254548934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b4013dc90 0x7f1b4009c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:56.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.297+0000 7f1b3dd9b640 1 -- 172.21.15.139:0/4254548934 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b280051d0 msgr2=0x7f1b401507f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:56.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.297+0000 7f1b3dd9b640 1 --2- 172.21.15.139:0/4254548934 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b280051d0 0x7f1b401507f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.297+0000 7f1b3dd9b640 1 -- 172.21.15.139:0/4254548934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b4013dc90 msgr2=0x7f1b4009c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:56.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.297+0000 7f1b3dd9b640 1 --2- 172.21.15.139:0/4254548934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b4013dc90 0x7f1b4009c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.297+0000 7f1b3dd9b640 1 -- 172.21.15.139:0/4254548934 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b401f2b40 con 0x7f1b4013c1b0 2024-09-19T08:37:56.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.297+0000 7f1b3e59c640 1 --2- 172.21.15.139:0/4254548934 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b280051d0 0x7f1b401507f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:37:56.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.297+0000 7f1b3dd9b640 1 --2- 172.21.15.139:0/4254548934 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b4013c1b0 0x7f1b4013d750 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1b340072c0 tx=0x7f1b34007780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:56.304 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.297+0000 7f1b2effd640 1 -- 172.21.15.139:0/4254548934 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3401a070 con 0x7f1b4013c1b0 2024-09-19T08:37:56.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.297+0000 7f1b2effd640 1 -- 172.21.15.139:0/4254548934 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b3400f960 con 0x7f1b4013c1b0 2024-09-19T08:37:56.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.297+0000 7f1b2effd640 1 -- 172.21.15.139:0/4254548934 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3400fd30 con 0x7f1b4013c1b0 2024-09-19T08:37:56.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.297+0000 7f1b44926640 1 -- 172.21.15.139:0/4254548934 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b4013c6b0 con 0x7f1b4013c1b0 2024-09-19T08:37:56.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.297+0000 7f1b44926640 1 -- 172.21.15.139:0/4254548934 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b4013cc70 con 0x7f1b4013c1b0 2024-09-19T08:37:56.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.297+0000 7f1b2effd640 1 -- 172.21.15.139:0/4254548934 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1b34014450 con 0x7f1b4013c1b0 2024-09-19T08:37:56.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.297+0000 7f1b2effd640 1 -- 172.21.15.139:0/4254548934 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f1b3401eb40 con 0x7f1b4013c1b0 2024-09-19T08:37:56.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.301+0000 7f1b44926640 1 -- 172.21.15.139:0/4254548934 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b400b2800 con 0x7f1b4013c1b0 2024-09-19T08:37:56.305 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.301+0000 7f1b2effd640 1 -- 172.21.15.139:0/4254548934 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f1b34014830 con 0x7f1b4013c1b0 2024-09-19T08:37:56.356 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:37:56.353+0000 7ff7b5cad180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T08:37:56.373 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.369+0000 7f1b44926640 1 -- 172.21.15.139:0/4254548934 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ed3494e5-e909-4d1a-a4f2-9080dbed7574", "id": 1} v 0) -- 0x7f1b40153200 con 0x7f1b4013c1b0 2024-09-19T08:37:56.373 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:37:56.370+0000 7f3765211180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T08:37:56.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.377+0000 7f1b2effd640 1 -- 172.21.15.139:0/4254548934 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ed3494e5-e909-4d1a-a4f2-9080dbed7574", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f1b34014a10 con 0x7f1b4013c1b0 2024-09-19T08:37:56.384 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-09-19T08:37:56.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.381+0000 7f1b44926640 1 -- 172.21.15.139:0/4254548934 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b4013c1b0 msgr2=0x7f1b4013d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:56.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.381+0000 7f1b44926640 1 --2- 172.21.15.139:0/4254548934 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b4013c1b0 0x7f1b4013d750 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1b340072c0 tx=0x7f1b34007780 comp rx=0 tx=0).stop 2024-09-19T08:37:56.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.381+0000 7f1b44926640 1 -- 172.21.15.139:0/4254548934 shutdown_connections 2024-09-19T08:37:56.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.381+0000 7f1b44926640 1 --2- 172.21.15.139:0/4254548934 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b4013dc90 0x7f1b4009c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.381+0000 7f1b44926640 1 --2- 172.21.15.139:0/4254548934 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b4013c1b0 0x7f1b4013d750 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.381+0000 7f1b44926640 1 --2- 172.21.15.139:0/4254548934 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b280051d0 0x7f1b401507f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.381+0000 7f1b44926640 1 -- 172.21.15.139:0/4254548934 >> 172.21.15.139:0/4254548934 conn(0x7f1b400b4d60 msgr2=0x7f1b40067a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:56.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.381+0000 7f1b44926640 1 -- 172.21.15.139:0/4254548934 shutdown_connections 2024-09-19T08:37:56.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.381+0000 7f1b44926640 1 -- 172.21.15.139:0/4254548934 wait complete. 2024-09-19T08:37:56.398 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --cluster ceph osd new 9760b7fd-c73e-4668-9cc9-a1efe5d5767c 2 2024-09-19T08:37:56.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.509+0000 7f45a3775640 1 Processor -- start 2024-09-19T08:37:56.511 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.509+0000 7f45a3775640 1 -- start start 2024-09-19T08:37:56.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.509+0000 7f45a3775640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f459c06c480 0x7f459c06c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:56.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.509+0000 7f45a3775640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f459c0b1bc0 0x7f459c0b2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:56.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.509+0000 7f45a3775640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f459c0af4a0 con 0x7f459c15b4f0 2024-09-19T08:37:56.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.509+0000 7f45a3775640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f459c06ce10 con 0x7f459c0b1bc0 2024-09-19T08:37:56.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.509+0000 7f45a3775640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f459c0af780 con 0x7f459c06c480 2024-09-19T08:37:56.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.509+0000 7f45a0d10640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f459c06c480 0x7f459c06c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:56.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.509+0000 7f45a1511640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f459c15b4f0 0x7f459c0b0530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.139:52076/0 (socket says 172.21.15.139:52076) 2024-09-19T08:37:56.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.509+0000 7f45a1511640 1 -- 172.21.15.139:0/3641175658 learned_addr learned my addr 172.21.15.139:0/3641175658 (peer_addr_for_me v1:172.21.15.139:0/0) 2024-09-19T08:37:56.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.509+0000 7f45a1d12640 1 --2- 172.21.15.139:0/3641175658 >> v2:172.21.15.98:3300/0 conn(0x7f459c0b1bc0 0x7f459c0b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:56.513 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.509+0000 7f45a0d10640 1 -- 172.21.15.139:0/3641175658 >> v2:172.21.15.98:3300/0 conn(0x7f459c0b1bc0 msgr2=0x7f459c0b2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:37:56.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.509+0000 7f45a0d10640 1 --2- 172.21.15.139:0/3641175658 >> v2:172.21.15.98:3300/0 conn(0x7f459c0b1bc0 0x7f459c0b2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.509+0000 7f45a0d10640 1 -- 172.21.15.139:0/3641175658 >> v1:172.21.15.139:6789/0 conn(0x7f459c15b4f0 legacy=0x7f459c0b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:56.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.509+0000 7f45a0d10640 1 -- 172.21.15.139:0/3641175658 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f459c0afa60 con 0x7f459c06c480 2024-09-19T08:37:56.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.509+0000 7f458ffff640 1 -- 172.21.15.139:0/3641175658 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1688716727 0 0) 0x7f459c0af4a0 con 0x7f459c15b4f0 2024-09-19T08:37:56.514 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.509+0000 7f45a1d12640 1 --2- 172.21.15.139:0/3641175658 >> v2:172.21.15.98:3300/0 conn(0x7f459c0b1bc0 0x7f459c0b2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:37:56.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.513+0000 7f45a0d10640 1 --2- 172.21.15.139:0/3641175658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f459c06c480 0x7f459c06c8d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f459c0b6600 tx=0x7f459801ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f83b0f47f9e81392 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:56.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.513+0000 7f458ffff640 1 -- 172.21.15.139:0/3641175658 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45980b5070 con 0x7f459c06c480 2024-09-19T08:37:56.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.513+0000 7f458ffff640 1 -- 172.21.15.139:0/3641175658 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45980a8970 con 0x7f459c06c480 2024-09-19T08:37:56.515 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.513+0000 7f458ffff640 1 -- 172.21.15.139:0/3641175658 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45980a8d40 con 0x7f459c06c480 2024-09-19T08:37:56.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.513+0000 7f45a3775640 1 -- 172.21.15.139:0/3641175658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f459c06c480 msgr2=0x7f459c06c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:56.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.513+0000 7f45a3775640 1 --2- 172.21.15.139:0/3641175658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f459c06c480 0x7f459c06c8d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f459c0b6600 tx=0x7f459801ec70 comp rx=0 tx=0).stop 2024-09-19T08:37:56.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.513+0000 7f45a3775640 1 -- 172.21.15.139:0/3641175658 shutdown_connections 2024-09-19T08:37:56.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.513+0000 7f45a3775640 1 --2- 172.21.15.139:0/3641175658 >> v2:172.21.15.98:3300/0 conn(0x7f459c0b1bc0 0x7f459c0b2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.513+0000 7f45a3775640 1 --2- 172.21.15.139:0/3641175658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f459c06c480 0x7f459c06c8d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.513+0000 7f45a3775640 1 -- 172.21.15.139:0/3641175658 >> 172.21.15.139:0/3641175658 conn(0x7f459c0b5f50 msgr2=0x7f459c0b6650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:56.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.513+0000 7f45a3775640 1 -- 172.21.15.139:0/3641175658 shutdown_connections 2024-09-19T08:37:56.516 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.513+0000 7f45a3775640 1 -- 172.21.15.139:0/3641175658 wait complete. 2024-09-19T08:37:56.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.513+0000 7f45a3775640 1 Processor -- start 2024-09-19T08:37:56.517 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.513+0000 7f45a3775640 1 -- start start 2024-09-19T08:37:56.519 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.513+0000 7f45a3775640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f459c06c480 0x7f459c0d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:56.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.513+0000 7f45a3775640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f459c0b1bc0 0x7f459c0ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:56.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.513+0000 7f45a3775640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f459c15b4f0 0x7f459c0ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:56.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.513+0000 7f45a3775640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f459c0b1670 con 0x7f459c15b4f0 2024-09-19T08:37:56.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.513+0000 7f45a3775640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f459c0b0c40 con 0x7f459c06c480 2024-09-19T08:37:56.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.513+0000 7f45a3775640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f459c15b8d0 con 0x7f459c0b1bc0 2024-09-19T08:37:56.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.513+0000 7f45a1511640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f459c06c480 0x7f459c0d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:56.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.517+0000 7f45a1511640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f459c06c480 0x7f459c0d1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60864/0 (socket says 172.21.15.139:60864) 2024-09-19T08:37:56.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.517+0000 7f45a0d10640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f459c0b1bc0 0x7f459c0ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:56.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.517+0000 7f45a1511640 1 -- 172.21.15.139:0/3646792699 learned_addr learned my addr 172.21.15.139:0/3646792699 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-19T08:37:56.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.517+0000 7f45a1d12640 1 --2- 172.21.15.139:0/3646792699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f459c15b4f0 0x7f459c0ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:56.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.517+0000 7f45a0d10640 1 -- 172.21.15.139:0/3646792699 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f459c06c480 msgr2=0x7f459c0d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:56.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.517+0000 7f45a0d10640 1 --2- 172.21.15.139:0/3646792699 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f459c06c480 0x7f459c0d1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.517+0000 7f45a0d10640 1 -- 172.21.15.139:0/3646792699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f459c15b4f0 msgr2=0x7f459c0ca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:56.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.517+0000 7f45a0d10640 1 --2- 172.21.15.139:0/3646792699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f459c15b4f0 0x7f459c0ca9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.517+0000 7f45a0d10640 1 -- 172.21.15.139:0/3646792699 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f459c1f3dd0 con 0x7f459c0b1bc0 2024-09-19T08:37:56.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.517+0000 7f45a1511640 1 --2- 172.21.15.139:0/3646792699 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f459c06c480 0x7f459c0d1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:37:56.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.517+0000 7f45a0d10640 1 --2- 172.21.15.139:0/3646792699 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f459c0b1bc0 0x7f459c0ca4b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f45980af000 tx=0x7f459801ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:56.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.517+0000 7f458dffb640 1 -- 172.21.15.139:0/3646792699 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45980b5040 con 0x7f459c0b1bc0 2024-09-19T08:37:56.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.517+0000 7f458dffb640 1 -- 172.21.15.139:0/3646792699 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4598004240 con 0x7f459c0b1bc0 2024-09-19T08:37:56.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.517+0000 7f45a3775640 1 -- 172.21.15.139:0/3646792699 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f459c0cf4b0 con 0x7f459c0b1bc0 2024-09-19T08:37:56.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.517+0000 7f458dffb640 1 -- 172.21.15.139:0/3646792699 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4598003620 con 0x7f459c0b1bc0 2024-09-19T08:37:56.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.517+0000 7f45a3775640 1 -- 172.21.15.139:0/3646792699 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f459c0d1200 con 0x7f459c0b1bc0 2024-09-19T08:37:56.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.517+0000 7f458dffb640 1 -- 172.21.15.139:0/3646792699 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f45980c2060 con 0x7f459c0b1bc0 2024-09-19T08:37:56.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.517+0000 7f458dffb640 1 -- 172.21.15.139:0/3646792699 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f45980c4450 con 0x7f459c0b1bc0 2024-09-19T08:37:56.521 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.517+0000 7f45a3775640 1 -- 172.21.15.139:0/3646792699 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f459c06c8d0 con 0x7f459c0b1bc0 2024-09-19T08:37:56.522 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.517+0000 7f458dffb640 1 -- 172.21.15.139:0/3646792699 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f45980ca080 con 0x7f459c0b1bc0 2024-09-19T08:37:56.590 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.589+0000 7f45a3775640 1 -- 172.21.15.139:0/3646792699 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9760b7fd-c73e-4668-9cc9-a1efe5d5767c", "id": 2} v 0) -- 0x7f459c06f8c0 con 0x7f459c0b1bc0 2024-09-19T08:37:56.597 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.593+0000 7f458dffb640 1 -- 172.21.15.139:0/3646792699 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9760b7fd-c73e-4668-9cc9-a1efe5d5767c", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f45980187d0 con 0x7f459c0b1bc0 2024-09-19T08:37:56.597 INFO:teuthology.orchestra.run.smithi139.stdout:2 2024-09-19T08:37:56.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.597+0000 7f45a3775640 1 -- 172.21.15.139:0/3646792699 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f459c0b1bc0 msgr2=0x7f459c0ca4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:56.598 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.597+0000 7f45a3775640 1 --2- 172.21.15.139:0/3646792699 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f459c0b1bc0 0x7f459c0ca4b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f45980af000 tx=0x7f459801ba10 comp rx=0 tx=0).stop 2024-09-19T08:37:56.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.597+0000 7f45a3775640 1 -- 172.21.15.139:0/3646792699 shutdown_connections 2024-09-19T08:37:56.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.597+0000 7f45a3775640 1 --2- 172.21.15.139:0/3646792699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f459c15b4f0 0x7f459c0ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.597+0000 7f45a3775640 1 --2- 172.21.15.139:0/3646792699 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f459c0b1bc0 0x7f459c0ca4b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.597+0000 7f45a3775640 1 --2- 172.21.15.139:0/3646792699 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f459c06c480 0x7f459c0d1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.597+0000 7f45a3775640 1 -- 172.21.15.139:0/3646792699 >> 172.21.15.139:0/3646792699 conn(0x7f459c0b5f50 msgr2=0x7f459c06ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:56.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.597+0000 7f45a3775640 1 -- 172.21.15.139:0/3646792699 shutdown_connections 2024-09-19T08:37:56.599 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.597+0000 7f45a3775640 1 -- 172.21.15.139:0/3646792699 wait complete. 2024-09-19T08:37:56.614 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --cluster ceph osd new cf08f97e-9975-4b66-ac1a-7c6eb6a24e12 3 2024-09-19T08:37:56.729 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.725+0000 7fad53f80640 1 Processor -- start 2024-09-19T08:37:56.730 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.725+0000 7fad53f80640 1 -- start start 2024-09-19T08:37:56.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.729+0000 7fad53f80640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fad4c156ae0 0x7fad4c0ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:56.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.729+0000 7fad53f80640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fad4c160610 0x7fad4c06f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:56.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.729+0000 7fad53f80640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad4c0cb5a0 con 0x7fad4c0cb0a0 2024-09-19T08:37:56.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.729+0000 7fad53f80640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fad4c0cb880 con 0x7fad4c156ae0 2024-09-19T08:37:56.731 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.729+0000 7fad53f80640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fad4c0cbb60 con 0x7fad4c160610 2024-09-19T08:37:56.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.729+0000 7fad5251d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fad4c160610 0x7fad4c06f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:56.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.729+0000 7fad5251d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fad4c160610 0x7fad4c06f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60874/0 (socket says 172.21.15.139:60874) 2024-09-19T08:37:56.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.729+0000 7fad5251d640 1 -- 172.21.15.139:0/2460839249 learned_addr learned my addr 172.21.15.139:0/2460839249 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-19T08:37:56.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.729+0000 7fad51d1c640 1 --2- 172.21.15.139:0/2460839249 >> v2:172.21.15.98:3300/0 conn(0x7fad4c156ae0 0x7fad4c0ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:56.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.729+0000 7fad5251d640 1 -- 172.21.15.139:0/2460839249 >> v2:172.21.15.98:3300/0 conn(0x7fad4c156ae0 msgr2=0x7fad4c0ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:37:56.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.729+0000 7fad5251d640 1 --2- 172.21.15.139:0/2460839249 >> v2:172.21.15.98:3300/0 conn(0x7fad4c156ae0 0x7fad4c0ccc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.729+0000 7fad5251d640 1 -- 172.21.15.139:0/2460839249 >> v1:172.21.15.98:6789/0 conn(0x7fad4c0cb0a0 legacy=0x7fad4c15ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:37:56.732 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.729+0000 7fad5251d640 1 -- 172.21.15.139:0/2460839249 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad4c0cbe40 con 0x7fad4c160610 2024-09-19T08:37:56.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.729+0000 7fad5251d640 1 --2- 172.21.15.139:0/2460839249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad4c160610 0x7fad4c06f730 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fad4c074050 tx=0x7fad4801c9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6372e4568a3b8b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:56.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.729+0000 7fad50d1a640 1 -- 172.21.15.139:0/2460839249 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad480a5780 con 0x7fad4c160610 2024-09-19T08:37:56.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.729+0000 7fad50d1a640 1 -- 172.21.15.139:0/2460839249 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad480a5920 con 0x7fad4c160610 2024-09-19T08:37:56.733 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.729+0000 7fad50d1a640 1 -- 172.21.15.139:0/2460839249 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad480a5cf0 con 0x7fad4c160610 2024-09-19T08:37:56.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.729+0000 7fad53f80640 1 -- 172.21.15.139:0/2460839249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad4c160610 msgr2=0x7fad4c06f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:56.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.729+0000 7fad53f80640 1 --2- 172.21.15.139:0/2460839249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad4c160610 0x7fad4c06f730 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fad4c074050 tx=0x7fad4801c9b0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.729+0000 7fad53f80640 1 -- 172.21.15.139:0/2460839249 shutdown_connections 2024-09-19T08:37:56.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.729+0000 7fad53f80640 1 --2- 172.21.15.139:0/2460839249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad4c160610 0x7fad4c06f730 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.729+0000 7fad53f80640 1 --2- 172.21.15.139:0/2460839249 >> v2:172.21.15.98:3300/0 conn(0x7fad4c156ae0 0x7fad4c0ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.729+0000 7fad53f80640 1 -- 172.21.15.139:0/2460839249 >> 172.21.15.139:0/2460839249 conn(0x7fad4c0d0a50 msgr2=0x7fad4c0d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:56.734 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.733+0000 7fad53f80640 1 -- 172.21.15.139:0/2460839249 shutdown_connections 2024-09-19T08:37:56.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.733+0000 7fad53f80640 1 -- 172.21.15.139:0/2460839249 wait complete. 2024-09-19T08:37:56.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.733+0000 7fad53f80640 1 Processor -- start 2024-09-19T08:37:56.735 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.733+0000 7fad53f80640 1 -- start start 2024-09-19T08:37:56.737 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.733+0000 7fad53f80640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad4c0cb0a0 0x7fad4c0b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:56.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.733+0000 7fad53f80640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad4c156ae0 0x7fad4c145a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:56.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.733+0000 7fad53f80640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad4c160610 0x7fad4c1540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:56.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.733+0000 7fad53f80640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c15d2a0 con 0x7fad4c156ae0 2024-09-19T08:37:56.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.733+0000 7fad53f80640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fad4c15d120 con 0x7fad4c0cb0a0 2024-09-19T08:37:56.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.733+0000 7fad53f80640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fad4c15d420 con 0x7fad4c160610 2024-09-19T08:37:56.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.733+0000 7fad51d1c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad4c0cb0a0 0x7fad4c0b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:56.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.733+0000 7fad51d1c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad4c0cb0a0 0x7fad4c0b6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60878/0 (socket says 172.21.15.139:60878) 2024-09-19T08:37:56.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.733+0000 7fad51d1c640 1 -- 172.21.15.139:0/213927936 learned_addr learned my addr 172.21.15.139:0/213927936 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-19T08:37:56.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.733+0000 7fad5251d640 1 --2- 172.21.15.139:0/213927936 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad4c160610 0x7fad4c1540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:56.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.733+0000 7fad5151b640 1 --2- 172.21.15.139:0/213927936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad4c156ae0 0x7fad4c145a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:56.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.733+0000 7fad51d1c640 1 -- 172.21.15.139:0/213927936 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad4c160610 msgr2=0x7fad4c1540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:56.738 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.733+0000 7fad51d1c640 1 --2- 172.21.15.139:0/213927936 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad4c160610 0x7fad4c1540d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.733+0000 7fad51d1c640 1 -- 172.21.15.139:0/213927936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad4c156ae0 msgr2=0x7fad4c145a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:56.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.733+0000 7fad51d1c640 1 --2- 172.21.15.139:0/213927936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad4c156ae0 0x7fad4c145a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.733+0000 7fad51d1c640 1 -- 172.21.15.139:0/213927936 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad4c0b5cf0 con 0x7fad4c0cb0a0 2024-09-19T08:37:56.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.733+0000 7fad5151b640 1 --2- 172.21.15.139:0/213927936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad4c156ae0 0x7fad4c145a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:37:56.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.733+0000 7fad51d1c640 1 --2- 172.21.15.139:0/213927936 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad4c0cb0a0 0x7fad4c0b6120 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fad4400c7f0 tx=0x7fad4400ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:56.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.733+0000 7fad427fc640 1 -- 172.21.15.139:0/213927936 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad44015070 con 0x7fad4c0cb0a0 2024-09-19T08:37:56.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.733+0000 7fad427fc640 1 -- 172.21.15.139:0/213927936 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad44004240 con 0x7fad4c0cb0a0 2024-09-19T08:37:56.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.733+0000 7fad53f80640 1 -- 172.21.15.139:0/213927936 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad4c0b5a10 con 0x7fad4c0cb0a0 2024-09-19T08:37:56.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.733+0000 7fad427fc640 1 -- 172.21.15.139:0/213927936 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad44003620 con 0x7fad4c0cb0a0 2024-09-19T08:37:56.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.733+0000 7fad53f80640 1 -- 172.21.15.139:0/213927936 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad4c14e6e0 con 0x7fad4c0cb0a0 2024-09-19T08:37:56.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.733+0000 7fad427fc640 1 -- 172.21.15.139:0/213927936 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fad440039d0 con 0x7fad4c0cb0a0 2024-09-19T08:37:56.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.733+0000 7fad427fc640 1 -- 172.21.15.139:0/213927936 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fad440198e0 con 0x7fad4c0cb0a0 2024-09-19T08:37:56.739 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.733+0000 7fad53f80640 1 -- 172.21.15.139:0/213927936 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad4c0ccc50 con 0x7fad4c0cb0a0 2024-09-19T08:37:56.740 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.737+0000 7fad427fc640 1 -- 172.21.15.139:0/213927936 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fad44019bc0 con 0x7fad4c0cb0a0 2024-09-19T08:37:56.808 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.805+0000 7fad53f80640 1 -- 172.21.15.139:0/213927936 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd new", "uuid": "cf08f97e-9975-4b66-ac1a-7c6eb6a24e12", "id": 3} v 0) -- 0x7fad4c0b6820 con 0x7fad4c0cb0a0 2024-09-19T08:37:56.815 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.813+0000 7fad427fc640 1 -- 172.21.15.139:0/213927936 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cf08f97e-9975-4b66-ac1a-7c6eb6a24e12", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fad44019450 con 0x7fad4c0cb0a0 2024-09-19T08:37:56.815 INFO:teuthology.orchestra.run.smithi139.stdout:3 2024-09-19T08:37:56.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.813+0000 7fad53f80640 1 -- 172.21.15.139:0/213927936 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad4c0cb0a0 msgr2=0x7fad4c0b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:56.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.813+0000 7fad53f80640 1 --2- 172.21.15.139:0/213927936 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad4c0cb0a0 0x7fad4c0b6120 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fad4400c7f0 tx=0x7fad4400ccb0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.813+0000 7fad53f80640 1 -- 172.21.15.139:0/213927936 shutdown_connections 2024-09-19T08:37:56.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.813+0000 7fad53f80640 1 --2- 172.21.15.139:0/213927936 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad4c160610 0x7fad4c1540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.816 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.813+0000 7fad53f80640 1 --2- 172.21.15.139:0/213927936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad4c156ae0 0x7fad4c145a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.813+0000 7fad53f80640 1 --2- 172.21.15.139:0/213927936 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad4c0cb0a0 0x7fad4c0b6120 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.813+0000 7fad53f80640 1 -- 172.21.15.139:0/213927936 >> 172.21.15.139:0/213927936 conn(0x7fad4c0d0a50 msgr2=0x7fad4c15da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:56.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.813+0000 7fad53f80640 1 -- 172.21.15.139:0/213927936 shutdown_connections 2024-09-19T08:37:56.817 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.813+0000 7fad53f80640 1 -- 172.21.15.139:0/213927936 wait complete. 2024-09-19T08:37:56.832 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --cluster ceph osd new af96ffe3-e63d-48be-a561-6e3d1bf005c3 4 2024-09-19T08:37:56.934 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:37:56.929+0000 7ff7b5cad180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T08:37:56.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.941+0000 7efe51e78640 1 Processor -- start 2024-09-19T08:37:56.944 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.941+0000 7efe51e78640 1 -- start start 2024-09-19T08:37:56.944 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:37:56.942+0000 7f3765211180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T08:37:56.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.941+0000 7efe51e78640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe4c15b4f0 0x7efe4c0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:56.946 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.941+0000 7efe51e78640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7efe4c06cb90 0x7efe4c0b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:56.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.941+0000 7efe51e78640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe4c0af4a0 con 0x7efe4c0b0a70 2024-09-19T08:37:56.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.941+0000 7efe51e78640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7efe4c0af780 con 0x7efe4c06cb90 2024-09-19T08:37:56.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.941+0000 7efe51e78640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7efe4c0afa60 con 0x7efe4c15b4f0 2024-09-19T08:37:56.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.941+0000 7efe4b7fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe4c15b4f0 0x7efe4c0b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:56.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.941+0000 7efe4b7fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe4c15b4f0 0x7efe4c0b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.139:57662/0 (socket says 172.21.15.139:57662) 2024-09-19T08:37:56.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.941+0000 7efe4b7fe640 1 -- 172.21.15.139:0/3796802398 learned_addr learned my addr 172.21.15.139:0/3796802398 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-19T08:37:56.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.941+0000 7efe4bfff640 1 --2- 172.21.15.139:0/3796802398 >> v2:172.21.15.98:3300/0 conn(0x7efe4c06cb90 0x7efe4c0b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:56.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.941+0000 7efe4b7fe640 1 -- 172.21.15.139:0/3796802398 >> v2:172.21.15.98:3300/0 conn(0x7efe4c06cb90 msgr2=0x7efe4c0b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:37:56.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.941+0000 7efe4b7fe640 1 --2- 172.21.15.139:0/3796802398 >> v2:172.21.15.98:3300/0 conn(0x7efe4c06cb90 0x7efe4c0b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.941+0000 7efe4b7fe640 1 -- 172.21.15.139:0/3796802398 >> v1:172.21.15.98:6789/0 conn(0x7efe4c0b0a70 legacy=0x7efe4c06c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:56.947 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.941+0000 7efe4b7fe640 1 -- 172.21.15.139:0/3796802398 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe4c0afd40 con 0x7efe4c15b4f0 2024-09-19T08:37:56.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.945+0000 7efe4b7fe640 1 --2- 172.21.15.139:0/3796802398 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe4c15b4f0 0x7efe4c0b0530 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efe3c00bb10 tx=0x7efe3c021420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fb2f1e25b818733 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:56.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.945+0000 7efe4a7fc640 1 -- 172.21.15.139:0/3796802398 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3c0b6070 con 0x7efe4c15b4f0 2024-09-19T08:37:56.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.945+0000 7efe4a7fc640 1 -- 172.21.15.139:0/3796802398 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe3c0b1600 con 0x7efe4c15b4f0 2024-09-19T08:37:56.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.945+0000 7efe4a7fc640 1 -- 172.21.15.139:0/3796802398 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3c0b19d0 con 0x7efe4c15b4f0 2024-09-19T08:37:56.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.945+0000 7efe51e78640 1 -- 172.21.15.139:0/3796802398 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe4c15b4f0 msgr2=0x7efe4c0b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:56.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.945+0000 7efe51e78640 1 --2- 172.21.15.139:0/3796802398 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe4c15b4f0 0x7efe4c0b0530 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efe3c00bb10 tx=0x7efe3c021420 comp rx=0 tx=0).stop 2024-09-19T08:37:56.949 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.945+0000 7efe51e78640 1 -- 172.21.15.139:0/3796802398 shutdown_connections 2024-09-19T08:37:56.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.945+0000 7efe51e78640 1 --2- 172.21.15.139:0/3796802398 >> v2:172.21.15.98:3300/0 conn(0x7efe4c06cb90 0x7efe4c0b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.945+0000 7efe51e78640 1 --2- 172.21.15.139:0/3796802398 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe4c15b4f0 0x7efe4c0b0530 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.945+0000 7efe51e78640 1 -- 172.21.15.139:0/3796802398 >> 172.21.15.139:0/3796802398 conn(0x7efe4c0b5f50 msgr2=0x7efe4c0b6650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:56.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.945+0000 7efe51e78640 1 -- 172.21.15.139:0/3796802398 shutdown_connections 2024-09-19T08:37:56.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.945+0000 7efe51e78640 1 -- 172.21.15.139:0/3796802398 wait complete. 2024-09-19T08:37:56.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.945+0000 7efe51e78640 1 Processor -- start 2024-09-19T08:37:56.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.945+0000 7efe51e78640 1 -- start start 2024-09-19T08:37:56.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.949+0000 7efe51e78640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe4c06cb90 0x7efe4c09ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:56.952 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.949+0000 7efe51e78640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe4c0b0a70 0x7efe4c1482f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:56.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.949+0000 7efe51e78640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe4c15b4f0 0x7efe4c158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:56.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.949+0000 7efe51e78640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c0af0e0 con 0x7efe4c15b4f0 2024-09-19T08:37:56.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.949+0000 7efe51e78640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efe4c15b8d0 con 0x7efe4c06cb90 2024-09-19T08:37:56.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.949+0000 7efe51e78640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7efe4c0af260 con 0x7efe4c0b0a70 2024-09-19T08:37:56.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.949+0000 7efe4b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe4c06cb90 0x7efe4c09ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:56.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.949+0000 7efe4affd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe4c0b0a70 0x7efe4c1482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:56.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.949+0000 7efe4b7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe4c06cb90 0x7efe4c09ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60886/0 (socket says 172.21.15.139:60886) 2024-09-19T08:37:56.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.949+0000 7efe4b7fe640 1 -- 172.21.15.139:0/347848604 learned_addr learned my addr 172.21.15.139:0/347848604 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-19T08:37:56.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.949+0000 7efe4bfff640 1 --2- 172.21.15.139:0/347848604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe4c15b4f0 0x7efe4c158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:56.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.949+0000 7efe4affd640 1 -- 172.21.15.139:0/347848604 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe4c06cb90 msgr2=0x7efe4c09ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:56.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.949+0000 7efe4affd640 1 --2- 172.21.15.139:0/347848604 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe4c06cb90 0x7efe4c09ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.949+0000 7efe4affd640 1 -- 172.21.15.139:0/347848604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe4c15b4f0 msgr2=0x7efe4c158360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:56.953 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.949+0000 7efe4affd640 1 --2- 172.21.15.139:0/347848604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe4c15b4f0 0x7efe4c158360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:56.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.949+0000 7efe4affd640 1 -- 172.21.15.139:0/347848604 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe4c06e930 con 0x7efe4c0b0a70 2024-09-19T08:37:56.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.949+0000 7efe4bfff640 1 --2- 172.21.15.139:0/347848604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe4c15b4f0 0x7efe4c158360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:37:56.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.949+0000 7efe4affd640 1 --2- 172.21.15.139:0/347848604 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe4c0b0a70 0x7efe4c1482f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efe4c0b0e50 tx=0x7efe4000c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:56.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.949+0000 7efe33fff640 1 -- 172.21.15.139:0/347848604 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe400052d0 con 0x7efe4c0b0a70 2024-09-19T08:37:56.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.949+0000 7efe33fff640 1 -- 172.21.15.139:0/347848604 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe40005470 con 0x7efe4c0b0a70 2024-09-19T08:37:56.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.949+0000 7efe33fff640 1 -- 172.21.15.139:0/347848604 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe40007260 con 0x7efe4c0b0a70 2024-09-19T08:37:56.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.949+0000 7efe51e78640 1 -- 172.21.15.139:0/347848604 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe4c1f45c0 con 0x7efe4c0b0a70 2024-09-19T08:37:56.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.949+0000 7efe51e78640 1 -- 172.21.15.139:0/347848604 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efe4c15bb80 con 0x7efe4c0b0a70 2024-09-19T08:37:56.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.949+0000 7efe33fff640 1 -- 172.21.15.139:0/347848604 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efe40007400 con 0x7efe4c0b0a70 2024-09-19T08:37:56.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.949+0000 7efe33fff640 1 -- 172.21.15.139:0/347848604 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7efe40011590 con 0x7efe4c0b0a70 2024-09-19T08:37:56.954 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.949+0000 7efe51e78640 1 -- 172.21.15.139:0/347848604 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe4c0b0530 con 0x7efe4c0b0a70 2024-09-19T08:37:56.955 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:56.953+0000 7efe33fff640 1 -- 172.21.15.139:0/347848604 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7efe40006970 con 0x7efe4c0b0a70 2024-09-19T08:37:56.990 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:37:56.989+0000 7ff7b5cad180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T08:37:56.998 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:37:56.994+0000 7f3765211180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T08:37:57.023 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.021+0000 7efe51e78640 1 -- 172.21.15.139:0/347848604 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "osd new", "uuid": "af96ffe3-e63d-48be-a561-6e3d1bf005c3", "id": 4} v 0) -- 0x7efe4c0b2460 con 0x7efe4c0b0a70 2024-09-19T08:37:57.030 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.025+0000 7efe33fff640 1 -- 172.21.15.139:0/347848604 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "af96ffe3-e63d-48be-a561-6e3d1bf005c3", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7efe40003600 con 0x7efe4c0b0a70 2024-09-19T08:37:57.030 INFO:teuthology.orchestra.run.smithi139.stdout:4 2024-09-19T08:37:57.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.029+0000 7efe51e78640 1 -- 172.21.15.139:0/347848604 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe4c0b0a70 msgr2=0x7efe4c1482f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:57.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.029+0000 7efe51e78640 1 --2- 172.21.15.139:0/347848604 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe4c0b0a70 0x7efe4c1482f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efe4c0b0e50 tx=0x7efe4000c820 comp rx=0 tx=0).stop 2024-09-19T08:37:57.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.029+0000 7efe51e78640 1 -- 172.21.15.139:0/347848604 shutdown_connections 2024-09-19T08:37:57.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.029+0000 7efe51e78640 1 --2- 172.21.15.139:0/347848604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe4c15b4f0 0x7efe4c158360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.031 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.029+0000 7efe51e78640 1 --2- 172.21.15.139:0/347848604 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe4c0b0a70 0x7efe4c1482f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.029+0000 7efe51e78640 1 --2- 172.21.15.139:0/347848604 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe4c06cb90 0x7efe4c09ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.029+0000 7efe51e78640 1 -- 172.21.15.139:0/347848604 >> 172.21.15.139:0/347848604 conn(0x7efe4c0b5f50 msgr2=0x7efe4c06e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:57.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.029+0000 7efe51e78640 1 -- 172.21.15.139:0/347848604 shutdown_connections 2024-09-19T08:37:57.032 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.029+0000 7efe51e78640 1 -- 172.21.15.139:0/347848604 wait complete. 2024-09-19T08:37:57.045 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:37:57.041+0000 7ff7b5cad180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T08:37:57.046 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --cluster ceph osd new 6225119f-e471-4453-a3c5-853dd8a58eb4 5 2024-09-19T08:37:57.051 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:37:57.050+0000 7f3765211180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T08:37:57.125 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:37:57.121+0000 7ff7b5cad180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T08:37:57.128 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:37:57.126+0000 7f3765211180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T08:37:57.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.157+0000 7f10ece07640 1 Processor -- start 2024-09-19T08:37:57.159 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.157+0000 7f10ece07640 1 -- start start 2024-09-19T08:37:57.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.157+0000 7f10ece07640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f10e80d1e20 0x7f10e806c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:57.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.157+0000 7f10ece07640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f10e815c9b0 0x7f10e815ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:57.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.157+0000 7f10ece07640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10e806c5b0 con 0x7f10e806bc90 2024-09-19T08:37:57.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.157+0000 7f10ece07640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f10e80d1360 con 0x7f10e815c9b0 2024-09-19T08:37:57.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.157+0000 7f10ece07640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f10e806d590 con 0x7f10e80d1e20 2024-09-19T08:37:57.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.157+0000 7f10e5d9b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f10e80d1e20 0x7f10e806c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:57.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.157+0000 7f10e5d9b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f10e80d1e20 0x7f10e806c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60900/0 (socket says 172.21.15.139:60900) 2024-09-19T08:37:57.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.157+0000 7f10e5d9b640 1 -- 172.21.15.139:0/3251806838 learned_addr learned my addr 172.21.15.139:0/3251806838 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-19T08:37:57.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.157+0000 7f10e6d9d640 1 --2- 172.21.15.139:0/3251806838 >> v2:172.21.15.98:3300/0 conn(0x7f10e815c9b0 0x7f10e815ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:57.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.157+0000 7f10e5d9b640 1 -- 172.21.15.139:0/3251806838 >> v2:172.21.15.98:3300/0 conn(0x7f10e815c9b0 msgr2=0x7f10e815ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:37:57.161 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.157+0000 7f10e5d9b640 1 --2- 172.21.15.139:0/3251806838 >> v2:172.21.15.98:3300/0 conn(0x7f10e815c9b0 0x7f10e815ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.157+0000 7f10e5d9b640 1 -- 172.21.15.139:0/3251806838 >> v1:172.21.15.98:6789/0 conn(0x7f10e806bc90 legacy=0x7f10e815c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:37:57.162 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.157+0000 7f10e5d9b640 1 -- 172.21.15.139:0/3251806838 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e80d1620 con 0x7f10e80d1e20 2024-09-19T08:37:57.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.161+0000 7f10e5d9b640 1 --2- 172.21.15.139:0/3251806838 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10e80d1e20 0x7f10e806c070 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f10e8071e50 tx=0x7f10dc01c4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dad2525f60f61f0d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:57.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.161+0000 7f10e559a640 1 -- 172.21.15.139:0/3251806838 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc0ab5f0 con 0x7f10e80d1e20 2024-09-19T08:37:57.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.161+0000 7f10e559a640 1 -- 172.21.15.139:0/3251806838 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10dc0b0070 con 0x7f10e80d1e20 2024-09-19T08:37:57.163 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.161+0000 7f10e559a640 1 -- 172.21.15.139:0/3251806838 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc0ab9b0 con 0x7f10e80d1e20 2024-09-19T08:37:57.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.161+0000 7f10ece07640 1 -- 172.21.15.139:0/3251806838 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10e80d1e20 msgr2=0x7f10e806c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:57.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.161+0000 7f10ece07640 1 --2- 172.21.15.139:0/3251806838 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10e80d1e20 0x7f10e806c070 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f10e8071e50 tx=0x7f10dc01c4e0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.161+0000 7f10ece07640 1 -- 172.21.15.139:0/3251806838 shutdown_connections 2024-09-19T08:37:57.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.161+0000 7f10ece07640 1 --2- 172.21.15.139:0/3251806838 >> v2:172.21.15.98:3300/0 conn(0x7f10e815c9b0 0x7f10e815ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.164 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.161+0000 7f10ece07640 1 --2- 172.21.15.139:0/3251806838 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10e80d1e20 0x7f10e806c070 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.161+0000 7f10ece07640 1 -- 172.21.15.139:0/3251806838 >> 172.21.15.139:0/3251806838 conn(0x7f10e80b4d60 msgr2=0x7f10e80b5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:57.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.161+0000 7f10ece07640 1 -- 172.21.15.139:0/3251806838 shutdown_connections 2024-09-19T08:37:57.165 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.161+0000 7f10ece07640 1 -- 172.21.15.139:0/3251806838 wait complete. 2024-09-19T08:37:57.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.161+0000 7f10ece07640 1 Processor -- start 2024-09-19T08:37:57.166 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.161+0000 7f10ece07640 1 -- start start 2024-09-19T08:37:57.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.165+0000 7f10ece07640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10e806bc90 0x7f10e8158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:57.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.165+0000 7f10ece07640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10e80d1e20 0x7f10e8074ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:57.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.165+0000 7f10ece07640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10e815c9b0 0x7f10e814a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:57.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.165+0000 7f10ece07640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f10e8155cc0 con 0x7f10e80d1e20 2024-09-19T08:37:57.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.165+0000 7f10ece07640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f10e8155b40 con 0x7f10e806bc90 2024-09-19T08:37:57.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.165+0000 7f10ece07640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f10e8168640 con 0x7f10e815c9b0 2024-09-19T08:37:57.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.165+0000 7f10e6d9d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10e815c9b0 0x7f10e814a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:57.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.165+0000 7f10e659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10e806bc90 0x7f10e8158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:57.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.165+0000 7f10e659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10e806bc90 0x7f10e8158a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60906/0 (socket says 172.21.15.139:60906) 2024-09-19T08:37:57.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.165+0000 7f10e6d9d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10e815c9b0 0x7f10e814a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.139:57684/0 (socket says 172.21.15.139:57684) 2024-09-19T08:37:57.169 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.165+0000 7f10e659c640 1 -- 172.21.15.139:0/2039874605 learned_addr learned my addr 172.21.15.139:0/2039874605 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-19T08:37:57.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.165+0000 7f10e5d9b640 1 --2- 172.21.15.139:0/2039874605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10e80d1e20 0x7f10e8074ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:57.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.165+0000 7f10e659c640 1 -- 172.21.15.139:0/2039874605 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10e815c9b0 msgr2=0x7f10e814a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:57.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.165+0000 7f10e659c640 1 --2- 172.21.15.139:0/2039874605 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10e815c9b0 0x7f10e814a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.165+0000 7f10e659c640 1 -- 172.21.15.139:0/2039874605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10e80d1e20 msgr2=0x7f10e8074ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:57.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.165+0000 7f10e659c640 1 --2- 172.21.15.139:0/2039874605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10e80d1e20 0x7f10e8074ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.165+0000 7f10e659c640 1 -- 172.21.15.139:0/2039874605 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e8154530 con 0x7f10e806bc90 2024-09-19T08:37:57.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.165+0000 7f10e6d9d640 1 --2- 172.21.15.139:0/2039874605 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10e815c9b0 0x7f10e814a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:37:57.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.165+0000 7f10e5d9b640 1 --2- 172.21.15.139:0/2039874605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10e80d1e20 0x7f10e8074ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:37:57.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.165+0000 7f10e659c640 1 --2- 172.21.15.139:0/2039874605 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10e806bc90 0x7f10e8158a10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f10d800b410 tx=0x7f10d800b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:57.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.165+0000 7f10d6ffd640 1 -- 172.21.15.139:0/2039874605 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8015030 con 0x7f10e806bc90 2024-09-19T08:37:57.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.165+0000 7f10d6ffd640 1 -- 172.21.15.139:0/2039874605 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10d8004020 con 0x7f10e806bc90 2024-09-19T08:37:57.170 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.165+0000 7f10ece07640 1 -- 172.21.15.139:0/2039874605 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e80d0520 con 0x7f10e806bc90 2024-09-19T08:37:57.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.165+0000 7f10d6ffd640 1 -- 172.21.15.139:0/2039874605 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8004450 con 0x7f10e806bc90 2024-09-19T08:37:57.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.165+0000 7f10ece07640 1 -- 172.21.15.139:0/2039874605 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e8075410 con 0x7f10e806bc90 2024-09-19T08:37:57.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.165+0000 7f10d6ffd640 1 -- 172.21.15.139:0/2039874605 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f10d8018070 con 0x7f10e806bc90 2024-09-19T08:37:57.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.165+0000 7f10d6ffd640 1 -- 172.21.15.139:0/2039874605 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f10d801a410 con 0x7f10e806bc90 2024-09-19T08:37:57.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.165+0000 7f10ece07640 1 -- 172.21.15.139:0/2039874605 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10e806c070 con 0x7f10e806bc90 2024-09-19T08:37:57.171 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.169+0000 7f10d6ffd640 1 -- 172.21.15.139:0/2039874605 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f10d8003620 con 0x7f10e806bc90 2024-09-19T08:37:57.180 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:37:57.178+0000 7f3765211180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T08:37:57.181 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:37:57.177+0000 7ff7b5cad180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T08:37:57.240 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.237+0000 7f10ece07640 1 -- 172.21.15.139:0/2039874605 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6225119f-e471-4453-a3c5-853dd8a58eb4", "id": 5} v 0) -- 0x7f10e8156290 con 0x7f10e806bc90 2024-09-19T08:37:57.247 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.245+0000 7f10d6ffd640 1 -- 172.21.15.139:0/2039874605 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6225119f-e471-4453-a3c5-853dd8a58eb4", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f10d8003bb0 con 0x7f10e806bc90 2024-09-19T08:37:57.247 INFO:teuthology.orchestra.run.smithi139.stdout:5 2024-09-19T08:37:57.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.245+0000 7f10ece07640 1 -- 172.21.15.139:0/2039874605 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10e806bc90 msgr2=0x7f10e8158a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:57.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.245+0000 7f10ece07640 1 --2- 172.21.15.139:0/2039874605 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10e806bc90 0x7f10e8158a10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f10d800b410 tx=0x7f10d800b8d0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.245+0000 7f10ece07640 1 -- 172.21.15.139:0/2039874605 shutdown_connections 2024-09-19T08:37:57.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.245+0000 7f10ece07640 1 --2- 172.21.15.139:0/2039874605 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10e815c9b0 0x7f10e814a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.245+0000 7f10ece07640 1 --2- 172.21.15.139:0/2039874605 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10e80d1e20 0x7f10e8074ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.245+0000 7f10ece07640 1 --2- 172.21.15.139:0/2039874605 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10e806bc90 0x7f10e8158a10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.245+0000 7f10ece07640 1 -- 172.21.15.139:0/2039874605 >> 172.21.15.139:0/2039874605 conn(0x7f10e80b4d60 msgr2=0x7f10e815d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:57.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.245+0000 7f10ece07640 1 -- 172.21.15.139:0/2039874605 shutdown_connections 2024-09-19T08:37:57.249 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.245+0000 7f10ece07640 1 -- 172.21.15.139:0/2039874605 wait complete. 2024-09-19T08:37:57.257 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:37:57.254+0000 7f3765211180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T08:37:57.263 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --cluster ceph osd new da297124-d02d-4cba-ba1c-9a6f53c62929 6 2024-09-19T08:37:57.265 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:37:57.261+0000 7ff7b5cad180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T08:37:57.313 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:37:57.310+0000 7f3765211180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T08:37:57.320 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:37:57.317+0000 7ff7b5cad180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T08:37:57.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.377+0000 7fdba3095640 1 Processor -- start 2024-09-19T08:37:57.380 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.377+0000 7fdba3095640 1 -- start start 2024-09-19T08:37:57.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.377+0000 7fdba3095640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fdb9c158db0 0x7fdb9c15f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:57.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.377+0000 7fdba3095640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb9c0b0620 0x7fdb9c074c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:57.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.377+0000 7fdba3095640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdb9c0751b0 0x7fdb9c1622e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:57.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.377+0000 7fdba3095640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fdb9c075ad0 con 0x7fdb9c158db0 2024-09-19T08:37:57.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.377+0000 7fdba3095640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fdb9c06bfa0 con 0x7fdb9c0751b0 2024-09-19T08:37:57.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.377+0000 7fdba3095640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdb9c0ce710 con 0x7fdb9c0b0620 2024-09-19T08:37:57.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.377+0000 7fdb9bfff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb9c0b0620 0x7fdb9c074c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:57.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.377+0000 7fdba1632640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdb9c0751b0 0x7fdb9c1622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:57.382 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.377+0000 7fdb9bfff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb9c0b0620 0x7fdb9c074c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.139:57688/0 (socket says 172.21.15.139:57688) 2024-09-19T08:37:57.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.377+0000 7fdb9bfff640 1 -- 172.21.15.139:0/4076535499 learned_addr learned my addr 172.21.15.139:0/4076535499 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-19T08:37:57.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.377+0000 7fdba0e31640 1 --2- 172.21.15.139:0/4076535499 >> v2:172.21.15.98:3300/0 conn(0x7fdb9c158db0 0x7fdb9c15f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:57.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.377+0000 7fdba1632640 1 -- 172.21.15.139:0/4076535499 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb9c0b0620 msgr2=0x7fdb9c074c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:37:57.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.377+0000 7fdba1632640 1 --2- 172.21.15.139:0/4076535499 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb9c0b0620 0x7fdb9c074c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.377+0000 7fdba1632640 1 -- 172.21.15.139:0/4076535499 >> v2:172.21.15.98:3300/0 conn(0x7fdb9c158db0 msgr2=0x7fdb9c15f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:37:57.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.377+0000 7fdba1632640 1 --2- 172.21.15.139:0/4076535499 >> v2:172.21.15.98:3300/0 conn(0x7fdb9c158db0 0x7fdb9c15f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.377+0000 7fdba1632640 1 -- 172.21.15.139:0/4076535499 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9c159190 con 0x7fdb9c0751b0 2024-09-19T08:37:57.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.377+0000 7fdb9bfff640 1 --2- 172.21.15.139:0/4076535499 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb9c0b0620 0x7fdb9c074c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:37:57.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.381+0000 7fdba1632640 1 --2- 172.21.15.139:0/4076535499 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb9c0751b0 0x7fdb9c1622e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c0718f0 tx=0x7fdb9401c670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5377f84684ec2f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:57.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.381+0000 7fdb9b7fe640 1 -- 172.21.15.139:0/4076535499 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb940ad070 con 0x7fdb9c0751b0 2024-09-19T08:37:57.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.381+0000 7fdb9b7fe640 1 -- 172.21.15.139:0/4076535499 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb940a9400 con 0x7fdb9c0751b0 2024-09-19T08:37:57.384 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.381+0000 7fdb9b7fe640 1 -- 172.21.15.139:0/4076535499 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb940a97d0 con 0x7fdb9c0751b0 2024-09-19T08:37:57.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.381+0000 7fdba3095640 1 -- 172.21.15.139:0/4076535499 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb9c0751b0 msgr2=0x7fdb9c1622e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:57.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.381+0000 7fdba3095640 1 --2- 172.21.15.139:0/4076535499 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb9c0751b0 0x7fdb9c1622e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c0718f0 tx=0x7fdb9401c670 comp rx=0 tx=0).stop 2024-09-19T08:37:57.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.381+0000 7fdba3095640 1 -- 172.21.15.139:0/4076535499 shutdown_connections 2024-09-19T08:37:57.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.381+0000 7fdba3095640 1 --2- 172.21.15.139:0/4076535499 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb9c0751b0 0x7fdb9c1622e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.381+0000 7fdba3095640 1 --2- 172.21.15.139:0/4076535499 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb9c0b0620 0x7fdb9c074c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.381+0000 7fdba3095640 1 --2- 172.21.15.139:0/4076535499 >> v2:172.21.15.98:3300/0 conn(0x7fdb9c158db0 0x7fdb9c15f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.385 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.381+0000 7fdba3095640 1 -- 172.21.15.139:0/4076535499 >> 172.21.15.139:0/4076535499 conn(0x7fdb9c0b5f50 msgr2=0x7fdb9c0b6650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:57.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.381+0000 7fdba3095640 1 -- 172.21.15.139:0/4076535499 shutdown_connections 2024-09-19T08:37:57.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.381+0000 7fdba3095640 1 -- 172.21.15.139:0/4076535499 wait complete. 2024-09-19T08:37:57.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.385+0000 7fdba3095640 1 Processor -- start 2024-09-19T08:37:57.386 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.385+0000 7fdba3095640 1 -- start start 2024-09-19T08:37:57.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.385+0000 7fdba3095640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb9c0751b0 0x7fdb9c148230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:57.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.385+0000 7fdba3095640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb9c0b0620 0x7fdb9c0cf180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:57.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.385+0000 7fdba3095640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb9c158db0 0x7fdb9c0b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:57.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.385+0000 7fdba3095640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c06c7a0 con 0x7fdb9c0751b0 2024-09-19T08:37:57.388 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.385+0000 7fdba3095640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c06c620 con 0x7fdb9c0b0620 2024-09-19T08:37:57.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.385+0000 7fdba3095640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdb9c06c920 con 0x7fdb9c158db0 2024-09-19T08:37:57.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.385+0000 7fdb9bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb9c0b0620 0x7fdb9c0cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:57.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.385+0000 7fdb9bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb9c0b0620 0x7fdb9c0cf180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60918/0 (socket says 172.21.15.139:60918) 2024-09-19T08:37:57.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.385+0000 7fdb9bfff640 1 -- 172.21.15.139:0/1921073571 learned_addr learned my addr 172.21.15.139:0/1921073571 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-19T08:37:57.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.385+0000 7fdba1632640 1 --2- 172.21.15.139:0/1921073571 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb9c158db0 0x7fdb9c0b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:57.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.385+0000 7fdba0e31640 1 --2- 172.21.15.139:0/1921073571 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb9c0751b0 0x7fdb9c148230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:57.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.385+0000 7fdb9bfff640 1 -- 172.21.15.139:0/1921073571 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb9c158db0 msgr2=0x7fdb9c0b11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:57.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.385+0000 7fdb9bfff640 1 --2- 172.21.15.139:0/1921073571 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb9c158db0 0x7fdb9c0b11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.385+0000 7fdb9bfff640 1 -- 172.21.15.139:0/1921073571 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb9c0751b0 msgr2=0x7fdb9c148230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:57.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.385+0000 7fdb9bfff640 1 --2- 172.21.15.139:0/1921073571 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb9c0751b0 0x7fdb9c148230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.385+0000 7fdb9bfff640 1 -- 172.21.15.139:0/1921073571 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9c0cf9b0 con 0x7fdb9c0b0620 2024-09-19T08:37:57.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.385+0000 7fdba0e31640 1 --2- 172.21.15.139:0/1921073571 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb9c0751b0 0x7fdb9c148230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:37:57.389 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.385+0000 7fdb9bfff640 1 --2- 172.21.15.139:0/1921073571 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb9c0b0620 0x7fdb9c0cf180 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00e980 tx=0x7fdb8c00ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:57.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.385+0000 7fdb997fa640 1 -- 172.21.15.139:0/1921073571 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c004030 con 0x7fdb9c0b0620 2024-09-19T08:37:57.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.385+0000 7fdb997fa640 1 -- 172.21.15.139:0/1921073571 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb8c0041d0 con 0x7fdb9c0b0620 2024-09-19T08:37:57.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.385+0000 7fdb997fa640 1 -- 172.21.15.139:0/1921073571 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c00c9a0 con 0x7fdb9c0b0620 2024-09-19T08:37:57.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.385+0000 7fdba3095640 1 -- 172.21.15.139:0/1921073571 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb9c0cfc90 con 0x7fdb9c0b0620 2024-09-19T08:37:57.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.385+0000 7fdba3095640 1 -- 172.21.15.139:0/1921073571 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb9c0d0250 con 0x7fdb9c0b0620 2024-09-19T08:37:57.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.385+0000 7fdb997fa640 1 -- 172.21.15.139:0/1921073571 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdb8c003620 con 0x7fdb9c0b0620 2024-09-19T08:37:57.390 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.385+0000 7fdb997fa640 1 -- 172.21.15.139:0/1921073571 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fdb8c013d80 con 0x7fdb9c0b0620 2024-09-19T08:37:57.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.385+0000 7fdba3095640 1 -- 172.21.15.139:0/1921073571 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb9c1622e0 con 0x7fdb9c0b0620 2024-09-19T08:37:57.391 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.389+0000 7fdb997fa640 1 -- 172.21.15.139:0/1921073571 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fdb8c017020 con 0x7fdb9c0b0620 2024-09-19T08:37:57.461 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.457+0000 7fdba3095640 1 -- 172.21.15.139:0/1921073571 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd new", "uuid": "da297124-d02d-4cba-ba1c-9a6f53c62929", "id": 6} v 0) -- 0x7fdb9c0ce4f0 con 0x7fdb9c0b0620 2024-09-19T08:37:57.463 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:37:57.462+0000 7f3765211180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T08:37:57.470 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.465+0000 7fdb997fa640 1 -- 172.21.15.139:0/1921073571 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "da297124-d02d-4cba-ba1c-9a6f53c62929", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fdb8c003b40 con 0x7fdb9c0b0620 2024-09-19T08:37:57.470 INFO:teuthology.orchestra.run.smithi139.stdout:6 2024-09-19T08:37:57.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.469+0000 7fdba3095640 1 -- 172.21.15.139:0/1921073571 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb9c0b0620 msgr2=0x7fdb9c0cf180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:57.471 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.469+0000 7fdba3095640 1 --2- 172.21.15.139:0/1921073571 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb9c0b0620 0x7fdb9c0cf180 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00e980 tx=0x7fdb8c00ee40 comp rx=0 tx=0).stop 2024-09-19T08:37:57.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.469+0000 7fdba3095640 1 -- 172.21.15.139:0/1921073571 shutdown_connections 2024-09-19T08:37:57.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.469+0000 7fdba3095640 1 --2- 172.21.15.139:0/1921073571 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb9c158db0 0x7fdb9c0b11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.469+0000 7fdba3095640 1 --2- 172.21.15.139:0/1921073571 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb9c0b0620 0x7fdb9c0cf180 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.469+0000 7fdba3095640 1 --2- 172.21.15.139:0/1921073571 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb9c0751b0 0x7fdb9c148230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.469+0000 7fdba3095640 1 -- 172.21.15.139:0/1921073571 >> 172.21.15.139:0/1921073571 conn(0x7fdb9c0b5f50 msgr2=0x7fdb9c0ae0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:57.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.469+0000 7fdba3095640 1 -- 172.21.15.139:0/1921073571 shutdown_connections 2024-09-19T08:37:57.472 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.469+0000 7fdba3095640 1 -- 172.21.15.139:0/1921073571 wait complete. 2024-09-19T08:37:57.478 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:37:57.477+0000 7ff7b5cad180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T08:37:57.487 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --cluster ceph osd new 4ce2f72e-c241-44c1-b4cc-7616cde855fd 7 2024-09-19T08:37:57.526 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:37:57.522+0000 7f3765211180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T08:37:57.542 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:37:57.541+0000 7ff7b5cad180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T08:37:57.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.601+0000 7f9e2f5d0640 1 Processor -- start 2024-09-19T08:37:57.605 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.601+0000 7f9e2f5d0640 1 -- start start 2024-09-19T08:37:57.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.601+0000 7f9e2f5d0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9e28158db0 0x7f9e2815f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:57.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.601+0000 7f9e2f5d0640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9e280b0620 0x7f9e28074c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:57.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.601+0000 7f9e2f5d0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9e280751b0 0x7f9e281622e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:57.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.601+0000 7f9e2f5d0640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f9e28075ad0 con 0x7f9e280b0620 2024-09-19T08:37:57.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.601+0000 7f9e2f5d0640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f9e2806bfa0 con 0x7f9e280751b0 2024-09-19T08:37:57.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.601+0000 7f9e2f5d0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9e280ce710 con 0x7f9e28158db0 2024-09-19T08:37:57.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.601+0000 7f9e2d36c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9e28158db0 0x7f9e2815f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:57.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.601+0000 7f9e2db6d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9e280751b0 0x7f9e281622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:57.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.605+0000 7f9e2db6d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9e280751b0 0x7f9e281622e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60932/0 (socket says 172.21.15.139:60932) 2024-09-19T08:37:57.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.605+0000 7f9e2d36c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9e28158db0 0x7f9e2815f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.139:57700/0 (socket says 172.21.15.139:57700) 2024-09-19T08:37:57.607 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.605+0000 7f9e2db6d640 1 -- 172.21.15.139:0/2132655858 learned_addr learned my addr 172.21.15.139:0/2132655858 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-19T08:37:57.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.605+0000 7f9e2cb6b640 1 --2- 172.21.15.139:0/2132655858 >> v2:172.21.15.98:3300/0 conn(0x7f9e280b0620 0x7f9e28074c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:57.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.605+0000 7f9e2d36c640 1 -- 172.21.15.139:0/2132655858 >> v2:172.21.15.139:3300/0 conn(0x7f9e280751b0 msgr2=0x7f9e281622e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:37:57.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.605+0000 7f9e2d36c640 1 --2- 172.21.15.139:0/2132655858 >> v2:172.21.15.139:3300/0 conn(0x7f9e280751b0 0x7f9e281622e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.605+0000 7f9e2d36c640 1 -- 172.21.15.139:0/2132655858 >> v2:172.21.15.98:3300/0 conn(0x7f9e280b0620 msgr2=0x7f9e28074c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:37:57.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.605+0000 7f9e2d36c640 1 --2- 172.21.15.139:0/2132655858 >> v2:172.21.15.98:3300/0 conn(0x7f9e280b0620 0x7f9e28074c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.605+0000 7f9e2d36c640 1 -- 172.21.15.139:0/2132655858 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e28159190 con 0x7f9e28158db0 2024-09-19T08:37:57.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.605+0000 7f9e2db6d640 1 --2- 172.21.15.139:0/2132655858 >> v2:172.21.15.139:3300/0 conn(0x7f9e280751b0 0x7f9e281622e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:37:57.608 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.605+0000 7f9e2cb6b640 1 --2- 172.21.15.139:0/2132655858 >> v2:172.21.15.98:3300/0 conn(0x7f9e280b0620 0x7f9e28074c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:37:57.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.605+0000 7f9e2d36c640 1 --2- 172.21.15.139:0/2132655858 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9e28158db0 0x7f9e2815f770 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9e280718f0 tx=0x7f9e1801c1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c4e9e49e9b9fe14 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:57.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.605+0000 7f9e1ffff640 1 -- 172.21.15.139:0/2132655858 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e180b0070 con 0x7f9e28158db0 2024-09-19T08:37:57.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.605+0000 7f9e1ffff640 1 -- 172.21.15.139:0/2132655858 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e180ab600 con 0x7f9e28158db0 2024-09-19T08:37:57.609 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.605+0000 7f9e1ffff640 1 -- 172.21.15.139:0/2132655858 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e180ab9d0 con 0x7f9e28158db0 2024-09-19T08:37:57.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.605+0000 7f9e2f5d0640 1 -- 172.21.15.139:0/2132655858 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9e28158db0 msgr2=0x7f9e2815f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:57.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.605+0000 7f9e2f5d0640 1 --2- 172.21.15.139:0/2132655858 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9e28158db0 0x7f9e2815f770 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9e280718f0 tx=0x7f9e1801c1c0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.605+0000 7f9e2f5d0640 1 -- 172.21.15.139:0/2132655858 shutdown_connections 2024-09-19T08:37:57.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.605+0000 7f9e2f5d0640 1 --2- 172.21.15.139:0/2132655858 >> v2:172.21.15.98:3300/0 conn(0x7f9e280b0620 0x7f9e28074c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.605+0000 7f9e2f5d0640 1 --2- 172.21.15.139:0/2132655858 >> v2:172.21.15.139:3300/0 conn(0x7f9e280751b0 0x7f9e281622e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.605+0000 7f9e2f5d0640 1 --2- 172.21.15.139:0/2132655858 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9e28158db0 0x7f9e2815f770 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.605+0000 7f9e2f5d0640 1 -- 172.21.15.139:0/2132655858 >> 172.21.15.139:0/2132655858 conn(0x7f9e280b5f50 msgr2=0x7f9e280b6650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:57.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.605+0000 7f9e2f5d0640 1 -- 172.21.15.139:0/2132655858 shutdown_connections 2024-09-19T08:37:57.612 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.605+0000 7f9e2f5d0640 1 -- 172.21.15.139:0/2132655858 wait complete. 2024-09-19T08:37:57.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.609+0000 7f9e2f5d0640 1 Processor -- start 2024-09-19T08:37:57.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.609+0000 7f9e2f5d0640 1 -- start start 2024-09-19T08:37:57.614 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.609+0000 7f9e2f5d0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e280751b0 0x7f9e2814f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:57.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.609+0000 7f9e2f5d0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9e280b0620 0x7f9e2814dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:57.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.609+0000 7f9e2f5d0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e28158db0 0x7f9e280d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:57.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.609+0000 7f9e2f5d0640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2806c7c0 con 0x7f9e280751b0 2024-09-19T08:37:57.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.609+0000 7f9e2f5d0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2806c640 con 0x7f9e28158db0 2024-09-19T08:37:57.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.609+0000 7f9e2f5d0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9e2806c940 con 0x7f9e280b0620 2024-09-19T08:37:57.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.609+0000 7f9e2cb6b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9e280b0620 0x7f9e2814dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:57.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.609+0000 7f9e2db6d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e28158db0 0x7f9e280d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:57.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.609+0000 7f9e2cb6b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9e280b0620 0x7f9e2814dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.139:57706/0 (socket says 172.21.15.139:57706) 2024-09-19T08:37:57.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.609+0000 7f9e2db6d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e28158db0 0x7f9e280d09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:60940/0 (socket says 172.21.15.139:60940) 2024-09-19T08:37:57.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.609+0000 7f9e2cb6b640 1 -- 172.21.15.139:0/2965965254 learned_addr learned my addr 172.21.15.139:0/2965965254 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-09-19T08:37:57.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.609+0000 7f9e2d36c640 1 --2- 172.21.15.139:0/2965965254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e280751b0 0x7f9e2814f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:57.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.609+0000 7f9e2db6d640 1 -- 172.21.15.139:0/2965965254 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9e280b0620 msgr2=0x7f9e2814dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:57.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.609+0000 7f9e2db6d640 1 --2- 172.21.15.139:0/2965965254 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9e280b0620 0x7f9e2814dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.609+0000 7f9e2db6d640 1 -- 172.21.15.139:0/2965965254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e280751b0 msgr2=0x7f9e2814f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:57.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.609+0000 7f9e2db6d640 1 --2- 172.21.15.139:0/2965965254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e280751b0 0x7f9e2814f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.609+0000 7f9e2db6d640 1 -- 172.21.15.139:0/2965965254 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e2814fe00 con 0x7f9e28158db0 2024-09-19T08:37:57.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.609+0000 7f9e2cb6b640 1 --2- 172.21.15.139:0/2965965254 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9e280b0620 0x7f9e2814dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:37:57.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.609+0000 7f9e2db6d640 1 --2- 172.21.15.139:0/2965965254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e28158db0 0x7f9e280d09a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9e2000ef00 tx=0x7f9e2000b480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:57.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.609+0000 7f9e1dffb640 1 -- 172.21.15.139:0/2965965254 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2000bab0 con 0x7f9e28158db0 2024-09-19T08:37:57.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.609+0000 7f9e1dffb640 1 -- 172.21.15.139:0/2965965254 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e2000bc70 con 0x7f9e28158db0 2024-09-19T08:37:57.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.609+0000 7f9e1dffb640 1 -- 172.21.15.139:0/2965965254 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e20003be0 con 0x7f9e28158db0 2024-09-19T08:37:57.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.609+0000 7f9e2f5d0640 1 -- 172.21.15.139:0/2965965254 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e2814f780 con 0x7f9e28158db0 2024-09-19T08:37:57.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.609+0000 7f9e2f5d0640 1 -- 172.21.15.139:0/2965965254 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e280d1be0 con 0x7f9e28158db0 2024-09-19T08:37:57.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.609+0000 7f9e1dffb640 1 -- 172.21.15.139:0/2965965254 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9e2001b030 con 0x7f9e28158db0 2024-09-19T08:37:57.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.609+0000 7f9e1dffb640 1 -- 172.21.15.139:0/2965965254 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f9e20014a60 con 0x7f9e28158db0 2024-09-19T08:37:57.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.609+0000 7f9e0b7fe640 1 -- 172.21.15.139:0/2965965254 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e040052a0 con 0x7f9e28158db0 2024-09-19T08:37:57.617 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.613+0000 7f9e1dffb640 1 -- 172.21.15.139:0/2965965254 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f9e20022020 con 0x7f9e28158db0 2024-09-19T08:37:57.683 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.681+0000 7f9e0b7fe640 1 -- 172.21.15.139:0/2965965254 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4ce2f72e-c241-44c1-b4cc-7616cde855fd", "id": 7} v 0) -- 0x7f9e04005590 con 0x7f9e28158db0 2024-09-19T08:37:57.691 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.689+0000 7f9e1dffb640 1 -- 172.21.15.139:0/2965965254 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4ce2f72e-c241-44c1-b4cc-7616cde855fd", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f9e2000be20 con 0x7f9e28158db0 2024-09-19T08:37:57.691 INFO:teuthology.orchestra.run.smithi139.stdout:7 2024-09-19T08:37:57.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.689+0000 7f9e0b7fe640 1 -- 172.21.15.139:0/2965965254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e28158db0 msgr2=0x7f9e280d09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:57.692 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.689+0000 7f9e0b7fe640 1 --2- 172.21.15.139:0/2965965254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e28158db0 0x7f9e280d09a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9e2000ef00 tx=0x7f9e2000b480 comp rx=0 tx=0).stop 2024-09-19T08:37:57.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.689+0000 7f9e0b7fe640 1 -- 172.21.15.139:0/2965965254 shutdown_connections 2024-09-19T08:37:57.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.689+0000 7f9e0b7fe640 1 --2- 172.21.15.139:0/2965965254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e28158db0 0x7f9e280d09a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.689+0000 7f9e0b7fe640 1 --2- 172.21.15.139:0/2965965254 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9e280b0620 0x7f9e2814dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.689+0000 7f9e0b7fe640 1 --2- 172.21.15.139:0/2965965254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e280751b0 0x7f9e2814f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.689+0000 7f9e0b7fe640 1 -- 172.21.15.139:0/2965965254 >> 172.21.15.139:0/2965965254 conn(0x7f9e280b5f50 msgr2=0x7f9e2806cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:57.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.689+0000 7f9e0b7fe640 1 -- 172.21.15.139:0/2965965254 shutdown_connections 2024-09-19T08:37:57.693 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:37:57.689+0000 7f9e0b7fe640 1 -- 172.21.15.139:0/2965965254 wait complete. 2024-09-19T08:37:57.707 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-19T08:37:57.707 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-19T08:37:57.710 INFO:tasks.ceph.osd.0:Started 2024-09-19T08:37:57.710 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-19T08:37:57.710 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-19T08:37:57.712 INFO:tasks.ceph.osd.1:Started 2024-09-19T08:37:57.712 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-19T08:37:57.712 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-19T08:37:57.714 INFO:tasks.ceph.osd.2:Started 2024-09-19T08:37:57.714 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-19T08:37:57.715 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-19T08:37:57.718 INFO:tasks.ceph.osd.3:Started 2024-09-19T08:37:57.718 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-19T08:37:57.718 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-19T08:37:57.720 INFO:tasks.ceph.osd.4:Started 2024-09-19T08:37:57.720 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-19T08:37:57.721 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-19T08:37:57.722 INFO:tasks.ceph.osd.5:Started 2024-09-19T08:37:57.723 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-19T08:37:57.723 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-19T08:37:57.724 INFO:tasks.ceph.osd.6:Started 2024-09-19T08:37:57.725 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-19T08:37:57.725 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-19T08:37:57.727 INFO:tasks.ceph.osd.7:Started 2024-09-19T08:37:57.727 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:37:57.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.850+0000 7fda8eb2f640 1 Processor -- start 2024-09-19T08:37:57.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.850+0000 7fda8eb2f640 1 -- start start 2024-09-19T08:37:57.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.854+0000 7fda8eb2f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda8815ec00 0x7fda88159d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:57.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.854+0000 7fda8eb2f640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fda880ce4a0 0x7fda8815a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:57.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.854+0000 7fda8eb2f640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda880b4800 con 0x7fda8815e110 2024-09-19T08:37:57.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.854+0000 7fda8eb2f640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fda88073e10 con 0x7fda880ce4a0 2024-09-19T08:37:57.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.854+0000 7fda8eb2f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fda880b4ae0 con 0x7fda8815ec00 2024-09-19T08:37:57.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.854+0000 7fda8c8cb640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fda8815e110 0x7fda8815e4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:38020/0 (socket says 172.21.15.98:38020) 2024-09-19T08:37:57.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.854+0000 7fda8d0cc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fda880ce4a0 0x7fda8815a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:57.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.854+0000 7fda8c8cb640 1 -- 172.21.15.98:0/3726537443 learned_addr learned my addr 172.21.15.98:0/3726537443 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:37:57.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.854+0000 7fda87fff640 1 --2- 172.21.15.98:0/3726537443 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda8815ec00 0x7fda88159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:57.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.854+0000 7fda87fff640 1 -- 172.21.15.98:0/3726537443 >> v2:172.21.15.98:3300/0 conn(0x7fda880ce4a0 msgr2=0x7fda8815a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:37:57.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.854+0000 7fda87fff640 1 --2- 172.21.15.98:0/3726537443 >> v2:172.21.15.98:3300/0 conn(0x7fda880ce4a0 0x7fda8815a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.854+0000 7fda87fff640 1 -- 172.21.15.98:0/3726537443 >> v1:172.21.15.98:6789/0 conn(0x7fda8815e110 legacy=0x7fda8815e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:57.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.854+0000 7fda87fff640 1 -- 172.21.15.98:0/3726537443 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda880b4dc0 con 0x7fda8815ec00 2024-09-19T08:37:57.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.858+0000 7fda87fff640 1 --2- 172.21.15.98:0/3726537443 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda8815ec00 0x7fda88159d90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fda7c0097c0 tx=0x7fda7c01e550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4861598a3cf6d0bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:57.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.858+0000 7fda877fe640 1 -- 172.21.15.98:0/3726537443 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7c0ac5f0 con 0x7fda8815ec00 2024-09-19T08:37:57.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.858+0000 7fda877fe640 1 -- 172.21.15.98:0/3726537443 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda7c0b1070 con 0x7fda8815ec00 2024-09-19T08:37:57.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.858+0000 7fda877fe640 1 -- 172.21.15.98:0/3726537443 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7c0ac9b0 con 0x7fda8815ec00 2024-09-19T08:37:57.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.862+0000 7fda8eb2f640 1 -- 172.21.15.98:0/3726537443 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda8815ec00 msgr2=0x7fda88159d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:57.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.862+0000 7fda8eb2f640 1 --2- 172.21.15.98:0/3726537443 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda8815ec00 0x7fda88159d90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fda7c0097c0 tx=0x7fda7c01e550 comp rx=0 tx=0).stop 2024-09-19T08:37:57.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.862+0000 7fda8eb2f640 1 -- 172.21.15.98:0/3726537443 shutdown_connections 2024-09-19T08:37:57.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.862+0000 7fda8eb2f640 1 --2- 172.21.15.98:0/3726537443 >> v2:172.21.15.98:3300/0 conn(0x7fda880ce4a0 0x7fda8815a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.862+0000 7fda8eb2f640 1 --2- 172.21.15.98:0/3726537443 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda8815ec00 0x7fda88159d90 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.862+0000 7fda8eb2f640 1 -- 172.21.15.98:0/3726537443 >> 172.21.15.98:0/3726537443 conn(0x7fda8806cda0 msgr2=0x7fda8806d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:57.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.862+0000 7fda8eb2f640 1 -- 172.21.15.98:0/3726537443 shutdown_connections 2024-09-19T08:37:57.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.862+0000 7fda8eb2f640 1 -- 172.21.15.98:0/3726537443 wait complete. 2024-09-19T08:37:57.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.862+0000 7fda8eb2f640 1 Processor -- start 2024-09-19T08:37:57.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.862+0000 7fda8eb2f640 1 -- start start 2024-09-19T08:37:57.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.866+0000 7fda8eb2f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda880ce4a0 0x7fda8814e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:57.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.866+0000 7fda8eb2f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda8815e110 0x7fda88150120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:57.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.866+0000 7fda8eb2f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda8815ec00 0x7fda8815bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:57.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.866+0000 7fda8eb2f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fda880b3db0 con 0x7fda8815e110 2024-09-19T08:37:57.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.866+0000 7fda8c8cb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda880ce4a0 0x7fda8814e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:57.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.866+0000 7fda8c8cb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda880ce4a0 0x7fda8814e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:41876/0 (socket says 172.21.15.98:41876) 2024-09-19T08:37:57.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.866+0000 7fda87fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda8815e110 0x7fda88150120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:57.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.866+0000 7fda8c8cb640 1 -- 172.21.15.98:0/3723343765 learned_addr learned my addr 172.21.15.98:0/3723343765 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:37:57.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.866+0000 7fda8d0cc640 1 --2- 172.21.15.98:0/3723343765 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda8815ec00 0x7fda8815bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:57.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.866+0000 7fda8eb2f640 1 -- 172.21.15.98:0/3723343765 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fda880b3c30 con 0x7fda8815ec00 2024-09-19T08:37:57.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.866+0000 7fda8eb2f640 1 -- 172.21.15.98:0/3723343765 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fda880b3f30 con 0x7fda880ce4a0 2024-09-19T08:37:57.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.866+0000 7fda8c8cb640 1 -- 172.21.15.98:0/3723343765 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda8815ec00 msgr2=0x7fda8815bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:57.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.866+0000 7fda8c8cb640 1 --2- 172.21.15.98:0/3723343765 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda8815ec00 0x7fda8815bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.866+0000 7fda8c8cb640 1 -- 172.21.15.98:0/3723343765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda8815e110 msgr2=0x7fda88150120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:57.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.866+0000 7fda8c8cb640 1 --2- 172.21.15.98:0/3723343765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda8815e110 0x7fda88150120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.866+0000 7fda8c8cb640 1 -- 172.21.15.98:0/3723343765 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda881529d0 con 0x7fda880ce4a0 2024-09-19T08:37:57.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.866+0000 7fda87fff640 1 --2- 172.21.15.98:0/3723343765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda8815e110 0x7fda88150120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:37:57.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.866+0000 7fda8c8cb640 1 --2- 172.21.15.98:0/3723343765 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda880ce4a0 0x7fda8814e550 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fda7800c780 tx=0x7fda7800cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:57.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.866+0000 7fda857fa640 1 -- 172.21.15.98:0/3723343765 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda78018070 con 0x7fda880ce4a0 2024-09-19T08:37:57.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.866+0000 7fda857fa640 1 -- 172.21.15.98:0/3723343765 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda780074e0 con 0x7fda880ce4a0 2024-09-19T08:37:57.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.870+0000 7fda857fa640 1 -- 172.21.15.98:0/3723343765 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda78004a30 con 0x7fda880ce4a0 2024-09-19T08:37:57.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.878+0000 7fda8eb2f640 1 -- 172.21.15.98:0/3723343765 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda88152cb0 con 0x7fda880ce4a0 2024-09-19T08:37:57.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.878+0000 7fda8eb2f640 1 -- 172.21.15.98:0/3723343765 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fda88153270 con 0x7fda880ce4a0 2024-09-19T08:37:57.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.878+0000 7fda857fa640 1 -- 172.21.15.98:0/3723343765 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fda78004bd0 con 0x7fda880ce4a0 2024-09-19T08:37:57.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.878+0000 7fda857fa640 1 -- 172.21.15.98:0/3723343765 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fda78012ad0 con 0x7fda880ce4a0 2024-09-19T08:37:57.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.878+0000 7fda8eb2f640 1 -- 172.21.15.98:0/3723343765 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda880b4340 con 0x7fda880ce4a0 2024-09-19T08:37:57.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.886+0000 7fda857fa640 1 -- 172.21.15.98:0/3723343765 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fda78012400 con 0x7fda880ce4a0 2024-09-19T08:37:57.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.954+0000 7fda8eb2f640 1 -- 172.21.15.98:0/3723343765 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda8806ecd0 con 0x7fda880ce4a0 2024-09-19T08:37:57.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.954+0000 7fda857fa640 1 -- 172.21.15.98:0/3723343765 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fda780125e0 con 0x7fda880ce4a0 2024-09-19T08:37:57.957 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:37:57.957 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":10,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:37:57.685486+0000","last_up_change":"0.000000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:37:57.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.954+0000 7fda8eb2f640 1 -- 172.21.15.98:0/3723343765 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda880ce4a0 msgr2=0x7fda8814e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:57.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.954+0000 7fda8eb2f640 1 --2- 172.21.15.98:0/3723343765 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda880ce4a0 0x7fda8814e550 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fda7800c780 tx=0x7fda7800cc40 comp rx=0 tx=0).stop 2024-09-19T08:37:57.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.954+0000 7fda8eb2f640 1 -- 172.21.15.98:0/3723343765 shutdown_connections 2024-09-19T08:37:57.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.954+0000 7fda8eb2f640 1 --2- 172.21.15.98:0/3723343765 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda8815ec00 0x7fda8815bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.954+0000 7fda8eb2f640 1 --2- 172.21.15.98:0/3723343765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda8815e110 0x7fda88150120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.954+0000 7fda8eb2f640 1 --2- 172.21.15.98:0/3723343765 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda880ce4a0 0x7fda8814e550 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:57.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.954+0000 7fda8eb2f640 1 -- 172.21.15.98:0/3723343765 >> 172.21.15.98:0/3723343765 conn(0x7fda8806cda0 msgr2=0x7fda88158860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:57.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.954+0000 7fda8eb2f640 1 -- 172.21.15.98:0/3723343765 shutdown_connections 2024-09-19T08:37:57.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:37:57.954+0000 7fda8eb2f640 1 -- 172.21.15.98:0/3723343765 wait complete. 2024-09-19T08:37:57.974 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-19T08:37:57.974 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-19T08:37:58.147 INFO:tasks.ceph.mgr.y.smithi098.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-19T08:37:58.148 INFO:tasks.ceph.mgr.y.smithi098.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-19T08:37:58.148 INFO:tasks.ceph.mgr.y.smithi098.stderr: from numpy import show_config as show_numpy_config 2024-09-19T08:37:58.193 INFO:tasks.ceph.mgr.x.smithi139.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-19T08:37:58.194 INFO:tasks.ceph.mgr.x.smithi139.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-19T08:37:58.194 INFO:tasks.ceph.mgr.x.smithi139.stderr: from numpy import show_config as show_numpy_config 2024-09-19T08:37:58.203 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:37:58.202+0000 7f3765211180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T08:37:58.248 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:37:58.245+0000 7ff7b5cad180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T08:37:58.337 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:37:58.334+0000 7f3765211180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:37:58.420 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:37:58.417+0000 7ff7b5cad180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:37:58.494 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:37:58.490+0000 7f3765211180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T08:37:58.558 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:37:58.554+0000 7f3765211180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T08:37:58.570 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:37:58.565+0000 7ff7b5cad180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T08:37:58.632 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:37:58.629+0000 7ff7b5cad180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T08:37:58.692 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:37:58.690+0000 7f3765211180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:37:58.767 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:37:58.765+0000 7ff7b5cad180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:37:58.780 DEBUG:teuthology.orchestra.run.smithi098:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-19T08:37:58.886 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.882+0000 7f71c33dd640 1 Processor -- start 2024-09-19T08:37:58.886 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.882+0000 7f71c33dd640 1 -- start start 2024-09-19T08:37:58.887 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.882+0000 7f71c33dd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f71bc15bec0 0x7f71bc06e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:58.887 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.882+0000 7f71c33dd640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71bc1620b0 con 0x7f71bc0cf770 2024-09-19T08:37:58.887 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.882+0000 7f71c33dd640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71bc162370 con 0x7f71bc06c310 2024-09-19T08:37:58.887 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.882+0000 7f71c33dd640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f71bc162650 con 0x7f71bc15bec0 2024-09-19T08:37:58.887 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.882+0000 7f71c1179640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f71bc15bec0 0x7f71bc06e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:58.887 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.882+0000 7f71c0978640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f71bc0cf770 0x7f71bc06bc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:38024/0 (socket says 172.21.15.98:38024) 2024-09-19T08:37:58.887 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.882+0000 7f71c0978640 1 -- 172.21.15.98:0/1590641406 learned_addr learned my addr 172.21.15.98:0/1590641406 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:37:58.887 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.886+0000 7f71c1179640 1 -- 172.21.15.98:0/1590641406 >> v1:172.21.15.139:6789/0 conn(0x7f71bc06c310 legacy=0x7f71bc0ca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:37:58.887 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.886+0000 7f71c1179640 1 -- 172.21.15.98:0/1590641406 >> v1:172.21.15.98:6789/0 conn(0x7f71bc0cf770 legacy=0x7f71bc06bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:58.887 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.886+0000 7f71c1179640 1 -- 172.21.15.98:0/1590641406 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71bc162930 con 0x7f71bc15bec0 2024-09-19T08:37:58.889 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.886+0000 7f71c1179640 1 --2- 172.21.15.98:0/1590641406 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71bc15bec0 0x7f71bc06e100 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f71ac0097c0 tx=0x7f71ac01ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fde5ccb5fc44dd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:58.889 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.886+0000 7f71b3fff640 1 -- 172.21.15.98:0/1590641406 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac0b5070 con 0x7f71bc15bec0 2024-09-19T08:37:58.890 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.886+0000 7f71b3fff640 1 -- 172.21.15.98:0/1590641406 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71ac0a8950 con 0x7f71bc15bec0 2024-09-19T08:37:58.890 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.886+0000 7f71b3fff640 1 -- 172.21.15.98:0/1590641406 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac0a8d20 con 0x7f71bc15bec0 2024-09-19T08:37:58.890 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.886+0000 7f71c33dd640 1 -- 172.21.15.98:0/1590641406 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71bc15bec0 msgr2=0x7f71bc06e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:58.890 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.886+0000 7f71c33dd640 1 --2- 172.21.15.98:0/1590641406 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71bc15bec0 0x7f71bc06e100 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f71ac0097c0 tx=0x7f71ac01ec50 comp rx=0 tx=0).stop 2024-09-19T08:37:58.891 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.886+0000 7f71c33dd640 1 -- 172.21.15.98:0/1590641406 shutdown_connections 2024-09-19T08:37:58.891 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.886+0000 7f71c33dd640 1 --2- 172.21.15.98:0/1590641406 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71bc15bec0 0x7f71bc06e100 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:58.891 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.886+0000 7f71c33dd640 1 -- 172.21.15.98:0/1590641406 >> 172.21.15.98:0/1590641406 conn(0x7f71bc0d1290 msgr2=0x7f71bc0d16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:58.891 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.886+0000 7f71c33dd640 1 -- 172.21.15.98:0/1590641406 shutdown_connections 2024-09-19T08:37:58.891 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.886+0000 7f71c33dd640 1 -- 172.21.15.98:0/1590641406 wait complete. 2024-09-19T08:37:58.891 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.890+0000 7f71c33dd640 1 Processor -- start 2024-09-19T08:37:58.892 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.890+0000 7f71c33dd640 1 -- start start 2024-09-19T08:37:58.894 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.890+0000 7f71c33dd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71bc06c310 0x7f71bc147050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:58.894 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.890+0000 7f71c33dd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71bc0cf770 0x7f71bc144630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:58.894 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.890+0000 7f71c33dd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71bc15bec0 0x7f71bc0b6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:37:58.894 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.890+0000 7f71c33dd640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc0cf020 con 0x7f71bc06c310 2024-09-19T08:37:58.894 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.890+0000 7f71c33dd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc162ef0 con 0x7f71bc0cf770 2024-09-19T08:37:58.894 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.890+0000 7f71c33dd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f71bc0cf1a0 con 0x7f71bc15bec0 2024-09-19T08:37:58.894 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.890+0000 7f71c1179640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71bc06c310 0x7f71bc147050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:58.894 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.890+0000 7f71c1179640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71bc06c310 0x7f71bc147050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39406/0 (socket says 172.21.15.98:39406) 2024-09-19T08:37:58.894 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.890+0000 7f71c1179640 1 -- 172.21.15.98:0/323901307 learned_addr learned my addr 172.21.15.98:0/323901307 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:37:58.894 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.890+0000 7f71c197a640 1 --2- 172.21.15.98:0/323901307 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71bc15bec0 0x7f71bc0b6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:58.894 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.890+0000 7f71c0978640 1 --2- 172.21.15.98:0/323901307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71bc0cf770 0x7f71bc144630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:37:58.894 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.890+0000 7f71c1179640 1 -- 172.21.15.98:0/323901307 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71bc15bec0 msgr2=0x7f71bc0b6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:58.895 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.890+0000 7f71c1179640 1 --2- 172.21.15.98:0/323901307 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71bc15bec0 0x7f71bc0b6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:58.895 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.890+0000 7f71c1179640 1 -- 172.21.15.98:0/323901307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71bc0cf770 msgr2=0x7f71bc144630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:58.895 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.890+0000 7f71c1179640 1 --2- 172.21.15.98:0/323901307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71bc0cf770 0x7f71bc144630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:58.895 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.890+0000 7f71c1179640 1 -- 172.21.15.98:0/323901307 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71bc074050 con 0x7f71bc06c310 2024-09-19T08:37:58.895 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.890+0000 7f71c197a640 1 --2- 172.21.15.98:0/323901307 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71bc15bec0 0x7f71bc0b6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:37:58.895 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.890+0000 7f71c0978640 1 --2- 172.21.15.98:0/323901307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71bc0cf770 0x7f71bc144630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:37:58.895 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.890+0000 7f71c1179640 1 --2- 172.21.15.98:0/323901307 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71bc06c310 0x7f71bc147050 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f71ac01b860 tx=0x7f71ac01b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:37:58.895 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.890+0000 7f71b1ffb640 1 -- 172.21.15.98:0/323901307 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac0b5040 con 0x7f71bc06c310 2024-09-19T08:37:58.895 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.890+0000 7f71b1ffb640 1 -- 172.21.15.98:0/323901307 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71ac0c0070 con 0x7f71bc06c310 2024-09-19T08:37:58.895 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.890+0000 7f71c33dd640 1 -- 172.21.15.98:0/323901307 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71bc0b6190 con 0x7f71bc06c310 2024-09-19T08:37:58.895 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.890+0000 7f71b1ffb640 1 -- 172.21.15.98:0/323901307 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac0b0a80 con 0x7f71bc06c310 2024-09-19T08:37:58.895 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.890+0000 7f71c33dd640 1 -- 172.21.15.98:0/323901307 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71bc147750 con 0x7f71bc06c310 2024-09-19T08:37:58.895 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.890+0000 7f71b1ffb640 1 -- 172.21.15.98:0/323901307 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f71ac0b0c20 con 0x7f71bc06c310 2024-09-19T08:37:58.896 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.890+0000 7f71b1ffb640 1 -- 172.21.15.98:0/323901307 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f71ac0c5be0 con 0x7f71bc06c310 2024-09-19T08:37:58.896 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.890+0000 7f71c33dd640 1 -- 172.21.15.98:0/323901307 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71bc15c2a0 con 0x7f71bc06c310 2024-09-19T08:37:58.897 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.894+0000 7f71b1ffb640 1 -- 172.21.15.98:0/323901307 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f71bc074050 con 0x7f71bc06c310 2024-09-19T08:37:58.955 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.954+0000 7f71c33dd640 1 -- 172.21.15.98:0/323901307 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f71bc074230 con 0x7f71bc06c310 2024-09-19T08:37:58.956 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.954+0000 7f71b1ffb640 1 -- 172.21.15.98:0/323901307 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f71ac0187b0 con 0x7f71bc06c310 2024-09-19T08:37:58.956 INFO:teuthology.misc.health.smithi098.stdout: 2024-09-19T08:37:58.956 INFO:teuthology.misc.health.smithi098.stdout:{"epoch":10,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:37:57.685486+0000","last_up_change":"0.000000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:37:58.957 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.954+0000 7f71c33dd640 1 -- 172.21.15.98:0/323901307 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71bc06c310 msgr2=0x7f71bc147050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:37:58.957 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.954+0000 7f71c33dd640 1 --2- 172.21.15.98:0/323901307 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71bc06c310 0x7f71bc147050 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f71ac01b860 tx=0x7f71ac01b9f0 comp rx=0 tx=0).stop 2024-09-19T08:37:58.957 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.954+0000 7f71c33dd640 1 -- 172.21.15.98:0/323901307 shutdown_connections 2024-09-19T08:37:58.958 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.954+0000 7f71c33dd640 1 --2- 172.21.15.98:0/323901307 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71bc15bec0 0x7f71bc0b6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:58.958 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.954+0000 7f71c33dd640 1 --2- 172.21.15.98:0/323901307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71bc0cf770 0x7f71bc144630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:58.958 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.954+0000 7f71c33dd640 1 --2- 172.21.15.98:0/323901307 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71bc06c310 0x7f71bc147050 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:37:58.958 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.954+0000 7f71c33dd640 1 -- 172.21.15.98:0/323901307 >> 172.21.15.98:0/323901307 conn(0x7f71bc0d1290 msgr2=0x7f71bc0ca8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:37:58.958 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.954+0000 7f71c33dd640 1 -- 172.21.15.98:0/323901307 shutdown_connections 2024-09-19T08:37:58.958 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:37:58.954+0000 7f71c33dd640 1 -- 172.21.15.98:0/323901307 wait complete. 2024-09-19T08:37:58.972 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-19T08:37:59.304 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:37:59.302+0000 7f3765211180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T08:37:59.372 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-19T08:37:59.370+0000 7fc0524df7c0 -1 Falling back to public interface 2024-09-19T08:37:59.375 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:37:59.373+0000 7ff7b5cad180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T08:37:59.380 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:37:59.378+0000 7f3765211180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T08:37:59.451 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:37:59.449+0000 7ff7b5cad180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T08:37:59.563 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:37:59.558+0000 7f3765211180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T08:37:59.633 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:37:59.629+0000 7ff7b5cad180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T08:37:59.636 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:37:59.634+0000 7f3765211180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T08:37:59.639 INFO:tasks.ceph.osd.6.smithi139.stderr:2024-09-19T08:37:59.637+0000 7f762f8617c0 -1 Falling back to public interface 2024-09-19T08:37:59.653 INFO:tasks.ceph.osd.2.smithi098.stderr:2024-09-19T08:37:59.650+0000 7f9c969017c0 -1 Falling back to public interface 2024-09-19T08:37:59.707 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:37:59.705+0000 7ff7b5cad180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T08:37:59.876 INFO:tasks.ceph.osd.0.smithi098.stderr:2024-09-19T08:37:59.874+0000 7f17883f47c0 -1 Falling back to public interface 2024-09-19T08:37:59.876 INFO:tasks.ceph.osd.1.smithi098.stderr:2024-09-19T08:37:59.874+0000 7f59a3ea07c0 -1 Falling back to public interface 2024-09-19T08:37:59.916 INFO:tasks.ceph.osd.5.smithi139.stderr:2024-09-19T08:37:59.913+0000 7f09401c47c0 -1 Falling back to public interface 2024-09-19T08:37:59.927 INFO:tasks.ceph.osd.7.smithi139.stderr:2024-09-19T08:37:59.925+0000 7fdc5bed27c0 -1 Falling back to public interface 2024-09-19T08:37:59.933 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:37:59.930+0000 7f3765211180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T08:37:59.934 INFO:tasks.ceph.osd.4.smithi139.stderr:2024-09-19T08:37:59.933+0000 7fd78730f7c0 -1 Falling back to public interface 2024-09-19T08:37:59.996 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:37:59.994+0000 7f3765211180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T08:37:59.999 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:37:59.997+0000 7ff7b5cad180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T08:38:00.065 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:38:00.061+0000 7ff7b5cad180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T08:38:01.039 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:38:01.034+0000 7f370bd48640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-19T08:38:01.040 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:38:01.038+0000 7f370bd48640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-19T08:38:03.736 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-19T08:38:03.734+0000 7fc0524df7c0 -1 osd.3 0 log_to_monitors true 2024-09-19T08:38:03.952 INFO:tasks.ceph.osd.6.smithi139.stderr:2024-09-19T08:38:03.949+0000 7f762f8617c0 -1 osd.6 0 log_to_monitors true 2024-09-19T08:38:04.167 INFO:tasks.ceph.osd.0.smithi098.stderr:2024-09-19T08:38:04.166+0000 7f17883f47c0 -1 osd.0 0 log_to_monitors true 2024-09-19T08:38:04.208 INFO:tasks.ceph.osd.1.smithi098.stderr:2024-09-19T08:38:04.206+0000 7f59a3ea07c0 -1 osd.1 0 log_to_monitors true 2024-09-19T08:38:04.306 INFO:tasks.ceph.osd.2.smithi098.stderr:2024-09-19T08:38:04.302+0000 7f9c969017c0 -1 osd.2 0 log_to_monitors true 2024-09-19T08:38:04.522 INFO:tasks.ceph.osd.4.smithi139.stderr:2024-09-19T08:38:04.521+0000 7fd78730f7c0 -1 osd.4 0 log_to_monitors true 2024-09-19T08:38:04.682 INFO:tasks.ceph.osd.7.smithi139.stderr:2024-09-19T08:38:04.681+0000 7fdc5bed27c0 -1 osd.7 0 log_to_monitors true 2024-09-19T08:38:04.773 INFO:tasks.ceph.osd.5.smithi139.stderr:2024-09-19T08:38:04.769+0000 7f09401c47c0 -1 osd.5 0 log_to_monitors true 2024-09-19T08:38:05.779 DEBUG:teuthology.orchestra.run.smithi098:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-19T08:38:05.894 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.890+0000 7feb32979640 1 Processor -- start 2024-09-19T08:38:05.894 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.890+0000 7feb32979640 1 -- start start 2024-09-19T08:38:05.895 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.894+0000 7feb32979640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7feb2c15fbd0 0x7feb2c0d2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:05.895 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.894+0000 7feb32979640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7feb2c0d0ec0 0x7feb2c0afa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:05.895 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.894+0000 7feb32979640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb2c0cdf40 con 0x7feb2c0ce720 2024-09-19T08:38:05.896 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.894+0000 7feb32979640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7feb2c0ce500 con 0x7feb2c15fbd0 2024-09-19T08:38:05.896 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.894+0000 7feb32979640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7feb2c0af590 con 0x7feb2c0d0ec0 2024-09-19T08:38:05.896 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.894+0000 7feb2bfff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7feb2c15fbd0 0x7feb2c0d2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:05.896 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.894+0000 7feb2bfff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7feb2c15fbd0 0x7feb2c0d2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39532/0 (socket says 172.21.15.98:39532) 2024-09-19T08:38:05.896 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.894+0000 7feb2bfff640 1 -- 172.21.15.98:0/2176611387 learned_addr learned my addr 172.21.15.98:0/2176611387 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:05.896 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.894+0000 7feb2bfff640 1 -- 172.21.15.98:0/2176611387 >> v2:172.21.15.139:3300/0 conn(0x7feb2c0d0ec0 msgr2=0x7feb2c0afa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:05.896 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.894+0000 7feb30f16640 1 --2- 172.21.15.98:0/2176611387 >> v2:172.21.15.139:3300/0 conn(0x7feb2c0d0ec0 0x7feb2c0afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:05.896 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.894+0000 7feb2affd640 1 -- 172.21.15.98:0/2176611387 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 273301595 0 0) 0x7feb2c0cdf40 con 0x7feb2c0ce720 2024-09-19T08:38:05.896 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.894+0000 7feb2bfff640 1 --2- 172.21.15.98:0/2176611387 >> v2:172.21.15.139:3300/0 conn(0x7feb2c0d0ec0 0x7feb2c0afa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:05.896 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.894+0000 7feb2bfff640 1 -- 172.21.15.98:0/2176611387 >> v1:172.21.15.98:6789/0 conn(0x7feb2c0ce720 legacy=0x7feb2c0d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:05.896 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.894+0000 7feb2bfff640 1 -- 172.21.15.98:0/2176611387 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb2c0ce220 con 0x7feb2c15fbd0 2024-09-19T08:38:05.897 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.894+0000 7feb2bfff640 1 --2- 172.21.15.98:0/2176611387 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb2c15fbd0 0x7feb2c0d2d80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7feb2c072e50 tx=0x7feb1c01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c0311c42b1ba1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:05.897 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.894+0000 7feb2affd640 1 -- 172.21.15.98:0/2176611387 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1c0a56c0 con 0x7feb2c15fbd0 2024-09-19T08:38:05.898 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.894+0000 7feb2affd640 1 -- 172.21.15.98:0/2176611387 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb1c0a5860 con 0x7feb2c15fbd0 2024-09-19T08:38:05.898 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.894+0000 7feb2affd640 1 -- 172.21.15.98:0/2176611387 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1c0a5c30 con 0x7feb2c15fbd0 2024-09-19T08:38:05.898 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.898+0000 7feb32979640 1 -- 172.21.15.98:0/2176611387 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb2c15fbd0 msgr2=0x7feb2c0d2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:05.899 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.898+0000 7feb32979640 1 --2- 172.21.15.98:0/2176611387 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb2c15fbd0 0x7feb2c0d2d80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7feb2c072e50 tx=0x7feb1c01c8a0 comp rx=0 tx=0).stop 2024-09-19T08:38:05.899 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.898+0000 7feb32979640 1 -- 172.21.15.98:0/2176611387 shutdown_connections 2024-09-19T08:38:05.899 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.898+0000 7feb32979640 1 --2- 172.21.15.98:0/2176611387 >> v2:172.21.15.139:3300/0 conn(0x7feb2c0d0ec0 0x7feb2c0afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:05.899 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.898+0000 7feb32979640 1 --2- 172.21.15.98:0/2176611387 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb2c15fbd0 0x7feb2c0d2d80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:05.899 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.898+0000 7feb32979640 1 -- 172.21.15.98:0/2176611387 >> 172.21.15.98:0/2176611387 conn(0x7feb2c09ba10 msgr2=0x7feb2c0d2460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:05.899 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.898+0000 7feb32979640 1 -- 172.21.15.98:0/2176611387 shutdown_connections 2024-09-19T08:38:05.899 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.898+0000 7feb32979640 1 -- 172.21.15.98:0/2176611387 wait complete. 2024-09-19T08:38:05.900 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.898+0000 7feb32979640 1 Processor -- start 2024-09-19T08:38:05.900 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.898+0000 7feb32979640 1 -- start start 2024-09-19T08:38:05.902 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.898+0000 7feb32979640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb2c0ce720 0x7feb2c166a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:05.902 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.898+0000 7feb32979640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb2c0d0ec0 0x7feb2c15cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:05.902 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.898+0000 7feb32979640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb2c15fbd0 0x7feb2c15a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:05.902 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.898+0000 7feb32979640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7feb2c0b08f0 con 0x7feb2c0d0ec0 2024-09-19T08:38:05.902 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.898+0000 7feb32979640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feb2c0b0770 con 0x7feb2c15fbd0 2024-09-19T08:38:05.902 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.898+0000 7feb32979640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7feb2c0b0a70 con 0x7feb2c0ce720 2024-09-19T08:38:05.902 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.898+0000 7feb2b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb2c0d0ec0 0x7feb2c15cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:05.902 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.898+0000 7feb2b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb2c0d0ec0 0x7feb2c15cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39536/0 (socket says 172.21.15.98:39536) 2024-09-19T08:38:05.903 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.898+0000 7feb2b7fe640 1 -- 172.21.15.98:0/168096224 learned_addr learned my addr 172.21.15.98:0/168096224 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:05.903 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.898+0000 7feb30f16640 1 --2- 172.21.15.98:0/168096224 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb2c15fbd0 0x7feb2c15a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:05.903 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.898+0000 7feb2bfff640 1 --2- 172.21.15.98:0/168096224 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb2c0ce720 0x7feb2c166a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:05.903 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.898+0000 7feb2b7fe640 1 -- 172.21.15.98:0/168096224 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb2c0ce720 msgr2=0x7feb2c166a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:05.903 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.898+0000 7feb2b7fe640 1 --2- 172.21.15.98:0/168096224 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb2c0ce720 0x7feb2c166a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:05.903 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.898+0000 7feb2b7fe640 1 -- 172.21.15.98:0/168096224 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb2c15fbd0 msgr2=0x7feb2c15a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:05.903 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.898+0000 7feb2b7fe640 1 --2- 172.21.15.98:0/168096224 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb2c15fbd0 0x7feb2c15a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:05.903 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.898+0000 7feb2b7fe640 1 -- 172.21.15.98:0/168096224 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb2c166f80 con 0x7feb2c0d0ec0 2024-09-19T08:38:05.903 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.898+0000 7feb30f16640 1 --2- 172.21.15.98:0/168096224 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb2c15fbd0 0x7feb2c15a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:38:05.903 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.898+0000 7feb2b7fe640 1 --2- 172.21.15.98:0/168096224 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb2c0d0ec0 0x7feb2c15cea0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7feb2000c480 tx=0x7feb2000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:05.904 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.898+0000 7feb28ff9640 1 -- 172.21.15.98:0/168096224 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb20005480 con 0x7feb2c0d0ec0 2024-09-19T08:38:05.904 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.898+0000 7feb28ff9640 1 -- 172.21.15.98:0/168096224 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb20003600 con 0x7feb2c0d0ec0 2024-09-19T08:38:05.904 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.898+0000 7feb32979640 1 -- 172.21.15.98:0/168096224 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb2c0b7e50 con 0x7feb2c0d0ec0 2024-09-19T08:38:05.904 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.898+0000 7feb28ff9640 1 -- 172.21.15.98:0/168096224 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb200039f0 con 0x7feb2c0d0ec0 2024-09-19T08:38:05.904 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.898+0000 7feb32979640 1 -- 172.21.15.98:0/168096224 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb2c15d3e0 con 0x7feb2c0d0ec0 2024-09-19T08:38:05.905 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.902+0000 7feb32979640 1 -- 172.21.15.98:0/168096224 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb2c0b8300 con 0x7feb2c0d0ec0 2024-09-19T08:38:05.905 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.902+0000 7feb28ff9640 1 -- 172.21.15.98:0/168096224 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 4) ==== 87389+0+0 (secure 0 0 0) 0x7feb20003b90 con 0x7feb2c0d0ec0 2024-09-19T08:38:05.905 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.902+0000 7feb28ff9640 1 --2- 172.21.15.98:0/168096224 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7feb18067830 0x7feb18069cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:05.905 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.902+0000 7feb28ff9640 1 -- 172.21.15.98:0/168096224 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2135+0+0 (secure 0 0 0) 0x7feb200853c0 con 0x7feb2c0d0ec0 2024-09-19T08:38:05.905 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.902+0000 7feb2bfff640 1 --2- 172.21.15.98:0/168096224 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7feb18067830 0x7feb18069cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:05.905 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.902+0000 7feb2bfff640 1 --2- 172.21.15.98:0/168096224 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7feb18067830 0x7feb18069cf0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7feb1c0ae000 tx=0x7feb1c0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:05.908 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:05.906+0000 7feb28ff9640 1 -- 172.21.15.98:0/168096224 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7feb20056f40 con 0x7feb2c0d0ec0 2024-09-19T08:38:05.963 INFO:tasks.ceph.osd.6.smithi139.stderr:2024-09-19T08:38:05.961+0000 7f762afe7640 -1 osd.6 0 waiting for initial osdmap 2024-09-19T08:38:05.987 INFO:tasks.ceph.osd.6.smithi139.stderr:2024-09-19T08:38:05.985+0000 7f762362e640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:38:06.008 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-19T08:38:06.006+0000 7fc04ec9a640 -1 osd.3 0 waiting for initial osdmap 2024-09-19T08:38:06.028 INFO:tasks.ceph.osd.3.smithi098.stderr:2024-09-19T08:38:06.026+0000 7fc0462ac640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:38:06.090 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:06.086+0000 7feb32979640 1 -- 172.21.15.98:0/168096224 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feb2c0cdf40 con 0x7feb2c0d0ec0 2024-09-19T08:38:06.091 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:06.090+0000 7feb28ff9640 1 -- 172.21.15.98:0/168096224 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+7567 (secure 0 0 0) 0x7feb20056840 con 0x7feb2c0d0ec0 2024-09-19T08:38:06.091 INFO:teuthology.misc.health.smithi098.stdout: 2024-09-19T08:38:06.091 INFO:teuthology.misc.health.smithi098.stdout:{"epoch":13,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:38:06.042470+0000","last_up_change":"2024-09-19T08:38:06.042470+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:38:06.093 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:06.090+0000 7feb32979640 1 -- 172.21.15.98:0/168096224 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7feb18067830 msgr2=0x7feb18069cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:06.093 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:06.090+0000 7feb32979640 1 --2- 172.21.15.98:0/168096224 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7feb18067830 0x7feb18069cf0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7feb1c0ae000 tx=0x7feb1c0a7070 comp rx=0 tx=0).stop 2024-09-19T08:38:06.093 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:06.090+0000 7feb32979640 1 -- 172.21.15.98:0/168096224 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb2c0d0ec0 msgr2=0x7feb2c15cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:06.093 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:06.090+0000 7feb32979640 1 --2- 172.21.15.98:0/168096224 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb2c0d0ec0 0x7feb2c15cea0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7feb2000c480 tx=0x7feb2000c940 comp rx=0 tx=0).stop 2024-09-19T08:38:06.093 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:06.090+0000 7feb32979640 1 -- 172.21.15.98:0/168096224 shutdown_connections 2024-09-19T08:38:06.094 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:06.090+0000 7feb32979640 1 --2- 172.21.15.98:0/168096224 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7feb18067830 0x7feb18069cf0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:06.094 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:06.090+0000 7feb32979640 1 --2- 172.21.15.98:0/168096224 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb2c15fbd0 0x7feb2c15a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:06.094 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:06.090+0000 7feb32979640 1 --2- 172.21.15.98:0/168096224 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb2c0d0ec0 0x7feb2c15cea0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:06.094 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:06.090+0000 7feb32979640 1 --2- 172.21.15.98:0/168096224 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb2c0ce720 0x7feb2c166a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:06.094 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:06.090+0000 7feb32979640 1 -- 172.21.15.98:0/168096224 >> 172.21.15.98:0/168096224 conn(0x7feb2c09ba10 msgr2=0x7feb2c0b5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:06.094 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:06.090+0000 7feb32979640 1 -- 172.21.15.98:0/168096224 shutdown_connections 2024-09-19T08:38:06.094 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:06.090+0000 7feb32979640 1 -- 172.21.15.98:0/168096224 wait complete. 2024-09-19T08:38:06.110 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-09-19T08:38:07.694 INFO:tasks.ceph.osd.0.smithi098.stderr:2024-09-19T08:38:07.690+0000 7f1784baf640 -1 osd.0 0 waiting for initial osdmap 2024-09-19T08:38:07.723 INFO:tasks.ceph.osd.0.smithi098.stderr:2024-09-19T08:38:07.722+0000 7f177c1c1640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:38:07.744 INFO:tasks.ceph.osd.4.smithi139.stderr:2024-09-19T08:38:07.741+0000 7fd782a95640 -1 osd.4 0 waiting for initial osdmap 2024-09-19T08:38:07.770 INFO:tasks.ceph.osd.4.smithi139.stderr:2024-09-19T08:38:07.769+0000 7fd77b0dc640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:38:07.781 INFO:tasks.ceph.osd.2.smithi098.stderr:2024-09-19T08:38:07.778+0000 7f9c92087640 -1 osd.2 0 waiting for initial osdmap 2024-09-19T08:38:07.802 INFO:tasks.ceph.osd.5.smithi139.stderr:2024-09-19T08:38:07.801+0000 7f093b94a640 -1 osd.5 0 waiting for initial osdmap 2024-09-19T08:38:07.804 INFO:tasks.ceph.osd.2.smithi098.stderr:2024-09-19T08:38:07.802+0000 7f9c8a6ce640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:38:07.807 INFO:tasks.ceph.osd.7.smithi139.stderr:2024-09-19T08:38:07.805+0000 7fdc57658640 -1 osd.7 0 waiting for initial osdmap 2024-09-19T08:38:07.808 INFO:tasks.ceph.osd.1.smithi098.stderr:2024-09-19T08:38:07.806+0000 7f59a065b640 -1 osd.1 0 waiting for initial osdmap 2024-09-19T08:38:07.831 INFO:tasks.ceph.osd.5.smithi139.stderr:2024-09-19T08:38:07.829+0000 7f0933f91640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:38:07.831 INFO:tasks.ceph.osd.1.smithi098.stderr:2024-09-19T08:38:07.830+0000 7f5997c6d640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:38:07.851 INFO:tasks.ceph.osd.7.smithi139.stderr:2024-09-19T08:38:07.849+0000 7fdc4fc9f640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T08:38:12.917 DEBUG:teuthology.orchestra.run.smithi098:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-19T08:38:13.038 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.035+0000 7f53a4a75640 1 Processor -- start 2024-09-19T08:38:13.038 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.039+0000 7f53a4a75640 1 -- start start 2024-09-19T08:38:13.040 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.039+0000 7f53a4a75640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f53a015c470 0x7f53a0073040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:13.040 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.039+0000 7f53a4a75640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53a015c850 con 0x7f53a0073580 2024-09-19T08:38:13.040 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.039+0000 7f53a4a75640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53a0160e40 con 0x7f53a00cdb40 2024-09-19T08:38:13.040 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.039+0000 7f53a4a75640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f53a0072bb0 con 0x7f53a015c470 2024-09-19T08:38:13.040 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.039+0000 7f539e59c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f53a015c470 0x7f53a0073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:13.040 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.039+0000 7f5395d9b640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f53a0073580 0x7f53a006bc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42704/0 (socket says 172.21.15.98:42704) 2024-09-19T08:38:13.040 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.039+0000 7f5395d9b640 1 -- 172.21.15.98:0/3977734086 learned_addr learned my addr 172.21.15.98:0/3977734086 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:13.040 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.039+0000 7f539e59c640 1 -- 172.21.15.98:0/3977734086 >> v1:172.21.15.139:6789/0 conn(0x7f53a00cdb40 legacy=0x7f53a00ce0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:13.040 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.039+0000 7f539e59c640 1 -- 172.21.15.98:0/3977734086 >> v1:172.21.15.98:6789/0 conn(0x7f53a0073580 legacy=0x7f53a006bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:13.041 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.039+0000 7f539e59c640 1 -- 172.21.15.98:0/3977734086 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a0072280 con 0x7f53a015c470 2024-09-19T08:38:13.041 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.039+0000 7f539dd9b640 1 -- 172.21.15.98:0/3977734086 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2762569499 0 0) 0x7f53a015c850 con 0x7f53a0073580 2024-09-19T08:38:13.042 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.039+0000 7f539e59c640 1 --2- 172.21.15.98:0/3977734086 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53a015c470 0x7f53a0073040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f53900097c0 tx=0x7f539001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d7d123586a6260 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:13.042 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.039+0000 7f539dd9b640 1 -- 172.21.15.98:0/3977734086 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53900ab430 con 0x7f53a015c470 2024-09-19T08:38:13.042 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.043+0000 7f539dd9b640 1 -- 172.21.15.98:0/3977734086 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53900b0070 con 0x7f53a015c470 2024-09-19T08:38:13.042 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.043+0000 7f539dd9b640 1 -- 172.21.15.98:0/3977734086 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53900ab7f0 con 0x7f53a015c470 2024-09-19T08:38:13.043 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.043+0000 7f53a4a75640 1 -- 172.21.15.98:0/3977734086 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53a015c470 msgr2=0x7f53a0073040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:13.043 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.043+0000 7f53a4a75640 1 --2- 172.21.15.98:0/3977734086 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53a015c470 0x7f53a0073040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f53900097c0 tx=0x7f539001c430 comp rx=0 tx=0).stop 2024-09-19T08:38:13.043 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.043+0000 7f53a4a75640 1 -- 172.21.15.98:0/3977734086 shutdown_connections 2024-09-19T08:38:13.043 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.043+0000 7f53a4a75640 1 --2- 172.21.15.98:0/3977734086 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53a015c470 0x7f53a0073040 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:13.043 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.043+0000 7f53a4a75640 1 -- 172.21.15.98:0/3977734086 >> 172.21.15.98:0/3977734086 conn(0x7f53a00b1f00 msgr2=0x7f53a00b2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:13.043 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.043+0000 7f53a4a75640 1 -- 172.21.15.98:0/3977734086 shutdown_connections 2024-09-19T08:38:13.044 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.043+0000 7f53a4a75640 1 -- 172.21.15.98:0/3977734086 wait complete. 2024-09-19T08:38:13.044 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.043+0000 7f53a4a75640 1 Processor -- start 2024-09-19T08:38:13.044 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.043+0000 7f53a4a75640 1 -- start start 2024-09-19T08:38:13.046 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.043+0000 7f53a4a75640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53a00cdb40 0x7f53a015af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:13.046 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.043+0000 7f53a4a75640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53a015c470 0x7f53a0151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:13.046 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.043+0000 7f53a4a75640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53a0156c80 0x7f53a0157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:13.046 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.043+0000 7f53a4a75640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f53a006c490 con 0x7f53a00cdb40 2024-09-19T08:38:13.046 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.043+0000 7f53a4a75640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f53a006c310 con 0x7f53a0156c80 2024-09-19T08:38:13.046 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.043+0000 7f53a4a75640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f53a00ce7c0 con 0x7f53a015c470 2024-09-19T08:38:13.046 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.043+0000 7f539e59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53a00cdb40 0x7f53a015af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:13.046 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.043+0000 7f539e59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53a00cdb40 0x7f53a015af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41658/0 (socket says 172.21.15.98:41658) 2024-09-19T08:38:13.046 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.043+0000 7f539e59c640 1 -- 172.21.15.98:0/1317166466 learned_addr learned my addr 172.21.15.98:0/1317166466 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:13.047 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.043+0000 7f539e59c640 1 -- 172.21.15.98:0/1317166466 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53a015c470 msgr2=0x7f53a0151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:13.047 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.043+0000 7f539ed9d640 1 --2- 172.21.15.98:0/1317166466 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53a0156c80 0x7f53a0157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:13.047 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.043+0000 7f5395d9b640 1 --2- 172.21.15.98:0/1317166466 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53a015c470 0x7f53a0151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:13.047 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.043+0000 7f539e59c640 1 --2- 172.21.15.98:0/1317166466 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53a015c470 0x7f53a0151070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:13.047 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.043+0000 7f539e59c640 1 -- 172.21.15.98:0/1317166466 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53a0156c80 msgr2=0x7f53a0157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:13.047 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.043+0000 7f539e59c640 1 --2- 172.21.15.98:0/1317166466 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53a0156c80 0x7f53a0157fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:13.047 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.043+0000 7f539e59c640 1 -- 172.21.15.98:0/1317166466 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a01587e0 con 0x7f53a00cdb40 2024-09-19T08:38:13.047 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.043+0000 7f539e59c640 1 --2- 172.21.15.98:0/1317166466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53a00cdb40 0x7f53a015af00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f53900097c0 tx=0x7f539001ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:13.047 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.043+0000 7f53977fe640 1 -- 172.21.15.98:0/1317166466 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5390004220 con 0x7f53a00cdb40 2024-09-19T08:38:13.047 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.043+0000 7f53977fe640 1 -- 172.21.15.98:0/1317166466 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53900043c0 con 0x7f53a00cdb40 2024-09-19T08:38:13.047 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.043+0000 7f53a4a75640 1 -- 172.21.15.98:0/1317166466 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53a0155ec0 con 0x7f53a00cdb40 2024-09-19T08:38:13.047 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.043+0000 7f53977fe640 1 -- 172.21.15.98:0/1317166466 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5390003820 con 0x7f53a00cdb40 2024-09-19T08:38:13.047 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.043+0000 7f53a4a75640 1 -- 172.21.15.98:0/1317166466 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53a0156480 con 0x7f53a00cdb40 2024-09-19T08:38:13.048 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.047+0000 7f53a4a75640 1 -- 172.21.15.98:0/1317166466 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53a00cd560 con 0x7f53a00cdb40 2024-09-19T08:38:13.048 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.047+0000 7f53977fe640 1 -- 172.21.15.98:0/1317166466 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f53900039c0 con 0x7f53a00cdb40 2024-09-19T08:38:13.049 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.047+0000 7f53977fe640 1 --2- 172.21.15.98:0/1317166466 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f53800678c0 0x7f5380069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:13.049 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.047+0000 7f53977fe640 1 -- 172.21.15.98:0/1317166466 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f53900a8020 con 0x7f53a00cdb40 2024-09-19T08:38:13.049 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.047+0000 7f5395d9b640 1 --2- 172.21.15.98:0/1317166466 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f53800678c0 0x7f5380069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:13.049 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.047+0000 7f5395d9b640 1 --2- 172.21.15.98:0/1317166466 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f53800678c0 0x7f5380069d80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f53a015be60 tx=0x7f5388009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:13.052 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.051+0000 7f53977fe640 1 -- 172.21.15.98:0/1317166466 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f53900c4070 con 0x7f53a00cdb40 2024-09-19T08:38:13.212 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.211+0000 7f53a4a75640 1 -- 172.21.15.98:0/1317166466 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f53a00cc490 con 0x7f53a00cdb40 2024-09-19T08:38:13.214 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.215+0000 7f53977fe640 1 -- 172.21.15.98:0/1317166466 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12896 (secure 0 0 0) 0x7f5390016050 con 0x7f53a00cdb40 2024-09-19T08:38:13.214 INFO:teuthology.misc.health.smithi098.stdout: 2024-09-19T08:38:13.215 INFO:teuthology.misc.health.smithi098.stdout:{"epoch":17,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:38:10.703074+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:38:13.217 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.215+0000 7f53a4a75640 1 -- 172.21.15.98:0/1317166466 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f53800678c0 msgr2=0x7f5380069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:13.217 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.215+0000 7f53a4a75640 1 --2- 172.21.15.98:0/1317166466 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f53800678c0 0x7f5380069d80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f53a015be60 tx=0x7f5388009040 comp rx=0 tx=0).stop 2024-09-19T08:38:13.217 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.215+0000 7f53a4a75640 1 -- 172.21.15.98:0/1317166466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53a00cdb40 msgr2=0x7f53a015af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:13.217 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.215+0000 7f53a4a75640 1 --2- 172.21.15.98:0/1317166466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53a00cdb40 0x7f53a015af00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f53900097c0 tx=0x7f539001ac70 comp rx=0 tx=0).stop 2024-09-19T08:38:13.217 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.215+0000 7f53a4a75640 1 -- 172.21.15.98:0/1317166466 shutdown_connections 2024-09-19T08:38:13.217 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.215+0000 7f53a4a75640 1 --2- 172.21.15.98:0/1317166466 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f53800678c0 0x7f5380069d80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:13.217 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.215+0000 7f53a4a75640 1 --2- 172.21.15.98:0/1317166466 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53a0156c80 0x7f53a0157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:13.217 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.215+0000 7f53a4a75640 1 --2- 172.21.15.98:0/1317166466 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53a015c470 0x7f53a0151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:13.217 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.215+0000 7f53a4a75640 1 --2- 172.21.15.98:0/1317166466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53a00cdb40 0x7f53a015af00 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:13.217 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.215+0000 7f53a4a75640 1 -- 172.21.15.98:0/1317166466 >> 172.21.15.98:0/1317166466 conn(0x7f53a00b1f00 msgr2=0x7f53a0167480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:13.217 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.215+0000 7f53a4a75640 1 -- 172.21.15.98:0/1317166466 shutdown_connections 2024-09-19T08:38:13.218 INFO:teuthology.misc.health.smithi098.stderr:2024-09-19T08:38:13.215+0000 7f53a4a75640 1 -- 172.21.15.98:0/1317166466 wait complete. 2024-09-19T08:38:13.231 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-19T08:38:13.231 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-19T08:38:13.231 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-19T08:38:13.231 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-19T08:38:13.234 INFO:tasks.ceph.mds.a:Started 2024-09-19T08:38:13.234 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-19T08:38:13.234 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-19T08:38:13.235 INFO:tasks.ceph.mds.c:Started 2024-09-19T08:38:13.236 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-19T08:38:13.236 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-19T08:38:13.238 INFO:tasks.ceph.mds.b:Started 2024-09-19T08:38:13.238 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-19T08:38:13.239 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-19T08:38:13.241 INFO:tasks.ceph.mds.d:Started 2024-09-19T08:38:13.241 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-19T08:38:13.241 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:38:13.310 INFO:tasks.ceph.mds.c.smithi098.stdout:starting mds.c at 2024-09-19T08:38:13.312 INFO:tasks.ceph.mds.a.smithi098.stdout:starting mds.a at 2024-09-19T08:38:13.331 INFO:tasks.ceph.mds.b.smithi139.stdout:starting mds.b at 2024-09-19T08:38:13.335 INFO:tasks.ceph.mds.d.smithi139.stdout:starting mds.d at 2024-09-19T08:38:13.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.359+0000 7fdf6c9aa640 1 Processor -- start 2024-09-19T08:38:13.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.359+0000 7fdf6c9aa640 1 -- start start 2024-09-19T08:38:13.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.363+0000 7fdf6c9aa640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fdf680b7f80 0x7fdf6815f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:13.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.363+0000 7fdf6c9aa640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdf680d4820 0x7fdf681621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:13.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.363+0000 7fdf6c9aa640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf68068440 con 0x7fdf680b9970 2024-09-19T08:38:13.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.363+0000 7fdf6c9aa640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fdf68163050 con 0x7fdf680b7f80 2024-09-19T08:38:13.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.363+0000 7fdf6c9aa640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fdf68162a90 con 0x7fdf680d4820 2024-09-19T08:38:13.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.363+0000 7fdf6659c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fdf680b7f80 0x7fdf6815f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:13.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.363+0000 7fdf6659c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fdf680b7f80 0x7fdf6815f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41682/0 (socket says 172.21.15.98:41682) 2024-09-19T08:38:13.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.363+0000 7fdf6659c640 1 -- 172.21.15.98:0/1510604812 learned_addr learned my addr 172.21.15.98:0/1510604812 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:13.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.363+0000 7fdf6659c640 1 -- 172.21.15.98:0/1510604812 >> v2:172.21.15.139:3300/0 conn(0x7fdf680d4820 msgr2=0x7fdf681621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:13.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.363+0000 7fdf6659c640 1 --2- 172.21.15.98:0/1510604812 >> v2:172.21.15.139:3300/0 conn(0x7fdf680d4820 0x7fdf681621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:13.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.363+0000 7fdf6659c640 1 -- 172.21.15.98:0/1510604812 >> v1:172.21.15.139:6789/0 conn(0x7fdf680b9970 legacy=0x7fdf680b9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:13.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.363+0000 7fdf6659c640 1 -- 172.21.15.98:0/1510604812 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf68162d70 con 0x7fdf680b7f80 2024-09-19T08:38:13.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.363+0000 7fdf6659c640 1 --2- 172.21.15.98:0/1510604812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf680b7f80 0x7fdf6815f560 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdf680ceb00 tx=0x7fdf5801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf03ff1742b3946b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:13.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.363+0000 7fdf6559a640 1 -- 172.21.15.98:0/1510604812 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf580b0070 con 0x7fdf680b7f80 2024-09-19T08:38:13.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.363+0000 7fdf6559a640 1 -- 172.21.15.98:0/1510604812 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf580ab600 con 0x7fdf680b7f80 2024-09-19T08:38:13.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.363+0000 7fdf6559a640 1 -- 172.21.15.98:0/1510604812 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf580ab9d0 con 0x7fdf680b7f80 2024-09-19T08:38:13.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.363+0000 7fdf6c9aa640 1 -- 172.21.15.98:0/1510604812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf680b7f80 msgr2=0x7fdf6815f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:13.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.363+0000 7fdf6c9aa640 1 --2- 172.21.15.98:0/1510604812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf680b7f80 0x7fdf6815f560 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdf680ceb00 tx=0x7fdf5801c450 comp rx=0 tx=0).stop 2024-09-19T08:38:13.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.363+0000 7fdf6c9aa640 1 -- 172.21.15.98:0/1510604812 shutdown_connections 2024-09-19T08:38:13.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.363+0000 7fdf6c9aa640 1 --2- 172.21.15.98:0/1510604812 >> v2:172.21.15.139:3300/0 conn(0x7fdf680d4820 0x7fdf681621b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:13.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.363+0000 7fdf6c9aa640 1 --2- 172.21.15.98:0/1510604812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf680b7f80 0x7fdf6815f560 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:13.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.363+0000 7fdf6c9aa640 1 -- 172.21.15.98:0/1510604812 >> 172.21.15.98:0/1510604812 conn(0x7fdf68074e40 msgr2=0x7fdf68075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:13.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.363+0000 7fdf6c9aa640 1 -- 172.21.15.98:0/1510604812 shutdown_connections 2024-09-19T08:38:13.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.363+0000 7fdf6c9aa640 1 -- 172.21.15.98:0/1510604812 wait complete. 2024-09-19T08:38:13.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.367+0000 7fdf6c9aa640 1 Processor -- start 2024-09-19T08:38:13.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.367+0000 7fdf6c9aa640 1 -- start start 2024-09-19T08:38:13.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.367+0000 7fdf6c9aa640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf680b7f80 0x7fdf6815fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:13.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.367+0000 7fdf6c9aa640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf680b9970 0x7fdf68158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:13.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.367+0000 7fdf6c9aa640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf680d4820 0x7fdf68154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:13.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.367+0000 7fdf6c9aa640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdf680b84e0 con 0x7fdf680b7f80 2024-09-19T08:38:13.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.367+0000 7fdf6c9aa640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdf680b8360 con 0x7fdf680b9970 2024-09-19T08:38:13.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.367+0000 7fdf6c9aa640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdf68144590 con 0x7fdf680d4820 2024-09-19T08:38:13.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.367+0000 7fdf6659c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf680b7f80 0x7fdf6815fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:13.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.367+0000 7fdf6659c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf680b7f80 0x7fdf6815fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41686/0 (socket says 172.21.15.98:41686) 2024-09-19T08:38:13.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.367+0000 7fdf6659c640 1 -- 172.21.15.98:0/2939838654 learned_addr learned my addr 172.21.15.98:0/2939838654 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:13.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.367+0000 7fdf65d9b640 1 --2- 172.21.15.98:0/2939838654 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf680b9970 0x7fdf68158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:13.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.367+0000 7fdf6659c640 1 -- 172.21.15.98:0/2939838654 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf680d4820 msgr2=0x7fdf68154790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:13.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.367+0000 7fdf66d9d640 1 --2- 172.21.15.98:0/2939838654 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf680d4820 0x7fdf68154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:13.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.367+0000 7fdf6659c640 1 --2- 172.21.15.98:0/2939838654 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf680d4820 0x7fdf68154790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:13.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.367+0000 7fdf6659c640 1 -- 172.21.15.98:0/2939838654 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf680b9970 msgr2=0x7fdf68158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:13.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.367+0000 7fdf6659c640 1 --2- 172.21.15.98:0/2939838654 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf680b9970 0x7fdf68158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:13.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.367+0000 7fdf6659c640 1 -- 172.21.15.98:0/2939838654 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf6813d5e0 con 0x7fdf680b7f80 2024-09-19T08:38:13.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.367+0000 7fdf65d9b640 1 --2- 172.21.15.98:0/2939838654 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf680b9970 0x7fdf68158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:13.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.367+0000 7fdf66d9d640 1 --2- 172.21.15.98:0/2939838654 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf680d4820 0x7fdf68154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:13.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.367+0000 7fdf6659c640 1 --2- 172.21.15.98:0/2939838654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf680b7f80 0x7fdf6815fcf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdf580ae040 tx=0x7fdf5801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:13.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.367+0000 7fdf56ffd640 1 -- 172.21.15.98:0/2939838654 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf580ba020 con 0x7fdf680b7f80 2024-09-19T08:38:13.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.367+0000 7fdf56ffd640 1 -- 172.21.15.98:0/2939838654 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf580b6640 con 0x7fdf680b7f80 2024-09-19T08:38:13.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.367+0000 7fdf6c9aa640 1 -- 172.21.15.98:0/2939838654 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf6813d8c0 con 0x7fdf680b7f80 2024-09-19T08:38:13.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.367+0000 7fdf56ffd640 1 -- 172.21.15.98:0/2939838654 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf580b0040 con 0x7fdf680b7f80 2024-09-19T08:38:13.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.367+0000 7fdf6c9aa640 1 -- 172.21.15.98:0/2939838654 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf6813de80 con 0x7fdf680b7f80 2024-09-19T08:38:13.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.367+0000 7fdf6c9aa640 1 -- 172.21.15.98:0/2939838654 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf68068f50 con 0x7fdf680b7f80 2024-09-19T08:38:13.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.367+0000 7fdf56ffd640 1 -- 172.21.15.98:0/2939838654 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdf580bd020 con 0x7fdf680b7f80 2024-09-19T08:38:13.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.367+0000 7fdf56ffd640 1 --2- 172.21.15.98:0/2939838654 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdf480678f0 0x7fdf48069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:13.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.371+0000 7fdf56ffd640 1 -- 172.21.15.98:0/2939838654 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fdf5811fb30 con 0x7fdf680b7f80 2024-09-19T08:38:13.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.371+0000 7fdf65d9b640 1 --2- 172.21.15.98:0/2939838654 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdf480678f0 0x7fdf48069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:13.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.371+0000 7fdf65d9b640 1 --2- 172.21.15.98:0/2939838654 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdf480678f0 0x7fdf48069db0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdf680d3900 tx=0x7fdf5c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:13.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.375+0000 7fdf56ffd640 1 -- 172.21.15.98:0/2939838654 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fdf580c4030 con 0x7fdf680b7f80 2024-09-19T08:38:13.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.571+0000 7fdf6c9aa640 1 -- 172.21.15.98:0/2939838654 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf68158b40 con 0x7fdf680b7f80 2024-09-19T08:38:13.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.575+0000 7fdf56ffd640 1 -- 172.21.15.98:0/2939838654 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1434 (secure 0 0 0) 0x7fdf580160d0 con 0x7fdf680b7f80 2024-09-19T08:38:13.575 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:38:13.575 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":2,"btime":"2024-09-19T08:38:13:318682+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2832328076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2832328076},{"type":"v1","addr":"172.21.15.98:6837","nonce":2832328076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T08:38:13.575 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 2 2024-09-19T08:38:13.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.575+0000 7fdf6c9aa640 1 -- 172.21.15.98:0/2939838654 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdf480678f0 msgr2=0x7fdf48069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:13.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.575+0000 7fdf6c9aa640 1 --2- 172.21.15.98:0/2939838654 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdf480678f0 0x7fdf48069db0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdf680d3900 tx=0x7fdf5c008040 comp rx=0 tx=0).stop 2024-09-19T08:38:13.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.575+0000 7fdf6c9aa640 1 -- 172.21.15.98:0/2939838654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf680b7f80 msgr2=0x7fdf6815fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:13.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.575+0000 7fdf6c9aa640 1 --2- 172.21.15.98:0/2939838654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf680b7f80 0x7fdf6815fcf0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdf580ae040 tx=0x7fdf5801ac90 comp rx=0 tx=0).stop 2024-09-19T08:38:13.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.575+0000 7fdf6c9aa640 1 -- 172.21.15.98:0/2939838654 shutdown_connections 2024-09-19T08:38:13.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.575+0000 7fdf6c9aa640 1 --2- 172.21.15.98:0/2939838654 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdf480678f0 0x7fdf48069db0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:13.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.575+0000 7fdf6c9aa640 1 --2- 172.21.15.98:0/2939838654 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf680d4820 0x7fdf68154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:13.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.575+0000 7fdf6c9aa640 1 --2- 172.21.15.98:0/2939838654 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf680b9970 0x7fdf68158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:13.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.575+0000 7fdf6c9aa640 1 --2- 172.21.15.98:0/2939838654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf680b7f80 0x7fdf6815fcf0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:13.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.575+0000 7fdf6c9aa640 1 -- 172.21.15.98:0/2939838654 >> 172.21.15.98:0/2939838654 conn(0x7fdf68074e40 msgr2=0x7fdf680d2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:13.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.575+0000 7fdf6c9aa640 1 -- 172.21.15.98:0/2939838654 shutdown_connections 2024-09-19T08:38:13.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:13.575+0000 7fdf6c9aa640 1 -- 172.21.15.98:0/2939838654 wait complete. 2024-09-19T08:38:15.594 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:38:15.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.719+0000 7fdbad8c0640 1 Processor -- start 2024-09-19T08:38:15.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.719+0000 7fdbad8c0640 1 -- start start 2024-09-19T08:38:15.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.723+0000 7fdbad8c0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdba806c3a0 0x7fdba80cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:15.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.723+0000 7fdbad8c0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdba80cf900 0x7fdba80723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:15.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.723+0000 7fdbad8c0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdba806d310 con 0x7fdba815c550 2024-09-19T08:38:15.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.723+0000 7fdbad8c0640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fdba806d5f0 con 0x7fdba80cf900 2024-09-19T08:38:15.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.723+0000 7fdbad8c0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdba806d8d0 con 0x7fdba806c3a0 2024-09-19T08:38:15.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.723+0000 7fdba77fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdba80cf900 0x7fdba80723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:15.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.723+0000 7fdba67fc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdba806c3a0 0x7fdba80cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:15.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.723+0000 7fdba6ffd640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fdba815c550 0x7fdba806bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:56130/0 (socket says 172.21.15.98:56130) 2024-09-19T08:38:15.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.723+0000 7fdba6ffd640 1 -- 172.21.15.98:0/2966679368 learned_addr learned my addr 172.21.15.98:0/2966679368 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:15.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.723+0000 7fdba67fc640 1 -- 172.21.15.98:0/2966679368 >> v2:172.21.15.139:3300/0 conn(0x7fdba80cf900 msgr2=0x7fdba80723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:15.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.723+0000 7fdba67fc640 1 --2- 172.21.15.98:0/2966679368 >> v2:172.21.15.139:3300/0 conn(0x7fdba80cf900 0x7fdba80723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:15.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.723+0000 7fdba67fc640 1 -- 172.21.15.98:0/2966679368 >> v1:172.21.15.139:6789/0 conn(0x7fdba815c550 legacy=0x7fdba806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:15.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.723+0000 7fdba67fc640 1 -- 172.21.15.98:0/2966679368 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdba806dbb0 con 0x7fdba806c3a0 2024-09-19T08:38:15.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.723+0000 7fdba77fe640 1 --2- 172.21.15.98:0/2966679368 >> v2:172.21.15.139:3300/0 conn(0x7fdba80cf900 0x7fdba80723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:38:15.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.727+0000 7fdba67fc640 1 --2- 172.21.15.98:0/2966679368 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdba806c3a0 0x7fdba80cf3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdb940097c0 tx=0x7fdb9401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=163ce33815dfaea6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:15.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.727+0000 7fdba5ffb640 1 -- 172.21.15.98:0/2966679368 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb940b1070 con 0x7fdba806c3a0 2024-09-19T08:38:15.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.727+0000 7fdba5ffb640 1 -- 172.21.15.98:0/2966679368 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb940ac410 con 0x7fdba806c3a0 2024-09-19T08:38:15.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.727+0000 7fdba5ffb640 1 -- 172.21.15.98:0/2966679368 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb940ac7e0 con 0x7fdba806c3a0 2024-09-19T08:38:15.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.727+0000 7fdbad8c0640 1 -- 172.21.15.98:0/2966679368 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdba806c3a0 msgr2=0x7fdba80cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:15.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.727+0000 7fdbad8c0640 1 --2- 172.21.15.98:0/2966679368 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdba806c3a0 0x7fdba80cf3c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdb940097c0 tx=0x7fdb9401c4a0 comp rx=0 tx=0).stop 2024-09-19T08:38:15.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.727+0000 7fdbad8c0640 1 -- 172.21.15.98:0/2966679368 shutdown_connections 2024-09-19T08:38:15.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.727+0000 7fdbad8c0640 1 --2- 172.21.15.98:0/2966679368 >> v2:172.21.15.139:3300/0 conn(0x7fdba80cf900 0x7fdba80723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:15.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.727+0000 7fdbad8c0640 1 --2- 172.21.15.98:0/2966679368 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdba806c3a0 0x7fdba80cf3c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:15.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.727+0000 7fdbad8c0640 1 -- 172.21.15.98:0/2966679368 >> 172.21.15.98:0/2966679368 conn(0x7fdba80b2010 msgr2=0x7fdba80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:15.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.727+0000 7fdbad8c0640 1 -- 172.21.15.98:0/2966679368 shutdown_connections 2024-09-19T08:38:15.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.727+0000 7fdbad8c0640 1 -- 172.21.15.98:0/2966679368 wait complete. 2024-09-19T08:38:15.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.727+0000 7fdbad8c0640 1 Processor -- start 2024-09-19T08:38:15.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.727+0000 7fdbad8c0640 1 -- start start 2024-09-19T08:38:15.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.727+0000 7fdbad8c0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdba806c3a0 0x7fdba8152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:15.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.727+0000 7fdbad8c0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdba80cf900 0x7fdba814f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:15.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.727+0000 7fdbad8c0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdba815c550 0x7fdba813b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:15.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.727+0000 7fdbad8c0640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdba8073480 con 0x7fdba815c550 2024-09-19T08:38:15.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.727+0000 7fdbad8c0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdba8073300 con 0x7fdba806c3a0 2024-09-19T08:38:15.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.727+0000 7fdbad8c0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdba8073600 con 0x7fdba80cf900 2024-09-19T08:38:15.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.727+0000 7fdba77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdba815c550 0x7fdba813b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:15.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.727+0000 7fdba77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdba815c550 0x7fdba813b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41688/0 (socket says 172.21.15.98:41688) 2024-09-19T08:38:15.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.727+0000 7fdba77fe640 1 -- 172.21.15.98:0/945282751 learned_addr learned my addr 172.21.15.98:0/945282751 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:15.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.727+0000 7fdba67fc640 1 --2- 172.21.15.98:0/945282751 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdba80cf900 0x7fdba814f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:15.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.727+0000 7fdba6ffd640 1 --2- 172.21.15.98:0/945282751 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdba806c3a0 0x7fdba8152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:15.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.727+0000 7fdba77fe640 1 -- 172.21.15.98:0/945282751 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdba80cf900 msgr2=0x7fdba814f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:15.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.727+0000 7fdba77fe640 1 --2- 172.21.15.98:0/945282751 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdba80cf900 0x7fdba814f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:15.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.727+0000 7fdba77fe640 1 -- 172.21.15.98:0/945282751 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdba806c3a0 msgr2=0x7fdba8152880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:15.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.727+0000 7fdba77fe640 1 --2- 172.21.15.98:0/945282751 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdba806c3a0 0x7fdba8152880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:15.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.727+0000 7fdba77fe640 1 -- 172.21.15.98:0/945282751 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdba8150370 con 0x7fdba815c550 2024-09-19T08:38:15.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.731+0000 7fdba77fe640 1 --2- 172.21.15.98:0/945282751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdba815c550 0x7fdba813b5c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdba0009e60 tx=0x7fdba000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:15.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.731+0000 7fdb9ffff640 1 -- 172.21.15.98:0/945282751 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba0018030 con 0x7fdba815c550 2024-09-19T08:38:15.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.731+0000 7fdb9ffff640 1 -- 172.21.15.98:0/945282751 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdba000cdf0 con 0x7fdba815c550 2024-09-19T08:38:15.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.731+0000 7fdbad8c0640 1 -- 172.21.15.98:0/945282751 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdba8151da0 con 0x7fdba815c550 2024-09-19T08:38:15.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.731+0000 7fdb9ffff640 1 -- 172.21.15.98:0/945282751 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba0003bc0 con 0x7fdba815c550 2024-09-19T08:38:15.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.731+0000 7fdbad8c0640 1 -- 172.21.15.98:0/945282751 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdba8151a50 con 0x7fdba815c550 2024-09-19T08:38:15.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.731+0000 7fdbad8c0640 1 -- 172.21.15.98:0/945282751 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdba80cf3c0 con 0x7fdba815c550 2024-09-19T08:38:15.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.731+0000 7fdb9ffff640 1 -- 172.21.15.98:0/945282751 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdba001b070 con 0x7fdba815c550 2024-09-19T08:38:15.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.731+0000 7fdb9ffff640 1 --2- 172.21.15.98:0/945282751 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdb900678c0 0x7fdb90069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:15.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.731+0000 7fdb9ffff640 1 -- 172.21.15.98:0/945282751 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fdba0086910 con 0x7fdba815c550 2024-09-19T08:38:15.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.731+0000 7fdba6ffd640 1 --2- 172.21.15.98:0/945282751 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdb900678c0 0x7fdb90069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:15.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.731+0000 7fdba6ffd640 1 --2- 172.21.15.98:0/945282751 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdb900678c0 0x7fdb90069d80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdba81552e0 tx=0x7fdb98009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:15.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.735+0000 7fdb9ffff640 1 -- 172.21.15.98:0/945282751 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fdba0010040 con 0x7fdba815c550 2024-09-19T08:38:15.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.915+0000 7fdbad8c0640 1 -- 172.21.15.98:0/945282751 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdba80cc910 con 0x7fdba815c550 2024-09-19T08:38:15.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.919+0000 7fdb9ffff640 1 -- 172.21.15.98:0/945282751 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3909 (secure 0 0 0) 0x7fdba001b360 con 0x7fdba815c550 2024-09-19T08:38:15.919 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:38:15.919 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":3,"btime":"2024-09-19T08:38:14:319850+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/2038744461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":2038744461},{"type":"v1","addr":"172.21.15.139:6833","nonce":2038744461}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/765134249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":765134249},{"type":"v1","addr":"172.21.15.98:6835","nonce":765134249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2832328076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2832328076},{"type":"v1","addr":"172.21.15.98:6837","nonce":2832328076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4261,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1879089763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1879089763},{"type":"v1","addr":"172.21.15.139:6835","nonce":1879089763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-19T08:38:15.919 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 3 2024-09-19T08:38:15.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.919+0000 7fdbad8c0640 1 -- 172.21.15.98:0/945282751 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdb900678c0 msgr2=0x7fdb90069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:15.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.919+0000 7fdbad8c0640 1 --2- 172.21.15.98:0/945282751 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdb900678c0 0x7fdb90069d80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdba81552e0 tx=0x7fdb98009040 comp rx=0 tx=0).stop 2024-09-19T08:38:15.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.919+0000 7fdbad8c0640 1 -- 172.21.15.98:0/945282751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdba815c550 msgr2=0x7fdba813b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:15.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.919+0000 7fdbad8c0640 1 --2- 172.21.15.98:0/945282751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdba815c550 0x7fdba813b5c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdba0009e60 tx=0x7fdba000c490 comp rx=0 tx=0).stop 2024-09-19T08:38:15.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.919+0000 7fdbad8c0640 1 -- 172.21.15.98:0/945282751 shutdown_connections 2024-09-19T08:38:15.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.919+0000 7fdbad8c0640 1 --2- 172.21.15.98:0/945282751 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdb900678c0 0x7fdb90069d80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:15.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.919+0000 7fdbad8c0640 1 --2- 172.21.15.98:0/945282751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdba815c550 0x7fdba813b5c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:15.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.919+0000 7fdbad8c0640 1 --2- 172.21.15.98:0/945282751 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdba80cf900 0x7fdba814f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:15.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.919+0000 7fdbad8c0640 1 --2- 172.21.15.98:0/945282751 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdba806c3a0 0x7fdba8152880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:15.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.919+0000 7fdbad8c0640 1 -- 172.21.15.98:0/945282751 >> 172.21.15.98:0/945282751 conn(0x7fdba80b2010 msgr2=0x7fdba81586e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:15.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.919+0000 7fdbad8c0640 1 -- 172.21.15.98:0/945282751 shutdown_connections 2024-09-19T08:38:15.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:15.919+0000 7fdbad8c0640 1 -- 172.21.15.98:0/945282751 wait complete. 2024-09-19T08:38:15.936 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-19T08:38:16.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.055+0000 7f8f745f6640 1 Processor -- start 2024-09-19T08:38:16.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.055+0000 7f8f745f6640 1 -- start start 2024-09-19T08:38:16.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.059+0000 7f8f745f6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8f6c15c550 0x7f8f6c06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:16.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.059+0000 7f8f745f6640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8f6c0cfad0 0x7f8f6c0723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:16.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.059+0000 7f8f745f6640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f6c06d310 con 0x7f8f6c06c1d0 2024-09-19T08:38:16.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.059+0000 7f8f745f6640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f8f6c06c640 con 0x7f8f6c0cfad0 2024-09-19T08:38:16.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.059+0000 7f8f745f6640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f8f6c06d5f0 con 0x7f8f6c15c550 2024-09-19T08:38:16.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.059+0000 7f8f72b93640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8f6c0cfad0 0x7f8f6c0723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:16.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.059+0000 7f8f72b93640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8f6c0cfad0 0x7f8f6c0723a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41702/0 (socket says 172.21.15.98:41702) 2024-09-19T08:38:16.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.059+0000 7f8f72b93640 1 -- 172.21.15.98:0/4240030570 learned_addr learned my addr 172.21.15.98:0/4240030570 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:16.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.059+0000 7f8f72392640 1 --2- 172.21.15.98:0/4240030570 >> v2:172.21.15.139:3300/0 conn(0x7f8f6c15c550 0x7f8f6c06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:16.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.059+0000 7f8f72b93640 1 -- 172.21.15.98:0/4240030570 >> v2:172.21.15.139:3300/0 conn(0x7f8f6c15c550 msgr2=0x7f8f6c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:16.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.059+0000 7f8f72b93640 1 --2- 172.21.15.98:0/4240030570 >> v2:172.21.15.139:3300/0 conn(0x7f8f6c15c550 0x7f8f6c06bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:16.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.059+0000 7f8f72b93640 1 -- 172.21.15.98:0/4240030570 >> v1:172.21.15.139:6789/0 conn(0x7f8f6c06c1d0 legacy=0x7f8f6c0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:16.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.059+0000 7f8f72b93640 1 -- 172.21.15.98:0/4240030570 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6c06d8d0 con 0x7f8f6c0cfad0 2024-09-19T08:38:16.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.059+0000 7f8f72392640 1 --2- 172.21.15.98:0/4240030570 >> v2:172.21.15.139:3300/0 conn(0x7f8f6c15c550 0x7f8f6c06bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:16.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.059+0000 7f8f72b93640 1 --2- 172.21.15.98:0/4240030570 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f6c0cfad0 0x7f8f6c0723a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c0b56b0 tx=0x7f8f6401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd704505b5e2bbc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:16.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.059+0000 7f8f71390640 1 -- 172.21.15.98:0/4240030570 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f640b0070 con 0x7f8f6c0cfad0 2024-09-19T08:38:16.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.059+0000 7f8f71390640 1 -- 172.21.15.98:0/4240030570 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f640ab600 con 0x7f8f6c0cfad0 2024-09-19T08:38:16.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.059+0000 7f8f71390640 1 -- 172.21.15.98:0/4240030570 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f640ab9d0 con 0x7f8f6c0cfad0 2024-09-19T08:38:16.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.059+0000 7f8f745f6640 1 -- 172.21.15.98:0/4240030570 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f6c0cfad0 msgr2=0x7f8f6c0723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:16.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.059+0000 7f8f745f6640 1 --2- 172.21.15.98:0/4240030570 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f6c0cfad0 0x7f8f6c0723a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c0b56b0 tx=0x7f8f6401c4e0 comp rx=0 tx=0).stop 2024-09-19T08:38:16.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.063+0000 7f8f745f6640 1 -- 172.21.15.98:0/4240030570 shutdown_connections 2024-09-19T08:38:16.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.063+0000 7f8f745f6640 1 --2- 172.21.15.98:0/4240030570 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f6c0cfad0 0x7f8f6c0723a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:16.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.063+0000 7f8f745f6640 1 --2- 172.21.15.98:0/4240030570 >> v2:172.21.15.139:3300/0 conn(0x7f8f6c15c550 0x7f8f6c06bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:16.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.063+0000 7f8f745f6640 1 -- 172.21.15.98:0/4240030570 >> 172.21.15.98:0/4240030570 conn(0x7f8f6c0b2010 msgr2=0x7f8f6c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:16.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.063+0000 7f8f745f6640 1 -- 172.21.15.98:0/4240030570 shutdown_connections 2024-09-19T08:38:16.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.063+0000 7f8f745f6640 1 -- 172.21.15.98:0/4240030570 wait complete. 2024-09-19T08:38:16.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.063+0000 7f8f745f6640 1 Processor -- start 2024-09-19T08:38:16.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.063+0000 7f8f745f6640 1 -- start start 2024-09-19T08:38:16.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.063+0000 7f8f745f6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f6c06c1d0 0x7f8f6c147810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:16.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.063+0000 7f8f745f6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f6c0cfad0 0x7f8f6c14a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:16.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.063+0000 7f8f745f6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f6c15c550 0x7f8f6c149b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:16.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.063+0000 7f8f745f6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c0734c0 con 0x7f8f6c0cfad0 2024-09-19T08:38:16.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.063+0000 7f8f745f6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c073340 con 0x7f8f6c06c1d0 2024-09-19T08:38:16.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.063+0000 7f8f745f6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8f6c073640 con 0x7f8f6c15c550 2024-09-19T08:38:16.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.063+0000 7f8f71b91640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f6c0cfad0 0x7f8f6c14a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:16.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.063+0000 7f8f71b91640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f6c0cfad0 0x7f8f6c14a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41714/0 (socket says 172.21.15.98:41714) 2024-09-19T08:38:16.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.063+0000 7f8f71b91640 1 -- 172.21.15.98:0/1872509609 learned_addr learned my addr 172.21.15.98:0/1872509609 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:16.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.063+0000 7f8f72b93640 1 --2- 172.21.15.98:0/1872509609 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f6c15c550 0x7f8f6c149b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:16.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.063+0000 7f8f72392640 1 --2- 172.21.15.98:0/1872509609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f6c06c1d0 0x7f8f6c147810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:16.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.063+0000 7f8f71b91640 1 -- 172.21.15.98:0/1872509609 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f6c15c550 msgr2=0x7f8f6c149b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:16.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.063+0000 7f8f71b91640 1 --2- 172.21.15.98:0/1872509609 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f6c15c550 0x7f8f6c149b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:16.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.063+0000 7f8f71b91640 1 -- 172.21.15.98:0/1872509609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f6c06c1d0 msgr2=0x7f8f6c147810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:16.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.063+0000 7f8f71b91640 1 --2- 172.21.15.98:0/1872509609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f6c06c1d0 0x7f8f6c147810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:16.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.063+0000 7f8f71b91640 1 -- 172.21.15.98:0/1872509609 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6c14a070 con 0x7f8f6c0cfad0 2024-09-19T08:38:16.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.063+0000 7f8f72b93640 1 --2- 172.21.15.98:0/1872509609 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f6c15c550 0x7f8f6c149b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:16.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.063+0000 7f8f72392640 1 --2- 172.21.15.98:0/1872509609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f6c06c1d0 0x7f8f6c147810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:16.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.063+0000 7f8f71b91640 1 --2- 172.21.15.98:0/1872509609 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f6c0cfad0 0x7f8f6c14a830 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8f5400ba80 tx=0x7f8f5400bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:16.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.063+0000 7f8f6affd640 1 -- 172.21.15.98:0/1872509609 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54019070 con 0x7f8f6c0cfad0 2024-09-19T08:38:16.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.063+0000 7f8f6affd640 1 -- 172.21.15.98:0/1872509609 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f5400cc90 con 0x7f8f6c0cfad0 2024-09-19T08:38:16.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.063+0000 7f8f745f6640 1 -- 172.21.15.98:0/1872509609 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f6c0b4800 con 0x7f8f6c0cfad0 2024-09-19T08:38:16.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.063+0000 7f8f6affd640 1 -- 172.21.15.98:0/1872509609 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54014680 con 0x7f8f6c0cfad0 2024-09-19T08:38:16.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.063+0000 7f8f745f6640 1 -- 172.21.15.98:0/1872509609 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f6c14a310 con 0x7f8f6c0cfad0 2024-09-19T08:38:16.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.063+0000 7f8f745f6640 1 -- 172.21.15.98:0/1872509609 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f6c0723a0 con 0x7f8f6c0cfad0 2024-09-19T08:38:16.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.067+0000 7f8f6affd640 1 -- 172.21.15.98:0/1872509609 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8f5401c070 con 0x7f8f6c0cfad0 2024-09-19T08:38:16.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.067+0000 7f8f6affd640 1 --2- 172.21.15.98:0/1872509609 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8f4c0678c0 0x7f8f4c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:16.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.067+0000 7f8f6affd640 1 -- 172.21.15.98:0/1872509609 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f8f54085800 con 0x7f8f6c0cfad0 2024-09-19T08:38:16.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.067+0000 7f8f72392640 1 --2- 172.21.15.98:0/1872509609 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8f4c0678c0 0x7f8f4c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:16.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.067+0000 7f8f72392640 1 --2- 172.21.15.98:0/1872509609 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8f4c0678c0 0x7f8f4c069d80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c0af4a0 tx=0x7f8f5c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:16.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.071+0000 7f8f6affd640 1 -- 172.21.15.98:0/1872509609 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f8f540565f0 con 0x7f8f6c0cfad0 2024-09-19T08:38:16.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.223+0000 7f8f745f6640 1 -- 172.21.15.98:0/1872509609 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f8f6c0b4d20 con 0x7f8f6c0cfad0 2024-09-19T08:38:16.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.223+0000 7f8f6affd640 1 -- 172.21.15.98:0/1872509609 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7f8f54055ef0 con 0x7f8f6c0cfad0 2024-09-19T08:38:16.225 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:38:16.225 INFO:teuthology.orchestra.run.smithi098.stdout:[ 2024-09-19T08:38:16.225 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:38:16.225 INFO:teuthology.orchestra.run.smithi098.stdout: "poolnum": 1, 2024-09-19T08:38:16.225 INFO:teuthology.orchestra.run.smithi098.stdout: "poolname": ".mgr" 2024-09-19T08:38:16.225 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:38:16.226 INFO:teuthology.orchestra.run.smithi098.stdout:] 2024-09-19T08:38:16.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.227+0000 7f8f745f6640 1 -- 172.21.15.98:0/1872509609 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8f4c0678c0 msgr2=0x7f8f4c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:16.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.227+0000 7f8f745f6640 1 --2- 172.21.15.98:0/1872509609 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8f4c0678c0 0x7f8f4c069d80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c0af4a0 tx=0x7f8f5c008040 comp rx=0 tx=0).stop 2024-09-19T08:38:16.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.227+0000 7f8f745f6640 1 -- 172.21.15.98:0/1872509609 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f6c0cfad0 msgr2=0x7f8f6c14a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:16.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.227+0000 7f8f745f6640 1 --2- 172.21.15.98:0/1872509609 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f6c0cfad0 0x7f8f6c14a830 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8f5400ba80 tx=0x7f8f5400bf40 comp rx=0 tx=0).stop 2024-09-19T08:38:16.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.227+0000 7f8f745f6640 1 -- 172.21.15.98:0/1872509609 shutdown_connections 2024-09-19T08:38:16.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.227+0000 7f8f745f6640 1 --2- 172.21.15.98:0/1872509609 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8f4c0678c0 0x7f8f4c069d80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:16.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.227+0000 7f8f745f6640 1 --2- 172.21.15.98:0/1872509609 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f6c15c550 0x7f8f6c149b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:16.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.227+0000 7f8f745f6640 1 --2- 172.21.15.98:0/1872509609 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f6c0cfad0 0x7f8f6c14a830 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:16.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.227+0000 7f8f745f6640 1 --2- 172.21.15.98:0/1872509609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f6c06c1d0 0x7f8f6c147810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:16.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.227+0000 7f8f745f6640 1 -- 172.21.15.98:0/1872509609 >> 172.21.15.98:0/1872509609 conn(0x7f8f6c0b2010 msgr2=0x7f8f6c0d0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:16.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.227+0000 7f8f745f6640 1 -- 172.21.15.98:0/1872509609 shutdown_connections 2024-09-19T08:38:16.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.227+0000 7f8f745f6640 1 -- 172.21.15.98:0/1872509609 wait complete. 2024-09-19T08:38:16.242 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T08:38:16.242 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T08:38:16.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.371+0000 7f8b4f040640 1 Processor -- start 2024-09-19T08:38:16.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.371+0000 7f8b4f040640 1 -- start start 2024-09-19T08:38:16.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.371+0000 7f8b4f040640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8b4815f6e0 0x7f8b4806d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:16.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.371+0000 7f8b4f040640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8b4815fac0 0x7f8b4806da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:16.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.371+0000 7f8b4f040640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b4815baf0 con 0x7f8b48073680 2024-09-19T08:38:16.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.371+0000 7f8b4f040640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f8b4815c8b0 con 0x7f8b4815f6e0 2024-09-19T08:38:16.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.371+0000 7f8b4f040640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f8b480742d0 con 0x7f8b4815fac0 2024-09-19T08:38:16.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.371+0000 7f8b4cddc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8b4815f6e0 0x7f8b4806d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:16.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.371+0000 7f8b4cddc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8b4815f6e0 0x7f8b4806d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41726/0 (socket says 172.21.15.98:41726) 2024-09-19T08:38:16.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.371+0000 7f8b4cddc640 1 -- 172.21.15.98:0/1178834687 learned_addr learned my addr 172.21.15.98:0/1178834687 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:16.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.371+0000 7f8b47fff640 1 --2- 172.21.15.98:0/1178834687 >> v2:172.21.15.139:3300/0 conn(0x7f8b4815fac0 0x7f8b4806da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:16.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.371+0000 7f8b4cddc640 1 -- 172.21.15.98:0/1178834687 >> v2:172.21.15.139:3300/0 conn(0x7f8b4815fac0 msgr2=0x7f8b4806da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:16.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.371+0000 7f8b4cddc640 1 --2- 172.21.15.98:0/1178834687 >> v2:172.21.15.139:3300/0 conn(0x7f8b4815fac0 0x7f8b4806da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:16.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.371+0000 7f8b4cddc640 1 -- 172.21.15.98:0/1178834687 >> v1:172.21.15.139:6789/0 conn(0x7f8b48073680 legacy=0x7f8b48073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:16.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.371+0000 7f8b4cddc640 1 -- 172.21.15.98:0/1178834687 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b4815bdd0 con 0x7f8b4815f6e0 2024-09-19T08:38:16.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.375+0000 7f8b4cddc640 1 --2- 172.21.15.98:0/1178834687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b4815f6e0 0x7f8b4806d4c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8b480af7f0 tx=0x7f8b3801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=203055a089ca8636 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:16.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.375+0000 7f8b477fe640 1 -- 172.21.15.98:0/1178834687 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b380a56d0 con 0x7f8b4815f6e0 2024-09-19T08:38:16.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.375+0000 7f8b477fe640 1 -- 172.21.15.98:0/1178834687 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b380a5870 con 0x7f8b4815f6e0 2024-09-19T08:38:16.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.375+0000 7f8b477fe640 1 -- 172.21.15.98:0/1178834687 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b380a5c40 con 0x7f8b4815f6e0 2024-09-19T08:38:16.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.375+0000 7f8b4f040640 1 -- 172.21.15.98:0/1178834687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b4815f6e0 msgr2=0x7f8b4806d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:16.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.375+0000 7f8b4f040640 1 --2- 172.21.15.98:0/1178834687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b4815f6e0 0x7f8b4806d4c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8b480af7f0 tx=0x7f8b3801c920 comp rx=0 tx=0).stop 2024-09-19T08:38:16.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.375+0000 7f8b4f040640 1 -- 172.21.15.98:0/1178834687 shutdown_connections 2024-09-19T08:38:16.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.375+0000 7f8b4f040640 1 --2- 172.21.15.98:0/1178834687 >> v2:172.21.15.139:3300/0 conn(0x7f8b4815fac0 0x7f8b4806da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:16.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.375+0000 7f8b4f040640 1 --2- 172.21.15.98:0/1178834687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b4815f6e0 0x7f8b4806d4c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:16.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.375+0000 7f8b4f040640 1 -- 172.21.15.98:0/1178834687 >> 172.21.15.98:0/1178834687 conn(0x7f8b4806c090 msgr2=0x7f8b4806c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:16.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.375+0000 7f8b4f040640 1 -- 172.21.15.98:0/1178834687 shutdown_connections 2024-09-19T08:38:16.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.375+0000 7f8b4f040640 1 -- 172.21.15.98:0/1178834687 wait complete. 2024-09-19T08:38:16.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.375+0000 7f8b4f040640 1 Processor -- start 2024-09-19T08:38:16.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.375+0000 7f8b4f040640 1 -- start start 2024-09-19T08:38:16.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.379+0000 7f8b4f040640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b48073680 0x7f8b4814fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:16.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.379+0000 7f8b4f040640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b4815f6e0 0x7f8b4814c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:16.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.379+0000 7f8b4f040640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b4815fac0 0x7f8b4814d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:16.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.379+0000 7f8b4f040640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8b480cefc0 con 0x7f8b4815f6e0 2024-09-19T08:38:16.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.379+0000 7f8b4f040640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8b480cee40 con 0x7f8b48073680 2024-09-19T08:38:16.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.379+0000 7f8b4f040640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8b480cf140 con 0x7f8b4815fac0 2024-09-19T08:38:16.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.379+0000 7f8b47fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b4815f6e0 0x7f8b4814c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:16.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.379+0000 7f8b47fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b4815f6e0 0x7f8b4814c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41732/0 (socket says 172.21.15.98:41732) 2024-09-19T08:38:16.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.379+0000 7f8b47fff640 1 -- 172.21.15.98:0/1995576947 learned_addr learned my addr 172.21.15.98:0/1995576947 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:16.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.379+0000 7f8b4cddc640 1 --2- 172.21.15.98:0/1995576947 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b48073680 0x7f8b4814fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:16.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.379+0000 7f8b4d5dd640 1 --2- 172.21.15.98:0/1995576947 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b4815fac0 0x7f8b4814d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:16.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.379+0000 7f8b47fff640 1 -- 172.21.15.98:0/1995576947 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b4815fac0 msgr2=0x7f8b4814d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:16.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.379+0000 7f8b47fff640 1 --2- 172.21.15.98:0/1995576947 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b4815fac0 0x7f8b4814d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:16.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.379+0000 7f8b47fff640 1 -- 172.21.15.98:0/1995576947 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b48073680 msgr2=0x7f8b4814fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:16.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.379+0000 7f8b47fff640 1 --2- 172.21.15.98:0/1995576947 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b48073680 0x7f8b4814fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:16.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.379+0000 7f8b47fff640 1 -- 172.21.15.98:0/1995576947 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b4814ee50 con 0x7f8b4815f6e0 2024-09-19T08:38:16.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.379+0000 7f8b47fff640 1 --2- 172.21.15.98:0/1995576947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b4815f6e0 0x7f8b4814c850 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000c7f0 tx=0x7f8b3000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:16.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.379+0000 7f8b457fa640 1 -- 172.21.15.98:0/1995576947 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b30015070 con 0x7f8b4815f6e0 2024-09-19T08:38:16.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.379+0000 7f8b457fa640 1 -- 172.21.15.98:0/1995576947 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b30004240 con 0x7f8b4815f6e0 2024-09-19T08:38:16.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.379+0000 7f8b4f040640 1 -- 172.21.15.98:0/1995576947 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b4814f130 con 0x7f8b4815f6e0 2024-09-19T08:38:16.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.379+0000 7f8b457fa640 1 -- 172.21.15.98:0/1995576947 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b30003620 con 0x7f8b4815f6e0 2024-09-19T08:38:16.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.379+0000 7f8b4f040640 1 -- 172.21.15.98:0/1995576947 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b4814f6f0 con 0x7f8b4815f6e0 2024-09-19T08:38:16.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.379+0000 7f8b4f040640 1 -- 172.21.15.98:0/1995576947 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b480cf530 con 0x7f8b4815f6e0 2024-09-19T08:38:16.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.379+0000 7f8b457fa640 1 -- 172.21.15.98:0/1995576947 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8b300039d0 con 0x7f8b4815f6e0 2024-09-19T08:38:16.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.379+0000 7f8b457fa640 1 --2- 172.21.15.98:0/1995576947 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8b280678c0 0x7f8b28069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:16.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.379+0000 7f8b457fa640 1 -- 172.21.15.98:0/1995576947 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f8b30085a10 con 0x7f8b4815f6e0 2024-09-19T08:38:16.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.379+0000 7f8b4cddc640 1 --2- 172.21.15.98:0/1995576947 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8b280678c0 0x7f8b28069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:16.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.379+0000 7f8b4cddc640 1 --2- 172.21.15.98:0/1995576947 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8b280678c0 0x7f8b28069d80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8b380ae000 tx=0x7f8b380a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:16.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.383+0000 7f8b457fa640 1 -- 172.21.15.98:0/1995576947 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f8b300568a0 con 0x7f8b4815f6e0 2024-09-19T08:38:16.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:16.547+0000 7f8b4f040640 1 -- 172.21.15.98:0/1995576947 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f8b48073f00 con 0x7f8b4815f6e0 2024-09-19T08:38:16.894 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:38:16.897 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:38:17.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.331+0000 7f8b457fa640 1 -- 172.21.15.98:0/1995576947 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7f8b300561a0 con 0x7f8b4815f6e0 2024-09-19T08:38:17.334 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_metadata' created 2024-09-19T08:38:17.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.339+0000 7f8b4f040640 1 -- 172.21.15.98:0/1995576947 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8b280678c0 msgr2=0x7f8b28069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:17.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.339+0000 7f8b4f040640 1 --2- 172.21.15.98:0/1995576947 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8b280678c0 0x7f8b28069d80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8b380ae000 tx=0x7f8b380a7070 comp rx=0 tx=0).stop 2024-09-19T08:38:17.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.339+0000 7f8b4f040640 1 -- 172.21.15.98:0/1995576947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b4815f6e0 msgr2=0x7f8b4814c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:17.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.339+0000 7f8b4f040640 1 --2- 172.21.15.98:0/1995576947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b4815f6e0 0x7f8b4814c850 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000c7f0 tx=0x7f8b3000ccb0 comp rx=0 tx=0).stop 2024-09-19T08:38:17.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.339+0000 7f8b4f040640 1 -- 172.21.15.98:0/1995576947 shutdown_connections 2024-09-19T08:38:17.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.339+0000 7f8b4f040640 1 --2- 172.21.15.98:0/1995576947 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8b280678c0 0x7f8b28069d80 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:17.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.339+0000 7f8b4f040640 1 --2- 172.21.15.98:0/1995576947 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b4815fac0 0x7f8b4814d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:17.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.339+0000 7f8b4f040640 1 --2- 172.21.15.98:0/1995576947 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b4815f6e0 0x7f8b4814c850 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:17.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.339+0000 7f8b4f040640 1 --2- 172.21.15.98:0/1995576947 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b48073680 0x7f8b4814fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:17.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.339+0000 7f8b4f040640 1 -- 172.21.15.98:0/1995576947 >> 172.21.15.98:0/1995576947 conn(0x7f8b4806c090 msgr2=0x7f8b480725c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:17.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.339+0000 7f8b4f040640 1 -- 172.21.15.98:0/1995576947 shutdown_connections 2024-09-19T08:38:17.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.339+0000 7f8b4f040640 1 -- 172.21.15.98:0/1995576947 wait complete. 2024-09-19T08:38:17.366 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T08:38:17.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.495+0000 7f85d53ea640 1 Processor -- start 2024-09-19T08:38:17.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.495+0000 7f85d53ea640 1 -- start start 2024-09-19T08:38:17.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.495+0000 7f85d53ea640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85d006c3a0 0x7f85d0165f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:17.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.495+0000 7f85d53ea640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85d0072ea0 con 0x7f85d0144d10 2024-09-19T08:38:17.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.495+0000 7f85d53ea640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85d0073180 con 0x7f85d00729f0 2024-09-19T08:38:17.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.495+0000 7f85d53ea640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f85d0073ab0 con 0x7f85d006c3a0 2024-09-19T08:38:17.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.495+0000 7f85ceffd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f85d0144d10 0x7f85d00722e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42724/0 (socket says 172.21.15.98:42724) 2024-09-19T08:38:17.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.495+0000 7f85ceffd640 1 -- 172.21.15.98:0/3076628724 learned_addr learned my addr 172.21.15.98:0/3076628724 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:17.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.495+0000 7f85ce7fc640 1 -- 172.21.15.98:0/3076628724 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 351146861 0 0) 0x7f85d0072ea0 con 0x7f85d0144d10 2024-09-19T08:38:17.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.495+0000 7f85ce7fc640 1 -- 172.21.15.98:0/3076628724 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f85bc003530 con 0x7f85d0144d10 2024-09-19T08:38:17.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.495+0000 7f85cf7fe640 1 --2- 172.21.15.98:0/3076628724 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85d006c3a0 0x7f85d0165f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:17.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.499+0000 7f85ce7fc640 1 -- 172.21.15.98:0/3076628724 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4244055913 0 0) 0x7f85bc003530 con 0x7f85d0144d10 2024-09-19T08:38:17.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.499+0000 7f85ce7fc640 1 -- 172.21.15.98:0/3076628724 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85d006c3a0 msgr2=0x7f85d0165f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:17.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.499+0000 7f85ce7fc640 1 --2- 172.21.15.98:0/3076628724 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85d006c3a0 0x7f85d0165f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:17.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.499+0000 7f85ce7fc640 1 -- 172.21.15.98:0/3076628724 >> v1:172.21.15.139:6789/0 conn(0x7f85d00729f0 legacy=0x7f85d006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:17.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.499+0000 7f85ce7fc640 1 -- 172.21.15.98:0/3076628724 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d0073460 con 0x7f85d0144d10 2024-09-19T08:38:17.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.499+0000 7f85ce7fc640 1 -- 172.21.15.98:0/3076628724 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f85c0001680 con 0x7f85d0144d10 2024-09-19T08:38:17.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.499+0000 7f85ce7fc640 1 -- 172.21.15.98:0/3076628724 >> v1:172.21.15.98:6789/0 conn(0x7f85d0144d10 legacy=0x7f85d00722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:17.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.499+0000 7f85ce7fc640 1 --2- 172.21.15.98:0/3076628724 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85d006c3a0 0x7f85bc0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:17.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.499+0000 7f85ce7fc640 1 --2- 172.21.15.98:0/3076628724 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85bc004020 0x7f85bc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:17.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.499+0000 7f85ce7fc640 1 --2- 172.21.15.98:0/3076628724 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f85bc004d50 0x7f85bc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:17.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.499+0000 7f85ce7fc640 1 -- 172.21.15.98:0/3076628724 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f85d0073ab0 con 0x7f85bc004d50 2024-09-19T08:38:17.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.499+0000 7f85ce7fc640 1 -- 172.21.15.98:0/3076628724 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f85bc0099f0 con 0x7f85bc004020 2024-09-19T08:38:17.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.499+0000 7f85ce7fc640 1 -- 172.21.15.98:0/3076628724 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f85bc009bf0 con 0x7f85d006c3a0 2024-09-19T08:38:17.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.499+0000 7f85cf7fe640 1 --2- 172.21.15.98:0/3076628724 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f85bc004d50 0x7f85bc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:17.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.499+0000 7f85c7fff640 1 --2- 172.21.15.98:0/3076628724 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85bc004020 0x7f85bc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:17.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.499+0000 7f85ceffd640 1 --2- 172.21.15.98:0/3076628724 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85d006c3a0 0x7f85bc0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:17.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.499+0000 7f85cf7fe640 1 -- 172.21.15.98:0/3076628724 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85d006c3a0 msgr2=0x7f85bc0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:17.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.499+0000 7f85cf7fe640 1 --2- 172.21.15.98:0/3076628724 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85d006c3a0 0x7f85bc0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:17.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.499+0000 7f85cf7fe640 1 -- 172.21.15.98:0/3076628724 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85bc004020 msgr2=0x7f85bc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:17.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.499+0000 7f85cf7fe640 1 --2- 172.21.15.98:0/3076628724 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85bc004020 0x7f85bc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:17.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.499+0000 7f85cf7fe640 1 -- 172.21.15.98:0/3076628724 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85bc003530 con 0x7f85bc004d50 2024-09-19T08:38:17.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.499+0000 7f85ceffd640 1 --2- 172.21.15.98:0/3076628724 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85d006c3a0 0x7f85bc0039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:17.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.499+0000 7f85cf7fe640 1 --2- 172.21.15.98:0/3076628724 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f85bc004d50 0x7f85bc0051d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f85d006f4d0 tx=0x7f85c801cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db7be019d5852b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:17.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.499+0000 7f85ce7fc640 1 -- 172.21.15.98:0/3076628724 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c80b2070 con 0x7f85bc004d50 2024-09-19T08:38:17.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.499+0000 7f85ce7fc640 1 -- 172.21.15.98:0/3076628724 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85c80a5ce0 con 0x7f85bc004d50 2024-09-19T08:38:17.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.499+0000 7f85ce7fc640 1 -- 172.21.15.98:0/3076628724 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c8009be0 con 0x7f85bc004d50 2024-09-19T08:38:17.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.499+0000 7f85d53ea640 1 -- 172.21.15.98:0/3076628724 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f85bc004d50 msgr2=0x7f85bc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:17.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.499+0000 7f85d53ea640 1 --2- 172.21.15.98:0/3076628724 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f85bc004d50 0x7f85bc0051d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f85d006f4d0 tx=0x7f85c801cde0 comp rx=0 tx=0).stop 2024-09-19T08:38:17.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.499+0000 7f85cf7fe640 1 -- 172.21.15.98:0/3076628724 reap_dead start 2024-09-19T08:38:17.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.499+0000 7f85d53ea640 1 -- 172.21.15.98:0/3076628724 shutdown_connections 2024-09-19T08:38:17.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.499+0000 7f85d53ea640 1 -- 172.21.15.98:0/3076628724 >> 172.21.15.98:0/3076628724 conn(0x7f85d00b2010 msgr2=0x7f85d00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:17.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.499+0000 7f85d53ea640 1 -- 172.21.15.98:0/3076628724 shutdown_connections 2024-09-19T08:38:17.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.499+0000 7f85d53ea640 1 -- 172.21.15.98:0/3076628724 wait complete. 2024-09-19T08:38:17.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.503+0000 7f85d53ea640 1 Processor -- start 2024-09-19T08:38:17.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.503+0000 7f85d53ea640 1 -- start start 2024-09-19T08:38:17.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.503+0000 7f85d53ea640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85d01ef250 0x7f85d015e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:17.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.503+0000 7f85d53ea640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85d0139f20 0x7f85d01ef630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:17.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.503+0000 7f85d53ea640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f85d013a380 0x7f85d00b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:17.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.503+0000 7f85d53ea640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f85d00af8b0 con 0x7f85d013a380 2024-09-19T08:38:17.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.503+0000 7f85d53ea640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f85d01450f0 con 0x7f85d0139f20 2024-09-19T08:38:17.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.503+0000 7f85d53ea640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f85d00afa30 con 0x7f85d01ef250 2024-09-19T08:38:17.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.503+0000 7f85cf7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f85d013a380 0x7f85d00b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:17.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.503+0000 7f85cf7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f85d013a380 0x7f85d00b11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41754/0 (socket says 172.21.15.98:41754) 2024-09-19T08:38:17.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.503+0000 7f85cf7fe640 1 -- 172.21.15.98:0/4162755593 learned_addr learned my addr 172.21.15.98:0/4162755593 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:17.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.503+0000 7f85c7fff640 1 --2- 172.21.15.98:0/4162755593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85d0139f20 0x7f85d01ef630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:17.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.503+0000 7f85ceffd640 1 --2- 172.21.15.98:0/4162755593 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85d01ef250 0x7f85d015e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:17.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.503+0000 7f85cf7fe640 1 -- 172.21.15.98:0/4162755593 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85d01ef250 msgr2=0x7f85d015e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:17.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.503+0000 7f85cf7fe640 1 --2- 172.21.15.98:0/4162755593 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85d01ef250 0x7f85d015e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:17.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.503+0000 7f85cf7fe640 1 -- 172.21.15.98:0/4162755593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85d0139f20 msgr2=0x7f85d01ef630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:17.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.503+0000 7f85cf7fe640 1 --2- 172.21.15.98:0/4162755593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85d0139f20 0x7f85d01ef630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:17.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.503+0000 7f85cf7fe640 1 -- 172.21.15.98:0/4162755593 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d01eff80 con 0x7f85d013a380 2024-09-19T08:38:17.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.503+0000 7f85cf7fe640 1 --2- 172.21.15.98:0/4162755593 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f85d013a380 0x7f85d00b11a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f85c8002240 tx=0x7f85bc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:17.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.503+0000 7f85c77fe640 1 -- 172.21.15.98:0/4162755593 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c80bb070 con 0x7f85d013a380 2024-09-19T08:38:17.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.503+0000 7f85c77fe640 1 -- 172.21.15.98:0/4162755593 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85c80a5ae0 con 0x7f85d013a380 2024-09-19T08:38:17.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.503+0000 7f85d53ea640 1 -- 172.21.15.98:0/4162755593 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85d01f0260 con 0x7f85d013a380 2024-09-19T08:38:17.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.503+0000 7f85c77fe640 1 -- 172.21.15.98:0/4162755593 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c80b2040 con 0x7f85d013a380 2024-09-19T08:38:17.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.503+0000 7f85d53ea640 1 -- 172.21.15.98:0/4162755593 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85d01f0820 con 0x7f85d013a380 2024-09-19T08:38:17.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.503+0000 7f85d53ea640 1 -- 172.21.15.98:0/4162755593 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85d0072e00 con 0x7f85d013a380 2024-09-19T08:38:17.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.507+0000 7f85c77fe640 1 -- 172.21.15.98:0/4162755593 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f85c80be070 con 0x7f85d013a380 2024-09-19T08:38:17.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.507+0000 7f85c77fe640 1 --2- 172.21.15.98:0/4162755593 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f85b00678c0 0x7f85b0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:17.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.507+0000 7f85c77fe640 1 -- 172.21.15.98:0/4162755593 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7f85c811f3e0 con 0x7f85d013a380 2024-09-19T08:38:17.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.507+0000 7f85ceffd640 1 --2- 172.21.15.98:0/4162755593 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f85b00678c0 0x7f85b0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:17.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.507+0000 7f85ceffd640 1 --2- 172.21.15.98:0/4162755593 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f85b00678c0 0x7f85b0069d80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f85c00017f0 tx=0x7f85c0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:17.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.511+0000 7f85c77fe640 1 -- 172.21.15.98:0/4162755593 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f85c8016af0 con 0x7f85d013a380 2024-09-19T08:38:17.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:17.659+0000 7f85d53ea640 1 -- 172.21.15.98:0/4162755593 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f85d0143ad0 con 0x7f85d013a380 2024-09-19T08:38:18.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.335+0000 7f85c77fe640 1 -- 172.21.15.98:0/4162755593 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) ==== 170+0+0 (secure 0 0 0) 0x7f85c80efab0 con 0x7f85d013a380 2024-09-19T08:38:18.338 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' created 2024-09-19T08:38:18.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.339+0000 7f85d53ea640 1 -- 172.21.15.98:0/4162755593 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f85b00678c0 msgr2=0x7f85b0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:18.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.339+0000 7f85d53ea640 1 --2- 172.21.15.98:0/4162755593 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f85b00678c0 0x7f85b0069d80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f85c00017f0 tx=0x7f85c0009040 comp rx=0 tx=0).stop 2024-09-19T08:38:18.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.339+0000 7f85d53ea640 1 -- 172.21.15.98:0/4162755593 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f85d013a380 msgr2=0x7f85d00b11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:18.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.339+0000 7f85d53ea640 1 --2- 172.21.15.98:0/4162755593 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f85d013a380 0x7f85d00b11a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f85c8002240 tx=0x7f85bc005840 comp rx=0 tx=0).stop 2024-09-19T08:38:18.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.343+0000 7f85d53ea640 1 -- 172.21.15.98:0/4162755593 shutdown_connections 2024-09-19T08:38:18.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.343+0000 7f85d53ea640 1 --2- 172.21.15.98:0/4162755593 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f85b00678c0 0x7f85b0069d80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:18.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.343+0000 7f85d53ea640 1 --2- 172.21.15.98:0/4162755593 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f85d013a380 0x7f85d00b11a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:18.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.343+0000 7f85d53ea640 1 --2- 172.21.15.98:0/4162755593 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85d0139f20 0x7f85d01ef630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:18.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.343+0000 7f85d53ea640 1 --2- 172.21.15.98:0/4162755593 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85d01ef250 0x7f85d015e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:18.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.343+0000 7f85d53ea640 1 -- 172.21.15.98:0/4162755593 >> 172.21.15.98:0/4162755593 conn(0x7f85d00b2010 msgr2=0x7f85d0142e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:18.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.347+0000 7f85d53ea640 1 -- 172.21.15.98:0/4162755593 shutdown_connections 2024-09-19T08:38:18.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.347+0000 7f85d53ea640 1 -- 172.21.15.98:0/4162755593 wait complete. 2024-09-19T08:38:18.379 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T08:38:18.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.503+0000 7f17c0948640 1 Processor -- start 2024-09-19T08:38:18.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.503+0000 7f17c0948640 1 -- start start 2024-09-19T08:38:18.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.503+0000 7f17c0948640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f17bc144d10 0x7f17bc0722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:18.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.503+0000 7f17c0948640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f17bc072820 0x7f17bc06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:18.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.503+0000 7f17c0948640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f17bc06c1d0 0x7f17bc0b4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:18.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.503+0000 7f17c0948640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f17bc072c60 con 0x7f17bc072820 2024-09-19T08:38:18.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.503+0000 7f17c0948640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f17bc073ab0 con 0x7f17bc06c1d0 2024-09-19T08:38:18.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.503+0000 7f17c0948640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f17bc1450f0 con 0x7f17bc144d10 2024-09-19T08:38:18.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.503+0000 7f17b9d9b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f17bc072820 0x7f17bc06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:18.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.503+0000 7f17b9d9b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f17bc072820 0x7f17bc06bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33234/0 (socket says 172.21.15.98:33234) 2024-09-19T08:38:18.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.503+0000 7f17b9d9b640 1 -- 172.21.15.98:0/1644045158 learned_addr learned my addr 172.21.15.98:0/1644045158 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:18.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.503+0000 7f17bad9d640 1 --2- 172.21.15.98:0/1644045158 >> v2:172.21.15.139:3300/0 conn(0x7f17bc06c1d0 0x7f17bc0b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:18.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.503+0000 7f17ba59c640 1 --2- 172.21.15.98:0/1644045158 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f17bc144d10 0x7f17bc0722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:18.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.503+0000 7f17b9d9b640 1 -- 172.21.15.98:0/1644045158 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f17bc144d10 msgr2=0x7f17bc0722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:18.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.503+0000 7f17b9d9b640 1 --2- 172.21.15.98:0/1644045158 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f17bc144d10 0x7f17bc0722e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:18.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.503+0000 7f17b9d9b640 1 -- 172.21.15.98:0/1644045158 >> v2:172.21.15.139:3300/0 conn(0x7f17bc06c1d0 msgr2=0x7f17bc0b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:18.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.503+0000 7f17b9d9b640 1 --2- 172.21.15.98:0/1644045158 >> v2:172.21.15.139:3300/0 conn(0x7f17bc06c1d0 0x7f17bc0b4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:18.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.503+0000 7f17b9d9b640 1 -- 172.21.15.98:0/1644045158 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17bc072ea0 con 0x7f17bc072820 2024-09-19T08:38:18.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.503+0000 7f17bad9d640 1 --2- 172.21.15.98:0/1644045158 >> v2:172.21.15.139:3300/0 conn(0x7f17bc06c1d0 0x7f17bc0b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:38:18.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.507+0000 7f17b9d9b640 1 --2- 172.21.15.98:0/1644045158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17bc072820 0x7f17bc06bc90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f17bc0b33a0 tx=0x7f17b001e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b49bf8128509f34c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:18.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.507+0000 7f17b959a640 1 -- 172.21.15.98:0/1644045158 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b00b4070 con 0x7f17bc072820 2024-09-19T08:38:18.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.507+0000 7f17b959a640 1 -- 172.21.15.98:0/1644045158 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17b00a78b0 con 0x7f17bc072820 2024-09-19T08:38:18.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.507+0000 7f17b959a640 1 -- 172.21.15.98:0/1644045158 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b00a7c80 con 0x7f17bc072820 2024-09-19T08:38:18.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.507+0000 7f17c0948640 1 -- 172.21.15.98:0/1644045158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17bc072820 msgr2=0x7f17bc06bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:18.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.507+0000 7f17c0948640 1 --2- 172.21.15.98:0/1644045158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17bc072820 0x7f17bc06bc90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f17bc0b33a0 tx=0x7f17b001e930 comp rx=0 tx=0).stop 2024-09-19T08:38:18.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.507+0000 7f17c0948640 1 -- 172.21.15.98:0/1644045158 shutdown_connections 2024-09-19T08:38:18.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.507+0000 7f17c0948640 1 --2- 172.21.15.98:0/1644045158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17bc072820 0x7f17bc06bc90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:18.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.507+0000 7f17c0948640 1 --2- 172.21.15.98:0/1644045158 >> v2:172.21.15.139:3300/0 conn(0x7f17bc06c1d0 0x7f17bc0b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:18.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.507+0000 7f17c0948640 1 --2- 172.21.15.98:0/1644045158 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f17bc144d10 0x7f17bc0722e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:18.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.507+0000 7f17c0948640 1 -- 172.21.15.98:0/1644045158 >> 172.21.15.98:0/1644045158 conn(0x7f17bc0b2010 msgr2=0x7f17bc0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:18.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.507+0000 7f17c0948640 1 -- 172.21.15.98:0/1644045158 shutdown_connections 2024-09-19T08:38:18.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.507+0000 7f17c0948640 1 -- 172.21.15.98:0/1644045158 wait complete. 2024-09-19T08:38:18.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.507+0000 7f17c0948640 1 Processor -- start 2024-09-19T08:38:18.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.507+0000 7f17c0948640 1 -- start start 2024-09-19T08:38:18.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.507+0000 7f17c0948640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17bc06c1d0 0x7f17bc134f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:18.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.507+0000 7f17c0948640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f17bc072820 0x7f17bc1368c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:18.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.507+0000 7f17c0948640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f17bc144d10 0x7f17bc15ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:18.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.507+0000 7f17c0948640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc0afa30 con 0x7f17bc06c1d0 2024-09-19T08:38:18.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.507+0000 7f17c0948640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc0af8b0 con 0x7f17bc072820 2024-09-19T08:38:18.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.507+0000 7f17c0948640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f17bc0b3410 con 0x7f17bc144d10 2024-09-19T08:38:18.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.507+0000 7f17ba59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17bc06c1d0 0x7f17bc134f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:18.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.507+0000 7f17ba59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17bc06c1d0 0x7f17bc134f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33250/0 (socket says 172.21.15.98:33250) 2024-09-19T08:38:18.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.507+0000 7f17ba59c640 1 -- 172.21.15.98:0/3774677298 learned_addr learned my addr 172.21.15.98:0/3774677298 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:18.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.507+0000 7f17b9d9b640 1 --2- 172.21.15.98:0/3774677298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f17bc072820 0x7f17bc1368c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:18.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.507+0000 7f17bad9d640 1 --2- 172.21.15.98:0/3774677298 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f17bc144d10 0x7f17bc15ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:18.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.507+0000 7f17ba59c640 1 -- 172.21.15.98:0/3774677298 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f17bc144d10 msgr2=0x7f17bc15ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:18.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.507+0000 7f17ba59c640 1 --2- 172.21.15.98:0/3774677298 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f17bc144d10 0x7f17bc15ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:18.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.507+0000 7f17ba59c640 1 -- 172.21.15.98:0/3774677298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f17bc072820 msgr2=0x7f17bc1368c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:18.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.507+0000 7f17ba59c640 1 --2- 172.21.15.98:0/3774677298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f17bc072820 0x7f17bc1368c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:18.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.507+0000 7f17ba59c640 1 -- 172.21.15.98:0/3774677298 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17bc1370c0 con 0x7f17bc06c1d0 2024-09-19T08:38:18.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.507+0000 7f17b9d9b640 1 --2- 172.21.15.98:0/3774677298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f17bc072820 0x7f17bc1368c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:18.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.507+0000 7f17ba59c640 1 --2- 172.21.15.98:0/3774677298 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17bc06c1d0 0x7f17bc134f10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f17ac00d620 tx=0x7f17ac00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:18.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.511+0000 7f17aaffd640 1 -- 172.21.15.98:0/3774677298 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac00b620 con 0x7f17bc06c1d0 2024-09-19T08:38:18.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.511+0000 7f17aaffd640 1 -- 172.21.15.98:0/3774677298 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17ac019070 con 0x7f17bc06c1d0 2024-09-19T08:38:18.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.511+0000 7f17c0948640 1 -- 172.21.15.98:0/3774677298 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17bc135450 con 0x7f17bc06c1d0 2024-09-19T08:38:18.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.511+0000 7f17aaffd640 1 -- 172.21.15.98:0/3774677298 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac00ba20 con 0x7f17bc06c1d0 2024-09-19T08:38:18.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.511+0000 7f17c0948640 1 -- 172.21.15.98:0/3774677298 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17bc134150 con 0x7f17bc06c1d0 2024-09-19T08:38:18.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.511+0000 7f17c0948640 1 -- 172.21.15.98:0/3774677298 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17bc06bc90 con 0x7f17bc06c1d0 2024-09-19T08:38:18.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.511+0000 7f17aaffd640 1 -- 172.21.15.98:0/3774677298 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f17ac00bbc0 con 0x7f17bc06c1d0 2024-09-19T08:38:18.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.511+0000 7f17aaffd640 1 --2- 172.21.15.98:0/3774677298 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f179c0678c0 0x7f179c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:18.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.511+0000 7f17aaffd640 1 -- 172.21.15.98:0/3774677298 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7f17ac086080 con 0x7f17bc06c1d0 2024-09-19T08:38:18.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.511+0000 7f17b9d9b640 1 --2- 172.21.15.98:0/3774677298 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f179c0678c0 0x7f179c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:18.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.511+0000 7f17b9d9b640 1 --2- 172.21.15.98:0/3774677298 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f179c0678c0 0x7f179c069d80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f17b00b1000 tx=0x7f17b00a9070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:18.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.515+0000 7f17aaffd640 1 -- 172.21.15.98:0/3774677298 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f17ac056c20 con 0x7f17bc06c1d0 2024-09-19T08:38:18.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:18.715+0000 7f17c0948640 1 -- 172.21.15.98:0/3774677298 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f17bc0757a0 con 0x7f17bc06c1d0 2024-09-19T08:38:19.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.399+0000 7f17aaffd640 1 -- 172.21.15.98:0/3774677298 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f17ac056520 con 0x7f17bc06c1d0 2024-09-19T08:38:19.402 INFO:teuthology.orchestra.run.smithi098.stderr:new fs with metadata pool 2 and data pool 3 2024-09-19T08:38:19.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.407+0000 7f17c0948640 1 -- 172.21.15.98:0/3774677298 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f179c0678c0 msgr2=0x7f179c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:19.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.407+0000 7f17c0948640 1 --2- 172.21.15.98:0/3774677298 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f179c0678c0 0x7f179c069d80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f17b00b1000 tx=0x7f17b00a9070 comp rx=0 tx=0).stop 2024-09-19T08:38:19.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.407+0000 7f17c0948640 1 -- 172.21.15.98:0/3774677298 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17bc06c1d0 msgr2=0x7f17bc134f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:19.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.407+0000 7f17c0948640 1 --2- 172.21.15.98:0/3774677298 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17bc06c1d0 0x7f17bc134f10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f17ac00d620 tx=0x7f17ac00dae0 comp rx=0 tx=0).stop 2024-09-19T08:38:19.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.407+0000 7f17c0948640 1 -- 172.21.15.98:0/3774677298 shutdown_connections 2024-09-19T08:38:19.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.407+0000 7f17c0948640 1 --2- 172.21.15.98:0/3774677298 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f179c0678c0 0x7f179c069d80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:19.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.407+0000 7f17c0948640 1 --2- 172.21.15.98:0/3774677298 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f17bc144d10 0x7f17bc15ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:19.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.407+0000 7f17c0948640 1 --2- 172.21.15.98:0/3774677298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f17bc072820 0x7f17bc1368c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:19.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.407+0000 7f17c0948640 1 --2- 172.21.15.98:0/3774677298 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17bc06c1d0 0x7f17bc134f10 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f17ac00d620 tx=0x7f17ac00dae0 comp rx=0 tx=0).stop 2024-09-19T08:38:19.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.407+0000 7f17c0948640 1 -- 172.21.15.98:0/3774677298 >> 172.21.15.98:0/3774677298 conn(0x7f17bc0b2010 msgr2=0x7f17bc0afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:19.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.407+0000 7f17c0948640 1 -- 172.21.15.98:0/3774677298 shutdown_connections 2024-09-19T08:38:19.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.407+0000 7f17c0948640 1 -- 172.21.15.98:0/3774677298 wait complete. 2024-09-19T08:38:19.440 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:38:19.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.555+0000 7fb931103640 1 Processor -- start 2024-09-19T08:38:19.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.555+0000 7fb931103640 1 -- start start 2024-09-19T08:38:19.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.559+0000 7fb931103640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb92c06c3a0 0x7fb92c0cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:19.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.559+0000 7fb931103640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb92c0cf900 0x7fb92c0723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:19.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.559+0000 7fb931103640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb92c06d310 con 0x7fb92c15c550 2024-09-19T08:38:19.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.559+0000 7fb931103640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fb92c06d5f0 con 0x7fb92c0cf900 2024-09-19T08:38:19.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.559+0000 7fb931103640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fb92c06d8d0 con 0x7fb92c06c3a0 2024-09-19T08:38:19.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.559+0000 7fb92ad9d640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fb92c15c550 0x7fb92c06bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41430/0 (socket says 172.21.15.98:41430) 2024-09-19T08:38:19.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.559+0000 7fb92ad9d640 1 -- 172.21.15.98:0/3078644978 learned_addr learned my addr 172.21.15.98:0/3078644978 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:19.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.559+0000 7fb92b59e640 1 --2- 172.21.15.98:0/3078644978 >> v2:172.21.15.98:3300/0 conn(0x7fb92c0cf900 0x7fb92c0723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:19.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.559+0000 7fb92a59c640 1 --2- 172.21.15.98:0/3078644978 >> v2:172.21.15.139:3300/0 conn(0x7fb92c06c3a0 0x7fb92c0cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:19.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.559+0000 7fb92b59e640 1 -- 172.21.15.98:0/3078644978 >> v2:172.21.15.139:3300/0 conn(0x7fb92c06c3a0 msgr2=0x7fb92c0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:19.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.559+0000 7fb92b59e640 1 --2- 172.21.15.98:0/3078644978 >> v2:172.21.15.139:3300/0 conn(0x7fb92c06c3a0 0x7fb92c0cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:19.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.559+0000 7fb92b59e640 1 -- 172.21.15.98:0/3078644978 >> v1:172.21.15.98:6789/0 conn(0x7fb92c15c550 legacy=0x7fb92c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:19.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.559+0000 7fb929d9b640 1 -- 172.21.15.98:0/3078644978 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1933663194 0 0) 0x7fb92c06d310 con 0x7fb92c15c550 2024-09-19T08:38:19.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.559+0000 7fb92b59e640 1 -- 172.21.15.98:0/3078644978 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92c06dbb0 con 0x7fb92c0cf900 2024-09-19T08:38:19.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.559+0000 7fb92b59e640 1 --2- 172.21.15.98:0/3078644978 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb92c0cf900 0x7fb92c0723a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb92c0ccc60 tx=0x7fb92401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=722396d03477c572 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:19.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.559+0000 7fb929d9b640 1 -- 172.21.15.98:0/3078644978 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9240b2070 con 0x7fb92c0cf900 2024-09-19T08:38:19.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.559+0000 7fb929d9b640 1 -- 172.21.15.98:0/3078644978 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9240a5900 con 0x7fb92c0cf900 2024-09-19T08:38:19.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.559+0000 7fb929d9b640 1 -- 172.21.15.98:0/3078644978 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9240a5cd0 con 0x7fb92c0cf900 2024-09-19T08:38:19.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.563+0000 7fb931103640 1 -- 172.21.15.98:0/3078644978 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb92c0cf900 msgr2=0x7fb92c0723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:19.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.563+0000 7fb931103640 1 --2- 172.21.15.98:0/3078644978 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb92c0cf900 0x7fb92c0723a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb92c0ccc60 tx=0x7fb92401c930 comp rx=0 tx=0).stop 2024-09-19T08:38:19.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.563+0000 7fb931103640 1 -- 172.21.15.98:0/3078644978 shutdown_connections 2024-09-19T08:38:19.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.563+0000 7fb931103640 1 --2- 172.21.15.98:0/3078644978 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb92c0cf900 0x7fb92c0723a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:19.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.563+0000 7fb931103640 1 --2- 172.21.15.98:0/3078644978 >> v2:172.21.15.139:3300/0 conn(0x7fb92c06c3a0 0x7fb92c0cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:19.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.563+0000 7fb931103640 1 -- 172.21.15.98:0/3078644978 >> 172.21.15.98:0/3078644978 conn(0x7fb92c0b2010 msgr2=0x7fb92c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:19.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.563+0000 7fb931103640 1 -- 172.21.15.98:0/3078644978 shutdown_connections 2024-09-19T08:38:19.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.563+0000 7fb931103640 1 -- 172.21.15.98:0/3078644978 wait complete. 2024-09-19T08:38:19.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.563+0000 7fb931103640 1 Processor -- start 2024-09-19T08:38:19.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.563+0000 7fb931103640 1 -- start start 2024-09-19T08:38:19.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.563+0000 7fb931103640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb92c06c3a0 0x7fb92c150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:19.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.563+0000 7fb931103640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb92c0cf900 0x7fb92c14b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:19.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.563+0000 7fb931103640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb92c15c550 0x7fb92c155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:19.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.563+0000 7fb931103640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c073480 con 0x7fb92c15c550 2024-09-19T08:38:19.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.563+0000 7fb931103640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c073300 con 0x7fb92c0cf900 2024-09-19T08:38:19.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.563+0000 7fb931103640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb92c073600 con 0x7fb92c06c3a0 2024-09-19T08:38:19.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.563+0000 7fb92b59e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb92c15c550 0x7fb92c155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:19.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.563+0000 7fb92b59e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb92c15c550 0x7fb92c155e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33264/0 (socket says 172.21.15.98:33264) 2024-09-19T08:38:19.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.563+0000 7fb92b59e640 1 -- 172.21.15.98:0/1955068968 learned_addr learned my addr 172.21.15.98:0/1955068968 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:19.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.563+0000 7fb92ad9d640 1 --2- 172.21.15.98:0/1955068968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb92c06c3a0 0x7fb92c150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:19.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.563+0000 7fb92a59c640 1 --2- 172.21.15.98:0/1955068968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb92c0cf900 0x7fb92c14b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:19.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.563+0000 7fb92b59e640 1 -- 172.21.15.98:0/1955068968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb92c06c3a0 msgr2=0x7fb92c150740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:19.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.563+0000 7fb92b59e640 1 --2- 172.21.15.98:0/1955068968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb92c06c3a0 0x7fb92c150740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:19.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.563+0000 7fb92b59e640 1 -- 172.21.15.98:0/1955068968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb92c0cf900 msgr2=0x7fb92c14b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:19.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.563+0000 7fb92b59e640 1 --2- 172.21.15.98:0/1955068968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb92c0cf900 0x7fb92c14b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:19.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.563+0000 7fb92b59e640 1 -- 172.21.15.98:0/1955068968 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92c14b980 con 0x7fb92c15c550 2024-09-19T08:38:19.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.563+0000 7fb92b59e640 1 --2- 172.21.15.98:0/1955068968 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb92c15c550 0x7fb92c155e90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb9240b0040 tx=0x7fb9240196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:19.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.563+0000 7fb9137fe640 1 -- 172.21.15.98:0/1955068968 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9240b2050 con 0x7fb92c15c550 2024-09-19T08:38:19.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.563+0000 7fb9137fe640 1 -- 172.21.15.98:0/1955068968 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9240b3040 con 0x7fb92c15c550 2024-09-19T08:38:19.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.563+0000 7fb931103640 1 -- 172.21.15.98:0/1955068968 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb92c157220 con 0x7fb92c15c550 2024-09-19T08:38:19.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.563+0000 7fb9137fe640 1 -- 172.21.15.98:0/1955068968 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9240ad850 con 0x7fb92c15c550 2024-09-19T08:38:19.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.563+0000 7fb931103640 1 -- 172.21.15.98:0/1955068968 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb92c14c0a0 con 0x7fb92c15c550 2024-09-19T08:38:19.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.563+0000 7fb931103640 1 -- 172.21.15.98:0/1955068968 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb92c151a80 con 0x7fb92c15c550 2024-09-19T08:38:19.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.567+0000 7fb9137fe640 1 -- 172.21.15.98:0/1955068968 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb9240c0020 con 0x7fb92c15c550 2024-09-19T08:38:19.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.567+0000 7fb9137fe640 1 --2- 172.21.15.98:0/1955068968 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb9180678c0 0x7fb918069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:19.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.567+0000 7fb9137fe640 1 -- 172.21.15.98:0/1955068968 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb9241206c0 con 0x7fb92c15c550 2024-09-19T08:38:19.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.567+0000 7fb92ad9d640 1 --2- 172.21.15.98:0/1955068968 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb9180678c0 0x7fb918069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:19.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.567+0000 7fb92ad9d640 1 --2- 172.21.15.98:0/1955068968 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb9180678c0 0x7fb918069d80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb91c000f70 tx=0x7fb91c007b60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:19.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.571+0000 7fb9137fe640 1 -- 172.21.15.98:0/1955068968 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb924016520 con 0x7fb92c15c550 2024-09-19T08:38:19.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.731+0000 7fb931103640 1 -- 172.21.15.98:0/1955068968 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb92c14bb70 con 0x7fb92c15c550 2024-09-19T08:38:19.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.735+0000 7fb9137fe640 1 -- 172.21.15.98:0/1955068968 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16678 (secure 0 0 0) 0x7fb9240f13b0 con 0x7fb92c15c550 2024-09-19T08:38:19.735 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:38:19.735 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":20,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:38:18.716482+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:38:16.588688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T08:38:17.700392+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:38:19.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.735+0000 7fb931103640 1 -- 172.21.15.98:0/1955068968 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb9180678c0 msgr2=0x7fb918069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:19.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.735+0000 7fb931103640 1 --2- 172.21.15.98:0/1955068968 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb9180678c0 0x7fb918069d80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb91c000f70 tx=0x7fb91c007b60 comp rx=0 tx=0).stop 2024-09-19T08:38:19.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.735+0000 7fb931103640 1 -- 172.21.15.98:0/1955068968 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb92c15c550 msgr2=0x7fb92c155e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:19.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.735+0000 7fb931103640 1 --2- 172.21.15.98:0/1955068968 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb92c15c550 0x7fb92c155e90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb9240b0040 tx=0x7fb9240196d0 comp rx=0 tx=0).stop 2024-09-19T08:38:19.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.735+0000 7fb931103640 1 -- 172.21.15.98:0/1955068968 shutdown_connections 2024-09-19T08:38:19.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.735+0000 7fb931103640 1 --2- 172.21.15.98:0/1955068968 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb9180678c0 0x7fb918069d80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:19.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.735+0000 7fb931103640 1 --2- 172.21.15.98:0/1955068968 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb92c15c550 0x7fb92c155e90 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:19.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.735+0000 7fb931103640 1 --2- 172.21.15.98:0/1955068968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb92c0cf900 0x7fb92c14b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:19.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.735+0000 7fb931103640 1 --2- 172.21.15.98:0/1955068968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb92c06c3a0 0x7fb92c150740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:19.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.735+0000 7fb931103640 1 -- 172.21.15.98:0/1955068968 >> 172.21.15.98:0/1955068968 conn(0x7fb92c0b2010 msgr2=0x7fb92c0d0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:19.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.735+0000 7fb931103640 1 -- 172.21.15.98:0/1955068968 shutdown_connections 2024-09-19T08:38:19.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.735+0000 7fb931103640 1 -- 172.21.15.98:0/1955068968 wait complete. 2024-09-19T08:38:19.754 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:38:19.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.871+0000 7fd932d14640 1 Processor -- start 2024-09-19T08:38:19.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.871+0000 7fd932d14640 1 -- start start 2024-09-19T08:38:19.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.875+0000 7fd932d14640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd92c15c550 0x7fd92c06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:19.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.875+0000 7fd932d14640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd92c06c1d0 0x7fd92c0cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:19.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.875+0000 7fd932d14640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd92c06d310 con 0x7fd92c0cf900 2024-09-19T08:38:19.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.875+0000 7fd932d14640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fd92c06c640 con 0x7fd92c06c1d0 2024-09-19T08:38:19.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.875+0000 7fd932d14640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fd92c06d5f0 con 0x7fd92c15c550 2024-09-19T08:38:19.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.875+0000 7fd92bfff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd92c06c1d0 0x7fd92c0cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:19.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.875+0000 7fd92bfff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd92c06c1d0 0x7fd92c0cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33278/0 (socket says 172.21.15.98:33278) 2024-09-19T08:38:19.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.875+0000 7fd92bfff640 1 -- 172.21.15.98:0/2188347128 learned_addr learned my addr 172.21.15.98:0/2188347128 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:19.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.875+0000 7fd930ab0640 1 --2- 172.21.15.98:0/2188347128 >> v2:172.21.15.139:3300/0 conn(0x7fd92c15c550 0x7fd92c06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:19.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.875+0000 7fd92bfff640 1 -- 172.21.15.98:0/2188347128 >> v2:172.21.15.139:3300/0 conn(0x7fd92c15c550 msgr2=0x7fd92c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:19.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.875+0000 7fd92bfff640 1 --2- 172.21.15.98:0/2188347128 >> v2:172.21.15.139:3300/0 conn(0x7fd92c15c550 0x7fd92c06bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:19.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.875+0000 7fd92bfff640 1 -- 172.21.15.98:0/2188347128 >> v1:172.21.15.139:6789/0 conn(0x7fd92c0cf900 legacy=0x7fd92c0ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:19.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.875+0000 7fd92bfff640 1 -- 172.21.15.98:0/2188347128 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd92c06d8d0 con 0x7fd92c06c1d0 2024-09-19T08:38:19.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.875+0000 7fd92bfff640 1 --2- 172.21.15.98:0/2188347128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd92c06c1d0 0x7fd92c0cf3c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd92c0b56b0 tx=0x7fd91c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c11aba9882ea4b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:19.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.875+0000 7fd92b7fe640 1 -- 172.21.15.98:0/2188347128 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91c0b0070 con 0x7fd92c06c1d0 2024-09-19T08:38:19.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.875+0000 7fd92b7fe640 1 -- 172.21.15.98:0/2188347128 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd91c0ab600 con 0x7fd92c06c1d0 2024-09-19T08:38:19.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.875+0000 7fd92b7fe640 1 -- 172.21.15.98:0/2188347128 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91c0ab9d0 con 0x7fd92c06c1d0 2024-09-19T08:38:19.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.879+0000 7fd932d14640 1 -- 172.21.15.98:0/2188347128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd92c06c1d0 msgr2=0x7fd92c0cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:19.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.879+0000 7fd932d14640 1 --2- 172.21.15.98:0/2188347128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd92c06c1d0 0x7fd92c0cf3c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd92c0b56b0 tx=0x7fd91c01c520 comp rx=0 tx=0).stop 2024-09-19T08:38:19.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.879+0000 7fd932d14640 1 -- 172.21.15.98:0/2188347128 shutdown_connections 2024-09-19T08:38:19.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.879+0000 7fd932d14640 1 --2- 172.21.15.98:0/2188347128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd92c06c1d0 0x7fd92c0cf3c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:19.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.879+0000 7fd932d14640 1 --2- 172.21.15.98:0/2188347128 >> v2:172.21.15.139:3300/0 conn(0x7fd92c15c550 0x7fd92c06bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:19.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.879+0000 7fd932d14640 1 -- 172.21.15.98:0/2188347128 >> 172.21.15.98:0/2188347128 conn(0x7fd92c0b2010 msgr2=0x7fd92c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:19.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.879+0000 7fd932d14640 1 -- 172.21.15.98:0/2188347128 shutdown_connections 2024-09-19T08:38:19.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.879+0000 7fd932d14640 1 -- 172.21.15.98:0/2188347128 wait complete. 2024-09-19T08:38:19.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.879+0000 7fd932d14640 1 Processor -- start 2024-09-19T08:38:19.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.879+0000 7fd932d14640 1 -- start start 2024-09-19T08:38:19.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.879+0000 7fd932d14640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd92c06c1d0 0x7fd92c14ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:19.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.879+0000 7fd932d14640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd92c0cf900 0x7fd92c14a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:19.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.879+0000 7fd932d14640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd92c15c550 0x7fd92c14ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:19.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.879+0000 7fd932d14640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd92c1693b0 con 0x7fd92c15c550 2024-09-19T08:38:19.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.879+0000 7fd932d14640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd92c169230 con 0x7fd92c0cf900 2024-09-19T08:38:19.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.879+0000 7fd932d14640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd92c169530 con 0x7fd92c06c1d0 2024-09-19T08:38:19.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.879+0000 7fd9312b1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd92c15c550 0x7fd92c14ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:19.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.879+0000 7fd9312b1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd92c15c550 0x7fd92c14ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33280/0 (socket says 172.21.15.98:33280) 2024-09-19T08:38:19.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.879+0000 7fd9312b1640 1 -- 172.21.15.98:0/415288308 learned_addr learned my addr 172.21.15.98:0/415288308 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:19.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.879+0000 7fd930ab0640 1 --2- 172.21.15.98:0/415288308 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd92c06c1d0 0x7fd92c14ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:19.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.879+0000 7fd92bfff640 1 --2- 172.21.15.98:0/415288308 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd92c0cf900 0x7fd92c14a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:19.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.879+0000 7fd9312b1640 1 -- 172.21.15.98:0/415288308 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd92c06c1d0 msgr2=0x7fd92c14ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:19.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.879+0000 7fd9312b1640 1 --2- 172.21.15.98:0/415288308 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd92c06c1d0 0x7fd92c14ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:19.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.879+0000 7fd9312b1640 1 -- 172.21.15.98:0/415288308 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd92c0cf900 msgr2=0x7fd92c14a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:19.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.879+0000 7fd9312b1640 1 --2- 172.21.15.98:0/415288308 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd92c0cf900 0x7fd92c14a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:19.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.879+0000 7fd9312b1640 1 -- 172.21.15.98:0/415288308 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd92c154840 con 0x7fd92c15c550 2024-09-19T08:38:19.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.879+0000 7fd92bfff640 1 --2- 172.21.15.98:0/415288308 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd92c0cf900 0x7fd92c14a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:38:19.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.879+0000 7fd9312b1640 1 --2- 172.21.15.98:0/415288308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd92c15c550 0x7fd92c14ae70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd92400c770 tx=0x7fd92400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:19.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.879+0000 7fd9297fa640 1 -- 172.21.15.98:0/415288308 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd924003b80 con 0x7fd92c15c550 2024-09-19T08:38:19.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.879+0000 7fd9297fa640 1 -- 172.21.15.98:0/415288308 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd924003d20 con 0x7fd92c15c550 2024-09-19T08:38:19.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.879+0000 7fd932d14640 1 -- 172.21.15.98:0/415288308 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd92c1504e0 con 0x7fd92c15c550 2024-09-19T08:38:19.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.879+0000 7fd9297fa640 1 -- 172.21.15.98:0/415288308 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92400aa90 con 0x7fd92c15c550 2024-09-19T08:38:19.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.879+0000 7fd932d14640 1 -- 172.21.15.98:0/415288308 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd92c152410 con 0x7fd92c15c550 2024-09-19T08:38:19.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.883+0000 7fd932d14640 1 -- 172.21.15.98:0/415288308 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd92c0cf3c0 con 0x7fd92c15c550 2024-09-19T08:38:19.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.883+0000 7fd9297fa640 1 -- 172.21.15.98:0/415288308 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd92400ac30 con 0x7fd92c15c550 2024-09-19T08:38:19.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.883+0000 7fd9297fa640 1 --2- 172.21.15.98:0/415288308 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd9180678c0 0x7fd918069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:19.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.883+0000 7fd9297fa640 1 -- 172.21.15.98:0/415288308 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd924085b80 con 0x7fd92c15c550 2024-09-19T08:38:19.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.883+0000 7fd930ab0640 1 --2- 172.21.15.98:0/415288308 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd9180678c0 0x7fd918069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:19.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.883+0000 7fd930ab0640 1 --2- 172.21.15.98:0/415288308 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd9180678c0 0x7fd918069d80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd920009af0 tx=0x7fd920008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:19.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:19.887+0000 7fd9297fa640 1 -- 172.21.15.98:0/415288308 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd924056750 con 0x7fd92c15c550 2024-09-19T08:38:20.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.055+0000 7fd932d14640 1 -- 172.21.15.98:0/415288308 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd92c1525e0 con 0x7fd92c15c550 2024-09-19T08:38:20.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.055+0000 7fd9297fa640 1 -- 172.21.15.98:0/415288308 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16678 (secure 0 0 0) 0x7fd924056050 con 0x7fd92c15c550 2024-09-19T08:38:20.058 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:38:20.058 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":20,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:38:18.716482+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:38:16.588688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T08:38:17.700392+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:38:20.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.059+0000 7fd932d14640 1 -- 172.21.15.98:0/415288308 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd9180678c0 msgr2=0x7fd918069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:20.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.059+0000 7fd932d14640 1 --2- 172.21.15.98:0/415288308 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd9180678c0 0x7fd918069d80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd920009af0 tx=0x7fd920008040 comp rx=0 tx=0).stop 2024-09-19T08:38:20.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.059+0000 7fd932d14640 1 -- 172.21.15.98:0/415288308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd92c15c550 msgr2=0x7fd92c14ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:20.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.059+0000 7fd932d14640 1 --2- 172.21.15.98:0/415288308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd92c15c550 0x7fd92c14ae70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd92400c770 tx=0x7fd92400cc30 comp rx=0 tx=0).stop 2024-09-19T08:38:20.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.059+0000 7fd932d14640 1 -- 172.21.15.98:0/415288308 shutdown_connections 2024-09-19T08:38:20.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.059+0000 7fd932d14640 1 --2- 172.21.15.98:0/415288308 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd9180678c0 0x7fd918069d80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:20.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.059+0000 7fd932d14640 1 --2- 172.21.15.98:0/415288308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd92c15c550 0x7fd92c14ae70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:20.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.059+0000 7fd932d14640 1 --2- 172.21.15.98:0/415288308 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd92c0cf900 0x7fd92c14a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:20.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.059+0000 7fd932d14640 1 --2- 172.21.15.98:0/415288308 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd92c06c1d0 0x7fd92c14ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:20.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.059+0000 7fd932d14640 1 -- 172.21.15.98:0/415288308 >> 172.21.15.98:0/415288308 conn(0x7fd92c0b2010 msgr2=0x7fd92c0d0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:20.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.059+0000 7fd932d14640 1 -- 172.21.15.98:0/415288308 shutdown_connections 2024-09-19T08:38:20.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.059+0000 7fd932d14640 1 -- 172.21.15.98:0/415288308 wait complete. 2024-09-19T08:38:20.077 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-19T08:38:20.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.195+0000 7fe0e3f7b640 1 Processor -- start 2024-09-19T08:38:20.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.195+0000 7fe0e3f7b640 1 -- start start 2024-09-19T08:38:20.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.195+0000 7fe0e3f7b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0dc15c320 0x7fe0dc0d0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:20.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.195+0000 7fe0e3f7b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe0dc0b5c60 0x7fe0dc073e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:20.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.195+0000 7fe0e3f7b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0dc15c700 con 0x7fe0dc073a10 2024-09-19T08:38:20.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.195+0000 7fe0e3f7b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe0dc0711c0 con 0x7fe0dc0b5c60 2024-09-19T08:38:20.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.195+0000 7fe0e3f7b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe0dc072fc0 con 0x7fe0dc15c320 2024-09-19T08:38:20.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.195+0000 7fe0e2518640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe0dc0b5c60 0x7fe0dc073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:20.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.195+0000 7fe0e1d17640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0dc15c320 0x7fe0dc0d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:20.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.195+0000 7fe0e1d17640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0dc15c320 0x7fe0dc0d0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:55720/0 (socket says 172.21.15.98:55720) 2024-09-19T08:38:20.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.195+0000 7fe0e2518640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe0dc0b5c60 0x7fe0dc073e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:35818/0 (socket says 172.21.15.98:35818) 2024-09-19T08:38:20.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.195+0000 7fe0e1516640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fe0dc073a10 0x7fe0dc0b5550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:50628/0 (socket says 172.21.15.98:50628) 2024-09-19T08:38:20.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.195+0000 7fe0e1d17640 1 -- 172.21.15.98:0/3213631687 learned_addr learned my addr 172.21.15.98:0/3213631687 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:20.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.195+0000 7fe0e1d17640 1 -- 172.21.15.98:0/3213631687 >> v2:172.21.15.139:3300/0 conn(0x7fe0dc0b5c60 msgr2=0x7fe0dc073e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:20.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.195+0000 7fe0e1d17640 1 --2- 172.21.15.98:0/3213631687 >> v2:172.21.15.139:3300/0 conn(0x7fe0dc0b5c60 0x7fe0dc073e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:20.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.195+0000 7fe0e1d17640 1 -- 172.21.15.98:0/3213631687 >> v1:172.21.15.139:6789/0 conn(0x7fe0dc073a10 legacy=0x7fe0dc0b5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:20.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.195+0000 7fe0e1d17640 1 -- 172.21.15.98:0/3213631687 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0dc071410 con 0x7fe0dc15c320 2024-09-19T08:38:20.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.199+0000 7fe0e1d17640 1 --2- 172.21.15.98:0/3213631687 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0dc15c320 0x7fe0dc0d0890 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80098a0 tx=0x7fe0d801ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3708a8ca56e125c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:20.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.199+0000 7fe0e0d15640 1 -- 172.21.15.98:0/3213631687 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d80b5070 con 0x7fe0dc15c320 2024-09-19T08:38:20.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.199+0000 7fe0e0d15640 1 -- 172.21.15.98:0/3213631687 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0d80a8a30 con 0x7fe0dc15c320 2024-09-19T08:38:20.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.199+0000 7fe0e0d15640 1 -- 172.21.15.98:0/3213631687 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d80a8e00 con 0x7fe0dc15c320 2024-09-19T08:38:20.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.199+0000 7fe0e3f7b640 1 -- 172.21.15.98:0/3213631687 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0dc15c320 msgr2=0x7fe0dc0d0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:20.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.199+0000 7fe0e3f7b640 1 --2- 172.21.15.98:0/3213631687 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0dc15c320 0x7fe0dc0d0890 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80098a0 tx=0x7fe0d801ed30 comp rx=0 tx=0).stop 2024-09-19T08:38:20.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.199+0000 7fe0e3f7b640 1 -- 172.21.15.98:0/3213631687 shutdown_connections 2024-09-19T08:38:20.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.199+0000 7fe0e3f7b640 1 --2- 172.21.15.98:0/3213631687 >> v2:172.21.15.139:3300/0 conn(0x7fe0dc0b5c60 0x7fe0dc073e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:20.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.199+0000 7fe0e3f7b640 1 --2- 172.21.15.98:0/3213631687 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0dc15c320 0x7fe0dc0d0890 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:20.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.199+0000 7fe0e3f7b640 1 -- 172.21.15.98:0/3213631687 >> 172.21.15.98:0/3213631687 conn(0x7fe0dc0cedf0 msgr2=0x7fe0dc0cf200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:20.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.199+0000 7fe0e3f7b640 1 -- 172.21.15.98:0/3213631687 shutdown_connections 2024-09-19T08:38:20.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.199+0000 7fe0e3f7b640 1 -- 172.21.15.98:0/3213631687 wait complete. 2024-09-19T08:38:20.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.199+0000 7fe0e3f7b640 1 Processor -- start 2024-09-19T08:38:20.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.199+0000 7fe0e3f7b640 1 -- start start 2024-09-19T08:38:20.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.203+0000 7fe0e3f7b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0dc073a10 0x7fe0dc157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:20.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.203+0000 7fe0e3f7b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0dc0b5c60 0x7fe0dc158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:20.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.203+0000 7fe0e3f7b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0dc15c320 0x7fe0dc1e7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:20.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.203+0000 7fe0e3f7b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc0734f0 con 0x7fe0dc0b5c60 2024-09-19T08:38:20.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.203+0000 7fe0e3f7b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe0dc073210 con 0x7fe0dc15c320 2024-09-19T08:38:20.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.203+0000 7fe0e3f7b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe0dc0d0dd0 con 0x7fe0dc073a10 2024-09-19T08:38:20.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.203+0000 7fe0e1516640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0dc0b5c60 0x7fe0dc158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:20.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.203+0000 7fe0e1516640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0dc0b5c60 0x7fe0dc158c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33290/0 (socket says 172.21.15.98:33290) 2024-09-19T08:38:20.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.203+0000 7fe0e1516640 1 -- 172.21.15.98:0/3469316771 learned_addr learned my addr 172.21.15.98:0/3469316771 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:20.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.203+0000 7fe0e2518640 1 --2- 172.21.15.98:0/3469316771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0dc15c320 0x7fe0dc1e7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:20.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.203+0000 7fe0e1d17640 1 --2- 172.21.15.98:0/3469316771 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0dc073a10 0x7fe0dc157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:20.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.203+0000 7fe0e1516640 1 -- 172.21.15.98:0/3469316771 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0dc073a10 msgr2=0x7fe0dc157690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:20.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.203+0000 7fe0e1516640 1 --2- 172.21.15.98:0/3469316771 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0dc073a10 0x7fe0dc157690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:20.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.203+0000 7fe0e1516640 1 -- 172.21.15.98:0/3469316771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0dc15c320 msgr2=0x7fe0dc1e7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:20.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.203+0000 7fe0e1516640 1 --2- 172.21.15.98:0/3469316771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0dc15c320 0x7fe0dc1e7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:20.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.203+0000 7fe0e1516640 1 -- 172.21.15.98:0/3469316771 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0dc159630 con 0x7fe0dc0b5c60 2024-09-19T08:38:20.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.203+0000 7fe0e1516640 1 --2- 172.21.15.98:0/3469316771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0dc0b5c60 0x7fe0dc158c50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc00c500 tx=0x7fe0cc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:20.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.203+0000 7fe0d27fc640 1 -- 172.21.15.98:0/3469316771 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc018070 con 0x7fe0dc0b5c60 2024-09-19T08:38:20.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.203+0000 7fe0d27fc640 1 -- 172.21.15.98:0/3469316771 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0cc007500 con 0x7fe0dc0b5c60 2024-09-19T08:38:20.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.203+0000 7fe0e3f7b640 1 -- 172.21.15.98:0/3469316771 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0dc159910 con 0x7fe0dc0b5c60 2024-09-19T08:38:20.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.203+0000 7fe0d27fc640 1 -- 172.21.15.98:0/3469316771 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc004a10 con 0x7fe0dc0b5c60 2024-09-19T08:38:20.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.203+0000 7fe0e3f7b640 1 -- 172.21.15.98:0/3469316771 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0dc159ed0 con 0x7fe0dc0b5c60 2024-09-19T08:38:20.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.203+0000 7fe0e3f7b640 1 -- 172.21.15.98:0/3469316771 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0dc0700d0 con 0x7fe0dc0b5c60 2024-09-19T08:38:20.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.203+0000 7fe0d27fc640 1 -- 172.21.15.98:0/3469316771 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe0cc01b070 con 0x7fe0dc0b5c60 2024-09-19T08:38:20.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.207+0000 7fe0d27fc640 1 --2- 172.21.15.98:0/3469316771 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe0c00678c0 0x7fe0c0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:20.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.207+0000 7fe0d27fc640 1 -- 172.21.15.98:0/3469316771 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe0cc085250 con 0x7fe0dc0b5c60 2024-09-19T08:38:20.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.207+0000 7fe0e1d17640 1 --2- 172.21.15.98:0/3469316771 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe0c00678c0 0x7fe0c0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:20.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.207+0000 7fe0e1d17640 1 --2- 172.21.15.98:0/3469316771 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe0c00678c0 0x7fe0c0069d80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80b2000 tx=0x7fe0d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:20.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.207+0000 7fe0d27fc640 1 -- 172.21.15.98:0/3469316771 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe0cc055f00 con 0x7fe0dc0b5c60 2024-09-19T08:38:20.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:20.407+0000 7fe0e3f7b640 1 -- 172.21.15.98:0/3469316771 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe0dc0cfb30 con 0x7fe0dc0b5c60 2024-09-19T08:38:21.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.403+0000 7fe0d27fc640 1 -- 172.21.15.98:0/3469316771 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) ==== 118+0+0 (secure 0 0 0) 0x7fe0cc055800 con 0x7fe0dc0b5c60 2024-09-19T08:38:21.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.407+0000 7fe0e3f7b640 1 -- 172.21.15.98:0/3469316771 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe0c00678c0 msgr2=0x7fe0c0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:21.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.407+0000 7fe0e3f7b640 1 --2- 172.21.15.98:0/3469316771 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe0c00678c0 0x7fe0c0069d80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80b2000 tx=0x7fe0d8008040 comp rx=0 tx=0).stop 2024-09-19T08:38:21.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.407+0000 7fe0e3f7b640 1 -- 172.21.15.98:0/3469316771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0dc0b5c60 msgr2=0x7fe0dc158c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:21.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.407+0000 7fe0e3f7b640 1 --2- 172.21.15.98:0/3469316771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0dc0b5c60 0x7fe0dc158c50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc00c500 tx=0x7fe0cc00c9c0 comp rx=0 tx=0).stop 2024-09-19T08:38:21.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.407+0000 7fe0e3f7b640 1 -- 172.21.15.98:0/3469316771 shutdown_connections 2024-09-19T08:38:21.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.407+0000 7fe0e3f7b640 1 --2- 172.21.15.98:0/3469316771 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe0c00678c0 0x7fe0c0069d80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:21.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.407+0000 7fe0e3f7b640 1 --2- 172.21.15.98:0/3469316771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0dc15c320 0x7fe0dc1e7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:21.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.407+0000 7fe0e3f7b640 1 --2- 172.21.15.98:0/3469316771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0dc0b5c60 0x7fe0dc158c50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:21.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.407+0000 7fe0e3f7b640 1 --2- 172.21.15.98:0/3469316771 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0dc073a10 0x7fe0dc157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:21.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.407+0000 7fe0e3f7b640 1 -- 172.21.15.98:0/3469316771 >> 172.21.15.98:0/3469316771 conn(0x7fe0dc0cedf0 msgr2=0x7fe0dc0709d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:21.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.407+0000 7fe0e3f7b640 1 -- 172.21.15.98:0/3469316771 shutdown_connections 2024-09-19T08:38:21.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.407+0000 7fe0e3f7b640 1 -- 172.21.15.98:0/3469316771 wait complete. 2024-09-19T08:38:21.427 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-19T08:38:21.427 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-19T08:38:21.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.555+0000 7fa189db5640 1 Processor -- start 2024-09-19T08:38:21.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.555+0000 7fa189db5640 1 -- start start 2024-09-19T08:38:21.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.555+0000 7fa189db5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa1841415a0 0x7fa1840d20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:21.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.555+0000 7fa189db5640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa18415eae0 con 0x7fa1840d3a70 2024-09-19T08:38:21.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.555+0000 7fa189db5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa18415edc0 con 0x7fa1840d25f0 2024-09-19T08:38:21.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.555+0000 7fa189db5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fa1840d2a30 con 0x7fa1841415a0 2024-09-19T08:38:21.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.555+0000 7fa183fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fa1840d3a70 0x7fa1840b8790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41434/0 (socket says 172.21.15.98:41434) 2024-09-19T08:38:21.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.555+0000 7fa183fff640 1 -- 172.21.15.98:0/1540782346 learned_addr learned my addr 172.21.15.98:0/1540782346 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:21.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.555+0000 7fa1837fe640 1 --2- 172.21.15.98:0/1540782346 >> v2:172.21.15.139:3300/0 conn(0x7fa1841415a0 0x7fa1840d20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:21.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.555+0000 7fa1827fc640 1 -- 172.21.15.98:0/1540782346 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1575511632 0 0) 0x7fa18415eae0 con 0x7fa1840d3a70 2024-09-19T08:38:21.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.555+0000 7fa1827fc640 1 -- 172.21.15.98:0/1540782346 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa16c003530 con 0x7fa1840d3a70 2024-09-19T08:38:21.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.555+0000 7fa1827fc640 1 -- 172.21.15.98:0/1540782346 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2048518764 0 0) 0x7fa16c003530 con 0x7fa1840d3a70 2024-09-19T08:38:21.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.555+0000 7fa1827fc640 1 -- 172.21.15.98:0/1540782346 >> v2:172.21.15.139:3300/0 conn(0x7fa1841415a0 msgr2=0x7fa1840d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:21.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.555+0000 7fa1827fc640 1 --2- 172.21.15.98:0/1540782346 >> v2:172.21.15.139:3300/0 conn(0x7fa1841415a0 0x7fa1840d20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:21.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.555+0000 7fa1827fc640 1 -- 172.21.15.98:0/1540782346 >> v1:172.21.15.139:6789/0 conn(0x7fa1840d25f0 legacy=0x7fa1840d3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:21.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.555+0000 7fa1827fc640 1 -- 172.21.15.98:0/1540782346 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa18415f0a0 con 0x7fa1840d3a70 2024-09-19T08:38:21.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.555+0000 7fa1827fc640 1 -- 172.21.15.98:0/1540782346 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fa17c001680 con 0x7fa1840d3a70 2024-09-19T08:38:21.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.555+0000 7fa1827fc640 1 -- 172.21.15.98:0/1540782346 >> v1:172.21.15.98:6789/0 conn(0x7fa1840d3a70 legacy=0x7fa1840b8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:21.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.555+0000 7fa1827fc640 1 --2- 172.21.15.98:0/1540782346 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa16c0039c0 0x7fa16c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:21.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.555+0000 7fa1827fc640 1 --2- 172.21.15.98:0/1540782346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa16c004810 0x7fa16c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:21.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.555+0000 7fa1827fc640 1 --2- 172.21.15.98:0/1540782346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa16c0051d0 0x7fa16c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:21.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.555+0000 7fa1827fc640 1 -- 172.21.15.98:0/1540782346 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa1840d2a30 con 0x7fa16c0051d0 2024-09-19T08:38:21.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.555+0000 7fa1827fc640 1 -- 172.21.15.98:0/1540782346 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa16c00c070 con 0x7fa16c004810 2024-09-19T08:38:21.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.555+0000 7fa1827fc640 1 -- 172.21.15.98:0/1540782346 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa16c00c270 con 0x7fa16c0039c0 2024-09-19T08:38:21.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.555+0000 7fa183fff640 1 --2- 172.21.15.98:0/1540782346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa16c0051d0 0x7fa16c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:21.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.555+0000 7fa1837fe640 1 --2- 172.21.15.98:0/1540782346 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa16c0039c0 0x7fa16c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:21.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.555+0000 7fa183fff640 1 -- 172.21.15.98:0/1540782346 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa16c0039c0 msgr2=0x7fa16c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:21.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.555+0000 7fa183fff640 1 --2- 172.21.15.98:0/1540782346 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa16c0039c0 0x7fa16c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:21.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.555+0000 7fa183fff640 1 -- 172.21.15.98:0/1540782346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa16c004810 msgr2=0x7fa16c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:21.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.555+0000 7fa183fff640 1 --2- 172.21.15.98:0/1540782346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa16c004810 0x7fa16c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:21.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.555+0000 7fa183fff640 1 -- 172.21.15.98:0/1540782346 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa16c003530 con 0x7fa16c0051d0 2024-09-19T08:38:21.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.555+0000 7fa1837fe640 1 --2- 172.21.15.98:0/1540782346 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa16c0039c0 0x7fa16c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:21.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.559+0000 7fa183fff640 1 -- 172.21.15.98:0/1540782346 reap_dead start 2024-09-19T08:38:21.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.559+0000 7fa183fff640 1 --2- 172.21.15.98:0/1540782346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa16c0051d0 0x7fa16c00b9d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00c240 tx=0x7fa17c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8409a6251afd229 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:21.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.559+0000 7fa1827fc640 1 -- 172.21.15.98:0/1540782346 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c007d20 con 0x7fa16c0051d0 2024-09-19T08:38:21.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.559+0000 7fa1827fc640 1 -- 172.21.15.98:0/1540782346 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa17c0a9070 con 0x7fa16c0051d0 2024-09-19T08:38:21.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.559+0000 7fa1827fc640 1 -- 172.21.15.98:0/1540782346 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c0a5dc0 con 0x7fa16c0051d0 2024-09-19T08:38:21.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.559+0000 7fa189db5640 1 -- 172.21.15.98:0/1540782346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa16c0051d0 msgr2=0x7fa16c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:21.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.559+0000 7fa189db5640 1 --2- 172.21.15.98:0/1540782346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa16c0051d0 0x7fa16c00b9d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00c240 tx=0x7fa17c01cc80 comp rx=0 tx=0).stop 2024-09-19T08:38:21.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.559+0000 7fa189db5640 1 -- 172.21.15.98:0/1540782346 shutdown_connections 2024-09-19T08:38:21.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.559+0000 7fa189db5640 1 --2- 172.21.15.98:0/1540782346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa16c0051d0 0x7fa16c00b9d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:21.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.559+0000 7fa189db5640 1 -- 172.21.15.98:0/1540782346 >> 172.21.15.98:0/1540782346 conn(0x7fa184074c70 msgr2=0x7fa184075080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:21.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.559+0000 7fa189db5640 1 -- 172.21.15.98:0/1540782346 shutdown_connections 2024-09-19T08:38:21.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.559+0000 7fa189db5640 1 -- 172.21.15.98:0/1540782346 wait complete. 2024-09-19T08:38:21.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.559+0000 7fa189db5640 1 Processor -- start 2024-09-19T08:38:21.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.559+0000 7fa189db5640 1 -- start start 2024-09-19T08:38:21.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.559+0000 7fa189db5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa16c0051d0 0x7fa1841623b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:21.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.559+0000 7fa189db5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa18415fa20 0x7fa184143450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:21.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.563+0000 7fa189db5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa184143990 0x7fa1841e9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:21.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.563+0000 7fa189db5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa1840d20b0 con 0x7fa16c0051d0 2024-09-19T08:38:21.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.563+0000 7fa189db5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa18415f8a0 con 0x7fa184143990 2024-09-19T08:38:21.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.563+0000 7fa189db5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa1840d2230 con 0x7fa18415fa20 2024-09-19T08:38:21.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.563+0000 7fa1837fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa16c0051d0 0x7fa1841623b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:21.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.563+0000 7fa1837fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa16c0051d0 0x7fa1841623b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33298/0 (socket says 172.21.15.98:33298) 2024-09-19T08:38:21.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.563+0000 7fa1837fe640 1 -- 172.21.15.98:0/866025728 learned_addr learned my addr 172.21.15.98:0/866025728 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:21.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.563+0000 7fa1837fe640 1 -- 172.21.15.98:0/866025728 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa18415fa20 msgr2=0x7fa184143450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:21.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.563+0000 7fa1837fe640 1 --2- 172.21.15.98:0/866025728 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa18415fa20 0x7fa184143450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:21.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.563+0000 7fa1837fe640 1 -- 172.21.15.98:0/866025728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa184143990 msgr2=0x7fa1841e9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:21.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.563+0000 7fa1837fe640 1 --2- 172.21.15.98:0/866025728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa184143990 0x7fa1841e9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:21.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.563+0000 7fa1837fe640 1 -- 172.21.15.98:0/866025728 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1840d2c10 con 0x7fa16c0051d0 2024-09-19T08:38:21.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.563+0000 7fa1837fe640 1 --2- 172.21.15.98:0/866025728 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa16c0051d0 0x7fa1841623b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa17400f350 tx=0x7fa17400f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:21.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.563+0000 7fa16bfff640 1 -- 172.21.15.98:0/866025728 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17401c070 con 0x7fa16c0051d0 2024-09-19T08:38:21.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.563+0000 7fa16bfff640 1 -- 172.21.15.98:0/866025728 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa174017600 con 0x7fa16c0051d0 2024-09-19T08:38:21.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.563+0000 7fa189db5640 1 -- 172.21.15.98:0/866025728 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1841e8270 con 0x7fa16c0051d0 2024-09-19T08:38:21.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.563+0000 7fa16bfff640 1 -- 172.21.15.98:0/866025728 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1740179b0 con 0x7fa16c0051d0 2024-09-19T08:38:21.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.563+0000 7fa189db5640 1 -- 172.21.15.98:0/866025728 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1841e8830 con 0x7fa16c0051d0 2024-09-19T08:38:21.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.563+0000 7fa189db5640 1 -- 172.21.15.98:0/866025728 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1840d3ec0 con 0x7fa16c0051d0 2024-09-19T08:38:21.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.563+0000 7fa16bfff640 1 -- 172.21.15.98:0/866025728 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa17401f070 con 0x7fa16c0051d0 2024-09-19T08:38:21.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.563+0000 7fa16bfff640 1 --2- 172.21.15.98:0/866025728 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa1700678f0 0x7fa170069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:21.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.563+0000 7fa182ffd640 1 --2- 172.21.15.98:0/866025728 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa1700678f0 0x7fa170069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:21.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.563+0000 7fa16bfff640 1 -- 172.21.15.98:0/866025728 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa174085ff0 con 0x7fa16c0051d0 2024-09-19T08:38:21.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.563+0000 7fa182ffd640 1 --2- 172.21.15.98:0/866025728 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa1700678f0 0x7fa170069db0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa178007fd0 tx=0x7fa1780079e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:21.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.567+0000 7fa16bfff640 1 -- 172.21.15.98:0/866025728 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa174012070 con 0x7fa16c0051d0 2024-09-19T08:38:21.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:21.767+0000 7fa189db5640 1 -- 172.21.15.98:0/866025728 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fa184140570 con 0x7fa16c0051d0 2024-09-19T08:38:22.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.415+0000 7fa16bfff640 1 -- 172.21.15.98:0/866025728 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) ==== 122+0+0 (secure 0 0 0) 0x7fa1740564e0 con 0x7fa16c0051d0 2024-09-19T08:38:22.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.415+0000 7fa189db5640 1 -- 172.21.15.98:0/866025728 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa1700678f0 msgr2=0x7fa170069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:22.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.415+0000 7fa189db5640 1 --2- 172.21.15.98:0/866025728 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa1700678f0 0x7fa170069db0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa178007fd0 tx=0x7fa1780079e0 comp rx=0 tx=0).stop 2024-09-19T08:38:22.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.415+0000 7fa189db5640 1 -- 172.21.15.98:0/866025728 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa16c0051d0 msgr2=0x7fa1841623b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:22.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.415+0000 7fa189db5640 1 --2- 172.21.15.98:0/866025728 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa16c0051d0 0x7fa1841623b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa17400f350 tx=0x7fa17400f810 comp rx=0 tx=0).stop 2024-09-19T08:38:22.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.415+0000 7fa189db5640 1 -- 172.21.15.98:0/866025728 shutdown_connections 2024-09-19T08:38:22.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.415+0000 7fa189db5640 1 --2- 172.21.15.98:0/866025728 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa1700678f0 0x7fa170069db0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:22.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.415+0000 7fa189db5640 1 --2- 172.21.15.98:0/866025728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa184143990 0x7fa1841e9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:22.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.415+0000 7fa189db5640 1 --2- 172.21.15.98:0/866025728 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa18415fa20 0x7fa184143450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:22.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.415+0000 7fa189db5640 1 --2- 172.21.15.98:0/866025728 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa16c0051d0 0x7fa1841623b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:22.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.415+0000 7fa189db5640 1 -- 172.21.15.98:0/866025728 >> 172.21.15.98:0/866025728 conn(0x7fa184074c70 msgr2=0x7fa1840cc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:22.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.419+0000 7fa189db5640 1 -- 172.21.15.98:0/866025728 shutdown_connections 2024-09-19T08:38:22.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.419+0000 7fa189db5640 1 -- 172.21.15.98:0/866025728 wait complete. 2024-09-19T08:38:22.436 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-19T08:38:22.437 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:38:22.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.555+0000 7fd14229c640 1 Processor -- start 2024-09-19T08:38:22.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.555+0000 7fd14229c640 1 -- start start 2024-09-19T08:38:22.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.555+0000 7fd14229c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd13c15c550 0x7fd13c06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:22.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.555+0000 7fd14229c640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd13c06d310 con 0x7fd13c06c1d0 2024-09-19T08:38:22.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.555+0000 7fd14229c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd13c06d5f0 con 0x7fd13c0cfad0 2024-09-19T08:38:22.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.555+0000 7fd14229c640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fd13c06c640 con 0x7fd13c15c550 2024-09-19T08:38:22.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.555+0000 7fd13bfff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd13c15c550 0x7fd13c06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:22.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.555+0000 7fd133fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fd13c06c1d0 0x7fd13c0cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41448/0 (socket says 172.21.15.98:41448) 2024-09-19T08:38:22.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.555+0000 7fd133fff640 1 -- 172.21.15.98:0/1361296043 learned_addr learned my addr 172.21.15.98:0/1361296043 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:22.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.555+0000 7fd13bfff640 1 --2- 172.21.15.98:0/1361296043 >> v2:172.21.15.98:3300/0 conn(0x7fd13c15c550 0x7fd13c06bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33314/0 (socket says 172.21.15.98:33314) 2024-09-19T08:38:22.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.555+0000 7fd13bfff640 1 -- 172.21.15.98:0/1361296043 >> v1:172.21.15.139:6789/0 conn(0x7fd13c0cfad0 legacy=0x7fd13c0723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:22.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.555+0000 7fd13bfff640 1 -- 172.21.15.98:0/1361296043 >> v1:172.21.15.98:6789/0 conn(0x7fd13c06c1d0 legacy=0x7fd13c0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:22.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.555+0000 7fd13bfff640 1 -- 172.21.15.98:0/1361296043 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd13c06d8d0 con 0x7fd13c15c550 2024-09-19T08:38:22.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.555+0000 7fd13b7fe640 1 -- 172.21.15.98:0/1361296043 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 565985398 0 0) 0x7fd13c06d310 con 0x7fd13c06c1d0 2024-09-19T08:38:22.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.559+0000 7fd13bfff640 1 --2- 172.21.15.98:0/1361296043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd13c15c550 0x7fd13c06bc90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd12c0097c0 tx=0x7fd12c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d5455be6246a48 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:22.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.559+0000 7fd13b7fe640 1 -- 172.21.15.98:0/1361296043 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c0b0070 con 0x7fd13c15c550 2024-09-19T08:38:22.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.559+0000 7fd13b7fe640 1 -- 172.21.15.98:0/1361296043 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd12c0ab600 con 0x7fd13c15c550 2024-09-19T08:38:22.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.559+0000 7fd13b7fe640 1 -- 172.21.15.98:0/1361296043 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c0ab9d0 con 0x7fd13c15c550 2024-09-19T08:38:22.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.559+0000 7fd14229c640 1 -- 172.21.15.98:0/1361296043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd13c15c550 msgr2=0x7fd13c06bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:22.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.559+0000 7fd14229c640 1 --2- 172.21.15.98:0/1361296043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd13c15c550 0x7fd13c06bc90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd12c0097c0 tx=0x7fd12c01c460 comp rx=0 tx=0).stop 2024-09-19T08:38:22.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.559+0000 7fd14229c640 1 -- 172.21.15.98:0/1361296043 shutdown_connections 2024-09-19T08:38:22.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.559+0000 7fd14229c640 1 --2- 172.21.15.98:0/1361296043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd13c15c550 0x7fd13c06bc90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:22.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.559+0000 7fd14229c640 1 -- 172.21.15.98:0/1361296043 >> 172.21.15.98:0/1361296043 conn(0x7fd13c0b2010 msgr2=0x7fd13c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:22.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.559+0000 7fd14229c640 1 -- 172.21.15.98:0/1361296043 shutdown_connections 2024-09-19T08:38:22.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.559+0000 7fd14229c640 1 -- 172.21.15.98:0/1361296043 wait complete. 2024-09-19T08:38:22.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.559+0000 7fd14229c640 1 Processor -- start 2024-09-19T08:38:22.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.559+0000 7fd14229c640 1 -- start start 2024-09-19T08:38:22.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.559+0000 7fd14229c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd13c06c1d0 0x7fd13c14add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:22.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.559+0000 7fd14229c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd13c0cfad0 0x7fd13c14a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:22.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.559+0000 7fd14229c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd13c15c550 0x7fd13c151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:22.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.559+0000 7fd14229c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c073690 con 0x7fd13c15c550 2024-09-19T08:38:22.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.559+0000 7fd14229c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c073510 con 0x7fd13c0cfad0 2024-09-19T08:38:22.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.559+0000 7fd14229c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd13c16a240 con 0x7fd13c06c1d0 2024-09-19T08:38:22.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.559+0000 7fd140839640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd13c15c550 0x7fd13c151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:22.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.559+0000 7fd140839640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd13c15c550 0x7fd13c151f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33330/0 (socket says 172.21.15.98:33330) 2024-09-19T08:38:22.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.559+0000 7fd140839640 1 -- 172.21.15.98:0/850000227 learned_addr learned my addr 172.21.15.98:0/850000227 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:22.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.559+0000 7fd13bfff640 1 --2- 172.21.15.98:0/850000227 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd13c06c1d0 0x7fd13c14add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:22.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.563+0000 7fd133fff640 1 --2- 172.21.15.98:0/850000227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd13c0cfad0 0x7fd13c14a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:22.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.563+0000 7fd140839640 1 -- 172.21.15.98:0/850000227 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd13c06c1d0 msgr2=0x7fd13c14add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:22.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.563+0000 7fd140839640 1 --2- 172.21.15.98:0/850000227 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd13c06c1d0 0x7fd13c14add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:22.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.563+0000 7fd140839640 1 -- 172.21.15.98:0/850000227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd13c0cfad0 msgr2=0x7fd13c14a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:22.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.563+0000 7fd140839640 1 --2- 172.21.15.98:0/850000227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd13c0cfad0 0x7fd13c14a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:22.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.563+0000 7fd140839640 1 -- 172.21.15.98:0/850000227 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd13c14b310 con 0x7fd13c15c550 2024-09-19T08:38:22.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.563+0000 7fd133fff640 1 --2- 172.21.15.98:0/850000227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd13c0cfad0 0x7fd13c14a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:22.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.563+0000 7fd140839640 1 --2- 172.21.15.98:0/850000227 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd13c15c550 0x7fd13c151f90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd13400c7b0 tx=0x7fd13400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:22.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.563+0000 7fd1397fa640 1 -- 172.21.15.98:0/850000227 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd134004b70 con 0x7fd13c15c550 2024-09-19T08:38:22.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.563+0000 7fd1397fa640 1 -- 172.21.15.98:0/850000227 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd134004d10 con 0x7fd13c15c550 2024-09-19T08:38:22.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.563+0000 7fd14229c640 1 -- 172.21.15.98:0/850000227 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd13c1549f0 con 0x7fd13c15c550 2024-09-19T08:38:22.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.563+0000 7fd1397fa640 1 -- 172.21.15.98:0/850000227 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13400aa90 con 0x7fd13c15c550 2024-09-19T08:38:22.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.563+0000 7fd14229c640 1 -- 172.21.15.98:0/850000227 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd13c152b60 con 0x7fd13c15c550 2024-09-19T08:38:22.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.563+0000 7fd14229c640 1 -- 172.21.15.98:0/850000227 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd13c06bc90 con 0x7fd13c15c550 2024-09-19T08:38:22.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.563+0000 7fd1397fa640 1 -- 172.21.15.98:0/850000227 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd13400ac30 con 0x7fd13c15c550 2024-09-19T08:38:22.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.563+0000 7fd1397fa640 1 --2- 172.21.15.98:0/850000227 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd11c0678c0 0x7fd11c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:22.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.563+0000 7fd13bfff640 1 --2- 172.21.15.98:0/850000227 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd11c0678c0 0x7fd11c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:22.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.563+0000 7fd1397fa640 1 -- 172.21.15.98:0/850000227 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd134085e40 con 0x7fd13c15c550 2024-09-19T08:38:22.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.563+0000 7fd13bfff640 1 --2- 172.21.15.98:0/850000227 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd11c0678c0 0x7fd11c069d80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd12c0ad000 tx=0x7fd12c0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:22.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.567+0000 7fd1397fa640 1 -- 172.21.15.98:0/850000227 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd134020070 con 0x7fd13c15c550 2024-09-19T08:38:22.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.759+0000 7fd14229c640 1 -- 172.21.15.98:0/850000227 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd13c15b850 con 0x7fd13c15c550 2024-09-19T08:38:22.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.759+0000 7fd1397fa640 1 -- 172.21.15.98:0/850000227 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5220 (secure 0 0 0) 0x7fd134056a10 con 0x7fd13c15c550 2024-09-19T08:38:22.759 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:38:22.759 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":7,"btime":"2024-09-19T08:38:22:407519+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/2038744461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":2038744461},{"type":"v1","addr":"172.21.15.139:6833","nonce":2038744461}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/765134249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":765134249},{"type":"v1","addr":"172.21.15.98:6835","nonce":765134249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2832328076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2832328076},{"type":"v1","addr":"172.21.15.98:6837","nonce":2832328076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:38:18.717221+0000","modified":"2024-09-19T08:38:21.769948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6835/1879089763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1879089763},{"type":"v1","addr":"172.21.15.139:6835","nonce":1879089763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4261,"qdb_cluster":[4261]},"id":1}]} 2024-09-19T08:38:22.759 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 7 2024-09-19T08:38:22.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.759+0000 7fd14229c640 1 -- 172.21.15.98:0/850000227 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd11c0678c0 msgr2=0x7fd11c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:22.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.759+0000 7fd14229c640 1 --2- 172.21.15.98:0/850000227 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd11c0678c0 0x7fd11c069d80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd12c0ad000 tx=0x7fd12c0a6000 comp rx=0 tx=0).stop 2024-09-19T08:38:22.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.759+0000 7fd14229c640 1 -- 172.21.15.98:0/850000227 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd13c15c550 msgr2=0x7fd13c151f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:22.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.759+0000 7fd14229c640 1 --2- 172.21.15.98:0/850000227 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd13c15c550 0x7fd13c151f90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd13400c7b0 tx=0x7fd13400cc70 comp rx=0 tx=0).stop 2024-09-19T08:38:22.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.759+0000 7fd14229c640 1 -- 172.21.15.98:0/850000227 shutdown_connections 2024-09-19T08:38:22.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.759+0000 7fd14229c640 1 --2- 172.21.15.98:0/850000227 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd11c0678c0 0x7fd11c069d80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:22.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.759+0000 7fd14229c640 1 --2- 172.21.15.98:0/850000227 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd13c15c550 0x7fd13c151f90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:22.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.759+0000 7fd14229c640 1 --2- 172.21.15.98:0/850000227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd13c0cfad0 0x7fd13c14a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:22.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.759+0000 7fd14229c640 1 --2- 172.21.15.98:0/850000227 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd13c06c1d0 0x7fd13c14add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:22.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.759+0000 7fd14229c640 1 -- 172.21.15.98:0/850000227 >> 172.21.15.98:0/850000227 conn(0x7fd13c0b2010 msgr2=0x7fd13c0b15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:22.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.759+0000 7fd14229c640 1 -- 172.21.15.98:0/850000227 shutdown_connections 2024-09-19T08:38:22.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.759+0000 7fd14229c640 1 -- 172.21.15.98:0/850000227 wait complete. 2024-09-19T08:38:22.776 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:38:22.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.887+0000 7fb6c2e19640 1 Processor -- start 2024-09-19T08:38:22.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.887+0000 7fb6c2e19640 1 -- start start 2024-09-19T08:38:22.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.887+0000 7fb6c2e19640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6bc15c550 0x7fb6bc06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:22.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.887+0000 7fb6c2e19640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb6bc0cfad0 0x7fb6bc0723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:22.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.887+0000 7fb6c2e19640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6bc06d310 con 0x7fb6bc06c1d0 2024-09-19T08:38:22.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.887+0000 7fb6c2e19640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fb6bc06c610 con 0x7fb6bc0cfad0 2024-09-19T08:38:22.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.887+0000 7fb6c2e19640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb6bc06d5f0 con 0x7fb6bc15c550 2024-09-19T08:38:22.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6bbfff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fb6bc06c1d0 0x7fb6bc0cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41458/0 (socket says 172.21.15.98:41458) 2024-09-19T08:38:22.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6bbfff640 1 -- 172.21.15.98:0/1391198087 learned_addr learned my addr 172.21.15.98:0/1391198087 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:22.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6c13b6640 1 --2- 172.21.15.98:0/1391198087 >> v2:172.21.15.139:3300/0 conn(0x7fb6bc0cfad0 0x7fb6bc0723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:22.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6c0bb5640 1 --2- 172.21.15.98:0/1391198087 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6bc15c550 0x7fb6bc06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:22.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6bb7fe640 1 -- 172.21.15.98:0/1391198087 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3588868691 0 0) 0x7fb6bc06d310 con 0x7fb6bc06c1d0 2024-09-19T08:38:22.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6bb7fe640 1 -- 172.21.15.98:0/1391198087 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6a4003530 con 0x7fb6bc06c1d0 2024-09-19T08:38:22.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6bb7fe640 1 -- 172.21.15.98:0/1391198087 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2393448059 0 0) 0x7fb6a4003530 con 0x7fb6bc06c1d0 2024-09-19T08:38:22.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6bb7fe640 1 -- 172.21.15.98:0/1391198087 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6bc15c550 msgr2=0x7fb6bc06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:22.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6bb7fe640 1 --2- 172.21.15.98:0/1391198087 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6bc15c550 0x7fb6bc06bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:22.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6bb7fe640 1 -- 172.21.15.98:0/1391198087 >> v2:172.21.15.139:3300/0 conn(0x7fb6bc0cfad0 msgr2=0x7fb6bc0723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:22.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6bb7fe640 1 --2- 172.21.15.98:0/1391198087 >> v2:172.21.15.139:3300/0 conn(0x7fb6bc0cfad0 0x7fb6bc0723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:22.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6bb7fe640 1 -- 172.21.15.98:0/1391198087 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6bc06d8d0 con 0x7fb6bc06c1d0 2024-09-19T08:38:22.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6bb7fe640 1 -- 172.21.15.98:0/1391198087 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fb6b0001680 con 0x7fb6bc06c1d0 2024-09-19T08:38:22.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6bb7fe640 1 -- 172.21.15.98:0/1391198087 >> v1:172.21.15.98:6789/0 conn(0x7fb6bc06c1d0 legacy=0x7fb6bc0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:22.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6bb7fe640 1 --2- 172.21.15.98:0/1391198087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6a40039c0 0x7fb6a4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:22.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6bb7fe640 1 --2- 172.21.15.98:0/1391198087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6a4004810 0x7fb6a4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:22.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6bb7fe640 1 --2- 172.21.15.98:0/1391198087 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6bc15c550 0x7fb6a40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:22.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6bb7fe640 1 -- 172.21.15.98:0/1391198087 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb6bc06c610 con 0x7fb6a40039c0 2024-09-19T08:38:22.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6bb7fe640 1 -- 172.21.15.98:0/1391198087 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb6bc06d5f0 con 0x7fb6a4004810 2024-09-19T08:38:22.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6bb7fe640 1 -- 172.21.15.98:0/1391198087 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb6a400b9d0 con 0x7fb6bc15c550 2024-09-19T08:38:22.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6c0bb5640 1 --2- 172.21.15.98:0/1391198087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6a40039c0 0x7fb6a4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:22.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6bbfff640 1 --2- 172.21.15.98:0/1391198087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6a4004810 0x7fb6a4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:22.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6c13b6640 1 --2- 172.21.15.98:0/1391198087 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6bc15c550 0x7fb6a40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:22.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6c0bb5640 1 -- 172.21.15.98:0/1391198087 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6bc15c550 msgr2=0x7fb6a40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:22.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6c0bb5640 1 --2- 172.21.15.98:0/1391198087 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6bc15c550 0x7fb6a40051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:22.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6c0bb5640 1 -- 172.21.15.98:0/1391198087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6a4004810 msgr2=0x7fb6a4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:22.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6c0bb5640 1 --2- 172.21.15.98:0/1391198087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6a4004810 0x7fb6a4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:22.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6c0bb5640 1 -- 172.21.15.98:0/1391198087 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6a4003530 con 0x7fb6a40039c0 2024-09-19T08:38:22.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6c0bb5640 1 --2- 172.21.15.98:0/1391198087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6a40039c0 0x7fb6a4003de0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac009ac0 tx=0x7fb6ac01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3992cd06c39824 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:22.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6bb7fe640 1 -- 172.21.15.98:0/1391198087 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac009690 con 0x7fb6a40039c0 2024-09-19T08:38:22.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6bb7fe640 1 -- 172.21.15.98:0/1391198087 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6ac0a9450 con 0x7fb6a40039c0 2024-09-19T08:38:22.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6bb7fe640 1 -- 172.21.15.98:0/1391198087 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac0a9820 con 0x7fb6a40039c0 2024-09-19T08:38:22.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6c2e19640 1 -- 172.21.15.98:0/1391198087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6a40039c0 msgr2=0x7fb6a4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:22.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6c2e19640 1 --2- 172.21.15.98:0/1391198087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6a40039c0 0x7fb6a4003de0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac009ac0 tx=0x7fb6ac01ca50 comp rx=0 tx=0).stop 2024-09-19T08:38:22.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6c13b6640 1 -- 172.21.15.98:0/1391198087 reap_dead start 2024-09-19T08:38:22.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6c2e19640 1 -- 172.21.15.98:0/1391198087 shutdown_connections 2024-09-19T08:38:22.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6c2e19640 1 -- 172.21.15.98:0/1391198087 >> 172.21.15.98:0/1391198087 conn(0x7fb6bc0b2010 msgr2=0x7fb6bc0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:22.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.891+0000 7fb6c2e19640 1 -- 172.21.15.98:0/1391198087 shutdown_connections 2024-09-19T08:38:22.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.895+0000 7fb6c2e19640 1 -- 172.21.15.98:0/1391198087 wait complete. 2024-09-19T08:38:22.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.895+0000 7fb6c2e19640 1 Processor -- start 2024-09-19T08:38:22.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.895+0000 7fb6c2e19640 1 -- start start 2024-09-19T08:38:22.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.895+0000 7fb6c2e19640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6a40039c0 0x7fb6bc150ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:22.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.895+0000 7fb6c2e19640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6bc154e00 0x7fb6bc0aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:22.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.895+0000 7fb6c2e19640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6bc1488b0 0x7fb6bc148d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:22.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.895+0000 7fb6c2e19640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb6bc073300 con 0x7fb6bc154e00 2024-09-19T08:38:22.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.895+0000 7fb6c2e19640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb6bc15cb40 con 0x7fb6bc1488b0 2024-09-19T08:38:22.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.895+0000 7fb6c2e19640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb6bc073480 con 0x7fb6a40039c0 2024-09-19T08:38:22.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.895+0000 7fb6bbfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6bc154e00 0x7fb6bc0aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:22.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.895+0000 7fb6bbfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6bc154e00 0x7fb6bc0aeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33356/0 (socket says 172.21.15.98:33356) 2024-09-19T08:38:22.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.895+0000 7fb6bbfff640 1 -- 172.21.15.98:0/1101458185 learned_addr learned my addr 172.21.15.98:0/1101458185 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:22.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.895+0000 7fb6c0bb5640 1 --2- 172.21.15.98:0/1101458185 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6a40039c0 0x7fb6bc150ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:22.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.895+0000 7fb6c13b6640 1 --2- 172.21.15.98:0/1101458185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6bc1488b0 0x7fb6bc148d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:22.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.895+0000 7fb6bbfff640 1 -- 172.21.15.98:0/1101458185 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6a40039c0 msgr2=0x7fb6bc150ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:22.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.895+0000 7fb6bbfff640 1 --2- 172.21.15.98:0/1101458185 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6a40039c0 0x7fb6bc150ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:22.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.895+0000 7fb6bbfff640 1 -- 172.21.15.98:0/1101458185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6bc1488b0 msgr2=0x7fb6bc148d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:22.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.895+0000 7fb6bbfff640 1 --2- 172.21.15.98:0/1101458185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6bc1488b0 0x7fb6bc148d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:22.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.895+0000 7fb6bbfff640 1 -- 172.21.15.98:0/1101458185 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6bc149250 con 0x7fb6bc154e00 2024-09-19T08:38:22.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.895+0000 7fb6bbfff640 1 --2- 172.21.15.98:0/1101458185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6bc154e00 0x7fb6bc0aeb10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0001b50 tx=0x7fb6b000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:22.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.895+0000 7fb6b97fa640 1 -- 172.21.15.98:0/1101458185 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b000ee60 con 0x7fb6bc154e00 2024-09-19T08:38:22.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.895+0000 7fb6b97fa640 1 -- 172.21.15.98:0/1101458185 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6b0018070 con 0x7fb6bc154e00 2024-09-19T08:38:22.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.895+0000 7fb6b97fa640 1 -- 172.21.15.98:0/1101458185 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b000ca70 con 0x7fb6bc154e00 2024-09-19T08:38:22.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.895+0000 7fb6c2e19640 1 -- 172.21.15.98:0/1101458185 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6bc155300 con 0x7fb6bc154e00 2024-09-19T08:38:22.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.895+0000 7fb6c2e19640 1 -- 172.21.15.98:0/1101458185 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6bc1558c0 con 0x7fb6bc154e00 2024-09-19T08:38:22.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.895+0000 7fb6c2e19640 1 -- 172.21.15.98:0/1101458185 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6bc0723a0 con 0x7fb6bc154e00 2024-09-19T08:38:22.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.899+0000 7fb6b97fa640 1 -- 172.21.15.98:0/1101458185 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb6b000cc10 con 0x7fb6bc154e00 2024-09-19T08:38:22.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.899+0000 7fb6b97fa640 1 --2- 172.21.15.98:0/1101458185 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb6a80678f0 0x7fb6a8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:22.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.899+0000 7fb6b97fa640 1 -- 172.21.15.98:0/1101458185 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb6b0085af0 con 0x7fb6bc154e00 2024-09-19T08:38:22.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.899+0000 7fb6c0bb5640 1 --2- 172.21.15.98:0/1101458185 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb6a80678f0 0x7fb6a8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:22.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.899+0000 7fb6c0bb5640 1 --2- 172.21.15.98:0/1101458185 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb6a80678f0 0x7fb6a8069db0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac0ae000 tx=0x7fb6ac0b3040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:22.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:22.903+0000 7fb6b97fa640 1 -- 172.21.15.98:0/1101458185 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb6b0056590 con 0x7fb6bc154e00 2024-09-19T08:38:23.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.059+0000 7fb6c2e19640 1 -- 172.21.15.98:0/1101458185 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6bc0d08f0 con 0x7fb6bc154e00 2024-09-19T08:38:23.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.059+0000 7fb6b97fa640 1 -- 172.21.15.98:0/1101458185 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16665 (secure 0 0 0) 0x7fb6b0055e90 con 0x7fb6bc154e00 2024-09-19T08:38:23.062 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:38:23.062 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":21,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:38:20.362527+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:38:16.588688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T08:38:17.700392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:38:23.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.063+0000 7fb6c2e19640 1 -- 172.21.15.98:0/1101458185 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb6a80678f0 msgr2=0x7fb6a8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:23.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.063+0000 7fb6c2e19640 1 --2- 172.21.15.98:0/1101458185 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb6a80678f0 0x7fb6a8069db0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac0ae000 tx=0x7fb6ac0b3040 comp rx=0 tx=0).stop 2024-09-19T08:38:23.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.063+0000 7fb6c2e19640 1 -- 172.21.15.98:0/1101458185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6bc154e00 msgr2=0x7fb6bc0aeb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:23.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.063+0000 7fb6c2e19640 1 --2- 172.21.15.98:0/1101458185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6bc154e00 0x7fb6bc0aeb10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0001b50 tx=0x7fb6b000ec10 comp rx=0 tx=0).stop 2024-09-19T08:38:23.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.063+0000 7fb6c2e19640 1 -- 172.21.15.98:0/1101458185 shutdown_connections 2024-09-19T08:38:23.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.063+0000 7fb6c2e19640 1 --2- 172.21.15.98:0/1101458185 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb6a80678f0 0x7fb6a8069db0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:23.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.063+0000 7fb6c2e19640 1 --2- 172.21.15.98:0/1101458185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6bc1488b0 0x7fb6bc148d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:23.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.063+0000 7fb6c2e19640 1 --2- 172.21.15.98:0/1101458185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6bc154e00 0x7fb6bc0aeb10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:23.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.063+0000 7fb6c2e19640 1 --2- 172.21.15.98:0/1101458185 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6a40039c0 0x7fb6bc150ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:23.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.063+0000 7fb6c2e19640 1 -- 172.21.15.98:0/1101458185 >> 172.21.15.98:0/1101458185 conn(0x7fb6bc0b2010 msgr2=0x7fb6bc067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:23.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.063+0000 7fb6c2e19640 1 -- 172.21.15.98:0/1101458185 shutdown_connections 2024-09-19T08:38:23.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.063+0000 7fb6c2e19640 1 -- 172.21.15.98:0/1101458185 wait complete. 2024-09-19T08:38:23.080 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T08:38:23.080 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:38:23.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.195+0000 7f0d1bf2c640 1 Processor -- start 2024-09-19T08:38:23.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.195+0000 7f0d1bf2c640 1 -- start start 2024-09-19T08:38:23.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.195+0000 7f0d1bf2c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0d1406f380 0x7f0d140ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:23.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.195+0000 7f0d1bf2c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0d140cec60 0x7f0d140aee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:23.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.195+0000 7f0d1bf2c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d140af4a0 con 0x7f0d1406efa0 2024-09-19T08:38:23.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.195+0000 7f0d1bf2c640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f0d1415ed00 con 0x7f0d1406f380 2024-09-19T08:38:23.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.195+0000 7f0d1bf2c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f0d140af780 con 0x7f0d140cec60 2024-09-19T08:38:23.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.195+0000 7f0d194c7640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0d1406f380 0x7f0d140ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:23.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.195+0000 7f0d194c7640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0d1406f380 0x7f0d140ae8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33372/0 (socket says 172.21.15.98:33372) 2024-09-19T08:38:23.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.195+0000 7f0d194c7640 1 -- 172.21.15.98:0/2903032835 learned_addr learned my addr 172.21.15.98:0/2903032835 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:23.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.195+0000 7f0d1a4c9640 1 --2- 172.21.15.98:0/2903032835 >> v2:172.21.15.139:3300/0 conn(0x7f0d140cec60 0x7f0d140aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:23.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.195+0000 7f0d194c7640 1 -- 172.21.15.98:0/2903032835 >> v2:172.21.15.139:3300/0 conn(0x7f0d140cec60 msgr2=0x7f0d140aee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:23.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.195+0000 7f0d194c7640 1 --2- 172.21.15.98:0/2903032835 >> v2:172.21.15.139:3300/0 conn(0x7f0d140cec60 0x7f0d140aee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:23.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.195+0000 7f0d194c7640 1 -- 172.21.15.98:0/2903032835 >> v1:172.21.15.139:6789/0 conn(0x7f0d1406efa0 legacy=0x7f0d140ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:23.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.195+0000 7f0d194c7640 1 -- 172.21.15.98:0/2903032835 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d140afa60 con 0x7f0d1406f380 2024-09-19T08:38:23.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.199+0000 7f0d194c7640 1 --2- 172.21.15.98:0/2903032835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d1406f380 0x7f0d140ae8c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0d10009870 tx=0x7f0d1001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8701d79cf810fb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:23.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.199+0000 7f0d18cc6640 1 -- 172.21.15.98:0/2903032835 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d100b0070 con 0x7f0d1406f380 2024-09-19T08:38:23.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.199+0000 7f0d18cc6640 1 -- 172.21.15.98:0/2903032835 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d100ab600 con 0x7f0d1406f380 2024-09-19T08:38:23.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.199+0000 7f0d18cc6640 1 -- 172.21.15.98:0/2903032835 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d100ab9d0 con 0x7f0d1406f380 2024-09-19T08:38:23.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.199+0000 7f0d1bf2c640 1 -- 172.21.15.98:0/2903032835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d1406f380 msgr2=0x7f0d140ae8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:23.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.199+0000 7f0d1bf2c640 1 --2- 172.21.15.98:0/2903032835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d1406f380 0x7f0d140ae8c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0d10009870 tx=0x7f0d1001c3c0 comp rx=0 tx=0).stop 2024-09-19T08:38:23.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.199+0000 7f0d1bf2c640 1 -- 172.21.15.98:0/2903032835 shutdown_connections 2024-09-19T08:38:23.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.199+0000 7f0d1bf2c640 1 --2- 172.21.15.98:0/2903032835 >> v2:172.21.15.139:3300/0 conn(0x7f0d140cec60 0x7f0d140aee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:23.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.199+0000 7f0d1bf2c640 1 --2- 172.21.15.98:0/2903032835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d1406f380 0x7f0d140ae8c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:23.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.199+0000 7f0d1bf2c640 1 -- 172.21.15.98:0/2903032835 >> 172.21.15.98:0/2903032835 conn(0x7f0d140d0980 msgr2=0x7f0d140d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:23.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.199+0000 7f0d1bf2c640 1 -- 172.21.15.98:0/2903032835 shutdown_connections 2024-09-19T08:38:23.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.199+0000 7f0d1bf2c640 1 -- 172.21.15.98:0/2903032835 wait complete. 2024-09-19T08:38:23.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.199+0000 7f0d1bf2c640 1 Processor -- start 2024-09-19T08:38:23.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.199+0000 7f0d1bf2c640 1 -- start start 2024-09-19T08:38:23.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.203+0000 7f0d1bf2c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d1406efa0 0x7f0d1414e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:23.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.203+0000 7f0d1bf2c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d140cec60 0x7f0d14154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:23.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.203+0000 7f0d1bf2c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d14155320 0x7f0d14154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:23.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.203+0000 7f0d1bf2c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1415c180 con 0x7f0d140cec60 2024-09-19T08:38:23.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.203+0000 7f0d1bf2c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1415c000 con 0x7f0d1406efa0 2024-09-19T08:38:23.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.203+0000 7f0d1bf2c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0d1415c300 con 0x7f0d14155320 2024-09-19T08:38:23.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.203+0000 7f0d194c7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d140cec60 0x7f0d14154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:23.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.203+0000 7f0d194c7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d140cec60 0x7f0d14154de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33388/0 (socket says 172.21.15.98:33388) 2024-09-19T08:38:23.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.203+0000 7f0d194c7640 1 -- 172.21.15.98:0/2789271816 learned_addr learned my addr 172.21.15.98:0/2789271816 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:23.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.203+0000 7f0d194c7640 1 -- 172.21.15.98:0/2789271816 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d14155320 msgr2=0x7f0d14154020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:23.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.203+0000 7f0d19cc8640 1 --2- 172.21.15.98:0/2789271816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d1406efa0 0x7f0d1414e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:23.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.203+0000 7f0d1a4c9640 1 --2- 172.21.15.98:0/2789271816 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d14155320 0x7f0d14154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:23.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.203+0000 7f0d194c7640 1 --2- 172.21.15.98:0/2789271816 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d14155320 0x7f0d14154020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:23.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.203+0000 7f0d194c7640 1 -- 172.21.15.98:0/2789271816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d1406efa0 msgr2=0x7f0d1414e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:23.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.203+0000 7f0d194c7640 1 --2- 172.21.15.98:0/2789271816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d1406efa0 0x7f0d1414e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:23.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.203+0000 7f0d194c7640 1 -- 172.21.15.98:0/2789271816 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d14155860 con 0x7f0d140cec60 2024-09-19T08:38:23.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.203+0000 7f0d19cc8640 1 --2- 172.21.15.98:0/2789271816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d1406efa0 0x7f0d1414e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:23.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.203+0000 7f0d194c7640 1 --2- 172.21.15.98:0/2789271816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d140cec60 0x7f0d14154de0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0d100ae040 tx=0x7f0d1001ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:23.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.203+0000 7f0d0a7fc640 1 -- 172.21.15.98:0/2789271816 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d100ba020 con 0x7f0d140cec60 2024-09-19T08:38:23.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.203+0000 7f0d0a7fc640 1 -- 172.21.15.98:0/2789271816 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d100abe10 con 0x7f0d140cec60 2024-09-19T08:38:23.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.203+0000 7f0d1bf2c640 1 -- 172.21.15.98:0/2789271816 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d14155b40 con 0x7f0d140cec60 2024-09-19T08:38:23.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.203+0000 7f0d0a7fc640 1 -- 172.21.15.98:0/2789271816 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d100b0040 con 0x7f0d140cec60 2024-09-19T08:38:23.216 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.203+0000 7f0d1bf2c640 1 -- 172.21.15.98:0/2789271816 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d14151510 con 0x7f0d140cec60 2024-09-19T08:38:23.216 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.203+0000 7f0d1bf2c640 1 -- 172.21.15.98:0/2789271816 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d140ce550 con 0x7f0d140cec60 2024-09-19T08:38:23.216 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.203+0000 7f0d0a7fc640 1 -- 172.21.15.98:0/2789271816 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0d100bd020 con 0x7f0d140cec60 2024-09-19T08:38:23.216 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.203+0000 7f0d0a7fc640 1 --2- 172.21.15.98:0/2789271816 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0cf80678c0 0x7f0cf8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:23.216 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.203+0000 7f0d19cc8640 1 --2- 172.21.15.98:0/2789271816 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0cf80678c0 0x7f0cf8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:23.216 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.203+0000 7f0d0a7fc640 1 -- 172.21.15.98:0/2789271816 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0d101200b0 con 0x7f0d140cec60 2024-09-19T08:38:23.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.203+0000 7f0d19cc8640 1 --2- 172.21.15.98:0/2789271816 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0cf80678c0 0x7f0cf8069d80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0d0400a850 tx=0x7f0d04008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:23.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.207+0000 7f0d0a7fc640 1 -- 172.21.15.98:0/2789271816 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f0d100c4030 con 0x7f0d140cec60 2024-09-19T08:38:23.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.355+0000 7f0d1bf2c640 1 -- 172.21.15.98:0/2789271816 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0d1406fee0 con 0x7f0cf80678c0 2024-09-19T08:38:23.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.367+0000 7f0d0a7fc640 1 -- 172.21.15.98:0/2789271816 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+244523 (secure 0 0 0) 0x7f0d1406fee0 con 0x7f0cf80678c0 2024-09-19T08:38:23.372 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:38:23.373 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:38:23.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.371+0000 7f0d1bf2c640 1 -- 172.21.15.98:0/2789271816 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0cf80678c0 msgr2=0x7f0cf8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:23.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.371+0000 7f0d1bf2c640 1 --2- 172.21.15.98:0/2789271816 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0cf80678c0 0x7f0cf8069d80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0d0400a850 tx=0x7f0d04008040 comp rx=0 tx=0).stop 2024-09-19T08:38:23.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.371+0000 7f0d1bf2c640 1 -- 172.21.15.98:0/2789271816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d140cec60 msgr2=0x7f0d14154de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:23.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.371+0000 7f0d1bf2c640 1 --2- 172.21.15.98:0/2789271816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d140cec60 0x7f0d14154de0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0d100ae040 tx=0x7f0d1001ac00 comp rx=0 tx=0).stop 2024-09-19T08:38:23.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.371+0000 7f0d1bf2c640 1 -- 172.21.15.98:0/2789271816 shutdown_connections 2024-09-19T08:38:23.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.371+0000 7f0d1bf2c640 1 --2- 172.21.15.98:0/2789271816 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0cf80678c0 0x7f0cf8069d80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:23.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.371+0000 7f0d1bf2c640 1 --2- 172.21.15.98:0/2789271816 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0d14155320 0x7f0d14154020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:23.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.371+0000 7f0d1bf2c640 1 --2- 172.21.15.98:0/2789271816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0d140cec60 0x7f0d14154de0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:23.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.371+0000 7f0d1bf2c640 1 --2- 172.21.15.98:0/2789271816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0d1406efa0 0x7f0d1414e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:23.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.371+0000 7f0d1bf2c640 1 -- 172.21.15.98:0/2789271816 >> 172.21.15.98:0/2789271816 conn(0x7f0d140d0980 msgr2=0x7f0d1415d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:23.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.371+0000 7f0d1bf2c640 1 -- 172.21.15.98:0/2789271816 shutdown_connections 2024-09-19T08:38:23.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.371+0000 7f0d1bf2c640 1 -- 172.21.15.98:0/2789271816 wait complete. 2024-09-19T08:38:23.387 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-19T08:38:22.027036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":148,"acting":148,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":38,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218772,"kb_used_data":2540,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755948,"statfs":{"total":773094113280,"available":772870090752,"internally_reserved":0,"allocated":2600960,"data_stored":1624955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.691138"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.368603+0000","last_change":"2024-09-19T08:38:18.368603+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.368603+0000","last_undegraded":"2024-09-19T08:38:18.368603+0000","last_fullsized":"2024-09-19T08:38:18.368603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.349394+0000","last_change":"2024-09-19T08:38:18.349394+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.349394+0000","last_undegraded":"2024-09-19T08:38:18.349394+0000","last_fullsized":"2024-09-19T08:38:18.349394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.352974+0000","last_change":"2024-09-19T08:38:18.352974+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.352974+0000","last_undegraded":"2024-09-19T08:38:18.352974+0000","last_fullsized":"2024-09-19T08:38:18.352974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.389392+0000","last_change":"2024-09-19T08:38:18.389392+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.389392+0000","last_undegraded":"2024-09-19T08:38:18.389392+0000","last_fullsized":"2024-09-19T08:38:18.389392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.361760+0000","last_change":"2024-09-19T08:38:18.361760+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.361760+0000","last_undegraded":"2024-09-19T08:38:18.361760+0000","last_fullsized":"2024-09-19T08:38:18.361760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.389879+0000","last_change":"2024-09-19T08:38:18.389879+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.389879+0000","last_undegraded":"2024-09-19T08:38:18.389879+0000","last_fullsized":"2024-09-19T08:38:18.389879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.363729+0000","last_change":"2024-09-19T08:38:18.363729+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.363729+0000","last_undegraded":"2024-09-19T08:38:18.363729+0000","last_fullsized":"2024-09-19T08:38:18.363729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:18.334613+0000","last_change":"2024-09-19T08:38:18.334613+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.334613+0000","last_undegraded":"2024-09-19T08:38:18.334613+0000","last_fullsized":"2024-09-19T08:38:18.334613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.366761+0000","last_change":"2024-09-19T08:38:18.366761+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.366761+0000","last_undegraded":"2024-09-19T08:38:18.366761+0000","last_fullsized":"2024-09-19T08:38:18.366761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:18.334613+0000","last_change":"2024-09-19T08:38:18.334613+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.334613+0000","last_undegraded":"2024-09-19T08:38:18.334613+0000","last_fullsized":"2024-09-19T08:38:18.334613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:18.334613+0000","last_change":"2024-09-19T08:38:18.334613+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.334613+0000","last_undegraded":"2024-09-19T08:38:18.334613+0000","last_fullsized":"2024-09-19T08:38:18.334613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.355429+0000","last_change":"2024-09-19T08:38:18.355429+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.355429+0000","last_undegraded":"2024-09-19T08:38:18.355429+0000","last_fullsized":"2024-09-19T08:38:18.355429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:18.334613+0000","last_change":"2024-09-19T08:38:18.334613+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.334613+0000","last_undegraded":"2024-09-19T08:38:18.334613+0000","last_fullsized":"2024-09-19T08:38:18.334613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.350914+0000","last_change":"2024-09-19T08:38:18.350914+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.350914+0000","last_undegraded":"2024-09-19T08:38:18.350914+0000","last_fullsized":"2024-09-19T08:38:18.350914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.349265+0000","last_change":"2024-09-19T08:38:18.349265+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.349265+0000","last_undegraded":"2024-09-19T08:38:18.349265+0000","last_fullsized":"2024-09-19T08:38:18.349265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.368264+0000","last_change":"2024-09-19T08:38:18.368264+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.368264+0000","last_undegraded":"2024-09-19T08:38:18.368264+0000","last_fullsized":"2024-09-19T08:38:18.368264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.366483+0000","last_change":"2024-09-19T08:38:18.366483+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.366483+0000","last_undegraded":"2024-09-19T08:38:18.366483+0000","last_fullsized":"2024-09-19T08:38:18.366483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.354376+0000","last_change":"2024-09-19T08:38:18.354376+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.354376+0000","last_undegraded":"2024-09-19T08:38:18.354376+0000","last_fullsized":"2024-09-19T08:38:18.354376+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.356210+0000","last_change":"2024-09-19T08:38:18.356210+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.356210+0000","last_undegraded":"2024-09-19T08:38:18.356210+0000","last_fullsized":"2024-09-19T08:38:18.356210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:18.334613+0000","last_change":"2024-09-19T08:38:18.334613+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.334613+0000","last_undegraded":"2024-09-19T08:38:18.334613+0000","last_fullsized":"2024-09-19T08:38:18.334613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.351092+0000","last_change":"2024-09-19T08:38:18.351092+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.351092+0000","last_undegraded":"2024-09-19T08:38:18.351092+0000","last_fullsized":"2024-09-19T08:38:18.351092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.371212+0000","last_change":"2024-09-19T08:38:18.371212+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.371212+0000","last_undegraded":"2024-09-19T08:38:18.371212+0000","last_fullsized":"2024-09-19T08:38:18.371212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:18.334613+0000","last_change":"2024-09-19T08:38:18.334613+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.334613+0000","last_undegraded":"2024-09-19T08:38:18.334613+0000","last_fullsized":"2024-09-19T08:38:18.334613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.363746+0000","last_change":"2024-09-19T08:38:18.363746+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.363746+0000","last_undegraded":"2024-09-19T08:38:18.363746+0000","last_fullsized":"2024-09-19T08:38:18.363746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:18.334613+0000","last_change":"2024-09-19T08:38:18.334613+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.334613+0000","last_undegraded":"2024-09-19T08:38:18.334613+0000","last_fullsized":"2024-09-19T08:38:18.334613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:18.334613+0000","last_change":"2024-09-19T08:38:18.334613+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.334613+0000","last_undegraded":"2024-09-19T08:38:18.334613+0000","last_fullsized":"2024-09-19T08:38:18.334613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.349428+0000","last_change":"2024-09-19T08:38:18.349428+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.349428+0000","last_undegraded":"2024-09-19T08:38:18.349428+0000","last_fullsized":"2024-09-19T08:38:18.349428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:18.334613+0000","last_change":"2024-09-19T08:38:18.334613+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.334613+0000","last_undegraded":"2024-09-19T08:38:18.334613+0000","last_fullsized":"2024-09-19T08:38:18.334613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.349992+0000","last_change":"2024-09-19T08:38:18.349992+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.349992+0000","last_undegraded":"2024-09-19T08:38:18.349992+0000","last_fullsized":"2024-09-19T08:38:18.349992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.368660+0000","last_change":"2024-09-19T08:38:18.368660+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.368660+0000","last_undegraded":"2024-09-19T08:38:18.368660+0000","last_fullsized":"2024-09-19T08:38:18.368660+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.353743+0000","last_change":"2024-09-19T08:38:18.353743+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.353743+0000","last_undegraded":"2024-09-19T08:38:18.353743+0000","last_fullsized":"2024-09-19T08:38:18.353743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:18.334613+0000","last_change":"2024-09-19T08:38:18.334613+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.334613+0000","last_undegraded":"2024-09-19T08:38:18.334613+0000","last_fullsized":"2024-09-19T08:38:18.334613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.351625+0000","last_change":"2024-09-19T08:38:18.351625+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.351625+0000","last_undegraded":"2024-09-19T08:38:18.351625+0000","last_fullsized":"2024-09-19T08:38:18.351625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-19T08:38:17.346646+0000","last_change":"2024-09-19T08:38:17.346646+0000","last_active":"2024-09-19T08:38:17.331127+0000","last_peered":"2024-09-19T08:38:17.331127+0000","last_clean":"2024-09-19T08:38:17.331127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:17.346646+0000","last_undegraded":"2024-09-19T08:38:17.346646+0000","last_fullsized":"2024-09-19T08:38:17.346646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.348441+0000","last_change":"2024-09-19T08:38:18.348441+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.348441+0000","last_undegraded":"2024-09-19T08:38:18.348441+0000","last_fullsized":"2024-09-19T08:38:18.348441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:17.331127+0000","last_change":"2024-09-19T08:38:17.331127+0000","last_active":"2024-09-19T08:38:17.331127+0000","last_peered":"2024-09-19T08:38:17.331127+0000","last_clean":"2024-09-19T08:38:17.331127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:17.331127+0000","last_undegraded":"2024-09-19T08:38:17.331127+0000","last_fullsized":"2024-09-19T08:38:17.331127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-19T08:38:17.346118+0000","last_change":"2024-09-19T08:38:17.346118+0000","last_active":"2024-09-19T08:38:17.331127+0000","last_peered":"2024-09-19T08:38:17.331127+0000","last_clean":"2024-09-19T08:38:17.331127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:17.346118+0000","last_undegraded":"2024-09-19T08:38:17.346118+0000","last_fullsized":"2024-09-19T08:38:17.346118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:18.334613+0000","last_change":"2024-09-19T08:38:18.334613+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.334613+0000","last_undegraded":"2024-09-19T08:38:18.334613+0000","last_fullsized":"2024-09-19T08:38:18.334613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.398624+0000","last_change":"2024-09-19T08:38:18.393405+0000","last_active":"2024-09-19T08:38:18.398624+0000","last_peered":"2024-09-19T08:38:18.398624+0000","last_clean":"2024-09-19T08:38:18.398624+0000","last_became_active":"2024-09-19T08:38:18.392613+0000","last_became_peered":"2024-09-19T08:38:18.392613+0000","last_unstale":"2024-09-19T08:38:18.398624+0000","last_undegraded":"2024-09-19T08:38:18.398624+0000","last_fullsized":"2024-09-19T08:38:18.398624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:30.341445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.350268+0000","last_change":"2024-09-19T08:38:18.350268+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.350268+0000","last_undegraded":"2024-09-19T08:38:18.350268+0000","last_fullsized":"2024-09-19T08:38:18.350268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:17.331127+0000","last_change":"2024-09-19T08:38:17.331127+0000","last_active":"2024-09-19T08:38:17.331127+0000","last_peered":"2024-09-19T08:38:17.331127+0000","last_clean":"2024-09-19T08:38:17.331127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:17.331127+0000","last_undegraded":"2024-09-19T08:38:17.331127+0000","last_fullsized":"2024-09-19T08:38:17.331127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.365694+0000","last_change":"2024-09-19T08:38:18.365694+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.365694+0000","last_undegraded":"2024-09-19T08:38:18.365694+0000","last_fullsized":"2024-09-19T08:38:18.365694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.398568+0000","last_change":"2024-09-19T08:38:18.393568+0000","last_active":"2024-09-19T08:38:18.398568+0000","last_peered":"2024-09-19T08:38:18.398568+0000","last_clean":"2024-09-19T08:38:18.398568+0000","last_became_active":"2024-09-19T08:38:18.390336+0000","last_became_peered":"2024-09-19T08:38:18.390336+0000","last_unstale":"2024-09-19T08:38:18.398568+0000","last_undegraded":"2024-09-19T08:38:18.398568+0000","last_fullsized":"2024-09-19T08:38:18.398568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:52.328114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.345935+0000","last_change":"2024-09-19T08:38:18.345935+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.345935+0000","last_undegraded":"2024-09-19T08:38:18.345935+0000","last_fullsized":"2024-09-19T08:38:18.345935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.394587+0000","last_change":"2024-09-19T08:38:18.390022+0000","last_active":"2024-09-19T08:38:18.394587+0000","last_peered":"2024-09-19T08:38:18.394587+0000","last_clean":"2024-09-19T08:38:18.394587+0000","last_became_active":"2024-09-19T08:38:18.389679+0000","last_became_peered":"2024-09-19T08:38:18.389679+0000","last_unstale":"2024-09-19T08:38:18.394587+0000","last_undegraded":"2024-09-19T08:38:18.394587+0000","last_fullsized":"2024-09-19T08:38:18.394587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:47.789309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.352042+0000","last_change":"2024-09-19T08:38:18.352042+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.352042+0000","last_undegraded":"2024-09-19T08:38:18.352042+0000","last_fullsized":"2024-09-19T08:38:18.352042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:17.331127+0000","last_change":"2024-09-19T08:38:17.331127+0000","last_active":"2024-09-19T08:38:17.331127+0000","last_peered":"2024-09-19T08:38:17.331127+0000","last_clean":"2024-09-19T08:38:17.331127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:17.331127+0000","last_undegraded":"2024-09-19T08:38:17.331127+0000","last_fullsized":"2024-09-19T08:38:17.331127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.367831+0000","last_change":"2024-09-19T08:38:18.367831+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.367831+0000","last_undegraded":"2024-09-19T08:38:18.367831+0000","last_fullsized":"2024-09-19T08:38:18.367831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.370392+0000","last_change":"2024-09-19T08:38:18.366653+0000","last_active":"2024-09-19T08:38:18.370392+0000","last_peered":"2024-09-19T08:38:18.370392+0000","last_clean":"2024-09-19T08:38:18.370392+0000","last_became_active":"2024-09-19T08:38:18.366387+0000","last_became_peered":"2024-09-19T08:38:18.366387+0000","last_unstale":"2024-09-19T08:38:18.370392+0000","last_undegraded":"2024-09-19T08:38:18.370392+0000","last_fullsized":"2024-09-19T08:38:18.370392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:17.554499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.345969+0000","last_change":"2024-09-19T08:38:18.345969+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.345969+0000","last_undegraded":"2024-09-19T08:38:18.345969+0000","last_fullsized":"2024-09-19T08:38:18.345969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.403987+0000","last_change":"2024-09-19T08:38:18.402871+0000","last_active":"2024-09-19T08:38:18.403987+0000","last_peered":"2024-09-19T08:38:18.403987+0000","last_clean":"2024-09-19T08:38:18.403987+0000","last_became_active":"2024-09-19T08:38:18.400331+0000","last_became_peered":"2024-09-19T08:38:18.400331+0000","last_unstale":"2024-09-19T08:38:18.403987+0000","last_undegraded":"2024-09-19T08:38:18.403987+0000","last_fullsized":"2024-09-19T08:38:18.403987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:34:58.036813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.360463+0000","last_change":"2024-09-19T08:38:18.360463+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.360463+0000","last_undegraded":"2024-09-19T08:38:18.360463+0000","last_fullsized":"2024-09-19T08:38:18.360463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:17.331127+0000","last_change":"2024-09-19T08:38:17.331127+0000","last_active":"2024-09-19T08:38:17.331127+0000","last_peered":"2024-09-19T08:38:17.331127+0000","last_clean":"2024-09-19T08:38:17.331127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:17.331127+0000","last_undegraded":"2024-09-19T08:38:17.331127+0000","last_fullsized":"2024-09-19T08:38:17.331127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:18.334613+0000","last_change":"2024-09-19T08:38:18.334613+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.334613+0000","last_undegraded":"2024-09-19T08:38:18.334613+0000","last_fullsized":"2024-09-19T08:38:18.334613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.365899+0000","last_change":"2024-09-19T08:38:18.360672+0000","last_active":"2024-09-19T08:38:18.365899+0000","last_peered":"2024-09-19T08:38:18.365899+0000","last_clean":"2024-09-19T08:38:18.365899+0000","last_became_active":"2024-09-19T08:38:18.360210+0000","last_became_peered":"2024-09-19T08:38:18.360210+0000","last_unstale":"2024-09-19T08:38:18.365899+0000","last_undegraded":"2024-09-19T08:38:18.365899+0000","last_fullsized":"2024-09-19T08:38:18.365899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:05.111193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.358717+0000","last_change":"2024-09-19T08:38:18.358717+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.358717+0000","last_undegraded":"2024-09-19T08:38:18.358717+0000","last_fullsized":"2024-09-19T08:38:18.358717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.380164+0000","last_change":"2024-09-19T08:38:18.377025+0000","last_active":"2024-09-19T08:38:18.380164+0000","last_peered":"2024-09-19T08:38:18.380164+0000","last_clean":"2024-09-19T08:38:18.380164+0000","last_became_active":"2024-09-19T08:38:18.372574+0000","last_became_peered":"2024-09-19T08:38:18.372574+0000","last_unstale":"2024-09-19T08:38:18.380164+0000","last_undegraded":"2024-09-19T08:38:18.380164+0000","last_fullsized":"2024-09-19T08:38:18.380164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:05.111193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.357541+0000","last_change":"2024-09-19T08:38:18.357541+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.357541+0000","last_undegraded":"2024-09-19T08:38:18.357541+0000","last_fullsized":"2024-09-19T08:38:18.357541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.368559+0000","last_change":"2024-09-19T08:38:18.366653+0000","last_active":"2024-09-19T08:38:18.368559+0000","last_peered":"2024-09-19T08:38:18.368559+0000","last_clean":"2024-09-19T08:38:18.368559+0000","last_became_active":"2024-09-19T08:38:18.366357+0000","last_became_peered":"2024-09-19T08:38:18.366357+0000","last_unstale":"2024-09-19T08:38:18.368559+0000","last_undegraded":"2024-09-19T08:38:18.368559+0000","last_fullsized":"2024-09-19T08:38:18.368559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:30.341445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.363646+0000","last_change":"2024-09-19T08:38:18.363646+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.363646+0000","last_undegraded":"2024-09-19T08:38:18.363646+0000","last_fullsized":"2024-09-19T08:38:18.363646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.367913+0000","last_change":"2024-09-19T08:38:18.366656+0000","last_active":"2024-09-19T08:38:18.367913+0000","last_peered":"2024-09-19T08:38:18.367913+0000","last_clean":"2024-09-19T08:38:18.367913+0000","last_became_active":"2024-09-19T08:38:18.366299+0000","last_became_peered":"2024-09-19T08:38:18.366299+0000","last_unstale":"2024-09-19T08:38:18.367913+0000","last_undegraded":"2024-09-19T08:38:18.367913+0000","last_fullsized":"2024-09-19T08:38:18.367913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:59.268711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.366406+0000","last_change":"2024-09-19T08:38:18.366406+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.366406+0000","last_undegraded":"2024-09-19T08:38:18.366406+0000","last_fullsized":"2024-09-19T08:38:18.366406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.399509+0000","last_change":"2024-09-19T08:38:18.394865+0000","last_active":"2024-09-19T08:38:18.399509+0000","last_peered":"2024-09-19T08:38:18.399509+0000","last_clean":"2024-09-19T08:38:18.399509+0000","last_became_active":"2024-09-19T08:38:18.394496+0000","last_became_peered":"2024-09-19T08:38:18.394496+0000","last_unstale":"2024-09-19T08:38:18.399509+0000","last_undegraded":"2024-09-19T08:38:18.399509+0000","last_fullsized":"2024-09-19T08:38:18.399509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:59.268711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.365122+0000","last_change":"2024-09-19T08:38:18.365122+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.365122+0000","last_undegraded":"2024-09-19T08:38:18.365122+0000","last_fullsized":"2024-09-19T08:38:18.365122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":166,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.375597+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:38:18.375597+0000","last_peered":"2024-09-19T08:38:18.375597+0000","last_clean":"2024-09-19T08:38:18.375597+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:38:18.375597+0000","last_undegraded":"2024-09-19T08:38:18.375597+0000","last_fullsized":"2024-09-19T08:38:18.375597+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.398411+0000","last_change":"2024-09-19T08:38:18.391179+0000","last_active":"2024-09-19T08:38:18.398411+0000","last_peered":"2024-09-19T08:38:18.398411+0000","last_clean":"2024-09-19T08:38:18.398411+0000","last_became_active":"2024-09-19T08:38:18.390393+0000","last_became_peered":"2024-09-19T08:38:18.390393+0000","last_unstale":"2024-09-19T08:38:18.398411+0000","last_undegraded":"2024-09-19T08:38:18.398411+0000","last_fullsized":"2024-09-19T08:38:18.398411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:05.111193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:18.334613+0000","last_change":"2024-09-19T08:38:18.334613+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.334613+0000","last_undegraded":"2024-09-19T08:38:18.334613+0000","last_fullsized":"2024-09-19T08:38:18.334613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.391428+0000","last_change":"2024-09-19T08:38:18.384759+0000","last_active":"2024-09-19T08:38:18.391428+0000","last_peered":"2024-09-19T08:38:18.391428+0000","last_clean":"2024-09-19T08:38:18.391428+0000","last_became_active":"2024-09-19T08:38:18.383974+0000","last_became_peered":"2024-09-19T08:38:18.383974+0000","last_unstale":"2024-09-19T08:38:18.391428+0000","last_undegraded":"2024-09-19T08:38:18.391428+0000","last_fullsized":"2024-09-19T08:38:18.391428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:52.328114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.345450+0000","last_change":"2024-09-19T08:38:18.345450+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.345450+0000","last_undegraded":"2024-09-19T08:38:18.345450+0000","last_fullsized":"2024-09-19T08:38:18.345450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.368575+0000","last_change":"2024-09-19T08:38:18.366746+0000","last_active":"2024-09-19T08:38:18.368575+0000","last_peered":"2024-09-19T08:38:18.368575+0000","last_clean":"2024-09-19T08:38:18.368575+0000","last_became_active":"2024-09-19T08:38:18.366346+0000","last_became_peered":"2024-09-19T08:38:18.366346+0000","last_unstale":"2024-09-19T08:38:18.368575+0000","last_undegraded":"2024-09-19T08:38:18.368575+0000","last_fullsized":"2024-09-19T08:38:18.368575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:52.328114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.346210+0000","last_change":"2024-09-19T08:38:18.346210+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.346210+0000","last_undegraded":"2024-09-19T08:38:18.346210+0000","last_fullsized":"2024-09-19T08:38:18.346210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.403231+0000","last_change":"2024-09-19T08:38:18.398888+0000","last_active":"2024-09-19T08:38:18.403231+0000","last_peered":"2024-09-19T08:38:18.403231+0000","last_clean":"2024-09-19T08:38:18.403231+0000","last_became_active":"2024-09-19T08:38:18.391135+0000","last_became_peered":"2024-09-19T08:38:18.391135+0000","last_unstale":"2024-09-19T08:38:18.403231+0000","last_undegraded":"2024-09-19T08:38:18.403231+0000","last_fullsized":"2024-09-19T08:38:18.403231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:59.268711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.353393+0000","last_change":"2024-09-19T08:38:18.353393+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.353393+0000","last_undegraded":"2024-09-19T08:38:18.353393+0000","last_fullsized":"2024-09-19T08:38:18.353393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.365863+0000","last_change":"2024-09-19T08:38:18.363154+0000","last_active":"2024-09-19T08:38:18.365863+0000","last_peered":"2024-09-19T08:38:18.365863+0000","last_clean":"2024-09-19T08:38:18.365863+0000","last_became_active":"2024-09-19T08:38:18.362876+0000","last_became_peered":"2024-09-19T08:38:18.362876+0000","last_unstale":"2024-09-19T08:38:18.365863+0000","last_undegraded":"2024-09-19T08:38:18.365863+0000","last_fullsized":"2024-09-19T08:38:18.365863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:52.328114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.365956+0000","last_change":"2024-09-19T08:38:18.365956+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.365956+0000","last_undegraded":"2024-09-19T08:38:18.365956+0000","last_fullsized":"2024-09-19T08:38:18.365956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.397433+0000","last_change":"2024-09-19T08:38:18.389212+0000","last_active":"2024-09-19T08:38:18.397433+0000","last_peered":"2024-09-19T08:38:18.397433+0000","last_clean":"2024-09-19T08:38:18.397433+0000","last_became_active":"2024-09-19T08:38:18.388588+0000","last_became_peered":"2024-09-19T08:38:18.388588+0000","last_unstale":"2024-09-19T08:38:18.397433+0000","last_undegraded":"2024-09-19T08:38:18.397433+0000","last_fullsized":"2024-09-19T08:38:18.397433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:55.536656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.350757+0000","last_change":"2024-09-19T08:38:18.350757+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.350757+0000","last_undegraded":"2024-09-19T08:38:18.350757+0000","last_fullsized":"2024-09-19T08:38:18.350757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.381033+0000","last_change":"2024-09-19T08:38:18.377131+0000","last_active":"2024-09-19T08:38:18.381033+0000","last_peered":"2024-09-19T08:38:18.381033+0000","last_clean":"2024-09-19T08:38:18.381033+0000","last_became_active":"2024-09-19T08:38:18.376860+0000","last_became_peered":"2024-09-19T08:38:18.376860+0000","last_unstale":"2024-09-19T08:38:18.381033+0000","last_undegraded":"2024-09-19T08:38:18.381033+0000","last_fullsized":"2024-09-19T08:38:18.381033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:34:58.036813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:18.334613+0000","last_change":"2024-09-19T08:38:18.334613+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.334613+0000","last_undegraded":"2024-09-19T08:38:18.334613+0000","last_fullsized":"2024-09-19T08:38:18.334613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:17.331127+0000","last_change":"2024-09-19T08:38:17.331127+0000","last_active":"2024-09-19T08:38:17.331127+0000","last_peered":"2024-09-19T08:38:17.331127+0000","last_clean":"2024-09-19T08:38:17.331127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:17.331127+0000","last_undegraded":"2024-09-19T08:38:17.331127+0000","last_fullsized":"2024-09-19T08:38:17.331127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.346457+0000","last_change":"2024-09-19T08:38:18.346457+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.346457+0000","last_undegraded":"2024-09-19T08:38:18.346457+0000","last_fullsized":"2024-09-19T08:38:18.346457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.391302+0000","last_change":"2024-09-19T08:38:18.384552+0000","last_active":"2024-09-19T08:38:18.391302+0000","last_peered":"2024-09-19T08:38:18.391302+0000","last_clean":"2024-09-19T08:38:18.391302+0000","last_became_active":"2024-09-19T08:38:18.383834+0000","last_became_peered":"2024-09-19T08:38:18.383834+0000","last_unstale":"2024-09-19T08:38:18.391302+0000","last_undegraded":"2024-09-19T08:38:18.391302+0000","last_fullsized":"2024-09-19T08:38:18.391302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:30.341445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.352521+0000","last_change":"2024-09-19T08:38:18.352521+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.352521+0000","last_undegraded":"2024-09-19T08:38:18.352521+0000","last_fullsized":"2024-09-19T08:38:18.352521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.383574+0000","last_change":"2024-09-19T08:38:18.378231+0000","last_active":"2024-09-19T08:38:18.383574+0000","last_peered":"2024-09-19T08:38:18.383574+0000","last_clean":"2024-09-19T08:38:18.383574+0000","last_became_active":"2024-09-19T08:38:18.377883+0000","last_became_peered":"2024-09-19T08:38:18.377883+0000","last_unstale":"2024-09-19T08:38:18.383574+0000","last_undegraded":"2024-09-19T08:38:18.383574+0000","last_fullsized":"2024-09-19T08:38:18.383574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:55.536656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:18.334613+0000","last_change":"2024-09-19T08:38:18.334613+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.334613+0000","last_undegraded":"2024-09-19T08:38:18.334613+0000","last_fullsized":"2024-09-19T08:38:18.334613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.364662+0000","last_change":"2024-09-19T08:38:18.359251+0000","last_active":"2024-09-19T08:38:18.364662+0000","last_peered":"2024-09-19T08:38:18.364662+0000","last_clean":"2024-09-19T08:38:18.364662+0000","last_became_active":"2024-09-19T08:38:18.358873+0000","last_became_peered":"2024-09-19T08:38:18.358873+0000","last_unstale":"2024-09-19T08:38:18.364662+0000","last_undegraded":"2024-09-19T08:38:18.364662+0000","last_fullsized":"2024-09-19T08:38:18.364662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:30.341445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:18.334613+0000","last_change":"2024-09-19T08:38:18.334613+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.334613+0000","last_undegraded":"2024-09-19T08:38:18.334613+0000","last_fullsized":"2024-09-19T08:38:18.334613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.397355+0000","last_change":"2024-09-19T08:38:18.387311+0000","last_active":"2024-09-19T08:38:18.397355+0000","last_peered":"2024-09-19T08:38:18.397355+0000","last_clean":"2024-09-19T08:38:18.397355+0000","last_became_active":"2024-09-19T08:38:18.386340+0000","last_became_peered":"2024-09-19T08:38:18.386340+0000","last_unstale":"2024-09-19T08:38:18.397355+0000","last_undegraded":"2024-09-19T08:38:18.397355+0000","last_fullsized":"2024-09-19T08:38:18.397355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:34:58.036813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:18.334613+0000","last_change":"2024-09-19T08:38:18.334613+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.334613+0000","last_undegraded":"2024-09-19T08:38:18.334613+0000","last_fullsized":"2024-09-19T08:38:18.334613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.372680+0000","last_change":"2024-09-19T08:38:18.372680+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.372680+0000","last_undegraded":"2024-09-19T08:38:18.372680+0000","last_fullsized":"2024-09-19T08:38:18.372680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.402413+0000","last_change":"2024-09-19T08:38:18.400304+0000","last_active":"2024-09-19T08:38:18.402413+0000","last_peered":"2024-09-19T08:38:18.402413+0000","last_clean":"2024-09-19T08:38:18.402413+0000","last_became_active":"2024-09-19T08:38:18.399609+0000","last_became_peered":"2024-09-19T08:38:18.399609+0000","last_unstale":"2024-09-19T08:38:18.402413+0000","last_undegraded":"2024-09-19T08:38:18.402413+0000","last_fullsized":"2024-09-19T08:38:18.402413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:47.789309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.373318+0000","last_change":"2024-09-19T08:38:18.373318+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.373318+0000","last_undegraded":"2024-09-19T08:38:18.373318+0000","last_fullsized":"2024-09-19T08:38:18.373318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.391342+0000","last_change":"2024-09-19T08:38:18.384861+0000","last_active":"2024-09-19T08:38:18.391342+0000","last_peered":"2024-09-19T08:38:18.391342+0000","last_clean":"2024-09-19T08:38:18.391342+0000","last_became_active":"2024-09-19T08:38:18.384070+0000","last_became_peered":"2024-09-19T08:38:18.384070+0000","last_unstale":"2024-09-19T08:38:18.391342+0000","last_undegraded":"2024-09-19T08:38:18.391342+0000","last_fullsized":"2024-09-19T08:38:18.391342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:59.268711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.347147+0000","last_change":"2024-09-19T08:38:18.347147+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.347147+0000","last_undegraded":"2024-09-19T08:38:18.347147+0000","last_fullsized":"2024-09-19T08:38:18.347147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-19T08:38:17.346199+0000","last_change":"2024-09-19T08:38:17.346199+0000","last_active":"2024-09-19T08:38:17.331127+0000","last_peered":"2024-09-19T08:38:17.331127+0000","last_clean":"2024-09-19T08:38:17.331127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:17.346199+0000","last_undegraded":"2024-09-19T08:38:17.346199+0000","last_fullsized":"2024-09-19T08:38:17.346199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:18.334613+0000","last_change":"2024-09-19T08:38:18.334613+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.334613+0000","last_undegraded":"2024-09-19T08:38:18.334613+0000","last_fullsized":"2024-09-19T08:38:18.334613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.386129+0000","last_change":"2024-09-19T08:38:18.380235+0000","last_active":"2024-09-19T08:38:18.386129+0000","last_peered":"2024-09-19T08:38:18.386129+0000","last_clean":"2024-09-19T08:38:18.386129+0000","last_became_active":"2024-09-19T08:38:18.380044+0000","last_became_peered":"2024-09-19T08:38:18.380044+0000","last_unstale":"2024-09-19T08:38:18.386129+0000","last_undegraded":"2024-09-19T08:38:18.386129+0000","last_fullsized":"2024-09-19T08:38:18.386129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:51.748538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":92,"acting":92,"num_store_stats":6},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":54,"acting":54,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":649760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":184320,"data_stored":59392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":184320,"data_stored":59392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509443,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":172032,"data_stored":53117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":172032,"data_stored":53117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509443,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":649760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27184,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":159744,"data_stored":47300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":172032,"data_stored":53117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:38:23.393 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:38:23.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.543+0000 7f18cf7ad640 1 Processor -- start 2024-09-19T08:38:23.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.547+0000 7f18cf7ad640 1 -- start start 2024-09-19T08:38:23.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.547+0000 7f18cf7ad640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18c80d4820 0x7f18c81621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:23.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.547+0000 7f18cf7ad640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f18c80b9970 0x7f18c80b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:23.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.547+0000 7f18cf7ad640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18c8068440 con 0x7f18c80b7f80 2024-09-19T08:38:23.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.547+0000 7f18cf7ad640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f18c8163050 con 0x7f18c80b9970 2024-09-19T08:38:23.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.547+0000 7f18cf7ad640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f18c80ba360 con 0x7f18c80d4820 2024-09-19T08:38:23.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.547+0000 7f18ccd48640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18c80d4820 0x7f18c81621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:23.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.547+0000 7f18ccd48640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18c80d4820 0x7f18c81621b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:55796/0 (socket says 172.21.15.98:55796) 2024-09-19T08:38:23.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.547+0000 7f18ccd48640 1 -- 172.21.15.98:0/2380487647 learned_addr learned my addr 172.21.15.98:0/2380487647 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:23.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.547+0000 7f18cdd4a640 1 --2- 172.21.15.98:0/2380487647 >> v2:172.21.15.139:3300/0 conn(0x7f18c80b9970 0x7f18c80b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:23.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.547+0000 7f18c7fff640 1 -- 172.21.15.98:0/2380487647 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3784983785 0 0) 0x7f18c8068440 con 0x7f18c80b7f80 2024-09-19T08:38:23.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.547+0000 7f18ccd48640 1 -- 172.21.15.98:0/2380487647 >> v2:172.21.15.139:3300/0 conn(0x7f18c80b9970 msgr2=0x7f18c80b9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:23.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.547+0000 7f18ccd48640 1 --2- 172.21.15.98:0/2380487647 >> v2:172.21.15.139:3300/0 conn(0x7f18c80b9970 0x7f18c80b9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:23.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.547+0000 7f18ccd48640 1 -- 172.21.15.98:0/2380487647 >> v1:172.21.15.139:6789/0 conn(0x7f18c80b7f80 legacy=0x7f18c815f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:23.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.547+0000 7f18ccd48640 1 -- 172.21.15.98:0/2380487647 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c8162a90 con 0x7f18c80d4820 2024-09-19T08:38:23.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.551+0000 7f18ccd48640 1 --2- 172.21.15.98:0/2380487647 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18c80d4820 0x7f18c81621b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f18bc009870 tx=0x7f18bc01c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7d4445e225a9683 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:23.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.551+0000 7f18c7fff640 1 -- 172.21.15.98:0/2380487647 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc0ab650 con 0x7f18c80d4820 2024-09-19T08:38:23.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.551+0000 7f18c7fff640 1 -- 172.21.15.98:0/2380487647 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18bc0b0070 con 0x7f18c80d4820 2024-09-19T08:38:23.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.551+0000 7f18c7fff640 1 -- 172.21.15.98:0/2380487647 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc0aba10 con 0x7f18c80d4820 2024-09-19T08:38:23.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.551+0000 7f18cf7ad640 1 -- 172.21.15.98:0/2380487647 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18c80d4820 msgr2=0x7f18c81621b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:23.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.551+0000 7f18cf7ad640 1 --2- 172.21.15.98:0/2380487647 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18c80d4820 0x7f18c81621b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f18bc009870 tx=0x7f18bc01c560 comp rx=0 tx=0).stop 2024-09-19T08:38:23.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.551+0000 7f18cf7ad640 1 -- 172.21.15.98:0/2380487647 shutdown_connections 2024-09-19T08:38:23.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.551+0000 7f18cf7ad640 1 --2- 172.21.15.98:0/2380487647 >> v2:172.21.15.139:3300/0 conn(0x7f18c80b9970 0x7f18c80b9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:23.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.551+0000 7f18cf7ad640 1 --2- 172.21.15.98:0/2380487647 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18c80d4820 0x7f18c81621b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:23.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.551+0000 7f18cf7ad640 1 -- 172.21.15.98:0/2380487647 >> 172.21.15.98:0/2380487647 conn(0x7f18c8074e40 msgr2=0x7f18c8075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:23.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.551+0000 7f18cf7ad640 1 -- 172.21.15.98:0/2380487647 shutdown_connections 2024-09-19T08:38:23.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.551+0000 7f18cf7ad640 1 -- 172.21.15.98:0/2380487647 wait complete. 2024-09-19T08:38:23.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.551+0000 7f18cf7ad640 1 Processor -- start 2024-09-19T08:38:23.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.551+0000 7f18cf7ad640 1 -- start start 2024-09-19T08:38:23.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.551+0000 7f18cf7ad640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18c80b7f80 0x7f18c8140bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:23.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.551+0000 7f18cf7ad640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18c80b9970 0x7f18c8158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:23.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.551+0000 7f18cf7ad640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18c80d4820 0x7f18c8152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:23.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.551+0000 7f18cf7ad640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f18c80b84e0 con 0x7f18c80b9970 2024-09-19T08:38:23.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.551+0000 7f18cf7ad640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f18c80b8360 con 0x7f18c80b7f80 2024-09-19T08:38:23.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.551+0000 7f18cf7ad640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f18c8144590 con 0x7f18c80d4820 2024-09-19T08:38:23.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.551+0000 7f18ccd48640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18c80b9970 0x7f18c8158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:23.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.551+0000 7f18ccd48640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18c80b9970 0x7f18c8158900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33396/0 (socket says 172.21.15.98:33396) 2024-09-19T08:38:23.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.551+0000 7f18ccd48640 1 -- 172.21.15.98:0/4207134345 learned_addr learned my addr 172.21.15.98:0/4207134345 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:23.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.551+0000 7f18cdd4a640 1 --2- 172.21.15.98:0/4207134345 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18c80d4820 0x7f18c8152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:23.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.551+0000 7f18cd549640 1 --2- 172.21.15.98:0/4207134345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18c80b7f80 0x7f18c8140bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:23.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.551+0000 7f18ccd48640 1 -- 172.21.15.98:0/4207134345 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18c80d4820 msgr2=0x7f18c8152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:23.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.551+0000 7f18ccd48640 1 --2- 172.21.15.98:0/4207134345 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18c80d4820 0x7f18c8152870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:23.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.551+0000 7f18ccd48640 1 -- 172.21.15.98:0/4207134345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18c80b7f80 msgr2=0x7f18c8140bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:23.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.551+0000 7f18ccd48640 1 --2- 172.21.15.98:0/4207134345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18c80b7f80 0x7f18c8140bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:23.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.551+0000 7f18ccd48640 1 -- 172.21.15.98:0/4207134345 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c8069b80 con 0x7f18c80b9970 2024-09-19T08:38:23.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.551+0000 7f18ccd48640 1 --2- 172.21.15.98:0/4207134345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18c80b9970 0x7f18c8158900 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f18bc01a190 tx=0x7f18bc01ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:23.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.551+0000 7f18c5ffb640 1 -- 172.21.15.98:0/4207134345 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc0ba070 con 0x7f18c80b9970 2024-09-19T08:38:23.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.551+0000 7f18c5ffb640 1 -- 172.21.15.98:0/4207134345 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18bc004220 con 0x7f18c80b9970 2024-09-19T08:38:23.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.551+0000 7f18cf7ad640 1 -- 172.21.15.98:0/4207134345 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18c8156dc0 con 0x7f18c80b9970 2024-09-19T08:38:23.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.551+0000 7f18c5ffb640 1 -- 172.21.15.98:0/4207134345 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc0ab950 con 0x7f18c80b9970 2024-09-19T08:38:23.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.551+0000 7f18cf7ad640 1 -- 172.21.15.98:0/4207134345 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18c8157080 con 0x7f18c80b9970 2024-09-19T08:38:23.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.555+0000 7f18cf7ad640 1 -- 172.21.15.98:0/4207134345 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18c81621b0 con 0x7f18c80b9970 2024-09-19T08:38:23.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.555+0000 7f18c5ffb640 1 -- 172.21.15.98:0/4207134345 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f18bc0abaf0 con 0x7f18c80b9970 2024-09-19T08:38:23.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.555+0000 7f18c5ffb640 1 --2- 172.21.15.98:0/4207134345 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f18b40678c0 0x7f18b4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:23.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.555+0000 7f18c5ffb640 1 -- 172.21.15.98:0/4207134345 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f18bc11fe90 con 0x7f18c80b9970 2024-09-19T08:38:23.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.555+0000 7f18cd549640 1 --2- 172.21.15.98:0/4207134345 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f18b40678c0 0x7f18b4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:23.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.555+0000 7f18cd549640 1 --2- 172.21.15.98:0/4207134345 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f18b40678c0 0x7f18b4069d80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f18c0000f70 tx=0x7f18c0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:23.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.559+0000 7f18c5ffb640 1 -- 172.21.15.98:0/4207134345 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f18bc016150 con 0x7f18c80b9970 2024-09-19T08:38:23.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.703+0000 7f18cf7ad640 1 -- 172.21.15.98:0/4207134345 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f18c80d3d00 con 0x7f18b40678c0 2024-09-19T08:38:23.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.719+0000 7f18c5ffb640 1 -- 172.21.15.98:0/4207134345 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+244523 (secure 0 0 0) 0x7f18c80d3d00 con 0x7f18b40678c0 2024-09-19T08:38:23.723 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:38:23.724 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:38:23.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.723+0000 7f18cf7ad640 1 -- 172.21.15.98:0/4207134345 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f18b40678c0 msgr2=0x7f18b4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:23.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.723+0000 7f18cf7ad640 1 --2- 172.21.15.98:0/4207134345 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f18b40678c0 0x7f18b4069d80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f18c0000f70 tx=0x7f18c0009040 comp rx=0 tx=0).stop 2024-09-19T08:38:23.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.723+0000 7f18cf7ad640 1 -- 172.21.15.98:0/4207134345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18c80b9970 msgr2=0x7f18c8158900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:23.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.723+0000 7f18cf7ad640 1 --2- 172.21.15.98:0/4207134345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18c80b9970 0x7f18c8158900 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f18bc01a190 tx=0x7f18bc01ada0 comp rx=0 tx=0).stop 2024-09-19T08:38:23.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.723+0000 7f18cf7ad640 1 -- 172.21.15.98:0/4207134345 shutdown_connections 2024-09-19T08:38:23.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.723+0000 7f18cf7ad640 1 --2- 172.21.15.98:0/4207134345 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f18b40678c0 0x7f18b4069d80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:23.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.723+0000 7f18cf7ad640 1 --2- 172.21.15.98:0/4207134345 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18c80d4820 0x7f18c8152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:23.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.723+0000 7f18cf7ad640 1 --2- 172.21.15.98:0/4207134345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18c80b9970 0x7f18c8158900 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:23.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.723+0000 7f18cf7ad640 1 --2- 172.21.15.98:0/4207134345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18c80b7f80 0x7f18c8140bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:23.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.723+0000 7f18cf7ad640 1 -- 172.21.15.98:0/4207134345 >> 172.21.15.98:0/4207134345 conn(0x7f18c8074e40 msgr2=0x7f18c8067a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:23.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.723+0000 7f18cf7ad640 1 -- 172.21.15.98:0/4207134345 shutdown_connections 2024-09-19T08:38:23.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.723+0000 7f18cf7ad640 1 -- 172.21.15.98:0/4207134345 wait complete. 2024-09-19T08:38:23.740 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-19T08:38:22.027036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":148,"acting":148,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":38,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218772,"kb_used_data":2540,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755948,"statfs":{"total":773094113280,"available":772870090752,"internally_reserved":0,"allocated":2600960,"data_stored":1624955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.691138"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.368603+0000","last_change":"2024-09-19T08:38:18.368603+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.368603+0000","last_undegraded":"2024-09-19T08:38:18.368603+0000","last_fullsized":"2024-09-19T08:38:18.368603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.349394+0000","last_change":"2024-09-19T08:38:18.349394+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.349394+0000","last_undegraded":"2024-09-19T08:38:18.349394+0000","last_fullsized":"2024-09-19T08:38:18.349394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.352974+0000","last_change":"2024-09-19T08:38:18.352974+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.352974+0000","last_undegraded":"2024-09-19T08:38:18.352974+0000","last_fullsized":"2024-09-19T08:38:18.352974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.389392+0000","last_change":"2024-09-19T08:38:18.389392+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.389392+0000","last_undegraded":"2024-09-19T08:38:18.389392+0000","last_fullsized":"2024-09-19T08:38:18.389392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.361760+0000","last_change":"2024-09-19T08:38:18.361760+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.361760+0000","last_undegraded":"2024-09-19T08:38:18.361760+0000","last_fullsized":"2024-09-19T08:38:18.361760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.389879+0000","last_change":"2024-09-19T08:38:18.389879+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.389879+0000","last_undegraded":"2024-09-19T08:38:18.389879+0000","last_fullsized":"2024-09-19T08:38:18.389879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.363729+0000","last_change":"2024-09-19T08:38:18.363729+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.363729+0000","last_undegraded":"2024-09-19T08:38:18.363729+0000","last_fullsized":"2024-09-19T08:38:18.363729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:18.334613+0000","last_change":"2024-09-19T08:38:18.334613+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.334613+0000","last_undegraded":"2024-09-19T08:38:18.334613+0000","last_fullsized":"2024-09-19T08:38:18.334613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.366761+0000","last_change":"2024-09-19T08:38:18.366761+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.366761+0000","last_undegraded":"2024-09-19T08:38:18.366761+0000","last_fullsized":"2024-09-19T08:38:18.366761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:18.334613+0000","last_change":"2024-09-19T08:38:18.334613+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.334613+0000","last_undegraded":"2024-09-19T08:38:18.334613+0000","last_fullsized":"2024-09-19T08:38:18.334613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:18.334613+0000","last_change":"2024-09-19T08:38:18.334613+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.334613+0000","last_undegraded":"2024-09-19T08:38:18.334613+0000","last_fullsized":"2024-09-19T08:38:18.334613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.355429+0000","last_change":"2024-09-19T08:38:18.355429+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.355429+0000","last_undegraded":"2024-09-19T08:38:18.355429+0000","last_fullsized":"2024-09-19T08:38:18.355429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:18.334613+0000","last_change":"2024-09-19T08:38:18.334613+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.334613+0000","last_undegraded":"2024-09-19T08:38:18.334613+0000","last_fullsized":"2024-09-19T08:38:18.334613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.350914+0000","last_change":"2024-09-19T08:38:18.350914+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.350914+0000","last_undegraded":"2024-09-19T08:38:18.350914+0000","last_fullsized":"2024-09-19T08:38:18.350914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.349265+0000","last_change":"2024-09-19T08:38:18.349265+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.349265+0000","last_undegraded":"2024-09-19T08:38:18.349265+0000","last_fullsized":"2024-09-19T08:38:18.349265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.368264+0000","last_change":"2024-09-19T08:38:18.368264+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.368264+0000","last_undegraded":"2024-09-19T08:38:18.368264+0000","last_fullsized":"2024-09-19T08:38:18.368264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.366483+0000","last_change":"2024-09-19T08:38:18.366483+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.366483+0000","last_undegraded":"2024-09-19T08:38:18.366483+0000","last_fullsized":"2024-09-19T08:38:18.366483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.354376+0000","last_change":"2024-09-19T08:38:18.354376+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.354376+0000","last_undegraded":"2024-09-19T08:38:18.354376+0000","last_fullsized":"2024-09-19T08:38:18.354376+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.356210+0000","last_change":"2024-09-19T08:38:18.356210+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.356210+0000","last_undegraded":"2024-09-19T08:38:18.356210+0000","last_fullsized":"2024-09-19T08:38:18.356210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:18.334613+0000","last_change":"2024-09-19T08:38:18.334613+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.334613+0000","last_undegraded":"2024-09-19T08:38:18.334613+0000","last_fullsized":"2024-09-19T08:38:18.334613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.351092+0000","last_change":"2024-09-19T08:38:18.351092+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.351092+0000","last_undegraded":"2024-09-19T08:38:18.351092+0000","last_fullsized":"2024-09-19T08:38:18.351092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.371212+0000","last_change":"2024-09-19T08:38:18.371212+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.371212+0000","last_undegraded":"2024-09-19T08:38:18.371212+0000","last_fullsized":"2024-09-19T08:38:18.371212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:18.334613+0000","last_change":"2024-09-19T08:38:18.334613+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.334613+0000","last_undegraded":"2024-09-19T08:38:18.334613+0000","last_fullsized":"2024-09-19T08:38:18.334613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.363746+0000","last_change":"2024-09-19T08:38:18.363746+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.363746+0000","last_undegraded":"2024-09-19T08:38:18.363746+0000","last_fullsized":"2024-09-19T08:38:18.363746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:18.334613+0000","last_change":"2024-09-19T08:38:18.334613+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.334613+0000","last_undegraded":"2024-09-19T08:38:18.334613+0000","last_fullsized":"2024-09-19T08:38:18.334613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:18.334613+0000","last_change":"2024-09-19T08:38:18.334613+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.334613+0000","last_undegraded":"2024-09-19T08:38:18.334613+0000","last_fullsized":"2024-09-19T08:38:18.334613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.349428+0000","last_change":"2024-09-19T08:38:18.349428+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.349428+0000","last_undegraded":"2024-09-19T08:38:18.349428+0000","last_fullsized":"2024-09-19T08:38:18.349428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:18.334613+0000","last_change":"2024-09-19T08:38:18.334613+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.334613+0000","last_undegraded":"2024-09-19T08:38:18.334613+0000","last_fullsized":"2024-09-19T08:38:18.334613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.349992+0000","last_change":"2024-09-19T08:38:18.349992+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.349992+0000","last_undegraded":"2024-09-19T08:38:18.349992+0000","last_fullsized":"2024-09-19T08:38:18.349992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.368660+0000","last_change":"2024-09-19T08:38:18.368660+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.368660+0000","last_undegraded":"2024-09-19T08:38:18.368660+0000","last_fullsized":"2024-09-19T08:38:18.368660+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.353743+0000","last_change":"2024-09-19T08:38:18.353743+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.353743+0000","last_undegraded":"2024-09-19T08:38:18.353743+0000","last_fullsized":"2024-09-19T08:38:18.353743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:18.334613+0000","last_change":"2024-09-19T08:38:18.334613+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.334613+0000","last_undegraded":"2024-09-19T08:38:18.334613+0000","last_fullsized":"2024-09-19T08:38:18.334613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.351625+0000","last_change":"2024-09-19T08:38:18.351625+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.351625+0000","last_undegraded":"2024-09-19T08:38:18.351625+0000","last_fullsized":"2024-09-19T08:38:18.351625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-19T08:38:17.346646+0000","last_change":"2024-09-19T08:38:17.346646+0000","last_active":"2024-09-19T08:38:17.331127+0000","last_peered":"2024-09-19T08:38:17.331127+0000","last_clean":"2024-09-19T08:38:17.331127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:17.346646+0000","last_undegraded":"2024-09-19T08:38:17.346646+0000","last_fullsized":"2024-09-19T08:38:17.346646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.348441+0000","last_change":"2024-09-19T08:38:18.348441+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.348441+0000","last_undegraded":"2024-09-19T08:38:18.348441+0000","last_fullsized":"2024-09-19T08:38:18.348441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:17.331127+0000","last_change":"2024-09-19T08:38:17.331127+0000","last_active":"2024-09-19T08:38:17.331127+0000","last_peered":"2024-09-19T08:38:17.331127+0000","last_clean":"2024-09-19T08:38:17.331127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:17.331127+0000","last_undegraded":"2024-09-19T08:38:17.331127+0000","last_fullsized":"2024-09-19T08:38:17.331127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-19T08:38:17.346118+0000","last_change":"2024-09-19T08:38:17.346118+0000","last_active":"2024-09-19T08:38:17.331127+0000","last_peered":"2024-09-19T08:38:17.331127+0000","last_clean":"2024-09-19T08:38:17.331127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:17.346118+0000","last_undegraded":"2024-09-19T08:38:17.346118+0000","last_fullsized":"2024-09-19T08:38:17.346118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:18.334613+0000","last_change":"2024-09-19T08:38:18.334613+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.334613+0000","last_undegraded":"2024-09-19T08:38:18.334613+0000","last_fullsized":"2024-09-19T08:38:18.334613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.398624+0000","last_change":"2024-09-19T08:38:18.393405+0000","last_active":"2024-09-19T08:38:18.398624+0000","last_peered":"2024-09-19T08:38:18.398624+0000","last_clean":"2024-09-19T08:38:18.398624+0000","last_became_active":"2024-09-19T08:38:18.392613+0000","last_became_peered":"2024-09-19T08:38:18.392613+0000","last_unstale":"2024-09-19T08:38:18.398624+0000","last_undegraded":"2024-09-19T08:38:18.398624+0000","last_fullsized":"2024-09-19T08:38:18.398624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:30.341445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.350268+0000","last_change":"2024-09-19T08:38:18.350268+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.350268+0000","last_undegraded":"2024-09-19T08:38:18.350268+0000","last_fullsized":"2024-09-19T08:38:18.350268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:17.331127+0000","last_change":"2024-09-19T08:38:17.331127+0000","last_active":"2024-09-19T08:38:17.331127+0000","last_peered":"2024-09-19T08:38:17.331127+0000","last_clean":"2024-09-19T08:38:17.331127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:17.331127+0000","last_undegraded":"2024-09-19T08:38:17.331127+0000","last_fullsized":"2024-09-19T08:38:17.331127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.365694+0000","last_change":"2024-09-19T08:38:18.365694+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.365694+0000","last_undegraded":"2024-09-19T08:38:18.365694+0000","last_fullsized":"2024-09-19T08:38:18.365694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.398568+0000","last_change":"2024-09-19T08:38:18.393568+0000","last_active":"2024-09-19T08:38:18.398568+0000","last_peered":"2024-09-19T08:38:18.398568+0000","last_clean":"2024-09-19T08:38:18.398568+0000","last_became_active":"2024-09-19T08:38:18.390336+0000","last_became_peered":"2024-09-19T08:38:18.390336+0000","last_unstale":"2024-09-19T08:38:18.398568+0000","last_undegraded":"2024-09-19T08:38:18.398568+0000","last_fullsized":"2024-09-19T08:38:18.398568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:52.328114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.345935+0000","last_change":"2024-09-19T08:38:18.345935+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.345935+0000","last_undegraded":"2024-09-19T08:38:18.345935+0000","last_fullsized":"2024-09-19T08:38:18.345935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.394587+0000","last_change":"2024-09-19T08:38:18.390022+0000","last_active":"2024-09-19T08:38:18.394587+0000","last_peered":"2024-09-19T08:38:18.394587+0000","last_clean":"2024-09-19T08:38:18.394587+0000","last_became_active":"2024-09-19T08:38:18.389679+0000","last_became_peered":"2024-09-19T08:38:18.389679+0000","last_unstale":"2024-09-19T08:38:18.394587+0000","last_undegraded":"2024-09-19T08:38:18.394587+0000","last_fullsized":"2024-09-19T08:38:18.394587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:47.789309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.352042+0000","last_change":"2024-09-19T08:38:18.352042+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.352042+0000","last_undegraded":"2024-09-19T08:38:18.352042+0000","last_fullsized":"2024-09-19T08:38:18.352042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:17.331127+0000","last_change":"2024-09-19T08:38:17.331127+0000","last_active":"2024-09-19T08:38:17.331127+0000","last_peered":"2024-09-19T08:38:17.331127+0000","last_clean":"2024-09-19T08:38:17.331127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:17.331127+0000","last_undegraded":"2024-09-19T08:38:17.331127+0000","last_fullsized":"2024-09-19T08:38:17.331127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.367831+0000","last_change":"2024-09-19T08:38:18.367831+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.367831+0000","last_undegraded":"2024-09-19T08:38:18.367831+0000","last_fullsized":"2024-09-19T08:38:18.367831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.370392+0000","last_change":"2024-09-19T08:38:18.366653+0000","last_active":"2024-09-19T08:38:18.370392+0000","last_peered":"2024-09-19T08:38:18.370392+0000","last_clean":"2024-09-19T08:38:18.370392+0000","last_became_active":"2024-09-19T08:38:18.366387+0000","last_became_peered":"2024-09-19T08:38:18.366387+0000","last_unstale":"2024-09-19T08:38:18.370392+0000","last_undegraded":"2024-09-19T08:38:18.370392+0000","last_fullsized":"2024-09-19T08:38:18.370392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:17.554499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.345969+0000","last_change":"2024-09-19T08:38:18.345969+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.345969+0000","last_undegraded":"2024-09-19T08:38:18.345969+0000","last_fullsized":"2024-09-19T08:38:18.345969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.403987+0000","last_change":"2024-09-19T08:38:18.402871+0000","last_active":"2024-09-19T08:38:18.403987+0000","last_peered":"2024-09-19T08:38:18.403987+0000","last_clean":"2024-09-19T08:38:18.403987+0000","last_became_active":"2024-09-19T08:38:18.400331+0000","last_became_peered":"2024-09-19T08:38:18.400331+0000","last_unstale":"2024-09-19T08:38:18.403987+0000","last_undegraded":"2024-09-19T08:38:18.403987+0000","last_fullsized":"2024-09-19T08:38:18.403987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:34:58.036813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.360463+0000","last_change":"2024-09-19T08:38:18.360463+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.360463+0000","last_undegraded":"2024-09-19T08:38:18.360463+0000","last_fullsized":"2024-09-19T08:38:18.360463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:17.331127+0000","last_change":"2024-09-19T08:38:17.331127+0000","last_active":"2024-09-19T08:38:17.331127+0000","last_peered":"2024-09-19T08:38:17.331127+0000","last_clean":"2024-09-19T08:38:17.331127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:17.331127+0000","last_undegraded":"2024-09-19T08:38:17.331127+0000","last_fullsized":"2024-09-19T08:38:17.331127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:18.334613+0000","last_change":"2024-09-19T08:38:18.334613+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.334613+0000","last_undegraded":"2024-09-19T08:38:18.334613+0000","last_fullsized":"2024-09-19T08:38:18.334613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.365899+0000","last_change":"2024-09-19T08:38:18.360672+0000","last_active":"2024-09-19T08:38:18.365899+0000","last_peered":"2024-09-19T08:38:18.365899+0000","last_clean":"2024-09-19T08:38:18.365899+0000","last_became_active":"2024-09-19T08:38:18.360210+0000","last_became_peered":"2024-09-19T08:38:18.360210+0000","last_unstale":"2024-09-19T08:38:18.365899+0000","last_undegraded":"2024-09-19T08:38:18.365899+0000","last_fullsized":"2024-09-19T08:38:18.365899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:05.111193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.358717+0000","last_change":"2024-09-19T08:38:18.358717+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.358717+0000","last_undegraded":"2024-09-19T08:38:18.358717+0000","last_fullsized":"2024-09-19T08:38:18.358717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.380164+0000","last_change":"2024-09-19T08:38:18.377025+0000","last_active":"2024-09-19T08:38:18.380164+0000","last_peered":"2024-09-19T08:38:18.380164+0000","last_clean":"2024-09-19T08:38:18.380164+0000","last_became_active":"2024-09-19T08:38:18.372574+0000","last_became_peered":"2024-09-19T08:38:18.372574+0000","last_unstale":"2024-09-19T08:38:18.380164+0000","last_undegraded":"2024-09-19T08:38:18.380164+0000","last_fullsized":"2024-09-19T08:38:18.380164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:05.111193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.357541+0000","last_change":"2024-09-19T08:38:18.357541+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.357541+0000","last_undegraded":"2024-09-19T08:38:18.357541+0000","last_fullsized":"2024-09-19T08:38:18.357541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.368559+0000","last_change":"2024-09-19T08:38:18.366653+0000","last_active":"2024-09-19T08:38:18.368559+0000","last_peered":"2024-09-19T08:38:18.368559+0000","last_clean":"2024-09-19T08:38:18.368559+0000","last_became_active":"2024-09-19T08:38:18.366357+0000","last_became_peered":"2024-09-19T08:38:18.366357+0000","last_unstale":"2024-09-19T08:38:18.368559+0000","last_undegraded":"2024-09-19T08:38:18.368559+0000","last_fullsized":"2024-09-19T08:38:18.368559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:30.341445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.363646+0000","last_change":"2024-09-19T08:38:18.363646+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.363646+0000","last_undegraded":"2024-09-19T08:38:18.363646+0000","last_fullsized":"2024-09-19T08:38:18.363646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.367913+0000","last_change":"2024-09-19T08:38:18.366656+0000","last_active":"2024-09-19T08:38:18.367913+0000","last_peered":"2024-09-19T08:38:18.367913+0000","last_clean":"2024-09-19T08:38:18.367913+0000","last_became_active":"2024-09-19T08:38:18.366299+0000","last_became_peered":"2024-09-19T08:38:18.366299+0000","last_unstale":"2024-09-19T08:38:18.367913+0000","last_undegraded":"2024-09-19T08:38:18.367913+0000","last_fullsized":"2024-09-19T08:38:18.367913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:59.268711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.366406+0000","last_change":"2024-09-19T08:38:18.366406+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.366406+0000","last_undegraded":"2024-09-19T08:38:18.366406+0000","last_fullsized":"2024-09-19T08:38:18.366406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.399509+0000","last_change":"2024-09-19T08:38:18.394865+0000","last_active":"2024-09-19T08:38:18.399509+0000","last_peered":"2024-09-19T08:38:18.399509+0000","last_clean":"2024-09-19T08:38:18.399509+0000","last_became_active":"2024-09-19T08:38:18.394496+0000","last_became_peered":"2024-09-19T08:38:18.394496+0000","last_unstale":"2024-09-19T08:38:18.399509+0000","last_undegraded":"2024-09-19T08:38:18.399509+0000","last_fullsized":"2024-09-19T08:38:18.399509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:59.268711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.365122+0000","last_change":"2024-09-19T08:38:18.365122+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.365122+0000","last_undegraded":"2024-09-19T08:38:18.365122+0000","last_fullsized":"2024-09-19T08:38:18.365122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":166,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.375597+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:38:18.375597+0000","last_peered":"2024-09-19T08:38:18.375597+0000","last_clean":"2024-09-19T08:38:18.375597+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:38:18.375597+0000","last_undegraded":"2024-09-19T08:38:18.375597+0000","last_fullsized":"2024-09-19T08:38:18.375597+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.398411+0000","last_change":"2024-09-19T08:38:18.391179+0000","last_active":"2024-09-19T08:38:18.398411+0000","last_peered":"2024-09-19T08:38:18.398411+0000","last_clean":"2024-09-19T08:38:18.398411+0000","last_became_active":"2024-09-19T08:38:18.390393+0000","last_became_peered":"2024-09-19T08:38:18.390393+0000","last_unstale":"2024-09-19T08:38:18.398411+0000","last_undegraded":"2024-09-19T08:38:18.398411+0000","last_fullsized":"2024-09-19T08:38:18.398411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:05.111193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:18.334613+0000","last_change":"2024-09-19T08:38:18.334613+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.334613+0000","last_undegraded":"2024-09-19T08:38:18.334613+0000","last_fullsized":"2024-09-19T08:38:18.334613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.391428+0000","last_change":"2024-09-19T08:38:18.384759+0000","last_active":"2024-09-19T08:38:18.391428+0000","last_peered":"2024-09-19T08:38:18.391428+0000","last_clean":"2024-09-19T08:38:18.391428+0000","last_became_active":"2024-09-19T08:38:18.383974+0000","last_became_peered":"2024-09-19T08:38:18.383974+0000","last_unstale":"2024-09-19T08:38:18.391428+0000","last_undegraded":"2024-09-19T08:38:18.391428+0000","last_fullsized":"2024-09-19T08:38:18.391428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:52.328114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.345450+0000","last_change":"2024-09-19T08:38:18.345450+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.345450+0000","last_undegraded":"2024-09-19T08:38:18.345450+0000","last_fullsized":"2024-09-19T08:38:18.345450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.368575+0000","last_change":"2024-09-19T08:38:18.366746+0000","last_active":"2024-09-19T08:38:18.368575+0000","last_peered":"2024-09-19T08:38:18.368575+0000","last_clean":"2024-09-19T08:38:18.368575+0000","last_became_active":"2024-09-19T08:38:18.366346+0000","last_became_peered":"2024-09-19T08:38:18.366346+0000","last_unstale":"2024-09-19T08:38:18.368575+0000","last_undegraded":"2024-09-19T08:38:18.368575+0000","last_fullsized":"2024-09-19T08:38:18.368575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:52.328114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.346210+0000","last_change":"2024-09-19T08:38:18.346210+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.346210+0000","last_undegraded":"2024-09-19T08:38:18.346210+0000","last_fullsized":"2024-09-19T08:38:18.346210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.403231+0000","last_change":"2024-09-19T08:38:18.398888+0000","last_active":"2024-09-19T08:38:18.403231+0000","last_peered":"2024-09-19T08:38:18.403231+0000","last_clean":"2024-09-19T08:38:18.403231+0000","last_became_active":"2024-09-19T08:38:18.391135+0000","last_became_peered":"2024-09-19T08:38:18.391135+0000","last_unstale":"2024-09-19T08:38:18.403231+0000","last_undegraded":"2024-09-19T08:38:18.403231+0000","last_fullsized":"2024-09-19T08:38:18.403231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:59.268711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.353393+0000","last_change":"2024-09-19T08:38:18.353393+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.353393+0000","last_undegraded":"2024-09-19T08:38:18.353393+0000","last_fullsized":"2024-09-19T08:38:18.353393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.365863+0000","last_change":"2024-09-19T08:38:18.363154+0000","last_active":"2024-09-19T08:38:18.365863+0000","last_peered":"2024-09-19T08:38:18.365863+0000","last_clean":"2024-09-19T08:38:18.365863+0000","last_became_active":"2024-09-19T08:38:18.362876+0000","last_became_peered":"2024-09-19T08:38:18.362876+0000","last_unstale":"2024-09-19T08:38:18.365863+0000","last_undegraded":"2024-09-19T08:38:18.365863+0000","last_fullsized":"2024-09-19T08:38:18.365863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:52.328114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.365956+0000","last_change":"2024-09-19T08:38:18.365956+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.365956+0000","last_undegraded":"2024-09-19T08:38:18.365956+0000","last_fullsized":"2024-09-19T08:38:18.365956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.397433+0000","last_change":"2024-09-19T08:38:18.389212+0000","last_active":"2024-09-19T08:38:18.397433+0000","last_peered":"2024-09-19T08:38:18.397433+0000","last_clean":"2024-09-19T08:38:18.397433+0000","last_became_active":"2024-09-19T08:38:18.388588+0000","last_became_peered":"2024-09-19T08:38:18.388588+0000","last_unstale":"2024-09-19T08:38:18.397433+0000","last_undegraded":"2024-09-19T08:38:18.397433+0000","last_fullsized":"2024-09-19T08:38:18.397433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:55.536656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.350757+0000","last_change":"2024-09-19T08:38:18.350757+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.350757+0000","last_undegraded":"2024-09-19T08:38:18.350757+0000","last_fullsized":"2024-09-19T08:38:18.350757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.381033+0000","last_change":"2024-09-19T08:38:18.377131+0000","last_active":"2024-09-19T08:38:18.381033+0000","last_peered":"2024-09-19T08:38:18.381033+0000","last_clean":"2024-09-19T08:38:18.381033+0000","last_became_active":"2024-09-19T08:38:18.376860+0000","last_became_peered":"2024-09-19T08:38:18.376860+0000","last_unstale":"2024-09-19T08:38:18.381033+0000","last_undegraded":"2024-09-19T08:38:18.381033+0000","last_fullsized":"2024-09-19T08:38:18.381033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:34:58.036813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:18.334613+0000","last_change":"2024-09-19T08:38:18.334613+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.334613+0000","last_undegraded":"2024-09-19T08:38:18.334613+0000","last_fullsized":"2024-09-19T08:38:18.334613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:17.331127+0000","last_change":"2024-09-19T08:38:17.331127+0000","last_active":"2024-09-19T08:38:17.331127+0000","last_peered":"2024-09-19T08:38:17.331127+0000","last_clean":"2024-09-19T08:38:17.331127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:17.331127+0000","last_undegraded":"2024-09-19T08:38:17.331127+0000","last_fullsized":"2024-09-19T08:38:17.331127+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.346457+0000","last_change":"2024-09-19T08:38:18.346457+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.346457+0000","last_undegraded":"2024-09-19T08:38:18.346457+0000","last_fullsized":"2024-09-19T08:38:18.346457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.391302+0000","last_change":"2024-09-19T08:38:18.384552+0000","last_active":"2024-09-19T08:38:18.391302+0000","last_peered":"2024-09-19T08:38:18.391302+0000","last_clean":"2024-09-19T08:38:18.391302+0000","last_became_active":"2024-09-19T08:38:18.383834+0000","last_became_peered":"2024-09-19T08:38:18.383834+0000","last_unstale":"2024-09-19T08:38:18.391302+0000","last_undegraded":"2024-09-19T08:38:18.391302+0000","last_fullsized":"2024-09-19T08:38:18.391302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:30.341445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.352521+0000","last_change":"2024-09-19T08:38:18.352521+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.352521+0000","last_undegraded":"2024-09-19T08:38:18.352521+0000","last_fullsized":"2024-09-19T08:38:18.352521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.383574+0000","last_change":"2024-09-19T08:38:18.378231+0000","last_active":"2024-09-19T08:38:18.383574+0000","last_peered":"2024-09-19T08:38:18.383574+0000","last_clean":"2024-09-19T08:38:18.383574+0000","last_became_active":"2024-09-19T08:38:18.377883+0000","last_became_peered":"2024-09-19T08:38:18.377883+0000","last_unstale":"2024-09-19T08:38:18.383574+0000","last_undegraded":"2024-09-19T08:38:18.383574+0000","last_fullsized":"2024-09-19T08:38:18.383574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:55.536656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:18.334613+0000","last_change":"2024-09-19T08:38:18.334613+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.334613+0000","last_undegraded":"2024-09-19T08:38:18.334613+0000","last_fullsized":"2024-09-19T08:38:18.334613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.364662+0000","last_change":"2024-09-19T08:38:18.359251+0000","last_active":"2024-09-19T08:38:18.364662+0000","last_peered":"2024-09-19T08:38:18.364662+0000","last_clean":"2024-09-19T08:38:18.364662+0000","last_became_active":"2024-09-19T08:38:18.358873+0000","last_became_peered":"2024-09-19T08:38:18.358873+0000","last_unstale":"2024-09-19T08:38:18.364662+0000","last_undegraded":"2024-09-19T08:38:18.364662+0000","last_fullsized":"2024-09-19T08:38:18.364662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:30.341445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:18.334613+0000","last_change":"2024-09-19T08:38:18.334613+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.334613+0000","last_undegraded":"2024-09-19T08:38:18.334613+0000","last_fullsized":"2024-09-19T08:38:18.334613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.397355+0000","last_change":"2024-09-19T08:38:18.387311+0000","last_active":"2024-09-19T08:38:18.397355+0000","last_peered":"2024-09-19T08:38:18.397355+0000","last_clean":"2024-09-19T08:38:18.397355+0000","last_became_active":"2024-09-19T08:38:18.386340+0000","last_became_peered":"2024-09-19T08:38:18.386340+0000","last_unstale":"2024-09-19T08:38:18.397355+0000","last_undegraded":"2024-09-19T08:38:18.397355+0000","last_fullsized":"2024-09-19T08:38:18.397355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:34:58.036813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:18.334613+0000","last_change":"2024-09-19T08:38:18.334613+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.334613+0000","last_undegraded":"2024-09-19T08:38:18.334613+0000","last_fullsized":"2024-09-19T08:38:18.334613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.372680+0000","last_change":"2024-09-19T08:38:18.372680+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.372680+0000","last_undegraded":"2024-09-19T08:38:18.372680+0000","last_fullsized":"2024-09-19T08:38:18.372680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.402413+0000","last_change":"2024-09-19T08:38:18.400304+0000","last_active":"2024-09-19T08:38:18.402413+0000","last_peered":"2024-09-19T08:38:18.402413+0000","last_clean":"2024-09-19T08:38:18.402413+0000","last_became_active":"2024-09-19T08:38:18.399609+0000","last_became_peered":"2024-09-19T08:38:18.399609+0000","last_unstale":"2024-09-19T08:38:18.402413+0000","last_undegraded":"2024-09-19T08:38:18.402413+0000","last_fullsized":"2024-09-19T08:38:18.402413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:47.789309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.373318+0000","last_change":"2024-09-19T08:38:18.373318+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.373318+0000","last_undegraded":"2024-09-19T08:38:18.373318+0000","last_fullsized":"2024-09-19T08:38:18.373318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.391342+0000","last_change":"2024-09-19T08:38:18.384861+0000","last_active":"2024-09-19T08:38:18.391342+0000","last_peered":"2024-09-19T08:38:18.391342+0000","last_clean":"2024-09-19T08:38:18.391342+0000","last_became_active":"2024-09-19T08:38:18.384070+0000","last_became_peered":"2024-09-19T08:38:18.384070+0000","last_unstale":"2024-09-19T08:38:18.391342+0000","last_undegraded":"2024-09-19T08:38:18.391342+0000","last_fullsized":"2024-09-19T08:38:18.391342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:59.268711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.347147+0000","last_change":"2024-09-19T08:38:18.347147+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.347147+0000","last_undegraded":"2024-09-19T08:38:18.347147+0000","last_fullsized":"2024-09-19T08:38:18.347147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-19T08:38:17.346199+0000","last_change":"2024-09-19T08:38:17.346199+0000","last_active":"2024-09-19T08:38:17.331127+0000","last_peered":"2024-09-19T08:38:17.331127+0000","last_clean":"2024-09-19T08:38:17.331127+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:17.346199+0000","last_undegraded":"2024-09-19T08:38:17.346199+0000","last_fullsized":"2024-09-19T08:38:17.346199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:38:18.334613+0000","last_change":"2024-09-19T08:38:18.334613+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.334613+0000","last_undegraded":"2024-09-19T08:38:18.334613+0000","last_fullsized":"2024-09-19T08:38:18.334613+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.386129+0000","last_change":"2024-09-19T08:38:18.380235+0000","last_active":"2024-09-19T08:38:18.386129+0000","last_peered":"2024-09-19T08:38:18.386129+0000","last_clean":"2024-09-19T08:38:18.386129+0000","last_became_active":"2024-09-19T08:38:18.380044+0000","last_became_peered":"2024-09-19T08:38:18.380044+0000","last_unstale":"2024-09-19T08:38:18.386129+0000","last_undegraded":"2024-09-19T08:38:18.386129+0000","last_fullsized":"2024-09-19T08:38:18.386129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:51.748538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":92,"acting":92,"num_store_stats":6},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":54,"acting":54,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":649760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":184320,"data_stored":59392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":184320,"data_stored":59392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509443,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":172032,"data_stored":53117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":172032,"data_stored":53117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509443,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":649760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27184,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":159744,"data_stored":47300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":172032,"data_stored":53117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:38:23.744 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:38:23.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.895+0000 7f5539a36640 1 Processor -- start 2024-09-19T08:38:23.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.895+0000 7f5539a36640 1 -- start start 2024-09-19T08:38:23.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.899+0000 7f5539a36640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5534158e40 0x7f553406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:23.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.899+0000 7f5539a36640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f553415fcf0 con 0x7f55340ce8b0 2024-09-19T08:38:23.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.899+0000 7f5539a36640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f553415ef30 con 0x7f553406c1d0 2024-09-19T08:38:23.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.899+0000 7f5539a36640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f553406c640 con 0x7f5534158e40 2024-09-19T08:38:23.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.899+0000 7f5532ffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5534158e40 0x7f553406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:23.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.899+0000 7f55337fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f55340ce8b0 0x7f55340b3560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41470/0 (socket says 172.21.15.98:41470) 2024-09-19T08:38:23.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.899+0000 7f5532ffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5534158e40 0x7f553406bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33400/0 (socket says 172.21.15.98:33400) 2024-09-19T08:38:23.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.899+0000 7f55337fe640 1 -- 172.21.15.98:0/2737994279 learned_addr learned my addr 172.21.15.98:0/2737994279 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:23.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.899+0000 7f5532ffd640 1 -- 172.21.15.98:0/2737994279 >> v1:172.21.15.139:6789/0 conn(0x7f553406c1d0 legacy=0x7f55340ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:23.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.899+0000 7f5532ffd640 1 -- 172.21.15.98:0/2737994279 >> v1:172.21.15.98:6789/0 conn(0x7f55340ce8b0 legacy=0x7f55340b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:23.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.899+0000 7f5531ffb640 1 -- 172.21.15.98:0/2737994279 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4258348277 0 0) 0x7f553415fcf0 con 0x7f55340ce8b0 2024-09-19T08:38:23.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.899+0000 7f5532ffd640 1 -- 172.21.15.98:0/2737994279 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f553415f210 con 0x7f5534158e40 2024-09-19T08:38:23.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.899+0000 7f5532ffd640 1 --2- 172.21.15.98:0/2737994279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5534158e40 0x7f553406bc90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f55280097c0 tx=0x7f552801ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c76c66f61e2c65b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:23.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.903+0000 7f5531ffb640 1 -- 172.21.15.98:0/2737994279 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55280b5070 con 0x7f5534158e40 2024-09-19T08:38:23.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.903+0000 7f5531ffb640 1 -- 172.21.15.98:0/2737994279 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55280a8990 con 0x7f5534158e40 2024-09-19T08:38:23.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.903+0000 7f5531ffb640 1 -- 172.21.15.98:0/2737994279 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55280a8d60 con 0x7f5534158e40 2024-09-19T08:38:23.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.903+0000 7f5539a36640 1 -- 172.21.15.98:0/2737994279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5534158e40 msgr2=0x7f553406bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:23.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.903+0000 7f5539a36640 1 --2- 172.21.15.98:0/2737994279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5534158e40 0x7f553406bc90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f55280097c0 tx=0x7f552801ec90 comp rx=0 tx=0).stop 2024-09-19T08:38:23.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.903+0000 7f5539a36640 1 -- 172.21.15.98:0/2737994279 shutdown_connections 2024-09-19T08:38:23.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.903+0000 7f5539a36640 1 --2- 172.21.15.98:0/2737994279 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5534158e40 0x7f553406bc90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:23.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.903+0000 7f5539a36640 1 -- 172.21.15.98:0/2737994279 >> 172.21.15.98:0/2737994279 conn(0x7f55340b2010 msgr2=0x7f55340b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:23.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.903+0000 7f5539a36640 1 -- 172.21.15.98:0/2737994279 shutdown_connections 2024-09-19T08:38:23.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.903+0000 7f5539a36640 1 -- 172.21.15.98:0/2737994279 wait complete. 2024-09-19T08:38:23.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.903+0000 7f5539a36640 1 Processor -- start 2024-09-19T08:38:23.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.903+0000 7f5539a36640 1 -- start start 2024-09-19T08:38:23.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.903+0000 7f5539a36640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f553406c1d0 0x7f5534153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:23.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.903+0000 7f5539a36640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f55340ce8b0 0x7f553415c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:23.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.903+0000 7f5539a36640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5534158e40 0x7f553414de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:23.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.903+0000 7f5539a36640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5534159220 con 0x7f5534158e40 2024-09-19T08:38:23.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.903+0000 7f5539a36640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f553415f7d0 con 0x7f553406c1d0 2024-09-19T08:38:23.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.903+0000 7f5539a36640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f55340cee20 con 0x7f55340ce8b0 2024-09-19T08:38:23.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.903+0000 7f55337fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5534158e40 0x7f553414de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:23.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.903+0000 7f55337fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5534158e40 0x7f553414de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33412/0 (socket says 172.21.15.98:33412) 2024-09-19T08:38:23.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.903+0000 7f55337fe640 1 -- 172.21.15.98:0/3952963470 learned_addr learned my addr 172.21.15.98:0/3952963470 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:23.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.903+0000 7f5532ffd640 1 --2- 172.21.15.98:0/3952963470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f553406c1d0 0x7f5534153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:23.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.903+0000 7f55337fe640 1 -- 172.21.15.98:0/3952963470 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f55340ce8b0 msgr2=0x7f553415c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:23.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.903+0000 7f55337fe640 1 --2- 172.21.15.98:0/3952963470 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f55340ce8b0 0x7f553415c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:23.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.903+0000 7f55337fe640 1 -- 172.21.15.98:0/3952963470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f553406c1d0 msgr2=0x7f5534153be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:23.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.903+0000 7f55337fe640 1 --2- 172.21.15.98:0/3952963470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f553406c1d0 0x7f5534153be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:23.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.903+0000 7f55337fe640 1 -- 172.21.15.98:0/3952963470 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55341541c0 con 0x7f5534158e40 2024-09-19T08:38:23.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.903+0000 7f55337fe640 1 --2- 172.21.15.98:0/3952963470 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5534158e40 0x7f553414de20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f552c00c7d0 tx=0x7f552c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:23.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.903+0000 7f551bfff640 1 -- 172.21.15.98:0/3952963470 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c018070 con 0x7f5534158e40 2024-09-19T08:38:23.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.903+0000 7f551bfff640 1 -- 172.21.15.98:0/3952963470 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f552c00a640 con 0x7f5534158e40 2024-09-19T08:38:23.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.903+0000 7f5539a36640 1 -- 172.21.15.98:0/3952963470 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55341544a0 con 0x7f5534158e40 2024-09-19T08:38:23.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.903+0000 7f551bfff640 1 -- 172.21.15.98:0/3952963470 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c00aa70 con 0x7f5534158e40 2024-09-19T08:38:23.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.903+0000 7f5539a36640 1 -- 172.21.15.98:0/3952963470 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5534154a60 con 0x7f5534158e40 2024-09-19T08:38:23.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.907+0000 7f5539a36640 1 -- 172.21.15.98:0/3952963470 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f553414e360 con 0x7f5534158e40 2024-09-19T08:38:23.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.907+0000 7f551bfff640 1 -- 172.21.15.98:0/3952963470 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f552c01b070 con 0x7f5534158e40 2024-09-19T08:38:23.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.907+0000 7f551bfff640 1 --2- 172.21.15.98:0/3952963470 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f55200678c0 0x7f5520069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:23.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.907+0000 7f551bfff640 1 -- 172.21.15.98:0/3952963470 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f552c085220 con 0x7f5534158e40 2024-09-19T08:38:23.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.907+0000 7f5532ffd640 1 --2- 172.21.15.98:0/3952963470 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f55200678c0 0x7f5520069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:23.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.907+0000 7f5532ffd640 1 --2- 172.21.15.98:0/3952963470 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f55200678c0 0x7f5520069d80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f55280b2000 tx=0x7f55280aa070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:23.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:23.911+0000 7f551bfff640 1 -- 172.21.15.98:0/3952963470 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f552c055df0 con 0x7f5534158e40 2024-09-19T08:38:24.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.071+0000 7f5539a36640 1 -- 172.21.15.98:0/3952963470 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f55340b3d50 con 0x7f55200678c0 2024-09-19T08:38:24.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.083+0000 7f551bfff640 1 -- 172.21.15.98:0/3952963470 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248134 (secure 0 0 0) 0x7f55340b3d50 con 0x7f55200678c0 2024-09-19T08:38:24.090 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:38:24.092 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:38:24.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.087+0000 7f5539a36640 1 -- 172.21.15.98:0/3952963470 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f55200678c0 msgr2=0x7f5520069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:24.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.087+0000 7f5539a36640 1 --2- 172.21.15.98:0/3952963470 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f55200678c0 0x7f5520069d80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f55280b2000 tx=0x7f55280aa070 comp rx=0 tx=0).stop 2024-09-19T08:38:24.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.087+0000 7f5539a36640 1 -- 172.21.15.98:0/3952963470 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5534158e40 msgr2=0x7f553414de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:24.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.087+0000 7f5539a36640 1 --2- 172.21.15.98:0/3952963470 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5534158e40 0x7f553414de20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f552c00c7d0 tx=0x7f552c00cc90 comp rx=0 tx=0).stop 2024-09-19T08:38:24.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.087+0000 7f5539a36640 1 -- 172.21.15.98:0/3952963470 shutdown_connections 2024-09-19T08:38:24.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.087+0000 7f5539a36640 1 --2- 172.21.15.98:0/3952963470 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f55200678c0 0x7f5520069d80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:24.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.087+0000 7f5539a36640 1 --2- 172.21.15.98:0/3952963470 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5534158e40 0x7f553414de20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:24.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.087+0000 7f5539a36640 1 --2- 172.21.15.98:0/3952963470 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f55340ce8b0 0x7f553415c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:24.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.087+0000 7f5539a36640 1 --2- 172.21.15.98:0/3952963470 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f553406c1d0 0x7f5534153be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:24.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.087+0000 7f5539a36640 1 -- 172.21.15.98:0/3952963470 >> 172.21.15.98:0/3952963470 conn(0x7f55340b2010 msgr2=0x7f55340cc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:24.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.087+0000 7f5539a36640 1 -- 172.21.15.98:0/3952963470 shutdown_connections 2024-09-19T08:38:24.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.087+0000 7f5539a36640 1 -- 172.21.15.98:0/3952963470 wait complete. 2024-09-19T08:38:24.105 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-19T08:38:24.027662+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591980,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":168,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218888,"kb_used_data":2648,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755832,"statfs":{"total":773094113280,"available":772869971968,"internally_reserved":0,"allocated":2711552,"data_stored":1672696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001937"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.368603+0000","last_change":"2024-09-19T08:38:18.368603+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.368603+0000","last_undegraded":"2024-09-19T08:38:18.368603+0000","last_fullsized":"2024-09-19T08:38:18.368603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.349394+0000","last_change":"2024-09-19T08:38:18.349394+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.349394+0000","last_undegraded":"2024-09-19T08:38:18.349394+0000","last_fullsized":"2024-09-19T08:38:18.349394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.352974+0000","last_change":"2024-09-19T08:38:18.352974+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.352974+0000","last_undegraded":"2024-09-19T08:38:18.352974+0000","last_fullsized":"2024-09-19T08:38:18.352974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.389392+0000","last_change":"2024-09-19T08:38:18.389392+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.389392+0000","last_undegraded":"2024-09-19T08:38:18.389392+0000","last_fullsized":"2024-09-19T08:38:18.389392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.361760+0000","last_change":"2024-09-19T08:38:18.361760+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.361760+0000","last_undegraded":"2024-09-19T08:38:18.361760+0000","last_fullsized":"2024-09-19T08:38:18.361760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.389879+0000","last_change":"2024-09-19T08:38:18.389879+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.389879+0000","last_undegraded":"2024-09-19T08:38:18.389879+0000","last_fullsized":"2024-09-19T08:38:18.389879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.363729+0000","last_change":"2024-09-19T08:38:18.363729+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.363729+0000","last_undegraded":"2024-09-19T08:38:18.363729+0000","last_fullsized":"2024-09-19T08:38:18.363729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.084591+0000","last_change":"2024-09-19T08:38:19.400917+0000","last_active":"2024-09-19T08:38:22.084591+0000","last_peered":"2024-09-19T08:38:22.084591+0000","last_clean":"2024-09-19T08:38:22.084591+0000","last_became_active":"2024-09-19T08:38:19.400188+0000","last_became_peered":"2024-09-19T08:38:19.400188+0000","last_unstale":"2024-09-19T08:38:22.084591+0000","last_undegraded":"2024-09-19T08:38:22.084591+0000","last_fullsized":"2024-09-19T08:38:22.084591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:24:38.033969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.366761+0000","last_change":"2024-09-19T08:38:18.366761+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.366761+0000","last_undegraded":"2024-09-19T08:38:18.366761+0000","last_fullsized":"2024-09-19T08:38:18.366761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.425359+0000","last_change":"2024-09-19T08:38:19.402509+0000","last_active":"2024-09-19T08:38:20.425359+0000","last_peered":"2024-09-19T08:38:20.425359+0000","last_clean":"2024-09-19T08:38:20.425359+0000","last_became_active":"2024-09-19T08:38:19.401833+0000","last_became_peered":"2024-09-19T08:38:19.401833+0000","last_unstale":"2024-09-19T08:38:20.425359+0000","last_undegraded":"2024-09-19T08:38:20.425359+0000","last_fullsized":"2024-09-19T08:38:20.425359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:08.065282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424901+0000","last_change":"2024-09-19T08:38:19.430286+0000","last_active":"2024-09-19T08:38:20.424901+0000","last_peered":"2024-09-19T08:38:20.424901+0000","last_clean":"2024-09-19T08:38:20.424901+0000","last_became_active":"2024-09-19T08:38:19.429901+0000","last_became_peered":"2024-09-19T08:38:19.429901+0000","last_unstale":"2024-09-19T08:38:20.424901+0000","last_undegraded":"2024-09-19T08:38:20.424901+0000","last_fullsized":"2024-09-19T08:38:20.424901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:31:42.999935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.355429+0000","last_change":"2024-09-19T08:38:18.355429+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.355429+0000","last_undegraded":"2024-09-19T08:38:18.355429+0000","last_fullsized":"2024-09-19T08:38:18.355429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.086556+0000","last_change":"2024-09-19T08:38:19.396759+0000","last_active":"2024-09-19T08:38:22.086556+0000","last_peered":"2024-09-19T08:38:22.086556+0000","last_clean":"2024-09-19T08:38:22.086556+0000","last_became_active":"2024-09-19T08:38:19.395976+0000","last_became_peered":"2024-09-19T08:38:19.395976+0000","last_unstale":"2024-09-19T08:38:22.086556+0000","last_undegraded":"2024-09-19T08:38:22.086556+0000","last_fullsized":"2024-09-19T08:38:22.086556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:20.116184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.350914+0000","last_change":"2024-09-19T08:38:18.350914+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.350914+0000","last_undegraded":"2024-09-19T08:38:18.350914+0000","last_fullsized":"2024-09-19T08:38:18.350914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.349265+0000","last_change":"2024-09-19T08:38:18.349265+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.349265+0000","last_undegraded":"2024-09-19T08:38:18.349265+0000","last_fullsized":"2024-09-19T08:38:18.349265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.368264+0000","last_change":"2024-09-19T08:38:18.368264+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.368264+0000","last_undegraded":"2024-09-19T08:38:18.368264+0000","last_fullsized":"2024-09-19T08:38:18.368264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.366483+0000","last_change":"2024-09-19T08:38:18.366483+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.366483+0000","last_undegraded":"2024-09-19T08:38:18.366483+0000","last_fullsized":"2024-09-19T08:38:18.366483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.354376+0000","last_change":"2024-09-19T08:38:18.354376+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.354376+0000","last_undegraded":"2024-09-19T08:38:18.354376+0000","last_fullsized":"2024-09-19T08:38:18.354376+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.356210+0000","last_change":"2024-09-19T08:38:18.356210+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.356210+0000","last_undegraded":"2024-09-19T08:38:18.356210+0000","last_fullsized":"2024-09-19T08:38:18.356210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.425280+0000","last_change":"2024-09-19T08:38:19.389509+0000","last_active":"2024-09-19T08:38:20.425280+0000","last_peered":"2024-09-19T08:38:20.425280+0000","last_clean":"2024-09-19T08:38:20.425280+0000","last_became_active":"2024-09-19T08:38:19.388736+0000","last_became_peered":"2024-09-19T08:38:19.388736+0000","last_unstale":"2024-09-19T08:38:20.425280+0000","last_undegraded":"2024-09-19T08:38:20.425280+0000","last_fullsized":"2024-09-19T08:38:20.425280+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:20:35.891069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.351092+0000","last_change":"2024-09-19T08:38:18.351092+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.351092+0000","last_undegraded":"2024-09-19T08:38:18.351092+0000","last_fullsized":"2024-09-19T08:38:18.351092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.371212+0000","last_change":"2024-09-19T08:38:18.371212+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.371212+0000","last_undegraded":"2024-09-19T08:38:18.371212+0000","last_fullsized":"2024-09-19T08:38:18.371212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.084821+0000","last_change":"2024-09-19T08:38:19.406260+0000","last_active":"2024-09-19T08:38:22.084821+0000","last_peered":"2024-09-19T08:38:22.084821+0000","last_clean":"2024-09-19T08:38:22.084821+0000","last_became_active":"2024-09-19T08:38:19.405769+0000","last_became_peered":"2024-09-19T08:38:19.405769+0000","last_unstale":"2024-09-19T08:38:22.084821+0000","last_undegraded":"2024-09-19T08:38:22.084821+0000","last_fullsized":"2024-09-19T08:38:22.084821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:31:42.999935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.363746+0000","last_change":"2024-09-19T08:38:18.363746+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.363746+0000","last_undegraded":"2024-09-19T08:38:18.363746+0000","last_fullsized":"2024-09-19T08:38:18.363746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085580+0000","last_change":"2024-09-19T08:38:19.406201+0000","last_active":"2024-09-19T08:38:22.085580+0000","last_peered":"2024-09-19T08:38:22.085580+0000","last_clean":"2024-09-19T08:38:22.085580+0000","last_became_active":"2024-09-19T08:38:19.405583+0000","last_became_peered":"2024-09-19T08:38:19.405583+0000","last_unstale":"2024-09-19T08:38:22.085580+0000","last_undegraded":"2024-09-19T08:38:22.085580+0000","last_fullsized":"2024-09-19T08:38:22.085580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:08.065282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.425374+0000","last_change":"2024-09-19T08:38:19.411842+0000","last_active":"2024-09-19T08:38:20.425374+0000","last_peered":"2024-09-19T08:38:20.425374+0000","last_clean":"2024-09-19T08:38:20.425374+0000","last_became_active":"2024-09-19T08:38:19.411092+0000","last_became_peered":"2024-09-19T08:38:19.411092+0000","last_unstale":"2024-09-19T08:38:20.425374+0000","last_undegraded":"2024-09-19T08:38:20.425374+0000","last_fullsized":"2024-09-19T08:38:20.425374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:39:22.441525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.349428+0000","last_change":"2024-09-19T08:38:18.349428+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.349428+0000","last_undegraded":"2024-09-19T08:38:18.349428+0000","last_fullsized":"2024-09-19T08:38:18.349428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085836+0000","last_change":"2024-09-19T08:38:19.408371+0000","last_active":"2024-09-19T08:38:22.085836+0000","last_peered":"2024-09-19T08:38:22.085836+0000","last_clean":"2024-09-19T08:38:22.085836+0000","last_became_active":"2024-09-19T08:38:19.407846+0000","last_became_peered":"2024-09-19T08:38:19.407846+0000","last_unstale":"2024-09-19T08:38:22.085836+0000","last_undegraded":"2024-09-19T08:38:22.085836+0000","last_fullsized":"2024-09-19T08:38:22.085836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:33.753905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.349992+0000","last_change":"2024-09-19T08:38:18.349992+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.349992+0000","last_undegraded":"2024-09-19T08:38:18.349992+0000","last_fullsized":"2024-09-19T08:38:18.349992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.368660+0000","last_change":"2024-09-19T08:38:18.368660+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.368660+0000","last_undegraded":"2024-09-19T08:38:18.368660+0000","last_fullsized":"2024-09-19T08:38:18.368660+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.353743+0000","last_change":"2024-09-19T08:38:18.353743+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.353743+0000","last_undegraded":"2024-09-19T08:38:18.353743+0000","last_fullsized":"2024-09-19T08:38:18.353743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424603+0000","last_change":"2024-09-19T08:38:19.414469+0000","last_active":"2024-09-19T08:38:20.424603+0000","last_peered":"2024-09-19T08:38:20.424603+0000","last_clean":"2024-09-19T08:38:20.424603+0000","last_became_active":"2024-09-19T08:38:19.413995+0000","last_became_peered":"2024-09-19T08:38:19.413995+0000","last_unstale":"2024-09-19T08:38:20.424603+0000","last_undegraded":"2024-09-19T08:38:20.424603+0000","last_fullsized":"2024-09-19T08:38:20.424603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:24:38.033969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.351625+0000","last_change":"2024-09-19T08:38:18.351625+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.351625+0000","last_undegraded":"2024-09-19T08:38:18.351625+0000","last_fullsized":"2024-09-19T08:38:18.351625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085745+0000","last_change":"2024-09-19T08:38:18.387502+0000","last_active":"2024-09-19T08:38:22.085745+0000","last_peered":"2024-09-19T08:38:22.085745+0000","last_clean":"2024-09-19T08:38:22.085745+0000","last_became_active":"2024-09-19T08:38:18.387154+0000","last_became_peered":"2024-09-19T08:38:18.387154+0000","last_unstale":"2024-09-19T08:38:22.085745+0000","last_undegraded":"2024-09-19T08:38:22.085745+0000","last_fullsized":"2024-09-19T08:38:22.085745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:53.708142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.348441+0000","last_change":"2024-09-19T08:38:18.348441+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.348441+0000","last_undegraded":"2024-09-19T08:38:18.348441+0000","last_fullsized":"2024-09-19T08:38:18.348441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424590+0000","last_change":"2024-09-19T08:38:18.381446+0000","last_active":"2024-09-19T08:38:20.424590+0000","last_peered":"2024-09-19T08:38:20.424590+0000","last_clean":"2024-09-19T08:38:20.424590+0000","last_became_active":"2024-09-19T08:38:18.380678+0000","last_became_peered":"2024-09-19T08:38:18.380678+0000","last_unstale":"2024-09-19T08:38:20.424590+0000","last_undegraded":"2024-09-19T08:38:20.424590+0000","last_fullsized":"2024-09-19T08:38:20.424590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:07:31.456618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.084969+0000","last_change":"2024-09-19T08:38:18.376428+0000","last_active":"2024-09-19T08:38:22.084969+0000","last_peered":"2024-09-19T08:38:22.084969+0000","last_clean":"2024-09-19T08:38:22.084969+0000","last_became_active":"2024-09-19T08:38:18.376015+0000","last_became_peered":"2024-09-19T08:38:18.376015+0000","last_unstale":"2024-09-19T08:38:22.084969+0000","last_undegraded":"2024-09-19T08:38:22.084969+0000","last_fullsized":"2024-09-19T08:38:22.084969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:49:33.943029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.084815+0000","last_change":"2024-09-19T08:38:19.414261+0000","last_active":"2024-09-19T08:38:22.084815+0000","last_peered":"2024-09-19T08:38:22.084815+0000","last_clean":"2024-09-19T08:38:22.084815+0000","last_became_active":"2024-09-19T08:38:19.413930+0000","last_became_peered":"2024-09-19T08:38:19.413930+0000","last_unstale":"2024-09-19T08:38:22.084815+0000","last_undegraded":"2024-09-19T08:38:22.084815+0000","last_fullsized":"2024-09-19T08:38:22.084815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:32.216306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.398624+0000","last_change":"2024-09-19T08:38:18.393405+0000","last_active":"2024-09-19T08:38:18.398624+0000","last_peered":"2024-09-19T08:38:18.398624+0000","last_clean":"2024-09-19T08:38:18.398624+0000","last_became_active":"2024-09-19T08:38:18.392613+0000","last_became_peered":"2024-09-19T08:38:18.392613+0000","last_unstale":"2024-09-19T08:38:18.398624+0000","last_undegraded":"2024-09-19T08:38:18.398624+0000","last_fullsized":"2024-09-19T08:38:18.398624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:30.341445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.350268+0000","last_change":"2024-09-19T08:38:18.350268+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.350268+0000","last_undegraded":"2024-09-19T08:38:18.350268+0000","last_fullsized":"2024-09-19T08:38:18.350268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424870+0000","last_change":"2024-09-19T08:38:18.398231+0000","last_active":"2024-09-19T08:38:20.424870+0000","last_peered":"2024-09-19T08:38:20.424870+0000","last_clean":"2024-09-19T08:38:20.424870+0000","last_became_active":"2024-09-19T08:38:18.397590+0000","last_became_peered":"2024-09-19T08:38:18.397590+0000","last_unstale":"2024-09-19T08:38:20.424870+0000","last_undegraded":"2024-09-19T08:38:20.424870+0000","last_fullsized":"2024-09-19T08:38:20.424870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:54:02.262454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.365694+0000","last_change":"2024-09-19T08:38:18.365694+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.365694+0000","last_undegraded":"2024-09-19T08:38:18.365694+0000","last_fullsized":"2024-09-19T08:38:18.365694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.398568+0000","last_change":"2024-09-19T08:38:18.393568+0000","last_active":"2024-09-19T08:38:18.398568+0000","last_peered":"2024-09-19T08:38:18.398568+0000","last_clean":"2024-09-19T08:38:18.398568+0000","last_became_active":"2024-09-19T08:38:18.390336+0000","last_became_peered":"2024-09-19T08:38:18.390336+0000","last_unstale":"2024-09-19T08:38:18.398568+0000","last_undegraded":"2024-09-19T08:38:18.398568+0000","last_fullsized":"2024-09-19T08:38:18.398568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:52.328114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.345935+0000","last_change":"2024-09-19T08:38:18.345935+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.345935+0000","last_undegraded":"2024-09-19T08:38:18.345935+0000","last_fullsized":"2024-09-19T08:38:18.345935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.394587+0000","last_change":"2024-09-19T08:38:18.390022+0000","last_active":"2024-09-19T08:38:18.394587+0000","last_peered":"2024-09-19T08:38:18.394587+0000","last_clean":"2024-09-19T08:38:18.394587+0000","last_became_active":"2024-09-19T08:38:18.389679+0000","last_became_peered":"2024-09-19T08:38:18.389679+0000","last_unstale":"2024-09-19T08:38:18.394587+0000","last_undegraded":"2024-09-19T08:38:18.394587+0000","last_fullsized":"2024-09-19T08:38:18.394587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:47.789309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.352042+0000","last_change":"2024-09-19T08:38:18.352042+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.352042+0000","last_undegraded":"2024-09-19T08:38:18.352042+0000","last_fullsized":"2024-09-19T08:38:18.352042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424913+0000","last_change":"2024-09-19T08:38:18.371315+0000","last_active":"2024-09-19T08:38:20.424913+0000","last_peered":"2024-09-19T08:38:20.424913+0000","last_clean":"2024-09-19T08:38:20.424913+0000","last_became_active":"2024-09-19T08:38:18.370524+0000","last_became_peered":"2024-09-19T08:38:18.370524+0000","last_unstale":"2024-09-19T08:38:20.424913+0000","last_undegraded":"2024-09-19T08:38:20.424913+0000","last_fullsized":"2024-09-19T08:38:20.424913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:49:33.943029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.367831+0000","last_change":"2024-09-19T08:38:18.367831+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.367831+0000","last_undegraded":"2024-09-19T08:38:18.367831+0000","last_fullsized":"2024-09-19T08:38:18.367831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.370392+0000","last_change":"2024-09-19T08:38:18.366653+0000","last_active":"2024-09-19T08:38:18.370392+0000","last_peered":"2024-09-19T08:38:18.370392+0000","last_clean":"2024-09-19T08:38:18.370392+0000","last_became_active":"2024-09-19T08:38:18.366387+0000","last_became_peered":"2024-09-19T08:38:18.366387+0000","last_unstale":"2024-09-19T08:38:18.370392+0000","last_undegraded":"2024-09-19T08:38:18.370392+0000","last_fullsized":"2024-09-19T08:38:18.370392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:17.554499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.345969+0000","last_change":"2024-09-19T08:38:18.345969+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.345969+0000","last_undegraded":"2024-09-19T08:38:18.345969+0000","last_fullsized":"2024-09-19T08:38:18.345969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.403987+0000","last_change":"2024-09-19T08:38:18.402871+0000","last_active":"2024-09-19T08:38:18.403987+0000","last_peered":"2024-09-19T08:38:18.403987+0000","last_clean":"2024-09-19T08:38:18.403987+0000","last_became_active":"2024-09-19T08:38:18.400331+0000","last_became_peered":"2024-09-19T08:38:18.400331+0000","last_unstale":"2024-09-19T08:38:18.403987+0000","last_undegraded":"2024-09-19T08:38:18.403987+0000","last_fullsized":"2024-09-19T08:38:18.403987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:34:58.036813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.360463+0000","last_change":"2024-09-19T08:38:18.360463+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.360463+0000","last_undegraded":"2024-09-19T08:38:18.360463+0000","last_fullsized":"2024-09-19T08:38:18.360463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424461+0000","last_change":"2024-09-19T08:38:18.396642+0000","last_active":"2024-09-19T08:38:20.424461+0000","last_peered":"2024-09-19T08:38:20.424461+0000","last_clean":"2024-09-19T08:38:20.424461+0000","last_became_active":"2024-09-19T08:38:18.396009+0000","last_became_peered":"2024-09-19T08:38:18.396009+0000","last_unstale":"2024-09-19T08:38:20.424461+0000","last_undegraded":"2024-09-19T08:38:20.424461+0000","last_fullsized":"2024-09-19T08:38:20.424461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:21.951846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424570+0000","last_change":"2024-09-19T08:38:19.409375+0000","last_active":"2024-09-19T08:38:20.424570+0000","last_peered":"2024-09-19T08:38:20.424570+0000","last_clean":"2024-09-19T08:38:20.424570+0000","last_became_active":"2024-09-19T08:38:19.408667+0000","last_became_peered":"2024-09-19T08:38:19.408667+0000","last_unstale":"2024-09-19T08:38:20.424570+0000","last_undegraded":"2024-09-19T08:38:20.424570+0000","last_fullsized":"2024-09-19T08:38:20.424570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:32.216306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.365899+0000","last_change":"2024-09-19T08:38:18.360672+0000","last_active":"2024-09-19T08:38:18.365899+0000","last_peered":"2024-09-19T08:38:18.365899+0000","last_clean":"2024-09-19T08:38:18.365899+0000","last_became_active":"2024-09-19T08:38:18.360210+0000","last_became_peered":"2024-09-19T08:38:18.360210+0000","last_unstale":"2024-09-19T08:38:18.365899+0000","last_undegraded":"2024-09-19T08:38:18.365899+0000","last_fullsized":"2024-09-19T08:38:18.365899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:05.111193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.358717+0000","last_change":"2024-09-19T08:38:18.358717+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.358717+0000","last_undegraded":"2024-09-19T08:38:18.358717+0000","last_fullsized":"2024-09-19T08:38:18.358717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.380164+0000","last_change":"2024-09-19T08:38:18.377025+0000","last_active":"2024-09-19T08:38:18.380164+0000","last_peered":"2024-09-19T08:38:18.380164+0000","last_clean":"2024-09-19T08:38:18.380164+0000","last_became_active":"2024-09-19T08:38:18.372574+0000","last_became_peered":"2024-09-19T08:38:18.372574+0000","last_unstale":"2024-09-19T08:38:18.380164+0000","last_undegraded":"2024-09-19T08:38:18.380164+0000","last_fullsized":"2024-09-19T08:38:18.380164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:05.111193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.357541+0000","last_change":"2024-09-19T08:38:18.357541+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.357541+0000","last_undegraded":"2024-09-19T08:38:18.357541+0000","last_fullsized":"2024-09-19T08:38:18.357541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.368559+0000","last_change":"2024-09-19T08:38:18.366653+0000","last_active":"2024-09-19T08:38:18.368559+0000","last_peered":"2024-09-19T08:38:18.368559+0000","last_clean":"2024-09-19T08:38:18.368559+0000","last_became_active":"2024-09-19T08:38:18.366357+0000","last_became_peered":"2024-09-19T08:38:18.366357+0000","last_unstale":"2024-09-19T08:38:18.368559+0000","last_undegraded":"2024-09-19T08:38:18.368559+0000","last_fullsized":"2024-09-19T08:38:18.368559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:30.341445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.363646+0000","last_change":"2024-09-19T08:38:18.363646+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.363646+0000","last_undegraded":"2024-09-19T08:38:18.363646+0000","last_fullsized":"2024-09-19T08:38:18.363646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.367913+0000","last_change":"2024-09-19T08:38:18.366656+0000","last_active":"2024-09-19T08:38:18.367913+0000","last_peered":"2024-09-19T08:38:18.367913+0000","last_clean":"2024-09-19T08:38:18.367913+0000","last_became_active":"2024-09-19T08:38:18.366299+0000","last_became_peered":"2024-09-19T08:38:18.366299+0000","last_unstale":"2024-09-19T08:38:18.367913+0000","last_undegraded":"2024-09-19T08:38:18.367913+0000","last_fullsized":"2024-09-19T08:38:18.367913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:59.268711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.366406+0000","last_change":"2024-09-19T08:38:18.366406+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.366406+0000","last_undegraded":"2024-09-19T08:38:18.366406+0000","last_fullsized":"2024-09-19T08:38:18.366406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.399509+0000","last_change":"2024-09-19T08:38:18.394865+0000","last_active":"2024-09-19T08:38:18.399509+0000","last_peered":"2024-09-19T08:38:18.399509+0000","last_clean":"2024-09-19T08:38:18.399509+0000","last_became_active":"2024-09-19T08:38:18.394496+0000","last_became_peered":"2024-09-19T08:38:18.394496+0000","last_unstale":"2024-09-19T08:38:18.399509+0000","last_undegraded":"2024-09-19T08:38:18.399509+0000","last_fullsized":"2024-09-19T08:38:18.399509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:59.268711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.365122+0000","last_change":"2024-09-19T08:38:18.365122+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.365122+0000","last_undegraded":"2024-09-19T08:38:18.365122+0000","last_fullsized":"2024-09-19T08:38:18.365122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":166,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.375597+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:38:18.375597+0000","last_peered":"2024-09-19T08:38:18.375597+0000","last_clean":"2024-09-19T08:38:18.375597+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:38:18.375597+0000","last_undegraded":"2024-09-19T08:38:18.375597+0000","last_fullsized":"2024-09-19T08:38:18.375597+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.398411+0000","last_change":"2024-09-19T08:38:18.391179+0000","last_active":"2024-09-19T08:38:18.398411+0000","last_peered":"2024-09-19T08:38:18.398411+0000","last_clean":"2024-09-19T08:38:18.398411+0000","last_became_active":"2024-09-19T08:38:18.390393+0000","last_became_peered":"2024-09-19T08:38:18.390393+0000","last_unstale":"2024-09-19T08:38:18.398411+0000","last_undegraded":"2024-09-19T08:38:18.398411+0000","last_fullsized":"2024-09-19T08:38:18.398411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:05.111193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085655+0000","last_change":"2024-09-19T08:38:19.430328+0000","last_active":"2024-09-19T08:38:22.085655+0000","last_peered":"2024-09-19T08:38:22.085655+0000","last_clean":"2024-09-19T08:38:22.085655+0000","last_became_active":"2024-09-19T08:38:19.429984+0000","last_became_peered":"2024-09-19T08:38:19.429984+0000","last_unstale":"2024-09-19T08:38:22.085655+0000","last_undegraded":"2024-09-19T08:38:22.085655+0000","last_fullsized":"2024-09-19T08:38:22.085655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:19:32.736687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.391428+0000","last_change":"2024-09-19T08:38:18.384759+0000","last_active":"2024-09-19T08:38:18.391428+0000","last_peered":"2024-09-19T08:38:18.391428+0000","last_clean":"2024-09-19T08:38:18.391428+0000","last_became_active":"2024-09-19T08:38:18.383974+0000","last_became_peered":"2024-09-19T08:38:18.383974+0000","last_unstale":"2024-09-19T08:38:18.391428+0000","last_undegraded":"2024-09-19T08:38:18.391428+0000","last_fullsized":"2024-09-19T08:38:18.391428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:52.328114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.345450+0000","last_change":"2024-09-19T08:38:18.345450+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.345450+0000","last_undegraded":"2024-09-19T08:38:18.345450+0000","last_fullsized":"2024-09-19T08:38:18.345450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.368575+0000","last_change":"2024-09-19T08:38:18.366746+0000","last_active":"2024-09-19T08:38:18.368575+0000","last_peered":"2024-09-19T08:38:18.368575+0000","last_clean":"2024-09-19T08:38:18.368575+0000","last_became_active":"2024-09-19T08:38:18.366346+0000","last_became_peered":"2024-09-19T08:38:18.366346+0000","last_unstale":"2024-09-19T08:38:18.368575+0000","last_undegraded":"2024-09-19T08:38:18.368575+0000","last_fullsized":"2024-09-19T08:38:18.368575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:52.328114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.346210+0000","last_change":"2024-09-19T08:38:18.346210+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.346210+0000","last_undegraded":"2024-09-19T08:38:18.346210+0000","last_fullsized":"2024-09-19T08:38:18.346210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.403231+0000","last_change":"2024-09-19T08:38:18.398888+0000","last_active":"2024-09-19T08:38:18.403231+0000","last_peered":"2024-09-19T08:38:18.403231+0000","last_clean":"2024-09-19T08:38:18.403231+0000","last_became_active":"2024-09-19T08:38:18.391135+0000","last_became_peered":"2024-09-19T08:38:18.391135+0000","last_unstale":"2024-09-19T08:38:18.403231+0000","last_undegraded":"2024-09-19T08:38:18.403231+0000","last_fullsized":"2024-09-19T08:38:18.403231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:59.268711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.353393+0000","last_change":"2024-09-19T08:38:18.353393+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.353393+0000","last_undegraded":"2024-09-19T08:38:18.353393+0000","last_fullsized":"2024-09-19T08:38:18.353393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.365863+0000","last_change":"2024-09-19T08:38:18.363154+0000","last_active":"2024-09-19T08:38:18.365863+0000","last_peered":"2024-09-19T08:38:18.365863+0000","last_clean":"2024-09-19T08:38:18.365863+0000","last_became_active":"2024-09-19T08:38:18.362876+0000","last_became_peered":"2024-09-19T08:38:18.362876+0000","last_unstale":"2024-09-19T08:38:18.365863+0000","last_undegraded":"2024-09-19T08:38:18.365863+0000","last_fullsized":"2024-09-19T08:38:18.365863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:52.328114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.365956+0000","last_change":"2024-09-19T08:38:18.365956+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.365956+0000","last_undegraded":"2024-09-19T08:38:18.365956+0000","last_fullsized":"2024-09-19T08:38:18.365956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.397433+0000","last_change":"2024-09-19T08:38:18.389212+0000","last_active":"2024-09-19T08:38:18.397433+0000","last_peered":"2024-09-19T08:38:18.397433+0000","last_clean":"2024-09-19T08:38:18.397433+0000","last_became_active":"2024-09-19T08:38:18.388588+0000","last_became_peered":"2024-09-19T08:38:18.388588+0000","last_unstale":"2024-09-19T08:38:18.397433+0000","last_undegraded":"2024-09-19T08:38:18.397433+0000","last_fullsized":"2024-09-19T08:38:18.397433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:55.536656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.350757+0000","last_change":"2024-09-19T08:38:18.350757+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.350757+0000","last_undegraded":"2024-09-19T08:38:18.350757+0000","last_fullsized":"2024-09-19T08:38:18.350757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.381033+0000","last_change":"2024-09-19T08:38:18.377131+0000","last_active":"2024-09-19T08:38:18.381033+0000","last_peered":"2024-09-19T08:38:18.381033+0000","last_clean":"2024-09-19T08:38:18.381033+0000","last_became_active":"2024-09-19T08:38:18.376860+0000","last_became_peered":"2024-09-19T08:38:18.376860+0000","last_unstale":"2024-09-19T08:38:18.381033+0000","last_undegraded":"2024-09-19T08:38:18.381033+0000","last_fullsized":"2024-09-19T08:38:18.381033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:34:58.036813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085974+0000","last_change":"2024-09-19T08:38:19.410025+0000","last_active":"2024-09-19T08:38:22.085974+0000","last_peered":"2024-09-19T08:38:22.085974+0000","last_clean":"2024-09-19T08:38:22.085974+0000","last_became_active":"2024-09-19T08:38:19.409390+0000","last_became_peered":"2024-09-19T08:38:19.409390+0000","last_unstale":"2024-09-19T08:38:22.085974+0000","last_undegraded":"2024-09-19T08:38:22.085974+0000","last_fullsized":"2024-09-19T08:38:22.085974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:10.400137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.425026+0000","last_change":"2024-09-19T08:38:18.392567+0000","last_active":"2024-09-19T08:38:20.425026+0000","last_peered":"2024-09-19T08:38:20.425026+0000","last_clean":"2024-09-19T08:38:20.425026+0000","last_became_active":"2024-09-19T08:38:18.387256+0000","last_became_peered":"2024-09-19T08:38:18.387256+0000","last_unstale":"2024-09-19T08:38:20.425026+0000","last_undegraded":"2024-09-19T08:38:20.425026+0000","last_fullsized":"2024-09-19T08:38:20.425026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:12:57.094603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.346457+0000","last_change":"2024-09-19T08:38:18.346457+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.346457+0000","last_undegraded":"2024-09-19T08:38:18.346457+0000","last_fullsized":"2024-09-19T08:38:18.346457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.391302+0000","last_change":"2024-09-19T08:38:18.384552+0000","last_active":"2024-09-19T08:38:18.391302+0000","last_peered":"2024-09-19T08:38:18.391302+0000","last_clean":"2024-09-19T08:38:18.391302+0000","last_became_active":"2024-09-19T08:38:18.383834+0000","last_became_peered":"2024-09-19T08:38:18.383834+0000","last_unstale":"2024-09-19T08:38:18.391302+0000","last_undegraded":"2024-09-19T08:38:18.391302+0000","last_fullsized":"2024-09-19T08:38:18.391302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:30.341445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.352521+0000","last_change":"2024-09-19T08:38:18.352521+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.352521+0000","last_undegraded":"2024-09-19T08:38:18.352521+0000","last_fullsized":"2024-09-19T08:38:18.352521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.383574+0000","last_change":"2024-09-19T08:38:18.378231+0000","last_active":"2024-09-19T08:38:18.383574+0000","last_peered":"2024-09-19T08:38:18.383574+0000","last_clean":"2024-09-19T08:38:18.383574+0000","last_became_active":"2024-09-19T08:38:18.377883+0000","last_became_peered":"2024-09-19T08:38:18.377883+0000","last_unstale":"2024-09-19T08:38:18.383574+0000","last_undegraded":"2024-09-19T08:38:18.383574+0000","last_fullsized":"2024-09-19T08:38:18.383574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:55.536656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085006+0000","last_change":"2024-09-19T08:38:19.392650+0000","last_active":"2024-09-19T08:38:22.085006+0000","last_peered":"2024-09-19T08:38:22.085006+0000","last_clean":"2024-09-19T08:38:22.085006+0000","last_became_active":"2024-09-19T08:38:19.391936+0000","last_became_peered":"2024-09-19T08:38:19.391936+0000","last_unstale":"2024-09-19T08:38:22.085006+0000","last_undegraded":"2024-09-19T08:38:22.085006+0000","last_fullsized":"2024-09-19T08:38:22.085006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:42:24.499497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.364662+0000","last_change":"2024-09-19T08:38:18.359251+0000","last_active":"2024-09-19T08:38:18.364662+0000","last_peered":"2024-09-19T08:38:18.364662+0000","last_clean":"2024-09-19T08:38:18.364662+0000","last_became_active":"2024-09-19T08:38:18.358873+0000","last_became_peered":"2024-09-19T08:38:18.358873+0000","last_unstale":"2024-09-19T08:38:18.364662+0000","last_undegraded":"2024-09-19T08:38:18.364662+0000","last_fullsized":"2024-09-19T08:38:18.364662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:30.341445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085621+0000","last_change":"2024-09-19T08:38:19.414125+0000","last_active":"2024-09-19T08:38:22.085621+0000","last_peered":"2024-09-19T08:38:22.085621+0000","last_clean":"2024-09-19T08:38:22.085621+0000","last_became_active":"2024-09-19T08:38:19.413714+0000","last_became_peered":"2024-09-19T08:38:19.413714+0000","last_unstale":"2024-09-19T08:38:22.085621+0000","last_undegraded":"2024-09-19T08:38:22.085621+0000","last_fullsized":"2024-09-19T08:38:22.085621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:39:22.441525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.397355+0000","last_change":"2024-09-19T08:38:18.387311+0000","last_active":"2024-09-19T08:38:18.397355+0000","last_peered":"2024-09-19T08:38:18.397355+0000","last_clean":"2024-09-19T08:38:18.397355+0000","last_became_active":"2024-09-19T08:38:18.386340+0000","last_became_peered":"2024-09-19T08:38:18.386340+0000","last_unstale":"2024-09-19T08:38:18.397355+0000","last_undegraded":"2024-09-19T08:38:18.397355+0000","last_fullsized":"2024-09-19T08:38:18.397355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:34:58.036813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424982+0000","last_change":"2024-09-19T08:38:19.407976+0000","last_active":"2024-09-19T08:38:20.424982+0000","last_peered":"2024-09-19T08:38:20.424982+0000","last_clean":"2024-09-19T08:38:20.424982+0000","last_became_active":"2024-09-19T08:38:19.407198+0000","last_became_peered":"2024-09-19T08:38:19.407198+0000","last_unstale":"2024-09-19T08:38:20.424982+0000","last_undegraded":"2024-09-19T08:38:20.424982+0000","last_fullsized":"2024-09-19T08:38:20.424982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:42:24.499497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.372680+0000","last_change":"2024-09-19T08:38:18.372680+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.372680+0000","last_undegraded":"2024-09-19T08:38:18.372680+0000","last_fullsized":"2024-09-19T08:38:18.372680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.402413+0000","last_change":"2024-09-19T08:38:18.400304+0000","last_active":"2024-09-19T08:38:18.402413+0000","last_peered":"2024-09-19T08:38:18.402413+0000","last_clean":"2024-09-19T08:38:18.402413+0000","last_became_active":"2024-09-19T08:38:18.399609+0000","last_became_peered":"2024-09-19T08:38:18.399609+0000","last_unstale":"2024-09-19T08:38:18.402413+0000","last_undegraded":"2024-09-19T08:38:18.402413+0000","last_fullsized":"2024-09-19T08:38:18.402413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:47.789309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.373318+0000","last_change":"2024-09-19T08:38:18.373318+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.373318+0000","last_undegraded":"2024-09-19T08:38:18.373318+0000","last_fullsized":"2024-09-19T08:38:18.373318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.391342+0000","last_change":"2024-09-19T08:38:18.384861+0000","last_active":"2024-09-19T08:38:18.391342+0000","last_peered":"2024-09-19T08:38:18.391342+0000","last_clean":"2024-09-19T08:38:18.391342+0000","last_became_active":"2024-09-19T08:38:18.384070+0000","last_became_peered":"2024-09-19T08:38:18.384070+0000","last_unstale":"2024-09-19T08:38:18.391342+0000","last_undegraded":"2024-09-19T08:38:18.391342+0000","last_fullsized":"2024-09-19T08:38:18.391342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:59.268711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.347147+0000","last_change":"2024-09-19T08:38:18.347147+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.347147+0000","last_undegraded":"2024-09-19T08:38:18.347147+0000","last_fullsized":"2024-09-19T08:38:18.347147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085411+0000","last_change":"2024-09-19T08:38:18.358117+0000","last_active":"2024-09-19T08:38:22.085411+0000","last_peered":"2024-09-19T08:38:22.085411+0000","last_clean":"2024-09-19T08:38:22.085411+0000","last_became_active":"2024-09-19T08:38:18.357791+0000","last_became_peered":"2024-09-19T08:38:18.357791+0000","last_unstale":"2024-09-19T08:38:22.085411+0000","last_undegraded":"2024-09-19T08:38:22.085411+0000","last_fullsized":"2024-09-19T08:38:22.085411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:00.838359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085487+0000","last_change":"2024-09-19T08:38:19.396783+0000","last_active":"2024-09-19T08:38:22.085487+0000","last_peered":"2024-09-19T08:38:22.085487+0000","last_clean":"2024-09-19T08:38:22.085487+0000","last_became_active":"2024-09-19T08:38:19.396012+0000","last_became_peered":"2024-09-19T08:38:19.396012+0000","last_unstale":"2024-09-19T08:38:22.085487+0000","last_undegraded":"2024-09-19T08:38:22.085487+0000","last_fullsized":"2024-09-19T08:38:22.085487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:20:35.891069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.386129+0000","last_change":"2024-09-19T08:38:18.380235+0000","last_active":"2024-09-19T08:38:18.386129+0000","last_peered":"2024-09-19T08:38:18.386129+0000","last_clean":"2024-09-19T08:38:18.386129+0000","last_became_active":"2024-09-19T08:38:18.380044+0000","last_became_peered":"2024-09-19T08:38:18.380044+0000","last_unstale":"2024-09-19T08:38:18.386129+0000","last_undegraded":"2024-09-19T08:38:18.386129+0000","last_fullsized":"2024-09-19T08:38:18.386129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:51.748538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":649760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":184320,"data_stored":59392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":184320,"data_stored":59392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509443,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":172032,"data_stored":53117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":172032,"data_stored":53117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509443,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":649760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":74079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":74079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:38:24.109 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3f is not active+clean 2024-09-19T08:38:24.109 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.368603+0000', 'last_change': '2024-09-19T08:38:18.368603+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.368603+0000', 'last_undegraded': '2024-09-19T08:38:18.368603+0000', 'last_fullsized': '2024-09-19T08:38:18.368603+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T08:38:24.109 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3e is not active+clean 2024-09-19T08:38:24.109 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.349394+0000', 'last_change': '2024-09-19T08:38:18.349394+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.349394+0000', 'last_undegraded': '2024-09-19T08:38:18.349394+0000', 'last_fullsized': '2024-09-19T08:38:18.349394+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T08:38:24.109 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3d is not active+clean 2024-09-19T08:38:24.109 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.352974+0000', 'last_change': '2024-09-19T08:38:18.352974+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.352974+0000', 'last_undegraded': '2024-09-19T08:38:18.352974+0000', 'last_fullsized': '2024-09-19T08:38:18.352974+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T08:38:24.110 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3c is not active+clean 2024-09-19T08:38:24.110 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.389392+0000', 'last_change': '2024-09-19T08:38:18.389392+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.389392+0000', 'last_undegraded': '2024-09-19T08:38:18.389392+0000', 'last_fullsized': '2024-09-19T08:38:18.389392+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T08:38:24.110 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3b is not active+clean 2024-09-19T08:38:24.110 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.361760+0000', 'last_change': '2024-09-19T08:38:18.361760+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.361760+0000', 'last_undegraded': '2024-09-19T08:38:18.361760+0000', 'last_fullsized': '2024-09-19T08:38:18.361760+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T08:38:24.110 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3a is not active+clean 2024-09-19T08:38:24.110 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.389879+0000', 'last_change': '2024-09-19T08:38:18.389879+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.389879+0000', 'last_undegraded': '2024-09-19T08:38:18.389879+0000', 'last_fullsized': '2024-09-19T08:38:18.389879+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T08:38:24.110 INFO:tasks.ceph.ceph_manager.ceph:PG 3.39 is not active+clean 2024-09-19T08:38:24.110 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.363729+0000', 'last_change': '2024-09-19T08:38:18.363729+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.363729+0000', 'last_undegraded': '2024-09-19T08:38:18.363729+0000', 'last_fullsized': '2024-09-19T08:38:18.363729+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T08:38:24.110 INFO:tasks.ceph.ceph_manager.ceph:PG 3.37 is not active+clean 2024-09-19T08:38:24.110 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.366761+0000', 'last_change': '2024-09-19T08:38:18.366761+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.366761+0000', 'last_undegraded': '2024-09-19T08:38:18.366761+0000', 'last_fullsized': '2024-09-19T08:38:18.366761+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T08:38:24.110 INFO:tasks.ceph.ceph_manager.ceph:PG 3.34 is not active+clean 2024-09-19T08:38:24.110 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.355429+0000', 'last_change': '2024-09-19T08:38:18.355429+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.355429+0000', 'last_undegraded': '2024-09-19T08:38:18.355429+0000', 'last_fullsized': '2024-09-19T08:38:18.355429+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T08:38:24.111 INFO:tasks.ceph.ceph_manager.ceph:PG 3.32 is not active+clean 2024-09-19T08:38:24.111 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.350914+0000', 'last_change': '2024-09-19T08:38:18.350914+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.350914+0000', 'last_undegraded': '2024-09-19T08:38:18.350914+0000', 'last_fullsized': '2024-09-19T08:38:18.350914+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T08:38:24.111 INFO:tasks.ceph.ceph_manager.ceph:PG 3.31 is not active+clean 2024-09-19T08:38:24.111 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.349265+0000', 'last_change': '2024-09-19T08:38:18.349265+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.349265+0000', 'last_undegraded': '2024-09-19T08:38:18.349265+0000', 'last_fullsized': '2024-09-19T08:38:18.349265+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T08:38:24.111 INFO:tasks.ceph.ceph_manager.ceph:PG 3.30 is not active+clean 2024-09-19T08:38:24.111 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.368264+0000', 'last_change': '2024-09-19T08:38:18.368264+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.368264+0000', 'last_undegraded': '2024-09-19T08:38:18.368264+0000', 'last_fullsized': '2024-09-19T08:38:18.368264+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T08:38:24.111 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2f is not active+clean 2024-09-19T08:38:24.111 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.366483+0000', 'last_change': '2024-09-19T08:38:18.366483+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.366483+0000', 'last_undegraded': '2024-09-19T08:38:18.366483+0000', 'last_fullsized': '2024-09-19T08:38:18.366483+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T08:38:24.111 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2e is not active+clean 2024-09-19T08:38:24.111 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.354376+0000', 'last_change': '2024-09-19T08:38:18.354376+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.354376+0000', 'last_undegraded': '2024-09-19T08:38:18.354376+0000', 'last_fullsized': '2024-09-19T08:38:18.354376+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-19T08:38:24.112 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2d is not active+clean 2024-09-19T08:38:24.112 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.356210+0000', 'last_change': '2024-09-19T08:38:18.356210+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.356210+0000', 'last_undegraded': '2024-09-19T08:38:18.356210+0000', 'last_fullsized': '2024-09-19T08:38:18.356210+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T08:38:24.112 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2b is not active+clean 2024-09-19T08:38:24.112 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.351092+0000', 'last_change': '2024-09-19T08:38:18.351092+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.351092+0000', 'last_undegraded': '2024-09-19T08:38:18.351092+0000', 'last_fullsized': '2024-09-19T08:38:18.351092+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T08:38:24.112 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2a is not active+clean 2024-09-19T08:38:24.112 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.371212+0000', 'last_change': '2024-09-19T08:38:18.371212+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.371212+0000', 'last_undegraded': '2024-09-19T08:38:18.371212+0000', 'last_fullsized': '2024-09-19T08:38:18.371212+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T08:38:24.112 INFO:tasks.ceph.ceph_manager.ceph:PG 3.28 is not active+clean 2024-09-19T08:38:24.112 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.363746+0000', 'last_change': '2024-09-19T08:38:18.363746+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.363746+0000', 'last_undegraded': '2024-09-19T08:38:18.363746+0000', 'last_fullsized': '2024-09-19T08:38:18.363746+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T08:38:24.112 INFO:tasks.ceph.ceph_manager.ceph:PG 3.25 is not active+clean 2024-09-19T08:38:24.112 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.349428+0000', 'last_change': '2024-09-19T08:38:18.349428+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.349428+0000', 'last_undegraded': '2024-09-19T08:38:18.349428+0000', 'last_fullsized': '2024-09-19T08:38:18.349428+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T08:38:24.112 INFO:tasks.ceph.ceph_manager.ceph:PG 3.23 is not active+clean 2024-09-19T08:38:24.113 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.349992+0000', 'last_change': '2024-09-19T08:38:18.349992+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.349992+0000', 'last_undegraded': '2024-09-19T08:38:18.349992+0000', 'last_fullsized': '2024-09-19T08:38:18.349992+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T08:38:24.113 INFO:tasks.ceph.ceph_manager.ceph:PG 3.22 is not active+clean 2024-09-19T08:38:24.113 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.368660+0000', 'last_change': '2024-09-19T08:38:18.368660+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.368660+0000', 'last_undegraded': '2024-09-19T08:38:18.368660+0000', 'last_fullsized': '2024-09-19T08:38:18.368660+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T08:38:24.113 INFO:tasks.ceph.ceph_manager.ceph:PG 3.21 is not active+clean 2024-09-19T08:38:24.113 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.353743+0000', 'last_change': '2024-09-19T08:38:18.353743+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.353743+0000', 'last_undegraded': '2024-09-19T08:38:18.353743+0000', 'last_fullsized': '2024-09-19T08:38:18.353743+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-19T08:38:24.113 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1a is not active+clean 2024-09-19T08:38:24.113 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.351625+0000', 'last_change': '2024-09-19T08:38:18.351625+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.351625+0000', 'last_undegraded': '2024-09-19T08:38:18.351625+0000', 'last_fullsized': '2024-09-19T08:38:18.351625+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T08:38:24.113 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1b is not active+clean 2024-09-19T08:38:24.113 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.348441+0000', 'last_change': '2024-09-19T08:38:18.348441+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.348441+0000', 'last_undegraded': '2024-09-19T08:38:18.348441+0000', 'last_fullsized': '2024-09-19T08:38:18.348441+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T08:38:24.114 INFO:tasks.ceph.ceph_manager.ceph:PG 3.19 is not active+clean 2024-09-19T08:38:24.114 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.350268+0000', 'last_change': '2024-09-19T08:38:18.350268+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.350268+0000', 'last_undegraded': '2024-09-19T08:38:18.350268+0000', 'last_fullsized': '2024-09-19T08:38:18.350268+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T08:38:24.114 INFO:tasks.ceph.ceph_manager.ceph:PG 3.16 is not active+clean 2024-09-19T08:38:24.114 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.365694+0000', 'last_change': '2024-09-19T08:38:18.365694+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.365694+0000', 'last_undegraded': '2024-09-19T08:38:18.365694+0000', 'last_fullsized': '2024-09-19T08:38:18.365694+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T08:38:24.114 INFO:tasks.ceph.ceph_manager.ceph:PG 3.17 is not active+clean 2024-09-19T08:38:24.114 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.345935+0000', 'last_change': '2024-09-19T08:38:18.345935+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.345935+0000', 'last_undegraded': '2024-09-19T08:38:18.345935+0000', 'last_fullsized': '2024-09-19T08:38:18.345935+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T08:38:24.114 INFO:tasks.ceph.ceph_manager.ceph:PG 3.14 is not active+clean 2024-09-19T08:38:24.114 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.352042+0000', 'last_change': '2024-09-19T08:38:18.352042+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.352042+0000', 'last_undegraded': '2024-09-19T08:38:18.352042+0000', 'last_fullsized': '2024-09-19T08:38:18.352042+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T08:38:24.114 INFO:tasks.ceph.ceph_manager.ceph:PG 3.15 is not active+clean 2024-09-19T08:38:24.114 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.367831+0000', 'last_change': '2024-09-19T08:38:18.367831+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.367831+0000', 'last_undegraded': '2024-09-19T08:38:18.367831+0000', 'last_fullsized': '2024-09-19T08:38:18.367831+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T08:38:24.114 INFO:tasks.ceph.ceph_manager.ceph:PG 3.12 is not active+clean 2024-09-19T08:38:24.115 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.345969+0000', 'last_change': '2024-09-19T08:38:18.345969+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.345969+0000', 'last_undegraded': '2024-09-19T08:38:18.345969+0000', 'last_fullsized': '2024-09-19T08:38:18.345969+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T08:38:24.115 INFO:tasks.ceph.ceph_manager.ceph:PG 3.13 is not active+clean 2024-09-19T08:38:24.115 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.360463+0000', 'last_change': '2024-09-19T08:38:18.360463+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.360463+0000', 'last_undegraded': '2024-09-19T08:38:18.360463+0000', 'last_fullsized': '2024-09-19T08:38:18.360463+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T08:38:24.115 INFO:tasks.ceph.ceph_manager.ceph:PG 3.11 is not active+clean 2024-09-19T08:38:24.115 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.358717+0000', 'last_change': '2024-09-19T08:38:18.358717+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.358717+0000', 'last_undegraded': '2024-09-19T08:38:18.358717+0000', 'last_fullsized': '2024-09-19T08:38:18.358717+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T08:38:24.115 INFO:tasks.ceph.ceph_manager.ceph:PG 3.e is not active+clean 2024-09-19T08:38:24.115 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.357541+0000', 'last_change': '2024-09-19T08:38:18.357541+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.357541+0000', 'last_undegraded': '2024-09-19T08:38:18.357541+0000', 'last_fullsized': '2024-09-19T08:38:18.357541+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T08:38:24.115 INFO:tasks.ceph.ceph_manager.ceph:PG 3.f is not active+clean 2024-09-19T08:38:24.115 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.363646+0000', 'last_change': '2024-09-19T08:38:18.363646+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.363646+0000', 'last_undegraded': '2024-09-19T08:38:18.363646+0000', 'last_fullsized': '2024-09-19T08:38:18.363646+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T08:38:24.115 INFO:tasks.ceph.ceph_manager.ceph:PG 3.c is not active+clean 2024-09-19T08:38:24.115 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.366406+0000', 'last_change': '2024-09-19T08:38:18.366406+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.366406+0000', 'last_undegraded': '2024-09-19T08:38:18.366406+0000', 'last_fullsized': '2024-09-19T08:38:18.366406+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T08:38:24.116 INFO:tasks.ceph.ceph_manager.ceph:PG 3.d is not active+clean 2024-09-19T08:38:24.116 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.365122+0000', 'last_change': '2024-09-19T08:38:18.365122+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.365122+0000', 'last_undegraded': '2024-09-19T08:38:18.365122+0000', 'last_fullsized': '2024-09-19T08:38:18.365122+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T08:38:24.116 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1 is not active+clean 2024-09-19T08:38:24.116 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.345450+0000', 'last_change': '2024-09-19T08:38:18.345450+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.345450+0000', 'last_undegraded': '2024-09-19T08:38:18.345450+0000', 'last_fullsized': '2024-09-19T08:38:18.345450+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T08:38:24.116 INFO:tasks.ceph.ceph_manager.ceph:PG 3.0 is not active+clean 2024-09-19T08:38:24.116 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.346210+0000', 'last_change': '2024-09-19T08:38:18.346210+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.346210+0000', 'last_undegraded': '2024-09-19T08:38:18.346210+0000', 'last_fullsized': '2024-09-19T08:38:18.346210+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T08:38:24.116 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3 is not active+clean 2024-09-19T08:38:24.116 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.353393+0000', 'last_change': '2024-09-19T08:38:18.353393+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.353393+0000', 'last_undegraded': '2024-09-19T08:38:18.353393+0000', 'last_fullsized': '2024-09-19T08:38:18.353393+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T08:38:24.116 INFO:tasks.ceph.ceph_manager.ceph:PG 3.5 is not active+clean 2024-09-19T08:38:24.117 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.365956+0000', 'last_change': '2024-09-19T08:38:18.365956+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.365956+0000', 'last_undegraded': '2024-09-19T08:38:18.365956+0000', 'last_fullsized': '2024-09-19T08:38:18.365956+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T08:38:24.117 INFO:tasks.ceph.ceph_manager.ceph:PG 3.4 is not active+clean 2024-09-19T08:38:24.117 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.350757+0000', 'last_change': '2024-09-19T08:38:18.350757+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.350757+0000', 'last_undegraded': '2024-09-19T08:38:18.350757+0000', 'last_fullsized': '2024-09-19T08:38:18.350757+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T08:38:24.117 INFO:tasks.ceph.ceph_manager.ceph:PG 3.6 is not active+clean 2024-09-19T08:38:24.117 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.346457+0000', 'last_change': '2024-09-19T08:38:18.346457+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.346457+0000', 'last_undegraded': '2024-09-19T08:38:18.346457+0000', 'last_fullsized': '2024-09-19T08:38:18.346457+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T08:38:24.117 INFO:tasks.ceph.ceph_manager.ceph:PG 3.9 is not active+clean 2024-09-19T08:38:24.117 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.352521+0000', 'last_change': '2024-09-19T08:38:18.352521+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.352521+0000', 'last_undegraded': '2024-09-19T08:38:18.352521+0000', 'last_fullsized': '2024-09-19T08:38:18.352521+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T08:38:24.117 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1d is not active+clean 2024-09-19T08:38:24.117 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.372680+0000', 'last_change': '2024-09-19T08:38:18.372680+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.372680+0000', 'last_undegraded': '2024-09-19T08:38:18.372680+0000', 'last_fullsized': '2024-09-19T08:38:18.372680+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T08:38:24.117 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1c is not active+clean 2024-09-19T08:38:24.117 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.373318+0000', 'last_change': '2024-09-19T08:38:18.373318+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.373318+0000', 'last_undegraded': '2024-09-19T08:38:18.373318+0000', 'last_fullsized': '2024-09-19T08:38:18.373318+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T08:38:24.118 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1f is not active+clean 2024-09-19T08:38:24.118 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 19, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:38:18.347147+0000', 'last_change': '2024-09-19T08:38:18.347147+0000', 'last_active': '2024-09-19T08:38:18.334613+0000', 'last_peered': '2024-09-19T08:38:18.334613+0000', 'last_clean': '2024-09-19T08:38:18.334613+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:38:18.347147+0000', 'last_undegraded': '2024-09-19T08:38:18.347147+0000', 'last_fullsized': '2024-09-19T08:38:18.347147+0000', 'mapping_epoch': 19, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 19, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'last_clean_scrub_stamp': '2024-09-19T08:38:18.334613+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T08:38:24.118 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T08:38:24.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.271+0000 7f4a107ce640 1 Processor -- start 2024-09-19T08:38:24.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.271+0000 7f4a107ce640 1 -- start start 2024-09-19T08:38:24.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.275+0000 7f4a107ce640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a080d05a0 0x7f4a08160be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:24.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.275+0000 7f4a107ce640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4a080d0980 0x7f4a080b3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:24.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.275+0000 7f4a107ce640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a080afec0 con 0x7f4a080b39a0 2024-09-19T08:38:24.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.275+0000 7f4a107ce640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f4a080b3fd0 con 0x7f4a080d0980 2024-09-19T08:38:24.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.275+0000 7f4a107ce640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4a08160100 con 0x7f4a080d05a0 2024-09-19T08:38:24.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.275+0000 7f4a0ed6b640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f4a080b39a0 0x7f4a0815bed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41480/0 (socket says 172.21.15.98:41480) 2024-09-19T08:38:24.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.275+0000 7f4a0ed6b640 1 -- 172.21.15.98:0/3860718562 learned_addr learned my addr 172.21.15.98:0/3860718562 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:24.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.275+0000 7f4a0dd69640 1 --2- 172.21.15.98:0/3860718562 >> v2:172.21.15.139:3300/0 conn(0x7f4a080d0980 0x7f4a080b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:24.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.275+0000 7f4a0e56a640 1 --2- 172.21.15.98:0/3860718562 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a080d05a0 0x7f4a08160be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:24.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.275+0000 7f4a0d568640 1 -- 172.21.15.98:0/3860718562 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3423734732 0 0) 0x7f4a080afec0 con 0x7f4a080b39a0 2024-09-19T08:38:24.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.275+0000 7f4a0d568640 1 -- 172.21.15.98:0/3860718562 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f49f0003530 con 0x7f4a080b39a0 2024-09-19T08:38:24.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.275+0000 7f4a0d568640 1 -- 172.21.15.98:0/3860718562 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 424798854 0 0) 0x7f49f0003530 con 0x7f4a080b39a0 2024-09-19T08:38:24.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.275+0000 7f4a0d568640 1 -- 172.21.15.98:0/3860718562 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a080d05a0 msgr2=0x7f4a08160be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:24.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.275+0000 7f4a0d568640 1 --2- 172.21.15.98:0/3860718562 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a080d05a0 0x7f4a08160be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:24.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.275+0000 7f4a0d568640 1 -- 172.21.15.98:0/3860718562 >> v2:172.21.15.139:3300/0 conn(0x7f4a080d0980 msgr2=0x7f4a080b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:24.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.275+0000 7f4a0d568640 1 --2- 172.21.15.98:0/3860718562 >> v2:172.21.15.139:3300/0 conn(0x7f4a080d0980 0x7f4a080b3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:24.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.275+0000 7f4a0d568640 1 -- 172.21.15.98:0/3860718562 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a081603e0 con 0x7f4a080b39a0 2024-09-19T08:38:24.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.275+0000 7f4a0d568640 1 -- 172.21.15.98:0/3860718562 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f49f8001a00 con 0x7f4a080b39a0 2024-09-19T08:38:24.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.275+0000 7f4a0dd69640 1 --2- 172.21.15.98:0/3860718562 >> v2:172.21.15.139:3300/0 conn(0x7f4a080d0980 0x7f4a080b3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:38:24.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.275+0000 7f4a0d568640 1 -- 172.21.15.98:0/3860718562 >> v1:172.21.15.98:6789/0 conn(0x7f4a080b39a0 legacy=0x7f4a0815bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:24.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.275+0000 7f4a0d568640 1 --2- 172.21.15.98:0/3860718562 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a080d05a0 0x7f49f00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:24.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.275+0000 7f4a0d568640 1 --2- 172.21.15.98:0/3860718562 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49f0003f40 0x7f49f0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:24.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.275+0000 7f4a0d568640 1 --2- 172.21.15.98:0/3860718562 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49f0004d50 0x7f49f00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:24.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.275+0000 7f4a0d568640 1 -- 172.21.15.98:0/3860718562 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4a080b3fd0 con 0x7f49f0003f40 2024-09-19T08:38:24.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.275+0000 7f4a0d568640 1 -- 172.21.15.98:0/3860718562 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4a08160100 con 0x7f49f0004d50 2024-09-19T08:38:24.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.275+0000 7f4a0d568640 1 -- 172.21.15.98:0/3860718562 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f49f000ba20 con 0x7f4a080d05a0 2024-09-19T08:38:24.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.275+0000 7f4a0dd69640 1 --2- 172.21.15.98:0/3860718562 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49f0003f40 0x7f49f0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:24.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.275+0000 7f4a0e56a640 1 --2- 172.21.15.98:0/3860718562 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a080d05a0 0x7f49f00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:24.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.275+0000 7f4a0ed6b640 1 --2- 172.21.15.98:0/3860718562 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49f0004d50 0x7f49f00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:24.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.275+0000 7f4a0dd69640 1 -- 172.21.15.98:0/3860718562 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a080d05a0 msgr2=0x7f49f00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:24.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.275+0000 7f4a0dd69640 1 --2- 172.21.15.98:0/3860718562 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a080d05a0 0x7f49f00039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:24.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.275+0000 7f4a0dd69640 1 -- 172.21.15.98:0/3860718562 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49f0004d50 msgr2=0x7f49f00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:24.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.275+0000 7f4a0dd69640 1 --2- 172.21.15.98:0/3860718562 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49f0004d50 0x7f49f00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:24.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.275+0000 7f4a0dd69640 1 -- 172.21.15.98:0/3860718562 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f0003530 con 0x7f49f0003f40 2024-09-19T08:38:24.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.279+0000 7f4a0dd69640 1 --2- 172.21.15.98:0/3860718562 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49f0003f40 0x7f49f0004810 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f49fc009cb0 tx=0x7f49fc01cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda0d5f4ddfdaba5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:24.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.279+0000 7f4a0d568640 1 -- 172.21.15.98:0/3860718562 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc009880 con 0x7f49f0003f40 2024-09-19T08:38:24.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.279+0000 7f4a0d568640 1 -- 172.21.15.98:0/3860718562 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49fc004020 con 0x7f49f0003f40 2024-09-19T08:38:24.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.279+0000 7f4a0d568640 1 -- 172.21.15.98:0/3860718562 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc004430 con 0x7f49f0003f40 2024-09-19T08:38:24.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.279+0000 7f4a107ce640 1 -- 172.21.15.98:0/3860718562 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49f0003f40 msgr2=0x7f49f0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:24.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.279+0000 7f4a107ce640 1 --2- 172.21.15.98:0/3860718562 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f49f0003f40 0x7f49f0004810 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f49fc009cb0 tx=0x7f49fc01cc50 comp rx=0 tx=0).stop 2024-09-19T08:38:24.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.279+0000 7f4a0ed6b640 1 -- 172.21.15.98:0/3860718562 reap_dead start 2024-09-19T08:38:24.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.279+0000 7f4a107ce640 1 -- 172.21.15.98:0/3860718562 shutdown_connections 2024-09-19T08:38:24.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.279+0000 7f4a107ce640 1 -- 172.21.15.98:0/3860718562 >> 172.21.15.98:0/3860718562 conn(0x7f4a0806bc90 msgr2=0x7f4a0806c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:24.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.279+0000 7f4a107ce640 1 -- 172.21.15.98:0/3860718562 shutdown_connections 2024-09-19T08:38:24.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.279+0000 7f4a107ce640 1 -- 172.21.15.98:0/3860718562 wait complete. 2024-09-19T08:38:24.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.279+0000 7f4a107ce640 1 Processor -- start 2024-09-19T08:38:24.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.279+0000 7f4a107ce640 1 -- start start 2024-09-19T08:38:24.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.279+0000 7f4a107ce640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f49f0003f40 0x7f4a08157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:24.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.279+0000 7f4a107ce640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a0814e430 0x7f4a080cd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:24.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.279+0000 7f4a107ce640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a0815eef0 0x7f4a0815f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:24.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.279+0000 7f4a107ce640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4a080b5320 con 0x7f4a0814e430 2024-09-19T08:38:24.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.279+0000 7f4a107ce640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4a080b51a0 con 0x7f4a0815eef0 2024-09-19T08:38:24.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.279+0000 7f4a107ce640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4a080b54a0 con 0x7f49f0003f40 2024-09-19T08:38:24.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.279+0000 7f4a0dd69640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a0814e430 0x7f4a080cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:24.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.279+0000 7f4a0dd69640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a0814e430 0x7f4a080cd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33428/0 (socket says 172.21.15.98:33428) 2024-09-19T08:38:24.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.279+0000 7f4a0dd69640 1 -- 172.21.15.98:0/1455004788 learned_addr learned my addr 172.21.15.98:0/1455004788 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:24.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.279+0000 7f4a0e56a640 1 --2- 172.21.15.98:0/1455004788 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f49f0003f40 0x7f4a08157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:24.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.279+0000 7f4a0ed6b640 1 --2- 172.21.15.98:0/1455004788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a0815eef0 0x7f4a0815f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:24.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.279+0000 7f4a0dd69640 1 -- 172.21.15.98:0/1455004788 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f49f0003f40 msgr2=0x7f4a08157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:24.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.279+0000 7f4a0dd69640 1 --2- 172.21.15.98:0/1455004788 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f49f0003f40 0x7f4a08157370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:24.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.279+0000 7f4a0dd69640 1 -- 172.21.15.98:0/1455004788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a0815eef0 msgr2=0x7f4a0815f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:24.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.279+0000 7f4a0dd69640 1 --2- 172.21.15.98:0/1455004788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a0815eef0 0x7f4a0815f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:24.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.279+0000 7f4a0dd69640 1 -- 172.21.15.98:0/1455004788 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0814e8b0 con 0x7f4a0814e430 2024-09-19T08:38:24.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.279+0000 7f4a0dd69640 1 --2- 172.21.15.98:0/1455004788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a0814e430 0x7f4a080cd680 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f49fc01ca80 tx=0x7f49f0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:24.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.279+0000 7f4a06ffd640 1 -- 172.21.15.98:0/1455004788 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc005db0 con 0x7f4a0814e430 2024-09-19T08:38:24.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.279+0000 7f4a06ffd640 1 -- 172.21.15.98:0/1455004788 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49fc0be070 con 0x7f4a0814e430 2024-09-19T08:38:24.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.279+0000 7f4a06ffd640 1 -- 172.21.15.98:0/1455004788 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc0a5930 con 0x7f4a0814e430 2024-09-19T08:38:24.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.283+0000 7f4a107ce640 1 -- 172.21.15.98:0/1455004788 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a080cded0 con 0x7f4a0814e430 2024-09-19T08:38:24.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.283+0000 7f4a107ce640 1 -- 172.21.15.98:0/1455004788 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a080ce490 con 0x7f4a0814e430 2024-09-19T08:38:24.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.283+0000 7f4a107ce640 1 -- 172.21.15.98:0/1455004788 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a0815f8b0 con 0x7f4a0814e430 2024-09-19T08:38:24.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.283+0000 7f4a06ffd640 1 -- 172.21.15.98:0/1455004788 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f49fc0a5ad0 con 0x7f4a0814e430 2024-09-19T08:38:24.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.283+0000 7f4a06ffd640 1 --2- 172.21.15.98:0/1455004788 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f49f40678c0 0x7f49f4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:24.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.283+0000 7f4a06ffd640 1 -- 172.21.15.98:0/1455004788 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f49fc120530 con 0x7f4a0814e430 2024-09-19T08:38:24.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.283+0000 7f4a0e56a640 1 --2- 172.21.15.98:0/1455004788 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f49f40678c0 0x7f49f4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:24.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.283+0000 7f4a0e56a640 1 --2- 172.21.15.98:0/1455004788 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f49f40678c0 0x7f49f4069d80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4a0815b600 tx=0x7f4a00008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:24.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.287+0000 7f4a06ffd640 1 -- 172.21.15.98:0/1455004788 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f49fc0169a0 con 0x7f4a0814e430 2024-09-19T08:38:24.332 INFO:tasks.ceph.mds.d.smithi139.stderr:2024-09-19T08:38:24.330+0000 7fe477378640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T08:38:24.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.499+0000 7f4a107ce640 1 -- 172.21.15.98:0/1455004788 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f4a080b3460 con 0x7f4a0814e430 2024-09-19T08:38:24.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.499+0000 7f4a06ffd640 1 -- 172.21.15.98:0/1455004788 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1137 (secure 0 0 0) 0x7f49fc0f1070 con 0x7f4a0814e430 2024-09-19T08:38:24.499 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:38:24.499 INFO:teuthology.orchestra.run.smithi098.stdout:{"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":29,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":21,"num_osds":8,"num_up_osds":8,"osd_up_since":1726735088,"num_in_osds":8,"osd_in_since":1726735077,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":51},{"state_name":"creating+peering","count":46}],"num_pgs":97,"num_pools":3,"num_objects":7,"data_bytes":591980,"bytes_used":224141312,"bytes_avail":772869971968,"bytes_total":773094113280,"inactive_pgs_ratio":0.47422680258750916,"write_bytes_sec":511,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":7,"btime":"2024-09-19T08:38:22:407519+0000","id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"d","status":"up:active","gid":4261}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-19T08:37:55.451005+0000","services":{}},"progress_events":{}} 2024-09-19T08:38:24.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.499+0000 7f4a107ce640 1 -- 172.21.15.98:0/1455004788 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f49f40678c0 msgr2=0x7f49f4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:24.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.499+0000 7f4a107ce640 1 --2- 172.21.15.98:0/1455004788 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f49f40678c0 0x7f49f4069d80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4a0815b600 tx=0x7f4a00008040 comp rx=0 tx=0).stop 2024-09-19T08:38:24.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.499+0000 7f4a107ce640 1 -- 172.21.15.98:0/1455004788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a0814e430 msgr2=0x7f4a080cd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:24.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.499+0000 7f4a107ce640 1 --2- 172.21.15.98:0/1455004788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a0814e430 0x7f4a080cd680 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f49fc01ca80 tx=0x7f49f0005840 comp rx=0 tx=0).stop 2024-09-19T08:38:24.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.499+0000 7f4a107ce640 1 -- 172.21.15.98:0/1455004788 shutdown_connections 2024-09-19T08:38:24.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.499+0000 7f4a107ce640 1 --2- 172.21.15.98:0/1455004788 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f49f40678c0 0x7f49f4069d80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:24.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.499+0000 7f4a107ce640 1 --2- 172.21.15.98:0/1455004788 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a0815eef0 0x7f4a0815f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:24.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.499+0000 7f4a107ce640 1 --2- 172.21.15.98:0/1455004788 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a0814e430 0x7f4a080cd680 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:24.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.499+0000 7f4a107ce640 1 --2- 172.21.15.98:0/1455004788 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f49f0003f40 0x7f4a08157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:24.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.499+0000 7f4a107ce640 1 -- 172.21.15.98:0/1455004788 >> 172.21.15.98:0/1455004788 conn(0x7f4a0806bc90 msgr2=0x7f4a0815d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:24.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.499+0000 7f4a107ce640 1 -- 172.21.15.98:0/1455004788 shutdown_connections 2024-09-19T08:38:24.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.499+0000 7f4a107ce640 1 -- 172.21.15.98:0/1455004788 wait complete. 2024-09-19T08:38:24.516 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-19T08:38:24.517 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:38:24.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.635+0000 7f2415b4f640 1 Processor -- start 2024-09-19T08:38:24.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.635+0000 7f2415b4f640 1 -- start start 2024-09-19T08:38:24.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.635+0000 7f2415b4f640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f241015c550 0x7f241006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:24.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.635+0000 7f2415b4f640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f241006d310 con 0x7f241006c1d0 2024-09-19T08:38:24.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.635+0000 7f2415b4f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f241006d5f0 con 0x7f24100cfad0 2024-09-19T08:38:24.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.635+0000 7f2415b4f640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f241006c640 con 0x7f241015c550 2024-09-19T08:38:24.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.635+0000 7f240f7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f241015c550 0x7f241006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:24.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.635+0000 7f240f7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f241015c550 0x7f241006bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33438/0 (socket says 172.21.15.98:33438) 2024-09-19T08:38:24.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.635+0000 7f240f7fe640 1 -- 172.21.15.98:0/3764167238 learned_addr learned my addr 172.21.15.98:0/3764167238 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:24.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.635+0000 7f240effd640 1 --1- 172.21.15.98:0/3764167238 >> v1:172.21.15.98:6789/0 conn(0x7f241006c1d0 0x7f24100cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41484/0 (socket says 172.21.15.98:41484) 2024-09-19T08:38:24.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.639+0000 7f240f7fe640 1 -- 172.21.15.98:0/3764167238 >> v1:172.21.15.139:6789/0 conn(0x7f24100cfad0 legacy=0x7f24100723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:24.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.639+0000 7f240f7fe640 1 -- 172.21.15.98:0/3764167238 >> v1:172.21.15.98:6789/0 conn(0x7f241006c1d0 legacy=0x7f24100cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:24.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.639+0000 7f240f7fe640 1 -- 172.21.15.98:0/3764167238 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241006d8d0 con 0x7f241015c550 2024-09-19T08:38:24.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.639+0000 7f240f7fe640 1 --2- 172.21.15.98:0/3764167238 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f241015c550 0x7f241006bc90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2400009870 tx=0x7f240001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=251e7b09d3c6169e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:24.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.639+0000 7f240e7fc640 1 -- 172.21.15.98:0/3764167238 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24000b0070 con 0x7f241015c550 2024-09-19T08:38:24.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.639+0000 7f240e7fc640 1 -- 172.21.15.98:0/3764167238 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24000ab600 con 0x7f241015c550 2024-09-19T08:38:24.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.639+0000 7f240e7fc640 1 -- 172.21.15.98:0/3764167238 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24000ab9d0 con 0x7f241015c550 2024-09-19T08:38:24.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.639+0000 7f2415b4f640 1 -- 172.21.15.98:0/3764167238 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f241015c550 msgr2=0x7f241006bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:24.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.639+0000 7f2415b4f640 1 --2- 172.21.15.98:0/3764167238 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f241015c550 0x7f241006bc90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2400009870 tx=0x7f240001c520 comp rx=0 tx=0).stop 2024-09-19T08:38:24.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.639+0000 7f2415b4f640 1 -- 172.21.15.98:0/3764167238 shutdown_connections 2024-09-19T08:38:24.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.639+0000 7f2415b4f640 1 --2- 172.21.15.98:0/3764167238 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f241015c550 0x7f241006bc90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:24.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.639+0000 7f2415b4f640 1 -- 172.21.15.98:0/3764167238 >> 172.21.15.98:0/3764167238 conn(0x7f24100b2010 msgr2=0x7f24100b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:24.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.639+0000 7f2415b4f640 1 -- 172.21.15.98:0/3764167238 shutdown_connections 2024-09-19T08:38:24.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.639+0000 7f2415b4f640 1 -- 172.21.15.98:0/3764167238 wait complete. 2024-09-19T08:38:24.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.643+0000 7f2415b4f640 1 Processor -- start 2024-09-19T08:38:24.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.643+0000 7f2415b4f640 1 -- start start 2024-09-19T08:38:24.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.643+0000 7f2415b4f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f241006c1d0 0x7f2410149f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:24.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.643+0000 7f2415b4f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24100cfad0 0x7f24100ae970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:24.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.643+0000 7f2415b4f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f241015c550 0x7f24101567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:24.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.643+0000 7f2415b4f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2410073690 con 0x7f241006c1d0 2024-09-19T08:38:24.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.643+0000 7f2415b4f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2410073510 con 0x7f241015c550 2024-09-19T08:38:24.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.643+0000 7f2415b4f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f241016a240 con 0x7f24100cfad0 2024-09-19T08:38:24.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.643+0000 7f240f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f241006c1d0 0x7f2410149f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:24.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.643+0000 7f240f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f241006c1d0 0x7f2410149f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33446/0 (socket says 172.21.15.98:33446) 2024-09-19T08:38:24.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.643+0000 7f240f7fe640 1 -- 172.21.15.98:0/710599103 learned_addr learned my addr 172.21.15.98:0/710599103 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:24.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.643+0000 7f240f7fe640 1 -- 172.21.15.98:0/710599103 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24100cfad0 msgr2=0x7f24100ae970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:24.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.643+0000 7f240ffff640 1 --2- 172.21.15.98:0/710599103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f241015c550 0x7f24101567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:24.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.643+0000 7f240f7fe640 1 --2- 172.21.15.98:0/710599103 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24100cfad0 0x7f24100ae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:24.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.643+0000 7f240f7fe640 1 -- 172.21.15.98:0/710599103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f241015c550 msgr2=0x7f24101567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:24.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.643+0000 7f240f7fe640 1 --2- 172.21.15.98:0/710599103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f241015c550 0x7f24101567d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:24.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.643+0000 7f240f7fe640 1 -- 172.21.15.98:0/710599103 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2410071690 con 0x7f241006c1d0 2024-09-19T08:38:24.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.643+0000 7f240f7fe640 1 --2- 172.21.15.98:0/710599103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f241006c1d0 0x7f2410149f20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f240001abd0 tx=0x7f240001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:24.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.643+0000 7f2407fff640 1 -- 172.21.15.98:0/710599103 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24000b0040 con 0x7f241006c1d0 2024-09-19T08:38:24.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.643+0000 7f2407fff640 1 -- 172.21.15.98:0/710599103 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24000bc070 con 0x7f241006c1d0 2024-09-19T08:38:24.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.643+0000 7f2415b4f640 1 -- 172.21.15.98:0/710599103 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24101555b0 con 0x7f241006c1d0 2024-09-19T08:38:24.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.643+0000 7f2407fff640 1 -- 172.21.15.98:0/710599103 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24000b6880 con 0x7f241006c1d0 2024-09-19T08:38:24.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.643+0000 7f2415b4f640 1 -- 172.21.15.98:0/710599103 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f241014a900 con 0x7f241006c1d0 2024-09-19T08:38:24.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.643+0000 7f2415b4f640 1 -- 172.21.15.98:0/710599103 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f241006bc90 con 0x7f241006c1d0 2024-09-19T08:38:24.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.643+0000 7f2407fff640 1 -- 172.21.15.98:0/710599103 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f24000b6a20 con 0x7f241006c1d0 2024-09-19T08:38:24.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.643+0000 7f2407fff640 1 --2- 172.21.15.98:0/710599103 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f23f00678c0 0x7f23f0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:24.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.647+0000 7f2407fff640 1 -- 172.21.15.98:0/710599103 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f240011ffe0 con 0x7f241006c1d0 2024-09-19T08:38:24.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.647+0000 7f240effd640 1 --2- 172.21.15.98:0/710599103 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f23f00678c0 0x7f23f0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:24.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.647+0000 7f240effd640 1 --2- 172.21.15.98:0/710599103 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f23f00678c0 0x7f23f0069d80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f23f8000f70 tx=0x7f23f8007a10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:24.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.651+0000 7f2407fff640 1 -- 172.21.15.98:0/710599103 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f24101555b0 con 0x7f241006c1d0 2024-09-19T08:38:24.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.803+0000 7f2415b4f640 1 -- 172.21.15.98:0/710599103 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2410156d10 con 0x7f23f00678c0 2024-09-19T08:38:24.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.819+0000 7f2407fff640 1 -- 172.21.15.98:0/710599103 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248134 (secure 0 0 0) 0x7f2410156d10 con 0x7f23f00678c0 2024-09-19T08:38:24.823 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:38:24.825 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:38:24.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.823+0000 7f2415b4f640 1 -- 172.21.15.98:0/710599103 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f23f00678c0 msgr2=0x7f23f0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:24.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.823+0000 7f2415b4f640 1 --2- 172.21.15.98:0/710599103 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f23f00678c0 0x7f23f0069d80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f23f8000f70 tx=0x7f23f8007a10 comp rx=0 tx=0).stop 2024-09-19T08:38:24.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.823+0000 7f2415b4f640 1 -- 172.21.15.98:0/710599103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f241006c1d0 msgr2=0x7f2410149f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:24.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.823+0000 7f2415b4f640 1 --2- 172.21.15.98:0/710599103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f241006c1d0 0x7f2410149f20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f240001abd0 tx=0x7f240001ad60 comp rx=0 tx=0).stop 2024-09-19T08:38:24.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.823+0000 7f2415b4f640 1 -- 172.21.15.98:0/710599103 shutdown_connections 2024-09-19T08:38:24.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.823+0000 7f2415b4f640 1 --2- 172.21.15.98:0/710599103 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f23f00678c0 0x7f23f0069d80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:24.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.823+0000 7f2415b4f640 1 --2- 172.21.15.98:0/710599103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f241015c550 0x7f24101567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:24.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.823+0000 7f2415b4f640 1 --2- 172.21.15.98:0/710599103 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24100cfad0 0x7f24100ae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:24.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.823+0000 7f2415b4f640 1 --2- 172.21.15.98:0/710599103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f241006c1d0 0x7f2410149f20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:24.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.823+0000 7f2415b4f640 1 -- 172.21.15.98:0/710599103 >> 172.21.15.98:0/710599103 conn(0x7f24100b2010 msgr2=0x7f2410152890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:24.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.823+0000 7f2415b4f640 1 -- 172.21.15.98:0/710599103 shutdown_connections 2024-09-19T08:38:24.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:24.823+0000 7f2415b4f640 1 -- 172.21.15.98:0/710599103 wait complete. 2024-09-19T08:38:24.840 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-19T08:38:24.027662+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591980,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":168,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218888,"kb_used_data":2648,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755832,"statfs":{"total":773094113280,"available":772869971968,"internally_reserved":0,"allocated":2711552,"data_stored":1672696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001937"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.368603+0000","last_change":"2024-09-19T08:38:18.368603+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.368603+0000","last_undegraded":"2024-09-19T08:38:18.368603+0000","last_fullsized":"2024-09-19T08:38:18.368603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.349394+0000","last_change":"2024-09-19T08:38:18.349394+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.349394+0000","last_undegraded":"2024-09-19T08:38:18.349394+0000","last_fullsized":"2024-09-19T08:38:18.349394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.352974+0000","last_change":"2024-09-19T08:38:18.352974+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.352974+0000","last_undegraded":"2024-09-19T08:38:18.352974+0000","last_fullsized":"2024-09-19T08:38:18.352974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.389392+0000","last_change":"2024-09-19T08:38:18.389392+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.389392+0000","last_undegraded":"2024-09-19T08:38:18.389392+0000","last_fullsized":"2024-09-19T08:38:18.389392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.361760+0000","last_change":"2024-09-19T08:38:18.361760+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.361760+0000","last_undegraded":"2024-09-19T08:38:18.361760+0000","last_fullsized":"2024-09-19T08:38:18.361760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.389879+0000","last_change":"2024-09-19T08:38:18.389879+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.389879+0000","last_undegraded":"2024-09-19T08:38:18.389879+0000","last_fullsized":"2024-09-19T08:38:18.389879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.363729+0000","last_change":"2024-09-19T08:38:18.363729+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.363729+0000","last_undegraded":"2024-09-19T08:38:18.363729+0000","last_fullsized":"2024-09-19T08:38:18.363729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.084591+0000","last_change":"2024-09-19T08:38:19.400917+0000","last_active":"2024-09-19T08:38:22.084591+0000","last_peered":"2024-09-19T08:38:22.084591+0000","last_clean":"2024-09-19T08:38:22.084591+0000","last_became_active":"2024-09-19T08:38:19.400188+0000","last_became_peered":"2024-09-19T08:38:19.400188+0000","last_unstale":"2024-09-19T08:38:22.084591+0000","last_undegraded":"2024-09-19T08:38:22.084591+0000","last_fullsized":"2024-09-19T08:38:22.084591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:24:38.033969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.366761+0000","last_change":"2024-09-19T08:38:18.366761+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.366761+0000","last_undegraded":"2024-09-19T08:38:18.366761+0000","last_fullsized":"2024-09-19T08:38:18.366761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.425359+0000","last_change":"2024-09-19T08:38:19.402509+0000","last_active":"2024-09-19T08:38:20.425359+0000","last_peered":"2024-09-19T08:38:20.425359+0000","last_clean":"2024-09-19T08:38:20.425359+0000","last_became_active":"2024-09-19T08:38:19.401833+0000","last_became_peered":"2024-09-19T08:38:19.401833+0000","last_unstale":"2024-09-19T08:38:20.425359+0000","last_undegraded":"2024-09-19T08:38:20.425359+0000","last_fullsized":"2024-09-19T08:38:20.425359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:08.065282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424901+0000","last_change":"2024-09-19T08:38:19.430286+0000","last_active":"2024-09-19T08:38:20.424901+0000","last_peered":"2024-09-19T08:38:20.424901+0000","last_clean":"2024-09-19T08:38:20.424901+0000","last_became_active":"2024-09-19T08:38:19.429901+0000","last_became_peered":"2024-09-19T08:38:19.429901+0000","last_unstale":"2024-09-19T08:38:20.424901+0000","last_undegraded":"2024-09-19T08:38:20.424901+0000","last_fullsized":"2024-09-19T08:38:20.424901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:31:42.999935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.355429+0000","last_change":"2024-09-19T08:38:18.355429+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.355429+0000","last_undegraded":"2024-09-19T08:38:18.355429+0000","last_fullsized":"2024-09-19T08:38:18.355429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.086556+0000","last_change":"2024-09-19T08:38:19.396759+0000","last_active":"2024-09-19T08:38:22.086556+0000","last_peered":"2024-09-19T08:38:22.086556+0000","last_clean":"2024-09-19T08:38:22.086556+0000","last_became_active":"2024-09-19T08:38:19.395976+0000","last_became_peered":"2024-09-19T08:38:19.395976+0000","last_unstale":"2024-09-19T08:38:22.086556+0000","last_undegraded":"2024-09-19T08:38:22.086556+0000","last_fullsized":"2024-09-19T08:38:22.086556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:20.116184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.350914+0000","last_change":"2024-09-19T08:38:18.350914+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.350914+0000","last_undegraded":"2024-09-19T08:38:18.350914+0000","last_fullsized":"2024-09-19T08:38:18.350914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.349265+0000","last_change":"2024-09-19T08:38:18.349265+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.349265+0000","last_undegraded":"2024-09-19T08:38:18.349265+0000","last_fullsized":"2024-09-19T08:38:18.349265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.368264+0000","last_change":"2024-09-19T08:38:18.368264+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.368264+0000","last_undegraded":"2024-09-19T08:38:18.368264+0000","last_fullsized":"2024-09-19T08:38:18.368264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.366483+0000","last_change":"2024-09-19T08:38:18.366483+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.366483+0000","last_undegraded":"2024-09-19T08:38:18.366483+0000","last_fullsized":"2024-09-19T08:38:18.366483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.354376+0000","last_change":"2024-09-19T08:38:18.354376+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.354376+0000","last_undegraded":"2024-09-19T08:38:18.354376+0000","last_fullsized":"2024-09-19T08:38:18.354376+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.356210+0000","last_change":"2024-09-19T08:38:18.356210+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.356210+0000","last_undegraded":"2024-09-19T08:38:18.356210+0000","last_fullsized":"2024-09-19T08:38:18.356210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.425280+0000","last_change":"2024-09-19T08:38:19.389509+0000","last_active":"2024-09-19T08:38:20.425280+0000","last_peered":"2024-09-19T08:38:20.425280+0000","last_clean":"2024-09-19T08:38:20.425280+0000","last_became_active":"2024-09-19T08:38:19.388736+0000","last_became_peered":"2024-09-19T08:38:19.388736+0000","last_unstale":"2024-09-19T08:38:20.425280+0000","last_undegraded":"2024-09-19T08:38:20.425280+0000","last_fullsized":"2024-09-19T08:38:20.425280+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:20:35.891069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.351092+0000","last_change":"2024-09-19T08:38:18.351092+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.351092+0000","last_undegraded":"2024-09-19T08:38:18.351092+0000","last_fullsized":"2024-09-19T08:38:18.351092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.371212+0000","last_change":"2024-09-19T08:38:18.371212+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.371212+0000","last_undegraded":"2024-09-19T08:38:18.371212+0000","last_fullsized":"2024-09-19T08:38:18.371212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.084821+0000","last_change":"2024-09-19T08:38:19.406260+0000","last_active":"2024-09-19T08:38:22.084821+0000","last_peered":"2024-09-19T08:38:22.084821+0000","last_clean":"2024-09-19T08:38:22.084821+0000","last_became_active":"2024-09-19T08:38:19.405769+0000","last_became_peered":"2024-09-19T08:38:19.405769+0000","last_unstale":"2024-09-19T08:38:22.084821+0000","last_undegraded":"2024-09-19T08:38:22.084821+0000","last_fullsized":"2024-09-19T08:38:22.084821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:31:42.999935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.363746+0000","last_change":"2024-09-19T08:38:18.363746+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.363746+0000","last_undegraded":"2024-09-19T08:38:18.363746+0000","last_fullsized":"2024-09-19T08:38:18.363746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085580+0000","last_change":"2024-09-19T08:38:19.406201+0000","last_active":"2024-09-19T08:38:22.085580+0000","last_peered":"2024-09-19T08:38:22.085580+0000","last_clean":"2024-09-19T08:38:22.085580+0000","last_became_active":"2024-09-19T08:38:19.405583+0000","last_became_peered":"2024-09-19T08:38:19.405583+0000","last_unstale":"2024-09-19T08:38:22.085580+0000","last_undegraded":"2024-09-19T08:38:22.085580+0000","last_fullsized":"2024-09-19T08:38:22.085580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:08.065282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.425374+0000","last_change":"2024-09-19T08:38:19.411842+0000","last_active":"2024-09-19T08:38:20.425374+0000","last_peered":"2024-09-19T08:38:20.425374+0000","last_clean":"2024-09-19T08:38:20.425374+0000","last_became_active":"2024-09-19T08:38:19.411092+0000","last_became_peered":"2024-09-19T08:38:19.411092+0000","last_unstale":"2024-09-19T08:38:20.425374+0000","last_undegraded":"2024-09-19T08:38:20.425374+0000","last_fullsized":"2024-09-19T08:38:20.425374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:39:22.441525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.349428+0000","last_change":"2024-09-19T08:38:18.349428+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.349428+0000","last_undegraded":"2024-09-19T08:38:18.349428+0000","last_fullsized":"2024-09-19T08:38:18.349428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085836+0000","last_change":"2024-09-19T08:38:19.408371+0000","last_active":"2024-09-19T08:38:22.085836+0000","last_peered":"2024-09-19T08:38:22.085836+0000","last_clean":"2024-09-19T08:38:22.085836+0000","last_became_active":"2024-09-19T08:38:19.407846+0000","last_became_peered":"2024-09-19T08:38:19.407846+0000","last_unstale":"2024-09-19T08:38:22.085836+0000","last_undegraded":"2024-09-19T08:38:22.085836+0000","last_fullsized":"2024-09-19T08:38:22.085836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:33.753905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.349992+0000","last_change":"2024-09-19T08:38:18.349992+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.349992+0000","last_undegraded":"2024-09-19T08:38:18.349992+0000","last_fullsized":"2024-09-19T08:38:18.349992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.368660+0000","last_change":"2024-09-19T08:38:18.368660+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.368660+0000","last_undegraded":"2024-09-19T08:38:18.368660+0000","last_fullsized":"2024-09-19T08:38:18.368660+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.353743+0000","last_change":"2024-09-19T08:38:18.353743+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.353743+0000","last_undegraded":"2024-09-19T08:38:18.353743+0000","last_fullsized":"2024-09-19T08:38:18.353743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424603+0000","last_change":"2024-09-19T08:38:19.414469+0000","last_active":"2024-09-19T08:38:20.424603+0000","last_peered":"2024-09-19T08:38:20.424603+0000","last_clean":"2024-09-19T08:38:20.424603+0000","last_became_active":"2024-09-19T08:38:19.413995+0000","last_became_peered":"2024-09-19T08:38:19.413995+0000","last_unstale":"2024-09-19T08:38:20.424603+0000","last_undegraded":"2024-09-19T08:38:20.424603+0000","last_fullsized":"2024-09-19T08:38:20.424603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:24:38.033969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.351625+0000","last_change":"2024-09-19T08:38:18.351625+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.351625+0000","last_undegraded":"2024-09-19T08:38:18.351625+0000","last_fullsized":"2024-09-19T08:38:18.351625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085745+0000","last_change":"2024-09-19T08:38:18.387502+0000","last_active":"2024-09-19T08:38:22.085745+0000","last_peered":"2024-09-19T08:38:22.085745+0000","last_clean":"2024-09-19T08:38:22.085745+0000","last_became_active":"2024-09-19T08:38:18.387154+0000","last_became_peered":"2024-09-19T08:38:18.387154+0000","last_unstale":"2024-09-19T08:38:22.085745+0000","last_undegraded":"2024-09-19T08:38:22.085745+0000","last_fullsized":"2024-09-19T08:38:22.085745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:53.708142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.348441+0000","last_change":"2024-09-19T08:38:18.348441+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.348441+0000","last_undegraded":"2024-09-19T08:38:18.348441+0000","last_fullsized":"2024-09-19T08:38:18.348441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424590+0000","last_change":"2024-09-19T08:38:18.381446+0000","last_active":"2024-09-19T08:38:20.424590+0000","last_peered":"2024-09-19T08:38:20.424590+0000","last_clean":"2024-09-19T08:38:20.424590+0000","last_became_active":"2024-09-19T08:38:18.380678+0000","last_became_peered":"2024-09-19T08:38:18.380678+0000","last_unstale":"2024-09-19T08:38:20.424590+0000","last_undegraded":"2024-09-19T08:38:20.424590+0000","last_fullsized":"2024-09-19T08:38:20.424590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:07:31.456618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.084969+0000","last_change":"2024-09-19T08:38:18.376428+0000","last_active":"2024-09-19T08:38:22.084969+0000","last_peered":"2024-09-19T08:38:22.084969+0000","last_clean":"2024-09-19T08:38:22.084969+0000","last_became_active":"2024-09-19T08:38:18.376015+0000","last_became_peered":"2024-09-19T08:38:18.376015+0000","last_unstale":"2024-09-19T08:38:22.084969+0000","last_undegraded":"2024-09-19T08:38:22.084969+0000","last_fullsized":"2024-09-19T08:38:22.084969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:49:33.943029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.084815+0000","last_change":"2024-09-19T08:38:19.414261+0000","last_active":"2024-09-19T08:38:22.084815+0000","last_peered":"2024-09-19T08:38:22.084815+0000","last_clean":"2024-09-19T08:38:22.084815+0000","last_became_active":"2024-09-19T08:38:19.413930+0000","last_became_peered":"2024-09-19T08:38:19.413930+0000","last_unstale":"2024-09-19T08:38:22.084815+0000","last_undegraded":"2024-09-19T08:38:22.084815+0000","last_fullsized":"2024-09-19T08:38:22.084815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:32.216306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.398624+0000","last_change":"2024-09-19T08:38:18.393405+0000","last_active":"2024-09-19T08:38:18.398624+0000","last_peered":"2024-09-19T08:38:18.398624+0000","last_clean":"2024-09-19T08:38:18.398624+0000","last_became_active":"2024-09-19T08:38:18.392613+0000","last_became_peered":"2024-09-19T08:38:18.392613+0000","last_unstale":"2024-09-19T08:38:18.398624+0000","last_undegraded":"2024-09-19T08:38:18.398624+0000","last_fullsized":"2024-09-19T08:38:18.398624+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:30.341445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.350268+0000","last_change":"2024-09-19T08:38:18.350268+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.350268+0000","last_undegraded":"2024-09-19T08:38:18.350268+0000","last_fullsized":"2024-09-19T08:38:18.350268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424870+0000","last_change":"2024-09-19T08:38:18.398231+0000","last_active":"2024-09-19T08:38:20.424870+0000","last_peered":"2024-09-19T08:38:20.424870+0000","last_clean":"2024-09-19T08:38:20.424870+0000","last_became_active":"2024-09-19T08:38:18.397590+0000","last_became_peered":"2024-09-19T08:38:18.397590+0000","last_unstale":"2024-09-19T08:38:20.424870+0000","last_undegraded":"2024-09-19T08:38:20.424870+0000","last_fullsized":"2024-09-19T08:38:20.424870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:54:02.262454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.365694+0000","last_change":"2024-09-19T08:38:18.365694+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.365694+0000","last_undegraded":"2024-09-19T08:38:18.365694+0000","last_fullsized":"2024-09-19T08:38:18.365694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.398568+0000","last_change":"2024-09-19T08:38:18.393568+0000","last_active":"2024-09-19T08:38:18.398568+0000","last_peered":"2024-09-19T08:38:18.398568+0000","last_clean":"2024-09-19T08:38:18.398568+0000","last_became_active":"2024-09-19T08:38:18.390336+0000","last_became_peered":"2024-09-19T08:38:18.390336+0000","last_unstale":"2024-09-19T08:38:18.398568+0000","last_undegraded":"2024-09-19T08:38:18.398568+0000","last_fullsized":"2024-09-19T08:38:18.398568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:52.328114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.345935+0000","last_change":"2024-09-19T08:38:18.345935+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.345935+0000","last_undegraded":"2024-09-19T08:38:18.345935+0000","last_fullsized":"2024-09-19T08:38:18.345935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.394587+0000","last_change":"2024-09-19T08:38:18.390022+0000","last_active":"2024-09-19T08:38:18.394587+0000","last_peered":"2024-09-19T08:38:18.394587+0000","last_clean":"2024-09-19T08:38:18.394587+0000","last_became_active":"2024-09-19T08:38:18.389679+0000","last_became_peered":"2024-09-19T08:38:18.389679+0000","last_unstale":"2024-09-19T08:38:18.394587+0000","last_undegraded":"2024-09-19T08:38:18.394587+0000","last_fullsized":"2024-09-19T08:38:18.394587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:47.789309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.352042+0000","last_change":"2024-09-19T08:38:18.352042+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.352042+0000","last_undegraded":"2024-09-19T08:38:18.352042+0000","last_fullsized":"2024-09-19T08:38:18.352042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424913+0000","last_change":"2024-09-19T08:38:18.371315+0000","last_active":"2024-09-19T08:38:20.424913+0000","last_peered":"2024-09-19T08:38:20.424913+0000","last_clean":"2024-09-19T08:38:20.424913+0000","last_became_active":"2024-09-19T08:38:18.370524+0000","last_became_peered":"2024-09-19T08:38:18.370524+0000","last_unstale":"2024-09-19T08:38:20.424913+0000","last_undegraded":"2024-09-19T08:38:20.424913+0000","last_fullsized":"2024-09-19T08:38:20.424913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:49:33.943029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.367831+0000","last_change":"2024-09-19T08:38:18.367831+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.367831+0000","last_undegraded":"2024-09-19T08:38:18.367831+0000","last_fullsized":"2024-09-19T08:38:18.367831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.370392+0000","last_change":"2024-09-19T08:38:18.366653+0000","last_active":"2024-09-19T08:38:18.370392+0000","last_peered":"2024-09-19T08:38:18.370392+0000","last_clean":"2024-09-19T08:38:18.370392+0000","last_became_active":"2024-09-19T08:38:18.366387+0000","last_became_peered":"2024-09-19T08:38:18.366387+0000","last_unstale":"2024-09-19T08:38:18.370392+0000","last_undegraded":"2024-09-19T08:38:18.370392+0000","last_fullsized":"2024-09-19T08:38:18.370392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:17.554499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.345969+0000","last_change":"2024-09-19T08:38:18.345969+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.345969+0000","last_undegraded":"2024-09-19T08:38:18.345969+0000","last_fullsized":"2024-09-19T08:38:18.345969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.403987+0000","last_change":"2024-09-19T08:38:18.402871+0000","last_active":"2024-09-19T08:38:18.403987+0000","last_peered":"2024-09-19T08:38:18.403987+0000","last_clean":"2024-09-19T08:38:18.403987+0000","last_became_active":"2024-09-19T08:38:18.400331+0000","last_became_peered":"2024-09-19T08:38:18.400331+0000","last_unstale":"2024-09-19T08:38:18.403987+0000","last_undegraded":"2024-09-19T08:38:18.403987+0000","last_fullsized":"2024-09-19T08:38:18.403987+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:34:58.036813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.360463+0000","last_change":"2024-09-19T08:38:18.360463+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.360463+0000","last_undegraded":"2024-09-19T08:38:18.360463+0000","last_fullsized":"2024-09-19T08:38:18.360463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424461+0000","last_change":"2024-09-19T08:38:18.396642+0000","last_active":"2024-09-19T08:38:20.424461+0000","last_peered":"2024-09-19T08:38:20.424461+0000","last_clean":"2024-09-19T08:38:20.424461+0000","last_became_active":"2024-09-19T08:38:18.396009+0000","last_became_peered":"2024-09-19T08:38:18.396009+0000","last_unstale":"2024-09-19T08:38:20.424461+0000","last_undegraded":"2024-09-19T08:38:20.424461+0000","last_fullsized":"2024-09-19T08:38:20.424461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:21.951846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424570+0000","last_change":"2024-09-19T08:38:19.409375+0000","last_active":"2024-09-19T08:38:20.424570+0000","last_peered":"2024-09-19T08:38:20.424570+0000","last_clean":"2024-09-19T08:38:20.424570+0000","last_became_active":"2024-09-19T08:38:19.408667+0000","last_became_peered":"2024-09-19T08:38:19.408667+0000","last_unstale":"2024-09-19T08:38:20.424570+0000","last_undegraded":"2024-09-19T08:38:20.424570+0000","last_fullsized":"2024-09-19T08:38:20.424570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:32.216306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.365899+0000","last_change":"2024-09-19T08:38:18.360672+0000","last_active":"2024-09-19T08:38:18.365899+0000","last_peered":"2024-09-19T08:38:18.365899+0000","last_clean":"2024-09-19T08:38:18.365899+0000","last_became_active":"2024-09-19T08:38:18.360210+0000","last_became_peered":"2024-09-19T08:38:18.360210+0000","last_unstale":"2024-09-19T08:38:18.365899+0000","last_undegraded":"2024-09-19T08:38:18.365899+0000","last_fullsized":"2024-09-19T08:38:18.365899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:05.111193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.358717+0000","last_change":"2024-09-19T08:38:18.358717+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.358717+0000","last_undegraded":"2024-09-19T08:38:18.358717+0000","last_fullsized":"2024-09-19T08:38:18.358717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.380164+0000","last_change":"2024-09-19T08:38:18.377025+0000","last_active":"2024-09-19T08:38:18.380164+0000","last_peered":"2024-09-19T08:38:18.380164+0000","last_clean":"2024-09-19T08:38:18.380164+0000","last_became_active":"2024-09-19T08:38:18.372574+0000","last_became_peered":"2024-09-19T08:38:18.372574+0000","last_unstale":"2024-09-19T08:38:18.380164+0000","last_undegraded":"2024-09-19T08:38:18.380164+0000","last_fullsized":"2024-09-19T08:38:18.380164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:05.111193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.357541+0000","last_change":"2024-09-19T08:38:18.357541+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.357541+0000","last_undegraded":"2024-09-19T08:38:18.357541+0000","last_fullsized":"2024-09-19T08:38:18.357541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.368559+0000","last_change":"2024-09-19T08:38:18.366653+0000","last_active":"2024-09-19T08:38:18.368559+0000","last_peered":"2024-09-19T08:38:18.368559+0000","last_clean":"2024-09-19T08:38:18.368559+0000","last_became_active":"2024-09-19T08:38:18.366357+0000","last_became_peered":"2024-09-19T08:38:18.366357+0000","last_unstale":"2024-09-19T08:38:18.368559+0000","last_undegraded":"2024-09-19T08:38:18.368559+0000","last_fullsized":"2024-09-19T08:38:18.368559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:30.341445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.363646+0000","last_change":"2024-09-19T08:38:18.363646+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.363646+0000","last_undegraded":"2024-09-19T08:38:18.363646+0000","last_fullsized":"2024-09-19T08:38:18.363646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.367913+0000","last_change":"2024-09-19T08:38:18.366656+0000","last_active":"2024-09-19T08:38:18.367913+0000","last_peered":"2024-09-19T08:38:18.367913+0000","last_clean":"2024-09-19T08:38:18.367913+0000","last_became_active":"2024-09-19T08:38:18.366299+0000","last_became_peered":"2024-09-19T08:38:18.366299+0000","last_unstale":"2024-09-19T08:38:18.367913+0000","last_undegraded":"2024-09-19T08:38:18.367913+0000","last_fullsized":"2024-09-19T08:38:18.367913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:59.268711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.366406+0000","last_change":"2024-09-19T08:38:18.366406+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.366406+0000","last_undegraded":"2024-09-19T08:38:18.366406+0000","last_fullsized":"2024-09-19T08:38:18.366406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.399509+0000","last_change":"2024-09-19T08:38:18.394865+0000","last_active":"2024-09-19T08:38:18.399509+0000","last_peered":"2024-09-19T08:38:18.399509+0000","last_clean":"2024-09-19T08:38:18.399509+0000","last_became_active":"2024-09-19T08:38:18.394496+0000","last_became_peered":"2024-09-19T08:38:18.394496+0000","last_unstale":"2024-09-19T08:38:18.399509+0000","last_undegraded":"2024-09-19T08:38:18.399509+0000","last_fullsized":"2024-09-19T08:38:18.399509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:59.268711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.365122+0000","last_change":"2024-09-19T08:38:18.365122+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.365122+0000","last_undegraded":"2024-09-19T08:38:18.365122+0000","last_fullsized":"2024-09-19T08:38:18.365122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":166,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.375597+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:38:18.375597+0000","last_peered":"2024-09-19T08:38:18.375597+0000","last_clean":"2024-09-19T08:38:18.375597+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:38:18.375597+0000","last_undegraded":"2024-09-19T08:38:18.375597+0000","last_fullsized":"2024-09-19T08:38:18.375597+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.398411+0000","last_change":"2024-09-19T08:38:18.391179+0000","last_active":"2024-09-19T08:38:18.398411+0000","last_peered":"2024-09-19T08:38:18.398411+0000","last_clean":"2024-09-19T08:38:18.398411+0000","last_became_active":"2024-09-19T08:38:18.390393+0000","last_became_peered":"2024-09-19T08:38:18.390393+0000","last_unstale":"2024-09-19T08:38:18.398411+0000","last_undegraded":"2024-09-19T08:38:18.398411+0000","last_fullsized":"2024-09-19T08:38:18.398411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:45:05.111193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085655+0000","last_change":"2024-09-19T08:38:19.430328+0000","last_active":"2024-09-19T08:38:22.085655+0000","last_peered":"2024-09-19T08:38:22.085655+0000","last_clean":"2024-09-19T08:38:22.085655+0000","last_became_active":"2024-09-19T08:38:19.429984+0000","last_became_peered":"2024-09-19T08:38:19.429984+0000","last_unstale":"2024-09-19T08:38:22.085655+0000","last_undegraded":"2024-09-19T08:38:22.085655+0000","last_fullsized":"2024-09-19T08:38:22.085655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:19:32.736687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.391428+0000","last_change":"2024-09-19T08:38:18.384759+0000","last_active":"2024-09-19T08:38:18.391428+0000","last_peered":"2024-09-19T08:38:18.391428+0000","last_clean":"2024-09-19T08:38:18.391428+0000","last_became_active":"2024-09-19T08:38:18.383974+0000","last_became_peered":"2024-09-19T08:38:18.383974+0000","last_unstale":"2024-09-19T08:38:18.391428+0000","last_undegraded":"2024-09-19T08:38:18.391428+0000","last_fullsized":"2024-09-19T08:38:18.391428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:52.328114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.345450+0000","last_change":"2024-09-19T08:38:18.345450+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.345450+0000","last_undegraded":"2024-09-19T08:38:18.345450+0000","last_fullsized":"2024-09-19T08:38:18.345450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.368575+0000","last_change":"2024-09-19T08:38:18.366746+0000","last_active":"2024-09-19T08:38:18.368575+0000","last_peered":"2024-09-19T08:38:18.368575+0000","last_clean":"2024-09-19T08:38:18.368575+0000","last_became_active":"2024-09-19T08:38:18.366346+0000","last_became_peered":"2024-09-19T08:38:18.366346+0000","last_unstale":"2024-09-19T08:38:18.368575+0000","last_undegraded":"2024-09-19T08:38:18.368575+0000","last_fullsized":"2024-09-19T08:38:18.368575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:52.328114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.346210+0000","last_change":"2024-09-19T08:38:18.346210+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.346210+0000","last_undegraded":"2024-09-19T08:38:18.346210+0000","last_fullsized":"2024-09-19T08:38:18.346210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.403231+0000","last_change":"2024-09-19T08:38:18.398888+0000","last_active":"2024-09-19T08:38:18.403231+0000","last_peered":"2024-09-19T08:38:18.403231+0000","last_clean":"2024-09-19T08:38:18.403231+0000","last_became_active":"2024-09-19T08:38:18.391135+0000","last_became_peered":"2024-09-19T08:38:18.391135+0000","last_unstale":"2024-09-19T08:38:18.403231+0000","last_undegraded":"2024-09-19T08:38:18.403231+0000","last_fullsized":"2024-09-19T08:38:18.403231+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:59.268711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.353393+0000","last_change":"2024-09-19T08:38:18.353393+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.353393+0000","last_undegraded":"2024-09-19T08:38:18.353393+0000","last_fullsized":"2024-09-19T08:38:18.353393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.365863+0000","last_change":"2024-09-19T08:38:18.363154+0000","last_active":"2024-09-19T08:38:18.365863+0000","last_peered":"2024-09-19T08:38:18.365863+0000","last_clean":"2024-09-19T08:38:18.365863+0000","last_became_active":"2024-09-19T08:38:18.362876+0000","last_became_peered":"2024-09-19T08:38:18.362876+0000","last_unstale":"2024-09-19T08:38:18.365863+0000","last_undegraded":"2024-09-19T08:38:18.365863+0000","last_fullsized":"2024-09-19T08:38:18.365863+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:52.328114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.365956+0000","last_change":"2024-09-19T08:38:18.365956+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.365956+0000","last_undegraded":"2024-09-19T08:38:18.365956+0000","last_fullsized":"2024-09-19T08:38:18.365956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.397433+0000","last_change":"2024-09-19T08:38:18.389212+0000","last_active":"2024-09-19T08:38:18.397433+0000","last_peered":"2024-09-19T08:38:18.397433+0000","last_clean":"2024-09-19T08:38:18.397433+0000","last_became_active":"2024-09-19T08:38:18.388588+0000","last_became_peered":"2024-09-19T08:38:18.388588+0000","last_unstale":"2024-09-19T08:38:18.397433+0000","last_undegraded":"2024-09-19T08:38:18.397433+0000","last_fullsized":"2024-09-19T08:38:18.397433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:55.536656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.350757+0000","last_change":"2024-09-19T08:38:18.350757+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.350757+0000","last_undegraded":"2024-09-19T08:38:18.350757+0000","last_fullsized":"2024-09-19T08:38:18.350757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.381033+0000","last_change":"2024-09-19T08:38:18.377131+0000","last_active":"2024-09-19T08:38:18.381033+0000","last_peered":"2024-09-19T08:38:18.381033+0000","last_clean":"2024-09-19T08:38:18.381033+0000","last_became_active":"2024-09-19T08:38:18.376860+0000","last_became_peered":"2024-09-19T08:38:18.376860+0000","last_unstale":"2024-09-19T08:38:18.381033+0000","last_undegraded":"2024-09-19T08:38:18.381033+0000","last_fullsized":"2024-09-19T08:38:18.381033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:34:58.036813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085974+0000","last_change":"2024-09-19T08:38:19.410025+0000","last_active":"2024-09-19T08:38:22.085974+0000","last_peered":"2024-09-19T08:38:22.085974+0000","last_clean":"2024-09-19T08:38:22.085974+0000","last_became_active":"2024-09-19T08:38:19.409390+0000","last_became_peered":"2024-09-19T08:38:19.409390+0000","last_unstale":"2024-09-19T08:38:22.085974+0000","last_undegraded":"2024-09-19T08:38:22.085974+0000","last_fullsized":"2024-09-19T08:38:22.085974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:10.400137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.425026+0000","last_change":"2024-09-19T08:38:18.392567+0000","last_active":"2024-09-19T08:38:20.425026+0000","last_peered":"2024-09-19T08:38:20.425026+0000","last_clean":"2024-09-19T08:38:20.425026+0000","last_became_active":"2024-09-19T08:38:18.387256+0000","last_became_peered":"2024-09-19T08:38:18.387256+0000","last_unstale":"2024-09-19T08:38:20.425026+0000","last_undegraded":"2024-09-19T08:38:20.425026+0000","last_fullsized":"2024-09-19T08:38:20.425026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:12:57.094603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.346457+0000","last_change":"2024-09-19T08:38:18.346457+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.346457+0000","last_undegraded":"2024-09-19T08:38:18.346457+0000","last_fullsized":"2024-09-19T08:38:18.346457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.391302+0000","last_change":"2024-09-19T08:38:18.384552+0000","last_active":"2024-09-19T08:38:18.391302+0000","last_peered":"2024-09-19T08:38:18.391302+0000","last_clean":"2024-09-19T08:38:18.391302+0000","last_became_active":"2024-09-19T08:38:18.383834+0000","last_became_peered":"2024-09-19T08:38:18.383834+0000","last_unstale":"2024-09-19T08:38:18.391302+0000","last_undegraded":"2024-09-19T08:38:18.391302+0000","last_fullsized":"2024-09-19T08:38:18.391302+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:30.341445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.352521+0000","last_change":"2024-09-19T08:38:18.352521+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.352521+0000","last_undegraded":"2024-09-19T08:38:18.352521+0000","last_fullsized":"2024-09-19T08:38:18.352521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.383574+0000","last_change":"2024-09-19T08:38:18.378231+0000","last_active":"2024-09-19T08:38:18.383574+0000","last_peered":"2024-09-19T08:38:18.383574+0000","last_clean":"2024-09-19T08:38:18.383574+0000","last_became_active":"2024-09-19T08:38:18.377883+0000","last_became_peered":"2024-09-19T08:38:18.377883+0000","last_unstale":"2024-09-19T08:38:18.383574+0000","last_undegraded":"2024-09-19T08:38:18.383574+0000","last_fullsized":"2024-09-19T08:38:18.383574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:55.536656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085006+0000","last_change":"2024-09-19T08:38:19.392650+0000","last_active":"2024-09-19T08:38:22.085006+0000","last_peered":"2024-09-19T08:38:22.085006+0000","last_clean":"2024-09-19T08:38:22.085006+0000","last_became_active":"2024-09-19T08:38:19.391936+0000","last_became_peered":"2024-09-19T08:38:19.391936+0000","last_unstale":"2024-09-19T08:38:22.085006+0000","last_undegraded":"2024-09-19T08:38:22.085006+0000","last_fullsized":"2024-09-19T08:38:22.085006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:42:24.499497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.364662+0000","last_change":"2024-09-19T08:38:18.359251+0000","last_active":"2024-09-19T08:38:18.364662+0000","last_peered":"2024-09-19T08:38:18.364662+0000","last_clean":"2024-09-19T08:38:18.364662+0000","last_became_active":"2024-09-19T08:38:18.358873+0000","last_became_peered":"2024-09-19T08:38:18.358873+0000","last_unstale":"2024-09-19T08:38:18.364662+0000","last_undegraded":"2024-09-19T08:38:18.364662+0000","last_fullsized":"2024-09-19T08:38:18.364662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:30.341445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085621+0000","last_change":"2024-09-19T08:38:19.414125+0000","last_active":"2024-09-19T08:38:22.085621+0000","last_peered":"2024-09-19T08:38:22.085621+0000","last_clean":"2024-09-19T08:38:22.085621+0000","last_became_active":"2024-09-19T08:38:19.413714+0000","last_became_peered":"2024-09-19T08:38:19.413714+0000","last_unstale":"2024-09-19T08:38:22.085621+0000","last_undegraded":"2024-09-19T08:38:22.085621+0000","last_fullsized":"2024-09-19T08:38:22.085621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:39:22.441525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.397355+0000","last_change":"2024-09-19T08:38:18.387311+0000","last_active":"2024-09-19T08:38:18.397355+0000","last_peered":"2024-09-19T08:38:18.397355+0000","last_clean":"2024-09-19T08:38:18.397355+0000","last_became_active":"2024-09-19T08:38:18.386340+0000","last_became_peered":"2024-09-19T08:38:18.386340+0000","last_unstale":"2024-09-19T08:38:18.397355+0000","last_undegraded":"2024-09-19T08:38:18.397355+0000","last_fullsized":"2024-09-19T08:38:18.397355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:34:58.036813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424982+0000","last_change":"2024-09-19T08:38:19.407976+0000","last_active":"2024-09-19T08:38:20.424982+0000","last_peered":"2024-09-19T08:38:20.424982+0000","last_clean":"2024-09-19T08:38:20.424982+0000","last_became_active":"2024-09-19T08:38:19.407198+0000","last_became_peered":"2024-09-19T08:38:19.407198+0000","last_unstale":"2024-09-19T08:38:20.424982+0000","last_undegraded":"2024-09-19T08:38:20.424982+0000","last_fullsized":"2024-09-19T08:38:20.424982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:42:24.499497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.372680+0000","last_change":"2024-09-19T08:38:18.372680+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.372680+0000","last_undegraded":"2024-09-19T08:38:18.372680+0000","last_fullsized":"2024-09-19T08:38:18.372680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.402413+0000","last_change":"2024-09-19T08:38:18.400304+0000","last_active":"2024-09-19T08:38:18.402413+0000","last_peered":"2024-09-19T08:38:18.402413+0000","last_clean":"2024-09-19T08:38:18.402413+0000","last_became_active":"2024-09-19T08:38:18.399609+0000","last_became_peered":"2024-09-19T08:38:18.399609+0000","last_unstale":"2024-09-19T08:38:18.402413+0000","last_undegraded":"2024-09-19T08:38:18.402413+0000","last_fullsized":"2024-09-19T08:38:18.402413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:47.789309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.373318+0000","last_change":"2024-09-19T08:38:18.373318+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.373318+0000","last_undegraded":"2024-09-19T08:38:18.373318+0000","last_fullsized":"2024-09-19T08:38:18.373318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.391342+0000","last_change":"2024-09-19T08:38:18.384861+0000","last_active":"2024-09-19T08:38:18.391342+0000","last_peered":"2024-09-19T08:38:18.391342+0000","last_clean":"2024-09-19T08:38:18.391342+0000","last_became_active":"2024-09-19T08:38:18.384070+0000","last_became_peered":"2024-09-19T08:38:18.384070+0000","last_unstale":"2024-09-19T08:38:18.391342+0000","last_undegraded":"2024-09-19T08:38:18.391342+0000","last_fullsized":"2024-09-19T08:38:18.391342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:59.268711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1,"reported_epoch":19,"state":"creating+peering","last_fresh":"2024-09-19T08:38:18.347147+0000","last_change":"2024-09-19T08:38:18.347147+0000","last_active":"2024-09-19T08:38:18.334613+0000","last_peered":"2024-09-19T08:38:18.334613+0000","last_clean":"2024-09-19T08:38:18.334613+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:38:18.347147+0000","last_undegraded":"2024-09-19T08:38:18.347147+0000","last_fullsized":"2024-09-19T08:38:18.347147+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085411+0000","last_change":"2024-09-19T08:38:18.358117+0000","last_active":"2024-09-19T08:38:22.085411+0000","last_peered":"2024-09-19T08:38:22.085411+0000","last_clean":"2024-09-19T08:38:22.085411+0000","last_became_active":"2024-09-19T08:38:18.357791+0000","last_became_peered":"2024-09-19T08:38:18.357791+0000","last_unstale":"2024-09-19T08:38:22.085411+0000","last_undegraded":"2024-09-19T08:38:22.085411+0000","last_fullsized":"2024-09-19T08:38:22.085411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:00.838359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085487+0000","last_change":"2024-09-19T08:38:19.396783+0000","last_active":"2024-09-19T08:38:22.085487+0000","last_peered":"2024-09-19T08:38:22.085487+0000","last_clean":"2024-09-19T08:38:22.085487+0000","last_became_active":"2024-09-19T08:38:19.396012+0000","last_became_peered":"2024-09-19T08:38:19.396012+0000","last_unstale":"2024-09-19T08:38:22.085487+0000","last_undegraded":"2024-09-19T08:38:22.085487+0000","last_fullsized":"2024-09-19T08:38:22.085487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:20:35.891069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-19T08:38:18.386129+0000","last_change":"2024-09-19T08:38:18.380235+0000","last_active":"2024-09-19T08:38:18.386129+0000","last_peered":"2024-09-19T08:38:18.386129+0000","last_clean":"2024-09-19T08:38:18.386129+0000","last_became_active":"2024-09-19T08:38:18.380044+0000","last_became_peered":"2024-09-19T08:38:18.380044+0000","last_unstale":"2024-09-19T08:38:18.386129+0000","last_undegraded":"2024-09-19T08:38:18.386129+0000","last_fullsized":"2024-09-19T08:38:18.386129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:51.748538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":649760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":184320,"data_stored":59392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":184320,"data_stored":59392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509443,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":172032,"data_stored":53117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":172032,"data_stored":53117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509443,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27788,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344052,"statfs":{"total":96636764160,"available":96608309248,"internally_reserved":0,"allocated":778240,"data_stored":649760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":74079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":74079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:38:27.847 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:38:27.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.979+0000 7f275094b640 1 Processor -- start 2024-09-19T08:38:27.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.979+0000 7f275094b640 1 -- start start 2024-09-19T08:38:27.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.979+0000 7f275094b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f274c15e750 0x7f274c073c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:27.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.979+0000 7f275094b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f274c075380 0x7f274c0cf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:27.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.979+0000 7f275094b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f274c075b80 con 0x7f274c0b28e0 2024-09-19T08:38:27.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.979+0000 7f275094b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f274c0d1bd0 con 0x7f274c15e750 2024-09-19T08:38:27.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.979+0000 7f275094b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f274c0d1eb0 con 0x7f274c075380 2024-09-19T08:38:27.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.979+0000 7f274ad9d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f274c075380 0x7f274c0cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:27.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.979+0000 7f274a59c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f274c15e750 0x7f274c073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:27.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.979+0000 7f274ad9d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f274c075380 0x7f274c0cf520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:37730/0 (socket says 172.21.15.98:37730) 2024-09-19T08:38:27.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.979+0000 7f274ad9d640 1 -- 172.21.15.98:0/3723239698 learned_addr learned my addr 172.21.15.98:0/3723239698 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:27.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.979+0000 7f274ad9d640 1 -- 172.21.15.98:0/3723239698 >> v2:172.21.15.139:3300/0 conn(0x7f274c15e750 msgr2=0x7f274c073c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:27.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.979+0000 7f274ad9d640 1 --2- 172.21.15.98:0/3723239698 >> v2:172.21.15.139:3300/0 conn(0x7f274c15e750 0x7f274c073c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:27.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.979+0000 7f274ad9d640 1 -- 172.21.15.98:0/3723239698 >> v1:172.21.15.139:6789/0 conn(0x7f274c0b28e0 legacy=0x7f274c074c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:27.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.979+0000 7f274ad9d640 1 -- 172.21.15.98:0/3723239698 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274c0cf000 con 0x7f274c075380 2024-09-19T08:38:27.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.979+0000 7f274a59c640 1 --2- 172.21.15.98:0/3723239698 >> v2:172.21.15.139:3300/0 conn(0x7f274c15e750 0x7f274c073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:38:27.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.983+0000 7f274ad9d640 1 --2- 172.21.15.98:0/3723239698 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f274c075380 0x7f274c0cf520 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2744009e50 tx=0x7f274401c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8769144de9f06f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:27.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.983+0000 7f2749d9b640 1 -- 172.21.15.98:0/3723239698 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27440a5880 con 0x7f274c075380 2024-09-19T08:38:27.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.983+0000 7f2749d9b640 1 -- 172.21.15.98:0/3723239698 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27440a5a20 con 0x7f274c075380 2024-09-19T08:38:27.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.983+0000 7f2749d9b640 1 -- 172.21.15.98:0/3723239698 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27440a5df0 con 0x7f274c075380 2024-09-19T08:38:27.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.983+0000 7f275094b640 1 -- 172.21.15.98:0/3723239698 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f274c075380 msgr2=0x7f274c0cf520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:27.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.983+0000 7f275094b640 1 --2- 172.21.15.98:0/3723239698 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f274c075380 0x7f274c0cf520 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2744009e50 tx=0x7f274401c9a0 comp rx=0 tx=0).stop 2024-09-19T08:38:27.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.983+0000 7f275094b640 1 -- 172.21.15.98:0/3723239698 shutdown_connections 2024-09-19T08:38:27.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.983+0000 7f275094b640 1 --2- 172.21.15.98:0/3723239698 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f274c075380 0x7f274c0cf520 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:27.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.983+0000 7f275094b640 1 --2- 172.21.15.98:0/3723239698 >> v2:172.21.15.139:3300/0 conn(0x7f274c15e750 0x7f274c073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:27.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.983+0000 7f275094b640 1 -- 172.21.15.98:0/3723239698 >> 172.21.15.98:0/3723239698 conn(0x7f274c073170 msgr2=0x7f274c073580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:27.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.983+0000 7f275094b640 1 -- 172.21.15.98:0/3723239698 shutdown_connections 2024-09-19T08:38:27.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.983+0000 7f275094b640 1 -- 172.21.15.98:0/3723239698 wait complete. 2024-09-19T08:38:27.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.983+0000 7f275094b640 1 Processor -- start 2024-09-19T08:38:27.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.983+0000 7f275094b640 1 -- start start 2024-09-19T08:38:27.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.987+0000 7f275094b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f274c075380 0x7f274c09a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:27.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.987+0000 7f275094b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f274c0b28e0 0x7f274c15b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:27.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.987+0000 7f275094b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f274c15e750 0x7f274c15b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:27.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.987+0000 7f275094b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f274c0d0600 con 0x7f274c0b28e0 2024-09-19T08:38:27.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.987+0000 7f275094b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f274c0d0480 con 0x7f274c075380 2024-09-19T08:38:27.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.987+0000 7f275094b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f274c0d0780 con 0x7f274c15e750 2024-09-19T08:38:27.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.987+0000 7f2743fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f274c0b28e0 0x7f274c15b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:27.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.987+0000 7f2743fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f274c0b28e0 0x7f274c15b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37702/0 (socket says 172.21.15.98:37702) 2024-09-19T08:38:27.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.987+0000 7f2743fff640 1 -- 172.21.15.98:0/654917376 learned_addr learned my addr 172.21.15.98:0/654917376 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:27.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.987+0000 7f274a59c640 1 --2- 172.21.15.98:0/654917376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f274c075380 0x7f274c09a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:27.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.987+0000 7f274ad9d640 1 --2- 172.21.15.98:0/654917376 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f274c15e750 0x7f274c15b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:27.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.987+0000 7f2743fff640 1 -- 172.21.15.98:0/654917376 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f274c15e750 msgr2=0x7f274c15b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:27.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.987+0000 7f2743fff640 1 --2- 172.21.15.98:0/654917376 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f274c15e750 0x7f274c15b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:27.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.987+0000 7f2743fff640 1 -- 172.21.15.98:0/654917376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f274c075380 msgr2=0x7f274c09a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:27.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.987+0000 7f2743fff640 1 --2- 172.21.15.98:0/654917376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f274c075380 0x7f274c09a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:27.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.987+0000 7f2743fff640 1 -- 172.21.15.98:0/654917376 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274c1549b0 con 0x7f274c0b28e0 2024-09-19T08:38:27.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.987+0000 7f2743fff640 1 --2- 172.21.15.98:0/654917376 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f274c0b28e0 0x7f274c15b4b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f273400c420 tx=0x7f273400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:27.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.987+0000 7f2742ffd640 1 -- 172.21.15.98:0/654917376 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27340052d0 con 0x7f274c0b28e0 2024-09-19T08:38:27.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.987+0000 7f275094b640 1 -- 172.21.15.98:0/654917376 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f274c154c50 con 0x7f274c0b28e0 2024-09-19T08:38:27.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.987+0000 7f2742ffd640 1 -- 172.21.15.98:0/654917376 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2734005470 con 0x7f274c0b28e0 2024-09-19T08:38:27.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.987+0000 7f2742ffd640 1 -- 172.21.15.98:0/654917376 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2734007260 con 0x7f274c0b28e0 2024-09-19T08:38:27.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.987+0000 7f275094b640 1 -- 172.21.15.98:0/654917376 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f274c155210 con 0x7f274c0b28e0 2024-09-19T08:38:27.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.987+0000 7f275094b640 1 -- 172.21.15.98:0/654917376 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f274c0cf520 con 0x7f274c0b28e0 2024-09-19T08:38:27.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.987+0000 7f2742ffd640 1 -- 172.21.15.98:0/654917376 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2734007400 con 0x7f274c0b28e0 2024-09-19T08:38:27.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.987+0000 7f2742ffd640 1 --2- 172.21.15.98:0/654917376 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f272c0678c0 0x7f272c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:27.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.987+0000 7f2742ffd640 1 -- 172.21.15.98:0/654917376 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2734085160 con 0x7f274c0b28e0 2024-09-19T08:38:27.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.987+0000 7f274a59c640 1 --2- 172.21.15.98:0/654917376 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f272c0678c0 0x7f272c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:27.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.987+0000 7f274a59c640 1 --2- 172.21.15.98:0/654917376 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f272c0678c0 0x7f272c069d80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f273c009580 tx=0x7f273c009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:27.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:27.995+0000 7f2742ffd640 1 -- 172.21.15.98:0/654917376 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f27340555d0 con 0x7f274c0b28e0 2024-09-19T08:38:28.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.147+0000 7f275094b640 1 -- 172.21.15.98:0/654917376 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f274c0c7bd0 con 0x7f272c0678c0 2024-09-19T08:38:28.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.163+0000 7f2742ffd640 1 -- 172.21.15.98:0/654917376 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251939 (secure 0 0 0) 0x7f274c0c7bd0 con 0x7f272c0678c0 2024-09-19T08:38:28.168 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:38:28.170 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:38:28.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.171+0000 7f275094b640 1 -- 172.21.15.98:0/654917376 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f272c0678c0 msgr2=0x7f272c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:28.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.171+0000 7f275094b640 1 --2- 172.21.15.98:0/654917376 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f272c0678c0 0x7f272c069d80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f273c009580 tx=0x7f273c009190 comp rx=0 tx=0).stop 2024-09-19T08:38:28.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.171+0000 7f275094b640 1 -- 172.21.15.98:0/654917376 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f274c0b28e0 msgr2=0x7f274c15b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:28.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.171+0000 7f275094b640 1 --2- 172.21.15.98:0/654917376 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f274c0b28e0 0x7f274c15b4b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f273400c420 tx=0x7f273400c8e0 comp rx=0 tx=0).stop 2024-09-19T08:38:28.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.171+0000 7f275094b640 1 -- 172.21.15.98:0/654917376 shutdown_connections 2024-09-19T08:38:28.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.171+0000 7f275094b640 1 --2- 172.21.15.98:0/654917376 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f272c0678c0 0x7f272c069d80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:28.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.171+0000 7f275094b640 1 --2- 172.21.15.98:0/654917376 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f274c15e750 0x7f274c15b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:28.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.171+0000 7f275094b640 1 --2- 172.21.15.98:0/654917376 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f274c0b28e0 0x7f274c15b4b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:28.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.171+0000 7f275094b640 1 --2- 172.21.15.98:0/654917376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f274c075380 0x7f274c09a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:28.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.171+0000 7f275094b640 1 -- 172.21.15.98:0/654917376 >> 172.21.15.98:0/654917376 conn(0x7f274c073170 msgr2=0x7f274c15cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:28.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.171+0000 7f275094b640 1 -- 172.21.15.98:0/654917376 shutdown_connections 2024-09-19T08:38:28.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.171+0000 7f275094b640 1 -- 172.21.15.98:0/654917376 wait complete. 2024-09-19T08:38:28.186 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-19T08:38:28.029135+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":169,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219100,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755620,"statfs":{"total":773094113280,"available":772869754880,"internally_reserved":0,"allocated":2924544,"data_stored":1765292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.664407"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.495053+0000","last_change":"2024-09-19T08:38:19.406406+0000","last_active":"2024-09-19T08:38:21.495053+0000","last_peered":"2024-09-19T08:38:21.495053+0000","last_clean":"2024-09-19T08:38:21.495053+0000","last_became_active":"2024-09-19T08:38:19.405760+0000","last_became_peered":"2024-09-19T08:38:19.405760+0000","last_unstale":"2024-09-19T08:38:21.495053+0000","last_undegraded":"2024-09-19T08:38:21.495053+0000","last_fullsized":"2024-09-19T08:38:21.495053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:26.845220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.376816+0000","last_change":"2024-09-19T08:38:19.406176+0000","last_active":"2024-09-19T08:38:20.376816+0000","last_peered":"2024-09-19T08:38:20.376816+0000","last_clean":"2024-09-19T08:38:20.376816+0000","last_became_active":"2024-09-19T08:38:19.405604+0000","last_became_peered":"2024-09-19T08:38:19.405604+0000","last_unstale":"2024-09-19T08:38:20.376816+0000","last_undegraded":"2024-09-19T08:38:20.376816+0000","last_fullsized":"2024-09-19T08:38:20.376816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:47:44.536264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.086412+0000","last_change":"2024-09-19T08:38:19.400726+0000","last_active":"2024-09-19T08:38:22.086412+0000","last_peered":"2024-09-19T08:38:22.086412+0000","last_clean":"2024-09-19T08:38:22.086412+0000","last_became_active":"2024-09-19T08:38:19.399956+0000","last_became_peered":"2024-09-19T08:38:19.399956+0000","last_unstale":"2024-09-19T08:38:22.086412+0000","last_undegraded":"2024-09-19T08:38:22.086412+0000","last_fullsized":"2024-09-19T08:38:22.086412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:17:03.830010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.384590+0000","last_change":"2024-09-19T08:38:19.425899+0000","last_active":"2024-09-19T08:38:20.384590+0000","last_peered":"2024-09-19T08:38:20.384590+0000","last_clean":"2024-09-19T08:38:20.384590+0000","last_became_active":"2024-09-19T08:38:19.425622+0000","last_became_peered":"2024-09-19T08:38:19.425622+0000","last_unstale":"2024-09-19T08:38:20.384590+0000","last_undegraded":"2024-09-19T08:38:20.384590+0000","last_fullsized":"2024-09-19T08:38:20.384590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:47:44.536264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.494366+0000","last_change":"2024-09-19T08:38:19.406714+0000","last_active":"2024-09-19T08:38:21.494366+0000","last_peered":"2024-09-19T08:38:21.494366+0000","last_clean":"2024-09-19T08:38:21.494366+0000","last_became_active":"2024-09-19T08:38:19.404949+0000","last_became_peered":"2024-09-19T08:38:19.404949+0000","last_unstale":"2024-09-19T08:38:21.494366+0000","last_undegraded":"2024-09-19T08:38:21.494366+0000","last_fullsized":"2024-09-19T08:38:21.494366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:02:31.580648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.384530+0000","last_change":"2024-09-19T08:38:19.426925+0000","last_active":"2024-09-19T08:38:20.384530+0000","last_peered":"2024-09-19T08:38:20.384530+0000","last_clean":"2024-09-19T08:38:20.384530+0000","last_became_active":"2024-09-19T08:38:19.426666+0000","last_became_peered":"2024-09-19T08:38:19.426666+0000","last_unstale":"2024-09-19T08:38:20.384530+0000","last_undegraded":"2024-09-19T08:38:20.384530+0000","last_fullsized":"2024-09-19T08:38:20.384530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:02:54.292567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.493827+0000","last_change":"2024-09-19T08:38:19.406462+0000","last_active":"2024-09-19T08:38:21.493827+0000","last_peered":"2024-09-19T08:38:21.493827+0000","last_clean":"2024-09-19T08:38:21.493827+0000","last_became_active":"2024-09-19T08:38:19.404851+0000","last_became_peered":"2024-09-19T08:38:19.404851+0000","last_unstale":"2024-09-19T08:38:21.493827+0000","last_undegraded":"2024-09-19T08:38:21.493827+0000","last_fullsized":"2024-09-19T08:38:21.493827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:02:54.292567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.084591+0000","last_change":"2024-09-19T08:38:19.400917+0000","last_active":"2024-09-19T08:38:22.084591+0000","last_peered":"2024-09-19T08:38:22.084591+0000","last_clean":"2024-09-19T08:38:22.084591+0000","last_became_active":"2024-09-19T08:38:19.400188+0000","last_became_peered":"2024-09-19T08:38:19.400188+0000","last_unstale":"2024-09-19T08:38:22.084591+0000","last_undegraded":"2024-09-19T08:38:22.084591+0000","last_fullsized":"2024-09-19T08:38:22.084591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:24:38.033969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.495025+0000","last_change":"2024-09-19T08:38:19.432423+0000","last_active":"2024-09-19T08:38:21.495025+0000","last_peered":"2024-09-19T08:38:21.495025+0000","last_clean":"2024-09-19T08:38:21.495025+0000","last_became_active":"2024-09-19T08:38:19.431886+0000","last_became_peered":"2024-09-19T08:38:19.431886+0000","last_unstale":"2024-09-19T08:38:21.495025+0000","last_undegraded":"2024-09-19T08:38:21.495025+0000","last_fullsized":"2024-09-19T08:38:21.495025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:06.577636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.425359+0000","last_change":"2024-09-19T08:38:19.402509+0000","last_active":"2024-09-19T08:38:20.425359+0000","last_peered":"2024-09-19T08:38:20.425359+0000","last_clean":"2024-09-19T08:38:20.425359+0000","last_became_active":"2024-09-19T08:38:19.401833+0000","last_became_peered":"2024-09-19T08:38:19.401833+0000","last_unstale":"2024-09-19T08:38:20.425359+0000","last_undegraded":"2024-09-19T08:38:20.425359+0000","last_fullsized":"2024-09-19T08:38:20.425359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:08.065282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424901+0000","last_change":"2024-09-19T08:38:19.430286+0000","last_active":"2024-09-19T08:38:20.424901+0000","last_peered":"2024-09-19T08:38:20.424901+0000","last_clean":"2024-09-19T08:38:20.424901+0000","last_became_active":"2024-09-19T08:38:19.429901+0000","last_became_peered":"2024-09-19T08:38:19.429901+0000","last_unstale":"2024-09-19T08:38:20.424901+0000","last_undegraded":"2024-09-19T08:38:20.424901+0000","last_fullsized":"2024-09-19T08:38:20.424901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:31:42.999935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085451+0000","last_change":"2024-09-19T08:38:19.397867+0000","last_active":"2024-09-19T08:38:22.085451+0000","last_peered":"2024-09-19T08:38:22.085451+0000","last_clean":"2024-09-19T08:38:22.085451+0000","last_became_active":"2024-09-19T08:38:19.397106+0000","last_became_peered":"2024-09-19T08:38:19.397106+0000","last_unstale":"2024-09-19T08:38:22.085451+0000","last_undegraded":"2024-09-19T08:38:22.085451+0000","last_fullsized":"2024-09-19T08:38:22.085451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:37.286310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.086556+0000","last_change":"2024-09-19T08:38:19.396759+0000","last_active":"2024-09-19T08:38:22.086556+0000","last_peered":"2024-09-19T08:38:22.086556+0000","last_clean":"2024-09-19T08:38:22.086556+0000","last_became_active":"2024-09-19T08:38:19.395976+0000","last_became_peered":"2024-09-19T08:38:19.395976+0000","last_unstale":"2024-09-19T08:38:22.086556+0000","last_undegraded":"2024-09-19T08:38:22.086556+0000","last_fullsized":"2024-09-19T08:38:22.086556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:20.116184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.084526+0000","last_change":"2024-09-19T08:38:19.405170+0000","last_active":"2024-09-19T08:38:22.084526+0000","last_peered":"2024-09-19T08:38:22.084526+0000","last_clean":"2024-09-19T08:38:22.084526+0000","last_became_active":"2024-09-19T08:38:19.401487+0000","last_became_peered":"2024-09-19T08:38:19.401487+0000","last_unstale":"2024-09-19T08:38:22.084526+0000","last_undegraded":"2024-09-19T08:38:22.084526+0000","last_fullsized":"2024-09-19T08:38:22.084526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:40.803843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.376223+0000","last_change":"2024-09-19T08:38:19.380536+0000","last_active":"2024-09-19T08:38:20.376223+0000","last_peered":"2024-09-19T08:38:20.376223+0000","last_clean":"2024-09-19T08:38:20.376223+0000","last_became_active":"2024-09-19T08:38:19.375034+0000","last_became_peered":"2024-09-19T08:38:19.375034+0000","last_unstale":"2024-09-19T08:38:20.376223+0000","last_undegraded":"2024-09-19T08:38:20.376223+0000","last_fullsized":"2024-09-19T08:38:20.376223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:07.305805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.384258+0000","last_change":"2024-09-19T08:38:19.426216+0000","last_active":"2024-09-19T08:38:20.384258+0000","last_peered":"2024-09-19T08:38:20.384258+0000","last_clean":"2024-09-19T08:38:20.384258+0000","last_became_active":"2024-09-19T08:38:19.425926+0000","last_became_peered":"2024-09-19T08:38:19.425926+0000","last_unstale":"2024-09-19T08:38:20.384258+0000","last_undegraded":"2024-09-19T08:38:20.384258+0000","last_fullsized":"2024-09-19T08:38:20.384258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:47.390332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.385670+0000","last_change":"2024-09-19T08:38:19.427199+0000","last_active":"2024-09-19T08:38:20.385670+0000","last_peered":"2024-09-19T08:38:20.385670+0000","last_clean":"2024-09-19T08:38:20.385670+0000","last_became_active":"2024-09-19T08:38:19.425571+0000","last_became_peered":"2024-09-19T08:38:19.425571+0000","last_unstale":"2024-09-19T08:38:20.385670+0000","last_undegraded":"2024-09-19T08:38:20.385670+0000","last_fullsized":"2024-09-19T08:38:20.385670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:51:44.907657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.084606+0000","last_change":"2024-09-19T08:38:19.394500+0000","last_active":"2024-09-19T08:38:22.084606+0000","last_peered":"2024-09-19T08:38:22.084606+0000","last_clean":"2024-09-19T08:38:22.084606+0000","last_became_active":"2024-09-19T08:38:19.393782+0000","last_became_peered":"2024-09-19T08:38:19.393782+0000","last_unstale":"2024-09-19T08:38:22.084606+0000","last_undegraded":"2024-09-19T08:38:22.084606+0000","last_fullsized":"2024-09-19T08:38:22.084606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:35:09.398239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.494842+0000","last_change":"2024-09-19T08:38:19.412148+0000","last_active":"2024-09-19T08:38:21.494842+0000","last_peered":"2024-09-19T08:38:21.494842+0000","last_clean":"2024-09-19T08:38:21.494842+0000","last_became_active":"2024-09-19T08:38:19.410061+0000","last_became_peered":"2024-09-19T08:38:19.410061+0000","last_unstale":"2024-09-19T08:38:21.494842+0000","last_undegraded":"2024-09-19T08:38:21.494842+0000","last_fullsized":"2024-09-19T08:38:21.494842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:51:44.907657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.425280+0000","last_change":"2024-09-19T08:38:19.389509+0000","last_active":"2024-09-19T08:38:20.425280+0000","last_peered":"2024-09-19T08:38:20.425280+0000","last_clean":"2024-09-19T08:38:20.425280+0000","last_became_active":"2024-09-19T08:38:19.388736+0000","last_became_peered":"2024-09-19T08:38:19.388736+0000","last_unstale":"2024-09-19T08:38:20.425280+0000","last_undegraded":"2024-09-19T08:38:20.425280+0000","last_fullsized":"2024-09-19T08:38:20.425280+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:20:35.891069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.086457+0000","last_change":"2024-09-19T08:38:19.415448+0000","last_active":"2024-09-19T08:38:22.086457+0000","last_peered":"2024-09-19T08:38:22.086457+0000","last_clean":"2024-09-19T08:38:22.086457+0000","last_became_active":"2024-09-19T08:38:19.415169+0000","last_became_peered":"2024-09-19T08:38:19.415169+0000","last_unstale":"2024-09-19T08:38:22.086457+0000","last_undegraded":"2024-09-19T08:38:22.086457+0000","last_fullsized":"2024-09-19T08:38:22.086457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:18:23.928102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.384606+0000","last_change":"2024-09-19T08:38:19.429495+0000","last_active":"2024-09-19T08:38:20.384606+0000","last_peered":"2024-09-19T08:38:20.384606+0000","last_clean":"2024-09-19T08:38:20.384606+0000","last_became_active":"2024-09-19T08:38:19.427201+0000","last_became_peered":"2024-09-19T08:38:19.427201+0000","last_unstale":"2024-09-19T08:38:20.384606+0000","last_undegraded":"2024-09-19T08:38:20.384606+0000","last_fullsized":"2024-09-19T08:38:20.384606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:08.554428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.084821+0000","last_change":"2024-09-19T08:38:19.406260+0000","last_active":"2024-09-19T08:38:22.084821+0000","last_peered":"2024-09-19T08:38:22.084821+0000","last_clean":"2024-09-19T08:38:22.084821+0000","last_became_active":"2024-09-19T08:38:19.405769+0000","last_became_peered":"2024-09-19T08:38:19.405769+0000","last_unstale":"2024-09-19T08:38:22.084821+0000","last_undegraded":"2024-09-19T08:38:22.084821+0000","last_fullsized":"2024-09-19T08:38:22.084821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:31:42.999935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.384446+0000","last_change":"2024-09-19T08:38:19.402246+0000","last_active":"2024-09-19T08:38:20.384446+0000","last_peered":"2024-09-19T08:38:20.384446+0000","last_clean":"2024-09-19T08:38:20.384446+0000","last_became_active":"2024-09-19T08:38:19.401487+0000","last_became_peered":"2024-09-19T08:38:19.401487+0000","last_unstale":"2024-09-19T08:38:20.384446+0000","last_undegraded":"2024-09-19T08:38:20.384446+0000","last_fullsized":"2024-09-19T08:38:20.384446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:54:01.480300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085580+0000","last_change":"2024-09-19T08:38:19.406201+0000","last_active":"2024-09-19T08:38:22.085580+0000","last_peered":"2024-09-19T08:38:22.085580+0000","last_clean":"2024-09-19T08:38:22.085580+0000","last_became_active":"2024-09-19T08:38:19.405583+0000","last_became_peered":"2024-09-19T08:38:19.405583+0000","last_unstale":"2024-09-19T08:38:22.085580+0000","last_undegraded":"2024-09-19T08:38:22.085580+0000","last_fullsized":"2024-09-19T08:38:22.085580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:08.065282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.425374+0000","last_change":"2024-09-19T08:38:19.411842+0000","last_active":"2024-09-19T08:38:20.425374+0000","last_peered":"2024-09-19T08:38:20.425374+0000","last_clean":"2024-09-19T08:38:20.425374+0000","last_became_active":"2024-09-19T08:38:19.411092+0000","last_became_peered":"2024-09-19T08:38:19.411092+0000","last_unstale":"2024-09-19T08:38:20.425374+0000","last_undegraded":"2024-09-19T08:38:20.425374+0000","last_fullsized":"2024-09-19T08:38:20.425374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:39:22.441525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.376568+0000","last_change":"2024-09-19T08:38:19.393312+0000","last_active":"2024-09-19T08:38:20.376568+0000","last_peered":"2024-09-19T08:38:20.376568+0000","last_clean":"2024-09-19T08:38:20.376568+0000","last_became_active":"2024-09-19T08:38:19.392591+0000","last_became_peered":"2024-09-19T08:38:19.392591+0000","last_unstale":"2024-09-19T08:38:20.376568+0000","last_undegraded":"2024-09-19T08:38:20.376568+0000","last_fullsized":"2024-09-19T08:38:20.376568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:54:01.480300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085836+0000","last_change":"2024-09-19T08:38:19.408371+0000","last_active":"2024-09-19T08:38:22.085836+0000","last_peered":"2024-09-19T08:38:22.085836+0000","last_clean":"2024-09-19T08:38:22.085836+0000","last_became_active":"2024-09-19T08:38:19.407846+0000","last_became_peered":"2024-09-19T08:38:19.407846+0000","last_unstale":"2024-09-19T08:38:22.085836+0000","last_undegraded":"2024-09-19T08:38:22.085836+0000","last_fullsized":"2024-09-19T08:38:22.085836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:33.753905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.086518+0000","last_change":"2024-09-19T08:38:19.406826+0000","last_active":"2024-09-19T08:38:22.086518+0000","last_peered":"2024-09-19T08:38:22.086518+0000","last_clean":"2024-09-19T08:38:22.086518+0000","last_became_active":"2024-09-19T08:38:19.406456+0000","last_became_peered":"2024-09-19T08:38:19.406456+0000","last_unstale":"2024-09-19T08:38:22.086518+0000","last_undegraded":"2024-09-19T08:38:22.086518+0000","last_fullsized":"2024-09-19T08:38:22.086518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:32:19.829820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.384963+0000","last_change":"2024-09-19T08:38:19.430413+0000","last_active":"2024-09-19T08:38:20.384963+0000","last_peered":"2024-09-19T08:38:20.384963+0000","last_clean":"2024-09-19T08:38:20.384963+0000","last_became_active":"2024-09-19T08:38:19.430122+0000","last_became_peered":"2024-09-19T08:38:19.430122+0000","last_unstale":"2024-09-19T08:38:20.384963+0000","last_undegraded":"2024-09-19T08:38:20.384963+0000","last_fullsized":"2024-09-19T08:38:20.384963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:13:57.163258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.083388+0000","last_change":"2024-09-19T08:38:19.429646+0000","last_active":"2024-09-19T08:38:22.083388+0000","last_peered":"2024-09-19T08:38:22.083388+0000","last_clean":"2024-09-19T08:38:22.083388+0000","last_became_active":"2024-09-19T08:38:19.429256+0000","last_became_peered":"2024-09-19T08:38:19.429256+0000","last_unstale":"2024-09-19T08:38:22.083388+0000","last_undegraded":"2024-09-19T08:38:22.083388+0000","last_fullsized":"2024-09-19T08:38:22.083388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:37.286310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424603+0000","last_change":"2024-09-19T08:38:19.414469+0000","last_active":"2024-09-19T08:38:20.424603+0000","last_peered":"2024-09-19T08:38:20.424603+0000","last_clean":"2024-09-19T08:38:20.424603+0000","last_became_active":"2024-09-19T08:38:19.413995+0000","last_became_peered":"2024-09-19T08:38:19.413995+0000","last_unstale":"2024-09-19T08:38:20.424603+0000","last_undegraded":"2024-09-19T08:38:20.424603+0000","last_fullsized":"2024-09-19T08:38:20.424603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:24:38.033969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085027+0000","last_change":"2024-09-19T08:38:19.377658+0000","last_active":"2024-09-19T08:38:22.085027+0000","last_peered":"2024-09-19T08:38:22.085027+0000","last_clean":"2024-09-19T08:38:22.085027+0000","last_became_active":"2024-09-19T08:38:19.376916+0000","last_became_peered":"2024-09-19T08:38:19.376916+0000","last_unstale":"2024-09-19T08:38:22.085027+0000","last_undegraded":"2024-09-19T08:38:22.085027+0000","last_fullsized":"2024-09-19T08:38:22.085027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:21.640545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085745+0000","last_change":"2024-09-19T08:38:18.387502+0000","last_active":"2024-09-19T08:38:22.085745+0000","last_peered":"2024-09-19T08:38:22.085745+0000","last_clean":"2024-09-19T08:38:22.085745+0000","last_became_active":"2024-09-19T08:38:18.387154+0000","last_became_peered":"2024-09-19T08:38:18.387154+0000","last_unstale":"2024-09-19T08:38:22.085745+0000","last_undegraded":"2024-09-19T08:38:22.085745+0000","last_fullsized":"2024-09-19T08:38:22.085745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:53.708142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.086691+0000","last_change":"2024-09-19T08:38:19.408323+0000","last_active":"2024-09-19T08:38:22.086691+0000","last_peered":"2024-09-19T08:38:22.086691+0000","last_clean":"2024-09-19T08:38:22.086691+0000","last_became_active":"2024-09-19T08:38:19.407418+0000","last_became_peered":"2024-09-19T08:38:19.407418+0000","last_unstale":"2024-09-19T08:38:22.086691+0000","last_undegraded":"2024-09-19T08:38:22.086691+0000","last_fullsized":"2024-09-19T08:38:22.086691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:32:35.027328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424590+0000","last_change":"2024-09-19T08:38:18.381446+0000","last_active":"2024-09-19T08:38:20.424590+0000","last_peered":"2024-09-19T08:38:20.424590+0000","last_clean":"2024-09-19T08:38:20.424590+0000","last_became_active":"2024-09-19T08:38:18.380678+0000","last_became_peered":"2024-09-19T08:38:18.380678+0000","last_unstale":"2024-09-19T08:38:20.424590+0000","last_undegraded":"2024-09-19T08:38:20.424590+0000","last_fullsized":"2024-09-19T08:38:20.424590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:07:31.456618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.084969+0000","last_change":"2024-09-19T08:38:18.376428+0000","last_active":"2024-09-19T08:38:22.084969+0000","last_peered":"2024-09-19T08:38:22.084969+0000","last_clean":"2024-09-19T08:38:22.084969+0000","last_became_active":"2024-09-19T08:38:18.376015+0000","last_became_peered":"2024-09-19T08:38:18.376015+0000","last_unstale":"2024-09-19T08:38:22.084969+0000","last_undegraded":"2024-09-19T08:38:22.084969+0000","last_fullsized":"2024-09-19T08:38:22.084969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:49:33.943029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.084815+0000","last_change":"2024-09-19T08:38:19.414261+0000","last_active":"2024-09-19T08:38:22.084815+0000","last_peered":"2024-09-19T08:38:22.084815+0000","last_clean":"2024-09-19T08:38:22.084815+0000","last_became_active":"2024-09-19T08:38:19.413930+0000","last_became_peered":"2024-09-19T08:38:19.413930+0000","last_unstale":"2024-09-19T08:38:22.084815+0000","last_undegraded":"2024-09-19T08:38:22.084815+0000","last_fullsized":"2024-09-19T08:38:22.084815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:32.216306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.384529+0000","last_change":"2024-09-19T08:38:18.393405+0000","last_active":"2024-09-19T08:38:20.384529+0000","last_peered":"2024-09-19T08:38:20.384529+0000","last_clean":"2024-09-19T08:38:20.384529+0000","last_became_active":"2024-09-19T08:38:18.392613+0000","last_became_peered":"2024-09-19T08:38:18.392613+0000","last_unstale":"2024-09-19T08:38:20.384529+0000","last_undegraded":"2024-09-19T08:38:20.384529+0000","last_fullsized":"2024-09-19T08:38:20.384529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:55.536656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.376013+0000","last_change":"2024-09-19T08:38:19.397074+0000","last_active":"2024-09-19T08:38:20.376013+0000","last_peered":"2024-09-19T08:38:20.376013+0000","last_clean":"2024-09-19T08:38:20.376013+0000","last_became_active":"2024-09-19T08:38:19.396130+0000","last_became_peered":"2024-09-19T08:38:19.396130+0000","last_unstale":"2024-09-19T08:38:20.376013+0000","last_undegraded":"2024-09-19T08:38:20.376013+0000","last_fullsized":"2024-09-19T08:38:20.376013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:32:35.027328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424870+0000","last_change":"2024-09-19T08:38:18.398231+0000","last_active":"2024-09-19T08:38:20.424870+0000","last_peered":"2024-09-19T08:38:20.424870+0000","last_clean":"2024-09-19T08:38:20.424870+0000","last_became_active":"2024-09-19T08:38:18.397590+0000","last_became_peered":"2024-09-19T08:38:18.397590+0000","last_unstale":"2024-09-19T08:38:20.424870+0000","last_undegraded":"2024-09-19T08:38:20.424870+0000","last_fullsized":"2024-09-19T08:38:20.424870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:54:02.262454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.385857+0000","last_change":"2024-09-19T08:38:19.415962+0000","last_active":"2024-09-19T08:38:20.385857+0000","last_peered":"2024-09-19T08:38:20.385857+0000","last_clean":"2024-09-19T08:38:20.385857+0000","last_became_active":"2024-09-19T08:38:19.415579+0000","last_became_peered":"2024-09-19T08:38:19.415579+0000","last_unstale":"2024-09-19T08:38:20.385857+0000","last_undegraded":"2024-09-19T08:38:20.385857+0000","last_fullsized":"2024-09-19T08:38:20.385857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:06.577636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.385384+0000","last_change":"2024-09-19T08:38:18.393568+0000","last_active":"2024-09-19T08:38:20.385384+0000","last_peered":"2024-09-19T08:38:20.385384+0000","last_clean":"2024-09-19T08:38:20.385384+0000","last_became_active":"2024-09-19T08:38:18.390336+0000","last_became_peered":"2024-09-19T08:38:18.390336+0000","last_unstale":"2024-09-19T08:38:20.385384+0000","last_undegraded":"2024-09-19T08:38:20.385384+0000","last_fullsized":"2024-09-19T08:38:20.385384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:20.637060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.087146+0000","last_change":"2024-09-19T08:38:19.428816+0000","last_active":"2024-09-19T08:38:22.087146+0000","last_peered":"2024-09-19T08:38:22.087146+0000","last_clean":"2024-09-19T08:38:22.087146+0000","last_became_active":"2024-09-19T08:38:19.428391+0000","last_became_peered":"2024-09-19T08:38:19.428391+0000","last_unstale":"2024-09-19T08:38:22.087146+0000","last_undegraded":"2024-09-19T08:38:22.087146+0000","last_fullsized":"2024-09-19T08:38:22.087146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:47.390332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.376655+0000","last_change":"2024-09-19T08:38:18.390022+0000","last_active":"2024-09-19T08:38:20.376655+0000","last_peered":"2024-09-19T08:38:20.376655+0000","last_clean":"2024-09-19T08:38:20.376655+0000","last_became_active":"2024-09-19T08:38:18.389679+0000","last_became_peered":"2024-09-19T08:38:18.389679+0000","last_unstale":"2024-09-19T08:38:20.376655+0000","last_undegraded":"2024-09-19T08:38:20.376655+0000","last_fullsized":"2024-09-19T08:38:20.376655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:39.064483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085614+0000","last_change":"2024-09-19T08:38:19.397504+0000","last_active":"2024-09-19T08:38:22.085614+0000","last_peered":"2024-09-19T08:38:22.085614+0000","last_clean":"2024-09-19T08:38:22.085614+0000","last_became_active":"2024-09-19T08:38:19.396770+0000","last_became_peered":"2024-09-19T08:38:19.396770+0000","last_unstale":"2024-09-19T08:38:22.085614+0000","last_undegraded":"2024-09-19T08:38:22.085614+0000","last_fullsized":"2024-09-19T08:38:22.085614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:35:09.398239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424913+0000","last_change":"2024-09-19T08:38:18.371315+0000","last_active":"2024-09-19T08:38:20.424913+0000","last_peered":"2024-09-19T08:38:20.424913+0000","last_clean":"2024-09-19T08:38:20.424913+0000","last_became_active":"2024-09-19T08:38:18.370524+0000","last_became_peered":"2024-09-19T08:38:18.370524+0000","last_unstale":"2024-09-19T08:38:20.424913+0000","last_undegraded":"2024-09-19T08:38:20.424913+0000","last_fullsized":"2024-09-19T08:38:20.424913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:49:33.943029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.494199+0000","last_change":"2024-09-19T08:38:19.409150+0000","last_active":"2024-09-19T08:38:21.494199+0000","last_peered":"2024-09-19T08:38:21.494199+0000","last_clean":"2024-09-19T08:38:21.494199+0000","last_became_active":"2024-09-19T08:38:19.408303+0000","last_became_peered":"2024-09-19T08:38:19.408303+0000","last_unstale":"2024-09-19T08:38:21.494199+0000","last_undegraded":"2024-09-19T08:38:21.494199+0000","last_fullsized":"2024-09-19T08:38:21.494199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:13:57.163258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.086837+0000","last_change":"2024-09-19T08:38:18.366653+0000","last_active":"2024-09-19T08:38:22.086837+0000","last_peered":"2024-09-19T08:38:22.086837+0000","last_clean":"2024-09-19T08:38:22.086837+0000","last_became_active":"2024-09-19T08:38:18.366387+0000","last_became_peered":"2024-09-19T08:38:18.366387+0000","last_unstale":"2024-09-19T08:38:22.086837+0000","last_undegraded":"2024-09-19T08:38:22.086837+0000","last_fullsized":"2024-09-19T08:38:22.086837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:52.328114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.086019+0000","last_change":"2024-09-19T08:38:19.415512+0000","last_active":"2024-09-19T08:38:22.086019+0000","last_peered":"2024-09-19T08:38:22.086019+0000","last_clean":"2024-09-19T08:38:22.086019+0000","last_became_active":"2024-09-19T08:38:19.415166+0000","last_became_peered":"2024-09-19T08:38:19.415166+0000","last_unstale":"2024-09-19T08:38:22.086019+0000","last_undegraded":"2024-09-19T08:38:22.086019+0000","last_fullsized":"2024-09-19T08:38:22.086019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:22.366004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.384829+0000","last_change":"2024-09-19T08:38:18.402871+0000","last_active":"2024-09-19T08:38:20.384829+0000","last_peered":"2024-09-19T08:38:20.384829+0000","last_clean":"2024-09-19T08:38:20.384829+0000","last_became_active":"2024-09-19T08:38:18.400331+0000","last_became_peered":"2024-09-19T08:38:18.400331+0000","last_unstale":"2024-09-19T08:38:20.384829+0000","last_undegraded":"2024-09-19T08:38:20.384829+0000","last_fullsized":"2024-09-19T08:38:20.384829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:39.064483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.494370+0000","last_change":"2024-09-19T08:38:19.406507+0000","last_active":"2024-09-19T08:38:21.494370+0000","last_peered":"2024-09-19T08:38:21.494370+0000","last_clean":"2024-09-19T08:38:21.494370+0000","last_became_active":"2024-09-19T08:38:19.405133+0000","last_became_peered":"2024-09-19T08:38:19.405133+0000","last_unstale":"2024-09-19T08:38:21.494370+0000","last_undegraded":"2024-09-19T08:38:21.494370+0000","last_fullsized":"2024-09-19T08:38:21.494370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:30:08.777799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424461+0000","last_change":"2024-09-19T08:38:18.396642+0000","last_active":"2024-09-19T08:38:20.424461+0000","last_peered":"2024-09-19T08:38:20.424461+0000","last_clean":"2024-09-19T08:38:20.424461+0000","last_became_active":"2024-09-19T08:38:18.396009+0000","last_became_peered":"2024-09-19T08:38:18.396009+0000","last_unstale":"2024-09-19T08:38:20.424461+0000","last_undegraded":"2024-09-19T08:38:20.424461+0000","last_fullsized":"2024-09-19T08:38:20.424461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:21.951846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424570+0000","last_change":"2024-09-19T08:38:19.409375+0000","last_active":"2024-09-19T08:38:20.424570+0000","last_peered":"2024-09-19T08:38:20.424570+0000","last_clean":"2024-09-19T08:38:20.424570+0000","last_became_active":"2024-09-19T08:38:19.408667+0000","last_became_peered":"2024-09-19T08:38:19.408667+0000","last_unstale":"2024-09-19T08:38:20.424570+0000","last_undegraded":"2024-09-19T08:38:20.424570+0000","last_fullsized":"2024-09-19T08:38:20.424570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:32.216306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.082894+0000","last_change":"2024-09-19T08:38:18.360672+0000","last_active":"2024-09-19T08:38:22.082894+0000","last_peered":"2024-09-19T08:38:22.082894+0000","last_clean":"2024-09-19T08:38:22.082894+0000","last_became_active":"2024-09-19T08:38:18.360210+0000","last_became_peered":"2024-09-19T08:38:18.360210+0000","last_unstale":"2024-09-19T08:38:22.082894+0000","last_undegraded":"2024-09-19T08:38:22.082894+0000","last_fullsized":"2024-09-19T08:38:22.082894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:20.637060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.493833+0000","last_change":"2024-09-19T08:38:19.407114+0000","last_active":"2024-09-19T08:38:21.493833+0000","last_peered":"2024-09-19T08:38:21.493833+0000","last_clean":"2024-09-19T08:38:21.493833+0000","last_became_active":"2024-09-19T08:38:19.405734+0000","last_became_peered":"2024-09-19T08:38:19.405734+0000","last_unstale":"2024-09-19T08:38:21.493833+0000","last_undegraded":"2024-09-19T08:38:21.493833+0000","last_fullsized":"2024-09-19T08:38:21.493833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:08.554428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085716+0000","last_change":"2024-09-19T08:38:18.377025+0000","last_active":"2024-09-19T08:38:22.085716+0000","last_peered":"2024-09-19T08:38:22.085716+0000","last_clean":"2024-09-19T08:38:22.085716+0000","last_became_active":"2024-09-19T08:38:18.372574+0000","last_became_peered":"2024-09-19T08:38:18.372574+0000","last_unstale":"2024-09-19T08:38:22.085716+0000","last_undegraded":"2024-09-19T08:38:22.085716+0000","last_fullsized":"2024-09-19T08:38:22.085716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:30.341445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.494907+0000","last_change":"2024-09-19T08:38:19.405731+0000","last_active":"2024-09-19T08:38:21.494907+0000","last_peered":"2024-09-19T08:38:21.494907+0000","last_clean":"2024-09-19T08:38:21.494907+0000","last_became_active":"2024-09-19T08:38:19.404769+0000","last_became_peered":"2024-09-19T08:38:19.404769+0000","last_unstale":"2024-09-19T08:38:21.494907+0000","last_undegraded":"2024-09-19T08:38:21.494907+0000","last_fullsized":"2024-09-19T08:38:21.494907+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:21.788118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.084854+0000","last_change":"2024-09-19T08:38:18.366653+0000","last_active":"2024-09-19T08:38:22.084854+0000","last_peered":"2024-09-19T08:38:22.084854+0000","last_clean":"2024-09-19T08:38:22.084854+0000","last_became_active":"2024-09-19T08:38:18.366357+0000","last_became_peered":"2024-09-19T08:38:18.366357+0000","last_unstale":"2024-09-19T08:38:22.084854+0000","last_undegraded":"2024-09-19T08:38:22.084854+0000","last_fullsized":"2024-09-19T08:38:22.084854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:47.789309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.495241+0000","last_change":"2024-09-19T08:38:19.408858+0000","last_active":"2024-09-19T08:38:21.495241+0000","last_peered":"2024-09-19T08:38:21.495241+0000","last_clean":"2024-09-19T08:38:21.495241+0000","last_became_active":"2024-09-19T08:38:19.408122+0000","last_became_peered":"2024-09-19T08:38:19.408122+0000","last_unstale":"2024-09-19T08:38:21.495241+0000","last_undegraded":"2024-09-19T08:38:21.495241+0000","last_fullsized":"2024-09-19T08:38:21.495241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:02.493804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.376783+0000","last_change":"2024-09-19T08:38:18.366656+0000","last_active":"2024-09-19T08:38:20.376783+0000","last_peered":"2024-09-19T08:38:20.376783+0000","last_clean":"2024-09-19T08:38:20.376783+0000","last_became_active":"2024-09-19T08:38:18.366299+0000","last_became_peered":"2024-09-19T08:38:18.366299+0000","last_unstale":"2024-09-19T08:38:20.376783+0000","last_undegraded":"2024-09-19T08:38:20.376783+0000","last_fullsized":"2024-09-19T08:38:20.376783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:35:08.394753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.385063+0000","last_change":"2024-09-19T08:38:19.414578+0000","last_active":"2024-09-19T08:38:20.385063+0000","last_peered":"2024-09-19T08:38:20.385063+0000","last_clean":"2024-09-19T08:38:20.385063+0000","last_became_active":"2024-09-19T08:38:19.413677+0000","last_became_peered":"2024-09-19T08:38:19.413677+0000","last_unstale":"2024-09-19T08:38:20.385063+0000","last_undegraded":"2024-09-19T08:38:20.385063+0000","last_fullsized":"2024-09-19T08:38:20.385063+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:30:08.777799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.083724+0000","last_change":"2024-09-19T08:38:18.394865+0000","last_active":"2024-09-19T08:38:22.083724+0000","last_peered":"2024-09-19T08:38:22.083724+0000","last_clean":"2024-09-19T08:38:22.083724+0000","last_became_active":"2024-09-19T08:38:18.394496+0000","last_became_peered":"2024-09-19T08:38:18.394496+0000","last_unstale":"2024-09-19T08:38:22.083724+0000","last_undegraded":"2024-09-19T08:38:22.083724+0000","last_fullsized":"2024-09-19T08:38:22.083724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:34:58.036813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.494357+0000","last_change":"2024-09-19T08:38:19.433593+0000","last_active":"2024-09-19T08:38:21.494357+0000","last_peered":"2024-09-19T08:38:21.494357+0000","last_clean":"2024-09-19T08:38:21.494357+0000","last_became_active":"2024-09-19T08:38:19.433322+0000","last_became_peered":"2024-09-19T08:38:19.433322+0000","last_unstale":"2024-09-19T08:38:21.494357+0000","last_undegraded":"2024-09-19T08:38:21.494357+0000","last_fullsized":"2024-09-19T08:38:21.494357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:44:57.127592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.493861+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:38:21.493861+0000","last_peered":"2024-09-19T08:38:21.493861+0000","last_clean":"2024-09-19T08:38:21.493861+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:38:21.493861+0000","last_undegraded":"2024-09-19T08:38:21.493861+0000","last_fullsized":"2024-09-19T08:38:21.493861+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.384222+0000","last_change":"2024-09-19T08:38:18.391179+0000","last_active":"2024-09-19T08:38:20.384222+0000","last_peered":"2024-09-19T08:38:20.384222+0000","last_clean":"2024-09-19T08:38:20.384222+0000","last_became_active":"2024-09-19T08:38:18.390393+0000","last_became_peered":"2024-09-19T08:38:18.390393+0000","last_unstale":"2024-09-19T08:38:20.384222+0000","last_undegraded":"2024-09-19T08:38:20.384222+0000","last_fullsized":"2024-09-19T08:38:20.384222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:47.789309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085655+0000","last_change":"2024-09-19T08:38:19.430328+0000","last_active":"2024-09-19T08:38:22.085655+0000","last_peered":"2024-09-19T08:38:22.085655+0000","last_clean":"2024-09-19T08:38:22.085655+0000","last_became_active":"2024-09-19T08:38:19.429984+0000","last_became_peered":"2024-09-19T08:38:19.429984+0000","last_unstale":"2024-09-19T08:38:22.085655+0000","last_undegraded":"2024-09-19T08:38:22.085655+0000","last_fullsized":"2024-09-19T08:38:22.085655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:19:32.736687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.493590+0000","last_change":"2024-09-19T08:38:18.384759+0000","last_active":"2024-09-19T08:38:21.493590+0000","last_peered":"2024-09-19T08:38:21.493590+0000","last_clean":"2024-09-19T08:38:21.493590+0000","last_became_active":"2024-09-19T08:38:18.383974+0000","last_became_peered":"2024-09-19T08:38:18.383974+0000","last_unstale":"2024-09-19T08:38:21.493590+0000","last_undegraded":"2024-09-19T08:38:21.493590+0000","last_fullsized":"2024-09-19T08:38:21.493590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:39.064483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.084012+0000","last_change":"2024-09-19T08:38:19.412025+0000","last_active":"2024-09-19T08:38:22.084012+0000","last_peered":"2024-09-19T08:38:22.084012+0000","last_clean":"2024-09-19T08:38:22.084012+0000","last_became_active":"2024-09-19T08:38:19.411701+0000","last_became_peered":"2024-09-19T08:38:19.411701+0000","last_unstale":"2024-09-19T08:38:22.084012+0000","last_undegraded":"2024-09-19T08:38:22.084012+0000","last_fullsized":"2024-09-19T08:38:22.084012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:13:02.635546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.083685+0000","last_change":"2024-09-19T08:38:18.366746+0000","last_active":"2024-09-19T08:38:22.083685+0000","last_peered":"2024-09-19T08:38:22.083685+0000","last_clean":"2024-09-19T08:38:22.083685+0000","last_became_active":"2024-09-19T08:38:18.366346+0000","last_became_peered":"2024-09-19T08:38:18.366346+0000","last_unstale":"2024-09-19T08:38:22.083685+0000","last_undegraded":"2024-09-19T08:38:22.083685+0000","last_fullsized":"2024-09-19T08:38:22.083685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:55.536656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.376256+0000","last_change":"2024-09-19T08:38:19.395656+0000","last_active":"2024-09-19T08:38:20.376256+0000","last_peered":"2024-09-19T08:38:20.376256+0000","last_clean":"2024-09-19T08:38:20.376256+0000","last_became_active":"2024-09-19T08:38:19.393805+0000","last_became_peered":"2024-09-19T08:38:19.393805+0000","last_unstale":"2024-09-19T08:38:20.376256+0000","last_undegraded":"2024-09-19T08:38:20.376256+0000","last_fullsized":"2024-09-19T08:38:20.376256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:08.505022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.385101+0000","last_change":"2024-09-19T08:38:18.398888+0000","last_active":"2024-09-19T08:38:20.385101+0000","last_peered":"2024-09-19T08:38:20.385101+0000","last_clean":"2024-09-19T08:38:20.385101+0000","last_became_active":"2024-09-19T08:38:18.391135+0000","last_became_peered":"2024-09-19T08:38:18.391135+0000","last_unstale":"2024-09-19T08:38:20.385101+0000","last_undegraded":"2024-09-19T08:38:20.385101+0000","last_fullsized":"2024-09-19T08:38:20.385101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:48.426450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085253+0000","last_change":"2024-09-19T08:38:19.418136+0000","last_active":"2024-09-19T08:38:22.085253+0000","last_peered":"2024-09-19T08:38:22.085253+0000","last_clean":"2024-09-19T08:38:22.085253+0000","last_became_active":"2024-09-19T08:38:19.411375+0000","last_became_peered":"2024-09-19T08:38:19.411375+0000","last_unstale":"2024-09-19T08:38:22.085253+0000","last_undegraded":"2024-09-19T08:38:22.085253+0000","last_fullsized":"2024-09-19T08:38:22.085253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:40.067968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.376493+0000","last_change":"2024-09-19T08:38:18.363154+0000","last_active":"2024-09-19T08:38:20.376493+0000","last_peered":"2024-09-19T08:38:20.376493+0000","last_clean":"2024-09-19T08:38:20.376493+0000","last_became_active":"2024-09-19T08:38:18.362876+0000","last_became_peered":"2024-09-19T08:38:18.362876+0000","last_unstale":"2024-09-19T08:38:20.376493+0000","last_undegraded":"2024-09-19T08:38:20.376493+0000","last_fullsized":"2024-09-19T08:38:20.376493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:48.426450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.385738+0000","last_change":"2024-09-19T08:38:19.413745+0000","last_active":"2024-09-19T08:38:20.385738+0000","last_peered":"2024-09-19T08:38:20.385738+0000","last_clean":"2024-09-19T08:38:20.385738+0000","last_became_active":"2024-09-19T08:38:19.413092+0000","last_became_peered":"2024-09-19T08:38:19.413092+0000","last_unstale":"2024-09-19T08:38:20.385738+0000","last_undegraded":"2024-09-19T08:38:20.385738+0000","last_fullsized":"2024-09-19T08:38:20.385738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:21.788118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.494741+0000","last_change":"2024-09-19T08:38:18.389212+0000","last_active":"2024-09-19T08:38:21.494741+0000","last_peered":"2024-09-19T08:38:21.494741+0000","last_clean":"2024-09-19T08:38:21.494741+0000","last_became_active":"2024-09-19T08:38:18.388588+0000","last_became_peered":"2024-09-19T08:38:18.388588+0000","last_unstale":"2024-09-19T08:38:21.494741+0000","last_undegraded":"2024-09-19T08:38:21.494741+0000","last_fullsized":"2024-09-19T08:38:21.494741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:46:34.417531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.376388+0000","last_change":"2024-09-19T08:38:19.386583+0000","last_active":"2024-09-19T08:38:20.376388+0000","last_peered":"2024-09-19T08:38:20.376388+0000","last_clean":"2024-09-19T08:38:20.376388+0000","last_became_active":"2024-09-19T08:38:19.385749+0000","last_became_peered":"2024-09-19T08:38:19.385749+0000","last_unstale":"2024-09-19T08:38:20.376388+0000","last_undegraded":"2024-09-19T08:38:20.376388+0000","last_fullsized":"2024-09-19T08:38:20.376388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:47.390332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.376949+0000","last_change":"2024-09-19T08:38:18.377131+0000","last_active":"2024-09-19T08:38:20.376949+0000","last_peered":"2024-09-19T08:38:20.376949+0000","last_clean":"2024-09-19T08:38:20.376949+0000","last_became_active":"2024-09-19T08:38:18.376860+0000","last_became_peered":"2024-09-19T08:38:18.376860+0000","last_unstale":"2024-09-19T08:38:20.376949+0000","last_undegraded":"2024-09-19T08:38:20.376949+0000","last_fullsized":"2024-09-19T08:38:20.376949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:46:34.417531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085974+0000","last_change":"2024-09-19T08:38:19.410025+0000","last_active":"2024-09-19T08:38:22.085974+0000","last_peered":"2024-09-19T08:38:22.085974+0000","last_clean":"2024-09-19T08:38:22.085974+0000","last_became_active":"2024-09-19T08:38:19.409390+0000","last_became_peered":"2024-09-19T08:38:19.409390+0000","last_unstale":"2024-09-19T08:38:22.085974+0000","last_undegraded":"2024-09-19T08:38:22.085974+0000","last_fullsized":"2024-09-19T08:38:22.085974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:10.400137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.425026+0000","last_change":"2024-09-19T08:38:18.392567+0000","last_active":"2024-09-19T08:38:20.425026+0000","last_peered":"2024-09-19T08:38:20.425026+0000","last_clean":"2024-09-19T08:38:20.425026+0000","last_became_active":"2024-09-19T08:38:18.387256+0000","last_became_peered":"2024-09-19T08:38:18.387256+0000","last_unstale":"2024-09-19T08:38:20.425026+0000","last_undegraded":"2024-09-19T08:38:20.425026+0000","last_fullsized":"2024-09-19T08:38:20.425026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:12:57.094603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.087052+0000","last_change":"2024-09-19T08:38:19.406115+0000","last_active":"2024-09-19T08:38:22.087052+0000","last_peered":"2024-09-19T08:38:22.087052+0000","last_clean":"2024-09-19T08:38:22.087052+0000","last_became_active":"2024-09-19T08:38:19.401366+0000","last_became_peered":"2024-09-19T08:38:19.401366+0000","last_unstale":"2024-09-19T08:38:22.087052+0000","last_undegraded":"2024-09-19T08:38:22.087052+0000","last_fullsized":"2024-09-19T08:38:22.087052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:08.505022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.493561+0000","last_change":"2024-09-19T08:38:18.384552+0000","last_active":"2024-09-19T08:38:21.493561+0000","last_peered":"2024-09-19T08:38:21.493561+0000","last_clean":"2024-09-19T08:38:21.493561+0000","last_became_active":"2024-09-19T08:38:18.383834+0000","last_became_peered":"2024-09-19T08:38:18.383834+0000","last_unstale":"2024-09-19T08:38:21.493561+0000","last_undegraded":"2024-09-19T08:38:21.493561+0000","last_fullsized":"2024-09-19T08:38:21.493561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:20.637060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085218+0000","last_change":"2024-09-19T08:38:19.395111+0000","last_active":"2024-09-19T08:38:22.085218+0000","last_peered":"2024-09-19T08:38:22.085218+0000","last_clean":"2024-09-19T08:38:22.085218+0000","last_became_active":"2024-09-19T08:38:19.394398+0000","last_became_peered":"2024-09-19T08:38:19.394398+0000","last_unstale":"2024-09-19T08:38:22.085218+0000","last_undegraded":"2024-09-19T08:38:22.085218+0000","last_fullsized":"2024-09-19T08:38:22.085218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:49.429936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.376132+0000","last_change":"2024-09-19T08:38:18.378231+0000","last_active":"2024-09-19T08:38:20.376132+0000","last_peered":"2024-09-19T08:38:20.376132+0000","last_clean":"2024-09-19T08:38:20.376132+0000","last_became_active":"2024-09-19T08:38:18.377883+0000","last_became_peered":"2024-09-19T08:38:18.377883+0000","last_unstale":"2024-09-19T08:38:20.376132+0000","last_undegraded":"2024-09-19T08:38:20.376132+0000","last_fullsized":"2024-09-19T08:38:20.376132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:36.282825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085006+0000","last_change":"2024-09-19T08:38:19.392650+0000","last_active":"2024-09-19T08:38:22.085006+0000","last_peered":"2024-09-19T08:38:22.085006+0000","last_clean":"2024-09-19T08:38:22.085006+0000","last_became_active":"2024-09-19T08:38:19.391936+0000","last_became_peered":"2024-09-19T08:38:19.391936+0000","last_unstale":"2024-09-19T08:38:22.085006+0000","last_undegraded":"2024-09-19T08:38:22.085006+0000","last_fullsized":"2024-09-19T08:38:22.085006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:42:24.499497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.376387+0000","last_change":"2024-09-19T08:38:18.359251+0000","last_active":"2024-09-19T08:38:20.376387+0000","last_peered":"2024-09-19T08:38:20.376387+0000","last_clean":"2024-09-19T08:38:20.376387+0000","last_became_active":"2024-09-19T08:38:18.358873+0000","last_became_peered":"2024-09-19T08:38:18.358873+0000","last_unstale":"2024-09-19T08:38:20.376387+0000","last_undegraded":"2024-09-19T08:38:20.376387+0000","last_fullsized":"2024-09-19T08:38:20.376387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:20.637060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085621+0000","last_change":"2024-09-19T08:38:19.414125+0000","last_active":"2024-09-19T08:38:22.085621+0000","last_peered":"2024-09-19T08:38:22.085621+0000","last_clean":"2024-09-19T08:38:22.085621+0000","last_became_active":"2024-09-19T08:38:19.413714+0000","last_became_peered":"2024-09-19T08:38:19.413714+0000","last_unstale":"2024-09-19T08:38:22.085621+0000","last_undegraded":"2024-09-19T08:38:22.085621+0000","last_fullsized":"2024-09-19T08:38:22.085621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:39:22.441525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.494666+0000","last_change":"2024-09-19T08:38:18.387311+0000","last_active":"2024-09-19T08:38:21.494666+0000","last_peered":"2024-09-19T08:38:21.494666+0000","last_clean":"2024-09-19T08:38:21.494666+0000","last_became_active":"2024-09-19T08:38:18.386340+0000","last_became_peered":"2024-09-19T08:38:18.386340+0000","last_unstale":"2024-09-19T08:38:21.494666+0000","last_undegraded":"2024-09-19T08:38:21.494666+0000","last_fullsized":"2024-09-19T08:38:21.494666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:36.282825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424982+0000","last_change":"2024-09-19T08:38:19.407976+0000","last_active":"2024-09-19T08:38:20.424982+0000","last_peered":"2024-09-19T08:38:20.424982+0000","last_clean":"2024-09-19T08:38:20.424982+0000","last_became_active":"2024-09-19T08:38:19.407198+0000","last_became_peered":"2024-09-19T08:38:19.407198+0000","last_unstale":"2024-09-19T08:38:20.424982+0000","last_undegraded":"2024-09-19T08:38:20.424982+0000","last_fullsized":"2024-09-19T08:38:20.424982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:42:24.499497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.385369+0000","last_change":"2024-09-19T08:38:19.413947+0000","last_active":"2024-09-19T08:38:20.385369+0000","last_peered":"2024-09-19T08:38:20.385369+0000","last_clean":"2024-09-19T08:38:20.385369+0000","last_became_active":"2024-09-19T08:38:19.413225+0000","last_became_peered":"2024-09-19T08:38:19.413225+0000","last_unstale":"2024-09-19T08:38:20.385369+0000","last_undegraded":"2024-09-19T08:38:20.385369+0000","last_fullsized":"2024-09-19T08:38:20.385369+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:02:31.580648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.494149+0000","last_change":"2024-09-19T08:38:18.400304+0000","last_active":"2024-09-19T08:38:21.494149+0000","last_peered":"2024-09-19T08:38:21.494149+0000","last_clean":"2024-09-19T08:38:21.494149+0000","last_became_active":"2024-09-19T08:38:18.399609+0000","last_became_peered":"2024-09-19T08:38:18.399609+0000","last_unstale":"2024-09-19T08:38:21.494149+0000","last_undegraded":"2024-09-19T08:38:21.494149+0000","last_fullsized":"2024-09-19T08:38:21.494149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:48.426450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.385061+0000","last_change":"2024-09-19T08:38:19.429826+0000","last_active":"2024-09-19T08:38:20.385061+0000","last_peered":"2024-09-19T08:38:20.385061+0000","last_clean":"2024-09-19T08:38:20.385061+0000","last_became_active":"2024-09-19T08:38:19.429543+0000","last_became_peered":"2024-09-19T08:38:19.429543+0000","last_unstale":"2024-09-19T08:38:20.385061+0000","last_undegraded":"2024-09-19T08:38:20.385061+0000","last_fullsized":"2024-09-19T08:38:20.385061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:44:57.127592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.494455+0000","last_change":"2024-09-19T08:38:18.384861+0000","last_active":"2024-09-19T08:38:21.494455+0000","last_peered":"2024-09-19T08:38:21.494455+0000","last_clean":"2024-09-19T08:38:21.494455+0000","last_became_active":"2024-09-19T08:38:18.384070+0000","last_became_peered":"2024-09-19T08:38:18.384070+0000","last_unstale":"2024-09-19T08:38:21.494455+0000","last_undegraded":"2024-09-19T08:38:21.494455+0000","last_fullsized":"2024-09-19T08:38:21.494455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:35:08.394753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.087092+0000","last_change":"2024-09-19T08:38:19.430701+0000","last_active":"2024-09-19T08:38:22.087092+0000","last_peered":"2024-09-19T08:38:22.087092+0000","last_clean":"2024-09-19T08:38:22.087092+0000","last_became_active":"2024-09-19T08:38:19.430335+0000","last_became_peered":"2024-09-19T08:38:19.430335+0000","last_unstale":"2024-09-19T08:38:22.087092+0000","last_undegraded":"2024-09-19T08:38:22.087092+0000","last_fullsized":"2024-09-19T08:38:22.087092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:07.305805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085411+0000","last_change":"2024-09-19T08:38:18.358117+0000","last_active":"2024-09-19T08:38:22.085411+0000","last_peered":"2024-09-19T08:38:22.085411+0000","last_clean":"2024-09-19T08:38:22.085411+0000","last_became_active":"2024-09-19T08:38:18.357791+0000","last_became_peered":"2024-09-19T08:38:18.357791+0000","last_unstale":"2024-09-19T08:38:22.085411+0000","last_undegraded":"2024-09-19T08:38:22.085411+0000","last_fullsized":"2024-09-19T08:38:22.085411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:00.838359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085487+0000","last_change":"2024-09-19T08:38:19.396783+0000","last_active":"2024-09-19T08:38:22.085487+0000","last_peered":"2024-09-19T08:38:22.085487+0000","last_clean":"2024-09-19T08:38:22.085487+0000","last_became_active":"2024-09-19T08:38:19.396012+0000","last_became_peered":"2024-09-19T08:38:19.396012+0000","last_unstale":"2024-09-19T08:38:22.085487+0000","last_undegraded":"2024-09-19T08:38:22.085487+0000","last_fullsized":"2024-09-19T08:38:22.085487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:20:35.891069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.087284+0000","last_change":"2024-09-19T08:38:18.380235+0000","last_active":"2024-09-19T08:38:22.087284+0000","last_peered":"2024-09-19T08:38:22.087284+0000","last_clean":"2024-09-19T08:38:22.087284+0000","last_became_active":"2024-09-19T08:38:18.380044+0000","last_became_peered":"2024-09-19T08:38:18.380044+0000","last_unstale":"2024-09-19T08:38:22.087284+0000","last_undegraded":"2024-09-19T08:38:22.087284+0000","last_fullsized":"2024-09-19T08:38:22.087284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:30.341445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":806912,"data_stored":663417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":72557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":72593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":72467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":217088,"data_stored":72627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":815104,"data_stored":663473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":74079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":74079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:38:28.190 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T08:38:28.191 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-19T08:38:28.191 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-19T08:38:28.191 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-19T08:38:28.192 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-19T08:38:28.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.335+0000 7f14a7686640 1 Processor -- start 2024-09-19T08:38:28.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.335+0000 7f14a7686640 1 -- start start 2024-09-19T08:38:28.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.339+0000 7f14a7686640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14a015c080 0x7f14a0073960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:28.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.339+0000 7f14a7686640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f14a00b7300 0x7f14a00b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:28.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.339+0000 7f14a7686640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14a015fcf0 con 0x7f14a0073ea0 2024-09-19T08:38:28.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.339+0000 7f14a7686640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f14a00742e0 con 0x7f14a00b7300 2024-09-19T08:38:28.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.339+0000 7f14a7686640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f14a015ffd0 con 0x7f14a015c080 2024-09-19T08:38:28.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.339+0000 7f14a5c23640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f14a00b7300 0x7f14a00b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:28.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.339+0000 7f14a5c23640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f14a00b7300 0x7f14a00b53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37714/0 (socket says 172.21.15.98:37714) 2024-09-19T08:38:28.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.339+0000 7f14a5c23640 1 -- 172.21.15.98:0/3962393124 learned_addr learned my addr 172.21.15.98:0/3962393124 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:28.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.339+0000 7f14a5422640 1 --2- 172.21.15.98:0/3962393124 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14a015c080 0x7f14a0073960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:28.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.339+0000 7f14a5c23640 1 -- 172.21.15.98:0/3962393124 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14a015c080 msgr2=0x7f14a0073960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:28.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.339+0000 7f14a5c23640 1 --2- 172.21.15.98:0/3962393124 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14a015c080 0x7f14a0073960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:28.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.339+0000 7f14a5c23640 1 -- 172.21.15.98:0/3962393124 >> v1:172.21.15.139:6789/0 conn(0x7f14a0073ea0 legacy=0x7f14a00b6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:28.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.339+0000 7f14a5c23640 1 -- 172.21.15.98:0/3962393124 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a01602b0 con 0x7f14a00b7300 2024-09-19T08:38:28.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.339+0000 7f14a5c23640 1 --2- 172.21.15.98:0/3962393124 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14a00b7300 0x7f14a00b53a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1498009e50 tx=0x7f149801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4641b2d8ed840c64 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:28.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.339+0000 7f1497fff640 1 -- 172.21.15.98:0/3962393124 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14980a5760 con 0x7f14a00b7300 2024-09-19T08:38:28.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.339+0000 7f1497fff640 1 -- 172.21.15.98:0/3962393124 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14980a5900 con 0x7f14a00b7300 2024-09-19T08:38:28.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.339+0000 7f1497fff640 1 -- 172.21.15.98:0/3962393124 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14980a5cd0 con 0x7f14a00b7300 2024-09-19T08:38:28.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.339+0000 7f14a7686640 1 -- 172.21.15.98:0/3962393124 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14a00b7300 msgr2=0x7f14a00b53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:28.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.339+0000 7f14a7686640 1 --2- 172.21.15.98:0/3962393124 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14a00b7300 0x7f14a00b53a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1498009e50 tx=0x7f149801c9f0 comp rx=0 tx=0).stop 2024-09-19T08:38:28.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.343+0000 7f14a7686640 1 -- 172.21.15.98:0/3962393124 shutdown_connections 2024-09-19T08:38:28.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.343+0000 7f14a7686640 1 --2- 172.21.15.98:0/3962393124 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14a00b7300 0x7f14a00b53a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:28.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.343+0000 7f14a7686640 1 --2- 172.21.15.98:0/3962393124 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14a015c080 0x7f14a0073960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:28.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.343+0000 7f14a7686640 1 -- 172.21.15.98:0/3962393124 >> 172.21.15.98:0/3962393124 conn(0x7f14a00b40a0 msgr2=0x7f14a00b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:28.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.343+0000 7f14a7686640 1 -- 172.21.15.98:0/3962393124 shutdown_connections 2024-09-19T08:38:28.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.343+0000 7f14a7686640 1 -- 172.21.15.98:0/3962393124 wait complete. 2024-09-19T08:38:28.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.343+0000 7f14a7686640 1 Processor -- start 2024-09-19T08:38:28.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.343+0000 7f14a7686640 1 -- start start 2024-09-19T08:38:28.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.343+0000 7f14a7686640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14a0073ea0 0x7f14a013dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:28.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.343+0000 7f14a7686640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14a00b7300 0x7f14a01549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:28.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.343+0000 7f14a7686640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14a015c080 0x7f14a0154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:28.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.343+0000 7f14a7686640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f14a00b6270 con 0x7f14a00b7300 2024-09-19T08:38:28.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.343+0000 7f14a7686640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f14a00b60f0 con 0x7f14a015c080 2024-09-19T08:38:28.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.343+0000 7f14a7686640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f14a00b63f0 con 0x7f14a0073ea0 2024-09-19T08:38:28.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.343+0000 7f14a4c21640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14a00b7300 0x7f14a01549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:28.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.343+0000 7f14a4c21640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14a00b7300 0x7f14a01549e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37718/0 (socket says 172.21.15.98:37718) 2024-09-19T08:38:28.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.343+0000 7f14a4c21640 1 -- 172.21.15.98:0/72411211 learned_addr learned my addr 172.21.15.98:0/72411211 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:28.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.343+0000 7f14a5422640 1 --2- 172.21.15.98:0/72411211 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14a0073ea0 0x7f14a013dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:28.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.343+0000 7f14a5c23640 1 --2- 172.21.15.98:0/72411211 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14a015c080 0x7f14a0154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:28.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.343+0000 7f14a4c21640 1 -- 172.21.15.98:0/72411211 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14a0073ea0 msgr2=0x7f14a013dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:28.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.343+0000 7f14a4c21640 1 --2- 172.21.15.98:0/72411211 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14a0073ea0 0x7f14a013dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:28.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.343+0000 7f14a4c21640 1 -- 172.21.15.98:0/72411211 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14a015c080 msgr2=0x7f14a0154f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:28.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.343+0000 7f14a4c21640 1 --2- 172.21.15.98:0/72411211 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14a015c080 0x7f14a0154f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:28.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.343+0000 7f14a4c21640 1 -- 172.21.15.98:0/72411211 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a0152dd0 con 0x7f14a00b7300 2024-09-19T08:38:28.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.343+0000 7f14a5c23640 1 --2- 172.21.15.98:0/72411211 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14a015c080 0x7f14a0154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:28.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.343+0000 7f14a4c21640 1 --2- 172.21.15.98:0/72411211 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14a00b7300 0x7f14a01549e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f149000c430 tx=0x7f149000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:28.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.343+0000 7f1495ffb640 1 -- 172.21.15.98:0/72411211 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1490011790 con 0x7f14a00b7300 2024-09-19T08:38:28.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.343+0000 7f1495ffb640 1 -- 172.21.15.98:0/72411211 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1490017070 con 0x7f14a00b7300 2024-09-19T08:38:28.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.343+0000 7f14a7686640 1 -- 172.21.15.98:0/72411211 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14a01530b0 con 0x7f14a00b7300 2024-09-19T08:38:28.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.343+0000 7f1495ffb640 1 -- 172.21.15.98:0/72411211 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1490011b70 con 0x7f14a00b7300 2024-09-19T08:38:28.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.343+0000 7f14a7686640 1 -- 172.21.15.98:0/72411211 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14a0153670 con 0x7f14a00b7300 2024-09-19T08:38:28.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.343+0000 7f14a7686640 1 -- 172.21.15.98:0/72411211 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14a0152b60 con 0x7f14a00b7300 2024-09-19T08:38:28.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.347+0000 7f1495ffb640 1 -- 172.21.15.98:0/72411211 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1490011d10 con 0x7f14a00b7300 2024-09-19T08:38:28.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.347+0000 7f1495ffb640 1 --2- 172.21.15.98:0/72411211 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f14840678c0 0x7f1484069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:28.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.347+0000 7f14a5422640 1 --2- 172.21.15.98:0/72411211 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f14840678c0 0x7f1484069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:28.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.347+0000 7f1495ffb640 1 -- 172.21.15.98:0/72411211 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1490085760 con 0x7f14a00b7300 2024-09-19T08:38:28.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.347+0000 7f14a5422640 1 --2- 172.21.15.98:0/72411211 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f14840678c0 0x7f1484069d80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f14a00b57f0 tx=0x7f149c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:28.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.351+0000 7f1495ffb640 1 -- 172.21.15.98:0/72411211 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f14900562a0 con 0x7f14a00b7300 2024-09-19T08:38:28.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.555+0000 7f14a7686640 1 -- 172.21.15.98:0/72411211 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f14a00b53a0 con 0x7f14a00b7300 2024-09-19T08:38:28.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.559+0000 7f1495ffb640 1 -- 172.21.15.98:0/72411211 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+166887 (secure 0 0 0) 0x7f1490055ba0 con 0x7f14a00b7300 2024-09-19T08:38:28.561 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:38:28.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.563+0000 7f14a7686640 1 -- 172.21.15.98:0/72411211 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f14840678c0 msgr2=0x7f1484069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:28.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.563+0000 7f14a7686640 1 --2- 172.21.15.98:0/72411211 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f14840678c0 0x7f1484069d80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f14a00b57f0 tx=0x7f149c008040 comp rx=0 tx=0).stop 2024-09-19T08:38:28.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.563+0000 7f14a7686640 1 -- 172.21.15.98:0/72411211 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14a00b7300 msgr2=0x7f14a01549e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:28.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.563+0000 7f14a7686640 1 --2- 172.21.15.98:0/72411211 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14a00b7300 0x7f14a01549e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f149000c430 tx=0x7f149000c8f0 comp rx=0 tx=0).stop 2024-09-19T08:38:28.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.563+0000 7f14a7686640 1 -- 172.21.15.98:0/72411211 shutdown_connections 2024-09-19T08:38:28.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.563+0000 7f14a7686640 1 --2- 172.21.15.98:0/72411211 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f14840678c0 0x7f1484069d80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:28.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.563+0000 7f14a7686640 1 --2- 172.21.15.98:0/72411211 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14a015c080 0x7f14a0154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:28.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.563+0000 7f14a7686640 1 --2- 172.21.15.98:0/72411211 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14a00b7300 0x7f14a01549e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:28.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.563+0000 7f14a7686640 1 --2- 172.21.15.98:0/72411211 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14a0073ea0 0x7f14a013dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:28.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.563+0000 7f14a7686640 1 -- 172.21.15.98:0/72411211 >> 172.21.15.98:0/72411211 conn(0x7f14a00b40a0 msgr2=0x7f14a0070550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:28.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.563+0000 7f14a7686640 1 -- 172.21.15.98:0/72411211 shutdown_connections 2024-09-19T08:38:28.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.563+0000 7f14a7686640 1 -- 172.21.15.98:0/72411211 wait complete. 2024-09-19T08:38:28.579 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2040310606},{"type":"v1","addr":"172.21.15.98:6833","nonce":2040310606}]},"active_addr":"172.21.15.98:6833/2040310606","active_change":"2024-09-19T08:38:00.016702+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4106,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.98:0","nonce":2131240070}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.98:0","nonce":1465974372}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.98:0","nonce":1109131925}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.98:0","nonce":3519546751}]}]} 2024-09-19T08:38:28.582 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-19T08:38:28.582 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-19T08:38:28.583 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:38:28.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.695+0000 7fd2be822640 1 Processor -- start 2024-09-19T08:38:28.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.695+0000 7fd2be822640 1 -- start start 2024-09-19T08:38:28.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.695+0000 7fd2be822640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd2b80733b0 0x7fd2b80cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:28.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.695+0000 7fd2be822640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2b806d310 con 0x7fd2b806c3a0 2024-09-19T08:38:28.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.695+0000 7fd2be822640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2b806d5f0 con 0x7fd2b815c550 2024-09-19T08:38:28.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.695+0000 7fd2be822640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd2b806d8d0 con 0x7fd2b80733b0 2024-09-19T08:38:28.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.695+0000 7fd2b77fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fd2b806c3a0 0x7fd2b80cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:60516/0 (socket says 172.21.15.98:60516) 2024-09-19T08:38:28.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.695+0000 7fd2b77fe640 1 -- 172.21.15.98:0/2675502964 learned_addr learned my addr 172.21.15.98:0/2675502964 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:28.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.695+0000 7fd2b6ffd640 1 -- 172.21.15.98:0/2675502964 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1228083868 0 0) 0x7fd2b806d310 con 0x7fd2b806c3a0 2024-09-19T08:38:28.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.695+0000 7fd2b6ffd640 1 -- 172.21.15.98:0/2675502964 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd2a4003530 con 0x7fd2b806c3a0 2024-09-19T08:38:28.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.695+0000 7fd2bcdbf640 1 --2- 172.21.15.98:0/2675502964 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd2b80733b0 0x7fd2b80cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:28.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.695+0000 7fd2b6ffd640 1 -- 172.21.15.98:0/2675502964 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4052485406 0 0) 0x7fd2a4003530 con 0x7fd2b806c3a0 2024-09-19T08:38:28.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.695+0000 7fd2b6ffd640 1 -- 172.21.15.98:0/2675502964 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd2b80733b0 msgr2=0x7fd2b80cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:28.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.695+0000 7fd2b6ffd640 1 --2- 172.21.15.98:0/2675502964 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd2b80733b0 0x7fd2b80cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:28.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.695+0000 7fd2b6ffd640 1 -- 172.21.15.98:0/2675502964 >> v1:172.21.15.139:6789/0 conn(0x7fd2b815c550 legacy=0x7fd2b806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:28.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.695+0000 7fd2b6ffd640 1 -- 172.21.15.98:0/2675502964 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2b806dbb0 con 0x7fd2b806c3a0 2024-09-19T08:38:28.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.695+0000 7fd2b6ffd640 1 -- 172.21.15.98:0/2675502964 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fd2a0001680 con 0x7fd2b806c3a0 2024-09-19T08:38:28.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.695+0000 7fd2b6ffd640 1 -- 172.21.15.98:0/2675502964 >> v1:172.21.15.98:6789/0 conn(0x7fd2b806c3a0 legacy=0x7fd2b80cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:28.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.695+0000 7fd2b6ffd640 1 --2- 172.21.15.98:0/2675502964 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2a40039f0 0x7fd2a4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:28.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.695+0000 7fd2b6ffd640 1 --2- 172.21.15.98:0/2675502964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2a4004810 0x7fd2a4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:28.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.695+0000 7fd2b6ffd640 1 --2- 172.21.15.98:0/2675502964 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd2b80733b0 0x7fd2a4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:28.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.695+0000 7fd2b6ffd640 1 -- 172.21.15.98:0/2675502964 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b806d8d0 con 0x7fd2a40039f0 2024-09-19T08:38:28.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.695+0000 7fd2b6ffd640 1 -- 172.21.15.98:0/2675502964 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a400bb00 con 0x7fd2a4004810 2024-09-19T08:38:28.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.695+0000 7fd2b6ffd640 1 -- 172.21.15.98:0/2675502964 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd2a400bd00 con 0x7fd2b80733b0 2024-09-19T08:38:28.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.695+0000 7fd2b7fff640 1 --2- 172.21.15.98:0/2675502964 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2a40039f0 0x7fd2a4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:28.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.695+0000 7fd2bcdbf640 1 --2- 172.21.15.98:0/2675502964 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd2b80733b0 0x7fd2a4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:28.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.695+0000 7fd2b77fe640 1 --2- 172.21.15.98:0/2675502964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2a4004810 0x7fd2a4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:28.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.695+0000 7fd2b7fff640 1 -- 172.21.15.98:0/2675502964 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd2b80733b0 msgr2=0x7fd2a4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:28.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.695+0000 7fd2b7fff640 1 --2- 172.21.15.98:0/2675502964 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd2b80733b0 0x7fd2a4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:28.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.695+0000 7fd2b7fff640 1 -- 172.21.15.98:0/2675502964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2a4004810 msgr2=0x7fd2a4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:28.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.695+0000 7fd2b7fff640 1 --2- 172.21.15.98:0/2675502964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2a4004810 0x7fd2a4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:28.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.695+0000 7fd2b7fff640 1 -- 172.21.15.98:0/2675502964 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a4003530 con 0x7fd2a40039f0 2024-09-19T08:38:28.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.695+0000 7fd2b77fe640 1 --2- 172.21.15.98:0/2675502964 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2a4004810 0x7fd2a4004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:28.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.699+0000 7fd2b7fff640 1 --2- 172.21.15.98:0/2675502964 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2a40039f0 0x7fd2a4003e10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8001e70 tx=0x7fd2a801c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414084c560987979 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:28.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.699+0000 7fd2b6ffd640 1 -- 172.21.15.98:0/2675502964 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a80af070 con 0x7fd2a40039f0 2024-09-19T08:38:28.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.699+0000 7fd2b6ffd640 1 -- 172.21.15.98:0/2675502964 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2a8003b80 con 0x7fd2a40039f0 2024-09-19T08:38:28.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.699+0000 7fd2b6ffd640 1 -- 172.21.15.98:0/2675502964 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a8001a90 con 0x7fd2a40039f0 2024-09-19T08:38:28.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.699+0000 7fd2be822640 1 -- 172.21.15.98:0/2675502964 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2a40039f0 msgr2=0x7fd2a4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:28.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.699+0000 7fd2be822640 1 --2- 172.21.15.98:0/2675502964 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2a40039f0 0x7fd2a4003e10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8001e70 tx=0x7fd2a801c780 comp rx=0 tx=0).stop 2024-09-19T08:38:28.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.699+0000 7fd2bcdbf640 1 -- 172.21.15.98:0/2675502964 reap_dead start 2024-09-19T08:38:28.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.699+0000 7fd2be822640 1 -- 172.21.15.98:0/2675502964 shutdown_connections 2024-09-19T08:38:28.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.699+0000 7fd2be822640 1 -- 172.21.15.98:0/2675502964 >> 172.21.15.98:0/2675502964 conn(0x7fd2b80b2010 msgr2=0x7fd2b80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:28.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.699+0000 7fd2be822640 1 -- 172.21.15.98:0/2675502964 shutdown_connections 2024-09-19T08:38:28.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.699+0000 7fd2be822640 1 -- 172.21.15.98:0/2675502964 wait complete. 2024-09-19T08:38:28.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.699+0000 7fd2be822640 1 Processor -- start 2024-09-19T08:38:28.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.699+0000 7fd2be822640 1 -- start start 2024-09-19T08:38:28.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.699+0000 7fd2be822640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2a40039f0 0x7fd2b8150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:28.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.699+0000 7fd2be822640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2b8160350 0x7fd2b8162400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:28.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.699+0000 7fd2be822640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd2b8150fb0 0x7fd2b8154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:28.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.699+0000 7fd2be822640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b8167c40 con 0x7fd2a40039f0 2024-09-19T08:38:28.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.699+0000 7fd2be822640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b815cb40 con 0x7fd2b8160350 2024-09-19T08:38:28.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.699+0000 7fd2be822640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd2b8167dc0 con 0x7fd2b8150fb0 2024-09-19T08:38:28.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.699+0000 7fd2b7fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2a40039f0 0x7fd2b8150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:28.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.699+0000 7fd2b7fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2a40039f0 0x7fd2b8150970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37724/0 (socket says 172.21.15.98:37724) 2024-09-19T08:38:28.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.699+0000 7fd2b7fff640 1 -- 172.21.15.98:0/3989485885 learned_addr learned my addr 172.21.15.98:0/3989485885 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:28.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.699+0000 7fd2b7fff640 1 -- 172.21.15.98:0/3989485885 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd2b8150fb0 msgr2=0x7fd2b8154ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:28.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.699+0000 7fd2b7fff640 1 --2- 172.21.15.98:0/3989485885 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd2b8150fb0 0x7fd2b8154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:28.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.699+0000 7fd2b7fff640 1 -- 172.21.15.98:0/3989485885 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2b8160350 msgr2=0x7fd2b8162400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:28.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.699+0000 7fd2b7fff640 1 --2- 172.21.15.98:0/3989485885 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2b8160350 0x7fd2b8162400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:28.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.699+0000 7fd2b7fff640 1 -- 172.21.15.98:0/3989485885 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2b8160e90 con 0x7fd2a40039f0 2024-09-19T08:38:28.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.699+0000 7fd2b7fff640 1 --2- 172.21.15.98:0/3989485885 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2a40039f0 0x7fd2b8150970 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd2a801c5b0 tx=0x7fd2a4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:28.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.699+0000 7fd2b4ff9640 1 -- 172.21.15.98:0/3989485885 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a80ba020 con 0x7fd2a40039f0 2024-09-19T08:38:28.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.699+0000 7fd2b4ff9640 1 -- 172.21.15.98:0/3989485885 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2a80b0040 con 0x7fd2a40039f0 2024-09-19T08:38:28.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.699+0000 7fd2b4ff9640 1 -- 172.21.15.98:0/3989485885 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a80af070 con 0x7fd2a40039f0 2024-09-19T08:38:28.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.699+0000 7fd2be822640 1 -- 172.21.15.98:0/3989485885 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2b81608d0 con 0x7fd2a40039f0 2024-09-19T08:38:28.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.699+0000 7fd2be822640 1 -- 172.21.15.98:0/3989485885 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2b815df00 con 0x7fd2a40039f0 2024-09-19T08:38:28.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.703+0000 7fd2be822640 1 -- 172.21.15.98:0/3989485885 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2b80cf3c0 con 0x7fd2a40039f0 2024-09-19T08:38:28.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.703+0000 7fd2b4ff9640 1 -- 172.21.15.98:0/3989485885 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd2a80bd020 con 0x7fd2a40039f0 2024-09-19T08:38:28.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.703+0000 7fd2b4ff9640 1 --2- 172.21.15.98:0/3989485885 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd2980678f0 0x7fd298069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:28.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.703+0000 7fd2b77fe640 1 --2- 172.21.15.98:0/3989485885 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd2980678f0 0x7fd298069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:28.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.703+0000 7fd2b4ff9640 1 -- 172.21.15.98:0/3989485885 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd2a811ff20 con 0x7fd2a40039f0 2024-09-19T08:38:28.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.703+0000 7fd2b77fe640 1 --2- 172.21.15.98:0/3989485885 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd2980678f0 0x7fd298069db0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd2a00017f0 tx=0x7fd2a0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:28.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.707+0000 7fd2b4ff9640 1 -- 172.21.15.98:0/3989485885 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd2a80c4030 con 0x7fd2a40039f0 2024-09-19T08:38:28.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.855+0000 7fd2be822640 1 -- 172.21.15.98:0/3989485885 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd2b81596e0 con 0x7fd2a40039f0 2024-09-19T08:38:28.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.859+0000 7fd2b4ff9640 1 -- 172.21.15.98:0/3989485885 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16665 (secure 0 0 0) 0x7fd2a8016490 con 0x7fd2a40039f0 2024-09-19T08:38:28.859 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:38:28.860 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":21,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:38:20.362527+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:38:16.588688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T08:38:17.700392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:38:28.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.859+0000 7fd2be822640 1 -- 172.21.15.98:0/3989485885 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd2980678f0 msgr2=0x7fd298069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:28.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.859+0000 7fd2be822640 1 --2- 172.21.15.98:0/3989485885 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd2980678f0 0x7fd298069db0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd2a00017f0 tx=0x7fd2a0009040 comp rx=0 tx=0).stop 2024-09-19T08:38:28.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.859+0000 7fd2be822640 1 -- 172.21.15.98:0/3989485885 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2a40039f0 msgr2=0x7fd2b8150970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:28.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.859+0000 7fd2be822640 1 --2- 172.21.15.98:0/3989485885 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2a40039f0 0x7fd2b8150970 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd2a801c5b0 tx=0x7fd2a4005840 comp rx=0 tx=0).stop 2024-09-19T08:38:28.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.863+0000 7fd2be822640 1 -- 172.21.15.98:0/3989485885 shutdown_connections 2024-09-19T08:38:28.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.863+0000 7fd2be822640 1 --2- 172.21.15.98:0/3989485885 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd2980678f0 0x7fd298069db0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:28.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.863+0000 7fd2be822640 1 --2- 172.21.15.98:0/3989485885 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd2b8150fb0 0x7fd2b8154ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:28.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.863+0000 7fd2be822640 1 --2- 172.21.15.98:0/3989485885 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd2b8160350 0x7fd2b8162400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:28.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.863+0000 7fd2be822640 1 --2- 172.21.15.98:0/3989485885 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2a40039f0 0x7fd2b8150970 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:28.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.863+0000 7fd2be822640 1 -- 172.21.15.98:0/3989485885 >> 172.21.15.98:0/3989485885 conn(0x7fd2b80b2010 msgr2=0x7fd2b8159ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:28.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.863+0000 7fd2be822640 1 -- 172.21.15.98:0/3989485885 shutdown_connections 2024-09-19T08:38:28.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.863+0000 7fd2be822640 1 -- 172.21.15.98:0/3989485885 wait complete. 2024-09-19T08:38:28.877 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-19T08:38:28.878 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:38:28.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.983+0000 7fe1cfabf640 1 Processor -- start 2024-09-19T08:38:28.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.983+0000 7fe1cfabf640 1 -- start start 2024-09-19T08:38:28.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.983+0000 7fe1cfabf640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe1c806c120 0x7fe1c80cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:28.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.983+0000 7fe1cfabf640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1c80ccc60 con 0x7fe1c8158fd0 2024-09-19T08:38:28.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.983+0000 7fe1cfabf640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1c80ccf40 con 0x7fe1c806bc90 2024-09-19T08:38:28.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.983+0000 7fe1cfabf640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe1c81593b0 con 0x7fe1c806c120 2024-09-19T08:38:28.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.983+0000 7fe1cd85b640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fe1c8158fd0 0x7fe1c80b3e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:60524/0 (socket says 172.21.15.98:60524) 2024-09-19T08:38:28.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.983+0000 7fe1cd85b640 1 -- 172.21.15.98:0/3780597260 learned_addr learned my addr 172.21.15.98:0/3780597260 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:28.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.983+0000 7fe1cc859640 1 -- 172.21.15.98:0/3780597260 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2461627475 0 0) 0x7fe1c80ccc60 con 0x7fe1c8158fd0 2024-09-19T08:38:28.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.983+0000 7fe1ce05c640 1 --2- 172.21.15.98:0/3780597260 >> v2:172.21.15.139:3300/0 conn(0x7fe1c806c120 0x7fe1c80cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:28.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.983+0000 7fe1cc859640 1 -- 172.21.15.98:0/3780597260 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1b4003530 con 0x7fe1c8158fd0 2024-09-19T08:38:28.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.983+0000 7fe1cc859640 1 -- 172.21.15.98:0/3780597260 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3208950014 0 0) 0x7fe1b4003530 con 0x7fe1c8158fd0 2024-09-19T08:38:28.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.983+0000 7fe1cc859640 1 -- 172.21.15.98:0/3780597260 >> v2:172.21.15.139:3300/0 conn(0x7fe1c806c120 msgr2=0x7fe1c80cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:28.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.983+0000 7fe1cc859640 1 --2- 172.21.15.98:0/3780597260 >> v2:172.21.15.139:3300/0 conn(0x7fe1c806c120 0x7fe1c80cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:28.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.983+0000 7fe1cc859640 1 -- 172.21.15.98:0/3780597260 >> v1:172.21.15.139:6789/0 conn(0x7fe1c806bc90 legacy=0x7fe1c80ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:28.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.983+0000 7fe1cc859640 1 -- 172.21.15.98:0/3780597260 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c80cd220 con 0x7fe1c8158fd0 2024-09-19T08:38:28.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.983+0000 7fe1cc859640 1 -- 172.21.15.98:0/3780597260 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fe1c4001680 con 0x7fe1c8158fd0 2024-09-19T08:38:28.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.983+0000 7fe1ce05c640 1 --2- 172.21.15.98:0/3780597260 >> v2:172.21.15.139:3300/0 conn(0x7fe1c806c120 0x7fe1c80cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:38:28.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.987+0000 7fe1cc859640 1 -- 172.21.15.98:0/3780597260 >> v1:172.21.15.98:6789/0 conn(0x7fe1c8158fd0 legacy=0x7fe1c80b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:28.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.987+0000 7fe1cc859640 1 --2- 172.21.15.98:0/3780597260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1b40039c0 0x7fe1b4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:28.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.987+0000 7fe1cc859640 1 --2- 172.21.15.98:0/3780597260 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1b4004810 0x7fe1b4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:28.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.987+0000 7fe1cc859640 1 --2- 172.21.15.98:0/3780597260 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1b40052f0 0x7fe1b400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:28.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.987+0000 7fe1cc859640 1 -- 172.21.15.98:0/3780597260 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c81593b0 con 0x7fe1b40039c0 2024-09-19T08:38:28.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.987+0000 7fe1cc859640 1 -- 172.21.15.98:0/3780597260 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b400c1c0 con 0x7fe1b40052f0 2024-09-19T08:38:28.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.987+0000 7fe1cc859640 1 -- 172.21.15.98:0/3780597260 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe1b400e5d0 con 0x7fe1b4004810 2024-09-19T08:38:28.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.987+0000 7fe1cd85b640 1 --2- 172.21.15.98:0/3780597260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1b40039c0 0x7fe1b4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:28.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.987+0000 7fe1ce05c640 1 --2- 172.21.15.98:0/3780597260 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1b40052f0 0x7fe1b400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:28.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.987+0000 7fe1cd85b640 1 -- 172.21.15.98:0/3780597260 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1b4004810 msgr2=0x7fe1b4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:28.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.987+0000 7fe1cd85b640 1 --2- 172.21.15.98:0/3780597260 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1b4004810 0x7fe1b4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:28.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.987+0000 7fe1cd85b640 1 -- 172.21.15.98:0/3780597260 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1b40052f0 msgr2=0x7fe1b400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:28.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.987+0000 7fe1cd85b640 1 --2- 172.21.15.98:0/3780597260 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1b40052f0 0x7fe1b400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:28.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.987+0000 7fe1ce05c640 1 -- 172.21.15.98:0/3780597260 reap_dead start 2024-09-19T08:38:28.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.987+0000 7fe1cd85b640 1 -- 172.21.15.98:0/3780597260 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1b4003530 con 0x7fe1b40039c0 2024-09-19T08:38:28.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.987+0000 7fe1cd85b640 1 --2- 172.21.15.98:0/3780597260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1b40039c0 0x7fe1b4003de0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe1c400c210 tx=0x7fe1c401cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626b7796f1444233 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:28.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.987+0000 7fe1cc859640 1 -- 172.21.15.98:0/3780597260 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c4007d20 con 0x7fe1b40039c0 2024-09-19T08:38:28.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.987+0000 7fe1cc859640 1 -- 172.21.15.98:0/3780597260 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1c40a9070 con 0x7fe1b40039c0 2024-09-19T08:38:28.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.987+0000 7fe1cc859640 1 -- 172.21.15.98:0/3780597260 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c40a5db0 con 0x7fe1b40039c0 2024-09-19T08:38:28.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.987+0000 7fe1cfabf640 1 -- 172.21.15.98:0/3780597260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1b40039c0 msgr2=0x7fe1b4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:28.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.987+0000 7fe1cfabf640 1 --2- 172.21.15.98:0/3780597260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1b40039c0 0x7fe1b4003de0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe1c400c210 tx=0x7fe1c401cc30 comp rx=0 tx=0).stop 2024-09-19T08:38:28.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.987+0000 7fe1cfabf640 1 -- 172.21.15.98:0/3780597260 shutdown_connections 2024-09-19T08:38:28.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.987+0000 7fe1cfabf640 1 --2- 172.21.15.98:0/3780597260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1b40039c0 0x7fe1b4003de0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:28.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.987+0000 7fe1cfabf640 1 -- 172.21.15.98:0/3780597260 >> 172.21.15.98:0/3780597260 conn(0x7fe1c80b2010 msgr2=0x7fe1c80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:28.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.987+0000 7fe1cfabf640 1 -- 172.21.15.98:0/3780597260 shutdown_connections 2024-09-19T08:38:28.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.987+0000 7fe1cfabf640 1 -- 172.21.15.98:0/3780597260 wait complete. 2024-09-19T08:38:28.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.991+0000 7fe1cfabf640 1 Processor -- start 2024-09-19T08:38:28.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.991+0000 7fe1cfabf640 1 -- start start 2024-09-19T08:38:28.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.991+0000 7fe1cfabf640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1b40039c0 0x7fe1c81f3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:28.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.991+0000 7fe1cfabf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1c81550b0 0x7fe1c8156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:28.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.991+0000 7fe1cfabf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1c8156550 0x7fe1c81f2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:28.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.991+0000 7fe1cfabf640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c815c920 con 0x7fe1c81550b0 2024-09-19T08:38:28.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.991+0000 7fe1cfabf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c815c7a0 con 0x7fe1c8156550 2024-09-19T08:38:28.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.991+0000 7fe1cfabf640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe1c815efc0 con 0x7fe1b40039c0 2024-09-19T08:38:28.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.991+0000 7fe1cd05a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1c81550b0 0x7fe1c8156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:28.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.991+0000 7fe1cd05a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1c81550b0 0x7fe1c8156010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37738/0 (socket says 172.21.15.98:37738) 2024-09-19T08:38:28.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.991+0000 7fe1cd05a640 1 -- 172.21.15.98:0/998888794 learned_addr learned my addr 172.21.15.98:0/998888794 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:28.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.991+0000 7fe1ce05c640 1 --2- 172.21.15.98:0/998888794 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1c8156550 0x7fe1c81f2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:28.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.991+0000 7fe1cd85b640 1 --2- 172.21.15.98:0/998888794 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1b40039c0 0x7fe1c81f3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:28.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.991+0000 7fe1cd05a640 1 -- 172.21.15.98:0/998888794 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1b40039c0 msgr2=0x7fe1c81f3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:28.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.991+0000 7fe1cd05a640 1 --2- 172.21.15.98:0/998888794 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1b40039c0 0x7fe1c81f3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:28.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.991+0000 7fe1cd05a640 1 -- 172.21.15.98:0/998888794 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1c8156550 msgr2=0x7fe1c81f2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:28.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.991+0000 7fe1cd05a640 1 --2- 172.21.15.98:0/998888794 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1c8156550 0x7fe1c81f2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:28.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.991+0000 7fe1cd05a640 1 -- 172.21.15.98:0/998888794 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c81f3bb0 con 0x7fe1c81550b0 2024-09-19T08:38:28.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.991+0000 7fe1cd85b640 1 --2- 172.21.15.98:0/998888794 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1b40039c0 0x7fe1c81f3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:28.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.991+0000 7fe1cd05a640 1 --2- 172.21.15.98:0/998888794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1c81550b0 0x7fe1c8156010 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe1b800c500 tx=0x7fe1b800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:28.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.991+0000 7fe1be7fc640 1 -- 172.21.15.98:0/998888794 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b80052d0 con 0x7fe1c81550b0 2024-09-19T08:38:28.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.991+0000 7fe1cfabf640 1 -- 172.21.15.98:0/998888794 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1c80727d0 con 0x7fe1c81550b0 2024-09-19T08:38:28.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.991+0000 7fe1cfabf640 1 -- 172.21.15.98:0/998888794 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1c8155530 con 0x7fe1c81550b0 2024-09-19T08:38:28.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.991+0000 7fe1be7fc640 1 -- 172.21.15.98:0/998888794 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1b8005470 con 0x7fe1c81550b0 2024-09-19T08:38:28.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.991+0000 7fe1be7fc640 1 -- 172.21.15.98:0/998888794 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8012830 con 0x7fe1c81550b0 2024-09-19T08:38:28.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.991+0000 7fe1cfabf640 1 -- 172.21.15.98:0/998888794 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1c8156930 con 0x7fe1c81550b0 2024-09-19T08:38:28.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.991+0000 7fe1be7fc640 1 -- 172.21.15.98:0/998888794 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe1b80129d0 con 0x7fe1c81550b0 2024-09-19T08:38:28.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.995+0000 7fe1be7fc640 1 --2- 172.21.15.98:0/998888794 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe1ac0678f0 0x7fe1ac069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:28.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.995+0000 7fe1be7fc640 1 -- 172.21.15.98:0/998888794 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe1b8085910 con 0x7fe1c81550b0 2024-09-19T08:38:28.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.995+0000 7fe1cd85b640 1 --2- 172.21.15.98:0/998888794 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe1ac0678f0 0x7fe1ac069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:28.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.995+0000 7fe1cd85b640 1 --2- 172.21.15.98:0/998888794 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe1ac0678f0 0x7fe1ac069db0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe1c40ab000 tx=0x7fe1c40a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:28.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:28.999+0000 7fe1be7fc640 1 -- 172.21.15.98:0/998888794 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe1b8056460 con 0x7fe1c81550b0 2024-09-19T08:38:29.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.147+0000 7fe1cfabf640 1 -- 172.21.15.98:0/998888794 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe1c815fab0 con 0x7fe1c81550b0 2024-09-19T08:38:29.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.151+0000 7fe1be7fc640 1 -- 172.21.15.98:0/998888794 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16665 (secure 0 0 0) 0x7fe1b8055d60 con 0x7fe1c81550b0 2024-09-19T08:38:29.153 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:38:29.154 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":21,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:38:20.362527+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:38:16.588688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T08:38:17.700392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:38:29.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.155+0000 7fe1cfabf640 1 -- 172.21.15.98:0/998888794 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe1ac0678f0 msgr2=0x7fe1ac069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.155+0000 7fe1cfabf640 1 --2- 172.21.15.98:0/998888794 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe1ac0678f0 0x7fe1ac069db0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe1c40ab000 tx=0x7fe1c40a7070 comp rx=0 tx=0).stop 2024-09-19T08:38:29.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.155+0000 7fe1cfabf640 1 -- 172.21.15.98:0/998888794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1c81550b0 msgr2=0x7fe1c8156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.155+0000 7fe1cfabf640 1 --2- 172.21.15.98:0/998888794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1c81550b0 0x7fe1c8156010 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe1b800c500 tx=0x7fe1b800c9c0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.155+0000 7fe1cfabf640 1 -- 172.21.15.98:0/998888794 shutdown_connections 2024-09-19T08:38:29.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.155+0000 7fe1cfabf640 1 --2- 172.21.15.98:0/998888794 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe1ac0678f0 0x7fe1ac069db0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.155+0000 7fe1cfabf640 1 --2- 172.21.15.98:0/998888794 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1c8156550 0x7fe1c81f2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.155+0000 7fe1cfabf640 1 --2- 172.21.15.98:0/998888794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1c81550b0 0x7fe1c8156010 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.155+0000 7fe1cfabf640 1 --2- 172.21.15.98:0/998888794 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1b40039c0 0x7fe1c81f3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.155+0000 7fe1cfabf640 1 -- 172.21.15.98:0/998888794 >> 172.21.15.98:0/998888794 conn(0x7fe1c80b2010 msgr2=0x7fe1c815cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:29.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.155+0000 7fe1cfabf640 1 -- 172.21.15.98:0/998888794 shutdown_connections 2024-09-19T08:38:29.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.155+0000 7fe1cfabf640 1 -- 172.21.15.98:0/998888794 wait complete. 2024-09-19T08:38:29.171 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-19T08:38:29.171 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-19T08:38:29.172 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-19T08:38:29.172 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-19T08:38:29.172 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-19T08:38:29.173 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-19T08:38:29.173 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-19T08:38:29.173 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-19T08:38:29.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.347+0000 7f8149aaf640 1 Processor -- start 2024-09-19T08:38:29.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.347+0000 7f8149aaf640 1 -- start start 2024-09-19T08:38:29.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.347+0000 7f8149aaf640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f814415ec00 0x7f8144159d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.347+0000 7f8149aaf640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f81440ce4a0 0x7f814415a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.347+0000 7f8149aaf640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81440b4800 con 0x7f814415e110 2024-09-19T08:38:29.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.347+0000 7f8149aaf640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f8144073e10 con 0x7f81440ce4a0 2024-09-19T08:38:29.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.347+0000 7f8149aaf640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f81440b4ae0 con 0x7f814415ec00 2024-09-19T08:38:29.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.347+0000 7f8143fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f81440ce4a0 0x7f814415a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.347+0000 7f8143fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f81440ce4a0 0x7f814415a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37748/0 (socket says 172.21.15.98:37748) 2024-09-19T08:38:29.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.347+0000 7f8143fff640 1 -- 172.21.15.98:0/3777253927 learned_addr learned my addr 172.21.15.98:0/3777253927 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:29.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.347+0000 7f8142ffd640 1 --2- 172.21.15.98:0/3777253927 >> v2:172.21.15.139:3300/0 conn(0x7f814415ec00 0x7f8144159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.347+0000 7f8143fff640 1 -- 172.21.15.98:0/3777253927 >> v2:172.21.15.139:3300/0 conn(0x7f814415ec00 msgr2=0x7f8144159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:29.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.347+0000 7f8143fff640 1 --2- 172.21.15.98:0/3777253927 >> v2:172.21.15.139:3300/0 conn(0x7f814415ec00 0x7f8144159d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.347+0000 7f8143fff640 1 -- 172.21.15.98:0/3777253927 >> v1:172.21.15.139:6789/0 conn(0x7f814415e110 legacy=0x7f814415e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.347+0000 7f8143fff640 1 -- 172.21.15.98:0/3777253927 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81440b4dc0 con 0x7f81440ce4a0 2024-09-19T08:38:29.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f4adbfff640 1 Processor -- start 2024-09-19T08:38:29.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f4adbfff640 1 -- start start 2024-09-19T08:38:29.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f633ffff640 1 Processor -- start 2024-09-19T08:38:29.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f633ffff640 1 -- start start 2024-09-19T08:38:29.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f5e22e46640 1 Processor -- start 2024-09-19T08:38:29.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f4adbfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4adc06c3a0 0x7f4adc0af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f4adbfff640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4adc0ce6b0 con 0x7f4adc144b10 2024-09-19T08:38:29.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f4adbfff640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4adc0ce990 con 0x7f4adc0cdff0 2024-09-19T08:38:29.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f4adbfff640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f4adc0ce460 con 0x7f4adc06c3a0 2024-09-19T08:38:29.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f633ffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f634015ecd0 0x7f63400b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f633ffff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63400cf210 0x7f63400cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f633ffff640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f634015f1b0 con 0x7f634015b2e0 2024-09-19T08:38:29.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f633ffff640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f634006bfe0 con 0x7f634015ecd0 2024-09-19T08:38:29.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f633ffff640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f63400cfba0 con 0x7f63400cf210 2024-09-19T08:38:29.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f4adb7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4adc06c3a0 0x7f4adc0af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f4adb7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4adc06c3a0 0x7f4adc0af0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:49816/0 (socket says 172.21.15.98:49816) 2024-09-19T08:38:29.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f4adb7fe640 1 -- 172.21.15.98:0/2569799535 learned_addr learned my addr 172.21.15.98:0/2569799535 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:29.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f633e7fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f634015ecd0 0x7f63400b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f633e7fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f634015ecd0 0x7f63400b3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:49828/0 (socket says 172.21.15.98:49828) 2024-09-19T08:38:29.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f633e7fc640 1 -- 172.21.15.98:0/139860119 learned_addr learned my addr 172.21.15.98:0/139860119 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:29.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f5e22e46640 1 -- start start 2024-09-19T08:38:29.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f5e22e46640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5e1c15f910 0x7f5e1c0d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f5e22e46640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5e1c0b70e0 0x7f5e1c0d2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f5e22e46640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5e1c0b7650 0x7f5e1c0d2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f5e22e46640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f5e1c0b7ba0 con 0x7f5e1c0b7650 2024-09-19T08:38:29.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f5e22e46640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f5e1c0d2ea0 con 0x7f5e1c0b70e0 2024-09-19T08:38:29.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f5e22e46640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5e1c0b7df0 con 0x7f5e1c15f910 2024-09-19T08:38:29.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f0eddbf9640 1 Processor -- start 2024-09-19T08:38:29.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f633f7fe640 1 --2- 172.21.15.98:0/139860119 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63400cf210 0x7f63400cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f5e1bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5e1c0b70e0 0x7f5e1c0d2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f633e7fc640 1 -- 172.21.15.98:0/139860119 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63400cf210 msgr2=0x7f63400cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:29.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f633e7fc640 1 --2- 172.21.15.98:0/139860119 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63400cf210 0x7f63400cf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f633e7fc640 1 -- 172.21.15.98:0/139860119 >> v1:172.21.15.98:6789/0 conn(0x7f634015b2e0 legacy=0x7f6340071e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f633e7fc640 1 -- 172.21.15.98:0/139860119 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634015f490 con 0x7f634015ecd0 2024-09-19T08:38:29.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f5e1bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5e1c0b70e0 0x7f5e1c0d2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:49850/0 (socket says 172.21.15.98:49850) 2024-09-19T08:38:29.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f5e1bfff640 1 -- 172.21.15.98:0/2293926374 learned_addr learned my addr 172.21.15.98:0/2293926374 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:29.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f8143fff640 1 --2- 172.21.15.98:0/3777253927 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81440ce4a0 0x7f814415a2d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f813c009e50 tx=0x7f813c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4720efba6a34d69 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f4cff917640 1 Processor -- start 2024-09-19T08:38:29.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f4cff917640 1 -- start start 2024-09-19T08:38:29.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f4cff917640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4cf00deb70 0x7f4cf0006d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f4cff917640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4cf000a7d0 0x7f4cf0006180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f4cff917640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4cf0050210 con 0x7f4cf000a380 2024-09-19T08:38:29.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f4cff917640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f4cf001c530 con 0x7f4cf000a7d0 2024-09-19T08:38:29.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f4cff917640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f4cf00504f0 con 0x7f4cf00deb70 2024-09-19T08:38:29.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f4cfdeb4640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4cf000a7d0 0x7f4cf0006180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f4cfceb2640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f4cf000a380 0x7f4cf0005a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:44942/0 (socket says 172.21.15.98:44942) 2024-09-19T08:38:29.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f4cfceb2640 1 -- 172.21.15.98:0/2253282807 learned_addr learned my addr 172.21.15.98:0/2253282807 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:29.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f4cfd6b3640 1 --2- 172.21.15.98:0/2253282807 >> v2:172.21.15.139:3300/0 conn(0x7f4cf00deb70 0x7f4cf0006d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f4cfdeb4640 1 -- 172.21.15.98:0/2253282807 >> v2:172.21.15.139:3300/0 conn(0x7f4cf00deb70 msgr2=0x7f4cf0006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:29.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f4cfdeb4640 1 --2- 172.21.15.98:0/2253282807 >> v2:172.21.15.139:3300/0 conn(0x7f4cf00deb70 0x7f4cf0006d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f4cfdeb4640 1 -- 172.21.15.98:0/2253282807 >> v1:172.21.15.139:6789/0 conn(0x7f4cf000a380 legacy=0x7f4cf0005a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f4cfdeb4640 1 -- 172.21.15.98:0/2253282807 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf00507d0 con 0x7f4cf000a7d0 2024-09-19T08:38:29.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f56861fe640 1 Processor -- start 2024-09-19T08:38:29.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f81427fc640 1 -- 172.21.15.98:0/3777253927 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813c0a58b0 con 0x7f81440ce4a0 2024-09-19T08:38:29.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f81427fc640 1 -- 172.21.15.98:0/3777253927 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f813c0a5a50 con 0x7f81440ce4a0 2024-09-19T08:38:29.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f81427fc640 1 -- 172.21.15.98:0/3777253927 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813c0a5e20 con 0x7f81440ce4a0 2024-09-19T08:38:29.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f56861fe640 1 -- start start 2024-09-19T08:38:29.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f56861fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f56800cd290 0x7f568016d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f5e213e3640 1 --2- 172.21.15.98:0/2293926374 >> v2:172.21.15.98:3300/0 conn(0x7f5e1c0b7650 0x7f5e1c0d2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f0eddbf9640 1 -- start start 2024-09-19T08:38:29.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f0eddbf9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0ed0054f50 0x7f0ed0054300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f0eddbf9640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0ed00e4f00 0x7f0ed00e18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f0eddbf9640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ed0054990 con 0x7f0ed0055330 2024-09-19T08:38:29.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f0eddbf9640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f0ed00e1e20 con 0x7f0ed00e4f00 2024-09-19T08:38:29.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f0eddbf9640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f0ed0038240 con 0x7f0ed0054f50 2024-09-19T08:38:29.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f4ad9ffb640 1 -- 172.21.15.98:0/2569799535 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2952997314 0 0) 0x7f4adc0ce990 con 0x7f4adc0cdff0 2024-09-19T08:38:29.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f4ad9ffb640 1 -- 172.21.15.98:0/2569799535 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ac4003530 con 0x7f4adc0cdff0 2024-09-19T08:38:29.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f4adb7fe640 1 -- 172.21.15.98:0/2569799535 >> v1:172.21.15.139:6789/0 conn(0x7f4adc0cdff0 legacy=0x7f4adc06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f4adb7fe640 1 -- 172.21.15.98:0/2569799535 >> v1:172.21.15.98:6789/0 conn(0x7f4adc144b10 legacy=0x7f4adc162750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f4adb7fe640 1 -- 172.21.15.98:0/2569799535 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4adc0cec70 con 0x7f4adc06c3a0 2024-09-19T08:38:29.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f5e20be2640 1 --2- 172.21.15.98:0/2293926374 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5e1c15f910 0x7f5e1c0d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f56861fe640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5680070d40 con 0x7f5680162700 2024-09-19T08:38:29.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f5e1bfff640 1 -- 172.21.15.98:0/2293926374 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5e1c15f910 msgr2=0x7f5e1c0d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:29.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f5e1bfff640 1 --2- 172.21.15.98:0/2293926374 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5e1c15f910 0x7f5e1c0d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f5e1bfff640 1 -- 172.21.15.98:0/2293926374 >> v2:172.21.15.98:3300/0 conn(0x7f5e1c0b7650 msgr2=0x7f5e1c0d2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:29.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f5e1bfff640 1 --2- 172.21.15.98:0/2293926374 >> v2:172.21.15.98:3300/0 conn(0x7f5e1c0b7650 0x7f5e1c0d2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f5e1bfff640 1 -- 172.21.15.98:0/2293926374 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e1c0d2ba0 con 0x7f5e1c0b70e0 2024-09-19T08:38:29.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f5e20be2640 1 --2- 172.21.15.98:0/2293926374 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5e1c15f910 0x7f5e1c0d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:29.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f56861fe640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56800b3cf0 con 0x7f5680073740 2024-09-19T08:38:29.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f56861fe640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f56800ce5b0 con 0x7f56800cd290 2024-09-19T08:38:29.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f0ed7fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0ed00e4f00 0x7f0ed00e18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f56849fb640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f5680073740 0x7f56800ccb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:44944/0 (socket says 172.21.15.98:44944) 2024-09-19T08:38:29.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.351+0000 7f56849fb640 1 -- 172.21.15.98:0/2983669385 learned_addr learned my addr 172.21.15.98:0/2983669385 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:29.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f8149aaf640 1 -- 172.21.15.98:0/3777253927 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81440ce4a0 msgr2=0x7f814415a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f8149aaf640 1 --2- 172.21.15.98:0/3777253927 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81440ce4a0 0x7f814415a2d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f813c009e50 tx=0x7f813c01c9f0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f56859fd640 1 --2- 172.21.15.98:0/2983669385 >> v2:172.21.15.98:3300/0 conn(0x7f56800cd290 0x7f568016d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f8149aaf640 1 -- 172.21.15.98:0/3777253927 shutdown_connections 2024-09-19T08:38:29.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f8149aaf640 1 --2- 172.21.15.98:0/3777253927 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81440ce4a0 0x7f814415a2d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f8149aaf640 1 --2- 172.21.15.98:0/3777253927 >> v2:172.21.15.139:3300/0 conn(0x7f814415ec00 0x7f8144159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f8149aaf640 1 -- 172.21.15.98:0/3777253927 >> 172.21.15.98:0/3777253927 conn(0x7f814406cda0 msgr2=0x7f814406d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:29.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f0ed7fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0ed00e4f00 0x7f0ed00e18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37790/0 (socket says 172.21.15.98:37790) 2024-09-19T08:38:29.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f8149aaf640 1 -- 172.21.15.98:0/3777253927 shutdown_connections 2024-09-19T08:38:29.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f0ed77fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0ed0054f50 0x7f0ed0054300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f0ed7fff640 1 -- 172.21.15.98:0/1029394861 learned_addr learned my addr 172.21.15.98:0/1029394861 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:29.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f5677fff640 1 -- 172.21.15.98:0/2983669385 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1236383200 0 0) 0x7f56800b3cf0 con 0x7f5680073740 2024-09-19T08:38:29.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f5677fff640 1 -- 172.21.15.98:0/2983669385 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f566c003530 con 0x7f5680073740 2024-09-19T08:38:29.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f5677fff640 1 -- 172.21.15.98:0/2983669385 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 556424388 0 0) 0x7f5670001120 con 0x7f5680162700 2024-09-19T08:38:29.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f5677fff640 1 -- 172.21.15.98:0/2983669385 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56800b3cf0 con 0x7f5680162700 2024-09-19T08:38:29.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f56859fd640 1 -- 172.21.15.98:0/2983669385 >> v1:172.21.15.139:6789/0 conn(0x7f5680073740 legacy=0x7f56800ccb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f56859fd640 1 -- 172.21.15.98:0/2983669385 >> v1:172.21.15.98:6789/0 conn(0x7f5680162700 legacy=0x7f5680166ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f56859fd640 1 -- 172.21.15.98:0/2983669385 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56800b3fd0 con 0x7f56800cd290 2024-09-19T08:38:29.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f0ed7fff640 1 -- 172.21.15.98:0/1029394861 >> v2:172.21.15.139:3300/0 conn(0x7f0ed0054f50 msgr2=0x7f0ed0054300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:29.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f0ed7fff640 1 --2- 172.21.15.98:0/1029394861 >> v2:172.21.15.139:3300/0 conn(0x7f0ed0054f50 0x7f0ed0054300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f0ed7fff640 1 -- 172.21.15.98:0/1029394861 >> v1:172.21.15.139:6789/0 conn(0x7f0ed0055330 legacy=0x7f0ed0051380 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:38:29.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f0ed7fff640 1 -- 172.21.15.98:0/1029394861 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed0038520 con 0x7f0ed00e4f00 2024-09-19T08:38:29.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f4cfdeb4640 1 --2- 172.21.15.98:0/2253282807 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4cf000a7d0 0x7f4cf0006180 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4cf802dd60 tx=0x7f4cf809b030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56d261d1c0e01c5c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f8149aaf640 1 -- 172.21.15.98:0/3777253927 wait complete. 2024-09-19T08:38:29.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f4ceffff640 1 -- 172.21.15.98:0/2253282807 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf8124c90 con 0x7f4cf000a7d0 2024-09-19T08:38:29.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f4ceffff640 1 -- 172.21.15.98:0/2253282807 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cf812c440 con 0x7f4cf000a7d0 2024-09-19T08:38:29.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f4ceffff640 1 -- 172.21.15.98:0/2253282807 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf812c8b0 con 0x7f4cf000a7d0 2024-09-19T08:38:29.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f633e7fc640 1 --2- 172.21.15.98:0/139860119 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f634015ecd0 0x7f63400b3bb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f633400bb10 tx=0x7f633401ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=502ea5958c38b68f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f633dffb640 1 -- 172.21.15.98:0/139860119 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63340b4070 con 0x7f634015ecd0 2024-09-19T08:38:29.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f4cff917640 1 -- 172.21.15.98:0/2253282807 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4cf000a7d0 msgr2=0x7f4cf0006180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f4cff917640 1 --2- 172.21.15.98:0/2253282807 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4cf000a7d0 0x7f4cf0006180 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4cf802dd60 tx=0x7f4cf809b030 comp rx=0 tx=0).stop 2024-09-19T08:38:29.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f8149aaf640 1 Processor -- start 2024-09-19T08:38:29.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f8149aaf640 1 -- start start 2024-09-19T08:38:29.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f4cff917640 1 -- 172.21.15.98:0/2253282807 shutdown_connections 2024-09-19T08:38:29.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f4cff917640 1 --2- 172.21.15.98:0/2253282807 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4cf000a7d0 0x7f4cf0006180 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f4cff917640 1 --2- 172.21.15.98:0/2253282807 >> v2:172.21.15.139:3300/0 conn(0x7f4cf00deb70 0x7f4cf0006d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f4cff917640 1 -- 172.21.15.98:0/2253282807 >> 172.21.15.98:0/2253282807 conn(0x7f4cf00360a0 msgr2=0x7f4cf00364b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:29.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f633dffb640 1 -- 172.21.15.98:0/139860119 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63340a7b00 con 0x7f634015ecd0 2024-09-19T08:38:29.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f633dffb640 1 -- 172.21.15.98:0/139860119 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63340af440 con 0x7f634015ecd0 2024-09-19T08:38:29.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f4adb7fe640 1 --2- 172.21.15.98:0/2569799535 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4adc06c3a0 0x7f4adc0af0c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4009e80 tx=0x7f4ad401c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9442be9702d4c045 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f4cff917640 1 -- 172.21.15.98:0/2253282807 shutdown_connections 2024-09-19T08:38:29.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f5e1bfff640 1 --2- 172.21.15.98:0/2293926374 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e1c0b70e0 0x7f5e1c0d2120 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c071780 tx=0x7f5e1001c4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86dcff7e92a1ebf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f633ffff640 1 -- 172.21.15.98:0/139860119 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f634015ecd0 msgr2=0x7f63400b3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f633ffff640 1 --2- 172.21.15.98:0/139860119 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f634015ecd0 0x7f63400b3bb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f633400bb10 tx=0x7f633401ec50 comp rx=0 tx=0).stop 2024-09-19T08:38:29.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f633ffff640 1 -- 172.21.15.98:0/139860119 shutdown_connections 2024-09-19T08:38:29.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f633ffff640 1 --2- 172.21.15.98:0/139860119 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63400cf210 0x7f63400cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f633ffff640 1 --2- 172.21.15.98:0/139860119 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f634015ecd0 0x7f63400b3bb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f633ffff640 1 -- 172.21.15.98:0/139860119 >> 172.21.15.98:0/139860119 conn(0x7f6340075070 msgr2=0x7f63400b1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:29.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.355+0000 7f4cff917640 1 -- 172.21.15.98:0/2253282807 wait complete. 2024-09-19T08:38:29.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f4ad9ffb640 1 -- 172.21.15.98:0/2569799535 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad0001600 con 0x7f4adc06c3a0 2024-09-19T08:38:29.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f4ad9ffb640 1 -- 172.21.15.98:0/2569799535 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ad40090d0 con 0x7f4adc06c3a0 2024-09-19T08:38:29.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f5e1b7fe640 1 -- 172.21.15.98:0/2293926374 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e100b2070 con 0x7f5e1c0b70e0 2024-09-19T08:38:29.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f633ffff640 1 -- 172.21.15.98:0/139860119 shutdown_connections 2024-09-19T08:38:29.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f8149aaf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81440ce4a0 0x7f8144153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f8149aaf640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f814415e110 0x7f8144154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f81437fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81440ce4a0 0x7f8144153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f81437fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81440ce4a0 0x7f8144153980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37794/0 (socket says 172.21.15.98:37794) 2024-09-19T08:38:29.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f5e1b7fe640 1 -- 172.21.15.98:0/2293926374 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e100ad680 con 0x7f5e1c0b70e0 2024-09-19T08:38:29.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f81437fe640 1 -- 172.21.15.98:0/3174940633 learned_addr learned my addr 172.21.15.98:0/3174940633 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:29.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f8149aaf640 1 --2- 172.21.15.98:0/3174940633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f814415ec00 0x7f81440b0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f8149aaf640 1 -- 172.21.15.98:0/3174940633 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f81440b3db0 con 0x7f81440ce4a0 2024-09-19T08:38:29.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f8149aaf640 1 -- 172.21.15.98:0/3174940633 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f81440b3c30 con 0x7f814415ec00 2024-09-19T08:38:29.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f8149aaf640 1 -- 172.21.15.98:0/3174940633 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f81440b3f30 con 0x7f814415e110 2024-09-19T08:38:29.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f4cff917640 1 Processor -- start 2024-09-19T08:38:29.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f8142ffd640 1 --2- 172.21.15.98:0/3174940633 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f814415e110 0x7f8144154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f8143fff640 1 --2- 172.21.15.98:0/3174940633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f814415ec00 0x7f81440b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f4ad9ffb640 1 -- 172.21.15.98:0/2569799535 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad40a66d0 con 0x7f4adc06c3a0 2024-09-19T08:38:29.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f4adbfff640 1 -- 172.21.15.98:0/2569799535 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4adc06c3a0 msgr2=0x7f4adc0af0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f56859fd640 1 --2- 172.21.15.98:0/2983669385 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56800cd290 0x7f568016d4a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f567c0097c0 tx=0x7f567c01ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c471f1531787261 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f5677fff640 1 -- 172.21.15.98:0/2983669385 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c0b5070 con 0x7f56800cd290 2024-09-19T08:38:29.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f4adbfff640 1 --2- 172.21.15.98:0/2569799535 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4adc06c3a0 0x7f4adc0af0c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4009e80 tx=0x7f4ad401c820 comp rx=0 tx=0).stop 2024-09-19T08:38:29.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f5677fff640 1 -- 172.21.15.98:0/2983669385 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f567c0a8970 con 0x7f56800cd290 2024-09-19T08:38:29.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f5677fff640 1 -- 172.21.15.98:0/2983669385 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c0a8d80 con 0x7f56800cd290 2024-09-19T08:38:29.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f81437fe640 1 -- 172.21.15.98:0/3174940633 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f814415e110 msgr2=0x7f8144154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f81437fe640 1 --2- 172.21.15.98:0/3174940633 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f814415e110 0x7f8144154050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f81437fe640 1 -- 172.21.15.98:0/3174940633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f814415ec00 msgr2=0x7f81440b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f81437fe640 1 --2- 172.21.15.98:0/3174940633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f814415ec00 0x7f81440b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f81437fe640 1 -- 172.21.15.98:0/3174940633 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f814415beb0 con 0x7f81440ce4a0 2024-09-19T08:38:29.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f56861fe640 1 -- 172.21.15.98:0/2983669385 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56800cd290 msgr2=0x7f568016d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f56861fe640 1 --2- 172.21.15.98:0/2983669385 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56800cd290 0x7f568016d4a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f567c0097c0 tx=0x7f567c01ec70 comp rx=0 tx=0).stop 2024-09-19T08:38:29.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f8142ffd640 1 --2- 172.21.15.98:0/3174940633 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f814415e110 0x7f8144154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:38:29.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f633ffff640 1 -- 172.21.15.98:0/139860119 wait complete. 2024-09-19T08:38:29.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f4cff917640 1 -- start start 2024-09-19T08:38:29.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f56861fe640 1 -- 172.21.15.98:0/2983669385 shutdown_connections 2024-09-19T08:38:29.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f56861fe640 1 --2- 172.21.15.98:0/2983669385 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56800cd290 0x7f568016d4a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f56861fe640 1 -- 172.21.15.98:0/2983669385 >> 172.21.15.98:0/2983669385 conn(0x7f568006d080 msgr2=0x7f568006d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:29.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f4adbfff640 1 -- 172.21.15.98:0/2569799535 shutdown_connections 2024-09-19T08:38:29.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f4adbfff640 1 --2- 172.21.15.98:0/2569799535 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4adc06c3a0 0x7f4adc0af0c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f4adbfff640 1 -- 172.21.15.98:0/2569799535 >> 172.21.15.98:0/2569799535 conn(0x7f4adc075070 msgr2=0x7f4adc0b1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:29.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f0ed7fff640 1 --2- 172.21.15.98:0/1029394861 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ed00e4f00 0x7f0ed00e18e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0ed807bab0 tx=0x7f0ed80987c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab793e769c0f6f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f5e22e46640 1 -- 172.21.15.98:0/2293926374 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e1c0b70e0 msgr2=0x7f5e1c0d2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f5e22e46640 1 --2- 172.21.15.98:0/2293926374 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e1c0b70e0 0x7f5e1c0d2120 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c071780 tx=0x7f5e1001c4b0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f5e1b7fe640 1 -- 172.21.15.98:0/2293926374 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e100ada30 con 0x7f5e1c0b70e0 2024-09-19T08:38:29.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f56861fe640 1 -- 172.21.15.98:0/2983669385 shutdown_connections 2024-09-19T08:38:29.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f633ffff640 1 Processor -- start 2024-09-19T08:38:29.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f5e22e46640 1 -- 172.21.15.98:0/2293926374 shutdown_connections 2024-09-19T08:38:29.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f5e22e46640 1 --2- 172.21.15.98:0/2293926374 >> v2:172.21.15.98:3300/0 conn(0x7f5e1c0b7650 0x7f5e1c0d2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f5e22e46640 1 --2- 172.21.15.98:0/2293926374 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e1c0b70e0 0x7f5e1c0d2120 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f5e22e46640 1 --2- 172.21.15.98:0/2293926374 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5e1c15f910 0x7f5e1c0d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f5e22e46640 1 -- 172.21.15.98:0/2293926374 >> 172.21.15.98:0/2293926374 conn(0x7f5e1c0b5160 msgr2=0x7f5e1c0b5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:29.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f5e22e46640 1 -- 172.21.15.98:0/2293926374 shutdown_connections 2024-09-19T08:38:29.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f81437fe640 1 --2- 172.21.15.98:0/3174940633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81440ce4a0 0x7f8144153980 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f813400c500 tx=0x7f813400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f0ed67fc640 1 -- 172.21.15.98:0/1029394861 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8122040 con 0x7f0ed00e4f00 2024-09-19T08:38:29.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f0ed67fc640 1 -- 172.21.15.98:0/1029394861 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ed8126740 con 0x7f0ed00e4f00 2024-09-19T08:38:29.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f0ed67fc640 1 -- 172.21.15.98:0/1029394861 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8126bb0 con 0x7f0ed00e4f00 2024-09-19T08:38:29.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f4adbfff640 1 -- 172.21.15.98:0/2569799535 shutdown_connections 2024-09-19T08:38:29.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f4cff917640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4cf000a380 0x7f4cf017cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f4cff917640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4cf000a7d0 0x7f4cf00e0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f4cff917640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4cf00deb70 0x7f4cf0039260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f4cff917640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf0033c00 con 0x7f4cf00deb70 2024-09-19T08:38:29.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f4cff917640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf000ac60 con 0x7f4cf000a380 2024-09-19T08:38:29.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f4cff917640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4cf0007260 con 0x7f4cf000a7d0 2024-09-19T08:38:29.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f4cfceb2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4cf000a7d0 0x7f4cf00e0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f4cfceb2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4cf000a7d0 0x7f4cf00e0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:37810/0 (socket says 172.21.15.98:37810) 2024-09-19T08:38:29.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f4cfceb2640 1 -- 172.21.15.98:0/1768905152 learned_addr learned my addr 172.21.15.98:0/1768905152 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:29.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f4cfd6b3640 1 --2- 172.21.15.98:0/1768905152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4cf000a380 0x7f4cf017cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f633ffff640 1 -- start start 2024-09-19T08:38:29.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f633ffff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63400cf210 0x7f634015db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f4cfd6b3640 1 -- 172.21.15.98:0/1768905152 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4cf000a7d0 msgr2=0x7f4cf00e0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f4cfd6b3640 1 --2- 172.21.15.98:0/1768905152 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4cf000a7d0 0x7f4cf00e0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f4cfd6b3640 1 -- 172.21.15.98:0/1768905152 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4cf00deb70 msgr2=0x7f4cf0039260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f4cfd6b3640 1 --2- 172.21.15.98:0/1768905152 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4cf00deb70 0x7f4cf0039260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f4cfd6b3640 1 -- 172.21.15.98:0/1768905152 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf00e1290 con 0x7f4cf000a380 2024-09-19T08:38:29.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f24bd3fc640 1 Processor -- start 2024-09-19T08:38:29.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f633ffff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f634015b2e0 0x7f6340154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f56861fe640 1 -- 172.21.15.98:0/2983669385 wait complete. 2024-09-19T08:38:29.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f24bd3fc640 1 -- start start 2024-09-19T08:38:29.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f633ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f634015ecd0 0x7f634015b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f812bfff640 1 -- 172.21.15.98:0/3174940633 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8134016020 con 0x7f81440ce4a0 2024-09-19T08:38:29.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f812bfff640 1 -- 172.21.15.98:0/3174940633 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8134005220 con 0x7f81440ce4a0 2024-09-19T08:38:29.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f5e22e46640 1 -- 172.21.15.98:0/2293926374 wait complete. 2024-09-19T08:38:29.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f812bfff640 1 -- 172.21.15.98:0/3174940633 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8134007030 con 0x7f81440ce4a0 2024-09-19T08:38:29.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f8149aaf640 1 -- 172.21.15.98:0/3174940633 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f814415c190 con 0x7f81440ce4a0 2024-09-19T08:38:29.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f8149aaf640 1 -- 172.21.15.98:0/3174940633 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f814415c750 con 0x7f81440ce4a0 2024-09-19T08:38:29.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f0eddbf9640 1 -- 172.21.15.98:0/1029394861 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ed00e4f00 msgr2=0x7f0ed00e18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f0eddbf9640 1 --2- 172.21.15.98:0/1029394861 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ed00e4f00 0x7f0ed00e18e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0ed807bab0 tx=0x7f0ed80987c0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f0eddbf9640 1 -- 172.21.15.98:0/1029394861 shutdown_connections 2024-09-19T08:38:29.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f0eddbf9640 1 --2- 172.21.15.98:0/1029394861 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ed00e4f00 0x7f0ed00e18e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f0eddbf9640 1 --2- 172.21.15.98:0/1029394861 >> v2:172.21.15.139:3300/0 conn(0x7f0ed0054f50 0x7f0ed0054300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f0eddbf9640 1 -- 172.21.15.98:0/1029394861 >> 172.21.15.98:0/1029394861 conn(0x7f0ed00567c0 msgr2=0x7f0ed0056bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:29.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f56861fe640 1 Processor -- start 2024-09-19T08:38:29.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f633ffff640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f63400b54b0 con 0x7f634015ecd0 2024-09-19T08:38:29.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f633ffff640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f63400b5330 con 0x7f634015b2e0 2024-09-19T08:38:29.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f24bd3fc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24b8154bc0 0x7f24b806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f24bd3fc640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24b80b48a0 con 0x7f24b81547e0 2024-09-19T08:38:29.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f24bd3fc640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24b80ccb30 con 0x7f24b806c1d0 2024-09-19T08:38:29.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f24bd3fc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f24b80ccdd0 con 0x7f24b8154bc0 2024-09-19T08:38:29.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f633ffff640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f63400ccbf0 con 0x7f63400cf210 2024-09-19T08:38:29.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.359+0000 7f4adbfff640 1 -- 172.21.15.98:0/2569799535 wait complete. 2024-09-19T08:38:29.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f4adbfff640 1 Processor -- start 2024-09-19T08:38:29.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f56861fe640 1 -- start start 2024-09-19T08:38:29.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f4cfd6b3640 1 --2- 172.21.15.98:0/1768905152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4cf000a380 0x7f4cf017cf50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4cf400d5d0 tx=0x7f4cf400da90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f633effd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63400cf210 0x7f634015db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f633effd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63400cf210 0x7f634015db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:37826/0 (socket says 172.21.15.98:37826) 2024-09-19T08:38:29.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f633effd640 1 -- 172.21.15.98:0/3631394867 learned_addr learned my addr 172.21.15.98:0/3631394867 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:29.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f24bcbfb640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f24b806c1d0 0x7f24b80ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:44962/0 (socket says 172.21.15.98:44962) 2024-09-19T08:38:29.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f24bcbfb640 1 -- 172.21.15.98:0/1615331204 learned_addr learned my addr 172.21.15.98:0/1615331204 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:29.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f24b77fe640 1 --2- 172.21.15.98:0/1615331204 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24b8154bc0 0x7f24b806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f5e22e46640 1 Processor -- start 2024-09-19T08:38:29.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f633e7fc640 1 --2- 172.21.15.98:0/3631394867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f634015b2e0 0x7f6340154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f5e22e46640 1 -- start start 2024-09-19T08:38:29.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f8149aaf640 1 -- 172.21.15.98:0/3174940633 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8124001040 con 0x7f81440ce4a0 2024-09-19T08:38:29.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f4cedffb640 1 -- 172.21.15.98:0/1768905152 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4005bb0 con 0x7f4cf000a380 2024-09-19T08:38:29.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f4cedffb640 1 -- 172.21.15.98:0/1768905152 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cf4018070 con 0x7f4cf000a380 2024-09-19T08:38:29.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f4cff917640 1 -- 172.21.15.98:0/1768905152 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cf00375a0 con 0x7f4cf000a380 2024-09-19T08:38:29.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f4cff917640 1 -- 172.21.15.98:0/1768905152 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cf0037b60 con 0x7f4cf000a380 2024-09-19T08:38:29.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f4cedffb640 1 -- 172.21.15.98:0/1768905152 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4006050 con 0x7f4cf000a380 2024-09-19T08:38:29.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f24b6ffd640 1 -- 172.21.15.98:0/1615331204 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1690142761 0 0) 0x7f24b80ccb30 con 0x7f24b806c1d0 2024-09-19T08:38:29.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f24b6ffd640 1 -- 172.21.15.98:0/1615331204 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24a0003530 con 0x7f24b806c1d0 2024-09-19T08:38:29.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f633f7fe640 1 --2- 172.21.15.98:0/3631394867 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f634015ecd0 0x7f634015b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f633effd640 1 -- 172.21.15.98:0/3631394867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f634015b2e0 msgr2=0x7f6340154ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f633effd640 1 --2- 172.21.15.98:0/3631394867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f634015b2e0 0x7f6340154ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f633effd640 1 -- 172.21.15.98:0/3631394867 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f634015ecd0 msgr2=0x7f634015b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f633effd640 1 --2- 172.21.15.98:0/3631394867 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f634015ecd0 0x7f634015b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f633effd640 1 -- 172.21.15.98:0/3631394867 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634015e250 con 0x7f63400cf210 2024-09-19T08:38:29.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f56861fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5680073740 0x7f56800af060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f56861fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56800cd290 0x7f568013b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f56861fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5680162700 0x7f5680138660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f56861fe640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5680162ae0 con 0x7f5680073740 2024-09-19T08:38:29.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f56861fe640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f56800ce890 con 0x7f56800cd290 2024-09-19T08:38:29.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f56861fe640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f56800cfd30 con 0x7f5680162700 2024-09-19T08:38:29.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f56851fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5680073740 0x7f56800af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f56851fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5680073740 0x7f56800af060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37816/0 (socket says 172.21.15.98:37816) 2024-09-19T08:38:29.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f56851fc640 1 -- 172.21.15.98:0/3290334630 learned_addr learned my addr 172.21.15.98:0/3290334630 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:29.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f633effd640 1 --2- 172.21.15.98:0/3631394867 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63400cf210 0x7f634015db90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f633000c500 tx=0x7f633000c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f5e22e46640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5e1c0b7650 0x7f5e1c14e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f5e22e46640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e1c15f910 0x7f5e1c06e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f5e22e46640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e1c157be0 0x7f5e1c0b2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f5e22e46640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1c0ce0b0 con 0x7f5e1c15f910 2024-09-19T08:38:29.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f5e22e46640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1c0cdf30 con 0x7f5e1c157be0 2024-09-19T08:38:29.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f5e22e46640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5e1c0ce230 con 0x7f5e1c0b7650 2024-09-19T08:38:29.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f5e1bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e1c15f910 0x7f5e1c06e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f5e1bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e1c15f910 0x7f5e1c06e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37828/0 (socket says 172.21.15.98:37828) 2024-09-19T08:38:29.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f5e1bfff640 1 -- 172.21.15.98:0/2924660291 learned_addr learned my addr 172.21.15.98:0/2924660291 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:29.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f5e20be2640 1 --2- 172.21.15.98:0/2924660291 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5e1c0b7650 0x7f5e1c14e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f24b77fe640 1 -- 172.21.15.98:0/1615331204 >> v1:172.21.15.139:6789/0 conn(0x7f24b806c1d0 legacy=0x7f24b80ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f24b77fe640 1 -- 172.21.15.98:0/1615331204 >> v1:172.21.15.98:6789/0 conn(0x7f24b81547e0 legacy=0x7f24b80b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:29.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f24b77fe640 1 -- 172.21.15.98:0/1615331204 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b80cd0b0 con 0x7f24b8154bc0 2024-09-19T08:38:29.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f0eddbf9640 1 -- 172.21.15.98:0/1029394861 shutdown_connections 2024-09-19T08:38:29.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f0eddbf9640 1 -- 172.21.15.98:0/1029394861 wait complete. 2024-09-19T08:38:29.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f4adbfff640 1 -- start start 2024-09-19T08:38:29.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f4adbfff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4adc06c3a0 0x7f4adc2039c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f4adbfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4adc0cdff0 0x7f4adc1571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f4adbfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4adc144b10 0x7f4adc157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f4adbfff640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4adc16c370 con 0x7f4adc144b10 2024-09-19T08:38:29.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f4adbfff640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4adc16c1f0 con 0x7f4adc0cdff0 2024-09-19T08:38:29.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f4adbfff640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4adc16c4f0 con 0x7f4adc06c3a0 2024-09-19T08:38:29.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f4ada7fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4adc0cdff0 0x7f4adc1571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f4adaffd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4adc06c3a0 0x7f4adc2039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f4adb7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4adc144b10 0x7f4adc157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f4adb7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4adc144b10 0x7f4adc157710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37818/0 (socket says 172.21.15.98:37818) 2024-09-19T08:38:29.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f4adb7fe640 1 -- 172.21.15.98:0/759147497 learned_addr learned my addr 172.21.15.98:0/759147497 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:29.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f63277fe640 1 -- 172.21.15.98:0/3631394867 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6330016020 con 0x7f63400cf210 2024-09-19T08:38:29.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f633ffff640 1 -- 172.21.15.98:0/3631394867 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f634015e530 con 0x7f63400cf210 2024-09-19T08:38:29.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f633ffff640 1 -- 172.21.15.98:0/3631394867 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f63401fcac0 con 0x7f63400cf210 2024-09-19T08:38:29.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f63277fe640 1 -- 172.21.15.98:0/3631394867 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6330005220 con 0x7f63400cf210 2024-09-19T08:38:29.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f63277fe640 1 -- 172.21.15.98:0/3631394867 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6330007030 con 0x7f63400cf210 2024-09-19T08:38:29.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f5e1bfff640 1 -- 172.21.15.98:0/2924660291 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5e1c0b7650 msgr2=0x7f5e1c14e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f5e1bfff640 1 --2- 172.21.15.98:0/2924660291 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5e1c0b7650 0x7f5e1c14e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f5e1bfff640 1 -- 172.21.15.98:0/2924660291 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e1c157be0 msgr2=0x7f5e1c0b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f5e1bfff640 1 --2- 172.21.15.98:0/2924660291 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5e1c157be0 0x7f5e1c0b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f56859fd640 1 --2- 172.21.15.98:0/3290334630 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5680162700 0x7f5680138660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f5e1bfff640 1 -- 172.21.15.98:0/2924660291 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e1c0705b0 con 0x7f5e1c15f910 2024-09-19T08:38:29.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f4cff917640 1 -- 172.21.15.98:0/1768905152 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4cf000bbd0 con 0x7f4cf000a380 2024-09-19T08:38:29.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f5e1bfff640 1 --2- 172.21.15.98:0/2924660291 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e1c15f910 0x7f5e1c06e8b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5e1001ab60 tx=0x7f5e10019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f0eddbf9640 1 Processor -- start 2024-09-19T08:38:29.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f633ffff640 1 -- 172.21.15.98:0/3631394867 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6320001040 con 0x7f63400cf210 2024-09-19T08:38:29.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f56851fc640 1 -- 172.21.15.98:0/3290334630 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5680162700 msgr2=0x7f5680138660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f56851fc640 1 --2- 172.21.15.98:0/3290334630 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5680162700 0x7f5680138660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f56851fc640 1 -- 172.21.15.98:0/3290334630 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56800cd290 msgr2=0x7f568013b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f56851fc640 1 --2- 172.21.15.98:0/3290334630 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56800cd290 0x7f568013b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f56851fc640 1 -- 172.21.15.98:0/3290334630 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568013c130 con 0x7f5680073740 2024-09-19T08:38:29.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f4ada7fc640 1 -- 172.21.15.98:0/759147497 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4adc06c3a0 msgr2=0x7f4adc2039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f4ada7fc640 1 --2- 172.21.15.98:0/759147497 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4adc06c3a0 0x7f4adc2039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f4ada7fc640 1 -- 172.21.15.98:0/759147497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4adc144b10 msgr2=0x7f4adc157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f4ada7fc640 1 --2- 172.21.15.98:0/759147497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4adc144b10 0x7f4adc157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f4ada7fc640 1 -- 172.21.15.98:0/759147497 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4adc0add10 con 0x7f4adc0cdff0 2024-09-19T08:38:29.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f0eddbf9640 1 -- start start 2024-09-19T08:38:29.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f812bfff640 1 -- 172.21.15.98:0/3174940633 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f81340071f0 con 0x7f81440ce4a0 2024-09-19T08:38:29.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f5e197fa640 1 -- 172.21.15.98:0/2924660291 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e100bb070 con 0x7f5e1c15f910 2024-09-19T08:38:29.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f5e197fa640 1 -- 172.21.15.98:0/2924660291 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e10004260 con 0x7f5e1c15f910 2024-09-19T08:38:29.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f4ada7fc640 1 --2- 172.21.15.98:0/759147497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4adc0cdff0 0x7f4adc1571d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4ad000c7b0 tx=0x7f4ad000cc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.363+0000 7f5e197fa640 1 -- 172.21.15.98:0/2924660291 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e100b2040 con 0x7f5e1c15f910 2024-09-19T08:38:29.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f56851fc640 1 --2- 172.21.15.98:0/3290334630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5680073740 0x7f56800af060 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f567000c480 tx=0x7f567000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f5e22e46640 1 -- 172.21.15.98:0/2924660291 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e1c14ea60 con 0x7f5e1c15f910 2024-09-19T08:38:29.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f5e22e46640 1 -- 172.21.15.98:0/2924660291 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e1c070820 con 0x7f5e1c15f910 2024-09-19T08:38:29.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f812bfff640 1 --2- 172.21.15.98:0/3174940633 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8130067910 0x7f8130069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f812bfff640 1 -- 172.21.15.98:0/3174940633 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f81340863e0 con 0x7f81440ce4a0 2024-09-19T08:38:29.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f812bfff640 1 --2- 172.21.15.98:0/3174940633 >> [v2:172.21.15.98:6800/3643795173,v1:172.21.15.98:6801/3643795173] conn(0x7f813006fb30 0x7f8130071f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f812bfff640 1 -- 172.21.15.98:0/3174940633 --> [v2:172.21.15.98:6800/3643795173,v1:172.21.15.98:6801/3643795173] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8130072600 con 0x7f813006fb30 2024-09-19T08:38:29.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f812bfff640 1 -- 172.21.15.98:0/3174940633 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8134086790 con 0x7f81440ce4a0 2024-09-19T08:38:29.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f4cedffb640 1 -- 172.21.15.98:0/1768905152 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4cf40061f0 con 0x7f4cf000a380 2024-09-19T08:38:29.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f8142ffd640 1 --2- 172.21.15.98:0/3174940633 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8130067910 0x7f8130069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f5675ffb640 1 -- 172.21.15.98:0/3290334630 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5670005480 con 0x7f5680073740 2024-09-19T08:38:29.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f56861fe640 1 -- 172.21.15.98:0/3290334630 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f568013bb70 con 0x7f5680073740 2024-09-19T08:38:29.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f56861fe640 1 -- 172.21.15.98:0/3290334630 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5680137e60 con 0x7f5680073740 2024-09-19T08:38:29.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f5675ffb640 1 -- 172.21.15.98:0/3290334630 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5670003600 con 0x7f5680073740 2024-09-19T08:38:29.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f566b7fe640 1 -- 172.21.15.98:0/3290334630 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5660001040 con 0x7f5680073740 2024-09-19T08:38:29.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f5e22e46640 1 -- 172.21.15.98:0/2924660291 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5e1c0b70e0 con 0x7f5e1c15f910 2024-09-19T08:38:29.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f8143fff640 1 --2- 172.21.15.98:0/3174940633 >> [v2:172.21.15.98:6800/3643795173,v1:172.21.15.98:6801/3643795173] conn(0x7f813006fb30 0x7f8130071f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f8142ffd640 1 --2- 172.21.15.98:0/3174940633 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8130067910 0x7f8130069dd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f81441fecb0 tx=0x7f813800d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f8143fff640 1 --2- 172.21.15.98:0/3174940633 >> [v2:172.21.15.98:6800/3643795173,v1:172.21.15.98:6801/3643795173] conn(0x7f813006fb30 0x7f8130071f50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f24b77fe640 1 --2- 172.21.15.98:0/1615331204 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24b8154bc0 0x7f24b806bc90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f24a80087f0 tx=0x7f24a801e9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ad78c4baf19df91 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f24b6ffd640 1 -- 172.21.15.98:0/1615331204 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a80ad070 con 0x7f24b8154bc0 2024-09-19T08:38:29.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f24b6ffd640 1 -- 172.21.15.98:0/1615331204 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24a80a7880 con 0x7f24b8154bc0 2024-09-19T08:38:29.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f24b6ffd640 1 -- 172.21.15.98:0/1615331204 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a80a7c50 con 0x7f24b8154bc0 2024-09-19T08:38:29.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f0eddbf9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ed0054f50 0x7f0ed00ddb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f0eddbf9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ed0055330 0x7f0ed00e2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f0eddbf9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ed00e4f00 0x7f0ed00395f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f0eddbf9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed0039000 con 0x7f0ed00e4f00 2024-09-19T08:38:29.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f0eddbf9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed00ed660 con 0x7f0ed0055330 2024-09-19T08:38:29.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f0eddbf9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0ed00e38f0 con 0x7f0ed0054f50 2024-09-19T08:38:29.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f0ed6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ed0055330 0x7f0ed00e2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f0ed6ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ed0055330 0x7f0ed00e2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:49900/0 (socket says 172.21.15.98:49900) 2024-09-19T08:38:29.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f0ed6ffd640 1 -- 172.21.15.98:0/3657062065 learned_addr learned my addr 172.21.15.98:0/3657062065 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:29.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f0ed7fff640 1 --2- 172.21.15.98:0/3657062065 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ed00e4f00 0x7f0ed00395f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f0ed77fe640 1 --2- 172.21.15.98:0/3657062065 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ed0054f50 0x7f0ed00ddb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f0ed7fff640 1 -- 172.21.15.98:0/3657062065 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ed0054f50 msgr2=0x7f0ed00ddb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f0ed7fff640 1 --2- 172.21.15.98:0/3657062065 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ed0054f50 0x7f0ed00ddb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f0ed7fff640 1 -- 172.21.15.98:0/3657062065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ed0055330 msgr2=0x7f0ed00e2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f0ed7fff640 1 --2- 172.21.15.98:0/3657062065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ed0055330 0x7f0ed00e2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f0ed7fff640 1 -- 172.21.15.98:0/3657062065 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed00e2ce0 con 0x7f0ed00e4f00 2024-09-19T08:38:29.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f0ed6ffd640 1 --2- 172.21.15.98:0/3657062065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ed0055330 0x7f0ed00e2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:38:29.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f0ed7fff640 1 --2- 172.21.15.98:0/3657062065 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ed00e4f00 0x7f0ed00395f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0ed806b250 tx=0x7f0ed806b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f4ac3fff640 1 -- 172.21.15.98:0/759147497 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad000ce60 con 0x7f4adc0cdff0 2024-09-19T08:38:29.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f4ac3fff640 1 -- 172.21.15.98:0/759147497 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ad000a640 con 0x7f4adc0cdff0 2024-09-19T08:38:29.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f4adbfff640 1 -- 172.21.15.98:0/759147497 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4adc0adff0 con 0x7f4adc0cdff0 2024-09-19T08:38:29.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f4adbfff640 1 -- 172.21.15.98:0/759147497 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4adc0ae5b0 con 0x7f4adc0cdff0 2024-09-19T08:38:29.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f4adbfff640 1 -- 172.21.15.98:0/759147497 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4adc203f00 con 0x7f4adc0cdff0 2024-09-19T08:38:29.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f4ac3fff640 1 -- 172.21.15.98:0/759147497 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad000aa30 con 0x7f4adc0cdff0 2024-09-19T08:38:29.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f4cedffb640 1 --2- 172.21.15.98:0/1768905152 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4cdc067910 0x7f4cdc069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f4cedffb640 1 -- 172.21.15.98:0/1768905152 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4cf4086720 con 0x7f4cf000a380 2024-09-19T08:38:29.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f4cedffb640 1 --2- 172.21.15.98:0/1768905152 >> [v2:172.21.15.139:6816/688014474,v1:172.21.15.139:6817/688014474] conn(0x7f4cdc06fb90 0x7f4cdc071fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f4cedffb640 1 -- 172.21.15.98:0/1768905152 --> [v2:172.21.15.139:6816/688014474,v1:172.21.15.139:6817/688014474] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4cdc072660 con 0x7f4cdc06fb90 2024-09-19T08:38:29.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f4cedffb640 1 -- 172.21.15.98:0/1768905152 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4cf4086b10 con 0x7f4cf000a380 2024-09-19T08:38:29.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f4cfceb2640 1 --2- 172.21.15.98:0/1768905152 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4cdc067910 0x7f4cdc069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f4cfdeb4640 1 --2- 172.21.15.98:0/1768905152 >> [v2:172.21.15.139:6816/688014474,v1:172.21.15.139:6817/688014474] conn(0x7f4cdc06fb90 0x7f4cdc071fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f4cfceb2640 1 --2- 172.21.15.98:0/1768905152 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4cdc067910 0x7f4cdc069dd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4cf00391b0 tx=0x7f4ce8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f63277fe640 1 -- 172.21.15.98:0/3631394867 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f63300071d0 con 0x7f63400cf210 2024-09-19T08:38:29.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f4cfdeb4640 1 --2- 172.21.15.98:0/1768905152 >> [v2:172.21.15.139:6816/688014474,v1:172.21.15.139:6817/688014474] conn(0x7f4cdc06fb90 0x7f4cdc071fb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f63277fe640 1 --2- 172.21.15.98:0/3631394867 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f632c0678c0 0x7f632c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f63277fe640 1 -- 172.21.15.98:0/3631394867 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6330086370 con 0x7f63400cf210 2024-09-19T08:38:29.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f63277fe640 1 --2- 172.21.15.98:0/3631394867 >> [v2:172.21.15.98:6817/1538482309,v1:172.21.15.98:6820/1538482309] conn(0x7f632c06fae0 0x7f632c071f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f63277fe640 1 -- 172.21.15.98:0/3631394867 --> [v2:172.21.15.98:6817/1538482309,v1:172.21.15.98:6820/1538482309] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f63300076a0 con 0x7f632c06fae0 2024-09-19T08:38:29.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f63277fe640 1 -- 172.21.15.98:0/3631394867 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6330086760 con 0x7f63400cf210 2024-09-19T08:38:29.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f5675ffb640 1 -- 172.21.15.98:0/3290334630 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56700039f0 con 0x7f5680073740 2024-09-19T08:38:29.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f633e7fc640 1 --2- 172.21.15.98:0/3631394867 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f632c0678c0 0x7f632c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f633f7fe640 1 --2- 172.21.15.98:0/3631394867 >> [v2:172.21.15.98:6817/1538482309,v1:172.21.15.98:6820/1538482309] conn(0x7f632c06fae0 0x7f632c071f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f24bd3fc640 1 -- 172.21.15.98:0/1615331204 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24b8154bc0 msgr2=0x7f24b806bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f24bd3fc640 1 --2- 172.21.15.98:0/1615331204 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24b8154bc0 0x7f24b806bc90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f24a80087f0 tx=0x7f24a801e9b0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f5e197fa640 1 -- 172.21.15.98:0/2924660291 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5e100ad680 con 0x7f5e1c15f910 2024-09-19T08:38:29.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f812bfff640 1 -- 172.21.15.98:0/3174940633 <== osd.3 v2:172.21.15.98:6800/3643795173 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8130072600 con 0x7f813006fb30 2024-09-19T08:38:29.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f633f7fe640 1 --2- 172.21.15.98:0/3631394867 >> [v2:172.21.15.98:6817/1538482309,v1:172.21.15.98:6820/1538482309] conn(0x7f632c06fae0 0x7f632c071f00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f633e7fc640 1 --2- 172.21.15.98:0/3631394867 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f632c0678c0 0x7f632c069d80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f63340b1000 tx=0x7f6334008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.367+0000 7f0ec3fff640 1 -- 172.21.15.98:0/3657062065 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8130400 con 0x7f0ed00e4f00 2024-09-19T08:38:29.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.371+0000 7f0eddbf9640 1 -- 172.21.15.98:0/3657062065 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ed00e2fc0 con 0x7f0ed00e4f00 2024-09-19T08:38:29.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.371+0000 7f0eddbf9640 1 -- 172.21.15.98:0/3657062065 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ed00e3580 con 0x7f0ed00e4f00 2024-09-19T08:38:29.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.371+0000 7f0ec3fff640 1 -- 172.21.15.98:0/3657062065 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ed8122030 con 0x7f0ed00e4f00 2024-09-19T08:38:29.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.371+0000 7f5e197fa640 1 --2- 172.21.15.98:0/2924660291 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5e00067990 0x7f5e00069e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.371+0000 7f5e197fa640 1 -- 172.21.15.98:0/2924660291 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5e1011eda0 con 0x7f5e1c15f910 2024-09-19T08:38:29.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.371+0000 7f5e197fa640 1 --2- 172.21.15.98:0/2924660291 >> [v2:172.21.15.98:6808/2466005517,v1:172.21.15.98:6809/2466005517] conn(0x7f5e0006fb90 0x7f5e00071fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.371+0000 7f5e197fa640 1 -- 172.21.15.98:0/2924660291 --> [v2:172.21.15.98:6808/2466005517,v1:172.21.15.98:6809/2466005517] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5e100bce50 con 0x7f5e0006fb90 2024-09-19T08:38:29.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.371+0000 7f5e20be2640 1 --2- 172.21.15.98:0/2924660291 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5e00067990 0x7f5e00069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.371+0000 7f5e213e3640 1 --2- 172.21.15.98:0/2924660291 >> [v2:172.21.15.98:6808/2466005517,v1:172.21.15.98:6809/2466005517] conn(0x7f5e0006fb90 0x7f5e00071fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.371+0000 7f5e20be2640 1 --2- 172.21.15.98:0/2924660291 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5e00067990 0x7f5e00069e50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5e0800ab60 tx=0x7f5e08008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.371+0000 7f24bd3fc640 1 -- 172.21.15.98:0/1615331204 shutdown_connections 2024-09-19T08:38:29.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.371+0000 7f24bd3fc640 1 --2- 172.21.15.98:0/1615331204 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24b8154bc0 0x7f24b806bc90 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.371+0000 7f24bd3fc640 1 -- 172.21.15.98:0/1615331204 >> 172.21.15.98:0/1615331204 conn(0x7f24b8075070 msgr2=0x7f24b80b1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:29.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.371+0000 7f0ec3fff640 1 -- 172.21.15.98:0/3657062065 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8130820 con 0x7f0ed00e4f00 2024-09-19T08:38:29.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.371+0000 7f0eddbf9640 1 -- 172.21.15.98:0/3657062065 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0eb8001040 con 0x7f0ed00e4f00 2024-09-19T08:38:29.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.371+0000 7f4cedffb640 1 -- 172.21.15.98:0/1768905152 <== osd.7 v2:172.21.15.139:6816/688014474 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4cdc072660 con 0x7f4cdc06fb90 2024-09-19T08:38:29.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.371+0000 7f0ec3fff640 1 -- 172.21.15.98:0/3657062065 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0ed8137030 con 0x7f0ed00e4f00 2024-09-19T08:38:29.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.371+0000 7f0ec3fff640 1 --2- 172.21.15.98:0/3657062065 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0ebc0678c0 0x7f0ebc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.371+0000 7f0ec3fff640 1 -- 172.21.15.98:0/3657062065 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0ed819b210 con 0x7f0ed00e4f00 2024-09-19T08:38:29.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.371+0000 7f63277fe640 1 -- 172.21.15.98:0/3631394867 <== osd.1 v2:172.21.15.98:6817/1538482309 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f63300076a0 con 0x7f632c06fae0 2024-09-19T08:38:29.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.371+0000 7f0ec3fff640 1 --2- 172.21.15.98:0/3657062065 >> [v2:172.21.15.139:6808/520689888,v1:172.21.15.139:6809/520689888] conn(0x7f0ebc06fa60 0x7f0ebc071e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.371+0000 7f0ed77fe640 1 --2- 172.21.15.98:0/3657062065 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0ebc0678c0 0x7f0ebc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.371+0000 7f4ac3fff640 1 -- 172.21.15.98:0/759147497 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4ad000abd0 con 0x7f4adc0cdff0 2024-09-19T08:38:29.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.371+0000 7f4ac3fff640 1 --2- 172.21.15.98:0/759147497 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4ac8067910 0x7f4ac8069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.371+0000 7f4ac3fff640 1 -- 172.21.15.98:0/759147497 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4ad0086500 con 0x7f4adc0cdff0 2024-09-19T08:38:29.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.371+0000 7f4ac3fff640 1 --2- 172.21.15.98:0/759147497 >> [v2:172.21.15.139:6800/2917689006,v1:172.21.15.139:6801/2917689006] conn(0x7f4ac806fab0 0x7f4ac8071ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.371+0000 7f4ac3fff640 1 -- 172.21.15.98:0/759147497 --> [v2:172.21.15.139:6800/2917689006,v1:172.21.15.139:6801/2917689006] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4ad0003e50 con 0x7f4ac806fab0 2024-09-19T08:38:29.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.371+0000 7f4ac3fff640 1 -- 172.21.15.98:0/759147497 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4ad0086990 con 0x7f4adc0cdff0 2024-09-19T08:38:29.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.371+0000 7f4adaffd640 1 --2- 172.21.15.98:0/759147497 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4ac8067910 0x7f4ac8069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.371+0000 7f0ec3fff640 1 -- 172.21.15.98:0/3657062065 --> [v2:172.21.15.139:6808/520689888,v1:172.21.15.139:6809/520689888] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0ed813cec0 con 0x7f0ebc06fa60 2024-09-19T08:38:29.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.371+0000 7f0ec3fff640 1 -- 172.21.15.98:0/3657062065 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0ed81a2050 con 0x7f0ed00e4f00 2024-09-19T08:38:29.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.371+0000 7f0ed6ffd640 1 --2- 172.21.15.98:0/3657062065 >> [v2:172.21.15.139:6808/520689888,v1:172.21.15.139:6809/520689888] conn(0x7f0ebc06fa60 0x7f0ebc071e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.371+0000 7f4adaffd640 1 --2- 172.21.15.98:0/759147497 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4ac8067910 0x7f4ac8069dd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4acc007fd0 tx=0x7f4acc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.371+0000 7f0ed77fe640 1 --2- 172.21.15.98:0/3657062065 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0ebc0678c0 0x7f0ebc069d80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4006fd0 tx=0x7f0ec4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.371+0000 7f5e213e3640 1 --2- 172.21.15.98:0/2924660291 >> [v2:172.21.15.98:6808/2466005517,v1:172.21.15.98:6809/2466005517] conn(0x7f5e0006fb90 0x7f5e00071fb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.375+0000 7f5e197fa640 1 -- 172.21.15.98:0/2924660291 <== osd.2 v2:172.21.15.98:6808/2466005517 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5e100bce50 con 0x7f5e0006fb90 2024-09-19T08:38:29.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.375+0000 7f0ed6ffd640 1 --2- 172.21.15.98:0/3657062065 >> [v2:172.21.15.139:6808/520689888,v1:172.21.15.139:6809/520689888] conn(0x7f0ebc06fa60 0x7f0ebc071e80 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.375+0000 7f0ec3fff640 1 -- 172.21.15.98:0/3657062065 <== osd.5 v2:172.21.15.139:6808/520689888 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0ed813cec0 con 0x7f0ebc06fa60 2024-09-19T08:38:29.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.375+0000 7f4adb7fe640 1 --2- 172.21.15.98:0/759147497 >> [v2:172.21.15.139:6800/2917689006,v1:172.21.15.139:6801/2917689006] conn(0x7f4ac806fab0 0x7f4ac8071ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.375+0000 7f4adb7fe640 1 --2- 172.21.15.98:0/759147497 >> [v2:172.21.15.139:6800/2917689006,v1:172.21.15.139:6801/2917689006] conn(0x7f4ac806fab0 0x7f4ac8071ed0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.379+0000 7f4ac3fff640 1 -- 172.21.15.98:0/759147497 <== osd.6 v2:172.21.15.139:6800/2917689006 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4ad0003e50 con 0x7f4ac806fab0 2024-09-19T08:38:29.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.379+0000 7f24bd3fc640 1 -- 172.21.15.98:0/1615331204 shutdown_connections 2024-09-19T08:38:29.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.379+0000 7f24bd3fc640 1 -- 172.21.15.98:0/1615331204 wait complete. 2024-09-19T08:38:29.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.379+0000 7f24bd3fc640 1 Processor -- start 2024-09-19T08:38:29.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.379+0000 7f24bd3fc640 1 -- start start 2024-09-19T08:38:29.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.383+0000 7f5675ffb640 1 -- 172.21.15.98:0/3290334630 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5670025430 con 0x7f5680073740 2024-09-19T08:38:29.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.383+0000 7f5e197fa640 1 -- 172.21.15.98:0/2924660291 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5e100eb510 con 0x7f5e1c15f910 2024-09-19T08:38:29.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.383+0000 7f24bd3fc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24b806c1d0 0x7f24b81527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.383+0000 7f5675ffb640 1 --2- 172.21.15.98:0/3290334630 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5664067990 0x7f5664069e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.383+0000 7f24bd3fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24b81547e0 0x7f24b81440e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.383+0000 7f5675ffb640 1 -- 172.21.15.98:0/3290334630 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5670086190 con 0x7f5680073740 2024-09-19T08:38:29.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.383+0000 7f24bd3fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24b8154bc0 0x7f24b8149dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.383+0000 7f24bd3fc640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f24b80cf880 con 0x7f24b81547e0 2024-09-19T08:38:29.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.383+0000 7f24bd3fc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f24b80cf700 con 0x7f24b8154bc0 2024-09-19T08:38:29.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.383+0000 7f24bd3fc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f24b80cfa00 con 0x7f24b806c1d0 2024-09-19T08:38:29.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.383+0000 7f24b7fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24b806c1d0 0x7f24b81527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.383+0000 7f24bcbfb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24b8154bc0 0x7f24b8149dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.383+0000 7f24b7fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24b806c1d0 0x7f24b81527b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:37888/0 (socket says 172.21.15.98:37888) 2024-09-19T08:38:29.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.383+0000 7f24b7fff640 1 -- 172.21.15.98:0/2827117165 learned_addr learned my addr 172.21.15.98:0/2827117165 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:29.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.383+0000 7f24b77fe640 1 --2- 172.21.15.98:0/2827117165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24b81547e0 0x7f24b81440e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.383+0000 7f5675ffb640 1 --2- 172.21.15.98:0/3290334630 >> [v2:172.21.15.98:6816/2191254527,v1:172.21.15.98:6818/2191254527] conn(0x7f566406fbb0 0x7f5664071fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.383+0000 7f5675ffb640 1 -- 172.21.15.98:0/3290334630 --> [v2:172.21.15.98:6816/2191254527,v1:172.21.15.98:6818/2191254527] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5670025e50 con 0x7f566406fbb0 2024-09-19T08:38:29.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.383+0000 7f5675ffb640 1 -- 172.21.15.98:0/3290334630 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5670086580 con 0x7f5680073740 2024-09-19T08:38:29.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.383+0000 7f56859fd640 1 --2- 172.21.15.98:0/3290334630 >> [v2:172.21.15.98:6816/2191254527,v1:172.21.15.98:6818/2191254527] conn(0x7f566406fbb0 0x7f5664071fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.383+0000 7f56849fb640 1 --2- 172.21.15.98:0/3290334630 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5664067990 0x7f5664069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.383+0000 7f24b77fe640 1 -- 172.21.15.98:0/2827117165 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24b806c1d0 msgr2=0x7f24b81527b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.383+0000 7f24b77fe640 1 --2- 172.21.15.98:0/2827117165 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24b806c1d0 0x7f24b81527b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.383+0000 7f24b77fe640 1 -- 172.21.15.98:0/2827117165 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24b8154bc0 msgr2=0x7f24b8149dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.383+0000 7f24b77fe640 1 --2- 172.21.15.98:0/2827117165 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24b8154bc0 0x7f24b8149dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.383+0000 7f24b77fe640 1 -- 172.21.15.98:0/2827117165 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b8146fd0 con 0x7f24b81547e0 2024-09-19T08:38:29.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.383+0000 7f24b7fff640 1 --2- 172.21.15.98:0/2827117165 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24b806c1d0 0x7f24b81527b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:38:29.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.383+0000 7f24b77fe640 1 --2- 172.21.15.98:0/2827117165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24b81547e0 0x7f24b81440e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f24a80af000 tx=0x7f24a801b750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.383+0000 7f56859fd640 1 --2- 172.21.15.98:0/3290334630 >> [v2:172.21.15.98:6816/2191254527,v1:172.21.15.98:6818/2191254527] conn(0x7f566406fbb0 0x7f5664071fd0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.383+0000 7f24b4ff9640 1 -- 172.21.15.98:0/2827117165 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a80ad040 con 0x7f24b81547e0 2024-09-19T08:38:29.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.383+0000 7f24b4ff9640 1 -- 172.21.15.98:0/2827117165 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24a8008df0 con 0x7f24b81547e0 2024-09-19T08:38:29.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.383+0000 7f24bd3fc640 1 -- 172.21.15.98:0/2827117165 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24b814b9b0 con 0x7f24b81547e0 2024-09-19T08:38:29.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.383+0000 7f24b4ff9640 1 -- 172.21.15.98:0/2827117165 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a80b97d0 con 0x7f24b81547e0 2024-09-19T08:38:29.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.383+0000 7f24bd3fc640 1 -- 172.21.15.98:0/2827117165 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24b8151fb0 con 0x7f24b81547e0 2024-09-19T08:38:29.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.383+0000 7f5675ffb640 1 -- 172.21.15.98:0/3290334630 <== osd.0 v2:172.21.15.98:6816/2191254527 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5670025e50 con 0x7f566406fbb0 2024-09-19T08:38:29.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.383+0000 7f56849fb640 1 --2- 172.21.15.98:0/3290334630 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5664067990 0x7f5664069e50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f568013d300 tx=0x7f56780016a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.387+0000 7f249e7fc640 1 -- 172.21.15.98:0/2827117165 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f24b806bc90 con 0x7f24b81547e0 2024-09-19T08:38:29.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.387+0000 7f24b4ff9640 1 -- 172.21.15.98:0/2827117165 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f24a80c1030 con 0x7f24b81547e0 2024-09-19T08:38:29.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.387+0000 7f24b4ff9640 1 --2- 172.21.15.98:0/2827117165 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f24a4067940 0x7f24a4069e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.387+0000 7f24b7fff640 1 --2- 172.21.15.98:0/2827117165 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f24a4067940 0x7f24a4069e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.387+0000 7f24b4ff9640 1 -- 172.21.15.98:0/2827117165 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f24a80f1400 con 0x7f24b81547e0 2024-09-19T08:38:29.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.387+0000 7f24b4ff9640 1 --2- 172.21.15.98:0/2827117165 >> [v2:172.21.15.139:6824/1412125138,v1:172.21.15.139:6825/1412125138] conn(0x7f24a406fbc0 0x7f24a4071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.387+0000 7f24b4ff9640 1 -- 172.21.15.98:0/2827117165 --> [v2:172.21.15.139:6824/1412125138,v1:172.21.15.139:6825/1412125138] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f24a8004480 con 0x7f24a406fbc0 2024-09-19T08:38:29.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.387+0000 7f24b4ff9640 1 -- 172.21.15.98:0/2827117165 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f24a8128060 con 0x7f24b81547e0 2024-09-19T08:38:29.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.387+0000 7f24bcbfb640 1 --2- 172.21.15.98:0/2827117165 >> [v2:172.21.15.139:6824/1412125138,v1:172.21.15.139:6825/1412125138] conn(0x7f24a406fbc0 0x7f24a4071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.387+0000 7f24b7fff640 1 --2- 172.21.15.98:0/2827117165 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f24a4067940 0x7f24a4069e00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f24ac004470 tx=0x7f24ac003700 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.387+0000 7f24bcbfb640 1 --2- 172.21.15.98:0/2827117165 >> [v2:172.21.15.139:6824/1412125138,v1:172.21.15.139:6825/1412125138] conn(0x7f24a406fbc0 0x7f24a4071fe0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.391+0000 7f24b4ff9640 1 -- 172.21.15.98:0/2827117165 <== osd.4 v2:172.21.15.139:6824/1412125138 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f24a8004480 con 0x7f24a406fbc0 2024-09-19T08:38:29.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f8149aaf640 1 -- 172.21.15.98:0/3174940633 --> [v2:172.21.15.98:6800/3643795173,v1:172.21.15.98:6801/3643795173] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8124002de0 con 0x7f813006fb30 2024-09-19T08:38:29.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f633ffff640 1 -- 172.21.15.98:0/3631394867 --> [v2:172.21.15.98:6817/1538482309,v1:172.21.15.98:6820/1538482309] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6320002d60 con 0x7f632c06fae0 2024-09-19T08:38:29.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f812bfff640 1 -- 172.21.15.98:0/3174940633 <== osd.3 v2:172.21.15.98:6800/3643795173 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8124002de0 con 0x7f813006fb30 2024-09-19T08:38:29.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f0eddbf9640 1 -- 172.21.15.98:0/3657062065 --> [v2:172.21.15.139:6808/520689888,v1:172.21.15.139:6809/520689888] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0eb8002de0 con 0x7f0ebc06fa60 2024-09-19T08:38:29.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f63277fe640 1 -- 172.21.15.98:0/3631394867 <== osd.1 v2:172.21.15.98:6817/1538482309 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6320002d60 con 0x7f632c06fae0 2024-09-19T08:38:29.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f8149aaf640 1 -- 172.21.15.98:0/3174940633 >> [v2:172.21.15.98:6800/3643795173,v1:172.21.15.98:6801/3643795173] conn(0x7f813006fb30 msgr2=0x7f8130071f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f8149aaf640 1 --2- 172.21.15.98:0/3174940633 >> [v2:172.21.15.98:6800/3643795173,v1:172.21.15.98:6801/3643795173] conn(0x7f813006fb30 0x7f8130071f50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f8149aaf640 1 -- 172.21.15.98:0/3174940633 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8130067910 msgr2=0x7f8130069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f8149aaf640 1 --2- 172.21.15.98:0/3174940633 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8130067910 0x7f8130069dd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f81441fecb0 tx=0x7f813800d040 comp rx=0 tx=0).stop 2024-09-19T08:38:29.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f0ec3fff640 1 -- 172.21.15.98:0/3657062065 <== osd.5 v2:172.21.15.139:6808/520689888 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0eb8002de0 con 0x7f0ebc06fa60 2024-09-19T08:38:29.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f8149aaf640 1 -- 172.21.15.98:0/3174940633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81440ce4a0 msgr2=0x7f8144153980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f8149aaf640 1 --2- 172.21.15.98:0/3174940633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81440ce4a0 0x7f8144153980 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f813400c500 tx=0x7f813400c9c0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f8143fff640 1 -- 172.21.15.98:0/3174940633 reap_dead start 2024-09-19T08:38:29.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f8149aaf640 1 -- 172.21.15.98:0/3174940633 shutdown_connections 2024-09-19T08:38:29.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f8149aaf640 1 -- 172.21.15.98:0/3174940633 >> 172.21.15.98:0/3174940633 conn(0x7f814406cda0 msgr2=0x7f81440ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:29.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f8149aaf640 1 -- 172.21.15.98:0/3174940633 shutdown_connections 2024-09-19T08:38:29.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f8149aaf640 1 -- 172.21.15.98:0/3174940633 wait complete. 2024-09-19T08:38:29.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f0eddbf9640 1 -- 172.21.15.98:0/3657062065 >> [v2:172.21.15.139:6808/520689888,v1:172.21.15.139:6809/520689888] conn(0x7f0ebc06fa60 msgr2=0x7f0ebc071e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f0eddbf9640 1 --2- 172.21.15.98:0/3657062065 >> [v2:172.21.15.139:6808/520689888,v1:172.21.15.139:6809/520689888] conn(0x7f0ebc06fa60 0x7f0ebc071e80 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f633ffff640 1 -- 172.21.15.98:0/3631394867 >> [v2:172.21.15.98:6817/1538482309,v1:172.21.15.98:6820/1538482309] conn(0x7f632c06fae0 msgr2=0x7f632c071f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f633ffff640 1 --2- 172.21.15.98:0/3631394867 >> [v2:172.21.15.98:6817/1538482309,v1:172.21.15.98:6820/1538482309] conn(0x7f632c06fae0 0x7f632c071f00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f0eddbf9640 1 -- 172.21.15.98:0/3657062065 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0ebc0678c0 msgr2=0x7f0ebc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f0eddbf9640 1 --2- 172.21.15.98:0/3657062065 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0ebc0678c0 0x7f0ebc069d80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4006fd0 tx=0x7f0ec4008040 comp rx=0 tx=0).stop 2024-09-19T08:38:29.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f633ffff640 1 -- 172.21.15.98:0/3631394867 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f632c0678c0 msgr2=0x7f632c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f633ffff640 1 --2- 172.21.15.98:0/3631394867 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f632c0678c0 0x7f632c069d80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f63340b1000 tx=0x7f6334008040 comp rx=0 tx=0).stop 2024-09-19T08:38:29.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f633ffff640 1 -- 172.21.15.98:0/3631394867 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63400cf210 msgr2=0x7f634015db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f633ffff640 1 --2- 172.21.15.98:0/3631394867 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63400cf210 0x7f634015db90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f633000c500 tx=0x7f633000c9c0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f0eddbf9640 1 -- 172.21.15.98:0/3657062065 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ed00e4f00 msgr2=0x7f0ed00395f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f0eddbf9640 1 --2- 172.21.15.98:0/3657062065 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ed00e4f00 0x7f0ed00395f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0ed806b250 tx=0x7f0ed806b3e0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f0ed7fff640 1 -- 172.21.15.98:0/3657062065 reap_dead start 2024-09-19T08:38:29.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f633f7fe640 1 -- 172.21.15.98:0/3631394867 reap_dead start 2024-09-19T08:38:29.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f633ffff640 1 -- 172.21.15.98:0/3631394867 shutdown_connections 2024-09-19T08:38:29.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f633ffff640 1 -- 172.21.15.98:0/3631394867 >> 172.21.15.98:0/3631394867 conn(0x7f6340075070 msgr2=0x7f6340155720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:29.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f633ffff640 1 -- 172.21.15.98:0/3631394867 shutdown_connections 2024-09-19T08:38:29.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f633ffff640 1 -- 172.21.15.98:0/3631394867 wait complete. 2024-09-19T08:38:29.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f0eddbf9640 1 -- 172.21.15.98:0/3657062065 shutdown_connections 2024-09-19T08:38:29.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.399+0000 7f0eddbf9640 1 -- 172.21.15.98:0/3657062065 >> 172.21.15.98:0/3657062065 conn(0x7f0ed00567c0 msgr2=0x7f0ed0052620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:29.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.403+0000 7f0eddbf9640 1 -- 172.21.15.98:0/3657062065 shutdown_connections 2024-09-19T08:38:29.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.403+0000 7f4cff917640 1 -- 172.21.15.98:0/1768905152 --> [v2:172.21.15.139:6816/688014474,v1:172.21.15.139:6817/688014474] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4cf0180540 con 0x7f4cdc06fb90 2024-09-19T08:38:29.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.403+0000 7f0eddbf9640 1 -- 172.21.15.98:0/3657062065 wait complete. 2024-09-19T08:38:29.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.403+0000 7f5e22e46640 1 -- 172.21.15.98:0/2924660291 --> [v2:172.21.15.98:6808/2466005517,v1:172.21.15.98:6809/2466005517] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5e1c070a20 con 0x7f5e0006fb90 2024-09-19T08:38:29.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.403+0000 7f4cedffb640 1 -- 172.21.15.98:0/1768905152 <== osd.7 v2:172.21.15.139:6816/688014474 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4cf0180540 con 0x7f4cdc06fb90 2024-09-19T08:38:29.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.403+0000 7f5e197fa640 1 -- 172.21.15.98:0/2924660291 <== osd.2 v2:172.21.15.98:6808/2466005517 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5e1c070a20 con 0x7f5e0006fb90 2024-09-19T08:38:29.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.403+0000 7f5e22e46640 1 -- 172.21.15.98:0/2924660291 >> [v2:172.21.15.98:6808/2466005517,v1:172.21.15.98:6809/2466005517] conn(0x7f5e0006fb90 msgr2=0x7f5e00071fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.403+0000 7f4cff917640 1 -- 172.21.15.98:0/1768905152 >> [v2:172.21.15.139:6816/688014474,v1:172.21.15.139:6817/688014474] conn(0x7f4cdc06fb90 msgr2=0x7f4cdc071fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.403+0000 7f4cff917640 1 --2- 172.21.15.98:0/1768905152 >> [v2:172.21.15.139:6816/688014474,v1:172.21.15.139:6817/688014474] conn(0x7f4cdc06fb90 0x7f4cdc071fb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.403+0000 7f5e22e46640 1 --2- 172.21.15.98:0/2924660291 >> [v2:172.21.15.98:6808/2466005517,v1:172.21.15.98:6809/2466005517] conn(0x7f5e0006fb90 0x7f5e00071fb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.403+0000 7f4cff917640 1 -- 172.21.15.98:0/1768905152 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4cdc067910 msgr2=0x7f4cdc069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.403+0000 7f4cff917640 1 --2- 172.21.15.98:0/1768905152 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4cdc067910 0x7f4cdc069dd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4cf00391b0 tx=0x7f4ce8009040 comp rx=0 tx=0).stop 2024-09-19T08:38:29.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.403+0000 7f5e22e46640 1 -- 172.21.15.98:0/2924660291 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5e00067990 msgr2=0x7f5e00069e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.403+0000 7f4cff917640 1 -- 172.21.15.98:0/1768905152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4cf000a380 msgr2=0x7f4cf017cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.403+0000 7f4cff917640 1 --2- 172.21.15.98:0/1768905152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4cf000a380 0x7f4cf017cf50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4cf400d5d0 tx=0x7f4cf400da90 comp rx=0 tx=0).stop 2024-09-19T08:38:29.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.403+0000 7f4cfdeb4640 1 -- 172.21.15.98:0/1768905152 reap_dead start 2024-09-19T08:38:29.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.403+0000 7f5e22e46640 1 --2- 172.21.15.98:0/2924660291 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5e00067990 0x7f5e00069e50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5e0800ab60 tx=0x7f5e08008040 comp rx=0 tx=0).stop 2024-09-19T08:38:29.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.403+0000 7f4cff917640 1 -- 172.21.15.98:0/1768905152 shutdown_connections 2024-09-19T08:38:29.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.403+0000 7f4cff917640 1 -- 172.21.15.98:0/1768905152 >> 172.21.15.98:0/1768905152 conn(0x7f4cf00360a0 msgr2=0x7f4cf0033f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:29.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.403+0000 7f4cff917640 1 -- 172.21.15.98:0/1768905152 shutdown_connections 2024-09-19T08:38:29.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.403+0000 7f4cff917640 1 -- 172.21.15.98:0/1768905152 wait complete. 2024-09-19T08:38:29.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.403+0000 7f5e22e46640 1 -- 172.21.15.98:0/2924660291 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e1c15f910 msgr2=0x7f5e1c06e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.403+0000 7f4adbfff640 1 -- 172.21.15.98:0/759147497 --> [v2:172.21.15.139:6800/2917689006,v1:172.21.15.139:6801/2917689006] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4adc06c180 con 0x7f4ac806fab0 2024-09-19T08:38:29.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.403+0000 7f5e22e46640 1 --2- 172.21.15.98:0/2924660291 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5e1c15f910 0x7f5e1c06e8b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5e1001ab60 tx=0x7f5e10019250 comp rx=0 tx=0).stop 2024-09-19T08:38:29.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.403+0000 7f5e213e3640 1 -- 172.21.15.98:0/2924660291 reap_dead start 2024-09-19T08:38:29.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.407+0000 7f5e22e46640 1 -- 172.21.15.98:0/2924660291 shutdown_connections 2024-09-19T08:38:29.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.407+0000 7f4ac3fff640 1 -- 172.21.15.98:0/759147497 <== osd.6 v2:172.21.15.139:6800/2917689006 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4adc06c180 con 0x7f4ac806fab0 2024-09-19T08:38:29.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.407+0000 7f5e22e46640 1 -- 172.21.15.98:0/2924660291 >> 172.21.15.98:0/2924660291 conn(0x7f5e1c0b5160 msgr2=0x7f5e1c0d1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:29.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.407+0000 7f4adbfff640 1 -- 172.21.15.98:0/759147497 >> [v2:172.21.15.139:6800/2917689006,v1:172.21.15.139:6801/2917689006] conn(0x7f4ac806fab0 msgr2=0x7f4ac8071ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.407+0000 7f4adbfff640 1 --2- 172.21.15.98:0/759147497 >> [v2:172.21.15.139:6800/2917689006,v1:172.21.15.139:6801/2917689006] conn(0x7f4ac806fab0 0x7f4ac8071ed0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.407+0000 7f5e22e46640 1 -- 172.21.15.98:0/2924660291 shutdown_connections 2024-09-19T08:38:29.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.407+0000 7f5e22e46640 1 -- 172.21.15.98:0/2924660291 wait complete. 2024-09-19T08:38:29.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.411+0000 7f4adbfff640 1 -- 172.21.15.98:0/759147497 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4ac8067910 msgr2=0x7f4ac8069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.411+0000 7f4adbfff640 1 --2- 172.21.15.98:0/759147497 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4ac8067910 0x7f4ac8069dd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4acc007fd0 tx=0x7f4acc009040 comp rx=0 tx=0).stop 2024-09-19T08:38:29.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.411+0000 7f4adbfff640 1 -- 172.21.15.98:0/759147497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4adc0cdff0 msgr2=0x7f4adc1571d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.411+0000 7f4adbfff640 1 --2- 172.21.15.98:0/759147497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4adc0cdff0 0x7f4adc1571d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4ad000c7b0 tx=0x7f4ad000cc70 comp rx=0 tx=0).stop 2024-09-19T08:38:29.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.411+0000 7f4adb7fe640 1 -- 172.21.15.98:0/759147497 reap_dead start 2024-09-19T08:38:29.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.411+0000 7f4adbfff640 1 -- 172.21.15.98:0/759147497 shutdown_connections 2024-09-19T08:38:29.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.411+0000 7f4adbfff640 1 -- 172.21.15.98:0/759147497 >> 172.21.15.98:0/759147497 conn(0x7f4adc075070 msgr2=0x7f4adc06d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:29.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.411+0000 7f4adbfff640 1 -- 172.21.15.98:0/759147497 shutdown_connections 2024-09-19T08:38:29.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.411+0000 7f4adbfff640 1 -- 172.21.15.98:0/759147497 wait complete. 2024-09-19T08:38:29.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.415+0000 7f566b7fe640 1 -- 172.21.15.98:0/3290334630 --> [v2:172.21.15.98:6816/2191254527,v1:172.21.15.98:6818/2191254527] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5660002de0 con 0x7f566406fbb0 2024-09-19T08:38:29.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.415+0000 7f5675ffb640 1 -- 172.21.15.98:0/3290334630 <== osd.0 v2:172.21.15.98:6816/2191254527 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5660002de0 con 0x7f566406fbb0 2024-09-19T08:38:29.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.415+0000 7f566b7fe640 1 -- 172.21.15.98:0/3290334630 >> [v2:172.21.15.98:6816/2191254527,v1:172.21.15.98:6818/2191254527] conn(0x7f566406fbb0 msgr2=0x7f5664071fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.415+0000 7f566b7fe640 1 --2- 172.21.15.98:0/3290334630 >> [v2:172.21.15.98:6816/2191254527,v1:172.21.15.98:6818/2191254527] conn(0x7f566406fbb0 0x7f5664071fd0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.419+0000 7f249e7fc640 1 -- 172.21.15.98:0/2827117165 --> [v2:172.21.15.139:6824/1412125138,v1:172.21.15.139:6825/1412125138] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f24b80b2200 con 0x7f24a406fbc0 2024-09-19T08:38:29.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.419+0000 7f566b7fe640 1 -- 172.21.15.98:0/3290334630 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5664067990 msgr2=0x7f5664069e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.419+0000 7f566b7fe640 1 --2- 172.21.15.98:0/3290334630 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5664067990 0x7f5664069e50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f568013d300 tx=0x7f56780016a0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.419+0000 7f566b7fe640 1 -- 172.21.15.98:0/3290334630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5680073740 msgr2=0x7f56800af060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.419+0000 7f566b7fe640 1 --2- 172.21.15.98:0/3290334630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5680073740 0x7f56800af060 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f567000c480 tx=0x7f567000c940 comp rx=0 tx=0).stop 2024-09-19T08:38:29.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.419+0000 7f56859fd640 1 -- 172.21.15.98:0/3290334630 reap_dead start 2024-09-19T08:38:29.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.419+0000 7f566b7fe640 1 -- 172.21.15.98:0/3290334630 shutdown_connections 2024-09-19T08:38:29.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.419+0000 7f566b7fe640 1 -- 172.21.15.98:0/3290334630 >> 172.21.15.98:0/3290334630 conn(0x7f568006d080 msgr2=0x7f5680150460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:29.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.419+0000 7f566b7fe640 1 -- 172.21.15.98:0/3290334630 shutdown_connections 2024-09-19T08:38:29.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.419+0000 7f566b7fe640 1 -- 172.21.15.98:0/3290334630 wait complete. 2024-09-19T08:38:29.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.419+0000 7f24b4ff9640 1 -- 172.21.15.98:0/2827117165 <== osd.4 v2:172.21.15.139:6824/1412125138 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f24b80b2200 con 0x7f24a406fbc0 2024-09-19T08:38:29.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.419+0000 7f249e7fc640 1 -- 172.21.15.98:0/2827117165 >> [v2:172.21.15.139:6824/1412125138,v1:172.21.15.139:6825/1412125138] conn(0x7f24a406fbc0 msgr2=0x7f24a4071fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.419+0000 7f249e7fc640 1 --2- 172.21.15.98:0/2827117165 >> [v2:172.21.15.139:6824/1412125138,v1:172.21.15.139:6825/1412125138] conn(0x7f24a406fbc0 0x7f24a4071fe0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.419+0000 7f249e7fc640 1 -- 172.21.15.98:0/2827117165 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f24a4067940 msgr2=0x7f24a4069e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.419+0000 7f249e7fc640 1 --2- 172.21.15.98:0/2827117165 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f24a4067940 0x7f24a4069e00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f24ac004470 tx=0x7f24ac003700 comp rx=0 tx=0).stop 2024-09-19T08:38:29.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.419+0000 7f249e7fc640 1 -- 172.21.15.98:0/2827117165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24b81547e0 msgr2=0x7f24b81440e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.419+0000 7f249e7fc640 1 --2- 172.21.15.98:0/2827117165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24b81547e0 0x7f24b81440e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f24a80af000 tx=0x7f24a801b750 comp rx=0 tx=0).stop 2024-09-19T08:38:29.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.419+0000 7f24bcbfb640 1 -- 172.21.15.98:0/2827117165 reap_dead start 2024-09-19T08:38:29.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.431+0000 7f249e7fc640 1 -- 172.21.15.98:0/2827117165 shutdown_connections 2024-09-19T08:38:29.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.431+0000 7f249e7fc640 1 -- 172.21.15.98:0/2827117165 >> 172.21.15.98:0/2827117165 conn(0x7f24b8075070 msgr2=0x7f24b8072400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:29.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.431+0000 7f249e7fc640 1 -- 172.21.15.98:0/2827117165 shutdown_connections 2024-09-19T08:38:29.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.431+0000 7f249e7fc640 1 -- 172.21.15.98:0/2827117165 wait complete. 2024-09-19T08:38:29.468 INFO:teuthology.orchestra.run.smithi098.stdout:64424509448 2024-09-19T08:38:29.468 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-19T08:38:29.469 INFO:teuthology.orchestra.run.smithi098.stdout:64424509449 2024-09-19T08:38:29.469 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-19T08:38:29.470 INFO:teuthology.orchestra.run.smithi098.stdout:64424509447 2024-09-19T08:38:29.470 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-19T08:38:29.471 INFO:teuthology.orchestra.run.smithi098.stdout:60129542151 2024-09-19T08:38:29.471 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-19T08:38:29.472 INFO:teuthology.orchestra.run.smithi098.stdout:64424509446 2024-09-19T08:38:29.473 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-19T08:38:29.473 INFO:teuthology.orchestra.run.smithi098.stdout:55834574855 2024-09-19T08:38:29.474 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-19T08:38:29.474 INFO:teuthology.orchestra.run.smithi098.stdout:64424509446 2024-09-19T08:38:29.475 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-19T08:38:29.475 INFO:teuthology.orchestra.run.smithi098.stdout:64424509449 2024-09-19T08:38:29.476 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-19T08:38:29.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.683+0000 7fab075b7640 1 Processor -- start 2024-09-19T08:38:29.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.683+0000 7fab075b7640 1 -- start start 2024-09-19T08:38:29.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.683+0000 7f3b67fff640 1 Processor -- start 2024-09-19T08:38:29.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f2679c1a640 1 Processor -- start 2024-09-19T08:38:29.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.683+0000 7f3b67fff640 1 -- start start 2024-09-19T08:38:29.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f54985b8640 1 Processor -- start 2024-09-19T08:38:29.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f369714f640 1 Processor -- start 2024-09-19T08:38:29.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f54985b8640 1 -- start start 2024-09-19T08:38:29.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7fab075b7640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fab0014c380 0x7fab000ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7fab075b7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fab000b5320 0x7fab000b5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7fab075b7640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab0014c760 con 0x7fab000b4e30 2024-09-19T08:38:29.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7fab075b7640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fab000b5c90 con 0x7fab0014c380 2024-09-19T08:38:29.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7fab075b7640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fab000cd4f0 con 0x7fab000b5320 2024-09-19T08:38:29.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f369714f640 1 -- start start 2024-09-19T08:38:29.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7fb4e3fb5640 1 Processor -- start 2024-09-19T08:38:29.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f3b67fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3b681627a0 0x7f3b680cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f3b67fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3b6806c3a0 0x7f3b68071770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f3b67fff640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b680cda30 con 0x7f3b680cf880 2024-09-19T08:38:29.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f3b67fff640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f3b680712e0 con 0x7f3b6806c3a0 2024-09-19T08:38:29.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f3b67fff640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f3b680b3c20 con 0x7f3b681627a0 2024-09-19T08:38:29.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7fb4e3fb5640 1 -- start start 2024-09-19T08:38:29.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f2679c1a640 1 -- start start 2024-09-19T08:38:29.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f18447b8640 1 Processor -- start 2024-09-19T08:38:29.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f2679c1a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f26740cdf80 0x7f26740cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f2cc7a7b640 1 Processor -- start 2024-09-19T08:38:29.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f18447b8640 1 -- start start 2024-09-19T08:38:29.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f2cc7a7b640 1 -- start start 2024-09-19T08:38:29.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f3b677fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3b6806c3a0 0x7f3b68071770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f2679c1a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f26741638f0 0x7f26740c5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f2679c1a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26740d00c0 con 0x7f26740c56c0 2024-09-19T08:38:29.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f2679c1a640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f26740c5b10 con 0x7f26740cdf80 2024-09-19T08:38:29.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f2679c1a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f26740d03a0 con 0x7f26741638f0 2024-09-19T08:38:29.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f54985b8640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5490145640 0x7f549015c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7fab05353640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fab0014c380 0x7fab000ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7fab05353640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fab0014c380 0x7fab000ccbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37846/0 (socket says 172.21.15.98:37846) 2024-09-19T08:38:29.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f54985b8640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5490073d70 0x7f54900cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7fab05353640 1 -- 172.21.15.98:0/248372406 learned_addr learned my addr 172.21.15.98:0/248372406 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:29.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7fab04b52640 1 --1- 172.21.15.98:0/248372406 >> v1:172.21.15.139:6789/0 conn(0x7fab000b4e30 0x7fab000ce590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:44966/0 (socket says 172.21.15.98:44966) 2024-09-19T08:38:29.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7fab05b54640 1 --2- 172.21.15.98:0/248372406 >> v2:172.21.15.139:3300/0 conn(0x7fab000b5320 0x7fab000b5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f369714f640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f369015b170 0x7f36900b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f369714f640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f369015c480 con 0x7f369006c830 2024-09-19T08:38:29.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f369714f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36900b1830 con 0x7f369006bc90 2024-09-19T08:38:29.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f369714f640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f36900b1e80 con 0x7f369015b170 2024-09-19T08:38:29.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f3b677fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3b6806c3a0 0x7f3b68071770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37852/0 (socket says 172.21.15.98:37852) 2024-09-19T08:38:29.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f3b677fe640 1 -- 172.21.15.98:0/3598049632 learned_addr learned my addr 172.21.15.98:0/3598049632 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:29.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f3b66ffd640 1 --2- 172.21.15.98:0/3598049632 >> v2:172.21.15.139:3300/0 conn(0x7f3b681627a0 0x7f3b680cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f368ffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f369015b170 0x7f36900b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f368ffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f369015b170 0x7f36900b4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37874/0 (socket says 172.21.15.98:37874) 2024-09-19T08:38:29.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f368ffff640 1 -- 172.21.15.98:0/2018521182 learned_addr learned my addr 172.21.15.98:0/2018521182 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:29.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f54985b8640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f549015cda0 con 0x7f5490070d40 2024-09-19T08:38:29.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f54985b8640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f54900cff40 con 0x7f5490145640 2024-09-19T08:38:29.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f54985b8640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f54900cf6a0 con 0x7f5490073d70 2024-09-19T08:38:29.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f5496354640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5490145640 0x7f549015c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7fb4e3fb5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb4d40475b0 0x7fb4d400a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7fb4e3fb5640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4d401ded0 con 0x7fb4d404eba0 2024-09-19T08:38:29.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7fb4e3fb5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4d400baf0 con 0x7fb4d40338f0 2024-09-19T08:38:29.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7fb4e3fb5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fb4d401d110 con 0x7fb4d40475b0 2024-09-19T08:38:29.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f5496354640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5490145640 0x7f549015c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37888/0 (socket says 172.21.15.98:37888) 2024-09-19T08:38:29.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f5496354640 1 -- 172.21.15.98:0/1261220039 learned_addr learned my addr 172.21.15.98:0/1261220039 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:29.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f5495b53640 1 --2- 172.21.15.98:0/1261220039 >> v2:172.21.15.139:3300/0 conn(0x7f5490073d70 0x7f54900cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f2673fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f26740c56c0 0x7f26740b6f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:44976/0 (socket says 172.21.15.98:44976) 2024-09-19T08:38:29.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f2673fff640 1 -- 172.21.15.98:0/393073247 learned_addr learned my addr 172.21.15.98:0/393073247 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:29.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f26737fe640 1 --2- 172.21.15.98:0/393073247 >> v2:172.21.15.98:3300/0 conn(0x7f26740cdf80 0x7f26740cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f2672ffd640 1 --2- 172.21.15.98:0/393073247 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f26741638f0 0x7f26740c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f18447b8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f183c15e110 0x7f183c15e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f18447b8640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f183c15ea30 0x7f183c159d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f18447b8640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f183c15a720 con 0x7f183c15a2d0 2024-09-19T08:38:29.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f18447b8640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f183c073e10 con 0x7f183c15ea30 2024-09-19T08:38:29.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f18447b8640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f183c0b4800 con 0x7f183c15e110 2024-09-19T08:38:29.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f2cc7a7b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2cb80e0a00 0x7f2cb801d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f2cc7a7b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2cb8006620 0x7f2cb8006a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f2cc7a7b640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cb801c910 con 0x7f2cb80e4380 2024-09-19T08:38:29.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f2cc7a7b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f2cb801def0 con 0x7f2cb8006620 2024-09-19T08:38:29.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f2cc7a7b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2cb801d520 con 0x7f2cb80e0a00 2024-09-19T08:38:29.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7fb4e2552640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb4d40475b0 0x7fb4d400a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7fb4e2552640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb4d40475b0 0x7fb4d400a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:49950/0 (socket says 172.21.15.98:49950) 2024-09-19T08:38:29.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7fb4e2552640 1 -- 172.21.15.98:0/2882164790 learned_addr learned my addr 172.21.15.98:0/2882164790 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:29.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7fab05353640 1 -- 172.21.15.98:0/248372406 >> v2:172.21.15.139:3300/0 conn(0x7fab000b5320 msgr2=0x7fab000b5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:29.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7fab05353640 1 --2- 172.21.15.98:0/248372406 >> v2:172.21.15.139:3300/0 conn(0x7fab000b5320 0x7fab000b5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7fab05353640 1 -- 172.21.15.98:0/248372406 >> v1:172.21.15.139:6789/0 conn(0x7fab000b4e30 legacy=0x7fab000ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7fab05353640 1 -- 172.21.15.98:0/248372406 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab000cd7d0 con 0x7fab0014c380 2024-09-19T08:38:29.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f3b66ffd640 1 -- 172.21.15.98:0/3598049632 >> v2:172.21.15.98:3300/0 conn(0x7f3b6806c3a0 msgr2=0x7f3b68071770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:29.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f3b66ffd640 1 --2- 172.21.15.98:0/3598049632 >> v2:172.21.15.98:3300/0 conn(0x7f3b6806c3a0 0x7f3b68071770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f3b66ffd640 1 -- 172.21.15.98:0/3598049632 >> v1:172.21.15.98:6789/0 conn(0x7f3b680cf880 legacy=0x7f3b6806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f368f7fe640 1 -- 172.21.15.98:0/2018521182 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2403781296 0 0) 0x7f369015c480 con 0x7f369006c830 2024-09-19T08:38:29.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f368f7fe640 1 -- 172.21.15.98:0/2018521182 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3678003530 con 0x7f369006c830 2024-09-19T08:38:29.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f3b66ffd640 1 -- 172.21.15.98:0/3598049632 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b68070cd0 con 0x7f3b681627a0 2024-09-19T08:38:29.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f1841d53640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f183c15ea30 0x7f183c159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f1841d53640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f183c15ea30 0x7f183c159d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:49958/0 (socket says 172.21.15.98:49958) 2024-09-19T08:38:29.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f1841d53640 1 -- 172.21.15.98:0/328393824 learned_addr learned my addr 172.21.15.98:0/328393824 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:29.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f368f7fe640 1 -- 172.21.15.98:0/2018521182 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3541233323 0 0) 0x7f36900b1830 con 0x7f369006bc90 2024-09-19T08:38:29.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f368f7fe640 1 -- 172.21.15.98:0/2018521182 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f369015c480 con 0x7f369006bc90 2024-09-19T08:38:29.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f368f7fe640 1 -- 172.21.15.98:0/2018521182 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4218651055 0 0) 0x7f3678003530 con 0x7f369006c830 2024-09-19T08:38:29.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f368f7fe640 1 -- 172.21.15.98:0/2018521182 >> v2:172.21.15.98:3300/0 conn(0x7f369015b170 msgr2=0x7f36900b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:29.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f368f7fe640 1 --2- 172.21.15.98:0/2018521182 >> v2:172.21.15.98:3300/0 conn(0x7f369015b170 0x7f36900b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f368f7fe640 1 -- 172.21.15.98:0/2018521182 >> v1:172.21.15.139:6789/0 conn(0x7f369006bc90 legacy=0x7f36900cd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f368f7fe640 1 -- 172.21.15.98:0/2018521182 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3690161c20 con 0x7f369006c830 2024-09-19T08:38:29.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f368f7fe640 1 -- 172.21.15.98:0/2018521182 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f3684001680 con 0x7f369006c830 2024-09-19T08:38:29.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f368f7fe640 1 -- 172.21.15.98:0/2018521182 >> v1:172.21.15.98:6789/0 conn(0x7f369006c830 legacy=0x7f36900b2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f368f7fe640 1 --2- 172.21.15.98:0/2018521182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3678003b50 0x7f3678003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f368f7fe640 1 --2- 172.21.15.98:0/2018521182 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3678004810 0x7f3678004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f368f7fe640 1 --2- 172.21.15.98:0/2018521182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36780051d0 0x7f36780141e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f368f7fe640 1 -- 172.21.15.98:0/2018521182 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f36900b1e80 con 0x7f36780051d0 2024-09-19T08:38:29.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f368f7fe640 1 -- 172.21.15.98:0/2018521182 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3678014910 con 0x7f3678003b50 2024-09-19T08:38:29.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f368f7fe640 1 -- 172.21.15.98:0/2018521182 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3678014b10 con 0x7f3678004810 2024-09-19T08:38:29.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f2cc6018640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f2cb80e4380 0x7f2cb8031480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:60614/0 (socket says 172.21.15.98:60614) 2024-09-19T08:38:29.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f2cc6018640 1 -- 172.21.15.98:0/2862898130 learned_addr learned my addr 172.21.15.98:0/2862898130 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:29.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f368ffff640 1 --2- 172.21.15.98:0/2018521182 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3678004810 0x7f3678004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f36956ec640 1 --2- 172.21.15.98:0/2018521182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36780051d0 0x7f36780141e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7fb4e2552640 1 -- 172.21.15.98:0/2882164790 >> v1:172.21.15.139:6789/0 conn(0x7fb4d40338f0 legacy=0x7fb4d4009b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7fb4e2552640 1 -- 172.21.15.98:0/2882164790 >> v1:172.21.15.98:6789/0 conn(0x7fb4d404eba0 legacy=0x7fb4d404ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:29.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7fb4e2552640 1 -- 172.21.15.98:0/2882164790 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d401d3f0 con 0x7fb4d40475b0 2024-09-19T08:38:29.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f3694eeb640 1 --2- 172.21.15.98:0/2018521182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3678003b50 0x7f3678003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f1842554640 1 --2- 172.21.15.98:0/328393824 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f183c15e110 0x7f183c15e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f1841d53640 1 -- 172.21.15.98:0/328393824 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f183c15e110 msgr2=0x7f183c15e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:29.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f1841d53640 1 --2- 172.21.15.98:0/328393824 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f183c15e110 0x7f183c15e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f1841d53640 1 -- 172.21.15.98:0/328393824 >> v1:172.21.15.98:6789/0 conn(0x7f183c15a2d0 legacy=0x7f183c0cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f1841d53640 1 -- 172.21.15.98:0/328393824 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183c0b4ae0 con 0x7f183c15ea30 2024-09-19T08:38:29.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7fb4e2552640 1 --2- 172.21.15.98:0/2882164790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4d40475b0 0x7fb4d400a240 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8009870 tx=0x7fb4d801c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f95685b359715248 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f26737fe640 1 -- 172.21.15.98:0/393073247 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f26741638f0 msgr2=0x7f26740c5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:29.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f26737fe640 1 --2- 172.21.15.98:0/393073247 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f26741638f0 0x7f26740c5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f26737fe640 1 -- 172.21.15.98:0/393073247 >> v1:172.21.15.139:6789/0 conn(0x7f26740c56c0 legacy=0x7f26740b6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f26737fe640 1 -- 172.21.15.98:0/393073247 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26740ced00 con 0x7f26740cdf80 2024-09-19T08:38:29.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f26737fe640 1 --2- 172.21.15.98:0/393073247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26740cdf80 0x7f26740cf7e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f26680097c0 tx=0x7f266801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85caf8e58a0b346 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7fb4e0d4f640 1 -- 172.21.15.98:0/2882164790 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d80ab610 con 0x7fb4d40475b0 2024-09-19T08:38:29.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f5495b53640 1 -- 172.21.15.98:0/1261220039 >> v2:172.21.15.98:3300/0 conn(0x7f5490145640 msgr2=0x7f549015c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:29.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f5495b53640 1 --2- 172.21.15.98:0/1261220039 >> v2:172.21.15.98:3300/0 conn(0x7f5490145640 0x7f549015c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f5495b53640 1 -- 172.21.15.98:0/1261220039 >> v1:172.21.15.139:6789/0 conn(0x7f5490070d40 legacy=0x7f5490071150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:29.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.687+0000 7f5495b53640 1 -- 172.21.15.98:0/1261220039 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54900cf980 con 0x7f5490073d70 2024-09-19T08:38:29.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7fb4e0d4f640 1 -- 172.21.15.98:0/2882164790 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4d80b0070 con 0x7fb4d40475b0 2024-09-19T08:38:29.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7fb4e0d4f640 1 -- 172.21.15.98:0/2882164790 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d80ab9d0 con 0x7fb4d40475b0 2024-09-19T08:38:29.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f3b66ffd640 1 --2- 172.21.15.98:0/3598049632 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b681627a0 0x7f3b680cd4f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3b680cdc60 tx=0x7f3b5c01c370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5b3e22912bd226c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f26727fc640 1 -- 172.21.15.98:0/393073247 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26680ab660 con 0x7f26740cdf80 2024-09-19T08:38:29.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f26727fc640 1 -- 172.21.15.98:0/393073247 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26680b0070 con 0x7f26740cdf80 2024-09-19T08:38:29.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f26727fc640 1 -- 172.21.15.98:0/393073247 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26680aba20 con 0x7f26740cdf80 2024-09-19T08:38:29.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f3b65ffb640 1 -- 172.21.15.98:0/3598049632 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c0b0070 con 0x7f3b681627a0 2024-09-19T08:38:29.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f1841d53640 1 --2- 172.21.15.98:0/328393824 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f183c15ea30 0x7f183c159d90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f183800afd0 tx=0x7f18380208f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dc8ffaef6f2e2fa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f5495b53640 1 --2- 172.21.15.98:0/1261220039 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5490073d70 0x7f54900cee70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f549006f230 tx=0x7f548c01c280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b62a03d4282dc8e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7fb4e3fb5640 1 -- 172.21.15.98:0/2882164790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4d40475b0 msgr2=0x7fb4d400a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7fb4e3fb5640 1 --2- 172.21.15.98:0/2882164790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4d40475b0 0x7fb4d400a240 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8009870 tx=0x7fb4d801c540 comp rx=0 tx=0).stop 2024-09-19T08:38:29.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f1841552640 1 -- 172.21.15.98:0/328393824 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1838008be0 con 0x7f183c15ea30 2024-09-19T08:38:29.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7fb4e3fb5640 1 -- 172.21.15.98:0/2882164790 shutdown_connections 2024-09-19T08:38:29.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7fb4e3fb5640 1 --2- 172.21.15.98:0/2882164790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4d40475b0 0x7fb4d400a240 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7fb4e3fb5640 1 -- 172.21.15.98:0/2882164790 >> 172.21.15.98:0/2882164790 conn(0x7fb4d40360d0 msgr2=0x7fb4d40364e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:29.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f1841552640 1 -- 172.21.15.98:0/328393824 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1838008d80 con 0x7f183c15ea30 2024-09-19T08:38:29.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f3b65ffb640 1 -- 172.21.15.98:0/3598049632 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b5c0ab600 con 0x7f3b681627a0 2024-09-19T08:38:29.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f3b65ffb640 1 -- 172.21.15.98:0/3598049632 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c0ab9f0 con 0x7f3b681627a0 2024-09-19T08:38:29.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f2679c1a640 1 -- 172.21.15.98:0/393073247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26740cdf80 msgr2=0x7f26740cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f2679c1a640 1 --2- 172.21.15.98:0/393073247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26740cdf80 0x7f26740cf7e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f26680097c0 tx=0x7f266801c4a0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f2679c1a640 1 -- 172.21.15.98:0/393073247 shutdown_connections 2024-09-19T08:38:29.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f2679c1a640 1 --2- 172.21.15.98:0/393073247 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f26741638f0 0x7f26740c5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f2679c1a640 1 --2- 172.21.15.98:0/393073247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26740cdf80 0x7f26740cf7e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f2679c1a640 1 -- 172.21.15.98:0/393073247 >> 172.21.15.98:0/393073247 conn(0x7f26740b5160 msgr2=0x7f26740b5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:29.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f2cc4815640 1 -- 172.21.15.98:0/2862898130 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2129575213 0 0) 0x7f2cb801c910 con 0x7f2cb80e4380 2024-09-19T08:38:29.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f2cc4815640 1 -- 172.21.15.98:0/2862898130 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2cac003530 con 0x7f2cb80e4380 2024-09-19T08:38:29.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f2cc5016640 1 --2- 172.21.15.98:0/2862898130 >> v2:172.21.15.139:3300/0 conn(0x7f2cb8006620 0x7f2cb8006a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7fb4e3fb5640 1 -- 172.21.15.98:0/2882164790 shutdown_connections 2024-09-19T08:38:29.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f2cc5817640 1 --2- 172.21.15.98:0/2862898130 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2cb80e0a00 0x7f2cb801d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f2cc4815640 1 -- 172.21.15.98:0/2862898130 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 552668591 0 0) 0x7f2cac003530 con 0x7f2cb80e4380 2024-09-19T08:38:29.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f2cc4815640 1 -- 172.21.15.98:0/2862898130 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2cb80e0a00 msgr2=0x7f2cb801d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:29.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f2cc4815640 1 --2- 172.21.15.98:0/2862898130 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2cb80e0a00 0x7f2cb801d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f2cc4815640 1 -- 172.21.15.98:0/2862898130 >> v2:172.21.15.139:3300/0 conn(0x7f2cb8006620 msgr2=0x7f2cb8006a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:29.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f2cc4815640 1 --2- 172.21.15.98:0/2862898130 >> v2:172.21.15.139:3300/0 conn(0x7f2cb8006620 0x7f2cb8006a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f2cc4815640 1 -- 172.21.15.98:0/2862898130 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb801cbf0 con 0x7f2cb80e4380 2024-09-19T08:38:29.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f2cc4815640 1 -- 172.21.15.98:0/2862898130 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f2cc0050810 con 0x7f2cb80e4380 2024-09-19T08:38:29.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f2cc4815640 1 -- 172.21.15.98:0/2862898130 >> v1:172.21.15.98:6789/0 conn(0x7f2cb80e4380 legacy=0x7f2cb8031480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f2cc4815640 1 --2- 172.21.15.98:0/2862898130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cac0039c0 0x7f2cac003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f2cc4815640 1 --2- 172.21.15.98:0/2862898130 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2cb80e0a00 0x7f2cac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f2cc4815640 1 --2- 172.21.15.98:0/2862898130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cac004d50 0x7f2cac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f2cc4815640 1 -- 172.21.15.98:0/2862898130 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb801def0 con 0x7f2cac004d50 2024-09-19T08:38:29.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f2cc4815640 1 -- 172.21.15.98:0/2862898130 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb801d520 con 0x7f2cac0039c0 2024-09-19T08:38:29.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f2cc4815640 1 -- 172.21.15.98:0/2862898130 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2cac00b9d0 con 0x7f2cb80e0a00 2024-09-19T08:38:29.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f2cc5817640 1 --2- 172.21.15.98:0/2862898130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cac0039c0 0x7f2cac003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f2cc5016640 1 --2- 172.21.15.98:0/2862898130 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2cb80e0a00 0x7f2cac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f36956ec640 1 -- 172.21.15.98:0/2018521182 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3678004810 msgr2=0x7f3678004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:29.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f36956ec640 1 --2- 172.21.15.98:0/2018521182 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3678004810 0x7f3678004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f36956ec640 1 -- 172.21.15.98:0/2018521182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3678003b50 msgr2=0x7f3678003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:29.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f36956ec640 1 --2- 172.21.15.98:0/2018521182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3678003b50 0x7f3678003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.691+0000 7f36956ec640 1 -- 172.21.15.98:0/2018521182 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3690161c20 con 0x7f36780051d0 2024-09-19T08:38:29.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f36956ec640 1 -- 172.21.15.98:0/2018521182 reap_dead start 2024-09-19T08:38:29.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f3b67fff640 1 -- 172.21.15.98:0/3598049632 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b681627a0 msgr2=0x7f3b680cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f3b67fff640 1 --2- 172.21.15.98:0/3598049632 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b681627a0 0x7f3b680cd4f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3b680cdc60 tx=0x7f3b5c01c370 comp rx=0 tx=0).stop 2024-09-19T08:38:29.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f2679c1a640 1 -- 172.21.15.98:0/393073247 shutdown_connections 2024-09-19T08:38:29.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f36956ec640 1 --2- 172.21.15.98:0/2018521182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36780051d0 0x7f36780141e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3680007fd0 tx=0x7f368001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15f0d026925d3f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f1841552640 1 -- 172.21.15.98:0/328393824 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183800a2f0 con 0x7f183c15ea30 2024-09-19T08:38:29.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f18447b8640 1 -- 172.21.15.98:0/328393824 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f183c15ea30 msgr2=0x7f183c159d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f18447b8640 1 --2- 172.21.15.98:0/328393824 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f183c15ea30 0x7f183c159d90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f183800afd0 tx=0x7f18380208f0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f18447b8640 1 -- 172.21.15.98:0/328393824 shutdown_connections 2024-09-19T08:38:29.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f18447b8640 1 --2- 172.21.15.98:0/328393824 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f183c15ea30 0x7f183c159d90 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f18447b8640 1 --2- 172.21.15.98:0/328393824 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f183c15e110 0x7f183c15e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f18447b8640 1 -- 172.21.15.98:0/328393824 >> 172.21.15.98:0/328393824 conn(0x7f183c06cda0 msgr2=0x7f183c06d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:29.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f5495352640 1 -- 172.21.15.98:0/1261220039 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c0b0070 con 0x7f5490073d70 2024-09-19T08:38:29.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f5495352640 1 -- 172.21.15.98:0/1261220039 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f548c0ab600 con 0x7f5490073d70 2024-09-19T08:38:29.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f5495352640 1 -- 172.21.15.98:0/1261220039 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c0aba10 con 0x7f5490073d70 2024-09-19T08:38:29.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f2cc6018640 1 --2- 172.21.15.98:0/2862898130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cac004d50 0x7f2cac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7fab05353640 1 --2- 172.21.15.98:0/248372406 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab0014c380 0x7fab000ccbf0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fab000b64d0 tx=0x7faafc01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c25a9c5a0c3697dd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7fb4e3fb5640 1 -- 172.21.15.98:0/2882164790 wait complete. 2024-09-19T08:38:29.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7fb4e3fb5640 1 Processor -- start 2024-09-19T08:38:29.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f2679c1a640 1 -- 172.21.15.98:0/393073247 wait complete. 2024-09-19T08:38:29.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f54985b8640 1 -- 172.21.15.98:0/1261220039 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5490073d70 msgr2=0x7f54900cee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f54985b8640 1 --2- 172.21.15.98:0/1261220039 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5490073d70 0x7f54900cee70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f549006f230 tx=0x7f548c01c280 comp rx=0 tx=0).stop 2024-09-19T08:38:29.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f2679c1a640 1 Processor -- start 2024-09-19T08:38:29.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7faaf7fff640 1 -- 172.21.15.98:0/248372406 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc0aa650 con 0x7fab0014c380 2024-09-19T08:38:29.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7faaf7fff640 1 -- 172.21.15.98:0/248372406 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faafc0af070 con 0x7fab0014c380 2024-09-19T08:38:29.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7faaf7fff640 1 -- 172.21.15.98:0/248372406 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc0aaa10 con 0x7fab0014c380 2024-09-19T08:38:29.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f368f7fe640 1 -- 172.21.15.98:0/2018521182 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3680001500 con 0x7f36780051d0 2024-09-19T08:38:29.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f368f7fe640 1 -- 172.21.15.98:0/2018521182 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36800a7070 con 0x7f36780051d0 2024-09-19T08:38:29.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f368f7fe640 1 -- 172.21.15.98:0/2018521182 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36800054b0 con 0x7f36780051d0 2024-09-19T08:38:29.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f54985b8640 1 -- 172.21.15.98:0/1261220039 shutdown_connections 2024-09-19T08:38:29.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f54985b8640 1 --2- 172.21.15.98:0/1261220039 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5490073d70 0x7f54900cee70 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f54985b8640 1 --2- 172.21.15.98:0/1261220039 >> v2:172.21.15.98:3300/0 conn(0x7f5490145640 0x7f549015c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f54985b8640 1 -- 172.21.15.98:0/1261220039 >> 172.21.15.98:0/1261220039 conn(0x7f54900b1f10 msgr2=0x7f54900b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:29.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7fb4e3fb5640 1 -- start start 2024-09-19T08:38:29.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f54985b8640 1 -- 172.21.15.98:0/1261220039 shutdown_connections 2024-09-19T08:38:29.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f2679c1a640 1 -- start start 2024-09-19T08:38:29.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f2679c1a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26740c56c0 0x7f2674154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f2679c1a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26740cdf80 0x7f2674162df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f18447b8640 1 -- 172.21.15.98:0/328393824 shutdown_connections 2024-09-19T08:38:29.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f2679c1a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f26741638f0 0x7f2674158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f2679c1a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2674163d00 con 0x7f26740c56c0 2024-09-19T08:38:29.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f369714f640 1 -- 172.21.15.98:0/2018521182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36780051d0 msgr2=0x7f36780141e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f369714f640 1 --2- 172.21.15.98:0/2018521182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36780051d0 0x7f36780141e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3680007fd0 tx=0x7f368001c560 comp rx=0 tx=0).stop 2024-09-19T08:38:29.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f2679c1a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f26740cfd60 con 0x7f26740cdf80 2024-09-19T08:38:29.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f2679c1a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2674154da0 con 0x7f26741638f0 2024-09-19T08:38:29.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f54985b8640 1 -- 172.21.15.98:0/1261220039 wait complete. 2024-09-19T08:38:29.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7fb4e3fb5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4d40338f0 0x7fb4d404d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f3b67fff640 1 -- 172.21.15.98:0/3598049632 shutdown_connections 2024-09-19T08:38:29.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f3b67fff640 1 --2- 172.21.15.98:0/3598049632 >> v2:172.21.15.98:3300/0 conn(0x7f3b6806c3a0 0x7f3b68071770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f3b67fff640 1 --2- 172.21.15.98:0/3598049632 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b681627a0 0x7f3b680cd4f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f3b67fff640 1 -- 172.21.15.98:0/3598049632 >> 172.21.15.98:0/3598049632 conn(0x7f3b68075070 msgr2=0x7f3b680b1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:29.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f369714f640 1 -- 172.21.15.98:0/2018521182 shutdown_connections 2024-09-19T08:38:29.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f369714f640 1 --2- 172.21.15.98:0/2018521182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36780051d0 0x7f36780141e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f369714f640 1 -- 172.21.15.98:0/2018521182 >> 172.21.15.98:0/2018521182 conn(0x7f36900b41c0 msgr2=0x7f36900b45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:29.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f3b67fff640 1 -- 172.21.15.98:0/3598049632 shutdown_connections 2024-09-19T08:38:29.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f2cc6018640 1 -- 172.21.15.98:0/2862898130 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2cb80e0a00 msgr2=0x7f2cac004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:29.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f2cc6018640 1 --2- 172.21.15.98:0/2862898130 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2cb80e0a00 0x7f2cac004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f2cc6018640 1 -- 172.21.15.98:0/2862898130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cac0039c0 msgr2=0x7f2cac003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:29.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f2cc6018640 1 --2- 172.21.15.98:0/2862898130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cac0039c0 0x7f2cac003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f2cc6018640 1 -- 172.21.15.98:0/2862898130 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb801cbf0 con 0x7f2cac004d50 2024-09-19T08:38:29.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f369714f640 1 -- 172.21.15.98:0/2018521182 shutdown_connections 2024-09-19T08:38:29.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7fb4e3fb5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4d40475b0 0x7fb4d40dbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f369714f640 1 -- 172.21.15.98:0/2018521182 wait complete. 2024-09-19T08:38:29.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7fb4e3fb5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb4d404eba0 0x7fb4d40db200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f3b67fff640 1 -- 172.21.15.98:0/3598049632 wait complete. 2024-09-19T08:38:29.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7fb4e3fb5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d4007440 con 0x7fb4d40475b0 2024-09-19T08:38:29.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7fb4e3fb5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d40072c0 con 0x7fb4d40338f0 2024-09-19T08:38:29.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7fb4e1d51640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4d40338f0 0x7fb4d404d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7fb4e1d51640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4d40338f0 0x7fb4d404d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:50000/0 (socket says 172.21.15.98:50000) 2024-09-19T08:38:29.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7fb4e1d51640 1 -- 172.21.15.98:0/929968759 learned_addr learned my addr 172.21.15.98:0/929968759 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:29.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7fab075b7640 1 -- 172.21.15.98:0/248372406 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab0014c380 msgr2=0x7fab000ccbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7fab075b7640 1 --2- 172.21.15.98:0/248372406 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab0014c380 0x7fab000ccbf0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fab000b64d0 tx=0x7faafc01c510 comp rx=0 tx=0).stop 2024-09-19T08:38:29.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7fb4e2552640 1 --2- 172.21.15.98:0/929968759 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb4d404eba0 0x7fb4d40db200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f54985b8640 1 Processor -- start 2024-09-19T08:38:29.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f54985b8640 1 -- start start 2024-09-19T08:38:29.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7fb4e3fb5640 1 -- 172.21.15.98:0/929968759 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb4d40f5610 con 0x7fb4d404eba0 2024-09-19T08:38:29.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f2cc6018640 1 --2- 172.21.15.98:0/2862898130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cac004d50 0x7f2cac0051d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0050e10 tx=0x7f2cc009ba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c03be341b69d8b1b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f3b67fff640 1 Processor -- start 2024-09-19T08:38:29.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f26737fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26740c56c0 0x7f2674154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f26737fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26740c56c0 0x7f2674154860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37900/0 (socket says 172.21.15.98:37900) 2024-09-19T08:38:29.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f26737fe640 1 -- 172.21.15.98:0/1337482219 learned_addr learned my addr 172.21.15.98:0/1337482219 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:29.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f2672ffd640 1 --2- 172.21.15.98:0/1337482219 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26740cdf80 0x7f2674162df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f2673fff640 1 --2- 172.21.15.98:0/1337482219 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f26741638f0 0x7f2674158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7fb4e1550640 1 --2- 172.21.15.98:0/929968759 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4d40475b0 0x7fb4d40dbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7fab075b7640 1 -- 172.21.15.98:0/248372406 shutdown_connections 2024-09-19T08:38:29.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7fab075b7640 1 --2- 172.21.15.98:0/248372406 >> v2:172.21.15.139:3300/0 conn(0x7fab000b5320 0x7fab000b5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7fab075b7640 1 --2- 172.21.15.98:0/248372406 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab0014c380 0x7fab000ccbf0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7fab075b7640 1 -- 172.21.15.98:0/248372406 >> 172.21.15.98:0/248372406 conn(0x7fab000b1f10 msgr2=0x7fab000b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:29.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.695+0000 7f18447b8640 1 -- 172.21.15.98:0/328393824 wait complete. 2024-09-19T08:38:29.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f2cc4815640 1 -- 172.21.15.98:0/2862898130 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0127450 con 0x7f2cac004d50 2024-09-19T08:38:29.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f2cc4815640 1 -- 172.21.15.98:0/2862898130 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cc0069e30 con 0x7f2cac004d50 2024-09-19T08:38:29.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f2cc4815640 1 -- 172.21.15.98:0/2862898130 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0127930 con 0x7f2cac004d50 2024-09-19T08:38:29.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f18447b8640 1 Processor -- start 2024-09-19T08:38:29.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f2672ffd640 1 -- 172.21.15.98:0/1337482219 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f26741638f0 msgr2=0x7f2674158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f2672ffd640 1 --2- 172.21.15.98:0/1337482219 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f26741638f0 0x7f2674158510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f2672ffd640 1 -- 172.21.15.98:0/1337482219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26740c56c0 msgr2=0x7f2674154860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f2672ffd640 1 --2- 172.21.15.98:0/1337482219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26740c56c0 0x7f2674154860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f2672ffd640 1 -- 172.21.15.98:0/1337482219 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26740b1530 con 0x7f26740cdf80 2024-09-19T08:38:29.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7fb4e2552640 1 -- 172.21.15.98:0/929968759 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4d40338f0 msgr2=0x7fb4d404d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7fb4e2552640 1 --2- 172.21.15.98:0/929968759 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4d40338f0 0x7fb4d404d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7fb4e2552640 1 -- 172.21.15.98:0/929968759 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4d40475b0 msgr2=0x7fb4d40dbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7fb4e2552640 1 --2- 172.21.15.98:0/929968759 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4d40475b0 0x7fb4d40dbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7fb4e2552640 1 -- 172.21.15.98:0/929968759 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d404d900 con 0x7fb4d404eba0 2024-09-19T08:38:29.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7fb4e2552640 1 --2- 172.21.15.98:0/929968759 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb4d404eba0 0x7fb4d40db200 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb4d80ac000 tx=0x7fb4d801ad80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f3b67fff640 1 -- start start 2024-09-19T08:38:29.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f3b67fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b6806c3a0 0x7f3b68156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f3b67fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b680cf880 0x7f3b6815ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f3b67fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b681627a0 0x7f3b68154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f3b67fff640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3b68162d00 con 0x7f3b6806c3a0 2024-09-19T08:38:29.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f3b67fff640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3b68162b80 con 0x7f3b681627a0 2024-09-19T08:38:29.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f3b67fff640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3b680724c0 con 0x7f3b680cf880 2024-09-19T08:38:29.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f3b677fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b681627a0 0x7f3b68154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f3b677fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b681627a0 0x7f3b68154770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:50002/0 (socket says 172.21.15.98:50002) 2024-09-19T08:38:29.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f3b677fe640 1 -- 172.21.15.98:0/2685158155 learned_addr learned my addr 172.21.15.98:0/2685158155 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:29.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f3b667fc640 1 --2- 172.21.15.98:0/2685158155 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b680cf880 0x7f3b6815ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f369714f640 1 Processor -- start 2024-09-19T08:38:29.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f54985b8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5490070d40 0x7f5490207d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f54985b8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5490073d70 0x7f549013c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f54985b8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5490145640 0x7f54900cd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f54985b8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f54900b5740 con 0x7f5490073d70 2024-09-19T08:38:29.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f54985b8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f54900b55c0 con 0x7f5490145640 2024-09-19T08:38:29.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f54985b8640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f54900b58c0 con 0x7f5490070d40 2024-09-19T08:38:29.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f5496b55640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5490145640 0x7f54900cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f5496b55640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5490145640 0x7f54900cd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:50014/0 (socket says 172.21.15.98:50014) 2024-09-19T08:38:29.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f5496b55640 1 -- 172.21.15.98:0/2896854217 learned_addr learned my addr 172.21.15.98:0/2896854217 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:29.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f5495b53640 1 --2- 172.21.15.98:0/2896854217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5490073d70 0x7f549013c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f5496354640 1 --2- 172.21.15.98:0/2896854217 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5490070d40 0x7f5490207d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7fab075b7640 1 -- 172.21.15.98:0/248372406 shutdown_connections 2024-09-19T08:38:29.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7fab075b7640 1 -- 172.21.15.98:0/248372406 wait complete. 2024-09-19T08:38:29.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f2672ffd640 1 --2- 172.21.15.98:0/1337482219 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26740cdf80 0x7f2674162df0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f266400e870 tx=0x7f266400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f265bfff640 1 -- 172.21.15.98:0/1337482219 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2664017070 con 0x7f26740cdf80 2024-09-19T08:38:29.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f2679c1a640 1 -- 172.21.15.98:0/1337482219 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2674158fb0 con 0x7f26740cdf80 2024-09-19T08:38:29.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f2679c1a640 1 -- 172.21.15.98:0/1337482219 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26741625f0 con 0x7f26740cdf80 2024-09-19T08:38:29.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f265bfff640 1 -- 172.21.15.98:0/1337482219 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f266400ee60 con 0x7f26740cdf80 2024-09-19T08:38:29.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f265bfff640 1 -- 172.21.15.98:0/1337482219 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266400cb60 con 0x7f26740cdf80 2024-09-19T08:38:29.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f18447b8640 1 -- start start 2024-09-19T08:38:29.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f2cc7a7b640 1 -- 172.21.15.98:0/2862898130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cac004d50 msgr2=0x7f2cac0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f2cc7a7b640 1 --2- 172.21.15.98:0/2862898130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cac004d50 0x7f2cac0051d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0050e10 tx=0x7f2cc009ba80 comp rx=0 tx=0).stop 2024-09-19T08:38:29.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7fb4d27fc640 1 -- 172.21.15.98:0/929968759 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d80be5f0 con 0x7fb4d404eba0 2024-09-19T08:38:29.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7fb4e3fb5640 1 -- 172.21.15.98:0/929968759 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4d40375d0 con 0x7fb4d404eba0 2024-09-19T08:38:29.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7fb4e3fb5640 1 -- 172.21.15.98:0/929968759 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4d40db740 con 0x7fb4d404eba0 2024-09-19T08:38:29.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f3b66ffd640 1 --2- 172.21.15.98:0/2685158155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b6806c3a0 0x7f3b68156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f3b677fe640 1 -- 172.21.15.98:0/2685158155 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b680cf880 msgr2=0x7f3b6815ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f3b677fe640 1 --2- 172.21.15.98:0/2685158155 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b680cf880 0x7f3b6815ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f3b677fe640 1 -- 172.21.15.98:0/2685158155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b6806c3a0 msgr2=0x7f3b68156470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f3b677fe640 1 --2- 172.21.15.98:0/2685158155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b6806c3a0 0x7f3b68156470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f3b677fe640 1 -- 172.21.15.98:0/2685158155 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b6815b260 con 0x7f3b681627a0 2024-09-19T08:38:29.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f5495b53640 1 -- 172.21.15.98:0/2896854217 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5490070d40 msgr2=0x7f5490207d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f5495b53640 1 --2- 172.21.15.98:0/2896854217 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5490070d40 0x7f5490207d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f5495b53640 1 -- 172.21.15.98:0/2896854217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5490145640 msgr2=0x7f54900cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f5495b53640 1 --2- 172.21.15.98:0/2896854217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5490145640 0x7f54900cd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f5495b53640 1 -- 172.21.15.98:0/2896854217 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5490204230 con 0x7f5490073d70 2024-09-19T08:38:29.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f5495b53640 1 --2- 172.21.15.98:0/2896854217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5490073d70 0x7f549013c230 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f548c0ab000 tx=0x7f548c019020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7fab075b7640 1 Processor -- start 2024-09-19T08:38:29.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f18447b8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f183c15a2d0 0x7f183c155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f18447b8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f183c15e110 0x7f183c0aa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f18447b8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f183c15ea30 0x7f183c0aa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f18447b8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f183c0ce530 con 0x7f183c15ea30 2024-09-19T08:38:29.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f18447b8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f183c0ce3b0 con 0x7f183c15a2d0 2024-09-19T08:38:29.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f18447b8640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f183c0ce6b0 con 0x7f183c15e110 2024-09-19T08:38:29.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f1842554640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f183c15a2d0 0x7f183c155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f1841d53640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f183c15e110 0x7f183c0aa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f1842554640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f183c15a2d0 0x7f183c155cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:50016/0 (socket says 172.21.15.98:50016) 2024-09-19T08:38:29.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f1842554640 1 -- 172.21.15.98:0/789221388 learned_addr learned my addr 172.21.15.98:0/789221388 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:29.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f1842d55640 1 --2- 172.21.15.98:0/789221388 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f183c15ea30 0x7f183c0aa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7fb4d27fc640 1 -- 172.21.15.98:0/929968759 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4d80be7d0 con 0x7fb4d404eba0 2024-09-19T08:38:29.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7fb4d27fc640 1 -- 172.21.15.98:0/929968759 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d80bebc0 con 0x7fb4d404eba0 2024-09-19T08:38:29.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7fab075b7640 1 -- start start 2024-09-19T08:38:29.744 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.699+0000 7f369714f640 1 -- start start 2024-09-19T08:38:29.744 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f3b677fe640 1 --2- 172.21.15.98:0/2685158155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b681627a0 0x7f3b68154770 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3b6000efc0 tx=0x7f3b6000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.744 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f2cc6018640 1 -- 172.21.15.98:0/2862898130 reap_dead start 2024-09-19T08:38:29.744 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f2cc7a7b640 1 -- 172.21.15.98:0/2862898130 shutdown_connections 2024-09-19T08:38:29.744 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f2cc7a7b640 1 -- 172.21.15.98:0/2862898130 >> 172.21.15.98:0/2862898130 conn(0x7f2cb80360a0 msgr2=0x7f2cb80364b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:29.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7fb4e3fb5640 1 -- 172.21.15.98:0/929968759 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4d404ef80 con 0x7fb4d404eba0 2024-09-19T08:38:29.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f2cc7a7b640 1 -- 172.21.15.98:0/2862898130 shutdown_connections 2024-09-19T08:38:29.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f3b4f7fe640 1 -- 172.21.15.98:0/2685158155 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6000cd20 con 0x7f3b681627a0 2024-09-19T08:38:29.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f3b67fff640 1 -- 172.21.15.98:0/2685158155 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b6815b900 con 0x7f3b681627a0 2024-09-19T08:38:29.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f3b67fff640 1 -- 172.21.15.98:0/2685158155 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b6815bec0 con 0x7f3b681627a0 2024-09-19T08:38:29.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f1842554640 1 -- 172.21.15.98:0/789221388 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f183c15e110 msgr2=0x7f183c0aa4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f1842554640 1 --2- 172.21.15.98:0/789221388 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f183c15e110 0x7f183c0aa4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f1842554640 1 -- 172.21.15.98:0/789221388 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f183c15ea30 msgr2=0x7f183c0aa9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f1842554640 1 --2- 172.21.15.98:0/789221388 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f183c15ea30 0x7f183c0aa9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f1842554640 1 -- 172.21.15.98:0/789221388 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183c0ac730 con 0x7f183c15a2d0 2024-09-19T08:38:29.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7fab075b7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab000b4e30 0x7fab00144b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7fab075b7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab000b5320 0x7fab001fb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7fab075b7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab0014c380 0x7fab000b4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7fab075b7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fab000cd130 con 0x7fab0014c380 2024-09-19T08:38:29.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7fab075b7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fab000ceca0 con 0x7fab000b5320 2024-09-19T08:38:29.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7fab075b7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fab000cd2b0 con 0x7fab000b4e30 2024-09-19T08:38:29.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7fab04b52640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab000b5320 0x7fab001fb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f369714f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36780051d0 0x7f36900d1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7fab04b52640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab000b5320 0x7fab001fb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:50022/0 (socket says 172.21.15.98:50022) 2024-09-19T08:38:29.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7fab04b52640 1 -- 172.21.15.98:0/572006557 learned_addr learned my addr 172.21.15.98:0/572006557 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:29.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f369714f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3690152780 0x7f369015f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f369714f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f369006efe0 0x7f369006f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f3b4f7fe640 1 -- 172.21.15.98:0/2685158155 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b600039a0 con 0x7f3b681627a0 2024-09-19T08:38:29.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f3b4f7fe640 1 -- 172.21.15.98:0/2685158155 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b60003d90 con 0x7f3b681627a0 2024-09-19T08:38:29.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f369714f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f36900b1610 con 0x7f369006efe0 2024-09-19T08:38:29.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f369714f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f36900b1490 con 0x7f36780051d0 2024-09-19T08:38:29.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f369714f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f369015c080 con 0x7f3690152780 2024-09-19T08:38:29.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7fab05b54640 1 --2- 172.21.15.98:0/572006557 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab0014c380 0x7fab000b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f36956ec640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f369006efe0 0x7f369006f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f36956ec640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f369006efe0 0x7f369006f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37938/0 (socket says 172.21.15.98:37938) 2024-09-19T08:38:29.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f36956ec640 1 -- 172.21.15.98:0/997321617 learned_addr learned my addr 172.21.15.98:0/997321617 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:29.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f1842554640 1 --2- 172.21.15.98:0/789221388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f183c15a2d0 0x7f183c155cb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f182c00d990 tx=0x7f182c00dd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f3b67fff640 1 -- 172.21.15.98:0/2685158155 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b480052a0 con 0x7f3b681627a0 2024-09-19T08:38:29.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7fab04b52640 1 -- 172.21.15.98:0/572006557 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab000b4e30 msgr2=0x7fab00144b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7fab04b52640 1 --2- 172.21.15.98:0/572006557 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab000b4e30 0x7fab00144b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7fab04b52640 1 -- 172.21.15.98:0/572006557 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab0014c380 msgr2=0x7fab000b4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7fab04b52640 1 --2- 172.21.15.98:0/572006557 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab0014c380 0x7fab000b4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7fab04b52640 1 -- 172.21.15.98:0/572006557 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab001450d0 con 0x7fab000b5320 2024-09-19T08:38:29.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f36956ec640 1 -- 172.21.15.98:0/997321617 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3690152780 msgr2=0x7f369015f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f36956ec640 1 --2- 172.21.15.98:0/997321617 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3690152780 0x7f369015f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f36956ec640 1 -- 172.21.15.98:0/997321617 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36780051d0 msgr2=0x7f36900d1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f36956ec640 1 --2- 172.21.15.98:0/997321617 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36780051d0 0x7f36900d1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f36956ec640 1 -- 172.21.15.98:0/997321617 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36900d0560 con 0x7f369006efe0 2024-09-19T08:38:29.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f1832ffd640 1 -- 172.21.15.98:0/789221388 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182c015070 con 0x7f183c15a2d0 2024-09-19T08:38:29.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f1832ffd640 1 -- 172.21.15.98:0/789221388 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f182c007b10 con 0x7f183c15a2d0 2024-09-19T08:38:29.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f18447b8640 1 -- 172.21.15.98:0/789221388 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f183c0aca10 con 0x7f183c15a2d0 2024-09-19T08:38:29.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f2cc7a7b640 1 -- 172.21.15.98:0/2862898130 wait complete. 2024-09-19T08:38:29.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f2cc7a7b640 1 Processor -- start 2024-09-19T08:38:29.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f36956ec640 1 --2- 172.21.15.98:0/997321617 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f369006efe0 0x7f369006f460 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f368001c390 tx=0x7f3678005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f2cc7a7b640 1 -- start start 2024-09-19T08:38:29.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f1832ffd640 1 -- 172.21.15.98:0/789221388 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182c010470 con 0x7f183c15a2d0 2024-09-19T08:38:29.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f18447b8640 1 -- 172.21.15.98:0/789221388 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f183c0acfd0 con 0x7f183c15a2d0 2024-09-19T08:38:29.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f368d7fa640 1 -- 172.21.15.98:0/997321617 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3680003800 con 0x7f369006efe0 2024-09-19T08:38:29.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f368d7fa640 1 -- 172.21.15.98:0/997321617 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36800039a0 con 0x7f369006efe0 2024-09-19T08:38:29.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f368d7fa640 1 -- 172.21.15.98:0/997321617 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3680003d50 con 0x7f369006efe0 2024-09-19T08:38:29.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f5486ffd640 1 -- 172.21.15.98:0/2896854217 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c0b8070 con 0x7f5490073d70 2024-09-19T08:38:29.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7fab04b52640 1 --2- 172.21.15.98:0/572006557 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab000b5320 0x7fab001fb3f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faaf000c4c0 tx=0x7faaf000c980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7fb4d27fc640 1 -- 172.21.15.98:0/929968759 <== mon.2 v2:172.21.15.139:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb4d80b4600 con 0x7fb4d404eba0 2024-09-19T08:38:29.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7fb4d27fc640 1 --2- 172.21.15.98:0/929968759 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb4c00678c0 0x7fb4c0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7fb4d27fc640 1 -- 172.21.15.98:0/929968759 <== mon.2 v2:172.21.15.139:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb4d8120b40 con 0x7fb4d404eba0 2024-09-19T08:38:29.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f2679c1a640 1 -- 172.21.15.98:0/1337482219 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26540052a0 con 0x7f26740cdf80 2024-09-19T08:38:29.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f265bfff640 1 -- 172.21.15.98:0/1337482219 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f266401a070 con 0x7f26740cdf80 2024-09-19T08:38:29.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f265bfff640 1 --2- 172.21.15.98:0/1337482219 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f26600678c0 0x7f2660069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f265bfff640 1 -- 172.21.15.98:0/1337482219 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f26640861a0 con 0x7f26740cdf80 2024-09-19T08:38:29.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f3b4f7fe640 1 -- 172.21.15.98:0/2685158155 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3b60012050 con 0x7f3b681627a0 2024-09-19T08:38:29.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f26737fe640 1 --2- 172.21.15.98:0/1337482219 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f26600678c0 0x7f2660069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.707+0000 7f3b4f7fe640 1 --2- 172.21.15.98:0/2685158155 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3b540678f0 0x7f3b54069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f369714f640 1 -- 172.21.15.98:0/997321617 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36900d0840 con 0x7f369006efe0 2024-09-19T08:38:29.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f369714f640 1 -- 172.21.15.98:0/997321617 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36900d0e00 con 0x7f369006efe0 2024-09-19T08:38:29.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f5486ffd640 1 -- 172.21.15.98:0/2896854217 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f548c0b4410 con 0x7f5490073d70 2024-09-19T08:38:29.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f5486ffd640 1 -- 172.21.15.98:0/2896854217 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c0b0040 con 0x7f5490073d70 2024-09-19T08:38:29.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f54985b8640 1 -- 172.21.15.98:0/2896854217 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5490204510 con 0x7f5490073d70 2024-09-19T08:38:29.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.703+0000 7f54985b8640 1 -- 172.21.15.98:0/2896854217 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5490204ad0 con 0x7f5490073d70 2024-09-19T08:38:29.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.707+0000 7fb4e1d51640 1 --2- 172.21.15.98:0/929968759 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb4c00678c0 0x7fb4c0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.707+0000 7f3b4f7fe640 1 -- 172.21.15.98:0/2685158155 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3b60086cb0 con 0x7f3b681627a0 2024-09-19T08:38:29.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.707+0000 7f3b66ffd640 1 --2- 172.21.15.98:0/2685158155 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3b540678f0 0x7f3b54069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.707+0000 7fb4e1d51640 1 --2- 172.21.15.98:0/929968759 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb4c00678c0 0x7fb4c0069d80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc050bf0 tx=0x7fb4dc06e040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.707+0000 7f26737fe640 1 --2- 172.21.15.98:0/1337482219 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f26600678c0 0x7f2660069d80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f266801cd40 tx=0x7f26680a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.707+0000 7f18447b8640 1 -- 172.21.15.98:0/789221388 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f183c15e4f0 con 0x7f183c15a2d0 2024-09-19T08:38:29.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.707+0000 7f54985b8640 1 -- 172.21.15.98:0/2896854217 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54900cee70 con 0x7f5490073d70 2024-09-19T08:38:29.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.707+0000 7f2cc7a7b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2cb80d8af0 0x7f2cb80da3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.707+0000 7f2cc7a7b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cb80dbab0 0x7f2cb80df6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.707+0000 7f2cc7a7b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cb80dca50 0x7f2cb80dcee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.707+0000 7f2cc7a7b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb801c530 con 0x7f2cb80dca50 2024-09-19T08:38:29.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.707+0000 7f2cc7a7b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb801d1b0 con 0x7f2cb80dbab0 2024-09-19T08:38:29.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.707+0000 7f2cc7a7b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2cb801c6b0 con 0x7f2cb80d8af0 2024-09-19T08:38:29.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.707+0000 7f369714f640 1 -- 172.21.15.98:0/997321617 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36900b4ce0 con 0x7f369006efe0 2024-09-19T08:38:29.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.707+0000 7f2cc5016640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cb80dbab0 0x7f2cb80df6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.707+0000 7f2cc5016640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cb80dbab0 0x7f2cb80df6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:50046/0 (socket says 172.21.15.98:50046) 2024-09-19T08:38:29.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.707+0000 7f2cc5016640 1 -- 172.21.15.98:0/3047273880 learned_addr learned my addr 172.21.15.98:0/3047273880 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:29.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.707+0000 7f2cc5817640 1 --2- 172.21.15.98:0/3047273880 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2cb80d8af0 0x7f2cb80da3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.707+0000 7f2cc6018640 1 --2- 172.21.15.98:0/3047273880 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cb80dca50 0x7f2cb80dcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.707+0000 7f2cc5016640 1 -- 172.21.15.98:0/3047273880 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2cb80d8af0 msgr2=0x7f2cb80da3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.707+0000 7f2cc5016640 1 --2- 172.21.15.98:0/3047273880 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2cb80d8af0 0x7f2cb80da3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.707+0000 7f2cc5016640 1 -- 172.21.15.98:0/3047273880 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cb80dca50 msgr2=0x7f2cb80dcee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.707+0000 7f2cc5016640 1 --2- 172.21.15.98:0/3047273880 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cb80dca50 0x7f2cb80dcee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.707+0000 7f2cc5016640 1 -- 172.21.15.98:0/3047273880 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb80dfbe0 con 0x7f2cb80dbab0 2024-09-19T08:38:29.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.707+0000 7f368d7fa640 1 -- 172.21.15.98:0/997321617 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f36800b2030 con 0x7f369006efe0 2024-09-19T08:38:29.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.707+0000 7f2cc5016640 1 --2- 172.21.15.98:0/3047273880 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cb80dbab0 0x7f2cb80df6a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc009f60 tx=0x7f2cbc009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.707+0000 7f368d7fa640 1 --2- 172.21.15.98:0/997321617 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f367c0678c0 0x7f367c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.707+0000 7f368d7fa640 1 -- 172.21.15.98:0/997321617 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f368011f7c0 con 0x7f369006efe0 2024-09-19T08:38:29.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.707+0000 7f5486ffd640 1 -- 172.21.15.98:0/2896854217 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f548c0bb070 con 0x7f5490073d70 2024-09-19T08:38:29.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.707+0000 7f3b66ffd640 1 --2- 172.21.15.98:0/2685158155 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3b540678f0 0x7f3b54069db0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c0ad000 tx=0x7f3b5c0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.707+0000 7f5486ffd640 1 --2- 172.21.15.98:0/2896854217 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5474067910 0x7f5474069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.707+0000 7f5486ffd640 1 -- 172.21.15.98:0/2896854217 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f548c11fbd0 con 0x7f5490073d70 2024-09-19T08:38:29.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.707+0000 7f3694eeb640 1 --2- 172.21.15.98:0/997321617 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f367c0678c0 0x7f367c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.707+0000 7f5496354640 1 --2- 172.21.15.98:0/2896854217 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5474067910 0x7f5474069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.711+0000 7f3694eeb640 1 --2- 172.21.15.98:0/997321617 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f367c0678c0 0x7f367c069d80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3684001000 tx=0x7f3684001d60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.711+0000 7f5496354640 1 --2- 172.21.15.98:0/2896854217 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5474067910 0x7f5474069dd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5488009d80 tx=0x7f548800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.711+0000 7f368d7fa640 1 -- 172.21.15.98:0/997321617 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f36800b0040 con 0x7f369006efe0 2024-09-19T08:38:29.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.719+0000 7f5486ffd640 1 -- 172.21.15.98:0/2896854217 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f549015cda0 con 0x7f5490073d70 2024-09-19T08:38:29.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.819+0000 7f2cb67fc640 1 -- 172.21.15.98:0/3047273880 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc023070 con 0x7f2cb80dbab0 2024-09-19T08:38:29.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.819+0000 7faaf5ffb640 1 -- 172.21.15.98:0/572006557 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf0018070 con 0x7fab000b5320 2024-09-19T08:38:29.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.819+0000 7fab075b7640 1 -- 172.21.15.98:0/572006557 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab000aca40 con 0x7fab000b5320 2024-09-19T08:38:29.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.819+0000 7f2cc7a7b640 1 -- 172.21.15.98:0/3047273880 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cb80dacf0 con 0x7f2cb80dbab0 2024-09-19T08:38:29.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.819+0000 7fab075b7640 1 -- 172.21.15.98:0/572006557 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab000ad000 con 0x7fab000b5320 2024-09-19T08:38:29.822 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.819+0000 7fb4d27fc640 1 -- 172.21.15.98:0/929968759 <== mon.2 v2:172.21.15.139:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb4d80c4060 con 0x7fb4d404eba0 2024-09-19T08:38:29.822 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.819+0000 7f2cc7a7b640 1 -- 172.21.15.98:0/3047273880 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cb80db2b0 con 0x7f2cb80dbab0 2024-09-19T08:38:29.822 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.819+0000 7f3b4f7fe640 1 -- 172.21.15.98:0/2685158155 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3b60010040 con 0x7f3b681627a0 2024-09-19T08:38:29.822 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.823+0000 7fab075b7640 1 -- 172.21.15.98:0/572006557 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab000ad700 con 0x7fab000b5320 2024-09-19T08:38:29.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.823+0000 7f2cc7a7b640 1 -- 172.21.15.98:0/3047273880 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cb801c870 con 0x7f2cb80dbab0 2024-09-19T08:38:29.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.831+0000 7f265bfff640 1 -- 172.21.15.98:0/1337482219 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f26640565f0 con 0x7f26740cdf80 2024-09-19T08:38:29.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.835+0000 7faaf5ffb640 1 -- 172.21.15.98:0/572006557 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaf0007480 con 0x7fab000b5320 2024-09-19T08:38:29.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.835+0000 7faaf5ffb640 1 -- 172.21.15.98:0/572006557 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf0022060 con 0x7fab000b5320 2024-09-19T08:38:29.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.835+0000 7f2cb67fc640 1 -- 172.21.15.98:0/3047273880 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cbc00faf0 con 0x7f2cb80dbab0 2024-09-19T08:38:29.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.835+0000 7f2cb67fc640 1 -- 172.21.15.98:0/3047273880 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc004030 con 0x7f2cb80dbab0 2024-09-19T08:38:29.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.835+0000 7f1832ffd640 1 -- 172.21.15.98:0/789221388 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f182c018070 con 0x7f183c15a2d0 2024-09-19T08:38:29.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.835+0000 7f1832ffd640 1 --2- 172.21.15.98:0/789221388 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f18200678c0 0x7f1820069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.835+0000 7f1832ffd640 1 -- 172.21.15.98:0/789221388 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f182c086330 con 0x7f183c15a2d0 2024-09-19T08:38:29.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.835+0000 7faaf5ffb640 1 -- 172.21.15.98:0/572006557 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7faaf001b070 con 0x7fab000b5320 2024-09-19T08:38:29.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.835+0000 7f1841d53640 1 --2- 172.21.15.98:0/789221388 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f18200678c0 0x7f1820069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.835+0000 7f2cb67fc640 1 -- 172.21.15.98:0/3047273880 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2cbc026070 con 0x7f2cb80dbab0 2024-09-19T08:38:29.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.839+0000 7f2cb67fc640 1 --2- 172.21.15.98:0/3047273880 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2ca40678c0 0x7f2ca4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.839+0000 7f1841d53640 1 --2- 172.21.15.98:0/789221388 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f18200678c0 0x7f1820069d80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f18380b2000 tx=0x7f1838008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.839+0000 7faaf5ffb640 1 --2- 172.21.15.98:0/572006557 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7faae40678c0 0x7faae4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:29.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.839+0000 7fab05353640 1 --2- 172.21.15.98:0/572006557 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7faae40678c0 0x7faae4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.839+0000 7fab05353640 1 --2- 172.21.15.98:0/572006557 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7faae40678c0 0x7faae4069d80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7faafc0ae000 tx=0x7faafc004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.847+0000 7faaf5ffb640 1 -- 172.21.15.98:0/572006557 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7faaf00539d0 con 0x7fab000b5320 2024-09-19T08:38:29.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.847+0000 7f2cb67fc640 1 -- 172.21.15.98:0/3047273880 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2cbc058900 con 0x7f2cb80dbab0 2024-09-19T08:38:29.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.847+0000 7f1832ffd640 1 -- 172.21.15.98:0/789221388 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f182c056810 con 0x7f183c15a2d0 2024-09-19T08:38:29.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.847+0000 7f2cc5817640 1 --2- 172.21.15.98:0/3047273880 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2ca40678c0 0x7f2ca4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:29.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.859+0000 7faaf5ffb640 1 -- 172.21.15.98:0/572006557 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7faaf0055930 con 0x7fab000b5320 2024-09-19T08:38:29.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.863+0000 7f2cc5817640 1 --2- 172.21.15.98:0/3047273880 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2ca40678c0 0x7f2ca4069d80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0006fd0 tx=0x7f2cb0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:29.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.871+0000 7f2cb67fc640 1 -- 172.21.15.98:0/3047273880 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2cbc02d080 con 0x7f2cb80dbab0 2024-09-19T08:38:29.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.911+0000 7f54985b8640 1 -- 172.21.15.98:0/2896854217 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f549015cf80 con 0x7f5490073d70 2024-09-19T08:38:29.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.911+0000 7f5486ffd640 1 -- 172.21.15.98:0/2896854217 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f548c0f0130 con 0x7f5490073d70 2024-09-19T08:38:29.912 INFO:teuthology.orchestra.run.smithi098.stdout:60129542149 2024-09-19T08:38:29.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.915+0000 7f5484ff9640 1 -- 172.21.15.98:0/2896854217 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5474067910 msgr2=0x7f5474069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.915+0000 7f5484ff9640 1 --2- 172.21.15.98:0/2896854217 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5474067910 0x7f5474069dd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5488009d80 tx=0x7f548800b040 comp rx=0 tx=0).stop 2024-09-19T08:38:29.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.915+0000 7f5484ff9640 1 -- 172.21.15.98:0/2896854217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5490073d70 msgr2=0x7f549013c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.915+0000 7f5484ff9640 1 --2- 172.21.15.98:0/2896854217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5490073d70 0x7f549013c230 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f548c0ab000 tx=0x7f548c019020 comp rx=0 tx=0).stop 2024-09-19T08:38:29.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.915+0000 7f5484ff9640 1 -- 172.21.15.98:0/2896854217 shutdown_connections 2024-09-19T08:38:29.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.915+0000 7f5484ff9640 1 --2- 172.21.15.98:0/2896854217 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5474067910 0x7f5474069dd0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.915+0000 7f5484ff9640 1 --2- 172.21.15.98:0/2896854217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5490145640 0x7f54900cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.915+0000 7f5484ff9640 1 --2- 172.21.15.98:0/2896854217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5490073d70 0x7f549013c230 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.915+0000 7f5484ff9640 1 --2- 172.21.15.98:0/2896854217 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5490070d40 0x7f5490207d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.915+0000 7f5484ff9640 1 -- 172.21.15.98:0/2896854217 >> 172.21.15.98:0/2896854217 conn(0x7f54900b1f10 msgr2=0x7f54900ce6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:29.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.915+0000 7f5484ff9640 1 -- 172.21.15.98:0/2896854217 shutdown_connections 2024-09-19T08:38:29.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.915+0000 7f5484ff9640 1 -- 172.21.15.98:0/2896854217 wait complete. 2024-09-19T08:38:29.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.931+0000 7f369714f640 1 -- 172.21.15.98:0/997321617 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f3690161890 con 0x7f369006efe0 2024-09-19T08:38:29.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.931+0000 7f368d7fa640 1 -- 172.21.15.98:0/997321617 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3680016270 con 0x7f369006efe0 2024-09-19T08:38:29.930 INFO:teuthology.orchestra.run.smithi098.stdout:64424509445 2024-09-19T08:38:29.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.935+0000 7f3676ffd640 1 -- 172.21.15.98:0/997321617 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f367c0678c0 msgr2=0x7f367c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.935+0000 7f3676ffd640 1 --2- 172.21.15.98:0/997321617 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f367c0678c0 0x7f367c069d80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3684001000 tx=0x7f3684001d60 comp rx=0 tx=0).stop 2024-09-19T08:38:29.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.935+0000 7f3676ffd640 1 -- 172.21.15.98:0/997321617 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f369006efe0 msgr2=0x7f369006f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:29.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.935+0000 7f3676ffd640 1 --2- 172.21.15.98:0/997321617 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f369006efe0 0x7f369006f460 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f368001c390 tx=0x7f3678005840 comp rx=0 tx=0).stop 2024-09-19T08:38:29.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.935+0000 7f3676ffd640 1 -- 172.21.15.98:0/997321617 shutdown_connections 2024-09-19T08:38:29.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.935+0000 7f3676ffd640 1 --2- 172.21.15.98:0/997321617 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f367c0678c0 0x7f367c069d80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.935+0000 7f3676ffd640 1 --2- 172.21.15.98:0/997321617 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f369006efe0 0x7f369006f460 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.935+0000 7f3676ffd640 1 --2- 172.21.15.98:0/997321617 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3690152780 0x7f369015f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.935+0000 7f3676ffd640 1 --2- 172.21.15.98:0/997321617 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36780051d0 0x7f36900d1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:29.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.935+0000 7f3676ffd640 1 -- 172.21.15.98:0/997321617 >> 172.21.15.98:0/997321617 conn(0x7f36900b41c0 msgr2=0x7f36900b2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:29.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.935+0000 7f3676ffd640 1 -- 172.21.15.98:0/997321617 shutdown_connections 2024-09-19T08:38:29.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:29.935+0000 7f3676ffd640 1 -- 172.21.15.98:0/997321617 wait complete. 2024-09-19T08:38:29.940 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542149 for osd.3 2024-09-19T08:38:29.959 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509445 for osd.7 2024-09-19T08:38:30.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.067+0000 7fb4e3fb5640 1 -- 172.21.15.98:0/929968759 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fb4d401caa0 con 0x7fb4d404eba0 2024-09-19T08:38:30.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.067+0000 7fb4d27fc640 1 -- 172.21.15.98:0/929968759 <== mon.2 v2:172.21.15.139:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb4d80a8050 con 0x7fb4d404eba0 2024-09-19T08:38:30.068 INFO:teuthology.orchestra.run.smithi098.stdout:64424509446 2024-09-19T08:38:30.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.071+0000 7fb4e3fb5640 1 -- 172.21.15.98:0/929968759 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb4c00678c0 msgr2=0x7fb4c0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:30.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.071+0000 7fb4e3fb5640 1 --2- 172.21.15.98:0/929968759 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb4c00678c0 0x7fb4c0069d80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc050bf0 tx=0x7fb4dc06e040 comp rx=0 tx=0).stop 2024-09-19T08:38:30.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.071+0000 7fb4e3fb5640 1 -- 172.21.15.98:0/929968759 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb4d404eba0 msgr2=0x7fb4d40db200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:30.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.071+0000 7fb4e3fb5640 1 --2- 172.21.15.98:0/929968759 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb4d404eba0 0x7fb4d40db200 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb4d80ac000 tx=0x7fb4d801ad80 comp rx=0 tx=0).stop 2024-09-19T08:38:30.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.071+0000 7fb4e3fb5640 1 -- 172.21.15.98:0/929968759 shutdown_connections 2024-09-19T08:38:30.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.071+0000 7fb4e3fb5640 1 --2- 172.21.15.98:0/929968759 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb4c00678c0 0x7fb4c0069d80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:30.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.071+0000 7fb4e3fb5640 1 --2- 172.21.15.98:0/929968759 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb4d404eba0 0x7fb4d40db200 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:30.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.071+0000 7fb4e3fb5640 1 --2- 172.21.15.98:0/929968759 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb4d40475b0 0x7fb4d40dbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:30.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.071+0000 7fb4e3fb5640 1 --2- 172.21.15.98:0/929968759 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb4d40338f0 0x7fb4d404d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:30.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.071+0000 7fb4e3fb5640 1 -- 172.21.15.98:0/929968759 >> 172.21.15.98:0/929968759 conn(0x7fb4d40360d0 msgr2=0x7fb4d40e33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:30.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.071+0000 7fb4e3fb5640 1 -- 172.21.15.98:0/929968759 shutdown_connections 2024-09-19T08:38:30.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.071+0000 7fb4e3fb5640 1 -- 172.21.15.98:0/929968759 wait complete. 2024-09-19T08:38:30.086 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509446 for osd.1 2024-09-19T08:38:30.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.091+0000 7f3b67fff640 1 -- 172.21.15.98:0/2685158155 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f3b48005860 con 0x7f3b681627a0 2024-09-19T08:38:30.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.091+0000 7f3b4f7fe640 1 -- 172.21.15.98:0/2685158155 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3b60057310 con 0x7f3b681627a0 2024-09-19T08:38:30.091 INFO:teuthology.orchestra.run.smithi098.stdout:64424509447 2024-09-19T08:38:30.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.095+0000 7f3b67fff640 1 -- 172.21.15.98:0/2685158155 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3b540678f0 msgr2=0x7f3b54069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:30.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.095+0000 7f3b67fff640 1 --2- 172.21.15.98:0/2685158155 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3b540678f0 0x7f3b54069db0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c0ad000 tx=0x7f3b5c0a6000 comp rx=0 tx=0).stop 2024-09-19T08:38:30.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.095+0000 7f3b67fff640 1 -- 172.21.15.98:0/2685158155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b681627a0 msgr2=0x7f3b68154770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:30.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.095+0000 7f3b67fff640 1 --2- 172.21.15.98:0/2685158155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b681627a0 0x7f3b68154770 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3b6000efc0 tx=0x7f3b6000c490 comp rx=0 tx=0).stop 2024-09-19T08:38:30.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.095+0000 7f3b67fff640 1 -- 172.21.15.98:0/2685158155 shutdown_connections 2024-09-19T08:38:30.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.095+0000 7f3b67fff640 1 --2- 172.21.15.98:0/2685158155 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3b540678f0 0x7f3b54069db0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:30.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.095+0000 7f3b67fff640 1 --2- 172.21.15.98:0/2685158155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b681627a0 0x7f3b68154770 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:30.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.095+0000 7f3b67fff640 1 --2- 172.21.15.98:0/2685158155 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b680cf880 0x7f3b6815ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:30.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.095+0000 7f3b67fff640 1 --2- 172.21.15.98:0/2685158155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b6806c3a0 0x7f3b68156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:30.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.095+0000 7f3b67fff640 1 -- 172.21.15.98:0/2685158155 >> 172.21.15.98:0/2685158155 conn(0x7f3b68075070 msgr2=0x7f3b6815fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:30.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.095+0000 7f3b67fff640 1 -- 172.21.15.98:0/2685158155 shutdown_connections 2024-09-19T08:38:30.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.095+0000 7f3b67fff640 1 -- 172.21.15.98:0/2685158155 wait complete. 2024-09-19T08:38:30.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.095+0000 7f2679c1a640 1 -- 172.21.15.98:0/1337482219 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f2654005590 con 0x7f26740cdf80 2024-09-19T08:38:30.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.095+0000 7f265bfff640 1 -- 172.21.15.98:0/1337482219 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f266405a8f0 con 0x7f26740cdf80 2024-09-19T08:38:30.096 INFO:teuthology.orchestra.run.smithi098.stdout:55834574854 2024-09-19T08:38:30.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.095+0000 7f2679c1a640 1 -- 172.21.15.98:0/1337482219 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f26600678c0 msgr2=0x7f2660069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:30.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.095+0000 7f2679c1a640 1 --2- 172.21.15.98:0/1337482219 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f26600678c0 0x7f2660069d80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f266801cd40 tx=0x7f26680a7000 comp rx=0 tx=0).stop 2024-09-19T08:38:30.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.095+0000 7f2679c1a640 1 -- 172.21.15.98:0/1337482219 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26740cdf80 msgr2=0x7f2674162df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:30.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.095+0000 7f2679c1a640 1 --2- 172.21.15.98:0/1337482219 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26740cdf80 0x7f2674162df0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f266400e870 tx=0x7f266400ed30 comp rx=0 tx=0).stop 2024-09-19T08:38:30.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.095+0000 7f2679c1a640 1 -- 172.21.15.98:0/1337482219 shutdown_connections 2024-09-19T08:38:30.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.095+0000 7f2679c1a640 1 --2- 172.21.15.98:0/1337482219 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f26600678c0 0x7f2660069d80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:30.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.095+0000 7f2679c1a640 1 --2- 172.21.15.98:0/1337482219 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f26741638f0 0x7f2674158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:30.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.095+0000 7f2679c1a640 1 --2- 172.21.15.98:0/1337482219 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26740cdf80 0x7f2674162df0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:30.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.095+0000 7f2679c1a640 1 --2- 172.21.15.98:0/1337482219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f26740c56c0 0x7f2674154860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:30.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.095+0000 7f2679c1a640 1 -- 172.21.15.98:0/1337482219 >> 172.21.15.98:0/1337482219 conn(0x7f26740b5160 msgr2=0x7f26740d18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:30.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.095+0000 7f2679c1a640 1 -- 172.21.15.98:0/1337482219 shutdown_connections 2024-09-19T08:38:30.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.095+0000 7f2679c1a640 1 -- 172.21.15.98:0/1337482219 wait complete. 2024-09-19T08:38:30.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.099+0000 7f2cc7a7b640 1 -- 172.21.15.98:0/3047273880 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f2cb8036fb0 con 0x7f2cb80dbab0 2024-09-19T08:38:30.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.099+0000 7f2cb67fc640 1 -- 172.21.15.98:0/3047273880 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2cbc05a860 con 0x7f2cb80dbab0 2024-09-19T08:38:30.100 INFO:teuthology.orchestra.run.smithi098.stdout:64424509447 2024-09-19T08:38:30.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.103+0000 7f2cc7a7b640 1 -- 172.21.15.98:0/3047273880 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2ca40678c0 msgr2=0x7f2ca4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:30.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.103+0000 7f2cc7a7b640 1 --2- 172.21.15.98:0/3047273880 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2ca40678c0 0x7f2ca4069d80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0006fd0 tx=0x7f2cb0008040 comp rx=0 tx=0).stop 2024-09-19T08:38:30.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.103+0000 7f2cc7a7b640 1 -- 172.21.15.98:0/3047273880 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cb80dbab0 msgr2=0x7f2cb80df6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:30.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.103+0000 7f2cc7a7b640 1 --2- 172.21.15.98:0/3047273880 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cb80dbab0 0x7f2cb80df6a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc009f60 tx=0x7f2cbc009f90 comp rx=0 tx=0).stop 2024-09-19T08:38:30.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.103+0000 7f2cc7a7b640 1 -- 172.21.15.98:0/3047273880 shutdown_connections 2024-09-19T08:38:30.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.103+0000 7f2cc7a7b640 1 --2- 172.21.15.98:0/3047273880 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2ca40678c0 0x7f2ca4069d80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:30.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.103+0000 7f2cc7a7b640 1 --2- 172.21.15.98:0/3047273880 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2cb80dca50 0x7f2cb80dcee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:30.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.103+0000 7f2cc7a7b640 1 --2- 172.21.15.98:0/3047273880 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2cb80dbab0 0x7f2cb80df6a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:30.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.103+0000 7f2cc7a7b640 1 --2- 172.21.15.98:0/3047273880 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2cb80d8af0 0x7f2cb80da3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:30.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.103+0000 7f2cc7a7b640 1 -- 172.21.15.98:0/3047273880 >> 172.21.15.98:0/3047273880 conn(0x7f2cb80360a0 msgr2=0x7f2cb80e2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:30.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.103+0000 7f2cc7a7b640 1 -- 172.21.15.98:0/3047273880 shutdown_connections 2024-09-19T08:38:30.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.103+0000 7f2cc7a7b640 1 -- 172.21.15.98:0/3047273880 wait complete. 2024-09-19T08:38:30.113 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509447 for osd.5 2024-09-19T08:38:30.117 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.6 2024-09-19T08:38:30.122 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509447 for osd.4 2024-09-19T08:38:30.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.155+0000 7fab075b7640 1 -- 172.21.15.98:0/572006557 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fab00071ec0 con 0x7fab000b5320 2024-09-19T08:38:30.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.159+0000 7faaf5ffb640 1 -- 172.21.15.98:0/572006557 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7faaf0059c30 con 0x7fab000b5320 2024-09-19T08:38:30.158 INFO:teuthology.orchestra.run.smithi098.stdout:64424509444 2024-09-19T08:38:30.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.159+0000 7f18447b8640 1 -- 172.21.15.98:0/789221388 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f183c0757a0 con 0x7f183c15a2d0 2024-09-19T08:38:30.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.159+0000 7f1832ffd640 1 -- 172.21.15.98:0/789221388 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f182c05ab10 con 0x7f183c15a2d0 2024-09-19T08:38:30.159 INFO:teuthology.orchestra.run.smithi098.stdout:64424509444 2024-09-19T08:38:30.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.159+0000 7fab075b7640 1 -- 172.21.15.98:0/572006557 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7faae40678c0 msgr2=0x7faae4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:30.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.159+0000 7fab075b7640 1 --2- 172.21.15.98:0/572006557 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7faae40678c0 0x7faae4069d80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7faafc0ae000 tx=0x7faafc004740 comp rx=0 tx=0).stop 2024-09-19T08:38:30.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.159+0000 7fab075b7640 1 -- 172.21.15.98:0/572006557 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab000b5320 msgr2=0x7fab001fb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:30.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.159+0000 7fab075b7640 1 --2- 172.21.15.98:0/572006557 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab000b5320 0x7fab001fb3f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faaf000c4c0 tx=0x7faaf000c980 comp rx=0 tx=0).stop 2024-09-19T08:38:30.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.159+0000 7fab075b7640 1 -- 172.21.15.98:0/572006557 shutdown_connections 2024-09-19T08:38:30.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.159+0000 7fab075b7640 1 --2- 172.21.15.98:0/572006557 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7faae40678c0 0x7faae4069d80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:30.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.159+0000 7fab075b7640 1 --2- 172.21.15.98:0/572006557 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab0014c380 0x7fab000b4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:30.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.159+0000 7fab075b7640 1 --2- 172.21.15.98:0/572006557 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab000b5320 0x7fab001fb3f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:30.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.159+0000 7fab075b7640 1 --2- 172.21.15.98:0/572006557 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab000b4e30 0x7fab00144b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:30.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.159+0000 7fab075b7640 1 -- 172.21.15.98:0/572006557 >> 172.21.15.98:0/572006557 conn(0x7fab000b1f10 msgr2=0x7fab000724d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:30.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.159+0000 7fab075b7640 1 -- 172.21.15.98:0/572006557 shutdown_connections 2024-09-19T08:38:30.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.159+0000 7fab075b7640 1 -- 172.21.15.98:0/572006557 wait complete. 2024-09-19T08:38:30.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.159+0000 7f18447b8640 1 -- 172.21.15.98:0/789221388 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f18200678c0 msgr2=0x7f1820069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:30.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.159+0000 7f18447b8640 1 --2- 172.21.15.98:0/789221388 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f18200678c0 0x7f1820069d80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f18380b2000 tx=0x7f1838008040 comp rx=0 tx=0).stop 2024-09-19T08:38:30.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.159+0000 7f18447b8640 1 -- 172.21.15.98:0/789221388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f183c15a2d0 msgr2=0x7f183c155cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:30.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.159+0000 7f18447b8640 1 --2- 172.21.15.98:0/789221388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f183c15a2d0 0x7f183c155cb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f182c00d990 tx=0x7f182c00dd90 comp rx=0 tx=0).stop 2024-09-19T08:38:30.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.159+0000 7f18447b8640 1 -- 172.21.15.98:0/789221388 shutdown_connections 2024-09-19T08:38:30.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.159+0000 7f18447b8640 1 --2- 172.21.15.98:0/789221388 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f18200678c0 0x7f1820069d80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:30.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.159+0000 7f18447b8640 1 --2- 172.21.15.98:0/789221388 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f183c15ea30 0x7f183c0aa9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:30.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.159+0000 7f18447b8640 1 --2- 172.21.15.98:0/789221388 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f183c15e110 0x7f183c0aa4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:30.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.159+0000 7f18447b8640 1 --2- 172.21.15.98:0/789221388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f183c15a2d0 0x7f183c155cb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:30.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.159+0000 7f18447b8640 1 -- 172.21.15.98:0/789221388 >> 172.21.15.98:0/789221388 conn(0x7f183c06cda0 msgr2=0x7f183c06e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:30.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.163+0000 7f18447b8640 1 -- 172.21.15.98:0/789221388 shutdown_connections 2024-09-19T08:38:30.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:30.163+0000 7f18447b8640 1 -- 172.21.15.98:0/789221388 wait complete. 2024-09-19T08:38:30.182 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509444 for osd.2 2024-09-19T08:38:30.183 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509444 for osd.0 2024-09-19T08:38:30.941 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-19T08:38:30.960 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-19T08:38:31.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.067+0000 7f685ef36640 1 Processor -- start 2024-09-19T08:38:31.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.067+0000 7f685ef36640 1 -- start start 2024-09-19T08:38:31.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.071+0000 7f685ef36640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f68580cef60 0x7f6858071520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.071+0000 7f685ef36640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68580cffe0 con 0x7f685815dee0 2024-09-19T08:38:31.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.071+0000 7f685ef36640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68580cf460 con 0x7f6858071a60 2024-09-19T08:38:31.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.071+0000 7f685ef36640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f6858071ed0 con 0x7f68580cef60 2024-09-19T08:38:31.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.071+0000 7f685ccd2640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f685815dee0 0x7f68580afaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:60616/0 (socket says 172.21.15.98:60616) 2024-09-19T08:38:31.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.071+0000 7f685ccd2640 1 -- 172.21.15.98:0/2740700614 learned_addr learned my addr 172.21.15.98:0/2740700614 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:31.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.071+0000 7f6857fff640 1 --2- 172.21.15.98:0/2740700614 >> v2:172.21.15.98:3300/0 conn(0x7f68580cef60 0x7f6858071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.071+0000 7f6857fff640 1 -- 172.21.15.98:0/2740700614 >> v1:172.21.15.139:6789/0 conn(0x7f6858071a60 legacy=0x7f68580cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:31.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.071+0000 7f6857fff640 1 -- 172.21.15.98:0/2740700614 >> v1:172.21.15.98:6789/0 conn(0x7f685815dee0 legacy=0x7f68580afaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.071+0000 7f68577fe640 1 -- 172.21.15.98:0/2740700614 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 807383541 0 0) 0x7f68580cffe0 con 0x7f685815dee0 2024-09-19T08:38:31.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.071+0000 7f6857fff640 1 -- 172.21.15.98:0/2740700614 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68580cf740 con 0x7f68580cef60 2024-09-19T08:38:31.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.071+0000 7f6857fff640 1 --2- 172.21.15.98:0/2740700614 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68580cef60 0x7f6858071520 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f68440097c0 tx=0x7f684401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66141abdcb80643a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:31.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.071+0000 7f68577fe640 1 -- 172.21.15.98:0/2740700614 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68440b0070 con 0x7f68580cef60 2024-09-19T08:38:31.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.071+0000 7f68577fe640 1 -- 172.21.15.98:0/2740700614 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68440ab600 con 0x7f68580cef60 2024-09-19T08:38:31.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.071+0000 7f68577fe640 1 -- 172.21.15.98:0/2740700614 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68440ab9d0 con 0x7f68580cef60 2024-09-19T08:38:31.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.075+0000 7f685ef36640 1 -- 172.21.15.98:0/2740700614 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68580cef60 msgr2=0x7f6858071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.075+0000 7f685ef36640 1 --2- 172.21.15.98:0/2740700614 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68580cef60 0x7f6858071520 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f68440097c0 tx=0x7f684401c460 comp rx=0 tx=0).stop 2024-09-19T08:38:31.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.075+0000 7f685ef36640 1 -- 172.21.15.98:0/2740700614 shutdown_connections 2024-09-19T08:38:31.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.075+0000 7f685ef36640 1 --2- 172.21.15.98:0/2740700614 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68580cef60 0x7f6858071520 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.075+0000 7f685ef36640 1 -- 172.21.15.98:0/2740700614 >> 172.21.15.98:0/2740700614 conn(0x7f68580d0980 msgr2=0x7f68580d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:31.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.075+0000 7f685ef36640 1 -- 172.21.15.98:0/2740700614 shutdown_connections 2024-09-19T08:38:31.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.075+0000 7f685ef36640 1 -- 172.21.15.98:0/2740700614 wait complete. 2024-09-19T08:38:31.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.075+0000 7f685ef36640 1 Processor -- start 2024-09-19T08:38:31.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.075+0000 7f685ef36640 1 -- start start 2024-09-19T08:38:31.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.075+0000 7f685ef36640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6858071a60 0x7f685806e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.075+0000 7f685ef36640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68580cef60 0x7f6858155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.075+0000 7f685ef36640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f685815dee0 0x7f68580c6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.075+0000 7f685ef36640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6858068220 con 0x7f68580cef60 2024-09-19T08:38:31.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.075+0000 7f685ef36640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f68580cc5a0 con 0x7f685815dee0 2024-09-19T08:38:31.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.075+0000 7f685ef36640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f68580683a0 con 0x7f6858071a60 2024-09-19T08:38:31.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.075+0000 7f6857fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68580cef60 0x7f6858155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.075+0000 7f6857fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68580cef60 0x7f6858155910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37964/0 (socket says 172.21.15.98:37964) 2024-09-19T08:38:31.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.075+0000 7f6857fff640 1 -- 172.21.15.98:0/1939437341 learned_addr learned my addr 172.21.15.98:0/1939437341 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:31.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.075+0000 7f685d4d3640 1 --2- 172.21.15.98:0/1939437341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f685815dee0 0x7f68580c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.075+0000 7f685ccd2640 1 --2- 172.21.15.98:0/1939437341 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6858071a60 0x7f685806e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.075+0000 7f6857fff640 1 -- 172.21.15.98:0/1939437341 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6858071a60 msgr2=0x7f685806e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.075+0000 7f6857fff640 1 --2- 172.21.15.98:0/1939437341 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6858071a60 0x7f685806e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.075+0000 7f6857fff640 1 -- 172.21.15.98:0/1939437341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f685815dee0 msgr2=0x7f68580c6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.075+0000 7f6857fff640 1 --2- 172.21.15.98:0/1939437341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f685815dee0 0x7f68580c6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.075+0000 7f6857fff640 1 -- 172.21.15.98:0/1939437341 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68580ca2b0 con 0x7f68580cef60 2024-09-19T08:38:31.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.075+0000 7f685d4d3640 1 --2- 172.21.15.98:0/1939437341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f685815dee0 0x7f68580c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:31.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.075+0000 7f6857fff640 1 --2- 172.21.15.98:0/1939437341 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68580cef60 0x7f6858155910 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6844019070 tx=0x7f684401aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:31.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.075+0000 7f68557fa640 1 -- 172.21.15.98:0/1939437341 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68440b0050 con 0x7f68580cef60 2024-09-19T08:38:31.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.075+0000 7f68557fa640 1 -- 172.21.15.98:0/1939437341 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68440bc070 con 0x7f68580cef60 2024-09-19T08:38:31.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.075+0000 7f685ef36640 1 -- 172.21.15.98:0/1939437341 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68580ca590 con 0x7f68580cef60 2024-09-19T08:38:31.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.075+0000 7f68557fa640 1 -- 172.21.15.98:0/1939437341 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68440b5a70 con 0x7f68580cef60 2024-09-19T08:38:31.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.075+0000 7f685ef36640 1 -- 172.21.15.98:0/1939437341 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68580cab50 con 0x7f68580cef60 2024-09-19T08:38:31.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.079+0000 7f685ef36640 1 -- 172.21.15.98:0/1939437341 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6858068730 con 0x7f68580cef60 2024-09-19T08:38:31.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.079+0000 7f68557fa640 1 -- 172.21.15.98:0/1939437341 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f68440ab480 con 0x7f68580cef60 2024-09-19T08:38:31.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.079+0000 7f68557fa640 1 --2- 172.21.15.98:0/1939437341 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f683c0678c0 0x7f683c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.079+0000 7f685ccd2640 1 --2- 172.21.15.98:0/1939437341 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f683c0678c0 0x7f683c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.079+0000 7f68557fa640 1 -- 172.21.15.98:0/1939437341 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6844121130 con 0x7f68580cef60 2024-09-19T08:38:31.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.079+0000 7f685ccd2640 1 --2- 172.21.15.98:0/1939437341 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f683c0678c0 0x7f683c069d80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f684c000f70 tx=0x7f684c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:31.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.083+0000 7f68557fa640 1 -- 172.21.15.98:0/1939437341 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6844016050 con 0x7f68580cef60 2024-09-19T08:38:31.086 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-19T08:38:31.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.103+0000 7f4348ae6640 1 Processor -- start 2024-09-19T08:38:31.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.103+0000 7f4348ae6640 1 -- start start 2024-09-19T08:38:31.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.103+0000 7f4348ae6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43440cec60 0x7f434415b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.103+0000 7f4348ae6640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43440af4a0 con 0x7f434406f380 2024-09-19T08:38:31.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.103+0000 7f4348ae6640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43440af780 con 0x7f434406efa0 2024-09-19T08:38:31.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.103+0000 7f4348ae6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f434415ed00 con 0x7f43440cec60 2024-09-19T08:38:31.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.103+0000 7f433bfff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f434406f380 0x7f43440ae8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:60626/0 (socket says 172.21.15.98:60626) 2024-09-19T08:38:31.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.103+0000 7f433bfff640 1 -- 172.21.15.98:0/4157837948 learned_addr learned my addr 172.21.15.98:0/4157837948 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:31.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.103+0000 7f4342d9d640 1 --2- 172.21.15.98:0/4157837948 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43440cec60 0x7f434415b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.103+0000 7f4341d9b640 1 -- 172.21.15.98:0/4157837948 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1859053367 0 0) 0x7f43440af4a0 con 0x7f434406f380 2024-09-19T08:38:31.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.103+0000 7f4341d9b640 1 -- 172.21.15.98:0/4157837948 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4330003530 con 0x7f434406f380 2024-09-19T08:38:31.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.107+0000 7f4341d9b640 1 -- 172.21.15.98:0/4157837948 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1696988396 0 0) 0x7f4330003530 con 0x7f434406f380 2024-09-19T08:38:31.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.107+0000 7f4341d9b640 1 -- 172.21.15.98:0/4157837948 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43440cec60 msgr2=0x7f434415b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:31.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.107+0000 7f4341d9b640 1 --2- 172.21.15.98:0/4157837948 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43440cec60 0x7f434415b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.107+0000 7f4341d9b640 1 -- 172.21.15.98:0/4157837948 >> v1:172.21.15.139:6789/0 conn(0x7f434406efa0 legacy=0x7f43440ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.107+0000 7f4341d9b640 1 -- 172.21.15.98:0/4157837948 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43440afa60 con 0x7f434406f380 2024-09-19T08:38:31.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.107+0000 7f4341d9b640 1 -- 172.21.15.98:0/4157837948 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f432c001680 con 0x7f434406f380 2024-09-19T08:38:31.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.107+0000 7f4341d9b640 1 -- 172.21.15.98:0/4157837948 >> v1:172.21.15.98:6789/0 conn(0x7f434406f380 legacy=0x7f43440ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.107+0000 7f4341d9b640 1 --2- 172.21.15.98:0/4157837948 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43440cec60 0x7f43300039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.107+0000 7f4341d9b640 1 --2- 172.21.15.98:0/4157837948 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4330003f30 0x7f4330004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.107+0000 7f4341d9b640 1 --2- 172.21.15.98:0/4157837948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4330004d50 0x7f43300051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.107+0000 7f4341d9b640 1 -- 172.21.15.98:0/4157837948 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f434415ed00 con 0x7f4330004d50 2024-09-19T08:38:31.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.107+0000 7f4341d9b640 1 -- 172.21.15.98:0/4157837948 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f433000bae0 con 0x7f4330003f30 2024-09-19T08:38:31.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.107+0000 7f4341d9b640 1 -- 172.21.15.98:0/4157837948 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f433000bce0 con 0x7f43440cec60 2024-09-19T08:38:31.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.107+0000 7f4342d9d640 1 --2- 172.21.15.98:0/4157837948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4330004d50 0x7f43300051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.107+0000 7f434259c640 1 --2- 172.21.15.98:0/4157837948 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43440cec60 0x7f43300039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.107+0000 7f433bfff640 1 --2- 172.21.15.98:0/4157837948 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4330003f30 0x7f4330004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.107+0000 7f4342d9d640 1 -- 172.21.15.98:0/4157837948 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43440cec60 msgr2=0x7f43300039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:31.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.107+0000 7f4342d9d640 1 --2- 172.21.15.98:0/4157837948 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43440cec60 0x7f43300039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.107+0000 7f4342d9d640 1 -- 172.21.15.98:0/4157837948 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4330003f30 msgr2=0x7f4330004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:31.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.107+0000 7f4342d9d640 1 --2- 172.21.15.98:0/4157837948 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4330003f30 0x7f4330004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.107+0000 7f4342d9d640 1 -- 172.21.15.98:0/4157837948 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4330003530 con 0x7f4330004d50 2024-09-19T08:38:31.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.107+0000 7f433bfff640 1 --2- 172.21.15.98:0/4157837948 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4330003f30 0x7f4330004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:31.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.111+0000 7f4342d9d640 1 --2- 172.21.15.98:0/4157837948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4330004d50 0x7f43300051d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f433c009f90 tx=0x7f433c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98995cfc16ec3cea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:31.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.111+0000 7f4341d9b640 1 -- 172.21.15.98:0/4157837948 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c0b2070 con 0x7f4330004d50 2024-09-19T08:38:31.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.111+0000 7f4341d9b640 1 -- 172.21.15.98:0/4157837948 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f433c0a5c50 con 0x7f4330004d50 2024-09-19T08:38:31.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.111+0000 7f4341d9b640 1 -- 172.21.15.98:0/4157837948 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c009c10 con 0x7f4330004d50 2024-09-19T08:38:31.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.111+0000 7f4348ae6640 1 -- 172.21.15.98:0/4157837948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4330004d50 msgr2=0x7f43300051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.111+0000 7f4348ae6640 1 --2- 172.21.15.98:0/4157837948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4330004d50 0x7f43300051d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f433c009f90 tx=0x7f433c01cc40 comp rx=0 tx=0).stop 2024-09-19T08:38:31.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.111+0000 7f4342d9d640 1 -- 172.21.15.98:0/4157837948 reap_dead start 2024-09-19T08:38:31.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.111+0000 7f4348ae6640 1 -- 172.21.15.98:0/4157837948 shutdown_connections 2024-09-19T08:38:31.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.111+0000 7f4348ae6640 1 -- 172.21.15.98:0/4157837948 >> 172.21.15.98:0/4157837948 conn(0x7f43440d0980 msgr2=0x7f43440d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:31.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.111+0000 7f4348ae6640 1 -- 172.21.15.98:0/4157837948 shutdown_connections 2024-09-19T08:38:31.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.111+0000 7f4348ae6640 1 -- 172.21.15.98:0/4157837948 wait complete. 2024-09-19T08:38:31.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.111+0000 7f4348ae6640 1 Processor -- start 2024-09-19T08:38:31.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.111+0000 7f4348ae6640 1 -- start start 2024-09-19T08:38:31.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.111+0000 7f4348ae6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f434414ecc0 0x7f43440b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.111+0000 7f4348ae6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f434414f0a0 0x7f43440caf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.111+0000 7f4348ae6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43440cb4c0 0x7f43440cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.111+0000 7f4348ae6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f43440af150 con 0x7f434414ecc0 2024-09-19T08:38:31.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.111+0000 7f4348ae6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f43440aefd0 con 0x7f434414f0a0 2024-09-19T08:38:31.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.111+0000 7f4348ae6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f434406e5c0 con 0x7f43440cb4c0 2024-09-19T08:38:31.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.111+0000 7f434259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f434414ecc0 0x7f43440b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.111+0000 7f434259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f434414ecc0 0x7f43440b0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37988/0 (socket says 172.21.15.98:37988) 2024-09-19T08:38:31.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.111+0000 7f434259c640 1 -- 172.21.15.98:0/2670590607 learned_addr learned my addr 172.21.15.98:0/2670590607 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:31.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.111+0000 7f433bfff640 1 --2- 172.21.15.98:0/2670590607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f434414f0a0 0x7f43440caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.111+0000 7f434259c640 1 -- 172.21.15.98:0/2670590607 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43440cb4c0 msgr2=0x7f43440cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.111+0000 7f4342d9d640 1 --2- 172.21.15.98:0/2670590607 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43440cb4c0 0x7f43440cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.111+0000 7f434259c640 1 --2- 172.21.15.98:0/2670590607 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43440cb4c0 0x7f43440cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.111+0000 7f434259c640 1 -- 172.21.15.98:0/2670590607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f434414f0a0 msgr2=0x7f43440caf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.111+0000 7f434259c640 1 --2- 172.21.15.98:0/2670590607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f434414f0a0 0x7f43440caf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.111+0000 7f434259c640 1 -- 172.21.15.98:0/2670590607 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43440b1840 con 0x7f434414ecc0 2024-09-19T08:38:31.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.111+0000 7f4342d9d640 1 --2- 172.21.15.98:0/2670590607 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43440cb4c0 0x7f43440cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:31.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.111+0000 7f434259c640 1 --2- 172.21.15.98:0/2670590607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f434414ecc0 0x7f43440b0ed0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4334010f40 tx=0x7f433400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:31.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.111+0000 7f433affd640 1 -- 172.21.15.98:0/2670590607 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4334018020 con 0x7f434414ecc0 2024-09-19T08:38:31.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.111+0000 7f433affd640 1 -- 172.21.15.98:0/2670590607 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4334005270 con 0x7f434414ecc0 2024-09-19T08:38:31.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.111+0000 7f4348ae6640 1 -- 172.21.15.98:0/2670590607 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43440b1b20 con 0x7f434414ecc0 2024-09-19T08:38:31.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.115+0000 7f433affd640 1 -- 172.21.15.98:0/2670590607 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433400ee50 con 0x7f434414ecc0 2024-09-19T08:38:31.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.115+0000 7f4348ae6640 1 -- 172.21.15.98:0/2670590607 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43440b1410 con 0x7f434414ecc0 2024-09-19T08:38:31.116 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-19T08:38:31.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.115+0000 7f4348ae6640 1 -- 172.21.15.98:0/2670590607 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43440cb8d0 con 0x7f434414ecc0 2024-09-19T08:38:31.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.115+0000 7f433affd640 1 -- 172.21.15.98:0/2670590607 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f433401b020 con 0x7f434414ecc0 2024-09-19T08:38:31.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.115+0000 7f433affd640 1 --2- 172.21.15.98:0/2670590607 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f43240678f0 0x7f4324069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.115+0000 7f433affd640 1 -- 172.21.15.98:0/2670590607 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4334089420 con 0x7f434414ecc0 2024-09-19T08:38:31.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.115+0000 7f433bfff640 1 --2- 172.21.15.98:0/2670590607 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f43240678f0 0x7f4324069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.115+0000 7f433bfff640 1 --2- 172.21.15.98:0/2670590607 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f43240678f0 0x7f4324069db0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f43440ce470 tx=0x7f432c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:31.118 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-19T08:38:31.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.119+0000 7f433affd640 1 -- 172.21.15.98:0/2670590607 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f43440cb8d0 con 0x7f434414ecc0 2024-09-19T08:38:31.123 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-19T08:38:31.183 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-19T08:38:31.184 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-19T08:38:31.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.239+0000 7f338fc67640 1 Processor -- start 2024-09-19T08:38:31.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.239+0000 7f338fc67640 1 -- start start 2024-09-19T08:38:31.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.239+0000 7f338fc67640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f338815e110 0x7f338815e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.239+0000 7f338fc67640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f33880b4220 0x7f338815ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.239+0000 7f338fc67640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33880b4800 con 0x7f338806f3e0 2024-09-19T08:38:31.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.239+0000 7f338fc67640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f3388073e10 con 0x7f338815e110 2024-09-19T08:38:31.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.239+0000 7f338fc67640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f33880b4ae0 con 0x7f33880b4220 2024-09-19T08:38:31.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.239+0000 7f338d202640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f338806f3e0 0x7f33880cd490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:60636/0 (socket says 172.21.15.98:60636) 2024-09-19T08:38:31.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.239+0000 7f338d202640 1 -- 172.21.15.98:0/1058414279 learned_addr learned my addr 172.21.15.98:0/1058414279 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:31.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.239+0000 7f338da03640 1 --2- 172.21.15.98:0/1058414279 >> v2:172.21.15.98:3300/0 conn(0x7f338815e110 0x7f338815e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.239+0000 7f338e204640 1 --2- 172.21.15.98:0/1058414279 >> v2:172.21.15.139:3300/0 conn(0x7f33880b4220 0x7f338815ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.243+0000 7f338e204640 1 -- 172.21.15.98:0/1058414279 >> v2:172.21.15.98:3300/0 conn(0x7f338815e110 msgr2=0x7f338815e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:31.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.243+0000 7fa17a575640 1 Processor -- start 2024-09-19T08:38:31.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.243+0000 7fa17a575640 1 -- start start 2024-09-19T08:38:31.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.243+0000 7fa17a575640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa17415e110 0x7fa17415e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.243+0000 7fa17a575640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1740b4800 con 0x7fa17415ea30 2024-09-19T08:38:31.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.243+0000 7fa17a575640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1740b4ae0 con 0x7fa17415a4a0 2024-09-19T08:38:31.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.243+0000 7fa17a575640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fa174073e10 con 0x7fa17415e110 2024-09-19T08:38:31.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.243+0000 7f338e204640 1 --2- 172.21.15.98:0/1058414279 >> v2:172.21.15.98:3300/0 conn(0x7f338815e110 0x7f338815e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.243+0000 7f338e204640 1 -- 172.21.15.98:0/1058414279 >> v1:172.21.15.98:6789/0 conn(0x7f338806f3e0 legacy=0x7f33880cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.243+0000 7f338e204640 1 -- 172.21.15.98:0/1058414279 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33880b4dc0 con 0x7f33880b4220 2024-09-19T08:38:31.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.243+0000 7f338da03640 1 --2- 172.21.15.98:0/1058414279 >> v2:172.21.15.98:3300/0 conn(0x7f338815e110 0x7f338815e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:38:31.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.243+0000 7fa173fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa17415e110 0x7fa17415e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.243+0000 7fa173fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa17415e110 0x7fa17415e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:50068/0 (socket says 172.21.15.98:50068) 2024-09-19T08:38:31.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.243+0000 7fa173fff640 1 -- 172.21.15.98:0/413596816 learned_addr learned my addr 172.21.15.98:0/413596816 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:31.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.243+0000 7fa173fff640 1 -- 172.21.15.98:0/413596816 >> v1:172.21.15.139:6789/0 conn(0x7fa17415a4a0 legacy=0x7fa1740b3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.243+0000 7fa173fff640 1 -- 172.21.15.98:0/413596816 >> v1:172.21.15.98:6789/0 conn(0x7fa17415ea30 legacy=0x7fa174159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:31.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.243+0000 7f338e204640 1 --2- 172.21.15.98:0/1058414279 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33880b4220 0x7f338815ea30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f33880cf200 tx=0x7f337801d560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=374d5a1e117a22b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:31.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.247+0000 7f338ca01640 1 -- 172.21.15.98:0/1058414279 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33780a9620 con 0x7f33880b4220 2024-09-19T08:38:31.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.247+0000 7f338ca01640 1 -- 172.21.15.98:0/1058414279 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33780a97c0 con 0x7f33880b4220 2024-09-19T08:38:31.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.247+0000 7f338fc67640 1 -- 172.21.15.98:0/1058414279 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33880b4220 msgr2=0x7f338815ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.247+0000 7f338fc67640 1 --2- 172.21.15.98:0/1058414279 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33880b4220 0x7f338815ea30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f33880cf200 tx=0x7f337801d560 comp rx=0 tx=0).stop 2024-09-19T08:38:31.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.247+0000 7f338fc67640 1 -- 172.21.15.98:0/1058414279 shutdown_connections 2024-09-19T08:38:31.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.247+0000 7f338fc67640 1 --2- 172.21.15.98:0/1058414279 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33880b4220 0x7f338815ea30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.247+0000 7f338fc67640 1 --2- 172.21.15.98:0/1058414279 >> v2:172.21.15.98:3300/0 conn(0x7f338815e110 0x7f338815e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.247+0000 7f338fc67640 1 -- 172.21.15.98:0/1058414279 >> 172.21.15.98:0/1058414279 conn(0x7f338806cda0 msgr2=0x7f338806d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:31.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.243+0000 7fa173fff640 1 -- 172.21.15.98:0/413596816 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1740b4dc0 con 0x7fa17415e110 2024-09-19T08:38:31.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.247+0000 7f338fc67640 1 -- 172.21.15.98:0/1058414279 shutdown_connections 2024-09-19T08:38:31.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.247+0000 7fa173fff640 1 --2- 172.21.15.98:0/413596816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa17415e110 0x7fa17415e4f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa16c00d370 tx=0x7fa16c01ed90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f02f51ada4a4ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:31.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.247+0000 7f338fc67640 1 -- 172.21.15.98:0/1058414279 wait complete. 2024-09-19T08:38:31.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.251+0000 7f338fc67640 1 Processor -- start 2024-09-19T08:38:31.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.251+0000 7fa172ffd640 1 -- 172.21.15.98:0/413596816 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c0ad070 con 0x7fa17415e110 2024-09-19T08:38:31.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.251+0000 7fa172ffd640 1 -- 172.21.15.98:0/413596816 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa16c0a7d70 con 0x7fa17415e110 2024-09-19T08:38:31.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.251+0000 7fa172ffd640 1 -- 172.21.15.98:0/413596816 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c00be60 con 0x7fa17415e110 2024-09-19T08:38:31.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.251+0000 7f338fc67640 1 -- start start 2024-09-19T08:38:31.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.251+0000 7f338fc67640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f338806f3e0 0x7f338815a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.251+0000 7f338fc67640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33880b4220 0x7f338814fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.251+0000 7f338fc67640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f338815e110 0x7f3388203fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.251+0000 7f338d202640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33880b4220 0x7f338814fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.251+0000 7f338d202640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33880b4220 0x7f338814fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:38072/0 (socket says 172.21.15.98:38072) 2024-09-19T08:38:31.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.251+0000 7fa17a575640 1 -- 172.21.15.98:0/413596816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa17415e110 msgr2=0x7fa17415e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.251+0000 7fa17a575640 1 --2- 172.21.15.98:0/413596816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa17415e110 0x7fa17415e4f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa16c00d370 tx=0x7fa16c01ed90 comp rx=0 tx=0).stop 2024-09-19T08:38:31.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.251+0000 7f338e204640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f338815e110 0x7f3388203fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.251+0000 7f338e204640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f338815e110 0x7f3388203fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38018/0 (socket says 172.21.15.98:38018) 2024-09-19T08:38:31.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.251+0000 7f338fc67640 1 -- 172.21.15.98:0/3696129206 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3388159d90 con 0x7f338815e110 2024-09-19T08:38:31.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.251+0000 7f338d202640 1 -- 172.21.15.98:0/3696129206 learned_addr learned my addr 172.21.15.98:0/3696129206 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:31.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.251+0000 7f338fc67640 1 -- 172.21.15.98:0/3696129206 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f33880ccf70 con 0x7f338806f3e0 2024-09-19T08:38:31.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.251+0000 7f338fc67640 1 -- 172.21.15.98:0/3696129206 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3388159f10 con 0x7f33880b4220 2024-09-19T08:38:31.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.251+0000 7fa17a575640 1 -- 172.21.15.98:0/413596816 shutdown_connections 2024-09-19T08:38:31.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.251+0000 7fa17a575640 1 --2- 172.21.15.98:0/413596816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa17415e110 0x7fa17415e4f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.251+0000 7fa17a575640 1 -- 172.21.15.98:0/413596816 >> 172.21.15.98:0/413596816 conn(0x7fa17406cda0 msgr2=0x7fa17406d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:31.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.251+0000 7fa17a575640 1 -- 172.21.15.98:0/413596816 shutdown_connections 2024-09-19T08:38:31.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.255+0000 7f338e204640 1 -- 172.21.15.98:0/3696129206 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33880b4220 msgr2=0x7f338814fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.255+0000 7f338e204640 1 --2- 172.21.15.98:0/3696129206 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33880b4220 0x7f338814fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.255+0000 7f338e204640 1 -- 172.21.15.98:0/3696129206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f338806f3e0 msgr2=0x7f338815a4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T08:38:31.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.255+0000 7f338e204640 1 --2- 172.21.15.98:0/3696129206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f338806f3e0 0x7f338815a4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.255+0000 7f338e204640 1 -- 172.21.15.98:0/3696129206 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f338815b950 con 0x7f338815e110 2024-09-19T08:38:31.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.255+0000 7f338e204640 1 --2- 172.21.15.98:0/3696129206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f338815e110 0x7f3388203fb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f33780ad000 tx=0x7f337801a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:31.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.255+0000 7fa17a575640 1 -- 172.21.15.98:0/413596816 wait complete. 2024-09-19T08:38:31.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.255+0000 7fa17a575640 1 Processor -- start 2024-09-19T08:38:31.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.255+0000 7fa17a575640 1 -- start start 2024-09-19T08:38:31.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.255+0000 7fa17a575640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa17415a4a0 0x7fa174072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.255+0000 7fa17a575640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa17415e110 0x7fa174146e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.259+0000 7fa17a575640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa17415ea30 0x7fa1740ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.259+0000 7fa17a575640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa1740b44c0 con 0x7fa17415e110 2024-09-19T08:38:31.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.259+0000 7fa17a575640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa1740b4340 con 0x7fa17415a4a0 2024-09-19T08:38:31.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.259+0000 7fa17a575640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa174168050 con 0x7fa17415ea30 2024-09-19T08:38:31.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.259+0000 7fa1737fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa17415e110 0x7fa174146e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.259+0000 7fa1737fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa17415e110 0x7fa174146e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38020/0 (socket says 172.21.15.98:38020) 2024-09-19T08:38:31.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.259+0000 7fa1737fe640 1 -- 172.21.15.98:0/1935546833 learned_addr learned my addr 172.21.15.98:0/1935546833 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:31.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.259+0000 7fa173fff640 1 --2- 172.21.15.98:0/1935546833 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa17415a4a0 0x7fa174072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.259+0000 7fa178b12640 1 --2- 172.21.15.98:0/1935546833 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa17415ea30 0x7fa1740ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.259+0000 7fa1737fe640 1 -- 172.21.15.98:0/1935546833 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa17415ea30 msgr2=0x7fa1740ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.259+0000 7fa1737fe640 1 --2- 172.21.15.98:0/1935546833 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa17415ea30 0x7fa1740ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.259+0000 7fa1737fe640 1 -- 172.21.15.98:0/1935546833 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa17415a4a0 msgr2=0x7fa174072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.259+0000 7fa1737fe640 1 --2- 172.21.15.98:0/1935546833 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa17415a4a0 0x7fa174072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.259+0000 7fa1737fe640 1 -- 172.21.15.98:0/1935546833 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa174073470 con 0x7fa17415e110 2024-09-19T08:38:31.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.259+0000 7fa1737fe640 1 --2- 172.21.15.98:0/1935546833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa17415e110 0x7fa174146e30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa16800b410 tx=0x7fa16800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:31.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.263+0000 7f337e7fc640 1 -- 172.21.15.98:0/3696129206 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33780b1cc0 con 0x7f338815e110 2024-09-19T08:38:31.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.263+0000 7f338fc67640 1 -- 172.21.15.98:0/3696129206 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f338815bc30 con 0x7f338815e110 2024-09-19T08:38:31.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.263+0000 7f338fc67640 1 -- 172.21.15.98:0/3696129206 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3388150000 con 0x7f338815e110 2024-09-19T08:38:31.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.263+0000 7f338fc67640 1 -- 172.21.15.98:0/3696129206 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f338815a2d0 con 0x7f338815e110 2024-09-19T08:38:31.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.267+0000 7f337e7fc640 1 -- 172.21.15.98:0/3696129206 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33780ba420 con 0x7f338815e110 2024-09-19T08:38:31.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.271+0000 7f337e7fc640 1 -- 172.21.15.98:0/3696129206 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33780b1e60 con 0x7f338815e110 2024-09-19T08:38:31.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.271+0000 7f337e7fc640 1 -- 172.21.15.98:0/3696129206 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f33780a73c0 con 0x7f338815e110 2024-09-19T08:38:31.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.271+0000 7f337e7fc640 1 --2- 172.21.15.98:0/3696129206 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f336c0679c0 0x7f336c069e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.271+0000 7f337e7fc640 1 -- 172.21.15.98:0/3696129206 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3378120790 con 0x7f338815e110 2024-09-19T08:38:31.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.271+0000 7fa170ff9640 1 -- 172.21.15.98:0/1935546833 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa168015030 con 0x7fa17415e110 2024-09-19T08:38:31.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.271+0000 7fa170ff9640 1 -- 172.21.15.98:0/1935546833 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa168004020 con 0x7fa17415e110 2024-09-19T08:38:31.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.271+0000 7f338da03640 1 --2- 172.21.15.98:0/3696129206 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f336c0679c0 0x7f336c069e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.271+0000 7f338da03640 1 --2- 172.21.15.98:0/3696129206 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f336c0679c0 0x7f336c069e80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f33880b1e20 tx=0x7f3384009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:31.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.271+0000 7fa170ff9640 1 -- 172.21.15.98:0/1935546833 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa168004450 con 0x7fa17415e110 2024-09-19T08:38:31.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.271+0000 7fa17a575640 1 -- 172.21.15.98:0/1935546833 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa174147370 con 0x7fa17415e110 2024-09-19T08:38:31.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.271+0000 7fa17a575640 1 -- 172.21.15.98:0/1935546833 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa174151b90 con 0x7fa17415e110 2024-09-19T08:38:31.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.275+0000 7fa17a575640 1 -- 172.21.15.98:0/1935546833 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa17415e4f0 con 0x7fa17415e110 2024-09-19T08:38:31.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.275+0000 7f337e7fc640 1 -- 172.21.15.98:0/3696129206 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f33780c0060 con 0x7f338815e110 2024-09-19T08:38:31.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.283+0000 7fa170ff9640 1 -- 172.21.15.98:0/1935546833 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa168018070 con 0x7fa17415e110 2024-09-19T08:38:31.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.283+0000 7fa170ff9640 1 --2- 172.21.15.98:0/1935546833 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa160067990 0x7fa160069e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.283+0000 7fa170ff9640 1 -- 172.21.15.98:0/1935546833 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa168085de0 con 0x7fa17415e110 2024-09-19T08:38:31.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.283+0000 7fa173fff640 1 --2- 172.21.15.98:0/1935546833 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa160067990 0x7fa160069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.283+0000 7fa170ff9640 1 -- 172.21.15.98:0/1935546833 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa168086200 con 0x7fa17415e110 2024-09-19T08:38:31.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.283+0000 7fa173fff640 1 --2- 172.21.15.98:0/1935546833 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa160067990 0x7fa160069e50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0b2000 tx=0x7fa16c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:31.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.295+0000 7ff707414640 1 Processor -- start 2024-09-19T08:38:31.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.295+0000 7ff707414640 1 -- start start 2024-09-19T08:38:31.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.295+0000 7ff707414640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7000ce5e0 0x7ff7000ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.295+0000 7ff707414640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7000cdae0 con 0x7ff7001687d0 2024-09-19T08:38:31.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.295+0000 7ff707414640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff70016c710 con 0x7ff700168bb0 2024-09-19T08:38:31.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.295+0000 7ff707414640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff70016c9f0 con 0x7ff7000ce5e0 2024-09-19T08:38:31.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.295+0000 7ff7051b0640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7ff7001687d0 0x7ff7000cded0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:60652/0 (socket says 172.21.15.98:60652) 2024-09-19T08:38:31.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.295+0000 7ff7051b0640 1 -- 172.21.15.98:0/1187694082 learned_addr learned my addr 172.21.15.98:0/1187694082 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:31.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.295+0000 7ff7049af640 1 --2- 172.21.15.98:0/1187694082 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7000ce5e0 0x7ff7000ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.295+0000 7ff6f7fff640 1 -- 172.21.15.98:0/1187694082 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1374050246 0 0) 0x7ff7000cdae0 con 0x7ff7001687d0 2024-09-19T08:38:31.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.295+0000 7ff6f7fff640 1 -- 172.21.15.98:0/1187694082 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff6ec003510 con 0x7ff7001687d0 2024-09-19T08:38:31.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.295+0000 7ff6f7fff640 1 -- 172.21.15.98:0/1187694082 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 925877625 0 0) 0x7ff70016c710 con 0x7ff700168bb0 2024-09-19T08:38:31.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.295+0000 7ff6f7fff640 1 -- 172.21.15.98:0/1187694082 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff7000cdae0 con 0x7ff700168bb0 2024-09-19T08:38:31.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.295+0000 7ff7049af640 1 -- 172.21.15.98:0/1187694082 >> v1:172.21.15.139:6789/0 conn(0x7ff700168bb0 legacy=0x7ff7000cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.295+0000 7ff7049af640 1 -- 172.21.15.98:0/1187694082 >> v1:172.21.15.98:6789/0 conn(0x7ff7001687d0 legacy=0x7ff7000cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.295+0000 7ff7049af640 1 -- 172.21.15.98:0/1187694082 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff70016ccd0 con 0x7ff7000ce5e0 2024-09-19T08:38:31.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.299+0000 7ff7049af640 1 --2- 172.21.15.98:0/1187694082 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7000ce5e0 0x7ff7000ccdb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc009880 tx=0x7ff6fc020580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=212de23d9209839c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:31.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.299+0000 7ff6f7fff640 1 -- 172.21.15.98:0/1187694082 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f8001980 con 0x7ff7000ce5e0 2024-09-19T08:38:31.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.299+0000 7ff6f7fff640 1 -- 172.21.15.98:0/1187694082 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6fc009030 con 0x7ff7000ce5e0 2024-09-19T08:38:31.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.303+0000 7ff707414640 1 -- 172.21.15.98:0/1187694082 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7000ce5e0 msgr2=0x7ff7000ccdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.303+0000 7ff707414640 1 --2- 172.21.15.98:0/1187694082 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7000ce5e0 0x7ff7000ccdb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc009880 tx=0x7ff6fc020580 comp rx=0 tx=0).stop 2024-09-19T08:38:31.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.303+0000 7ff707414640 1 -- 172.21.15.98:0/1187694082 shutdown_connections 2024-09-19T08:38:31.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.303+0000 7ff707414640 1 --2- 172.21.15.98:0/1187694082 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7000ce5e0 0x7ff7000ccdb0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.303+0000 7ff707414640 1 -- 172.21.15.98:0/1187694082 >> 172.21.15.98:0/1187694082 conn(0x7ff70006cda0 msgr2=0x7ff70006d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:31.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.307+0000 7ff707414640 1 -- 172.21.15.98:0/1187694082 shutdown_connections 2024-09-19T08:38:31.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.307+0000 7f685ef36640 1 -- 172.21.15.98:0/1939437341 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f6858069220 con 0x7f68580cef60 2024-09-19T08:38:31.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.307+0000 7ff707414640 1 -- 172.21.15.98:0/1187694082 wait complete. 2024-09-19T08:38:31.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.307+0000 7ff707414640 1 Processor -- start 2024-09-19T08:38:31.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.307+0000 7f68557fa640 1 -- 172.21.15.98:0/1939437341 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6858069220 con 0x7f68580cef60 2024-09-19T08:38:31.308 INFO:teuthology.orchestra.run.smithi098.stdout:60129542151 2024-09-19T08:38:31.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.307+0000 7ff707414640 1 -- start start 2024-09-19T08:38:31.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.307+0000 7ff707414640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7000ce5e0 0x7ff700161d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.307+0000 7ff707414640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7001687d0 0x7ff70015d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.307+0000 7ff707414640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff700168bb0 0x7ff7000898d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.307+0000 7ff707414640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff700178b90 con 0x7ff7000ce5e0 2024-09-19T08:38:31.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.307+0000 7ff707414640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff700178a10 con 0x7ff700168bb0 2024-09-19T08:38:31.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.307+0000 7ff707414640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff700178d10 con 0x7ff7001687d0 2024-09-19T08:38:31.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.311+0000 7ff7051b0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7000ce5e0 0x7ff700161d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.311+0000 7ff7051b0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7000ce5e0 0x7ff700161d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38034/0 (socket says 172.21.15.98:38034) 2024-09-19T08:38:31.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.311+0000 7ff7051b0640 1 -- 172.21.15.98:0/1595878631 learned_addr learned my addr 172.21.15.98:0/1595878631 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:31.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.311+0000 7ff7049af640 1 --2- 172.21.15.98:0/1595878631 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7001687d0 0x7ff70015d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.311+0000 7ff7059b1640 1 --2- 172.21.15.98:0/1595878631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff700168bb0 0x7ff7000898d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.311+0000 7ff7051b0640 1 -- 172.21.15.98:0/1595878631 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7001687d0 msgr2=0x7ff70015d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.311+0000 7ff7051b0640 1 --2- 172.21.15.98:0/1595878631 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7001687d0 0x7ff70015d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.311+0000 7ff7051b0640 1 -- 172.21.15.98:0/1595878631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff700168bb0 msgr2=0x7ff7000898d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.311+0000 7ff7051b0640 1 --2- 172.21.15.98:0/1595878631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff700168bb0 0x7ff7000898d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.311+0000 7ff7051b0640 1 -- 172.21.15.98:0/1595878631 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7001622c0 con 0x7ff7000ce5e0 2024-09-19T08:38:31.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.311+0000 7ff7051b0640 1 --2- 172.21.15.98:0/1595878631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7000ce5e0 0x7ff700161d80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff6f800cb10 tx=0x7ff6f800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:31.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.311+0000 7ff6f5ffb640 1 -- 172.21.15.98:0/1595878631 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f8018070 con 0x7ff7000ce5e0 2024-09-19T08:38:31.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.311+0000 7ff6f5ffb640 1 -- 172.21.15.98:0/1595878631 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6f800a860 con 0x7ff7000ce5e0 2024-09-19T08:38:31.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.311+0000 7ff707414640 1 -- 172.21.15.98:0/1595878631 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff70015d8e0 con 0x7ff7000ce5e0 2024-09-19T08:38:31.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.311+0000 7ff6f5ffb640 1 -- 172.21.15.98:0/1595878631 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f800ac50 con 0x7ff7000ce5e0 2024-09-19T08:38:31.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.311+0000 7ff707414640 1 -- 172.21.15.98:0/1595878631 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff70015dea0 con 0x7ff7000ce5e0 2024-09-19T08:38:31.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.311+0000 7ff707414640 1 -- 172.21.15.98:0/1595878631 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7000cded0 con 0x7ff7000ce5e0 2024-09-19T08:38:31.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.315+0000 7ff6f5ffb640 1 -- 172.21.15.98:0/1595878631 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff6f801b070 con 0x7ff7000ce5e0 2024-09-19T08:38:31.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.315+0000 7ff6f5ffb640 1 --2- 172.21.15.98:0/1595878631 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff6e4067910 0x7ff6e4069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.315+0000 7ff6f5ffb640 1 -- 172.21.15.98:0/1595878631 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff6f8085600 con 0x7ff7000ce5e0 2024-09-19T08:38:31.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.315+0000 7ff7049af640 1 --2- 172.21.15.98:0/1595878631 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff6e4067910 0x7ff6e4069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.315+0000 7ff7049af640 1 --2- 172.21.15.98:0/1595878631 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff6e4067910 0x7ff6e4069dd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc0b1000 tx=0x7ff6fc0ab000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:31.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.319+0000 7f685ef36640 1 -- 172.21.15.98:0/1939437341 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f683c0678c0 msgr2=0x7f683c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.319+0000 7f685ef36640 1 --2- 172.21.15.98:0/1939437341 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f683c0678c0 0x7f683c069d80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f684c000f70 tx=0x7f684c009040 comp rx=0 tx=0).stop 2024-09-19T08:38:31.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.319+0000 7f685ef36640 1 -- 172.21.15.98:0/1939437341 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68580cef60 msgr2=0x7f6858155910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.319+0000 7f685ef36640 1 --2- 172.21.15.98:0/1939437341 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68580cef60 0x7f6858155910 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6844019070 tx=0x7f684401aca0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.319+0000 7f685ef36640 1 -- 172.21.15.98:0/1939437341 shutdown_connections 2024-09-19T08:38:31.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.319+0000 7f685ef36640 1 --2- 172.21.15.98:0/1939437341 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f683c0678c0 0x7f683c069d80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.319+0000 7f685ef36640 1 --2- 172.21.15.98:0/1939437341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f685815dee0 0x7f68580c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.319+0000 7f685ef36640 1 --2- 172.21.15.98:0/1939437341 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f68580cef60 0x7f6858155910 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.319+0000 7f685ef36640 1 --2- 172.21.15.98:0/1939437341 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6858071a60 0x7f685806e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.319+0000 7f685ef36640 1 -- 172.21.15.98:0/1939437341 >> 172.21.15.98:0/1939437341 conn(0x7f68580d0980 msgr2=0x7f68580b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:31.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.319+0000 7f685ef36640 1 -- 172.21.15.98:0/1939437341 shutdown_connections 2024-09-19T08:38:31.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.319+0000 7f685ef36640 1 -- 172.21.15.98:0/1939437341 wait complete. 2024-09-19T08:38:31.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.323+0000 7ff6f5ffb640 1 -- 172.21.15.98:0/1595878631 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff7001622c0 con 0x7ff7000ce5e0 2024-09-19T08:38:31.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.323+0000 7f50dd981640 1 Processor -- start 2024-09-19T08:38:31.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.323+0000 7f50dd981640 1 -- start start 2024-09-19T08:38:31.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.327+0000 7f50dd981640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f50d815a220 0x7f50d80b52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.327+0000 7f50dd981640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50d815a600 0x7f50d80ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.327+0000 7f50dd981640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50d80cdb70 con 0x7f50d80b5c30 2024-09-19T08:38:31.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.327+0000 7f50dd981640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f50d80cd360 con 0x7f50d815a220 2024-09-19T08:38:31.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.327+0000 7f50dd981640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f50d80cd5b0 con 0x7f50d815a600 2024-09-19T08:38:31.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.327+0000 7f50d6ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f50d815a220 0x7f50d80b52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.327+0000 7f50d67fc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50d815a600 0x7f50d80ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.327+0000 7f50d6ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f50d815a220 0x7f50d80b52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:50102/0 (socket says 172.21.15.98:50102) 2024-09-19T08:38:31.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.327+0000 7f50d67fc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50d815a600 0x7f50d80ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:38116/0 (socket says 172.21.15.98:38116) 2024-09-19T08:38:31.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.327+0000 7f50d6ffd640 1 -- 172.21.15.98:0/289043326 learned_addr learned my addr 172.21.15.98:0/289043326 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:31.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.327+0000 7f50d67fc640 1 -- 172.21.15.98:0/289043326 >> v2:172.21.15.139:3300/0 conn(0x7f50d815a220 msgr2=0x7f50d80b52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:31.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.327+0000 7f50d67fc640 1 --2- 172.21.15.98:0/289043326 >> v2:172.21.15.139:3300/0 conn(0x7f50d815a220 0x7f50d80b52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.327+0000 7f50d67fc640 1 -- 172.21.15.98:0/289043326 >> v1:172.21.15.98:6789/0 conn(0x7f50d80b5c30 legacy=0x7f50d815dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:31.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.327+0000 7f50d67fc640 1 -- 172.21.15.98:0/289043326 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d80cd890 con 0x7f50d815a600 2024-09-19T08:38:31.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.331+0000 7f50d67fc640 1 --2- 172.21.15.98:0/289043326 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50d815a600 0x7f50d80ce410 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f50c800b150 tx=0x7f50c801cb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40c908f219b1b501 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:31.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.331+0000 7f50d5ffb640 1 -- 172.21.15.98:0/289043326 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c80a5ae0 con 0x7f50d815a600 2024-09-19T08:38:31.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.331+0000 7f50d5ffb640 1 -- 172.21.15.98:0/289043326 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50c80a5c80 con 0x7f50d815a600 2024-09-19T08:38:31.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.331+0000 7f50d5ffb640 1 -- 172.21.15.98:0/289043326 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c8007af0 con 0x7f50d815a600 2024-09-19T08:38:31.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.331+0000 7f50dd981640 1 -- 172.21.15.98:0/289043326 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50d815a600 msgr2=0x7f50d80ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.331+0000 7f50dd981640 1 --2- 172.21.15.98:0/289043326 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50d815a600 0x7f50d80ce410 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f50c800b150 tx=0x7f50c801cb60 comp rx=0 tx=0).stop 2024-09-19T08:38:31.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.331+0000 7f50dd981640 1 -- 172.21.15.98:0/289043326 shutdown_connections 2024-09-19T08:38:31.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.331+0000 7f50dd981640 1 --2- 172.21.15.98:0/289043326 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50d815a600 0x7f50d80ce410 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.331+0000 7f50dd981640 1 --2- 172.21.15.98:0/289043326 >> v2:172.21.15.139:3300/0 conn(0x7f50d815a220 0x7f50d80b52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.331+0000 7f50dd981640 1 -- 172.21.15.98:0/289043326 >> 172.21.15.98:0/289043326 conn(0x7f50d80b1f10 msgr2=0x7f50d80b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:31.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.331+0000 7f50dd981640 1 -- 172.21.15.98:0/289043326 shutdown_connections 2024-09-19T08:38:31.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.331+0000 7f50dd981640 1 -- 172.21.15.98:0/289043326 wait complete. 2024-09-19T08:38:31.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.335+0000 7f50dd981640 1 Processor -- start 2024-09-19T08:38:31.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.335+0000 7f50dd981640 1 -- start start 2024-09-19T08:38:31.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.335+0000 7f50dd981640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50d80b5c30 0x7f50d81451b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.335+0000 7f50dd981640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50d815a220 0x7f50d814ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.335+0000 7f50d6ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50d80b5c30 0x7f50d81451b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.335+0000 7f50dd981640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50d815a600 0x7f50d814f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.335+0000 7f50d6ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50d80b5c30 0x7f50d81451b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38046/0 (socket says 172.21.15.98:38046) 2024-09-19T08:38:31.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.335+0000 7f50dd981640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f50d8071200 con 0x7f50d80b5c30 2024-09-19T08:38:31.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.335+0000 7f50dd981640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f50d80ced60 con 0x7f50d815a220 2024-09-19T08:38:31.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.335+0000 7f50dd981640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f50d8071380 con 0x7f50d815a600 2024-09-19T08:38:31.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.335+0000 7f50d6ffd640 1 -- 172.21.15.98:0/2338269541 learned_addr learned my addr 172.21.15.98:0/2338269541 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:31.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.335+0000 7f50d67fc640 1 --2- 172.21.15.98:0/2338269541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50d815a220 0x7f50d814ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.335+0000 7f50d77fe640 1 --2- 172.21.15.98:0/2338269541 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50d815a600 0x7f50d814f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.335+0000 7f50d6ffd640 1 -- 172.21.15.98:0/2338269541 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50d815a600 msgr2=0x7f50d814f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.335+0000 7f50d6ffd640 1 --2- 172.21.15.98:0/2338269541 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50d815a600 0x7f50d814f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.335+0000 7f50d6ffd640 1 -- 172.21.15.98:0/2338269541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50d815a220 msgr2=0x7f50d814ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.335+0000 7f50d6ffd640 1 --2- 172.21.15.98:0/2338269541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50d815a220 0x7f50d814ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.335+0000 7f50d6ffd640 1 -- 172.21.15.98:0/2338269541 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d8155370 con 0x7f50d80b5c30 2024-09-19T08:38:31.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.335+0000 7f50d6ffd640 1 --2- 172.21.15.98:0/2338269541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50d80b5c30 0x7f50d81451b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f50cc00e890 tx=0x7f50cc00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:31.343 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.3 2024-09-19T08:38:31.343 DEBUG:teuthology.parallel:result is None 2024-09-19T08:38:31.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.347+0000 7f50bffff640 1 -- 172.21.15.98:0/2338269541 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50cc020070 con 0x7f50d80b5c30 2024-09-19T08:38:31.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.347+0000 7f50dd981640 1 -- 172.21.15.98:0/2338269541 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50d8155650 con 0x7f50d80b5c30 2024-09-19T08:38:31.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.347+0000 7f50dd981640 1 -- 172.21.15.98:0/2338269541 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50d8155c10 con 0x7f50d80b5c30 2024-09-19T08:38:31.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.347+0000 7f50bffff640 1 -- 172.21.15.98:0/2338269541 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50cc00c940 con 0x7f50d80b5c30 2024-09-19T08:38:31.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.347+0000 7f50bffff640 1 -- 172.21.15.98:0/2338269541 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50cc00cd30 con 0x7f50d80b5c30 2024-09-19T08:38:31.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.347+0000 7f50bffff640 1 -- 172.21.15.98:0/2338269541 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f50cc023070 con 0x7f50d80b5c30 2024-09-19T08:38:31.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.347+0000 7f50dd981640 1 -- 172.21.15.98:0/2338269541 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50b80052a0 con 0x7f50d80b5c30 2024-09-19T08:38:31.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.347+0000 7f50bffff640 1 --2- 172.21.15.98:0/2338269541 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f50c4067910 0x7f50c4069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.347+0000 7f50bffff640 1 -- 172.21.15.98:0/2338269541 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f50cc089980 con 0x7f50d80b5c30 2024-09-19T08:38:31.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.347+0000 7f50d67fc640 1 --2- 172.21.15.98:0/2338269541 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f50c4067910 0x7f50c4069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.351+0000 7f50d67fc640 1 --2- 172.21.15.98:0/2338269541 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f50c4067910 0x7f50c4069dd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f50c80ae000 tx=0x7f50c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:31.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.359+0000 7f50bffff640 1 -- 172.21.15.98:0/2338269541 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f50cc059ee0 con 0x7f50d80b5c30 2024-09-19T08:38:31.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.367+0000 7fdbd153c640 1 Processor -- start 2024-09-19T08:38:31.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.367+0000 7fdbd153c640 1 -- start start 2024-09-19T08:38:31.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.367+0000 7fdbd153c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdbcc0cd350 0x7fdbcc0b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.367+0000 7fdbd153c640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbcc0cd730 con 0x7fdbcc0b6f50 2024-09-19T08:38:31.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.367+0000 7fdbd153c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbcc0b6190 con 0x7fdbcc0ce380 2024-09-19T08:38:31.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.367+0000 7fdbd153c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdbcc0b5f60 con 0x7fdbcc0cd350 2024-09-19T08:38:31.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.367+0000 7fdbca7fc640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fdbcc0b6f50 0x7fdbcc0cdc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:60662/0 (socket says 172.21.15.98:60662) 2024-09-19T08:38:31.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.371+0000 7fdbca7fc640 1 -- 172.21.15.98:0/1569760302 learned_addr learned my addr 172.21.15.98:0/1569760302 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:31.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.371+0000 7fdbcaffd640 1 --2- 172.21.15.98:0/1569760302 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdbcc0cd350 0x7fdbcc0b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.371+0000 7fdbc9ffb640 1 -- 172.21.15.98:0/1569760302 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 706407773 0 0) 0x7fdbcc0cd730 con 0x7fdbcc0b6f50 2024-09-19T08:38:31.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.371+0000 7fdbc9ffb640 1 -- 172.21.15.98:0/1569760302 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdbb4003530 con 0x7fdbcc0b6f50 2024-09-19T08:38:31.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.371+0000 7fdbc9ffb640 1 -- 172.21.15.98:0/1569760302 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3152016888 0 0) 0x7fdbcc0b6190 con 0x7fdbcc0ce380 2024-09-19T08:38:31.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.371+0000 7fdbc9ffb640 1 -- 172.21.15.98:0/1569760302 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdbcc0cd730 con 0x7fdbcc0ce380 2024-09-19T08:38:31.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.371+0000 7fdbcaffd640 1 -- 172.21.15.98:0/1569760302 >> v1:172.21.15.139:6789/0 conn(0x7fdbcc0ce380 legacy=0x7fdbcc0cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.371+0000 7fdbcaffd640 1 -- 172.21.15.98:0/1569760302 >> v1:172.21.15.98:6789/0 conn(0x7fdbcc0b6f50 legacy=0x7fdbcc0cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.371+0000 7fdbcaffd640 1 -- 172.21.15.98:0/1569760302 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbcc0b6470 con 0x7fdbcc0cd350 2024-09-19T08:38:31.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.371+0000 7fdbcaffd640 1 --2- 172.21.15.98:0/1569760302 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdbcc0cd350 0x7fdbcc0b24b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00097f0 tx=0x7fdbc001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=969529c35b7f8304 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:31.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.371+0000 7fdbc9ffb640 1 -- 172.21.15.98:0/1569760302 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc00b0070 con 0x7fdbcc0cd350 2024-09-19T08:38:31.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.371+0000 7fdbc9ffb640 1 -- 172.21.15.98:0/1569760302 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbc00ab600 con 0x7fdbcc0cd350 2024-09-19T08:38:31.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.371+0000 7fdbc9ffb640 1 -- 172.21.15.98:0/1569760302 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc00ab9d0 con 0x7fdbcc0cd350 2024-09-19T08:38:31.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.375+0000 7fdbd153c640 1 -- 172.21.15.98:0/1569760302 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdbcc0cd350 msgr2=0x7fdbcc0b24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.375+0000 7fdbd153c640 1 --2- 172.21.15.98:0/1569760302 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdbcc0cd350 0x7fdbcc0b24b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00097f0 tx=0x7fdbc001c470 comp rx=0 tx=0).stop 2024-09-19T08:38:31.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.375+0000 7fdbd153c640 1 -- 172.21.15.98:0/1569760302 shutdown_connections 2024-09-19T08:38:31.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.375+0000 7fdbd153c640 1 --2- 172.21.15.98:0/1569760302 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdbcc0cd350 0x7fdbcc0b24b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.375+0000 7fdbd153c640 1 -- 172.21.15.98:0/1569760302 >> 172.21.15.98:0/1569760302 conn(0x7fdbcc073170 msgr2=0x7fdbcc073580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:31.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.375+0000 7fdbd153c640 1 -- 172.21.15.98:0/1569760302 shutdown_connections 2024-09-19T08:38:31.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.375+0000 7fdbd153c640 1 -- 172.21.15.98:0/1569760302 wait complete. 2024-09-19T08:38:31.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.375+0000 7fdbd153c640 1 Processor -- start 2024-09-19T08:38:31.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.375+0000 7fdbd153c640 1 -- start start 2024-09-19T08:38:31.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.375+0000 7fdbd153c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbcc0b6f50 0x7fdbcc09aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.375+0000 7fdbd153c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdbcc0cd350 0x7fdbcc15dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.375+0000 7fdbd153c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdbcc0ce380 0x7fdbcc15e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.375+0000 7fdbd153c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdbcc1691a0 con 0x7fdbcc0b6f50 2024-09-19T08:38:31.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.375+0000 7fdbd153c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdbcc169020 con 0x7fdbcc0cd350 2024-09-19T08:38:31.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.375+0000 7fdbd153c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdbcc169320 con 0x7fdbcc0ce380 2024-09-19T08:38:31.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.375+0000 7fdbcaffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbcc0b6f50 0x7fdbcc09aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.375+0000 7fdbcaffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbcc0b6f50 0x7fdbcc09aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38054/0 (socket says 172.21.15.98:38054) 2024-09-19T08:38:31.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.375+0000 7fdbcaffd640 1 -- 172.21.15.98:0/441585864 learned_addr learned my addr 172.21.15.98:0/441585864 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:31.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.379+0000 7fdbca7fc640 1 --2- 172.21.15.98:0/441585864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdbcc0cd350 0x7fdbcc15dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.379+0000 7fdbcb7fe640 1 --2- 172.21.15.98:0/441585864 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdbcc0ce380 0x7fdbcc15e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.379+0000 7fdbcaffd640 1 -- 172.21.15.98:0/441585864 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdbcc0ce380 msgr2=0x7fdbcc15e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.379+0000 7fdbcaffd640 1 --2- 172.21.15.98:0/441585864 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdbcc0ce380 0x7fdbcc15e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.379+0000 7fdbcaffd640 1 -- 172.21.15.98:0/441585864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdbcc0cd350 msgr2=0x7fdbcc15dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.379+0000 7fdbcaffd640 1 --2- 172.21.15.98:0/441585864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdbcc0cd350 0x7fdbcc15dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.379+0000 7fdbcaffd640 1 -- 172.21.15.98:0/441585864 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbcc15b260 con 0x7fdbcc0b6f50 2024-09-19T08:38:31.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.379+0000 7fdbcaffd640 1 --2- 172.21.15.98:0/441585864 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbcc0b6f50 0x7fdbcc09aaf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdbc001a0a0 tx=0x7fdbc001acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:31.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.379+0000 7fdbb37fe640 1 -- 172.21.15.98:0/441585864 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc00b0050 con 0x7fdbcc0b6f50 2024-09-19T08:38:31.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.379+0000 7fdbd153c640 1 -- 172.21.15.98:0/441585864 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbcc1529e0 con 0x7fdbcc0b6f50 2024-09-19T08:38:31.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.379+0000 7fdbd153c640 1 -- 172.21.15.98:0/441585864 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbcc152fa0 con 0x7fdbcc0b6f50 2024-09-19T08:38:31.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.379+0000 7fdbb37fe640 1 -- 172.21.15.98:0/441585864 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbc00b1040 con 0x7fdbcc0b6f50 2024-09-19T08:38:31.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.379+0000 7fdbb37fe640 1 -- 172.21.15.98:0/441585864 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0006e20 con 0x7fdbcc0b6f50 2024-09-19T08:38:31.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.379+0000 7fdbd153c640 1 -- 172.21.15.98:0/441585864 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbcc0b24b0 con 0x7fdbcc0b6f50 2024-09-19T08:38:31.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.379+0000 7fdbb37fe640 1 -- 172.21.15.98:0/441585864 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdbc00be030 con 0x7fdbcc0b6f50 2024-09-19T08:38:31.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.383+0000 7fdbb37fe640 1 --2- 172.21.15.98:0/441585864 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdbb8070120 0x7fdbb80725e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.383+0000 7fdbb37fe640 1 -- 172.21.15.98:0/441585864 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fdbc011f850 con 0x7fdbcc0b6f50 2024-09-19T08:38:31.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.383+0000 7fdbca7fc640 1 --2- 172.21.15.98:0/441585864 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdbb8070120 0x7fdbb80725e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.383+0000 7fdbca7fc640 1 --2- 172.21.15.98:0/441585864 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdbb8070120 0x7fdbb80725e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc000c10 tx=0x7fdbbc007d90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:31.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.383+0000 7fb083fff640 1 Processor -- start 2024-09-19T08:38:31.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.383+0000 7fb083fff640 1 -- start start 2024-09-19T08:38:31.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.383+0000 7fb083fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb0841558f0 0x7fb08415f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.383+0000 7fb083fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb0840b0360 0x7fb08406c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.383+0000 7fb083fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb08406ca50 0x7fb0840731d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.383+0000 7fb083fff640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fb084155cd0 con 0x7fb08406ca50 2024-09-19T08:38:31.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.383+0000 7fb083fff640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fb0840b0b00 con 0x7fb0841558f0 2024-09-19T08:38:31.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.383+0000 7fb083fff640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb08415b8c0 con 0x7fb0840b0360 2024-09-19T08:38:31.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.383+0000 7fb0837fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb08406ca50 0x7fb0840731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.383+0000 7fb0837fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb08406ca50 0x7fb0840731d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38060/0 (socket says 172.21.15.98:38060) 2024-09-19T08:38:31.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.383+0000 7fb0837fe640 1 -- 172.21.15.98:0/1661398467 learned_addr learned my addr 172.21.15.98:0/1661398467 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:31.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.383+0000 7fb0827fc640 1 --2- 172.21.15.98:0/1661398467 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb0840b0360 0x7fb08406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.383+0000 7fb082ffd640 1 --2- 172.21.15.98:0/1661398467 >> v2:172.21.15.139:3300/0 conn(0x7fb0841558f0 0x7fb08415f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.383+0000 7fb0837fe640 1 -- 172.21.15.98:0/1661398467 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb0840b0360 msgr2=0x7fb08406c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:31.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.383+0000 7fb0837fe640 1 --2- 172.21.15.98:0/1661398467 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb0840b0360 0x7fb08406c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.383+0000 7fb0837fe640 1 -- 172.21.15.98:0/1661398467 >> v2:172.21.15.139:3300/0 conn(0x7fb0841558f0 msgr2=0x7fb08415f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:31.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.383+0000 7fb0837fe640 1 --2- 172.21.15.98:0/1661398467 >> v2:172.21.15.139:3300/0 conn(0x7fb0841558f0 0x7fb08415f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.383+0000 7fb0837fe640 1 -- 172.21.15.98:0/1661398467 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb08415bba0 con 0x7fb08406ca50 2024-09-19T08:38:31.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.387+0000 7fb0837fe640 1 --2- 172.21.15.98:0/1661398467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb08406ca50 0x7fb0840731d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb084070690 tx=0x7fb07c01c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ea3d1baafd6fb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:31.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.391+0000 7fdbb37fe640 1 -- 172.21.15.98:0/441585864 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fdbc00c5060 con 0x7fdbcc0b6f50 2024-09-19T08:38:31.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.391+0000 7fb081ffb640 1 -- 172.21.15.98:0/1661398467 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c0a64f0 con 0x7fb08406ca50 2024-09-19T08:38:31.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.391+0000 7fb081ffb640 1 -- 172.21.15.98:0/1661398467 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb07c0a6690 con 0x7fb08406ca50 2024-09-19T08:38:31.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.391+0000 7fb083fff640 1 -- 172.21.15.98:0/1661398467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb08406ca50 msgr2=0x7fb0840731d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.391+0000 7fb083fff640 1 --2- 172.21.15.98:0/1661398467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb08406ca50 0x7fb0840731d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb084070690 tx=0x7fb07c01c770 comp rx=0 tx=0).stop 2024-09-19T08:38:31.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.391+0000 7fb083fff640 1 -- 172.21.15.98:0/1661398467 shutdown_connections 2024-09-19T08:38:31.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.391+0000 7fb083fff640 1 --2- 172.21.15.98:0/1661398467 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb08406ca50 0x7fb0840731d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.391+0000 7fb083fff640 1 --2- 172.21.15.98:0/1661398467 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb0840b0360 0x7fb08406c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.391+0000 7fb083fff640 1 --2- 172.21.15.98:0/1661398467 >> v2:172.21.15.139:3300/0 conn(0x7fb0841558f0 0x7fb08415f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.391+0000 7fb083fff640 1 -- 172.21.15.98:0/1661398467 >> 172.21.15.98:0/1661398467 conn(0x7fb0840b2dc0 msgr2=0x7fb0840b31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:31.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.391+0000 7fb083fff640 1 -- 172.21.15.98:0/1661398467 shutdown_connections 2024-09-19T08:38:31.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.391+0000 7fb083fff640 1 -- 172.21.15.98:0/1661398467 wait complete. 2024-09-19T08:38:31.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.395+0000 7fb083fff640 1 Processor -- start 2024-09-19T08:38:31.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.395+0000 7fb083fff640 1 -- start start 2024-09-19T08:38:31.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.395+0000 7fb083fff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb08406ca50 0x7fb084151160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.395+0000 7fb083fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb0840b0360 0x7fb0840cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.395+0000 7fb083fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0841558f0 0x7fb0840d0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.395+0000 7fb083fff640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb0840b5f20 con 0x7fb0841558f0 2024-09-19T08:38:31.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.395+0000 7fb0837fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0841558f0 0x7fb0840d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.395+0000 7fb082ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb08406ca50 0x7fb084151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.395+0000 7fb0837fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0841558f0 0x7fb0840d0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38064/0 (socket says 172.21.15.98:38064) 2024-09-19T08:38:31.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.395+0000 7fb0837fe640 1 -- 172.21.15.98:0/1422991439 learned_addr learned my addr 172.21.15.98:0/1422991439 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:31.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.395+0000 7fb083fff640 1 -- 172.21.15.98:0/1422991439 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb0840b07c0 con 0x7fb08406ca50 2024-09-19T08:38:31.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.395+0000 7fb083fff640 1 -- 172.21.15.98:0/1422991439 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb0840b60a0 con 0x7fb0840b0360 2024-09-19T08:38:31.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.395+0000 7fb0837fe640 1 -- 172.21.15.98:0/1422991439 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb0840b0360 msgr2=0x7fb0840cfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.395+0000 7fb0837fe640 1 --2- 172.21.15.98:0/1422991439 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb0840b0360 0x7fb0840cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.395+0000 7fb0837fe640 1 -- 172.21.15.98:0/1422991439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb08406ca50 msgr2=0x7fb084151160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.399+0000 7fb0837fe640 1 --2- 172.21.15.98:0/1422991439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb08406ca50 0x7fb084151160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.399+0000 7fb0837fe640 1 -- 172.21.15.98:0/1422991439 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0840d0890 con 0x7fb0841558f0 2024-09-19T08:38:31.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.399+0000 7fb0837fe640 1 --2- 172.21.15.98:0/1422991439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0841558f0 0x7fb0840d0170 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb07c01ae20 tx=0x7fb07c019510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:31.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.403+0000 7fb06b7fe640 1 -- 172.21.15.98:0/1422991439 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c0a64f0 con 0x7fb0841558f0 2024-09-19T08:38:31.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.403+0000 7fb083fff640 1 -- 172.21.15.98:0/1422991439 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0840d0b70 con 0x7fb0841558f0 2024-09-19T08:38:31.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.403+0000 7fb083fff640 1 -- 172.21.15.98:0/1422991439 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0841516a0 con 0x7fb0841558f0 2024-09-19T08:38:31.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.403+0000 7f4348ae6640 1 -- 172.21.15.98:0/2670590607 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f434406fee0 con 0x7f434414ecc0 2024-09-19T08:38:31.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.403+0000 7fb06b7fe640 1 -- 172.21.15.98:0/1422991439 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb07c0a6690 con 0x7fb0841558f0 2024-09-19T08:38:31.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.403+0000 7fb06b7fe640 1 -- 172.21.15.98:0/1422991439 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c003ba0 con 0x7fb0841558f0 2024-09-19T08:38:31.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.403+0000 7fb083fff640 1 -- 172.21.15.98:0/1422991439 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0640052a0 con 0x7fb0841558f0 2024-09-19T08:38:31.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.403+0000 7f433affd640 1 -- 172.21.15.98:0/2670590607 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4334022020 con 0x7f434414ecc0 2024-09-19T08:38:31.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.403+0000 7fb06b7fe640 1 -- 172.21.15.98:0/1422991439 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb07c0a6840 con 0x7fb0841558f0 2024-09-19T08:38:31.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.407+0000 7fb06b7fe640 1 --2- 172.21.15.98:0/1422991439 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb070067910 0x7fb070069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.407+0000 7fb06b7fe640 1 -- 172.21.15.98:0/1422991439 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb07c11ff20 con 0x7fb0841558f0 2024-09-19T08:38:31.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.407+0000 7fb082ffd640 1 --2- 172.21.15.98:0/1422991439 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb070067910 0x7fb070069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.407+0000 7fb082ffd640 1 --2- 172.21.15.98:0/1422991439 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb070067910 0x7fb070069dd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb0780045f0 tx=0x7fb078008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:31.407 INFO:teuthology.orchestra.run.smithi098.stdout:64424509447 2024-09-19T08:38:31.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.411+0000 7f4348ae6640 1 -- 172.21.15.98:0/2670590607 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f43240678f0 msgr2=0x7f4324069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.411+0000 7f4348ae6640 1 --2- 172.21.15.98:0/2670590607 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f43240678f0 0x7f4324069db0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f43440ce470 tx=0x7f432c009040 comp rx=0 tx=0).stop 2024-09-19T08:38:31.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.411+0000 7f4348ae6640 1 -- 172.21.15.98:0/2670590607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f434414ecc0 msgr2=0x7f43440b0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.411+0000 7f4348ae6640 1 --2- 172.21.15.98:0/2670590607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f434414ecc0 0x7f43440b0ed0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4334010f40 tx=0x7f433400e530 comp rx=0 tx=0).stop 2024-09-19T08:38:31.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.411+0000 7f4348ae6640 1 -- 172.21.15.98:0/2670590607 shutdown_connections 2024-09-19T08:38:31.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.411+0000 7f4348ae6640 1 --2- 172.21.15.98:0/2670590607 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f43240678f0 0x7f4324069db0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.411+0000 7f4348ae6640 1 --2- 172.21.15.98:0/2670590607 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43440cb4c0 0x7f43440cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.411+0000 7f4348ae6640 1 --2- 172.21.15.98:0/2670590607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f434414f0a0 0x7f43440caf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.411+0000 7f4348ae6640 1 --2- 172.21.15.98:0/2670590607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f434414ecc0 0x7f43440b0ed0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.411+0000 7f4348ae6640 1 -- 172.21.15.98:0/2670590607 >> 172.21.15.98:0/2670590607 conn(0x7f43440d0980 msgr2=0x7f43440ae8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:31.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.411+0000 7f4348ae6640 1 -- 172.21.15.98:0/2670590607 shutdown_connections 2024-09-19T08:38:31.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.411+0000 7f4348ae6640 1 -- 172.21.15.98:0/2670590607 wait complete. 2024-09-19T08:38:31.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.415+0000 7fb06b7fe640 1 -- 172.21.15.98:0/1422991439 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb07c016620 con 0x7fb0841558f0 2024-09-19T08:38:31.435 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.7 2024-09-19T08:38:31.435 DEBUG:teuthology.parallel:result is None 2024-09-19T08:38:31.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.535+0000 7f50dd981640 1 -- 172.21.15.98:0/2338269541 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f50b8005590 con 0x7f50d80b5c30 2024-09-19T08:38:31.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.535+0000 7f50bffff640 1 -- 172.21.15.98:0/2338269541 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f50cc05e1e0 con 0x7f50d80b5c30 2024-09-19T08:38:31.535 INFO:teuthology.orchestra.run.smithi098.stdout:64424509449 2024-09-19T08:38:31.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.535+0000 7f50dd981640 1 -- 172.21.15.98:0/2338269541 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f50c4067910 msgr2=0x7f50c4069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.535+0000 7f50dd981640 1 --2- 172.21.15.98:0/2338269541 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f50c4067910 0x7f50c4069dd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f50c80ae000 tx=0x7f50c8008040 comp rx=0 tx=0).stop 2024-09-19T08:38:31.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.535+0000 7f50dd981640 1 -- 172.21.15.98:0/2338269541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50d80b5c30 msgr2=0x7f50d81451b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.535+0000 7f50dd981640 1 --2- 172.21.15.98:0/2338269541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50d80b5c30 0x7f50d81451b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f50cc00e890 tx=0x7f50cc00ed50 comp rx=0 tx=0).stop 2024-09-19T08:38:31.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.535+0000 7f50dd981640 1 -- 172.21.15.98:0/2338269541 shutdown_connections 2024-09-19T08:38:31.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.535+0000 7f50dd981640 1 --2- 172.21.15.98:0/2338269541 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f50c4067910 0x7f50c4069dd0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.535+0000 7f50dd981640 1 --2- 172.21.15.98:0/2338269541 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50d815a600 0x7f50d814f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.535+0000 7f50dd981640 1 --2- 172.21.15.98:0/2338269541 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50d815a220 0x7f50d814ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.535+0000 7f50dd981640 1 --2- 172.21.15.98:0/2338269541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50d80b5c30 0x7f50d81451b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.535+0000 7f50dd981640 1 -- 172.21.15.98:0/2338269541 >> 172.21.15.98:0/2338269541 conn(0x7f50d80b1f10 msgr2=0x7f50d80b1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:31.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.535+0000 7f50dd981640 1 -- 172.21.15.98:0/2338269541 shutdown_connections 2024-09-19T08:38:31.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.535+0000 7f50dd981640 1 -- 172.21.15.98:0/2338269541 wait complete. 2024-09-19T08:38:31.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.539+0000 7ff707414640 1 -- 172.21.15.98:0/1595878631 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7ff70016e7c0 con 0x7ff7000ce5e0 2024-09-19T08:38:31.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.539+0000 7ff6f5ffb640 1 -- 172.21.15.98:0/1595878631 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff6f8055bd0 con 0x7ff7000ce5e0 2024-09-19T08:38:31.540 INFO:teuthology.orchestra.run.smithi098.stdout:55834574855 2024-09-19T08:38:31.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.543+0000 7ff707414640 1 -- 172.21.15.98:0/1595878631 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff6e4067910 msgr2=0x7ff6e4069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.543+0000 7ff707414640 1 --2- 172.21.15.98:0/1595878631 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff6e4067910 0x7ff6e4069dd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc0b1000 tx=0x7ff6fc0ab000 comp rx=0 tx=0).stop 2024-09-19T08:38:31.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.543+0000 7ff707414640 1 -- 172.21.15.98:0/1595878631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7000ce5e0 msgr2=0x7ff700161d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.543+0000 7ff707414640 1 --2- 172.21.15.98:0/1595878631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7000ce5e0 0x7ff700161d80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff6f800cb10 tx=0x7ff6f800a060 comp rx=0 tx=0).stop 2024-09-19T08:38:31.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.543+0000 7ff707414640 1 -- 172.21.15.98:0/1595878631 shutdown_connections 2024-09-19T08:38:31.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.543+0000 7ff707414640 1 --2- 172.21.15.98:0/1595878631 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff6e4067910 0x7ff6e4069dd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.543+0000 7ff707414640 1 --2- 172.21.15.98:0/1595878631 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff700168bb0 0x7ff7000898d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.543+0000 7ff707414640 1 --2- 172.21.15.98:0/1595878631 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7001687d0 0x7ff70015d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.543+0000 7ff707414640 1 --2- 172.21.15.98:0/1595878631 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7000ce5e0 0x7ff700161d80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.543+0000 7ff707414640 1 -- 172.21.15.98:0/1595878631 >> 172.21.15.98:0/1595878631 conn(0x7ff70006cda0 msgr2=0x7ff70015ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:31.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.543+0000 7ff707414640 1 -- 172.21.15.98:0/1595878631 shutdown_connections 2024-09-19T08:38:31.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.543+0000 7ff707414640 1 -- 172.21.15.98:0/1595878631 wait complete. 2024-09-19T08:38:31.556 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.4 2024-09-19T08:38:31.557 DEBUG:teuthology.parallel:result is None 2024-09-19T08:38:31.559 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2024-09-19T08:38:31.559 DEBUG:teuthology.parallel:result is None 2024-09-19T08:38:31.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.583+0000 7f338fc67640 1 -- 172.21.15.98:0/3696129206 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f33880ce5b0 con 0x7f338815e110 2024-09-19T08:38:31.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.583+0000 7f337e7fc640 1 -- 172.21.15.98:0/3696129206 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f33780ad030 con 0x7f338815e110 2024-09-19T08:38:31.584 INFO:teuthology.orchestra.run.smithi098.stdout:64424509448 2024-09-19T08:38:31.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.587+0000 7f338fc67640 1 -- 172.21.15.98:0/3696129206 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f336c0679c0 msgr2=0x7f336c069e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.587+0000 7f338fc67640 1 --2- 172.21.15.98:0/3696129206 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f336c0679c0 0x7f336c069e80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f33880b1e20 tx=0x7f3384009190 comp rx=0 tx=0).stop 2024-09-19T08:38:31.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.587+0000 7f338fc67640 1 -- 172.21.15.98:0/3696129206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f338815e110 msgr2=0x7f3388203fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.587+0000 7f338fc67640 1 --2- 172.21.15.98:0/3696129206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f338815e110 0x7f3388203fb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f33780ad000 tx=0x7f337801a300 comp rx=0 tx=0).stop 2024-09-19T08:38:31.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.587+0000 7f338fc67640 1 -- 172.21.15.98:0/3696129206 shutdown_connections 2024-09-19T08:38:31.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.587+0000 7f338fc67640 1 --2- 172.21.15.98:0/3696129206 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f336c0679c0 0x7f336c069e80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.587+0000 7f338fc67640 1 --2- 172.21.15.98:0/3696129206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f338815e110 0x7f3388203fb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.587+0000 7f338fc67640 1 --2- 172.21.15.98:0/3696129206 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33880b4220 0x7f338814fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.587+0000 7f338fc67640 1 --2- 172.21.15.98:0/3696129206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f338806f3e0 0x7f338815a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.587+0000 7f338fc67640 1 -- 172.21.15.98:0/3696129206 >> 172.21.15.98:0/3696129206 conn(0x7f338806cda0 msgr2=0x7f33881590c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:31.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.587+0000 7f338fc67640 1 -- 172.21.15.98:0/3696129206 shutdown_connections 2024-09-19T08:38:31.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.587+0000 7f338fc67640 1 -- 172.21.15.98:0/3696129206 wait complete. 2024-09-19T08:38:31.602 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.1 2024-09-19T08:38:31.602 DEBUG:teuthology.parallel:result is None 2024-09-19T08:38:31.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.603+0000 7fa17a575640 1 -- 172.21.15.98:0/1935546833 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fa174159db0 con 0x7fa17415e110 2024-09-19T08:38:31.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.603+0000 7fa170ff9640 1 -- 172.21.15.98:0/1935546833 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa168056340 con 0x7fa17415e110 2024-09-19T08:38:31.609 INFO:teuthology.orchestra.run.smithi098.stdout:64424509449 2024-09-19T08:38:31.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.607+0000 7fa17a575640 1 -- 172.21.15.98:0/1935546833 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa160067990 msgr2=0x7fa160069e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.607+0000 7fa17a575640 1 --2- 172.21.15.98:0/1935546833 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa160067990 0x7fa160069e50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0b2000 tx=0x7fa16c00b040 comp rx=0 tx=0).stop 2024-09-19T08:38:31.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.607+0000 7fa17a575640 1 -- 172.21.15.98:0/1935546833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa17415e110 msgr2=0x7fa174146e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.607+0000 7fa17a575640 1 --2- 172.21.15.98:0/1935546833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa17415e110 0x7fa174146e30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa16800b410 tx=0x7fa16800b8d0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.607+0000 7fa17a575640 1 -- 172.21.15.98:0/1935546833 shutdown_connections 2024-09-19T08:38:31.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.607+0000 7fa17a575640 1 --2- 172.21.15.98:0/1935546833 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa160067990 0x7fa160069e50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.607+0000 7fa17a575640 1 --2- 172.21.15.98:0/1935546833 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa17415ea30 0x7fa1740ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.607+0000 7fa17a575640 1 --2- 172.21.15.98:0/1935546833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa17415e110 0x7fa174146e30 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.607+0000 7fa17a575640 1 --2- 172.21.15.98:0/1935546833 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa17415a4a0 0x7fa174072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.607+0000 7fa17a575640 1 -- 172.21.15.98:0/1935546833 >> 172.21.15.98:0/1935546833 conn(0x7fa17406cda0 msgr2=0x7fa174072160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:31.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.607+0000 7fa17a575640 1 -- 172.21.15.98:0/1935546833 shutdown_connections 2024-09-19T08:38:31.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.607+0000 7fa17a575640 1 -- 172.21.15.98:0/1935546833 wait complete. 2024-09-19T08:38:31.622 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.5 2024-09-19T08:38:31.622 DEBUG:teuthology.parallel:result is None 2024-09-19T08:38:31.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.659+0000 7fdbd153c640 1 -- 172.21.15.98:0/441585864 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fdbcc154ad0 con 0x7fdbcc0b6f50 2024-09-19T08:38:31.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.659+0000 7fdbb37fe640 1 -- 172.21.15.98:0/441585864 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdbc0016090 con 0x7fdbcc0b6f50 2024-09-19T08:38:31.659 INFO:teuthology.orchestra.run.smithi098.stdout:64424509446 2024-09-19T08:38:31.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.659+0000 7fdbd153c640 1 -- 172.21.15.98:0/441585864 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdbb8070120 msgr2=0x7fdbb80725e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.659+0000 7fdbd153c640 1 --2- 172.21.15.98:0/441585864 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdbb8070120 0x7fdbb80725e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc000c10 tx=0x7fdbbc007d90 comp rx=0 tx=0).stop 2024-09-19T08:38:31.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.659+0000 7fdbd153c640 1 -- 172.21.15.98:0/441585864 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbcc0b6f50 msgr2=0x7fdbcc09aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.659+0000 7fdbd153c640 1 --2- 172.21.15.98:0/441585864 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbcc0b6f50 0x7fdbcc09aaf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdbc001a0a0 tx=0x7fdbc001acb0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.659+0000 7fdbd153c640 1 -- 172.21.15.98:0/441585864 shutdown_connections 2024-09-19T08:38:31.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.659+0000 7fdbd153c640 1 --2- 172.21.15.98:0/441585864 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdbb8070120 0x7fdbb80725e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.659+0000 7fdbd153c640 1 --2- 172.21.15.98:0/441585864 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdbcc0ce380 0x7fdbcc15e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.659+0000 7fdbd153c640 1 --2- 172.21.15.98:0/441585864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdbcc0cd350 0x7fdbcc15dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.659+0000 7fdbd153c640 1 --2- 172.21.15.98:0/441585864 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdbcc0b6f50 0x7fdbcc09aaf0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.659+0000 7fdbd153c640 1 -- 172.21.15.98:0/441585864 >> 172.21.15.98:0/441585864 conn(0x7fdbcc073170 msgr2=0x7fdbcc06f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:31.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.659+0000 7fdbd153c640 1 -- 172.21.15.98:0/441585864 shutdown_connections 2024-09-19T08:38:31.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.659+0000 7fdbd153c640 1 -- 172.21.15.98:0/441585864 wait complete. 2024-09-19T08:38:31.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.671+0000 7fb083fff640 1 -- 172.21.15.98:0/1422991439 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fb064005860 con 0x7fb0841558f0 2024-09-19T08:38:31.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.671+0000 7fb06b7fe640 1 -- 172.21.15.98:0/1422991439 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb07c0f0bd0 con 0x7fb0841558f0 2024-09-19T08:38:31.673 INFO:teuthology.orchestra.run.smithi098.stdout:64424509446 2024-09-19T08:38:31.676 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.2 2024-09-19T08:38:31.676 DEBUG:teuthology.parallel:result is None 2024-09-19T08:38:31.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.675+0000 7fb083fff640 1 -- 172.21.15.98:0/1422991439 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb070067910 msgr2=0x7fb070069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.675+0000 7fb083fff640 1 --2- 172.21.15.98:0/1422991439 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb070067910 0x7fb070069dd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb0780045f0 tx=0x7fb078008040 comp rx=0 tx=0).stop 2024-09-19T08:38:31.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.675+0000 7fb083fff640 1 -- 172.21.15.98:0/1422991439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0841558f0 msgr2=0x7fb0840d0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.675+0000 7fb083fff640 1 --2- 172.21.15.98:0/1422991439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0841558f0 0x7fb0840d0170 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb07c01ae20 tx=0x7fb07c019510 comp rx=0 tx=0).stop 2024-09-19T08:38:31.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.675+0000 7fb083fff640 1 -- 172.21.15.98:0/1422991439 shutdown_connections 2024-09-19T08:38:31.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.675+0000 7fb083fff640 1 --2- 172.21.15.98:0/1422991439 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb070067910 0x7fb070069dd0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.675+0000 7fb083fff640 1 --2- 172.21.15.98:0/1422991439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0841558f0 0x7fb0840d0170 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.675+0000 7fb083fff640 1 --2- 172.21.15.98:0/1422991439 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb0840b0360 0x7fb0840cfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.675+0000 7fb083fff640 1 --2- 172.21.15.98:0/1422991439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb08406ca50 0x7fb084151160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.675+0000 7fb083fff640 1 -- 172.21.15.98:0/1422991439 >> 172.21.15.98:0/1422991439 conn(0x7fb0840b2dc0 msgr2=0x7fb0840b1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:31.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.675+0000 7fb083fff640 1 -- 172.21.15.98:0/1422991439 shutdown_connections 2024-09-19T08:38:31.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.675+0000 7fb083fff640 1 -- 172.21.15.98:0/1422991439 wait complete. 2024-09-19T08:38:31.690 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.0 2024-09-19T08:38:31.690 DEBUG:teuthology.parallel:result is None 2024-09-19T08:38:31.691 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T08:38:31.691 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:38:31.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.839+0000 7f6d1b5cf640 1 Processor -- start 2024-09-19T08:38:31.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.839+0000 7f6d1b5cf640 1 -- start start 2024-09-19T08:38:31.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.839+0000 7f6d1b5cf640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d140b28e0 0x7f6d14073c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.839+0000 7f6d1b5cf640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d14075b80 con 0x7f6d14075380 2024-09-19T08:38:31.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.839+0000 7f6d1b5cf640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d140d1bd0 con 0x7f6d1415e750 2024-09-19T08:38:31.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.839+0000 7f6d1b5cf640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6d140d1eb0 con 0x7f6d140b28e0 2024-09-19T08:38:31.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.839+0000 7f6d19b6c640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f6d14075380 0x7f6d140cf520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:60666/0 (socket says 172.21.15.98:60666) 2024-09-19T08:38:31.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.839+0000 7f6d19b6c640 1 -- 172.21.15.98:0/710061681 learned_addr learned my addr 172.21.15.98:0/710061681 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:31.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.839+0000 7f6d18b6a640 1 --2- 172.21.15.98:0/710061681 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d140b28e0 0x7f6d14073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.839+0000 7f6d0bfff640 1 -- 172.21.15.98:0/710061681 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4229921132 0 0) 0x7f6d14075b80 con 0x7f6d14075380 2024-09-19T08:38:31.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.839+0000 7f6d0bfff640 1 -- 172.21.15.98:0/710061681 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d00003530 con 0x7f6d14075380 2024-09-19T08:38:31.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.839+0000 7f6d0bfff640 1 -- 172.21.15.98:0/710061681 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 802781411 0 0) 0x7f6d00003530 con 0x7f6d14075380 2024-09-19T08:38:31.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.839+0000 7f6d0bfff640 1 -- 172.21.15.98:0/710061681 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d140b28e0 msgr2=0x7f6d14073c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:31.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.839+0000 7f6d0bfff640 1 --2- 172.21.15.98:0/710061681 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d140b28e0 0x7f6d14073c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.839+0000 7f6d0bfff640 1 -- 172.21.15.98:0/710061681 >> v1:172.21.15.139:6789/0 conn(0x7f6d1415e750 legacy=0x7f6d14074c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.839+0000 7f6d0bfff640 1 -- 172.21.15.98:0/710061681 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d140cf000 con 0x7f6d14075380 2024-09-19T08:38:31.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.839+0000 7f6d0bfff640 1 -- 172.21.15.98:0/710061681 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f6d0c001a00 con 0x7f6d14075380 2024-09-19T08:38:31.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.839+0000 7f6d0bfff640 1 -- 172.21.15.98:0/710061681 >> v1:172.21.15.98:6789/0 conn(0x7f6d14075380 legacy=0x7f6d140cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.839+0000 7f6d0bfff640 1 --2- 172.21.15.98:0/710061681 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d000039f0 0x7f6d00003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.839+0000 7f6d0bfff640 1 --2- 172.21.15.98:0/710061681 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d140b28e0 0x7f6d00004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.839+0000 7f6d0bfff640 1 --2- 172.21.15.98:0/710061681 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d00004d50 0x7f6d000051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.839+0000 7f6d0bfff640 1 -- 172.21.15.98:0/710061681 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6d140d1eb0 con 0x7f6d000039f0 2024-09-19T08:38:31.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.839+0000 7f6d0bfff640 1 -- 172.21.15.98:0/710061681 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0000bb30 con 0x7f6d00004d50 2024-09-19T08:38:31.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.839+0000 7f6d0bfff640 1 -- 172.21.15.98:0/710061681 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6d0000bd30 con 0x7f6d140b28e0 2024-09-19T08:38:31.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.839+0000 7f6d1936b640 1 --2- 172.21.15.98:0/710061681 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d000039f0 0x7f6d00003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.839+0000 7f6d1936b640 1 -- 172.21.15.98:0/710061681 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d140b28e0 msgr2=0x7f6d00004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:31.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.839+0000 7f6d1936b640 1 --2- 172.21.15.98:0/710061681 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d140b28e0 0x7f6d00004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.839+0000 7f6d19b6c640 1 --2- 172.21.15.98:0/710061681 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d00004d50 0x7f6d000051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.839+0000 7f6d1936b640 1 -- 172.21.15.98:0/710061681 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d00004d50 msgr2=0x7f6d000051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:31.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.839+0000 7f6d1936b640 1 --2- 172.21.15.98:0/710061681 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d00004d50 0x7f6d000051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.839+0000 7f6d1936b640 1 -- 172.21.15.98:0/710061681 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d00003530 con 0x7f6d000039f0 2024-09-19T08:38:31.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.843+0000 7f6d1936b640 1 --2- 172.21.15.98:0/710061681 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d000039f0 0x7f6d00003e10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6d10001780 tx=0x7f6d1001c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50b026177695b94 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:31.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.843+0000 7f6d0bfff640 1 -- 172.21.15.98:0/710061681 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d100013a0 con 0x7f6d000039f0 2024-09-19T08:38:31.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.843+0000 7f6d0bfff640 1 -- 172.21.15.98:0/710061681 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d100052f0 con 0x7f6d000039f0 2024-09-19T08:38:31.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.843+0000 7f6d0bfff640 1 -- 172.21.15.98:0/710061681 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d10003840 con 0x7f6d000039f0 2024-09-19T08:38:31.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.843+0000 7f6d1b5cf640 1 -- 172.21.15.98:0/710061681 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d000039f0 msgr2=0x7f6d00003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.843+0000 7f6d1b5cf640 1 --2- 172.21.15.98:0/710061681 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d000039f0 0x7f6d00003e10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6d10001780 tx=0x7f6d1001c360 comp rx=0 tx=0).stop 2024-09-19T08:38:31.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.843+0000 7f6d19b6c640 1 -- 172.21.15.98:0/710061681 reap_dead start 2024-09-19T08:38:31.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.843+0000 7f6d1b5cf640 1 -- 172.21.15.98:0/710061681 shutdown_connections 2024-09-19T08:38:31.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.843+0000 7f6d1b5cf640 1 -- 172.21.15.98:0/710061681 >> 172.21.15.98:0/710061681 conn(0x7f6d14073170 msgr2=0x7f6d14073580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:31.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.843+0000 7f6d1b5cf640 1 -- 172.21.15.98:0/710061681 shutdown_connections 2024-09-19T08:38:31.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.843+0000 7f6d1b5cf640 1 -- 172.21.15.98:0/710061681 wait complete. 2024-09-19T08:38:31.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.843+0000 7f6d1b5cf640 1 Processor -- start 2024-09-19T08:38:31.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.843+0000 7f6d1b5cf640 1 -- start start 2024-09-19T08:38:31.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.843+0000 7f6d1b5cf640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d000039f0 0x7f6d140caf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.843+0000 7f6d1b5cf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d140c6510 0x7f6d1415f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.843+0000 7f6d1b5cf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d140c6970 0x7f6d1415f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.843+0000 7f6d1b5cf640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6d140d0650 con 0x7f6d140c6970 2024-09-19T08:38:31.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.843+0000 7f6d1b5cf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6d140740a0 con 0x7f6d140c6510 2024-09-19T08:38:31.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.843+0000 7f6d1b5cf640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6d140d07d0 con 0x7f6d000039f0 2024-09-19T08:38:31.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.843+0000 7f6d19b6c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d140c6970 0x7f6d1415f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.843+0000 7f6d19b6c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d140c6970 0x7f6d1415f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38090/0 (socket says 172.21.15.98:38090) 2024-09-19T08:38:31.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.843+0000 7f6d19b6c640 1 -- 172.21.15.98:0/3161794499 learned_addr learned my addr 172.21.15.98:0/3161794499 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:31.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.843+0000 7f6d18b6a640 1 --2- 172.21.15.98:0/3161794499 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d140c6510 0x7f6d1415f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.843+0000 7f6d1936b640 1 --2- 172.21.15.98:0/3161794499 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d000039f0 0x7f6d140caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.843+0000 7f6d19b6c640 1 -- 172.21.15.98:0/3161794499 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d000039f0 msgr2=0x7f6d140caf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.843+0000 7f6d19b6c640 1 --2- 172.21.15.98:0/3161794499 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d000039f0 0x7f6d140caf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.843+0000 7f6d19b6c640 1 -- 172.21.15.98:0/3161794499 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d140c6510 msgr2=0x7f6d1415f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:31.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.843+0000 7f6d19b6c640 1 --2- 172.21.15.98:0/3161794499 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d140c6510 0x7f6d1415f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:31.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.843+0000 7f6d19b6c640 1 -- 172.21.15.98:0/3161794499 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d1415cb70 con 0x7f6d140c6970 2024-09-19T08:38:31.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.843+0000 7f6d19b6c640 1 --2- 172.21.15.98:0/3161794499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d140c6970 0x7f6d1415f760 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c001590 tx=0x7f6d0c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:31.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.847+0000 7f6d09ffb640 1 -- 172.21.15.98:0/3161794499 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c00bb80 con 0x7f6d140c6970 2024-09-19T08:38:31.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.847+0000 7f6d09ffb640 1 -- 172.21.15.98:0/3161794499 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d0c00bd20 con 0x7f6d140c6970 2024-09-19T08:38:31.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.847+0000 7f6d1b5cf640 1 -- 172.21.15.98:0/3161794499 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d140c5470 con 0x7f6d140c6970 2024-09-19T08:38:31.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.847+0000 7f6d09ffb640 1 -- 172.21.15.98:0/3161794499 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c007310 con 0x7f6d140c6970 2024-09-19T08:38:31.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.847+0000 7f6d1b5cf640 1 -- 172.21.15.98:0/3161794499 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d140c5a30 con 0x7f6d140c6970 2024-09-19T08:38:31.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.847+0000 7f6d1b5cf640 1 -- 172.21.15.98:0/3161794499 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d140cf520 con 0x7f6d140c6970 2024-09-19T08:38:31.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.847+0000 7f6d09ffb640 1 -- 172.21.15.98:0/3161794499 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6d0c01b060 con 0x7f6d140c6970 2024-09-19T08:38:31.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.847+0000 7f6d09ffb640 1 --2- 172.21.15.98:0/3161794499 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6cf80678f0 0x7f6cf8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:31.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.847+0000 7f6d09ffb640 1 -- 172.21.15.98:0/3161794499 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6d0c085c30 con 0x7f6d140c6970 2024-09-19T08:38:31.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.847+0000 7f6d1936b640 1 --2- 172.21.15.98:0/3161794499 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6cf80678f0 0x7f6cf8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:31.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.847+0000 7f6d1936b640 1 --2- 172.21.15.98:0/3161794499 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6cf80678f0 0x7f6cf8069db0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6d10001750 tx=0x7f6d100011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:31.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.851+0000 7f6d09ffb640 1 -- 172.21.15.98:0/3161794499 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6d0c056800 con 0x7f6d140c6970 2024-09-19T08:38:31.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:31.999+0000 7f6d1b5cf640 1 -- 172.21.15.98:0/3161794499 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6d14074ff0 con 0x7f6cf80678f0 2024-09-19T08:38:32.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.003+0000 7f6d09ffb640 1 -- 172.21.15.98:0/3161794499 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251837 (secure 0 0 0) 0x7f6d14074ff0 con 0x7f6cf80678f0 2024-09-19T08:38:32.008 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:38:32.010 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:38:32.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.011+0000 7f6d1b5cf640 1 -- 172.21.15.98:0/3161794499 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6cf80678f0 msgr2=0x7f6cf8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:32.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.011+0000 7f6d1b5cf640 1 --2- 172.21.15.98:0/3161794499 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6cf80678f0 0x7f6cf8069db0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6d10001750 tx=0x7f6d100011a0 comp rx=0 tx=0).stop 2024-09-19T08:38:32.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.011+0000 7f6d1b5cf640 1 -- 172.21.15.98:0/3161794499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d140c6970 msgr2=0x7f6d1415f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:32.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.011+0000 7f6d1b5cf640 1 --2- 172.21.15.98:0/3161794499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d140c6970 0x7f6d1415f760 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c001590 tx=0x7f6d0c00b4a0 comp rx=0 tx=0).stop 2024-09-19T08:38:32.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.011+0000 7f6d1b5cf640 1 -- 172.21.15.98:0/3161794499 shutdown_connections 2024-09-19T08:38:32.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.011+0000 7f6d1b5cf640 1 --2- 172.21.15.98:0/3161794499 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6cf80678f0 0x7f6cf8069db0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:32.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.011+0000 7f6d1b5cf640 1 --2- 172.21.15.98:0/3161794499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d140c6970 0x7f6d1415f760 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:32.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.011+0000 7f6d1b5cf640 1 --2- 172.21.15.98:0/3161794499 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d140c6510 0x7f6d1415f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:32.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.011+0000 7f6d1b5cf640 1 --2- 172.21.15.98:0/3161794499 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d000039f0 0x7f6d140caf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:32.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.011+0000 7f6d1b5cf640 1 -- 172.21.15.98:0/3161794499 >> 172.21.15.98:0/3161794499 conn(0x7f6d14073170 msgr2=0x7f6d14073580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:32.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.011+0000 7f6d1b5cf640 1 -- 172.21.15.98:0/3161794499 shutdown_connections 2024-09-19T08:38:32.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.011+0000 7f6d1b5cf640 1 -- 172.21.15.98:0/3161794499 wait complete. 2024-09-19T08:38:32.025 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-19T08:38:30.029585+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219100,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755620,"statfs":{"total":773094113280,"available":772869754880,"internally_reserved":0,"allocated":2924544,"data_stored":1765292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003170"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.495053+0000","last_change":"2024-09-19T08:38:19.406406+0000","last_active":"2024-09-19T08:38:21.495053+0000","last_peered":"2024-09-19T08:38:21.495053+0000","last_clean":"2024-09-19T08:38:21.495053+0000","last_became_active":"2024-09-19T08:38:19.405760+0000","last_became_peered":"2024-09-19T08:38:19.405760+0000","last_unstale":"2024-09-19T08:38:21.495053+0000","last_undegraded":"2024-09-19T08:38:21.495053+0000","last_fullsized":"2024-09-19T08:38:21.495053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:26.845220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.376816+0000","last_change":"2024-09-19T08:38:19.406176+0000","last_active":"2024-09-19T08:38:20.376816+0000","last_peered":"2024-09-19T08:38:20.376816+0000","last_clean":"2024-09-19T08:38:20.376816+0000","last_became_active":"2024-09-19T08:38:19.405604+0000","last_became_peered":"2024-09-19T08:38:19.405604+0000","last_unstale":"2024-09-19T08:38:20.376816+0000","last_undegraded":"2024-09-19T08:38:20.376816+0000","last_fullsized":"2024-09-19T08:38:20.376816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:47:44.536264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.086412+0000","last_change":"2024-09-19T08:38:19.400726+0000","last_active":"2024-09-19T08:38:22.086412+0000","last_peered":"2024-09-19T08:38:22.086412+0000","last_clean":"2024-09-19T08:38:22.086412+0000","last_became_active":"2024-09-19T08:38:19.399956+0000","last_became_peered":"2024-09-19T08:38:19.399956+0000","last_unstale":"2024-09-19T08:38:22.086412+0000","last_undegraded":"2024-09-19T08:38:22.086412+0000","last_fullsized":"2024-09-19T08:38:22.086412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:17:03.830010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.384590+0000","last_change":"2024-09-19T08:38:19.425899+0000","last_active":"2024-09-19T08:38:20.384590+0000","last_peered":"2024-09-19T08:38:20.384590+0000","last_clean":"2024-09-19T08:38:20.384590+0000","last_became_active":"2024-09-19T08:38:19.425622+0000","last_became_peered":"2024-09-19T08:38:19.425622+0000","last_unstale":"2024-09-19T08:38:20.384590+0000","last_undegraded":"2024-09-19T08:38:20.384590+0000","last_fullsized":"2024-09-19T08:38:20.384590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:47:44.536264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.494366+0000","last_change":"2024-09-19T08:38:19.406714+0000","last_active":"2024-09-19T08:38:21.494366+0000","last_peered":"2024-09-19T08:38:21.494366+0000","last_clean":"2024-09-19T08:38:21.494366+0000","last_became_active":"2024-09-19T08:38:19.404949+0000","last_became_peered":"2024-09-19T08:38:19.404949+0000","last_unstale":"2024-09-19T08:38:21.494366+0000","last_undegraded":"2024-09-19T08:38:21.494366+0000","last_fullsized":"2024-09-19T08:38:21.494366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:02:31.580648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.384530+0000","last_change":"2024-09-19T08:38:19.426925+0000","last_active":"2024-09-19T08:38:20.384530+0000","last_peered":"2024-09-19T08:38:20.384530+0000","last_clean":"2024-09-19T08:38:20.384530+0000","last_became_active":"2024-09-19T08:38:19.426666+0000","last_became_peered":"2024-09-19T08:38:19.426666+0000","last_unstale":"2024-09-19T08:38:20.384530+0000","last_undegraded":"2024-09-19T08:38:20.384530+0000","last_fullsized":"2024-09-19T08:38:20.384530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:02:54.292567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.493827+0000","last_change":"2024-09-19T08:38:19.406462+0000","last_active":"2024-09-19T08:38:21.493827+0000","last_peered":"2024-09-19T08:38:21.493827+0000","last_clean":"2024-09-19T08:38:21.493827+0000","last_became_active":"2024-09-19T08:38:19.404851+0000","last_became_peered":"2024-09-19T08:38:19.404851+0000","last_unstale":"2024-09-19T08:38:21.493827+0000","last_undegraded":"2024-09-19T08:38:21.493827+0000","last_fullsized":"2024-09-19T08:38:21.493827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:02:54.292567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.084591+0000","last_change":"2024-09-19T08:38:19.400917+0000","last_active":"2024-09-19T08:38:22.084591+0000","last_peered":"2024-09-19T08:38:22.084591+0000","last_clean":"2024-09-19T08:38:22.084591+0000","last_became_active":"2024-09-19T08:38:19.400188+0000","last_became_peered":"2024-09-19T08:38:19.400188+0000","last_unstale":"2024-09-19T08:38:22.084591+0000","last_undegraded":"2024-09-19T08:38:22.084591+0000","last_fullsized":"2024-09-19T08:38:22.084591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:24:38.033969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.495025+0000","last_change":"2024-09-19T08:38:19.432423+0000","last_active":"2024-09-19T08:38:21.495025+0000","last_peered":"2024-09-19T08:38:21.495025+0000","last_clean":"2024-09-19T08:38:21.495025+0000","last_became_active":"2024-09-19T08:38:19.431886+0000","last_became_peered":"2024-09-19T08:38:19.431886+0000","last_unstale":"2024-09-19T08:38:21.495025+0000","last_undegraded":"2024-09-19T08:38:21.495025+0000","last_fullsized":"2024-09-19T08:38:21.495025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:06.577636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.425359+0000","last_change":"2024-09-19T08:38:19.402509+0000","last_active":"2024-09-19T08:38:20.425359+0000","last_peered":"2024-09-19T08:38:20.425359+0000","last_clean":"2024-09-19T08:38:20.425359+0000","last_became_active":"2024-09-19T08:38:19.401833+0000","last_became_peered":"2024-09-19T08:38:19.401833+0000","last_unstale":"2024-09-19T08:38:20.425359+0000","last_undegraded":"2024-09-19T08:38:20.425359+0000","last_fullsized":"2024-09-19T08:38:20.425359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:08.065282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424901+0000","last_change":"2024-09-19T08:38:19.430286+0000","last_active":"2024-09-19T08:38:20.424901+0000","last_peered":"2024-09-19T08:38:20.424901+0000","last_clean":"2024-09-19T08:38:20.424901+0000","last_became_active":"2024-09-19T08:38:19.429901+0000","last_became_peered":"2024-09-19T08:38:19.429901+0000","last_unstale":"2024-09-19T08:38:20.424901+0000","last_undegraded":"2024-09-19T08:38:20.424901+0000","last_fullsized":"2024-09-19T08:38:20.424901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:31:42.999935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085451+0000","last_change":"2024-09-19T08:38:19.397867+0000","last_active":"2024-09-19T08:38:22.085451+0000","last_peered":"2024-09-19T08:38:22.085451+0000","last_clean":"2024-09-19T08:38:22.085451+0000","last_became_active":"2024-09-19T08:38:19.397106+0000","last_became_peered":"2024-09-19T08:38:19.397106+0000","last_unstale":"2024-09-19T08:38:22.085451+0000","last_undegraded":"2024-09-19T08:38:22.085451+0000","last_fullsized":"2024-09-19T08:38:22.085451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:37.286310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.086556+0000","last_change":"2024-09-19T08:38:19.396759+0000","last_active":"2024-09-19T08:38:22.086556+0000","last_peered":"2024-09-19T08:38:22.086556+0000","last_clean":"2024-09-19T08:38:22.086556+0000","last_became_active":"2024-09-19T08:38:19.395976+0000","last_became_peered":"2024-09-19T08:38:19.395976+0000","last_unstale":"2024-09-19T08:38:22.086556+0000","last_undegraded":"2024-09-19T08:38:22.086556+0000","last_fullsized":"2024-09-19T08:38:22.086556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:20.116184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.084526+0000","last_change":"2024-09-19T08:38:19.405170+0000","last_active":"2024-09-19T08:38:22.084526+0000","last_peered":"2024-09-19T08:38:22.084526+0000","last_clean":"2024-09-19T08:38:22.084526+0000","last_became_active":"2024-09-19T08:38:19.401487+0000","last_became_peered":"2024-09-19T08:38:19.401487+0000","last_unstale":"2024-09-19T08:38:22.084526+0000","last_undegraded":"2024-09-19T08:38:22.084526+0000","last_fullsized":"2024-09-19T08:38:22.084526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:40.803843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.376223+0000","last_change":"2024-09-19T08:38:19.380536+0000","last_active":"2024-09-19T08:38:20.376223+0000","last_peered":"2024-09-19T08:38:20.376223+0000","last_clean":"2024-09-19T08:38:20.376223+0000","last_became_active":"2024-09-19T08:38:19.375034+0000","last_became_peered":"2024-09-19T08:38:19.375034+0000","last_unstale":"2024-09-19T08:38:20.376223+0000","last_undegraded":"2024-09-19T08:38:20.376223+0000","last_fullsized":"2024-09-19T08:38:20.376223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:07.305805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.384258+0000","last_change":"2024-09-19T08:38:19.426216+0000","last_active":"2024-09-19T08:38:20.384258+0000","last_peered":"2024-09-19T08:38:20.384258+0000","last_clean":"2024-09-19T08:38:20.384258+0000","last_became_active":"2024-09-19T08:38:19.425926+0000","last_became_peered":"2024-09-19T08:38:19.425926+0000","last_unstale":"2024-09-19T08:38:20.384258+0000","last_undegraded":"2024-09-19T08:38:20.384258+0000","last_fullsized":"2024-09-19T08:38:20.384258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:47.390332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.385670+0000","last_change":"2024-09-19T08:38:19.427199+0000","last_active":"2024-09-19T08:38:20.385670+0000","last_peered":"2024-09-19T08:38:20.385670+0000","last_clean":"2024-09-19T08:38:20.385670+0000","last_became_active":"2024-09-19T08:38:19.425571+0000","last_became_peered":"2024-09-19T08:38:19.425571+0000","last_unstale":"2024-09-19T08:38:20.385670+0000","last_undegraded":"2024-09-19T08:38:20.385670+0000","last_fullsized":"2024-09-19T08:38:20.385670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:51:44.907657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.084606+0000","last_change":"2024-09-19T08:38:19.394500+0000","last_active":"2024-09-19T08:38:22.084606+0000","last_peered":"2024-09-19T08:38:22.084606+0000","last_clean":"2024-09-19T08:38:22.084606+0000","last_became_active":"2024-09-19T08:38:19.393782+0000","last_became_peered":"2024-09-19T08:38:19.393782+0000","last_unstale":"2024-09-19T08:38:22.084606+0000","last_undegraded":"2024-09-19T08:38:22.084606+0000","last_fullsized":"2024-09-19T08:38:22.084606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:35:09.398239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.494842+0000","last_change":"2024-09-19T08:38:19.412148+0000","last_active":"2024-09-19T08:38:21.494842+0000","last_peered":"2024-09-19T08:38:21.494842+0000","last_clean":"2024-09-19T08:38:21.494842+0000","last_became_active":"2024-09-19T08:38:19.410061+0000","last_became_peered":"2024-09-19T08:38:19.410061+0000","last_unstale":"2024-09-19T08:38:21.494842+0000","last_undegraded":"2024-09-19T08:38:21.494842+0000","last_fullsized":"2024-09-19T08:38:21.494842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:51:44.907657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.425280+0000","last_change":"2024-09-19T08:38:19.389509+0000","last_active":"2024-09-19T08:38:20.425280+0000","last_peered":"2024-09-19T08:38:20.425280+0000","last_clean":"2024-09-19T08:38:20.425280+0000","last_became_active":"2024-09-19T08:38:19.388736+0000","last_became_peered":"2024-09-19T08:38:19.388736+0000","last_unstale":"2024-09-19T08:38:20.425280+0000","last_undegraded":"2024-09-19T08:38:20.425280+0000","last_fullsized":"2024-09-19T08:38:20.425280+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:20:35.891069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.086457+0000","last_change":"2024-09-19T08:38:19.415448+0000","last_active":"2024-09-19T08:38:22.086457+0000","last_peered":"2024-09-19T08:38:22.086457+0000","last_clean":"2024-09-19T08:38:22.086457+0000","last_became_active":"2024-09-19T08:38:19.415169+0000","last_became_peered":"2024-09-19T08:38:19.415169+0000","last_unstale":"2024-09-19T08:38:22.086457+0000","last_undegraded":"2024-09-19T08:38:22.086457+0000","last_fullsized":"2024-09-19T08:38:22.086457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:18:23.928102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.384606+0000","last_change":"2024-09-19T08:38:19.429495+0000","last_active":"2024-09-19T08:38:20.384606+0000","last_peered":"2024-09-19T08:38:20.384606+0000","last_clean":"2024-09-19T08:38:20.384606+0000","last_became_active":"2024-09-19T08:38:19.427201+0000","last_became_peered":"2024-09-19T08:38:19.427201+0000","last_unstale":"2024-09-19T08:38:20.384606+0000","last_undegraded":"2024-09-19T08:38:20.384606+0000","last_fullsized":"2024-09-19T08:38:20.384606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:08.554428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.084821+0000","last_change":"2024-09-19T08:38:19.406260+0000","last_active":"2024-09-19T08:38:22.084821+0000","last_peered":"2024-09-19T08:38:22.084821+0000","last_clean":"2024-09-19T08:38:22.084821+0000","last_became_active":"2024-09-19T08:38:19.405769+0000","last_became_peered":"2024-09-19T08:38:19.405769+0000","last_unstale":"2024-09-19T08:38:22.084821+0000","last_undegraded":"2024-09-19T08:38:22.084821+0000","last_fullsized":"2024-09-19T08:38:22.084821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:31:42.999935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.384446+0000","last_change":"2024-09-19T08:38:19.402246+0000","last_active":"2024-09-19T08:38:20.384446+0000","last_peered":"2024-09-19T08:38:20.384446+0000","last_clean":"2024-09-19T08:38:20.384446+0000","last_became_active":"2024-09-19T08:38:19.401487+0000","last_became_peered":"2024-09-19T08:38:19.401487+0000","last_unstale":"2024-09-19T08:38:20.384446+0000","last_undegraded":"2024-09-19T08:38:20.384446+0000","last_fullsized":"2024-09-19T08:38:20.384446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:54:01.480300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085580+0000","last_change":"2024-09-19T08:38:19.406201+0000","last_active":"2024-09-19T08:38:22.085580+0000","last_peered":"2024-09-19T08:38:22.085580+0000","last_clean":"2024-09-19T08:38:22.085580+0000","last_became_active":"2024-09-19T08:38:19.405583+0000","last_became_peered":"2024-09-19T08:38:19.405583+0000","last_unstale":"2024-09-19T08:38:22.085580+0000","last_undegraded":"2024-09-19T08:38:22.085580+0000","last_fullsized":"2024-09-19T08:38:22.085580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:08.065282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.425374+0000","last_change":"2024-09-19T08:38:19.411842+0000","last_active":"2024-09-19T08:38:20.425374+0000","last_peered":"2024-09-19T08:38:20.425374+0000","last_clean":"2024-09-19T08:38:20.425374+0000","last_became_active":"2024-09-19T08:38:19.411092+0000","last_became_peered":"2024-09-19T08:38:19.411092+0000","last_unstale":"2024-09-19T08:38:20.425374+0000","last_undegraded":"2024-09-19T08:38:20.425374+0000","last_fullsized":"2024-09-19T08:38:20.425374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:39:22.441525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.376568+0000","last_change":"2024-09-19T08:38:19.393312+0000","last_active":"2024-09-19T08:38:20.376568+0000","last_peered":"2024-09-19T08:38:20.376568+0000","last_clean":"2024-09-19T08:38:20.376568+0000","last_became_active":"2024-09-19T08:38:19.392591+0000","last_became_peered":"2024-09-19T08:38:19.392591+0000","last_unstale":"2024-09-19T08:38:20.376568+0000","last_undegraded":"2024-09-19T08:38:20.376568+0000","last_fullsized":"2024-09-19T08:38:20.376568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:54:01.480300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085836+0000","last_change":"2024-09-19T08:38:19.408371+0000","last_active":"2024-09-19T08:38:22.085836+0000","last_peered":"2024-09-19T08:38:22.085836+0000","last_clean":"2024-09-19T08:38:22.085836+0000","last_became_active":"2024-09-19T08:38:19.407846+0000","last_became_peered":"2024-09-19T08:38:19.407846+0000","last_unstale":"2024-09-19T08:38:22.085836+0000","last_undegraded":"2024-09-19T08:38:22.085836+0000","last_fullsized":"2024-09-19T08:38:22.085836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:33.753905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.086518+0000","last_change":"2024-09-19T08:38:19.406826+0000","last_active":"2024-09-19T08:38:22.086518+0000","last_peered":"2024-09-19T08:38:22.086518+0000","last_clean":"2024-09-19T08:38:22.086518+0000","last_became_active":"2024-09-19T08:38:19.406456+0000","last_became_peered":"2024-09-19T08:38:19.406456+0000","last_unstale":"2024-09-19T08:38:22.086518+0000","last_undegraded":"2024-09-19T08:38:22.086518+0000","last_fullsized":"2024-09-19T08:38:22.086518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:32:19.829820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.384963+0000","last_change":"2024-09-19T08:38:19.430413+0000","last_active":"2024-09-19T08:38:20.384963+0000","last_peered":"2024-09-19T08:38:20.384963+0000","last_clean":"2024-09-19T08:38:20.384963+0000","last_became_active":"2024-09-19T08:38:19.430122+0000","last_became_peered":"2024-09-19T08:38:19.430122+0000","last_unstale":"2024-09-19T08:38:20.384963+0000","last_undegraded":"2024-09-19T08:38:20.384963+0000","last_fullsized":"2024-09-19T08:38:20.384963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:13:57.163258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.083388+0000","last_change":"2024-09-19T08:38:19.429646+0000","last_active":"2024-09-19T08:38:22.083388+0000","last_peered":"2024-09-19T08:38:22.083388+0000","last_clean":"2024-09-19T08:38:22.083388+0000","last_became_active":"2024-09-19T08:38:19.429256+0000","last_became_peered":"2024-09-19T08:38:19.429256+0000","last_unstale":"2024-09-19T08:38:22.083388+0000","last_undegraded":"2024-09-19T08:38:22.083388+0000","last_fullsized":"2024-09-19T08:38:22.083388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:37.286310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424603+0000","last_change":"2024-09-19T08:38:19.414469+0000","last_active":"2024-09-19T08:38:20.424603+0000","last_peered":"2024-09-19T08:38:20.424603+0000","last_clean":"2024-09-19T08:38:20.424603+0000","last_became_active":"2024-09-19T08:38:19.413995+0000","last_became_peered":"2024-09-19T08:38:19.413995+0000","last_unstale":"2024-09-19T08:38:20.424603+0000","last_undegraded":"2024-09-19T08:38:20.424603+0000","last_fullsized":"2024-09-19T08:38:20.424603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:24:38.033969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085027+0000","last_change":"2024-09-19T08:38:19.377658+0000","last_active":"2024-09-19T08:38:22.085027+0000","last_peered":"2024-09-19T08:38:22.085027+0000","last_clean":"2024-09-19T08:38:22.085027+0000","last_became_active":"2024-09-19T08:38:19.376916+0000","last_became_peered":"2024-09-19T08:38:19.376916+0000","last_unstale":"2024-09-19T08:38:22.085027+0000","last_undegraded":"2024-09-19T08:38:22.085027+0000","last_fullsized":"2024-09-19T08:38:22.085027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:21.640545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085745+0000","last_change":"2024-09-19T08:38:18.387502+0000","last_active":"2024-09-19T08:38:22.085745+0000","last_peered":"2024-09-19T08:38:22.085745+0000","last_clean":"2024-09-19T08:38:22.085745+0000","last_became_active":"2024-09-19T08:38:18.387154+0000","last_became_peered":"2024-09-19T08:38:18.387154+0000","last_unstale":"2024-09-19T08:38:22.085745+0000","last_undegraded":"2024-09-19T08:38:22.085745+0000","last_fullsized":"2024-09-19T08:38:22.085745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:53.708142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.086691+0000","last_change":"2024-09-19T08:38:19.408323+0000","last_active":"2024-09-19T08:38:22.086691+0000","last_peered":"2024-09-19T08:38:22.086691+0000","last_clean":"2024-09-19T08:38:22.086691+0000","last_became_active":"2024-09-19T08:38:19.407418+0000","last_became_peered":"2024-09-19T08:38:19.407418+0000","last_unstale":"2024-09-19T08:38:22.086691+0000","last_undegraded":"2024-09-19T08:38:22.086691+0000","last_fullsized":"2024-09-19T08:38:22.086691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:32:35.027328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424590+0000","last_change":"2024-09-19T08:38:18.381446+0000","last_active":"2024-09-19T08:38:20.424590+0000","last_peered":"2024-09-19T08:38:20.424590+0000","last_clean":"2024-09-19T08:38:20.424590+0000","last_became_active":"2024-09-19T08:38:18.380678+0000","last_became_peered":"2024-09-19T08:38:18.380678+0000","last_unstale":"2024-09-19T08:38:20.424590+0000","last_undegraded":"2024-09-19T08:38:20.424590+0000","last_fullsized":"2024-09-19T08:38:20.424590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:07:31.456618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.084969+0000","last_change":"2024-09-19T08:38:18.376428+0000","last_active":"2024-09-19T08:38:22.084969+0000","last_peered":"2024-09-19T08:38:22.084969+0000","last_clean":"2024-09-19T08:38:22.084969+0000","last_became_active":"2024-09-19T08:38:18.376015+0000","last_became_peered":"2024-09-19T08:38:18.376015+0000","last_unstale":"2024-09-19T08:38:22.084969+0000","last_undegraded":"2024-09-19T08:38:22.084969+0000","last_fullsized":"2024-09-19T08:38:22.084969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:49:33.943029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.084815+0000","last_change":"2024-09-19T08:38:19.414261+0000","last_active":"2024-09-19T08:38:22.084815+0000","last_peered":"2024-09-19T08:38:22.084815+0000","last_clean":"2024-09-19T08:38:22.084815+0000","last_became_active":"2024-09-19T08:38:19.413930+0000","last_became_peered":"2024-09-19T08:38:19.413930+0000","last_unstale":"2024-09-19T08:38:22.084815+0000","last_undegraded":"2024-09-19T08:38:22.084815+0000","last_fullsized":"2024-09-19T08:38:22.084815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:32.216306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.384529+0000","last_change":"2024-09-19T08:38:18.393405+0000","last_active":"2024-09-19T08:38:20.384529+0000","last_peered":"2024-09-19T08:38:20.384529+0000","last_clean":"2024-09-19T08:38:20.384529+0000","last_became_active":"2024-09-19T08:38:18.392613+0000","last_became_peered":"2024-09-19T08:38:18.392613+0000","last_unstale":"2024-09-19T08:38:20.384529+0000","last_undegraded":"2024-09-19T08:38:20.384529+0000","last_fullsized":"2024-09-19T08:38:20.384529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:55.536656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.376013+0000","last_change":"2024-09-19T08:38:19.397074+0000","last_active":"2024-09-19T08:38:20.376013+0000","last_peered":"2024-09-19T08:38:20.376013+0000","last_clean":"2024-09-19T08:38:20.376013+0000","last_became_active":"2024-09-19T08:38:19.396130+0000","last_became_peered":"2024-09-19T08:38:19.396130+0000","last_unstale":"2024-09-19T08:38:20.376013+0000","last_undegraded":"2024-09-19T08:38:20.376013+0000","last_fullsized":"2024-09-19T08:38:20.376013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:32:35.027328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424870+0000","last_change":"2024-09-19T08:38:18.398231+0000","last_active":"2024-09-19T08:38:20.424870+0000","last_peered":"2024-09-19T08:38:20.424870+0000","last_clean":"2024-09-19T08:38:20.424870+0000","last_became_active":"2024-09-19T08:38:18.397590+0000","last_became_peered":"2024-09-19T08:38:18.397590+0000","last_unstale":"2024-09-19T08:38:20.424870+0000","last_undegraded":"2024-09-19T08:38:20.424870+0000","last_fullsized":"2024-09-19T08:38:20.424870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:54:02.262454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.385857+0000","last_change":"2024-09-19T08:38:19.415962+0000","last_active":"2024-09-19T08:38:20.385857+0000","last_peered":"2024-09-19T08:38:20.385857+0000","last_clean":"2024-09-19T08:38:20.385857+0000","last_became_active":"2024-09-19T08:38:19.415579+0000","last_became_peered":"2024-09-19T08:38:19.415579+0000","last_unstale":"2024-09-19T08:38:20.385857+0000","last_undegraded":"2024-09-19T08:38:20.385857+0000","last_fullsized":"2024-09-19T08:38:20.385857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:06.577636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.385384+0000","last_change":"2024-09-19T08:38:18.393568+0000","last_active":"2024-09-19T08:38:20.385384+0000","last_peered":"2024-09-19T08:38:20.385384+0000","last_clean":"2024-09-19T08:38:20.385384+0000","last_became_active":"2024-09-19T08:38:18.390336+0000","last_became_peered":"2024-09-19T08:38:18.390336+0000","last_unstale":"2024-09-19T08:38:20.385384+0000","last_undegraded":"2024-09-19T08:38:20.385384+0000","last_fullsized":"2024-09-19T08:38:20.385384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:20.637060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.087146+0000","last_change":"2024-09-19T08:38:19.428816+0000","last_active":"2024-09-19T08:38:22.087146+0000","last_peered":"2024-09-19T08:38:22.087146+0000","last_clean":"2024-09-19T08:38:22.087146+0000","last_became_active":"2024-09-19T08:38:19.428391+0000","last_became_peered":"2024-09-19T08:38:19.428391+0000","last_unstale":"2024-09-19T08:38:22.087146+0000","last_undegraded":"2024-09-19T08:38:22.087146+0000","last_fullsized":"2024-09-19T08:38:22.087146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:47.390332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.376655+0000","last_change":"2024-09-19T08:38:18.390022+0000","last_active":"2024-09-19T08:38:20.376655+0000","last_peered":"2024-09-19T08:38:20.376655+0000","last_clean":"2024-09-19T08:38:20.376655+0000","last_became_active":"2024-09-19T08:38:18.389679+0000","last_became_peered":"2024-09-19T08:38:18.389679+0000","last_unstale":"2024-09-19T08:38:20.376655+0000","last_undegraded":"2024-09-19T08:38:20.376655+0000","last_fullsized":"2024-09-19T08:38:20.376655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:39.064483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085614+0000","last_change":"2024-09-19T08:38:19.397504+0000","last_active":"2024-09-19T08:38:22.085614+0000","last_peered":"2024-09-19T08:38:22.085614+0000","last_clean":"2024-09-19T08:38:22.085614+0000","last_became_active":"2024-09-19T08:38:19.396770+0000","last_became_peered":"2024-09-19T08:38:19.396770+0000","last_unstale":"2024-09-19T08:38:22.085614+0000","last_undegraded":"2024-09-19T08:38:22.085614+0000","last_fullsized":"2024-09-19T08:38:22.085614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:35:09.398239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424913+0000","last_change":"2024-09-19T08:38:18.371315+0000","last_active":"2024-09-19T08:38:20.424913+0000","last_peered":"2024-09-19T08:38:20.424913+0000","last_clean":"2024-09-19T08:38:20.424913+0000","last_became_active":"2024-09-19T08:38:18.370524+0000","last_became_peered":"2024-09-19T08:38:18.370524+0000","last_unstale":"2024-09-19T08:38:20.424913+0000","last_undegraded":"2024-09-19T08:38:20.424913+0000","last_fullsized":"2024-09-19T08:38:20.424913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:49:33.943029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.494199+0000","last_change":"2024-09-19T08:38:19.409150+0000","last_active":"2024-09-19T08:38:21.494199+0000","last_peered":"2024-09-19T08:38:21.494199+0000","last_clean":"2024-09-19T08:38:21.494199+0000","last_became_active":"2024-09-19T08:38:19.408303+0000","last_became_peered":"2024-09-19T08:38:19.408303+0000","last_unstale":"2024-09-19T08:38:21.494199+0000","last_undegraded":"2024-09-19T08:38:21.494199+0000","last_fullsized":"2024-09-19T08:38:21.494199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:13:57.163258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.086837+0000","last_change":"2024-09-19T08:38:18.366653+0000","last_active":"2024-09-19T08:38:22.086837+0000","last_peered":"2024-09-19T08:38:22.086837+0000","last_clean":"2024-09-19T08:38:22.086837+0000","last_became_active":"2024-09-19T08:38:18.366387+0000","last_became_peered":"2024-09-19T08:38:18.366387+0000","last_unstale":"2024-09-19T08:38:22.086837+0000","last_undegraded":"2024-09-19T08:38:22.086837+0000","last_fullsized":"2024-09-19T08:38:22.086837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:52.328114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.086019+0000","last_change":"2024-09-19T08:38:19.415512+0000","last_active":"2024-09-19T08:38:22.086019+0000","last_peered":"2024-09-19T08:38:22.086019+0000","last_clean":"2024-09-19T08:38:22.086019+0000","last_became_active":"2024-09-19T08:38:19.415166+0000","last_became_peered":"2024-09-19T08:38:19.415166+0000","last_unstale":"2024-09-19T08:38:22.086019+0000","last_undegraded":"2024-09-19T08:38:22.086019+0000","last_fullsized":"2024-09-19T08:38:22.086019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:22.366004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.384829+0000","last_change":"2024-09-19T08:38:18.402871+0000","last_active":"2024-09-19T08:38:20.384829+0000","last_peered":"2024-09-19T08:38:20.384829+0000","last_clean":"2024-09-19T08:38:20.384829+0000","last_became_active":"2024-09-19T08:38:18.400331+0000","last_became_peered":"2024-09-19T08:38:18.400331+0000","last_unstale":"2024-09-19T08:38:20.384829+0000","last_undegraded":"2024-09-19T08:38:20.384829+0000","last_fullsized":"2024-09-19T08:38:20.384829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:39.064483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.494370+0000","last_change":"2024-09-19T08:38:19.406507+0000","last_active":"2024-09-19T08:38:21.494370+0000","last_peered":"2024-09-19T08:38:21.494370+0000","last_clean":"2024-09-19T08:38:21.494370+0000","last_became_active":"2024-09-19T08:38:19.405133+0000","last_became_peered":"2024-09-19T08:38:19.405133+0000","last_unstale":"2024-09-19T08:38:21.494370+0000","last_undegraded":"2024-09-19T08:38:21.494370+0000","last_fullsized":"2024-09-19T08:38:21.494370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:30:08.777799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424461+0000","last_change":"2024-09-19T08:38:18.396642+0000","last_active":"2024-09-19T08:38:20.424461+0000","last_peered":"2024-09-19T08:38:20.424461+0000","last_clean":"2024-09-19T08:38:20.424461+0000","last_became_active":"2024-09-19T08:38:18.396009+0000","last_became_peered":"2024-09-19T08:38:18.396009+0000","last_unstale":"2024-09-19T08:38:20.424461+0000","last_undegraded":"2024-09-19T08:38:20.424461+0000","last_fullsized":"2024-09-19T08:38:20.424461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:21.951846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424570+0000","last_change":"2024-09-19T08:38:19.409375+0000","last_active":"2024-09-19T08:38:20.424570+0000","last_peered":"2024-09-19T08:38:20.424570+0000","last_clean":"2024-09-19T08:38:20.424570+0000","last_became_active":"2024-09-19T08:38:19.408667+0000","last_became_peered":"2024-09-19T08:38:19.408667+0000","last_unstale":"2024-09-19T08:38:20.424570+0000","last_undegraded":"2024-09-19T08:38:20.424570+0000","last_fullsized":"2024-09-19T08:38:20.424570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:32.216306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.082894+0000","last_change":"2024-09-19T08:38:18.360672+0000","last_active":"2024-09-19T08:38:22.082894+0000","last_peered":"2024-09-19T08:38:22.082894+0000","last_clean":"2024-09-19T08:38:22.082894+0000","last_became_active":"2024-09-19T08:38:18.360210+0000","last_became_peered":"2024-09-19T08:38:18.360210+0000","last_unstale":"2024-09-19T08:38:22.082894+0000","last_undegraded":"2024-09-19T08:38:22.082894+0000","last_fullsized":"2024-09-19T08:38:22.082894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:20.637060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.493833+0000","last_change":"2024-09-19T08:38:19.407114+0000","last_active":"2024-09-19T08:38:21.493833+0000","last_peered":"2024-09-19T08:38:21.493833+0000","last_clean":"2024-09-19T08:38:21.493833+0000","last_became_active":"2024-09-19T08:38:19.405734+0000","last_became_peered":"2024-09-19T08:38:19.405734+0000","last_unstale":"2024-09-19T08:38:21.493833+0000","last_undegraded":"2024-09-19T08:38:21.493833+0000","last_fullsized":"2024-09-19T08:38:21.493833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:08.554428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085716+0000","last_change":"2024-09-19T08:38:18.377025+0000","last_active":"2024-09-19T08:38:22.085716+0000","last_peered":"2024-09-19T08:38:22.085716+0000","last_clean":"2024-09-19T08:38:22.085716+0000","last_became_active":"2024-09-19T08:38:18.372574+0000","last_became_peered":"2024-09-19T08:38:18.372574+0000","last_unstale":"2024-09-19T08:38:22.085716+0000","last_undegraded":"2024-09-19T08:38:22.085716+0000","last_fullsized":"2024-09-19T08:38:22.085716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:30.341445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.494907+0000","last_change":"2024-09-19T08:38:19.405731+0000","last_active":"2024-09-19T08:38:21.494907+0000","last_peered":"2024-09-19T08:38:21.494907+0000","last_clean":"2024-09-19T08:38:21.494907+0000","last_became_active":"2024-09-19T08:38:19.404769+0000","last_became_peered":"2024-09-19T08:38:19.404769+0000","last_unstale":"2024-09-19T08:38:21.494907+0000","last_undegraded":"2024-09-19T08:38:21.494907+0000","last_fullsized":"2024-09-19T08:38:21.494907+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:21.788118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.084854+0000","last_change":"2024-09-19T08:38:18.366653+0000","last_active":"2024-09-19T08:38:22.084854+0000","last_peered":"2024-09-19T08:38:22.084854+0000","last_clean":"2024-09-19T08:38:22.084854+0000","last_became_active":"2024-09-19T08:38:18.366357+0000","last_became_peered":"2024-09-19T08:38:18.366357+0000","last_unstale":"2024-09-19T08:38:22.084854+0000","last_undegraded":"2024-09-19T08:38:22.084854+0000","last_fullsized":"2024-09-19T08:38:22.084854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:47.789309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.495241+0000","last_change":"2024-09-19T08:38:19.408858+0000","last_active":"2024-09-19T08:38:21.495241+0000","last_peered":"2024-09-19T08:38:21.495241+0000","last_clean":"2024-09-19T08:38:21.495241+0000","last_became_active":"2024-09-19T08:38:19.408122+0000","last_became_peered":"2024-09-19T08:38:19.408122+0000","last_unstale":"2024-09-19T08:38:21.495241+0000","last_undegraded":"2024-09-19T08:38:21.495241+0000","last_fullsized":"2024-09-19T08:38:21.495241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:02.493804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.376783+0000","last_change":"2024-09-19T08:38:18.366656+0000","last_active":"2024-09-19T08:38:20.376783+0000","last_peered":"2024-09-19T08:38:20.376783+0000","last_clean":"2024-09-19T08:38:20.376783+0000","last_became_active":"2024-09-19T08:38:18.366299+0000","last_became_peered":"2024-09-19T08:38:18.366299+0000","last_unstale":"2024-09-19T08:38:20.376783+0000","last_undegraded":"2024-09-19T08:38:20.376783+0000","last_fullsized":"2024-09-19T08:38:20.376783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:35:08.394753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.385063+0000","last_change":"2024-09-19T08:38:19.414578+0000","last_active":"2024-09-19T08:38:20.385063+0000","last_peered":"2024-09-19T08:38:20.385063+0000","last_clean":"2024-09-19T08:38:20.385063+0000","last_became_active":"2024-09-19T08:38:19.413677+0000","last_became_peered":"2024-09-19T08:38:19.413677+0000","last_unstale":"2024-09-19T08:38:20.385063+0000","last_undegraded":"2024-09-19T08:38:20.385063+0000","last_fullsized":"2024-09-19T08:38:20.385063+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:30:08.777799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.083724+0000","last_change":"2024-09-19T08:38:18.394865+0000","last_active":"2024-09-19T08:38:22.083724+0000","last_peered":"2024-09-19T08:38:22.083724+0000","last_clean":"2024-09-19T08:38:22.083724+0000","last_became_active":"2024-09-19T08:38:18.394496+0000","last_became_peered":"2024-09-19T08:38:18.394496+0000","last_unstale":"2024-09-19T08:38:22.083724+0000","last_undegraded":"2024-09-19T08:38:22.083724+0000","last_fullsized":"2024-09-19T08:38:22.083724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:34:58.036813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.494357+0000","last_change":"2024-09-19T08:38:19.433593+0000","last_active":"2024-09-19T08:38:21.494357+0000","last_peered":"2024-09-19T08:38:21.494357+0000","last_clean":"2024-09-19T08:38:21.494357+0000","last_became_active":"2024-09-19T08:38:19.433322+0000","last_became_peered":"2024-09-19T08:38:19.433322+0000","last_unstale":"2024-09-19T08:38:21.494357+0000","last_undegraded":"2024-09-19T08:38:21.494357+0000","last_fullsized":"2024-09-19T08:38:21.494357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:44:57.127592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.493861+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:38:21.493861+0000","last_peered":"2024-09-19T08:38:21.493861+0000","last_clean":"2024-09-19T08:38:21.493861+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:38:21.493861+0000","last_undegraded":"2024-09-19T08:38:21.493861+0000","last_fullsized":"2024-09-19T08:38:21.493861+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.384222+0000","last_change":"2024-09-19T08:38:18.391179+0000","last_active":"2024-09-19T08:38:20.384222+0000","last_peered":"2024-09-19T08:38:20.384222+0000","last_clean":"2024-09-19T08:38:20.384222+0000","last_became_active":"2024-09-19T08:38:18.390393+0000","last_became_peered":"2024-09-19T08:38:18.390393+0000","last_unstale":"2024-09-19T08:38:20.384222+0000","last_undegraded":"2024-09-19T08:38:20.384222+0000","last_fullsized":"2024-09-19T08:38:20.384222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:47.789309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085655+0000","last_change":"2024-09-19T08:38:19.430328+0000","last_active":"2024-09-19T08:38:22.085655+0000","last_peered":"2024-09-19T08:38:22.085655+0000","last_clean":"2024-09-19T08:38:22.085655+0000","last_became_active":"2024-09-19T08:38:19.429984+0000","last_became_peered":"2024-09-19T08:38:19.429984+0000","last_unstale":"2024-09-19T08:38:22.085655+0000","last_undegraded":"2024-09-19T08:38:22.085655+0000","last_fullsized":"2024-09-19T08:38:22.085655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:19:32.736687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.493590+0000","last_change":"2024-09-19T08:38:18.384759+0000","last_active":"2024-09-19T08:38:21.493590+0000","last_peered":"2024-09-19T08:38:21.493590+0000","last_clean":"2024-09-19T08:38:21.493590+0000","last_became_active":"2024-09-19T08:38:18.383974+0000","last_became_peered":"2024-09-19T08:38:18.383974+0000","last_unstale":"2024-09-19T08:38:21.493590+0000","last_undegraded":"2024-09-19T08:38:21.493590+0000","last_fullsized":"2024-09-19T08:38:21.493590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:39.064483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.084012+0000","last_change":"2024-09-19T08:38:19.412025+0000","last_active":"2024-09-19T08:38:22.084012+0000","last_peered":"2024-09-19T08:38:22.084012+0000","last_clean":"2024-09-19T08:38:22.084012+0000","last_became_active":"2024-09-19T08:38:19.411701+0000","last_became_peered":"2024-09-19T08:38:19.411701+0000","last_unstale":"2024-09-19T08:38:22.084012+0000","last_undegraded":"2024-09-19T08:38:22.084012+0000","last_fullsized":"2024-09-19T08:38:22.084012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:13:02.635546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.083685+0000","last_change":"2024-09-19T08:38:18.366746+0000","last_active":"2024-09-19T08:38:22.083685+0000","last_peered":"2024-09-19T08:38:22.083685+0000","last_clean":"2024-09-19T08:38:22.083685+0000","last_became_active":"2024-09-19T08:38:18.366346+0000","last_became_peered":"2024-09-19T08:38:18.366346+0000","last_unstale":"2024-09-19T08:38:22.083685+0000","last_undegraded":"2024-09-19T08:38:22.083685+0000","last_fullsized":"2024-09-19T08:38:22.083685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:55.536656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.376256+0000","last_change":"2024-09-19T08:38:19.395656+0000","last_active":"2024-09-19T08:38:20.376256+0000","last_peered":"2024-09-19T08:38:20.376256+0000","last_clean":"2024-09-19T08:38:20.376256+0000","last_became_active":"2024-09-19T08:38:19.393805+0000","last_became_peered":"2024-09-19T08:38:19.393805+0000","last_unstale":"2024-09-19T08:38:20.376256+0000","last_undegraded":"2024-09-19T08:38:20.376256+0000","last_fullsized":"2024-09-19T08:38:20.376256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:08.505022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.385101+0000","last_change":"2024-09-19T08:38:18.398888+0000","last_active":"2024-09-19T08:38:20.385101+0000","last_peered":"2024-09-19T08:38:20.385101+0000","last_clean":"2024-09-19T08:38:20.385101+0000","last_became_active":"2024-09-19T08:38:18.391135+0000","last_became_peered":"2024-09-19T08:38:18.391135+0000","last_unstale":"2024-09-19T08:38:20.385101+0000","last_undegraded":"2024-09-19T08:38:20.385101+0000","last_fullsized":"2024-09-19T08:38:20.385101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:48.426450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085253+0000","last_change":"2024-09-19T08:38:19.418136+0000","last_active":"2024-09-19T08:38:22.085253+0000","last_peered":"2024-09-19T08:38:22.085253+0000","last_clean":"2024-09-19T08:38:22.085253+0000","last_became_active":"2024-09-19T08:38:19.411375+0000","last_became_peered":"2024-09-19T08:38:19.411375+0000","last_unstale":"2024-09-19T08:38:22.085253+0000","last_undegraded":"2024-09-19T08:38:22.085253+0000","last_fullsized":"2024-09-19T08:38:22.085253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:40.067968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.376493+0000","last_change":"2024-09-19T08:38:18.363154+0000","last_active":"2024-09-19T08:38:20.376493+0000","last_peered":"2024-09-19T08:38:20.376493+0000","last_clean":"2024-09-19T08:38:20.376493+0000","last_became_active":"2024-09-19T08:38:18.362876+0000","last_became_peered":"2024-09-19T08:38:18.362876+0000","last_unstale":"2024-09-19T08:38:20.376493+0000","last_undegraded":"2024-09-19T08:38:20.376493+0000","last_fullsized":"2024-09-19T08:38:20.376493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:48.426450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.385738+0000","last_change":"2024-09-19T08:38:19.413745+0000","last_active":"2024-09-19T08:38:20.385738+0000","last_peered":"2024-09-19T08:38:20.385738+0000","last_clean":"2024-09-19T08:38:20.385738+0000","last_became_active":"2024-09-19T08:38:19.413092+0000","last_became_peered":"2024-09-19T08:38:19.413092+0000","last_unstale":"2024-09-19T08:38:20.385738+0000","last_undegraded":"2024-09-19T08:38:20.385738+0000","last_fullsized":"2024-09-19T08:38:20.385738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:21.788118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.494741+0000","last_change":"2024-09-19T08:38:18.389212+0000","last_active":"2024-09-19T08:38:21.494741+0000","last_peered":"2024-09-19T08:38:21.494741+0000","last_clean":"2024-09-19T08:38:21.494741+0000","last_became_active":"2024-09-19T08:38:18.388588+0000","last_became_peered":"2024-09-19T08:38:18.388588+0000","last_unstale":"2024-09-19T08:38:21.494741+0000","last_undegraded":"2024-09-19T08:38:21.494741+0000","last_fullsized":"2024-09-19T08:38:21.494741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:46:34.417531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.376388+0000","last_change":"2024-09-19T08:38:19.386583+0000","last_active":"2024-09-19T08:38:20.376388+0000","last_peered":"2024-09-19T08:38:20.376388+0000","last_clean":"2024-09-19T08:38:20.376388+0000","last_became_active":"2024-09-19T08:38:19.385749+0000","last_became_peered":"2024-09-19T08:38:19.385749+0000","last_unstale":"2024-09-19T08:38:20.376388+0000","last_undegraded":"2024-09-19T08:38:20.376388+0000","last_fullsized":"2024-09-19T08:38:20.376388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:47.390332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.376949+0000","last_change":"2024-09-19T08:38:18.377131+0000","last_active":"2024-09-19T08:38:20.376949+0000","last_peered":"2024-09-19T08:38:20.376949+0000","last_clean":"2024-09-19T08:38:20.376949+0000","last_became_active":"2024-09-19T08:38:18.376860+0000","last_became_peered":"2024-09-19T08:38:18.376860+0000","last_unstale":"2024-09-19T08:38:20.376949+0000","last_undegraded":"2024-09-19T08:38:20.376949+0000","last_fullsized":"2024-09-19T08:38:20.376949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:46:34.417531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085974+0000","last_change":"2024-09-19T08:38:19.410025+0000","last_active":"2024-09-19T08:38:22.085974+0000","last_peered":"2024-09-19T08:38:22.085974+0000","last_clean":"2024-09-19T08:38:22.085974+0000","last_became_active":"2024-09-19T08:38:19.409390+0000","last_became_peered":"2024-09-19T08:38:19.409390+0000","last_unstale":"2024-09-19T08:38:22.085974+0000","last_undegraded":"2024-09-19T08:38:22.085974+0000","last_fullsized":"2024-09-19T08:38:22.085974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:10.400137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.425026+0000","last_change":"2024-09-19T08:38:18.392567+0000","last_active":"2024-09-19T08:38:20.425026+0000","last_peered":"2024-09-19T08:38:20.425026+0000","last_clean":"2024-09-19T08:38:20.425026+0000","last_became_active":"2024-09-19T08:38:18.387256+0000","last_became_peered":"2024-09-19T08:38:18.387256+0000","last_unstale":"2024-09-19T08:38:20.425026+0000","last_undegraded":"2024-09-19T08:38:20.425026+0000","last_fullsized":"2024-09-19T08:38:20.425026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:12:57.094603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.087052+0000","last_change":"2024-09-19T08:38:19.406115+0000","last_active":"2024-09-19T08:38:22.087052+0000","last_peered":"2024-09-19T08:38:22.087052+0000","last_clean":"2024-09-19T08:38:22.087052+0000","last_became_active":"2024-09-19T08:38:19.401366+0000","last_became_peered":"2024-09-19T08:38:19.401366+0000","last_unstale":"2024-09-19T08:38:22.087052+0000","last_undegraded":"2024-09-19T08:38:22.087052+0000","last_fullsized":"2024-09-19T08:38:22.087052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:08.505022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.493561+0000","last_change":"2024-09-19T08:38:18.384552+0000","last_active":"2024-09-19T08:38:21.493561+0000","last_peered":"2024-09-19T08:38:21.493561+0000","last_clean":"2024-09-19T08:38:21.493561+0000","last_became_active":"2024-09-19T08:38:18.383834+0000","last_became_peered":"2024-09-19T08:38:18.383834+0000","last_unstale":"2024-09-19T08:38:21.493561+0000","last_undegraded":"2024-09-19T08:38:21.493561+0000","last_fullsized":"2024-09-19T08:38:21.493561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:20.637060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085218+0000","last_change":"2024-09-19T08:38:19.395111+0000","last_active":"2024-09-19T08:38:22.085218+0000","last_peered":"2024-09-19T08:38:22.085218+0000","last_clean":"2024-09-19T08:38:22.085218+0000","last_became_active":"2024-09-19T08:38:19.394398+0000","last_became_peered":"2024-09-19T08:38:19.394398+0000","last_unstale":"2024-09-19T08:38:22.085218+0000","last_undegraded":"2024-09-19T08:38:22.085218+0000","last_fullsized":"2024-09-19T08:38:22.085218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:49.429936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.376132+0000","last_change":"2024-09-19T08:38:18.378231+0000","last_active":"2024-09-19T08:38:20.376132+0000","last_peered":"2024-09-19T08:38:20.376132+0000","last_clean":"2024-09-19T08:38:20.376132+0000","last_became_active":"2024-09-19T08:38:18.377883+0000","last_became_peered":"2024-09-19T08:38:18.377883+0000","last_unstale":"2024-09-19T08:38:20.376132+0000","last_undegraded":"2024-09-19T08:38:20.376132+0000","last_fullsized":"2024-09-19T08:38:20.376132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:36.282825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085006+0000","last_change":"2024-09-19T08:38:19.392650+0000","last_active":"2024-09-19T08:38:22.085006+0000","last_peered":"2024-09-19T08:38:22.085006+0000","last_clean":"2024-09-19T08:38:22.085006+0000","last_became_active":"2024-09-19T08:38:19.391936+0000","last_became_peered":"2024-09-19T08:38:19.391936+0000","last_unstale":"2024-09-19T08:38:22.085006+0000","last_undegraded":"2024-09-19T08:38:22.085006+0000","last_fullsized":"2024-09-19T08:38:22.085006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:42:24.499497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.376387+0000","last_change":"2024-09-19T08:38:18.359251+0000","last_active":"2024-09-19T08:38:20.376387+0000","last_peered":"2024-09-19T08:38:20.376387+0000","last_clean":"2024-09-19T08:38:20.376387+0000","last_became_active":"2024-09-19T08:38:18.358873+0000","last_became_peered":"2024-09-19T08:38:18.358873+0000","last_unstale":"2024-09-19T08:38:20.376387+0000","last_undegraded":"2024-09-19T08:38:20.376387+0000","last_fullsized":"2024-09-19T08:38:20.376387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:20.637060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085621+0000","last_change":"2024-09-19T08:38:19.414125+0000","last_active":"2024-09-19T08:38:22.085621+0000","last_peered":"2024-09-19T08:38:22.085621+0000","last_clean":"2024-09-19T08:38:22.085621+0000","last_became_active":"2024-09-19T08:38:19.413714+0000","last_became_peered":"2024-09-19T08:38:19.413714+0000","last_unstale":"2024-09-19T08:38:22.085621+0000","last_undegraded":"2024-09-19T08:38:22.085621+0000","last_fullsized":"2024-09-19T08:38:22.085621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:39:22.441525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.494666+0000","last_change":"2024-09-19T08:38:18.387311+0000","last_active":"2024-09-19T08:38:21.494666+0000","last_peered":"2024-09-19T08:38:21.494666+0000","last_clean":"2024-09-19T08:38:21.494666+0000","last_became_active":"2024-09-19T08:38:18.386340+0000","last_became_peered":"2024-09-19T08:38:18.386340+0000","last_unstale":"2024-09-19T08:38:21.494666+0000","last_undegraded":"2024-09-19T08:38:21.494666+0000","last_fullsized":"2024-09-19T08:38:21.494666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:36.282825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424982+0000","last_change":"2024-09-19T08:38:19.407976+0000","last_active":"2024-09-19T08:38:20.424982+0000","last_peered":"2024-09-19T08:38:20.424982+0000","last_clean":"2024-09-19T08:38:20.424982+0000","last_became_active":"2024-09-19T08:38:19.407198+0000","last_became_peered":"2024-09-19T08:38:19.407198+0000","last_unstale":"2024-09-19T08:38:20.424982+0000","last_undegraded":"2024-09-19T08:38:20.424982+0000","last_fullsized":"2024-09-19T08:38:20.424982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:42:24.499497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.385369+0000","last_change":"2024-09-19T08:38:19.413947+0000","last_active":"2024-09-19T08:38:20.385369+0000","last_peered":"2024-09-19T08:38:20.385369+0000","last_clean":"2024-09-19T08:38:20.385369+0000","last_became_active":"2024-09-19T08:38:19.413225+0000","last_became_peered":"2024-09-19T08:38:19.413225+0000","last_unstale":"2024-09-19T08:38:20.385369+0000","last_undegraded":"2024-09-19T08:38:20.385369+0000","last_fullsized":"2024-09-19T08:38:20.385369+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:02:31.580648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.494149+0000","last_change":"2024-09-19T08:38:18.400304+0000","last_active":"2024-09-19T08:38:21.494149+0000","last_peered":"2024-09-19T08:38:21.494149+0000","last_clean":"2024-09-19T08:38:21.494149+0000","last_became_active":"2024-09-19T08:38:18.399609+0000","last_became_peered":"2024-09-19T08:38:18.399609+0000","last_unstale":"2024-09-19T08:38:21.494149+0000","last_undegraded":"2024-09-19T08:38:21.494149+0000","last_fullsized":"2024-09-19T08:38:21.494149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:48.426450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.385061+0000","last_change":"2024-09-19T08:38:19.429826+0000","last_active":"2024-09-19T08:38:20.385061+0000","last_peered":"2024-09-19T08:38:20.385061+0000","last_clean":"2024-09-19T08:38:20.385061+0000","last_became_active":"2024-09-19T08:38:19.429543+0000","last_became_peered":"2024-09-19T08:38:19.429543+0000","last_unstale":"2024-09-19T08:38:20.385061+0000","last_undegraded":"2024-09-19T08:38:20.385061+0000","last_fullsized":"2024-09-19T08:38:20.385061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:44:57.127592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.494455+0000","last_change":"2024-09-19T08:38:18.384861+0000","last_active":"2024-09-19T08:38:21.494455+0000","last_peered":"2024-09-19T08:38:21.494455+0000","last_clean":"2024-09-19T08:38:21.494455+0000","last_became_active":"2024-09-19T08:38:18.384070+0000","last_became_peered":"2024-09-19T08:38:18.384070+0000","last_unstale":"2024-09-19T08:38:21.494455+0000","last_undegraded":"2024-09-19T08:38:21.494455+0000","last_fullsized":"2024-09-19T08:38:21.494455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:35:08.394753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.087092+0000","last_change":"2024-09-19T08:38:19.430701+0000","last_active":"2024-09-19T08:38:22.087092+0000","last_peered":"2024-09-19T08:38:22.087092+0000","last_clean":"2024-09-19T08:38:22.087092+0000","last_became_active":"2024-09-19T08:38:19.430335+0000","last_became_peered":"2024-09-19T08:38:19.430335+0000","last_unstale":"2024-09-19T08:38:22.087092+0000","last_undegraded":"2024-09-19T08:38:22.087092+0000","last_fullsized":"2024-09-19T08:38:22.087092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:07.305805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085411+0000","last_change":"2024-09-19T08:38:18.358117+0000","last_active":"2024-09-19T08:38:22.085411+0000","last_peered":"2024-09-19T08:38:22.085411+0000","last_clean":"2024-09-19T08:38:22.085411+0000","last_became_active":"2024-09-19T08:38:18.357791+0000","last_became_peered":"2024-09-19T08:38:18.357791+0000","last_unstale":"2024-09-19T08:38:22.085411+0000","last_undegraded":"2024-09-19T08:38:22.085411+0000","last_fullsized":"2024-09-19T08:38:22.085411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:00.838359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085487+0000","last_change":"2024-09-19T08:38:19.396783+0000","last_active":"2024-09-19T08:38:22.085487+0000","last_peered":"2024-09-19T08:38:22.085487+0000","last_clean":"2024-09-19T08:38:22.085487+0000","last_became_active":"2024-09-19T08:38:19.396012+0000","last_became_peered":"2024-09-19T08:38:19.396012+0000","last_unstale":"2024-09-19T08:38:22.085487+0000","last_undegraded":"2024-09-19T08:38:22.085487+0000","last_fullsized":"2024-09-19T08:38:22.085487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:20:35.891069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.087284+0000","last_change":"2024-09-19T08:38:18.380235+0000","last_active":"2024-09-19T08:38:22.087284+0000","last_peered":"2024-09-19T08:38:22.087284+0000","last_clean":"2024-09-19T08:38:22.087284+0000","last_became_active":"2024-09-19T08:38:18.380044+0000","last_became_peered":"2024-09-19T08:38:18.380044+0000","last_unstale":"2024-09-19T08:38:22.087284+0000","last_undegraded":"2024-09-19T08:38:22.087284+0000","last_fullsized":"2024-09-19T08:38:22.087284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:30.341445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":806912,"data_stored":663417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509449,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":72557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509449,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":72593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":72467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":217088,"data_stored":72627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":815104,"data_stored":663473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":74079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":74079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:38:32.029 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:38:32.177 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.175+0000 7f9910077640 1 Processor -- start 2024-09-19T08:38:32.177 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.175+0000 7f9910077640 1 -- start start 2024-09-19T08:38:32.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.179+0000 7f9910077640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f99080b7f80 0x7f990815f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:32.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.179+0000 7f9910077640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f99080b9970 0x7f99080b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:32.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.179+0000 7f9910077640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9908068440 con 0x7f99080d4820 2024-09-19T08:38:32.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.179+0000 7f9910077640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f9908163050 con 0x7f99080b9970 2024-09-19T08:38:32.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.179+0000 7f9910077640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f9908162a90 con 0x7f99080b7f80 2024-09-19T08:38:32.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.179+0000 7f990e614640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f99080b9970 0x7f99080b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:32.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.179+0000 7f990d612640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f99080d4820 0x7f99081621b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:60674/0 (socket says 172.21.15.98:60674) 2024-09-19T08:38:32.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.179+0000 7f990e614640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f99080b9970 0x7f99080b9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38104/0 (socket says 172.21.15.98:38104) 2024-09-19T08:38:32.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.179+0000 7f990d612640 1 -- 172.21.15.98:0/3726768909 learned_addr learned my addr 172.21.15.98:0/3726768909 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:32.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.179+0000 7f990de13640 1 --2- 172.21.15.98:0/3726768909 >> v2:172.21.15.139:3300/0 conn(0x7f99080b7f80 0x7f990815f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:32.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.179+0000 7f990e614640 1 -- 172.21.15.98:0/3726768909 >> v2:172.21.15.139:3300/0 conn(0x7f99080b7f80 msgr2=0x7f990815f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:32.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.179+0000 7f990e614640 1 --2- 172.21.15.98:0/3726768909 >> v2:172.21.15.139:3300/0 conn(0x7f99080b7f80 0x7f990815f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:32.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.179+0000 7f990e614640 1 -- 172.21.15.98:0/3726768909 >> v1:172.21.15.98:6789/0 conn(0x7f99080d4820 legacy=0x7f99081621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:32.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.179+0000 7f990e614640 1 -- 172.21.15.98:0/3726768909 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9908162d70 con 0x7f99080b9970 2024-09-19T08:38:32.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.179+0000 7f990de13640 1 --2- 172.21.15.98:0/3726768909 >> v2:172.21.15.139:3300/0 conn(0x7f99080b7f80 0x7f990815f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:32.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.179+0000 7f990e614640 1 --2- 172.21.15.98:0/3726768909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99080b9970 0x7f99080b9de0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f99080d0dd0 tx=0x7f990001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0eb677179fe7282 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:32.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.179+0000 7f990ce11640 1 -- 172.21.15.98:0/3726768909 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99000b2070 con 0x7f99080b9970 2024-09-19T08:38:32.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.179+0000 7f990ce11640 1 -- 172.21.15.98:0/3726768909 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99000a57f0 con 0x7f99080b9970 2024-09-19T08:38:32.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.179+0000 7f990ce11640 1 -- 172.21.15.98:0/3726768909 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99000a5bc0 con 0x7f99080b9970 2024-09-19T08:38:32.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.179+0000 7f9910077640 1 -- 172.21.15.98:0/3726768909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99080b9970 msgr2=0x7f99080b9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:32.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.179+0000 7f9910077640 1 --2- 172.21.15.98:0/3726768909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99080b9970 0x7f99080b9de0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f99080d0dd0 tx=0x7f990001c930 comp rx=0 tx=0).stop 2024-09-19T08:38:32.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.179+0000 7f9910077640 1 -- 172.21.15.98:0/3726768909 shutdown_connections 2024-09-19T08:38:32.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.179+0000 7f9910077640 1 --2- 172.21.15.98:0/3726768909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99080b9970 0x7f99080b9de0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:32.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.179+0000 7f9910077640 1 --2- 172.21.15.98:0/3726768909 >> v2:172.21.15.139:3300/0 conn(0x7f99080b7f80 0x7f990815f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:32.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.179+0000 7f9910077640 1 -- 172.21.15.98:0/3726768909 >> 172.21.15.98:0/3726768909 conn(0x7f9908074e40 msgr2=0x7f9908075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:32.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.179+0000 7f9910077640 1 -- 172.21.15.98:0/3726768909 shutdown_connections 2024-09-19T08:38:32.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.179+0000 7f9910077640 1 -- 172.21.15.98:0/3726768909 wait complete. 2024-09-19T08:38:32.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.183+0000 7f9910077640 1 Processor -- start 2024-09-19T08:38:32.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.183+0000 7f9910077640 1 -- start start 2024-09-19T08:38:32.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.183+0000 7f9910077640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99080b7f80 0x7f990815fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:32.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.183+0000 7f9910077640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99080b9970 0x7f9908152a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:32.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.183+0000 7f9910077640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99080d4820 0x7f990813c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:32.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.183+0000 7f9910077640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f99080b84e0 con 0x7f99080b7f80 2024-09-19T08:38:32.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.183+0000 7f9910077640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f99080b8360 con 0x7f99080d4820 2024-09-19T08:38:32.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.183+0000 7f9910077640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9908144590 con 0x7f99080b9970 2024-09-19T08:38:32.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.183+0000 7f990de13640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99080b7f80 0x7f990815fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:32.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.183+0000 7f990de13640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99080b7f80 0x7f990815fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38118/0 (socket says 172.21.15.98:38118) 2024-09-19T08:38:32.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.183+0000 7f990de13640 1 -- 172.21.15.98:0/1926931999 learned_addr learned my addr 172.21.15.98:0/1926931999 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:32.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.183+0000 7f990d612640 1 --2- 172.21.15.98:0/1926931999 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99080b9970 0x7f9908152a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:32.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.183+0000 7f990de13640 1 -- 172.21.15.98:0/1926931999 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99080b9970 msgr2=0x7f9908152a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:32.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.183+0000 7f990e614640 1 --2- 172.21.15.98:0/1926931999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99080d4820 0x7f990813c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:32.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.183+0000 7f990de13640 1 --2- 172.21.15.98:0/1926931999 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99080b9970 0x7f9908152a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:32.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.183+0000 7f990de13640 1 -- 172.21.15.98:0/1926931999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99080d4820 msgr2=0x7f990813c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:32.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.183+0000 7f990de13640 1 --2- 172.21.15.98:0/1926931999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99080d4820 0x7f990813c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:32.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.183+0000 7f990de13640 1 -- 172.21.15.98:0/1926931999 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9908160ba0 con 0x7f99080b7f80 2024-09-19T08:38:32.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.183+0000 7f990de13640 1 --2- 172.21.15.98:0/1926931999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99080b7f80 0x7f990815fe00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f990400c7f0 tx=0x7f990400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:32.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.183+0000 7f98fe7fc640 1 -- 172.21.15.98:0/1926931999 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9904015070 con 0x7f99080b7f80 2024-09-19T08:38:32.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.183+0000 7f98fe7fc640 1 -- 172.21.15.98:0/1926931999 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9904007820 con 0x7f99080b7f80 2024-09-19T08:38:32.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.183+0000 7f9910077640 1 -- 172.21.15.98:0/1926931999 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99081395c0 con 0x7f99080b7f80 2024-09-19T08:38:32.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.183+0000 7f98fe7fc640 1 -- 172.21.15.98:0/1926931999 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9904007c10 con 0x7f99080b7f80 2024-09-19T08:38:32.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.183+0000 7f9910077640 1 -- 172.21.15.98:0/1926931999 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9908160340 con 0x7f99080b7f80 2024-09-19T08:38:32.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.183+0000 7f9910077640 1 -- 172.21.15.98:0/1926931999 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99080681e0 con 0x7f99080b7f80 2024-09-19T08:38:32.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.183+0000 7f98fe7fc640 1 -- 172.21.15.98:0/1926931999 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9904018070 con 0x7f99080b7f80 2024-09-19T08:38:32.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.187+0000 7f98fe7fc640 1 --2- 172.21.15.98:0/1926931999 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f98ec0678c0 0x7f98ec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:32.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.187+0000 7f98fe7fc640 1 -- 172.21.15.98:0/1926931999 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9904085b80 con 0x7f99080b7f80 2024-09-19T08:38:32.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.187+0000 7f990d612640 1 --2- 172.21.15.98:0/1926931999 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f98ec0678c0 0x7f98ec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:32.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.187+0000 7f990d612640 1 --2- 172.21.15.98:0/1926931999 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f98ec0678c0 0x7f98ec069d80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f99080d24b0 tx=0x7f98f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:32.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.191+0000 7f98fe7fc640 1 -- 172.21.15.98:0/1926931999 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9904056750 con 0x7f99080b7f80 2024-09-19T08:38:32.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.335+0000 7f9910077640 1 -- 172.21.15.98:0/1926931999 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9908139cd0 con 0x7f98ec0678c0 2024-09-19T08:38:32.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.343+0000 7f98fe7fc640 1 -- 172.21.15.98:0/1926931999 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251837 (secure 0 0 0) 0x7f9908139cd0 con 0x7f98ec0678c0 2024-09-19T08:38:32.348 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:38:32.349 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:38:32.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.347+0000 7f9910077640 1 -- 172.21.15.98:0/1926931999 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f98ec0678c0 msgr2=0x7f98ec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:32.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.347+0000 7f9910077640 1 --2- 172.21.15.98:0/1926931999 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f98ec0678c0 0x7f98ec069d80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f99080d24b0 tx=0x7f98f8009040 comp rx=0 tx=0).stop 2024-09-19T08:38:32.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.347+0000 7f9910077640 1 -- 172.21.15.98:0/1926931999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99080b7f80 msgr2=0x7f990815fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:32.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.347+0000 7f9910077640 1 --2- 172.21.15.98:0/1926931999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99080b7f80 0x7f990815fe00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f990400c7f0 tx=0x7f990400ccb0 comp rx=0 tx=0).stop 2024-09-19T08:38:32.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.347+0000 7f9910077640 1 -- 172.21.15.98:0/1926931999 shutdown_connections 2024-09-19T08:38:32.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.351+0000 7f9910077640 1 --2- 172.21.15.98:0/1926931999 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f98ec0678c0 0x7f98ec069d80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:32.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.351+0000 7f9910077640 1 --2- 172.21.15.98:0/1926931999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99080d4820 0x7f990813c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:32.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.351+0000 7f9910077640 1 --2- 172.21.15.98:0/1926931999 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99080b9970 0x7f9908152a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:32.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.351+0000 7f9910077640 1 --2- 172.21.15.98:0/1926931999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99080b7f80 0x7f990815fe00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:32.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.351+0000 7f9910077640 1 -- 172.21.15.98:0/1926931999 >> 172.21.15.98:0/1926931999 conn(0x7f9908074e40 msgr2=0x7f990813b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:32.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.351+0000 7f9910077640 1 -- 172.21.15.98:0/1926931999 shutdown_connections 2024-09-19T08:38:32.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.351+0000 7f9910077640 1 -- 172.21.15.98:0/1926931999 wait complete. 2024-09-19T08:38:32.365 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-19T08:38:32.030226+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219100,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755620,"statfs":{"total":773094113280,"available":772869754880,"internally_reserved":0,"allocated":2924544,"data_stored":1765292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.657126"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.495053+0000","last_change":"2024-09-19T08:38:19.406406+0000","last_active":"2024-09-19T08:38:21.495053+0000","last_peered":"2024-09-19T08:38:21.495053+0000","last_clean":"2024-09-19T08:38:21.495053+0000","last_became_active":"2024-09-19T08:38:19.405760+0000","last_became_peered":"2024-09-19T08:38:19.405760+0000","last_unstale":"2024-09-19T08:38:21.495053+0000","last_undegraded":"2024-09-19T08:38:21.495053+0000","last_fullsized":"2024-09-19T08:38:21.495053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:26.845220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.376816+0000","last_change":"2024-09-19T08:38:19.406176+0000","last_active":"2024-09-19T08:38:20.376816+0000","last_peered":"2024-09-19T08:38:20.376816+0000","last_clean":"2024-09-19T08:38:20.376816+0000","last_became_active":"2024-09-19T08:38:19.405604+0000","last_became_peered":"2024-09-19T08:38:19.405604+0000","last_unstale":"2024-09-19T08:38:20.376816+0000","last_undegraded":"2024-09-19T08:38:20.376816+0000","last_fullsized":"2024-09-19T08:38:20.376816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:47:44.536264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.086412+0000","last_change":"2024-09-19T08:38:19.400726+0000","last_active":"2024-09-19T08:38:22.086412+0000","last_peered":"2024-09-19T08:38:22.086412+0000","last_clean":"2024-09-19T08:38:22.086412+0000","last_became_active":"2024-09-19T08:38:19.399956+0000","last_became_peered":"2024-09-19T08:38:19.399956+0000","last_unstale":"2024-09-19T08:38:22.086412+0000","last_undegraded":"2024-09-19T08:38:22.086412+0000","last_fullsized":"2024-09-19T08:38:22.086412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:17:03.830010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.384590+0000","last_change":"2024-09-19T08:38:19.425899+0000","last_active":"2024-09-19T08:38:20.384590+0000","last_peered":"2024-09-19T08:38:20.384590+0000","last_clean":"2024-09-19T08:38:20.384590+0000","last_became_active":"2024-09-19T08:38:19.425622+0000","last_became_peered":"2024-09-19T08:38:19.425622+0000","last_unstale":"2024-09-19T08:38:20.384590+0000","last_undegraded":"2024-09-19T08:38:20.384590+0000","last_fullsized":"2024-09-19T08:38:20.384590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:47:44.536264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.494366+0000","last_change":"2024-09-19T08:38:19.406714+0000","last_active":"2024-09-19T08:38:21.494366+0000","last_peered":"2024-09-19T08:38:21.494366+0000","last_clean":"2024-09-19T08:38:21.494366+0000","last_became_active":"2024-09-19T08:38:19.404949+0000","last_became_peered":"2024-09-19T08:38:19.404949+0000","last_unstale":"2024-09-19T08:38:21.494366+0000","last_undegraded":"2024-09-19T08:38:21.494366+0000","last_fullsized":"2024-09-19T08:38:21.494366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:02:31.580648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.384530+0000","last_change":"2024-09-19T08:38:19.426925+0000","last_active":"2024-09-19T08:38:20.384530+0000","last_peered":"2024-09-19T08:38:20.384530+0000","last_clean":"2024-09-19T08:38:20.384530+0000","last_became_active":"2024-09-19T08:38:19.426666+0000","last_became_peered":"2024-09-19T08:38:19.426666+0000","last_unstale":"2024-09-19T08:38:20.384530+0000","last_undegraded":"2024-09-19T08:38:20.384530+0000","last_fullsized":"2024-09-19T08:38:20.384530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:02:54.292567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.493827+0000","last_change":"2024-09-19T08:38:19.406462+0000","last_active":"2024-09-19T08:38:21.493827+0000","last_peered":"2024-09-19T08:38:21.493827+0000","last_clean":"2024-09-19T08:38:21.493827+0000","last_became_active":"2024-09-19T08:38:19.404851+0000","last_became_peered":"2024-09-19T08:38:19.404851+0000","last_unstale":"2024-09-19T08:38:21.493827+0000","last_undegraded":"2024-09-19T08:38:21.493827+0000","last_fullsized":"2024-09-19T08:38:21.493827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:02:54.292567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.084591+0000","last_change":"2024-09-19T08:38:19.400917+0000","last_active":"2024-09-19T08:38:22.084591+0000","last_peered":"2024-09-19T08:38:22.084591+0000","last_clean":"2024-09-19T08:38:22.084591+0000","last_became_active":"2024-09-19T08:38:19.400188+0000","last_became_peered":"2024-09-19T08:38:19.400188+0000","last_unstale":"2024-09-19T08:38:22.084591+0000","last_undegraded":"2024-09-19T08:38:22.084591+0000","last_fullsized":"2024-09-19T08:38:22.084591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:24:38.033969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.495025+0000","last_change":"2024-09-19T08:38:19.432423+0000","last_active":"2024-09-19T08:38:21.495025+0000","last_peered":"2024-09-19T08:38:21.495025+0000","last_clean":"2024-09-19T08:38:21.495025+0000","last_became_active":"2024-09-19T08:38:19.431886+0000","last_became_peered":"2024-09-19T08:38:19.431886+0000","last_unstale":"2024-09-19T08:38:21.495025+0000","last_undegraded":"2024-09-19T08:38:21.495025+0000","last_fullsized":"2024-09-19T08:38:21.495025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:06.577636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.425359+0000","last_change":"2024-09-19T08:38:19.402509+0000","last_active":"2024-09-19T08:38:20.425359+0000","last_peered":"2024-09-19T08:38:20.425359+0000","last_clean":"2024-09-19T08:38:20.425359+0000","last_became_active":"2024-09-19T08:38:19.401833+0000","last_became_peered":"2024-09-19T08:38:19.401833+0000","last_unstale":"2024-09-19T08:38:20.425359+0000","last_undegraded":"2024-09-19T08:38:20.425359+0000","last_fullsized":"2024-09-19T08:38:20.425359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:08.065282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424901+0000","last_change":"2024-09-19T08:38:19.430286+0000","last_active":"2024-09-19T08:38:20.424901+0000","last_peered":"2024-09-19T08:38:20.424901+0000","last_clean":"2024-09-19T08:38:20.424901+0000","last_became_active":"2024-09-19T08:38:19.429901+0000","last_became_peered":"2024-09-19T08:38:19.429901+0000","last_unstale":"2024-09-19T08:38:20.424901+0000","last_undegraded":"2024-09-19T08:38:20.424901+0000","last_fullsized":"2024-09-19T08:38:20.424901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:31:42.999935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085451+0000","last_change":"2024-09-19T08:38:19.397867+0000","last_active":"2024-09-19T08:38:22.085451+0000","last_peered":"2024-09-19T08:38:22.085451+0000","last_clean":"2024-09-19T08:38:22.085451+0000","last_became_active":"2024-09-19T08:38:19.397106+0000","last_became_peered":"2024-09-19T08:38:19.397106+0000","last_unstale":"2024-09-19T08:38:22.085451+0000","last_undegraded":"2024-09-19T08:38:22.085451+0000","last_fullsized":"2024-09-19T08:38:22.085451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:37.286310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.086556+0000","last_change":"2024-09-19T08:38:19.396759+0000","last_active":"2024-09-19T08:38:22.086556+0000","last_peered":"2024-09-19T08:38:22.086556+0000","last_clean":"2024-09-19T08:38:22.086556+0000","last_became_active":"2024-09-19T08:38:19.395976+0000","last_became_peered":"2024-09-19T08:38:19.395976+0000","last_unstale":"2024-09-19T08:38:22.086556+0000","last_undegraded":"2024-09-19T08:38:22.086556+0000","last_fullsized":"2024-09-19T08:38:22.086556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:20.116184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.084526+0000","last_change":"2024-09-19T08:38:19.405170+0000","last_active":"2024-09-19T08:38:22.084526+0000","last_peered":"2024-09-19T08:38:22.084526+0000","last_clean":"2024-09-19T08:38:22.084526+0000","last_became_active":"2024-09-19T08:38:19.401487+0000","last_became_peered":"2024-09-19T08:38:19.401487+0000","last_unstale":"2024-09-19T08:38:22.084526+0000","last_undegraded":"2024-09-19T08:38:22.084526+0000","last_fullsized":"2024-09-19T08:38:22.084526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:40.803843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.376223+0000","last_change":"2024-09-19T08:38:19.380536+0000","last_active":"2024-09-19T08:38:20.376223+0000","last_peered":"2024-09-19T08:38:20.376223+0000","last_clean":"2024-09-19T08:38:20.376223+0000","last_became_active":"2024-09-19T08:38:19.375034+0000","last_became_peered":"2024-09-19T08:38:19.375034+0000","last_unstale":"2024-09-19T08:38:20.376223+0000","last_undegraded":"2024-09-19T08:38:20.376223+0000","last_fullsized":"2024-09-19T08:38:20.376223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:07.305805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.384258+0000","last_change":"2024-09-19T08:38:19.426216+0000","last_active":"2024-09-19T08:38:20.384258+0000","last_peered":"2024-09-19T08:38:20.384258+0000","last_clean":"2024-09-19T08:38:20.384258+0000","last_became_active":"2024-09-19T08:38:19.425926+0000","last_became_peered":"2024-09-19T08:38:19.425926+0000","last_unstale":"2024-09-19T08:38:20.384258+0000","last_undegraded":"2024-09-19T08:38:20.384258+0000","last_fullsized":"2024-09-19T08:38:20.384258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:47.390332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.385670+0000","last_change":"2024-09-19T08:38:19.427199+0000","last_active":"2024-09-19T08:38:20.385670+0000","last_peered":"2024-09-19T08:38:20.385670+0000","last_clean":"2024-09-19T08:38:20.385670+0000","last_became_active":"2024-09-19T08:38:19.425571+0000","last_became_peered":"2024-09-19T08:38:19.425571+0000","last_unstale":"2024-09-19T08:38:20.385670+0000","last_undegraded":"2024-09-19T08:38:20.385670+0000","last_fullsized":"2024-09-19T08:38:20.385670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:51:44.907657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.084606+0000","last_change":"2024-09-19T08:38:19.394500+0000","last_active":"2024-09-19T08:38:22.084606+0000","last_peered":"2024-09-19T08:38:22.084606+0000","last_clean":"2024-09-19T08:38:22.084606+0000","last_became_active":"2024-09-19T08:38:19.393782+0000","last_became_peered":"2024-09-19T08:38:19.393782+0000","last_unstale":"2024-09-19T08:38:22.084606+0000","last_undegraded":"2024-09-19T08:38:22.084606+0000","last_fullsized":"2024-09-19T08:38:22.084606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:35:09.398239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.494842+0000","last_change":"2024-09-19T08:38:19.412148+0000","last_active":"2024-09-19T08:38:21.494842+0000","last_peered":"2024-09-19T08:38:21.494842+0000","last_clean":"2024-09-19T08:38:21.494842+0000","last_became_active":"2024-09-19T08:38:19.410061+0000","last_became_peered":"2024-09-19T08:38:19.410061+0000","last_unstale":"2024-09-19T08:38:21.494842+0000","last_undegraded":"2024-09-19T08:38:21.494842+0000","last_fullsized":"2024-09-19T08:38:21.494842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:51:44.907657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.425280+0000","last_change":"2024-09-19T08:38:19.389509+0000","last_active":"2024-09-19T08:38:20.425280+0000","last_peered":"2024-09-19T08:38:20.425280+0000","last_clean":"2024-09-19T08:38:20.425280+0000","last_became_active":"2024-09-19T08:38:19.388736+0000","last_became_peered":"2024-09-19T08:38:19.388736+0000","last_unstale":"2024-09-19T08:38:20.425280+0000","last_undegraded":"2024-09-19T08:38:20.425280+0000","last_fullsized":"2024-09-19T08:38:20.425280+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:20:35.891069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.086457+0000","last_change":"2024-09-19T08:38:19.415448+0000","last_active":"2024-09-19T08:38:22.086457+0000","last_peered":"2024-09-19T08:38:22.086457+0000","last_clean":"2024-09-19T08:38:22.086457+0000","last_became_active":"2024-09-19T08:38:19.415169+0000","last_became_peered":"2024-09-19T08:38:19.415169+0000","last_unstale":"2024-09-19T08:38:22.086457+0000","last_undegraded":"2024-09-19T08:38:22.086457+0000","last_fullsized":"2024-09-19T08:38:22.086457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:18:23.928102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.384606+0000","last_change":"2024-09-19T08:38:19.429495+0000","last_active":"2024-09-19T08:38:20.384606+0000","last_peered":"2024-09-19T08:38:20.384606+0000","last_clean":"2024-09-19T08:38:20.384606+0000","last_became_active":"2024-09-19T08:38:19.427201+0000","last_became_peered":"2024-09-19T08:38:19.427201+0000","last_unstale":"2024-09-19T08:38:20.384606+0000","last_undegraded":"2024-09-19T08:38:20.384606+0000","last_fullsized":"2024-09-19T08:38:20.384606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:08.554428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.084821+0000","last_change":"2024-09-19T08:38:19.406260+0000","last_active":"2024-09-19T08:38:22.084821+0000","last_peered":"2024-09-19T08:38:22.084821+0000","last_clean":"2024-09-19T08:38:22.084821+0000","last_became_active":"2024-09-19T08:38:19.405769+0000","last_became_peered":"2024-09-19T08:38:19.405769+0000","last_unstale":"2024-09-19T08:38:22.084821+0000","last_undegraded":"2024-09-19T08:38:22.084821+0000","last_fullsized":"2024-09-19T08:38:22.084821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:31:42.999935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.384446+0000","last_change":"2024-09-19T08:38:19.402246+0000","last_active":"2024-09-19T08:38:20.384446+0000","last_peered":"2024-09-19T08:38:20.384446+0000","last_clean":"2024-09-19T08:38:20.384446+0000","last_became_active":"2024-09-19T08:38:19.401487+0000","last_became_peered":"2024-09-19T08:38:19.401487+0000","last_unstale":"2024-09-19T08:38:20.384446+0000","last_undegraded":"2024-09-19T08:38:20.384446+0000","last_fullsized":"2024-09-19T08:38:20.384446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:54:01.480300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085580+0000","last_change":"2024-09-19T08:38:19.406201+0000","last_active":"2024-09-19T08:38:22.085580+0000","last_peered":"2024-09-19T08:38:22.085580+0000","last_clean":"2024-09-19T08:38:22.085580+0000","last_became_active":"2024-09-19T08:38:19.405583+0000","last_became_peered":"2024-09-19T08:38:19.405583+0000","last_unstale":"2024-09-19T08:38:22.085580+0000","last_undegraded":"2024-09-19T08:38:22.085580+0000","last_fullsized":"2024-09-19T08:38:22.085580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:08.065282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.425374+0000","last_change":"2024-09-19T08:38:19.411842+0000","last_active":"2024-09-19T08:38:20.425374+0000","last_peered":"2024-09-19T08:38:20.425374+0000","last_clean":"2024-09-19T08:38:20.425374+0000","last_became_active":"2024-09-19T08:38:19.411092+0000","last_became_peered":"2024-09-19T08:38:19.411092+0000","last_unstale":"2024-09-19T08:38:20.425374+0000","last_undegraded":"2024-09-19T08:38:20.425374+0000","last_fullsized":"2024-09-19T08:38:20.425374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:39:22.441525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.376568+0000","last_change":"2024-09-19T08:38:19.393312+0000","last_active":"2024-09-19T08:38:20.376568+0000","last_peered":"2024-09-19T08:38:20.376568+0000","last_clean":"2024-09-19T08:38:20.376568+0000","last_became_active":"2024-09-19T08:38:19.392591+0000","last_became_peered":"2024-09-19T08:38:19.392591+0000","last_unstale":"2024-09-19T08:38:20.376568+0000","last_undegraded":"2024-09-19T08:38:20.376568+0000","last_fullsized":"2024-09-19T08:38:20.376568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:54:01.480300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085836+0000","last_change":"2024-09-19T08:38:19.408371+0000","last_active":"2024-09-19T08:38:22.085836+0000","last_peered":"2024-09-19T08:38:22.085836+0000","last_clean":"2024-09-19T08:38:22.085836+0000","last_became_active":"2024-09-19T08:38:19.407846+0000","last_became_peered":"2024-09-19T08:38:19.407846+0000","last_unstale":"2024-09-19T08:38:22.085836+0000","last_undegraded":"2024-09-19T08:38:22.085836+0000","last_fullsized":"2024-09-19T08:38:22.085836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:33.753905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.086518+0000","last_change":"2024-09-19T08:38:19.406826+0000","last_active":"2024-09-19T08:38:22.086518+0000","last_peered":"2024-09-19T08:38:22.086518+0000","last_clean":"2024-09-19T08:38:22.086518+0000","last_became_active":"2024-09-19T08:38:19.406456+0000","last_became_peered":"2024-09-19T08:38:19.406456+0000","last_unstale":"2024-09-19T08:38:22.086518+0000","last_undegraded":"2024-09-19T08:38:22.086518+0000","last_fullsized":"2024-09-19T08:38:22.086518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:32:19.829820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.384963+0000","last_change":"2024-09-19T08:38:19.430413+0000","last_active":"2024-09-19T08:38:20.384963+0000","last_peered":"2024-09-19T08:38:20.384963+0000","last_clean":"2024-09-19T08:38:20.384963+0000","last_became_active":"2024-09-19T08:38:19.430122+0000","last_became_peered":"2024-09-19T08:38:19.430122+0000","last_unstale":"2024-09-19T08:38:20.384963+0000","last_undegraded":"2024-09-19T08:38:20.384963+0000","last_fullsized":"2024-09-19T08:38:20.384963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:13:57.163258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.083388+0000","last_change":"2024-09-19T08:38:19.429646+0000","last_active":"2024-09-19T08:38:22.083388+0000","last_peered":"2024-09-19T08:38:22.083388+0000","last_clean":"2024-09-19T08:38:22.083388+0000","last_became_active":"2024-09-19T08:38:19.429256+0000","last_became_peered":"2024-09-19T08:38:19.429256+0000","last_unstale":"2024-09-19T08:38:22.083388+0000","last_undegraded":"2024-09-19T08:38:22.083388+0000","last_fullsized":"2024-09-19T08:38:22.083388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:37.286310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424603+0000","last_change":"2024-09-19T08:38:19.414469+0000","last_active":"2024-09-19T08:38:20.424603+0000","last_peered":"2024-09-19T08:38:20.424603+0000","last_clean":"2024-09-19T08:38:20.424603+0000","last_became_active":"2024-09-19T08:38:19.413995+0000","last_became_peered":"2024-09-19T08:38:19.413995+0000","last_unstale":"2024-09-19T08:38:20.424603+0000","last_undegraded":"2024-09-19T08:38:20.424603+0000","last_fullsized":"2024-09-19T08:38:20.424603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:24:38.033969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085027+0000","last_change":"2024-09-19T08:38:19.377658+0000","last_active":"2024-09-19T08:38:22.085027+0000","last_peered":"2024-09-19T08:38:22.085027+0000","last_clean":"2024-09-19T08:38:22.085027+0000","last_became_active":"2024-09-19T08:38:19.376916+0000","last_became_peered":"2024-09-19T08:38:19.376916+0000","last_unstale":"2024-09-19T08:38:22.085027+0000","last_undegraded":"2024-09-19T08:38:22.085027+0000","last_fullsized":"2024-09-19T08:38:22.085027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:21.640545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085745+0000","last_change":"2024-09-19T08:38:18.387502+0000","last_active":"2024-09-19T08:38:22.085745+0000","last_peered":"2024-09-19T08:38:22.085745+0000","last_clean":"2024-09-19T08:38:22.085745+0000","last_became_active":"2024-09-19T08:38:18.387154+0000","last_became_peered":"2024-09-19T08:38:18.387154+0000","last_unstale":"2024-09-19T08:38:22.085745+0000","last_undegraded":"2024-09-19T08:38:22.085745+0000","last_fullsized":"2024-09-19T08:38:22.085745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:53.708142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.086691+0000","last_change":"2024-09-19T08:38:19.408323+0000","last_active":"2024-09-19T08:38:22.086691+0000","last_peered":"2024-09-19T08:38:22.086691+0000","last_clean":"2024-09-19T08:38:22.086691+0000","last_became_active":"2024-09-19T08:38:19.407418+0000","last_became_peered":"2024-09-19T08:38:19.407418+0000","last_unstale":"2024-09-19T08:38:22.086691+0000","last_undegraded":"2024-09-19T08:38:22.086691+0000","last_fullsized":"2024-09-19T08:38:22.086691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:32:35.027328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424590+0000","last_change":"2024-09-19T08:38:18.381446+0000","last_active":"2024-09-19T08:38:20.424590+0000","last_peered":"2024-09-19T08:38:20.424590+0000","last_clean":"2024-09-19T08:38:20.424590+0000","last_became_active":"2024-09-19T08:38:18.380678+0000","last_became_peered":"2024-09-19T08:38:18.380678+0000","last_unstale":"2024-09-19T08:38:20.424590+0000","last_undegraded":"2024-09-19T08:38:20.424590+0000","last_fullsized":"2024-09-19T08:38:20.424590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:07:31.456618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.084969+0000","last_change":"2024-09-19T08:38:18.376428+0000","last_active":"2024-09-19T08:38:22.084969+0000","last_peered":"2024-09-19T08:38:22.084969+0000","last_clean":"2024-09-19T08:38:22.084969+0000","last_became_active":"2024-09-19T08:38:18.376015+0000","last_became_peered":"2024-09-19T08:38:18.376015+0000","last_unstale":"2024-09-19T08:38:22.084969+0000","last_undegraded":"2024-09-19T08:38:22.084969+0000","last_fullsized":"2024-09-19T08:38:22.084969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:49:33.943029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.084815+0000","last_change":"2024-09-19T08:38:19.414261+0000","last_active":"2024-09-19T08:38:22.084815+0000","last_peered":"2024-09-19T08:38:22.084815+0000","last_clean":"2024-09-19T08:38:22.084815+0000","last_became_active":"2024-09-19T08:38:19.413930+0000","last_became_peered":"2024-09-19T08:38:19.413930+0000","last_unstale":"2024-09-19T08:38:22.084815+0000","last_undegraded":"2024-09-19T08:38:22.084815+0000","last_fullsized":"2024-09-19T08:38:22.084815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:32.216306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.384529+0000","last_change":"2024-09-19T08:38:18.393405+0000","last_active":"2024-09-19T08:38:20.384529+0000","last_peered":"2024-09-19T08:38:20.384529+0000","last_clean":"2024-09-19T08:38:20.384529+0000","last_became_active":"2024-09-19T08:38:18.392613+0000","last_became_peered":"2024-09-19T08:38:18.392613+0000","last_unstale":"2024-09-19T08:38:20.384529+0000","last_undegraded":"2024-09-19T08:38:20.384529+0000","last_fullsized":"2024-09-19T08:38:20.384529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:55.536656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.376013+0000","last_change":"2024-09-19T08:38:19.397074+0000","last_active":"2024-09-19T08:38:20.376013+0000","last_peered":"2024-09-19T08:38:20.376013+0000","last_clean":"2024-09-19T08:38:20.376013+0000","last_became_active":"2024-09-19T08:38:19.396130+0000","last_became_peered":"2024-09-19T08:38:19.396130+0000","last_unstale":"2024-09-19T08:38:20.376013+0000","last_undegraded":"2024-09-19T08:38:20.376013+0000","last_fullsized":"2024-09-19T08:38:20.376013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:32:35.027328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424870+0000","last_change":"2024-09-19T08:38:18.398231+0000","last_active":"2024-09-19T08:38:20.424870+0000","last_peered":"2024-09-19T08:38:20.424870+0000","last_clean":"2024-09-19T08:38:20.424870+0000","last_became_active":"2024-09-19T08:38:18.397590+0000","last_became_peered":"2024-09-19T08:38:18.397590+0000","last_unstale":"2024-09-19T08:38:20.424870+0000","last_undegraded":"2024-09-19T08:38:20.424870+0000","last_fullsized":"2024-09-19T08:38:20.424870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:54:02.262454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.385857+0000","last_change":"2024-09-19T08:38:19.415962+0000","last_active":"2024-09-19T08:38:20.385857+0000","last_peered":"2024-09-19T08:38:20.385857+0000","last_clean":"2024-09-19T08:38:20.385857+0000","last_became_active":"2024-09-19T08:38:19.415579+0000","last_became_peered":"2024-09-19T08:38:19.415579+0000","last_unstale":"2024-09-19T08:38:20.385857+0000","last_undegraded":"2024-09-19T08:38:20.385857+0000","last_fullsized":"2024-09-19T08:38:20.385857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:06.577636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.385384+0000","last_change":"2024-09-19T08:38:18.393568+0000","last_active":"2024-09-19T08:38:20.385384+0000","last_peered":"2024-09-19T08:38:20.385384+0000","last_clean":"2024-09-19T08:38:20.385384+0000","last_became_active":"2024-09-19T08:38:18.390336+0000","last_became_peered":"2024-09-19T08:38:18.390336+0000","last_unstale":"2024-09-19T08:38:20.385384+0000","last_undegraded":"2024-09-19T08:38:20.385384+0000","last_fullsized":"2024-09-19T08:38:20.385384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:20.637060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.087146+0000","last_change":"2024-09-19T08:38:19.428816+0000","last_active":"2024-09-19T08:38:22.087146+0000","last_peered":"2024-09-19T08:38:22.087146+0000","last_clean":"2024-09-19T08:38:22.087146+0000","last_became_active":"2024-09-19T08:38:19.428391+0000","last_became_peered":"2024-09-19T08:38:19.428391+0000","last_unstale":"2024-09-19T08:38:22.087146+0000","last_undegraded":"2024-09-19T08:38:22.087146+0000","last_fullsized":"2024-09-19T08:38:22.087146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:47.390332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.376655+0000","last_change":"2024-09-19T08:38:18.390022+0000","last_active":"2024-09-19T08:38:20.376655+0000","last_peered":"2024-09-19T08:38:20.376655+0000","last_clean":"2024-09-19T08:38:20.376655+0000","last_became_active":"2024-09-19T08:38:18.389679+0000","last_became_peered":"2024-09-19T08:38:18.389679+0000","last_unstale":"2024-09-19T08:38:20.376655+0000","last_undegraded":"2024-09-19T08:38:20.376655+0000","last_fullsized":"2024-09-19T08:38:20.376655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:39.064483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085614+0000","last_change":"2024-09-19T08:38:19.397504+0000","last_active":"2024-09-19T08:38:22.085614+0000","last_peered":"2024-09-19T08:38:22.085614+0000","last_clean":"2024-09-19T08:38:22.085614+0000","last_became_active":"2024-09-19T08:38:19.396770+0000","last_became_peered":"2024-09-19T08:38:19.396770+0000","last_unstale":"2024-09-19T08:38:22.085614+0000","last_undegraded":"2024-09-19T08:38:22.085614+0000","last_fullsized":"2024-09-19T08:38:22.085614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:35:09.398239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424913+0000","last_change":"2024-09-19T08:38:18.371315+0000","last_active":"2024-09-19T08:38:20.424913+0000","last_peered":"2024-09-19T08:38:20.424913+0000","last_clean":"2024-09-19T08:38:20.424913+0000","last_became_active":"2024-09-19T08:38:18.370524+0000","last_became_peered":"2024-09-19T08:38:18.370524+0000","last_unstale":"2024-09-19T08:38:20.424913+0000","last_undegraded":"2024-09-19T08:38:20.424913+0000","last_fullsized":"2024-09-19T08:38:20.424913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:49:33.943029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.494199+0000","last_change":"2024-09-19T08:38:19.409150+0000","last_active":"2024-09-19T08:38:21.494199+0000","last_peered":"2024-09-19T08:38:21.494199+0000","last_clean":"2024-09-19T08:38:21.494199+0000","last_became_active":"2024-09-19T08:38:19.408303+0000","last_became_peered":"2024-09-19T08:38:19.408303+0000","last_unstale":"2024-09-19T08:38:21.494199+0000","last_undegraded":"2024-09-19T08:38:21.494199+0000","last_fullsized":"2024-09-19T08:38:21.494199+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:13:57.163258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.086837+0000","last_change":"2024-09-19T08:38:18.366653+0000","last_active":"2024-09-19T08:38:22.086837+0000","last_peered":"2024-09-19T08:38:22.086837+0000","last_clean":"2024-09-19T08:38:22.086837+0000","last_became_active":"2024-09-19T08:38:18.366387+0000","last_became_peered":"2024-09-19T08:38:18.366387+0000","last_unstale":"2024-09-19T08:38:22.086837+0000","last_undegraded":"2024-09-19T08:38:22.086837+0000","last_fullsized":"2024-09-19T08:38:22.086837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:52.328114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.086019+0000","last_change":"2024-09-19T08:38:19.415512+0000","last_active":"2024-09-19T08:38:22.086019+0000","last_peered":"2024-09-19T08:38:22.086019+0000","last_clean":"2024-09-19T08:38:22.086019+0000","last_became_active":"2024-09-19T08:38:19.415166+0000","last_became_peered":"2024-09-19T08:38:19.415166+0000","last_unstale":"2024-09-19T08:38:22.086019+0000","last_undegraded":"2024-09-19T08:38:22.086019+0000","last_fullsized":"2024-09-19T08:38:22.086019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:22.366004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.384829+0000","last_change":"2024-09-19T08:38:18.402871+0000","last_active":"2024-09-19T08:38:20.384829+0000","last_peered":"2024-09-19T08:38:20.384829+0000","last_clean":"2024-09-19T08:38:20.384829+0000","last_became_active":"2024-09-19T08:38:18.400331+0000","last_became_peered":"2024-09-19T08:38:18.400331+0000","last_unstale":"2024-09-19T08:38:20.384829+0000","last_undegraded":"2024-09-19T08:38:20.384829+0000","last_fullsized":"2024-09-19T08:38:20.384829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:39.064483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.494370+0000","last_change":"2024-09-19T08:38:19.406507+0000","last_active":"2024-09-19T08:38:21.494370+0000","last_peered":"2024-09-19T08:38:21.494370+0000","last_clean":"2024-09-19T08:38:21.494370+0000","last_became_active":"2024-09-19T08:38:19.405133+0000","last_became_peered":"2024-09-19T08:38:19.405133+0000","last_unstale":"2024-09-19T08:38:21.494370+0000","last_undegraded":"2024-09-19T08:38:21.494370+0000","last_fullsized":"2024-09-19T08:38:21.494370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:30:08.777799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424461+0000","last_change":"2024-09-19T08:38:18.396642+0000","last_active":"2024-09-19T08:38:20.424461+0000","last_peered":"2024-09-19T08:38:20.424461+0000","last_clean":"2024-09-19T08:38:20.424461+0000","last_became_active":"2024-09-19T08:38:18.396009+0000","last_became_peered":"2024-09-19T08:38:18.396009+0000","last_unstale":"2024-09-19T08:38:20.424461+0000","last_undegraded":"2024-09-19T08:38:20.424461+0000","last_fullsized":"2024-09-19T08:38:20.424461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:21.951846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424570+0000","last_change":"2024-09-19T08:38:19.409375+0000","last_active":"2024-09-19T08:38:20.424570+0000","last_peered":"2024-09-19T08:38:20.424570+0000","last_clean":"2024-09-19T08:38:20.424570+0000","last_became_active":"2024-09-19T08:38:19.408667+0000","last_became_peered":"2024-09-19T08:38:19.408667+0000","last_unstale":"2024-09-19T08:38:20.424570+0000","last_undegraded":"2024-09-19T08:38:20.424570+0000","last_fullsized":"2024-09-19T08:38:20.424570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:32.216306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.082894+0000","last_change":"2024-09-19T08:38:18.360672+0000","last_active":"2024-09-19T08:38:22.082894+0000","last_peered":"2024-09-19T08:38:22.082894+0000","last_clean":"2024-09-19T08:38:22.082894+0000","last_became_active":"2024-09-19T08:38:18.360210+0000","last_became_peered":"2024-09-19T08:38:18.360210+0000","last_unstale":"2024-09-19T08:38:22.082894+0000","last_undegraded":"2024-09-19T08:38:22.082894+0000","last_fullsized":"2024-09-19T08:38:22.082894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:20.637060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.493833+0000","last_change":"2024-09-19T08:38:19.407114+0000","last_active":"2024-09-19T08:38:21.493833+0000","last_peered":"2024-09-19T08:38:21.493833+0000","last_clean":"2024-09-19T08:38:21.493833+0000","last_became_active":"2024-09-19T08:38:19.405734+0000","last_became_peered":"2024-09-19T08:38:19.405734+0000","last_unstale":"2024-09-19T08:38:21.493833+0000","last_undegraded":"2024-09-19T08:38:21.493833+0000","last_fullsized":"2024-09-19T08:38:21.493833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:08.554428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085716+0000","last_change":"2024-09-19T08:38:18.377025+0000","last_active":"2024-09-19T08:38:22.085716+0000","last_peered":"2024-09-19T08:38:22.085716+0000","last_clean":"2024-09-19T08:38:22.085716+0000","last_became_active":"2024-09-19T08:38:18.372574+0000","last_became_peered":"2024-09-19T08:38:18.372574+0000","last_unstale":"2024-09-19T08:38:22.085716+0000","last_undegraded":"2024-09-19T08:38:22.085716+0000","last_fullsized":"2024-09-19T08:38:22.085716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:30.341445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.494907+0000","last_change":"2024-09-19T08:38:19.405731+0000","last_active":"2024-09-19T08:38:21.494907+0000","last_peered":"2024-09-19T08:38:21.494907+0000","last_clean":"2024-09-19T08:38:21.494907+0000","last_became_active":"2024-09-19T08:38:19.404769+0000","last_became_peered":"2024-09-19T08:38:19.404769+0000","last_unstale":"2024-09-19T08:38:21.494907+0000","last_undegraded":"2024-09-19T08:38:21.494907+0000","last_fullsized":"2024-09-19T08:38:21.494907+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:21.788118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.084854+0000","last_change":"2024-09-19T08:38:18.366653+0000","last_active":"2024-09-19T08:38:22.084854+0000","last_peered":"2024-09-19T08:38:22.084854+0000","last_clean":"2024-09-19T08:38:22.084854+0000","last_became_active":"2024-09-19T08:38:18.366357+0000","last_became_peered":"2024-09-19T08:38:18.366357+0000","last_unstale":"2024-09-19T08:38:22.084854+0000","last_undegraded":"2024-09-19T08:38:22.084854+0000","last_fullsized":"2024-09-19T08:38:22.084854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:47.789309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.495241+0000","last_change":"2024-09-19T08:38:19.408858+0000","last_active":"2024-09-19T08:38:21.495241+0000","last_peered":"2024-09-19T08:38:21.495241+0000","last_clean":"2024-09-19T08:38:21.495241+0000","last_became_active":"2024-09-19T08:38:19.408122+0000","last_became_peered":"2024-09-19T08:38:19.408122+0000","last_unstale":"2024-09-19T08:38:21.495241+0000","last_undegraded":"2024-09-19T08:38:21.495241+0000","last_fullsized":"2024-09-19T08:38:21.495241+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:02.493804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.376783+0000","last_change":"2024-09-19T08:38:18.366656+0000","last_active":"2024-09-19T08:38:20.376783+0000","last_peered":"2024-09-19T08:38:20.376783+0000","last_clean":"2024-09-19T08:38:20.376783+0000","last_became_active":"2024-09-19T08:38:18.366299+0000","last_became_peered":"2024-09-19T08:38:18.366299+0000","last_unstale":"2024-09-19T08:38:20.376783+0000","last_undegraded":"2024-09-19T08:38:20.376783+0000","last_fullsized":"2024-09-19T08:38:20.376783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:35:08.394753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.385063+0000","last_change":"2024-09-19T08:38:19.414578+0000","last_active":"2024-09-19T08:38:20.385063+0000","last_peered":"2024-09-19T08:38:20.385063+0000","last_clean":"2024-09-19T08:38:20.385063+0000","last_became_active":"2024-09-19T08:38:19.413677+0000","last_became_peered":"2024-09-19T08:38:19.413677+0000","last_unstale":"2024-09-19T08:38:20.385063+0000","last_undegraded":"2024-09-19T08:38:20.385063+0000","last_fullsized":"2024-09-19T08:38:20.385063+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:30:08.777799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.083724+0000","last_change":"2024-09-19T08:38:18.394865+0000","last_active":"2024-09-19T08:38:22.083724+0000","last_peered":"2024-09-19T08:38:22.083724+0000","last_clean":"2024-09-19T08:38:22.083724+0000","last_became_active":"2024-09-19T08:38:18.394496+0000","last_became_peered":"2024-09-19T08:38:18.394496+0000","last_unstale":"2024-09-19T08:38:22.083724+0000","last_undegraded":"2024-09-19T08:38:22.083724+0000","last_fullsized":"2024-09-19T08:38:22.083724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:34:58.036813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.494357+0000","last_change":"2024-09-19T08:38:19.433593+0000","last_active":"2024-09-19T08:38:21.494357+0000","last_peered":"2024-09-19T08:38:21.494357+0000","last_clean":"2024-09-19T08:38:21.494357+0000","last_became_active":"2024-09-19T08:38:19.433322+0000","last_became_peered":"2024-09-19T08:38:19.433322+0000","last_unstale":"2024-09-19T08:38:21.494357+0000","last_undegraded":"2024-09-19T08:38:21.494357+0000","last_fullsized":"2024-09-19T08:38:21.494357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:44:57.127592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.493861+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:38:21.493861+0000","last_peered":"2024-09-19T08:38:21.493861+0000","last_clean":"2024-09-19T08:38:21.493861+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:38:21.493861+0000","last_undegraded":"2024-09-19T08:38:21.493861+0000","last_fullsized":"2024-09-19T08:38:21.493861+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.384222+0000","last_change":"2024-09-19T08:38:18.391179+0000","last_active":"2024-09-19T08:38:20.384222+0000","last_peered":"2024-09-19T08:38:20.384222+0000","last_clean":"2024-09-19T08:38:20.384222+0000","last_became_active":"2024-09-19T08:38:18.390393+0000","last_became_peered":"2024-09-19T08:38:18.390393+0000","last_unstale":"2024-09-19T08:38:20.384222+0000","last_undegraded":"2024-09-19T08:38:20.384222+0000","last_fullsized":"2024-09-19T08:38:20.384222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:47.789309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085655+0000","last_change":"2024-09-19T08:38:19.430328+0000","last_active":"2024-09-19T08:38:22.085655+0000","last_peered":"2024-09-19T08:38:22.085655+0000","last_clean":"2024-09-19T08:38:22.085655+0000","last_became_active":"2024-09-19T08:38:19.429984+0000","last_became_peered":"2024-09-19T08:38:19.429984+0000","last_unstale":"2024-09-19T08:38:22.085655+0000","last_undegraded":"2024-09-19T08:38:22.085655+0000","last_fullsized":"2024-09-19T08:38:22.085655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:19:32.736687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.493590+0000","last_change":"2024-09-19T08:38:18.384759+0000","last_active":"2024-09-19T08:38:21.493590+0000","last_peered":"2024-09-19T08:38:21.493590+0000","last_clean":"2024-09-19T08:38:21.493590+0000","last_became_active":"2024-09-19T08:38:18.383974+0000","last_became_peered":"2024-09-19T08:38:18.383974+0000","last_unstale":"2024-09-19T08:38:21.493590+0000","last_undegraded":"2024-09-19T08:38:21.493590+0000","last_fullsized":"2024-09-19T08:38:21.493590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:39.064483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.084012+0000","last_change":"2024-09-19T08:38:19.412025+0000","last_active":"2024-09-19T08:38:22.084012+0000","last_peered":"2024-09-19T08:38:22.084012+0000","last_clean":"2024-09-19T08:38:22.084012+0000","last_became_active":"2024-09-19T08:38:19.411701+0000","last_became_peered":"2024-09-19T08:38:19.411701+0000","last_unstale":"2024-09-19T08:38:22.084012+0000","last_undegraded":"2024-09-19T08:38:22.084012+0000","last_fullsized":"2024-09-19T08:38:22.084012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:13:02.635546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.083685+0000","last_change":"2024-09-19T08:38:18.366746+0000","last_active":"2024-09-19T08:38:22.083685+0000","last_peered":"2024-09-19T08:38:22.083685+0000","last_clean":"2024-09-19T08:38:22.083685+0000","last_became_active":"2024-09-19T08:38:18.366346+0000","last_became_peered":"2024-09-19T08:38:18.366346+0000","last_unstale":"2024-09-19T08:38:22.083685+0000","last_undegraded":"2024-09-19T08:38:22.083685+0000","last_fullsized":"2024-09-19T08:38:22.083685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:16:55.536656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.376256+0000","last_change":"2024-09-19T08:38:19.395656+0000","last_active":"2024-09-19T08:38:20.376256+0000","last_peered":"2024-09-19T08:38:20.376256+0000","last_clean":"2024-09-19T08:38:20.376256+0000","last_became_active":"2024-09-19T08:38:19.393805+0000","last_became_peered":"2024-09-19T08:38:19.393805+0000","last_unstale":"2024-09-19T08:38:20.376256+0000","last_undegraded":"2024-09-19T08:38:20.376256+0000","last_fullsized":"2024-09-19T08:38:20.376256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:08.505022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.385101+0000","last_change":"2024-09-19T08:38:18.398888+0000","last_active":"2024-09-19T08:38:20.385101+0000","last_peered":"2024-09-19T08:38:20.385101+0000","last_clean":"2024-09-19T08:38:20.385101+0000","last_became_active":"2024-09-19T08:38:18.391135+0000","last_became_peered":"2024-09-19T08:38:18.391135+0000","last_unstale":"2024-09-19T08:38:20.385101+0000","last_undegraded":"2024-09-19T08:38:20.385101+0000","last_fullsized":"2024-09-19T08:38:20.385101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:48.426450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085253+0000","last_change":"2024-09-19T08:38:19.418136+0000","last_active":"2024-09-19T08:38:22.085253+0000","last_peered":"2024-09-19T08:38:22.085253+0000","last_clean":"2024-09-19T08:38:22.085253+0000","last_became_active":"2024-09-19T08:38:19.411375+0000","last_became_peered":"2024-09-19T08:38:19.411375+0000","last_unstale":"2024-09-19T08:38:22.085253+0000","last_undegraded":"2024-09-19T08:38:22.085253+0000","last_fullsized":"2024-09-19T08:38:22.085253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:40.067968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.376493+0000","last_change":"2024-09-19T08:38:18.363154+0000","last_active":"2024-09-19T08:38:20.376493+0000","last_peered":"2024-09-19T08:38:20.376493+0000","last_clean":"2024-09-19T08:38:20.376493+0000","last_became_active":"2024-09-19T08:38:18.362876+0000","last_became_peered":"2024-09-19T08:38:18.362876+0000","last_unstale":"2024-09-19T08:38:20.376493+0000","last_undegraded":"2024-09-19T08:38:20.376493+0000","last_fullsized":"2024-09-19T08:38:20.376493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:48.426450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.385738+0000","last_change":"2024-09-19T08:38:19.413745+0000","last_active":"2024-09-19T08:38:20.385738+0000","last_peered":"2024-09-19T08:38:20.385738+0000","last_clean":"2024-09-19T08:38:20.385738+0000","last_became_active":"2024-09-19T08:38:19.413092+0000","last_became_peered":"2024-09-19T08:38:19.413092+0000","last_unstale":"2024-09-19T08:38:20.385738+0000","last_undegraded":"2024-09-19T08:38:20.385738+0000","last_fullsized":"2024-09-19T08:38:20.385738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:21.788118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.494741+0000","last_change":"2024-09-19T08:38:18.389212+0000","last_active":"2024-09-19T08:38:21.494741+0000","last_peered":"2024-09-19T08:38:21.494741+0000","last_clean":"2024-09-19T08:38:21.494741+0000","last_became_active":"2024-09-19T08:38:18.388588+0000","last_became_peered":"2024-09-19T08:38:18.388588+0000","last_unstale":"2024-09-19T08:38:21.494741+0000","last_undegraded":"2024-09-19T08:38:21.494741+0000","last_fullsized":"2024-09-19T08:38:21.494741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:46:34.417531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.376388+0000","last_change":"2024-09-19T08:38:19.386583+0000","last_active":"2024-09-19T08:38:20.376388+0000","last_peered":"2024-09-19T08:38:20.376388+0000","last_clean":"2024-09-19T08:38:20.376388+0000","last_became_active":"2024-09-19T08:38:19.385749+0000","last_became_peered":"2024-09-19T08:38:19.385749+0000","last_unstale":"2024-09-19T08:38:20.376388+0000","last_undegraded":"2024-09-19T08:38:20.376388+0000","last_fullsized":"2024-09-19T08:38:20.376388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:47.390332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.376949+0000","last_change":"2024-09-19T08:38:18.377131+0000","last_active":"2024-09-19T08:38:20.376949+0000","last_peered":"2024-09-19T08:38:20.376949+0000","last_clean":"2024-09-19T08:38:20.376949+0000","last_became_active":"2024-09-19T08:38:18.376860+0000","last_became_peered":"2024-09-19T08:38:18.376860+0000","last_unstale":"2024-09-19T08:38:20.376949+0000","last_undegraded":"2024-09-19T08:38:20.376949+0000","last_fullsized":"2024-09-19T08:38:20.376949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:46:34.417531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085974+0000","last_change":"2024-09-19T08:38:19.410025+0000","last_active":"2024-09-19T08:38:22.085974+0000","last_peered":"2024-09-19T08:38:22.085974+0000","last_clean":"2024-09-19T08:38:22.085974+0000","last_became_active":"2024-09-19T08:38:19.409390+0000","last_became_peered":"2024-09-19T08:38:19.409390+0000","last_unstale":"2024-09-19T08:38:22.085974+0000","last_undegraded":"2024-09-19T08:38:22.085974+0000","last_fullsized":"2024-09-19T08:38:22.085974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:10.400137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.425026+0000","last_change":"2024-09-19T08:38:18.392567+0000","last_active":"2024-09-19T08:38:20.425026+0000","last_peered":"2024-09-19T08:38:20.425026+0000","last_clean":"2024-09-19T08:38:20.425026+0000","last_became_active":"2024-09-19T08:38:18.387256+0000","last_became_peered":"2024-09-19T08:38:18.387256+0000","last_unstale":"2024-09-19T08:38:20.425026+0000","last_undegraded":"2024-09-19T08:38:20.425026+0000","last_fullsized":"2024-09-19T08:38:20.425026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:12:57.094603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.087052+0000","last_change":"2024-09-19T08:38:19.406115+0000","last_active":"2024-09-19T08:38:22.087052+0000","last_peered":"2024-09-19T08:38:22.087052+0000","last_clean":"2024-09-19T08:38:22.087052+0000","last_became_active":"2024-09-19T08:38:19.401366+0000","last_became_peered":"2024-09-19T08:38:19.401366+0000","last_unstale":"2024-09-19T08:38:22.087052+0000","last_undegraded":"2024-09-19T08:38:22.087052+0000","last_fullsized":"2024-09-19T08:38:22.087052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:08.505022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.493561+0000","last_change":"2024-09-19T08:38:18.384552+0000","last_active":"2024-09-19T08:38:21.493561+0000","last_peered":"2024-09-19T08:38:21.493561+0000","last_clean":"2024-09-19T08:38:21.493561+0000","last_became_active":"2024-09-19T08:38:18.383834+0000","last_became_peered":"2024-09-19T08:38:18.383834+0000","last_unstale":"2024-09-19T08:38:21.493561+0000","last_undegraded":"2024-09-19T08:38:21.493561+0000","last_fullsized":"2024-09-19T08:38:21.493561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:20.637060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085218+0000","last_change":"2024-09-19T08:38:19.395111+0000","last_active":"2024-09-19T08:38:22.085218+0000","last_peered":"2024-09-19T08:38:22.085218+0000","last_clean":"2024-09-19T08:38:22.085218+0000","last_became_active":"2024-09-19T08:38:19.394398+0000","last_became_peered":"2024-09-19T08:38:19.394398+0000","last_unstale":"2024-09-19T08:38:22.085218+0000","last_undegraded":"2024-09-19T08:38:22.085218+0000","last_fullsized":"2024-09-19T08:38:22.085218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:49.429936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.376132+0000","last_change":"2024-09-19T08:38:18.378231+0000","last_active":"2024-09-19T08:38:20.376132+0000","last_peered":"2024-09-19T08:38:20.376132+0000","last_clean":"2024-09-19T08:38:20.376132+0000","last_became_active":"2024-09-19T08:38:18.377883+0000","last_became_peered":"2024-09-19T08:38:18.377883+0000","last_unstale":"2024-09-19T08:38:20.376132+0000","last_undegraded":"2024-09-19T08:38:20.376132+0000","last_fullsized":"2024-09-19T08:38:20.376132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:36.282825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085006+0000","last_change":"2024-09-19T08:38:19.392650+0000","last_active":"2024-09-19T08:38:22.085006+0000","last_peered":"2024-09-19T08:38:22.085006+0000","last_clean":"2024-09-19T08:38:22.085006+0000","last_became_active":"2024-09-19T08:38:19.391936+0000","last_became_peered":"2024-09-19T08:38:19.391936+0000","last_unstale":"2024-09-19T08:38:22.085006+0000","last_undegraded":"2024-09-19T08:38:22.085006+0000","last_fullsized":"2024-09-19T08:38:22.085006+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:42:24.499497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.376387+0000","last_change":"2024-09-19T08:38:18.359251+0000","last_active":"2024-09-19T08:38:20.376387+0000","last_peered":"2024-09-19T08:38:20.376387+0000","last_clean":"2024-09-19T08:38:20.376387+0000","last_became_active":"2024-09-19T08:38:18.358873+0000","last_became_peered":"2024-09-19T08:38:18.358873+0000","last_unstale":"2024-09-19T08:38:20.376387+0000","last_undegraded":"2024-09-19T08:38:20.376387+0000","last_fullsized":"2024-09-19T08:38:20.376387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:20.637060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085621+0000","last_change":"2024-09-19T08:38:19.414125+0000","last_active":"2024-09-19T08:38:22.085621+0000","last_peered":"2024-09-19T08:38:22.085621+0000","last_clean":"2024-09-19T08:38:22.085621+0000","last_became_active":"2024-09-19T08:38:19.413714+0000","last_became_peered":"2024-09-19T08:38:19.413714+0000","last_unstale":"2024-09-19T08:38:22.085621+0000","last_undegraded":"2024-09-19T08:38:22.085621+0000","last_fullsized":"2024-09-19T08:38:22.085621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:39:22.441525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.494666+0000","last_change":"2024-09-19T08:38:18.387311+0000","last_active":"2024-09-19T08:38:21.494666+0000","last_peered":"2024-09-19T08:38:21.494666+0000","last_clean":"2024-09-19T08:38:21.494666+0000","last_became_active":"2024-09-19T08:38:18.386340+0000","last_became_peered":"2024-09-19T08:38:18.386340+0000","last_unstale":"2024-09-19T08:38:21.494666+0000","last_undegraded":"2024-09-19T08:38:21.494666+0000","last_fullsized":"2024-09-19T08:38:21.494666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:36.282825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.424982+0000","last_change":"2024-09-19T08:38:19.407976+0000","last_active":"2024-09-19T08:38:20.424982+0000","last_peered":"2024-09-19T08:38:20.424982+0000","last_clean":"2024-09-19T08:38:20.424982+0000","last_became_active":"2024-09-19T08:38:19.407198+0000","last_became_peered":"2024-09-19T08:38:19.407198+0000","last_unstale":"2024-09-19T08:38:20.424982+0000","last_undegraded":"2024-09-19T08:38:20.424982+0000","last_fullsized":"2024-09-19T08:38:20.424982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:42:24.499497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.385369+0000","last_change":"2024-09-19T08:38:19.413947+0000","last_active":"2024-09-19T08:38:20.385369+0000","last_peered":"2024-09-19T08:38:20.385369+0000","last_clean":"2024-09-19T08:38:20.385369+0000","last_became_active":"2024-09-19T08:38:19.413225+0000","last_became_peered":"2024-09-19T08:38:19.413225+0000","last_unstale":"2024-09-19T08:38:20.385369+0000","last_undegraded":"2024-09-19T08:38:20.385369+0000","last_fullsized":"2024-09-19T08:38:20.385369+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:02:31.580648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.494149+0000","last_change":"2024-09-19T08:38:18.400304+0000","last_active":"2024-09-19T08:38:21.494149+0000","last_peered":"2024-09-19T08:38:21.494149+0000","last_clean":"2024-09-19T08:38:21.494149+0000","last_became_active":"2024-09-19T08:38:18.399609+0000","last_became_peered":"2024-09-19T08:38:18.399609+0000","last_unstale":"2024-09-19T08:38:21.494149+0000","last_undegraded":"2024-09-19T08:38:21.494149+0000","last_fullsized":"2024-09-19T08:38:21.494149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:48.426450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:20.385061+0000","last_change":"2024-09-19T08:38:19.429826+0000","last_active":"2024-09-19T08:38:20.385061+0000","last_peered":"2024-09-19T08:38:20.385061+0000","last_clean":"2024-09-19T08:38:20.385061+0000","last_became_active":"2024-09-19T08:38:19.429543+0000","last_became_peered":"2024-09-19T08:38:19.429543+0000","last_unstale":"2024-09-19T08:38:20.385061+0000","last_undegraded":"2024-09-19T08:38:20.385061+0000","last_fullsized":"2024-09-19T08:38:20.385061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:44:57.127592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:21.494455+0000","last_change":"2024-09-19T08:38:18.384861+0000","last_active":"2024-09-19T08:38:21.494455+0000","last_peered":"2024-09-19T08:38:21.494455+0000","last_clean":"2024-09-19T08:38:21.494455+0000","last_became_active":"2024-09-19T08:38:18.384070+0000","last_became_peered":"2024-09-19T08:38:18.384070+0000","last_unstale":"2024-09-19T08:38:21.494455+0000","last_undegraded":"2024-09-19T08:38:21.494455+0000","last_fullsized":"2024-09-19T08:38:21.494455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:35:08.394753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.087092+0000","last_change":"2024-09-19T08:38:19.430701+0000","last_active":"2024-09-19T08:38:22.087092+0000","last_peered":"2024-09-19T08:38:22.087092+0000","last_clean":"2024-09-19T08:38:22.087092+0000","last_became_active":"2024-09-19T08:38:19.430335+0000","last_became_peered":"2024-09-19T08:38:19.430335+0000","last_unstale":"2024-09-19T08:38:22.087092+0000","last_undegraded":"2024-09-19T08:38:22.087092+0000","last_fullsized":"2024-09-19T08:38:22.087092+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:07.305805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085411+0000","last_change":"2024-09-19T08:38:18.358117+0000","last_active":"2024-09-19T08:38:22.085411+0000","last_peered":"2024-09-19T08:38:22.085411+0000","last_clean":"2024-09-19T08:38:22.085411+0000","last_became_active":"2024-09-19T08:38:18.357791+0000","last_became_peered":"2024-09-19T08:38:18.357791+0000","last_unstale":"2024-09-19T08:38:22.085411+0000","last_undegraded":"2024-09-19T08:38:22.085411+0000","last_fullsized":"2024-09-19T08:38:22.085411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:00.838359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.085487+0000","last_change":"2024-09-19T08:38:19.396783+0000","last_active":"2024-09-19T08:38:22.085487+0000","last_peered":"2024-09-19T08:38:22.085487+0000","last_clean":"2024-09-19T08:38:22.085487+0000","last_became_active":"2024-09-19T08:38:19.396012+0000","last_became_peered":"2024-09-19T08:38:19.396012+0000","last_unstale":"2024-09-19T08:38:22.085487+0000","last_undegraded":"2024-09-19T08:38:22.085487+0000","last_fullsized":"2024-09-19T08:38:22.085487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:18.334613+0000","last_clean_scrub_stamp":"2024-09-19T08:38:18.334613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:20:35.891069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-19T08:38:22.087284+0000","last_change":"2024-09-19T08:38:18.380235+0000","last_active":"2024-09-19T08:38:22.087284+0000","last_peered":"2024-09-19T08:38:22.087284+0000","last_clean":"2024-09-19T08:38:22.087284+0000","last_became_active":"2024-09-19T08:38:18.380044+0000","last_became_peered":"2024-09-19T08:38:18.380044+0000","last_unstale":"2024-09-19T08:38:22.087284+0000","last_undegraded":"2024-09-19T08:38:22.087284+0000","last_fullsized":"2024-09-19T08:38:22.087284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:17.331127+0000","last_clean_scrub_stamp":"2024-09-19T08:38:17.331127+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:36:30.341445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":806912,"data_stored":663417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509449,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":212992,"data_stored":72557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509449,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":72593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":72467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":217088,"data_stored":72627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":815104,"data_stored":663473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":74079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":74079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:38:32.369 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T08:38:32.369 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-19T08:38:32.369 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-19T08:38:32.369 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T08:38:32.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.515+0000 7f131f0e6640 1 Processor -- start 2024-09-19T08:38:32.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.515+0000 7f131f0e6640 1 -- start start 2024-09-19T08:38:32.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.519+0000 7f131f0e6640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f131815e510 0x7f13180b6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:32.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.519+0000 7f131f0e6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13180ceed0 0x7f131806cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:32.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.519+0000 7f131f0e6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f13180cf2f0 0x7f131806d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:32.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.519+0000 7f131f0e6640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f13180b3b90 con 0x7f131815e510 2024-09-19T08:38:32.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.519+0000 7f131f0e6640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f13180ca930 con 0x7f13180cf2f0 2024-09-19T08:38:32.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.519+0000 7f131f0e6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f13180ca650 con 0x7f13180ceed0 2024-09-19T08:38:32.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.519+0000 7f131ce82640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f131815e510 0x7f13180b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:32.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.519+0000 7f131ce82640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f131815e510 0x7f13180b6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38128/0 (socket says 172.21.15.98:38128) 2024-09-19T08:38:32.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.519+0000 7f131ce82640 1 -- 172.21.15.98:0/222569260 learned_addr learned my addr 172.21.15.98:0/222569260 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:32.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.519+0000 7f1317fff640 1 --2- 172.21.15.98:0/222569260 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13180ceed0 0x7f131806cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:32.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.519+0000 7f131d683640 1 --2- 172.21.15.98:0/222569260 >> v2:172.21.15.139:3300/0 conn(0x7f13180cf2f0 0x7f131806d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:32.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.519+0000 7f131ce82640 1 -- 172.21.15.98:0/222569260 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13180ceed0 msgr2=0x7f131806cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:32.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.519+0000 7f131ce82640 1 --2- 172.21.15.98:0/222569260 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13180ceed0 0x7f131806cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:32.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.519+0000 7f131ce82640 1 -- 172.21.15.98:0/222569260 >> v2:172.21.15.139:3300/0 conn(0x7f13180cf2f0 msgr2=0x7f131806d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:32.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.519+0000 7f131ce82640 1 --2- 172.21.15.98:0/222569260 >> v2:172.21.15.139:3300/0 conn(0x7f13180cf2f0 0x7f131806d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:32.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.519+0000 7f131ce82640 1 -- 172.21.15.98:0/222569260 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13180cb0e0 con 0x7f131815e510 2024-09-19T08:38:32.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.519+0000 7f131d683640 1 --2- 172.21.15.98:0/222569260 >> v2:172.21.15.139:3300/0 conn(0x7f13180cf2f0 0x7f131806d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:32.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.519+0000 7f1317fff640 1 --2- 172.21.15.98:0/222569260 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13180ceed0 0x7f131806cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:32.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.519+0000 7f131ce82640 1 --2- 172.21.15.98:0/222569260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f131815e510 0x7f13180b6130 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f13180cf7a0 tx=0x7f130801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52273047cd0a27ae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:32.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.519+0000 7f13177fe640 1 -- 172.21.15.98:0/222569260 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13080ab650 con 0x7f131815e510 2024-09-19T08:38:32.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.519+0000 7f13177fe640 1 -- 172.21.15.98:0/222569260 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13080b0070 con 0x7f131815e510 2024-09-19T08:38:32.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.519+0000 7f13177fe640 1 -- 172.21.15.98:0/222569260 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13080aba10 con 0x7f131815e510 2024-09-19T08:38:32.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.519+0000 7f131f0e6640 1 -- 172.21.15.98:0/222569260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f131815e510 msgr2=0x7f13180b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:32.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.519+0000 7f131f0e6640 1 --2- 172.21.15.98:0/222569260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f131815e510 0x7f13180b6130 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f13180cf7a0 tx=0x7f130801c4f0 comp rx=0 tx=0).stop 2024-09-19T08:38:32.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.519+0000 7f131f0e6640 1 -- 172.21.15.98:0/222569260 shutdown_connections 2024-09-19T08:38:32.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.519+0000 7f131f0e6640 1 --2- 172.21.15.98:0/222569260 >> v2:172.21.15.139:3300/0 conn(0x7f13180cf2f0 0x7f131806d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:32.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.519+0000 7f131f0e6640 1 --2- 172.21.15.98:0/222569260 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13180ceed0 0x7f131806cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:32.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.519+0000 7f131f0e6640 1 --2- 172.21.15.98:0/222569260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f131815e510 0x7f13180b6130 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:32.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.519+0000 7f131f0e6640 1 -- 172.21.15.98:0/222569260 >> 172.21.15.98:0/222569260 conn(0x7f13180b2fd0 msgr2=0x7f13180b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:32.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.519+0000 7f131f0e6640 1 -- 172.21.15.98:0/222569260 shutdown_connections 2024-09-19T08:38:32.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.519+0000 7f131f0e6640 1 -- 172.21.15.98:0/222569260 wait complete. 2024-09-19T08:38:32.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.523+0000 7f131f0e6640 1 Processor -- start 2024-09-19T08:38:32.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.523+0000 7f131f0e6640 1 -- start start 2024-09-19T08:38:32.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.523+0000 7f131f0e6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13180ceed0 0x7f131814ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:32.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.523+0000 7f131f0e6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13180cf2f0 0x7f1318149470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:32.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.523+0000 7f131f0e6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f131815e510 0x7f131815f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:32.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.523+0000 7f131f0e6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1318073c20 con 0x7f13180ceed0 2024-09-19T08:38:32.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.523+0000 7f131f0e6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1318073aa0 con 0x7f131815e510 2024-09-19T08:38:32.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.523+0000 7f131f0e6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1318073da0 con 0x7f13180cf2f0 2024-09-19T08:38:32.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.523+0000 7f131ce82640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13180ceed0 0x7f131814ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:32.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.523+0000 7f131ce82640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13180ceed0 0x7f131814ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38140/0 (socket says 172.21.15.98:38140) 2024-09-19T08:38:32.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.523+0000 7f131ce82640 1 -- 172.21.15.98:0/1840300928 learned_addr learned my addr 172.21.15.98:0/1840300928 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:32.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.523+0000 7f1317fff640 1 --2- 172.21.15.98:0/1840300928 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13180cf2f0 0x7f1318149470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:32.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.523+0000 7f131d683640 1 --2- 172.21.15.98:0/1840300928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f131815e510 0x7f131815f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:32.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.523+0000 7f131ce82640 1 -- 172.21.15.98:0/1840300928 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13180cf2f0 msgr2=0x7f1318149470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:32.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.523+0000 7f131ce82640 1 --2- 172.21.15.98:0/1840300928 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13180cf2f0 0x7f1318149470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:32.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.523+0000 7f131ce82640 1 -- 172.21.15.98:0/1840300928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f131815e510 msgr2=0x7f131815f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:32.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.523+0000 7f131ce82640 1 --2- 172.21.15.98:0/1840300928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f131815e510 0x7f131815f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:32.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.523+0000 7f131ce82640 1 -- 172.21.15.98:0/1840300928 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13180cb9d0 con 0x7f13180ceed0 2024-09-19T08:38:32.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.523+0000 7f1317fff640 1 --2- 172.21.15.98:0/1840300928 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13180cf2f0 0x7f1318149470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:32.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.523+0000 7f131d683640 1 --2- 172.21.15.98:0/1840300928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f131815e510 0x7f131815f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:32.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.523+0000 7f131ce82640 1 --2- 172.21.15.98:0/1840300928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13180ceed0 0x7f131814ed70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f130801c320 tx=0x7f130801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:32.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.523+0000 7f13157fa640 1 -- 172.21.15.98:0/1840300928 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13080b4800 con 0x7f13180ceed0 2024-09-19T08:38:32.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.523+0000 7f13157fa640 1 -- 172.21.15.98:0/1840300928 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13080b49a0 con 0x7f13180ceed0 2024-09-19T08:38:32.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.523+0000 7f131f0e6640 1 -- 172.21.15.98:0/1840300928 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13180cbc70 con 0x7f13180ceed0 2024-09-19T08:38:32.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.523+0000 7f13157fa640 1 -- 172.21.15.98:0/1840300928 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13080b4d50 con 0x7f13180ceed0 2024-09-19T08:38:32.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.523+0000 7f131f0e6640 1 -- 172.21.15.98:0/1840300928 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13180cc230 con 0x7f13180ceed0 2024-09-19T08:38:32.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.523+0000 7f131f0e6640 1 -- 172.21.15.98:0/1840300928 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13180b6130 con 0x7f13180ceed0 2024-09-19T08:38:32.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.527+0000 7f13157fa640 1 -- 172.21.15.98:0/1840300928 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f13080c7390 con 0x7f13180ceed0 2024-09-19T08:38:32.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.527+0000 7f13157fa640 1 --2- 172.21.15.98:0/1840300928 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f13040678c0 0x7f1304069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:32.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.527+0000 7f13157fa640 1 -- 172.21.15.98:0/1840300928 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f13081205f0 con 0x7f13180ceed0 2024-09-19T08:38:32.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.527+0000 7f1317fff640 1 --2- 172.21.15.98:0/1840300928 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f13040678c0 0x7f1304069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:32.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.527+0000 7f1317fff640 1 --2- 172.21.15.98:0/1840300928 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f13040678c0 0x7f1304069d80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f130c006fd0 tx=0x7f130c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:32.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.531+0000 7f13157fa640 1 -- 172.21.15.98:0/1840300928 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f13080c4060 con 0x7f13180ceed0 2024-09-19T08:38:32.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.715+0000 7f131f0e6640 1 -- 172.21.15.98:0/1840300928 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f13180b0360 con 0x7f13180ceed0 2024-09-19T08:38:32.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.715+0000 7f13157fa640 1 -- 172.21.15.98:0/1840300928 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f1308016110 con 0x7f13180ceed0 2024-09-19T08:38:32.714 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:38:32.714 INFO:teuthology.orchestra.run.smithi098.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T08:38:32.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.715+0000 7f131f0e6640 1 -- 172.21.15.98:0/1840300928 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f13040678c0 msgr2=0x7f1304069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:32.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.715+0000 7f131f0e6640 1 --2- 172.21.15.98:0/1840300928 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f13040678c0 0x7f1304069d80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f130c006fd0 tx=0x7f130c008040 comp rx=0 tx=0).stop 2024-09-19T08:38:32.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.715+0000 7f131f0e6640 1 -- 172.21.15.98:0/1840300928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13180ceed0 msgr2=0x7f131814ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:32.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.715+0000 7f131f0e6640 1 --2- 172.21.15.98:0/1840300928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13180ceed0 0x7f131814ed70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f130801c320 tx=0x7f130801ad30 comp rx=0 tx=0).stop 2024-09-19T08:38:32.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.715+0000 7f131f0e6640 1 -- 172.21.15.98:0/1840300928 shutdown_connections 2024-09-19T08:38:32.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.715+0000 7f131f0e6640 1 --2- 172.21.15.98:0/1840300928 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f13040678c0 0x7f1304069d80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:32.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.715+0000 7f131f0e6640 1 --2- 172.21.15.98:0/1840300928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f131815e510 0x7f131815f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:32.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.715+0000 7f131f0e6640 1 --2- 172.21.15.98:0/1840300928 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13180cf2f0 0x7f1318149470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:32.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.715+0000 7f131f0e6640 1 --2- 172.21.15.98:0/1840300928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13180ceed0 0x7f131814ed70 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:32.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.715+0000 7f131f0e6640 1 -- 172.21.15.98:0/1840300928 >> 172.21.15.98:0/1840300928 conn(0x7f13180b2fd0 msgr2=0x7f131814fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:32.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.715+0000 7f131f0e6640 1 -- 172.21.15.98:0/1840300928 shutdown_connections 2024-09-19T08:38:32.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.715+0000 7f131f0e6640 1 -- 172.21.15.98:0/1840300928 wait complete. 2024-09-19T08:38:32.732 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-19T08:38:32.732 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:38:32.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.847+0000 7fe38139b640 1 Processor -- start 2024-09-19T08:38:32.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.847+0000 7fe38139b640 1 -- start start 2024-09-19T08:38:32.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.847+0000 7fe38139b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe37c0b7f80 0x7fe37c15f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:32.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.847+0000 7fe38139b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe37c0b9970 0x7fe37c0b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:32.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.847+0000 7fe38139b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe37c068440 con 0x7fe37c0d4820 2024-09-19T08:38:32.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.847+0000 7fe38139b640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fe37c163050 con 0x7fe37c0b7f80 2024-09-19T08:38:32.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.847+0000 7fe38139b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe37c162a90 con 0x7fe37c0b9970 2024-09-19T08:38:32.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.847+0000 7fe37affd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe37c0b7f80 0x7fe37c15f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:32.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.847+0000 7fe37affd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe37c0b7f80 0x7fe37c15f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38148/0 (socket says 172.21.15.98:38148) 2024-09-19T08:38:32.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.847+0000 7fe37affd640 1 -- 172.21.15.98:0/3365992733 learned_addr learned my addr 172.21.15.98:0/3365992733 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:32.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.847+0000 7fe37b7fe640 1 --2- 172.21.15.98:0/3365992733 >> v2:172.21.15.139:3300/0 conn(0x7fe37c0b9970 0x7fe37c0b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:32.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.847+0000 7fe37affd640 1 -- 172.21.15.98:0/3365992733 >> v2:172.21.15.139:3300/0 conn(0x7fe37c0b9970 msgr2=0x7fe37c0b9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:32.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.847+0000 7fe37affd640 1 --2- 172.21.15.98:0/3365992733 >> v2:172.21.15.139:3300/0 conn(0x7fe37c0b9970 0x7fe37c0b9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:32.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.847+0000 7fe37affd640 1 -- 172.21.15.98:0/3365992733 >> v1:172.21.15.139:6789/0 conn(0x7fe37c0d4820 legacy=0x7fe37c1621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:32.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.847+0000 7fe37affd640 1 -- 172.21.15.98:0/3365992733 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37c162d70 con 0x7fe37c0b7f80 2024-09-19T08:38:32.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.847+0000 7fe37b7fe640 1 --2- 172.21.15.98:0/3365992733 >> v2:172.21.15.139:3300/0 conn(0x7fe37c0b9970 0x7fe37c0b9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:38:32.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe37affd640 1 --2- 172.21.15.98:0/3365992733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe37c0b7f80 0x7fe37c15f560 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0b69c0 tx=0x7fe36c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dfee7ed3c5e03fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:32.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe37a7fc640 1 -- 172.21.15.98:0/3365992733 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c0b0070 con 0x7fe37c0b7f80 2024-09-19T08:38:32.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe37a7fc640 1 -- 172.21.15.98:0/3365992733 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe36c0ab600 con 0x7fe37c0b7f80 2024-09-19T08:38:32.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe37a7fc640 1 -- 172.21.15.98:0/3365992733 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c0ab9d0 con 0x7fe37c0b7f80 2024-09-19T08:38:32.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe38139b640 1 -- 172.21.15.98:0/3365992733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe37c0b7f80 msgr2=0x7fe37c15f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:32.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe38139b640 1 --2- 172.21.15.98:0/3365992733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe37c0b7f80 0x7fe37c15f560 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0b69c0 tx=0x7fe36c01c520 comp rx=0 tx=0).stop 2024-09-19T08:38:32.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe38139b640 1 -- 172.21.15.98:0/3365992733 shutdown_connections 2024-09-19T08:38:32.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe38139b640 1 --2- 172.21.15.98:0/3365992733 >> v2:172.21.15.139:3300/0 conn(0x7fe37c0b9970 0x7fe37c0b9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:32.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe38139b640 1 --2- 172.21.15.98:0/3365992733 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe37c0b7f80 0x7fe37c15f560 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:32.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe38139b640 1 -- 172.21.15.98:0/3365992733 >> 172.21.15.98:0/3365992733 conn(0x7fe37c074e40 msgr2=0x7fe37c075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:32.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe38139b640 1 -- 172.21.15.98:0/3365992733 shutdown_connections 2024-09-19T08:38:32.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe38139b640 1 -- 172.21.15.98:0/3365992733 wait complete. 2024-09-19T08:38:32.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe38139b640 1 Processor -- start 2024-09-19T08:38:32.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe38139b640 1 -- start start 2024-09-19T08:38:32.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe38139b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe37c0b7f80 0x7fe37c152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:32.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe38139b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe37c0b9970 0x7fe37c139720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:32.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe38139b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe37c0d4820 0x7fe37c1614c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:32.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe38139b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c0b84e0 con 0x7fe37c0b9970 2024-09-19T08:38:32.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe38139b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c0b8360 con 0x7fe37c0b7f80 2024-09-19T08:38:32.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe38139b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe37c144590 con 0x7fe37c0d4820 2024-09-19T08:38:32.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe3727fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe37c0b9970 0x7fe37c139720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:32.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe3727fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe37c0b9970 0x7fe37c139720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38164/0 (socket says 172.21.15.98:38164) 2024-09-19T08:38:32.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe3727fc640 1 -- 172.21.15.98:0/4275583075 learned_addr learned my addr 172.21.15.98:0/4275583075 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:32.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe37affd640 1 --2- 172.21.15.98:0/4275583075 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe37c0b7f80 0x7fe37c152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:32.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe37b7fe640 1 --2- 172.21.15.98:0/4275583075 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe37c0d4820 0x7fe37c1614c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:32.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe3727fc640 1 -- 172.21.15.98:0/4275583075 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe37c0d4820 msgr2=0x7fe37c1614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:32.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe3727fc640 1 --2- 172.21.15.98:0/4275583075 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe37c0d4820 0x7fe37c1614c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:32.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe3727fc640 1 -- 172.21.15.98:0/4275583075 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe37c0b7f80 msgr2=0x7fe37c152970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:32.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe3727fc640 1 --2- 172.21.15.98:0/4275583075 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe37c0b7f80 0x7fe37c152970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:32.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe3727fc640 1 -- 172.21.15.98:0/4275583075 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37c152eb0 con 0x7fe37c0b9970 2024-09-19T08:38:32.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe37b7fe640 1 --2- 172.21.15.98:0/4275583075 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe37c0d4820 0x7fe37c1614c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:32.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe3727fc640 1 --2- 172.21.15.98:0/4275583075 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe37c0b9970 0x7fe37c139720 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe36400c420 tx=0x7fe36400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:32.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe373fff640 1 -- 172.21.15.98:0/4275583075 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3640052d0 con 0x7fe37c0b9970 2024-09-19T08:38:32.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe373fff640 1 -- 172.21.15.98:0/4275583075 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe364005470 con 0x7fe37c0b9970 2024-09-19T08:38:32.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe38139b640 1 -- 172.21.15.98:0/4275583075 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe37c160420 con 0x7fe37c0b9970 2024-09-19T08:38:32.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe373fff640 1 -- 172.21.15.98:0/4275583075 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe364007260 con 0x7fe37c0b9970 2024-09-19T08:38:32.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.851+0000 7fe38139b640 1 -- 172.21.15.98:0/4275583075 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe37c1609e0 con 0x7fe37c0b9970 2024-09-19T08:38:32.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.855+0000 7fe38139b640 1 -- 172.21.15.98:0/4275583075 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe37c0ce8e0 con 0x7fe37c0b9970 2024-09-19T08:38:32.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.855+0000 7fe373fff640 1 -- 172.21.15.98:0/4275583075 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe364007400 con 0x7fe37c0b9970 2024-09-19T08:38:32.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.855+0000 7fe373fff640 1 --2- 172.21.15.98:0/4275583075 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe35c0678c0 0x7fe35c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:32.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.855+0000 7fe373fff640 1 -- 172.21.15.98:0/4275583075 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe364085940 con 0x7fe37c0b9970 2024-09-19T08:38:32.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.855+0000 7fe37affd640 1 --2- 172.21.15.98:0/4275583075 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe35c0678c0 0x7fe35c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:32.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.855+0000 7fe37affd640 1 --2- 172.21.15.98:0/4275583075 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe35c0678c0 0x7fe35c069d80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe36c0ad000 tx=0x7fe36c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:32.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:32.859+0000 7fe373fff640 1 -- 172.21.15.98:0/4275583075 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe364056560 con 0x7fe37c0b9970 2024-09-19T08:38:33.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.039+0000 7fe38139b640 1 -- 172.21.15.98:0/4275583075 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe37c0d4df0 con 0x7fe37c0b9970 2024-09-19T08:38:33.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.043+0000 7fe373fff640 1 -- 172.21.15.98:0/4275583075 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5220 (secure 0 0 0) 0x7fe364055e60 con 0x7fe37c0b9970 2024-09-19T08:38:33.045 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:38:33.045 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":7,"btime":"2024-09-19T08:38:22:407519+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/2038744461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":2038744461},{"type":"v1","addr":"172.21.15.139:6833","nonce":2038744461}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/765134249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":765134249},{"type":"v1","addr":"172.21.15.98:6835","nonce":765134249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2832328076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2832328076},{"type":"v1","addr":"172.21.15.98:6837","nonce":2832328076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:38:18.717221+0000","modified":"2024-09-19T08:38:21.769948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6835/1879089763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1879089763},{"type":"v1","addr":"172.21.15.139:6835","nonce":1879089763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4261,"qdb_cluster":[4261]},"id":1}]} 2024-09-19T08:38:33.046 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 7 2024-09-19T08:38:33.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.043+0000 7fe38139b640 1 -- 172.21.15.98:0/4275583075 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe35c0678c0 msgr2=0x7fe35c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:33.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.043+0000 7fe38139b640 1 --2- 172.21.15.98:0/4275583075 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe35c0678c0 0x7fe35c069d80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe36c0ad000 tx=0x7fe36c008040 comp rx=0 tx=0).stop 2024-09-19T08:38:33.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.043+0000 7fe38139b640 1 -- 172.21.15.98:0/4275583075 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe37c0b9970 msgr2=0x7fe37c139720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:33.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.043+0000 7fe38139b640 1 --2- 172.21.15.98:0/4275583075 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe37c0b9970 0x7fe37c139720 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe36400c420 tx=0x7fe36400c8e0 comp rx=0 tx=0).stop 2024-09-19T08:38:33.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.043+0000 7fe38139b640 1 -- 172.21.15.98:0/4275583075 shutdown_connections 2024-09-19T08:38:33.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.043+0000 7fe38139b640 1 --2- 172.21.15.98:0/4275583075 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe35c0678c0 0x7fe35c069d80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:33.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.043+0000 7fe38139b640 1 --2- 172.21.15.98:0/4275583075 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe37c0d4820 0x7fe37c1614c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:33.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.043+0000 7fe38139b640 1 --2- 172.21.15.98:0/4275583075 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe37c0b9970 0x7fe37c139720 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:33.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.043+0000 7fe38139b640 1 --2- 172.21.15.98:0/4275583075 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe37c0b7f80 0x7fe37c152970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:33.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.043+0000 7fe38139b640 1 -- 172.21.15.98:0/4275583075 >> 172.21.15.98:0/4275583075 conn(0x7fe37c074e40 msgr2=0x7fe37c0d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:33.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.043+0000 7fe38139b640 1 -- 172.21.15.98:0/4275583075 shutdown_connections 2024-09-19T08:38:33.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.043+0000 7fe38139b640 1 -- 172.21.15.98:0/4275583075 wait complete. 2024-09-19T08:38:33.060 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:38:33.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.179+0000 7f09dbfb4640 1 Processor -- start 2024-09-19T08:38:33.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.179+0000 7f09dbfb4640 1 -- start start 2024-09-19T08:38:33.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.183+0000 7f09dbfb4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f09d406bc90 0x7f09d406c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:33.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.183+0000 7f09dbfb4640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09d40ccc60 con 0x7f09d40ce840 2024-09-19T08:38:33.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.183+0000 7f09dbfb4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09d40ccf40 con 0x7f09d4158fd0 2024-09-19T08:38:33.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.183+0000 7f09dbfb4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f09d41593b0 con 0x7f09d406bc90 2024-09-19T08:38:33.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.183+0000 7f09da551640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f09d40ce840 0x7f09d40cec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:60678/0 (socket says 172.21.15.98:60678) 2024-09-19T08:38:33.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.183+0000 7f09da551640 1 -- 172.21.15.98:0/2794363148 learned_addr learned my addr 172.21.15.98:0/2794363148 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:33.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.183+0000 7f09d954f640 1 --2- 172.21.15.98:0/2794363148 >> v2:172.21.15.139:3300/0 conn(0x7f09d406bc90 0x7f09d406c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:33.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.183+0000 7f09d8d4e640 1 -- 172.21.15.98:0/2794363148 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3233383907 0 0) 0x7f09d40ccc60 con 0x7f09d40ce840 2024-09-19T08:38:33.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.183+0000 7f09d8d4e640 1 -- 172.21.15.98:0/2794363148 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f09c0003530 con 0x7f09d40ce840 2024-09-19T08:38:33.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.183+0000 7f09d8d4e640 1 -- 172.21.15.98:0/2794363148 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3106001672 0 0) 0x7f09c0003530 con 0x7f09d40ce840 2024-09-19T08:38:33.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.183+0000 7f09d8d4e640 1 -- 172.21.15.98:0/2794363148 >> v2:172.21.15.139:3300/0 conn(0x7f09d406bc90 msgr2=0x7f09d406c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:33.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.183+0000 7f09d8d4e640 1 --2- 172.21.15.98:0/2794363148 >> v2:172.21.15.139:3300/0 conn(0x7f09d406bc90 0x7f09d406c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:33.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.183+0000 7f09d8d4e640 1 -- 172.21.15.98:0/2794363148 >> v1:172.21.15.139:6789/0 conn(0x7f09d4158fd0 legacy=0x7f09d40b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:33.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.183+0000 7f09d8d4e640 1 -- 172.21.15.98:0/2794363148 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d40cd220 con 0x7f09d40ce840 2024-09-19T08:38:33.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.183+0000 7f09d8d4e640 1 -- 172.21.15.98:0/2794363148 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f09cc001a00 con 0x7f09d40ce840 2024-09-19T08:38:33.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.183+0000 7f09d8d4e640 1 -- 172.21.15.98:0/2794363148 >> v1:172.21.15.98:6789/0 conn(0x7f09d40ce840 legacy=0x7f09d40cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:33.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.183+0000 7f09d8d4e640 1 --2- 172.21.15.98:0/2794363148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09c00039c0 0x7f09c0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:33.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.183+0000 7f09d8d4e640 1 --2- 172.21.15.98:0/2794363148 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09c0004810 0x7f09c0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:33.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.183+0000 7f09d8d4e640 1 --2- 172.21.15.98:0/2794363148 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f09c00051d0 0x7f09c000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:33.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.183+0000 7f09d8d4e640 1 -- 172.21.15.98:0/2794363148 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f09d41593b0 con 0x7f09c00039c0 2024-09-19T08:38:33.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.183+0000 7f09d8d4e640 1 -- 172.21.15.98:0/2794363148 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f09c000c170 con 0x7f09c0004810 2024-09-19T08:38:33.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.183+0000 7f09d8d4e640 1 -- 172.21.15.98:0/2794363148 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f09c000c370 con 0x7f09c00051d0 2024-09-19T08:38:33.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.183+0000 7f09d9d50640 1 --2- 172.21.15.98:0/2794363148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09c00039c0 0x7f09c0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:33.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.183+0000 7f09da551640 1 --2- 172.21.15.98:0/2794363148 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f09c00051d0 0x7f09c000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:33.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.183+0000 7f09d954f640 1 --2- 172.21.15.98:0/2794363148 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09c0004810 0x7f09c0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:33.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.183+0000 7f09d9d50640 1 -- 172.21.15.98:0/2794363148 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f09c00051d0 msgr2=0x7f09c000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:33.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.183+0000 7f09d9d50640 1 --2- 172.21.15.98:0/2794363148 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f09c00051d0 0x7f09c000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:33.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.183+0000 7f09d9d50640 1 -- 172.21.15.98:0/2794363148 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09c0004810 msgr2=0x7f09c0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:33.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.183+0000 7f09d9d50640 1 --2- 172.21.15.98:0/2794363148 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09c0004810 0x7f09c0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:33.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.183+0000 7f09d9d50640 1 -- 172.21.15.98:0/2794363148 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c0003530 con 0x7f09c00039c0 2024-09-19T08:38:33.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.183+0000 7f09d954f640 1 --2- 172.21.15.98:0/2794363148 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09c0004810 0x7f09c0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:33.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.183+0000 7f09da551640 1 --2- 172.21.15.98:0/2794363148 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f09c00051d0 0x7f09c000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:33.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.183+0000 7f09da551640 1 -- 172.21.15.98:0/2794363148 reap_dead start 2024-09-19T08:38:33.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.187+0000 7f09d9d50640 1 --2- 172.21.15.98:0/2794363148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09c00039c0 0x7f09c0003de0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f09d0001780 tx=0x7f09d001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=674597c9f6cbf091 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:33.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.187+0000 7f09d8d4e640 1 -- 172.21.15.98:0/2794363148 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d00013a0 con 0x7f09c00039c0 2024-09-19T08:38:33.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.187+0000 7f09d8d4e640 1 -- 172.21.15.98:0/2794363148 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09d00052f0 con 0x7f09c00039c0 2024-09-19T08:38:33.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.187+0000 7f09d8d4e640 1 -- 172.21.15.98:0/2794363148 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d0003840 con 0x7f09c00039c0 2024-09-19T08:38:33.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.187+0000 7f09dbfb4640 1 -- 172.21.15.98:0/2794363148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09c00039c0 msgr2=0x7f09c0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:33.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.187+0000 7f09dbfb4640 1 --2- 172.21.15.98:0/2794363148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09c00039c0 0x7f09c0003de0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f09d0001780 tx=0x7f09d001c3d0 comp rx=0 tx=0).stop 2024-09-19T08:38:33.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.187+0000 7f09dbfb4640 1 -- 172.21.15.98:0/2794363148 shutdown_connections 2024-09-19T08:38:33.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.187+0000 7f09dbfb4640 1 --2- 172.21.15.98:0/2794363148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09c00039c0 0x7f09c0003de0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:33.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.187+0000 7f09dbfb4640 1 -- 172.21.15.98:0/2794363148 >> 172.21.15.98:0/2794363148 conn(0x7f09d40b2010 msgr2=0x7f09d40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:33.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.187+0000 7f09dbfb4640 1 -- 172.21.15.98:0/2794363148 shutdown_connections 2024-09-19T08:38:33.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.187+0000 7f09dbfb4640 1 -- 172.21.15.98:0/2794363148 wait complete. 2024-09-19T08:38:33.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.187+0000 7f09dbfb4640 1 Processor -- start 2024-09-19T08:38:33.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.187+0000 7f09dbfb4640 1 -- start start 2024-09-19T08:38:33.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.187+0000 7f09dbfb4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09c00039c0 0x7f09d4153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:33.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.187+0000 7f09dbfb4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f09d414fe20 0x7f09d41526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:33.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.187+0000 7f09dbfb4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09d4155390 0x7f09d415de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:33.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.187+0000 7f09dbfb4640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f09d415c920 con 0x7f09c00039c0 2024-09-19T08:38:33.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.187+0000 7f09dbfb4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f09d415c7a0 con 0x7f09d4155390 2024-09-19T08:38:33.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.187+0000 7f09dbfb4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f09d415efc0 con 0x7f09d414fe20 2024-09-19T08:38:33.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.191+0000 7f09d9d50640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09c00039c0 0x7f09d4153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:33.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.191+0000 7f09d9d50640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09c00039c0 0x7f09d4153d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38174/0 (socket says 172.21.15.98:38174) 2024-09-19T08:38:33.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.191+0000 7f09d9d50640 1 -- 172.21.15.98:0/3189507275 learned_addr learned my addr 172.21.15.98:0/3189507275 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:33.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.191+0000 7f09da551640 1 --2- 172.21.15.98:0/3189507275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09d4155390 0x7f09d415de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:33.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.191+0000 7f09d954f640 1 --2- 172.21.15.98:0/3189507275 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f09d414fe20 0x7f09d41526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:33.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.191+0000 7f09d9d50640 1 -- 172.21.15.98:0/3189507275 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f09d414fe20 msgr2=0x7f09d41526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:33.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.191+0000 7f09d9d50640 1 --2- 172.21.15.98:0/3189507275 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f09d414fe20 0x7f09d41526a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:33.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.191+0000 7f09d9d50640 1 -- 172.21.15.98:0/3189507275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09d4155390 msgr2=0x7f09d415de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:33.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.191+0000 7f09d9d50640 1 --2- 172.21.15.98:0/3189507275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09d4155390 0x7f09d415de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:33.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.191+0000 7f09d9d50640 1 -- 172.21.15.98:0/3189507275 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09d41542f0 con 0x7f09c00039c0 2024-09-19T08:38:33.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.191+0000 7f09da551640 1 --2- 172.21.15.98:0/3189507275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09d4155390 0x7f09d415de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:33.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.191+0000 7f09d9d50640 1 --2- 172.21.15.98:0/3189507275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09c00039c0 0x7f09d4153d10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f09d0005040 tx=0x7f09c0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:33.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.191+0000 7f09ca7fc640 1 -- 172.21.15.98:0/3189507275 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d0007a40 con 0x7f09c00039c0 2024-09-19T08:38:33.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.191+0000 7f09ca7fc640 1 -- 172.21.15.98:0/3189507275 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09d00ba070 con 0x7f09c00039c0 2024-09-19T08:38:33.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.191+0000 7f09dbfb4640 1 -- 172.21.15.98:0/3189507275 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09d41545d0 con 0x7f09c00039c0 2024-09-19T08:38:33.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.191+0000 7f09ca7fc640 1 -- 172.21.15.98:0/3189507275 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d00054a0 con 0x7f09c00039c0 2024-09-19T08:38:33.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.191+0000 7f09dbfb4640 1 -- 172.21.15.98:0/3189507275 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09d4154b90 con 0x7f09c00039c0 2024-09-19T08:38:33.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.191+0000 7f09dbfb4640 1 -- 172.21.15.98:0/3189507275 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09d415f2f0 con 0x7f09c00039c0 2024-09-19T08:38:33.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.191+0000 7f09ca7fc640 1 -- 172.21.15.98:0/3189507275 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f09d0007030 con 0x7f09c00039c0 2024-09-19T08:38:33.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.191+0000 7f09ca7fc640 1 --2- 172.21.15.98:0/3189507275 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f09b80678c0 0x7f09b8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:33.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.191+0000 7f09ca7fc640 1 -- 172.21.15.98:0/3189507275 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f09d011fb20 con 0x7f09c00039c0 2024-09-19T08:38:33.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.191+0000 7f09d954f640 1 --2- 172.21.15.98:0/3189507275 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f09b80678c0 0x7f09b8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:33.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.191+0000 7f09d954f640 1 --2- 172.21.15.98:0/3189507275 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f09b80678c0 0x7f09b8069d80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f09d41505e0 tx=0x7f09c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:33.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.195+0000 7f09ca7fc640 1 -- 172.21.15.98:0/3189507275 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f09d0016110 con 0x7f09c00039c0 2024-09-19T08:38:33.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.379+0000 7f09dbfb4640 1 -- 172.21.15.98:0/3189507275 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09d4150200 con 0x7f09c00039c0 2024-09-19T08:38:33.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.379+0000 7f09ca7fc640 1 -- 172.21.15.98:0/3189507275 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5220 (secure 0 0 0) 0x7f09d00f0660 con 0x7f09c00039c0 2024-09-19T08:38:33.382 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:38:33.382 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":7,"btime":"2024-09-19T08:38:22:407519+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/2038744461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":2038744461},{"type":"v1","addr":"172.21.15.139:6833","nonce":2038744461}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/765134249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":765134249},{"type":"v1","addr":"172.21.15.98:6835","nonce":765134249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2832328076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2832328076},{"type":"v1","addr":"172.21.15.98:6837","nonce":2832328076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:38:18.717221+0000","modified":"2024-09-19T08:38:21.769948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6835/1879089763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1879089763},{"type":"v1","addr":"172.21.15.139:6835","nonce":1879089763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4261,"qdb_cluster":[4261]},"id":1}]} 2024-09-19T08:38:33.382 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 7 2024-09-19T08:38:33.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.383+0000 7f09dbfb4640 1 -- 172.21.15.98:0/3189507275 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f09b80678c0 msgr2=0x7f09b8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:33.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.383+0000 7f09dbfb4640 1 --2- 172.21.15.98:0/3189507275 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f09b80678c0 0x7f09b8069d80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f09d41505e0 tx=0x7f09c4008040 comp rx=0 tx=0).stop 2024-09-19T08:38:33.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.383+0000 7f09dbfb4640 1 -- 172.21.15.98:0/3189507275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09c00039c0 msgr2=0x7f09d4153d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:33.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.383+0000 7f09dbfb4640 1 --2- 172.21.15.98:0/3189507275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09c00039c0 0x7f09d4153d10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f09d0005040 tx=0x7f09c0005840 comp rx=0 tx=0).stop 2024-09-19T08:38:33.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.383+0000 7f09dbfb4640 1 -- 172.21.15.98:0/3189507275 shutdown_connections 2024-09-19T08:38:33.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.383+0000 7f09dbfb4640 1 --2- 172.21.15.98:0/3189507275 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f09b80678c0 0x7f09b8069d80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:33.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.383+0000 7f09dbfb4640 1 --2- 172.21.15.98:0/3189507275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f09d4155390 0x7f09d415de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:33.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.383+0000 7f09dbfb4640 1 --2- 172.21.15.98:0/3189507275 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f09d414fe20 0x7f09d41526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:33.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.383+0000 7f09dbfb4640 1 --2- 172.21.15.98:0/3189507275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f09c00039c0 0x7f09d4153d10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:33.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.383+0000 7f09dbfb4640 1 -- 172.21.15.98:0/3189507275 >> 172.21.15.98:0/3189507275 conn(0x7f09d40b2010 msgr2=0x7f09d415cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:33.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.383+0000 7f09dbfb4640 1 -- 172.21.15.98:0/3189507275 shutdown_connections 2024-09-19T08:38:33.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.383+0000 7f09dbfb4640 1 -- 172.21.15.98:0/3189507275 wait complete. 2024-09-19T08:38:33.399 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:38:33.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.515+0000 7f7fa5888640 1 Processor -- start 2024-09-19T08:38:33.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.515+0000 7f7fa5888640 1 -- start start 2024-09-19T08:38:33.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.515+0000 7f7fa5888640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7fa015c550 0x7f7fa006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:33.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.515+0000 7f7fa5888640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7fa00cfad0 0x7f7fa00723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:33.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.515+0000 7f7fa5888640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fa006d310 con 0x7f7fa006c1d0 2024-09-19T08:38:33.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.515+0000 7f7fa5888640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f7fa006c640 con 0x7f7fa015c550 2024-09-19T08:38:33.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.515+0000 7f7fa5888640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f7fa006d5f0 con 0x7f7fa00cfad0 2024-09-19T08:38:33.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.515+0000 7f7f9effd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7fa015c550 0x7f7fa006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:33.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.515+0000 7f7f9effd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7fa015c550 0x7f7fa006bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38186/0 (socket says 172.21.15.98:38186) 2024-09-19T08:38:33.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.515+0000 7f7f9effd640 1 -- 172.21.15.98:0/1388575743 learned_addr learned my addr 172.21.15.98:0/1388575743 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:33.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.515+0000 7f7f97fff640 1 --1- 172.21.15.98:0/1388575743 >> v1:172.21.15.98:6789/0 conn(0x7f7fa006c1d0 0x7f7fa00cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:60690/0 (socket says 172.21.15.98:60690) 2024-09-19T08:38:33.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.515+0000 7f7f9effd640 1 -- 172.21.15.98:0/1388575743 >> v2:172.21.15.139:3300/0 conn(0x7f7fa00cfad0 msgr2=0x7f7fa00723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:33.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.515+0000 7f7f9f7fe640 1 --2- 172.21.15.98:0/1388575743 >> v2:172.21.15.139:3300/0 conn(0x7f7fa00cfad0 0x7f7fa00723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:33.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.515+0000 7f7f9effd640 1 --2- 172.21.15.98:0/1388575743 >> v2:172.21.15.139:3300/0 conn(0x7f7fa00cfad0 0x7f7fa00723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:33.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.515+0000 7f7f9effd640 1 -- 172.21.15.98:0/1388575743 >> v1:172.21.15.98:6789/0 conn(0x7f7fa006c1d0 legacy=0x7f7fa00cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:33.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.515+0000 7f7f9effd640 1 -- 172.21.15.98:0/1388575743 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa006d8d0 con 0x7f7fa015c550 2024-09-19T08:38:33.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.515+0000 7f7f9f7fe640 1 --2- 172.21.15.98:0/1388575743 >> v2:172.21.15.139:3300/0 conn(0x7f7fa00cfad0 0x7f7fa00723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:33.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.515+0000 7f7f9effd640 1 --2- 172.21.15.98:0/1388575743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fa015c550 0x7f7fa006bc90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7fa00b33a0 tx=0x7f7f8801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb4b45482839f45c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:33.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.519+0000 7f7f9e7fc640 1 -- 172.21.15.98:0/1388575743 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f880b0070 con 0x7f7fa015c550 2024-09-19T08:38:33.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.519+0000 7f7f9e7fc640 1 -- 172.21.15.98:0/1388575743 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f880ab600 con 0x7f7fa015c550 2024-09-19T08:38:33.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.519+0000 7f7f9e7fc640 1 -- 172.21.15.98:0/1388575743 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f880ab9d0 con 0x7f7fa015c550 2024-09-19T08:38:33.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.519+0000 7f7fa5888640 1 -- 172.21.15.98:0/1388575743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fa015c550 msgr2=0x7f7fa006bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:33.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.519+0000 7f7fa5888640 1 --2- 172.21.15.98:0/1388575743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fa015c550 0x7f7fa006bc90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7fa00b33a0 tx=0x7f7f8801c520 comp rx=0 tx=0).stop 2024-09-19T08:38:33.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.519+0000 7f7fa5888640 1 -- 172.21.15.98:0/1388575743 shutdown_connections 2024-09-19T08:38:33.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.519+0000 7f7fa5888640 1 --2- 172.21.15.98:0/1388575743 >> v2:172.21.15.139:3300/0 conn(0x7f7fa00cfad0 0x7f7fa00723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:33.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.519+0000 7f7fa5888640 1 --2- 172.21.15.98:0/1388575743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fa015c550 0x7f7fa006bc90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:33.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.519+0000 7f7fa5888640 1 -- 172.21.15.98:0/1388575743 >> 172.21.15.98:0/1388575743 conn(0x7f7fa00b2010 msgr2=0x7f7fa00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:33.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.519+0000 7f7fa5888640 1 -- 172.21.15.98:0/1388575743 shutdown_connections 2024-09-19T08:38:33.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.519+0000 7f7fa5888640 1 -- 172.21.15.98:0/1388575743 wait complete. 2024-09-19T08:38:33.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.519+0000 7f7fa5888640 1 Processor -- start 2024-09-19T08:38:33.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.519+0000 7f7fa5888640 1 -- start start 2024-09-19T08:38:33.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.519+0000 7f7fa5888640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fa006c1d0 0x7f7fa014cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:33.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.519+0000 7f7fa5888640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fa00cfad0 0x7f7fa01f52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:33.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.519+0000 7f7fa5888640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7fa015c550 0x7f7fa01f5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:33.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.519+0000 7f7fa5888640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa00734c0 con 0x7f7fa006c1d0 2024-09-19T08:38:33.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.519+0000 7f7fa5888640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa0073340 con 0x7f7fa00cfad0 2024-09-19T08:38:33.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.519+0000 7f7fa5888640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7fa0073640 con 0x7f7fa015c550 2024-09-19T08:38:33.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.519+0000 7f7f9effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fa006c1d0 0x7f7fa014cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:33.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.519+0000 7f7f9effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fa006c1d0 0x7f7fa014cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38198/0 (socket says 172.21.15.98:38198) 2024-09-19T08:38:33.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.519+0000 7f7f9effd640 1 -- 172.21.15.98:0/3036357355 learned_addr learned my addr 172.21.15.98:0/3036357355 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:33.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.519+0000 7f7f9effd640 1 -- 172.21.15.98:0/3036357355 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7fa015c550 msgr2=0x7f7fa01f5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:33.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.519+0000 7f7f9effd640 1 --2- 172.21.15.98:0/3036357355 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7fa015c550 0x7f7fa01f5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:33.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.519+0000 7f7f9effd640 1 -- 172.21.15.98:0/3036357355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fa00cfad0 msgr2=0x7f7fa01f52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:33.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.519+0000 7f7f97fff640 1 --2- 172.21.15.98:0/3036357355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fa00cfad0 0x7f7fa01f52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:33.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.519+0000 7f7f9effd640 1 --2- 172.21.15.98:0/3036357355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fa00cfad0 0x7f7fa01f52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:33.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.519+0000 7f7f9effd640 1 -- 172.21.15.98:0/3036357355 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa0159ac0 con 0x7f7fa006c1d0 2024-09-19T08:38:33.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.519+0000 7f7f9effd640 1 --2- 172.21.15.98:0/3036357355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fa006c1d0 0x7f7fa014cf10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7f8801abd0 tx=0x7f7f880192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:33.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.519+0000 7f7f9cff9640 1 -- 172.21.15.98:0/3036357355 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f880bb070 con 0x7f7fa006c1d0 2024-09-19T08:38:33.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.519+0000 7f7f9cff9640 1 -- 172.21.15.98:0/3036357355 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f880b6630 con 0x7f7fa006c1d0 2024-09-19T08:38:33.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.519+0000 7f7f9cff9640 1 -- 172.21.15.98:0/3036357355 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f880b0040 con 0x7f7fa006c1d0 2024-09-19T08:38:33.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.519+0000 7f7fa5888640 1 -- 172.21.15.98:0/3036357355 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fa014e540 con 0x7f7fa006c1d0 2024-09-19T08:38:33.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.519+0000 7f7fa5888640 1 -- 172.21.15.98:0/3036357355 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fa01593d0 con 0x7f7fa006c1d0 2024-09-19T08:38:33.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.523+0000 7f7fa5888640 1 -- 172.21.15.98:0/3036357355 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fa01596c0 con 0x7f7fa006c1d0 2024-09-19T08:38:33.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.523+0000 7f7f9cff9640 1 -- 172.21.15.98:0/3036357355 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7f880be070 con 0x7f7fa006c1d0 2024-09-19T08:38:33.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.523+0000 7f7f9cff9640 1 --2- 172.21.15.98:0/3036357355 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7f800678c0 0x7f7f80069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:33.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.523+0000 7f7f97fff640 1 --2- 172.21.15.98:0/3036357355 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7f800678c0 0x7f7f80069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:33.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.523+0000 7f7f9cff9640 1 -- 172.21.15.98:0/3036357355 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7f8811f3d0 con 0x7f7fa006c1d0 2024-09-19T08:38:33.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.523+0000 7f7f97fff640 1 --2- 172.21.15.98:0/3036357355 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7f800678c0 0x7f7f80069d80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7fa015a340 tx=0x7f7f90009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:33.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.527+0000 7f7f9cff9640 1 -- 172.21.15.98:0/3036357355 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f7f88016110 con 0x7f7fa006c1d0 2024-09-19T08:38:33.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.671+0000 7f7fa5888640 1 -- 172.21.15.98:0/3036357355 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7fa0158fe0 con 0x7f7fa006c1d0 2024-09-19T08:38:33.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.675+0000 7f7f9cff9640 1 -- 172.21.15.98:0/3036357355 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16665 (secure 0 0 0) 0x7f7f880efe80 con 0x7f7fa006c1d0 2024-09-19T08:38:33.678 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:38:33.678 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":21,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:38:20.362527+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:38:16.588688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T08:38:17.700392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:38:33.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.675+0000 7f7fa5888640 1 -- 172.21.15.98:0/3036357355 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7f800678c0 msgr2=0x7f7f80069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:33.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.675+0000 7f7fa5888640 1 --2- 172.21.15.98:0/3036357355 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7f800678c0 0x7f7f80069d80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7fa015a340 tx=0x7f7f90009040 comp rx=0 tx=0).stop 2024-09-19T08:38:33.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.675+0000 7f7fa5888640 1 -- 172.21.15.98:0/3036357355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fa006c1d0 msgr2=0x7f7fa014cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:33.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.675+0000 7f7fa5888640 1 --2- 172.21.15.98:0/3036357355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fa006c1d0 0x7f7fa014cf10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7f8801abd0 tx=0x7f7f880192c0 comp rx=0 tx=0).stop 2024-09-19T08:38:33.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.675+0000 7f7fa5888640 1 -- 172.21.15.98:0/3036357355 shutdown_connections 2024-09-19T08:38:33.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.675+0000 7f7fa5888640 1 --2- 172.21.15.98:0/3036357355 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7f800678c0 0x7f7f80069d80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:33.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.675+0000 7f7fa5888640 1 --2- 172.21.15.98:0/3036357355 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7fa015c550 0x7f7fa01f5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:33.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.675+0000 7f7fa5888640 1 --2- 172.21.15.98:0/3036357355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7fa00cfad0 0x7f7fa01f52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:33.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.675+0000 7f7fa5888640 1 --2- 172.21.15.98:0/3036357355 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7fa006c1d0 0x7f7fa014cf10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:33.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.675+0000 7f7fa5888640 1 -- 172.21.15.98:0/3036357355 >> 172.21.15.98:0/3036357355 conn(0x7f7fa00b2010 msgr2=0x7f7fa00cd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:33.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.675+0000 7f7fa5888640 1 -- 172.21.15.98:0/3036357355 shutdown_connections 2024-09-19T08:38:33.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.675+0000 7f7fa5888640 1 -- 172.21.15.98:0/3036357355 wait complete. 2024-09-19T08:38:33.696 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:38:33.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.811+0000 7f4f22f87640 1 Processor -- start 2024-09-19T08:38:33.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.811+0000 7f4f22f87640 1 -- start start 2024-09-19T08:38:33.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.811+0000 7f4f22f87640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4f1c0d4820 0x7f4f1c1621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:33.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.811+0000 7f4f22f87640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f1c0b9970 0x7f4f1c0b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:33.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.811+0000 7f4f22f87640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f1c068440 con 0x7f4f1c0b7f80 2024-09-19T08:38:33.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.811+0000 7f4f22f87640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f4f1c163050 con 0x7f4f1c0d4820 2024-09-19T08:38:33.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.811+0000 7f4f22f87640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4f1c162a90 con 0x7f4f1c0b9970 2024-09-19T08:38:33.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.811+0000 7f4f21524640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f1c0b9970 0x7f4f1c0b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:33.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.811+0000 7f4f1bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4f1c0d4820 0x7f4f1c1621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:33.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.811+0000 7f4f21524640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f1c0b9970 0x7f4f1c0b9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:38230/0 (socket says 172.21.15.98:38230) 2024-09-19T08:38:33.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.811+0000 7f4f21524640 1 -- 172.21.15.98:0/793063639 learned_addr learned my addr 172.21.15.98:0/793063639 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:33.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.811+0000 7f4f21524640 1 -- 172.21.15.98:0/793063639 >> v2:172.21.15.139:3300/0 conn(0x7f4f1c0d4820 msgr2=0x7f4f1c1621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:33.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.811+0000 7f4f21524640 1 --2- 172.21.15.98:0/793063639 >> v2:172.21.15.139:3300/0 conn(0x7f4f1c0d4820 0x7f4f1c1621b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:33.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.811+0000 7f4f21524640 1 -- 172.21.15.98:0/793063639 >> v1:172.21.15.139:6789/0 conn(0x7f4f1c0b7f80 legacy=0x7f4f1c15f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:33.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.811+0000 7f4f21524640 1 -- 172.21.15.98:0/793063639 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f1c162d70 con 0x7f4f1c0b9970 2024-09-19T08:38:33.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.815+0000 7f4f21524640 1 --2- 172.21.15.98:0/793063639 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f1c0b9970 0x7f4f1c0b9de0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4f14009e50 tx=0x7f4f1401ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76c4f9c4a0c1825f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:33.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.815+0000 7f4f1b7fe640 1 -- 172.21.15.98:0/793063639 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f140a5810 con 0x7f4f1c0b9970 2024-09-19T08:38:33.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.815+0000 7f4f1b7fe640 1 -- 172.21.15.98:0/793063639 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f140a59b0 con 0x7f4f1c0b9970 2024-09-19T08:38:33.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.815+0000 7f4f1b7fe640 1 -- 172.21.15.98:0/793063639 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f140a5d80 con 0x7f4f1c0b9970 2024-09-19T08:38:33.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.815+0000 7f4f22f87640 1 -- 172.21.15.98:0/793063639 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f1c0b9970 msgr2=0x7f4f1c0b9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:33.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.815+0000 7f4f22f87640 1 --2- 172.21.15.98:0/793063639 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f1c0b9970 0x7f4f1c0b9de0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4f14009e50 tx=0x7f4f1401ca30 comp rx=0 tx=0).stop 2024-09-19T08:38:33.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.815+0000 7f4f22f87640 1 -- 172.21.15.98:0/793063639 shutdown_connections 2024-09-19T08:38:33.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.815+0000 7f4f22f87640 1 --2- 172.21.15.98:0/793063639 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f1c0b9970 0x7f4f1c0b9de0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:33.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.815+0000 7f4f22f87640 1 --2- 172.21.15.98:0/793063639 >> v2:172.21.15.139:3300/0 conn(0x7f4f1c0d4820 0x7f4f1c1621b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:33.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.815+0000 7f4f22f87640 1 -- 172.21.15.98:0/793063639 >> 172.21.15.98:0/793063639 conn(0x7f4f1c074e40 msgr2=0x7f4f1c075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:33.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.815+0000 7f4f22f87640 1 -- 172.21.15.98:0/793063639 shutdown_connections 2024-09-19T08:38:33.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.815+0000 7f4f22f87640 1 -- 172.21.15.98:0/793063639 wait complete. 2024-09-19T08:38:33.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.815+0000 7f4f22f87640 1 Processor -- start 2024-09-19T08:38:33.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.815+0000 7f4f22f87640 1 -- start start 2024-09-19T08:38:33.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.819+0000 7f4f22f87640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f1c0b7f80 0x7f4f1c150c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:33.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.819+0000 7f4f22f87640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f1c0b9970 0x7f4f1c0d3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:33.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.819+0000 7f4f22f87640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f1c0d4820 0x7f4f1c0d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:33.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.819+0000 7f4f22f87640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c0b84e0 con 0x7f4f1c0d4820 2024-09-19T08:38:33.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.819+0000 7f4f22f87640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c0b8360 con 0x7f4f1c0b9970 2024-09-19T08:38:33.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.819+0000 7f4f22f87640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4f1c144590 con 0x7f4f1c0b7f80 2024-09-19T08:38:33.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.819+0000 7f4f21524640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f1c0d4820 0x7f4f1c0d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:33.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.819+0000 7f4f21524640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f1c0d4820 0x7f4f1c0d3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38212/0 (socket says 172.21.15.98:38212) 2024-09-19T08:38:33.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.819+0000 7f4f21524640 1 -- 172.21.15.98:0/1433042943 learned_addr learned my addr 172.21.15.98:0/1433042943 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:33.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.819+0000 7f4f1bfff640 1 --2- 172.21.15.98:0/1433042943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f1c0b9970 0x7f4f1c0d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:33.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.819+0000 7f4f20d23640 1 --2- 172.21.15.98:0/1433042943 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f1c0b7f80 0x7f4f1c150c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:33.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.819+0000 7f4f21524640 1 -- 172.21.15.98:0/1433042943 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f1c0b7f80 msgr2=0x7f4f1c150c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:33.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.819+0000 7f4f21524640 1 --2- 172.21.15.98:0/1433042943 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f1c0b7f80 0x7f4f1c150c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:33.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.819+0000 7f4f21524640 1 -- 172.21.15.98:0/1433042943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f1c0b9970 msgr2=0x7f4f1c0d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:33.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.819+0000 7f4f21524640 1 --2- 172.21.15.98:0/1433042943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f1c0b9970 0x7f4f1c0d3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:33.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.819+0000 7f4f21524640 1 -- 172.21.15.98:0/1433042943 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f1c1513f0 con 0x7f4f1c0d4820 2024-09-19T08:38:33.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.819+0000 7f4f21524640 1 --2- 172.21.15.98:0/1433042943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f1c0d4820 0x7f4f1c0d3b70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4f1401a660 tx=0x7f4f140197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:33.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.819+0000 7f4f197fa640 1 -- 172.21.15.98:0/1433042943 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f140b8670 con 0x7f4f1c0d4820 2024-09-19T08:38:33.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.819+0000 7f4f197fa640 1 -- 172.21.15.98:0/1433042943 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f140be070 con 0x7f4f1c0d4820 2024-09-19T08:38:33.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.819+0000 7f4f197fa640 1 -- 172.21.15.98:0/1433042943 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f140b8a30 con 0x7f4f1c0d4820 2024-09-19T08:38:33.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.819+0000 7f4f22f87640 1 -- 172.21.15.98:0/1433042943 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f1c151170 con 0x7f4f1c0d4820 2024-09-19T08:38:33.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.819+0000 7f4f22f87640 1 -- 172.21.15.98:0/1433042943 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f1c154c90 con 0x7f4f1c0d4820 2024-09-19T08:38:33.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.819+0000 7f4f22f87640 1 -- 172.21.15.98:0/1433042943 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f1c0b9de0 con 0x7f4f1c0d4820 2024-09-19T08:38:33.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.819+0000 7f4f197fa640 1 -- 172.21.15.98:0/1433042943 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4f140b8bd0 con 0x7f4f1c0d4820 2024-09-19T08:38:33.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.819+0000 7f4f197fa640 1 --2- 172.21.15.98:0/1433042943 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4efc0678c0 0x7f4efc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:33.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.819+0000 7f4f197fa640 1 -- 172.21.15.98:0/1433042943 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4f14120de0 con 0x7f4f1c0d4820 2024-09-19T08:38:33.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.819+0000 7f4f20d23640 1 --2- 172.21.15.98:0/1433042943 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4efc0678c0 0x7f4efc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:33.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.819+0000 7f4f20d23640 1 --2- 172.21.15.98:0/1433042943 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4efc0678c0 0x7f4efc069d80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c000f70 tx=0x7f4f0c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:33.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:33.823+0000 7f4f197fa640 1 -- 172.21.15.98:0/1433042943 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f4f1c151170 con 0x7f4f1c0d4820 2024-09-19T08:38:34.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.023+0000 7f4f22f87640 1 -- 172.21.15.98:0/1433042943 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f1c141f50 con 0x7f4f1c0d4820 2024-09-19T08:38:34.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.023+0000 7f4f197fa640 1 -- 172.21.15.98:0/1433042943 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5220 (secure 0 0 0) 0x7f4f1c1513f0 con 0x7f4f1c0d4820 2024-09-19T08:38:34.027 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:38:34.027 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":7,"btime":"2024-09-19T08:38:22:407519+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/2038744461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":2038744461},{"type":"v1","addr":"172.21.15.139:6833","nonce":2038744461}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/765134249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":765134249},{"type":"v1","addr":"172.21.15.98:6835","nonce":765134249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2832328076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2832328076},{"type":"v1","addr":"172.21.15.98:6837","nonce":2832328076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:38:18.717221+0000","modified":"2024-09-19T08:38:21.769948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6835/1879089763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1879089763},{"type":"v1","addr":"172.21.15.139:6835","nonce":1879089763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4261,"qdb_cluster":[4261]},"id":1}]} 2024-09-19T08:38:34.027 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 7 2024-09-19T08:38:34.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.023+0000 7f4f22f87640 1 -- 172.21.15.98:0/1433042943 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4efc0678c0 msgr2=0x7f4efc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:34.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.023+0000 7f4f22f87640 1 --2- 172.21.15.98:0/1433042943 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4efc0678c0 0x7f4efc069d80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c000f70 tx=0x7f4f0c009040 comp rx=0 tx=0).stop 2024-09-19T08:38:34.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.023+0000 7f4f22f87640 1 -- 172.21.15.98:0/1433042943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f1c0d4820 msgr2=0x7f4f1c0d3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:34.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.023+0000 7f4f22f87640 1 --2- 172.21.15.98:0/1433042943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f1c0d4820 0x7f4f1c0d3b70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4f1401a660 tx=0x7f4f140197d0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.023+0000 7f4f22f87640 1 -- 172.21.15.98:0/1433042943 shutdown_connections 2024-09-19T08:38:34.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.023+0000 7f4f22f87640 1 --2- 172.21.15.98:0/1433042943 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4efc0678c0 0x7f4efc069d80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.023+0000 7f4f22f87640 1 --2- 172.21.15.98:0/1433042943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f1c0d4820 0x7f4f1c0d3b70 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.023+0000 7f4f22f87640 1 --2- 172.21.15.98:0/1433042943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f1c0b9970 0x7f4f1c0d3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.023+0000 7f4f22f87640 1 --2- 172.21.15.98:0/1433042943 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f1c0b7f80 0x7f4f1c150c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.023+0000 7f4f22f87640 1 -- 172.21.15.98:0/1433042943 >> 172.21.15.98:0/1433042943 conn(0x7f4f1c074e40 msgr2=0x7f4f1c1603e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:34.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.023+0000 7f4f22f87640 1 -- 172.21.15.98:0/1433042943 shutdown_connections 2024-09-19T08:38:34.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.023+0000 7f4f22f87640 1 -- 172.21.15.98:0/1433042943 wait complete. 2024-09-19T08:38:34.448 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T08:38:18.717221+0000', 'modified': '2024-09-19T08:38:21.769948+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4261}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4261': {'gid': 4261, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.139:6835/1879089763', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6834', 'nonce': 1879089763}, {'type': 'v1', 'addr': '172.21.15.139:6835', 'nonce': 1879089763}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4261, 'qdb_cluster': [4261]} 2024-09-19T08:38:34.449 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T08:38:34.449 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-19T08:38:34.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.575+0000 7f5a1188f640 1 Processor -- start 2024-09-19T08:38:34.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.575+0000 7f5a1188f640 1 -- start start 2024-09-19T08:38:34.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.575+0000 7f5a1188f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a0c15c550 0x7f5a0c06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:34.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.575+0000 7f5a1188f640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5a0c06c1d0 0x7f5a0c0cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:34.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.575+0000 7f5a1188f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a0c06d310 con 0x7f5a0c0cf900 2024-09-19T08:38:34.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.575+0000 7f5a1188f640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f5a0c06c610 con 0x7f5a0c06c1d0 2024-09-19T08:38:34.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.575+0000 7f5a1188f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5a0c06d5f0 con 0x7f5a0c15c550 2024-09-19T08:38:34.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.575+0000 7f5a0a7fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5a0c06c1d0 0x7f5a0c0cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:34.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.575+0000 7f5a0a7fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5a0c06c1d0 0x7f5a0c0cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38224/0 (socket says 172.21.15.98:38224) 2024-09-19T08:38:34.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.575+0000 7f5a0a7fc640 1 -- 172.21.15.98:0/3231125933 learned_addr learned my addr 172.21.15.98:0/3231125933 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:34.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.575+0000 7f5a0affd640 1 --2- 172.21.15.98:0/3231125933 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a0c15c550 0x7f5a0c06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:34.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.575+0000 7f5a0a7fc640 1 -- 172.21.15.98:0/3231125933 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a0c15c550 msgr2=0x7f5a0c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:34.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.575+0000 7f5a0a7fc640 1 --2- 172.21.15.98:0/3231125933 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a0c15c550 0x7f5a0c06bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.575+0000 7f5a0a7fc640 1 -- 172.21.15.98:0/3231125933 >> v1:172.21.15.139:6789/0 conn(0x7f5a0c0cf900 legacy=0x7f5a0c0ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:34.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.575+0000 7f5a0a7fc640 1 -- 172.21.15.98:0/3231125933 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a0c06d8d0 con 0x7f5a0c06c1d0 2024-09-19T08:38:34.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.579+0000 7f5a0a7fc640 1 --2- 172.21.15.98:0/3231125933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a0c06c1d0 0x7f5a0c0cf3c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5a00009870 tx=0x7f5a0001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834f8bb9298060b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:34.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.579+0000 7f5a09ffb640 1 -- 172.21.15.98:0/3231125933 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a000b2070 con 0x7f5a0c06c1d0 2024-09-19T08:38:34.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.579+0000 7f5a09ffb640 1 -- 172.21.15.98:0/3231125933 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a000a9070 con 0x7f5a0c06c1d0 2024-09-19T08:38:34.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.579+0000 7f5a09ffb640 1 -- 172.21.15.98:0/3231125933 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a000ad880 con 0x7f5a0c06c1d0 2024-09-19T08:38:34.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.579+0000 7f5a1188f640 1 -- 172.21.15.98:0/3231125933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a0c06c1d0 msgr2=0x7f5a0c0cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:34.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.579+0000 7f5a1188f640 1 --2- 172.21.15.98:0/3231125933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a0c06c1d0 0x7f5a0c0cf3c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5a00009870 tx=0x7f5a0001c560 comp rx=0 tx=0).stop 2024-09-19T08:38:34.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.579+0000 7f5a1188f640 1 -- 172.21.15.98:0/3231125933 shutdown_connections 2024-09-19T08:38:34.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.579+0000 7f5a1188f640 1 --2- 172.21.15.98:0/3231125933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a0c06c1d0 0x7f5a0c0cf3c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.579+0000 7f5a1188f640 1 --2- 172.21.15.98:0/3231125933 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a0c15c550 0x7f5a0c06bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.579+0000 7f5a1188f640 1 -- 172.21.15.98:0/3231125933 >> 172.21.15.98:0/3231125933 conn(0x7f5a0c0b2010 msgr2=0x7f5a0c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:34.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.579+0000 7f5a1188f640 1 -- 172.21.15.98:0/3231125933 shutdown_connections 2024-09-19T08:38:34.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.579+0000 7f5a1188f640 1 -- 172.21.15.98:0/3231125933 wait complete. 2024-09-19T08:38:34.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.579+0000 7f5a1188f640 1 Processor -- start 2024-09-19T08:38:34.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.579+0000 7f5a1188f640 1 -- start start 2024-09-19T08:38:34.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.583+0000 7f5a1188f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a0c0cf900 0x7f5a0c1496d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:34.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.583+0000 7f5a1188f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a0c15c550 0x7f5a0c1fb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:34.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.583+0000 7f5a1188f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a0c0b4920 0x7f5a0c153620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:34.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.583+0000 7f5a1188f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0c169230 con 0x7f5a0c15c550 2024-09-19T08:38:34.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.583+0000 7f5a1188f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0c15cb40 con 0x7f5a0c0cf900 2024-09-19T08:38:34.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.583+0000 7f5a1188f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5a0c1693b0 con 0x7f5a0c0b4920 2024-09-19T08:38:34.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.583+0000 7f5a0a7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a0c15c550 0x7f5a0c1fb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:34.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.583+0000 7f5a0a7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a0c15c550 0x7f5a0c1fb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38236/0 (socket says 172.21.15.98:38236) 2024-09-19T08:38:34.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.583+0000 7f5a0a7fc640 1 -- 172.21.15.98:0/169045273 learned_addr learned my addr 172.21.15.98:0/169045273 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:34.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.583+0000 7f5a0affd640 1 --2- 172.21.15.98:0/169045273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a0c0cf900 0x7f5a0c1496d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:34.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.583+0000 7f5a0b7fe640 1 --2- 172.21.15.98:0/169045273 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a0c0b4920 0x7f5a0c153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:34.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.583+0000 7f5a0a7fc640 1 -- 172.21.15.98:0/169045273 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a0c0b4920 msgr2=0x7f5a0c153620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:34.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.583+0000 7f5a0a7fc640 1 --2- 172.21.15.98:0/169045273 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a0c0b4920 0x7f5a0c153620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.583+0000 7f5a0a7fc640 1 -- 172.21.15.98:0/169045273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a0c0cf900 msgr2=0x7f5a0c1496d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:34.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.583+0000 7f5a0a7fc640 1 --2- 172.21.15.98:0/169045273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a0c0cf900 0x7f5a0c1496d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.583+0000 7f5a0a7fc640 1 -- 172.21.15.98:0/169045273 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a0c0b38a0 con 0x7f5a0c15c550 2024-09-19T08:38:34.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.583+0000 7f5a0affd640 1 --2- 172.21.15.98:0/169045273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a0c0cf900 0x7f5a0c1496d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:38:34.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.583+0000 7f5a0a7fc640 1 --2- 172.21.15.98:0/169045273 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a0c15c550 0x7f5a0c1fb8c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5a000b0040 tx=0x7f5a0001ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:34.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.583+0000 7f59f3fff640 1 -- 172.21.15.98:0/169045273 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a000be070 con 0x7f5a0c15c550 2024-09-19T08:38:34.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.583+0000 7f59f3fff640 1 -- 172.21.15.98:0/169045273 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a000addd0 con 0x7f5a0c15c550 2024-09-19T08:38:34.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.583+0000 7f5a1188f640 1 -- 172.21.15.98:0/169045273 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a0c0b3e40 con 0x7f5a0c15c550 2024-09-19T08:38:34.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.583+0000 7f59f3fff640 1 -- 172.21.15.98:0/169045273 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a000b2040 con 0x7f5a0c15c550 2024-09-19T08:38:34.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.583+0000 7f5a1188f640 1 -- 172.21.15.98:0/169045273 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a0c1fbe00 con 0x7f5a0c15c550 2024-09-19T08:38:34.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.583+0000 7f59f3fff640 1 -- 172.21.15.98:0/169045273 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5a000c1070 con 0x7f5a0c15c550 2024-09-19T08:38:34.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.583+0000 7f59f3fff640 1 --2- 172.21.15.98:0/169045273 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f59f80678f0 0x7f59f8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:34.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.583+0000 7f59f3fff640 1 -- 172.21.15.98:0/169045273 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5a0011f480 con 0x7f5a0c15c550 2024-09-19T08:38:34.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.583+0000 7f5a0affd640 1 --2- 172.21.15.98:0/169045273 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f59f80678f0 0x7f59f8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:34.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.587+0000 7f5a0affd640 1 --2- 172.21.15.98:0/169045273 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f59f80678f0 0x7f59f8069db0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00b9f0 tx=0x7f59fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:34.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.587+0000 7f5a1494b000 1 Processor -- start 2024-09-19T08:38:34.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.587+0000 7f5a1494b000 1 -- start start 2024-09-19T08:38:34.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.587+0000 7f5a1494b000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2bffab20 0x562c2bffaf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:34.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.587+0000 7f5a1494b000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c2bffb440 0x562c2bfb0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:34.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.587+0000 7f5a1494b000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c2bfb1290 0x562c2bfb1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:34.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.587+0000 7f5a1494b000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562c2be98b60 con 0x562c2bffab20 2024-09-19T08:38:34.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.587+0000 7f5a1494b000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562c2bdffc70 con 0x562c2bfb1290 2024-09-19T08:38:34.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.587+0000 7f5a1494b000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x562c2bda5ed0 con 0x562c2bffb440 2024-09-19T08:38:34.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.587+0000 7f5a0affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2bffab20 0x562c2bffaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:34.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.587+0000 7f5a0affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2bffab20 0x562c2bffaf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38242/0 (socket says 172.21.15.98:38242) 2024-09-19T08:38:34.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.587+0000 7f5a0affd640 1 -- 172.21.15.98:0/531073176 learned_addr learned my addr 172.21.15.98:0/531073176 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:34.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.587+0000 7f5a0b7fe640 1 --2- 172.21.15.98:0/531073176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c2bfb1290 0x562c2bfb1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:34.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.587+0000 7f5a0a7fc640 1 --2- 172.21.15.98:0/531073176 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c2bffb440 0x562c2bfb0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:34.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.587+0000 7f5a0affd640 1 -- 172.21.15.98:0/531073176 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c2bffb440 msgr2=0x562c2bfb0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:34.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.587+0000 7f5a0affd640 1 --2- 172.21.15.98:0/531073176 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c2bffb440 0x562c2bfb0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.587+0000 7f5a0affd640 1 -- 172.21.15.98:0/531073176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c2bfb1290 msgr2=0x562c2bfb1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:34.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.587+0000 7f5a0affd640 1 --2- 172.21.15.98:0/531073176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c2bfb1290 0x562c2bfb1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.587+0000 7f5a0affd640 1 -- 172.21.15.98:0/531073176 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c2bdb7e50 con 0x562c2bffab20 2024-09-19T08:38:34.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.587+0000 7f5a0affd640 1 --2- 172.21.15.98:0/531073176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2bffab20 0x562c2bffaf00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00fab0 tx=0x7f59fc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f0ec11147118e22 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:34.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.587+0000 7f59f17fa640 1 -- 172.21.15.98:0/531073176 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59fc01b050 con 0x562c2bffab20 2024-09-19T08:38:34.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.587+0000 7f59f17fa640 1 -- 172.21.15.98:0/531073176 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59fc025070 con 0x562c2bffab20 2024-09-19T08:38:34.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.587+0000 7f59f17fa640 1 -- 172.21.15.98:0/531073176 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59fc020440 con 0x562c2bffab20 2024-09-19T08:38:34.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.587+0000 7f5a1494b000 1 -- 172.21.15.98:0/531073176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2bffab20 msgr2=0x562c2bffaf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:34.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.587+0000 7f5a1494b000 1 --2- 172.21.15.98:0/531073176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2bffab20 0x562c2bffaf00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00fab0 tx=0x7f59fc012000 comp rx=0 tx=0).stop 2024-09-19T08:38:34.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.587+0000 7f5a1494b000 1 -- 172.21.15.98:0/531073176 shutdown_connections 2024-09-19T08:38:34.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.587+0000 7f5a1494b000 1 --2- 172.21.15.98:0/531073176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c2bfb1290 0x562c2bfb1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.587+0000 7f5a1494b000 1 --2- 172.21.15.98:0/531073176 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c2bffb440 0x562c2bfb0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.587+0000 7f5a1494b000 1 --2- 172.21.15.98:0/531073176 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2bffab20 0x562c2bffaf00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.587+0000 7f5a1494b000 1 -- 172.21.15.98:0/531073176 >> 172.21.15.98:0/531073176 conn(0x562c2bfb7600 msgr2=0x562c2bfaf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:34.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.587+0000 7f5a1494b000 1 -- 172.21.15.98:0/531073176 shutdown_connections 2024-09-19T08:38:34.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.587+0000 7f5a1494b000 1 -- 172.21.15.98:0/531073176 wait complete. 2024-09-19T08:38:34.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.587+0000 7f5a1494b000 1 -- start start 2024-09-19T08:38:34.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.587+0000 7f5a1494b000 1 Processor -- start 2024-09-19T08:38:34.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f5a1494b000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2bffab20 0x562c2bfb5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:34.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f5a1494b000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c2bffb440 0x562c2bfbbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:34.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f5a1494b000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c2bfb6390 0x562c2bfbc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:34.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f5a1494b000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562c2be61b00 con 0x562c2bffab20 2024-09-19T08:38:34.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f5a1494b000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562c2bdda360 con 0x562c2bffb440 2024-09-19T08:38:34.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f5a1494b000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x562c2bdd72d0 con 0x562c2bfb6390 2024-09-19T08:38:34.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f5a1494b000 10 client.0 _ll_register_callbacks cb 0x562c2bf9c820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:38:34.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f5a0affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2bffab20 0x562c2bfb5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:34.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f5a0affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2bffab20 0x562c2bfb5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38246/0 (socket says 172.21.15.98:38246) 2024-09-19T08:38:34.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f5a0affd640 1 -- 172.21.15.98:0/3087545481 learned_addr learned my addr 172.21.15.98:0/3087545481 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:34.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f5a0a7fc640 1 --2- 172.21.15.98:0/3087545481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c2bffb440 0x562c2bfbbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:34.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f5a0b7fe640 1 --2- 172.21.15.98:0/3087545481 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c2bfb6390 0x562c2bfbc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:34.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f5a0affd640 1 -- 172.21.15.98:0/3087545481 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c2bfb6390 msgr2=0x562c2bfbc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:34.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f5a0affd640 1 --2- 172.21.15.98:0/3087545481 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c2bfb6390 0x562c2bfbc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f5a0affd640 1 -- 172.21.15.98:0/3087545481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c2bffb440 msgr2=0x562c2bfbbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:34.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f5a0affd640 1 --2- 172.21.15.98:0/3087545481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c2bffb440 0x562c2bfbbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f5a0affd640 1 -- 172.21.15.98:0/3087545481 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c2bda6120 con 0x562c2bffab20 2024-09-19T08:38:34.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f5a0a7fc640 1 --2- 172.21.15.98:0/3087545481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c2bffb440 0x562c2bfbbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:34.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f5a0affd640 1 --2- 172.21.15.98:0/3087545481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2bffab20 0x562c2bfb5e50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00b600 tx=0x7f59fc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a63be7b434e4cdbc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:34.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f59ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-19T08:38:34.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f59ea7fc640 1 -- 172.21.15.98:0/3087545481 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59fc020c80 con 0x562c2bffab20 2024-09-19T08:38:34.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f59ea7fc640 1 -- 172.21.15.98:0/3087545481 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f59fc034470 con 0x562c2bffab20 2024-09-19T08:38:34.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f5a1494b000 1 -- 172.21.15.98:0/3087545481 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59fc025070 con 0x562c2bffab20 2024-09-19T08:38:34.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f59ea7fc640 1 -- 172.21.15.98:0/3087545481 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f59fc01b020 con 0x562c2bffab20 2024-09-19T08:38:34.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f5a1494b000 10 client.4599 fetch_fsmap learned FSMap version 7 2024-09-19T08:38:34.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f5a1494b000 1 -- 172.21.15.98:0/3087545481 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x562c2bd9f450 con 0x562c2bffab20 2024-09-19T08:38:34.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f59ea7fc640 1 -- 172.21.15.98:0/3087545481 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59fc025070 con 0x562c2bffab20 2024-09-19T08:38:34.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f59ea7fc640 1 -- 172.21.15.98:0/3087545481 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f59fc034670 con 0x562c2bffab20 2024-09-19T08:38:34.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f5a1494b000 10 client.4599 fetch_fsmap finished waiting for FSMap version 7 2024-09-19T08:38:34.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f5a1494b000 10 client.4599 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-19T08:38:34.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f5a1494b000 20 client.4599 populate_metadata read hostname 'smithi098' 2024-09-19T08:38:34.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f5a1494b000 1 --2- 172.21.15.98:0/3087545481 >> [v2:172.21.15.139:6834/1879089763,v1:172.21.15.139:6835/1879089763] conn(0x562c2c009280 0x562c2c00b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:34.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f5a1494b000 4 client.4599 mds_command: new command op to 4261 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:38:34.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f5a1494b000 1 -- 172.21.15.98:0/3087545481 --> [v2:172.21.15.139:6834/1879089763,v1:172.21.15.139:6835/1879089763] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c2bd99c00 con 0x562c2c009280 2024-09-19T08:38:34.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f5a0a7fc640 1 --2- 172.21.15.98:0/3087545481 >> [v2:172.21.15.139:6834/1879089763,v1:172.21.15.139:6835/1879089763] conn(0x562c2c009280 0x562c2c00b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:34.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f59ea7fc640 10 client.4599 ms_handle_connect on v2:172.21.15.139:6834/1879089763 2024-09-19T08:38:34.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.591+0000 7f5a0a7fc640 1 --2- 172.21.15.98:0/3087545481 >> [v2:172.21.15.139:6834/1879089763,v1:172.21.15.139:6835/1879089763] conn(0x562c2c009280 0x562c2c00b670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=944f1efbdbbc4c84 server_cookie=fa781af8dc890a08 in_seq=0 out_seq=0 2024-09-19T08:38:34.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.595+0000 7f59ea7fc640 1 -- 172.21.15.98:0/3087545481 <== mds.0 v2:172.21.15.139:6834/1879089763 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562c2bd99c00 con 0x562c2c009280 2024-09-19T08:38:34.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.595+0000 7f59ea7fc640 10 client.4599 handle_command_reply: tid=0 2024-09-19T08:38:34.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.595+0000 7f5a1494b000 1 client.4599 shutdown 2024-09-19T08:38:34.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.595+0000 7f5a1494b000 1 -- 172.21.15.98:0/3087545481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2bffab20 msgr2=0x562c2bfb5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:34.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.595+0000 7f5a1494b000 1 --2- 172.21.15.98:0/3087545481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2bffab20 0x562c2bfb5e50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00b600 tx=0x7f59fc004740 comp rx=0 tx=0).stop 2024-09-19T08:38:34.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.595+0000 7f5a1494b000 1 -- 172.21.15.98:0/3087545481 shutdown_connections 2024-09-19T08:38:34.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.595+0000 7f5a1494b000 1 --2- 172.21.15.98:0/3087545481 >> [v2:172.21.15.139:6834/1879089763,v1:172.21.15.139:6835/1879089763] conn(0x562c2c009280 0x562c2c00b670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.595+0000 7f5a1494b000 1 --2- 172.21.15.98:0/3087545481 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c2bfb6390 0x562c2bfbc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.595+0000 7f5a1494b000 1 --2- 172.21.15.98:0/3087545481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c2bffb440 0x562c2bfbbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.595+0000 7f5a1494b000 1 --2- 172.21.15.98:0/3087545481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2bffab20 0x562c2bfb5e50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.595+0000 7f59ea7fc640 0 client.4599 ms_handle_reset on v2:172.21.15.139:6834/1879089763 2024-09-19T08:38:34.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.595+0000 7f5a1494b000 1 -- 172.21.15.98:0/3087545481 >> 172.21.15.98:0/3087545481 conn(0x562c2bfb1290 msgr2=0x562c2bfafd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:34.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.595+0000 7f5a1494b000 1 -- 172.21.15.98:0/3087545481 shutdown_connections 2024-09-19T08:38:34.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.595+0000 7f5a1494b000 1 -- 172.21.15.98:0/3087545481 wait complete. 2024-09-19T08:38:34.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.595+0000 7f5a1494b000 20 client.4599 trim_cache size 0 max 16384 2024-09-19T08:38:34.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a1494b000 1 Processor -- start 2024-09-19T08:38:34.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a1494b000 1 -- start start 2024-09-19T08:38:34.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a1494b000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c2bfb6390 0x562c2bff9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:34.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a1494b000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c2bffab20 0x562c2bffa200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:34.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a1494b000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2bffb440 0x562c2c00b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:34.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a1494b000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0c203350 con 0x562c2bffb440 2024-09-19T08:38:34.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a1494b000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0c158d70 con 0x562c2bfb6390 2024-09-19T08:38:34.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a1494b000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5a0c1593c0 con 0x562c2bffab20 2024-09-19T08:38:34.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a0b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2bffb440 0x562c2c00b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:34.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a0b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2bffb440 0x562c2c00b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38254/0 (socket says 172.21.15.98:38254) 2024-09-19T08:38:34.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a0b7fe640 1 -- 172.21.15.98:0/1398366836 learned_addr learned my addr 172.21.15.98:0/1398366836 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:34.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a0affd640 1 --2- 172.21.15.98:0/1398366836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c2bfb6390 0x562c2bff9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:34.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a0a7fc640 1 --2- 172.21.15.98:0/1398366836 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c2bffab20 0x562c2bffa200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:34.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a0b7fe640 1 -- 172.21.15.98:0/1398366836 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c2bffab20 msgr2=0x562c2bffa200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:34.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a0b7fe640 1 --2- 172.21.15.98:0/1398366836 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c2bffab20 0x562c2bffa200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a0b7fe640 1 -- 172.21.15.98:0/1398366836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c2bfb6390 msgr2=0x562c2bff9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:34.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a0b7fe640 1 --2- 172.21.15.98:0/1398366836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c2bfb6390 0x562c2bff9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a0b7fe640 1 -- 172.21.15.98:0/1398366836 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c2bd86fd0 con 0x562c2bffb440 2024-09-19T08:38:34.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a0b7fe640 1 --2- 172.21.15.98:0/1398366836 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2bffb440 0x562c2c00b670 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5a040051b0 tx=0x7f5a04005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d23690f4b000efe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:34.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f59f0ff9640 1 -- 172.21.15.98:0/1398366836 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a04010040 con 0x562c2bffb440 2024-09-19T08:38:34.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f59f0ff9640 1 -- 172.21.15.98:0/1398366836 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a04001cb0 con 0x562c2bffb440 2024-09-19T08:38:34.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f59f0ff9640 1 -- 172.21.15.98:0/1398366836 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a04002060 con 0x562c2bffb440 2024-09-19T08:38:34.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a1494b000 1 -- 172.21.15.98:0/1398366836 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2bffb440 msgr2=0x562c2c00b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:34.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a1494b000 1 --2- 172.21.15.98:0/1398366836 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2bffb440 0x562c2c00b670 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5a040051b0 tx=0x7f5a04005670 comp rx=0 tx=0).stop 2024-09-19T08:38:34.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a1494b000 1 -- 172.21.15.98:0/1398366836 shutdown_connections 2024-09-19T08:38:34.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a1494b000 1 --2- 172.21.15.98:0/1398366836 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2bffb440 0x562c2c00b670 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a1494b000 1 --2- 172.21.15.98:0/1398366836 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c2bffab20 0x562c2bffa200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a1494b000 1 --2- 172.21.15.98:0/1398366836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c2bfb6390 0x562c2bff9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a1494b000 1 -- 172.21.15.98:0/1398366836 >> 172.21.15.98:0/1398366836 conn(0x562c2bfb1290 msgr2=0x562c2bfb5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:34.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a1494b000 1 -- 172.21.15.98:0/1398366836 shutdown_connections 2024-09-19T08:38:34.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a1494b000 1 -- 172.21.15.98:0/1398366836 wait complete. 2024-09-19T08:38:34.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a1494b000 1 -- start start 2024-09-19T08:38:34.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a1494b000 1 Processor -- start 2024-09-19T08:38:34.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a1494b000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2bfb6390 0x562c2c00e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:34.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a1494b000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c2bffab20 0x562c2c00e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:34.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a1494b000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c2bffb440 0x562c2c029960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:34.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a1494b000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562c2bdfef80 con 0x562c2bfb6390 2024-09-19T08:38:34.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a1494b000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562c2bdb7ca0 con 0x562c2bffab20 2024-09-19T08:38:34.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a1494b000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x562c2be042a0 con 0x562c2bffb440 2024-09-19T08:38:34.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a1494b000 10 client.0 _ll_register_callbacks cb 0x7f59fc00f9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:38:34.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a0affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2bfb6390 0x562c2c00e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:34.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a0affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2bfb6390 0x562c2c00e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38264/0 (socket says 172.21.15.98:38264) 2024-09-19T08:38:34.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a0affd640 1 -- 172.21.15.98:0/1851520234 learned_addr learned my addr 172.21.15.98:0/1851520234 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:34.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a0b7fe640 1 --2- 172.21.15.98:0/1851520234 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c2bffb440 0x562c2c029960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:34.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a0affd640 1 -- 172.21.15.98:0/1851520234 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c2bffb440 msgr2=0x562c2c029960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:34.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a0affd640 1 --2- 172.21.15.98:0/1851520234 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c2bffb440 0x562c2c029960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a0affd640 1 -- 172.21.15.98:0/1851520234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c2bffab20 msgr2=0x562c2c00e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:34.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a0affd640 1 --2- 172.21.15.98:0/1851520234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c2bffab20 0x562c2c00e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a0affd640 1 -- 172.21.15.98:0/1851520234 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c2be82750 con 0x562c2bfb6390 2024-09-19T08:38:34.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a0affd640 1 --2- 172.21.15.98:0/1851520234 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2bfb6390 0x562c2c00e320 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f59fc0148b0 tx=0x7f59fc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d5b95e63a44e6f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:34.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f59ea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-19T08:38:34.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f59ea7fc640 1 -- 172.21.15.98:0/1851520234 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59fc044020 con 0x562c2bfb6390 2024-09-19T08:38:34.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f59ea7fc640 1 -- 172.21.15.98:0/1851520234 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f59fc020a80 con 0x562c2bfb6390 2024-09-19T08:38:34.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a1494b000 1 -- 172.21.15.98:0/1851520234 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a04001cb0 con 0x562c2bfb6390 2024-09-19T08:38:34.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f59ea7fc640 1 -- 172.21.15.98:0/1851520234 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f59fc020c80 con 0x562c2bfb6390 2024-09-19T08:38:34.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a1494b000 10 client.4605 fetch_fsmap learned FSMap version 7 2024-09-19T08:38:34.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a1494b000 1 -- 172.21.15.98:0/1851520234 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x562c2bd8da30 con 0x562c2bfb6390 2024-09-19T08:38:34.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f59ea7fc640 1 -- 172.21.15.98:0/1851520234 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a04001cb0 con 0x562c2bfb6390 2024-09-19T08:38:34.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f59ea7fc640 1 -- 172.21.15.98:0/1851520234 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f59fc0059a0 con 0x562c2bfb6390 2024-09-19T08:38:34.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a1494b000 10 client.4605 fetch_fsmap finished waiting for FSMap version 7 2024-09-19T08:38:34.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a1494b000 10 client.4605 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-19T08:38:34.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a1494b000 20 client.4605 populate_metadata read hostname 'smithi098' 2024-09-19T08:38:34.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a1494b000 1 --2- 172.21.15.98:0/1851520234 >> [v2:172.21.15.139:6834/1879089763,v1:172.21.15.139:6835/1879089763] conn(0x562c2c02a320 0x562c2c06dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:34.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a1494b000 4 client.4605 mds_command: new command op to 4261 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T08:38:34.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a1494b000 1 -- 172.21.15.98:0/1851520234 --> [v2:172.21.15.139:6834/1879089763,v1:172.21.15.139:6835/1879089763] -- command(tid 0: {"prefix": "status"}) -- 0x562c2be0d3a0 con 0x562c2c02a320 2024-09-19T08:38:34.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.615+0000 7f5a0a7fc640 1 --2- 172.21.15.98:0/1851520234 >> [v2:172.21.15.139:6834/1879089763,v1:172.21.15.139:6835/1879089763] conn(0x562c2c02a320 0x562c2c06dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:34.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.619+0000 7f59ea7fc640 10 client.4605 ms_handle_connect on v2:172.21.15.139:6834/1879089763 2024-09-19T08:38:34.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.619+0000 7f5a0a7fc640 1 --2- 172.21.15.98:0/1851520234 >> [v2:172.21.15.139:6834/1879089763,v1:172.21.15.139:6835/1879089763] conn(0x562c2c02a320 0x562c2c06dc00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=899993795cdfb04 server_cookie=bb20dde5e90fa39f in_seq=0 out_seq=0 2024-09-19T08:38:34.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.619+0000 7f59ea7fc640 1 -- 172.21.15.98:0/1851520234 <== mds.0 v2:172.21.15.139:6834/1879089763 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x562c2be0d3a0 con 0x562c2c02a320 2024-09-19T08:38:34.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.619+0000 7f59ea7fc640 10 client.4605 handle_command_reply: tid=0 2024-09-19T08:38:34.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.619+0000 7f5a1494b000 1 client.4605 shutdown 2024-09-19T08:38:34.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.619+0000 7f5a1494b000 1 -- 172.21.15.98:0/1851520234 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2bfb6390 msgr2=0x562c2c00e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:34.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.619+0000 7f5a1494b000 1 --2- 172.21.15.98:0/1851520234 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2bfb6390 0x562c2c00e320 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f59fc0148b0 tx=0x7f59fc015000 comp rx=0 tx=0).stop 2024-09-19T08:38:34.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.619+0000 7f5a1494b000 1 -- 172.21.15.98:0/1851520234 shutdown_connections 2024-09-19T08:38:34.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.619+0000 7f5a1494b000 1 --2- 172.21.15.98:0/1851520234 >> [v2:172.21.15.139:6834/1879089763,v1:172.21.15.139:6835/1879089763] conn(0x562c2c02a320 0x562c2c06dc00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.619+0000 7f5a1494b000 1 --2- 172.21.15.98:0/1851520234 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562c2bffb440 0x562c2c029960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.619+0000 7f5a1494b000 1 --2- 172.21.15.98:0/1851520234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562c2bffab20 0x562c2c00e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.619+0000 7f59ea7fc640 0 client.4605 ms_handle_reset on v2:172.21.15.139:6834/1879089763 2024-09-19T08:38:34.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.619+0000 7f5a1494b000 1 --2- 172.21.15.98:0/1851520234 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562c2bfb6390 0x562c2c00e320 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.619+0000 7f5a1494b000 1 -- 172.21.15.98:0/1851520234 >> 172.21.15.98:0/1851520234 conn(0x562c2bfb1290 msgr2=0x562c2bff8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:34.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.619+0000 7f5a1494b000 1 -- 172.21.15.98:0/1851520234 shutdown_connections 2024-09-19T08:38:34.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.619+0000 7f5a1494b000 1 -- 172.21.15.98:0/1851520234 wait complete. 2024-09-19T08:38:34.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.619+0000 7f5a1494b000 20 client.4605 trim_cache size 0 max 16384 2024-09-19T08:38:34.629 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:38:34.629 INFO:teuthology.orchestra.run.smithi098.stdout: "cluster_fsid": "552bf0ee-1f4f-4644-ba7c-d0931e55a451", 2024-09-19T08:38:34.629 INFO:teuthology.orchestra.run.smithi098.stdout: "whoami": 0, 2024-09-19T08:38:34.629 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 4261, 2024-09-19T08:38:34.629 INFO:teuthology.orchestra.run.smithi098.stdout: "want_state": "up:active", 2024-09-19T08:38:34.629 INFO:teuthology.orchestra.run.smithi098.stdout: "state": "up:active", 2024-09-19T08:38:34.630 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "cephfs", 2024-09-19T08:38:34.630 INFO:teuthology.orchestra.run.smithi098.stdout: "rank_uptime": 15.213014361999999, 2024-09-19T08:38:34.630 INFO:teuthology.orchestra.run.smithi098.stdout: "mdsmap_epoch": 7, 2024-09-19T08:38:34.630 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch": 21, 2024-09-19T08:38:34.630 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch_barrier": 21, 2024-09-19T08:38:34.630 INFO:teuthology.orchestra.run.smithi098.stdout: "uptime": 21.286123171 2024-09-19T08:38:34.630 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:38:34.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.619+0000 7f5a1188f640 1 -- 172.21.15.98:0/169045273 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f59f80678f0 msgr2=0x7f59f8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:34.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.619+0000 7f5a1188f640 1 --2- 172.21.15.98:0/169045273 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f59f80678f0 0x7f59f8069db0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00b9f0 tx=0x7f59fc008040 comp rx=0 tx=0).stop 2024-09-19T08:38:34.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.619+0000 7f5a1188f640 1 -- 172.21.15.98:0/169045273 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a0c15c550 msgr2=0x7f5a0c1fb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:34.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.619+0000 7f5a1188f640 1 --2- 172.21.15.98:0/169045273 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a0c15c550 0x7f5a0c1fb8c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5a000b0040 tx=0x7f5a0001ada0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.619+0000 7f5a1188f640 1 -- 172.21.15.98:0/169045273 shutdown_connections 2024-09-19T08:38:34.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.619+0000 7f5a1188f640 1 --2- 172.21.15.98:0/169045273 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f59f80678f0 0x7f59f8069db0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.619+0000 7f5a1188f640 1 --2- 172.21.15.98:0/169045273 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a0c0b4920 0x7f5a0c153620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.619+0000 7f5a1188f640 1 --2- 172.21.15.98:0/169045273 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a0c15c550 0x7f5a0c1fb8c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.619+0000 7f5a1188f640 1 --2- 172.21.15.98:0/169045273 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a0c0cf900 0x7f5a0c1496d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.619+0000 7f5a1188f640 1 -- 172.21.15.98:0/169045273 >> 172.21.15.98:0/169045273 conn(0x7f5a0c0b2010 msgr2=0x7f5a0c156fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:34.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.619+0000 7f5a1188f640 1 -- 172.21.15.98:0/169045273 shutdown_connections 2024-09-19T08:38:34.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.619+0000 7f5a1188f640 1 -- 172.21.15.98:0/169045273 wait complete. 2024-09-19T08:38:34.638 INFO:teuthology.run_tasks:Running task sequential... 2024-09-19T08:38:34.649 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-19T08:38:34.649 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-19T08:38:34.649 INFO:teuthology.task.print:Enabling mgr modules 2024-09-19T08:38:34.650 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-19T08:38:34.687 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-19T08:38:34.688 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-19T08:38:34.688 INFO:tasks.ceph_fuse:client.0 config is {} 2024-09-19T08:38:34.688 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T08:38:34.688 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T08:38:34.688 INFO:tasks.ceph_fuse:client.2 config is {} 2024-09-19T08:38:34.688 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T08:38:34.688 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-19T08:38:34.689 INFO:tasks.ceph_fuse:client.1 config is {} 2024-09-19T08:38:34.689 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T08:38:34.689 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-19T08:38:34.689 INFO:tasks.ceph_fuse:client.3 config is {} 2024-09-19T08:38:34.689 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T08:38:34.689 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-19T08:38:34.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:34.689 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-09-19T08:38:34.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:34.697 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link delete ceph-brx 2024-09-19T08:38:34.757 INFO:teuthology.orchestra.run.smithi139.stderr:Cannot find device "ceph-brx" 2024-09-19T08:38:34.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:38:34.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:34.759 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2024-09-19T08:38:34.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:34.766 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link delete ceph-brx 2024-09-19T08:38:34.824 INFO:teuthology.orchestra.run.smithi098.stderr:Cannot find device "ceph-brx" 2024-09-19T08:38:34.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:38:34.825 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-19T08:38:34.826 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-19T08:38:34.826 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T08:38:34.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.983+0000 7f2e2e6ab640 1 Processor -- start 2024-09-19T08:38:34.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.983+0000 7f2e2e6ab640 1 -- start start 2024-09-19T08:38:34.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.983+0000 7f2e2e6ab640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e280733b0 0x7f2e280cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:34.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.983+0000 7f2e2e6ab640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e2806d310 con 0x7f2e2815c550 2024-09-19T08:38:34.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.983+0000 7f2e2e6ab640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e2806d5f0 con 0x7f2e2806c3a0 2024-09-19T08:38:34.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.983+0000 7f2e2e6ab640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2e2806d8d0 con 0x7f2e280733b0 2024-09-19T08:38:34.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.983+0000 7f2e27fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f2e2815c550 0x7f2e2806bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:60694/0 (socket says 172.21.15.98:60694) 2024-09-19T08:38:34.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.983+0000 7f2e27fff640 1 -- 172.21.15.98:0/600953795 learned_addr learned my addr 172.21.15.98:0/600953795 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:34.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.983+0000 7f2e2cc48640 1 --2- 172.21.15.98:0/600953795 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e280733b0 0x7f2e280cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:34.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.983+0000 7f2e26ffd640 1 -- 172.21.15.98:0/600953795 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1799285248 0 0) 0x7f2e2806d310 con 0x7f2e2815c550 2024-09-19T08:38:34.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.983+0000 7f2e26ffd640 1 -- 172.21.15.98:0/600953795 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e10003530 con 0x7f2e2815c550 2024-09-19T08:38:34.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.983+0000 7f2e26ffd640 1 -- 172.21.15.98:0/600953795 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2918803307 0 0) 0x7f2e10003530 con 0x7f2e2815c550 2024-09-19T08:38:34.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.983+0000 7f2e26ffd640 1 -- 172.21.15.98:0/600953795 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e280733b0 msgr2=0x7f2e280cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:34.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.983+0000 7f2e26ffd640 1 --2- 172.21.15.98:0/600953795 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e280733b0 0x7f2e280cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.983+0000 7f2e26ffd640 1 -- 172.21.15.98:0/600953795 >> v1:172.21.15.139:6789/0 conn(0x7f2e2806c3a0 legacy=0x7f2e280cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:34.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.983+0000 7f2e26ffd640 1 -- 172.21.15.98:0/600953795 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e2806dbb0 con 0x7f2e2815c550 2024-09-19T08:38:34.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.983+0000 7f2e26ffd640 1 -- 172.21.15.98:0/600953795 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f2e1c001680 con 0x7f2e2815c550 2024-09-19T08:38:34.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.983+0000 7f2e26ffd640 1 -- 172.21.15.98:0/600953795 >> v1:172.21.15.98:6789/0 conn(0x7f2e2815c550 legacy=0x7f2e2806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:34.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.983+0000 7f2e26ffd640 1 --2- 172.21.15.98:0/600953795 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e100039f0 0x7f2e10003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:34.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.983+0000 7f2e26ffd640 1 --2- 172.21.15.98:0/600953795 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e280733b0 0x7f2e10004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:34.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.983+0000 7f2e26ffd640 1 --2- 172.21.15.98:0/600953795 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e10004d50 0x7f2e100051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:34.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.983+0000 7f2e26ffd640 1 -- 172.21.15.98:0/600953795 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2806d8d0 con 0x7f2e100039f0 2024-09-19T08:38:34.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.983+0000 7f2e26ffd640 1 -- 172.21.15.98:0/600953795 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1000ba80 con 0x7f2e10004d50 2024-09-19T08:38:34.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.983+0000 7f2e26ffd640 1 -- 172.21.15.98:0/600953795 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2e1000bc80 con 0x7f2e280733b0 2024-09-19T08:38:34.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.983+0000 7f2e27fff640 1 --2- 172.21.15.98:0/600953795 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e100039f0 0x7f2e10003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:34.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.983+0000 7f2e2cc48640 1 --2- 172.21.15.98:0/600953795 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e10004d50 0x7f2e100051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:34.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.983+0000 7f2e277fe640 1 --2- 172.21.15.98:0/600953795 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e280733b0 0x7f2e10004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:34.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.983+0000 7f2e27fff640 1 -- 172.21.15.98:0/600953795 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e280733b0 msgr2=0x7f2e10004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:34.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.983+0000 7f2e27fff640 1 --2- 172.21.15.98:0/600953795 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e280733b0 0x7f2e10004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.983+0000 7f2e27fff640 1 -- 172.21.15.98:0/600953795 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e10004d50 msgr2=0x7f2e100051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:34.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.983+0000 7f2e27fff640 1 --2- 172.21.15.98:0/600953795 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e10004d50 0x7f2e100051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.983+0000 7f2e27fff640 1 -- 172.21.15.98:0/600953795 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e10003530 con 0x7f2e100039f0 2024-09-19T08:38:34.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.983+0000 7f2e277fe640 1 --2- 172.21.15.98:0/600953795 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e280733b0 0x7f2e10004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:34.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.987+0000 7f2e27fff640 1 --2- 172.21.15.98:0/600953795 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e100039f0 0x7f2e10003e10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c00c190 tx=0x7f2e1c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5428e20ebf806243 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:34.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.987+0000 7f2e26ffd640 1 -- 172.21.15.98:0/600953795 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c007d00 con 0x7f2e100039f0 2024-09-19T08:38:34.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.987+0000 7f2e26ffd640 1 -- 172.21.15.98:0/600953795 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e1c0a9020 con 0x7f2e100039f0 2024-09-19T08:38:34.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.987+0000 7f2e26ffd640 1 -- 172.21.15.98:0/600953795 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c005b20 con 0x7f2e100039f0 2024-09-19T08:38:34.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.987+0000 7f2e2e6ab640 1 -- 172.21.15.98:0/600953795 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e100039f0 msgr2=0x7f2e10003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:34.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.987+0000 7f2e2e6ab640 1 --2- 172.21.15.98:0/600953795 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e100039f0 0x7f2e10003e10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c00c190 tx=0x7f2e1c01cc40 comp rx=0 tx=0).stop 2024-09-19T08:38:34.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.987+0000 7f2e2cc48640 1 -- 172.21.15.98:0/600953795 reap_dead start 2024-09-19T08:38:34.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.987+0000 7f2e2e6ab640 1 -- 172.21.15.98:0/600953795 shutdown_connections 2024-09-19T08:38:34.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.987+0000 7f2e2e6ab640 1 -- 172.21.15.98:0/600953795 >> 172.21.15.98:0/600953795 conn(0x7f2e280b2010 msgr2=0x7f2e280b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:34.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.987+0000 7f2e2e6ab640 1 -- 172.21.15.98:0/600953795 shutdown_connections 2024-09-19T08:38:34.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.987+0000 7f2e2e6ab640 1 -- 172.21.15.98:0/600953795 wait complete. 2024-09-19T08:38:34.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.987+0000 7f2e2e6ab640 1 Processor -- start 2024-09-19T08:38:34.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.987+0000 7f2e2e6ab640 1 -- start start 2024-09-19T08:38:34.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.987+0000 7f2e2e6ab640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e100039f0 0x7f2e281538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:34.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.987+0000 7f2e2e6ab640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e28160b60 0x7f2e28155950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:34.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.987+0000 7f2e2e6ab640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e28160fe0 0x7f2e28155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:34.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.987+0000 7f2e2e6ab640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2e28167c40 con 0x7f2e28160fe0 2024-09-19T08:38:34.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.987+0000 7f2e2e6ab640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2815cb40 con 0x7f2e28160b60 2024-09-19T08:38:34.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.987+0000 7f2e2e6ab640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2e28167dc0 con 0x7f2e100039f0 2024-09-19T08:38:34.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.987+0000 7f2e2cc48640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e28160fe0 0x7f2e28155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:34.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.987+0000 7f2e2cc48640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e28160fe0 0x7f2e28155e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38276/0 (socket says 172.21.15.98:38276) 2024-09-19T08:38:34.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.987+0000 7f2e2cc48640 1 -- 172.21.15.98:0/824831944 learned_addr learned my addr 172.21.15.98:0/824831944 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:34.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.991+0000 7f2e277fe640 1 --2- 172.21.15.98:0/824831944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e28160b60 0x7f2e28155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:34.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.991+0000 7f2e27fff640 1 --2- 172.21.15.98:0/824831944 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e100039f0 0x7f2e281538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:34.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.991+0000 7f2e2cc48640 1 -- 172.21.15.98:0/824831944 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e100039f0 msgr2=0x7f2e281538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:34.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.991+0000 7f2e2cc48640 1 --2- 172.21.15.98:0/824831944 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e100039f0 0x7f2e281538f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.991+0000 7f2e2cc48640 1 -- 172.21.15.98:0/824831944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e28160b60 msgr2=0x7f2e28155950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:34.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.991+0000 7f2e2cc48640 1 --2- 172.21.15.98:0/824831944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e28160b60 0x7f2e28155950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:34.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.991+0000 7f2e2cc48640 1 -- 172.21.15.98:0/824831944 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e281645c0 con 0x7f2e28160fe0 2024-09-19T08:38:34.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.991+0000 7f2e277fe640 1 --2- 172.21.15.98:0/824831944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e28160b60 0x7f2e28155950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:34.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.991+0000 7f2e2cc48640 1 --2- 172.21.15.98:0/824831944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e28160fe0 0x7f2e28155e90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2e280d12d0 tx=0x7f2e200115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:34.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.991+0000 7f2e24ff9640 1 -- 172.21.15.98:0/824831944 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e20011e10 con 0x7f2e28160fe0 2024-09-19T08:38:34.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.991+0000 7f2e24ff9640 1 -- 172.21.15.98:0/824831944 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e2001b070 con 0x7f2e28160fe0 2024-09-19T08:38:34.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.991+0000 7f2e2e6ab640 1 -- 172.21.15.98:0/824831944 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e281648a0 con 0x7f2e28160fe0 2024-09-19T08:38:34.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.991+0000 7f2e24ff9640 1 -- 172.21.15.98:0/824831944 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e200039a0 con 0x7f2e28160fe0 2024-09-19T08:38:34.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.991+0000 7f2e2e6ab640 1 -- 172.21.15.98:0/824831944 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e28164e60 con 0x7f2e28160fe0 2024-09-19T08:38:34.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.991+0000 7f2e2e6ab640 1 -- 172.21.15.98:0/824831944 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e280cf3c0 con 0x7f2e28160fe0 2024-09-19T08:38:34.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.991+0000 7f2e24ff9640 1 -- 172.21.15.98:0/824831944 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2e20003b40 con 0x7f2e28160fe0 2024-09-19T08:38:34.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.991+0000 7f2e24ff9640 1 --2- 172.21.15.98:0/824831944 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2e140678f0 0x7f2e14069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:34.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.991+0000 7f2e24ff9640 1 -- 172.21.15.98:0/824831944 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2e20086230 con 0x7f2e28160fe0 2024-09-19T08:38:34.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.991+0000 7f2e27fff640 1 --2- 172.21.15.98:0/824831944 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2e140678f0 0x7f2e14069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:34.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.991+0000 7f2e27fff640 1 --2- 172.21.15.98:0/824831944 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2e140678f0 0x7f2e14069db0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c0aa000 tx=0x7f2e28073860 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:35.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:34.995+0000 7f2e24ff9640 1 -- 172.21.15.98:0/824831944 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2e20056d70 con 0x7f2e28160fe0 2024-09-19T08:38:35.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:35.179+0000 7f2e2e6ab640 1 -- 172.21.15.98:0/824831944 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2e280757a0 con 0x7f2e28160fe0 2024-09-19T08:38:35.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:35.179+0000 7f2e24ff9640 1 -- 172.21.15.98:0/824831944 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f2e20056670 con 0x7f2e28160fe0 2024-09-19T08:38:35.185 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T08:38:35.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:35.183+0000 7f2e2e6ab640 1 -- 172.21.15.98:0/824831944 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2e140678f0 msgr2=0x7f2e14069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:35.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:35.183+0000 7f2e2e6ab640 1 --2- 172.21.15.98:0/824831944 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2e140678f0 0x7f2e14069db0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c0aa000 tx=0x7f2e28073860 comp rx=0 tx=0).stop 2024-09-19T08:38:35.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:35.183+0000 7f2e2e6ab640 1 -- 172.21.15.98:0/824831944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e28160fe0 msgr2=0x7f2e28155e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:35.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:35.183+0000 7f2e2e6ab640 1 --2- 172.21.15.98:0/824831944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e28160fe0 0x7f2e28155e90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2e280d12d0 tx=0x7f2e200115a0 comp rx=0 tx=0).stop 2024-09-19T08:38:35.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:35.183+0000 7f2e2e6ab640 1 -- 172.21.15.98:0/824831944 shutdown_connections 2024-09-19T08:38:35.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:35.183+0000 7f2e2e6ab640 1 --2- 172.21.15.98:0/824831944 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2e140678f0 0x7f2e14069db0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:35.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:35.183+0000 7f2e2e6ab640 1 --2- 172.21.15.98:0/824831944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e28160fe0 0x7f2e28155e90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:35.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:35.183+0000 7f2e2e6ab640 1 --2- 172.21.15.98:0/824831944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e28160b60 0x7f2e28155950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:35.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:35.183+0000 7f2e2e6ab640 1 --2- 172.21.15.98:0/824831944 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e100039f0 0x7f2e281538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:35.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:35.183+0000 7f2e2e6ab640 1 -- 172.21.15.98:0/824831944 >> 172.21.15.98:0/824831944 conn(0x7f2e280b2010 msgr2=0x7f2e28156cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:35.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:35.183+0000 7f2e2e6ab640 1 -- 172.21.15.98:0/824831944 shutdown_connections 2024-09-19T08:38:35.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:35.183+0000 7f2e2e6ab640 1 -- 172.21.15.98:0/824831944 wait complete. 2024-09-19T08:38:35.203 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T08:38:35.203 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T08:38:35.203 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi098.front.sepia.ceph.com 2024-09-19T08:38:35.203 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T08:38:35.203 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T08:38:35.203 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T08:38:35.203 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T08:38:35.203 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T08:38:35.203 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:38:35.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:35.204 DEBUG:teuthology.orchestra.run.smithi098:> ip addr 2024-09-19T08:38:35.209 INFO:teuthology.orchestra.run.smithi098.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T08:38:35.209 INFO:teuthology.orchestra.run.smithi098.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T08:38:35.210 INFO:teuthology.orchestra.run.smithi098.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T08:38:35.210 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:35.210 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 ::1/128 scope host 2024-09-19T08:38:35.210 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:35.210 INFO:teuthology.orchestra.run.smithi098.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:38:35.210 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b8 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:38:35.210 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f0 2024-09-19T08:38:35.210 INFO:teuthology.orchestra.run.smithi098.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:38:35.210 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b9 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:38:35.210 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f1 2024-09-19T08:38:35.210 INFO:teuthology.orchestra.run.smithi098.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:38:35.210 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6a brd ff:ff:ff:ff:ff:ff 2024-09-19T08:38:35.210 INFO:teuthology.orchestra.run.smithi098.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T08:38:35.210 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6b brd ff:ff:ff:ff:ff:ff 2024-09-19T08:38:35.211 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.21.15.98/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T08:38:35.211 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft 42493sec preferred_lft 42493sec 2024-09-19T08:38:35.211 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::ec4:7aff:fe8f:d26b/64 scope link 2024-09-19T08:38:35.211 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:35.211 INFO:teuthology.orchestra.run.smithi098.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T08:38:35.211 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 02:42:98:dd:32:2d brd ff:ff:ff:ff:ff:ff 2024-09-19T08:38:35.211 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T08:38:35.211 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:35.211 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-19T08:38:35.212 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:38:35.212 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-09-19T08:38:35.212 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link add name ceph-brx type bridge 2024-09-19T08:38:35.212 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip addr flush dev ceph-brx 2024-09-19T08:38:35.212 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set ceph-brx up 2024-09-19T08:38:35.212 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-19T08:38:35.212 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-09-19T08:38:35.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:38:35.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:38:35.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:35.332 DEBUG:teuthology.orchestra.run.smithi098:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T08:38:35.381 INFO:teuthology.orchestra.run.smithi098.stdout:1 2024-09-19T08:38:35.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:35.383 DEBUG:teuthology.orchestra.run.smithi098:> route 2024-09-19T08:38:35.462 INFO:teuthology.orchestra.run.smithi098.stdout:Kernel IP routing table 2024-09-19T08:38:35.463 INFO:teuthology.orchestra.run.smithi098.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T08:38:35.463 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T08:38:35.463 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T08:38:35.463 INFO:teuthology.orchestra.run.smithi098.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T08:38:35.463 INFO:teuthology.orchestra.run.smithi098.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T08:38:35.463 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T08:38:35.463 INFO:teuthology.orchestra.run.smithi098.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:38:35.463 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:38:35.463 INFO:teuthology.orchestra.run.smithi098.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:38:35.463 INFO:teuthology.orchestra.run.smithi098.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T08:38:35.464 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:38:35.464 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-09-19T08:38:35.464 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T08:38:35.464 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T08:38:35.464 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T08:38:35.464 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-09-19T08:38:35.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:38:35.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:38:35.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:35.582 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2024-09-19T08:38:35.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:35.631 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list-id 2024-09-19T08:38:35.678 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:38:35.678 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-09-19T08:38:35.678 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:38:35.679 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-19T08:38:35.679 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-09-19T08:38:35.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:38:35.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:38:35.795 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-19T08:38:35.795 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:38:35.796 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-09-19T08:38:35.796 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-19T08:38:35.796 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-19T08:38:35.796 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-19T08:38:35.796 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-19T08:38:35.796 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-19T08:38:35.796 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-09-19T08:38:35.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:38:35.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:38:35.912 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:38:35.912 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-09-19T08:38:35.912 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set brx.0 up 2024-09-19T08:38:35.912 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set dev brx.0 master ceph-brx 2024-09-19T08:38:35.912 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-09-19T08:38:35.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:38:36.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:38:36.014 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-19T08:38:36.015 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:38:36.015 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T08:38:36.062 INFO:teuthology.orchestra.run.smithi098.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:38:36.063 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:38:36.063 DEBUG:teuthology.orchestra.run.smithi098:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T08:38:36.109 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:38:36.166 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:38:36.215 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:38:36.215 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:38:36.215 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:38:36.215 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:38:36.215 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:38:36.215 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:38:36.215 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:38:36.215 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:38:36.215 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:38:36.215 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:38:36.215 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:38:36.215 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:38:36.216 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:38:36.216 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:38:36.216 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:38:36.216 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:38:36.216 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:38:36.216 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:38:36.216 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:38:36.216 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:38:36.216 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:38:36.216 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:38:36.216 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:38:36.216 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:38:36.216 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:38:36.217 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:38:36.217 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:38:36.217 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:38:36.217 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:38:36.217 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:38:36.217 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:38:36.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:36.218 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:38:36.266 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T08:38:36.267 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-19T08:38:36.315 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:38:36.330 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:38:36.378 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:38:36.378 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:38:36.378 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:38:36.379 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:38:36.379 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:38:36.379 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:38:36.379 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:38:36.379 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:38:36.379 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:38:36.379 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:38:36.379 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:38:36.379 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:38:36.379 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:38:36.379 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:38:36.379 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:38:36.379 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:38:36.379 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:38:36.380 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:38:36.380 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:38:36.380 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:38:36.380 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:38:36.380 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:38:36.380 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:38:36.380 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:38:36.380 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:38:36.380 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:38:36.380 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:38:36.380 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:38:36.380 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:38:36.380 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:38:36.381 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:38:36.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:36.381 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:38:36.399 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:2024-09-19T08:38:36.395+0000 7f7d8c6b0580 -1 init, newargv = 0x558b33e45680 newargc=15 2024-09-19T08:38:36.399 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:ceph-fuse[35966]: starting ceph client 2024-09-19T08:38:36.406 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:ceph-fuse[35966]: starting fuse 2024-09-19T08:38:37.504 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:38:37.520 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:38:37.567 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:38:37.568 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:38:37.568 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:38:37.568 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:38:37.568 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:38:37.568 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:38:37.568 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:38:37.568 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:38:37.568 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:38:37.568 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:38:37.568 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:38:37.569 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:38:37.569 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:38:37.569 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:38:37.569 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:38:37.569 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:38:37.569 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:38:37.569 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:38:37.569 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:38:37.569 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:38:37.569 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:38:37.569 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:38:37.569 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:38:37.570 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:38:37.570 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:38:37.570 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:38:37.570 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:38:37.570 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:38:37.570 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:38:37.570 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:38:37.570 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:38:37.570 INFO:teuthology.orchestra.run.smithi098.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-19T08:38:37.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:37.571 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:38:37.618 INFO:teuthology.orchestra.run.smithi098.stdout:45 2024-09-19T08:38:37.618 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-19T08:38:37.618 DEBUG:teuthology.orchestra.run.smithi098:> sudo stdin-killer -- python3 -c ' 2024-09-19T08:38:37.618 DEBUG:teuthology.orchestra.run.smithi098:> import glob 2024-09-19T08:38:37.619 DEBUG:teuthology.orchestra.run.smithi098:> import re 2024-09-19T08:38:37.619 DEBUG:teuthology.orchestra.run.smithi098:> import os 2024-09-19T08:38:37.619 DEBUG:teuthology.orchestra.run.smithi098:> import subprocess 2024-09-19T08:38:37.619 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:38:37.619 DEBUG:teuthology.orchestra.run.smithi098:> def _find_admin_socket(client_name): 2024-09-19T08:38:37.619 DEBUG:teuthology.orchestra.run.smithi098:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T08:38:37.619 DEBUG:teuthology.orchestra.run.smithi098:> files = glob.glob(asok_path) 2024-09-19T08:38:37.619 DEBUG:teuthology.orchestra.run.smithi098:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T08:38:37.619 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:38:37.619 DEBUG:teuthology.orchestra.run.smithi098:> # Given a non-glob path, it better be there 2024-09-19T08:38:37.619 DEBUG:teuthology.orchestra.run.smithi098:> if "*" not in asok_path: 2024-09-19T08:38:37.619 DEBUG:teuthology.orchestra.run.smithi098:> assert(len(files) == 1) 2024-09-19T08:38:37.619 DEBUG:teuthology.orchestra.run.smithi098:> return files[0] 2024-09-19T08:38:37.619 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:38:37.620 DEBUG:teuthology.orchestra.run.smithi098:> for f in files: 2024-09-19T08:38:37.620 DEBUG:teuthology.orchestra.run.smithi098:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T08:38:37.620 DEBUG:teuthology.orchestra.run.smithi098:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T08:38:37.620 DEBUG:teuthology.orchestra.run.smithi098:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T08:38:37.620 DEBUG:teuthology.orchestra.run.smithi098:> contents = proc_f.read() 2024-09-19T08:38:37.620 DEBUG:teuthology.orchestra.run.smithi098:> if mountpoint in contents: 2024-09-19T08:38:37.620 DEBUG:teuthology.orchestra.run.smithi098:> return f 2024-09-19T08:38:37.620 DEBUG:teuthology.orchestra.run.smithi098:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T08:38:37.620 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:38:37.620 DEBUG:teuthology.orchestra.run.smithi098:> print(_find_admin_socket("client.0")) 2024-09-19T08:38:37.620 DEBUG:teuthology.orchestra.run.smithi098:> ' 2024-09-19T08:38:37.743 INFO:teuthology.orchestra.run.smithi098.stdout:/var/run/ceph/ceph-client.0.35966.asok 2024-09-19T08:38:37.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:38:37.755 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.35966.asok 2024-09-19T08:38:37.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:37.756 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.35966.asok status 2024-09-19T08:38:37.907 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:38:37.907 INFO:teuthology.orchestra.run.smithi098.stdout: "metadata": { 2024-09-19T08:38:37.907 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T08:38:37.907 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T08:38:37.907 INFO:teuthology.orchestra.run.smithi098.stdout: "entity_id": "0", 2024-09-19T08:38:37.907 INFO:teuthology.orchestra.run.smithi098.stdout: "hostname": "smithi098", 2024-09-19T08:38:37.907 INFO:teuthology.orchestra.run.smithi098.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T08:38:37.907 INFO:teuthology.orchestra.run.smithi098.stdout: "pid": "35966", 2024-09-19T08:38:37.907 INFO:teuthology.orchestra.run.smithi098.stdout: "root": "/" 2024-09-19T08:38:37.907 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:38:37.907 INFO:teuthology.orchestra.run.smithi098.stdout: "dentry_count": 0, 2024-09-19T08:38:37.908 INFO:teuthology.orchestra.run.smithi098.stdout: "dentry_pinned_count": 0, 2024-09-19T08:38:37.908 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 4617, 2024-09-19T08:38:37.908 INFO:teuthology.orchestra.run.smithi098.stdout: "inst": { 2024-09-19T08:38:37.908 INFO:teuthology.orchestra.run.smithi098.stdout: "name": { 2024-09-19T08:38:37.908 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "client", 2024-09-19T08:38:37.908 INFO:teuthology.orchestra.run.smithi098.stdout: "num": 4617 2024-09-19T08:38:37.908 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:38:37.908 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": { 2024-09-19T08:38:37.908 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v1", 2024-09-19T08:38:37.908 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:38:37.908 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 1622010032 2024-09-19T08:38:37.908 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:38:37.908 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:38:37.909 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": { 2024-09-19T08:38:37.909 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v1", 2024-09-19T08:38:37.909 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:38:37.909 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 1622010032 2024-09-19T08:38:37.909 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:38:37.909 INFO:teuthology.orchestra.run.smithi098.stdout: "inst_str": "client.4617 192.168.0.1:0/1622010032", 2024-09-19T08:38:37.909 INFO:teuthology.orchestra.run.smithi098.stdout: "addr_str": "192.168.0.1:0/1622010032", 2024-09-19T08:38:37.909 INFO:teuthology.orchestra.run.smithi098.stdout: "inode_count": 1, 2024-09-19T08:38:37.909 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_epoch": 7, 2024-09-19T08:38:37.909 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_epoch": 21, 2024-09-19T08:38:37.909 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_epoch_barrier": 0, 2024-09-19T08:38:37.909 INFO:teuthology.orchestra.run.smithi098.stdout: "blocklisted": false, 2024-09-19T08:38:37.909 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "cephfs" 2024-09-19T08:38:37.909 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:38:37.921 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-19T08:38:37.921 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T08:38:38.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.043+0000 7f23db20e640 1 Processor -- start 2024-09-19T08:38:38.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.043+0000 7f23db20e640 1 -- start start 2024-09-19T08:38:38.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.043+0000 7f23db20e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23d4160740 0x7f23d4163170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:38.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.043+0000 7f23db20e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f23d406c3a0 0x7f23d40ceb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:38.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.043+0000 7f23db20e640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23d4163bf0 con 0x7f23d41636b0 2024-09-19T08:38:38.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.043+0000 7f23db20e640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f23d4163ed0 con 0x7f23d406c3a0 2024-09-19T08:38:38.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.043+0000 7f23db20e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f23d40ce320 con 0x7f23d4160740 2024-09-19T08:38:38.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.043+0000 7f23d97ab640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f23d406c3a0 0x7f23d40ceb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:38.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.043+0000 7f23d3fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f23d41636b0 0x7f23d406bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:46380/0 (socket says 172.21.15.98:46380) 2024-09-19T08:38:38.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.043+0000 7f23d3fff640 1 -- 172.21.15.98:0/1562075394 learned_addr learned my addr 172.21.15.98:0/1562075394 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:38.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.043+0000 7f23d8faa640 1 --2- 172.21.15.98:0/1562075394 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23d4160740 0x7f23d4163170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:38.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.043+0000 7f23d97ab640 1 -- 172.21.15.98:0/1562075394 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23d4160740 msgr2=0x7f23d4163170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:38.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.043+0000 7f23d97ab640 1 --2- 172.21.15.98:0/1562075394 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23d4160740 0x7f23d4163170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:38.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.043+0000 7f23d97ab640 1 -- 172.21.15.98:0/1562075394 >> v1:172.21.15.98:6789/0 conn(0x7f23d41636b0 legacy=0x7f23d406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:38.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.043+0000 7f23d97ab640 1 -- 172.21.15.98:0/1562075394 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d40ce600 con 0x7f23d406c3a0 2024-09-19T08:38:38.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.047+0000 7f23d97ab640 1 --2- 172.21.15.98:0/1562075394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23d406c3a0 0x7f23d40ceb20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f23c40097c0 tx=0x7f23c401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4beed7bf2b9e06f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:38.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.047+0000 7f23d37fe640 1 -- 172.21.15.98:0/1562075394 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c40b0070 con 0x7f23d406c3a0 2024-09-19T08:38:38.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.047+0000 7f23d37fe640 1 -- 172.21.15.98:0/1562075394 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23c40ab600 con 0x7f23d406c3a0 2024-09-19T08:38:38.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.047+0000 7f23d37fe640 1 -- 172.21.15.98:0/1562075394 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c40ab9d0 con 0x7f23d406c3a0 2024-09-19T08:38:38.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.047+0000 7f23db20e640 1 -- 172.21.15.98:0/1562075394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23d406c3a0 msgr2=0x7f23d40ceb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:38.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.047+0000 7f23db20e640 1 --2- 172.21.15.98:0/1562075394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23d406c3a0 0x7f23d40ceb20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f23c40097c0 tx=0x7f23c401c4a0 comp rx=0 tx=0).stop 2024-09-19T08:38:38.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.047+0000 7f23db20e640 1 -- 172.21.15.98:0/1562075394 shutdown_connections 2024-09-19T08:38:38.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.047+0000 7f23db20e640 1 --2- 172.21.15.98:0/1562075394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23d406c3a0 0x7f23d40ceb20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:38.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.047+0000 7f23db20e640 1 --2- 172.21.15.98:0/1562075394 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23d4160740 0x7f23d4163170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:38.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.047+0000 7f23db20e640 1 -- 172.21.15.98:0/1562075394 >> 172.21.15.98:0/1562075394 conn(0x7f23d40b2010 msgr2=0x7f23d40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:38.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.047+0000 7f23db20e640 1 -- 172.21.15.98:0/1562075394 shutdown_connections 2024-09-19T08:38:38.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.047+0000 7f23db20e640 1 -- 172.21.15.98:0/1562075394 wait complete. 2024-09-19T08:38:38.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.047+0000 7f23db20e640 1 Processor -- start 2024-09-19T08:38:38.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.047+0000 7f23db20e640 1 -- start start 2024-09-19T08:38:38.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.047+0000 7f23db20e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23d406c3a0 0x7f23d41fc280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:38.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.047+0000 7f23db20e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23d4160740 0x7f23d414ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:38.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.047+0000 7f23db20e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23d41636b0 0x7f23d41fec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:38.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.047+0000 7f23db20e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f23d40cfc40 con 0x7f23d406c3a0 2024-09-19T08:38:38.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.047+0000 7f23db20e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f23d40cfac0 con 0x7f23d41636b0 2024-09-19T08:38:38.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.047+0000 7f23db20e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f23d40cfdc0 con 0x7f23d4160740 2024-09-19T08:38:38.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.047+0000 7f23d8faa640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23d406c3a0 0x7f23d41fc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:38.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.047+0000 7f23d8faa640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23d406c3a0 0x7f23d41fc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34442/0 (socket says 172.21.15.98:34442) 2024-09-19T08:38:38.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.047+0000 7f23d8faa640 1 -- 172.21.15.98:0/3634839558 learned_addr learned my addr 172.21.15.98:0/3634839558 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:38.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.047+0000 7f23d3fff640 1 --2- 172.21.15.98:0/3634839558 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23d4160740 0x7f23d414ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:38.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.047+0000 7f23d97ab640 1 --2- 172.21.15.98:0/3634839558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23d41636b0 0x7f23d41fec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:38.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.047+0000 7f23d8faa640 1 -- 172.21.15.98:0/3634839558 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23d4160740 msgr2=0x7f23d414ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:38.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.047+0000 7f23d8faa640 1 --2- 172.21.15.98:0/3634839558 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23d4160740 0x7f23d414ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:38.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.047+0000 7f23d8faa640 1 -- 172.21.15.98:0/3634839558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23d41636b0 msgr2=0x7f23d41fec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:38.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.047+0000 7f23d8faa640 1 --2- 172.21.15.98:0/3634839558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23d41636b0 0x7f23d41fec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:38.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.047+0000 7f23d8faa640 1 -- 172.21.15.98:0/3634839558 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d41ffb30 con 0x7f23d406c3a0 2024-09-19T08:38:38.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.047+0000 7f23d3fff640 1 --2- 172.21.15.98:0/3634839558 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23d4160740 0x7f23d414ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:38:38.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.051+0000 7f23d8faa640 1 --2- 172.21.15.98:0/3634839558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23d406c3a0 0x7f23d41fc280 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f23cc00d5f0 tx=0x7f23cc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:38.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.051+0000 7f23d17fa640 1 -- 172.21.15.98:0/3634839558 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc0125f0 con 0x7f23d406c3a0 2024-09-19T08:38:38.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.051+0000 7f23d17fa640 1 -- 172.21.15.98:0/3634839558 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23cc019070 con 0x7f23d406c3a0 2024-09-19T08:38:38.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.051+0000 7f23d17fa640 1 -- 172.21.15.98:0/3634839558 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc0129b0 con 0x7f23d406c3a0 2024-09-19T08:38:38.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.051+0000 7f23db20e640 1 -- 172.21.15.98:0/3634839558 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23d41ff570 con 0x7f23d406c3a0 2024-09-19T08:38:38.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.051+0000 7f23db20e640 1 -- 172.21.15.98:0/3634839558 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23d41569c0 con 0x7f23d406c3a0 2024-09-19T08:38:38.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.051+0000 7f23db20e640 1 -- 172.21.15.98:0/3634839558 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23d40ceb20 con 0x7f23d406c3a0 2024-09-19T08:38:38.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.051+0000 7f23d17fa640 1 -- 172.21.15.98:0/3634839558 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f23cc012b50 con 0x7f23d406c3a0 2024-09-19T08:38:38.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.051+0000 7f23d17fa640 1 --2- 172.21.15.98:0/3634839558 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f23b40678f0 0x7f23b4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:38.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.051+0000 7f23d17fa640 1 -- 172.21.15.98:0/3634839558 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f23cc086000 con 0x7f23d406c3a0 2024-09-19T08:38:38.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.051+0000 7f23d3fff640 1 --2- 172.21.15.98:0/3634839558 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f23b40678f0 0x7f23b4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:38.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.051+0000 7f23d3fff640 1 --2- 172.21.15.98:0/3634839558 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f23b40678f0 0x7f23b4069db0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f23d415a0e0 tx=0x7f23bc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:38.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.055+0000 7f23d17fa640 1 -- 172.21.15.98:0/3634839558 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f23cc056bd0 con 0x7f23d406c3a0 2024-09-19T08:38:38.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.255+0000 7f23db20e640 1 -- 172.21.15.98:0/3634839558 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f23d40742b0 con 0x7f23d406c3a0 2024-09-19T08:38:38.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.255+0000 7f23d17fa640 1 -- 172.21.15.98:0/3634839558 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f23cc0564d0 con 0x7f23d406c3a0 2024-09-19T08:38:38.260 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T08:38:38.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.259+0000 7f23db20e640 1 -- 172.21.15.98:0/3634839558 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f23b40678f0 msgr2=0x7f23b4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:38.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.259+0000 7f23db20e640 1 --2- 172.21.15.98:0/3634839558 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f23b40678f0 0x7f23b4069db0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f23d415a0e0 tx=0x7f23bc009040 comp rx=0 tx=0).stop 2024-09-19T08:38:38.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.259+0000 7f23db20e640 1 -- 172.21.15.98:0/3634839558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23d406c3a0 msgr2=0x7f23d41fc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:38.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.259+0000 7f23db20e640 1 --2- 172.21.15.98:0/3634839558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23d406c3a0 0x7f23d41fc280 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f23cc00d5f0 tx=0x7f23cc00dab0 comp rx=0 tx=0).stop 2024-09-19T08:38:38.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.259+0000 7f23db20e640 1 -- 172.21.15.98:0/3634839558 shutdown_connections 2024-09-19T08:38:38.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.259+0000 7f23db20e640 1 --2- 172.21.15.98:0/3634839558 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f23b40678f0 0x7f23b4069db0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:38.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.259+0000 7f23db20e640 1 --2- 172.21.15.98:0/3634839558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23d41636b0 0x7f23d41fec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:38.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.259+0000 7f23db20e640 1 --2- 172.21.15.98:0/3634839558 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23d4160740 0x7f23d414ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:38.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.259+0000 7f23db20e640 1 --2- 172.21.15.98:0/3634839558 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23d406c3a0 0x7f23d41fc280 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:38.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.259+0000 7f23db20e640 1 -- 172.21.15.98:0/3634839558 >> 172.21.15.98:0/3634839558 conn(0x7f23d40b2010 msgr2=0x7f23d4160d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:38.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.259+0000 7f23db20e640 1 -- 172.21.15.98:0/3634839558 shutdown_connections 2024-09-19T08:38:38.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38.259+0000 7f23db20e640 1 -- 172.21.15.98:0/3634839558 wait complete. 2024-09-19T08:38:38.279 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T08:38:38.279 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T08:38:38.279 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi098.front.sepia.ceph.com 2024-09-19T08:38:38.279 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-19T08:38:38.279 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-19T08:38:38.279 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T08:38:38.280 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T08:38:38.280 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T08:38:38.280 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-19T08:38:38.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:38.280 DEBUG:teuthology.orchestra.run.smithi098:> ip addr 2024-09-19T08:38:38.286 INFO:teuthology.orchestra.run.smithi098.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T08:38:38.286 INFO:teuthology.orchestra.run.smithi098.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T08:38:38.286 INFO:teuthology.orchestra.run.smithi098.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T08:38:38.286 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:38.286 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 ::1/128 scope host 2024-09-19T08:38:38.286 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:38.286 INFO:teuthology.orchestra.run.smithi098.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:38:38.286 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b8 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:38:38.286 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f0 2024-09-19T08:38:38.287 INFO:teuthology.orchestra.run.smithi098.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:38:38.287 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b9 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:38:38.287 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f1 2024-09-19T08:38:38.287 INFO:teuthology.orchestra.run.smithi098.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:38:38.287 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6a brd ff:ff:ff:ff:ff:ff 2024-09-19T08:38:38.287 INFO:teuthology.orchestra.run.smithi098.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T08:38:38.287 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6b brd ff:ff:ff:ff:ff:ff 2024-09-19T08:38:38.287 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.21.15.98/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T08:38:38.287 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft 42490sec preferred_lft 42490sec 2024-09-19T08:38:38.287 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::ec4:7aff:fe8f:d26b/64 scope link 2024-09-19T08:38:38.287 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:38.287 INFO:teuthology.orchestra.run.smithi098.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T08:38:38.288 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 02:42:98:dd:32:2d brd ff:ff:ff:ff:ff:ff 2024-09-19T08:38:38.288 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T08:38:38.288 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:38.288 INFO:teuthology.orchestra.run.smithi098.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T08:38:38.288 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-19T08:38:38.288 INFO:teuthology.orchestra.run.smithi098.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T08:38:38.288 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:38.288 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-19T08:38:38.288 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:38.288 INFO:teuthology.orchestra.run.smithi098.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:38:38.288 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:38:38.289 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-19T08:38:38.289 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:38.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:38.289 DEBUG:teuthology.orchestra.run.smithi098:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T08:38:38.339 INFO:teuthology.orchestra.run.smithi098.stdout:1 2024-09-19T08:38:38.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:38.340 DEBUG:teuthology.orchestra.run.smithi098:> route 2024-09-19T08:38:38.394 INFO:teuthology.orchestra.run.smithi098.stdout:Kernel IP routing table 2024-09-19T08:38:38.395 INFO:teuthology.orchestra.run.smithi098.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T08:38:38.395 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T08:38:38.395 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T08:38:38.395 INFO:teuthology.orchestra.run.smithi098.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T08:38:38.395 INFO:teuthology.orchestra.run.smithi098.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T08:38:38.395 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T08:38:38.395 INFO:teuthology.orchestra.run.smithi098.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:38:38.395 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:38:38.395 INFO:teuthology.orchestra.run.smithi098.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:38:38.395 INFO:teuthology.orchestra.run.smithi098.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T08:38:38.396 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:38:38.396 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-09-19T08:38:38.396 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T08:38:38.396 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T08:38:38.396 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T08:38:38.396 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-09-19T08:38:38.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:38:38.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:38:38.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:38.521 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2024-09-19T08:38:38.527 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T08:38:38.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:38.528 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list-id 2024-09-19T08:38:38.574 INFO:teuthology.orchestra.run.smithi098.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-19T08:38:38.574 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:38:38.574 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-09-19T08:38:38.574 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:38:38.574 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-19T08:38:38.574 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-09-19T08:38:38.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:38:38.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:38:38.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:38.691 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-19T08:38:38.703 INFO:teuthology.orchestra.run.smithi098.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T08:38:38.703 INFO:teuthology.orchestra.run.smithi098.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T08:38:38.703 INFO:teuthology.orchestra.run.smithi098.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T08:38:38.703 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:38.704 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 ::1/128 scope host 2024-09-19T08:38:38.704 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:38.704 INFO:teuthology.orchestra.run.smithi098.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T08:38:38.704 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether ba:3c:9a:d7:e4:12 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-19T08:38:38.704 INFO:teuthology.orchestra.run.smithi098.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-19T08:38:38.704 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:38.704 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::b83c:9aff:fed7:e412/64 scope link 2024-09-19T08:38:38.704 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:38.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:38.706 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-19T08:38:38.766 INFO:teuthology.orchestra.run.smithi098.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T08:38:38.766 INFO:teuthology.orchestra.run.smithi098.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T08:38:38.766 INFO:teuthology.orchestra.run.smithi098.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T08:38:38.766 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:38.766 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 ::1/128 scope host 2024-09-19T08:38:38.766 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:38.767 INFO:teuthology.orchestra.run.smithi098.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T08:38:38.767 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether ba:3c:9a:d7:e4:12 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-19T08:38:38.767 INFO:teuthology.orchestra.run.smithi098.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-19T08:38:38.767 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:38.767 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::b83c:9aff:fed7:e412/64 scope link 2024-09-19T08:38:38.767 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:38.768 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-19T08:38:38.768 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:38:38.768 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-09-19T08:38:38.769 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-19T08:38:38.769 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-19T08:38:38.769 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-19T08:38:38.769 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-19T08:38:38.769 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-19T08:38:38.769 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-09-19T08:38:38.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:38:38.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:38:38.910 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:38:38.910 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-09-19T08:38:38.910 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set brx.1 up 2024-09-19T08:38:38.910 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set dev brx.1 master ceph-brx 2024-09-19T08:38:38.911 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-09-19T08:38:38.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:38:38.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:38:38.973 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-09-19T08:38:38.974 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:38:38.974 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-19T08:38:39.022 INFO:teuthology.orchestra.run.smithi098.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-19T08:38:39.022 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:38:39.022 DEBUG:teuthology.orchestra.run.smithi098:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-19T08:38:39.069 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:38:39.127 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:38:39.176 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:38:39.176 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:38:39.176 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:38:39.176 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:38:39.176 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:38:39.177 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:38:39.177 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:38:39.177 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:38:39.177 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:38:39.177 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:38:39.177 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:38:39.177 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:38:39.177 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:38:39.177 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:38:39.178 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:38:39.178 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:38:39.178 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:38:39.178 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:38:39.178 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:38:39.178 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:38:39.178 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:38:39.178 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:38:39.178 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:38:39.178 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:38:39.178 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:38:39.179 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:38:39.179 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:38:39.179 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:38:39.179 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:38:39.179 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:38:39.179 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:38:39.179 INFO:teuthology.orchestra.run.smithi098.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-19T08:38:39.179 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:38:39.179 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:38:39.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:39.181 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:38:39.226 INFO:teuthology.orchestra.run.smithi098.stdout:45 2024-09-19T08:38:39.226 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-09-19T08:38:39.226 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-09-19T08:38:39.270 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:38:39.284 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:38:39.331 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:38:39.331 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:38:39.331 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:38:39.331 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:38:39.331 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:38:39.332 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:38:39.332 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:38:39.332 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:38:39.332 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:38:39.332 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:38:39.332 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:38:39.332 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:38:39.332 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:38:39.332 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:38:39.333 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:38:39.333 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:38:39.333 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:38:39.333 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:38:39.333 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:38:39.333 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:38:39.333 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:38:39.333 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:38:39.333 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:38:39.333 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:38:39.333 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:38:39.333 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:38:39.333 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:38:39.334 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:38:39.334 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:38:39.334 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:38:39.334 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:38:39.334 INFO:teuthology.orchestra.run.smithi098.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-19T08:38:39.334 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:38:39.334 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:38:39.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:39.334 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:38:39.347 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi098.stderr:2024-09-19T08:38:39.343+0000 7f5ce04af580 -1 init, newargv = 0x556586006020 newargc=15 2024-09-19T08:38:39.348 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi098.stderr:ceph-fuse[36111]: starting ceph client 2024-09-19T08:38:39.352 INFO:teuthology.orchestra.run.smithi098.stdout:45 2024-09-19T08:38:39.355 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi098.stderr:ceph-fuse[36111]: starting fuse 2024-09-19T08:38:40.453 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:38:40.470 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:38:40.520 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:38:40.520 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:38:40.521 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:38:40.521 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:38:40.521 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:38:40.521 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:38:40.521 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:38:40.521 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:38:40.521 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:38:40.521 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:38:40.521 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:38:40.521 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:38:40.521 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:38:40.521 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:38:40.521 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:38:40.522 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:38:40.522 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:38:40.522 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:38:40.522 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:38:40.522 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:38:40.522 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:38:40.522 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:38:40.522 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:38:40.522 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:38:40.522 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:38:40.522 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:38:40.522 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:38:40.522 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:38:40.522 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:38:40.523 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:38:40.523 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:38:40.523 INFO:teuthology.orchestra.run.smithi098.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-19T08:38:40.523 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:38:40.523 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:38:40.523 INFO:teuthology.orchestra.run.smithi098.stdout:/home/ubuntu/cephtest/mnt.2 2024-09-19T08:38:40.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:40.524 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:38:40.571 INFO:teuthology.orchestra.run.smithi098.stdout:45 2024-09-19T08:38:40.571 INFO:teuthology.orchestra.run.smithi098.stdout:46 2024-09-19T08:38:40.571 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-09-19T08:38:40.571 DEBUG:teuthology.orchestra.run.smithi098:> sudo stdin-killer -- python3 -c ' 2024-09-19T08:38:40.571 DEBUG:teuthology.orchestra.run.smithi098:> import glob 2024-09-19T08:38:40.571 DEBUG:teuthology.orchestra.run.smithi098:> import re 2024-09-19T08:38:40.572 DEBUG:teuthology.orchestra.run.smithi098:> import os 2024-09-19T08:38:40.572 DEBUG:teuthology.orchestra.run.smithi098:> import subprocess 2024-09-19T08:38:40.572 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:38:40.572 DEBUG:teuthology.orchestra.run.smithi098:> def _find_admin_socket(client_name): 2024-09-19T08:38:40.572 DEBUG:teuthology.orchestra.run.smithi098:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-09-19T08:38:40.572 DEBUG:teuthology.orchestra.run.smithi098:> files = glob.glob(asok_path) 2024-09-19T08:38:40.572 DEBUG:teuthology.orchestra.run.smithi098:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-09-19T08:38:40.572 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:38:40.572 DEBUG:teuthology.orchestra.run.smithi098:> # Given a non-glob path, it better be there 2024-09-19T08:38:40.572 DEBUG:teuthology.orchestra.run.smithi098:> if "*" not in asok_path: 2024-09-19T08:38:40.573 DEBUG:teuthology.orchestra.run.smithi098:> assert(len(files) == 1) 2024-09-19T08:38:40.573 DEBUG:teuthology.orchestra.run.smithi098:> return files[0] 2024-09-19T08:38:40.573 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:38:40.573 DEBUG:teuthology.orchestra.run.smithi098:> for f in files: 2024-09-19T08:38:40.573 DEBUG:teuthology.orchestra.run.smithi098:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T08:38:40.573 DEBUG:teuthology.orchestra.run.smithi098:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T08:38:40.573 DEBUG:teuthology.orchestra.run.smithi098:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T08:38:40.573 DEBUG:teuthology.orchestra.run.smithi098:> contents = proc_f.read() 2024-09-19T08:38:40.573 DEBUG:teuthology.orchestra.run.smithi098:> if mountpoint in contents: 2024-09-19T08:38:40.573 DEBUG:teuthology.orchestra.run.smithi098:> return f 2024-09-19T08:38:40.574 DEBUG:teuthology.orchestra.run.smithi098:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T08:38:40.574 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:38:40.574 DEBUG:teuthology.orchestra.run.smithi098:> print(_find_admin_socket("client.2")) 2024-09-19T08:38:40.574 DEBUG:teuthology.orchestra.run.smithi098:> ' 2024-09-19T08:38:40.695 INFO:teuthology.orchestra.run.smithi098.stdout:/var/run/ceph/ceph-client.2.36111.asok 2024-09-19T08:38:40.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:38:40.706 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.36111.asok 2024-09-19T08:38:40.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:40.707 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.36111.asok status 2024-09-19T08:38:40.863 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:38:40.863 INFO:teuthology.orchestra.run.smithi098.stdout: "metadata": { 2024-09-19T08:38:40.863 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T08:38:40.863 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T08:38:40.863 INFO:teuthology.orchestra.run.smithi098.stdout: "entity_id": "2", 2024-09-19T08:38:40.863 INFO:teuthology.orchestra.run.smithi098.stdout: "hostname": "smithi098", 2024-09-19T08:38:40.863 INFO:teuthology.orchestra.run.smithi098.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-09-19T08:38:40.863 INFO:teuthology.orchestra.run.smithi098.stdout: "pid": "36111", 2024-09-19T08:38:40.863 INFO:teuthology.orchestra.run.smithi098.stdout: "root": "/" 2024-09-19T08:38:40.864 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:38:40.864 INFO:teuthology.orchestra.run.smithi098.stdout: "dentry_count": 0, 2024-09-19T08:38:40.864 INFO:teuthology.orchestra.run.smithi098.stdout: "dentry_pinned_count": 0, 2024-09-19T08:38:40.864 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 4635, 2024-09-19T08:38:40.864 INFO:teuthology.orchestra.run.smithi098.stdout: "inst": { 2024-09-19T08:38:40.864 INFO:teuthology.orchestra.run.smithi098.stdout: "name": { 2024-09-19T08:38:40.864 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "client", 2024-09-19T08:38:40.864 INFO:teuthology.orchestra.run.smithi098.stdout: "num": 4635 2024-09-19T08:38:40.864 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:38:40.864 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": { 2024-09-19T08:38:40.864 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v1", 2024-09-19T08:38:40.864 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "192.168.0.2:0", 2024-09-19T08:38:40.864 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 1554631277 2024-09-19T08:38:40.865 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:38:40.865 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:38:40.865 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": { 2024-09-19T08:38:40.865 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v1", 2024-09-19T08:38:40.865 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "192.168.0.2:0", 2024-09-19T08:38:40.865 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 1554631277 2024-09-19T08:38:40.865 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:38:40.865 INFO:teuthology.orchestra.run.smithi098.stdout: "inst_str": "client.4635 192.168.0.2:0/1554631277", 2024-09-19T08:38:40.865 INFO:teuthology.orchestra.run.smithi098.stdout: "addr_str": "192.168.0.2:0/1554631277", 2024-09-19T08:38:40.865 INFO:teuthology.orchestra.run.smithi098.stdout: "inode_count": 1, 2024-09-19T08:38:40.865 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_epoch": 7, 2024-09-19T08:38:40.865 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_epoch": 21, 2024-09-19T08:38:40.865 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_epoch_barrier": 0, 2024-09-19T08:38:40.865 INFO:teuthology.orchestra.run.smithi098.stdout: "blocklisted": false, 2024-09-19T08:38:40.866 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "cephfs" 2024-09-19T08:38:40.866 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:38:40.878 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-19T08:38:40.878 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T08:38:41.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:40.999+0000 7f08b1191640 1 Processor -- start 2024-09-19T08:38:41.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:40.999+0000 7f08b1191640 1 -- start start 2024-09-19T08:38:41.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:40.999+0000 7f08b1191640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f08ac158fd0 0x7f08ac0b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:41.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:40.999+0000 7f08b1191640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08ac0ccc60 con 0x7f08ac06c3a0 2024-09-19T08:38:41.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:40.999+0000 7f08b1191640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08ac0ccf40 con 0x7f08ac0b4360 2024-09-19T08:38:41.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:40.999+0000 7f08b1191640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f08ac1593b0 con 0x7f08ac158fd0 2024-09-19T08:38:41.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:40.999+0000 7f08ab59e640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f08ac06c3a0 0x7f08ac0ce840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:46382/0 (socket says 172.21.15.98:46382) 2024-09-19T08:38:41.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:40.999+0000 7f08ab59e640 1 -- 172.21.15.98:0/963961657 learned_addr learned my addr 172.21.15.98:0/963961657 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:41.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:40.999+0000 7f08aad9d640 1 --2- 172.21.15.98:0/963961657 >> v2:172.21.15.98:3300/0 conn(0x7f08ac158fd0 0x7f08ac0b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:41.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:40.999+0000 7f08aad9d640 1 -- 172.21.15.98:0/963961657 >> v1:172.21.15.139:6789/0 conn(0x7f08ac0b4360 legacy=0x7f08ac06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:41.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:40.999+0000 7f08aad9d640 1 -- 172.21.15.98:0/963961657 >> v1:172.21.15.98:6789/0 conn(0x7f08ac06c3a0 legacy=0x7f08ac0ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:41.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:40.999+0000 7f08aad9d640 1 -- 172.21.15.98:0/963961657 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08ac0cd220 con 0x7f08ac158fd0 2024-09-19T08:38:41.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.003+0000 7f08aad9d640 1 --2- 172.21.15.98:0/963961657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08ac158fd0 0x7f08ac0b3e20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f089c0097c0 tx=0x7f089c01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f87c3eaf6803483 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:41.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.003+0000 7f08a9d9b640 1 -- 172.21.15.98:0/963961657 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089c0b0070 con 0x7f08ac158fd0 2024-09-19T08:38:41.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.003+0000 7f08a9d9b640 1 -- 172.21.15.98:0/963961657 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f089c0ab600 con 0x7f08ac158fd0 2024-09-19T08:38:41.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.003+0000 7f08a9d9b640 1 -- 172.21.15.98:0/963961657 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089c0ab9d0 con 0x7f08ac158fd0 2024-09-19T08:38:41.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.003+0000 7f08b1191640 1 -- 172.21.15.98:0/963961657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08ac158fd0 msgr2=0x7f08ac0b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:41.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.003+0000 7f08b1191640 1 --2- 172.21.15.98:0/963961657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08ac158fd0 0x7f08ac0b3e20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f089c0097c0 tx=0x7f089c01c260 comp rx=0 tx=0).stop 2024-09-19T08:38:41.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.003+0000 7f08b1191640 1 -- 172.21.15.98:0/963961657 shutdown_connections 2024-09-19T08:38:41.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.003+0000 7f08b1191640 1 --2- 172.21.15.98:0/963961657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08ac158fd0 0x7f08ac0b3e20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:41.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.003+0000 7f08b1191640 1 -- 172.21.15.98:0/963961657 >> 172.21.15.98:0/963961657 conn(0x7f08ac0b2010 msgr2=0x7f08ac0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:41.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.003+0000 7f08b1191640 1 -- 172.21.15.98:0/963961657 shutdown_connections 2024-09-19T08:38:41.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.003+0000 7f08b1191640 1 -- 172.21.15.98:0/963961657 wait complete. 2024-09-19T08:38:41.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.003+0000 7f08b1191640 1 Processor -- start 2024-09-19T08:38:41.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.003+0000 7f08b1191640 1 -- start start 2024-09-19T08:38:41.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.003+0000 7f08b1191640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08ac06c3a0 0x7f08ac153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:41.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.003+0000 7f08b1191640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08ac0b4360 0x7f08ac0aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:41.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.003+0000 7f08b1191640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08ac158fd0 0x7f08ac14c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:41.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.003+0000 7f08b1191640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f08ac15c7a0 con 0x7f08ac06c3a0 2024-09-19T08:38:41.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.003+0000 7f08b1191640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f08ac0cd7e0 con 0x7f08ac158fd0 2024-09-19T08:38:41.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.003+0000 7f08b1191640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f08ac15c920 con 0x7f08ac0b4360 2024-09-19T08:38:41.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.003+0000 7f08aad9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08ac06c3a0 0x7f08ac153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:41.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.003+0000 7f08aad9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08ac06c3a0 0x7f08ac153980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34458/0 (socket says 172.21.15.98:34458) 2024-09-19T08:38:41.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.003+0000 7f08aad9d640 1 -- 172.21.15.98:0/1663319750 learned_addr learned my addr 172.21.15.98:0/1663319750 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:41.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.003+0000 7f08aad9d640 1 -- 172.21.15.98:0/1663319750 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08ac0b4360 msgr2=0x7f08ac0aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:41.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.003+0000 7f08aad9d640 1 --2- 172.21.15.98:0/1663319750 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08ac0b4360 0x7f08ac0aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:41.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.003+0000 7f08aad9d640 1 -- 172.21.15.98:0/1663319750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08ac158fd0 msgr2=0x7f08ac14c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:41.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.003+0000 7f08aad9d640 1 --2- 172.21.15.98:0/1663319750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08ac158fd0 0x7f08ac14c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:41.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.003+0000 7f08aad9d640 1 -- 172.21.15.98:0/1663319750 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08ac14d530 con 0x7f08ac06c3a0 2024-09-19T08:38:41.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.003+0000 7f08aad9d640 1 --2- 172.21.15.98:0/1663319750 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08ac06c3a0 0x7f08ac153980 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f089c019e90 tx=0x7f089c01aaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:41.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.003+0000 7f08937fe640 1 -- 172.21.15.98:0/1663319750 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089c0b0050 con 0x7f08ac06c3a0 2024-09-19T08:38:41.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.003+0000 7f08937fe640 1 -- 172.21.15.98:0/1663319750 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f089c0bc070 con 0x7f08ac06c3a0 2024-09-19T08:38:41.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.003+0000 7f08b1191640 1 -- 172.21.15.98:0/1663319750 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08ac15d3e0 con 0x7f08ac06c3a0 2024-09-19T08:38:41.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.007+0000 7f08937fe640 1 -- 172.21.15.98:0/1663319750 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089c0aae40 con 0x7f08ac06c3a0 2024-09-19T08:38:41.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.007+0000 7f08b1191640 1 -- 172.21.15.98:0/1663319750 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08ac15d9a0 con 0x7f08ac06c3a0 2024-09-19T08:38:41.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.007+0000 7f08b1191640 1 -- 172.21.15.98:0/1663319750 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08ac15c3c0 con 0x7f08ac06c3a0 2024-09-19T08:38:41.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.007+0000 7f08937fe640 1 -- 172.21.15.98:0/1663319750 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f089c0ab4c0 con 0x7f08ac06c3a0 2024-09-19T08:38:41.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.007+0000 7f08937fe640 1 --2- 172.21.15.98:0/1663319750 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f08980678c0 0x7f0898069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:41.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.007+0000 7f08937fe640 1 -- 172.21.15.98:0/1663319750 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f089c120a00 con 0x7f08ac06c3a0 2024-09-19T08:38:41.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.007+0000 7f08aa59c640 1 --2- 172.21.15.98:0/1663319750 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f08980678c0 0x7f0898069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:41.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.007+0000 7f08aa59c640 1 --2- 172.21.15.98:0/1663319750 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f08980678c0 0x7f0898069d80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f08ac150c20 tx=0x7f08a00079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:41.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.011+0000 7f08937fe640 1 -- 172.21.15.98:0/1663319750 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f089c015fb0 con 0x7f08ac06c3a0 2024-09-19T08:38:41.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.215+0000 7f08b1191640 1 -- 172.21.15.98:0/1663319750 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f08ac15c0e0 con 0x7f08ac06c3a0 2024-09-19T08:38:41.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.215+0000 7f08937fe640 1 -- 172.21.15.98:0/1663319750 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f089c0f1530 con 0x7f08ac06c3a0 2024-09-19T08:38:41.219 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T08:38:41.221 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.215+0000 7f08b1191640 1 -- 172.21.15.98:0/1663319750 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f08980678c0 msgr2=0x7f0898069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:41.221 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.215+0000 7f08b1191640 1 --2- 172.21.15.98:0/1663319750 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f08980678c0 0x7f0898069d80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f08ac150c20 tx=0x7f08a00079c0 comp rx=0 tx=0).stop 2024-09-19T08:38:41.221 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.215+0000 7f08b1191640 1 -- 172.21.15.98:0/1663319750 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08ac06c3a0 msgr2=0x7f08ac153980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:41.221 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.215+0000 7f08b1191640 1 --2- 172.21.15.98:0/1663319750 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08ac06c3a0 0x7f08ac153980 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f089c019e90 tx=0x7f089c01aaa0 comp rx=0 tx=0).stop 2024-09-19T08:38:41.221 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.215+0000 7f08b1191640 1 -- 172.21.15.98:0/1663319750 shutdown_connections 2024-09-19T08:38:41.221 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.215+0000 7f08b1191640 1 --2- 172.21.15.98:0/1663319750 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f08980678c0 0x7f0898069d80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:41.221 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.215+0000 7f08b1191640 1 --2- 172.21.15.98:0/1663319750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f08ac158fd0 0x7f08ac14c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:41.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.215+0000 7f08b1191640 1 --2- 172.21.15.98:0/1663319750 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f08ac0b4360 0x7f08ac0aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:41.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.215+0000 7f08b1191640 1 --2- 172.21.15.98:0/1663319750 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08ac06c3a0 0x7f08ac153980 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:41.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.215+0000 7f08b1191640 1 -- 172.21.15.98:0/1663319750 >> 172.21.15.98:0/1663319750 conn(0x7f08ac0b2010 msgr2=0x7f08ac0b1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:41.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.215+0000 7f08b1191640 1 -- 172.21.15.98:0/1663319750 shutdown_connections 2024-09-19T08:38:41.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:41.215+0000 7f08b1191640 1 -- 172.21.15.98:0/1663319750 wait complete. 2024-09-19T08:38:41.238 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T08:38:41.238 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T08:38:41.238 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2024-09-19T08:38:41.238 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-19T08:38:41.238 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-19T08:38:41.239 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T08:38:41.239 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T08:38:41.239 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T08:38:41.239 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T08:38:41.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:41.239 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2024-09-19T08:38:41.246 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T08:38:41.246 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T08:38:41.246 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T08:38:41.246 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:41.247 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-09-19T08:38:41.247 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:41.247 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:38:41.247 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:38:41.247 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2024-09-19T08:38:41.247 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:38:41.247 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:38:41.247 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2024-09-19T08:38:41.247 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:38:41.247 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:38:41.247 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T08:38:41.247 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:38:41.247 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T08:38:41.248 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 42499sec preferred_lft 42499sec 2024-09-19T08:38:41.248 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::ec4:7aff:fe88:7f59/64 scope link 2024-09-19T08:38:41.248 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:41.248 INFO:teuthology.orchestra.run.smithi139.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T08:38:41.248 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 02:42:74:ed:e4:60 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:38:41.248 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T08:38:41.248 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:41.248 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-19T08:38:41.248 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:38:41.249 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-09-19T08:38:41.249 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link add name ceph-brx type bridge 2024-09-19T08:38:41.249 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip addr flush dev ceph-brx 2024-09-19T08:38:41.249 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set ceph-brx up 2024-09-19T08:38:41.249 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-19T08:38:41.249 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-09-19T08:38:41.335 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:38:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:38:41.370 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:38:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:38:41.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:41.377 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T08:38:41.387 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-09-19T08:38:41.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:41.388 DEBUG:teuthology.orchestra.run.smithi139:> route 2024-09-19T08:38:41.538 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2024-09-19T08:38:41.538 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T08:38:41.538 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T08:38:41.538 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T08:38:41.538 INFO:teuthology.orchestra.run.smithi139.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T08:38:41.538 INFO:teuthology.orchestra.run.smithi139.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T08:38:41.538 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T08:38:41.538 INFO:teuthology.orchestra.run.smithi139.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:38:41.539 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:38:41.539 INFO:teuthology.orchestra.run.smithi139.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:38:41.539 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T08:38:41.539 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:38:41.539 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-09-19T08:38:41.539 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T08:38:41.539 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T08:38:41.539 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T08:38:41.539 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-09-19T08:38:41.586 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:38:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:38:41.613 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:38:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:38:41.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:41.620 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-09-19T08:38:41.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:41.669 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list-id 2024-09-19T08:38:41.716 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:38:41.717 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-09-19T08:38:41.717 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:38:41.717 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-19T08:38:41.717 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-09-19T08:38:41.806 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:38:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:38:41.828 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:38:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:38:41.835 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-19T08:38:41.836 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:38:41.836 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-09-19T08:38:41.836 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-19T08:38:41.836 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-19T08:38:41.836 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-19T08:38:41.836 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-19T08:38:41.836 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-19T08:38:41.836 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-09-19T08:38:41.876 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:38:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:38:41.934 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:38:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:38:41.941 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:38:41.941 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-09-19T08:38:41.941 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.0 up 2024-09-19T08:38:41.941 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set dev brx.0 master ceph-brx 2024-09-19T08:38:41.941 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-09-19T08:38:42.022 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:38:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:38:42.036 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:38:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:38:42.042 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-19T08:38:42.042 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:38:42.043 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-19T08:38:42.087 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-19T08:38:42.088 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:38:42.088 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-19T08:38:42.135 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:38:42.186 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:38:42.234 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:38:42.234 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:38:42.234 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:38:42.234 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:38:42.234 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:38:42.234 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:38:42.234 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:38:42.234 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:38:42.234 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:38:42.234 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:38:42.234 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:38:42.235 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:38:42.235 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:38:42.235 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:38:42.235 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:38:42.235 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:38:42.235 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:38:42.235 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:38:42.235 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:38:42.235 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:38:42.235 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:38:42.235 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:38:42.235 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:38:42.235 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:38:42.235 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:38:42.236 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:38:42.236 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:38:42.236 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:38:42.236 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:38:42.236 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:38:42.236 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:38:42.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:42.236 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:38:42.280 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T08:38:42.280 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-19T08:38:42.325 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:38:42.339 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:38:42.388 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:38:42.389 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:38:42.389 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:38:42.389 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:38:42.389 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:38:42.389 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:38:42.389 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:38:42.389 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:38:42.389 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:38:42.389 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:38:42.389 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:38:42.389 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:38:42.389 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:38:42.390 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:38:42.390 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:38:42.390 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:38:42.390 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:38:42.390 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:38:42.390 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:38:42.390 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:38:42.390 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:38:42.390 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:38:42.390 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:38:42.390 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:38:42.390 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:38:42.390 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:38:42.390 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:38:42.391 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:38:42.391 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:38:42.391 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:38:42.391 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:38:42.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:42.392 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:38:42.403 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:2024-09-19T08:38:42.403+0000 7f8534b0f580 -1 init, newargv = 0x55e566b92390 newargc=15 2024-09-19T08:38:42.403 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[34958]: starting ceph client 2024-09-19T08:38:42.409 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[34958]: starting fuse 2024-09-19T08:38:43.509 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:38:43.525 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:38:43.575 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:38:43.575 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:38:43.575 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:38:43.575 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:38:43.575 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:38:43.575 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:38:43.575 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:38:43.575 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:38:43.576 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:38:43.576 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:38:43.576 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:38:43.576 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:38:43.576 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:38:43.576 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:38:43.576 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:38:43.576 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:38:43.576 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:38:43.576 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:38:43.576 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:38:43.576 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:38:43.576 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:38:43.577 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:38:43.577 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:38:43.577 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:38:43.577 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:38:43.577 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:38:43.577 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:38:43.577 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:38:43.577 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:38:43.577 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:38:43.577 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:38:43.577 INFO:teuthology.orchestra.run.smithi139.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-19T08:38:43.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:43.578 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:38:43.625 INFO:teuthology.orchestra.run.smithi139.stdout:45 2024-09-19T08:38:43.625 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-19T08:38:43.625 DEBUG:teuthology.orchestra.run.smithi139:> sudo stdin-killer -- python3 -c ' 2024-09-19T08:38:43.625 DEBUG:teuthology.orchestra.run.smithi139:> import glob 2024-09-19T08:38:43.625 DEBUG:teuthology.orchestra.run.smithi139:> import re 2024-09-19T08:38:43.625 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-09-19T08:38:43.625 DEBUG:teuthology.orchestra.run.smithi139:> import subprocess 2024-09-19T08:38:43.626 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:38:43.626 DEBUG:teuthology.orchestra.run.smithi139:> def _find_admin_socket(client_name): 2024-09-19T08:38:43.626 DEBUG:teuthology.orchestra.run.smithi139:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-19T08:38:43.626 DEBUG:teuthology.orchestra.run.smithi139:> files = glob.glob(asok_path) 2024-09-19T08:38:43.626 DEBUG:teuthology.orchestra.run.smithi139:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-19T08:38:43.626 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:38:43.626 DEBUG:teuthology.orchestra.run.smithi139:> # Given a non-glob path, it better be there 2024-09-19T08:38:43.626 DEBUG:teuthology.orchestra.run.smithi139:> if "*" not in asok_path: 2024-09-19T08:38:43.626 DEBUG:teuthology.orchestra.run.smithi139:> assert(len(files) == 1) 2024-09-19T08:38:43.626 DEBUG:teuthology.orchestra.run.smithi139:> return files[0] 2024-09-19T08:38:43.626 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:38:43.626 DEBUG:teuthology.orchestra.run.smithi139:> for f in files: 2024-09-19T08:38:43.626 DEBUG:teuthology.orchestra.run.smithi139:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T08:38:43.626 DEBUG:teuthology.orchestra.run.smithi139:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T08:38:43.627 DEBUG:teuthology.orchestra.run.smithi139:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T08:38:43.627 DEBUG:teuthology.orchestra.run.smithi139:> contents = proc_f.read() 2024-09-19T08:38:43.627 DEBUG:teuthology.orchestra.run.smithi139:> if mountpoint in contents: 2024-09-19T08:38:43.627 DEBUG:teuthology.orchestra.run.smithi139:> return f 2024-09-19T08:38:43.627 DEBUG:teuthology.orchestra.run.smithi139:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T08:38:43.627 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:38:43.627 DEBUG:teuthology.orchestra.run.smithi139:> print(_find_admin_socket("client.1")) 2024-09-19T08:38:43.627 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-09-19T08:38:43.747 INFO:teuthology.orchestra.run.smithi139.stdout:/var/run/ceph/ceph-client.1.34958.asok 2024-09-19T08:38:43.750 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:38:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:38:43.759 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.34958.asok 2024-09-19T08:38:43.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:43.759 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.34958.asok status 2024-09-19T08:38:43.909 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-19T08:38:43.909 INFO:teuthology.orchestra.run.smithi139.stdout: "metadata": { 2024-09-19T08:38:43.909 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T08:38:43.909 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T08:38:43.909 INFO:teuthology.orchestra.run.smithi139.stdout: "entity_id": "1", 2024-09-19T08:38:43.909 INFO:teuthology.orchestra.run.smithi139.stdout: "hostname": "smithi139", 2024-09-19T08:38:43.909 INFO:teuthology.orchestra.run.smithi139.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-19T08:38:43.909 INFO:teuthology.orchestra.run.smithi139.stdout: "pid": "34958", 2024-09-19T08:38:43.909 INFO:teuthology.orchestra.run.smithi139.stdout: "root": "/" 2024-09-19T08:38:43.910 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:38:43.910 INFO:teuthology.orchestra.run.smithi139.stdout: "dentry_count": 0, 2024-09-19T08:38:43.910 INFO:teuthology.orchestra.run.smithi139.stdout: "dentry_pinned_count": 0, 2024-09-19T08:38:43.910 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 4546, 2024-09-19T08:38:43.910 INFO:teuthology.orchestra.run.smithi139.stdout: "inst": { 2024-09-19T08:38:43.910 INFO:teuthology.orchestra.run.smithi139.stdout: "name": { 2024-09-19T08:38:43.910 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "client", 2024-09-19T08:38:43.910 INFO:teuthology.orchestra.run.smithi139.stdout: "num": 4546 2024-09-19T08:38:43.910 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:38:43.910 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": { 2024-09-19T08:38:43.910 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-19T08:38:43.910 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:38:43.910 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 101072868 2024-09-19T08:38:43.911 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-19T08:38:43.911 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:38:43.911 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": { 2024-09-19T08:38:43.911 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-19T08:38:43.911 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:38:43.911 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 101072868 2024-09-19T08:38:43.911 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:38:43.911 INFO:teuthology.orchestra.run.smithi139.stdout: "inst_str": "client.4546 192.168.0.1:0/101072868", 2024-09-19T08:38:43.911 INFO:teuthology.orchestra.run.smithi139.stdout: "addr_str": "192.168.0.1:0/101072868", 2024-09-19T08:38:43.911 INFO:teuthology.orchestra.run.smithi139.stdout: "inode_count": 1, 2024-09-19T08:38:43.911 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_epoch": 7, 2024-09-19T08:38:43.911 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_epoch": 21, 2024-09-19T08:38:43.911 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_epoch_barrier": 0, 2024-09-19T08:38:43.912 INFO:teuthology.orchestra.run.smithi139.stdout: "blocklisted": false, 2024-09-19T08:38:43.912 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs" 2024-09-19T08:38:43.912 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-19T08:38:43.922 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-19T08:38:43.922 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T08:38:44.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.047+0000 7f39ad293640 1 Processor -- start 2024-09-19T08:38:44.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.047+0000 7f39ad293640 1 -- start start 2024-09-19T08:38:44.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.047+0000 7f39ad293640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39a815c550 0x7f39a806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:44.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.047+0000 7f39ad293640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f39a806c1d0 0x7f39a80cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:44.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.047+0000 7f39ad293640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39a806d310 con 0x7f39a80cf900 2024-09-19T08:38:44.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.047+0000 7f39ad293640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f39a806c610 con 0x7f39a806c1d0 2024-09-19T08:38:44.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.047+0000 7f39ad293640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f39a806d5f0 con 0x7f39a815c550 2024-09-19T08:38:44.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.051+0000 7f39a6ffd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39a815c550 0x7f39a806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:44.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.051+0000 7f39a6ffd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39a815c550 0x7f39a806bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:51720/0 (socket says 172.21.15.98:51720) 2024-09-19T08:38:44.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.051+0000 7f39a67fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f39a806c1d0 0x7f39a80cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:44.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.051+0000 7f39a6ffd640 1 -- 172.21.15.98:0/3179245163 learned_addr learned my addr 172.21.15.98:0/3179245163 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:44.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.051+0000 7f39a67fc640 1 -- 172.21.15.98:0/3179245163 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39a815c550 msgr2=0x7f39a806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:44.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.051+0000 7f39a67fc640 1 --2- 172.21.15.98:0/3179245163 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39a815c550 0x7f39a806bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:44.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.051+0000 7f39a67fc640 1 -- 172.21.15.98:0/3179245163 >> v1:172.21.15.139:6789/0 conn(0x7f39a80cf900 legacy=0x7f39a80ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:44.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.051+0000 7f39a67fc640 1 -- 172.21.15.98:0/3179245163 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a806d8d0 con 0x7f39a806c1d0 2024-09-19T08:38:44.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.051+0000 7f39a6ffd640 1 --2- 172.21.15.98:0/3179245163 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39a815c550 0x7f39a806bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:38:44.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.051+0000 7f39a67fc640 1 --2- 172.21.15.98:0/3179245163 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39a806c1d0 0x7f39a80cf3c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f39980097c0 tx=0x7f399801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1e915f913a1783f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:44.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.051+0000 7f39a5ffb640 1 -- 172.21.15.98:0/3179245163 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39980b0070 con 0x7f39a806c1d0 2024-09-19T08:38:44.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.051+0000 7f39a5ffb640 1 -- 172.21.15.98:0/3179245163 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39980ab600 con 0x7f39a806c1d0 2024-09-19T08:38:44.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.051+0000 7f39a5ffb640 1 -- 172.21.15.98:0/3179245163 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39980ab9d0 con 0x7f39a806c1d0 2024-09-19T08:38:44.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.051+0000 7f39ad293640 1 -- 172.21.15.98:0/3179245163 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39a806c1d0 msgr2=0x7f39a80cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:44.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.051+0000 7f39ad293640 1 --2- 172.21.15.98:0/3179245163 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39a806c1d0 0x7f39a80cf3c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f39980097c0 tx=0x7f399801c4a0 comp rx=0 tx=0).stop 2024-09-19T08:38:44.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.055+0000 7f39ad293640 1 -- 172.21.15.98:0/3179245163 shutdown_connections 2024-09-19T08:38:44.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.055+0000 7f39ad293640 1 --2- 172.21.15.98:0/3179245163 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39a806c1d0 0x7f39a80cf3c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:44.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.055+0000 7f39ad293640 1 --2- 172.21.15.98:0/3179245163 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39a815c550 0x7f39a806bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:44.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.055+0000 7f39ad293640 1 -- 172.21.15.98:0/3179245163 >> 172.21.15.98:0/3179245163 conn(0x7f39a80b2010 msgr2=0x7f39a80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:44.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.055+0000 7f39ad293640 1 -- 172.21.15.98:0/3179245163 shutdown_connections 2024-09-19T08:38:44.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.055+0000 7f39ad293640 1 -- 172.21.15.98:0/3179245163 wait complete. 2024-09-19T08:38:44.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.055+0000 7f39ad293640 1 Processor -- start 2024-09-19T08:38:44.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.055+0000 7f39ad293640 1 -- start start 2024-09-19T08:38:44.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.055+0000 7f39ad293640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39a806c1d0 0x7f39a8149c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:44.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.055+0000 7f39ad293640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39a80cf900 0x7f39a81577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:44.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.055+0000 7f39ad293640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39a815c550 0x7f39a8157d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:44.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.055+0000 7f39ad293640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f39a8169230 con 0x7f39a80cf900 2024-09-19T08:38:44.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.055+0000 7f39ad293640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f39a815cb40 con 0x7f39a806c1d0 2024-09-19T08:38:44.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.055+0000 7f39ad293640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f39a81693b0 con 0x7f39a815c550 2024-09-19T08:38:44.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.055+0000 7f39a67fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39a80cf900 0x7f39a81577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:44.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.055+0000 7f39a67fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39a80cf900 0x7f39a81577c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34472/0 (socket says 172.21.15.98:34472) 2024-09-19T08:38:44.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.055+0000 7f39a67fc640 1 -- 172.21.15.98:0/472101212 learned_addr learned my addr 172.21.15.98:0/472101212 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:44.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.055+0000 7f39a6ffd640 1 --2- 172.21.15.98:0/472101212 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39a806c1d0 0x7f39a8149c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:44.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.055+0000 7f39a77fe640 1 --2- 172.21.15.98:0/472101212 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39a815c550 0x7f39a8157d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:44.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.055+0000 7f39a67fc640 1 -- 172.21.15.98:0/472101212 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39a815c550 msgr2=0x7f39a8157d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:44.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.055+0000 7f39a67fc640 1 --2- 172.21.15.98:0/472101212 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39a815c550 0x7f39a8157d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:44.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.055+0000 7f39a67fc640 1 -- 172.21.15.98:0/472101212 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39a806c1d0 msgr2=0x7f39a8149c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:44.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.055+0000 7f39a67fc640 1 --2- 172.21.15.98:0/472101212 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39a806c1d0 0x7f39a8149c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:44.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.055+0000 7f39a67fc640 1 -- 172.21.15.98:0/472101212 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a8159620 con 0x7f39a80cf900 2024-09-19T08:38:44.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.055+0000 7f39a77fe640 1 --2- 172.21.15.98:0/472101212 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39a815c550 0x7f39a8157d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:44.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.055+0000 7f39a67fc640 1 --2- 172.21.15.98:0/472101212 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39a80cf900 0x7f39a81577c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f399801a0d0 tx=0x7f399801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:44.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.055+0000 7f398f7fe640 1 -- 172.21.15.98:0/472101212 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39980b0050 con 0x7f39a80cf900 2024-09-19T08:38:44.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.055+0000 7f398f7fe640 1 -- 172.21.15.98:0/472101212 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39980bc070 con 0x7f39a80cf900 2024-09-19T08:38:44.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.055+0000 7f39ad293640 1 -- 172.21.15.98:0/472101212 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a8158310 con 0x7f39a80cf900 2024-09-19T08:38:44.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.055+0000 7f398f7fe640 1 -- 172.21.15.98:0/472101212 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39980b5830 con 0x7f39a80cf900 2024-09-19T08:38:44.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.055+0000 7f39ad293640 1 -- 172.21.15.98:0/472101212 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a81588d0 con 0x7f39a80cf900 2024-09-19T08:38:44.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.059+0000 7f39ad293640 1 -- 172.21.15.98:0/472101212 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39a80cf3c0 con 0x7f39a80cf900 2024-09-19T08:38:44.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.059+0000 7f398f7fe640 1 -- 172.21.15.98:0/472101212 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f39980b59d0 con 0x7f39a80cf900 2024-09-19T08:38:44.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.059+0000 7f398f7fe640 1 --2- 172.21.15.98:0/472101212 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f39940678c0 0x7f3994069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:44.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.059+0000 7f398f7fe640 1 -- 172.21.15.98:0/472101212 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3998120440 con 0x7f39a80cf900 2024-09-19T08:38:44.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.059+0000 7f39a6ffd640 1 --2- 172.21.15.98:0/472101212 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f39940678c0 0x7f3994069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:44.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.059+0000 7f39a6ffd640 1 --2- 172.21.15.98:0/472101212 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f39940678c0 0x7f3994069d80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f399c009a40 tx=0x7f399c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:44.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.063+0000 7f398f7fe640 1 -- 172.21.15.98:0/472101212 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3998016090 con 0x7f39a80cf900 2024-09-19T08:38:44.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.255+0000 7f39ad293640 1 -- 172.21.15.98:0/472101212 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f39a815bab0 con 0x7f39a80cf900 2024-09-19T08:38:44.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.255+0000 7f398f7fe640 1 -- 172.21.15.98:0/472101212 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f39980f0f80 con 0x7f39a80cf900 2024-09-19T08:38:44.258 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T08:38:44.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.255+0000 7f39ad293640 1 -- 172.21.15.98:0/472101212 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f39940678c0 msgr2=0x7f3994069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:44.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.255+0000 7f39ad293640 1 --2- 172.21.15.98:0/472101212 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f39940678c0 0x7f3994069d80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f399c009a40 tx=0x7f399c008040 comp rx=0 tx=0).stop 2024-09-19T08:38:44.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.255+0000 7f39ad293640 1 -- 172.21.15.98:0/472101212 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39a80cf900 msgr2=0x7f39a81577c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:44.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.255+0000 7f39ad293640 1 --2- 172.21.15.98:0/472101212 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39a80cf900 0x7f39a81577c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f399801a0d0 tx=0x7f399801ace0 comp rx=0 tx=0).stop 2024-09-19T08:38:44.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.255+0000 7f39ad293640 1 -- 172.21.15.98:0/472101212 shutdown_connections 2024-09-19T08:38:44.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.255+0000 7f39ad293640 1 --2- 172.21.15.98:0/472101212 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f39940678c0 0x7f3994069d80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:44.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.255+0000 7f39ad293640 1 --2- 172.21.15.98:0/472101212 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39a815c550 0x7f39a8157d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:44.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.255+0000 7f39ad293640 1 --2- 172.21.15.98:0/472101212 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39a80cf900 0x7f39a81577c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:44.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.255+0000 7f39ad293640 1 --2- 172.21.15.98:0/472101212 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39a806c1d0 0x7f39a8149c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:44.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.255+0000 7f39ad293640 1 -- 172.21.15.98:0/472101212 >> 172.21.15.98:0/472101212 conn(0x7f39a80b2010 msgr2=0x7f39a8156f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:44.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.255+0000 7f39ad293640 1 -- 172.21.15.98:0/472101212 shutdown_connections 2024-09-19T08:38:44.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:44.255+0000 7f39ad293640 1 -- 172.21.15.98:0/472101212 wait complete. 2024-09-19T08:38:44.276 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T08:38:44.276 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T08:38:44.276 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2024-09-19T08:38:44.276 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-19T08:38:44.276 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-19T08:38:44.276 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T08:38:44.276 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T08:38:44.277 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T08:38:44.277 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-19T08:38:44.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:44.277 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2024-09-19T08:38:44.285 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T08:38:44.285 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T08:38:44.285 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T08:38:44.286 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:44.286 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-09-19T08:38:44.286 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:44.286 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:38:44.286 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:38:44.286 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2024-09-19T08:38:44.286 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:38:44.286 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:38:44.286 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2024-09-19T08:38:44.286 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:38:44.286 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:38:44.286 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T08:38:44.286 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:38:44.287 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T08:38:44.287 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 42496sec preferred_lft 42496sec 2024-09-19T08:38:44.287 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::ec4:7aff:fe88:7f59/64 scope link 2024-09-19T08:38:44.287 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:44.287 INFO:teuthology.orchestra.run.smithi139.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T08:38:44.287 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 02:42:74:ed:e4:60 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:38:44.287 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T08:38:44.287 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:44.287 INFO:teuthology.orchestra.run.smithi139.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T08:38:44.287 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-19T08:38:44.287 INFO:teuthology.orchestra.run.smithi139.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T08:38:44.288 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:44.288 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-19T08:38:44.288 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:44.288 INFO:teuthology.orchestra.run.smithi139.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:38:44.288 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:38:44.288 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-19T08:38:44.289 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:44.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:44.289 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T08:38:44.345 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-09-19T08:38:44.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:44.347 DEBUG:teuthology.orchestra.run.smithi139:> route 2024-09-19T08:38:44.397 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2024-09-19T08:38:44.397 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T08:38:44.397 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T08:38:44.397 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T08:38:44.397 INFO:teuthology.orchestra.run.smithi139.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T08:38:44.397 INFO:teuthology.orchestra.run.smithi139.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T08:38:44.397 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T08:38:44.398 INFO:teuthology.orchestra.run.smithi139.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:38:44.398 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:38:44.398 INFO:teuthology.orchestra.run.smithi139.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:38:44.398 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T08:38:44.398 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:38:44.398 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-09-19T08:38:44.398 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T08:38:44.398 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T08:38:44.398 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T08:38:44.398 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-09-19T08:38:44.490 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:38:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:38:44.520 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:38:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:38:44.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:44.527 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-09-19T08:38:44.533 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-19T08:38:44.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:44.533 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list-id 2024-09-19T08:38:44.580 INFO:teuthology.orchestra.run.smithi139.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-19T08:38:44.580 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:38:44.580 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-09-19T08:38:44.580 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:38:44.580 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-19T08:38:44.581 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-09-19T08:38:44.669 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:38:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:38:44.690 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:38:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:38:44.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:44.697 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-19T08:38:44.710 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T08:38:44.710 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T08:38:44.710 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T08:38:44.710 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:44.711 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-09-19T08:38:44.711 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:44.711 INFO:teuthology.orchestra.run.smithi139.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T08:38:44.711 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 7a:48:ba:60:91:b3 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-19T08:38:44.711 INFO:teuthology.orchestra.run.smithi139.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-19T08:38:44.711 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:44.711 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::7848:baff:fe60:91b3/64 scope link 2024-09-19T08:38:44.711 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:44.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:44.712 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-19T08:38:44.767 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T08:38:44.767 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T08:38:44.767 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T08:38:44.768 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:44.768 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-09-19T08:38:44.768 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:44.768 INFO:teuthology.orchestra.run.smithi139.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T08:38:44.768 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 7a:48:ba:60:91:b3 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-19T08:38:44.768 INFO:teuthology.orchestra.run.smithi139.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-19T08:38:44.768 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:44.768 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::7848:baff:fe60:91b3/64 scope link 2024-09-19T08:38:44.768 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:38:44.769 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-19T08:38:44.769 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:38:44.769 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-09-19T08:38:44.770 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-19T08:38:44.770 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-19T08:38:44.770 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-19T08:38:44.770 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-19T08:38:44.770 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-19T08:38:44.770 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-09-19T08:38:44.857 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:38:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:38:44.903 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:38:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:38:44.910 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:38:44.910 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-09-19T08:38:44.910 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.1 up 2024-09-19T08:38:44.910 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set dev brx.1 master ceph-brx 2024-09-19T08:38:44.910 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-09-19T08:38:44.950 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:38:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:38:44.963 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:38:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:38:44.970 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-09-19T08:38:44.970 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:38:44.970 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-19T08:38:45.016 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-19T08:38:45.016 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:38:45.016 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-19T08:38:45.067 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:38:45.123 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:38:45.169 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:38:45.169 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:38:45.169 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:38:45.169 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:38:45.169 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:38:45.170 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:38:45.170 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:38:45.170 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:38:45.170 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:38:45.170 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:38:45.170 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:38:45.170 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:38:45.170 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:38:45.170 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:38:45.171 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:38:45.171 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:38:45.171 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:38:45.171 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:38:45.171 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:38:45.171 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:38:45.171 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:38:45.171 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:38:45.171 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:38:45.171 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:38:45.171 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:38:45.171 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:38:45.172 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:38:45.172 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:38:45.172 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:38:45.172 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:38:45.172 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:38:45.172 INFO:teuthology.orchestra.run.smithi139.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-19T08:38:45.172 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:38:45.172 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:38:45.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:45.173 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:38:45.220 INFO:teuthology.orchestra.run.smithi139.stdout:45 2024-09-19T08:38:45.220 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-09-19T08:38:45.220 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-09-19T08:38:45.265 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:38:45.281 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:38:45.328 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:38:45.328 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:38:45.328 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:38:45.329 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:38:45.329 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:38:45.329 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:38:45.329 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:38:45.329 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:38:45.329 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:38:45.329 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:38:45.329 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:38:45.329 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:38:45.333 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:38:45.333 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:38:45.333 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:38:45.333 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:38:45.333 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:38:45.333 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:38:45.333 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:38:45.333 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:38:45.333 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:38:45.333 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:38:45.334 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:38:45.334 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:38:45.334 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:38:45.334 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:38:45.334 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:38:45.334 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:38:45.334 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:38:45.334 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:38:45.334 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:38:45.334 INFO:teuthology.orchestra.run.smithi139.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-19T08:38:45.334 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:38:45.334 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:38:45.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:45.336 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:38:45.349 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi139.stderr:2024-09-19T08:38:45.351+0000 7f50e2ff2580 -1 init, newargv = 0x55d8c300e3a0 newargc=15 2024-09-19T08:38:45.349 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi139.stderr:ceph-fuse[35077]: starting ceph client 2024-09-19T08:38:45.353 INFO:teuthology.orchestra.run.smithi139.stdout:45 2024-09-19T08:38:45.354 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi139.stderr:ceph-fuse[35077]: starting fuse 2024-09-19T08:38:46.456 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:38:46.473 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:38:46.522 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:38:46.522 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:38:46.522 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:38:46.522 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:38:46.522 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:38:46.522 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:38:46.522 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:38:46.522 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:38:46.523 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:38:46.523 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:38:46.523 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:38:46.523 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:38:46.523 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:38:46.523 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:38:46.523 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:38:46.523 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:38:46.523 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:38:46.523 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:38:46.523 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:38:46.523 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:38:46.523 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:38:46.524 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:38:46.524 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:38:46.524 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:38:46.524 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:38:46.524 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:38:46.524 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:38:46.524 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:38:46.524 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:38:46.524 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:38:46.524 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:38:46.524 INFO:teuthology.orchestra.run.smithi139.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-19T08:38:46.524 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:38:46.524 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:38:46.525 INFO:teuthology.orchestra.run.smithi139.stdout:/home/ubuntu/cephtest/mnt.3 2024-09-19T08:38:46.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:46.526 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:38:46.576 INFO:teuthology.orchestra.run.smithi139.stdout:45 2024-09-19T08:38:46.577 INFO:teuthology.orchestra.run.smithi139.stdout:46 2024-09-19T08:38:46.577 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-09-19T08:38:46.577 DEBUG:teuthology.orchestra.run.smithi139:> sudo stdin-killer -- python3 -c ' 2024-09-19T08:38:46.577 DEBUG:teuthology.orchestra.run.smithi139:> import glob 2024-09-19T08:38:46.577 DEBUG:teuthology.orchestra.run.smithi139:> import re 2024-09-19T08:38:46.577 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-09-19T08:38:46.577 DEBUG:teuthology.orchestra.run.smithi139:> import subprocess 2024-09-19T08:38:46.577 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:38:46.577 DEBUG:teuthology.orchestra.run.smithi139:> def _find_admin_socket(client_name): 2024-09-19T08:38:46.578 DEBUG:teuthology.orchestra.run.smithi139:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-09-19T08:38:46.578 DEBUG:teuthology.orchestra.run.smithi139:> files = glob.glob(asok_path) 2024-09-19T08:38:46.578 DEBUG:teuthology.orchestra.run.smithi139:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-09-19T08:38:46.578 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:38:46.578 DEBUG:teuthology.orchestra.run.smithi139:> # Given a non-glob path, it better be there 2024-09-19T08:38:46.578 DEBUG:teuthology.orchestra.run.smithi139:> if "*" not in asok_path: 2024-09-19T08:38:46.578 DEBUG:teuthology.orchestra.run.smithi139:> assert(len(files) == 1) 2024-09-19T08:38:46.578 DEBUG:teuthology.orchestra.run.smithi139:> return files[0] 2024-09-19T08:38:46.578 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:38:46.578 DEBUG:teuthology.orchestra.run.smithi139:> for f in files: 2024-09-19T08:38:46.578 DEBUG:teuthology.orchestra.run.smithi139:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T08:38:46.578 DEBUG:teuthology.orchestra.run.smithi139:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T08:38:46.578 DEBUG:teuthology.orchestra.run.smithi139:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T08:38:46.578 DEBUG:teuthology.orchestra.run.smithi139:> contents = proc_f.read() 2024-09-19T08:38:46.579 DEBUG:teuthology.orchestra.run.smithi139:> if mountpoint in contents: 2024-09-19T08:38:46.579 DEBUG:teuthology.orchestra.run.smithi139:> return f 2024-09-19T08:38:46.579 DEBUG:teuthology.orchestra.run.smithi139:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T08:38:46.579 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:38:46.579 DEBUG:teuthology.orchestra.run.smithi139:> print(_find_admin_socket("client.3")) 2024-09-19T08:38:46.579 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-09-19T08:38:46.699 INFO:teuthology.orchestra.run.smithi139.stdout:/var/run/ceph/ceph-client.3.35077.asok 2024-09-19T08:38:46.703 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:38:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:38:46.710 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.35077.asok 2024-09-19T08:38:46.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:46.711 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.35077.asok status 2024-09-19T08:38:46.862 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-19T08:38:46.862 INFO:teuthology.orchestra.run.smithi139.stdout: "metadata": { 2024-09-19T08:38:46.863 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T08:38:46.863 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T08:38:46.863 INFO:teuthology.orchestra.run.smithi139.stdout: "entity_id": "3", 2024-09-19T08:38:46.863 INFO:teuthology.orchestra.run.smithi139.stdout: "hostname": "smithi139", 2024-09-19T08:38:46.863 INFO:teuthology.orchestra.run.smithi139.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-09-19T08:38:46.863 INFO:teuthology.orchestra.run.smithi139.stdout: "pid": "35077", 2024-09-19T08:38:46.863 INFO:teuthology.orchestra.run.smithi139.stdout: "root": "/" 2024-09-19T08:38:46.863 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:38:46.863 INFO:teuthology.orchestra.run.smithi139.stdout: "dentry_count": 0, 2024-09-19T08:38:46.863 INFO:teuthology.orchestra.run.smithi139.stdout: "dentry_pinned_count": 0, 2024-09-19T08:38:46.863 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 4561, 2024-09-19T08:38:46.864 INFO:teuthology.orchestra.run.smithi139.stdout: "inst": { 2024-09-19T08:38:46.864 INFO:teuthology.orchestra.run.smithi139.stdout: "name": { 2024-09-19T08:38:46.864 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "client", 2024-09-19T08:38:46.864 INFO:teuthology.orchestra.run.smithi139.stdout: "num": 4561 2024-09-19T08:38:46.864 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:38:46.864 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": { 2024-09-19T08:38:46.864 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-19T08:38:46.864 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "192.168.0.2:0", 2024-09-19T08:38:46.864 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1646168695 2024-09-19T08:38:46.864 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-19T08:38:46.864 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:38:46.864 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": { 2024-09-19T08:38:46.864 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-19T08:38:46.864 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "192.168.0.2:0", 2024-09-19T08:38:46.865 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1646168695 2024-09-19T08:38:46.865 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:38:46.865 INFO:teuthology.orchestra.run.smithi139.stdout: "inst_str": "client.4561 192.168.0.2:0/1646168695", 2024-09-19T08:38:46.865 INFO:teuthology.orchestra.run.smithi139.stdout: "addr_str": "192.168.0.2:0/1646168695", 2024-09-19T08:38:46.865 INFO:teuthology.orchestra.run.smithi139.stdout: "inode_count": 1, 2024-09-19T08:38:46.865 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_epoch": 7, 2024-09-19T08:38:46.865 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_epoch": 21, 2024-09-19T08:38:46.865 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_epoch_barrier": 0, 2024-09-19T08:38:46.865 INFO:teuthology.orchestra.run.smithi139.stdout: "blocklisted": false, 2024-09-19T08:38:46.865 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs" 2024-09-19T08:38:46.865 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-19T08:38:46.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:46.878 DEBUG:teuthology.orchestra.run.smithi098:> stat --file-system '--printf=%T 2024-09-19T08:38:46.878 DEBUG:teuthology.orchestra.run.smithi098:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T08:38:46.885 INFO:teuthology.orchestra.run.smithi098.stdout:fuseblk 2024-09-19T08:38:46.886 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T08:38:46.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:46.886 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T08:38:46.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:46.946 DEBUG:teuthology.orchestra.run.smithi098:> stat --file-system '--printf=%T 2024-09-19T08:38:46.946 DEBUG:teuthology.orchestra.run.smithi098:> ' -- /home/ubuntu/cephtest/mnt.2 2024-09-19T08:38:46.967 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:38:46.992 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:38:46.996 INFO:teuthology.orchestra.run.smithi098.stdout:fuseblk 2024-09-19T08:38:46.996 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-09-19T08:38:46.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:46.996 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-19T08:38:47.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:47.094 DEBUG:teuthology.orchestra.run.smithi139:> stat --file-system '--printf=%T 2024-09-19T08:38:47.094 DEBUG:teuthology.orchestra.run.smithi139:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-19T08:38:47.101 INFO:teuthology.orchestra.run.smithi139.stdout:fuseblk 2024-09-19T08:38:47.101 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-19T08:38:47.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:47.101 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-19T08:38:47.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:47.157 DEBUG:teuthology.orchestra.run.smithi139:> stat --file-system '--printf=%T 2024-09-19T08:38:47.157 DEBUG:teuthology.orchestra.run.smithi139:> ' -- /home/ubuntu/cephtest/mnt.3 2024-09-19T08:38:47.205 INFO:teuthology.orchestra.run.smithi139.stdout:fuseblk 2024-09-19T08:38:47.205 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-09-19T08:38:47.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:47.206 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-19T08:38:47.260 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-19T08:38:47.407 INFO:tasks.cephfs_test_runner:Starting test: test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate binary .meta file on legacy subvol root doesn't break the system 2024-09-19T08:38:47.407 INFO:tasks.cephfs_test_runner:test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-09-19T08:38:47.408 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:38:47.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.535+0000 7f739523c640 1 Processor -- start 2024-09-19T08:38:47.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.535+0000 7f739523c640 1 -- start start 2024-09-19T08:38:47.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.535+0000 7f739523c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f739015c550 0x7f739006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:47.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.535+0000 7f739523c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f739006c1d0 0x7f73900cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:47.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.535+0000 7f739523c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73900cf900 0x7f73900ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:47.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.535+0000 7f739523c640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f739006c640 con 0x7f739015c550 2024-09-19T08:38:47.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.535+0000 7f739523c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f739006d310 con 0x7f739006c1d0 2024-09-19T08:38:47.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.535+0000 7f739523c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f739006d5f0 con 0x7f73900cf900 2024-09-19T08:38:47.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.535+0000 7f738ed9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f739015c550 0x7f739006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:47.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.535+0000 7f738ed9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f739015c550 0x7f739006bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40552/0 (socket says 172.21.15.98:40552) 2024-09-19T08:38:47.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.535+0000 7f738ed9d640 1 -- 172.21.15.98:0/2425197193 learned_addr learned my addr 172.21.15.98:0/2425197193 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:47.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.535+0000 7f738e59c640 1 --2- 172.21.15.98:0/2425197193 >> v2:172.21.15.139:3300/0 conn(0x7f739006c1d0 0x7f73900cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:47.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.535+0000 7f738ed9d640 1 -- 172.21.15.98:0/2425197193 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73900cf900 msgr2=0x7f73900ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:47.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.535+0000 7f738f59e640 1 --2- 172.21.15.98:0/2425197193 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73900cf900 0x7f73900ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:47.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.535+0000 7f738ed9d640 1 --2- 172.21.15.98:0/2425197193 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73900cf900 0x7f73900ce1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:47.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.535+0000 7f738ed9d640 1 -- 172.21.15.98:0/2425197193 >> v2:172.21.15.139:3300/0 conn(0x7f739006c1d0 msgr2=0x7f73900cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:47.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.535+0000 7f738ed9d640 1 --2- 172.21.15.98:0/2425197193 >> v2:172.21.15.139:3300/0 conn(0x7f739006c1d0 0x7f73900cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:47.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.535+0000 7f738ed9d640 1 -- 172.21.15.98:0/2425197193 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f739006d8d0 con 0x7f739015c550 2024-09-19T08:38:47.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.535+0000 7f738e59c640 1 --2- 172.21.15.98:0/2425197193 >> v2:172.21.15.139:3300/0 conn(0x7f739006c1d0 0x7f73900cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:47.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.535+0000 7f738f59e640 1 --2- 172.21.15.98:0/2425197193 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73900cf900 0x7f73900ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:47.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.539+0000 7f738ed9d640 1 --2- 172.21.15.98:0/2425197193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f739015c550 0x7f739006bc90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f73900ccc60 tx=0x7f738001c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb669e07aeb39b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:47.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.539+0000 7f738dd9b640 1 -- 172.21.15.98:0/2425197193 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73800b0070 con 0x7f739015c550 2024-09-19T08:38:47.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.539+0000 7f738dd9b640 1 -- 172.21.15.98:0/2425197193 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73800ab600 con 0x7f739015c550 2024-09-19T08:38:47.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.539+0000 7f738dd9b640 1 -- 172.21.15.98:0/2425197193 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73800ab9d0 con 0x7f739015c550 2024-09-19T08:38:47.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.539+0000 7f739523c640 1 -- 172.21.15.98:0/2425197193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f739015c550 msgr2=0x7f739006bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:47.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.539+0000 7f739523c640 1 --2- 172.21.15.98:0/2425197193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f739015c550 0x7f739006bc90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f73900ccc60 tx=0x7f738001c260 comp rx=0 tx=0).stop 2024-09-19T08:38:47.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.539+0000 7f739523c640 1 -- 172.21.15.98:0/2425197193 shutdown_connections 2024-09-19T08:38:47.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.539+0000 7f739523c640 1 --2- 172.21.15.98:0/2425197193 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73900cf900 0x7f73900ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:47.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.539+0000 7f739523c640 1 --2- 172.21.15.98:0/2425197193 >> v2:172.21.15.139:3300/0 conn(0x7f739006c1d0 0x7f73900cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:47.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.539+0000 7f739523c640 1 --2- 172.21.15.98:0/2425197193 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f739015c550 0x7f739006bc90 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:47.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.539+0000 7f739523c640 1 -- 172.21.15.98:0/2425197193 >> 172.21.15.98:0/2425197193 conn(0x7f73900b2010 msgr2=0x7f73900b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:47.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.539+0000 7f739523c640 1 -- 172.21.15.98:0/2425197193 shutdown_connections 2024-09-19T08:38:47.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.539+0000 7f739523c640 1 -- 172.21.15.98:0/2425197193 wait complete. 2024-09-19T08:38:47.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.539+0000 7f739523c640 1 Processor -- start 2024-09-19T08:38:47.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.539+0000 7f739523c640 1 -- start start 2024-09-19T08:38:47.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.539+0000 7f739523c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f739006c1d0 0x7f73901fa410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:47.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.539+0000 7f739523c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73900cf900 0x7f73900b46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:47.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.539+0000 7f739523c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f739015c550 0x7f73900ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:47.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.539+0000 7f739523c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7390072520 con 0x7f739015c550 2024-09-19T08:38:47.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.539+0000 7f739523c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f73900723a0 con 0x7f739006c1d0 2024-09-19T08:38:47.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.539+0000 7f739523c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f73900726a0 con 0x7f73900cf900 2024-09-19T08:38:47.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.539+0000 7f738f59e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f739015c550 0x7f73900ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:47.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.539+0000 7f738f59e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f739015c550 0x7f73900ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40568/0 (socket says 172.21.15.98:40568) 2024-09-19T08:38:47.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.539+0000 7f738f59e640 1 -- 172.21.15.98:0/293000984 learned_addr learned my addr 172.21.15.98:0/293000984 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:47.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.539+0000 7f738e59c640 1 --2- 172.21.15.98:0/293000984 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73900cf900 0x7f73900b46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:47.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.539+0000 7f738ed9d640 1 --2- 172.21.15.98:0/293000984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f739006c1d0 0x7f73901fa410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:47.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.543+0000 7f738f59e640 1 -- 172.21.15.98:0/293000984 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73900cf900 msgr2=0x7f73900b46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:47.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.543+0000 7f738f59e640 1 --2- 172.21.15.98:0/293000984 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73900cf900 0x7f73900b46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:47.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.543+0000 7f738f59e640 1 -- 172.21.15.98:0/293000984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f739006c1d0 msgr2=0x7f73901fa410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:47.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.543+0000 7f738f59e640 1 --2- 172.21.15.98:0/293000984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f739006c1d0 0x7f73901fa410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:47.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.543+0000 7f738f59e640 1 -- 172.21.15.98:0/293000984 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73900b5360 con 0x7f739015c550 2024-09-19T08:38:47.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.543+0000 7f738f59e640 1 --2- 172.21.15.98:0/293000984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f739015c550 0x7f73900ae8b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f738800cf20 tx=0x7f738800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:47.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.543+0000 7f73777fe640 1 -- 172.21.15.98:0/293000984 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738801a070 con 0x7f739015c550 2024-09-19T08:38:47.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.543+0000 7f73777fe640 1 -- 172.21.15.98:0/293000984 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f738800d840 con 0x7f739015c550 2024-09-19T08:38:47.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.543+0000 7f739523c640 1 -- 172.21.15.98:0/293000984 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7390157b30 con 0x7f739015c550 2024-09-19T08:38:47.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.543+0000 7f73777fe640 1 -- 172.21.15.98:0/293000984 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738800dc30 con 0x7f739015c550 2024-09-19T08:38:47.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.543+0000 7f739523c640 1 -- 172.21.15.98:0/293000984 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73901580b0 con 0x7f739015c550 2024-09-19T08:38:47.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.543+0000 7f739523c640 1 -- 172.21.15.98:0/293000984 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73900ce1a0 con 0x7f739015c550 2024-09-19T08:38:47.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.543+0000 7f73777fe640 1 -- 172.21.15.98:0/293000984 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f738801d070 con 0x7f739015c550 2024-09-19T08:38:47.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.543+0000 7f73777fe640 1 --2- 172.21.15.98:0/293000984 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f737c0678c0 0x7f737c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:47.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.543+0000 7f738ed9d640 1 --2- 172.21.15.98:0/293000984 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f737c0678c0 0x7f737c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:47.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.543+0000 7f73777fe640 1 -- 172.21.15.98:0/293000984 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7388086540 con 0x7f739015c550 2024-09-19T08:38:47.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.543+0000 7f738ed9d640 1 --2- 172.21.15.98:0/293000984 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f737c0678c0 0x7f737c069d80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f73800ad000 tx=0x7f7380008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:47.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.547+0000 7f73777fe640 1 -- 172.21.15.98:0/293000984 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f7388057080 con 0x7f739015c550 2024-09-19T08:38:47.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.707+0000 7f739523c640 1 -- 172.21.15.98:0/293000984 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73900b0f10 con 0x7f739015c550 2024-09-19T08:38:47.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.707+0000 7f73777fe640 1 -- 172.21.15.98:0/293000984 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16665 (secure 0 0 0) 0x7f7388056980 con 0x7f739015c550 2024-09-19T08:38:47.712 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:38:47.712 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":21,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:38:20.362527+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:38:16.588688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T08:38:17.700392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:38:47.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.711+0000 7f739523c640 1 -- 172.21.15.98:0/293000984 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f737c0678c0 msgr2=0x7f737c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:47.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.711+0000 7f739523c640 1 --2- 172.21.15.98:0/293000984 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f737c0678c0 0x7f737c069d80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f73800ad000 tx=0x7f7380008040 comp rx=0 tx=0).stop 2024-09-19T08:38:47.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.711+0000 7f739523c640 1 -- 172.21.15.98:0/293000984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f739015c550 msgr2=0x7f73900ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:47.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.711+0000 7f739523c640 1 --2- 172.21.15.98:0/293000984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f739015c550 0x7f73900ae8b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f738800cf20 tx=0x7f738800a7d0 comp rx=0 tx=0).stop 2024-09-19T08:38:47.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.711+0000 7f739523c640 1 -- 172.21.15.98:0/293000984 shutdown_connections 2024-09-19T08:38:47.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.711+0000 7f739523c640 1 --2- 172.21.15.98:0/293000984 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f737c0678c0 0x7f737c069d80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:47.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.711+0000 7f739523c640 1 --2- 172.21.15.98:0/293000984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f739015c550 0x7f73900ae8b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:47.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.711+0000 7f739523c640 1 --2- 172.21.15.98:0/293000984 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73900cf900 0x7f73900b46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:47.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.711+0000 7f739523c640 1 --2- 172.21.15.98:0/293000984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f739006c1d0 0x7f73901fa410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:47.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.711+0000 7f739523c640 1 -- 172.21.15.98:0/293000984 >> 172.21.15.98:0/293000984 conn(0x7f73900b2010 msgr2=0x7f73900b0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:47.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.711+0000 7f739523c640 1 -- 172.21.15.98:0/293000984 shutdown_connections 2024-09-19T08:38:47.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.711+0000 7f739523c640 1 -- 172.21.15.98:0/293000984 wait complete. 2024-09-19T08:38:47.730 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T08:38:08.105684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T08:38:16.588688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T08:38:17.700392+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T08:38:47.730 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T08:38:47.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.839+0000 7f323657c640 1 Processor -- start 2024-09-19T08:38:47.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.839+0000 7f323657c640 1 -- start start 2024-09-19T08:38:47.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.839+0000 7f323657c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f32300729f0 0x7f323006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:47.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.839+0000 7f323657c640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3230072ea0 con 0x7f3230144d10 2024-09-19T08:38:47.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.839+0000 7f323657c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3230073180 con 0x7f323006c1d0 2024-09-19T08:38:47.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.839+0000 7f323657c640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f323006c620 con 0x7f32300729f0 2024-09-19T08:38:47.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.839+0000 7f322f7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f32300729f0 0x7f323006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:47.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.839+0000 7f322f7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f32300729f0 0x7f323006bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40572/0 (socket says 172.21.15.98:40572) 2024-09-19T08:38:47.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.839+0000 7f322f7fe640 1 -- 172.21.15.98:0/1807369396 learned_addr learned my addr 172.21.15.98:0/1807369396 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:47.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.839+0000 7f322f7fe640 1 -- 172.21.15.98:0/1807369396 >> v1:172.21.15.139:6789/0 conn(0x7f323006c1d0 legacy=0x7f32300cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:47.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.839+0000 7f322f7fe640 1 -- 172.21.15.98:0/1807369396 >> v1:172.21.15.98:6789/0 conn(0x7f3230144d10 legacy=0x7f32300722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:47.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.839+0000 7f322f7fe640 1 -- 172.21.15.98:0/1807369396 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3230073460 con 0x7f32300729f0 2024-09-19T08:38:47.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.843+0000 7f322f7fe640 1 --2- 172.21.15.98:0/1807369396 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32300729f0 0x7f323006bc90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f322400bb10 tx=0x7f322401ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=604a090b3808e931 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:47.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.843+0000 7f322effd640 1 -- 172.21.15.98:0/1807369396 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32240b4070 con 0x7f32300729f0 2024-09-19T08:38:47.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.843+0000 7f322effd640 1 -- 172.21.15.98:0/1807369396 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32240a7ac0 con 0x7f32300729f0 2024-09-19T08:38:47.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.843+0000 7f322effd640 1 -- 172.21.15.98:0/1807369396 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32240af440 con 0x7f32300729f0 2024-09-19T08:38:47.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.843+0000 7f323657c640 1 -- 172.21.15.98:0/1807369396 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32300729f0 msgr2=0x7f323006bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:47.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.843+0000 7f323657c640 1 --2- 172.21.15.98:0/1807369396 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32300729f0 0x7f323006bc90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f322400bb10 tx=0x7f322401ebf0 comp rx=0 tx=0).stop 2024-09-19T08:38:47.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.843+0000 7f323657c640 1 -- 172.21.15.98:0/1807369396 shutdown_connections 2024-09-19T08:38:47.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.843+0000 7f323657c640 1 --2- 172.21.15.98:0/1807369396 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32300729f0 0x7f323006bc90 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:47.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.843+0000 7f323657c640 1 -- 172.21.15.98:0/1807369396 >> 172.21.15.98:0/1807369396 conn(0x7f32300b2010 msgr2=0x7f32300b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:47.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.843+0000 7f323657c640 1 -- 172.21.15.98:0/1807369396 shutdown_connections 2024-09-19T08:38:47.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.843+0000 7f323657c640 1 -- 172.21.15.98:0/1807369396 wait complete. 2024-09-19T08:38:47.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.843+0000 7f323657c640 1 Processor -- start 2024-09-19T08:38:47.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.843+0000 7f323657c640 1 -- start start 2024-09-19T08:38:47.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.843+0000 7f323657c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f323006c1d0 0x7f3230157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:47.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.843+0000 7f323657c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32300729f0 0x7f323015a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:47.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.843+0000 7f323657c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3230144d10 0x7f323015a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:47.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.843+0000 7f323657c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f32300ce540 con 0x7f3230144d10 2024-09-19T08:38:47.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.843+0000 7f323657c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f32300ce3c0 con 0x7f32300729f0 2024-09-19T08:38:47.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.843+0000 7f323657c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f32300b3410 con 0x7f323006c1d0 2024-09-19T08:38:47.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.843+0000 7f3234b19640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3230144d10 0x7f323015a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:47.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.843+0000 7f3234b19640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3230144d10 0x7f323015a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40576/0 (socket says 172.21.15.98:40576) 2024-09-19T08:38:47.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.843+0000 7f3234b19640 1 -- 172.21.15.98:0/2649929063 learned_addr learned my addr 172.21.15.98:0/2649929063 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:47.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.843+0000 7f322ffff640 1 --2- 172.21.15.98:0/2649929063 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f323006c1d0 0x7f3230157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:47.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.843+0000 7f322f7fe640 1 --2- 172.21.15.98:0/2649929063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32300729f0 0x7f323015a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:47.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.843+0000 7f3234b19640 1 -- 172.21.15.98:0/2649929063 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f323006c1d0 msgr2=0x7f3230157b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:47.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.843+0000 7f3234b19640 1 --2- 172.21.15.98:0/2649929063 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f323006c1d0 0x7f3230157b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:47.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.843+0000 7f3234b19640 1 -- 172.21.15.98:0/2649929063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32300729f0 msgr2=0x7f323015a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:47.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.843+0000 7f3234b19640 1 --2- 172.21.15.98:0/2649929063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32300729f0 0x7f323015a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:47.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.843+0000 7f3234b19640 1 -- 172.21.15.98:0/2649929063 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f323006df40 con 0x7f3230144d10 2024-09-19T08:38:47.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.843+0000 7f322ffff640 1 --2- 172.21.15.98:0/2649929063 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f323006c1d0 0x7f3230157b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:47.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.843+0000 7f3234b19640 1 --2- 172.21.15.98:0/2649929063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3230144d10 0x7f323015a910 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f322800c7b0 tx=0x7f322800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:47.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.843+0000 7f322cff9640 1 -- 172.21.15.98:0/2649929063 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3228004b70 con 0x7f3230144d10 2024-09-19T08:38:47.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.843+0000 7f322cff9640 1 -- 172.21.15.98:0/2649929063 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3228004d10 con 0x7f3230144d10 2024-09-19T08:38:47.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.843+0000 7f323657c640 1 -- 172.21.15.98:0/2649929063 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3230141ec0 con 0x7f3230144d10 2024-09-19T08:38:47.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.843+0000 7f322cff9640 1 -- 172.21.15.98:0/2649929063 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322800aa90 con 0x7f3230144d10 2024-09-19T08:38:47.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.843+0000 7f323657c640 1 -- 172.21.15.98:0/2649929063 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3230142440 con 0x7f3230144d10 2024-09-19T08:38:47.853 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.847+0000 7f323657c640 1 -- 172.21.15.98:0/2649929063 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f323006bc90 con 0x7f3230144d10 2024-09-19T08:38:47.853 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.847+0000 7f322cff9640 1 -- 172.21.15.98:0/2649929063 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f322800ac30 con 0x7f3230144d10 2024-09-19T08:38:47.853 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.847+0000 7f322cff9640 1 --2- 172.21.15.98:0/2649929063 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f321c0678c0 0x7f321c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:47.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.847+0000 7f322ffff640 1 --2- 172.21.15.98:0/2649929063 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f321c0678c0 0x7f321c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:47.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.847+0000 7f322cff9640 1 -- 172.21.15.98:0/2649929063 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3228085ed0 con 0x7f3230144d10 2024-09-19T08:38:47.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.847+0000 7f322ffff640 1 --2- 172.21.15.98:0/2649929063 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f321c0678c0 0x7f321c069d80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3220001650 tx=0x7f3220009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:47.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.851+0000 7f322cff9640 1 -- 172.21.15.98:0/2649929063 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3228020070 con 0x7f3230144d10 2024-09-19T08:38:48.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.999+0000 7f323657c640 1 -- 172.21.15.98:0/2649929063 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f32300b43c0 con 0x7f3230144d10 2024-09-19T08:38:48.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:47.999+0000 7f322cff9640 1 -- 172.21.15.98:0/2649929063 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f3228056a10 con 0x7f3230144d10 2024-09-19T08:38:48.005 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2024-09-19T08:38:48.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.003+0000 7f323657c640 1 -- 172.21.15.98:0/2649929063 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f321c0678c0 msgr2=0x7f321c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:48.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.003+0000 7f323657c640 1 --2- 172.21.15.98:0/2649929063 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f321c0678c0 0x7f321c069d80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3220001650 tx=0x7f3220009040 comp rx=0 tx=0).stop 2024-09-19T08:38:48.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.003+0000 7f323657c640 1 -- 172.21.15.98:0/2649929063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3230144d10 msgr2=0x7f323015a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:48.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.003+0000 7f323657c640 1 --2- 172.21.15.98:0/2649929063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3230144d10 0x7f323015a910 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f322800c7b0 tx=0x7f322800cc70 comp rx=0 tx=0).stop 2024-09-19T08:38:48.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.003+0000 7f323657c640 1 -- 172.21.15.98:0/2649929063 shutdown_connections 2024-09-19T08:38:48.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.003+0000 7f323657c640 1 --2- 172.21.15.98:0/2649929063 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f321c0678c0 0x7f321c069d80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:48.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.003+0000 7f323657c640 1 --2- 172.21.15.98:0/2649929063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3230144d10 0x7f323015a910 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:48.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.003+0000 7f323657c640 1 --2- 172.21.15.98:0/2649929063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32300729f0 0x7f323015a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:48.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.003+0000 7f323657c640 1 --2- 172.21.15.98:0/2649929063 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f323006c1d0 0x7f3230157b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:48.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.003+0000 7f323657c640 1 -- 172.21.15.98:0/2649929063 >> 172.21.15.98:0/2649929063 conn(0x7f32300b2010 msgr2=0x7f32300afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:48.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.003+0000 7f323657c640 1 -- 172.21.15.98:0/2649929063 shutdown_connections 2024-09-19T08:38:48.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.003+0000 7f323657c640 1 -- 172.21.15.98:0/2649929063 wait complete. 2024-09-19T08:38:48.022 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-19T08:38:48.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.139+0000 7f42804c4640 1 Processor -- start 2024-09-19T08:38:48.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.139+0000 7f42804c4640 1 -- start start 2024-09-19T08:38:48.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.143+0000 7f42804c4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f427806c3a0 0x7f42780713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:48.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.143+0000 7f42804c4640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f42780718f0 0x7f42780b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:48.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.143+0000 7f42804c4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4278073530 con 0x7f4278145b70 2024-09-19T08:38:48.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.143+0000 7f42804c4640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f4278073810 con 0x7f42780718f0 2024-09-19T08:38:48.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.143+0000 7f42804c4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4278073af0 con 0x7f427806c3a0 2024-09-19T08:38:48.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.143+0000 7f427ea61640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f42780718f0 0x7f42780b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:48.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.143+0000 7f427ea61640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f42780718f0 0x7f42780b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40590/0 (socket says 172.21.15.98:40590) 2024-09-19T08:38:48.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.143+0000 7f427ea61640 1 -- 172.21.15.98:0/569567652 learned_addr learned my addr 172.21.15.98:0/569567652 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:48.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.143+0000 7f427ea61640 1 -- 172.21.15.98:0/569567652 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f427806c3a0 msgr2=0x7f42780713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:48.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.143+0000 7f427ea61640 1 --2- 172.21.15.98:0/569567652 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f427806c3a0 0x7f42780713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:48.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.143+0000 7f427ea61640 1 -- 172.21.15.98:0/569567652 >> v1:172.21.15.139:6789/0 conn(0x7f4278145b70 legacy=0x7f427806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:48.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.143+0000 7f427ea61640 1 -- 172.21.15.98:0/569567652 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4278073dd0 con 0x7f42780718f0 2024-09-19T08:38:48.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.143+0000 7f427ea61640 1 --2- 172.21.15.98:0/569567652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42780718f0 0x7f42780b3410 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4270009e50 tx=0x7f427001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=703a2a93b49b847f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:48.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.143+0000 7f427d25e640 1 -- 172.21.15.98:0/569567652 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42700a55e0 con 0x7f42780718f0 2024-09-19T08:38:48.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.143+0000 7f427d25e640 1 -- 172.21.15.98:0/569567652 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42700a5780 con 0x7f42780718f0 2024-09-19T08:38:48.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.143+0000 7f427d25e640 1 -- 172.21.15.98:0/569567652 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42700a5b50 con 0x7f42780718f0 2024-09-19T08:38:48.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.147+0000 7f42804c4640 1 -- 172.21.15.98:0/569567652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42780718f0 msgr2=0x7f42780b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:48.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.147+0000 7f42804c4640 1 --2- 172.21.15.98:0/569567652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42780718f0 0x7f42780b3410 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4270009e50 tx=0x7f427001c960 comp rx=0 tx=0).stop 2024-09-19T08:38:48.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.147+0000 7f42804c4640 1 -- 172.21.15.98:0/569567652 shutdown_connections 2024-09-19T08:38:48.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.147+0000 7f42804c4640 1 --2- 172.21.15.98:0/569567652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42780718f0 0x7f42780b3410 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:48.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.147+0000 7f42804c4640 1 --2- 172.21.15.98:0/569567652 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f427806c3a0 0x7f42780713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:48.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.147+0000 7f42804c4640 1 -- 172.21.15.98:0/569567652 >> 172.21.15.98:0/569567652 conn(0x7f42780b2010 msgr2=0x7f42780b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:48.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.147+0000 7f42804c4640 1 -- 172.21.15.98:0/569567652 shutdown_connections 2024-09-19T08:38:48.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.147+0000 7f42804c4640 1 -- 172.21.15.98:0/569567652 wait complete. 2024-09-19T08:38:48.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.147+0000 7f42804c4640 1 Processor -- start 2024-09-19T08:38:48.152 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.147+0000 7f42804c4640 1 -- start start 2024-09-19T08:38:48.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.147+0000 7f42804c4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f427806c3a0 0x7f4278158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:48.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.147+0000 7f42804c4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42780718f0 0x7f427815ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:48.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.147+0000 7f42804c4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4278145b70 0x7f427813cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:48.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.147+0000 7f42804c4640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f42780b4530 con 0x7f42780718f0 2024-09-19T08:38:48.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.147+0000 7f42804c4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f42780b43b0 con 0x7f427806c3a0 2024-09-19T08:38:48.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.147+0000 7f42804c4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f42780b46b0 con 0x7f4278145b70 2024-09-19T08:38:48.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.147+0000 7f427da5f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42780718f0 0x7f427815ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:48.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.147+0000 7f427da5f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42780718f0 0x7f427815ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40602/0 (socket says 172.21.15.98:40602) 2024-09-19T08:38:48.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.147+0000 7f427da5f640 1 -- 172.21.15.98:0/3629500595 learned_addr learned my addr 172.21.15.98:0/3629500595 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:48.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.147+0000 7f427da5f640 1 -- 172.21.15.98:0/3629500595 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4278145b70 msgr2=0x7f427813cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:48.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.147+0000 7f427e260640 1 --2- 172.21.15.98:0/3629500595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f427806c3a0 0x7f4278158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:48.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.147+0000 7f427da5f640 1 --2- 172.21.15.98:0/3629500595 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4278145b70 0x7f427813cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:48.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.147+0000 7f427da5f640 1 -- 172.21.15.98:0/3629500595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f427806c3a0 msgr2=0x7f4278158f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:48.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.147+0000 7f427da5f640 1 --2- 172.21.15.98:0/3629500595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f427806c3a0 0x7f4278158f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:48.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.147+0000 7f427da5f640 1 -- 172.21.15.98:0/3629500595 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4278159490 con 0x7f42780718f0 2024-09-19T08:38:48.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.147+0000 7f427da5f640 1 --2- 172.21.15.98:0/3629500595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42780718f0 0x7f427815ca20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f426800b550 tx=0x7f426800ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:48.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.147+0000 7f426effd640 1 -- 172.21.15.98:0/3629500595 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4268014070 con 0x7f42780718f0 2024-09-19T08:38:48.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.147+0000 7f426effd640 1 -- 172.21.15.98:0/3629500595 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4268010890 con 0x7f42780718f0 2024-09-19T08:38:48.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.147+0000 7f426effd640 1 -- 172.21.15.98:0/3629500595 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4268010c80 con 0x7f42780718f0 2024-09-19T08:38:48.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.147+0000 7f42804c4640 1 -- 172.21.15.98:0/3629500595 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f427815a850 con 0x7f42780718f0 2024-09-19T08:38:48.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.147+0000 7f42804c4640 1 -- 172.21.15.98:0/3629500595 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f427815ae10 con 0x7f42780718f0 2024-09-19T08:38:48.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.151+0000 7f42804c4640 1 -- 172.21.15.98:0/3629500595 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42780713b0 con 0x7f42780718f0 2024-09-19T08:38:48.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.151+0000 7f426effd640 1 -- 172.21.15.98:0/3629500595 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4268017070 con 0x7f42780718f0 2024-09-19T08:38:48.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.151+0000 7f426effd640 1 --2- 172.21.15.98:0/3629500595 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f425c0678c0 0x7f425c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:48.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.151+0000 7f426effd640 1 -- 172.21.15.98:0/3629500595 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4268085a60 con 0x7f42780718f0 2024-09-19T08:38:48.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.151+0000 7f427e260640 1 --2- 172.21.15.98:0/3629500595 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f425c0678c0 0x7f425c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:48.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.151+0000 7f427e260640 1 --2- 172.21.15.98:0/3629500595 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f425c0678c0 0x7f425c069d80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f427400a790 tx=0x7f4274008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:48.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.155+0000 7f426effd640 1 -- 172.21.15.98:0/3629500595 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f42680565a0 con 0x7f42780718f0 2024-09-19T08:38:48.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.315+0000 7f42804c4640 1 -- 172.21.15.98:0/3629500595 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f427815a0f0 con 0x7f42780718f0 2024-09-19T08:38:48.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.315+0000 7f426effd640 1 -- 172.21.15.98:0/3629500595 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f4268010e20 con 0x7f42780718f0 2024-09-19T08:38:48.320 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 32 2024-09-19T08:38:48.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.319+0000 7f42804c4640 1 -- 172.21.15.98:0/3629500595 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f425c0678c0 msgr2=0x7f425c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:48.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.319+0000 7f42804c4640 1 --2- 172.21.15.98:0/3629500595 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f425c0678c0 0x7f425c069d80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f427400a790 tx=0x7f4274008040 comp rx=0 tx=0).stop 2024-09-19T08:38:48.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.319+0000 7f42804c4640 1 -- 172.21.15.98:0/3629500595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42780718f0 msgr2=0x7f427815ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:48.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.319+0000 7f42804c4640 1 --2- 172.21.15.98:0/3629500595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42780718f0 0x7f427815ca20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f426800b550 tx=0x7f426800ba10 comp rx=0 tx=0).stop 2024-09-19T08:38:48.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.319+0000 7f42804c4640 1 -- 172.21.15.98:0/3629500595 shutdown_connections 2024-09-19T08:38:48.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.319+0000 7f42804c4640 1 --2- 172.21.15.98:0/3629500595 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f425c0678c0 0x7f425c069d80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:48.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.319+0000 7f42804c4640 1 --2- 172.21.15.98:0/3629500595 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4278145b70 0x7f427813cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:48.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.319+0000 7f42804c4640 1 --2- 172.21.15.98:0/3629500595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42780718f0 0x7f427815ca20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:48.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.319+0000 7f42804c4640 1 --2- 172.21.15.98:0/3629500595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f427806c3a0 0x7f4278158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:48.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.319+0000 7f42804c4640 1 -- 172.21.15.98:0/3629500595 >> 172.21.15.98:0/3629500595 conn(0x7f42780b2010 msgr2=0x7f42780cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:48.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.319+0000 7f42804c4640 1 -- 172.21.15.98:0/3629500595 shutdown_connections 2024-09-19T08:38:48.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.319+0000 7f42804c4640 1 -- 172.21.15.98:0/3629500595 wait complete. 2024-09-19T08:38:48.340 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-19T08:38:48.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.463+0000 7fc0ae5af640 1 Processor -- start 2024-09-19T08:38:48.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.463+0000 7fc0ae5af640 1 -- start start 2024-09-19T08:38:48.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.463+0000 7fc0ae5af640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc0a806ce10 0x7fc0a8074c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:48.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.463+0000 7fc0ae5af640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc0a80751b0 0x7fc0a80adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:48.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.463+0000 7fc0ae5af640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0a806edb0 con 0x7fc0a815c790 2024-09-19T08:38:48.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.463+0000 7fc0ae5af640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fc0a80cfd30 con 0x7fc0a80751b0 2024-09-19T08:38:48.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.463+0000 7fc0ae5af640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fc0a80d0010 con 0x7fc0a806ce10 2024-09-19T08:38:48.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.463+0000 7fc0acb4c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc0a80751b0 0x7fc0a80adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:48.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.463+0000 7fc0acb4c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc0a80751b0 0x7fc0a80adfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40606/0 (socket says 172.21.15.98:40606) 2024-09-19T08:38:48.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.463+0000 7fc0acb4c640 1 -- 172.21.15.98:0/3309255859 learned_addr learned my addr 172.21.15.98:0/3309255859 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:48.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.463+0000 7fc0a77fe640 1 --2- 172.21.15.98:0/3309255859 >> v2:172.21.15.139:3300/0 conn(0x7fc0a806ce10 0x7fc0a8074c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:48.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.463+0000 7fc0acb4c640 1 -- 172.21.15.98:0/3309255859 >> v2:172.21.15.139:3300/0 conn(0x7fc0a806ce10 msgr2=0x7fc0a8074c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:48.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.463+0000 7fc0acb4c640 1 --2- 172.21.15.98:0/3309255859 >> v2:172.21.15.139:3300/0 conn(0x7fc0a806ce10 0x7fc0a8074c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:48.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.463+0000 7fc0acb4c640 1 -- 172.21.15.98:0/3309255859 >> v1:172.21.15.139:6789/0 conn(0x7fc0a815c790 legacy=0x7fc0a80cf4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:48.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.463+0000 7fc0acb4c640 1 -- 172.21.15.98:0/3309255859 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a80d02f0 con 0x7fc0a80751b0 2024-09-19T08:38:48.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.463+0000 7fc0a77fe640 1 --2- 172.21.15.98:0/3309255859 >> v2:172.21.15.139:3300/0 conn(0x7fc0a806ce10 0x7fc0a8074c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:48.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.463+0000 7fc0acb4c640 1 --2- 172.21.15.98:0/3309255859 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0a80751b0 0x7fc0a80adfa0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc0a8071bf0 tx=0x7fc0a001c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ceeb9926612549 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:48.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.467+0000 7fc0a6ffd640 1 -- 172.21.15.98:0/3309255859 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a00ac020 con 0x7fc0a80751b0 2024-09-19T08:38:48.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.467+0000 7fc0a6ffd640 1 -- 172.21.15.98:0/3309255859 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0a00a64a0 con 0x7fc0a80751b0 2024-09-19T08:38:48.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.467+0000 7fc0a6ffd640 1 -- 172.21.15.98:0/3309255859 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a00a6870 con 0x7fc0a80751b0 2024-09-19T08:38:48.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.467+0000 7fc0ae5af640 1 -- 172.21.15.98:0/3309255859 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0a80751b0 msgr2=0x7fc0a80adfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:48.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.467+0000 7fc0ae5af640 1 --2- 172.21.15.98:0/3309255859 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0a80751b0 0x7fc0a80adfa0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc0a8071bf0 tx=0x7fc0a001c7d0 comp rx=0 tx=0).stop 2024-09-19T08:38:48.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.467+0000 7fc0ae5af640 1 -- 172.21.15.98:0/3309255859 shutdown_connections 2024-09-19T08:38:48.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.467+0000 7fc0ae5af640 1 --2- 172.21.15.98:0/3309255859 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0a80751b0 0x7fc0a80adfa0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:48.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.467+0000 7fc0ae5af640 1 --2- 172.21.15.98:0/3309255859 >> v2:172.21.15.139:3300/0 conn(0x7fc0a806ce10 0x7fc0a8074c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:48.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.467+0000 7fc0ae5af640 1 -- 172.21.15.98:0/3309255859 >> 172.21.15.98:0/3309255859 conn(0x7fc0a80d1250 msgr2=0x7fc0a80d1660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:48.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.467+0000 7fc0ae5af640 1 -- 172.21.15.98:0/3309255859 shutdown_connections 2024-09-19T08:38:48.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.467+0000 7fc0ae5af640 1 -- 172.21.15.98:0/3309255859 wait complete. 2024-09-19T08:38:48.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.467+0000 7fc0ae5af640 1 Processor -- start 2024-09-19T08:38:48.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.467+0000 7fc0ae5af640 1 -- start start 2024-09-19T08:38:48.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.467+0000 7fc0ae5af640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0a806ce10 0x7fc0a80c7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:48.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.467+0000 7fc0ae5af640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0a80751b0 0x7fc0a8158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:48.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.467+0000 7fc0ae5af640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0a815c790 0x7fc0a80b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:48.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.467+0000 7fc0ae5af640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a80af080 con 0x7fc0a815c790 2024-09-19T08:38:48.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.467+0000 7fc0ae5af640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a80aef00 con 0x7fc0a80751b0 2024-09-19T08:38:48.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.467+0000 7fc0ae5af640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc0a80af200 con 0x7fc0a806ce10 2024-09-19T08:38:48.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.467+0000 7fc0acb4c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0a815c790 0x7fc0a80b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:48.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.467+0000 7fc0acb4c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0a815c790 0x7fc0a80b0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40612/0 (socket says 172.21.15.98:40612) 2024-09-19T08:38:48.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.467+0000 7fc0acb4c640 1 -- 172.21.15.98:0/2440669660 learned_addr learned my addr 172.21.15.98:0/2440669660 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:48.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.467+0000 7fc0a77fe640 1 --2- 172.21.15.98:0/2440669660 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0a80751b0 0x7fc0a8158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:48.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.467+0000 7fc0acb4c640 1 -- 172.21.15.98:0/2440669660 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0a806ce10 msgr2=0x7fc0a80c7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:48.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.467+0000 7fc0acb4c640 1 --2- 172.21.15.98:0/2440669660 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0a806ce10 0x7fc0a80c7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:48.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.467+0000 7fc0acb4c640 1 -- 172.21.15.98:0/2440669660 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0a80751b0 msgr2=0x7fc0a8158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:48.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.467+0000 7fc0acb4c640 1 --2- 172.21.15.98:0/2440669660 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0a80751b0 0x7fc0a8158ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:48.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.467+0000 7fc0acb4c640 1 -- 172.21.15.98:0/2440669660 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a80b5d20 con 0x7fc0a815c790 2024-09-19T08:38:48.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.467+0000 7fc0acb4c640 1 --2- 172.21.15.98:0/2440669660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0a815c790 0x7fc0a80b0ed0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc0a00ab040 tx=0x7fc0a0019570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:48.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.467+0000 7fc0a4ff9640 1 -- 172.21.15.98:0/2440669660 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a00ac040 con 0x7fc0a815c790 2024-09-19T08:38:48.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.467+0000 7fc0a4ff9640 1 -- 172.21.15.98:0/2440669660 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0a00a7680 con 0x7fc0a815c790 2024-09-19T08:38:48.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.467+0000 7fc0ae5af640 1 -- 172.21.15.98:0/2440669660 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0a80c68b0 con 0x7fc0a815c790 2024-09-19T08:38:48.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.467+0000 7fc0a4ff9640 1 -- 172.21.15.98:0/2440669660 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a00a7a70 con 0x7fc0a815c790 2024-09-19T08:38:48.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.467+0000 7fc0ae5af640 1 -- 172.21.15.98:0/2440669660 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0a80c6e70 con 0x7fc0a815c790 2024-09-19T08:38:48.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.471+0000 7fc0ae5af640 1 -- 172.21.15.98:0/2440669660 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0a8074c70 con 0x7fc0a815c790 2024-09-19T08:38:48.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.471+0000 7fc0a4ff9640 1 -- 172.21.15.98:0/2440669660 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc0a00af020 con 0x7fc0a815c790 2024-09-19T08:38:48.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.471+0000 7fc0a4ff9640 1 --2- 172.21.15.98:0/2440669660 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc0880678c0 0x7fc088069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:48.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.471+0000 7fc0a7fff640 1 --2- 172.21.15.98:0/2440669660 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc0880678c0 0x7fc088069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:48.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.471+0000 7fc0a4ff9640 1 -- 172.21.15.98:0/2440669660 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc0a0120200 con 0x7fc0a815c790 2024-09-19T08:38:48.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.471+0000 7fc0a7fff640 1 --2- 172.21.15.98:0/2440669660 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc0880678c0 0x7fc088069d80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc098007fd0 tx=0x7fc0980079a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:48.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.475+0000 7fc0a4ff9640 1 -- 172.21.15.98:0/2440669660 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc0a00c1060 con 0x7fc0a815c790 2024-09-19T08:38:48.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.635+0000 7fc0ae5af640 1 -- 172.21.15.98:0/2440669660 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc0a8072c70 con 0x7fc0a815c790 2024-09-19T08:38:48.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.635+0000 7fc0a4ff9640 1 -- 172.21.15.98:0/2440669660 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7fc0a806edb0 con 0x7fc0a815c790 2024-09-19T08:38:48.640 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 64 2024-09-19T08:38:48.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.639+0000 7fc0ae5af640 1 -- 172.21.15.98:0/2440669660 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc0880678c0 msgr2=0x7fc088069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:48.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.639+0000 7fc0ae5af640 1 --2- 172.21.15.98:0/2440669660 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc0880678c0 0x7fc088069d80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc098007fd0 tx=0x7fc0980079a0 comp rx=0 tx=0).stop 2024-09-19T08:38:48.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.639+0000 7fc0ae5af640 1 -- 172.21.15.98:0/2440669660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0a815c790 msgr2=0x7fc0a80b0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:48.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.639+0000 7fc0ae5af640 1 --2- 172.21.15.98:0/2440669660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0a815c790 0x7fc0a80b0ed0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc0a00ab040 tx=0x7fc0a0019570 comp rx=0 tx=0).stop 2024-09-19T08:38:48.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.639+0000 7fc0ae5af640 1 -- 172.21.15.98:0/2440669660 shutdown_connections 2024-09-19T08:38:48.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.639+0000 7fc0ae5af640 1 --2- 172.21.15.98:0/2440669660 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc0880678c0 0x7fc088069d80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:48.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.639+0000 7fc0ae5af640 1 --2- 172.21.15.98:0/2440669660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0a815c790 0x7fc0a80b0ed0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:48.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.639+0000 7fc0ae5af640 1 --2- 172.21.15.98:0/2440669660 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0a80751b0 0x7fc0a8158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:48.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.639+0000 7fc0ae5af640 1 --2- 172.21.15.98:0/2440669660 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0a806ce10 0x7fc0a80c7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:48.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.639+0000 7fc0ae5af640 1 -- 172.21.15.98:0/2440669660 >> 172.21.15.98:0/2440669660 conn(0x7fc0a80d1250 msgr2=0x7fc0a815f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:48.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.639+0000 7fc0ae5af640 1 -- 172.21.15.98:0/2440669660 shutdown_connections 2024-09-19T08:38:48.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.639+0000 7fc0ae5af640 1 -- 172.21.15.98:0/2440669660 wait complete. 2024-09-19T08:38:48.660 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2024-09-19T08:38:48.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.775+0000 7faad509c640 1 Processor -- start 2024-09-19T08:38:48.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.775+0000 7faad509c640 1 -- start start 2024-09-19T08:38:48.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.775+0000 7faad509c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7faad00af710 0x7faad00cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:48.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.775+0000 7faad509c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faad0163af0 0x7faad00afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:48.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.775+0000 7faad509c640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faad00b0030 con 0x7faad00cf240 2024-09-19T08:38:48.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.775+0000 7faad509c640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7faad00cca20 con 0x7faad00af710 2024-09-19T08:38:48.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.775+0000 7faad509c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7faad00cedb0 con 0x7faad0163af0 2024-09-19T08:38:48.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.775+0000 7faaced9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7faad00af710 0x7faad00cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:48.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.775+0000 7faace59c640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7faad00cf240 0x7faad00cfde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:35908/0 (socket says 172.21.15.98:35908) 2024-09-19T08:38:48.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.775+0000 7faace59c640 1 -- 172.21.15.98:0/1510125823 learned_addr learned my addr 172.21.15.98:0/1510125823 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:48.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.775+0000 7faacdd9b640 1 -- 172.21.15.98:0/1510125823 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 155335498 0 0) 0x7faad00b0030 con 0x7faad00cf240 2024-09-19T08:38:48.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.775+0000 7faacf59e640 1 --2- 172.21.15.98:0/1510125823 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faad0163af0 0x7faad00afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:48.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.775+0000 7faacdd9b640 1 -- 172.21.15.98:0/1510125823 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faabc003530 con 0x7faad00cf240 2024-09-19T08:38:48.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.775+0000 7faaced9d640 1 -- 172.21.15.98:0/1510125823 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faad0163af0 msgr2=0x7faad00afaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:48.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.775+0000 7faaced9d640 1 --2- 172.21.15.98:0/1510125823 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faad0163af0 0x7faad00afaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:48.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.775+0000 7faaced9d640 1 -- 172.21.15.98:0/1510125823 >> v1:172.21.15.98:6789/0 conn(0x7faad00cf240 legacy=0x7faad00cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:48.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.775+0000 7faaced9d640 1 -- 172.21.15.98:0/1510125823 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad00d05d0 con 0x7faad00af710 2024-09-19T08:38:48.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.775+0000 7faacf59e640 1 --2- 172.21.15.98:0/1510125823 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faad0163af0 0x7faad00afaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:48.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.779+0000 7faaced9d640 1 --2- 172.21.15.98:0/1510125823 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faad00af710 0x7faad00cceb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7faac0009d90 tx=0x7faac001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa5b59579d23ae61 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:48.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.779+0000 7faacdd9b640 1 -- 172.21.15.98:0/1510125823 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac00b2070 con 0x7faad00af710 2024-09-19T08:38:48.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.779+0000 7faacdd9b640 1 -- 172.21.15.98:0/1510125823 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faac00a5830 con 0x7faad00af710 2024-09-19T08:38:48.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.779+0000 7faacdd9b640 1 -- 172.21.15.98:0/1510125823 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac00a5c00 con 0x7faad00af710 2024-09-19T08:38:48.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.779+0000 7faad509c640 1 -- 172.21.15.98:0/1510125823 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faad00af710 msgr2=0x7faad00cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:48.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.779+0000 7faad509c640 1 --2- 172.21.15.98:0/1510125823 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faad00af710 0x7faad00cceb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7faac0009d90 tx=0x7faac001c970 comp rx=0 tx=0).stop 2024-09-19T08:38:48.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.779+0000 7faad509c640 1 -- 172.21.15.98:0/1510125823 shutdown_connections 2024-09-19T08:38:48.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.779+0000 7faad509c640 1 --2- 172.21.15.98:0/1510125823 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faad0163af0 0x7faad00afaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:48.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.779+0000 7faad509c640 1 --2- 172.21.15.98:0/1510125823 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faad00af710 0x7faad00cceb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:48.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.779+0000 7faad509c640 1 -- 172.21.15.98:0/1510125823 >> 172.21.15.98:0/1510125823 conn(0x7faad00d0980 msgr2=0x7faad00d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:48.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.779+0000 7faad509c640 1 -- 172.21.15.98:0/1510125823 shutdown_connections 2024-09-19T08:38:48.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.779+0000 7faad509c640 1 -- 172.21.15.98:0/1510125823 wait complete. 2024-09-19T08:38:48.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.779+0000 7faad509c640 1 Processor -- start 2024-09-19T08:38:48.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.779+0000 7faad509c640 1 -- start start 2024-09-19T08:38:48.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.783+0000 7faad509c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faad00af710 0x7faad015ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:48.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.783+0000 7faad509c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faad00cf240 0x7faad00c7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:48.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.783+0000 7faad509c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faad0163af0 0x7faad01fb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:48.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.783+0000 7faad509c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faad00c4140 con 0x7faad00af710 2024-09-19T08:38:48.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.783+0000 7faad509c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faad00c3fc0 con 0x7faad0163af0 2024-09-19T08:38:48.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.783+0000 7faad509c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7faad00c42c0 con 0x7faad00cf240 2024-09-19T08:38:48.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.783+0000 7faaced9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faad00af710 0x7faad015ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:48.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.783+0000 7faaced9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faad00af710 0x7faad015ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40628/0 (socket says 172.21.15.98:40628) 2024-09-19T08:38:48.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.783+0000 7faaced9d640 1 -- 172.21.15.98:0/3669780876 learned_addr learned my addr 172.21.15.98:0/3669780876 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:48.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.783+0000 7faace59c640 1 --2- 172.21.15.98:0/3669780876 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faad00cf240 0x7faad00c7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:48.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.783+0000 7faaced9d640 1 -- 172.21.15.98:0/3669780876 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faad00cf240 msgr2=0x7faad00c7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:48.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.783+0000 7faaced9d640 1 --2- 172.21.15.98:0/3669780876 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faad00cf240 0x7faad00c7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:48.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.783+0000 7faacf59e640 1 --2- 172.21.15.98:0/3669780876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faad0163af0 0x7faad01fb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:48.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.783+0000 7faaced9d640 1 -- 172.21.15.98:0/3669780876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faad0163af0 msgr2=0x7faad01fb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:48.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.783+0000 7faaced9d640 1 --2- 172.21.15.98:0/3669780876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faad0163af0 0x7faad01fb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:48.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.783+0000 7faaced9d640 1 -- 172.21.15.98:0/3669780876 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad015efc0 con 0x7faad00af710 2024-09-19T08:38:48.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.783+0000 7faaced9d640 1 --2- 172.21.15.98:0/3669780876 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faad00af710 0x7faad015ea80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7faac001a5a0 tx=0x7faac0019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:48.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.783+0000 7faac77fe640 1 -- 172.21.15.98:0/3669780876 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac00b2050 con 0x7faad00af710 2024-09-19T08:38:48.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.783+0000 7faac77fe640 1 -- 172.21.15.98:0/3669780876 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faac00be070 con 0x7faad00af710 2024-09-19T08:38:48.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.783+0000 7faad509c640 1 -- 172.21.15.98:0/3669780876 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faad00c7560 con 0x7faad00af710 2024-09-19T08:38:48.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.783+0000 7faac77fe640 1 -- 172.21.15.98:0/3669780876 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac00ad850 con 0x7faad00af710 2024-09-19T08:38:48.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.783+0000 7faad509c640 1 -- 172.21.15.98:0/3669780876 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faad00b5070 con 0x7faad00af710 2024-09-19T08:38:48.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.783+0000 7faad509c640 1 -- 172.21.15.98:0/3669780876 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faad00afd60 con 0x7faad00af710 2024-09-19T08:38:48.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.783+0000 7faac77fe640 1 -- 172.21.15.98:0/3669780876 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7faac00ad9f0 con 0x7faad00af710 2024-09-19T08:38:48.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.783+0000 7faac77fe640 1 --2- 172.21.15.98:0/3669780876 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7faab00678f0 0x7faab0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:48.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.783+0000 7faac77fe640 1 -- 172.21.15.98:0/3669780876 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7faac01208e0 con 0x7faad00af710 2024-09-19T08:38:48.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.783+0000 7faace59c640 1 --2- 172.21.15.98:0/3669780876 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7faab00678f0 0x7faab0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:48.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.783+0000 7faace59c640 1 --2- 172.21.15.98:0/3669780876 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7faab00678f0 0x7faab0069db0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7faab8007fd0 tx=0x7faab8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:48.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:48.787+0000 7faac77fe640 1 -- 172.21.15.98:0/3669780876 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7faac00c7020 con 0x7faad00af710 2024-09-19T08:38:49.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.015+0000 7faad509c640 1 -- 172.21.15.98:0/3669780876 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7faad01fb860 con 0x7faad00af710 2024-09-19T08:38:49.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.099+0000 7faac77fe640 1 -- 172.21.15.98:0/3669780876 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v48) ==== 161+0+0 (secure 0 0 0) 0x7faad015efc0 con 0x7faad00af710 2024-09-19T08:38:49.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.103+0000 7faad509c640 1 -- 172.21.15.98:0/3669780876 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7faab00678f0 msgr2=0x7faab0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:49.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.103+0000 7faad509c640 1 --2- 172.21.15.98:0/3669780876 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7faab00678f0 0x7faab0069db0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7faab8007fd0 tx=0x7faab8009040 comp rx=0 tx=0).stop 2024-09-19T08:38:49.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.103+0000 7faad509c640 1 -- 172.21.15.98:0/3669780876 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faad00af710 msgr2=0x7faad015ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:49.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.103+0000 7faad509c640 1 --2- 172.21.15.98:0/3669780876 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faad00af710 0x7faad015ea80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7faac001a5a0 tx=0x7faac0019710 comp rx=0 tx=0).stop 2024-09-19T08:38:49.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.103+0000 7faad509c640 1 -- 172.21.15.98:0/3669780876 shutdown_connections 2024-09-19T08:38:49.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.103+0000 7faad509c640 1 --2- 172.21.15.98:0/3669780876 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7faab00678f0 0x7faab0069db0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:49.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.103+0000 7faad509c640 1 --2- 172.21.15.98:0/3669780876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faad0163af0 0x7faad01fb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:49.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.103+0000 7faad509c640 1 --2- 172.21.15.98:0/3669780876 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faad00cf240 0x7faad00c7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:49.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.103+0000 7faad509c640 1 --2- 172.21.15.98:0/3669780876 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faad00af710 0x7faad015ea80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:49.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.103+0000 7faad509c640 1 -- 172.21.15.98:0/3669780876 >> 172.21.15.98:0/3669780876 conn(0x7faad00d0980 msgr2=0x7faad0165820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:49.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.103+0000 7faad509c640 1 -- 172.21.15.98:0/3669780876 shutdown_connections 2024-09-19T08:38:49.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.103+0000 7faad509c640 1 -- 172.21.15.98:0/3669780876 wait complete. 2024-09-19T08:38:49.127 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T08:38:49.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.251+0000 7f3d1eac3640 1 Processor -- start 2024-09-19T08:38:49.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.251+0000 7f3d1eac3640 1 -- start start 2024-09-19T08:38:49.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.251+0000 7f3d1eac3640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d180d27c0 0x7f3d180d3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:49.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.251+0000 7f3d1eac3640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d1815eae0 con 0x7f3d180d38a0 2024-09-19T08:38:49.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.251+0000 7f3d1eac3640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d1815edc0 con 0x7f3d181415a0 2024-09-19T08:38:49.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.251+0000 7f3d1eac3640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3d1815f0a0 con 0x7f3d180d27c0 2024-09-19T08:38:49.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.251+0000 7f3d1d060640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f3d180d38a0 0x7f3d180b8790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:35920/0 (socket says 172.21.15.98:35920) 2024-09-19T08:38:49.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.251+0000 7f3d1d060640 1 -- 172.21.15.98:0/1570481400 learned_addr learned my addr 172.21.15.98:0/1570481400 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:49.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.251+0000 7f3d17fff640 1 --2- 172.21.15.98:0/1570481400 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d180d27c0 0x7f3d180d3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:49.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.251+0000 7f3d177fe640 1 -- 172.21.15.98:0/1570481400 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1617030665 0 0) 0x7f3d1815eae0 con 0x7f3d180d38a0 2024-09-19T08:38:49.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.251+0000 7f3d177fe640 1 -- 172.21.15.98:0/1570481400 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d08003530 con 0x7f3d180d38a0 2024-09-19T08:38:49.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.255+0000 7f3d177fe640 1 -- 172.21.15.98:0/1570481400 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3773019462 0 0) 0x7f3d08003530 con 0x7f3d180d38a0 2024-09-19T08:38:49.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.255+0000 7f3d177fe640 1 -- 172.21.15.98:0/1570481400 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d180d27c0 msgr2=0x7f3d180d3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:49.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.255+0000 7f3d177fe640 1 --2- 172.21.15.98:0/1570481400 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d180d27c0 0x7f3d180d3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:49.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.255+0000 7f3d177fe640 1 -- 172.21.15.98:0/1570481400 >> v1:172.21.15.139:6789/0 conn(0x7f3d181415a0 legacy=0x7f3d180d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:49.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.255+0000 7f3d177fe640 1 -- 172.21.15.98:0/1570481400 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d1815f380 con 0x7f3d180d38a0 2024-09-19T08:38:49.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.255+0000 7f3d177fe640 1 -- 172.21.15.98:0/1570481400 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f3d10001a00 con 0x7f3d180d38a0 2024-09-19T08:38:49.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.255+0000 7f3d177fe640 1 -- 172.21.15.98:0/1570481400 >> v1:172.21.15.98:6789/0 conn(0x7f3d180d38a0 legacy=0x7f3d180b8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:49.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.255+0000 7f3d177fe640 1 --2- 172.21.15.98:0/1570481400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d080039f0 0x7f3d08003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:49.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.255+0000 7f3d177fe640 1 --2- 172.21.15.98:0/1570481400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d08004810 0x7f3d08004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:49.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.255+0000 7f3d177fe640 1 --2- 172.21.15.98:0/1570481400 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d180d27c0 0x7f3d080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:49.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.255+0000 7f3d177fe640 1 -- 172.21.15.98:0/1570481400 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1815f0a0 con 0x7f3d08004810 2024-09-19T08:38:49.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.255+0000 7f3d177fe640 1 -- 172.21.15.98:0/1570481400 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0800ba80 con 0x7f3d080039f0 2024-09-19T08:38:49.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.255+0000 7f3d177fe640 1 -- 172.21.15.98:0/1570481400 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3d0800bc80 con 0x7f3d180d27c0 2024-09-19T08:38:49.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.255+0000 7f3d17fff640 1 --2- 172.21.15.98:0/1570481400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d08004810 0x7f3d08004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:49.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.255+0000 7f3d1c85f640 1 --2- 172.21.15.98:0/1570481400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d080039f0 0x7f3d08003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:49.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.255+0000 7f3d1d060640 1 --2- 172.21.15.98:0/1570481400 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d180d27c0 0x7f3d080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:49.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.255+0000 7f3d17fff640 1 -- 172.21.15.98:0/1570481400 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d180d27c0 msgr2=0x7f3d080051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:49.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.255+0000 7f3d17fff640 1 --2- 172.21.15.98:0/1570481400 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d180d27c0 0x7f3d080051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:49.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.255+0000 7f3d17fff640 1 -- 172.21.15.98:0/1570481400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d080039f0 msgr2=0x7f3d08003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:49.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.255+0000 7f3d17fff640 1 --2- 172.21.15.98:0/1570481400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d080039f0 0x7f3d08003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:49.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.255+0000 7f3d17fff640 1 -- 172.21.15.98:0/1570481400 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d08003530 con 0x7f3d08004810 2024-09-19T08:38:49.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.255+0000 7f3d17fff640 1 --2- 172.21.15.98:0/1570481400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d08004810 0x7f3d08004c90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3d04009ab0 tx=0x7f3d0401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d466173723674541 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:49.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.255+0000 7f3d177fe640 1 -- 172.21.15.98:0/1570481400 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d040b2070 con 0x7f3d08004810 2024-09-19T08:38:49.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.255+0000 7f3d177fe640 1 -- 172.21.15.98:0/1570481400 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d040a5510 con 0x7f3d08004810 2024-09-19T08:38:49.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.255+0000 7f3d177fe640 1 -- 172.21.15.98:0/1570481400 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d040096b0 con 0x7f3d08004810 2024-09-19T08:38:49.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.255+0000 7f3d1eac3640 1 -- 172.21.15.98:0/1570481400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d08004810 msgr2=0x7f3d08004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:49.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.255+0000 7f3d1eac3640 1 --2- 172.21.15.98:0/1570481400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d08004810 0x7f3d08004c90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3d04009ab0 tx=0x7f3d0401ca30 comp rx=0 tx=0).stop 2024-09-19T08:38:49.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.255+0000 7f3d1d060640 1 -- 172.21.15.98:0/1570481400 reap_dead start 2024-09-19T08:38:49.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.255+0000 7f3d1eac3640 1 -- 172.21.15.98:0/1570481400 shutdown_connections 2024-09-19T08:38:49.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.255+0000 7f3d1eac3640 1 -- 172.21.15.98:0/1570481400 >> 172.21.15.98:0/1570481400 conn(0x7f3d18074c70 msgr2=0x7f3d18075080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:49.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.255+0000 7f3d1eac3640 1 -- 172.21.15.98:0/1570481400 shutdown_connections 2024-09-19T08:38:49.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.255+0000 7f3d1eac3640 1 -- 172.21.15.98:0/1570481400 wait complete. 2024-09-19T08:38:49.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.259+0000 7f3d1eac3640 1 Processor -- start 2024-09-19T08:38:49.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.259+0000 7f3d1eac3640 1 -- start start 2024-09-19T08:38:49.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.259+0000 7f3d1eac3640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d08004810 0x7f3d181549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:49.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.259+0000 7f3d1eac3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d18140180 0x7f3d18161f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:49.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.259+0000 7f3d1eac3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d181604f0 0x7f3d18162460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:49.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.259+0000 7f3d1eac3640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3d180d3ed0 con 0x7f3d181604f0 2024-09-19T08:38:49.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.259+0000 7f3d1eac3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3d180d3d50 con 0x7f3d18140180 2024-09-19T08:38:49.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.259+0000 7f3d1eac3640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3d180d4120 con 0x7f3d08004810 2024-09-19T08:38:49.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.259+0000 7f3d1d060640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d181604f0 0x7f3d18162460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:49.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.259+0000 7f3d1d060640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d181604f0 0x7f3d18162460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40642/0 (socket says 172.21.15.98:40642) 2024-09-19T08:38:49.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.259+0000 7f3d1d060640 1 -- 172.21.15.98:0/1963287595 learned_addr learned my addr 172.21.15.98:0/1963287595 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:49.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.259+0000 7f3d17fff640 1 --2- 172.21.15.98:0/1963287595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d18140180 0x7f3d18161f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:49.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.259+0000 7f3d1c85f640 1 --2- 172.21.15.98:0/1963287595 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d08004810 0x7f3d181549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:49.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.259+0000 7f3d1d060640 1 -- 172.21.15.98:0/1963287595 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d08004810 msgr2=0x7f3d181549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:49.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.259+0000 7f3d1d060640 1 --2- 172.21.15.98:0/1963287595 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d08004810 0x7f3d181549e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:49.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.259+0000 7f3d1d060640 1 -- 172.21.15.98:0/1963287595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d18140180 msgr2=0x7f3d18161f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:49.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.259+0000 7f3d1d060640 1 --2- 172.21.15.98:0/1963287595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d18140180 0x7f3d18161f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:49.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.259+0000 7f3d1d060640 1 -- 172.21.15.98:0/1963287595 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d18160ab0 con 0x7f3d181604f0 2024-09-19T08:38:49.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.259+0000 7f3d1d060640 1 --2- 172.21.15.98:0/1963287595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d181604f0 0x7f3d18162460 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3d180764b0 tx=0x7f3d10011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:49.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.259+0000 7f3d157fa640 1 -- 172.21.15.98:0/1963287595 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d10010040 con 0x7f3d181604f0 2024-09-19T08:38:49.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.259+0000 7f3d157fa640 1 -- 172.21.15.98:0/1963287595 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d1001a070 con 0x7f3d181604f0 2024-09-19T08:38:49.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.259+0000 7f3d1eac3640 1 -- 172.21.15.98:0/1963287595 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d18142730 con 0x7f3d181604f0 2024-09-19T08:38:49.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.259+0000 7f3d157fa640 1 -- 172.21.15.98:0/1963287595 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1000fb60 con 0x7f3d181604f0 2024-09-19T08:38:49.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.259+0000 7f3d1eac3640 1 -- 172.21.15.98:0/1963287595 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d18142cf0 con 0x7f3d181604f0 2024-09-19T08:38:49.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.259+0000 7f3d1eac3640 1 -- 172.21.15.98:0/1963287595 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d180d3360 con 0x7f3d181604f0 2024-09-19T08:38:49.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.263+0000 7f3d157fa640 1 -- 172.21.15.98:0/1963287595 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3d1000fd00 con 0x7f3d181604f0 2024-09-19T08:38:49.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.263+0000 7f3d157fa640 1 --2- 172.21.15.98:0/1963287595 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3cfc0678c0 0x7f3cfc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:49.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.263+0000 7f3d157fa640 1 -- 172.21.15.98:0/1963287595 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3d10088ca0 con 0x7f3d181604f0 2024-09-19T08:38:49.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.263+0000 7f3d1c85f640 1 --2- 172.21.15.98:0/1963287595 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3cfc0678c0 0x7f3cfc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:49.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.263+0000 7f3d1c85f640 1 --2- 172.21.15.98:0/1963287595 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3cfc0678c0 0x7f3cfc069d80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c001620 tx=0x7f3d0c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:49.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.267+0000 7f3d157fa640 1 -- 172.21.15.98:0/1963287595 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3d100597e0 con 0x7f3d181604f0 2024-09-19T08:38:49.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.427+0000 7f3d1eac3640 1 -- 172.21.15.98:0/1963287595 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f3d1806caf0 con 0x7f3d181604f0 2024-09-19T08:38:49.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.435+0000 7f3d157fa640 1 -- 172.21.15.98:0/1963287595 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f3d100590e0 con 0x7f3d181604f0 2024-09-19T08:38:49.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.435+0000 7f3d1eac3640 1 -- 172.21.15.98:0/1963287595 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3cfc0678c0 msgr2=0x7f3cfc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:49.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.435+0000 7f3d1eac3640 1 --2- 172.21.15.98:0/1963287595 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3cfc0678c0 0x7f3cfc069d80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c001620 tx=0x7f3d0c009040 comp rx=0 tx=0).stop 2024-09-19T08:38:49.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.435+0000 7f3d1eac3640 1 -- 172.21.15.98:0/1963287595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d181604f0 msgr2=0x7f3d18162460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:49.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.435+0000 7f3d1eac3640 1 --2- 172.21.15.98:0/1963287595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d181604f0 0x7f3d18162460 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3d180764b0 tx=0x7f3d10011c20 comp rx=0 tx=0).stop 2024-09-19T08:38:49.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.435+0000 7f3d1eac3640 1 -- 172.21.15.98:0/1963287595 shutdown_connections 2024-09-19T08:38:49.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.435+0000 7f3d1eac3640 1 --2- 172.21.15.98:0/1963287595 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3cfc0678c0 0x7f3cfc069d80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:49.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.435+0000 7f3d1eac3640 1 --2- 172.21.15.98:0/1963287595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d181604f0 0x7f3d18162460 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:49.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.435+0000 7f3d1eac3640 1 --2- 172.21.15.98:0/1963287595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d18140180 0x7f3d18161f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:49.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.435+0000 7f3d1eac3640 1 --2- 172.21.15.98:0/1963287595 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d08004810 0x7f3d181549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:49.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.435+0000 7f3d1eac3640 1 -- 172.21.15.98:0/1963287595 >> 172.21.15.98:0/1963287595 conn(0x7f3d18074c70 msgr2=0x7f3d1806c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:49.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.435+0000 7f3d1eac3640 1 -- 172.21.15.98:0/1963287595 shutdown_connections 2024-09-19T08:38:49.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.435+0000 7f3d1eac3640 1 -- 172.21.15.98:0/1963287595 wait complete. 2024-09-19T08:38:49.458 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T08:38:49.458 DEBUG:teuthology.orchestra.run.smithi098:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:38:49.542 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:38:49.629 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T08:38:49.629 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:38:49.629 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:38:49.637 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:38:49.637 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:38:49.686 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T08:38:49.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.851+0000 7f1f928b4640 1 Processor -- start 2024-09-19T08:38:49.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.851+0000 7f1f928b4640 1 -- start start 2024-09-19T08:38:49.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.851+0000 7f1f928b4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f8c0733b0 0x7f1f8c0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:49.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.851+0000 7f1f928b4640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f8c06d310 con 0x7f1f8c15c550 2024-09-19T08:38:49.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.851+0000 7f1f928b4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f8c06d5f0 con 0x7f1f8c06c3a0 2024-09-19T08:38:49.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.851+0000 7f1f928b4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1f8c06d8d0 con 0x7f1f8c0733b0 2024-09-19T08:38:49.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.851+0000 7f1f8bfff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f1f8c15c550 0x7f1f8c06bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:35922/0 (socket says 172.21.15.98:35922) 2024-09-19T08:38:49.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.851+0000 7f1f8bfff640 1 -- 172.21.15.98:0/1212019369 learned_addr learned my addr 172.21.15.98:0/1212019369 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:49.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.851+0000 7f1f8affd640 1 -- 172.21.15.98:0/1212019369 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3706177130 0 0) 0x7f1f8c06d310 con 0x7f1f8c15c550 2024-09-19T08:38:49.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.851+0000 7f1f8affd640 1 -- 172.21.15.98:0/1212019369 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f74003530 con 0x7f1f8c15c550 2024-09-19T08:38:49.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.851+0000 7f1f90e51640 1 --2- 172.21.15.98:0/1212019369 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f8c0733b0 0x7f1f8c0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:49.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.851+0000 7f1f8affd640 1 -- 172.21.15.98:0/1212019369 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2315955076 0 0) 0x7f1f74003530 con 0x7f1f8c15c550 2024-09-19T08:38:49.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.851+0000 7f1f8affd640 1 -- 172.21.15.98:0/1212019369 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f8c0733b0 msgr2=0x7f1f8c0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:49.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.851+0000 7f1f8affd640 1 --2- 172.21.15.98:0/1212019369 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f8c0733b0 0x7f1f8c0cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:49.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.851+0000 7f1f8affd640 1 -- 172.21.15.98:0/1212019369 >> v1:172.21.15.139:6789/0 conn(0x7f1f8c06c3a0 legacy=0x7f1f8c0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:49.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.851+0000 7f1f8affd640 1 -- 172.21.15.98:0/1212019369 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8c06dbb0 con 0x7f1f8c15c550 2024-09-19T08:38:49.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.851+0000 7f1f8affd640 1 -- 172.21.15.98:0/1212019369 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f1f84001680 con 0x7f1f8c15c550 2024-09-19T08:38:49.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.851+0000 7f1f8affd640 1 -- 172.21.15.98:0/1212019369 >> v1:172.21.15.98:6789/0 conn(0x7f1f8c15c550 legacy=0x7f1f8c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:49.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.851+0000 7f1f8affd640 1 --2- 172.21.15.98:0/1212019369 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f8c0733b0 0x7f1f740039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:49.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.851+0000 7f1f8affd640 1 --2- 172.21.15.98:0/1212019369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f74004020 0x7f1f74004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:49.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.851+0000 7f1f8affd640 1 --2- 172.21.15.98:0/1212019369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f74004d50 0x7f1f740051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:49.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.851+0000 7f1f8affd640 1 -- 172.21.15.98:0/1212019369 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c06d8d0 con 0x7f1f74004d50 2024-09-19T08:38:49.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.851+0000 7f1f8affd640 1 -- 172.21.15.98:0/1212019369 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7400bb90 con 0x7f1f74004020 2024-09-19T08:38:49.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.851+0000 7f1f8affd640 1 -- 172.21.15.98:0/1212019369 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1f7400bd90 con 0x7f1f8c0733b0 2024-09-19T08:38:49.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.855+0000 7f1f8b7fe640 1 --2- 172.21.15.98:0/1212019369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f74004020 0x7f1f74004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:49.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.855+0000 7f1f90e51640 1 --2- 172.21.15.98:0/1212019369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f74004d50 0x7f1f740051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:49.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.855+0000 7f1f8bfff640 1 --2- 172.21.15.98:0/1212019369 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f8c0733b0 0x7f1f740039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:49.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.855+0000 7f1f8b7fe640 1 -- 172.21.15.98:0/1212019369 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f8c0733b0 msgr2=0x7f1f740039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:49.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.855+0000 7f1f8b7fe640 1 --2- 172.21.15.98:0/1212019369 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f8c0733b0 0x7f1f740039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:49.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.855+0000 7f1f8b7fe640 1 -- 172.21.15.98:0/1212019369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f74004d50 msgr2=0x7f1f740051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:49.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.855+0000 7f1f8b7fe640 1 --2- 172.21.15.98:0/1212019369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f74004d50 0x7f1f740051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:49.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.855+0000 7f1f8b7fe640 1 -- 172.21.15.98:0/1212019369 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f74003530 con 0x7f1f74004020 2024-09-19T08:38:49.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.855+0000 7f1f8b7fe640 1 --2- 172.21.15.98:0/1212019369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f74004020 0x7f1f74004810 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1f80001780 tx=0x7f1f8001c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c2a5ad950bbedea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:49.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.855+0000 7f1f8affd640 1 -- 172.21.15.98:0/1212019369 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f800013a0 con 0x7f1f74004020 2024-09-19T08:38:49.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.855+0000 7f1f8affd640 1 -- 172.21.15.98:0/1212019369 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f800a9070 con 0x7f1f74004020 2024-09-19T08:38:49.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.855+0000 7f1f8affd640 1 -- 172.21.15.98:0/1212019369 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f80003600 con 0x7f1f74004020 2024-09-19T08:38:49.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.855+0000 7f1f928b4640 1 -- 172.21.15.98:0/1212019369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f74004020 msgr2=0x7f1f74004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:49.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.855+0000 7f1f928b4640 1 --2- 172.21.15.98:0/1212019369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f74004020 0x7f1f74004810 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1f80001780 tx=0x7f1f8001c420 comp rx=0 tx=0).stop 2024-09-19T08:38:49.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.855+0000 7f1f90e51640 1 -- 172.21.15.98:0/1212019369 reap_dead start 2024-09-19T08:38:49.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.855+0000 7f1f928b4640 1 -- 172.21.15.98:0/1212019369 shutdown_connections 2024-09-19T08:38:49.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.855+0000 7f1f928b4640 1 -- 172.21.15.98:0/1212019369 >> 172.21.15.98:0/1212019369 conn(0x7f1f8c0b2010 msgr2=0x7f1f8c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:49.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.855+0000 7f1f928b4640 1 -- 172.21.15.98:0/1212019369 shutdown_connections 2024-09-19T08:38:49.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.859+0000 7f1f928b4640 1 -- 172.21.15.98:0/1212019369 wait complete. 2024-09-19T08:38:49.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.859+0000 7f1f928b4640 1 Processor -- start 2024-09-19T08:38:49.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.859+0000 7f1f928b4640 1 -- start start 2024-09-19T08:38:49.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.859+0000 7f1f928b4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f74004020 0x7f1f8c14fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:49.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.859+0000 7f1f928b4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f8c15cb40 0x7f1f8c150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:49.865 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.859+0000 7f1f928b4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f8c152b40 0x7f1f8c151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:49.865 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.859+0000 7f1f928b4640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c0cfc50 con 0x7f1f8c15cb40 2024-09-19T08:38:49.865 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.859+0000 7f1f928b4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c0cfad0 con 0x7f1f8c152b40 2024-09-19T08:38:49.865 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.859+0000 7f1f928b4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1f8c0cfdd0 con 0x7f1f74004020 2024-09-19T08:38:49.865 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.859+0000 7f1f8b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f8c15cb40 0x7f1f8c150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:49.865 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.859+0000 7f1f8b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f8c15cb40 0x7f1f8c150b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40650/0 (socket says 172.21.15.98:40650) 2024-09-19T08:38:49.865 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.859+0000 7f1f8b7fe640 1 -- 172.21.15.98:0/1255200236 learned_addr learned my addr 172.21.15.98:0/1255200236 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:49.865 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.859+0000 7f1f8bfff640 1 --2- 172.21.15.98:0/1255200236 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f74004020 0x7f1f8c14fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:49.865 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.859+0000 7f1f90e51640 1 --2- 172.21.15.98:0/1255200236 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f8c152b40 0x7f1f8c151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:49.865 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.859+0000 7f1f8b7fe640 1 -- 172.21.15.98:0/1255200236 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f74004020 msgr2=0x7f1f8c14fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:49.865 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.859+0000 7f1f8b7fe640 1 --2- 172.21.15.98:0/1255200236 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f74004020 0x7f1f8c14fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:49.865 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.859+0000 7f1f8b7fe640 1 -- 172.21.15.98:0/1255200236 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f8c152b40 msgr2=0x7f1f8c151040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:49.865 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.859+0000 7f1f8b7fe640 1 --2- 172.21.15.98:0/1255200236 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f8c152b40 0x7f1f8c151040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:49.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.859+0000 7f1f8b7fe640 1 -- 172.21.15.98:0/1255200236 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8c1517d0 con 0x7f1f8c15cb40 2024-09-19T08:38:49.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.859+0000 7f1f90e51640 1 --2- 172.21.15.98:0/1255200236 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f8c152b40 0x7f1f8c151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:49.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.859+0000 7f1f8b7fe640 1 --2- 172.21.15.98:0/1255200236 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f8c15cb40 0x7f1f8c150b00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1f80005040 tx=0x7f1f74005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:49.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.859+0000 7f1f88ff9640 1 -- 172.21.15.98:0/1255200236 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f800b2040 con 0x7f1f8c15cb40 2024-09-19T08:38:49.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.859+0000 7f1f88ff9640 1 -- 172.21.15.98:0/1255200236 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f800bc070 con 0x7f1f8c15cb40 2024-09-19T08:38:49.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.859+0000 7f1f928b4640 1 -- 172.21.15.98:0/1255200236 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f8c151ab0 con 0x7f1f8c15cb40 2024-09-19T08:38:49.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.859+0000 7f1f88ff9640 1 -- 172.21.15.98:0/1255200236 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f80007a40 con 0x7f1f8c15cb40 2024-09-19T08:38:49.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.859+0000 7f1f928b4640 1 -- 172.21.15.98:0/1255200236 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f8c150110 con 0x7f1f8c15cb40 2024-09-19T08:38:49.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.859+0000 7f1f928b4640 1 -- 172.21.15.98:0/1255200236 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f8c0cf3c0 con 0x7f1f8c15cb40 2024-09-19T08:38:49.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.863+0000 7f1f88ff9640 1 -- 172.21.15.98:0/1255200236 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1f800ab070 con 0x7f1f8c15cb40 2024-09-19T08:38:49.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.863+0000 7f1f88ff9640 1 --2- 172.21.15.98:0/1255200236 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f1f780678c0 0x7f1f78069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:49.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.863+0000 7f1f88ff9640 1 -- 172.21.15.98:0/1255200236 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1f8011fda0 con 0x7f1f8c15cb40 2024-09-19T08:38:49.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.863+0000 7f1f8bfff640 1 --2- 172.21.15.98:0/1255200236 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f1f780678c0 0x7f1f78069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:49.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.863+0000 7f1f8bfff640 1 --2- 172.21.15.98:0/1255200236 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f1f780678c0 0x7f1f78069d80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1f84001000 tx=0x7f1f84009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:49.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:49.867+0000 7f1f88ff9640 1 -- 172.21.15.98:0/1255200236 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f1f80016130 con 0x7f1f8c15cb40 2024-09-19T08:38:50.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.043+0000 7f1f928b4640 1 -- 172.21.15.98:0/1255200236 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1f8c0cd650 con 0x7f1f8c15cb40 2024-09-19T08:38:50.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.043+0000 7f1f88ff9640 1 -- 172.21.15.98:0/1255200236 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f1f80007670 con 0x7f1f8c15cb40 2024-09-19T08:38:50.046 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-19T08:38:50.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.043+0000 7f1f928b4640 1 -- 172.21.15.98:0/1255200236 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f1f780678c0 msgr2=0x7f1f78069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:50.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.043+0000 7f1f928b4640 1 --2- 172.21.15.98:0/1255200236 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f1f780678c0 0x7f1f78069d80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1f84001000 tx=0x7f1f84009040 comp rx=0 tx=0).stop 2024-09-19T08:38:50.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.043+0000 7f1f928b4640 1 -- 172.21.15.98:0/1255200236 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f8c15cb40 msgr2=0x7f1f8c150b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:50.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.043+0000 7f1f928b4640 1 --2- 172.21.15.98:0/1255200236 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f8c15cb40 0x7f1f8c150b00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1f80005040 tx=0x7f1f74005840 comp rx=0 tx=0).stop 2024-09-19T08:38:50.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.043+0000 7f1f928b4640 1 -- 172.21.15.98:0/1255200236 shutdown_connections 2024-09-19T08:38:50.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.043+0000 7f1f928b4640 1 --2- 172.21.15.98:0/1255200236 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f1f780678c0 0x7f1f78069d80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:50.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.043+0000 7f1f928b4640 1 --2- 172.21.15.98:0/1255200236 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f8c152b40 0x7f1f8c151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:50.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.043+0000 7f1f928b4640 1 --2- 172.21.15.98:0/1255200236 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f8c15cb40 0x7f1f8c150b00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:50.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.043+0000 7f1f928b4640 1 --2- 172.21.15.98:0/1255200236 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f74004020 0x7f1f8c14fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:50.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.043+0000 7f1f928b4640 1 -- 172.21.15.98:0/1255200236 >> 172.21.15.98:0/1255200236 conn(0x7f1f8c0b2010 msgr2=0x7f1f8c1599c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:50.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.043+0000 7f1f928b4640 1 -- 172.21.15.98:0/1255200236 shutdown_connections 2024-09-19T08:38:50.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.043+0000 7f1f928b4640 1 -- 172.21.15.98:0/1255200236 wait complete. 2024-09-19T08:38:50.066 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:38:50.066 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:38:50.072 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T08:38:50.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.231+0000 7f95f8d9f640 1 Processor -- start 2024-09-19T08:38:50.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.231+0000 7f95f8d9f640 1 -- start start 2024-09-19T08:38:50.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.231+0000 7f95f8d9f640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f95f415c550 0x7f95f406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:50.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.231+0000 7f95f8d9f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f95f40cfad0 0x7f95f40723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:50.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.231+0000 7f95f8d9f640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95f406d310 con 0x7f95f406c1d0 2024-09-19T08:38:50.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.231+0000 7f95f8d9f640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f95f406c640 con 0x7f95f415c550 2024-09-19T08:38:50.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.231+0000 7f95f8d9f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f95f406d5f0 con 0x7f95f40cfad0 2024-09-19T08:38:50.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.231+0000 7f95f259c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f95f415c550 0x7f95f406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:50.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.231+0000 7f95f259c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f95f415c550 0x7f95f406bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40658/0 (socket says 172.21.15.98:40658) 2024-09-19T08:38:50.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.231+0000 7f95f1d9b640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f95f406c1d0 0x7f95f40cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:35938/0 (socket says 172.21.15.98:35938) 2024-09-19T08:38:50.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.231+0000 7f95f259c640 1 -- 172.21.15.98:0/3994030851 learned_addr learned my addr 172.21.15.98:0/3994030851 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:50.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.231+0000 7f95f2d9d640 1 --2- 172.21.15.98:0/3994030851 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f95f40cfad0 0x7f95f40723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:50.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.231+0000 7f95f259c640 1 -- 172.21.15.98:0/3994030851 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f95f40cfad0 msgr2=0x7f95f40723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:50.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.231+0000 7f95f259c640 1 --2- 172.21.15.98:0/3994030851 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f95f40cfad0 0x7f95f40723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:50.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.231+0000 7f95f259c640 1 -- 172.21.15.98:0/3994030851 >> v1:172.21.15.98:6789/0 conn(0x7f95f406c1d0 legacy=0x7f95f40cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:50.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.231+0000 7f95f259c640 1 -- 172.21.15.98:0/3994030851 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f406d8d0 con 0x7f95f415c550 2024-09-19T08:38:50.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.235+0000 7f95f259c640 1 --2- 172.21.15.98:0/3994030851 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95f415c550 0x7f95f406bc90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f95e4009870 tx=0x7f95e401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99059b631a24fc1f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:50.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.235+0000 7f95f159a640 1 -- 172.21.15.98:0/3994030851 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e40b2070 con 0x7f95f415c550 2024-09-19T08:38:50.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.235+0000 7f95f159a640 1 -- 172.21.15.98:0/3994030851 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95e40a9070 con 0x7f95f415c550 2024-09-19T08:38:50.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.235+0000 7f95f159a640 1 -- 172.21.15.98:0/3994030851 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e40ad880 con 0x7f95f415c550 2024-09-19T08:38:50.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.235+0000 7f95f8d9f640 1 -- 172.21.15.98:0/3994030851 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95f415c550 msgr2=0x7f95f406bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:50.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.235+0000 7f95f8d9f640 1 --2- 172.21.15.98:0/3994030851 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95f415c550 0x7f95f406bc90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f95e4009870 tx=0x7f95e401c560 comp rx=0 tx=0).stop 2024-09-19T08:38:50.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.235+0000 7f95f8d9f640 1 -- 172.21.15.98:0/3994030851 shutdown_connections 2024-09-19T08:38:50.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.235+0000 7f95f8d9f640 1 --2- 172.21.15.98:0/3994030851 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f95f40cfad0 0x7f95f40723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:50.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.235+0000 7f95f8d9f640 1 --2- 172.21.15.98:0/3994030851 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95f415c550 0x7f95f406bc90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:50.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.235+0000 7f95f8d9f640 1 -- 172.21.15.98:0/3994030851 >> 172.21.15.98:0/3994030851 conn(0x7f95f40b2010 msgr2=0x7f95f40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:50.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.235+0000 7f95f8d9f640 1 -- 172.21.15.98:0/3994030851 shutdown_connections 2024-09-19T08:38:50.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.235+0000 7f95f8d9f640 1 -- 172.21.15.98:0/3994030851 wait complete. 2024-09-19T08:38:50.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.235+0000 7f95f8d9f640 1 Processor -- start 2024-09-19T08:38:50.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.235+0000 7f95f8d9f640 1 -- start start 2024-09-19T08:38:50.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.235+0000 7f95f8d9f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95f406c1d0 0x7f95f41505c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:50.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.235+0000 7f95f8d9f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95f40cfad0 0x7f95f414b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:50.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.235+0000 7f95f8d9f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f95f415c550 0x7f95f4138690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:50.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.235+0000 7f95f8d9f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f95f4073380 con 0x7f95f40cfad0 2024-09-19T08:38:50.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.235+0000 7f95f8d9f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f95f415cb40 con 0x7f95f406c1d0 2024-09-19T08:38:50.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.235+0000 7f95f8d9f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f95f4073500 con 0x7f95f415c550 2024-09-19T08:38:50.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.235+0000 7f95f1d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95f40cfad0 0x7f95f414b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:50.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.235+0000 7f95f1d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95f40cfad0 0x7f95f414b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40662/0 (socket says 172.21.15.98:40662) 2024-09-19T08:38:50.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.235+0000 7f95f1d9b640 1 -- 172.21.15.98:0/2012348084 learned_addr learned my addr 172.21.15.98:0/2012348084 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:50.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.235+0000 7f95f259c640 1 --2- 172.21.15.98:0/2012348084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95f406c1d0 0x7f95f41505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:50.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.239+0000 7f95f2d9d640 1 --2- 172.21.15.98:0/2012348084 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f95f415c550 0x7f95f4138690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:50.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.239+0000 7f95f1d9b640 1 -- 172.21.15.98:0/2012348084 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f95f415c550 msgr2=0x7f95f4138690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:50.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.239+0000 7f95f1d9b640 1 --2- 172.21.15.98:0/2012348084 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f95f415c550 0x7f95f4138690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:50.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.239+0000 7f95f1d9b640 1 -- 172.21.15.98:0/2012348084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95f406c1d0 msgr2=0x7f95f41505c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:50.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.239+0000 7f95f1d9b640 1 --2- 172.21.15.98:0/2012348084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95f406c1d0 0x7f95f41505c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:50.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.239+0000 7f95f1d9b640 1 -- 172.21.15.98:0/2012348084 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f414db90 con 0x7f95f40cfad0 2024-09-19T08:38:50.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.239+0000 7f95f1d9b640 1 --2- 172.21.15.98:0/2012348084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95f40cfad0 0x7f95f414b530 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f95e800c420 tx=0x7f95e800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:50.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.239+0000 7f95e2ffd640 1 -- 172.21.15.98:0/2012348084 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e80074e0 con 0x7f95f40cfad0 2024-09-19T08:38:50.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.239+0000 7f95e2ffd640 1 -- 172.21.15.98:0/2012348084 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95e8007680 con 0x7f95f40cfad0 2024-09-19T08:38:50.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.239+0000 7f95e2ffd640 1 -- 172.21.15.98:0/2012348084 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e8004a30 con 0x7f95f40cfad0 2024-09-19T08:38:50.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.239+0000 7f95f8d9f640 1 -- 172.21.15.98:0/2012348084 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f414de70 con 0x7f95f40cfad0 2024-09-19T08:38:50.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.239+0000 7f95f8d9f640 1 -- 172.21.15.98:0/2012348084 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f414e430 con 0x7f95f40cfad0 2024-09-19T08:38:50.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.239+0000 7f95f8d9f640 1 -- 172.21.15.98:0/2012348084 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95f40723a0 con 0x7f95f40cfad0 2024-09-19T08:38:50.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.239+0000 7f95e2ffd640 1 -- 172.21.15.98:0/2012348084 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f95e801a020 con 0x7f95f40cfad0 2024-09-19T08:38:50.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.239+0000 7f95e2ffd640 1 --2- 172.21.15.98:0/2012348084 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f95d40678c0 0x7f95d4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:50.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.239+0000 7f95e2ffd640 1 -- 172.21.15.98:0/2012348084 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f95e8085440 con 0x7f95f40cfad0 2024-09-19T08:38:50.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.239+0000 7f95f259c640 1 --2- 172.21.15.98:0/2012348084 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f95d40678c0 0x7f95d4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:50.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.239+0000 7f95f259c640 1 --2- 172.21.15.98:0/2012348084 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f95d40678c0 0x7f95d4069d80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f95e40af000 tx=0x7f95e40a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:50.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.243+0000 7f95e2ffd640 1 -- 172.21.15.98:0/2012348084 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f95e8056010 con 0x7f95f40cfad0 2024-09-19T08:38:50.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.419+0000 7f95f8d9f640 1 -- 172.21.15.98:0/2012348084 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f95f40d0fa0 con 0x7f95f40cfad0 2024-09-19T08:38:50.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.419+0000 7f95e2ffd640 1 -- 172.21.15.98:0/2012348084 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f95e8055910 con 0x7f95f40cfad0 2024-09-19T08:38:50.423 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-19T08:38:50.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.419+0000 7f95f8d9f640 1 -- 172.21.15.98:0/2012348084 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f95d40678c0 msgr2=0x7f95d4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:50.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.419+0000 7f95f8d9f640 1 --2- 172.21.15.98:0/2012348084 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f95d40678c0 0x7f95d4069d80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f95e40af000 tx=0x7f95e40a7000 comp rx=0 tx=0).stop 2024-09-19T08:38:50.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.419+0000 7f95f8d9f640 1 -- 172.21.15.98:0/2012348084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95f40cfad0 msgr2=0x7f95f414b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:50.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.419+0000 7f95f8d9f640 1 --2- 172.21.15.98:0/2012348084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95f40cfad0 0x7f95f414b530 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f95e800c420 tx=0x7f95e800c8e0 comp rx=0 tx=0).stop 2024-09-19T08:38:50.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.419+0000 7f95f8d9f640 1 -- 172.21.15.98:0/2012348084 shutdown_connections 2024-09-19T08:38:50.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.419+0000 7f95f8d9f640 1 --2- 172.21.15.98:0/2012348084 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f95d40678c0 0x7f95d4069d80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:50.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.419+0000 7f95f8d9f640 1 --2- 172.21.15.98:0/2012348084 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f95f415c550 0x7f95f4138690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:50.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.419+0000 7f95f8d9f640 1 --2- 172.21.15.98:0/2012348084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95f40cfad0 0x7f95f414b530 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:50.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.419+0000 7f95f8d9f640 1 --2- 172.21.15.98:0/2012348084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f95f406c1d0 0x7f95f41505c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:50.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.419+0000 7f95f8d9f640 1 -- 172.21.15.98:0/2012348084 >> 172.21.15.98:0/2012348084 conn(0x7f95f40b2010 msgr2=0x7f95f41563d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:50.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.419+0000 7f95f8d9f640 1 -- 172.21.15.98:0/2012348084 shutdown_connections 2024-09-19T08:38:50.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.423+0000 7f95f8d9f640 1 -- 172.21.15.98:0/2012348084 wait complete. 2024-09-19T08:38:50.442 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi098.front.sepia.ceph.com... 2024-09-19T08:38:50.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:50.442 DEBUG:teuthology.orchestra.run.smithi098:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T08:38:50.462 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:38:50.463 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:38:50.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:50.463 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T08:38:50.520 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:38:50.520 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:38:50.526 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:38:50.526 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:38:50.573 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T08:38:50.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.695+0000 7fc4d14a7640 1 Processor -- start 2024-09-19T08:38:50.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.695+0000 7fc4d14a7640 1 -- start start 2024-09-19T08:38:50.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.695+0000 7fc4d14a7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc4cc0b0280 0x7fc4cc06d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:50.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.695+0000 7fc4d14a7640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc4cc074c70 0x7fc4cc0b6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:50.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.695+0000 7fc4d14a7640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4cc160350 con 0x7fc4cc0b6d90 2024-09-19T08:38:50.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.695+0000 7fc4d14a7640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fc4cc0b71c0 con 0x7fc4cc074c70 2024-09-19T08:38:50.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.695+0000 7fc4d14a7640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fc4cc160630 con 0x7fc4cc0b0280 2024-09-19T08:38:50.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.695+0000 7fc4ca7fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc4cc074c70 0x7fc4cc0b6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:50.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.695+0000 7fc4caffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc4cc0b0280 0x7fc4cc06d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:50.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.695+0000 7fc4cb7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fc4cc0b6d90 0x7fc4cc0d1200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:35474/0 (socket says 172.21.15.98:35474) 2024-09-19T08:38:50.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.695+0000 7fc4caffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc4cc0b0280 0x7fc4cc06d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:46736/0 (socket says 172.21.15.98:46736) 2024-09-19T08:38:50.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.695+0000 7fc4ca7fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc4cc074c70 0x7fc4cc0b6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40666/0 (socket says 172.21.15.98:40666) 2024-09-19T08:38:50.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.695+0000 7fc4cb7fe640 1 -- 172.21.15.98:0/350235357 learned_addr learned my addr 172.21.15.98:0/350235357 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:50.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.695+0000 7fc4ca7fc640 1 -- 172.21.15.98:0/350235357 >> v2:172.21.15.139:3300/0 conn(0x7fc4cc0b0280 msgr2=0x7fc4cc06d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:50.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.695+0000 7fc4ca7fc640 1 --2- 172.21.15.98:0/350235357 >> v2:172.21.15.139:3300/0 conn(0x7fc4cc0b0280 0x7fc4cc06d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:50.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.695+0000 7fc4c9ffb640 1 -- 172.21.15.98:0/350235357 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 836788439 0 0) 0x7fc4cc160350 con 0x7fc4cc0b6d90 2024-09-19T08:38:50.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.695+0000 7fc4ca7fc640 1 -- 172.21.15.98:0/350235357 >> v1:172.21.15.139:6789/0 conn(0x7fc4cc0b6d90 legacy=0x7fc4cc0d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:50.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.695+0000 7fc4ca7fc640 1 -- 172.21.15.98:0/350235357 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4cc160910 con 0x7fc4cc074c70 2024-09-19T08:38:50.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.699+0000 7fc4ca7fc640 1 --2- 172.21.15.98:0/350235357 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4cc074c70 0x7fc4cc0b6850 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc069910 tx=0x7fc4c001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c723cbe061afeb81 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:50.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.699+0000 7fc4c9ffb640 1 -- 172.21.15.98:0/350235357 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c00b0070 con 0x7fc4cc074c70 2024-09-19T08:38:50.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.699+0000 7fc4c9ffb640 1 -- 172.21.15.98:0/350235357 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4c00ab600 con 0x7fc4cc074c70 2024-09-19T08:38:50.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.699+0000 7fc4c9ffb640 1 -- 172.21.15.98:0/350235357 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c00ab9d0 con 0x7fc4cc074c70 2024-09-19T08:38:50.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.699+0000 7fc4d14a7640 1 -- 172.21.15.98:0/350235357 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4cc074c70 msgr2=0x7fc4cc0b6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:50.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.699+0000 7fc4d14a7640 1 --2- 172.21.15.98:0/350235357 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4cc074c70 0x7fc4cc0b6850 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc069910 tx=0x7fc4c001c3f0 comp rx=0 tx=0).stop 2024-09-19T08:38:50.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.699+0000 7fc4d14a7640 1 -- 172.21.15.98:0/350235357 shutdown_connections 2024-09-19T08:38:50.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.699+0000 7fc4d14a7640 1 --2- 172.21.15.98:0/350235357 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4cc074c70 0x7fc4cc0b6850 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:50.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.699+0000 7fc4d14a7640 1 --2- 172.21.15.98:0/350235357 >> v2:172.21.15.139:3300/0 conn(0x7fc4cc0b0280 0x7fc4cc06d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:50.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.699+0000 7fc4d14a7640 1 -- 172.21.15.98:0/350235357 >> 172.21.15.98:0/350235357 conn(0x7fc4cc073170 msgr2=0x7fc4cc073580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:50.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.699+0000 7fc4d14a7640 1 -- 172.21.15.98:0/350235357 shutdown_connections 2024-09-19T08:38:50.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.699+0000 7fc4d14a7640 1 -- 172.21.15.98:0/350235357 wait complete. 2024-09-19T08:38:50.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.699+0000 7fc4d14a7640 1 Processor -- start 2024-09-19T08:38:50.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.699+0000 7fc4d14a7640 1 -- start start 2024-09-19T08:38:50.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.703+0000 7fc4d14a7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4cc074c70 0x7fc4cc155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:50.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.703+0000 7fc4d14a7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4cc0b0280 0x7fc4cc09a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:50.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.703+0000 7fc4d14a7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4cc0b6d90 0x7fc4cc0cd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:50.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.703+0000 7fc4d14a7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc169500 con 0x7fc4cc0b0280 2024-09-19T08:38:50.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.703+0000 7fc4d14a7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc169380 con 0x7fc4cc0b6d90 2024-09-19T08:38:50.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.703+0000 7fc4d14a7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc4cc169680 con 0x7fc4cc074c70 2024-09-19T08:38:50.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.703+0000 7fc4ca7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4cc0b0280 0x7fc4cc09a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:50.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.703+0000 7fc4ca7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4cc0b0280 0x7fc4cc09a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40680/0 (socket says 172.21.15.98:40680) 2024-09-19T08:38:50.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.703+0000 7fc4ca7fc640 1 -- 172.21.15.98:0/2082263694 learned_addr learned my addr 172.21.15.98:0/2082263694 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:50.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.703+0000 7fc4cb7fe640 1 --2- 172.21.15.98:0/2082263694 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4cc0b6d90 0x7fc4cc0cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:50.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.703+0000 7fc4caffd640 1 --2- 172.21.15.98:0/2082263694 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4cc074c70 0x7fc4cc155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:50.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.703+0000 7fc4ca7fc640 1 -- 172.21.15.98:0/2082263694 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4cc074c70 msgr2=0x7fc4cc155840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:50.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.703+0000 7fc4ca7fc640 1 --2- 172.21.15.98:0/2082263694 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4cc074c70 0x7fc4cc155840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:50.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.703+0000 7fc4ca7fc640 1 -- 172.21.15.98:0/2082263694 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4cc0b6d90 msgr2=0x7fc4cc0cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:50.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.703+0000 7fc4ca7fc640 1 --2- 172.21.15.98:0/2082263694 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4cc0b6d90 0x7fc4cc0cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:50.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.703+0000 7fc4ca7fc640 1 -- 172.21.15.98:0/2082263694 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4cc0d0e60 con 0x7fc4cc0b0280 2024-09-19T08:38:50.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.703+0000 7fc4ca7fc640 1 --2- 172.21.15.98:0/2082263694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4cc0b0280 0x7fc4cc09a850 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00ae040 tx=0x7fc4c0019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:50.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.703+0000 7fc4b37fe640 1 -- 172.21.15.98:0/2082263694 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c00b0040 con 0x7fc4cc0b0280 2024-09-19T08:38:50.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.703+0000 7fc4b37fe640 1 -- 172.21.15.98:0/2082263694 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4c00b9070 con 0x7fc4cc0b0280 2024-09-19T08:38:50.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.703+0000 7fc4d14a7640 1 -- 172.21.15.98:0/2082263694 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4cc0cee60 con 0x7fc4cc0b0280 2024-09-19T08:38:50.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.703+0000 7fc4b37fe640 1 -- 172.21.15.98:0/2082263694 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0006e20 con 0x7fc4cc0b0280 2024-09-19T08:38:50.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.703+0000 7fc4d14a7640 1 -- 172.21.15.98:0/2082263694 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4cc0d0b80 con 0x7fc4cc0b0280 2024-09-19T08:38:50.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.703+0000 7fc4d14a7640 1 -- 172.21.15.98:0/2082263694 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4cc0b6850 con 0x7fc4cc0b0280 2024-09-19T08:38:50.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.703+0000 7fc4b37fe640 1 -- 172.21.15.98:0/2082263694 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc4c00b4650 con 0x7fc4cc0b0280 2024-09-19T08:38:50.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.703+0000 7fc4b37fe640 1 --2- 172.21.15.98:0/2082263694 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc4b80678c0 0x7fc4b8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:50.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.703+0000 7fc4b37fe640 1 -- 172.21.15.98:0/2082263694 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc4c011f7f0 con 0x7fc4cc0b0280 2024-09-19T08:38:50.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.703+0000 7fc4caffd640 1 --2- 172.21.15.98:0/2082263694 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc4b80678c0 0x7fc4b8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:50.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.707+0000 7fc4caffd640 1 --2- 172.21.15.98:0/2082263694 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc4b80678c0 0x7fc4b8069d80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009a50 tx=0x7fc4bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:50.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.707+0000 7fc4b37fe640 1 -- 172.21.15.98:0/2082263694 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc4c0016070 con 0x7fc4cc0b0280 2024-09-19T08:38:50.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.871+0000 7fc4d14a7640 1 -- 172.21.15.98:0/2082263694 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc4cc0754f0 con 0x7fc4cc0b0280 2024-09-19T08:38:50.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.871+0000 7fc4b37fe640 1 -- 172.21.15.98:0/2082263694 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7fc4c00f0330 con 0x7fc4cc0b0280 2024-09-19T08:38:50.876 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-19T08:38:50.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.875+0000 7fc4d14a7640 1 -- 172.21.15.98:0/2082263694 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc4b80678c0 msgr2=0x7fc4b8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:50.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.875+0000 7fc4d14a7640 1 --2- 172.21.15.98:0/2082263694 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc4b80678c0 0x7fc4b8069d80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009a50 tx=0x7fc4bc008040 comp rx=0 tx=0).stop 2024-09-19T08:38:50.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.875+0000 7fc4d14a7640 1 -- 172.21.15.98:0/2082263694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4cc0b0280 msgr2=0x7fc4cc09a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:50.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.875+0000 7fc4d14a7640 1 --2- 172.21.15.98:0/2082263694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4cc0b0280 0x7fc4cc09a850 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00ae040 tx=0x7fc4c0019190 comp rx=0 tx=0).stop 2024-09-19T08:38:50.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.875+0000 7fc4d14a7640 1 -- 172.21.15.98:0/2082263694 shutdown_connections 2024-09-19T08:38:50.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.875+0000 7fc4d14a7640 1 --2- 172.21.15.98:0/2082263694 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc4b80678c0 0x7fc4b8069d80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:50.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.875+0000 7fc4d14a7640 1 --2- 172.21.15.98:0/2082263694 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4cc0b6d90 0x7fc4cc0cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:50.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.875+0000 7fc4d14a7640 1 --2- 172.21.15.98:0/2082263694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4cc0b0280 0x7fc4cc09a850 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:50.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.875+0000 7fc4d14a7640 1 --2- 172.21.15.98:0/2082263694 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4cc074c70 0x7fc4cc155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:50.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.875+0000 7fc4d14a7640 1 -- 172.21.15.98:0/2082263694 >> 172.21.15.98:0/2082263694 conn(0x7fc4cc073170 msgr2=0x7fc4cc0b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:50.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.875+0000 7fc4d14a7640 1 -- 172.21.15.98:0/2082263694 shutdown_connections 2024-09-19T08:38:50.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:50.875+0000 7fc4d14a7640 1 -- 172.21.15.98:0/2082263694 wait complete. 2024-09-19T08:38:50.894 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:38:50.894 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:38:50.901 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T08:38:51.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.007+0000 7faff3b93640 1 Processor -- start 2024-09-19T08:38:51.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.007+0000 7faff3b93640 1 -- start start 2024-09-19T08:38:51.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.007+0000 7faff3b93640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafec0d4820 0x7fafec1621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:51.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.007+0000 7faff3b93640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fafec0b9970 0x7fafec0b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:51.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.007+0000 7faff3b93640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafec068440 con 0x7fafec0b7f80 2024-09-19T08:38:51.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.007+0000 7faff3b93640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fafec163050 con 0x7fafec0b9970 2024-09-19T08:38:51.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.007+0000 7faff3b93640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fafec0ba360 con 0x7fafec0d4820 2024-09-19T08:38:51.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.007+0000 7faff192f640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fafec0b7f80 0x7fafec15f560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:35944/0 (socket says 172.21.15.98:35944) 2024-09-19T08:38:51.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.007+0000 7faff192f640 1 -- 172.21.15.98:0/1557245437 learned_addr learned my addr 172.21.15.98:0/1557245437 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:51.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.007+0000 7faff2130640 1 --2- 172.21.15.98:0/1557245437 >> v2:172.21.15.139:3300/0 conn(0x7fafec0b9970 0x7fafec0b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:51.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.007+0000 7faff092d640 1 -- 172.21.15.98:0/1557245437 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1836681104 0 0) 0x7fafec068440 con 0x7fafec0b7f80 2024-09-19T08:38:51.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.007+0000 7faff092d640 1 -- 172.21.15.98:0/1557245437 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fafd4003530 con 0x7fafec0b7f80 2024-09-19T08:38:51.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.007+0000 7faff112e640 1 --2- 172.21.15.98:0/1557245437 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafec0d4820 0x7fafec1621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:51.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.007+0000 7faff092d640 1 -- 172.21.15.98:0/1557245437 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2095019995 0 0) 0x7fafd4003530 con 0x7fafec0b7f80 2024-09-19T08:38:51.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.007+0000 7faff092d640 1 -- 172.21.15.98:0/1557245437 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafec0d4820 msgr2=0x7fafec1621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:51.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.007+0000 7faff092d640 1 --2- 172.21.15.98:0/1557245437 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafec0d4820 0x7fafec1621b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:51.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.007+0000 7faff092d640 1 -- 172.21.15.98:0/1557245437 >> v2:172.21.15.139:3300/0 conn(0x7fafec0b9970 msgr2=0x7fafec0b9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:51.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.007+0000 7faff092d640 1 --2- 172.21.15.98:0/1557245437 >> v2:172.21.15.139:3300/0 conn(0x7fafec0b9970 0x7fafec0b9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:51.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.007+0000 7faff092d640 1 -- 172.21.15.98:0/1557245437 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafec162a90 con 0x7fafec0b7f80 2024-09-19T08:38:51.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.007+0000 7faff092d640 1 -- 172.21.15.98:0/1557245437 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fafe4001680 con 0x7fafec0b7f80 2024-09-19T08:38:51.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.007+0000 7faff092d640 1 -- 172.21.15.98:0/1557245437 >> v1:172.21.15.98:6789/0 conn(0x7fafec0b7f80 legacy=0x7fafec15f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:51.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.007+0000 7faff092d640 1 --2- 172.21.15.98:0/1557245437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fafd40039c0 0x7fafd4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:51.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.007+0000 7faff092d640 1 --2- 172.21.15.98:0/1557245437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafd4004810 0x7fafd4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:51.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.007+0000 7faff092d640 1 --2- 172.21.15.98:0/1557245437 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafec0d4820 0x7fafd40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:51.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.007+0000 7faff092d640 1 -- 172.21.15.98:0/1557245437 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fafec163050 con 0x7fafd4004810 2024-09-19T08:38:51.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.007+0000 7faff092d640 1 -- 172.21.15.98:0/1557245437 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fafec0ba360 con 0x7fafd40039c0 2024-09-19T08:38:51.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.007+0000 7faff092d640 1 -- 172.21.15.98:0/1557245437 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fafd400b9d0 con 0x7fafec0d4820 2024-09-19T08:38:51.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.007+0000 7faff112e640 1 --2- 172.21.15.98:0/1557245437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafd4004810 0x7fafd4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:51.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.007+0000 7faff192f640 1 --2- 172.21.15.98:0/1557245437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fafd40039c0 0x7fafd4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:51.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.007+0000 7faff2130640 1 --2- 172.21.15.98:0/1557245437 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafec0d4820 0x7fafd40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:51.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.007+0000 7faff112e640 1 -- 172.21.15.98:0/1557245437 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafec0d4820 msgr2=0x7fafd40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:51.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.007+0000 7faff112e640 1 --2- 172.21.15.98:0/1557245437 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafec0d4820 0x7fafd40051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:51.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.007+0000 7faff112e640 1 -- 172.21.15.98:0/1557245437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fafd40039c0 msgr2=0x7fafd4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:51.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.007+0000 7faff112e640 1 --2- 172.21.15.98:0/1557245437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fafd40039c0 0x7fafd4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:51.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.007+0000 7faff112e640 1 -- 172.21.15.98:0/1557245437 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd4003530 con 0x7fafd4004810 2024-09-19T08:38:51.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.011+0000 7faff112e640 1 --2- 172.21.15.98:0/1557245437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafd4004810 0x7fafd4004c90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fafe0009ab0 tx=0x7fafe001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660201cb0381168f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:51.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.011+0000 7faff092d640 1 -- 172.21.15.98:0/1557245437 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe00096b0 con 0x7fafd4004810 2024-09-19T08:38:51.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.011+0000 7faff092d640 1 -- 172.21.15.98:0/1557245437 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafe00a6460 con 0x7fafd4004810 2024-09-19T08:38:51.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.011+0000 7faff092d640 1 -- 172.21.15.98:0/1557245437 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe00a6810 con 0x7fafd4004810 2024-09-19T08:38:51.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.011+0000 7faff3b93640 1 -- 172.21.15.98:0/1557245437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafd4004810 msgr2=0x7fafd4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:51.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.011+0000 7faff3b93640 1 --2- 172.21.15.98:0/1557245437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafd4004810 0x7fafd4004c90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fafe0009ab0 tx=0x7fafe001ca50 comp rx=0 tx=0).stop 2024-09-19T08:38:51.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.011+0000 7faff2130640 1 -- 172.21.15.98:0/1557245437 reap_dead start 2024-09-19T08:38:51.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.011+0000 7faff3b93640 1 -- 172.21.15.98:0/1557245437 shutdown_connections 2024-09-19T08:38:51.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.011+0000 7faff3b93640 1 -- 172.21.15.98:0/1557245437 >> 172.21.15.98:0/1557245437 conn(0x7fafec074e40 msgr2=0x7fafec075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:51.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.011+0000 7faff3b93640 1 -- 172.21.15.98:0/1557245437 shutdown_connections 2024-09-19T08:38:51.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.011+0000 7faff3b93640 1 -- 172.21.15.98:0/1557245437 wait complete. 2024-09-19T08:38:51.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.011+0000 7faff3b93640 1 Processor -- start 2024-09-19T08:38:51.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.011+0000 7faff3b93640 1 -- start start 2024-09-19T08:38:51.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.015+0000 7faff3b93640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafd4004810 0x7fafec159220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:51.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.015+0000 7faff3b93640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fafec13fc00 0x7fafec152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:51.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.015+0000 7faff3b93640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafec140080 0x7fafec153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:51.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.015+0000 7faff3b93640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fafec0b8360 con 0x7fafec140080 2024-09-19T08:38:51.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.015+0000 7faff3b93640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fafec15ed60 con 0x7fafec13fc00 2024-09-19T08:38:51.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.015+0000 7faff3b93640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fafec0b84e0 con 0x7fafd4004810 2024-09-19T08:38:51.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.015+0000 7faff2130640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafec140080 0x7fafec153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:51.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.015+0000 7faff2130640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafec140080 0x7fafec153060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40700/0 (socket says 172.21.15.98:40700) 2024-09-19T08:38:51.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.015+0000 7faff2130640 1 -- 172.21.15.98:0/1395691880 learned_addr learned my addr 172.21.15.98:0/1395691880 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:51.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.015+0000 7faff112e640 1 --2- 172.21.15.98:0/1395691880 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fafec13fc00 0x7fafec152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:51.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.015+0000 7faff192f640 1 --2- 172.21.15.98:0/1395691880 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafd4004810 0x7fafec159220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:51.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.015+0000 7faff2130640 1 -- 172.21.15.98:0/1395691880 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafd4004810 msgr2=0x7fafec159220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:51.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.015+0000 7faff2130640 1 --2- 172.21.15.98:0/1395691880 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafd4004810 0x7fafec159220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:51.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.015+0000 7faff2130640 1 -- 172.21.15.98:0/1395691880 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fafec13fc00 msgr2=0x7fafec152b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:51.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.015+0000 7faff2130640 1 --2- 172.21.15.98:0/1395691880 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fafec13fc00 0x7fafec152b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:51.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.015+0000 7faff2130640 1 -- 172.21.15.98:0/1395691880 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafec13eb60 con 0x7fafec140080 2024-09-19T08:38:51.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.015+0000 7faff2130640 1 --2- 172.21.15.98:0/1395691880 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafec140080 0x7fafec153060 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fafec0b6670 tx=0x7fafdc00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:51.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.015+0000 7fafea7fc640 1 -- 172.21.15.98:0/1395691880 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafdc01c070 con 0x7fafec140080 2024-09-19T08:38:51.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.015+0000 7fafea7fc640 1 -- 172.21.15.98:0/1395691880 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafdc017400 con 0x7fafec140080 2024-09-19T08:38:51.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.015+0000 7faff3b93640 1 -- 172.21.15.98:0/1395691880 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafec13ee40 con 0x7fafec140080 2024-09-19T08:38:51.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.015+0000 7fafea7fc640 1 -- 172.21.15.98:0/1395691880 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafdc0177f0 con 0x7fafec140080 2024-09-19T08:38:51.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.015+0000 7faff3b93640 1 -- 172.21.15.98:0/1395691880 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafec13f400 con 0x7fafec140080 2024-09-19T08:38:51.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.015+0000 7faff3b93640 1 -- 172.21.15.98:0/1395691880 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafec0b9e20 con 0x7fafec140080 2024-09-19T08:38:51.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.015+0000 7fafea7fc640 1 -- 172.21.15.98:0/1395691880 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fafdc01f070 con 0x7fafec140080 2024-09-19T08:38:51.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.015+0000 7fafea7fc640 1 --2- 172.21.15.98:0/1395691880 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fafd80678c0 0x7fafd8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:51.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.015+0000 7fafea7fc640 1 -- 172.21.15.98:0/1395691880 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fafdc085d00 con 0x7fafec140080 2024-09-19T08:38:51.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.015+0000 7faff192f640 1 --2- 172.21.15.98:0/1395691880 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fafd80678c0 0x7fafd8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:51.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.015+0000 7faff192f640 1 --2- 172.21.15.98:0/1395691880 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fafd80678c0 0x7fafd8069d80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fafe4000fc0 tx=0x7fafe40047f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:51.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.019+0000 7fafea7fc640 1 -- 172.21.15.98:0/1395691880 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fafdc012070 con 0x7fafec140080 2024-09-19T08:38:51.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.191+0000 7faff3b93640 1 -- 172.21.15.98:0/1395691880 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fafec1604a0 con 0x7fafec140080 2024-09-19T08:38:51.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.195+0000 7fafea7fc640 1 -- 172.21.15.98:0/1395691880 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7fafdc0568d0 con 0x7fafec140080 2024-09-19T08:38:51.197 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-19T08:38:51.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.195+0000 7faff3b93640 1 -- 172.21.15.98:0/1395691880 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fafd80678c0 msgr2=0x7fafd8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:51.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.195+0000 7faff3b93640 1 --2- 172.21.15.98:0/1395691880 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fafd80678c0 0x7fafd8069d80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fafe4000fc0 tx=0x7fafe40047f0 comp rx=0 tx=0).stop 2024-09-19T08:38:51.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.195+0000 7faff3b93640 1 -- 172.21.15.98:0/1395691880 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafec140080 msgr2=0x7fafec153060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:51.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.195+0000 7faff3b93640 1 --2- 172.21.15.98:0/1395691880 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafec140080 0x7fafec153060 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fafec0b6670 tx=0x7fafdc00f4a0 comp rx=0 tx=0).stop 2024-09-19T08:38:51.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.195+0000 7faff3b93640 1 -- 172.21.15.98:0/1395691880 shutdown_connections 2024-09-19T08:38:51.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.195+0000 7faff3b93640 1 --2- 172.21.15.98:0/1395691880 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fafd80678c0 0x7fafd8069d80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:51.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.195+0000 7faff3b93640 1 --2- 172.21.15.98:0/1395691880 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafec140080 0x7fafec153060 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:51.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.195+0000 7faff3b93640 1 --2- 172.21.15.98:0/1395691880 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fafec13fc00 0x7fafec152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:51.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.195+0000 7faff3b93640 1 --2- 172.21.15.98:0/1395691880 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafd4004810 0x7fafec159220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:51.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.195+0000 7faff3b93640 1 -- 172.21.15.98:0/1395691880 >> 172.21.15.98:0/1395691880 conn(0x7fafec074e40 msgr2=0x7fafec067a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:51.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.195+0000 7faff3b93640 1 -- 172.21.15.98:0/1395691880 shutdown_connections 2024-09-19T08:38:51.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.195+0000 7faff3b93640 1 -- 172.21.15.98:0/1395691880 wait complete. 2024-09-19T08:38:51.215 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi139.front.sepia.ceph.com... 2024-09-19T08:38:51.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:51.215 DEBUG:teuthology.orchestra.run.smithi139:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-19T08:38:51.245 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:38:51.245 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:38:51.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:51.245 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-19T08:38:51.302 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:38:51.302 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:38:51.308 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:38:51.308 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:38:51.355 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T08:38:51.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.507+0000 7f0a12da1640 1 Processor -- start 2024-09-19T08:38:51.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.507+0000 7f0a12da1640 1 -- start start 2024-09-19T08:38:51.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.507+0000 7f0a12da1640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0a0c15cc10 0x7f0a0c071520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:51.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.507+0000 7f0a12da1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0a0c0d01b0 0x7f0a0c071a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:51.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.507+0000 7f0a12da1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0a0c071fa0 0x7f0a0c06ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:51.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.507+0000 7f0a12da1640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f0a0c15d3a0 con 0x7f0a0c15cc10 2024-09-19T08:38:51.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.507+0000 7f0a12da1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f0a0c073080 con 0x7f0a0c071fa0 2024-09-19T08:38:51.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.507+0000 7f0a12da1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0a0c073360 con 0x7f0a0c0d01b0 2024-09-19T08:38:51.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.507+0000 7f0a10b3d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0a0c15cc10 0x7f0a0c071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:51.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.507+0000 7f0a10b3d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0a0c15cc10 0x7f0a0c071520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40702/0 (socket says 172.21.15.98:40702) 2024-09-19T08:38:51.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.507+0000 7f0a10b3d640 1 -- 172.21.15.98:0/40312687 learned_addr learned my addr 172.21.15.98:0/40312687 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:51.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.507+0000 7f0a10b3d640 1 -- 172.21.15.98:0/40312687 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0a0c0d01b0 msgr2=0x7f0a0c071a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:51.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.507+0000 7f0a1133e640 1 --2- 172.21.15.98:0/40312687 >> v2:172.21.15.139:3300/0 conn(0x7f0a0c071fa0 0x7f0a0c06ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:51.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.507+0000 7f0a10b3d640 1 --2- 172.21.15.98:0/40312687 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0a0c0d01b0 0x7f0a0c071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:51.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.507+0000 7f0a10b3d640 1 -- 172.21.15.98:0/40312687 >> v2:172.21.15.139:3300/0 conn(0x7f0a0c071fa0 msgr2=0x7f0a0c06ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:51.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.507+0000 7f0a10b3d640 1 --2- 172.21.15.98:0/40312687 >> v2:172.21.15.139:3300/0 conn(0x7f0a0c071fa0 0x7f0a0c06ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:51.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.507+0000 7f0a10b3d640 1 -- 172.21.15.98:0/40312687 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a0c073640 con 0x7f0a0c15cc10 2024-09-19T08:38:51.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.511+0000 7f0a10b3d640 1 --2- 172.21.15.98:0/40312687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a0c15cc10 0x7f0a0c071520 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c0d0950 tx=0x7f09f801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e65b4b7d4e2d335 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:51.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.511+0000 7f0a0b7fe640 1 -- 172.21.15.98:0/40312687 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f80b2070 con 0x7f0a0c15cc10 2024-09-19T08:38:51.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.511+0000 7f0a0b7fe640 1 -- 172.21.15.98:0/40312687 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09f80a9070 con 0x7f0a0c15cc10 2024-09-19T08:38:51.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.511+0000 7f0a0b7fe640 1 -- 172.21.15.98:0/40312687 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f80ad880 con 0x7f0a0c15cc10 2024-09-19T08:38:51.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.511+0000 7f0a12da1640 1 -- 172.21.15.98:0/40312687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a0c15cc10 msgr2=0x7f0a0c071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:51.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.511+0000 7f0a12da1640 1 --2- 172.21.15.98:0/40312687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a0c15cc10 0x7f0a0c071520 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c0d0950 tx=0x7f09f801c490 comp rx=0 tx=0).stop 2024-09-19T08:38:51.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.511+0000 7f0a12da1640 1 -- 172.21.15.98:0/40312687 shutdown_connections 2024-09-19T08:38:51.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.511+0000 7f0a12da1640 1 --2- 172.21.15.98:0/40312687 >> v2:172.21.15.139:3300/0 conn(0x7f0a0c071fa0 0x7f0a0c06ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:51.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.511+0000 7f0a12da1640 1 --2- 172.21.15.98:0/40312687 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0a0c0d01b0 0x7f0a0c071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:51.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.511+0000 7f0a12da1640 1 --2- 172.21.15.98:0/40312687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a0c15cc10 0x7f0a0c071520 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:51.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.511+0000 7f0a12da1640 1 -- 172.21.15.98:0/40312687 >> 172.21.15.98:0/40312687 conn(0x7f0a0c0d0980 msgr2=0x7f0a0c0d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:51.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.511+0000 7f0a12da1640 1 -- 172.21.15.98:0/40312687 shutdown_connections 2024-09-19T08:38:51.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.511+0000 7f0a12da1640 1 -- 172.21.15.98:0/40312687 wait complete. 2024-09-19T08:38:51.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.511+0000 7f0a12da1640 1 Processor -- start 2024-09-19T08:38:51.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.511+0000 7f0a12da1640 1 -- start start 2024-09-19T08:38:51.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.511+0000 7f0a12da1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a0c071fa0 0x7f0a0c0c4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:51.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.511+0000 7f0a12da1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a0c0d01b0 0x7f0a0c153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:51.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.511+0000 7f0a12da1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0a0c15cc10 0x7f0a0c0af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:51.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.511+0000 7f0a12da1640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0c15cff0 con 0x7f0a0c0d01b0 2024-09-19T08:38:51.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.511+0000 7f0a12da1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0c06fc70 con 0x7f0a0c071fa0 2024-09-19T08:38:51.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.511+0000 7f0a12da1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0a0c0b02a0 con 0x7f0a0c15cc10 2024-09-19T08:38:51.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.511+0000 7f0a0bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a0c0d01b0 0x7f0a0c153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:51.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.511+0000 7f0a0bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a0c0d01b0 0x7f0a0c153fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40710/0 (socket says 172.21.15.98:40710) 2024-09-19T08:38:51.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.511+0000 7f0a0bfff640 1 -- 172.21.15.98:0/2825123400 learned_addr learned my addr 172.21.15.98:0/2825123400 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:51.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.511+0000 7f0a10b3d640 1 --2- 172.21.15.98:0/2825123400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a0c071fa0 0x7f0a0c0c4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:51.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.511+0000 7f0a0bfff640 1 -- 172.21.15.98:0/2825123400 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0a0c15cc10 msgr2=0x7f0a0c0af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:51.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.511+0000 7f0a0bfff640 1 --2- 172.21.15.98:0/2825123400 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0a0c15cc10 0x7f0a0c0af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:51.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.511+0000 7f0a0bfff640 1 -- 172.21.15.98:0/2825123400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a0c071fa0 msgr2=0x7f0a0c0c4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:51.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.511+0000 7f0a0bfff640 1 --2- 172.21.15.98:0/2825123400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a0c071fa0 0x7f0a0c0c4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:51.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.511+0000 7f0a0bfff640 1 -- 172.21.15.98:0/2825123400 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a0c0c66c0 con 0x7f0a0c0d01b0 2024-09-19T08:38:51.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.511+0000 7f0a0bfff640 1 --2- 172.21.15.98:0/2825123400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a0c0d01b0 0x7f0a0c153fd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000b550 tx=0x7f0a0000ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:51.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.511+0000 7f0a097fa640 1 -- 172.21.15.98:0/2825123400 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a00014070 con 0x7f0a0c0d01b0 2024-09-19T08:38:51.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.511+0000 7f0a097fa640 1 -- 172.21.15.98:0/2825123400 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a00010890 con 0x7f0a0c0d01b0 2024-09-19T08:38:51.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.511+0000 7f0a12da1640 1 -- 172.21.15.98:0/2825123400 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a0c0c69a0 con 0x7f0a0c0d01b0 2024-09-19T08:38:51.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.511+0000 7f0a097fa640 1 -- 172.21.15.98:0/2825123400 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a00010c80 con 0x7f0a0c0d01b0 2024-09-19T08:38:51.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.511+0000 7f0a12da1640 1 -- 172.21.15.98:0/2825123400 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a0c0c6f60 con 0x7f0a0c0d01b0 2024-09-19T08:38:51.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.515+0000 7f0a12da1640 1 -- 172.21.15.98:0/2825123400 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a0c0c7160 con 0x7f0a0c0d01b0 2024-09-19T08:38:51.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.515+0000 7f0a097fa640 1 -- 172.21.15.98:0/2825123400 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0a00017070 con 0x7f0a0c0d01b0 2024-09-19T08:38:51.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.515+0000 7f0a097fa640 1 --2- 172.21.15.98:0/2825123400 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f09f00678c0 0x7f09f0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:51.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.515+0000 7f0a10b3d640 1 --2- 172.21.15.98:0/2825123400 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f09f00678c0 0x7f09f0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:51.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.515+0000 7f0a097fa640 1 -- 172.21.15.98:0/2825123400 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0a00085a90 con 0x7f0a0c0d01b0 2024-09-19T08:38:51.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.515+0000 7f0a10b3d640 1 --2- 172.21.15.98:0/2825123400 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f09f00678c0 0x7f09f0069d80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f09f80af000 tx=0x7f09f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:51.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.519+0000 7f0a097fa640 1 -- 172.21.15.98:0/2825123400 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f0a0c0c7160 con 0x7f0a0c0d01b0 2024-09-19T08:38:51.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.679+0000 7f0a12da1640 1 -- 172.21.15.98:0/2825123400 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0a0c06ef20 con 0x7f0a0c0d01b0 2024-09-19T08:38:51.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.679+0000 7f0a097fa640 1 -- 172.21.15.98:0/2825123400 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f0a00010e20 con 0x7f0a0c0d01b0 2024-09-19T08:38:51.683 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-19T08:38:51.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.679+0000 7f0a12da1640 1 -- 172.21.15.98:0/2825123400 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f09f00678c0 msgr2=0x7f09f0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:51.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.679+0000 7f0a12da1640 1 --2- 172.21.15.98:0/2825123400 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f09f00678c0 0x7f09f0069d80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f09f80af000 tx=0x7f09f8008040 comp rx=0 tx=0).stop 2024-09-19T08:38:51.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.679+0000 7f0a12da1640 1 -- 172.21.15.98:0/2825123400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a0c0d01b0 msgr2=0x7f0a0c153fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:51.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.679+0000 7f0a12da1640 1 --2- 172.21.15.98:0/2825123400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a0c0d01b0 0x7f0a0c153fd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000b550 tx=0x7f0a0000ba10 comp rx=0 tx=0).stop 2024-09-19T08:38:51.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.679+0000 7f0a12da1640 1 -- 172.21.15.98:0/2825123400 shutdown_connections 2024-09-19T08:38:51.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.679+0000 7f0a12da1640 1 --2- 172.21.15.98:0/2825123400 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f09f00678c0 0x7f09f0069d80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:51.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.679+0000 7f0a12da1640 1 --2- 172.21.15.98:0/2825123400 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0a0c15cc10 0x7f0a0c0af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:51.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.679+0000 7f0a12da1640 1 --2- 172.21.15.98:0/2825123400 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0a0c0d01b0 0x7f0a0c153fd0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:51.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.679+0000 7f0a12da1640 1 --2- 172.21.15.98:0/2825123400 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0a0c071fa0 0x7f0a0c0c4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:51.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.679+0000 7f0a12da1640 1 -- 172.21.15.98:0/2825123400 >> 172.21.15.98:0/2825123400 conn(0x7f0a0c0d0980 msgr2=0x7f0a0c067a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:51.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.683+0000 7f0a12da1640 1 -- 172.21.15.98:0/2825123400 shutdown_connections 2024-09-19T08:38:51.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.683+0000 7f0a12da1640 1 -- 172.21.15.98:0/2825123400 wait complete. 2024-09-19T08:38:51.700 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:38:51.700 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:38:51.706 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T08:38:51.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.867+0000 7fa44e33f640 1 Processor -- start 2024-09-19T08:38:51.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.867+0000 7fa44e33f640 1 -- start start 2024-09-19T08:38:51.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.867+0000 7fa44e33f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4480b7f80 0x7fa44815f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:51.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.867+0000 7fa44e33f640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa4480b9970 0x7fa4480b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:51.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.867+0000 7fa44e33f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa448068440 con 0x7fa4480d4820 2024-09-19T08:38:51.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.867+0000 7fa44e33f640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fa448163050 con 0x7fa4480b9970 2024-09-19T08:38:51.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.867+0000 7fa44e33f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa4480ba360 con 0x7fa4480b7f80 2024-09-19T08:38:51.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.867+0000 7fa44c8dc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa4480b9970 0x7fa4480b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:51.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.867+0000 7fa44c8dc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa4480b9970 0x7fa4480b9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40726/0 (socket says 172.21.15.98:40726) 2024-09-19T08:38:51.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.867+0000 7fa44c8dc640 1 -- 172.21.15.98:0/2444845076 learned_addr learned my addr 172.21.15.98:0/2444845076 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:51.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.867+0000 7fa447fff640 1 --2- 172.21.15.98:0/2444845076 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4480b7f80 0x7fa44815f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:51.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.867+0000 7fa44c8dc640 1 -- 172.21.15.98:0/2444845076 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4480b7f80 msgr2=0x7fa44815f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:51.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.867+0000 7fa44c8dc640 1 --2- 172.21.15.98:0/2444845076 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4480b7f80 0x7fa44815f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:51.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.867+0000 7fa44c8dc640 1 -- 172.21.15.98:0/2444845076 >> v1:172.21.15.139:6789/0 conn(0x7fa4480d4820 legacy=0x7fa4481621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:51.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.867+0000 7fa44c8dc640 1 -- 172.21.15.98:0/2444845076 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa448162a90 con 0x7fa4480b9970 2024-09-19T08:38:51.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.871+0000 7fa44c8dc640 1 --2- 172.21.15.98:0/2444845076 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4480b9970 0x7fa4480b9e20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa440009e50 tx=0x7fa44001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca5619f08c3460e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:51.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.871+0000 7fa446ffd640 1 -- 172.21.15.98:0/2444845076 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4400a5700 con 0x7fa4480b9970 2024-09-19T08:38:51.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.871+0000 7fa446ffd640 1 -- 172.21.15.98:0/2444845076 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4400a58a0 con 0x7fa4480b9970 2024-09-19T08:38:51.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.871+0000 7fa446ffd640 1 -- 172.21.15.98:0/2444845076 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4400a5c70 con 0x7fa4480b9970 2024-09-19T08:38:51.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.871+0000 7fa44e33f640 1 -- 172.21.15.98:0/2444845076 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4480b9970 msgr2=0x7fa4480b9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:51.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.871+0000 7fa44e33f640 1 --2- 172.21.15.98:0/2444845076 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4480b9970 0x7fa4480b9e20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa440009e50 tx=0x7fa44001ca30 comp rx=0 tx=0).stop 2024-09-19T08:38:51.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.871+0000 7fa44e33f640 1 -- 172.21.15.98:0/2444845076 shutdown_connections 2024-09-19T08:38:51.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.871+0000 7fa44e33f640 1 --2- 172.21.15.98:0/2444845076 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4480b9970 0x7fa4480b9e20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:51.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.871+0000 7fa44e33f640 1 --2- 172.21.15.98:0/2444845076 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4480b7f80 0x7fa44815f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:51.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.871+0000 7fa44e33f640 1 -- 172.21.15.98:0/2444845076 >> 172.21.15.98:0/2444845076 conn(0x7fa448074e40 msgr2=0x7fa448075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:51.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.871+0000 7fa44e33f640 1 -- 172.21.15.98:0/2444845076 shutdown_connections 2024-09-19T08:38:51.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.871+0000 7fa44e33f640 1 -- 172.21.15.98:0/2444845076 wait complete. 2024-09-19T08:38:51.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.871+0000 7fa44e33f640 1 Processor -- start 2024-09-19T08:38:51.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.871+0000 7fa44e33f640 1 -- start start 2024-09-19T08:38:51.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.875+0000 7fa44e33f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4480b7f80 0x7fa4480b5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:51.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.875+0000 7fa44e33f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4480b9970 0x7fa448141bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:51.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.875+0000 7fa44e33f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4480d4820 0x7fa4481602d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:51.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.875+0000 7fa44e33f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa4480b84e0 con 0x7fa4480d4820 2024-09-19T08:38:51.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.875+0000 7fa44e33f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa4480b8360 con 0x7fa4480b7f80 2024-09-19T08:38:51.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.875+0000 7fa44e33f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa448144590 con 0x7fa4480b9970 2024-09-19T08:38:51.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.875+0000 7fa44c8dc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4480d4820 0x7fa4481602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:51.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.875+0000 7fa44c8dc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4480d4820 0x7fa4481602d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40736/0 (socket says 172.21.15.98:40736) 2024-09-19T08:38:51.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.875+0000 7fa44c8dc640 1 -- 172.21.15.98:0/72672275 learned_addr learned my addr 172.21.15.98:0/72672275 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:51.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.875+0000 7fa4477fe640 1 --2- 172.21.15.98:0/72672275 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4480b9970 0x7fa448141bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:51.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.875+0000 7fa44c8dc640 1 -- 172.21.15.98:0/72672275 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4480b9970 msgr2=0x7fa448141bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:51.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.875+0000 7fa44c8dc640 1 --2- 172.21.15.98:0/72672275 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4480b9970 0x7fa448141bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:51.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.875+0000 7fa447fff640 1 --2- 172.21.15.98:0/72672275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4480b7f80 0x7fa4480b5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:51.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.875+0000 7fa44c8dc640 1 -- 172.21.15.98:0/72672275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4480b7f80 msgr2=0x7fa4480b5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:51.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.875+0000 7fa44c8dc640 1 --2- 172.21.15.98:0/72672275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4480b7f80 0x7fa4480b5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:51.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.875+0000 7fa44c8dc640 1 -- 172.21.15.98:0/72672275 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4480b41d0 con 0x7fa4480d4820 2024-09-19T08:38:51.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.875+0000 7fa44c8dc640 1 --2- 172.21.15.98:0/72672275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4480d4820 0x7fa4481602d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa44001a660 tx=0x7fa4400197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:51.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.875+0000 7fa444ff9640 1 -- 172.21.15.98:0/72672275 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4400bc020 con 0x7fa4480d4820 2024-09-19T08:38:51.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.875+0000 7fa444ff9640 1 -- 172.21.15.98:0/72672275 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4400b8670 con 0x7fa4480d4820 2024-09-19T08:38:51.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.875+0000 7fa44e33f640 1 -- 172.21.15.98:0/72672275 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa448160810 con 0x7fa4480d4820 2024-09-19T08:38:51.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.875+0000 7fa444ff9640 1 -- 172.21.15.98:0/72672275 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4400b8a60 con 0x7fa4480d4820 2024-09-19T08:38:51.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.875+0000 7fa44e33f640 1 -- 172.21.15.98:0/72672275 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4480b3b10 con 0x7fa4480d4820 2024-09-19T08:38:51.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.875+0000 7fa44e33f640 1 -- 172.21.15.98:0/72672275 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa44815f560 con 0x7fa4480d4820 2024-09-19T08:38:51.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.875+0000 7fa444ff9640 1 -- 172.21.15.98:0/72672275 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa4400bf020 con 0x7fa4480d4820 2024-09-19T08:38:51.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.875+0000 7fa444ff9640 1 --2- 172.21.15.98:0/72672275 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa4300678c0 0x7fa430069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:51.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.875+0000 7fa447fff640 1 --2- 172.21.15.98:0/72672275 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa4300678c0 0x7fa430069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:51.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.875+0000 7fa444ff9640 1 -- 172.21.15.98:0/72672275 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa440121100 con 0x7fa4480d4820 2024-09-19T08:38:51.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.879+0000 7fa447fff640 1 --2- 172.21.15.98:0/72672275 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa4300678c0 0x7fa430069d80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa4380096f0 tx=0x7fa438008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:51.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:51.879+0000 7fa444ff9640 1 -- 172.21.15.98:0/72672275 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa4400c6030 con 0x7fa4480d4820 2024-09-19T08:38:52.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.059+0000 7fa44e33f640 1 -- 172.21.15.98:0/72672275 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa448161c70 con 0x7fa4480d4820 2024-09-19T08:38:52.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.059+0000 7fa444ff9640 1 -- 172.21.15.98:0/72672275 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7fa440016620 con 0x7fa4480d4820 2024-09-19T08:38:52.063 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-19T08:38:52.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.063+0000 7fa44e33f640 1 -- 172.21.15.98:0/72672275 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa4300678c0 msgr2=0x7fa430069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:52.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.063+0000 7fa44e33f640 1 --2- 172.21.15.98:0/72672275 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa4300678c0 0x7fa430069d80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa4380096f0 tx=0x7fa438008040 comp rx=0 tx=0).stop 2024-09-19T08:38:52.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.063+0000 7fa44e33f640 1 -- 172.21.15.98:0/72672275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4480d4820 msgr2=0x7fa4481602d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:52.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.063+0000 7fa44e33f640 1 --2- 172.21.15.98:0/72672275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4480d4820 0x7fa4481602d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa44001a660 tx=0x7fa4400197d0 comp rx=0 tx=0).stop 2024-09-19T08:38:52.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.063+0000 7fa44e33f640 1 -- 172.21.15.98:0/72672275 shutdown_connections 2024-09-19T08:38:52.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.063+0000 7fa44e33f640 1 --2- 172.21.15.98:0/72672275 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa4300678c0 0x7fa430069d80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:52.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.063+0000 7fa44e33f640 1 --2- 172.21.15.98:0/72672275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4480d4820 0x7fa4481602d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:52.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.063+0000 7fa44e33f640 1 --2- 172.21.15.98:0/72672275 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4480b9970 0x7fa448141bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:52.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.063+0000 7fa44e33f640 1 --2- 172.21.15.98:0/72672275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4480b7f80 0x7fa4480b5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:52.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.063+0000 7fa44e33f640 1 -- 172.21.15.98:0/72672275 >> 172.21.15.98:0/72672275 conn(0x7fa448074e40 msgr2=0x7fa448154b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:52.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.063+0000 7fa44e33f640 1 -- 172.21.15.98:0/72672275 shutdown_connections 2024-09-19T08:38:52.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.063+0000 7fa44e33f640 1 -- 172.21.15.98:0/72672275 wait complete. 2024-09-19T08:38:52.082 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi098.front.sepia.ceph.com... 2024-09-19T08:38:52.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:52.083 DEBUG:teuthology.orchestra.run.smithi098:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-09-19T08:38:52.096 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:38:52.096 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:38:52.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:52.097 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-19T08:38:52.142 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:38:52.142 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:38:52.148 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:38:52.148 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:38:52.197 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T08:38:52.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.323+0000 7fcfcb0f3640 1 Processor -- start 2024-09-19T08:38:52.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.323+0000 7fcfcb0f3640 1 -- start start 2024-09-19T08:38:52.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.323+0000 7fcfcb0f3640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcfc40b9970 0x7fcfc40b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:52.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.323+0000 7fcfcb0f3640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfc4068440 con 0x7fcfc40d4820 2024-09-19T08:38:52.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.323+0000 7fcfcb0f3640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfc4162a90 con 0x7fcfc40b7f80 2024-09-19T08:38:52.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.323+0000 7fcfcb0f3640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fcfc4163050 con 0x7fcfc40b9970 2024-09-19T08:38:52.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.323+0000 7fcfc9690640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcfc40b9970 0x7fcfc40b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:52.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.323+0000 7fcfc3fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fcfc40d4820 0x7fcfc41621b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:35946/0 (socket says 172.21.15.98:35946) 2024-09-19T08:38:52.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.323+0000 7fcfc3fff640 1 -- 172.21.15.98:0/948457054 learned_addr learned my addr 172.21.15.98:0/948457054 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:52.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.323+0000 7fcfc9690640 1 -- 172.21.15.98:0/948457054 >> v1:172.21.15.139:6789/0 conn(0x7fcfc40b7f80 legacy=0x7fcfc415f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:52.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.323+0000 7fcfc9690640 1 -- 172.21.15.98:0/948457054 >> v1:172.21.15.98:6789/0 conn(0x7fcfc40d4820 legacy=0x7fcfc41621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:52.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.323+0000 7fcfc9690640 1 -- 172.21.15.98:0/948457054 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc4162d70 con 0x7fcfc40b9970 2024-09-19T08:38:52.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.327+0000 7fcfc9690640 1 --2- 172.21.15.98:0/948457054 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfc40b9970 0x7fcfc40b9de0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc009d90 tx=0x7fcfbc01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69e8266268c134da server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:52.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.327+0000 7fcfc37fe640 1 -- 172.21.15.98:0/948457054 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc0b2070 con 0x7fcfc40b9970 2024-09-19T08:38:52.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.327+0000 7fcfc37fe640 1 -- 172.21.15.98:0/948457054 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfbc0a5830 con 0x7fcfc40b9970 2024-09-19T08:38:52.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.327+0000 7fcfc37fe640 1 -- 172.21.15.98:0/948457054 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc0a5c00 con 0x7fcfc40b9970 2024-09-19T08:38:52.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.327+0000 7fcfcb0f3640 1 -- 172.21.15.98:0/948457054 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfc40b9970 msgr2=0x7fcfc40b9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:52.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.327+0000 7fcfcb0f3640 1 --2- 172.21.15.98:0/948457054 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfc40b9970 0x7fcfc40b9de0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc009d90 tx=0x7fcfbc01c970 comp rx=0 tx=0).stop 2024-09-19T08:38:52.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.327+0000 7fcfcb0f3640 1 -- 172.21.15.98:0/948457054 shutdown_connections 2024-09-19T08:38:52.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.327+0000 7fcfcb0f3640 1 --2- 172.21.15.98:0/948457054 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfc40b9970 0x7fcfc40b9de0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:52.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.327+0000 7fcfcb0f3640 1 -- 172.21.15.98:0/948457054 >> 172.21.15.98:0/948457054 conn(0x7fcfc4074e40 msgr2=0x7fcfc4075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:52.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.327+0000 7fcfcb0f3640 1 -- 172.21.15.98:0/948457054 shutdown_connections 2024-09-19T08:38:52.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.327+0000 7fcfcb0f3640 1 -- 172.21.15.98:0/948457054 wait complete. 2024-09-19T08:38:52.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.327+0000 7fcfcb0f3640 1 Processor -- start 2024-09-19T08:38:52.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.327+0000 7fcfcb0f3640 1 -- start start 2024-09-19T08:38:52.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.327+0000 7fcfcb0f3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfc40b7f80 0x7fcfc413e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:52.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.327+0000 7fcfcb0f3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfc40b9970 0x7fcfc41f7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:52.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.327+0000 7fcfcb0f3640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcfc40d4820 0x7fcfc41406d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:52.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.327+0000 7fcfcb0f3640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc4144710 con 0x7fcfc40b7f80 2024-09-19T08:38:52.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.327+0000 7fcfcb0f3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc4144590 con 0x7fcfc40b9970 2024-09-19T08:38:52.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.327+0000 7fcfcb0f3640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcfc4144890 con 0x7fcfc40d4820 2024-09-19T08:38:52.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.327+0000 7fcfc8e8f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfc40b7f80 0x7fcfc413e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:52.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.327+0000 7fcfc8e8f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfc40b7f80 0x7fcfc413e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40758/0 (socket says 172.21.15.98:40758) 2024-09-19T08:38:52.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.327+0000 7fcfc8e8f640 1 -- 172.21.15.98:0/734847639 learned_addr learned my addr 172.21.15.98:0/734847639 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:52.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.327+0000 7fcfc3fff640 1 --2- 172.21.15.98:0/734847639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfc40b9970 0x7fcfc41f7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:52.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.331+0000 7fcfc9690640 1 --2- 172.21.15.98:0/734847639 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcfc40d4820 0x7fcfc41406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:52.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.331+0000 7fcfc8e8f640 1 -- 172.21.15.98:0/734847639 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcfc40d4820 msgr2=0x7fcfc41406d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:52.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.331+0000 7fcfc8e8f640 1 --2- 172.21.15.98:0/734847639 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcfc40d4820 0x7fcfc41406d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:52.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.331+0000 7fcfc8e8f640 1 -- 172.21.15.98:0/734847639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfc40b9970 msgr2=0x7fcfc41f7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:52.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.331+0000 7fcfc8e8f640 1 --2- 172.21.15.98:0/734847639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfc40b9970 0x7fcfc41f7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:52.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.331+0000 7fcfc8e8f640 1 -- 172.21.15.98:0/734847639 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc41f8590 con 0x7fcfc40b7f80 2024-09-19T08:38:52.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.331+0000 7fcfc8e8f640 1 --2- 172.21.15.98:0/734847639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfc40b7f80 0x7fcfc413e600 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcfb400c370 tx=0x7fcfb400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:52.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.331+0000 7fcfc17fa640 1 -- 172.21.15.98:0/734847639 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb40052d0 con 0x7fcfc40b7f80 2024-09-19T08:38:52.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.331+0000 7fcfc17fa640 1 -- 172.21.15.98:0/734847639 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfb4005470 con 0x7fcfc40b7f80 2024-09-19T08:38:52.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.331+0000 7fcfcb0f3640 1 -- 172.21.15.98:0/734847639 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfc41f8870 con 0x7fcfc40b7f80 2024-09-19T08:38:52.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.331+0000 7fcfc17fa640 1 -- 172.21.15.98:0/734847639 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb4007260 con 0x7fcfc40b7f80 2024-09-19T08:38:52.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.331+0000 7fcfcb0f3640 1 -- 172.21.15.98:0/734847639 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfc41f8e30 con 0x7fcfc40b7f80 2024-09-19T08:38:52.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.331+0000 7fcfcb0f3640 1 -- 172.21.15.98:0/734847639 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfc40b9de0 con 0x7fcfc40b7f80 2024-09-19T08:38:52.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.331+0000 7fcfc17fa640 1 -- 172.21.15.98:0/734847639 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcfb4007400 con 0x7fcfc40b7f80 2024-09-19T08:38:52.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.331+0000 7fcfc17fa640 1 --2- 172.21.15.98:0/734847639 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fcfb00678c0 0x7fcfb0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:52.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.331+0000 7fcfc17fa640 1 -- 172.21.15.98:0/734847639 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fcfb4084fd0 con 0x7fcfc40b7f80 2024-09-19T08:38:52.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.331+0000 7fcfc3fff640 1 --2- 172.21.15.98:0/734847639 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fcfb00678c0 0x7fcfb0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:52.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.331+0000 7fcfc3fff640 1 --2- 172.21.15.98:0/734847639 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fcfb00678c0 0x7fcfb0069d80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcfc415f740 tx=0x7fcfb8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:52.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.335+0000 7fcfc17fa640 1 -- 172.21.15.98:0/734847639 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fcfb4055550 con 0x7fcfc40b7f80 2024-09-19T08:38:52.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.507+0000 7fcfcb0f3640 1 -- 172.21.15.98:0/734847639 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcfc40688c0 con 0x7fcfc40b7f80 2024-09-19T08:38:52.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.507+0000 7fcfc17fa640 1 -- 172.21.15.98:0/734847639 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7fcfb4059850 con 0x7fcfc40b7f80 2024-09-19T08:38:52.512 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-19T08:38:52.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.511+0000 7fcfcb0f3640 1 -- 172.21.15.98:0/734847639 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fcfb00678c0 msgr2=0x7fcfb0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:52.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.511+0000 7fcfcb0f3640 1 --2- 172.21.15.98:0/734847639 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fcfb00678c0 0x7fcfb0069d80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcfc415f740 tx=0x7fcfb8009040 comp rx=0 tx=0).stop 2024-09-19T08:38:52.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.511+0000 7fcfcb0f3640 1 -- 172.21.15.98:0/734847639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfc40b7f80 msgr2=0x7fcfc413e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:52.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.511+0000 7fcfcb0f3640 1 --2- 172.21.15.98:0/734847639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfc40b7f80 0x7fcfc413e600 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcfb400c370 tx=0x7fcfb400c830 comp rx=0 tx=0).stop 2024-09-19T08:38:52.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.511+0000 7fcfcb0f3640 1 -- 172.21.15.98:0/734847639 shutdown_connections 2024-09-19T08:38:52.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.511+0000 7fcfcb0f3640 1 --2- 172.21.15.98:0/734847639 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fcfb00678c0 0x7fcfb0069d80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:52.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.511+0000 7fcfcb0f3640 1 --2- 172.21.15.98:0/734847639 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcfc40d4820 0x7fcfc41406d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:52.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.511+0000 7fcfcb0f3640 1 --2- 172.21.15.98:0/734847639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfc40b9970 0x7fcfc41f7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:52.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.511+0000 7fcfcb0f3640 1 --2- 172.21.15.98:0/734847639 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcfc40b7f80 0x7fcfc413e600 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:52.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.511+0000 7fcfcb0f3640 1 -- 172.21.15.98:0/734847639 >> 172.21.15.98:0/734847639 conn(0x7fcfc4074e40 msgr2=0x7fcfc40d4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:52.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.511+0000 7fcfcb0f3640 1 -- 172.21.15.98:0/734847639 shutdown_connections 2024-09-19T08:38:52.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.511+0000 7fcfcb0f3640 1 -- 172.21.15.98:0/734847639 wait complete. 2024-09-19T08:38:52.530 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:38:52.530 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:38:52.537 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T08:38:52.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.647+0000 7feed1990640 1 Processor -- start 2024-09-19T08:38:52.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.647+0000 7feed1990640 1 -- start start 2024-09-19T08:38:52.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.647+0000 7feed1990640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7feecc06c120 0x7feecc0cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:52.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.647+0000 7feed1990640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feecc0ccc60 con 0x7feecc06bc90 2024-09-19T08:38:52.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.647+0000 7feed1990640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feecc0ccf40 con 0x7feecc158fd0 2024-09-19T08:38:52.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.647+0000 7feed1990640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7feecc1593b0 con 0x7feecc06c120 2024-09-19T08:38:52.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.647+0000 7feec27fc640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7feecc06bc90 0x7feecc0ce840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:35962/0 (socket says 172.21.15.98:35962) 2024-09-19T08:38:52.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.647+0000 7feec27fc640 1 -- 172.21.15.98:0/3028591608 learned_addr learned my addr 172.21.15.98:0/3028591608 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:52.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.647+0000 7feecb7fe640 1 --2- 172.21.15.98:0/3028591608 >> v2:172.21.15.139:3300/0 conn(0x7feecc06c120 0x7feecc0cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:52.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.647+0000 7feeca7fc640 1 -- 172.21.15.98:0/3028591608 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4086003226 0 0) 0x7feecc0ccc60 con 0x7feecc06bc90 2024-09-19T08:38:52.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.647+0000 7feeca7fc640 1 -- 172.21.15.98:0/3028591608 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feeb8003530 con 0x7feecc06bc90 2024-09-19T08:38:52.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.647+0000 7feeca7fc640 1 -- 172.21.15.98:0/3028591608 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2695057403 0 0) 0x7feeb8003530 con 0x7feecc06bc90 2024-09-19T08:38:52.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.647+0000 7feeca7fc640 1 -- 172.21.15.98:0/3028591608 >> v2:172.21.15.139:3300/0 conn(0x7feecc06c120 msgr2=0x7feecc0cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:52.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.647+0000 7feeca7fc640 1 --2- 172.21.15.98:0/3028591608 >> v2:172.21.15.139:3300/0 conn(0x7feecc06c120 0x7feecc0cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:52.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.647+0000 7feeca7fc640 1 -- 172.21.15.98:0/3028591608 >> v1:172.21.15.139:6789/0 conn(0x7feecc158fd0 legacy=0x7feecc0b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:52.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.647+0000 7feeca7fc640 1 -- 172.21.15.98:0/3028591608 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feecc0cd220 con 0x7feecc06bc90 2024-09-19T08:38:52.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.647+0000 7feeca7fc640 1 -- 172.21.15.98:0/3028591608 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7feeb4001680 con 0x7feecc06bc90 2024-09-19T08:38:52.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.647+0000 7feeca7fc640 1 -- 172.21.15.98:0/3028591608 >> v1:172.21.15.98:6789/0 conn(0x7feecc06bc90 legacy=0x7feecc0ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:52.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.647+0000 7feeca7fc640 1 --2- 172.21.15.98:0/3028591608 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feeb80039c0 0x7feeb8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:52.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.647+0000 7feeca7fc640 1 --2- 172.21.15.98:0/3028591608 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feeb8004810 0x7feeb8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:52.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.647+0000 7feeca7fc640 1 --2- 172.21.15.98:0/3028591608 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feeb80051d0 0x7feeb800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:52.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.647+0000 7feeca7fc640 1 -- 172.21.15.98:0/3028591608 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7feecc1593b0 con 0x7feeb8004810 2024-09-19T08:38:52.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.647+0000 7feeca7fc640 1 -- 172.21.15.98:0/3028591608 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feeb800c170 con 0x7feeb80039c0 2024-09-19T08:38:52.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.647+0000 7feeca7fc640 1 -- 172.21.15.98:0/3028591608 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7feeb800c370 con 0x7feeb80051d0 2024-09-19T08:38:52.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.647+0000 7feec27fc640 1 --2- 172.21.15.98:0/3028591608 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feeb8004810 0x7feeb8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:52.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.647+0000 7feecb7fe640 1 --2- 172.21.15.98:0/3028591608 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feeb80051d0 0x7feeb800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:52.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.647+0000 7feec27fc640 1 -- 172.21.15.98:0/3028591608 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feeb80051d0 msgr2=0x7feeb800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:52.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.647+0000 7feecaffd640 1 --2- 172.21.15.98:0/3028591608 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feeb80039c0 0x7feeb8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:52.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.647+0000 7feec27fc640 1 --2- 172.21.15.98:0/3028591608 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feeb80051d0 0x7feeb800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:52.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.647+0000 7feec27fc640 1 -- 172.21.15.98:0/3028591608 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feeb80039c0 msgr2=0x7feeb8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:52.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.651+0000 7feec27fc640 1 --2- 172.21.15.98:0/3028591608 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feeb80039c0 0x7feeb8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:52.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.651+0000 7feec27fc640 1 -- 172.21.15.98:0/3028591608 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeb8003530 con 0x7feeb8004810 2024-09-19T08:38:52.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.651+0000 7feecb7fe640 1 --2- 172.21.15.98:0/3028591608 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feeb80051d0 0x7feeb800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:52.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.651+0000 7feecaffd640 1 --2- 172.21.15.98:0/3028591608 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feeb80039c0 0x7feeb8003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:52.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.651+0000 7feecb7fe640 1 -- 172.21.15.98:0/3028591608 reap_dead start 2024-09-19T08:38:52.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.651+0000 7feec27fc640 1 --2- 172.21.15.98:0/3028591608 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feeb8004810 0x7feeb8004c90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7feeb400c240 tx=0x7feeb401cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=174d7f036465db11 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:52.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.651+0000 7feeca7fc640 1 -- 172.21.15.98:0/3028591608 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb4007d20 con 0x7feeb8004810 2024-09-19T08:38:52.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.651+0000 7feeca7fc640 1 -- 172.21.15.98:0/3028591608 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb40a9070 con 0x7feeb8004810 2024-09-19T08:38:52.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.651+0000 7feeca7fc640 1 -- 172.21.15.98:0/3028591608 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb4005da0 con 0x7feeb8004810 2024-09-19T08:38:52.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.651+0000 7feed1990640 1 -- 172.21.15.98:0/3028591608 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feeb8004810 msgr2=0x7feeb8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:52.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.651+0000 7feed1990640 1 --2- 172.21.15.98:0/3028591608 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feeb8004810 0x7feeb8004c90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7feeb400c240 tx=0x7feeb401cb90 comp rx=0 tx=0).stop 2024-09-19T08:38:52.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.651+0000 7feed1990640 1 -- 172.21.15.98:0/3028591608 shutdown_connections 2024-09-19T08:38:52.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.651+0000 7feed1990640 1 --2- 172.21.15.98:0/3028591608 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feeb8004810 0x7feeb8004c90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:52.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.651+0000 7feed1990640 1 -- 172.21.15.98:0/3028591608 >> 172.21.15.98:0/3028591608 conn(0x7feecc0b2010 msgr2=0x7feecc0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:52.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.651+0000 7feed1990640 1 -- 172.21.15.98:0/3028591608 shutdown_connections 2024-09-19T08:38:52.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.651+0000 7feed1990640 1 -- 172.21.15.98:0/3028591608 wait complete. 2024-09-19T08:38:52.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.655+0000 7feed1990640 1 Processor -- start 2024-09-19T08:38:52.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.655+0000 7feed1990640 1 -- start start 2024-09-19T08:38:52.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.655+0000 7feed1990640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feeb8004810 0x7feecc151ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:52.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.655+0000 7feed1990640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feecc1f5e30 0x7feecc15f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:52.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.655+0000 7feed1990640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feecc1f62b0 0x7feecc15fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:52.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.655+0000 7feed1990640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7feecc15c920 con 0x7feecc1f62b0 2024-09-19T08:38:52.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.655+0000 7feed1990640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feecc15c7a0 con 0x7feecc1f5e30 2024-09-19T08:38:52.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.655+0000 7feed1990640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7feecc15efc0 con 0x7feeb8004810 2024-09-19T08:38:52.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.655+0000 7feecb7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feecc1f62b0 0x7feecc15fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:52.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.655+0000 7feecb7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feecc1f62b0 0x7feecc15fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40784/0 (socket says 172.21.15.98:40784) 2024-09-19T08:38:52.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.655+0000 7feecb7fe640 1 -- 172.21.15.98:0/1583967805 learned_addr learned my addr 172.21.15.98:0/1583967805 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:52.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.655+0000 7feecaffd640 1 --2- 172.21.15.98:0/1583967805 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feeb8004810 0x7feecc151ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:52.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.655+0000 7feec27fc640 1 --2- 172.21.15.98:0/1583967805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feecc1f5e30 0x7feecc15f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:52.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.655+0000 7feecb7fe640 1 -- 172.21.15.98:0/1583967805 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feeb8004810 msgr2=0x7feecc151ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:52.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.655+0000 7feecb7fe640 1 --2- 172.21.15.98:0/1583967805 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feeb8004810 0x7feecc151ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:52.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.655+0000 7feecb7fe640 1 -- 172.21.15.98:0/1583967805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feecc1f5e30 msgr2=0x7feecc15f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:52.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.655+0000 7feecb7fe640 1 --2- 172.21.15.98:0/1583967805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feecc1f5e30 0x7feecc15f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:52.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.655+0000 7feecb7fe640 1 -- 172.21.15.98:0/1583967805 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feecc152410 con 0x7feecc1f62b0 2024-09-19T08:38:52.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.655+0000 7feec27fc640 1 --2- 172.21.15.98:0/1583967805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feecc1f5e30 0x7feecc15f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:38:52.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.655+0000 7feecb7fe640 1 --2- 172.21.15.98:0/1583967805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feecc1f62b0 0x7feecc15fa40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7feecc15c770 tx=0x7feec400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:52.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.655+0000 7feec8ff9640 1 -- 172.21.15.98:0/1583967805 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec4016620 con 0x7feecc1f62b0 2024-09-19T08:38:52.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.655+0000 7feec8ff9640 1 -- 172.21.15.98:0/1583967805 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec401b070 con 0x7feecc1f62b0 2024-09-19T08:38:52.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.655+0000 7feec8ff9640 1 -- 172.21.15.98:0/1583967805 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec40169e0 con 0x7feecc1f62b0 2024-09-19T08:38:52.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.655+0000 7feed1990640 1 -- 172.21.15.98:0/1583967805 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feecc155500 con 0x7feecc1f62b0 2024-09-19T08:38:52.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.655+0000 7feed1990640 1 -- 172.21.15.98:0/1583967805 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feecc152a30 con 0x7feecc1f62b0 2024-09-19T08:38:52.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.655+0000 7feed1990640 1 -- 172.21.15.98:0/1583967805 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feecc15f2f0 con 0x7feecc1f62b0 2024-09-19T08:38:52.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.655+0000 7feec8ff9640 1 -- 172.21.15.98:0/1583967805 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7feec4016b80 con 0x7feecc1f62b0 2024-09-19T08:38:52.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.659+0000 7feec8ff9640 1 --2- 172.21.15.98:0/1583967805 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7feeac0678c0 0x7feeac069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:52.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.659+0000 7feecaffd640 1 --2- 172.21.15.98:0/1583967805 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7feeac0678c0 0x7feeac069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:52.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.659+0000 7feec8ff9640 1 -- 172.21.15.98:0/1583967805 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7feec4086b50 con 0x7feecc1f62b0 2024-09-19T08:38:52.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.659+0000 7feecaffd640 1 --2- 172.21.15.98:0/1583967805 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7feeac0678c0 0x7feeac069d80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7feebc007fd0 tx=0x7feebc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:52.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.663+0000 7feec8ff9640 1 -- 172.21.15.98:0/1583967805 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7feec40575f0 con 0x7feecc1f62b0 2024-09-19T08:38:52.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.835+0000 7feed1990640 1 -- 172.21.15.98:0/1583967805 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7feecc0757a0 con 0x7feecc1f62b0 2024-09-19T08:38:52.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.835+0000 7feec8ff9640 1 -- 172.21.15.98:0/1583967805 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7feec4056ef0 con 0x7feecc1f62b0 2024-09-19T08:38:52.839 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-19T08:38:52.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.835+0000 7feed1990640 1 -- 172.21.15.98:0/1583967805 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7feeac0678c0 msgr2=0x7feeac069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:52.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.835+0000 7feed1990640 1 --2- 172.21.15.98:0/1583967805 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7feeac0678c0 0x7feeac069d80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7feebc007fd0 tx=0x7feebc009040 comp rx=0 tx=0).stop 2024-09-19T08:38:52.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.835+0000 7feed1990640 1 -- 172.21.15.98:0/1583967805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feecc1f62b0 msgr2=0x7feecc15fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:52.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.835+0000 7feed1990640 1 --2- 172.21.15.98:0/1583967805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feecc1f62b0 0x7feecc15fa40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7feecc15c770 tx=0x7feec400f520 comp rx=0 tx=0).stop 2024-09-19T08:38:52.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.835+0000 7feed1990640 1 -- 172.21.15.98:0/1583967805 shutdown_connections 2024-09-19T08:38:52.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.839+0000 7feed1990640 1 --2- 172.21.15.98:0/1583967805 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7feeac0678c0 0x7feeac069d80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:52.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.839+0000 7feed1990640 1 --2- 172.21.15.98:0/1583967805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feecc1f62b0 0x7feecc15fa40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:52.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.839+0000 7feed1990640 1 --2- 172.21.15.98:0/1583967805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feecc1f5e30 0x7feecc15f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:52.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.839+0000 7feed1990640 1 --2- 172.21.15.98:0/1583967805 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feeb8004810 0x7feecc151ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:52.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.839+0000 7feed1990640 1 -- 172.21.15.98:0/1583967805 >> 172.21.15.98:0/1583967805 conn(0x7feecc0b2010 msgr2=0x7feecc15df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:52.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.839+0000 7feed1990640 1 -- 172.21.15.98:0/1583967805 shutdown_connections 2024-09-19T08:38:52.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:52.839+0000 7feed1990640 1 -- 172.21.15.98:0/1583967805 wait complete. 2024-09-19T08:38:52.859 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi139.front.sepia.ceph.com... 2024-09-19T08:38:52.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:52.859 DEBUG:teuthology.orchestra.run.smithi139:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-09-19T08:38:52.874 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:38:52.875 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:38:52.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:38:52.875 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-19T08:38:52.921 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-19T08:38:52.922 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:38:53.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.039+0000 7fe7a9cfd640 1 Processor -- start 2024-09-19T08:38:53.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.039+0000 7fe7a9cfd640 1 -- start start 2024-09-19T08:38:53.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.039+0000 7fe7a9cfd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe7a415c550 0x7fe7a406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:53.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.039+0000 7fe7a9cfd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7a406c1d0 0x7fe7a40cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:53.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.039+0000 7fe7a9cfd640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7a406d310 con 0x7fe7a40cf900 2024-09-19T08:38:53.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.039+0000 7fe7a9cfd640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe7a406c610 con 0x7fe7a415c550 2024-09-19T08:38:53.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.039+0000 7fe7a9cfd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe7a406d5f0 con 0x7fe7a406c1d0 2024-09-19T08:38:53.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.039+0000 7fe7a3fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fe7a40cf900 0x7fe7a40ce1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:35970/0 (socket says 172.21.15.98:35970) 2024-09-19T08:38:53.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.039+0000 7fe7a3fff640 1 -- 172.21.15.98:0/2933693270 learned_addr learned my addr 172.21.15.98:0/2933693270 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:53.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.039+0000 7fe7a2ffd640 1 --2- 172.21.15.98:0/2933693270 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7a406c1d0 0x7fe7a40cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:53.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.039+0000 7fe7a37fe640 1 --2- 172.21.15.98:0/2933693270 >> v2:172.21.15.139:3300/0 conn(0x7fe7a415c550 0x7fe7a406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:53.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.039+0000 7fe7a27fc640 1 -- 172.21.15.98:0/2933693270 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1731189647 0 0) 0x7fe7a406d310 con 0x7fe7a40cf900 2024-09-19T08:38:53.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.039+0000 7fe7a27fc640 1 -- 172.21.15.98:0/2933693270 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe790003530 con 0x7fe7a40cf900 2024-09-19T08:38:53.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.039+0000 7fe7a27fc640 1 -- 172.21.15.98:0/2933693270 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2088343171 0 0) 0x7fe790003530 con 0x7fe7a40cf900 2024-09-19T08:38:53.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.039+0000 7fe7a27fc640 1 -- 172.21.15.98:0/2933693270 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7a406c1d0 msgr2=0x7fe7a40cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:53.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.039+0000 7fe7a27fc640 1 --2- 172.21.15.98:0/2933693270 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7a406c1d0 0x7fe7a40cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:53.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.039+0000 7fe7a27fc640 1 -- 172.21.15.98:0/2933693270 >> v2:172.21.15.139:3300/0 conn(0x7fe7a415c550 msgr2=0x7fe7a406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:53.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.039+0000 7fe7a27fc640 1 --2- 172.21.15.98:0/2933693270 >> v2:172.21.15.139:3300/0 conn(0x7fe7a415c550 0x7fe7a406bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:53.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.039+0000 7fe7a27fc640 1 -- 172.21.15.98:0/2933693270 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7a406d8d0 con 0x7fe7a40cf900 2024-09-19T08:38:53.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.039+0000 7fe7a27fc640 1 -- 172.21.15.98:0/2933693270 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fe79c001a00 con 0x7fe7a40cf900 2024-09-19T08:38:53.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.039+0000 7fe7a27fc640 1 -- 172.21.15.98:0/2933693270 >> v1:172.21.15.98:6789/0 conn(0x7fe7a40cf900 legacy=0x7fe7a40ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:53.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.039+0000 7fe7a27fc640 1 --2- 172.21.15.98:0/2933693270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7900039c0 0x7fe790003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:53.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.039+0000 7fe7a27fc640 1 --2- 172.21.15.98:0/2933693270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe790004810 0x7fe790004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:53.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.039+0000 7fe7a27fc640 1 --2- 172.21.15.98:0/2933693270 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7a406c1d0 0x7fe7900051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:53.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.039+0000 7fe7a27fc640 1 -- 172.21.15.98:0/2933693270 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a406c610 con 0x7fe790004810 2024-09-19T08:38:53.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.039+0000 7fe7a27fc640 1 -- 172.21.15.98:0/2933693270 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a406d5f0 con 0x7fe7900039c0 2024-09-19T08:38:53.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.039+0000 7fe7a27fc640 1 -- 172.21.15.98:0/2933693270 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe79000b9d0 con 0x7fe7a406c1d0 2024-09-19T08:38:53.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.039+0000 7fe7a2ffd640 1 --2- 172.21.15.98:0/2933693270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe790004810 0x7fe790004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:53.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.039+0000 7fe7a37fe640 1 --2- 172.21.15.98:0/2933693270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7900039c0 0x7fe790003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:53.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.039+0000 7fe7a3fff640 1 --2- 172.21.15.98:0/2933693270 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7a406c1d0 0x7fe7900051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:53.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.039+0000 7fe7a2ffd640 1 -- 172.21.15.98:0/2933693270 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7a406c1d0 msgr2=0x7fe7900051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:53.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.039+0000 7fe7a2ffd640 1 --2- 172.21.15.98:0/2933693270 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7a406c1d0 0x7fe7900051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:53.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.039+0000 7fe7a2ffd640 1 -- 172.21.15.98:0/2933693270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7900039c0 msgr2=0x7fe790003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:53.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.039+0000 7fe7a2ffd640 1 --2- 172.21.15.98:0/2933693270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7900039c0 0x7fe790003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:53.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.039+0000 7fe7a2ffd640 1 -- 172.21.15.98:0/2933693270 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe790003530 con 0x7fe790004810 2024-09-19T08:38:53.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.043+0000 7fe7a2ffd640 1 --2- 172.21.15.98:0/2933693270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe790004810 0x7fe790004c90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe78c009ae0 tx=0x7fe78c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b753f57974e4eadc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:53.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.043+0000 7fe7a27fc640 1 -- 172.21.15.98:0/2933693270 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c0b2070 con 0x7fe790004810 2024-09-19T08:38:53.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.043+0000 7fe7a27fc640 1 -- 172.21.15.98:0/2933693270 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe78c0a5580 con 0x7fe790004810 2024-09-19T08:38:53.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.043+0000 7fe7a27fc640 1 -- 172.21.15.98:0/2933693270 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c0096b0 con 0x7fe790004810 2024-09-19T08:38:53.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.043+0000 7fe7a9cfd640 1 -- 172.21.15.98:0/2933693270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe790004810 msgr2=0x7fe790004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:53.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.043+0000 7fe7a9cfd640 1 --2- 172.21.15.98:0/2933693270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe790004810 0x7fe790004c90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe78c009ae0 tx=0x7fe78c01ca70 comp rx=0 tx=0).stop 2024-09-19T08:38:53.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.043+0000 7fe7a3fff640 1 -- 172.21.15.98:0/2933693270 reap_dead start 2024-09-19T08:38:53.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.043+0000 7fe7a9cfd640 1 -- 172.21.15.98:0/2933693270 shutdown_connections 2024-09-19T08:38:53.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.043+0000 7fe7a9cfd640 1 -- 172.21.15.98:0/2933693270 >> 172.21.15.98:0/2933693270 conn(0x7fe7a40b2010 msgr2=0x7fe7a40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:53.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.043+0000 7fe7a9cfd640 1 -- 172.21.15.98:0/2933693270 shutdown_connections 2024-09-19T08:38:53.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.043+0000 7fe7a9cfd640 1 -- 172.21.15.98:0/2933693270 wait complete. 2024-09-19T08:38:53.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.043+0000 7fe7a9cfd640 1 Processor -- start 2024-09-19T08:38:53.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.043+0000 7fe7a9cfd640 1 -- start start 2024-09-19T08:38:53.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.043+0000 7fe7a9cfd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe790004810 0x7fe7a414ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:53.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.043+0000 7fe7a9cfd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7a415de70 0x7fe7a41e72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:53.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.043+0000 7fe7a9cfd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7a415e2d0 0x7fe7a4152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:53.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.043+0000 7fe7a9cfd640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a415c930 con 0x7fe7a415e2d0 2024-09-19T08:38:53.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.043+0000 7fe7a9cfd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a40cfd90 con 0x7fe790004810 2024-09-19T08:38:53.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.047+0000 7fe7a9cfd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe7a415cab0 con 0x7fe7a415de70 2024-09-19T08:38:53.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.047+0000 7fe7a3fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7a415e2d0 0x7fe7a4152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:53.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.047+0000 7fe7a3fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7a415e2d0 0x7fe7a4152f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40798/0 (socket says 172.21.15.98:40798) 2024-09-19T08:38:53.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.047+0000 7fe7a3fff640 1 -- 172.21.15.98:0/4125909007 learned_addr learned my addr 172.21.15.98:0/4125909007 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:53.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.047+0000 7fe7a37fe640 1 --2- 172.21.15.98:0/4125909007 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe790004810 0x7fe7a414ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:53.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.047+0000 7fe7a2ffd640 1 --2- 172.21.15.98:0/4125909007 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7a415de70 0x7fe7a41e72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:53.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.047+0000 7fe7a3fff640 1 -- 172.21.15.98:0/4125909007 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7a415de70 msgr2=0x7fe7a41e72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:53.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.047+0000 7fe7a3fff640 1 --2- 172.21.15.98:0/4125909007 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7a415de70 0x7fe7a41e72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:53.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.047+0000 7fe7a3fff640 1 -- 172.21.15.98:0/4125909007 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe790004810 msgr2=0x7fe7a414ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:53.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.047+0000 7fe7a3fff640 1 --2- 172.21.15.98:0/4125909007 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe790004810 0x7fe7a414ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:53.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.047+0000 7fe7a3fff640 1 -- 172.21.15.98:0/4125909007 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7a415cdd0 con 0x7fe7a415e2d0 2024-09-19T08:38:53.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.047+0000 7fe7a3fff640 1 --2- 172.21.15.98:0/4125909007 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7a415e2d0 0x7fe7a4152f50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe7a40d12d0 tx=0x7fe79c005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:53.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.047+0000 7fe79bfff640 1 -- 172.21.15.98:0/4125909007 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c01a070 con 0x7fe7a415e2d0 2024-09-19T08:38:53.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.047+0000 7fe79bfff640 1 -- 172.21.15.98:0/4125909007 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe79c011040 con 0x7fe7a415e2d0 2024-09-19T08:38:53.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.047+0000 7fe7a9cfd640 1 -- 172.21.15.98:0/4125909007 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7a415d0b0 con 0x7fe7a415e2d0 2024-09-19T08:38:53.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.047+0000 7fe79bfff640 1 -- 172.21.15.98:0/4125909007 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c00edc0 con 0x7fe7a415e2d0 2024-09-19T08:38:53.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.047+0000 7fe7a9cfd640 1 -- 172.21.15.98:0/4125909007 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7a415d670 con 0x7fe7a415e2d0 2024-09-19T08:38:53.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.047+0000 7fe7a9cfd640 1 -- 172.21.15.98:0/4125909007 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7a406bc90 con 0x7fe7a415e2d0 2024-09-19T08:38:53.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.047+0000 7fe79bfff640 1 -- 172.21.15.98:0/4125909007 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe79c01d070 con 0x7fe7a415e2d0 2024-09-19T08:38:53.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.047+0000 7fe79bfff640 1 --2- 172.21.15.98:0/4125909007 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe7840678c0 0x7fe784069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:53.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.047+0000 7fe7a37fe640 1 --2- 172.21.15.98:0/4125909007 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe7840678c0 0x7fe784069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:53.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.047+0000 7fe79bfff640 1 -- 172.21.15.98:0/4125909007 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe79c088650 con 0x7fe7a415e2d0 2024-09-19T08:38:53.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.047+0000 7fe7a37fe640 1 --2- 172.21.15.98:0/4125909007 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe7840678c0 0x7fe784069d80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe794009ab0 tx=0x7fe794008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:53.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.051+0000 7fe79bfff640 1 -- 172.21.15.98:0/4125909007 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe79c059100 con 0x7fe7a415e2d0 2024-09-19T08:38:53.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.251+0000 7fe7a9cfd640 1 -- 172.21.15.98:0/4125909007 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7a415a810 con 0x7fe7a415e2d0 2024-09-19T08:38:53.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.251+0000 7fe79bfff640 1 -- 172.21.15.98:0/4125909007 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5220 (secure 0 0 0) 0x7fe79c058a00 con 0x7fe7a415e2d0 2024-09-19T08:38:53.255 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:38:53.255 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":7,"btime":"2024-09-19T08:38:22:407519+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/2038744461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":2038744461},{"type":"v1","addr":"172.21.15.139:6833","nonce":2038744461}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/765134249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":765134249},{"type":"v1","addr":"172.21.15.98:6835","nonce":765134249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2832328076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2832328076},{"type":"v1","addr":"172.21.15.98:6837","nonce":2832328076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:38:18.717221+0000","modified":"2024-09-19T08:38:21.769948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6835/1879089763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1879089763},{"type":"v1","addr":"172.21.15.139:6835","nonce":1879089763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4261,"qdb_cluster":[4261]},"id":1}]} 2024-09-19T08:38:53.256 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 7 2024-09-19T08:38:53.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.255+0000 7fe7a9cfd640 1 -- 172.21.15.98:0/4125909007 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe7840678c0 msgr2=0x7fe784069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:53.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.255+0000 7fe7a9cfd640 1 --2- 172.21.15.98:0/4125909007 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe7840678c0 0x7fe784069d80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe794009ab0 tx=0x7fe794008040 comp rx=0 tx=0).stop 2024-09-19T08:38:53.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.255+0000 7fe7a9cfd640 1 -- 172.21.15.98:0/4125909007 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7a415e2d0 msgr2=0x7fe7a4152f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:53.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.255+0000 7fe7a9cfd640 1 --2- 172.21.15.98:0/4125909007 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7a415e2d0 0x7fe7a4152f50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe7a40d12d0 tx=0x7fe79c005b00 comp rx=0 tx=0).stop 2024-09-19T08:38:53.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.255+0000 7fe7a9cfd640 1 -- 172.21.15.98:0/4125909007 shutdown_connections 2024-09-19T08:38:53.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.255+0000 7fe7a9cfd640 1 --2- 172.21.15.98:0/4125909007 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe7840678c0 0x7fe784069d80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:53.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.255+0000 7fe7a9cfd640 1 --2- 172.21.15.98:0/4125909007 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7a415e2d0 0x7fe7a4152f50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:53.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.255+0000 7fe7a9cfd640 1 --2- 172.21.15.98:0/4125909007 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7a415de70 0x7fe7a41e72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:53.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.255+0000 7fe7a9cfd640 1 --2- 172.21.15.98:0/4125909007 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe790004810 0x7fe7a414ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:53.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.255+0000 7fe7a9cfd640 1 -- 172.21.15.98:0/4125909007 >> 172.21.15.98:0/4125909007 conn(0x7fe7a40b2010 msgr2=0x7fe7a4073c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:53.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.255+0000 7fe7a9cfd640 1 -- 172.21.15.98:0/4125909007 shutdown_connections 2024-09-19T08:38:53.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.255+0000 7fe7a9cfd640 1 -- 172.21.15.98:0/4125909007 wait complete. 2024-09-19T08:38:53.274 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:38:53.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.383+0000 7ff114cf0640 1 Processor -- start 2024-09-19T08:38:53.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.383+0000 7ff114cf0640 1 -- start start 2024-09-19T08:38:53.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.387+0000 7ff114cf0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff11006f9a0 0x7ff11006da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:53.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.387+0000 7ff114cf0640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff1100aff00 0x7ff1100cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:53.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.387+0000 7ff114cf0640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff110072ad0 con 0x7ff1100cfba0 2024-09-19T08:38:53.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.387+0000 7ff114cf0640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7ff11006dfb0 con 0x7ff1100aff00 2024-09-19T08:38:53.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.387+0000 7ff114cf0640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff1100cb440 con 0x7ff11006f9a0 2024-09-19T08:38:53.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.387+0000 7ff10dd9b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff1100aff00 0x7ff1100cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:53.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.387+0000 7ff10dd9b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff1100aff00 0x7ff1100cf660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40806/0 (socket says 172.21.15.98:40806) 2024-09-19T08:38:53.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.387+0000 7ff10dd9b640 1 -- 172.21.15.98:0/1745838507 learned_addr learned my addr 172.21.15.98:0/1745838507 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:53.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.387+0000 7ff10e59c640 1 --2- 172.21.15.98:0/1745838507 >> v2:172.21.15.139:3300/0 conn(0x7ff11006f9a0 0x7ff11006da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:53.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.387+0000 7ff10dd9b640 1 -- 172.21.15.98:0/1745838507 >> v2:172.21.15.139:3300/0 conn(0x7ff11006f9a0 msgr2=0x7ff11006da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:53.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.387+0000 7ff10dd9b640 1 --2- 172.21.15.98:0/1745838507 >> v2:172.21.15.139:3300/0 conn(0x7ff11006f9a0 0x7ff11006da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:53.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.387+0000 7ff10dd9b640 1 -- 172.21.15.98:0/1745838507 >> v1:172.21.15.98:6789/0 conn(0x7ff1100cfba0 legacy=0x7ff1100cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:53.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.387+0000 7ff10d59a640 1 -- 172.21.15.98:0/1745838507 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2463855451 0 0) 0x7ff110072ad0 con 0x7ff1100cfba0 2024-09-19T08:38:53.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.387+0000 7ff10dd9b640 1 -- 172.21.15.98:0/1745838507 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1100cb720 con 0x7ff1100aff00 2024-09-19T08:38:53.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.387+0000 7ff10dd9b640 1 --2- 172.21.15.98:0/1745838507 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1100aff00 0x7ff1100cf660 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff110075990 tx=0x7ff10001eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db78d0ddc3d16bc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:53.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.387+0000 7ff10d59a640 1 -- 172.21.15.98:0/1745838507 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1000b4070 con 0x7ff1100aff00 2024-09-19T08:38:53.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.387+0000 7ff10d59a640 1 -- 172.21.15.98:0/1745838507 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1000a7930 con 0x7ff1100aff00 2024-09-19T08:38:53.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.387+0000 7ff10d59a640 1 -- 172.21.15.98:0/1745838507 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1000a7d00 con 0x7ff1100aff00 2024-09-19T08:38:53.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.391+0000 7ff114cf0640 1 -- 172.21.15.98:0/1745838507 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1100aff00 msgr2=0x7ff1100cf660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:53.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.391+0000 7ff114cf0640 1 --2- 172.21.15.98:0/1745838507 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1100aff00 0x7ff1100cf660 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff110075990 tx=0x7ff10001eb60 comp rx=0 tx=0).stop 2024-09-19T08:38:53.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.391+0000 7ff114cf0640 1 -- 172.21.15.98:0/1745838507 shutdown_connections 2024-09-19T08:38:53.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.391+0000 7ff114cf0640 1 --2- 172.21.15.98:0/1745838507 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1100aff00 0x7ff1100cf660 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:53.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.391+0000 7ff114cf0640 1 --2- 172.21.15.98:0/1745838507 >> v2:172.21.15.139:3300/0 conn(0x7ff11006f9a0 0x7ff11006da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:53.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.391+0000 7ff114cf0640 1 -- 172.21.15.98:0/1745838507 >> 172.21.15.98:0/1745838507 conn(0x7ff1100d0980 msgr2=0x7ff1100d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:53.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.391+0000 7ff114cf0640 1 -- 172.21.15.98:0/1745838507 shutdown_connections 2024-09-19T08:38:53.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.391+0000 7ff114cf0640 1 -- 172.21.15.98:0/1745838507 wait complete. 2024-09-19T08:38:53.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.391+0000 7ff114cf0640 1 Processor -- start 2024-09-19T08:38:53.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.391+0000 7ff114cf0640 1 -- start start 2024-09-19T08:38:53.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.391+0000 7ff114cf0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff11006f9a0 0x7ff110157a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:53.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.391+0000 7ff114cf0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1100aff00 0x7ff11015ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:53.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.391+0000 7ff114cf0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1100cfba0 0x7ff1100b0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:53.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.391+0000 7ff114cf0640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff1100cd000 con 0x7ff11006f9a0 2024-09-19T08:38:53.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.391+0000 7ff114cf0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff1100cfff0 con 0x7ff1100aff00 2024-09-19T08:38:53.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.391+0000 7ff114cf0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff1100cd180 con 0x7ff1100cfba0 2024-09-19T08:38:53.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.391+0000 7ff10e59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff11006f9a0 0x7ff110157a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:53.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.391+0000 7ff10e59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff11006f9a0 0x7ff110157a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40818/0 (socket says 172.21.15.98:40818) 2024-09-19T08:38:53.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.391+0000 7ff10e59c640 1 -- 172.21.15.98:0/2155550168 learned_addr learned my addr 172.21.15.98:0/2155550168 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:53.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.391+0000 7ff10dd9b640 1 --2- 172.21.15.98:0/2155550168 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1100aff00 0x7ff11015ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:53.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.391+0000 7ff10e59c640 1 -- 172.21.15.98:0/2155550168 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1100cfba0 msgr2=0x7ff1100b0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:53.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.391+0000 7ff10ed9d640 1 --2- 172.21.15.98:0/2155550168 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1100cfba0 0x7ff1100b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:53.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.391+0000 7ff10e59c640 1 --2- 172.21.15.98:0/2155550168 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1100cfba0 0x7ff1100b0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:53.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.391+0000 7ff10e59c640 1 -- 172.21.15.98:0/2155550168 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1100aff00 msgr2=0x7ff11015ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:53.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.391+0000 7ff10e59c640 1 --2- 172.21.15.98:0/2155550168 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1100aff00 0x7ff11015ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:53.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.391+0000 7ff10e59c640 1 -- 172.21.15.98:0/2155550168 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1100c7e80 con 0x7ff11006f9a0 2024-09-19T08:38:53.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.391+0000 7ff10dd9b640 1 --2- 172.21.15.98:0/2155550168 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1100aff00 0x7ff11015ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:53.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.391+0000 7ff10e59c640 1 --2- 172.21.15.98:0/2155550168 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff11006f9a0 0x7ff110157a10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff108007e40 tx=0x7ff10800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:53.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.391+0000 7ff106ffd640 1 -- 172.21.15.98:0/2155550168 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff108014620 con 0x7ff11006f9a0 2024-09-19T08:38:53.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.391+0000 7ff106ffd640 1 -- 172.21.15.98:0/2155550168 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff108019070 con 0x7ff11006f9a0 2024-09-19T08:38:53.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.391+0000 7ff106ffd640 1 -- 172.21.15.98:0/2155550168 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff108014a00 con 0x7ff11006f9a0 2024-09-19T08:38:53.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.391+0000 7ff114cf0640 1 -- 172.21.15.98:0/2155550168 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1100c8160 con 0x7ff11006f9a0 2024-09-19T08:38:53.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.391+0000 7ff114cf0640 1 -- 172.21.15.98:0/2155550168 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1100c8720 con 0x7ff11006f9a0 2024-09-19T08:38:53.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.395+0000 7ff114cf0640 1 -- 172.21.15.98:0/2155550168 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff11006fe10 con 0x7ff11006f9a0 2024-09-19T08:38:53.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.395+0000 7ff106ffd640 1 -- 172.21.15.98:0/2155550168 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff108014ba0 con 0x7ff11006f9a0 2024-09-19T08:38:53.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.395+0000 7ff106ffd640 1 --2- 172.21.15.98:0/2155550168 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff0f00678c0 0x7ff0f0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:53.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.395+0000 7ff10dd9b640 1 --2- 172.21.15.98:0/2155550168 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff0f00678c0 0x7ff0f0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:53.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.395+0000 7ff106ffd640 1 -- 172.21.15.98:0/2155550168 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff108086480 con 0x7ff11006f9a0 2024-09-19T08:38:53.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.395+0000 7ff10dd9b640 1 --2- 172.21.15.98:0/2155550168 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff0f00678c0 0x7ff0f0069d80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff1000b1000 tx=0x7ff1000a9070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:53.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.399+0000 7ff106ffd640 1 -- 172.21.15.98:0/2155550168 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff108057050 con 0x7ff11006f9a0 2024-09-19T08:38:53.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.579+0000 7ff114cf0640 1 -- 172.21.15.98:0/2155550168 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1100d0690 con 0x7ff11006f9a0 2024-09-19T08:38:53.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.579+0000 7ff106ffd640 1 -- 172.21.15.98:0/2155550168 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5220 (secure 0 0 0) 0x7ff108056950 con 0x7ff11006f9a0 2024-09-19T08:38:53.585 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:38:53.585 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":7,"btime":"2024-09-19T08:38:22:407519+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/2038744461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":2038744461},{"type":"v1","addr":"172.21.15.139:6833","nonce":2038744461}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/765134249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":765134249},{"type":"v1","addr":"172.21.15.98:6835","nonce":765134249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2832328076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2832328076},{"type":"v1","addr":"172.21.15.98:6837","nonce":2832328076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:38:18.717221+0000","modified":"2024-09-19T08:38:21.769948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6835/1879089763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1879089763},{"type":"v1","addr":"172.21.15.139:6835","nonce":1879089763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4261,"qdb_cluster":[4261]},"id":1}]} 2024-09-19T08:38:53.585 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 7 2024-09-19T08:38:53.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.583+0000 7ff114cf0640 1 -- 172.21.15.98:0/2155550168 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff0f00678c0 msgr2=0x7ff0f0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:53.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.583+0000 7ff114cf0640 1 --2- 172.21.15.98:0/2155550168 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff0f00678c0 0x7ff0f0069d80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff1000b1000 tx=0x7ff1000a9070 comp rx=0 tx=0).stop 2024-09-19T08:38:53.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.583+0000 7ff114cf0640 1 -- 172.21.15.98:0/2155550168 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff11006f9a0 msgr2=0x7ff110157a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:53.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.583+0000 7ff114cf0640 1 --2- 172.21.15.98:0/2155550168 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff11006f9a0 0x7ff110157a10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff108007e40 tx=0x7ff10800e4b0 comp rx=0 tx=0).stop 2024-09-19T08:38:53.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.583+0000 7ff114cf0640 1 -- 172.21.15.98:0/2155550168 shutdown_connections 2024-09-19T08:38:53.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.583+0000 7ff114cf0640 1 --2- 172.21.15.98:0/2155550168 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff0f00678c0 0x7ff0f0069d80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:53.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.583+0000 7ff114cf0640 1 --2- 172.21.15.98:0/2155550168 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1100cfba0 0x7ff1100b0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:53.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.583+0000 7ff114cf0640 1 --2- 172.21.15.98:0/2155550168 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1100aff00 0x7ff11015ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:53.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.583+0000 7ff114cf0640 1 --2- 172.21.15.98:0/2155550168 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff11006f9a0 0x7ff110157a10 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:53.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.583+0000 7ff114cf0640 1 -- 172.21.15.98:0/2155550168 >> 172.21.15.98:0/2155550168 conn(0x7ff1100d0980 msgr2=0x7ff1100cc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:53.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.583+0000 7ff114cf0640 1 -- 172.21.15.98:0/2155550168 shutdown_connections 2024-09-19T08:38:53.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.583+0000 7ff114cf0640 1 -- 172.21.15.98:0/2155550168 wait complete. 2024-09-19T08:38:53.602 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:38:53.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.707+0000 7f8013e9c640 1 Processor -- start 2024-09-19T08:38:53.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.711+0000 7f8013e9c640 1 -- start start 2024-09-19T08:38:53.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.711+0000 7f8013e9c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f800c0733b0 0x7f800c0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:53.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.711+0000 7f8013e9c640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f800c06d310 con 0x7f800c15c550 2024-09-19T08:38:53.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.711+0000 7f8013e9c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f800c06d5f0 con 0x7f800c06c3a0 2024-09-19T08:38:53.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.711+0000 7f8013e9c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f800c06d8d0 con 0x7f800c0733b0 2024-09-19T08:38:53.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.711+0000 7f8011c38640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f800c15c550 0x7f800c06bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:35986/0 (socket says 172.21.15.98:35986) 2024-09-19T08:38:53.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.711+0000 7f8011c38640 1 -- 172.21.15.98:0/1048588336 learned_addr learned my addr 172.21.15.98:0/1048588336 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:53.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.711+0000 7f8010c36640 1 -- 172.21.15.98:0/1048588336 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2036330728 0 0) 0x7f800c06d310 con 0x7f800c15c550 2024-09-19T08:38:53.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.711+0000 7f8012439640 1 --2- 172.21.15.98:0/1048588336 >> v2:172.21.15.139:3300/0 conn(0x7f800c0733b0 0x7f800c0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:53.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.711+0000 7f8010c36640 1 -- 172.21.15.98:0/1048588336 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ff8003530 con 0x7f800c15c550 2024-09-19T08:38:53.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.711+0000 7f8010c36640 1 -- 172.21.15.98:0/1048588336 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 332332577 0 0) 0x7f7ff8003530 con 0x7f800c15c550 2024-09-19T08:38:53.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.711+0000 7f8010c36640 1 -- 172.21.15.98:0/1048588336 >> v2:172.21.15.139:3300/0 conn(0x7f800c0733b0 msgr2=0x7f800c0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:53.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.711+0000 7f8010c36640 1 --2- 172.21.15.98:0/1048588336 >> v2:172.21.15.139:3300/0 conn(0x7f800c0733b0 0x7f800c0cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:53.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.711+0000 7f8010c36640 1 -- 172.21.15.98:0/1048588336 >> v1:172.21.15.139:6789/0 conn(0x7f800c06c3a0 legacy=0x7f800c0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:53.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.711+0000 7f8010c36640 1 -- 172.21.15.98:0/1048588336 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f800c06dbb0 con 0x7f800c15c550 2024-09-19T08:38:53.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.711+0000 7f8010c36640 1 -- 172.21.15.98:0/1048588336 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f8008001680 con 0x7f800c15c550 2024-09-19T08:38:53.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.711+0000 7f8010c36640 1 -- 172.21.15.98:0/1048588336 >> v1:172.21.15.98:6789/0 conn(0x7f800c15c550 legacy=0x7f800c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:53.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.711+0000 7f8010c36640 1 --2- 172.21.15.98:0/1048588336 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ff8003a10 0x7f7ff8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:53.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.711+0000 7f8010c36640 1 --2- 172.21.15.98:0/1048588336 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ff8004810 0x7f7ff8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:53.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.711+0000 7f8010c36640 1 --2- 172.21.15.98:0/1048588336 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ff80051d0 0x7f7ff800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:53.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.711+0000 7f8010c36640 1 -- 172.21.15.98:0/1048588336 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f800c06d8d0 con 0x7f7ff8004810 2024-09-19T08:38:53.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.711+0000 7f8010c36640 1 -- 172.21.15.98:0/1048588336 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff800c0d0 con 0x7f7ff80051d0 2024-09-19T08:38:53.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.711+0000 7f8010c36640 1 -- 172.21.15.98:0/1048588336 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7ff800c2d0 con 0x7f7ff8003a10 2024-09-19T08:38:53.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.711+0000 7f8011437640 1 --2- 172.21.15.98:0/1048588336 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ff8004810 0x7f7ff8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:53.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.711+0000 7f8011c38640 1 --2- 172.21.15.98:0/1048588336 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ff8003a10 0x7f7ff8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:53.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.711+0000 7f8011437640 1 -- 172.21.15.98:0/1048588336 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ff8003a10 msgr2=0x7f7ff8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:53.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.711+0000 7f8012439640 1 --2- 172.21.15.98:0/1048588336 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ff80051d0 0x7f7ff800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:53.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.711+0000 7f8011437640 1 --2- 172.21.15.98:0/1048588336 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ff8003a10 0x7f7ff8003e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:53.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.711+0000 7f8011437640 1 -- 172.21.15.98:0/1048588336 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ff80051d0 msgr2=0x7f7ff800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:53.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.711+0000 7f8011437640 1 --2- 172.21.15.98:0/1048588336 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ff80051d0 0x7f7ff800b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:53.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.711+0000 7f8012439640 1 -- 172.21.15.98:0/1048588336 reap_dead start 2024-09-19T08:38:53.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.711+0000 7f8011437640 1 -- 172.21.15.98:0/1048588336 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff8003530 con 0x7f7ff8004810 2024-09-19T08:38:53.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.715+0000 7f8011437640 1 --2- 172.21.15.98:0/1048588336 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ff8004810 0x7f7ff8004c90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8000001780 tx=0x7f800001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd1fcfb672897f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:53.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.715+0000 7f8010c36640 1 -- 172.21.15.98:0/1048588336 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80000013a0 con 0x7f7ff8004810 2024-09-19T08:38:53.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.715+0000 7f8010c36640 1 -- 172.21.15.98:0/1048588336 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80000052f0 con 0x7f7ff8004810 2024-09-19T08:38:53.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.715+0000 7f8010c36640 1 -- 172.21.15.98:0/1048588336 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8000003840 con 0x7f7ff8004810 2024-09-19T08:38:53.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.715+0000 7f8013e9c640 1 -- 172.21.15.98:0/1048588336 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ff8004810 msgr2=0x7f7ff8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:53.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.715+0000 7f8013e9c640 1 --2- 172.21.15.98:0/1048588336 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ff8004810 0x7f7ff8004c90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8000001780 tx=0x7f800001c3c0 comp rx=0 tx=0).stop 2024-09-19T08:38:53.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.715+0000 7f8013e9c640 1 -- 172.21.15.98:0/1048588336 shutdown_connections 2024-09-19T08:38:53.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.715+0000 7f8013e9c640 1 --2- 172.21.15.98:0/1048588336 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ff8004810 0x7f7ff8004c90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:53.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.715+0000 7f8013e9c640 1 -- 172.21.15.98:0/1048588336 >> 172.21.15.98:0/1048588336 conn(0x7f800c0b2010 msgr2=0x7f800c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:53.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.715+0000 7f8013e9c640 1 -- 172.21.15.98:0/1048588336 shutdown_connections 2024-09-19T08:38:53.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.715+0000 7f8013e9c640 1 -- 172.21.15.98:0/1048588336 wait complete. 2024-09-19T08:38:53.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.715+0000 7f8013e9c640 1 Processor -- start 2024-09-19T08:38:53.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.715+0000 7f8013e9c640 1 -- start start 2024-09-19T08:38:53.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.715+0000 7f8013e9c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ff8004810 0x7f800c153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:53.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.715+0000 7f8013e9c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f800c156150 0x7f800c1f2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:53.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.715+0000 7f8013e9c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f800c1f2670 0x7f800c1f2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:53.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.715+0000 7f8013e9c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f800c06bc90 con 0x7f800c1f2670 2024-09-19T08:38:53.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.715+0000 7f8013e9c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f800c15cb40 con 0x7f800c156150 2024-09-19T08:38:53.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.715+0000 7f8013e9c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f800c06be10 con 0x7f7ff8004810 2024-09-19T08:38:53.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.715+0000 7f8012439640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f800c1f2670 0x7f800c1f2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:53.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.715+0000 7f8012439640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f800c1f2670 0x7f800c1f2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40832/0 (socket says 172.21.15.98:40832) 2024-09-19T08:38:53.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.715+0000 7f8012439640 1 -- 172.21.15.98:0/3592280293 learned_addr learned my addr 172.21.15.98:0/3592280293 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:53.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.715+0000 7f8011437640 1 --2- 172.21.15.98:0/3592280293 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f800c156150 0x7f800c1f2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:53.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.715+0000 7f8011c38640 1 --2- 172.21.15.98:0/3592280293 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ff8004810 0x7f800c153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:53.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.715+0000 7f8012439640 1 -- 172.21.15.98:0/3592280293 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ff8004810 msgr2=0x7f800c153ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:53.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.715+0000 7f8012439640 1 --2- 172.21.15.98:0/3592280293 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ff8004810 0x7f800c153ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:53.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.715+0000 7f8012439640 1 -- 172.21.15.98:0/3592280293 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f800c156150 msgr2=0x7f800c1f2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:53.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.715+0000 7f8012439640 1 --2- 172.21.15.98:0/3592280293 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f800c156150 0x7f800c1f2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:53.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.715+0000 7f8012439640 1 -- 172.21.15.98:0/3592280293 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f800c156650 con 0x7f800c1f2670 2024-09-19T08:38:53.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.715+0000 7f8011437640 1 --2- 172.21.15.98:0/3592280293 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f800c156150 0x7f800c1f2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:38:53.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.715+0000 7f8012439640 1 --2- 172.21.15.98:0/3592280293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f800c1f2670 0x7f800c1f2b00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f800c073b20 tx=0x7f800400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:53.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.719+0000 7f7ffe7fc640 1 -- 172.21.15.98:0/3592280293 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004016420 con 0x7f800c1f2670 2024-09-19T08:38:53.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.719+0000 7f7ffe7fc640 1 -- 172.21.15.98:0/3592280293 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f800401b070 con 0x7f800c1f2670 2024-09-19T08:38:53.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.719+0000 7f8013e9c640 1 -- 172.21.15.98:0/3592280293 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f800c156930 con 0x7f800c1f2670 2024-09-19T08:38:53.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.719+0000 7f7ffe7fc640 1 -- 172.21.15.98:0/3592280293 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004016820 con 0x7f800c1f2670 2024-09-19T08:38:53.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.719+0000 7f8013e9c640 1 -- 172.21.15.98:0/3592280293 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f800c1501d0 con 0x7f800c1f2670 2024-09-19T08:38:53.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.719+0000 7f8013e9c640 1 -- 172.21.15.98:0/3592280293 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f800c15dd10 con 0x7f800c1f2670 2024-09-19T08:38:53.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.719+0000 7f7ffe7fc640 1 -- 172.21.15.98:0/3592280293 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f80040169c0 con 0x7f800c1f2670 2024-09-19T08:38:53.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.719+0000 7f7ffe7fc640 1 --2- 172.21.15.98:0/3592280293 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7ff00678c0 0x7f7ff0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:53.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.719+0000 7f7ffe7fc640 1 -- 172.21.15.98:0/3592280293 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8004086d20 con 0x7f800c1f2670 2024-09-19T08:38:53.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.719+0000 7f8011c38640 1 --2- 172.21.15.98:0/3592280293 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7ff00678c0 0x7f7ff0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:53.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.719+0000 7f8011c38640 1 --2- 172.21.15.98:0/3592280293 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7ff00678c0 0x7f7ff0069d80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8008000c10 tx=0x7f8008001520 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:53.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.723+0000 7f7ffe7fc640 1 -- 172.21.15.98:0/3592280293 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f8004025020 con 0x7f800c1f2670 2024-09-19T08:38:53.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.871+0000 7f8013e9c640 1 -- 172.21.15.98:0/3592280293 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f800c0cf3c0 con 0x7f800c1f2670 2024-09-19T08:38:53.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.875+0000 7f7ffe7fc640 1 -- 172.21.15.98:0/3592280293 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16665 (secure 0 0 0) 0x7f8004057850 con 0x7f800c1f2670 2024-09-19T08:38:53.879 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:38:53.879 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":21,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:38:20.362527+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:38:16.588688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T08:38:17.700392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:38:53.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.879+0000 7f8013e9c640 1 -- 172.21.15.98:0/3592280293 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7ff00678c0 msgr2=0x7f7ff0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:53.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.879+0000 7f8013e9c640 1 --2- 172.21.15.98:0/3592280293 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7ff00678c0 0x7f7ff0069d80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8008000c10 tx=0x7f8008001520 comp rx=0 tx=0).stop 2024-09-19T08:38:53.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.879+0000 7f8013e9c640 1 -- 172.21.15.98:0/3592280293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f800c1f2670 msgr2=0x7f800c1f2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:53.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.879+0000 7f8013e9c640 1 --2- 172.21.15.98:0/3592280293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f800c1f2670 0x7f800c1f2b00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f800c073b20 tx=0x7f800400f520 comp rx=0 tx=0).stop 2024-09-19T08:38:53.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.879+0000 7f8013e9c640 1 -- 172.21.15.98:0/3592280293 shutdown_connections 2024-09-19T08:38:53.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.879+0000 7f8013e9c640 1 --2- 172.21.15.98:0/3592280293 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7ff00678c0 0x7f7ff0069d80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:53.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.879+0000 7f8013e9c640 1 --2- 172.21.15.98:0/3592280293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f800c1f2670 0x7f800c1f2b00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:53.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.879+0000 7f8013e9c640 1 --2- 172.21.15.98:0/3592280293 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f800c156150 0x7f800c1f2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:53.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.879+0000 7f8013e9c640 1 --2- 172.21.15.98:0/3592280293 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ff8004810 0x7f800c153ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:53.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.879+0000 7f8013e9c640 1 -- 172.21.15.98:0/3592280293 >> 172.21.15.98:0/3592280293 conn(0x7f800c0b2010 msgr2=0x7f800c0b1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:53.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.879+0000 7f8013e9c640 1 -- 172.21.15.98:0/3592280293 shutdown_connections 2024-09-19T08:38:53.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:53.879+0000 7f8013e9c640 1 -- 172.21.15.98:0/3592280293 wait complete. 2024-09-19T08:38:53.897 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T08:38:53.897 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:38:54.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.007+0000 7fccaa4c1640 1 Processor -- start 2024-09-19T08:38:54.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.007+0000 7fccaa4c1640 1 -- start start 2024-09-19T08:38:54.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.007+0000 7fccaa4c1640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcca4140d30 0x7fcca406d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:54.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.007+0000 7fccaa4c1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcca4141110 0x7fcca406d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:54.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.007+0000 7fccaa4c1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcca40b3310 0x7fcca40b3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:54.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.007+0000 7fccaa4c1640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fcca415e7c0 con 0x7fcca4140d30 2024-09-19T08:38:54.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.007+0000 7fccaa4c1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fcca40b3ca0 con 0x7fcca40b3310 2024-09-19T08:38:54.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.007+0000 7fccaa4c1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcca415ea10 con 0x7fcca4141110 2024-09-19T08:38:54.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.007+0000 7fcca3fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcca4140d30 0x7fcca406d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:54.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.007+0000 7fcca3fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcca4140d30 0x7fcca406d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40834/0 (socket says 172.21.15.98:40834) 2024-09-19T08:38:54.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.007+0000 7fcca3fff640 1 -- 172.21.15.98:0/183708578 learned_addr learned my addr 172.21.15.98:0/183708578 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:54.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.007+0000 7fcca8a5e640 1 --2- 172.21.15.98:0/183708578 >> v2:172.21.15.139:3300/0 conn(0x7fcca40b3310 0x7fcca40b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:54.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.007+0000 7fcca37fe640 1 --2- 172.21.15.98:0/183708578 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcca4141110 0x7fcca406d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:54.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.007+0000 7fcca3fff640 1 -- 172.21.15.98:0/183708578 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcca4141110 msgr2=0x7fcca406d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:54.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.007+0000 7fcca3fff640 1 --2- 172.21.15.98:0/183708578 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcca4141110 0x7fcca406d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.007+0000 7fcca3fff640 1 -- 172.21.15.98:0/183708578 >> v2:172.21.15.139:3300/0 conn(0x7fcca40b3310 msgr2=0x7fcca40b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:54.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.007+0000 7fcca3fff640 1 --2- 172.21.15.98:0/183708578 >> v2:172.21.15.139:3300/0 conn(0x7fcca40b3310 0x7fcca40b3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.007+0000 7fcca3fff640 1 -- 172.21.15.98:0/183708578 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca415ecf0 con 0x7fcca4140d30 2024-09-19T08:38:54.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.007+0000 7fcca8a5e640 1 --2- 172.21.15.98:0/183708578 >> v2:172.21.15.139:3300/0 conn(0x7fcca40b3310 0x7fcca40b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:54.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.007+0000 7fcca3fff640 1 --2- 172.21.15.98:0/183708578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcca4140d30 0x7fcca406d170 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcca40b2d90 tx=0x7fcc9401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa5cf589f0c0109 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:54.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.011+0000 7fcca2ffd640 1 -- 172.21.15.98:0/183708578 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc940a6040 con 0x7fcca4140d30 2024-09-19T08:38:54.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.011+0000 7fcca2ffd640 1 -- 172.21.15.98:0/183708578 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc940af070 con 0x7fcca4140d30 2024-09-19T08:38:54.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.011+0000 7fcca2ffd640 1 -- 172.21.15.98:0/183708578 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc940aa860 con 0x7fcca4140d30 2024-09-19T08:38:54.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.011+0000 7fccaa4c1640 1 -- 172.21.15.98:0/183708578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcca4140d30 msgr2=0x7fcca406d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:54.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.011+0000 7fccaa4c1640 1 --2- 172.21.15.98:0/183708578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcca4140d30 0x7fcca406d170 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcca40b2d90 tx=0x7fcc9401c500 comp rx=0 tx=0).stop 2024-09-19T08:38:54.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.011+0000 7fccaa4c1640 1 -- 172.21.15.98:0/183708578 shutdown_connections 2024-09-19T08:38:54.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.011+0000 7fccaa4c1640 1 --2- 172.21.15.98:0/183708578 >> v2:172.21.15.139:3300/0 conn(0x7fcca40b3310 0x7fcca40b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.011+0000 7fccaa4c1640 1 --2- 172.21.15.98:0/183708578 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcca4141110 0x7fcca406d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.011+0000 7fccaa4c1640 1 --2- 172.21.15.98:0/183708578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcca4140d30 0x7fcca406d170 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.011+0000 7fccaa4c1640 1 -- 172.21.15.98:0/183708578 >> 172.21.15.98:0/183708578 conn(0x7fcca40b2010 msgr2=0x7fcca40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:54.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.011+0000 7fccaa4c1640 1 -- 172.21.15.98:0/183708578 shutdown_connections 2024-09-19T08:38:54.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.011+0000 7fccaa4c1640 1 -- 172.21.15.98:0/183708578 wait complete. 2024-09-19T08:38:54.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.011+0000 7fccaa4c1640 1 Processor -- start 2024-09-19T08:38:54.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.011+0000 7fccaa4c1640 1 -- start start 2024-09-19T08:38:54.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.011+0000 7fccaa4c1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcca40b3310 0x7fcca41f7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:54.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.011+0000 7fccaa4c1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcca4140d30 0x7fcca41640f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:54.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.011+0000 7fccaa4c1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcca4141110 0x7fcca4164630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:54.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.011+0000 7fccaa4c1640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcca40cd660 con 0x7fcca4141110 2024-09-19T08:38:54.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.011+0000 7fccaa4c1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcca40cd4e0 con 0x7fcca4140d30 2024-09-19T08:38:54.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.011+0000 7fccaa4c1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcca40cd7e0 con 0x7fcca40b3310 2024-09-19T08:38:54.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.011+0000 7fcca8a5e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcca4141110 0x7fcca4164630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:54.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.011+0000 7fcca8a5e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcca4141110 0x7fcca4164630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40850/0 (socket says 172.21.15.98:40850) 2024-09-19T08:38:54.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.011+0000 7fcca8a5e640 1 -- 172.21.15.98:0/1532504842 learned_addr learned my addr 172.21.15.98:0/1532504842 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:54.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.011+0000 7fcca37fe640 1 --2- 172.21.15.98:0/1532504842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcca4140d30 0x7fcca41640f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:54.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.011+0000 7fcca3fff640 1 --2- 172.21.15.98:0/1532504842 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcca40b3310 0x7fcca41f7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:54.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.011+0000 7fcca8a5e640 1 -- 172.21.15.98:0/1532504842 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcca40b3310 msgr2=0x7fcca41f7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:54.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.011+0000 7fcca8a5e640 1 --2- 172.21.15.98:0/1532504842 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcca40b3310 0x7fcca41f7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.011+0000 7fcca8a5e640 1 -- 172.21.15.98:0/1532504842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcca4140d30 msgr2=0x7fcca41640f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:54.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.011+0000 7fcca8a5e640 1 --2- 172.21.15.98:0/1532504842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcca4140d30 0x7fcca41640f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.011+0000 7fcca8a5e640 1 -- 172.21.15.98:0/1532504842 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca41625f0 con 0x7fcca4141110 2024-09-19T08:38:54.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.011+0000 7fcca8a5e640 1 --2- 172.21.15.98:0/1532504842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcca4141110 0x7fcca4164630 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c00cef0 tx=0x7fcc9c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:54.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.011+0000 7fcca0ff9640 1 -- 172.21.15.98:0/1532504842 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c01a070 con 0x7fcca4141110 2024-09-19T08:38:54.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.011+0000 7fcca0ff9640 1 -- 172.21.15.98:0/1532504842 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc9c00d840 con 0x7fcca4141110 2024-09-19T08:38:54.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.011+0000 7fccaa4c1640 1 -- 172.21.15.98:0/1532504842 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcca41628d0 con 0x7fcca4141110 2024-09-19T08:38:54.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.011+0000 7fcca0ff9640 1 -- 172.21.15.98:0/1532504842 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c00dc30 con 0x7fcca4141110 2024-09-19T08:38:54.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.011+0000 7fccaa4c1640 1 -- 172.21.15.98:0/1532504842 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcca4162e90 con 0x7fcca4141110 2024-09-19T08:38:54.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.015+0000 7fccaa4c1640 1 -- 172.21.15.98:0/1532504842 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcca41631d0 con 0x7fcca4141110 2024-09-19T08:38:54.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.015+0000 7fcca0ff9640 1 -- 172.21.15.98:0/1532504842 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcc9c01d070 con 0x7fcca4141110 2024-09-19T08:38:54.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.015+0000 7fcca0ff9640 1 --2- 172.21.15.98:0/1532504842 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fcc840678c0 0x7fcc84069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:54.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.015+0000 7fcca0ff9640 1 -- 172.21.15.98:0/1532504842 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fcc9c0864c0 con 0x7fcca4141110 2024-09-19T08:38:54.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.015+0000 7fcca3fff640 1 --2- 172.21.15.98:0/1532504842 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fcc840678c0 0x7fcc84069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:54.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.015+0000 7fcca3fff640 1 --2- 172.21.15.98:0/1532504842 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fcc840678c0 0x7fcc84069d80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcc940ae000 tx=0x7fcc940a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:54.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.019+0000 7fcca0ff9640 1 -- 172.21.15.98:0/1532504842 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fcca41631d0 con 0x7fcca4141110 2024-09-19T08:38:54.216 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.211+0000 7fccaa4c1640 1 -- 172.21.15.98:0/1532504842 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcca40cf9d0 con 0x7fcca4141110 2024-09-19T08:38:54.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.215+0000 7fcca0ff9640 1 -- 172.21.15.98:0/1532504842 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5220 (secure 0 0 0) 0x7fcc9c057000 con 0x7fcca4141110 2024-09-19T08:38:54.217 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:38:54.217 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":7,"btime":"2024-09-19T08:38:22:407519+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/2038744461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":2038744461},{"type":"v1","addr":"172.21.15.139:6833","nonce":2038744461}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/765134249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":765134249},{"type":"v1","addr":"172.21.15.98:6835","nonce":765134249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2832328076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2832328076},{"type":"v1","addr":"172.21.15.98:6837","nonce":2832328076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:38:18.717221+0000","modified":"2024-09-19T08:38:21.769948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6835/1879089763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1879089763},{"type":"v1","addr":"172.21.15.139:6835","nonce":1879089763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4261,"qdb_cluster":[4261]},"id":1}]} 2024-09-19T08:38:54.218 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 7 2024-09-19T08:38:54.221 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.219+0000 7fccaa4c1640 1 -- 172.21.15.98:0/1532504842 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fcc840678c0 msgr2=0x7fcc84069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:54.221 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.219+0000 7fccaa4c1640 1 --2- 172.21.15.98:0/1532504842 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fcc840678c0 0x7fcc84069d80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcc940ae000 tx=0x7fcc940a7000 comp rx=0 tx=0).stop 2024-09-19T08:38:54.221 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.219+0000 7fccaa4c1640 1 -- 172.21.15.98:0/1532504842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcca4141110 msgr2=0x7fcca4164630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:54.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.219+0000 7fccaa4c1640 1 --2- 172.21.15.98:0/1532504842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcca4141110 0x7fcca4164630 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c00cef0 tx=0x7fcc9c00a710 comp rx=0 tx=0).stop 2024-09-19T08:38:54.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.219+0000 7fccaa4c1640 1 -- 172.21.15.98:0/1532504842 shutdown_connections 2024-09-19T08:38:54.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.219+0000 7fccaa4c1640 1 --2- 172.21.15.98:0/1532504842 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fcc840678c0 0x7fcc84069d80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.219+0000 7fccaa4c1640 1 --2- 172.21.15.98:0/1532504842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcca4141110 0x7fcca4164630 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.219+0000 7fccaa4c1640 1 --2- 172.21.15.98:0/1532504842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcca4140d30 0x7fcca41640f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.219+0000 7fccaa4c1640 1 --2- 172.21.15.98:0/1532504842 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcca40b3310 0x7fcca41f7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.219+0000 7fccaa4c1640 1 -- 172.21.15.98:0/1532504842 >> 172.21.15.98:0/1532504842 conn(0x7fcca40b2010 msgr2=0x7fcca40730e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:54.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.219+0000 7fccaa4c1640 1 -- 172.21.15.98:0/1532504842 shutdown_connections 2024-09-19T08:38:54.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.219+0000 7fccaa4c1640 1 -- 172.21.15.98:0/1532504842 wait complete. 2024-09-19T08:38:54.237 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:38:54.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.339+0000 7feff2a97640 1 Processor -- start 2024-09-19T08:38:54.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.339+0000 7feff2a97640 1 -- start start 2024-09-19T08:38:54.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.339+0000 7feff2a97640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fefec06c3a0 0x7fefec0cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:54.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.339+0000 7feff2a97640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fefec0cf900 0x7fefec0723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:54.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.339+0000 7feff2a97640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefec06d310 con 0x7fefec15c550 2024-09-19T08:38:54.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.339+0000 7feff2a97640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fefec06d5f0 con 0x7fefec06c3a0 2024-09-19T08:38:54.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.339+0000 7feff2a97640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fefec06d8d0 con 0x7fefec0cf900 2024-09-19T08:38:54.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.339+0000 7fefebfff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fefec06c3a0 0x7fefec0cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:54.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.339+0000 7fefebfff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fefec06c3a0 0x7fefec0cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40852/0 (socket says 172.21.15.98:40852) 2024-09-19T08:38:54.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.339+0000 7fefebfff640 1 -- 172.21.15.98:0/559373345 learned_addr learned my addr 172.21.15.98:0/559373345 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:54.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.339+0000 7feff1034640 1 --2- 172.21.15.98:0/559373345 >> v2:172.21.15.139:3300/0 conn(0x7fefec0cf900 0x7fefec0723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:54.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.339+0000 7fefebfff640 1 -- 172.21.15.98:0/559373345 >> v2:172.21.15.139:3300/0 conn(0x7fefec0cf900 msgr2=0x7fefec0723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:54.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.339+0000 7fefebfff640 1 --2- 172.21.15.98:0/559373345 >> v2:172.21.15.139:3300/0 conn(0x7fefec0cf900 0x7fefec0723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.339+0000 7fefebfff640 1 -- 172.21.15.98:0/559373345 >> v1:172.21.15.139:6789/0 conn(0x7fefec15c550 legacy=0x7fefec06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:54.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.339+0000 7fefebfff640 1 -- 172.21.15.98:0/559373345 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefec06dbb0 con 0x7fefec06c3a0 2024-09-19T08:38:54.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.339+0000 7feff1034640 1 --2- 172.21.15.98:0/559373345 >> v2:172.21.15.139:3300/0 conn(0x7fefec0cf900 0x7fefec0723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:54.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.343+0000 7fefebfff640 1 --2- 172.21.15.98:0/559373345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefec06c3a0 0x7fefec0cf3c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fefec06e270 tx=0x7fefd801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b0f7e86803a2642 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:54.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.343+0000 7fefeb7fe640 1 -- 172.21.15.98:0/559373345 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd80ab5f0 con 0x7fefec06c3a0 2024-09-19T08:38:54.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.343+0000 7fefeb7fe640 1 -- 172.21.15.98:0/559373345 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefd80b0070 con 0x7fefec06c3a0 2024-09-19T08:38:54.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.343+0000 7fefeb7fe640 1 -- 172.21.15.98:0/559373345 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd80ab9b0 con 0x7fefec06c3a0 2024-09-19T08:38:54.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.343+0000 7feff2a97640 1 -- 172.21.15.98:0/559373345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefec06c3a0 msgr2=0x7fefec0cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:54.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.343+0000 7feff2a97640 1 --2- 172.21.15.98:0/559373345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefec06c3a0 0x7fefec0cf3c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fefec06e270 tx=0x7fefd801c4e0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.343+0000 7feff2a97640 1 -- 172.21.15.98:0/559373345 shutdown_connections 2024-09-19T08:38:54.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.343+0000 7feff2a97640 1 --2- 172.21.15.98:0/559373345 >> v2:172.21.15.139:3300/0 conn(0x7fefec0cf900 0x7fefec0723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.343+0000 7feff2a97640 1 --2- 172.21.15.98:0/559373345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefec06c3a0 0x7fefec0cf3c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.343+0000 7feff2a97640 1 -- 172.21.15.98:0/559373345 >> 172.21.15.98:0/559373345 conn(0x7fefec0b2010 msgr2=0x7fefec0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:54.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.343+0000 7feff2a97640 1 -- 172.21.15.98:0/559373345 shutdown_connections 2024-09-19T08:38:54.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.343+0000 7feff2a97640 1 -- 172.21.15.98:0/559373345 wait complete. 2024-09-19T08:38:54.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.343+0000 7feff2a97640 1 Processor -- start 2024-09-19T08:38:54.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.343+0000 7feff2a97640 1 -- start start 2024-09-19T08:38:54.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.343+0000 7feff2a97640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefec06c3a0 0x7fefec067a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:54.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.343+0000 7feff2a97640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fefec0cf900 0x7fefec155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:54.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.343+0000 7feff2a97640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fefec15c550 0x7fefec1f88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:54.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.343+0000 7feff2a97640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fefec073500 con 0x7fefec06c3a0 2024-09-19T08:38:54.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.343+0000 7feff2a97640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fefec073380 con 0x7fefec15c550 2024-09-19T08:38:54.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.343+0000 7feff2a97640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fefec073680 con 0x7fefec0cf900 2024-09-19T08:38:54.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.343+0000 7feff0833640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefec06c3a0 0x7fefec067a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:54.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.343+0000 7feff0833640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefec06c3a0 0x7fefec067a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40866/0 (socket says 172.21.15.98:40866) 2024-09-19T08:38:54.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.343+0000 7feff0833640 1 -- 172.21.15.98:0/2190401674 learned_addr learned my addr 172.21.15.98:0/2190401674 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:54.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.343+0000 7fefebfff640 1 --2- 172.21.15.98:0/2190401674 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fefec0cf900 0x7fefec155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:54.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.343+0000 7feff1034640 1 --2- 172.21.15.98:0/2190401674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fefec15c550 0x7fefec1f88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:54.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.343+0000 7feff0833640 1 -- 172.21.15.98:0/2190401674 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fefec0cf900 msgr2=0x7fefec155920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:54.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.343+0000 7feff0833640 1 --2- 172.21.15.98:0/2190401674 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fefec0cf900 0x7fefec155920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.343+0000 7feff0833640 1 -- 172.21.15.98:0/2190401674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fefec15c550 msgr2=0x7fefec1f88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:54.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.343+0000 7feff0833640 1 --2- 172.21.15.98:0/2190401674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fefec15c550 0x7fefec1f88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.343+0000 7feff0833640 1 -- 172.21.15.98:0/2190401674 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefec152b50 con 0x7fefec06c3a0 2024-09-19T08:38:54.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.343+0000 7feff0833640 1 --2- 172.21.15.98:0/2190401674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefec06c3a0 0x7fefec067a00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fefe000b410 tx=0x7fefe000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:54.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.343+0000 7fefe97fa640 1 -- 172.21.15.98:0/2190401674 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe0015030 con 0x7fefec06c3a0 2024-09-19T08:38:54.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.343+0000 7fefe97fa640 1 -- 172.21.15.98:0/2190401674 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefe0004020 con 0x7fefec06c3a0 2024-09-19T08:38:54.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.343+0000 7fefe97fa640 1 -- 172.21.15.98:0/2190401674 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe0004450 con 0x7fefec06c3a0 2024-09-19T08:38:54.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.343+0000 7feff2a97640 1 -- 172.21.15.98:0/2190401674 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefec152df0 con 0x7fefec06c3a0 2024-09-19T08:38:54.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.343+0000 7feff2a97640 1 -- 172.21.15.98:0/2190401674 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefec1533b0 con 0x7fefec06c3a0 2024-09-19T08:38:54.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.347+0000 7feff2a97640 1 -- 172.21.15.98:0/2190401674 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefec0cf3c0 con 0x7fefec06c3a0 2024-09-19T08:38:54.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.347+0000 7fefe97fa640 1 -- 172.21.15.98:0/2190401674 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fefe0018070 con 0x7fefec06c3a0 2024-09-19T08:38:54.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.347+0000 7fefe97fa640 1 --2- 172.21.15.98:0/2190401674 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fefd00678f0 0x7fefd0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:54.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.347+0000 7fefe97fa640 1 -- 172.21.15.98:0/2190401674 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fefe00857e0 con 0x7fefec06c3a0 2024-09-19T08:38:54.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.347+0000 7fefebfff640 1 --2- 172.21.15.98:0/2190401674 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fefd00678f0 0x7fefd0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:54.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.347+0000 7fefebfff640 1 --2- 172.21.15.98:0/2190401674 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fefd00678f0 0x7fefd0069db0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fefd80af000 tx=0x7fefd80a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:54.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.351+0000 7fefe97fa640 1 -- 172.21.15.98:0/2190401674 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fefe0056320 con 0x7fefec06c3a0 2024-09-19T08:38:54.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.531+0000 7feff2a97640 1 -- 172.21.15.98:0/2190401674 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fefec154da0 con 0x7fefec06c3a0 2024-09-19T08:38:54.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.535+0000 7fefe97fa640 1 -- 172.21.15.98:0/2190401674 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5220 (secure 0 0 0) 0x7fefe0055c20 con 0x7fefec06c3a0 2024-09-19T08:38:54.538 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:38:54.538 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":7,"btime":"2024-09-19T08:38:22:407519+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/2038744461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":2038744461},{"type":"v1","addr":"172.21.15.139:6833","nonce":2038744461}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/765134249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":765134249},{"type":"v1","addr":"172.21.15.98:6835","nonce":765134249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/2832328076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":2832328076},{"type":"v1","addr":"172.21.15.98:6837","nonce":2832328076}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:38:18.717221+0000","modified":"2024-09-19T08:38:21.769948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4261},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4261":{"gid":4261,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6835/1879089763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1879089763},{"type":"v1","addr":"172.21.15.139:6835","nonce":1879089763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4261,"qdb_cluster":[4261]},"id":1}]} 2024-09-19T08:38:54.538 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 7 2024-09-19T08:38:54.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.535+0000 7feff2a97640 1 -- 172.21.15.98:0/2190401674 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fefd00678f0 msgr2=0x7fefd0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:54.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.535+0000 7feff2a97640 1 --2- 172.21.15.98:0/2190401674 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fefd00678f0 0x7fefd0069db0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fefd80af000 tx=0x7fefd80a7000 comp rx=0 tx=0).stop 2024-09-19T08:38:54.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.535+0000 7feff2a97640 1 -- 172.21.15.98:0/2190401674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefec06c3a0 msgr2=0x7fefec067a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:54.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.535+0000 7feff2a97640 1 --2- 172.21.15.98:0/2190401674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefec06c3a0 0x7fefec067a00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fefe000b410 tx=0x7fefe000b8d0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.535+0000 7feff2a97640 1 -- 172.21.15.98:0/2190401674 shutdown_connections 2024-09-19T08:38:54.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.535+0000 7feff2a97640 1 --2- 172.21.15.98:0/2190401674 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fefd00678f0 0x7fefd0069db0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.535+0000 7feff2a97640 1 --2- 172.21.15.98:0/2190401674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fefec15c550 0x7fefec1f88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.535+0000 7feff2a97640 1 --2- 172.21.15.98:0/2190401674 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fefec0cf900 0x7fefec155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.535+0000 7feff2a97640 1 --2- 172.21.15.98:0/2190401674 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fefec06c3a0 0x7fefec067a00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.535+0000 7feff2a97640 1 -- 172.21.15.98:0/2190401674 >> 172.21.15.98:0/2190401674 conn(0x7fefec0b2010 msgr2=0x7fefec159720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:54.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.535+0000 7feff2a97640 1 -- 172.21.15.98:0/2190401674 shutdown_connections 2024-09-19T08:38:54.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.535+0000 7feff2a97640 1 -- 172.21.15.98:0/2190401674 wait complete. 2024-09-19T08:38:54.555 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:38:54.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.667+0000 7fe5772c6640 1 Processor -- start 2024-09-19T08:38:54.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.671+0000 7fe5772c6640 1 -- start start 2024-09-19T08:38:54.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.671+0000 7fe5772c6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe57015c220 0x7fe5700733f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:54.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.671+0000 7fe5772c6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe570073930 0x7fe5700d0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:54.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.671+0000 7fe5772c6640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5700740b0 con 0x7fe5700d1530 2024-09-19T08:38:54.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.671+0000 7fe5772c6640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe570073dd0 con 0x7fe57015c220 2024-09-19T08:38:54.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.671+0000 7fe5772c6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe5700b6460 con 0x7fe570073930 2024-09-19T08:38:54.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.671+0000 7fe575062640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe57015c220 0x7fe5700733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:54.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.671+0000 7fe574861640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe570073930 0x7fe5700d0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:54.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.671+0000 7fe575062640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe57015c220 0x7fe5700733f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:46958/0 (socket says 172.21.15.98:46958) 2024-09-19T08:38:54.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.671+0000 7fe575062640 1 -- 172.21.15.98:0/1172690089 learned_addr learned my addr 172.21.15.98:0/1172690089 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:54.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.671+0000 7fe575863640 1 --1- 172.21.15.98:0/1172690089 >> v1:172.21.15.139:6789/0 conn(0x7fe5700d1530 0x7fe5700b6f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:35516/0 (socket says 172.21.15.98:35516) 2024-09-19T08:38:54.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.671+0000 7fe575062640 1 -- 172.21.15.98:0/1172690089 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe570073930 msgr2=0x7fe5700d0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:54.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.671+0000 7fe575062640 1 --2- 172.21.15.98:0/1172690089 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe570073930 0x7fe5700d0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.671+0000 7fe575062640 1 -- 172.21.15.98:0/1172690089 >> v1:172.21.15.139:6789/0 conn(0x7fe5700d1530 legacy=0x7fe5700b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:54.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.671+0000 7fe575062640 1 -- 172.21.15.98:0/1172690089 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5700b6740 con 0x7fe57015c220 2024-09-19T08:38:54.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.671+0000 7fe574861640 1 --2- 172.21.15.98:0/1172690089 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe570073930 0x7fe5700d0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:38:54.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.675+0000 7fe575062640 1 --2- 172.21.15.98:0/1172690089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe57015c220 0x7fe5700733f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe568009870 tx=0x7fe56801c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcb66dd2e6f186c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:54.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.675+0000 7fe56ffff640 1 -- 172.21.15.98:0/1172690089 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5680a7070 con 0x7fe57015c220 2024-09-19T08:38:54.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.675+0000 7fe56ffff640 1 -- 172.21.15.98:0/1172690089 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5680b0070 con 0x7fe57015c220 2024-09-19T08:38:54.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.675+0000 7fe56ffff640 1 -- 172.21.15.98:0/1172690089 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5680ab890 con 0x7fe57015c220 2024-09-19T08:38:54.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.675+0000 7fe5772c6640 1 -- 172.21.15.98:0/1172690089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe57015c220 msgr2=0x7fe5700733f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:54.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.675+0000 7fe5772c6640 1 --2- 172.21.15.98:0/1172690089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe57015c220 0x7fe5700733f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe568009870 tx=0x7fe56801c560 comp rx=0 tx=0).stop 2024-09-19T08:38:54.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.675+0000 7fe5772c6640 1 -- 172.21.15.98:0/1172690089 shutdown_connections 2024-09-19T08:38:54.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.675+0000 7fe5772c6640 1 --2- 172.21.15.98:0/1172690089 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe570073930 0x7fe5700d0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.675+0000 7fe5772c6640 1 --2- 172.21.15.98:0/1172690089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe57015c220 0x7fe5700733f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.675+0000 7fe5772c6640 1 -- 172.21.15.98:0/1172690089 >> 172.21.15.98:0/1172690089 conn(0x7fe5700b40a0 msgr2=0x7fe5700b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:54.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.675+0000 7fe5772c6640 1 -- 172.21.15.98:0/1172690089 shutdown_connections 2024-09-19T08:38:54.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.675+0000 7fe5772c6640 1 -- 172.21.15.98:0/1172690089 wait complete. 2024-09-19T08:38:54.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.675+0000 7fe5772c6640 1 Processor -- start 2024-09-19T08:38:54.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.675+0000 7fe5772c6640 1 -- start start 2024-09-19T08:38:54.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.675+0000 7fe5772c6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe570073930 0x7fe570153120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:54.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.675+0000 7fe5772c6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5700d1530 0x7fe570150e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:54.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.675+0000 7fe5772c6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe57015c220 0x7fe570148f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:54.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.675+0000 7fe5772c6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe570168ea0 con 0x7fe570073930 2024-09-19T08:38:54.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.675+0000 7fe5772c6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe57015d0a0 con 0x7fe5700d1530 2024-09-19T08:38:54.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.675+0000 7fe5772c6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe570169020 con 0x7fe57015c220 2024-09-19T08:38:54.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.675+0000 7fe575062640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe570073930 0x7fe570153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:54.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.675+0000 7fe575062640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe570073930 0x7fe570153120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40874/0 (socket says 172.21.15.98:40874) 2024-09-19T08:38:54.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.675+0000 7fe575062640 1 -- 172.21.15.98:0/290468 learned_addr learned my addr 172.21.15.98:0/290468 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:54.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.675+0000 7fe574861640 1 --2- 172.21.15.98:0/290468 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5700d1530 0x7fe570150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:54.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.675+0000 7fe575863640 1 --2- 172.21.15.98:0/290468 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe57015c220 0x7fe570148f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:54.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.675+0000 7fe575062640 1 -- 172.21.15.98:0/290468 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe57015c220 msgr2=0x7fe570148f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:54.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.675+0000 7fe575062640 1 --2- 172.21.15.98:0/290468 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe57015c220 0x7fe570148f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.675+0000 7fe575062640 1 -- 172.21.15.98:0/290468 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5700d1530 msgr2=0x7fe570150e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:54.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.675+0000 7fe575062640 1 --2- 172.21.15.98:0/290468 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5700d1530 0x7fe570150e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.675+0000 7fe575062640 1 -- 172.21.15.98:0/290468 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5700b23b0 con 0x7fe570073930 2024-09-19T08:38:54.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.675+0000 7fe574861640 1 --2- 172.21.15.98:0/290468 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5700d1530 0x7fe570150e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:54.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.675+0000 7fe575062640 1 --2- 172.21.15.98:0/290468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe570073930 0x7fe570153120 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe5680ad040 tx=0x7fe56801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:54.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.679+0000 7fe56dffb640 1 -- 172.21.15.98:0/290468 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5680be5f0 con 0x7fe570073930 2024-09-19T08:38:54.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.679+0000 7fe56dffb640 1 -- 172.21.15.98:0/290468 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5680be790 con 0x7fe570073930 2024-09-19T08:38:54.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.679+0000 7fe5772c6640 1 -- 172.21.15.98:0/290468 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5700b26c0 con 0x7fe570073930 2024-09-19T08:38:54.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.679+0000 7fe56dffb640 1 -- 172.21.15.98:0/290468 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5680beb60 con 0x7fe570073930 2024-09-19T08:38:54.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.679+0000 7fe5772c6640 1 -- 172.21.15.98:0/290468 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5700b2f00 con 0x7fe570073930 2024-09-19T08:38:54.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.679+0000 7fe5772c6640 1 -- 172.21.15.98:0/290468 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5700d0ff0 con 0x7fe570073930 2024-09-19T08:38:54.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.679+0000 7fe56dffb640 1 -- 172.21.15.98:0/290468 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe5680b43f0 con 0x7fe570073930 2024-09-19T08:38:54.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.679+0000 7fe56dffb640 1 --2- 172.21.15.98:0/290468 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe55c0678c0 0x7fe55c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:54.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.679+0000 7fe56dffb640 1 -- 172.21.15.98:0/290468 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe568120750 con 0x7fe570073930 2024-09-19T08:38:54.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.679+0000 7fe574861640 1 --2- 172.21.15.98:0/290468 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe55c0678c0 0x7fe55c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:54.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.679+0000 7fe574861640 1 --2- 172.21.15.98:0/290468 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe55c0678c0 0x7fe55c069d80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe570150bf0 tx=0x7fe564008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:54.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.683+0000 7fe56dffb640 1 -- 172.21.15.98:0/290468 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe5700b26c0 con 0x7fe570073930 2024-09-19T08:38:54.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.839+0000 7fe5772c6640 1 -- 172.21.15.98:0/290468 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe5700b8160 con 0x7fe570073930 2024-09-19T08:38:54.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.843+0000 7fe56dffb640 1 -- 172.21.15.98:0/290468 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16665 (secure 0 0 0) 0x7fe5680c4060 con 0x7fe570073930 2024-09-19T08:38:54.848 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:38:54.848 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":21,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:38:20.362527+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:38:16.588688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T08:38:17.700392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:38:54.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.847+0000 7fe5772c6640 1 -- 172.21.15.98:0/290468 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe55c0678c0 msgr2=0x7fe55c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:54.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.847+0000 7fe5772c6640 1 --2- 172.21.15.98:0/290468 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe55c0678c0 0x7fe55c069d80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe570150bf0 tx=0x7fe564008040 comp rx=0 tx=0).stop 2024-09-19T08:38:54.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.847+0000 7fe5772c6640 1 -- 172.21.15.98:0/290468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe570073930 msgr2=0x7fe570153120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:54.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.847+0000 7fe5772c6640 1 --2- 172.21.15.98:0/290468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe570073930 0x7fe570153120 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe5680ad040 tx=0x7fe56801ada0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.847+0000 7fe5772c6640 1 -- 172.21.15.98:0/290468 shutdown_connections 2024-09-19T08:38:54.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.847+0000 7fe5772c6640 1 --2- 172.21.15.98:0/290468 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe55c0678c0 0x7fe55c069d80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.847+0000 7fe5772c6640 1 --2- 172.21.15.98:0/290468 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe57015c220 0x7fe570148f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.847+0000 7fe5772c6640 1 --2- 172.21.15.98:0/290468 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5700d1530 0x7fe570150e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.847+0000 7fe5772c6640 1 --2- 172.21.15.98:0/290468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe570073930 0x7fe570153120 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.847+0000 7fe5772c6640 1 -- 172.21.15.98:0/290468 >> 172.21.15.98:0/290468 conn(0x7fe5700b40a0 msgr2=0x7fe570070350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:54.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.847+0000 7fe5772c6640 1 -- 172.21.15.98:0/290468 shutdown_connections 2024-09-19T08:38:54.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.847+0000 7fe5772c6640 1 -- 172.21.15.98:0/290468 wait complete. 2024-09-19T08:38:54.865 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-19T08:38:54.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.971+0000 7f6610a94640 1 Processor -- start 2024-09-19T08:38:54.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.971+0000 7f6610a94640 1 -- start start 2024-09-19T08:38:54.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.971+0000 7f6610a94640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f660c0b28e0 0x7f660c073c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:54.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.971+0000 7f6610a94640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f660c075380 0x7f660c0cf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:54.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.971+0000 7f6610a94640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f660c075b80 con 0x7f660c15e750 2024-09-19T08:38:54.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.971+0000 7f6610a94640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f660c0d1bd0 con 0x7f660c0b28e0 2024-09-19T08:38:54.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.971+0000 7f6610a94640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f660c0d1eb0 con 0x7f660c075380 2024-09-19T08:38:54.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.971+0000 7f6609d9b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f660c0b28e0 0x7f660c073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:54.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.971+0000 7f660a59c640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f660c15e750 0x7f660c074c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36000/0 (socket says 172.21.15.98:36000) 2024-09-19T08:38:54.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.971+0000 7f660a59c640 1 -- 172.21.15.98:0/2577413846 learned_addr learned my addr 172.21.15.98:0/2577413846 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:54.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.971+0000 7f660ad9d640 1 --2- 172.21.15.98:0/2577413846 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f660c075380 0x7f660c0cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:54.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.971+0000 7f6609d9b640 1 -- 172.21.15.98:0/2577413846 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f660c075380 msgr2=0x7f660c0cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:54.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.971+0000 7f6609d9b640 1 --2- 172.21.15.98:0/2577413846 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f660c075380 0x7f660c0cf520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.971+0000 7f6609d9b640 1 -- 172.21.15.98:0/2577413846 >> v1:172.21.15.98:6789/0 conn(0x7f660c15e750 legacy=0x7f660c074c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:54.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.971+0000 7f6609d9b640 1 -- 172.21.15.98:0/2577413846 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f660c0cf000 con 0x7f660c0b28e0 2024-09-19T08:38:54.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.971+0000 7f660959a640 1 -- 172.21.15.98:0/2577413846 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 726605551 0 0) 0x7f660c075b80 con 0x7f660c15e750 2024-09-19T08:38:54.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.975+0000 7f6609d9b640 1 --2- 172.21.15.98:0/2577413846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f660c0b28e0 0x7f660c073c90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f66000097c0 tx=0x7f660001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de5dec725728a6e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:54.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.975+0000 7f660959a640 1 -- 172.21.15.98:0/2577413846 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66000ab430 con 0x7f660c0b28e0 2024-09-19T08:38:54.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.975+0000 7f660959a640 1 -- 172.21.15.98:0/2577413846 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66000b0070 con 0x7f660c0b28e0 2024-09-19T08:38:54.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.975+0000 7f660959a640 1 -- 172.21.15.98:0/2577413846 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66000ab7f0 con 0x7f660c0b28e0 2024-09-19T08:38:54.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.975+0000 7f6610a94640 1 -- 172.21.15.98:0/2577413846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f660c0b28e0 msgr2=0x7f660c073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:54.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.975+0000 7f6610a94640 1 --2- 172.21.15.98:0/2577413846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f660c0b28e0 0x7f660c073c90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f66000097c0 tx=0x7f660001c410 comp rx=0 tx=0).stop 2024-09-19T08:38:54.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.975+0000 7f6610a94640 1 -- 172.21.15.98:0/2577413846 shutdown_connections 2024-09-19T08:38:54.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.975+0000 7f6610a94640 1 --2- 172.21.15.98:0/2577413846 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f660c075380 0x7f660c0cf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.975+0000 7f6610a94640 1 --2- 172.21.15.98:0/2577413846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f660c0b28e0 0x7f660c073c90 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.975+0000 7f6610a94640 1 -- 172.21.15.98:0/2577413846 >> 172.21.15.98:0/2577413846 conn(0x7f660c073170 msgr2=0x7f660c073580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:54.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.975+0000 7f6610a94640 1 -- 172.21.15.98:0/2577413846 shutdown_connections 2024-09-19T08:38:54.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.975+0000 7f6610a94640 1 -- 172.21.15.98:0/2577413846 wait complete. 2024-09-19T08:38:54.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.975+0000 7f6610a94640 1 Processor -- start 2024-09-19T08:38:54.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.975+0000 7f6610a94640 1 -- start start 2024-09-19T08:38:54.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.975+0000 7f6610a94640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f660c075380 0x7f660c153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:54.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.975+0000 7f6610a94640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f660c0b28e0 0x7f660c099e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:54.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.975+0000 7f6610a94640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f660c15e750 0x7f660c0c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:54.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.975+0000 7f6610a94640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f660c0d0600 con 0x7f660c15e750 2024-09-19T08:38:54.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.975+0000 7f6610a94640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f660c0d0480 con 0x7f660c0b28e0 2024-09-19T08:38:54.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.975+0000 7f6610a94640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f660c0d0780 con 0x7f660c075380 2024-09-19T08:38:54.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.975+0000 7f660ad9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f660c15e750 0x7f660c0c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:54.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.975+0000 7f660ad9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f660c15e750 0x7f660c0c5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40886/0 (socket says 172.21.15.98:40886) 2024-09-19T08:38:54.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.975+0000 7f660ad9d640 1 -- 172.21.15.98:0/2630906287 learned_addr learned my addr 172.21.15.98:0/2630906287 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:54.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.975+0000 7f6609d9b640 1 --2- 172.21.15.98:0/2630906287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f660c0b28e0 0x7f660c099e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:54.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.975+0000 7f660ad9d640 1 -- 172.21.15.98:0/2630906287 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f660c075380 msgr2=0x7f660c153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:54.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.975+0000 7f660ad9d640 1 --2- 172.21.15.98:0/2630906287 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f660c075380 0x7f660c153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.975+0000 7f660ad9d640 1 -- 172.21.15.98:0/2630906287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f660c0b28e0 msgr2=0x7f660c099e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:54.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.975+0000 7f660ad9d640 1 --2- 172.21.15.98:0/2630906287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f660c0b28e0 0x7f660c099e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:54.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.975+0000 7f660ad9d640 1 -- 172.21.15.98:0/2630906287 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f660c160410 con 0x7f660c15e750 2024-09-19T08:38:54.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.975+0000 7f660ad9d640 1 --2- 172.21.15.98:0/2630906287 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f660c15e750 0x7f660c0c5ef0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f660400cc30 tx=0x7f660400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:54.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.975+0000 7f65faffd640 1 -- 172.21.15.98:0/2630906287 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660400ce40 con 0x7f660c15e750 2024-09-19T08:38:54.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.979+0000 7f65faffd640 1 -- 172.21.15.98:0/2630906287 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66040039a0 con 0x7f660c15e750 2024-09-19T08:38:54.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.979+0000 7f65faffd640 1 -- 172.21.15.98:0/2630906287 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6604003d50 con 0x7f660c15e750 2024-09-19T08:38:54.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.979+0000 7f6610a94640 1 -- 172.21.15.98:0/2630906287 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f660c15a100 con 0x7f660c15e750 2024-09-19T08:38:54.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.979+0000 7f6610a94640 1 -- 172.21.15.98:0/2630906287 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f660c160990 con 0x7f660c15e750 2024-09-19T08:38:54.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.979+0000 7f6610a94640 1 -- 172.21.15.98:0/2630906287 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f660c0cf520 con 0x7f660c15e750 2024-09-19T08:38:54.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.979+0000 7f65faffd640 1 -- 172.21.15.98:0/2630906287 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6604003f30 con 0x7f660c15e750 2024-09-19T08:38:54.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.979+0000 7f65faffd640 1 --2- 172.21.15.98:0/2630906287 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f65ec0678f0 0x7f65ec069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:54.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.979+0000 7f65faffd640 1 -- 172.21.15.98:0/2630906287 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f66040860d0 con 0x7f660c15e750 2024-09-19T08:38:54.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.979+0000 7f660a59c640 1 --2- 172.21.15.98:0/2630906287 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f65ec0678f0 0x7f65ec069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:54.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.979+0000 7f660a59c640 1 --2- 172.21.15.98:0/2630906287 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f65ec0678f0 0x7f65ec069db0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f65fc0011f0 tx=0x7f65fc007930 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:54.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:54.983+0000 7f65faffd640 1 -- 172.21.15.98:0/2630906287 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6604014030 con 0x7f660c15e750 2024-09-19T08:38:55.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.163+0000 7f6610a94640 1 -- 172.21.15.98:0/2630906287 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f660c073dd0 con 0x7f660c15e750 2024-09-19T08:38:55.169 INFO:tasks.ceph.mon.a.smithi098.stderr:2024-09-19T08:38:55.167+0000 7f5e49017640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T08:38:55.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.167+0000 7f65faffd640 1 -- 172.21.15.98:0/2630906287 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) ==== 161+0+0 (secure 0 0 0) 0x7f6604056ca0 con 0x7f660c15e750 2024-09-19T08:38:55.172 INFO:teuthology.orchestra.run.smithi098.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T08:38:55.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.171+0000 7f6610a94640 1 -- 172.21.15.98:0/2630906287 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f65ec0678f0 msgr2=0x7f65ec069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:55.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.171+0000 7f6610a94640 1 --2- 172.21.15.98:0/2630906287 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f65ec0678f0 0x7f65ec069db0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f65fc0011f0 tx=0x7f65fc007930 comp rx=0 tx=0).stop 2024-09-19T08:38:55.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.171+0000 7f6610a94640 1 -- 172.21.15.98:0/2630906287 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f660c15e750 msgr2=0x7f660c0c5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:55.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.171+0000 7f6610a94640 1 --2- 172.21.15.98:0/2630906287 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f660c15e750 0x7f660c0c5ef0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f660400cc30 tx=0x7f660400a5c0 comp rx=0 tx=0).stop 2024-09-19T08:38:55.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.171+0000 7f6610a94640 1 -- 172.21.15.98:0/2630906287 shutdown_connections 2024-09-19T08:38:55.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.171+0000 7f6610a94640 1 --2- 172.21.15.98:0/2630906287 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f65ec0678f0 0x7f65ec069db0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:55.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.171+0000 7f6610a94640 1 --2- 172.21.15.98:0/2630906287 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f660c15e750 0x7f660c0c5ef0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:55.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.171+0000 7f6610a94640 1 --2- 172.21.15.98:0/2630906287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f660c0b28e0 0x7f660c099e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:55.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.171+0000 7f6610a94640 1 --2- 172.21.15.98:0/2630906287 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f660c075380 0x7f660c153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:55.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.171+0000 7f6610a94640 1 -- 172.21.15.98:0/2630906287 >> 172.21.15.98:0/2630906287 conn(0x7f660c073170 msgr2=0x7f660c0b2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:55.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.171+0000 7f6610a94640 1 -- 172.21.15.98:0/2630906287 shutdown_connections 2024-09-19T08:38:55.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.171+0000 7f6610a94640 1 -- 172.21.15.98:0/2630906287 wait complete. 2024-09-19T08:38:55.185 INFO:tasks.ceph.mds.d.smithi139.stderr: -2292> 2024-09-19T08:38:24.330+0000 7fe477378640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T08:38:55.191 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T08:38:55.229 INFO:tasks.ceph.mds.d.smithi139.stdout:starting mds.d at 2024-09-19T08:38:55.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.307+0000 7f8504464640 1 Processor -- start 2024-09-19T08:38:55.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.307+0000 7f8504464640 1 -- start start 2024-09-19T08:38:55.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.307+0000 7f8504464640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f84fc145b70 0x7f84fc06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:55.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.307+0000 7f8504464640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f84fc06c1d0 0x7f84fc0713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:55.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.307+0000 7f8504464640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84fc073530 con 0x7f84fc0718f0 2024-09-19T08:38:55.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.307+0000 7f8504464640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f84fc06c620 con 0x7f84fc06c1d0 2024-09-19T08:38:55.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.307+0000 7f8504464640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f84fc073810 con 0x7f84fc145b70 2024-09-19T08:38:55.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.307+0000 7f85019ff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f84fc06c1d0 0x7f84fc0713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:55.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.307+0000 7f85019ff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f84fc06c1d0 0x7f84fc0713b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40898/0 (socket says 172.21.15.98:40898) 2024-09-19T08:38:55.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.307+0000 7f85019ff640 1 -- 172.21.15.98:0/853608137 learned_addr learned my addr 172.21.15.98:0/853608137 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:55.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.307+0000 7f8502200640 1 --2- 172.21.15.98:0/853608137 >> v2:172.21.15.139:3300/0 conn(0x7f84fc145b70 0x7f84fc06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:55.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.307+0000 7f85019ff640 1 -- 172.21.15.98:0/853608137 >> v2:172.21.15.139:3300/0 conn(0x7f84fc145b70 msgr2=0x7f84fc06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:55.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.307+0000 7f85019ff640 1 --2- 172.21.15.98:0/853608137 >> v2:172.21.15.139:3300/0 conn(0x7f84fc145b70 0x7f84fc06bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:55.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.307+0000 7f85019ff640 1 -- 172.21.15.98:0/853608137 >> v1:172.21.15.139:6789/0 conn(0x7f84fc0718f0 legacy=0x7f84fc0cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:55.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.307+0000 7f85019ff640 1 -- 172.21.15.98:0/853608137 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84fc073af0 con 0x7f84fc06c1d0 2024-09-19T08:38:55.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.311+0000 7f85019ff640 1 --2- 172.21.15.98:0/853608137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84fc06c1d0 0x7f84fc0713b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f84fc0b2d30 tx=0x7f84ec01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51fd480c4f48955 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:55.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.311+0000 7f85011fe640 1 -- 172.21.15.98:0/853608137 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ec0b0070 con 0x7f84fc06c1d0 2024-09-19T08:38:55.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.311+0000 7f85011fe640 1 -- 172.21.15.98:0/853608137 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84ec0ab600 con 0x7f84fc06c1d0 2024-09-19T08:38:55.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.311+0000 7f85011fe640 1 -- 172.21.15.98:0/853608137 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ec0ab9d0 con 0x7f84fc06c1d0 2024-09-19T08:38:55.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.311+0000 7f8504464640 1 -- 172.21.15.98:0/853608137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84fc06c1d0 msgr2=0x7f84fc0713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:55.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.311+0000 7f8504464640 1 --2- 172.21.15.98:0/853608137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84fc06c1d0 0x7f84fc0713b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f84fc0b2d30 tx=0x7f84ec01c450 comp rx=0 tx=0).stop 2024-09-19T08:38:55.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.311+0000 7f8504464640 1 -- 172.21.15.98:0/853608137 shutdown_connections 2024-09-19T08:38:55.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.311+0000 7f8504464640 1 --2- 172.21.15.98:0/853608137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84fc06c1d0 0x7f84fc0713b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:55.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.311+0000 7f8504464640 1 --2- 172.21.15.98:0/853608137 >> v2:172.21.15.139:3300/0 conn(0x7f84fc145b70 0x7f84fc06bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:55.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.311+0000 7f8504464640 1 -- 172.21.15.98:0/853608137 >> 172.21.15.98:0/853608137 conn(0x7f84fc0b2010 msgr2=0x7f84fc0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:55.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.311+0000 7f8504464640 1 -- 172.21.15.98:0/853608137 shutdown_connections 2024-09-19T08:38:55.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.311+0000 7f8504464640 1 -- 172.21.15.98:0/853608137 wait complete. 2024-09-19T08:38:55.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.311+0000 7f8504464640 1 Processor -- start 2024-09-19T08:38:55.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.311+0000 7f8504464640 1 -- start start 2024-09-19T08:38:55.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.315+0000 7f8504464640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84fc06c1d0 0x7f84fc142040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:55.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.315+0000 7f8504464640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84fc0718f0 0x7f84fc1280e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:55.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.315+0000 7f8504464640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f84fc145b70 0x7f84fc128620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:55.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.315+0000 7f8504464640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f84fc1691f0 con 0x7f84fc06c1d0 2024-09-19T08:38:55.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.315+0000 7f8504464640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f84fc169070 con 0x7f84fc0718f0 2024-09-19T08:38:55.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.315+0000 7f8504464640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f84fc169370 con 0x7f84fc145b70 2024-09-19T08:38:55.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.315+0000 7f8502200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84fc06c1d0 0x7f84fc142040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:55.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.315+0000 7f8502200640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84fc06c1d0 0x7f84fc142040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40914/0 (socket says 172.21.15.98:40914) 2024-09-19T08:38:55.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.315+0000 7f8502200640 1 -- 172.21.15.98:0/1572562200 learned_addr learned my addr 172.21.15.98:0/1572562200 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:55.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.315+0000 7f8502200640 1 -- 172.21.15.98:0/1572562200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f84fc145b70 msgr2=0x7f84fc128620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:55.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.315+0000 7f8502200640 1 --2- 172.21.15.98:0/1572562200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f84fc145b70 0x7f84fc128620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:55.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.315+0000 7f8502200640 1 -- 172.21.15.98:0/1572562200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84fc0718f0 msgr2=0x7f84fc1280e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:55.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.315+0000 7f8502200640 1 --2- 172.21.15.98:0/1572562200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84fc0718f0 0x7f84fc1280e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:55.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.315+0000 7f8502200640 1 -- 172.21.15.98:0/1572562200 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84fc140fe0 con 0x7f84fc06c1d0 2024-09-19T08:38:55.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.315+0000 7f8502200640 1 --2- 172.21.15.98:0/1572562200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84fc06c1d0 0x7f84fc142040 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f84f800d5f0 tx=0x7f84f800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:55.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.315+0000 7f84f2ffd640 1 -- 172.21.15.98:0/1572562200 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f80125f0 con 0x7f84fc06c1d0 2024-09-19T08:38:55.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.315+0000 7f84f2ffd640 1 -- 172.21.15.98:0/1572562200 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84f8019070 con 0x7f84fc06c1d0 2024-09-19T08:38:55.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.315+0000 7f8504464640 1 -- 172.21.15.98:0/1572562200 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84fc141280 con 0x7f84fc06c1d0 2024-09-19T08:38:55.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.315+0000 7f84f2ffd640 1 -- 172.21.15.98:0/1572562200 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f80129b0 con 0x7f84fc06c1d0 2024-09-19T08:38:55.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.315+0000 7f8504464640 1 -- 172.21.15.98:0/1572562200 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84fc141840 con 0x7f84fc06c1d0 2024-09-19T08:38:55.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.315+0000 7f8504464640 1 -- 172.21.15.98:0/1572562200 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84fc0713b0 con 0x7f84fc06c1d0 2024-09-19T08:38:55.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.315+0000 7f84f2ffd640 1 -- 172.21.15.98:0/1572562200 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f84f8012b50 con 0x7f84fc06c1d0 2024-09-19T08:38:55.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.315+0000 7f84f2ffd640 1 --2- 172.21.15.98:0/1572562200 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f84e00678c0 0x7f84e0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:55.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.315+0000 7f84f2ffd640 1 -- 172.21.15.98:0/1572562200 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5795+0+0 (secure 0 0 0) 0x7f84f80860a0 con 0x7f84fc06c1d0 2024-09-19T08:38:55.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.315+0000 7f85019ff640 1 --2- 172.21.15.98:0/1572562200 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f84e00678c0 0x7f84e0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:55.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.315+0000 7f85019ff640 1 --2- 172.21.15.98:0/1572562200 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f84e00678c0 0x7f84e0069d80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f84ec0ad000 tx=0x7f84ec004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:55.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.319+0000 7f84f2ffd640 1 -- 172.21.15.98:0/1572562200 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f84f8056c10 con 0x7f84fc06c1d0 2024-09-19T08:38:55.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:55.507+0000 7f8504464640 1 -- 172.21.15.98:0/1572562200 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f84fc0cc9c0 con 0x7f84fc06c1d0 2024-09-19T08:38:56.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.175+0000 7f84f2ffd640 1 -- 172.21.15.98:0/1572562200 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) ==== 104+0+0 (secure 0 0 0) 0x7f84f8056510 con 0x7f84fc06c1d0 2024-09-19T08:38:56.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.179+0000 7f8504464640 1 -- 172.21.15.98:0/1572562200 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f84e00678c0 msgr2=0x7f84e0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:56.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.179+0000 7f8504464640 1 --2- 172.21.15.98:0/1572562200 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f84e00678c0 0x7f84e0069d80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f84ec0ad000 tx=0x7f84ec004740 comp rx=0 tx=0).stop 2024-09-19T08:38:56.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.179+0000 7f8504464640 1 -- 172.21.15.98:0/1572562200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84fc06c1d0 msgr2=0x7f84fc142040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:56.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.179+0000 7f8504464640 1 --2- 172.21.15.98:0/1572562200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84fc06c1d0 0x7f84fc142040 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f84f800d5f0 tx=0x7f84f800dab0 comp rx=0 tx=0).stop 2024-09-19T08:38:56.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.179+0000 7f8504464640 1 -- 172.21.15.98:0/1572562200 shutdown_connections 2024-09-19T08:38:56.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.179+0000 7f8504464640 1 --2- 172.21.15.98:0/1572562200 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f84e00678c0 0x7f84e0069d80 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:56.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.179+0000 7f8504464640 1 --2- 172.21.15.98:0/1572562200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f84fc145b70 0x7f84fc128620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:56.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.179+0000 7f8504464640 1 --2- 172.21.15.98:0/1572562200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f84fc0718f0 0x7f84fc1280e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:56.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.179+0000 7f8504464640 1 --2- 172.21.15.98:0/1572562200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84fc06c1d0 0x7f84fc142040 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:56.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.179+0000 7f8504464640 1 -- 172.21.15.98:0/1572562200 >> 172.21.15.98:0/1572562200 conn(0x7f84fc0b2010 msgr2=0x7f84fc0b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:56.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.179+0000 7f8504464640 1 -- 172.21.15.98:0/1572562200 shutdown_connections 2024-09-19T08:38:56.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.179+0000 7f8504464640 1 -- 172.21.15.98:0/1572562200 wait complete. 2024-09-19T08:38:56.202 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T08:38:56.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.323+0000 7fa453e3c640 1 Processor -- start 2024-09-19T08:38:56.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.323+0000 7fa453e3c640 1 -- start start 2024-09-19T08:38:56.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.323+0000 7fa453e3c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa44c15b220 0x7fa44c06e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:56.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.323+0000 7fa453e3c640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa44c0cbe90 con 0x7fa44c06d600 2024-09-19T08:38:56.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.323+0000 7fa453e3c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa44c15b600 con 0x7fa44c0cb730 2024-09-19T08:38:56.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.323+0000 7fa453e3c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fa44c06efe0 con 0x7fa44c15b220 2024-09-19T08:38:56.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.323+0000 7fa4513d7640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fa44c06d600 0x7fa44c0cb020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36008/0 (socket says 172.21.15.98:36008) 2024-09-19T08:38:56.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.323+0000 7fa4513d7640 1 -- 172.21.15.98:0/1416647744 learned_addr learned my addr 172.21.15.98:0/1416647744 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:56.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.323+0000 7fa450bd6640 1 -- 172.21.15.98:0/1416647744 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2801710725 0 0) 0x7fa44c0cbe90 con 0x7fa44c06d600 2024-09-19T08:38:56.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.323+0000 7fa450bd6640 1 -- 172.21.15.98:0/1416647744 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa438005740 con 0x7fa44c06d600 2024-09-19T08:38:56.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.323+0000 7fa451bd8640 1 --2- 172.21.15.98:0/1416647744 >> v2:172.21.15.139:3300/0 conn(0x7fa44c15b220 0x7fa44c06e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:56.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.323+0000 7fa450bd6640 1 -- 172.21.15.98:0/1416647744 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3459427446 0 0) 0x7fa438005740 con 0x7fa44c06d600 2024-09-19T08:38:56.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.323+0000 7fa450bd6640 1 -- 172.21.15.98:0/1416647744 >> v2:172.21.15.139:3300/0 conn(0x7fa44c15b220 msgr2=0x7fa44c06e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:56.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.323+0000 7fa450bd6640 1 --2- 172.21.15.98:0/1416647744 >> v2:172.21.15.139:3300/0 conn(0x7fa44c15b220 0x7fa44c06e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:56.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.323+0000 7fa450bd6640 1 -- 172.21.15.98:0/1416647744 >> v1:172.21.15.139:6789/0 conn(0x7fa44c0cb730 legacy=0x7fa44c0cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:56.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.323+0000 7fa450bd6640 1 -- 172.21.15.98:0/1416647744 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa44c06ea60 con 0x7fa44c06d600 2024-09-19T08:38:56.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.323+0000 7fa450bd6640 1 -- 172.21.15.98:0/1416647744 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fa43c001680 con 0x7fa44c06d600 2024-09-19T08:38:56.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.323+0000 7fa450bd6640 1 -- 172.21.15.98:0/1416647744 >> v1:172.21.15.98:6789/0 conn(0x7fa44c06d600 legacy=0x7fa44c0cb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:56.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.323+0000 7fa450bd6640 1 --2- 172.21.15.98:0/1416647744 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa438005bd0 0x7fa438006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:56.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.323+0000 7fa450bd6640 1 --2- 172.21.15.98:0/1416647744 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa438006a20 0x7fa438006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:56.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.323+0000 7fa450bd6640 1 --2- 172.21.15.98:0/1416647744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa438007500 0x7fa43800dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:56.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.323+0000 7fa450bd6640 1 -- 172.21.15.98:0/1416647744 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa44c06efe0 con 0x7fa438007500 2024-09-19T08:38:56.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.323+0000 7fa450bd6640 1 -- 172.21.15.98:0/1416647744 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa43800e400 con 0x7fa438005bd0 2024-09-19T08:38:56.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.323+0000 7fa450bd6640 1 -- 172.21.15.98:0/1416647744 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa43800e600 con 0x7fa438006a20 2024-09-19T08:38:56.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.323+0000 7fa4523d9640 1 --2- 172.21.15.98:0/1416647744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa438007500 0x7fa43800dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:56.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.323+0000 7fa451bd8640 1 --2- 172.21.15.98:0/1416647744 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa438005bd0 0x7fa438006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:56.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.323+0000 7fa4513d7640 1 --2- 172.21.15.98:0/1416647744 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa438006a20 0x7fa438006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:56.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.323+0000 7fa4523d9640 1 -- 172.21.15.98:0/1416647744 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa438006a20 msgr2=0x7fa438006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:56.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.323+0000 7fa4523d9640 1 --2- 172.21.15.98:0/1416647744 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa438006a20 0x7fa438006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:56.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.323+0000 7fa4523d9640 1 -- 172.21.15.98:0/1416647744 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa438005bd0 msgr2=0x7fa438006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:56.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.323+0000 7fa4523d9640 1 --2- 172.21.15.98:0/1416647744 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa438005bd0 0x7fa438006010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:56.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.323+0000 7fa4523d9640 1 -- 172.21.15.98:0/1416647744 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa438005740 con 0x7fa438007500 2024-09-19T08:38:56.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.323+0000 7fa451bd8640 1 --2- 172.21.15.98:0/1416647744 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa438005bd0 0x7fa438006010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:38:56.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.327+0000 7fa4523d9640 1 -- 172.21.15.98:0/1416647744 reap_dead start 2024-09-19T08:38:56.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.327+0000 7fa4523d9640 1 --2- 172.21.15.98:0/1416647744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa438007500 0x7fa43800dbe0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa44c0b0500 tx=0x7fa44401c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47569a3eeff822fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:56.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.327+0000 7fa450bd6640 1 -- 172.21.15.98:0/1416647744 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa444001a90 con 0x7fa438007500 2024-09-19T08:38:56.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.327+0000 7fa450bd6640 1 -- 172.21.15.98:0/1416647744 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa444003b80 con 0x7fa438007500 2024-09-19T08:38:56.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.327+0000 7fa450bd6640 1 -- 172.21.15.98:0/1416647744 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4440050d0 con 0x7fa438007500 2024-09-19T08:38:56.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.327+0000 7fa453e3c640 1 -- 172.21.15.98:0/1416647744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa438007500 msgr2=0x7fa43800dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:56.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.327+0000 7fa453e3c640 1 --2- 172.21.15.98:0/1416647744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa438007500 0x7fa43800dbe0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa44c0b0500 tx=0x7fa44401c720 comp rx=0 tx=0).stop 2024-09-19T08:38:56.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.327+0000 7fa453e3c640 1 -- 172.21.15.98:0/1416647744 shutdown_connections 2024-09-19T08:38:56.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.327+0000 7fa453e3c640 1 --2- 172.21.15.98:0/1416647744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa438007500 0x7fa43800dbe0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:56.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.327+0000 7fa453e3c640 1 -- 172.21.15.98:0/1416647744 >> 172.21.15.98:0/1416647744 conn(0x7fa44c0b25c0 msgr2=0x7fa44c0b29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:56.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.327+0000 7fa453e3c640 1 -- 172.21.15.98:0/1416647744 shutdown_connections 2024-09-19T08:38:56.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.327+0000 7fa453e3c640 1 -- 172.21.15.98:0/1416647744 wait complete. 2024-09-19T08:38:56.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.331+0000 7fa453e3c640 1 Processor -- start 2024-09-19T08:38:56.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.331+0000 7fa453e3c640 1 -- start start 2024-09-19T08:38:56.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.331+0000 7fa453e3c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa438007500 0x7fa44c1589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:56.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.331+0000 7fa453e3c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa44c0c6f40 0x7fa44c0c4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:56.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.331+0000 7fa453e3c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa44c0c73c0 0x7fa44c1f3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:56.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.331+0000 7fa453e3c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa44c0cec90 con 0x7fa44c0c73c0 2024-09-19T08:38:56.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.331+0000 7fa453e3c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa44c0ce9b0 con 0x7fa44c0c6f40 2024-09-19T08:38:56.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.331+0000 7fa453e3c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa44c06ec30 con 0x7fa438007500 2024-09-19T08:38:56.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.331+0000 7fa4523d9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa44c0c73c0 0x7fa44c1f3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:56.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.331+0000 7fa4523d9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa44c0c73c0 0x7fa44c1f3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40946/0 (socket says 172.21.15.98:40946) 2024-09-19T08:38:56.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.331+0000 7fa4523d9640 1 -- 172.21.15.98:0/3287317093 learned_addr learned my addr 172.21.15.98:0/3287317093 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:56.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.331+0000 7fa451bd8640 1 --2- 172.21.15.98:0/3287317093 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa438007500 0x7fa44c1589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:56.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.331+0000 7fa4513d7640 1 --2- 172.21.15.98:0/3287317093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa44c0c6f40 0x7fa44c0c4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:56.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.331+0000 7fa4523d9640 1 -- 172.21.15.98:0/3287317093 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa438007500 msgr2=0x7fa44c1589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:56.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.331+0000 7fa4523d9640 1 --2- 172.21.15.98:0/3287317093 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa438007500 0x7fa44c1589d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:56.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.331+0000 7fa4523d9640 1 -- 172.21.15.98:0/3287317093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa44c0c6f40 msgr2=0x7fa44c0c4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:56.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.331+0000 7fa4523d9640 1 --2- 172.21.15.98:0/3287317093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa44c0c6f40 0x7fa44c0c4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:56.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.331+0000 7fa4523d9640 1 -- 172.21.15.98:0/3287317093 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa44c0c5970 con 0x7fa44c0c73c0 2024-09-19T08:38:56.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.331+0000 7fa451bd8640 1 --2- 172.21.15.98:0/3287317093 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa438007500 0x7fa44c1589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:38:56.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.331+0000 7fa4523d9640 1 --2- 172.21.15.98:0/3287317093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa44c0c73c0 0x7fa44c1f3970 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa444098020 tx=0x7fa438007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:56.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.331+0000 7fa4427fc640 1 -- 172.21.15.98:0/3287317093 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa444007dc0 con 0x7fa44c0c73c0 2024-09-19T08:38:56.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.331+0000 7fa4427fc640 1 -- 172.21.15.98:0/3287317093 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa444001260 con 0x7fa44c0c73c0 2024-09-19T08:38:56.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.331+0000 7fa453e3c640 1 -- 172.21.15.98:0/3287317093 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa44c0c6180 con 0x7fa44c0c73c0 2024-09-19T08:38:56.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.331+0000 7fa4427fc640 1 -- 172.21.15.98:0/3287317093 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4440b4640 con 0x7fa44c0c73c0 2024-09-19T08:38:56.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.331+0000 7fa453e3c640 1 -- 172.21.15.98:0/3287317093 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa44c0c6740 con 0x7fa44c0c73c0 2024-09-19T08:38:56.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.331+0000 7fa453e3c640 1 -- 172.21.15.98:0/3287317093 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa44c0cafe0 con 0x7fa44c0c73c0 2024-09-19T08:38:56.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.331+0000 7fa4427fc640 1 -- 172.21.15.98:0/3287317093 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa4440a9410 con 0x7fa44c0c73c0 2024-09-19T08:38:56.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.335+0000 7fa4427fc640 1 --2- 172.21.15.98:0/3287317093 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa4300678c0 0x7fa430069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:56.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.335+0000 7fa4427fc640 1 -- 172.21.15.98:0/3287317093 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5795+0+0 (secure 0 0 0) 0x7fa44411f340 con 0x7fa44c0c73c0 2024-09-19T08:38:56.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.335+0000 7fa451bd8640 1 --2- 172.21.15.98:0/3287317093 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa4300678c0 0x7fa430069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:56.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.335+0000 7fa451bd8640 1 --2- 172.21.15.98:0/3287317093 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa4300678c0 0x7fa430069d80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa448003870 tx=0x7fa448008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:56.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.339+0000 7fa4427fc640 1 -- 172.21.15.98:0/3287317093 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa4440ba060 con 0x7fa44c0c73c0 2024-09-19T08:38:56.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:56.491+0000 7fa453e3c640 1 -- 172.21.15.98:0/3287317093 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa44c0b0750 con 0x7fa44c0c73c0 2024-09-19T08:38:57.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.179+0000 7fa4427fc640 1 -- 172.21.15.98:0/3287317093 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v23) ==== 181+0+0 (secure 0 0 0) 0x7fa444016430 con 0x7fa44c0c73c0 2024-09-19T08:38:57.184 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_metadata' removed 2024-09-19T08:38:57.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.183+0000 7fa453e3c640 1 -- 172.21.15.98:0/3287317093 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa4300678c0 msgr2=0x7fa430069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:57.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.183+0000 7fa453e3c640 1 --2- 172.21.15.98:0/3287317093 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa4300678c0 0x7fa430069d80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa448003870 tx=0x7fa448008040 comp rx=0 tx=0).stop 2024-09-19T08:38:57.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.183+0000 7fa453e3c640 1 -- 172.21.15.98:0/3287317093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa44c0c73c0 msgr2=0x7fa44c1f3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:57.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.183+0000 7fa453e3c640 1 --2- 172.21.15.98:0/3287317093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa44c0c73c0 0x7fa44c1f3970 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa444098020 tx=0x7fa438007a50 comp rx=0 tx=0).stop 2024-09-19T08:38:57.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.183+0000 7fa453e3c640 1 -- 172.21.15.98:0/3287317093 shutdown_connections 2024-09-19T08:38:57.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.183+0000 7fa453e3c640 1 --2- 172.21.15.98:0/3287317093 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa4300678c0 0x7fa430069d80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:57.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.183+0000 7fa453e3c640 1 --2- 172.21.15.98:0/3287317093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa44c0c73c0 0x7fa44c1f3970 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:57.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.183+0000 7fa453e3c640 1 --2- 172.21.15.98:0/3287317093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa44c0c6f40 0x7fa44c0c4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:57.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.183+0000 7fa453e3c640 1 --2- 172.21.15.98:0/3287317093 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa438007500 0x7fa44c1589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:57.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.183+0000 7fa453e3c640 1 -- 172.21.15.98:0/3287317093 >> 172.21.15.98:0/3287317093 conn(0x7fa44c0b25c0 msgr2=0x7fa44c06dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:57.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.183+0000 7fa453e3c640 1 -- 172.21.15.98:0/3287317093 shutdown_connections 2024-09-19T08:38:57.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.183+0000 7fa453e3c640 1 -- 172.21.15.98:0/3287317093 wait complete. 2024-09-19T08:38:57.209 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T08:38:57.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.331+0000 7f4e6be72640 1 Processor -- start 2024-09-19T08:38:57.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.331+0000 7f4e6be72640 1 -- start start 2024-09-19T08:38:57.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.331+0000 7f4e6be72640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4e640cf700 0x7f4e6415ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:57.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.331+0000 7f4e6be72640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e641608c0 con 0x7f4e6406d4c0 2024-09-19T08:38:57.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.331+0000 7f4e6be72640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e64160ba0 con 0x7f4e6415c290 2024-09-19T08:38:57.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.331+0000 7f4e6be72640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f4e64160e80 con 0x7f4e640cf700 2024-09-19T08:38:57.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.331+0000 7f4e6a40f640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f4e6406d4c0 0x7f4e6406d8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36020/0 (socket says 172.21.15.98:36020) 2024-09-19T08:38:57.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.331+0000 7f4e6a40f640 1 -- 172.21.15.98:0/1566114817 learned_addr learned my addr 172.21.15.98:0/1566114817 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:57.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.331+0000 7f4e68c0c640 1 -- 172.21.15.98:0/1566114817 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3765704223 0 0) 0x7f4e641608c0 con 0x7f4e6406d4c0 2024-09-19T08:38:57.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.331+0000 7f4e68c0c640 1 -- 172.21.15.98:0/1566114817 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e50003530 con 0x7f4e6406d4c0 2024-09-19T08:38:57.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.331+0000 7f4e6940d640 1 --2- 172.21.15.98:0/1566114817 >> v2:172.21.15.139:3300/0 conn(0x7f4e640cf700 0x7f4e6415ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:57.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.335+0000 7f4e68c0c640 1 -- 172.21.15.98:0/1566114817 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 442954697 0 0) 0x7f4e50003530 con 0x7f4e6406d4c0 2024-09-19T08:38:57.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.335+0000 7f4e68c0c640 1 -- 172.21.15.98:0/1566114817 >> v2:172.21.15.139:3300/0 conn(0x7f4e640cf700 msgr2=0x7f4e6415ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:57.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.335+0000 7f4e68c0c640 1 --2- 172.21.15.98:0/1566114817 >> v2:172.21.15.139:3300/0 conn(0x7f4e640cf700 0x7f4e6415ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:57.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.335+0000 7f4e68c0c640 1 -- 172.21.15.98:0/1566114817 >> v1:172.21.15.139:6789/0 conn(0x7f4e6415c290 legacy=0x7f4e640cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:57.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.335+0000 7f4e68c0c640 1 -- 172.21.15.98:0/1566114817 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e640cd1a0 con 0x7f4e6406d4c0 2024-09-19T08:38:57.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.335+0000 7f4e68c0c640 1 -- 172.21.15.98:0/1566114817 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f4e5c001680 con 0x7f4e6406d4c0 2024-09-19T08:38:57.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.335+0000 7f4e68c0c640 1 -- 172.21.15.98:0/1566114817 >> v1:172.21.15.98:6789/0 conn(0x7f4e6406d4c0 legacy=0x7f4e6406d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:57.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.335+0000 7f4e68c0c640 1 --2- 172.21.15.98:0/1566114817 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e500039c0 0x7f4e50003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:57.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.335+0000 7f4e68c0c640 1 --2- 172.21.15.98:0/1566114817 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e50004810 0x7f4e50004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:57.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.335+0000 7f4e68c0c640 1 --2- 172.21.15.98:0/1566114817 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e500052f0 0x7f4e5000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:57.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.335+0000 7f4e68c0c640 1 -- 172.21.15.98:0/1566114817 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5000c190 con 0x7f4e500052f0 2024-09-19T08:38:57.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.335+0000 7f4e68c0c640 1 -- 172.21.15.98:0/1566114817 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5000c340 con 0x7f4e50004810 2024-09-19T08:38:57.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.335+0000 7f4e68c0c640 1 -- 172.21.15.98:0/1566114817 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4e5000c540 con 0x7f4e500039c0 2024-09-19T08:38:57.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.335+0000 7f4e6a40f640 1 --2- 172.21.15.98:0/1566114817 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e500052f0 0x7f4e5000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:57.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.335+0000 7f4e6940d640 1 --2- 172.21.15.98:0/1566114817 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e50004810 0x7f4e50004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:57.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.335+0000 7f4e69c0e640 1 --2- 172.21.15.98:0/1566114817 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e500039c0 0x7f4e50003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:57.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.335+0000 7f4e6a40f640 1 -- 172.21.15.98:0/1566114817 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e500039c0 msgr2=0x7f4e50003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:57.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.335+0000 7f4e6a40f640 1 --2- 172.21.15.98:0/1566114817 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e500039c0 0x7f4e50003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:57.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.335+0000 7f4e6a40f640 1 -- 172.21.15.98:0/1566114817 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e50004810 msgr2=0x7f4e50004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:57.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.335+0000 7f4e6a40f640 1 --2- 172.21.15.98:0/1566114817 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e50004810 0x7f4e50004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:57.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.335+0000 7f4e6a40f640 1 -- 172.21.15.98:0/1566114817 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e50003530 con 0x7f4e500052f0 2024-09-19T08:38:57.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.335+0000 7f4e6a40f640 1 -- 172.21.15.98:0/1566114817 reap_dead start 2024-09-19T08:38:57.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.335+0000 7f4e6a40f640 1 --2- 172.21.15.98:0/1566114817 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e500052f0 0x7f4e5000b9d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00c190 tx=0x7f4e5c01f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b3337705236d51 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:57.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.335+0000 7f4e68c0c640 1 -- 172.21.15.98:0/1566114817 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c007d00 con 0x7f4e500052f0 2024-09-19T08:38:57.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.335+0000 7f4e68c0c640 1 -- 172.21.15.98:0/1566114817 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e5c005b00 con 0x7f4e500052f0 2024-09-19T08:38:57.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.335+0000 7f4e68c0c640 1 -- 172.21.15.98:0/1566114817 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c007070 con 0x7f4e500052f0 2024-09-19T08:38:57.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.335+0000 7f4e6be72640 1 -- 172.21.15.98:0/1566114817 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e500052f0 msgr2=0x7f4e5000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:57.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.335+0000 7f4e6be72640 1 --2- 172.21.15.98:0/1566114817 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e500052f0 0x7f4e5000b9d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00c190 tx=0x7f4e5c01f390 comp rx=0 tx=0).stop 2024-09-19T08:38:57.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.335+0000 7f4e6be72640 1 -- 172.21.15.98:0/1566114817 shutdown_connections 2024-09-19T08:38:57.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.335+0000 7f4e6be72640 1 --2- 172.21.15.98:0/1566114817 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e500052f0 0x7f4e5000b9d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:57.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.335+0000 7f4e6be72640 1 -- 172.21.15.98:0/1566114817 >> 172.21.15.98:0/1566114817 conn(0x7f4e6406c090 msgr2=0x7f4e6406c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:57.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.335+0000 7f4e6be72640 1 -- 172.21.15.98:0/1566114817 shutdown_connections 2024-09-19T08:38:57.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.335+0000 7f4e6be72640 1 -- 172.21.15.98:0/1566114817 wait complete. 2024-09-19T08:38:57.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.339+0000 7f4e6be72640 1 Processor -- start 2024-09-19T08:38:57.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.339+0000 7f4e6be72640 1 -- start start 2024-09-19T08:38:57.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.339+0000 7f4e6be72640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e500052f0 0x7f4e64072b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:57.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.339+0000 7f4e6be72640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e641f51f0 0x7f4e6415ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:57.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.339+0000 7f4e6be72640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e640ae8b0 0x7f4e640aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:57.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.339+0000 7f4e6be72640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6406d8a0 con 0x7f4e500052f0 2024-09-19T08:38:57.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.339+0000 7f4e6be72640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4e640cd510 con 0x7f4e641f51f0 2024-09-19T08:38:57.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.339+0000 7f4e6be72640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4e6406da20 con 0x7f4e640ae8b0 2024-09-19T08:38:57.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.339+0000 7f4e69c0e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e500052f0 0x7f4e64072b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:57.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.339+0000 7f4e69c0e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e500052f0 0x7f4e64072b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40968/0 (socket says 172.21.15.98:40968) 2024-09-19T08:38:57.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.339+0000 7f4e69c0e640 1 -- 172.21.15.98:0/2824426314 learned_addr learned my addr 172.21.15.98:0/2824426314 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:57.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.339+0000 7f4e6a40f640 1 --2- 172.21.15.98:0/2824426314 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e640ae8b0 0x7f4e640aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:57.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.339+0000 7f4e6940d640 1 --2- 172.21.15.98:0/2824426314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e641f51f0 0x7f4e6415ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:57.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.339+0000 7f4e69c0e640 1 -- 172.21.15.98:0/2824426314 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e640ae8b0 msgr2=0x7f4e640aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:57.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.339+0000 7f4e69c0e640 1 --2- 172.21.15.98:0/2824426314 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e640ae8b0 0x7f4e640aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:57.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.339+0000 7f4e69c0e640 1 -- 172.21.15.98:0/2824426314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e641f51f0 msgr2=0x7f4e6415ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:57.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.339+0000 7f4e69c0e640 1 --2- 172.21.15.98:0/2824426314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e641f51f0 0x7f4e6415ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:57.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.339+0000 7f4e69c0e640 1 -- 172.21.15.98:0/2824426314 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e641f4c90 con 0x7f4e500052f0 2024-09-19T08:38:57.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.339+0000 7f4e69c0e640 1 --2- 172.21.15.98:0/2824426314 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e500052f0 0x7f4e64072b60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4e5400d570 tx=0x7f4e5400da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:57.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.339+0000 7f4e5a7fc640 1 -- 172.21.15.98:0/2824426314 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e54005480 con 0x7f4e500052f0 2024-09-19T08:38:57.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.339+0000 7f4e5a7fc640 1 -- 172.21.15.98:0/2824426314 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e54017070 con 0x7f4e500052f0 2024-09-19T08:38:57.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.339+0000 7f4e6be72640 1 -- 172.21.15.98:0/2824426314 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e641f49b0 con 0x7f4e500052f0 2024-09-19T08:38:57.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.339+0000 7f4e5a7fc640 1 -- 172.21.15.98:0/2824426314 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e54003820 con 0x7f4e500052f0 2024-09-19T08:38:57.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.339+0000 7f4e6be72640 1 -- 172.21.15.98:0/2824426314 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e640c59a0 con 0x7f4e500052f0 2024-09-19T08:38:57.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.339+0000 7f4e6be72640 1 -- 172.21.15.98:0/2824426314 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e640cd9a0 con 0x7f4e500052f0 2024-09-19T08:38:57.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.343+0000 7f4e5a7fc640 1 -- 172.21.15.98:0/2824426314 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4e540039c0 con 0x7f4e500052f0 2024-09-19T08:38:57.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.343+0000 7f4e5a7fc640 1 --2- 172.21.15.98:0/2824426314 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4e480678f0 0x7f4e48069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:57.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.343+0000 7f4e6940d640 1 --2- 172.21.15.98:0/2824426314 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4e480678f0 0x7f4e48069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:57.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.343+0000 7f4e5a7fc640 1 -- 172.21.15.98:0/2824426314 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5357+0+0 (secure 0 0 0) 0x7f4e54085790 con 0x7f4e500052f0 2024-09-19T08:38:57.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.343+0000 7f4e6940d640 1 --2- 172.21.15.98:0/2824426314 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4e480678f0 0x7f4e48069db0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4e64072b10 tx=0x7f4e6000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:57.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.347+0000 7f4e5a7fc640 1 -- 172.21.15.98:0/2824426314 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f4e5400f2f0 con 0x7f4e500052f0 2024-09-19T08:38:57.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:57.519+0000 7f4e6be72640 1 -- 172.21.15.98:0/2824426314 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4e640af2a0 con 0x7f4e500052f0 2024-09-19T08:38:58.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.187+0000 7f4e5a7fc640 1 -- 172.21.15.98:0/2824426314 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v24) ==== 169+0+0 (secure 0 0 0) 0x7f4e540564c0 con 0x7f4e500052f0 2024-09-19T08:38:58.193 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' removed 2024-09-19T08:38:58.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.195+0000 7f4e6be72640 1 -- 172.21.15.98:0/2824426314 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4e480678f0 msgr2=0x7f4e48069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:58.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.195+0000 7f4e6be72640 1 --2- 172.21.15.98:0/2824426314 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4e480678f0 0x7f4e48069db0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4e64072b10 tx=0x7f4e6000b040 comp rx=0 tx=0).stop 2024-09-19T08:38:58.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.195+0000 7f4e6be72640 1 -- 172.21.15.98:0/2824426314 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e500052f0 msgr2=0x7f4e64072b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:58.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.195+0000 7f4e6be72640 1 --2- 172.21.15.98:0/2824426314 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e500052f0 0x7f4e64072b60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4e5400d570 tx=0x7f4e5400da30 comp rx=0 tx=0).stop 2024-09-19T08:38:58.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.195+0000 7f4e6be72640 1 -- 172.21.15.98:0/2824426314 shutdown_connections 2024-09-19T08:38:58.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.195+0000 7f4e6be72640 1 --2- 172.21.15.98:0/2824426314 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4e480678f0 0x7f4e48069db0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:58.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.195+0000 7f4e6be72640 1 --2- 172.21.15.98:0/2824426314 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e640ae8b0 0x7f4e640aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:58.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.195+0000 7f4e6be72640 1 --2- 172.21.15.98:0/2824426314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e641f51f0 0x7f4e6415ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:58.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.195+0000 7f4e6be72640 1 --2- 172.21.15.98:0/2824426314 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e500052f0 0x7f4e64072b60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:58.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.195+0000 7f4e6be72640 1 -- 172.21.15.98:0/2824426314 >> 172.21.15.98:0/2824426314 conn(0x7f4e6406c090 msgr2=0x7f4e6415d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:58.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.195+0000 7f4e6be72640 1 -- 172.21.15.98:0/2824426314 shutdown_connections 2024-09-19T08:38:58.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.195+0000 7f4e6be72640 1 -- 172.21.15.98:0/2824426314 wait complete. 2024-09-19T08:38:58.220 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-19T08:38:58.220 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-19T08:38:58.220 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-19T08:38:58.220 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:38:58.221 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-19T08:38:58.221 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-19T08:38:58.221 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-19T08:38:58.221 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:38:58.221 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-19T08:38:58.221 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-19T08:38:58.221 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-19T08:38:58.222 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:38:58.222 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-19T08:38:58.222 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-19T08:38:58.222 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-19T08:38:58.222 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:38:58.231 INFO:tasks.ceph.mds.c:Stopped 2024-09-19T08:38:58.231 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-19T08:38:58.233 INFO:tasks.ceph.mds.a:Stopped 2024-09-19T08:38:58.233 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-19T08:38:58.240 INFO:tasks.ceph.mds.b:Stopped 2024-09-19T08:38:58.240 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-19T08:38:58.241 INFO:tasks.ceph.mds.d:Stopped 2024-09-19T08:38:58.244 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-19T08:38:58.282 INFO:tasks.ceph.mds.c:Started 2024-09-19T08:38:58.282 DEBUG:teuthology.parallel:result is None 2024-09-19T08:38:58.283 INFO:tasks.ceph.mds.a:Started 2024-09-19T08:38:58.283 DEBUG:teuthology.parallel:result is None 2024-09-19T08:38:58.285 INFO:tasks.ceph.mds.b:Started 2024-09-19T08:38:58.285 DEBUG:teuthology.parallel:result is None 2024-09-19T08:38:58.286 INFO:tasks.ceph.mds.d:Started 2024-09-19T08:38:58.286 DEBUG:teuthology.parallel:result is None 2024-09-19T08:38:58.286 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-19T08:38:58.347 INFO:tasks.ceph.mds.a.smithi098.stdout:starting mds.a at 2024-09-19T08:38:58.351 INFO:tasks.ceph.mds.c.smithi098.stdout:starting mds.c at 2024-09-19T08:38:58.355 INFO:tasks.ceph.mds.d.smithi139.stdout:starting mds.d at 2024-09-19T08:38:58.355 INFO:tasks.ceph.mds.b.smithi139.stdout:starting mds.b at 2024-09-19T08:38:58.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.411+0000 7f6527431640 1 Processor -- start 2024-09-19T08:38:58.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.411+0000 7f6527431640 1 -- start start 2024-09-19T08:38:58.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.411+0000 7f6527431640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65200d4820 0x7f65201621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:58.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.411+0000 7f6527431640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f65200b9970 0x7f65200b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:58.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.411+0000 7f6527431640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6520068440 con 0x7f65200b7f80 2024-09-19T08:38:58.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.411+0000 7f6527431640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f6520163050 con 0x7f65200b9970 2024-09-19T08:38:58.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.411+0000 7f6527431640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f65200ba360 con 0x7f65200d4820 2024-09-19T08:38:58.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.411+0000 7f65249cc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65200d4820 0x7f65201621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:58.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.411+0000 7f65249cc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65200d4820 0x7f65201621b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:57388/0 (socket says 172.21.15.98:57388) 2024-09-19T08:38:58.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.411+0000 7f65249cc640 1 -- 172.21.15.98:0/1565253334 learned_addr learned my addr 172.21.15.98:0/1565253334 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:58.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.411+0000 7f65259ce640 1 --2- 172.21.15.98:0/1565253334 >> v2:172.21.15.139:3300/0 conn(0x7f65200b9970 0x7f65200b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:58.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.411+0000 7f65259ce640 1 -- 172.21.15.98:0/1565253334 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65200d4820 msgr2=0x7f65201621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:58.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.411+0000 7f65259ce640 1 --2- 172.21.15.98:0/1565253334 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65200d4820 0x7f65201621b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:58.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.411+0000 7f65259ce640 1 -- 172.21.15.98:0/1565253334 >> v1:172.21.15.139:6789/0 conn(0x7f65200b7f80 legacy=0x7f652015f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:58.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.411+0000 7f65259ce640 1 -- 172.21.15.98:0/1565253334 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6520162a90 con 0x7f65200b9970 2024-09-19T08:38:58.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.415+0000 7f65259ce640 1 --2- 172.21.15.98:0/1565253334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65200b9970 0x7f65200b9e20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6518009d90 tx=0x7f651801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9e13bf3dcccf292 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:58.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.415+0000 7f6517fff640 1 -- 172.21.15.98:0/1565253334 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65180b2070 con 0x7f65200b9970 2024-09-19T08:38:58.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.415+0000 7f6517fff640 1 -- 172.21.15.98:0/1565253334 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65180a5920 con 0x7f65200b9970 2024-09-19T08:38:58.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.415+0000 7f6517fff640 1 -- 172.21.15.98:0/1565253334 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65180a5cf0 con 0x7f65200b9970 2024-09-19T08:38:58.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.415+0000 7f6527431640 1 -- 172.21.15.98:0/1565253334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65200b9970 msgr2=0x7f65200b9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:58.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.415+0000 7f6527431640 1 --2- 172.21.15.98:0/1565253334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65200b9970 0x7f65200b9e20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6518009d90 tx=0x7f651801c970 comp rx=0 tx=0).stop 2024-09-19T08:38:58.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.415+0000 7f6527431640 1 -- 172.21.15.98:0/1565253334 shutdown_connections 2024-09-19T08:38:58.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.415+0000 7f6527431640 1 --2- 172.21.15.98:0/1565253334 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65200b9970 0x7f65200b9e20 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:58.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.415+0000 7f6527431640 1 --2- 172.21.15.98:0/1565253334 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65200d4820 0x7f65201621b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:58.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.415+0000 7f6527431640 1 -- 172.21.15.98:0/1565253334 >> 172.21.15.98:0/1565253334 conn(0x7f6520074e40 msgr2=0x7f6520075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:58.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.415+0000 7f6527431640 1 -- 172.21.15.98:0/1565253334 shutdown_connections 2024-09-19T08:38:58.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.415+0000 7f6527431640 1 -- 172.21.15.98:0/1565253334 wait complete. 2024-09-19T08:38:58.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.415+0000 7f6527431640 1 Processor -- start 2024-09-19T08:38:58.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.415+0000 7f6527431640 1 -- start start 2024-09-19T08:38:58.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.415+0000 7f6527431640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65200b7f80 0x7f652013e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:58.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.415+0000 7f6527431640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65200b9970 0x7f6520141bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:58.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.415+0000 7f6527431640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65200d4820 0x7f6520139bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:58.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.415+0000 7f6527431640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f65200b84e0 con 0x7f65200b7f80 2024-09-19T08:38:58.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.415+0000 7f6527431640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f65200b8360 con 0x7f65200d4820 2024-09-19T08:38:58.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.415+0000 7f6527431640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6520144590 con 0x7f65200b9970 2024-09-19T08:38:58.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.415+0000 7f65251cd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65200b7f80 0x7f652013e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:58.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.415+0000 7f65251cd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65200b7f80 0x7f652013e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35864/0 (socket says 172.21.15.98:35864) 2024-09-19T08:38:58.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.415+0000 7f65251cd640 1 -- 172.21.15.98:0/521214294 learned_addr learned my addr 172.21.15.98:0/521214294 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:58.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.415+0000 7f65259ce640 1 --2- 172.21.15.98:0/521214294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65200d4820 0x7f6520139bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:58.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.415+0000 7f65249cc640 1 --2- 172.21.15.98:0/521214294 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65200b9970 0x7f6520141bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:58.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.415+0000 7f65251cd640 1 -- 172.21.15.98:0/521214294 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65200b9970 msgr2=0x7f6520141bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:58.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.415+0000 7f65251cd640 1 --2- 172.21.15.98:0/521214294 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65200b9970 0x7f6520141bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:58.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.415+0000 7f65251cd640 1 -- 172.21.15.98:0/521214294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65200d4820 msgr2=0x7f6520139bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:58.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.415+0000 7f65251cd640 1 --2- 172.21.15.98:0/521214294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65200d4820 0x7f6520139bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:58.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.415+0000 7f65251cd640 1 -- 172.21.15.98:0/521214294 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f652013eb00 con 0x7f65200b7f80 2024-09-19T08:38:58.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.415+0000 7f65259ce640 1 --2- 172.21.15.98:0/521214294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65200d4820 0x7f6520139bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:38:58.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.415+0000 7f65251cd640 1 --2- 172.21.15.98:0/521214294 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65200b7f80 0x7f652013e5c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f651c00c370 tx=0x7f651c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:58.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.419+0000 7f6515ffb640 1 -- 172.21.15.98:0/521214294 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c0052d0 con 0x7f65200b7f80 2024-09-19T08:38:58.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.419+0000 7f6527431640 1 -- 172.21.15.98:0/521214294 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6520142660 con 0x7f65200b7f80 2024-09-19T08:38:58.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.419+0000 7f6515ffb640 1 -- 172.21.15.98:0/521214294 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f651c005470 con 0x7f65200b7f80 2024-09-19T08:38:58.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.419+0000 7f6515ffb640 1 -- 172.21.15.98:0/521214294 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c007260 con 0x7f65200b7f80 2024-09-19T08:38:58.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.419+0000 7f6527431640 1 -- 172.21.15.98:0/521214294 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6520142c20 con 0x7f65200b7f80 2024-09-19T08:38:58.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.419+0000 7f6527431640 1 -- 172.21.15.98:0/521214294 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65200b9e20 con 0x7f65200b7f80 2024-09-19T08:38:58.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.419+0000 7f6515ffb640 1 -- 172.21.15.98:0/521214294 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f651c007400 con 0x7f65200b7f80 2024-09-19T08:38:58.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.419+0000 7f6515ffb640 1 --2- 172.21.15.98:0/521214294 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f65040678f0 0x7f6504069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:58.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.419+0000 7f6515ffb640 1 -- 172.21.15.98:0/521214294 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 4943+0+0 (secure 0 0 0) 0x7f651c084f00 con 0x7f65200b7f80 2024-09-19T08:38:58.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.419+0000 7f65249cc640 1 --2- 172.21.15.98:0/521214294 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f65040678f0 0x7f6504069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:58.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.419+0000 7f65249cc640 1 --2- 172.21.15.98:0/521214294 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f65040678f0 0x7f6504069db0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f652015e5e0 tx=0x7f6510008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:58.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.423+0000 7f6515ffb640 1 -- 172.21.15.98:0/521214294 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f651c055c20 con 0x7f65200b7f80 2024-09-19T08:38:58.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:58.599+0000 7f6527431640 1 -- 172.21.15.98:0/521214294 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f6520067a00 con 0x7f65200b7f80 2024-09-19T08:38:59.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.211+0000 7f6515ffb640 1 -- 172.21.15.98:0/521214294 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v26) ==== 97+0+0 (secure 0 0 0) 0x7f651c055520 con 0x7f65200b7f80 2024-09-19T08:38:59.214 INFO:teuthology.orchestra.run.smithi098.stderr: removed all blocklist entries 2024-09-19T08:38:59.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.211+0000 7f6527431640 1 -- 172.21.15.98:0/521214294 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f65040678f0 msgr2=0x7f6504069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:59.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.211+0000 7f6527431640 1 --2- 172.21.15.98:0/521214294 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f65040678f0 0x7f6504069db0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f652015e5e0 tx=0x7f6510008040 comp rx=0 tx=0).stop 2024-09-19T08:38:59.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.211+0000 7f6527431640 1 -- 172.21.15.98:0/521214294 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65200b7f80 msgr2=0x7f652013e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:59.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.211+0000 7f6527431640 1 --2- 172.21.15.98:0/521214294 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65200b7f80 0x7f652013e5c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f651c00c370 tx=0x7f651c00c830 comp rx=0 tx=0).stop 2024-09-19T08:38:59.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.211+0000 7f6527431640 1 -- 172.21.15.98:0/521214294 shutdown_connections 2024-09-19T08:38:59.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.211+0000 7f6527431640 1 --2- 172.21.15.98:0/521214294 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f65040678f0 0x7f6504069db0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:59.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.211+0000 7f6527431640 1 --2- 172.21.15.98:0/521214294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65200d4820 0x7f6520139bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:59.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.211+0000 7f6527431640 1 --2- 172.21.15.98:0/521214294 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65200b9970 0x7f6520141bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:59.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.211+0000 7f6527431640 1 --2- 172.21.15.98:0/521214294 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65200b7f80 0x7f652013e5c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:59.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.211+0000 7f6527431640 1 -- 172.21.15.98:0/521214294 >> 172.21.15.98:0/521214294 conn(0x7f6520074e40 msgr2=0x7f65200d34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:59.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.211+0000 7f6527431640 1 -- 172.21.15.98:0/521214294 shutdown_connections 2024-09-19T08:38:59.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.211+0000 7f6527431640 1 -- 172.21.15.98:0/521214294 wait complete. 2024-09-19T08:38:59.239 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-19T08:38:59.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.355+0000 7f4d713e8640 1 Processor -- start 2024-09-19T08:38:59.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.359+0000 7f4d713e8640 1 -- start start 2024-09-19T08:38:59.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.359+0000 7f4d713e8640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4d6c0b7f80 0x7f4d6c15f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:59.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.359+0000 7f4d713e8640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4d6c0d4820 0x7f4d6c1621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:59.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.359+0000 7f4d713e8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d6c0b9970 0x7f4d6c0b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:59.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.359+0000 7f4d713e8640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f4d6c163050 con 0x7f4d6c0d4820 2024-09-19T08:38:59.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.359+0000 7f4d713e8640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f4d6c162a90 con 0x7f4d6c0b7f80 2024-09-19T08:38:59.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.359+0000 7f4d713e8640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4d6c162d70 con 0x7f4d6c0b9970 2024-09-19T08:38:59.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.359+0000 7f4d6a7fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4d6c0d4820 0x7f4d6c1621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:59.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.359+0000 7f4d6a7fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4d6c0d4820 0x7f4d6c1621b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35866/0 (socket says 172.21.15.98:35866) 2024-09-19T08:38:59.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.359+0000 7f4d6a7fc640 1 -- 172.21.15.98:0/1176113521 learned_addr learned my addr 172.21.15.98:0/1176113521 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:59.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.359+0000 7f4d6b7fe640 1 --2- 172.21.15.98:0/1176113521 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d6c0b9970 0x7f4d6c0b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:59.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.359+0000 7f4d6affd640 1 --2- 172.21.15.98:0/1176113521 >> v2:172.21.15.139:3300/0 conn(0x7f4d6c0b7f80 0x7f4d6c15f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:59.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.359+0000 7f4d6a7fc640 1 -- 172.21.15.98:0/1176113521 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d6c0b9970 msgr2=0x7f4d6c0b9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:59.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.359+0000 7f4d6a7fc640 1 --2- 172.21.15.98:0/1176113521 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d6c0b9970 0x7f4d6c0b9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:59.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.359+0000 7f4d6a7fc640 1 -- 172.21.15.98:0/1176113521 >> v2:172.21.15.139:3300/0 conn(0x7f4d6c0b7f80 msgr2=0x7f4d6c15f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:59.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.359+0000 7f4d6a7fc640 1 --2- 172.21.15.98:0/1176113521 >> v2:172.21.15.139:3300/0 conn(0x7f4d6c0b7f80 0x7f4d6c15f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:59.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.359+0000 7f4d6a7fc640 1 -- 172.21.15.98:0/1176113521 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6c068440 con 0x7f4d6c0d4820 2024-09-19T08:38:59.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.359+0000 7f4d6b7fe640 1 --2- 172.21.15.98:0/1176113521 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d6c0b9970 0x7f4d6c0b9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:59.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.359+0000 7f4d6a7fc640 1 --2- 172.21.15.98:0/1176113521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d6c0d4820 0x7f4d6c1621b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c0d0dd0 tx=0x7f4d5c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad11960186dadfa6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:59.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.359+0000 7f4d69ffb640 1 -- 172.21.15.98:0/1176113521 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c0b1070 con 0x7f4d6c0d4820 2024-09-19T08:38:59.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.359+0000 7f4d69ffb640 1 -- 172.21.15.98:0/1176113521 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d5c0ac410 con 0x7f4d6c0d4820 2024-09-19T08:38:59.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.359+0000 7f4d69ffb640 1 -- 172.21.15.98:0/1176113521 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c0ac7e0 con 0x7f4d6c0d4820 2024-09-19T08:38:59.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.363+0000 7f4d713e8640 1 -- 172.21.15.98:0/1176113521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d6c0d4820 msgr2=0x7f4d6c1621b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:59.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.363+0000 7f4d713e8640 1 --2- 172.21.15.98:0/1176113521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d6c0d4820 0x7f4d6c1621b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c0d0dd0 tx=0x7f4d5c01c520 comp rx=0 tx=0).stop 2024-09-19T08:38:59.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.363+0000 7f4d713e8640 1 -- 172.21.15.98:0/1176113521 shutdown_connections 2024-09-19T08:38:59.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.363+0000 7f4d713e8640 1 --2- 172.21.15.98:0/1176113521 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d6c0d4820 0x7f4d6c1621b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:59.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.363+0000 7f4d713e8640 1 --2- 172.21.15.98:0/1176113521 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d6c0b9970 0x7f4d6c0b9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:59.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.363+0000 7f4d713e8640 1 --2- 172.21.15.98:0/1176113521 >> v2:172.21.15.139:3300/0 conn(0x7f4d6c0b7f80 0x7f4d6c15f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:59.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.363+0000 7f4d713e8640 1 -- 172.21.15.98:0/1176113521 >> 172.21.15.98:0/1176113521 conn(0x7f4d6c074e40 msgr2=0x7f4d6c075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:59.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.363+0000 7f4d713e8640 1 -- 172.21.15.98:0/1176113521 shutdown_connections 2024-09-19T08:38:59.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.363+0000 7f4d713e8640 1 -- 172.21.15.98:0/1176113521 wait complete. 2024-09-19T08:38:59.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.363+0000 7f4d713e8640 1 Processor -- start 2024-09-19T08:38:59.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.363+0000 7f4d713e8640 1 -- start start 2024-09-19T08:38:59.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.363+0000 7f4d713e8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d6c0b7f80 0x7f4d6c13e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:59.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.363+0000 7f4d713e8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d6c0b9970 0x7f4d6c0d0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:59.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.363+0000 7f4d713e8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d6c0d4820 0x7f4d6c0d1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:59.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.363+0000 7f4d713e8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c162870 con 0x7f4d6c0b9970 2024-09-19T08:38:59.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.363+0000 7f4d713e8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c1626f0 con 0x7f4d6c0b7f80 2024-09-19T08:38:59.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.363+0000 7f4d713e8640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4d6c0b8360 con 0x7f4d6c0d4820 2024-09-19T08:38:59.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.363+0000 7f4d6a7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d6c0b9970 0x7f4d6c0d0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:59.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.363+0000 7f4d6a7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d6c0b9970 0x7f4d6c0d0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35872/0 (socket says 172.21.15.98:35872) 2024-09-19T08:38:59.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.363+0000 7f4d6a7fc640 1 -- 172.21.15.98:0/943836348 learned_addr learned my addr 172.21.15.98:0/943836348 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:59.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.363+0000 7f4d6b7fe640 1 --2- 172.21.15.98:0/943836348 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d6c0d4820 0x7f4d6c0d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:59.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.363+0000 7f4d6affd640 1 --2- 172.21.15.98:0/943836348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d6c0b7f80 0x7f4d6c13e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:59.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.363+0000 7f4d6a7fc640 1 -- 172.21.15.98:0/943836348 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d6c0d4820 msgr2=0x7f4d6c0d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:59.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.363+0000 7f4d6a7fc640 1 --2- 172.21.15.98:0/943836348 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d6c0d4820 0x7f4d6c0d1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:59.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.363+0000 7f4d6a7fc640 1 -- 172.21.15.98:0/943836348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d6c0b7f80 msgr2=0x7f4d6c13e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:59.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.363+0000 7f4d6a7fc640 1 --2- 172.21.15.98:0/943836348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d6c0b7f80 0x7f4d6c13e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:59.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.363+0000 7f4d6a7fc640 1 -- 172.21.15.98:0/943836348 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6c1517e0 con 0x7f4d6c0b9970 2024-09-19T08:38:59.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.363+0000 7f4d6b7fe640 1 --2- 172.21.15.98:0/943836348 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d6c0d4820 0x7f4d6c0d1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:59.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.363+0000 7f4d6affd640 1 --2- 172.21.15.98:0/943836348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d6c0b7f80 0x7f4d6c13e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:38:59.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.363+0000 7f4d6a7fc640 1 --2- 172.21.15.98:0/943836348 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d6c0b9970 0x7f4d6c0d0ed0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c01abd0 tx=0x7f4d5c0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:59.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.363+0000 7f4d637fe640 1 -- 172.21.15.98:0/943836348 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c0b1040 con 0x7f4d6c0b9970 2024-09-19T08:38:59.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.363+0000 7f4d637fe640 1 -- 172.21.15.98:0/943836348 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d5c0ade30 con 0x7f4d6c0b9970 2024-09-19T08:38:59.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.363+0000 7f4d637fe640 1 -- 172.21.15.98:0/943836348 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c0b7450 con 0x7f4d6c0b9970 2024-09-19T08:38:59.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.367+0000 7f4d713e8640 1 -- 172.21.15.98:0/943836348 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d6c157f00 con 0x7f4d6c0b9970 2024-09-19T08:38:59.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.367+0000 7f4d713e8640 1 -- 172.21.15.98:0/943836348 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d6c1584c0 con 0x7f4d6c0b9970 2024-09-19T08:38:59.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.367+0000 7f4d713e8640 1 -- 172.21.15.98:0/943836348 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d6c0b9de0 con 0x7f4d6c0b9970 2024-09-19T08:38:59.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.367+0000 7f4d637fe640 1 -- 172.21.15.98:0/943836348 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4d5c0be020 con 0x7f4d6c0b9970 2024-09-19T08:38:59.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.367+0000 7f4d637fe640 1 --2- 172.21.15.98:0/943836348 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4d4c0678f0 0x7f4d4c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:59.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.367+0000 7f4d637fe640 1 -- 172.21.15.98:0/943836348 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f4d5c11f460 con 0x7f4d6c0b9970 2024-09-19T08:38:59.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.367+0000 7f4d6affd640 1 --2- 172.21.15.98:0/943836348 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4d4c0678f0 0x7f4d4c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:59.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.367+0000 7f4d6affd640 1 --2- 172.21.15.98:0/943836348 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4d4c0678f0 0x7f4d4c069db0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c14fee0 tx=0x7f4d54008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:59.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.371+0000 7f4d637fe640 1 -- 172.21.15.98:0/943836348 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f4d5c016110 con 0x7f4d6c0b9970 2024-09-19T08:38:59.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.591+0000 7f4d713e8640 1 -- 172.21.15.98:0/943836348 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f4d6c152920 con 0x7f4d6c0b9970 2024-09-19T08:38:59.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.591+0000 7f4d637fe640 1 -- 172.21.15.98:0/943836348 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) ==== 80+0+6645 (secure 0 0 0) 0x7f4d5c0b5310 con 0x7f4d6c0b9970 2024-09-19T08:38:59.597 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:38:59.597 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:38:59.597 INFO:teuthology.orchestra.run.smithi098.stdout: "auth_dump": [ 2024-09-19T08:38:59.597 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:38:59.597 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.a", 2024-09-19T08:38:59.597 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmuujGBRAAa2k/kW68NJI6nUKx+W+G2Q==", 2024-09-19T08:38:59.597 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:38:59.597 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:38:59.597 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:38:59.597 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:38:59.597 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:38:59.597 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:38:59.598 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:38:59.598 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:38:59.598 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.b", 2024-09-19T08:38:59.598 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmFoojFBAAaOZPpa41fdjBPEn3CGsJnA==", 2024-09-19T08:38:59.598 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:38:59.598 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:38:59.598 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:38:59.598 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:38:59.598 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:38:59.598 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:38:59.598 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:38:59.598 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:38:59.598 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.c", 2024-09-19T08:38:59.599 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmcQcDDhAAF0QJMlljMH+DiHfWW+/Sog==", 2024-09-19T08:38:59.599 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:38:59.599 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:38:59.599 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:38:59.599 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:38:59.599 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:38:59.599 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:38:59.599 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:38:59.599 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:38:59.599 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.d", 2024-09-19T08:38:59.599 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmwWmqHBAAf15VcrdeHf8AzQjGsN9LoA==", 2024-09-19T08:38:59.599 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:38:59.599 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:38:59.599 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:38:59.599 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:38:59.600 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:38:59.600 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:38:59.600 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:38:59.600 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:38:59.600 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.0", 2024-09-19T08:38:59.600 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDC4utm9PjNKxAAa6SG9AE4smPiu4azRAFKvg==", 2024-09-19T08:38:59.600 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:38:59.600 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:38:59.600 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:38:59.600 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:38:59.600 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:38:59.600 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:38:59.600 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:38:59.601 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.1", 2024-09-19T08:38:59.601 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDG4utmHy6UCxAAN6hpG4DJEp81iG/KoloTvQ==", 2024-09-19T08:38:59.601 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:38:59.601 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:38:59.601 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:38:59.601 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:38:59.601 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:38:59.601 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:38:59.601 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:38:59.601 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.2", 2024-09-19T08:38:59.601 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDJ4utmUVP4GhAAJNAL30sEAseoDG2fE6YbZQ==", 2024-09-19T08:38:59.601 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:38:59.601 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:38:59.602 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:38:59.602 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:38:59.602 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:38:59.602 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:38:59.602 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:38:59.602 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.3", 2024-09-19T08:38:59.602 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDM4utmTKgoLRAAiCrPRLML+wvjOt6RhKay3w==", 2024-09-19T08:38:59.602 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:38:59.602 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:38:59.602 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:38:59.602 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:38:59.602 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:38:59.602 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:38:59.602 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:38:59.603 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.4", 2024-09-19T08:38:59.603 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDS4utmAf6bMhAAI3uENuFoXLicUeuspgFjYA==", 2024-09-19T08:38:59.603 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:38:59.603 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:38:59.603 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:38:59.603 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:38:59.603 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:38:59.603 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:38:59.603 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:38:59.603 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.5", 2024-09-19T08:38:59.603 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDW4utmRjxwEhAAF1A2GtrZXVx9BlmjHjhUDQ==", 2024-09-19T08:38:59.603 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:38:59.603 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:38:59.603 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:38:59.604 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:38:59.604 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:38:59.604 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:38:59.604 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:38:59.604 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.6", 2024-09-19T08:38:59.604 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDZ4utmSk/XIRAAROSfzGL7smR88ZkDSx5jAg==", 2024-09-19T08:38:59.604 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:38:59.604 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:38:59.604 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:38:59.604 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:38:59.604 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:38:59.604 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:38:59.604 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:38:59.605 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.7", 2024-09-19T08:38:59.605 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDc4utmXfpbMxAAqEM4I/ACFvfu/sMA8bMcrQ==", 2024-09-19T08:38:59.605 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:38:59.605 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:38:59.605 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:38:59.605 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:38:59.605 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:38:59.605 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:38:59.605 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:38:59.605 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.0", 2024-09-19T08:38:59.605 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utm0vNpIhAA4S1E8dFZ0+ibmSKhmZ+bIA==", 2024-09-19T08:38:59.605 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:38:59.605 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:38:59.605 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-19T08:38:59.606 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-19T08:38:59.606 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-19T08:38:59.606 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:38:59.606 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:38:59.606 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:38:59.606 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.1", 2024-09-19T08:38:59.606 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmx7xyKhAAC84lykPEb01MiypXKQVREQ==", 2024-09-19T08:38:59.606 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:38:59.606 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:38:59.606 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-19T08:38:59.606 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-19T08:38:59.606 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-19T08:38:59.606 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:38:59.607 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:38:59.607 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:38:59.607 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.2", 2024-09-19T08:38:59.607 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmteOQJxAAtTF2Py21ZE2WRQjeQSrkKQ==", 2024-09-19T08:38:59.607 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:38:59.607 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:38:59.607 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-19T08:38:59.607 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-19T08:38:59.607 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-19T08:38:59.607 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:38:59.607 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:38:59.607 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:38:59.608 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.3", 2024-09-19T08:38:59.608 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmKN92LxAAMqraZOWRT6IIfx5AuDWS2g==", 2024-09-19T08:38:59.608 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:38:59.608 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:38:59.608 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-19T08:38:59.608 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-19T08:38:59.608 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-19T08:38:59.608 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:38:59.608 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:38:59.608 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:38:59.608 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.admin", 2024-09-19T08:38:59.608 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC+4utmLH97KhAA1DTYW5bTrGUG+7cb8+d7gA==", 2024-09-19T08:38:59.608 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:38:59.609 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-09-19T08:38:59.609 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:38:59.609 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:38:59.609 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:38:59.609 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:38:59.609 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:38:59.609 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:38:59.609 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.bootstrap-mds", 2024-09-19T08:38:59.609 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDj4utmxXbGGhAAeeTuqagfaAJR1KlF+tmW6A==", 2024-09-19T08:38:59.609 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:38:59.609 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile bootstrap-mds" 2024-09-19T08:38:59.609 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:38:59.609 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:38:59.609 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:38:59.610 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.bootstrap-mgr", 2024-09-19T08:38:59.610 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDj4utmCZLGGhAACCEgpIhuQgGE+ULHupg2HA==", 2024-09-19T08:38:59.610 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:38:59.610 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-19T08:38:59.610 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:38:59.610 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:38:59.610 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:38:59.610 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.bootstrap-osd", 2024-09-19T08:38:59.610 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDj4utmSajGGhAApgCtr9q0iK4vI7oS+txUoA==", 2024-09-19T08:38:59.610 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:38:59.610 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile bootstrap-osd" 2024-09-19T08:38:59.610 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:38:59.610 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:38:59.610 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:38:59.611 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.bootstrap-rbd", 2024-09-19T08:38:59.611 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDj4utmH77GGhAA0F7hplelM/JZNBU0zEU5kA==", 2024-09-19T08:38:59.611 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:38:59.611 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-19T08:38:59.611 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:38:59.611 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:38:59.611 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:38:59.611 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-19T08:38:59.611 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDj4utm1dTGGhAAyLk/nar33DBlY0hcYkjpXw==", 2024-09-19T08:38:59.611 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:38:59.611 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-19T08:38:59.611 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:38:59.611 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:38:59.611 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:38:59.612 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.bootstrap-rgw", 2024-09-19T08:38:59.612 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDj4utm+OvGGhAAOWLDJTMwJ/WvkJW5CWcj/Q==", 2024-09-19T08:38:59.612 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:38:59.612 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-19T08:38:59.612 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:38:59.612 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:38:59.612 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:38:59.612 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mgr.x", 2024-09-19T08:38:59.612 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmbKuyAhAA0/XjHUIPOIV/YvNEhB8rEQ==", 2024-09-19T08:38:59.612 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:38:59.612 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-09-19T08:38:59.612 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile mgr", 2024-09-19T08:38:59.612 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:38:59.613 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:38:59.613 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:38:59.613 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:38:59.613 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mgr.y", 2024-09-19T08:38:59.613 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC+4utmU+ZrOxAAsvcwINH6qyxfN7N+rDrkqg==", 2024-09-19T08:38:59.613 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:38:59.613 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-09-19T08:38:59.613 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile mgr", 2024-09-19T08:38:59.613 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:38:59.613 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:38:59.613 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:38:59.613 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-19T08:38:59.613 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:38:59.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.595+0000 7f4d713e8640 1 -- 172.21.15.98:0/943836348 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4d4c0678f0 msgr2=0x7f4d4c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:59.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.595+0000 7f4d713e8640 1 --2- 172.21.15.98:0/943836348 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4d4c0678f0 0x7f4d4c069db0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c14fee0 tx=0x7f4d54008040 comp rx=0 tx=0).stop 2024-09-19T08:38:59.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.595+0000 7f4d713e8640 1 -- 172.21.15.98:0/943836348 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d6c0b9970 msgr2=0x7f4d6c0d0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:59.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.595+0000 7f4d713e8640 1 --2- 172.21.15.98:0/943836348 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d6c0b9970 0x7f4d6c0d0ed0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c01abd0 tx=0x7f4d5c0192c0 comp rx=0 tx=0).stop 2024-09-19T08:38:59.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.595+0000 7f4d713e8640 1 -- 172.21.15.98:0/943836348 shutdown_connections 2024-09-19T08:38:59.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.595+0000 7f4d713e8640 1 --2- 172.21.15.98:0/943836348 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4d4c0678f0 0x7f4d4c069db0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:59.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.595+0000 7f4d713e8640 1 --2- 172.21.15.98:0/943836348 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d6c0d4820 0x7f4d6c0d1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:59.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.595+0000 7f4d713e8640 1 --2- 172.21.15.98:0/943836348 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d6c0b9970 0x7f4d6c0d0ed0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:59.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.595+0000 7f4d713e8640 1 --2- 172.21.15.98:0/943836348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d6c0b7f80 0x7f4d6c13e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:59.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.595+0000 7f4d713e8640 1 -- 172.21.15.98:0/943836348 >> 172.21.15.98:0/943836348 conn(0x7f4d6c074e40 msgr2=0x7f4d6c0752b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:59.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.595+0000 7f4d713e8640 1 -- 172.21.15.98:0/943836348 shutdown_connections 2024-09-19T08:38:59.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.595+0000 7f4d713e8640 1 -- 172.21.15.98:0/943836348 wait complete. 2024-09-19T08:38:59.616 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-19T08:38:59.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.731+0000 7fa276f4f640 1 Processor -- start 2024-09-19T08:38:59.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.731+0000 7fa276f4f640 1 -- start start 2024-09-19T08:38:59.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.731+0000 7fa276f4f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa27015e750 0x7fa270073c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:59.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.731+0000 7fa276f4f640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa2700b28e0 0x7fa270074c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:59.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.731+0000 7fa276f4f640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa270075b80 con 0x7fa2700751b0 2024-09-19T08:38:59.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.731+0000 7fa276f4f640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fa2700d1bd0 con 0x7fa2700b28e0 2024-09-19T08:38:59.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.731+0000 7fa276f4f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fa2700d1eb0 con 0x7fa27015e750 2024-09-19T08:38:59.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.731+0000 7fa26ffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa2700b28e0 0x7fa270074c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:59.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.731+0000 7fa2754ec640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fa2700751b0 0x7fa2700b6200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:55386/0 (socket says 172.21.15.98:55386) 2024-09-19T08:38:59.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.731+0000 7fa2754ec640 1 -- 172.21.15.98:0/4242326417 learned_addr learned my addr 172.21.15.98:0/4242326417 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:38:59.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.731+0000 7fa274ceb640 1 --2- 172.21.15.98:0/4242326417 >> v2:172.21.15.139:3300/0 conn(0x7fa27015e750 0x7fa270073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:59.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.731+0000 7fa26ffff640 1 -- 172.21.15.98:0/4242326417 >> v2:172.21.15.139:3300/0 conn(0x7fa27015e750 msgr2=0x7fa270073c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:38:59.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.731+0000 7fa26ffff640 1 --2- 172.21.15.98:0/4242326417 >> v2:172.21.15.139:3300/0 conn(0x7fa27015e750 0x7fa270073c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:59.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.731+0000 7fa26ffff640 1 -- 172.21.15.98:0/4242326417 >> v1:172.21.15.98:6789/0 conn(0x7fa2700751b0 legacy=0x7fa2700b6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:59.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.731+0000 7fa26f7fe640 1 -- 172.21.15.98:0/4242326417 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3584727753 0 0) 0x7fa270075b80 con 0x7fa2700751b0 2024-09-19T08:38:59.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.731+0000 7fa26ffff640 1 -- 172.21.15.98:0/4242326417 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2700cf000 con 0x7fa2700b28e0 2024-09-19T08:38:59.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.731+0000 7fa274ceb640 1 --2- 172.21.15.98:0/4242326417 >> v2:172.21.15.139:3300/0 conn(0x7fa27015e750 0x7fa270073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:38:59.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.735+0000 7fa26ffff640 1 --2- 172.21.15.98:0/4242326417 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2700b28e0 0x7fa270074c70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa270069910 tx=0x7fa26401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f55793aebf79c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:59.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.735+0000 7fa26f7fe640 1 -- 172.21.15.98:0/4242326417 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2640b0070 con 0x7fa2700b28e0 2024-09-19T08:38:59.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.735+0000 7fa26f7fe640 1 -- 172.21.15.98:0/4242326417 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2640ab600 con 0x7fa2700b28e0 2024-09-19T08:38:59.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.735+0000 7fa26f7fe640 1 -- 172.21.15.98:0/4242326417 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2640ab9d0 con 0x7fa2700b28e0 2024-09-19T08:38:59.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.735+0000 7fa276f4f640 1 -- 172.21.15.98:0/4242326417 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2700b28e0 msgr2=0x7fa270074c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:59.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.735+0000 7fa276f4f640 1 --2- 172.21.15.98:0/4242326417 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2700b28e0 0x7fa270074c70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa270069910 tx=0x7fa26401c3f0 comp rx=0 tx=0).stop 2024-09-19T08:38:59.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.735+0000 7fa276f4f640 1 -- 172.21.15.98:0/4242326417 shutdown_connections 2024-09-19T08:38:59.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.735+0000 7fa276f4f640 1 --2- 172.21.15.98:0/4242326417 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2700b28e0 0x7fa270074c70 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:59.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.735+0000 7fa276f4f640 1 --2- 172.21.15.98:0/4242326417 >> v2:172.21.15.139:3300/0 conn(0x7fa27015e750 0x7fa270073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:59.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.735+0000 7fa276f4f640 1 -- 172.21.15.98:0/4242326417 >> 172.21.15.98:0/4242326417 conn(0x7fa270073170 msgr2=0x7fa270073580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:59.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.735+0000 7fa276f4f640 1 -- 172.21.15.98:0/4242326417 shutdown_connections 2024-09-19T08:38:59.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.735+0000 7fa276f4f640 1 -- 172.21.15.98:0/4242326417 wait complete. 2024-09-19T08:38:59.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.735+0000 7fa276f4f640 1 Processor -- start 2024-09-19T08:38:59.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.735+0000 7fa276f4f640 1 -- start start 2024-09-19T08:38:59.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.735+0000 7fa276f4f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2700751b0 0x7fa270161820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:59.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.735+0000 7fa276f4f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2700b28e0 0x7fa27009aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:59.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.735+0000 7fa276f4f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa27015e750 0x7fa270157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:59.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.735+0000 7fa276f4f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa27016b500 con 0x7fa27015e750 2024-09-19T08:38:59.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.735+0000 7fa276f4f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa27016b380 con 0x7fa2700751b0 2024-09-19T08:38:59.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.735+0000 7fa276f4f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa27016b680 con 0x7fa2700b28e0 2024-09-19T08:38:59.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.735+0000 7fa2754ec640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa27015e750 0x7fa270157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:59.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.735+0000 7fa2754ec640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa27015e750 0x7fa270157da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35884/0 (socket says 172.21.15.98:35884) 2024-09-19T08:38:59.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.735+0000 7fa2754ec640 1 -- 172.21.15.98:0/3557060866 learned_addr learned my addr 172.21.15.98:0/3557060866 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:38:59.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.735+0000 7fa274ceb640 1 --2- 172.21.15.98:0/3557060866 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2700751b0 0x7fa270161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:59.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.739+0000 7fa26ffff640 1 --2- 172.21.15.98:0/3557060866 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2700b28e0 0x7fa27009aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:59.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.739+0000 7fa2754ec640 1 -- 172.21.15.98:0/3557060866 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2700b28e0 msgr2=0x7fa27009aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:59.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.739+0000 7fa2754ec640 1 --2- 172.21.15.98:0/3557060866 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2700b28e0 0x7fa27009aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:59.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.739+0000 7fa2754ec640 1 -- 172.21.15.98:0/3557060866 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2700751b0 msgr2=0x7fa270161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:59.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.739+0000 7fa2754ec640 1 --2- 172.21.15.98:0/3557060866 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2700751b0 0x7fa270161820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:59.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.739+0000 7fa2754ec640 1 -- 172.21.15.98:0/3557060866 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2701582e0 con 0x7fa27015e750 2024-09-19T08:38:59.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.739+0000 7fa2754ec640 1 --2- 172.21.15.98:0/3557060866 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa27015e750 0x7fa270157da0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa26800c4b0 tx=0x7fa26800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:59.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.739+0000 7fa26d7fa640 1 -- 172.21.15.98:0/3557060866 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa268018070 con 0x7fa27015e750 2024-09-19T08:38:59.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.739+0000 7fa26d7fa640 1 -- 172.21.15.98:0/3557060866 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2680074e0 con 0x7fa27015e750 2024-09-19T08:38:59.744 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.739+0000 7fa276f4f640 1 -- 172.21.15.98:0/3557060866 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa270156a20 con 0x7fa27015e750 2024-09-19T08:38:59.744 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.739+0000 7fa26d7fa640 1 -- 172.21.15.98:0/3557060866 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa268004a30 con 0x7fa27015e750 2024-09-19T08:38:59.744 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.739+0000 7fa276f4f640 1 -- 172.21.15.98:0/3557060866 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa270156fe0 con 0x7fa27015e750 2024-09-19T08:38:59.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.739+0000 7fa276f4f640 1 -- 172.21.15.98:0/3557060866 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa270074c70 con 0x7fa27015e750 2024-09-19T08:38:59.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.739+0000 7fa26d7fa640 1 -- 172.21.15.98:0/3557060866 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa26801b070 con 0x7fa27015e750 2024-09-19T08:38:59.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.739+0000 7fa26d7fa640 1 --2- 172.21.15.98:0/3557060866 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa25c0678c0 0x7fa25c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:38:59.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.739+0000 7fa26d7fa640 1 -- 172.21.15.98:0/3557060866 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7fa268084dd0 con 0x7fa27015e750 2024-09-19T08:38:59.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.739+0000 7fa274ceb640 1 --2- 172.21.15.98:0/3557060866 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa25c0678c0 0x7fa25c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:38:59.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.739+0000 7fa274ceb640 1 --2- 172.21.15.98:0/3557060866 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa25c0678c0 0x7fa25c069d80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa260009fd0 tx=0x7fa2600097f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:38:59.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.743+0000 7fa26d7fa640 1 -- 172.21.15.98:0/3557060866 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa2680556f0 con 0x7fa27015e750 2024-09-19T08:38:59.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.955+0000 7fa276f4f640 1 -- 172.21.15.98:0/3557060866 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fa270159080 con 0x7fa27015e750 2024-09-19T08:38:59.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.959+0000 7fa26d7fa640 1 -- 172.21.15.98:0/3557060866 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7fa2680599f0 con 0x7fa27015e750 2024-09-19T08:38:59.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.959+0000 7fa276f4f640 1 -- 172.21.15.98:0/3557060866 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa25c0678c0 msgr2=0x7fa25c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:59.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.959+0000 7fa276f4f640 1 --2- 172.21.15.98:0/3557060866 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa25c0678c0 0x7fa25c069d80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa260009fd0 tx=0x7fa2600097f0 comp rx=0 tx=0).stop 2024-09-19T08:38:59.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.959+0000 7fa276f4f640 1 -- 172.21.15.98:0/3557060866 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa27015e750 msgr2=0x7fa270157da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:38:59.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.959+0000 7fa276f4f640 1 --2- 172.21.15.98:0/3557060866 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa27015e750 0x7fa270157da0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa26800c4b0 tx=0x7fa26800c970 comp rx=0 tx=0).stop 2024-09-19T08:38:59.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.959+0000 7fa276f4f640 1 -- 172.21.15.98:0/3557060866 shutdown_connections 2024-09-19T08:38:59.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.959+0000 7fa276f4f640 1 --2- 172.21.15.98:0/3557060866 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa25c0678c0 0x7fa25c069d80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:59.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.959+0000 7fa276f4f640 1 --2- 172.21.15.98:0/3557060866 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa27015e750 0x7fa270157da0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:59.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.959+0000 7fa276f4f640 1 --2- 172.21.15.98:0/3557060866 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2700b28e0 0x7fa27009aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:59.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.959+0000 7fa276f4f640 1 --2- 172.21.15.98:0/3557060866 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2700751b0 0x7fa270161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:38:59.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.959+0000 7fa276f4f640 1 -- 172.21.15.98:0/3557060866 >> 172.21.15.98:0/3557060866 conn(0x7fa270073170 msgr2=0x7fa2701585f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:38:59.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.959+0000 7fa276f4f640 1 -- 172.21.15.98:0/3557060866 shutdown_connections 2024-09-19T08:38:59.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:38:59.959+0000 7fa276f4f640 1 -- 172.21.15.98:0/3557060866 wait complete. 2024-09-19T08:38:59.980 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-19T08:39:00.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.087+0000 7ff4f2973640 1 Processor -- start 2024-09-19T08:39:00.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.087+0000 7ff4f2973640 1 -- start start 2024-09-19T08:39:00.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.087+0000 7ff4f2973640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff4ec15e750 0x7ff4ec073c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:00.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.087+0000 7ff4f2973640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff4ec075380 0x7ff4ec0cf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:00.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.087+0000 7ff4f2973640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4ec075b80 con 0x7ff4ec0b28e0 2024-09-19T08:39:00.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.087+0000 7ff4f2973640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7ff4ec0d1bd0 con 0x7ff4ec075380 2024-09-19T08:39:00.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.087+0000 7ff4f2973640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff4ec0d1eb0 con 0x7ff4ec15e750 2024-09-19T08:39:00.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.087+0000 7ff4f0f10640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff4ec075380 0x7ff4ec0cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:00.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.087+0000 7ff4f0f10640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff4ec075380 0x7ff4ec0cf520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35892/0 (socket says 172.21.15.98:35892) 2024-09-19T08:39:00.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.087+0000 7ff4f0f10640 1 -- 172.21.15.98:0/1592093306 learned_addr learned my addr 172.21.15.98:0/1592093306 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:00.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.087+0000 7ff4eb7fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7ff4ec0b28e0 0x7ff4ec074c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:55390/0 (socket says 172.21.15.98:55390) 2024-09-19T08:39:00.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.087+0000 7ff4ebfff640 1 --2- 172.21.15.98:0/1592093306 >> v2:172.21.15.139:3300/0 conn(0x7ff4ec15e750 0x7ff4ec073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:00.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.087+0000 7ff4f0f10640 1 -- 172.21.15.98:0/1592093306 >> v2:172.21.15.139:3300/0 conn(0x7ff4ec15e750 msgr2=0x7ff4ec073c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:00.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.087+0000 7ff4f0f10640 1 --2- 172.21.15.98:0/1592093306 >> v2:172.21.15.139:3300/0 conn(0x7ff4ec15e750 0x7ff4ec073c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:00.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.087+0000 7ff4f0f10640 1 -- 172.21.15.98:0/1592093306 >> v1:172.21.15.98:6789/0 conn(0x7ff4ec0b28e0 legacy=0x7ff4ec074c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:00.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.087+0000 7ff4f0f10640 1 -- 172.21.15.98:0/1592093306 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4ec0cf000 con 0x7ff4ec075380 2024-09-19T08:39:00.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.091+0000 7ff4f0f10640 1 --2- 172.21.15.98:0/1592093306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4ec075380 0x7ff4ec0cf520 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec069910 tx=0x7ff4e401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dda67f6e457dfdc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:00.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.091+0000 7ff4eaffd640 1 -- 172.21.15.98:0/1592093306 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e40a5840 con 0x7ff4ec075380 2024-09-19T08:39:00.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.091+0000 7ff4eaffd640 1 -- 172.21.15.98:0/1592093306 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4e40a59e0 con 0x7ff4ec075380 2024-09-19T08:39:00.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.091+0000 7ff4eaffd640 1 -- 172.21.15.98:0/1592093306 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e40a5db0 con 0x7ff4ec075380 2024-09-19T08:39:00.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.091+0000 7ff4f2973640 1 -- 172.21.15.98:0/1592093306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4ec075380 msgr2=0x7ff4ec0cf520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:00.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.091+0000 7ff4f2973640 1 --2- 172.21.15.98:0/1592093306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4ec075380 0x7ff4ec0cf520 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec069910 tx=0x7ff4e401c960 comp rx=0 tx=0).stop 2024-09-19T08:39:00.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.091+0000 7ff4f2973640 1 -- 172.21.15.98:0/1592093306 shutdown_connections 2024-09-19T08:39:00.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.091+0000 7ff4f2973640 1 --2- 172.21.15.98:0/1592093306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4ec075380 0x7ff4ec0cf520 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:00.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.091+0000 7ff4f2973640 1 --2- 172.21.15.98:0/1592093306 >> v2:172.21.15.139:3300/0 conn(0x7ff4ec15e750 0x7ff4ec073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:00.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.091+0000 7ff4f2973640 1 -- 172.21.15.98:0/1592093306 >> 172.21.15.98:0/1592093306 conn(0x7ff4ec073170 msgr2=0x7ff4ec073580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:00.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.091+0000 7ff4f2973640 1 -- 172.21.15.98:0/1592093306 shutdown_connections 2024-09-19T08:39:00.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.091+0000 7ff4f2973640 1 -- 172.21.15.98:0/1592093306 wait complete. 2024-09-19T08:39:00.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.091+0000 7ff4f2973640 1 Processor -- start 2024-09-19T08:39:00.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.091+0000 7ff4f2973640 1 -- start start 2024-09-19T08:39:00.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.091+0000 7ff4f2973640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4ec075380 0x7ff4ec1f96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:00.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.091+0000 7ff4f2973640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4ec0b28e0 0x7ff4ec0c63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:00.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.091+0000 7ff4f2973640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4ec15e750 0x7ff4ec0b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:00.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.091+0000 7ff4f2973640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ec0d0600 con 0x7ff4ec075380 2024-09-19T08:39:00.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.091+0000 7ff4f2973640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ec0d0480 con 0x7ff4ec0b28e0 2024-09-19T08:39:00.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.091+0000 7ff4f2973640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff4ec0d0780 con 0x7ff4ec15e750 2024-09-19T08:39:00.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.091+0000 7ff4ebfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4ec075380 0x7ff4ec1f96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:00.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.091+0000 7ff4ebfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4ec075380 0x7ff4ec1f96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35902/0 (socket says 172.21.15.98:35902) 2024-09-19T08:39:00.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.091+0000 7ff4ebfff640 1 -- 172.21.15.98:0/2529455849 learned_addr learned my addr 172.21.15.98:0/2529455849 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:00.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.091+0000 7ff4ebfff640 1 -- 172.21.15.98:0/2529455849 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4ec15e750 msgr2=0x7ff4ec0b0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:00.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.091+0000 7ff4f0f10640 1 --2- 172.21.15.98:0/2529455849 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4ec15e750 0x7ff4ec0b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:00.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.091+0000 7ff4eb7fe640 1 --2- 172.21.15.98:0/2529455849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4ec0b28e0 0x7ff4ec0c63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:00.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.091+0000 7ff4ebfff640 1 --2- 172.21.15.98:0/2529455849 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4ec15e750 0x7ff4ec0b0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:00.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.091+0000 7ff4ebfff640 1 -- 172.21.15.98:0/2529455849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4ec0b28e0 msgr2=0x7ff4ec0c63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:00.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.091+0000 7ff4ebfff640 1 --2- 172.21.15.98:0/2529455849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4ec0b28e0 0x7ff4ec0c63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:00.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.091+0000 7ff4ebfff640 1 -- 172.21.15.98:0/2529455849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4ec15c050 con 0x7ff4ec075380 2024-09-19T08:39:00.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.091+0000 7ff4eb7fe640 1 --2- 172.21.15.98:0/2529455849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4ec0b28e0 0x7ff4ec0c63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:00.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.095+0000 7ff4ebfff640 1 --2- 172.21.15.98:0/2529455849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4ec075380 0x7ff4ec1f96c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff4d400d5f0 tx=0x7ff4d400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:00.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.095+0000 7ff4e8ff9640 1 -- 172.21.15.98:0/2529455849 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d40125f0 con 0x7ff4ec075380 2024-09-19T08:39:00.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.095+0000 7ff4e8ff9640 1 -- 172.21.15.98:0/2529455849 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4d4019070 con 0x7ff4ec075380 2024-09-19T08:39:00.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.095+0000 7ff4f2973640 1 -- 172.21.15.98:0/2529455849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4ec15c330 con 0x7ff4ec075380 2024-09-19T08:39:00.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.095+0000 7ff4e8ff9640 1 -- 172.21.15.98:0/2529455849 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d40129b0 con 0x7ff4ec075380 2024-09-19T08:39:00.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.095+0000 7ff4f2973640 1 -- 172.21.15.98:0/2529455849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4ec09eed0 con 0x7ff4ec075380 2024-09-19T08:39:00.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.095+0000 7ff4f2973640 1 -- 172.21.15.98:0/2529455849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4ec09e830 con 0x7ff4ec075380 2024-09-19T08:39:00.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.095+0000 7ff4e8ff9640 1 -- 172.21.15.98:0/2529455849 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff4d4012b50 con 0x7ff4ec075380 2024-09-19T08:39:00.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.095+0000 7ff4e8ff9640 1 --2- 172.21.15.98:0/2529455849 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff4cc0678f0 0x7ff4cc069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:00.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.095+0000 7ff4e8ff9640 1 -- 172.21.15.98:0/2529455849 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7ff4d4085c80 con 0x7ff4ec075380 2024-09-19T08:39:00.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.095+0000 7ff4eb7fe640 1 --2- 172.21.15.98:0/2529455849 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff4cc0678f0 0x7ff4cc069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:00.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.095+0000 7ff4eb7fe640 1 --2- 172.21.15.98:0/2529455849 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff4cc0678f0 0x7ff4cc069db0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc007fd0 tx=0x7ff4dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:00.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.099+0000 7ff4e8ff9640 1 -- 172.21.15.98:0/2529455849 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff4d4056ba0 con 0x7ff4ec075380 2024-09-19T08:39:00.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.319+0000 7ff4f2973640 1 -- 172.21.15.98:0/2529455849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7ff4ec15c500 con 0x7ff4ec075380 2024-09-19T08:39:00.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.323+0000 7ff4e8ff9640 1 -- 172.21.15.98:0/2529455849 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7ff4d40564a0 con 0x7ff4ec075380 2024-09-19T08:39:00.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.323+0000 7ff4f2973640 1 -- 172.21.15.98:0/2529455849 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff4cc0678f0 msgr2=0x7ff4cc069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:00.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.323+0000 7ff4f2973640 1 --2- 172.21.15.98:0/2529455849 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff4cc0678f0 0x7ff4cc069db0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc007fd0 tx=0x7ff4dc009040 comp rx=0 tx=0).stop 2024-09-19T08:39:00.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.323+0000 7ff4f2973640 1 -- 172.21.15.98:0/2529455849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4ec075380 msgr2=0x7ff4ec1f96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:00.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.323+0000 7ff4f2973640 1 --2- 172.21.15.98:0/2529455849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4ec075380 0x7ff4ec1f96c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff4d400d5f0 tx=0x7ff4d400dab0 comp rx=0 tx=0).stop 2024-09-19T08:39:00.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.323+0000 7ff4f2973640 1 -- 172.21.15.98:0/2529455849 shutdown_connections 2024-09-19T08:39:00.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.323+0000 7ff4f2973640 1 --2- 172.21.15.98:0/2529455849 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff4cc0678f0 0x7ff4cc069db0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:00.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.323+0000 7ff4f2973640 1 --2- 172.21.15.98:0/2529455849 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff4ec15e750 0x7ff4ec0b0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:00.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.323+0000 7ff4f2973640 1 --2- 172.21.15.98:0/2529455849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff4ec0b28e0 0x7ff4ec0c63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:00.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.323+0000 7ff4f2973640 1 --2- 172.21.15.98:0/2529455849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff4ec075380 0x7ff4ec1f96c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:00.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.323+0000 7ff4f2973640 1 -- 172.21.15.98:0/2529455849 >> 172.21.15.98:0/2529455849 conn(0x7ff4ec073170 msgr2=0x7ff4ec0b6c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:00.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.323+0000 7ff4f2973640 1 -- 172.21.15.98:0/2529455849 shutdown_connections 2024-09-19T08:39:00.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.323+0000 7ff4f2973640 1 -- 172.21.15.98:0/2529455849 wait complete. 2024-09-19T08:39:00.343 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-19T08:39:00.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.459+0000 7faacdcee640 1 Processor -- start 2024-09-19T08:39:00.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.459+0000 7faacdcee640 1 -- start start 2024-09-19T08:39:00.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.459+0000 7faacdcee640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7faac806d880 0x7faac80b3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:00.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.459+0000 7faacdcee640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faac8141110 0x7faac80b3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:00.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.459+0000 7faacdcee640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faac815ea10 con 0x7faac8140d30 2024-09-19T08:39:00.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.459+0000 7faacdcee640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7faac815e7c0 con 0x7faac806d880 2024-09-19T08:39:00.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.459+0000 7faacdcee640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7faac815ecf0 con 0x7faac8141110 2024-09-19T08:39:00.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.459+0000 7faac7fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faac8141110 0x7faac80b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:00.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.459+0000 7faac6ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7faac806d880 0x7faac80b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:00.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.459+0000 7faac7fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faac8141110 0x7faac80b3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:57456/0 (socket says 172.21.15.98:57456) 2024-09-19T08:39:00.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.459+0000 7faac7fff640 1 -- 172.21.15.98:0/3862568023 learned_addr learned my addr 172.21.15.98:0/3862568023 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:00.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.459+0000 7faac6ffd640 1 -- 172.21.15.98:0/3862568023 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faac8141110 msgr2=0x7faac80b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:00.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.459+0000 7faac6ffd640 1 --2- 172.21.15.98:0/3862568023 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faac8141110 0x7faac80b3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:00.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.459+0000 7faac6ffd640 1 -- 172.21.15.98:0/3862568023 >> v1:172.21.15.139:6789/0 conn(0x7faac8140d30 legacy=0x7faac806d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:00.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.459+0000 7faac6ffd640 1 -- 172.21.15.98:0/3862568023 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac815efd0 con 0x7faac806d880 2024-09-19T08:39:00.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.459+0000 7faac7fff640 1 --2- 172.21.15.98:0/3862568023 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faac8141110 0x7faac80b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:39:00.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.463+0000 7faac6ffd640 1 --2- 172.21.15.98:0/3862568023 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faac806d880 0x7faac80b3310 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faabc0097c0 tx=0x7faabc01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=938a48d4e9406f74 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:00.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.463+0000 7faac67fc640 1 -- 172.21.15.98:0/3862568023 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faabc0aa620 con 0x7faac806d880 2024-09-19T08:39:00.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.463+0000 7faac67fc640 1 -- 172.21.15.98:0/3862568023 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faabc0af070 con 0x7faac806d880 2024-09-19T08:39:00.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.463+0000 7faac67fc640 1 -- 172.21.15.98:0/3862568023 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faabc0aa9e0 con 0x7faac806d880 2024-09-19T08:39:00.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.463+0000 7faacdcee640 1 -- 172.21.15.98:0/3862568023 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faac806d880 msgr2=0x7faac80b3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:00.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.463+0000 7faacdcee640 1 --2- 172.21.15.98:0/3862568023 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faac806d880 0x7faac80b3310 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faabc0097c0 tx=0x7faabc01c4a0 comp rx=0 tx=0).stop 2024-09-19T08:39:00.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.463+0000 7faacdcee640 1 -- 172.21.15.98:0/3862568023 shutdown_connections 2024-09-19T08:39:00.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.463+0000 7faacdcee640 1 --2- 172.21.15.98:0/3862568023 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faac8141110 0x7faac80b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:00.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.463+0000 7faacdcee640 1 --2- 172.21.15.98:0/3862568023 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faac806d880 0x7faac80b3310 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:00.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.463+0000 7faacdcee640 1 -- 172.21.15.98:0/3862568023 >> 172.21.15.98:0/3862568023 conn(0x7faac80b2010 msgr2=0x7faac80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:00.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.463+0000 7faacdcee640 1 -- 172.21.15.98:0/3862568023 shutdown_connections 2024-09-19T08:39:00.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.463+0000 7faacdcee640 1 -- 172.21.15.98:0/3862568023 wait complete. 2024-09-19T08:39:00.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.463+0000 7faacdcee640 1 Processor -- start 2024-09-19T08:39:00.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.463+0000 7faacdcee640 1 -- start start 2024-09-19T08:39:00.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.467+0000 7faacdcee640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faac806d880 0x7faac8073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:00.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.467+0000 7faacdcee640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faac8140d30 0x7faac8163d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:00.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.467+0000 7faacdcee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faac8141110 0x7faac81612d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:00.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.467+0000 7faacdcee640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faac815bdd0 con 0x7faac806d880 2024-09-19T08:39:00.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.467+0000 7faacdcee640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faac815bc50 con 0x7faac8141110 2024-09-19T08:39:00.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.467+0000 7faacdcee640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7faac815bf50 con 0x7faac8140d30 2024-09-19T08:39:00.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.467+0000 7faac77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faac806d880 0x7faac8073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:00.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.467+0000 7faac77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faac806d880 0x7faac8073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35904/0 (socket says 172.21.15.98:35904) 2024-09-19T08:39:00.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.467+0000 7faac77fe640 1 -- 172.21.15.98:0/397429822 learned_addr learned my addr 172.21.15.98:0/397429822 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:00.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.467+0000 7faac6ffd640 1 --2- 172.21.15.98:0/397429822 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faac8140d30 0x7faac8163d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:00.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.467+0000 7faac7fff640 1 --2- 172.21.15.98:0/397429822 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faac8141110 0x7faac81612d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:00.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.467+0000 7faac77fe640 1 -- 172.21.15.98:0/397429822 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faac8140d30 msgr2=0x7faac8163d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:00.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.467+0000 7faac77fe640 1 --2- 172.21.15.98:0/397429822 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faac8140d30 0x7faac8163d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:00.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.467+0000 7faac77fe640 1 -- 172.21.15.98:0/397429822 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faac8141110 msgr2=0x7faac81612d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:00.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.467+0000 7faac77fe640 1 --2- 172.21.15.98:0/397429822 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faac8141110 0x7faac81612d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:00.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.467+0000 7faac77fe640 1 -- 172.21.15.98:0/397429822 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac8160660 con 0x7faac806d880 2024-09-19T08:39:00.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.467+0000 7faac6ffd640 1 --2- 172.21.15.98:0/397429822 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faac8140d30 0x7faac8163d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:00.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.467+0000 7faac7fff640 1 --2- 172.21.15.98:0/397429822 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faac8141110 0x7faac81612d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:00.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.467+0000 7faac77fe640 1 --2- 172.21.15.98:0/397429822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faac806d880 0x7faac8073140 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7faab800c450 tx=0x7faab800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:00.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.467+0000 7faaaffff640 1 -- 172.21.15.98:0/397429822 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab80052d0 con 0x7faac806d880 2024-09-19T08:39:00.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.467+0000 7faaaffff640 1 -- 172.21.15.98:0/397429822 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faab8005490 con 0x7faac806d880 2024-09-19T08:39:00.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.467+0000 7faacdcee640 1 -- 172.21.15.98:0/397429822 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faac8163250 con 0x7faac806d880 2024-09-19T08:39:00.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.467+0000 7faaaffff640 1 -- 172.21.15.98:0/397429822 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab8012850 con 0x7faac806d880 2024-09-19T08:39:00.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.467+0000 7faacdcee640 1 -- 172.21.15.98:0/397429822 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faac8160940 con 0x7faac806d880 2024-09-19T08:39:00.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.467+0000 7faacdcee640 1 -- 172.21.15.98:0/397429822 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faac81639e0 con 0x7faac806d880 2024-09-19T08:39:00.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.467+0000 7faaaffff640 1 -- 172.21.15.98:0/397429822 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7faab80129f0 con 0x7faac806d880 2024-09-19T08:39:00.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.467+0000 7faaaffff640 1 --2- 172.21.15.98:0/397429822 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7faab40678c0 0x7faab4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:00.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.467+0000 7faac6ffd640 1 --2- 172.21.15.98:0/397429822 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7faab40678c0 0x7faab4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:00.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.467+0000 7faaaffff640 1 -- 172.21.15.98:0/397429822 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7faab8085570 con 0x7faac806d880 2024-09-19T08:39:00.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.467+0000 7faac6ffd640 1 --2- 172.21.15.98:0/397429822 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7faab40678c0 0x7faab4069d80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7faabc0ae000 tx=0x7faabc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:00.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.471+0000 7faaaffff640 1 -- 172.21.15.98:0/397429822 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7faab8056400 con 0x7faac806d880 2024-09-19T08:39:00.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.675+0000 7faacdcee640 1 -- 172.21.15.98:0/397429822 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7faac806d0f0 con 0x7faac806d880 2024-09-19T08:39:00.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.679+0000 7faaaffff640 1 -- 172.21.15.98:0/397429822 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7faab8055d00 con 0x7faac806d880 2024-09-19T08:39:00.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.683+0000 7faacdcee640 1 -- 172.21.15.98:0/397429822 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7faab40678c0 msgr2=0x7faab4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:00.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.683+0000 7faacdcee640 1 --2- 172.21.15.98:0/397429822 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7faab40678c0 0x7faab4069d80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7faabc0ae000 tx=0x7faabc008040 comp rx=0 tx=0).stop 2024-09-19T08:39:00.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.683+0000 7faacdcee640 1 -- 172.21.15.98:0/397429822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faac806d880 msgr2=0x7faac8073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:00.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.683+0000 7faacdcee640 1 --2- 172.21.15.98:0/397429822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faac806d880 0x7faac8073140 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7faab800c450 tx=0x7faab800c910 comp rx=0 tx=0).stop 2024-09-19T08:39:00.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.683+0000 7faacdcee640 1 -- 172.21.15.98:0/397429822 shutdown_connections 2024-09-19T08:39:00.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.683+0000 7faacdcee640 1 --2- 172.21.15.98:0/397429822 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7faab40678c0 0x7faab4069d80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:00.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.683+0000 7faacdcee640 1 --2- 172.21.15.98:0/397429822 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faac8141110 0x7faac81612d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:00.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.683+0000 7faacdcee640 1 --2- 172.21.15.98:0/397429822 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faac8140d30 0x7faac8163d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:00.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.683+0000 7faacdcee640 1 --2- 172.21.15.98:0/397429822 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faac806d880 0x7faac8073140 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:00.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.683+0000 7faacdcee640 1 -- 172.21.15.98:0/397429822 >> 172.21.15.98:0/397429822 conn(0x7faac80b2010 msgr2=0x7faac80cede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:00.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.683+0000 7faacdcee640 1 -- 172.21.15.98:0/397429822 shutdown_connections 2024-09-19T08:39:00.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.683+0000 7faacdcee640 1 -- 172.21.15.98:0/397429822 wait complete. 2024-09-19T08:39:00.704 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-19T08:39:00.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.823+0000 7f9a034af640 1 Processor -- start 2024-09-19T08:39:00.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.823+0000 7f9a034af640 1 -- start start 2024-09-19T08:39:00.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.823+0000 7f9a034af640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f99fc163af0 0x7f99fc0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:00.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.823+0000 7f9a034af640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99fc0cf240 0x7f99fc0afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:00.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.823+0000 7f9a034af640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99fc0b0030 con 0x7f99fc0af710 2024-09-19T08:39:00.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.823+0000 7f9a034af640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f99fc0cf670 con 0x7f99fc163af0 2024-09-19T08:39:00.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.823+0000 7f9a034af640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f99fc0cca20 con 0x7f99fc0cf240 2024-09-19T08:39:00.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.827+0000 7f9a0124b640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f99fc0af710 0x7f99fc0cfde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:55404/0 (socket says 172.21.15.98:55404) 2024-09-19T08:39:00.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.827+0000 7f9a0124b640 1 -- 172.21.15.98:0/1146528200 learned_addr learned my addr 172.21.15.98:0/1146528200 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:00.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.827+0000 7f9a00a4a640 1 --2- 172.21.15.98:0/1146528200 >> v2:172.21.15.139:3300/0 conn(0x7f99fc163af0 0x7f99fc0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:00.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.827+0000 7f9a01a4c640 1 --2- 172.21.15.98:0/1146528200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99fc0cf240 0x7f99fc0afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:00.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.827+0000 7f99f3fff640 1 -- 172.21.15.98:0/1146528200 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4062667038 0 0) 0x7f99fc0b0030 con 0x7f99fc0af710 2024-09-19T08:39:00.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.827+0000 7f99f3fff640 1 -- 172.21.15.98:0/1146528200 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99e8003530 con 0x7f99fc0af710 2024-09-19T08:39:00.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.827+0000 7f99f3fff640 1 -- 172.21.15.98:0/1146528200 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2156331337 0 0) 0x7f99e8003530 con 0x7f99fc0af710 2024-09-19T08:39:00.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.827+0000 7f99f3fff640 1 -- 172.21.15.98:0/1146528200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99fc0cf240 msgr2=0x7f99fc0afaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:00.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.827+0000 7f99f3fff640 1 --2- 172.21.15.98:0/1146528200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99fc0cf240 0x7f99fc0afaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:00.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.827+0000 7f99f3fff640 1 -- 172.21.15.98:0/1146528200 >> v2:172.21.15.139:3300/0 conn(0x7f99fc163af0 msgr2=0x7f99fc0cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:00.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.827+0000 7f99f3fff640 1 --2- 172.21.15.98:0/1146528200 >> v2:172.21.15.139:3300/0 conn(0x7f99fc163af0 0x7f99fc0cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:00.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.827+0000 7f99f3fff640 1 -- 172.21.15.98:0/1146528200 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99fc0d05d0 con 0x7f99fc0af710 2024-09-19T08:39:00.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.827+0000 7f99f3fff640 1 -- 172.21.15.98:0/1146528200 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f99f80016a0 con 0x7f99fc0af710 2024-09-19T08:39:00.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.827+0000 7f99f3fff640 1 -- 172.21.15.98:0/1146528200 >> v1:172.21.15.98:6789/0 conn(0x7f99fc0af710 legacy=0x7f99fc0cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:00.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.827+0000 7f99f3fff640 1 --2- 172.21.15.98:0/1146528200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99e80039c0 0x7f99e8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:00.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.827+0000 7f99f3fff640 1 --2- 172.21.15.98:0/1146528200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99e8004810 0x7f99e8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:00.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.827+0000 7f99f3fff640 1 --2- 172.21.15.98:0/1146528200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99fc0cf240 0x7f99e80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:00.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.827+0000 7f99f3fff640 1 -- 172.21.15.98:0/1146528200 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc0cf670 con 0x7f99e8004810 2024-09-19T08:39:00.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.827+0000 7f99f3fff640 1 -- 172.21.15.98:0/1146528200 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc0cca20 con 0x7f99e80039c0 2024-09-19T08:39:00.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.827+0000 7f99f3fff640 1 -- 172.21.15.98:0/1146528200 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f99e800b9d0 con 0x7f99fc0cf240 2024-09-19T08:39:00.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.827+0000 7f9a00a4a640 1 --2- 172.21.15.98:0/1146528200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99e8004810 0x7f99e8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:00.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.827+0000 7f9a0124b640 1 --2- 172.21.15.98:0/1146528200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99e80039c0 0x7f99e8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:00.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.827+0000 7f9a01a4c640 1 --2- 172.21.15.98:0/1146528200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99fc0cf240 0x7f99e80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:00.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.827+0000 7f9a00a4a640 1 -- 172.21.15.98:0/1146528200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99fc0cf240 msgr2=0x7f99e80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:00.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.827+0000 7f9a00a4a640 1 --2- 172.21.15.98:0/1146528200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99fc0cf240 0x7f99e80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:00.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.827+0000 7f9a00a4a640 1 -- 172.21.15.98:0/1146528200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99e80039c0 msgr2=0x7f99e8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:00.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.827+0000 7f9a00a4a640 1 --2- 172.21.15.98:0/1146528200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99e80039c0 0x7f99e8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:00.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.827+0000 7f9a00a4a640 1 -- 172.21.15.98:0/1146528200 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e8003530 con 0x7f99e8004810 2024-09-19T08:39:00.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.827+0000 7f9a01a4c640 1 --2- 172.21.15.98:0/1146528200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99fc0cf240 0x7f99e80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:00.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.827+0000 7f9a00a4a640 1 --2- 172.21.15.98:0/1146528200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99e8004810 0x7f99e8004c90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009ab0 tx=0x7f99ec01ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0848a86d948dc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:00.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.827+0000 7f99f3fff640 1 -- 172.21.15.98:0/1146528200 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec0096b0 con 0x7f99e8004810 2024-09-19T08:39:00.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.827+0000 7f99f3fff640 1 -- 172.21.15.98:0/1146528200 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99ec0a64a0 con 0x7f99e8004810 2024-09-19T08:39:00.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.827+0000 7f99f3fff640 1 -- 172.21.15.98:0/1146528200 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec0a6850 con 0x7f99e8004810 2024-09-19T08:39:00.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.831+0000 7f9a034af640 1 -- 172.21.15.98:0/1146528200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99e8004810 msgr2=0x7f99e8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:00.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.831+0000 7f9a034af640 1 --2- 172.21.15.98:0/1146528200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99e8004810 0x7f99e8004c90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009ab0 tx=0x7f99ec01ca90 comp rx=0 tx=0).stop 2024-09-19T08:39:00.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.831+0000 7f9a01a4c640 1 -- 172.21.15.98:0/1146528200 reap_dead start 2024-09-19T08:39:00.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.831+0000 7f9a034af640 1 -- 172.21.15.98:0/1146528200 shutdown_connections 2024-09-19T08:39:00.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.831+0000 7f9a034af640 1 -- 172.21.15.98:0/1146528200 >> 172.21.15.98:0/1146528200 conn(0x7f99fc0d0980 msgr2=0x7f99fc0d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:00.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.831+0000 7f9a034af640 1 -- 172.21.15.98:0/1146528200 shutdown_connections 2024-09-19T08:39:00.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.831+0000 7f9a034af640 1 -- 172.21.15.98:0/1146528200 wait complete. 2024-09-19T08:39:00.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.831+0000 7f9a034af640 1 Processor -- start 2024-09-19T08:39:00.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.831+0000 7f9a034af640 1 -- start start 2024-09-19T08:39:00.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.831+0000 7f9a034af640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99e8004810 0x7f99fc0b4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:00.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.831+0000 7f9a034af640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99fc06ea40 0x7f99fc15de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:00.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.831+0000 7f9a034af640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99fc06eec0 0x7f99fc15e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:00.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.831+0000 7f9a034af640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc0cea40 con 0x7f99e8004810 2024-09-19T08:39:00.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.831+0000 7f9a034af640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc0cedb0 con 0x7f99fc06ea40 2024-09-19T08:39:00.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.831+0000 7f9a034af640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f99fc0afaf0 con 0x7f99fc06eec0 2024-09-19T08:39:00.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.831+0000 7f9a0124b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99e8004810 0x7f99fc0b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:00.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.831+0000 7f9a0124b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99e8004810 0x7f99fc0b4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35926/0 (socket says 172.21.15.98:35926) 2024-09-19T08:39:00.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.831+0000 7f9a0124b640 1 -- 172.21.15.98:0/2552638406 learned_addr learned my addr 172.21.15.98:0/2552638406 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:00.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.831+0000 7f9a00a4a640 1 --2- 172.21.15.98:0/2552638406 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99fc06ea40 0x7f99fc15de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:00.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.831+0000 7f9a01a4c640 1 --2- 172.21.15.98:0/2552638406 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99fc06eec0 0x7f99fc15e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:00.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.831+0000 7f9a0124b640 1 -- 172.21.15.98:0/2552638406 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99fc06eec0 msgr2=0x7f99fc15e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:00.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.831+0000 7f9a0124b640 1 --2- 172.21.15.98:0/2552638406 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99fc06eec0 0x7f99fc15e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:00.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.831+0000 7f9a0124b640 1 -- 172.21.15.98:0/2552638406 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99fc06ea40 msgr2=0x7f99fc15de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:00.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.831+0000 7f9a0124b640 1 --2- 172.21.15.98:0/2552638406 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99fc06ea40 0x7f99fc15de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:00.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.831+0000 7f9a0124b640 1 -- 172.21.15.98:0/2552638406 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99fc15c300 con 0x7f99e8004810 2024-09-19T08:39:00.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.831+0000 7f9a01a4c640 1 --2- 172.21.15.98:0/2552638406 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99fc06eec0 0x7f99fc15e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:00.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.831+0000 7f9a0124b640 1 --2- 172.21.15.98:0/2552638406 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99e8004810 0x7f99fc0b4260 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f99f8011800 tx=0x7f99f8011cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:00.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.831+0000 7f99f1ffb640 1 -- 172.21.15.98:0/2552638406 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f8010040 con 0x7f99e8004810 2024-09-19T08:39:00.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.831+0000 7f99f1ffb640 1 -- 172.21.15.98:0/2552638406 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99f801a070 con 0x7f99e8004810 2024-09-19T08:39:00.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.831+0000 7f9a034af640 1 -- 172.21.15.98:0/2552638406 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99fc0b5780 con 0x7f99e8004810 2024-09-19T08:39:00.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.831+0000 7f99f1ffb640 1 -- 172.21.15.98:0/2552638406 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f800fde0 con 0x7f99e8004810 2024-09-19T08:39:00.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.831+0000 7f9a034af640 1 -- 172.21.15.98:0/2552638406 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99fc0b5d40 con 0x7f99e8004810 2024-09-19T08:39:00.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.835+0000 7f9a034af640 1 -- 172.21.15.98:0/2552638406 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99fc0afed0 con 0x7f99e8004810 2024-09-19T08:39:00.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.835+0000 7f99f1ffb640 1 -- 172.21.15.98:0/2552638406 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f99f801f460 con 0x7f99e8004810 2024-09-19T08:39:00.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.835+0000 7f99f1ffb640 1 --2- 172.21.15.98:0/2552638406 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f99e00678f0 0x7f99e0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:00.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.835+0000 7f99f1ffb640 1 -- 172.21.15.98:0/2552638406 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f99f8088a70 con 0x7f99e8004810 2024-09-19T08:39:00.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.835+0000 7f9a00a4a640 1 --2- 172.21.15.98:0/2552638406 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f99e00678f0 0x7f99e0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:00.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.835+0000 7f9a00a4a640 1 --2- 172.21.15.98:0/2552638406 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f99e00678f0 0x7f99e0069db0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f99ec0ad000 tx=0x7f99ec0b2040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:00.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:00.839+0000 7f99f1ffb640 1 -- 172.21.15.98:0/2552638406 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f99f8059900 con 0x7f99e8004810 2024-09-19T08:39:01.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.059+0000 7f9a034af640 1 -- 172.21.15.98:0/2552638406 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f99fc0b1160 con 0x7f99e8004810 2024-09-19T08:39:01.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.063+0000 7f99f1ffb640 1 -- 172.21.15.98:0/2552638406 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f99f8059200 con 0x7f99e8004810 2024-09-19T08:39:01.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.067+0000 7f9a034af640 1 -- 172.21.15.98:0/2552638406 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f99e00678f0 msgr2=0x7f99e0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:01.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.067+0000 7f9a034af640 1 --2- 172.21.15.98:0/2552638406 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f99e00678f0 0x7f99e0069db0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f99ec0ad000 tx=0x7f99ec0b2040 comp rx=0 tx=0).stop 2024-09-19T08:39:01.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.067+0000 7f9a034af640 1 -- 172.21.15.98:0/2552638406 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99e8004810 msgr2=0x7f99fc0b4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:01.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.067+0000 7f9a034af640 1 --2- 172.21.15.98:0/2552638406 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99e8004810 0x7f99fc0b4260 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f99f8011800 tx=0x7f99f8011cc0 comp rx=0 tx=0).stop 2024-09-19T08:39:01.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.067+0000 7f9a034af640 1 -- 172.21.15.98:0/2552638406 shutdown_connections 2024-09-19T08:39:01.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.067+0000 7f9a034af640 1 --2- 172.21.15.98:0/2552638406 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f99e00678f0 0x7f99e0069db0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:01.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.067+0000 7f9a034af640 1 --2- 172.21.15.98:0/2552638406 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99fc06eec0 0x7f99fc15e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:01.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.067+0000 7f9a034af640 1 --2- 172.21.15.98:0/2552638406 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99fc06ea40 0x7f99fc15de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:01.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.067+0000 7f9a034af640 1 --2- 172.21.15.98:0/2552638406 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99e8004810 0x7f99fc0b4260 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:01.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.067+0000 7f9a034af640 1 -- 172.21.15.98:0/2552638406 >> 172.21.15.98:0/2552638406 conn(0x7f99fc0d0980 msgr2=0x7f99fc0ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:01.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.067+0000 7f9a034af640 1 -- 172.21.15.98:0/2552638406 shutdown_connections 2024-09-19T08:39:01.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.067+0000 7f9a034af640 1 -- 172.21.15.98:0/2552638406 wait complete. 2024-09-19T08:39:01.087 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-19T08:39:01.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.195+0000 7fc51902d640 1 Processor -- start 2024-09-19T08:39:01.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.195+0000 7fc51902d640 1 -- start start 2024-09-19T08:39:01.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.195+0000 7fc51902d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc5140733b0 0x7fc5140cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:01.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.195+0000 7fc51902d640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc51406d310 con 0x7fc51415c550 2024-09-19T08:39:01.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.195+0000 7fc51902d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc51406d5f0 con 0x7fc51406c3a0 2024-09-19T08:39:01.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.195+0000 7fc51902d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fc51406d8d0 con 0x7fc5140733b0 2024-09-19T08:39:01.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.195+0000 7fc512d9d640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fc51415c550 0x7fc51406bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:55406/0 (socket says 172.21.15.98:55406) 2024-09-19T08:39:01.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.195+0000 7fc512d9d640 1 -- 172.21.15.98:0/3379983175 learned_addr learned my addr 172.21.15.98:0/3379983175 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:01.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.195+0000 7fc51359e640 1 --2- 172.21.15.98:0/3379983175 >> v2:172.21.15.139:3300/0 conn(0x7fc5140733b0 0x7fc5140cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:01.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.195+0000 7fc511d9b640 1 -- 172.21.15.98:0/3379983175 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1620412472 0 0) 0x7fc51406d310 con 0x7fc51415c550 2024-09-19T08:39:01.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.195+0000 7fc511d9b640 1 -- 172.21.15.98:0/3379983175 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc4fc003530 con 0x7fc51415c550 2024-09-19T08:39:01.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.195+0000 7fc511d9b640 1 -- 172.21.15.98:0/3379983175 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1194313644 0 0) 0x7fc4fc003530 con 0x7fc51415c550 2024-09-19T08:39:01.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.195+0000 7fc511d9b640 1 -- 172.21.15.98:0/3379983175 >> v2:172.21.15.139:3300/0 conn(0x7fc5140733b0 msgr2=0x7fc5140cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:01.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.195+0000 7fc511d9b640 1 --2- 172.21.15.98:0/3379983175 >> v2:172.21.15.139:3300/0 conn(0x7fc5140733b0 0x7fc5140cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:01.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.195+0000 7fc511d9b640 1 -- 172.21.15.98:0/3379983175 >> v1:172.21.15.139:6789/0 conn(0x7fc51406c3a0 legacy=0x7fc5140cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:01.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.195+0000 7fc511d9b640 1 -- 172.21.15.98:0/3379983175 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc51406dbb0 con 0x7fc51415c550 2024-09-19T08:39:01.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.195+0000 7fc511d9b640 1 -- 172.21.15.98:0/3379983175 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fc504001680 con 0x7fc51415c550 2024-09-19T08:39:01.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.195+0000 7fc511d9b640 1 -- 172.21.15.98:0/3379983175 >> v1:172.21.15.98:6789/0 conn(0x7fc51415c550 legacy=0x7fc51406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:01.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.195+0000 7fc511d9b640 1 --2- 172.21.15.98:0/3379983175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4fc0039c0 0x7fc4fc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:01.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.195+0000 7fc511d9b640 1 --2- 172.21.15.98:0/3379983175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4fc004810 0x7fc4fc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:01.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.195+0000 7fc511d9b640 1 --2- 172.21.15.98:0/3379983175 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4fc0051d0 0x7fc4fc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:01.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.195+0000 7fc511d9b640 1 -- 172.21.15.98:0/3379983175 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc51406d8d0 con 0x7fc4fc0039c0 2024-09-19T08:39:01.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.195+0000 7fc511d9b640 1 -- 172.21.15.98:0/3379983175 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc4fc00c070 con 0x7fc4fc004810 2024-09-19T08:39:01.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.195+0000 7fc511d9b640 1 -- 172.21.15.98:0/3379983175 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc4fc00c270 con 0x7fc4fc0051d0 2024-09-19T08:39:01.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.195+0000 7fc512d9d640 1 --2- 172.21.15.98:0/3379983175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4fc0039c0 0x7fc4fc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:01.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.195+0000 7fc51259c640 1 --2- 172.21.15.98:0/3379983175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4fc004810 0x7fc4fc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:01.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.195+0000 7fc512d9d640 1 -- 172.21.15.98:0/3379983175 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4fc0051d0 msgr2=0x7fc4fc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:01.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.195+0000 7fc512d9d640 1 --2- 172.21.15.98:0/3379983175 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4fc0051d0 0x7fc4fc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:01.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.195+0000 7fc512d9d640 1 -- 172.21.15.98:0/3379983175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4fc004810 msgr2=0x7fc4fc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:01.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.195+0000 7fc512d9d640 1 --2- 172.21.15.98:0/3379983175 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4fc004810 0x7fc4fc004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:01.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.195+0000 7fc512d9d640 1 -- 172.21.15.98:0/3379983175 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4fc003530 con 0x7fc4fc0039c0 2024-09-19T08:39:01.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.195+0000 7fc51359e640 1 -- 172.21.15.98:0/3379983175 reap_dead start 2024-09-19T08:39:01.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.199+0000 7fc512d9d640 1 --2- 172.21.15.98:0/3379983175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4fc0039c0 0x7fc4fc003de0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc50400c1c0 tx=0x7fc50401cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eab3bdcfd23d32e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:01.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.199+0000 7fc511d9b640 1 -- 172.21.15.98:0/3379983175 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc504007d00 con 0x7fc4fc0039c0 2024-09-19T08:39:01.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.199+0000 7fc511d9b640 1 -- 172.21.15.98:0/3379983175 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5040a9020 con 0x7fc4fc0039c0 2024-09-19T08:39:01.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.199+0000 7fc511d9b640 1 -- 172.21.15.98:0/3379983175 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc504005ae0 con 0x7fc4fc0039c0 2024-09-19T08:39:01.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.199+0000 7fc51902d640 1 -- 172.21.15.98:0/3379983175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4fc0039c0 msgr2=0x7fc4fc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:01.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.199+0000 7fc51902d640 1 --2- 172.21.15.98:0/3379983175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4fc0039c0 0x7fc4fc003de0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc50400c1c0 tx=0x7fc50401cc20 comp rx=0 tx=0).stop 2024-09-19T08:39:01.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.199+0000 7fc51902d640 1 -- 172.21.15.98:0/3379983175 shutdown_connections 2024-09-19T08:39:01.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.199+0000 7fc51902d640 1 --2- 172.21.15.98:0/3379983175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4fc0039c0 0x7fc4fc003de0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:01.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.199+0000 7fc51902d640 1 -- 172.21.15.98:0/3379983175 >> 172.21.15.98:0/3379983175 conn(0x7fc5140b2010 msgr2=0x7fc5140b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:01.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.199+0000 7fc51902d640 1 -- 172.21.15.98:0/3379983175 shutdown_connections 2024-09-19T08:39:01.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.199+0000 7fc51902d640 1 -- 172.21.15.98:0/3379983175 wait complete. 2024-09-19T08:39:01.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.199+0000 7fc51902d640 1 Processor -- start 2024-09-19T08:39:01.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.199+0000 7fc51902d640 1 -- start start 2024-09-19T08:39:01.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.199+0000 7fc51902d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4fc0039c0 0x7fc514151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:01.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.199+0000 7fc51902d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc514142470 0x7fc51414e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:01.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.199+0000 7fc51902d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc51414fe50 0x7fc514150300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:01.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.199+0000 7fc51902d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc514072ae0 con 0x7fc4fc0039c0 2024-09-19T08:39:01.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.199+0000 7fc51902d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc51415cb40 con 0x7fc514142470 2024-09-19T08:39:01.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.199+0000 7fc51902d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc514072c60 con 0x7fc51414fe50 2024-09-19T08:39:01.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.199+0000 7fc512d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4fc0039c0 0x7fc514151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:01.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.199+0000 7fc512d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4fc0039c0 0x7fc514151ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35948/0 (socket says 172.21.15.98:35948) 2024-09-19T08:39:01.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.199+0000 7fc512d9d640 1 -- 172.21.15.98:0/1868270452 learned_addr learned my addr 172.21.15.98:0/1868270452 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:01.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.203+0000 7fc512d9d640 1 -- 172.21.15.98:0/1868270452 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc51414fe50 msgr2=0x7fc514150300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:01.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.203+0000 7fc512d9d640 1 --2- 172.21.15.98:0/1868270452 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc51414fe50 0x7fc514150300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:01.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.203+0000 7fc512d9d640 1 -- 172.21.15.98:0/1868270452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc514142470 msgr2=0x7fc51414e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:01.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.203+0000 7fc512d9d640 1 --2- 172.21.15.98:0/1868270452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc514142470 0x7fc51414e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:01.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.203+0000 7fc512d9d640 1 -- 172.21.15.98:0/1868270452 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5141531e0 con 0x7fc4fc0039c0 2024-09-19T08:39:01.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.203+0000 7fc512d9d640 1 --2- 172.21.15.98:0/1868270452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4fc0039c0 0x7fc514151ab0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc5040985c0 tx=0x7fc4fc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:01.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.203+0000 7fc4fb7fe640 1 -- 172.21.15.98:0/1868270452 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc504007d00 con 0x7fc4fc0039c0 2024-09-19T08:39:01.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.203+0000 7fc4fb7fe640 1 -- 172.21.15.98:0/1868270452 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5040049f0 con 0x7fc4fc0039c0 2024-09-19T08:39:01.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.203+0000 7fc51902d640 1 -- 172.21.15.98:0/1868270452 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc51414edc0 con 0x7fc4fc0039c0 2024-09-19T08:39:01.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.203+0000 7fc4fb7fe640 1 -- 172.21.15.98:0/1868270452 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc504001c40 con 0x7fc4fc0039c0 2024-09-19T08:39:01.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.203+0000 7fc51902d640 1 -- 172.21.15.98:0/1868270452 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc514152420 con 0x7fc4fc0039c0 2024-09-19T08:39:01.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.203+0000 7fc51902d640 1 -- 172.21.15.98:0/1868270452 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc514072ff0 con 0x7fc4fc0039c0 2024-09-19T08:39:01.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.203+0000 7fc4fb7fe640 1 -- 172.21.15.98:0/1868270452 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc5040bd060 con 0x7fc4fc0039c0 2024-09-19T08:39:01.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.203+0000 7fc4fb7fe640 1 --2- 172.21.15.98:0/1868270452 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc5000678f0 0x7fc500069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:01.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.203+0000 7fc4fb7fe640 1 -- 172.21.15.98:0/1868270452 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7fc50411ebc0 con 0x7fc4fc0039c0 2024-09-19T08:39:01.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.203+0000 7fc51259c640 1 --2- 172.21.15.98:0/1868270452 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc5000678f0 0x7fc500069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:01.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.203+0000 7fc51259c640 1 --2- 172.21.15.98:0/1868270452 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc5000678f0 0x7fc500069db0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc508007fd0 tx=0x7fc508009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:01.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.207+0000 7fc4fb7fe640 1 -- 172.21.15.98:0/1868270452 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc504016930 con 0x7fc4fc0039c0 2024-09-19T08:39:01.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.427+0000 7fc51902d640 1 -- 172.21.15.98:0/1868270452 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fc514160b40 con 0x7fc4fc0039c0 2024-09-19T08:39:01.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.431+0000 7fc4fb7fe640 1 -- 172.21.15.98:0/1868270452 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) ==== 97+0+0 (secure 0 0 0) 0x7fc5040efa50 con 0x7fc4fc0039c0 2024-09-19T08:39:01.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.431+0000 7fc51902d640 1 -- 172.21.15.98:0/1868270452 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc5000678f0 msgr2=0x7fc500069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:01.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.431+0000 7fc51902d640 1 --2- 172.21.15.98:0/1868270452 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc5000678f0 0x7fc500069db0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc508007fd0 tx=0x7fc508009040 comp rx=0 tx=0).stop 2024-09-19T08:39:01.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.431+0000 7fc51902d640 1 -- 172.21.15.98:0/1868270452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4fc0039c0 msgr2=0x7fc514151ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:01.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.431+0000 7fc51902d640 1 --2- 172.21.15.98:0/1868270452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4fc0039c0 0x7fc514151ab0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc5040985c0 tx=0x7fc4fc005840 comp rx=0 tx=0).stop 2024-09-19T08:39:01.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.431+0000 7fc51902d640 1 -- 172.21.15.98:0/1868270452 shutdown_connections 2024-09-19T08:39:01.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.431+0000 7fc51902d640 1 --2- 172.21.15.98:0/1868270452 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc5000678f0 0x7fc500069db0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:01.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.431+0000 7fc51902d640 1 --2- 172.21.15.98:0/1868270452 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc51414fe50 0x7fc514150300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:01.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.431+0000 7fc51902d640 1 --2- 172.21.15.98:0/1868270452 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc514142470 0x7fc51414e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:01.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.431+0000 7fc51902d640 1 --2- 172.21.15.98:0/1868270452 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4fc0039c0 0x7fc514151ab0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:01.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.431+0000 7fc51902d640 1 -- 172.21.15.98:0/1868270452 >> 172.21.15.98:0/1868270452 conn(0x7fc5140b2010 msgr2=0x7fc51415b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:01.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.435+0000 7fc51902d640 1 -- 172.21.15.98:0/1868270452 shutdown_connections 2024-09-19T08:39:01.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.435+0000 7fc51902d640 1 -- 172.21.15.98:0/1868270452 wait complete. 2024-09-19T08:39:01.452 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-19T08:39:01.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.555+0000 7f587fd43640 1 Processor -- start 2024-09-19T08:39:01.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.555+0000 7f587fd43640 1 -- start start 2024-09-19T08:39:01.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.555+0000 7f587fd43640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f58780cec60 0x7f587815b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:01.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.555+0000 7f587fd43640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58780af4a0 con 0x7f587806efa0 2024-09-19T08:39:01.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.555+0000 7f587fd43640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58780af780 con 0x7f587806f380 2024-09-19T08:39:01.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.555+0000 7f587fd43640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f587815ed00 con 0x7f58780cec60 2024-09-19T08:39:01.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.555+0000 7f587dadf640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f587806efa0 0x7f58780ce550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:55422/0 (socket says 172.21.15.98:55422) 2024-09-19T08:39:01.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.555+0000 7f587dadf640 1 -- 172.21.15.98:0/2867234438 learned_addr learned my addr 172.21.15.98:0/2867234438 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:01.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.555+0000 7f587cadd640 1 -- 172.21.15.98:0/2867234438 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3934415902 0 0) 0x7f58780af4a0 con 0x7f587806efa0 2024-09-19T08:39:01.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.555+0000 7f587cadd640 1 -- 172.21.15.98:0/2867234438 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5864003530 con 0x7f587806efa0 2024-09-19T08:39:01.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.555+0000 7f587e2e0640 1 --2- 172.21.15.98:0/2867234438 >> v2:172.21.15.139:3300/0 conn(0x7f58780cec60 0x7f587815b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:01.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.555+0000 7f587cadd640 1 -- 172.21.15.98:0/2867234438 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3696635573 0 0) 0x7f5864003530 con 0x7f587806efa0 2024-09-19T08:39:01.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.555+0000 7f587cadd640 1 -- 172.21.15.98:0/2867234438 >> v2:172.21.15.139:3300/0 conn(0x7f58780cec60 msgr2=0x7f587815b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:01.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.555+0000 7f587cadd640 1 --2- 172.21.15.98:0/2867234438 >> v2:172.21.15.139:3300/0 conn(0x7f58780cec60 0x7f587815b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:01.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.555+0000 7f587cadd640 1 -- 172.21.15.98:0/2867234438 >> v1:172.21.15.139:6789/0 conn(0x7f587806f380 legacy=0x7f58780ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:01.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.555+0000 7f587cadd640 1 -- 172.21.15.98:0/2867234438 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58780afa60 con 0x7f587806efa0 2024-09-19T08:39:01.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.555+0000 7f587cadd640 1 -- 172.21.15.98:0/2867234438 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f5874001680 con 0x7f587806efa0 2024-09-19T08:39:01.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.555+0000 7f587e2e0640 1 --2- 172.21.15.98:0/2867234438 >> v2:172.21.15.139:3300/0 conn(0x7f58780cec60 0x7f587815b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:39:01.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.555+0000 7f587cadd640 1 -- 172.21.15.98:0/2867234438 >> v1:172.21.15.98:6789/0 conn(0x7f587806efa0 legacy=0x7f58780ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:01.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.555+0000 7f587cadd640 1 --2- 172.21.15.98:0/2867234438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58640039c0 0x7f5864003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:01.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.555+0000 7f587cadd640 1 --2- 172.21.15.98:0/2867234438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5864004810 0x7f5864004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:01.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.555+0000 7f587cadd640 1 --2- 172.21.15.98:0/2867234438 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58640052f0 0x7f586400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:01.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.555+0000 7f587cadd640 1 -- 172.21.15.98:0/2867234438 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f587815ed00 con 0x7f5864004810 2024-09-19T08:39:01.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.555+0000 7f587cadd640 1 -- 172.21.15.98:0/2867234438 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f586400c240 con 0x7f58640039c0 2024-09-19T08:39:01.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.555+0000 7f587cadd640 1 -- 172.21.15.98:0/2867234438 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f586400c440 con 0x7f58640052f0 2024-09-19T08:39:01.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.555+0000 7f587d2de640 1 --2- 172.21.15.98:0/2867234438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5864004810 0x7f5864004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:01.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.555+0000 7f587dadf640 1 --2- 172.21.15.98:0/2867234438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58640039c0 0x7f5864003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:01.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.555+0000 7f587e2e0640 1 --2- 172.21.15.98:0/2867234438 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58640052f0 0x7f586400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:01.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.555+0000 7f587d2de640 1 -- 172.21.15.98:0/2867234438 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58640052f0 msgr2=0x7f586400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:01.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.555+0000 7f587d2de640 1 --2- 172.21.15.98:0/2867234438 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58640052f0 0x7f586400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:01.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.555+0000 7f587d2de640 1 -- 172.21.15.98:0/2867234438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58640039c0 msgr2=0x7f5864003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:01.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.555+0000 7f587d2de640 1 --2- 172.21.15.98:0/2867234438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58640039c0 0x7f5864003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:01.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.555+0000 7f587e2e0640 1 -- 172.21.15.98:0/2867234438 reap_dead start 2024-09-19T08:39:01.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.555+0000 7f587d2de640 1 -- 172.21.15.98:0/2867234438 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5864003530 con 0x7f5864004810 2024-09-19T08:39:01.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.559+0000 7f587d2de640 1 --2- 172.21.15.98:0/2867234438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5864004810 0x7f5864004c90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5868007fd0 tx=0x7f586801c280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8bd880ee36cd29e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:01.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.559+0000 7f587cadd640 1 -- 172.21.15.98:0/2867234438 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58680b0070 con 0x7f5864004810 2024-09-19T08:39:01.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.559+0000 7f587cadd640 1 -- 172.21.15.98:0/2867234438 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58680ab620 con 0x7f5864004810 2024-09-19T08:39:01.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.559+0000 7f587cadd640 1 -- 172.21.15.98:0/2867234438 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58680039b0 con 0x7f5864004810 2024-09-19T08:39:01.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.559+0000 7f587fd43640 1 -- 172.21.15.98:0/2867234438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5864004810 msgr2=0x7f5864004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:01.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.559+0000 7f587fd43640 1 --2- 172.21.15.98:0/2867234438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5864004810 0x7f5864004c90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5868007fd0 tx=0x7f586801c280 comp rx=0 tx=0).stop 2024-09-19T08:39:01.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.559+0000 7f587fd43640 1 -- 172.21.15.98:0/2867234438 shutdown_connections 2024-09-19T08:39:01.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.559+0000 7f587fd43640 1 --2- 172.21.15.98:0/2867234438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5864004810 0x7f5864004c90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:01.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.559+0000 7f587fd43640 1 -- 172.21.15.98:0/2867234438 >> 172.21.15.98:0/2867234438 conn(0x7f58780d0980 msgr2=0x7f58780d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:01.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.559+0000 7f587fd43640 1 -- 172.21.15.98:0/2867234438 shutdown_connections 2024-09-19T08:39:01.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.559+0000 7f587fd43640 1 -- 172.21.15.98:0/2867234438 wait complete. 2024-09-19T08:39:01.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.559+0000 7f587fd43640 1 Processor -- start 2024-09-19T08:39:01.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.559+0000 7f587fd43640 1 -- start start 2024-09-19T08:39:01.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.559+0000 7f587fd43640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5864004810 0x7f58781ecaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:01.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.559+0000 7f587fd43640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58780c94d0 0x7f587815adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:01.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.559+0000 7f587fd43640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f587806ddb0 0x7f587806e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:01.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.559+0000 7f587fd43640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f587815b970 con 0x7f587806ddb0 2024-09-19T08:39:01.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.559+0000 7f587fd43640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f587815b7f0 con 0x7f58780c94d0 2024-09-19T08:39:01.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.559+0000 7f587fd43640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f587815baf0 con 0x7f5864004810 2024-09-19T08:39:01.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.559+0000 7f587e2e0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f587806ddb0 0x7f587806e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:01.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.559+0000 7f587e2e0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f587806ddb0 0x7f587806e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35964/0 (socket says 172.21.15.98:35964) 2024-09-19T08:39:01.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.559+0000 7f587e2e0640 1 -- 172.21.15.98:0/2146635806 learned_addr learned my addr 172.21.15.98:0/2146635806 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:01.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.563+0000 7f587d2de640 1 --2- 172.21.15.98:0/2146635806 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58780c94d0 0x7f587815adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:01.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.563+0000 7f587dadf640 1 --2- 172.21.15.98:0/2146635806 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5864004810 0x7f58781ecaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:01.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.563+0000 7f587e2e0640 1 -- 172.21.15.98:0/2146635806 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5864004810 msgr2=0x7f58781ecaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:01.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.563+0000 7f587e2e0640 1 --2- 172.21.15.98:0/2146635806 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5864004810 0x7f58781ecaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:01.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.563+0000 7f587e2e0640 1 -- 172.21.15.98:0/2146635806 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58780c94d0 msgr2=0x7f587815adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:01.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.563+0000 7f587e2e0640 1 --2- 172.21.15.98:0/2146635806 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58780c94d0 0x7f587815adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:01.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.563+0000 7f587e2e0640 1 -- 172.21.15.98:0/2146635806 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58781ed0d0 con 0x7f587806ddb0 2024-09-19T08:39:01.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.563+0000 7f587d2de640 1 --2- 172.21.15.98:0/2146635806 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58780c94d0 0x7f587815adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:01.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.563+0000 7f587e2e0640 1 --2- 172.21.15.98:0/2146635806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f587806ddb0 0x7f587806e1f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5878068240 tx=0x7f5870007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:01.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.563+0000 7f586e7fc640 1 -- 172.21.15.98:0/2146635806 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5870014410 con 0x7f587806ddb0 2024-09-19T08:39:01.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.563+0000 7f586e7fc640 1 -- 172.21.15.98:0/2146635806 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5870019070 con 0x7f587806ddb0 2024-09-19T08:39:01.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.563+0000 7f586e7fc640 1 -- 172.21.15.98:0/2146635806 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58700147d0 con 0x7f587806ddb0 2024-09-19T08:39:01.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.563+0000 7f587fd43640 1 -- 172.21.15.98:0/2146635806 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58781ed3b0 con 0x7f587806ddb0 2024-09-19T08:39:01.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.563+0000 7f587fd43640 1 -- 172.21.15.98:0/2146635806 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58781ed970 con 0x7f587806ddb0 2024-09-19T08:39:01.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.563+0000 7f587fd43640 1 -- 172.21.15.98:0/2146635806 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f587815be80 con 0x7f587806ddb0 2024-09-19T08:39:01.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.563+0000 7f586e7fc640 1 -- 172.21.15.98:0/2146635806 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5870014970 con 0x7f587806ddb0 2024-09-19T08:39:01.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.563+0000 7f586e7fc640 1 --2- 172.21.15.98:0/2146635806 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f585c0678c0 0x7f585c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:01.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.563+0000 7f586e7fc640 1 -- 172.21.15.98:0/2146635806 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f5870085ff0 con 0x7f587806ddb0 2024-09-19T08:39:01.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.563+0000 7f587dadf640 1 --2- 172.21.15.98:0/2146635806 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f585c0678c0 0x7f585c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:01.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.563+0000 7f587dadf640 1 --2- 172.21.15.98:0/2146635806 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f585c0678c0 0x7f585c069d80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f58780d0310 tx=0x7f5874001d80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:01.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.567+0000 7f586e7fc640 1 -- 172.21.15.98:0/2146635806 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f5870056e80 con 0x7f587806ddb0 2024-09-19T08:39:01.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.771+0000 7f587fd43640 1 -- 172.21.15.98:0/2146635806 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f587806fee0 con 0x7f587806ddb0 2024-09-19T08:39:01.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.775+0000 7f586e7fc640 1 -- 172.21.15.98:0/2146635806 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7f5870056780 con 0x7f587806ddb0 2024-09-19T08:39:01.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.775+0000 7f587fd43640 1 -- 172.21.15.98:0/2146635806 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f585c0678c0 msgr2=0x7f585c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:01.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.775+0000 7f587fd43640 1 --2- 172.21.15.98:0/2146635806 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f585c0678c0 0x7f585c069d80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f58780d0310 tx=0x7f5874001d80 comp rx=0 tx=0).stop 2024-09-19T08:39:01.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.775+0000 7f587fd43640 1 -- 172.21.15.98:0/2146635806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f587806ddb0 msgr2=0x7f587806e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:01.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.775+0000 7f587fd43640 1 --2- 172.21.15.98:0/2146635806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f587806ddb0 0x7f587806e1f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5878068240 tx=0x7f5870007700 comp rx=0 tx=0).stop 2024-09-19T08:39:01.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.775+0000 7f587fd43640 1 -- 172.21.15.98:0/2146635806 shutdown_connections 2024-09-19T08:39:01.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.775+0000 7f587fd43640 1 --2- 172.21.15.98:0/2146635806 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f585c0678c0 0x7f585c069d80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:01.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.775+0000 7f587fd43640 1 --2- 172.21.15.98:0/2146635806 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f587806ddb0 0x7f587806e1f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:01.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.775+0000 7f587fd43640 1 --2- 172.21.15.98:0/2146635806 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58780c94d0 0x7f587815adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:01.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.775+0000 7f587fd43640 1 --2- 172.21.15.98:0/2146635806 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5864004810 0x7f58781ecaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:01.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.775+0000 7f587fd43640 1 -- 172.21.15.98:0/2146635806 >> 172.21.15.98:0/2146635806 conn(0x7f58780d0980 msgr2=0x7f587815fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:01.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.775+0000 7f587fd43640 1 -- 172.21.15.98:0/2146635806 shutdown_connections 2024-09-19T08:39:01.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.775+0000 7f587fd43640 1 -- 172.21.15.98:0/2146635806 wait complete. 2024-09-19T08:39:01.796 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-19T08:39:01.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.907+0000 7ff2ddcc0640 1 Processor -- start 2024-09-19T08:39:01.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.907+0000 7ff2ddcc0640 1 -- start start 2024-09-19T08:39:01.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.907+0000 7ff2ddcc0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff2d80d4820 0x7ff2d81621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:01.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.907+0000 7ff2ddcc0640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff2d80b9970 0x7ff2d80b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:01.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.907+0000 7ff2ddcc0640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2d8068440 con 0x7ff2d80b7f80 2024-09-19T08:39:01.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.907+0000 7ff2ddcc0640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7ff2d8163050 con 0x7ff2d80b9970 2024-09-19T08:39:01.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.907+0000 7ff2ddcc0640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff2d8162a90 con 0x7ff2d80d4820 2024-09-19T08:39:01.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.907+0000 7ff2d77fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7ff2d80b7f80 0x7ff2d815f560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:55428/0 (socket says 172.21.15.98:55428) 2024-09-19T08:39:01.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.911+0000 7ff2d77fe640 1 -- 172.21.15.98:0/483246730 learned_addr learned my addr 172.21.15.98:0/483246730 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:01.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.911+0000 7ff2d7fff640 1 --2- 172.21.15.98:0/483246730 >> v2:172.21.15.98:3300/0 conn(0x7ff2d80b9970 0x7ff2d80b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:01.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.911+0000 7ff2d6ffd640 1 --2- 172.21.15.98:0/483246730 >> v2:172.21.15.139:3300/0 conn(0x7ff2d80d4820 0x7ff2d81621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:01.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.911+0000 7ff2d67fc640 1 -- 172.21.15.98:0/483246730 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4193901205 0 0) 0x7ff2d8068440 con 0x7ff2d80b7f80 2024-09-19T08:39:01.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.911+0000 7ff2d67fc640 1 -- 172.21.15.98:0/483246730 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff2c0003530 con 0x7ff2d80b7f80 2024-09-19T08:39:01.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.911+0000 7ff2d7fff640 1 -- 172.21.15.98:0/483246730 >> v2:172.21.15.139:3300/0 conn(0x7ff2d80d4820 msgr2=0x7ff2d81621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:01.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.911+0000 7ff2d7fff640 1 --2- 172.21.15.98:0/483246730 >> v2:172.21.15.139:3300/0 conn(0x7ff2d80d4820 0x7ff2d81621b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:01.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.911+0000 7ff2d7fff640 1 -- 172.21.15.98:0/483246730 >> v1:172.21.15.98:6789/0 conn(0x7ff2d80b7f80 legacy=0x7ff2d815f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:01.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.911+0000 7ff2d7fff640 1 -- 172.21.15.98:0/483246730 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d8162d70 con 0x7ff2d80b9970 2024-09-19T08:39:01.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.911+0000 7ff2d7fff640 1 --2- 172.21.15.98:0/483246730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2d80b9970 0x7ff2d80b9de0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff2d80ceb00 tx=0x7ff2d001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34cca1d7fa34051 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:01.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.911+0000 7ff2d67fc640 1 -- 172.21.15.98:0/483246730 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d00b2070 con 0x7ff2d80b9970 2024-09-19T08:39:01.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.911+0000 7ff2d67fc640 1 -- 172.21.15.98:0/483246730 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2d00a5900 con 0x7ff2d80b9970 2024-09-19T08:39:01.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.911+0000 7ff2d67fc640 1 -- 172.21.15.98:0/483246730 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d00a5cd0 con 0x7ff2d80b9970 2024-09-19T08:39:01.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.911+0000 7ff2ddcc0640 1 -- 172.21.15.98:0/483246730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2d80b9970 msgr2=0x7ff2d80b9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:01.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.911+0000 7ff2ddcc0640 1 --2- 172.21.15.98:0/483246730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2d80b9970 0x7ff2d80b9de0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff2d80ceb00 tx=0x7ff2d001c930 comp rx=0 tx=0).stop 2024-09-19T08:39:01.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.911+0000 7ff2ddcc0640 1 -- 172.21.15.98:0/483246730 shutdown_connections 2024-09-19T08:39:01.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.911+0000 7ff2ddcc0640 1 --2- 172.21.15.98:0/483246730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2d80b9970 0x7ff2d80b9de0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:01.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.911+0000 7ff2ddcc0640 1 --2- 172.21.15.98:0/483246730 >> v2:172.21.15.139:3300/0 conn(0x7ff2d80d4820 0x7ff2d81621b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:01.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.911+0000 7ff2ddcc0640 1 -- 172.21.15.98:0/483246730 >> 172.21.15.98:0/483246730 conn(0x7ff2d8074e40 msgr2=0x7ff2d8075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:01.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.911+0000 7ff2ddcc0640 1 -- 172.21.15.98:0/483246730 shutdown_connections 2024-09-19T08:39:01.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.911+0000 7ff2ddcc0640 1 -- 172.21.15.98:0/483246730 wait complete. 2024-09-19T08:39:01.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.915+0000 7ff2ddcc0640 1 Processor -- start 2024-09-19T08:39:01.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.915+0000 7ff2ddcc0640 1 -- start start 2024-09-19T08:39:01.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.915+0000 7ff2ddcc0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2d80b7f80 0x7ff2d8139fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:01.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.915+0000 7ff2ddcc0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2d80b9970 0x7ff2d813d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:01.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.915+0000 7ff2ddcc0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2d80d4820 0x7ff2d813d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:01.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.915+0000 7ff2ddcc0640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d80b84e0 con 0x7ff2d80b7f80 2024-09-19T08:39:01.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.915+0000 7ff2ddcc0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d80b8360 con 0x7ff2d80d4820 2024-09-19T08:39:01.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.915+0000 7ff2ddcc0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff2d8144590 con 0x7ff2d80b9970 2024-09-19T08:39:01.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.915+0000 7ff2d77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2d80b7f80 0x7ff2d8139fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:01.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.915+0000 7ff2d77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2d80b7f80 0x7ff2d8139fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35982/0 (socket says 172.21.15.98:35982) 2024-09-19T08:39:01.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.915+0000 7ff2d77fe640 1 -- 172.21.15.98:0/3361441143 learned_addr learned my addr 172.21.15.98:0/3361441143 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:01.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.915+0000 7ff2d7fff640 1 --2- 172.21.15.98:0/3361441143 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2d80d4820 0x7ff2d813d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:01.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.915+0000 7ff2d6ffd640 1 --2- 172.21.15.98:0/3361441143 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2d80b9970 0x7ff2d813d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:01.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.915+0000 7ff2d77fe640 1 -- 172.21.15.98:0/3361441143 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2d80b9970 msgr2=0x7ff2d813d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:01.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.915+0000 7ff2d77fe640 1 --2- 172.21.15.98:0/3361441143 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2d80b9970 0x7ff2d813d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:01.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.915+0000 7ff2d77fe640 1 -- 172.21.15.98:0/3361441143 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2d80d4820 msgr2=0x7ff2d813d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:01.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.915+0000 7ff2d77fe640 1 --2- 172.21.15.98:0/3361441143 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2d80d4820 0x7ff2d813d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:01.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.915+0000 7ff2d77fe640 1 -- 172.21.15.98:0/3361441143 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d8154b20 con 0x7ff2d80b7f80 2024-09-19T08:39:01.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.915+0000 7ff2d77fe640 1 --2- 172.21.15.98:0/3361441143 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2d80b7f80 0x7ff2d8139fb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800c3f0 tx=0x7ff2c800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:01.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.915+0000 7ff2bffff640 1 -- 172.21.15.98:0/3361441143 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8017070 con 0x7ff2d80b7f80 2024-09-19T08:39:01.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.915+0000 7ff2bffff640 1 -- 172.21.15.98:0/3361441143 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2c8005cf0 con 0x7ff2d80b7f80 2024-09-19T08:39:01.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.915+0000 7ff2ddcc0640 1 -- 172.21.15.98:0/3361441143 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2d815df70 con 0x7ff2d80b7f80 2024-09-19T08:39:01.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.915+0000 7ff2bffff640 1 -- 172.21.15.98:0/3361441143 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c80072f0 con 0x7ff2d80b7f80 2024-09-19T08:39:01.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.915+0000 7ff2ddcc0640 1 -- 172.21.15.98:0/3361441143 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2d81390b0 con 0x7ff2d80b7f80 2024-09-19T08:39:01.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.915+0000 7ff2ddcc0640 1 -- 172.21.15.98:0/3361441143 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2d8068f50 con 0x7ff2d80b7f80 2024-09-19T08:39:01.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.915+0000 7ff2bffff640 1 -- 172.21.15.98:0/3361441143 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff2c801a070 con 0x7ff2d80b7f80 2024-09-19T08:39:01.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.919+0000 7ff2bffff640 1 --2- 172.21.15.98:0/3361441143 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff2c40678f0 0x7ff2c4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:01.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.919+0000 7ff2bffff640 1 -- 172.21.15.98:0/3361441143 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7ff2c8084d50 con 0x7ff2d80b7f80 2024-09-19T08:39:01.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.919+0000 7ff2d6ffd640 1 --2- 172.21.15.98:0/3361441143 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff2c40678f0 0x7ff2c4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:01.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.919+0000 7ff2d6ffd640 1 --2- 172.21.15.98:0/3361441143 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff2c40678f0 0x7ff2c4069db0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff2d81fa7a0 tx=0x7ff2cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:01.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:01.923+0000 7ff2bffff640 1 -- 172.21.15.98:0/3361441143 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff2c8088030 con 0x7ff2d80b7f80 2024-09-19T08:39:02.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.083+0000 7ff2ddcc0640 1 -- 172.21.15.98:0/3361441143 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff2d81448c0 con 0x7ff2d80b7f80 2024-09-19T08:39:02.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.087+0000 7ff2bffff640 1 -- 172.21.15.98:0/3361441143 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v28) ==== 84+0+65 (secure 0 0 0) 0x7ff2c8055be0 con 0x7ff2d80b7f80 2024-09-19T08:39:02.089 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:39:02.089 INFO:teuthology.orchestra.run.smithi098.stdout:[ 2024-09-19T08:39:02.090 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:39:02.090 INFO:teuthology.orchestra.run.smithi098.stdout: "poolnum": 1, 2024-09-19T08:39:02.090 INFO:teuthology.orchestra.run.smithi098.stdout: "poolname": ".mgr" 2024-09-19T08:39:02.090 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:39:02.090 INFO:teuthology.orchestra.run.smithi098.stdout:] 2024-09-19T08:39:02.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.087+0000 7ff2ddcc0640 1 -- 172.21.15.98:0/3361441143 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff2c40678f0 msgr2=0x7ff2c4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:02.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.087+0000 7ff2ddcc0640 1 --2- 172.21.15.98:0/3361441143 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff2c40678f0 0x7ff2c4069db0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff2d81fa7a0 tx=0x7ff2cc008040 comp rx=0 tx=0).stop 2024-09-19T08:39:02.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.087+0000 7ff2ddcc0640 1 -- 172.21.15.98:0/3361441143 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2d80b7f80 msgr2=0x7ff2d8139fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:02.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.087+0000 7ff2ddcc0640 1 --2- 172.21.15.98:0/3361441143 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2d80b7f80 0x7ff2d8139fb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800c3f0 tx=0x7ff2c800c8b0 comp rx=0 tx=0).stop 2024-09-19T08:39:02.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.087+0000 7ff2ddcc0640 1 -- 172.21.15.98:0/3361441143 shutdown_connections 2024-09-19T08:39:02.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.087+0000 7ff2ddcc0640 1 --2- 172.21.15.98:0/3361441143 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff2c40678f0 0x7ff2c4069db0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:02.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.087+0000 7ff2ddcc0640 1 --2- 172.21.15.98:0/3361441143 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2d80d4820 0x7ff2d813d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:02.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.087+0000 7ff2ddcc0640 1 --2- 172.21.15.98:0/3361441143 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2d80b9970 0x7ff2d813d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:02.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.087+0000 7ff2ddcc0640 1 --2- 172.21.15.98:0/3361441143 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2d80b7f80 0x7ff2d8139fb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:02.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.087+0000 7ff2ddcc0640 1 -- 172.21.15.98:0/3361441143 >> 172.21.15.98:0/3361441143 conn(0x7ff2d8074e40 msgr2=0x7ff2d81431b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:02.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.087+0000 7ff2ddcc0640 1 -- 172.21.15.98:0/3361441143 shutdown_connections 2024-09-19T08:39:02.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.087+0000 7ff2ddcc0640 1 -- 172.21.15.98:0/3361441143 wait complete. 2024-09-19T08:39:02.107 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T08:39:02.107 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T08:39:02.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.215+0000 7ffa8676d640 1 Processor -- start 2024-09-19T08:39:02.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.215+0000 7ffa8676d640 1 -- start start 2024-09-19T08:39:02.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.215+0000 7ffa8676d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffa800cdac0 0x7ffa8006e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:02.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.215+0000 7ffa8676d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ffa8006edd0 0x7ffa8015ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:02.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.215+0000 7ffa8676d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa8015c480 con 0x7ffa8015c0a0 2024-09-19T08:39:02.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.215+0000 7ffa8676d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ffa80162b70 con 0x7ffa8006edd0 2024-09-19T08:39:02.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.215+0000 7ffa8676d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ffa80162240 con 0x7ffa800cdac0 2024-09-19T08:39:02.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.215+0000 7ffa7f7fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffa800cdac0 0x7ffa8006e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:02.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.215+0000 7ffa84d0a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ffa8006edd0 0x7ffa8015ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:02.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.215+0000 7ffa7f7fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffa800cdac0 0x7ffa8006e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:57544/0 (socket says 172.21.15.98:57544) 2024-09-19T08:39:02.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.215+0000 7ffa84d0a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ffa8006edd0 0x7ffa8015ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:46268/0 (socket says 172.21.15.98:46268) 2024-09-19T08:39:02.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.215+0000 7ffa7f7fe640 1 -- 172.21.15.98:0/1922600113 learned_addr learned my addr 172.21.15.98:0/1922600113 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:02.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.215+0000 7ffa7f7fe640 1 -- 172.21.15.98:0/1922600113 >> v2:172.21.15.139:3300/0 conn(0x7ffa8006edd0 msgr2=0x7ffa8015ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:02.221 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.215+0000 7ffa7f7fe640 1 --2- 172.21.15.98:0/1922600113 >> v2:172.21.15.139:3300/0 conn(0x7ffa8006edd0 0x7ffa8015ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:02.221 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.215+0000 7ffa7f7fe640 1 -- 172.21.15.98:0/1922600113 >> v1:172.21.15.139:6789/0 conn(0x7ffa8015c0a0 legacy=0x7ffa800cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:02.221 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.215+0000 7ffa7f7fe640 1 -- 172.21.15.98:0/1922600113 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa80162520 con 0x7ffa800cdac0 2024-09-19T08:39:02.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.219+0000 7ffa7f7fe640 1 --2- 172.21.15.98:0/1922600113 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffa800cdac0 0x7ffa8006e890 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ffa74009870 tx=0x7ffa7401c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d3b8128e64dbdad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:02.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.227+0000 7ffa7effd640 1 -- 172.21.15.98:0/1922600113 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa740b0070 con 0x7ffa800cdac0 2024-09-19T08:39:02.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.227+0000 7ffa7effd640 1 -- 172.21.15.98:0/1922600113 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa740ab600 con 0x7ffa800cdac0 2024-09-19T08:39:02.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.227+0000 7ffa7effd640 1 -- 172.21.15.98:0/1922600113 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa740ab9d0 con 0x7ffa800cdac0 2024-09-19T08:39:02.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.227+0000 7ffa8676d640 1 -- 172.21.15.98:0/1922600113 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffa800cdac0 msgr2=0x7ffa8006e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:02.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.227+0000 7ffa8676d640 1 --2- 172.21.15.98:0/1922600113 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffa800cdac0 0x7ffa8006e890 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ffa74009870 tx=0x7ffa7401c4d0 comp rx=0 tx=0).stop 2024-09-19T08:39:02.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.227+0000 7ffa8676d640 1 -- 172.21.15.98:0/1922600113 shutdown_connections 2024-09-19T08:39:02.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.227+0000 7ffa8676d640 1 --2- 172.21.15.98:0/1922600113 >> v2:172.21.15.139:3300/0 conn(0x7ffa8006edd0 0x7ffa8015ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:02.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.227+0000 7ffa8676d640 1 --2- 172.21.15.98:0/1922600113 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffa800cdac0 0x7ffa8006e890 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:02.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.227+0000 7ffa8676d640 1 -- 172.21.15.98:0/1922600113 >> 172.21.15.98:0/1922600113 conn(0x7ffa800b25c0 msgr2=0x7ffa800b29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:02.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.227+0000 7ffa8676d640 1 -- 172.21.15.98:0/1922600113 shutdown_connections 2024-09-19T08:39:02.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.227+0000 7ffa8676d640 1 -- 172.21.15.98:0/1922600113 wait complete. 2024-09-19T08:39:02.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.227+0000 7ffa8676d640 1 Processor -- start 2024-09-19T08:39:02.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.227+0000 7ffa8676d640 1 -- start start 2024-09-19T08:39:02.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.231+0000 7ffa8676d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffa8006edd0 0x7ffa80152ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:02.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.231+0000 7ffa8676d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa800cdac0 0x7ffa80155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:02.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.231+0000 7ffa8676d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa8015c0a0 0x7ffa8014e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:02.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.231+0000 7ffa8676d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8015f910 con 0x7ffa800cdac0 2024-09-19T08:39:02.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.231+0000 7ffa8676d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8015f790 con 0x7ffa8015c0a0 2024-09-19T08:39:02.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.231+0000 7ffa8676d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ffa8016b3d0 con 0x7ffa8006edd0 2024-09-19T08:39:02.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.231+0000 7ffa7f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa800cdac0 0x7ffa80155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:02.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.231+0000 7ffa7f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa800cdac0 0x7ffa80155230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35988/0 (socket says 172.21.15.98:35988) 2024-09-19T08:39:02.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.231+0000 7ffa7f7fe640 1 -- 172.21.15.98:0/2844044985 learned_addr learned my addr 172.21.15.98:0/2844044985 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:02.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.231+0000 7ffa7ffff640 1 --2- 172.21.15.98:0/2844044985 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffa8006edd0 0x7ffa80152ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:02.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.231+0000 7ffa84d0a640 1 --2- 172.21.15.98:0/2844044985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa8015c0a0 0x7ffa8014e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:02.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.231+0000 7ffa7f7fe640 1 -- 172.21.15.98:0/2844044985 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffa8006edd0 msgr2=0x7ffa80152ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:02.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.231+0000 7ffa7f7fe640 1 --2- 172.21.15.98:0/2844044985 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffa8006edd0 0x7ffa80152ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:02.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.231+0000 7ffa7f7fe640 1 -- 172.21.15.98:0/2844044985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa8015c0a0 msgr2=0x7ffa8014e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:02.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.231+0000 7ffa7f7fe640 1 --2- 172.21.15.98:0/2844044985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa8015c0a0 0x7ffa8014e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:02.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.231+0000 7ffa7f7fe640 1 -- 172.21.15.98:0/2844044985 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa801537a0 con 0x7ffa800cdac0 2024-09-19T08:39:02.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.231+0000 7ffa7f7fe640 1 --2- 172.21.15.98:0/2844044985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa800cdac0 0x7ffa80155230 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ffa7401ab80 tx=0x7ffa74019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:02.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.231+0000 7ffa7cff9640 1 -- 172.21.15.98:0/2844044985 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa740ba070 con 0x7ffa800cdac0 2024-09-19T08:39:02.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.231+0000 7ffa7cff9640 1 -- 172.21.15.98:0/2844044985 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa740b4420 con 0x7ffa800cdac0 2024-09-19T08:39:02.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.231+0000 7ffa8676d640 1 -- 172.21.15.98:0/2844044985 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa80153a80 con 0x7ffa800cdac0 2024-09-19T08:39:02.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.231+0000 7ffa7cff9640 1 -- 172.21.15.98:0/2844044985 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa740b0040 con 0x7ffa800cdac0 2024-09-19T08:39:02.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.231+0000 7ffa8676d640 1 -- 172.21.15.98:0/2844044985 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa80154040 con 0x7ffa800cdac0 2024-09-19T08:39:02.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.231+0000 7ffa8676d640 1 -- 172.21.15.98:0/2844044985 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa8015ea40 con 0x7ffa800cdac0 2024-09-19T08:39:02.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.231+0000 7ffa7cff9640 1 -- 172.21.15.98:0/2844044985 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ffa740bd070 con 0x7ffa800cdac0 2024-09-19T08:39:02.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.231+0000 7ffa7cff9640 1 --2- 172.21.15.98:0/2844044985 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ffa6c0678f0 0x7ffa6c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:02.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.231+0000 7ffa7cff9640 1 -- 172.21.15.98:0/2844044985 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7ffa7411e940 con 0x7ffa800cdac0 2024-09-19T08:39:02.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.231+0000 7ffa7ffff640 1 --2- 172.21.15.98:0/2844044985 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ffa6c0678f0 0x7ffa6c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:02.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.231+0000 7ffa7ffff640 1 --2- 172.21.15.98:0/2844044985 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ffa6c0678f0 0x7ffa6c069db0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ffa78001e00 tx=0x7ffa78009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:02.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.235+0000 7ffa7cff9640 1 -- 172.21.15.98:0/2844044985 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ffa74016150 con 0x7ffa800cdac0 2024-09-19T08:39:02.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:02.387+0000 7ffa8676d640 1 -- 172.21.15.98:0/2844044985 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ffa80160320 con 0x7ffa800cdac0 2024-09-19T08:39:03.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.231+0000 7ffa7cff9640 1 -- 172.21.15.98:0/2844044985 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v29) ==== 138+0+0 (secure 0 0 0) 0x7ffa740ef180 con 0x7ffa800cdac0 2024-09-19T08:39:03.234 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_metadata' created 2024-09-19T08:39:03.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.235+0000 7ffa8676d640 1 -- 172.21.15.98:0/2844044985 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ffa6c0678f0 msgr2=0x7ffa6c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:03.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.235+0000 7ffa8676d640 1 --2- 172.21.15.98:0/2844044985 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ffa6c0678f0 0x7ffa6c069db0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ffa78001e00 tx=0x7ffa78009040 comp rx=0 tx=0).stop 2024-09-19T08:39:03.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.235+0000 7ffa8676d640 1 -- 172.21.15.98:0/2844044985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa800cdac0 msgr2=0x7ffa80155230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:03.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.235+0000 7ffa8676d640 1 --2- 172.21.15.98:0/2844044985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa800cdac0 0x7ffa80155230 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ffa7401ab80 tx=0x7ffa74019270 comp rx=0 tx=0).stop 2024-09-19T08:39:03.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.235+0000 7ffa8676d640 1 -- 172.21.15.98:0/2844044985 shutdown_connections 2024-09-19T08:39:03.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.235+0000 7ffa8676d640 1 --2- 172.21.15.98:0/2844044985 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ffa6c0678f0 0x7ffa6c069db0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:03.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.235+0000 7ffa8676d640 1 --2- 172.21.15.98:0/2844044985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffa8015c0a0 0x7ffa8014e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:03.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.235+0000 7ffa8676d640 1 --2- 172.21.15.98:0/2844044985 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffa800cdac0 0x7ffa80155230 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:03.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.235+0000 7ffa8676d640 1 --2- 172.21.15.98:0/2844044985 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffa8006edd0 0x7ffa80152ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:03.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.235+0000 7ffa8676d640 1 -- 172.21.15.98:0/2844044985 >> 172.21.15.98:0/2844044985 conn(0x7ffa800b25c0 msgr2=0x7ffa80160b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:03.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.235+0000 7ffa8676d640 1 -- 172.21.15.98:0/2844044985 shutdown_connections 2024-09-19T08:39:03.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.235+0000 7ffa8676d640 1 -- 172.21.15.98:0/2844044985 wait complete. 2024-09-19T08:39:03.263 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T08:39:03.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.399+0000 7fa91b37a640 1 Processor -- start 2024-09-19T08:39:03.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.399+0000 7fa91b37a640 1 -- start start 2024-09-19T08:39:03.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.399+0000 7fa91b37a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa9140729f0 0x7fa91406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:03.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.399+0000 7fa91b37a640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa914072ea0 con 0x7fa91406c1d0 2024-09-19T08:39:03.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.399+0000 7fa91b37a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa914073180 con 0x7fa914144d10 2024-09-19T08:39:03.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.399+0000 7fa91b37a640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fa91406c620 con 0x7fa9140729f0 2024-09-19T08:39:03.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.399+0000 7fa918915640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa9140729f0 0x7fa91406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:03.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.399+0000 7fa919917640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fa91406c1d0 0x7fa9140cd250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:55436/0 (socket says 172.21.15.98:55436) 2024-09-19T08:39:03.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.399+0000 7fa919917640 1 -- 172.21.15.98:0/379657926 learned_addr learned my addr 172.21.15.98:0/379657926 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:03.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.399+0000 7fa918915640 1 -- 172.21.15.98:0/379657926 >> v1:172.21.15.139:6789/0 conn(0x7fa914144d10 legacy=0x7fa9140722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:03.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.399+0000 7fa90bfff640 1 -- 172.21.15.98:0/379657926 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1558603593 0 0) 0x7fa914072ea0 con 0x7fa91406c1d0 2024-09-19T08:39:03.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.399+0000 7fa918915640 1 -- 172.21.15.98:0/379657926 >> v1:172.21.15.98:6789/0 conn(0x7fa91406c1d0 legacy=0x7fa9140cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:03.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.399+0000 7fa918915640 1 -- 172.21.15.98:0/379657926 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa914073460 con 0x7fa9140729f0 2024-09-19T08:39:03.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.399+0000 7fa918915640 1 --2- 172.21.15.98:0/379657926 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9140729f0 0x7fa91406bc90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa9040097c0 tx=0x7fa90401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171d1eea953fd93 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:03.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.399+0000 7fa90bfff640 1 -- 172.21.15.98:0/379657926 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9040b0070 con 0x7fa9140729f0 2024-09-19T08:39:03.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.399+0000 7fa90bfff640 1 -- 172.21.15.98:0/379657926 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9040ab600 con 0x7fa9140729f0 2024-09-19T08:39:03.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.403+0000 7fa90bfff640 1 -- 172.21.15.98:0/379657926 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9040ab9d0 con 0x7fa9140729f0 2024-09-19T08:39:03.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.403+0000 7fa91b37a640 1 -- 172.21.15.98:0/379657926 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9140729f0 msgr2=0x7fa91406bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:03.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.403+0000 7fa91b37a640 1 --2- 172.21.15.98:0/379657926 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9140729f0 0x7fa91406bc90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa9040097c0 tx=0x7fa90401c4a0 comp rx=0 tx=0).stop 2024-09-19T08:39:03.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.403+0000 7fa91b37a640 1 -- 172.21.15.98:0/379657926 shutdown_connections 2024-09-19T08:39:03.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.403+0000 7fa91b37a640 1 --2- 172.21.15.98:0/379657926 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9140729f0 0x7fa91406bc90 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:03.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.403+0000 7fa91b37a640 1 -- 172.21.15.98:0/379657926 >> 172.21.15.98:0/379657926 conn(0x7fa9140b2010 msgr2=0x7fa9140b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:03.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.403+0000 7fa91b37a640 1 -- 172.21.15.98:0/379657926 shutdown_connections 2024-09-19T08:39:03.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.403+0000 7fa91b37a640 1 -- 172.21.15.98:0/379657926 wait complete. 2024-09-19T08:39:03.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.403+0000 7fa91b37a640 1 Processor -- start 2024-09-19T08:39:03.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.403+0000 7fa91b37a640 1 -- start start 2024-09-19T08:39:03.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.403+0000 7fa91b37a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa91406c1d0 0x7fa914142c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:03.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.403+0000 7fa91b37a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9140729f0 0x7fa9140aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:03.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.403+0000 7fa91b37a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa914144d10 0x7fa91415ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:03.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.403+0000 7fa91b37a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa9140ce500 con 0x7fa9140729f0 2024-09-19T08:39:03.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.403+0000 7fa91b37a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa9140ce380 con 0x7fa91406c1d0 2024-09-19T08:39:03.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.403+0000 7fa91b37a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa914168f70 con 0x7fa914144d10 2024-09-19T08:39:03.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.403+0000 7fa918915640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9140729f0 0x7fa9140aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:03.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.403+0000 7fa918915640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9140729f0 0x7fa9140aeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36012/0 (socket says 172.21.15.98:36012) 2024-09-19T08:39:03.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.403+0000 7fa918915640 1 -- 172.21.15.98:0/3439729016 learned_addr learned my addr 172.21.15.98:0/3439729016 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:03.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.403+0000 7fa919917640 1 --2- 172.21.15.98:0/3439729016 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa914144d10 0x7fa91415ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:03.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.403+0000 7fa919116640 1 --2- 172.21.15.98:0/3439729016 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa91406c1d0 0x7fa914142c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:03.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.403+0000 7fa918915640 1 -- 172.21.15.98:0/3439729016 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa914144d10 msgr2=0x7fa91415ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:03.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.403+0000 7fa918915640 1 --2- 172.21.15.98:0/3439729016 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa914144d10 0x7fa91415ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:03.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.403+0000 7fa918915640 1 -- 172.21.15.98:0/3439729016 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa91406c1d0 msgr2=0x7fa914142c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:03.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.403+0000 7fa918915640 1 --2- 172.21.15.98:0/3439729016 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa91406c1d0 0x7fa914142c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:03.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.403+0000 7fa918915640 1 -- 172.21.15.98:0/3439729016 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9140742b0 con 0x7fa9140729f0 2024-09-19T08:39:03.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.403+0000 7fa919116640 1 --2- 172.21.15.98:0/3439729016 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa91406c1d0 0x7fa914142c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:03.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.403+0000 7fa918915640 1 --2- 172.21.15.98:0/3439729016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9140729f0 0x7fa9140aeb10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa9040097c0 tx=0x7fa904019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:03.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.403+0000 7fa909ffb640 1 -- 172.21.15.98:0/3439729016 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9040b8030 con 0x7fa9140729f0 2024-09-19T08:39:03.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.403+0000 7fa909ffb640 1 -- 172.21.15.98:0/3439729016 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9040b4410 con 0x7fa9140729f0 2024-09-19T08:39:03.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.403+0000 7fa91b37a640 1 -- 172.21.15.98:0/3439729016 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9141432d0 con 0x7fa9140729f0 2024-09-19T08:39:03.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.403+0000 7fa909ffb640 1 -- 172.21.15.98:0/3439729016 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9040b0040 con 0x7fa9140729f0 2024-09-19T08:39:03.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.403+0000 7fa91b37a640 1 -- 172.21.15.98:0/3439729016 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa914143890 con 0x7fa9140729f0 2024-09-19T08:39:03.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.407+0000 7fa91b37a640 1 -- 172.21.15.98:0/3439729016 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa91406bc90 con 0x7fa9140729f0 2024-09-19T08:39:03.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.407+0000 7fa909ffb640 1 -- 172.21.15.98:0/3439729016 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa9040bb070 con 0x7fa9140729f0 2024-09-19T08:39:03.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.407+0000 7fa909ffb640 1 --2- 172.21.15.98:0/3439729016 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa8f80678c0 0x7fa8f8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:03.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.407+0000 7fa909ffb640 1 -- 172.21.15.98:0/3439729016 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5227+0+0 (secure 0 0 0) 0x7fa90411eed0 con 0x7fa9140729f0 2024-09-19T08:39:03.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.407+0000 7fa919116640 1 --2- 172.21.15.98:0/3439729016 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa8f80678c0 0x7fa8f8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:03.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.407+0000 7fa919116640 1 --2- 172.21.15.98:0/3439729016 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa8f80678c0 0x7fa8f8069d80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa9140b44b0 tx=0x7fa910009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:03.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.411+0000 7fa909ffb640 1 -- 172.21.15.98:0/3439729016 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa9040c3020 con 0x7fa9140729f0 2024-09-19T08:39:03.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:03.575+0000 7fa91b37a640 1 -- 172.21.15.98:0/3439729016 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa9140b2bb0 con 0x7fa9140729f0 2024-09-19T08:39:04.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.239+0000 7fa909ffb640 1 -- 172.21.15.98:0/3439729016 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v30) ==== 170+0+0 (secure 0 0 0) 0x7fa904016090 con 0x7fa9140729f0 2024-09-19T08:39:04.245 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' created 2024-09-19T08:39:04.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.243+0000 7fa91b37a640 1 -- 172.21.15.98:0/3439729016 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa8f80678c0 msgr2=0x7fa8f8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:04.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.243+0000 7fa91b37a640 1 --2- 172.21.15.98:0/3439729016 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa8f80678c0 0x7fa8f8069d80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa9140b44b0 tx=0x7fa910009040 comp rx=0 tx=0).stop 2024-09-19T08:39:04.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.243+0000 7fa91b37a640 1 -- 172.21.15.98:0/3439729016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9140729f0 msgr2=0x7fa9140aeb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:04.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.243+0000 7fa91b37a640 1 --2- 172.21.15.98:0/3439729016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9140729f0 0x7fa9140aeb10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa9040097c0 tx=0x7fa904019240 comp rx=0 tx=0).stop 2024-09-19T08:39:04.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.243+0000 7fa91b37a640 1 -- 172.21.15.98:0/3439729016 shutdown_connections 2024-09-19T08:39:04.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.243+0000 7fa91b37a640 1 --2- 172.21.15.98:0/3439729016 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa8f80678c0 0x7fa8f8069d80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:04.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.243+0000 7fa91b37a640 1 --2- 172.21.15.98:0/3439729016 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa914144d10 0x7fa91415ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:04.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.243+0000 7fa91b37a640 1 --2- 172.21.15.98:0/3439729016 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa9140729f0 0x7fa9140aeb10 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:04.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.243+0000 7fa91b37a640 1 --2- 172.21.15.98:0/3439729016 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa91406c1d0 0x7fa914142c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:04.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.243+0000 7fa91b37a640 1 -- 172.21.15.98:0/3439729016 >> 172.21.15.98:0/3439729016 conn(0x7fa9140b2010 msgr2=0x7fa91406d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:04.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.243+0000 7fa91b37a640 1 -- 172.21.15.98:0/3439729016 shutdown_connections 2024-09-19T08:39:04.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.243+0000 7fa91b37a640 1 -- 172.21.15.98:0/3439729016 wait complete. 2024-09-19T08:39:04.284 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T08:39:04.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.403+0000 7f4a77bff640 1 Processor -- start 2024-09-19T08:39:04.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.403+0000 7f4a77bff640 1 -- start start 2024-09-19T08:39:04.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.407+0000 7f4a77bff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4a70145b70 0x7f4a7006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:04.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.407+0000 7f4a77bff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4a7006c1d0 0x7f4a700713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:04.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.407+0000 7f4a77bff640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a70073530 con 0x7f4a700718f0 2024-09-19T08:39:04.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.407+0000 7f4a77bff640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f4a7006c620 con 0x7f4a70145b70 2024-09-19T08:39:04.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.407+0000 7f4a77bff640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f4a70073810 con 0x7f4a7006c1d0 2024-09-19T08:39:04.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.407+0000 7f4a7599b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4a70145b70 0x7f4a7006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:04.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.407+0000 7f4a7599b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4a70145b70 0x7f4a7006bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36028/0 (socket says 172.21.15.98:36028) 2024-09-19T08:39:04.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.407+0000 7f4a7599b640 1 -- 172.21.15.98:0/698814050 learned_addr learned my addr 172.21.15.98:0/698814050 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:04.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.407+0000 7f4a7519a640 1 --2- 172.21.15.98:0/698814050 >> v2:172.21.15.139:3300/0 conn(0x7f4a7006c1d0 0x7f4a700713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:04.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.407+0000 7f4a7599b640 1 -- 172.21.15.98:0/698814050 >> v2:172.21.15.139:3300/0 conn(0x7f4a7006c1d0 msgr2=0x7f4a700713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:04.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.407+0000 7f4a7599b640 1 --2- 172.21.15.98:0/698814050 >> v2:172.21.15.139:3300/0 conn(0x7f4a7006c1d0 0x7f4a700713b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:04.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.407+0000 7f4a7599b640 1 -- 172.21.15.98:0/698814050 >> v1:172.21.15.139:6789/0 conn(0x7f4a700718f0 legacy=0x7f4a700cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:04.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.407+0000 7f4a7599b640 1 -- 172.21.15.98:0/698814050 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a70073af0 con 0x7f4a70145b70 2024-09-19T08:39:04.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.407+0000 7f4a7599b640 1 --2- 172.21.15.98:0/698814050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a70145b70 0x7f4a7006bc90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4a700ccd30 tx=0x7f4a6c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a87ada499160ac0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:04.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.407+0000 7f4a74999640 1 -- 172.21.15.98:0/698814050 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c0b0070 con 0x7f4a70145b70 2024-09-19T08:39:04.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.411+0000 7f4a74999640 1 -- 172.21.15.98:0/698814050 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a6c0ab600 con 0x7f4a70145b70 2024-09-19T08:39:04.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.411+0000 7f4a74999640 1 -- 172.21.15.98:0/698814050 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c0ab9d0 con 0x7f4a70145b70 2024-09-19T08:39:04.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.411+0000 7f4a77bff640 1 -- 172.21.15.98:0/698814050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a70145b70 msgr2=0x7f4a7006bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:04.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.411+0000 7f4a77bff640 1 --2- 172.21.15.98:0/698814050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a70145b70 0x7f4a7006bc90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4a700ccd30 tx=0x7f4a6c01c450 comp rx=0 tx=0).stop 2024-09-19T08:39:04.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.411+0000 7f4a77bff640 1 -- 172.21.15.98:0/698814050 shutdown_connections 2024-09-19T08:39:04.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.411+0000 7f4a77bff640 1 --2- 172.21.15.98:0/698814050 >> v2:172.21.15.139:3300/0 conn(0x7f4a7006c1d0 0x7f4a700713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:04.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.411+0000 7f4a77bff640 1 --2- 172.21.15.98:0/698814050 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a70145b70 0x7f4a7006bc90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:04.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.411+0000 7f4a77bff640 1 -- 172.21.15.98:0/698814050 >> 172.21.15.98:0/698814050 conn(0x7f4a700b2010 msgr2=0x7f4a700b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:04.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.411+0000 7f4a77bff640 1 -- 172.21.15.98:0/698814050 shutdown_connections 2024-09-19T08:39:04.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.411+0000 7f4a77bff640 1 -- 172.21.15.98:0/698814050 wait complete. 2024-09-19T08:39:04.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.411+0000 7f4a77bff640 1 Processor -- start 2024-09-19T08:39:04.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.411+0000 7f4a77bff640 1 -- start start 2024-09-19T08:39:04.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.411+0000 7f4a77bff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a7006c1d0 0x7f4a70140ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:04.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.411+0000 7f4a77bff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a700718f0 0x7f4a701599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:04.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.411+0000 7f4a77bff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a70145b70 0x7f4a70159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:04.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.411+0000 7f4a77bff640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4a701691f0 con 0x7f4a700718f0 2024-09-19T08:39:04.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.411+0000 7f4a77bff640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4a70169070 con 0x7f4a7006c1d0 2024-09-19T08:39:04.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.411+0000 7f4a77bff640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4a70169370 con 0x7f4a70145b70 2024-09-19T08:39:04.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.411+0000 7f4a7519a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a700718f0 0x7f4a701599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:04.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.411+0000 7f4a7519a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a700718f0 0x7f4a701599a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36042/0 (socket says 172.21.15.98:36042) 2024-09-19T08:39:04.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.411+0000 7f4a7519a640 1 -- 172.21.15.98:0/2395365130 learned_addr learned my addr 172.21.15.98:0/2395365130 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:04.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.411+0000 7f4a7599b640 1 --2- 172.21.15.98:0/2395365130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a7006c1d0 0x7f4a70140ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:04.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.411+0000 7f4a7619c640 1 --2- 172.21.15.98:0/2395365130 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a70145b70 0x7f4a70159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:04.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.411+0000 7f4a7519a640 1 -- 172.21.15.98:0/2395365130 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a70145b70 msgr2=0x7f4a70159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:04.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.411+0000 7f4a7519a640 1 --2- 172.21.15.98:0/2395365130 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a70145b70 0x7f4a70159ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:04.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.411+0000 7f4a7519a640 1 -- 172.21.15.98:0/2395365130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a7006c1d0 msgr2=0x7f4a70140ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:04.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.411+0000 7f4a7519a640 1 --2- 172.21.15.98:0/2395365130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a7006c1d0 0x7f4a70140ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:04.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.411+0000 7f4a7519a640 1 -- 172.21.15.98:0/2395365130 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a70135000 con 0x7f4a700718f0 2024-09-19T08:39:04.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.411+0000 7f4a7519a640 1 --2- 172.21.15.98:0/2395365130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a700718f0 0x7f4a701599a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000c7f0 tx=0x7f4a6000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:04.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.411+0000 7f4a667fc640 1 -- 172.21.15.98:0/2395365130 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a60015070 con 0x7f4a700718f0 2024-09-19T08:39:04.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.411+0000 7f4a667fc640 1 -- 172.21.15.98:0/2395365130 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a60004240 con 0x7f4a700718f0 2024-09-19T08:39:04.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.411+0000 7f4a77bff640 1 -- 172.21.15.98:0/2395365130 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a70145350 con 0x7f4a700718f0 2024-09-19T08:39:04.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.411+0000 7f4a667fc640 1 -- 172.21.15.98:0/2395365130 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a60003620 con 0x7f4a700718f0 2024-09-19T08:39:04.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.411+0000 7f4a77bff640 1 -- 172.21.15.98:0/2395365130 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a700cd1c0 con 0x7f4a700718f0 2024-09-19T08:39:04.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.415+0000 7f4a77bff640 1 -- 172.21.15.98:0/2395365130 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a700713b0 con 0x7f4a700718f0 2024-09-19T08:39:04.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.415+0000 7f4a667fc640 1 -- 172.21.15.98:0/2395365130 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4a600039d0 con 0x7f4a700718f0 2024-09-19T08:39:04.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.415+0000 7f4a667fc640 1 --2- 172.21.15.98:0/2395365130 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4a540678c0 0x7f4a54069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:04.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.415+0000 7f4a7599b640 1 --2- 172.21.15.98:0/2395365130 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4a540678c0 0x7f4a54069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:04.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.415+0000 7f4a667fc640 1 -- 172.21.15.98:0/2395365130 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5609+0+0 (secure 0 0 0) 0x7f4a60086140 con 0x7f4a700718f0 2024-09-19T08:39:04.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.415+0000 7f4a7599b640 1 --2- 172.21.15.98:0/2395365130 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4a540678c0 0x7f4a54069d80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c0ad000 tx=0x7f4a6c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:04.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.419+0000 7f4a667fc640 1 -- 172.21.15.98:0/2395365130 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f4a60056d50 con 0x7f4a700718f0 2024-09-19T08:39:04.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:04.623+0000 7f4a77bff640 1 -- 172.21.15.98:0/2395365130 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4a700ccae0 con 0x7f4a700718f0 2024-09-19T08:39:05.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.243+0000 7f4a667fc640 1 -- 172.21.15.98:0/2395365130 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) ==== 172+0+0 (secure 0 0 0) 0x7f4a60056650 con 0x7f4a700718f0 2024-09-19T08:39:05.247 INFO:teuthology.orchestra.run.smithi098.stderr:new fs with metadata pool 4 and data pool 5 2024-09-19T08:39:05.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.251+0000 7f4a77bff640 1 -- 172.21.15.98:0/2395365130 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4a540678c0 msgr2=0x7f4a54069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:05.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.251+0000 7f4a77bff640 1 --2- 172.21.15.98:0/2395365130 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4a540678c0 0x7f4a54069d80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c0ad000 tx=0x7f4a6c008040 comp rx=0 tx=0).stop 2024-09-19T08:39:05.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.251+0000 7f4a77bff640 1 -- 172.21.15.98:0/2395365130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a700718f0 msgr2=0x7f4a701599a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:05.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.251+0000 7f4a77bff640 1 --2- 172.21.15.98:0/2395365130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a700718f0 0x7f4a701599a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000c7f0 tx=0x7f4a6000ccb0 comp rx=0 tx=0).stop 2024-09-19T08:39:05.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.251+0000 7f4a77bff640 1 -- 172.21.15.98:0/2395365130 shutdown_connections 2024-09-19T08:39:05.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.251+0000 7f4a77bff640 1 --2- 172.21.15.98:0/2395365130 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4a540678c0 0x7f4a54069d80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:05.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.251+0000 7f4a77bff640 1 --2- 172.21.15.98:0/2395365130 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a70145b70 0x7f4a70159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:05.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.251+0000 7f4a77bff640 1 --2- 172.21.15.98:0/2395365130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a700718f0 0x7f4a701599a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:05.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.251+0000 7f4a77bff640 1 --2- 172.21.15.98:0/2395365130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a7006c1d0 0x7f4a70140ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:05.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.251+0000 7f4a77bff640 1 -- 172.21.15.98:0/2395365130 >> 172.21.15.98:0/2395365130 conn(0x7f4a700b2010 msgr2=0x7f4a7015fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:05.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.251+0000 7f4a77bff640 1 -- 172.21.15.98:0/2395365130 shutdown_connections 2024-09-19T08:39:05.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.251+0000 7f4a77bff640 1 -- 172.21.15.98:0/2395365130 wait complete. 2024-09-19T08:39:05.296 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:39:05.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.407+0000 7f9d6ff99640 1 Processor -- start 2024-09-19T08:39:05.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.407+0000 7f9d6ff99640 1 -- start start 2024-09-19T08:39:05.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.407+0000 7f9d6ff99640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9d680733b0 0x7f9d680cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:05.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.407+0000 7f9d6ff99640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d6806d310 con 0x7f9d6815c550 2024-09-19T08:39:05.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.407+0000 7f9d6ff99640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d6806d5f0 con 0x7f9d6806c3a0 2024-09-19T08:39:05.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.407+0000 7f9d6ff99640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f9d6806d8d0 con 0x7f9d680733b0 2024-09-19T08:39:05.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.407+0000 7f9d6dd35640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f9d6815c550 0x7f9d6806bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:55446/0 (socket says 172.21.15.98:55446) 2024-09-19T08:39:05.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.407+0000 7f9d6dd35640 1 -- 172.21.15.98:0/1888442980 learned_addr learned my addr 172.21.15.98:0/1888442980 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:05.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.407+0000 7f9d6e536640 1 --2- 172.21.15.98:0/1888442980 >> v2:172.21.15.139:3300/0 conn(0x7f9d680733b0 0x7f9d680cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:05.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.407+0000 7f9d6cd33640 1 -- 172.21.15.98:0/1888442980 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3977335403 0 0) 0x7f9d6806d310 con 0x7f9d6815c550 2024-09-19T08:39:05.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.407+0000 7f9d6cd33640 1 -- 172.21.15.98:0/1888442980 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d54003530 con 0x7f9d6815c550 2024-09-19T08:39:05.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.407+0000 7f9d6cd33640 1 -- 172.21.15.98:0/1888442980 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 503016849 0 0) 0x7f9d54003530 con 0x7f9d6815c550 2024-09-19T08:39:05.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.407+0000 7f9d6cd33640 1 -- 172.21.15.98:0/1888442980 >> v2:172.21.15.139:3300/0 conn(0x7f9d680733b0 msgr2=0x7f9d680cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:05.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.407+0000 7f9d6cd33640 1 --2- 172.21.15.98:0/1888442980 >> v2:172.21.15.139:3300/0 conn(0x7f9d680733b0 0x7f9d680cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:05.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.407+0000 7f9d6cd33640 1 -- 172.21.15.98:0/1888442980 >> v1:172.21.15.139:6789/0 conn(0x7f9d6806c3a0 legacy=0x7f9d680cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:05.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.407+0000 7f9d6cd33640 1 -- 172.21.15.98:0/1888442980 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d6806dbb0 con 0x7f9d6815c550 2024-09-19T08:39:05.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.407+0000 7f9d6cd33640 1 -- 172.21.15.98:0/1888442980 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f9d64001680 con 0x7f9d6815c550 2024-09-19T08:39:05.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.407+0000 7f9d6cd33640 1 -- 172.21.15.98:0/1888442980 >> v1:172.21.15.98:6789/0 conn(0x7f9d6815c550 legacy=0x7f9d6806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:05.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.407+0000 7f9d6cd33640 1 --2- 172.21.15.98:0/1888442980 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9d540039c0 0x7f9d54003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:05.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.407+0000 7f9d6cd33640 1 --2- 172.21.15.98:0/1888442980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d54004810 0x7f9d54004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:05.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.407+0000 7f9d6cd33640 1 --2- 172.21.15.98:0/1888442980 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d540051d0 0x7f9d5400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:05.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.407+0000 7f9d6cd33640 1 -- 172.21.15.98:0/1888442980 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6806d8d0 con 0x7f9d54004810 2024-09-19T08:39:05.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.407+0000 7f9d6cd33640 1 -- 172.21.15.98:0/1888442980 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9d5400c070 con 0x7f9d540051d0 2024-09-19T08:39:05.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.407+0000 7f9d6cd33640 1 -- 172.21.15.98:0/1888442980 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9d5400c270 con 0x7f9d540039c0 2024-09-19T08:39:05.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.407+0000 7f9d6d534640 1 --2- 172.21.15.98:0/1888442980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d54004810 0x7f9d54004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:05.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.407+0000 7f9d6dd35640 1 --2- 172.21.15.98:0/1888442980 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9d540039c0 0x7f9d54003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:05.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.407+0000 7f9d6d534640 1 -- 172.21.15.98:0/1888442980 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9d540039c0 msgr2=0x7f9d54003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:05.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.407+0000 7f9d6d534640 1 --2- 172.21.15.98:0/1888442980 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9d540039c0 0x7f9d54003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:05.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.407+0000 7f9d6d534640 1 -- 172.21.15.98:0/1888442980 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d540051d0 msgr2=0x7f9d5400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:05.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.407+0000 7f9d6d534640 1 --2- 172.21.15.98:0/1888442980 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d540051d0 0x7f9d5400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:05.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.407+0000 7f9d6d534640 1 -- 172.21.15.98:0/1888442980 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d54003530 con 0x7f9d54004810 2024-09-19T08:39:05.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.407+0000 7f9d6e536640 1 -- 172.21.15.98:0/1888442980 reap_dead start 2024-09-19T08:39:05.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.411+0000 7f9d6d534640 1 --2- 172.21.15.98:0/1888442980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d54004810 0x7f9d54004c90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9d58001780 tx=0x7f9d5801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc9d509985baa788 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:05.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.411+0000 7f9d6cd33640 1 -- 172.21.15.98:0/1888442980 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d580013a0 con 0x7f9d54004810 2024-09-19T08:39:05.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.411+0000 7f9d6cd33640 1 -- 172.21.15.98:0/1888442980 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d580052f0 con 0x7f9d54004810 2024-09-19T08:39:05.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.411+0000 7f9d6cd33640 1 -- 172.21.15.98:0/1888442980 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d58003840 con 0x7f9d54004810 2024-09-19T08:39:05.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.411+0000 7f9d6ff99640 1 -- 172.21.15.98:0/1888442980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d54004810 msgr2=0x7f9d54004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:05.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.411+0000 7f9d6ff99640 1 --2- 172.21.15.98:0/1888442980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d54004810 0x7f9d54004c90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9d58001780 tx=0x7f9d5801c400 comp rx=0 tx=0).stop 2024-09-19T08:39:05.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.411+0000 7f9d6ff99640 1 -- 172.21.15.98:0/1888442980 shutdown_connections 2024-09-19T08:39:05.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.411+0000 7f9d6ff99640 1 --2- 172.21.15.98:0/1888442980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d54004810 0x7f9d54004c90 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:05.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.411+0000 7f9d6ff99640 1 -- 172.21.15.98:0/1888442980 >> 172.21.15.98:0/1888442980 conn(0x7f9d680b2010 msgr2=0x7f9d680b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:05.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.411+0000 7f9d6ff99640 1 -- 172.21.15.98:0/1888442980 shutdown_connections 2024-09-19T08:39:05.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.411+0000 7f9d6ff99640 1 -- 172.21.15.98:0/1888442980 wait complete. 2024-09-19T08:39:05.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.411+0000 7f9d6ff99640 1 Processor -- start 2024-09-19T08:39:05.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.411+0000 7f9d6ff99640 1 -- start start 2024-09-19T08:39:05.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.415+0000 7f9d6ff99640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d54004810 0x7f9d680c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:05.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.415+0000 7f9d6ff99640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9d681f4cf0 0x7f9d6815f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:05.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.415+0000 7f9d6ff99640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d680aeb10 0x7f9d68156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:05.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.415+0000 7f9d6ff99640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6806bc90 con 0x7f9d54004810 2024-09-19T08:39:05.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.415+0000 7f9d6ff99640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6815cb40 con 0x7f9d680aeb10 2024-09-19T08:39:05.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.415+0000 7f9d6ff99640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9d6806be10 con 0x7f9d681f4cf0 2024-09-19T08:39:05.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.415+0000 7f9d6dd35640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d54004810 0x7f9d680c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:05.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.415+0000 7f9d6dd35640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d54004810 0x7f9d680c5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36074/0 (socket says 172.21.15.98:36074) 2024-09-19T08:39:05.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.415+0000 7f9d6dd35640 1 -- 172.21.15.98:0/130162064 learned_addr learned my addr 172.21.15.98:0/130162064 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:05.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.415+0000 7f9d6d534640 1 --2- 172.21.15.98:0/130162064 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9d681f4cf0 0x7f9d6815f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:05.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.415+0000 7f9d6e536640 1 --2- 172.21.15.98:0/130162064 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d680aeb10 0x7f9d68156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:05.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.415+0000 7f9d6dd35640 1 -- 172.21.15.98:0/130162064 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9d681f4cf0 msgr2=0x7f9d6815f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:05.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.415+0000 7f9d6dd35640 1 --2- 172.21.15.98:0/130162064 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9d681f4cf0 0x7f9d6815f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:05.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.415+0000 7f9d6dd35640 1 -- 172.21.15.98:0/130162064 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d680aeb10 msgr2=0x7f9d68156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:05.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.415+0000 7f9d6dd35640 1 --2- 172.21.15.98:0/130162064 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d680aeb10 0x7f9d68156b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:05.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.415+0000 7f9d6dd35640 1 -- 172.21.15.98:0/130162064 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d681f5170 con 0x7f9d54004810 2024-09-19T08:39:05.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.415+0000 7f9d6dd35640 1 --2- 172.21.15.98:0/130162064 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d54004810 0x7f9d680c5fa0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9d64011950 tx=0x7f9d64011e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:05.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.415+0000 7f9d5e7fc640 1 -- 172.21.15.98:0/130162064 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d64010040 con 0x7f9d54004810 2024-09-19T08:39:05.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.415+0000 7f9d5e7fc640 1 -- 172.21.15.98:0/130162064 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d6401a070 con 0x7f9d54004810 2024-09-19T08:39:05.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.415+0000 7f9d6ff99640 1 -- 172.21.15.98:0/130162064 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d6815fcf0 con 0x7f9d54004810 2024-09-19T08:39:05.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.415+0000 7f9d5e7fc640 1 -- 172.21.15.98:0/130162064 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6400fe40 con 0x7f9d54004810 2024-09-19T08:39:05.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.415+0000 7f9d6ff99640 1 -- 172.21.15.98:0/130162064 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d680c6980 con 0x7f9d54004810 2024-09-19T08:39:05.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.415+0000 7f9d6ff99640 1 -- 172.21.15.98:0/130162064 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d680cf3c0 con 0x7f9d54004810 2024-09-19T08:39:05.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.415+0000 7f9d5e7fc640 1 -- 172.21.15.98:0/130162064 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9d6401f460 con 0x7f9d54004810 2024-09-19T08:39:05.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.415+0000 7f9d5e7fc640 1 --2- 172.21.15.98:0/130162064 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9d4c0678f0 0x7f9d4c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:05.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.415+0000 7f9d5e7fc640 1 -- 172.21.15.98:0/130162064 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f9d64088ef0 con 0x7f9d54004810 2024-09-19T08:39:05.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.415+0000 7f9d6d534640 1 --2- 172.21.15.98:0/130162064 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9d4c0678f0 0x7f9d4c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:05.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.415+0000 7f9d6d534640 1 --2- 172.21.15.98:0/130162064 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9d4c0678f0 0x7f9d4c069db0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9d58001750 tx=0x7f9d580011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:05.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.419+0000 7f9d5e7fc640 1 -- 172.21.15.98:0/130162064 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9d64059af0 con 0x7f9d54004810 2024-09-19T08:39:05.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.587+0000 7f9d6ff99640 1 -- 172.21.15.98:0/130162064 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9d680afb80 con 0x7f9d54004810 2024-09-19T08:39:05.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.587+0000 7f9d5e7fc640 1 -- 172.21.15.98:0/130162064 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16742 (secure 0 0 0) 0x7f9d640593f0 con 0x7f9d54004810 2024-09-19T08:39:05.591 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:39:05.591 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":31,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:39:04.632086+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:39:02.429044+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-19T08:39:03.620637+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:39:05.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.591+0000 7f9d6ff99640 1 -- 172.21.15.98:0/130162064 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9d4c0678f0 msgr2=0x7f9d4c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:05.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.591+0000 7f9d6ff99640 1 --2- 172.21.15.98:0/130162064 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9d4c0678f0 0x7f9d4c069db0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9d58001750 tx=0x7f9d580011a0 comp rx=0 tx=0).stop 2024-09-19T08:39:05.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.591+0000 7f9d6ff99640 1 -- 172.21.15.98:0/130162064 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d54004810 msgr2=0x7f9d680c5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:05.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.591+0000 7f9d6ff99640 1 --2- 172.21.15.98:0/130162064 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d54004810 0x7f9d680c5fa0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9d64011950 tx=0x7f9d64011e10 comp rx=0 tx=0).stop 2024-09-19T08:39:05.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.591+0000 7f9d6ff99640 1 -- 172.21.15.98:0/130162064 shutdown_connections 2024-09-19T08:39:05.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.591+0000 7f9d6ff99640 1 --2- 172.21.15.98:0/130162064 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9d4c0678f0 0x7f9d4c069db0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:05.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.591+0000 7f9d6ff99640 1 --2- 172.21.15.98:0/130162064 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d680aeb10 0x7f9d68156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:05.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.591+0000 7f9d6ff99640 1 --2- 172.21.15.98:0/130162064 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9d681f4cf0 0x7f9d6815f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:05.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.591+0000 7f9d6ff99640 1 --2- 172.21.15.98:0/130162064 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d54004810 0x7f9d680c5fa0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:05.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.591+0000 7f9d6ff99640 1 -- 172.21.15.98:0/130162064 >> 172.21.15.98:0/130162064 conn(0x7f9d680b2010 msgr2=0x7f9d6815a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:05.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.591+0000 7f9d6ff99640 1 -- 172.21.15.98:0/130162064 shutdown_connections 2024-09-19T08:39:05.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.591+0000 7f9d6ff99640 1 -- 172.21.15.98:0/130162064 wait complete. 2024-09-19T08:39:05.610 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:39:05.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.719+0000 7faa180d8640 1 Processor -- start 2024-09-19T08:39:05.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.719+0000 7faa180d8640 1 -- start start 2024-09-19T08:39:05.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.719+0000 7faa180d8640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7faa100d4820 0x7faa101621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:05.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.719+0000 7faa180d8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa100b9970 0x7faa100b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:05.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.719+0000 7faa180d8640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa10068440 con 0x7faa100b7f80 2024-09-19T08:39:05.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.719+0000 7faa180d8640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7faa10163050 con 0x7faa100d4820 2024-09-19T08:39:05.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.719+0000 7faa180d8640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7faa10162a90 con 0x7faa100b9970 2024-09-19T08:39:05.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.719+0000 7faa16675640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa100b9970 0x7faa100b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:05.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.719+0000 7faa16675640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa100b9970 0x7faa100b9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:57600/0 (socket says 172.21.15.98:57600) 2024-09-19T08:39:05.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.719+0000 7faa16675640 1 -- 172.21.15.98:0/2124124904 learned_addr learned my addr 172.21.15.98:0/2124124904 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:05.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.719+0000 7faa15673640 1 --2- 172.21.15.98:0/2124124904 >> v2:172.21.15.139:3300/0 conn(0x7faa100d4820 0x7faa101621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:05.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.719+0000 7faa16675640 1 -- 172.21.15.98:0/2124124904 >> v2:172.21.15.139:3300/0 conn(0x7faa100d4820 msgr2=0x7faa101621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:05.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.719+0000 7faa16675640 1 --2- 172.21.15.98:0/2124124904 >> v2:172.21.15.139:3300/0 conn(0x7faa100d4820 0x7faa101621b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:05.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.719+0000 7faa16675640 1 -- 172.21.15.98:0/2124124904 >> v1:172.21.15.139:6789/0 conn(0x7faa100b7f80 legacy=0x7faa1015f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:05.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.719+0000 7faa16675640 1 -- 172.21.15.98:0/2124124904 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa10162d70 con 0x7faa100b9970 2024-09-19T08:39:05.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.723+0000 7faa16675640 1 --2- 172.21.15.98:0/2124124904 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa100b9970 0x7faa100b9de0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7faa00009870 tx=0x7faa0001c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b753f070f257ce6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:05.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.723+0000 7faa14e72640 1 -- 172.21.15.98:0/2124124904 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa000a7070 con 0x7faa100b9970 2024-09-19T08:39:05.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.723+0000 7faa14e72640 1 -- 172.21.15.98:0/2124124904 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa000b0070 con 0x7faa100b9970 2024-09-19T08:39:05.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.723+0000 7faa14e72640 1 -- 172.21.15.98:0/2124124904 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa000ab890 con 0x7faa100b9970 2024-09-19T08:39:05.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.723+0000 7faa180d8640 1 -- 172.21.15.98:0/2124124904 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa100b9970 msgr2=0x7faa100b9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:05.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.723+0000 7faa180d8640 1 --2- 172.21.15.98:0/2124124904 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa100b9970 0x7faa100b9de0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7faa00009870 tx=0x7faa0001c560 comp rx=0 tx=0).stop 2024-09-19T08:39:05.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.723+0000 7faa180d8640 1 -- 172.21.15.98:0/2124124904 shutdown_connections 2024-09-19T08:39:05.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.723+0000 7faa180d8640 1 --2- 172.21.15.98:0/2124124904 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa100b9970 0x7faa100b9de0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:05.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.723+0000 7faa180d8640 1 --2- 172.21.15.98:0/2124124904 >> v2:172.21.15.139:3300/0 conn(0x7faa100d4820 0x7faa101621b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:05.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.723+0000 7faa180d8640 1 -- 172.21.15.98:0/2124124904 >> 172.21.15.98:0/2124124904 conn(0x7faa10074e40 msgr2=0x7faa10075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:05.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.723+0000 7faa180d8640 1 -- 172.21.15.98:0/2124124904 shutdown_connections 2024-09-19T08:39:05.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.723+0000 7faa180d8640 1 -- 172.21.15.98:0/2124124904 wait complete. 2024-09-19T08:39:05.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.727+0000 7faa180d8640 1 Processor -- start 2024-09-19T08:39:05.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.727+0000 7faa180d8640 1 -- start start 2024-09-19T08:39:05.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.727+0000 7faa180d8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa100b7f80 0x7faa1013b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:05.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.727+0000 7faa180d8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa100b9970 0x7faa1013e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:05.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.727+0000 7faa180d8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa100d4820 0x7faa1013a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:05.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.727+0000 7faa180d8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faa100b84e0 con 0x7faa100b7f80 2024-09-19T08:39:05.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.727+0000 7faa180d8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faa100b8360 con 0x7faa100d4820 2024-09-19T08:39:05.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.727+0000 7faa180d8640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7faa10144590 con 0x7faa100b9970 2024-09-19T08:39:05.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.727+0000 7faa15e74640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa100b7f80 0x7faa1013b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:05.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.727+0000 7faa15e74640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa100b7f80 0x7faa1013b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36088/0 (socket says 172.21.15.98:36088) 2024-09-19T08:39:05.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.727+0000 7faa15e74640 1 -- 172.21.15.98:0/1913278039 learned_addr learned my addr 172.21.15.98:0/1913278039 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:05.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.727+0000 7faa15673640 1 --2- 172.21.15.98:0/1913278039 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa100b9970 0x7faa1013e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:05.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.727+0000 7faa16675640 1 --2- 172.21.15.98:0/1913278039 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa100d4820 0x7faa1013a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:05.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.727+0000 7faa15e74640 1 -- 172.21.15.98:0/1913278039 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa100b9970 msgr2=0x7faa1013e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:05.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.727+0000 7faa15e74640 1 --2- 172.21.15.98:0/1913278039 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa100b9970 0x7faa1013e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:05.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.727+0000 7faa15e74640 1 -- 172.21.15.98:0/1913278039 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa100d4820 msgr2=0x7faa1013a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:05.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.727+0000 7faa15e74640 1 --2- 172.21.15.98:0/1913278039 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa100d4820 0x7faa1013a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:05.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.727+0000 7faa15e74640 1 -- 172.21.15.98:0/1913278039 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa100d3260 con 0x7faa100b7f80 2024-09-19T08:39:05.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.727+0000 7faa15e74640 1 --2- 172.21.15.98:0/1913278039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa100b7f80 0x7faa1013b0d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7faa0800c730 tx=0x7faa0800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:05.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.727+0000 7faa0e7fc640 1 -- 172.21.15.98:0/1913278039 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa08003ba0 con 0x7faa100b7f80 2024-09-19T08:39:05.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.727+0000 7faa0e7fc640 1 -- 172.21.15.98:0/1913278039 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa08003d60 con 0x7faa100b7f80 2024-09-19T08:39:05.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.727+0000 7faa180d8640 1 -- 172.21.15.98:0/1913278039 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa100d3540 con 0x7faa100b7f80 2024-09-19T08:39:05.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.727+0000 7faa0e7fc640 1 -- 172.21.15.98:0/1913278039 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0800aa30 con 0x7faa100b7f80 2024-09-19T08:39:05.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.727+0000 7faa180d8640 1 -- 172.21.15.98:0/1913278039 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa100d3b00 con 0x7faa100b7f80 2024-09-19T08:39:05.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.727+0000 7faa180d8640 1 -- 172.21.15.98:0/1913278039 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa100b9de0 con 0x7faa100b7f80 2024-09-19T08:39:05.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.727+0000 7faa0e7fc640 1 -- 172.21.15.98:0/1913278039 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7faa0800abd0 con 0x7faa100b7f80 2024-09-19T08:39:05.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.731+0000 7faa0e7fc640 1 --2- 172.21.15.98:0/1913278039 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa9fc0678c0 0x7fa9fc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:05.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.731+0000 7faa0e7fc640 1 -- 172.21.15.98:0/1913278039 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7faa08085a90 con 0x7faa100b7f80 2024-09-19T08:39:05.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.731+0000 7faa15673640 1 --2- 172.21.15.98:0/1913278039 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa9fc0678c0 0x7fa9fc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:05.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.731+0000 7faa15673640 1 --2- 172.21.15.98:0/1913278039 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa9fc0678c0 0x7fa9fc069d80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7faa10160710 tx=0x7faa04008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:05.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.735+0000 7faa0e7fc640 1 -- 172.21.15.98:0/1913278039 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7faa08056660 con 0x7faa100b7f80 2024-09-19T08:39:05.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.895+0000 7faa180d8640 1 -- 172.21.15.98:0/1913278039 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faa10067ce0 con 0x7faa100b7f80 2024-09-19T08:39:05.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.899+0000 7faa0e7fc640 1 -- 172.21.15.98:0/1913278039 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16742 (secure 0 0 0) 0x7faa08055f60 con 0x7faa100b7f80 2024-09-19T08:39:05.902 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:39:05.903 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":31,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:39:04.632086+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:39:02.429044+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-19T08:39:03.620637+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:39:05.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.899+0000 7faa180d8640 1 -- 172.21.15.98:0/1913278039 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa9fc0678c0 msgr2=0x7fa9fc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:05.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.899+0000 7faa180d8640 1 --2- 172.21.15.98:0/1913278039 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa9fc0678c0 0x7fa9fc069d80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7faa10160710 tx=0x7faa04008040 comp rx=0 tx=0).stop 2024-09-19T08:39:05.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.899+0000 7faa180d8640 1 -- 172.21.15.98:0/1913278039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa100b7f80 msgr2=0x7faa1013b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:05.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.899+0000 7faa180d8640 1 --2- 172.21.15.98:0/1913278039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa100b7f80 0x7faa1013b0d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7faa0800c730 tx=0x7faa0800cbf0 comp rx=0 tx=0).stop 2024-09-19T08:39:05.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.899+0000 7faa180d8640 1 -- 172.21.15.98:0/1913278039 shutdown_connections 2024-09-19T08:39:05.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.899+0000 7faa180d8640 1 --2- 172.21.15.98:0/1913278039 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa9fc0678c0 0x7fa9fc069d80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:05.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.899+0000 7faa180d8640 1 --2- 172.21.15.98:0/1913278039 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa100d4820 0x7faa1013a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:05.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.899+0000 7faa180d8640 1 --2- 172.21.15.98:0/1913278039 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa100b9970 0x7faa1013e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:05.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.899+0000 7faa180d8640 1 --2- 172.21.15.98:0/1913278039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa100b7f80 0x7faa1013b0d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:05.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.899+0000 7faa180d8640 1 -- 172.21.15.98:0/1913278039 >> 172.21.15.98:0/1913278039 conn(0x7faa10074e40 msgr2=0x7faa1015fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:05.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.899+0000 7faa180d8640 1 -- 172.21.15.98:0/1913278039 shutdown_connections 2024-09-19T08:39:05.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:05.899+0000 7faa180d8640 1 -- 172.21.15.98:0/1913278039 wait complete. 2024-09-19T08:39:05.922 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-19T08:39:06.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.031+0000 7fb88d892640 1 Processor -- start 2024-09-19T08:39:06.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.031+0000 7fb88d892640 1 -- start start 2024-09-19T08:39:06.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.035+0000 7fb88d892640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb888144d10 0x7fb8880722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:06.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.035+0000 7fb88d892640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb888072820 0x7fb88806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:06.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.035+0000 7fb88d892640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb888072ea0 con 0x7fb88806c1d0 2024-09-19T08:39:06.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.035+0000 7fb88d892640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fb888072c60 con 0x7fb888144d10 2024-09-19T08:39:06.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.035+0000 7fb88d892640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb888073ab0 con 0x7fb888072820 2024-09-19T08:39:06.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.035+0000 7fb8867fc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb888072820 0x7fb88806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:06.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.035+0000 7fb8867fc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb888072820 0x7fb88806bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:57616/0 (socket says 172.21.15.98:57616) 2024-09-19T08:39:06.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.035+0000 7fb8867fc640 1 -- 172.21.15.98:0/2560878296 learned_addr learned my addr 172.21.15.98:0/2560878296 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:06.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.035+0000 7fb886ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb888144d10 0x7fb8880722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:06.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.035+0000 7fb8867fc640 1 -- 172.21.15.98:0/2560878296 >> v2:172.21.15.139:3300/0 conn(0x7fb888144d10 msgr2=0x7fb8880722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:06.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.035+0000 7fb8867fc640 1 --2- 172.21.15.98:0/2560878296 >> v2:172.21.15.139:3300/0 conn(0x7fb888144d10 0x7fb8880722e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:06.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.035+0000 7fb8867fc640 1 -- 172.21.15.98:0/2560878296 >> v1:172.21.15.139:6789/0 conn(0x7fb88806c1d0 legacy=0x7fb8880b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:06.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.035+0000 7fb8867fc640 1 -- 172.21.15.98:0/2560878296 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb888073180 con 0x7fb888072820 2024-09-19T08:39:06.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.035+0000 7fb8867fc640 1 --2- 172.21.15.98:0/2560878296 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb888072820 0x7fb88806bc90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb87c00bb10 tx=0x7fb87c01ec30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29c6e4f65aa900e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:06.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.039+0000 7fb885ffb640 1 -- 172.21.15.98:0/2560878296 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87c0b4070 con 0x7fb888072820 2024-09-19T08:39:06.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.039+0000 7fb885ffb640 1 -- 172.21.15.98:0/2560878296 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb87c0a7b00 con 0x7fb888072820 2024-09-19T08:39:06.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.039+0000 7fb885ffb640 1 -- 172.21.15.98:0/2560878296 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87c0af440 con 0x7fb888072820 2024-09-19T08:39:06.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.039+0000 7fb88d892640 1 -- 172.21.15.98:0/2560878296 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb888072820 msgr2=0x7fb88806bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:06.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.039+0000 7fb88d892640 1 --2- 172.21.15.98:0/2560878296 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb888072820 0x7fb88806bc90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb87c00bb10 tx=0x7fb87c01ec30 comp rx=0 tx=0).stop 2024-09-19T08:39:06.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.039+0000 7fb88d892640 1 -- 172.21.15.98:0/2560878296 shutdown_connections 2024-09-19T08:39:06.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.039+0000 7fb88d892640 1 --2- 172.21.15.98:0/2560878296 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb888072820 0x7fb88806bc90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:06.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.039+0000 7fb88d892640 1 --2- 172.21.15.98:0/2560878296 >> v2:172.21.15.139:3300/0 conn(0x7fb888144d10 0x7fb8880722e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:06.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.039+0000 7fb88d892640 1 -- 172.21.15.98:0/2560878296 >> 172.21.15.98:0/2560878296 conn(0x7fb8880b2010 msgr2=0x7fb8880b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:06.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.039+0000 7fb88d892640 1 -- 172.21.15.98:0/2560878296 shutdown_connections 2024-09-19T08:39:06.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.039+0000 7fb88d892640 1 -- 172.21.15.98:0/2560878296 wait complete. 2024-09-19T08:39:06.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.039+0000 7fb88d892640 1 Processor -- start 2024-09-19T08:39:06.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.039+0000 7fb88d892640 1 -- start start 2024-09-19T08:39:06.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.039+0000 7fb88d892640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb88806c1d0 0x7fb888157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:06.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.039+0000 7fb88d892640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb888072820 0x7fb88813f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:06.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.039+0000 7fb88d892640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb888144d10 0x7fb88813fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:06.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.039+0000 7fb88d892640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb8880b3590 con 0x7fb888072820 2024-09-19T08:39:06.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.039+0000 7fb88d892640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb8880b3410 con 0x7fb88806c1d0 2024-09-19T08:39:06.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.039+0000 7fb88d892640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb8880b3710 con 0x7fb888144d10 2024-09-19T08:39:06.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.039+0000 7fb8867fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb888072820 0x7fb88813f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:06.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.039+0000 7fb8867fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb888072820 0x7fb88813f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36096/0 (socket says 172.21.15.98:36096) 2024-09-19T08:39:06.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.039+0000 7fb8867fc640 1 -- 172.21.15.98:0/1470020650 learned_addr learned my addr 172.21.15.98:0/1470020650 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:06.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.039+0000 7fb886ffd640 1 --2- 172.21.15.98:0/1470020650 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb88806c1d0 0x7fb888157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:06.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.039+0000 7fb8877fe640 1 --2- 172.21.15.98:0/1470020650 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb888144d10 0x7fb88813fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:06.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.039+0000 7fb8867fc640 1 -- 172.21.15.98:0/1470020650 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb888144d10 msgr2=0x7fb88813fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:06.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.039+0000 7fb8867fc640 1 --2- 172.21.15.98:0/1470020650 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb888144d10 0x7fb88813fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:06.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.039+0000 7fb8867fc640 1 -- 172.21.15.98:0/1470020650 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb88806c1d0 msgr2=0x7fb888157eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:06.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.039+0000 7fb8867fc640 1 --2- 172.21.15.98:0/1470020650 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb88806c1d0 0x7fb888157eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:06.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.039+0000 7fb8867fc640 1 -- 172.21.15.98:0/1470020650 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb88813e2c0 con 0x7fb888072820 2024-09-19T08:39:06.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.039+0000 7fb8877fe640 1 --2- 172.21.15.98:0/1470020650 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb888144d10 0x7fb88813fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:06.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.039+0000 7fb8867fc640 1 --2- 172.21.15.98:0/1470020650 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb888072820 0x7fb88813f590 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb87c01d2e0 tx=0x7fb87c01b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:06.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.039+0000 7fb86ffff640 1 -- 172.21.15.98:0/1470020650 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87c0b4040 con 0x7fb888072820 2024-09-19T08:39:06.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.039+0000 7fb86ffff640 1 -- 172.21.15.98:0/1470020650 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb87c0bf070 con 0x7fb888072820 2024-09-19T08:39:06.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.039+0000 7fb86ffff640 1 -- 172.21.15.98:0/1470020650 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87c0ba660 con 0x7fb888072820 2024-09-19T08:39:06.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.039+0000 7fb88d892640 1 -- 172.21.15.98:0/1470020650 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb88813e5a0 con 0x7fb888072820 2024-09-19T08:39:06.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.039+0000 7fb88d892640 1 -- 172.21.15.98:0/1470020650 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb88813eb60 con 0x7fb888072820 2024-09-19T08:39:06.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.043+0000 7fb88d892640 1 -- 172.21.15.98:0/1470020650 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb88806bc90 con 0x7fb888072820 2024-09-19T08:39:06.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.043+0000 7fb86ffff640 1 -- 172.21.15.98:0/1470020650 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb87c0ba800 con 0x7fb888072820 2024-09-19T08:39:06.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.043+0000 7fb86ffff640 1 --2- 172.21.15.98:0/1470020650 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb8740678c0 0x7fb874069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:06.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.043+0000 7fb86ffff640 1 -- 172.21.15.98:0/1470020650 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fb87c1231f0 con 0x7fb888072820 2024-09-19T08:39:06.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.043+0000 7fb886ffd640 1 --2- 172.21.15.98:0/1470020650 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb8740678c0 0x7fb874069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:06.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.043+0000 7fb886ffd640 1 --2- 172.21.15.98:0/1470020650 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb8740678c0 0x7fb874069d80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb878009a50 tx=0x7fb878008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:06.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.047+0000 7fb86ffff640 1 -- 172.21.15.98:0/1470020650 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb87c018820 con 0x7fb888072820 2024-09-19T08:39:06.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.231+0000 7fb88d892640 1 -- 172.21.15.98:0/1470020650 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb88813d470 con 0x7fb888072820 2024-09-19T08:39:06.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.263+0000 7fb86ffff640 1 -- 172.21.15.98:0/1470020650 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v13) ==== 118+0+0 (secure 0 0 0) 0x7fb87c0ade20 con 0x7fb888072820 2024-09-19T08:39:06.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.267+0000 7fb88d892640 1 -- 172.21.15.98:0/1470020650 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb8740678c0 msgr2=0x7fb874069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:06.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.267+0000 7fb88d892640 1 --2- 172.21.15.98:0/1470020650 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb8740678c0 0x7fb874069d80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb878009a50 tx=0x7fb878008040 comp rx=0 tx=0).stop 2024-09-19T08:39:06.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.267+0000 7fb88d892640 1 -- 172.21.15.98:0/1470020650 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb888072820 msgr2=0x7fb88813f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:06.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.267+0000 7fb88d892640 1 --2- 172.21.15.98:0/1470020650 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb888072820 0x7fb88813f590 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb87c01d2e0 tx=0x7fb87c01b9d0 comp rx=0 tx=0).stop 2024-09-19T08:39:06.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.267+0000 7fb88d892640 1 -- 172.21.15.98:0/1470020650 shutdown_connections 2024-09-19T08:39:06.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.267+0000 7fb88d892640 1 --2- 172.21.15.98:0/1470020650 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb8740678c0 0x7fb874069d80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:06.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.267+0000 7fb88d892640 1 --2- 172.21.15.98:0/1470020650 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb888144d10 0x7fb88813fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:06.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.267+0000 7fb88d892640 1 --2- 172.21.15.98:0/1470020650 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb888072820 0x7fb88813f590 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:06.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.267+0000 7fb88d892640 1 --2- 172.21.15.98:0/1470020650 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb88806c1d0 0x7fb888157eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:06.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.267+0000 7fb88d892640 1 -- 172.21.15.98:0/1470020650 >> 172.21.15.98:0/1470020650 conn(0x7fb8880b2010 msgr2=0x7fb8880b0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:06.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.267+0000 7fb88d892640 1 -- 172.21.15.98:0/1470020650 shutdown_connections 2024-09-19T08:39:06.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.267+0000 7fb88d892640 1 -- 172.21.15.98:0/1470020650 wait complete. 2024-09-19T08:39:06.287 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-19T08:39:06.287 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-19T08:39:06.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.395+0000 7fc8f4d14640 1 Processor -- start 2024-09-19T08:39:06.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.395+0000 7fc8f4d14640 1 -- start start 2024-09-19T08:39:06.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.395+0000 7fc8f4d14640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc8f0145b70 0x7fc8f006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:06.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.395+0000 7fc8f4d14640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc8f0071ac0 0x7fc8f00b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:06.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.395+0000 7fc8f4d14640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8f0073530 con 0x7fc8f006c1d0 2024-09-19T08:39:06.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.395+0000 7fc8f4d14640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fc8f0071f10 con 0x7fc8f0145b70 2024-09-19T08:39:06.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.395+0000 7fc8f4d14640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fc8f006c620 con 0x7fc8f0071ac0 2024-09-19T08:39:06.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.395+0000 7fc8ee59c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc8f0145b70 0x7fc8f006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:06.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.395+0000 7fc8edd9b640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fc8f006c1d0 0x7fc8f00713b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:55458/0 (socket says 172.21.15.98:55458) 2024-09-19T08:39:06.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.395+0000 7fc8edd9b640 1 -- 172.21.15.98:0/1735962301 learned_addr learned my addr 172.21.15.98:0/1735962301 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:06.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.395+0000 7fc8ee59c640 1 -- 172.21.15.98:0/1735962301 >> v2:172.21.15.139:3300/0 conn(0x7fc8f0071ac0 msgr2=0x7fc8f00b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:06.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.395+0000 7fc8eed9d640 1 --2- 172.21.15.98:0/1735962301 >> v2:172.21.15.139:3300/0 conn(0x7fc8f0071ac0 0x7fc8f00b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:06.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.395+0000 7fc8ee59c640 1 --2- 172.21.15.98:0/1735962301 >> v2:172.21.15.139:3300/0 conn(0x7fc8f0071ac0 0x7fc8f00b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:06.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.395+0000 7fc8ee59c640 1 -- 172.21.15.98:0/1735962301 >> v1:172.21.15.98:6789/0 conn(0x7fc8f006c1d0 legacy=0x7fc8f00713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:06.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.395+0000 7fc8ed59a640 1 -- 172.21.15.98:0/1735962301 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2337562081 0 0) 0x7fc8f0073530 con 0x7fc8f006c1d0 2024-09-19T08:39:06.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.395+0000 7fc8ee59c640 1 -- 172.21.15.98:0/1735962301 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f0073810 con 0x7fc8f0145b70 2024-09-19T08:39:06.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.395+0000 7fc8eed9d640 1 --2- 172.21.15.98:0/1735962301 >> v2:172.21.15.139:3300/0 conn(0x7fc8f0071ac0 0x7fc8f00b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:06.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.399+0000 7fc8ee59c640 1 --2- 172.21.15.98:0/1735962301 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8f0145b70 0x7fc8f006bc90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0074450 tx=0x7fc8e001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0ec1d7fd79672c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:06.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.399+0000 7fc8ed59a640 1 -- 172.21.15.98:0/1735962301 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e00b0070 con 0x7fc8f0145b70 2024-09-19T08:39:06.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.399+0000 7fc8ed59a640 1 -- 172.21.15.98:0/1735962301 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8e00ab600 con 0x7fc8f0145b70 2024-09-19T08:39:06.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.399+0000 7fc8ed59a640 1 -- 172.21.15.98:0/1735962301 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e00ab9d0 con 0x7fc8f0145b70 2024-09-19T08:39:06.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.399+0000 7fc8f4d14640 1 -- 172.21.15.98:0/1735962301 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8f0145b70 msgr2=0x7fc8f006bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:06.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.399+0000 7fc8f4d14640 1 --2- 172.21.15.98:0/1735962301 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8f0145b70 0x7fc8f006bc90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0074450 tx=0x7fc8e001c3f0 comp rx=0 tx=0).stop 2024-09-19T08:39:06.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.399+0000 7fc8f4d14640 1 -- 172.21.15.98:0/1735962301 shutdown_connections 2024-09-19T08:39:06.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.399+0000 7fc8f4d14640 1 --2- 172.21.15.98:0/1735962301 >> v2:172.21.15.139:3300/0 conn(0x7fc8f0071ac0 0x7fc8f00b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:06.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.399+0000 7fc8f4d14640 1 --2- 172.21.15.98:0/1735962301 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8f0145b70 0x7fc8f006bc90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:06.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.399+0000 7fc8f4d14640 1 -- 172.21.15.98:0/1735962301 >> 172.21.15.98:0/1735962301 conn(0x7fc8f00b2010 msgr2=0x7fc8f00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:06.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.399+0000 7fc8f4d14640 1 -- 172.21.15.98:0/1735962301 shutdown_connections 2024-09-19T08:39:06.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.399+0000 7fc8f4d14640 1 -- 172.21.15.98:0/1735962301 wait complete. 2024-09-19T08:39:06.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.399+0000 7fc8f4d14640 1 Processor -- start 2024-09-19T08:39:06.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.399+0000 7fc8f4d14640 1 -- start start 2024-09-19T08:39:06.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.399+0000 7fc8f4d14640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8f006c1d0 0x7fc8f00aa580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:06.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.399+0000 7fc8f4d14640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8f0071ac0 0x7fc8f0141080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:06.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.399+0000 7fc8f4d14640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8f0145b70 0x7fc8f015a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:06.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.399+0000 7fc8f4d14640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f00b42e0 con 0x7fc8f0145b70 2024-09-19T08:39:06.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.399+0000 7fc8f4d14640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f00b4160 con 0x7fc8f006c1d0 2024-09-19T08:39:06.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.399+0000 7fc8f4d14640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc8f00b4460 con 0x7fc8f0071ac0 2024-09-19T08:39:06.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.399+0000 7fc8eed9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8f0145b70 0x7fc8f015a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:06.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.399+0000 7fc8eed9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8f0145b70 0x7fc8f015a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36112/0 (socket says 172.21.15.98:36112) 2024-09-19T08:39:06.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.399+0000 7fc8eed9d640 1 -- 172.21.15.98:0/781542337 learned_addr learned my addr 172.21.15.98:0/781542337 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:06.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.403+0000 7fc8ee59c640 1 --2- 172.21.15.98:0/781542337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8f006c1d0 0x7fc8f00aa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:06.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.403+0000 7fc8edd9b640 1 --2- 172.21.15.98:0/781542337 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8f0071ac0 0x7fc8f0141080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:06.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.403+0000 7fc8eed9d640 1 -- 172.21.15.98:0/781542337 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8f0071ac0 msgr2=0x7fc8f0141080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:06.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.403+0000 7fc8eed9d640 1 --2- 172.21.15.98:0/781542337 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8f0071ac0 0x7fc8f0141080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:06.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.403+0000 7fc8eed9d640 1 -- 172.21.15.98:0/781542337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8f006c1d0 msgr2=0x7fc8f00aa580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:06.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.403+0000 7fc8eed9d640 1 --2- 172.21.15.98:0/781542337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8f006c1d0 0x7fc8f00aa580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:06.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.403+0000 7fc8eed9d640 1 -- 172.21.15.98:0/781542337 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f013de60 con 0x7fc8f0145b70 2024-09-19T08:39:06.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.403+0000 7fc8eed9d640 1 --2- 172.21.15.98:0/781542337 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8f0145b70 0x7fc8f015a950 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc8e800cef0 tx=0x7fc8e800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:06.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.403+0000 7fc8deffd640 1 -- 172.21.15.98:0/781542337 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e801a070 con 0x7fc8f0145b70 2024-09-19T08:39:06.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.403+0000 7fc8deffd640 1 -- 172.21.15.98:0/781542337 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8e800d860 con 0x7fc8f0145b70 2024-09-19T08:39:06.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.403+0000 7fc8deffd640 1 -- 172.21.15.98:0/781542337 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e800dc50 con 0x7fc8f0145b70 2024-09-19T08:39:06.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.403+0000 7fc8f4d14640 1 -- 172.21.15.98:0/781542337 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8f013eb10 con 0x7fc8f0145b70 2024-09-19T08:39:06.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.403+0000 7fc8f4d14640 1 -- 172.21.15.98:0/781542337 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8f013f0d0 con 0x7fc8f0145b70 2024-09-19T08:39:06.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.403+0000 7fc8f4d14640 1 -- 172.21.15.98:0/781542337 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8f00b3410 con 0x7fc8f0145b70 2024-09-19T08:39:06.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.403+0000 7fc8deffd640 1 -- 172.21.15.98:0/781542337 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc8e801d070 con 0x7fc8f0145b70 2024-09-19T08:39:06.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.403+0000 7fc8deffd640 1 --2- 172.21.15.98:0/781542337 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc8d00678c0 0x7fc8d0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:06.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.403+0000 7fc8ee59c640 1 --2- 172.21.15.98:0/781542337 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc8d00678c0 0x7fc8d0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:06.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.403+0000 7fc8deffd640 1 -- 172.21.15.98:0/781542337 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fc8e8086560 con 0x7fc8f0145b70 2024-09-19T08:39:06.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.403+0000 7fc8ee59c640 1 --2- 172.21.15.98:0/781542337 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc8d00678c0 0x7fc8d0069d80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc8e00ad000 tx=0x7fc8e0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:06.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.407+0000 7fc8deffd640 1 -- 172.21.15.98:0/781542337 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc8e8057160 con 0x7fc8f0145b70 2024-09-19T08:39:06.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:06.611+0000 7fc8f4d14640 1 -- 172.21.15.98:0/781542337 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fc8f013f2a0 con 0x7fc8f0145b70 2024-09-19T08:39:07.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.275+0000 7fc8deffd640 1 -- 172.21.15.98:0/781542337 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7fc8e8056a60 con 0x7fc8f0145b70 2024-09-19T08:39:07.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.279+0000 7fc8f4d14640 1 -- 172.21.15.98:0/781542337 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc8d00678c0 msgr2=0x7fc8d0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:07.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.279+0000 7fc8f4d14640 1 --2- 172.21.15.98:0/781542337 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc8d00678c0 0x7fc8d0069d80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc8e00ad000 tx=0x7fc8e0008040 comp rx=0 tx=0).stop 2024-09-19T08:39:07.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.279+0000 7fc8f4d14640 1 -- 172.21.15.98:0/781542337 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8f0145b70 msgr2=0x7fc8f015a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:07.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.279+0000 7fc8f4d14640 1 --2- 172.21.15.98:0/781542337 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8f0145b70 0x7fc8f015a950 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc8e800cef0 tx=0x7fc8e800a710 comp rx=0 tx=0).stop 2024-09-19T08:39:07.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.279+0000 7fc8f4d14640 1 -- 172.21.15.98:0/781542337 shutdown_connections 2024-09-19T08:39:07.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.279+0000 7fc8f4d14640 1 --2- 172.21.15.98:0/781542337 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc8d00678c0 0x7fc8d0069d80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:07.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.279+0000 7fc8f4d14640 1 --2- 172.21.15.98:0/781542337 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8f0145b70 0x7fc8f015a950 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:07.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.279+0000 7fc8f4d14640 1 --2- 172.21.15.98:0/781542337 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8f0071ac0 0x7fc8f0141080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:07.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.279+0000 7fc8f4d14640 1 --2- 172.21.15.98:0/781542337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8f006c1d0 0x7fc8f00aa580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:07.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.279+0000 7fc8f4d14640 1 -- 172.21.15.98:0/781542337 >> 172.21.15.98:0/781542337 conn(0x7fc8f00b2010 msgr2=0x7fc8f00b0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:07.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.279+0000 7fc8f4d14640 1 -- 172.21.15.98:0/781542337 shutdown_connections 2024-09-19T08:39:07.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.279+0000 7fc8f4d14640 1 -- 172.21.15.98:0/781542337 wait complete. 2024-09-19T08:39:07.304 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-19T08:39:07.304 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:39:07.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.427+0000 7f0564cfa640 1 Processor -- start 2024-09-19T08:39:07.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.427+0000 7f0564cfa640 1 -- start start 2024-09-19T08:39:07.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.427+0000 7f0564cfa640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0560140d30 0x7f056006d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:07.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.427+0000 7f0564cfa640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f056015ea10 con 0x7f0560141110 2024-09-19T08:39:07.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.427+0000 7f0564cfa640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f056015ecf0 con 0x7f05600b3a20 2024-09-19T08:39:07.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.427+0000 7f0564cfa640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f056015e7c0 con 0x7f0560140d30 2024-09-19T08:39:07.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.427+0000 7f055dd9b640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f0560141110 0x7f05600b3310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:55464/0 (socket says 172.21.15.98:55464) 2024-09-19T08:39:07.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.427+0000 7f055dd9b640 1 -- 172.21.15.98:0/3386032340 learned_addr learned my addr 172.21.15.98:0/3386032340 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:07.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.427+0000 7f055e59c640 1 --2- 172.21.15.98:0/3386032340 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0560140d30 0x7f056006d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:07.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.427+0000 7f055d59a640 1 -- 172.21.15.98:0/3386032340 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2519305886 0 0) 0x7f056015ea10 con 0x7f0560141110 2024-09-19T08:39:07.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.427+0000 7f055d59a640 1 -- 172.21.15.98:0/3386032340 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f054c003530 con 0x7f0560141110 2024-09-19T08:39:07.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.427+0000 7f055d59a640 1 -- 172.21.15.98:0/3386032340 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2863364108 0 0) 0x7f054c003530 con 0x7f0560141110 2024-09-19T08:39:07.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.427+0000 7f055d59a640 1 -- 172.21.15.98:0/3386032340 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0560140d30 msgr2=0x7f056006d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:07.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.427+0000 7f055d59a640 1 --2- 172.21.15.98:0/3386032340 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0560140d30 0x7f056006d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:07.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.427+0000 7f055d59a640 1 -- 172.21.15.98:0/3386032340 >> v1:172.21.15.139:6789/0 conn(0x7f05600b3a20 legacy=0x7f056015b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:07.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.427+0000 7f055d59a640 1 -- 172.21.15.98:0/3386032340 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f056015efd0 con 0x7f0560141110 2024-09-19T08:39:07.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.427+0000 7f055d59a640 1 -- 172.21.15.98:0/3386032340 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f0548001680 con 0x7f0560141110 2024-09-19T08:39:07.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.427+0000 7f055d59a640 1 -- 172.21.15.98:0/3386032340 >> v1:172.21.15.98:6789/0 conn(0x7f0560141110 legacy=0x7f05600b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:07.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.427+0000 7f055d59a640 1 --2- 172.21.15.98:0/3386032340 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0560140d30 0x7f054c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:07.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.427+0000 7f055d59a640 1 --2- 172.21.15.98:0/3386032340 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f054c003f70 0x7f054c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:07.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.427+0000 7f055d59a640 1 --2- 172.21.15.98:0/3386032340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f054c004d50 0x7f054c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:07.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.427+0000 7f055d59a640 1 -- 172.21.15.98:0/3386032340 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f056015e7c0 con 0x7f054c004d50 2024-09-19T08:39:07.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.427+0000 7f055d59a640 1 -- 172.21.15.98:0/3386032340 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f054c00bab0 con 0x7f054c003f70 2024-09-19T08:39:07.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.427+0000 7f055d59a640 1 -- 172.21.15.98:0/3386032340 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f054c00bcb0 con 0x7f0560140d30 2024-09-19T08:39:07.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.427+0000 7f055ed9d640 1 --2- 172.21.15.98:0/3386032340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f054c004d50 0x7f054c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:07.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.427+0000 7f055e59c640 1 --2- 172.21.15.98:0/3386032340 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0560140d30 0x7f054c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:07.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.427+0000 7f055dd9b640 1 --2- 172.21.15.98:0/3386032340 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f054c003f70 0x7f054c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:07.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.427+0000 7f055ed9d640 1 -- 172.21.15.98:0/3386032340 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0560140d30 msgr2=0x7f054c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:07.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.427+0000 7f055ed9d640 1 --2- 172.21.15.98:0/3386032340 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0560140d30 0x7f054c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:07.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.427+0000 7f055ed9d640 1 -- 172.21.15.98:0/3386032340 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f054c003f70 msgr2=0x7f054c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:07.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.427+0000 7f055ed9d640 1 --2- 172.21.15.98:0/3386032340 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f054c003f70 0x7f054c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:07.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.427+0000 7f055ed9d640 1 -- 172.21.15.98:0/3386032340 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f054c003530 con 0x7f054c004d50 2024-09-19T08:39:07.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.427+0000 7f055e59c640 1 --2- 172.21.15.98:0/3386032340 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0560140d30 0x7f054c0039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:39:07.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.431+0000 7f055ed9d640 1 --2- 172.21.15.98:0/3386032340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f054c004d50 0x7f054c0051d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0550001780 tx=0x7f055001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=552324f61813ef09 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:07.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.431+0000 7f055d59a640 1 -- 172.21.15.98:0/3386032340 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05500013a0 con 0x7f054c004d50 2024-09-19T08:39:07.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.431+0000 7f055d59a640 1 -- 172.21.15.98:0/3386032340 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05500a9070 con 0x7f054c004d50 2024-09-19T08:39:07.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.431+0000 7f055d59a640 1 -- 172.21.15.98:0/3386032340 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0550003600 con 0x7f054c004d50 2024-09-19T08:39:07.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.431+0000 7f0564cfa640 1 -- 172.21.15.98:0/3386032340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f054c004d50 msgr2=0x7f054c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:07.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.431+0000 7f0564cfa640 1 --2- 172.21.15.98:0/3386032340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f054c004d50 0x7f054c0051d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0550001780 tx=0x7f055001c420 comp rx=0 tx=0).stop 2024-09-19T08:39:07.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.431+0000 7f055ed9d640 1 -- 172.21.15.98:0/3386032340 reap_dead start 2024-09-19T08:39:07.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.431+0000 7f0564cfa640 1 -- 172.21.15.98:0/3386032340 shutdown_connections 2024-09-19T08:39:07.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.431+0000 7f0564cfa640 1 -- 172.21.15.98:0/3386032340 >> 172.21.15.98:0/3386032340 conn(0x7f05600b2010 msgr2=0x7f05600b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:07.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.431+0000 7f0564cfa640 1 -- 172.21.15.98:0/3386032340 shutdown_connections 2024-09-19T08:39:07.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.431+0000 7f0564cfa640 1 -- 172.21.15.98:0/3386032340 wait complete. 2024-09-19T08:39:07.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.431+0000 7f0564cfa640 1 Processor -- start 2024-09-19T08:39:07.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.431+0000 7f0564cfa640 1 -- start start 2024-09-19T08:39:07.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.435+0000 7f0564cfa640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f054c004d50 0x7f056015c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:07.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.435+0000 7f0564cfa640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0560157f50 0x7f05600ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:07.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.435+0000 7f0564cfa640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05600cadd0 0x7f05601f45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:07.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.435+0000 7f0564cfa640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f056006bc90 con 0x7f05600cadd0 2024-09-19T08:39:07.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.435+0000 7f0564cfa640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f05600ce1a0 con 0x7f0560157f50 2024-09-19T08:39:07.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.435+0000 7f0564cfa640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f056006be10 con 0x7f054c004d50 2024-09-19T08:39:07.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.435+0000 7f055ed9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05600cadd0 0x7f05601f45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:07.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.435+0000 7f055ed9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05600cadd0 0x7f05601f45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36132/0 (socket says 172.21.15.98:36132) 2024-09-19T08:39:07.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.435+0000 7f055ed9d640 1 -- 172.21.15.98:0/346256902 learned_addr learned my addr 172.21.15.98:0/346256902 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:07.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.435+0000 7f055e59c640 1 --2- 172.21.15.98:0/346256902 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f054c004d50 0x7f056015c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:07.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.435+0000 7f055dd9b640 1 --2- 172.21.15.98:0/346256902 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0560157f50 0x7f05600ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:07.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.435+0000 7f055ed9d640 1 -- 172.21.15.98:0/346256902 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f054c004d50 msgr2=0x7f056015c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:07.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.435+0000 7f055ed9d640 1 --2- 172.21.15.98:0/346256902 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f054c004d50 0x7f056015c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:07.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.435+0000 7f055ed9d640 1 -- 172.21.15.98:0/346256902 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0560157f50 msgr2=0x7f05600ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:07.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.435+0000 7f055ed9d640 1 --2- 172.21.15.98:0/346256902 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0560157f50 0x7f05600ca890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:07.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.435+0000 7f055ed9d640 1 -- 172.21.15.98:0/346256902 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f056015b930 con 0x7f05600cadd0 2024-09-19T08:39:07.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.435+0000 7f055ed9d640 1 --2- 172.21.15.98:0/346256902 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05600cadd0 0x7f05601f45a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0550005040 tx=0x7f054c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:07.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.435+0000 7f0556ffd640 1 -- 172.21.15.98:0/346256902 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0550007a40 con 0x7f05600cadd0 2024-09-19T08:39:07.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.435+0000 7f0556ffd640 1 -- 172.21.15.98:0/346256902 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05500bc070 con 0x7f05600cadd0 2024-09-19T08:39:07.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.435+0000 7f0564cfa640 1 -- 172.21.15.98:0/346256902 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f056015bbd0 con 0x7f05600cadd0 2024-09-19T08:39:07.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.435+0000 7f0556ffd640 1 -- 172.21.15.98:0/346256902 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0550007450 con 0x7f05600cadd0 2024-09-19T08:39:07.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.435+0000 7f0564cfa640 1 -- 172.21.15.98:0/346256902 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f056015c190 con 0x7f05600cadd0 2024-09-19T08:39:07.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.435+0000 7f0564cfa640 1 -- 172.21.15.98:0/346256902 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f056006c190 con 0x7f05600cadd0 2024-09-19T08:39:07.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.435+0000 7f0556ffd640 1 -- 172.21.15.98:0/346256902 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f05500b5070 con 0x7f05600cadd0 2024-09-19T08:39:07.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.435+0000 7f0556ffd640 1 --2- 172.21.15.98:0/346256902 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f05400678f0 0x7f0540069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:07.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.435+0000 7f0556ffd640 1 -- 172.21.15.98:0/346256902 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f055011f4a0 con 0x7f05600cadd0 2024-09-19T08:39:07.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.435+0000 7f055e59c640 1 --2- 172.21.15.98:0/346256902 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f05400678f0 0x7f0540069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:07.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.435+0000 7f055e59c640 1 --2- 172.21.15.98:0/346256902 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f05400678f0 0x7f0540069db0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f055800a240 tx=0x7f055800c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:07.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.439+0000 7f0556ffd640 1 -- 172.21.15.98:0/346256902 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f0550016130 con 0x7f05600cadd0 2024-09-19T08:39:07.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.635+0000 7f0564cfa640 1 -- 172.21.15.98:0/346256902 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f056015ab00 con 0x7f05600cadd0 2024-09-19T08:39:07.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.639+0000 7f0556ffd640 1 -- 172.21.15.98:0/346256902 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5229 (secure 0 0 0) 0x7f05500f0100 con 0x7f05600cadd0 2024-09-19T08:39:07.641 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:39:07.641 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":14,"btime":"2024-09-19T08:39:07:270861+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4481,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3797406671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3797406671},{"type":"v1","addr":"172.21.15.98:6835","nonce":3797406671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/4112109751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4112109751},{"type":"v1","addr":"172.21.15.139:6834","nonce":4112109751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4684,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1071477338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6833","nonce":1071477338},{"type":"v1","addr":"172.21.15.139:6835","nonce":1071477338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:39:04.632502+0000","modified":"2024-09-19T08:39:06.618098+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4833},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4833":{"gid":4833,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/1820856497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1820856497},{"type":"v1","addr":"172.21.15.98:6837","nonce":1820856497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4833,"qdb_cluster":[4833]},"id":2}]} 2024-09-19T08:39:07.641 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 14 2024-09-19T08:39:07.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.639+0000 7f0564cfa640 1 -- 172.21.15.98:0/346256902 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f05400678f0 msgr2=0x7f0540069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:07.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.639+0000 7f0564cfa640 1 --2- 172.21.15.98:0/346256902 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f05400678f0 0x7f0540069db0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f055800a240 tx=0x7f055800c040 comp rx=0 tx=0).stop 2024-09-19T08:39:07.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.639+0000 7f0564cfa640 1 -- 172.21.15.98:0/346256902 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05600cadd0 msgr2=0x7f05601f45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:07.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.639+0000 7f0564cfa640 1 --2- 172.21.15.98:0/346256902 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05600cadd0 0x7f05601f45a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0550005040 tx=0x7f054c005840 comp rx=0 tx=0).stop 2024-09-19T08:39:07.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.639+0000 7f0564cfa640 1 -- 172.21.15.98:0/346256902 shutdown_connections 2024-09-19T08:39:07.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.639+0000 7f0564cfa640 1 --2- 172.21.15.98:0/346256902 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f05400678f0 0x7f0540069db0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:07.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.639+0000 7f0564cfa640 1 --2- 172.21.15.98:0/346256902 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f05600cadd0 0x7f05601f45a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:07.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.639+0000 7f0564cfa640 1 --2- 172.21.15.98:0/346256902 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0560157f50 0x7f05600ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:07.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.639+0000 7f0564cfa640 1 --2- 172.21.15.98:0/346256902 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f054c004d50 0x7f056015c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:07.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.639+0000 7f0564cfa640 1 -- 172.21.15.98:0/346256902 >> 172.21.15.98:0/346256902 conn(0x7f05600b2010 msgr2=0x7f05600cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:07.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.639+0000 7f0564cfa640 1 -- 172.21.15.98:0/346256902 shutdown_connections 2024-09-19T08:39:07.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.639+0000 7f0564cfa640 1 -- 172.21.15.98:0/346256902 wait complete. 2024-09-19T08:39:07.660 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:39:07.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.775+0000 7fd16b306640 1 Processor -- start 2024-09-19T08:39:07.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.775+0000 7fd16b306640 1 -- start start 2024-09-19T08:39:07.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.779+0000 7fd16b306640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd16406c3a0 0x7fd1640cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:07.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.779+0000 7fd16b306640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd1640cf900 0x7fd1640723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:07.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.779+0000 7fd16b306640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd16406d310 con 0x7fd16415c550 2024-09-19T08:39:07.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.779+0000 7fd16b306640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fd16406d5f0 con 0x7fd1640cf900 2024-09-19T08:39:07.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.779+0000 7fd16b306640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fd16406d8d0 con 0x7fd16406c3a0 2024-09-19T08:39:07.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.779+0000 7fd1698a3640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd1640cf900 0x7fd1640723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:07.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.779+0000 7fd1690a2640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fd16415c550 0x7fd16406bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:45554/0 (socket says 172.21.15.98:45554) 2024-09-19T08:39:07.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.779+0000 7fd1690a2640 1 -- 172.21.15.98:0/3607312149 learned_addr learned my addr 172.21.15.98:0/3607312149 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:07.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.779+0000 7fd1688a1640 1 --2- 172.21.15.98:0/3607312149 >> v2:172.21.15.139:3300/0 conn(0x7fd16406c3a0 0x7fd1640cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:07.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.779+0000 7fd1698a3640 1 -- 172.21.15.98:0/3607312149 >> v2:172.21.15.139:3300/0 conn(0x7fd16406c3a0 msgr2=0x7fd1640cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:07.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.779+0000 7fd1698a3640 1 --2- 172.21.15.98:0/3607312149 >> v2:172.21.15.139:3300/0 conn(0x7fd16406c3a0 0x7fd1640cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:07.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.779+0000 7fd1698a3640 1 -- 172.21.15.98:0/3607312149 >> v1:172.21.15.98:6789/0 conn(0x7fd16415c550 legacy=0x7fd16406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:07.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.779+0000 7fd1698a3640 1 -- 172.21.15.98:0/3607312149 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd16406dbb0 con 0x7fd1640cf900 2024-09-19T08:39:07.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.779+0000 7fd15bfff640 1 -- 172.21.15.98:0/3607312149 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1062358560 0 0) 0x7fd16406d310 con 0x7fd16415c550 2024-09-19T08:39:07.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.779+0000 7fd1698a3640 1 --2- 172.21.15.98:0/3607312149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1640cf900 0x7fd1640723a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd1640ccc60 tx=0x7fd15c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f9a44da13d625eb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:07.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.779+0000 7fd15bfff640 1 -- 172.21.15.98:0/3607312149 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c0b2070 con 0x7fd1640cf900 2024-09-19T08:39:07.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.779+0000 7fd15bfff640 1 -- 172.21.15.98:0/3607312149 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd15c0a5900 con 0x7fd1640cf900 2024-09-19T08:39:07.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.779+0000 7fd15bfff640 1 -- 172.21.15.98:0/3607312149 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c0a5cd0 con 0x7fd1640cf900 2024-09-19T08:39:07.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.783+0000 7fd16b306640 1 -- 172.21.15.98:0/3607312149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1640cf900 msgr2=0x7fd1640723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:07.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.783+0000 7fd16b306640 1 --2- 172.21.15.98:0/3607312149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1640cf900 0x7fd1640723a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd1640ccc60 tx=0x7fd15c01c930 comp rx=0 tx=0).stop 2024-09-19T08:39:07.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.783+0000 7fd16b306640 1 -- 172.21.15.98:0/3607312149 shutdown_connections 2024-09-19T08:39:07.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.783+0000 7fd16b306640 1 --2- 172.21.15.98:0/3607312149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1640cf900 0x7fd1640723a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:07.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.783+0000 7fd16b306640 1 --2- 172.21.15.98:0/3607312149 >> v2:172.21.15.139:3300/0 conn(0x7fd16406c3a0 0x7fd1640cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:07.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.783+0000 7fd16b306640 1 -- 172.21.15.98:0/3607312149 >> 172.21.15.98:0/3607312149 conn(0x7fd1640b2010 msgr2=0x7fd1640b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:07.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.783+0000 7fd16b306640 1 -- 172.21.15.98:0/3607312149 shutdown_connections 2024-09-19T08:39:07.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.783+0000 7fd16b306640 1 -- 172.21.15.98:0/3607312149 wait complete. 2024-09-19T08:39:07.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.783+0000 7fd16b306640 1 Processor -- start 2024-09-19T08:39:07.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.783+0000 7fd16b306640 1 -- start start 2024-09-19T08:39:07.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.783+0000 7fd16b306640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd16406c3a0 0x7fd16414bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:07.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.783+0000 7fd16b306640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1640cf900 0x7fd16414b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:07.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.783+0000 7fd16b306640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd16415c550 0x7fd1641560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:07.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.783+0000 7fd16b306640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd164073480 con 0x7fd16406c3a0 2024-09-19T08:39:07.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.783+0000 7fd16b306640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd164073300 con 0x7fd16415c550 2024-09-19T08:39:07.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.783+0000 7fd16b306640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd164073600 con 0x7fd1640cf900 2024-09-19T08:39:07.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.783+0000 7fd1690a2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd16406c3a0 0x7fd16414bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:07.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.783+0000 7fd1690a2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd16406c3a0 0x7fd16414bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36378/0 (socket says 172.21.15.98:36378) 2024-09-19T08:39:07.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.783+0000 7fd1690a2640 1 -- 172.21.15.98:0/2725420623 learned_addr learned my addr 172.21.15.98:0/2725420623 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:07.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.783+0000 7fd1698a3640 1 --2- 172.21.15.98:0/2725420623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd16415c550 0x7fd1641560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:07.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.783+0000 7fd1690a2640 1 -- 172.21.15.98:0/2725420623 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1640cf900 msgr2=0x7fd16414b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:07.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.783+0000 7fd1690a2640 1 --2- 172.21.15.98:0/2725420623 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1640cf900 0x7fd16414b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:07.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.783+0000 7fd1690a2640 1 -- 172.21.15.98:0/2725420623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd16415c550 msgr2=0x7fd1641560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:07.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.783+0000 7fd1690a2640 1 --2- 172.21.15.98:0/2725420623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd16415c550 0x7fd1641560f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:07.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.783+0000 7fd1690a2640 1 -- 172.21.15.98:0/2725420623 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd16415a250 con 0x7fd16406c3a0 2024-09-19T08:39:07.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.783+0000 7fd1690a2640 1 --2- 172.21.15.98:0/2725420623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd16406c3a0 0x7fd16414bf10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd1600011f0 tx=0x7fd16000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:07.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.783+0000 7fd159ffb640 1 -- 172.21.15.98:0/2725420623 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160018070 con 0x7fd16406c3a0 2024-09-19T08:39:07.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.783+0000 7fd159ffb640 1 -- 172.21.15.98:0/2725420623 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1600074e0 con 0x7fd16406c3a0 2024-09-19T08:39:07.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.783+0000 7fd16b306640 1 -- 172.21.15.98:0/2725420623 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd16415a530 con 0x7fd16406c3a0 2024-09-19T08:39:07.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.783+0000 7fd159ffb640 1 -- 172.21.15.98:0/2725420623 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160004a30 con 0x7fd16406c3a0 2024-09-19T08:39:07.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.783+0000 7fd16b306640 1 -- 172.21.15.98:0/2725420623 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd164155bd0 con 0x7fd16406c3a0 2024-09-19T08:39:07.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.783+0000 7fd16b306640 1 -- 172.21.15.98:0/2725420623 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1640cf3c0 con 0x7fd16406c3a0 2024-09-19T08:39:07.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.787+0000 7fd159ffb640 1 -- 172.21.15.98:0/2725420623 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd16001b070 con 0x7fd16406c3a0 2024-09-19T08:39:07.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.787+0000 7fd159ffb640 1 --2- 172.21.15.98:0/2725420623 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd1480678c0 0x7fd148069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:07.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.787+0000 7fd159ffb640 1 -- 172.21.15.98:0/2725420623 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fd160084fc0 con 0x7fd16406c3a0 2024-09-19T08:39:07.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.787+0000 7fd1688a1640 1 --2- 172.21.15.98:0/2725420623 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd1480678c0 0x7fd148069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:07.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.787+0000 7fd1688a1640 1 --2- 172.21.15.98:0/2725420623 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd1480678c0 0x7fd148069d80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd1640b51b0 tx=0x7fd154008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:07.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.791+0000 7fd159ffb640 1 -- 172.21.15.98:0/2725420623 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd160055b90 con 0x7fd16406c3a0 2024-09-19T08:39:07.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.955+0000 7fd16b306640 1 -- 172.21.15.98:0/2725420623 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd16406c080 con 0x7fd16406c3a0 2024-09-19T08:39:07.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.955+0000 7fd159ffb640 1 -- 172.21.15.98:0/2725420623 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16729 (secure 0 0 0) 0x7fd160055490 con 0x7fd16406c3a0 2024-09-19T08:39:07.961 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:39:07.961 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":32,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:39:06.263597+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:39:02.429044+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-19T08:39:03.620637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:39:07.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.959+0000 7fd16b306640 1 -- 172.21.15.98:0/2725420623 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd1480678c0 msgr2=0x7fd148069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:07.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.959+0000 7fd16b306640 1 --2- 172.21.15.98:0/2725420623 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd1480678c0 0x7fd148069d80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd1640b51b0 tx=0x7fd154008040 comp rx=0 tx=0).stop 2024-09-19T08:39:07.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.959+0000 7fd16b306640 1 -- 172.21.15.98:0/2725420623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd16406c3a0 msgr2=0x7fd16414bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:07.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.959+0000 7fd16b306640 1 --2- 172.21.15.98:0/2725420623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd16406c3a0 0x7fd16414bf10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd1600011f0 tx=0x7fd16000c9b0 comp rx=0 tx=0).stop 2024-09-19T08:39:07.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.959+0000 7fd16b306640 1 -- 172.21.15.98:0/2725420623 shutdown_connections 2024-09-19T08:39:07.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.959+0000 7fd16b306640 1 --2- 172.21.15.98:0/2725420623 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd1480678c0 0x7fd148069d80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:07.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.959+0000 7fd16b306640 1 --2- 172.21.15.98:0/2725420623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd16415c550 0x7fd1641560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:07.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.959+0000 7fd16b306640 1 --2- 172.21.15.98:0/2725420623 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1640cf900 0x7fd16414b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:07.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.959+0000 7fd16b306640 1 --2- 172.21.15.98:0/2725420623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd16406c3a0 0x7fd16414bf10 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:07.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.959+0000 7fd16b306640 1 -- 172.21.15.98:0/2725420623 >> 172.21.15.98:0/2725420623 conn(0x7fd1640b2010 msgr2=0x7fd164067a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:07.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.959+0000 7fd16b306640 1 -- 172.21.15.98:0/2725420623 shutdown_connections 2024-09-19T08:39:07.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:07.959+0000 7fd16b306640 1 -- 172.21.15.98:0/2725420623 wait complete. 2024-09-19T08:39:07.980 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T08:39:07.981 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:39:08.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.091+0000 7fcb0106d640 1 Processor -- start 2024-09-19T08:39:08.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.091+0000 7fcb0106d640 1 -- start start 2024-09-19T08:39:08.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.095+0000 7fcb0106d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcafc06d880 0x7fcafc0b3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:08.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.095+0000 7fcb0106d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcafc141110 0x7fcafc0b3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:08.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.095+0000 7fcb0106d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcafc15ea10 con 0x7fcafc140d30 2024-09-19T08:39:08.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.095+0000 7fcb0106d640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fcafc15e7c0 con 0x7fcafc06d880 2024-09-19T08:39:08.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.095+0000 7fcb0106d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fcafc15ecf0 con 0x7fcafc141110 2024-09-19T08:39:08.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.095+0000 7fcafa59c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcafc06d880 0x7fcafc0b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:08.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.095+0000 7fcafa59c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcafc06d880 0x7fcafc0b3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36386/0 (socket says 172.21.15.98:36386) 2024-09-19T08:39:08.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.095+0000 7fcafa59c640 1 -- 172.21.15.98:0/323002376 learned_addr learned my addr 172.21.15.98:0/323002376 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:08.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.095+0000 7fcafb59e640 1 --2- 172.21.15.98:0/323002376 >> v2:172.21.15.139:3300/0 conn(0x7fcafc141110 0x7fcafc0b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:08.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.095+0000 7fcafa59c640 1 -- 172.21.15.98:0/323002376 >> v2:172.21.15.139:3300/0 conn(0x7fcafc141110 msgr2=0x7fcafc0b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:08.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.095+0000 7fcafa59c640 1 --2- 172.21.15.98:0/323002376 >> v2:172.21.15.139:3300/0 conn(0x7fcafc141110 0x7fcafc0b3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:08.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.095+0000 7fcafa59c640 1 -- 172.21.15.98:0/323002376 >> v1:172.21.15.139:6789/0 conn(0x7fcafc140d30 legacy=0x7fcafc06d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:08.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.095+0000 7fcafa59c640 1 -- 172.21.15.98:0/323002376 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcafc15efd0 con 0x7fcafc06d880 2024-09-19T08:39:08.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.095+0000 7fcafa59c640 1 --2- 172.21.15.98:0/323002376 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcafc06d880 0x7fcafc0b3310 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fcafc06cd40 tx=0x7fcaf001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d529b801f8fb60 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:08.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.095+0000 7fcaf9d9b640 1 -- 172.21.15.98:0/323002376 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf00ab680 con 0x7fcafc06d880 2024-09-19T08:39:08.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.095+0000 7fcaf9d9b640 1 -- 172.21.15.98:0/323002376 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaf00b0070 con 0x7fcafc06d880 2024-09-19T08:39:08.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.095+0000 7fcaf9d9b640 1 -- 172.21.15.98:0/323002376 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf00aba40 con 0x7fcafc06d880 2024-09-19T08:39:08.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.099+0000 7fcb0106d640 1 -- 172.21.15.98:0/323002376 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcafc06d880 msgr2=0x7fcafc0b3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:08.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.099+0000 7fcb0106d640 1 --2- 172.21.15.98:0/323002376 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcafc06d880 0x7fcafc0b3310 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fcafc06cd40 tx=0x7fcaf001c4e0 comp rx=0 tx=0).stop 2024-09-19T08:39:08.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.099+0000 7fcb0106d640 1 -- 172.21.15.98:0/323002376 shutdown_connections 2024-09-19T08:39:08.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.099+0000 7fcb0106d640 1 --2- 172.21.15.98:0/323002376 >> v2:172.21.15.139:3300/0 conn(0x7fcafc141110 0x7fcafc0b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:08.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.099+0000 7fcb0106d640 1 --2- 172.21.15.98:0/323002376 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcafc06d880 0x7fcafc0b3310 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:08.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.099+0000 7fcb0106d640 1 -- 172.21.15.98:0/323002376 >> 172.21.15.98:0/323002376 conn(0x7fcafc0b2010 msgr2=0x7fcafc0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:08.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.099+0000 7fcb0106d640 1 -- 172.21.15.98:0/323002376 shutdown_connections 2024-09-19T08:39:08.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.099+0000 7fcb0106d640 1 -- 172.21.15.98:0/323002376 wait complete. 2024-09-19T08:39:08.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.099+0000 7fcb0106d640 1 Processor -- start 2024-09-19T08:39:08.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.099+0000 7fcb0106d640 1 -- start start 2024-09-19T08:39:08.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.099+0000 7fcb0106d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcafc06d880 0x7fcafc160300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:08.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.099+0000 7fcb0106d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcafc140d30 0x7fcafc164240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:08.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.099+0000 7fcb0106d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcafc141110 0x7fcafc161f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:08.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.099+0000 7fcb0106d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc167ce0 con 0x7fcafc140d30 2024-09-19T08:39:08.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.099+0000 7fcb0106d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc167b60 con 0x7fcafc141110 2024-09-19T08:39:08.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.099+0000 7fcb0106d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcafc167e60 con 0x7fcafc06d880 2024-09-19T08:39:08.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.099+0000 7fcafa59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcafc140d30 0x7fcafc164240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:08.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.099+0000 7fcafa59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcafc140d30 0x7fcafc164240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36392/0 (socket says 172.21.15.98:36392) 2024-09-19T08:39:08.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.099+0000 7fcafa59c640 1 -- 172.21.15.98:0/1316649420 learned_addr learned my addr 172.21.15.98:0/1316649420 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:08.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.099+0000 7fcafad9d640 1 --2- 172.21.15.98:0/1316649420 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcafc06d880 0x7fcafc160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:08.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.099+0000 7fcafb59e640 1 --2- 172.21.15.98:0/1316649420 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcafc141110 0x7fcafc161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:08.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.099+0000 7fcafa59c640 1 -- 172.21.15.98:0/1316649420 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcafc06d880 msgr2=0x7fcafc160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:08.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.099+0000 7fcafa59c640 1 --2- 172.21.15.98:0/1316649420 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcafc06d880 0x7fcafc160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:08.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.099+0000 7fcafa59c640 1 -- 172.21.15.98:0/1316649420 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcafc141110 msgr2=0x7fcafc161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:08.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.099+0000 7fcafa59c640 1 --2- 172.21.15.98:0/1316649420 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcafc141110 0x7fcafc161f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:08.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.099+0000 7fcafa59c640 1 -- 172.21.15.98:0/1316649420 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcafc140570 con 0x7fcafc140d30 2024-09-19T08:39:08.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.099+0000 7fcafa59c640 1 --2- 172.21.15.98:0/1316649420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcafc140d30 0x7fcafc164240 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcaf001a110 tx=0x7fcaf001ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:08.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.099+0000 7fcae37fe640 1 -- 172.21.15.98:0/1316649420 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf00be5f0 con 0x7fcafc140d30 2024-09-19T08:39:08.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.099+0000 7fcae37fe640 1 -- 172.21.15.98:0/1316649420 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaf00a7070 con 0x7fcafc140d30 2024-09-19T08:39:08.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.099+0000 7fcb0106d640 1 -- 172.21.15.98:0/1316649420 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcafc160a00 con 0x7fcafc140d30 2024-09-19T08:39:08.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.099+0000 7fcae37fe640 1 -- 172.21.15.98:0/1316649420 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf00be990 con 0x7fcafc140d30 2024-09-19T08:39:08.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.099+0000 7fcb0106d640 1 -- 172.21.15.98:0/1316649420 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcafc072380 con 0x7fcafc140d30 2024-09-19T08:39:08.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.103+0000 7fcb0106d640 1 -- 172.21.15.98:0/1316649420 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcafc0b3520 con 0x7fcafc140d30 2024-09-19T08:39:08.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.103+0000 7fcae37fe640 1 -- 172.21.15.98:0/1316649420 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcaf00b43f0 con 0x7fcafc140d30 2024-09-19T08:39:08.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.103+0000 7fcae37fe640 1 --2- 172.21.15.98:0/1316649420 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fcae80678f0 0x7fcae8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:08.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.103+0000 7fcae37fe640 1 -- 172.21.15.98:0/1316649420 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fcaf01200e0 con 0x7fcafc140d30 2024-09-19T08:39:08.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.103+0000 7fcafad9d640 1 --2- 172.21.15.98:0/1316649420 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fcae80678f0 0x7fcae8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:08.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.103+0000 7fcafad9d640 1 --2- 172.21.15.98:0/1316649420 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fcae80678f0 0x7fcae8069db0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcaec004e30 tx=0x7fcaec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:08.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.107+0000 7fcae37fe640 1 -- 172.21.15.98:0/1316649420 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fcaf00c4060 con 0x7fcafc140d30 2024-09-19T08:39:08.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.267+0000 7fcb0106d640 1 -- 172.21.15.98:0/1316649420 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcafc164780 con 0x7fcae80678f0 2024-09-19T08:39:08.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.283+0000 7fcae37fe640 1 -- 172.21.15.98:0/1316649420 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240452 (secure 0 0 0) 0x7fcaec0053b0 con 0x7fcae80678f0 2024-09-19T08:39:08.289 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:39:08.291 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:39:08.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.287+0000 7fcb0106d640 1 -- 172.21.15.98:0/1316649420 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fcae80678f0 msgr2=0x7fcae8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:08.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.287+0000 7fcb0106d640 1 --2- 172.21.15.98:0/1316649420 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fcae80678f0 0x7fcae8069db0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcaec004e30 tx=0x7fcaec009040 comp rx=0 tx=0).stop 2024-09-19T08:39:08.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.287+0000 7fcb0106d640 1 -- 172.21.15.98:0/1316649420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcafc140d30 msgr2=0x7fcafc164240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:08.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.287+0000 7fcb0106d640 1 --2- 172.21.15.98:0/1316649420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcafc140d30 0x7fcafc164240 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcaf001a110 tx=0x7fcaf001ad20 comp rx=0 tx=0).stop 2024-09-19T08:39:08.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.287+0000 7fcb0106d640 1 -- 172.21.15.98:0/1316649420 shutdown_connections 2024-09-19T08:39:08.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.287+0000 7fcb0106d640 1 --2- 172.21.15.98:0/1316649420 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fcae80678f0 0x7fcae8069db0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:08.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.287+0000 7fcb0106d640 1 --2- 172.21.15.98:0/1316649420 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcafc141110 0x7fcafc161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:08.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.287+0000 7fcb0106d640 1 --2- 172.21.15.98:0/1316649420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcafc140d30 0x7fcafc164240 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:08.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.287+0000 7fcb0106d640 1 --2- 172.21.15.98:0/1316649420 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcafc06d880 0x7fcafc160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:08.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.287+0000 7fcb0106d640 1 -- 172.21.15.98:0/1316649420 >> 172.21.15.98:0/1316649420 conn(0x7fcafc0b2010 msgr2=0x7fcafc0d02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:08.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.287+0000 7fcb0106d640 1 -- 172.21.15.98:0/1316649420 shutdown_connections 2024-09-19T08:39:08.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.287+0000 7fcb0106d640 1 -- 172.21.15.98:0/1316649420 wait complete. 2024-09-19T08:39:08.308 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2024-09-19T08:39:08.040412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590950,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":163,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121,"ondisk_log_size":121,"up":62,"acting":62,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":219856,"kb_used_data":3632,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754864,"statfs":{"total":773094113280,"available":772868980736,"internally_reserved":0,"allocated":3719168,"data_stored":2113779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.801923"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.458834+0000","last_change":"2024-09-19T08:39:05.300955+0000","last_active":"2024-09-19T08:39:06.458834+0000","last_peered":"2024-09-19T08:39:06.458834+0000","last_clean":"2024-09-19T08:39:06.458834+0000","last_became_active":"2024-09-19T08:39:05.300228+0000","last_became_peered":"2024-09-19T08:39:05.300228+0000","last_unstale":"2024-09-19T08:39:06.458834+0000","last_undegraded":"2024-09-19T08:39:06.458834+0000","last_fullsized":"2024-09-19T08:39:06.458834+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:05.814581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.459141+0000","last_change":"2024-09-19T08:39:05.276219+0000","last_active":"2024-09-19T08:39:06.459141+0000","last_peered":"2024-09-19T08:39:06.459141+0000","last_clean":"2024-09-19T08:39:06.459141+0000","last_became_active":"2024-09-19T08:39:05.275466+0000","last_became_peered":"2024-09-19T08:39:05.275466+0000","last_unstale":"2024-09-19T08:39:06.459141+0000","last_undegraded":"2024-09-19T08:39:06.459141+0000","last_fullsized":"2024-09-19T08:39:06.459141+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:09.038404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.459079+0000","last_change":"2024-09-19T08:39:04.292831+0000","last_active":"2024-09-19T08:39:06.459079+0000","last_peered":"2024-09-19T08:39:06.459079+0000","last_clean":"2024-09-19T08:39:06.459079+0000","last_became_active":"2024-09-19T08:39:04.292058+0000","last_became_peered":"2024-09-19T08:39:04.292058+0000","last_unstale":"2024-09-19T08:39:06.459079+0000","last_undegraded":"2024-09-19T08:39:06.459079+0000","last_fullsized":"2024-09-19T08:39:06.459079+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:38:41.573373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.459180+0000","last_change":"2024-09-19T08:39:05.305411+0000","last_active":"2024-09-19T08:39:06.459180+0000","last_peered":"2024-09-19T08:39:06.459180+0000","last_clean":"2024-09-19T08:39:06.459180+0000","last_became_active":"2024-09-19T08:39:05.304823+0000","last_became_peered":"2024-09-19T08:39:05.304823+0000","last_unstale":"2024-09-19T08:39:06.459180+0000","last_undegraded":"2024-09-19T08:39:06.459180+0000","last_fullsized":"2024-09-19T08:39:06.459180+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:11.979445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.300418+0000","last_change":"2024-09-19T08:39:03.300418+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.300418+0000","last_undegraded":"2024-09-19T08:39:03.300418+0000","last_fullsized":"2024-09-19T08:39:03.300418+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:03.224537+0000","last_change":"2024-09-19T08:39:03.224537+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.224537+0000","last_undegraded":"2024-09-19T08:39:03.224537+0000","last_fullsized":"2024-09-19T08:39:03.224537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.243996+0000","last_change":"2024-09-19T08:39:03.243996+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.243996+0000","last_undegraded":"2024-09-19T08:39:03.243996+0000","last_fullsized":"2024-09-19T08:39:03.243996+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.245284+0000","last_change":"2024-09-19T08:39:03.245284+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.245284+0000","last_undegraded":"2024-09-19T08:39:03.245284+0000","last_fullsized":"2024-09-19T08:39:03.245284+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.243944+0000","last_change":"2024-09-19T08:39:03.243944+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.243944+0000","last_undegraded":"2024-09-19T08:39:03.243944+0000","last_fullsized":"2024-09-19T08:39:03.243944+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:03.224537+0000","last_change":"2024-09-19T08:39:03.224537+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.224537+0000","last_undegraded":"2024-09-19T08:39:03.224537+0000","last_fullsized":"2024-09-19T08:39:03.224537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.243920+0000","last_change":"2024-09-19T08:39:03.243920+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.243920+0000","last_undegraded":"2024-09-19T08:39:03.243920+0000","last_fullsized":"2024-09-19T08:39:03.243920+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:03.224537+0000","last_change":"2024-09-19T08:39:03.224537+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.224537+0000","last_undegraded":"2024-09-19T08:39:03.224537+0000","last_fullsized":"2024-09-19T08:39:03.224537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.299294+0000","last_change":"2024-09-19T08:39:03.299294+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.299294+0000","last_undegraded":"2024-09-19T08:39:03.299294+0000","last_fullsized":"2024-09-19T08:39:03.299294+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.249325+0000","last_change":"2024-09-19T08:39:03.249325+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.249325+0000","last_undegraded":"2024-09-19T08:39:03.249325+0000","last_fullsized":"2024-09-19T08:39:03.249325+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:03.224537+0000","last_change":"2024-09-19T08:39:03.224537+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.224537+0000","last_undegraded":"2024-09-19T08:39:03.224537+0000","last_fullsized":"2024-09-19T08:39:03.224537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.243959+0000","last_change":"2024-09-19T08:39:03.243959+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.243959+0000","last_undegraded":"2024-09-19T08:39:03.243959+0000","last_fullsized":"2024-09-19T08:39:03.243959+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.248308+0000","last_change":"2024-09-19T08:39:03.248308+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.248308+0000","last_undegraded":"2024-09-19T08:39:03.248308+0000","last_fullsized":"2024-09-19T08:39:03.248308+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:03.224537+0000","last_change":"2024-09-19T08:39:03.224537+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.224537+0000","last_undegraded":"2024-09-19T08:39:03.224537+0000","last_fullsized":"2024-09-19T08:39:03.224537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.459475+0000","last_change":"2024-09-19T08:39:05.305633+0000","last_active":"2024-09-19T08:39:06.459475+0000","last_peered":"2024-09-19T08:39:06.459475+0000","last_clean":"2024-09-19T08:39:06.459475+0000","last_became_active":"2024-09-19T08:39:05.304913+0000","last_became_peered":"2024-09-19T08:39:05.304913+0000","last_unstale":"2024-09-19T08:39:06.459475+0000","last_undegraded":"2024-09-19T08:39:06.459475+0000","last_fullsized":"2024-09-19T08:39:06.459475+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:41:53.594289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:03.224537+0000","last_change":"2024-09-19T08:39:03.224537+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.224537+0000","last_undegraded":"2024-09-19T08:39:03.224537+0000","last_fullsized":"2024-09-19T08:39:03.224537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.248171+0000","last_change":"2024-09-19T08:39:03.248171+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.248171+0000","last_undegraded":"2024-09-19T08:39:03.248171+0000","last_fullsized":"2024-09-19T08:39:03.248171+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.245152+0000","last_change":"2024-09-19T08:39:03.245152+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.245152+0000","last_undegraded":"2024-09-19T08:39:03.245152+0000","last_fullsized":"2024-09-19T08:39:03.245152+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.244013+0000","last_change":"2024-09-19T08:39:03.244013+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.244013+0000","last_undegraded":"2024-09-19T08:39:03.244013+0000","last_fullsized":"2024-09-19T08:39:03.244013+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.244800+0000","last_change":"2024-09-19T08:39:03.244800+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.244800+0000","last_undegraded":"2024-09-19T08:39:03.244800+0000","last_fullsized":"2024-09-19T08:39:03.244800+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.299894+0000","last_change":"2024-09-19T08:39:03.299894+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.299894+0000","last_undegraded":"2024-09-19T08:39:03.299894+0000","last_fullsized":"2024-09-19T08:39:03.299894+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.458762+0000","last_change":"2024-09-19T08:39:04.285062+0000","last_active":"2024-09-19T08:39:06.458762+0000","last_peered":"2024-09-19T08:39:06.458762+0000","last_clean":"2024-09-19T08:39:06.458762+0000","last_became_active":"2024-09-19T08:39:04.284140+0000","last_became_peered":"2024-09-19T08:39:04.284140+0000","last_unstale":"2024-09-19T08:39:06.458762+0000","last_undegraded":"2024-09-19T08:39:06.458762+0000","last_fullsized":"2024-09-19T08:39:06.458762+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:00:00.495561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.245674+0000","last_change":"2024-09-19T08:39:03.245674+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.245674+0000","last_undegraded":"2024-09-19T08:39:03.245674+0000","last_fullsized":"2024-09-19T08:39:03.245674+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.248687+0000","last_change":"2024-09-19T08:39:03.248687+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.248687+0000","last_undegraded":"2024-09-19T08:39:03.248687+0000","last_fullsized":"2024-09-19T08:39:03.248687+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.246436+0000","last_change":"2024-09-19T08:39:03.246436+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.246436+0000","last_undegraded":"2024-09-19T08:39:03.246436+0000","last_fullsized":"2024-09-19T08:39:03.246436+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.299915+0000","last_change":"2024-09-19T08:39:03.299915+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.299915+0000","last_undegraded":"2024-09-19T08:39:03.299915+0000","last_fullsized":"2024-09-19T08:39:03.299915+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.246320+0000","last_change":"2024-09-19T08:39:03.246320+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.246320+0000","last_undegraded":"2024-09-19T08:39:03.246320+0000","last_fullsized":"2024-09-19T08:39:03.246320+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.458915+0000","last_change":"2024-09-19T08:39:05.276652+0000","last_active":"2024-09-19T08:39:06.458915+0000","last_peered":"2024-09-19T08:39:06.458915+0000","last_clean":"2024-09-19T08:39:06.458915+0000","last_became_active":"2024-09-19T08:39:05.275913+0000","last_became_peered":"2024-09-19T08:39:05.275913+0000","last_unstale":"2024-09-19T08:39:06.458915+0000","last_undegraded":"2024-09-19T08:39:06.458915+0000","last_fullsized":"2024-09-19T08:39:06.458915+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:36.037852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.299263+0000","last_change":"2024-09-19T08:39:03.299263+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.299263+0000","last_undegraded":"2024-09-19T08:39:03.299263+0000","last_fullsized":"2024-09-19T08:39:03.299263+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.246224+0000","last_change":"2024-09-19T08:39:03.246224+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.246224+0000","last_undegraded":"2024-09-19T08:39:03.246224+0000","last_fullsized":"2024-09-19T08:39:03.246224+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.243968+0000","last_change":"2024-09-19T08:39:03.243968+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.243968+0000","last_undegraded":"2024-09-19T08:39:03.243968+0000","last_fullsized":"2024-09-19T08:39:03.243968+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:03.224537+0000","last_change":"2024-09-19T08:39:03.224537+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.224537+0000","last_undegraded":"2024-09-19T08:39:03.224537+0000","last_fullsized":"2024-09-19T08:39:03.224537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.458454+0000","last_change":"2024-09-19T08:39:04.294673+0000","last_active":"2024-09-19T08:39:06.458454+0000","last_peered":"2024-09-19T08:39:06.458454+0000","last_clean":"2024-09-19T08:39:06.458454+0000","last_became_active":"2024-09-19T08:39:04.293913+0000","last_became_peered":"2024-09-19T08:39:04.293913+0000","last_unstale":"2024-09-19T08:39:06.458454+0000","last_undegraded":"2024-09-19T08:39:06.458454+0000","last_fullsized":"2024-09-19T08:39:06.458454+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:12.830812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":182,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-19T08:39:03.254315+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:39:03.254315+0000","last_peered":"2024-09-19T08:39:03.254315+0000","last_clean":"2024-09-19T08:39:03.254315+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:39:03.254315+0000","last_undegraded":"2024-09-19T08:39:03.254315+0000","last_fullsized":"2024-09-19T08:39:03.254315+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":10,"acting":10,"num_store_stats":1},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":50,"acting":50,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509454,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":897024,"data_stored":700868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":315392,"data_stored":116317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":315392,"data_stored":116317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509453,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":315392,"data_stored":116317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509455,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":303104,"data_stored":110500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509453,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27916,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343924,"statfs":{"total":96636764160,"available":96608178176,"internally_reserved":0,"allocated":909312,"data_stored":706685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":360448,"data_stored":136275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27324,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":303104,"data_stored":110500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:39:08.314 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:39:08.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.463+0000 7f367b948640 1 Processor -- start 2024-09-19T08:39:08.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.463+0000 7f367b948640 1 -- start start 2024-09-19T08:39:08.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.467+0000 7f367b948640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f367406bc90 0x7f367406c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:08.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.467+0000 7f367b948640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f36740ce840 0x7f36740cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:08.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.467+0000 7f367b948640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36740ccc60 con 0x7f3674158fd0 2024-09-19T08:39:08.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.467+0000 7f367b948640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f36741593b0 con 0x7f36740ce840 2024-09-19T08:39:08.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.467+0000 7f367b948640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f36740ccf40 con 0x7f367406bc90 2024-09-19T08:39:08.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.467+0000 7f3679ee5640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f36740ce840 0x7f36740cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:08.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.467+0000 7f3679ee5640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f36740ce840 0x7f36740cec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36396/0 (socket says 172.21.15.98:36396) 2024-09-19T08:39:08.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.467+0000 7f3679ee5640 1 -- 172.21.15.98:0/3510535289 learned_addr learned my addr 172.21.15.98:0/3510535289 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:08.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.467+0000 7f3678ee3640 1 --2- 172.21.15.98:0/3510535289 >> v2:172.21.15.139:3300/0 conn(0x7f367406bc90 0x7f367406c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:08.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.467+0000 7f3679ee5640 1 -- 172.21.15.98:0/3510535289 >> v2:172.21.15.139:3300/0 conn(0x7f367406bc90 msgr2=0x7f367406c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:08.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.467+0000 7f3679ee5640 1 --2- 172.21.15.98:0/3510535289 >> v2:172.21.15.139:3300/0 conn(0x7f367406bc90 0x7f367406c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:08.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.467+0000 7f3679ee5640 1 -- 172.21.15.98:0/3510535289 >> v1:172.21.15.139:6789/0 conn(0x7f3674158fd0 legacy=0x7f36740b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:08.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.467+0000 7f3679ee5640 1 -- 172.21.15.98:0/3510535289 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36740cd220 con 0x7f36740ce840 2024-09-19T08:39:08.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.471+0000 7f3679ee5640 1 --2- 172.21.15.98:0/3510535289 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36740ce840 0x7f36740cec70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f36740cdef0 tx=0x7f366c01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245ec41581bc5752 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:08.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.471+0000 7f366bfff640 1 -- 172.21.15.98:0/3510535289 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366c0a56d0 con 0x7f36740ce840 2024-09-19T08:39:08.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.471+0000 7f366bfff640 1 -- 172.21.15.98:0/3510535289 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f366c0a5870 con 0x7f36740ce840 2024-09-19T08:39:08.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.471+0000 7f366bfff640 1 -- 172.21.15.98:0/3510535289 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366c0a5c40 con 0x7f36740ce840 2024-09-19T08:39:08.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.471+0000 7f367b948640 1 -- 172.21.15.98:0/3510535289 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36740ce840 msgr2=0x7f36740cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:08.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.471+0000 7f367b948640 1 --2- 172.21.15.98:0/3510535289 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36740ce840 0x7f36740cec70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f36740cdef0 tx=0x7f366c01c920 comp rx=0 tx=0).stop 2024-09-19T08:39:08.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.471+0000 7f367b948640 1 -- 172.21.15.98:0/3510535289 shutdown_connections 2024-09-19T08:39:08.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.471+0000 7f367b948640 1 --2- 172.21.15.98:0/3510535289 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36740ce840 0x7f36740cec70 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:08.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.471+0000 7f367b948640 1 --2- 172.21.15.98:0/3510535289 >> v2:172.21.15.139:3300/0 conn(0x7f367406bc90 0x7f367406c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:08.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.471+0000 7f367b948640 1 -- 172.21.15.98:0/3510535289 >> 172.21.15.98:0/3510535289 conn(0x7f36740b2010 msgr2=0x7f36740b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:08.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.471+0000 7f367b948640 1 -- 172.21.15.98:0/3510535289 shutdown_connections 2024-09-19T08:39:08.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.471+0000 7f367b948640 1 -- 172.21.15.98:0/3510535289 wait complete. 2024-09-19T08:39:08.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.471+0000 7f367b948640 1 Processor -- start 2024-09-19T08:39:08.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.471+0000 7f367b948640 1 -- start start 2024-09-19T08:39:08.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.471+0000 7f367b948640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f367406bc90 0x7f3674150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:08.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.471+0000 7f367b948640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36740ce840 0x7f3674154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:08.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.471+0000 7f367b948640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3674158fd0 0x7f367415a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:08.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.471+0000 7f367b948640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f367415c920 con 0x7f367406bc90 2024-09-19T08:39:08.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.471+0000 7f367b948640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f367415c7a0 con 0x7f36740ce840 2024-09-19T08:39:08.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.471+0000 7f367b948640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f367406d170 con 0x7f3674158fd0 2024-09-19T08:39:08.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.471+0000 7f36796e4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f367406bc90 0x7f3674150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:08.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.471+0000 7f36796e4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f367406bc90 0x7f3674150690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36408/0 (socket says 172.21.15.98:36408) 2024-09-19T08:39:08.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.471+0000 7f36796e4640 1 -- 172.21.15.98:0/784377509 learned_addr learned my addr 172.21.15.98:0/784377509 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:08.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.471+0000 7f3678ee3640 1 --2- 172.21.15.98:0/784377509 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36740ce840 0x7f3674154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:08.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.471+0000 7f3679ee5640 1 --2- 172.21.15.98:0/784377509 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3674158fd0 0x7f367415a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:08.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.471+0000 7f36796e4640 1 -- 172.21.15.98:0/784377509 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3674158fd0 msgr2=0x7f367415a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:08.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.471+0000 7f36796e4640 1 --2- 172.21.15.98:0/784377509 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3674158fd0 0x7f367415a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:08.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.471+0000 7f36796e4640 1 -- 172.21.15.98:0/784377509 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36740ce840 msgr2=0x7f3674154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:08.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.471+0000 7f36796e4640 1 --2- 172.21.15.98:0/784377509 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36740ce840 0x7f3674154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:08.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.471+0000 7f36796e4640 1 -- 172.21.15.98:0/784377509 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36741552c0 con 0x7f367406bc90 2024-09-19T08:39:08.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.471+0000 7f3678ee3640 1 --2- 172.21.15.98:0/784377509 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36740ce840 0x7f3674154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:08.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.471+0000 7f36796e4640 1 --2- 172.21.15.98:0/784377509 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f367406bc90 0x7f3674150690 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f367000b410 tx=0x7f367000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:08.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.471+0000 7f3669ffb640 1 -- 172.21.15.98:0/784377509 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3670015030 con 0x7f367406bc90 2024-09-19T08:39:08.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.471+0000 7f3669ffb640 1 -- 172.21.15.98:0/784377509 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3670004020 con 0x7f367406bc90 2024-09-19T08:39:08.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.471+0000 7f3669ffb640 1 -- 172.21.15.98:0/784377509 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3670004450 con 0x7f367406bc90 2024-09-19T08:39:08.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.471+0000 7f367b948640 1 -- 172.21.15.98:0/784377509 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36741555a0 con 0x7f367406bc90 2024-09-19T08:39:08.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.475+0000 7f367b948640 1 -- 172.21.15.98:0/784377509 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3674155b60 con 0x7f367406bc90 2024-09-19T08:39:08.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.475+0000 7f367b948640 1 -- 172.21.15.98:0/784377509 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f367406c120 con 0x7f367406bc90 2024-09-19T08:39:08.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.475+0000 7f3669ffb640 1 -- 172.21.15.98:0/784377509 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3670018070 con 0x7f367406bc90 2024-09-19T08:39:08.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.475+0000 7f3669ffb640 1 --2- 172.21.15.98:0/784377509 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f36580678f0 0x7f3658069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:08.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.475+0000 7f3669ffb640 1 -- 172.21.15.98:0/784377509 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f3670085910 con 0x7f367406bc90 2024-09-19T08:39:08.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.475+0000 7f3678ee3640 1 --2- 172.21.15.98:0/784377509 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f36580678f0 0x7f3658069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:08.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.475+0000 7f3678ee3640 1 --2- 172.21.15.98:0/784377509 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f36580678f0 0x7f3658069db0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3674069f60 tx=0x7f3664008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:08.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.479+0000 7f3669ffb640 1 -- 172.21.15.98:0/784377509 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f36700563c0 con 0x7f367406bc90 2024-09-19T08:39:08.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.639+0000 7f367b948640 1 -- 172.21.15.98:0/784377509 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f36740af9b0 con 0x7f36580678f0 2024-09-19T08:39:08.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.651+0000 7f3669ffb640 1 -- 172.21.15.98:0/784377509 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240452 (secure 0 0 0) 0x7f36740af9b0 con 0x7f36580678f0 2024-09-19T08:39:08.656 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:39:08.657 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:39:08.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.655+0000 7f367b948640 1 -- 172.21.15.98:0/784377509 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f36580678f0 msgr2=0x7f3658069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:08.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.655+0000 7f367b948640 1 --2- 172.21.15.98:0/784377509 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f36580678f0 0x7f3658069db0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3674069f60 tx=0x7f3664008040 comp rx=0 tx=0).stop 2024-09-19T08:39:08.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.655+0000 7f367b948640 1 -- 172.21.15.98:0/784377509 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f367406bc90 msgr2=0x7f3674150690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:08.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.655+0000 7f367b948640 1 --2- 172.21.15.98:0/784377509 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f367406bc90 0x7f3674150690 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f367000b410 tx=0x7f367000b8d0 comp rx=0 tx=0).stop 2024-09-19T08:39:08.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.655+0000 7f367b948640 1 -- 172.21.15.98:0/784377509 shutdown_connections 2024-09-19T08:39:08.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.655+0000 7f367b948640 1 --2- 172.21.15.98:0/784377509 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f36580678f0 0x7f3658069db0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:08.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.655+0000 7f367b948640 1 --2- 172.21.15.98:0/784377509 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3674158fd0 0x7f367415a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:08.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.655+0000 7f367b948640 1 --2- 172.21.15.98:0/784377509 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36740ce840 0x7f3674154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:08.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.655+0000 7f367b948640 1 --2- 172.21.15.98:0/784377509 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f367406bc90 0x7f3674150690 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:08.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.655+0000 7f367b948640 1 -- 172.21.15.98:0/784377509 >> 172.21.15.98:0/784377509 conn(0x7f36740b2010 msgr2=0x7f3674165330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:08.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.655+0000 7f367b948640 1 -- 172.21.15.98:0/784377509 shutdown_connections 2024-09-19T08:39:08.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.655+0000 7f367b948640 1 -- 172.21.15.98:0/784377509 wait complete. 2024-09-19T08:39:08.675 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2024-09-19T08:39:08.040412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590950,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":163,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121,"ondisk_log_size":121,"up":62,"acting":62,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":219856,"kb_used_data":3632,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754864,"statfs":{"total":773094113280,"available":772868980736,"internally_reserved":0,"allocated":3719168,"data_stored":2113779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.801923"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.458834+0000","last_change":"2024-09-19T08:39:05.300955+0000","last_active":"2024-09-19T08:39:06.458834+0000","last_peered":"2024-09-19T08:39:06.458834+0000","last_clean":"2024-09-19T08:39:06.458834+0000","last_became_active":"2024-09-19T08:39:05.300228+0000","last_became_peered":"2024-09-19T08:39:05.300228+0000","last_unstale":"2024-09-19T08:39:06.458834+0000","last_undegraded":"2024-09-19T08:39:06.458834+0000","last_fullsized":"2024-09-19T08:39:06.458834+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:05.814581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.459141+0000","last_change":"2024-09-19T08:39:05.276219+0000","last_active":"2024-09-19T08:39:06.459141+0000","last_peered":"2024-09-19T08:39:06.459141+0000","last_clean":"2024-09-19T08:39:06.459141+0000","last_became_active":"2024-09-19T08:39:05.275466+0000","last_became_peered":"2024-09-19T08:39:05.275466+0000","last_unstale":"2024-09-19T08:39:06.459141+0000","last_undegraded":"2024-09-19T08:39:06.459141+0000","last_fullsized":"2024-09-19T08:39:06.459141+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:09.038404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.459079+0000","last_change":"2024-09-19T08:39:04.292831+0000","last_active":"2024-09-19T08:39:06.459079+0000","last_peered":"2024-09-19T08:39:06.459079+0000","last_clean":"2024-09-19T08:39:06.459079+0000","last_became_active":"2024-09-19T08:39:04.292058+0000","last_became_peered":"2024-09-19T08:39:04.292058+0000","last_unstale":"2024-09-19T08:39:06.459079+0000","last_undegraded":"2024-09-19T08:39:06.459079+0000","last_fullsized":"2024-09-19T08:39:06.459079+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:38:41.573373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.459180+0000","last_change":"2024-09-19T08:39:05.305411+0000","last_active":"2024-09-19T08:39:06.459180+0000","last_peered":"2024-09-19T08:39:06.459180+0000","last_clean":"2024-09-19T08:39:06.459180+0000","last_became_active":"2024-09-19T08:39:05.304823+0000","last_became_peered":"2024-09-19T08:39:05.304823+0000","last_unstale":"2024-09-19T08:39:06.459180+0000","last_undegraded":"2024-09-19T08:39:06.459180+0000","last_fullsized":"2024-09-19T08:39:06.459180+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:11.979445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.300418+0000","last_change":"2024-09-19T08:39:03.300418+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.300418+0000","last_undegraded":"2024-09-19T08:39:03.300418+0000","last_fullsized":"2024-09-19T08:39:03.300418+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:03.224537+0000","last_change":"2024-09-19T08:39:03.224537+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.224537+0000","last_undegraded":"2024-09-19T08:39:03.224537+0000","last_fullsized":"2024-09-19T08:39:03.224537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.243996+0000","last_change":"2024-09-19T08:39:03.243996+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.243996+0000","last_undegraded":"2024-09-19T08:39:03.243996+0000","last_fullsized":"2024-09-19T08:39:03.243996+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.245284+0000","last_change":"2024-09-19T08:39:03.245284+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.245284+0000","last_undegraded":"2024-09-19T08:39:03.245284+0000","last_fullsized":"2024-09-19T08:39:03.245284+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.243944+0000","last_change":"2024-09-19T08:39:03.243944+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.243944+0000","last_undegraded":"2024-09-19T08:39:03.243944+0000","last_fullsized":"2024-09-19T08:39:03.243944+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:03.224537+0000","last_change":"2024-09-19T08:39:03.224537+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.224537+0000","last_undegraded":"2024-09-19T08:39:03.224537+0000","last_fullsized":"2024-09-19T08:39:03.224537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.243920+0000","last_change":"2024-09-19T08:39:03.243920+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.243920+0000","last_undegraded":"2024-09-19T08:39:03.243920+0000","last_fullsized":"2024-09-19T08:39:03.243920+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:03.224537+0000","last_change":"2024-09-19T08:39:03.224537+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.224537+0000","last_undegraded":"2024-09-19T08:39:03.224537+0000","last_fullsized":"2024-09-19T08:39:03.224537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.299294+0000","last_change":"2024-09-19T08:39:03.299294+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.299294+0000","last_undegraded":"2024-09-19T08:39:03.299294+0000","last_fullsized":"2024-09-19T08:39:03.299294+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.249325+0000","last_change":"2024-09-19T08:39:03.249325+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.249325+0000","last_undegraded":"2024-09-19T08:39:03.249325+0000","last_fullsized":"2024-09-19T08:39:03.249325+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:03.224537+0000","last_change":"2024-09-19T08:39:03.224537+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.224537+0000","last_undegraded":"2024-09-19T08:39:03.224537+0000","last_fullsized":"2024-09-19T08:39:03.224537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.243959+0000","last_change":"2024-09-19T08:39:03.243959+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.243959+0000","last_undegraded":"2024-09-19T08:39:03.243959+0000","last_fullsized":"2024-09-19T08:39:03.243959+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.248308+0000","last_change":"2024-09-19T08:39:03.248308+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.248308+0000","last_undegraded":"2024-09-19T08:39:03.248308+0000","last_fullsized":"2024-09-19T08:39:03.248308+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:03.224537+0000","last_change":"2024-09-19T08:39:03.224537+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.224537+0000","last_undegraded":"2024-09-19T08:39:03.224537+0000","last_fullsized":"2024-09-19T08:39:03.224537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.459475+0000","last_change":"2024-09-19T08:39:05.305633+0000","last_active":"2024-09-19T08:39:06.459475+0000","last_peered":"2024-09-19T08:39:06.459475+0000","last_clean":"2024-09-19T08:39:06.459475+0000","last_became_active":"2024-09-19T08:39:05.304913+0000","last_became_peered":"2024-09-19T08:39:05.304913+0000","last_unstale":"2024-09-19T08:39:06.459475+0000","last_undegraded":"2024-09-19T08:39:06.459475+0000","last_fullsized":"2024-09-19T08:39:06.459475+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:41:53.594289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:03.224537+0000","last_change":"2024-09-19T08:39:03.224537+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.224537+0000","last_undegraded":"2024-09-19T08:39:03.224537+0000","last_fullsized":"2024-09-19T08:39:03.224537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.248171+0000","last_change":"2024-09-19T08:39:03.248171+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.248171+0000","last_undegraded":"2024-09-19T08:39:03.248171+0000","last_fullsized":"2024-09-19T08:39:03.248171+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.245152+0000","last_change":"2024-09-19T08:39:03.245152+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.245152+0000","last_undegraded":"2024-09-19T08:39:03.245152+0000","last_fullsized":"2024-09-19T08:39:03.245152+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.244013+0000","last_change":"2024-09-19T08:39:03.244013+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.244013+0000","last_undegraded":"2024-09-19T08:39:03.244013+0000","last_fullsized":"2024-09-19T08:39:03.244013+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.244800+0000","last_change":"2024-09-19T08:39:03.244800+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.244800+0000","last_undegraded":"2024-09-19T08:39:03.244800+0000","last_fullsized":"2024-09-19T08:39:03.244800+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.299894+0000","last_change":"2024-09-19T08:39:03.299894+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.299894+0000","last_undegraded":"2024-09-19T08:39:03.299894+0000","last_fullsized":"2024-09-19T08:39:03.299894+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.458762+0000","last_change":"2024-09-19T08:39:04.285062+0000","last_active":"2024-09-19T08:39:06.458762+0000","last_peered":"2024-09-19T08:39:06.458762+0000","last_clean":"2024-09-19T08:39:06.458762+0000","last_became_active":"2024-09-19T08:39:04.284140+0000","last_became_peered":"2024-09-19T08:39:04.284140+0000","last_unstale":"2024-09-19T08:39:06.458762+0000","last_undegraded":"2024-09-19T08:39:06.458762+0000","last_fullsized":"2024-09-19T08:39:06.458762+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:00:00.495561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.245674+0000","last_change":"2024-09-19T08:39:03.245674+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.245674+0000","last_undegraded":"2024-09-19T08:39:03.245674+0000","last_fullsized":"2024-09-19T08:39:03.245674+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.248687+0000","last_change":"2024-09-19T08:39:03.248687+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.248687+0000","last_undegraded":"2024-09-19T08:39:03.248687+0000","last_fullsized":"2024-09-19T08:39:03.248687+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.246436+0000","last_change":"2024-09-19T08:39:03.246436+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.246436+0000","last_undegraded":"2024-09-19T08:39:03.246436+0000","last_fullsized":"2024-09-19T08:39:03.246436+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.299915+0000","last_change":"2024-09-19T08:39:03.299915+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.299915+0000","last_undegraded":"2024-09-19T08:39:03.299915+0000","last_fullsized":"2024-09-19T08:39:03.299915+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.246320+0000","last_change":"2024-09-19T08:39:03.246320+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.246320+0000","last_undegraded":"2024-09-19T08:39:03.246320+0000","last_fullsized":"2024-09-19T08:39:03.246320+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.458915+0000","last_change":"2024-09-19T08:39:05.276652+0000","last_active":"2024-09-19T08:39:06.458915+0000","last_peered":"2024-09-19T08:39:06.458915+0000","last_clean":"2024-09-19T08:39:06.458915+0000","last_became_active":"2024-09-19T08:39:05.275913+0000","last_became_peered":"2024-09-19T08:39:05.275913+0000","last_unstale":"2024-09-19T08:39:06.458915+0000","last_undegraded":"2024-09-19T08:39:06.458915+0000","last_fullsized":"2024-09-19T08:39:06.458915+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:36.037852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.299263+0000","last_change":"2024-09-19T08:39:03.299263+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.299263+0000","last_undegraded":"2024-09-19T08:39:03.299263+0000","last_fullsized":"2024-09-19T08:39:03.299263+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.246224+0000","last_change":"2024-09-19T08:39:03.246224+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.246224+0000","last_undegraded":"2024-09-19T08:39:03.246224+0000","last_fullsized":"2024-09-19T08:39:03.246224+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.243968+0000","last_change":"2024-09-19T08:39:03.243968+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.243968+0000","last_undegraded":"2024-09-19T08:39:03.243968+0000","last_fullsized":"2024-09-19T08:39:03.243968+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:03.224537+0000","last_change":"2024-09-19T08:39:03.224537+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.224537+0000","last_undegraded":"2024-09-19T08:39:03.224537+0000","last_fullsized":"2024-09-19T08:39:03.224537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.458454+0000","last_change":"2024-09-19T08:39:04.294673+0000","last_active":"2024-09-19T08:39:06.458454+0000","last_peered":"2024-09-19T08:39:06.458454+0000","last_clean":"2024-09-19T08:39:06.458454+0000","last_became_active":"2024-09-19T08:39:04.293913+0000","last_became_peered":"2024-09-19T08:39:04.293913+0000","last_unstale":"2024-09-19T08:39:06.458454+0000","last_undegraded":"2024-09-19T08:39:06.458454+0000","last_fullsized":"2024-09-19T08:39:06.458454+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:12.830812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":182,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-19T08:39:03.254315+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:39:03.254315+0000","last_peered":"2024-09-19T08:39:03.254315+0000","last_clean":"2024-09-19T08:39:03.254315+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:39:03.254315+0000","last_undegraded":"2024-09-19T08:39:03.254315+0000","last_fullsized":"2024-09-19T08:39:03.254315+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":10,"acting":10,"num_store_stats":1},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":50,"acting":50,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509454,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":897024,"data_stored":700868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":315392,"data_stored":116317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":315392,"data_stored":116317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509453,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":315392,"data_stored":116317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509455,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":303104,"data_stored":110500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509453,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27916,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343924,"statfs":{"total":96636764160,"available":96608178176,"internally_reserved":0,"allocated":909312,"data_stored":706685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":360448,"data_stored":136275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27324,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":303104,"data_stored":110500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:39:08.679 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:39:08.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.823+0000 7f99ab197640 1 Processor -- start 2024-09-19T08:39:08.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.823+0000 7f99ab197640 1 -- start start 2024-09-19T08:39:08.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.827+0000 7f99ab197640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f99a40b0000 0x7f99a415d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:08.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.827+0000 7f99ab197640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99a40d0140 0x7f99a4072b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:08.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.827+0000 7f99ab197640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f99a40730d0 0x7f99a4071520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:08.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.827+0000 7f99ab197640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f99a4073520 con 0x7f99a40b0000 2024-09-19T08:39:08.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.827+0000 7f99ab197640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f99a406eb00 con 0x7f99a40730d0 2024-09-19T08:39:08.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.827+0000 7f99ab197640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f99a406fe40 con 0x7f99a40d0140 2024-09-19T08:39:08.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.827+0000 7f99a8f33640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f99a40b0000 0x7f99a415d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:08.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.827+0000 7f99a8f33640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f99a40b0000 0x7f99a415d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36422/0 (socket says 172.21.15.98:36422) 2024-09-19T08:39:08.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.827+0000 7f99a8f33640 1 -- 172.21.15.98:0/1707937033 learned_addr learned my addr 172.21.15.98:0/1707937033 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:08.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.827+0000 7f99a9734640 1 --2- 172.21.15.98:0/1707937033 >> v2:172.21.15.139:3300/0 conn(0x7f99a40730d0 0x7f99a4071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:08.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.827+0000 7f99a3fff640 1 --2- 172.21.15.98:0/1707937033 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99a40d0140 0x7f99a4072b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:08.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.827+0000 7f99a8f33640 1 -- 172.21.15.98:0/1707937033 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99a40d0140 msgr2=0x7f99a4072b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:08.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.827+0000 7f99a8f33640 1 --2- 172.21.15.98:0/1707937033 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99a40d0140 0x7f99a4072b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:08.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.827+0000 7f99a8f33640 1 -- 172.21.15.98:0/1707937033 >> v2:172.21.15.139:3300/0 conn(0x7f99a40730d0 msgr2=0x7f99a4071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:08.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.827+0000 7f99a8f33640 1 --2- 172.21.15.98:0/1707937033 >> v2:172.21.15.139:3300/0 conn(0x7f99a40730d0 0x7f99a4071520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:08.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.827+0000 7f99a8f33640 1 -- 172.21.15.98:0/1707937033 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a406eda0 con 0x7f99a40b0000 2024-09-19T08:39:08.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.827+0000 7f99a3fff640 1 --2- 172.21.15.98:0/1707937033 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99a40d0140 0x7f99a4072b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:08.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.827+0000 7f99a8f33640 1 --2- 172.21.15.98:0/1707937033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99a40b0000 0x7f99a415d250 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f99a4069a10 tx=0x7f999801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd2558770913eaba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:08.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.827+0000 7f99a37fe640 1 -- 172.21.15.98:0/1707937033 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99980b0070 con 0x7f99a40b0000 2024-09-19T08:39:08.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.827+0000 7f99a37fe640 1 -- 172.21.15.98:0/1707937033 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99980ab600 con 0x7f99a40b0000 2024-09-19T08:39:08.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.827+0000 7f99a37fe640 1 -- 172.21.15.98:0/1707937033 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99980ab9d0 con 0x7f99a40b0000 2024-09-19T08:39:08.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.831+0000 7f99ab197640 1 -- 172.21.15.98:0/1707937033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99a40b0000 msgr2=0x7f99a415d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:08.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.831+0000 7f99ab197640 1 --2- 172.21.15.98:0/1707937033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99a40b0000 0x7f99a415d250 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f99a4069a10 tx=0x7f999801c260 comp rx=0 tx=0).stop 2024-09-19T08:39:08.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.831+0000 7f99ab197640 1 -- 172.21.15.98:0/1707937033 shutdown_connections 2024-09-19T08:39:08.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.831+0000 7f99ab197640 1 --2- 172.21.15.98:0/1707937033 >> v2:172.21.15.139:3300/0 conn(0x7f99a40730d0 0x7f99a4071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:08.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.831+0000 7f99ab197640 1 --2- 172.21.15.98:0/1707937033 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99a40d0140 0x7f99a4072b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:08.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.831+0000 7f99ab197640 1 --2- 172.21.15.98:0/1707937033 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99a40b0000 0x7f99a415d250 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:08.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.831+0000 7f99ab197640 1 -- 172.21.15.98:0/1707937033 >> 172.21.15.98:0/1707937033 conn(0x7f99a40d0980 msgr2=0x7f99a40d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:08.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.831+0000 7f99ab197640 1 -- 172.21.15.98:0/1707937033 shutdown_connections 2024-09-19T08:39:08.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.831+0000 7f99ab197640 1 -- 172.21.15.98:0/1707937033 wait complete. 2024-09-19T08:39:08.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.831+0000 7f99ab197640 1 Processor -- start 2024-09-19T08:39:08.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.831+0000 7f99ab197640 1 -- start start 2024-09-19T08:39:08.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.831+0000 7f99ab197640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a40730d0 0x7f99a4153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:08.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.831+0000 7f99ab197640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99a40b0000 0x7f99a40ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:08.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.831+0000 7f99ab197640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99a40d0140 0x7f99a4067a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:08.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.831+0000 7f99ab197640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f99a40ce550 con 0x7f99a40b0000 2024-09-19T08:39:08.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.831+0000 7f99ab197640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f99a406f920 con 0x7f99a40730d0 2024-09-19T08:39:08.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.831+0000 7f99ab197640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f99a40ce6d0 con 0x7f99a40d0140 2024-09-19T08:39:08.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.831+0000 7f99a3fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99a40b0000 0x7f99a40ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:08.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.831+0000 7f99a3fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99a40b0000 0x7f99a40ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36428/0 (socket says 172.21.15.98:36428) 2024-09-19T08:39:08.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.831+0000 7f99a3fff640 1 -- 172.21.15.98:0/3334485641 learned_addr learned my addr 172.21.15.98:0/3334485641 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:08.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.831+0000 7f99a8f33640 1 --2- 172.21.15.98:0/3334485641 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a40730d0 0x7f99a4153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:08.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.831+0000 7f99a3fff640 1 -- 172.21.15.98:0/3334485641 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99a40d0140 msgr2=0x7f99a4067a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:08.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.831+0000 7f99a9734640 1 --2- 172.21.15.98:0/3334485641 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99a40d0140 0x7f99a4067a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:08.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.831+0000 7f99a3fff640 1 --2- 172.21.15.98:0/3334485641 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99a40d0140 0x7f99a4067a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:08.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.831+0000 7f99a3fff640 1 -- 172.21.15.98:0/3334485641 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a40730d0 msgr2=0x7f99a4153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:08.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.831+0000 7f99a3fff640 1 --2- 172.21.15.98:0/3334485641 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a40730d0 0x7f99a4153e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:08.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.831+0000 7f99a3fff640 1 -- 172.21.15.98:0/3334485641 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a41f51b0 con 0x7f99a40b0000 2024-09-19T08:39:08.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.831+0000 7f99a8f33640 1 --2- 172.21.15.98:0/3334485641 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a40730d0 0x7f99a4153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:08.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.831+0000 7f99a3fff640 1 --2- 172.21.15.98:0/3334485641 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99a40b0000 0x7f99a40ca240 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f999000c7f0 tx=0x7f999000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:08.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.831+0000 7f99a17fa640 1 -- 172.21.15.98:0/3334485641 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9990015070 con 0x7f99a40b0000 2024-09-19T08:39:08.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.831+0000 7f99a17fa640 1 -- 172.21.15.98:0/3334485641 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9990007820 con 0x7f99a40b0000 2024-09-19T08:39:08.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.831+0000 7f99a17fa640 1 -- 172.21.15.98:0/3334485641 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9990007c10 con 0x7f99a40b0000 2024-09-19T08:39:08.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.831+0000 7f99ab197640 1 -- 172.21.15.98:0/3334485641 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99a41f5450 con 0x7f99a40b0000 2024-09-19T08:39:08.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.831+0000 7f99ab197640 1 -- 172.21.15.98:0/3334485641 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99a41f5a10 con 0x7f99a40b0000 2024-09-19T08:39:08.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.835+0000 7f99ab197640 1 -- 172.21.15.98:0/3334485641 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99a415d250 con 0x7f99a40b0000 2024-09-19T08:39:08.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.835+0000 7f99a17fa640 1 -- 172.21.15.98:0/3334485641 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9990018070 con 0x7f99a40b0000 2024-09-19T08:39:08.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.835+0000 7f99a17fa640 1 --2- 172.21.15.98:0/3334485641 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f99880678c0 0x7f9988069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:08.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.835+0000 7f99a17fa640 1 -- 172.21.15.98:0/3334485641 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f9990085dc0 con 0x7f99a40b0000 2024-09-19T08:39:08.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.835+0000 7f99a8f33640 1 --2- 172.21.15.98:0/3334485641 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f99880678c0 0x7f9988069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:08.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.835+0000 7f99a8f33640 1 --2- 172.21.15.98:0/3334485641 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f99880678c0 0x7f9988069d80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f99980ad000 tx=0x7f9998008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:08.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.839+0000 7f99a17fa640 1 -- 172.21.15.98:0/3334485641 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9990056900 con 0x7f99a40b0000 2024-09-19T08:39:09.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:08.999+0000 7f99ab197640 1 -- 172.21.15.98:0/3334485641 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f99a40cef40 con 0x7f99880678c0 2024-09-19T08:39:09.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.015+0000 7f99a17fa640 1 -- 172.21.15.98:0/3334485641 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240452 (secure 0 0 0) 0x7f99a40cef40 con 0x7f99880678c0 2024-09-19T08:39:09.020 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:39:09.022 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:39:09.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.019+0000 7f99ab197640 1 -- 172.21.15.98:0/3334485641 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f99880678c0 msgr2=0x7f9988069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:09.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.019+0000 7f99ab197640 1 --2- 172.21.15.98:0/3334485641 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f99880678c0 0x7f9988069d80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f99980ad000 tx=0x7f9998008040 comp rx=0 tx=0).stop 2024-09-19T08:39:09.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.019+0000 7f99ab197640 1 -- 172.21.15.98:0/3334485641 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99a40b0000 msgr2=0x7f99a40ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:09.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.019+0000 7f99ab197640 1 --2- 172.21.15.98:0/3334485641 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99a40b0000 0x7f99a40ca240 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f999000c7f0 tx=0x7f999000ccb0 comp rx=0 tx=0).stop 2024-09-19T08:39:09.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.019+0000 7f99ab197640 1 -- 172.21.15.98:0/3334485641 shutdown_connections 2024-09-19T08:39:09.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.019+0000 7f99ab197640 1 --2- 172.21.15.98:0/3334485641 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f99880678c0 0x7f9988069d80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:09.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.019+0000 7f99ab197640 1 --2- 172.21.15.98:0/3334485641 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99a40d0140 0x7f99a4067a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:09.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.019+0000 7f99ab197640 1 --2- 172.21.15.98:0/3334485641 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99a40b0000 0x7f99a40ca240 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:09.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.019+0000 7f99ab197640 1 --2- 172.21.15.98:0/3334485641 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99a40730d0 0x7f99a4153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:09.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.019+0000 7f99ab197640 1 -- 172.21.15.98:0/3334485641 >> 172.21.15.98:0/3334485641 conn(0x7f99a40d0980 msgr2=0x7f99a415f190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:09.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.019+0000 7f99ab197640 1 -- 172.21.15.98:0/3334485641 shutdown_connections 2024-09-19T08:39:09.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.019+0000 7f99ab197640 1 -- 172.21.15.98:0/3334485641 wait complete. 2024-09-19T08:39:09.038 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2024-09-19T08:39:08.040412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590950,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":163,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121,"ondisk_log_size":121,"up":62,"acting":62,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":219856,"kb_used_data":3632,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754864,"statfs":{"total":773094113280,"available":772868980736,"internally_reserved":0,"allocated":3719168,"data_stored":2113779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.801923"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.458834+0000","last_change":"2024-09-19T08:39:05.300955+0000","last_active":"2024-09-19T08:39:06.458834+0000","last_peered":"2024-09-19T08:39:06.458834+0000","last_clean":"2024-09-19T08:39:06.458834+0000","last_became_active":"2024-09-19T08:39:05.300228+0000","last_became_peered":"2024-09-19T08:39:05.300228+0000","last_unstale":"2024-09-19T08:39:06.458834+0000","last_undegraded":"2024-09-19T08:39:06.458834+0000","last_fullsized":"2024-09-19T08:39:06.458834+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:05.814581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.459141+0000","last_change":"2024-09-19T08:39:05.276219+0000","last_active":"2024-09-19T08:39:06.459141+0000","last_peered":"2024-09-19T08:39:06.459141+0000","last_clean":"2024-09-19T08:39:06.459141+0000","last_became_active":"2024-09-19T08:39:05.275466+0000","last_became_peered":"2024-09-19T08:39:05.275466+0000","last_unstale":"2024-09-19T08:39:06.459141+0000","last_undegraded":"2024-09-19T08:39:06.459141+0000","last_fullsized":"2024-09-19T08:39:06.459141+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:09.038404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.459079+0000","last_change":"2024-09-19T08:39:04.292831+0000","last_active":"2024-09-19T08:39:06.459079+0000","last_peered":"2024-09-19T08:39:06.459079+0000","last_clean":"2024-09-19T08:39:06.459079+0000","last_became_active":"2024-09-19T08:39:04.292058+0000","last_became_peered":"2024-09-19T08:39:04.292058+0000","last_unstale":"2024-09-19T08:39:06.459079+0000","last_undegraded":"2024-09-19T08:39:06.459079+0000","last_fullsized":"2024-09-19T08:39:06.459079+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:38:41.573373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.459180+0000","last_change":"2024-09-19T08:39:05.305411+0000","last_active":"2024-09-19T08:39:06.459180+0000","last_peered":"2024-09-19T08:39:06.459180+0000","last_clean":"2024-09-19T08:39:06.459180+0000","last_became_active":"2024-09-19T08:39:05.304823+0000","last_became_peered":"2024-09-19T08:39:05.304823+0000","last_unstale":"2024-09-19T08:39:06.459180+0000","last_undegraded":"2024-09-19T08:39:06.459180+0000","last_fullsized":"2024-09-19T08:39:06.459180+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:11.979445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.300418+0000","last_change":"2024-09-19T08:39:03.300418+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.300418+0000","last_undegraded":"2024-09-19T08:39:03.300418+0000","last_fullsized":"2024-09-19T08:39:03.300418+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:03.224537+0000","last_change":"2024-09-19T08:39:03.224537+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.224537+0000","last_undegraded":"2024-09-19T08:39:03.224537+0000","last_fullsized":"2024-09-19T08:39:03.224537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.243996+0000","last_change":"2024-09-19T08:39:03.243996+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.243996+0000","last_undegraded":"2024-09-19T08:39:03.243996+0000","last_fullsized":"2024-09-19T08:39:03.243996+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.245284+0000","last_change":"2024-09-19T08:39:03.245284+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.245284+0000","last_undegraded":"2024-09-19T08:39:03.245284+0000","last_fullsized":"2024-09-19T08:39:03.245284+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.243944+0000","last_change":"2024-09-19T08:39:03.243944+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.243944+0000","last_undegraded":"2024-09-19T08:39:03.243944+0000","last_fullsized":"2024-09-19T08:39:03.243944+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:03.224537+0000","last_change":"2024-09-19T08:39:03.224537+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.224537+0000","last_undegraded":"2024-09-19T08:39:03.224537+0000","last_fullsized":"2024-09-19T08:39:03.224537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.243920+0000","last_change":"2024-09-19T08:39:03.243920+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.243920+0000","last_undegraded":"2024-09-19T08:39:03.243920+0000","last_fullsized":"2024-09-19T08:39:03.243920+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:03.224537+0000","last_change":"2024-09-19T08:39:03.224537+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.224537+0000","last_undegraded":"2024-09-19T08:39:03.224537+0000","last_fullsized":"2024-09-19T08:39:03.224537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.299294+0000","last_change":"2024-09-19T08:39:03.299294+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.299294+0000","last_undegraded":"2024-09-19T08:39:03.299294+0000","last_fullsized":"2024-09-19T08:39:03.299294+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.249325+0000","last_change":"2024-09-19T08:39:03.249325+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.249325+0000","last_undegraded":"2024-09-19T08:39:03.249325+0000","last_fullsized":"2024-09-19T08:39:03.249325+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:03.224537+0000","last_change":"2024-09-19T08:39:03.224537+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.224537+0000","last_undegraded":"2024-09-19T08:39:03.224537+0000","last_fullsized":"2024-09-19T08:39:03.224537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.243959+0000","last_change":"2024-09-19T08:39:03.243959+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.243959+0000","last_undegraded":"2024-09-19T08:39:03.243959+0000","last_fullsized":"2024-09-19T08:39:03.243959+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.248308+0000","last_change":"2024-09-19T08:39:03.248308+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.248308+0000","last_undegraded":"2024-09-19T08:39:03.248308+0000","last_fullsized":"2024-09-19T08:39:03.248308+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:03.224537+0000","last_change":"2024-09-19T08:39:03.224537+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.224537+0000","last_undegraded":"2024-09-19T08:39:03.224537+0000","last_fullsized":"2024-09-19T08:39:03.224537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.459475+0000","last_change":"2024-09-19T08:39:05.305633+0000","last_active":"2024-09-19T08:39:06.459475+0000","last_peered":"2024-09-19T08:39:06.459475+0000","last_clean":"2024-09-19T08:39:06.459475+0000","last_became_active":"2024-09-19T08:39:05.304913+0000","last_became_peered":"2024-09-19T08:39:05.304913+0000","last_unstale":"2024-09-19T08:39:06.459475+0000","last_undegraded":"2024-09-19T08:39:06.459475+0000","last_fullsized":"2024-09-19T08:39:06.459475+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:41:53.594289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:03.224537+0000","last_change":"2024-09-19T08:39:03.224537+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.224537+0000","last_undegraded":"2024-09-19T08:39:03.224537+0000","last_fullsized":"2024-09-19T08:39:03.224537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.248171+0000","last_change":"2024-09-19T08:39:03.248171+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.248171+0000","last_undegraded":"2024-09-19T08:39:03.248171+0000","last_fullsized":"2024-09-19T08:39:03.248171+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.245152+0000","last_change":"2024-09-19T08:39:03.245152+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.245152+0000","last_undegraded":"2024-09-19T08:39:03.245152+0000","last_fullsized":"2024-09-19T08:39:03.245152+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.244013+0000","last_change":"2024-09-19T08:39:03.244013+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.244013+0000","last_undegraded":"2024-09-19T08:39:03.244013+0000","last_fullsized":"2024-09-19T08:39:03.244013+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.244800+0000","last_change":"2024-09-19T08:39:03.244800+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.244800+0000","last_undegraded":"2024-09-19T08:39:03.244800+0000","last_fullsized":"2024-09-19T08:39:03.244800+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.299894+0000","last_change":"2024-09-19T08:39:03.299894+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.299894+0000","last_undegraded":"2024-09-19T08:39:03.299894+0000","last_fullsized":"2024-09-19T08:39:03.299894+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.458762+0000","last_change":"2024-09-19T08:39:04.285062+0000","last_active":"2024-09-19T08:39:06.458762+0000","last_peered":"2024-09-19T08:39:06.458762+0000","last_clean":"2024-09-19T08:39:06.458762+0000","last_became_active":"2024-09-19T08:39:04.284140+0000","last_became_peered":"2024-09-19T08:39:04.284140+0000","last_unstale":"2024-09-19T08:39:06.458762+0000","last_undegraded":"2024-09-19T08:39:06.458762+0000","last_fullsized":"2024-09-19T08:39:06.458762+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:00:00.495561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.245674+0000","last_change":"2024-09-19T08:39:03.245674+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.245674+0000","last_undegraded":"2024-09-19T08:39:03.245674+0000","last_fullsized":"2024-09-19T08:39:03.245674+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.248687+0000","last_change":"2024-09-19T08:39:03.248687+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.248687+0000","last_undegraded":"2024-09-19T08:39:03.248687+0000","last_fullsized":"2024-09-19T08:39:03.248687+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.246436+0000","last_change":"2024-09-19T08:39:03.246436+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.246436+0000","last_undegraded":"2024-09-19T08:39:03.246436+0000","last_fullsized":"2024-09-19T08:39:03.246436+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.299915+0000","last_change":"2024-09-19T08:39:03.299915+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.299915+0000","last_undegraded":"2024-09-19T08:39:03.299915+0000","last_fullsized":"2024-09-19T08:39:03.299915+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.246320+0000","last_change":"2024-09-19T08:39:03.246320+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.246320+0000","last_undegraded":"2024-09-19T08:39:03.246320+0000","last_fullsized":"2024-09-19T08:39:03.246320+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.458915+0000","last_change":"2024-09-19T08:39:05.276652+0000","last_active":"2024-09-19T08:39:06.458915+0000","last_peered":"2024-09-19T08:39:06.458915+0000","last_clean":"2024-09-19T08:39:06.458915+0000","last_became_active":"2024-09-19T08:39:05.275913+0000","last_became_peered":"2024-09-19T08:39:05.275913+0000","last_unstale":"2024-09-19T08:39:06.458915+0000","last_undegraded":"2024-09-19T08:39:06.458915+0000","last_fullsized":"2024-09-19T08:39:06.458915+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:36.037852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.299263+0000","last_change":"2024-09-19T08:39:03.299263+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.299263+0000","last_undegraded":"2024-09-19T08:39:03.299263+0000","last_fullsized":"2024-09-19T08:39:03.299263+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.246224+0000","last_change":"2024-09-19T08:39:03.246224+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.246224+0000","last_undegraded":"2024-09-19T08:39:03.246224+0000","last_fullsized":"2024-09-19T08:39:03.246224+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.243968+0000","last_change":"2024-09-19T08:39:03.243968+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.243968+0000","last_undegraded":"2024-09-19T08:39:03.243968+0000","last_fullsized":"2024-09-19T08:39:03.243968+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:03.224537+0000","last_change":"2024-09-19T08:39:03.224537+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.224537+0000","last_undegraded":"2024-09-19T08:39:03.224537+0000","last_fullsized":"2024-09-19T08:39:03.224537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.458454+0000","last_change":"2024-09-19T08:39:04.294673+0000","last_active":"2024-09-19T08:39:06.458454+0000","last_peered":"2024-09-19T08:39:06.458454+0000","last_clean":"2024-09-19T08:39:06.458454+0000","last_became_active":"2024-09-19T08:39:04.293913+0000","last_became_peered":"2024-09-19T08:39:04.293913+0000","last_unstale":"2024-09-19T08:39:06.458454+0000","last_undegraded":"2024-09-19T08:39:06.458454+0000","last_fullsized":"2024-09-19T08:39:06.458454+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:12.830812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":182,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-19T08:39:03.254315+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:39:03.254315+0000","last_peered":"2024-09-19T08:39:03.254315+0000","last_clean":"2024-09-19T08:39:03.254315+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:39:03.254315+0000","last_undegraded":"2024-09-19T08:39:03.254315+0000","last_fullsized":"2024-09-19T08:39:03.254315+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":10,"acting":10,"num_store_stats":1},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":50,"acting":50,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509454,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":897024,"data_stored":700868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":315392,"data_stored":116317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":315392,"data_stored":116317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509453,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":315392,"data_stored":116317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509455,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":303104,"data_stored":110500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509453,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27916,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343924,"statfs":{"total":96636764160,"available":96608178176,"internally_reserved":0,"allocated":909312,"data_stored":706685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":360448,"data_stored":136275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27324,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":303104,"data_stored":110500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:39:09.042 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3e is not active+clean 2024-09-19T08:39:09.042 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.042 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3d is not active+clean 2024-09-19T08:39:09.042 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.042 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3c is not active+clean 2024-09-19T08:39:09.043 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.043 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3b is not active+clean 2024-09-19T08:39:09.043 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.043 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3a is not active+clean 2024-09-19T08:39:09.043 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.043 INFO:tasks.ceph.ceph_manager.ceph:PG 5.39 is not active+clean 2024-09-19T08:39:09.043 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.043 INFO:tasks.ceph.ceph_manager.ceph:PG 5.38 is not active+clean 2024-09-19T08:39:09.043 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.043 INFO:tasks.ceph.ceph_manager.ceph:PG 5.37 is not active+clean 2024-09-19T08:39:09.043 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.044 INFO:tasks.ceph.ceph_manager.ceph:PG 5.36 is not active+clean 2024-09-19T08:39:09.044 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.044 INFO:tasks.ceph.ceph_manager.ceph:PG 5.35 is not active+clean 2024-09-19T08:39:09.044 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.044 INFO:tasks.ceph.ceph_manager.ceph:PG 5.34 is not active+clean 2024-09-19T08:39:09.044 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.044 INFO:tasks.ceph.ceph_manager.ceph:PG 5.33 is not active+clean 2024-09-19T08:39:09.044 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.044 INFO:tasks.ceph.ceph_manager.ceph:PG 5.32 is not active+clean 2024-09-19T08:39:09.044 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.044 INFO:tasks.ceph.ceph_manager.ceph:PG 5.31 is not active+clean 2024-09-19T08:39:09.045 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.045 INFO:tasks.ceph.ceph_manager.ceph:PG 5.30 is not active+clean 2024-09-19T08:39:09.045 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.045 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2f is not active+clean 2024-09-19T08:39:09.045 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.045 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2e is not active+clean 2024-09-19T08:39:09.045 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.045 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2d is not active+clean 2024-09-19T08:39:09.045 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.045 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2c is not active+clean 2024-09-19T08:39:09.045 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.046 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2b is not active+clean 2024-09-19T08:39:09.046 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.046 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2a is not active+clean 2024-09-19T08:39:09.046 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.046 INFO:tasks.ceph.ceph_manager.ceph:PG 5.29 is not active+clean 2024-09-19T08:39:09.046 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.046 INFO:tasks.ceph.ceph_manager.ceph:PG 5.28 is not active+clean 2024-09-19T08:39:09.046 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.046 INFO:tasks.ceph.ceph_manager.ceph:PG 5.27 is not active+clean 2024-09-19T08:39:09.046 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.046 INFO:tasks.ceph.ceph_manager.ceph:PG 5.26 is not active+clean 2024-09-19T08:39:09.047 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.047 INFO:tasks.ceph.ceph_manager.ceph:PG 5.25 is not active+clean 2024-09-19T08:39:09.047 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.047 INFO:tasks.ceph.ceph_manager.ceph:PG 5.24 is not active+clean 2024-09-19T08:39:09.047 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.047 INFO:tasks.ceph.ceph_manager.ceph:PG 5.23 is not active+clean 2024-09-19T08:39:09.047 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.047 INFO:tasks.ceph.ceph_manager.ceph:PG 5.22 is not active+clean 2024-09-19T08:39:09.047 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.047 INFO:tasks.ceph.ceph_manager.ceph:PG 5.20 is not active+clean 2024-09-19T08:39:09.047 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.047 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1e is not active+clean 2024-09-19T08:39:09.048 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.048 INFO:tasks.ceph.ceph_manager.ceph:PG 4.1e is not active+clean 2024-09-19T08:39:09.048 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:39:03.300418+0000', 'last_change': '2024-09-19T08:39:03.300418+0000', 'last_active': '2024-09-19T08:39:03.224537+0000', 'last_peered': '2024-09-19T08:39:03.224537+0000', 'last_clean': '2024-09-19T08:39:03.224537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:03.300418+0000', 'last_undegraded': '2024-09-19T08:39:03.300418+0000', 'last_fullsized': '2024-09-19T08:39:03.300418+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T08:39:09.048 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1c is not active+clean 2024-09-19T08:39:09.048 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.048 INFO:tasks.ceph.ceph_manager.ceph:PG 4.1d is not active+clean 2024-09-19T08:39:09.048 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:03.224537+0000', 'last_change': '2024-09-19T08:39:03.224537+0000', 'last_active': '2024-09-19T08:39:03.224537+0000', 'last_peered': '2024-09-19T08:39:03.224537+0000', 'last_clean': '2024-09-19T08:39:03.224537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:03.224537+0000', 'last_undegraded': '2024-09-19T08:39:03.224537+0000', 'last_fullsized': '2024-09-19T08:39:03.224537+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.048 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1d is not active+clean 2024-09-19T08:39:09.048 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.048 INFO:tasks.ceph.ceph_manager.ceph:PG 4.1c is not active+clean 2024-09-19T08:39:09.048 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:39:03.243996+0000', 'last_change': '2024-09-19T08:39:03.243996+0000', 'last_active': '2024-09-19T08:39:03.224537+0000', 'last_peered': '2024-09-19T08:39:03.224537+0000', 'last_clean': '2024-09-19T08:39:03.224537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:03.243996+0000', 'last_undegraded': '2024-09-19T08:39:03.243996+0000', 'last_fullsized': '2024-09-19T08:39:03.243996+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-19T08:39:09.049 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1a is not active+clean 2024-09-19T08:39:09.049 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.049 INFO:tasks.ceph.ceph_manager.ceph:PG 4.1b is not active+clean 2024-09-19T08:39:09.049 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:39:03.245284+0000', 'last_change': '2024-09-19T08:39:03.245284+0000', 'last_active': '2024-09-19T08:39:03.224537+0000', 'last_peered': '2024-09-19T08:39:03.224537+0000', 'last_clean': '2024-09-19T08:39:03.224537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:03.245284+0000', 'last_undegraded': '2024-09-19T08:39:03.245284+0000', 'last_fullsized': '2024-09-19T08:39:03.245284+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T08:39:09.049 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1b is not active+clean 2024-09-19T08:39:09.049 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.049 INFO:tasks.ceph.ceph_manager.ceph:PG 4.1a is not active+clean 2024-09-19T08:39:09.049 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:39:03.243944+0000', 'last_change': '2024-09-19T08:39:03.243944+0000', 'last_active': '2024-09-19T08:39:03.224537+0000', 'last_peered': '2024-09-19T08:39:03.224537+0000', 'last_clean': '2024-09-19T08:39:03.224537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:03.243944+0000', 'last_undegraded': '2024-09-19T08:39:03.243944+0000', 'last_fullsized': '2024-09-19T08:39:03.243944+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T08:39:09.049 INFO:tasks.ceph.ceph_manager.ceph:PG 5.18 is not active+clean 2024-09-19T08:39:09.049 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.049 INFO:tasks.ceph.ceph_manager.ceph:PG 4.19 is not active+clean 2024-09-19T08:39:09.050 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:03.224537+0000', 'last_change': '2024-09-19T08:39:03.224537+0000', 'last_active': '2024-09-19T08:39:03.224537+0000', 'last_peered': '2024-09-19T08:39:03.224537+0000', 'last_clean': '2024-09-19T08:39:03.224537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:03.224537+0000', 'last_undegraded': '2024-09-19T08:39:03.224537+0000', 'last_fullsized': '2024-09-19T08:39:03.224537+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.050 INFO:tasks.ceph.ceph_manager.ceph:PG 5.19 is not active+clean 2024-09-19T08:39:09.050 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.050 INFO:tasks.ceph.ceph_manager.ceph:PG 4.18 is not active+clean 2024-09-19T08:39:09.050 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:39:03.243920+0000', 'last_change': '2024-09-19T08:39:03.243920+0000', 'last_active': '2024-09-19T08:39:03.224537+0000', 'last_peered': '2024-09-19T08:39:03.224537+0000', 'last_clean': '2024-09-19T08:39:03.224537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:03.243920+0000', 'last_undegraded': '2024-09-19T08:39:03.243920+0000', 'last_fullsized': '2024-09-19T08:39:03.243920+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T08:39:09.050 INFO:tasks.ceph.ceph_manager.ceph:PG 5.16 is not active+clean 2024-09-19T08:39:09.050 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.050 INFO:tasks.ceph.ceph_manager.ceph:PG 4.17 is not active+clean 2024-09-19T08:39:09.050 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:03.224537+0000', 'last_change': '2024-09-19T08:39:03.224537+0000', 'last_active': '2024-09-19T08:39:03.224537+0000', 'last_peered': '2024-09-19T08:39:03.224537+0000', 'last_clean': '2024-09-19T08:39:03.224537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:03.224537+0000', 'last_undegraded': '2024-09-19T08:39:03.224537+0000', 'last_fullsized': '2024-09-19T08:39:03.224537+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.050 INFO:tasks.ceph.ceph_manager.ceph:PG 5.17 is not active+clean 2024-09-19T08:39:09.050 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.051 INFO:tasks.ceph.ceph_manager.ceph:PG 4.16 is not active+clean 2024-09-19T08:39:09.051 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:39:03.299294+0000', 'last_change': '2024-09-19T08:39:03.299294+0000', 'last_active': '2024-09-19T08:39:03.224537+0000', 'last_peered': '2024-09-19T08:39:03.224537+0000', 'last_clean': '2024-09-19T08:39:03.224537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:03.299294+0000', 'last_undegraded': '2024-09-19T08:39:03.299294+0000', 'last_fullsized': '2024-09-19T08:39:03.299294+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T08:39:09.051 INFO:tasks.ceph.ceph_manager.ceph:PG 5.14 is not active+clean 2024-09-19T08:39:09.051 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.051 INFO:tasks.ceph.ceph_manager.ceph:PG 4.15 is not active+clean 2024-09-19T08:39:09.051 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:39:03.249325+0000', 'last_change': '2024-09-19T08:39:03.249325+0000', 'last_active': '2024-09-19T08:39:03.224537+0000', 'last_peered': '2024-09-19T08:39:03.224537+0000', 'last_clean': '2024-09-19T08:39:03.224537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:03.249325+0000', 'last_undegraded': '2024-09-19T08:39:03.249325+0000', 'last_fullsized': '2024-09-19T08:39:03.249325+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T08:39:09.051 INFO:tasks.ceph.ceph_manager.ceph:PG 5.15 is not active+clean 2024-09-19T08:39:09.051 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.051 INFO:tasks.ceph.ceph_manager.ceph:PG 4.14 is not active+clean 2024-09-19T08:39:09.051 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:03.224537+0000', 'last_change': '2024-09-19T08:39:03.224537+0000', 'last_active': '2024-09-19T08:39:03.224537+0000', 'last_peered': '2024-09-19T08:39:03.224537+0000', 'last_clean': '2024-09-19T08:39:03.224537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:03.224537+0000', 'last_undegraded': '2024-09-19T08:39:03.224537+0000', 'last_fullsized': '2024-09-19T08:39:03.224537+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.051 INFO:tasks.ceph.ceph_manager.ceph:PG 5.12 is not active+clean 2024-09-19T08:39:09.052 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.052 INFO:tasks.ceph.ceph_manager.ceph:PG 4.13 is not active+clean 2024-09-19T08:39:09.052 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:39:03.243959+0000', 'last_change': '2024-09-19T08:39:03.243959+0000', 'last_active': '2024-09-19T08:39:03.224537+0000', 'last_peered': '2024-09-19T08:39:03.224537+0000', 'last_clean': '2024-09-19T08:39:03.224537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:03.243959+0000', 'last_undegraded': '2024-09-19T08:39:03.243959+0000', 'last_fullsized': '2024-09-19T08:39:03.243959+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T08:39:09.052 INFO:tasks.ceph.ceph_manager.ceph:PG 5.13 is not active+clean 2024-09-19T08:39:09.052 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.052 INFO:tasks.ceph.ceph_manager.ceph:PG 4.12 is not active+clean 2024-09-19T08:39:09.052 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:39:03.248308+0000', 'last_change': '2024-09-19T08:39:03.248308+0000', 'last_active': '2024-09-19T08:39:03.224537+0000', 'last_peered': '2024-09-19T08:39:03.224537+0000', 'last_clean': '2024-09-19T08:39:03.224537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:03.248308+0000', 'last_undegraded': '2024-09-19T08:39:03.248308+0000', 'last_fullsized': '2024-09-19T08:39:03.248308+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T08:39:09.052 INFO:tasks.ceph.ceph_manager.ceph:PG 5.10 is not active+clean 2024-09-19T08:39:09.052 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.052 INFO:tasks.ceph.ceph_manager.ceph:PG 4.11 is not active+clean 2024-09-19T08:39:09.052 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:03.224537+0000', 'last_change': '2024-09-19T08:39:03.224537+0000', 'last_active': '2024-09-19T08:39:03.224537+0000', 'last_peered': '2024-09-19T08:39:03.224537+0000', 'last_clean': '2024-09-19T08:39:03.224537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:03.224537+0000', 'last_undegraded': '2024-09-19T08:39:03.224537+0000', 'last_fullsized': '2024-09-19T08:39:03.224537+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.053 INFO:tasks.ceph.ceph_manager.ceph:PG 4.10 is not active+clean 2024-09-19T08:39:09.053 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:03.224537+0000', 'last_change': '2024-09-19T08:39:03.224537+0000', 'last_active': '2024-09-19T08:39:03.224537+0000', 'last_peered': '2024-09-19T08:39:03.224537+0000', 'last_clean': '2024-09-19T08:39:03.224537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:03.224537+0000', 'last_undegraded': '2024-09-19T08:39:03.224537+0000', 'last_fullsized': '2024-09-19T08:39:03.224537+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.053 INFO:tasks.ceph.ceph_manager.ceph:PG 5.e is not active+clean 2024-09-19T08:39:09.053 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.053 INFO:tasks.ceph.ceph_manager.ceph:PG 4.f is not active+clean 2024-09-19T08:39:09.053 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:39:03.248171+0000', 'last_change': '2024-09-19T08:39:03.248171+0000', 'last_active': '2024-09-19T08:39:03.224537+0000', 'last_peered': '2024-09-19T08:39:03.224537+0000', 'last_clean': '2024-09-19T08:39:03.224537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:03.248171+0000', 'last_undegraded': '2024-09-19T08:39:03.248171+0000', 'last_fullsized': '2024-09-19T08:39:03.248171+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T08:39:09.053 INFO:tasks.ceph.ceph_manager.ceph:PG 5.f is not active+clean 2024-09-19T08:39:09.053 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.053 INFO:tasks.ceph.ceph_manager.ceph:PG 4.e is not active+clean 2024-09-19T08:39:09.053 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:39:03.245152+0000', 'last_change': '2024-09-19T08:39:03.245152+0000', 'last_active': '2024-09-19T08:39:03.224537+0000', 'last_peered': '2024-09-19T08:39:03.224537+0000', 'last_clean': '2024-09-19T08:39:03.224537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:03.245152+0000', 'last_undegraded': '2024-09-19T08:39:03.245152+0000', 'last_fullsized': '2024-09-19T08:39:03.245152+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T08:39:09.053 INFO:tasks.ceph.ceph_manager.ceph:PG 5.c is not active+clean 2024-09-19T08:39:09.054 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.054 INFO:tasks.ceph.ceph_manager.ceph:PG 4.d is not active+clean 2024-09-19T08:39:09.054 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:39:03.244013+0000', 'last_change': '2024-09-19T08:39:03.244013+0000', 'last_active': '2024-09-19T08:39:03.224537+0000', 'last_peered': '2024-09-19T08:39:03.224537+0000', 'last_clean': '2024-09-19T08:39:03.224537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:03.244013+0000', 'last_undegraded': '2024-09-19T08:39:03.244013+0000', 'last_fullsized': '2024-09-19T08:39:03.244013+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T08:39:09.054 INFO:tasks.ceph.ceph_manager.ceph:PG 5.d is not active+clean 2024-09-19T08:39:09.054 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.054 INFO:tasks.ceph.ceph_manager.ceph:PG 4.c is not active+clean 2024-09-19T08:39:09.054 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:39:03.244800+0000', 'last_change': '2024-09-19T08:39:03.244800+0000', 'last_active': '2024-09-19T08:39:03.224537+0000', 'last_peered': '2024-09-19T08:39:03.224537+0000', 'last_clean': '2024-09-19T08:39:03.224537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:03.244800+0000', 'last_undegraded': '2024-09-19T08:39:03.244800+0000', 'last_fullsized': '2024-09-19T08:39:03.244800+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T08:39:09.054 INFO:tasks.ceph.ceph_manager.ceph:PG 5.a is not active+clean 2024-09-19T08:39:09.054 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.054 INFO:tasks.ceph.ceph_manager.ceph:PG 4.b is not active+clean 2024-09-19T08:39:09.054 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:39:03.299894+0000', 'last_change': '2024-09-19T08:39:03.299894+0000', 'last_active': '2024-09-19T08:39:03.224537+0000', 'last_peered': '2024-09-19T08:39:03.224537+0000', 'last_clean': '2024-09-19T08:39:03.224537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:03.299894+0000', 'last_undegraded': '2024-09-19T08:39:03.299894+0000', 'last_fullsized': '2024-09-19T08:39:03.299894+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T08:39:09.055 INFO:tasks.ceph.ceph_manager.ceph:PG 5.b is not active+clean 2024-09-19T08:39:09.055 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.055 INFO:tasks.ceph.ceph_manager.ceph:PG 5.8 is not active+clean 2024-09-19T08:39:09.055 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.055 INFO:tasks.ceph.ceph_manager.ceph:PG 4.9 is not active+clean 2024-09-19T08:39:09.055 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:39:03.245674+0000', 'last_change': '2024-09-19T08:39:03.245674+0000', 'last_active': '2024-09-19T08:39:03.224537+0000', 'last_peered': '2024-09-19T08:39:03.224537+0000', 'last_clean': '2024-09-19T08:39:03.224537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:03.245674+0000', 'last_undegraded': '2024-09-19T08:39:03.245674+0000', 'last_fullsized': '2024-09-19T08:39:03.245674+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T08:39:09.055 INFO:tasks.ceph.ceph_manager.ceph:PG 5.9 is not active+clean 2024-09-19T08:39:09.055 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.055 INFO:tasks.ceph.ceph_manager.ceph:PG 4.8 is not active+clean 2024-09-19T08:39:09.055 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:39:03.248687+0000', 'last_change': '2024-09-19T08:39:03.248687+0000', 'last_active': '2024-09-19T08:39:03.224537+0000', 'last_peered': '2024-09-19T08:39:03.224537+0000', 'last_clean': '2024-09-19T08:39:03.224537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:03.248687+0000', 'last_undegraded': '2024-09-19T08:39:03.248687+0000', 'last_fullsized': '2024-09-19T08:39:03.248687+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T08:39:09.055 INFO:tasks.ceph.ceph_manager.ceph:PG 5.6 is not active+clean 2024-09-19T08:39:09.056 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.056 INFO:tasks.ceph.ceph_manager.ceph:PG 4.7 is not active+clean 2024-09-19T08:39:09.056 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:39:03.246436+0000', 'last_change': '2024-09-19T08:39:03.246436+0000', 'last_active': '2024-09-19T08:39:03.224537+0000', 'last_peered': '2024-09-19T08:39:03.224537+0000', 'last_clean': '2024-09-19T08:39:03.224537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:03.246436+0000', 'last_undegraded': '2024-09-19T08:39:03.246436+0000', 'last_fullsized': '2024-09-19T08:39:03.246436+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T08:39:09.056 INFO:tasks.ceph.ceph_manager.ceph:PG 5.7 is not active+clean 2024-09-19T08:39:09.056 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.056 INFO:tasks.ceph.ceph_manager.ceph:PG 4.6 is not active+clean 2024-09-19T08:39:09.056 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:39:03.299915+0000', 'last_change': '2024-09-19T08:39:03.299915+0000', 'last_active': '2024-09-19T08:39:03.224537+0000', 'last_peered': '2024-09-19T08:39:03.224537+0000', 'last_clean': '2024-09-19T08:39:03.224537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:03.299915+0000', 'last_undegraded': '2024-09-19T08:39:03.299915+0000', 'last_fullsized': '2024-09-19T08:39:03.299915+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T08:39:09.056 INFO:tasks.ceph.ceph_manager.ceph:PG 5.5 is not active+clean 2024-09-19T08:39:09.056 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.056 INFO:tasks.ceph.ceph_manager.ceph:PG 4.4 is not active+clean 2024-09-19T08:39:09.056 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:39:03.246320+0000', 'last_change': '2024-09-19T08:39:03.246320+0000', 'last_active': '2024-09-19T08:39:03.224537+0000', 'last_peered': '2024-09-19T08:39:03.224537+0000', 'last_clean': '2024-09-19T08:39:03.224537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:03.246320+0000', 'last_undegraded': '2024-09-19T08:39:03.246320+0000', 'last_fullsized': '2024-09-19T08:39:03.246320+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T08:39:09.057 INFO:tasks.ceph.ceph_manager.ceph:PG 4.3 is not active+clean 2024-09-19T08:39:09.057 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:39:03.299263+0000', 'last_change': '2024-09-19T08:39:03.299263+0000', 'last_active': '2024-09-19T08:39:03.224537+0000', 'last_peered': '2024-09-19T08:39:03.224537+0000', 'last_clean': '2024-09-19T08:39:03.224537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:03.299263+0000', 'last_undegraded': '2024-09-19T08:39:03.299263+0000', 'last_fullsized': '2024-09-19T08:39:03.299263+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T08:39:09.057 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3 is not active+clean 2024-09-19T08:39:09.057 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.057 INFO:tasks.ceph.ceph_manager.ceph:PG 4.2 is not active+clean 2024-09-19T08:39:09.057 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:39:03.246224+0000', 'last_change': '2024-09-19T08:39:03.246224+0000', 'last_active': '2024-09-19T08:39:03.224537+0000', 'last_peered': '2024-09-19T08:39:03.224537+0000', 'last_clean': '2024-09-19T08:39:03.224537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:03.246224+0000', 'last_undegraded': '2024-09-19T08:39:03.246224+0000', 'last_fullsized': '2024-09-19T08:39:03.246224+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T08:39:09.057 INFO:tasks.ceph.ceph_manager.ceph:PG 5.0 is not active+clean 2024-09-19T08:39:09.057 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.057 INFO:tasks.ceph.ceph_manager.ceph:PG 4.1 is not active+clean 2024-09-19T08:39:09.057 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 29, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:39:03.243968+0000', 'last_change': '2024-09-19T08:39:03.243968+0000', 'last_active': '2024-09-19T08:39:03.224537+0000', 'last_peered': '2024-09-19T08:39:03.224537+0000', 'last_clean': '2024-09-19T08:39:03.224537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:03.243968+0000', 'last_undegraded': '2024-09-19T08:39:03.243968+0000', 'last_fullsized': '2024-09-19T08:39:03.243968+0000', 'mapping_epoch': 29, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 29, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T08:39:09.057 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1 is not active+clean 2024-09-19T08:39:09.058 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.058 INFO:tasks.ceph.ceph_manager.ceph:PG 4.0 is not active+clean 2024-09-19T08:39:09.058 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:03.224537+0000', 'last_change': '2024-09-19T08:39:03.224537+0000', 'last_active': '2024-09-19T08:39:03.224537+0000', 'last_peered': '2024-09-19T08:39:03.224537+0000', 'last_clean': '2024-09-19T08:39:03.224537+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:03.224537+0000', 'last_undegraded': '2024-09-19T08:39:03.224537+0000', 'last_fullsized': '2024-09-19T08:39:03.224537+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:03.224537+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.058 INFO:tasks.ceph.ceph_manager.ceph:PG 5.4 is not active+clean 2024-09-19T08:39:09.058 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:04.236483+0000', 'last_change': '2024-09-19T08:39:04.236483+0000', 'last_active': '2024-09-19T08:39:04.236483+0000', 'last_peered': '2024-09-19T08:39:04.236483+0000', 'last_clean': '2024-09-19T08:39:04.236483+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:04.236483+0000', 'last_undegraded': '2024-09-19T08:39:04.236483+0000', 'last_fullsized': '2024-09-19T08:39:04.236483+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:04.236483+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:09.059 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T08:39:09.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.195+0000 7f4e3537e640 1 Processor -- start 2024-09-19T08:39:09.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.195+0000 7f4e3537e640 1 -- start start 2024-09-19T08:39:09.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.195+0000 7f4e3537e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e3015b440 0x7f4e300ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:09.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.195+0000 7f4e3537e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4e300ce860 0x7f4e300d0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:09.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.195+0000 7f4e3537e640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e30071100 con 0x7f4e300d0850 2024-09-19T08:39:09.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.195+0000 7f4e3537e640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f4e3015b820 con 0x7f4e300ce860 2024-09-19T08:39:09.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.195+0000 7f4e3537e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4e300ced00 con 0x7f4e3015b440 2024-09-19T08:39:09.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.195+0000 7f4e2e7fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4e300ce860 0x7f4e300d0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:09.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.195+0000 7f4e2f7fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f4e300d0850 0x7f4e300b20f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:45560/0 (socket says 172.21.15.98:45560) 2024-09-19T08:39:09.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.195+0000 7f4e2f7fe640 1 -- 172.21.15.98:0/2532877654 learned_addr learned my addr 172.21.15.98:0/2532877654 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:09.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.199+0000 7f4e2effd640 1 --2- 172.21.15.98:0/2532877654 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e3015b440 0x7f4e300ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:09.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.199+0000 7f4e2e7fc640 1 -- 172.21.15.98:0/2532877654 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e3015b440 msgr2=0x7f4e300ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:09.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.199+0000 7f4e2e7fc640 1 --2- 172.21.15.98:0/2532877654 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e3015b440 0x7f4e300ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:09.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.199+0000 7f4e2e7fc640 1 -- 172.21.15.98:0/2532877654 >> v1:172.21.15.98:6789/0 conn(0x7f4e300d0850 legacy=0x7f4e300b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:09.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.199+0000 7f4e2e7fc640 1 -- 172.21.15.98:0/2532877654 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e30071380 con 0x7f4e300ce860 2024-09-19T08:39:09.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.199+0000 7f4e2dffb640 1 -- 172.21.15.98:0/2532877654 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2215581897 0 0) 0x7f4e28001120 con 0x7f4e300d0850 2024-09-19T08:39:09.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.199+0000 7f4e2e7fc640 1 --2- 172.21.15.98:0/2532877654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e300ce860 0x7f4e300d0310 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4e180097c0 tx=0x7f4e1801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4f62dc048b9272 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:09.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.199+0000 7f4e2dffb640 1 -- 172.21.15.98:0/2532877654 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e180b2070 con 0x7f4e300ce860 2024-09-19T08:39:09.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.199+0000 7f4e2dffb640 1 -- 172.21.15.98:0/2532877654 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e180ad640 con 0x7f4e300ce860 2024-09-19T08:39:09.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.199+0000 7f4e2dffb640 1 -- 172.21.15.98:0/2532877654 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e180ada10 con 0x7f4e300ce860 2024-09-19T08:39:09.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.199+0000 7f4e3537e640 1 -- 172.21.15.98:0/2532877654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e300ce860 msgr2=0x7f4e300d0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:09.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.199+0000 7f4e3537e640 1 --2- 172.21.15.98:0/2532877654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e300ce860 0x7f4e300d0310 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4e180097c0 tx=0x7f4e1801c4a0 comp rx=0 tx=0).stop 2024-09-19T08:39:09.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.199+0000 7f4e3537e640 1 -- 172.21.15.98:0/2532877654 shutdown_connections 2024-09-19T08:39:09.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.199+0000 7f4e3537e640 1 --2- 172.21.15.98:0/2532877654 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e300ce860 0x7f4e300d0310 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:09.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.199+0000 7f4e3537e640 1 --2- 172.21.15.98:0/2532877654 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e3015b440 0x7f4e300ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:09.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.199+0000 7f4e3537e640 1 -- 172.21.15.98:0/2532877654 >> 172.21.15.98:0/2532877654 conn(0x7f4e3006bc90 msgr2=0x7f4e3006c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:09.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.199+0000 7f4e3537e640 1 -- 172.21.15.98:0/2532877654 shutdown_connections 2024-09-19T08:39:09.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.199+0000 7f4e3537e640 1 -- 172.21.15.98:0/2532877654 wait complete. 2024-09-19T08:39:09.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.203+0000 7f4e3537e640 1 Processor -- start 2024-09-19T08:39:09.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.203+0000 7f4e3537e640 1 -- start start 2024-09-19T08:39:09.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.203+0000 7f4e3537e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e300ce860 0x7f4e30156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:09.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.203+0000 7f4e3537e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e300d0850 0x7f4e3006d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:09.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.203+0000 7f4e3537e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e3015b440 0x7f4e3006d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:09.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.203+0000 7f4e3537e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4e30168060 con 0x7f4e3015b440 2024-09-19T08:39:09.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.203+0000 7f4e3537e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3015fe50 con 0x7f4e300d0850 2024-09-19T08:39:09.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.203+0000 7f4e3537e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4e301681e0 con 0x7f4e300ce860 2024-09-19T08:39:09.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.203+0000 7f4e2f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e3015b440 0x7f4e3006d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:09.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.203+0000 7f4e2f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e3015b440 0x7f4e3006d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36438/0 (socket says 172.21.15.98:36438) 2024-09-19T08:39:09.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.203+0000 7f4e2f7fe640 1 -- 172.21.15.98:0/3921937568 learned_addr learned my addr 172.21.15.98:0/3921937568 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:09.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.203+0000 7f4e2effd640 1 --2- 172.21.15.98:0/3921937568 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e300ce860 0x7f4e30156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:09.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.203+0000 7f4e2e7fc640 1 --2- 172.21.15.98:0/3921937568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e300d0850 0x7f4e3006d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:09.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.203+0000 7f4e2f7fe640 1 -- 172.21.15.98:0/3921937568 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e300ce860 msgr2=0x7f4e30156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:09.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.203+0000 7f4e2f7fe640 1 --2- 172.21.15.98:0/3921937568 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e300ce860 0x7f4e30156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:09.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.203+0000 7f4e2f7fe640 1 -- 172.21.15.98:0/3921937568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e300d0850 msgr2=0x7f4e3006d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:09.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.203+0000 7f4e2f7fe640 1 --2- 172.21.15.98:0/3921937568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e300d0850 0x7f4e3006d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:09.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.203+0000 7f4e2f7fe640 1 -- 172.21.15.98:0/3921937568 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e30072a00 con 0x7f4e3015b440 2024-09-19T08:39:09.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.203+0000 7f4e2effd640 1 --2- 172.21.15.98:0/3921937568 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e300ce860 0x7f4e30156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:39:09.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.203+0000 7f4e2f7fe640 1 --2- 172.21.15.98:0/3921937568 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e3015b440 0x7f4e3006d6b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4e2800c3e0 tx=0x7f4e2800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:09.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.203+0000 7f4e277fe640 1 -- 172.21.15.98:0/3921937568 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e280074e0 con 0x7f4e3015b440 2024-09-19T08:39:09.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.203+0000 7f4e277fe640 1 -- 172.21.15.98:0/3921937568 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e280047f0 con 0x7f4e3015b440 2024-09-19T08:39:09.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.203+0000 7f4e277fe640 1 -- 172.21.15.98:0/3921937568 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e28004be0 con 0x7f4e3015b440 2024-09-19T08:39:09.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.203+0000 7f4e3537e640 1 -- 172.21.15.98:0/3921937568 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e300af260 con 0x7f4e3015b440 2024-09-19T08:39:09.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.203+0000 7f4e3537e640 1 -- 172.21.15.98:0/3921937568 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e300aef80 con 0x7f4e3015b440 2024-09-19T08:39:09.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.203+0000 7f4e3537e640 1 -- 172.21.15.98:0/3921937568 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e300d0310 con 0x7f4e3015b440 2024-09-19T08:39:09.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.207+0000 7f4e277fe640 1 -- 172.21.15.98:0/3921937568 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4e28011070 con 0x7f4e3015b440 2024-09-19T08:39:09.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.207+0000 7f4e277fe640 1 --2- 172.21.15.98:0/3921937568 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4e100678f0 0x7f4e10069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:09.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.207+0000 7f4e277fe640 1 -- 172.21.15.98:0/3921937568 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f4e28085830 con 0x7f4e3015b440 2024-09-19T08:39:09.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.207+0000 7f4e2effd640 1 --2- 172.21.15.98:0/3921937568 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4e100678f0 0x7f4e10069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:09.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.207+0000 7f4e2effd640 1 --2- 172.21.15.98:0/3921937568 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4e100678f0 0x7f4e10069db0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4e300b1310 tx=0x7f4e2000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:09.214 INFO:tasks.ceph.mds.c.smithi098.stderr:2024-09-19T08:39:09.211+0000 7f5a2c1ec640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T08:39:09.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.211+0000 7f4e277fe640 1 -- 172.21.15.98:0/3921937568 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f4e30071100 con 0x7f4e3015b440 2024-09-19T08:39:09.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.423+0000 7f4e3537e640 1 -- 172.21.15.98:0/3921937568 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f4e300008e0 con 0x7f4e3015b440 2024-09-19T08:39:09.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.427+0000 7f4e277fe640 1 -- 172.21.15.98:0/3921937568 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2697 (secure 0 0 0) 0x7f4e2805c030 con 0x7f4e3015b440 2024-09-19T08:39:09.429 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:39:09.429 INFO:teuthology.orchestra.run.smithi098.stdout:{"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":73,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":32,"num_osds":8,"num_up_osds":8,"osd_up_since":1726735088,"num_in_osds":8,"osd_in_since":1726735077,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":66},{"state_name":"creating+peering","count":22},{"state_name":"active+clean","count":9}],"num_pgs":97,"num_pools":3,"num_objects":4,"data_bytes":590950,"bytes_used":225132544,"bytes_avail":772868980736,"bytes_total":773094113280,"unknown_pgs_ratio":0.68041235208511353,"inactive_pgs_ratio":0.22680412232875824,"write_bytes_sec":213,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":14,"btime":"2024-09-19T08:39:07:270861+0000","id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":4833}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-19T08:39:08.040483+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-19T08:39:09.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.427+0000 7f4e3537e640 1 -- 172.21.15.98:0/3921937568 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4e100678f0 msgr2=0x7f4e10069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:09.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.427+0000 7f4e3537e640 1 --2- 172.21.15.98:0/3921937568 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4e100678f0 0x7f4e10069db0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4e300b1310 tx=0x7f4e2000b040 comp rx=0 tx=0).stop 2024-09-19T08:39:09.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.427+0000 7f4e3537e640 1 -- 172.21.15.98:0/3921937568 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e3015b440 msgr2=0x7f4e3006d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:09.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.427+0000 7f4e3537e640 1 --2- 172.21.15.98:0/3921937568 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e3015b440 0x7f4e3006d6b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4e2800c3e0 tx=0x7f4e2800c8a0 comp rx=0 tx=0).stop 2024-09-19T08:39:09.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.427+0000 7f4e3537e640 1 -- 172.21.15.98:0/3921937568 shutdown_connections 2024-09-19T08:39:09.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.427+0000 7f4e3537e640 1 --2- 172.21.15.98:0/3921937568 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4e100678f0 0x7f4e10069db0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:09.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.427+0000 7f4e3537e640 1 --2- 172.21.15.98:0/3921937568 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4e3015b440 0x7f4e3006d6b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:09.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.427+0000 7f4e3537e640 1 --2- 172.21.15.98:0/3921937568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e300d0850 0x7f4e3006d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:09.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.427+0000 7f4e3537e640 1 --2- 172.21.15.98:0/3921937568 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4e300ce860 0x7f4e30156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:09.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.427+0000 7f4e3537e640 1 -- 172.21.15.98:0/3921937568 >> 172.21.15.98:0/3921937568 conn(0x7f4e3006bc90 msgr2=0x7f4e3006c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:09.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.427+0000 7f4e3537e640 1 -- 172.21.15.98:0/3921937568 shutdown_connections 2024-09-19T08:39:09.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.427+0000 7f4e3537e640 1 -- 172.21.15.98:0/3921937568 wait complete. 2024-09-19T08:39:09.446 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-19T08:39:09.447 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:39:09.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.559+0000 7f37457a0640 1 Processor -- start 2024-09-19T08:39:09.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.559+0000 7f37457a0640 1 -- start start 2024-09-19T08:39:09.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.563+0000 7f37457a0640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3740160740 0x7f3740163170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:09.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.563+0000 7f37457a0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f37401636b0 0x7f374006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:09.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.563+0000 7f37457a0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f374006c1d0 0x7f374006d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:09.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.563+0000 7f37457a0640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f374006c600 con 0x7f3740160740 2024-09-19T08:39:09.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.563+0000 7f37457a0640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f3740163bf0 con 0x7f374006c1d0 2024-09-19T08:39:09.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.563+0000 7f37457a0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3740163ed0 con 0x7f37401636b0 2024-09-19T08:39:09.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.563+0000 7f373effd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3740160740 0x7f3740163170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:09.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.563+0000 7f373effd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3740160740 0x7f3740163170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36442/0 (socket says 172.21.15.98:36442) 2024-09-19T08:39:09.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.563+0000 7f373effd640 1 -- 172.21.15.98:0/76446618 learned_addr learned my addr 172.21.15.98:0/76446618 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:09.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.563+0000 7f373e7fc640 1 --2- 172.21.15.98:0/76446618 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f37401636b0 0x7f374006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:09.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.563+0000 7f373f7fe640 1 --2- 172.21.15.98:0/76446618 >> v2:172.21.15.139:3300/0 conn(0x7f374006c1d0 0x7f374006d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:09.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.563+0000 7f373effd640 1 -- 172.21.15.98:0/76446618 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f37401636b0 msgr2=0x7f374006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:09.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.563+0000 7f373effd640 1 --2- 172.21.15.98:0/76446618 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f37401636b0 0x7f374006bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:09.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.563+0000 7f373effd640 1 -- 172.21.15.98:0/76446618 >> v2:172.21.15.139:3300/0 conn(0x7f374006c1d0 msgr2=0x7f374006d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:09.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.563+0000 7f373effd640 1 --2- 172.21.15.98:0/76446618 >> v2:172.21.15.139:3300/0 conn(0x7f374006c1d0 0x7f374006d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:09.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.563+0000 7f373effd640 1 -- 172.21.15.98:0/76446618 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37400ce320 con 0x7f3740160740 2024-09-19T08:39:09.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.563+0000 7f373f7fe640 1 --2- 172.21.15.98:0/76446618 >> v2:172.21.15.139:3300/0 conn(0x7f374006c1d0 0x7f374006d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:39:09.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.563+0000 7f373effd640 1 --2- 172.21.15.98:0/76446618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3740160740 0x7f3740163170 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f374006f3a0 tx=0x7f373401c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f1e920ba392bd0b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:09.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.563+0000 7f373dffb640 1 -- 172.21.15.98:0/76446618 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37340b0070 con 0x7f3740160740 2024-09-19T08:39:09.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.563+0000 7f373dffb640 1 -- 172.21.15.98:0/76446618 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37340ab600 con 0x7f3740160740 2024-09-19T08:39:09.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.563+0000 7f373dffb640 1 -- 172.21.15.98:0/76446618 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37340ab9d0 con 0x7f3740160740 2024-09-19T08:39:09.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.567+0000 7f37457a0640 1 -- 172.21.15.98:0/76446618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3740160740 msgr2=0x7f3740163170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:09.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.567+0000 7f37457a0640 1 --2- 172.21.15.98:0/76446618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3740160740 0x7f3740163170 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f374006f3a0 tx=0x7f373401c260 comp rx=0 tx=0).stop 2024-09-19T08:39:09.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.567+0000 7f37457a0640 1 -- 172.21.15.98:0/76446618 shutdown_connections 2024-09-19T08:39:09.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.567+0000 7f37457a0640 1 --2- 172.21.15.98:0/76446618 >> v2:172.21.15.139:3300/0 conn(0x7f374006c1d0 0x7f374006d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:09.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.567+0000 7f37457a0640 1 --2- 172.21.15.98:0/76446618 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f37401636b0 0x7f374006bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:09.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.567+0000 7f37457a0640 1 --2- 172.21.15.98:0/76446618 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3740160740 0x7f3740163170 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:09.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.567+0000 7f37457a0640 1 -- 172.21.15.98:0/76446618 >> 172.21.15.98:0/76446618 conn(0x7f37400b2010 msgr2=0x7f37400b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:09.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.567+0000 7f37457a0640 1 -- 172.21.15.98:0/76446618 shutdown_connections 2024-09-19T08:39:09.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.567+0000 7f37457a0640 1 -- 172.21.15.98:0/76446618 wait complete. 2024-09-19T08:39:09.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.567+0000 7f37457a0640 1 Processor -- start 2024-09-19T08:39:09.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.567+0000 7f37457a0640 1 -- start start 2024-09-19T08:39:09.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.567+0000 7f37457a0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f374006c1d0 0x7f374015cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:09.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.567+0000 7f37457a0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3740160740 0x7f3740150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:09.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.567+0000 7f37457a0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37401636b0 0x7f3740151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:09.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.567+0000 7f37457a0640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f37400ceca0 con 0x7f37401636b0 2024-09-19T08:39:09.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.567+0000 7f37457a0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f37400ceb20 con 0x7f374006c1d0 2024-09-19T08:39:09.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.567+0000 7f37457a0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f37400cee20 con 0x7f3740160740 2024-09-19T08:39:09.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.567+0000 7f373f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37401636b0 0x7f3740151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:09.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.567+0000 7f373f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37401636b0 0x7f3740151f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36456/0 (socket says 172.21.15.98:36456) 2024-09-19T08:39:09.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.567+0000 7f373f7fe640 1 -- 172.21.15.98:0/3487862903 learned_addr learned my addr 172.21.15.98:0/3487862903 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:09.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.567+0000 7f373effd640 1 --2- 172.21.15.98:0/3487862903 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f374006c1d0 0x7f374015cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:09.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.567+0000 7f373e7fc640 1 --2- 172.21.15.98:0/3487862903 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3740160740 0x7f3740150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:09.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.567+0000 7f373f7fe640 1 -- 172.21.15.98:0/3487862903 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3740160740 msgr2=0x7f3740150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:09.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.567+0000 7f373f7fe640 1 --2- 172.21.15.98:0/3487862903 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3740160740 0x7f3740150fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:09.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.567+0000 7f373f7fe640 1 -- 172.21.15.98:0/3487862903 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f374006c1d0 msgr2=0x7f374015cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:09.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.567+0000 7f373f7fe640 1 --2- 172.21.15.98:0/3487862903 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f374006c1d0 0x7f374015cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:09.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.567+0000 7f373f7fe640 1 -- 172.21.15.98:0/3487862903 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37401527e0 con 0x7f37401636b0 2024-09-19T08:39:09.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.567+0000 7f373effd640 1 --2- 172.21.15.98:0/3487862903 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f374006c1d0 0x7f374015cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:09.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.567+0000 7f373f7fe640 1 --2- 172.21.15.98:0/3487862903 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37401636b0 0x7f3740151f90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3738007e40 tx=0x7f373800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:09.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.567+0000 7f37277fe640 1 -- 172.21.15.98:0/3487862903 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373801a070 con 0x7f37401636b0 2024-09-19T08:39:09.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.567+0000 7f37277fe640 1 -- 172.21.15.98:0/3487862903 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3738003ba0 con 0x7f37401636b0 2024-09-19T08:39:09.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.567+0000 7f37457a0640 1 -- 172.21.15.98:0/3487862903 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3740159c50 con 0x7f37401636b0 2024-09-19T08:39:09.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.567+0000 7f37277fe640 1 -- 172.21.15.98:0/3487862903 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373800de80 con 0x7f37401636b0 2024-09-19T08:39:09.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.567+0000 7f37457a0640 1 -- 172.21.15.98:0/3487862903 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f374015a210 con 0x7f37401636b0 2024-09-19T08:39:09.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.571+0000 7f37457a0640 1 -- 172.21.15.98:0/3487862903 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3740163170 con 0x7f37401636b0 2024-09-19T08:39:09.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.571+0000 7f37277fe640 1 -- 172.21.15.98:0/3487862903 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f373801d070 con 0x7f37401636b0 2024-09-19T08:39:09.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.571+0000 7f37277fe640 1 --2- 172.21.15.98:0/3487862903 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f372c0678c0 0x7f372c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:09.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.571+0000 7f373effd640 1 --2- 172.21.15.98:0/3487862903 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f372c0678c0 0x7f372c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:09.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.571+0000 7f37277fe640 1 -- 172.21.15.98:0/3487862903 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f3738085ed0 con 0x7f37401636b0 2024-09-19T08:39:09.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.571+0000 7f373effd640 1 --2- 172.21.15.98:0/3487862903 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f372c0678c0 0x7f372c069d80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f37340ad000 tx=0x7f3734008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:09.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.575+0000 7f37277fe640 1 -- 172.21.15.98:0/3487862903 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3738010070 con 0x7f37401636b0 2024-09-19T08:39:09.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.735+0000 7f37457a0640 1 -- 172.21.15.98:0/3487862903 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f37400742b0 con 0x7f372c0678c0 2024-09-19T08:39:09.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.739+0000 7f37277fe640 1 -- 172.21.15.98:0/3487862903 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240452 (secure 0 0 0) 0x7f37400742b0 con 0x7f372c0678c0 2024-09-19T08:39:09.746 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:39:09.747 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:39:09.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.743+0000 7f37457a0640 1 -- 172.21.15.98:0/3487862903 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f372c0678c0 msgr2=0x7f372c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:09.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.743+0000 7f37457a0640 1 --2- 172.21.15.98:0/3487862903 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f372c0678c0 0x7f372c069d80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f37340ad000 tx=0x7f3734008040 comp rx=0 tx=0).stop 2024-09-19T08:39:09.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.743+0000 7f37457a0640 1 -- 172.21.15.98:0/3487862903 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37401636b0 msgr2=0x7f3740151f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:09.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.743+0000 7f37457a0640 1 --2- 172.21.15.98:0/3487862903 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37401636b0 0x7f3740151f90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3738007e40 tx=0x7f373800d4a0 comp rx=0 tx=0).stop 2024-09-19T08:39:09.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.743+0000 7f37457a0640 1 -- 172.21.15.98:0/3487862903 shutdown_connections 2024-09-19T08:39:09.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.743+0000 7f37457a0640 1 --2- 172.21.15.98:0/3487862903 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f372c0678c0 0x7f372c069d80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:09.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.743+0000 7f37457a0640 1 --2- 172.21.15.98:0/3487862903 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f37401636b0 0x7f3740151f90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:09.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.743+0000 7f37457a0640 1 --2- 172.21.15.98:0/3487862903 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3740160740 0x7f3740150fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:09.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.743+0000 7f37457a0640 1 --2- 172.21.15.98:0/3487862903 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f374006c1d0 0x7f374015cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:09.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.743+0000 7f37457a0640 1 -- 172.21.15.98:0/3487862903 >> 172.21.15.98:0/3487862903 conn(0x7f37400b2010 msgr2=0x7f37400d0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:09.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.743+0000 7f37457a0640 1 -- 172.21.15.98:0/3487862903 shutdown_connections 2024-09-19T08:39:09.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:09.743+0000 7f37457a0640 1 -- 172.21.15.98:0/3487862903 wait complete. 2024-09-19T08:39:09.764 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2024-09-19T08:39:08.040412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590950,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":163,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121,"ondisk_log_size":121,"up":62,"acting":62,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":219856,"kb_used_data":3632,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754864,"statfs":{"total":773094113280,"available":772868980736,"internally_reserved":0,"allocated":3719168,"data_stored":2113779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.801923"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.458834+0000","last_change":"2024-09-19T08:39:05.300955+0000","last_active":"2024-09-19T08:39:06.458834+0000","last_peered":"2024-09-19T08:39:06.458834+0000","last_clean":"2024-09-19T08:39:06.458834+0000","last_became_active":"2024-09-19T08:39:05.300228+0000","last_became_peered":"2024-09-19T08:39:05.300228+0000","last_unstale":"2024-09-19T08:39:06.458834+0000","last_undegraded":"2024-09-19T08:39:06.458834+0000","last_fullsized":"2024-09-19T08:39:06.458834+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:05.814581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.459141+0000","last_change":"2024-09-19T08:39:05.276219+0000","last_active":"2024-09-19T08:39:06.459141+0000","last_peered":"2024-09-19T08:39:06.459141+0000","last_clean":"2024-09-19T08:39:06.459141+0000","last_became_active":"2024-09-19T08:39:05.275466+0000","last_became_peered":"2024-09-19T08:39:05.275466+0000","last_unstale":"2024-09-19T08:39:06.459141+0000","last_undegraded":"2024-09-19T08:39:06.459141+0000","last_fullsized":"2024-09-19T08:39:06.459141+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:09.038404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.459079+0000","last_change":"2024-09-19T08:39:04.292831+0000","last_active":"2024-09-19T08:39:06.459079+0000","last_peered":"2024-09-19T08:39:06.459079+0000","last_clean":"2024-09-19T08:39:06.459079+0000","last_became_active":"2024-09-19T08:39:04.292058+0000","last_became_peered":"2024-09-19T08:39:04.292058+0000","last_unstale":"2024-09-19T08:39:06.459079+0000","last_undegraded":"2024-09-19T08:39:06.459079+0000","last_fullsized":"2024-09-19T08:39:06.459079+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:38:41.573373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.459180+0000","last_change":"2024-09-19T08:39:05.305411+0000","last_active":"2024-09-19T08:39:06.459180+0000","last_peered":"2024-09-19T08:39:06.459180+0000","last_clean":"2024-09-19T08:39:06.459180+0000","last_became_active":"2024-09-19T08:39:05.304823+0000","last_became_peered":"2024-09-19T08:39:05.304823+0000","last_unstale":"2024-09-19T08:39:06.459180+0000","last_undegraded":"2024-09-19T08:39:06.459180+0000","last_fullsized":"2024-09-19T08:39:06.459180+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:11.979445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.300418+0000","last_change":"2024-09-19T08:39:03.300418+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.300418+0000","last_undegraded":"2024-09-19T08:39:03.300418+0000","last_fullsized":"2024-09-19T08:39:03.300418+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:03.224537+0000","last_change":"2024-09-19T08:39:03.224537+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.224537+0000","last_undegraded":"2024-09-19T08:39:03.224537+0000","last_fullsized":"2024-09-19T08:39:03.224537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.243996+0000","last_change":"2024-09-19T08:39:03.243996+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.243996+0000","last_undegraded":"2024-09-19T08:39:03.243996+0000","last_fullsized":"2024-09-19T08:39:03.243996+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.245284+0000","last_change":"2024-09-19T08:39:03.245284+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.245284+0000","last_undegraded":"2024-09-19T08:39:03.245284+0000","last_fullsized":"2024-09-19T08:39:03.245284+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.243944+0000","last_change":"2024-09-19T08:39:03.243944+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.243944+0000","last_undegraded":"2024-09-19T08:39:03.243944+0000","last_fullsized":"2024-09-19T08:39:03.243944+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:03.224537+0000","last_change":"2024-09-19T08:39:03.224537+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.224537+0000","last_undegraded":"2024-09-19T08:39:03.224537+0000","last_fullsized":"2024-09-19T08:39:03.224537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.243920+0000","last_change":"2024-09-19T08:39:03.243920+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.243920+0000","last_undegraded":"2024-09-19T08:39:03.243920+0000","last_fullsized":"2024-09-19T08:39:03.243920+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:03.224537+0000","last_change":"2024-09-19T08:39:03.224537+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.224537+0000","last_undegraded":"2024-09-19T08:39:03.224537+0000","last_fullsized":"2024-09-19T08:39:03.224537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.299294+0000","last_change":"2024-09-19T08:39:03.299294+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.299294+0000","last_undegraded":"2024-09-19T08:39:03.299294+0000","last_fullsized":"2024-09-19T08:39:03.299294+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.249325+0000","last_change":"2024-09-19T08:39:03.249325+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.249325+0000","last_undegraded":"2024-09-19T08:39:03.249325+0000","last_fullsized":"2024-09-19T08:39:03.249325+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:03.224537+0000","last_change":"2024-09-19T08:39:03.224537+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.224537+0000","last_undegraded":"2024-09-19T08:39:03.224537+0000","last_fullsized":"2024-09-19T08:39:03.224537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.243959+0000","last_change":"2024-09-19T08:39:03.243959+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.243959+0000","last_undegraded":"2024-09-19T08:39:03.243959+0000","last_fullsized":"2024-09-19T08:39:03.243959+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.248308+0000","last_change":"2024-09-19T08:39:03.248308+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.248308+0000","last_undegraded":"2024-09-19T08:39:03.248308+0000","last_fullsized":"2024-09-19T08:39:03.248308+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:03.224537+0000","last_change":"2024-09-19T08:39:03.224537+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.224537+0000","last_undegraded":"2024-09-19T08:39:03.224537+0000","last_fullsized":"2024-09-19T08:39:03.224537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.459475+0000","last_change":"2024-09-19T08:39:05.305633+0000","last_active":"2024-09-19T08:39:06.459475+0000","last_peered":"2024-09-19T08:39:06.459475+0000","last_clean":"2024-09-19T08:39:06.459475+0000","last_became_active":"2024-09-19T08:39:05.304913+0000","last_became_peered":"2024-09-19T08:39:05.304913+0000","last_unstale":"2024-09-19T08:39:06.459475+0000","last_undegraded":"2024-09-19T08:39:06.459475+0000","last_fullsized":"2024-09-19T08:39:06.459475+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:41:53.594289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:03.224537+0000","last_change":"2024-09-19T08:39:03.224537+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.224537+0000","last_undegraded":"2024-09-19T08:39:03.224537+0000","last_fullsized":"2024-09-19T08:39:03.224537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.248171+0000","last_change":"2024-09-19T08:39:03.248171+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.248171+0000","last_undegraded":"2024-09-19T08:39:03.248171+0000","last_fullsized":"2024-09-19T08:39:03.248171+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.245152+0000","last_change":"2024-09-19T08:39:03.245152+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.245152+0000","last_undegraded":"2024-09-19T08:39:03.245152+0000","last_fullsized":"2024-09-19T08:39:03.245152+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.244013+0000","last_change":"2024-09-19T08:39:03.244013+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.244013+0000","last_undegraded":"2024-09-19T08:39:03.244013+0000","last_fullsized":"2024-09-19T08:39:03.244013+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.244800+0000","last_change":"2024-09-19T08:39:03.244800+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.244800+0000","last_undegraded":"2024-09-19T08:39:03.244800+0000","last_fullsized":"2024-09-19T08:39:03.244800+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.299894+0000","last_change":"2024-09-19T08:39:03.299894+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.299894+0000","last_undegraded":"2024-09-19T08:39:03.299894+0000","last_fullsized":"2024-09-19T08:39:03.299894+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.458762+0000","last_change":"2024-09-19T08:39:04.285062+0000","last_active":"2024-09-19T08:39:06.458762+0000","last_peered":"2024-09-19T08:39:06.458762+0000","last_clean":"2024-09-19T08:39:06.458762+0000","last_became_active":"2024-09-19T08:39:04.284140+0000","last_became_peered":"2024-09-19T08:39:04.284140+0000","last_unstale":"2024-09-19T08:39:06.458762+0000","last_undegraded":"2024-09-19T08:39:06.458762+0000","last_fullsized":"2024-09-19T08:39:06.458762+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:00:00.495561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.245674+0000","last_change":"2024-09-19T08:39:03.245674+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.245674+0000","last_undegraded":"2024-09-19T08:39:03.245674+0000","last_fullsized":"2024-09-19T08:39:03.245674+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.248687+0000","last_change":"2024-09-19T08:39:03.248687+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.248687+0000","last_undegraded":"2024-09-19T08:39:03.248687+0000","last_fullsized":"2024-09-19T08:39:03.248687+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.246436+0000","last_change":"2024-09-19T08:39:03.246436+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.246436+0000","last_undegraded":"2024-09-19T08:39:03.246436+0000","last_fullsized":"2024-09-19T08:39:03.246436+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.299915+0000","last_change":"2024-09-19T08:39:03.299915+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.299915+0000","last_undegraded":"2024-09-19T08:39:03.299915+0000","last_fullsized":"2024-09-19T08:39:03.299915+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.246320+0000","last_change":"2024-09-19T08:39:03.246320+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.246320+0000","last_undegraded":"2024-09-19T08:39:03.246320+0000","last_fullsized":"2024-09-19T08:39:03.246320+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.458915+0000","last_change":"2024-09-19T08:39:05.276652+0000","last_active":"2024-09-19T08:39:06.458915+0000","last_peered":"2024-09-19T08:39:06.458915+0000","last_clean":"2024-09-19T08:39:06.458915+0000","last_became_active":"2024-09-19T08:39:05.275913+0000","last_became_peered":"2024-09-19T08:39:05.275913+0000","last_unstale":"2024-09-19T08:39:06.458915+0000","last_undegraded":"2024-09-19T08:39:06.458915+0000","last_fullsized":"2024-09-19T08:39:06.458915+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:36.037852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.299263+0000","last_change":"2024-09-19T08:39:03.299263+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.299263+0000","last_undegraded":"2024-09-19T08:39:03.299263+0000","last_fullsized":"2024-09-19T08:39:03.299263+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.246224+0000","last_change":"2024-09-19T08:39:03.246224+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.246224+0000","last_undegraded":"2024-09-19T08:39:03.246224+0000","last_fullsized":"2024-09-19T08:39:03.246224+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":29,"state":"creating+peering","last_fresh":"2024-09-19T08:39:03.243968+0000","last_change":"2024-09-19T08:39:03.243968+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.243968+0000","last_undegraded":"2024-09-19T08:39:03.243968+0000","last_fullsized":"2024-09-19T08:39:03.243968+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:03.224537+0000","last_change":"2024-09-19T08:39:03.224537+0000","last_active":"2024-09-19T08:39:03.224537+0000","last_peered":"2024-09-19T08:39:03.224537+0000","last_clean":"2024-09-19T08:39:03.224537+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:03.224537+0000","last_undegraded":"2024-09-19T08:39:03.224537+0000","last_fullsized":"2024-09-19T08:39:03.224537+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:04.236483+0000","last_change":"2024-09-19T08:39:04.236483+0000","last_active":"2024-09-19T08:39:04.236483+0000","last_peered":"2024-09-19T08:39:04.236483+0000","last_clean":"2024-09-19T08:39:04.236483+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:04.236483+0000","last_undegraded":"2024-09-19T08:39:04.236483+0000","last_fullsized":"2024-09-19T08:39:04.236483+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.458454+0000","last_change":"2024-09-19T08:39:04.294673+0000","last_active":"2024-09-19T08:39:06.458454+0000","last_peered":"2024-09-19T08:39:06.458454+0000","last_clean":"2024-09-19T08:39:06.458454+0000","last_became_active":"2024-09-19T08:39:04.293913+0000","last_became_peered":"2024-09-19T08:39:04.293913+0000","last_unstale":"2024-09-19T08:39:06.458454+0000","last_undegraded":"2024-09-19T08:39:06.458454+0000","last_fullsized":"2024-09-19T08:39:06.458454+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:12.830812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":182,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-19T08:39:03.254315+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:39:03.254315+0000","last_peered":"2024-09-19T08:39:03.254315+0000","last_clean":"2024-09-19T08:39:03.254315+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:39:03.254315+0000","last_undegraded":"2024-09-19T08:39:03.254315+0000","last_fullsized":"2024-09-19T08:39:03.254315+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":10,"acting":10,"num_store_stats":1},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":50,"acting":50,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509454,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":897024,"data_stored":700868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":315392,"data_stored":116317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":315392,"data_stored":116317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509453,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":315392,"data_stored":116317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509455,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":303104,"data_stored":110500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509453,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27916,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343924,"statfs":{"total":96636764160,"available":96608178176,"internally_reserved":0,"allocated":909312,"data_stored":706685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":360448,"data_stored":136275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27324,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":303104,"data_stored":110500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:39:12.768 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:39:12.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.895+0000 7f11c9e46640 1 Processor -- start 2024-09-19T08:39:12.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.895+0000 7f11c9e46640 1 -- start start 2024-09-19T08:39:12.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.895+0000 7f11c9e46640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f11c406dd10 0x7f11c40b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:12.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.895+0000 7f11c9e46640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f11c406c070 0x7f11c4070ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:12.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.895+0000 7f11c9e46640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11c40b3a20 con 0x7f11c406bc90 2024-09-19T08:39:12.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.895+0000 7f11c9e46640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f11c40cc9a0 con 0x7f11c406dd10 2024-09-19T08:39:12.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.895+0000 7f11c9e46640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f11c40ccc80 con 0x7f11c406c070 2024-09-19T08:39:12.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.895+0000 7f11c3fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f11c406c070 0x7f11c4070ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:12.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.895+0000 7f11bbfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f11c406dd10 0x7f11c40b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:12.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.895+0000 7f11c3fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f11c406c070 0x7f11c4070ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:44478/0 (socket says 172.21.15.98:44478) 2024-09-19T08:39:12.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.895+0000 7f11c3fff640 1 -- 172.21.15.98:0/1676409695 learned_addr learned my addr 172.21.15.98:0/1676409695 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:12.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.895+0000 7f11bbfff640 1 -- 172.21.15.98:0/1676409695 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f11c406c070 msgr2=0x7f11c4070ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:12.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.895+0000 7f11bbfff640 1 --2- 172.21.15.98:0/1676409695 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f11c406c070 0x7f11c4070ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:12.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.895+0000 7f11bbfff640 1 -- 172.21.15.98:0/1676409695 >> v1:172.21.15.139:6789/0 conn(0x7f11c406bc90 legacy=0x7f11c40cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:12.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.895+0000 7f11bbfff640 1 -- 172.21.15.98:0/1676409695 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11c40b51a0 con 0x7f11c406dd10 2024-09-19T08:39:12.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.899+0000 7f11bbfff640 1 --2- 172.21.15.98:0/1676409695 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11c406dd10 0x7f11c40b34e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f11ac0097c0 tx=0x7f11ac01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de81be7735fba9c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:12.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.899+0000 7f11c2ffd640 1 -- 172.21.15.98:0/1676409695 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac0b0070 con 0x7f11c406dd10 2024-09-19T08:39:12.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.899+0000 7f11c2ffd640 1 -- 172.21.15.98:0/1676409695 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11ac0ab600 con 0x7f11c406dd10 2024-09-19T08:39:12.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.899+0000 7f11c2ffd640 1 -- 172.21.15.98:0/1676409695 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac0ab9d0 con 0x7f11c406dd10 2024-09-19T08:39:12.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.899+0000 7f11c9e46640 1 -- 172.21.15.98:0/1676409695 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11c406dd10 msgr2=0x7f11c40b34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:12.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.899+0000 7f11c9e46640 1 --2- 172.21.15.98:0/1676409695 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11c406dd10 0x7f11c40b34e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f11ac0097c0 tx=0x7f11ac01c410 comp rx=0 tx=0).stop 2024-09-19T08:39:12.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.899+0000 7f11c9e46640 1 -- 172.21.15.98:0/1676409695 shutdown_connections 2024-09-19T08:39:12.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.899+0000 7f11c9e46640 1 --2- 172.21.15.98:0/1676409695 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f11c406c070 0x7f11c4070ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:12.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.899+0000 7f11c9e46640 1 --2- 172.21.15.98:0/1676409695 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11c406dd10 0x7f11c40b34e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:12.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.899+0000 7f11c9e46640 1 -- 172.21.15.98:0/1676409695 >> 172.21.15.98:0/1676409695 conn(0x7f11c40b2010 msgr2=0x7f11c40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:12.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.899+0000 7f11c9e46640 1 -- 172.21.15.98:0/1676409695 shutdown_connections 2024-09-19T08:39:12.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.899+0000 7f11c9e46640 1 -- 172.21.15.98:0/1676409695 wait complete. 2024-09-19T08:39:12.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.899+0000 7f11c9e46640 1 Processor -- start 2024-09-19T08:39:12.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.899+0000 7f11c9e46640 1 -- start start 2024-09-19T08:39:12.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.903+0000 7f11c9e46640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f11c406bc90 0x7f11c414a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:12.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.903+0000 7f11c9e46640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11c406c070 0x7f11c40ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:12.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.903+0000 7f11c9e46640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11c406dd10 0x7f11c40aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:12.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.903+0000 7f11c9e46640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f11c4167510 con 0x7f11c406dd10 2024-09-19T08:39:12.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.903+0000 7f11c9e46640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f11c4167390 con 0x7f11c406c070 2024-09-19T08:39:12.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.903+0000 7f11c9e46640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f11c4167690 con 0x7f11c406bc90 2024-09-19T08:39:12.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.903+0000 7f11c3fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11c406dd10 0x7f11c40aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:12.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.903+0000 7f11c3fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11c406dd10 0x7f11c40aedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36460/0 (socket says 172.21.15.98:36460) 2024-09-19T08:39:12.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.903+0000 7f11c3fff640 1 -- 172.21.15.98:0/879810832 learned_addr learned my addr 172.21.15.98:0/879810832 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:12.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.903+0000 7f11c37fe640 1 --2- 172.21.15.98:0/879810832 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f11c406bc90 0x7f11c414a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:12.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.903+0000 7f11bbfff640 1 --2- 172.21.15.98:0/879810832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11c406c070 0x7f11c40ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:12.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.903+0000 7f11c3fff640 1 -- 172.21.15.98:0/879810832 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f11c406bc90 msgr2=0x7f11c414a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:12.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.903+0000 7f11c3fff640 1 --2- 172.21.15.98:0/879810832 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f11c406bc90 0x7f11c414a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:12.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.903+0000 7f11c3fff640 1 -- 172.21.15.98:0/879810832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11c406c070 msgr2=0x7f11c40ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:12.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.903+0000 7f11c3fff640 1 --2- 172.21.15.98:0/879810832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11c406c070 0x7f11c40ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:12.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.903+0000 7f11c3fff640 1 -- 172.21.15.98:0/879810832 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11c4073490 con 0x7f11c406dd10 2024-09-19T08:39:12.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.903+0000 7f11c3fff640 1 --2- 172.21.15.98:0/879810832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11c406dd10 0x7f11c40aedf0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f11bc009e50 tx=0x7f11bc0075e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:12.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.903+0000 7f11c0ff9640 1 -- 172.21.15.98:0/879810832 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc019070 con 0x7f11c406dd10 2024-09-19T08:39:12.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.903+0000 7f11c9e46640 1 -- 172.21.15.98:0/879810832 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11c4156ea0 con 0x7f11c406dd10 2024-09-19T08:39:12.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.903+0000 7f11c0ff9640 1 -- 172.21.15.98:0/879810832 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11bc0039a0 con 0x7f11c406dd10 2024-09-19T08:39:12.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.903+0000 7f11c0ff9640 1 -- 172.21.15.98:0/879810832 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc003d70 con 0x7f11c406dd10 2024-09-19T08:39:12.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.903+0000 7f11c9e46640 1 -- 172.21.15.98:0/879810832 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11c4157460 con 0x7f11c406dd10 2024-09-19T08:39:12.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.903+0000 7f11c9e46640 1 -- 172.21.15.98:0/879810832 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11c4070ed0 con 0x7f11c406dd10 2024-09-19T08:39:12.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.903+0000 7f11c0ff9640 1 -- 172.21.15.98:0/879810832 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f11bc01c070 con 0x7f11c406dd10 2024-09-19T08:39:12.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.903+0000 7f11c0ff9640 1 --2- 172.21.15.98:0/879810832 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f11a40678c0 0x7f11a4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:12.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.903+0000 7f11c0ff9640 1 -- 172.21.15.98:0/879810832 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f11bc085a50 con 0x7f11c406dd10 2024-09-19T08:39:12.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.903+0000 7f11c37fe640 1 --2- 172.21.15.98:0/879810832 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f11a40678c0 0x7f11a4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:12.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.907+0000 7f11c37fe640 1 --2- 172.21.15.98:0/879810832 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f11a40678c0 0x7f11a4069d80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f11b4001770 tx=0x7f11b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:12.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:12.911+0000 7f11c0ff9640 1 -- 172.21.15.98:0/879810832 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f11bc011070 con 0x7f11c406dd10 2024-09-19T08:39:13.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.071+0000 7f11c9e46640 1 -- 172.21.15.98:0/879810832 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f11c414aec0 con 0x7f11a40678c0 2024-09-19T08:39:13.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.075+0000 7f11c0ff9640 1 -- 172.21.15.98:0/879810832 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251957 (secure 0 0 0) 0x7f11b40013b0 con 0x7f11a40678c0 2024-09-19T08:39:13.082 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:39:13.083 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:39:13.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.079+0000 7f11c9e46640 1 -- 172.21.15.98:0/879810832 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f11a40678c0 msgr2=0x7f11a4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:13.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.079+0000 7f11c9e46640 1 --2- 172.21.15.98:0/879810832 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f11a40678c0 0x7f11a4069d80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f11b4001770 tx=0x7f11b4009040 comp rx=0 tx=0).stop 2024-09-19T08:39:13.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.079+0000 7f11c9e46640 1 -- 172.21.15.98:0/879810832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11c406dd10 msgr2=0x7f11c40aedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:13.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.079+0000 7f11c9e46640 1 --2- 172.21.15.98:0/879810832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11c406dd10 0x7f11c40aedf0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f11bc009e50 tx=0x7f11bc0075e0 comp rx=0 tx=0).stop 2024-09-19T08:39:13.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.079+0000 7f11c9e46640 1 -- 172.21.15.98:0/879810832 shutdown_connections 2024-09-19T08:39:13.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.079+0000 7f11c9e46640 1 --2- 172.21.15.98:0/879810832 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f11a40678c0 0x7f11a4069d80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:13.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.079+0000 7f11c9e46640 1 --2- 172.21.15.98:0/879810832 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11c406dd10 0x7f11c40aedf0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:13.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.079+0000 7f11c9e46640 1 --2- 172.21.15.98:0/879810832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11c406c070 0x7f11c40ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:13.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.079+0000 7f11c9e46640 1 --2- 172.21.15.98:0/879810832 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f11c406bc90 0x7f11c414a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:13.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.079+0000 7f11c9e46640 1 -- 172.21.15.98:0/879810832 >> 172.21.15.98:0/879810832 conn(0x7f11c40b2010 msgr2=0x7f11c40d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:13.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.079+0000 7f11c9e46640 1 -- 172.21.15.98:0/879810832 shutdown_connections 2024-09-19T08:39:13.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.079+0000 7f11c9e46640 1 -- 172.21.15.98:0/879810832 wait complete. 2024-09-19T08:39:13.100 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2024-09-19T08:39:12.041792+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":155,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220224,"kb_used_data":3976,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754496,"statfs":{"total":773094113280,"available":772868603904,"internally_reserved":0,"allocated":4071424,"data_stored":2270828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":50,"apply_latency_ms":50,"commit_latency_ns":50000000,"apply_latency_ns":50000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.793712"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.458834+0000","last_change":"2024-09-19T08:39:05.300955+0000","last_active":"2024-09-19T08:39:06.458834+0000","last_peered":"2024-09-19T08:39:06.458834+0000","last_clean":"2024-09-19T08:39:06.458834+0000","last_became_active":"2024-09-19T08:39:05.300228+0000","last_became_peered":"2024-09-19T08:39:05.300228+0000","last_unstale":"2024-09-19T08:39:06.458834+0000","last_undegraded":"2024-09-19T08:39:06.458834+0000","last_fullsized":"2024-09-19T08:39:06.458834+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:05.814581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.274258+0000","last_change":"2024-09-19T08:39:05.275094+0000","last_active":"2024-09-19T08:39:06.274258+0000","last_peered":"2024-09-19T08:39:06.274258+0000","last_clean":"2024-09-19T08:39:06.274258+0000","last_became_active":"2024-09-19T08:39:05.274807+0000","last_became_peered":"2024-09-19T08:39:05.274807+0000","last_unstale":"2024-09-19T08:39:06.274258+0000","last_undegraded":"2024-09-19T08:39:06.274258+0000","last_fullsized":"2024-09-19T08:39:06.274258+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:36.497290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.461192+0000","last_change":"2024-09-19T08:39:05.292241+0000","last_active":"2024-09-19T08:39:06.461192+0000","last_peered":"2024-09-19T08:39:06.461192+0000","last_clean":"2024-09-19T08:39:06.461192+0000","last_became_active":"2024-09-19T08:39:05.290242+0000","last_became_peered":"2024-09-19T08:39:05.290242+0000","last_unstale":"2024-09-19T08:39:06.461192+0000","last_undegraded":"2024-09-19T08:39:06.461192+0000","last_fullsized":"2024-09-19T08:39:06.461192+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:55.703531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.453611+0000","last_change":"2024-09-19T08:39:05.291420+0000","last_active":"2024-09-19T08:39:06.453611+0000","last_peered":"2024-09-19T08:39:06.453611+0000","last_clean":"2024-09-19T08:39:06.453611+0000","last_became_active":"2024-09-19T08:39:05.291168+0000","last_became_peered":"2024-09-19T08:39:05.291168+0000","last_unstale":"2024-09-19T08:39:06.453611+0000","last_undegraded":"2024-09-19T08:39:06.453611+0000","last_fullsized":"2024-09-19T08:39:06.453611+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:31:44.442577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.454063+0000","last_change":"2024-09-19T08:39:05.269411+0000","last_active":"2024-09-19T08:39:06.454063+0000","last_peered":"2024-09-19T08:39:06.454063+0000","last_clean":"2024-09-19T08:39:06.454063+0000","last_became_active":"2024-09-19T08:39:05.269049+0000","last_became_peered":"2024-09-19T08:39:05.269049+0000","last_unstale":"2024-09-19T08:39:06.454063+0000","last_undegraded":"2024-09-19T08:39:06.454063+0000","last_fullsized":"2024-09-19T08:39:06.454063+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:16:30.685035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.273411+0000","last_change":"2024-09-19T08:39:05.294504+0000","last_active":"2024-09-19T08:39:06.273411+0000","last_peered":"2024-09-19T08:39:06.273411+0000","last_clean":"2024-09-19T08:39:06.273411+0000","last_became_active":"2024-09-19T08:39:05.294299+0000","last_became_peered":"2024-09-19T08:39:05.294299+0000","last_unstale":"2024-09-19T08:39:06.273411+0000","last_undegraded":"2024-09-19T08:39:06.273411+0000","last_fullsized":"2024-09-19T08:39:06.273411+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:33.706222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.460026+0000","last_change":"2024-09-19T08:39:05.310062+0000","last_active":"2024-09-19T08:39:06.460026+0000","last_peered":"2024-09-19T08:39:06.460026+0000","last_clean":"2024-09-19T08:39:06.460026+0000","last_became_active":"2024-09-19T08:39:05.309295+0000","last_became_peered":"2024-09-19T08:39:05.309295+0000","last_unstale":"2024-09-19T08:39:06.460026+0000","last_undegraded":"2024-09-19T08:39:06.460026+0000","last_fullsized":"2024-09-19T08:39:06.460026+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:16.032596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.459693+0000","last_change":"2024-09-19T08:39:05.296643+0000","last_active":"2024-09-19T08:39:06.459693+0000","last_peered":"2024-09-19T08:39:06.459693+0000","last_clean":"2024-09-19T08:39:06.459693+0000","last_became_active":"2024-09-19T08:39:05.293978+0000","last_became_peered":"2024-09-19T08:39:05.293978+0000","last_unstale":"2024-09-19T08:39:06.459693+0000","last_undegraded":"2024-09-19T08:39:06.459693+0000","last_fullsized":"2024-09-19T08:39:06.459693+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:07.521988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.280933+0000","last_change":"2024-09-19T08:39:05.269459+0000","last_active":"2024-09-19T08:39:06.280933+0000","last_peered":"2024-09-19T08:39:06.280933+0000","last_clean":"2024-09-19T08:39:06.280933+0000","last_became_active":"2024-09-19T08:39:05.265720+0000","last_became_peered":"2024-09-19T08:39:05.265720+0000","last_unstale":"2024-09-19T08:39:06.280933+0000","last_undegraded":"2024-09-19T08:39:06.280933+0000","last_fullsized":"2024-09-19T08:39:06.280933+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:21.553060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.455909+0000","last_change":"2024-09-19T08:39:05.269077+0000","last_active":"2024-09-19T08:39:06.455909+0000","last_peered":"2024-09-19T08:39:06.455909+0000","last_clean":"2024-09-19T08:39:06.455909+0000","last_became_active":"2024-09-19T08:39:05.268585+0000","last_became_peered":"2024-09-19T08:39:05.268585+0000","last_unstale":"2024-09-19T08:39:06.455909+0000","last_undegraded":"2024-09-19T08:39:06.455909+0000","last_fullsized":"2024-09-19T08:39:06.455909+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:34.799076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.461608+0000","last_change":"2024-09-19T08:39:05.298410+0000","last_active":"2024-09-19T08:39:06.461608+0000","last_peered":"2024-09-19T08:39:06.461608+0000","last_clean":"2024-09-19T08:39:06.461608+0000","last_became_active":"2024-09-19T08:39:05.297764+0000","last_became_peered":"2024-09-19T08:39:05.297764+0000","last_unstale":"2024-09-19T08:39:06.461608+0000","last_undegraded":"2024-09-19T08:39:06.461608+0000","last_fullsized":"2024-09-19T08:39:06.461608+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:47.034506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.460330+0000","last_change":"2024-09-19T08:39:05.291183+0000","last_active":"2024-09-19T08:39:06.460330+0000","last_peered":"2024-09-19T08:39:06.460330+0000","last_clean":"2024-09-19T08:39:06.460330+0000","last_became_active":"2024-09-19T08:39:05.290494+0000","last_became_peered":"2024-09-19T08:39:05.290494+0000","last_unstale":"2024-09-19T08:39:06.460330+0000","last_undegraded":"2024-09-19T08:39:06.460330+0000","last_fullsized":"2024-09-19T08:39:06.460330+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:18:49.490651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.454075+0000","last_change":"2024-09-19T08:39:05.268385+0000","last_active":"2024-09-19T08:39:06.454075+0000","last_peered":"2024-09-19T08:39:06.454075+0000","last_clean":"2024-09-19T08:39:06.454075+0000","last_became_active":"2024-09-19T08:39:05.268079+0000","last_became_peered":"2024-09-19T08:39:05.268079+0000","last_unstale":"2024-09-19T08:39:06.454075+0000","last_undegraded":"2024-09-19T08:39:06.454075+0000","last_fullsized":"2024-09-19T08:39:06.454075+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:00:43.205293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.273317+0000","last_change":"2024-09-19T08:39:05.273636+0000","last_active":"2024-09-19T08:39:06.273317+0000","last_peered":"2024-09-19T08:39:06.273317+0000","last_clean":"2024-09-19T08:39:06.273317+0000","last_became_active":"2024-09-19T08:39:05.273296+0000","last_became_peered":"2024-09-19T08:39:05.273296+0000","last_unstale":"2024-09-19T08:39:06.273317+0000","last_undegraded":"2024-09-19T08:39:06.273317+0000","last_fullsized":"2024-09-19T08:39:06.273317+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:46.998357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.458250+0000","last_change":"2024-09-19T08:39:05.312052+0000","last_active":"2024-09-19T08:39:06.458250+0000","last_peered":"2024-09-19T08:39:06.458250+0000","last_clean":"2024-09-19T08:39:06.458250+0000","last_became_active":"2024-09-19T08:39:05.311355+0000","last_became_peered":"2024-09-19T08:39:05.311355+0000","last_unstale":"2024-09-19T08:39:06.458250+0000","last_undegraded":"2024-09-19T08:39:06.458250+0000","last_fullsized":"2024-09-19T08:39:06.458250+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:22.498460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.460992+0000","last_change":"2024-09-19T08:39:05.299731+0000","last_active":"2024-09-19T08:39:06.460992+0000","last_peered":"2024-09-19T08:39:06.460992+0000","last_clean":"2024-09-19T08:39:06.460992+0000","last_became_active":"2024-09-19T08:39:05.299083+0000","last_became_peered":"2024-09-19T08:39:05.299083+0000","last_unstale":"2024-09-19T08:39:06.460992+0000","last_undegraded":"2024-09-19T08:39:06.460992+0000","last_fullsized":"2024-09-19T08:39:06.460992+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:33:09.963564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.462349+0000","last_change":"2024-09-19T08:39:05.309568+0000","last_active":"2024-09-19T08:39:06.462349+0000","last_peered":"2024-09-19T08:39:06.462349+0000","last_clean":"2024-09-19T08:39:06.462349+0000","last_became_active":"2024-09-19T08:39:05.308859+0000","last_became_peered":"2024-09-19T08:39:05.308859+0000","last_unstale":"2024-09-19T08:39:06.462349+0000","last_undegraded":"2024-09-19T08:39:06.462349+0000","last_fullsized":"2024-09-19T08:39:06.462349+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:55.703531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.274103+0000","last_change":"2024-09-19T08:39:05.273262+0000","last_active":"2024-09-19T08:39:06.274103+0000","last_peered":"2024-09-19T08:39:06.274103+0000","last_clean":"2024-09-19T08:39:06.274103+0000","last_became_active":"2024-09-19T08:39:05.272934+0000","last_became_peered":"2024-09-19T08:39:05.272934+0000","last_unstale":"2024-09-19T08:39:06.274103+0000","last_undegraded":"2024-09-19T08:39:06.274103+0000","last_fullsized":"2024-09-19T08:39:06.274103+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:09.146478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.273863+0000","last_change":"2024-09-19T08:39:05.280901+0000","last_active":"2024-09-19T08:39:06.273863+0000","last_peered":"2024-09-19T08:39:06.273863+0000","last_clean":"2024-09-19T08:39:06.273863+0000","last_became_active":"2024-09-19T08:39:05.280402+0000","last_became_peered":"2024-09-19T08:39:05.280402+0000","last_unstale":"2024-09-19T08:39:06.273863+0000","last_undegraded":"2024-09-19T08:39:06.273863+0000","last_fullsized":"2024-09-19T08:39:06.273863+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:54.618548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.454594+0000","last_change":"2024-09-19T08:39:05.292712+0000","last_active":"2024-09-19T08:39:06.454594+0000","last_peered":"2024-09-19T08:39:06.454594+0000","last_clean":"2024-09-19T08:39:06.454594+0000","last_became_active":"2024-09-19T08:39:05.292462+0000","last_became_peered":"2024-09-19T08:39:05.292462+0000","last_unstale":"2024-09-19T08:39:06.454594+0000","last_undegraded":"2024-09-19T08:39:06.454594+0000","last_fullsized":"2024-09-19T08:39:06.454594+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:22.025013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.278949+0000","last_change":"2024-09-19T08:39:05.292831+0000","last_active":"2024-09-19T08:39:06.278949+0000","last_peered":"2024-09-19T08:39:06.278949+0000","last_clean":"2024-09-19T08:39:06.278949+0000","last_became_active":"2024-09-19T08:39:05.290014+0000","last_became_peered":"2024-09-19T08:39:05.290014+0000","last_unstale":"2024-09-19T08:39:06.278949+0000","last_undegraded":"2024-09-19T08:39:06.278949+0000","last_fullsized":"2024-09-19T08:39:06.278949+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:36:26.198031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.279244+0000","last_change":"2024-09-19T08:39:05.276750+0000","last_active":"2024-09-19T08:39:06.279244+0000","last_peered":"2024-09-19T08:39:06.279244+0000","last_clean":"2024-09-19T08:39:06.279244+0000","last_became_active":"2024-09-19T08:39:05.276037+0000","last_became_peered":"2024-09-19T08:39:05.276037+0000","last_unstale":"2024-09-19T08:39:06.279244+0000","last_undegraded":"2024-09-19T08:39:06.279244+0000","last_fullsized":"2024-09-19T08:39:06.279244+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:42:42.724415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.453148+0000","last_change":"2024-09-19T08:39:05.289518+0000","last_active":"2024-09-19T08:39:06.453148+0000","last_peered":"2024-09-19T08:39:06.453148+0000","last_clean":"2024-09-19T08:39:06.453148+0000","last_became_active":"2024-09-19T08:39:05.289023+0000","last_became_peered":"2024-09-19T08:39:05.289023+0000","last_unstale":"2024-09-19T08:39:06.453148+0000","last_undegraded":"2024-09-19T08:39:06.453148+0000","last_fullsized":"2024-09-19T08:39:06.453148+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:16.522784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.457790+0000","last_change":"2024-09-19T08:39:05.297103+0000","last_active":"2024-09-19T08:39:06.457790+0000","last_peered":"2024-09-19T08:39:06.457790+0000","last_clean":"2024-09-19T08:39:06.457790+0000","last_became_active":"2024-09-19T08:39:05.294582+0000","last_became_peered":"2024-09-19T08:39:05.294582+0000","last_unstale":"2024-09-19T08:39:06.457790+0000","last_undegraded":"2024-09-19T08:39:06.457790+0000","last_fullsized":"2024-09-19T08:39:06.457790+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:41:23.042119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.455885+0000","last_change":"2024-09-19T08:39:05.293312+0000","last_active":"2024-09-19T08:39:06.455885+0000","last_peered":"2024-09-19T08:39:06.455885+0000","last_clean":"2024-09-19T08:39:06.455885+0000","last_became_active":"2024-09-19T08:39:05.292952+0000","last_became_peered":"2024-09-19T08:39:05.292952+0000","last_unstale":"2024-09-19T08:39:06.455885+0000","last_undegraded":"2024-09-19T08:39:06.455885+0000","last_fullsized":"2024-09-19T08:39:06.455885+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:23:34.885764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.274176+0000","last_change":"2024-09-19T08:39:05.293159+0000","last_active":"2024-09-19T08:39:06.274176+0000","last_peered":"2024-09-19T08:39:06.274176+0000","last_clean":"2024-09-19T08:39:06.274176+0000","last_became_active":"2024-09-19T08:39:05.292844+0000","last_became_peered":"2024-09-19T08:39:05.292844+0000","last_unstale":"2024-09-19T08:39:06.274176+0000","last_undegraded":"2024-09-19T08:39:06.274176+0000","last_fullsized":"2024-09-19T08:39:06.274176+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:36.815356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.455937+0000","last_change":"2024-09-19T08:39:05.291447+0000","last_active":"2024-09-19T08:39:06.455937+0000","last_peered":"2024-09-19T08:39:06.455937+0000","last_clean":"2024-09-19T08:39:06.455937+0000","last_became_active":"2024-09-19T08:39:05.291136+0000","last_became_peered":"2024-09-19T08:39:05.291136+0000","last_unstale":"2024-09-19T08:39:06.455937+0000","last_undegraded":"2024-09-19T08:39:06.455937+0000","last_fullsized":"2024-09-19T08:39:06.455937+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:07.521988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.460742+0000","last_change":"2024-09-19T08:39:05.312472+0000","last_active":"2024-09-19T08:39:06.460742+0000","last_peered":"2024-09-19T08:39:06.460742+0000","last_clean":"2024-09-19T08:39:06.460742+0000","last_became_active":"2024-09-19T08:39:05.311858+0000","last_became_peered":"2024-09-19T08:39:05.311858+0000","last_unstale":"2024-09-19T08:39:06.460742+0000","last_undegraded":"2024-09-19T08:39:06.460742+0000","last_fullsized":"2024-09-19T08:39:06.460742+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:22:55.001406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.279346+0000","last_change":"2024-09-19T08:39:05.273228+0000","last_active":"2024-09-19T08:39:06.279346+0000","last_peered":"2024-09-19T08:39:06.279346+0000","last_clean":"2024-09-19T08:39:06.279346+0000","last_became_active":"2024-09-19T08:39:05.272501+0000","last_became_peered":"2024-09-19T08:39:05.272501+0000","last_unstale":"2024-09-19T08:39:06.279346+0000","last_undegraded":"2024-09-19T08:39:06.279346+0000","last_fullsized":"2024-09-19T08:39:06.279346+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:02.155985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.459964+0000","last_change":"2024-09-19T08:39:05.304057+0000","last_active":"2024-09-19T08:39:06.459964+0000","last_peered":"2024-09-19T08:39:06.459964+0000","last_clean":"2024-09-19T08:39:06.459964+0000","last_became_active":"2024-09-19T08:39:05.303354+0000","last_became_peered":"2024-09-19T08:39:05.303354+0000","last_unstale":"2024-09-19T08:39:06.459964+0000","last_undegraded":"2024-09-19T08:39:06.459964+0000","last_fullsized":"2024-09-19T08:39:06.459964+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:41:23.042119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.459141+0000","last_change":"2024-09-19T08:39:05.276219+0000","last_active":"2024-09-19T08:39:06.459141+0000","last_peered":"2024-09-19T08:39:06.459141+0000","last_clean":"2024-09-19T08:39:06.459141+0000","last_became_active":"2024-09-19T08:39:05.275466+0000","last_became_peered":"2024-09-19T08:39:05.275466+0000","last_unstale":"2024-09-19T08:39:06.459141+0000","last_undegraded":"2024-09-19T08:39:06.459141+0000","last_fullsized":"2024-09-19T08:39:06.459141+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:09.038404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.459934+0000","last_change":"2024-09-19T08:39:05.317125+0000","last_active":"2024-09-19T08:39:06.459934+0000","last_peered":"2024-09-19T08:39:06.459934+0000","last_clean":"2024-09-19T08:39:06.459934+0000","last_became_active":"2024-09-19T08:39:05.310831+0000","last_became_peered":"2024-09-19T08:39:05.310831+0000","last_unstale":"2024-09-19T08:39:06.459934+0000","last_undegraded":"2024-09-19T08:39:06.459934+0000","last_fullsized":"2024-09-19T08:39:06.459934+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:38.339269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.454853+0000","last_change":"2024-09-19T08:39:05.291105+0000","last_active":"2024-09-19T08:39:06.454853+0000","last_peered":"2024-09-19T08:39:06.454853+0000","last_clean":"2024-09-19T08:39:06.454853+0000","last_became_active":"2024-09-19T08:39:05.290561+0000","last_became_peered":"2024-09-19T08:39:05.290561+0000","last_unstale":"2024-09-19T08:39:06.454853+0000","last_undegraded":"2024-09-19T08:39:06.454853+0000","last_fullsized":"2024-09-19T08:39:06.454853+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:42:42.724415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.459079+0000","last_change":"2024-09-19T08:39:04.292831+0000","last_active":"2024-09-19T08:39:06.459079+0000","last_peered":"2024-09-19T08:39:06.459079+0000","last_clean":"2024-09-19T08:39:06.459079+0000","last_became_active":"2024-09-19T08:39:04.292058+0000","last_became_peered":"2024-09-19T08:39:04.292058+0000","last_unstale":"2024-09-19T08:39:06.459079+0000","last_undegraded":"2024-09-19T08:39:06.459079+0000","last_fullsized":"2024-09-19T08:39:06.459079+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:38:41.573373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.459180+0000","last_change":"2024-09-19T08:39:05.305411+0000","last_active":"2024-09-19T08:39:06.459180+0000","last_peered":"2024-09-19T08:39:06.459180+0000","last_clean":"2024-09-19T08:39:06.459180+0000","last_became_active":"2024-09-19T08:39:05.304823+0000","last_became_peered":"2024-09-19T08:39:05.304823+0000","last_unstale":"2024-09-19T08:39:06.459180+0000","last_undegraded":"2024-09-19T08:39:06.459180+0000","last_fullsized":"2024-09-19T08:39:06.459180+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:11.979445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.454906+0000","last_change":"2024-09-19T08:39:04.285630+0000","last_active":"2024-09-19T08:39:06.454906+0000","last_peered":"2024-09-19T08:39:06.454906+0000","last_clean":"2024-09-19T08:39:06.454906+0000","last_became_active":"2024-09-19T08:39:04.285135+0000","last_became_peered":"2024-09-19T08:39:04.285135+0000","last_unstale":"2024-09-19T08:39:06.454906+0000","last_undegraded":"2024-09-19T08:39:06.454906+0000","last_fullsized":"2024-09-19T08:39:06.454906+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:48:51.832038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.280472+0000","last_change":"2024-09-19T08:39:05.286027+0000","last_active":"2024-09-19T08:39:06.280472+0000","last_peered":"2024-09-19T08:39:06.280472+0000","last_clean":"2024-09-19T08:39:06.280472+0000","last_became_active":"2024-09-19T08:39:05.280465+0000","last_became_peered":"2024-09-19T08:39:05.280465+0000","last_unstale":"2024-09-19T08:39:06.280472+0000","last_undegraded":"2024-09-19T08:39:06.280472+0000","last_fullsized":"2024-09-19T08:39:06.280472+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:31:44.442577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.273899+0000","last_change":"2024-09-19T08:39:04.274746+0000","last_active":"2024-09-19T08:39:06.273899+0000","last_peered":"2024-09-19T08:39:06.273899+0000","last_clean":"2024-09-19T08:39:06.273899+0000","last_became_active":"2024-09-19T08:39:04.270304+0000","last_became_peered":"2024-09-19T08:39:04.270304+0000","last_unstale":"2024-09-19T08:39:06.273899+0000","last_undegraded":"2024-09-19T08:39:06.273899+0000","last_fullsized":"2024-09-19T08:39:06.273899+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:15.074411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.456121+0000","last_change":"2024-09-19T08:39:05.269016+0000","last_active":"2024-09-19T08:39:06.456121+0000","last_peered":"2024-09-19T08:39:06.456121+0000","last_clean":"2024-09-19T08:39:06.456121+0000","last_became_active":"2024-09-19T08:39:05.268734+0000","last_became_peered":"2024-09-19T08:39:05.268734+0000","last_unstale":"2024-09-19T08:39:06.456121+0000","last_undegraded":"2024-09-19T08:39:06.456121+0000","last_fullsized":"2024-09-19T08:39:06.456121+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:38.339269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.454434+0000","last_change":"2024-09-19T08:39:04.273916+0000","last_active":"2024-09-19T08:39:06.454434+0000","last_peered":"2024-09-19T08:39:06.454434+0000","last_clean":"2024-09-19T08:39:06.454434+0000","last_became_active":"2024-09-19T08:39:04.273627+0000","last_became_peered":"2024-09-19T08:39:04.273627+0000","last_unstale":"2024-09-19T08:39:06.454434+0000","last_undegraded":"2024-09-19T08:39:06.454434+0000","last_fullsized":"2024-09-19T08:39:06.454434+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:14:42.053182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.460356+0000","last_change":"2024-09-19T08:39:05.310429+0000","last_active":"2024-09-19T08:39:06.460356+0000","last_peered":"2024-09-19T08:39:06.460356+0000","last_clean":"2024-09-19T08:39:06.460356+0000","last_became_active":"2024-09-19T08:39:05.309844+0000","last_became_peered":"2024-09-19T08:39:05.309844+0000","last_unstale":"2024-09-19T08:39:06.460356+0000","last_undegraded":"2024-09-19T08:39:06.460356+0000","last_fullsized":"2024-09-19T08:39:06.460356+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:22.498460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.279477+0000","last_change":"2024-09-19T08:39:04.291083+0000","last_active":"2024-09-19T08:39:06.279477+0000","last_peered":"2024-09-19T08:39:06.279477+0000","last_clean":"2024-09-19T08:39:06.279477+0000","last_became_active":"2024-09-19T08:39:04.290351+0000","last_became_peered":"2024-09-19T08:39:04.290351+0000","last_unstale":"2024-09-19T08:39:06.279477+0000","last_undegraded":"2024-09-19T08:39:06.279477+0000","last_fullsized":"2024-09-19T08:39:06.279477+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:48:51.832038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.457992+0000","last_change":"2024-09-19T08:39:05.278165+0000","last_active":"2024-09-19T08:39:06.457992+0000","last_peered":"2024-09-19T08:39:06.457992+0000","last_clean":"2024-09-19T08:39:06.457992+0000","last_became_active":"2024-09-19T08:39:05.277474+0000","last_became_peered":"2024-09-19T08:39:05.277474+0000","last_unstale":"2024-09-19T08:39:06.457992+0000","last_undegraded":"2024-09-19T08:39:06.457992+0000","last_fullsized":"2024-09-19T08:39:06.457992+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:16.032596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.279344+0000","last_change":"2024-09-19T08:39:04.273879+0000","last_active":"2024-09-19T08:39:06.279344+0000","last_peered":"2024-09-19T08:39:06.279344+0000","last_clean":"2024-09-19T08:39:06.279344+0000","last_became_active":"2024-09-19T08:39:04.273125+0000","last_became_peered":"2024-09-19T08:39:04.273125+0000","last_unstale":"2024-09-19T08:39:06.279344+0000","last_undegraded":"2024-09-19T08:39:06.279344+0000","last_fullsized":"2024-09-19T08:39:06.279344+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:00:42.193347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.278947+0000","last_change":"2024-09-19T08:39:05.285741+0000","last_active":"2024-09-19T08:39:06.278947+0000","last_peered":"2024-09-19T08:39:06.278947+0000","last_clean":"2024-09-19T08:39:06.278947+0000","last_became_active":"2024-09-19T08:39:05.285026+0000","last_became_peered":"2024-09-19T08:39:05.285026+0000","last_unstale":"2024-09-19T08:39:06.278947+0000","last_undegraded":"2024-09-19T08:39:06.278947+0000","last_fullsized":"2024-09-19T08:39:06.278947+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:54.222762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.273251+0000","last_change":"2024-09-19T08:39:04.276660+0000","last_active":"2024-09-19T08:39:06.273251+0000","last_peered":"2024-09-19T08:39:06.273251+0000","last_clean":"2024-09-19T08:39:06.273251+0000","last_became_active":"2024-09-19T08:39:04.275978+0000","last_became_peered":"2024-09-19T08:39:04.275978+0000","last_unstale":"2024-09-19T08:39:06.273251+0000","last_undegraded":"2024-09-19T08:39:06.273251+0000","last_fullsized":"2024-09-19T08:39:06.273251+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:10.967498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.453581+0000","last_change":"2024-09-19T08:39:05.283407+0000","last_active":"2024-09-19T08:39:06.453581+0000","last_peered":"2024-09-19T08:39:06.453581+0000","last_clean":"2024-09-19T08:39:06.453581+0000","last_became_active":"2024-09-19T08:39:05.283153+0000","last_became_peered":"2024-09-19T08:39:05.283153+0000","last_unstale":"2024-09-19T08:39:06.453581+0000","last_undegraded":"2024-09-19T08:39:06.453581+0000","last_fullsized":"2024-09-19T08:39:06.453581+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:02.155985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.278405+0000","last_change":"2024-09-19T08:39:04.283095+0000","last_active":"2024-09-19T08:39:06.278405+0000","last_peered":"2024-09-19T08:39:06.278405+0000","last_clean":"2024-09-19T08:39:06.278405+0000","last_became_active":"2024-09-19T08:39:04.282373+0000","last_became_peered":"2024-09-19T08:39:04.282373+0000","last_unstale":"2024-09-19T08:39:06.278405+0000","last_undegraded":"2024-09-19T08:39:06.278405+0000","last_fullsized":"2024-09-19T08:39:06.278405+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:54:09.641358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.460764+0000","last_change":"2024-09-19T08:39:05.298283+0000","last_active":"2024-09-19T08:39:06.460764+0000","last_peered":"2024-09-19T08:39:06.460764+0000","last_clean":"2024-09-19T08:39:06.460764+0000","last_became_active":"2024-09-19T08:39:05.297539+0000","last_became_peered":"2024-09-19T08:39:05.297539+0000","last_unstale":"2024-09-19T08:39:06.460764+0000","last_undegraded":"2024-09-19T08:39:06.460764+0000","last_fullsized":"2024-09-19T08:39:06.460764+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:33:09.963564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.273975+0000","last_change":"2024-09-19T08:39:04.288851+0000","last_active":"2024-09-19T08:39:06.273975+0000","last_peered":"2024-09-19T08:39:06.273975+0000","last_clean":"2024-09-19T08:39:06.273975+0000","last_became_active":"2024-09-19T08:39:04.288321+0000","last_became_peered":"2024-09-19T08:39:04.288321+0000","last_unstale":"2024-09-19T08:39:06.273975+0000","last_undegraded":"2024-09-19T08:39:06.273975+0000","last_fullsized":"2024-09-19T08:39:06.273975+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:23:06.984397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.273858+0000","last_change":"2024-09-19T08:39:05.280435+0000","last_active":"2024-09-19T08:39:06.273858+0000","last_peered":"2024-09-19T08:39:06.273858+0000","last_clean":"2024-09-19T08:39:06.273858+0000","last_became_active":"2024-09-19T08:39:05.280069+0000","last_became_peered":"2024-09-19T08:39:05.280069+0000","last_unstale":"2024-09-19T08:39:06.273858+0000","last_undegraded":"2024-09-19T08:39:06.273858+0000","last_fullsized":"2024-09-19T08:39:06.273858+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:30.977226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.455087+0000","last_change":"2024-09-19T08:39:04.269552+0000","last_active":"2024-09-19T08:39:06.455087+0000","last_peered":"2024-09-19T08:39:06.455087+0000","last_clean":"2024-09-19T08:39:06.455087+0000","last_became_active":"2024-09-19T08:39:04.269236+0000","last_became_peered":"2024-09-19T08:39:04.269236+0000","last_unstale":"2024-09-19T08:39:06.455087+0000","last_undegraded":"2024-09-19T08:39:06.455087+0000","last_fullsized":"2024-09-19T08:39:06.455087+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:21:25.044458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.273827+0000","last_change":"2024-09-19T08:39:05.289473+0000","last_active":"2024-09-19T08:39:06.273827+0000","last_peered":"2024-09-19T08:39:06.273827+0000","last_clean":"2024-09-19T08:39:06.273827+0000","last_became_active":"2024-09-19T08:39:05.289032+0000","last_became_peered":"2024-09-19T08:39:05.289032+0000","last_unstale":"2024-09-19T08:39:06.273827+0000","last_undegraded":"2024-09-19T08:39:06.273827+0000","last_fullsized":"2024-09-19T08:39:06.273827+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:17:56.604632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.461269+0000","last_change":"2024-09-19T08:39:04.300019+0000","last_active":"2024-09-19T08:39:06.461269+0000","last_peered":"2024-09-19T08:39:06.461269+0000","last_clean":"2024-09-19T08:39:06.461269+0000","last_became_active":"2024-09-19T08:39:04.299285+0000","last_became_peered":"2024-09-19T08:39:04.299285+0000","last_unstale":"2024-09-19T08:39:06.461269+0000","last_undegraded":"2024-09-19T08:39:06.461269+0000","last_fullsized":"2024-09-19T08:39:06.461269+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:42:41.712469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.460995+0000","last_change":"2024-09-19T08:39:05.290869+0000","last_active":"2024-09-19T08:39:06.460995+0000","last_peered":"2024-09-19T08:39:06.460995+0000","last_clean":"2024-09-19T08:39:06.460995+0000","last_became_active":"2024-09-19T08:39:05.290154+0000","last_became_peered":"2024-09-19T08:39:05.290154+0000","last_unstale":"2024-09-19T08:39:06.460995+0000","last_undegraded":"2024-09-19T08:39:06.460995+0000","last_fullsized":"2024-09-19T08:39:06.460995+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:47.034506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"31'6","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.273942+0000","last_change":"2024-09-19T08:39:04.268499+0000","last_active":"2024-09-19T08:39:06.273942+0000","last_peered":"2024-09-19T08:39:06.273942+0000","last_clean":"2024-09-19T08:39:06.273942+0000","last_became_active":"2024-09-19T08:39:04.268117+0000","last_became_peered":"2024-09-19T08:39:04.268117+0000","last_unstale":"2024-09-19T08:39:06.273942+0000","last_undegraded":"2024-09-19T08:39:06.273942+0000","last_fullsized":"2024-09-19T08:39:06.273942+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:55:13.176419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.453989+0000","last_change":"2024-09-19T08:39:05.283768+0000","last_active":"2024-09-19T08:39:06.453989+0000","last_peered":"2024-09-19T08:39:06.453989+0000","last_clean":"2024-09-19T08:39:06.453989+0000","last_became_active":"2024-09-19T08:39:05.283267+0000","last_became_peered":"2024-09-19T08:39:05.283267+0000","last_unstale":"2024-09-19T08:39:06.453989+0000","last_undegraded":"2024-09-19T08:39:06.453989+0000","last_fullsized":"2024-09-19T08:39:06.453989+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:07:40.948944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.279739+0000","last_change":"2024-09-19T08:39:04.287554+0000","last_active":"2024-09-19T08:39:06.279739+0000","last_peered":"2024-09-19T08:39:06.279739+0000","last_clean":"2024-09-19T08:39:06.279739+0000","last_became_active":"2024-09-19T08:39:04.286788+0000","last_became_peered":"2024-09-19T08:39:04.286788+0000","last_unstale":"2024-09-19T08:39:06.279739+0000","last_undegraded":"2024-09-19T08:39:06.279739+0000","last_fullsized":"2024-09-19T08:39:06.279739+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:21.013066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.273689+0000","last_change":"2024-09-19T08:39:05.269143+0000","last_active":"2024-09-19T08:39:06.273689+0000","last_peered":"2024-09-19T08:39:06.273689+0000","last_clean":"2024-09-19T08:39:06.273689+0000","last_became_active":"2024-09-19T08:39:05.268864+0000","last_became_peered":"2024-09-19T08:39:05.268864+0000","last_unstale":"2024-09-19T08:39:06.273689+0000","last_undegraded":"2024-09-19T08:39:06.273689+0000","last_fullsized":"2024-09-19T08:39:06.273689+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:16.269711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.454078+0000","last_change":"2024-09-19T08:39:04.281904+0000","last_active":"2024-09-19T08:39:06.454078+0000","last_peered":"2024-09-19T08:39:06.454078+0000","last_clean":"2024-09-19T08:39:06.454078+0000","last_became_active":"2024-09-19T08:39:04.281575+0000","last_became_peered":"2024-09-19T08:39:04.281575+0000","last_unstale":"2024-09-19T08:39:06.454078+0000","last_undegraded":"2024-09-19T08:39:06.454078+0000","last_fullsized":"2024-09-19T08:39:06.454078+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:21.013066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.460790+0000","last_change":"2024-09-19T08:39:05.311174+0000","last_active":"2024-09-19T08:39:06.460790+0000","last_peered":"2024-09-19T08:39:06.460790+0000","last_clean":"2024-09-19T08:39:06.460790+0000","last_became_active":"2024-09-19T08:39:05.310638+0000","last_became_peered":"2024-09-19T08:39:05.310638+0000","last_unstale":"2024-09-19T08:39:06.460790+0000","last_undegraded":"2024-09-19T08:39:06.460790+0000","last_fullsized":"2024-09-19T08:39:06.460790+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:22:55.001406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.273365+0000","last_change":"2024-09-19T08:39:04.288373+0000","last_active":"2024-09-19T08:39:06.273365+0000","last_peered":"2024-09-19T08:39:06.273365+0000","last_clean":"2024-09-19T08:39:06.273365+0000","last_became_active":"2024-09-19T08:39:04.288043+0000","last_became_peered":"2024-09-19T08:39:04.288043+0000","last_unstale":"2024-09-19T08:39:06.273365+0000","last_undegraded":"2024-09-19T08:39:06.273365+0000","last_fullsized":"2024-09-19T08:39:06.273365+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:41:52.582342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.459475+0000","last_change":"2024-09-19T08:39:05.305633+0000","last_active":"2024-09-19T08:39:06.459475+0000","last_peered":"2024-09-19T08:39:06.459475+0000","last_clean":"2024-09-19T08:39:06.459475+0000","last_became_active":"2024-09-19T08:39:05.304913+0000","last_became_peered":"2024-09-19T08:39:05.304913+0000","last_unstale":"2024-09-19T08:39:06.459475+0000","last_undegraded":"2024-09-19T08:39:06.459475+0000","last_fullsized":"2024-09-19T08:39:06.459475+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:41:53.594289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.273473+0000","last_change":"2024-09-19T08:39:04.275012+0000","last_active":"2024-09-19T08:39:06.273473+0000","last_peered":"2024-09-19T08:39:06.273473+0000","last_clean":"2024-09-19T08:39:06.273473+0000","last_became_active":"2024-09-19T08:39:04.268630+0000","last_became_peered":"2024-09-19T08:39:04.268630+0000","last_unstale":"2024-09-19T08:39:06.273473+0000","last_undegraded":"2024-09-19T08:39:06.273473+0000","last_fullsized":"2024-09-19T08:39:06.273473+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:08:08.026458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.281265+0000","last_change":"2024-09-19T08:39:05.287665+0000","last_active":"2024-09-19T08:39:06.281265+0000","last_peered":"2024-09-19T08:39:06.281265+0000","last_clean":"2024-09-19T08:39:06.281265+0000","last_became_active":"2024-09-19T08:39:05.286967+0000","last_became_peered":"2024-09-19T08:39:05.286967+0000","last_unstale":"2024-09-19T08:39:06.281265+0000","last_undegraded":"2024-09-19T08:39:06.281265+0000","last_fullsized":"2024-09-19T08:39:06.281265+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:07:40.948944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.455888+0000","last_change":"2024-09-19T08:39:04.267304+0000","last_active":"2024-09-19T08:39:06.455888+0000","last_peered":"2024-09-19T08:39:06.455888+0000","last_clean":"2024-09-19T08:39:06.455888+0000","last_became_active":"2024-09-19T08:39:04.266821+0000","last_became_peered":"2024-09-19T08:39:04.266821+0000","last_unstale":"2024-09-19T08:39:06.455888+0000","last_undegraded":"2024-09-19T08:39:06.455888+0000","last_fullsized":"2024-09-19T08:39:06.455888+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:24:14.301248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.461215+0000","last_change":"2024-09-19T08:39:05.315944+0000","last_active":"2024-09-19T08:39:06.461215+0000","last_peered":"2024-09-19T08:39:06.461215+0000","last_clean":"2024-09-19T08:39:06.461215+0000","last_became_active":"2024-09-19T08:39:05.315296+0000","last_became_peered":"2024-09-19T08:39:05.315296+0000","last_unstale":"2024-09-19T08:39:06.461215+0000","last_undegraded":"2024-09-19T08:39:06.461215+0000","last_fullsized":"2024-09-19T08:39:06.461215+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:48:16.380386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.280460+0000","last_change":"2024-09-19T08:39:04.289318+0000","last_active":"2024-09-19T08:39:06.280460+0000","last_peered":"2024-09-19T08:39:06.280460+0000","last_clean":"2024-09-19T08:39:06.280460+0000","last_became_active":"2024-09-19T08:39:04.288617+0000","last_became_peered":"2024-09-19T08:39:04.288617+0000","last_unstale":"2024-09-19T08:39:06.280460+0000","last_undegraded":"2024-09-19T08:39:06.280460+0000","last_fullsized":"2024-09-19T08:39:06.280460+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:11:50.689661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.453807+0000","last_change":"2024-09-19T08:39:05.294121+0000","last_active":"2024-09-19T08:39:06.453807+0000","last_peered":"2024-09-19T08:39:06.453807+0000","last_clean":"2024-09-19T08:39:06.453807+0000","last_became_active":"2024-09-19T08:39:05.293869+0000","last_became_peered":"2024-09-19T08:39:05.293869+0000","last_unstale":"2024-09-19T08:39:06.453807+0000","last_undegraded":"2024-09-19T08:39:06.453807+0000","last_fullsized":"2024-09-19T08:39:06.453807+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:21.553060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.280534+0000","last_change":"2024-09-19T08:39:04.267432+0000","last_active":"2024-09-19T08:39:06.280534+0000","last_peered":"2024-09-19T08:39:06.280534+0000","last_clean":"2024-09-19T08:39:06.280534+0000","last_became_active":"2024-09-19T08:39:04.266603+0000","last_became_peered":"2024-09-19T08:39:04.266603+0000","last_unstale":"2024-09-19T08:39:06.280534+0000","last_undegraded":"2024-09-19T08:39:06.280534+0000","last_fullsized":"2024-09-19T08:39:06.280534+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:03.926682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.454246+0000","last_change":"2024-09-19T08:39:05.302230+0000","last_active":"2024-09-19T08:39:06.454246+0000","last_peered":"2024-09-19T08:39:06.454246+0000","last_clean":"2024-09-19T08:39:06.454246+0000","last_became_active":"2024-09-19T08:39:05.301944+0000","last_became_peered":"2024-09-19T08:39:05.301944+0000","last_unstale":"2024-09-19T08:39:06.454246+0000","last_undegraded":"2024-09-19T08:39:06.454246+0000","last_fullsized":"2024-09-19T08:39:06.454246+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:04.938629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.280134+0000","last_change":"2024-09-19T08:39:04.279661+0000","last_active":"2024-09-19T08:39:06.280134+0000","last_peered":"2024-09-19T08:39:06.280134+0000","last_clean":"2024-09-19T08:39:06.280134+0000","last_became_active":"2024-09-19T08:39:04.278891+0000","last_became_peered":"2024-09-19T08:39:04.278891+0000","last_unstale":"2024-09-19T08:39:06.280134+0000","last_undegraded":"2024-09-19T08:39:06.280134+0000","last_fullsized":"2024-09-19T08:39:06.280134+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:00.768392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.454428+0000","last_change":"2024-09-19T08:39:05.292210+0000","last_active":"2024-09-19T08:39:06.454428+0000","last_peered":"2024-09-19T08:39:06.454428+0000","last_clean":"2024-09-19T08:39:06.454428+0000","last_became_active":"2024-09-19T08:39:05.291817+0000","last_became_peered":"2024-09-19T08:39:05.291817+0000","last_unstale":"2024-09-19T08:39:06.454428+0000","last_undegraded":"2024-09-19T08:39:06.454428+0000","last_fullsized":"2024-09-19T08:39:06.454428+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:48:52.843985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.454013+0000","last_change":"2024-09-19T08:39:04.270348+0000","last_active":"2024-09-19T08:39:06.454013+0000","last_peered":"2024-09-19T08:39:06.454013+0000","last_clean":"2024-09-19T08:39:06.454013+0000","last_became_active":"2024-09-19T08:39:04.269982+0000","last_became_peered":"2024-09-19T08:39:04.269982+0000","last_unstale":"2024-09-19T08:39:06.454013+0000","last_undegraded":"2024-09-19T08:39:06.454013+0000","last_fullsized":"2024-09-19T08:39:06.454013+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:21.013066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.454225+0000","last_change":"2024-09-19T08:39:05.269964+0000","last_active":"2024-09-19T08:39:06.454225+0000","last_peered":"2024-09-19T08:39:06.454225+0000","last_clean":"2024-09-19T08:39:06.454225+0000","last_became_active":"2024-09-19T08:39:05.269685+0000","last_became_peered":"2024-09-19T08:39:05.269685+0000","last_unstale":"2024-09-19T08:39:06.454225+0000","last_undegraded":"2024-09-19T08:39:06.454225+0000","last_fullsized":"2024-09-19T08:39:06.454225+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:01.780338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.458762+0000","last_change":"2024-09-19T08:39:04.285062+0000","last_active":"2024-09-19T08:39:06.458762+0000","last_peered":"2024-09-19T08:39:06.458762+0000","last_clean":"2024-09-19T08:39:06.458762+0000","last_became_active":"2024-09-19T08:39:04.284140+0000","last_became_peered":"2024-09-19T08:39:04.284140+0000","last_unstale":"2024-09-19T08:39:06.458762+0000","last_undegraded":"2024-09-19T08:39:06.458762+0000","last_fullsized":"2024-09-19T08:39:06.458762+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:00:00.495561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.454313+0000","last_change":"2024-09-19T08:39:05.269101+0000","last_active":"2024-09-19T08:39:06.454313+0000","last_peered":"2024-09-19T08:39:06.454313+0000","last_clean":"2024-09-19T08:39:06.454313+0000","last_became_active":"2024-09-19T08:39:05.268597+0000","last_became_peered":"2024-09-19T08:39:05.268597+0000","last_unstale":"2024-09-19T08:39:06.454313+0000","last_undegraded":"2024-09-19T08:39:06.454313+0000","last_fullsized":"2024-09-19T08:39:06.454313+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:54:10.653305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.278554+0000","last_change":"2024-09-19T08:39:04.273274+0000","last_active":"2024-09-19T08:39:06.278554+0000","last_peered":"2024-09-19T08:39:06.278554+0000","last_clean":"2024-09-19T08:39:06.278554+0000","last_became_active":"2024-09-19T08:39:04.272496+0000","last_became_peered":"2024-09-19T08:39:04.272496+0000","last_unstale":"2024-09-19T08:39:06.278554+0000","last_undegraded":"2024-09-19T08:39:06.278554+0000","last_fullsized":"2024-09-19T08:39:06.278554+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:47.383728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.459686+0000","last_change":"2024-09-19T08:39:05.304080+0000","last_active":"2024-09-19T08:39:06.459686+0000","last_peered":"2024-09-19T08:39:06.459686+0000","last_clean":"2024-09-19T08:39:06.459686+0000","last_became_active":"2024-09-19T08:39:05.303429+0000","last_became_peered":"2024-09-19T08:39:05.303429+0000","last_unstale":"2024-09-19T08:39:06.459686+0000","last_undegraded":"2024-09-19T08:39:06.459686+0000","last_fullsized":"2024-09-19T08:39:06.459686+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:34.799076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.457938+0000","last_change":"2024-09-19T08:39:04.300053+0000","last_active":"2024-09-19T08:39:06.457938+0000","last_peered":"2024-09-19T08:39:06.457938+0000","last_clean":"2024-09-19T08:39:06.457938+0000","last_became_active":"2024-09-19T08:39:04.299311+0000","last_became_peered":"2024-09-19T08:39:04.299311+0000","last_unstale":"2024-09-19T08:39:06.457938+0000","last_undegraded":"2024-09-19T08:39:06.457938+0000","last_fullsized":"2024-09-19T08:39:06.457938+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:36:25.186084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.454172+0000","last_change":"2024-09-19T08:39:05.297890+0000","last_active":"2024-09-19T08:39:06.454172+0000","last_peered":"2024-09-19T08:39:06.454172+0000","last_clean":"2024-09-19T08:39:06.454172+0000","last_became_active":"2024-09-19T08:39:05.297588+0000","last_became_peered":"2024-09-19T08:39:05.297588+0000","last_unstale":"2024-09-19T08:39:06.454172+0000","last_undegraded":"2024-09-19T08:39:06.454172+0000","last_fullsized":"2024-09-19T08:39:06.454172+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:11:51.701607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.455934+0000","last_change":"2024-09-19T08:39:04.289505+0000","last_active":"2024-09-19T08:39:06.455934+0000","last_peered":"2024-09-19T08:39:06.455934+0000","last_clean":"2024-09-19T08:39:06.455934+0000","last_became_active":"2024-09-19T08:39:04.289087+0000","last_became_peered":"2024-09-19T08:39:04.289087+0000","last_unstale":"2024-09-19T08:39:06.455934+0000","last_undegraded":"2024-09-19T08:39:06.455934+0000","last_fullsized":"2024-09-19T08:39:06.455934+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:21:25.044458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.461605+0000","last_change":"2024-09-19T08:39:05.287054+0000","last_active":"2024-09-19T08:39:06.461605+0000","last_peered":"2024-09-19T08:39:06.461605+0000","last_clean":"2024-09-19T08:39:06.461605+0000","last_became_active":"2024-09-19T08:39:05.285751+0000","last_became_peered":"2024-09-19T08:39:05.285751+0000","last_unstale":"2024-09-19T08:39:06.461605+0000","last_undegraded":"2024-09-19T08:39:06.461605+0000","last_fullsized":"2024-09-19T08:39:06.461605+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:35.641987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.455000+0000","last_change":"2024-09-19T08:39:04.270773+0000","last_active":"2024-09-19T08:39:06.455000+0000","last_peered":"2024-09-19T08:39:06.455000+0000","last_clean":"2024-09-19T08:39:06.455000+0000","last_became_active":"2024-09-19T08:39:04.270101+0000","last_became_peered":"2024-09-19T08:39:04.270101+0000","last_unstale":"2024-09-19T08:39:06.455000+0000","last_undegraded":"2024-09-19T08:39:06.455000+0000","last_fullsized":"2024-09-19T08:39:06.455000+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:28:53.942271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.453331+0000","last_change":"2024-09-19T08:39:05.263130+0000","last_active":"2024-09-19T08:39:06.453331+0000","last_peered":"2024-09-19T08:39:06.453331+0000","last_clean":"2024-09-19T08:39:06.453331+0000","last_became_active":"2024-09-19T08:39:05.262841+0000","last_became_peered":"2024-09-19T08:39:05.262841+0000","last_unstale":"2024-09-19T08:39:06.453331+0000","last_undegraded":"2024-09-19T08:39:06.453331+0000","last_fullsized":"2024-09-19T08:39:06.453331+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:36:26.198031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.455451+0000","last_change":"2024-09-19T08:39:04.269602+0000","last_active":"2024-09-19T08:39:06.455451+0000","last_peered":"2024-09-19T08:39:06.455451+0000","last_clean":"2024-09-19T08:39:06.455451+0000","last_became_active":"2024-09-19T08:39:04.269258+0000","last_became_peered":"2024-09-19T08:39:04.269258+0000","last_unstale":"2024-09-19T08:39:06.455451+0000","last_undegraded":"2024-09-19T08:39:06.455451+0000","last_fullsized":"2024-09-19T08:39:06.455451+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:28:53.942271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.458915+0000","last_change":"2024-09-19T08:39:05.276652+0000","last_active":"2024-09-19T08:39:06.458915+0000","last_peered":"2024-09-19T08:39:06.458915+0000","last_clean":"2024-09-19T08:39:06.458915+0000","last_became_active":"2024-09-19T08:39:05.275913+0000","last_became_peered":"2024-09-19T08:39:05.275913+0000","last_unstale":"2024-09-19T08:39:06.458915+0000","last_undegraded":"2024-09-19T08:39:06.458915+0000","last_fullsized":"2024-09-19T08:39:06.458915+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:36.037852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.454335+0000","last_change":"2024-09-19T08:39:04.288965+0000","last_active":"2024-09-19T08:39:06.454335+0000","last_peered":"2024-09-19T08:39:06.454335+0000","last_clean":"2024-09-19T08:39:06.454335+0000","last_became_active":"2024-09-19T08:39:04.288652+0000","last_became_peered":"2024-09-19T08:39:04.288652+0000","last_unstale":"2024-09-19T08:39:06.454335+0000","last_undegraded":"2024-09-19T08:39:06.454335+0000","last_fullsized":"2024-09-19T08:39:06.454335+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:24:14.301248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.453917+0000","last_change":"2024-09-19T08:39:05.297045+0000","last_active":"2024-09-19T08:39:06.453917+0000","last_peered":"2024-09-19T08:39:06.453917+0000","last_clean":"2024-09-19T08:39:06.453917+0000","last_became_active":"2024-09-19T08:39:05.296623+0000","last_became_peered":"2024-09-19T08:39:05.296623+0000","last_unstale":"2024-09-19T08:39:06.453917+0000","last_undegraded":"2024-09-19T08:39:06.453917+0000","last_fullsized":"2024-09-19T08:39:06.453917+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:54.222762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.454248+0000","last_change":"2024-09-19T08:39:04.286748+0000","last_active":"2024-09-19T08:39:06.454248+0000","last_peered":"2024-09-19T08:39:06.454248+0000","last_clean":"2024-09-19T08:39:06.454248+0000","last_became_active":"2024-09-19T08:39:04.286504+0000","last_became_peered":"2024-09-19T08:39:04.286504+0000","last_unstale":"2024-09-19T08:39:06.454248+0000","last_undegraded":"2024-09-19T08:39:06.454248+0000","last_fullsized":"2024-09-19T08:39:06.454248+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:07:39.437649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.273253+0000","last_change":"2024-09-19T08:39:05.271856+0000","last_active":"2024-09-19T08:39:06.273253+0000","last_peered":"2024-09-19T08:39:06.273253+0000","last_clean":"2024-09-19T08:39:06.273253+0000","last_became_active":"2024-09-19T08:39:05.271554+0000","last_became_peered":"2024-09-19T08:39:05.271554+0000","last_unstale":"2024-09-19T08:39:06.273253+0000","last_undegraded":"2024-09-19T08:39:06.273253+0000","last_fullsized":"2024-09-19T08:39:06.273253+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:46:17.548500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.278981+0000","last_change":"2024-09-19T08:39:04.295108+0000","last_active":"2024-09-19T08:39:06.278981+0000","last_peered":"2024-09-19T08:39:06.278981+0000","last_clean":"2024-09-19T08:39:06.278981+0000","last_became_active":"2024-09-19T08:39:04.294195+0000","last_became_peered":"2024-09-19T08:39:04.294195+0000","last_unstale":"2024-09-19T08:39:06.278981+0000","last_undegraded":"2024-09-19T08:39:06.278981+0000","last_fullsized":"2024-09-19T08:39:06.278981+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:16:29.673088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.278724+0000","last_change":"2024-09-19T08:39:05.273370+0000","last_active":"2024-09-19T08:39:06.278724+0000","last_peered":"2024-09-19T08:39:06.278724+0000","last_clean":"2024-09-19T08:39:06.278724+0000","last_became_active":"2024-09-19T08:39:05.272592+0000","last_became_peered":"2024-09-19T08:39:05.272592+0000","last_unstale":"2024-09-19T08:39:06.278724+0000","last_undegraded":"2024-09-19T08:39:06.278724+0000","last_fullsized":"2024-09-19T08:39:06.278724+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:16.522784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.273400+0000","last_change":"2024-09-19T08:39:04.295400+0000","last_active":"2024-09-19T08:39:06.273400+0000","last_peered":"2024-09-19T08:39:06.273400+0000","last_clean":"2024-09-19T08:39:06.273400+0000","last_became_active":"2024-09-19T08:39:04.295078+0000","last_became_peered":"2024-09-19T08:39:04.295078+0000","last_unstale":"2024-09-19T08:39:06.273400+0000","last_undegraded":"2024-09-19T08:39:06.273400+0000","last_fullsized":"2024-09-19T08:39:06.273400+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:57.757107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.460507+0000","last_change":"2024-09-19T08:39:05.294664+0000","last_active":"2024-09-19T08:39:06.460507+0000","last_peered":"2024-09-19T08:39:06.460507+0000","last_clean":"2024-09-19T08:39:06.460507+0000","last_became_active":"2024-09-19T08:39:05.293664+0000","last_became_peered":"2024-09-19T08:39:05.293664+0000","last_unstale":"2024-09-19T08:39:06.460507+0000","last_undegraded":"2024-09-19T08:39:06.460507+0000","last_fullsized":"2024-09-19T08:39:06.460507+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:04.236483+0000","last_clean_scrub_stamp":"2024-09-19T08:39:04.236483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:18:49.490651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.458454+0000","last_change":"2024-09-19T08:39:04.294673+0000","last_active":"2024-09-19T08:39:06.458454+0000","last_peered":"2024-09-19T08:39:06.458454+0000","last_clean":"2024-09-19T08:39:06.458454+0000","last_became_active":"2024-09-19T08:39:04.293913+0000","last_became_peered":"2024-09-19T08:39:04.293913+0000","last_unstale":"2024-09-19T08:39:06.458454+0000","last_undegraded":"2024-09-19T08:39:06.458454+0000","last_fullsized":"2024-09-19T08:39:06.458454+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:03.224537+0000","last_clean_scrub_stamp":"2024-09-19T08:39:03.224537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:12.830812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":188,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-19T08:39:06.461072+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:39:06.461072+0000","last_peered":"2024-09-19T08:39:06.461072+0000","last_clean":"2024-09-19T08:39:06.461072+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:39:06.461072+0000","last_undegraded":"2024-09-19T08:39:06.461072+0000","last_fullsized":"2024-09-19T08:39:06.461072+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509455,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":950272,"data_stored":726117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":360448,"data_stored":136331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":360448,"data_stored":135763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509454,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":356352,"data_stored":135729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509456,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":368640,"data_stored":137293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509454,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":946176,"data_stored":726027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":360448,"data_stored":136275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542159,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":368640,"data_stored":137293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:39:13.103 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T08:39:13.104 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-19T08:39:13.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.255+0000 7f45f504d640 1 Processor -- start 2024-09-19T08:39:13.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.255+0000 7f45f504d640 1 -- start start 2024-09-19T08:39:13.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.255+0000 7f45f504d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45f015c730 0x7f45f00b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:13.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.255+0000 7f45f504d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f45f00b3770 0x7f45f00cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:13.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.255+0000 7f45f504d640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45f00b08c0 con 0x7f45f006caf0 2024-09-19T08:39:13.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.255+0000 7f45f504d640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f45f00b0ba0 con 0x7f45f00b3770 2024-09-19T08:39:13.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.255+0000 7f45f504d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f45f00b0e80 con 0x7f45f015c730 2024-09-19T08:39:13.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.255+0000 7f45eed9d640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f45f006caf0 0x7f45f006ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:45574/0 (socket says 172.21.15.98:45574) 2024-09-19T08:39:13.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.255+0000 7f45eed9d640 1 -- 172.21.15.98:0/12488927 learned_addr learned my addr 172.21.15.98:0/12488927 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:13.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.255+0000 7f45ef59e640 1 --2- 172.21.15.98:0/12488927 >> v2:172.21.15.98:3300/0 conn(0x7f45f00b3770 0x7f45f00cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:13.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.255+0000 7f45edd9b640 1 -- 172.21.15.98:0/12488927 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2226581080 0 0) 0x7f45f00b08c0 con 0x7f45f006caf0 2024-09-19T08:39:13.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.255+0000 7f45edd9b640 1 -- 172.21.15.98:0/12488927 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45d8003530 con 0x7f45f006caf0 2024-09-19T08:39:13.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.255+0000 7f45ee59c640 1 --2- 172.21.15.98:0/12488927 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45f015c730 0x7f45f00b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:13.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.255+0000 7f45ef59e640 1 -- 172.21.15.98:0/12488927 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45f015c730 msgr2=0x7f45f00b02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:13.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.255+0000 7f45ef59e640 1 --2- 172.21.15.98:0/12488927 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45f015c730 0x7f45f00b02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:13.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.255+0000 7f45ef59e640 1 -- 172.21.15.98:0/12488927 >> v1:172.21.15.98:6789/0 conn(0x7f45f006caf0 legacy=0x7f45f006ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:13.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.255+0000 7f45ef59e640 1 -- 172.21.15.98:0/12488927 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45f00b1160 con 0x7f45f00b3770 2024-09-19T08:39:13.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.255+0000 7f45ef59e640 1 --2- 172.21.15.98:0/12488927 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45f00b3770 0x7f45f00cbc00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f45e8009d90 tx=0x7f45e801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33811b6ff45cf681 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:13.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.259+0000 7f45edd9b640 1 -- 172.21.15.98:0/12488927 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e80b2070 con 0x7f45f00b3770 2024-09-19T08:39:13.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.259+0000 7f45edd9b640 1 -- 172.21.15.98:0/12488927 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45e80a5a20 con 0x7f45f00b3770 2024-09-19T08:39:13.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.259+0000 7f45edd9b640 1 -- 172.21.15.98:0/12488927 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e80a5dd0 con 0x7f45f00b3770 2024-09-19T08:39:13.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.259+0000 7f45f504d640 1 -- 172.21.15.98:0/12488927 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45f00b3770 msgr2=0x7f45f00cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:13.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.259+0000 7f45f504d640 1 --2- 172.21.15.98:0/12488927 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45f00b3770 0x7f45f00cbc00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f45e8009d90 tx=0x7f45e801c940 comp rx=0 tx=0).stop 2024-09-19T08:39:13.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.259+0000 7f45f504d640 1 -- 172.21.15.98:0/12488927 shutdown_connections 2024-09-19T08:39:13.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.259+0000 7f45f504d640 1 --2- 172.21.15.98:0/12488927 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45f00b3770 0x7f45f00cbc00 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:13.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.259+0000 7f45f504d640 1 --2- 172.21.15.98:0/12488927 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45f015c730 0x7f45f00b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:13.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.259+0000 7f45f504d640 1 -- 172.21.15.98:0/12488927 >> 172.21.15.98:0/12488927 conn(0x7f45f00b5f50 msgr2=0x7f45f00b6650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:13.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.259+0000 7f45f504d640 1 -- 172.21.15.98:0/12488927 shutdown_connections 2024-09-19T08:39:13.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.259+0000 7f45f504d640 1 -- 172.21.15.98:0/12488927 wait complete. 2024-09-19T08:39:13.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.259+0000 7f45f504d640 1 Processor -- start 2024-09-19T08:39:13.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.259+0000 7f45f504d640 1 -- start start 2024-09-19T08:39:13.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.259+0000 7f45f504d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45f006caf0 0x7f45f014bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:13.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.259+0000 7f45f504d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45f00b3770 0x7f45f0154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:13.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.259+0000 7f45f504d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45f015c730 0x7f45f014b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:13.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.259+0000 7f45f504d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f45f0169120 con 0x7f45f00b3770 2024-09-19T08:39:13.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.259+0000 7f45f504d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f45f00cc350 con 0x7f45f015c730 2024-09-19T08:39:13.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.259+0000 7f45f504d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f45f01692a0 con 0x7f45f006caf0 2024-09-19T08:39:13.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.259+0000 7f45ee59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45f00b3770 0x7f45f0154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:13.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.259+0000 7f45ee59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45f00b3770 0x7f45f0154240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36474/0 (socket says 172.21.15.98:36474) 2024-09-19T08:39:13.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.259+0000 7f45ee59c640 1 -- 172.21.15.98:0/281815867 learned_addr learned my addr 172.21.15.98:0/281815867 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:13.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.259+0000 7f45eed9d640 1 --2- 172.21.15.98:0/281815867 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45f006caf0 0x7f45f014bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:13.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.259+0000 7f45ef59e640 1 --2- 172.21.15.98:0/281815867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45f015c730 0x7f45f014b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:13.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.259+0000 7f45ee59c640 1 -- 172.21.15.98:0/281815867 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45f006caf0 msgr2=0x7f45f014bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:13.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.259+0000 7f45ee59c640 1 --2- 172.21.15.98:0/281815867 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45f006caf0 0x7f45f014bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:13.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.259+0000 7f45ee59c640 1 -- 172.21.15.98:0/281815867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45f015c730 msgr2=0x7f45f014b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:13.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.259+0000 7f45ee59c640 1 --2- 172.21.15.98:0/281815867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45f015c730 0x7f45f014b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:13.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.259+0000 7f45ee59c640 1 -- 172.21.15.98:0/281815867 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45f006eae0 con 0x7f45f00b3770 2024-09-19T08:39:13.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.259+0000 7f45ee59c640 1 --2- 172.21.15.98:0/281815867 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45f00b3770 0x7f45f0154240 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f45e400d800 tx=0x7f45e400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:13.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.259+0000 7f45d77fe640 1 -- 172.21.15.98:0/281815867 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e400b6c0 con 0x7f45f00b3770 2024-09-19T08:39:13.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.259+0000 7f45d77fe640 1 -- 172.21.15.98:0/281815867 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45e4019070 con 0x7f45f00b3770 2024-09-19T08:39:13.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.259+0000 7f45f504d640 1 -- 172.21.15.98:0/281815867 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45f00cf320 con 0x7f45f00b3770 2024-09-19T08:39:13.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.259+0000 7f45d77fe640 1 -- 172.21.15.98:0/281815867 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e400ba60 con 0x7f45f00b3770 2024-09-19T08:39:13.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.259+0000 7f45f504d640 1 -- 172.21.15.98:0/281815867 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45f00cf8e0 con 0x7f45f00b3770 2024-09-19T08:39:13.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.263+0000 7f45f504d640 1 -- 172.21.15.98:0/281815867 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45f00cbc00 con 0x7f45f00b3770 2024-09-19T08:39:13.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.263+0000 7f45d77fe640 1 -- 172.21.15.98:0/281815867 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f45e400bc00 con 0x7f45f00b3770 2024-09-19T08:39:13.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.263+0000 7f45d77fe640 1 --2- 172.21.15.98:0/281815867 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f45dc0678c0 0x7f45dc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:13.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.263+0000 7f45d77fe640 1 -- 172.21.15.98:0/281815867 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f45e4086070 con 0x7f45f00b3770 2024-09-19T08:39:13.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.263+0000 7f45eed9d640 1 --2- 172.21.15.98:0/281815867 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f45dc0678c0 0x7f45dc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:13.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.263+0000 7f45eed9d640 1 --2- 172.21.15.98:0/281815867 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f45dc0678c0 0x7f45dc069d80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f45e0007fd0 tx=0x7f45e0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:13.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.267+0000 7f45d77fe640 1 -- 172.21.15.98:0/281815867 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f45e4056c40 con 0x7f45f00b3770 2024-09-19T08:39:13.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.479+0000 7f45f504d640 1 -- 172.21.15.98:0/281815867 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f45f015a190 con 0x7f45f00b3770 2024-09-19T08:39:13.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.483+0000 7f45d77fe640 1 -- 172.21.15.98:0/281815867 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f45e4056540 con 0x7f45f00b3770 2024-09-19T08:39:13.488 INFO:teuthology.orchestra.run.smithi098.stdout:[client.0] 2024-09-19T08:39:13.488 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQC/4utm0vNpIhAA4S1E8dFZ0+ibmSKhmZ+bIA== 2024-09-19T08:39:13.488 INFO:teuthology.orchestra.run.smithi098.stdout: caps mds = "allow" 2024-09-19T08:39:13.488 INFO:teuthology.orchestra.run.smithi098.stdout: caps mon = "allow r" 2024-09-19T08:39:13.488 INFO:teuthology.orchestra.run.smithi098.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-19T08:39:13.488 INFO:teuthology.orchestra.run.smithi098.stderr:updated caps for client.0 2024-09-19T08:39:13.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.487+0000 7f45f504d640 1 -- 172.21.15.98:0/281815867 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f45dc0678c0 msgr2=0x7f45dc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:13.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.487+0000 7f45f504d640 1 --2- 172.21.15.98:0/281815867 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f45dc0678c0 0x7f45dc069d80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f45e0007fd0 tx=0x7f45e0009040 comp rx=0 tx=0).stop 2024-09-19T08:39:13.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.487+0000 7f45f504d640 1 -- 172.21.15.98:0/281815867 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45f00b3770 msgr2=0x7f45f0154240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:13.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.487+0000 7f45f504d640 1 --2- 172.21.15.98:0/281815867 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45f00b3770 0x7f45f0154240 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f45e400d800 tx=0x7f45e400dcc0 comp rx=0 tx=0).stop 2024-09-19T08:39:13.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.487+0000 7f45f504d640 1 -- 172.21.15.98:0/281815867 shutdown_connections 2024-09-19T08:39:13.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.487+0000 7f45f504d640 1 --2- 172.21.15.98:0/281815867 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f45dc0678c0 0x7f45dc069d80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:13.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.487+0000 7f45f504d640 1 --2- 172.21.15.98:0/281815867 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45f015c730 0x7f45f014b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:13.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.487+0000 7f45f504d640 1 --2- 172.21.15.98:0/281815867 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45f00b3770 0x7f45f0154240 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:13.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.487+0000 7f45f504d640 1 --2- 172.21.15.98:0/281815867 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45f006caf0 0x7f45f014bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:13.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.487+0000 7f45f504d640 1 -- 172.21.15.98:0/281815867 >> 172.21.15.98:0/281815867 conn(0x7f45f00b5f50 msgr2=0x7f45f006df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:13.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.487+0000 7f45f504d640 1 -- 172.21.15.98:0/281815867 shutdown_connections 2024-09-19T08:39:13.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.487+0000 7f45f504d640 1 -- 172.21.15.98:0/281815867 wait complete. 2024-09-19T08:39:13.505 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:39:13.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.607+0000 7fc8180ea640 1 Processor -- start 2024-09-19T08:39:13.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.607+0000 7fc8180ea640 1 -- start start 2024-09-19T08:39:13.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.607+0000 7fc8180ea640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc81015c550 0x7fc81006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:13.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.607+0000 7fc8180ea640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc81006c1d0 0x7fc8100cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:13.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.607+0000 7fc8180ea640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc81006d310 con 0x7fc8100cf900 2024-09-19T08:39:13.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.607+0000 7fc8180ea640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fc81006c610 con 0x7fc81006c1d0 2024-09-19T08:39:13.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.607+0000 7fc8180ea640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc81006d5f0 con 0x7fc81015c550 2024-09-19T08:39:13.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.607+0000 7fc815685640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc81006c1d0 0x7fc8100cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:13.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.607+0000 7fc815685640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc81006c1d0 0x7fc8100cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36490/0 (socket says 172.21.15.98:36490) 2024-09-19T08:39:13.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.607+0000 7fc815685640 1 -- 172.21.15.98:0/1490524143 learned_addr learned my addr 172.21.15.98:0/1490524143 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:13.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.607+0000 7fc816687640 1 --1- 172.21.15.98:0/1490524143 >> v1:172.21.15.98:6789/0 conn(0x7fc8100cf900 0x7fc8100ce1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:45576/0 (socket says 172.21.15.98:45576) 2024-09-19T08:39:13.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.607+0000 7fc815e86640 1 --2- 172.21.15.98:0/1490524143 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc81015c550 0x7fc81006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:13.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.607+0000 7fc815685640 1 -- 172.21.15.98:0/1490524143 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc81015c550 msgr2=0x7fc81006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:13.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.607+0000 7fc815685640 1 --2- 172.21.15.98:0/1490524143 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc81015c550 0x7fc81006bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:13.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.607+0000 7fc815685640 1 -- 172.21.15.98:0/1490524143 >> v1:172.21.15.98:6789/0 conn(0x7fc8100cf900 legacy=0x7fc8100ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:13.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.607+0000 7fc815685640 1 -- 172.21.15.98:0/1490524143 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc81006d8d0 con 0x7fc81006c1d0 2024-09-19T08:39:13.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.611+0000 7fc815685640 1 --2- 172.21.15.98:0/1490524143 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc81006c1d0 0x7fc8100cf3c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc800009870 tx=0x7fc80001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dadd3fe67baadd1e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:13.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.611+0000 7fc814e84640 1 -- 172.21.15.98:0/1490524143 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8000b2070 con 0x7fc81006c1d0 2024-09-19T08:39:13.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.611+0000 7fc814e84640 1 -- 172.21.15.98:0/1490524143 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8000a9070 con 0x7fc81006c1d0 2024-09-19T08:39:13.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.611+0000 7fc814e84640 1 -- 172.21.15.98:0/1490524143 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8000ad880 con 0x7fc81006c1d0 2024-09-19T08:39:13.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.611+0000 7fc8180ea640 1 -- 172.21.15.98:0/1490524143 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc81006c1d0 msgr2=0x7fc8100cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:13.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.611+0000 7fc8180ea640 1 --2- 172.21.15.98:0/1490524143 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc81006c1d0 0x7fc8100cf3c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc800009870 tx=0x7fc80001c560 comp rx=0 tx=0).stop 2024-09-19T08:39:13.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.611+0000 7fc8180ea640 1 -- 172.21.15.98:0/1490524143 shutdown_connections 2024-09-19T08:39:13.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.611+0000 7fc8180ea640 1 --2- 172.21.15.98:0/1490524143 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc81006c1d0 0x7fc8100cf3c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:13.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.611+0000 7fc8180ea640 1 --2- 172.21.15.98:0/1490524143 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc81015c550 0x7fc81006bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:13.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.611+0000 7fc8180ea640 1 -- 172.21.15.98:0/1490524143 >> 172.21.15.98:0/1490524143 conn(0x7fc8100b2010 msgr2=0x7fc8100b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:13.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.611+0000 7fc8180ea640 1 -- 172.21.15.98:0/1490524143 shutdown_connections 2024-09-19T08:39:13.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.611+0000 7fc8180ea640 1 -- 172.21.15.98:0/1490524143 wait complete. 2024-09-19T08:39:13.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.611+0000 7fc8180ea640 1 Processor -- start 2024-09-19T08:39:13.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.611+0000 7fc8180ea640 1 -- start start 2024-09-19T08:39:13.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.611+0000 7fc8180ea640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc81006c1d0 0x7fc810149a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:13.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.611+0000 7fc8180ea640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8100cf900 0x7fc81014b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:13.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.611+0000 7fc8180ea640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc81015c550 0x7fc8100b4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:13.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.611+0000 7fc8180ea640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc810169230 con 0x7fc8100cf900 2024-09-19T08:39:13.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.611+0000 7fc8180ea640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc81015cb40 con 0x7fc81015c550 2024-09-19T08:39:13.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.611+0000 7fc8180ea640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc8101693b0 con 0x7fc81006c1d0 2024-09-19T08:39:13.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.611+0000 7fc815685640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8100cf900 0x7fc81014b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:13.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.611+0000 7fc815685640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8100cf900 0x7fc81014b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36504/0 (socket says 172.21.15.98:36504) 2024-09-19T08:39:13.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.611+0000 7fc815685640 1 -- 172.21.15.98:0/3069793159 learned_addr learned my addr 172.21.15.98:0/3069793159 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:13.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.611+0000 7fc815e86640 1 --2- 172.21.15.98:0/3069793159 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc81006c1d0 0x7fc810149a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:13.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.611+0000 7fc815685640 1 -- 172.21.15.98:0/3069793159 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc81006c1d0 msgr2=0x7fc810149a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:13.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.611+0000 7fc815685640 1 --2- 172.21.15.98:0/3069793159 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc81006c1d0 0x7fc810149a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:13.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.611+0000 7fc815685640 1 -- 172.21.15.98:0/3069793159 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc81015c550 msgr2=0x7fc8100b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:13.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.611+0000 7fc815685640 1 --2- 172.21.15.98:0/3069793159 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc81015c550 0x7fc8100b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:13.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.611+0000 7fc815685640 1 -- 172.21.15.98:0/3069793159 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc810149f90 con 0x7fc8100cf900 2024-09-19T08:39:13.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.611+0000 7fc815685640 1 --2- 172.21.15.98:0/3069793159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8100cf900 0x7fc81014b800 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc8000b0040 tx=0x7fc80001ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:13.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.611+0000 7fc8067fc640 1 -- 172.21.15.98:0/3069793159 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8000b2040 con 0x7fc8100cf900 2024-09-19T08:39:13.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.611+0000 7fc8067fc640 1 -- 172.21.15.98:0/3069793159 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8000be070 con 0x7fc8100cf900 2024-09-19T08:39:13.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.611+0000 7fc8067fc640 1 -- 172.21.15.98:0/3069793159 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8000b8660 con 0x7fc8100cf900 2024-09-19T08:39:13.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.611+0000 7fc8180ea640 1 -- 172.21.15.98:0/3069793159 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc81015e570 con 0x7fc8100cf900 2024-09-19T08:39:13.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.611+0000 7fc8180ea640 1 -- 172.21.15.98:0/3069793159 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8100b3940 con 0x7fc8100cf900 2024-09-19T08:39:13.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.615+0000 7fc8180ea640 1 -- 172.21.15.98:0/3069793159 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8100cf3c0 con 0x7fc8100cf900 2024-09-19T08:39:13.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.615+0000 7fc8067fc640 1 -- 172.21.15.98:0/3069793159 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc8000ab070 con 0x7fc8100cf900 2024-09-19T08:39:13.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.615+0000 7fc8067fc640 1 --2- 172.21.15.98:0/3069793159 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc7f40678c0 0x7fc7f4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:13.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.615+0000 7fc8067fc640 1 -- 172.21.15.98:0/3069793159 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fc80011fe40 con 0x7fc8100cf900 2024-09-19T08:39:13.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.615+0000 7fc815e86640 1 --2- 172.21.15.98:0/3069793159 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc7f40678c0 0x7fc7f4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:13.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.615+0000 7fc815e86640 1 --2- 172.21.15.98:0/3069793159 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc7f40678c0 0x7fc7f4069d80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc80c0096f0 tx=0x7fc80c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:13.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.619+0000 7fc8067fc640 1 -- 172.21.15.98:0/3069793159 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc800016150 con 0x7fc8100cf900 2024-09-19T08:39:13.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.811+0000 7fc8180ea640 1 -- 172.21.15.98:0/3069793159 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc8100cd300 con 0x7fc8100cf900 2024-09-19T08:39:13.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.815+0000 7fc8067fc640 1 -- 172.21.15.98:0/3069793159 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5229 (secure 0 0 0) 0x7fc8000f08f0 con 0x7fc8100cf900 2024-09-19T08:39:13.818 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:39:13.818 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":14,"btime":"2024-09-19T08:39:07:270861+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4481,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3797406671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3797406671},{"type":"v1","addr":"172.21.15.98:6835","nonce":3797406671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/4112109751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4112109751},{"type":"v1","addr":"172.21.15.139:6834","nonce":4112109751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4684,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1071477338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6833","nonce":1071477338},{"type":"v1","addr":"172.21.15.139:6835","nonce":1071477338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:39:04.632502+0000","modified":"2024-09-19T08:39:06.618098+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4833},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4833":{"gid":4833,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/1820856497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1820856497},{"type":"v1","addr":"172.21.15.98:6837","nonce":1820856497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4833,"qdb_cluster":[4833]},"id":2}]} 2024-09-19T08:39:13.818 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 14 2024-09-19T08:39:13.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.815+0000 7fc8180ea640 1 -- 172.21.15.98:0/3069793159 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc7f40678c0 msgr2=0x7fc7f4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:13.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.815+0000 7fc8180ea640 1 --2- 172.21.15.98:0/3069793159 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc7f40678c0 0x7fc7f4069d80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc80c0096f0 tx=0x7fc80c008040 comp rx=0 tx=0).stop 2024-09-19T08:39:13.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.815+0000 7fc8180ea640 1 -- 172.21.15.98:0/3069793159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8100cf900 msgr2=0x7fc81014b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:13.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.815+0000 7fc8180ea640 1 --2- 172.21.15.98:0/3069793159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8100cf900 0x7fc81014b800 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc8000b0040 tx=0x7fc80001ada0 comp rx=0 tx=0).stop 2024-09-19T08:39:13.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.815+0000 7fc8180ea640 1 -- 172.21.15.98:0/3069793159 shutdown_connections 2024-09-19T08:39:13.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.815+0000 7fc8180ea640 1 --2- 172.21.15.98:0/3069793159 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc7f40678c0 0x7fc7f4069d80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:13.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.815+0000 7fc8180ea640 1 --2- 172.21.15.98:0/3069793159 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc81015c550 0x7fc8100b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:13.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.815+0000 7fc8180ea640 1 --2- 172.21.15.98:0/3069793159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8100cf900 0x7fc81014b800 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:13.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.815+0000 7fc8180ea640 1 --2- 172.21.15.98:0/3069793159 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc81006c1d0 0x7fc810149a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:13.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.815+0000 7fc8180ea640 1 -- 172.21.15.98:0/3069793159 >> 172.21.15.98:0/3069793159 conn(0x7fc8100b2010 msgr2=0x7fc8100ce1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:13.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.815+0000 7fc8180ea640 1 -- 172.21.15.98:0/3069793159 shutdown_connections 2024-09-19T08:39:13.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:13.815+0000 7fc8180ea640 1 -- 172.21.15.98:0/3069793159 wait complete. 2024-09-19T08:39:14.239 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 14, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T08:39:04.632502+0000', 'modified': '2024-09-19T08:39:06.618098+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4833}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4833': {'gid': 4833, 'name': 'c', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.98:6837/1820856497', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6836', 'nonce': 1820856497}, {'type': 'v1', 'addr': '172.21.15.98:6837', 'nonce': 1820856497}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4833, 'qdb_cluster': [4833]} 2024-09-19T08:39:14.240 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T08:39:14.240 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-19T08:39:14.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.363+0000 7f676e047640 1 Processor -- start 2024-09-19T08:39:14.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.363+0000 7f676e047640 1 -- start start 2024-09-19T08:39:14.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.367+0000 7f676e047640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6768158fd0 0x7f67680b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:14.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.367+0000 7f676e047640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f676806c3a0 0x7f67680ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:14.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.367+0000 7f676e047640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67680ccc60 con 0x7f67680b4360 2024-09-19T08:39:14.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.367+0000 7f676e047640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f67681593b0 con 0x7f6768158fd0 2024-09-19T08:39:14.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.367+0000 7f676e047640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f67680ccf40 con 0x7f676806c3a0 2024-09-19T08:39:14.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.367+0000 7f67677fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6768158fd0 0x7f67680b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:14.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.367+0000 7f67677fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6768158fd0 0x7f67680b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36516/0 (socket says 172.21.15.98:36516) 2024-09-19T08:39:14.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.367+0000 7f67677fe640 1 -- 172.21.15.98:0/2671020662 learned_addr learned my addr 172.21.15.98:0/2671020662 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:14.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.367+0000 7f6767fff640 1 --2- 172.21.15.98:0/2671020662 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f676806c3a0 0x7f67680ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:14.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.367+0000 7f67677fe640 1 -- 172.21.15.98:0/2671020662 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f676806c3a0 msgr2=0x7f67680ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:14.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.367+0000 7f67677fe640 1 --2- 172.21.15.98:0/2671020662 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f676806c3a0 0x7f67680ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.367+0000 7f67677fe640 1 -- 172.21.15.98:0/2671020662 >> v1:172.21.15.98:6789/0 conn(0x7f67680b4360 legacy=0x7f676806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:14.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.367+0000 7f67677fe640 1 -- 172.21.15.98:0/2671020662 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67680cd220 con 0x7f6768158fd0 2024-09-19T08:39:14.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.367+0000 7f6767fff640 1 --2- 172.21.15.98:0/2671020662 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f676806c3a0 0x7f67680ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:14.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.367+0000 7f67677fe640 1 --2- 172.21.15.98:0/2671020662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6768158fd0 0x7f67680b3e20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6758009870 tx=0x7f675801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a0694bc6405dd86 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:14.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.367+0000 7f67667fc640 1 -- 172.21.15.98:0/2671020662 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67580aa600 con 0x7f6768158fd0 2024-09-19T08:39:14.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.367+0000 7f67667fc640 1 -- 172.21.15.98:0/2671020662 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67580af070 con 0x7f6768158fd0 2024-09-19T08:39:14.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.367+0000 7f67667fc640 1 -- 172.21.15.98:0/2671020662 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67580aa9c0 con 0x7f6768158fd0 2024-09-19T08:39:14.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.371+0000 7f676e047640 1 -- 172.21.15.98:0/2671020662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6768158fd0 msgr2=0x7f67680b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:14.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.371+0000 7f676e047640 1 --2- 172.21.15.98:0/2671020662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6768158fd0 0x7f67680b3e20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6758009870 tx=0x7f675801c4a0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.371+0000 7f676e047640 1 -- 172.21.15.98:0/2671020662 shutdown_connections 2024-09-19T08:39:14.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.371+0000 7f676e047640 1 --2- 172.21.15.98:0/2671020662 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f676806c3a0 0x7f67680ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.371+0000 7f676e047640 1 --2- 172.21.15.98:0/2671020662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6768158fd0 0x7f67680b3e20 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.371+0000 7f676e047640 1 -- 172.21.15.98:0/2671020662 >> 172.21.15.98:0/2671020662 conn(0x7f67680b2010 msgr2=0x7f67680b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:14.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.371+0000 7f676e047640 1 -- 172.21.15.98:0/2671020662 shutdown_connections 2024-09-19T08:39:14.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.371+0000 7f676e047640 1 -- 172.21.15.98:0/2671020662 wait complete. 2024-09-19T08:39:14.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.371+0000 7f676e047640 1 Processor -- start 2024-09-19T08:39:14.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.371+0000 7f676e047640 1 -- start start 2024-09-19T08:39:14.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.371+0000 7f676e047640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f676806c3a0 0x7f6768073b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:14.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.371+0000 7f676e047640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67680b4360 0x7f6768151e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:14.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.371+0000 7f676e047640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6768158fd0 0x7f6768071220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:14.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.371+0000 7f676e047640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f67680cf710 con 0x7f676806c3a0 2024-09-19T08:39:14.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.371+0000 7f676e047640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f67680cf590 con 0x7f67680b4360 2024-09-19T08:39:14.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.371+0000 7f676e047640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f67680cf890 con 0x7f6768158fd0 2024-09-19T08:39:14.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.371+0000 7f67677fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f676806c3a0 0x7f6768073b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:14.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.371+0000 7f67677fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f676806c3a0 0x7f6768073b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36530/0 (socket says 172.21.15.98:36530) 2024-09-19T08:39:14.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.371+0000 7f67677fe640 1 -- 172.21.15.98:0/1836653369 learned_addr learned my addr 172.21.15.98:0/1836653369 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:14.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.371+0000 7f67677fe640 1 -- 172.21.15.98:0/1836653369 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6768158fd0 msgr2=0x7f6768071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:14.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.371+0000 7f6767fff640 1 --2- 172.21.15.98:0/1836653369 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6768158fd0 0x7f6768071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:14.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.371+0000 7f6766ffd640 1 --2- 172.21.15.98:0/1836653369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67680b4360 0x7f6768151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:14.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.371+0000 7f67677fe640 1 --2- 172.21.15.98:0/1836653369 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6768158fd0 0x7f6768071220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.371+0000 7f67677fe640 1 -- 172.21.15.98:0/1836653369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67680b4360 msgr2=0x7f6768151e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:14.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.371+0000 7f67677fe640 1 --2- 172.21.15.98:0/1836653369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67680b4360 0x7f6768151e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.371+0000 7f67677fe640 1 -- 172.21.15.98:0/1836653369 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6768071dc0 con 0x7f676806c3a0 2024-09-19T08:39:14.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.371+0000 7f67677fe640 1 --2- 172.21.15.98:0/1836653369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f676806c3a0 0x7f6768073b50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f67580ac040 tx=0x7f675801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:14.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.371+0000 7f674ffff640 1 -- 172.21.15.98:0/1836653369 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67580b3800 con 0x7f676806c3a0 2024-09-19T08:39:14.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.371+0000 7f674ffff640 1 -- 172.21.15.98:0/1836653369 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67580b39c0 con 0x7f676806c3a0 2024-09-19T08:39:14.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.371+0000 7f674ffff640 1 -- 172.21.15.98:0/1836653369 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67580b3db0 con 0x7f676806c3a0 2024-09-19T08:39:14.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.371+0000 7f676e047640 1 -- 172.21.15.98:0/1836653369 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67680720a0 con 0x7f676806c3a0 2024-09-19T08:39:14.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.371+0000 7f676e047640 1 -- 172.21.15.98:0/1836653369 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6768074090 con 0x7f676806c3a0 2024-09-19T08:39:14.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.375+0000 7f674ffff640 1 -- 172.21.15.98:0/1836653369 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f67580a9070 con 0x7f676806c3a0 2024-09-19T08:39:14.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.375+0000 7f674ffff640 1 --2- 172.21.15.98:0/1836653369 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f67540678c0 0x7f6754069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:14.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.375+0000 7f674ffff640 1 -- 172.21.15.98:0/1836653369 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f67581202b0 con 0x7f676806c3a0 2024-09-19T08:39:14.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.375+0000 7f6766ffd640 1 --2- 172.21.15.98:0/1836653369 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f67540678c0 0x7f6754069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:14.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.375+0000 7f6766ffd640 1 --2- 172.21.15.98:0/1836653369 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f67540678c0 0x7f6754069d80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f675c007fd0 tx=0x7f675c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:14.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.375+0000 7f6771103000 1 Processor -- start 2024-09-19T08:39:14.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.375+0000 7f6771103000 1 -- start start 2024-09-19T08:39:14.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.375+0000 7f6771103000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8a7d17b20 0x55a8a7d17f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:14.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.375+0000 7f6771103000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8a7d18440 0x55a8a7ccdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:14.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.375+0000 7f6771103000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a8a7cce290 0x55a8a7cce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:14.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.375+0000 7f6771103000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a8a7bb5b60 con 0x55a8a7d18440 2024-09-19T08:39:14.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.375+0000 7f6771103000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a8a7b1cc70 con 0x55a8a7d17b20 2024-09-19T08:39:14.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.375+0000 7f6771103000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x55a8a7ac2ed0 con 0x55a8a7cce290 2024-09-19T08:39:14.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.375+0000 7f6766ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8a7d18440 0x55a8a7ccdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:14.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.375+0000 7f6766ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8a7d18440 0x55a8a7ccdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36536/0 (socket says 172.21.15.98:36536) 2024-09-19T08:39:14.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.375+0000 7f6766ffd640 1 -- 172.21.15.98:0/3212298093 learned_addr learned my addr 172.21.15.98:0/3212298093 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:14.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.375+0000 7f6767fff640 1 --2- 172.21.15.98:0/3212298093 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a8a7cce290 0x55a8a7cce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:14.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.375+0000 7f67677fe640 1 --2- 172.21.15.98:0/3212298093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8a7d17b20 0x55a8a7d17f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:14.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.375+0000 7f6766ffd640 1 -- 172.21.15.98:0/3212298093 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a8a7cce290 msgr2=0x55a8a7cce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:14.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.375+0000 7f6766ffd640 1 --2- 172.21.15.98:0/3212298093 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a8a7cce290 0x55a8a7cce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.375+0000 7f6766ffd640 1 -- 172.21.15.98:0/3212298093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8a7d17b20 msgr2=0x55a8a7d17f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:14.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.375+0000 7f6766ffd640 1 --2- 172.21.15.98:0/3212298093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8a7d17b20 0x55a8a7d17f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.375+0000 7f6766ffd640 1 -- 172.21.15.98:0/3212298093 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8a7ad4e50 con 0x55a8a7d18440 2024-09-19T08:39:14.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.375+0000 7f67677fe640 1 --2- 172.21.15.98:0/3212298093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8a7d17b20 0x55a8a7d17f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:14.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.375+0000 7f6766ffd640 1 --2- 172.21.15.98:0/3212298093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8a7d18440 0x55a8a7ccdd50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f675c00d850 tx=0x7f675c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142519c94eebe710 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:14.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.375+0000 7f674d7fa640 1 -- 172.21.15.98:0/3212298093 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c024070 con 0x55a8a7d18440 2024-09-19T08:39:14.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.375+0000 7f674d7fa640 1 -- 172.21.15.98:0/3212298093 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f675c014ba0 con 0x55a8a7d18440 2024-09-19T08:39:14.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f674d7fa640 1 -- 172.21.15.98:0/3212298093 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c01f400 con 0x55a8a7d18440 2024-09-19T08:39:14.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6771103000 1 -- 172.21.15.98:0/3212298093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8a7d18440 msgr2=0x55a8a7ccdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:14.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6771103000 1 --2- 172.21.15.98:0/3212298093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8a7d18440 0x55a8a7ccdd50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f675c00d850 tx=0x7f675c010000 comp rx=0 tx=0).stop 2024-09-19T08:39:14.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6771103000 1 -- 172.21.15.98:0/3212298093 shutdown_connections 2024-09-19T08:39:14.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6771103000 1 --2- 172.21.15.98:0/3212298093 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a8a7cce290 0x55a8a7cce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6771103000 1 --2- 172.21.15.98:0/3212298093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8a7d18440 0x55a8a7ccdd50 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6771103000 1 --2- 172.21.15.98:0/3212298093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8a7d17b20 0x55a8a7d17f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6771103000 1 -- 172.21.15.98:0/3212298093 >> 172.21.15.98:0/3212298093 conn(0x55a8a7cd4600 msgr2=0x55a8a7ccc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:14.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6771103000 1 -- 172.21.15.98:0/3212298093 shutdown_connections 2024-09-19T08:39:14.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6771103000 1 -- 172.21.15.98:0/3212298093 wait complete. 2024-09-19T08:39:14.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6771103000 1 -- start start 2024-09-19T08:39:14.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6771103000 1 Processor -- start 2024-09-19T08:39:14.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6771103000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a8a7d17b20 0x55a8a7cd2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:14.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6771103000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8a7d18440 0x55a8a7cd8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:14.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6771103000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8a7cd3390 0x55a8a7cd9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:14.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6771103000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a8a7b7eb00 con 0x55a8a7cd3390 2024-09-19T08:39:14.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6771103000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a8a7af7360 con 0x55a8a7d18440 2024-09-19T08:39:14.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6771103000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x55a8a7af42d0 con 0x55a8a7d17b20 2024-09-19T08:39:14.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6771103000 10 client.0 _ll_register_callbacks cb 0x55a8a7cb9820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:39:14.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6767fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8a7cd3390 0x55a8a7cd9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:14.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6767fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8a7cd3390 0x55a8a7cd9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36550/0 (socket says 172.21.15.98:36550) 2024-09-19T08:39:14.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6767fff640 1 -- 172.21.15.98:0/1031848594 learned_addr learned my addr 172.21.15.98:0/1031848594 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:14.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f67677fe640 1 --2- 172.21.15.98:0/1031848594 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a8a7d17b20 0x55a8a7cd2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:14.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6766ffd640 1 --2- 172.21.15.98:0/1031848594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8a7d18440 0x55a8a7cd8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:14.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6767fff640 1 -- 172.21.15.98:0/1031848594 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a8a7d17b20 msgr2=0x55a8a7cd2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:14.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6767fff640 1 --2- 172.21.15.98:0/1031848594 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a8a7d17b20 0x55a8a7cd2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6767fff640 1 -- 172.21.15.98:0/1031848594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8a7d18440 msgr2=0x55a8a7cd8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:14.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6767fff640 1 --2- 172.21.15.98:0/1031848594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8a7d18440 0x55a8a7cd8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6767fff640 1 -- 172.21.15.98:0/1031848594 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8a7ac3120 con 0x55a8a7cd3390 2024-09-19T08:39:14.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6767fff640 1 --2- 172.21.15.98:0/1031848594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8a7cd3390 0x55a8a7cd9230 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f676000eaa0 tx=0x7f676000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda7ee86419d9cb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:14.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f67467fc640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-19T08:39:14.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f67467fc640 1 -- 172.21.15.98:0/1031848594 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676001c060 con 0x55a8a7cd3390 2024-09-19T08:39:14.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f67467fc640 1 -- 172.21.15.98:0/1031848594 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f676000f450 con 0x55a8a7cd3390 2024-09-19T08:39:14.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6771103000 1 -- 172.21.15.98:0/1031848594 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f675c014ba0 con 0x55a8a7cd3390 2024-09-19T08:39:14.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f67467fc640 1 -- 172.21.15.98:0/1031848594 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7f6760022050 con 0x55a8a7cd3390 2024-09-19T08:39:14.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6771103000 10 client.5019 fetch_fsmap learned FSMap version 14 2024-09-19T08:39:14.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6771103000 1 -- 172.21.15.98:0/1031848594 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) -- 0x55a8a7abc450 con 0x55a8a7cd3390 2024-09-19T08:39:14.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f67467fc640 1 -- 172.21.15.98:0/1031848594 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f675c014ba0 con 0x55a8a7cd3390 2024-09-19T08:39:14.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f67467fc640 1 -- 172.21.15.98:0/1031848594 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 14) ==== 3502+0+0 (secure 0 0 0) 0x7f6760025020 con 0x55a8a7cd3390 2024-09-19T08:39:14.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6771103000 10 client.5019 fetch_fsmap finished waiting for FSMap version 14 2024-09-19T08:39:14.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6771103000 10 client.5019 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T08:39:14.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6771103000 20 client.5019 populate_metadata read hostname 'smithi098' 2024-09-19T08:39:14.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6771103000 1 --2- 172.21.15.98:0/1031848594 >> [v2:172.21.15.98:6836/1820856497,v1:172.21.15.98:6837/1820856497] conn(0x55a8a7d26280 0x55a8a7d28670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:14.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6771103000 4 client.5019 mds_command: new command op to 4833 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:39:14.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f6771103000 1 -- 172.21.15.98:0/1031848594 --> [v2:172.21.15.98:6836/1820856497,v1:172.21.15.98:6837/1820856497] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8a7ab6c00 con 0x55a8a7d26280 2024-09-19T08:39:14.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f67677fe640 1 --2- 172.21.15.98:0/1031848594 >> [v2:172.21.15.98:6836/1820856497,v1:172.21.15.98:6837/1820856497] conn(0x55a8a7d26280 0x55a8a7d28670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:14.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f67677fe640 1 --2- 172.21.15.98:0/1031848594 >> [v2:172.21.15.98:6836/1820856497,v1:172.21.15.98:6837/1820856497] conn(0x55a8a7d26280 0x55a8a7d28670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6125e3da788fe28 server_cookie=8983951f155a508f in_seq=0 out_seq=0 2024-09-19T08:39:14.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.379+0000 7f67467fc640 10 client.5019 ms_handle_connect on v2:172.21.15.98:6836/1820856497 2024-09-19T08:39:14.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.383+0000 7f67467fc640 1 -- 172.21.15.98:0/1031848594 <== mds.0 v2:172.21.15.98:6836/1820856497 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a8a7ab6c00 con 0x55a8a7d26280 2024-09-19T08:39:14.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.383+0000 7f67467fc640 10 client.5019 handle_command_reply: tid=0 2024-09-19T08:39:14.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.383+0000 7f6771103000 1 client.5019 shutdown 2024-09-19T08:39:14.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.383+0000 7f6771103000 1 -- 172.21.15.98:0/1031848594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8a7cd3390 msgr2=0x55a8a7cd9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:14.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.383+0000 7f6771103000 1 --2- 172.21.15.98:0/1031848594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8a7cd3390 0x55a8a7cd9230 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f676000eaa0 tx=0x7f676000ef60 comp rx=0 tx=0).stop 2024-09-19T08:39:14.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.383+0000 7f6771103000 1 -- 172.21.15.98:0/1031848594 shutdown_connections 2024-09-19T08:39:14.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.383+0000 7f6771103000 1 --2- 172.21.15.98:0/1031848594 >> [v2:172.21.15.98:6836/1820856497,v1:172.21.15.98:6837/1820856497] conn(0x55a8a7d26280 0x55a8a7d28670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.383+0000 7f6771103000 1 --2- 172.21.15.98:0/1031848594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8a7cd3390 0x55a8a7cd9230 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.383+0000 7f6771103000 1 --2- 172.21.15.98:0/1031848594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8a7d18440 0x55a8a7cd8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.383+0000 7f6771103000 1 --2- 172.21.15.98:0/1031848594 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a8a7d17b20 0x55a8a7cd2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.383+0000 7f67467fc640 0 client.5019 ms_handle_reset on v2:172.21.15.98:6836/1820856497 2024-09-19T08:39:14.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.383+0000 7f6771103000 1 -- 172.21.15.98:0/1031848594 >> 172.21.15.98:0/1031848594 conn(0x55a8a7cce290 msgr2=0x55a8a7cccd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:14.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.383+0000 7f6771103000 1 -- 172.21.15.98:0/1031848594 shutdown_connections 2024-09-19T08:39:14.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.383+0000 7f6771103000 1 -- 172.21.15.98:0/1031848594 wait complete. 2024-09-19T08:39:14.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.383+0000 7f6771103000 20 client.5019 trim_cache size 0 max 16384 2024-09-19T08:39:14.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.399+0000 7f6771103000 1 Processor -- start 2024-09-19T08:39:14.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.399+0000 7f6771103000 1 -- start start 2024-09-19T08:39:14.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.399+0000 7f6771103000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8a7cd3390 0x55a8a7d16cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:14.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.399+0000 7f6771103000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a8a7d17b20 0x55a8a7d17200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:14.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.399+0000 7f6771103000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8a7d18440 0x55a8a7d28670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:14.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.399+0000 7f6771103000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f67681fa240 con 0x55a8a7cd3390 2024-09-19T08:39:14.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.399+0000 7f6771103000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f676814c780 con 0x55a8a7d18440 2024-09-19T08:39:14.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.399+0000 7f6771103000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f676814cdd0 con 0x55a8a7d17b20 2024-09-19T08:39:14.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.399+0000 7f67677fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8a7cd3390 0x55a8a7d16cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:14.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.399+0000 7f67677fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8a7cd3390 0x55a8a7d16cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36564/0 (socket says 172.21.15.98:36564) 2024-09-19T08:39:14.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.399+0000 7f67677fe640 1 -- 172.21.15.98:0/2274284116 learned_addr learned my addr 172.21.15.98:0/2274284116 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:14.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.399+0000 7f67677fe640 1 -- 172.21.15.98:0/2274284116 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a8a7d17b20 msgr2=0x55a8a7d17200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:14.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.399+0000 7f67677fe640 1 --2- 172.21.15.98:0/2274284116 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a8a7d17b20 0x55a8a7d17200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.399+0000 7f67677fe640 1 -- 172.21.15.98:0/2274284116 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8a7d18440 msgr2=0x55a8a7d28670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:14.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.399+0000 7f67677fe640 1 --2- 172.21.15.98:0/2274284116 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8a7d18440 0x55a8a7d28670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.399+0000 7f67677fe640 1 -- 172.21.15.98:0/2274284116 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8a7aa3fd0 con 0x55a8a7cd3390 2024-09-19T08:39:14.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.399+0000 7f67677fe640 1 --2- 172.21.15.98:0/2274284116 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8a7cd3390 0x55a8a7d16cc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f675801c980 tx=0x7f675811e740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=911ee426c1b048ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:14.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f674cff9640 1 -- 172.21.15.98:0/2274284116 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67580f02c0 con 0x55a8a7cd3390 2024-09-19T08:39:14.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f674cff9640 1 -- 172.21.15.98:0/2274284116 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67580af040 con 0x55a8a7cd3390 2024-09-19T08:39:14.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f674cff9640 1 -- 172.21.15.98:0/2274284116 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67580f00e0 con 0x55a8a7cd3390 2024-09-19T08:39:14.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6771103000 1 -- 172.21.15.98:0/2274284116 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8a7cd3390 msgr2=0x55a8a7d16cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:14.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6771103000 1 --2- 172.21.15.98:0/2274284116 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8a7cd3390 0x55a8a7d16cc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f675801c980 tx=0x7f675811e740 comp rx=0 tx=0).stop 2024-09-19T08:39:14.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6771103000 1 -- 172.21.15.98:0/2274284116 shutdown_connections 2024-09-19T08:39:14.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6771103000 1 --2- 172.21.15.98:0/2274284116 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8a7d18440 0x55a8a7d28670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6771103000 1 --2- 172.21.15.98:0/2274284116 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a8a7d17b20 0x55a8a7d17200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6771103000 1 --2- 172.21.15.98:0/2274284116 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8a7cd3390 0x55a8a7d16cc0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6771103000 1 -- 172.21.15.98:0/2274284116 >> 172.21.15.98:0/2274284116 conn(0x55a8a7cce290 msgr2=0x55a8a7cd2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:14.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6771103000 1 -- 172.21.15.98:0/2274284116 shutdown_connections 2024-09-19T08:39:14.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6771103000 1 -- 172.21.15.98:0/2274284116 wait complete. 2024-09-19T08:39:14.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6771103000 1 -- start start 2024-09-19T08:39:14.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6771103000 1 Processor -- start 2024-09-19T08:39:14.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6771103000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8a7cd3390 0x55a8a7d2b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:14.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6771103000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8a7d17b20 0x55a8a7d46960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:14.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6771103000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a8a7d18440 0x55a8a7d46ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:14.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6771103000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55a8a7b1bf80 con 0x55a8a7d17b20 2024-09-19T08:39:14.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6771103000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55a8a7ad4ca0 con 0x55a8a7cd3390 2024-09-19T08:39:14.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6771103000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x55a8a7b212a0 con 0x55a8a7d18440 2024-09-19T08:39:14.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6771103000 10 client.0 _ll_register_callbacks cb 0x7f67600096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:39:14.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6766ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8a7d17b20 0x55a8a7d46960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:14.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6766ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8a7d17b20 0x55a8a7d46960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36574/0 (socket says 172.21.15.98:36574) 2024-09-19T08:39:14.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6766ffd640 1 -- 172.21.15.98:0/1930902393 learned_addr learned my addr 172.21.15.98:0/1930902393 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:14.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f67677fe640 1 --2- 172.21.15.98:0/1930902393 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8a7cd3390 0x55a8a7d2b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:14.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6767fff640 1 --2- 172.21.15.98:0/1930902393 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a8a7d18440 0x55a8a7d46ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:14.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6766ffd640 1 -- 172.21.15.98:0/1930902393 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a8a7d18440 msgr2=0x55a8a7d46ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:14.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6766ffd640 1 --2- 172.21.15.98:0/1930902393 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a8a7d18440 0x55a8a7d46ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6766ffd640 1 -- 172.21.15.98:0/1930902393 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8a7cd3390 msgr2=0x55a8a7d2b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:14.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6766ffd640 1 --2- 172.21.15.98:0/1930902393 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8a7cd3390 0x55a8a7d2b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6766ffd640 1 -- 172.21.15.98:0/1930902393 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8a7b9f750 con 0x55a8a7d17b20 2024-09-19T08:39:14.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f67677fe640 1 --2- 172.21.15.98:0/1930902393 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8a7cd3390 0x55a8a7d2b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:14.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6766ffd640 1 --2- 172.21.15.98:0/1930902393 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8a7d17b20 0x55a8a7d46960 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f675c001e30 tx=0x7f675c000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c175b9b72a368b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:14.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f67467fc640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-19T08:39:14.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f67467fc640 1 -- 172.21.15.98:0/1930902393 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c024050 con 0x55a8a7d17b20 2024-09-19T08:39:14.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f67467fc640 1 -- 172.21.15.98:0/1930902393 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f675c0298d0 con 0x55a8a7d17b20 2024-09-19T08:39:14.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6771103000 1 -- 172.21.15.98:0/1930902393 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67580af040 con 0x55a8a7d17b20 2024-09-19T08:39:14.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f67467fc640 1 -- 172.21.15.98:0/1930902393 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) ==== 24+0+0 (secure 0 0 0) 0x7f675c038050 con 0x55a8a7d17b20 2024-09-19T08:39:14.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6771103000 10 client.5025 fetch_fsmap learned FSMap version 14 2024-09-19T08:39:14.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6771103000 1 -- 172.21.15.98:0/1930902393 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) -- 0x55a8a7aaaa30 con 0x55a8a7d17b20 2024-09-19T08:39:14.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f67467fc640 1 -- 172.21.15.98:0/1930902393 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67580af040 con 0x55a8a7d17b20 2024-09-19T08:39:14.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f67467fc640 1 -- 172.21.15.98:0/1930902393 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 14) ==== 3502+0+0 (secure 0 0 0) 0x7f675c03b020 con 0x55a8a7d17b20 2024-09-19T08:39:14.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6771103000 10 client.5025 fetch_fsmap finished waiting for FSMap version 14 2024-09-19T08:39:14.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6771103000 10 client.5025 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T08:39:14.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6771103000 20 client.5025 populate_metadata read hostname 'smithi098' 2024-09-19T08:39:14.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6771103000 1 --2- 172.21.15.98:0/1930902393 >> [v2:172.21.15.98:6836/1820856497,v1:172.21.15.98:6837/1820856497] conn(0x55a8a7d473e0 0x55a8a7d8af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:14.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6771103000 4 client.5025 mds_command: new command op to 4833 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T08:39:14.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f6771103000 1 -- 172.21.15.98:0/1930902393 --> [v2:172.21.15.98:6836/1820856497,v1:172.21.15.98:6837/1820856497] -- command(tid 0: {"prefix": "status"}) -- 0x55a8a7b2a3a0 con 0x55a8a7d473e0 2024-09-19T08:39:14.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f67677fe640 1 --2- 172.21.15.98:0/1930902393 >> [v2:172.21.15.98:6836/1820856497,v1:172.21.15.98:6837/1820856497] conn(0x55a8a7d473e0 0x55a8a7d8af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:14.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f67677fe640 1 --2- 172.21.15.98:0/1930902393 >> [v2:172.21.15.98:6836/1820856497,v1:172.21.15.98:6837/1820856497] conn(0x55a8a7d473e0 0x55a8a7d8af30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c84df457caae28f server_cookie=ec4e1be346dcfd24 in_seq=0 out_seq=0 2024-09-19T08:39:14.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.403+0000 7f67467fc640 10 client.5025 ms_handle_connect on v2:172.21.15.98:6836/1820856497 2024-09-19T08:39:14.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.407+0000 7f67467fc640 1 -- 172.21.15.98:0/1930902393 <== mds.0 v2:172.21.15.98:6836/1820856497 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55a8a7b2a3a0 con 0x55a8a7d473e0 2024-09-19T08:39:14.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.407+0000 7f67467fc640 10 client.5025 handle_command_reply: tid=0 2024-09-19T08:39:14.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.407+0000 7f6771103000 1 client.5025 shutdown 2024-09-19T08:39:14.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.407+0000 7f6771103000 1 -- 172.21.15.98:0/1930902393 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8a7d17b20 msgr2=0x55a8a7d46960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:14.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.407+0000 7f6771103000 1 --2- 172.21.15.98:0/1930902393 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8a7d17b20 0x55a8a7d46960 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f675c001e30 tx=0x7f675c000bd0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.407+0000 7f6771103000 1 -- 172.21.15.98:0/1930902393 shutdown_connections 2024-09-19T08:39:14.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.407+0000 7f6771103000 1 --2- 172.21.15.98:0/1930902393 >> [v2:172.21.15.98:6836/1820856497,v1:172.21.15.98:6837/1820856497] conn(0x55a8a7d473e0 0x55a8a7d8af30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.407+0000 7f6771103000 1 --2- 172.21.15.98:0/1930902393 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55a8a7d18440 0x55a8a7d46ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.407+0000 7f67467fc640 0 client.5025 ms_handle_reset on v2:172.21.15.98:6836/1820856497 2024-09-19T08:39:14.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.407+0000 7f6771103000 1 --2- 172.21.15.98:0/1930902393 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55a8a7d17b20 0x55a8a7d46960 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.407+0000 7f6771103000 1 --2- 172.21.15.98:0/1930902393 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55a8a7cd3390 0x55a8a7d2b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.407+0000 7f6771103000 1 -- 172.21.15.98:0/1930902393 >> 172.21.15.98:0/1930902393 conn(0x55a8a7cce290 msgr2=0x55a8a7d15cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:14.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.407+0000 7f6771103000 1 -- 172.21.15.98:0/1930902393 shutdown_connections 2024-09-19T08:39:14.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.407+0000 7f6771103000 1 -- 172.21.15.98:0/1930902393 wait complete. 2024-09-19T08:39:14.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.407+0000 7f6771103000 20 client.5025 trim_cache size 0 max 16384 2024-09-19T08:39:14.415 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:39:14.415 INFO:teuthology.orchestra.run.smithi098.stdout: "cluster_fsid": "552bf0ee-1f4f-4644-ba7c-d0931e55a451", 2024-09-19T08:39:14.415 INFO:teuthology.orchestra.run.smithi098.stdout: "whoami": 0, 2024-09-19T08:39:14.415 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 4833, 2024-09-19T08:39:14.415 INFO:teuthology.orchestra.run.smithi098.stdout: "want_state": "up:active", 2024-09-19T08:39:14.415 INFO:teuthology.orchestra.run.smithi098.stdout: "state": "up:active", 2024-09-19T08:39:14.416 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "cephfs", 2024-09-19T08:39:14.416 INFO:teuthology.orchestra.run.smithi098.stdout: "rank_uptime": 9.1609115429999992, 2024-09-19T08:39:14.416 INFO:teuthology.orchestra.run.smithi098.stdout: "mdsmap_epoch": 14, 2024-09-19T08:39:14.416 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch": 32, 2024-09-19T08:39:14.416 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch_barrier": 32, 2024-09-19T08:39:14.416 INFO:teuthology.orchestra.run.smithi098.stdout: "uptime": 16.056444511999999 2024-09-19T08:39:14.416 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:39:14.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.407+0000 7f676e047640 1 -- 172.21.15.98:0/1836653369 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f67540678c0 msgr2=0x7f6754069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:14.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.407+0000 7f676e047640 1 --2- 172.21.15.98:0/1836653369 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f67540678c0 0x7f6754069d80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f675c007fd0 tx=0x7f675c009040 comp rx=0 tx=0).stop 2024-09-19T08:39:14.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.407+0000 7f676e047640 1 -- 172.21.15.98:0/1836653369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f676806c3a0 msgr2=0x7f6768073b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:14.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.407+0000 7f676e047640 1 --2- 172.21.15.98:0/1836653369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f676806c3a0 0x7f6768073b50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f67580ac040 tx=0x7f675801ace0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.407+0000 7f676e047640 1 -- 172.21.15.98:0/1836653369 shutdown_connections 2024-09-19T08:39:14.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.407+0000 7f676e047640 1 --2- 172.21.15.98:0/1836653369 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f67540678c0 0x7f6754069d80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.407+0000 7f676e047640 1 --2- 172.21.15.98:0/1836653369 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6768158fd0 0x7f6768071220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.407+0000 7f676e047640 1 --2- 172.21.15.98:0/1836653369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67680b4360 0x7f6768151e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.407+0000 7f676e047640 1 --2- 172.21.15.98:0/1836653369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f676806c3a0 0x7f6768073b50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.407+0000 7f676e047640 1 -- 172.21.15.98:0/1836653369 >> 172.21.15.98:0/1836653369 conn(0x7f67680b2010 msgr2=0x7f67680ae8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:14.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.407+0000 7f676e047640 1 -- 172.21.15.98:0/1836653369 shutdown_connections 2024-09-19T08:39:14.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.407+0000 7f676e047640 1 -- 172.21.15.98:0/1836653369 wait complete. 2024-09-19T08:39:14.425 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T08:39:14.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.523+0000 7fada8227640 1 Processor -- start 2024-09-19T08:39:14.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.523+0000 7fada8227640 1 -- start start 2024-09-19T08:39:14.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.527+0000 7fada8227640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fada015c550 0x7fada006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:14.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.527+0000 7fada8227640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fada006c1d0 0x7fada00cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:14.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.527+0000 7fada8227640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fada006d310 con 0x7fada00cf900 2024-09-19T08:39:14.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.527+0000 7fada8227640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fada006c610 con 0x7fada006c1d0 2024-09-19T08:39:14.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.527+0000 7fada8227640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fada006d5f0 con 0x7fada015c550 2024-09-19T08:39:14.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.527+0000 7fada67c4640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fada00cf900 0x7fada00ce1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:45596/0 (socket says 172.21.15.98:45596) 2024-09-19T08:39:14.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.527+0000 7fada67c4640 1 -- 172.21.15.98:0/1668579676 learned_addr learned my addr 172.21.15.98:0/1668579676 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:14.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.527+0000 7fada5fc3640 1 --2- 172.21.15.98:0/1668579676 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fada015c550 0x7fada006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:14.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.527+0000 7fada57c2640 1 --2- 172.21.15.98:0/1668579676 >> v2:172.21.15.139:3300/0 conn(0x7fada006c1d0 0x7fada00cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:14.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.527+0000 7fada4fc1640 1 -- 172.21.15.98:0/1668579676 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 629395437 0 0) 0x7fada006d310 con 0x7fada00cf900 2024-09-19T08:39:14.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.527+0000 7fada4fc1640 1 -- 172.21.15.98:0/1668579676 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad8c003530 con 0x7fada00cf900 2024-09-19T08:39:14.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.527+0000 7fada4fc1640 1 -- 172.21.15.98:0/1668579676 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2105351 0 0) 0x7fad8c003530 con 0x7fada00cf900 2024-09-19T08:39:14.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.527+0000 7fada4fc1640 1 -- 172.21.15.98:0/1668579676 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fada015c550 msgr2=0x7fada006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:14.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.527+0000 7fada4fc1640 1 --2- 172.21.15.98:0/1668579676 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fada015c550 0x7fada006bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.527+0000 7fada4fc1640 1 -- 172.21.15.98:0/1668579676 >> v2:172.21.15.139:3300/0 conn(0x7fada006c1d0 msgr2=0x7fada00cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:14.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.527+0000 7fada4fc1640 1 --2- 172.21.15.98:0/1668579676 >> v2:172.21.15.139:3300/0 conn(0x7fada006c1d0 0x7fada00cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.527+0000 7fada4fc1640 1 -- 172.21.15.98:0/1668579676 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada006d8d0 con 0x7fada00cf900 2024-09-19T08:39:14.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.527+0000 7fada4fc1640 1 -- 172.21.15.98:0/1668579676 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fad98001680 con 0x7fada00cf900 2024-09-19T08:39:14.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.527+0000 7fada4fc1640 1 -- 172.21.15.98:0/1668579676 >> v1:172.21.15.98:6789/0 conn(0x7fada00cf900 legacy=0x7fada00ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:14.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.527+0000 7fada4fc1640 1 --2- 172.21.15.98:0/1668579676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad8c0039c0 0x7fad8c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:14.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.527+0000 7fada4fc1640 1 --2- 172.21.15.98:0/1668579676 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fada015c550 0x7fad8c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:14.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.527+0000 7fada4fc1640 1 --2- 172.21.15.98:0/1668579676 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad8c004d50 0x7fad8c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:14.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.527+0000 7fada4fc1640 1 -- 172.21.15.98:0/1668579676 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fada006c610 con 0x7fad8c0039c0 2024-09-19T08:39:14.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.527+0000 7fada4fc1640 1 -- 172.21.15.98:0/1668579676 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fada006d5f0 con 0x7fad8c004d50 2024-09-19T08:39:14.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.527+0000 7fada4fc1640 1 -- 172.21.15.98:0/1668579676 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fad8c00b9d0 con 0x7fada015c550 2024-09-19T08:39:14.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.527+0000 7fada5fc3640 1 --2- 172.21.15.98:0/1668579676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad8c0039c0 0x7fad8c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:14.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.527+0000 7fada5fc3640 1 -- 172.21.15.98:0/1668579676 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fada015c550 msgr2=0x7fad8c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:14.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.527+0000 7fada5fc3640 1 --2- 172.21.15.98:0/1668579676 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fada015c550 0x7fad8c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.527+0000 7fada5fc3640 1 -- 172.21.15.98:0/1668579676 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad8c004d50 msgr2=0x7fad8c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:14.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.527+0000 7fada5fc3640 1 --2- 172.21.15.98:0/1668579676 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad8c004d50 0x7fad8c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.527+0000 7fada5fc3640 1 -- 172.21.15.98:0/1668579676 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad8c003530 con 0x7fad8c0039c0 2024-09-19T08:39:14.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.531+0000 7fada5fc3640 1 --2- 172.21.15.98:0/1668579676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad8c0039c0 0x7fad8c003de0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fad9c009ac0 tx=0x7fad9c01ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c906d056996230 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:14.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.531+0000 7fada4fc1640 1 -- 172.21.15.98:0/1668579676 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c009690 con 0x7fad8c0039c0 2024-09-19T08:39:14.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.531+0000 7fada4fc1640 1 -- 172.21.15.98:0/1668579676 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad9c0a6470 con 0x7fad8c0039c0 2024-09-19T08:39:14.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.531+0000 7fada4fc1640 1 -- 172.21.15.98:0/1668579676 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c0a6840 con 0x7fad8c0039c0 2024-09-19T08:39:14.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.531+0000 7fada8227640 1 -- 172.21.15.98:0/1668579676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad8c0039c0 msgr2=0x7fad8c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:14.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.531+0000 7fada8227640 1 --2- 172.21.15.98:0/1668579676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad8c0039c0 0x7fad8c003de0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fad9c009ac0 tx=0x7fad9c01ca90 comp rx=0 tx=0).stop 2024-09-19T08:39:14.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.531+0000 7fada67c4640 1 -- 172.21.15.98:0/1668579676 reap_dead start 2024-09-19T08:39:14.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.531+0000 7fada8227640 1 -- 172.21.15.98:0/1668579676 shutdown_connections 2024-09-19T08:39:14.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.531+0000 7fada8227640 1 -- 172.21.15.98:0/1668579676 >> 172.21.15.98:0/1668579676 conn(0x7fada00b2010 msgr2=0x7fada00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:14.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.531+0000 7fada8227640 1 -- 172.21.15.98:0/1668579676 shutdown_connections 2024-09-19T08:39:14.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.531+0000 7fada8227640 1 -- 172.21.15.98:0/1668579676 wait complete. 2024-09-19T08:39:14.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.531+0000 7fada8227640 1 Processor -- start 2024-09-19T08:39:14.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.531+0000 7fada8227640 1 -- start start 2024-09-19T08:39:14.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.531+0000 7fada8227640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad8c0039c0 0x7fada014d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:14.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.531+0000 7fada8227640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fada00b1990 0x7fada01f5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:14.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.531+0000 7fada8227640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fada0157ba0 0x7fada0158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:14.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.531+0000 7fada8227640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fada0167220 con 0x7fad8c0039c0 2024-09-19T08:39:14.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.531+0000 7fada8227640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fada015cb40 con 0x7fada0157ba0 2024-09-19T08:39:14.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.531+0000 7fada8227640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fada01673a0 con 0x7fada00b1990 2024-09-19T08:39:14.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.531+0000 7fada5fc3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad8c0039c0 0x7fada014d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:14.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.531+0000 7fada5fc3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad8c0039c0 0x7fada014d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36586/0 (socket says 172.21.15.98:36586) 2024-09-19T08:39:14.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.531+0000 7fada5fc3640 1 -- 172.21.15.98:0/388379958 learned_addr learned my addr 172.21.15.98:0/388379958 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:14.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.531+0000 7fada57c2640 1 --2- 172.21.15.98:0/388379958 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fada00b1990 0x7fada01f5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:14.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.531+0000 7fada67c4640 1 --2- 172.21.15.98:0/388379958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fada0157ba0 0x7fada0158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:14.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.531+0000 7fada5fc3640 1 -- 172.21.15.98:0/388379958 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fada00b1990 msgr2=0x7fada01f5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:14.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.531+0000 7fada5fc3640 1 --2- 172.21.15.98:0/388379958 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fada00b1990 0x7fada01f5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.531+0000 7fada5fc3640 1 -- 172.21.15.98:0/388379958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fada0157ba0 msgr2=0x7fada0158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:14.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.531+0000 7fada5fc3640 1 --2- 172.21.15.98:0/388379958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fada0157ba0 0x7fada0158020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.531+0000 7fada5fc3640 1 -- 172.21.15.98:0/388379958 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada00b1e10 con 0x7fad8c0039c0 2024-09-19T08:39:14.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.531+0000 7fada57c2640 1 --2- 172.21.15.98:0/388379958 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fada00b1990 0x7fada01f5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:14.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.531+0000 7fada5fc3640 1 --2- 172.21.15.98:0/388379958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad8c0039c0 0x7fada014d8b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fad9c01c8c0 tx=0x7fad8c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:14.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.531+0000 7fad927fc640 1 -- 172.21.15.98:0/388379958 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c00f000 con 0x7fad8c0039c0 2024-09-19T08:39:14.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.531+0000 7fad927fc640 1 -- 172.21.15.98:0/388379958 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad9c0a6ca0 con 0x7fad8c0039c0 2024-09-19T08:39:14.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.531+0000 7fad927fc640 1 -- 172.21.15.98:0/388379958 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9c0a7690 con 0x7fad8c0039c0 2024-09-19T08:39:14.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.535+0000 7fada8227640 1 -- 172.21.15.98:0/388379958 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada01f6000 con 0x7fad8c0039c0 2024-09-19T08:39:14.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.535+0000 7fada8227640 1 -- 172.21.15.98:0/388379958 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fada01f65c0 con 0x7fad8c0039c0 2024-09-19T08:39:14.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.535+0000 7fada8227640 1 -- 172.21.15.98:0/388379958 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fada01f6800 con 0x7fad8c0039c0 2024-09-19T08:39:14.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.535+0000 7fad927fc640 1 -- 172.21.15.98:0/388379958 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fad9c0a7830 con 0x7fad8c0039c0 2024-09-19T08:39:14.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.535+0000 7fad927fc640 1 --2- 172.21.15.98:0/388379958 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fad840678c0 0x7fad84069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:14.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.535+0000 7fad927fc640 1 -- 172.21.15.98:0/388379958 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fad9c11ffe0 con 0x7fad8c0039c0 2024-09-19T08:39:14.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.535+0000 7fada57c2640 1 --2- 172.21.15.98:0/388379958 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fad840678c0 0x7fad84069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:14.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.535+0000 7fada57c2640 1 --2- 172.21.15.98:0/388379958 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fad840678c0 0x7fad84069d80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fada01611c0 tx=0x7fad9400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:14.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.539+0000 7fad927fc640 1 -- 172.21.15.98:0/388379958 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fad9c0ac020 con 0x7fad8c0039c0 2024-09-19T08:39:14.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.719+0000 7fada8227640 1 -- 172.21.15.98:0/388379958 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fada00cf3c0 con 0x7fad8c0039c0 2024-09-19T08:39:14.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.723+0000 7fad927fc640 1 -- 172.21.15.98:0/388379958 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) ==== 53+0+73 (secure 0 0 0) 0x7fad9c0167a0 con 0x7fad8c0039c0 2024-09-19T08:39:14.725 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T08:39:14.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.723+0000 7fada8227640 1 -- 172.21.15.98:0/388379958 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fad840678c0 msgr2=0x7fad84069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:14.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.723+0000 7fada8227640 1 --2- 172.21.15.98:0/388379958 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fad840678c0 0x7fad84069d80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fada01611c0 tx=0x7fad9400b040 comp rx=0 tx=0).stop 2024-09-19T08:39:14.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.723+0000 7fada8227640 1 -- 172.21.15.98:0/388379958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad8c0039c0 msgr2=0x7fada014d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:14.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.723+0000 7fada8227640 1 --2- 172.21.15.98:0/388379958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad8c0039c0 0x7fada014d8b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fad9c01c8c0 tx=0x7fad8c005840 comp rx=0 tx=0).stop 2024-09-19T08:39:14.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.723+0000 7fada8227640 1 -- 172.21.15.98:0/388379958 shutdown_connections 2024-09-19T08:39:14.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.723+0000 7fada8227640 1 --2- 172.21.15.98:0/388379958 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fad840678c0 0x7fad84069d80 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.723+0000 7fada8227640 1 --2- 172.21.15.98:0/388379958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fada0157ba0 0x7fada0158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.723+0000 7fada8227640 1 --2- 172.21.15.98:0/388379958 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fada00b1990 0x7fada01f5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.723+0000 7fada8227640 1 --2- 172.21.15.98:0/388379958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad8c0039c0 0x7fada014d8b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:14.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.723+0000 7fada8227640 1 -- 172.21.15.98:0/388379958 >> 172.21.15.98:0/388379958 conn(0x7fada00b2010 msgr2=0x7fada00cc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:14.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.723+0000 7fada8227640 1 -- 172.21.15.98:0/388379958 shutdown_connections 2024-09-19T08:39:14.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14.723+0000 7fada8227640 1 -- 172.21.15.98:0/388379958 wait complete. 2024-09-19T08:39:14.743 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T08:39:14.743 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T08:39:14.743 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi098.front.sepia.ceph.com 2024-09-19T08:39:14.743 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T08:39:14.743 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T08:39:14.743 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T08:39:14.743 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T08:39:14.743 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T08:39:14.743 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:39:14.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:14.744 DEBUG:teuthology.orchestra.run.smithi098:> ip addr 2024-09-19T08:39:14.749 INFO:teuthology.orchestra.run.smithi098.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T08:39:14.749 INFO:teuthology.orchestra.run.smithi098.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T08:39:14.750 INFO:teuthology.orchestra.run.smithi098.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T08:39:14.750 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:39:14.750 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 ::1/128 scope host 2024-09-19T08:39:14.750 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:39:14.750 INFO:teuthology.orchestra.run.smithi098.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:39:14.750 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b8 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:39:14.750 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f0 2024-09-19T08:39:14.750 INFO:teuthology.orchestra.run.smithi098.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:39:14.750 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b9 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:39:14.750 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f1 2024-09-19T08:39:14.750 INFO:teuthology.orchestra.run.smithi098.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:39:14.750 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6a brd ff:ff:ff:ff:ff:ff 2024-09-19T08:39:14.750 INFO:teuthology.orchestra.run.smithi098.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T08:39:14.751 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6b brd ff:ff:ff:ff:ff:ff 2024-09-19T08:39:14.751 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.21.15.98/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T08:39:14.751 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft 42454sec preferred_lft 42454sec 2024-09-19T08:39:14.751 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::ec4:7aff:fe8f:d26b/64 scope link 2024-09-19T08:39:14.751 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:39:14.751 INFO:teuthology.orchestra.run.smithi098.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T08:39:14.751 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 02:42:98:dd:32:2d brd ff:ff:ff:ff:ff:ff 2024-09-19T08:39:14.751 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T08:39:14.751 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:39:14.751 INFO:teuthology.orchestra.run.smithi098.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T08:39:14.751 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-19T08:39:14.751 INFO:teuthology.orchestra.run.smithi098.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T08:39:14.751 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:39:14.751 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-19T08:39:14.752 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:39:14.752 INFO:teuthology.orchestra.run.smithi098.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:39:14.752 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:39:14.752 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-19T08:39:14.752 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:39:14.752 INFO:teuthology.orchestra.run.smithi098.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:39:14.752 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:39:14.752 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-19T08:39:14.752 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:39:14.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:14.753 DEBUG:teuthology.orchestra.run.smithi098:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T08:39:14.805 INFO:teuthology.orchestra.run.smithi098.stdout:1 2024-09-19T08:39:14.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:14.806 DEBUG:teuthology.orchestra.run.smithi098:> route 2024-09-19T08:39:14.856 INFO:teuthology.orchestra.run.smithi098.stdout:Kernel IP routing table 2024-09-19T08:39:14.856 INFO:teuthology.orchestra.run.smithi098.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T08:39:14.857 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T08:39:14.857 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T08:39:14.857 INFO:teuthology.orchestra.run.smithi098.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T08:39:14.857 INFO:teuthology.orchestra.run.smithi098.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T08:39:14.857 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T08:39:14.857 INFO:teuthology.orchestra.run.smithi098.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:39:14.857 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:39:14.857 INFO:teuthology.orchestra.run.smithi098.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:39:14.857 INFO:teuthology.orchestra.run.smithi098.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T08:39:14.858 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:39:14.858 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-09-19T08:39:14.858 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T08:39:14.858 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T08:39:14.858 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T08:39:14.858 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-09-19T08:39:14.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:39:14.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:39:14.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:14.978 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2024-09-19T08:39:15.025 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-19T08:39:15.025 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T08:39:15.025 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:39:15.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:15.026 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set brx.0 up 2024-09-19T08:39:15.079 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-19T08:39:15.079 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:39:15.080 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T08:39:15.125 INFO:teuthology.orchestra.run.smithi098.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:39:15.126 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:39:15.126 DEBUG:teuthology.orchestra.run.smithi098:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T08:39:15.173 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:39:15.231 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:39:15.281 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:39:15.281 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:39:15.281 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:39:15.281 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:39:15.281 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:39:15.281 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:39:15.281 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:39:15.281 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:39:15.281 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:39:15.281 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:39:15.282 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:39:15.282 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:39:15.282 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:39:15.282 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:39:15.282 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:39:15.282 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:39:15.282 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:39:15.282 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:39:15.282 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:39:15.282 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:39:15.282 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:39:15.282 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:39:15.283 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:39:15.283 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:39:15.283 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:39:15.283 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:39:15.283 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:39:15.283 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:39:15.283 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:39:15.283 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:39:15.283 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:39:15.283 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:39:15.283 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:39:15.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:15.284 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:39:15.333 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T08:39:15.334 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-19T08:39:15.378 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:39:15.393 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:39:15.442 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:39:15.442 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:39:15.442 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:39:15.442 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:39:15.442 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:39:15.442 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:39:15.442 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:39:15.442 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:39:15.442 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:39:15.442 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:39:15.442 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:39:15.442 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:39:15.443 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:39:15.443 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:39:15.443 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:39:15.443 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:39:15.443 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:39:15.443 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:39:15.443 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:39:15.443 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:39:15.443 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:39:15.443 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:39:15.443 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:39:15.443 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:39:15.443 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:39:15.444 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:39:15.444 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:39:15.444 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:39:15.444 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:39:15.444 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:39:15.444 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:39:15.444 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:39:15.444 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:39:15.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:15.445 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:39:15.457 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:2024-09-19T08:39:15.455+0000 7f34134ca580 -1 init, newargv = 0x55c80b8495b0 newargc=15 2024-09-19T08:39:15.457 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:ceph-fuse[37711]: starting ceph client 2024-09-19T08:39:15.462 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:ceph-fuse[37711]: starting fuse 2024-09-19T08:39:16.563 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:39:16.579 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:39:16.627 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:39:16.627 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:39:16.627 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:39:16.627 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:39:16.627 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:39:16.627 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:39:16.627 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:39:16.627 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:39:16.627 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:39:16.627 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:39:16.628 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:39:16.628 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:39:16.628 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:39:16.628 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:39:16.628 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:39:16.628 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:39:16.628 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:39:16.628 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:39:16.628 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:39:16.628 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:39:16.628 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:39:16.628 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:39:16.628 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:39:16.629 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:39:16.629 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:39:16.629 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:39:16.629 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:39:16.629 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:39:16.629 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:39:16.629 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:39:16.629 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:39:16.629 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:39:16.629 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:39:16.629 INFO:teuthology.orchestra.run.smithi098.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-19T08:39:16.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:16.630 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:39:16.682 INFO:teuthology.orchestra.run.smithi098.stdout:45 2024-09-19T08:39:16.682 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-19T08:39:16.682 DEBUG:teuthology.orchestra.run.smithi098:> sudo stdin-killer -- python3 -c ' 2024-09-19T08:39:16.682 DEBUG:teuthology.orchestra.run.smithi098:> import glob 2024-09-19T08:39:16.682 DEBUG:teuthology.orchestra.run.smithi098:> import re 2024-09-19T08:39:16.682 DEBUG:teuthology.orchestra.run.smithi098:> import os 2024-09-19T08:39:16.683 DEBUG:teuthology.orchestra.run.smithi098:> import subprocess 2024-09-19T08:39:16.683 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:39:16.683 DEBUG:teuthology.orchestra.run.smithi098:> def _find_admin_socket(client_name): 2024-09-19T08:39:16.683 DEBUG:teuthology.orchestra.run.smithi098:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T08:39:16.683 DEBUG:teuthology.orchestra.run.smithi098:> files = glob.glob(asok_path) 2024-09-19T08:39:16.683 DEBUG:teuthology.orchestra.run.smithi098:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T08:39:16.683 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:39:16.683 DEBUG:teuthology.orchestra.run.smithi098:> # Given a non-glob path, it better be there 2024-09-19T08:39:16.683 DEBUG:teuthology.orchestra.run.smithi098:> if "*" not in asok_path: 2024-09-19T08:39:16.683 DEBUG:teuthology.orchestra.run.smithi098:> assert(len(files) == 1) 2024-09-19T08:39:16.683 DEBUG:teuthology.orchestra.run.smithi098:> return files[0] 2024-09-19T08:39:16.683 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:39:16.683 DEBUG:teuthology.orchestra.run.smithi098:> for f in files: 2024-09-19T08:39:16.684 DEBUG:teuthology.orchestra.run.smithi098:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T08:39:16.684 DEBUG:teuthology.orchestra.run.smithi098:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T08:39:16.684 DEBUG:teuthology.orchestra.run.smithi098:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T08:39:16.684 DEBUG:teuthology.orchestra.run.smithi098:> contents = proc_f.read() 2024-09-19T08:39:16.684 DEBUG:teuthology.orchestra.run.smithi098:> if mountpoint in contents: 2024-09-19T08:39:16.684 DEBUG:teuthology.orchestra.run.smithi098:> return f 2024-09-19T08:39:16.684 DEBUG:teuthology.orchestra.run.smithi098:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T08:39:16.684 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:39:16.684 DEBUG:teuthology.orchestra.run.smithi098:> print(_find_admin_socket("client.0")) 2024-09-19T08:39:16.684 DEBUG:teuthology.orchestra.run.smithi098:> ' 2024-09-19T08:39:16.806 INFO:teuthology.orchestra.run.smithi098.stdout:/var/run/ceph/ceph-client.0.37711.asok 2024-09-19T08:39:16.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:39:16.818 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.37711.asok 2024-09-19T08:39:16.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:16.819 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.37711.asok status 2024-09-19T08:39:16.970 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:39:16.970 INFO:teuthology.orchestra.run.smithi098.stdout: "metadata": { 2024-09-19T08:39:16.970 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T08:39:16.970 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T08:39:16.970 INFO:teuthology.orchestra.run.smithi098.stdout: "entity_id": "0", 2024-09-19T08:39:16.970 INFO:teuthology.orchestra.run.smithi098.stdout: "hostname": "smithi098", 2024-09-19T08:39:16.970 INFO:teuthology.orchestra.run.smithi098.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T08:39:16.970 INFO:teuthology.orchestra.run.smithi098.stdout: "pid": "37711", 2024-09-19T08:39:16.970 INFO:teuthology.orchestra.run.smithi098.stdout: "root": "/" 2024-09-19T08:39:16.970 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:39:16.971 INFO:teuthology.orchestra.run.smithi098.stdout: "dentry_count": 0, 2024-09-19T08:39:16.971 INFO:teuthology.orchestra.run.smithi098.stdout: "dentry_pinned_count": 0, 2024-09-19T08:39:16.971 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 5037, 2024-09-19T08:39:16.971 INFO:teuthology.orchestra.run.smithi098.stdout: "inst": { 2024-09-19T08:39:16.971 INFO:teuthology.orchestra.run.smithi098.stdout: "name": { 2024-09-19T08:39:16.971 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "client", 2024-09-19T08:39:16.971 INFO:teuthology.orchestra.run.smithi098.stdout: "num": 5037 2024-09-19T08:39:16.971 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:39:16.971 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": { 2024-09-19T08:39:16.971 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v1", 2024-09-19T08:39:16.971 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:39:16.971 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 3394799537 2024-09-19T08:39:16.971 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:39:16.971 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:39:16.972 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": { 2024-09-19T08:39:16.972 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v1", 2024-09-19T08:39:16.972 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:39:16.972 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 3394799537 2024-09-19T08:39:16.972 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:39:16.972 INFO:teuthology.orchestra.run.smithi098.stdout: "inst_str": "client.5037 192.168.0.1:0/3394799537", 2024-09-19T08:39:16.972 INFO:teuthology.orchestra.run.smithi098.stdout: "addr_str": "192.168.0.1:0/3394799537", 2024-09-19T08:39:16.972 INFO:teuthology.orchestra.run.smithi098.stdout: "inode_count": 1, 2024-09-19T08:39:16.972 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_epoch": 14, 2024-09-19T08:39:16.972 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_epoch": 32, 2024-09-19T08:39:16.972 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_epoch_barrier": 0, 2024-09-19T08:39:16.972 INFO:teuthology.orchestra.run.smithi098.stdout: "blocklisted": false, 2024-09-19T08:39:16.972 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "cephfs" 2024-09-19T08:39:16.973 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:39:16.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:16.984 DEBUG:teuthology.orchestra.run.smithi098:> stat --file-system '--printf=%T 2024-09-19T08:39:16.985 DEBUG:teuthology.orchestra.run.smithi098:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T08:39:16.991 INFO:teuthology.orchestra.run.smithi098.stdout:fuseblk 2024-09-19T08:39:16.991 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T08:39:16.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:16.991 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T08:39:17.037 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:39:17.045 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T08:39:17.090 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:39:17.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.215+0000 7f7c0f83c640 1 Processor -- start 2024-09-19T08:39:17.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.215+0000 7f7c0f83c640 1 -- start start 2024-09-19T08:39:17.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.215+0000 7f7c0f83c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c0815c550 0x7f7c0806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:17.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.215+0000 7f7c0f83c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7c0806c1d0 0x7f7c080cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:17.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.215+0000 7f7c0f83c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c0806d310 con 0x7f7c080cf900 2024-09-19T08:39:17.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.215+0000 7f7c0f83c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f7c0806c610 con 0x7f7c0806c1d0 2024-09-19T08:39:17.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.215+0000 7f7c0f83c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7c0806d5f0 con 0x7f7c0815c550 2024-09-19T08:39:17.221 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.215+0000 7f7c0d5d8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c0815c550 0x7f7c0806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:17.221 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.215+0000 7f7c0cdd7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7c0806c1d0 0x7f7c080cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:17.221 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.215+0000 7f7c0cdd7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7c0806c1d0 0x7f7c080cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:37214/0 (socket says 172.21.15.98:37214) 2024-09-19T08:39:17.221 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.215+0000 7f7c0cdd7640 1 -- 172.21.15.98:0/846330348 learned_addr learned my addr 172.21.15.98:0/846330348 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:17.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.219+0000 7f7c0cdd7640 1 -- 172.21.15.98:0/846330348 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c0815c550 msgr2=0x7f7c0806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:17.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.219+0000 7f7c0cdd7640 1 --2- 172.21.15.98:0/846330348 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c0815c550 0x7f7c0806bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:17.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.219+0000 7f7c0cdd7640 1 -- 172.21.15.98:0/846330348 >> v1:172.21.15.139:6789/0 conn(0x7f7c080cf900 legacy=0x7f7c080ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:17.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.219+0000 7f7c0cdd7640 1 -- 172.21.15.98:0/846330348 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0806d8d0 con 0x7f7c0806c1d0 2024-09-19T08:39:17.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.219+0000 7f7c0cdd7640 1 --2- 172.21.15.98:0/846330348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c0806c1d0 0x7f7c080cf3c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8009870 tx=0x7f7bf801c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2532626b88efeaa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:17.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.219+0000 7f7bfffff640 1 -- 172.21.15.98:0/846330348 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf80b2070 con 0x7f7c0806c1d0 2024-09-19T08:39:17.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.219+0000 7f7bfffff640 1 -- 172.21.15.98:0/846330348 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bf80a9070 con 0x7f7c0806c1d0 2024-09-19T08:39:17.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.219+0000 7f7bfffff640 1 -- 172.21.15.98:0/846330348 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf80ad880 con 0x7f7c0806c1d0 2024-09-19T08:39:17.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.219+0000 7f7c0f83c640 1 -- 172.21.15.98:0/846330348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c0806c1d0 msgr2=0x7f7c080cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:17.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.219+0000 7f7c0f83c640 1 --2- 172.21.15.98:0/846330348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c0806c1d0 0x7f7c080cf3c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8009870 tx=0x7f7bf801c560 comp rx=0 tx=0).stop 2024-09-19T08:39:17.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.219+0000 7f7c0f83c640 1 -- 172.21.15.98:0/846330348 shutdown_connections 2024-09-19T08:39:17.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.219+0000 7f7c0f83c640 1 --2- 172.21.15.98:0/846330348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c0806c1d0 0x7f7c080cf3c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:17.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.219+0000 7f7c0f83c640 1 --2- 172.21.15.98:0/846330348 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c0815c550 0x7f7c0806bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:17.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.219+0000 7f7c0f83c640 1 -- 172.21.15.98:0/846330348 >> 172.21.15.98:0/846330348 conn(0x7f7c080b2010 msgr2=0x7f7c080b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:17.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.219+0000 7f7c0f83c640 1 -- 172.21.15.98:0/846330348 shutdown_connections 2024-09-19T08:39:17.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.219+0000 7f7c0f83c640 1 -- 172.21.15.98:0/846330348 wait complete. 2024-09-19T08:39:17.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.223+0000 7f7c0f83c640 1 Processor -- start 2024-09-19T08:39:17.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.223+0000 7f7c0f83c640 1 -- start start 2024-09-19T08:39:17.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.223+0000 7f7c0f83c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c0806c1d0 0x7f7c08151a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:17.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.223+0000 7f7c0f83c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c080cf900 0x7f7c081581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:17.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.223+0000 7f7c0f83c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c0815c550 0x7f7c0814e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:17.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.223+0000 7f7c0f83c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7c08169230 con 0x7f7c0815c550 2024-09-19T08:39:17.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.223+0000 7f7c0f83c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0815cb40 con 0x7f7c0806c1d0 2024-09-19T08:39:17.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.223+0000 7f7c0f83c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7c081693b0 con 0x7f7c080cf900 2024-09-19T08:39:17.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.223+0000 7f7c0ddd9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c0815c550 0x7f7c0814e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:17.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.223+0000 7f7c0ddd9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c0815c550 0x7f7c0814e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36590/0 (socket says 172.21.15.98:36590) 2024-09-19T08:39:17.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.223+0000 7f7c0ddd9640 1 -- 172.21.15.98:0/1038365684 learned_addr learned my addr 172.21.15.98:0/1038365684 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:17.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.223+0000 7f7c0d5d8640 1 --2- 172.21.15.98:0/1038365684 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c0806c1d0 0x7f7c08151a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:17.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.223+0000 7f7c0cdd7640 1 --2- 172.21.15.98:0/1038365684 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c080cf900 0x7f7c081581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:17.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.223+0000 7f7c0ddd9640 1 -- 172.21.15.98:0/1038365684 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c080cf900 msgr2=0x7f7c081581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:17.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.223+0000 7f7c0ddd9640 1 --2- 172.21.15.98:0/1038365684 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c080cf900 0x7f7c081581d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:17.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.223+0000 7f7c0ddd9640 1 -- 172.21.15.98:0/1038365684 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c0806c1d0 msgr2=0x7f7c08151a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:17.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.223+0000 7f7c0ddd9640 1 --2- 172.21.15.98:0/1038365684 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c0806c1d0 0x7f7c08151a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:17.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.223+0000 7f7c0ddd9640 1 -- 172.21.15.98:0/1038365684 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c08158c90 con 0x7f7c0815c550 2024-09-19T08:39:17.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.223+0000 7f7c0cdd7640 1 --2- 172.21.15.98:0/1038365684 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c080cf900 0x7f7c081581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:17.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.223+0000 7f7c0ddd9640 1 --2- 172.21.15.98:0/1038365684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c0815c550 0x7f7c0814e150 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7c0000c770 tx=0x7f7c0000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:17.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.223+0000 7f7bfdffb640 1 -- 172.21.15.98:0/1038365684 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c00003b80 con 0x7f7c0815c550 2024-09-19T08:39:17.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.223+0000 7f7bfdffb640 1 -- 172.21.15.98:0/1038365684 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c00003d20 con 0x7f7c0815c550 2024-09-19T08:39:17.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.223+0000 7f7bfdffb640 1 -- 172.21.15.98:0/1038365684 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0000aa90 con 0x7f7c0815c550 2024-09-19T08:39:17.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.223+0000 7f7c0f83c640 1 -- 172.21.15.98:0/1038365684 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c08158f70 con 0x7f7c0815c550 2024-09-19T08:39:17.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.223+0000 7f7c0f83c640 1 -- 172.21.15.98:0/1038365684 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c08149da0 con 0x7f7c0815c550 2024-09-19T08:39:17.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.227+0000 7f7c0f83c640 1 -- 172.21.15.98:0/1038365684 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c0814d740 con 0x7f7c0815c550 2024-09-19T08:39:17.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.227+0000 7f7bfdffb640 1 -- 172.21.15.98:0/1038365684 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7c0000ac30 con 0x7f7c0815c550 2024-09-19T08:39:17.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.227+0000 7f7bfdffb640 1 --2- 172.21.15.98:0/1038365684 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7bec0678c0 0x7f7bec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:17.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.227+0000 7f7bfdffb640 1 -- 172.21.15.98:0/1038365684 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f7c00085cb0 con 0x7f7c0815c550 2024-09-19T08:39:17.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.227+0000 7f7c0d5d8640 1 --2- 172.21.15.98:0/1038365684 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7bec0678c0 0x7f7bec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:17.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.227+0000 7f7c0d5d8640 1 --2- 172.21.15.98:0/1038365684 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7bec0678c0 0x7f7bec069d80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7c04009a50 tx=0x7f7c04008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:17.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.231+0000 7f7bfdffb640 1 -- 172.21.15.98:0/1038365684 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f7c00056750 con 0x7f7c0815c550 2024-09-19T08:39:17.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.423+0000 7f7c0f83c640 1 -- 172.21.15.98:0/1038365684 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7c080cf3c0 con 0x7f7c0815c550 2024-09-19T08:39:17.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.423+0000 7f7bfdffb640 1 -- 172.21.15.98:0/1038365684 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) ==== 53+0+73 (secure 0 0 0) 0x7f7c00056050 con 0x7f7c0815c550 2024-09-19T08:39:17.428 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T08:39:17.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.427+0000 7f7c0f83c640 1 -- 172.21.15.98:0/1038365684 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7bec0678c0 msgr2=0x7f7bec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:17.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.427+0000 7f7c0f83c640 1 --2- 172.21.15.98:0/1038365684 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7bec0678c0 0x7f7bec069d80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7c04009a50 tx=0x7f7c04008040 comp rx=0 tx=0).stop 2024-09-19T08:39:17.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.427+0000 7f7c0f83c640 1 -- 172.21.15.98:0/1038365684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c0815c550 msgr2=0x7f7c0814e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:17.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.427+0000 7f7c0f83c640 1 --2- 172.21.15.98:0/1038365684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c0815c550 0x7f7c0814e150 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7c0000c770 tx=0x7f7c0000cc30 comp rx=0 tx=0).stop 2024-09-19T08:39:17.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.427+0000 7f7c0f83c640 1 -- 172.21.15.98:0/1038365684 shutdown_connections 2024-09-19T08:39:17.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.427+0000 7f7c0f83c640 1 --2- 172.21.15.98:0/1038365684 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7bec0678c0 0x7f7bec069d80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:17.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.427+0000 7f7c0f83c640 1 --2- 172.21.15.98:0/1038365684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c0815c550 0x7f7c0814e150 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:17.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.427+0000 7f7c0f83c640 1 --2- 172.21.15.98:0/1038365684 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c080cf900 0x7f7c081581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:17.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.427+0000 7f7c0f83c640 1 --2- 172.21.15.98:0/1038365684 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c0806c1d0 0x7f7c08151a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:17.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.427+0000 7f7c0f83c640 1 -- 172.21.15.98:0/1038365684 >> 172.21.15.98:0/1038365684 conn(0x7f7c080b2010 msgr2=0x7f7c08156f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:17.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.427+0000 7f7c0f83c640 1 -- 172.21.15.98:0/1038365684 shutdown_connections 2024-09-19T08:39:17.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:17.427+0000 7f7c0f83c640 1 -- 172.21.15.98:0/1038365684 wait complete. 2024-09-19T08:39:17.445 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T08:39:17.445 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T08:39:17.445 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2024-09-19T08:39:17.445 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-19T08:39:17.445 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-19T08:39:17.445 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T08:39:17.445 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T08:39:17.445 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T08:39:17.445 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T08:39:17.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:17.446 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2024-09-19T08:39:17.452 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T08:39:17.452 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T08:39:17.452 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T08:39:17.453 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:39:17.453 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-09-19T08:39:17.453 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:39:17.453 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:39:17.453 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:39:17.453 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2024-09-19T08:39:17.453 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:39:17.453 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:39:17.453 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2024-09-19T08:39:17.453 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:39:17.453 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:39:17.453 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T08:39:17.453 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:39:17.454 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T08:39:17.454 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 42462sec preferred_lft 42462sec 2024-09-19T08:39:17.454 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::ec4:7aff:fe88:7f59/64 scope link 2024-09-19T08:39:17.454 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:39:17.454 INFO:teuthology.orchestra.run.smithi139.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T08:39:17.454 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 02:42:74:ed:e4:60 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:39:17.454 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T08:39:17.454 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:39:17.454 INFO:teuthology.orchestra.run.smithi139.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T08:39:17.454 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-19T08:39:17.454 INFO:teuthology.orchestra.run.smithi139.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T08:39:17.454 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:39:17.454 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-19T08:39:17.454 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:39:17.455 INFO:teuthology.orchestra.run.smithi139.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:39:17.455 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:39:17.455 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-19T08:39:17.455 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:39:17.455 INFO:teuthology.orchestra.run.smithi139.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:39:17.455 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:39:17.455 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-19T08:39:17.455 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:39:17.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:17.455 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T08:39:17.506 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-09-19T08:39:17.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:17.508 DEBUG:teuthology.orchestra.run.smithi139:> route 2024-09-19T08:39:17.560 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2024-09-19T08:39:17.560 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T08:39:17.561 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T08:39:17.561 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T08:39:17.561 INFO:teuthology.orchestra.run.smithi139.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T08:39:17.561 INFO:teuthology.orchestra.run.smithi139.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T08:39:17.561 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T08:39:17.561 INFO:teuthology.orchestra.run.smithi139.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:39:17.561 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:39:17.561 INFO:teuthology.orchestra.run.smithi139.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:39:17.561 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T08:39:17.561 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:39:17.562 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-09-19T08:39:17.562 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T08:39:17.562 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T08:39:17.562 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T08:39:17.562 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-09-19T08:39:17.649 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:39:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:39:17.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:39:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:39:17.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:17.684 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-09-19T08:39:17.690 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-19T08:39:17.690 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-19T08:39:17.690 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T08:39:17.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:17.690 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.0 up 2024-09-19T08:39:17.743 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-19T08:39:17.743 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:39:17.743 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-19T08:39:17.793 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-19T08:39:17.793 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:39:17.793 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-19T08:39:17.839 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:39:17.897 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:39:17.945 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:39:17.945 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:39:17.945 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:39:17.945 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:39:17.945 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:39:17.945 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:39:17.945 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:39:17.945 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:39:17.946 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:39:17.946 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:39:17.946 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:39:17.946 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:39:17.946 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:39:17.946 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:39:17.946 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:39:17.946 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:39:17.946 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:39:17.946 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:39:17.946 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:39:17.946 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:39:17.947 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:39:17.947 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:39:17.947 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:39:17.947 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:39:17.947 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:39:17.947 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:39:17.947 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:39:17.947 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:39:17.947 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:39:17.947 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:39:17.947 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:39:17.947 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:39:17.947 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:39:17.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:17.949 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:39:17.996 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T08:39:17.996 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-19T08:39:18.041 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:39:18.055 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:39:18.104 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:39:18.104 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:39:18.104 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:39:18.104 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:39:18.104 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:39:18.104 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:39:18.105 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:39:18.105 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:39:18.105 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:39:18.105 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:39:18.105 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:39:18.105 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:39:18.105 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:39:18.105 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:39:18.105 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:39:18.105 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:39:18.105 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:39:18.105 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:39:18.105 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:39:18.105 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:39:18.106 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:39:18.106 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:39:18.106 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:39:18.106 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:39:18.106 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:39:18.106 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:39:18.106 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:39:18.106 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:39:18.106 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:39:18.106 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:39:18.106 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:39:18.106 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:39:18.106 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:39:18.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:18.107 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:39:18.118 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:2024-09-19T08:39:18.118+0000 7fc245526580 -1 init, newargv = 0x5555af246020 newargc=15 2024-09-19T08:39:18.119 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[35255]: starting ceph client 2024-09-19T08:39:18.126 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[35255]: starting fuse 2024-09-19T08:39:19.224 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:39:19.241 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:39:19.289 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:39:19.289 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:39:19.289 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:39:19.289 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:39:19.289 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:39:19.289 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:39:19.289 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:39:19.290 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:39:19.290 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:39:19.290 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:39:19.290 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:39:19.290 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:39:19.290 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:39:19.290 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:39:19.290 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:39:19.290 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:39:19.290 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:39:19.290 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:39:19.290 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:39:19.290 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:39:19.290 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:39:19.291 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:39:19.291 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:39:19.291 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:39:19.291 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:39:19.291 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:39:19.291 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:39:19.291 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:39:19.291 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:39:19.291 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:39:19.291 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:39:19.291 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:39:19.291 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:39:19.291 INFO:teuthology.orchestra.run.smithi139.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-19T08:39:19.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:19.292 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:39:19.344 INFO:teuthology.orchestra.run.smithi139.stdout:45 2024-09-19T08:39:19.344 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-19T08:39:19.344 DEBUG:teuthology.orchestra.run.smithi139:> sudo stdin-killer -- python3 -c ' 2024-09-19T08:39:19.344 DEBUG:teuthology.orchestra.run.smithi139:> import glob 2024-09-19T08:39:19.345 DEBUG:teuthology.orchestra.run.smithi139:> import re 2024-09-19T08:39:19.345 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-09-19T08:39:19.345 DEBUG:teuthology.orchestra.run.smithi139:> import subprocess 2024-09-19T08:39:19.345 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:39:19.345 DEBUG:teuthology.orchestra.run.smithi139:> def _find_admin_socket(client_name): 2024-09-19T08:39:19.345 DEBUG:teuthology.orchestra.run.smithi139:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-19T08:39:19.345 DEBUG:teuthology.orchestra.run.smithi139:> files = glob.glob(asok_path) 2024-09-19T08:39:19.345 DEBUG:teuthology.orchestra.run.smithi139:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-19T08:39:19.345 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:39:19.345 DEBUG:teuthology.orchestra.run.smithi139:> # Given a non-glob path, it better be there 2024-09-19T08:39:19.348 DEBUG:teuthology.orchestra.run.smithi139:> if "*" not in asok_path: 2024-09-19T08:39:19.348 DEBUG:teuthology.orchestra.run.smithi139:> assert(len(files) == 1) 2024-09-19T08:39:19.348 DEBUG:teuthology.orchestra.run.smithi139:> return files[0] 2024-09-19T08:39:19.348 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:39:19.348 DEBUG:teuthology.orchestra.run.smithi139:> for f in files: 2024-09-19T08:39:19.348 DEBUG:teuthology.orchestra.run.smithi139:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T08:39:19.349 DEBUG:teuthology.orchestra.run.smithi139:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T08:39:19.349 DEBUG:teuthology.orchestra.run.smithi139:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T08:39:19.349 DEBUG:teuthology.orchestra.run.smithi139:> contents = proc_f.read() 2024-09-19T08:39:19.349 DEBUG:teuthology.orchestra.run.smithi139:> if mountpoint in contents: 2024-09-19T08:39:19.349 DEBUG:teuthology.orchestra.run.smithi139:> return f 2024-09-19T08:39:19.349 DEBUG:teuthology.orchestra.run.smithi139:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T08:39:19.349 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:39:19.349 DEBUG:teuthology.orchestra.run.smithi139:> print(_find_admin_socket("client.1")) 2024-09-19T08:39:19.349 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-09-19T08:39:19.469 INFO:teuthology.orchestra.run.smithi139.stdout:/var/run/ceph/ceph-client.1.35255.asok 2024-09-19T08:39:19.473 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:39:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:39:19.481 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.35255.asok 2024-09-19T08:39:19.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:19.482 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.35255.asok status 2024-09-19T08:39:19.636 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-19T08:39:19.636 INFO:teuthology.orchestra.run.smithi139.stdout: "metadata": { 2024-09-19T08:39:19.636 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T08:39:19.637 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T08:39:19.637 INFO:teuthology.orchestra.run.smithi139.stdout: "entity_id": "1", 2024-09-19T08:39:19.637 INFO:teuthology.orchestra.run.smithi139.stdout: "hostname": "smithi139", 2024-09-19T08:39:19.637 INFO:teuthology.orchestra.run.smithi139.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-19T08:39:19.637 INFO:teuthology.orchestra.run.smithi139.stdout: "pid": "35255", 2024-09-19T08:39:19.637 INFO:teuthology.orchestra.run.smithi139.stdout: "root": "/" 2024-09-19T08:39:19.637 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:39:19.637 INFO:teuthology.orchestra.run.smithi139.stdout: "dentry_count": 0, 2024-09-19T08:39:19.637 INFO:teuthology.orchestra.run.smithi139.stdout: "dentry_pinned_count": 0, 2024-09-19T08:39:19.637 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 4574, 2024-09-19T08:39:19.637 INFO:teuthology.orchestra.run.smithi139.stdout: "inst": { 2024-09-19T08:39:19.637 INFO:teuthology.orchestra.run.smithi139.stdout: "name": { 2024-09-19T08:39:19.638 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "client", 2024-09-19T08:39:19.638 INFO:teuthology.orchestra.run.smithi139.stdout: "num": 4574 2024-09-19T08:39:19.638 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:39:19.638 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": { 2024-09-19T08:39:19.638 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-19T08:39:19.638 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:39:19.638 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2689638534 2024-09-19T08:39:19.638 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-19T08:39:19.638 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:39:19.638 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": { 2024-09-19T08:39:19.638 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-19T08:39:19.638 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:39:19.638 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2689638534 2024-09-19T08:39:19.638 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:39:19.639 INFO:teuthology.orchestra.run.smithi139.stdout: "inst_str": "client.4574 192.168.0.1:0/2689638534", 2024-09-19T08:39:19.639 INFO:teuthology.orchestra.run.smithi139.stdout: "addr_str": "192.168.0.1:0/2689638534", 2024-09-19T08:39:19.639 INFO:teuthology.orchestra.run.smithi139.stdout: "inode_count": 1, 2024-09-19T08:39:19.639 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_epoch": 14, 2024-09-19T08:39:19.639 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_epoch": 32, 2024-09-19T08:39:19.639 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_epoch_barrier": 0, 2024-09-19T08:39:19.639 INFO:teuthology.orchestra.run.smithi139.stdout: "blocklisted": false, 2024-09-19T08:39:19.639 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs" 2024-09-19T08:39:19.639 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-19T08:39:19.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:19.650 DEBUG:teuthology.orchestra.run.smithi139:> stat --file-system '--printf=%T 2024-09-19T08:39:19.650 DEBUG:teuthology.orchestra.run.smithi139:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-19T08:39:19.656 INFO:teuthology.orchestra.run.smithi139.stdout:fuseblk 2024-09-19T08:39:19.657 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-19T08:39:19.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:19.657 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-19T08:39:19.714 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-19T08:39:19.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.851+0000 7efca4ba6640 1 Processor -- start 2024-09-19T08:39:19.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.851+0000 7efca4ba6640 1 -- start start 2024-09-19T08:39:19.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.851+0000 7efca4ba6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7efca00cf700 0x7efca015ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:19.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.851+0000 7efca4ba6640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efca01608c0 con 0x7efca015c290 2024-09-19T08:39:19.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.851+0000 7efca4ba6640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efca0160ba0 con 0x7efca006d4c0 2024-09-19T08:39:19.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.851+0000 7efca4ba6640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7efca0160e80 con 0x7efca00cf700 2024-09-19T08:39:19.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.851+0000 7efc9e59c640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7efca015c290 0x7efca00cd9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:46284/0 (socket says 172.21.15.98:46284) 2024-09-19T08:39:19.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.851+0000 7efc9e59c640 1 -- 172.21.15.98:0/3845909983 learned_addr learned my addr 172.21.15.98:0/3845909983 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:19.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.851+0000 7efc9d59a640 1 -- 172.21.15.98:0/3845909983 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2330832643 0 0) 0x7efca01608c0 con 0x7efca015c290 2024-09-19T08:39:19.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.851+0000 7efc9d59a640 1 -- 172.21.15.98:0/3845909983 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc88003530 con 0x7efca015c290 2024-09-19T08:39:19.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.851+0000 7efc9dd9b640 1 --2- 172.21.15.98:0/3845909983 >> v2:172.21.15.139:3300/0 conn(0x7efca00cf700 0x7efca015ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:19.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.851+0000 7efc9d59a640 1 -- 172.21.15.98:0/3845909983 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1004881726 0 0) 0x7efc88003530 con 0x7efca015c290 2024-09-19T08:39:19.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.851+0000 7efc9d59a640 1 -- 172.21.15.98:0/3845909983 >> v2:172.21.15.139:3300/0 conn(0x7efca00cf700 msgr2=0x7efca015ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:19.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.851+0000 7efc9d59a640 1 --2- 172.21.15.98:0/3845909983 >> v2:172.21.15.139:3300/0 conn(0x7efca00cf700 0x7efca015ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:19.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.851+0000 7efc9d59a640 1 -- 172.21.15.98:0/3845909983 >> v1:172.21.15.139:6789/0 conn(0x7efca006d4c0 legacy=0x7efca006d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:19.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.851+0000 7efc9d59a640 1 -- 172.21.15.98:0/3845909983 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca00cd1a0 con 0x7efca015c290 2024-09-19T08:39:19.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.851+0000 7efc9dd9b640 1 --2- 172.21.15.98:0/3845909983 >> v2:172.21.15.139:3300/0 conn(0x7efca00cf700 0x7efca015ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:39:19.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.851+0000 7efc9d59a640 1 -- 172.21.15.98:0/3845909983 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7efc90001680 con 0x7efca015c290 2024-09-19T08:39:19.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.851+0000 7efc9d59a640 1 -- 172.21.15.98:0/3845909983 >> v1:172.21.15.98:6789/0 conn(0x7efca015c290 legacy=0x7efca00cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:19.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.851+0000 7efc9d59a640 1 --2- 172.21.15.98:0/3845909983 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc880039c0 0x7efc88003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:19.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.851+0000 7efc9d59a640 1 --2- 172.21.15.98:0/3845909983 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc88004810 0x7efc88004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:19.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.851+0000 7efc9d59a640 1 --2- 172.21.15.98:0/3845909983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc880051d0 0x7efc8800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:19.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.851+0000 7efc9d59a640 1 -- 172.21.15.98:0/3845909983 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efc8800c140 con 0x7efc880051d0 2024-09-19T08:39:19.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.851+0000 7efc9d59a640 1 -- 172.21.15.98:0/3845909983 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efc8800c2f0 con 0x7efc880039c0 2024-09-19T08:39:19.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.851+0000 7efc9d59a640 1 -- 172.21.15.98:0/3845909983 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7efc8800c4f0 con 0x7efc88004810 2024-09-19T08:39:19.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.851+0000 7efc9ed9d640 1 --2- 172.21.15.98:0/3845909983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc880051d0 0x7efc8800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:19.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.851+0000 7efc9e59c640 1 --2- 172.21.15.98:0/3845909983 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc880039c0 0x7efc88003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:19.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.851+0000 7efc9dd9b640 1 --2- 172.21.15.98:0/3845909983 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc88004810 0x7efc88004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:19.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.851+0000 7efc9ed9d640 1 -- 172.21.15.98:0/3845909983 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc88004810 msgr2=0x7efc88004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:19.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.851+0000 7efc9ed9d640 1 --2- 172.21.15.98:0/3845909983 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc88004810 0x7efc88004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:19.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.851+0000 7efc9ed9d640 1 -- 172.21.15.98:0/3845909983 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc880039c0 msgr2=0x7efc88003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:19.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.851+0000 7efc9ed9d640 1 --2- 172.21.15.98:0/3845909983 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc880039c0 0x7efc88003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:19.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.851+0000 7efc9ed9d640 1 -- 172.21.15.98:0/3845909983 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc88003530 con 0x7efc880051d0 2024-09-19T08:39:19.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.855+0000 7efc9ed9d640 1 -- 172.21.15.98:0/3845909983 reap_dead start 2024-09-19T08:39:19.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.855+0000 7efc9ed9d640 1 --2- 172.21.15.98:0/3845909983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc880051d0 0x7efc8800b9d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7efc98001e70 tx=0x7efc9801c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05209e692cb25e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:19.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.855+0000 7efc9d59a640 1 -- 172.21.15.98:0/3845909983 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc98001a90 con 0x7efc880051d0 2024-09-19T08:39:19.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.855+0000 7efc9d59a640 1 -- 172.21.15.98:0/3845909983 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc98003bc0 con 0x7efc880051d0 2024-09-19T08:39:19.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.855+0000 7efc9d59a640 1 -- 172.21.15.98:0/3845909983 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc980050d0 con 0x7efc880051d0 2024-09-19T08:39:19.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.855+0000 7efca4ba6640 1 -- 172.21.15.98:0/3845909983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc880051d0 msgr2=0x7efc8800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:19.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.855+0000 7efca4ba6640 1 --2- 172.21.15.98:0/3845909983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc880051d0 0x7efc8800b9d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7efc98001e70 tx=0x7efc9801c780 comp rx=0 tx=0).stop 2024-09-19T08:39:19.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.855+0000 7efca4ba6640 1 -- 172.21.15.98:0/3845909983 shutdown_connections 2024-09-19T08:39:19.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.855+0000 7efca4ba6640 1 --2- 172.21.15.98:0/3845909983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc880051d0 0x7efc8800b9d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:19.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.855+0000 7efca4ba6640 1 -- 172.21.15.98:0/3845909983 >> 172.21.15.98:0/3845909983 conn(0x7efca006c090 msgr2=0x7efca006c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:19.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.855+0000 7efca4ba6640 1 -- 172.21.15.98:0/3845909983 shutdown_connections 2024-09-19T08:39:19.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.855+0000 7efca4ba6640 1 -- 172.21.15.98:0/3845909983 wait complete. 2024-09-19T08:39:19.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.859+0000 7efca4ba6640 1 Processor -- start 2024-09-19T08:39:19.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.859+0000 7efca4ba6640 1 -- start start 2024-09-19T08:39:19.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.859+0000 7efca4ba6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc880051d0 0x7efca0155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:19.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.859+0000 7efca4ba6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efca0158e40 0x7efca01592c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:19.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.859+0000 7efca4ba6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efca00ace20 0x7efca00ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:19.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.859+0000 7efca4ba6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efca006d8a0 con 0x7efca00ace20 2024-09-19T08:39:19.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.859+0000 7efca4ba6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efca00cd510 con 0x7efca0158e40 2024-09-19T08:39:19.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.859+0000 7efca4ba6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7efca006da20 con 0x7efc880051d0 2024-09-19T08:39:19.865 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.859+0000 7efc9ed9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efca00ace20 0x7efca00ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:19.865 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.859+0000 7efc9ed9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efca00ace20 0x7efca00ad2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35050/0 (socket says 172.21.15.98:35050) 2024-09-19T08:39:19.865 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.859+0000 7efc9ed9d640 1 -- 172.21.15.98:0/1751033425 learned_addr learned my addr 172.21.15.98:0/1751033425 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:19.865 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.859+0000 7efc9e59c640 1 --2- 172.21.15.98:0/1751033425 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc880051d0 0x7efca0155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:19.865 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.859+0000 7efc9dd9b640 1 --2- 172.21.15.98:0/1751033425 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efca0158e40 0x7efca01592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:19.865 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.859+0000 7efc9ed9d640 1 -- 172.21.15.98:0/1751033425 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc880051d0 msgr2=0x7efca0155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:19.865 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.859+0000 7efc9ed9d640 1 --2- 172.21.15.98:0/1751033425 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc880051d0 0x7efca0155e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:19.865 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.859+0000 7efc9ed9d640 1 -- 172.21.15.98:0/1751033425 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efca0158e40 msgr2=0x7efca01592c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:19.865 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.859+0000 7efc9ed9d640 1 --2- 172.21.15.98:0/1751033425 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efca0158e40 0x7efca01592c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:19.865 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.859+0000 7efc9ed9d640 1 -- 172.21.15.98:0/1751033425 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca01598c0 con 0x7efca00ace20 2024-09-19T08:39:19.865 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.859+0000 7efc9dd9b640 1 --2- 172.21.15.98:0/1751033425 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efca0158e40 0x7efca01592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:39:19.865 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.859+0000 7efc9ed9d640 1 --2- 172.21.15.98:0/1751033425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efca00ace20 0x7efca00ad2a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efc98098080 tx=0x7efc88005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:19.865 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.859+0000 7efc8effd640 1 -- 172.21.15.98:0/1751033425 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc98007dc0 con 0x7efca00ace20 2024-09-19T08:39:19.865 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.859+0000 7efc8effd640 1 -- 172.21.15.98:0/1751033425 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc980af070 con 0x7efca00ace20 2024-09-19T08:39:19.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.859+0000 7efca4ba6640 1 -- 172.21.15.98:0/1751033425 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efca0159ba0 con 0x7efca00ace20 2024-09-19T08:39:19.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.859+0000 7efc8effd640 1 -- 172.21.15.98:0/1751033425 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc98001a90 con 0x7efca00ace20 2024-09-19T08:39:19.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.859+0000 7efca4ba6640 1 -- 172.21.15.98:0/1751033425 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efca015a160 con 0x7efca00ace20 2024-09-19T08:39:19.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.859+0000 7efca4ba6640 1 -- 172.21.15.98:0/1751033425 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efca00cd9a0 con 0x7efca00ace20 2024-09-19T08:39:19.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.859+0000 7efc8effd640 1 -- 172.21.15.98:0/1751033425 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7efc980b1030 con 0x7efca00ace20 2024-09-19T08:39:19.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.863+0000 7efc8effd640 1 --2- 172.21.15.98:0/1751033425 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7efc800678c0 0x7efc80069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:19.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.863+0000 7efc8effd640 1 -- 172.21.15.98:0/1751033425 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7efc9811eb20 con 0x7efca00ace20 2024-09-19T08:39:19.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.863+0000 7efc9e59c640 1 --2- 172.21.15.98:0/1751033425 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7efc800678c0 0x7efc80069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:19.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.863+0000 7efc9e59c640 1 --2- 172.21.15.98:0/1751033425 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7efc800678c0 0x7efc80069d80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7efc90000c10 tx=0x7efc90001cf0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:19.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:19.867+0000 7efc8effd640 1 -- 172.21.15.98:0/1751033425 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7efc98016490 con 0x7efca00ace20 2024-09-19T08:39:20.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.059+0000 7efca4ba6640 1 -- 172.21.15.98:0/1751033425 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7efca015dc20 con 0x7efca00ace20 2024-09-19T08:39:20.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.111+0000 7efc8effd640 1 -- 172.21.15.98:0/1751033425 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v15) ==== 136+0+0 (secure 0 0 0) 0x7efc980ef690 con 0x7efca00ace20 2024-09-19T08:39:20.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.115+0000 7efca4ba6640 1 -- 172.21.15.98:0/1751033425 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7efc800678c0 msgr2=0x7efc80069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:20.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.115+0000 7efca4ba6640 1 --2- 172.21.15.98:0/1751033425 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7efc800678c0 0x7efc80069d80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7efc90000c10 tx=0x7efc90001cf0 comp rx=0 tx=0).stop 2024-09-19T08:39:20.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.115+0000 7efca4ba6640 1 -- 172.21.15.98:0/1751033425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efca00ace20 msgr2=0x7efca00ad2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:20.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.115+0000 7efca4ba6640 1 --2- 172.21.15.98:0/1751033425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efca00ace20 0x7efca00ad2a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efc98098080 tx=0x7efc88005840 comp rx=0 tx=0).stop 2024-09-19T08:39:20.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.115+0000 7efca4ba6640 1 -- 172.21.15.98:0/1751033425 shutdown_connections 2024-09-19T08:39:20.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.115+0000 7efca4ba6640 1 --2- 172.21.15.98:0/1751033425 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7efc800678c0 0x7efc80069d80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:20.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.115+0000 7efca4ba6640 1 --2- 172.21.15.98:0/1751033425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efca00ace20 0x7efca00ad2a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:20.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.115+0000 7efca4ba6640 1 --2- 172.21.15.98:0/1751033425 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efca0158e40 0x7efca01592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:20.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.115+0000 7efca4ba6640 1 --2- 172.21.15.98:0/1751033425 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc880051d0 0x7efca0155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:20.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.115+0000 7efca4ba6640 1 -- 172.21.15.98:0/1751033425 >> 172.21.15.98:0/1751033425 conn(0x7efca006c090 msgr2=0x7efca00b2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:20.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.115+0000 7efca4ba6640 1 -- 172.21.15.98:0/1751033425 shutdown_connections 2024-09-19T08:39:20.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.115+0000 7efca4ba6640 1 -- 172.21.15.98:0/1751033425 wait complete. 2024-09-19T08:39:20.137 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-19T08:39:20.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.255+0000 7f3d7919b640 1 Processor -- start 2024-09-19T08:39:20.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.255+0000 7f3d7919b640 1 -- start start 2024-09-19T08:39:20.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.255+0000 7f3d7919b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3d7415c550 0x7f3d7406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:20.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.255+0000 7f3d7919b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d7406c1d0 0x7f3d740cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:20.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.255+0000 7f3d7919b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3d740cf900 0x7f3d740ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:20.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.255+0000 7f3d7919b640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f3d7406c610 con 0x7f3d7415c550 2024-09-19T08:39:20.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.255+0000 7f3d7919b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f3d7406d310 con 0x7f3d740cf900 2024-09-19T08:39:20.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.255+0000 7f3d7919b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3d7406d5f0 con 0x7f3d7406c1d0 2024-09-19T08:39:20.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.255+0000 7f3d72d9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3d7415c550 0x7f3d7406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:20.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.255+0000 7f3d72d9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3d7415c550 0x7f3d7406bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35066/0 (socket says 172.21.15.98:35066) 2024-09-19T08:39:20.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.255+0000 7f3d72d9d640 1 -- 172.21.15.98:0/2068090933 learned_addr learned my addr 172.21.15.98:0/2068090933 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:20.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.255+0000 7f3d7359e640 1 --2- 172.21.15.98:0/2068090933 >> v2:172.21.15.139:3300/0 conn(0x7f3d740cf900 0x7f3d740ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:20.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.255+0000 7f3d72d9d640 1 -- 172.21.15.98:0/2068090933 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d7406c1d0 msgr2=0x7f3d740cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:20.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.255+0000 7f3d7259c640 1 --2- 172.21.15.98:0/2068090933 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d7406c1d0 0x7f3d740cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:20.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.255+0000 7f3d72d9d640 1 --2- 172.21.15.98:0/2068090933 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d7406c1d0 0x7f3d740cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:20.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.255+0000 7f3d72d9d640 1 -- 172.21.15.98:0/2068090933 >> v2:172.21.15.139:3300/0 conn(0x7f3d740cf900 msgr2=0x7f3d740ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:20.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.255+0000 7f3d72d9d640 1 --2- 172.21.15.98:0/2068090933 >> v2:172.21.15.139:3300/0 conn(0x7f3d740cf900 0x7f3d740ce1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:20.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.255+0000 7f3d72d9d640 1 -- 172.21.15.98:0/2068090933 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d7406d8d0 con 0x7f3d7415c550 2024-09-19T08:39:20.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.255+0000 7f3d7359e640 1 --2- 172.21.15.98:0/2068090933 >> v2:172.21.15.139:3300/0 conn(0x7f3d740cf900 0x7f3d740ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:20.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.255+0000 7f3d7259c640 1 --2- 172.21.15.98:0/2068090933 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d7406c1d0 0x7f3d740cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:20.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.259+0000 7f3d72d9d640 1 --2- 172.21.15.98:0/2068090933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d7415c550 0x7f3d7406bc90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3d74074090 tx=0x7f3d6401c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e389a4ef353caf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:20.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.259+0000 7f3d71d9b640 1 -- 172.21.15.98:0/2068090933 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d640b0070 con 0x7f3d7415c550 2024-09-19T08:39:20.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.259+0000 7f3d71d9b640 1 -- 172.21.15.98:0/2068090933 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d640ab600 con 0x7f3d7415c550 2024-09-19T08:39:20.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.259+0000 7f3d71d9b640 1 -- 172.21.15.98:0/2068090933 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d640ab9d0 con 0x7f3d7415c550 2024-09-19T08:39:20.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.259+0000 7f3d7919b640 1 -- 172.21.15.98:0/2068090933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d7415c550 msgr2=0x7f3d7406bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:20.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.259+0000 7f3d7919b640 1 --2- 172.21.15.98:0/2068090933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d7415c550 0x7f3d7406bc90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3d74074090 tx=0x7f3d6401c260 comp rx=0 tx=0).stop 2024-09-19T08:39:20.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.259+0000 7f3d7919b640 1 -- 172.21.15.98:0/2068090933 shutdown_connections 2024-09-19T08:39:20.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.259+0000 7f3d7919b640 1 --2- 172.21.15.98:0/2068090933 >> v2:172.21.15.139:3300/0 conn(0x7f3d740cf900 0x7f3d740ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:20.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.259+0000 7f3d7919b640 1 --2- 172.21.15.98:0/2068090933 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d7406c1d0 0x7f3d740cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:20.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.259+0000 7f3d7919b640 1 --2- 172.21.15.98:0/2068090933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d7415c550 0x7f3d7406bc90 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:20.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.259+0000 7f3d7919b640 1 -- 172.21.15.98:0/2068090933 >> 172.21.15.98:0/2068090933 conn(0x7f3d740b2010 msgr2=0x7f3d740b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:20.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.259+0000 7f3d7919b640 1 -- 172.21.15.98:0/2068090933 shutdown_connections 2024-09-19T08:39:20.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.259+0000 7f3d7919b640 1 -- 172.21.15.98:0/2068090933 wait complete. 2024-09-19T08:39:20.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.259+0000 7f3d7919b640 1 Processor -- start 2024-09-19T08:39:20.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.259+0000 7f3d7919b640 1 -- start start 2024-09-19T08:39:20.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.259+0000 7f3d7919b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d7406c1d0 0x7f3d7413db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:20.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.259+0000 7f3d7919b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d740cf900 0x7f3d7414c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:20.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.259+0000 7f3d7919b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d7415c550 0x7f3d74156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:20.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.259+0000 7f3d7919b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3d740723a0 con 0x7f3d740cf900 2024-09-19T08:39:20.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.259+0000 7f3d7919b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7415cb40 con 0x7f3d7415c550 2024-09-19T08:39:20.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.259+0000 7f3d7919b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3d74072520 con 0x7f3d7406c1d0 2024-09-19T08:39:20.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.259+0000 7f3d7259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d740cf900 0x7f3d7414c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:20.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.259+0000 7f3d7259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d740cf900 0x7f3d7414c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35068/0 (socket says 172.21.15.98:35068) 2024-09-19T08:39:20.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.259+0000 7f3d7259c640 1 -- 172.21.15.98:0/3202490637 learned_addr learned my addr 172.21.15.98:0/3202490637 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:20.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.259+0000 7f3d7359e640 1 --2- 172.21.15.98:0/3202490637 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d7415c550 0x7f3d74156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:20.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.259+0000 7f3d72d9d640 1 --2- 172.21.15.98:0/3202490637 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d7406c1d0 0x7f3d7413db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:20.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.259+0000 7f3d7259c640 1 -- 172.21.15.98:0/3202490637 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d7406c1d0 msgr2=0x7f3d7413db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:20.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.259+0000 7f3d7259c640 1 --2- 172.21.15.98:0/3202490637 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d7406c1d0 0x7f3d7413db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:20.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.259+0000 7f3d7259c640 1 -- 172.21.15.98:0/3202490637 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d7415c550 msgr2=0x7f3d74156cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:20.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.259+0000 7f3d7259c640 1 --2- 172.21.15.98:0/3202490637 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d7415c550 0x7f3d74156cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:20.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.259+0000 7f3d7259c640 1 -- 172.21.15.98:0/3202490637 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d741599c0 con 0x7f3d740cf900 2024-09-19T08:39:20.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.259+0000 7f3d7259c640 1 --2- 172.21.15.98:0/3202490637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d740cf900 0x7f3d7414c730 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3d6800c7f0 tx=0x7f3d6800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:20.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.263+0000 7f3d5b7fe640 1 -- 172.21.15.98:0/3202490637 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d68015070 con 0x7f3d740cf900 2024-09-19T08:39:20.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.263+0000 7f3d5b7fe640 1 -- 172.21.15.98:0/3202490637 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d68007a30 con 0x7f3d740cf900 2024-09-19T08:39:20.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.263+0000 7f3d7919b640 1 -- 172.21.15.98:0/3202490637 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d74159ca0 con 0x7f3d740cf900 2024-09-19T08:39:20.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.263+0000 7f3d5b7fe640 1 -- 172.21.15.98:0/3202490637 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d68007e00 con 0x7f3d740cf900 2024-09-19T08:39:20.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.263+0000 7f3d7919b640 1 -- 172.21.15.98:0/3202490637 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d7414dc30 con 0x7f3d740cf900 2024-09-19T08:39:20.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.263+0000 7f3d7919b640 1 -- 172.21.15.98:0/3202490637 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d740ce1a0 con 0x7f3d740cf900 2024-09-19T08:39:20.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.263+0000 7f3d5b7fe640 1 -- 172.21.15.98:0/3202490637 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3d68018070 con 0x7f3d740cf900 2024-09-19T08:39:20.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.263+0000 7f3d5b7fe640 1 --2- 172.21.15.98:0/3202490637 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3d600678c0 0x7f3d60069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:20.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.263+0000 7f3d5b7fe640 1 -- 172.21.15.98:0/3202490637 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f3d68085be0 con 0x7f3d740cf900 2024-09-19T08:39:20.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.263+0000 7f3d72d9d640 1 --2- 172.21.15.98:0/3202490637 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3d600678c0 0x7f3d60069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:20.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.263+0000 7f3d72d9d640 1 --2- 172.21.15.98:0/3202490637 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3d600678c0 0x7f3d60069d80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3d640ad000 tx=0x7f3d64008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:20.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.267+0000 7f3d5b7fe640 1 -- 172.21.15.98:0/3202490637 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3d68056710 con 0x7f3d740cf900 2024-09-19T08:39:20.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.447+0000 7f3d7919b640 1 -- 172.21.15.98:0/3202490637 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f3d7413e0a0 con 0x7f3d600678c0 2024-09-19T08:39:20.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.451+0000 7f3d5b7fe640 1 -- 172.21.15.98:0/3202490637 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f3d7413e0a0 con 0x7f3d600678c0 2024-09-19T08:39:20.453 INFO:teuthology.orchestra.run.smithi098.stdout:[ 2024-09-19T08:39:20.453 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:39:20.453 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "cephfs" 2024-09-19T08:39:20.453 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:39:20.453 INFO:teuthology.orchestra.run.smithi098.stdout:] 2024-09-19T08:39:20.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.451+0000 7f3d7919b640 1 -- 172.21.15.98:0/3202490637 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3d600678c0 msgr2=0x7f3d60069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:20.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.451+0000 7f3d7919b640 1 --2- 172.21.15.98:0/3202490637 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3d600678c0 0x7f3d60069d80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3d640ad000 tx=0x7f3d64008040 comp rx=0 tx=0).stop 2024-09-19T08:39:20.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.451+0000 7f3d7919b640 1 -- 172.21.15.98:0/3202490637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d740cf900 msgr2=0x7f3d7414c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:20.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.451+0000 7f3d7919b640 1 --2- 172.21.15.98:0/3202490637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d740cf900 0x7f3d7414c730 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3d6800c7f0 tx=0x7f3d6800ccb0 comp rx=0 tx=0).stop 2024-09-19T08:39:20.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.451+0000 7f3d7919b640 1 -- 172.21.15.98:0/3202490637 shutdown_connections 2024-09-19T08:39:20.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.451+0000 7f3d7919b640 1 --2- 172.21.15.98:0/3202490637 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3d600678c0 0x7f3d60069d80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:20.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.451+0000 7f3d7919b640 1 --2- 172.21.15.98:0/3202490637 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d7415c550 0x7f3d74156cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:20.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.451+0000 7f3d7919b640 1 --2- 172.21.15.98:0/3202490637 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d740cf900 0x7f3d7414c730 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:20.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.451+0000 7f3d7919b640 1 --2- 172.21.15.98:0/3202490637 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d7406c1d0 0x7f3d7413db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:20.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.451+0000 7f3d7919b640 1 -- 172.21.15.98:0/3202490637 >> 172.21.15.98:0/3202490637 conn(0x7f3d740b2010 msgr2=0x7f3d740afd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:20.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.451+0000 7f3d7919b640 1 -- 172.21.15.98:0/3202490637 shutdown_connections 2024-09-19T08:39:20.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.451+0000 7f3d7919b640 1 -- 172.21.15.98:0/3202490637 wait complete. 2024-09-19T08:39:20.471 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T08:39:20.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.587+0000 7fe92fd9f640 1 Processor -- start 2024-09-19T08:39:20.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.587+0000 7fe92fd9f640 1 -- start start 2024-09-19T08:39:20.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.587+0000 7fe92fd9f640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe928145b70 0x7fe92806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:20.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.587+0000 7fe92fd9f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe92806c1d0 0x7fe9280713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:20.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.587+0000 7fe92fd9f640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe92806c5f0 con 0x7fe9280718f0 2024-09-19T08:39:20.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.587+0000 7fe92fd9f640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fe928073530 con 0x7fe928145b70 2024-09-19T08:39:20.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.587+0000 7fe92fd9f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe928073810 con 0x7fe92806c1d0 2024-09-19T08:39:20.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.587+0000 7fe92db3b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe928145b70 0x7fe92806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:20.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.587+0000 7fe92db3b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe928145b70 0x7fe92806bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35082/0 (socket says 172.21.15.98:35082) 2024-09-19T08:39:20.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.587+0000 7fe92db3b640 1 -- 172.21.15.98:0/3685845092 learned_addr learned my addr 172.21.15.98:0/3685845092 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:20.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.587+0000 7fe92e33c640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fe9280718f0 0x7fe9280cef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:46296/0 (socket says 172.21.15.98:46296) 2024-09-19T08:39:20.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.587+0000 7fe92d33a640 1 --2- 172.21.15.98:0/3685845092 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe92806c1d0 0x7fe9280713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:20.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.587+0000 7fe92db3b640 1 -- 172.21.15.98:0/3685845092 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe92806c1d0 msgr2=0x7fe9280713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:20.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.587+0000 7fe92db3b640 1 --2- 172.21.15.98:0/3685845092 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe92806c1d0 0x7fe9280713b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:20.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.587+0000 7fe92db3b640 1 -- 172.21.15.98:0/3685845092 >> v1:172.21.15.98:6789/0 conn(0x7fe9280718f0 legacy=0x7fe9280cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:20.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.587+0000 7fe92db3b640 1 -- 172.21.15.98:0/3685845092 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe928073af0 con 0x7fe928145b70 2024-09-19T08:39:20.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.587+0000 7fe92db3b640 1 --2- 172.21.15.98:0/3685845092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe928145b70 0x7fe92806bc90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe924009870 tx=0x7fe92401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4334f037843b2214 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:20.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.591+0000 7fe92cb39640 1 -- 172.21.15.98:0/3685845092 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9240b0070 con 0x7fe928145b70 2024-09-19T08:39:20.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.591+0000 7fe92cb39640 1 -- 172.21.15.98:0/3685845092 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9240ab600 con 0x7fe928145b70 2024-09-19T08:39:20.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.591+0000 7fe92cb39640 1 -- 172.21.15.98:0/3685845092 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9240ab9d0 con 0x7fe928145b70 2024-09-19T08:39:20.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.591+0000 7fe92fd9f640 1 -- 172.21.15.98:0/3685845092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe928145b70 msgr2=0x7fe92806bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:20.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.591+0000 7fe92fd9f640 1 --2- 172.21.15.98:0/3685845092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe928145b70 0x7fe92806bc90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe924009870 tx=0x7fe92401c4d0 comp rx=0 tx=0).stop 2024-09-19T08:39:20.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.591+0000 7fe92fd9f640 1 -- 172.21.15.98:0/3685845092 shutdown_connections 2024-09-19T08:39:20.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.591+0000 7fe92fd9f640 1 --2- 172.21.15.98:0/3685845092 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe92806c1d0 0x7fe9280713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:20.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.591+0000 7fe92fd9f640 1 --2- 172.21.15.98:0/3685845092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe928145b70 0x7fe92806bc90 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:20.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.591+0000 7fe92fd9f640 1 -- 172.21.15.98:0/3685845092 >> 172.21.15.98:0/3685845092 conn(0x7fe9280b2010 msgr2=0x7fe9280b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:20.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.591+0000 7fe92fd9f640 1 -- 172.21.15.98:0/3685845092 shutdown_connections 2024-09-19T08:39:20.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.591+0000 7fe92fd9f640 1 -- 172.21.15.98:0/3685845092 wait complete. 2024-09-19T08:39:20.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.591+0000 7fe92fd9f640 1 Processor -- start 2024-09-19T08:39:20.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.591+0000 7fe92fd9f640 1 -- start start 2024-09-19T08:39:20.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.591+0000 7fe92fd9f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe92806c1d0 0x7fe92813be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:20.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.591+0000 7fe92fd9f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9280718f0 0x7fe9280aa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:20.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.591+0000 7fe92fd9f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe928145b70 0x7fe92815b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:20.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.591+0000 7fe92fd9f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe928169070 con 0x7fe928145b70 2024-09-19T08:39:20.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.591+0000 7fe92fd9f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe928071f20 con 0x7fe9280718f0 2024-09-19T08:39:20.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.591+0000 7fe92fd9f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe9281691f0 con 0x7fe92806c1d0 2024-09-19T08:39:20.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.591+0000 7fe92e33c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe928145b70 0x7fe92815b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:20.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.591+0000 7fe92e33c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe928145b70 0x7fe92815b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35094/0 (socket says 172.21.15.98:35094) 2024-09-19T08:39:20.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.591+0000 7fe92e33c640 1 -- 172.21.15.98:0/2602657274 learned_addr learned my addr 172.21.15.98:0/2602657274 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:20.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.591+0000 7fe92e33c640 1 -- 172.21.15.98:0/2602657274 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe92806c1d0 msgr2=0x7fe92813be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:20.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.591+0000 7fe92db3b640 1 --2- 172.21.15.98:0/2602657274 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe92806c1d0 0x7fe92813be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:20.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.591+0000 7fe92d33a640 1 --2- 172.21.15.98:0/2602657274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9280718f0 0x7fe9280aa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:20.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.591+0000 7fe92e33c640 1 --2- 172.21.15.98:0/2602657274 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe92806c1d0 0x7fe92813be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:20.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.591+0000 7fe92e33c640 1 -- 172.21.15.98:0/2602657274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9280718f0 msgr2=0x7fe9280aa4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:20.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.591+0000 7fe92e33c640 1 --2- 172.21.15.98:0/2602657274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9280718f0 0x7fe9280aa4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:20.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.591+0000 7fe92e33c640 1 -- 172.21.15.98:0/2602657274 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9280aae10 con 0x7fe928145b70 2024-09-19T08:39:20.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.591+0000 7fe92e33c640 1 --2- 172.21.15.98:0/2602657274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe928145b70 0x7fe92815b540 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe92000c740 tx=0x7fe92000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:20.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.591+0000 7fe91e7fc640 1 -- 172.21.15.98:0/2602657274 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe920017070 con 0x7fe928145b70 2024-09-19T08:39:20.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.591+0000 7fe91e7fc640 1 -- 172.21.15.98:0/2602657274 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe92000ce50 con 0x7fe928145b70 2024-09-19T08:39:20.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.591+0000 7fe92fd9f640 1 -- 172.21.15.98:0/2602657274 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9280ab0f0 con 0x7fe928145b70 2024-09-19T08:39:20.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.591+0000 7fe91e7fc640 1 -- 172.21.15.98:0/2602657274 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92000a860 con 0x7fe928145b70 2024-09-19T08:39:20.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.591+0000 7fe92fd9f640 1 -- 172.21.15.98:0/2602657274 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9280ab6b0 con 0x7fe928145b70 2024-09-19T08:39:20.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.595+0000 7fe92fd9f640 1 -- 172.21.15.98:0/2602657274 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9280713b0 con 0x7fe928145b70 2024-09-19T08:39:20.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.595+0000 7fe91e7fc640 1 -- 172.21.15.98:0/2602657274 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe92001a070 con 0x7fe928145b70 2024-09-19T08:39:20.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.595+0000 7fe91e7fc640 1 --2- 172.21.15.98:0/2602657274 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe90c0678f0 0x7fe90c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:20.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.595+0000 7fe91e7fc640 1 -- 172.21.15.98:0/2602657274 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fe920085290 con 0x7fe928145b70 2024-09-19T08:39:20.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.595+0000 7fe92db3b640 1 --2- 172.21.15.98:0/2602657274 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe90c0678f0 0x7fe90c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:20.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.595+0000 7fe92db3b640 1 --2- 172.21.15.98:0/2602657274 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe90c0678f0 0x7fe90c069db0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe9240ad000 tx=0x7fe924008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:20.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.599+0000 7fe91e7fc640 1 -- 172.21.15.98:0/2602657274 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe920055dd0 con 0x7fe928145b70 2024-09-19T08:39:20.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.759+0000 7fe92fd9f640 1 -- 172.21.15.98:0/2602657274 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe9280008e0 con 0x7fe928145b70 2024-09-19T08:39:20.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.759+0000 7fe91e7fc640 1 -- 172.21.15.98:0/2602657274 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7fe9200556d0 con 0x7fe928145b70 2024-09-19T08:39:20.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.763+0000 7fe92fd9f640 1 -- 172.21.15.98:0/2602657274 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe90c0678f0 msgr2=0x7fe90c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:20.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.763+0000 7fe92fd9f640 1 --2- 172.21.15.98:0/2602657274 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe90c0678f0 0x7fe90c069db0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe9240ad000 tx=0x7fe924008040 comp rx=0 tx=0).stop 2024-09-19T08:39:20.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.763+0000 7fe92fd9f640 1 -- 172.21.15.98:0/2602657274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe928145b70 msgr2=0x7fe92815b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:20.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.763+0000 7fe92fd9f640 1 --2- 172.21.15.98:0/2602657274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe928145b70 0x7fe92815b540 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe92000c740 tx=0x7fe92000cc00 comp rx=0 tx=0).stop 2024-09-19T08:39:20.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.763+0000 7fe92fd9f640 1 -- 172.21.15.98:0/2602657274 shutdown_connections 2024-09-19T08:39:20.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.763+0000 7fe92fd9f640 1 --2- 172.21.15.98:0/2602657274 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe90c0678f0 0x7fe90c069db0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:20.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.763+0000 7fe92fd9f640 1 --2- 172.21.15.98:0/2602657274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe928145b70 0x7fe92815b540 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:20.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.763+0000 7fe92fd9f640 1 --2- 172.21.15.98:0/2602657274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9280718f0 0x7fe9280aa4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:20.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.763+0000 7fe92fd9f640 1 --2- 172.21.15.98:0/2602657274 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe92806c1d0 0x7fe92813be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:20.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.763+0000 7fe92fd9f640 1 -- 172.21.15.98:0/2602657274 >> 172.21.15.98:0/2602657274 conn(0x7fe9280b2010 msgr2=0x7fe928141d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:20.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.763+0000 7fe92fd9f640 1 -- 172.21.15.98:0/2602657274 shutdown_connections 2024-09-19T08:39:20.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:20.763+0000 7fe92fd9f640 1 -- 172.21.15.98:0/2602657274 wait complete. 2024-09-19T08:39:20.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:20.783 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp66610/subvol44050) 2024-09-19T08:39:20.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:20.805 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-19T08:39:20.865 INFO:teuthology.orchestra.run.smithi098.stdout:cephfs_data 2024-09-19T08:39:20.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:20.866 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/subvol_grp66610/subvol44050) 2024-09-19T08:39:20.928 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:39:20.928 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/home/ubuntu/cephtest/mnt.0/./volumes/subvol_grp66610/subvol44050/.meta 2024-09-19T08:39:20.987 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol44050 subvol_grp66610 2024-09-19T08:39:21.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.151+0000 7f1219cf8640 1 Processor -- start 2024-09-19T08:39:21.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.151+0000 7f1219cf8640 1 -- start start 2024-09-19T08:39:21.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.151+0000 7f1219cf8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f121415e690 0x7f121406f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:21.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.151+0000 7f1219cf8640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f121406cf40 0x7f121406f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:21.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.151+0000 7f1219cf8640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12140d14b0 con 0x7f1214073490 2024-09-19T08:39:21.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.151+0000 7f1219cf8640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f1214073a00 con 0x7f121406cf40 2024-09-19T08:39:21.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.151+0000 7f1219cf8640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f12140d1750 con 0x7f121415e690 2024-09-19T08:39:21.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.151+0000 7f1212ffd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f1214073490 0x7f1214070790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:46310/0 (socket says 172.21.15.98:46310) 2024-09-19T08:39:21.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.151+0000 7f1212ffd640 1 -- 172.21.15.98:0/3692926619 learned_addr learned my addr 172.21.15.98:0/3692926619 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:21.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.151+0000 7f1213fff640 1 --2- 172.21.15.98:0/3692926619 >> v2:172.21.15.139:3300/0 conn(0x7f121406cf40 0x7f121406f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:21.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.151+0000 7f12127fc640 1 -- 172.21.15.98:0/3692926619 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2856155787 0 0) 0x7f12140d14b0 con 0x7f1214073490 2024-09-19T08:39:21.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.151+0000 7f12137fe640 1 --2- 172.21.15.98:0/3692926619 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f121415e690 0x7f121406f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:21.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.151+0000 7f12127fc640 1 -- 172.21.15.98:0/3692926619 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11fc003530 con 0x7f1214073490 2024-09-19T08:39:21.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.151+0000 7f12127fc640 1 -- 172.21.15.98:0/3692926619 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1626112726 0 0) 0x7f11fc003530 con 0x7f1214073490 2024-09-19T08:39:21.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.151+0000 7f12127fc640 1 -- 172.21.15.98:0/3692926619 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f121415e690 msgr2=0x7f121406f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:21.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.151+0000 7f12127fc640 1 --2- 172.21.15.98:0/3692926619 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f121415e690 0x7f121406f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:21.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.151+0000 7f12127fc640 1 -- 172.21.15.98:0/3692926619 >> v2:172.21.15.139:3300/0 conn(0x7f121406cf40 msgr2=0x7f121406f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:21.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.151+0000 7f12127fc640 1 --2- 172.21.15.98:0/3692926619 >> v2:172.21.15.139:3300/0 conn(0x7f121406cf40 0x7f121406f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:21.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.151+0000 7f12127fc640 1 -- 172.21.15.98:0/3692926619 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12140d1a30 con 0x7f1214073490 2024-09-19T08:39:21.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.151+0000 7f12127fc640 1 -- 172.21.15.98:0/3692926619 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f1208001680 con 0x7f1214073490 2024-09-19T08:39:21.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.151+0000 7f12127fc640 1 -- 172.21.15.98:0/3692926619 >> v1:172.21.15.98:6789/0 conn(0x7f1214073490 legacy=0x7f1214070790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:21.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.151+0000 7f12127fc640 1 --2- 172.21.15.98:0/3692926619 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11fc0039c0 0x7f11fc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:21.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.151+0000 7f12127fc640 1 --2- 172.21.15.98:0/3692926619 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f121415e690 0x7f11fc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:21.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.151+0000 7f12127fc640 1 --2- 172.21.15.98:0/3692926619 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11fc004d50 0x7f11fc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:21.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.151+0000 7f12127fc640 1 -- 172.21.15.98:0/3692926619 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1214073a00 con 0x7f11fc004d50 2024-09-19T08:39:21.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.151+0000 7f12127fc640 1 -- 172.21.15.98:0/3692926619 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f12140d1750 con 0x7f11fc0039c0 2024-09-19T08:39:21.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.151+0000 7f12127fc640 1 -- 172.21.15.98:0/3692926619 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f11fc00b9d0 con 0x7f121415e690 2024-09-19T08:39:21.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.151+0000 7f1213fff640 1 --2- 172.21.15.98:0/3692926619 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11fc004d50 0x7f11fc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:21.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.151+0000 7f1212ffd640 1 --2- 172.21.15.98:0/3692926619 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f121415e690 0x7f11fc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:21.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.151+0000 7f1213fff640 1 -- 172.21.15.98:0/3692926619 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f121415e690 msgr2=0x7f11fc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:21.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.151+0000 7f1213fff640 1 --2- 172.21.15.98:0/3692926619 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f121415e690 0x7f11fc004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:21.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.151+0000 7f1213fff640 1 -- 172.21.15.98:0/3692926619 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11fc0039c0 msgr2=0x7f11fc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:21.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.151+0000 7f1213fff640 1 --2- 172.21.15.98:0/3692926619 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11fc0039c0 0x7f11fc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:21.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.151+0000 7f1213fff640 1 -- 172.21.15.98:0/3692926619 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11fc003530 con 0x7f11fc004d50 2024-09-19T08:39:21.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.155+0000 7f1213fff640 1 --2- 172.21.15.98:0/3692926619 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11fc004d50 0x7f11fc0051d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f120c009f90 tx=0x7f120c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f91bf7f352101fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:21.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.155+0000 7f12127fc640 1 -- 172.21.15.98:0/3692926619 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c0b2070 con 0x7f11fc004d50 2024-09-19T08:39:21.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.155+0000 7f12127fc640 1 -- 172.21.15.98:0/3692926619 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f120c0a5c90 con 0x7f11fc004d50 2024-09-19T08:39:21.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.155+0000 7f12127fc640 1 -- 172.21.15.98:0/3692926619 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c009c10 con 0x7f11fc004d50 2024-09-19T08:39:21.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.155+0000 7f1219cf8640 1 -- 172.21.15.98:0/3692926619 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11fc004d50 msgr2=0x7f11fc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:21.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.155+0000 7f1219cf8640 1 --2- 172.21.15.98:0/3692926619 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11fc004d50 0x7f11fc0051d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f120c009f90 tx=0x7f120c01cc80 comp rx=0 tx=0).stop 2024-09-19T08:39:21.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.155+0000 7f1213fff640 1 -- 172.21.15.98:0/3692926619 reap_dead start 2024-09-19T08:39:21.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.155+0000 7f1219cf8640 1 -- 172.21.15.98:0/3692926619 shutdown_connections 2024-09-19T08:39:21.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.155+0000 7f1219cf8640 1 -- 172.21.15.98:0/3692926619 >> 172.21.15.98:0/3692926619 conn(0x7f12140cedf0 msgr2=0x7f12140cf200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:21.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.155+0000 7f1219cf8640 1 -- 172.21.15.98:0/3692926619 shutdown_connections 2024-09-19T08:39:21.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.155+0000 7f1219cf8640 1 -- 172.21.15.98:0/3692926619 wait complete. 2024-09-19T08:39:21.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.155+0000 7f1219cf8640 1 Processor -- start 2024-09-19T08:39:21.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.155+0000 7f1219cf8640 1 -- start start 2024-09-19T08:39:21.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.159+0000 7f1219cf8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11fc004d50 0x7f1214147c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:21.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.159+0000 7f1219cf8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f12140b0b10 0x7f12140b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:21.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.159+0000 7f1219cf8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f121415cab0 0x7f121415cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:21.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.159+0000 7f1219cf8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f12140730a0 con 0x7f12140b0b10 2024-09-19T08:39:21.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.159+0000 7f1219cf8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f121415ea70 con 0x7f11fc004d50 2024-09-19T08:39:21.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.159+0000 7f1219cf8640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1214073220 con 0x7f121415cab0 2024-09-19T08:39:21.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.159+0000 7f1212ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f12140b0b10 0x7f12140b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:21.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.159+0000 7f1212ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f12140b0b10 0x7f12140b0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35112/0 (socket says 172.21.15.98:35112) 2024-09-19T08:39:21.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.159+0000 7f1212ffd640 1 -- 172.21.15.98:0/1556594619 learned_addr learned my addr 172.21.15.98:0/1556594619 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:21.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.159+0000 7f12137fe640 1 --2- 172.21.15.98:0/1556594619 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11fc004d50 0x7f1214147c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:21.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.159+0000 7f1213fff640 1 --2- 172.21.15.98:0/1556594619 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f121415cab0 0x7f121415cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:21.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.159+0000 7f1212ffd640 1 -- 172.21.15.98:0/1556594619 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f121415cab0 msgr2=0x7f121415cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:21.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.159+0000 7f1212ffd640 1 --2- 172.21.15.98:0/1556594619 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f121415cab0 0x7f121415cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:21.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.159+0000 7f1212ffd640 1 -- 172.21.15.98:0/1556594619 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11fc004d50 msgr2=0x7f1214147c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:21.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.159+0000 7f1212ffd640 1 --2- 172.21.15.98:0/1556594619 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11fc004d50 0x7f1214147c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:21.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.159+0000 7f1212ffd640 1 -- 172.21.15.98:0/1556594619 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12140b1660 con 0x7f12140b0b10 2024-09-19T08:39:21.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.159+0000 7f1212ffd640 1 --2- 172.21.15.98:0/1556594619 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f12140b0b10 0x7f12140b0f90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1208010c80 tx=0x7f120800e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:21.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.159+0000 7f11fbfff640 1 -- 172.21.15.98:0/1556594619 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120801b070 con 0x7f12140b0b10 2024-09-19T08:39:21.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.159+0000 7f11fbfff640 1 -- 172.21.15.98:0/1556594619 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1208011040 con 0x7f12140b0b10 2024-09-19T08:39:21.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.159+0000 7f1219cf8640 1 -- 172.21.15.98:0/1556594619 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12140b1940 con 0x7f12140b0b10 2024-09-19T08:39:21.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.159+0000 7f11fbfff640 1 -- 172.21.15.98:0/1556594619 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12080143a0 con 0x7f12140b0b10 2024-09-19T08:39:21.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.159+0000 7f1219cf8640 1 -- 172.21.15.98:0/1556594619 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1214154520 con 0x7f12140b0b10 2024-09-19T08:39:21.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.159+0000 7f1219cf8640 1 -- 172.21.15.98:0/1556594619 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f121415d470 con 0x7f12140b0b10 2024-09-19T08:39:21.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.159+0000 7f11fbfff640 1 -- 172.21.15.98:0/1556594619 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f120801e070 con 0x7f12140b0b10 2024-09-19T08:39:21.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.159+0000 7f11fbfff640 1 --2- 172.21.15.98:0/1556594619 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f12000678c0 0x7f1200069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:21.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.159+0000 7f11fbfff640 1 -- 172.21.15.98:0/1556594619 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f1208088870 con 0x7f12140b0b10 2024-09-19T08:39:21.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.159+0000 7f12137fe640 1 --2- 172.21.15.98:0/1556594619 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f12000678c0 0x7f1200069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:21.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.159+0000 7f12137fe640 1 --2- 172.21.15.98:0/1556594619 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f12000678c0 0x7f1200069d80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f12040047c0 tx=0x7f1204008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:21.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.163+0000 7f11fbfff640 1 -- 172.21.15.98:0/1556594619 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f12080593b0 con 0x7f12140b0b10 2024-09-19T08:39:21.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.343+0000 7f1219cf8640 1 -- 172.21.15.98:0/1556594619 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol44050", "group_name": "subvol_grp66610", "target": ["mon-mgr", ""]}) -- 0x7f12140b1df0 con 0x7f12000678c0 2024-09-19T08:39:21.353 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:39:21.351+0000 7f36e9c04640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:39:21.353 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:39:21.351+0000 7f36e9c04640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:39:21.353 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:39:21.351+0000 7f36e9c04640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:39:21.354 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:39:21.351+0000 7f36e9c04640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:39:21.354 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:39:21.351+0000 7f36e9c04640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:39:21.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.451+0000 7f11fbfff640 1 -- 172.21.15.98:0/1556594619 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f12140b1df0 con 0x7f12000678c0 2024-09-19T08:39:21.454 INFO:teuthology.orchestra.run.smithi098.stdout:/volumes/subvol_grp66610/subvol44050 2024-09-19T08:39:21.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.451+0000 7f1219cf8640 1 -- 172.21.15.98:0/1556594619 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f12000678c0 msgr2=0x7f1200069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:21.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.451+0000 7f1219cf8640 1 --2- 172.21.15.98:0/1556594619 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f12000678c0 0x7f1200069d80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f12040047c0 tx=0x7f1204008040 comp rx=0 tx=0).stop 2024-09-19T08:39:21.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.451+0000 7f1219cf8640 1 -- 172.21.15.98:0/1556594619 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f12140b0b10 msgr2=0x7f12140b0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:21.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.451+0000 7f1219cf8640 1 --2- 172.21.15.98:0/1556594619 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f12140b0b10 0x7f12140b0f90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1208010c80 tx=0x7f120800e4d0 comp rx=0 tx=0).stop 2024-09-19T08:39:21.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.451+0000 7f1219cf8640 1 -- 172.21.15.98:0/1556594619 shutdown_connections 2024-09-19T08:39:21.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.451+0000 7f1219cf8640 1 --2- 172.21.15.98:0/1556594619 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f12000678c0 0x7f1200069d80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:21.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.451+0000 7f1219cf8640 1 --2- 172.21.15.98:0/1556594619 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f121415cab0 0x7f121415cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:21.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.451+0000 7f1219cf8640 1 --2- 172.21.15.98:0/1556594619 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f12140b0b10 0x7f12140b0f90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:21.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.451+0000 7f1219cf8640 1 --2- 172.21.15.98:0/1556594619 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11fc004d50 0x7f1214147c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:21.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.451+0000 7f1219cf8640 1 -- 172.21.15.98:0/1556594619 >> 172.21.15.98:0/1556594619 conn(0x7f12140cedf0 msgr2=0x7f121415bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:21.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.455+0000 7f1219cf8640 1 -- 172.21.15.98:0/1556594619 shutdown_connections 2024-09-19T08:39:21.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.455+0000 7f1219cf8640 1 -- 172.21.15.98:0/1556594619 wait complete. 2024-09-19T08:39:21.472 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol44050 subvol_grp66610 2024-09-19T08:39:21.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.587+0000 7fb665a16640 1 Processor -- start 2024-09-19T08:39:21.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.587+0000 7fb665a16640 1 -- start start 2024-09-19T08:39:21.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.591+0000 7fb665a16640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb6600cf700 0x7fb66015ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:21.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.591+0000 7fb665a16640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb66006d4c0 0x7fb66006d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:21.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.591+0000 7fb665a16640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6601608c0 con 0x7fb66015c290 2024-09-19T08:39:21.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.591+0000 7fb665a16640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fb660160e80 con 0x7fb6600cf700 2024-09-19T08:39:21.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.591+0000 7fb665a16640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb6600cd510 con 0x7fb66006d4c0 2024-09-19T08:39:21.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.591+0000 7fb6567fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb6600cf700 0x7fb66015ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:21.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.591+0000 7fb6567fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb6600cf700 0x7fb66015ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35142/0 (socket says 172.21.15.98:35142) 2024-09-19T08:39:21.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.591+0000 7fb6567fc640 1 -- 172.21.15.98:0/2747862175 learned_addr learned my addr 172.21.15.98:0/2747862175 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:21.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.591+0000 7fb65f7fe640 1 --2- 172.21.15.98:0/2747862175 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb66006d4c0 0x7fb66006d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:21.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.591+0000 7fb6567fc640 1 -- 172.21.15.98:0/2747862175 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb66006d4c0 msgr2=0x7fb66006d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:21.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.591+0000 7fb6567fc640 1 --2- 172.21.15.98:0/2747862175 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb66006d4c0 0x7fb66006d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:21.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.591+0000 7fb6567fc640 1 -- 172.21.15.98:0/2747862175 >> v1:172.21.15.139:6789/0 conn(0x7fb66015c290 legacy=0x7fb6600cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:21.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.591+0000 7fb6567fc640 1 -- 172.21.15.98:0/2747862175 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb660160ba0 con 0x7fb6600cf700 2024-09-19T08:39:21.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.591+0000 7fb6567fc640 1 --2- 172.21.15.98:0/2747862175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6600cf700 0x7fb66015ff50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb648009870 tx=0x7fb64801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9ef1036549e954 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:21.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.591+0000 7fb65e7fc640 1 -- 172.21.15.98:0/2747862175 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6480b2070 con 0x7fb6600cf700 2024-09-19T08:39:21.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.591+0000 7fb65e7fc640 1 -- 172.21.15.98:0/2747862175 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6480a9070 con 0x7fb6600cf700 2024-09-19T08:39:21.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.591+0000 7fb65e7fc640 1 -- 172.21.15.98:0/2747862175 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6480ad880 con 0x7fb6600cf700 2024-09-19T08:39:21.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.595+0000 7fb665a16640 1 -- 172.21.15.98:0/2747862175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6600cf700 msgr2=0x7fb66015ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:21.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.595+0000 7fb665a16640 1 --2- 172.21.15.98:0/2747862175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6600cf700 0x7fb66015ff50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb648009870 tx=0x7fb64801c560 comp rx=0 tx=0).stop 2024-09-19T08:39:21.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.595+0000 7fb665a16640 1 -- 172.21.15.98:0/2747862175 shutdown_connections 2024-09-19T08:39:21.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.595+0000 7fb665a16640 1 --2- 172.21.15.98:0/2747862175 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb66006d4c0 0x7fb66006d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:21.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.595+0000 7fb665a16640 1 --2- 172.21.15.98:0/2747862175 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6600cf700 0x7fb66015ff50 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:21.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.595+0000 7fb665a16640 1 -- 172.21.15.98:0/2747862175 >> 172.21.15.98:0/2747862175 conn(0x7fb66006c090 msgr2=0x7fb66006c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:21.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.595+0000 7fb665a16640 1 -- 172.21.15.98:0/2747862175 shutdown_connections 2024-09-19T08:39:21.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.595+0000 7fb665a16640 1 -- 172.21.15.98:0/2747862175 wait complete. 2024-09-19T08:39:21.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.595+0000 7fb665a16640 1 Processor -- start 2024-09-19T08:39:21.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.595+0000 7fb665a16640 1 -- start start 2024-09-19T08:39:21.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.595+0000 7fb665a16640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb66006d4c0 0x7fb660165b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:21.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.595+0000 7fb665a16640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6600cf700 0x7fb660163bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:21.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.595+0000 7fb665a16640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb66015c290 0x7fb6600b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:21.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.595+0000 7fb665a16640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb6600b5580 con 0x7fb66006d4c0 2024-09-19T08:39:21.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.595+0000 7fb665a16640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb6600b5400 con 0x7fb66015c290 2024-09-19T08:39:21.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.595+0000 7fb665a16640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb6601690c0 con 0x7fb6600cf700 2024-09-19T08:39:21.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.595+0000 7fb65effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb66006d4c0 0x7fb660165b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:21.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.595+0000 7fb65effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb66006d4c0 0x7fb660165b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35154/0 (socket says 172.21.15.98:35154) 2024-09-19T08:39:21.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.595+0000 7fb65effd640 1 -- 172.21.15.98:0/1095099128 learned_addr learned my addr 172.21.15.98:0/1095099128 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:21.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.595+0000 7fb65f7fe640 1 --2- 172.21.15.98:0/1095099128 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb66015c290 0x7fb6600b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:21.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.595+0000 7fb6567fc640 1 --2- 172.21.15.98:0/1095099128 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6600cf700 0x7fb660163bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:21.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.595+0000 7fb65effd640 1 -- 172.21.15.98:0/1095099128 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6600cf700 msgr2=0x7fb660163bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:21.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.595+0000 7fb65effd640 1 --2- 172.21.15.98:0/1095099128 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6600cf700 0x7fb660163bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:21.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.595+0000 7fb65effd640 1 -- 172.21.15.98:0/1095099128 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb66015c290 msgr2=0x7fb6600b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:21.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.595+0000 7fb65effd640 1 --2- 172.21.15.98:0/1095099128 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb66015c290 0x7fb6600b1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:21.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.595+0000 7fb65effd640 1 -- 172.21.15.98:0/1095099128 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb660163110 con 0x7fb66006d4c0 2024-09-19T08:39:21.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.595+0000 7fb65effd640 1 --2- 172.21.15.98:0/1095099128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb66006d4c0 0x7fb660165b60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb65000c390 tx=0x7fb65000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:21.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.595+0000 7fb65cff9640 1 -- 172.21.15.98:0/1095099128 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb650017070 con 0x7fb66006d4c0 2024-09-19T08:39:21.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.595+0000 7fb65cff9640 1 -- 172.21.15.98:0/1095099128 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6500052f0 con 0x7fb66006d4c0 2024-09-19T08:39:21.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.595+0000 7fb65cff9640 1 -- 172.21.15.98:0/1095099128 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb650003840 con 0x7fb66006d4c0 2024-09-19T08:39:21.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.599+0000 7fb665a16640 1 -- 172.21.15.98:0/1095099128 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb660164d70 con 0x7fb66006d4c0 2024-09-19T08:39:21.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.599+0000 7fb665a16640 1 -- 172.21.15.98:0/1095099128 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb660164ff0 con 0x7fb66006d4c0 2024-09-19T08:39:21.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.599+0000 7fb665a16640 1 -- 172.21.15.98:0/1095099128 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb66006d8f0 con 0x7fb66006d4c0 2024-09-19T08:39:21.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.599+0000 7fb65cff9640 1 -- 172.21.15.98:0/1095099128 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb65001a070 con 0x7fb66006d4c0 2024-09-19T08:39:21.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.599+0000 7fb65cff9640 1 --2- 172.21.15.98:0/1095099128 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb6400678c0 0x7fb640069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:21.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.599+0000 7fb65cff9640 1 -- 172.21.15.98:0/1095099128 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fb650084f70 con 0x7fb66006d4c0 2024-09-19T08:39:21.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.599+0000 7fb6567fc640 1 --2- 172.21.15.98:0/1095099128 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb6400678c0 0x7fb640069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:21.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.599+0000 7fb6567fc640 1 --2- 172.21.15.98:0/1095099128 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb6400678c0 0x7fb640069d80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb6480af000 tx=0x7fb648008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:21.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.603+0000 7fb65cff9640 1 -- 172.21.15.98:0/1095099128 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb660164d70 con 0x7fb66006d4c0 2024-09-19T08:39:21.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.795+0000 7fb665a16640 1 -- 172.21.15.98:0/1095099128 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol44050", "group_name": "subvol_grp66610", "target": ["mon-mgr", ""]}) -- 0x7fb66015de50 con 0x7fb6400678c0 2024-09-19T08:39:21.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.803+0000 7fb65cff9640 1 -- 172.21.15.98:0/1095099128 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb66015de50 con 0x7fb6400678c0 2024-09-19T08:39:21.806 INFO:teuthology.orchestra.run.smithi098.stdout:/volumes/subvol_grp66610/subvol44050 2024-09-19T08:39:21.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.803+0000 7fb665a16640 1 -- 172.21.15.98:0/1095099128 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb6400678c0 msgr2=0x7fb640069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:21.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.803+0000 7fb665a16640 1 --2- 172.21.15.98:0/1095099128 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb6400678c0 0x7fb640069d80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb6480af000 tx=0x7fb648008040 comp rx=0 tx=0).stop 2024-09-19T08:39:21.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.803+0000 7fb665a16640 1 -- 172.21.15.98:0/1095099128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb66006d4c0 msgr2=0x7fb660165b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:21.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.803+0000 7fb665a16640 1 --2- 172.21.15.98:0/1095099128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb66006d4c0 0x7fb660165b60 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb65000c390 tx=0x7fb65000c850 comp rx=0 tx=0).stop 2024-09-19T08:39:21.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.803+0000 7fb665a16640 1 -- 172.21.15.98:0/1095099128 shutdown_connections 2024-09-19T08:39:21.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.803+0000 7fb665a16640 1 --2- 172.21.15.98:0/1095099128 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb6400678c0 0x7fb640069d80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:21.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.803+0000 7fb665a16640 1 --2- 172.21.15.98:0/1095099128 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb66015c290 0x7fb6600b1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:21.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.803+0000 7fb665a16640 1 --2- 172.21.15.98:0/1095099128 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6600cf700 0x7fb660163bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:21.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.803+0000 7fb665a16640 1 --2- 172.21.15.98:0/1095099128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb66006d4c0 0x7fb660165b60 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:21.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.803+0000 7fb665a16640 1 -- 172.21.15.98:0/1095099128 >> 172.21.15.98:0/1095099128 conn(0x7fb66006c090 msgr2=0x7fb6600b5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:21.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.807+0000 7fb665a16640 1 -- 172.21.15.98:0/1095099128 shutdown_connections 2024-09-19T08:39:21.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:21.807+0000 7fb665a16640 1 -- 172.21.15.98:0/1095099128 wait complete. 2024-09-19T08:39:21.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:21.825 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/d850646f86efe4fb640ecb81229d4966.meta) 2024-09-19T08:39:23.363 INFO:teuthology.orchestra.run.smithi098.stdout:[GLOBAL] 2024-09-19T08:39:23.363 INFO:teuthology.orchestra.run.smithi098.stdout:version = 1 2024-09-19T08:39:23.363 INFO:teuthology.orchestra.run.smithi098.stdout:type = subvolume 2024-09-19T08:39:23.363 INFO:teuthology.orchestra.run.smithi098.stdout:path = /volumes/subvol_grp66610/subvol44050 2024-09-19T08:39:23.363 INFO:teuthology.orchestra.run.smithi098.stdout:state = complete 2024-09-19T08:39:23.363 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:39:23.366 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol44050 subvol_grp66610 2024-09-19T08:39:23.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.495+0000 7efc92aa9640 1 Processor -- start 2024-09-19T08:39:23.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.495+0000 7efc92aa9640 1 -- start start 2024-09-19T08:39:23.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.495+0000 7efc92aa9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7efc8c06dbd0 0x7efc8c073680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:23.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.495+0000 7efc92aa9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc8c15fac0 0x7efc8c073bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:23.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.495+0000 7efc92aa9640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc8c15baf0 con 0x7efc8c15f6e0 2024-09-19T08:39:23.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.495+0000 7efc92aa9640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7efc8c15c8b0 con 0x7efc8c06dbd0 2024-09-19T08:39:23.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.495+0000 7efc92aa9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7efc8c0742d0 con 0x7efc8c15fac0 2024-09-19T08:39:23.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.495+0000 7efc91046640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc8c15fac0 0x7efc8c073bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:23.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.495+0000 7efc8bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7efc8c06dbd0 0x7efc8c073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:23.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.495+0000 7efc90845640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7efc8c15f6e0 0x7efc8c06d4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:45372/0 (socket says 172.21.15.98:45372) 2024-09-19T08:39:23.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.495+0000 7efc91046640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc8c15fac0 0x7efc8c073bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:43712/0 (socket says 172.21.15.98:43712) 2024-09-19T08:39:23.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.495+0000 7efc90845640 1 -- 172.21.15.98:0/2009610992 learned_addr learned my addr 172.21.15.98:0/2009610992 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:23.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.495+0000 7efc91046640 1 -- 172.21.15.98:0/2009610992 >> v2:172.21.15.139:3300/0 conn(0x7efc8c06dbd0 msgr2=0x7efc8c073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:23.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.495+0000 7efc91046640 1 --2- 172.21.15.98:0/2009610992 >> v2:172.21.15.139:3300/0 conn(0x7efc8c06dbd0 0x7efc8c073680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:23.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.495+0000 7efc91046640 1 -- 172.21.15.98:0/2009610992 >> v1:172.21.15.139:6789/0 conn(0x7efc8c15f6e0 legacy=0x7efc8c06d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:23.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.495+0000 7efc91046640 1 -- 172.21.15.98:0/2009610992 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc8c15bdd0 con 0x7efc8c15fac0 2024-09-19T08:39:23.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.495+0000 7efc8bfff640 1 --2- 172.21.15.98:0/2009610992 >> v2:172.21.15.139:3300/0 conn(0x7efc8c06dbd0 0x7efc8c073680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:39:23.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.499+0000 7efc91046640 1 --2- 172.21.15.98:0/2009610992 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc8c15fac0 0x7efc8c073bc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efc84009d90 tx=0x7efc8401c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94f58d24b6372009 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:23.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.499+0000 7efc8b7fe640 1 -- 172.21.15.98:0/2009610992 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc840b2070 con 0x7efc8c15fac0 2024-09-19T08:39:23.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.499+0000 7efc8b7fe640 1 -- 172.21.15.98:0/2009610992 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc840a5730 con 0x7efc8c15fac0 2024-09-19T08:39:23.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.499+0000 7efc8b7fe640 1 -- 172.21.15.98:0/2009610992 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc840a5b00 con 0x7efc8c15fac0 2024-09-19T08:39:23.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.499+0000 7efc92aa9640 1 -- 172.21.15.98:0/2009610992 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc8c15fac0 msgr2=0x7efc8c073bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:23.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.499+0000 7efc92aa9640 1 --2- 172.21.15.98:0/2009610992 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc8c15fac0 0x7efc8c073bc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efc84009d90 tx=0x7efc8401c970 comp rx=0 tx=0).stop 2024-09-19T08:39:23.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.499+0000 7efc92aa9640 1 -- 172.21.15.98:0/2009610992 shutdown_connections 2024-09-19T08:39:23.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.499+0000 7efc92aa9640 1 --2- 172.21.15.98:0/2009610992 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc8c15fac0 0x7efc8c073bc0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:23.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.499+0000 7efc92aa9640 1 --2- 172.21.15.98:0/2009610992 >> v2:172.21.15.139:3300/0 conn(0x7efc8c06dbd0 0x7efc8c073680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:23.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.499+0000 7efc92aa9640 1 -- 172.21.15.98:0/2009610992 >> 172.21.15.98:0/2009610992 conn(0x7efc8c06c090 msgr2=0x7efc8c06c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:23.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.499+0000 7efc92aa9640 1 -- 172.21.15.98:0/2009610992 shutdown_connections 2024-09-19T08:39:23.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.499+0000 7efc92aa9640 1 -- 172.21.15.98:0/2009610992 wait complete. 2024-09-19T08:39:23.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.503+0000 7efc92aa9640 1 Processor -- start 2024-09-19T08:39:23.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.503+0000 7efc92aa9640 1 -- start start 2024-09-19T08:39:23.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.503+0000 7efc92aa9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc8c06dbd0 0x7efc8c0b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:23.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.503+0000 7efc92aa9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc8c15f6e0 0x7efc8c148850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:23.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.503+0000 7efc92aa9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc8c15fac0 0x7efc8c0b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:23.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.503+0000 7efc92aa9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efc8c0b42c0 con 0x7efc8c06dbd0 2024-09-19T08:39:23.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.503+0000 7efc92aa9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efc8c0b4140 con 0x7efc8c15f6e0 2024-09-19T08:39:23.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.503+0000 7efc92aa9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7efc8c0b4440 con 0x7efc8c15fac0 2024-09-19T08:39:23.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.503+0000 7efc90845640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc8c06dbd0 0x7efc8c0b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:23.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.503+0000 7efc90845640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc8c06dbd0 0x7efc8c0b21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35156/0 (socket says 172.21.15.98:35156) 2024-09-19T08:39:23.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.503+0000 7efc90845640 1 -- 172.21.15.98:0/3785499380 learned_addr learned my addr 172.21.15.98:0/3785499380 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:23.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.503+0000 7efc91046640 1 --2- 172.21.15.98:0/3785499380 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc8c15fac0 0x7efc8c0b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:23.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.503+0000 7efc8bfff640 1 --2- 172.21.15.98:0/3785499380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc8c15f6e0 0x7efc8c148850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:23.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.503+0000 7efc90845640 1 -- 172.21.15.98:0/3785499380 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc8c15fac0 msgr2=0x7efc8c0b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:23.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.503+0000 7efc90845640 1 --2- 172.21.15.98:0/3785499380 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc8c15fac0 0x7efc8c0b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:23.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.503+0000 7efc90845640 1 -- 172.21.15.98:0/3785499380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc8c15f6e0 msgr2=0x7efc8c148850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:23.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.503+0000 7efc90845640 1 --2- 172.21.15.98:0/3785499380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc8c15f6e0 0x7efc8c148850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:23.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.503+0000 7efc90845640 1 -- 172.21.15.98:0/3785499380 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc8c0b27d0 con 0x7efc8c06dbd0 2024-09-19T08:39:23.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.503+0000 7efc90845640 1 --2- 172.21.15.98:0/3785499380 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc8c06dbd0 0x7efc8c0b21d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00c420 tx=0x7efc7c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:23.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.503+0000 7efc897fa640 1 -- 172.21.15.98:0/3785499380 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c017070 con 0x7efc8c06dbd0 2024-09-19T08:39:23.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.503+0000 7efc897fa640 1 -- 172.21.15.98:0/3785499380 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc7c005cf0 con 0x7efc8c06dbd0 2024-09-19T08:39:23.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.503+0000 7efc897fa640 1 -- 172.21.15.98:0/3785499380 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c0072f0 con 0x7efc8c06dbd0 2024-09-19T08:39:23.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.503+0000 7efc92aa9640 1 -- 172.21.15.98:0/3785499380 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc8c0b2ab0 con 0x7efc8c06dbd0 2024-09-19T08:39:23.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.503+0000 7efc92aa9640 1 -- 172.21.15.98:0/3785499380 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc8c0b3070 con 0x7efc8c06dbd0 2024-09-19T08:39:23.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.503+0000 7efc92aa9640 1 -- 172.21.15.98:0/3785499380 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc8c06d4c0 con 0x7efc8c06dbd0 2024-09-19T08:39:23.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.503+0000 7efc897fa640 1 -- 172.21.15.98:0/3785499380 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 6) ==== 87496+0+0 (secure 0 0 0) 0x7efc7c01a070 con 0x7efc8c06dbd0 2024-09-19T08:39:23.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.507+0000 7efc897fa640 1 --2- 172.21.15.98:0/3785499380 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7efc78067950 0x7efc78069e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:23.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.507+0000 7efc897fa640 1 -- 172.21.15.98:0/3785499380 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7efc7c085530 con 0x7efc8c06dbd0 2024-09-19T08:39:23.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.507+0000 7efc8bfff640 1 --2- 172.21.15.98:0/3785499380 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7efc78067950 0x7efc78069e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:23.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.507+0000 7efc8bfff640 1 --2- 172.21.15.98:0/3785499380 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7efc78067950 0x7efc78069e10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7efc8c1f62a0 tx=0x7efc8000b3a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:23.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.511+0000 7efc897fa640 1 -- 172.21.15.98:0/3785499380 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7efc7c021020 con 0x7efc8c06dbd0 2024-09-19T08:39:23.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.695+0000 7efc92aa9640 1 -- 172.21.15.98:0/3785499380 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol44050", "group_name": "subvol_grp66610", "target": ["mon-mgr", ""]}) -- 0x7efc8c15b3b0 con 0x7efc78067950 2024-09-19T08:39:23.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.723+0000 7efc897fa640 1 -- 172.21.15.98:0/3785499380 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efc8c15b3b0 con 0x7efc78067950 2024-09-19T08:39:23.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.723+0000 7efc92aa9640 1 -- 172.21.15.98:0/3785499380 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7efc78067950 msgr2=0x7efc78069e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:23.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.723+0000 7efc92aa9640 1 --2- 172.21.15.98:0/3785499380 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7efc78067950 0x7efc78069e10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7efc8c1f62a0 tx=0x7efc8000b3a0 comp rx=0 tx=0).stop 2024-09-19T08:39:23.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.723+0000 7efc92aa9640 1 -- 172.21.15.98:0/3785499380 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc8c06dbd0 msgr2=0x7efc8c0b21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:23.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.723+0000 7efc92aa9640 1 --2- 172.21.15.98:0/3785499380 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc8c06dbd0 0x7efc8c0b21d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00c420 tx=0x7efc7c00c8e0 comp rx=0 tx=0).stop 2024-09-19T08:39:23.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.723+0000 7efc92aa9640 1 -- 172.21.15.98:0/3785499380 shutdown_connections 2024-09-19T08:39:23.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.723+0000 7efc92aa9640 1 --2- 172.21.15.98:0/3785499380 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7efc78067950 0x7efc78069e10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:23.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.723+0000 7efc92aa9640 1 --2- 172.21.15.98:0/3785499380 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc8c15fac0 0x7efc8c0b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:23.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.723+0000 7efc92aa9640 1 --2- 172.21.15.98:0/3785499380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc8c15f6e0 0x7efc8c148850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:23.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.723+0000 7efc92aa9640 1 --2- 172.21.15.98:0/3785499380 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc8c06dbd0 0x7efc8c0b21d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:23.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.723+0000 7efc92aa9640 1 -- 172.21.15.98:0/3785499380 >> 172.21.15.98:0/3785499380 conn(0x7efc8c06c090 msgr2=0x7efc8c06c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:23.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.723+0000 7efc92aa9640 1 -- 172.21.15.98:0/3785499380 shutdown_connections 2024-09-19T08:39:23.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.723+0000 7efc92aa9640 1 -- 172.21.15.98:0/3785499380 wait complete. 2024-09-19T08:39:23.730 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:39:23.727+0000 7f36ecc0a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:39:23.730 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:39:23.727+0000 7f36ecc0a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:39:23.730 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:39:23.727+0000 7f36ecc0a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:39:23.730 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:39:23.727+0000 7f36ecc0a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:39:23.730 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:39:23.727+0000 7f36ecc0a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:39:23.746 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:39:23.743+0000 7f36ebc08640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:39:23.746 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:39:23.743+0000 7f36ebc08640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:39:23.747 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:39:23.743+0000 7f36ebc08640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:39:23.747 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:39:23.743+0000 7f36ebc08640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:39:23.747 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:39:23.743+0000 7f36ebc08640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:39:23.748 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-19T08:39:23.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T08:39:23.790 INFO:teuthology.orchestra.run.smithi098.stdout:2 2024-09-19T08:39:23.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:39:23.796 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp66610 2024-09-19T08:39:23.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.959+0000 7f9073830640 1 Processor -- start 2024-09-19T08:39:23.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.959+0000 7f9073830640 1 -- start start 2024-09-19T08:39:23.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.959+0000 7f9073830640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f906c06c3a0 0x7f906c0713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:23.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.959+0000 7f9073830640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f906c0718f0 0x7f906c0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:23.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.959+0000 7f9073830640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f906c073530 con 0x7f906c145b70 2024-09-19T08:39:23.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.959+0000 7f9073830640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f906c073810 con 0x7f906c0718f0 2024-09-19T08:39:23.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.959+0000 7f9073830640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f906c073af0 con 0x7f906c06c3a0 2024-09-19T08:39:23.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.959+0000 7f9070dcb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f906c06c3a0 0x7f906c0713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:23.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.959+0000 7f9071dcd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f906c0718f0 0x7f906c0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:23.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.959+0000 7f9070dcb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f906c06c3a0 0x7f906c0713b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:43754/0 (socket says 172.21.15.98:43754) 2024-09-19T08:39:23.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.959+0000 7f9070dcb640 1 -- 172.21.15.98:0/2654566266 learned_addr learned my addr 172.21.15.98:0/2654566266 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:23.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.959+0000 7f90715cc640 1 --1- 172.21.15.98:0/2654566266 >> v1:172.21.15.139:6789/0 conn(0x7f906c145b70 0x7f906c06bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:45386/0 (socket says 172.21.15.98:45386) 2024-09-19T08:39:23.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.959+0000 7f9070dcb640 1 -- 172.21.15.98:0/2654566266 >> v2:172.21.15.139:3300/0 conn(0x7f906c0718f0 msgr2=0x7f906c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:23.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.959+0000 7f9070dcb640 1 --2- 172.21.15.98:0/2654566266 >> v2:172.21.15.139:3300/0 conn(0x7f906c0718f0 0x7f906c0b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:23.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.959+0000 7f9070dcb640 1 -- 172.21.15.98:0/2654566266 >> v1:172.21.15.139:6789/0 conn(0x7f906c145b70 legacy=0x7f906c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:23.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.959+0000 7f9070dcb640 1 -- 172.21.15.98:0/2654566266 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f906c073dd0 con 0x7f906c06c3a0 2024-09-19T08:39:23.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f9070dcb640 1 --2- 172.21.15.98:0/2654566266 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f906c06c3a0 0x7f906c0713b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f905c009870 tx=0x7f905c01c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d17c7b01c0f0fbf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:23.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f9063fff640 1 -- 172.21.15.98:0/2654566266 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c0b0070 con 0x7f906c06c3a0 2024-09-19T08:39:23.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f9063fff640 1 -- 172.21.15.98:0/2654566266 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f905c0ab600 con 0x7f906c06c3a0 2024-09-19T08:39:23.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f9063fff640 1 -- 172.21.15.98:0/2654566266 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c0ab9d0 con 0x7f906c06c3a0 2024-09-19T08:39:23.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f9073830640 1 -- 172.21.15.98:0/2654566266 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f906c06c3a0 msgr2=0x7f906c0713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:23.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f9073830640 1 --2- 172.21.15.98:0/2654566266 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f906c06c3a0 0x7f906c0713b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f905c009870 tx=0x7f905c01c4d0 comp rx=0 tx=0).stop 2024-09-19T08:39:23.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f9073830640 1 -- 172.21.15.98:0/2654566266 shutdown_connections 2024-09-19T08:39:23.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f9073830640 1 --2- 172.21.15.98:0/2654566266 >> v2:172.21.15.139:3300/0 conn(0x7f906c0718f0 0x7f906c0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:23.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f9073830640 1 --2- 172.21.15.98:0/2654566266 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f906c06c3a0 0x7f906c0713b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:23.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f9073830640 1 -- 172.21.15.98:0/2654566266 >> 172.21.15.98:0/2654566266 conn(0x7f906c0b2010 msgr2=0x7f906c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:23.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f9073830640 1 -- 172.21.15.98:0/2654566266 shutdown_connections 2024-09-19T08:39:23.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f9073830640 1 -- 172.21.15.98:0/2654566266 wait complete. 2024-09-19T08:39:23.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f9073830640 1 Processor -- start 2024-09-19T08:39:23.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f9073830640 1 -- start start 2024-09-19T08:39:23.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f9073830640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f906c06c3a0 0x7f906c13b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:23.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f9073830640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f906c0718f0 0x7f906c144d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:23.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f9073830640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f906c145b70 0x7f906c0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:23.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f9073830640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f906c0b4530 con 0x7f906c06c3a0 2024-09-19T08:39:23.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f9073830640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f906c0b43b0 con 0x7f906c0718f0 2024-09-19T08:39:23.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f9073830640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f906c0b46b0 con 0x7f906c145b70 2024-09-19T08:39:23.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f90715cc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f906c06c3a0 0x7f906c13b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:23.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f90715cc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f906c06c3a0 0x7f906c13b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35208/0 (socket says 172.21.15.98:35208) 2024-09-19T08:39:23.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f90715cc640 1 -- 172.21.15.98:0/3048909286 learned_addr learned my addr 172.21.15.98:0/3048909286 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:23.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f9070dcb640 1 --2- 172.21.15.98:0/3048909286 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f906c0718f0 0x7f906c144d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:23.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f9071dcd640 1 --2- 172.21.15.98:0/3048909286 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f906c145b70 0x7f906c0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:23.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f90715cc640 1 -- 172.21.15.98:0/3048909286 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f906c145b70 msgr2=0x7f906c0ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:23.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f90715cc640 1 --2- 172.21.15.98:0/3048909286 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f906c145b70 0x7f906c0ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:23.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f90715cc640 1 -- 172.21.15.98:0/3048909286 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f906c0718f0 msgr2=0x7f906c144d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:23.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f90715cc640 1 --2- 172.21.15.98:0/3048909286 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f906c0718f0 0x7f906c144d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:23.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f90715cc640 1 -- 172.21.15.98:0/3048909286 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f906c0cc8c0 con 0x7f906c06c3a0 2024-09-19T08:39:23.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f9071dcd640 1 --2- 172.21.15.98:0/3048909286 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f906c145b70 0x7f906c0ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:39:23.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f90715cc640 1 --2- 172.21.15.98:0/3048909286 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f906c06c3a0 0x7f906c13b8c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f906800c3a0 tx=0x7f906800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:23.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f9061ffb640 1 -- 172.21.15.98:0/3048909286 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90680074e0 con 0x7f906c06c3a0 2024-09-19T08:39:23.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f9061ffb640 1 -- 172.21.15.98:0/3048909286 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9068007680 con 0x7f906c06c3a0 2024-09-19T08:39:23.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f9073830640 1 -- 172.21.15.98:0/3048909286 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f906c15e2d0 con 0x7f906c06c3a0 2024-09-19T08:39:23.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f9061ffb640 1 -- 172.21.15.98:0/3048909286 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068004a30 con 0x7f906c06c3a0 2024-09-19T08:39:23.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f9073830640 1 -- 172.21.15.98:0/3048909286 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f906c15e890 con 0x7f906c06c3a0 2024-09-19T08:39:23.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f9073830640 1 -- 172.21.15.98:0/3048909286 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f906c0713b0 con 0x7f906c06c3a0 2024-09-19T08:39:23.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f9061ffb640 1 -- 172.21.15.98:0/3048909286 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 6) ==== 87496+0+0 (secure 0 0 0) 0x7f906801a020 con 0x7f906c06c3a0 2024-09-19T08:39:23.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f9061ffb640 1 --2- 172.21.15.98:0/3048909286 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9050067950 0x7f9050069e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:23.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f9061ffb640 1 -- 172.21.15.98:0/3048909286 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f90680853c0 con 0x7f906c06c3a0 2024-09-19T08:39:23.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f9070dcb640 1 --2- 172.21.15.98:0/3048909286 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9050067950 0x7f9050069e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:23.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.967+0000 7f9070dcb640 1 --2- 172.21.15.98:0/3048909286 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9050067950 0x7f9050069e10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f905c0ad000 tx=0x7f905c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:23.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:23.971+0000 7f9061ffb640 1 -- 172.21.15.98:0/3048909286 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9068055f10 con 0x7f906c06c3a0 2024-09-19T08:39:24.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.163+0000 7f9073830640 1 -- 172.21.15.98:0/3048909286 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp66610", "target": ["mon-mgr", ""]}) -- 0x7f906c13be00 con 0x7f9050067950 2024-09-19T08:39:24.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.175+0000 7f9061ffb640 1 -- 172.21.15.98:0/3048909286 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f906c13be00 con 0x7f9050067950 2024-09-19T08:39:24.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.175+0000 7f9073830640 1 -- 172.21.15.98:0/3048909286 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9050067950 msgr2=0x7f9050069e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:24.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.175+0000 7f9073830640 1 --2- 172.21.15.98:0/3048909286 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9050067950 0x7f9050069e10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f905c0ad000 tx=0x7f905c008040 comp rx=0 tx=0).stop 2024-09-19T08:39:24.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.175+0000 7f9073830640 1 -- 172.21.15.98:0/3048909286 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f906c06c3a0 msgr2=0x7f906c13b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:24.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.175+0000 7f9073830640 1 --2- 172.21.15.98:0/3048909286 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f906c06c3a0 0x7f906c13b8c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f906800c3a0 tx=0x7f906800c860 comp rx=0 tx=0).stop 2024-09-19T08:39:24.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.175+0000 7f9073830640 1 -- 172.21.15.98:0/3048909286 shutdown_connections 2024-09-19T08:39:24.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.175+0000 7f9073830640 1 --2- 172.21.15.98:0/3048909286 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9050067950 0x7f9050069e10 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:24.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.175+0000 7f9073830640 1 --2- 172.21.15.98:0/3048909286 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f906c145b70 0x7f906c0ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:24.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.175+0000 7f9073830640 1 --2- 172.21.15.98:0/3048909286 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f906c0718f0 0x7f906c144d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:24.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.175+0000 7f9073830640 1 --2- 172.21.15.98:0/3048909286 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f906c06c3a0 0x7f906c13b8c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:24.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.175+0000 7f9073830640 1 -- 172.21.15.98:0/3048909286 >> 172.21.15.98:0/3048909286 conn(0x7f906c0b2010 msgr2=0x7f906c0aff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:24.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.175+0000 7f9073830640 1 -- 172.21.15.98:0/3048909286 shutdown_connections 2024-09-19T08:39:24.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.175+0000 7f9073830640 1 -- 172.21.15.98:0/3048909286 wait complete. 2024-09-19T08:39:24.192 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T08:39:24.192 DEBUG:teuthology.orchestra.run.smithi098:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:39:24.278 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:39:24.349 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T08:39:24.349 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:39:24.350 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:39:24.350 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:39:24.356 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T08:39:24.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.527+0000 7f40b0957640 1 Processor -- start 2024-09-19T08:39:24.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.527+0000 7f40b0957640 1 -- start start 2024-09-19T08:39:24.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.527+0000 7f40b0957640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f40ac15f590 0x7f40ac0cf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:24.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.527+0000 7f40b0957640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40ac0cf8b0 0x7f40ac06bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:24.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.527+0000 7f40b0957640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40ac15fb50 con 0x7f40ac06c4e0 2024-09-19T08:39:24.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.527+0000 7f40b0957640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f40ac06c950 con 0x7f40ac15f590 2024-09-19T08:39:24.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.527+0000 7f40b0957640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f40ac15f970 con 0x7f40ac0cf8b0 2024-09-19T08:39:24.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.531+0000 7f40aa59c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f40ac15f590 0x7f40ac0cf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:24.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.531+0000 7f40aad9d640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f40ac06c4e0 0x7f40ac162030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:46318/0 (socket says 172.21.15.98:46318) 2024-09-19T08:39:24.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.531+0000 7f40aad9d640 1 -- 172.21.15.98:0/817019635 learned_addr learned my addr 172.21.15.98:0/817019635 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:24.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.531+0000 7f40a3fff640 1 --2- 172.21.15.98:0/817019635 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40ac0cf8b0 0x7f40ac06bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:24.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.531+0000 7f40aa59c640 1 -- 172.21.15.98:0/817019635 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40ac0cf8b0 msgr2=0x7f40ac06bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:24.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.531+0000 7f40aa59c640 1 --2- 172.21.15.98:0/817019635 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40ac0cf8b0 0x7f40ac06bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:24.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.531+0000 7f40aa59c640 1 -- 172.21.15.98:0/817019635 >> v1:172.21.15.98:6789/0 conn(0x7f40ac06c4e0 legacy=0x7f40ac162030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:24.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.531+0000 7f40aa59c640 1 -- 172.21.15.98:0/817019635 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40ac15fdc0 con 0x7f40ac15f590 2024-09-19T08:39:24.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.531+0000 7f40a9d9b640 1 -- 172.21.15.98:0/817019635 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1293778604 0 0) 0x7f40ac15fb50 con 0x7f40ac06c4e0 2024-09-19T08:39:24.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.531+0000 7f40aa59c640 1 --2- 172.21.15.98:0/817019635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40ac15f590 0x7f40ac0cf370 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f409c0097c0 tx=0x7f409c01ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9485e64b73901b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:24.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.531+0000 7f40a9d9b640 1 -- 172.21.15.98:0/817019635 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c0b5070 con 0x7f40ac15f590 2024-09-19T08:39:24.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.531+0000 7f40a9d9b640 1 -- 172.21.15.98:0/817019635 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f409c0a8990 con 0x7f40ac15f590 2024-09-19T08:39:24.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.531+0000 7f40a9d9b640 1 -- 172.21.15.98:0/817019635 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c0a8d60 con 0x7f40ac15f590 2024-09-19T08:39:24.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.531+0000 7f40b0957640 1 -- 172.21.15.98:0/817019635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40ac15f590 msgr2=0x7f40ac0cf370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:24.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.531+0000 7f40b0957640 1 --2- 172.21.15.98:0/817019635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40ac15f590 0x7f40ac0cf370 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f409c0097c0 tx=0x7f409c01ec90 comp rx=0 tx=0).stop 2024-09-19T08:39:24.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.531+0000 7f40b0957640 1 -- 172.21.15.98:0/817019635 shutdown_connections 2024-09-19T08:39:24.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.531+0000 7f40b0957640 1 --2- 172.21.15.98:0/817019635 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40ac0cf8b0 0x7f40ac06bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:24.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.531+0000 7f40b0957640 1 --2- 172.21.15.98:0/817019635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40ac15f590 0x7f40ac0cf370 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:24.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.531+0000 7f40b0957640 1 -- 172.21.15.98:0/817019635 >> 172.21.15.98:0/817019635 conn(0x7f40ac074c70 msgr2=0x7f40ac075080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:24.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.531+0000 7f40b0957640 1 -- 172.21.15.98:0/817019635 shutdown_connections 2024-09-19T08:39:24.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.531+0000 7f40b0957640 1 -- 172.21.15.98:0/817019635 wait complete. 2024-09-19T08:39:24.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.535+0000 7f40b0957640 1 Processor -- start 2024-09-19T08:39:24.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.535+0000 7f40b0957640 1 -- start start 2024-09-19T08:39:24.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.535+0000 7f40b0957640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40ac06c4e0 0x7f40ac1539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:24.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.535+0000 7f40b0957640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40ac0cf8b0 0x7f40ac0d3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:24.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.535+0000 7f40b0957640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40ac15f590 0x7f40ac14ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:24.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.535+0000 7f40b0957640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f40ac160660 con 0x7f40ac06c4e0 2024-09-19T08:39:24.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.535+0000 7f40b0957640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f40ac160380 con 0x7f40ac15f590 2024-09-19T08:39:24.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.535+0000 7f40b0957640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f40ac160940 con 0x7f40ac0cf8b0 2024-09-19T08:39:24.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.535+0000 7f40aa59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40ac06c4e0 0x7f40ac1539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:24.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.535+0000 7f40aa59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40ac06c4e0 0x7f40ac1539d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35222/0 (socket says 172.21.15.98:35222) 2024-09-19T08:39:24.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.535+0000 7f40aa59c640 1 -- 172.21.15.98:0/594911992 learned_addr learned my addr 172.21.15.98:0/594911992 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:24.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.535+0000 7f40aad9d640 1 --2- 172.21.15.98:0/594911992 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40ac15f590 0x7f40ac14ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:24.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.535+0000 7f40aa59c640 1 -- 172.21.15.98:0/594911992 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40ac0cf8b0 msgr2=0x7f40ac0d3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:24.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.535+0000 7f40aa59c640 1 --2- 172.21.15.98:0/594911992 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40ac0cf8b0 0x7f40ac0d3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:24.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.535+0000 7f40aa59c640 1 -- 172.21.15.98:0/594911992 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40ac15f590 msgr2=0x7f40ac14ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:24.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.535+0000 7f40aa59c640 1 --2- 172.21.15.98:0/594911992 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40ac15f590 0x7f40ac14ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:24.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.535+0000 7f40aa59c640 1 -- 172.21.15.98:0/594911992 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40ac15a0f0 con 0x7f40ac06c4e0 2024-09-19T08:39:24.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.535+0000 7f40aa59c640 1 --2- 172.21.15.98:0/594911992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40ac06c4e0 0x7f40ac1539d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f409c0b3040 tx=0x7f409c01ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:24.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.535+0000 7f40a2ffd640 1 -- 172.21.15.98:0/594911992 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c0b5040 con 0x7f40ac06c4e0 2024-09-19T08:39:24.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.535+0000 7f40a2ffd640 1 -- 172.21.15.98:0/594911992 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f409c004220 con 0x7f40ac06c4e0 2024-09-19T08:39:24.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.535+0000 7f40b0957640 1 -- 172.21.15.98:0/594911992 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40ac15a3d0 con 0x7f40ac06c4e0 2024-09-19T08:39:24.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.535+0000 7f40a2ffd640 1 -- 172.21.15.98:0/594911992 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c003620 con 0x7f40ac06c4e0 2024-09-19T08:39:24.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.535+0000 7f40b0957640 1 -- 172.21.15.98:0/594911992 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40ac15a990 con 0x7f40ac06c4e0 2024-09-19T08:39:24.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.535+0000 7f40b0957640 1 -- 172.21.15.98:0/594911992 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40ac15b160 con 0x7f40ac06c4e0 2024-09-19T08:39:24.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.535+0000 7f40a2ffd640 1 -- 172.21.15.98:0/594911992 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 6) ==== 87496+0+0 (secure 0 0 0) 0x7f409c0037c0 con 0x7f40ac06c4e0 2024-09-19T08:39:24.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.539+0000 7f40a2ffd640 1 --2- 172.21.15.98:0/594911992 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f408c067950 0x7f408c069e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:24.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.539+0000 7f40a2ffd640 1 -- 172.21.15.98:0/594911992 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f409c123250 con 0x7f40ac06c4e0 2024-09-19T08:39:24.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.539+0000 7f40a3fff640 1 --2- 172.21.15.98:0/594911992 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f408c067950 0x7f408c069e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:24.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.539+0000 7f40a3fff640 1 --2- 172.21.15.98:0/594911992 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f408c067950 0x7f408c069e10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f40ac1554e0 tx=0x7f4094008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:24.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.543+0000 7f40a2ffd640 1 -- 172.21.15.98:0/594911992 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f409c0c6030 con 0x7f40ac06c4e0 2024-09-19T08:39:24.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.715+0000 7f40b0957640 1 -- 172.21.15.98:0/594911992 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f40ac0ba500 con 0x7f40ac06c4e0 2024-09-19T08:39:24.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.715+0000 7f40a2ffd640 1 -- 172.21.15.98:0/594911992 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v32) ==== 80+0+0 (secure 0 0 0) 0x7f409c0bd030 con 0x7f40ac06c4e0 2024-09-19T08:39:24.716 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-19T08:39:24.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.719+0000 7f40b0957640 1 -- 172.21.15.98:0/594911992 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f408c067950 msgr2=0x7f408c069e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:24.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.719+0000 7f40b0957640 1 --2- 172.21.15.98:0/594911992 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f408c067950 0x7f408c069e10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f40ac1554e0 tx=0x7f4094008040 comp rx=0 tx=0).stop 2024-09-19T08:39:24.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.719+0000 7f40b0957640 1 -- 172.21.15.98:0/594911992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40ac06c4e0 msgr2=0x7f40ac1539d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:24.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.719+0000 7f40b0957640 1 --2- 172.21.15.98:0/594911992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40ac06c4e0 0x7f40ac1539d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f409c0b3040 tx=0x7f409c01ba30 comp rx=0 tx=0).stop 2024-09-19T08:39:24.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.719+0000 7f40b0957640 1 -- 172.21.15.98:0/594911992 shutdown_connections 2024-09-19T08:39:24.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.719+0000 7f40b0957640 1 --2- 172.21.15.98:0/594911992 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f408c067950 0x7f408c069e10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:24.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.719+0000 7f40b0957640 1 --2- 172.21.15.98:0/594911992 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40ac15f590 0x7f40ac14ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:24.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.719+0000 7f40b0957640 1 --2- 172.21.15.98:0/594911992 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40ac0cf8b0 0x7f40ac0d3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:24.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.719+0000 7f40b0957640 1 --2- 172.21.15.98:0/594911992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40ac06c4e0 0x7f40ac1539d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:24.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.719+0000 7f40b0957640 1 -- 172.21.15.98:0/594911992 >> 172.21.15.98:0/594911992 conn(0x7f40ac074c70 msgr2=0x7f40ac15c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:24.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.719+0000 7f40b0957640 1 -- 172.21.15.98:0/594911992 shutdown_connections 2024-09-19T08:39:24.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.719+0000 7f40b0957640 1 -- 172.21.15.98:0/594911992 wait complete. 2024-09-19T08:39:24.735 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi098.front.sepia.ceph.com... 2024-09-19T08:39:24.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:24.736 DEBUG:teuthology.orchestra.run.smithi098:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T08:39:24.745 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:ceph-fuse[37711]: fuse finished with error 0 and tester_r 0 2024-09-19T08:39:24.746 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:39:24.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:24.746 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T08:39:24.857 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:39:24.857 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:39:24.857 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:39:24.864 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T08:39:24.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.983+0000 7f2256c6c640 1 Processor -- start 2024-09-19T08:39:24.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.983+0000 7f2256c6c640 1 -- start start 2024-09-19T08:39:24.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.983+0000 7f2256c6c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f225015c550 0x7f225006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:24.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.983+0000 7f2256c6c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22500cfad0 0x7f22500723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:24.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.983+0000 7f2256c6c640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f225006d310 con 0x7f225006c1d0 2024-09-19T08:39:24.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.983+0000 7f2256c6c640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f225006c640 con 0x7f225015c550 2024-09-19T08:39:24.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.983+0000 7f2256c6c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f225006d5f0 con 0x7f22500cfad0 2024-09-19T08:39:24.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.983+0000 7f224ffff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f225006c1d0 0x7f22500cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:46322/0 (socket says 172.21.15.98:46322) 2024-09-19T08:39:24.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.983+0000 7f2254a08640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f225015c550 0x7f225006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:24.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.983+0000 7f224ffff640 1 -- 172.21.15.98:0/1070892930 learned_addr learned my addr 172.21.15.98:0/1070892930 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:24.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.983+0000 7f2255209640 1 --2- 172.21.15.98:0/1070892930 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22500cfad0 0x7f22500723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:24.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.983+0000 7f2254a08640 1 -- 172.21.15.98:0/1070892930 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22500cfad0 msgr2=0x7f22500723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:24.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.983+0000 7f2254a08640 1 --2- 172.21.15.98:0/1070892930 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22500cfad0 0x7f22500723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:24.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.983+0000 7f2254a08640 1 -- 172.21.15.98:0/1070892930 >> v1:172.21.15.98:6789/0 conn(0x7f225006c1d0 legacy=0x7f22500cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:24.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.983+0000 7f224f7fe640 1 -- 172.21.15.98:0/1070892930 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1243339998 0 0) 0x7f225006d310 con 0x7f225006c1d0 2024-09-19T08:39:24.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.983+0000 7f2254a08640 1 -- 172.21.15.98:0/1070892930 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225006d8d0 con 0x7f225015c550 2024-09-19T08:39:24.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.987+0000 7f2254a08640 1 --2- 172.21.15.98:0/1070892930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f225015c550 0x7f225006bc90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f22400097c0 tx=0x7f224001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2be92ed306df4c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:24.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.987+0000 7f224f7fe640 1 -- 172.21.15.98:0/1070892930 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22400b0070 con 0x7f225015c550 2024-09-19T08:39:24.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.987+0000 7f224f7fe640 1 -- 172.21.15.98:0/1070892930 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22400ab600 con 0x7f225015c550 2024-09-19T08:39:24.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.987+0000 7f224f7fe640 1 -- 172.21.15.98:0/1070892930 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22400ab9d0 con 0x7f225015c550 2024-09-19T08:39:24.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.987+0000 7f2256c6c640 1 -- 172.21.15.98:0/1070892930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f225015c550 msgr2=0x7f225006bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:24.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.987+0000 7f2256c6c640 1 --2- 172.21.15.98:0/1070892930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f225015c550 0x7f225006bc90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f22400097c0 tx=0x7f224001c4a0 comp rx=0 tx=0).stop 2024-09-19T08:39:24.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.987+0000 7f2256c6c640 1 -- 172.21.15.98:0/1070892930 shutdown_connections 2024-09-19T08:39:24.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.987+0000 7f2256c6c640 1 --2- 172.21.15.98:0/1070892930 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22500cfad0 0x7f22500723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:24.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.987+0000 7f2256c6c640 1 --2- 172.21.15.98:0/1070892930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f225015c550 0x7f225006bc90 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:24.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.987+0000 7f2256c6c640 1 -- 172.21.15.98:0/1070892930 >> 172.21.15.98:0/1070892930 conn(0x7f22500b2010 msgr2=0x7f22500b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:24.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.987+0000 7f2256c6c640 1 -- 172.21.15.98:0/1070892930 shutdown_connections 2024-09-19T08:39:24.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.987+0000 7f2256c6c640 1 -- 172.21.15.98:0/1070892930 wait complete. 2024-09-19T08:39:24.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.987+0000 7f2256c6c640 1 Processor -- start 2024-09-19T08:39:24.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.987+0000 7f2256c6c640 1 -- start start 2024-09-19T08:39:24.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.991+0000 7f2256c6c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f225006c1d0 0x7f225014fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:24.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.991+0000 7f2256c6c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22500cfad0 0x7f22500ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:24.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.991+0000 7f2256c6c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f225015c550 0x7f22500aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:24.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.991+0000 7f2256c6c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2250073340 con 0x7f225015c550 2024-09-19T08:39:24.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.991+0000 7f2256c6c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f225015cb40 con 0x7f22500cfad0 2024-09-19T08:39:24.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.991+0000 7f2256c6c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f22500734c0 con 0x7f225006c1d0 2024-09-19T08:39:24.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.991+0000 7f2255209640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f225015c550 0x7f22500aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:24.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.991+0000 7f2255209640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f225015c550 0x7f22500aedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35234/0 (socket says 172.21.15.98:35234) 2024-09-19T08:39:24.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.991+0000 7f2255209640 1 -- 172.21.15.98:0/2308444715 learned_addr learned my addr 172.21.15.98:0/2308444715 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:24.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.991+0000 7f2254a08640 1 --2- 172.21.15.98:0/2308444715 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f225006c1d0 0x7f225014fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:24.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.991+0000 7f224ffff640 1 --2- 172.21.15.98:0/2308444715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22500cfad0 0x7f22500ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:24.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.991+0000 7f2255209640 1 -- 172.21.15.98:0/2308444715 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f225006c1d0 msgr2=0x7f225014fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:24.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.991+0000 7f2255209640 1 --2- 172.21.15.98:0/2308444715 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f225006c1d0 0x7f225014fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:24.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.991+0000 7f2255209640 1 -- 172.21.15.98:0/2308444715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22500cfad0 msgr2=0x7f22500ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:24.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.991+0000 7f2255209640 1 --2- 172.21.15.98:0/2308444715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22500cfad0 0x7f22500ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:24.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.991+0000 7f2255209640 1 -- 172.21.15.98:0/2308444715 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2250150360 con 0x7f225015c550 2024-09-19T08:39:24.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.991+0000 7f224ffff640 1 --2- 172.21.15.98:0/2308444715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22500cfad0 0x7f22500ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:24.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.991+0000 7f2255209640 1 --2- 172.21.15.98:0/2308444715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f225015c550 0x7f22500aedf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2248007e40 tx=0x7f224800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:24.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.991+0000 7f224d7fa640 1 -- 172.21.15.98:0/2308444715 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2248014620 con 0x7f225015c550 2024-09-19T08:39:24.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.991+0000 7f224d7fa640 1 -- 172.21.15.98:0/2308444715 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2248019070 con 0x7f225015c550 2024-09-19T08:39:24.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.991+0000 7f2256c6c640 1 -- 172.21.15.98:0/2308444715 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2250158720 con 0x7f225015c550 2024-09-19T08:39:24.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.991+0000 7f224d7fa640 1 -- 172.21.15.98:0/2308444715 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2248014a20 con 0x7f225015c550 2024-09-19T08:39:24.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.991+0000 7f2256c6c640 1 -- 172.21.15.98:0/2308444715 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2250158ce0 con 0x7f225015c550 2024-09-19T08:39:24.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.991+0000 7f2256c6c640 1 -- 172.21.15.98:0/2308444715 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22500723a0 con 0x7f225015c550 2024-09-19T08:39:24.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.991+0000 7f224d7fa640 1 -- 172.21.15.98:0/2308444715 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 6) ==== 87496+0+0 (secure 0 0 0) 0x7f2248014bc0 con 0x7f225015c550 2024-09-19T08:39:24.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.991+0000 7f224d7fa640 1 --2- 172.21.15.98:0/2308444715 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f223c067950 0x7f223c069e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:24.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.991+0000 7f224d7fa640 1 -- 172.21.15.98:0/2308444715 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f22480865a0 con 0x7f225015c550 2024-09-19T08:39:24.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.991+0000 7f2254a08640 1 --2- 172.21.15.98:0/2308444715 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f223c067950 0x7f223c069e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:24.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.991+0000 7f2254a08640 1 --2- 172.21.15.98:0/2308444715 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f223c067950 0x7f223c069e10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f22400ad000 tx=0x7f22400a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:24.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:24.995+0000 7f224d7fa640 1 -- 172.21.15.98:0/2308444715 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f22480570f0 con 0x7f225015c550 2024-09-19T08:39:25.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.171+0000 7f2256c6c640 1 -- 172.21.15.98:0/2308444715 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2250157020 con 0x7f225015c550 2024-09-19T08:39:25.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.179+0000 7f224d7fa640 1 -- 172.21.15.98:0/2308444715 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f2248053060 con 0x7f225015c550 2024-09-19T08:39:25.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.183+0000 7f224d7fa640 1 -- 172.21.15.98:0/2308444715 <== mon.0 v2:172.21.15.98:3300/0 8 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v32) ==== 80+0+0 (secure 0 0 0) 0x7f2248071210 con 0x7f225015c550 2024-09-19T08:39:25.181 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-19T08:39:25.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.183+0000 7f2256c6c640 1 -- 172.21.15.98:0/2308444715 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f223c067950 msgr2=0x7f223c069e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:25.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.183+0000 7f2256c6c640 1 --2- 172.21.15.98:0/2308444715 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f223c067950 0x7f223c069e10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f22400ad000 tx=0x7f22400a6000 comp rx=0 tx=0).stop 2024-09-19T08:39:25.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.183+0000 7f2256c6c640 1 -- 172.21.15.98:0/2308444715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f225015c550 msgr2=0x7f22500aedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:25.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.183+0000 7f2256c6c640 1 --2- 172.21.15.98:0/2308444715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f225015c550 0x7f22500aedf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2248007e40 tx=0x7f224800e4b0 comp rx=0 tx=0).stop 2024-09-19T08:39:25.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.183+0000 7f2256c6c640 1 -- 172.21.15.98:0/2308444715 shutdown_connections 2024-09-19T08:39:25.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.183+0000 7f2256c6c640 1 --2- 172.21.15.98:0/2308444715 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f223c067950 0x7f223c069e10 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:25.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.183+0000 7f2256c6c640 1 --2- 172.21.15.98:0/2308444715 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f225015c550 0x7f22500aedf0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:25.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.183+0000 7f2256c6c640 1 --2- 172.21.15.98:0/2308444715 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22500cfad0 0x7f22500ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:25.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.183+0000 7f2256c6c640 1 --2- 172.21.15.98:0/2308444715 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f225006c1d0 0x7f225014fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:25.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.183+0000 7f2256c6c640 1 -- 172.21.15.98:0/2308444715 >> 172.21.15.98:0/2308444715 conn(0x7f22500b2010 msgr2=0x7f22500b0520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:25.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.183+0000 7f2256c6c640 1 -- 172.21.15.98:0/2308444715 shutdown_connections 2024-09-19T08:39:25.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.183+0000 7f2256c6c640 1 -- 172.21.15.98:0/2308444715 wait complete. 2024-09-19T08:39:25.201 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi139.front.sepia.ceph.com... 2024-09-19T08:39:25.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:25.201 DEBUG:teuthology.orchestra.run.smithi139:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-19T08:39:25.213 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[35255]: fuse finished with error 0 and tester_r 0 2024-09-19T08:39:25.215 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:39:25.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:25.215 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-19T08:39:25.318 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:39:25.318 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:39:25.318 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:39:25.325 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15846 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T08:39:25.325 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:39:25.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:25.326 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-19T08:39:25.378 INFO:teuthology.orchestra.run.smithi098.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-19T08:39:25.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:39:25.379 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:39:25.380 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:39:25.380 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:39:25.386 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T08:39:25.387 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:39:25.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:25.387 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-19T08:39:25.432 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-19T08:39:25.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:39:25.433 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:39:25.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.563+0000 7f4d4d6e3640 1 Processor -- start 2024-09-19T08:39:25.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.563+0000 7f4d4d6e3640 1 -- start start 2024-09-19T08:39:25.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.563+0000 7f4d4d6e3640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4d4806c3a0 0x7f4d480cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:25.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.563+0000 7f4d4d6e3640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d4806d310 con 0x7f4d4815c550 2024-09-19T08:39:25.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.563+0000 7f4d4d6e3640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d4806d5f0 con 0x7f4d480cf900 2024-09-19T08:39:25.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.563+0000 7f4d4d6e3640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f4d4806d8d0 con 0x7f4d4806c3a0 2024-09-19T08:39:25.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.563+0000 7f4d46ffd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f4d4815c550 0x7f4d4806bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:46328/0 (socket says 172.21.15.98:46328) 2024-09-19T08:39:25.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.563+0000 7f4d46ffd640 1 -- 172.21.15.98:0/4283444761 learned_addr learned my addr 172.21.15.98:0/4283444761 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:25.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.563+0000 7f4d45ffb640 1 -- 172.21.15.98:0/4283444761 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1025576223 0 0) 0x7f4d4806d310 con 0x7f4d4815c550 2024-09-19T08:39:25.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.563+0000 7f4d45ffb640 1 -- 172.21.15.98:0/4283444761 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d30003530 con 0x7f4d4815c550 2024-09-19T08:39:25.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.563+0000 7f4d467fc640 1 --2- 172.21.15.98:0/4283444761 >> v2:172.21.15.139:3300/0 conn(0x7f4d4806c3a0 0x7f4d480cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:25.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.563+0000 7f4d45ffb640 1 -- 172.21.15.98:0/4283444761 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2540878668 0 0) 0x7f4d30003530 con 0x7f4d4815c550 2024-09-19T08:39:25.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.563+0000 7f4d45ffb640 1 -- 172.21.15.98:0/4283444761 >> v2:172.21.15.139:3300/0 conn(0x7f4d4806c3a0 msgr2=0x7f4d480cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:25.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.563+0000 7f4d45ffb640 1 --2- 172.21.15.98:0/4283444761 >> v2:172.21.15.139:3300/0 conn(0x7f4d4806c3a0 0x7f4d480cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:25.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.563+0000 7f4d45ffb640 1 -- 172.21.15.98:0/4283444761 >> v1:172.21.15.139:6789/0 conn(0x7f4d480cf900 legacy=0x7f4d480723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:25.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.563+0000 7f4d45ffb640 1 -- 172.21.15.98:0/4283444761 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4806dbb0 con 0x7f4d4815c550 2024-09-19T08:39:25.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.563+0000 7f4d45ffb640 1 -- 172.21.15.98:0/4283444761 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f4d3c001680 con 0x7f4d4815c550 2024-09-19T08:39:25.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.563+0000 7f4d45ffb640 1 -- 172.21.15.98:0/4283444761 >> v1:172.21.15.98:6789/0 conn(0x7f4d4815c550 legacy=0x7f4d4806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:25.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.563+0000 7f4d45ffb640 1 --2- 172.21.15.98:0/4283444761 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d300039c0 0x7f4d30003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:25.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.563+0000 7f4d45ffb640 1 --2- 172.21.15.98:0/4283444761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d30004810 0x7f4d30004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:25.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.563+0000 7f4d45ffb640 1 --2- 172.21.15.98:0/4283444761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d300051d0 0x7f4d3000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:25.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.563+0000 7f4d45ffb640 1 -- 172.21.15.98:0/4283444761 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4806d8d0 con 0x7f4d300051d0 2024-09-19T08:39:25.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.563+0000 7f4d45ffb640 1 -- 172.21.15.98:0/4283444761 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3000c070 con 0x7f4d30004810 2024-09-19T08:39:25.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.563+0000 7f4d45ffb640 1 -- 172.21.15.98:0/4283444761 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4d3000c270 con 0x7f4d300039c0 2024-09-19T08:39:25.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.563+0000 7f4d477fe640 1 --2- 172.21.15.98:0/4283444761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d300051d0 0x7f4d3000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:25.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.563+0000 7f4d46ffd640 1 --2- 172.21.15.98:0/4283444761 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d300039c0 0x7f4d30003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:25.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.563+0000 7f4d477fe640 1 -- 172.21.15.98:0/4283444761 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d300039c0 msgr2=0x7f4d30003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:25.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.563+0000 7f4d477fe640 1 --2- 172.21.15.98:0/4283444761 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d300039c0 0x7f4d30003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:25.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.563+0000 7f4d477fe640 1 -- 172.21.15.98:0/4283444761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d30004810 msgr2=0x7f4d30004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:25.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.563+0000 7f4d477fe640 1 --2- 172.21.15.98:0/4283444761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d30004810 0x7f4d30004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:25.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.563+0000 7f4d477fe640 1 -- 172.21.15.98:0/4283444761 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d30003530 con 0x7f4d300051d0 2024-09-19T08:39:25.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.563+0000 7f4d46ffd640 1 --2- 172.21.15.98:0/4283444761 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d300039c0 0x7f4d30003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:25.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.567+0000 7f4d477fe640 1 -- 172.21.15.98:0/4283444761 reap_dead start 2024-09-19T08:39:25.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.567+0000 7f4d477fe640 1 --2- 172.21.15.98:0/4283444761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d300051d0 0x7f4d3000b9d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4d40001e70 tx=0x7f4d4001c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=548b0f449903d49e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:25.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.567+0000 7f4d45ffb640 1 -- 172.21.15.98:0/4283444761 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d40001a90 con 0x7f4d300051d0 2024-09-19T08:39:25.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.567+0000 7f4d45ffb640 1 -- 172.21.15.98:0/4283444761 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d40003bc0 con 0x7f4d300051d0 2024-09-19T08:39:25.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.567+0000 7f4d45ffb640 1 -- 172.21.15.98:0/4283444761 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d40005020 con 0x7f4d300051d0 2024-09-19T08:39:25.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.567+0000 7f4d4d6e3640 1 -- 172.21.15.98:0/4283444761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d300051d0 msgr2=0x7f4d3000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:25.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.567+0000 7f4d4d6e3640 1 --2- 172.21.15.98:0/4283444761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d300051d0 0x7f4d3000b9d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4d40001e70 tx=0x7f4d4001c780 comp rx=0 tx=0).stop 2024-09-19T08:39:25.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.567+0000 7f4d4d6e3640 1 -- 172.21.15.98:0/4283444761 shutdown_connections 2024-09-19T08:39:25.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.567+0000 7f4d4d6e3640 1 --2- 172.21.15.98:0/4283444761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d300051d0 0x7f4d3000b9d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:25.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.567+0000 7f4d4d6e3640 1 -- 172.21.15.98:0/4283444761 >> 172.21.15.98:0/4283444761 conn(0x7f4d480b2010 msgr2=0x7f4d480b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:25.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.567+0000 7f4d4d6e3640 1 -- 172.21.15.98:0/4283444761 shutdown_connections 2024-09-19T08:39:25.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.567+0000 7f4d4d6e3640 1 -- 172.21.15.98:0/4283444761 wait complete. 2024-09-19T08:39:25.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.567+0000 7f4d4d6e3640 1 Processor -- start 2024-09-19T08:39:25.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.567+0000 7f4d4d6e3640 1 -- start start 2024-09-19T08:39:25.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.571+0000 7f4d4d6e3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d300051d0 0x7f4d481f1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:25.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.571+0000 7f4d4d6e3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d4813e9c0 0x7f4d48150350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:25.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.571+0000 7f4d4d6e3640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d48150890 0x7f4d48152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:25.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.571+0000 7f4d4d6e3640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4d480cfd70 con 0x7f4d300051d0 2024-09-19T08:39:25.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.571+0000 7f4d4d6e3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4815cb40 con 0x7f4d4813e9c0 2024-09-19T08:39:25.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.571+0000 7f4d4d6e3640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4d480cfef0 con 0x7f4d48150890 2024-09-19T08:39:25.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.571+0000 7f4d46ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d300051d0 0x7f4d481f1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:25.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.571+0000 7f4d46ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d300051d0 0x7f4d481f1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35246/0 (socket says 172.21.15.98:35246) 2024-09-19T08:39:25.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.571+0000 7f4d46ffd640 1 -- 172.21.15.98:0/871775855 learned_addr learned my addr 172.21.15.98:0/871775855 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:25.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.571+0000 7f4d467fc640 1 --2- 172.21.15.98:0/871775855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d4813e9c0 0x7f4d48150350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:25.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.571+0000 7f4d477fe640 1 --2- 172.21.15.98:0/871775855 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d48150890 0x7f4d48152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:25.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.571+0000 7f4d46ffd640 1 -- 172.21.15.98:0/871775855 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d48150890 msgr2=0x7f4d48152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:25.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.571+0000 7f4d46ffd640 1 --2- 172.21.15.98:0/871775855 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d48150890 0x7f4d48152630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:25.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.571+0000 7f4d46ffd640 1 -- 172.21.15.98:0/871775855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d4813e9c0 msgr2=0x7f4d48150350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:25.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.571+0000 7f4d46ffd640 1 --2- 172.21.15.98:0/871775855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d4813e9c0 0x7f4d48150350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:25.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.571+0000 7f4d46ffd640 1 -- 172.21.15.98:0/871775855 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d48153360 con 0x7f4d300051d0 2024-09-19T08:39:25.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.571+0000 7f4d46ffd640 1 --2- 172.21.15.98:0/871775855 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d300051d0 0x7f4d481f1710 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c00d7e0 tx=0x7f4d3c00dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:25.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.571+0000 7f4d2f7fe640 1 -- 172.21.15.98:0/871775855 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c019070 con 0x7f4d300051d0 2024-09-19T08:39:25.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.571+0000 7f4d2f7fe640 1 -- 172.21.15.98:0/871775855 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d3c0074e0 con 0x7f4d300051d0 2024-09-19T08:39:25.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.571+0000 7f4d4d6e3640 1 -- 172.21.15.98:0/871775855 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d4815dfe0 con 0x7f4d300051d0 2024-09-19T08:39:25.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.571+0000 7f4d2f7fe640 1 -- 172.21.15.98:0/871775855 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c0049f0 con 0x7f4d300051d0 2024-09-19T08:39:25.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.571+0000 7f4d4d6e3640 1 -- 172.21.15.98:0/871775855 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d4815e5a0 con 0x7f4d300051d0 2024-09-19T08:39:25.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.571+0000 7f4d4d6e3640 1 -- 172.21.15.98:0/871775855 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d48150ca0 con 0x7f4d300051d0 2024-09-19T08:39:25.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.571+0000 7f4d2f7fe640 1 -- 172.21.15.98:0/871775855 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f4d3c01c070 con 0x7f4d300051d0 2024-09-19T08:39:25.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.571+0000 7f4d2f7fe640 1 --2- 172.21.15.98:0/871775855 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4d34067a70 0x7f4d34069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:25.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.571+0000 7f4d467fc640 1 --2- 172.21.15.98:0/871775855 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4d34067a70 0x7f4d34069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:25.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.571+0000 7f4d2f7fe640 1 -- 172.21.15.98:0/871775855 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f4d3c085b80 con 0x7f4d300051d0 2024-09-19T08:39:25.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.571+0000 7f4d467fc640 1 --2- 172.21.15.98:0/871775855 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4d34067a70 0x7f4d34069f30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4d380047c0 tx=0x7f4d38008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:25.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.575+0000 7f4d2f7fe640 1 -- 172.21.15.98:0/871775855 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f4d48150ca0 con 0x7f4d300051d0 2024-09-19T08:39:25.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.771+0000 7f4d4d6e3640 1 -- 172.21.15.98:0/871775855 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d480cf3c0 con 0x7f4d300051d0 2024-09-19T08:39:25.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.771+0000 7f4d2f7fe640 1 -- 172.21.15.98:0/871775855 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5229 (secure 0 0 0) 0x7f4d3c056330 con 0x7f4d300051d0 2024-09-19T08:39:25.772 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:39:25.772 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":15,"btime":"2024-09-19T08:39:20:116664+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4481,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3797406671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3797406671},{"type":"v1","addr":"172.21.15.98:6835","nonce":3797406671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/4112109751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4112109751},{"type":"v1","addr":"172.21.15.139:6834","nonce":4112109751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4684,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1071477338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6833","nonce":1071477338},{"type":"v1","addr":"172.21.15.139:6835","nonce":1071477338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:39:04.632502+0000","modified":"2024-09-19T08:39:06.618098+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4833},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4833":{"gid":4833,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/1820856497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1820856497},{"type":"v1","addr":"172.21.15.98:6837","nonce":1820856497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4833,"qdb_cluster":[4833]},"id":2}]} 2024-09-19T08:39:25.773 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 15 2024-09-19T08:39:25.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.775+0000 7f4d4d6e3640 1 -- 172.21.15.98:0/871775855 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4d34067a70 msgr2=0x7f4d34069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:25.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.775+0000 7f4d4d6e3640 1 --2- 172.21.15.98:0/871775855 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4d34067a70 0x7f4d34069f30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4d380047c0 tx=0x7f4d38008040 comp rx=0 tx=0).stop 2024-09-19T08:39:25.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.775+0000 7f4d4d6e3640 1 -- 172.21.15.98:0/871775855 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d300051d0 msgr2=0x7f4d481f1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:25.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.775+0000 7f4d4d6e3640 1 --2- 172.21.15.98:0/871775855 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d300051d0 0x7f4d481f1710 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c00d7e0 tx=0x7f4d3c00dca0 comp rx=0 tx=0).stop 2024-09-19T08:39:25.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.775+0000 7f4d4d6e3640 1 -- 172.21.15.98:0/871775855 shutdown_connections 2024-09-19T08:39:25.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.775+0000 7f4d4d6e3640 1 --2- 172.21.15.98:0/871775855 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4d34067a70 0x7f4d34069f30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:25.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.775+0000 7f4d4d6e3640 1 --2- 172.21.15.98:0/871775855 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d48150890 0x7f4d48152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:25.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.775+0000 7f4d4d6e3640 1 --2- 172.21.15.98:0/871775855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d4813e9c0 0x7f4d48150350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:25.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.775+0000 7f4d4d6e3640 1 --2- 172.21.15.98:0/871775855 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d300051d0 0x7f4d481f1710 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:25.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.775+0000 7f4d4d6e3640 1 -- 172.21.15.98:0/871775855 >> 172.21.15.98:0/871775855 conn(0x7f4d480b2010 msgr2=0x7f4d48072840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:25.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.775+0000 7f4d4d6e3640 1 -- 172.21.15.98:0/871775855 shutdown_connections 2024-09-19T08:39:25.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.775+0000 7f4d4d6e3640 1 -- 172.21.15.98:0/871775855 wait complete. 2024-09-19T08:39:25.794 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:39:25.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.915+0000 7f79b5f26640 1 Processor -- start 2024-09-19T08:39:25.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.919+0000 7f79b5f26640 1 -- start start 2024-09-19T08:39:25.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.919+0000 7f79b5f26640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79b00aff00 0x7f79b00cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:25.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.919+0000 7f79b5f26640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f79b00cfba0 0x7f79b00cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:25.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.919+0000 7f79b5f26640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79b0072ad0 con 0x7f79b006f9a0 2024-09-19T08:39:25.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.919+0000 7f79b5f26640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f79b00d0010 con 0x7f79b00cfba0 2024-09-19T08:39:25.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.919+0000 7f79b5f26640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f79b00cb440 con 0x7f79b00aff00 2024-09-19T08:39:25.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.919+0000 7f79affff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f79b00cfba0 0x7f79b00cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:25.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.919+0000 7f79af7fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f79b006f9a0 0x7f79b006da70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:46344/0 (socket says 172.21.15.98:46344) 2024-09-19T08:39:25.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.919+0000 7f79af7fe640 1 -- 172.21.15.98:0/1024208596 learned_addr learned my addr 172.21.15.98:0/1024208596 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:25.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.919+0000 7f79a7fff640 1 --2- 172.21.15.98:0/1024208596 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79b00aff00 0x7f79b00cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:25.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.919+0000 7f79affff640 1 -- 172.21.15.98:0/1024208596 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79b00aff00 msgr2=0x7f79b00cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:25.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.919+0000 7f79affff640 1 --2- 172.21.15.98:0/1024208596 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79b00aff00 0x7f79b00cf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:25.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.919+0000 7f79affff640 1 -- 172.21.15.98:0/1024208596 >> v1:172.21.15.98:6789/0 conn(0x7f79b006f9a0 legacy=0x7f79b006da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:25.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.919+0000 7f79affff640 1 -- 172.21.15.98:0/1024208596 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b00cb720 con 0x7f79b00cfba0 2024-09-19T08:39:25.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.919+0000 7f79aeffd640 1 -- 172.21.15.98:0/1024208596 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3389745726 0 0) 0x7f79b0072ad0 con 0x7f79b006f9a0 2024-09-19T08:39:25.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.919+0000 7f79a7fff640 1 --2- 172.21.15.98:0/1024208596 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79b00aff00 0x7f79b00cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:39:25.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.919+0000 7f79affff640 1 --2- 172.21.15.98:0/1024208596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79b00cfba0 0x7f79b00cc4e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f79a8009d90 tx=0x7f79a801c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d1996ab21a3a6d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:25.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.919+0000 7f79aeffd640 1 -- 172.21.15.98:0/1024208596 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a80b2070 con 0x7f79b00cfba0 2024-09-19T08:39:25.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.919+0000 7f79aeffd640 1 -- 172.21.15.98:0/1024208596 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79a80a5710 con 0x7f79b00cfba0 2024-09-19T08:39:25.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.923+0000 7f79aeffd640 1 -- 172.21.15.98:0/1024208596 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a80a5ae0 con 0x7f79b00cfba0 2024-09-19T08:39:25.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.923+0000 7f79b5f26640 1 -- 172.21.15.98:0/1024208596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79b00cfba0 msgr2=0x7f79b00cc4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:25.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.923+0000 7f79b5f26640 1 --2- 172.21.15.98:0/1024208596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79b00cfba0 0x7f79b00cc4e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f79a8009d90 tx=0x7f79a801c850 comp rx=0 tx=0).stop 2024-09-19T08:39:25.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.923+0000 7f79b5f26640 1 -- 172.21.15.98:0/1024208596 shutdown_connections 2024-09-19T08:39:25.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.923+0000 7f79b5f26640 1 --2- 172.21.15.98:0/1024208596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79b00cfba0 0x7f79b00cc4e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:25.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.923+0000 7f79b5f26640 1 --2- 172.21.15.98:0/1024208596 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79b00aff00 0x7f79b00cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:25.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.923+0000 7f79b5f26640 1 -- 172.21.15.98:0/1024208596 >> 172.21.15.98:0/1024208596 conn(0x7f79b00d0980 msgr2=0x7f79b00d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:25.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.923+0000 7f79b5f26640 1 -- 172.21.15.98:0/1024208596 shutdown_connections 2024-09-19T08:39:25.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.923+0000 7f79b5f26640 1 -- 172.21.15.98:0/1024208596 wait complete. 2024-09-19T08:39:25.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.923+0000 7f79b5f26640 1 Processor -- start 2024-09-19T08:39:25.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.923+0000 7f79b5f26640 1 -- start start 2024-09-19T08:39:25.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.923+0000 7f79b5f26640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79b006f9a0 0x7f79b0154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:25.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.923+0000 7f79b5f26640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79b00aff00 0x7f79b0151db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:25.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.923+0000 7f79b5f26640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79b00cfba0 0x7f79b0154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:25.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.923+0000 7f79b5f26640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f79b006ff00 con 0x7f79b00aff00 2024-09-19T08:39:25.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.923+0000 7f79b5f26640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f79b006fd80 con 0x7f79b006f9a0 2024-09-19T08:39:25.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.923+0000 7f79b5f26640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f79b00cafd0 con 0x7f79b00cfba0 2024-09-19T08:39:25.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.923+0000 7f79a7fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79b00aff00 0x7f79b0151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:25.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.923+0000 7f79a7fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79b00aff00 0x7f79b0151db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35268/0 (socket says 172.21.15.98:35268) 2024-09-19T08:39:25.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.923+0000 7f79a7fff640 1 -- 172.21.15.98:0/901722582 learned_addr learned my addr 172.21.15.98:0/901722582 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:25.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.923+0000 7f79af7fe640 1 --2- 172.21.15.98:0/901722582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79b006f9a0 0x7f79b0154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:25.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.923+0000 7f79affff640 1 --2- 172.21.15.98:0/901722582 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79b00cfba0 0x7f79b0154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:25.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.923+0000 7f79a7fff640 1 -- 172.21.15.98:0/901722582 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79b00cfba0 msgr2=0x7f79b0154fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:25.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.923+0000 7f79a7fff640 1 --2- 172.21.15.98:0/901722582 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79b00cfba0 0x7f79b0154fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:25.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.923+0000 7f79a7fff640 1 -- 172.21.15.98:0/901722582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79b006f9a0 msgr2=0x7f79b0154480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:25.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.923+0000 7f79a7fff640 1 --2- 172.21.15.98:0/901722582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79b006f9a0 0x7f79b0154480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:25.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.923+0000 7f79a7fff640 1 -- 172.21.15.98:0/901722582 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79b0153240 con 0x7f79b00aff00 2024-09-19T08:39:25.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.927+0000 7f79a7fff640 1 --2- 172.21.15.98:0/901722582 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79b00aff00 0x7f79b0151db0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f799800d800 tx=0x7f799800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:25.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.927+0000 7f79acff9640 1 -- 172.21.15.98:0/901722582 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7998018020 con 0x7f79b00aff00 2024-09-19T08:39:25.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.927+0000 7f79acff9640 1 -- 172.21.15.98:0/901722582 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79980096c0 con 0x7f79b00aff00 2024-09-19T08:39:25.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.927+0000 7f79b5f26640 1 -- 172.21.15.98:0/901722582 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79b0153520 con 0x7f79b00aff00 2024-09-19T08:39:25.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.927+0000 7f79acff9640 1 -- 172.21.15.98:0/901722582 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7998009ab0 con 0x7f79b00aff00 2024-09-19T08:39:25.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.927+0000 7f79b5f26640 1 -- 172.21.15.98:0/901722582 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79b0159a70 con 0x7f79b00aff00 2024-09-19T08:39:25.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.927+0000 7f79b5f26640 1 -- 172.21.15.98:0/901722582 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79b00cc4e0 con 0x7f79b00aff00 2024-09-19T08:39:25.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.927+0000 7f79acff9640 1 -- 172.21.15.98:0/901722582 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f799801b020 con 0x7f79b00aff00 2024-09-19T08:39:25.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.927+0000 7f79acff9640 1 --2- 172.21.15.98:0/901722582 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7990067a70 0x7f7990069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:25.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.927+0000 7f79af7fe640 1 --2- 172.21.15.98:0/901722582 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7990067a70 0x7f7990069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:25.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.927+0000 7f79acff9640 1 -- 172.21.15.98:0/901722582 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f79980861b0 con 0x7f79b00aff00 2024-09-19T08:39:25.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.927+0000 7f79af7fe640 1 --2- 172.21.15.98:0/901722582 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7990067a70 0x7f7990069f30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f79a0000f70 tx=0x7f79a0009e90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:25.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:25.931+0000 7f79acff9640 1 -- 172.21.15.98:0/901722582 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f7998022030 con 0x7f79b00aff00 2024-09-19T08:39:26.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.115+0000 7f79b5f26640 1 -- 172.21.15.98:0/901722582 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79b0072cc0 con 0x7f79b00aff00 2024-09-19T08:39:26.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.115+0000 7f79acff9640 1 -- 172.21.15.98:0/901722582 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5229 (secure 0 0 0) 0x7f7998056b00 con 0x7f79b00aff00 2024-09-19T08:39:26.114 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:39:26.114 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":15,"btime":"2024-09-19T08:39:20:116664+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4481,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3797406671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3797406671},{"type":"v1","addr":"172.21.15.98:6835","nonce":3797406671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/4112109751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4112109751},{"type":"v1","addr":"172.21.15.139:6834","nonce":4112109751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4684,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1071477338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6833","nonce":1071477338},{"type":"v1","addr":"172.21.15.139:6835","nonce":1071477338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:39:04.632502+0000","modified":"2024-09-19T08:39:06.618098+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4833},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4833":{"gid":4833,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/1820856497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1820856497},{"type":"v1","addr":"172.21.15.98:6837","nonce":1820856497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4833,"qdb_cluster":[4833]},"id":2}]} 2024-09-19T08:39:26.114 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 15 2024-09-19T08:39:26.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.115+0000 7f79b5f26640 1 -- 172.21.15.98:0/901722582 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7990067a70 msgr2=0x7f7990069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:26.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.115+0000 7f79b5f26640 1 --2- 172.21.15.98:0/901722582 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7990067a70 0x7f7990069f30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f79a0000f70 tx=0x7f79a0009e90 comp rx=0 tx=0).stop 2024-09-19T08:39:26.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.115+0000 7f79b5f26640 1 -- 172.21.15.98:0/901722582 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79b00aff00 msgr2=0x7f79b0151db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:26.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.115+0000 7f79b5f26640 1 --2- 172.21.15.98:0/901722582 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79b00aff00 0x7f79b0151db0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f799800d800 tx=0x7f799800dcc0 comp rx=0 tx=0).stop 2024-09-19T08:39:26.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.115+0000 7f79b5f26640 1 -- 172.21.15.98:0/901722582 shutdown_connections 2024-09-19T08:39:26.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.115+0000 7f79b5f26640 1 --2- 172.21.15.98:0/901722582 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7990067a70 0x7f7990069f30 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:26.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.115+0000 7f79b5f26640 1 --2- 172.21.15.98:0/901722582 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79b00cfba0 0x7f79b0154fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:26.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.115+0000 7f79b5f26640 1 --2- 172.21.15.98:0/901722582 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79b00aff00 0x7f79b0151db0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:26.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.115+0000 7f79b5f26640 1 --2- 172.21.15.98:0/901722582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79b006f9a0 0x7f79b0154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:26.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.115+0000 7f79b5f26640 1 -- 172.21.15.98:0/901722582 >> 172.21.15.98:0/901722582 conn(0x7f79b00d0980 msgr2=0x7f79b00ce530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:26.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.115+0000 7f79b5f26640 1 -- 172.21.15.98:0/901722582 shutdown_connections 2024-09-19T08:39:26.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.115+0000 7f79b5f26640 1 -- 172.21.15.98:0/901722582 wait complete. 2024-09-19T08:39:26.132 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:39:26.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.235+0000 7f79adb6d640 1 Processor -- start 2024-09-19T08:39:26.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.235+0000 7f79adb6d640 1 -- start start 2024-09-19T08:39:26.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.239+0000 7f79adb6d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79a815c550 0x7f79a806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:26.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.239+0000 7f79adb6d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f79a806c1d0 0x7f79a80cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:26.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.239+0000 7f79adb6d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79a806d310 con 0x7f79a80cf900 2024-09-19T08:39:26.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.239+0000 7f79adb6d640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f79a806c610 con 0x7f79a806c1d0 2024-09-19T08:39:26.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.239+0000 7f79adb6d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f79a806d5f0 con 0x7f79a815c550 2024-09-19T08:39:26.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.239+0000 7f799effd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f79a806c1d0 0x7f79a80cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:26.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.239+0000 7f799effd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f79a806c1d0 0x7f79a80cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35274/0 (socket says 172.21.15.98:35274) 2024-09-19T08:39:26.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.239+0000 7f799effd640 1 -- 172.21.15.98:0/3443976658 learned_addr learned my addr 172.21.15.98:0/3443976658 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:26.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.239+0000 7f79a77fe640 1 --2- 172.21.15.98:0/3443976658 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79a815c550 0x7f79a806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:26.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.239+0000 7f799effd640 1 -- 172.21.15.98:0/3443976658 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79a815c550 msgr2=0x7f79a806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:26.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.239+0000 7f799effd640 1 --2- 172.21.15.98:0/3443976658 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79a815c550 0x7f79a806bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:26.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.239+0000 7f799effd640 1 -- 172.21.15.98:0/3443976658 >> v1:172.21.15.139:6789/0 conn(0x7f79a80cf900 legacy=0x7f79a80ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:26.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.239+0000 7f799effd640 1 -- 172.21.15.98:0/3443976658 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a806d8d0 con 0x7f79a806c1d0 2024-09-19T08:39:26.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.239+0000 7f799effd640 1 --2- 172.21.15.98:0/3443976658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79a806c1d0 0x7f79a80cf3c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7990009870 tx=0x7f799001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a2b9e06629af5e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:26.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.239+0000 7f79a6ffd640 1 -- 172.21.15.98:0/3443976658 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79900b0070 con 0x7f79a806c1d0 2024-09-19T08:39:26.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.239+0000 7f79a6ffd640 1 -- 172.21.15.98:0/3443976658 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79900ab600 con 0x7f79a806c1d0 2024-09-19T08:39:26.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.239+0000 7f79a6ffd640 1 -- 172.21.15.98:0/3443976658 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79900ab9d0 con 0x7f79a806c1d0 2024-09-19T08:39:26.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.243+0000 7f79adb6d640 1 -- 172.21.15.98:0/3443976658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79a806c1d0 msgr2=0x7f79a80cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:26.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.243+0000 7f79adb6d640 1 --2- 172.21.15.98:0/3443976658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79a806c1d0 0x7f79a80cf3c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7990009870 tx=0x7f799001c520 comp rx=0 tx=0).stop 2024-09-19T08:39:26.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.243+0000 7f79adb6d640 1 -- 172.21.15.98:0/3443976658 shutdown_connections 2024-09-19T08:39:26.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.243+0000 7f79adb6d640 1 --2- 172.21.15.98:0/3443976658 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79a806c1d0 0x7f79a80cf3c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:26.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.243+0000 7f79adb6d640 1 --2- 172.21.15.98:0/3443976658 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79a815c550 0x7f79a806bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:26.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.243+0000 7f79adb6d640 1 -- 172.21.15.98:0/3443976658 >> 172.21.15.98:0/3443976658 conn(0x7f79a80b2010 msgr2=0x7f79a80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:26.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.243+0000 7f79adb6d640 1 -- 172.21.15.98:0/3443976658 shutdown_connections 2024-09-19T08:39:26.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.243+0000 7f79adb6d640 1 -- 172.21.15.98:0/3443976658 wait complete. 2024-09-19T08:39:26.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.243+0000 7f79adb6d640 1 Processor -- start 2024-09-19T08:39:26.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.243+0000 7f79adb6d640 1 -- start start 2024-09-19T08:39:26.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.243+0000 7f79adb6d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79a806c1d0 0x7f79a8149ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:26.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.243+0000 7f79adb6d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79a80cf900 0x7f79a814a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:26.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.243+0000 7f79adb6d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79a815c550 0x7f79a814d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:26.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.243+0000 7f79adb6d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f79a8169230 con 0x7f79a815c550 2024-09-19T08:39:26.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.243+0000 7f79adb6d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f79a815cb40 con 0x7f79a80cf900 2024-09-19T08:39:26.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.243+0000 7f79adb6d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f79a81693b0 con 0x7f79a806c1d0 2024-09-19T08:39:26.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.243+0000 7f79a7fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79a815c550 0x7f79a814d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:26.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.243+0000 7f79a7fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79a815c550 0x7f79a814d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35278/0 (socket says 172.21.15.98:35278) 2024-09-19T08:39:26.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.243+0000 7f79a7fff640 1 -- 172.21.15.98:0/3872227766 learned_addr learned my addr 172.21.15.98:0/3872227766 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:26.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.243+0000 7f79a77fe640 1 --2- 172.21.15.98:0/3872227766 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79a806c1d0 0x7f79a8149ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:26.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.243+0000 7f799effd640 1 --2- 172.21.15.98:0/3872227766 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79a80cf900 0x7f79a814a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:26.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.243+0000 7f79a7fff640 1 -- 172.21.15.98:0/3872227766 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79a806c1d0 msgr2=0x7f79a8149ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:26.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.243+0000 7f79a7fff640 1 --2- 172.21.15.98:0/3872227766 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79a806c1d0 0x7f79a8149ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:26.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.243+0000 7f79a7fff640 1 -- 172.21.15.98:0/3872227766 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79a80cf900 msgr2=0x7f79a814a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:26.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.243+0000 7f79a7fff640 1 --2- 172.21.15.98:0/3872227766 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79a80cf900 0x7f79a814a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:26.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.243+0000 7f79a7fff640 1 -- 172.21.15.98:0/3872227766 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a8155a40 con 0x7f79a815c550 2024-09-19T08:39:26.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.243+0000 7f799effd640 1 --2- 172.21.15.98:0/3872227766 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79a80cf900 0x7f79a814a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:39:26.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.243+0000 7f79a7fff640 1 --2- 172.21.15.98:0/3872227766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79a815c550 0x7f79a814d0c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f79a000c7b0 tx=0x7f79a000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:26.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.243+0000 7f79a4ff9640 1 -- 172.21.15.98:0/3872227766 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a0004b70 con 0x7f79a815c550 2024-09-19T08:39:26.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.243+0000 7f79a4ff9640 1 -- 172.21.15.98:0/3872227766 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79a0004d10 con 0x7f79a815c550 2024-09-19T08:39:26.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.243+0000 7f79adb6d640 1 -- 172.21.15.98:0/3872227766 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79a8155d20 con 0x7f79a815c550 2024-09-19T08:39:26.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.243+0000 7f79a4ff9640 1 -- 172.21.15.98:0/3872227766 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a000aa90 con 0x7f79a815c550 2024-09-19T08:39:26.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.243+0000 7f79adb6d640 1 -- 172.21.15.98:0/3872227766 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79a81556c0 con 0x7f79a815c550 2024-09-19T08:39:26.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.247+0000 7f79adb6d640 1 -- 172.21.15.98:0/3872227766 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79a815ccc0 con 0x7f79a815c550 2024-09-19T08:39:26.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.247+0000 7f79a4ff9640 1 -- 172.21.15.98:0/3872227766 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f79a000ac30 con 0x7f79a815c550 2024-09-19T08:39:26.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.247+0000 7f79a4ff9640 1 --2- 172.21.15.98:0/3872227766 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7988067a70 0x7f7988069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:26.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.247+0000 7f79a4ff9640 1 -- 172.21.15.98:0/3872227766 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f79a0086220 con 0x7f79a815c550 2024-09-19T08:39:26.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.247+0000 7f79a77fe640 1 --2- 172.21.15.98:0/3872227766 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7988067a70 0x7f7988069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:26.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.247+0000 7f79a77fe640 1 --2- 172.21.15.98:0/3872227766 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7988067a70 0x7f7988069f30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f799800aae0 tx=0x7f7998008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:26.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.251+0000 7f79a4ff9640 1 -- 172.21.15.98:0/3872227766 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f79a0020070 con 0x7f79a815c550 2024-09-19T08:39:26.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.407+0000 7f79adb6d640 1 -- 172.21.15.98:0/3872227766 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f79a80cf3c0 con 0x7f79a815c550 2024-09-19T08:39:26.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.407+0000 7f79a4ff9640 1 -- 172.21.15.98:0/3872227766 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16729 (secure 0 0 0) 0x7f79a0056a60 con 0x7f79a815c550 2024-09-19T08:39:26.409 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:39:26.409 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":32,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:39:06.263597+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:39:02.429044+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-19T08:39:03.620637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:39:26.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.411+0000 7f79adb6d640 1 -- 172.21.15.98:0/3872227766 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7988067a70 msgr2=0x7f7988069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:26.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.411+0000 7f79adb6d640 1 --2- 172.21.15.98:0/3872227766 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7988067a70 0x7f7988069f30 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f799800aae0 tx=0x7f7998008040 comp rx=0 tx=0).stop 2024-09-19T08:39:26.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.411+0000 7f79adb6d640 1 -- 172.21.15.98:0/3872227766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79a815c550 msgr2=0x7f79a814d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:26.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.411+0000 7f79adb6d640 1 --2- 172.21.15.98:0/3872227766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79a815c550 0x7f79a814d0c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f79a000c7b0 tx=0x7f79a000cc70 comp rx=0 tx=0).stop 2024-09-19T08:39:26.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.411+0000 7f79adb6d640 1 -- 172.21.15.98:0/3872227766 shutdown_connections 2024-09-19T08:39:26.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.411+0000 7f79adb6d640 1 --2- 172.21.15.98:0/3872227766 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7988067a70 0x7f7988069f30 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:26.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.411+0000 7f79adb6d640 1 --2- 172.21.15.98:0/3872227766 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79a815c550 0x7f79a814d0c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:26.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.411+0000 7f79adb6d640 1 --2- 172.21.15.98:0/3872227766 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79a80cf900 0x7f79a814a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:26.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.411+0000 7f79adb6d640 1 --2- 172.21.15.98:0/3872227766 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79a806c1d0 0x7f79a8149ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:26.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.411+0000 7f79adb6d640 1 -- 172.21.15.98:0/3872227766 >> 172.21.15.98:0/3872227766 conn(0x7f79a80b2010 msgr2=0x7f79a8157360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:26.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.411+0000 7f79adb6d640 1 -- 172.21.15.98:0/3872227766 shutdown_connections 2024-09-19T08:39:26.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.411+0000 7f79adb6d640 1 -- 172.21.15.98:0/3872227766 wait complete. 2024-09-19T08:39:26.426 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T08:39:26.426 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:39:26.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.547+0000 7ff3f2695640 1 Processor -- start 2024-09-19T08:39:26.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.547+0000 7ff3f2695640 1 -- start start 2024-09-19T08:39:26.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.547+0000 7ff3f2695640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff3ec06c3a0 0x7ff3ec0cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:26.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.547+0000 7ff3f2695640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3ec0cf900 0x7ff3ec0723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:26.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.547+0000 7ff3f2695640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3ec06d310 con 0x7ff3ec15c550 2024-09-19T08:39:26.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.547+0000 7ff3f2695640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7ff3ec06d5f0 con 0x7ff3ec06c3a0 2024-09-19T08:39:26.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.547+0000 7ff3f2695640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff3ec06d8d0 con 0x7ff3ec0cf900 2024-09-19T08:39:26.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.547+0000 7ff3eb7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff3ec06c3a0 0x7ff3ec0cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:26.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.547+0000 7ff3eb7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff3ec06c3a0 0x7ff3ec0cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35284/0 (socket says 172.21.15.98:35284) 2024-09-19T08:39:26.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.547+0000 7ff3eb7fe640 1 -- 172.21.15.98:0/2789695714 learned_addr learned my addr 172.21.15.98:0/2789695714 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:26.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.547+0000 7ff3f0c32640 1 --2- 172.21.15.98:0/2789695714 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3ec0cf900 0x7ff3ec0723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:26.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.547+0000 7ff3eb7fe640 1 -- 172.21.15.98:0/2789695714 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3ec0cf900 msgr2=0x7ff3ec0723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:26.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.547+0000 7ff3eb7fe640 1 --2- 172.21.15.98:0/2789695714 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3ec0cf900 0x7ff3ec0723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:26.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.547+0000 7ff3eb7fe640 1 -- 172.21.15.98:0/2789695714 >> v1:172.21.15.139:6789/0 conn(0x7ff3ec15c550 legacy=0x7ff3ec06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:26.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.547+0000 7ff3eb7fe640 1 -- 172.21.15.98:0/2789695714 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3ec06dbb0 con 0x7ff3ec06c3a0 2024-09-19T08:39:26.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.551+0000 7ff3eb7fe640 1 --2- 172.21.15.98:0/2789695714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3ec06c3a0 0x7ff3ec0cf3c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0009870 tx=0x7ff3e001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a529585f662edf10 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:26.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.551+0000 7ff3eaffd640 1 -- 172.21.15.98:0/2789695714 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e00a7070 con 0x7ff3ec06c3a0 2024-09-19T08:39:26.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.551+0000 7ff3eaffd640 1 -- 172.21.15.98:0/2789695714 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3e00b0070 con 0x7ff3ec06c3a0 2024-09-19T08:39:26.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.551+0000 7ff3eaffd640 1 -- 172.21.15.98:0/2789695714 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e00ab890 con 0x7ff3ec06c3a0 2024-09-19T08:39:26.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.551+0000 7ff3f2695640 1 -- 172.21.15.98:0/2789695714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3ec06c3a0 msgr2=0x7ff3ec0cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:26.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.551+0000 7ff3f2695640 1 --2- 172.21.15.98:0/2789695714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3ec06c3a0 0x7ff3ec0cf3c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0009870 tx=0x7ff3e001c560 comp rx=0 tx=0).stop 2024-09-19T08:39:26.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.551+0000 7ff3f2695640 1 -- 172.21.15.98:0/2789695714 shutdown_connections 2024-09-19T08:39:26.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.551+0000 7ff3f2695640 1 --2- 172.21.15.98:0/2789695714 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3ec0cf900 0x7ff3ec0723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:26.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.551+0000 7ff3f2695640 1 --2- 172.21.15.98:0/2789695714 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3ec06c3a0 0x7ff3ec0cf3c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:26.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.551+0000 7ff3f2695640 1 -- 172.21.15.98:0/2789695714 >> 172.21.15.98:0/2789695714 conn(0x7ff3ec0b2010 msgr2=0x7ff3ec0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:26.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.551+0000 7ff3f2695640 1 -- 172.21.15.98:0/2789695714 shutdown_connections 2024-09-19T08:39:26.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.551+0000 7ff3f2695640 1 -- 172.21.15.98:0/2789695714 wait complete. 2024-09-19T08:39:26.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.551+0000 7ff3f2695640 1 Processor -- start 2024-09-19T08:39:26.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.551+0000 7ff3f2695640 1 -- start start 2024-09-19T08:39:26.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.555+0000 7ff3f2695640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3ec06c3a0 0x7ff3ec1528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:26.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.555+0000 7ff3f2695640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3ec0cf900 0x7ff3ec0b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:26.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.555+0000 7ff3f2695640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3ec15c550 0x7ff3ec0b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:26.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.555+0000 7ff3f2695640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ec073500 con 0x7ff3ec0cf900 2024-09-19T08:39:26.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.555+0000 7ff3f2695640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ec073380 con 0x7ff3ec15c550 2024-09-19T08:39:26.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.555+0000 7ff3f2695640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff3ec073680 con 0x7ff3ec06c3a0 2024-09-19T08:39:26.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.555+0000 7ff3eb7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3ec0cf900 0x7ff3ec0b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:26.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.555+0000 7ff3eb7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3ec0cf900 0x7ff3ec0b0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35300/0 (socket says 172.21.15.98:35300) 2024-09-19T08:39:26.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.555+0000 7ff3eb7fe640 1 -- 172.21.15.98:0/365173624 learned_addr learned my addr 172.21.15.98:0/365173624 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:26.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.555+0000 7ff3ebfff640 1 --2- 172.21.15.98:0/365173624 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3ec06c3a0 0x7ff3ec1528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:26.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.555+0000 7ff3f0c32640 1 --2- 172.21.15.98:0/365173624 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3ec15c550 0x7ff3ec0b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:26.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.555+0000 7ff3eb7fe640 1 -- 172.21.15.98:0/365173624 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3ec06c3a0 msgr2=0x7ff3ec1528a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:26.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.555+0000 7ff3eb7fe640 1 --2- 172.21.15.98:0/365173624 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3ec06c3a0 0x7ff3ec1528a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:26.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.555+0000 7ff3eb7fe640 1 -- 172.21.15.98:0/365173624 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3ec15c550 msgr2=0x7ff3ec0b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:26.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.555+0000 7ff3eb7fe640 1 --2- 172.21.15.98:0/365173624 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3ec15c550 0x7ff3ec0b1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:26.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.555+0000 7ff3eb7fe640 1 -- 172.21.15.98:0/365173624 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3ec1551b0 con 0x7ff3ec0cf900 2024-09-19T08:39:26.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.555+0000 7ff3f0c32640 1 --2- 172.21.15.98:0/365173624 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3ec15c550 0x7ff3ec0b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:26.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.555+0000 7ff3eb7fe640 1 --2- 172.21.15.98:0/365173624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3ec0cf900 0x7ff3ec0b0ef0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff3e001ac10 tx=0x7ff3e001ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:26.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.555+0000 7ff3e8ff9640 1 -- 172.21.15.98:0/365173624 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e00be5f0 con 0x7ff3ec0cf900 2024-09-19T08:39:26.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.555+0000 7ff3e8ff9640 1 -- 172.21.15.98:0/365173624 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3e00be7b0 con 0x7ff3ec0cf900 2024-09-19T08:39:26.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.555+0000 7ff3e8ff9640 1 -- 172.21.15.98:0/365173624 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e00beba0 con 0x7ff3ec0cf900 2024-09-19T08:39:26.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.555+0000 7ff3f2695640 1 -- 172.21.15.98:0/365173624 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3ec152de0 con 0x7ff3ec0cf900 2024-09-19T08:39:26.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.555+0000 7ff3f2695640 1 -- 172.21.15.98:0/365173624 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3ec14ea40 con 0x7ff3ec0cf900 2024-09-19T08:39:26.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.555+0000 7ff3f2695640 1 -- 172.21.15.98:0/365173624 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3ec0cf3c0 con 0x7ff3ec0cf900 2024-09-19T08:39:26.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.555+0000 7ff3e8ff9640 1 -- 172.21.15.98:0/365173624 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7ff3e00b43f0 con 0x7ff3ec0cf900 2024-09-19T08:39:26.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.555+0000 7ff3e8ff9640 1 --2- 172.21.15.98:0/365173624 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff3d8067a70 0x7ff3d8069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:26.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.559+0000 7ff3ebfff640 1 --2- 172.21.15.98:0/365173624 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff3d8067a70 0x7ff3d8069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:26.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.559+0000 7ff3e8ff9640 1 -- 172.21.15.98:0/365173624 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7ff3e0120270 con 0x7ff3ec0cf900 2024-09-19T08:39:26.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.559+0000 7ff3ebfff640 1 --2- 172.21.15.98:0/365173624 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff3d8067a70 0x7ff3d8069f30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc001cb0 tx=0x7ff3dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:26.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.563+0000 7ff3e8ff9640 1 -- 172.21.15.98:0/365173624 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff3e0016150 con 0x7ff3ec0cf900 2024-09-19T08:39:26.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.759+0000 7ff3f2695640 1 -- 172.21.15.98:0/365173624 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3ec072570 con 0x7ff3ec0cf900 2024-09-19T08:39:26.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.759+0000 7ff3e8ff9640 1 -- 172.21.15.98:0/365173624 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5229 (secure 0 0 0) 0x7ff3e00f0ab0 con 0x7ff3ec0cf900 2024-09-19T08:39:26.760 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:39:26.760 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":15,"btime":"2024-09-19T08:39:20:116664+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4481,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3797406671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3797406671},{"type":"v1","addr":"172.21.15.98:6835","nonce":3797406671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/4112109751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4112109751},{"type":"v1","addr":"172.21.15.139:6834","nonce":4112109751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4684,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1071477338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6833","nonce":1071477338},{"type":"v1","addr":"172.21.15.139:6835","nonce":1071477338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:39:04.632502+0000","modified":"2024-09-19T08:39:06.618098+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4833},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4833":{"gid":4833,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/1820856497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1820856497},{"type":"v1","addr":"172.21.15.98:6837","nonce":1820856497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4833,"qdb_cluster":[4833]},"id":2}]} 2024-09-19T08:39:26.760 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 15 2024-09-19T08:39:26.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.763+0000 7ff3f2695640 1 -- 172.21.15.98:0/365173624 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff3d8067a70 msgr2=0x7ff3d8069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:26.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.763+0000 7ff3f2695640 1 --2- 172.21.15.98:0/365173624 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff3d8067a70 0x7ff3d8069f30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc001cb0 tx=0x7ff3dc009040 comp rx=0 tx=0).stop 2024-09-19T08:39:26.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.763+0000 7ff3f2695640 1 -- 172.21.15.98:0/365173624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3ec0cf900 msgr2=0x7ff3ec0b0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:26.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.763+0000 7ff3f2695640 1 --2- 172.21.15.98:0/365173624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3ec0cf900 0x7ff3ec0b0ef0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff3e001ac10 tx=0x7ff3e001ada0 comp rx=0 tx=0).stop 2024-09-19T08:39:26.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.763+0000 7ff3f2695640 1 -- 172.21.15.98:0/365173624 shutdown_connections 2024-09-19T08:39:26.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.763+0000 7ff3f2695640 1 --2- 172.21.15.98:0/365173624 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff3d8067a70 0x7ff3d8069f30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:26.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.763+0000 7ff3f2695640 1 --2- 172.21.15.98:0/365173624 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3ec15c550 0x7ff3ec0b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:26.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.763+0000 7ff3f2695640 1 --2- 172.21.15.98:0/365173624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3ec0cf900 0x7ff3ec0b0ef0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:26.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.763+0000 7ff3f2695640 1 --2- 172.21.15.98:0/365173624 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3ec06c3a0 0x7ff3ec1528a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:26.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.763+0000 7ff3f2695640 1 -- 172.21.15.98:0/365173624 >> 172.21.15.98:0/365173624 conn(0x7ff3ec0b2010 msgr2=0x7ff3ec0d0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:26.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.763+0000 7ff3f2695640 1 -- 172.21.15.98:0/365173624 shutdown_connections 2024-09-19T08:39:26.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.763+0000 7ff3f2695640 1 -- 172.21.15.98:0/365173624 wait complete. 2024-09-19T08:39:26.779 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:39:26.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.895+0000 7f6a31d22640 1 Processor -- start 2024-09-19T08:39:26.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.895+0000 7f6a31d22640 1 -- start start 2024-09-19T08:39:26.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.899+0000 7f6a31d22640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6a2c160740 0x7f6a2c163170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:26.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.899+0000 7f6a31d22640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6a2c1636b0 0x7f6a2c06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:26.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.899+0000 7f6a31d22640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a2c06c1d0 0x7f6a2c06d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:26.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.899+0000 7f6a31d22640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f6a2c06c600 con 0x7f6a2c160740 2024-09-19T08:39:26.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.899+0000 7f6a31d22640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f6a2c163bf0 con 0x7f6a2c1636b0 2024-09-19T08:39:26.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.899+0000 7f6a31d22640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6a2c163ed0 con 0x7f6a2c06c1d0 2024-09-19T08:39:26.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.899+0000 7f6a2b7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6a2c160740 0x7f6a2c163170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:26.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.899+0000 7f6a2b7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6a2c160740 0x7f6a2c163170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35312/0 (socket says 172.21.15.98:35312) 2024-09-19T08:39:26.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.899+0000 7f6a2b7fe640 1 -- 172.21.15.98:0/3356320950 learned_addr learned my addr 172.21.15.98:0/3356320950 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:26.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.899+0000 7f6a2affd640 1 --2- 172.21.15.98:0/3356320950 >> v2:172.21.15.139:3300/0 conn(0x7f6a2c1636b0 0x7f6a2c06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:26.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.899+0000 7f6a2bfff640 1 --2- 172.21.15.98:0/3356320950 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a2c06c1d0 0x7f6a2c06d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:26.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.899+0000 7f6a2b7fe640 1 -- 172.21.15.98:0/3356320950 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a2c06c1d0 msgr2=0x7f6a2c06d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:26.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.899+0000 7f6a2b7fe640 1 --2- 172.21.15.98:0/3356320950 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a2c06c1d0 0x7f6a2c06d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:26.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.899+0000 7f6a2b7fe640 1 -- 172.21.15.98:0/3356320950 >> v2:172.21.15.139:3300/0 conn(0x7f6a2c1636b0 msgr2=0x7f6a2c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:26.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.899+0000 7f6a2b7fe640 1 --2- 172.21.15.98:0/3356320950 >> v2:172.21.15.139:3300/0 conn(0x7f6a2c1636b0 0x7f6a2c06bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:26.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.899+0000 7f6a2b7fe640 1 -- 172.21.15.98:0/3356320950 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2c0ce320 con 0x7f6a2c160740 2024-09-19T08:39:26.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.899+0000 7f6a2affd640 1 --2- 172.21.15.98:0/3356320950 >> v2:172.21.15.139:3300/0 conn(0x7f6a2c1636b0 0x7f6a2c06bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:39:26.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.899+0000 7f6a2b7fe640 1 --2- 172.21.15.98:0/3356320950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a2c160740 0x7f6a2c163170 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c163e00 tx=0x7f6a2401c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be99be1f44bde44e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:26.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.899+0000 7f6a2a7fc640 1 -- 172.21.15.98:0/3356320950 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a240ac020 con 0x7f6a2c160740 2024-09-19T08:39:26.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.899+0000 7f6a2a7fc640 1 -- 172.21.15.98:0/3356320950 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a240a6470 con 0x7f6a2c160740 2024-09-19T08:39:26.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.899+0000 7f6a2a7fc640 1 -- 172.21.15.98:0/3356320950 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a240a6840 con 0x7f6a2c160740 2024-09-19T08:39:26.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.903+0000 7f6a31d22640 1 -- 172.21.15.98:0/3356320950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a2c160740 msgr2=0x7f6a2c163170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:26.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.903+0000 7f6a31d22640 1 --2- 172.21.15.98:0/3356320950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a2c160740 0x7f6a2c163170 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c163e00 tx=0x7f6a2401c730 comp rx=0 tx=0).stop 2024-09-19T08:39:26.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.903+0000 7f6a31d22640 1 -- 172.21.15.98:0/3356320950 shutdown_connections 2024-09-19T08:39:26.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.903+0000 7f6a31d22640 1 --2- 172.21.15.98:0/3356320950 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a2c06c1d0 0x7f6a2c06d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:26.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.903+0000 7f6a31d22640 1 --2- 172.21.15.98:0/3356320950 >> v2:172.21.15.139:3300/0 conn(0x7f6a2c1636b0 0x7f6a2c06bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:26.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.903+0000 7f6a31d22640 1 --2- 172.21.15.98:0/3356320950 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a2c160740 0x7f6a2c163170 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:26.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.903+0000 7f6a31d22640 1 -- 172.21.15.98:0/3356320950 >> 172.21.15.98:0/3356320950 conn(0x7f6a2c0b2010 msgr2=0x7f6a2c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:26.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.903+0000 7f6a31d22640 1 -- 172.21.15.98:0/3356320950 shutdown_connections 2024-09-19T08:39:26.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.903+0000 7f6a31d22640 1 -- 172.21.15.98:0/3356320950 wait complete. 2024-09-19T08:39:26.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.903+0000 7f6a31d22640 1 Processor -- start 2024-09-19T08:39:26.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.903+0000 7f6a31d22640 1 -- start start 2024-09-19T08:39:26.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.903+0000 7f6a31d22640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6a2c06c1d0 0x7f6a2c073aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:26.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.903+0000 7f6a31d22640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a2c160740 0x7f6a2c156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:26.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.903+0000 7f6a31d22640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a2c1636b0 0x7f6a2c152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:26.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.903+0000 7f6a31d22640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c0ceca0 con 0x7f6a2c160740 2024-09-19T08:39:26.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.903+0000 7f6a31d22640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c0ceb20 con 0x7f6a2c06c1d0 2024-09-19T08:39:26.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.903+0000 7f6a31d22640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6a2c0cee20 con 0x7f6a2c1636b0 2024-09-19T08:39:26.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.903+0000 7f6a2affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a2c160740 0x7f6a2c156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:26.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.903+0000 7f6a2affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a2c160740 0x7f6a2c156650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35322/0 (socket says 172.21.15.98:35322) 2024-09-19T08:39:26.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.903+0000 7f6a2affd640 1 -- 172.21.15.98:0/2751051266 learned_addr learned my addr 172.21.15.98:0/2751051266 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:26.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.903+0000 7f6a2bfff640 1 --2- 172.21.15.98:0/2751051266 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a2c1636b0 0x7f6a2c152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:26.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.903+0000 7f6a2affd640 1 -- 172.21.15.98:0/2751051266 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a2c1636b0 msgr2=0x7f6a2c152880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:26.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.903+0000 7f6a2affd640 1 --2- 172.21.15.98:0/2751051266 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a2c1636b0 0x7f6a2c152880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:26.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.903+0000 7f6a2b7fe640 1 --2- 172.21.15.98:0/2751051266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6a2c06c1d0 0x7f6a2c073aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:26.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.903+0000 7f6a2affd640 1 -- 172.21.15.98:0/2751051266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6a2c06c1d0 msgr2=0x7f6a2c073aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:26.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.903+0000 7f6a2affd640 1 --2- 172.21.15.98:0/2751051266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6a2c06c1d0 0x7f6a2c073aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:26.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.903+0000 7f6a2affd640 1 -- 172.21.15.98:0/2751051266 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2c152dc0 con 0x7f6a2c160740 2024-09-19T08:39:26.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.903+0000 7f6a2affd640 1 --2- 172.21.15.98:0/2751051266 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a2c160740 0x7f6a2c156650 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6a2000d830 tx=0x7f6a2000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:26.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.903+0000 7f6a13fff640 1 -- 172.21.15.98:0/2751051266 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a20018020 con 0x7f6a2c160740 2024-09-19T08:39:26.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.903+0000 7f6a13fff640 1 -- 172.21.15.98:0/2751051266 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a200096c0 con 0x7f6a2c160740 2024-09-19T08:39:26.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.903+0000 7f6a31d22640 1 -- 172.21.15.98:0/2751051266 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a2c156dc0 con 0x7f6a2c160740 2024-09-19T08:39:26.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.903+0000 7f6a13fff640 1 -- 172.21.15.98:0/2751051266 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a20009ab0 con 0x7f6a2c160740 2024-09-19T08:39:26.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.903+0000 7f6a31d22640 1 -- 172.21.15.98:0/2751051266 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a2c157380 con 0x7f6a2c160740 2024-09-19T08:39:26.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.907+0000 7f6a31d22640 1 -- 172.21.15.98:0/2751051266 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a2c163170 con 0x7f6a2c160740 2024-09-19T08:39:26.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.907+0000 7f6a13fff640 1 -- 172.21.15.98:0/2751051266 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f6a2001b020 con 0x7f6a2c160740 2024-09-19T08:39:26.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.907+0000 7f6a13fff640 1 --2- 172.21.15.98:0/2751051266 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6a18067aa0 0x7f6a18069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:26.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.907+0000 7f6a2b7fe640 1 --2- 172.21.15.98:0/2751051266 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6a18067aa0 0x7f6a18069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:26.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.907+0000 7f6a13fff640 1 -- 172.21.15.98:0/2751051266 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f6a200864e0 con 0x7f6a2c160740 2024-09-19T08:39:26.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.907+0000 7f6a2b7fe640 1 --2- 172.21.15.98:0/2751051266 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6a18067aa0 0x7f6a18069f60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6a240ad000 tx=0x7f6a240a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:26.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:26.911+0000 7f6a13fff640 1 -- 172.21.15.98:0/2751051266 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6a20022020 con 0x7f6a2c160740 2024-09-19T08:39:27.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.107+0000 7f6a31d22640 1 -- 172.21.15.98:0/2751051266 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a2c0742b0 con 0x7f6a2c160740 2024-09-19T08:39:27.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.111+0000 7f6a13fff640 1 -- 172.21.15.98:0/2751051266 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5229 (secure 0 0 0) 0x7f6a20056da0 con 0x7f6a2c160740 2024-09-19T08:39:27.109 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:39:27.110 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":15,"btime":"2024-09-19T08:39:20:116664+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4481,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3797406671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3797406671},{"type":"v1","addr":"172.21.15.98:6835","nonce":3797406671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/4112109751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4112109751},{"type":"v1","addr":"172.21.15.139:6834","nonce":4112109751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4684,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1071477338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6833","nonce":1071477338},{"type":"v1","addr":"172.21.15.139:6835","nonce":1071477338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:39:04.632502+0000","modified":"2024-09-19T08:39:06.618098+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4833},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4833":{"gid":4833,"name":"c","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/1820856497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":1820856497},{"type":"v1","addr":"172.21.15.98:6837","nonce":1820856497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4833,"qdb_cluster":[4833]},"id":2}]} 2024-09-19T08:39:27.110 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 15 2024-09-19T08:39:27.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.111+0000 7f6a31d22640 1 -- 172.21.15.98:0/2751051266 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6a18067aa0 msgr2=0x7f6a18069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:27.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.111+0000 7f6a31d22640 1 --2- 172.21.15.98:0/2751051266 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6a18067aa0 0x7f6a18069f60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6a240ad000 tx=0x7f6a240a7000 comp rx=0 tx=0).stop 2024-09-19T08:39:27.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.111+0000 7f6a31d22640 1 -- 172.21.15.98:0/2751051266 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a2c160740 msgr2=0x7f6a2c156650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:27.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.111+0000 7f6a31d22640 1 --2- 172.21.15.98:0/2751051266 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a2c160740 0x7f6a2c156650 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6a2000d830 tx=0x7f6a2000dcf0 comp rx=0 tx=0).stop 2024-09-19T08:39:27.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.111+0000 7f6a31d22640 1 -- 172.21.15.98:0/2751051266 shutdown_connections 2024-09-19T08:39:27.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.111+0000 7f6a31d22640 1 --2- 172.21.15.98:0/2751051266 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6a18067aa0 0x7f6a18069f60 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:27.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.111+0000 7f6a31d22640 1 --2- 172.21.15.98:0/2751051266 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a2c1636b0 0x7f6a2c152880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:27.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.111+0000 7f6a31d22640 1 --2- 172.21.15.98:0/2751051266 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a2c160740 0x7f6a2c156650 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:27.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.111+0000 7f6a31d22640 1 --2- 172.21.15.98:0/2751051266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6a2c06c1d0 0x7f6a2c073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:27.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.111+0000 7f6a31d22640 1 -- 172.21.15.98:0/2751051266 >> 172.21.15.98:0/2751051266 conn(0x7f6a2c0b2010 msgr2=0x7f6a2c0b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:27.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.111+0000 7f6a31d22640 1 -- 172.21.15.98:0/2751051266 shutdown_connections 2024-09-19T08:39:27.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.111+0000 7f6a31d22640 1 -- 172.21.15.98:0/2751051266 wait complete. 2024-09-19T08:39:27.130 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:39:27.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.247+0000 7fa0200db640 1 Processor -- start 2024-09-19T08:39:27.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.247+0000 7fa0200db640 1 -- start start 2024-09-19T08:39:27.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.251+0000 7fa0200db640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa01815c550 0x7fa01806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:27.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.251+0000 7fa0200db640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa01806c1d0 0x7fa0180cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:27.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.251+0000 7fa0200db640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa01806d310 con 0x7fa0180cf900 2024-09-19T08:39:27.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.251+0000 7fa0200db640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fa01806c610 con 0x7fa01815c550 2024-09-19T08:39:27.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.251+0000 7fa0200db640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa01806d5f0 con 0x7fa01806c1d0 2024-09-19T08:39:27.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.251+0000 7fa01de77640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa01815c550 0x7fa01806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:27.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.251+0000 7fa01e678640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fa0180cf900 0x7fa0180ce1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:46358/0 (socket says 172.21.15.98:46358) 2024-09-19T08:39:27.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.251+0000 7fa01e678640 1 -- 172.21.15.98:0/1432059432 learned_addr learned my addr 172.21.15.98:0/1432059432 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:27.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.251+0000 7fa01d676640 1 --2- 172.21.15.98:0/1432059432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa01806c1d0 0x7fa0180cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:27.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.251+0000 7fa01de77640 1 -- 172.21.15.98:0/1432059432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa01806c1d0 msgr2=0x7fa0180cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:27.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.251+0000 7fa01de77640 1 --2- 172.21.15.98:0/1432059432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa01806c1d0 0x7fa0180cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:27.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.251+0000 7fa01de77640 1 -- 172.21.15.98:0/1432059432 >> v1:172.21.15.98:6789/0 conn(0x7fa0180cf900 legacy=0x7fa0180ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:27.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.251+0000 7fa01ce75640 1 -- 172.21.15.98:0/1432059432 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2258525173 0 0) 0x7fa01806d310 con 0x7fa0180cf900 2024-09-19T08:39:27.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.251+0000 7fa01de77640 1 -- 172.21.15.98:0/1432059432 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa01806d8d0 con 0x7fa01815c550 2024-09-19T08:39:27.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.251+0000 7fa01d676640 1 --2- 172.21.15.98:0/1432059432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa01806c1d0 0x7fa0180cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:27.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.251+0000 7fa01de77640 1 --2- 172.21.15.98:0/1432059432 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa01815c550 0x7fa01806bc90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa0140097c0 tx=0x7fa01401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=825b2b96d5aaee42 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:27.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.251+0000 7fa01ce75640 1 -- 172.21.15.98:0/1432059432 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0140b0070 con 0x7fa01815c550 2024-09-19T08:39:27.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.251+0000 7fa01ce75640 1 -- 172.21.15.98:0/1432059432 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0140ab600 con 0x7fa01815c550 2024-09-19T08:39:27.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.251+0000 7fa01ce75640 1 -- 172.21.15.98:0/1432059432 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0140ab9d0 con 0x7fa01815c550 2024-09-19T08:39:27.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.251+0000 7fa0200db640 1 -- 172.21.15.98:0/1432059432 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa01815c550 msgr2=0x7fa01806bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:27.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.251+0000 7fa0200db640 1 --2- 172.21.15.98:0/1432059432 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa01815c550 0x7fa01806bc90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa0140097c0 tx=0x7fa01401c4a0 comp rx=0 tx=0).stop 2024-09-19T08:39:27.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.251+0000 7fa0200db640 1 -- 172.21.15.98:0/1432059432 shutdown_connections 2024-09-19T08:39:27.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.251+0000 7fa0200db640 1 --2- 172.21.15.98:0/1432059432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa01806c1d0 0x7fa0180cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:27.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.251+0000 7fa0200db640 1 --2- 172.21.15.98:0/1432059432 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa01815c550 0x7fa01806bc90 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:27.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.251+0000 7fa0200db640 1 -- 172.21.15.98:0/1432059432 >> 172.21.15.98:0/1432059432 conn(0x7fa0180b2010 msgr2=0x7fa0180b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:27.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.251+0000 7fa0200db640 1 -- 172.21.15.98:0/1432059432 shutdown_connections 2024-09-19T08:39:27.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.251+0000 7fa0200db640 1 -- 172.21.15.98:0/1432059432 wait complete. 2024-09-19T08:39:27.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.255+0000 7fa0200db640 1 Processor -- start 2024-09-19T08:39:27.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.255+0000 7fa0200db640 1 -- start start 2024-09-19T08:39:27.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.255+0000 7fa0200db640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa01806c1d0 0x7fa018141070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:27.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.255+0000 7fa0200db640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0180cf900 0x7fa0181572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:27.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.255+0000 7fa0200db640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa01815c550 0x7fa01813f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:27.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.255+0000 7fa0200db640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa018169230 con 0x7fa0180cf900 2024-09-19T08:39:27.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.255+0000 7fa0200db640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa01815cb40 con 0x7fa01815c550 2024-09-19T08:39:27.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.255+0000 7fa0200db640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa0181693b0 con 0x7fa01806c1d0 2024-09-19T08:39:27.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.255+0000 7fa01d676640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0180cf900 0x7fa0181572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:27.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.255+0000 7fa01d676640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0180cf900 0x7fa0181572c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35338/0 (socket says 172.21.15.98:35338) 2024-09-19T08:39:27.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.255+0000 7fa01d676640 1 -- 172.21.15.98:0/16290034 learned_addr learned my addr 172.21.15.98:0/16290034 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:27.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.255+0000 7fa01de77640 1 --2- 172.21.15.98:0/16290034 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa01806c1d0 0x7fa018141070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:27.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.255+0000 7fa01d676640 1 -- 172.21.15.98:0/16290034 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa01806c1d0 msgr2=0x7fa018141070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:27.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.255+0000 7fa01d676640 1 --2- 172.21.15.98:0/16290034 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa01806c1d0 0x7fa018141070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:27.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.255+0000 7fa01d676640 1 -- 172.21.15.98:0/16290034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa01815c550 msgr2=0x7fa01813f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:27.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.255+0000 7fa01d676640 1 --2- 172.21.15.98:0/16290034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa01815c550 0x7fa01813f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:27.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.255+0000 7fa01d676640 1 -- 172.21.15.98:0/16290034 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0180b0770 con 0x7fa0180cf900 2024-09-19T08:39:27.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.255+0000 7fa01d676640 1 --2- 172.21.15.98:0/16290034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0180cf900 0x7fa0181572c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa00800c7f0 tx=0x7fa00800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:27.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.255+0000 7fa00e7fc640 1 -- 172.21.15.98:0/16290034 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa008015070 con 0x7fa0180cf900 2024-09-19T08:39:27.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.255+0000 7fa00e7fc640 1 -- 172.21.15.98:0/16290034 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa008007820 con 0x7fa0180cf900 2024-09-19T08:39:27.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.255+0000 7fa0200db640 1 -- 172.21.15.98:0/16290034 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa01813f760 con 0x7fa0180cf900 2024-09-19T08:39:27.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.255+0000 7fa00e7fc640 1 -- 172.21.15.98:0/16290034 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa008007c10 con 0x7fa0180cf900 2024-09-19T08:39:27.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.255+0000 7fa0200db640 1 -- 172.21.15.98:0/16290034 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa018141710 con 0x7fa0180cf900 2024-09-19T08:39:27.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.255+0000 7fa0200db640 1 -- 172.21.15.98:0/16290034 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0180cf3c0 con 0x7fa0180cf900 2024-09-19T08:39:27.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.259+0000 7fa00e7fc640 1 -- 172.21.15.98:0/16290034 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fa008018070 con 0x7fa0180cf900 2024-09-19T08:39:27.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.259+0000 7fa00e7fc640 1 --2- 172.21.15.98:0/16290034 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9ffc067a70 0x7f9ffc069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:27.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.259+0000 7fa00e7fc640 1 -- 172.21.15.98:0/16290034 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fa008086010 con 0x7fa0180cf900 2024-09-19T08:39:27.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.259+0000 7fa01de77640 1 --2- 172.21.15.98:0/16290034 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9ffc067a70 0x7f9ffc069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:27.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.259+0000 7fa01de77640 1 --2- 172.21.15.98:0/16290034 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9ffc067a70 0x7f9ffc069f30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa0140ad000 tx=0x7fa0140a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:27.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.263+0000 7fa00e7fc640 1 -- 172.21.15.98:0/16290034 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa0080567c0 con 0x7fa0180cf900 2024-09-19T08:39:27.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.415+0000 7fa0200db640 1 -- 172.21.15.98:0/16290034 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa01815b900 con 0x7fa0180cf900 2024-09-19T08:39:27.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.415+0000 7fa00e7fc640 1 -- 172.21.15.98:0/16290034 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16729 (secure 0 0 0) 0x7fa0080560c0 con 0x7fa0180cf900 2024-09-19T08:39:27.416 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:39:27.416 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":32,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:39:06.263597+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:39:02.429044+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-19T08:39:03.620637+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:39:27.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.419+0000 7fa0200db640 1 -- 172.21.15.98:0/16290034 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9ffc067a70 msgr2=0x7f9ffc069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:27.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.419+0000 7fa0200db640 1 --2- 172.21.15.98:0/16290034 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9ffc067a70 0x7f9ffc069f30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa0140ad000 tx=0x7fa0140a6000 comp rx=0 tx=0).stop 2024-09-19T08:39:27.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.419+0000 7fa0200db640 1 -- 172.21.15.98:0/16290034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0180cf900 msgr2=0x7fa0181572c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:27.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.419+0000 7fa0200db640 1 --2- 172.21.15.98:0/16290034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0180cf900 0x7fa0181572c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa00800c7f0 tx=0x7fa00800ccb0 comp rx=0 tx=0).stop 2024-09-19T08:39:27.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.419+0000 7fa0200db640 1 -- 172.21.15.98:0/16290034 shutdown_connections 2024-09-19T08:39:27.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.419+0000 7fa0200db640 1 --2- 172.21.15.98:0/16290034 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9ffc067a70 0x7f9ffc069f30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:27.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.419+0000 7fa0200db640 1 --2- 172.21.15.98:0/16290034 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa01815c550 0x7fa01813f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:27.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.419+0000 7fa0200db640 1 --2- 172.21.15.98:0/16290034 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0180cf900 0x7fa0181572c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:27.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.419+0000 7fa0200db640 1 --2- 172.21.15.98:0/16290034 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa01806c1d0 0x7fa018141070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:27.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.419+0000 7fa0200db640 1 -- 172.21.15.98:0/16290034 >> 172.21.15.98:0/16290034 conn(0x7fa0180b2010 msgr2=0x7fa0180b15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:27.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.419+0000 7fa0200db640 1 -- 172.21.15.98:0/16290034 shutdown_connections 2024-09-19T08:39:27.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.419+0000 7fa0200db640 1 -- 172.21.15.98:0/16290034 wait complete. 2024-09-19T08:39:27.434 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-19T08:39:27.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.547+0000 7f0f2fd5b640 1 Processor -- start 2024-09-19T08:39:27.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.547+0000 7f0f2fd5b640 1 -- start start 2024-09-19T08:39:27.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.547+0000 7f0f2fd5b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0f28158fd0 0x7f0f280b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:27.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.547+0000 7f0f2fd5b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0f280b4360 0x7f0f2806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:27.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.547+0000 7f0f2fd5b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f280ccc60 con 0x7f0f2806c1d0 2024-09-19T08:39:27.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.547+0000 7f0f2fd5b640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f0f2806c640 con 0x7f0f28158fd0 2024-09-19T08:39:27.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.547+0000 7f0f2fd5b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f0f281593b0 con 0x7f0f280b4360 2024-09-19T08:39:27.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.547+0000 7f0f2daf7640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0f28158fd0 0x7f0f280b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:27.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.547+0000 7f0f2d2f6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0f280b4360 0x7f0f2806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:27.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.547+0000 7f0f2daf7640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0f28158fd0 0x7f0f280b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48340/0 (socket says 172.21.15.98:48340) 2024-09-19T08:39:27.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.547+0000 7f0f2daf7640 1 -- 172.21.15.98:0/2313692501 learned_addr learned my addr 172.21.15.98:0/2313692501 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:27.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.547+0000 7f0f2daf7640 1 -- 172.21.15.98:0/2313692501 >> v2:172.21.15.139:3300/0 conn(0x7f0f280b4360 msgr2=0x7f0f2806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:27.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.547+0000 7f0f2daf7640 1 --2- 172.21.15.98:0/2313692501 >> v2:172.21.15.139:3300/0 conn(0x7f0f280b4360 0x7f0f2806bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:27.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.547+0000 7f0f2daf7640 1 -- 172.21.15.98:0/2313692501 >> v1:172.21.15.139:6789/0 conn(0x7f0f2806c1d0 legacy=0x7f0f2815efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:27.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.547+0000 7f0f2daf7640 1 -- 172.21.15.98:0/2313692501 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f280ccf40 con 0x7f0f28158fd0 2024-09-19T08:39:27.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.551+0000 7f0f2daf7640 1 --2- 172.21.15.98:0/2313692501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f28158fd0 0x7f0f280b3e20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0f280cdef0 tx=0x7f0f2001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58c60128fc0a862 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:27.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.551+0000 7f0f2caf5640 1 -- 172.21.15.98:0/2313692501 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f200ab660 con 0x7f0f28158fd0 2024-09-19T08:39:27.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.551+0000 7f0f2caf5640 1 -- 172.21.15.98:0/2313692501 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f200b0070 con 0x7f0f28158fd0 2024-09-19T08:39:27.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.551+0000 7f0f2caf5640 1 -- 172.21.15.98:0/2313692501 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f200aba20 con 0x7f0f28158fd0 2024-09-19T08:39:27.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.551+0000 7f0f2fd5b640 1 -- 172.21.15.98:0/2313692501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f28158fd0 msgr2=0x7f0f280b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:27.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.551+0000 7f0f2fd5b640 1 --2- 172.21.15.98:0/2313692501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f28158fd0 0x7f0f280b3e20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0f280cdef0 tx=0x7f0f2001c460 comp rx=0 tx=0).stop 2024-09-19T08:39:27.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.551+0000 7f0f2fd5b640 1 -- 172.21.15.98:0/2313692501 shutdown_connections 2024-09-19T08:39:27.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.551+0000 7f0f2fd5b640 1 --2- 172.21.15.98:0/2313692501 >> v2:172.21.15.139:3300/0 conn(0x7f0f280b4360 0x7f0f2806bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:27.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.551+0000 7f0f2fd5b640 1 --2- 172.21.15.98:0/2313692501 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f28158fd0 0x7f0f280b3e20 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:27.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.551+0000 7f0f2fd5b640 1 -- 172.21.15.98:0/2313692501 >> 172.21.15.98:0/2313692501 conn(0x7f0f280b2010 msgr2=0x7f0f280b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:27.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.551+0000 7f0f2fd5b640 1 -- 172.21.15.98:0/2313692501 shutdown_connections 2024-09-19T08:39:27.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.551+0000 7f0f2fd5b640 1 -- 172.21.15.98:0/2313692501 wait complete. 2024-09-19T08:39:27.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.551+0000 7f0f2fd5b640 1 Processor -- start 2024-09-19T08:39:27.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.551+0000 7f0f2fd5b640 1 -- start start 2024-09-19T08:39:27.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.551+0000 7f0f2fd5b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f2806c1d0 0x7f0f28152d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:27.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.551+0000 7f0f2fd5b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f280b4360 0x7f0f28154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:27.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.551+0000 7f0f2fd5b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0f28158fd0 0x7f0f281550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:27.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.551+0000 7f0f2fd5b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2815c920 con 0x7f0f280b4360 2024-09-19T08:39:27.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.551+0000 7f0f2fd5b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2815c7a0 con 0x7f0f2806c1d0 2024-09-19T08:39:27.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.551+0000 7f0f2fd5b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0f28168110 con 0x7f0f28158fd0 2024-09-19T08:39:27.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.551+0000 7f0f2d2f6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f280b4360 0x7f0f28154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:27.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.551+0000 7f0f2d2f6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f280b4360 0x7f0f28154ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48348/0 (socket says 172.21.15.98:48348) 2024-09-19T08:39:27.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.551+0000 7f0f2d2f6640 1 -- 172.21.15.98:0/2458235923 learned_addr learned my addr 172.21.15.98:0/2458235923 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:27.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.551+0000 7f0f2daf7640 1 --2- 172.21.15.98:0/2458235923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f2806c1d0 0x7f0f28152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:27.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.551+0000 7f0f2e2f8640 1 --2- 172.21.15.98:0/2458235923 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0f28158fd0 0x7f0f281550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:27.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.551+0000 7f0f2d2f6640 1 -- 172.21.15.98:0/2458235923 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0f28158fd0 msgr2=0x7f0f281550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:27.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.551+0000 7f0f2d2f6640 1 --2- 172.21.15.98:0/2458235923 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0f28158fd0 0x7f0f281550e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:27.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.551+0000 7f0f2d2f6640 1 -- 172.21.15.98:0/2458235923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f2806c1d0 msgr2=0x7f0f28152d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:27.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.551+0000 7f0f2d2f6640 1 --2- 172.21.15.98:0/2458235923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f2806c1d0 0x7f0f28152d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:27.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.551+0000 7f0f2d2f6640 1 -- 172.21.15.98:0/2458235923 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f28072800 con 0x7f0f280b4360 2024-09-19T08:39:27.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.551+0000 7f0f2daf7640 1 --2- 172.21.15.98:0/2458235923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f2806c1d0 0x7f0f28152d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:27.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.551+0000 7f0f2e2f8640 1 --2- 172.21.15.98:0/2458235923 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0f28158fd0 0x7f0f281550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:27.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.551+0000 7f0f2d2f6640 1 --2- 172.21.15.98:0/2458235923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f280b4360 0x7f0f28154ba0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0f18010b40 tx=0x7f0f1800e360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:27.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.551+0000 7f0f1e7fc640 1 -- 172.21.15.98:0/2458235923 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1801e070 con 0x7f0f280b4360 2024-09-19T08:39:27.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.555+0000 7f0f1e7fc640 1 -- 172.21.15.98:0/2458235923 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f18011450 con 0x7f0f280b4360 2024-09-19T08:39:27.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.555+0000 7f0f2fd5b640 1 -- 172.21.15.98:0/2458235923 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f28072ae0 con 0x7f0f280b4360 2024-09-19T08:39:27.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.555+0000 7f0f1e7fc640 1 -- 172.21.15.98:0/2458235923 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f18011800 con 0x7f0f280b4360 2024-09-19T08:39:27.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.555+0000 7f0f2fd5b640 1 -- 172.21.15.98:0/2458235923 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f280730a0 con 0x7f0f280b4360 2024-09-19T08:39:27.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.555+0000 7f0f2fd5b640 1 -- 172.21.15.98:0/2458235923 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f2806bc90 con 0x7f0f280b4360 2024-09-19T08:39:27.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.555+0000 7f0f1e7fc640 1 -- 172.21.15.98:0/2458235923 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f0f18021070 con 0x7f0f280b4360 2024-09-19T08:39:27.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.555+0000 7f0f1e7fc640 1 --2- 172.21.15.98:0/2458235923 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0f0c067a70 0x7f0f0c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:27.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.555+0000 7f0f1e7fc640 1 -- 172.21.15.98:0/2458235923 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f0f18088200 con 0x7f0f280b4360 2024-09-19T08:39:27.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.555+0000 7f0f2daf7640 1 --2- 172.21.15.98:0/2458235923 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0f0c067a70 0x7f0f0c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:27.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.555+0000 7f0f2daf7640 1 --2- 172.21.15.98:0/2458235923 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0f0c067a70 0x7f0f0c069f30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0f200af000 tx=0x7f0f20008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:27.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.559+0000 7f0f1e7fc640 1 -- 172.21.15.98:0/2458235923 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f0f18058470 con 0x7f0f280b4360 2024-09-19T08:39:27.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.739+0000 7f0f2fd5b640 1 -- 172.21.15.98:0/2458235923 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0f2815f4b0 con 0x7f0f280b4360 2024-09-19T08:39:27.741 INFO:tasks.ceph.mon.a.smithi098.stderr:2024-09-19T08:39:27.739+0000 7f5e49017640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T08:39:27.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.743+0000 7f0f1e7fc640 1 -- 172.21.15.98:0/2458235923 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v16) ==== 161+0+0 (secure 0 0 0) 0x7f0f1805c770 con 0x7f0f280b4360 2024-09-19T08:39:27.742 INFO:teuthology.orchestra.run.smithi098.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T08:39:27.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.743+0000 7f0f2fd5b640 1 -- 172.21.15.98:0/2458235923 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0f0c067a70 msgr2=0x7f0f0c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:27.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.743+0000 7f0f2fd5b640 1 --2- 172.21.15.98:0/2458235923 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0f0c067a70 0x7f0f0c069f30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0f200af000 tx=0x7f0f20008040 comp rx=0 tx=0).stop 2024-09-19T08:39:27.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.743+0000 7f0f2fd5b640 1 -- 172.21.15.98:0/2458235923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f280b4360 msgr2=0x7f0f28154ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:27.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.743+0000 7f0f2fd5b640 1 --2- 172.21.15.98:0/2458235923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f280b4360 0x7f0f28154ba0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0f18010b40 tx=0x7f0f1800e360 comp rx=0 tx=0).stop 2024-09-19T08:39:27.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.743+0000 7f0f2fd5b640 1 -- 172.21.15.98:0/2458235923 shutdown_connections 2024-09-19T08:39:27.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.743+0000 7f0f2fd5b640 1 --2- 172.21.15.98:0/2458235923 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0f0c067a70 0x7f0f0c069f30 secure :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0f200af000 tx=0x7f0f20008040 comp rx=0 tx=0).stop 2024-09-19T08:39:27.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.743+0000 7f0f2fd5b640 1 --2- 172.21.15.98:0/2458235923 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0f28158fd0 0x7f0f281550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:27.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.743+0000 7f0f2fd5b640 1 --2- 172.21.15.98:0/2458235923 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f280b4360 0x7f0f28154ba0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:27.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.743+0000 7f0f2fd5b640 1 --2- 172.21.15.98:0/2458235923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f2806c1d0 0x7f0f28152d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:27.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.747+0000 7f0f2fd5b640 1 -- 172.21.15.98:0/2458235923 >> 172.21.15.98:0/2458235923 conn(0x7f0f280b2010 msgr2=0x7f0f280af700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:27.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.747+0000 7f0f2fd5b640 1 -- 172.21.15.98:0/2458235923 shutdown_connections 2024-09-19T08:39:27.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.747+0000 7f0f2fd5b640 1 -- 172.21.15.98:0/2458235923 wait complete. 2024-09-19T08:39:27.762 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T08:39:27.813 INFO:tasks.ceph.mds.c.smithi098.stdout:starting mds.c at 2024-09-19T08:39:27.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.879+0000 7ff3d6372640 1 Processor -- start 2024-09-19T08:39:27.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.879+0000 7ff3d6372640 1 -- start start 2024-09-19T08:39:27.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.879+0000 7ff3d6372640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff3d0144d10 0x7ff3d00722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:27.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.879+0000 7ff3d6372640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3d0072820 0x7ff3d006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:27.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.879+0000 7ff3d6372640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3d0072ea0 con 0x7ff3d006c1d0 2024-09-19T08:39:27.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.879+0000 7ff3d6372640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff3d0072c60 con 0x7ff3d0144d10 2024-09-19T08:39:27.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.879+0000 7ff3d6372640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff3d0073ab0 con 0x7ff3d0072820 2024-09-19T08:39:27.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.879+0000 7ff3cffff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff3d0144d10 0x7ff3d00722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:27.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.879+0000 7ff3cf7fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3d0072820 0x7ff3d006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:27.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.879+0000 7ff3cf7fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3d0072820 0x7ff3d006bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:46158/0 (socket says 172.21.15.98:46158) 2024-09-19T08:39:27.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.879+0000 7ff3cf7fe640 1 -- 172.21.15.98:0/3530631761 learned_addr learned my addr 172.21.15.98:0/3530631761 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:27.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.879+0000 7ff3cf7fe640 1 -- 172.21.15.98:0/3530631761 >> v2:172.21.15.139:3300/0 conn(0x7ff3d0144d10 msgr2=0x7ff3d00722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:27.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.879+0000 7ff3cf7fe640 1 --2- 172.21.15.98:0/3530631761 >> v2:172.21.15.139:3300/0 conn(0x7ff3d0144d10 0x7ff3d00722e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:27.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.879+0000 7ff3cf7fe640 1 -- 172.21.15.98:0/3530631761 >> v1:172.21.15.139:6789/0 conn(0x7ff3d006c1d0 legacy=0x7ff3d00b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:27.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.879+0000 7ff3cf7fe640 1 -- 172.21.15.98:0/3530631761 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d0073180 con 0x7ff3d0072820 2024-09-19T08:39:27.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.883+0000 7ff3cf7fe640 1 --2- 172.21.15.98:0/3530631761 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3d0072820 0x7ff3d006bc90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0009870 tx=0x7ff3c001c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45a304c88c93497c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:27.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.883+0000 7ff3ceffd640 1 -- 172.21.15.98:0/3530631761 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c00b2070 con 0x7ff3d0072820 2024-09-19T08:39:27.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.883+0000 7ff3ceffd640 1 -- 172.21.15.98:0/3530631761 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3c00a9070 con 0x7ff3d0072820 2024-09-19T08:39:27.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.883+0000 7ff3ceffd640 1 -- 172.21.15.98:0/3530631761 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c00ad880 con 0x7ff3d0072820 2024-09-19T08:39:27.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.883+0000 7ff3d6372640 1 -- 172.21.15.98:0/3530631761 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3d0072820 msgr2=0x7ff3d006bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:27.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.883+0000 7ff3d6372640 1 --2- 172.21.15.98:0/3530631761 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3d0072820 0x7ff3d006bc90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0009870 tx=0x7ff3c001c560 comp rx=0 tx=0).stop 2024-09-19T08:39:27.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.883+0000 7ff3d6372640 1 -- 172.21.15.98:0/3530631761 shutdown_connections 2024-09-19T08:39:27.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.883+0000 7ff3d6372640 1 --2- 172.21.15.98:0/3530631761 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3d0072820 0x7ff3d006bc90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:27.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.883+0000 7ff3d6372640 1 --2- 172.21.15.98:0/3530631761 >> v2:172.21.15.139:3300/0 conn(0x7ff3d0144d10 0x7ff3d00722e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:27.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.883+0000 7ff3d6372640 1 -- 172.21.15.98:0/3530631761 >> 172.21.15.98:0/3530631761 conn(0x7ff3d00b2010 msgr2=0x7ff3d00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:27.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.883+0000 7ff3d6372640 1 -- 172.21.15.98:0/3530631761 shutdown_connections 2024-09-19T08:39:27.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.883+0000 7ff3d6372640 1 -- 172.21.15.98:0/3530631761 wait complete. 2024-09-19T08:39:27.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.887+0000 7ff3d6372640 1 Processor -- start 2024-09-19T08:39:27.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.887+0000 7ff3d6372640 1 -- start start 2024-09-19T08:39:27.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.887+0000 7ff3d6372640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3d006c1d0 0x7ff3d013e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:27.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.887+0000 7ff3d6372640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3d0072820 0x7ff3d00c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:27.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.887+0000 7ff3d6372640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3d0144d10 0x7ff3d015aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:27.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.887+0000 7ff3d6372640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d01680e0 con 0x7ff3d0144d10 2024-09-19T08:39:27.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.887+0000 7ff3d6372640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d0167f60 con 0x7ff3d0072820 2024-09-19T08:39:27.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.887+0000 7ff3d6372640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff3d0168260 con 0x7ff3d006c1d0 2024-09-19T08:39:27.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.887+0000 7ff3d490f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3d0144d10 0x7ff3d015aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:27.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.887+0000 7ff3d490f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3d0144d10 0x7ff3d015aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48374/0 (socket says 172.21.15.98:48374) 2024-09-19T08:39:27.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.887+0000 7ff3d490f640 1 -- 172.21.15.98:0/119414234 learned_addr learned my addr 172.21.15.98:0/119414234 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:27.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.887+0000 7ff3cffff640 1 --2- 172.21.15.98:0/119414234 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3d006c1d0 0x7ff3d013e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:27.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.887+0000 7ff3cf7fe640 1 --2- 172.21.15.98:0/119414234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3d0072820 0x7ff3d00c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:27.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.887+0000 7ff3d490f640 1 -- 172.21.15.98:0/119414234 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3d006c1d0 msgr2=0x7ff3d013e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:27.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.887+0000 7ff3d490f640 1 --2- 172.21.15.98:0/119414234 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3d006c1d0 0x7ff3d013e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:27.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.887+0000 7ff3d490f640 1 -- 172.21.15.98:0/119414234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3d0072820 msgr2=0x7ff3d00c58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:27.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.887+0000 7ff3d490f640 1 --2- 172.21.15.98:0/119414234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3d0072820 0x7ff3d00c58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:27.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.887+0000 7ff3d490f640 1 -- 172.21.15.98:0/119414234 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d013d640 con 0x7ff3d0144d10 2024-09-19T08:39:27.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.887+0000 7ff3d490f640 1 --2- 172.21.15.98:0/119414234 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3d0144d10 0x7ff3d015aee0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800c7a0 tx=0x7ff3c800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:27.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.887+0000 7ff3ccff9640 1 -- 172.21.15.98:0/119414234 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c8003b80 con 0x7ff3d0144d10 2024-09-19T08:39:27.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.887+0000 7ff3ccff9640 1 -- 172.21.15.98:0/119414234 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3c8003d20 con 0x7ff3d0144d10 2024-09-19T08:39:27.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.887+0000 7ff3d6372640 1 -- 172.21.15.98:0/119414234 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3d013d920 con 0x7ff3d0144d10 2024-09-19T08:39:27.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.887+0000 7ff3ccff9640 1 -- 172.21.15.98:0/119414234 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c800aa90 con 0x7ff3d0144d10 2024-09-19T08:39:27.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.887+0000 7ff3d6372640 1 -- 172.21.15.98:0/119414234 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3d013dee0 con 0x7ff3d0144d10 2024-09-19T08:39:27.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.887+0000 7ff3d6372640 1 -- 172.21.15.98:0/119414234 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3d006bc90 con 0x7ff3d0144d10 2024-09-19T08:39:27.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.887+0000 7ff3ccff9640 1 -- 172.21.15.98:0/119414234 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7ff3c800ac30 con 0x7ff3d0144d10 2024-09-19T08:39:27.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.887+0000 7ff3ccff9640 1 --2- 172.21.15.98:0/119414234 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff3b0067a70 0x7ff3b0069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:27.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.891+0000 7ff3ccff9640 1 -- 172.21.15.98:0/119414234 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5795+0+0 (secure 0 0 0) 0x7ff3c8085ef0 con 0x7ff3d0144d10 2024-09-19T08:39:27.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.891+0000 7ff3cffff640 1 --2- 172.21.15.98:0/119414234 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff3b0067a70 0x7ff3b0069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:27.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.891+0000 7ff3cffff640 1 --2- 172.21.15.98:0/119414234 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff3b0067a70 0x7ff3b0069f30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8009ac0 tx=0x7ff3b8009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:27.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:27.895+0000 7ff3ccff9640 1 -- 172.21.15.98:0/119414234 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff3c80566d0 con 0x7ff3d0144d10 2024-09-19T08:39:28.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.075+0000 7ff3d6372640 1 -- 172.21.15.98:0/119414234 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff3d01414d0 con 0x7ff3d0144d10 2024-09-19T08:39:28.744 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.743+0000 7ff3ccff9640 1 -- 172.21.15.98:0/119414234 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7ff3c8055fd0 con 0x7ff3d0144d10 2024-09-19T08:39:28.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.747+0000 7ff3d6372640 1 -- 172.21.15.98:0/119414234 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff3b0067a70 msgr2=0x7ff3b0069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:28.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.747+0000 7ff3d6372640 1 --2- 172.21.15.98:0/119414234 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff3b0067a70 0x7ff3b0069f30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8009ac0 tx=0x7ff3b8009190 comp rx=0 tx=0).stop 2024-09-19T08:39:28.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.747+0000 7ff3d6372640 1 -- 172.21.15.98:0/119414234 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3d0144d10 msgr2=0x7ff3d015aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:28.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.747+0000 7ff3d6372640 1 --2- 172.21.15.98:0/119414234 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3d0144d10 0x7ff3d015aee0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800c7a0 tx=0x7ff3c800cc60 comp rx=0 tx=0).stop 2024-09-19T08:39:28.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.747+0000 7ff3d6372640 1 -- 172.21.15.98:0/119414234 shutdown_connections 2024-09-19T08:39:28.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.747+0000 7ff3d6372640 1 --2- 172.21.15.98:0/119414234 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff3b0067a70 0x7ff3b0069f30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:28.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.747+0000 7ff3d6372640 1 --2- 172.21.15.98:0/119414234 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3d0144d10 0x7ff3d015aee0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:28.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.747+0000 7ff3d6372640 1 --2- 172.21.15.98:0/119414234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3d0072820 0x7ff3d00c58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:28.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.747+0000 7ff3d6372640 1 --2- 172.21.15.98:0/119414234 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3d006c1d0 0x7ff3d013e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:28.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.747+0000 7ff3d6372640 1 -- 172.21.15.98:0/119414234 >> 172.21.15.98:0/119414234 conn(0x7ff3d00b2010 msgr2=0x7ff3d01419f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:28.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.747+0000 7ff3d6372640 1 -- 172.21.15.98:0/119414234 shutdown_connections 2024-09-19T08:39:28.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.747+0000 7ff3d6372640 1 -- 172.21.15.98:0/119414234 wait complete. 2024-09-19T08:39:28.770 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T08:39:28.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.891+0000 7f93ac9bf640 1 Processor -- start 2024-09-19T08:39:28.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.891+0000 7f93ac9bf640 1 -- start start 2024-09-19T08:39:28.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.891+0000 7f93ac9bf640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93a80cf700 0x7f93a815ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:28.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.891+0000 7f93ac9bf640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f93a806d4c0 0x7f93a806d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:28.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.891+0000 7f93ac9bf640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93a81608c0 con 0x7f93a815c290 2024-09-19T08:39:28.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.891+0000 7f93ac9bf640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f93a806de70 con 0x7f93a806d4c0 2024-09-19T08:39:28.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.891+0000 7f93ac9bf640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f93a8160e80 con 0x7f93a80cf700 2024-09-19T08:39:28.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.891+0000 7f93a659c640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f93a815c290 0x7f93a80cd9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36190/0 (socket says 172.21.15.98:36190) 2024-09-19T08:39:28.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.891+0000 7f93a659c640 1 -- 172.21.15.98:0/3673196330 learned_addr learned my addr 172.21.15.98:0/3673196330 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:28.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.891+0000 7f93a5d9b640 1 --2- 172.21.15.98:0/3673196330 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93a80cf700 0x7f93a815ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:28.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.891+0000 7f93a559a640 1 -- 172.21.15.98:0/3673196330 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1428213772 0 0) 0x7f93a81608c0 con 0x7f93a815c290 2024-09-19T08:39:28.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.891+0000 7f93a6d9d640 1 --2- 172.21.15.98:0/3673196330 >> v2:172.21.15.139:3300/0 conn(0x7f93a806d4c0 0x7f93a806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:28.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.891+0000 7f93a559a640 1 -- 172.21.15.98:0/3673196330 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9390003530 con 0x7f93a815c290 2024-09-19T08:39:28.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.895+0000 7f93a559a640 1 -- 172.21.15.98:0/3673196330 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 455237472 0 0) 0x7f9390003530 con 0x7f93a815c290 2024-09-19T08:39:28.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.895+0000 7f93a559a640 1 -- 172.21.15.98:0/3673196330 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93a80cf700 msgr2=0x7f93a815ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:28.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.895+0000 7f93a559a640 1 --2- 172.21.15.98:0/3673196330 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93a80cf700 0x7f93a815ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:28.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.895+0000 7f93a559a640 1 -- 172.21.15.98:0/3673196330 >> v2:172.21.15.139:3300/0 conn(0x7f93a806d4c0 msgr2=0x7f93a806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:28.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.895+0000 7f93a559a640 1 --2- 172.21.15.98:0/3673196330 >> v2:172.21.15.139:3300/0 conn(0x7f93a806d4c0 0x7f93a806d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:28.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.895+0000 7f93a559a640 1 -- 172.21.15.98:0/3673196330 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a8160ba0 con 0x7f93a815c290 2024-09-19T08:39:28.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.895+0000 7f93a559a640 1 -- 172.21.15.98:0/3673196330 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f9398001a00 con 0x7f93a815c290 2024-09-19T08:39:28.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.895+0000 7f93a559a640 1 -- 172.21.15.98:0/3673196330 >> v1:172.21.15.98:6789/0 conn(0x7f93a815c290 legacy=0x7f93a80cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:28.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.895+0000 7f93a559a640 1 --2- 172.21.15.98:0/3673196330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93900039c0 0x7f9390003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:28.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.895+0000 7f93a559a640 1 --2- 172.21.15.98:0/3673196330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9390004810 0x7f9390004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:28.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.895+0000 7f93a559a640 1 --2- 172.21.15.98:0/3673196330 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93a80cf700 0x7f93900051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:28.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.895+0000 7f93a559a640 1 -- 172.21.15.98:0/3673196330 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f93a806de70 con 0x7f93900039c0 2024-09-19T08:39:28.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.895+0000 7f93a559a640 1 -- 172.21.15.98:0/3673196330 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f939000b9d0 con 0x7f9390004810 2024-09-19T08:39:28.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.895+0000 7f93a559a640 1 -- 172.21.15.98:0/3673196330 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f939000bb50 con 0x7f93a80cf700 2024-09-19T08:39:28.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.895+0000 7f93a659c640 1 --2- 172.21.15.98:0/3673196330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93900039c0 0x7f9390003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:28.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.895+0000 7f93a6d9d640 1 --2- 172.21.15.98:0/3673196330 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93a80cf700 0x7f93900051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:28.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.895+0000 7f93a5d9b640 1 --2- 172.21.15.98:0/3673196330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9390004810 0x7f9390004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:28.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.895+0000 7f93a659c640 1 -- 172.21.15.98:0/3673196330 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93a80cf700 msgr2=0x7f93900051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:28.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.895+0000 7f93a659c640 1 --2- 172.21.15.98:0/3673196330 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93a80cf700 0x7f93900051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:28.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.895+0000 7f93a659c640 1 -- 172.21.15.98:0/3673196330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9390004810 msgr2=0x7f9390004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:28.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.895+0000 7f93a659c640 1 --2- 172.21.15.98:0/3673196330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9390004810 0x7f9390004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:28.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.895+0000 7f93a659c640 1 -- 172.21.15.98:0/3673196330 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9390003530 con 0x7f93900039c0 2024-09-19T08:39:28.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.895+0000 7f93a659c640 1 --2- 172.21.15.98:0/3673196330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93900039c0 0x7f9390003de0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f939800a880 tx=0x7f939801d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a43b5838574eddd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:28.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.895+0000 7f93a559a640 1 -- 172.21.15.98:0/3673196330 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939800a4a0 con 0x7f93900039c0 2024-09-19T08:39:28.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.895+0000 7f93a559a640 1 -- 172.21.15.98:0/3673196330 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93980a9020 con 0x7f93900039c0 2024-09-19T08:39:28.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.895+0000 7f93a559a640 1 -- 172.21.15.98:0/3673196330 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93980070d0 con 0x7f93900039c0 2024-09-19T08:39:28.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.895+0000 7f93ac9bf640 1 -- 172.21.15.98:0/3673196330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93900039c0 msgr2=0x7f9390003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:28.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.895+0000 7f93ac9bf640 1 --2- 172.21.15.98:0/3673196330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93900039c0 0x7f9390003de0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f939800a880 tx=0x7f939801d2e0 comp rx=0 tx=0).stop 2024-09-19T08:39:28.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.895+0000 7f93a6d9d640 1 -- 172.21.15.98:0/3673196330 reap_dead start 2024-09-19T08:39:28.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.895+0000 7f93ac9bf640 1 -- 172.21.15.98:0/3673196330 shutdown_connections 2024-09-19T08:39:28.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.895+0000 7f93ac9bf640 1 -- 172.21.15.98:0/3673196330 >> 172.21.15.98:0/3673196330 conn(0x7f93a806c090 msgr2=0x7f93a806c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:28.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.895+0000 7f93ac9bf640 1 -- 172.21.15.98:0/3673196330 shutdown_connections 2024-09-19T08:39:28.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.895+0000 7f93ac9bf640 1 -- 172.21.15.98:0/3673196330 wait complete. 2024-09-19T08:39:28.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.899+0000 7f93ac9bf640 1 Processor -- start 2024-09-19T08:39:28.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.899+0000 7f93ac9bf640 1 -- start start 2024-09-19T08:39:28.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.899+0000 7f93ac9bf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93900039c0 0x7f93a814bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:28.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.899+0000 7f93ac9bf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93a815a530 0x7f93a815a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:28.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.899+0000 7f93ac9bf640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93a81f5190 0x7f93a81f5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:28.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.899+0000 7f93ac9bf640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f93a80b4fc0 con 0x7f93900039c0 2024-09-19T08:39:28.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.899+0000 7f93ac9bf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f93a80b4e40 con 0x7f93a815a530 2024-09-19T08:39:28.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.899+0000 7f93ac9bf640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f93a80b5140 con 0x7f93a81f5190 2024-09-19T08:39:28.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.899+0000 7f93a659c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93900039c0 0x7f93a814bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:28.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.899+0000 7f93a659c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93900039c0 0x7f93a814bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48396/0 (socket says 172.21.15.98:48396) 2024-09-19T08:39:28.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.899+0000 7f93a659c640 1 -- 172.21.15.98:0/1845140234 learned_addr learned my addr 172.21.15.98:0/1845140234 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:28.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.899+0000 7f93a5d9b640 1 --2- 172.21.15.98:0/1845140234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93a815a530 0x7f93a815a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:28.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.899+0000 7f93a6d9d640 1 --2- 172.21.15.98:0/1845140234 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93a81f5190 0x7f93a81f5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:28.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.899+0000 7f93a659c640 1 -- 172.21.15.98:0/1845140234 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93a81f5190 msgr2=0x7f93a81f5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:28.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.899+0000 7f93a659c640 1 --2- 172.21.15.98:0/1845140234 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93a81f5190 0x7f93a81f5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:28.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.899+0000 7f93a659c640 1 -- 172.21.15.98:0/1845140234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93a815a530 msgr2=0x7f93a815a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:28.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.899+0000 7f93a659c640 1 --2- 172.21.15.98:0/1845140234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93a815a530 0x7f93a815a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:28.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.899+0000 7f93a659c640 1 -- 172.21.15.98:0/1845140234 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a81f60f0 con 0x7f93900039c0 2024-09-19T08:39:28.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.899+0000 7f93a659c640 1 --2- 172.21.15.98:0/1845140234 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93900039c0 0x7f93a814bdb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f939801d110 tx=0x7f9390005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:28.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.899+0000 7f9396ffd640 1 -- 172.21.15.98:0/1845140234 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9398001940 con 0x7f93900039c0 2024-09-19T08:39:28.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.899+0000 7f9396ffd640 1 -- 172.21.15.98:0/1845140234 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93980070d0 con 0x7f93900039c0 2024-09-19T08:39:28.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.899+0000 7f93ac9bf640 1 -- 172.21.15.98:0/1845140234 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93a814c2f0 con 0x7f93900039c0 2024-09-19T08:39:28.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.899+0000 7f9396ffd640 1 -- 172.21.15.98:0/1845140234 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93980074c0 con 0x7f93900039c0 2024-09-19T08:39:28.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.899+0000 7f93ac9bf640 1 -- 172.21.15.98:0/1845140234 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93a81f5e10 con 0x7f93900039c0 2024-09-19T08:39:28.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.899+0000 7f93ac9bf640 1 -- 172.21.15.98:0/1845140234 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93a80b5530 con 0x7f93900039c0 2024-09-19T08:39:28.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.899+0000 7f9396ffd640 1 -- 172.21.15.98:0/1845140234 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f9398007660 con 0x7f93900039c0 2024-09-19T08:39:28.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.903+0000 7f9396ffd640 1 --2- 172.21.15.98:0/1845140234 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9388067a70 0x7f9388069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:28.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.903+0000 7f9396ffd640 1 -- 172.21.15.98:0/1845140234 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5795+0+0 (secure 0 0 0) 0x7f9398120120 con 0x7f93900039c0 2024-09-19T08:39:28.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.903+0000 7f93a5d9b640 1 --2- 172.21.15.98:0/1845140234 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9388067a70 0x7f9388069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:28.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.903+0000 7f93a5d9b640 1 --2- 172.21.15.98:0/1845140234 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9388067a70 0x7f9388069f30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f93a81f86c0 tx=0x7f939c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:28.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:28.907+0000 7f9396ffd640 1 -- 172.21.15.98:0/1845140234 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f93980170b0 con 0x7f93900039c0 2024-09-19T08:39:29.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.071+0000 7f93ac9bf640 1 -- 172.21.15.98:0/1845140234 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f93a815e3c0 con 0x7f93900039c0 2024-09-19T08:39:29.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.755+0000 7f9396ffd640 1 -- 172.21.15.98:0/1845140234 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v34) ==== 181+0+0 (secure 0 0 0) 0x7f93980f08e0 con 0x7f93900039c0 2024-09-19T08:39:29.755 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_metadata' removed 2024-09-19T08:39:29.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.759+0000 7f93ac9bf640 1 -- 172.21.15.98:0/1845140234 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9388067a70 msgr2=0x7f9388069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:29.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.759+0000 7f93ac9bf640 1 --2- 172.21.15.98:0/1845140234 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9388067a70 0x7f9388069f30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f93a81f86c0 tx=0x7f939c008040 comp rx=0 tx=0).stop 2024-09-19T08:39:29.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.759+0000 7f93ac9bf640 1 -- 172.21.15.98:0/1845140234 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93900039c0 msgr2=0x7f93a814bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:29.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.759+0000 7f93ac9bf640 1 --2- 172.21.15.98:0/1845140234 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93900039c0 0x7f93a814bdb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f939801d110 tx=0x7f9390005840 comp rx=0 tx=0).stop 2024-09-19T08:39:29.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.759+0000 7f93ac9bf640 1 -- 172.21.15.98:0/1845140234 shutdown_connections 2024-09-19T08:39:29.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.759+0000 7f93ac9bf640 1 --2- 172.21.15.98:0/1845140234 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9388067a70 0x7f9388069f30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:29.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.759+0000 7f93ac9bf640 1 --2- 172.21.15.98:0/1845140234 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93a81f5190 0x7f93a81f5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:29.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.759+0000 7f93ac9bf640 1 --2- 172.21.15.98:0/1845140234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f93a815a530 0x7f93a815a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:29.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.759+0000 7f93ac9bf640 1 --2- 172.21.15.98:0/1845140234 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f93900039c0 0x7f93a814bdb0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:29.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.759+0000 7f93ac9bf640 1 -- 172.21.15.98:0/1845140234 >> 172.21.15.98:0/1845140234 conn(0x7f93a806c090 msgr2=0x7f93a815daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:29.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.759+0000 7f93ac9bf640 1 -- 172.21.15.98:0/1845140234 shutdown_connections 2024-09-19T08:39:29.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.763+0000 7f93ac9bf640 1 -- 172.21.15.98:0/1845140234 wait complete. 2024-09-19T08:39:29.783 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T08:39:29.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.911+0000 7f65ce2be640 1 Processor -- start 2024-09-19T08:39:29.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.911+0000 7f65ce2be640 1 -- start start 2024-09-19T08:39:29.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.915+0000 7f65ce2be640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f65c80cf430 0x7f65c8166050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:29.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.915+0000 7f65ce2be640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65c815baf0 con 0x7f65c815f6e0 2024-09-19T08:39:29.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.915+0000 7f65ce2be640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65c815bdd0 con 0x7f65c806dbd0 2024-09-19T08:39:29.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.915+0000 7f65ce2be640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f65c815c8b0 con 0x7f65c80cf430 2024-09-19T08:39:29.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.915+0000 7f65c7fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f65c815f6e0 0x7f65c806d4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36194/0 (socket says 172.21.15.98:36194) 2024-09-19T08:39:29.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.915+0000 7f65c7fff640 1 -- 172.21.15.98:0/4070487369 learned_addr learned my addr 172.21.15.98:0/4070487369 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:29.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.915+0000 7f65c6ffd640 1 -- 172.21.15.98:0/4070487369 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 407916833 0 0) 0x7f65c815baf0 con 0x7f65c815f6e0 2024-09-19T08:39:29.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.915+0000 7f65c6ffd640 1 -- 172.21.15.98:0/4070487369 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65b4003530 con 0x7f65c815f6e0 2024-09-19T08:39:29.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.915+0000 7f65cc85b640 1 --2- 172.21.15.98:0/4070487369 >> v2:172.21.15.139:3300/0 conn(0x7f65c80cf430 0x7f65c8166050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:29.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.915+0000 7f65c6ffd640 1 -- 172.21.15.98:0/4070487369 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3549559290 0 0) 0x7f65b4003530 con 0x7f65c815f6e0 2024-09-19T08:39:29.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.915+0000 7f65c6ffd640 1 -- 172.21.15.98:0/4070487369 >> v2:172.21.15.139:3300/0 conn(0x7f65c80cf430 msgr2=0x7f65c8166050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:29.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.915+0000 7f65c6ffd640 1 --2- 172.21.15.98:0/4070487369 >> v2:172.21.15.139:3300/0 conn(0x7f65c80cf430 0x7f65c8166050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:29.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.915+0000 7f65c6ffd640 1 -- 172.21.15.98:0/4070487369 >> v1:172.21.15.139:6789/0 conn(0x7f65c806dbd0 legacy=0x7f65c8073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:29.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.915+0000 7f65c6ffd640 1 -- 172.21.15.98:0/4070487369 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c815c0b0 con 0x7f65c815f6e0 2024-09-19T08:39:29.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.915+0000 7f65c6ffd640 1 -- 172.21.15.98:0/4070487369 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f65b8001a00 con 0x7f65c815f6e0 2024-09-19T08:39:29.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.915+0000 7f65c6ffd640 1 -- 172.21.15.98:0/4070487369 >> v1:172.21.15.98:6789/0 conn(0x7f65c815f6e0 legacy=0x7f65c806d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:29.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.915+0000 7f65c6ffd640 1 --2- 172.21.15.98:0/4070487369 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65b40039c0 0x7f65b4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:29.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.915+0000 7f65c6ffd640 1 --2- 172.21.15.98:0/4070487369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65b4004810 0x7f65b4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:29.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.915+0000 7f65c6ffd640 1 --2- 172.21.15.98:0/4070487369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65b40051d0 0x7f65b400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:29.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.915+0000 7f65c6ffd640 1 -- 172.21.15.98:0/4070487369 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f65c815c8b0 con 0x7f65b4004810 2024-09-19T08:39:29.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.915+0000 7f65c6ffd640 1 -- 172.21.15.98:0/4070487369 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f65b400c120 con 0x7f65b40051d0 2024-09-19T08:39:29.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.915+0000 7f65c6ffd640 1 -- 172.21.15.98:0/4070487369 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f65b400c320 con 0x7f65b40039c0 2024-09-19T08:39:29.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.915+0000 7f65c77fe640 1 --2- 172.21.15.98:0/4070487369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65b4004810 0x7f65b4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:29.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.915+0000 7f65cc85b640 1 --2- 172.21.15.98:0/4070487369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65b40051d0 0x7f65b400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:29.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.915+0000 7f65c7fff640 1 --2- 172.21.15.98:0/4070487369 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65b40039c0 0x7f65b4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:29.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.915+0000 7f65c77fe640 1 -- 172.21.15.98:0/4070487369 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65b40039c0 msgr2=0x7f65b4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:29.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.915+0000 7f65c77fe640 1 --2- 172.21.15.98:0/4070487369 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65b40039c0 0x7f65b4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:29.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.915+0000 7f65c77fe640 1 -- 172.21.15.98:0/4070487369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65b40051d0 msgr2=0x7f65b400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:29.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.915+0000 7f65c77fe640 1 --2- 172.21.15.98:0/4070487369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65b40051d0 0x7f65b400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:29.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.915+0000 7f65cc85b640 1 -- 172.21.15.98:0/4070487369 reap_dead start 2024-09-19T08:39:29.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.915+0000 7f65c77fe640 1 -- 172.21.15.98:0/4070487369 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b4003530 con 0x7f65b4004810 2024-09-19T08:39:29.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.919+0000 7f65c77fe640 1 --2- 172.21.15.98:0/4070487369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65b4004810 0x7f65b4004c90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f65b0001750 tx=0x7f65b001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e836d71d3028e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:29.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.919+0000 7f65c6ffd640 1 -- 172.21.15.98:0/4070487369 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b00013a0 con 0x7f65b4004810 2024-09-19T08:39:29.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.919+0000 7f65c6ffd640 1 -- 172.21.15.98:0/4070487369 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65b00052f0 con 0x7f65b4004810 2024-09-19T08:39:29.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.919+0000 7f65c6ffd640 1 -- 172.21.15.98:0/4070487369 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b0003840 con 0x7f65b4004810 2024-09-19T08:39:29.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.919+0000 7f65ce2be640 1 -- 172.21.15.98:0/4070487369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65b4004810 msgr2=0x7f65b4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:29.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.919+0000 7f65ce2be640 1 --2- 172.21.15.98:0/4070487369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65b4004810 0x7f65b4004c90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f65b0001750 tx=0x7f65b001c3c0 comp rx=0 tx=0).stop 2024-09-19T08:39:29.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.919+0000 7f65ce2be640 1 -- 172.21.15.98:0/4070487369 shutdown_connections 2024-09-19T08:39:29.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.919+0000 7f65ce2be640 1 --2- 172.21.15.98:0/4070487369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65b4004810 0x7f65b4004c90 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:29.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.919+0000 7f65ce2be640 1 -- 172.21.15.98:0/4070487369 >> 172.21.15.98:0/4070487369 conn(0x7f65c806c090 msgr2=0x7f65c806c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:29.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.919+0000 7f65ce2be640 1 -- 172.21.15.98:0/4070487369 shutdown_connections 2024-09-19T08:39:29.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.919+0000 7f65ce2be640 1 -- 172.21.15.98:0/4070487369 wait complete. 2024-09-19T08:39:29.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.919+0000 7f65ce2be640 1 Processor -- start 2024-09-19T08:39:29.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.919+0000 7f65ce2be640 1 -- start start 2024-09-19T08:39:29.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.919+0000 7f65ce2be640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65b4004810 0x7f65c8140dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:29.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.919+0000 7f65ce2be640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65c8155360 0x7f65c81557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:29.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.919+0000 7f65ce2be640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65c80b48a0 0x7f65c80b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:29.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.919+0000 7f65ce2be640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f65c815b890 con 0x7f65b4004810 2024-09-19T08:39:29.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.919+0000 7f65ce2be640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f65c80742d0 con 0x7f65c8155360 2024-09-19T08:39:29.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.919+0000 7f65ce2be640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f65c815fac0 con 0x7f65c80b48a0 2024-09-19T08:39:29.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.919+0000 7f65c7fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65b4004810 0x7f65c8140dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:29.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.919+0000 7f65c7fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65b4004810 0x7f65c8140dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48402/0 (socket says 172.21.15.98:48402) 2024-09-19T08:39:29.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.919+0000 7f65c7fff640 1 -- 172.21.15.98:0/3457402029 learned_addr learned my addr 172.21.15.98:0/3457402029 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:29.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.919+0000 7f65c77fe640 1 --2- 172.21.15.98:0/3457402029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65c8155360 0x7f65c81557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:29.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.919+0000 7f65cc85b640 1 --2- 172.21.15.98:0/3457402029 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65c80b48a0 0x7f65c80b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:29.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.919+0000 7f65c7fff640 1 -- 172.21.15.98:0/3457402029 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65c80b48a0 msgr2=0x7f65c80b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:29.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.919+0000 7f65c7fff640 1 --2- 172.21.15.98:0/3457402029 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65c80b48a0 0x7f65c80b4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:29.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.919+0000 7f65c7fff640 1 -- 172.21.15.98:0/3457402029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65c8155360 msgr2=0x7f65c81557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:29.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.919+0000 7f65c7fff640 1 --2- 172.21.15.98:0/3457402029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65c8155360 0x7f65c81557e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:29.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.919+0000 7f65c7fff640 1 -- 172.21.15.98:0/3457402029 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c814dad0 con 0x7f65b4004810 2024-09-19T08:39:29.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.919+0000 7f65c7fff640 1 --2- 172.21.15.98:0/3457402029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65b4004810 0x7f65c8140dc0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f65b80118e0 tx=0x7f65b8011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:29.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.919+0000 7f65c4ff9640 1 -- 172.21.15.98:0/3457402029 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b8010040 con 0x7f65b4004810 2024-09-19T08:39:29.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.923+0000 7f65c4ff9640 1 -- 172.21.15.98:0/3457402029 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65b801a070 con 0x7f65b4004810 2024-09-19T08:39:29.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.923+0000 7f65ce2be640 1 -- 172.21.15.98:0/3457402029 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c814ca30 con 0x7f65b4004810 2024-09-19T08:39:29.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.923+0000 7f65c4ff9640 1 -- 172.21.15.98:0/3457402029 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b800fd20 con 0x7f65b4004810 2024-09-19T08:39:29.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.923+0000 7f65ce2be640 1 -- 172.21.15.98:0/3457402029 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c814cff0 con 0x7f65b4004810 2024-09-19T08:39:29.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.923+0000 7f65ce2be640 1 -- 172.21.15.98:0/3457402029 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65c80b5260 con 0x7f65b4004810 2024-09-19T08:39:29.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.923+0000 7f65c4ff9640 1 -- 172.21.15.98:0/3457402029 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f65b801f460 con 0x7f65b4004810 2024-09-19T08:39:29.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.923+0000 7f65c4ff9640 1 --2- 172.21.15.98:0/3457402029 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f65a8067a70 0x7f65a8069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:29.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.923+0000 7f65c4ff9640 1 -- 172.21.15.98:0/3457402029 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5357+0+0 (secure 0 0 0) 0x7f65b8089bb0 con 0x7f65b4004810 2024-09-19T08:39:29.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.923+0000 7f65c77fe640 1 --2- 172.21.15.98:0/3457402029 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f65a8067a70 0x7f65a8069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:29.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.923+0000 7f65c77fe640 1 --2- 172.21.15.98:0/3457402029 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f65a8067a70 0x7f65a8069f30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f65c8070e10 tx=0x7f65b00011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:29.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:29.927+0000 7f65c4ff9640 1 -- 172.21.15.98:0/3457402029 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f65b805a6b0 con 0x7f65b4004810 2024-09-19T08:39:30.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.087+0000 7f65ce2be640 1 -- 172.21.15.98:0/3457402029 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f65c806d4c0 con 0x7f65b4004810 2024-09-19T08:39:30.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.759+0000 7f65c4ff9640 1 -- 172.21.15.98:0/3457402029 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v35) ==== 169+0+0 (secure 0 0 0) 0x7f65b801de40 con 0x7f65b4004810 2024-09-19T08:39:30.757 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' removed 2024-09-19T08:39:30.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.763+0000 7f65ce2be640 1 -- 172.21.15.98:0/3457402029 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f65a8067a70 msgr2=0x7f65a8069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:30.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.763+0000 7f65ce2be640 1 --2- 172.21.15.98:0/3457402029 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f65a8067a70 0x7f65a8069f30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f65c8070e10 tx=0x7f65b00011a0 comp rx=0 tx=0).stop 2024-09-19T08:39:30.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.763+0000 7f65ce2be640 1 -- 172.21.15.98:0/3457402029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65b4004810 msgr2=0x7f65c8140dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:30.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.763+0000 7f65ce2be640 1 --2- 172.21.15.98:0/3457402029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65b4004810 0x7f65c8140dc0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f65b80118e0 tx=0x7f65b8011da0 comp rx=0 tx=0).stop 2024-09-19T08:39:30.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.763+0000 7f65ce2be640 1 -- 172.21.15.98:0/3457402029 shutdown_connections 2024-09-19T08:39:30.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.763+0000 7f65ce2be640 1 --2- 172.21.15.98:0/3457402029 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f65a8067a70 0x7f65a8069f30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:30.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.763+0000 7f65ce2be640 1 --2- 172.21.15.98:0/3457402029 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65c80b48a0 0x7f65c80b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:30.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.763+0000 7f65ce2be640 1 --2- 172.21.15.98:0/3457402029 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65c8155360 0x7f65c81557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:30.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.763+0000 7f65ce2be640 1 --2- 172.21.15.98:0/3457402029 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f65b4004810 0x7f65c8140dc0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:30.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.763+0000 7f65ce2be640 1 -- 172.21.15.98:0/3457402029 >> 172.21.15.98:0/3457402029 conn(0x7f65c806c090 msgr2=0x7f65c815dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:30.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.763+0000 7f65ce2be640 1 -- 172.21.15.98:0/3457402029 shutdown_connections 2024-09-19T08:39:30.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.763+0000 7f65ce2be640 1 -- 172.21.15.98:0/3457402029 wait complete. 2024-09-19T08:39:30.787 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-19T08:39:30.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.911+0000 7fcd221d0640 1 Processor -- start 2024-09-19T08:39:30.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.911+0000 7fcd221d0640 1 -- start start 2024-09-19T08:39:30.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.911+0000 7fcd221d0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd1c15c730 0x7fcd1c0b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:30.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.911+0000 7fcd221d0640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcd1c0b3770 0x7fcd1c0cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:30.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.911+0000 7fcd221d0640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd1c0b08c0 con 0x7fcd1c06caf0 2024-09-19T08:39:30.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.911+0000 7fcd221d0640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fcd1c0b0ba0 con 0x7fcd1c0b3770 2024-09-19T08:39:30.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.911+0000 7fcd221d0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcd1c0b0e80 con 0x7fcd1c15c730 2024-09-19T08:39:30.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.911+0000 7fcd1b7fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fcd1c06caf0 0x7fcd1c06ced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36196/0 (socket says 172.21.15.98:36196) 2024-09-19T08:39:30.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.911+0000 7fcd1b7fe640 1 -- 172.21.15.98:0/2835835556 learned_addr learned my addr 172.21.15.98:0/2835835556 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:30.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.911+0000 7fcd1bfff640 1 --2- 172.21.15.98:0/2835835556 >> v2:172.21.15.98:3300/0 conn(0x7fcd1c0b3770 0x7fcd1c0cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:30.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.911+0000 7fcd1affd640 1 --2- 172.21.15.98:0/2835835556 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd1c15c730 0x7fcd1c0b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:30.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.911+0000 7fcd1a7fc640 1 -- 172.21.15.98:0/2835835556 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3884223976 0 0) 0x7fcd1c0b08c0 con 0x7fcd1c06caf0 2024-09-19T08:39:30.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.911+0000 7fcd1a7fc640 1 -- 172.21.15.98:0/2835835556 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd04003530 con 0x7fcd1c06caf0 2024-09-19T08:39:30.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.911+0000 7fcd1bfff640 1 -- 172.21.15.98:0/2835835556 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd1c15c730 msgr2=0x7fcd1c0b02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:30.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.911+0000 7fcd1bfff640 1 --2- 172.21.15.98:0/2835835556 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd1c15c730 0x7fcd1c0b02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:30.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.911+0000 7fcd1bfff640 1 -- 172.21.15.98:0/2835835556 >> v1:172.21.15.98:6789/0 conn(0x7fcd1c06caf0 legacy=0x7fcd1c06ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:30.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.911+0000 7fcd1bfff640 1 -- 172.21.15.98:0/2835835556 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd1c0b1160 con 0x7fcd1c0b3770 2024-09-19T08:39:30.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.915+0000 7fcd1bfff640 1 --2- 172.21.15.98:0/2835835556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd1c0b3770 0x7fcd1c0cbc00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fcd140097c0 tx=0x7fcd1401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52baf61915fb49af server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:30.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.915+0000 7fcd1a7fc640 1 -- 172.21.15.98:0/2835835556 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd140aa5f0 con 0x7fcd1c0b3770 2024-09-19T08:39:30.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.915+0000 7fcd1a7fc640 1 -- 172.21.15.98:0/2835835556 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd140af070 con 0x7fcd1c0b3770 2024-09-19T08:39:30.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.915+0000 7fcd1a7fc640 1 -- 172.21.15.98:0/2835835556 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd140aa9b0 con 0x7fcd1c0b3770 2024-09-19T08:39:30.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.915+0000 7fcd221d0640 1 -- 172.21.15.98:0/2835835556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd1c0b3770 msgr2=0x7fcd1c0cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:30.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.915+0000 7fcd221d0640 1 --2- 172.21.15.98:0/2835835556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd1c0b3770 0x7fcd1c0cbc00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fcd140097c0 tx=0x7fcd1401c470 comp rx=0 tx=0).stop 2024-09-19T08:39:30.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.915+0000 7fcd221d0640 1 -- 172.21.15.98:0/2835835556 shutdown_connections 2024-09-19T08:39:30.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.915+0000 7fcd221d0640 1 --2- 172.21.15.98:0/2835835556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd1c0b3770 0x7fcd1c0cbc00 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:30.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.915+0000 7fcd221d0640 1 --2- 172.21.15.98:0/2835835556 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd1c15c730 0x7fcd1c0b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:30.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.915+0000 7fcd221d0640 1 -- 172.21.15.98:0/2835835556 >> 172.21.15.98:0/2835835556 conn(0x7fcd1c0b5f50 msgr2=0x7fcd1c0b6650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:30.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.915+0000 7fcd221d0640 1 -- 172.21.15.98:0/2835835556 shutdown_connections 2024-09-19T08:39:30.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.915+0000 7fcd221d0640 1 -- 172.21.15.98:0/2835835556 wait complete. 2024-09-19T08:39:30.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.915+0000 7fcd221d0640 1 Processor -- start 2024-09-19T08:39:30.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.915+0000 7fcd221d0640 1 -- start start 2024-09-19T08:39:30.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.915+0000 7fcd221d0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd1c06caf0 0x7fcd1c152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:30.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.915+0000 7fcd221d0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd1c0b3770 0x7fcd1c147220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:30.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.915+0000 7fcd221d0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd1c15c730 0x7fcd1c1577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:30.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.915+0000 7fcd221d0640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c169120 con 0x7fcd1c15c730 2024-09-19T08:39:30.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.915+0000 7fcd221d0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c0cc350 con 0x7fcd1c06caf0 2024-09-19T08:39:30.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.915+0000 7fcd221d0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcd1c1692a0 con 0x7fcd1c0b3770 2024-09-19T08:39:30.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.915+0000 7fcd1bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd1c15c730 0x7fcd1c1577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:30.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.915+0000 7fcd1bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd1c15c730 0x7fcd1c1577c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48426/0 (socket says 172.21.15.98:48426) 2024-09-19T08:39:30.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.915+0000 7fcd1bfff640 1 -- 172.21.15.98:0/3161741358 learned_addr learned my addr 172.21.15.98:0/3161741358 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:30.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.915+0000 7fcd1b7fe640 1 --2- 172.21.15.98:0/3161741358 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd1c06caf0 0x7fcd1c152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:30.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.915+0000 7fcd1affd640 1 --2- 172.21.15.98:0/3161741358 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd1c0b3770 0x7fcd1c147220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:30.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.915+0000 7fcd1bfff640 1 -- 172.21.15.98:0/3161741358 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd1c0b3770 msgr2=0x7fcd1c147220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:30.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.915+0000 7fcd1bfff640 1 --2- 172.21.15.98:0/3161741358 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd1c0b3770 0x7fcd1c147220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:30.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.915+0000 7fcd1bfff640 1 -- 172.21.15.98:0/3161741358 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd1c06caf0 msgr2=0x7fcd1c152030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:30.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.915+0000 7fcd1bfff640 1 --2- 172.21.15.98:0/3161741358 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd1c06caf0 0x7fcd1c152030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:30.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.915+0000 7fcd1bfff640 1 -- 172.21.15.98:0/3161741358 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd1c1f9470 con 0x7fcd1c15c730 2024-09-19T08:39:30.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.915+0000 7fcd1bfff640 1 --2- 172.21.15.98:0/3161741358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd1c15c730 0x7fcd1c1577c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcd140ac040 tx=0x7fcd1401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:30.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.915+0000 7fcd03fff640 1 -- 172.21.15.98:0/3161741358 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd140be410 con 0x7fcd1c15c730 2024-09-19T08:39:30.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.915+0000 7fcd03fff640 1 -- 172.21.15.98:0/3161741358 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd140be5b0 con 0x7fcd1c15c730 2024-09-19T08:39:30.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.915+0000 7fcd221d0640 1 -- 172.21.15.98:0/3161741358 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd1c1f9750 con 0x7fcd1c15c730 2024-09-19T08:39:30.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.915+0000 7fcd03fff640 1 -- 172.21.15.98:0/3161741358 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd140be9a0 con 0x7fcd1c15c730 2024-09-19T08:39:30.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.919+0000 7fcd221d0640 1 -- 172.21.15.98:0/3161741358 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd1c1f9d10 con 0x7fcd1c15c730 2024-09-19T08:39:30.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.919+0000 7fcd221d0640 1 -- 172.21.15.98:0/3161741358 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd1c0cbc00 con 0x7fcd1c15c730 2024-09-19T08:39:30.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.919+0000 7fcd03fff640 1 -- 172.21.15.98:0/3161741358 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fcd140bc020 con 0x7fcd1c15c730 2024-09-19T08:39:30.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.919+0000 7fcd03fff640 1 --2- 172.21.15.98:0/3161741358 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fcd08067aa0 0x7fcd08069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:30.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.919+0000 7fcd03fff640 1 -- 172.21.15.98:0/3161741358 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7fcd14120300 con 0x7fcd1c15c730 2024-09-19T08:39:30.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.919+0000 7fcd1b7fe640 1 --2- 172.21.15.98:0/3161741358 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fcd08067aa0 0x7fcd08069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:30.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.919+0000 7fcd1b7fe640 1 --2- 172.21.15.98:0/3161741358 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fcd08067aa0 0x7fcd08069f60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c005c10 tx=0x7fcd0c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:30.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:30.923+0000 7fcd03fff640 1 -- 172.21.15.98:0/3161741358 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fcd14016090 con 0x7fcd1c15c730 2024-09-19T08:39:31.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.087+0000 7fcd221d0640 1 -- 172.21.15.98:0/3161741358 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fcd1c0b7000 con 0x7fcd1c15c730 2024-09-19T08:39:31.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.095+0000 7fcd03fff640 1 -- 172.21.15.98:0/3161741358 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7fcd140f0e40 con 0x7fcd1c15c730 2024-09-19T08:39:31.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.095+0000 7fcd221d0640 1 -- 172.21.15.98:0/3161741358 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fcd08067aa0 msgr2=0x7fcd08069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:31.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.095+0000 7fcd221d0640 1 --2- 172.21.15.98:0/3161741358 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fcd08067aa0 0x7fcd08069f60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c005c10 tx=0x7fcd0c009040 comp rx=0 tx=0).stop 2024-09-19T08:39:31.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.095+0000 7fcd221d0640 1 -- 172.21.15.98:0/3161741358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd1c15c730 msgr2=0x7fcd1c1577c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:31.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.095+0000 7fcd221d0640 1 --2- 172.21.15.98:0/3161741358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd1c15c730 0x7fcd1c1577c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcd140ac040 tx=0x7fcd1401acb0 comp rx=0 tx=0).stop 2024-09-19T08:39:31.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.095+0000 7fcd221d0640 1 -- 172.21.15.98:0/3161741358 shutdown_connections 2024-09-19T08:39:31.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.095+0000 7fcd221d0640 1 --2- 172.21.15.98:0/3161741358 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fcd08067aa0 0x7fcd08069f60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:31.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.095+0000 7fcd221d0640 1 --2- 172.21.15.98:0/3161741358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd1c15c730 0x7fcd1c1577c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:31.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.095+0000 7fcd221d0640 1 --2- 172.21.15.98:0/3161741358 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd1c0b3770 0x7fcd1c147220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:31.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.095+0000 7fcd221d0640 1 --2- 172.21.15.98:0/3161741358 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd1c06caf0 0x7fcd1c152030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:31.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.095+0000 7fcd221d0640 1 -- 172.21.15.98:0/3161741358 >> 172.21.15.98:0/3161741358 conn(0x7fcd1c0b5f50 msgr2=0x7fcd1c06f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:31.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.095+0000 7fcd221d0640 1 -- 172.21.15.98:0/3161741358 shutdown_connections 2024-09-19T08:39:31.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.095+0000 7fcd221d0640 1 -- 172.21.15.98:0/3161741358 wait complete. 2024-09-19T08:39:31.113 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2024-09-19T08:39:31.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.235+0000 7f3578a80640 1 Processor -- start 2024-09-19T08:39:31.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.235+0000 7f3578a80640 1 -- start start 2024-09-19T08:39:31.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.235+0000 7f3578a80640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f35740cd350 0x7f35740b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:31.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.235+0000 7f3578a80640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f35740ce380 0x7f35740cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:31.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.235+0000 7f3578a80640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35740cd730 con 0x7f35740b6f50 2024-09-19T08:39:31.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.235+0000 7f3578a80640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f35740b5f60 con 0x7f35740cd350 2024-09-19T08:39:31.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.235+0000 7f3578a80640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f35740b6190 con 0x7f35740ce380 2024-09-19T08:39:31.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.235+0000 7f357259c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f35740cd350 0x7f35740b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:31.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.235+0000 7f357259c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f35740cd350 0x7f35740b24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48432/0 (socket says 172.21.15.98:48432) 2024-09-19T08:39:31.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.235+0000 7f357259c640 1 -- 172.21.15.98:0/493312385 learned_addr learned my addr 172.21.15.98:0/493312385 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:31.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.235+0000 7f3572d9d640 1 --2- 172.21.15.98:0/493312385 >> v2:172.21.15.139:3300/0 conn(0x7f35740ce380 0x7f35740cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:31.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.235+0000 7f357259c640 1 -- 172.21.15.98:0/493312385 >> v2:172.21.15.139:3300/0 conn(0x7f35740ce380 msgr2=0x7f35740cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:31.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.235+0000 7f357259c640 1 --2- 172.21.15.98:0/493312385 >> v2:172.21.15.139:3300/0 conn(0x7f35740ce380 0x7f35740cb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:31.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.235+0000 7f357259c640 1 -- 172.21.15.98:0/493312385 >> v1:172.21.15.139:6789/0 conn(0x7f35740b6f50 legacy=0x7f35740cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:31.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.235+0000 7f357259c640 1 -- 172.21.15.98:0/493312385 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35740b6470 con 0x7f35740cd350 2024-09-19T08:39:31.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.239+0000 7f357259c640 1 --2- 172.21.15.98:0/493312385 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35740cd350 0x7f35740b24b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3574070700 tx=0x7f355c01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e38daf1547a458e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:31.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.239+0000 7f357159a640 1 -- 172.21.15.98:0/493312385 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c0aa610 con 0x7f35740cd350 2024-09-19T08:39:31.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.239+0000 7f357159a640 1 -- 172.21.15.98:0/493312385 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f355c0af070 con 0x7f35740cd350 2024-09-19T08:39:31.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.239+0000 7f357159a640 1 -- 172.21.15.98:0/493312385 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c0aa9d0 con 0x7f35740cd350 2024-09-19T08:39:31.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.239+0000 7f3578a80640 1 -- 172.21.15.98:0/493312385 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35740cd350 msgr2=0x7f35740b24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:31.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.239+0000 7f3578a80640 1 --2- 172.21.15.98:0/493312385 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35740cd350 0x7f35740b24b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3574070700 tx=0x7f355c01c4b0 comp rx=0 tx=0).stop 2024-09-19T08:39:31.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.239+0000 7f3578a80640 1 -- 172.21.15.98:0/493312385 shutdown_connections 2024-09-19T08:39:31.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.239+0000 7f3578a80640 1 --2- 172.21.15.98:0/493312385 >> v2:172.21.15.139:3300/0 conn(0x7f35740ce380 0x7f35740cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:31.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.239+0000 7f3578a80640 1 --2- 172.21.15.98:0/493312385 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35740cd350 0x7f35740b24b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:31.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.239+0000 7f3578a80640 1 -- 172.21.15.98:0/493312385 >> 172.21.15.98:0/493312385 conn(0x7f3574073170 msgr2=0x7f3574073580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:31.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.239+0000 7f3578a80640 1 -- 172.21.15.98:0/493312385 shutdown_connections 2024-09-19T08:39:31.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.239+0000 7f3578a80640 1 -- 172.21.15.98:0/493312385 wait complete. 2024-09-19T08:39:31.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.239+0000 7f3578a80640 1 Processor -- start 2024-09-19T08:39:31.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.239+0000 7f3578a80640 1 -- start start 2024-09-19T08:39:31.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.239+0000 7f3578a80640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35740b6f50 0x7f3574157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:31.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.239+0000 7f3578a80640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35740cd350 0x7f3574157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:31.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.239+0000 7f3578a80640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f35740ce380 0x7f357406c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:31.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.239+0000 7f3578a80640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3574166180 con 0x7f35740b6f50 2024-09-19T08:39:31.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.239+0000 7f3578a80640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3574166000 con 0x7f35740cd350 2024-09-19T08:39:31.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.239+0000 7f3578a80640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3574166300 con 0x7f35740ce380 2024-09-19T08:39:31.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.239+0000 7f357259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35740b6f50 0x7f3574157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:31.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.239+0000 7f357259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35740b6f50 0x7f3574157390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48434/0 (socket says 172.21.15.98:48434) 2024-09-19T08:39:31.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.239+0000 7f357259c640 1 -- 172.21.15.98:0/1740057430 learned_addr learned my addr 172.21.15.98:0/1740057430 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:31.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.239+0000 7f3572d9d640 1 --2- 172.21.15.98:0/1740057430 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f35740ce380 0x7f357406c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:31.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.239+0000 7f357259c640 1 -- 172.21.15.98:0/1740057430 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f35740ce380 msgr2=0x7f357406c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:31.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.239+0000 7f3571d9b640 1 --2- 172.21.15.98:0/1740057430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35740cd350 0x7f3574157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:31.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.239+0000 7f357259c640 1 --2- 172.21.15.98:0/1740057430 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f35740ce380 0x7f357406c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:31.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.239+0000 7f357259c640 1 -- 172.21.15.98:0/1740057430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35740cd350 msgr2=0x7f3574157c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:31.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.239+0000 7f357259c640 1 --2- 172.21.15.98:0/1740057430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35740cd350 0x7f3574157c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:31.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.239+0000 7f357259c640 1 -- 172.21.15.98:0/1740057430 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3574158610 con 0x7f35740b6f50 2024-09-19T08:39:31.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.239+0000 7f3572d9d640 1 --2- 172.21.15.98:0/1740057430 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f35740ce380 0x7f357406c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:31.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.239+0000 7f3571d9b640 1 --2- 172.21.15.98:0/1740057430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35740cd350 0x7f3574157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:31.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.239+0000 7f357259c640 1 --2- 172.21.15.98:0/1740057430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35740b6f50 0x7f3574157390 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f355c01a0e0 tx=0x7f355c01acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:31.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.243+0000 7f356affd640 1 -- 172.21.15.98:0/1740057430 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c0bd5f0 con 0x7f35740b6f50 2024-09-19T08:39:31.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.243+0000 7f356affd640 1 -- 172.21.15.98:0/1740057430 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f355c0bd790 con 0x7f35740b6f50 2024-09-19T08:39:31.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.243+0000 7f3578a80640 1 -- 172.21.15.98:0/1740057430 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35741588f0 con 0x7f35740b6f50 2024-09-19T08:39:31.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.243+0000 7f356affd640 1 -- 172.21.15.98:0/1740057430 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c0bdb40 con 0x7f35740b6f50 2024-09-19T08:39:31.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.243+0000 7f3578a80640 1 -- 172.21.15.98:0/1740057430 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3574158eb0 con 0x7f35740b6f50 2024-09-19T08:39:31.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.243+0000 7f3578a80640 1 -- 172.21.15.98:0/1740057430 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35740b24b0 con 0x7f35740b6f50 2024-09-19T08:39:31.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.243+0000 7f356affd640 1 -- 172.21.15.98:0/1740057430 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f355c0b33f0 con 0x7f35740b6f50 2024-09-19T08:39:31.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.243+0000 7f356affd640 1 --2- 172.21.15.98:0/1740057430 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3558067a70 0x7f3558069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:31.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.243+0000 7f356affd640 1 -- 172.21.15.98:0/1740057430 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f355c120b90 con 0x7f35740b6f50 2024-09-19T08:39:31.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.243+0000 7f3571d9b640 1 --2- 172.21.15.98:0/1740057430 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3558067a70 0x7f3558069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:31.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.243+0000 7f3571d9b640 1 --2- 172.21.15.98:0/1740057430 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3558067a70 0x7f3558069f30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f357406d190 tx=0x7f3564009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:31.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.247+0000 7f356affd640 1 -- 172.21.15.98:0/1740057430 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f355c0c3060 con 0x7f35740b6f50 2024-09-19T08:39:31.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.471+0000 7f3578a80640 1 -- 172.21.15.98:0/1740057430 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7f35740cef70 con 0x7f35740b6f50 2024-09-19T08:39:31.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.771+0000 7f356affd640 1 -- 172.21.15.98:0/1740057430 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v87) ==== 158+0+0 (secure 0 0 0) 0x7f355c0a7070 con 0x7f35740b6f50 2024-09-19T08:39:31.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.775+0000 7f3578a80640 1 -- 172.21.15.98:0/1740057430 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3558067a70 msgr2=0x7f3558069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:31.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.775+0000 7f3578a80640 1 --2- 172.21.15.98:0/1740057430 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3558067a70 0x7f3558069f30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f357406d190 tx=0x7f3564009040 comp rx=0 tx=0).stop 2024-09-19T08:39:31.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.775+0000 7f3578a80640 1 -- 172.21.15.98:0/1740057430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35740b6f50 msgr2=0x7f3574157390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:31.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.775+0000 7f3578a80640 1 --2- 172.21.15.98:0/1740057430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35740b6f50 0x7f3574157390 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f355c01a0e0 tx=0x7f355c01acf0 comp rx=0 tx=0).stop 2024-09-19T08:39:31.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.775+0000 7f3578a80640 1 -- 172.21.15.98:0/1740057430 shutdown_connections 2024-09-19T08:39:31.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.775+0000 7f3578a80640 1 --2- 172.21.15.98:0/1740057430 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3558067a70 0x7f3558069f30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:31.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.775+0000 7f3578a80640 1 --2- 172.21.15.98:0/1740057430 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f35740ce380 0x7f357406c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:31.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.775+0000 7f3578a80640 1 --2- 172.21.15.98:0/1740057430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35740cd350 0x7f3574157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:31.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.775+0000 7f3578a80640 1 --2- 172.21.15.98:0/1740057430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35740b6f50 0x7f3574157390 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:31.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.775+0000 7f3578a80640 1 -- 172.21.15.98:0/1740057430 >> 172.21.15.98:0/1740057430 conn(0x7f3574073170 msgr2=0x7f357415e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:31.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.775+0000 7f3578a80640 1 -- 172.21.15.98:0/1740057430 shutdown_connections 2024-09-19T08:39:31.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.775+0000 7f3578a80640 1 -- 172.21.15.98:0/1740057430 wait complete. 2024-09-19T08:39:31.794 INFO:tasks.cephfs_test_runner:Validate binary .meta file on legacy subvol root doesn't break the system ... ok 2024-09-19T08:39:31.794 INFO:tasks.cephfs_test_runner:Starting test: test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) 2024-09-19T08:39:31.794 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:39:31.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.923+0000 7f06415d6640 1 Processor -- start 2024-09-19T08:39:31.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.923+0000 7f06415d6640 1 -- start start 2024-09-19T08:39:31.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.923+0000 7f06415d6640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f063c15e750 0x7f063c073c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:31.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.923+0000 7f06415d6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f063c0b28e0 0x7f063c074c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:31.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.923+0000 7f06415d6640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f063c075b80 con 0x7f063c0751b0 2024-09-19T08:39:31.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.923+0000 7f06415d6640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f063c0d1bd0 con 0x7f063c15e750 2024-09-19T08:39:31.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.923+0000 7f06415d6640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f063c0d1eb0 con 0x7f063c0b28e0 2024-09-19T08:39:31.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.923+0000 7f063affd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f063c15e750 0x7f063c073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:31.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.923+0000 7f063affd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f063c15e750 0x7f063c073c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48442/0 (socket says 172.21.15.98:48442) 2024-09-19T08:39:31.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.923+0000 7f063affd640 1 -- 172.21.15.98:0/1454247412 learned_addr learned my addr 172.21.15.98:0/1454247412 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:31.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.923+0000 7f06327fc640 1 --2- 172.21.15.98:0/1454247412 >> v2:172.21.15.139:3300/0 conn(0x7f063c0b28e0 0x7f063c074c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:31.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.923+0000 7f063affd640 1 -- 172.21.15.98:0/1454247412 >> v2:172.21.15.139:3300/0 conn(0x7f063c0b28e0 msgr2=0x7f063c074c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:31.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.923+0000 7f063affd640 1 --2- 172.21.15.98:0/1454247412 >> v2:172.21.15.139:3300/0 conn(0x7f063c0b28e0 0x7f063c074c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:31.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.923+0000 7f063affd640 1 -- 172.21.15.98:0/1454247412 >> v1:172.21.15.139:6789/0 conn(0x7f063c0751b0 legacy=0x7f063c0b6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:31.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.923+0000 7f063affd640 1 -- 172.21.15.98:0/1454247412 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f063c0cf000 con 0x7f063c15e750 2024-09-19T08:39:31.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.927+0000 7f063affd640 1 --2- 172.21.15.98:0/1454247412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f063c15e750 0x7f063c073c90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f063c071340 tx=0x7f062c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=822c98ecf3278aee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:31.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.927+0000 7f063a7fc640 1 -- 172.21.15.98:0/1454247412 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c0a7070 con 0x7f063c15e750 2024-09-19T08:39:31.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.927+0000 7f063a7fc640 1 -- 172.21.15.98:0/1454247412 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f062c0b0070 con 0x7f063c15e750 2024-09-19T08:39:31.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.927+0000 7f063a7fc640 1 -- 172.21.15.98:0/1454247412 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c0ab890 con 0x7f063c15e750 2024-09-19T08:39:31.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.927+0000 7f06415d6640 1 -- 172.21.15.98:0/1454247412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f063c15e750 msgr2=0x7f063c073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:31.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.927+0000 7f06415d6640 1 --2- 172.21.15.98:0/1454247412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f063c15e750 0x7f063c073c90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f063c071340 tx=0x7f062c01c450 comp rx=0 tx=0).stop 2024-09-19T08:39:31.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.927+0000 7f06415d6640 1 -- 172.21.15.98:0/1454247412 shutdown_connections 2024-09-19T08:39:31.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.927+0000 7f06415d6640 1 --2- 172.21.15.98:0/1454247412 >> v2:172.21.15.139:3300/0 conn(0x7f063c0b28e0 0x7f063c074c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:31.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.927+0000 7f06415d6640 1 --2- 172.21.15.98:0/1454247412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f063c15e750 0x7f063c073c90 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:31.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.927+0000 7f06415d6640 1 -- 172.21.15.98:0/1454247412 >> 172.21.15.98:0/1454247412 conn(0x7f063c073170 msgr2=0x7f063c073580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:31.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.927+0000 7f06415d6640 1 -- 172.21.15.98:0/1454247412 shutdown_connections 2024-09-19T08:39:31.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.927+0000 7f06415d6640 1 -- 172.21.15.98:0/1454247412 wait complete. 2024-09-19T08:39:31.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.927+0000 7f06415d6640 1 Processor -- start 2024-09-19T08:39:31.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.927+0000 7f06415d6640 1 -- start start 2024-09-19T08:39:31.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.927+0000 7f06415d6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f063c0751b0 0x7f063c0ca110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:31.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.927+0000 7f06415d6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f063c0b28e0 0x7f063c09a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:31.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.927+0000 7f06415d6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f063c15e750 0x7f063c15ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:31.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.927+0000 7f06415d6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f063c16b500 con 0x7f063c15e750 2024-09-19T08:39:31.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.927+0000 7f06415d6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f063c16b380 con 0x7f063c0b28e0 2024-09-19T08:39:31.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.927+0000 7f06415d6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f063c16b680 con 0x7f063c0751b0 2024-09-19T08:39:31.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.927+0000 7f063b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f063c15e750 0x7f063c15ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:31.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.927+0000 7f063b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f063c15e750 0x7f063c15ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48446/0 (socket says 172.21.15.98:48446) 2024-09-19T08:39:31.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.927+0000 7f063b7fe640 1 -- 172.21.15.98:0/61712805 learned_addr learned my addr 172.21.15.98:0/61712805 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:31.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.931+0000 7f06327fc640 1 --2- 172.21.15.98:0/61712805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f063c0b28e0 0x7f063c09a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:31.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.931+0000 7f063affd640 1 --2- 172.21.15.98:0/61712805 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f063c0751b0 0x7f063c0ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:31.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.931+0000 7f063b7fe640 1 -- 172.21.15.98:0/61712805 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f063c0751b0 msgr2=0x7f063c0ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:31.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.931+0000 7f063b7fe640 1 --2- 172.21.15.98:0/61712805 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f063c0751b0 0x7f063c0ca110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:31.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.931+0000 7f063b7fe640 1 -- 172.21.15.98:0/61712805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f063c0b28e0 msgr2=0x7f063c09a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:31.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.931+0000 7f063b7fe640 1 --2- 172.21.15.98:0/61712805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f063c0b28e0 0x7f063c09a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:31.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.931+0000 7f063b7fe640 1 -- 172.21.15.98:0/61712805 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f063c0c9070 con 0x7f063c15e750 2024-09-19T08:39:31.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.931+0000 7f063b7fe640 1 --2- 172.21.15.98:0/61712805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f063c15e750 0x7f063c15ae50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f063400c780 tx=0x7f063400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:31.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.931+0000 7f0633fff640 1 -- 172.21.15.98:0/61712805 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0634004b70 con 0x7f063c15e750 2024-09-19T08:39:31.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.931+0000 7f0633fff640 1 -- 172.21.15.98:0/61712805 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0634004d10 con 0x7f063c15e750 2024-09-19T08:39:31.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.931+0000 7f06415d6640 1 -- 172.21.15.98:0/61712805 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f063c0c9350 con 0x7f063c15e750 2024-09-19T08:39:31.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.931+0000 7f0633fff640 1 -- 172.21.15.98:0/61712805 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063400aa90 con 0x7f063c15e750 2024-09-19T08:39:31.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.931+0000 7f06415d6640 1 -- 172.21.15.98:0/61712805 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f063c0c9910 con 0x7f063c15e750 2024-09-19T08:39:31.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.931+0000 7f06415d6640 1 -- 172.21.15.98:0/61712805 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f063c074c70 con 0x7f063c15e750 2024-09-19T08:39:31.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.931+0000 7f0633fff640 1 -- 172.21.15.98:0/61712805 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f063400ac30 con 0x7f063c15e750 2024-09-19T08:39:31.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.931+0000 7f0633fff640 1 --2- 172.21.15.98:0/61712805 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f061c067a70 0x7f061c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:31.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.931+0000 7f063affd640 1 --2- 172.21.15.98:0/61712805 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f061c067a70 0x7f061c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:31.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.931+0000 7f0633fff640 1 -- 172.21.15.98:0/61712805 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f0634085f10 con 0x7f063c15e750 2024-09-19T08:39:31.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.931+0000 7f063affd640 1 --2- 172.21.15.98:0/61712805 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f061c067a70 0x7f061c069f30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f062c0af000 tx=0x7f062c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:31.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:31.935+0000 7f0633fff640 1 -- 172.21.15.98:0/61712805 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f0634020070 con 0x7f063c15e750 2024-09-19T08:39:32.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.099+0000 7f06415d6640 1 -- 172.21.15.98:0/61712805 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f063c15d820 con 0x7f063c15e750 2024-09-19T08:39:32.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.099+0000 7f0633fff640 1 -- 172.21.15.98:0/61712805 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+13032 (secure 0 0 0) 0x7f0634056b60 con 0x7f063c15e750 2024-09-19T08:39:32.100 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:39:32.100 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":35,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:39:30.756823+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.98:6837/1820856497":"2024-09-20T08:39:27.741875+0000","172.21.15.98:6836/1820856497":"2024-09-20T08:39:27.741875+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:39:32.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.103+0000 7f06415d6640 1 -- 172.21.15.98:0/61712805 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f061c067a70 msgr2=0x7f061c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:32.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.103+0000 7f06415d6640 1 --2- 172.21.15.98:0/61712805 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f061c067a70 0x7f061c069f30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f062c0af000 tx=0x7f062c0a7000 comp rx=0 tx=0).stop 2024-09-19T08:39:32.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.103+0000 7f06415d6640 1 -- 172.21.15.98:0/61712805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f063c15e750 msgr2=0x7f063c15ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:32.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.103+0000 7f06415d6640 1 --2- 172.21.15.98:0/61712805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f063c15e750 0x7f063c15ae50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f063400c780 tx=0x7f063400cc40 comp rx=0 tx=0).stop 2024-09-19T08:39:32.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.103+0000 7f06415d6640 1 -- 172.21.15.98:0/61712805 shutdown_connections 2024-09-19T08:39:32.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.103+0000 7f06415d6640 1 --2- 172.21.15.98:0/61712805 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f061c067a70 0x7f061c069f30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:32.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.103+0000 7f06415d6640 1 --2- 172.21.15.98:0/61712805 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f063c15e750 0x7f063c15ae50 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:32.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.103+0000 7f06415d6640 1 --2- 172.21.15.98:0/61712805 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f063c0b28e0 0x7f063c09a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:32.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.103+0000 7f06415d6640 1 --2- 172.21.15.98:0/61712805 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f063c0751b0 0x7f063c0ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:32.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.103+0000 7f06415d6640 1 -- 172.21.15.98:0/61712805 >> 172.21.15.98:0/61712805 conn(0x7f063c073170 msgr2=0x7f063c06e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:32.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.103+0000 7f06415d6640 1 -- 172.21.15.98:0/61712805 shutdown_connections 2024-09-19T08:39:32.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.103+0000 7f06415d6640 1 -- 172.21.15.98:0/61712805 wait complete. 2024-09-19T08:39:32.119 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T08:38:08.105684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T08:39:32.119 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T08:39:32.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.239+0000 7fd990bec640 1 Processor -- start 2024-09-19T08:39:32.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.239+0000 7fd990bec640 1 -- start start 2024-09-19T08:39:32.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.239+0000 7fd990bec640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd98c15e490 0x7fd98c0d46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:32.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.239+0000 7fd990bec640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd98c0d35e0 0x7fd98c0cb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:32.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.239+0000 7fd990bec640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd98c0b75d0 con 0x7fd98c0d4bf0 2024-09-19T08:39:32.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.239+0000 7fd990bec640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fd98c0b8390 con 0x7fd98c15e490 2024-09-19T08:39:32.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.239+0000 7fd990bec640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd98c0d5060 con 0x7fd98c0d35e0 2024-09-19T08:39:32.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.239+0000 7fd98a59c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd98c15e490 0x7fd98c0d46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:32.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.239+0000 7fd98a59c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd98c15e490 0x7fd98c0d46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48460/0 (socket says 172.21.15.98:48460) 2024-09-19T08:39:32.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.239+0000 7fd98a59c640 1 -- 172.21.15.98:0/3389795915 learned_addr learned my addr 172.21.15.98:0/3389795915 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:32.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.239+0000 7fd98ad9d640 1 --2- 172.21.15.98:0/3389795915 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd98c0d35e0 0x7fd98c0cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:32.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.239+0000 7fd98a59c640 1 -- 172.21.15.98:0/3389795915 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd98c0d35e0 msgr2=0x7fd98c0cb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:32.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.239+0000 7fd98a59c640 1 --2- 172.21.15.98:0/3389795915 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd98c0d35e0 0x7fd98c0cb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:32.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.239+0000 7fd98a59c640 1 -- 172.21.15.98:0/3389795915 >> v1:172.21.15.98:6789/0 conn(0x7fd98c0d4bf0 legacy=0x7fd98c0d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:32.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.239+0000 7fd98959a640 1 -- 172.21.15.98:0/3389795915 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3319169204 0 0) 0x7fd98c0b75d0 con 0x7fd98c0d4bf0 2024-09-19T08:39:32.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.239+0000 7fd98a59c640 1 -- 172.21.15.98:0/3389795915 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd98c0b78b0 con 0x7fd98c15e490 2024-09-19T08:39:32.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.239+0000 7fd98ad9d640 1 --2- 172.21.15.98:0/3389795915 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd98c0d35e0 0x7fd98c0cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:39:32.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.243+0000 7fd98a59c640 1 --2- 172.21.15.98:0/3389795915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd98c15e490 0x7fd98c0d46b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd97c009870 tx=0x7fd97c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9280974426cc86a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:32.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.243+0000 7fd98959a640 1 -- 172.21.15.98:0/3389795915 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c0b0070 con 0x7fd98c15e490 2024-09-19T08:39:32.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.243+0000 7fd98959a640 1 -- 172.21.15.98:0/3389795915 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd97c0ab600 con 0x7fd98c15e490 2024-09-19T08:39:32.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.243+0000 7fd98959a640 1 -- 172.21.15.98:0/3389795915 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c0ab9d0 con 0x7fd98c15e490 2024-09-19T08:39:32.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.243+0000 7fd990bec640 1 -- 172.21.15.98:0/3389795915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd98c15e490 msgr2=0x7fd98c0d46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:32.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.243+0000 7fd990bec640 1 --2- 172.21.15.98:0/3389795915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd98c15e490 0x7fd98c0d46b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd97c009870 tx=0x7fd97c01c4d0 comp rx=0 tx=0).stop 2024-09-19T08:39:32.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.243+0000 7fd990bec640 1 -- 172.21.15.98:0/3389795915 shutdown_connections 2024-09-19T08:39:32.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.243+0000 7fd990bec640 1 --2- 172.21.15.98:0/3389795915 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd98c0d35e0 0x7fd98c0cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:32.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.243+0000 7fd990bec640 1 --2- 172.21.15.98:0/3389795915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd98c15e490 0x7fd98c0d46b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:32.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.243+0000 7fd990bec640 1 -- 172.21.15.98:0/3389795915 >> 172.21.15.98:0/3389795915 conn(0x7fd98c074c70 msgr2=0x7fd98c075080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:32.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.243+0000 7fd990bec640 1 -- 172.21.15.98:0/3389795915 shutdown_connections 2024-09-19T08:39:32.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.243+0000 7fd990bec640 1 -- 172.21.15.98:0/3389795915 wait complete. 2024-09-19T08:39:32.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.247+0000 7fd990bec640 1 Processor -- start 2024-09-19T08:39:32.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.247+0000 7fd990bec640 1 -- start start 2024-09-19T08:39:32.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.247+0000 7fd990bec640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd98c0d35e0 0x7fd98c158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:32.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.247+0000 7fd990bec640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd98c0d4bf0 0x7fd98c15aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:32.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.247+0000 7fd990bec640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd98c15e490 0x7fd98c1536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:32.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.247+0000 7fd990bec640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd98c0cc9f0 con 0x7fd98c15e490 2024-09-19T08:39:32.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.247+0000 7fd990bec640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd98c0cc870 con 0x7fd98c0d35e0 2024-09-19T08:39:32.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.247+0000 7fd990bec640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd98c0ccb70 con 0x7fd98c0d4bf0 2024-09-19T08:39:32.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.247+0000 7fd98ad9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd98c15e490 0x7fd98c1536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:32.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.247+0000 7fd98ad9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd98c15e490 0x7fd98c1536d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48466/0 (socket says 172.21.15.98:48466) 2024-09-19T08:39:32.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.247+0000 7fd98ad9d640 1 -- 172.21.15.98:0/2662332583 learned_addr learned my addr 172.21.15.98:0/2662332583 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:32.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.247+0000 7fd989d9b640 1 --2- 172.21.15.98:0/2662332583 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd98c0d4bf0 0x7fd98c15aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:32.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.247+0000 7fd98a59c640 1 --2- 172.21.15.98:0/2662332583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd98c0d35e0 0x7fd98c158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:32.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.247+0000 7fd98ad9d640 1 -- 172.21.15.98:0/2662332583 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd98c0d4bf0 msgr2=0x7fd98c15aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:32.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.247+0000 7fd98ad9d640 1 --2- 172.21.15.98:0/2662332583 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd98c0d4bf0 0x7fd98c15aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:32.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.247+0000 7fd98ad9d640 1 -- 172.21.15.98:0/2662332583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd98c0d35e0 msgr2=0x7fd98c158650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:32.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.247+0000 7fd98ad9d640 1 --2- 172.21.15.98:0/2662332583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd98c0d35e0 0x7fd98c158650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:32.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.247+0000 7fd98ad9d640 1 -- 172.21.15.98:0/2662332583 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd98c1598e0 con 0x7fd98c15e490 2024-09-19T08:39:32.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.247+0000 7fd98a59c640 1 --2- 172.21.15.98:0/2662332583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd98c0d35e0 0x7fd98c158650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:32.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.247+0000 7fd989d9b640 1 --2- 172.21.15.98:0/2662332583 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd98c0d4bf0 0x7fd98c15aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:32.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.247+0000 7fd98ad9d640 1 --2- 172.21.15.98:0/2662332583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd98c15e490 0x7fd98c1536d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd984007e70 tx=0x7fd98400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:32.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.247+0000 7fd97affd640 1 -- 172.21.15.98:0/2662332583 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98401a070 con 0x7fd98c15e490 2024-09-19T08:39:32.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.247+0000 7fd97affd640 1 -- 172.21.15.98:0/2662332583 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd984003bc0 con 0x7fd98c15e490 2024-09-19T08:39:32.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.247+0000 7fd990bec640 1 -- 172.21.15.98:0/2662332583 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd98c159bc0 con 0x7fd98c15e490 2024-09-19T08:39:32.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.247+0000 7fd97affd640 1 -- 172.21.15.98:0/2662332583 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd984004030 con 0x7fd98c15e490 2024-09-19T08:39:32.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.247+0000 7fd990bec640 1 -- 172.21.15.98:0/2662332583 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd98c15a180 con 0x7fd98c15e490 2024-09-19T08:39:32.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.247+0000 7fd990bec640 1 -- 172.21.15.98:0/2662332583 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd98c0cb910 con 0x7fd98c15e490 2024-09-19T08:39:32.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.247+0000 7fd97affd640 1 -- 172.21.15.98:0/2662332583 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fd98401d070 con 0x7fd98c15e490 2024-09-19T08:39:32.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.251+0000 7fd97affd640 1 --2- 172.21.15.98:0/2662332583 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd96c067a70 0x7fd96c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:32.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.251+0000 7fd98a59c640 1 --2- 172.21.15.98:0/2662332583 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd96c067a70 0x7fd96c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:32.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.251+0000 7fd97affd640 1 -- 172.21.15.98:0/2662332583 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7fd984086010 con 0x7fd98c15e490 2024-09-19T08:39:32.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.251+0000 7fd98a59c640 1 --2- 172.21.15.98:0/2662332583 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd96c067a70 0x7fd96c069f30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0ad000 tx=0x7fd97c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:32.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.255+0000 7fd97affd640 1 -- 172.21.15.98:0/2662332583 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd984010070 con 0x7fd98c15e490 2024-09-19T08:39:32.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.415+0000 7fd990bec640 1 -- 172.21.15.98:0/2662332583 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd98c0cd7d0 con 0x7fd98c15e490 2024-09-19T08:39:32.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.415+0000 7fd97affd640 1 -- 172.21.15.98:0/2662332583 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7fd984056b50 con 0x7fd98c15e490 2024-09-19T08:39:32.416 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2024-09-19T08:39:32.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.419+0000 7fd990bec640 1 -- 172.21.15.98:0/2662332583 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd96c067a70 msgr2=0x7fd96c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:32.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.419+0000 7fd990bec640 1 --2- 172.21.15.98:0/2662332583 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd96c067a70 0x7fd96c069f30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0ad000 tx=0x7fd97c008040 comp rx=0 tx=0).stop 2024-09-19T08:39:32.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.419+0000 7fd990bec640 1 -- 172.21.15.98:0/2662332583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd98c15e490 msgr2=0x7fd98c1536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:32.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.419+0000 7fd990bec640 1 --2- 172.21.15.98:0/2662332583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd98c15e490 0x7fd98c1536d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd984007e70 tx=0x7fd98400d4a0 comp rx=0 tx=0).stop 2024-09-19T08:39:32.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.419+0000 7fd990bec640 1 -- 172.21.15.98:0/2662332583 shutdown_connections 2024-09-19T08:39:32.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.419+0000 7fd990bec640 1 --2- 172.21.15.98:0/2662332583 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd96c067a70 0x7fd96c069f30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:32.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.419+0000 7fd990bec640 1 --2- 172.21.15.98:0/2662332583 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd98c15e490 0x7fd98c1536d0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:32.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.419+0000 7fd990bec640 1 --2- 172.21.15.98:0/2662332583 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd98c0d4bf0 0x7fd98c15aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:32.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.419+0000 7fd990bec640 1 --2- 172.21.15.98:0/2662332583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd98c0d35e0 0x7fd98c158650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:32.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.419+0000 7fd990bec640 1 -- 172.21.15.98:0/2662332583 >> 172.21.15.98:0/2662332583 conn(0x7fd98c074c70 msgr2=0x7fd98c0cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:32.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.419+0000 7fd990bec640 1 -- 172.21.15.98:0/2662332583 shutdown_connections 2024-09-19T08:39:32.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.419+0000 7fd990bec640 1 -- 172.21.15.98:0/2662332583 wait complete. 2024-09-19T08:39:32.435 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2024-09-19T08:39:32.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.555+0000 7f819d891640 1 Processor -- start 2024-09-19T08:39:32.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.555+0000 7f819d891640 1 -- start start 2024-09-19T08:39:32.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.555+0000 7f819d891640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f819815c550 0x7f819806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:32.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.555+0000 7f819d891640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f819806c1d0 0x7f81980cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:32.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.555+0000 7f819d891640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f81980cf900 0x7f81980ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:32.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.555+0000 7f819d891640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f819806c610 con 0x7f819806c1d0 2024-09-19T08:39:32.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.555+0000 7f819d891640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f819806d310 con 0x7f81980cf900 2024-09-19T08:39:32.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.555+0000 7f819d891640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f819806d5f0 con 0x7f819815c550 2024-09-19T08:39:32.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.555+0000 7f81967fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f819806c1d0 0x7f81980cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:32.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.555+0000 7f81967fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f819806c1d0 0x7f81980cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48472/0 (socket says 172.21.15.98:48472) 2024-09-19T08:39:32.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.555+0000 7f81967fc640 1 -- 172.21.15.98:0/3740674453 learned_addr learned my addr 172.21.15.98:0/3740674453 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:32.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.555+0000 7f81977fe640 1 --2- 172.21.15.98:0/3740674453 >> v2:172.21.15.139:3300/0 conn(0x7f81980cf900 0x7f81980ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:32.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.555+0000 7f8196ffd640 1 --2- 172.21.15.98:0/3740674453 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f819815c550 0x7f819806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:32.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.555+0000 7f81967fc640 1 -- 172.21.15.98:0/3740674453 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f819815c550 msgr2=0x7f819806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:32.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.555+0000 7f81967fc640 1 --2- 172.21.15.98:0/3740674453 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f819815c550 0x7f819806bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:32.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.555+0000 7f81967fc640 1 -- 172.21.15.98:0/3740674453 >> v2:172.21.15.139:3300/0 conn(0x7f81980cf900 msgr2=0x7f81980ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:32.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.555+0000 7f81967fc640 1 --2- 172.21.15.98:0/3740674453 >> v2:172.21.15.139:3300/0 conn(0x7f81980cf900 0x7f81980ce1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:32.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.555+0000 7f81967fc640 1 -- 172.21.15.98:0/3740674453 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f819806d8d0 con 0x7f819806c1d0 2024-09-19T08:39:32.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.555+0000 7f8196ffd640 1 --2- 172.21.15.98:0/3740674453 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f819815c550 0x7f819806bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:32.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.559+0000 7f81967fc640 1 --2- 172.21.15.98:0/3740674453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f819806c1d0 0x7f81980cf3c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8198074090 tx=0x7f818001c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d0a8b96aa7cb938 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:32.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.559+0000 7f8195ffb640 1 -- 172.21.15.98:0/3740674453 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81800b0070 con 0x7f819806c1d0 2024-09-19T08:39:32.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.559+0000 7f8195ffb640 1 -- 172.21.15.98:0/3740674453 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81800ab600 con 0x7f819806c1d0 2024-09-19T08:39:32.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.559+0000 7f8195ffb640 1 -- 172.21.15.98:0/3740674453 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81800ab9d0 con 0x7f819806c1d0 2024-09-19T08:39:32.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.559+0000 7f819d891640 1 -- 172.21.15.98:0/3740674453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f819806c1d0 msgr2=0x7f81980cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:32.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.559+0000 7f819d891640 1 --2- 172.21.15.98:0/3740674453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f819806c1d0 0x7f81980cf3c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8198074090 tx=0x7f818001c260 comp rx=0 tx=0).stop 2024-09-19T08:39:32.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.559+0000 7f819d891640 1 -- 172.21.15.98:0/3740674453 shutdown_connections 2024-09-19T08:39:32.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.559+0000 7f819d891640 1 --2- 172.21.15.98:0/3740674453 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f819806c1d0 0x7f81980cf3c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:32.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.559+0000 7f819d891640 1 --2- 172.21.15.98:0/3740674453 >> v2:172.21.15.139:3300/0 conn(0x7f81980cf900 0x7f81980ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:32.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.559+0000 7f819d891640 1 --2- 172.21.15.98:0/3740674453 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f819815c550 0x7f819806bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:32.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.559+0000 7f819d891640 1 -- 172.21.15.98:0/3740674453 >> 172.21.15.98:0/3740674453 conn(0x7f81980b2010 msgr2=0x7f81980b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:32.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.559+0000 7f819d891640 1 -- 172.21.15.98:0/3740674453 shutdown_connections 2024-09-19T08:39:32.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.559+0000 7f819d891640 1 -- 172.21.15.98:0/3740674453 wait complete. 2024-09-19T08:39:32.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.559+0000 7f819d891640 1 Processor -- start 2024-09-19T08:39:32.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.559+0000 7f819d891640 1 -- start start 2024-09-19T08:39:32.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.559+0000 7f819d891640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81980cf900 0x7f819814dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:32.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.559+0000 7f819d891640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f819815c550 0x7f819815a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:32.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.559+0000 7f819d891640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8198146ef0 0x7f8198147380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:32.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.559+0000 7f819d891640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f81980723a0 con 0x7f81980cf900 2024-09-19T08:39:32.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.559+0000 7f819d891640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f819815cb40 con 0x7f8198146ef0 2024-09-19T08:39:32.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.559+0000 7f819d891640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8198072520 con 0x7f819815c550 2024-09-19T08:39:32.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.559+0000 7f8196ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81980cf900 0x7f819814dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:32.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.559+0000 7f8196ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81980cf900 0x7f819814dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48480/0 (socket says 172.21.15.98:48480) 2024-09-19T08:39:32.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.559+0000 7f8196ffd640 1 -- 172.21.15.98:0/2664924342 learned_addr learned my addr 172.21.15.98:0/2664924342 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:32.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.559+0000 7f81967fc640 1 --2- 172.21.15.98:0/2664924342 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f819815c550 0x7f819815a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:32.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.559+0000 7f81977fe640 1 --2- 172.21.15.98:0/2664924342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8198146ef0 0x7f8198147380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:32.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.559+0000 7f8196ffd640 1 -- 172.21.15.98:0/2664924342 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f819815c550 msgr2=0x7f819815a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:32.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.559+0000 7f8196ffd640 1 --2- 172.21.15.98:0/2664924342 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f819815c550 0x7f819815a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:32.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.559+0000 7f8196ffd640 1 -- 172.21.15.98:0/2664924342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8198146ef0 msgr2=0x7f8198147380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:32.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.559+0000 7f8196ffd640 1 --2- 172.21.15.98:0/2664924342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8198146ef0 0x7f8198147380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:32.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.559+0000 7f8196ffd640 1 -- 172.21.15.98:0/2664924342 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81981504d0 con 0x7f81980cf900 2024-09-19T08:39:32.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.559+0000 7f81977fe640 1 --2- 172.21.15.98:0/2664924342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8198146ef0 0x7f8198147380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:32.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.563+0000 7f8196ffd640 1 --2- 172.21.15.98:0/2664924342 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81980cf900 0x7f819814dc20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f818800ce00 tx=0x7f818800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:32.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.563+0000 7f818ffff640 1 -- 172.21.15.98:0/2664924342 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818800f680 con 0x7f81980cf900 2024-09-19T08:39:32.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.563+0000 7f818ffff640 1 -- 172.21.15.98:0/2664924342 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f818800f820 con 0x7f81980cf900 2024-09-19T08:39:32.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.563+0000 7f818ffff640 1 -- 172.21.15.98:0/2664924342 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818800fc30 con 0x7f81980cf900 2024-09-19T08:39:32.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.563+0000 7f819d891640 1 -- 172.21.15.98:0/2664924342 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81981507b0 con 0x7f81980cf900 2024-09-19T08:39:32.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.563+0000 7f819d891640 1 -- 172.21.15.98:0/2664924342 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f819815aa80 con 0x7f81980cf900 2024-09-19T08:39:32.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.563+0000 7f819d891640 1 -- 172.21.15.98:0/2664924342 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81980ce1a0 con 0x7f81980cf900 2024-09-19T08:39:32.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.563+0000 7f818ffff640 1 -- 172.21.15.98:0/2664924342 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f8188018400 con 0x7f81980cf900 2024-09-19T08:39:32.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.563+0000 7f818ffff640 1 --2- 172.21.15.98:0/2664924342 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f817c067aa0 0x7f817c069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:32.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.563+0000 7f818ffff640 1 -- 172.21.15.98:0/2664924342 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f81880869f0 con 0x7f81980cf900 2024-09-19T08:39:32.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.563+0000 7f81967fc640 1 --2- 172.21.15.98:0/2664924342 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f817c067aa0 0x7f817c069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:32.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.563+0000 7f81967fc640 1 --2- 172.21.15.98:0/2664924342 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f817c067aa0 0x7f817c069f60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f81800ad000 tx=0x7f81800a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:32.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.567+0000 7f818ffff640 1 -- 172.21.15.98:0/2664924342 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f8188014030 con 0x7f81980cf900 2024-09-19T08:39:32.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:32.791+0000 7f819d891640 1 -- 172.21.15.98:0/2664924342 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) -- 0x7f8198153f30 con 0x7f81980cf900 2024-09-19T08:39:33.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.783+0000 7f818ffff640 1 -- 172.21.15.98:0/2664924342 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v89) ==== 143+0+0 (secure 0 0 0) 0x7f8188056f90 con 0x7f81980cf900 2024-09-19T08:39:33.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.783+0000 7f819d891640 1 -- 172.21.15.98:0/2664924342 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f817c067aa0 msgr2=0x7f817c069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:33.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.783+0000 7f819d891640 1 --2- 172.21.15.98:0/2664924342 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f817c067aa0 0x7f817c069f60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f81800ad000 tx=0x7f81800a6000 comp rx=0 tx=0).stop 2024-09-19T08:39:33.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.783+0000 7f819d891640 1 -- 172.21.15.98:0/2664924342 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81980cf900 msgr2=0x7f819814dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:33.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.783+0000 7f819d891640 1 --2- 172.21.15.98:0/2664924342 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81980cf900 0x7f819814dc20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f818800ce00 tx=0x7f818800a650 comp rx=0 tx=0).stop 2024-09-19T08:39:33.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.783+0000 7f819d891640 1 -- 172.21.15.98:0/2664924342 shutdown_connections 2024-09-19T08:39:33.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.783+0000 7f819d891640 1 --2- 172.21.15.98:0/2664924342 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f817c067aa0 0x7f817c069f60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:33.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.783+0000 7f819d891640 1 --2- 172.21.15.98:0/2664924342 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8198146ef0 0x7f8198147380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:33.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.783+0000 7f819d891640 1 --2- 172.21.15.98:0/2664924342 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f819815c550 0x7f819815a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:33.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.783+0000 7f819d891640 1 --2- 172.21.15.98:0/2664924342 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81980cf900 0x7f819814dc20 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:33.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.783+0000 7f819d891640 1 -- 172.21.15.98:0/2664924342 >> 172.21.15.98:0/2664924342 conn(0x7f81980b2010 msgr2=0x7f81980aff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:33.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.783+0000 7f819d891640 1 -- 172.21.15.98:0/2664924342 shutdown_connections 2024-09-19T08:39:33.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.787+0000 7f819d891640 1 -- 172.21.15.98:0/2664924342 wait complete. 2024-09-19T08:39:33.806 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T08:39:33.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.935+0000 7f3b51343640 1 Processor -- start 2024-09-19T08:39:33.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.935+0000 7f3b51343640 1 -- start start 2024-09-19T08:39:33.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.935+0000 7f3b51343640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3b4c145b70 0x7f3b4c06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:33.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.935+0000 7f3b51343640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b4c06c1d0 0x7f3b4c0713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:33.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.935+0000 7f3b51343640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b4c06c5f0 con 0x7f3b4c0718f0 2024-09-19T08:39:33.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.935+0000 7f3b51343640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f3b4c073530 con 0x7f3b4c145b70 2024-09-19T08:39:33.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.935+0000 7f3b51343640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3b4c073810 con 0x7f3b4c06c1d0 2024-09-19T08:39:33.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.935+0000 7f3b4b7fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f3b4c0718f0 0x7f3b4c0cef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36202/0 (socket says 172.21.15.98:36202) 2024-09-19T08:39:33.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.935+0000 7f3b4b7fe640 1 -- 172.21.15.98:0/805889816 learned_addr learned my addr 172.21.15.98:0/805889816 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:33.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.935+0000 7f3b4affd640 1 --2- 172.21.15.98:0/805889816 >> v2:172.21.15.139:3300/0 conn(0x7f3b4c145b70 0x7f3b4c06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:33.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.935+0000 7f3b4a7fc640 1 --2- 172.21.15.98:0/805889816 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b4c06c1d0 0x7f3b4c0713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:33.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.935+0000 7f3b49ffb640 1 -- 172.21.15.98:0/805889816 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3780250642 0 0) 0x7f3b4c06c5f0 con 0x7f3b4c0718f0 2024-09-19T08:39:33.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.935+0000 7f3b49ffb640 1 -- 172.21.15.98:0/805889816 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b38003530 con 0x7f3b4c0718f0 2024-09-19T08:39:33.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.935+0000 7f3b49ffb640 1 -- 172.21.15.98:0/805889816 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 25532036 0 0) 0x7f3b38003530 con 0x7f3b4c0718f0 2024-09-19T08:39:33.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.935+0000 7f3b49ffb640 1 -- 172.21.15.98:0/805889816 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b4c06c1d0 msgr2=0x7f3b4c0713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:33.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.935+0000 7f3b49ffb640 1 --2- 172.21.15.98:0/805889816 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b4c06c1d0 0x7f3b4c0713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:33.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.935+0000 7f3b49ffb640 1 -- 172.21.15.98:0/805889816 >> v2:172.21.15.139:3300/0 conn(0x7f3b4c145b70 msgr2=0x7f3b4c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:33.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.935+0000 7f3b49ffb640 1 --2- 172.21.15.98:0/805889816 >> v2:172.21.15.139:3300/0 conn(0x7f3b4c145b70 0x7f3b4c06bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:33.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.935+0000 7f3b49ffb640 1 -- 172.21.15.98:0/805889816 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4c073af0 con 0x7f3b4c0718f0 2024-09-19T08:39:33.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.935+0000 7f3b49ffb640 1 -- 172.21.15.98:0/805889816 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f3b3c001a00 con 0x7f3b4c0718f0 2024-09-19T08:39:33.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.935+0000 7f3b4affd640 1 --2- 172.21.15.98:0/805889816 >> v2:172.21.15.139:3300/0 conn(0x7f3b4c145b70 0x7f3b4c06bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:39:33.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.935+0000 7f3b49ffb640 1 -- 172.21.15.98:0/805889816 >> v1:172.21.15.98:6789/0 conn(0x7f3b4c0718f0 legacy=0x7f3b4c0cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:33.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.935+0000 7f3b49ffb640 1 --2- 172.21.15.98:0/805889816 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b4c06c1d0 0x7f3b380039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:33.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.935+0000 7f3b49ffb640 1 --2- 172.21.15.98:0/805889816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b38003f40 0x7f3b38004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:33.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.935+0000 7f3b49ffb640 1 --2- 172.21.15.98:0/805889816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b38004d50 0x7f3b380051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:33.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.935+0000 7f3b49ffb640 1 -- 172.21.15.98:0/805889816 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c073530 con 0x7f3b38004d50 2024-09-19T08:39:33.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.935+0000 7f3b49ffb640 1 -- 172.21.15.98:0/805889816 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c073810 con 0x7f3b38003f40 2024-09-19T08:39:33.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.935+0000 7f3b49ffb640 1 -- 172.21.15.98:0/805889816 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3b3800ba20 con 0x7f3b4c06c1d0 2024-09-19T08:39:33.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.935+0000 7f3b4b7fe640 1 --2- 172.21.15.98:0/805889816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b38004d50 0x7f3b380051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:33.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.935+0000 7f3b4affd640 1 --2- 172.21.15.98:0/805889816 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b4c06c1d0 0x7f3b380039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:33.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.935+0000 7f3b4a7fc640 1 --2- 172.21.15.98:0/805889816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b38003f40 0x7f3b38004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:33.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.935+0000 7f3b4b7fe640 1 -- 172.21.15.98:0/805889816 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b4c06c1d0 msgr2=0x7f3b380039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:33.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.935+0000 7f3b4b7fe640 1 --2- 172.21.15.98:0/805889816 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b4c06c1d0 0x7f3b380039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:33.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.935+0000 7f3b4b7fe640 1 -- 172.21.15.98:0/805889816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b38003f40 msgr2=0x7f3b38004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:33.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.935+0000 7f3b4b7fe640 1 --2- 172.21.15.98:0/805889816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b38003f40 0x7f3b38004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:33.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.935+0000 7f3b4b7fe640 1 -- 172.21.15.98:0/805889816 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b38003530 con 0x7f3b38004d50 2024-09-19T08:39:33.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.939+0000 7f3b4b7fe640 1 --2- 172.21.15.98:0/805889816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b38004d50 0x7f3b380051d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c00a8a0 tx=0x7f3b3c01fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b052afe4c7ebc940 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:33.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.939+0000 7f3b49ffb640 1 -- 172.21.15.98:0/805889816 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c00a4c0 con 0x7f3b38004d50 2024-09-19T08:39:33.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.939+0000 7f3b49ffb640 1 -- 172.21.15.98:0/805889816 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b3c0aa710 con 0x7f3b38004d50 2024-09-19T08:39:33.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.939+0000 7f3b49ffb640 1 -- 172.21.15.98:0/805889816 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c0aab00 con 0x7f3b38004d50 2024-09-19T08:39:33.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.939+0000 7f3b51343640 1 -- 172.21.15.98:0/805889816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b38004d50 msgr2=0x7f3b380051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:33.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.939+0000 7f3b51343640 1 --2- 172.21.15.98:0/805889816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b38004d50 0x7f3b380051d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c00a8a0 tx=0x7f3b3c01fae0 comp rx=0 tx=0).stop 2024-09-19T08:39:33.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.939+0000 7f3b4b7fe640 1 -- 172.21.15.98:0/805889816 reap_dead start 2024-09-19T08:39:33.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.939+0000 7f3b51343640 1 -- 172.21.15.98:0/805889816 shutdown_connections 2024-09-19T08:39:33.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.939+0000 7f3b51343640 1 -- 172.21.15.98:0/805889816 >> 172.21.15.98:0/805889816 conn(0x7f3b4c0b2010 msgr2=0x7f3b4c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:33.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.939+0000 7f3b51343640 1 -- 172.21.15.98:0/805889816 shutdown_connections 2024-09-19T08:39:33.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.939+0000 7f3b51343640 1 -- 172.21.15.98:0/805889816 wait complete. 2024-09-19T08:39:33.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.939+0000 7f3b51343640 1 Processor -- start 2024-09-19T08:39:33.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.939+0000 7f3b51343640 1 -- start start 2024-09-19T08:39:33.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.943+0000 7f3b51343640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b4c15b6a0 0x7f3b4c13be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:33.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.943+0000 7f3b51343640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b4c13c350 0x7f3b4c13f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:33.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.943+0000 7f3b51343640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b4c15ba80 0x7f3b4c13fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:33.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.943+0000 7f3b51343640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c1460d0 con 0x7f3b4c15b6a0 2024-09-19T08:39:33.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.943+0000 7f3b51343640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c145f50 con 0x7f3b4c13c350 2024-09-19T08:39:33.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.943+0000 7f3b51343640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3b4c071d30 con 0x7f3b4c15ba80 2024-09-19T08:39:33.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.943+0000 7f3b4affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b4c15b6a0 0x7f3b4c13be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:33.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.943+0000 7f3b4affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b4c15b6a0 0x7f3b4c13be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48512/0 (socket says 172.21.15.98:48512) 2024-09-19T08:39:33.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.943+0000 7f3b4affd640 1 -- 172.21.15.98:0/2998218169 learned_addr learned my addr 172.21.15.98:0/2998218169 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:33.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.943+0000 7f3b4a7fc640 1 --2- 172.21.15.98:0/2998218169 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b4c13c350 0x7f3b4c13f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:33.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.943+0000 7f3b4b7fe640 1 --2- 172.21.15.98:0/2998218169 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b4c15ba80 0x7f3b4c13fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:33.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.943+0000 7f3b4affd640 1 -- 172.21.15.98:0/2998218169 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b4c15ba80 msgr2=0x7f3b4c13fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:33.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.943+0000 7f3b4affd640 1 --2- 172.21.15.98:0/2998218169 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b4c15ba80 0x7f3b4c13fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:33.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.943+0000 7f3b4affd640 1 -- 172.21.15.98:0/2998218169 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b4c13c350 msgr2=0x7f3b4c13f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:33.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.943+0000 7f3b4affd640 1 --2- 172.21.15.98:0/2998218169 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b4c13c350 0x7f3b4c13f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:33.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.943+0000 7f3b4affd640 1 -- 172.21.15.98:0/2998218169 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4c13c7f0 con 0x7f3b4c15b6a0 2024-09-19T08:39:33.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.943+0000 7f3b4b7fe640 1 --2- 172.21.15.98:0/2998218169 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b4c15ba80 0x7f3b4c13fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:33.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.943+0000 7f3b4affd640 1 --2- 172.21.15.98:0/2998218169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b4c15b6a0 0x7f3b4c13be10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3b44011900 tx=0x7f3b44011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:33.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.943+0000 7f3b437fe640 1 -- 172.21.15.98:0/2998218169 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4400fb70 con 0x7f3b4c15b6a0 2024-09-19T08:39:33.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.943+0000 7f3b437fe640 1 -- 172.21.15.98:0/2998218169 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b4400fd10 con 0x7f3b4c15b6a0 2024-09-19T08:39:33.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.943+0000 7f3b51343640 1 -- 172.21.15.98:0/2998218169 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b4c13ad30 con 0x7f3b4c15b6a0 2024-09-19T08:39:33.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.943+0000 7f3b437fe640 1 -- 172.21.15.98:0/2998218169 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b44015660 con 0x7f3b4c15b6a0 2024-09-19T08:39:33.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.943+0000 7f3b51343640 1 -- 172.21.15.98:0/2998218169 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b4c13b2f0 con 0x7f3b4c15b6a0 2024-09-19T08:39:33.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.943+0000 7f3b51343640 1 -- 172.21.15.98:0/2998218169 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b4c073320 con 0x7f3b4c15b6a0 2024-09-19T08:39:33.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.943+0000 7f3b437fe640 1 -- 172.21.15.98:0/2998218169 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f3b44015800 con 0x7f3b4c15b6a0 2024-09-19T08:39:33.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.943+0000 7f3b437fe640 1 --2- 172.21.15.98:0/2998218169 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3b2c067a70 0x7f3b2c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:33.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.943+0000 7f3b437fe640 1 -- 172.21.15.98:0/2998218169 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f3b44086820 con 0x7f3b4c15b6a0 2024-09-19T08:39:33.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.943+0000 7f3b4a7fc640 1 --2- 172.21.15.98:0/2998218169 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3b2c067a70 0x7f3b2c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:33.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.943+0000 7f3b4a7fc640 1 --2- 172.21.15.98:0/2998218169 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3b2c067a70 0x7f3b2c069f30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c13a430 tx=0x7f3b34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:33.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:33.947+0000 7f3b437fe640 1 -- 172.21.15.98:0/2998218169 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3b44019030 con 0x7f3b4c15b6a0 2024-09-19T08:39:34.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.111+0000 7f3b51343640 1 -- 172.21.15.98:0/2998218169 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f3b4c0b4a60 con 0x7f3b4c15b6a0 2024-09-19T08:39:34.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.115+0000 7f3b437fe640 1 -- 172.21.15.98:0/2998218169 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f3b440573e0 con 0x7f3b4c15b6a0 2024-09-19T08:39:34.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.115+0000 7f3b51343640 1 -- 172.21.15.98:0/2998218169 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3b2c067a70 msgr2=0x7f3b2c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:34.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.119+0000 7f3b51343640 1 --2- 172.21.15.98:0/2998218169 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3b2c067a70 0x7f3b2c069f30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c13a430 tx=0x7f3b34008040 comp rx=0 tx=0).stop 2024-09-19T08:39:34.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.119+0000 7f3b51343640 1 -- 172.21.15.98:0/2998218169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b4c15b6a0 msgr2=0x7f3b4c13be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:34.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.119+0000 7f3b51343640 1 --2- 172.21.15.98:0/2998218169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b4c15b6a0 0x7f3b4c13be10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3b44011900 tx=0x7f3b44011dc0 comp rx=0 tx=0).stop 2024-09-19T08:39:34.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.119+0000 7f3b51343640 1 -- 172.21.15.98:0/2998218169 shutdown_connections 2024-09-19T08:39:34.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.119+0000 7f3b51343640 1 --2- 172.21.15.98:0/2998218169 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3b2c067a70 0x7f3b2c069f30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:34.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.119+0000 7f3b51343640 1 --2- 172.21.15.98:0/2998218169 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3b4c15ba80 0x7f3b4c13fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:34.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.119+0000 7f3b51343640 1 --2- 172.21.15.98:0/2998218169 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3b4c13c350 0x7f3b4c13f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:34.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.119+0000 7f3b51343640 1 --2- 172.21.15.98:0/2998218169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3b4c15b6a0 0x7f3b4c13be10 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:34.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.119+0000 7f3b51343640 1 -- 172.21.15.98:0/2998218169 >> 172.21.15.98:0/2998218169 conn(0x7f3b4c0b2010 msgr2=0x7f3b4c072130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:34.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.119+0000 7f3b51343640 1 -- 172.21.15.98:0/2998218169 shutdown_connections 2024-09-19T08:39:34.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.119+0000 7f3b51343640 1 -- 172.21.15.98:0/2998218169 wait complete. 2024-09-19T08:39:34.134 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T08:39:34.134 DEBUG:teuthology.orchestra.run.smithi098:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:39:34.206 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:39:34.274 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T08:39:34.274 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:39:34.274 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:39:34.280 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15846 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T08:39:34.280 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:39:34.280 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:39:34.286 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T08:39:34.287 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:39:34.287 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:39:34.326 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15846 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T08:39:34.326 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:39:34.326 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:39:34.333 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T08:39:34.333 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-19T08:39:34.333 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:39:34.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.499+0000 7f429574d640 1 Processor -- start 2024-09-19T08:39:34.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.499+0000 7f429574d640 1 -- start start 2024-09-19T08:39:34.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.503+0000 7f429574d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f429015c550 0x7f429006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:34.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.503+0000 7f429574d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f429006c1d0 0x7f42900cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:34.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.503+0000 7f429574d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f42900cf900 0x7f42900ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:34.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.503+0000 7f429574d640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f429006c610 con 0x7f429015c550 2024-09-19T08:39:34.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.503+0000 7f429574d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f429006d310 con 0x7f42900cf900 2024-09-19T08:39:34.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.503+0000 7f429574d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f429006d5f0 con 0x7f429006c1d0 2024-09-19T08:39:34.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.503+0000 7f428effd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f429015c550 0x7f429006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:34.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.503+0000 7f428effd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f429015c550 0x7f429006bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48528/0 (socket says 172.21.15.98:48528) 2024-09-19T08:39:34.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.503+0000 7f428effd640 1 -- 172.21.15.98:0/3213766472 learned_addr learned my addr 172.21.15.98:0/3213766472 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:34.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.503+0000 7f428e7fc640 1 --2- 172.21.15.98:0/3213766472 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f429006c1d0 0x7f42900cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:34.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.503+0000 7f428f7fe640 1 --2- 172.21.15.98:0/3213766472 >> v2:172.21.15.139:3300/0 conn(0x7f42900cf900 0x7f42900ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:34.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.503+0000 7f428effd640 1 -- 172.21.15.98:0/3213766472 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f429006c1d0 msgr2=0x7f42900cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:34.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.503+0000 7f428effd640 1 --2- 172.21.15.98:0/3213766472 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f429006c1d0 0x7f42900cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:34.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.503+0000 7f428effd640 1 -- 172.21.15.98:0/3213766472 >> v2:172.21.15.139:3300/0 conn(0x7f42900cf900 msgr2=0x7f42900ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:34.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.503+0000 7f428effd640 1 --2- 172.21.15.98:0/3213766472 >> v2:172.21.15.139:3300/0 conn(0x7f42900cf900 0x7f42900ce1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:34.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.503+0000 7f428effd640 1 -- 172.21.15.98:0/3213766472 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f429006d8d0 con 0x7f429015c550 2024-09-19T08:39:34.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.503+0000 7f428effd640 1 --2- 172.21.15.98:0/3213766472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f429015c550 0x7f429006bc90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4290074090 tx=0x7f427801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb08e7af7f3eac0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:34.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.503+0000 7f428dffb640 1 -- 172.21.15.98:0/3213766472 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42780b0070 con 0x7f429015c550 2024-09-19T08:39:34.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.503+0000 7f428dffb640 1 -- 172.21.15.98:0/3213766472 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42780ab600 con 0x7f429015c550 2024-09-19T08:39:34.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.503+0000 7f428dffb640 1 -- 172.21.15.98:0/3213766472 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42780ab9d0 con 0x7f429015c550 2024-09-19T08:39:34.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.503+0000 7f429574d640 1 -- 172.21.15.98:0/3213766472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f429015c550 msgr2=0x7f429006bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:34.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.503+0000 7f429574d640 1 --2- 172.21.15.98:0/3213766472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f429015c550 0x7f429006bc90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4290074090 tx=0x7f427801c260 comp rx=0 tx=0).stop 2024-09-19T08:39:34.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.503+0000 7f429574d640 1 -- 172.21.15.98:0/3213766472 shutdown_connections 2024-09-19T08:39:34.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.507+0000 7f429574d640 1 --2- 172.21.15.98:0/3213766472 >> v2:172.21.15.139:3300/0 conn(0x7f42900cf900 0x7f42900ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:34.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.507+0000 7f429574d640 1 --2- 172.21.15.98:0/3213766472 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f429006c1d0 0x7f42900cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:34.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.507+0000 7f429574d640 1 --2- 172.21.15.98:0/3213766472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f429015c550 0x7f429006bc90 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:34.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.507+0000 7f429574d640 1 -- 172.21.15.98:0/3213766472 >> 172.21.15.98:0/3213766472 conn(0x7f42900b2010 msgr2=0x7f42900b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:34.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.507+0000 7f429574d640 1 -- 172.21.15.98:0/3213766472 shutdown_connections 2024-09-19T08:39:34.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.507+0000 7f429574d640 1 -- 172.21.15.98:0/3213766472 wait complete. 2024-09-19T08:39:34.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.507+0000 7f429574d640 1 Processor -- start 2024-09-19T08:39:34.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.507+0000 7f429574d640 1 -- start start 2024-09-19T08:39:34.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.507+0000 7f429574d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f429006c1d0 0x7f4290149e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:34.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.507+0000 7f429574d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42900cf900 0x7f429014f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:34.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.507+0000 7f429574d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f429015c550 0x7f4290152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:34.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.507+0000 7f429574d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f42900723a0 con 0x7f429015c550 2024-09-19T08:39:34.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.507+0000 7f429574d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f429015cb40 con 0x7f42900cf900 2024-09-19T08:39:34.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.507+0000 7f429574d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4290072520 con 0x7f429006c1d0 2024-09-19T08:39:34.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.507+0000 7f428f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f429015c550 0x7f4290152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:34.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.507+0000 7f428f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f429015c550 0x7f4290152400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48530/0 (socket says 172.21.15.98:48530) 2024-09-19T08:39:34.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.507+0000 7f428f7fe640 1 -- 172.21.15.98:0/809583683 learned_addr learned my addr 172.21.15.98:0/809583683 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:34.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.507+0000 7f428effd640 1 --2- 172.21.15.98:0/809583683 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f429006c1d0 0x7f4290149e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:34.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.507+0000 7f428e7fc640 1 --2- 172.21.15.98:0/809583683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42900cf900 0x7f429014f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:34.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.507+0000 7f428f7fe640 1 -- 172.21.15.98:0/809583683 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f429006c1d0 msgr2=0x7f4290149e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:34.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.507+0000 7f428f7fe640 1 --2- 172.21.15.98:0/809583683 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f429006c1d0 0x7f4290149e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:34.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.507+0000 7f428f7fe640 1 -- 172.21.15.98:0/809583683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42900cf900 msgr2=0x7f429014f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:34.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.507+0000 7f428f7fe640 1 --2- 172.21.15.98:0/809583683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42900cf900 0x7f429014f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:34.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.507+0000 7f428f7fe640 1 -- 172.21.15.98:0/809583683 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4290150df0 con 0x7f429015c550 2024-09-19T08:39:34.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.507+0000 7f428f7fe640 1 --2- 172.21.15.98:0/809583683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f429015c550 0x7f4290152400 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f428800cd70 tx=0x7f428800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:34.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.507+0000 7f42877fe640 1 -- 172.21.15.98:0/809583683 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4288003ba0 con 0x7f429015c550 2024-09-19T08:39:34.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.507+0000 7f42877fe640 1 -- 172.21.15.98:0/809583683 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4288003d40 con 0x7f429015c550 2024-09-19T08:39:34.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.507+0000 7f429574d640 1 -- 172.21.15.98:0/809583683 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f429014a6f0 con 0x7f429015c550 2024-09-19T08:39:34.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.507+0000 7f42877fe640 1 -- 172.21.15.98:0/809583683 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4288004240 con 0x7f429015c550 2024-09-19T08:39:34.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.507+0000 7f429574d640 1 -- 172.21.15.98:0/809583683 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f429014acb0 con 0x7f429015c550 2024-09-19T08:39:34.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.511+0000 7f429574d640 1 -- 172.21.15.98:0/809583683 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42900ce1a0 con 0x7f429015c550 2024-09-19T08:39:34.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.511+0000 7f42877fe640 1 -- 172.21.15.98:0/809583683 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f42880043e0 con 0x7f429015c550 2024-09-19T08:39:34.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.511+0000 7f42877fe640 1 --2- 172.21.15.98:0/809583683 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4270067aa0 0x7f4270069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:34.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.511+0000 7f42877fe640 1 -- 172.21.15.98:0/809583683 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f4288086b60 con 0x7f429015c550 2024-09-19T08:39:34.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.511+0000 7f428effd640 1 --2- 172.21.15.98:0/809583683 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4270067aa0 0x7f4270069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:34.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.511+0000 7f428effd640 1 --2- 172.21.15.98:0/809583683 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4270067aa0 0x7f4270069f60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f42780ad000 tx=0x7f4278008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:34.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.515+0000 7f42877fe640 1 -- 172.21.15.98:0/809583683 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f428801d340 con 0x7f429015c550 2024-09-19T08:39:34.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.711+0000 7f429574d640 1 -- 172.21.15.98:0/809583683 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f42900734d0 con 0x7f429015c550 2024-09-19T08:39:34.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.711+0000 7f42877fe640 1 -- 172.21.15.98:0/809583683 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+3917 (secure 0 0 0) 0x7f4288057780 con 0x7f429015c550 2024-09-19T08:39:34.714 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:39:34.714 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":17,"btime":"2024-09-19T08:39:28:744315+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4481,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3797406671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3797406671},{"type":"v1","addr":"172.21.15.98:6835","nonce":3797406671}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4490,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/4112109751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":4112109751},{"type":"v1","addr":"172.21.15.139:6834","nonce":4112109751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4684,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1071477338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6833","nonce":1071477338},{"type":"v1","addr":"172.21.15.139:6835","nonce":1071477338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5169,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/3312936780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3312936780},{"type":"v1","addr":"172.21.15.98:6837","nonce":3312936780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[]} 2024-09-19T08:39:34.715 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 17 2024-09-19T08:39:34.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.715+0000 7f429574d640 1 -- 172.21.15.98:0/809583683 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4270067aa0 msgr2=0x7f4270069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:34.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.715+0000 7f429574d640 1 --2- 172.21.15.98:0/809583683 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4270067aa0 0x7f4270069f60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f42780ad000 tx=0x7f4278008040 comp rx=0 tx=0).stop 2024-09-19T08:39:34.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.715+0000 7f429574d640 1 -- 172.21.15.98:0/809583683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f429015c550 msgr2=0x7f4290152400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:34.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.715+0000 7f429574d640 1 --2- 172.21.15.98:0/809583683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f429015c550 0x7f4290152400 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f428800cd70 tx=0x7f428800a660 comp rx=0 tx=0).stop 2024-09-19T08:39:34.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.715+0000 7f429574d640 1 -- 172.21.15.98:0/809583683 shutdown_connections 2024-09-19T08:39:34.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.715+0000 7f429574d640 1 --2- 172.21.15.98:0/809583683 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4270067aa0 0x7f4270069f60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:34.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.715+0000 7f429574d640 1 --2- 172.21.15.98:0/809583683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f429015c550 0x7f4290152400 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:34.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.715+0000 7f429574d640 1 --2- 172.21.15.98:0/809583683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42900cf900 0x7f429014f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:34.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.715+0000 7f429574d640 1 --2- 172.21.15.98:0/809583683 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f429006c1d0 0x7f4290149e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:34.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.715+0000 7f429574d640 1 -- 172.21.15.98:0/809583683 >> 172.21.15.98:0/809583683 conn(0x7f42900b2010 msgr2=0x7f42900afbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:34.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.715+0000 7f429574d640 1 -- 172.21.15.98:0/809583683 shutdown_connections 2024-09-19T08:39:34.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.715+0000 7f429574d640 1 -- 172.21.15.98:0/809583683 wait complete. 2024-09-19T08:39:34.732 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-19T08:39:34.733 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-19T08:39:34.733 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-19T08:39:34.733 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:39:34.733 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-19T08:39:34.733 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-19T08:39:34.733 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-19T08:39:34.733 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:39:34.734 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-19T08:39:34.734 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-19T08:39:34.734 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-19T08:39:34.734 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:39:34.734 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-19T08:39:34.734 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-19T08:39:34.735 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-19T08:39:34.735 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:39:34.743 INFO:tasks.ceph.mds.a:Stopped 2024-09-19T08:39:34.743 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-19T08:39:34.744 INFO:tasks.ceph.mds.c:Stopped 2024-09-19T08:39:34.744 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-19T08:39:34.753 INFO:tasks.ceph.mds.b:Stopped 2024-09-19T08:39:34.754 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-19T08:39:34.754 INFO:tasks.ceph.mds.d:Stopped 2024-09-19T08:39:34.754 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-19T08:39:34.786 INFO:tasks.ceph.mds.a:Started 2024-09-19T08:39:34.786 DEBUG:teuthology.parallel:result is None 2024-09-19T08:39:34.786 INFO:tasks.ceph.mds.c:Started 2024-09-19T08:39:34.787 DEBUG:teuthology.parallel:result is None 2024-09-19T08:39:34.797 INFO:tasks.ceph.mds.b:Started 2024-09-19T08:39:34.798 DEBUG:teuthology.parallel:result is None 2024-09-19T08:39:34.798 INFO:tasks.ceph.mds.d:Started 2024-09-19T08:39:34.798 DEBUG:teuthology.parallel:result is None 2024-09-19T08:39:34.798 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-19T08:39:34.847 INFO:tasks.ceph.mds.a.smithi098.stdout:starting mds.a at 2024-09-19T08:39:34.848 INFO:tasks.ceph.mds.c.smithi098.stdout:starting mds.c at 2024-09-19T08:39:34.870 INFO:tasks.ceph.mds.d.smithi139.stdout:starting mds.d at 2024-09-19T08:39:34.871 INFO:tasks.ceph.mds.b.smithi139.stdout:starting mds.b at 2024-09-19T08:39:34.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.919+0000 7f92ab856640 1 Processor -- start 2024-09-19T08:39:34.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.919+0000 7f92ab856640 1 -- start start 2024-09-19T08:39:34.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.919+0000 7f92ab856640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f92a40d4820 0x7f92a41621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:34.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.919+0000 7f92ab856640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f92a40b9970 0x7f92a40b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:34.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.919+0000 7f92ab856640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92a4068440 con 0x7f92a40b7f80 2024-09-19T08:39:34.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.919+0000 7f92ab856640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f92a4163050 con 0x7f92a40b9970 2024-09-19T08:39:34.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.919+0000 7f92ab856640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f92a4162a90 con 0x7f92a40d4820 2024-09-19T08:39:34.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.919+0000 7f92a9df3640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f92a40b9970 0x7f92a40b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:34.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.919+0000 7f92a95f2640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f92a40b7f80 0x7f92a415f560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36234/0 (socket says 172.21.15.98:36234) 2024-09-19T08:39:34.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.919+0000 7f92a95f2640 1 -- 172.21.15.98:0/2374924773 learned_addr learned my addr 172.21.15.98:0/2374924773 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:34.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.919+0000 7f92a8df1640 1 --2- 172.21.15.98:0/2374924773 >> v2:172.21.15.139:3300/0 conn(0x7f92a40d4820 0x7f92a41621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:34.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.919+0000 7f92a9df3640 1 -- 172.21.15.98:0/2374924773 >> v2:172.21.15.139:3300/0 conn(0x7f92a40d4820 msgr2=0x7f92a41621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:34.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.919+0000 7f92a9df3640 1 --2- 172.21.15.98:0/2374924773 >> v2:172.21.15.139:3300/0 conn(0x7f92a40d4820 0x7f92a41621b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:34.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.919+0000 7f92a9df3640 1 -- 172.21.15.98:0/2374924773 >> v1:172.21.15.98:6789/0 conn(0x7f92a40b7f80 legacy=0x7f92a415f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:34.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.919+0000 7f92a9df3640 1 -- 172.21.15.98:0/2374924773 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a4162d70 con 0x7f92a40b9970 2024-09-19T08:39:34.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.919+0000 7f92a8df1640 1 --2- 172.21.15.98:0/2374924773 >> v2:172.21.15.139:3300/0 conn(0x7f92a40d4820 0x7f92a41621b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:39:34.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.923+0000 7f92a9df3640 1 --2- 172.21.15.98:0/2374924773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92a40b9970 0x7f92a40b9de0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f92a40ceb00 tx=0x7f929c01c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6ea514007649ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:34.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.923+0000 7f929bfff640 1 -- 172.21.15.98:0/2374924773 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c0b2070 con 0x7f92a40b9970 2024-09-19T08:39:34.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.923+0000 7f929bfff640 1 -- 172.21.15.98:0/2374924773 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f929c0a5710 con 0x7f92a40b9970 2024-09-19T08:39:34.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.923+0000 7f929bfff640 1 -- 172.21.15.98:0/2374924773 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c0a5ae0 con 0x7f92a40b9970 2024-09-19T08:39:34.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.923+0000 7f92ab856640 1 -- 172.21.15.98:0/2374924773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92a40b9970 msgr2=0x7f92a40b9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:34.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.923+0000 7f92ab856640 1 --2- 172.21.15.98:0/2374924773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92a40b9970 0x7f92a40b9de0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f92a40ceb00 tx=0x7f929c01c7c0 comp rx=0 tx=0).stop 2024-09-19T08:39:34.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.923+0000 7f92ab856640 1 -- 172.21.15.98:0/2374924773 shutdown_connections 2024-09-19T08:39:34.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.923+0000 7f92ab856640 1 --2- 172.21.15.98:0/2374924773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92a40b9970 0x7f92a40b9de0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:34.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.923+0000 7f92ab856640 1 --2- 172.21.15.98:0/2374924773 >> v2:172.21.15.139:3300/0 conn(0x7f92a40d4820 0x7f92a41621b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:34.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.923+0000 7f92ab856640 1 -- 172.21.15.98:0/2374924773 >> 172.21.15.98:0/2374924773 conn(0x7f92a4074e40 msgr2=0x7f92a4075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:34.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.923+0000 7f92ab856640 1 -- 172.21.15.98:0/2374924773 shutdown_connections 2024-09-19T08:39:34.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.923+0000 7f92ab856640 1 -- 172.21.15.98:0/2374924773 wait complete. 2024-09-19T08:39:34.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.923+0000 7f92ab856640 1 Processor -- start 2024-09-19T08:39:34.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.923+0000 7f92ab856640 1 -- start start 2024-09-19T08:39:34.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.923+0000 7f92ab856640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92a40b7f80 0x7f92a41399e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:34.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.923+0000 7f92ab856640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92a40b9970 0x7f92a4139210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:34.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.923+0000 7f92ab856640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92a40d4820 0x7f92a4140380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:34.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.923+0000 7f92ab856640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f92a40b84e0 con 0x7f92a40b7f80 2024-09-19T08:39:34.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.923+0000 7f92ab856640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f92a40b8360 con 0x7f92a40d4820 2024-09-19T08:39:34.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.923+0000 7f92ab856640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f92a4144590 con 0x7f92a40b9970 2024-09-19T08:39:34.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.923+0000 7f92a95f2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92a40b7f80 0x7f92a41399e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:34.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.923+0000 7f92a95f2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92a40b7f80 0x7f92a41399e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48582/0 (socket says 172.21.15.98:48582) 2024-09-19T08:39:34.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.923+0000 7f92a95f2640 1 -- 172.21.15.98:0/2807617157 learned_addr learned my addr 172.21.15.98:0/2807617157 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:34.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.923+0000 7f92a9df3640 1 --2- 172.21.15.98:0/2807617157 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92a40d4820 0x7f92a4140380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:34.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.923+0000 7f92a8df1640 1 --2- 172.21.15.98:0/2807617157 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92a40b9970 0x7f92a4139210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:34.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.923+0000 7f92a95f2640 1 -- 172.21.15.98:0/2807617157 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92a40b9970 msgr2=0x7f92a4139210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:34.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.923+0000 7f92a95f2640 1 --2- 172.21.15.98:0/2807617157 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92a40b9970 0x7f92a4139210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:34.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.923+0000 7f92a95f2640 1 -- 172.21.15.98:0/2807617157 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92a40d4820 msgr2=0x7f92a4140380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:34.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.923+0000 7f92a95f2640 1 --2- 172.21.15.98:0/2807617157 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92a40d4820 0x7f92a4140380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:34.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.923+0000 7f92a95f2640 1 -- 172.21.15.98:0/2807617157 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a4155a40 con 0x7f92a40b7f80 2024-09-19T08:39:34.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.923+0000 7f92a95f2640 1 --2- 172.21.15.98:0/2807617157 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92a40b7f80 0x7f92a41399e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f92a000c7b0 tx=0x7f92a000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:34.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.923+0000 7f9299ffb640 1 -- 172.21.15.98:0/2807617157 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a0018070 con 0x7f92a40b7f80 2024-09-19T08:39:34.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.923+0000 7f9299ffb640 1 -- 172.21.15.98:0/2807617157 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92a00072d0 con 0x7f92a40b7f80 2024-09-19T08:39:34.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.923+0000 7f92ab856640 1 -- 172.21.15.98:0/2807617157 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92a4157b70 con 0x7f92a40b7f80 2024-09-19T08:39:34.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.923+0000 7f9299ffb640 1 -- 172.21.15.98:0/2807617157 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a00047f0 con 0x7f92a40b7f80 2024-09-19T08:39:34.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.923+0000 7f92ab856640 1 -- 172.21.15.98:0/2807617157 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92a413ef90 con 0x7f92a40b7f80 2024-09-19T08:39:34.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.927+0000 7f92ab856640 1 -- 172.21.15.98:0/2807617157 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92a4068f50 con 0x7f92a40b7f80 2024-09-19T08:39:34.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.927+0000 7f9299ffb640 1 -- 172.21.15.98:0/2807617157 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f92a001b070 con 0x7f92a40b7f80 2024-09-19T08:39:34.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.927+0000 7f9299ffb640 1 --2- 172.21.15.98:0/2807617157 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9288067a70 0x7f9288069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:34.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.927+0000 7f92a8df1640 1 --2- 172.21.15.98:0/2807617157 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9288067a70 0x7f9288069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:34.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.927+0000 7f9299ffb640 1 -- 172.21.15.98:0/2807617157 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f92a0085010 con 0x7f92a40b7f80 2024-09-19T08:39:34.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.927+0000 7f92a8df1640 1 --2- 172.21.15.98:0/2807617157 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9288067a70 0x7f9288069f30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f92a41608a0 tx=0x7f9294008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:34.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:34.931+0000 7f9299ffb640 1 -- 172.21.15.98:0/2807617157 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f92a4157b70 con 0x7f92a40b7f80 2024-09-19T08:39:35.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.099+0000 7f92ab856640 1 -- 172.21.15.98:0/2807617157 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f92a41448c0 con 0x7f92a40b7f80 2024-09-19T08:39:35.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.811+0000 7f9299ffb640 1 -- 172.21.15.98:0/2807617157 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v37) ==== 97+0+0 (secure 0 0 0) 0x7f92a0055640 con 0x7f92a40b7f80 2024-09-19T08:39:35.812 INFO:teuthology.orchestra.run.smithi098.stderr: removed all blocklist entries 2024-09-19T08:39:35.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.815+0000 7f92ab856640 1 -- 172.21.15.98:0/2807617157 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9288067a70 msgr2=0x7f9288069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:35.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.815+0000 7f92ab856640 1 --2- 172.21.15.98:0/2807617157 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9288067a70 0x7f9288069f30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f92a41608a0 tx=0x7f9294008040 comp rx=0 tx=0).stop 2024-09-19T08:39:35.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.815+0000 7f92ab856640 1 -- 172.21.15.98:0/2807617157 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92a40b7f80 msgr2=0x7f92a41399e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:35.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.815+0000 7f92ab856640 1 --2- 172.21.15.98:0/2807617157 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92a40b7f80 0x7f92a41399e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f92a000c7b0 tx=0x7f92a000cc70 comp rx=0 tx=0).stop 2024-09-19T08:39:35.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.815+0000 7f92ab856640 1 -- 172.21.15.98:0/2807617157 shutdown_connections 2024-09-19T08:39:35.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.815+0000 7f92ab856640 1 --2- 172.21.15.98:0/2807617157 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9288067a70 0x7f9288069f30 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:35.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.815+0000 7f92ab856640 1 --2- 172.21.15.98:0/2807617157 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92a40d4820 0x7f92a4140380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:35.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.815+0000 7f92ab856640 1 --2- 172.21.15.98:0/2807617157 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92a40b9970 0x7f92a4139210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:35.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.815+0000 7f92ab856640 1 --2- 172.21.15.98:0/2807617157 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f92a40b7f80 0x7f92a41399e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:35.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.815+0000 7f92ab856640 1 -- 172.21.15.98:0/2807617157 >> 172.21.15.98:0/2807617157 conn(0x7f92a4074e40 msgr2=0x7f92a40d0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:35.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.815+0000 7f92ab856640 1 -- 172.21.15.98:0/2807617157 shutdown_connections 2024-09-19T08:39:35.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.815+0000 7f92ab856640 1 -- 172.21.15.98:0/2807617157 wait complete. 2024-09-19T08:39:35.836 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-19T08:39:35.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.951+0000 7f88ba7c3640 1 Processor -- start 2024-09-19T08:39:35.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.951+0000 7f88ba7c3640 1 -- start start 2024-09-19T08:39:35.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.955+0000 7f88ba7c3640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f88b406bc90 0x7f88b406c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:35.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.955+0000 7f88ba7c3640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f88b40ce840 0x7f88b40cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:35.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.955+0000 7f88ba7c3640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88b40ccc60 con 0x7f88b4158fd0 2024-09-19T08:39:35.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.955+0000 7f88ba7c3640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f88b41593b0 con 0x7f88b406bc90 2024-09-19T08:39:35.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.955+0000 7f88ba7c3640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f88b40ccf40 con 0x7f88b40ce840 2024-09-19T08:39:35.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.955+0000 7f88b37fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f88b406bc90 0x7f88b406c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:35.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.955+0000 7f88b3fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f88b4158fd0 0x7f88b40b3e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36244/0 (socket says 172.21.15.98:36244) 2024-09-19T08:39:35.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.955+0000 7f88b3fff640 1 -- 172.21.15.98:0/3966549961 learned_addr learned my addr 172.21.15.98:0/3966549961 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:35.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.955+0000 7f88b8d60640 1 --2- 172.21.15.98:0/3966549961 >> v2:172.21.15.139:3300/0 conn(0x7f88b40ce840 0x7f88b40cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:35.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.955+0000 7f88b37fe640 1 -- 172.21.15.98:0/3966549961 >> v2:172.21.15.139:3300/0 conn(0x7f88b40ce840 msgr2=0x7f88b40cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:35.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.955+0000 7f88b37fe640 1 --2- 172.21.15.98:0/3966549961 >> v2:172.21.15.139:3300/0 conn(0x7f88b40ce840 0x7f88b40cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:35.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.955+0000 7f88b37fe640 1 -- 172.21.15.98:0/3966549961 >> v1:172.21.15.98:6789/0 conn(0x7f88b4158fd0 legacy=0x7f88b40b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:35.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.955+0000 7f88b37fe640 1 -- 172.21.15.98:0/3966549961 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b40cd220 con 0x7f88b406bc90 2024-09-19T08:39:35.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.955+0000 7f88b8d60640 1 --2- 172.21.15.98:0/3966549961 >> v2:172.21.15.139:3300/0 conn(0x7f88b40ce840 0x7f88b40cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:35.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.955+0000 7f88b2ffd640 1 -- 172.21.15.98:0/3966549961 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 672120876 0 0) 0x7f88b40ccc60 con 0x7f88b4158fd0 2024-09-19T08:39:35.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.955+0000 7f88b37fe640 1 --2- 172.21.15.98:0/3966549961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88b406bc90 0x7f88b406c120 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f88b40afe30 tx=0x7f88a401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbfdc52f334622d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:35.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.955+0000 7f88b2ffd640 1 -- 172.21.15.98:0/3966549961 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a40b0070 con 0x7f88b406bc90 2024-09-19T08:39:35.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.955+0000 7f88b2ffd640 1 -- 172.21.15.98:0/3966549961 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88a40ab600 con 0x7f88b406bc90 2024-09-19T08:39:35.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.955+0000 7f88b2ffd640 1 -- 172.21.15.98:0/3966549961 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a40ab9d0 con 0x7f88b406bc90 2024-09-19T08:39:35.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.959+0000 7f88ba7c3640 1 -- 172.21.15.98:0/3966549961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88b406bc90 msgr2=0x7f88b406c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:35.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.959+0000 7f88ba7c3640 1 --2- 172.21.15.98:0/3966549961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88b406bc90 0x7f88b406c120 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f88b40afe30 tx=0x7f88a401c3f0 comp rx=0 tx=0).stop 2024-09-19T08:39:35.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.959+0000 7f88ba7c3640 1 -- 172.21.15.98:0/3966549961 shutdown_connections 2024-09-19T08:39:35.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.959+0000 7f88ba7c3640 1 --2- 172.21.15.98:0/3966549961 >> v2:172.21.15.139:3300/0 conn(0x7f88b40ce840 0x7f88b40cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:35.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.959+0000 7f88ba7c3640 1 --2- 172.21.15.98:0/3966549961 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88b406bc90 0x7f88b406c120 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:35.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.959+0000 7f88ba7c3640 1 -- 172.21.15.98:0/3966549961 >> 172.21.15.98:0/3966549961 conn(0x7f88b40b2010 msgr2=0x7f88b40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:35.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.959+0000 7f88ba7c3640 1 -- 172.21.15.98:0/3966549961 shutdown_connections 2024-09-19T08:39:35.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.959+0000 7f88ba7c3640 1 -- 172.21.15.98:0/3966549961 wait complete. 2024-09-19T08:39:35.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.959+0000 7f88ba7c3640 1 Processor -- start 2024-09-19T08:39:35.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.959+0000 7f88ba7c3640 1 -- start start 2024-09-19T08:39:35.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.959+0000 7f88ba7c3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88b406bc90 0x7f88b41607c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:35.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.959+0000 7f88ba7c3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88b40ce840 0x7f88b4149ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:35.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.959+0000 7f88ba7c3640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f88b4158fd0 0x7f88b40ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:35.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.959+0000 7f88ba7c3640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f88b415c920 con 0x7f88b40ce840 2024-09-19T08:39:35.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.959+0000 7f88ba7c3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f88b415c7a0 con 0x7f88b406bc90 2024-09-19T08:39:35.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.959+0000 7f88ba7c3640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f88b406d170 con 0x7f88b4158fd0 2024-09-19T08:39:35.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.959+0000 7f88b37fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88b40ce840 0x7f88b4149ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:35.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.959+0000 7f88b37fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88b40ce840 0x7f88b4149ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48602/0 (socket says 172.21.15.98:48602) 2024-09-19T08:39:35.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.959+0000 7f88b37fe640 1 -- 172.21.15.98:0/4187639606 learned_addr learned my addr 172.21.15.98:0/4187639606 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:35.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.959+0000 7f88b3fff640 1 --2- 172.21.15.98:0/4187639606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88b406bc90 0x7f88b41607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:35.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.959+0000 7f88b8d60640 1 --2- 172.21.15.98:0/4187639606 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f88b4158fd0 0x7f88b40ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:35.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.959+0000 7f88b37fe640 1 -- 172.21.15.98:0/4187639606 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f88b4158fd0 msgr2=0x7f88b40ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:35.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.959+0000 7f88b37fe640 1 --2- 172.21.15.98:0/4187639606 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f88b4158fd0 0x7f88b40ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:35.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.959+0000 7f88b37fe640 1 -- 172.21.15.98:0/4187639606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88b406bc90 msgr2=0x7f88b41607c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:35.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.959+0000 7f88b37fe640 1 --2- 172.21.15.98:0/4187639606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88b406bc90 0x7f88b41607c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:35.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.959+0000 7f88b37fe640 1 -- 172.21.15.98:0/4187639606 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b4160e30 con 0x7f88b40ce840 2024-09-19T08:39:35.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.959+0000 7f88b37fe640 1 --2- 172.21.15.98:0/4187639606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88b40ce840 0x7f88b4149ce0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f88a401a020 tx=0x7f88a401ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:35.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.959+0000 7f88b0ff9640 1 -- 172.21.15.98:0/4187639606 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a40b0050 con 0x7f88b40ce840 2024-09-19T08:39:35.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.959+0000 7f88b0ff9640 1 -- 172.21.15.98:0/4187639606 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88a40b1040 con 0x7f88b40ce840 2024-09-19T08:39:35.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.959+0000 7f88ba7c3640 1 -- 172.21.15.98:0/4187639606 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88b4161110 con 0x7f88b40ce840 2024-09-19T08:39:35.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.959+0000 7f88b0ff9640 1 -- 172.21.15.98:0/4187639606 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a4004220 con 0x7f88b40ce840 2024-09-19T08:39:35.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.959+0000 7f88ba7c3640 1 -- 172.21.15.98:0/4187639606 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88b415dda0 con 0x7f88b40ce840 2024-09-19T08:39:35.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.963+0000 7f88ba7c3640 1 -- 172.21.15.98:0/4187639606 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88b415dfa0 con 0x7f88b40ce840 2024-09-19T08:39:35.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.963+0000 7f88b0ff9640 1 -- 172.21.15.98:0/4187639606 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f88a40be060 con 0x7f88b40ce840 2024-09-19T08:39:35.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.963+0000 7f88b0ff9640 1 --2- 172.21.15.98:0/4187639606 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f88a0067a70 0x7f88a0069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:35.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.963+0000 7f88b0ff9640 1 -- 172.21.15.98:0/4187639606 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4857+0+0 (secure 0 0 0) 0x7f88a411f740 con 0x7f88b40ce840 2024-09-19T08:39:35.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.963+0000 7f88b3fff640 1 --2- 172.21.15.98:0/4187639606 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f88a0067a70 0x7f88a0069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:35.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.963+0000 7f88b3fff640 1 --2- 172.21.15.98:0/4187639606 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f88a0067a70 0x7f88a0069f30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f88a80012f0 tx=0x7f88a8005ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:35.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:35.967+0000 7f88b0ff9640 1 -- 172.21.15.98:0/4187639606 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f88a4016070 con 0x7f88b40ce840 2024-09-19T08:39:36.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.187+0000 7f88ba7c3640 1 -- 172.21.15.98:0/4187639606 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f88b415a0e0 con 0x7f88b40ce840 2024-09-19T08:39:36.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.187+0000 7f88b0ff9640 1 -- 172.21.15.98:0/4187639606 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7f88a40f02d0 con 0x7f88b40ce840 2024-09-19T08:39:36.187 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:39:36.188 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:39:36.188 INFO:teuthology.orchestra.run.smithi098.stdout: "auth_dump": [ 2024-09-19T08:39:36.188 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:39:36.188 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.a", 2024-09-19T08:39:36.188 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmuujGBRAAa2k/kW68NJI6nUKx+W+G2Q==", 2024-09-19T08:39:36.188 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:39:36.188 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:39:36.188 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:39:36.188 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:39:36.188 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:39:36.188 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:39:36.189 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:39:36.189 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:39:36.189 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.b", 2024-09-19T08:39:36.189 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmFoojFBAAaOZPpa41fdjBPEn3CGsJnA==", 2024-09-19T08:39:36.189 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:39:36.189 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:39:36.189 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:39:36.189 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:39:36.189 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:39:36.189 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:39:36.189 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:39:36.189 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:39:36.189 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.c", 2024-09-19T08:39:36.189 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmcQcDDhAAF0QJMlljMH+DiHfWW+/Sog==", 2024-09-19T08:39:36.190 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:39:36.190 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:39:36.190 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:39:36.190 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:39:36.190 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:39:36.190 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:39:36.190 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:39:36.190 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:39:36.190 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.d", 2024-09-19T08:39:36.190 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmwWmqHBAAf15VcrdeHf8AzQjGsN9LoA==", 2024-09-19T08:39:36.190 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:39:36.190 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:39:36.190 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:39:36.190 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:39:36.191 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:39:36.191 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:39:36.191 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:39:36.191 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:39:36.191 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.0", 2024-09-19T08:39:36.191 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDC4utm9PjNKxAAa6SG9AE4smPiu4azRAFKvg==", 2024-09-19T08:39:36.191 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:39:36.191 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:39:36.191 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:39:36.191 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:39:36.191 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:39:36.191 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:39:36.191 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:39:36.191 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.1", 2024-09-19T08:39:36.191 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDG4utmHy6UCxAAN6hpG4DJEp81iG/KoloTvQ==", 2024-09-19T08:39:36.192 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:39:36.192 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:39:36.192 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:39:36.192 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:39:36.192 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:39:36.192 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:39:36.192 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:39:36.192 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.2", 2024-09-19T08:39:36.192 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDJ4utmUVP4GhAAJNAL30sEAseoDG2fE6YbZQ==", 2024-09-19T08:39:36.192 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:39:36.192 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:39:36.192 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:39:36.192 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:39:36.193 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:39:36.193 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:39:36.193 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:39:36.193 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.3", 2024-09-19T08:39:36.193 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDM4utmTKgoLRAAiCrPRLML+wvjOt6RhKay3w==", 2024-09-19T08:39:36.193 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:39:36.193 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:39:36.193 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:39:36.193 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:39:36.193 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:39:36.193 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:39:36.193 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:39:36.193 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.4", 2024-09-19T08:39:36.193 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDS4utmAf6bMhAAI3uENuFoXLicUeuspgFjYA==", 2024-09-19T08:39:36.194 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:39:36.194 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:39:36.194 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:39:36.194 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:39:36.194 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:39:36.194 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:39:36.194 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:39:36.194 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.5", 2024-09-19T08:39:36.194 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDW4utmRjxwEhAAF1A2GtrZXVx9BlmjHjhUDQ==", 2024-09-19T08:39:36.194 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:39:36.194 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:39:36.194 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:39:36.194 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:39:36.194 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:39:36.195 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:39:36.195 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:39:36.195 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.6", 2024-09-19T08:39:36.195 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDZ4utmSk/XIRAAROSfzGL7smR88ZkDSx5jAg==", 2024-09-19T08:39:36.195 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:39:36.195 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:39:36.195 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:39:36.195 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:39:36.195 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:39:36.195 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:39:36.195 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:39:36.195 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.7", 2024-09-19T08:39:36.195 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDc4utmXfpbMxAAqEM4I/ACFvfu/sMA8bMcrQ==", 2024-09-19T08:39:36.195 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:39:36.195 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:39:36.196 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:39:36.196 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:39:36.196 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:39:36.196 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:39:36.196 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:39:36.196 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.0", 2024-09-19T08:39:36.196 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utm0vNpIhAA4S1E8dFZ0+ibmSKhmZ+bIA==", 2024-09-19T08:39:36.196 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:39:36.196 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:39:36.196 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow r", 2024-09-19T08:39:36.196 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-19T08:39:36.196 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:39:36.196 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:39:36.196 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:39:36.197 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.1", 2024-09-19T08:39:36.197 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmx7xyKhAAC84lykPEb01MiypXKQVREQ==", 2024-09-19T08:39:36.197 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:39:36.197 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:39:36.197 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-19T08:39:36.197 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-19T08:39:36.197 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-19T08:39:36.197 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:39:36.197 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:39:36.197 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:39:36.197 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.2", 2024-09-19T08:39:36.197 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmteOQJxAAtTF2Py21ZE2WRQjeQSrkKQ==", 2024-09-19T08:39:36.197 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:39:36.197 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:39:36.198 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-19T08:39:36.198 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-19T08:39:36.198 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-19T08:39:36.198 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:39:36.198 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:39:36.198 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:39:36.198 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.3", 2024-09-19T08:39:36.198 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmKN92LxAAMqraZOWRT6IIfx5AuDWS2g==", 2024-09-19T08:39:36.198 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:39:36.198 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:39:36.198 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-19T08:39:36.198 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-19T08:39:36.198 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-19T08:39:36.198 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:39:36.199 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:39:36.199 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:39:36.199 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.admin", 2024-09-19T08:39:36.199 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC+4utmLH97KhAA1DTYW5bTrGUG+7cb8+d7gA==", 2024-09-19T08:39:36.199 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:39:36.199 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-09-19T08:39:36.199 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:39:36.199 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:39:36.199 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:39:36.199 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:39:36.199 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:39:36.199 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:39:36.199 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mgr.x", 2024-09-19T08:39:36.200 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmbKuyAhAA0/XjHUIPOIV/YvNEhB8rEQ==", 2024-09-19T08:39:36.200 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:39:36.200 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-09-19T08:39:36.200 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile mgr", 2024-09-19T08:39:36.200 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:39:36.200 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:39:36.200 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:39:36.200 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:39:36.200 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mgr.y", 2024-09-19T08:39:36.200 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC+4utmU+ZrOxAAsvcwINH6qyxfN7N+rDrkqg==", 2024-09-19T08:39:36.200 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:39:36.200 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-09-19T08:39:36.200 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile mgr", 2024-09-19T08:39:36.200 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:39:36.200 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:39:36.200 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:39:36.201 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-19T08:39:36.201 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:39:36.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.191+0000 7f88ba7c3640 1 -- 172.21.15.98:0/4187639606 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f88a0067a70 msgr2=0x7f88a0069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:36.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.191+0000 7f88ba7c3640 1 --2- 172.21.15.98:0/4187639606 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f88a0067a70 0x7f88a0069f30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f88a80012f0 tx=0x7f88a8005ec0 comp rx=0 tx=0).stop 2024-09-19T08:39:36.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.191+0000 7f88ba7c3640 1 -- 172.21.15.98:0/4187639606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88b40ce840 msgr2=0x7f88b4149ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:36.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.191+0000 7f88ba7c3640 1 --2- 172.21.15.98:0/4187639606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88b40ce840 0x7f88b4149ce0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f88a401a020 tx=0x7f88a401ac30 comp rx=0 tx=0).stop 2024-09-19T08:39:36.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.191+0000 7f88ba7c3640 1 -- 172.21.15.98:0/4187639606 shutdown_connections 2024-09-19T08:39:36.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.191+0000 7f88ba7c3640 1 --2- 172.21.15.98:0/4187639606 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f88a0067a70 0x7f88a0069f30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:36.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.191+0000 7f88ba7c3640 1 --2- 172.21.15.98:0/4187639606 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f88b4158fd0 0x7f88b40ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:36.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.191+0000 7f88ba7c3640 1 --2- 172.21.15.98:0/4187639606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88b40ce840 0x7f88b4149ce0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:36.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.191+0000 7f88ba7c3640 1 --2- 172.21.15.98:0/4187639606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88b406bc90 0x7f88b41607c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:36.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.191+0000 7f88ba7c3640 1 -- 172.21.15.98:0/4187639606 >> 172.21.15.98:0/4187639606 conn(0x7f88b40b2010 msgr2=0x7f88b40b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:36.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.191+0000 7f88ba7c3640 1 -- 172.21.15.98:0/4187639606 shutdown_connections 2024-09-19T08:39:36.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.191+0000 7f88ba7c3640 1 -- 172.21.15.98:0/4187639606 wait complete. 2024-09-19T08:39:36.206 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-19T08:39:36.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.327+0000 7f6d2c789640 1 Processor -- start 2024-09-19T08:39:36.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.327+0000 7f6d2c789640 1 -- start start 2024-09-19T08:39:36.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.327+0000 7f6d2c789640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6d2406bc90 0x7f6d2406c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:36.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.327+0000 7f6d2c789640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d240ccc60 con 0x7f6d24158fd0 2024-09-19T08:39:36.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.327+0000 7f6d2c789640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d240ccf40 con 0x7f6d240ce840 2024-09-19T08:39:36.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.327+0000 7f6d2c789640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f6d241593b0 con 0x7f6d2406bc90 2024-09-19T08:39:36.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.327+0000 7f6d29d24640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6d2406bc90 0x7f6d2406c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:36.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.327+0000 7f6d2a525640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f6d24158fd0 0x7f6d240b3e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36248/0 (socket says 172.21.15.98:36248) 2024-09-19T08:39:36.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.327+0000 7f6d2a525640 1 -- 172.21.15.98:0/1859147346 learned_addr learned my addr 172.21.15.98:0/1859147346 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:36.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.327+0000 7f6d29523640 1 -- 172.21.15.98:0/1859147346 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 410896852 0 0) 0x7f6d240ccc60 con 0x7f6d24158fd0 2024-09-19T08:39:36.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.327+0000 7f6d29523640 1 -- 172.21.15.98:0/1859147346 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d10003530 con 0x7f6d24158fd0 2024-09-19T08:39:36.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.327+0000 7f6d29d24640 1 -- 172.21.15.98:0/1859147346 >> v1:172.21.15.139:6789/0 conn(0x7f6d240ce840 legacy=0x7f6d240cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:36.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.327+0000 7f6d29523640 1 -- 172.21.15.98:0/1859147346 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3703327553 0 0) 0x7f6d10003530 con 0x7f6d24158fd0 2024-09-19T08:39:36.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.327+0000 7f6d29d24640 1 -- 172.21.15.98:0/1859147346 >> v1:172.21.15.98:6789/0 conn(0x7f6d24158fd0 legacy=0x7f6d240b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:36.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.327+0000 7f6d29d24640 1 -- 172.21.15.98:0/1859147346 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d240cd220 con 0x7f6d2406bc90 2024-09-19T08:39:36.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.331+0000 7f6d29d24640 1 --2- 172.21.15.98:0/1859147346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d2406bc90 0x7f6d2406c120 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6d140097c0 tx=0x7f6d1401c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2789504f31ca4fb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:36.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.331+0000 7f6d29523640 1 -- 172.21.15.98:0/1859147346 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d140b0070 con 0x7f6d2406bc90 2024-09-19T08:39:36.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.331+0000 7f6d29523640 1 -- 172.21.15.98:0/1859147346 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d140ab400 con 0x7f6d2406bc90 2024-09-19T08:39:36.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.331+0000 7f6d29523640 1 -- 172.21.15.98:0/1859147346 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d20001680 con 0x7f6d2406bc90 2024-09-19T08:39:36.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.331+0000 7f6d2c789640 1 -- 172.21.15.98:0/1859147346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d2406bc90 msgr2=0x7f6d2406c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:36.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.331+0000 7f6d2c789640 1 --2- 172.21.15.98:0/1859147346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d2406bc90 0x7f6d2406c120 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6d140097c0 tx=0x7f6d1401c330 comp rx=0 tx=0).stop 2024-09-19T08:39:36.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.331+0000 7f6d2c789640 1 -- 172.21.15.98:0/1859147346 shutdown_connections 2024-09-19T08:39:36.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.331+0000 7f6d2c789640 1 --2- 172.21.15.98:0/1859147346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d2406bc90 0x7f6d2406c120 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:36.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.331+0000 7f6d2c789640 1 -- 172.21.15.98:0/1859147346 >> 172.21.15.98:0/1859147346 conn(0x7f6d240b2010 msgr2=0x7f6d240b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:36.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.331+0000 7f6d2c789640 1 -- 172.21.15.98:0/1859147346 shutdown_connections 2024-09-19T08:39:36.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.331+0000 7f6d2c789640 1 -- 172.21.15.98:0/1859147346 wait complete. 2024-09-19T08:39:36.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.331+0000 7f6d2c789640 1 Processor -- start 2024-09-19T08:39:36.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.331+0000 7f6d2c789640 1 -- start start 2024-09-19T08:39:36.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.331+0000 7f6d2c789640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d2406bc90 0x7f6d24153930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:36.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.331+0000 7f6d2c789640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d240ce840 0x7f6d240ac540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:36.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.331+0000 7f6d2c789640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d24158fd0 0x7f6d24144010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:36.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.331+0000 7f6d2c789640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2415c920 con 0x7f6d24158fd0 2024-09-19T08:39:36.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.331+0000 7f6d2c789640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2415c7a0 con 0x7f6d240ce840 2024-09-19T08:39:36.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.331+0000 7f6d2c789640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6d24169120 con 0x7f6d2406bc90 2024-09-19T08:39:36.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.331+0000 7f6d2ad26640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d24158fd0 0x7f6d24144010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:36.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.331+0000 7f6d2ad26640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d24158fd0 0x7f6d24144010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48622/0 (socket says 172.21.15.98:48622) 2024-09-19T08:39:36.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.331+0000 7f6d2ad26640 1 -- 172.21.15.98:0/2735857437 learned_addr learned my addr 172.21.15.98:0/2735857437 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:36.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.331+0000 7f6d29d24640 1 --2- 172.21.15.98:0/2735857437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d240ce840 0x7f6d240ac540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:36.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.331+0000 7f6d2a525640 1 --2- 172.21.15.98:0/2735857437 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d2406bc90 0x7f6d24153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:36.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.331+0000 7f6d2ad26640 1 -- 172.21.15.98:0/2735857437 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d2406bc90 msgr2=0x7f6d24153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:36.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.331+0000 7f6d2ad26640 1 --2- 172.21.15.98:0/2735857437 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d2406bc90 0x7f6d24153930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:36.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.331+0000 7f6d2ad26640 1 -- 172.21.15.98:0/2735857437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d240ce840 msgr2=0x7f6d240ac540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:36.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.331+0000 7f6d2ad26640 1 --2- 172.21.15.98:0/2735857437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d240ce840 0x7f6d240ac540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:36.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.331+0000 7f6d2ad26640 1 -- 172.21.15.98:0/2735857437 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d2415dee0 con 0x7f6d24158fd0 2024-09-19T08:39:36.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.331+0000 7f6d2a525640 1 --2- 172.21.15.98:0/2735857437 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d2406bc90 0x7f6d24153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:36.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.331+0000 7f6d2ad26640 1 --2- 172.21.15.98:0/2735857437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d24158fd0 0x7f6d24144010 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00c770 tx=0x7f6d1c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:36.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.335+0000 7f6d1affd640 1 -- 172.21.15.98:0/2735857437 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c003b80 con 0x7f6d24158fd0 2024-09-19T08:39:36.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.335+0000 7f6d1affd640 1 -- 172.21.15.98:0/2735857437 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d1c003d20 con 0x7f6d24158fd0 2024-09-19T08:39:36.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.335+0000 7f6d1affd640 1 -- 172.21.15.98:0/2735857437 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c00aa90 con 0x7f6d24158fd0 2024-09-19T08:39:36.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.335+0000 7f6d2c789640 1 -- 172.21.15.98:0/2735857437 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d2415e1c0 con 0x7f6d24158fd0 2024-09-19T08:39:36.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.335+0000 7f6d2c789640 1 -- 172.21.15.98:0/2735857437 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d2415e780 con 0x7f6d24158fd0 2024-09-19T08:39:36.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.335+0000 7f6d2c789640 1 -- 172.21.15.98:0/2735857437 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d2406c120 con 0x7f6d24158fd0 2024-09-19T08:39:36.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.335+0000 7f6d1affd640 1 -- 172.21.15.98:0/2735857437 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f6d1c00ac30 con 0x7f6d24158fd0 2024-09-19T08:39:36.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.335+0000 7f6d1affd640 1 --2- 172.21.15.98:0/2735857437 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6d08067a70 0x7f6d08069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:36.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.335+0000 7f6d1affd640 1 -- 172.21.15.98:0/2735857437 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4857+0+0 (secure 0 0 0) 0x7f6d1c085bd0 con 0x7f6d24158fd0 2024-09-19T08:39:36.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.335+0000 7f6d2a525640 1 --2- 172.21.15.98:0/2735857437 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6d08067a70 0x7f6d08069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:36.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.335+0000 7f6d2a525640 1 --2- 172.21.15.98:0/2735857437 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6d08067a70 0x7f6d08069f30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6d20001ec0 tx=0x7f6d20009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:36.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.339+0000 7f6d1affd640 1 -- 172.21.15.98:0/2735857437 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6d1c0567e0 con 0x7f6d24158fd0 2024-09-19T08:39:36.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.491+0000 7f6d2c789640 1 -- 172.21.15.98:0/2735857437 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f6d240757a0 con 0x7f6d24158fd0 2024-09-19T08:39:36.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.491+0000 7f6d1affd640 1 -- 172.21.15.98:0/2735857437 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v39) ==== 84+0+65 (secure 0 0 0) 0x7f6d1c0560e0 con 0x7f6d24158fd0 2024-09-19T08:39:36.491 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:39:36.491 INFO:teuthology.orchestra.run.smithi098.stdout:[ 2024-09-19T08:39:36.491 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:39:36.491 INFO:teuthology.orchestra.run.smithi098.stdout: "poolnum": 1, 2024-09-19T08:39:36.491 INFO:teuthology.orchestra.run.smithi098.stdout: "poolname": ".mgr" 2024-09-19T08:39:36.491 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:39:36.491 INFO:teuthology.orchestra.run.smithi098.stdout:] 2024-09-19T08:39:36.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.491+0000 7f6d2c789640 1 -- 172.21.15.98:0/2735857437 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6d08067a70 msgr2=0x7f6d08069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:36.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.491+0000 7f6d2c789640 1 --2- 172.21.15.98:0/2735857437 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6d08067a70 0x7f6d08069f30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6d20001ec0 tx=0x7f6d20009040 comp rx=0 tx=0).stop 2024-09-19T08:39:36.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.491+0000 7f6d2c789640 1 -- 172.21.15.98:0/2735857437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d24158fd0 msgr2=0x7f6d24144010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:36.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.491+0000 7f6d2c789640 1 --2- 172.21.15.98:0/2735857437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d24158fd0 0x7f6d24144010 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00c770 tx=0x7f6d1c00cc30 comp rx=0 tx=0).stop 2024-09-19T08:39:36.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.495+0000 7f6d2c789640 1 -- 172.21.15.98:0/2735857437 shutdown_connections 2024-09-19T08:39:36.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.495+0000 7f6d2c789640 1 --2- 172.21.15.98:0/2735857437 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6d08067a70 0x7f6d08069f30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:36.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.495+0000 7f6d2c789640 1 --2- 172.21.15.98:0/2735857437 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6d24158fd0 0x7f6d24144010 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:36.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.495+0000 7f6d2c789640 1 --2- 172.21.15.98:0/2735857437 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6d240ce840 0x7f6d240ac540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:36.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.495+0000 7f6d2c789640 1 --2- 172.21.15.98:0/2735857437 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6d2406bc90 0x7f6d24153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:36.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.495+0000 7f6d2c789640 1 -- 172.21.15.98:0/2735857437 >> 172.21.15.98:0/2735857437 conn(0x7f6d240b2010 msgr2=0x7f6d240b1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:36.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.495+0000 7f6d2c789640 1 -- 172.21.15.98:0/2735857437 shutdown_connections 2024-09-19T08:39:36.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.495+0000 7f6d2c789640 1 -- 172.21.15.98:0/2735857437 wait complete. 2024-09-19T08:39:36.508 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T08:39:36.508 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T08:39:36.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.627+0000 7f348d721640 1 Processor -- start 2024-09-19T08:39:36.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.627+0000 7f348d721640 1 -- start start 2024-09-19T08:39:36.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.627+0000 7f348d721640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f348815c290 0x7f34880cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:36.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.627+0000 7f348d721640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f348806d4c0 0x7f348806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:36.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.627+0000 7f348d721640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f348806dde0 con 0x7f34880cf700 2024-09-19T08:39:36.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.627+0000 7f348d721640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f3488160e80 con 0x7f348815c290 2024-09-19T08:39:36.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.627+0000 7f348d721640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f34880cd510 con 0x7f348806d4c0 2024-09-19T08:39:36.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.627+0000 7f3486ffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f348815c290 0x7f34880cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:36.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.627+0000 7f34867fc640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f34880cf700 0x7f348815ff50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36250/0 (socket says 172.21.15.98:36250) 2024-09-19T08:39:36.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.627+0000 7f34867fc640 1 -- 172.21.15.98:0/1966605358 learned_addr learned my addr 172.21.15.98:0/1966605358 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:36.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.627+0000 7f34877fe640 1 --2- 172.21.15.98:0/1966605358 >> v2:172.21.15.139:3300/0 conn(0x7f348806d4c0 0x7f348806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:36.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.627+0000 7f3486ffd640 1 -- 172.21.15.98:0/1966605358 >> v2:172.21.15.139:3300/0 conn(0x7f348806d4c0 msgr2=0x7f348806d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:36.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.627+0000 7f3486ffd640 1 --2- 172.21.15.98:0/1966605358 >> v2:172.21.15.139:3300/0 conn(0x7f348806d4c0 0x7f348806d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:36.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.627+0000 7f3486ffd640 1 -- 172.21.15.98:0/1966605358 >> v1:172.21.15.98:6789/0 conn(0x7f34880cf700 legacy=0x7f348815ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:36.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.627+0000 7f3486ffd640 1 -- 172.21.15.98:0/1966605358 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34881608c0 con 0x7f348815c290 2024-09-19T08:39:36.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.627+0000 7f3485ffb640 1 -- 172.21.15.98:0/1966605358 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4079049612 0 0) 0x7f348806dde0 con 0x7f34880cf700 2024-09-19T08:39:36.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.627+0000 7f34877fe640 1 --2- 172.21.15.98:0/1966605358 >> v2:172.21.15.139:3300/0 conn(0x7f348806d4c0 0x7f348806d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:36.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.631+0000 7f3486ffd640 1 --2- 172.21.15.98:0/1966605358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f348815c290 0x7f34880cd9a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f348806e0d0 tx=0x7f347801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce23fcee0102dc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:36.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.631+0000 7f3485ffb640 1 -- 172.21.15.98:0/1966605358 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34780b0070 con 0x7f348815c290 2024-09-19T08:39:36.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.631+0000 7f3485ffb640 1 -- 172.21.15.98:0/1966605358 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34780ab600 con 0x7f348815c290 2024-09-19T08:39:36.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.631+0000 7f3485ffb640 1 -- 172.21.15.98:0/1966605358 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34780ab9d0 con 0x7f348815c290 2024-09-19T08:39:36.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.631+0000 7f348d721640 1 -- 172.21.15.98:0/1966605358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f348815c290 msgr2=0x7f34880cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:36.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.631+0000 7f348d721640 1 --2- 172.21.15.98:0/1966605358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f348815c290 0x7f34880cd9a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f348806e0d0 tx=0x7f347801c3f0 comp rx=0 tx=0).stop 2024-09-19T08:39:36.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.631+0000 7f348d721640 1 -- 172.21.15.98:0/1966605358 shutdown_connections 2024-09-19T08:39:36.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.631+0000 7f348d721640 1 --2- 172.21.15.98:0/1966605358 >> v2:172.21.15.139:3300/0 conn(0x7f348806d4c0 0x7f348806d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:36.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.631+0000 7f348d721640 1 --2- 172.21.15.98:0/1966605358 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f348815c290 0x7f34880cd9a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:36.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.631+0000 7f348d721640 1 -- 172.21.15.98:0/1966605358 >> 172.21.15.98:0/1966605358 conn(0x7f348806c090 msgr2=0x7f348806c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:36.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.631+0000 7f348d721640 1 -- 172.21.15.98:0/1966605358 shutdown_connections 2024-09-19T08:39:36.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.631+0000 7f348d721640 1 -- 172.21.15.98:0/1966605358 wait complete. 2024-09-19T08:39:36.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.631+0000 7f348d721640 1 Processor -- start 2024-09-19T08:39:36.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.635+0000 7f348d721640 1 -- start start 2024-09-19T08:39:36.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.635+0000 7f348d721640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f348806d4c0 0x7f34881639a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:36.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.635+0000 7f348d721640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f34880cf700 0x7f3488163ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:36.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.635+0000 7f348d721640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f348815c290 0x7f34881445b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:36.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.635+0000 7f348d721640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f348815c7f0 con 0x7f34880cf700 2024-09-19T08:39:36.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.635+0000 7f348d721640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f348815c670 con 0x7f348815c290 2024-09-19T08:39:36.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.635+0000 7f348d721640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f34880b5400 con 0x7f348806d4c0 2024-09-19T08:39:36.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.635+0000 7f34867fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f34880cf700 0x7f3488163ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:36.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.635+0000 7f34867fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f34880cf700 0x7f3488163ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48636/0 (socket says 172.21.15.98:48636) 2024-09-19T08:39:36.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.635+0000 7f34867fc640 1 -- 172.21.15.98:0/3448891914 learned_addr learned my addr 172.21.15.98:0/3448891914 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:36.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.635+0000 7f34877fe640 1 --2- 172.21.15.98:0/3448891914 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f348815c290 0x7f34881445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:36.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.635+0000 7f34867fc640 1 -- 172.21.15.98:0/3448891914 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f348806d4c0 msgr2=0x7f34881639a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:36.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.635+0000 7f34867fc640 1 --2- 172.21.15.98:0/3448891914 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f348806d4c0 0x7f34881639a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:36.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.635+0000 7f34867fc640 1 -- 172.21.15.98:0/3448891914 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f348815c290 msgr2=0x7f34881445b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:36.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.635+0000 7f34867fc640 1 --2- 172.21.15.98:0/3448891914 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f348815c290 0x7f34881445b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:36.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.635+0000 7f34867fc640 1 -- 172.21.15.98:0/3448891914 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34880ae220 con 0x7f34880cf700 2024-09-19T08:39:36.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.635+0000 7f34877fe640 1 --2- 172.21.15.98:0/3448891914 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f348815c290 0x7f34881445b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:36.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.635+0000 7f34867fc640 1 --2- 172.21.15.98:0/3448891914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f34880cf700 0x7f3488163ee0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f347c0011f0 tx=0x7f347c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:36.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.635+0000 7f346f7fe640 1 -- 172.21.15.98:0/3448891914 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c018070 con 0x7f34880cf700 2024-09-19T08:39:36.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.635+0000 7f346f7fe640 1 -- 172.21.15.98:0/3448891914 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f347c0074e0 con 0x7f34880cf700 2024-09-19T08:39:36.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.635+0000 7f348d721640 1 -- 172.21.15.98:0/3448891914 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34880ae500 con 0x7f34880cf700 2024-09-19T08:39:36.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.635+0000 7f346f7fe640 1 -- 172.21.15.98:0/3448891914 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c004a10 con 0x7f34880cf700 2024-09-19T08:39:36.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.635+0000 7f348d721640 1 -- 172.21.15.98:0/3448891914 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3488145530 con 0x7f34880cf700 2024-09-19T08:39:36.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.635+0000 7f348d721640 1 -- 172.21.15.98:0/3448891914 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34880adfb0 con 0x7f34880cf700 2024-09-19T08:39:36.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.635+0000 7f346f7fe640 1 -- 172.21.15.98:0/3448891914 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f347c01b070 con 0x7f34880cf700 2024-09-19T08:39:36.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.635+0000 7f346f7fe640 1 --2- 172.21.15.98:0/3448891914 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3474067a70 0x7f3474069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:36.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.635+0000 7f346f7fe640 1 -- 172.21.15.98:0/3448891914 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4857+0+0 (secure 0 0 0) 0x7f347c084f90 con 0x7f34880cf700 2024-09-19T08:39:36.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.635+0000 7f3486ffd640 1 --2- 172.21.15.98:0/3448891914 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3474067a70 0x7f3474069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:36.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.639+0000 7f3486ffd640 1 --2- 172.21.15.98:0/3448891914 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3474067a70 0x7f3474069f30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f34780ad000 tx=0x7f3478004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:36.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.639+0000 7f346f7fe640 1 -- 172.21.15.98:0/3448891914 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f347c055ba0 con 0x7f34880cf700 2024-09-19T08:39:36.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.791+0000 7f348d721640 1 -- 172.21.15.98:0/3448891914 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f34880cd980 con 0x7f34880cf700 2024-09-19T08:39:36.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.879+0000 7f346f7fe640 1 -- 172.21.15.98:0/3448891914 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v40) ==== 138+0+0 (secure 0 0 0) 0x7f347c0554a0 con 0x7f34880cf700 2024-09-19T08:39:36.881 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_metadata' created 2024-09-19T08:39:36.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.883+0000 7f348d721640 1 -- 172.21.15.98:0/3448891914 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3474067a70 msgr2=0x7f3474069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:36.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.883+0000 7f348d721640 1 --2- 172.21.15.98:0/3448891914 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3474067a70 0x7f3474069f30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f34780ad000 tx=0x7f3478004740 comp rx=0 tx=0).stop 2024-09-19T08:39:36.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.883+0000 7f348d721640 1 -- 172.21.15.98:0/3448891914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f34880cf700 msgr2=0x7f3488163ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:36.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.883+0000 7f348d721640 1 --2- 172.21.15.98:0/3448891914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f34880cf700 0x7f3488163ee0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f347c0011f0 tx=0x7f347c00c9b0 comp rx=0 tx=0).stop 2024-09-19T08:39:36.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.883+0000 7f348d721640 1 -- 172.21.15.98:0/3448891914 shutdown_connections 2024-09-19T08:39:36.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.883+0000 7f348d721640 1 --2- 172.21.15.98:0/3448891914 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3474067a70 0x7f3474069f30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:36.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.883+0000 7f348d721640 1 --2- 172.21.15.98:0/3448891914 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f348815c290 0x7f34881445b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:36.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.883+0000 7f348d721640 1 --2- 172.21.15.98:0/3448891914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f34880cf700 0x7f3488163ee0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:36.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.883+0000 7f348d721640 1 --2- 172.21.15.98:0/3448891914 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f348806d4c0 0x7f34881639a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:36.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.883+0000 7f348d721640 1 -- 172.21.15.98:0/3448891914 >> 172.21.15.98:0/3448891914 conn(0x7f348806c090 msgr2=0x7f34880b0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:36.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.883+0000 7f348d721640 1 -- 172.21.15.98:0/3448891914 shutdown_connections 2024-09-19T08:39:36.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:36.883+0000 7f348d721640 1 -- 172.21.15.98:0/3448891914 wait complete. 2024-09-19T08:39:36.906 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T08:39:37.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.031+0000 7f512466a640 1 Processor -- start 2024-09-19T08:39:37.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.031+0000 7f512466a640 1 -- start start 2024-09-19T08:39:37.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.031+0000 7f512466a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f511c15c440 0x7f511c0706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:37.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.031+0000 7f512466a640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f511c070c20 con 0x7f511c0b53a0 2024-09-19T08:39:37.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.031+0000 7f512466a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f511c15c820 con 0x7f511c072f90 2024-09-19T08:39:37.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.031+0000 7f512466a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f511c070ec0 con 0x7f511c15c440 2024-09-19T08:39:37.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.035+0000 7f5122c07640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f511c0b53a0 0x7f511c06f380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36262/0 (socket says 172.21.15.98:36262) 2024-09-19T08:39:37.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.035+0000 7f5122c07640 1 -- 172.21.15.98:0/3433017762 learned_addr learned my addr 172.21.15.98:0/3433017762 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:37.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.035+0000 7f5122406640 1 --2- 172.21.15.98:0/3433017762 >> v2:172.21.15.139:3300/0 conn(0x7f511c15c440 0x7f511c0706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:37.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.035+0000 7f5121404640 1 -- 172.21.15.98:0/3433017762 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2609793240 0 0) 0x7f511c070c20 con 0x7f511c0b53a0 2024-09-19T08:39:37.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.035+0000 7f5121404640 1 -- 172.21.15.98:0/3433017762 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5108003530 con 0x7f511c0b53a0 2024-09-19T08:39:37.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.035+0000 7f5121404640 1 -- 172.21.15.98:0/3433017762 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2837760085 0 0) 0x7f5108003530 con 0x7f511c0b53a0 2024-09-19T08:39:37.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.035+0000 7f5121404640 1 -- 172.21.15.98:0/3433017762 >> v2:172.21.15.139:3300/0 conn(0x7f511c15c440 msgr2=0x7f511c0706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:37.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.035+0000 7f5121404640 1 --2- 172.21.15.98:0/3433017762 >> v2:172.21.15.139:3300/0 conn(0x7f511c15c440 0x7f511c0706e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:37.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.035+0000 7f5121404640 1 -- 172.21.15.98:0/3433017762 >> v1:172.21.15.139:6789/0 conn(0x7f511c072f90 legacy=0x7f511c0ce400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:37.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.035+0000 7f5121404640 1 -- 172.21.15.98:0/3433017762 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f511c0711a0 con 0x7f511c0b53a0 2024-09-19T08:39:37.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.035+0000 7f5121404640 1 -- 172.21.15.98:0/3433017762 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f5114001a20 con 0x7f511c0b53a0 2024-09-19T08:39:37.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.035+0000 7f5121404640 1 -- 172.21.15.98:0/3433017762 >> v1:172.21.15.98:6789/0 conn(0x7f511c0b53a0 legacy=0x7f511c06f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:37.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.035+0000 7f5121404640 1 --2- 172.21.15.98:0/3433017762 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51080039c0 0x7f5108003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:37.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.035+0000 7f5121404640 1 --2- 172.21.15.98:0/3433017762 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5108004810 0x7f5108004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:37.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.035+0000 7f5121404640 1 --2- 172.21.15.98:0/3433017762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51080051d0 0x7f510800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:37.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.035+0000 7f5121404640 1 -- 172.21.15.98:0/3433017762 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f511c070ec0 con 0x7f51080039c0 2024-09-19T08:39:37.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.035+0000 7f5121404640 1 -- 172.21.15.98:0/3433017762 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f510800c070 con 0x7f51080051d0 2024-09-19T08:39:37.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.035+0000 7f5121404640 1 -- 172.21.15.98:0/3433017762 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f510800c270 con 0x7f5108004810 2024-09-19T08:39:37.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.035+0000 7f5122406640 1 --2- 172.21.15.98:0/3433017762 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51080039c0 0x7f5108003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:37.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.035+0000 7f5122c07640 1 --2- 172.21.15.98:0/3433017762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51080051d0 0x7f510800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:37.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.035+0000 7f5121c05640 1 --2- 172.21.15.98:0/3433017762 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5108004810 0x7f5108004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:37.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.035+0000 7f5122406640 1 -- 172.21.15.98:0/3433017762 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5108004810 msgr2=0x7f5108004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:37.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.035+0000 7f5122406640 1 --2- 172.21.15.98:0/3433017762 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5108004810 0x7f5108004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:37.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.035+0000 7f5122406640 1 -- 172.21.15.98:0/3433017762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51080051d0 msgr2=0x7f510800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:37.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.035+0000 7f5122406640 1 --2- 172.21.15.98:0/3433017762 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51080051d0 0x7f510800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:37.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.035+0000 7f5122c07640 1 -- 172.21.15.98:0/3433017762 reap_dead start 2024-09-19T08:39:37.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.035+0000 7f5122406640 1 -- 172.21.15.98:0/3433017762 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5108003530 con 0x7f51080039c0 2024-09-19T08:39:37.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.035+0000 7f5121c05640 1 --2- 172.21.15.98:0/3433017762 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5108004810 0x7f5108004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:37.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.035+0000 7f5122406640 1 --2- 172.21.15.98:0/3433017762 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51080039c0 0x7f5108003de0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5118009ae0 tx=0x7f511801ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffcf1f4c6ee70a25 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:37.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.035+0000 7f5121404640 1 -- 172.21.15.98:0/3433017762 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51180b2070 con 0x7f51080039c0 2024-09-19T08:39:37.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.035+0000 7f5121404640 1 -- 172.21.15.98:0/3433017762 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51180a5520 con 0x7f51080039c0 2024-09-19T08:39:37.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.035+0000 7f5121404640 1 -- 172.21.15.98:0/3433017762 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51180096b0 con 0x7f51080039c0 2024-09-19T08:39:37.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.039+0000 7f512466a640 1 -- 172.21.15.98:0/3433017762 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51080039c0 msgr2=0x7f5108003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:37.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.039+0000 7f512466a640 1 --2- 172.21.15.98:0/3433017762 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51080039c0 0x7f5108003de0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5118009ae0 tx=0x7f511801ca40 comp rx=0 tx=0).stop 2024-09-19T08:39:37.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.039+0000 7f512466a640 1 -- 172.21.15.98:0/3433017762 shutdown_connections 2024-09-19T08:39:37.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.039+0000 7f512466a640 1 --2- 172.21.15.98:0/3433017762 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51080039c0 0x7f5108003de0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:37.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.039+0000 7f512466a640 1 -- 172.21.15.98:0/3433017762 >> 172.21.15.98:0/3433017762 conn(0x7f511c0d23d0 msgr2=0x7f511c0d27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:37.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.039+0000 7f512466a640 1 -- 172.21.15.98:0/3433017762 shutdown_connections 2024-09-19T08:39:37.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.039+0000 7f512466a640 1 -- 172.21.15.98:0/3433017762 wait complete. 2024-09-19T08:39:37.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.039+0000 7f512466a640 1 Processor -- start 2024-09-19T08:39:37.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.039+0000 7f512466a640 1 -- start start 2024-09-19T08:39:37.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.039+0000 7f512466a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f51080039c0 0x7f511c152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:37.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.039+0000 7f512466a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f511c151f60 0x7f511c14e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:37.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.039+0000 7f512466a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f511c1ea090 0x7f511c1ea510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:37.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.039+0000 7f512466a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f511c0cdee0 con 0x7f511c151f60 2024-09-19T08:39:37.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.039+0000 7f512466a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f511c0cec10 con 0x7f511c1ea090 2024-09-19T08:39:37.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.039+0000 7f512466a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f511c06f380 con 0x7f51080039c0 2024-09-19T08:39:37.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.039+0000 7f5121c05640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f511c151f60 0x7f511c14e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:37.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.039+0000 7f5121c05640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f511c151f60 0x7f511c14e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48646/0 (socket says 172.21.15.98:48646) 2024-09-19T08:39:37.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.039+0000 7f5121c05640 1 -- 172.21.15.98:0/2292646552 learned_addr learned my addr 172.21.15.98:0/2292646552 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:37.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.039+0000 7f5122406640 1 --2- 172.21.15.98:0/2292646552 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f51080039c0 0x7f511c152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:37.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.039+0000 7f5122c07640 1 --2- 172.21.15.98:0/2292646552 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f511c1ea090 0x7f511c1ea510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:37.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.039+0000 7f5121c05640 1 -- 172.21.15.98:0/2292646552 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f51080039c0 msgr2=0x7f511c152d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:37.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.039+0000 7f5121c05640 1 --2- 172.21.15.98:0/2292646552 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f51080039c0 0x7f511c152d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:37.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.039+0000 7f5121c05640 1 -- 172.21.15.98:0/2292646552 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f511c1ea090 msgr2=0x7f511c1ea510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:37.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.039+0000 7f5121c05640 1 --2- 172.21.15.98:0/2292646552 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f511c1ea090 0x7f511c1ea510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:37.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.039+0000 7f5121c05640 1 -- 172.21.15.98:0/2292646552 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f511c14e890 con 0x7f511c151f60 2024-09-19T08:39:37.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.039+0000 7f5121c05640 1 --2- 172.21.15.98:0/2292646552 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f511c151f60 0x7f511c14e060 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f510c00d500 tx=0x7f510c00d9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:37.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.039+0000 7f5112ffd640 1 -- 172.21.15.98:0/2292646552 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510c019070 con 0x7f511c151f60 2024-09-19T08:39:37.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.039+0000 7f5112ffd640 1 -- 172.21.15.98:0/2292646552 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f510c007030 con 0x7f511c151f60 2024-09-19T08:39:37.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.039+0000 7f5112ffd640 1 -- 172.21.15.98:0/2292646552 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510c007420 con 0x7f511c151f60 2024-09-19T08:39:37.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.039+0000 7f512466a640 1 -- 172.21.15.98:0/2292646552 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f511c14eb70 con 0x7f511c151f60 2024-09-19T08:39:37.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.039+0000 7f512466a640 1 -- 172.21.15.98:0/2292646552 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f511c14f130 con 0x7f511c151f60 2024-09-19T08:39:37.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.043+0000 7f512466a640 1 -- 172.21.15.98:0/2292646552 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f511c06f710 con 0x7f511c151f60 2024-09-19T08:39:37.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.043+0000 7f5112ffd640 1 -- 172.21.15.98:0/2292646552 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f510c0067e0 con 0x7f511c151f60 2024-09-19T08:39:37.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.043+0000 7f5112ffd640 1 --2- 172.21.15.98:0/2292646552 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5100067a70 0x7f5100069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:37.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.043+0000 7f5112ffd640 1 -- 172.21.15.98:0/2292646552 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5227+0+0 (secure 0 0 0) 0x7f510c085470 con 0x7f511c151f60 2024-09-19T08:39:37.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.043+0000 7f5122406640 1 --2- 172.21.15.98:0/2292646552 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5100067a70 0x7f5100069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:37.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.043+0000 7f5122406640 1 --2- 172.21.15.98:0/2292646552 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5100067a70 0x7f5100069f30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f51180af000 tx=0x7f5118008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:37.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.047+0000 7f5112ffd640 1 -- 172.21.15.98:0/2292646552 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f510c055f50 con 0x7f511c151f60 2024-09-19T08:39:37.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.195+0000 7f512466a640 1 -- 172.21.15.98:0/2292646552 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f511c1e7c80 con 0x7f511c151f60 2024-09-19T08:39:37.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.887+0000 7f5112ffd640 1 -- 172.21.15.98:0/2292646552 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v41) ==== 170+0+0 (secure 0 0 0) 0x7f510c055850 con 0x7f511c151f60 2024-09-19T08:39:37.886 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' created 2024-09-19T08:39:37.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.891+0000 7f512466a640 1 -- 172.21.15.98:0/2292646552 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5100067a70 msgr2=0x7f5100069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:37.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.891+0000 7f512466a640 1 --2- 172.21.15.98:0/2292646552 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5100067a70 0x7f5100069f30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f51180af000 tx=0x7f5118008040 comp rx=0 tx=0).stop 2024-09-19T08:39:37.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.891+0000 7f512466a640 1 -- 172.21.15.98:0/2292646552 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f511c151f60 msgr2=0x7f511c14e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:37.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.891+0000 7f512466a640 1 --2- 172.21.15.98:0/2292646552 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f511c151f60 0x7f511c14e060 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f510c00d500 tx=0x7f510c00d9c0 comp rx=0 tx=0).stop 2024-09-19T08:39:37.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.891+0000 7f512466a640 1 -- 172.21.15.98:0/2292646552 shutdown_connections 2024-09-19T08:39:37.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.891+0000 7f512466a640 1 --2- 172.21.15.98:0/2292646552 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5100067a70 0x7f5100069f30 secure :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f51180af000 tx=0x7f5118008040 comp rx=0 tx=0).stop 2024-09-19T08:39:37.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.891+0000 7f512466a640 1 --2- 172.21.15.98:0/2292646552 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f511c1ea090 0x7f511c1ea510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:37.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.891+0000 7f512466a640 1 --2- 172.21.15.98:0/2292646552 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f511c151f60 0x7f511c14e060 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:37.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.891+0000 7f512466a640 1 --2- 172.21.15.98:0/2292646552 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f51080039c0 0x7f511c152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:37.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.891+0000 7f512466a640 1 -- 172.21.15.98:0/2292646552 >> 172.21.15.98:0/2292646552 conn(0x7f511c0d23d0 msgr2=0x7f511c15a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:37.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.891+0000 7f512466a640 1 -- 172.21.15.98:0/2292646552 shutdown_connections 2024-09-19T08:39:37.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:37.891+0000 7f512466a640 1 -- 172.21.15.98:0/2292646552 wait complete. 2024-09-19T08:39:37.922 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T08:39:38.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.047+0000 7f898bc34640 1 Processor -- start 2024-09-19T08:39:38.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.047+0000 7f898bc34640 1 -- start start 2024-09-19T08:39:38.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.047+0000 7f898bc34640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f89840b4420 0x7f8984071ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:38.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.047+0000 7f898bc34640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8984073530 con 0x7f898406c3a0 2024-09-19T08:39:38.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.047+0000 7f898bc34640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8984073810 con 0x7f8984145b70 2024-09-19T08:39:38.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.047+0000 7f898bc34640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8984073af0 con 0x7f89840b4420 2024-09-19T08:39:38.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.047+0000 7f89891cf640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f898406c3a0 0x7f89840713b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:44688/0 (socket says 172.21.15.98:44688) 2024-09-19T08:39:38.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.047+0000 7f89891cf640 1 -- 172.21.15.98:0/98683043 learned_addr learned my addr 172.21.15.98:0/98683043 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:38.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.047+0000 7f898a1d1640 1 --2- 172.21.15.98:0/98683043 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f89840b4420 0x7f8984071ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:38.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.047+0000 7f89889ce640 1 -- 172.21.15.98:0/98683043 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2308180106 0 0) 0x7f8984073530 con 0x7f898406c3a0 2024-09-19T08:39:38.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.047+0000 7f89889ce640 1 -- 172.21.15.98:0/98683043 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8970003530 con 0x7f898406c3a0 2024-09-19T08:39:38.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.047+0000 7f89889ce640 1 -- 172.21.15.98:0/98683043 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2689831642 0 0) 0x7f8970003530 con 0x7f898406c3a0 2024-09-19T08:39:38.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.047+0000 7f89889ce640 1 -- 172.21.15.98:0/98683043 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f89840b4420 msgr2=0x7f8984071ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:38.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.047+0000 7f89889ce640 1 --2- 172.21.15.98:0/98683043 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f89840b4420 0x7f8984071ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:38.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.047+0000 7f89889ce640 1 -- 172.21.15.98:0/98683043 >> v1:172.21.15.139:6789/0 conn(0x7f8984145b70 legacy=0x7f898406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:38.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.047+0000 7f89889ce640 1 -- 172.21.15.98:0/98683043 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8984073dd0 con 0x7f898406c3a0 2024-09-19T08:39:38.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.047+0000 7f89889ce640 1 -- 172.21.15.98:0/98683043 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f8980001680 con 0x7f898406c3a0 2024-09-19T08:39:38.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.047+0000 7f89889ce640 1 -- 172.21.15.98:0/98683043 >> v1:172.21.15.98:6789/0 conn(0x7f898406c3a0 legacy=0x7f89840713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:38.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.047+0000 7f89889ce640 1 --2- 172.21.15.98:0/98683043 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89700039f0 0x7f8970003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:38.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.047+0000 7f89889ce640 1 --2- 172.21.15.98:0/98683043 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f89840b4420 0x7f8970004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:38.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.047+0000 7f89889ce640 1 --2- 172.21.15.98:0/98683043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8970004d50 0x7f89700051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:38.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.047+0000 7f89889ce640 1 -- 172.21.15.98:0/98683043 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8984073af0 con 0x7f8970004d50 2024-09-19T08:39:38.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.047+0000 7f89889ce640 1 -- 172.21.15.98:0/98683043 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f897000bb00 con 0x7f89700039f0 2024-09-19T08:39:38.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.047+0000 7f89889ce640 1 -- 172.21.15.98:0/98683043 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f897000bd00 con 0x7f89840b4420 2024-09-19T08:39:38.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.047+0000 7f898a1d1640 1 --2- 172.21.15.98:0/98683043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8970004d50 0x7f89700051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:38.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.047+0000 7f89899d0640 1 --2- 172.21.15.98:0/98683043 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89700039f0 0x7f8970003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:38.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.047+0000 7f89891cf640 1 --2- 172.21.15.98:0/98683043 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f89840b4420 0x7f8970004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:38.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.047+0000 7f898a1d1640 1 -- 172.21.15.98:0/98683043 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f89840b4420 msgr2=0x7f8970004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:38.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.047+0000 7f898a1d1640 1 --2- 172.21.15.98:0/98683043 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f89840b4420 0x7f8970004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:38.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.047+0000 7f898a1d1640 1 -- 172.21.15.98:0/98683043 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89700039f0 msgr2=0x7f8970003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:38.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.047+0000 7f898a1d1640 1 --2- 172.21.15.98:0/98683043 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89700039f0 0x7f8970003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:38.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.047+0000 7f898a1d1640 1 -- 172.21.15.98:0/98683043 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8970003530 con 0x7f8970004d50 2024-09-19T08:39:38.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.051+0000 7f898a1d1640 1 --2- 172.21.15.98:0/98683043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8970004d50 0x7f89700051d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f897c009fc0 tx=0x7f897c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf9c071cdadaefb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:38.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.051+0000 7f89889ce640 1 -- 172.21.15.98:0/98683043 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c0b2070 con 0x7f8970004d50 2024-09-19T08:39:38.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.051+0000 7f89889ce640 1 -- 172.21.15.98:0/98683043 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f897c0a5b80 con 0x7f8970004d50 2024-09-19T08:39:38.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.051+0000 7f89889ce640 1 -- 172.21.15.98:0/98683043 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c009c10 con 0x7f8970004d50 2024-09-19T08:39:38.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.051+0000 7f898bc34640 1 -- 172.21.15.98:0/98683043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8970004d50 msgr2=0x7f89700051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:38.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.051+0000 7f898bc34640 1 --2- 172.21.15.98:0/98683043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8970004d50 0x7f89700051d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f897c009fc0 tx=0x7f897c01cc80 comp rx=0 tx=0).stop 2024-09-19T08:39:38.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.051+0000 7f898a1d1640 1 -- 172.21.15.98:0/98683043 reap_dead start 2024-09-19T08:39:38.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.051+0000 7f898bc34640 1 -- 172.21.15.98:0/98683043 shutdown_connections 2024-09-19T08:39:38.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.051+0000 7f898bc34640 1 -- 172.21.15.98:0/98683043 >> 172.21.15.98:0/98683043 conn(0x7f89840b2010 msgr2=0x7f89840b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:38.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.051+0000 7f898bc34640 1 -- 172.21.15.98:0/98683043 shutdown_connections 2024-09-19T08:39:38.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.051+0000 7f898bc34640 1 -- 172.21.15.98:0/98683043 wait complete. 2024-09-19T08:39:38.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.051+0000 7f898bc34640 1 Processor -- start 2024-09-19T08:39:38.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.051+0000 7f898bc34640 1 -- start start 2024-09-19T08:39:38.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.055+0000 7f898bc34640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f898415ac80 0x7f8984140cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:38.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.055+0000 7f898bc34640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8984141260 0x7f89840ad0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:38.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.055+0000 7f898bc34640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89840b3310 0x7f89840b3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:38.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.055+0000 7f898bc34640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8984071c40 con 0x7f898415ac80 2024-09-19T08:39:38.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.055+0000 7f898bc34640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8984071ac0 con 0x7f89840b3310 2024-09-19T08:39:38.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.055+0000 7f898bc34640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8984071dc0 con 0x7f8984141260 2024-09-19T08:39:38.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.055+0000 7f89899d0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f898415ac80 0x7f8984140cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:38.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.055+0000 7f89899d0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f898415ac80 0x7f8984140cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49454/0 (socket says 172.21.15.98:49454) 2024-09-19T08:39:38.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.055+0000 7f89899d0640 1 -- 172.21.15.98:0/2732806559 learned_addr learned my addr 172.21.15.98:0/2732806559 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:38.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.055+0000 7f89891cf640 1 --2- 172.21.15.98:0/2732806559 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8984141260 0x7f89840ad0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:38.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.055+0000 7f898a1d1640 1 --2- 172.21.15.98:0/2732806559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89840b3310 0x7f89840b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:38.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.055+0000 7f89899d0640 1 -- 172.21.15.98:0/2732806559 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8984141260 msgr2=0x7f89840ad0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:38.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.055+0000 7f89899d0640 1 --2- 172.21.15.98:0/2732806559 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8984141260 0x7f89840ad0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:38.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.055+0000 7f89899d0640 1 -- 172.21.15.98:0/2732806559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89840b3310 msgr2=0x7f89840b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:38.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.055+0000 7f89899d0640 1 --2- 172.21.15.98:0/2732806559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89840b3310 0x7f89840b3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:38.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.055+0000 7f89899d0640 1 -- 172.21.15.98:0/2732806559 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89840cd9b0 con 0x7f898415ac80 2024-09-19T08:39:38.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.055+0000 7f89891cf640 1 --2- 172.21.15.98:0/2732806559 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8984141260 0x7f89840ad0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:38.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.055+0000 7f89899d0640 1 --2- 172.21.15.98:0/2732806559 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f898415ac80 0x7f8984140cb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f897400f7f0 tx=0x7f897400fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:38.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.055+0000 7f897a7fc640 1 -- 172.21.15.98:0/2732806559 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89740050d0 con 0x7f898415ac80 2024-09-19T08:39:38.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.055+0000 7f897a7fc640 1 -- 172.21.15.98:0/2732806559 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8974018070 con 0x7f898415ac80 2024-09-19T08:39:38.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.055+0000 7f898bc34640 1 -- 172.21.15.98:0/2732806559 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89840cdc90 con 0x7f898415ac80 2024-09-19T08:39:38.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.055+0000 7f897a7fc640 1 -- 172.21.15.98:0/2732806559 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897400d730 con 0x7f898415ac80 2024-09-19T08:39:38.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.055+0000 7f898bc34640 1 -- 172.21.15.98:0/2732806559 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89840ce250 con 0x7f898415ac80 2024-09-19T08:39:38.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.055+0000 7f898bc34640 1 -- 172.21.15.98:0/2732806559 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89840713b0 con 0x7f898415ac80 2024-09-19T08:39:38.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.055+0000 7f897a7fc640 1 -- 172.21.15.98:0/2732806559 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f897400d8d0 con 0x7f898415ac80 2024-09-19T08:39:38.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.055+0000 7f897a7fc640 1 --2- 172.21.15.98:0/2732806559 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8968067aa0 0x7f8968069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:38.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.055+0000 7f897a7fc640 1 -- 172.21.15.98:0/2732806559 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5609+0+0 (secure 0 0 0) 0x7f89740867c0 con 0x7f898415ac80 2024-09-19T08:39:38.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.055+0000 7f89891cf640 1 --2- 172.21.15.98:0/2732806559 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8968067aa0 0x7f8968069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:38.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.055+0000 7f89891cf640 1 --2- 172.21.15.98:0/2732806559 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8968067aa0 0x7f8968069f60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f89800017f0 tx=0x7f8980001160 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:38.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.059+0000 7f897a7fc640 1 -- 172.21.15.98:0/2732806559 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f8974057250 con 0x7f898415ac80 2024-09-19T08:39:38.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.259+0000 7f898bc34640 1 -- 172.21.15.98:0/2732806559 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f898413c220 con 0x7f898415ac80 2024-09-19T08:39:38.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.923+0000 7f897a7fc640 1 -- 172.21.15.98:0/2732806559 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v20) ==== 172+0+0 (secure 0 0 0) 0x7f8974056b50 con 0x7f898415ac80 2024-09-19T08:39:38.927 INFO:teuthology.orchestra.run.smithi098.stderr:new fs with metadata pool 6 and data pool 7 2024-09-19T08:39:38.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.939+0000 7f898bc34640 1 -- 172.21.15.98:0/2732806559 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8968067aa0 msgr2=0x7f8968069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:38.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.939+0000 7f898bc34640 1 --2- 172.21.15.98:0/2732806559 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8968067aa0 0x7f8968069f60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f89800017f0 tx=0x7f8980001160 comp rx=0 tx=0).stop 2024-09-19T08:39:38.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.939+0000 7f898bc34640 1 -- 172.21.15.98:0/2732806559 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f898415ac80 msgr2=0x7f8984140cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:38.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.939+0000 7f898bc34640 1 --2- 172.21.15.98:0/2732806559 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f898415ac80 0x7f8984140cb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f897400f7f0 tx=0x7f897400fcb0 comp rx=0 tx=0).stop 2024-09-19T08:39:38.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.939+0000 7f898bc34640 1 -- 172.21.15.98:0/2732806559 shutdown_connections 2024-09-19T08:39:38.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.939+0000 7f898bc34640 1 --2- 172.21.15.98:0/2732806559 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8968067aa0 0x7f8968069f60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:38.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.939+0000 7f898bc34640 1 --2- 172.21.15.98:0/2732806559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f89840b3310 0x7f89840b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:38.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.939+0000 7f898bc34640 1 --2- 172.21.15.98:0/2732806559 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8984141260 0x7f89840ad0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:38.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.939+0000 7f898bc34640 1 --2- 172.21.15.98:0/2732806559 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f898415ac80 0x7f8984140cb0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:38.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.939+0000 7f898bc34640 1 -- 172.21.15.98:0/2732806559 >> 172.21.15.98:0/2732806559 conn(0x7f89840b2010 msgr2=0x7f89841434a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:38.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.939+0000 7f898bc34640 1 -- 172.21.15.98:0/2732806559 shutdown_connections 2024-09-19T08:39:38.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:38.939+0000 7f898bc34640 1 -- 172.21.15.98:0/2732806559 wait complete. 2024-09-19T08:39:38.966 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:39:39.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.079+0000 7f91d96eb640 1 Processor -- start 2024-09-19T08:39:39.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.079+0000 7f91d96eb640 1 -- start start 2024-09-19T08:39:39.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.083+0000 7f91d96eb640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f91d40733b0 0x7f91d40cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:39.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.083+0000 7f91d96eb640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91d406d310 con 0x7f91d406c3a0 2024-09-19T08:39:39.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.083+0000 7f91d96eb640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91d406d5f0 con 0x7f91d415c550 2024-09-19T08:39:39.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.083+0000 7f91d96eb640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f91d406d8d0 con 0x7f91d40733b0 2024-09-19T08:39:39.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.083+0000 7f91cbfff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f91d406c3a0 0x7f91d40cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:44690/0 (socket says 172.21.15.98:44690) 2024-09-19T08:39:39.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.083+0000 7f91cbfff640 1 -- 172.21.15.98:0/3956305514 learned_addr learned my addr 172.21.15.98:0/3956305514 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:39.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.083+0000 7f91d37fe640 1 --2- 172.21.15.98:0/3956305514 >> v2:172.21.15.139:3300/0 conn(0x7f91d40733b0 0x7f91d40cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:39.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.083+0000 7f91d27fc640 1 -- 172.21.15.98:0/3956305514 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3123612670 0 0) 0x7f91d406d310 con 0x7f91d406c3a0 2024-09-19T08:39:39.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.083+0000 7f91d27fc640 1 -- 172.21.15.98:0/3956305514 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91b4003530 con 0x7f91d406c3a0 2024-09-19T08:39:39.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.083+0000 7f91d27fc640 1 -- 172.21.15.98:0/3956305514 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3991077283 0 0) 0x7f91b4003530 con 0x7f91d406c3a0 2024-09-19T08:39:39.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.083+0000 7f91d27fc640 1 -- 172.21.15.98:0/3956305514 >> v2:172.21.15.139:3300/0 conn(0x7f91d40733b0 msgr2=0x7f91d40cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:39.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.083+0000 7f91d27fc640 1 --2- 172.21.15.98:0/3956305514 >> v2:172.21.15.139:3300/0 conn(0x7f91d40733b0 0x7f91d40cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:39.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.083+0000 7f91d27fc640 1 -- 172.21.15.98:0/3956305514 >> v1:172.21.15.139:6789/0 conn(0x7f91d415c550 legacy=0x7f91d406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:39.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.083+0000 7f91d27fc640 1 -- 172.21.15.98:0/3956305514 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d406dbb0 con 0x7f91d406c3a0 2024-09-19T08:39:39.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.083+0000 7f91d27fc640 1 -- 172.21.15.98:0/3956305514 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f91bc001680 con 0x7f91d406c3a0 2024-09-19T08:39:39.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.083+0000 7f91d27fc640 1 -- 172.21.15.98:0/3956305514 >> v1:172.21.15.98:6789/0 conn(0x7f91d406c3a0 legacy=0x7f91d40cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:39.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.083+0000 7f91d27fc640 1 --2- 172.21.15.98:0/3956305514 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91b40039c0 0x7f91b4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:39.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.083+0000 7f91d27fc640 1 --2- 172.21.15.98:0/3956305514 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91b4004810 0x7f91b4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:39.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.083+0000 7f91d27fc640 1 --2- 172.21.15.98:0/3956305514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91b40051d0 0x7f91b400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:39.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.083+0000 7f91d27fc640 1 -- 172.21.15.98:0/3956305514 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f91d406d8d0 con 0x7f91b40051d0 2024-09-19T08:39:39.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.083+0000 7f91d27fc640 1 -- 172.21.15.98:0/3956305514 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f91b400c070 con 0x7f91b4004810 2024-09-19T08:39:39.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.083+0000 7f91d27fc640 1 -- 172.21.15.98:0/3956305514 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f91b400c270 con 0x7f91b40039c0 2024-09-19T08:39:39.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.083+0000 7f91d37fe640 1 --2- 172.21.15.98:0/3956305514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91b40051d0 0x7f91b400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:39.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.083+0000 7f91cbfff640 1 --2- 172.21.15.98:0/3956305514 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91b4004810 0x7f91b4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:39.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.083+0000 7f91d37fe640 1 -- 172.21.15.98:0/3956305514 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91b40039c0 msgr2=0x7f91b4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:39.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.083+0000 7f91d37fe640 1 --2- 172.21.15.98:0/3956305514 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91b40039c0 0x7f91b4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:39.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.083+0000 7f91d37fe640 1 -- 172.21.15.98:0/3956305514 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91b4004810 msgr2=0x7f91b4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:39.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.083+0000 7f91d37fe640 1 --2- 172.21.15.98:0/3956305514 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91b4004810 0x7f91b4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:39.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.083+0000 7f91d37fe640 1 -- 172.21.15.98:0/3956305514 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b4003530 con 0x7f91b40051d0 2024-09-19T08:39:39.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.083+0000 7f91cbfff640 1 --2- 172.21.15.98:0/3956305514 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91b4004810 0x7f91b4004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:39.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.087+0000 7f91d37fe640 1 -- 172.21.15.98:0/3956305514 reap_dead start 2024-09-19T08:39:39.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.087+0000 7f91d37fe640 1 --2- 172.21.15.98:0/3956305514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91b40051d0 0x7f91b400b9d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f91cc009fc0 tx=0x7f91cc01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5e3098841c2f4f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:39.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.087+0000 7f91d27fc640 1 -- 172.21.15.98:0/3956305514 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91cc009c10 con 0x7f91b40051d0 2024-09-19T08:39:39.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.087+0000 7f91d27fc640 1 -- 172.21.15.98:0/3956305514 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91cc0a5d10 con 0x7f91b40051d0 2024-09-19T08:39:39.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.087+0000 7f91d27fc640 1 -- 172.21.15.98:0/3956305514 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91cc003be0 con 0x7f91b40051d0 2024-09-19T08:39:39.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.087+0000 7f91d96eb640 1 -- 172.21.15.98:0/3956305514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91b40051d0 msgr2=0x7f91b400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:39.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.087+0000 7f91d96eb640 1 --2- 172.21.15.98:0/3956305514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91b40051d0 0x7f91b400b9d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f91cc009fc0 tx=0x7f91cc01ccc0 comp rx=0 tx=0).stop 2024-09-19T08:39:39.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.087+0000 7f91d96eb640 1 -- 172.21.15.98:0/3956305514 shutdown_connections 2024-09-19T08:39:39.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.087+0000 7f91d96eb640 1 --2- 172.21.15.98:0/3956305514 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91b40051d0 0x7f91b400b9d0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:39.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.087+0000 7f91d96eb640 1 -- 172.21.15.98:0/3956305514 >> 172.21.15.98:0/3956305514 conn(0x7f91d40b2010 msgr2=0x7f91d40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:39.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.087+0000 7f91d96eb640 1 -- 172.21.15.98:0/3956305514 shutdown_connections 2024-09-19T08:39:39.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.087+0000 7f91d96eb640 1 -- 172.21.15.98:0/3956305514 wait complete. 2024-09-19T08:39:39.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.087+0000 7f91d96eb640 1 Processor -- start 2024-09-19T08:39:39.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.087+0000 7f91d96eb640 1 -- start start 2024-09-19T08:39:39.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.087+0000 7f91d96eb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91b40051d0 0x7f91d415ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:39.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.087+0000 7f91d96eb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91d4155170 0x7f91d4145bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:39.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.087+0000 7f91d96eb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91d40b1000 0x7f91d40b1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:39.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.087+0000 7f91d96eb640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f91d40d0050 con 0x7f91d4155170 2024-09-19T08:39:39.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.087+0000 7f91d96eb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f91d415cb40 con 0x7f91b40051d0 2024-09-19T08:39:39.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.087+0000 7f91d96eb640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f91d406bc90 con 0x7f91d40b1000 2024-09-19T08:39:39.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.087+0000 7f91cbfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91d4155170 0x7f91d4145bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:39.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.087+0000 7f91cbfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91d4155170 0x7f91d4145bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49462/0 (socket says 172.21.15.98:49462) 2024-09-19T08:39:39.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.087+0000 7f91cbfff640 1 -- 172.21.15.98:0/2258945181 learned_addr learned my addr 172.21.15.98:0/2258945181 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:39.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.087+0000 7f91d37fe640 1 --2- 172.21.15.98:0/2258945181 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91d40b1000 0x7f91d40b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:39.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.087+0000 7f91d2ffd640 1 --2- 172.21.15.98:0/2258945181 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91b40051d0 0x7f91d415ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:39.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.087+0000 7f91cbfff640 1 -- 172.21.15.98:0/2258945181 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91d40b1000 msgr2=0x7f91d40b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:39.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.087+0000 7f91cbfff640 1 --2- 172.21.15.98:0/2258945181 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91d40b1000 0x7f91d40b1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:39.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.087+0000 7f91cbfff640 1 -- 172.21.15.98:0/2258945181 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91b40051d0 msgr2=0x7f91d415ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:39.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.087+0000 7f91cbfff640 1 --2- 172.21.15.98:0/2258945181 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91b40051d0 0x7f91d415ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:39.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.087+0000 7f91cbfff640 1 -- 172.21.15.98:0/2258945181 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d415d350 con 0x7f91d4155170 2024-09-19T08:39:39.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.087+0000 7f91cbfff640 1 --2- 172.21.15.98:0/2258945181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91d4155170 0x7f91d4145bc0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f91bc007fd0 tx=0x7f91bc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:39.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.091+0000 7f91cb7fe640 1 -- 172.21.15.98:0/2258945181 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc019070 con 0x7f91d4155170 2024-09-19T08:39:39.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.091+0000 7f91cb7fe640 1 -- 172.21.15.98:0/2258945181 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91bc0074e0 con 0x7f91d4155170 2024-09-19T08:39:39.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.091+0000 7f91d96eb640 1 -- 172.21.15.98:0/2258945181 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91d415d630 con 0x7f91d4155170 2024-09-19T08:39:39.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.091+0000 7f91cb7fe640 1 -- 172.21.15.98:0/2258945181 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc0049f0 con 0x7f91d4155170 2024-09-19T08:39:39.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.091+0000 7f91d96eb640 1 -- 172.21.15.98:0/2258945181 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91d415dbf0 con 0x7f91d4155170 2024-09-19T08:39:39.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.091+0000 7f91d96eb640 1 -- 172.21.15.98:0/2258945181 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91d406c020 con 0x7f91d4155170 2024-09-19T08:39:39.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.091+0000 7f91cb7fe640 1 -- 172.21.15.98:0/2258945181 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f91bc01c070 con 0x7f91d4155170 2024-09-19T08:39:39.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.091+0000 7f91cb7fe640 1 --2- 172.21.15.98:0/2258945181 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f91b8067aa0 0x7f91b8069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:39.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.091+0000 7f91cb7fe640 1 -- 172.21.15.98:0/2258945181 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5709+0+0 (secure 0 0 0) 0x7f91bc085a40 con 0x7f91d4155170 2024-09-19T08:39:39.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.091+0000 7f91d2ffd640 1 --2- 172.21.15.98:0/2258945181 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f91b8067aa0 0x7f91b8069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:39.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.091+0000 7f91d2ffd640 1 --2- 172.21.15.98:0/2258945181 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f91b8067aa0 0x7f91b8069f60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f91c4001670 tx=0x7f91c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:39.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.095+0000 7f91cb7fe640 1 -- 172.21.15.98:0/2258945181 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f91bc0561f0 con 0x7f91d4155170 2024-09-19T08:39:39.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.255+0000 7f91d96eb640 1 -- 172.21.15.98:0/2258945181 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f91d415a850 con 0x7f91d4155170 2024-09-19T08:39:39.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.259+0000 7f91cb7fe640 1 -- 172.21.15.98:0/2258945181 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+16686 (secure 0 0 0) 0x7f91bc085e20 con 0x7f91d4155170 2024-09-19T08:39:39.258 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:39:39.258 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":42,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:39:38.262210+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:39:36.829570+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-19T08:39:37.237199+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:39:39.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.259+0000 7f91d96eb640 1 -- 172.21.15.98:0/2258945181 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f91b8067aa0 msgr2=0x7f91b8069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:39.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.259+0000 7f91d96eb640 1 --2- 172.21.15.98:0/2258945181 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f91b8067aa0 0x7f91b8069f60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f91c4001670 tx=0x7f91c4009040 comp rx=0 tx=0).stop 2024-09-19T08:39:39.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.259+0000 7f91d96eb640 1 -- 172.21.15.98:0/2258945181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91d4155170 msgr2=0x7f91d4145bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:39.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.259+0000 7f91d96eb640 1 --2- 172.21.15.98:0/2258945181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91d4155170 0x7f91d4145bc0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f91bc007fd0 tx=0x7f91bc00dbe0 comp rx=0 tx=0).stop 2024-09-19T08:39:39.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.259+0000 7f91d96eb640 1 -- 172.21.15.98:0/2258945181 shutdown_connections 2024-09-19T08:39:39.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.259+0000 7f91d96eb640 1 --2- 172.21.15.98:0/2258945181 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f91b8067aa0 0x7f91b8069f60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:39.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.259+0000 7f91d96eb640 1 --2- 172.21.15.98:0/2258945181 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91d40b1000 0x7f91d40b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:39.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.259+0000 7f91d96eb640 1 --2- 172.21.15.98:0/2258945181 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91d4155170 0x7f91d4145bc0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:39.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.259+0000 7f91d96eb640 1 --2- 172.21.15.98:0/2258945181 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91b40051d0 0x7f91d415ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:39.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.259+0000 7f91d96eb640 1 -- 172.21.15.98:0/2258945181 >> 172.21.15.98:0/2258945181 conn(0x7f91d40b2010 msgr2=0x7f91d40cc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:39.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.259+0000 7f91d96eb640 1 -- 172.21.15.98:0/2258945181 shutdown_connections 2024-09-19T08:39:39.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.259+0000 7f91d96eb640 1 -- 172.21.15.98:0/2258945181 wait complete. 2024-09-19T08:39:39.278 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:39:39.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.391+0000 7f3283a5b640 1 Processor -- start 2024-09-19T08:39:39.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.391+0000 7f3283a5b640 1 -- start start 2024-09-19T08:39:39.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.395+0000 7f3283a5b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f327c06d4c0 0x7f327c06f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:39.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.395+0000 7f3283a5b640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f327c15c310 con 0x7f327c15bf30 2024-09-19T08:39:39.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.395+0000 7f3283a5b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f327c162310 con 0x7f327c06d950 2024-09-19T08:39:39.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.395+0000 7f3283a5b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f327c162e90 con 0x7f327c06d4c0 2024-09-19T08:39:39.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.395+0000 7f32817f7640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f327c15bf30 0x7f327c073170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:44698/0 (socket says 172.21.15.98:44698) 2024-09-19T08:39:39.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.395+0000 7f32817f7640 1 -- 172.21.15.98:0/3429769224 learned_addr learned my addr 172.21.15.98:0/3429769224 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:39.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.395+0000 7f3280ff6640 1 --2- 172.21.15.98:0/3429769224 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f327c06d4c0 0x7f327c06f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:39.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.395+0000 7f3273fff640 1 -- 172.21.15.98:0/3429769224 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3930342167 0 0) 0x7f327c15c310 con 0x7f327c15bf30 2024-09-19T08:39:39.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.395+0000 7f3273fff640 1 -- 172.21.15.98:0/3429769224 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3268003530 con 0x7f327c15bf30 2024-09-19T08:39:39.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.395+0000 7f3273fff640 1 -- 172.21.15.98:0/3429769224 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 960834128 0 0) 0x7f3268003530 con 0x7f327c15bf30 2024-09-19T08:39:39.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.395+0000 7f3273fff640 1 -- 172.21.15.98:0/3429769224 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f327c06d4c0 msgr2=0x7f327c06f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:39.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.395+0000 7f3273fff640 1 --2- 172.21.15.98:0/3429769224 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f327c06d4c0 0x7f327c06f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:39.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.395+0000 7f3273fff640 1 -- 172.21.15.98:0/3429769224 >> v1:172.21.15.139:6789/0 conn(0x7f327c06d950 legacy=0x7f327c15ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:39.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.395+0000 7f3273fff640 1 -- 172.21.15.98:0/3429769224 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f327c1625f0 con 0x7f327c15bf30 2024-09-19T08:39:39.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.395+0000 7f3273fff640 1 -- 172.21.15.98:0/3429769224 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f3278001680 con 0x7f327c15bf30 2024-09-19T08:39:39.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.395+0000 7f3273fff640 1 -- 172.21.15.98:0/3429769224 >> v1:172.21.15.98:6789/0 conn(0x7f327c15bf30 legacy=0x7f327c073170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:39.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.395+0000 7f3273fff640 1 --2- 172.21.15.98:0/3429769224 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f327c06d4c0 0x7f32680039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:39.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.395+0000 7f3273fff640 1 --2- 172.21.15.98:0/3429769224 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3268003f70 0x7f3268004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:39.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.395+0000 7f3273fff640 1 --2- 172.21.15.98:0/3429769224 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3268004d50 0x7f32680051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:39.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.395+0000 7f3273fff640 1 -- 172.21.15.98:0/3429769224 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f327c162e90 con 0x7f3268004d50 2024-09-19T08:39:39.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.395+0000 7f3273fff640 1 -- 172.21.15.98:0/3429769224 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f326800bab0 con 0x7f3268003f70 2024-09-19T08:39:39.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.395+0000 7f3273fff640 1 -- 172.21.15.98:0/3429769224 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f326800bcb0 con 0x7f327c06d4c0 2024-09-19T08:39:39.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.395+0000 7f3281ff8640 1 --2- 172.21.15.98:0/3429769224 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3268004d50 0x7f32680051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:39.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.395+0000 7f3280ff6640 1 --2- 172.21.15.98:0/3429769224 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3268003f70 0x7f3268004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:39.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.395+0000 7f32817f7640 1 --2- 172.21.15.98:0/3429769224 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f327c06d4c0 0x7f32680039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:39.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.395+0000 7f3281ff8640 1 -- 172.21.15.98:0/3429769224 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f327c06d4c0 msgr2=0x7f32680039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:39.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.395+0000 7f3281ff8640 1 --2- 172.21.15.98:0/3429769224 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f327c06d4c0 0x7f32680039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:39.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.395+0000 7f3281ff8640 1 -- 172.21.15.98:0/3429769224 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3268003f70 msgr2=0x7f3268004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:39.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.395+0000 7f3281ff8640 1 --2- 172.21.15.98:0/3429769224 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3268003f70 0x7f3268004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:39.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.395+0000 7f3281ff8640 1 -- 172.21.15.98:0/3429769224 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3268003530 con 0x7f3268004d50 2024-09-19T08:39:39.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.395+0000 7f32817f7640 1 --2- 172.21.15.98:0/3429769224 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f327c06d4c0 0x7f32680039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:39.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.395+0000 7f3280ff6640 1 --2- 172.21.15.98:0/3429769224 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3268003f70 0x7f3268004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:39:39.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.399+0000 7f3281ff8640 1 --2- 172.21.15.98:0/3429769224 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3268004d50 0x7f32680051d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f327c15c4e0 tx=0x7f327401c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739ded4cbc16be19 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:39.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.399+0000 7f3273fff640 1 -- 172.21.15.98:0/3429769224 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3274001aa0 con 0x7f3268004d50 2024-09-19T08:39:39.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.399+0000 7f3273fff640 1 -- 172.21.15.98:0/3429769224 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3274003b80 con 0x7f3268004d50 2024-09-19T08:39:39.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.399+0000 7f3273fff640 1 -- 172.21.15.98:0/3429769224 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3274005080 con 0x7f3268004d50 2024-09-19T08:39:39.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.399+0000 7f3283a5b640 1 -- 172.21.15.98:0/3429769224 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3268004d50 msgr2=0x7f32680051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:39.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.399+0000 7f3283a5b640 1 --2- 172.21.15.98:0/3429769224 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3268004d50 0x7f32680051d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f327c15c4e0 tx=0x7f327401c760 comp rx=0 tx=0).stop 2024-09-19T08:39:39.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.399+0000 7f3281ff8640 1 -- 172.21.15.98:0/3429769224 reap_dead start 2024-09-19T08:39:39.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.399+0000 7f3283a5b640 1 -- 172.21.15.98:0/3429769224 shutdown_connections 2024-09-19T08:39:39.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.399+0000 7f3283a5b640 1 -- 172.21.15.98:0/3429769224 >> 172.21.15.98:0/3429769224 conn(0x7f327c0b40a0 msgr2=0x7f327c0b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:39.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.399+0000 7f3283a5b640 1 -- 172.21.15.98:0/3429769224 shutdown_connections 2024-09-19T08:39:39.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.399+0000 7f3283a5b640 1 -- 172.21.15.98:0/3429769224 wait complete. 2024-09-19T08:39:39.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.399+0000 7f3283a5b640 1 Processor -- start 2024-09-19T08:39:39.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.399+0000 7f3283a5b640 1 -- start start 2024-09-19T08:39:39.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.399+0000 7f3283a5b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3268004d50 0x7f327c148e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:39.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.399+0000 7f3283a5b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f327c0af270 0x7f327c0b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:39.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.399+0000 7f3283a5b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f327c15d360 0x7f327c15d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:39.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.399+0000 7f3283a5b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f327c161f70 con 0x7f3268004d50 2024-09-19T08:39:39.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.399+0000 7f3283a5b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f327c162bb0 con 0x7f327c15d360 2024-09-19T08:39:39.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.399+0000 7f3283a5b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f327c1620f0 con 0x7f327c0af270 2024-09-19T08:39:39.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.399+0000 7f32817f7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3268004d50 0x7f327c148e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:39.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.399+0000 7f32817f7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3268004d50 0x7f327c148e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49484/0 (socket says 172.21.15.98:49484) 2024-09-19T08:39:39.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.399+0000 7f32817f7640 1 -- 172.21.15.98:0/2766631376 learned_addr learned my addr 172.21.15.98:0/2766631376 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:39.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.399+0000 7f3281ff8640 1 --2- 172.21.15.98:0/2766631376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f327c15d360 0x7f327c15d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:39.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.399+0000 7f32817f7640 1 -- 172.21.15.98:0/2766631376 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f327c0af270 msgr2=0x7f327c0b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:39.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.399+0000 7f32817f7640 1 --2- 172.21.15.98:0/2766631376 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f327c0af270 0x7f327c0b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:39.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.399+0000 7f32817f7640 1 -- 172.21.15.98:0/2766631376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f327c15d360 msgr2=0x7f327c15d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:39.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.399+0000 7f32817f7640 1 --2- 172.21.15.98:0/2766631376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f327c15d360 0x7f327c15d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:39.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.399+0000 7f32817f7640 1 -- 172.21.15.98:0/2766631376 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f327c0b1890 con 0x7f3268004d50 2024-09-19T08:39:39.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.399+0000 7f3281ff8640 1 --2- 172.21.15.98:0/2766631376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f327c15d360 0x7f327c15d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:39.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.399+0000 7f32817f7640 1 --2- 172.21.15.98:0/2766631376 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3268004d50 0x7f327c148e00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f327800d8b0 tx=0x7f327800dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:39.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.399+0000 7f3271ffb640 1 -- 172.21.15.98:0/2766631376 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278019070 con 0x7f3268004d50 2024-09-19T08:39:39.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.399+0000 7f3271ffb640 1 -- 172.21.15.98:0/2766631376 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32780074e0 con 0x7f3268004d50 2024-09-19T08:39:39.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.399+0000 7f3283a5b640 1 -- 172.21.15.98:0/2766631376 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f327c0b0190 con 0x7f3268004d50 2024-09-19T08:39:39.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.399+0000 7f3271ffb640 1 -- 172.21.15.98:0/2766631376 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32780049f0 con 0x7f3268004d50 2024-09-19T08:39:39.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.403+0000 7f3283a5b640 1 -- 172.21.15.98:0/2766631376 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f327c0b0b50 con 0x7f3268004d50 2024-09-19T08:39:39.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.403+0000 7f3283a5b640 1 -- 172.21.15.98:0/2766631376 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f327c073170 con 0x7f3268004d50 2024-09-19T08:39:39.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.403+0000 7f3271ffb640 1 -- 172.21.15.98:0/2766631376 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f327801c070 con 0x7f3268004d50 2024-09-19T08:39:39.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.403+0000 7f3271ffb640 1 --2- 172.21.15.98:0/2766631376 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3260067a70 0x7f3260069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:39.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.403+0000 7f3271ffb640 1 -- 172.21.15.98:0/2766631376 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5709+0+0 (secure 0 0 0) 0x7f3278085cc0 con 0x7f3268004d50 2024-09-19T08:39:39.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.403+0000 7f3280ff6640 1 --2- 172.21.15.98:0/2766631376 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3260067a70 0x7f3260069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:39.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.403+0000 7f3280ff6640 1 --2- 172.21.15.98:0/2766631376 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3260067a70 0x7f3260069f30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f327c15e6e0 tx=0x7f326c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:39.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.407+0000 7f3271ffb640 1 -- 172.21.15.98:0/2766631376 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3278056500 con 0x7f3268004d50 2024-09-19T08:39:39.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.571+0000 7f3283a5b640 1 -- 172.21.15.98:0/2766631376 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f327c15b740 con 0x7f3268004d50 2024-09-19T08:39:39.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.575+0000 7f3271ffb640 1 -- 172.21.15.98:0/2766631376 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+16686 (secure 0 0 0) 0x7f3278055e00 con 0x7f3268004d50 2024-09-19T08:39:39.574 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:39:39.574 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":42,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:39:38.262210+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:39:36.829570+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-19T08:39:37.237199+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:39:39.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.575+0000 7f3283a5b640 1 -- 172.21.15.98:0/2766631376 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3260067a70 msgr2=0x7f3260069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:39.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.575+0000 7f3283a5b640 1 --2- 172.21.15.98:0/2766631376 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3260067a70 0x7f3260069f30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f327c15e6e0 tx=0x7f326c008040 comp rx=0 tx=0).stop 2024-09-19T08:39:39.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.575+0000 7f3283a5b640 1 -- 172.21.15.98:0/2766631376 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3268004d50 msgr2=0x7f327c148e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:39.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.575+0000 7f3283a5b640 1 --2- 172.21.15.98:0/2766631376 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3268004d50 0x7f327c148e00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f327800d8b0 tx=0x7f327800dd70 comp rx=0 tx=0).stop 2024-09-19T08:39:39.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.575+0000 7f3283a5b640 1 -- 172.21.15.98:0/2766631376 shutdown_connections 2024-09-19T08:39:39.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.575+0000 7f3283a5b640 1 --2- 172.21.15.98:0/2766631376 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3260067a70 0x7f3260069f30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:39.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.575+0000 7f3283a5b640 1 --2- 172.21.15.98:0/2766631376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f327c15d360 0x7f327c15d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:39.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.575+0000 7f3283a5b640 1 --2- 172.21.15.98:0/2766631376 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f327c0af270 0x7f327c0b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:39.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.575+0000 7f3283a5b640 1 --2- 172.21.15.98:0/2766631376 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3268004d50 0x7f327c148e00 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:39.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.575+0000 7f3283a5b640 1 -- 172.21.15.98:0/2766631376 >> 172.21.15.98:0/2766631376 conn(0x7f327c0b40a0 msgr2=0x7f327c06bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:39.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.575+0000 7f3283a5b640 1 -- 172.21.15.98:0/2766631376 shutdown_connections 2024-09-19T08:39:39.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.575+0000 7f3283a5b640 1 -- 172.21.15.98:0/2766631376 wait complete. 2024-09-19T08:39:39.594 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-19T08:39:39.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.715+0000 7fbdbb699640 1 Processor -- start 2024-09-19T08:39:39.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.715+0000 7fbdbb699640 1 -- start start 2024-09-19T08:39:39.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.719+0000 7fbdbb699640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbdb40d27c0 0x7fbdb40d3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:39.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.719+0000 7fbdbb699640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdb415eae0 con 0x7fbdb40d38a0 2024-09-19T08:39:39.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.719+0000 7fbdbb699640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdb415edc0 con 0x7fbdb41415a0 2024-09-19T08:39:39.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.719+0000 7fbdbb699640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fbdb415f0a0 con 0x7fbdb40d27c0 2024-09-19T08:39:39.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.719+0000 7fbdb9c36640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fbdb40d38a0 0x7fbdb40b8790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:44704/0 (socket says 172.21.15.98:44704) 2024-09-19T08:39:39.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.719+0000 7fbdb9c36640 1 -- 172.21.15.98:0/2940994352 learned_addr learned my addr 172.21.15.98:0/2940994352 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:39.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.719+0000 7fbdb8c34640 1 --2- 172.21.15.98:0/2940994352 >> v2:172.21.15.139:3300/0 conn(0x7fbdb40d27c0 0x7fbdb40d3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:39.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.719+0000 7fbdabfff640 1 -- 172.21.15.98:0/2940994352 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2063711062 0 0) 0x7fbdb415eae0 con 0x7fbdb40d38a0 2024-09-19T08:39:39.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.719+0000 7fbdabfff640 1 -- 172.21.15.98:0/2940994352 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbda0003530 con 0x7fbdb40d38a0 2024-09-19T08:39:39.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.719+0000 7fbdabfff640 1 -- 172.21.15.98:0/2940994352 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 180988414 0 0) 0x7fbda0003530 con 0x7fbdb40d38a0 2024-09-19T08:39:39.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.719+0000 7fbdabfff640 1 -- 172.21.15.98:0/2940994352 >> v2:172.21.15.139:3300/0 conn(0x7fbdb40d27c0 msgr2=0x7fbdb40d3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:39.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.719+0000 7fbdabfff640 1 --2- 172.21.15.98:0/2940994352 >> v2:172.21.15.139:3300/0 conn(0x7fbdb40d27c0 0x7fbdb40d3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:39.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.719+0000 7fbdabfff640 1 -- 172.21.15.98:0/2940994352 >> v1:172.21.15.139:6789/0 conn(0x7fbdb41415a0 legacy=0x7fbdb40d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:39.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.719+0000 7fbdabfff640 1 -- 172.21.15.98:0/2940994352 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb415f380 con 0x7fbdb40d38a0 2024-09-19T08:39:39.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.719+0000 7fbdabfff640 1 -- 172.21.15.98:0/2940994352 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fbdac001a00 con 0x7fbdb40d38a0 2024-09-19T08:39:39.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.719+0000 7fbdabfff640 1 -- 172.21.15.98:0/2940994352 >> v1:172.21.15.98:6789/0 conn(0x7fbdb40d38a0 legacy=0x7fbdb40b8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:39.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.719+0000 7fbdabfff640 1 --2- 172.21.15.98:0/2940994352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbda00039c0 0x7fbda0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:39.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.719+0000 7fbdabfff640 1 --2- 172.21.15.98:0/2940994352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbda0004810 0x7fbda0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:39.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.719+0000 7fbdabfff640 1 --2- 172.21.15.98:0/2940994352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbda00051d0 0x7fbda000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:39.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.719+0000 7fbdabfff640 1 -- 172.21.15.98:0/2940994352 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb415f0a0 con 0x7fbda00051d0 2024-09-19T08:39:39.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.719+0000 7fbdabfff640 1 -- 172.21.15.98:0/2940994352 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbda000c070 con 0x7fbda00039c0 2024-09-19T08:39:39.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.719+0000 7fbdabfff640 1 -- 172.21.15.98:0/2940994352 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbda000c270 con 0x7fbda0004810 2024-09-19T08:39:39.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.719+0000 7fbdb9c36640 1 --2- 172.21.15.98:0/2940994352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbda00051d0 0x7fbda000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:39.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.719+0000 7fbdb8c34640 1 --2- 172.21.15.98:0/2940994352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbda0004810 0x7fbda0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:39.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.719+0000 7fbdb9435640 1 --2- 172.21.15.98:0/2940994352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbda00039c0 0x7fbda0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:39.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.719+0000 7fbdb9c36640 1 -- 172.21.15.98:0/2940994352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbda0004810 msgr2=0x7fbda0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:39.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.719+0000 7fbdb9c36640 1 --2- 172.21.15.98:0/2940994352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbda0004810 0x7fbda0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:39.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.719+0000 7fbdb9c36640 1 -- 172.21.15.98:0/2940994352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbda00039c0 msgr2=0x7fbda0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:39.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.719+0000 7fbdb9c36640 1 --2- 172.21.15.98:0/2940994352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbda00039c0 0x7fbda0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:39.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.719+0000 7fbdb9c36640 1 -- 172.21.15.98:0/2940994352 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda0003530 con 0x7fbda00051d0 2024-09-19T08:39:39.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.719+0000 7fbdb9c36640 1 -- 172.21.15.98:0/2940994352 reap_dead start 2024-09-19T08:39:39.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.723+0000 7fbdb9c36640 1 --2- 172.21.15.98:0/2940994352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbda00051d0 0x7fbda000b9d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbdac00a860 tx=0x7fbdac01d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b63835b16fd91c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:39.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.723+0000 7fbdabfff640 1 -- 172.21.15.98:0/2940994352 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdac00a480 con 0x7fbda00051d0 2024-09-19T08:39:39.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.723+0000 7fbdabfff640 1 -- 172.21.15.98:0/2940994352 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdac0ad630 con 0x7fbda00051d0 2024-09-19T08:39:39.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.723+0000 7fbdabfff640 1 -- 172.21.15.98:0/2940994352 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdac0ada20 con 0x7fbda00051d0 2024-09-19T08:39:39.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.723+0000 7fbdbb699640 1 -- 172.21.15.98:0/2940994352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbda00051d0 msgr2=0x7fbda000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:39.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.723+0000 7fbdbb699640 1 --2- 172.21.15.98:0/2940994352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbda00051d0 0x7fbda000b9d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbdac00a860 tx=0x7fbdac01d380 comp rx=0 tx=0).stop 2024-09-19T08:39:39.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.723+0000 7fbdbb699640 1 -- 172.21.15.98:0/2940994352 shutdown_connections 2024-09-19T08:39:39.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.723+0000 7fbdbb699640 1 --2- 172.21.15.98:0/2940994352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbda00051d0 0x7fbda000b9d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:39.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.723+0000 7fbdbb699640 1 -- 172.21.15.98:0/2940994352 >> 172.21.15.98:0/2940994352 conn(0x7fbdb4074c70 msgr2=0x7fbdb4075080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:39.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.723+0000 7fbdbb699640 1 -- 172.21.15.98:0/2940994352 shutdown_connections 2024-09-19T08:39:39.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.723+0000 7fbdbb699640 1 -- 172.21.15.98:0/2940994352 wait complete. 2024-09-19T08:39:39.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.723+0000 7fbdbb699640 1 Processor -- start 2024-09-19T08:39:39.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.723+0000 7fbdbb699640 1 -- start start 2024-09-19T08:39:39.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.723+0000 7fbdbb699640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbda00051d0 0x7fbdb4154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:39.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.723+0000 7fbdbb699640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbdb4155eb0 0x7fbdb4157880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:39.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.723+0000 7fbdbb699640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbdb4157dc0 0x7fbdb4142bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:39.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.723+0000 7fbdbb699640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb40d34e0 con 0x7fbdb4155eb0 2024-09-19T08:39:39.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.723+0000 7fbdbb699640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb40d3360 con 0x7fbdb4157dc0 2024-09-19T08:39:39.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.723+0000 7fbdbb699640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbdb40d3660 con 0x7fbda00051d0 2024-09-19T08:39:39.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.723+0000 7fbdb8c34640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbdb4155eb0 0x7fbdb4157880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:39.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.723+0000 7fbdb8c34640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbdb4155eb0 0x7fbdb4157880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49510/0 (socket says 172.21.15.98:49510) 2024-09-19T08:39:39.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.723+0000 7fbdb8c34640 1 -- 172.21.15.98:0/3398721720 learned_addr learned my addr 172.21.15.98:0/3398721720 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:39.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.723+0000 7fbdb9435640 1 --2- 172.21.15.98:0/3398721720 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbda00051d0 0x7fbdb4154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:39.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.723+0000 7fbdb9c36640 1 --2- 172.21.15.98:0/3398721720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbdb4157dc0 0x7fbdb4142bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:39.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.723+0000 7fbdb8c34640 1 -- 172.21.15.98:0/3398721720 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbda00051d0 msgr2=0x7fbdb4154390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:39.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.723+0000 7fbdb8c34640 1 --2- 172.21.15.98:0/3398721720 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbda00051d0 0x7fbdb4154390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:39.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.723+0000 7fbdb8c34640 1 -- 172.21.15.98:0/3398721720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbdb4157dc0 msgr2=0x7fbdb4142bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:39.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.723+0000 7fbdb8c34640 1 --2- 172.21.15.98:0/3398721720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbdb4157dc0 0x7fbdb4142bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:39.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.723+0000 7fbdb8c34640 1 -- 172.21.15.98:0/3398721720 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb40d1300 con 0x7fbdb4155eb0 2024-09-19T08:39:39.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.723+0000 7fbdb8c34640 1 --2- 172.21.15.98:0/3398721720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbdb4155eb0 0x7fbdb4157880 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbda40115f0 tx=0x7fbda4011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:39.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.727+0000 7fbda9ffb640 1 -- 172.21.15.98:0/3398721720 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda4015870 con 0x7fbdb4155eb0 2024-09-19T08:39:39.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.727+0000 7fbda9ffb640 1 -- 172.21.15.98:0/3398721720 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbda4015a10 con 0x7fbdb4155eb0 2024-09-19T08:39:39.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.727+0000 7fbda9ffb640 1 -- 172.21.15.98:0/3398721720 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda4015dc0 con 0x7fbdb4155eb0 2024-09-19T08:39:39.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.727+0000 7fbdbb699640 1 -- 172.21.15.98:0/3398721720 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdb41420e0 con 0x7fbdb4155eb0 2024-09-19T08:39:39.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.727+0000 7fbdbb699640 1 -- 172.21.15.98:0/3398721720 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdb41426a0 con 0x7fbdb4155eb0 2024-09-19T08:39:39.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.727+0000 7fbdbb699640 1 -- 172.21.15.98:0/3398721720 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdb4162da0 con 0x7fbdb4155eb0 2024-09-19T08:39:39.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.727+0000 7fbda9ffb640 1 -- 172.21.15.98:0/3398721720 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fbda4013400 con 0x7fbdb4155eb0 2024-09-19T08:39:39.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.727+0000 7fbda9ffb640 1 --2- 172.21.15.98:0/3398721720 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fbd98067a70 0x7fbd98069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:39.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.727+0000 7fbda9ffb640 1 -- 172.21.15.98:0/3398721720 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5709+0+0 (secure 0 0 0) 0x7fbda40867f0 con 0x7fbdb4155eb0 2024-09-19T08:39:39.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.727+0000 7fbdb9435640 1 --2- 172.21.15.98:0/3398721720 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fbd98067a70 0x7fbd98069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:39.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.727+0000 7fbdb9435640 1 --2- 172.21.15.98:0/3398721720 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fbd98067a70 0x7fbd98069f30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0000c10 tx=0x7fbdb0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:39.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.731+0000 7fbda9ffb640 1 -- 172.21.15.98:0/3398721720 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fbda4019030 con 0x7fbdb4155eb0 2024-09-19T08:39:39.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:39.927+0000 7fbdbb699640 1 -- 172.21.15.98:0/3398721720 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fbdb40d15e0 con 0x7fbdb4155eb0 2024-09-19T08:39:40.816 INFO:tasks.ceph.mds.c.smithi098.stderr:2024-09-19T08:39:40.815+0000 7f1a5bd60640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T08:39:40.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:40.939+0000 7fbda9ffb640 1 -- 172.21.15.98:0/3398721720 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v22) ==== 118+0+0 (secure 0 0 0) 0x7fbda40570b0 con 0x7fbdb4155eb0 2024-09-19T08:39:40.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:40.943+0000 7fbdbb699640 1 -- 172.21.15.98:0/3398721720 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fbd98067a70 msgr2=0x7fbd98069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:40.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:40.943+0000 7fbdbb699640 1 --2- 172.21.15.98:0/3398721720 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fbd98067a70 0x7fbd98069f30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbdb0000c10 tx=0x7fbdb0009040 comp rx=0 tx=0).stop 2024-09-19T08:39:40.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:40.943+0000 7fbdbb699640 1 -- 172.21.15.98:0/3398721720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbdb4155eb0 msgr2=0x7fbdb4157880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:40.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:40.943+0000 7fbdbb699640 1 --2- 172.21.15.98:0/3398721720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbdb4155eb0 0x7fbdb4157880 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbda40115f0 tx=0x7fbda4011ab0 comp rx=0 tx=0).stop 2024-09-19T08:39:40.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:40.943+0000 7fbdbb699640 1 -- 172.21.15.98:0/3398721720 shutdown_connections 2024-09-19T08:39:40.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:40.943+0000 7fbdbb699640 1 --2- 172.21.15.98:0/3398721720 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fbd98067a70 0x7fbd98069f30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:40.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:40.943+0000 7fbdbb699640 1 --2- 172.21.15.98:0/3398721720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbdb4157dc0 0x7fbdb4142bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:40.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:40.943+0000 7fbdbb699640 1 --2- 172.21.15.98:0/3398721720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbdb4155eb0 0x7fbdb4157880 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:40.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:40.943+0000 7fbdbb699640 1 --2- 172.21.15.98:0/3398721720 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbda00051d0 0x7fbdb4154390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:40.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:40.943+0000 7fbdbb699640 1 -- 172.21.15.98:0/3398721720 >> 172.21.15.98:0/3398721720 conn(0x7fbdb4074c70 msgr2=0x7fbdb40c9f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:40.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:40.943+0000 7fbdbb699640 1 -- 172.21.15.98:0/3398721720 shutdown_connections 2024-09-19T08:39:40.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:40.943+0000 7fbdbb699640 1 -- 172.21.15.98:0/3398721720 wait complete. 2024-09-19T08:39:40.961 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-19T08:39:40.961 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-19T08:39:41.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.079+0000 7ff726291640 1 Processor -- start 2024-09-19T08:39:41.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.079+0000 7ff726291640 1 -- start start 2024-09-19T08:39:41.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.079+0000 7ff726291640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff720142540 0x7ff7200ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:41.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.079+0000 7ff726291640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff7201455a0 0x7ff720072370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:41.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.079+0000 7ff726291640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff720072d40 con 0x7ff7200728b0 2024-09-19T08:39:41.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.079+0000 7ff726291640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff7200d02b0 con 0x7ff7201455a0 2024-09-19T08:39:41.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.079+0000 7ff726291640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff720073920 con 0x7ff720142540 2024-09-19T08:39:41.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.079+0000 7ff72482e640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7ff7200728b0 0x7ff72006ee10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:44708/0 (socket says 172.21.15.98:44708) 2024-09-19T08:39:41.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.079+0000 7ff72482e640 1 -- 172.21.15.98:0/3758283215 learned_addr learned my addr 172.21.15.98:0/3758283215 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:41.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.079+0000 7ff7177fe640 1 --2- 172.21.15.98:0/3758283215 >> v2:172.21.15.139:3300/0 conn(0x7ff7201455a0 0x7ff720072370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:41.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.079+0000 7ff71f7fe640 1 -- 172.21.15.98:0/3758283215 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4188167324 0 0) 0x7ff720072d40 con 0x7ff7200728b0 2024-09-19T08:39:41.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.079+0000 7ff71f7fe640 1 -- 172.21.15.98:0/3758283215 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff70c003530 con 0x7ff7200728b0 2024-09-19T08:39:41.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.079+0000 7ff71ffff640 1 --2- 172.21.15.98:0/3758283215 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff720142540 0x7ff7200ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:41.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.079+0000 7ff71f7fe640 1 -- 172.21.15.98:0/3758283215 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2274613665 0 0) 0x7ff70c003530 con 0x7ff7200728b0 2024-09-19T08:39:41.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.079+0000 7ff71f7fe640 1 -- 172.21.15.98:0/3758283215 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff720142540 msgr2=0x7ff7200ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:41.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.079+0000 7ff71f7fe640 1 --2- 172.21.15.98:0/3758283215 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff720142540 0x7ff7200ce5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:41.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.079+0000 7ff71f7fe640 1 -- 172.21.15.98:0/3758283215 >> v2:172.21.15.139:3300/0 conn(0x7ff7201455a0 msgr2=0x7ff720072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:41.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.079+0000 7ff71f7fe640 1 --2- 172.21.15.98:0/3758283215 >> v2:172.21.15.139:3300/0 conn(0x7ff7201455a0 0x7ff720072370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:41.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.079+0000 7ff71f7fe640 1 -- 172.21.15.98:0/3758283215 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7200d0500 con 0x7ff7200728b0 2024-09-19T08:39:41.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.079+0000 7ff71f7fe640 1 -- 172.21.15.98:0/3758283215 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7ff718001a00 con 0x7ff7200728b0 2024-09-19T08:39:41.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.079+0000 7ff71f7fe640 1 -- 172.21.15.98:0/3758283215 >> v1:172.21.15.98:6789/0 conn(0x7ff7200728b0 legacy=0x7ff72006ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:41.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.079+0000 7ff71f7fe640 1 --2- 172.21.15.98:0/3758283215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff70c0039c0 0x7ff70c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:41.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.079+0000 7ff71f7fe640 1 --2- 172.21.15.98:0/3758283215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff70c004810 0x7ff70c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:41.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.079+0000 7ff71f7fe640 1 --2- 172.21.15.98:0/3758283215 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff720142540 0x7ff70c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:41.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.079+0000 7ff71f7fe640 1 -- 172.21.15.98:0/3758283215 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff7200d02b0 con 0x7ff70c004810 2024-09-19T08:39:41.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.079+0000 7ff71f7fe640 1 -- 172.21.15.98:0/3758283215 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff720073920 con 0x7ff70c0039c0 2024-09-19T08:39:41.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.079+0000 7ff71f7fe640 1 -- 172.21.15.98:0/3758283215 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff70c00b9d0 con 0x7ff720142540 2024-09-19T08:39:41.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.079+0000 7ff7177fe640 1 --2- 172.21.15.98:0/3758283215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff70c004810 0x7ff70c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:41.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.079+0000 7ff71ffff640 1 --2- 172.21.15.98:0/3758283215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff70c0039c0 0x7ff70c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:41.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.079+0000 7ff72482e640 1 --2- 172.21.15.98:0/3758283215 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff720142540 0x7ff70c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:41.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.079+0000 7ff7177fe640 1 -- 172.21.15.98:0/3758283215 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff720142540 msgr2=0x7ff70c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:41.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.079+0000 7ff7177fe640 1 --2- 172.21.15.98:0/3758283215 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff720142540 0x7ff70c0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:41.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.079+0000 7ff7177fe640 1 -- 172.21.15.98:0/3758283215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff70c0039c0 msgr2=0x7ff70c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:41.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.079+0000 7ff7177fe640 1 --2- 172.21.15.98:0/3758283215 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff70c0039c0 0x7ff70c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:41.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.079+0000 7ff7177fe640 1 -- 172.21.15.98:0/3758283215 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff70c003530 con 0x7ff70c004810 2024-09-19T08:39:41.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.083+0000 7ff7177fe640 1 --2- 172.21.15.98:0/3758283215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff70c004810 0x7ff70c004c90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff708009ae0 tx=0x7ff70801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304b84031a6266b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:41.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.083+0000 7ff71f7fe640 1 -- 172.21.15.98:0/3758283215 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7080b2070 con 0x7ff70c004810 2024-09-19T08:39:41.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.083+0000 7ff71f7fe640 1 -- 172.21.15.98:0/3758283215 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7080a5580 con 0x7ff70c004810 2024-09-19T08:39:41.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.083+0000 7ff71f7fe640 1 -- 172.21.15.98:0/3758283215 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7080096b0 con 0x7ff70c004810 2024-09-19T08:39:41.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.083+0000 7ff726291640 1 -- 172.21.15.98:0/3758283215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff70c004810 msgr2=0x7ff70c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:41.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.083+0000 7ff726291640 1 --2- 172.21.15.98:0/3758283215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff70c004810 0x7ff70c004c90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff708009ae0 tx=0x7ff70801ca70 comp rx=0 tx=0).stop 2024-09-19T08:39:41.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.083+0000 7ff72482e640 1 -- 172.21.15.98:0/3758283215 reap_dead start 2024-09-19T08:39:41.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.083+0000 7ff726291640 1 -- 172.21.15.98:0/3758283215 shutdown_connections 2024-09-19T08:39:41.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.083+0000 7ff726291640 1 -- 172.21.15.98:0/3758283215 >> 172.21.15.98:0/3758283215 conn(0x7ff7200d0a10 msgr2=0x7ff7200d0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:41.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.083+0000 7ff726291640 1 -- 172.21.15.98:0/3758283215 shutdown_connections 2024-09-19T08:39:41.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.083+0000 7ff726291640 1 -- 172.21.15.98:0/3758283215 wait complete. 2024-09-19T08:39:41.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.083+0000 7ff726291640 1 Processor -- start 2024-09-19T08:39:41.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.083+0000 7ff726291640 1 -- start start 2024-09-19T08:39:41.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.083+0000 7ff726291640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff70c004810 0x7ff7201446a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:41.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.083+0000 7ff726291640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff72013f140 0x7ff72013f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:41.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.083+0000 7ff726291640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7200b43b0 0x7ff7200b4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:41.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.083+0000 7ff726291640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff720142920 con 0x7ff7200b43b0 2024-09-19T08:39:41.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.083+0000 7ff726291640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff7200735b0 con 0x7ff70c004810 2024-09-19T08:39:41.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.083+0000 7ff726291640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff720142aa0 con 0x7ff72013f140 2024-09-19T08:39:41.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.083+0000 7ff72482e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7200b43b0 0x7ff7200b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:41.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.083+0000 7ff72482e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7200b43b0 0x7ff7200b4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49526/0 (socket says 172.21.15.98:49526) 2024-09-19T08:39:41.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.083+0000 7ff72482e640 1 -- 172.21.15.98:0/1176475394 learned_addr learned my addr 172.21.15.98:0/1176475394 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:41.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.083+0000 7ff71ffff640 1 --2- 172.21.15.98:0/1176475394 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff70c004810 0x7ff7201446a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:41.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.087+0000 7ff7177fe640 1 --2- 172.21.15.98:0/1176475394 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff72013f140 0x7ff72013f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:41.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.087+0000 7ff72482e640 1 -- 172.21.15.98:0/1176475394 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff72013f140 msgr2=0x7ff72013f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:41.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.087+0000 7ff72482e640 1 --2- 172.21.15.98:0/1176475394 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff72013f140 0x7ff72013f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:41.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.087+0000 7ff72482e640 1 -- 172.21.15.98:0/1176475394 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff70c004810 msgr2=0x7ff7201446a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:41.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.087+0000 7ff72482e640 1 --2- 172.21.15.98:0/1176475394 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff70c004810 0x7ff7201446a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:41.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.087+0000 7ff72482e640 1 -- 172.21.15.98:0/1176475394 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7200c5e10 con 0x7ff7200b43b0 2024-09-19T08:39:41.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.087+0000 7ff72482e640 1 --2- 172.21.15.98:0/1176475394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7200b43b0 0x7ff7200b4830 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff7200d0240 tx=0x7ff718005aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:41.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.087+0000 7ff71d7fa640 1 -- 172.21.15.98:0/1176475394 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71801a070 con 0x7ff7200b43b0 2024-09-19T08:39:41.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.087+0000 7ff71d7fa640 1 -- 172.21.15.98:0/1176475394 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff718011040 con 0x7ff7200b43b0 2024-09-19T08:39:41.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.087+0000 7ff726291640 1 -- 172.21.15.98:0/1176475394 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7200b50e0 con 0x7ff7200b43b0 2024-09-19T08:39:41.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.087+0000 7ff71d7fa640 1 -- 172.21.15.98:0/1176475394 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71800ed80 con 0x7ff7200b43b0 2024-09-19T08:39:41.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.087+0000 7ff726291640 1 -- 172.21.15.98:0/1176475394 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7200b56a0 con 0x7ff7200b43b0 2024-09-19T08:39:41.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.087+0000 7ff726291640 1 -- 172.21.15.98:0/1176475394 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7200c5fe0 con 0x7ff7200b43b0 2024-09-19T08:39:41.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.087+0000 7ff71d7fa640 1 -- 172.21.15.98:0/1176475394 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7ff71801d070 con 0x7ff7200b43b0 2024-09-19T08:39:41.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.087+0000 7ff71d7fa640 1 --2- 172.21.15.98:0/1176475394 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff700067a70 0x7ff700069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:41.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.087+0000 7ff71d7fa640 1 -- 172.21.15.98:0/1176475394 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7ff718088750 con 0x7ff7200b43b0 2024-09-19T08:39:41.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.087+0000 7ff71ffff640 1 --2- 172.21.15.98:0/1176475394 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff700067a70 0x7ff700069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:41.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.087+0000 7ff71ffff640 1 --2- 172.21.15.98:0/1176475394 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff700067a70 0x7ff700069f30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff7200caff0 tx=0x7ff710008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:41.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.091+0000 7ff71d7fa640 1 -- 172.21.15.98:0/1176475394 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff718058a50 con 0x7ff7200b43b0 2024-09-19T08:39:41.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.275+0000 7ff726291640 1 -- 172.21.15.98:0/1176475394 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7ff720072370 con 0x7ff7200b43b0 2024-09-19T08:39:41.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.943+0000 7ff71d7fa640 1 -- 172.21.15.98:0/1176475394 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v23) ==== 122+0+0 (secure 0 0 0) 0x7ff71805cd50 con 0x7ff7200b43b0 2024-09-19T08:39:41.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.947+0000 7ff726291640 1 -- 172.21.15.98:0/1176475394 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff700067a70 msgr2=0x7ff700069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:41.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.947+0000 7ff726291640 1 --2- 172.21.15.98:0/1176475394 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff700067a70 0x7ff700069f30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff7200caff0 tx=0x7ff710008040 comp rx=0 tx=0).stop 2024-09-19T08:39:41.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.947+0000 7ff726291640 1 -- 172.21.15.98:0/1176475394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7200b43b0 msgr2=0x7ff7200b4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:41.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.947+0000 7ff726291640 1 --2- 172.21.15.98:0/1176475394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7200b43b0 0x7ff7200b4830 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff7200d0240 tx=0x7ff718005aa0 comp rx=0 tx=0).stop 2024-09-19T08:39:41.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.947+0000 7ff726291640 1 -- 172.21.15.98:0/1176475394 shutdown_connections 2024-09-19T08:39:41.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.947+0000 7ff726291640 1 --2- 172.21.15.98:0/1176475394 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff700067a70 0x7ff700069f30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:41.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.947+0000 7ff726291640 1 --2- 172.21.15.98:0/1176475394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff7200b43b0 0x7ff7200b4830 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:41.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.947+0000 7ff726291640 1 --2- 172.21.15.98:0/1176475394 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff72013f140 0x7ff72013f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:41.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.947+0000 7ff726291640 1 --2- 172.21.15.98:0/1176475394 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff70c004810 0x7ff7201446a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:41.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.947+0000 7ff726291640 1 -- 172.21.15.98:0/1176475394 >> 172.21.15.98:0/1176475394 conn(0x7ff7200d0a10 msgr2=0x7ff7200b09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:41.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.947+0000 7ff726291640 1 -- 172.21.15.98:0/1176475394 shutdown_connections 2024-09-19T08:39:41.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:41.947+0000 7ff726291640 1 -- 172.21.15.98:0/1176475394 wait complete. 2024-09-19T08:39:41.968 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-19T08:39:41.969 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:39:42.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.095+0000 7fbe9f3c2640 1 Processor -- start 2024-09-19T08:39:42.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.095+0000 7fbe9f3c2640 1 -- start start 2024-09-19T08:39:42.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.095+0000 7fbe9f3c2640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbe9815c550 0x7fbe9806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:42.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.095+0000 7fbe9f3c2640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbe980cfad0 0x7fbe980723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:42.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.095+0000 7fbe9f3c2640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe9806d310 con 0x7fbe9806c1d0 2024-09-19T08:39:42.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.095+0000 7fbe9f3c2640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fbe9806c640 con 0x7fbe9815c550 2024-09-19T08:39:42.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.095+0000 7fbe9f3c2640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fbe9806d5f0 con 0x7fbe980cfad0 2024-09-19T08:39:42.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.095+0000 7fbe9d15e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbe9815c550 0x7fbe9806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:42.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.095+0000 7fbe9d15e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbe9815c550 0x7fbe9806bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49542/0 (socket says 172.21.15.98:49542) 2024-09-19T08:39:42.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.095+0000 7fbe9d15e640 1 -- 172.21.15.98:0/2672789500 learned_addr learned my addr 172.21.15.98:0/2672789500 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:42.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.095+0000 7fbe9d15e640 1 -- 172.21.15.98:0/2672789500 >> v2:172.21.15.139:3300/0 conn(0x7fbe980cfad0 msgr2=0x7fbe980723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:42.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.095+0000 7fbe9d15e640 1 --2- 172.21.15.98:0/2672789500 >> v2:172.21.15.139:3300/0 conn(0x7fbe980cfad0 0x7fbe980723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:42.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.095+0000 7fbe9d15e640 1 -- 172.21.15.98:0/2672789500 >> v1:172.21.15.139:6789/0 conn(0x7fbe9806c1d0 legacy=0x7fbe980cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:42.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.095+0000 7fbe9d15e640 1 -- 172.21.15.98:0/2672789500 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe9806d8d0 con 0x7fbe9815c550 2024-09-19T08:39:42.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.099+0000 7fbe9d15e640 1 --2- 172.21.15.98:0/2672789500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe9815c550 0x7fbe9806bc90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fbe980b33a0 tx=0x7fbe9401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9ae806b58573a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:42.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.099+0000 7fbe8ffff640 1 -- 172.21.15.98:0/2672789500 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe940b0070 con 0x7fbe9815c550 2024-09-19T08:39:42.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.099+0000 7fbe8ffff640 1 -- 172.21.15.98:0/2672789500 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe940ab600 con 0x7fbe9815c550 2024-09-19T08:39:42.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.099+0000 7fbe8ffff640 1 -- 172.21.15.98:0/2672789500 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe940ab9d0 con 0x7fbe9815c550 2024-09-19T08:39:42.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.099+0000 7fbe9f3c2640 1 -- 172.21.15.98:0/2672789500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe9815c550 msgr2=0x7fbe9806bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:42.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.099+0000 7fbe9f3c2640 1 --2- 172.21.15.98:0/2672789500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe9815c550 0x7fbe9806bc90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fbe980b33a0 tx=0x7fbe9401c4e0 comp rx=0 tx=0).stop 2024-09-19T08:39:42.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.099+0000 7fbe9f3c2640 1 -- 172.21.15.98:0/2672789500 shutdown_connections 2024-09-19T08:39:42.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.099+0000 7fbe9f3c2640 1 --2- 172.21.15.98:0/2672789500 >> v2:172.21.15.139:3300/0 conn(0x7fbe980cfad0 0x7fbe980723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:42.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.099+0000 7fbe9f3c2640 1 --2- 172.21.15.98:0/2672789500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe9815c550 0x7fbe9806bc90 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:42.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.099+0000 7fbe9f3c2640 1 -- 172.21.15.98:0/2672789500 >> 172.21.15.98:0/2672789500 conn(0x7fbe980b2010 msgr2=0x7fbe980b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:42.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.099+0000 7fbe9f3c2640 1 -- 172.21.15.98:0/2672789500 shutdown_connections 2024-09-19T08:39:42.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.099+0000 7fbe9f3c2640 1 -- 172.21.15.98:0/2672789500 wait complete. 2024-09-19T08:39:42.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.099+0000 7fbe9f3c2640 1 Processor -- start 2024-09-19T08:39:42.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.099+0000 7fbe9f3c2640 1 -- start start 2024-09-19T08:39:42.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.099+0000 7fbe9f3c2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe9806c1d0 0x7fbe98151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:42.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.099+0000 7fbe9f3c2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe980cfad0 0x7fbe980aa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:42.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.099+0000 7fbe9f3c2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe9815c550 0x7fbe980d0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:42.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.099+0000 7fbe9f3c2640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbe980734c0 con 0x7fbe9806c1d0 2024-09-19T08:39:42.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.099+0000 7fbe9f3c2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbe98073340 con 0x7fbe980cfad0 2024-09-19T08:39:42.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.099+0000 7fbe9f3c2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbe98073640 con 0x7fbe9815c550 2024-09-19T08:39:42.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.099+0000 7fbe9d15e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe9806c1d0 0x7fbe98151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:42.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.099+0000 7fbe9d15e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe9806c1d0 0x7fbe98151fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49544/0 (socket says 172.21.15.98:49544) 2024-09-19T08:39:42.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.099+0000 7fbe9d15e640 1 -- 172.21.15.98:0/4236817594 learned_addr learned my addr 172.21.15.98:0/4236817594 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:42.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.103+0000 7fbe9c95d640 1 --2- 172.21.15.98:0/4236817594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe980cfad0 0x7fbe980aa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:42.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.103+0000 7fbe9d95f640 1 --2- 172.21.15.98:0/4236817594 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe9815c550 0x7fbe980d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:42.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.103+0000 7fbe9d15e640 1 -- 172.21.15.98:0/4236817594 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe9815c550 msgr2=0x7fbe980d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:42.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.103+0000 7fbe9d15e640 1 --2- 172.21.15.98:0/4236817594 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe9815c550 0x7fbe980d0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:42.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.103+0000 7fbe9d15e640 1 -- 172.21.15.98:0/4236817594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe980cfad0 msgr2=0x7fbe980aa4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:42.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.103+0000 7fbe9d15e640 1 --2- 172.21.15.98:0/4236817594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe980cfad0 0x7fbe980aa4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:42.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.103+0000 7fbe9d15e640 1 -- 172.21.15.98:0/4236817594 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe98156390 con 0x7fbe9806c1d0 2024-09-19T08:39:42.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.103+0000 7fbe9d15e640 1 --2- 172.21.15.98:0/4236817594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe9806c1d0 0x7fbe98151fd0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbe940ae040 tx=0x7fbe9401ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:42.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.103+0000 7fbe8dffb640 1 -- 172.21.15.98:0/4236817594 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe940b0040 con 0x7fbe9806c1d0 2024-09-19T08:39:42.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.103+0000 7fbe8dffb640 1 -- 172.21.15.98:0/4236817594 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe940bc070 con 0x7fbe9806c1d0 2024-09-19T08:39:42.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.103+0000 7fbe9f3c2640 1 -- 172.21.15.98:0/4236817594 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe98156670 con 0x7fbe9806c1d0 2024-09-19T08:39:42.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.103+0000 7fbe8dffb640 1 -- 172.21.15.98:0/4236817594 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe940b6880 con 0x7fbe9806c1d0 2024-09-19T08:39:42.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.103+0000 7fbe9f3c2640 1 -- 172.21.15.98:0/4236817594 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe98156c30 con 0x7fbe9806c1d0 2024-09-19T08:39:42.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.103+0000 7fbe9f3c2640 1 -- 172.21.15.98:0/4236817594 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe980723a0 con 0x7fbe9806c1d0 2024-09-19T08:39:42.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.103+0000 7fbe8dffb640 1 -- 172.21.15.98:0/4236817594 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fbe940b6a20 con 0x7fbe9806c1d0 2024-09-19T08:39:42.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.103+0000 7fbe8dffb640 1 --2- 172.21.15.98:0/4236817594 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fbe7c067a70 0x7fbe7c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:42.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.103+0000 7fbe8dffb640 1 -- 172.21.15.98:0/4236817594 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fbe94120ef0 con 0x7fbe9806c1d0 2024-09-19T08:39:42.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.103+0000 7fbe9c95d640 1 --2- 172.21.15.98:0/4236817594 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fbe7c067a70 0x7fbe7c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:42.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.103+0000 7fbe9c95d640 1 --2- 172.21.15.98:0/4236817594 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fbe7c067a70 0x7fbe7c069f30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbe981545f0 tx=0x7fbe88008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:42.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.107+0000 7fbe8dffb640 1 -- 172.21.15.98:0/4236817594 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fbe940160d0 con 0x7fbe9806c1d0 2024-09-19T08:39:42.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.303+0000 7fbe9f3c2640 1 -- 172.21.15.98:0/4236817594 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe9815eff0 con 0x7fbe9806c1d0 2024-09-19T08:39:42.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.303+0000 7fbe8dffb640 1 -- 172.21.15.98:0/4236817594 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5229 (secure 0 0 0) 0x7fbe940f1780 con 0x7fbe9806c1d0 2024-09-19T08:39:42.305 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:39:42.305 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":23,"btime":"2024-09-19T08:39:41:944534+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/2874086421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":2874086421},{"type":"v1","addr":"172.21.15.139:6833","nonce":2874086421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4996,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/4178563848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":4178563848},{"type":"v1","addr":"172.21.15.139:6835","nonce":4178563848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3584568343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3584568343},{"type":"v1","addr":"172.21.15.98:6835","nonce":3584568343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:39:38.262938+0000","modified":"2024-09-19T08:39:41.276985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/3085042202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3085042202},{"type":"v1","addr":"172.21.15.98:6837","nonce":3085042202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244]},"id":3}]} 2024-09-19T08:39:42.305 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 23 2024-09-19T08:39:42.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.307+0000 7fbe9f3c2640 1 -- 172.21.15.98:0/4236817594 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fbe7c067a70 msgr2=0x7fbe7c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:42.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.307+0000 7fbe9f3c2640 1 --2- 172.21.15.98:0/4236817594 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fbe7c067a70 0x7fbe7c069f30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbe981545f0 tx=0x7fbe88008040 comp rx=0 tx=0).stop 2024-09-19T08:39:42.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.307+0000 7fbe9f3c2640 1 -- 172.21.15.98:0/4236817594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe9806c1d0 msgr2=0x7fbe98151fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:42.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.307+0000 7fbe9f3c2640 1 --2- 172.21.15.98:0/4236817594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe9806c1d0 0x7fbe98151fd0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbe940ae040 tx=0x7fbe9401ad20 comp rx=0 tx=0).stop 2024-09-19T08:39:42.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.307+0000 7fbe9f3c2640 1 -- 172.21.15.98:0/4236817594 shutdown_connections 2024-09-19T08:39:42.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.307+0000 7fbe9f3c2640 1 --2- 172.21.15.98:0/4236817594 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fbe7c067a70 0x7fbe7c069f30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:42.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.307+0000 7fbe9f3c2640 1 --2- 172.21.15.98:0/4236817594 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe9815c550 0x7fbe980d0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:42.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.307+0000 7fbe9f3c2640 1 --2- 172.21.15.98:0/4236817594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe980cfad0 0x7fbe980aa4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:42.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.307+0000 7fbe9f3c2640 1 --2- 172.21.15.98:0/4236817594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe9806c1d0 0x7fbe98151fd0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:42.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.307+0000 7fbe9f3c2640 1 -- 172.21.15.98:0/4236817594 >> 172.21.15.98:0/4236817594 conn(0x7fbe980b2010 msgr2=0x7fbe98067a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:42.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.307+0000 7fbe9f3c2640 1 -- 172.21.15.98:0/4236817594 shutdown_connections 2024-09-19T08:39:42.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.307+0000 7fbe9f3c2640 1 -- 172.21.15.98:0/4236817594 wait complete. 2024-09-19T08:39:42.324 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:39:42.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.447+0000 7f2c7dfe7640 1 Processor -- start 2024-09-19T08:39:42.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.447+0000 7f2c7dfe7640 1 -- start start 2024-09-19T08:39:42.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.447+0000 7f2c7dfe7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2c7806c3a0 0x7f2c780cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:42.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.447+0000 7f2c7dfe7640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c7806d310 con 0x7f2c7815c550 2024-09-19T08:39:42.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.447+0000 7f2c7dfe7640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c7806d5f0 con 0x7f2c780cf900 2024-09-19T08:39:42.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.447+0000 7f2c7dfe7640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f2c7806d8d0 con 0x7f2c7806c3a0 2024-09-19T08:39:42.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.447+0000 7f2c777fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f2c7815c550 0x7f2c7806bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:44712/0 (socket says 172.21.15.98:44712) 2024-09-19T08:39:42.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.447+0000 7f2c777fe640 1 -- 172.21.15.98:0/848560488 learned_addr learned my addr 172.21.15.98:0/848560488 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:42.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.447+0000 7f2c76ffd640 1 -- 172.21.15.98:0/848560488 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 149170068 0 0) 0x7f2c7806d310 con 0x7f2c7815c550 2024-09-19T08:39:42.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.447+0000 7f2c6ffff640 1 --2- 172.21.15.98:0/848560488 >> v2:172.21.15.139:3300/0 conn(0x7f2c7806c3a0 0x7f2c780cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:42.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.447+0000 7f2c76ffd640 1 -- 172.21.15.98:0/848560488 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c58003530 con 0x7f2c7815c550 2024-09-19T08:39:42.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.447+0000 7f2c76ffd640 1 -- 172.21.15.98:0/848560488 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4018215016 0 0) 0x7f2c58003530 con 0x7f2c7815c550 2024-09-19T08:39:42.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.447+0000 7f2c76ffd640 1 -- 172.21.15.98:0/848560488 >> v2:172.21.15.139:3300/0 conn(0x7f2c7806c3a0 msgr2=0x7f2c780cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:42.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.447+0000 7f2c76ffd640 1 --2- 172.21.15.98:0/848560488 >> v2:172.21.15.139:3300/0 conn(0x7f2c7806c3a0 0x7f2c780cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:42.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.447+0000 7f2c76ffd640 1 -- 172.21.15.98:0/848560488 >> v1:172.21.15.139:6789/0 conn(0x7f2c780cf900 legacy=0x7f2c780723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:42.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.447+0000 7f2c76ffd640 1 -- 172.21.15.98:0/848560488 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c7806dbb0 con 0x7f2c7815c550 2024-09-19T08:39:42.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.447+0000 7f2c76ffd640 1 -- 172.21.15.98:0/848560488 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f2c68001680 con 0x7f2c7815c550 2024-09-19T08:39:42.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.447+0000 7f2c76ffd640 1 -- 172.21.15.98:0/848560488 >> v1:172.21.15.98:6789/0 conn(0x7f2c7815c550 legacy=0x7f2c7806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:42.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.447+0000 7f2c76ffd640 1 --2- 172.21.15.98:0/848560488 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c580039c0 0x7f2c58003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:42.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.447+0000 7f2c76ffd640 1 --2- 172.21.15.98:0/848560488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c58004810 0x7f2c58004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:42.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.447+0000 7f2c76ffd640 1 --2- 172.21.15.98:0/848560488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c580051d0 0x7f2c5800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:42.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.447+0000 7f2c76ffd640 1 -- 172.21.15.98:0/848560488 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7806d8d0 con 0x7f2c58004810 2024-09-19T08:39:42.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.447+0000 7f2c76ffd640 1 -- 172.21.15.98:0/848560488 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5800c070 con 0x7f2c580051d0 2024-09-19T08:39:42.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.447+0000 7f2c76ffd640 1 -- 172.21.15.98:0/848560488 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2c5800c270 con 0x7f2c580039c0 2024-09-19T08:39:42.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.447+0000 7f2c6ffff640 1 --2- 172.21.15.98:0/848560488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c58004810 0x7f2c58004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:42.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.447+0000 7f2c777fe640 1 --2- 172.21.15.98:0/848560488 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c580039c0 0x7f2c58003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:42.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.447+0000 7f2c6ffff640 1 -- 172.21.15.98:0/848560488 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c580039c0 msgr2=0x7f2c58003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:42.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.447+0000 7f2c6ffff640 1 --2- 172.21.15.98:0/848560488 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c580039c0 0x7f2c58003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:42.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.447+0000 7f2c6ffff640 1 -- 172.21.15.98:0/848560488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c580051d0 msgr2=0x7f2c5800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:42.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.447+0000 7f2c6ffff640 1 --2- 172.21.15.98:0/848560488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c580051d0 0x7f2c5800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:42.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.447+0000 7f2c6ffff640 1 -- 172.21.15.98:0/848560488 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c58003530 con 0x7f2c58004810 2024-09-19T08:39:42.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.447+0000 7f2c77fff640 1 -- 172.21.15.98:0/848560488 reap_dead start 2024-09-19T08:39:42.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.447+0000 7f2c777fe640 1 --2- 172.21.15.98:0/848560488 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c580039c0 0x7f2c58003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:42.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.451+0000 7f2c6ffff640 1 --2- 172.21.15.98:0/848560488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c58004810 0x7f2c58004c90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2c60009ac0 tx=0x7f2c6001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba707b2973a47c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:42.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.451+0000 7f2c76ffd640 1 -- 172.21.15.98:0/848560488 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c600b2070 con 0x7f2c58004810 2024-09-19T08:39:42.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.451+0000 7f2c76ffd640 1 -- 172.21.15.98:0/848560488 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c600a5660 con 0x7f2c58004810 2024-09-19T08:39:42.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.451+0000 7f2c76ffd640 1 -- 172.21.15.98:0/848560488 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c60009690 con 0x7f2c58004810 2024-09-19T08:39:42.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.451+0000 7f2c7dfe7640 1 -- 172.21.15.98:0/848560488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c58004810 msgr2=0x7f2c58004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:42.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.451+0000 7f2c7dfe7640 1 --2- 172.21.15.98:0/848560488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c58004810 0x7f2c58004c90 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2c60009ac0 tx=0x7f2c6001ca70 comp rx=0 tx=0).stop 2024-09-19T08:39:42.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.451+0000 7f2c7dfe7640 1 -- 172.21.15.98:0/848560488 shutdown_connections 2024-09-19T08:39:42.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.451+0000 7f2c7dfe7640 1 --2- 172.21.15.98:0/848560488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c58004810 0x7f2c58004c90 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:42.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.451+0000 7f2c7dfe7640 1 -- 172.21.15.98:0/848560488 >> 172.21.15.98:0/848560488 conn(0x7f2c780b2010 msgr2=0x7f2c780b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:42.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.451+0000 7f2c7dfe7640 1 -- 172.21.15.98:0/848560488 shutdown_connections 2024-09-19T08:39:42.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.451+0000 7f2c7dfe7640 1 -- 172.21.15.98:0/848560488 wait complete. 2024-09-19T08:39:42.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.451+0000 7f2c7dfe7640 1 Processor -- start 2024-09-19T08:39:42.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.451+0000 7f2c7dfe7640 1 -- start start 2024-09-19T08:39:42.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.451+0000 7f2c7dfe7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c58004810 0x7f2c781530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:42.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.451+0000 7f2c7dfe7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c7814f920 0x7f2c7814fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:42.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.451+0000 7f2c7dfe7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c7815cf20 0x7f2c7815d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:42.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.451+0000 7f2c7dfe7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7815cab0 con 0x7f2c58004810 2024-09-19T08:39:42.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.451+0000 7f2c7dfe7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7815c930 con 0x7f2c7814f920 2024-09-19T08:39:42.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.451+0000 7f2c7dfe7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2c780cf3c0 con 0x7f2c7815cf20 2024-09-19T08:39:42.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.451+0000 7f2c777fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c58004810 0x7f2c781530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:42.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.451+0000 7f2c777fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c58004810 0x7f2c781530a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49554/0 (socket says 172.21.15.98:49554) 2024-09-19T08:39:42.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.451+0000 7f2c777fe640 1 -- 172.21.15.98:0/1095519137 learned_addr learned my addr 172.21.15.98:0/1095519137 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:42.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.451+0000 7f2c6ffff640 1 --2- 172.21.15.98:0/1095519137 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c7814f920 0x7f2c7814fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:42.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.451+0000 7f2c777fe640 1 -- 172.21.15.98:0/1095519137 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c7815cf20 msgr2=0x7f2c7815d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:42.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.451+0000 7f2c777fe640 1 --2- 172.21.15.98:0/1095519137 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c7815cf20 0x7f2c7815d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:42.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.451+0000 7f2c777fe640 1 -- 172.21.15.98:0/1095519137 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c7814f920 msgr2=0x7f2c7814fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:42.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.451+0000 7f2c777fe640 1 --2- 172.21.15.98:0/1095519137 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c7814f920 0x7f2c7814fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:42.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.451+0000 7f2c777fe640 1 -- 172.21.15.98:0/1095519137 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c7815dbd0 con 0x7f2c58004810 2024-09-19T08:39:42.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.455+0000 7f2c777fe640 1 --2- 172.21.15.98:0/1095519137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c58004810 0x7f2c781530a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2c68010b10 tx=0x7f2c6800e100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:42.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.455+0000 7f2c74ff9640 1 -- 172.21.15.98:0/1095519137 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6801b070 con 0x7f2c58004810 2024-09-19T08:39:42.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.455+0000 7f2c74ff9640 1 -- 172.21.15.98:0/1095519137 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c68011040 con 0x7f2c58004810 2024-09-19T08:39:42.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.455+0000 7f2c7dfe7640 1 -- 172.21.15.98:0/1095519137 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c7815deb0 con 0x7f2c58004810 2024-09-19T08:39:42.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.455+0000 7f2c74ff9640 1 -- 172.21.15.98:0/1095519137 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c680143a0 con 0x7f2c58004810 2024-09-19T08:39:42.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.455+0000 7f2c7dfe7640 1 -- 172.21.15.98:0/1095519137 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c781528a0 con 0x7f2c58004810 2024-09-19T08:39:42.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.455+0000 7f2c7dfe7640 1 -- 172.21.15.98:0/1095519137 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c780cf6c0 con 0x7f2c58004810 2024-09-19T08:39:42.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.455+0000 7f2c74ff9640 1 -- 172.21.15.98:0/1095519137 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f2c6801e070 con 0x7f2c58004810 2024-09-19T08:39:42.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.455+0000 7f2c74ff9640 1 --2- 172.21.15.98:0/1095519137 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2c5c067a70 0x7f2c5c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:42.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.455+0000 7f2c74ff9640 1 -- 172.21.15.98:0/1095519137 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f2c68088b30 con 0x7f2c58004810 2024-09-19T08:39:42.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.455+0000 7f2c6ffff640 1 --2- 172.21.15.98:0/1095519137 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2c5c067a70 0x7f2c5c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:42.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.455+0000 7f2c6ffff640 1 --2- 172.21.15.98:0/1095519137 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2c5c067a70 0x7f2c5c069f30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2c600af000 tx=0x7f2c60008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:42.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.459+0000 7f2c74ff9640 1 -- 172.21.15.98:0/1095519137 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2c680593f0 con 0x7f2c58004810 2024-09-19T08:39:42.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.623+0000 7f2c7dfe7640 1 -- 172.21.15.98:0/1095519137 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c78067cc0 con 0x7f2c58004810 2024-09-19T08:39:42.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.627+0000 7f2c74ff9640 1 -- 172.21.15.98:0/1095519137 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+16673 (secure 0 0 0) 0x7f2c68058cf0 con 0x7f2c58004810 2024-09-19T08:39:42.627 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:39:42.628 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":43,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:39:39.890452+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:39:36.829570+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-19T08:39:37.237199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:39:42.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.631+0000 7f2c7dfe7640 1 -- 172.21.15.98:0/1095519137 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2c5c067a70 msgr2=0x7f2c5c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:42.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.631+0000 7f2c7dfe7640 1 --2- 172.21.15.98:0/1095519137 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2c5c067a70 0x7f2c5c069f30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2c600af000 tx=0x7f2c60008040 comp rx=0 tx=0).stop 2024-09-19T08:39:42.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.631+0000 7f2c7dfe7640 1 -- 172.21.15.98:0/1095519137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c58004810 msgr2=0x7f2c781530a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:42.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.631+0000 7f2c7dfe7640 1 --2- 172.21.15.98:0/1095519137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c58004810 0x7f2c781530a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2c68010b10 tx=0x7f2c6800e100 comp rx=0 tx=0).stop 2024-09-19T08:39:42.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.631+0000 7f2c7dfe7640 1 -- 172.21.15.98:0/1095519137 shutdown_connections 2024-09-19T08:39:42.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.631+0000 7f2c7dfe7640 1 --2- 172.21.15.98:0/1095519137 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2c5c067a70 0x7f2c5c069f30 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:42.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.631+0000 7f2c7dfe7640 1 --2- 172.21.15.98:0/1095519137 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c7815cf20 0x7f2c7815d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:42.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.631+0000 7f2c7dfe7640 1 --2- 172.21.15.98:0/1095519137 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c7814f920 0x7f2c7814fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:42.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.631+0000 7f2c7dfe7640 1 --2- 172.21.15.98:0/1095519137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c58004810 0x7f2c781530a0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:42.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.631+0000 7f2c7dfe7640 1 -- 172.21.15.98:0/1095519137 >> 172.21.15.98:0/1095519137 conn(0x7f2c780b2010 msgr2=0x7f2c78073300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:42.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.631+0000 7f2c7dfe7640 1 -- 172.21.15.98:0/1095519137 shutdown_connections 2024-09-19T08:39:42.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.631+0000 7f2c7dfe7640 1 -- 172.21.15.98:0/1095519137 wait complete. 2024-09-19T08:39:42.646 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T08:39:42.647 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:39:42.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.763+0000 7f70049f0640 1 Processor -- start 2024-09-19T08:39:42.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.763+0000 7f70049f0640 1 -- start start 2024-09-19T08:39:42.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.763+0000 7f70049f0640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f700015c550 0x7f700006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:42.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.763+0000 7f70049f0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f70000cfad0 0x7f70000723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:42.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.763+0000 7f70049f0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f700006d310 con 0x7f700006c1d0 2024-09-19T08:39:42.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.763+0000 7f70049f0640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f700006c640 con 0x7f700015c550 2024-09-19T08:39:42.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.763+0000 7f70049f0640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f700006d5f0 con 0x7f70000cfad0 2024-09-19T08:39:42.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.763+0000 7f6ffe59c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f700015c550 0x7f700006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:42.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.763+0000 7f6ffe59c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f700015c550 0x7f700006bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49560/0 (socket says 172.21.15.98:49560) 2024-09-19T08:39:42.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.763+0000 7f6ffe59c640 1 -- 172.21.15.98:0/347250484 learned_addr learned my addr 172.21.15.98:0/347250484 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:42.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.763+0000 7f6ffed9d640 1 --2- 172.21.15.98:0/347250484 >> v2:172.21.15.139:3300/0 conn(0x7f70000cfad0 0x7f70000723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:42.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.763+0000 7f6ffe59c640 1 -- 172.21.15.98:0/347250484 >> v2:172.21.15.139:3300/0 conn(0x7f70000cfad0 msgr2=0x7f70000723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:42.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.763+0000 7f6ffe59c640 1 --2- 172.21.15.98:0/347250484 >> v2:172.21.15.139:3300/0 conn(0x7f70000cfad0 0x7f70000723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:42.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.763+0000 7f6ffe59c640 1 -- 172.21.15.98:0/347250484 >> v1:172.21.15.139:6789/0 conn(0x7f700006c1d0 legacy=0x7f70000cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:42.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.763+0000 7f6ffe59c640 1 -- 172.21.15.98:0/347250484 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f700006d8d0 con 0x7f700015c550 2024-09-19T08:39:42.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.763+0000 7f6ffed9d640 1 --2- 172.21.15.98:0/347250484 >> v2:172.21.15.139:3300/0 conn(0x7f70000cfad0 0x7f70000723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:42.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.767+0000 7f6ffe59c640 1 --2- 172.21.15.98:0/347250484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f700015c550 0x7f700006bc90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f70000b33a0 tx=0x7f6ff001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa06f3ca34c591fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:42.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.767+0000 7f6ffd59a640 1 -- 172.21.15.98:0/347250484 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff00b0070 con 0x7f700015c550 2024-09-19T08:39:42.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.767+0000 7f6ffd59a640 1 -- 172.21.15.98:0/347250484 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ff00ab600 con 0x7f700015c550 2024-09-19T08:39:42.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.767+0000 7f6ffd59a640 1 -- 172.21.15.98:0/347250484 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff00ab9d0 con 0x7f700015c550 2024-09-19T08:39:42.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.767+0000 7f70049f0640 1 -- 172.21.15.98:0/347250484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f700015c550 msgr2=0x7f700006bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:42.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.767+0000 7f70049f0640 1 --2- 172.21.15.98:0/347250484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f700015c550 0x7f700006bc90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f70000b33a0 tx=0x7f6ff001c4e0 comp rx=0 tx=0).stop 2024-09-19T08:39:42.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.767+0000 7f70049f0640 1 -- 172.21.15.98:0/347250484 shutdown_connections 2024-09-19T08:39:42.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.767+0000 7f70049f0640 1 --2- 172.21.15.98:0/347250484 >> v2:172.21.15.139:3300/0 conn(0x7f70000cfad0 0x7f70000723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:42.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.767+0000 7f70049f0640 1 --2- 172.21.15.98:0/347250484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f700015c550 0x7f700006bc90 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:42.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.767+0000 7f70049f0640 1 -- 172.21.15.98:0/347250484 >> 172.21.15.98:0/347250484 conn(0x7f70000b2010 msgr2=0x7f70000b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:42.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.767+0000 7f70049f0640 1 -- 172.21.15.98:0/347250484 shutdown_connections 2024-09-19T08:39:42.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.767+0000 7f70049f0640 1 -- 172.21.15.98:0/347250484 wait complete. 2024-09-19T08:39:42.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.767+0000 7f70049f0640 1 Processor -- start 2024-09-19T08:39:42.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.767+0000 7f70049f0640 1 -- start start 2024-09-19T08:39:42.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.771+0000 7f70049f0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f700006c1d0 0x7f700014d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:42.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.771+0000 7f70049f0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70000cfad0 0x7f7000158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:42.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.771+0000 7f70049f0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7000158580 0x7f700014ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:42.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.771+0000 7f70049f0640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f70000734c0 con 0x7f7000158580 2024-09-19T08:39:42.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.771+0000 7f70049f0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7000073340 con 0x7f70000cfad0 2024-09-19T08:39:42.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.771+0000 7f70049f0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7000073640 con 0x7f700006c1d0 2024-09-19T08:39:42.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.771+0000 7f6ffed9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7000158580 0x7f700014ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:42.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.771+0000 7f6ffed9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7000158580 0x7f700014ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49572/0 (socket says 172.21.15.98:49572) 2024-09-19T08:39:42.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.771+0000 7f6ffed9d640 1 -- 172.21.15.98:0/144256398 learned_addr learned my addr 172.21.15.98:0/144256398 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:42.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.771+0000 7f6ffe59c640 1 --2- 172.21.15.98:0/144256398 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f700006c1d0 0x7f700014d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:42.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.771+0000 7f6ffdd9b640 1 --2- 172.21.15.98:0/144256398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70000cfad0 0x7f7000158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:42.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.771+0000 7f6ffed9d640 1 -- 172.21.15.98:0/144256398 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f700006c1d0 msgr2=0x7f700014d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:42.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.771+0000 7f6ffed9d640 1 --2- 172.21.15.98:0/144256398 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f700006c1d0 0x7f700014d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:42.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.771+0000 7f6ffed9d640 1 -- 172.21.15.98:0/144256398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70000cfad0 msgr2=0x7f7000158040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:42.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.771+0000 7f6ffed9d640 1 --2- 172.21.15.98:0/144256398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70000cfad0 0x7f7000158040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:42.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.771+0000 7f6ffed9d640 1 -- 172.21.15.98:0/144256398 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f700014d780 con 0x7f7000158580 2024-09-19T08:39:42.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.771+0000 7f6ffdd9b640 1 --2- 172.21.15.98:0/144256398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70000cfad0 0x7f7000158040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:42.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.771+0000 7f6ffed9d640 1 --2- 172.21.15.98:0/144256398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7000158580 0x7f700014ef00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6ff800cd80 tx=0x7f6ff800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:42.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.771+0000 7f6feeffd640 1 -- 172.21.15.98:0/144256398 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff800f680 con 0x7f7000158580 2024-09-19T08:39:42.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.771+0000 7f6feeffd640 1 -- 172.21.15.98:0/144256398 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ff800f840 con 0x7f7000158580 2024-09-19T08:39:42.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.771+0000 7f70049f0640 1 -- 172.21.15.98:0/144256398 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f700014fbf0 con 0x7f7000158580 2024-09-19T08:39:42.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.771+0000 7f6feeffd640 1 -- 172.21.15.98:0/144256398 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff800fc30 con 0x7f7000158580 2024-09-19T08:39:42.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.771+0000 7f70049f0640 1 -- 172.21.15.98:0/144256398 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7000151400 con 0x7f7000158580 2024-09-19T08:39:42.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.771+0000 7f70049f0640 1 -- 172.21.15.98:0/144256398 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7000150f10 con 0x7f7000158580 2024-09-19T08:39:42.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.771+0000 7f6feeffd640 1 -- 172.21.15.98:0/144256398 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f6ff80213f0 con 0x7f7000158580 2024-09-19T08:39:42.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.771+0000 7f6feeffd640 1 --2- 172.21.15.98:0/144256398 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6fe0067aa0 0x7f6fe0069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:42.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.771+0000 7f6ffe59c640 1 --2- 172.21.15.98:0/144256398 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6fe0067aa0 0x7f6fe0069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:42.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.771+0000 7f6feeffd640 1 -- 172.21.15.98:0/144256398 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f6ff8086e50 con 0x7f7000158580 2024-09-19T08:39:42.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.771+0000 7f6ffe59c640 1 --2- 172.21.15.98:0/144256398 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6fe0067aa0 0x7f6fe0069f60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00ad000 tx=0x7f6ff0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:42.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.775+0000 7f6feeffd640 1 -- 172.21.15.98:0/144256398 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6ff8014030 con 0x7f7000158580 2024-09-19T08:39:42.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.923+0000 7f70049f0640 1 -- 172.21.15.98:0/144256398 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f700006bc90 con 0x7f6fe0067aa0 2024-09-19T08:39:42.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.931+0000 7f6feeffd640 1 -- 172.21.15.98:0/144256398 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249325 (secure 0 0 0) 0x7f700006bc90 con 0x7f6fe0067aa0 2024-09-19T08:39:42.933 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:39:42.935 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:39:42.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.935+0000 7f70049f0640 1 -- 172.21.15.98:0/144256398 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6fe0067aa0 msgr2=0x7f6fe0069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:42.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.935+0000 7f70049f0640 1 --2- 172.21.15.98:0/144256398 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6fe0067aa0 0x7f6fe0069f60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00ad000 tx=0x7f6ff0008040 comp rx=0 tx=0).stop 2024-09-19T08:39:42.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.935+0000 7f70049f0640 1 -- 172.21.15.98:0/144256398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7000158580 msgr2=0x7f700014ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:42.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.935+0000 7f70049f0640 1 --2- 172.21.15.98:0/144256398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7000158580 0x7f700014ef00 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6ff800cd80 tx=0x7f6ff800a650 comp rx=0 tx=0).stop 2024-09-19T08:39:42.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.935+0000 7f70049f0640 1 -- 172.21.15.98:0/144256398 shutdown_connections 2024-09-19T08:39:42.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.935+0000 7f70049f0640 1 --2- 172.21.15.98:0/144256398 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6fe0067aa0 0x7f6fe0069f60 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:42.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.935+0000 7f70049f0640 1 --2- 172.21.15.98:0/144256398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7000158580 0x7f700014ef00 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:42.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.935+0000 7f70049f0640 1 --2- 172.21.15.98:0/144256398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70000cfad0 0x7f7000158040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:42.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.935+0000 7f70049f0640 1 --2- 172.21.15.98:0/144256398 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f700006c1d0 0x7f700014d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:42.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.935+0000 7f70049f0640 1 -- 172.21.15.98:0/144256398 >> 172.21.15.98:0/144256398 conn(0x7f70000b2010 msgr2=0x7f7000067a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:42.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.935+0000 7f70049f0640 1 -- 172.21.15.98:0/144256398 shutdown_connections 2024-09-19T08:39:42.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:42.935+0000 7f70049f0640 1 -- 172.21.15.98:0/144256398 wait complete. 2024-09-19T08:39:42.950 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2024-09-19T08:39:42.050723+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592162,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":195,"num_write_kb":2422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":170,"acting":170,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":30,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":229392,"kb_used_data":4832,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754745328,"statfs":{"total":773094113280,"available":772859215872,"internally_reserved":0,"allocated":4947968,"data_stored":2661275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1794,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.166397"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.958882+0000","last_change":"2024-09-19T08:39:38.938293+0000","last_active":"2024-09-19T08:39:38.958882+0000","last_peered":"2024-09-19T08:39:38.958882+0000","last_clean":"2024-09-19T08:39:38.958882+0000","last_became_active":"2024-09-19T08:39:38.936784+0000","last_became_peered":"2024-09-19T08:39:38.936784+0000","last_unstale":"2024-09-19T08:39:38.958882+0000","last_undegraded":"2024-09-19T08:39:38.958882+0000","last_fullsized":"2024-09-19T08:39:38.958882+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:08:18.820654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.944459+0000","last_change":"2024-09-19T08:39:38.942347+0000","last_active":"2024-09-19T08:39:38.944459+0000","last_peered":"2024-09-19T08:39:38.944459+0000","last_clean":"2024-09-19T08:39:38.944459+0000","last_became_active":"2024-09-19T08:39:38.941641+0000","last_became_peered":"2024-09-19T08:39:38.941641+0000","last_unstale":"2024-09-19T08:39:38.944459+0000","last_undegraded":"2024-09-19T08:39:38.944459+0000","last_fullsized":"2024-09-19T08:39:38.944459+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:29.732835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.933674+0000","last_change":"2024-09-19T08:39:38.918421+0000","last_active":"2024-09-19T08:39:38.933674+0000","last_peered":"2024-09-19T08:39:38.933674+0000","last_clean":"2024-09-19T08:39:38.933674+0000","last_became_active":"2024-09-19T08:39:38.917761+0000","last_became_peered":"2024-09-19T08:39:38.917761+0000","last_unstale":"2024-09-19T08:39:38.933674+0000","last_undegraded":"2024-09-19T08:39:38.933674+0000","last_fullsized":"2024-09-19T08:39:38.933674+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:31.825354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.936913+0000","last_change":"2024-09-19T08:39:38.932965+0000","last_active":"2024-09-19T08:39:38.936913+0000","last_peered":"2024-09-19T08:39:38.936913+0000","last_clean":"2024-09-19T08:39:38.936913+0000","last_became_active":"2024-09-19T08:39:38.929490+0000","last_became_peered":"2024-09-19T08:39:38.929490+0000","last_unstale":"2024-09-19T08:39:38.936913+0000","last_undegraded":"2024-09-19T08:39:38.936913+0000","last_fullsized":"2024-09-19T08:39:38.936913+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:30:09.863827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.942469+0000","last_change":"2024-09-19T08:39:38.938221+0000","last_active":"2024-09-19T08:39:38.942469+0000","last_peered":"2024-09-19T08:39:38.942469+0000","last_clean":"2024-09-19T08:39:38.942469+0000","last_became_active":"2024-09-19T08:39:38.937356+0000","last_became_peered":"2024-09-19T08:39:38.937356+0000","last_unstale":"2024-09-19T08:39:38.942469+0000","last_undegraded":"2024-09-19T08:39:38.942469+0000","last_fullsized":"2024-09-19T08:39:38.942469+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:00:09.742802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-09-19T08:39:37.898206+0000","last_change":"2024-09-19T08:39:37.898206+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.898206+0000","last_undegraded":"2024-09-19T08:39:37.898206+0000","last_fullsized":"2024-09-19T08:39:37.898206+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.961008+0000","last_change":"2024-09-19T08:39:38.945329+0000","last_active":"2024-09-19T08:39:38.961008+0000","last_peered":"2024-09-19T08:39:38.961008+0000","last_clean":"2024-09-19T08:39:38.961008+0000","last_became_active":"2024-09-19T08:39:38.944121+0000","last_became_peered":"2024-09-19T08:39:38.944121+0000","last_unstale":"2024-09-19T08:39:38.961008+0000","last_undegraded":"2024-09-19T08:39:38.961008+0000","last_fullsized":"2024-09-19T08:39:38.961008+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:37:14.987023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.937026+0000","last_change":"2024-09-19T08:39:38.929792+0000","last_active":"2024-09-19T08:39:38.937026+0000","last_peered":"2024-09-19T08:39:38.937026+0000","last_clean":"2024-09-19T08:39:38.937026+0000","last_became_active":"2024-09-19T08:39:38.928743+0000","last_became_peered":"2024-09-19T08:39:38.928743+0000","last_unstale":"2024-09-19T08:39:38.937026+0000","last_undegraded":"2024-09-19T08:39:38.937026+0000","last_fullsized":"2024-09-19T08:39:38.937026+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:14:32.273857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.937132+0000","last_change":"2024-09-19T08:39:38.934773+0000","last_active":"2024-09-19T08:39:38.937132+0000","last_peered":"2024-09-19T08:39:38.937132+0000","last_clean":"2024-09-19T08:39:38.937132+0000","last_became_active":"2024-09-19T08:39:38.934054+0000","last_became_peered":"2024-09-19T08:39:38.934054+0000","last_unstale":"2024-09-19T08:39:38.937132+0000","last_undegraded":"2024-09-19T08:39:38.937132+0000","last_fullsized":"2024-09-19T08:39:38.937132+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:31.825354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.937106+0000","last_change":"2024-09-19T08:39:38.934619+0000","last_active":"2024-09-19T08:39:38.937106+0000","last_peered":"2024-09-19T08:39:38.937106+0000","last_clean":"2024-09-19T08:39:38.937106+0000","last_became_active":"2024-09-19T08:39:38.934244+0000","last_became_peered":"2024-09-19T08:39:38.934244+0000","last_unstale":"2024-09-19T08:39:38.937106+0000","last_undegraded":"2024-09-19T08:39:38.937106+0000","last_fullsized":"2024-09-19T08:39:38.937106+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:08:44.560138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.937073+0000","last_change":"2024-09-19T08:39:38.933595+0000","last_active":"2024-09-19T08:39:38.937073+0000","last_peered":"2024-09-19T08:39:38.937073+0000","last_clean":"2024-09-19T08:39:38.937073+0000","last_became_active":"2024-09-19T08:39:38.933316+0000","last_became_peered":"2024-09-19T08:39:38.933316+0000","last_unstale":"2024-09-19T08:39:38.937073+0000","last_undegraded":"2024-09-19T08:39:38.937073+0000","last_fullsized":"2024-09-19T08:39:38.937073+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:43.876237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.928186+0000","last_change":"2024-09-19T08:39:38.923904+0000","last_active":"2024-09-19T08:39:38.928186+0000","last_peered":"2024-09-19T08:39:38.928186+0000","last_clean":"2024-09-19T08:39:38.928186+0000","last_became_active":"2024-09-19T08:39:38.923135+0000","last_became_peered":"2024-09-19T08:39:38.923135+0000","last_unstale":"2024-09-19T08:39:38.928186+0000","last_undegraded":"2024-09-19T08:39:38.928186+0000","last_fullsized":"2024-09-19T08:39:38.928186+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:06.457317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.938802+0000","last_change":"2024-09-19T08:39:38.933031+0000","last_active":"2024-09-19T08:39:38.938802+0000","last_peered":"2024-09-19T08:39:38.938802+0000","last_clean":"2024-09-19T08:39:38.938802+0000","last_became_active":"2024-09-19T08:39:38.932568+0000","last_became_peered":"2024-09-19T08:39:38.932568+0000","last_unstale":"2024-09-19T08:39:38.938802+0000","last_undegraded":"2024-09-19T08:39:38.938802+0000","last_fullsized":"2024-09-19T08:39:38.938802+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:41:56.688886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.940685+0000","last_change":"2024-09-19T08:39:38.936306+0000","last_active":"2024-09-19T08:39:38.940685+0000","last_peered":"2024-09-19T08:39:38.940685+0000","last_clean":"2024-09-19T08:39:38.940685+0000","last_became_active":"2024-09-19T08:39:38.935988+0000","last_became_peered":"2024-09-19T08:39:38.935988+0000","last_unstale":"2024-09-19T08:39:38.940685+0000","last_undegraded":"2024-09-19T08:39:38.940685+0000","last_fullsized":"2024-09-19T08:39:38.940685+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:49:14.068588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-09-19T08:39:37.899012+0000","last_change":"2024-09-19T08:39:37.899012+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.899012+0000","last_undegraded":"2024-09-19T08:39:37.899012+0000","last_fullsized":"2024-09-19T08:39:37.899012+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:37.883250+0000","last_change":"2024-09-19T08:39:37.883250+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.883250+0000","last_undegraded":"2024-09-19T08:39:37.883250+0000","last_fullsized":"2024-09-19T08:39:37.883250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.933907+0000","last_change":"2024-09-19T08:39:38.924533+0000","last_active":"2024-09-19T08:39:38.933907+0000","last_peered":"2024-09-19T08:39:38.933907+0000","last_clean":"2024-09-19T08:39:38.933907+0000","last_became_active":"2024-09-19T08:39:38.923817+0000","last_became_peered":"2024-09-19T08:39:38.923817+0000","last_unstale":"2024-09-19T08:39:38.933907+0000","last_undegraded":"2024-09-19T08:39:38.933907+0000","last_fullsized":"2024-09-19T08:39:38.933907+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:29.744836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.943119+0000","last_change":"2024-09-19T08:39:38.938932+0000","last_active":"2024-09-19T08:39:38.943119+0000","last_peered":"2024-09-19T08:39:38.943119+0000","last_clean":"2024-09-19T08:39:38.943119+0000","last_became_active":"2024-09-19T08:39:38.937330+0000","last_became_peered":"2024-09-19T08:39:38.937330+0000","last_unstale":"2024-09-19T08:39:38.943119+0000","last_undegraded":"2024-09-19T08:39:38.943119+0000","last_fullsized":"2024-09-19T08:39:38.943119+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:06.457317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:37.883250+0000","last_change":"2024-09-19T08:39:37.883250+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.883250+0000","last_undegraded":"2024-09-19T08:39:37.883250+0000","last_fullsized":"2024-09-19T08:39:37.883250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.938851+0000","last_change":"2024-09-19T08:39:38.936241+0000","last_active":"2024-09-19T08:39:38.938851+0000","last_peered":"2024-09-19T08:39:38.938851+0000","last_clean":"2024-09-19T08:39:38.938851+0000","last_became_active":"2024-09-19T08:39:38.935936+0000","last_became_peered":"2024-09-19T08:39:38.935936+0000","last_unstale":"2024-09-19T08:39:38.938851+0000","last_undegraded":"2024-09-19T08:39:38.938851+0000","last_fullsized":"2024-09-19T08:39:38.938851+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:49.679363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-09-19T08:39:37.898343+0000","last_change":"2024-09-19T08:39:37.898343+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.898343+0000","last_undegraded":"2024-09-19T08:39:37.898343+0000","last_fullsized":"2024-09-19T08:39:37.898343+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.938460+0000","last_change":"2024-09-19T08:39:38.935680+0000","last_active":"2024-09-19T08:39:38.938460+0000","last_peered":"2024-09-19T08:39:38.938460+0000","last_clean":"2024-09-19T08:39:38.938460+0000","last_became_active":"2024-09-19T08:39:38.929257+0000","last_became_peered":"2024-09-19T08:39:38.929257+0000","last_unstale":"2024-09-19T08:39:38.938460+0000","last_undegraded":"2024-09-19T08:39:38.938460+0000","last_fullsized":"2024-09-19T08:39:38.938460+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:23.048070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:37.883250+0000","last_change":"2024-09-19T08:39:37.883250+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.883250+0000","last_undegraded":"2024-09-19T08:39:37.883250+0000","last_fullsized":"2024-09-19T08:39:37.883250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:37.883250+0000","last_change":"2024-09-19T08:39:37.883250+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.883250+0000","last_undegraded":"2024-09-19T08:39:37.883250+0000","last_fullsized":"2024-09-19T08:39:37.883250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:37.883250+0000","last_change":"2024-09-19T08:39:37.883250+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.883250+0000","last_undegraded":"2024-09-19T08:39:37.883250+0000","last_fullsized":"2024-09-19T08:39:37.883250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.938757+0000","last_change":"2024-09-19T08:39:38.932467+0000","last_active":"2024-09-19T08:39:38.938757+0000","last_peered":"2024-09-19T08:39:38.938757+0000","last_clean":"2024-09-19T08:39:38.938757+0000","last_became_active":"2024-09-19T08:39:38.932023+0000","last_became_peered":"2024-09-19T08:39:38.932023+0000","last_unstale":"2024-09-19T08:39:38.938757+0000","last_undegraded":"2024-09-19T08:39:38.938757+0000","last_fullsized":"2024-09-19T08:39:38.938757+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:11.986036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.933854+0000","last_change":"2024-09-19T08:39:38.924547+0000","last_active":"2024-09-19T08:39:38.933854+0000","last_peered":"2024-09-19T08:39:38.933854+0000","last_clean":"2024-09-19T08:39:38.933854+0000","last_became_active":"2024-09-19T08:39:38.923802+0000","last_became_peered":"2024-09-19T08:39:38.923802+0000","last_unstale":"2024-09-19T08:39:38.933854+0000","last_undegraded":"2024-09-19T08:39:38.933854+0000","last_fullsized":"2024-09-19T08:39:38.933854+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:23.048070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.933859+0000","last_change":"2024-09-19T08:39:38.927516+0000","last_active":"2024-09-19T08:39:38.933859+0000","last_peered":"2024-09-19T08:39:38.933859+0000","last_clean":"2024-09-19T08:39:38.933859+0000","last_became_active":"2024-09-19T08:39:38.926015+0000","last_became_peered":"2024-09-19T08:39:38.926015+0000","last_unstale":"2024-09-19T08:39:38.933859+0000","last_undegraded":"2024-09-19T08:39:38.933859+0000","last_fullsized":"2024-09-19T08:39:38.933859+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:33:44.717869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:37.883250+0000","last_change":"2024-09-19T08:39:37.883250+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.883250+0000","last_undegraded":"2024-09-19T08:39:37.883250+0000","last_fullsized":"2024-09-19T08:39:37.883250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.934797+0000","last_change":"2024-09-19T08:39:38.931320+0000","last_active":"2024-09-19T08:39:38.934797+0000","last_peered":"2024-09-19T08:39:38.934797+0000","last_clean":"2024-09-19T08:39:38.934797+0000","last_became_active":"2024-09-19T08:39:38.930578+0000","last_became_peered":"2024-09-19T08:39:38.930578+0000","last_unstale":"2024-09-19T08:39:38.934797+0000","last_undegraded":"2024-09-19T08:39:38.934797+0000","last_fullsized":"2024-09-19T08:39:38.934797+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:49:14.068588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-09-19T08:39:37.897514+0000","last_change":"2024-09-19T08:39:37.897514+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.897514+0000","last_undegraded":"2024-09-19T08:39:37.897514+0000","last_fullsized":"2024-09-19T08:39:37.897514+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.930583+0000","last_change":"2024-09-19T08:39:38.921905+0000","last_active":"2024-09-19T08:39:38.930583+0000","last_peered":"2024-09-19T08:39:38.930583+0000","last_clean":"2024-09-19T08:39:38.930583+0000","last_became_active":"2024-09-19T08:39:38.920862+0000","last_became_peered":"2024-09-19T08:39:38.920862+0000","last_unstale":"2024-09-19T08:39:38.930583+0000","last_undegraded":"2024-09-19T08:39:38.930583+0000","last_fullsized":"2024-09-19T08:39:38.930583+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:43:08.445843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.950250+0000","last_change":"2024-09-19T08:39:38.942801+0000","last_active":"2024-09-19T08:39:38.950250+0000","last_peered":"2024-09-19T08:39:38.950250+0000","last_clean":"2024-09-19T08:39:38.950250+0000","last_became_active":"2024-09-19T08:39:38.942568+0000","last_became_peered":"2024-09-19T08:39:38.942568+0000","last_unstale":"2024-09-19T08:39:38.950250+0000","last_undegraded":"2024-09-19T08:39:38.950250+0000","last_fullsized":"2024-09-19T08:39:38.950250+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:31.825354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-19T08:39:37.942662+0000","last_change":"2024-09-19T08:39:37.937397+0000","last_active":"2024-09-19T08:39:37.942662+0000","last_peered":"2024-09-19T08:39:37.942662+0000","last_clean":"2024-09-19T08:39:37.942662+0000","last_became_active":"2024-09-19T08:39:37.937128+0000","last_became_peered":"2024-09-19T08:39:37.937128+0000","last_unstale":"2024-09-19T08:39:37.942662+0000","last_undegraded":"2024-09-19T08:39:37.942662+0000","last_fullsized":"2024-09-19T08:39:37.942662+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:47.949747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:37.883250+0000","last_change":"2024-09-19T08:39:37.883250+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.883250+0000","last_undegraded":"2024-09-19T08:39:37.883250+0000","last_fullsized":"2024-09-19T08:39:37.883250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"42'1","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.959478+0000","last_change":"2024-09-19T08:39:37.932635+0000","last_active":"2024-09-19T08:39:38.959478+0000","last_peered":"2024-09-19T08:39:38.959478+0000","last_clean":"2024-09-19T08:39:38.959478+0000","last_became_active":"2024-09-19T08:39:37.931841+0000","last_became_peered":"2024-09-19T08:39:37.931841+0000","last_unstale":"2024-09-19T08:39:38.959478+0000","last_undegraded":"2024-09-19T08:39:38.959478+0000","last_fullsized":"2024-09-19T08:39:38.959478+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:42.872698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.943117+0000","last_change":"2024-09-19T08:39:38.940685+0000","last_active":"2024-09-19T08:39:38.943117+0000","last_peered":"2024-09-19T08:39:38.943117+0000","last_clean":"2024-09-19T08:39:38.943117+0000","last_became_active":"2024-09-19T08:39:38.935293+0000","last_became_peered":"2024-09-19T08:39:38.935293+0000","last_unstale":"2024-09-19T08:39:38.943117+0000","last_undegraded":"2024-09-19T08:39:38.943117+0000","last_fullsized":"2024-09-19T08:39:38.943117+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:08:18.820654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.956642+0000","last_change":"2024-09-19T08:39:37.917790+0000","last_active":"2024-09-19T08:39:38.956642+0000","last_peered":"2024-09-19T08:39:38.956642+0000","last_clean":"2024-09-19T08:39:38.956642+0000","last_became_active":"2024-09-19T08:39:37.917479+0000","last_became_peered":"2024-09-19T08:39:37.917479+0000","last_unstale":"2024-09-19T08:39:38.956642+0000","last_undegraded":"2024-09-19T08:39:38.956642+0000","last_fullsized":"2024-09-19T08:39:38.956642+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:30:08.860289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.927097+0000","last_change":"2024-09-19T08:39:38.920238+0000","last_active":"2024-09-19T08:39:38.927097+0000","last_peered":"2024-09-19T08:39:38.927097+0000","last_clean":"2024-09-19T08:39:38.927097+0000","last_became_active":"2024-09-19T08:39:38.919486+0000","last_became_peered":"2024-09-19T08:39:38.919486+0000","last_unstale":"2024-09-19T08:39:38.927097+0000","last_undegraded":"2024-09-19T08:39:38.927097+0000","last_fullsized":"2024-09-19T08:39:38.927097+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:43.876237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.961365+0000","last_change":"2024-09-19T08:39:37.927771+0000","last_active":"2024-09-19T08:39:38.961365+0000","last_peered":"2024-09-19T08:39:38.961365+0000","last_clean":"2024-09-19T08:39:38.961365+0000","last_became_active":"2024-09-19T08:39:37.926946+0000","last_became_peered":"2024-09-19T08:39:37.926946+0000","last_unstale":"2024-09-19T08:39:38.961365+0000","last_undegraded":"2024-09-19T08:39:38.961365+0000","last_fullsized":"2024-09-19T08:39:38.961365+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:39.361088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.933755+0000","last_change":"2024-09-19T08:39:38.918556+0000","last_active":"2024-09-19T08:39:38.933755+0000","last_peered":"2024-09-19T08:39:38.933755+0000","last_clean":"2024-09-19T08:39:38.933755+0000","last_became_active":"2024-09-19T08:39:38.917830+0000","last_became_peered":"2024-09-19T08:39:38.917830+0000","last_unstale":"2024-09-19T08:39:38.933755+0000","last_undegraded":"2024-09-19T08:39:38.933755+0000","last_fullsized":"2024-09-19T08:39:38.933755+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:33.862515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-19T08:39:37.919363+0000","last_change":"2024-09-19T08:39:37.913524+0000","last_active":"2024-09-19T08:39:37.919363+0000","last_peered":"2024-09-19T08:39:37.919363+0000","last_clean":"2024-09-19T08:39:37.919363+0000","last_became_active":"2024-09-19T08:39:37.912853+0000","last_became_peered":"2024-09-19T08:39:37.912853+0000","last_unstale":"2024-09-19T08:39:37.919363+0000","last_undegraded":"2024-09-19T08:39:37.919363+0000","last_fullsized":"2024-09-19T08:39:37.919363+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:24.628377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.936990+0000","last_change":"2024-09-19T08:39:38.929719+0000","last_active":"2024-09-19T08:39:38.936990+0000","last_peered":"2024-09-19T08:39:38.936990+0000","last_clean":"2024-09-19T08:39:38.936990+0000","last_became_active":"2024-09-19T08:39:38.929158+0000","last_became_peered":"2024-09-19T08:39:38.929158+0000","last_unstale":"2024-09-19T08:39:38.936990+0000","last_undegraded":"2024-09-19T08:39:38.936990+0000","last_fullsized":"2024-09-19T08:39:38.936990+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:07:16.957999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"42'1","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.958601+0000","last_change":"2024-09-19T08:39:37.929415+0000","last_active":"2024-09-19T08:39:38.958601+0000","last_peered":"2024-09-19T08:39:38.958601+0000","last_clean":"2024-09-19T08:39:38.958601+0000","last_became_active":"2024-09-19T08:39:37.928610+0000","last_became_peered":"2024-09-19T08:39:37.928610+0000","last_unstale":"2024-09-19T08:39:38.958601+0000","last_undegraded":"2024-09-19T08:39:38.958601+0000","last_fullsized":"2024-09-19T08:39:38.958601+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:36.032108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.936735+0000","last_change":"2024-09-19T08:39:38.928185+0000","last_active":"2024-09-19T08:39:38.936735+0000","last_peered":"2024-09-19T08:39:38.936735+0000","last_clean":"2024-09-19T08:39:38.936735+0000","last_became_active":"2024-09-19T08:39:38.924812+0000","last_became_peered":"2024-09-19T08:39:38.924812+0000","last_unstale":"2024-09-19T08:39:38.936735+0000","last_undegraded":"2024-09-19T08:39:38.936735+0000","last_fullsized":"2024-09-19T08:39:38.936735+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:08.958729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.912110+0000","last_change":"2024-09-19T08:39:37.921882+0000","last_active":"2024-09-19T08:39:38.912110+0000","last_peered":"2024-09-19T08:39:38.912110+0000","last_clean":"2024-09-19T08:39:38.912110+0000","last_became_active":"2024-09-19T08:39:37.921061+0000","last_became_peered":"2024-09-19T08:39:37.921061+0000","last_unstale":"2024-09-19T08:39:38.912110+0000","last_undegraded":"2024-09-19T08:39:38.912110+0000","last_fullsized":"2024-09-19T08:39:38.912110+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:49:13.065050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:37.883250+0000","last_change":"2024-09-19T08:39:37.883250+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.883250+0000","last_undegraded":"2024-09-19T08:39:37.883250+0000","last_fullsized":"2024-09-19T08:39:37.883250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.902185+0000","last_change":"2024-09-19T08:39:37.914508+0000","last_active":"2024-09-19T08:39:38.902185+0000","last_peered":"2024-09-19T08:39:38.902185+0000","last_clean":"2024-09-19T08:39:38.902185+0000","last_became_active":"2024-09-19T08:39:37.914186+0000","last_became_peered":"2024-09-19T08:39:37.914186+0000","last_unstale":"2024-09-19T08:39:38.902185+0000","last_undegraded":"2024-09-19T08:39:38.902185+0000","last_fullsized":"2024-09-19T08:39:38.902185+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:48:49.023614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.943169+0000","last_change":"2024-09-19T08:39:38.939531+0000","last_active":"2024-09-19T08:39:38.943169+0000","last_peered":"2024-09-19T08:39:38.943169+0000","last_clean":"2024-09-19T08:39:38.943169+0000","last_became_active":"2024-09-19T08:39:38.938231+0000","last_became_peered":"2024-09-19T08:39:38.938231+0000","last_unstale":"2024-09-19T08:39:38.943169+0000","last_undegraded":"2024-09-19T08:39:38.943169+0000","last_fullsized":"2024-09-19T08:39:38.943169+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:33:44.717869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.901355+0000","last_change":"2024-09-19T08:39:37.930095+0000","last_active":"2024-09-19T08:39:38.901355+0000","last_peered":"2024-09-19T08:39:38.901355+0000","last_clean":"2024-09-19T08:39:38.901355+0000","last_became_active":"2024-09-19T08:39:37.921804+0000","last_became_peered":"2024-09-19T08:39:37.921804+0000","last_unstale":"2024-09-19T08:39:38.901355+0000","last_undegraded":"2024-09-19T08:39:38.901355+0000","last_fullsized":"2024-09-19T08:39:38.901355+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:30:08.860289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.958802+0000","last_change":"2024-09-19T08:39:38.941102+0000","last_active":"2024-09-19T08:39:38.958802+0000","last_peered":"2024-09-19T08:39:38.958802+0000","last_clean":"2024-09-19T08:39:38.958802+0000","last_became_active":"2024-09-19T08:39:38.937500+0000","last_became_peered":"2024-09-19T08:39:38.937500+0000","last_unstale":"2024-09-19T08:39:38.958802+0000","last_undegraded":"2024-09-19T08:39:38.958802+0000","last_fullsized":"2024-09-19T08:39:38.958802+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:29.732835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.911564+0000","last_change":"2024-09-19T08:39:37.912838+0000","last_active":"2024-09-19T08:39:38.911564+0000","last_peered":"2024-09-19T08:39:38.911564+0000","last_clean":"2024-09-19T08:39:38.911564+0000","last_became_active":"2024-09-19T08:39:37.912442+0000","last_became_peered":"2024-09-19T08:39:37.912442+0000","last_unstale":"2024-09-19T08:39:38.911564+0000","last_undegraded":"2024-09-19T08:39:38.911564+0000","last_fullsized":"2024-09-19T08:39:38.911564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:46.135290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-09-19T08:39:37.896926+0000","last_change":"2024-09-19T08:39:37.896926+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.896926+0000","last_undegraded":"2024-09-19T08:39:37.896926+0000","last_fullsized":"2024-09-19T08:39:37.896926+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"42'3","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.961471+0000","last_change":"2024-09-19T08:39:37.936425+0000","last_active":"2024-09-19T08:39:38.961471+0000","last_peered":"2024-09-19T08:39:38.961471+0000","last_clean":"2024-09-19T08:39:38.961471+0000","last_became_active":"2024-09-19T08:39:37.935418+0000","last_became_peered":"2024-09-19T08:39:37.935418+0000","last_unstale":"2024-09-19T08:39:38.961471+0000","last_undegraded":"2024-09-19T08:39:38.961471+0000","last_fullsized":"2024-09-19T08:39:38.961471+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:05.453778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.958959+0000","last_change":"2024-09-19T08:39:38.942176+0000","last_active":"2024-09-19T08:39:38.958959+0000","last_peered":"2024-09-19T08:39:38.958959+0000","last_clean":"2024-09-19T08:39:38.958959+0000","last_became_active":"2024-09-19T08:39:38.939625+0000","last_became_peered":"2024-09-19T08:39:38.939625+0000","last_unstale":"2024-09-19T08:39:38.958959+0000","last_undegraded":"2024-09-19T08:39:38.958959+0000","last_fullsized":"2024-09-19T08:39:38.958959+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:22:21.538914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"42'6","reported_seq":18,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.962495+0000","last_change":"2024-09-19T08:39:37.921227+0000","last_active":"2024-09-19T08:39:38.962495+0000","last_peered":"2024-09-19T08:39:38.962495+0000","last_clean":"2024-09-19T08:39:38.962495+0000","last_became_active":"2024-09-19T08:39:37.920773+0000","last_became_peered":"2024-09-19T08:39:37.920773+0000","last_unstale":"2024-09-19T08:39:38.962495+0000","last_undegraded":"2024-09-19T08:39:38.962495+0000","last_fullsized":"2024-09-19T08:39:38.962495+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:24:07.528992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:37.883250+0000","last_change":"2024-09-19T08:39:37.883250+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.883250+0000","last_undegraded":"2024-09-19T08:39:37.883250+0000","last_fullsized":"2024-09-19T08:39:37.883250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-19T08:39:37.912133+0000","last_change":"2024-09-19T08:39:37.905231+0000","last_active":"2024-09-19T08:39:37.912133+0000","last_peered":"2024-09-19T08:39:37.912133+0000","last_clean":"2024-09-19T08:39:37.912133+0000","last_became_active":"2024-09-19T08:39:37.904660+0000","last_became_peered":"2024-09-19T08:39:37.904660+0000","last_unstale":"2024-09-19T08:39:37.912133+0000","last_undegraded":"2024-09-19T08:39:37.912133+0000","last_fullsized":"2024-09-19T08:39:37.912133+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:38:48.929675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.940617+0000","last_change":"2024-09-19T08:39:38.936213+0000","last_active":"2024-09-19T08:39:38.940617+0000","last_peered":"2024-09-19T08:39:38.940617+0000","last_clean":"2024-09-19T08:39:38.940617+0000","last_became_active":"2024-09-19T08:39:38.935856+0000","last_became_peered":"2024-09-19T08:39:38.935856+0000","last_unstale":"2024-09-19T08:39:38.940617+0000","last_undegraded":"2024-09-19T08:39:38.940617+0000","last_fullsized":"2024-09-19T08:39:38.940617+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:29.744836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.908538+0000","last_change":"2024-09-19T08:39:37.914414+0000","last_active":"2024-09-19T08:39:38.908538+0000","last_peered":"2024-09-19T08:39:38.908538+0000","last_clean":"2024-09-19T08:39:38.908538+0000","last_became_active":"2024-09-19T08:39:37.913576+0000","last_became_peered":"2024-09-19T08:39:37.913576+0000","last_unstale":"2024-09-19T08:39:38.908538+0000","last_undegraded":"2024-09-19T08:39:38.908538+0000","last_fullsized":"2024-09-19T08:39:38.908538+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:28.741298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-09-19T08:39:37.896086+0000","last_change":"2024-09-19T08:39:37.896086+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.896086+0000","last_undegraded":"2024-09-19T08:39:37.896086+0000","last_fullsized":"2024-09-19T08:39:37.896086+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-19T08:39:37.912720+0000","last_change":"2024-09-19T08:39:37.910119+0000","last_active":"2024-09-19T08:39:37.912720+0000","last_peered":"2024-09-19T08:39:37.912720+0000","last_clean":"2024-09-19T08:39:37.912720+0000","last_became_active":"2024-09-19T08:39:37.909646+0000","last_became_peered":"2024-09-19T08:39:37.909646+0000","last_unstale":"2024-09-19T08:39:37.912720+0000","last_undegraded":"2024-09-19T08:39:37.912720+0000","last_fullsized":"2024-09-19T08:39:37.912720+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:30.311453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.959355+0000","last_change":"2024-09-19T08:39:38.937339+0000","last_active":"2024-09-19T08:39:38.959355+0000","last_peered":"2024-09-19T08:39:38.959355+0000","last_clean":"2024-09-19T08:39:38.959355+0000","last_became_active":"2024-09-19T08:39:38.936404+0000","last_became_peered":"2024-09-19T08:39:38.936404+0000","last_unstale":"2024-09-19T08:39:38.959355+0000","last_undegraded":"2024-09-19T08:39:38.959355+0000","last_fullsized":"2024-09-19T08:39:38.959355+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:00:09.742802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"42'8","reported_seq":20,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.957690+0000","last_change":"2024-09-19T08:39:37.918124+0000","last_active":"2024-09-19T08:39:38.957690+0000","last_peered":"2024-09-19T08:39:38.957690+0000","last_clean":"2024-09-19T08:39:38.957690+0000","last_became_active":"2024-09-19T08:39:37.917521+0000","last_became_peered":"2024-09-19T08:39:37.917521+0000","last_unstale":"2024-09-19T08:39:38.957690+0000","last_undegraded":"2024-09-19T08:39:38.957690+0000","last_fullsized":"2024-09-19T08:39:38.957690+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:38:39.741349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.937824+0000","last_change":"2024-09-19T08:39:38.926671+0000","last_active":"2024-09-19T08:39:38.937824+0000","last_peered":"2024-09-19T08:39:38.937824+0000","last_clean":"2024-09-19T08:39:38.937824+0000","last_became_active":"2024-09-19T08:39:38.925343+0000","last_became_peered":"2024-09-19T08:39:38.925343+0000","last_unstale":"2024-09-19T08:39:38.937824+0000","last_undegraded":"2024-09-19T08:39:38.937824+0000","last_fullsized":"2024-09-19T08:39:38.937824+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:41.168755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"42'1","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.949593+0000","last_change":"2024-09-19T08:39:37.911789+0000","last_active":"2024-09-19T08:39:38.949593+0000","last_peered":"2024-09-19T08:39:38.949593+0000","last_clean":"2024-09-19T08:39:38.949593+0000","last_became_active":"2024-09-19T08:39:37.911305+0000","last_became_peered":"2024-09-19T08:39:37.911305+0000","last_unstale":"2024-09-19T08:39:38.949593+0000","last_undegraded":"2024-09-19T08:39:38.949593+0000","last_fullsized":"2024-09-19T08:39:38.949593+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:13.592173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.938438+0000","last_change":"2024-09-19T08:39:38.935012+0000","last_active":"2024-09-19T08:39:38.938438+0000","last_peered":"2024-09-19T08:39:38.938438+0000","last_clean":"2024-09-19T08:39:38.938438+0000","last_became_active":"2024-09-19T08:39:38.934473+0000","last_became_peered":"2024-09-19T08:39:38.934473+0000","last_unstale":"2024-09-19T08:39:38.938438+0000","last_undegraded":"2024-09-19T08:39:38.938438+0000","last_fullsized":"2024-09-19T08:39:38.938438+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:33.862515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.959551+0000","last_change":"2024-09-19T08:39:37.923082+0000","last_active":"2024-09-19T08:39:38.959551+0000","last_peered":"2024-09-19T08:39:38.959551+0000","last_clean":"2024-09-19T08:39:38.959551+0000","last_became_active":"2024-09-19T08:39:37.922381+0000","last_became_peered":"2024-09-19T08:39:37.922381+0000","last_unstale":"2024-09-19T08:39:38.959551+0000","last_undegraded":"2024-09-19T08:39:38.959551+0000","last_fullsized":"2024-09-19T08:39:38.959551+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:08:43.556600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:37.883250+0000","last_change":"2024-09-19T08:39:37.883250+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.883250+0000","last_undegraded":"2024-09-19T08:39:37.883250+0000","last_fullsized":"2024-09-19T08:39:37.883250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"42'1","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.956915+0000","last_change":"2024-09-19T08:39:37.917129+0000","last_active":"2024-09-19T08:39:38.956915+0000","last_peered":"2024-09-19T08:39:38.956915+0000","last_clean":"2024-09-19T08:39:38.956915+0000","last_became_active":"2024-09-19T08:39:37.916360+0000","last_became_peered":"2024-09-19T08:39:37.916360+0000","last_unstale":"2024-09-19T08:39:38.956915+0000","last_undegraded":"2024-09-19T08:39:38.956915+0000","last_fullsized":"2024-09-19T08:39:38.956915+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:39.361088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:37.883250+0000","last_change":"2024-09-19T08:39:37.883250+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.883250+0000","last_undegraded":"2024-09-19T08:39:37.883250+0000","last_fullsized":"2024-09-19T08:39:37.883250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-19T08:39:37.937075+0000","last_change":"2024-09-19T08:39:37.932493+0000","last_active":"2024-09-19T08:39:37.937075+0000","last_peered":"2024-09-19T08:39:37.937075+0000","last_clean":"2024-09-19T08:39:37.937075+0000","last_became_active":"2024-09-19T08:39:37.932195+0000","last_became_peered":"2024-09-19T08:39:37.932195+0000","last_unstale":"2024-09-19T08:39:37.937075+0000","last_undegraded":"2024-09-19T08:39:37.937075+0000","last_fullsized":"2024-09-19T08:39:37.937075+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:32:30.169809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-09-19T08:39:37.896687+0000","last_change":"2024-09-19T08:39:37.896687+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.896687+0000","last_undegraded":"2024-09-19T08:39:37.896687+0000","last_fullsized":"2024-09-19T08:39:37.896687+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-19T08:39:37.919394+0000","last_change":"2024-09-19T08:39:37.913727+0000","last_active":"2024-09-19T08:39:37.919394+0000","last_peered":"2024-09-19T08:39:37.919394+0000","last_clean":"2024-09-19T08:39:37.919394+0000","last_became_active":"2024-09-19T08:39:37.913224+0000","last_became_peered":"2024-09-19T08:39:37.913224+0000","last_unstale":"2024-09-19T08:39:37.919394+0000","last_undegraded":"2024-09-19T08:39:37.919394+0000","last_fullsized":"2024-09-19T08:39:37.919394+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:54:58.881538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.928775+0000","last_change":"2024-09-19T08:39:38.923972+0000","last_active":"2024-09-19T08:39:38.928775+0000","last_peered":"2024-09-19T08:39:38.928775+0000","last_clean":"2024-09-19T08:39:38.928775+0000","last_became_active":"2024-09-19T08:39:38.923101+0000","last_became_peered":"2024-09-19T08:39:38.923101+0000","last_unstale":"2024-09-19T08:39:38.928775+0000","last_undegraded":"2024-09-19T08:39:38.928775+0000","last_fullsized":"2024-09-19T08:39:38.928775+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:08:44.560138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.911722+0000","last_change":"2024-09-19T08:39:37.934259+0000","last_active":"2024-09-19T08:39:38.911722+0000","last_peered":"2024-09-19T08:39:38.911722+0000","last_clean":"2024-09-19T08:39:38.911722+0000","last_became_active":"2024-09-19T08:39:37.933506+0000","last_became_peered":"2024-09-19T08:39:37.933506+0000","last_unstale":"2024-09-19T08:39:38.911722+0000","last_undegraded":"2024-09-19T08:39:38.911722+0000","last_fullsized":"2024-09-19T08:39:38.911722+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:22.044532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.948894+0000","last_change":"2024-09-19T08:39:38.944450+0000","last_active":"2024-09-19T08:39:38.948894+0000","last_peered":"2024-09-19T08:39:38.948894+0000","last_clean":"2024-09-19T08:39:38.948894+0000","last_became_active":"2024-09-19T08:39:38.944108+0000","last_became_peered":"2024-09-19T08:39:38.944108+0000","last_unstale":"2024-09-19T08:39:38.948894+0000","last_undegraded":"2024-09-19T08:39:38.948894+0000","last_fullsized":"2024-09-19T08:39:38.948894+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:56.145227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.901835+0000","last_change":"2024-09-19T08:39:37.922726+0000","last_active":"2024-09-19T08:39:38.901835+0000","last_peered":"2024-09-19T08:39:38.901835+0000","last_clean":"2024-09-19T08:39:38.901835+0000","last_became_active":"2024-09-19T08:39:37.922312+0000","last_became_peered":"2024-09-19T08:39:37.922312+0000","last_unstale":"2024-09-19T08:39:38.901835+0000","last_undegraded":"2024-09-19T08:39:38.901835+0000","last_fullsized":"2024-09-19T08:39:38.901835+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:05:28.346760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.935437+0000","last_change":"2024-09-19T08:39:38.920148+0000","last_active":"2024-09-19T08:39:38.935437+0000","last_peered":"2024-09-19T08:39:38.935437+0000","last_clean":"2024-09-19T08:39:38.935437+0000","last_became_active":"2024-09-19T08:39:38.919337+0000","last_became_peered":"2024-09-19T08:39:38.919337+0000","last_unstale":"2024-09-19T08:39:38.935437+0000","last_undegraded":"2024-09-19T08:39:38.935437+0000","last_fullsized":"2024-09-19T08:39:38.935437+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:27:20.365465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.908966+0000","last_change":"2024-09-19T08:39:37.915040+0000","last_active":"2024-09-19T08:39:38.908966+0000","last_peered":"2024-09-19T08:39:38.908966+0000","last_clean":"2024-09-19T08:39:38.908966+0000","last_became_active":"2024-09-19T08:39:37.914232+0000","last_became_peered":"2024-09-19T08:39:37.914232+0000","last_unstale":"2024-09-19T08:39:38.908966+0000","last_undegraded":"2024-09-19T08:39:38.908966+0000","last_fullsized":"2024-09-19T08:39:38.908966+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:49:13.065050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:37.883250+0000","last_change":"2024-09-19T08:39:37.883250+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.883250+0000","last_undegraded":"2024-09-19T08:39:37.883250+0000","last_fullsized":"2024-09-19T08:39:37.883250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-19T08:39:37.922144+0000","last_change":"2024-09-19T08:39:37.918815+0000","last_active":"2024-09-19T08:39:37.922144+0000","last_peered":"2024-09-19T08:39:37.922144+0000","last_clean":"2024-09-19T08:39:37.922144+0000","last_became_active":"2024-09-19T08:39:37.918481+0000","last_became_peered":"2024-09-19T08:39:37.918481+0000","last_unstale":"2024-09-19T08:39:37.922144+0000","last_undegraded":"2024-09-19T08:39:37.922144+0000","last_fullsized":"2024-09-19T08:39:37.922144+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:55:34.226831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.936888+0000","last_change":"2024-09-19T08:39:38.929785+0000","last_active":"2024-09-19T08:39:38.936888+0000","last_peered":"2024-09-19T08:39:38.936888+0000","last_clean":"2024-09-19T08:39:38.936888+0000","last_became_active":"2024-09-19T08:39:38.929074+0000","last_became_peered":"2024-09-19T08:39:38.929074+0000","last_unstale":"2024-09-19T08:39:38.936888+0000","last_undegraded":"2024-09-19T08:39:38.936888+0000","last_fullsized":"2024-09-19T08:39:38.936888+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:37.035646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.918780+0000","last_change":"2024-09-19T08:39:37.936655+0000","last_active":"2024-09-19T08:39:38.918780+0000","last_peered":"2024-09-19T08:39:38.918780+0000","last_clean":"2024-09-19T08:39:38.918780+0000","last_became_active":"2024-09-19T08:39:37.936112+0000","last_became_peered":"2024-09-19T08:39:37.936112+0000","last_unstale":"2024-09-19T08:39:38.918780+0000","last_undegraded":"2024-09-19T08:39:38.918780+0000","last_fullsized":"2024-09-19T08:39:38.918780+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:33:43.714330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.959000+0000","last_change":"2024-09-19T08:39:38.942782+0000","last_active":"2024-09-19T08:39:38.959000+0000","last_peered":"2024-09-19T08:39:38.959000+0000","last_clean":"2024-09-19T08:39:38.959000+0000","last_became_active":"2024-09-19T08:39:38.939689+0000","last_became_peered":"2024-09-19T08:39:38.939689+0000","last_unstale":"2024-09-19T08:39:38.959000+0000","last_undegraded":"2024-09-19T08:39:38.959000+0000","last_fullsized":"2024-09-19T08:39:38.959000+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:37.082611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.895030+0000","last_change":"2024-09-19T08:39:37.924237+0000","last_active":"2024-09-19T08:39:38.895030+0000","last_peered":"2024-09-19T08:39:38.895030+0000","last_clean":"2024-09-19T08:39:38.895030+0000","last_became_active":"2024-09-19T08:39:37.923832+0000","last_became_peered":"2024-09-19T08:39:37.923832+0000","last_unstale":"2024-09-19T08:39:38.895030+0000","last_undegraded":"2024-09-19T08:39:38.895030+0000","last_fullsized":"2024-09-19T08:39:38.895030+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:14:31.270319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-09-19T08:39:37.896039+0000","last_change":"2024-09-19T08:39:37.896039+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.896039+0000","last_undegraded":"2024-09-19T08:39:37.896039+0000","last_fullsized":"2024-09-19T08:39:37.896039+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.919124+0000","last_change":"2024-09-19T08:39:37.934015+0000","last_active":"2024-09-19T08:39:38.919124+0000","last_peered":"2024-09-19T08:39:38.919124+0000","last_clean":"2024-09-19T08:39:38.919124+0000","last_became_active":"2024-09-19T08:39:37.933256+0000","last_became_peered":"2024-09-19T08:39:37.933256+0000","last_unstale":"2024-09-19T08:39:38.919124+0000","last_undegraded":"2024-09-19T08:39:38.919124+0000","last_fullsized":"2024-09-19T08:39:38.919124+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:55:09.923681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-09-19T08:39:37.896215+0000","last_change":"2024-09-19T08:39:37.896215+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.896215+0000","last_undegraded":"2024-09-19T08:39:37.896215+0000","last_fullsized":"2024-09-19T08:39:37.896215+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.901249+0000","last_change":"2024-09-19T08:39:37.920347+0000","last_active":"2024-09-19T08:39:38.901249+0000","last_peered":"2024-09-19T08:39:38.901249+0000","last_clean":"2024-09-19T08:39:38.901249+0000","last_became_active":"2024-09-19T08:39:37.919634+0000","last_became_peered":"2024-09-19T08:39:37.919634+0000","last_unstale":"2024-09-19T08:39:38.901249+0000","last_undegraded":"2024-09-19T08:39:38.901249+0000","last_fullsized":"2024-09-19T08:39:38.901249+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:14:31.270319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.933874+0000","last_change":"2024-09-19T08:39:38.927845+0000","last_active":"2024-09-19T08:39:38.933874+0000","last_peered":"2024-09-19T08:39:38.933874+0000","last_clean":"2024-09-19T08:39:38.933874+0000","last_became_active":"2024-09-19T08:39:38.926989+0000","last_became_peered":"2024-09-19T08:39:38.926989+0000","last_unstale":"2024-09-19T08:39:38.933874+0000","last_undegraded":"2024-09-19T08:39:38.933874+0000","last_fullsized":"2024-09-19T08:39:38.933874+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:55:10.927219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.895000+0000","last_change":"2024-09-19T08:39:37.910329+0000","last_active":"2024-09-19T08:39:38.895000+0000","last_peered":"2024-09-19T08:39:38.895000+0000","last_clean":"2024-09-19T08:39:38.895000+0000","last_became_active":"2024-09-19T08:39:37.909966+0000","last_became_peered":"2024-09-19T08:39:37.909966+0000","last_unstale":"2024-09-19T08:39:38.895000+0000","last_undegraded":"2024-09-19T08:39:38.895000+0000","last_fullsized":"2024-09-19T08:39:38.895000+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:36.032108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.942950+0000","last_change":"2024-09-19T08:39:38.936998+0000","last_active":"2024-09-19T08:39:38.942950+0000","last_peered":"2024-09-19T08:39:38.942950+0000","last_clean":"2024-09-19T08:39:38.942950+0000","last_became_active":"2024-09-19T08:39:38.935884+0000","last_became_peered":"2024-09-19T08:39:38.935884+0000","last_unstale":"2024-09-19T08:39:38.942950+0000","last_undegraded":"2024-09-19T08:39:38.942950+0000","last_fullsized":"2024-09-19T08:39:38.942950+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:40.364626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.903222+0000","last_change":"2024-09-19T08:39:37.907408+0000","last_active":"2024-09-19T08:39:38.903222+0000","last_peered":"2024-09-19T08:39:38.903222+0000","last_clean":"2024-09-19T08:39:38.903222+0000","last_became_active":"2024-09-19T08:39:37.906928+0000","last_became_peered":"2024-09-19T08:39:37.906928+0000","last_unstale":"2024-09-19T08:39:38.903222+0000","last_undegraded":"2024-09-19T08:39:38.903222+0000","last_fullsized":"2024-09-19T08:39:38.903222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:08.285216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.945594+0000","last_change":"2024-09-19T08:39:38.940058+0000","last_active":"2024-09-19T08:39:38.945594+0000","last_peered":"2024-09-19T08:39:38.945594+0000","last_clean":"2024-09-19T08:39:38.945594+0000","last_became_active":"2024-09-19T08:39:38.939663+0000","last_became_peered":"2024-09-19T08:39:38.939663+0000","last_unstale":"2024-09-19T08:39:38.945594+0000","last_undegraded":"2024-09-19T08:39:38.945594+0000","last_fullsized":"2024-09-19T08:39:38.945594+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:40.364626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.957136+0000","last_change":"2024-09-19T08:39:37.922845+0000","last_active":"2024-09-19T08:39:38.957136+0000","last_peered":"2024-09-19T08:39:38.957136+0000","last_clean":"2024-09-19T08:39:38.957136+0000","last_became_active":"2024-09-19T08:39:37.922153+0000","last_became_peered":"2024-09-19T08:39:37.922153+0000","last_unstale":"2024-09-19T08:39:38.957136+0000","last_undegraded":"2024-09-19T08:39:38.957136+0000","last_fullsized":"2024-09-19T08:39:38.957136+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:28.741298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":204,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.908394+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:39:38.908394+0000","last_peered":"2024-09-19T08:39:38.908394+0000","last_clean":"2024-09-19T08:39:38.908394+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:39:38.908394+0000","last_undegraded":"2024-09-19T08:39:38.908394+0000","last_fullsized":"2024-09-19T08:39:38.908394+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":104,"acting":104,"num_store_stats":7},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1794,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":29,"ondisk_log_size":29,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509461,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1081344,"data_stored":783009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509463,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":185776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509463,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":471040,"data_stored":185776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509460,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":185776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509462,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":185776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509460,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28072,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343768,"statfs":{"total":96636764160,"available":96608018432,"internally_reserved":0,"allocated":1064960,"data_stored":776144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":179509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542165,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31636,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340204,"statfs":{"total":96636764160,"available":96604368896,"internally_reserved":0,"allocated":458752,"data_stored":179509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:39:42.954 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:39:43.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.107+0000 7feb56740640 1 Processor -- start 2024-09-19T08:39:43.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.107+0000 7feb56740640 1 -- start start 2024-09-19T08:39:43.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.111+0000 7feb56740640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb50158fd0 0x7feb500b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:43.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.111+0000 7feb56740640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7feb5006c3a0 0x7feb500ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:43.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.111+0000 7feb56740640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb500ccc60 con 0x7feb500b4360 2024-09-19T08:39:43.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.111+0000 7feb56740640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7feb501593b0 con 0x7feb5006c3a0 2024-09-19T08:39:43.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.111+0000 7feb56740640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7feb500ccf40 con 0x7feb50158fd0 2024-09-19T08:39:43.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.111+0000 7feb54cdd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7feb5006c3a0 0x7feb500ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:43.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.111+0000 7feb54cdd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7feb5006c3a0 0x7feb500ce840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49578/0 (socket says 172.21.15.98:49578) 2024-09-19T08:39:43.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.111+0000 7feb54cdd640 1 -- 172.21.15.98:0/1865068097 learned_addr learned my addr 172.21.15.98:0/1865068097 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:43.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.111+0000 7feb4ffff640 1 --2- 172.21.15.98:0/1865068097 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb50158fd0 0x7feb500b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:43.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.111+0000 7feb54cdd640 1 -- 172.21.15.98:0/1865068097 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb50158fd0 msgr2=0x7feb500b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:43.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.111+0000 7feb54cdd640 1 --2- 172.21.15.98:0/1865068097 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb50158fd0 0x7feb500b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:43.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.111+0000 7feb54cdd640 1 -- 172.21.15.98:0/1865068097 >> v1:172.21.15.139:6789/0 conn(0x7feb500b4360 legacy=0x7feb5006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:43.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.111+0000 7feb54cdd640 1 -- 172.21.15.98:0/1865068097 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb500cd220 con 0x7feb5006c3a0 2024-09-19T08:39:43.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.111+0000 7feb54cdd640 1 --2- 172.21.15.98:0/1865068097 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb5006c3a0 0x7feb500ce840 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7feb40009e50 tx=0x7feb4001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb24c60fea07792 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:43.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.111+0000 7feb4effd640 1 -- 172.21.15.98:0/1865068097 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb400a57b0 con 0x7feb5006c3a0 2024-09-19T08:39:43.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.111+0000 7feb4effd640 1 -- 172.21.15.98:0/1865068097 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb400a5950 con 0x7feb5006c3a0 2024-09-19T08:39:43.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.111+0000 7feb4effd640 1 -- 172.21.15.98:0/1865068097 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb400a5d20 con 0x7feb5006c3a0 2024-09-19T08:39:43.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.115+0000 7feb56740640 1 -- 172.21.15.98:0/1865068097 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb5006c3a0 msgr2=0x7feb500ce840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:43.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.115+0000 7feb56740640 1 --2- 172.21.15.98:0/1865068097 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb5006c3a0 0x7feb500ce840 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7feb40009e50 tx=0x7feb4001c930 comp rx=0 tx=0).stop 2024-09-19T08:39:43.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.115+0000 7feb56740640 1 -- 172.21.15.98:0/1865068097 shutdown_connections 2024-09-19T08:39:43.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.115+0000 7feb56740640 1 --2- 172.21.15.98:0/1865068097 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb5006c3a0 0x7feb500ce840 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:43.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.115+0000 7feb56740640 1 --2- 172.21.15.98:0/1865068097 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb50158fd0 0x7feb500b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:43.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.115+0000 7feb56740640 1 -- 172.21.15.98:0/1865068097 >> 172.21.15.98:0/1865068097 conn(0x7feb500b2010 msgr2=0x7feb500b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:43.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.115+0000 7feb56740640 1 -- 172.21.15.98:0/1865068097 shutdown_connections 2024-09-19T08:39:43.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.115+0000 7feb56740640 1 -- 172.21.15.98:0/1865068097 wait complete. 2024-09-19T08:39:43.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.115+0000 7feb56740640 1 Processor -- start 2024-09-19T08:39:43.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.115+0000 7feb56740640 1 -- start start 2024-09-19T08:39:43.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.115+0000 7feb56740640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb5006c3a0 0x7feb50160bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:43.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.115+0000 7feb56740640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb500b4360 0x7feb5015da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:43.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.115+0000 7feb56740640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb50158fd0 0x7feb50154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:43.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.115+0000 7feb56740640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7feb500cf710 con 0x7feb500b4360 2024-09-19T08:39:43.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.115+0000 7feb56740640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feb500cf590 con 0x7feb5006c3a0 2024-09-19T08:39:43.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.115+0000 7feb56740640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7feb500cf890 con 0x7feb50158fd0 2024-09-19T08:39:43.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.115+0000 7feb4f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb500b4360 0x7feb5015da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:43.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.115+0000 7feb4f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb500b4360 0x7feb5015da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49588/0 (socket says 172.21.15.98:49588) 2024-09-19T08:39:43.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.115+0000 7feb4f7fe640 1 -- 172.21.15.98:0/129117959 learned_addr learned my addr 172.21.15.98:0/129117959 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:43.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.115+0000 7feb4f7fe640 1 -- 172.21.15.98:0/129117959 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb50158fd0 msgr2=0x7feb50154cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:43.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.115+0000 7feb4f7fe640 1 --2- 172.21.15.98:0/129117959 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb50158fd0 0x7feb50154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:43.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.115+0000 7feb4f7fe640 1 -- 172.21.15.98:0/129117959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb5006c3a0 msgr2=0x7feb50160bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:43.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.115+0000 7feb4f7fe640 1 --2- 172.21.15.98:0/129117959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb5006c3a0 0x7feb50160bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:43.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.115+0000 7feb4f7fe640 1 -- 172.21.15.98:0/129117959 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb501fe7e0 con 0x7feb500b4360 2024-09-19T08:39:43.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.115+0000 7feb4f7fe640 1 --2- 172.21.15.98:0/129117959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb500b4360 0x7feb5015da00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7feb3800c440 tx=0x7feb3800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:43.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.115+0000 7feb4cff9640 1 -- 172.21.15.98:0/129117959 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb38016020 con 0x7feb500b4360 2024-09-19T08:39:43.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.115+0000 7feb4cff9640 1 -- 172.21.15.98:0/129117959 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb38012830 con 0x7feb500b4360 2024-09-19T08:39:43.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.115+0000 7feb4cff9640 1 -- 172.21.15.98:0/129117959 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb38012c20 con 0x7feb500b4360 2024-09-19T08:39:43.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.115+0000 7feb56740640 1 -- 172.21.15.98:0/129117959 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb501feac0 con 0x7feb500b4360 2024-09-19T08:39:43.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.115+0000 7feb56740640 1 -- 172.21.15.98:0/129117959 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb501ff080 con 0x7feb500b4360 2024-09-19T08:39:43.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.119+0000 7feb56740640 1 -- 172.21.15.98:0/129117959 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb5015fe60 con 0x7feb500b4360 2024-09-19T08:39:43.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.119+0000 7feb4cff9640 1 -- 172.21.15.98:0/129117959 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7feb38019020 con 0x7feb500b4360 2024-09-19T08:39:43.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.119+0000 7feb4cff9640 1 --2- 172.21.15.98:0/129117959 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7feb30067a70 0x7feb30069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:43.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.119+0000 7feb4ffff640 1 --2- 172.21.15.98:0/129117959 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7feb30067a70 0x7feb30069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:43.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.119+0000 7feb4cff9640 1 -- 172.21.15.98:0/129117959 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7feb38085ff0 con 0x7feb500b4360 2024-09-19T08:39:43.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.119+0000 7feb4ffff640 1 --2- 172.21.15.98:0/129117959 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7feb30067a70 0x7feb30069f30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7feb4800a810 tx=0x7feb48008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:43.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.123+0000 7feb4cff9640 1 -- 172.21.15.98:0/129117959 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7feb38020020 con 0x7feb500b4360 2024-09-19T08:39:43.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.271+0000 7feb56740640 1 -- 172.21.15.98:0/129117959 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7feb500ce7c0 con 0x7feb30067a70 2024-09-19T08:39:43.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.291+0000 7feb4cff9640 1 -- 172.21.15.98:0/129117959 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249325 (secure 0 0 0) 0x7feb500ce7c0 con 0x7feb30067a70 2024-09-19T08:39:43.292 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:39:43.294 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:39:43.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.295+0000 7feb56740640 1 -- 172.21.15.98:0/129117959 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7feb30067a70 msgr2=0x7feb30069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:43.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.295+0000 7feb56740640 1 --2- 172.21.15.98:0/129117959 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7feb30067a70 0x7feb30069f30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7feb4800a810 tx=0x7feb48008040 comp rx=0 tx=0).stop 2024-09-19T08:39:43.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.295+0000 7feb56740640 1 -- 172.21.15.98:0/129117959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb500b4360 msgr2=0x7feb5015da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:43.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.295+0000 7feb56740640 1 --2- 172.21.15.98:0/129117959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb500b4360 0x7feb5015da00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7feb3800c440 tx=0x7feb3800c900 comp rx=0 tx=0).stop 2024-09-19T08:39:43.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.295+0000 7feb56740640 1 -- 172.21.15.98:0/129117959 shutdown_connections 2024-09-19T08:39:43.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.295+0000 7feb56740640 1 --2- 172.21.15.98:0/129117959 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7feb30067a70 0x7feb30069f30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:43.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.295+0000 7feb56740640 1 --2- 172.21.15.98:0/129117959 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb50158fd0 0x7feb50154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:43.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.295+0000 7feb56740640 1 --2- 172.21.15.98:0/129117959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb500b4360 0x7feb5015da00 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:43.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.295+0000 7feb56740640 1 --2- 172.21.15.98:0/129117959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb5006c3a0 0x7feb50160bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:43.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.295+0000 7feb56740640 1 -- 172.21.15.98:0/129117959 >> 172.21.15.98:0/129117959 conn(0x7feb500b2010 msgr2=0x7feb500b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:43.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.295+0000 7feb56740640 1 -- 172.21.15.98:0/129117959 shutdown_connections 2024-09-19T08:39:43.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.295+0000 7feb56740640 1 -- 172.21.15.98:0/129117959 wait complete. 2024-09-19T08:39:43.309 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2024-09-19T08:39:42.050723+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592162,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":195,"num_write_kb":2422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":170,"acting":170,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":30,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":229392,"kb_used_data":4832,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754745328,"statfs":{"total":773094113280,"available":772859215872,"internally_reserved":0,"allocated":4947968,"data_stored":2661275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1794,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.166397"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.958882+0000","last_change":"2024-09-19T08:39:38.938293+0000","last_active":"2024-09-19T08:39:38.958882+0000","last_peered":"2024-09-19T08:39:38.958882+0000","last_clean":"2024-09-19T08:39:38.958882+0000","last_became_active":"2024-09-19T08:39:38.936784+0000","last_became_peered":"2024-09-19T08:39:38.936784+0000","last_unstale":"2024-09-19T08:39:38.958882+0000","last_undegraded":"2024-09-19T08:39:38.958882+0000","last_fullsized":"2024-09-19T08:39:38.958882+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:08:18.820654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.944459+0000","last_change":"2024-09-19T08:39:38.942347+0000","last_active":"2024-09-19T08:39:38.944459+0000","last_peered":"2024-09-19T08:39:38.944459+0000","last_clean":"2024-09-19T08:39:38.944459+0000","last_became_active":"2024-09-19T08:39:38.941641+0000","last_became_peered":"2024-09-19T08:39:38.941641+0000","last_unstale":"2024-09-19T08:39:38.944459+0000","last_undegraded":"2024-09-19T08:39:38.944459+0000","last_fullsized":"2024-09-19T08:39:38.944459+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:29.732835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.933674+0000","last_change":"2024-09-19T08:39:38.918421+0000","last_active":"2024-09-19T08:39:38.933674+0000","last_peered":"2024-09-19T08:39:38.933674+0000","last_clean":"2024-09-19T08:39:38.933674+0000","last_became_active":"2024-09-19T08:39:38.917761+0000","last_became_peered":"2024-09-19T08:39:38.917761+0000","last_unstale":"2024-09-19T08:39:38.933674+0000","last_undegraded":"2024-09-19T08:39:38.933674+0000","last_fullsized":"2024-09-19T08:39:38.933674+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:31.825354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.936913+0000","last_change":"2024-09-19T08:39:38.932965+0000","last_active":"2024-09-19T08:39:38.936913+0000","last_peered":"2024-09-19T08:39:38.936913+0000","last_clean":"2024-09-19T08:39:38.936913+0000","last_became_active":"2024-09-19T08:39:38.929490+0000","last_became_peered":"2024-09-19T08:39:38.929490+0000","last_unstale":"2024-09-19T08:39:38.936913+0000","last_undegraded":"2024-09-19T08:39:38.936913+0000","last_fullsized":"2024-09-19T08:39:38.936913+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:30:09.863827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.942469+0000","last_change":"2024-09-19T08:39:38.938221+0000","last_active":"2024-09-19T08:39:38.942469+0000","last_peered":"2024-09-19T08:39:38.942469+0000","last_clean":"2024-09-19T08:39:38.942469+0000","last_became_active":"2024-09-19T08:39:38.937356+0000","last_became_peered":"2024-09-19T08:39:38.937356+0000","last_unstale":"2024-09-19T08:39:38.942469+0000","last_undegraded":"2024-09-19T08:39:38.942469+0000","last_fullsized":"2024-09-19T08:39:38.942469+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:00:09.742802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-09-19T08:39:37.898206+0000","last_change":"2024-09-19T08:39:37.898206+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.898206+0000","last_undegraded":"2024-09-19T08:39:37.898206+0000","last_fullsized":"2024-09-19T08:39:37.898206+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.961008+0000","last_change":"2024-09-19T08:39:38.945329+0000","last_active":"2024-09-19T08:39:38.961008+0000","last_peered":"2024-09-19T08:39:38.961008+0000","last_clean":"2024-09-19T08:39:38.961008+0000","last_became_active":"2024-09-19T08:39:38.944121+0000","last_became_peered":"2024-09-19T08:39:38.944121+0000","last_unstale":"2024-09-19T08:39:38.961008+0000","last_undegraded":"2024-09-19T08:39:38.961008+0000","last_fullsized":"2024-09-19T08:39:38.961008+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:37:14.987023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.937026+0000","last_change":"2024-09-19T08:39:38.929792+0000","last_active":"2024-09-19T08:39:38.937026+0000","last_peered":"2024-09-19T08:39:38.937026+0000","last_clean":"2024-09-19T08:39:38.937026+0000","last_became_active":"2024-09-19T08:39:38.928743+0000","last_became_peered":"2024-09-19T08:39:38.928743+0000","last_unstale":"2024-09-19T08:39:38.937026+0000","last_undegraded":"2024-09-19T08:39:38.937026+0000","last_fullsized":"2024-09-19T08:39:38.937026+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:14:32.273857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.937132+0000","last_change":"2024-09-19T08:39:38.934773+0000","last_active":"2024-09-19T08:39:38.937132+0000","last_peered":"2024-09-19T08:39:38.937132+0000","last_clean":"2024-09-19T08:39:38.937132+0000","last_became_active":"2024-09-19T08:39:38.934054+0000","last_became_peered":"2024-09-19T08:39:38.934054+0000","last_unstale":"2024-09-19T08:39:38.937132+0000","last_undegraded":"2024-09-19T08:39:38.937132+0000","last_fullsized":"2024-09-19T08:39:38.937132+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:31.825354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.937106+0000","last_change":"2024-09-19T08:39:38.934619+0000","last_active":"2024-09-19T08:39:38.937106+0000","last_peered":"2024-09-19T08:39:38.937106+0000","last_clean":"2024-09-19T08:39:38.937106+0000","last_became_active":"2024-09-19T08:39:38.934244+0000","last_became_peered":"2024-09-19T08:39:38.934244+0000","last_unstale":"2024-09-19T08:39:38.937106+0000","last_undegraded":"2024-09-19T08:39:38.937106+0000","last_fullsized":"2024-09-19T08:39:38.937106+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:08:44.560138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.937073+0000","last_change":"2024-09-19T08:39:38.933595+0000","last_active":"2024-09-19T08:39:38.937073+0000","last_peered":"2024-09-19T08:39:38.937073+0000","last_clean":"2024-09-19T08:39:38.937073+0000","last_became_active":"2024-09-19T08:39:38.933316+0000","last_became_peered":"2024-09-19T08:39:38.933316+0000","last_unstale":"2024-09-19T08:39:38.937073+0000","last_undegraded":"2024-09-19T08:39:38.937073+0000","last_fullsized":"2024-09-19T08:39:38.937073+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:43.876237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.928186+0000","last_change":"2024-09-19T08:39:38.923904+0000","last_active":"2024-09-19T08:39:38.928186+0000","last_peered":"2024-09-19T08:39:38.928186+0000","last_clean":"2024-09-19T08:39:38.928186+0000","last_became_active":"2024-09-19T08:39:38.923135+0000","last_became_peered":"2024-09-19T08:39:38.923135+0000","last_unstale":"2024-09-19T08:39:38.928186+0000","last_undegraded":"2024-09-19T08:39:38.928186+0000","last_fullsized":"2024-09-19T08:39:38.928186+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:06.457317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.938802+0000","last_change":"2024-09-19T08:39:38.933031+0000","last_active":"2024-09-19T08:39:38.938802+0000","last_peered":"2024-09-19T08:39:38.938802+0000","last_clean":"2024-09-19T08:39:38.938802+0000","last_became_active":"2024-09-19T08:39:38.932568+0000","last_became_peered":"2024-09-19T08:39:38.932568+0000","last_unstale":"2024-09-19T08:39:38.938802+0000","last_undegraded":"2024-09-19T08:39:38.938802+0000","last_fullsized":"2024-09-19T08:39:38.938802+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:41:56.688886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.940685+0000","last_change":"2024-09-19T08:39:38.936306+0000","last_active":"2024-09-19T08:39:38.940685+0000","last_peered":"2024-09-19T08:39:38.940685+0000","last_clean":"2024-09-19T08:39:38.940685+0000","last_became_active":"2024-09-19T08:39:38.935988+0000","last_became_peered":"2024-09-19T08:39:38.935988+0000","last_unstale":"2024-09-19T08:39:38.940685+0000","last_undegraded":"2024-09-19T08:39:38.940685+0000","last_fullsized":"2024-09-19T08:39:38.940685+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:49:14.068588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-09-19T08:39:37.899012+0000","last_change":"2024-09-19T08:39:37.899012+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.899012+0000","last_undegraded":"2024-09-19T08:39:37.899012+0000","last_fullsized":"2024-09-19T08:39:37.899012+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:37.883250+0000","last_change":"2024-09-19T08:39:37.883250+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.883250+0000","last_undegraded":"2024-09-19T08:39:37.883250+0000","last_fullsized":"2024-09-19T08:39:37.883250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.933907+0000","last_change":"2024-09-19T08:39:38.924533+0000","last_active":"2024-09-19T08:39:38.933907+0000","last_peered":"2024-09-19T08:39:38.933907+0000","last_clean":"2024-09-19T08:39:38.933907+0000","last_became_active":"2024-09-19T08:39:38.923817+0000","last_became_peered":"2024-09-19T08:39:38.923817+0000","last_unstale":"2024-09-19T08:39:38.933907+0000","last_undegraded":"2024-09-19T08:39:38.933907+0000","last_fullsized":"2024-09-19T08:39:38.933907+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:29.744836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.943119+0000","last_change":"2024-09-19T08:39:38.938932+0000","last_active":"2024-09-19T08:39:38.943119+0000","last_peered":"2024-09-19T08:39:38.943119+0000","last_clean":"2024-09-19T08:39:38.943119+0000","last_became_active":"2024-09-19T08:39:38.937330+0000","last_became_peered":"2024-09-19T08:39:38.937330+0000","last_unstale":"2024-09-19T08:39:38.943119+0000","last_undegraded":"2024-09-19T08:39:38.943119+0000","last_fullsized":"2024-09-19T08:39:38.943119+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:06.457317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:37.883250+0000","last_change":"2024-09-19T08:39:37.883250+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.883250+0000","last_undegraded":"2024-09-19T08:39:37.883250+0000","last_fullsized":"2024-09-19T08:39:37.883250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.938851+0000","last_change":"2024-09-19T08:39:38.936241+0000","last_active":"2024-09-19T08:39:38.938851+0000","last_peered":"2024-09-19T08:39:38.938851+0000","last_clean":"2024-09-19T08:39:38.938851+0000","last_became_active":"2024-09-19T08:39:38.935936+0000","last_became_peered":"2024-09-19T08:39:38.935936+0000","last_unstale":"2024-09-19T08:39:38.938851+0000","last_undegraded":"2024-09-19T08:39:38.938851+0000","last_fullsized":"2024-09-19T08:39:38.938851+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:49.679363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-09-19T08:39:37.898343+0000","last_change":"2024-09-19T08:39:37.898343+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.898343+0000","last_undegraded":"2024-09-19T08:39:37.898343+0000","last_fullsized":"2024-09-19T08:39:37.898343+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.938460+0000","last_change":"2024-09-19T08:39:38.935680+0000","last_active":"2024-09-19T08:39:38.938460+0000","last_peered":"2024-09-19T08:39:38.938460+0000","last_clean":"2024-09-19T08:39:38.938460+0000","last_became_active":"2024-09-19T08:39:38.929257+0000","last_became_peered":"2024-09-19T08:39:38.929257+0000","last_unstale":"2024-09-19T08:39:38.938460+0000","last_undegraded":"2024-09-19T08:39:38.938460+0000","last_fullsized":"2024-09-19T08:39:38.938460+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:23.048070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:37.883250+0000","last_change":"2024-09-19T08:39:37.883250+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.883250+0000","last_undegraded":"2024-09-19T08:39:37.883250+0000","last_fullsized":"2024-09-19T08:39:37.883250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:37.883250+0000","last_change":"2024-09-19T08:39:37.883250+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.883250+0000","last_undegraded":"2024-09-19T08:39:37.883250+0000","last_fullsized":"2024-09-19T08:39:37.883250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:37.883250+0000","last_change":"2024-09-19T08:39:37.883250+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.883250+0000","last_undegraded":"2024-09-19T08:39:37.883250+0000","last_fullsized":"2024-09-19T08:39:37.883250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.938757+0000","last_change":"2024-09-19T08:39:38.932467+0000","last_active":"2024-09-19T08:39:38.938757+0000","last_peered":"2024-09-19T08:39:38.938757+0000","last_clean":"2024-09-19T08:39:38.938757+0000","last_became_active":"2024-09-19T08:39:38.932023+0000","last_became_peered":"2024-09-19T08:39:38.932023+0000","last_unstale":"2024-09-19T08:39:38.938757+0000","last_undegraded":"2024-09-19T08:39:38.938757+0000","last_fullsized":"2024-09-19T08:39:38.938757+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:11.986036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.933854+0000","last_change":"2024-09-19T08:39:38.924547+0000","last_active":"2024-09-19T08:39:38.933854+0000","last_peered":"2024-09-19T08:39:38.933854+0000","last_clean":"2024-09-19T08:39:38.933854+0000","last_became_active":"2024-09-19T08:39:38.923802+0000","last_became_peered":"2024-09-19T08:39:38.923802+0000","last_unstale":"2024-09-19T08:39:38.933854+0000","last_undegraded":"2024-09-19T08:39:38.933854+0000","last_fullsized":"2024-09-19T08:39:38.933854+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:23.048070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.933859+0000","last_change":"2024-09-19T08:39:38.927516+0000","last_active":"2024-09-19T08:39:38.933859+0000","last_peered":"2024-09-19T08:39:38.933859+0000","last_clean":"2024-09-19T08:39:38.933859+0000","last_became_active":"2024-09-19T08:39:38.926015+0000","last_became_peered":"2024-09-19T08:39:38.926015+0000","last_unstale":"2024-09-19T08:39:38.933859+0000","last_undegraded":"2024-09-19T08:39:38.933859+0000","last_fullsized":"2024-09-19T08:39:38.933859+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:33:44.717869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:37.883250+0000","last_change":"2024-09-19T08:39:37.883250+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.883250+0000","last_undegraded":"2024-09-19T08:39:37.883250+0000","last_fullsized":"2024-09-19T08:39:37.883250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.934797+0000","last_change":"2024-09-19T08:39:38.931320+0000","last_active":"2024-09-19T08:39:38.934797+0000","last_peered":"2024-09-19T08:39:38.934797+0000","last_clean":"2024-09-19T08:39:38.934797+0000","last_became_active":"2024-09-19T08:39:38.930578+0000","last_became_peered":"2024-09-19T08:39:38.930578+0000","last_unstale":"2024-09-19T08:39:38.934797+0000","last_undegraded":"2024-09-19T08:39:38.934797+0000","last_fullsized":"2024-09-19T08:39:38.934797+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:49:14.068588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-09-19T08:39:37.897514+0000","last_change":"2024-09-19T08:39:37.897514+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.897514+0000","last_undegraded":"2024-09-19T08:39:37.897514+0000","last_fullsized":"2024-09-19T08:39:37.897514+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.930583+0000","last_change":"2024-09-19T08:39:38.921905+0000","last_active":"2024-09-19T08:39:38.930583+0000","last_peered":"2024-09-19T08:39:38.930583+0000","last_clean":"2024-09-19T08:39:38.930583+0000","last_became_active":"2024-09-19T08:39:38.920862+0000","last_became_peered":"2024-09-19T08:39:38.920862+0000","last_unstale":"2024-09-19T08:39:38.930583+0000","last_undegraded":"2024-09-19T08:39:38.930583+0000","last_fullsized":"2024-09-19T08:39:38.930583+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:43:08.445843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.950250+0000","last_change":"2024-09-19T08:39:38.942801+0000","last_active":"2024-09-19T08:39:38.950250+0000","last_peered":"2024-09-19T08:39:38.950250+0000","last_clean":"2024-09-19T08:39:38.950250+0000","last_became_active":"2024-09-19T08:39:38.942568+0000","last_became_peered":"2024-09-19T08:39:38.942568+0000","last_unstale":"2024-09-19T08:39:38.950250+0000","last_undegraded":"2024-09-19T08:39:38.950250+0000","last_fullsized":"2024-09-19T08:39:38.950250+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:31.825354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-19T08:39:37.942662+0000","last_change":"2024-09-19T08:39:37.937397+0000","last_active":"2024-09-19T08:39:37.942662+0000","last_peered":"2024-09-19T08:39:37.942662+0000","last_clean":"2024-09-19T08:39:37.942662+0000","last_became_active":"2024-09-19T08:39:37.937128+0000","last_became_peered":"2024-09-19T08:39:37.937128+0000","last_unstale":"2024-09-19T08:39:37.942662+0000","last_undegraded":"2024-09-19T08:39:37.942662+0000","last_fullsized":"2024-09-19T08:39:37.942662+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:47.949747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:37.883250+0000","last_change":"2024-09-19T08:39:37.883250+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.883250+0000","last_undegraded":"2024-09-19T08:39:37.883250+0000","last_fullsized":"2024-09-19T08:39:37.883250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"42'1","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.959478+0000","last_change":"2024-09-19T08:39:37.932635+0000","last_active":"2024-09-19T08:39:38.959478+0000","last_peered":"2024-09-19T08:39:38.959478+0000","last_clean":"2024-09-19T08:39:38.959478+0000","last_became_active":"2024-09-19T08:39:37.931841+0000","last_became_peered":"2024-09-19T08:39:37.931841+0000","last_unstale":"2024-09-19T08:39:38.959478+0000","last_undegraded":"2024-09-19T08:39:38.959478+0000","last_fullsized":"2024-09-19T08:39:38.959478+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:42.872698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.943117+0000","last_change":"2024-09-19T08:39:38.940685+0000","last_active":"2024-09-19T08:39:38.943117+0000","last_peered":"2024-09-19T08:39:38.943117+0000","last_clean":"2024-09-19T08:39:38.943117+0000","last_became_active":"2024-09-19T08:39:38.935293+0000","last_became_peered":"2024-09-19T08:39:38.935293+0000","last_unstale":"2024-09-19T08:39:38.943117+0000","last_undegraded":"2024-09-19T08:39:38.943117+0000","last_fullsized":"2024-09-19T08:39:38.943117+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:08:18.820654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.956642+0000","last_change":"2024-09-19T08:39:37.917790+0000","last_active":"2024-09-19T08:39:38.956642+0000","last_peered":"2024-09-19T08:39:38.956642+0000","last_clean":"2024-09-19T08:39:38.956642+0000","last_became_active":"2024-09-19T08:39:37.917479+0000","last_became_peered":"2024-09-19T08:39:37.917479+0000","last_unstale":"2024-09-19T08:39:38.956642+0000","last_undegraded":"2024-09-19T08:39:38.956642+0000","last_fullsized":"2024-09-19T08:39:38.956642+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:30:08.860289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.927097+0000","last_change":"2024-09-19T08:39:38.920238+0000","last_active":"2024-09-19T08:39:38.927097+0000","last_peered":"2024-09-19T08:39:38.927097+0000","last_clean":"2024-09-19T08:39:38.927097+0000","last_became_active":"2024-09-19T08:39:38.919486+0000","last_became_peered":"2024-09-19T08:39:38.919486+0000","last_unstale":"2024-09-19T08:39:38.927097+0000","last_undegraded":"2024-09-19T08:39:38.927097+0000","last_fullsized":"2024-09-19T08:39:38.927097+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:43.876237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.961365+0000","last_change":"2024-09-19T08:39:37.927771+0000","last_active":"2024-09-19T08:39:38.961365+0000","last_peered":"2024-09-19T08:39:38.961365+0000","last_clean":"2024-09-19T08:39:38.961365+0000","last_became_active":"2024-09-19T08:39:37.926946+0000","last_became_peered":"2024-09-19T08:39:37.926946+0000","last_unstale":"2024-09-19T08:39:38.961365+0000","last_undegraded":"2024-09-19T08:39:38.961365+0000","last_fullsized":"2024-09-19T08:39:38.961365+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:39.361088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.933755+0000","last_change":"2024-09-19T08:39:38.918556+0000","last_active":"2024-09-19T08:39:38.933755+0000","last_peered":"2024-09-19T08:39:38.933755+0000","last_clean":"2024-09-19T08:39:38.933755+0000","last_became_active":"2024-09-19T08:39:38.917830+0000","last_became_peered":"2024-09-19T08:39:38.917830+0000","last_unstale":"2024-09-19T08:39:38.933755+0000","last_undegraded":"2024-09-19T08:39:38.933755+0000","last_fullsized":"2024-09-19T08:39:38.933755+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:33.862515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-19T08:39:37.919363+0000","last_change":"2024-09-19T08:39:37.913524+0000","last_active":"2024-09-19T08:39:37.919363+0000","last_peered":"2024-09-19T08:39:37.919363+0000","last_clean":"2024-09-19T08:39:37.919363+0000","last_became_active":"2024-09-19T08:39:37.912853+0000","last_became_peered":"2024-09-19T08:39:37.912853+0000","last_unstale":"2024-09-19T08:39:37.919363+0000","last_undegraded":"2024-09-19T08:39:37.919363+0000","last_fullsized":"2024-09-19T08:39:37.919363+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:24.628377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.936990+0000","last_change":"2024-09-19T08:39:38.929719+0000","last_active":"2024-09-19T08:39:38.936990+0000","last_peered":"2024-09-19T08:39:38.936990+0000","last_clean":"2024-09-19T08:39:38.936990+0000","last_became_active":"2024-09-19T08:39:38.929158+0000","last_became_peered":"2024-09-19T08:39:38.929158+0000","last_unstale":"2024-09-19T08:39:38.936990+0000","last_undegraded":"2024-09-19T08:39:38.936990+0000","last_fullsized":"2024-09-19T08:39:38.936990+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:07:16.957999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"42'1","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.958601+0000","last_change":"2024-09-19T08:39:37.929415+0000","last_active":"2024-09-19T08:39:38.958601+0000","last_peered":"2024-09-19T08:39:38.958601+0000","last_clean":"2024-09-19T08:39:38.958601+0000","last_became_active":"2024-09-19T08:39:37.928610+0000","last_became_peered":"2024-09-19T08:39:37.928610+0000","last_unstale":"2024-09-19T08:39:38.958601+0000","last_undegraded":"2024-09-19T08:39:38.958601+0000","last_fullsized":"2024-09-19T08:39:38.958601+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:36.032108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.936735+0000","last_change":"2024-09-19T08:39:38.928185+0000","last_active":"2024-09-19T08:39:38.936735+0000","last_peered":"2024-09-19T08:39:38.936735+0000","last_clean":"2024-09-19T08:39:38.936735+0000","last_became_active":"2024-09-19T08:39:38.924812+0000","last_became_peered":"2024-09-19T08:39:38.924812+0000","last_unstale":"2024-09-19T08:39:38.936735+0000","last_undegraded":"2024-09-19T08:39:38.936735+0000","last_fullsized":"2024-09-19T08:39:38.936735+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:08.958729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.912110+0000","last_change":"2024-09-19T08:39:37.921882+0000","last_active":"2024-09-19T08:39:38.912110+0000","last_peered":"2024-09-19T08:39:38.912110+0000","last_clean":"2024-09-19T08:39:38.912110+0000","last_became_active":"2024-09-19T08:39:37.921061+0000","last_became_peered":"2024-09-19T08:39:37.921061+0000","last_unstale":"2024-09-19T08:39:38.912110+0000","last_undegraded":"2024-09-19T08:39:38.912110+0000","last_fullsized":"2024-09-19T08:39:38.912110+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:49:13.065050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:37.883250+0000","last_change":"2024-09-19T08:39:37.883250+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.883250+0000","last_undegraded":"2024-09-19T08:39:37.883250+0000","last_fullsized":"2024-09-19T08:39:37.883250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.902185+0000","last_change":"2024-09-19T08:39:37.914508+0000","last_active":"2024-09-19T08:39:38.902185+0000","last_peered":"2024-09-19T08:39:38.902185+0000","last_clean":"2024-09-19T08:39:38.902185+0000","last_became_active":"2024-09-19T08:39:37.914186+0000","last_became_peered":"2024-09-19T08:39:37.914186+0000","last_unstale":"2024-09-19T08:39:38.902185+0000","last_undegraded":"2024-09-19T08:39:38.902185+0000","last_fullsized":"2024-09-19T08:39:38.902185+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:48:49.023614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.943169+0000","last_change":"2024-09-19T08:39:38.939531+0000","last_active":"2024-09-19T08:39:38.943169+0000","last_peered":"2024-09-19T08:39:38.943169+0000","last_clean":"2024-09-19T08:39:38.943169+0000","last_became_active":"2024-09-19T08:39:38.938231+0000","last_became_peered":"2024-09-19T08:39:38.938231+0000","last_unstale":"2024-09-19T08:39:38.943169+0000","last_undegraded":"2024-09-19T08:39:38.943169+0000","last_fullsized":"2024-09-19T08:39:38.943169+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:33:44.717869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.901355+0000","last_change":"2024-09-19T08:39:37.930095+0000","last_active":"2024-09-19T08:39:38.901355+0000","last_peered":"2024-09-19T08:39:38.901355+0000","last_clean":"2024-09-19T08:39:38.901355+0000","last_became_active":"2024-09-19T08:39:37.921804+0000","last_became_peered":"2024-09-19T08:39:37.921804+0000","last_unstale":"2024-09-19T08:39:38.901355+0000","last_undegraded":"2024-09-19T08:39:38.901355+0000","last_fullsized":"2024-09-19T08:39:38.901355+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:30:08.860289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.958802+0000","last_change":"2024-09-19T08:39:38.941102+0000","last_active":"2024-09-19T08:39:38.958802+0000","last_peered":"2024-09-19T08:39:38.958802+0000","last_clean":"2024-09-19T08:39:38.958802+0000","last_became_active":"2024-09-19T08:39:38.937500+0000","last_became_peered":"2024-09-19T08:39:38.937500+0000","last_unstale":"2024-09-19T08:39:38.958802+0000","last_undegraded":"2024-09-19T08:39:38.958802+0000","last_fullsized":"2024-09-19T08:39:38.958802+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:29.732835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.911564+0000","last_change":"2024-09-19T08:39:37.912838+0000","last_active":"2024-09-19T08:39:38.911564+0000","last_peered":"2024-09-19T08:39:38.911564+0000","last_clean":"2024-09-19T08:39:38.911564+0000","last_became_active":"2024-09-19T08:39:37.912442+0000","last_became_peered":"2024-09-19T08:39:37.912442+0000","last_unstale":"2024-09-19T08:39:38.911564+0000","last_undegraded":"2024-09-19T08:39:38.911564+0000","last_fullsized":"2024-09-19T08:39:38.911564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:46.135290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-09-19T08:39:37.896926+0000","last_change":"2024-09-19T08:39:37.896926+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.896926+0000","last_undegraded":"2024-09-19T08:39:37.896926+0000","last_fullsized":"2024-09-19T08:39:37.896926+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"42'3","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.961471+0000","last_change":"2024-09-19T08:39:37.936425+0000","last_active":"2024-09-19T08:39:38.961471+0000","last_peered":"2024-09-19T08:39:38.961471+0000","last_clean":"2024-09-19T08:39:38.961471+0000","last_became_active":"2024-09-19T08:39:37.935418+0000","last_became_peered":"2024-09-19T08:39:37.935418+0000","last_unstale":"2024-09-19T08:39:38.961471+0000","last_undegraded":"2024-09-19T08:39:38.961471+0000","last_fullsized":"2024-09-19T08:39:38.961471+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:05.453778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.958959+0000","last_change":"2024-09-19T08:39:38.942176+0000","last_active":"2024-09-19T08:39:38.958959+0000","last_peered":"2024-09-19T08:39:38.958959+0000","last_clean":"2024-09-19T08:39:38.958959+0000","last_became_active":"2024-09-19T08:39:38.939625+0000","last_became_peered":"2024-09-19T08:39:38.939625+0000","last_unstale":"2024-09-19T08:39:38.958959+0000","last_undegraded":"2024-09-19T08:39:38.958959+0000","last_fullsized":"2024-09-19T08:39:38.958959+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:22:21.538914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"42'6","reported_seq":18,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.962495+0000","last_change":"2024-09-19T08:39:37.921227+0000","last_active":"2024-09-19T08:39:38.962495+0000","last_peered":"2024-09-19T08:39:38.962495+0000","last_clean":"2024-09-19T08:39:38.962495+0000","last_became_active":"2024-09-19T08:39:37.920773+0000","last_became_peered":"2024-09-19T08:39:37.920773+0000","last_unstale":"2024-09-19T08:39:38.962495+0000","last_undegraded":"2024-09-19T08:39:38.962495+0000","last_fullsized":"2024-09-19T08:39:38.962495+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:24:07.528992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:37.883250+0000","last_change":"2024-09-19T08:39:37.883250+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.883250+0000","last_undegraded":"2024-09-19T08:39:37.883250+0000","last_fullsized":"2024-09-19T08:39:37.883250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-19T08:39:37.912133+0000","last_change":"2024-09-19T08:39:37.905231+0000","last_active":"2024-09-19T08:39:37.912133+0000","last_peered":"2024-09-19T08:39:37.912133+0000","last_clean":"2024-09-19T08:39:37.912133+0000","last_became_active":"2024-09-19T08:39:37.904660+0000","last_became_peered":"2024-09-19T08:39:37.904660+0000","last_unstale":"2024-09-19T08:39:37.912133+0000","last_undegraded":"2024-09-19T08:39:37.912133+0000","last_fullsized":"2024-09-19T08:39:37.912133+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:38:48.929675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.940617+0000","last_change":"2024-09-19T08:39:38.936213+0000","last_active":"2024-09-19T08:39:38.940617+0000","last_peered":"2024-09-19T08:39:38.940617+0000","last_clean":"2024-09-19T08:39:38.940617+0000","last_became_active":"2024-09-19T08:39:38.935856+0000","last_became_peered":"2024-09-19T08:39:38.935856+0000","last_unstale":"2024-09-19T08:39:38.940617+0000","last_undegraded":"2024-09-19T08:39:38.940617+0000","last_fullsized":"2024-09-19T08:39:38.940617+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:29.744836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.908538+0000","last_change":"2024-09-19T08:39:37.914414+0000","last_active":"2024-09-19T08:39:38.908538+0000","last_peered":"2024-09-19T08:39:38.908538+0000","last_clean":"2024-09-19T08:39:38.908538+0000","last_became_active":"2024-09-19T08:39:37.913576+0000","last_became_peered":"2024-09-19T08:39:37.913576+0000","last_unstale":"2024-09-19T08:39:38.908538+0000","last_undegraded":"2024-09-19T08:39:38.908538+0000","last_fullsized":"2024-09-19T08:39:38.908538+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:28.741298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-09-19T08:39:37.896086+0000","last_change":"2024-09-19T08:39:37.896086+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.896086+0000","last_undegraded":"2024-09-19T08:39:37.896086+0000","last_fullsized":"2024-09-19T08:39:37.896086+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-19T08:39:37.912720+0000","last_change":"2024-09-19T08:39:37.910119+0000","last_active":"2024-09-19T08:39:37.912720+0000","last_peered":"2024-09-19T08:39:37.912720+0000","last_clean":"2024-09-19T08:39:37.912720+0000","last_became_active":"2024-09-19T08:39:37.909646+0000","last_became_peered":"2024-09-19T08:39:37.909646+0000","last_unstale":"2024-09-19T08:39:37.912720+0000","last_undegraded":"2024-09-19T08:39:37.912720+0000","last_fullsized":"2024-09-19T08:39:37.912720+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:30.311453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.959355+0000","last_change":"2024-09-19T08:39:38.937339+0000","last_active":"2024-09-19T08:39:38.959355+0000","last_peered":"2024-09-19T08:39:38.959355+0000","last_clean":"2024-09-19T08:39:38.959355+0000","last_became_active":"2024-09-19T08:39:38.936404+0000","last_became_peered":"2024-09-19T08:39:38.936404+0000","last_unstale":"2024-09-19T08:39:38.959355+0000","last_undegraded":"2024-09-19T08:39:38.959355+0000","last_fullsized":"2024-09-19T08:39:38.959355+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:00:09.742802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"42'8","reported_seq":20,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.957690+0000","last_change":"2024-09-19T08:39:37.918124+0000","last_active":"2024-09-19T08:39:38.957690+0000","last_peered":"2024-09-19T08:39:38.957690+0000","last_clean":"2024-09-19T08:39:38.957690+0000","last_became_active":"2024-09-19T08:39:37.917521+0000","last_became_peered":"2024-09-19T08:39:37.917521+0000","last_unstale":"2024-09-19T08:39:38.957690+0000","last_undegraded":"2024-09-19T08:39:38.957690+0000","last_fullsized":"2024-09-19T08:39:38.957690+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:38:39.741349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.937824+0000","last_change":"2024-09-19T08:39:38.926671+0000","last_active":"2024-09-19T08:39:38.937824+0000","last_peered":"2024-09-19T08:39:38.937824+0000","last_clean":"2024-09-19T08:39:38.937824+0000","last_became_active":"2024-09-19T08:39:38.925343+0000","last_became_peered":"2024-09-19T08:39:38.925343+0000","last_unstale":"2024-09-19T08:39:38.937824+0000","last_undegraded":"2024-09-19T08:39:38.937824+0000","last_fullsized":"2024-09-19T08:39:38.937824+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:41.168755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"42'1","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.949593+0000","last_change":"2024-09-19T08:39:37.911789+0000","last_active":"2024-09-19T08:39:38.949593+0000","last_peered":"2024-09-19T08:39:38.949593+0000","last_clean":"2024-09-19T08:39:38.949593+0000","last_became_active":"2024-09-19T08:39:37.911305+0000","last_became_peered":"2024-09-19T08:39:37.911305+0000","last_unstale":"2024-09-19T08:39:38.949593+0000","last_undegraded":"2024-09-19T08:39:38.949593+0000","last_fullsized":"2024-09-19T08:39:38.949593+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:13.592173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.938438+0000","last_change":"2024-09-19T08:39:38.935012+0000","last_active":"2024-09-19T08:39:38.938438+0000","last_peered":"2024-09-19T08:39:38.938438+0000","last_clean":"2024-09-19T08:39:38.938438+0000","last_became_active":"2024-09-19T08:39:38.934473+0000","last_became_peered":"2024-09-19T08:39:38.934473+0000","last_unstale":"2024-09-19T08:39:38.938438+0000","last_undegraded":"2024-09-19T08:39:38.938438+0000","last_fullsized":"2024-09-19T08:39:38.938438+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:33.862515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.959551+0000","last_change":"2024-09-19T08:39:37.923082+0000","last_active":"2024-09-19T08:39:38.959551+0000","last_peered":"2024-09-19T08:39:38.959551+0000","last_clean":"2024-09-19T08:39:38.959551+0000","last_became_active":"2024-09-19T08:39:37.922381+0000","last_became_peered":"2024-09-19T08:39:37.922381+0000","last_unstale":"2024-09-19T08:39:38.959551+0000","last_undegraded":"2024-09-19T08:39:38.959551+0000","last_fullsized":"2024-09-19T08:39:38.959551+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:08:43.556600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:37.883250+0000","last_change":"2024-09-19T08:39:37.883250+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.883250+0000","last_undegraded":"2024-09-19T08:39:37.883250+0000","last_fullsized":"2024-09-19T08:39:37.883250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"42'1","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.956915+0000","last_change":"2024-09-19T08:39:37.917129+0000","last_active":"2024-09-19T08:39:38.956915+0000","last_peered":"2024-09-19T08:39:38.956915+0000","last_clean":"2024-09-19T08:39:38.956915+0000","last_became_active":"2024-09-19T08:39:37.916360+0000","last_became_peered":"2024-09-19T08:39:37.916360+0000","last_unstale":"2024-09-19T08:39:38.956915+0000","last_undegraded":"2024-09-19T08:39:38.956915+0000","last_fullsized":"2024-09-19T08:39:38.956915+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:39.361088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:37.883250+0000","last_change":"2024-09-19T08:39:37.883250+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.883250+0000","last_undegraded":"2024-09-19T08:39:37.883250+0000","last_fullsized":"2024-09-19T08:39:37.883250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-19T08:39:37.937075+0000","last_change":"2024-09-19T08:39:37.932493+0000","last_active":"2024-09-19T08:39:37.937075+0000","last_peered":"2024-09-19T08:39:37.937075+0000","last_clean":"2024-09-19T08:39:37.937075+0000","last_became_active":"2024-09-19T08:39:37.932195+0000","last_became_peered":"2024-09-19T08:39:37.932195+0000","last_unstale":"2024-09-19T08:39:37.937075+0000","last_undegraded":"2024-09-19T08:39:37.937075+0000","last_fullsized":"2024-09-19T08:39:37.937075+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:32:30.169809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-09-19T08:39:37.896687+0000","last_change":"2024-09-19T08:39:37.896687+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.896687+0000","last_undegraded":"2024-09-19T08:39:37.896687+0000","last_fullsized":"2024-09-19T08:39:37.896687+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-19T08:39:37.919394+0000","last_change":"2024-09-19T08:39:37.913727+0000","last_active":"2024-09-19T08:39:37.919394+0000","last_peered":"2024-09-19T08:39:37.919394+0000","last_clean":"2024-09-19T08:39:37.919394+0000","last_became_active":"2024-09-19T08:39:37.913224+0000","last_became_peered":"2024-09-19T08:39:37.913224+0000","last_unstale":"2024-09-19T08:39:37.919394+0000","last_undegraded":"2024-09-19T08:39:37.919394+0000","last_fullsized":"2024-09-19T08:39:37.919394+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:54:58.881538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.928775+0000","last_change":"2024-09-19T08:39:38.923972+0000","last_active":"2024-09-19T08:39:38.928775+0000","last_peered":"2024-09-19T08:39:38.928775+0000","last_clean":"2024-09-19T08:39:38.928775+0000","last_became_active":"2024-09-19T08:39:38.923101+0000","last_became_peered":"2024-09-19T08:39:38.923101+0000","last_unstale":"2024-09-19T08:39:38.928775+0000","last_undegraded":"2024-09-19T08:39:38.928775+0000","last_fullsized":"2024-09-19T08:39:38.928775+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:08:44.560138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.911722+0000","last_change":"2024-09-19T08:39:37.934259+0000","last_active":"2024-09-19T08:39:38.911722+0000","last_peered":"2024-09-19T08:39:38.911722+0000","last_clean":"2024-09-19T08:39:38.911722+0000","last_became_active":"2024-09-19T08:39:37.933506+0000","last_became_peered":"2024-09-19T08:39:37.933506+0000","last_unstale":"2024-09-19T08:39:38.911722+0000","last_undegraded":"2024-09-19T08:39:38.911722+0000","last_fullsized":"2024-09-19T08:39:38.911722+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:22.044532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.948894+0000","last_change":"2024-09-19T08:39:38.944450+0000","last_active":"2024-09-19T08:39:38.948894+0000","last_peered":"2024-09-19T08:39:38.948894+0000","last_clean":"2024-09-19T08:39:38.948894+0000","last_became_active":"2024-09-19T08:39:38.944108+0000","last_became_peered":"2024-09-19T08:39:38.944108+0000","last_unstale":"2024-09-19T08:39:38.948894+0000","last_undegraded":"2024-09-19T08:39:38.948894+0000","last_fullsized":"2024-09-19T08:39:38.948894+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:56.145227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.901835+0000","last_change":"2024-09-19T08:39:37.922726+0000","last_active":"2024-09-19T08:39:38.901835+0000","last_peered":"2024-09-19T08:39:38.901835+0000","last_clean":"2024-09-19T08:39:38.901835+0000","last_became_active":"2024-09-19T08:39:37.922312+0000","last_became_peered":"2024-09-19T08:39:37.922312+0000","last_unstale":"2024-09-19T08:39:38.901835+0000","last_undegraded":"2024-09-19T08:39:38.901835+0000","last_fullsized":"2024-09-19T08:39:38.901835+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:05:28.346760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.935437+0000","last_change":"2024-09-19T08:39:38.920148+0000","last_active":"2024-09-19T08:39:38.935437+0000","last_peered":"2024-09-19T08:39:38.935437+0000","last_clean":"2024-09-19T08:39:38.935437+0000","last_became_active":"2024-09-19T08:39:38.919337+0000","last_became_peered":"2024-09-19T08:39:38.919337+0000","last_unstale":"2024-09-19T08:39:38.935437+0000","last_undegraded":"2024-09-19T08:39:38.935437+0000","last_fullsized":"2024-09-19T08:39:38.935437+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:27:20.365465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.908966+0000","last_change":"2024-09-19T08:39:37.915040+0000","last_active":"2024-09-19T08:39:38.908966+0000","last_peered":"2024-09-19T08:39:38.908966+0000","last_clean":"2024-09-19T08:39:38.908966+0000","last_became_active":"2024-09-19T08:39:37.914232+0000","last_became_peered":"2024-09-19T08:39:37.914232+0000","last_unstale":"2024-09-19T08:39:38.908966+0000","last_undegraded":"2024-09-19T08:39:38.908966+0000","last_fullsized":"2024-09-19T08:39:38.908966+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:49:13.065050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:37.883250+0000","last_change":"2024-09-19T08:39:37.883250+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.883250+0000","last_undegraded":"2024-09-19T08:39:37.883250+0000","last_fullsized":"2024-09-19T08:39:37.883250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-19T08:39:37.922144+0000","last_change":"2024-09-19T08:39:37.918815+0000","last_active":"2024-09-19T08:39:37.922144+0000","last_peered":"2024-09-19T08:39:37.922144+0000","last_clean":"2024-09-19T08:39:37.922144+0000","last_became_active":"2024-09-19T08:39:37.918481+0000","last_became_peered":"2024-09-19T08:39:37.918481+0000","last_unstale":"2024-09-19T08:39:37.922144+0000","last_undegraded":"2024-09-19T08:39:37.922144+0000","last_fullsized":"2024-09-19T08:39:37.922144+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:55:34.226831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.936888+0000","last_change":"2024-09-19T08:39:38.929785+0000","last_active":"2024-09-19T08:39:38.936888+0000","last_peered":"2024-09-19T08:39:38.936888+0000","last_clean":"2024-09-19T08:39:38.936888+0000","last_became_active":"2024-09-19T08:39:38.929074+0000","last_became_peered":"2024-09-19T08:39:38.929074+0000","last_unstale":"2024-09-19T08:39:38.936888+0000","last_undegraded":"2024-09-19T08:39:38.936888+0000","last_fullsized":"2024-09-19T08:39:38.936888+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:37.035646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.918780+0000","last_change":"2024-09-19T08:39:37.936655+0000","last_active":"2024-09-19T08:39:38.918780+0000","last_peered":"2024-09-19T08:39:38.918780+0000","last_clean":"2024-09-19T08:39:38.918780+0000","last_became_active":"2024-09-19T08:39:37.936112+0000","last_became_peered":"2024-09-19T08:39:37.936112+0000","last_unstale":"2024-09-19T08:39:38.918780+0000","last_undegraded":"2024-09-19T08:39:38.918780+0000","last_fullsized":"2024-09-19T08:39:38.918780+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:33:43.714330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.959000+0000","last_change":"2024-09-19T08:39:38.942782+0000","last_active":"2024-09-19T08:39:38.959000+0000","last_peered":"2024-09-19T08:39:38.959000+0000","last_clean":"2024-09-19T08:39:38.959000+0000","last_became_active":"2024-09-19T08:39:38.939689+0000","last_became_peered":"2024-09-19T08:39:38.939689+0000","last_unstale":"2024-09-19T08:39:38.959000+0000","last_undegraded":"2024-09-19T08:39:38.959000+0000","last_fullsized":"2024-09-19T08:39:38.959000+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:37.082611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.895030+0000","last_change":"2024-09-19T08:39:37.924237+0000","last_active":"2024-09-19T08:39:38.895030+0000","last_peered":"2024-09-19T08:39:38.895030+0000","last_clean":"2024-09-19T08:39:38.895030+0000","last_became_active":"2024-09-19T08:39:37.923832+0000","last_became_peered":"2024-09-19T08:39:37.923832+0000","last_unstale":"2024-09-19T08:39:38.895030+0000","last_undegraded":"2024-09-19T08:39:38.895030+0000","last_fullsized":"2024-09-19T08:39:38.895030+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:14:31.270319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-09-19T08:39:37.896039+0000","last_change":"2024-09-19T08:39:37.896039+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.896039+0000","last_undegraded":"2024-09-19T08:39:37.896039+0000","last_fullsized":"2024-09-19T08:39:37.896039+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.919124+0000","last_change":"2024-09-19T08:39:37.934015+0000","last_active":"2024-09-19T08:39:38.919124+0000","last_peered":"2024-09-19T08:39:38.919124+0000","last_clean":"2024-09-19T08:39:38.919124+0000","last_became_active":"2024-09-19T08:39:37.933256+0000","last_became_peered":"2024-09-19T08:39:37.933256+0000","last_unstale":"2024-09-19T08:39:38.919124+0000","last_undegraded":"2024-09-19T08:39:38.919124+0000","last_fullsized":"2024-09-19T08:39:38.919124+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:55:09.923681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-09-19T08:39:37.896215+0000","last_change":"2024-09-19T08:39:37.896215+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.896215+0000","last_undegraded":"2024-09-19T08:39:37.896215+0000","last_fullsized":"2024-09-19T08:39:37.896215+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.901249+0000","last_change":"2024-09-19T08:39:37.920347+0000","last_active":"2024-09-19T08:39:38.901249+0000","last_peered":"2024-09-19T08:39:38.901249+0000","last_clean":"2024-09-19T08:39:38.901249+0000","last_became_active":"2024-09-19T08:39:37.919634+0000","last_became_peered":"2024-09-19T08:39:37.919634+0000","last_unstale":"2024-09-19T08:39:38.901249+0000","last_undegraded":"2024-09-19T08:39:38.901249+0000","last_fullsized":"2024-09-19T08:39:38.901249+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:14:31.270319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.933874+0000","last_change":"2024-09-19T08:39:38.927845+0000","last_active":"2024-09-19T08:39:38.933874+0000","last_peered":"2024-09-19T08:39:38.933874+0000","last_clean":"2024-09-19T08:39:38.933874+0000","last_became_active":"2024-09-19T08:39:38.926989+0000","last_became_peered":"2024-09-19T08:39:38.926989+0000","last_unstale":"2024-09-19T08:39:38.933874+0000","last_undegraded":"2024-09-19T08:39:38.933874+0000","last_fullsized":"2024-09-19T08:39:38.933874+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:55:10.927219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.895000+0000","last_change":"2024-09-19T08:39:37.910329+0000","last_active":"2024-09-19T08:39:38.895000+0000","last_peered":"2024-09-19T08:39:38.895000+0000","last_clean":"2024-09-19T08:39:38.895000+0000","last_became_active":"2024-09-19T08:39:37.909966+0000","last_became_peered":"2024-09-19T08:39:37.909966+0000","last_unstale":"2024-09-19T08:39:38.895000+0000","last_undegraded":"2024-09-19T08:39:38.895000+0000","last_fullsized":"2024-09-19T08:39:38.895000+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:36.032108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.942950+0000","last_change":"2024-09-19T08:39:38.936998+0000","last_active":"2024-09-19T08:39:38.942950+0000","last_peered":"2024-09-19T08:39:38.942950+0000","last_clean":"2024-09-19T08:39:38.942950+0000","last_became_active":"2024-09-19T08:39:38.935884+0000","last_became_peered":"2024-09-19T08:39:38.935884+0000","last_unstale":"2024-09-19T08:39:38.942950+0000","last_undegraded":"2024-09-19T08:39:38.942950+0000","last_fullsized":"2024-09-19T08:39:38.942950+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:40.364626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.903222+0000","last_change":"2024-09-19T08:39:37.907408+0000","last_active":"2024-09-19T08:39:38.903222+0000","last_peered":"2024-09-19T08:39:38.903222+0000","last_clean":"2024-09-19T08:39:38.903222+0000","last_became_active":"2024-09-19T08:39:37.906928+0000","last_became_peered":"2024-09-19T08:39:37.906928+0000","last_unstale":"2024-09-19T08:39:38.903222+0000","last_undegraded":"2024-09-19T08:39:38.903222+0000","last_fullsized":"2024-09-19T08:39:38.903222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:08.285216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.945594+0000","last_change":"2024-09-19T08:39:38.940058+0000","last_active":"2024-09-19T08:39:38.945594+0000","last_peered":"2024-09-19T08:39:38.945594+0000","last_clean":"2024-09-19T08:39:38.945594+0000","last_became_active":"2024-09-19T08:39:38.939663+0000","last_became_peered":"2024-09-19T08:39:38.939663+0000","last_unstale":"2024-09-19T08:39:38.945594+0000","last_undegraded":"2024-09-19T08:39:38.945594+0000","last_fullsized":"2024-09-19T08:39:38.945594+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:40.364626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.957136+0000","last_change":"2024-09-19T08:39:37.922845+0000","last_active":"2024-09-19T08:39:38.957136+0000","last_peered":"2024-09-19T08:39:38.957136+0000","last_clean":"2024-09-19T08:39:38.957136+0000","last_became_active":"2024-09-19T08:39:37.922153+0000","last_became_peered":"2024-09-19T08:39:37.922153+0000","last_unstale":"2024-09-19T08:39:38.957136+0000","last_undegraded":"2024-09-19T08:39:38.957136+0000","last_fullsized":"2024-09-19T08:39:38.957136+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:28.741298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":204,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.908394+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:39:38.908394+0000","last_peered":"2024-09-19T08:39:38.908394+0000","last_clean":"2024-09-19T08:39:38.908394+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:39:38.908394+0000","last_undegraded":"2024-09-19T08:39:38.908394+0000","last_fullsized":"2024-09-19T08:39:38.908394+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":104,"acting":104,"num_store_stats":7},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1794,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":29,"ondisk_log_size":29,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509461,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1081344,"data_stored":783009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509463,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":185776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509463,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":471040,"data_stored":185776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509460,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":185776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509462,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":185776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509460,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28072,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343768,"statfs":{"total":96636764160,"available":96608018432,"internally_reserved":0,"allocated":1064960,"data_stored":776144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":179509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542165,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31636,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340204,"statfs":{"total":96636764160,"available":96604368896,"internally_reserved":0,"allocated":458752,"data_stored":179509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:39:43.313 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:39:43.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.467+0000 7fd65ced7640 1 Processor -- start 2024-09-19T08:39:43.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.467+0000 7fd65ced7640 1 -- start start 2024-09-19T08:39:43.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.471+0000 7fd65ced7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd658140d30 0x7fd65806d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:43.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.471+0000 7fd65ced7640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd658141110 0x7fd65806d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:43.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.471+0000 7fd65ced7640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd65815ea10 con 0x7fd6580b3310 2024-09-19T08:39:43.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.471+0000 7fd65ced7640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fd65815e7c0 con 0x7fd658141110 2024-09-19T08:39:43.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.471+0000 7fd65ced7640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fd65815ecf0 con 0x7fd658140d30 2024-09-19T08:39:43.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.471+0000 7fd64ffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd658141110 0x7fd65806d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:43.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.471+0000 7fd64ffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd658141110 0x7fd65806d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49604/0 (socket says 172.21.15.98:49604) 2024-09-19T08:39:43.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.471+0000 7fd64ffff640 1 -- 172.21.15.98:0/3521330694 learned_addr learned my addr 172.21.15.98:0/3521330694 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:43.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.471+0000 7fd65659c640 1 --2- 172.21.15.98:0/3521330694 >> v2:172.21.15.139:3300/0 conn(0x7fd658140d30 0x7fd65806d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:43.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.471+0000 7fd64ffff640 1 -- 172.21.15.98:0/3521330694 >> v2:172.21.15.139:3300/0 conn(0x7fd658140d30 msgr2=0x7fd65806d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:43.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.471+0000 7fd64ffff640 1 --2- 172.21.15.98:0/3521330694 >> v2:172.21.15.139:3300/0 conn(0x7fd658140d30 0x7fd65806d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:43.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.471+0000 7fd64ffff640 1 -- 172.21.15.98:0/3521330694 >> v1:172.21.15.139:6789/0 conn(0x7fd6580b3310 legacy=0x7fd6580b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:43.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.471+0000 7fd64ffff640 1 -- 172.21.15.98:0/3521330694 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd65815efd0 con 0x7fd658141110 2024-09-19T08:39:43.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.471+0000 7fd65659c640 1 --2- 172.21.15.98:0/3521330694 >> v2:172.21.15.139:3300/0 conn(0x7fd658140d30 0x7fd65806d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:43.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.471+0000 7fd64ffff640 1 --2- 172.21.15.98:0/3521330694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd658141110 0x7fd65806d6b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd6580d1370 tx=0x7fd64001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90131b89ec9ff338 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:43.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.471+0000 7fd655d9b640 1 -- 172.21.15.98:0/3521330694 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6400aa660 con 0x7fd658141110 2024-09-19T08:39:43.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.471+0000 7fd655d9b640 1 -- 172.21.15.98:0/3521330694 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6400af070 con 0x7fd658141110 2024-09-19T08:39:43.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.471+0000 7fd655d9b640 1 -- 172.21.15.98:0/3521330694 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6400aaa20 con 0x7fd658141110 2024-09-19T08:39:43.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.475+0000 7fd65ced7640 1 -- 172.21.15.98:0/3521330694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd658141110 msgr2=0x7fd65806d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:43.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.475+0000 7fd65ced7640 1 --2- 172.21.15.98:0/3521330694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd658141110 0x7fd65806d6b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd6580d1370 tx=0x7fd64001c4e0 comp rx=0 tx=0).stop 2024-09-19T08:39:43.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.475+0000 7fd65ced7640 1 -- 172.21.15.98:0/3521330694 shutdown_connections 2024-09-19T08:39:43.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.475+0000 7fd65ced7640 1 --2- 172.21.15.98:0/3521330694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd658141110 0x7fd65806d6b0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:43.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.475+0000 7fd65ced7640 1 --2- 172.21.15.98:0/3521330694 >> v2:172.21.15.139:3300/0 conn(0x7fd658140d30 0x7fd65806d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:43.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.475+0000 7fd65ced7640 1 -- 172.21.15.98:0/3521330694 >> 172.21.15.98:0/3521330694 conn(0x7fd6580b2010 msgr2=0x7fd6580b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:43.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.475+0000 7fd65ced7640 1 -- 172.21.15.98:0/3521330694 shutdown_connections 2024-09-19T08:39:43.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.475+0000 7fd65ced7640 1 -- 172.21.15.98:0/3521330694 wait complete. 2024-09-19T08:39:43.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.475+0000 7fd65ced7640 1 Processor -- start 2024-09-19T08:39:43.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.475+0000 7fd65ced7640 1 -- start start 2024-09-19T08:39:43.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.475+0000 7fd65ced7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6580b3310 0x7fd6581535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:43.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.475+0000 7fd65ced7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd658140d30 0x7fd658140040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:43.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.475+0000 7fd65ced7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd658141110 0x7fd658159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:43.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.475+0000 7fd65ced7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd65815bdd0 con 0x7fd6580b3310 2024-09-19T08:39:43.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.475+0000 7fd65ced7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd65815bc50 con 0x7fd658141110 2024-09-19T08:39:43.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.475+0000 7fd65ced7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd65815bf50 con 0x7fd658140d30 2024-09-19T08:39:43.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.475+0000 7fd65659c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6580b3310 0x7fd6581535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:43.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.475+0000 7fd65659c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6580b3310 0x7fd6581535d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49608/0 (socket says 172.21.15.98:49608) 2024-09-19T08:39:43.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.475+0000 7fd65659c640 1 -- 172.21.15.98:0/3377366158 learned_addr learned my addr 172.21.15.98:0/3377366158 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:43.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.475+0000 7fd656d9d640 1 --2- 172.21.15.98:0/3377366158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd658141110 0x7fd658159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:43.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.475+0000 7fd64ffff640 1 --2- 172.21.15.98:0/3377366158 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd658140d30 0x7fd658140040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:43.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.475+0000 7fd65659c640 1 -- 172.21.15.98:0/3377366158 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd658140d30 msgr2=0x7fd658140040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:43.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.475+0000 7fd65659c640 1 --2- 172.21.15.98:0/3377366158 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd658140d30 0x7fd658140040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:43.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.475+0000 7fd65659c640 1 -- 172.21.15.98:0/3377366158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd658141110 msgr2=0x7fd658159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:43.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.475+0000 7fd65659c640 1 --2- 172.21.15.98:0/3377366158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd658141110 0x7fd658159190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:43.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.475+0000 7fd65659c640 1 -- 172.21.15.98:0/3377366158 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd658152850 con 0x7fd6580b3310 2024-09-19T08:39:43.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.475+0000 7fd65659c640 1 --2- 172.21.15.98:0/3377366158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6580b3310 0x7fd6581535d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd64800c7c0 tx=0x7fd64800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:43.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.475+0000 7fd64effd640 1 -- 172.21.15.98:0/3377366158 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd648007a30 con 0x7fd6580b3310 2024-09-19T08:39:43.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.475+0000 7fd64effd640 1 -- 172.21.15.98:0/3377366158 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd648007bd0 con 0x7fd6580b3310 2024-09-19T08:39:43.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.475+0000 7fd64effd640 1 -- 172.21.15.98:0/3377366158 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6480113f0 con 0x7fd6580b3310 2024-09-19T08:39:43.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.475+0000 7fd65ced7640 1 -- 172.21.15.98:0/3377366158 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd658140580 con 0x7fd6580b3310 2024-09-19T08:39:43.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.475+0000 7fd65ced7640 1 -- 172.21.15.98:0/3377366158 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd658152dd0 con 0x7fd6580b3310 2024-09-19T08:39:43.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.479+0000 7fd65ced7640 1 -- 172.21.15.98:0/3377366158 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6580b36f0 con 0x7fd6580b3310 2024-09-19T08:39:43.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.479+0000 7fd64effd640 1 -- 172.21.15.98:0/3377366158 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fd648011590 con 0x7fd6580b3310 2024-09-19T08:39:43.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.479+0000 7fd64effd640 1 --2- 172.21.15.98:0/3377366158 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd638067aa0 0x7fd638069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:43.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.479+0000 7fd64effd640 1 -- 172.21.15.98:0/3377366158 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fd648086240 con 0x7fd6580b3310 2024-09-19T08:39:43.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.479+0000 7fd64ffff640 1 --2- 172.21.15.98:0/3377366158 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd638067aa0 0x7fd638069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:43.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.479+0000 7fd64ffff640 1 --2- 172.21.15.98:0/3377366158 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd638067aa0 0x7fd638069f60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd6400ae000 tx=0x7fd6400a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:43.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.483+0000 7fd64effd640 1 -- 172.21.15.98:0/3377366158 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd648015020 con 0x7fd6580b3310 2024-09-19T08:39:43.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.639+0000 7fd65ced7640 1 -- 172.21.15.98:0/3377366158 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd658160920 con 0x7fd638067aa0 2024-09-19T08:39:43.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.651+0000 7fd64effd640 1 -- 172.21.15.98:0/3377366158 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249325 (secure 0 0 0) 0x7fd6400af040 con 0x7fd638067aa0 2024-09-19T08:39:43.651 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:39:43.653 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:39:43.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.655+0000 7fd65ced7640 1 -- 172.21.15.98:0/3377366158 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd638067aa0 msgr2=0x7fd638069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:43.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.655+0000 7fd65ced7640 1 --2- 172.21.15.98:0/3377366158 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd638067aa0 0x7fd638069f60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd6400ae000 tx=0x7fd6400a7000 comp rx=0 tx=0).stop 2024-09-19T08:39:43.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.655+0000 7fd65ced7640 1 -- 172.21.15.98:0/3377366158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6580b3310 msgr2=0x7fd6581535d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:43.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.655+0000 7fd65ced7640 1 --2- 172.21.15.98:0/3377366158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6580b3310 0x7fd6581535d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd64800c7c0 tx=0x7fd64800cc80 comp rx=0 tx=0).stop 2024-09-19T08:39:43.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.655+0000 7fd65ced7640 1 -- 172.21.15.98:0/3377366158 shutdown_connections 2024-09-19T08:39:43.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.655+0000 7fd65ced7640 1 --2- 172.21.15.98:0/3377366158 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd638067aa0 0x7fd638069f60 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:43.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.655+0000 7fd65ced7640 1 --2- 172.21.15.98:0/3377366158 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd658141110 0x7fd658159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:43.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.655+0000 7fd65ced7640 1 --2- 172.21.15.98:0/3377366158 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd658140d30 0x7fd658140040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:43.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.655+0000 7fd65ced7640 1 --2- 172.21.15.98:0/3377366158 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6580b3310 0x7fd6581535d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:43.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.655+0000 7fd65ced7640 1 -- 172.21.15.98:0/3377366158 >> 172.21.15.98:0/3377366158 conn(0x7fd6580b2010 msgr2=0x7fd6581582a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:43.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.655+0000 7fd65ced7640 1 -- 172.21.15.98:0/3377366158 shutdown_connections 2024-09-19T08:39:43.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.655+0000 7fd65ced7640 1 -- 172.21.15.98:0/3377366158 wait complete. 2024-09-19T08:39:43.670 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2024-09-19T08:39:42.050723+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592162,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":195,"num_write_kb":2422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":170,"acting":170,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":30,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":229392,"kb_used_data":4832,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754745328,"statfs":{"total":773094113280,"available":772859215872,"internally_reserved":0,"allocated":4947968,"data_stored":2661275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1794,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.166397"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.958882+0000","last_change":"2024-09-19T08:39:38.938293+0000","last_active":"2024-09-19T08:39:38.958882+0000","last_peered":"2024-09-19T08:39:38.958882+0000","last_clean":"2024-09-19T08:39:38.958882+0000","last_became_active":"2024-09-19T08:39:38.936784+0000","last_became_peered":"2024-09-19T08:39:38.936784+0000","last_unstale":"2024-09-19T08:39:38.958882+0000","last_undegraded":"2024-09-19T08:39:38.958882+0000","last_fullsized":"2024-09-19T08:39:38.958882+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:08:18.820654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.944459+0000","last_change":"2024-09-19T08:39:38.942347+0000","last_active":"2024-09-19T08:39:38.944459+0000","last_peered":"2024-09-19T08:39:38.944459+0000","last_clean":"2024-09-19T08:39:38.944459+0000","last_became_active":"2024-09-19T08:39:38.941641+0000","last_became_peered":"2024-09-19T08:39:38.941641+0000","last_unstale":"2024-09-19T08:39:38.944459+0000","last_undegraded":"2024-09-19T08:39:38.944459+0000","last_fullsized":"2024-09-19T08:39:38.944459+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:29.732835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.933674+0000","last_change":"2024-09-19T08:39:38.918421+0000","last_active":"2024-09-19T08:39:38.933674+0000","last_peered":"2024-09-19T08:39:38.933674+0000","last_clean":"2024-09-19T08:39:38.933674+0000","last_became_active":"2024-09-19T08:39:38.917761+0000","last_became_peered":"2024-09-19T08:39:38.917761+0000","last_unstale":"2024-09-19T08:39:38.933674+0000","last_undegraded":"2024-09-19T08:39:38.933674+0000","last_fullsized":"2024-09-19T08:39:38.933674+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:31.825354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.936913+0000","last_change":"2024-09-19T08:39:38.932965+0000","last_active":"2024-09-19T08:39:38.936913+0000","last_peered":"2024-09-19T08:39:38.936913+0000","last_clean":"2024-09-19T08:39:38.936913+0000","last_became_active":"2024-09-19T08:39:38.929490+0000","last_became_peered":"2024-09-19T08:39:38.929490+0000","last_unstale":"2024-09-19T08:39:38.936913+0000","last_undegraded":"2024-09-19T08:39:38.936913+0000","last_fullsized":"2024-09-19T08:39:38.936913+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:30:09.863827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.942469+0000","last_change":"2024-09-19T08:39:38.938221+0000","last_active":"2024-09-19T08:39:38.942469+0000","last_peered":"2024-09-19T08:39:38.942469+0000","last_clean":"2024-09-19T08:39:38.942469+0000","last_became_active":"2024-09-19T08:39:38.937356+0000","last_became_peered":"2024-09-19T08:39:38.937356+0000","last_unstale":"2024-09-19T08:39:38.942469+0000","last_undegraded":"2024-09-19T08:39:38.942469+0000","last_fullsized":"2024-09-19T08:39:38.942469+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:00:09.742802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-09-19T08:39:37.898206+0000","last_change":"2024-09-19T08:39:37.898206+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.898206+0000","last_undegraded":"2024-09-19T08:39:37.898206+0000","last_fullsized":"2024-09-19T08:39:37.898206+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.961008+0000","last_change":"2024-09-19T08:39:38.945329+0000","last_active":"2024-09-19T08:39:38.961008+0000","last_peered":"2024-09-19T08:39:38.961008+0000","last_clean":"2024-09-19T08:39:38.961008+0000","last_became_active":"2024-09-19T08:39:38.944121+0000","last_became_peered":"2024-09-19T08:39:38.944121+0000","last_unstale":"2024-09-19T08:39:38.961008+0000","last_undegraded":"2024-09-19T08:39:38.961008+0000","last_fullsized":"2024-09-19T08:39:38.961008+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:37:14.987023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.937026+0000","last_change":"2024-09-19T08:39:38.929792+0000","last_active":"2024-09-19T08:39:38.937026+0000","last_peered":"2024-09-19T08:39:38.937026+0000","last_clean":"2024-09-19T08:39:38.937026+0000","last_became_active":"2024-09-19T08:39:38.928743+0000","last_became_peered":"2024-09-19T08:39:38.928743+0000","last_unstale":"2024-09-19T08:39:38.937026+0000","last_undegraded":"2024-09-19T08:39:38.937026+0000","last_fullsized":"2024-09-19T08:39:38.937026+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:14:32.273857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.937132+0000","last_change":"2024-09-19T08:39:38.934773+0000","last_active":"2024-09-19T08:39:38.937132+0000","last_peered":"2024-09-19T08:39:38.937132+0000","last_clean":"2024-09-19T08:39:38.937132+0000","last_became_active":"2024-09-19T08:39:38.934054+0000","last_became_peered":"2024-09-19T08:39:38.934054+0000","last_unstale":"2024-09-19T08:39:38.937132+0000","last_undegraded":"2024-09-19T08:39:38.937132+0000","last_fullsized":"2024-09-19T08:39:38.937132+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:31.825354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.937106+0000","last_change":"2024-09-19T08:39:38.934619+0000","last_active":"2024-09-19T08:39:38.937106+0000","last_peered":"2024-09-19T08:39:38.937106+0000","last_clean":"2024-09-19T08:39:38.937106+0000","last_became_active":"2024-09-19T08:39:38.934244+0000","last_became_peered":"2024-09-19T08:39:38.934244+0000","last_unstale":"2024-09-19T08:39:38.937106+0000","last_undegraded":"2024-09-19T08:39:38.937106+0000","last_fullsized":"2024-09-19T08:39:38.937106+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:08:44.560138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.937073+0000","last_change":"2024-09-19T08:39:38.933595+0000","last_active":"2024-09-19T08:39:38.937073+0000","last_peered":"2024-09-19T08:39:38.937073+0000","last_clean":"2024-09-19T08:39:38.937073+0000","last_became_active":"2024-09-19T08:39:38.933316+0000","last_became_peered":"2024-09-19T08:39:38.933316+0000","last_unstale":"2024-09-19T08:39:38.937073+0000","last_undegraded":"2024-09-19T08:39:38.937073+0000","last_fullsized":"2024-09-19T08:39:38.937073+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:43.876237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.928186+0000","last_change":"2024-09-19T08:39:38.923904+0000","last_active":"2024-09-19T08:39:38.928186+0000","last_peered":"2024-09-19T08:39:38.928186+0000","last_clean":"2024-09-19T08:39:38.928186+0000","last_became_active":"2024-09-19T08:39:38.923135+0000","last_became_peered":"2024-09-19T08:39:38.923135+0000","last_unstale":"2024-09-19T08:39:38.928186+0000","last_undegraded":"2024-09-19T08:39:38.928186+0000","last_fullsized":"2024-09-19T08:39:38.928186+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:06.457317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.938802+0000","last_change":"2024-09-19T08:39:38.933031+0000","last_active":"2024-09-19T08:39:38.938802+0000","last_peered":"2024-09-19T08:39:38.938802+0000","last_clean":"2024-09-19T08:39:38.938802+0000","last_became_active":"2024-09-19T08:39:38.932568+0000","last_became_peered":"2024-09-19T08:39:38.932568+0000","last_unstale":"2024-09-19T08:39:38.938802+0000","last_undegraded":"2024-09-19T08:39:38.938802+0000","last_fullsized":"2024-09-19T08:39:38.938802+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:41:56.688886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.940685+0000","last_change":"2024-09-19T08:39:38.936306+0000","last_active":"2024-09-19T08:39:38.940685+0000","last_peered":"2024-09-19T08:39:38.940685+0000","last_clean":"2024-09-19T08:39:38.940685+0000","last_became_active":"2024-09-19T08:39:38.935988+0000","last_became_peered":"2024-09-19T08:39:38.935988+0000","last_unstale":"2024-09-19T08:39:38.940685+0000","last_undegraded":"2024-09-19T08:39:38.940685+0000","last_fullsized":"2024-09-19T08:39:38.940685+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:49:14.068588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-09-19T08:39:37.899012+0000","last_change":"2024-09-19T08:39:37.899012+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.899012+0000","last_undegraded":"2024-09-19T08:39:37.899012+0000","last_fullsized":"2024-09-19T08:39:37.899012+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:37.883250+0000","last_change":"2024-09-19T08:39:37.883250+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.883250+0000","last_undegraded":"2024-09-19T08:39:37.883250+0000","last_fullsized":"2024-09-19T08:39:37.883250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.933907+0000","last_change":"2024-09-19T08:39:38.924533+0000","last_active":"2024-09-19T08:39:38.933907+0000","last_peered":"2024-09-19T08:39:38.933907+0000","last_clean":"2024-09-19T08:39:38.933907+0000","last_became_active":"2024-09-19T08:39:38.923817+0000","last_became_peered":"2024-09-19T08:39:38.923817+0000","last_unstale":"2024-09-19T08:39:38.933907+0000","last_undegraded":"2024-09-19T08:39:38.933907+0000","last_fullsized":"2024-09-19T08:39:38.933907+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:29.744836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.943119+0000","last_change":"2024-09-19T08:39:38.938932+0000","last_active":"2024-09-19T08:39:38.943119+0000","last_peered":"2024-09-19T08:39:38.943119+0000","last_clean":"2024-09-19T08:39:38.943119+0000","last_became_active":"2024-09-19T08:39:38.937330+0000","last_became_peered":"2024-09-19T08:39:38.937330+0000","last_unstale":"2024-09-19T08:39:38.943119+0000","last_undegraded":"2024-09-19T08:39:38.943119+0000","last_fullsized":"2024-09-19T08:39:38.943119+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:06.457317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:37.883250+0000","last_change":"2024-09-19T08:39:37.883250+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.883250+0000","last_undegraded":"2024-09-19T08:39:37.883250+0000","last_fullsized":"2024-09-19T08:39:37.883250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.938851+0000","last_change":"2024-09-19T08:39:38.936241+0000","last_active":"2024-09-19T08:39:38.938851+0000","last_peered":"2024-09-19T08:39:38.938851+0000","last_clean":"2024-09-19T08:39:38.938851+0000","last_became_active":"2024-09-19T08:39:38.935936+0000","last_became_peered":"2024-09-19T08:39:38.935936+0000","last_unstale":"2024-09-19T08:39:38.938851+0000","last_undegraded":"2024-09-19T08:39:38.938851+0000","last_fullsized":"2024-09-19T08:39:38.938851+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:49.679363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-09-19T08:39:37.898343+0000","last_change":"2024-09-19T08:39:37.898343+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.898343+0000","last_undegraded":"2024-09-19T08:39:37.898343+0000","last_fullsized":"2024-09-19T08:39:37.898343+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.938460+0000","last_change":"2024-09-19T08:39:38.935680+0000","last_active":"2024-09-19T08:39:38.938460+0000","last_peered":"2024-09-19T08:39:38.938460+0000","last_clean":"2024-09-19T08:39:38.938460+0000","last_became_active":"2024-09-19T08:39:38.929257+0000","last_became_peered":"2024-09-19T08:39:38.929257+0000","last_unstale":"2024-09-19T08:39:38.938460+0000","last_undegraded":"2024-09-19T08:39:38.938460+0000","last_fullsized":"2024-09-19T08:39:38.938460+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:23.048070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:37.883250+0000","last_change":"2024-09-19T08:39:37.883250+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.883250+0000","last_undegraded":"2024-09-19T08:39:37.883250+0000","last_fullsized":"2024-09-19T08:39:37.883250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:37.883250+0000","last_change":"2024-09-19T08:39:37.883250+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.883250+0000","last_undegraded":"2024-09-19T08:39:37.883250+0000","last_fullsized":"2024-09-19T08:39:37.883250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:37.883250+0000","last_change":"2024-09-19T08:39:37.883250+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.883250+0000","last_undegraded":"2024-09-19T08:39:37.883250+0000","last_fullsized":"2024-09-19T08:39:37.883250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.938757+0000","last_change":"2024-09-19T08:39:38.932467+0000","last_active":"2024-09-19T08:39:38.938757+0000","last_peered":"2024-09-19T08:39:38.938757+0000","last_clean":"2024-09-19T08:39:38.938757+0000","last_became_active":"2024-09-19T08:39:38.932023+0000","last_became_peered":"2024-09-19T08:39:38.932023+0000","last_unstale":"2024-09-19T08:39:38.938757+0000","last_undegraded":"2024-09-19T08:39:38.938757+0000","last_fullsized":"2024-09-19T08:39:38.938757+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:11.986036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.933854+0000","last_change":"2024-09-19T08:39:38.924547+0000","last_active":"2024-09-19T08:39:38.933854+0000","last_peered":"2024-09-19T08:39:38.933854+0000","last_clean":"2024-09-19T08:39:38.933854+0000","last_became_active":"2024-09-19T08:39:38.923802+0000","last_became_peered":"2024-09-19T08:39:38.923802+0000","last_unstale":"2024-09-19T08:39:38.933854+0000","last_undegraded":"2024-09-19T08:39:38.933854+0000","last_fullsized":"2024-09-19T08:39:38.933854+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:23.048070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.933859+0000","last_change":"2024-09-19T08:39:38.927516+0000","last_active":"2024-09-19T08:39:38.933859+0000","last_peered":"2024-09-19T08:39:38.933859+0000","last_clean":"2024-09-19T08:39:38.933859+0000","last_became_active":"2024-09-19T08:39:38.926015+0000","last_became_peered":"2024-09-19T08:39:38.926015+0000","last_unstale":"2024-09-19T08:39:38.933859+0000","last_undegraded":"2024-09-19T08:39:38.933859+0000","last_fullsized":"2024-09-19T08:39:38.933859+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:33:44.717869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:37.883250+0000","last_change":"2024-09-19T08:39:37.883250+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.883250+0000","last_undegraded":"2024-09-19T08:39:37.883250+0000","last_fullsized":"2024-09-19T08:39:37.883250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.934797+0000","last_change":"2024-09-19T08:39:38.931320+0000","last_active":"2024-09-19T08:39:38.934797+0000","last_peered":"2024-09-19T08:39:38.934797+0000","last_clean":"2024-09-19T08:39:38.934797+0000","last_became_active":"2024-09-19T08:39:38.930578+0000","last_became_peered":"2024-09-19T08:39:38.930578+0000","last_unstale":"2024-09-19T08:39:38.934797+0000","last_undegraded":"2024-09-19T08:39:38.934797+0000","last_fullsized":"2024-09-19T08:39:38.934797+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:49:14.068588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-09-19T08:39:37.897514+0000","last_change":"2024-09-19T08:39:37.897514+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.897514+0000","last_undegraded":"2024-09-19T08:39:37.897514+0000","last_fullsized":"2024-09-19T08:39:37.897514+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.930583+0000","last_change":"2024-09-19T08:39:38.921905+0000","last_active":"2024-09-19T08:39:38.930583+0000","last_peered":"2024-09-19T08:39:38.930583+0000","last_clean":"2024-09-19T08:39:38.930583+0000","last_became_active":"2024-09-19T08:39:38.920862+0000","last_became_peered":"2024-09-19T08:39:38.920862+0000","last_unstale":"2024-09-19T08:39:38.930583+0000","last_undegraded":"2024-09-19T08:39:38.930583+0000","last_fullsized":"2024-09-19T08:39:38.930583+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:43:08.445843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.950250+0000","last_change":"2024-09-19T08:39:38.942801+0000","last_active":"2024-09-19T08:39:38.950250+0000","last_peered":"2024-09-19T08:39:38.950250+0000","last_clean":"2024-09-19T08:39:38.950250+0000","last_became_active":"2024-09-19T08:39:38.942568+0000","last_became_peered":"2024-09-19T08:39:38.942568+0000","last_unstale":"2024-09-19T08:39:38.950250+0000","last_undegraded":"2024-09-19T08:39:38.950250+0000","last_fullsized":"2024-09-19T08:39:38.950250+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:31.825354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-19T08:39:37.942662+0000","last_change":"2024-09-19T08:39:37.937397+0000","last_active":"2024-09-19T08:39:37.942662+0000","last_peered":"2024-09-19T08:39:37.942662+0000","last_clean":"2024-09-19T08:39:37.942662+0000","last_became_active":"2024-09-19T08:39:37.937128+0000","last_became_peered":"2024-09-19T08:39:37.937128+0000","last_unstale":"2024-09-19T08:39:37.942662+0000","last_undegraded":"2024-09-19T08:39:37.942662+0000","last_fullsized":"2024-09-19T08:39:37.942662+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:47.949747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:37.883250+0000","last_change":"2024-09-19T08:39:37.883250+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.883250+0000","last_undegraded":"2024-09-19T08:39:37.883250+0000","last_fullsized":"2024-09-19T08:39:37.883250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"42'1","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.959478+0000","last_change":"2024-09-19T08:39:37.932635+0000","last_active":"2024-09-19T08:39:38.959478+0000","last_peered":"2024-09-19T08:39:38.959478+0000","last_clean":"2024-09-19T08:39:38.959478+0000","last_became_active":"2024-09-19T08:39:37.931841+0000","last_became_peered":"2024-09-19T08:39:37.931841+0000","last_unstale":"2024-09-19T08:39:38.959478+0000","last_undegraded":"2024-09-19T08:39:38.959478+0000","last_fullsized":"2024-09-19T08:39:38.959478+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:42.872698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.943117+0000","last_change":"2024-09-19T08:39:38.940685+0000","last_active":"2024-09-19T08:39:38.943117+0000","last_peered":"2024-09-19T08:39:38.943117+0000","last_clean":"2024-09-19T08:39:38.943117+0000","last_became_active":"2024-09-19T08:39:38.935293+0000","last_became_peered":"2024-09-19T08:39:38.935293+0000","last_unstale":"2024-09-19T08:39:38.943117+0000","last_undegraded":"2024-09-19T08:39:38.943117+0000","last_fullsized":"2024-09-19T08:39:38.943117+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:08:18.820654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.956642+0000","last_change":"2024-09-19T08:39:37.917790+0000","last_active":"2024-09-19T08:39:38.956642+0000","last_peered":"2024-09-19T08:39:38.956642+0000","last_clean":"2024-09-19T08:39:38.956642+0000","last_became_active":"2024-09-19T08:39:37.917479+0000","last_became_peered":"2024-09-19T08:39:37.917479+0000","last_unstale":"2024-09-19T08:39:38.956642+0000","last_undegraded":"2024-09-19T08:39:38.956642+0000","last_fullsized":"2024-09-19T08:39:38.956642+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:30:08.860289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.927097+0000","last_change":"2024-09-19T08:39:38.920238+0000","last_active":"2024-09-19T08:39:38.927097+0000","last_peered":"2024-09-19T08:39:38.927097+0000","last_clean":"2024-09-19T08:39:38.927097+0000","last_became_active":"2024-09-19T08:39:38.919486+0000","last_became_peered":"2024-09-19T08:39:38.919486+0000","last_unstale":"2024-09-19T08:39:38.927097+0000","last_undegraded":"2024-09-19T08:39:38.927097+0000","last_fullsized":"2024-09-19T08:39:38.927097+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:43.876237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.961365+0000","last_change":"2024-09-19T08:39:37.927771+0000","last_active":"2024-09-19T08:39:38.961365+0000","last_peered":"2024-09-19T08:39:38.961365+0000","last_clean":"2024-09-19T08:39:38.961365+0000","last_became_active":"2024-09-19T08:39:37.926946+0000","last_became_peered":"2024-09-19T08:39:37.926946+0000","last_unstale":"2024-09-19T08:39:38.961365+0000","last_undegraded":"2024-09-19T08:39:38.961365+0000","last_fullsized":"2024-09-19T08:39:38.961365+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:39.361088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.933755+0000","last_change":"2024-09-19T08:39:38.918556+0000","last_active":"2024-09-19T08:39:38.933755+0000","last_peered":"2024-09-19T08:39:38.933755+0000","last_clean":"2024-09-19T08:39:38.933755+0000","last_became_active":"2024-09-19T08:39:38.917830+0000","last_became_peered":"2024-09-19T08:39:38.917830+0000","last_unstale":"2024-09-19T08:39:38.933755+0000","last_undegraded":"2024-09-19T08:39:38.933755+0000","last_fullsized":"2024-09-19T08:39:38.933755+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:33.862515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-19T08:39:37.919363+0000","last_change":"2024-09-19T08:39:37.913524+0000","last_active":"2024-09-19T08:39:37.919363+0000","last_peered":"2024-09-19T08:39:37.919363+0000","last_clean":"2024-09-19T08:39:37.919363+0000","last_became_active":"2024-09-19T08:39:37.912853+0000","last_became_peered":"2024-09-19T08:39:37.912853+0000","last_unstale":"2024-09-19T08:39:37.919363+0000","last_undegraded":"2024-09-19T08:39:37.919363+0000","last_fullsized":"2024-09-19T08:39:37.919363+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:37:24.628377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.936990+0000","last_change":"2024-09-19T08:39:38.929719+0000","last_active":"2024-09-19T08:39:38.936990+0000","last_peered":"2024-09-19T08:39:38.936990+0000","last_clean":"2024-09-19T08:39:38.936990+0000","last_became_active":"2024-09-19T08:39:38.929158+0000","last_became_peered":"2024-09-19T08:39:38.929158+0000","last_unstale":"2024-09-19T08:39:38.936990+0000","last_undegraded":"2024-09-19T08:39:38.936990+0000","last_fullsized":"2024-09-19T08:39:38.936990+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:07:16.957999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"42'1","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.958601+0000","last_change":"2024-09-19T08:39:37.929415+0000","last_active":"2024-09-19T08:39:38.958601+0000","last_peered":"2024-09-19T08:39:38.958601+0000","last_clean":"2024-09-19T08:39:38.958601+0000","last_became_active":"2024-09-19T08:39:37.928610+0000","last_became_peered":"2024-09-19T08:39:37.928610+0000","last_unstale":"2024-09-19T08:39:38.958601+0000","last_undegraded":"2024-09-19T08:39:38.958601+0000","last_fullsized":"2024-09-19T08:39:38.958601+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:36.032108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.936735+0000","last_change":"2024-09-19T08:39:38.928185+0000","last_active":"2024-09-19T08:39:38.936735+0000","last_peered":"2024-09-19T08:39:38.936735+0000","last_clean":"2024-09-19T08:39:38.936735+0000","last_became_active":"2024-09-19T08:39:38.924812+0000","last_became_peered":"2024-09-19T08:39:38.924812+0000","last_unstale":"2024-09-19T08:39:38.936735+0000","last_undegraded":"2024-09-19T08:39:38.936735+0000","last_fullsized":"2024-09-19T08:39:38.936735+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:08.958729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.912110+0000","last_change":"2024-09-19T08:39:37.921882+0000","last_active":"2024-09-19T08:39:38.912110+0000","last_peered":"2024-09-19T08:39:38.912110+0000","last_clean":"2024-09-19T08:39:38.912110+0000","last_became_active":"2024-09-19T08:39:37.921061+0000","last_became_peered":"2024-09-19T08:39:37.921061+0000","last_unstale":"2024-09-19T08:39:38.912110+0000","last_undegraded":"2024-09-19T08:39:38.912110+0000","last_fullsized":"2024-09-19T08:39:38.912110+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:49:13.065050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:37.883250+0000","last_change":"2024-09-19T08:39:37.883250+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.883250+0000","last_undegraded":"2024-09-19T08:39:37.883250+0000","last_fullsized":"2024-09-19T08:39:37.883250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.902185+0000","last_change":"2024-09-19T08:39:37.914508+0000","last_active":"2024-09-19T08:39:38.902185+0000","last_peered":"2024-09-19T08:39:38.902185+0000","last_clean":"2024-09-19T08:39:38.902185+0000","last_became_active":"2024-09-19T08:39:37.914186+0000","last_became_peered":"2024-09-19T08:39:37.914186+0000","last_unstale":"2024-09-19T08:39:38.902185+0000","last_undegraded":"2024-09-19T08:39:38.902185+0000","last_fullsized":"2024-09-19T08:39:38.902185+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:48:49.023614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.943169+0000","last_change":"2024-09-19T08:39:38.939531+0000","last_active":"2024-09-19T08:39:38.943169+0000","last_peered":"2024-09-19T08:39:38.943169+0000","last_clean":"2024-09-19T08:39:38.943169+0000","last_became_active":"2024-09-19T08:39:38.938231+0000","last_became_peered":"2024-09-19T08:39:38.938231+0000","last_unstale":"2024-09-19T08:39:38.943169+0000","last_undegraded":"2024-09-19T08:39:38.943169+0000","last_fullsized":"2024-09-19T08:39:38.943169+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:33:44.717869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.901355+0000","last_change":"2024-09-19T08:39:37.930095+0000","last_active":"2024-09-19T08:39:38.901355+0000","last_peered":"2024-09-19T08:39:38.901355+0000","last_clean":"2024-09-19T08:39:38.901355+0000","last_became_active":"2024-09-19T08:39:37.921804+0000","last_became_peered":"2024-09-19T08:39:37.921804+0000","last_unstale":"2024-09-19T08:39:38.901355+0000","last_undegraded":"2024-09-19T08:39:38.901355+0000","last_fullsized":"2024-09-19T08:39:38.901355+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:30:08.860289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.958802+0000","last_change":"2024-09-19T08:39:38.941102+0000","last_active":"2024-09-19T08:39:38.958802+0000","last_peered":"2024-09-19T08:39:38.958802+0000","last_clean":"2024-09-19T08:39:38.958802+0000","last_became_active":"2024-09-19T08:39:38.937500+0000","last_became_peered":"2024-09-19T08:39:38.937500+0000","last_unstale":"2024-09-19T08:39:38.958802+0000","last_undegraded":"2024-09-19T08:39:38.958802+0000","last_fullsized":"2024-09-19T08:39:38.958802+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:29.732835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.911564+0000","last_change":"2024-09-19T08:39:37.912838+0000","last_active":"2024-09-19T08:39:38.911564+0000","last_peered":"2024-09-19T08:39:38.911564+0000","last_clean":"2024-09-19T08:39:38.911564+0000","last_became_active":"2024-09-19T08:39:37.912442+0000","last_became_peered":"2024-09-19T08:39:37.912442+0000","last_unstale":"2024-09-19T08:39:38.911564+0000","last_undegraded":"2024-09-19T08:39:38.911564+0000","last_fullsized":"2024-09-19T08:39:38.911564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:46.135290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-09-19T08:39:37.896926+0000","last_change":"2024-09-19T08:39:37.896926+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.896926+0000","last_undegraded":"2024-09-19T08:39:37.896926+0000","last_fullsized":"2024-09-19T08:39:37.896926+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"42'3","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.961471+0000","last_change":"2024-09-19T08:39:37.936425+0000","last_active":"2024-09-19T08:39:38.961471+0000","last_peered":"2024-09-19T08:39:38.961471+0000","last_clean":"2024-09-19T08:39:38.961471+0000","last_became_active":"2024-09-19T08:39:37.935418+0000","last_became_peered":"2024-09-19T08:39:37.935418+0000","last_unstale":"2024-09-19T08:39:38.961471+0000","last_undegraded":"2024-09-19T08:39:38.961471+0000","last_fullsized":"2024-09-19T08:39:38.961471+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:05.453778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.958959+0000","last_change":"2024-09-19T08:39:38.942176+0000","last_active":"2024-09-19T08:39:38.958959+0000","last_peered":"2024-09-19T08:39:38.958959+0000","last_clean":"2024-09-19T08:39:38.958959+0000","last_became_active":"2024-09-19T08:39:38.939625+0000","last_became_peered":"2024-09-19T08:39:38.939625+0000","last_unstale":"2024-09-19T08:39:38.958959+0000","last_undegraded":"2024-09-19T08:39:38.958959+0000","last_fullsized":"2024-09-19T08:39:38.958959+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:22:21.538914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"42'6","reported_seq":18,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.962495+0000","last_change":"2024-09-19T08:39:37.921227+0000","last_active":"2024-09-19T08:39:38.962495+0000","last_peered":"2024-09-19T08:39:38.962495+0000","last_clean":"2024-09-19T08:39:38.962495+0000","last_became_active":"2024-09-19T08:39:37.920773+0000","last_became_peered":"2024-09-19T08:39:37.920773+0000","last_unstale":"2024-09-19T08:39:38.962495+0000","last_undegraded":"2024-09-19T08:39:38.962495+0000","last_fullsized":"2024-09-19T08:39:38.962495+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:24:07.528992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:37.883250+0000","last_change":"2024-09-19T08:39:37.883250+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.883250+0000","last_undegraded":"2024-09-19T08:39:37.883250+0000","last_fullsized":"2024-09-19T08:39:37.883250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-19T08:39:37.912133+0000","last_change":"2024-09-19T08:39:37.905231+0000","last_active":"2024-09-19T08:39:37.912133+0000","last_peered":"2024-09-19T08:39:37.912133+0000","last_clean":"2024-09-19T08:39:37.912133+0000","last_became_active":"2024-09-19T08:39:37.904660+0000","last_became_peered":"2024-09-19T08:39:37.904660+0000","last_unstale":"2024-09-19T08:39:37.912133+0000","last_undegraded":"2024-09-19T08:39:37.912133+0000","last_fullsized":"2024-09-19T08:39:37.912133+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:38:48.929675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.940617+0000","last_change":"2024-09-19T08:39:38.936213+0000","last_active":"2024-09-19T08:39:38.940617+0000","last_peered":"2024-09-19T08:39:38.940617+0000","last_clean":"2024-09-19T08:39:38.940617+0000","last_became_active":"2024-09-19T08:39:38.935856+0000","last_became_peered":"2024-09-19T08:39:38.935856+0000","last_unstale":"2024-09-19T08:39:38.940617+0000","last_undegraded":"2024-09-19T08:39:38.940617+0000","last_fullsized":"2024-09-19T08:39:38.940617+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:29.744836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.908538+0000","last_change":"2024-09-19T08:39:37.914414+0000","last_active":"2024-09-19T08:39:38.908538+0000","last_peered":"2024-09-19T08:39:38.908538+0000","last_clean":"2024-09-19T08:39:38.908538+0000","last_became_active":"2024-09-19T08:39:37.913576+0000","last_became_peered":"2024-09-19T08:39:37.913576+0000","last_unstale":"2024-09-19T08:39:38.908538+0000","last_undegraded":"2024-09-19T08:39:38.908538+0000","last_fullsized":"2024-09-19T08:39:38.908538+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:28.741298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-09-19T08:39:37.896086+0000","last_change":"2024-09-19T08:39:37.896086+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.896086+0000","last_undegraded":"2024-09-19T08:39:37.896086+0000","last_fullsized":"2024-09-19T08:39:37.896086+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-19T08:39:37.912720+0000","last_change":"2024-09-19T08:39:37.910119+0000","last_active":"2024-09-19T08:39:37.912720+0000","last_peered":"2024-09-19T08:39:37.912720+0000","last_clean":"2024-09-19T08:39:37.912720+0000","last_became_active":"2024-09-19T08:39:37.909646+0000","last_became_peered":"2024-09-19T08:39:37.909646+0000","last_unstale":"2024-09-19T08:39:37.912720+0000","last_undegraded":"2024-09-19T08:39:37.912720+0000","last_fullsized":"2024-09-19T08:39:37.912720+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:30.311453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.959355+0000","last_change":"2024-09-19T08:39:38.937339+0000","last_active":"2024-09-19T08:39:38.959355+0000","last_peered":"2024-09-19T08:39:38.959355+0000","last_clean":"2024-09-19T08:39:38.959355+0000","last_became_active":"2024-09-19T08:39:38.936404+0000","last_became_peered":"2024-09-19T08:39:38.936404+0000","last_unstale":"2024-09-19T08:39:38.959355+0000","last_undegraded":"2024-09-19T08:39:38.959355+0000","last_fullsized":"2024-09-19T08:39:38.959355+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:00:09.742802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"42'8","reported_seq":20,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.957690+0000","last_change":"2024-09-19T08:39:37.918124+0000","last_active":"2024-09-19T08:39:38.957690+0000","last_peered":"2024-09-19T08:39:38.957690+0000","last_clean":"2024-09-19T08:39:38.957690+0000","last_became_active":"2024-09-19T08:39:37.917521+0000","last_became_peered":"2024-09-19T08:39:37.917521+0000","last_unstale":"2024-09-19T08:39:38.957690+0000","last_undegraded":"2024-09-19T08:39:38.957690+0000","last_fullsized":"2024-09-19T08:39:38.957690+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:38:39.741349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.937824+0000","last_change":"2024-09-19T08:39:38.926671+0000","last_active":"2024-09-19T08:39:38.937824+0000","last_peered":"2024-09-19T08:39:38.937824+0000","last_clean":"2024-09-19T08:39:38.937824+0000","last_became_active":"2024-09-19T08:39:38.925343+0000","last_became_peered":"2024-09-19T08:39:38.925343+0000","last_unstale":"2024-09-19T08:39:38.937824+0000","last_undegraded":"2024-09-19T08:39:38.937824+0000","last_fullsized":"2024-09-19T08:39:38.937824+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:41.168755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"42'1","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.949593+0000","last_change":"2024-09-19T08:39:37.911789+0000","last_active":"2024-09-19T08:39:38.949593+0000","last_peered":"2024-09-19T08:39:38.949593+0000","last_clean":"2024-09-19T08:39:38.949593+0000","last_became_active":"2024-09-19T08:39:37.911305+0000","last_became_peered":"2024-09-19T08:39:37.911305+0000","last_unstale":"2024-09-19T08:39:38.949593+0000","last_undegraded":"2024-09-19T08:39:38.949593+0000","last_fullsized":"2024-09-19T08:39:38.949593+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:13.592173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.938438+0000","last_change":"2024-09-19T08:39:38.935012+0000","last_active":"2024-09-19T08:39:38.938438+0000","last_peered":"2024-09-19T08:39:38.938438+0000","last_clean":"2024-09-19T08:39:38.938438+0000","last_became_active":"2024-09-19T08:39:38.934473+0000","last_became_peered":"2024-09-19T08:39:38.934473+0000","last_unstale":"2024-09-19T08:39:38.938438+0000","last_undegraded":"2024-09-19T08:39:38.938438+0000","last_fullsized":"2024-09-19T08:39:38.938438+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:33.862515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.959551+0000","last_change":"2024-09-19T08:39:37.923082+0000","last_active":"2024-09-19T08:39:38.959551+0000","last_peered":"2024-09-19T08:39:38.959551+0000","last_clean":"2024-09-19T08:39:38.959551+0000","last_became_active":"2024-09-19T08:39:37.922381+0000","last_became_peered":"2024-09-19T08:39:37.922381+0000","last_unstale":"2024-09-19T08:39:38.959551+0000","last_undegraded":"2024-09-19T08:39:38.959551+0000","last_fullsized":"2024-09-19T08:39:38.959551+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:08:43.556600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:37.883250+0000","last_change":"2024-09-19T08:39:37.883250+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.883250+0000","last_undegraded":"2024-09-19T08:39:37.883250+0000","last_fullsized":"2024-09-19T08:39:37.883250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"42'1","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.956915+0000","last_change":"2024-09-19T08:39:37.917129+0000","last_active":"2024-09-19T08:39:38.956915+0000","last_peered":"2024-09-19T08:39:38.956915+0000","last_clean":"2024-09-19T08:39:38.956915+0000","last_became_active":"2024-09-19T08:39:37.916360+0000","last_became_peered":"2024-09-19T08:39:37.916360+0000","last_unstale":"2024-09-19T08:39:38.956915+0000","last_undegraded":"2024-09-19T08:39:38.956915+0000","last_fullsized":"2024-09-19T08:39:38.956915+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:39.361088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:37.883250+0000","last_change":"2024-09-19T08:39:37.883250+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.883250+0000","last_undegraded":"2024-09-19T08:39:37.883250+0000","last_fullsized":"2024-09-19T08:39:37.883250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-19T08:39:37.937075+0000","last_change":"2024-09-19T08:39:37.932493+0000","last_active":"2024-09-19T08:39:37.937075+0000","last_peered":"2024-09-19T08:39:37.937075+0000","last_clean":"2024-09-19T08:39:37.937075+0000","last_became_active":"2024-09-19T08:39:37.932195+0000","last_became_peered":"2024-09-19T08:39:37.932195+0000","last_unstale":"2024-09-19T08:39:37.937075+0000","last_undegraded":"2024-09-19T08:39:37.937075+0000","last_fullsized":"2024-09-19T08:39:37.937075+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:32:30.169809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-09-19T08:39:37.896687+0000","last_change":"2024-09-19T08:39:37.896687+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.896687+0000","last_undegraded":"2024-09-19T08:39:37.896687+0000","last_fullsized":"2024-09-19T08:39:37.896687+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-19T08:39:37.919394+0000","last_change":"2024-09-19T08:39:37.913727+0000","last_active":"2024-09-19T08:39:37.919394+0000","last_peered":"2024-09-19T08:39:37.919394+0000","last_clean":"2024-09-19T08:39:37.919394+0000","last_became_active":"2024-09-19T08:39:37.913224+0000","last_became_peered":"2024-09-19T08:39:37.913224+0000","last_unstale":"2024-09-19T08:39:37.919394+0000","last_undegraded":"2024-09-19T08:39:37.919394+0000","last_fullsized":"2024-09-19T08:39:37.919394+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:54:58.881538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.928775+0000","last_change":"2024-09-19T08:39:38.923972+0000","last_active":"2024-09-19T08:39:38.928775+0000","last_peered":"2024-09-19T08:39:38.928775+0000","last_clean":"2024-09-19T08:39:38.928775+0000","last_became_active":"2024-09-19T08:39:38.923101+0000","last_became_peered":"2024-09-19T08:39:38.923101+0000","last_unstale":"2024-09-19T08:39:38.928775+0000","last_undegraded":"2024-09-19T08:39:38.928775+0000","last_fullsized":"2024-09-19T08:39:38.928775+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:08:44.560138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.911722+0000","last_change":"2024-09-19T08:39:37.934259+0000","last_active":"2024-09-19T08:39:38.911722+0000","last_peered":"2024-09-19T08:39:38.911722+0000","last_clean":"2024-09-19T08:39:38.911722+0000","last_became_active":"2024-09-19T08:39:37.933506+0000","last_became_peered":"2024-09-19T08:39:37.933506+0000","last_unstale":"2024-09-19T08:39:38.911722+0000","last_undegraded":"2024-09-19T08:39:38.911722+0000","last_fullsized":"2024-09-19T08:39:38.911722+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:22.044532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.948894+0000","last_change":"2024-09-19T08:39:38.944450+0000","last_active":"2024-09-19T08:39:38.948894+0000","last_peered":"2024-09-19T08:39:38.948894+0000","last_clean":"2024-09-19T08:39:38.948894+0000","last_became_active":"2024-09-19T08:39:38.944108+0000","last_became_peered":"2024-09-19T08:39:38.944108+0000","last_unstale":"2024-09-19T08:39:38.948894+0000","last_undegraded":"2024-09-19T08:39:38.948894+0000","last_fullsized":"2024-09-19T08:39:38.948894+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:56.145227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.901835+0000","last_change":"2024-09-19T08:39:37.922726+0000","last_active":"2024-09-19T08:39:38.901835+0000","last_peered":"2024-09-19T08:39:38.901835+0000","last_clean":"2024-09-19T08:39:38.901835+0000","last_became_active":"2024-09-19T08:39:37.922312+0000","last_became_peered":"2024-09-19T08:39:37.922312+0000","last_unstale":"2024-09-19T08:39:38.901835+0000","last_undegraded":"2024-09-19T08:39:38.901835+0000","last_fullsized":"2024-09-19T08:39:38.901835+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:05:28.346760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.935437+0000","last_change":"2024-09-19T08:39:38.920148+0000","last_active":"2024-09-19T08:39:38.935437+0000","last_peered":"2024-09-19T08:39:38.935437+0000","last_clean":"2024-09-19T08:39:38.935437+0000","last_became_active":"2024-09-19T08:39:38.919337+0000","last_became_peered":"2024-09-19T08:39:38.919337+0000","last_unstale":"2024-09-19T08:39:38.935437+0000","last_undegraded":"2024-09-19T08:39:38.935437+0000","last_fullsized":"2024-09-19T08:39:38.935437+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:27:20.365465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.908966+0000","last_change":"2024-09-19T08:39:37.915040+0000","last_active":"2024-09-19T08:39:38.908966+0000","last_peered":"2024-09-19T08:39:38.908966+0000","last_clean":"2024-09-19T08:39:38.908966+0000","last_became_active":"2024-09-19T08:39:37.914232+0000","last_became_peered":"2024-09-19T08:39:37.914232+0000","last_unstale":"2024-09-19T08:39:38.908966+0000","last_undegraded":"2024-09-19T08:39:38.908966+0000","last_fullsized":"2024-09-19T08:39:38.908966+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:49:13.065050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:39:37.883250+0000","last_change":"2024-09-19T08:39:37.883250+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.883250+0000","last_undegraded":"2024-09-19T08:39:37.883250+0000","last_fullsized":"2024-09-19T08:39:37.883250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-19T08:39:37.922144+0000","last_change":"2024-09-19T08:39:37.918815+0000","last_active":"2024-09-19T08:39:37.922144+0000","last_peered":"2024-09-19T08:39:37.922144+0000","last_clean":"2024-09-19T08:39:37.922144+0000","last_became_active":"2024-09-19T08:39:37.918481+0000","last_became_peered":"2024-09-19T08:39:37.918481+0000","last_unstale":"2024-09-19T08:39:37.922144+0000","last_undegraded":"2024-09-19T08:39:37.922144+0000","last_fullsized":"2024-09-19T08:39:37.922144+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:55:34.226831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.936888+0000","last_change":"2024-09-19T08:39:38.929785+0000","last_active":"2024-09-19T08:39:38.936888+0000","last_peered":"2024-09-19T08:39:38.936888+0000","last_clean":"2024-09-19T08:39:38.936888+0000","last_became_active":"2024-09-19T08:39:38.929074+0000","last_became_peered":"2024-09-19T08:39:38.929074+0000","last_unstale":"2024-09-19T08:39:38.936888+0000","last_undegraded":"2024-09-19T08:39:38.936888+0000","last_fullsized":"2024-09-19T08:39:38.936888+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:37.035646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.918780+0000","last_change":"2024-09-19T08:39:37.936655+0000","last_active":"2024-09-19T08:39:38.918780+0000","last_peered":"2024-09-19T08:39:38.918780+0000","last_clean":"2024-09-19T08:39:38.918780+0000","last_became_active":"2024-09-19T08:39:37.936112+0000","last_became_peered":"2024-09-19T08:39:37.936112+0000","last_unstale":"2024-09-19T08:39:38.918780+0000","last_undegraded":"2024-09-19T08:39:38.918780+0000","last_fullsized":"2024-09-19T08:39:38.918780+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:33:43.714330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.959000+0000","last_change":"2024-09-19T08:39:38.942782+0000","last_active":"2024-09-19T08:39:38.959000+0000","last_peered":"2024-09-19T08:39:38.959000+0000","last_clean":"2024-09-19T08:39:38.959000+0000","last_became_active":"2024-09-19T08:39:38.939689+0000","last_became_peered":"2024-09-19T08:39:38.939689+0000","last_unstale":"2024-09-19T08:39:38.959000+0000","last_undegraded":"2024-09-19T08:39:38.959000+0000","last_fullsized":"2024-09-19T08:39:38.959000+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:37.082611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.895030+0000","last_change":"2024-09-19T08:39:37.924237+0000","last_active":"2024-09-19T08:39:38.895030+0000","last_peered":"2024-09-19T08:39:38.895030+0000","last_clean":"2024-09-19T08:39:38.895030+0000","last_became_active":"2024-09-19T08:39:37.923832+0000","last_became_peered":"2024-09-19T08:39:37.923832+0000","last_unstale":"2024-09-19T08:39:38.895030+0000","last_undegraded":"2024-09-19T08:39:38.895030+0000","last_fullsized":"2024-09-19T08:39:38.895030+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:14:31.270319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-09-19T08:39:37.896039+0000","last_change":"2024-09-19T08:39:37.896039+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.896039+0000","last_undegraded":"2024-09-19T08:39:37.896039+0000","last_fullsized":"2024-09-19T08:39:37.896039+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.919124+0000","last_change":"2024-09-19T08:39:37.934015+0000","last_active":"2024-09-19T08:39:38.919124+0000","last_peered":"2024-09-19T08:39:38.919124+0000","last_clean":"2024-09-19T08:39:38.919124+0000","last_became_active":"2024-09-19T08:39:37.933256+0000","last_became_peered":"2024-09-19T08:39:37.933256+0000","last_unstale":"2024-09-19T08:39:38.919124+0000","last_undegraded":"2024-09-19T08:39:38.919124+0000","last_fullsized":"2024-09-19T08:39:38.919124+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:55:09.923681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":41,"state":"creating+peering","last_fresh":"2024-09-19T08:39:37.896215+0000","last_change":"2024-09-19T08:39:37.896215+0000","last_active":"2024-09-19T08:39:37.883250+0000","last_peered":"2024-09-19T08:39:37.883250+0000","last_clean":"2024-09-19T08:39:37.883250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:39:37.896215+0000","last_undegraded":"2024-09-19T08:39:37.896215+0000","last_fullsized":"2024-09-19T08:39:37.896215+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.901249+0000","last_change":"2024-09-19T08:39:37.920347+0000","last_active":"2024-09-19T08:39:38.901249+0000","last_peered":"2024-09-19T08:39:38.901249+0000","last_clean":"2024-09-19T08:39:38.901249+0000","last_became_active":"2024-09-19T08:39:37.919634+0000","last_became_peered":"2024-09-19T08:39:37.919634+0000","last_unstale":"2024-09-19T08:39:38.901249+0000","last_undegraded":"2024-09-19T08:39:38.901249+0000","last_fullsized":"2024-09-19T08:39:38.901249+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:14:31.270319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.933874+0000","last_change":"2024-09-19T08:39:38.927845+0000","last_active":"2024-09-19T08:39:38.933874+0000","last_peered":"2024-09-19T08:39:38.933874+0000","last_clean":"2024-09-19T08:39:38.933874+0000","last_became_active":"2024-09-19T08:39:38.926989+0000","last_became_peered":"2024-09-19T08:39:38.926989+0000","last_unstale":"2024-09-19T08:39:38.933874+0000","last_undegraded":"2024-09-19T08:39:38.933874+0000","last_fullsized":"2024-09-19T08:39:38.933874+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:55:10.927219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.895000+0000","last_change":"2024-09-19T08:39:37.910329+0000","last_active":"2024-09-19T08:39:38.895000+0000","last_peered":"2024-09-19T08:39:38.895000+0000","last_clean":"2024-09-19T08:39:38.895000+0000","last_became_active":"2024-09-19T08:39:37.909966+0000","last_became_peered":"2024-09-19T08:39:37.909966+0000","last_unstale":"2024-09-19T08:39:38.895000+0000","last_undegraded":"2024-09-19T08:39:38.895000+0000","last_fullsized":"2024-09-19T08:39:38.895000+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:36.032108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.942950+0000","last_change":"2024-09-19T08:39:38.936998+0000","last_active":"2024-09-19T08:39:38.942950+0000","last_peered":"2024-09-19T08:39:38.942950+0000","last_clean":"2024-09-19T08:39:38.942950+0000","last_became_active":"2024-09-19T08:39:38.935884+0000","last_became_peered":"2024-09-19T08:39:38.935884+0000","last_unstale":"2024-09-19T08:39:38.942950+0000","last_undegraded":"2024-09-19T08:39:38.942950+0000","last_fullsized":"2024-09-19T08:39:38.942950+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:40.364626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.903222+0000","last_change":"2024-09-19T08:39:37.907408+0000","last_active":"2024-09-19T08:39:38.903222+0000","last_peered":"2024-09-19T08:39:38.903222+0000","last_clean":"2024-09-19T08:39:38.903222+0000","last_became_active":"2024-09-19T08:39:37.906928+0000","last_became_peered":"2024-09-19T08:39:37.906928+0000","last_unstale":"2024-09-19T08:39:38.903222+0000","last_undegraded":"2024-09-19T08:39:38.903222+0000","last_fullsized":"2024-09-19T08:39:38.903222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:08.285216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.945594+0000","last_change":"2024-09-19T08:39:38.940058+0000","last_active":"2024-09-19T08:39:38.945594+0000","last_peered":"2024-09-19T08:39:38.945594+0000","last_clean":"2024-09-19T08:39:38.945594+0000","last_became_active":"2024-09-19T08:39:38.939663+0000","last_became_peered":"2024-09-19T08:39:38.939663+0000","last_unstale":"2024-09-19T08:39:38.945594+0000","last_undegraded":"2024-09-19T08:39:38.945594+0000","last_fullsized":"2024-09-19T08:39:38.945594+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:40.364626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.957136+0000","last_change":"2024-09-19T08:39:37.922845+0000","last_active":"2024-09-19T08:39:38.957136+0000","last_peered":"2024-09-19T08:39:38.957136+0000","last_clean":"2024-09-19T08:39:38.957136+0000","last_became_active":"2024-09-19T08:39:37.922153+0000","last_became_peered":"2024-09-19T08:39:37.922153+0000","last_unstale":"2024-09-19T08:39:38.957136+0000","last_undegraded":"2024-09-19T08:39:38.957136+0000","last_fullsized":"2024-09-19T08:39:38.957136+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:28.741298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":204,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.908394+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:39:38.908394+0000","last_peered":"2024-09-19T08:39:38.908394+0000","last_clean":"2024-09-19T08:39:38.908394+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:39:38.908394+0000","last_undegraded":"2024-09-19T08:39:38.908394+0000","last_fullsized":"2024-09-19T08:39:38.908394+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":104,"acting":104,"num_store_stats":7},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":1794,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":29,"ondisk_log_size":29,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509461,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1081344,"data_stored":783009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509463,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":185776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509463,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":471040,"data_stored":185776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509460,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":185776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509462,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":185776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509460,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28072,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343768,"statfs":{"total":96636764160,"available":96608018432,"internally_reserved":0,"allocated":1064960,"data_stored":776144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":179509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542165,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31636,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340204,"statfs":{"total":96636764160,"available":96604368896,"internally_reserved":0,"allocated":458752,"data_stored":179509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:39:43.674 INFO:tasks.ceph.ceph_manager.ceph:PG 7.3a is not active+clean 2024-09-19T08:39:43.674 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:39:37.898206+0000', 'last_change': '2024-09-19T08:39:37.898206+0000', 'last_active': '2024-09-19T08:39:37.883250+0000', 'last_peered': '2024-09-19T08:39:37.883250+0000', 'last_clean': '2024-09-19T08:39:37.883250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:37.898206+0000', 'last_undegraded': '2024-09-19T08:39:37.898206+0000', 'last_fullsized': '2024-09-19T08:39:37.898206+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T08:39:43.674 INFO:tasks.ceph.ceph_manager.ceph:PG 7.31 is not active+clean 2024-09-19T08:39:43.674 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:39:37.899012+0000', 'last_change': '2024-09-19T08:39:37.899012+0000', 'last_active': '2024-09-19T08:39:37.883250+0000', 'last_peered': '2024-09-19T08:39:37.883250+0000', 'last_clean': '2024-09-19T08:39:37.883250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:37.899012+0000', 'last_undegraded': '2024-09-19T08:39:37.899012+0000', 'last_fullsized': '2024-09-19T08:39:37.899012+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T08:39:43.674 INFO:tasks.ceph.ceph_manager.ceph:PG 7.30 is not active+clean 2024-09-19T08:39:43.674 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:37.883250+0000', 'last_change': '2024-09-19T08:39:37.883250+0000', 'last_active': '2024-09-19T08:39:37.883250+0000', 'last_peered': '2024-09-19T08:39:37.883250+0000', 'last_clean': '2024-09-19T08:39:37.883250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:37.883250+0000', 'last_undegraded': '2024-09-19T08:39:37.883250+0000', 'last_fullsized': '2024-09-19T08:39:37.883250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:43.675 INFO:tasks.ceph.ceph_manager.ceph:PG 7.2d is not active+clean 2024-09-19T08:39:43.675 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:37.883250+0000', 'last_change': '2024-09-19T08:39:37.883250+0000', 'last_active': '2024-09-19T08:39:37.883250+0000', 'last_peered': '2024-09-19T08:39:37.883250+0000', 'last_clean': '2024-09-19T08:39:37.883250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:37.883250+0000', 'last_undegraded': '2024-09-19T08:39:37.883250+0000', 'last_fullsized': '2024-09-19T08:39:37.883250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:43.675 INFO:tasks.ceph.ceph_manager.ceph:PG 7.2b is not active+clean 2024-09-19T08:39:43.675 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:39:37.898343+0000', 'last_change': '2024-09-19T08:39:37.898343+0000', 'last_active': '2024-09-19T08:39:37.883250+0000', 'last_peered': '2024-09-19T08:39:37.883250+0000', 'last_clean': '2024-09-19T08:39:37.883250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:37.898343+0000', 'last_undegraded': '2024-09-19T08:39:37.898343+0000', 'last_fullsized': '2024-09-19T08:39:37.898343+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T08:39:43.675 INFO:tasks.ceph.ceph_manager.ceph:PG 7.29 is not active+clean 2024-09-19T08:39:43.675 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:37.883250+0000', 'last_change': '2024-09-19T08:39:37.883250+0000', 'last_active': '2024-09-19T08:39:37.883250+0000', 'last_peered': '2024-09-19T08:39:37.883250+0000', 'last_clean': '2024-09-19T08:39:37.883250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:37.883250+0000', 'last_undegraded': '2024-09-19T08:39:37.883250+0000', 'last_fullsized': '2024-09-19T08:39:37.883250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:43.675 INFO:tasks.ceph.ceph_manager.ceph:PG 7.28 is not active+clean 2024-09-19T08:39:43.675 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:37.883250+0000', 'last_change': '2024-09-19T08:39:37.883250+0000', 'last_active': '2024-09-19T08:39:37.883250+0000', 'last_peered': '2024-09-19T08:39:37.883250+0000', 'last_clean': '2024-09-19T08:39:37.883250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:37.883250+0000', 'last_undegraded': '2024-09-19T08:39:37.883250+0000', 'last_fullsized': '2024-09-19T08:39:37.883250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:43.675 INFO:tasks.ceph.ceph_manager.ceph:PG 7.27 is not active+clean 2024-09-19T08:39:43.676 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:37.883250+0000', 'last_change': '2024-09-19T08:39:37.883250+0000', 'last_active': '2024-09-19T08:39:37.883250+0000', 'last_peered': '2024-09-19T08:39:37.883250+0000', 'last_clean': '2024-09-19T08:39:37.883250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:37.883250+0000', 'last_undegraded': '2024-09-19T08:39:37.883250+0000', 'last_fullsized': '2024-09-19T08:39:37.883250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:43.676 INFO:tasks.ceph.ceph_manager.ceph:PG 7.23 is not active+clean 2024-09-19T08:39:43.676 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:37.883250+0000', 'last_change': '2024-09-19T08:39:37.883250+0000', 'last_active': '2024-09-19T08:39:37.883250+0000', 'last_peered': '2024-09-19T08:39:37.883250+0000', 'last_clean': '2024-09-19T08:39:37.883250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:37.883250+0000', 'last_undegraded': '2024-09-19T08:39:37.883250+0000', 'last_fullsized': '2024-09-19T08:39:37.883250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:43.676 INFO:tasks.ceph.ceph_manager.ceph:PG 7.21 is not active+clean 2024-09-19T08:39:43.676 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:39:37.897514+0000', 'last_change': '2024-09-19T08:39:37.897514+0000', 'last_active': '2024-09-19T08:39:37.883250+0000', 'last_peered': '2024-09-19T08:39:37.883250+0000', 'last_clean': '2024-09-19T08:39:37.883250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:37.897514+0000', 'last_undegraded': '2024-09-19T08:39:37.897514+0000', 'last_fullsized': '2024-09-19T08:39:37.897514+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T08:39:43.676 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1f is not active+clean 2024-09-19T08:39:43.676 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:37.883250+0000', 'last_change': '2024-09-19T08:39:37.883250+0000', 'last_active': '2024-09-19T08:39:37.883250+0000', 'last_peered': '2024-09-19T08:39:37.883250+0000', 'last_clean': '2024-09-19T08:39:37.883250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:37.883250+0000', 'last_undegraded': '2024-09-19T08:39:37.883250+0000', 'last_fullsized': '2024-09-19T08:39:37.883250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:43.676 INFO:tasks.ceph.ceph_manager.ceph:PG 7.19 is not active+clean 2024-09-19T08:39:43.676 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:37.883250+0000', 'last_change': '2024-09-19T08:39:37.883250+0000', 'last_active': '2024-09-19T08:39:37.883250+0000', 'last_peered': '2024-09-19T08:39:37.883250+0000', 'last_clean': '2024-09-19T08:39:37.883250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:37.883250+0000', 'last_undegraded': '2024-09-19T08:39:37.883250+0000', 'last_fullsized': '2024-09-19T08:39:37.883250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:43.677 INFO:tasks.ceph.ceph_manager.ceph:PG 7.14 is not active+clean 2024-09-19T08:39:43.677 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:39:37.896926+0000', 'last_change': '2024-09-19T08:39:37.896926+0000', 'last_active': '2024-09-19T08:39:37.883250+0000', 'last_peered': '2024-09-19T08:39:37.883250+0000', 'last_clean': '2024-09-19T08:39:37.883250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:37.896926+0000', 'last_undegraded': '2024-09-19T08:39:37.896926+0000', 'last_fullsized': '2024-09-19T08:39:37.896926+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T08:39:43.677 INFO:tasks.ceph.ceph_manager.ceph:PG 7.12 is not active+clean 2024-09-19T08:39:43.677 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:37.883250+0000', 'last_change': '2024-09-19T08:39:37.883250+0000', 'last_active': '2024-09-19T08:39:37.883250+0000', 'last_peered': '2024-09-19T08:39:37.883250+0000', 'last_clean': '2024-09-19T08:39:37.883250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:37.883250+0000', 'last_undegraded': '2024-09-19T08:39:37.883250+0000', 'last_fullsized': '2024-09-19T08:39:37.883250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:43.677 INFO:tasks.ceph.ceph_manager.ceph:PG 7.10 is not active+clean 2024-09-19T08:39:43.677 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:39:37.896086+0000', 'last_change': '2024-09-19T08:39:37.896086+0000', 'last_active': '2024-09-19T08:39:37.883250+0000', 'last_peered': '2024-09-19T08:39:37.883250+0000', 'last_clean': '2024-09-19T08:39:37.883250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:37.896086+0000', 'last_undegraded': '2024-09-19T08:39:37.896086+0000', 'last_fullsized': '2024-09-19T08:39:37.896086+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T08:39:43.677 INFO:tasks.ceph.ceph_manager.ceph:PG 7.c is not active+clean 2024-09-19T08:39:43.677 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:37.883250+0000', 'last_change': '2024-09-19T08:39:37.883250+0000', 'last_active': '2024-09-19T08:39:37.883250+0000', 'last_peered': '2024-09-19T08:39:37.883250+0000', 'last_clean': '2024-09-19T08:39:37.883250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:37.883250+0000', 'last_undegraded': '2024-09-19T08:39:37.883250+0000', 'last_fullsized': '2024-09-19T08:39:37.883250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:43.678 INFO:tasks.ceph.ceph_manager.ceph:PG 7.d is not active+clean 2024-09-19T08:39:43.678 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:37.883250+0000', 'last_change': '2024-09-19T08:39:37.883250+0000', 'last_active': '2024-09-19T08:39:37.883250+0000', 'last_peered': '2024-09-19T08:39:37.883250+0000', 'last_clean': '2024-09-19T08:39:37.883250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:37.883250+0000', 'last_undegraded': '2024-09-19T08:39:37.883250+0000', 'last_fullsized': '2024-09-19T08:39:37.883250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:43.678 INFO:tasks.ceph.ceph_manager.ceph:PG 7.a is not active+clean 2024-09-19T08:39:43.678 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:39:37.896687+0000', 'last_change': '2024-09-19T08:39:37.896687+0000', 'last_active': '2024-09-19T08:39:37.883250+0000', 'last_peered': '2024-09-19T08:39:37.883250+0000', 'last_clean': '2024-09-19T08:39:37.883250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:37.896687+0000', 'last_undegraded': '2024-09-19T08:39:37.896687+0000', 'last_fullsized': '2024-09-19T08:39:37.896687+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T08:39:43.678 INFO:tasks.ceph.ceph_manager.ceph:PG 7.7 is not active+clean 2024-09-19T08:39:43.678 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:39:37.883250+0000', 'last_change': '2024-09-19T08:39:37.883250+0000', 'last_active': '2024-09-19T08:39:37.883250+0000', 'last_peered': '2024-09-19T08:39:37.883250+0000', 'last_clean': '2024-09-19T08:39:37.883250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:37.883250+0000', 'last_undegraded': '2024-09-19T08:39:37.883250+0000', 'last_fullsized': '2024-09-19T08:39:37.883250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:39:43.678 INFO:tasks.ceph.ceph_manager.ceph:PG 7.2 is not active+clean 2024-09-19T08:39:43.678 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:39:37.896039+0000', 'last_change': '2024-09-19T08:39:37.896039+0000', 'last_active': '2024-09-19T08:39:37.883250+0000', 'last_peered': '2024-09-19T08:39:37.883250+0000', 'last_clean': '2024-09-19T08:39:37.883250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:37.896039+0000', 'last_undegraded': '2024-09-19T08:39:37.896039+0000', 'last_fullsized': '2024-09-19T08:39:37.896039+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T08:39:43.678 INFO:tasks.ceph.ceph_manager.ceph:PG 7.3 is not active+clean 2024-09-19T08:39:43.679 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 41, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:39:37.896215+0000', 'last_change': '2024-09-19T08:39:37.896215+0000', 'last_active': '2024-09-19T08:39:37.883250+0000', 'last_peered': '2024-09-19T08:39:37.883250+0000', 'last_clean': '2024-09-19T08:39:37.883250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:39:37.896215+0000', 'last_undegraded': '2024-09-19T08:39:37.896215+0000', 'last_fullsized': '2024-09-19T08:39:37.896215+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 41, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'last_clean_scrub_stamp': '2024-09-19T08:39:37.883250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T08:39:43.679 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T08:39:43.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.835+0000 7f2bca8c1640 1 Processor -- start 2024-09-19T08:39:43.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.835+0000 7f2bca8c1640 1 -- start start 2024-09-19T08:39:43.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.835+0000 7f2bca8c1640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2bc40d0760 0x7f2bc4072570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:43.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.835+0000 7f2bca8c1640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bc4071100 con 0x7f2bc415b440 2024-09-19T08:39:43.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.835+0000 7f2bca8c1640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bc415b820 con 0x7f2bc40d0310 2024-09-19T08:39:43.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.835+0000 7f2bca8c1640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f2bc40cea30 con 0x7f2bc40d0760 2024-09-19T08:39:43.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.839+0000 7f2bc8e5e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2bc40d0760 0x7f2bc4072570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:43.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.839+0000 7f2bc3fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f2bc415b440 0x7f2bc40ce320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:44722/0 (socket says 172.21.15.98:44722) 2024-09-19T08:39:43.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.839+0000 7f2bc8e5e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2bc40d0760 0x7f2bc4072570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49624/0 (socket says 172.21.15.98:49624) 2024-09-19T08:39:43.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.839+0000 7f2bc3fff640 1 -- 172.21.15.98:0/1854744740 learned_addr learned my addr 172.21.15.98:0/1854744740 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:43.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.839+0000 7f2bc8e5e640 1 -- 172.21.15.98:0/1854744740 >> v1:172.21.15.139:6789/0 conn(0x7f2bc40d0310 legacy=0x7f2bc4071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:43.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.839+0000 7f2bc8e5e640 1 -- 172.21.15.98:0/1854744740 >> v1:172.21.15.98:6789/0 conn(0x7f2bc415b440 legacy=0x7f2bc40ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:43.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.839+0000 7f2bc8e5e640 1 -- 172.21.15.98:0/1854744740 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bc40ced00 con 0x7f2bc40d0760 2024-09-19T08:39:43.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.839+0000 7f2bc2ffd640 1 -- 172.21.15.98:0/1854744740 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3310724683 0 0) 0x7f2bb4001120 con 0x7f2bc415b440 2024-09-19T08:39:43.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.839+0000 7f2bc8e5e640 1 --2- 172.21.15.98:0/1854744740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bc40d0760 0x7f2bc4072570 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc009dc0 tx=0x7f2bbc01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7458b128844caecb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:43.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.839+0000 7f2bc2ffd640 1 -- 172.21.15.98:0/1854744740 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc0b2070 con 0x7f2bc40d0760 2024-09-19T08:39:43.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.839+0000 7f2bc2ffd640 1 -- 172.21.15.98:0/1854744740 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bbc0a59e0 con 0x7f2bc40d0760 2024-09-19T08:39:43.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.839+0000 7f2bc2ffd640 1 -- 172.21.15.98:0/1854744740 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc0a5db0 con 0x7f2bc40d0760 2024-09-19T08:39:43.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.839+0000 7f2bca8c1640 1 -- 172.21.15.98:0/1854744740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bc40d0760 msgr2=0x7f2bc4072570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:43.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.839+0000 7f2bca8c1640 1 --2- 172.21.15.98:0/1854744740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bc40d0760 0x7f2bc4072570 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc009dc0 tx=0x7f2bbc01c920 comp rx=0 tx=0).stop 2024-09-19T08:39:43.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.839+0000 7f2bca8c1640 1 -- 172.21.15.98:0/1854744740 shutdown_connections 2024-09-19T08:39:43.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.839+0000 7f2bca8c1640 1 --2- 172.21.15.98:0/1854744740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bc40d0760 0x7f2bc4072570 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:43.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.839+0000 7f2bca8c1640 1 -- 172.21.15.98:0/1854744740 >> 172.21.15.98:0/1854744740 conn(0x7f2bc406bc90 msgr2=0x7f2bc406c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:43.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.839+0000 7f2bca8c1640 1 -- 172.21.15.98:0/1854744740 shutdown_connections 2024-09-19T08:39:43.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.839+0000 7f2bca8c1640 1 -- 172.21.15.98:0/1854744740 wait complete. 2024-09-19T08:39:43.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.843+0000 7f2bca8c1640 1 Processor -- start 2024-09-19T08:39:43.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.843+0000 7f2bca8c1640 1 -- start start 2024-09-19T08:39:43.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.843+0000 7f2bca8c1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2bc40d0310 0x7f2bc41518b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:43.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.843+0000 7f2bca8c1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bc40d0760 0x7f2bc414c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:43.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.843+0000 7f2bca8c1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2bc415b440 0x7f2bc40ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:43.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.843+0000 7f2bca8c1640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc415fc40 con 0x7f2bc40d0760 2024-09-19T08:39:43.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.843+0000 7f2bca8c1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc40d0bf0 con 0x7f2bc415b440 2024-09-19T08:39:43.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.843+0000 7f2bca8c1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2bc415fdc0 con 0x7f2bc40d0310 2024-09-19T08:39:43.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.843+0000 7f2bc37fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bc40d0760 0x7f2bc414c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:43.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.843+0000 7f2bc37fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bc40d0760 0x7f2bc414c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49628/0 (socket says 172.21.15.98:49628) 2024-09-19T08:39:43.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.843+0000 7f2bc37fe640 1 -- 172.21.15.98:0/1171633990 learned_addr learned my addr 172.21.15.98:0/1171633990 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:43.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.843+0000 7f2bc3fff640 1 --2- 172.21.15.98:0/1171633990 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2bc40d0310 0x7f2bc41518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:43.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.843+0000 7f2bc8e5e640 1 --2- 172.21.15.98:0/1171633990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2bc415b440 0x7f2bc40ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:43.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.843+0000 7f2bc37fe640 1 -- 172.21.15.98:0/1171633990 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2bc40d0310 msgr2=0x7f2bc41518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:43.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.843+0000 7f2bc37fe640 1 --2- 172.21.15.98:0/1171633990 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2bc40d0310 0x7f2bc41518b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:43.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.843+0000 7f2bc37fe640 1 -- 172.21.15.98:0/1171633990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2bc415b440 msgr2=0x7f2bc40ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:43.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.843+0000 7f2bc37fe640 1 --2- 172.21.15.98:0/1171633990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2bc415b440 0x7f2bc40ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:43.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.843+0000 7f2bc37fe640 1 -- 172.21.15.98:0/1171633990 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bc414cce0 con 0x7f2bc40d0760 2024-09-19T08:39:43.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.843+0000 7f2bc37fe640 1 --2- 172.21.15.98:0/1171633990 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bc40d0760 0x7f2bc414c7a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800c500 tx=0x7f2bb800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:43.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.843+0000 7f2bc0ff9640 1 -- 172.21.15.98:0/1171633990 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8016020 con 0x7f2bc40d0760 2024-09-19T08:39:43.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.843+0000 7f2bc0ff9640 1 -- 172.21.15.98:0/1171633990 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bb8005220 con 0x7f2bc40d0760 2024-09-19T08:39:43.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.843+0000 7f2bca8c1640 1 -- 172.21.15.98:0/1171633990 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bc4152e10 con 0x7f2bc40d0760 2024-09-19T08:39:43.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.843+0000 7f2bc0ff9640 1 -- 172.21.15.98:0/1171633990 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8007030 con 0x7f2bc40d0760 2024-09-19T08:39:43.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.843+0000 7f2bca8c1640 1 -- 172.21.15.98:0/1171633990 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bc41524e0 con 0x7f2bc40d0760 2024-09-19T08:39:43.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.843+0000 7f2bca8c1640 1 -- 172.21.15.98:0/1171633990 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bc40ce320 con 0x7f2bc40d0760 2024-09-19T08:39:43.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.847+0000 7f2bc0ff9640 1 -- 172.21.15.98:0/1171633990 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f2bb80071d0 con 0x7f2bc40d0760 2024-09-19T08:39:43.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.847+0000 7f2bc0ff9640 1 --2- 172.21.15.98:0/1171633990 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2bb0067a70 0x7f2bb0069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:43.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.847+0000 7f2bc0ff9640 1 -- 172.21.15.98:0/1171633990 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f2bb8085900 con 0x7f2bc40d0760 2024-09-19T08:39:43.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.847+0000 7f2bc3fff640 1 --2- 172.21.15.98:0/1171633990 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2bb0067a70 0x7f2bb0069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:43.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.847+0000 7f2bc3fff640 1 --2- 172.21.15.98:0/1171633990 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2bb0067a70 0x7f2bb0069f30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4000f70 tx=0x7f2bb4007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:43.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:43.851+0000 7f2bc0ff9640 1 -- 172.21.15.98:0/1171633990 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2bb8020020 con 0x7f2bc40d0760 2024-09-19T08:39:44.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.059+0000 7f2bca8c1640 1 -- 172.21.15.98:0/1171633990 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f2bc4155190 con 0x7f2bc40d0760 2024-09-19T08:39:44.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.059+0000 7f2bc0ff9640 1 -- 172.21.15.98:0/1171633990 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1636 (secure 0 0 0) 0x7f2bb80074c0 con 0x7f2bc40d0760 2024-09-19T08:39:44.060 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:39:44.060 INFO:teuthology.orchestra.run.smithi098.stdout:{"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":108,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":8,"num_up_osds":8,"osd_up_since":1726735088,"num_in_osds":8,"osd_in_since":1726735077,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":76},{"state_name":"unknown","count":12},{"state_name":"creating+peering","count":9}],"num_pgs":97,"num_pools":3,"num_objects":18,"data_bytes":592162,"bytes_used":234897408,"bytes_avail":772859215872,"bytes_total":773094113280,"unknown_pgs_ratio":0.12371134012937546,"inactive_pgs_ratio":0.092783503234386444,"write_bytes_sec":1189,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":23,"btime":"2024-09-19T08:39:41:944534+0000","id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":5244}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-19T08:39:28.047226+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-19T08:39:44.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.063+0000 7f2bca8c1640 1 -- 172.21.15.98:0/1171633990 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2bb0067a70 msgr2=0x7f2bb0069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:44.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.063+0000 7f2bca8c1640 1 --2- 172.21.15.98:0/1171633990 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2bb0067a70 0x7f2bb0069f30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4000f70 tx=0x7f2bb4007d30 comp rx=0 tx=0).stop 2024-09-19T08:39:44.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.063+0000 7f2bca8c1640 1 -- 172.21.15.98:0/1171633990 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bc40d0760 msgr2=0x7f2bc414c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:44.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.063+0000 7f2bca8c1640 1 --2- 172.21.15.98:0/1171633990 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bc40d0760 0x7f2bc414c7a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800c500 tx=0x7f2bb800c9c0 comp rx=0 tx=0).stop 2024-09-19T08:39:44.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.063+0000 7f2bca8c1640 1 -- 172.21.15.98:0/1171633990 shutdown_connections 2024-09-19T08:39:44.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.063+0000 7f2bca8c1640 1 --2- 172.21.15.98:0/1171633990 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2bb0067a70 0x7f2bb0069f30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:44.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.063+0000 7f2bca8c1640 1 --2- 172.21.15.98:0/1171633990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2bc415b440 0x7f2bc40ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:44.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.063+0000 7f2bca8c1640 1 --2- 172.21.15.98:0/1171633990 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bc40d0760 0x7f2bc414c7a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:44.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.063+0000 7f2bca8c1640 1 --2- 172.21.15.98:0/1171633990 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2bc40d0310 0x7f2bc41518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:44.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.063+0000 7f2bca8c1640 1 -- 172.21.15.98:0/1171633990 >> 172.21.15.98:0/1171633990 conn(0x7f2bc406bc90 msgr2=0x7f2bc4067a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:44.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.063+0000 7f2bca8c1640 1 -- 172.21.15.98:0/1171633990 shutdown_connections 2024-09-19T08:39:44.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.063+0000 7f2bca8c1640 1 -- 172.21.15.98:0/1171633990 wait complete. 2024-09-19T08:39:44.078 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-19T08:39:44.079 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:39:44.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.199+0000 7fb25c2df640 1 Processor -- start 2024-09-19T08:39:44.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.199+0000 7fb25c2df640 1 -- start start 2024-09-19T08:39:44.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.199+0000 7fb25c2df640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb25415c550 0x7fb25406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:44.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.199+0000 7fb25c2df640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb25406c1d0 0x7fb2540cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:44.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.199+0000 7fb25c2df640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb25406d310 con 0x7fb2540cf900 2024-09-19T08:39:44.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.199+0000 7fb25c2df640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fb25406c610 con 0x7fb25415c550 2024-09-19T08:39:44.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.199+0000 7fb25c2df640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb25406d5f0 con 0x7fb25406c1d0 2024-09-19T08:39:44.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.199+0000 7fb25a87c640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fb2540cf900 0x7fb2540ce1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:44726/0 (socket says 172.21.15.98:44726) 2024-09-19T08:39:44.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.199+0000 7fb25a87c640 1 -- 172.21.15.98:0/4071281391 learned_addr learned my addr 172.21.15.98:0/4071281391 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:44.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.199+0000 7fb25a07b640 1 --2- 172.21.15.98:0/4071281391 >> v2:172.21.15.98:3300/0 conn(0x7fb25415c550 0x7fb25406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:44.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.199+0000 7fb259079640 1 -- 172.21.15.98:0/4071281391 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 804893663 0 0) 0x7fb25406d310 con 0x7fb2540cf900 2024-09-19T08:39:44.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.199+0000 7fb259079640 1 -- 172.21.15.98:0/4071281391 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb240003530 con 0x7fb2540cf900 2024-09-19T08:39:44.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.199+0000 7fb25987a640 1 --2- 172.21.15.98:0/4071281391 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb25406c1d0 0x7fb2540cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:44.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.199+0000 7fb25a07b640 1 -- 172.21.15.98:0/4071281391 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb25406c1d0 msgr2=0x7fb2540cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:44.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.199+0000 7fb25a07b640 1 --2- 172.21.15.98:0/4071281391 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb25406c1d0 0x7fb2540cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:44.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.199+0000 7fb25a07b640 1 -- 172.21.15.98:0/4071281391 >> v1:172.21.15.98:6789/0 conn(0x7fb2540cf900 legacy=0x7fb2540ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:44.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.199+0000 7fb25a07b640 1 -- 172.21.15.98:0/4071281391 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb25406d8d0 con 0x7fb25415c550 2024-09-19T08:39:44.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.199+0000 7fb25987a640 1 --2- 172.21.15.98:0/4071281391 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb25406c1d0 0x7fb2540cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:44.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.203+0000 7fb25a07b640 1 --2- 172.21.15.98:0/4071281391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb25415c550 0x7fb25406bc90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb2440097c0 tx=0x7fb24401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f09085fdfe5b7e51 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:44.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.203+0000 7fb259079640 1 -- 172.21.15.98:0/4071281391 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2440b0070 con 0x7fb25415c550 2024-09-19T08:39:44.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.203+0000 7fb259079640 1 -- 172.21.15.98:0/4071281391 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2440ab600 con 0x7fb25415c550 2024-09-19T08:39:44.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.203+0000 7fb259079640 1 -- 172.21.15.98:0/4071281391 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2440ab9b0 con 0x7fb25415c550 2024-09-19T08:39:44.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.203+0000 7fb25c2df640 1 -- 172.21.15.98:0/4071281391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb25415c550 msgr2=0x7fb25406bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:44.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.203+0000 7fb25c2df640 1 --2- 172.21.15.98:0/4071281391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb25415c550 0x7fb25406bc90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb2440097c0 tx=0x7fb24401c4a0 comp rx=0 tx=0).stop 2024-09-19T08:39:44.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.203+0000 7fb25c2df640 1 -- 172.21.15.98:0/4071281391 shutdown_connections 2024-09-19T08:39:44.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.203+0000 7fb25c2df640 1 --2- 172.21.15.98:0/4071281391 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb25406c1d0 0x7fb2540cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:44.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.203+0000 7fb25c2df640 1 --2- 172.21.15.98:0/4071281391 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb25415c550 0x7fb25406bc90 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:44.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.203+0000 7fb25c2df640 1 -- 172.21.15.98:0/4071281391 >> 172.21.15.98:0/4071281391 conn(0x7fb2540b2010 msgr2=0x7fb2540b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:44.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.203+0000 7fb25c2df640 1 -- 172.21.15.98:0/4071281391 shutdown_connections 2024-09-19T08:39:44.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.203+0000 7fb25c2df640 1 -- 172.21.15.98:0/4071281391 wait complete. 2024-09-19T08:39:44.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.203+0000 7fb25c2df640 1 Processor -- start 2024-09-19T08:39:44.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.203+0000 7fb25c2df640 1 -- start start 2024-09-19T08:39:44.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.203+0000 7fb25c2df640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb25406c1d0 0x7fb25414b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:44.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.203+0000 7fb25c2df640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2540cf900 0x7fb254159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:44.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.203+0000 7fb25c2df640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb25415c550 0x7fb254151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:44.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.203+0000 7fb25c2df640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb254169230 con 0x7fb25406c1d0 2024-09-19T08:39:44.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.203+0000 7fb25c2df640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb25415cb40 con 0x7fb2540cf900 2024-09-19T08:39:44.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.203+0000 7fb25c2df640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb2541693b0 con 0x7fb25415c550 2024-09-19T08:39:44.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.203+0000 7fb25a07b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb25406c1d0 0x7fb25414b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:44.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.203+0000 7fb25a07b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb25406c1d0 0x7fb25414b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49646/0 (socket says 172.21.15.98:49646) 2024-09-19T08:39:44.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.203+0000 7fb25a07b640 1 -- 172.21.15.98:0/270441616 learned_addr learned my addr 172.21.15.98:0/270441616 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:44.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.203+0000 7fb25a07b640 1 -- 172.21.15.98:0/270441616 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb25415c550 msgr2=0x7fb254151da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:44.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.203+0000 7fb25a07b640 1 --2- 172.21.15.98:0/270441616 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb25415c550 0x7fb254151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:44.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.203+0000 7fb25a07b640 1 -- 172.21.15.98:0/270441616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2540cf900 msgr2=0x7fb254159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:44.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.203+0000 7fb25a07b640 1 --2- 172.21.15.98:0/270441616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2540cf900 0x7fb254159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:44.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.203+0000 7fb25a07b640 1 -- 172.21.15.98:0/270441616 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb25414c500 con 0x7fb25406c1d0 2024-09-19T08:39:44.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.203+0000 7fb25a07b640 1 --2- 172.21.15.98:0/270441616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb25406c1d0 0x7fb25414b240 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb24401a0d0 tx=0x7fb24401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:44.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.203+0000 7fb24affd640 1 -- 172.21.15.98:0/270441616 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2440b0050 con 0x7fb25406c1d0 2024-09-19T08:39:44.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.203+0000 7fb24affd640 1 -- 172.21.15.98:0/270441616 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2440b1040 con 0x7fb25406c1d0 2024-09-19T08:39:44.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.203+0000 7fb25c2df640 1 -- 172.21.15.98:0/270441616 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb25414c7e0 con 0x7fb25406c1d0 2024-09-19T08:39:44.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.203+0000 7fb24affd640 1 -- 172.21.15.98:0/270441616 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2440b5610 con 0x7fb25406c1d0 2024-09-19T08:39:44.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.203+0000 7fb25c2df640 1 -- 172.21.15.98:0/270441616 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2541596b0 con 0x7fb25406c1d0 2024-09-19T08:39:44.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.207+0000 7fb25c2df640 1 -- 172.21.15.98:0/270441616 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2540cf3c0 con 0x7fb25406c1d0 2024-09-19T08:39:44.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.207+0000 7fb24affd640 1 -- 172.21.15.98:0/270441616 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fb2440be030 con 0x7fb25406c1d0 2024-09-19T08:39:44.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.207+0000 7fb24affd640 1 --2- 172.21.15.98:0/270441616 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb238067aa0 0x7fb238069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:44.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.207+0000 7fb24affd640 1 -- 172.21.15.98:0/270441616 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fb24411fde0 con 0x7fb25406c1d0 2024-09-19T08:39:44.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.207+0000 7fb25987a640 1 --2- 172.21.15.98:0/270441616 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb238067aa0 0x7fb238069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:44.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.207+0000 7fb25987a640 1 --2- 172.21.15.98:0/270441616 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb238067aa0 0x7fb238069f60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb2540d0d20 tx=0x7fb250008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:44.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.211+0000 7fb24affd640 1 -- 172.21.15.98:0/270441616 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb2440c5020 con 0x7fb25406c1d0 2024-09-19T08:39:44.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.359+0000 7fb25c2df640 1 -- 172.21.15.98:0/270441616 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb2540ccc90 con 0x7fb238067aa0 2024-09-19T08:39:44.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.363+0000 7fb24affd640 1 -- 172.21.15.98:0/270441616 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251914 (secure 0 0 0) 0x7fb2540ccc90 con 0x7fb238067aa0 2024-09-19T08:39:44.365 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:39:44.367 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:39:44.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.367+0000 7fb25c2df640 1 -- 172.21.15.98:0/270441616 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb238067aa0 msgr2=0x7fb238069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:44.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.367+0000 7fb25c2df640 1 --2- 172.21.15.98:0/270441616 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb238067aa0 0x7fb238069f60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb2540d0d20 tx=0x7fb250008040 comp rx=0 tx=0).stop 2024-09-19T08:39:44.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.367+0000 7fb25c2df640 1 -- 172.21.15.98:0/270441616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb25406c1d0 msgr2=0x7fb25414b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:44.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.367+0000 7fb25c2df640 1 --2- 172.21.15.98:0/270441616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb25406c1d0 0x7fb25414b240 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb24401a0d0 tx=0x7fb24401ace0 comp rx=0 tx=0).stop 2024-09-19T08:39:44.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.367+0000 7fb25c2df640 1 -- 172.21.15.98:0/270441616 shutdown_connections 2024-09-19T08:39:44.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.367+0000 7fb25c2df640 1 --2- 172.21.15.98:0/270441616 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb238067aa0 0x7fb238069f60 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:44.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.367+0000 7fb25c2df640 1 --2- 172.21.15.98:0/270441616 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb25415c550 0x7fb254151da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:44.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.367+0000 7fb25c2df640 1 --2- 172.21.15.98:0/270441616 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2540cf900 0x7fb254159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:44.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.367+0000 7fb25c2df640 1 --2- 172.21.15.98:0/270441616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb25406c1d0 0x7fb25414b240 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:44.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.367+0000 7fb25c2df640 1 -- 172.21.15.98:0/270441616 >> 172.21.15.98:0/270441616 conn(0x7fb2540b2010 msgr2=0x7fb25415a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:44.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.367+0000 7fb25c2df640 1 -- 172.21.15.98:0/270441616 shutdown_connections 2024-09-19T08:39:44.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:44.367+0000 7fb25c2df640 1 -- 172.21.15.98:0/270441616 wait complete. 2024-09-19T08:39:44.383 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2024-09-19T08:39:44.051128+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229484,"kb_used_data":4924,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754745236,"statfs":{"total":773094113280,"available":772859121664,"internally_reserved":0,"allocated":5042176,"data_stored":2701269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":51,"apply_latency_ms":51,"commit_latency_ns":51000000,"apply_latency_ns":51000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.162052"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.958882+0000","last_change":"2024-09-19T08:39:38.938293+0000","last_active":"2024-09-19T08:39:38.958882+0000","last_peered":"2024-09-19T08:39:38.958882+0000","last_clean":"2024-09-19T08:39:38.958882+0000","last_became_active":"2024-09-19T08:39:38.936784+0000","last_became_peered":"2024-09-19T08:39:38.936784+0000","last_unstale":"2024-09-19T08:39:38.958882+0000","last_undegraded":"2024-09-19T08:39:38.958882+0000","last_fullsized":"2024-09-19T08:39:38.958882+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:08:18.820654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.944459+0000","last_change":"2024-09-19T08:39:38.942347+0000","last_active":"2024-09-19T08:39:38.944459+0000","last_peered":"2024-09-19T08:39:38.944459+0000","last_clean":"2024-09-19T08:39:38.944459+0000","last_became_active":"2024-09-19T08:39:38.941641+0000","last_became_peered":"2024-09-19T08:39:38.941641+0000","last_unstale":"2024-09-19T08:39:38.944459+0000","last_undegraded":"2024-09-19T08:39:38.944459+0000","last_fullsized":"2024-09-19T08:39:38.944459+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:29.732835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.933674+0000","last_change":"2024-09-19T08:39:38.918421+0000","last_active":"2024-09-19T08:39:38.933674+0000","last_peered":"2024-09-19T08:39:38.933674+0000","last_clean":"2024-09-19T08:39:38.933674+0000","last_became_active":"2024-09-19T08:39:38.917761+0000","last_became_peered":"2024-09-19T08:39:38.917761+0000","last_unstale":"2024-09-19T08:39:38.933674+0000","last_undegraded":"2024-09-19T08:39:38.933674+0000","last_fullsized":"2024-09-19T08:39:38.933674+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:31.825354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.936913+0000","last_change":"2024-09-19T08:39:38.932965+0000","last_active":"2024-09-19T08:39:38.936913+0000","last_peered":"2024-09-19T08:39:38.936913+0000","last_clean":"2024-09-19T08:39:38.936913+0000","last_became_active":"2024-09-19T08:39:38.929490+0000","last_became_peered":"2024-09-19T08:39:38.929490+0000","last_unstale":"2024-09-19T08:39:38.936913+0000","last_undegraded":"2024-09-19T08:39:38.936913+0000","last_fullsized":"2024-09-19T08:39:38.936913+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:30:09.863827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.942469+0000","last_change":"2024-09-19T08:39:38.938221+0000","last_active":"2024-09-19T08:39:38.942469+0000","last_peered":"2024-09-19T08:39:38.942469+0000","last_clean":"2024-09-19T08:39:38.942469+0000","last_became_active":"2024-09-19T08:39:38.937356+0000","last_became_peered":"2024-09-19T08:39:38.937356+0000","last_unstale":"2024-09-19T08:39:38.942469+0000","last_undegraded":"2024-09-19T08:39:38.942469+0000","last_fullsized":"2024-09-19T08:39:38.942469+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:00:09.742802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.957225+0000","last_change":"2024-09-19T08:39:38.937688+0000","last_active":"2024-09-19T08:39:39.957225+0000","last_peered":"2024-09-19T08:39:39.957225+0000","last_clean":"2024-09-19T08:39:39.957225+0000","last_became_active":"2024-09-19T08:39:38.937458+0000","last_became_peered":"2024-09-19T08:39:38.937458+0000","last_unstale":"2024-09-19T08:39:39.957225+0000","last_undegraded":"2024-09-19T08:39:39.957225+0000","last_fullsized":"2024-09-19T08:39:39.957225+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:12:54.866527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.961008+0000","last_change":"2024-09-19T08:39:38.945329+0000","last_active":"2024-09-19T08:39:38.961008+0000","last_peered":"2024-09-19T08:39:38.961008+0000","last_clean":"2024-09-19T08:39:38.961008+0000","last_became_active":"2024-09-19T08:39:38.944121+0000","last_became_peered":"2024-09-19T08:39:38.944121+0000","last_unstale":"2024-09-19T08:39:38.961008+0000","last_undegraded":"2024-09-19T08:39:38.961008+0000","last_fullsized":"2024-09-19T08:39:38.961008+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:37:14.987023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.937026+0000","last_change":"2024-09-19T08:39:38.929792+0000","last_active":"2024-09-19T08:39:38.937026+0000","last_peered":"2024-09-19T08:39:38.937026+0000","last_clean":"2024-09-19T08:39:38.937026+0000","last_became_active":"2024-09-19T08:39:38.928743+0000","last_became_peered":"2024-09-19T08:39:38.928743+0000","last_unstale":"2024-09-19T08:39:38.937026+0000","last_undegraded":"2024-09-19T08:39:38.937026+0000","last_fullsized":"2024-09-19T08:39:38.937026+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:14:32.273857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.937132+0000","last_change":"2024-09-19T08:39:38.934773+0000","last_active":"2024-09-19T08:39:38.937132+0000","last_peered":"2024-09-19T08:39:38.937132+0000","last_clean":"2024-09-19T08:39:38.937132+0000","last_became_active":"2024-09-19T08:39:38.934054+0000","last_became_peered":"2024-09-19T08:39:38.934054+0000","last_unstale":"2024-09-19T08:39:38.937132+0000","last_undegraded":"2024-09-19T08:39:38.937132+0000","last_fullsized":"2024-09-19T08:39:38.937132+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:31.825354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.937106+0000","last_change":"2024-09-19T08:39:38.934619+0000","last_active":"2024-09-19T08:39:38.937106+0000","last_peered":"2024-09-19T08:39:38.937106+0000","last_clean":"2024-09-19T08:39:38.937106+0000","last_became_active":"2024-09-19T08:39:38.934244+0000","last_became_peered":"2024-09-19T08:39:38.934244+0000","last_unstale":"2024-09-19T08:39:38.937106+0000","last_undegraded":"2024-09-19T08:39:38.937106+0000","last_fullsized":"2024-09-19T08:39:38.937106+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:08:44.560138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.937073+0000","last_change":"2024-09-19T08:39:38.933595+0000","last_active":"2024-09-19T08:39:38.937073+0000","last_peered":"2024-09-19T08:39:38.937073+0000","last_clean":"2024-09-19T08:39:38.937073+0000","last_became_active":"2024-09-19T08:39:38.933316+0000","last_became_peered":"2024-09-19T08:39:38.933316+0000","last_unstale":"2024-09-19T08:39:38.937073+0000","last_undegraded":"2024-09-19T08:39:38.937073+0000","last_fullsized":"2024-09-19T08:39:38.937073+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:43.876237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.928186+0000","last_change":"2024-09-19T08:39:38.923904+0000","last_active":"2024-09-19T08:39:38.928186+0000","last_peered":"2024-09-19T08:39:38.928186+0000","last_clean":"2024-09-19T08:39:38.928186+0000","last_became_active":"2024-09-19T08:39:38.923135+0000","last_became_peered":"2024-09-19T08:39:38.923135+0000","last_unstale":"2024-09-19T08:39:38.928186+0000","last_undegraded":"2024-09-19T08:39:38.928186+0000","last_fullsized":"2024-09-19T08:39:38.928186+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:06.457317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.938802+0000","last_change":"2024-09-19T08:39:38.933031+0000","last_active":"2024-09-19T08:39:38.938802+0000","last_peered":"2024-09-19T08:39:38.938802+0000","last_clean":"2024-09-19T08:39:38.938802+0000","last_became_active":"2024-09-19T08:39:38.932568+0000","last_became_peered":"2024-09-19T08:39:38.932568+0000","last_unstale":"2024-09-19T08:39:38.938802+0000","last_undegraded":"2024-09-19T08:39:38.938802+0000","last_fullsized":"2024-09-19T08:39:38.938802+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:41:56.688886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.940685+0000","last_change":"2024-09-19T08:39:38.936306+0000","last_active":"2024-09-19T08:39:38.940685+0000","last_peered":"2024-09-19T08:39:38.940685+0000","last_clean":"2024-09-19T08:39:38.940685+0000","last_became_active":"2024-09-19T08:39:38.935988+0000","last_became_peered":"2024-09-19T08:39:38.935988+0000","last_unstale":"2024-09-19T08:39:38.940685+0000","last_undegraded":"2024-09-19T08:39:38.940685+0000","last_fullsized":"2024-09-19T08:39:38.940685+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:49:14.068588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.957339+0000","last_change":"2024-09-19T08:39:38.935062+0000","last_active":"2024-09-19T08:39:39.957339+0000","last_peered":"2024-09-19T08:39:39.957339+0000","last_clean":"2024-09-19T08:39:39.957339+0000","last_became_active":"2024-09-19T08:39:38.933932+0000","last_became_peered":"2024-09-19T08:39:38.933932+0000","last_unstale":"2024-09-19T08:39:39.957339+0000","last_undegraded":"2024-09-19T08:39:39.957339+0000","last_fullsized":"2024-09-19T08:39:39.957339+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:18:18.796878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.275549+0000","last_change":"2024-09-19T08:39:38.940029+0000","last_active":"2024-09-19T08:39:40.275549+0000","last_peered":"2024-09-19T08:39:40.275549+0000","last_clean":"2024-09-19T08:39:40.275549+0000","last_became_active":"2024-09-19T08:39:38.939350+0000","last_became_peered":"2024-09-19T08:39:38.939350+0000","last_unstale":"2024-09-19T08:39:40.275549+0000","last_undegraded":"2024-09-19T08:39:40.275549+0000","last_fullsized":"2024-09-19T08:39:40.275549+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:01:10.462123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.933907+0000","last_change":"2024-09-19T08:39:38.924533+0000","last_active":"2024-09-19T08:39:38.933907+0000","last_peered":"2024-09-19T08:39:38.933907+0000","last_clean":"2024-09-19T08:39:38.933907+0000","last_became_active":"2024-09-19T08:39:38.923817+0000","last_became_peered":"2024-09-19T08:39:38.923817+0000","last_unstale":"2024-09-19T08:39:38.933907+0000","last_undegraded":"2024-09-19T08:39:38.933907+0000","last_fullsized":"2024-09-19T08:39:38.933907+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:29.744836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.943119+0000","last_change":"2024-09-19T08:39:38.938932+0000","last_active":"2024-09-19T08:39:38.943119+0000","last_peered":"2024-09-19T08:39:38.943119+0000","last_clean":"2024-09-19T08:39:38.943119+0000","last_became_active":"2024-09-19T08:39:38.937330+0000","last_became_peered":"2024-09-19T08:39:38.937330+0000","last_unstale":"2024-09-19T08:39:38.943119+0000","last_undegraded":"2024-09-19T08:39:38.943119+0000","last_fullsized":"2024-09-19T08:39:38.943119+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:06.457317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.276486+0000","last_change":"2024-09-19T08:39:38.939262+0000","last_active":"2024-09-19T08:39:40.276486+0000","last_peered":"2024-09-19T08:39:40.276486+0000","last_clean":"2024-09-19T08:39:40.276486+0000","last_became_active":"2024-09-19T08:39:38.938546+0000","last_became_peered":"2024-09-19T08:39:38.938546+0000","last_unstale":"2024-09-19T08:39:40.276486+0000","last_undegraded":"2024-09-19T08:39:40.276486+0000","last_fullsized":"2024-09-19T08:39:40.276486+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:38:49.933214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.938851+0000","last_change":"2024-09-19T08:39:38.936241+0000","last_active":"2024-09-19T08:39:38.938851+0000","last_peered":"2024-09-19T08:39:38.938851+0000","last_clean":"2024-09-19T08:39:38.938851+0000","last_became_active":"2024-09-19T08:39:38.935936+0000","last_became_peered":"2024-09-19T08:39:38.935936+0000","last_unstale":"2024-09-19T08:39:38.938851+0000","last_undegraded":"2024-09-19T08:39:38.938851+0000","last_fullsized":"2024-09-19T08:39:38.938851+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:49.679363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.957241+0000","last_change":"2024-09-19T08:39:38.931744+0000","last_active":"2024-09-19T08:39:39.957241+0000","last_peered":"2024-09-19T08:39:39.957241+0000","last_clean":"2024-09-19T08:39:39.957241+0000","last_became_active":"2024-09-19T08:39:38.931246+0000","last_became_peered":"2024-09-19T08:39:38.931246+0000","last_unstale":"2024-09-19T08:39:39.957241+0000","last_undegraded":"2024-09-19T08:39:39.957241+0000","last_fullsized":"2024-09-19T08:39:39.957241+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:01.279485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.938460+0000","last_change":"2024-09-19T08:39:38.935680+0000","last_active":"2024-09-19T08:39:38.938460+0000","last_peered":"2024-09-19T08:39:38.938460+0000","last_clean":"2024-09-19T08:39:38.938460+0000","last_became_active":"2024-09-19T08:39:38.929257+0000","last_became_peered":"2024-09-19T08:39:38.929257+0000","last_unstale":"2024-09-19T08:39:38.938460+0000","last_undegraded":"2024-09-19T08:39:38.938460+0000","last_fullsized":"2024-09-19T08:39:38.938460+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:23.048070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.276434+0000","last_change":"2024-09-19T08:39:38.947346+0000","last_active":"2024-09-19T08:39:40.276434+0000","last_peered":"2024-09-19T08:39:40.276434+0000","last_clean":"2024-09-19T08:39:40.276434+0000","last_became_active":"2024-09-19T08:39:38.946660+0000","last_became_peered":"2024-09-19T08:39:38.946660+0000","last_unstale":"2024-09-19T08:39:40.276434+0000","last_undegraded":"2024-09-19T08:39:40.276434+0000","last_fullsized":"2024-09-19T08:39:40.276434+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:19.465116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.275549+0000","last_change":"2024-09-19T08:39:38.947521+0000","last_active":"2024-09-19T08:39:40.275549+0000","last_peered":"2024-09-19T08:39:40.275549+0000","last_clean":"2024-09-19T08:39:40.275549+0000","last_became_active":"2024-09-19T08:39:38.946692+0000","last_became_peered":"2024-09-19T08:39:38.946692+0000","last_unstale":"2024-09-19T08:39:40.275549+0000","last_undegraded":"2024-09-19T08:39:40.275549+0000","last_fullsized":"2024-09-19T08:39:40.275549+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:17:10.144057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.276728+0000","last_change":"2024-09-19T08:39:38.932625+0000","last_active":"2024-09-19T08:39:40.276728+0000","last_peered":"2024-09-19T08:39:40.276728+0000","last_clean":"2024-09-19T08:39:40.276728+0000","last_became_active":"2024-09-19T08:39:38.931787+0000","last_became_peered":"2024-09-19T08:39:38.931787+0000","last_unstale":"2024-09-19T08:39:40.276728+0000","last_undegraded":"2024-09-19T08:39:40.276728+0000","last_fullsized":"2024-09-19T08:39:40.276728+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:31.314991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.938757+0000","last_change":"2024-09-19T08:39:38.932467+0000","last_active":"2024-09-19T08:39:38.938757+0000","last_peered":"2024-09-19T08:39:38.938757+0000","last_clean":"2024-09-19T08:39:38.938757+0000","last_became_active":"2024-09-19T08:39:38.932023+0000","last_became_peered":"2024-09-19T08:39:38.932023+0000","last_unstale":"2024-09-19T08:39:38.938757+0000","last_undegraded":"2024-09-19T08:39:38.938757+0000","last_fullsized":"2024-09-19T08:39:38.938757+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:11.986036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.933854+0000","last_change":"2024-09-19T08:39:38.924547+0000","last_active":"2024-09-19T08:39:38.933854+0000","last_peered":"2024-09-19T08:39:38.933854+0000","last_clean":"2024-09-19T08:39:38.933854+0000","last_became_active":"2024-09-19T08:39:38.923802+0000","last_became_peered":"2024-09-19T08:39:38.923802+0000","last_unstale":"2024-09-19T08:39:38.933854+0000","last_undegraded":"2024-09-19T08:39:38.933854+0000","last_fullsized":"2024-09-19T08:39:38.933854+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:23.048070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.933859+0000","last_change":"2024-09-19T08:39:38.927516+0000","last_active":"2024-09-19T08:39:38.933859+0000","last_peered":"2024-09-19T08:39:38.933859+0000","last_clean":"2024-09-19T08:39:38.933859+0000","last_became_active":"2024-09-19T08:39:38.926015+0000","last_became_peered":"2024-09-19T08:39:38.926015+0000","last_unstale":"2024-09-19T08:39:38.933859+0000","last_undegraded":"2024-09-19T08:39:38.933859+0000","last_fullsized":"2024-09-19T08:39:38.933859+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:33:44.717869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.276400+0000","last_change":"2024-09-19T08:39:38.924180+0000","last_active":"2024-09-19T08:39:40.276400+0000","last_peered":"2024-09-19T08:39:40.276400+0000","last_clean":"2024-09-19T08:39:40.276400+0000","last_became_active":"2024-09-19T08:39:38.923352+0000","last_became_peered":"2024-09-19T08:39:38.923352+0000","last_unstale":"2024-09-19T08:39:40.276400+0000","last_undegraded":"2024-09-19T08:39:40.276400+0000","last_fullsized":"2024-09-19T08:39:40.276400+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:11.722134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.934797+0000","last_change":"2024-09-19T08:39:38.931320+0000","last_active":"2024-09-19T08:39:38.934797+0000","last_peered":"2024-09-19T08:39:38.934797+0000","last_clean":"2024-09-19T08:39:38.934797+0000","last_became_active":"2024-09-19T08:39:38.930578+0000","last_became_peered":"2024-09-19T08:39:38.930578+0000","last_unstale":"2024-09-19T08:39:38.934797+0000","last_undegraded":"2024-09-19T08:39:38.934797+0000","last_fullsized":"2024-09-19T08:39:38.934797+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:49:14.068588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.958078+0000","last_change":"2024-09-19T08:39:38.933604+0000","last_active":"2024-09-19T08:39:39.958078+0000","last_peered":"2024-09-19T08:39:39.958078+0000","last_clean":"2024-09-19T08:39:39.958078+0000","last_became_active":"2024-09-19T08:39:38.931912+0000","last_became_peered":"2024-09-19T08:39:38.931912+0000","last_unstale":"2024-09-19T08:39:39.958078+0000","last_undegraded":"2024-09-19T08:39:39.958078+0000","last_fullsized":"2024-09-19T08:39:39.958078+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:31:29.880113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.930583+0000","last_change":"2024-09-19T08:39:38.921905+0000","last_active":"2024-09-19T08:39:38.930583+0000","last_peered":"2024-09-19T08:39:38.930583+0000","last_clean":"2024-09-19T08:39:38.930583+0000","last_became_active":"2024-09-19T08:39:38.920862+0000","last_became_peered":"2024-09-19T08:39:38.920862+0000","last_unstale":"2024-09-19T08:39:38.930583+0000","last_undegraded":"2024-09-19T08:39:38.930583+0000","last_fullsized":"2024-09-19T08:39:38.930583+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:43:08.445843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.950250+0000","last_change":"2024-09-19T08:39:38.942801+0000","last_active":"2024-09-19T08:39:38.950250+0000","last_peered":"2024-09-19T08:39:38.950250+0000","last_clean":"2024-09-19T08:39:38.950250+0000","last_became_active":"2024-09-19T08:39:38.942568+0000","last_became_peered":"2024-09-19T08:39:38.942568+0000","last_unstale":"2024-09-19T08:39:38.950250+0000","last_undegraded":"2024-09-19T08:39:38.950250+0000","last_fullsized":"2024-09-19T08:39:38.950250+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:27:31.825354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"42'4","reported_seq":18,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.957674+0000","last_change":"2024-09-19T08:39:37.937397+0000","last_active":"2024-09-19T08:39:39.957674+0000","last_peered":"2024-09-19T08:39:39.957674+0000","last_clean":"2024-09-19T08:39:39.957674+0000","last_became_active":"2024-09-19T08:39:37.937128+0000","last_became_peered":"2024-09-19T08:39:37.937128+0000","last_unstale":"2024-09-19T08:39:39.957674+0000","last_undegraded":"2024-09-19T08:39:39.957674+0000","last_fullsized":"2024-09-19T08:39:39.957674+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:19.834362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.276453+0000","last_change":"2024-09-19T08:39:38.929983+0000","last_active":"2024-09-19T08:39:40.276453+0000","last_peered":"2024-09-19T08:39:40.276453+0000","last_clean":"2024-09-19T08:39:40.276453+0000","last_became_active":"2024-09-19T08:39:38.929254+0000","last_became_peered":"2024-09-19T08:39:38.929254+0000","last_unstale":"2024-09-19T08:39:40.276453+0000","last_undegraded":"2024-09-19T08:39:40.276453+0000","last_fullsized":"2024-09-19T08:39:40.276453+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:21.872035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"42'1","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.959478+0000","last_change":"2024-09-19T08:39:37.932635+0000","last_active":"2024-09-19T08:39:38.959478+0000","last_peered":"2024-09-19T08:39:38.959478+0000","last_clean":"2024-09-19T08:39:38.959478+0000","last_became_active":"2024-09-19T08:39:37.931841+0000","last_became_peered":"2024-09-19T08:39:37.931841+0000","last_unstale":"2024-09-19T08:39:38.959478+0000","last_undegraded":"2024-09-19T08:39:38.959478+0000","last_fullsized":"2024-09-19T08:39:38.959478+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:42.872698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.943117+0000","last_change":"2024-09-19T08:39:38.940685+0000","last_active":"2024-09-19T08:39:38.943117+0000","last_peered":"2024-09-19T08:39:38.943117+0000","last_clean":"2024-09-19T08:39:38.943117+0000","last_became_active":"2024-09-19T08:39:38.935293+0000","last_became_peered":"2024-09-19T08:39:38.935293+0000","last_unstale":"2024-09-19T08:39:38.943117+0000","last_undegraded":"2024-09-19T08:39:38.943117+0000","last_fullsized":"2024-09-19T08:39:38.943117+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:08:18.820654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.956642+0000","last_change":"2024-09-19T08:39:37.917790+0000","last_active":"2024-09-19T08:39:38.956642+0000","last_peered":"2024-09-19T08:39:38.956642+0000","last_clean":"2024-09-19T08:39:38.956642+0000","last_became_active":"2024-09-19T08:39:37.917479+0000","last_became_peered":"2024-09-19T08:39:37.917479+0000","last_unstale":"2024-09-19T08:39:38.956642+0000","last_undegraded":"2024-09-19T08:39:38.956642+0000","last_fullsized":"2024-09-19T08:39:38.956642+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:30:08.860289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.927097+0000","last_change":"2024-09-19T08:39:38.920238+0000","last_active":"2024-09-19T08:39:38.927097+0000","last_peered":"2024-09-19T08:39:38.927097+0000","last_clean":"2024-09-19T08:39:38.927097+0000","last_became_active":"2024-09-19T08:39:38.919486+0000","last_became_peered":"2024-09-19T08:39:38.919486+0000","last_unstale":"2024-09-19T08:39:38.927097+0000","last_undegraded":"2024-09-19T08:39:38.927097+0000","last_fullsized":"2024-09-19T08:39:38.927097+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:43.876237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.961365+0000","last_change":"2024-09-19T08:39:37.927771+0000","last_active":"2024-09-19T08:39:38.961365+0000","last_peered":"2024-09-19T08:39:38.961365+0000","last_clean":"2024-09-19T08:39:38.961365+0000","last_became_active":"2024-09-19T08:39:37.926946+0000","last_became_peered":"2024-09-19T08:39:37.926946+0000","last_unstale":"2024-09-19T08:39:38.961365+0000","last_undegraded":"2024-09-19T08:39:38.961365+0000","last_fullsized":"2024-09-19T08:39:38.961365+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:39.361088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.933755+0000","last_change":"2024-09-19T08:39:38.918556+0000","last_active":"2024-09-19T08:39:38.933755+0000","last_peered":"2024-09-19T08:39:38.933755+0000","last_clean":"2024-09-19T08:39:38.933755+0000","last_became_active":"2024-09-19T08:39:38.917830+0000","last_became_peered":"2024-09-19T08:39:38.917830+0000","last_unstale":"2024-09-19T08:39:38.933755+0000","last_undegraded":"2024-09-19T08:39:38.933755+0000","last_fullsized":"2024-09-19T08:39:38.933755+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:33.862515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.957345+0000","last_change":"2024-09-19T08:39:37.913524+0000","last_active":"2024-09-19T08:39:39.957345+0000","last_peered":"2024-09-19T08:39:39.957345+0000","last_clean":"2024-09-19T08:39:39.957345+0000","last_became_active":"2024-09-19T08:39:37.912853+0000","last_became_peered":"2024-09-19T08:39:37.912853+0000","last_unstale":"2024-09-19T08:39:39.957345+0000","last_undegraded":"2024-09-19T08:39:39.957345+0000","last_fullsized":"2024-09-19T08:39:39.957345+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:32.036009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.936990+0000","last_change":"2024-09-19T08:39:38.929719+0000","last_active":"2024-09-19T08:39:38.936990+0000","last_peered":"2024-09-19T08:39:38.936990+0000","last_clean":"2024-09-19T08:39:38.936990+0000","last_became_active":"2024-09-19T08:39:38.929158+0000","last_became_peered":"2024-09-19T08:39:38.929158+0000","last_unstale":"2024-09-19T08:39:38.936990+0000","last_undegraded":"2024-09-19T08:39:38.936990+0000","last_fullsized":"2024-09-19T08:39:38.936990+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:07:16.957999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"42'1","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.958601+0000","last_change":"2024-09-19T08:39:37.929415+0000","last_active":"2024-09-19T08:39:38.958601+0000","last_peered":"2024-09-19T08:39:38.958601+0000","last_clean":"2024-09-19T08:39:38.958601+0000","last_became_active":"2024-09-19T08:39:37.928610+0000","last_became_peered":"2024-09-19T08:39:37.928610+0000","last_unstale":"2024-09-19T08:39:38.958601+0000","last_undegraded":"2024-09-19T08:39:38.958601+0000","last_fullsized":"2024-09-19T08:39:38.958601+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:36.032108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.936735+0000","last_change":"2024-09-19T08:39:38.928185+0000","last_active":"2024-09-19T08:39:38.936735+0000","last_peered":"2024-09-19T08:39:38.936735+0000","last_clean":"2024-09-19T08:39:38.936735+0000","last_became_active":"2024-09-19T08:39:38.924812+0000","last_became_peered":"2024-09-19T08:39:38.924812+0000","last_unstale":"2024-09-19T08:39:38.936735+0000","last_undegraded":"2024-09-19T08:39:38.936735+0000","last_fullsized":"2024-09-19T08:39:38.936735+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:08.958729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.912110+0000","last_change":"2024-09-19T08:39:37.921882+0000","last_active":"2024-09-19T08:39:38.912110+0000","last_peered":"2024-09-19T08:39:38.912110+0000","last_clean":"2024-09-19T08:39:38.912110+0000","last_became_active":"2024-09-19T08:39:37.921061+0000","last_became_peered":"2024-09-19T08:39:37.921061+0000","last_unstale":"2024-09-19T08:39:38.912110+0000","last_undegraded":"2024-09-19T08:39:38.912110+0000","last_fullsized":"2024-09-19T08:39:38.912110+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:49:13.065050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.275438+0000","last_change":"2024-09-19T08:39:38.948118+0000","last_active":"2024-09-19T08:39:40.275438+0000","last_peered":"2024-09-19T08:39:40.275438+0000","last_clean":"2024-09-19T08:39:40.275438+0000","last_became_active":"2024-09-19T08:39:38.947360+0000","last_became_peered":"2024-09-19T08:39:38.947360+0000","last_unstale":"2024-09-19T08:39:40.275438+0000","last_undegraded":"2024-09-19T08:39:40.275438+0000","last_fullsized":"2024-09-19T08:39:40.275438+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:42.793245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.902185+0000","last_change":"2024-09-19T08:39:37.914508+0000","last_active":"2024-09-19T08:39:38.902185+0000","last_peered":"2024-09-19T08:39:38.902185+0000","last_clean":"2024-09-19T08:39:38.902185+0000","last_became_active":"2024-09-19T08:39:37.914186+0000","last_became_peered":"2024-09-19T08:39:37.914186+0000","last_unstale":"2024-09-19T08:39:38.902185+0000","last_undegraded":"2024-09-19T08:39:38.902185+0000","last_fullsized":"2024-09-19T08:39:38.902185+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:48:49.023614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.943169+0000","last_change":"2024-09-19T08:39:38.939531+0000","last_active":"2024-09-19T08:39:38.943169+0000","last_peered":"2024-09-19T08:39:38.943169+0000","last_clean":"2024-09-19T08:39:38.943169+0000","last_became_active":"2024-09-19T08:39:38.938231+0000","last_became_peered":"2024-09-19T08:39:38.938231+0000","last_unstale":"2024-09-19T08:39:38.943169+0000","last_undegraded":"2024-09-19T08:39:38.943169+0000","last_fullsized":"2024-09-19T08:39:38.943169+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:33:44.717869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.901355+0000","last_change":"2024-09-19T08:39:37.930095+0000","last_active":"2024-09-19T08:39:38.901355+0000","last_peered":"2024-09-19T08:39:38.901355+0000","last_clean":"2024-09-19T08:39:38.901355+0000","last_became_active":"2024-09-19T08:39:37.921804+0000","last_became_peered":"2024-09-19T08:39:37.921804+0000","last_unstale":"2024-09-19T08:39:38.901355+0000","last_undegraded":"2024-09-19T08:39:38.901355+0000","last_fullsized":"2024-09-19T08:39:38.901355+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:30:08.860289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.958802+0000","last_change":"2024-09-19T08:39:38.941102+0000","last_active":"2024-09-19T08:39:38.958802+0000","last_peered":"2024-09-19T08:39:38.958802+0000","last_clean":"2024-09-19T08:39:38.958802+0000","last_became_active":"2024-09-19T08:39:38.937500+0000","last_became_peered":"2024-09-19T08:39:38.937500+0000","last_unstale":"2024-09-19T08:39:38.958802+0000","last_undegraded":"2024-09-19T08:39:38.958802+0000","last_fullsized":"2024-09-19T08:39:38.958802+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:29.732835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.911564+0000","last_change":"2024-09-19T08:39:37.912838+0000","last_active":"2024-09-19T08:39:38.911564+0000","last_peered":"2024-09-19T08:39:38.911564+0000","last_clean":"2024-09-19T08:39:38.911564+0000","last_became_active":"2024-09-19T08:39:37.912442+0000","last_became_peered":"2024-09-19T08:39:37.912442+0000","last_unstale":"2024-09-19T08:39:38.911564+0000","last_undegraded":"2024-09-19T08:39:38.911564+0000","last_fullsized":"2024-09-19T08:39:38.911564+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:46.135290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.957630+0000","last_change":"2024-09-19T08:39:38.931347+0000","last_active":"2024-09-19T08:39:39.957630+0000","last_peered":"2024-09-19T08:39:39.957630+0000","last_clean":"2024-09-19T08:39:39.957630+0000","last_became_active":"2024-09-19T08:39:38.930982+0000","last_became_peered":"2024-09-19T08:39:38.930982+0000","last_unstale":"2024-09-19T08:39:39.957630+0000","last_undegraded":"2024-09-19T08:39:39.957630+0000","last_fullsized":"2024-09-19T08:39:39.957630+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:36.448393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"42'3","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.961471+0000","last_change":"2024-09-19T08:39:37.936425+0000","last_active":"2024-09-19T08:39:38.961471+0000","last_peered":"2024-09-19T08:39:38.961471+0000","last_clean":"2024-09-19T08:39:38.961471+0000","last_became_active":"2024-09-19T08:39:37.935418+0000","last_became_peered":"2024-09-19T08:39:37.935418+0000","last_unstale":"2024-09-19T08:39:38.961471+0000","last_undegraded":"2024-09-19T08:39:38.961471+0000","last_fullsized":"2024-09-19T08:39:38.961471+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:05.453778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.958959+0000","last_change":"2024-09-19T08:39:38.942176+0000","last_active":"2024-09-19T08:39:38.958959+0000","last_peered":"2024-09-19T08:39:38.958959+0000","last_clean":"2024-09-19T08:39:38.958959+0000","last_became_active":"2024-09-19T08:39:38.939625+0000","last_became_peered":"2024-09-19T08:39:38.939625+0000","last_unstale":"2024-09-19T08:39:38.958959+0000","last_undegraded":"2024-09-19T08:39:38.958959+0000","last_fullsized":"2024-09-19T08:39:38.958959+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:22:21.538914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"42'6","reported_seq":18,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.962495+0000","last_change":"2024-09-19T08:39:37.921227+0000","last_active":"2024-09-19T08:39:38.962495+0000","last_peered":"2024-09-19T08:39:38.962495+0000","last_clean":"2024-09-19T08:39:38.962495+0000","last_became_active":"2024-09-19T08:39:37.920773+0000","last_became_peered":"2024-09-19T08:39:37.920773+0000","last_unstale":"2024-09-19T08:39:38.962495+0000","last_undegraded":"2024-09-19T08:39:38.962495+0000","last_fullsized":"2024-09-19T08:39:38.962495+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:24:07.528992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.275773+0000","last_change":"2024-09-19T08:39:38.939598+0000","last_active":"2024-09-19T08:39:40.275773+0000","last_peered":"2024-09-19T08:39:40.275773+0000","last_clean":"2024-09-19T08:39:40.275773+0000","last_became_active":"2024-09-19T08:39:38.938870+0000","last_became_peered":"2024-09-19T08:39:38.938870+0000","last_unstale":"2024-09-19T08:39:40.275773+0000","last_undegraded":"2024-09-19T08:39:40.275773+0000","last_fullsized":"2024-09-19T08:39:40.275773+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:30:00.712901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"42'2","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.957446+0000","last_change":"2024-09-19T08:39:37.905231+0000","last_active":"2024-09-19T08:39:39.957446+0000","last_peered":"2024-09-19T08:39:39.957446+0000","last_clean":"2024-09-19T08:39:39.957446+0000","last_became_active":"2024-09-19T08:39:37.904660+0000","last_became_peered":"2024-09-19T08:39:37.904660+0000","last_unstale":"2024-09-19T08:39:39.957446+0000","last_undegraded":"2024-09-19T08:39:39.957446+0000","last_fullsized":"2024-09-19T08:39:39.957446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:12:31.651879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.940617+0000","last_change":"2024-09-19T08:39:38.936213+0000","last_active":"2024-09-19T08:39:38.940617+0000","last_peered":"2024-09-19T08:39:38.940617+0000","last_clean":"2024-09-19T08:39:38.940617+0000","last_became_active":"2024-09-19T08:39:38.935856+0000","last_became_peered":"2024-09-19T08:39:38.935856+0000","last_unstale":"2024-09-19T08:39:38.940617+0000","last_undegraded":"2024-09-19T08:39:38.940617+0000","last_fullsized":"2024-09-19T08:39:38.940617+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:29.744836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.908538+0000","last_change":"2024-09-19T08:39:37.914414+0000","last_active":"2024-09-19T08:39:38.908538+0000","last_peered":"2024-09-19T08:39:38.908538+0000","last_clean":"2024-09-19T08:39:38.908538+0000","last_became_active":"2024-09-19T08:39:37.913576+0000","last_became_peered":"2024-09-19T08:39:37.913576+0000","last_unstale":"2024-09-19T08:39:38.908538+0000","last_undegraded":"2024-09-19T08:39:38.908538+0000","last_fullsized":"2024-09-19T08:39:38.908538+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:28.741298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.957112+0000","last_change":"2024-09-19T08:39:38.920776+0000","last_active":"2024-09-19T08:39:39.957112+0000","last_peered":"2024-09-19T08:39:39.957112+0000","last_clean":"2024-09-19T08:39:39.957112+0000","last_became_active":"2024-09-19T08:39:38.919807+0000","last_became_peered":"2024-09-19T08:39:38.919807+0000","last_unstale":"2024-09-19T08:39:39.957112+0000","last_undegraded":"2024-09-19T08:39:39.957112+0000","last_fullsized":"2024-09-19T08:39:39.957112+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:35.525085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.957610+0000","last_change":"2024-09-19T08:39:37.910119+0000","last_active":"2024-09-19T08:39:39.957610+0000","last_peered":"2024-09-19T08:39:39.957610+0000","last_clean":"2024-09-19T08:39:39.957610+0000","last_became_active":"2024-09-19T08:39:37.909646+0000","last_became_peered":"2024-09-19T08:39:37.909646+0000","last_unstale":"2024-09-19T08:39:39.957610+0000","last_undegraded":"2024-09-19T08:39:39.957610+0000","last_fullsized":"2024-09-19T08:39:39.957610+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:17:02.556780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.959355+0000","last_change":"2024-09-19T08:39:38.937339+0000","last_active":"2024-09-19T08:39:38.959355+0000","last_peered":"2024-09-19T08:39:38.959355+0000","last_clean":"2024-09-19T08:39:38.959355+0000","last_became_active":"2024-09-19T08:39:38.936404+0000","last_became_peered":"2024-09-19T08:39:38.936404+0000","last_unstale":"2024-09-19T08:39:38.959355+0000","last_undegraded":"2024-09-19T08:39:38.959355+0000","last_fullsized":"2024-09-19T08:39:38.959355+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:00:09.742802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"42'8","reported_seq":20,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.957690+0000","last_change":"2024-09-19T08:39:37.918124+0000","last_active":"2024-09-19T08:39:38.957690+0000","last_peered":"2024-09-19T08:39:38.957690+0000","last_clean":"2024-09-19T08:39:38.957690+0000","last_became_active":"2024-09-19T08:39:37.917521+0000","last_became_peered":"2024-09-19T08:39:37.917521+0000","last_unstale":"2024-09-19T08:39:38.957690+0000","last_undegraded":"2024-09-19T08:39:38.957690+0000","last_fullsized":"2024-09-19T08:39:38.957690+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:38:39.741349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.937824+0000","last_change":"2024-09-19T08:39:38.926671+0000","last_active":"2024-09-19T08:39:38.937824+0000","last_peered":"2024-09-19T08:39:38.937824+0000","last_clean":"2024-09-19T08:39:38.937824+0000","last_became_active":"2024-09-19T08:39:38.925343+0000","last_became_peered":"2024-09-19T08:39:38.925343+0000","last_unstale":"2024-09-19T08:39:38.937824+0000","last_undegraded":"2024-09-19T08:39:38.937824+0000","last_fullsized":"2024-09-19T08:39:38.937824+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:21:41.168755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"42'1","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.949593+0000","last_change":"2024-09-19T08:39:37.911789+0000","last_active":"2024-09-19T08:39:38.949593+0000","last_peered":"2024-09-19T08:39:38.949593+0000","last_clean":"2024-09-19T08:39:38.949593+0000","last_became_active":"2024-09-19T08:39:37.911305+0000","last_became_peered":"2024-09-19T08:39:37.911305+0000","last_unstale":"2024-09-19T08:39:38.949593+0000","last_undegraded":"2024-09-19T08:39:38.949593+0000","last_fullsized":"2024-09-19T08:39:38.949593+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:13.592173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.938438+0000","last_change":"2024-09-19T08:39:38.935012+0000","last_active":"2024-09-19T08:39:38.938438+0000","last_peered":"2024-09-19T08:39:38.938438+0000","last_clean":"2024-09-19T08:39:38.938438+0000","last_became_active":"2024-09-19T08:39:38.934473+0000","last_became_peered":"2024-09-19T08:39:38.934473+0000","last_unstale":"2024-09-19T08:39:38.938438+0000","last_undegraded":"2024-09-19T08:39:38.938438+0000","last_fullsized":"2024-09-19T08:39:38.938438+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:33.862515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.959551+0000","last_change":"2024-09-19T08:39:37.923082+0000","last_active":"2024-09-19T08:39:38.959551+0000","last_peered":"2024-09-19T08:39:38.959551+0000","last_clean":"2024-09-19T08:39:38.959551+0000","last_became_active":"2024-09-19T08:39:37.922381+0000","last_became_peered":"2024-09-19T08:39:37.922381+0000","last_unstale":"2024-09-19T08:39:38.959551+0000","last_undegraded":"2024-09-19T08:39:38.959551+0000","last_fullsized":"2024-09-19T08:39:38.959551+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:08:43.556600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.276444+0000","last_change":"2024-09-19T08:39:38.937247+0000","last_active":"2024-09-19T08:39:40.276444+0000","last_peered":"2024-09-19T08:39:40.276444+0000","last_clean":"2024-09-19T08:39:40.276444+0000","last_became_active":"2024-09-19T08:39:38.936501+0000","last_became_peered":"2024-09-19T08:39:38.936501+0000","last_unstale":"2024-09-19T08:39:40.276444+0000","last_undegraded":"2024-09-19T08:39:40.276444+0000","last_fullsized":"2024-09-19T08:39:40.276444+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:55.400643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"42'1","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.956915+0000","last_change":"2024-09-19T08:39:37.917129+0000","last_active":"2024-09-19T08:39:38.956915+0000","last_peered":"2024-09-19T08:39:38.956915+0000","last_clean":"2024-09-19T08:39:38.956915+0000","last_became_active":"2024-09-19T08:39:37.916360+0000","last_became_peered":"2024-09-19T08:39:37.916360+0000","last_unstale":"2024-09-19T08:39:38.956915+0000","last_undegraded":"2024-09-19T08:39:38.956915+0000","last_fullsized":"2024-09-19T08:39:38.956915+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:39.361088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.276433+0000","last_change":"2024-09-19T08:39:38.932462+0000","last_active":"2024-09-19T08:39:40.276433+0000","last_peered":"2024-09-19T08:39:40.276433+0000","last_clean":"2024-09-19T08:39:40.276433+0000","last_became_active":"2024-09-19T08:39:38.931552+0000","last_became_peered":"2024-09-19T08:39:38.931552+0000","last_unstale":"2024-09-19T08:39:40.276433+0000","last_undegraded":"2024-09-19T08:39:40.276433+0000","last_fullsized":"2024-09-19T08:39:40.276433+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:50:32.514250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.957547+0000","last_change":"2024-09-19T08:39:37.932493+0000","last_active":"2024-09-19T08:39:39.957547+0000","last_peered":"2024-09-19T08:39:39.957547+0000","last_clean":"2024-09-19T08:39:39.957547+0000","last_became_active":"2024-09-19T08:39:37.932195+0000","last_became_peered":"2024-09-19T08:39:37.932195+0000","last_unstale":"2024-09-19T08:39:39.957547+0000","last_undegraded":"2024-09-19T08:39:39.957547+0000","last_fullsized":"2024-09-19T08:39:39.957547+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:46:44.370790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.957912+0000","last_change":"2024-09-19T08:39:38.932384+0000","last_active":"2024-09-19T08:39:39.957912+0000","last_peered":"2024-09-19T08:39:39.957912+0000","last_clean":"2024-09-19T08:39:39.957912+0000","last_became_active":"2024-09-19T08:39:38.932029+0000","last_became_peered":"2024-09-19T08:39:38.932029+0000","last_unstale":"2024-09-19T08:39:39.957912+0000","last_undegraded":"2024-09-19T08:39:39.957912+0000","last_fullsized":"2024-09-19T08:39:39.957912+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:17:30.846841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"42'2","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.957587+0000","last_change":"2024-09-19T08:39:37.913727+0000","last_active":"2024-09-19T08:39:39.957587+0000","last_peered":"2024-09-19T08:39:39.957587+0000","last_clean":"2024-09-19T08:39:39.957587+0000","last_became_active":"2024-09-19T08:39:37.913224+0000","last_became_peered":"2024-09-19T08:39:37.913224+0000","last_unstale":"2024-09-19T08:39:39.957587+0000","last_undegraded":"2024-09-19T08:39:39.957587+0000","last_fullsized":"2024-09-19T08:39:39.957587+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:18.771121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.928775+0000","last_change":"2024-09-19T08:39:38.923972+0000","last_active":"2024-09-19T08:39:38.928775+0000","last_peered":"2024-09-19T08:39:38.928775+0000","last_clean":"2024-09-19T08:39:38.928775+0000","last_became_active":"2024-09-19T08:39:38.923101+0000","last_became_peered":"2024-09-19T08:39:38.923101+0000","last_unstale":"2024-09-19T08:39:38.928775+0000","last_undegraded":"2024-09-19T08:39:38.928775+0000","last_fullsized":"2024-09-19T08:39:38.928775+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:08:44.560138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.911722+0000","last_change":"2024-09-19T08:39:37.934259+0000","last_active":"2024-09-19T08:39:38.911722+0000","last_peered":"2024-09-19T08:39:38.911722+0000","last_clean":"2024-09-19T08:39:38.911722+0000","last_became_active":"2024-09-19T08:39:37.933506+0000","last_became_peered":"2024-09-19T08:39:37.933506+0000","last_unstale":"2024-09-19T08:39:38.911722+0000","last_undegraded":"2024-09-19T08:39:38.911722+0000","last_fullsized":"2024-09-19T08:39:38.911722+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:22.044532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.948894+0000","last_change":"2024-09-19T08:39:38.944450+0000","last_active":"2024-09-19T08:39:38.948894+0000","last_peered":"2024-09-19T08:39:38.948894+0000","last_clean":"2024-09-19T08:39:38.948894+0000","last_became_active":"2024-09-19T08:39:38.944108+0000","last_became_peered":"2024-09-19T08:39:38.944108+0000","last_unstale":"2024-09-19T08:39:38.948894+0000","last_undegraded":"2024-09-19T08:39:38.948894+0000","last_fullsized":"2024-09-19T08:39:38.948894+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:56.145227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.901835+0000","last_change":"2024-09-19T08:39:37.922726+0000","last_active":"2024-09-19T08:39:38.901835+0000","last_peered":"2024-09-19T08:39:38.901835+0000","last_clean":"2024-09-19T08:39:38.901835+0000","last_became_active":"2024-09-19T08:39:37.922312+0000","last_became_peered":"2024-09-19T08:39:37.922312+0000","last_unstale":"2024-09-19T08:39:38.901835+0000","last_undegraded":"2024-09-19T08:39:38.901835+0000","last_fullsized":"2024-09-19T08:39:38.901835+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:05:28.346760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.935437+0000","last_change":"2024-09-19T08:39:38.920148+0000","last_active":"2024-09-19T08:39:38.935437+0000","last_peered":"2024-09-19T08:39:38.935437+0000","last_clean":"2024-09-19T08:39:38.935437+0000","last_became_active":"2024-09-19T08:39:38.919337+0000","last_became_peered":"2024-09-19T08:39:38.919337+0000","last_unstale":"2024-09-19T08:39:38.935437+0000","last_undegraded":"2024-09-19T08:39:38.935437+0000","last_fullsized":"2024-09-19T08:39:38.935437+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:27:20.365465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.908966+0000","last_change":"2024-09-19T08:39:37.915040+0000","last_active":"2024-09-19T08:39:38.908966+0000","last_peered":"2024-09-19T08:39:38.908966+0000","last_clean":"2024-09-19T08:39:38.908966+0000","last_became_active":"2024-09-19T08:39:37.914232+0000","last_became_peered":"2024-09-19T08:39:37.914232+0000","last_unstale":"2024-09-19T08:39:38.908966+0000","last_undegraded":"2024-09-19T08:39:38.908966+0000","last_fullsized":"2024-09-19T08:39:38.908966+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:49:13.065050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.276434+0000","last_change":"2024-09-19T08:39:38.940179+0000","last_active":"2024-09-19T08:39:40.276434+0000","last_peered":"2024-09-19T08:39:40.276434+0000","last_clean":"2024-09-19T08:39:40.276434+0000","last_became_active":"2024-09-19T08:39:38.939475+0000","last_became_peered":"2024-09-19T08:39:38.939475+0000","last_unstale":"2024-09-19T08:39:40.276434+0000","last_undegraded":"2024-09-19T08:39:40.276434+0000","last_fullsized":"2024-09-19T08:39:40.276434+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:36:50.598742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"43'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.957521+0000","last_change":"2024-09-19T08:39:37.918815+0000","last_active":"2024-09-19T08:39:39.957521+0000","last_peered":"2024-09-19T08:39:39.957521+0000","last_clean":"2024-09-19T08:39:39.957521+0000","last_became_active":"2024-09-19T08:39:37.918481+0000","last_became_peered":"2024-09-19T08:39:37.918481+0000","last_unstale":"2024-09-19T08:39:39.957521+0000","last_undegraded":"2024-09-19T08:39:39.957521+0000","last_fullsized":"2024-09-19T08:39:39.957521+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:32.513917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.936888+0000","last_change":"2024-09-19T08:39:38.929785+0000","last_active":"2024-09-19T08:39:38.936888+0000","last_peered":"2024-09-19T08:39:38.936888+0000","last_clean":"2024-09-19T08:39:38.936888+0000","last_became_active":"2024-09-19T08:39:38.929074+0000","last_became_peered":"2024-09-19T08:39:38.929074+0000","last_unstale":"2024-09-19T08:39:38.936888+0000","last_undegraded":"2024-09-19T08:39:38.936888+0000","last_fullsized":"2024-09-19T08:39:38.936888+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:37.035646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.918780+0000","last_change":"2024-09-19T08:39:37.936655+0000","last_active":"2024-09-19T08:39:38.918780+0000","last_peered":"2024-09-19T08:39:38.918780+0000","last_clean":"2024-09-19T08:39:38.918780+0000","last_became_active":"2024-09-19T08:39:37.936112+0000","last_became_peered":"2024-09-19T08:39:37.936112+0000","last_unstale":"2024-09-19T08:39:38.918780+0000","last_undegraded":"2024-09-19T08:39:38.918780+0000","last_fullsized":"2024-09-19T08:39:38.918780+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:33:43.714330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.959000+0000","last_change":"2024-09-19T08:39:38.942782+0000","last_active":"2024-09-19T08:39:38.959000+0000","last_peered":"2024-09-19T08:39:38.959000+0000","last_clean":"2024-09-19T08:39:38.959000+0000","last_became_active":"2024-09-19T08:39:38.939689+0000","last_became_peered":"2024-09-19T08:39:38.939689+0000","last_unstale":"2024-09-19T08:39:38.959000+0000","last_undegraded":"2024-09-19T08:39:38.959000+0000","last_fullsized":"2024-09-19T08:39:38.959000+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:37.082611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.895030+0000","last_change":"2024-09-19T08:39:37.924237+0000","last_active":"2024-09-19T08:39:38.895030+0000","last_peered":"2024-09-19T08:39:38.895030+0000","last_clean":"2024-09-19T08:39:38.895030+0000","last_became_active":"2024-09-19T08:39:37.923832+0000","last_became_peered":"2024-09-19T08:39:37.923832+0000","last_unstale":"2024-09-19T08:39:38.895030+0000","last_undegraded":"2024-09-19T08:39:38.895030+0000","last_fullsized":"2024-09-19T08:39:38.895030+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:14:31.270319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.957703+0000","last_change":"2024-09-19T08:39:38.931776+0000","last_active":"2024-09-19T08:39:39.957703+0000","last_peered":"2024-09-19T08:39:39.957703+0000","last_clean":"2024-09-19T08:39:39.957703+0000","last_became_active":"2024-09-19T08:39:38.931055+0000","last_became_peered":"2024-09-19T08:39:38.931055+0000","last_unstale":"2024-09-19T08:39:39.957703+0000","last_undegraded":"2024-09-19T08:39:39.957703+0000","last_fullsized":"2024-09-19T08:39:39.957703+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:14.144870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.919124+0000","last_change":"2024-09-19T08:39:37.934015+0000","last_active":"2024-09-19T08:39:38.919124+0000","last_peered":"2024-09-19T08:39:38.919124+0000","last_clean":"2024-09-19T08:39:38.919124+0000","last_became_active":"2024-09-19T08:39:37.933256+0000","last_became_peered":"2024-09-19T08:39:37.933256+0000","last_unstale":"2024-09-19T08:39:38.919124+0000","last_undegraded":"2024-09-19T08:39:38.919124+0000","last_fullsized":"2024-09-19T08:39:38.919124+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:55:09.923681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.957357+0000","last_change":"2024-09-19T08:39:38.931817+0000","last_active":"2024-09-19T08:39:39.957357+0000","last_peered":"2024-09-19T08:39:39.957357+0000","last_clean":"2024-09-19T08:39:39.957357+0000","last_became_active":"2024-09-19T08:39:38.931021+0000","last_became_peered":"2024-09-19T08:39:38.931021+0000","last_unstale":"2024-09-19T08:39:39.957357+0000","last_undegraded":"2024-09-19T08:39:39.957357+0000","last_fullsized":"2024-09-19T08:39:39.957357+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:53:30.156065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.901249+0000","last_change":"2024-09-19T08:39:37.920347+0000","last_active":"2024-09-19T08:39:38.901249+0000","last_peered":"2024-09-19T08:39:38.901249+0000","last_clean":"2024-09-19T08:39:38.901249+0000","last_became_active":"2024-09-19T08:39:37.919634+0000","last_became_peered":"2024-09-19T08:39:37.919634+0000","last_unstale":"2024-09-19T08:39:38.901249+0000","last_undegraded":"2024-09-19T08:39:38.901249+0000","last_fullsized":"2024-09-19T08:39:38.901249+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:14:31.270319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.933874+0000","last_change":"2024-09-19T08:39:38.927845+0000","last_active":"2024-09-19T08:39:38.933874+0000","last_peered":"2024-09-19T08:39:38.933874+0000","last_clean":"2024-09-19T08:39:38.933874+0000","last_became_active":"2024-09-19T08:39:38.926989+0000","last_became_peered":"2024-09-19T08:39:38.926989+0000","last_unstale":"2024-09-19T08:39:38.933874+0000","last_undegraded":"2024-09-19T08:39:38.933874+0000","last_fullsized":"2024-09-19T08:39:38.933874+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:55:10.927219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.895000+0000","last_change":"2024-09-19T08:39:37.910329+0000","last_active":"2024-09-19T08:39:38.895000+0000","last_peered":"2024-09-19T08:39:38.895000+0000","last_clean":"2024-09-19T08:39:38.895000+0000","last_became_active":"2024-09-19T08:39:37.909966+0000","last_became_peered":"2024-09-19T08:39:37.909966+0000","last_unstale":"2024-09-19T08:39:38.895000+0000","last_undegraded":"2024-09-19T08:39:38.895000+0000","last_fullsized":"2024-09-19T08:39:38.895000+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:36.032108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.942950+0000","last_change":"2024-09-19T08:39:38.936998+0000","last_active":"2024-09-19T08:39:38.942950+0000","last_peered":"2024-09-19T08:39:38.942950+0000","last_clean":"2024-09-19T08:39:38.942950+0000","last_became_active":"2024-09-19T08:39:38.935884+0000","last_became_peered":"2024-09-19T08:39:38.935884+0000","last_unstale":"2024-09-19T08:39:38.942950+0000","last_undegraded":"2024-09-19T08:39:38.942950+0000","last_fullsized":"2024-09-19T08:39:38.942950+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:40.364626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.903222+0000","last_change":"2024-09-19T08:39:37.907408+0000","last_active":"2024-09-19T08:39:38.903222+0000","last_peered":"2024-09-19T08:39:38.903222+0000","last_clean":"2024-09-19T08:39:38.903222+0000","last_became_active":"2024-09-19T08:39:37.906928+0000","last_became_peered":"2024-09-19T08:39:37.906928+0000","last_unstale":"2024-09-19T08:39:38.903222+0000","last_undegraded":"2024-09-19T08:39:38.903222+0000","last_fullsized":"2024-09-19T08:39:38.903222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:08.285216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.945594+0000","last_change":"2024-09-19T08:39:38.940058+0000","last_active":"2024-09-19T08:39:38.945594+0000","last_peered":"2024-09-19T08:39:38.945594+0000","last_clean":"2024-09-19T08:39:38.945594+0000","last_became_active":"2024-09-19T08:39:38.939663+0000","last_became_peered":"2024-09-19T08:39:38.939663+0000","last_unstale":"2024-09-19T08:39:38.945594+0000","last_undegraded":"2024-09-19T08:39:38.945594+0000","last_fullsized":"2024-09-19T08:39:38.945594+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:40.364626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"42'2","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.957136+0000","last_change":"2024-09-19T08:39:37.922845+0000","last_active":"2024-09-19T08:39:38.957136+0000","last_peered":"2024-09-19T08:39:38.957136+0000","last_clean":"2024-09-19T08:39:38.957136+0000","last_became_active":"2024-09-19T08:39:37.922153+0000","last_became_peered":"2024-09-19T08:39:37.922153+0000","last_unstale":"2024-09-19T08:39:38.957136+0000","last_undegraded":"2024-09-19T08:39:38.957136+0000","last_fullsized":"2024-09-19T08:39:38.957136+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:28.741298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":204,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-19T08:39:38.908394+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:39:38.908394+0000","last_peered":"2024-09-19T08:39:38.908394+0000","last_clean":"2024-09-19T08:39:38.908394+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:39:38.908394+0000","last_undegraded":"2024-09-19T08:39:38.908394+0000","last_fullsized":"2024-09-19T08:39:38.908394+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509461,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1081344,"data_stored":783009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509463,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":185776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509463,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":471040,"data_stored":185776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509460,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":185776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509462,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":185776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509460,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28072,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343768,"statfs":{"total":96636764160,"available":96608018432,"internally_reserved":0,"allocated":1064960,"data_stored":776144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":503808,"data_stored":199523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542166,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31684,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340156,"statfs":{"total":96636764160,"available":96604319744,"internally_reserved":0,"allocated":507904,"data_stored":199489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:39:47.227 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:39:47.231 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:39:47.388 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:39:47.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.519+0000 7f17cc002640 1 Processor -- start 2024-09-19T08:39:47.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.519+0000 7f17cc002640 1 -- start start 2024-09-19T08:39:47.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.523+0000 7f17cc002640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f17c4158fd0 0x7f17c40b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:47.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.523+0000 7f17cc002640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17c40ccc60 con 0x7f17c406c3a0 2024-09-19T08:39:47.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.523+0000 7f17cc002640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17c40ccf40 con 0x7f17c40b4360 2024-09-19T08:39:47.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.523+0000 7f17cc002640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f17c41593b0 con 0x7f17c4158fd0 2024-09-19T08:39:47.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.523+0000 7f17ca59f640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f17c406c3a0 0x7f17c40ce840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:35818/0 (socket says 172.21.15.98:35818) 2024-09-19T08:39:47.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.523+0000 7f17ca59f640 1 -- 172.21.15.98:0/2508531764 learned_addr learned my addr 172.21.15.98:0/2508531764 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:47.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.523+0000 7f17c9d9e640 1 --2- 172.21.15.98:0/2508531764 >> v2:172.21.15.139:3300/0 conn(0x7f17c4158fd0 0x7f17c40b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:47.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.523+0000 7f17c8d9c640 1 -- 172.21.15.98:0/2508531764 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 46878048 0 0) 0x7f17c40ccc60 con 0x7f17c406c3a0 2024-09-19T08:39:47.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.523+0000 7f17c8d9c640 1 -- 172.21.15.98:0/2508531764 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f17b0003530 con 0x7f17c406c3a0 2024-09-19T08:39:47.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.523+0000 7f17c8d9c640 1 -- 172.21.15.98:0/2508531764 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3197352641 0 0) 0x7f17b0003530 con 0x7f17c406c3a0 2024-09-19T08:39:47.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.523+0000 7f17c8d9c640 1 -- 172.21.15.98:0/2508531764 >> v2:172.21.15.139:3300/0 conn(0x7f17c4158fd0 msgr2=0x7f17c40b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:47.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.523+0000 7f17c8d9c640 1 --2- 172.21.15.98:0/2508531764 >> v2:172.21.15.139:3300/0 conn(0x7f17c4158fd0 0x7f17c40b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:47.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.523+0000 7f17c8d9c640 1 -- 172.21.15.98:0/2508531764 >> v1:172.21.15.139:6789/0 conn(0x7f17c40b4360 legacy=0x7f17c406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:47.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.523+0000 7f17c8d9c640 1 -- 172.21.15.98:0/2508531764 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c40cd220 con 0x7f17c406c3a0 2024-09-19T08:39:47.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.523+0000 7f17c8d9c640 1 -- 172.21.15.98:0/2508531764 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f17bc001680 con 0x7f17c406c3a0 2024-09-19T08:39:47.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.523+0000 7f17c8d9c640 1 -- 172.21.15.98:0/2508531764 >> v1:172.21.15.98:6789/0 conn(0x7f17c406c3a0 legacy=0x7f17c40ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:47.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.523+0000 7f17c8d9c640 1 --2- 172.21.15.98:0/2508531764 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f17b0003a10 0x7f17b0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:47.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.523+0000 7f17c8d9c640 1 --2- 172.21.15.98:0/2508531764 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f17b0004810 0x7f17b0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:47.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.523+0000 7f17c8d9c640 1 --2- 172.21.15.98:0/2508531764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17b0005220 0x7f17b000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:47.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.523+0000 7f17c8d9c640 1 -- 172.21.15.98:0/2508531764 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f17c41593b0 con 0x7f17b0005220 2024-09-19T08:39:47.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.523+0000 7f17c8d9c640 1 -- 172.21.15.98:0/2508531764 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f17b000e380 con 0x7f17b0003a10 2024-09-19T08:39:47.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.523+0000 7f17c8d9c640 1 -- 172.21.15.98:0/2508531764 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f17b000e580 con 0x7f17b0004810 2024-09-19T08:39:47.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.523+0000 7f17ca59f640 1 --2- 172.21.15.98:0/2508531764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17b0005220 0x7f17b000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:47.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.523+0000 7f17c9d9e640 1 --2- 172.21.15.98:0/2508531764 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f17b0003a10 0x7f17b0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:47.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.523+0000 7f17c959d640 1 --2- 172.21.15.98:0/2508531764 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f17b0004810 0x7f17b0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:47.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.523+0000 7f17ca59f640 1 -- 172.21.15.98:0/2508531764 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f17b0004810 msgr2=0x7f17b0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:47.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.523+0000 7f17ca59f640 1 --2- 172.21.15.98:0/2508531764 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f17b0004810 0x7f17b0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:47.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.523+0000 7f17ca59f640 1 -- 172.21.15.98:0/2508531764 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f17b0003a10 msgr2=0x7f17b0003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:47.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.523+0000 7f17ca59f640 1 --2- 172.21.15.98:0/2508531764 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f17b0003a10 0x7f17b0003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:47.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.523+0000 7f17ca59f640 1 -- 172.21.15.98:0/2508531764 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17b0003530 con 0x7f17b0005220 2024-09-19T08:39:47.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.527+0000 7f17ca59f640 1 -- 172.21.15.98:0/2508531764 reap_dead start 2024-09-19T08:39:47.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.527+0000 7f17ca59f640 1 --2- 172.21.15.98:0/2508531764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17b0005220 0x7f17b000b9d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f17c40cdd40 tx=0x7f17bc01f290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d791c64bec8323 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:47.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.527+0000 7f17c8d9c640 1 -- 172.21.15.98:0/2508531764 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc007c50 con 0x7f17b0005220 2024-09-19T08:39:47.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.527+0000 7f17c8d9c640 1 -- 172.21.15.98:0/2508531764 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17bc0ac070 con 0x7f17b0005220 2024-09-19T08:39:47.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.527+0000 7f17c8d9c640 1 -- 172.21.15.98:0/2508531764 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc005b20 con 0x7f17b0005220 2024-09-19T08:39:47.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.527+0000 7f17cc002640 1 -- 172.21.15.98:0/2508531764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17b0005220 msgr2=0x7f17b000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:47.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.527+0000 7f17cc002640 1 --2- 172.21.15.98:0/2508531764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17b0005220 0x7f17b000b9d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f17c40cdd40 tx=0x7f17bc01f290 comp rx=0 tx=0).stop 2024-09-19T08:39:47.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.527+0000 7f17cc002640 1 -- 172.21.15.98:0/2508531764 shutdown_connections 2024-09-19T08:39:47.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.527+0000 7f17cc002640 1 --2- 172.21.15.98:0/2508531764 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17b0005220 0x7f17b000b9d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:47.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.527+0000 7f17cc002640 1 -- 172.21.15.98:0/2508531764 >> 172.21.15.98:0/2508531764 conn(0x7f17c40b2010 msgr2=0x7f17c40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:47.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.527+0000 7f17cc002640 1 -- 172.21.15.98:0/2508531764 shutdown_connections 2024-09-19T08:39:47.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.527+0000 7f17cc002640 1 -- 172.21.15.98:0/2508531764 wait complete. 2024-09-19T08:39:47.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.527+0000 7f17cc002640 1 Processor -- start 2024-09-19T08:39:47.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.527+0000 7f17cc002640 1 -- start start 2024-09-19T08:39:47.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.527+0000 7f17cc002640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17b0005220 0x7f17c4145d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:47.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.527+0000 7f17cc002640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f17c41f32b0 0x7f17c415f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:47.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.527+0000 7f17cc002640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f17c40727c0 0x7f17c4154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:47.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.527+0000 7f17cc002640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f17c415c7a0 con 0x7f17b0005220 2024-09-19T08:39:47.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.527+0000 7f17cc002640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f17c40cd7e0 con 0x7f17c40727c0 2024-09-19T08:39:47.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.527+0000 7f17cc002640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f17c415c920 con 0x7f17c41f32b0 2024-09-19T08:39:47.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.527+0000 7f17c9d9e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17b0005220 0x7f17c4145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:47.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.527+0000 7f17c9d9e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17b0005220 0x7f17c4145d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57048/0 (socket says 172.21.15.98:57048) 2024-09-19T08:39:47.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.527+0000 7f17c9d9e640 1 -- 172.21.15.98:0/3699837208 learned_addr learned my addr 172.21.15.98:0/3699837208 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:47.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.527+0000 7f17ca59f640 1 --2- 172.21.15.98:0/3699837208 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f17c40727c0 0x7f17c4154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:47.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.527+0000 7f17c959d640 1 --2- 172.21.15.98:0/3699837208 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f17c41f32b0 0x7f17c415f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:47.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.527+0000 7f17c9d9e640 1 -- 172.21.15.98:0/3699837208 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f17c41f32b0 msgr2=0x7f17c415f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:47.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.527+0000 7f17c9d9e640 1 --2- 172.21.15.98:0/3699837208 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f17c41f32b0 0x7f17c415f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:47.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.527+0000 7f17c9d9e640 1 -- 172.21.15.98:0/3699837208 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f17c40727c0 msgr2=0x7f17c4154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:47.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.527+0000 7f17c9d9e640 1 --2- 172.21.15.98:0/3699837208 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f17c40727c0 0x7f17c4154ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:47.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.527+0000 7f17c9d9e640 1 -- 172.21.15.98:0/3699837208 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c41f3820 con 0x7f17b0005220 2024-09-19T08:39:47.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.531+0000 7f17c9d9e640 1 --2- 172.21.15.98:0/3699837208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17b0005220 0x7f17c4145d10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f17c0007f30 tx=0x7f17c0011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:47.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.531+0000 7f17ba7fc640 1 -- 172.21.15.98:0/3699837208 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c001a020 con 0x7f17b0005220 2024-09-19T08:39:47.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.531+0000 7f17ba7fc640 1 -- 172.21.15.98:0/3699837208 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17c0016470 con 0x7f17b0005220 2024-09-19T08:39:47.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.531+0000 7f17cc002640 1 -- 172.21.15.98:0/3699837208 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17c41f3b00 con 0x7f17b0005220 2024-09-19T08:39:47.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.531+0000 7f17ba7fc640 1 -- 172.21.15.98:0/3699837208 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c0016860 con 0x7f17b0005220 2024-09-19T08:39:47.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.531+0000 7f17cc002640 1 -- 172.21.15.98:0/3699837208 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17c41f40c0 con 0x7f17b0005220 2024-09-19T08:39:47.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.531+0000 7f17cc002640 1 -- 172.21.15.98:0/3699837208 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17c41574d0 con 0x7f17b0005220 2024-09-19T08:39:47.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.531+0000 7f17ba7fc640 1 -- 172.21.15.98:0/3699837208 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f17c001d020 con 0x7f17b0005220 2024-09-19T08:39:47.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.531+0000 7f17ba7fc640 1 --2- 172.21.15.98:0/3699837208 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f17a8067aa0 0x7f17a8069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:47.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.531+0000 7f17ba7fc640 1 -- 172.21.15.98:0/3699837208 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f17c0086ef0 con 0x7f17b0005220 2024-09-19T08:39:47.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.531+0000 7f17c959d640 1 --2- 172.21.15.98:0/3699837208 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f17a8067aa0 0x7f17a8069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:47.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.531+0000 7f17c959d640 1 --2- 172.21.15.98:0/3699837208 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f17a8067aa0 0x7f17a8069f60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f17c415d040 tx=0x7f17b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:47.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.535+0000 7f17ba7fc640 1 -- 172.21.15.98:0/3699837208 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f17c0024030 con 0x7f17b0005220 2024-09-19T08:39:47.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.695+0000 7f17cc002640 1 -- 172.21.15.98:0/3699837208 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f17c41571f0 con 0x7f17a8067aa0 2024-09-19T08:39:47.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.715+0000 7f17ba7fc640 1 -- 172.21.15.98:0/3699837208 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251959 (secure 0 0 0) 0x7f17c41571f0 con 0x7f17a8067aa0 2024-09-19T08:39:47.716 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:39:47.718 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:39:47.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.719+0000 7f17cc002640 1 -- 172.21.15.98:0/3699837208 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f17a8067aa0 msgr2=0x7f17a8069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:47.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.719+0000 7f17cc002640 1 --2- 172.21.15.98:0/3699837208 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f17a8067aa0 0x7f17a8069f60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f17c415d040 tx=0x7f17b4009040 comp rx=0 tx=0).stop 2024-09-19T08:39:47.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.719+0000 7f17cc002640 1 -- 172.21.15.98:0/3699837208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17b0005220 msgr2=0x7f17c4145d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:47.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.719+0000 7f17cc002640 1 --2- 172.21.15.98:0/3699837208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17b0005220 0x7f17c4145d10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f17c0007f30 tx=0x7f17c0011660 comp rx=0 tx=0).stop 2024-09-19T08:39:47.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.719+0000 7f17cc002640 1 -- 172.21.15.98:0/3699837208 shutdown_connections 2024-09-19T08:39:47.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.719+0000 7f17cc002640 1 --2- 172.21.15.98:0/3699837208 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f17a8067aa0 0x7f17a8069f60 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:47.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.719+0000 7f17cc002640 1 --2- 172.21.15.98:0/3699837208 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f17c40727c0 0x7f17c4154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:47.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.719+0000 7f17cc002640 1 --2- 172.21.15.98:0/3699837208 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f17c41f32b0 0x7f17c415f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:47.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.719+0000 7f17cc002640 1 --2- 172.21.15.98:0/3699837208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f17b0005220 0x7f17c4145d10 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:47.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.719+0000 7f17cc002640 1 -- 172.21.15.98:0/3699837208 >> 172.21.15.98:0/3699837208 conn(0x7f17c40b2010 msgr2=0x7f17c406bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:47.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.719+0000 7f17cc002640 1 -- 172.21.15.98:0/3699837208 shutdown_connections 2024-09-19T08:39:47.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.719+0000 7f17cc002640 1 -- 172.21.15.98:0/3699837208 wait complete. 2024-09-19T08:39:47.736 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":85,"stamp":"2024-09-19T08:39:46.051744+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229648,"kb_used_data":5096,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754745072,"statfs":{"total":773094113280,"available":772858953728,"internally_reserved":0,"allocated":5218304,"data_stored":2776364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002007"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:41.061208+0000","last_change":"2024-09-19T08:39:38.938293+0000","last_active":"2024-09-19T08:39:41.061208+0000","last_peered":"2024-09-19T08:39:41.061208+0000","last_clean":"2024-09-19T08:39:41.061208+0000","last_became_active":"2024-09-19T08:39:38.936784+0000","last_became_peered":"2024-09-19T08:39:38.936784+0000","last_unstale":"2024-09-19T08:39:41.061208+0000","last_undegraded":"2024-09-19T08:39:41.061208+0000","last_fullsized":"2024-09-19T08:39:41.061208+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:32:54.953594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.699430+0000","last_change":"2024-09-19T08:39:38.942347+0000","last_active":"2024-09-19T08:39:40.699430+0000","last_peered":"2024-09-19T08:39:40.699430+0000","last_clean":"2024-09-19T08:39:40.699430+0000","last_became_active":"2024-09-19T08:39:38.941641+0000","last_became_peered":"2024-09-19T08:39:38.941641+0000","last_unstale":"2024-09-19T08:39:40.699430+0000","last_undegraded":"2024-09-19T08:39:40.699430+0000","last_fullsized":"2024-09-19T08:39:40.699430+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:57.785025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.898803+0000","last_change":"2024-09-19T08:39:38.918421+0000","last_active":"2024-09-19T08:39:39.898803+0000","last_peered":"2024-09-19T08:39:39.898803+0000","last_clean":"2024-09-19T08:39:39.898803+0000","last_became_active":"2024-09-19T08:39:38.917761+0000","last_became_peered":"2024-09-19T08:39:38.917761+0000","last_unstale":"2024-09-19T08:39:39.898803+0000","last_undegraded":"2024-09-19T08:39:39.898803+0000","last_fullsized":"2024-09-19T08:39:39.898803+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:29.732835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.591064+0000","last_change":"2024-09-19T08:39:38.932965+0000","last_active":"2024-09-19T08:39:40.591064+0000","last_peered":"2024-09-19T08:39:40.591064+0000","last_clean":"2024-09-19T08:39:40.591064+0000","last_became_active":"2024-09-19T08:39:38.929490+0000","last_became_peered":"2024-09-19T08:39:38.929490+0000","last_unstale":"2024-09-19T08:39:40.591064+0000","last_undegraded":"2024-09-19T08:39:40.591064+0000","last_fullsized":"2024-09-19T08:39:40.591064+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:37:14.987023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.698862+0000","last_change":"2024-09-19T08:39:38.938221+0000","last_active":"2024-09-19T08:39:40.698862+0000","last_peered":"2024-09-19T08:39:40.698862+0000","last_clean":"2024-09-19T08:39:40.698862+0000","last_became_active":"2024-09-19T08:39:38.937356+0000","last_became_peered":"2024-09-19T08:39:38.937356+0000","last_unstale":"2024-09-19T08:39:40.698862+0000","last_undegraded":"2024-09-19T08:39:40.698862+0000","last_fullsized":"2024-09-19T08:39:40.698862+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:37:14.987023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.957225+0000","last_change":"2024-09-19T08:39:38.937688+0000","last_active":"2024-09-19T08:39:39.957225+0000","last_peered":"2024-09-19T08:39:39.957225+0000","last_clean":"2024-09-19T08:39:39.957225+0000","last_became_active":"2024-09-19T08:39:38.937458+0000","last_became_peered":"2024-09-19T08:39:38.937458+0000","last_unstale":"2024-09-19T08:39:39.957225+0000","last_undegraded":"2024-09-19T08:39:39.957225+0000","last_fullsized":"2024-09-19T08:39:39.957225+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:12:54.866527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:41.060649+0000","last_change":"2024-09-19T08:39:38.945329+0000","last_active":"2024-09-19T08:39:41.060649+0000","last_peered":"2024-09-19T08:39:41.060649+0000","last_clean":"2024-09-19T08:39:41.060649+0000","last_became_active":"2024-09-19T08:39:38.944121+0000","last_became_peered":"2024-09-19T08:39:38.944121+0000","last_unstale":"2024-09-19T08:39:41.060649+0000","last_undegraded":"2024-09-19T08:39:41.060649+0000","last_fullsized":"2024-09-19T08:39:41.060649+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:06:12.303817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.590060+0000","last_change":"2024-09-19T08:39:38.929792+0000","last_active":"2024-09-19T08:39:40.590060+0000","last_peered":"2024-09-19T08:39:40.590060+0000","last_clean":"2024-09-19T08:39:40.590060+0000","last_became_active":"2024-09-19T08:39:38.928743+0000","last_became_peered":"2024-09-19T08:39:38.928743+0000","last_unstale":"2024-09-19T08:39:40.590060+0000","last_undegraded":"2024-09-19T08:39:40.590060+0000","last_fullsized":"2024-09-19T08:39:40.590060+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:33:44.717869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.591210+0000","last_change":"2024-09-19T08:39:38.934773+0000","last_active":"2024-09-19T08:39:40.591210+0000","last_peered":"2024-09-19T08:39:40.591210+0000","last_clean":"2024-09-19T08:39:40.591210+0000","last_became_active":"2024-09-19T08:39:38.934054+0000","last_became_peered":"2024-09-19T08:39:38.934054+0000","last_unstale":"2024-09-19T08:39:40.591210+0000","last_undegraded":"2024-09-19T08:39:40.591210+0000","last_fullsized":"2024-09-19T08:39:40.591210+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:57.785025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.591642+0000","last_change":"2024-09-19T08:39:38.934619+0000","last_active":"2024-09-19T08:39:40.591642+0000","last_peered":"2024-09-19T08:39:40.591642+0000","last_clean":"2024-09-19T08:39:40.591642+0000","last_became_active":"2024-09-19T08:39:38.934244+0000","last_became_peered":"2024-09-19T08:39:38.934244+0000","last_unstale":"2024-09-19T08:39:40.591642+0000","last_undegraded":"2024-09-19T08:39:40.591642+0000","last_fullsized":"2024-09-19T08:39:40.591642+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:32:54.953594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.591015+0000","last_change":"2024-09-19T08:39:38.933595+0000","last_active":"2024-09-19T08:39:40.591015+0000","last_peered":"2024-09-19T08:39:40.591015+0000","last_clean":"2024-09-19T08:39:40.591015+0000","last_became_active":"2024-09-19T08:39:38.933316+0000","last_became_peered":"2024-09-19T08:39:38.933316+0000","last_unstale":"2024-09-19T08:39:40.591015+0000","last_undegraded":"2024-09-19T08:39:40.591015+0000","last_fullsized":"2024-09-19T08:39:40.591015+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:37.082611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.698868+0000","last_change":"2024-09-19T08:39:38.923904+0000","last_active":"2024-09-19T08:39:40.698868+0000","last_peered":"2024-09-19T08:39:40.698868+0000","last_clean":"2024-09-19T08:39:40.698868+0000","last_became_active":"2024-09-19T08:39:38.923135+0000","last_became_peered":"2024-09-19T08:39:38.923135+0000","last_unstale":"2024-09-19T08:39:40.698868+0000","last_undegraded":"2024-09-19T08:39:40.698868+0000","last_fullsized":"2024-09-19T08:39:40.698868+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:06:12.303817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.897312+0000","last_change":"2024-09-19T08:39:38.933031+0000","last_active":"2024-09-19T08:39:39.897312+0000","last_peered":"2024-09-19T08:39:39.897312+0000","last_clean":"2024-09-19T08:39:39.897312+0000","last_became_active":"2024-09-19T08:39:38.932568+0000","last_became_peered":"2024-09-19T08:39:38.932568+0000","last_unstale":"2024-09-19T08:39:39.897312+0000","last_undegraded":"2024-09-19T08:39:39.897312+0000","last_fullsized":"2024-09-19T08:39:39.897312+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:05:29.350298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.590295+0000","last_change":"2024-09-19T08:39:38.936306+0000","last_active":"2024-09-19T08:39:40.590295+0000","last_peered":"2024-09-19T08:39:40.590295+0000","last_clean":"2024-09-19T08:39:40.590295+0000","last_became_active":"2024-09-19T08:39:38.935988+0000","last_became_peered":"2024-09-19T08:39:38.935988+0000","last_unstale":"2024-09-19T08:39:40.590295+0000","last_undegraded":"2024-09-19T08:39:40.590295+0000","last_fullsized":"2024-09-19T08:39:40.590295+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:55:10.927219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.957339+0000","last_change":"2024-09-19T08:39:38.935062+0000","last_active":"2024-09-19T08:39:39.957339+0000","last_peered":"2024-09-19T08:39:39.957339+0000","last_clean":"2024-09-19T08:39:39.957339+0000","last_became_active":"2024-09-19T08:39:38.933932+0000","last_became_peered":"2024-09-19T08:39:38.933932+0000","last_unstale":"2024-09-19T08:39:39.957339+0000","last_undegraded":"2024-09-19T08:39:39.957339+0000","last_fullsized":"2024-09-19T08:39:39.957339+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:18:18.796878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.275549+0000","last_change":"2024-09-19T08:39:38.940029+0000","last_active":"2024-09-19T08:39:40.275549+0000","last_peered":"2024-09-19T08:39:40.275549+0000","last_clean":"2024-09-19T08:39:40.275549+0000","last_became_active":"2024-09-19T08:39:38.939350+0000","last_became_peered":"2024-09-19T08:39:38.939350+0000","last_unstale":"2024-09-19T08:39:40.275549+0000","last_undegraded":"2024-09-19T08:39:40.275549+0000","last_fullsized":"2024-09-19T08:39:40.275549+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:01:10.462123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.899975+0000","last_change":"2024-09-19T08:39:38.924533+0000","last_active":"2024-09-19T08:39:39.899975+0000","last_peered":"2024-09-19T08:39:39.899975+0000","last_clean":"2024-09-19T08:39:39.899975+0000","last_became_active":"2024-09-19T08:39:38.923817+0000","last_became_peered":"2024-09-19T08:39:38.923817+0000","last_unstale":"2024-09-19T08:39:39.899975+0000","last_undegraded":"2024-09-19T08:39:39.899975+0000","last_fullsized":"2024-09-19T08:39:39.899975+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:06:12.303817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.899314+0000","last_change":"2024-09-19T08:39:38.938932+0000","last_active":"2024-09-19T08:39:39.899314+0000","last_peered":"2024-09-19T08:39:39.899314+0000","last_clean":"2024-09-19T08:39:39.899314+0000","last_became_active":"2024-09-19T08:39:38.937330+0000","last_became_peered":"2024-09-19T08:39:38.937330+0000","last_unstale":"2024-09-19T08:39:39.899314+0000","last_undegraded":"2024-09-19T08:39:39.899314+0000","last_fullsized":"2024-09-19T08:39:39.899314+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:37.082611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.276486+0000","last_change":"2024-09-19T08:39:38.939262+0000","last_active":"2024-09-19T08:39:40.276486+0000","last_peered":"2024-09-19T08:39:40.276486+0000","last_clean":"2024-09-19T08:39:40.276486+0000","last_became_active":"2024-09-19T08:39:38.938546+0000","last_became_peered":"2024-09-19T08:39:38.938546+0000","last_unstale":"2024-09-19T08:39:40.276486+0000","last_undegraded":"2024-09-19T08:39:40.276486+0000","last_fullsized":"2024-09-19T08:39:40.276486+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:38:49.933214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.897023+0000","last_change":"2024-09-19T08:39:38.936241+0000","last_active":"2024-09-19T08:39:39.897023+0000","last_peered":"2024-09-19T08:39:39.897023+0000","last_clean":"2024-09-19T08:39:39.897023+0000","last_became_active":"2024-09-19T08:39:38.935936+0000","last_became_peered":"2024-09-19T08:39:38.935936+0000","last_unstale":"2024-09-19T08:39:39.897023+0000","last_undegraded":"2024-09-19T08:39:39.897023+0000","last_fullsized":"2024-09-19T08:39:39.897023+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:23:28.648173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.957241+0000","last_change":"2024-09-19T08:39:38.931744+0000","last_active":"2024-09-19T08:39:39.957241+0000","last_peered":"2024-09-19T08:39:39.957241+0000","last_clean":"2024-09-19T08:39:39.957241+0000","last_became_active":"2024-09-19T08:39:38.931246+0000","last_became_peered":"2024-09-19T08:39:38.931246+0000","last_unstale":"2024-09-19T08:39:39.957241+0000","last_undegraded":"2024-09-19T08:39:39.957241+0000","last_fullsized":"2024-09-19T08:39:39.957241+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:01.279485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.591072+0000","last_change":"2024-09-19T08:39:38.935680+0000","last_active":"2024-09-19T08:39:40.591072+0000","last_peered":"2024-09-19T08:39:40.591072+0000","last_clean":"2024-09-19T08:39:40.591072+0000","last_became_active":"2024-09-19T08:39:38.929257+0000","last_became_peered":"2024-09-19T08:39:38.929257+0000","last_unstale":"2024-09-19T08:39:40.591072+0000","last_undegraded":"2024-09-19T08:39:40.591072+0000","last_fullsized":"2024-09-19T08:39:40.591072+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:42.809692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.276434+0000","last_change":"2024-09-19T08:39:38.947346+0000","last_active":"2024-09-19T08:39:40.276434+0000","last_peered":"2024-09-19T08:39:40.276434+0000","last_clean":"2024-09-19T08:39:40.276434+0000","last_became_active":"2024-09-19T08:39:38.946660+0000","last_became_peered":"2024-09-19T08:39:38.946660+0000","last_unstale":"2024-09-19T08:39:40.276434+0000","last_undegraded":"2024-09-19T08:39:40.276434+0000","last_fullsized":"2024-09-19T08:39:40.276434+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:19.465116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.275549+0000","last_change":"2024-09-19T08:39:38.947521+0000","last_active":"2024-09-19T08:39:40.275549+0000","last_peered":"2024-09-19T08:39:40.275549+0000","last_clean":"2024-09-19T08:39:40.275549+0000","last_became_active":"2024-09-19T08:39:38.946692+0000","last_became_peered":"2024-09-19T08:39:38.946692+0000","last_unstale":"2024-09-19T08:39:40.275549+0000","last_undegraded":"2024-09-19T08:39:40.275549+0000","last_fullsized":"2024-09-19T08:39:40.275549+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:17:10.144057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.276728+0000","last_change":"2024-09-19T08:39:38.932625+0000","last_active":"2024-09-19T08:39:40.276728+0000","last_peered":"2024-09-19T08:39:40.276728+0000","last_clean":"2024-09-19T08:39:40.276728+0000","last_became_active":"2024-09-19T08:39:38.931787+0000","last_became_peered":"2024-09-19T08:39:38.931787+0000","last_unstale":"2024-09-19T08:39:40.276728+0000","last_undegraded":"2024-09-19T08:39:40.276728+0000","last_fullsized":"2024-09-19T08:39:40.276728+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:31.314991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.897016+0000","last_change":"2024-09-19T08:39:38.932467+0000","last_active":"2024-09-19T08:39:39.897016+0000","last_peered":"2024-09-19T08:39:39.897016+0000","last_clean":"2024-09-19T08:39:39.897016+0000","last_became_active":"2024-09-19T08:39:38.932023+0000","last_became_peered":"2024-09-19T08:39:38.932023+0000","last_unstale":"2024-09-19T08:39:39.897016+0000","last_undegraded":"2024-09-19T08:39:39.897016+0000","last_fullsized":"2024-09-19T08:39:39.897016+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:33:43.610331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.899214+0000","last_change":"2024-09-19T08:39:38.924547+0000","last_active":"2024-09-19T08:39:39.899214+0000","last_peered":"2024-09-19T08:39:39.899214+0000","last_clean":"2024-09-19T08:39:39.899214+0000","last_became_active":"2024-09-19T08:39:38.923802+0000","last_became_peered":"2024-09-19T08:39:38.923802+0000","last_unstale":"2024-09-19T08:39:39.899214+0000","last_undegraded":"2024-09-19T08:39:39.899214+0000","last_fullsized":"2024-09-19T08:39:39.899214+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:22:21.538914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.699092+0000","last_change":"2024-09-19T08:39:38.927516+0000","last_active":"2024-09-19T08:39:40.699092+0000","last_peered":"2024-09-19T08:39:40.699092+0000","last_clean":"2024-09-19T08:39:40.699092+0000","last_became_active":"2024-09-19T08:39:38.926015+0000","last_became_peered":"2024-09-19T08:39:38.926015+0000","last_unstale":"2024-09-19T08:39:40.699092+0000","last_undegraded":"2024-09-19T08:39:40.699092+0000","last_fullsized":"2024-09-19T08:39:40.699092+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:03.770865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.276400+0000","last_change":"2024-09-19T08:39:38.924180+0000","last_active":"2024-09-19T08:39:40.276400+0000","last_peered":"2024-09-19T08:39:40.276400+0000","last_clean":"2024-09-19T08:39:40.276400+0000","last_became_active":"2024-09-19T08:39:38.923352+0000","last_became_peered":"2024-09-19T08:39:38.923352+0000","last_unstale":"2024-09-19T08:39:40.276400+0000","last_undegraded":"2024-09-19T08:39:40.276400+0000","last_fullsized":"2024-09-19T08:39:40.276400+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:11.722134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.898639+0000","last_change":"2024-09-19T08:39:38.931320+0000","last_active":"2024-09-19T08:39:39.898639+0000","last_peered":"2024-09-19T08:39:39.898639+0000","last_clean":"2024-09-19T08:39:39.898639+0000","last_became_active":"2024-09-19T08:39:38.930578+0000","last_became_peered":"2024-09-19T08:39:38.930578+0000","last_unstale":"2024-09-19T08:39:39.898639+0000","last_undegraded":"2024-09-19T08:39:39.898639+0000","last_fullsized":"2024-09-19T08:39:39.898639+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:55:10.927219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.958078+0000","last_change":"2024-09-19T08:39:38.933604+0000","last_active":"2024-09-19T08:39:39.958078+0000","last_peered":"2024-09-19T08:39:39.958078+0000","last_clean":"2024-09-19T08:39:39.958078+0000","last_became_active":"2024-09-19T08:39:38.931912+0000","last_became_peered":"2024-09-19T08:39:38.931912+0000","last_unstale":"2024-09-19T08:39:39.958078+0000","last_undegraded":"2024-09-19T08:39:39.958078+0000","last_fullsized":"2024-09-19T08:39:39.958078+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:31:29.880113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.896854+0000","last_change":"2024-09-19T08:39:38.921905+0000","last_active":"2024-09-19T08:39:39.896854+0000","last_peered":"2024-09-19T08:39:39.896854+0000","last_clean":"2024-09-19T08:39:39.896854+0000","last_became_active":"2024-09-19T08:39:38.920862+0000","last_became_peered":"2024-09-19T08:39:38.920862+0000","last_unstale":"2024-09-19T08:39:39.896854+0000","last_undegraded":"2024-09-19T08:39:39.896854+0000","last_fullsized":"2024-09-19T08:39:39.896854+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:19:23.137418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:41.057439+0000","last_change":"2024-09-19T08:39:38.942801+0000","last_active":"2024-09-19T08:39:41.057439+0000","last_peered":"2024-09-19T08:39:41.057439+0000","last_clean":"2024-09-19T08:39:41.057439+0000","last_became_active":"2024-09-19T08:39:38.942568+0000","last_became_peered":"2024-09-19T08:39:38.942568+0000","last_unstale":"2024-09-19T08:39:41.057439+0000","last_undegraded":"2024-09-19T08:39:41.057439+0000","last_fullsized":"2024-09-19T08:39:41.057439+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:23.048070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"42'4","reported_seq":18,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.957674+0000","last_change":"2024-09-19T08:39:37.937397+0000","last_active":"2024-09-19T08:39:39.957674+0000","last_peered":"2024-09-19T08:39:39.957674+0000","last_clean":"2024-09-19T08:39:39.957674+0000","last_became_active":"2024-09-19T08:39:37.937128+0000","last_became_peered":"2024-09-19T08:39:37.937128+0000","last_unstale":"2024-09-19T08:39:39.957674+0000","last_undegraded":"2024-09-19T08:39:39.957674+0000","last_fullsized":"2024-09-19T08:39:39.957674+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:19.834362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.276453+0000","last_change":"2024-09-19T08:39:38.929983+0000","last_active":"2024-09-19T08:39:40.276453+0000","last_peered":"2024-09-19T08:39:40.276453+0000","last_clean":"2024-09-19T08:39:40.276453+0000","last_became_active":"2024-09-19T08:39:38.929254+0000","last_became_peered":"2024-09-19T08:39:38.929254+0000","last_unstale":"2024-09-19T08:39:40.276453+0000","last_undegraded":"2024-09-19T08:39:40.276453+0000","last_fullsized":"2024-09-19T08:39:40.276453+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:21.872035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.899852+0000","last_change":"2024-09-19T08:39:37.932635+0000","last_active":"2024-09-19T08:39:39.899852+0000","last_peered":"2024-09-19T08:39:39.899852+0000","last_clean":"2024-09-19T08:39:39.899852+0000","last_became_active":"2024-09-19T08:39:37.931841+0000","last_became_peered":"2024-09-19T08:39:37.931841+0000","last_unstale":"2024-09-19T08:39:39.899852+0000","last_undegraded":"2024-09-19T08:39:39.899852+0000","last_fullsized":"2024-09-19T08:39:39.899852+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:42.872698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.699398+0000","last_change":"2024-09-19T08:39:38.940685+0000","last_active":"2024-09-19T08:39:40.699398+0000","last_peered":"2024-09-19T08:39:40.699398+0000","last_clean":"2024-09-19T08:39:40.699398+0000","last_became_active":"2024-09-19T08:39:38.935293+0000","last_became_peered":"2024-09-19T08:39:38.935293+0000","last_unstale":"2024-09-19T08:39:40.699398+0000","last_undegraded":"2024-09-19T08:39:40.699398+0000","last_fullsized":"2024-09-19T08:39:40.699398+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:42.809692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"42'2","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:41.057588+0000","last_change":"2024-09-19T08:39:37.917790+0000","last_active":"2024-09-19T08:39:41.057588+0000","last_peered":"2024-09-19T08:39:41.057588+0000","last_clean":"2024-09-19T08:39:41.057588+0000","last_became_active":"2024-09-19T08:39:37.917479+0000","last_became_peered":"2024-09-19T08:39:37.917479+0000","last_unstale":"2024-09-19T08:39:41.057588+0000","last_undegraded":"2024-09-19T08:39:41.057588+0000","last_fullsized":"2024-09-19T08:39:41.057588+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:30:08.860289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:41.057573+0000","last_change":"2024-09-19T08:39:38.920238+0000","last_active":"2024-09-19T08:39:41.057573+0000","last_peered":"2024-09-19T08:39:41.057573+0000","last_clean":"2024-09-19T08:39:41.057573+0000","last_became_active":"2024-09-19T08:39:38.919486+0000","last_became_peered":"2024-09-19T08:39:38.919486+0000","last_unstale":"2024-09-19T08:39:41.057573+0000","last_undegraded":"2024-09-19T08:39:41.057573+0000","last_fullsized":"2024-09-19T08:39:41.057573+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:29.744836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"42'2","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:41.060600+0000","last_change":"2024-09-19T08:39:37.927771+0000","last_active":"2024-09-19T08:39:41.060600+0000","last_peered":"2024-09-19T08:39:41.060600+0000","last_clean":"2024-09-19T08:39:41.060600+0000","last_became_active":"2024-09-19T08:39:37.926946+0000","last_became_peered":"2024-09-19T08:39:37.926946+0000","last_unstale":"2024-09-19T08:39:41.060600+0000","last_undegraded":"2024-09-19T08:39:41.060600+0000","last_fullsized":"2024-09-19T08:39:41.060600+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:39.361088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.898697+0000","last_change":"2024-09-19T08:39:38.918556+0000","last_active":"2024-09-19T08:39:39.898697+0000","last_peered":"2024-09-19T08:39:39.898697+0000","last_clean":"2024-09-19T08:39:39.898697+0000","last_became_active":"2024-09-19T08:39:38.917830+0000","last_became_peered":"2024-09-19T08:39:38.917830+0000","last_unstale":"2024-09-19T08:39:39.898697+0000","last_undegraded":"2024-09-19T08:39:39.898697+0000","last_fullsized":"2024-09-19T08:39:39.898697+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:08:18.820654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.957345+0000","last_change":"2024-09-19T08:39:37.913524+0000","last_active":"2024-09-19T08:39:39.957345+0000","last_peered":"2024-09-19T08:39:39.957345+0000","last_clean":"2024-09-19T08:39:39.957345+0000","last_became_active":"2024-09-19T08:39:37.912853+0000","last_became_peered":"2024-09-19T08:39:37.912853+0000","last_unstale":"2024-09-19T08:39:39.957345+0000","last_undegraded":"2024-09-19T08:39:39.957345+0000","last_fullsized":"2024-09-19T08:39:39.957345+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:32.036009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.590701+0000","last_change":"2024-09-19T08:39:38.929719+0000","last_active":"2024-09-19T08:39:40.590701+0000","last_peered":"2024-09-19T08:39:40.590701+0000","last_clean":"2024-09-19T08:39:40.590701+0000","last_became_active":"2024-09-19T08:39:38.929158+0000","last_became_peered":"2024-09-19T08:39:38.929158+0000","last_unstale":"2024-09-19T08:39:40.590701+0000","last_undegraded":"2024-09-19T08:39:40.590701+0000","last_fullsized":"2024-09-19T08:39:40.590701+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:29.732835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.898884+0000","last_change":"2024-09-19T08:39:37.929415+0000","last_active":"2024-09-19T08:39:39.898884+0000","last_peered":"2024-09-19T08:39:39.898884+0000","last_clean":"2024-09-19T08:39:39.898884+0000","last_became_active":"2024-09-19T08:39:37.928610+0000","last_became_peered":"2024-09-19T08:39:37.928610+0000","last_unstale":"2024-09-19T08:39:39.898884+0000","last_undegraded":"2024-09-19T08:39:39.898884+0000","last_fullsized":"2024-09-19T08:39:39.898884+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:36.032108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.590331+0000","last_change":"2024-09-19T08:39:38.928185+0000","last_active":"2024-09-19T08:39:40.590331+0000","last_peered":"2024-09-19T08:39:40.590331+0000","last_clean":"2024-09-19T08:39:40.590331+0000","last_became_active":"2024-09-19T08:39:38.924812+0000","last_became_peered":"2024-09-19T08:39:38.924812+0000","last_unstale":"2024-09-19T08:39:40.590331+0000","last_undegraded":"2024-09-19T08:39:40.590331+0000","last_fullsized":"2024-09-19T08:39:40.590331+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:08:18.820654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.698498+0000","last_change":"2024-09-19T08:39:37.921882+0000","last_active":"2024-09-19T08:39:40.698498+0000","last_peered":"2024-09-19T08:39:40.698498+0000","last_clean":"2024-09-19T08:39:40.698498+0000","last_became_active":"2024-09-19T08:39:37.921061+0000","last_became_peered":"2024-09-19T08:39:37.921061+0000","last_unstale":"2024-09-19T08:39:40.698498+0000","last_undegraded":"2024-09-19T08:39:40.698498+0000","last_fullsized":"2024-09-19T08:39:40.698498+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:49:13.065050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.275438+0000","last_change":"2024-09-19T08:39:38.948118+0000","last_active":"2024-09-19T08:39:40.275438+0000","last_peered":"2024-09-19T08:39:40.275438+0000","last_clean":"2024-09-19T08:39:40.275438+0000","last_became_active":"2024-09-19T08:39:38.947360+0000","last_became_peered":"2024-09-19T08:39:38.947360+0000","last_unstale":"2024-09-19T08:39:40.275438+0000","last_undegraded":"2024-09-19T08:39:40.275438+0000","last_fullsized":"2024-09-19T08:39:40.275438+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:42.793245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.590326+0000","last_change":"2024-09-19T08:39:37.914508+0000","last_active":"2024-09-19T08:39:40.590326+0000","last_peered":"2024-09-19T08:39:40.590326+0000","last_clean":"2024-09-19T08:39:40.590326+0000","last_became_active":"2024-09-19T08:39:37.914186+0000","last_became_peered":"2024-09-19T08:39:37.914186+0000","last_unstale":"2024-09-19T08:39:40.590326+0000","last_undegraded":"2024-09-19T08:39:40.590326+0000","last_fullsized":"2024-09-19T08:39:40.590326+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:48:49.023614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.899775+0000","last_change":"2024-09-19T08:39:38.939531+0000","last_active":"2024-09-19T08:39:39.899775+0000","last_peered":"2024-09-19T08:39:39.899775+0000","last_clean":"2024-09-19T08:39:39.899775+0000","last_became_active":"2024-09-19T08:39:38.938231+0000","last_became_peered":"2024-09-19T08:39:38.938231+0000","last_unstale":"2024-09-19T08:39:39.899775+0000","last_undegraded":"2024-09-19T08:39:39.899775+0000","last_fullsized":"2024-09-19T08:39:39.899775+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:37:14.987023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.899519+0000","last_change":"2024-09-19T08:39:37.930095+0000","last_active":"2024-09-19T08:39:39.899519+0000","last_peered":"2024-09-19T08:39:39.899519+0000","last_clean":"2024-09-19T08:39:39.899519+0000","last_became_active":"2024-09-19T08:39:37.921804+0000","last_became_peered":"2024-09-19T08:39:37.921804+0000","last_unstale":"2024-09-19T08:39:39.899519+0000","last_undegraded":"2024-09-19T08:39:39.899519+0000","last_fullsized":"2024-09-19T08:39:39.899519+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:30:08.860289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:41.061365+0000","last_change":"2024-09-19T08:39:38.941102+0000","last_active":"2024-09-19T08:39:41.061365+0000","last_peered":"2024-09-19T08:39:41.061365+0000","last_clean":"2024-09-19T08:39:41.061365+0000","last_became_active":"2024-09-19T08:39:38.937500+0000","last_became_peered":"2024-09-19T08:39:38.937500+0000","last_unstale":"2024-09-19T08:39:41.061365+0000","last_undegraded":"2024-09-19T08:39:41.061365+0000","last_fullsized":"2024-09-19T08:39:41.061365+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:32:29.190529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.591865+0000","last_change":"2024-09-19T08:39:37.912838+0000","last_active":"2024-09-19T08:39:40.591865+0000","last_peered":"2024-09-19T08:39:40.591865+0000","last_clean":"2024-09-19T08:39:40.591865+0000","last_became_active":"2024-09-19T08:39:37.912442+0000","last_became_peered":"2024-09-19T08:39:37.912442+0000","last_unstale":"2024-09-19T08:39:40.591865+0000","last_undegraded":"2024-09-19T08:39:40.591865+0000","last_fullsized":"2024-09-19T08:39:40.591865+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:46.135290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.957630+0000","last_change":"2024-09-19T08:39:38.931347+0000","last_active":"2024-09-19T08:39:39.957630+0000","last_peered":"2024-09-19T08:39:39.957630+0000","last_clean":"2024-09-19T08:39:39.957630+0000","last_became_active":"2024-09-19T08:39:38.930982+0000","last_became_peered":"2024-09-19T08:39:38.930982+0000","last_unstale":"2024-09-19T08:39:39.957630+0000","last_undegraded":"2024-09-19T08:39:39.957630+0000","last_fullsized":"2024-09-19T08:39:39.957630+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:36.448393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"42'3","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:41.061111+0000","last_change":"2024-09-19T08:39:37.936425+0000","last_active":"2024-09-19T08:39:41.061111+0000","last_peered":"2024-09-19T08:39:41.061111+0000","last_clean":"2024-09-19T08:39:41.061111+0000","last_became_active":"2024-09-19T08:39:37.935418+0000","last_became_peered":"2024-09-19T08:39:37.935418+0000","last_unstale":"2024-09-19T08:39:41.061111+0000","last_undegraded":"2024-09-19T08:39:41.061111+0000","last_fullsized":"2024-09-19T08:39:41.061111+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:05.453778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:41.060849+0000","last_change":"2024-09-19T08:39:38.942176+0000","last_active":"2024-09-19T08:39:41.060849+0000","last_peered":"2024-09-19T08:39:41.060849+0000","last_clean":"2024-09-19T08:39:41.060849+0000","last_became_active":"2024-09-19T08:39:38.939625+0000","last_became_peered":"2024-09-19T08:39:38.939625+0000","last_unstale":"2024-09-19T08:39:41.060849+0000","last_undegraded":"2024-09-19T08:39:41.060849+0000","last_fullsized":"2024-09-19T08:39:41.060849+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:23:42.809692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"42'6","reported_seq":20,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.897147+0000","last_change":"2024-09-19T08:39:37.921227+0000","last_active":"2024-09-19T08:39:39.897147+0000","last_peered":"2024-09-19T08:39:39.897147+0000","last_clean":"2024-09-19T08:39:39.897147+0000","last_became_active":"2024-09-19T08:39:37.920773+0000","last_became_peered":"2024-09-19T08:39:37.920773+0000","last_unstale":"2024-09-19T08:39:39.897147+0000","last_undegraded":"2024-09-19T08:39:39.897147+0000","last_fullsized":"2024-09-19T08:39:39.897147+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:24:07.528992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.275773+0000","last_change":"2024-09-19T08:39:38.939598+0000","last_active":"2024-09-19T08:39:40.275773+0000","last_peered":"2024-09-19T08:39:40.275773+0000","last_clean":"2024-09-19T08:39:40.275773+0000","last_became_active":"2024-09-19T08:39:38.938870+0000","last_became_peered":"2024-09-19T08:39:38.938870+0000","last_unstale":"2024-09-19T08:39:40.275773+0000","last_undegraded":"2024-09-19T08:39:40.275773+0000","last_fullsized":"2024-09-19T08:39:40.275773+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:30:00.712901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"42'2","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.957446+0000","last_change":"2024-09-19T08:39:37.905231+0000","last_active":"2024-09-19T08:39:39.957446+0000","last_peered":"2024-09-19T08:39:39.957446+0000","last_clean":"2024-09-19T08:39:39.957446+0000","last_became_active":"2024-09-19T08:39:37.904660+0000","last_became_peered":"2024-09-19T08:39:37.904660+0000","last_unstale":"2024-09-19T08:39:39.957446+0000","last_undegraded":"2024-09-19T08:39:39.957446+0000","last_fullsized":"2024-09-19T08:39:39.957446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:12:31.651879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.590324+0000","last_change":"2024-09-19T08:39:38.936213+0000","last_active":"2024-09-19T08:39:40.590324+0000","last_peered":"2024-09-19T08:39:40.590324+0000","last_clean":"2024-09-19T08:39:40.590324+0000","last_became_active":"2024-09-19T08:39:38.935856+0000","last_became_peered":"2024-09-19T08:39:38.935856+0000","last_unstale":"2024-09-19T08:39:40.590324+0000","last_undegraded":"2024-09-19T08:39:40.590324+0000","last_fullsized":"2024-09-19T08:39:40.590324+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:00:09.742802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:41.060543+0000","last_change":"2024-09-19T08:39:37.914414+0000","last_active":"2024-09-19T08:39:41.060543+0000","last_peered":"2024-09-19T08:39:41.060543+0000","last_clean":"2024-09-19T08:39:41.060543+0000","last_became_active":"2024-09-19T08:39:37.913576+0000","last_became_peered":"2024-09-19T08:39:37.913576+0000","last_unstale":"2024-09-19T08:39:41.060543+0000","last_undegraded":"2024-09-19T08:39:41.060543+0000","last_fullsized":"2024-09-19T08:39:41.060543+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:28.741298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.957112+0000","last_change":"2024-09-19T08:39:38.920776+0000","last_active":"2024-09-19T08:39:39.957112+0000","last_peered":"2024-09-19T08:39:39.957112+0000","last_clean":"2024-09-19T08:39:39.957112+0000","last_became_active":"2024-09-19T08:39:38.919807+0000","last_became_peered":"2024-09-19T08:39:38.919807+0000","last_unstale":"2024-09-19T08:39:39.957112+0000","last_undegraded":"2024-09-19T08:39:39.957112+0000","last_fullsized":"2024-09-19T08:39:39.957112+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:35.525085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.957610+0000","last_change":"2024-09-19T08:39:37.910119+0000","last_active":"2024-09-19T08:39:39.957610+0000","last_peered":"2024-09-19T08:39:39.957610+0000","last_clean":"2024-09-19T08:39:39.957610+0000","last_became_active":"2024-09-19T08:39:37.909646+0000","last_became_peered":"2024-09-19T08:39:37.909646+0000","last_unstale":"2024-09-19T08:39:39.957610+0000","last_undegraded":"2024-09-19T08:39:39.957610+0000","last_fullsized":"2024-09-19T08:39:39.957610+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:17:02.556780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:41.060905+0000","last_change":"2024-09-19T08:39:38.937339+0000","last_active":"2024-09-19T08:39:41.060905+0000","last_peered":"2024-09-19T08:39:41.060905+0000","last_clean":"2024-09-19T08:39:41.060905+0000","last_became_active":"2024-09-19T08:39:38.936404+0000","last_became_peered":"2024-09-19T08:39:38.936404+0000","last_unstale":"2024-09-19T08:39:41.060905+0000","last_undegraded":"2024-09-19T08:39:41.060905+0000","last_fullsized":"2024-09-19T08:39:41.060905+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:57.785025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"42'8","reported_seq":22,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.590615+0000","last_change":"2024-09-19T08:39:37.918124+0000","last_active":"2024-09-19T08:39:40.590615+0000","last_peered":"2024-09-19T08:39:40.590615+0000","last_clean":"2024-09-19T08:39:40.590615+0000","last_became_active":"2024-09-19T08:39:37.917521+0000","last_became_peered":"2024-09-19T08:39:37.917521+0000","last_unstale":"2024-09-19T08:39:40.590615+0000","last_undegraded":"2024-09-19T08:39:40.590615+0000","last_fullsized":"2024-09-19T08:39:40.590615+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:38:39.741349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.897479+0000","last_change":"2024-09-19T08:39:38.926671+0000","last_active":"2024-09-19T08:39:39.897479+0000","last_peered":"2024-09-19T08:39:39.897479+0000","last_clean":"2024-09-19T08:39:39.897479+0000","last_became_active":"2024-09-19T08:39:38.925343+0000","last_became_peered":"2024-09-19T08:39:38.925343+0000","last_unstale":"2024-09-19T08:39:39.897479+0000","last_undegraded":"2024-09-19T08:39:39.897479+0000","last_fullsized":"2024-09-19T08:39:39.897479+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:48:50.027153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.897407+0000","last_change":"2024-09-19T08:39:37.911789+0000","last_active":"2024-09-19T08:39:39.897407+0000","last_peered":"2024-09-19T08:39:39.897407+0000","last_clean":"2024-09-19T08:39:39.897407+0000","last_became_active":"2024-09-19T08:39:37.911305+0000","last_became_peered":"2024-09-19T08:39:37.911305+0000","last_unstale":"2024-09-19T08:39:39.897407+0000","last_undegraded":"2024-09-19T08:39:39.897407+0000","last_fullsized":"2024-09-19T08:39:39.897407+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:13.592173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.591441+0000","last_change":"2024-09-19T08:39:38.935012+0000","last_active":"2024-09-19T08:39:40.591441+0000","last_peered":"2024-09-19T08:39:40.591441+0000","last_clean":"2024-09-19T08:39:40.591441+0000","last_became_active":"2024-09-19T08:39:38.934473+0000","last_became_peered":"2024-09-19T08:39:38.934473+0000","last_unstale":"2024-09-19T08:39:40.591441+0000","last_undegraded":"2024-09-19T08:39:40.591441+0000","last_fullsized":"2024-09-19T08:39:40.591441+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:03.770865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"42'2","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.900030+0000","last_change":"2024-09-19T08:39:37.923082+0000","last_active":"2024-09-19T08:39:39.900030+0000","last_peered":"2024-09-19T08:39:39.900030+0000","last_clean":"2024-09-19T08:39:39.900030+0000","last_became_active":"2024-09-19T08:39:37.922381+0000","last_became_peered":"2024-09-19T08:39:37.922381+0000","last_unstale":"2024-09-19T08:39:39.900030+0000","last_undegraded":"2024-09-19T08:39:39.900030+0000","last_fullsized":"2024-09-19T08:39:39.900030+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:08:43.556600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.276444+0000","last_change":"2024-09-19T08:39:38.937247+0000","last_active":"2024-09-19T08:39:40.276444+0000","last_peered":"2024-09-19T08:39:40.276444+0000","last_clean":"2024-09-19T08:39:40.276444+0000","last_became_active":"2024-09-19T08:39:38.936501+0000","last_became_peered":"2024-09-19T08:39:38.936501+0000","last_unstale":"2024-09-19T08:39:40.276444+0000","last_undegraded":"2024-09-19T08:39:40.276444+0000","last_fullsized":"2024-09-19T08:39:40.276444+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:55.400643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"42'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.699170+0000","last_change":"2024-09-19T08:39:37.917129+0000","last_active":"2024-09-19T08:39:40.699170+0000","last_peered":"2024-09-19T08:39:40.699170+0000","last_clean":"2024-09-19T08:39:40.699170+0000","last_became_active":"2024-09-19T08:39:37.916360+0000","last_became_peered":"2024-09-19T08:39:37.916360+0000","last_unstale":"2024-09-19T08:39:40.699170+0000","last_undegraded":"2024-09-19T08:39:40.699170+0000","last_fullsized":"2024-09-19T08:39:40.699170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:39.361088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.276433+0000","last_change":"2024-09-19T08:39:38.932462+0000","last_active":"2024-09-19T08:39:40.276433+0000","last_peered":"2024-09-19T08:39:40.276433+0000","last_clean":"2024-09-19T08:39:40.276433+0000","last_became_active":"2024-09-19T08:39:38.931552+0000","last_became_peered":"2024-09-19T08:39:38.931552+0000","last_unstale":"2024-09-19T08:39:40.276433+0000","last_undegraded":"2024-09-19T08:39:40.276433+0000","last_fullsized":"2024-09-19T08:39:40.276433+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:50:32.514250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.957547+0000","last_change":"2024-09-19T08:39:37.932493+0000","last_active":"2024-09-19T08:39:39.957547+0000","last_peered":"2024-09-19T08:39:39.957547+0000","last_clean":"2024-09-19T08:39:39.957547+0000","last_became_active":"2024-09-19T08:39:37.932195+0000","last_became_peered":"2024-09-19T08:39:37.932195+0000","last_unstale":"2024-09-19T08:39:39.957547+0000","last_undegraded":"2024-09-19T08:39:39.957547+0000","last_fullsized":"2024-09-19T08:39:39.957547+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:46:44.370790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.957912+0000","last_change":"2024-09-19T08:39:38.932384+0000","last_active":"2024-09-19T08:39:39.957912+0000","last_peered":"2024-09-19T08:39:39.957912+0000","last_clean":"2024-09-19T08:39:39.957912+0000","last_became_active":"2024-09-19T08:39:38.932029+0000","last_became_peered":"2024-09-19T08:39:38.932029+0000","last_unstale":"2024-09-19T08:39:39.957912+0000","last_undegraded":"2024-09-19T08:39:39.957912+0000","last_fullsized":"2024-09-19T08:39:39.957912+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:17:30.846841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"42'2","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.957587+0000","last_change":"2024-09-19T08:39:37.913727+0000","last_active":"2024-09-19T08:39:39.957587+0000","last_peered":"2024-09-19T08:39:39.957587+0000","last_clean":"2024-09-19T08:39:39.957587+0000","last_became_active":"2024-09-19T08:39:37.913224+0000","last_became_peered":"2024-09-19T08:39:37.913224+0000","last_unstale":"2024-09-19T08:39:39.957587+0000","last_undegraded":"2024-09-19T08:39:39.957587+0000","last_fullsized":"2024-09-19T08:39:39.957587+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:18.771121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:41.057397+0000","last_change":"2024-09-19T08:39:38.923972+0000","last_active":"2024-09-19T08:39:41.057397+0000","last_peered":"2024-09-19T08:39:41.057397+0000","last_clean":"2024-09-19T08:39:41.057397+0000","last_became_active":"2024-09-19T08:39:38.923101+0000","last_became_peered":"2024-09-19T08:39:38.923101+0000","last_unstale":"2024-09-19T08:39:41.057397+0000","last_undegraded":"2024-09-19T08:39:41.057397+0000","last_fullsized":"2024-09-19T08:39:41.057397+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:33.862515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.698606+0000","last_change":"2024-09-19T08:39:37.934259+0000","last_active":"2024-09-19T08:39:40.698606+0000","last_peered":"2024-09-19T08:39:40.698606+0000","last_clean":"2024-09-19T08:39:40.698606+0000","last_became_active":"2024-09-19T08:39:37.933506+0000","last_became_peered":"2024-09-19T08:39:37.933506+0000","last_unstale":"2024-09-19T08:39:40.698606+0000","last_undegraded":"2024-09-19T08:39:40.698606+0000","last_fullsized":"2024-09-19T08:39:40.698606+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:22.044532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.897284+0000","last_change":"2024-09-19T08:39:38.944450+0000","last_active":"2024-09-19T08:39:39.897284+0000","last_peered":"2024-09-19T08:39:39.897284+0000","last_clean":"2024-09-19T08:39:39.897284+0000","last_became_active":"2024-09-19T08:39:38.944108+0000","last_became_peered":"2024-09-19T08:39:38.944108+0000","last_unstale":"2024-09-19T08:39:39.897284+0000","last_undegraded":"2024-09-19T08:39:39.897284+0000","last_fullsized":"2024-09-19T08:39:39.897284+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:20.681273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.590292+0000","last_change":"2024-09-19T08:39:37.922726+0000","last_active":"2024-09-19T08:39:40.590292+0000","last_peered":"2024-09-19T08:39:40.590292+0000","last_clean":"2024-09-19T08:39:40.590292+0000","last_became_active":"2024-09-19T08:39:37.922312+0000","last_became_peered":"2024-09-19T08:39:37.922312+0000","last_unstale":"2024-09-19T08:39:40.590292+0000","last_undegraded":"2024-09-19T08:39:40.590292+0000","last_fullsized":"2024-09-19T08:39:40.590292+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:05:28.346760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.591078+0000","last_change":"2024-09-19T08:39:38.920148+0000","last_active":"2024-09-19T08:39:40.591078+0000","last_peered":"2024-09-19T08:39:40.591078+0000","last_clean":"2024-09-19T08:39:40.591078+0000","last_became_active":"2024-09-19T08:39:38.919337+0000","last_became_peered":"2024-09-19T08:39:38.919337+0000","last_unstale":"2024-09-19T08:39:40.591078+0000","last_undegraded":"2024-09-19T08:39:40.591078+0000","last_fullsized":"2024-09-19T08:39:40.591078+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:06:12.303817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:41.060146+0000","last_change":"2024-09-19T08:39:37.915040+0000","last_active":"2024-09-19T08:39:41.060146+0000","last_peered":"2024-09-19T08:39:41.060146+0000","last_clean":"2024-09-19T08:39:41.060146+0000","last_became_active":"2024-09-19T08:39:37.914232+0000","last_became_peered":"2024-09-19T08:39:37.914232+0000","last_unstale":"2024-09-19T08:39:41.060146+0000","last_undegraded":"2024-09-19T08:39:41.060146+0000","last_fullsized":"2024-09-19T08:39:41.060146+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:49:13.065050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.276434+0000","last_change":"2024-09-19T08:39:38.940179+0000","last_active":"2024-09-19T08:39:40.276434+0000","last_peered":"2024-09-19T08:39:40.276434+0000","last_clean":"2024-09-19T08:39:40.276434+0000","last_became_active":"2024-09-19T08:39:38.939475+0000","last_became_peered":"2024-09-19T08:39:38.939475+0000","last_unstale":"2024-09-19T08:39:40.276434+0000","last_undegraded":"2024-09-19T08:39:40.276434+0000","last_fullsized":"2024-09-19T08:39:40.276434+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:36:50.598742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"43'1","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.957521+0000","last_change":"2024-09-19T08:39:37.918815+0000","last_active":"2024-09-19T08:39:39.957521+0000","last_peered":"2024-09-19T08:39:39.957521+0000","last_clean":"2024-09-19T08:39:39.957521+0000","last_became_active":"2024-09-19T08:39:37.918481+0000","last_became_peered":"2024-09-19T08:39:37.918481+0000","last_unstale":"2024-09-19T08:39:39.957521+0000","last_undegraded":"2024-09-19T08:39:39.957521+0000","last_fullsized":"2024-09-19T08:39:39.957521+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:56:32.513917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.590797+0000","last_change":"2024-09-19T08:39:38.929785+0000","last_active":"2024-09-19T08:39:40.590797+0000","last_peered":"2024-09-19T08:39:40.590797+0000","last_clean":"2024-09-19T08:39:40.590797+0000","last_became_active":"2024-09-19T08:39:38.929074+0000","last_became_peered":"2024-09-19T08:39:38.929074+0000","last_unstale":"2024-09-19T08:39:40.590797+0000","last_undegraded":"2024-09-19T08:39:40.590797+0000","last_fullsized":"2024-09-19T08:39:40.590797+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:22:21.538914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:41.061284+0000","last_change":"2024-09-19T08:39:37.936655+0000","last_active":"2024-09-19T08:39:41.061284+0000","last_peered":"2024-09-19T08:39:41.061284+0000","last_clean":"2024-09-19T08:39:41.061284+0000","last_became_active":"2024-09-19T08:39:37.936112+0000","last_became_peered":"2024-09-19T08:39:37.936112+0000","last_unstale":"2024-09-19T08:39:41.061284+0000","last_undegraded":"2024-09-19T08:39:41.061284+0000","last_fullsized":"2024-09-19T08:39:41.061284+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:33:43.714330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:41.061027+0000","last_change":"2024-09-19T08:39:38.942782+0000","last_active":"2024-09-19T08:39:41.061027+0000","last_peered":"2024-09-19T08:39:41.061027+0000","last_clean":"2024-09-19T08:39:41.061027+0000","last_became_active":"2024-09-19T08:39:38.939689+0000","last_became_peered":"2024-09-19T08:39:38.939689+0000","last_unstale":"2024-09-19T08:39:41.061027+0000","last_undegraded":"2024-09-19T08:39:41.061027+0000","last_fullsized":"2024-09-19T08:39:41.061027+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:03.770865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:41.057233+0000","last_change":"2024-09-19T08:39:37.924237+0000","last_active":"2024-09-19T08:39:41.057233+0000","last_peered":"2024-09-19T08:39:41.057233+0000","last_clean":"2024-09-19T08:39:41.057233+0000","last_became_active":"2024-09-19T08:39:37.923832+0000","last_became_peered":"2024-09-19T08:39:37.923832+0000","last_unstale":"2024-09-19T08:39:41.057233+0000","last_undegraded":"2024-09-19T08:39:41.057233+0000","last_fullsized":"2024-09-19T08:39:41.057233+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:14:31.270319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.957703+0000","last_change":"2024-09-19T08:39:38.931776+0000","last_active":"2024-09-19T08:39:39.957703+0000","last_peered":"2024-09-19T08:39:39.957703+0000","last_clean":"2024-09-19T08:39:39.957703+0000","last_became_active":"2024-09-19T08:39:38.931055+0000","last_became_peered":"2024-09-19T08:39:38.931055+0000","last_unstale":"2024-09-19T08:39:39.957703+0000","last_undegraded":"2024-09-19T08:39:39.957703+0000","last_fullsized":"2024-09-19T08:39:39.957703+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:14.144870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:41.060731+0000","last_change":"2024-09-19T08:39:37.934015+0000","last_active":"2024-09-19T08:39:41.060731+0000","last_peered":"2024-09-19T08:39:41.060731+0000","last_clean":"2024-09-19T08:39:41.060731+0000","last_became_active":"2024-09-19T08:39:37.933256+0000","last_became_peered":"2024-09-19T08:39:37.933256+0000","last_unstale":"2024-09-19T08:39:41.060731+0000","last_undegraded":"2024-09-19T08:39:41.060731+0000","last_fullsized":"2024-09-19T08:39:41.060731+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:55:09.923681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.957357+0000","last_change":"2024-09-19T08:39:38.931817+0000","last_active":"2024-09-19T08:39:39.957357+0000","last_peered":"2024-09-19T08:39:39.957357+0000","last_clean":"2024-09-19T08:39:39.957357+0000","last_became_active":"2024-09-19T08:39:38.931021+0000","last_became_peered":"2024-09-19T08:39:38.931021+0000","last_unstale":"2024-09-19T08:39:39.957357+0000","last_undegraded":"2024-09-19T08:39:39.957357+0000","last_fullsized":"2024-09-19T08:39:39.957357+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:53:30.156065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.898974+0000","last_change":"2024-09-19T08:39:37.920347+0000","last_active":"2024-09-19T08:39:39.898974+0000","last_peered":"2024-09-19T08:39:39.898974+0000","last_clean":"2024-09-19T08:39:39.898974+0000","last_became_active":"2024-09-19T08:39:37.919634+0000","last_became_peered":"2024-09-19T08:39:37.919634+0000","last_unstale":"2024-09-19T08:39:39.898974+0000","last_undegraded":"2024-09-19T08:39:39.898974+0000","last_fullsized":"2024-09-19T08:39:39.898974+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:14:31.270319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.698582+0000","last_change":"2024-09-19T08:39:38.927845+0000","last_active":"2024-09-19T08:39:40.698582+0000","last_peered":"2024-09-19T08:39:40.698582+0000","last_clean":"2024-09-19T08:39:40.698582+0000","last_became_active":"2024-09-19T08:39:38.926989+0000","last_became_peered":"2024-09-19T08:39:38.926989+0000","last_unstale":"2024-09-19T08:39:40.698582+0000","last_undegraded":"2024-09-19T08:39:40.698582+0000","last_fullsized":"2024-09-19T08:39:40.698582+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:37.082611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:41.057238+0000","last_change":"2024-09-19T08:39:37.910329+0000","last_active":"2024-09-19T08:39:41.057238+0000","last_peered":"2024-09-19T08:39:41.057238+0000","last_clean":"2024-09-19T08:39:41.057238+0000","last_became_active":"2024-09-19T08:39:37.909966+0000","last_became_peered":"2024-09-19T08:39:37.909966+0000","last_unstale":"2024-09-19T08:39:41.057238+0000","last_undegraded":"2024-09-19T08:39:41.057238+0000","last_fullsized":"2024-09-19T08:39:41.057238+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:36.032108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:39.898666+0000","last_change":"2024-09-19T08:39:38.936998+0000","last_active":"2024-09-19T08:39:39.898666+0000","last_peered":"2024-09-19T08:39:39.898666+0000","last_clean":"2024-09-19T08:39:39.898666+0000","last_became_active":"2024-09-19T08:39:38.935884+0000","last_became_peered":"2024-09-19T08:39:38.935884+0000","last_unstale":"2024-09-19T08:39:39.898666+0000","last_undegraded":"2024-09-19T08:39:39.898666+0000","last_fullsized":"2024-09-19T08:39:39.898666+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:00:09.742802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.590678+0000","last_change":"2024-09-19T08:39:37.907408+0000","last_active":"2024-09-19T08:39:40.590678+0000","last_peered":"2024-09-19T08:39:40.590678+0000","last_clean":"2024-09-19T08:39:40.590678+0000","last_became_active":"2024-09-19T08:39:37.906928+0000","last_became_peered":"2024-09-19T08:39:37.906928+0000","last_unstale":"2024-09-19T08:39:40.590678+0000","last_undegraded":"2024-09-19T08:39:40.590678+0000","last_fullsized":"2024-09-19T08:39:40.590678+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:08.285216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.591741+0000","last_change":"2024-09-19T08:39:38.940058+0000","last_active":"2024-09-19T08:39:40.591741+0000","last_peered":"2024-09-19T08:39:40.591741+0000","last_clean":"2024-09-19T08:39:40.591741+0000","last_became_active":"2024-09-19T08:39:38.939663+0000","last_became_peered":"2024-09-19T08:39:38.939663+0000","last_unstale":"2024-09-19T08:39:40.591741+0000","last_undegraded":"2024-09-19T08:39:40.591741+0000","last_fullsized":"2024-09-19T08:39:40.591741+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:37.883250+0000","last_clean_scrub_stamp":"2024-09-19T08:39:37.883250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:32:29.190529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"42'2","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:40.699435+0000","last_change":"2024-09-19T08:39:37.922845+0000","last_active":"2024-09-19T08:39:40.699435+0000","last_peered":"2024-09-19T08:39:40.699435+0000","last_clean":"2024-09-19T08:39:40.699435+0000","last_became_active":"2024-09-19T08:39:37.922153+0000","last_became_peered":"2024-09-19T08:39:37.922153+0000","last_unstale":"2024-09-19T08:39:40.699435+0000","last_undegraded":"2024-09-19T08:39:40.699435+0000","last_fullsized":"2024-09-19T08:39:40.699435+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:39:36.879712+0000","last_clean_scrub_stamp":"2024-09-19T08:39:36.879712+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:49:28.741298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":206,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-19T08:39:41.060271+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:39:41.060271+0000","last_peered":"2024-09-19T08:39:41.060271+0000","last_clean":"2024-09-19T08:39:41.060271+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:39:41.060271+0000","last_undegraded":"2024-09-19T08:39:41.060271+0000","last_fullsized":"2024-09-19T08:39:41.060271+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1093632,"data_stored":789309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509464,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":499712,"data_stored":198921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509464,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":507904,"data_stored":200463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509461,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":512000,"data_stored":200485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509463,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":499712,"data_stored":198921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509461,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28096,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343744,"statfs":{"total":96636764160,"available":96607993856,"internally_reserved":0,"allocated":1093632,"data_stored":789253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27520,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":503808,"data_stored":199523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542166,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31684,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340156,"statfs":{"total":96636764160,"available":96604319744,"internally_reserved":0,"allocated":507904,"data_stored":199489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:39:47.806 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T08:39:47.806 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-19T08:39:47.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.943+0000 7fa2ea080640 1 Processor -- start 2024-09-19T08:39:47.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.943+0000 7fa2ea080640 1 -- start start 2024-09-19T08:39:47.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.943+0000 7fa2ea080640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2e415e640 0x7fa2e40d2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:47.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.943+0000 7fa2ea080640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa2e406c3a0 0x7fa2e40b7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:47.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.943+0000 7fa2ea080640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2e40d0270 con 0x7fa2e40cfcf0 2024-09-19T08:39:47.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.943+0000 7fa2ea080640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fa2e40b75f0 con 0x7fa2e406c3a0 2024-09-19T08:39:47.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.943+0000 7fa2ea080640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa2e415ea20 con 0x7fa2e415e640 2024-09-19T08:39:47.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.947+0000 7fa2dbfff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fa2e40cfcf0 0x7fa2e406bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:35822/0 (socket says 172.21.15.98:35822) 2024-09-19T08:39:47.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.947+0000 7fa2dbfff640 1 -- 172.21.15.98:0/382130743 learned_addr learned my addr 172.21.15.98:0/382130743 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:47.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.947+0000 7fa2e3fff640 1 --2- 172.21.15.98:0/382130743 >> v2:172.21.15.139:3300/0 conn(0x7fa2e406c3a0 0x7fa2e40b7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:47.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.947+0000 7fa2e37fe640 1 --2- 172.21.15.98:0/382130743 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2e415e640 0x7fa2e40d2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:47.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.947+0000 7fa2e2ffd640 1 -- 172.21.15.98:0/382130743 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 374190470 0 0) 0x7fa2e40d0270 con 0x7fa2e40cfcf0 2024-09-19T08:39:47.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.947+0000 7fa2e2ffd640 1 -- 172.21.15.98:0/382130743 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2d0003530 con 0x7fa2e40cfcf0 2024-09-19T08:39:47.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.947+0000 7fa2e2ffd640 1 -- 172.21.15.98:0/382130743 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2722948356 0 0) 0x7fa2d0003530 con 0x7fa2e40cfcf0 2024-09-19T08:39:47.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.947+0000 7fa2e2ffd640 1 -- 172.21.15.98:0/382130743 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2e415e640 msgr2=0x7fa2e40d2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:47.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.947+0000 7fa2e2ffd640 1 --2- 172.21.15.98:0/382130743 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2e415e640 0x7fa2e40d2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:47.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.947+0000 7fa2e2ffd640 1 -- 172.21.15.98:0/382130743 >> v2:172.21.15.139:3300/0 conn(0x7fa2e406c3a0 msgr2=0x7fa2e40b7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:47.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.947+0000 7fa2e2ffd640 1 --2- 172.21.15.98:0/382130743 >> v2:172.21.15.139:3300/0 conn(0x7fa2e406c3a0 0x7fa2e40b7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:47.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.947+0000 7fa2e2ffd640 1 -- 172.21.15.98:0/382130743 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e40d0550 con 0x7fa2e40cfcf0 2024-09-19T08:39:47.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.947+0000 7fa2e2ffd640 1 -- 172.21.15.98:0/382130743 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fa2cc001680 con 0x7fa2e40cfcf0 2024-09-19T08:39:47.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.947+0000 7fa2e2ffd640 1 -- 172.21.15.98:0/382130743 >> v1:172.21.15.98:6789/0 conn(0x7fa2e40cfcf0 legacy=0x7fa2e406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:47.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.947+0000 7fa2e2ffd640 1 --2- 172.21.15.98:0/382130743 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2e415e640 0x7fa2d00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:47.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.947+0000 7fa2e2ffd640 1 --2- 172.21.15.98:0/382130743 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2d0003f40 0x7fa2d0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:47.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.947+0000 7fa2e2ffd640 1 --2- 172.21.15.98:0/382130743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2d0004d50 0x7fa2d00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:47.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.947+0000 7fa2e2ffd640 1 -- 172.21.15.98:0/382130743 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e415ea20 con 0x7fa2d0004d50 2024-09-19T08:39:47.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.947+0000 7fa2e2ffd640 1 -- 172.21.15.98:0/382130743 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d000b9d0 con 0x7fa2d0003f40 2024-09-19T08:39:47.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.947+0000 7fa2e2ffd640 1 -- 172.21.15.98:0/382130743 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa2d000bb50 con 0x7fa2e415e640 2024-09-19T08:39:47.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.947+0000 7fa2e3fff640 1 --2- 172.21.15.98:0/382130743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2d0004d50 0x7fa2d00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:47.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.947+0000 7fa2e37fe640 1 --2- 172.21.15.98:0/382130743 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2e415e640 0x7fa2d00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:47.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.947+0000 7fa2dbfff640 1 --2- 172.21.15.98:0/382130743 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2d0003f40 0x7fa2d0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:47.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.947+0000 7fa2e3fff640 1 -- 172.21.15.98:0/382130743 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2e415e640 msgr2=0x7fa2d00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:47.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.947+0000 7fa2e3fff640 1 --2- 172.21.15.98:0/382130743 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2e415e640 0x7fa2d00039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:47.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.947+0000 7fa2e3fff640 1 -- 172.21.15.98:0/382130743 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2d0003f40 msgr2=0x7fa2d0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:47.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.947+0000 7fa2e3fff640 1 --2- 172.21.15.98:0/382130743 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2d0003f40 0x7fa2d0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:47.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.947+0000 7fa2e3fff640 1 -- 172.21.15.98:0/382130743 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d0003530 con 0x7fa2d0004d50 2024-09-19T08:39:47.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.947+0000 7fa2e3fff640 1 --2- 172.21.15.98:0/382130743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2d0004d50 0x7fa2d00051d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009f90 tx=0x7fa2dc01cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20878029bdc0ed29 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:47.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.947+0000 7fa2e2ffd640 1 -- 172.21.15.98:0/382130743 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc0b2070 con 0x7fa2d0004d50 2024-09-19T08:39:47.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.947+0000 7fa2e2ffd640 1 -- 172.21.15.98:0/382130743 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2dc0a5c90 con 0x7fa2d0004d50 2024-09-19T08:39:47.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.947+0000 7fa2e2ffd640 1 -- 172.21.15.98:0/382130743 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc009c10 con 0x7fa2d0004d50 2024-09-19T08:39:47.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.951+0000 7fa2ea080640 1 -- 172.21.15.98:0/382130743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2d0004d50 msgr2=0x7fa2d00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:47.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.951+0000 7fa2ea080640 1 --2- 172.21.15.98:0/382130743 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2d0004d50 0x7fa2d00051d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009f90 tx=0x7fa2dc01cc20 comp rx=0 tx=0).stop 2024-09-19T08:39:47.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.951+0000 7fa2e3fff640 1 -- 172.21.15.98:0/382130743 reap_dead start 2024-09-19T08:39:47.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.951+0000 7fa2ea080640 1 -- 172.21.15.98:0/382130743 shutdown_connections 2024-09-19T08:39:47.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.951+0000 7fa2ea080640 1 -- 172.21.15.98:0/382130743 >> 172.21.15.98:0/382130743 conn(0x7fa2e40726b0 msgr2=0x7fa2e4072ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:47.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.951+0000 7fa2ea080640 1 -- 172.21.15.98:0/382130743 shutdown_connections 2024-09-19T08:39:47.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.951+0000 7fa2ea080640 1 -- 172.21.15.98:0/382130743 wait complete. 2024-09-19T08:39:47.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.951+0000 7fa2ea080640 1 Processor -- start 2024-09-19T08:39:47.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.951+0000 7fa2ea080640 1 -- start start 2024-09-19T08:39:47.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.951+0000 7fa2ea080640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2d0004d50 0x7fa2e40cb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:47.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.951+0000 7fa2ea080640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2e41f7970 0x7fa2e41557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:47.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.951+0000 7fa2ea080640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2e40ccec0 0x7fa2e40cd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:47.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.951+0000 7fa2ea080640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e40d2ab0 con 0x7fa2d0004d50 2024-09-19T08:39:47.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.951+0000 7fa2ea080640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e40d0b10 con 0x7fa2e41f7970 2024-09-19T08:39:47.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.951+0000 7fa2ea080640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa2e40d2c30 con 0x7fa2e40ccec0 2024-09-19T08:39:47.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.951+0000 7fa2e37fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2d0004d50 0x7fa2e40cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:47.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.951+0000 7fa2e37fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2d0004d50 0x7fa2e40cb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57062/0 (socket says 172.21.15.98:57062) 2024-09-19T08:39:47.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.951+0000 7fa2e37fe640 1 -- 172.21.15.98:0/1447257513 learned_addr learned my addr 172.21.15.98:0/1447257513 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:47.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.951+0000 7fa2e3fff640 1 --2- 172.21.15.98:0/1447257513 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2e40ccec0 0x7fa2e40cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:47.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.951+0000 7fa2dbfff640 1 --2- 172.21.15.98:0/1447257513 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2e41f7970 0x7fa2e41557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:47.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.951+0000 7fa2e37fe640 1 -- 172.21.15.98:0/1447257513 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2e40ccec0 msgr2=0x7fa2e40cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:47.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.951+0000 7fa2e37fe640 1 --2- 172.21.15.98:0/1447257513 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2e40ccec0 0x7fa2e40cd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:47.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.951+0000 7fa2e37fe640 1 -- 172.21.15.98:0/1447257513 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2e41f7970 msgr2=0x7fa2e41557d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:47.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.951+0000 7fa2e37fe640 1 --2- 172.21.15.98:0/1447257513 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2e41f7970 0x7fa2e41557d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:47.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.951+0000 7fa2e37fe640 1 -- 172.21.15.98:0/1447257513 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e41f7df0 con 0x7fa2d0004d50 2024-09-19T08:39:47.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.951+0000 7fa2e37fe640 1 --2- 172.21.15.98:0/1447257513 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2d0004d50 0x7fa2e40cb630 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4011610 tx=0x7fa2d4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:47.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.951+0000 7fa2e0ff9640 1 -- 172.21.15.98:0/1447257513 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d400f8f0 con 0x7fa2d0004d50 2024-09-19T08:39:47.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.951+0000 7fa2e0ff9640 1 -- 172.21.15.98:0/1447257513 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2d400fa90 con 0x7fa2d0004d50 2024-09-19T08:39:47.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.951+0000 7fa2e0ff9640 1 -- 172.21.15.98:0/1447257513 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d400fe40 con 0x7fa2d0004d50 2024-09-19T08:39:47.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.951+0000 7fa2ea080640 1 -- 172.21.15.98:0/1447257513 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2e40cbcc0 con 0x7fa2d0004d50 2024-09-19T08:39:47.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.951+0000 7fa2ea080640 1 -- 172.21.15.98:0/1447257513 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2e40cc280 con 0x7fa2d0004d50 2024-09-19T08:39:47.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.955+0000 7fa2ea080640 1 -- 172.21.15.98:0/1447257513 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2e40d0130 con 0x7fa2d0004d50 2024-09-19T08:39:47.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.955+0000 7fa2e0ff9640 1 -- 172.21.15.98:0/1447257513 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fa2d4015460 con 0x7fa2d0004d50 2024-09-19T08:39:47.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.955+0000 7fa2e0ff9640 1 --2- 172.21.15.98:0/1447257513 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa2c4067a70 0x7fa2c4069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:47.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.955+0000 7fa2e0ff9640 1 -- 172.21.15.98:0/1447257513 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fa2d40868c0 con 0x7fa2d0004d50 2024-09-19T08:39:47.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.955+0000 7fa2dbfff640 1 --2- 172.21.15.98:0/1447257513 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa2c4067a70 0x7fa2c4069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:47.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.955+0000 7fa2dbfff640 1 --2- 172.21.15.98:0/1447257513 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa2c4067a70 0x7fa2c4069f30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc001ec0 tx=0x7fa2cc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:47.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:47.959+0000 7fa2e0ff9640 1 -- 172.21.15.98:0/1447257513 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa2d4019030 con 0x7fa2d0004d50 2024-09-19T08:39:48.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.175+0000 7fa2ea080640 1 -- 172.21.15.98:0/1447257513 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fa2e415ed20 con 0x7fa2d0004d50 2024-09-19T08:39:48.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.183+0000 7fa2e0ff9640 1 -- 172.21.15.98:0/1447257513 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7fa2d4057210 con 0x7fa2d0004d50 2024-09-19T08:39:48.182 INFO:teuthology.orchestra.run.smithi098.stdout:[client.0] 2024-09-19T08:39:48.182 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQC/4utm0vNpIhAA4S1E8dFZ0+ibmSKhmZ+bIA== 2024-09-19T08:39:48.182 INFO:teuthology.orchestra.run.smithi098.stdout: caps mds = "allow" 2024-09-19T08:39:48.182 INFO:teuthology.orchestra.run.smithi098.stdout: caps mon = "allow r" 2024-09-19T08:39:48.182 INFO:teuthology.orchestra.run.smithi098.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-19T08:39:48.183 INFO:teuthology.orchestra.run.smithi098.stderr:updated caps for client.0 2024-09-19T08:39:48.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.183+0000 7fa2ea080640 1 -- 172.21.15.98:0/1447257513 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa2c4067a70 msgr2=0x7fa2c4069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:48.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.183+0000 7fa2ea080640 1 --2- 172.21.15.98:0/1447257513 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa2c4067a70 0x7fa2c4069f30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc001ec0 tx=0x7fa2cc009040 comp rx=0 tx=0).stop 2024-09-19T08:39:48.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.183+0000 7fa2ea080640 1 -- 172.21.15.98:0/1447257513 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2d0004d50 msgr2=0x7fa2e40cb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:48.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.183+0000 7fa2ea080640 1 --2- 172.21.15.98:0/1447257513 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2d0004d50 0x7fa2e40cb630 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4011610 tx=0x7fa2d4011ad0 comp rx=0 tx=0).stop 2024-09-19T08:39:48.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.183+0000 7fa2ea080640 1 -- 172.21.15.98:0/1447257513 shutdown_connections 2024-09-19T08:39:48.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.183+0000 7fa2ea080640 1 --2- 172.21.15.98:0/1447257513 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa2c4067a70 0x7fa2c4069f30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:48.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.183+0000 7fa2ea080640 1 --2- 172.21.15.98:0/1447257513 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2e40ccec0 0x7fa2e40cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:48.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.183+0000 7fa2ea080640 1 --2- 172.21.15.98:0/1447257513 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2e41f7970 0x7fa2e41557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:48.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.183+0000 7fa2ea080640 1 --2- 172.21.15.98:0/1447257513 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2d0004d50 0x7fa2e40cb630 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:48.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.183+0000 7fa2ea080640 1 -- 172.21.15.98:0/1447257513 >> 172.21.15.98:0/1447257513 conn(0x7fa2e40726b0 msgr2=0x7fa2e40cf2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:48.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.183+0000 7fa2ea080640 1 -- 172.21.15.98:0/1447257513 shutdown_connections 2024-09-19T08:39:48.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.183+0000 7fa2ea080640 1 -- 172.21.15.98:0/1447257513 wait complete. 2024-09-19T08:39:48.199 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:39:48.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.315+0000 7f9e13e6d640 1 Processor -- start 2024-09-19T08:39:48.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.315+0000 7f9e13e6d640 1 -- start start 2024-09-19T08:39:48.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.319+0000 7f9e13e6d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9e0c06c3a0 0x7f9e0c0cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:48.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.319+0000 7f9e13e6d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9e0c0cf900 0x7f9e0c0723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:48.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.319+0000 7f9e13e6d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e0c06d310 con 0x7f9e0c15c550 2024-09-19T08:39:48.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.319+0000 7f9e13e6d640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f9e0c06d5f0 con 0x7f9e0c06c3a0 2024-09-19T08:39:48.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.319+0000 7f9e13e6d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9e0c06d8d0 con 0x7f9e0c0cf900 2024-09-19T08:39:48.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.319+0000 7f9e11408640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9e0c06c3a0 0x7f9e0c0cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:48.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.319+0000 7f9e11408640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9e0c06c3a0 0x7f9e0c0cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57064/0 (socket says 172.21.15.98:57064) 2024-09-19T08:39:48.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.319+0000 7f9e11408640 1 -- 172.21.15.98:0/1570961740 learned_addr learned my addr 172.21.15.98:0/1570961740 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:48.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.319+0000 7f9e1240a640 1 --2- 172.21.15.98:0/1570961740 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9e0c0cf900 0x7f9e0c0723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:48.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.319+0000 7f9e11408640 1 -- 172.21.15.98:0/1570961740 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9e0c0cf900 msgr2=0x7f9e0c0723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:48.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.319+0000 7f9e11408640 1 --2- 172.21.15.98:0/1570961740 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9e0c0cf900 0x7f9e0c0723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:48.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.319+0000 7f9e11408640 1 -- 172.21.15.98:0/1570961740 >> v1:172.21.15.139:6789/0 conn(0x7f9e0c15c550 legacy=0x7f9e0c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:48.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.319+0000 7f9e11408640 1 -- 172.21.15.98:0/1570961740 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e0c06dbb0 con 0x7f9e0c06c3a0 2024-09-19T08:39:48.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.319+0000 7f9e11408640 1 --2- 172.21.15.98:0/1570961740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e0c06c3a0 0x7f9e0c0cf3c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc009870 tx=0x7f9dfc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6557b12d2b04ee34 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:48.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.319+0000 7f9e10c07640 1 -- 172.21.15.98:0/1570961740 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc0a7070 con 0x7f9e0c06c3a0 2024-09-19T08:39:48.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.319+0000 7f9e10c07640 1 -- 172.21.15.98:0/1570961740 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dfc0b0070 con 0x7f9e0c06c3a0 2024-09-19T08:39:48.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.319+0000 7f9e10c07640 1 -- 172.21.15.98:0/1570961740 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc0ab890 con 0x7f9e0c06c3a0 2024-09-19T08:39:48.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.319+0000 7f9e13e6d640 1 -- 172.21.15.98:0/1570961740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e0c06c3a0 msgr2=0x7f9e0c0cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:48.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.323+0000 7f9e13e6d640 1 --2- 172.21.15.98:0/1570961740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e0c06c3a0 0x7f9e0c0cf3c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc009870 tx=0x7f9dfc01c560 comp rx=0 tx=0).stop 2024-09-19T08:39:48.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.323+0000 7f9e13e6d640 1 -- 172.21.15.98:0/1570961740 shutdown_connections 2024-09-19T08:39:48.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.323+0000 7f9e13e6d640 1 --2- 172.21.15.98:0/1570961740 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9e0c0cf900 0x7f9e0c0723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:48.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.323+0000 7f9e13e6d640 1 --2- 172.21.15.98:0/1570961740 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e0c06c3a0 0x7f9e0c0cf3c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:48.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.323+0000 7f9e13e6d640 1 -- 172.21.15.98:0/1570961740 >> 172.21.15.98:0/1570961740 conn(0x7f9e0c0b2010 msgr2=0x7f9e0c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:48.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.323+0000 7f9e13e6d640 1 -- 172.21.15.98:0/1570961740 shutdown_connections 2024-09-19T08:39:48.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.323+0000 7f9e13e6d640 1 -- 172.21.15.98:0/1570961740 wait complete. 2024-09-19T08:39:48.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.323+0000 7f9e13e6d640 1 Processor -- start 2024-09-19T08:39:48.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.323+0000 7f9e13e6d640 1 -- start start 2024-09-19T08:39:48.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.323+0000 7f9e13e6d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e0c06c3a0 0x7f9e0c1f7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:48.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.323+0000 7f9e13e6d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9e0c0cf900 0x7f9e0c156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:48.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.323+0000 7f9e13e6d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e0c15c550 0x7f9e0c15df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:48.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.323+0000 7f9e13e6d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0c073500 con 0x7f9e0c06c3a0 2024-09-19T08:39:48.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.323+0000 7f9e13e6d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0c073380 con 0x7f9e0c15c550 2024-09-19T08:39:48.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.323+0000 7f9e13e6d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9e0c073680 con 0x7f9e0c0cf900 2024-09-19T08:39:48.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.323+0000 7f9e11c09640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e0c06c3a0 0x7f9e0c1f7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:48.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.323+0000 7f9e11c09640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e0c06c3a0 0x7f9e0c1f7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57076/0 (socket says 172.21.15.98:57076) 2024-09-19T08:39:48.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.323+0000 7f9e11c09640 1 -- 172.21.15.98:0/4194389260 learned_addr learned my addr 172.21.15.98:0/4194389260 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:48.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.323+0000 7f9e11408640 1 --2- 172.21.15.98:0/4194389260 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9e0c0cf900 0x7f9e0c156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:48.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.323+0000 7f9e1240a640 1 --2- 172.21.15.98:0/4194389260 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e0c15c550 0x7f9e0c15df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:48.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.323+0000 7f9e11c09640 1 -- 172.21.15.98:0/4194389260 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9e0c0cf900 msgr2=0x7f9e0c156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:48.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.323+0000 7f9e11c09640 1 --2- 172.21.15.98:0/4194389260 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9e0c0cf900 0x7f9e0c156ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:48.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.323+0000 7f9e11c09640 1 -- 172.21.15.98:0/4194389260 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e0c15c550 msgr2=0x7f9e0c15df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:48.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.323+0000 7f9e11c09640 1 --2- 172.21.15.98:0/4194389260 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e0c15c550 0x7f9e0c15df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:48.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.323+0000 7f9e11c09640 1 -- 172.21.15.98:0/4194389260 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e0c0b5120 con 0x7f9e0c06c3a0 2024-09-19T08:39:48.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.323+0000 7f9e1240a640 1 --2- 172.21.15.98:0/4194389260 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e0c15c550 0x7f9e0c15df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:48.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.323+0000 7f9e11c09640 1 --2- 172.21.15.98:0/4194389260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e0c06c3a0 0x7f9e0c1f7350 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9e0800c370 tx=0x7f9e0800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:48.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.323+0000 7f9e027fc640 1 -- 172.21.15.98:0/4194389260 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e080052d0 con 0x7f9e0c06c3a0 2024-09-19T08:39:48.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.323+0000 7f9e027fc640 1 -- 172.21.15.98:0/4194389260 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e08005470 con 0x7f9e0c06c3a0 2024-09-19T08:39:48.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.323+0000 7f9e13e6d640 1 -- 172.21.15.98:0/4194389260 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e0c1547d0 con 0x7f9e0c06c3a0 2024-09-19T08:39:48.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.323+0000 7f9e027fc640 1 -- 172.21.15.98:0/4194389260 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e08007260 con 0x7f9e0c06c3a0 2024-09-19T08:39:48.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.323+0000 7f9e13e6d640 1 -- 172.21.15.98:0/4194389260 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e0c154d90 con 0x7f9e0c06c3a0 2024-09-19T08:39:48.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.323+0000 7f9e13e6d640 1 -- 172.21.15.98:0/4194389260 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e0c154f90 con 0x7f9e0c06c3a0 2024-09-19T08:39:48.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.327+0000 7f9e027fc640 1 -- 172.21.15.98:0/4194389260 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f9e08007400 con 0x7f9e0c06c3a0 2024-09-19T08:39:48.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.327+0000 7f9e027fc640 1 --2- 172.21.15.98:0/4194389260 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9df0067a70 0x7f9df0069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:48.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.327+0000 7f9e027fc640 1 -- 172.21.15.98:0/4194389260 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f9e08085420 con 0x7f9e0c06c3a0 2024-09-19T08:39:48.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.327+0000 7f9e11408640 1 --2- 172.21.15.98:0/4194389260 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9df0067a70 0x7f9df0069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:48.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.327+0000 7f9e11408640 1 --2- 172.21.15.98:0/4194389260 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9df0067a70 0x7f9df0069f30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc0af000 tx=0x7f9dfc0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:48.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.331+0000 7f9e027fc640 1 -- 172.21.15.98:0/4194389260 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9e0c154f90 con 0x7f9e0c06c3a0 2024-09-19T08:39:48.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.511+0000 7f9e13e6d640 1 -- 172.21.15.98:0/4194389260 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e0c0cf3c0 con 0x7f9e0c06c3a0 2024-09-19T08:39:48.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.511+0000 7f9e027fc640 1 -- 172.21.15.98:0/4194389260 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5229 (secure 0 0 0) 0x7f9e08055ce0 con 0x7f9e0c06c3a0 2024-09-19T08:39:48.512 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:39:48.512 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":23,"btime":"2024-09-19T08:39:41:944534+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/2874086421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":2874086421},{"type":"v1","addr":"172.21.15.139:6833","nonce":2874086421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4996,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/4178563848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":4178563848},{"type":"v1","addr":"172.21.15.139:6835","nonce":4178563848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3584568343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3584568343},{"type":"v1","addr":"172.21.15.98:6835","nonce":3584568343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:39:38.262938+0000","modified":"2024-09-19T08:39:41.276985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/3085042202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3085042202},{"type":"v1","addr":"172.21.15.98:6837","nonce":3085042202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244]},"id":3}]} 2024-09-19T08:39:48.512 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 23 2024-09-19T08:39:48.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.515+0000 7f9e13e6d640 1 -- 172.21.15.98:0/4194389260 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9df0067a70 msgr2=0x7f9df0069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:48.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.515+0000 7f9e13e6d640 1 --2- 172.21.15.98:0/4194389260 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9df0067a70 0x7f9df0069f30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc0af000 tx=0x7f9dfc0a7000 comp rx=0 tx=0).stop 2024-09-19T08:39:48.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.515+0000 7f9e13e6d640 1 -- 172.21.15.98:0/4194389260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e0c06c3a0 msgr2=0x7f9e0c1f7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:48.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.515+0000 7f9e13e6d640 1 --2- 172.21.15.98:0/4194389260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e0c06c3a0 0x7f9e0c1f7350 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9e0800c370 tx=0x7f9e0800c830 comp rx=0 tx=0).stop 2024-09-19T08:39:48.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.515+0000 7f9e13e6d640 1 -- 172.21.15.98:0/4194389260 shutdown_connections 2024-09-19T08:39:48.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.515+0000 7f9e13e6d640 1 --2- 172.21.15.98:0/4194389260 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9df0067a70 0x7f9df0069f30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:48.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.515+0000 7f9e13e6d640 1 --2- 172.21.15.98:0/4194389260 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e0c15c550 0x7f9e0c15df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:48.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.515+0000 7f9e13e6d640 1 --2- 172.21.15.98:0/4194389260 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9e0c0cf900 0x7f9e0c156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:48.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.515+0000 7f9e13e6d640 1 --2- 172.21.15.98:0/4194389260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9e0c06c3a0 0x7f9e0c1f7350 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:48.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.515+0000 7f9e13e6d640 1 -- 172.21.15.98:0/4194389260 >> 172.21.15.98:0/4194389260 conn(0x7f9e0c0b2010 msgr2=0x7f9e0c1560f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:48.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.515+0000 7f9e13e6d640 1 -- 172.21.15.98:0/4194389260 shutdown_connections 2024-09-19T08:39:48.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:48.515+0000 7f9e13e6d640 1 -- 172.21.15.98:0/4194389260 wait complete. 2024-09-19T08:39:48.934 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T08:39:38.262938+0000', 'modified': '2024-09-19T08:39:41.276985+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5244}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5244': {'gid': 5244, 'name': 'c', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.98:6837/3085042202', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6836', 'nonce': 3085042202}, {'type': 'v1', 'addr': '172.21.15.98:6837', 'nonce': 3085042202}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5244, 'qdb_cluster': [5244]} 2024-09-19T08:39:48.934 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T08:39:48.934 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-19T08:39:49.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.067+0000 7fd25809e640 1 Processor -- start 2024-09-19T08:39:49.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.067+0000 7fd25809e640 1 -- start start 2024-09-19T08:39:49.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.067+0000 7fd25809e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd250158fd0 0x7fd2500b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:49.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.067+0000 7fd25809e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd2500b4360 0x7fd25006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:49.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.067+0000 7fd25809e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2500ccc60 con 0x7fd25006c1d0 2024-09-19T08:39:49.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.067+0000 7fd25809e640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fd25006c640 con 0x7fd2500b4360 2024-09-19T08:39:49.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.067+0000 7fd25809e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fd2501593b0 con 0x7fd250158fd0 2024-09-19T08:39:49.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.071+0000 7fd255639640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd2500b4360 0x7fd25006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:49.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.071+0000 7fd255639640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd2500b4360 0x7fd25006bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57080/0 (socket says 172.21.15.98:57080) 2024-09-19T08:39:49.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.071+0000 7fd255639640 1 -- 172.21.15.98:0/3796599535 learned_addr learned my addr 172.21.15.98:0/3796599535 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:49.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.071+0000 7fd255e3a640 1 --2- 172.21.15.98:0/3796599535 >> v2:172.21.15.139:3300/0 conn(0x7fd250158fd0 0x7fd2500b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:49.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.071+0000 7fd255639640 1 -- 172.21.15.98:0/3796599535 >> v2:172.21.15.139:3300/0 conn(0x7fd250158fd0 msgr2=0x7fd2500b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:49.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.071+0000 7fd255639640 1 --2- 172.21.15.98:0/3796599535 >> v2:172.21.15.139:3300/0 conn(0x7fd250158fd0 0x7fd2500b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.071+0000 7fd255639640 1 -- 172.21.15.98:0/3796599535 >> v1:172.21.15.139:6789/0 conn(0x7fd25006c1d0 legacy=0x7fd25015efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:49.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.071+0000 7fd255639640 1 -- 172.21.15.98:0/3796599535 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2500ccf40 con 0x7fd2500b4360 2024-09-19T08:39:49.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.071+0000 7fd255639640 1 --2- 172.21.15.98:0/3796599535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2500b4360 0x7fd25006bc90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd25006e310 tx=0x7fd24001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d4ee23036c1d9c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:49.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.071+0000 7fd254e38640 1 -- 172.21.15.98:0/3796599535 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2400ab620 con 0x7fd2500b4360 2024-09-19T08:39:49.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.071+0000 7fd254e38640 1 -- 172.21.15.98:0/3796599535 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2400b0070 con 0x7fd2500b4360 2024-09-19T08:39:49.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.071+0000 7fd254e38640 1 -- 172.21.15.98:0/3796599535 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2400ab9e0 con 0x7fd2500b4360 2024-09-19T08:39:49.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.071+0000 7fd25809e640 1 -- 172.21.15.98:0/3796599535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2500b4360 msgr2=0x7fd25006bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:49.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.071+0000 7fd25809e640 1 --2- 172.21.15.98:0/3796599535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2500b4360 0x7fd25006bc90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd25006e310 tx=0x7fd24001c420 comp rx=0 tx=0).stop 2024-09-19T08:39:49.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.071+0000 7fd25809e640 1 -- 172.21.15.98:0/3796599535 shutdown_connections 2024-09-19T08:39:49.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.071+0000 7fd25809e640 1 --2- 172.21.15.98:0/3796599535 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2500b4360 0x7fd25006bc90 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.071+0000 7fd25809e640 1 --2- 172.21.15.98:0/3796599535 >> v2:172.21.15.139:3300/0 conn(0x7fd250158fd0 0x7fd2500b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.071+0000 7fd25809e640 1 -- 172.21.15.98:0/3796599535 >> 172.21.15.98:0/3796599535 conn(0x7fd2500b2010 msgr2=0x7fd2500b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:49.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.071+0000 7fd25809e640 1 -- 172.21.15.98:0/3796599535 shutdown_connections 2024-09-19T08:39:49.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.075+0000 7fd25809e640 1 -- 172.21.15.98:0/3796599535 wait complete. 2024-09-19T08:39:49.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.075+0000 7fd25809e640 1 Processor -- start 2024-09-19T08:39:49.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.075+0000 7fd25809e640 1 -- start start 2024-09-19T08:39:49.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.075+0000 7fd25809e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd25006c1d0 0x7fd250152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:49.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.075+0000 7fd25809e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2500b4360 0x7fd250072f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:49.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.075+0000 7fd25809e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd250158fd0 0x7fd2501548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:49.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.075+0000 7fd25809e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd25015c920 con 0x7fd2500b4360 2024-09-19T08:39:49.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.075+0000 7fd25809e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd25015c7a0 con 0x7fd250158fd0 2024-09-19T08:39:49.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.075+0000 7fd25809e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd250168110 con 0x7fd25006c1d0 2024-09-19T08:39:49.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.075+0000 7fd255639640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2500b4360 0x7fd250072f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:49.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.075+0000 7fd255639640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2500b4360 0x7fd250072f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57084/0 (socket says 172.21.15.98:57084) 2024-09-19T08:39:49.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.075+0000 7fd255639640 1 -- 172.21.15.98:0/1336534483 learned_addr learned my addr 172.21.15.98:0/1336534483 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:49.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.075+0000 7fd25663b640 1 --2- 172.21.15.98:0/1336534483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd250158fd0 0x7fd2501548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:49.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.075+0000 7fd255e3a640 1 --2- 172.21.15.98:0/1336534483 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd25006c1d0 0x7fd250152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:49.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.075+0000 7fd255639640 1 -- 172.21.15.98:0/1336534483 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd25006c1d0 msgr2=0x7fd250152e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:49.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.075+0000 7fd255639640 1 --2- 172.21.15.98:0/1336534483 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd25006c1d0 0x7fd250152e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.075+0000 7fd255639640 1 -- 172.21.15.98:0/1336534483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd250158fd0 msgr2=0x7fd2501548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:49.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.075+0000 7fd255639640 1 --2- 172.21.15.98:0/1336534483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd250158fd0 0x7fd2501548e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.075+0000 7fd255639640 1 -- 172.21.15.98:0/1336534483 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd250155670 con 0x7fd2500b4360 2024-09-19T08:39:49.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.075+0000 7fd25663b640 1 --2- 172.21.15.98:0/1336534483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd250158fd0 0x7fd2501548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:49.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.075+0000 7fd255639640 1 --2- 172.21.15.98:0/1336534483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2500b4360 0x7fd250072f60 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd24001a050 tx=0x7fd24001ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:49.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.075+0000 7fd2467fc640 1 -- 172.21.15.98:0/1336534483 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2400abdb0 con 0x7fd2500b4360 2024-09-19T08:39:49.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.075+0000 7fd2467fc640 1 -- 172.21.15.98:0/1336534483 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2400be3f0 con 0x7fd2500b4360 2024-09-19T08:39:49.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.075+0000 7fd25809e640 1 -- 172.21.15.98:0/1336534483 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd250072480 con 0x7fd2500b4360 2024-09-19T08:39:49.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.075+0000 7fd2467fc640 1 -- 172.21.15.98:0/1336534483 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2400be7a0 con 0x7fd2500b4360 2024-09-19T08:39:49.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.075+0000 7fd25809e640 1 -- 172.21.15.98:0/1336534483 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd250155290 con 0x7fd2500b4360 2024-09-19T08:39:49.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.079+0000 7fd2467fc640 1 -- 172.21.15.98:0/1336534483 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fd2400c7440 con 0x7fd2500b4360 2024-09-19T08:39:49.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.079+0000 7fd2467fc640 1 --2- 172.21.15.98:0/1336534483 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd234067a70 0x7fd234069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:49.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.079+0000 7fd2467fc640 1 -- 172.21.15.98:0/1336534483 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fd2401205a0 con 0x7fd2500b4360 2024-09-19T08:39:49.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.079+0000 7fd255e3a640 1 --2- 172.21.15.98:0/1336534483 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd234067a70 0x7fd234069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:49.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.079+0000 7fd255e3a640 1 --2- 172.21.15.98:0/1336534483 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd234067a70 0x7fd234069f30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd250165ca0 tx=0x7fd24c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:49.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.079+0000 7fd25b15a000 1 Processor -- start 2024-09-19T08:39:49.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.079+0000 7fd25b15a000 1 -- start start 2024-09-19T08:39:49.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.079+0000 7fd25b15a000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559807f75b20 0x559807f75f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:49.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.079+0000 7fd25b15a000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x559807f76440 0x559807f2bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:49.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.079+0000 7fd25b15a000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559807f2c290 0x559807f2c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:49.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.079+0000 7fd25b15a000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559807e13b60 con 0x559807f2c290 2024-09-19T08:39:49.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.079+0000 7fd25b15a000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x559807d7ac70 con 0x559807f75b20 2024-09-19T08:39:49.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.079+0000 7fd25b15a000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x559807d20ed0 con 0x559807f76440 2024-09-19T08:39:49.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.079+0000 7fd25663b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559807f2c290 0x559807f2c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:49.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.079+0000 7fd25663b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559807f2c290 0x559807f2c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57100/0 (socket says 172.21.15.98:57100) 2024-09-19T08:39:49.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.079+0000 7fd25663b640 1 -- 172.21.15.98:0/605616726 learned_addr learned my addr 172.21.15.98:0/605616726 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:49.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.079+0000 7fd25663b640 1 -- 172.21.15.98:0/605616726 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x559807f76440 msgr2=0x559807f2bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:49.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.079+0000 7fd255639640 1 --2- 172.21.15.98:0/605616726 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x559807f76440 0x559807f2bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:49.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.079+0000 7fd255e3a640 1 --2- 172.21.15.98:0/605616726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559807f75b20 0x559807f75f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:49.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.079+0000 7fd25663b640 1 --2- 172.21.15.98:0/605616726 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x559807f76440 0x559807f2bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.079+0000 7fd25663b640 1 -- 172.21.15.98:0/605616726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559807f75b20 msgr2=0x559807f75f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:49.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.079+0000 7fd25663b640 1 --2- 172.21.15.98:0/605616726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559807f75b20 0x559807f75f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.079+0000 7fd25663b640 1 -- 172.21.15.98:0/605616726 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559807d32e50 con 0x559807f2c290 2024-09-19T08:39:49.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.079+0000 7fd25663b640 1 --2- 172.21.15.98:0/605616726 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559807f2c290 0x559807f2c670 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd248003ea0 tx=0x7fd248004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b719982706d2cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:49.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.079+0000 7fd2337fe640 1 -- 172.21.15.98:0/605616726 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd248005270 con 0x559807f2c290 2024-09-19T08:39:49.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.079+0000 7fd2337fe640 1 -- 172.21.15.98:0/605616726 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd248005430 con 0x559807f2c290 2024-09-19T08:39:49.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.079+0000 7fd2337fe640 1 -- 172.21.15.98:0/605616726 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd24800db80 con 0x559807f2c290 2024-09-19T08:39:49.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd25b15a000 1 -- 172.21.15.98:0/605616726 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559807f2c290 msgr2=0x559807f2c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:49.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd25b15a000 1 --2- 172.21.15.98:0/605616726 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559807f2c290 0x559807f2c670 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd248003ea0 tx=0x7fd248004360 comp rx=0 tx=0).stop 2024-09-19T08:39:49.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd25b15a000 1 -- 172.21.15.98:0/605616726 shutdown_connections 2024-09-19T08:39:49.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd25b15a000 1 --2- 172.21.15.98:0/605616726 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559807f2c290 0x559807f2c670 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd25b15a000 1 --2- 172.21.15.98:0/605616726 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x559807f76440 0x559807f2bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd25b15a000 1 --2- 172.21.15.98:0/605616726 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559807f75b20 0x559807f75f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd25b15a000 1 -- 172.21.15.98:0/605616726 >> 172.21.15.98:0/605616726 conn(0x559807f32600 msgr2=0x559807f2a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:49.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd25b15a000 1 -- 172.21.15.98:0/605616726 shutdown_connections 2024-09-19T08:39:49.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd25b15a000 1 -- 172.21.15.98:0/605616726 wait complete. 2024-09-19T08:39:49.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd25b15a000 1 -- start start 2024-09-19T08:39:49.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd25b15a000 1 Processor -- start 2024-09-19T08:39:49.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd25b15a000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559807f75b20 0x559807f30e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:49.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd25b15a000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559807f76440 0x559807f36cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:49.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd25b15a000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x559807f31390 0x559807f37230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:49.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd25b15a000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559807ddcb00 con 0x559807f75b20 2024-09-19T08:39:49.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd25b15a000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x559807d55360 con 0x559807f76440 2024-09-19T08:39:49.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd25b15a000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x559807d522d0 con 0x559807f31390 2024-09-19T08:39:49.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd25b15a000 10 client.0 _ll_register_callbacks cb 0x559807f17820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:39:49.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd255e3a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559807f75b20 0x559807f30e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:49.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd255e3a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559807f75b20 0x559807f30e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57110/0 (socket says 172.21.15.98:57110) 2024-09-19T08:39:49.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd255e3a640 1 -- 172.21.15.98:0/3860008760 learned_addr learned my addr 172.21.15.98:0/3860008760 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:49.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd255e3a640 1 -- 172.21.15.98:0/3860008760 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x559807f31390 msgr2=0x559807f37230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:49.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd25663b640 1 --2- 172.21.15.98:0/3860008760 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x559807f31390 0x559807f37230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:49.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd255639640 1 --2- 172.21.15.98:0/3860008760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559807f76440 0x559807f36cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:49.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd255e3a640 1 --2- 172.21.15.98:0/3860008760 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x559807f31390 0x559807f37230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd255e3a640 1 -- 172.21.15.98:0/3860008760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559807f76440 msgr2=0x559807f36cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:49.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd255e3a640 1 --2- 172.21.15.98:0/3860008760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559807f76440 0x559807f36cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd255e3a640 1 -- 172.21.15.98:0/3860008760 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559807d21120 con 0x559807f75b20 2024-09-19T08:39:49.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd255e3a640 1 --2- 172.21.15.98:0/3860008760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559807f75b20 0x559807f30e50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd24c0169a0 tx=0x7fd24c004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a11cf830e56341f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:49.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd230ff9640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-19T08:39:49.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd230ff9640 1 -- 172.21.15.98:0/3860008760 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd24c014ae0 con 0x559807f75b20 2024-09-19T08:39:49.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd230ff9640 1 -- 172.21.15.98:0/3860008760 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fd24c014c80 con 0x559807f75b20 2024-09-19T08:39:49.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd25b15a000 1 -- 172.21.15.98:0/3860008760 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd248005430 con 0x559807f75b20 2024-09-19T08:39:49.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd230ff9640 1 -- 172.21.15.98:0/3860008760 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd24c02d050 con 0x559807f75b20 2024-09-19T08:39:49.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd25b15a000 10 client.5409 fetch_fsmap learned FSMap version 23 2024-09-19T08:39:49.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd25b15a000 1 -- 172.21.15.98:0/3860008760 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x559807d1a450 con 0x559807f75b20 2024-09-19T08:39:49.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd230ff9640 1 -- 172.21.15.98:0/3860008760 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd248005430 con 0x559807f75b20 2024-09-19T08:39:49.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd230ff9640 1 -- 172.21.15.98:0/3860008760 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7fd24c030020 con 0x559807f75b20 2024-09-19T08:39:49.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd25b15a000 10 client.5409 fetch_fsmap finished waiting for FSMap version 23 2024-09-19T08:39:49.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd25b15a000 10 client.5409 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T08:39:49.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd25b15a000 20 client.5409 populate_metadata read hostname 'smithi098' 2024-09-19T08:39:49.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd25b15a000 1 --2- 172.21.15.98:0/3860008760 >> [v2:172.21.15.98:6836/3085042202,v1:172.21.15.98:6837/3085042202] conn(0x559807f84280 0x559807f86670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:49.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd25b15a000 4 client.5409 mds_command: new command op to 5244 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:39:49.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd25b15a000 1 -- 172.21.15.98:0/3860008760 --> [v2:172.21.15.98:6836/3085042202,v1:172.21.15.98:6837/3085042202] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559807d14c00 con 0x559807f84280 2024-09-19T08:39:49.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd255639640 1 --2- 172.21.15.98:0/3860008760 >> [v2:172.21.15.98:6836/3085042202,v1:172.21.15.98:6837/3085042202] conn(0x559807f84280 0x559807f86670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:49.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd255639640 1 --2- 172.21.15.98:0/3860008760 >> [v2:172.21.15.98:6836/3085042202,v1:172.21.15.98:6837/3085042202] conn(0x559807f84280 0x559807f86670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e11b8efc2c2601e2 server_cookie=6200a899aa78f5a4 in_seq=0 out_seq=0 2024-09-19T08:39:49.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.083+0000 7fd230ff9640 10 client.5409 ms_handle_connect on v2:172.21.15.98:6836/3085042202 2024-09-19T08:39:49.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.087+0000 7fd230ff9640 1 -- 172.21.15.98:0/3860008760 <== mds.0 v2:172.21.15.98:6836/3085042202 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559807d14c00 con 0x559807f84280 2024-09-19T08:39:49.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.087+0000 7fd230ff9640 10 client.5409 handle_command_reply: tid=0 2024-09-19T08:39:49.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.087+0000 7fd25b15a000 1 client.5409 shutdown 2024-09-19T08:39:49.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.087+0000 7fd25b15a000 1 -- 172.21.15.98:0/3860008760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559807f75b20 msgr2=0x559807f30e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:49.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.087+0000 7fd25b15a000 1 --2- 172.21.15.98:0/3860008760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559807f75b20 0x559807f30e50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd24c0169a0 tx=0x7fd24c004ac0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.087+0000 7fd25b15a000 1 -- 172.21.15.98:0/3860008760 shutdown_connections 2024-09-19T08:39:49.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.087+0000 7fd25b15a000 1 --2- 172.21.15.98:0/3860008760 >> [v2:172.21.15.98:6836/3085042202,v1:172.21.15.98:6837/3085042202] conn(0x559807f84280 0x559807f86670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.087+0000 7fd25b15a000 1 --2- 172.21.15.98:0/3860008760 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x559807f31390 0x559807f37230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.087+0000 7fd25b15a000 1 --2- 172.21.15.98:0/3860008760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559807f76440 0x559807f36cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.087+0000 7fd25b15a000 1 --2- 172.21.15.98:0/3860008760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559807f75b20 0x559807f30e50 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.087+0000 7fd230ff9640 0 client.5409 ms_handle_reset on v2:172.21.15.98:6836/3085042202 2024-09-19T08:39:49.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.087+0000 7fd25b15a000 1 -- 172.21.15.98:0/3860008760 >> 172.21.15.98:0/3860008760 conn(0x559807f2c290 msgr2=0x559807f2ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:49.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.087+0000 7fd25b15a000 1 -- 172.21.15.98:0/3860008760 shutdown_connections 2024-09-19T08:39:49.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.087+0000 7fd25b15a000 1 -- 172.21.15.98:0/3860008760 wait complete. 2024-09-19T08:39:49.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.087+0000 7fd25b15a000 20 client.5409 trim_cache size 0 max 16384 2024-09-19T08:39:49.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.103+0000 7fd25b15a000 1 Processor -- start 2024-09-19T08:39:49.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.103+0000 7fd25b15a000 1 -- start start 2024-09-19T08:39:49.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.103+0000 7fd25b15a000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559807f31390 0x559807f74cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:49.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.103+0000 7fd25b15a000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x559807f75b20 0x559807f75200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:49.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25b15a000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559807f76440 0x559807f86670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:49.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25b15a000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd250200cb0 con 0x559807f76440 2024-09-19T08:39:49.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25b15a000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd250071690 con 0x559807f31390 2024-09-19T08:39:49.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25b15a000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd250071ce0 con 0x559807f75b20 2024-09-19T08:39:49.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25663b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559807f76440 0x559807f86670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:49.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25663b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559807f76440 0x559807f86670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57114/0 (socket says 172.21.15.98:57114) 2024-09-19T08:39:49.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25663b640 1 -- 172.21.15.98:0/3191619048 learned_addr learned my addr 172.21.15.98:0/3191619048 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:49.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd255639640 1 --2- 172.21.15.98:0/3191619048 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x559807f75b20 0x559807f75200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:49.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25663b640 1 -- 172.21.15.98:0/3191619048 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x559807f75b20 msgr2=0x559807f75200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:49.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd255e3a640 1 --2- 172.21.15.98:0/3191619048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559807f31390 0x559807f74cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:49.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25663b640 1 --2- 172.21.15.98:0/3191619048 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x559807f75b20 0x559807f75200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25663b640 1 -- 172.21.15.98:0/3191619048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559807f31390 msgr2=0x559807f74cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:49.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25663b640 1 --2- 172.21.15.98:0/3191619048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559807f31390 0x559807f74cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25663b640 1 -- 172.21.15.98:0/3191619048 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559807d01fd0 con 0x559807f76440 2024-09-19T08:39:49.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25663b640 1 --2- 172.21.15.98:0/3191619048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559807f76440 0x559807f86670 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x559807e32ac0 tx=0x7fd248012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9798179ca60e916d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:49.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd232ffd640 1 -- 172.21.15.98:0/3191619048 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd24800d940 con 0x559807f76440 2024-09-19T08:39:49.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd232ffd640 1 -- 172.21.15.98:0/3191619048 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd248022070 con 0x559807f76440 2024-09-19T08:39:49.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd232ffd640 1 -- 172.21.15.98:0/3191619048 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd24800dd20 con 0x559807f76440 2024-09-19T08:39:49.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25b15a000 1 -- 172.21.15.98:0/3191619048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559807f76440 msgr2=0x559807f86670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:49.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25b15a000 1 --2- 172.21.15.98:0/3191619048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559807f76440 0x559807f86670 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x559807e32ac0 tx=0x7fd248012000 comp rx=0 tx=0).stop 2024-09-19T08:39:49.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25b15a000 1 -- 172.21.15.98:0/3191619048 shutdown_connections 2024-09-19T08:39:49.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25b15a000 1 --2- 172.21.15.98:0/3191619048 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559807f76440 0x559807f86670 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25b15a000 1 --2- 172.21.15.98:0/3191619048 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x559807f75b20 0x559807f75200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25b15a000 1 --2- 172.21.15.98:0/3191619048 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559807f31390 0x559807f74cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25b15a000 1 -- 172.21.15.98:0/3191619048 >> 172.21.15.98:0/3191619048 conn(0x559807f2c290 msgr2=0x559807f30650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:49.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25b15a000 1 -- 172.21.15.98:0/3191619048 shutdown_connections 2024-09-19T08:39:49.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25b15a000 1 -- 172.21.15.98:0/3191619048 wait complete. 2024-09-19T08:39:49.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25b15a000 1 -- start start 2024-09-19T08:39:49.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25b15a000 1 Processor -- start 2024-09-19T08:39:49.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25b15a000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559807f31390 0x559807f89500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:49.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25b15a000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x559807f75b20 0x559807fa4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:49.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25b15a000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559807f76440 0x559807fa4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:49.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25b15a000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x559807d79f80 con 0x559807f76440 2024-09-19T08:39:49.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25b15a000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x559807d32ca0 con 0x559807f31390 2024-09-19T08:39:49.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25b15a000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x559807d7f2a0 con 0x559807f75b20 2024-09-19T08:39:49.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25b15a000 10 client.0 _ll_register_callbacks cb 0x7fd24c007e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:39:49.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25663b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559807f76440 0x559807fa4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:49.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25663b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559807f76440 0x559807fa4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57118/0 (socket says 172.21.15.98:57118) 2024-09-19T08:39:49.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25663b640 1 -- 172.21.15.98:0/3814331850 learned_addr learned my addr 172.21.15.98:0/3814331850 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:49.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd255e3a640 1 --2- 172.21.15.98:0/3814331850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559807f31390 0x559807f89500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:49.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd255639640 1 --2- 172.21.15.98:0/3814331850 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x559807f75b20 0x559807fa4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:49.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25663b640 1 -- 172.21.15.98:0/3814331850 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x559807f75b20 msgr2=0x559807fa4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:49.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25663b640 1 --2- 172.21.15.98:0/3814331850 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x559807f75b20 0x559807fa4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25663b640 1 -- 172.21.15.98:0/3814331850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559807f31390 msgr2=0x559807f89500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:49.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25663b640 1 --2- 172.21.15.98:0/3814331850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559807f31390 0x559807f89500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25663b640 1 -- 172.21.15.98:0/3814331850 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559807dfd750 con 0x559807f76440 2024-09-19T08:39:49.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd25663b640 1 --2- 172.21.15.98:0/3814331850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559807f76440 0x559807fa4ea0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd248000c10 tx=0x7fd248003700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe585c4cbd04228 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:49.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.107+0000 7fd230ff9640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-19T08:39:49.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.111+0000 7fd230ff9640 1 -- 172.21.15.98:0/3814331850 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2480069e0 con 0x559807f76440 2024-09-19T08:39:49.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.111+0000 7fd230ff9640 1 -- 172.21.15.98:0/3814331850 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fd248006b80 con 0x559807f76440 2024-09-19T08:39:49.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.111+0000 7fd25b15a000 1 -- 172.21.15.98:0/3814331850 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd248022070 con 0x559807f76440 2024-09-19T08:39:49.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.111+0000 7fd230ff9640 1 -- 172.21.15.98:0/3814331850 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd248006d80 con 0x559807f76440 2024-09-19T08:39:49.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.111+0000 7fd25b15a000 10 client.5415 fetch_fsmap learned FSMap version 23 2024-09-19T08:39:49.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.111+0000 7fd25b15a000 1 -- 172.21.15.98:0/3814331850 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) -- 0x559807d08a30 con 0x559807f76440 2024-09-19T08:39:49.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.111+0000 7fd230ff9640 1 -- 172.21.15.98:0/3814331850 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd248022070 con 0x559807f76440 2024-09-19T08:39:49.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.111+0000 7fd230ff9640 1 -- 172.21.15.98:0/3814331850 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 23) ==== 3502+0+0 (secure 0 0 0) 0x7fd248007030 con 0x559807f76440 2024-09-19T08:39:49.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.111+0000 7fd25b15a000 10 client.5415 fetch_fsmap finished waiting for FSMap version 23 2024-09-19T08:39:49.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.111+0000 7fd25b15a000 10 client.5415 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T08:39:49.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.111+0000 7fd25b15a000 20 client.5415 populate_metadata read hostname 'smithi098' 2024-09-19T08:39:49.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.111+0000 7fd25b15a000 1 --2- 172.21.15.98:0/3814331850 >> [v2:172.21.15.98:6836/3085042202,v1:172.21.15.98:6837/3085042202] conn(0x559807fa53e0 0x559807fe8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:49.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.111+0000 7fd25b15a000 4 client.5415 mds_command: new command op to 5244 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T08:39:49.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.111+0000 7fd25b15a000 1 -- 172.21.15.98:0/3814331850 --> [v2:172.21.15.98:6836/3085042202,v1:172.21.15.98:6837/3085042202] -- command(tid 0: {"prefix": "status"}) -- 0x559807d883a0 con 0x559807fa53e0 2024-09-19T08:39:49.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.111+0000 7fd255e3a640 1 --2- 172.21.15.98:0/3814331850 >> [v2:172.21.15.98:6836/3085042202,v1:172.21.15.98:6837/3085042202] conn(0x559807fa53e0 0x559807fe8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:49.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.111+0000 7fd255e3a640 1 --2- 172.21.15.98:0/3814331850 >> [v2:172.21.15.98:6836/3085042202,v1:172.21.15.98:6837/3085042202] conn(0x559807fa53e0 0x559807fe8d20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f9615a0ee750c2f server_cookie=d568dbf2d8e058a3 in_seq=0 out_seq=0 2024-09-19T08:39:49.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.111+0000 7fd230ff9640 10 client.5415 ms_handle_connect on v2:172.21.15.98:6836/3085042202 2024-09-19T08:39:49.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.111+0000 7fd230ff9640 1 -- 172.21.15.98:0/3814331850 <== mds.0 v2:172.21.15.98:6836/3085042202 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x559807d883a0 con 0x559807fa53e0 2024-09-19T08:39:49.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.111+0000 7fd230ff9640 10 client.5415 handle_command_reply: tid=0 2024-09-19T08:39:49.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.111+0000 7fd25b15a000 1 client.5415 shutdown 2024-09-19T08:39:49.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.111+0000 7fd25b15a000 1 -- 172.21.15.98:0/3814331850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559807f76440 msgr2=0x559807fa4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:49.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.111+0000 7fd25b15a000 1 --2- 172.21.15.98:0/3814331850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559807f76440 0x559807fa4ea0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd248000c10 tx=0x7fd248003700 comp rx=0 tx=0).stop 2024-09-19T08:39:49.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.111+0000 7fd25b15a000 1 -- 172.21.15.98:0/3814331850 shutdown_connections 2024-09-19T08:39:49.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.111+0000 7fd25b15a000 1 --2- 172.21.15.98:0/3814331850 >> [v2:172.21.15.98:6836/3085042202,v1:172.21.15.98:6837/3085042202] conn(0x559807fa53e0 0x559807fe8d20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.111+0000 7fd25b15a000 1 --2- 172.21.15.98:0/3814331850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x559807f76440 0x559807fa4ea0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.111+0000 7fd25b15a000 1 --2- 172.21.15.98:0/3814331850 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x559807f75b20 0x559807fa4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.111+0000 7fd230ff9640 0 client.5415 ms_handle_reset on v2:172.21.15.98:6836/3085042202 2024-09-19T08:39:49.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.111+0000 7fd25b15a000 1 --2- 172.21.15.98:0/3814331850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559807f31390 0x559807f89500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.111+0000 7fd25b15a000 1 -- 172.21.15.98:0/3814331850 >> 172.21.15.98:0/3814331850 conn(0x559807f2c290 msgr2=0x559807f73cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:49.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.111+0000 7fd25b15a000 1 -- 172.21.15.98:0/3814331850 shutdown_connections 2024-09-19T08:39:49.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.111+0000 7fd25b15a000 1 -- 172.21.15.98:0/3814331850 wait complete. 2024-09-19T08:39:49.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.111+0000 7fd25b15a000 20 client.5415 trim_cache size 0 max 16384 2024-09-19T08:39:49.118 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:39:49.118 INFO:teuthology.orchestra.run.smithi098.stdout: "cluster_fsid": "552bf0ee-1f4f-4644-ba7c-d0931e55a451", 2024-09-19T08:39:49.118 INFO:teuthology.orchestra.run.smithi098.stdout: "whoami": 0, 2024-09-19T08:39:49.119 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 5244, 2024-09-19T08:39:49.119 INFO:teuthology.orchestra.run.smithi098.stdout: "want_state": "up:active", 2024-09-19T08:39:49.119 INFO:teuthology.orchestra.run.smithi098.stdout: "state": "up:active", 2024-09-19T08:39:49.119 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "cephfs", 2024-09-19T08:39:49.119 INFO:teuthology.orchestra.run.smithi098.stdout: "rank_uptime": 10.185946314000001, 2024-09-19T08:39:49.119 INFO:teuthology.orchestra.run.smithi098.stdout: "mdsmap_epoch": 23, 2024-09-19T08:39:49.119 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch": 43, 2024-09-19T08:39:49.119 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch_barrier": 43, 2024-09-19T08:39:49.119 INFO:teuthology.orchestra.run.smithi098.stdout: "uptime": 14.261749504000001 2024-09-19T08:39:49.119 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:39:49.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.115+0000 7fd25809e640 1 -- 172.21.15.98:0/1336534483 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd234067a70 msgr2=0x7fd234069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:49.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.115+0000 7fd25809e640 1 --2- 172.21.15.98:0/1336534483 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd234067a70 0x7fd234069f30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd250165ca0 tx=0x7fd24c00b040 comp rx=0 tx=0).stop 2024-09-19T08:39:49.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.115+0000 7fd25809e640 1 -- 172.21.15.98:0/1336534483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2500b4360 msgr2=0x7fd250072f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:49.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.115+0000 7fd25809e640 1 --2- 172.21.15.98:0/1336534483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2500b4360 0x7fd250072f60 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd24001a050 tx=0x7fd24001ac60 comp rx=0 tx=0).stop 2024-09-19T08:39:49.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.115+0000 7fd25809e640 1 -- 172.21.15.98:0/1336534483 shutdown_connections 2024-09-19T08:39:49.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.115+0000 7fd25809e640 1 --2- 172.21.15.98:0/1336534483 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd234067a70 0x7fd234069f30 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.115+0000 7fd25809e640 1 --2- 172.21.15.98:0/1336534483 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd250158fd0 0x7fd2501548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.115+0000 7fd25809e640 1 --2- 172.21.15.98:0/1336534483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd2500b4360 0x7fd250072f60 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.115+0000 7fd25809e640 1 --2- 172.21.15.98:0/1336534483 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd25006c1d0 0x7fd250152e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.115+0000 7fd25809e640 1 -- 172.21.15.98:0/1336534483 >> 172.21.15.98:0/1336534483 conn(0x7fd2500b2010 msgr2=0x7fd250155ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:49.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.115+0000 7fd25809e640 1 -- 172.21.15.98:0/1336534483 shutdown_connections 2024-09-19T08:39:49.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.115+0000 7fd25809e640 1 -- 172.21.15.98:0/1336534483 wait complete. 2024-09-19T08:39:49.129 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T08:39:49.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.231+0000 7f933ad4b640 1 Processor -- start 2024-09-19T08:39:49.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.231+0000 7f933ad4b640 1 -- start start 2024-09-19T08:39:49.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.235+0000 7f933ad4b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9334158fd0 0x7f93340b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:49.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.235+0000 7f933ad4b640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93340ccc60 con 0x7f93340b4360 2024-09-19T08:39:49.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.235+0000 7f933ad4b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93340ccf40 con 0x7f933406c3a0 2024-09-19T08:39:49.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.235+0000 7f933ad4b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f93341593b0 con 0x7f9334158fd0 2024-09-19T08:39:49.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.235+0000 7f9333fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f93340b4360 0x7f933406bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:35836/0 (socket says 172.21.15.98:35836) 2024-09-19T08:39:49.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.235+0000 7f9333fff640 1 -- 172.21.15.98:0/3064079038 learned_addr learned my addr 172.21.15.98:0/3064079038 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:49.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.235+0000 7f9338ae7640 1 --2- 172.21.15.98:0/3064079038 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9334158fd0 0x7f93340b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:49.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.235+0000 7f93337fe640 1 -- 172.21.15.98:0/3064079038 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1091648298 0 0) 0x7f93340ccc60 con 0x7f93340b4360 2024-09-19T08:39:49.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.235+0000 7f93337fe640 1 -- 172.21.15.98:0/3064079038 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9320003530 con 0x7f93340b4360 2024-09-19T08:39:49.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.235+0000 7f93337fe640 1 -- 172.21.15.98:0/3064079038 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 44411006 0 0) 0x7f9320003530 con 0x7f93340b4360 2024-09-19T08:39:49.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.235+0000 7f93337fe640 1 -- 172.21.15.98:0/3064079038 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9334158fd0 msgr2=0x7f93340b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:49.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.235+0000 7f93337fe640 1 --2- 172.21.15.98:0/3064079038 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9334158fd0 0x7f93340b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.235+0000 7f93337fe640 1 -- 172.21.15.98:0/3064079038 >> v1:172.21.15.139:6789/0 conn(0x7f933406c3a0 legacy=0x7f93340ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:49.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.235+0000 7f93337fe640 1 -- 172.21.15.98:0/3064079038 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93340cd220 con 0x7f93340b4360 2024-09-19T08:39:49.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.235+0000 7f93337fe640 1 -- 172.21.15.98:0/3064079038 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f931c001680 con 0x7f93340b4360 2024-09-19T08:39:49.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.235+0000 7f93337fe640 1 -- 172.21.15.98:0/3064079038 >> v1:172.21.15.98:6789/0 conn(0x7f93340b4360 legacy=0x7f933406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:49.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.235+0000 7f93337fe640 1 --2- 172.21.15.98:0/3064079038 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9334158fd0 0x7f93200039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:49.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.235+0000 7f93337fe640 1 --2- 172.21.15.98:0/3064079038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9320003f70 0x7f9320004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:49.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.235+0000 7f93337fe640 1 --2- 172.21.15.98:0/3064079038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9320004d50 0x7f93200051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:49.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.235+0000 7f93337fe640 1 -- 172.21.15.98:0/3064079038 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f93341593b0 con 0x7f9320004d50 2024-09-19T08:39:49.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.235+0000 7f93337fe640 1 -- 172.21.15.98:0/3064079038 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f932000dd70 con 0x7f9320003f70 2024-09-19T08:39:49.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.235+0000 7f93337fe640 1 -- 172.21.15.98:0/3064079038 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f932000df70 con 0x7f9334158fd0 2024-09-19T08:39:49.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.235+0000 7f93392e8640 1 --2- 172.21.15.98:0/3064079038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9320004d50 0x7f93200051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:49.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.235+0000 7f9338ae7640 1 --2- 172.21.15.98:0/3064079038 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9334158fd0 0x7f93200039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:49.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.235+0000 7f9333fff640 1 --2- 172.21.15.98:0/3064079038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9320003f70 0x7f9320004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:49.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.235+0000 7f93392e8640 1 -- 172.21.15.98:0/3064079038 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9334158fd0 msgr2=0x7f93200039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:49.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.235+0000 7f93392e8640 1 --2- 172.21.15.98:0/3064079038 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9334158fd0 0x7f93200039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.235+0000 7f93392e8640 1 -- 172.21.15.98:0/3064079038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9320003f70 msgr2=0x7f9320004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:49.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.235+0000 7f93392e8640 1 --2- 172.21.15.98:0/3064079038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9320003f70 0x7f9320004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.235+0000 7f93392e8640 1 -- 172.21.15.98:0/3064079038 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9320003530 con 0x7f9320004d50 2024-09-19T08:39:49.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.235+0000 7f9333fff640 1 --2- 172.21.15.98:0/3064079038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9320003f70 0x7f9320004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:49.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.235+0000 7f93392e8640 1 --2- 172.21.15.98:0/3064079038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9320004d50 0x7f93200051d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f932c001780 tx=0x7f932c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eed6a666e149d19 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:49.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.235+0000 7f93337fe640 1 -- 172.21.15.98:0/3064079038 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932c0013a0 con 0x7f9320004d50 2024-09-19T08:39:49.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.235+0000 7f93337fe640 1 -- 172.21.15.98:0/3064079038 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f932c0a9070 con 0x7f9320004d50 2024-09-19T08:39:49.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.235+0000 7f93337fe640 1 -- 172.21.15.98:0/3064079038 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c001d50 con 0x7f9320004d50 2024-09-19T08:39:49.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.239+0000 7f933ad4b640 1 -- 172.21.15.98:0/3064079038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9320004d50 msgr2=0x7f93200051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:49.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.239+0000 7f933ad4b640 1 --2- 172.21.15.98:0/3064079038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9320004d50 0x7f93200051d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f932c001780 tx=0x7f932c01c420 comp rx=0 tx=0).stop 2024-09-19T08:39:49.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.239+0000 7f93392e8640 1 -- 172.21.15.98:0/3064079038 reap_dead start 2024-09-19T08:39:49.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.239+0000 7f933ad4b640 1 -- 172.21.15.98:0/3064079038 shutdown_connections 2024-09-19T08:39:49.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.239+0000 7f933ad4b640 1 -- 172.21.15.98:0/3064079038 >> 172.21.15.98:0/3064079038 conn(0x7f93340b2010 msgr2=0x7f93340b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:49.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.239+0000 7f933ad4b640 1 -- 172.21.15.98:0/3064079038 shutdown_connections 2024-09-19T08:39:49.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.239+0000 7f933ad4b640 1 -- 172.21.15.98:0/3064079038 wait complete. 2024-09-19T08:39:49.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.239+0000 7f933ad4b640 1 Processor -- start 2024-09-19T08:39:49.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.239+0000 7f933ad4b640 1 -- start start 2024-09-19T08:39:49.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.239+0000 7f933ad4b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9320004d50 0x7f9334150600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:49.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.239+0000 7f933ad4b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f933415d900 0x7f933414f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:49.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.239+0000 7f933ad4b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9334152cf0 0x7f9334153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:49.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.239+0000 7f933ad4b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f933415c7a0 con 0x7f9320004d50 2024-09-19T08:39:49.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.239+0000 7f933ad4b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f93340cd7e0 con 0x7f9334152cf0 2024-09-19T08:39:49.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.239+0000 7f933ad4b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f933415c920 con 0x7f933415d900 2024-09-19T08:39:49.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.239+0000 7f9338ae7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9320004d50 0x7f9334150600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:49.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.239+0000 7f9338ae7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9320004d50 0x7f9334150600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57128/0 (socket says 172.21.15.98:57128) 2024-09-19T08:39:49.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.239+0000 7f9338ae7640 1 -- 172.21.15.98:0/917789232 learned_addr learned my addr 172.21.15.98:0/917789232 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:49.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.239+0000 7f93392e8640 1 --2- 172.21.15.98:0/917789232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9334152cf0 0x7f9334153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:49.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.239+0000 7f9338ae7640 1 -- 172.21.15.98:0/917789232 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f933415d900 msgr2=0x7f933414f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:49.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.239+0000 7f9338ae7640 1 --2- 172.21.15.98:0/917789232 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f933415d900 0x7f933414f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.239+0000 7f9338ae7640 1 -- 172.21.15.98:0/917789232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9334152cf0 msgr2=0x7f9334153170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:49.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.239+0000 7f9338ae7640 1 --2- 172.21.15.98:0/917789232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9334152cf0 0x7f9334153170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.239+0000 7f9338ae7640 1 -- 172.21.15.98:0/917789232 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9334072ad0 con 0x7f9320004d50 2024-09-19T08:39:49.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.239+0000 7f93392e8640 1 --2- 172.21.15.98:0/917789232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9334152cf0 0x7f9334153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:49.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.239+0000 7f9338ae7640 1 --2- 172.21.15.98:0/917789232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9320004d50 0x7f9334150600 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9324007f60 tx=0x7f9324007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:49.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.239+0000 7f93317fa640 1 -- 172.21.15.98:0/917789232 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932401a020 con 0x7f9320004d50 2024-09-19T08:39:49.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.239+0000 7f93317fa640 1 -- 172.21.15.98:0/917789232 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9324016470 con 0x7f9320004d50 2024-09-19T08:39:49.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.239+0000 7f933ad4b640 1 -- 172.21.15.98:0/917789232 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9334072db0 con 0x7f9320004d50 2024-09-19T08:39:49.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.239+0000 7f93317fa640 1 -- 172.21.15.98:0/917789232 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9324016860 con 0x7f9320004d50 2024-09-19T08:39:49.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.239+0000 7f933ad4b640 1 -- 172.21.15.98:0/917789232 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93341536b0 con 0x7f9320004d50 2024-09-19T08:39:49.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.243+0000 7f933ad4b640 1 -- 172.21.15.98:0/917789232 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f933415de30 con 0x7f9320004d50 2024-09-19T08:39:49.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.243+0000 7f93317fa640 1 -- 172.21.15.98:0/917789232 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f932401d020 con 0x7f9320004d50 2024-09-19T08:39:49.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.243+0000 7f93317fa640 1 --2- 172.21.15.98:0/917789232 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9314067a70 0x7f9314069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:49.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.243+0000 7f93317fa640 1 -- 172.21.15.98:0/917789232 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f9324087180 con 0x7f9320004d50 2024-09-19T08:39:49.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.243+0000 7f9333fff640 1 --2- 172.21.15.98:0/917789232 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9314067a70 0x7f9314069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:49.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.243+0000 7f9333fff640 1 --2- 172.21.15.98:0/917789232 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9314067a70 0x7f9314069f30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f931c0017f0 tx=0x7f931c0011d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:49.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.247+0000 7f93317fa640 1 -- 172.21.15.98:0/917789232 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9324024030 con 0x7f9320004d50 2024-09-19T08:39:49.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.435+0000 7f933ad4b640 1 -- 172.21.15.98:0/917789232 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9334157cf0 con 0x7f9320004d50 2024-09-19T08:39:49.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.435+0000 7f93317fa640 1 -- 172.21.15.98:0/917789232 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) ==== 53+0+73 (secure 0 0 0) 0x7f9324057370 con 0x7f9320004d50 2024-09-19T08:39:49.436 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T08:39:49.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.439+0000 7f933ad4b640 1 -- 172.21.15.98:0/917789232 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9314067a70 msgr2=0x7f9314069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:49.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.439+0000 7f933ad4b640 1 --2- 172.21.15.98:0/917789232 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9314067a70 0x7f9314069f30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f931c0017f0 tx=0x7f931c0011d0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.439+0000 7f933ad4b640 1 -- 172.21.15.98:0/917789232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9320004d50 msgr2=0x7f9334150600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:49.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.439+0000 7f933ad4b640 1 --2- 172.21.15.98:0/917789232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9320004d50 0x7f9334150600 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9324007f60 tx=0x7f9324007f90 comp rx=0 tx=0).stop 2024-09-19T08:39:49.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.439+0000 7f933ad4b640 1 -- 172.21.15.98:0/917789232 shutdown_connections 2024-09-19T08:39:49.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.439+0000 7f933ad4b640 1 --2- 172.21.15.98:0/917789232 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9314067a70 0x7f9314069f30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.439+0000 7f933ad4b640 1 --2- 172.21.15.98:0/917789232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9334152cf0 0x7f9334153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.439+0000 7f933ad4b640 1 --2- 172.21.15.98:0/917789232 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f933415d900 0x7f933414f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.439+0000 7f933ad4b640 1 --2- 172.21.15.98:0/917789232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9320004d50 0x7f9334150600 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:49.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.439+0000 7f933ad4b640 1 -- 172.21.15.98:0/917789232 >> 172.21.15.98:0/917789232 conn(0x7f93340b2010 msgr2=0x7f933415ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:49.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.439+0000 7f933ad4b640 1 -- 172.21.15.98:0/917789232 shutdown_connections 2024-09-19T08:39:49.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49.439+0000 7f933ad4b640 1 -- 172.21.15.98:0/917789232 wait complete. 2024-09-19T08:39:49.454 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T08:39:49.454 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T08:39:49.454 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi098.front.sepia.ceph.com 2024-09-19T08:39:49.454 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T08:39:49.454 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T08:39:49.454 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T08:39:49.454 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T08:39:49.454 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T08:39:49.455 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:39:49.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:49.455 DEBUG:teuthology.orchestra.run.smithi098:> ip addr 2024-09-19T08:39:49.461 INFO:teuthology.orchestra.run.smithi098.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T08:39:49.461 INFO:teuthology.orchestra.run.smithi098.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T08:39:49.461 INFO:teuthology.orchestra.run.smithi098.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T08:39:49.461 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:39:49.461 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 ::1/128 scope host 2024-09-19T08:39:49.461 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:39:49.462 INFO:teuthology.orchestra.run.smithi098.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:39:49.462 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b8 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:39:49.462 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f0 2024-09-19T08:39:49.462 INFO:teuthology.orchestra.run.smithi098.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:39:49.462 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b9 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:39:49.462 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f1 2024-09-19T08:39:49.462 INFO:teuthology.orchestra.run.smithi098.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:39:49.462 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6a brd ff:ff:ff:ff:ff:ff 2024-09-19T08:39:49.462 INFO:teuthology.orchestra.run.smithi098.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T08:39:49.462 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6b brd ff:ff:ff:ff:ff:ff 2024-09-19T08:39:49.462 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.21.15.98/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T08:39:49.462 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft 42419sec preferred_lft 42419sec 2024-09-19T08:39:49.462 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::ec4:7aff:fe8f:d26b/64 scope link 2024-09-19T08:39:49.463 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:39:49.463 INFO:teuthology.orchestra.run.smithi098.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T08:39:49.463 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 02:42:98:dd:32:2d brd ff:ff:ff:ff:ff:ff 2024-09-19T08:39:49.463 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T08:39:49.463 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:39:49.463 INFO:teuthology.orchestra.run.smithi098.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T08:39:49.463 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-19T08:39:49.463 INFO:teuthology.orchestra.run.smithi098.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T08:39:49.463 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:39:49.463 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-19T08:39:49.463 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:39:49.463 INFO:teuthology.orchestra.run.smithi098.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:39:49.464 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:39:49.464 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-19T08:39:49.464 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:39:49.464 INFO:teuthology.orchestra.run.smithi098.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:39:49.464 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:39:49.464 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-19T08:39:49.464 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:39:49.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:49.465 DEBUG:teuthology.orchestra.run.smithi098:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T08:39:49.513 INFO:teuthology.orchestra.run.smithi098.stdout:1 2024-09-19T08:39:49.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:49.514 DEBUG:teuthology.orchestra.run.smithi098:> route 2024-09-19T08:39:49.564 INFO:teuthology.orchestra.run.smithi098.stdout:Kernel IP routing table 2024-09-19T08:39:49.564 INFO:teuthology.orchestra.run.smithi098.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T08:39:49.564 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T08:39:49.564 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T08:39:49.564 INFO:teuthology.orchestra.run.smithi098.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T08:39:49.565 INFO:teuthology.orchestra.run.smithi098.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T08:39:49.565 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T08:39:49.565 INFO:teuthology.orchestra.run.smithi098.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:39:49.565 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:39:49.565 INFO:teuthology.orchestra.run.smithi098.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:39:49.565 INFO:teuthology.orchestra.run.smithi098.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T08:39:49.565 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:39:49.565 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-09-19T08:39:49.565 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T08:39:49.566 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T08:39:49.566 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T08:39:49.566 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-09-19T08:39:49.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:39:49.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:39:49.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:49.681 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2024-09-19T08:39:49.729 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-19T08:39:49.729 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T08:39:49.729 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:39:49.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:49.730 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set brx.0 up 2024-09-19T08:39:49.782 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-19T08:39:49.782 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:39:49.782 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T08:39:49.829 INFO:teuthology.orchestra.run.smithi098.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:39:49.829 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:39:49.830 DEBUG:teuthology.orchestra.run.smithi098:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T08:39:49.876 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:39:49.926 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:39:49.974 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:39:49.974 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:39:49.974 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:39:49.974 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:39:49.974 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:39:49.974 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:39:49.974 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:39:49.975 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:39:49.975 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:39:49.975 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:39:49.975 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:39:49.975 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:39:49.975 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:39:49.975 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:39:49.975 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:39:49.975 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:39:49.975 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:39:49.975 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:39:49.975 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:39:49.975 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:39:49.976 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:39:49.976 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:39:49.976 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:39:49.976 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:39:49.976 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:39:49.976 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:39:49.976 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:39:49.976 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:39:49.976 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:39:49.976 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:39:49.976 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:39:49.976 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:39:49.976 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:39:49.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:49.978 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:39:50.024 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T08:39:50.025 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-19T08:39:50.074 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:39:50.087 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:39:50.138 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:39:50.138 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:39:50.139 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:39:50.139 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:39:50.139 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:39:50.139 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:39:50.139 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:39:50.139 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:39:50.139 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:39:50.139 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:39:50.139 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:39:50.139 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:39:50.139 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:39:50.139 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:39:50.139 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:39:50.140 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:39:50.140 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:39:50.140 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:39:50.140 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:39:50.140 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:39:50.140 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:39:50.140 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:39:50.140 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:39:50.140 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:39:50.140 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:39:50.140 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:39:50.140 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:39:50.141 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:39:50.141 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:39:50.141 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:39:50.141 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:39:50.141 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:39:50.141 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:39:50.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:50.141 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:39:50.152 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:2024-09-19T08:39:50.151+0000 7ff1195e5580 -1 init, newargv = 0x563790619180 newargc=15 2024-09-19T08:39:50.152 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:ceph-fuse[39257]: starting ceph client 2024-09-19T08:39:50.158 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:ceph-fuse[39257]: starting fuse 2024-09-19T08:39:51.258 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:39:51.274 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:39:51.323 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:39:51.323 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:39:51.323 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:39:51.323 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:39:51.323 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:39:51.323 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:39:51.323 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:39:51.323 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:39:51.323 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:39:51.323 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:39:51.323 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:39:51.323 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:39:51.323 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:39:51.324 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:39:51.324 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:39:51.324 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:39:51.324 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:39:51.324 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:39:51.324 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:39:51.324 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:39:51.324 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:39:51.324 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:39:51.324 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:39:51.324 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:39:51.324 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:39:51.324 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:39:51.325 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:39:51.325 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:39:51.325 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:39:51.325 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:39:51.325 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:39:51.325 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:39:51.325 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:39:51.325 INFO:teuthology.orchestra.run.smithi098.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-19T08:39:51.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:51.326 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:39:51.373 INFO:teuthology.orchestra.run.smithi098.stdout:45 2024-09-19T08:39:51.374 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-19T08:39:51.374 DEBUG:teuthology.orchestra.run.smithi098:> sudo stdin-killer -- python3 -c ' 2024-09-19T08:39:51.374 DEBUG:teuthology.orchestra.run.smithi098:> import glob 2024-09-19T08:39:51.374 DEBUG:teuthology.orchestra.run.smithi098:> import re 2024-09-19T08:39:51.374 DEBUG:teuthology.orchestra.run.smithi098:> import os 2024-09-19T08:39:51.374 DEBUG:teuthology.orchestra.run.smithi098:> import subprocess 2024-09-19T08:39:51.374 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:39:51.374 DEBUG:teuthology.orchestra.run.smithi098:> def _find_admin_socket(client_name): 2024-09-19T08:39:51.374 DEBUG:teuthology.orchestra.run.smithi098:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T08:39:51.374 DEBUG:teuthology.orchestra.run.smithi098:> files = glob.glob(asok_path) 2024-09-19T08:39:51.374 DEBUG:teuthology.orchestra.run.smithi098:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T08:39:51.375 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:39:51.375 DEBUG:teuthology.orchestra.run.smithi098:> # Given a non-glob path, it better be there 2024-09-19T08:39:51.375 DEBUG:teuthology.orchestra.run.smithi098:> if "*" not in asok_path: 2024-09-19T08:39:51.375 DEBUG:teuthology.orchestra.run.smithi098:> assert(len(files) == 1) 2024-09-19T08:39:51.375 DEBUG:teuthology.orchestra.run.smithi098:> return files[0] 2024-09-19T08:39:51.375 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:39:51.375 DEBUG:teuthology.orchestra.run.smithi098:> for f in files: 2024-09-19T08:39:51.375 DEBUG:teuthology.orchestra.run.smithi098:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T08:39:51.375 DEBUG:teuthology.orchestra.run.smithi098:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T08:39:51.375 DEBUG:teuthology.orchestra.run.smithi098:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T08:39:51.375 DEBUG:teuthology.orchestra.run.smithi098:> contents = proc_f.read() 2024-09-19T08:39:51.375 DEBUG:teuthology.orchestra.run.smithi098:> if mountpoint in contents: 2024-09-19T08:39:51.375 DEBUG:teuthology.orchestra.run.smithi098:> return f 2024-09-19T08:39:51.375 DEBUG:teuthology.orchestra.run.smithi098:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T08:39:51.375 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:39:51.376 DEBUG:teuthology.orchestra.run.smithi098:> print(_find_admin_socket("client.0")) 2024-09-19T08:39:51.376 DEBUG:teuthology.orchestra.run.smithi098:> ' 2024-09-19T08:39:51.498 INFO:teuthology.orchestra.run.smithi098.stdout:/var/run/ceph/ceph-client.0.39257.asok 2024-09-19T08:39:51.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:39:51.509 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.39257.asok 2024-09-19T08:39:51.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:51.510 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.39257.asok status 2024-09-19T08:39:51.665 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:39:51.666 INFO:teuthology.orchestra.run.smithi098.stdout: "metadata": { 2024-09-19T08:39:51.666 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T08:39:51.666 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T08:39:51.666 INFO:teuthology.orchestra.run.smithi098.stdout: "entity_id": "0", 2024-09-19T08:39:51.666 INFO:teuthology.orchestra.run.smithi098.stdout: "hostname": "smithi098", 2024-09-19T08:39:51.666 INFO:teuthology.orchestra.run.smithi098.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T08:39:51.666 INFO:teuthology.orchestra.run.smithi098.stdout: "pid": "39257", 2024-09-19T08:39:51.666 INFO:teuthology.orchestra.run.smithi098.stdout: "root": "/" 2024-09-19T08:39:51.666 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:39:51.666 INFO:teuthology.orchestra.run.smithi098.stdout: "dentry_count": 0, 2024-09-19T08:39:51.666 INFO:teuthology.orchestra.run.smithi098.stdout: "dentry_pinned_count": 0, 2024-09-19T08:39:51.666 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 5430, 2024-09-19T08:39:51.667 INFO:teuthology.orchestra.run.smithi098.stdout: "inst": { 2024-09-19T08:39:51.667 INFO:teuthology.orchestra.run.smithi098.stdout: "name": { 2024-09-19T08:39:51.667 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "client", 2024-09-19T08:39:51.667 INFO:teuthology.orchestra.run.smithi098.stdout: "num": 5430 2024-09-19T08:39:51.667 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:39:51.667 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": { 2024-09-19T08:39:51.667 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v1", 2024-09-19T08:39:51.667 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:39:51.667 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 1644656876 2024-09-19T08:39:51.667 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:39:51.667 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:39:51.668 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": { 2024-09-19T08:39:51.668 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v1", 2024-09-19T08:39:51.668 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:39:51.668 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 1644656876 2024-09-19T08:39:51.668 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:39:51.668 INFO:teuthology.orchestra.run.smithi098.stdout: "inst_str": "client.5430 192.168.0.1:0/1644656876", 2024-09-19T08:39:51.668 INFO:teuthology.orchestra.run.smithi098.stdout: "addr_str": "192.168.0.1:0/1644656876", 2024-09-19T08:39:51.668 INFO:teuthology.orchestra.run.smithi098.stdout: "inode_count": 1, 2024-09-19T08:39:51.668 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_epoch": 23, 2024-09-19T08:39:51.668 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_epoch": 43, 2024-09-19T08:39:51.668 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_epoch_barrier": 0, 2024-09-19T08:39:51.668 INFO:teuthology.orchestra.run.smithi098.stdout: "blocklisted": false, 2024-09-19T08:39:51.669 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "cephfs" 2024-09-19T08:39:51.669 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:39:51.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:51.679 DEBUG:teuthology.orchestra.run.smithi098:> stat --file-system '--printf=%T 2024-09-19T08:39:51.679 DEBUG:teuthology.orchestra.run.smithi098:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T08:39:51.685 INFO:teuthology.orchestra.run.smithi098.stdout:fuseblk 2024-09-19T08:39:51.685 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T08:39:51.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:51.686 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T08:39:51.742 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T08:39:51.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.911+0000 7feb3a396640 1 Processor -- start 2024-09-19T08:39:51.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.911+0000 7feb3a396640 1 -- start start 2024-09-19T08:39:51.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.911+0000 7feb3a396640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7feb34147770 0x7feb340b9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:51.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.911+0000 7feb3a396640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb340b6200 0x7feb340b66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:51.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.911+0000 7feb3a396640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb340b7fe0 con 0x7feb340b97d0 2024-09-19T08:39:51.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.911+0000 7feb3a396640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7feb340b85a0 con 0x7feb34147770 2024-09-19T08:39:51.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.911+0000 7feb3a396640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7feb340b82c0 con 0x7feb340b6200 2024-09-19T08:39:51.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.911+0000 7feb38933640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb340b6200 0x7feb340b66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:51.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.911+0000 7feb33fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7feb34147770 0x7feb340b9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:51.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.911+0000 7feb33fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7feb34147770 0x7feb340b9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:55640/0 (socket says 172.21.15.98:55640) 2024-09-19T08:39:51.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.911+0000 7feb33fff640 1 -- 172.21.15.98:0/937566780 learned_addr learned my addr 172.21.15.98:0/937566780 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:51.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.911+0000 7feb33fff640 1 -- 172.21.15.98:0/937566780 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb340b6200 msgr2=0x7feb340b66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:51.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.911+0000 7feb33fff640 1 --2- 172.21.15.98:0/937566780 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb340b6200 0x7feb340b66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:51.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.911+0000 7feb33fff640 1 -- 172.21.15.98:0/937566780 >> v1:172.21.15.139:6789/0 conn(0x7feb340b97d0 legacy=0x7feb340b7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:51.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.911+0000 7feb33fff640 1 -- 172.21.15.98:0/937566780 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb340b8a90 con 0x7feb34147770 2024-09-19T08:39:51.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.911+0000 7feb38933640 1 --2- 172.21.15.98:0/937566780 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb340b6200 0x7feb340b66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:39:51.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.915+0000 7feb33fff640 1 --2- 172.21.15.98:0/937566780 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb34147770 0x7feb340b9290 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7feb24009870 tx=0x7feb2401c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92be276fb9281871 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:51.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.915+0000 7feb32ffd640 1 -- 172.21.15.98:0/937566780 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb240b1070 con 0x7feb34147770 2024-09-19T08:39:51.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.915+0000 7feb32ffd640 1 -- 172.21.15.98:0/937566780 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb240a8050 con 0x7feb34147770 2024-09-19T08:39:51.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.915+0000 7feb32ffd640 1 -- 172.21.15.98:0/937566780 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb240ac860 con 0x7feb34147770 2024-09-19T08:39:51.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.915+0000 7feb3a396640 1 -- 172.21.15.98:0/937566780 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb34147770 msgr2=0x7feb340b9290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:51.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.915+0000 7feb3a396640 1 --2- 172.21.15.98:0/937566780 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb34147770 0x7feb340b9290 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7feb24009870 tx=0x7feb2401c4d0 comp rx=0 tx=0).stop 2024-09-19T08:39:51.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.915+0000 7feb3a396640 1 -- 172.21.15.98:0/937566780 shutdown_connections 2024-09-19T08:39:51.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.915+0000 7feb3a396640 1 --2- 172.21.15.98:0/937566780 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb340b6200 0x7feb340b66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:51.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.915+0000 7feb3a396640 1 --2- 172.21.15.98:0/937566780 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb34147770 0x7feb340b9290 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:51.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.915+0000 7feb3a396640 1 -- 172.21.15.98:0/937566780 >> 172.21.15.98:0/937566780 conn(0x7feb34074e40 msgr2=0x7feb340722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:51.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.915+0000 7feb3a396640 1 -- 172.21.15.98:0/937566780 shutdown_connections 2024-09-19T08:39:51.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.915+0000 7feb3a396640 1 -- 172.21.15.98:0/937566780 wait complete. 2024-09-19T08:39:51.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.915+0000 7feb3a396640 1 Processor -- start 2024-09-19T08:39:51.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.915+0000 7feb3a396640 1 -- start start 2024-09-19T08:39:51.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.919+0000 7feb3a396640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb340b6200 0x7feb34137a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:51.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.919+0000 7feb3a396640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb34147770 0x7feb3415df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:51.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.919+0000 7feb3a396640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb34144610 0x7feb341360d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:51.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.919+0000 7feb3a396640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7feb34147cd0 con 0x7feb34147770 2024-09-19T08:39:51.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.919+0000 7feb3a396640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feb34147b50 con 0x7feb34144610 2024-09-19T08:39:51.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.919+0000 7feb3a396640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7feb3416b1f0 con 0x7feb340b6200 2024-09-19T08:39:51.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.919+0000 7feb337fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb34147770 0x7feb3415df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:51.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.919+0000 7feb337fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb34147770 0x7feb3415df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57142/0 (socket says 172.21.15.98:57142) 2024-09-19T08:39:51.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.919+0000 7feb337fe640 1 -- 172.21.15.98:0/2845859540 learned_addr learned my addr 172.21.15.98:0/2845859540 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:51.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.919+0000 7feb38933640 1 --2- 172.21.15.98:0/2845859540 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb34144610 0x7feb341360d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:51.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.919+0000 7feb33fff640 1 --2- 172.21.15.98:0/2845859540 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb340b6200 0x7feb34137a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:51.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.919+0000 7feb337fe640 1 -- 172.21.15.98:0/2845859540 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb340b6200 msgr2=0x7feb34137a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:51.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.919+0000 7feb337fe640 1 --2- 172.21.15.98:0/2845859540 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb340b6200 0x7feb34137a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:51.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.919+0000 7feb337fe640 1 -- 172.21.15.98:0/2845859540 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb34144610 msgr2=0x7feb341360d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:51.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.919+0000 7feb337fe640 1 --2- 172.21.15.98:0/2845859540 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb34144610 0x7feb341360d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:51.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.919+0000 7feb337fe640 1 -- 172.21.15.98:0/2845859540 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb34137fb0 con 0x7feb34147770 2024-09-19T08:39:51.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.919+0000 7feb337fe640 1 --2- 172.21.15.98:0/2845859540 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb34147770 0x7feb3415df90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7feb2800c790 tx=0x7feb2800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:51.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.919+0000 7feb30ff9640 1 -- 172.21.15.98:0/2845859540 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb28016020 con 0x7feb34147770 2024-09-19T08:39:51.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.919+0000 7feb30ff9640 1 -- 172.21.15.98:0/2845859540 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb28006d60 con 0x7feb34147770 2024-09-19T08:39:51.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.919+0000 7feb30ff9640 1 -- 172.21.15.98:0/2845859540 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb28003bc0 con 0x7feb34147770 2024-09-19T08:39:51.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.919+0000 7feb3a396640 1 -- 172.21.15.98:0/2845859540 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb341369d0 con 0x7feb34147770 2024-09-19T08:39:51.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.919+0000 7feb3a396640 1 -- 172.21.15.98:0/2845859540 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb34136f90 con 0x7feb34147770 2024-09-19T08:39:51.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.919+0000 7feb3a396640 1 -- 172.21.15.98:0/2845859540 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb340b66b0 con 0x7feb34147770 2024-09-19T08:39:51.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.919+0000 7feb30ff9640 1 -- 172.21.15.98:0/2845859540 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7feb280050d0 con 0x7feb34147770 2024-09-19T08:39:51.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.919+0000 7feb30ff9640 1 --2- 172.21.15.98:0/2845859540 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7feb20067a70 0x7feb20069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:51.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.919+0000 7feb30ff9640 1 -- 172.21.15.98:0/2845859540 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7feb28085d10 con 0x7feb34147770 2024-09-19T08:39:51.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.919+0000 7feb33fff640 1 --2- 172.21.15.98:0/2845859540 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7feb20067a70 0x7feb20069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:51.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.923+0000 7feb33fff640 1 --2- 172.21.15.98:0/2845859540 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7feb20067a70 0x7feb20069f30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7feb240ae000 tx=0x7feb240a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:51.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:51.927+0000 7feb30ff9640 1 -- 172.21.15.98:0/2845859540 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7feb28020020 con 0x7feb34147770 2024-09-19T08:39:52.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:52.123+0000 7feb3a396640 1 -- 172.21.15.98:0/2845859540 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7feb34146050 con 0x7feb34147770 2024-09-19T08:39:52.122 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:52.123+0000 7feb30ff9640 1 -- 172.21.15.98:0/2845859540 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) ==== 53+0+73 (secure 0 0 0) 0x7feb28056660 con 0x7feb34147770 2024-09-19T08:39:52.122 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T08:39:52.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:52.127+0000 7feb3a396640 1 -- 172.21.15.98:0/2845859540 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7feb20067a70 msgr2=0x7feb20069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:52.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:52.127+0000 7feb3a396640 1 --2- 172.21.15.98:0/2845859540 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7feb20067a70 0x7feb20069f30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7feb240ae000 tx=0x7feb240a7000 comp rx=0 tx=0).stop 2024-09-19T08:39:52.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:52.127+0000 7feb3a396640 1 -- 172.21.15.98:0/2845859540 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb34147770 msgr2=0x7feb3415df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:52.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:52.127+0000 7feb3a396640 1 --2- 172.21.15.98:0/2845859540 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb34147770 0x7feb3415df90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7feb2800c790 tx=0x7feb2800cb90 comp rx=0 tx=0).stop 2024-09-19T08:39:52.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:52.127+0000 7feb3a396640 1 -- 172.21.15.98:0/2845859540 shutdown_connections 2024-09-19T08:39:52.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:52.127+0000 7feb3a396640 1 --2- 172.21.15.98:0/2845859540 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7feb20067a70 0x7feb20069f30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:52.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:52.127+0000 7feb3a396640 1 --2- 172.21.15.98:0/2845859540 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb34144610 0x7feb341360d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:52.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:52.127+0000 7feb3a396640 1 --2- 172.21.15.98:0/2845859540 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb34147770 0x7feb3415df90 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:52.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:52.127+0000 7feb3a396640 1 --2- 172.21.15.98:0/2845859540 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb340b6200 0x7feb34137a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:52.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:52.127+0000 7feb3a396640 1 -- 172.21.15.98:0/2845859540 >> 172.21.15.98:0/2845859540 conn(0x7feb34074e40 msgr2=0x7feb3413e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:52.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:52.127+0000 7feb3a396640 1 -- 172.21.15.98:0/2845859540 shutdown_connections 2024-09-19T08:39:52.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:52.127+0000 7feb3a396640 1 -- 172.21.15.98:0/2845859540 wait complete. 2024-09-19T08:39:52.141 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T08:39:52.142 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T08:39:52.142 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2024-09-19T08:39:52.142 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-19T08:39:52.142 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-19T08:39:52.142 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T08:39:52.142 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T08:39:52.142 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T08:39:52.142 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T08:39:52.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:52.143 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2024-09-19T08:39:52.150 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T08:39:52.150 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T08:39:52.150 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T08:39:52.150 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:39:52.150 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-09-19T08:39:52.150 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:39:52.150 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:39:52.150 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:39:52.151 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2024-09-19T08:39:52.151 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:39:52.151 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:39:52.151 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2024-09-19T08:39:52.151 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:39:52.151 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:39:52.151 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T08:39:52.151 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:39:52.151 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T08:39:52.151 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 42428sec preferred_lft 42428sec 2024-09-19T08:39:52.151 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::ec4:7aff:fe88:7f59/64 scope link 2024-09-19T08:39:52.151 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:39:52.151 INFO:teuthology.orchestra.run.smithi139.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T08:39:52.151 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 02:42:74:ed:e4:60 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:39:52.152 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T08:39:52.152 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:39:52.152 INFO:teuthology.orchestra.run.smithi139.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T08:39:52.152 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-19T08:39:52.152 INFO:teuthology.orchestra.run.smithi139.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T08:39:52.152 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:39:52.152 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-19T08:39:52.152 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:39:52.152 INFO:teuthology.orchestra.run.smithi139.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:39:52.152 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:39:52.152 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-19T08:39:52.152 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:39:52.153 INFO:teuthology.orchestra.run.smithi139.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:39:52.153 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:39:52.153 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-19T08:39:52.153 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:39:52.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:52.153 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T08:39:52.207 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-09-19T08:39:52.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:52.208 DEBUG:teuthology.orchestra.run.smithi139:> route 2024-09-19T08:39:52.261 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2024-09-19T08:39:52.262 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T08:39:52.262 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T08:39:52.262 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T08:39:52.262 INFO:teuthology.orchestra.run.smithi139.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T08:39:52.262 INFO:teuthology.orchestra.run.smithi139.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T08:39:52.262 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T08:39:52.262 INFO:teuthology.orchestra.run.smithi139.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:39:52.262 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:39:52.262 INFO:teuthology.orchestra.run.smithi139.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:39:52.262 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T08:39:52.262 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:39:52.263 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-09-19T08:39:52.263 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T08:39:52.263 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T08:39:52.263 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T08:39:52.263 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-09-19T08:39:52.353 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:39:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:39:52.383 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:39:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:39:52.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:52.391 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-09-19T08:39:52.396 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-19T08:39:52.397 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-19T08:39:52.397 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T08:39:52.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:52.397 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.0 up 2024-09-19T08:39:52.458 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-19T08:39:52.458 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:39:52.459 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-19T08:39:52.506 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-19T08:39:52.506 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:39:52.506 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-19T08:39:52.554 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:39:52.611 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:39:52.659 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:39:52.660 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:39:52.660 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:39:52.660 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:39:52.660 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:39:52.660 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:39:52.661 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:39:52.661 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:39:52.661 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:39:52.661 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:39:52.661 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:39:52.661 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:39:52.661 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:39:52.661 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:39:52.662 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:39:52.662 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:39:52.662 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:39:52.662 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:39:52.662 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:39:52.662 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:39:52.663 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:39:52.663 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:39:52.663 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:39:52.663 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:39:52.663 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:39:52.663 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:39:52.663 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:39:52.663 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:39:52.663 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:39:52.663 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:39:52.663 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:39:52.663 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:39:52.663 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:39:52.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:52.664 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:39:52.706 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T08:39:52.706 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-19T08:39:52.751 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:39:52.765 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:39:52.818 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:39:52.818 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:39:52.818 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:39:52.818 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:39:52.818 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:39:52.818 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:39:52.818 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:39:52.818 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:39:52.819 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:39:52.819 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:39:52.819 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:39:52.819 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:39:52.819 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:39:52.819 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:39:52.819 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:39:52.819 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:39:52.819 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:39:52.819 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:39:52.819 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:39:52.819 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:39:52.819 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:39:52.819 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:39:52.820 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:39:52.820 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:39:52.820 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:39:52.820 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:39:52.820 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:39:52.820 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:39:52.820 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:39:52.820 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:39:52.820 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:39:52.820 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:39:52.820 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:39:52.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:52.821 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:39:52.831 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:2024-09-19T08:39:52.828+0000 7f63a0590580 -1 init, newargv = 0x55e2e9aa4ff0 newargc=15 2024-09-19T08:39:52.831 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[35408]: starting ceph client 2024-09-19T08:39:52.838 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[35408]: starting fuse 2024-09-19T08:39:53.937 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:39:53.952 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:39:53.999 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:39:53.999 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:39:53.999 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:39:53.999 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:39:53.999 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:39:53.999 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:39:53.999 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:39:53.999 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:39:53.999 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:39:53.999 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:39:53.999 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:39:53.999 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:39:54.000 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:39:54.000 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:39:54.000 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:39:54.000 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:39:54.000 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:39:54.000 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:39:54.000 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:39:54.000 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:39:54.000 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:39:54.000 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:39:54.000 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:39:54.000 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:39:54.000 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:39:54.000 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:39:54.001 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:39:54.001 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:39:54.001 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:39:54.001 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:39:54.001 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:39:54.001 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:39:54.001 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:39:54.001 INFO:teuthology.orchestra.run.smithi139.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-19T08:39:54.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:54.003 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:39:54.050 INFO:teuthology.orchestra.run.smithi139.stdout:45 2024-09-19T08:39:54.050 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-19T08:39:54.050 DEBUG:teuthology.orchestra.run.smithi139:> sudo stdin-killer -- python3 -c ' 2024-09-19T08:39:54.050 DEBUG:teuthology.orchestra.run.smithi139:> import glob 2024-09-19T08:39:54.050 DEBUG:teuthology.orchestra.run.smithi139:> import re 2024-09-19T08:39:54.051 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-09-19T08:39:54.051 DEBUG:teuthology.orchestra.run.smithi139:> import subprocess 2024-09-19T08:39:54.051 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:39:54.051 DEBUG:teuthology.orchestra.run.smithi139:> def _find_admin_socket(client_name): 2024-09-19T08:39:54.051 DEBUG:teuthology.orchestra.run.smithi139:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-19T08:39:54.051 DEBUG:teuthology.orchestra.run.smithi139:> files = glob.glob(asok_path) 2024-09-19T08:39:54.051 DEBUG:teuthology.orchestra.run.smithi139:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-19T08:39:54.051 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:39:54.051 DEBUG:teuthology.orchestra.run.smithi139:> # Given a non-glob path, it better be there 2024-09-19T08:39:54.051 DEBUG:teuthology.orchestra.run.smithi139:> if "*" not in asok_path: 2024-09-19T08:39:54.051 DEBUG:teuthology.orchestra.run.smithi139:> assert(len(files) == 1) 2024-09-19T08:39:54.052 DEBUG:teuthology.orchestra.run.smithi139:> return files[0] 2024-09-19T08:39:54.052 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:39:54.052 DEBUG:teuthology.orchestra.run.smithi139:> for f in files: 2024-09-19T08:39:54.052 DEBUG:teuthology.orchestra.run.smithi139:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T08:39:54.052 DEBUG:teuthology.orchestra.run.smithi139:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T08:39:54.052 DEBUG:teuthology.orchestra.run.smithi139:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T08:39:54.052 DEBUG:teuthology.orchestra.run.smithi139:> contents = proc_f.read() 2024-09-19T08:39:54.052 DEBUG:teuthology.orchestra.run.smithi139:> if mountpoint in contents: 2024-09-19T08:39:54.052 DEBUG:teuthology.orchestra.run.smithi139:> return f 2024-09-19T08:39:54.052 DEBUG:teuthology.orchestra.run.smithi139:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T08:39:54.052 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:39:54.052 DEBUG:teuthology.orchestra.run.smithi139:> print(_find_admin_socket("client.1")) 2024-09-19T08:39:54.052 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-09-19T08:39:54.174 INFO:teuthology.orchestra.run.smithi139.stdout:/var/run/ceph/ceph-client.1.35408.asok 2024-09-19T08:39:54.179 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:39:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:39:54.188 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.35408.asok 2024-09-19T08:39:54.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:54.188 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.35408.asok status 2024-09-19T08:39:54.342 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-19T08:39:54.343 INFO:teuthology.orchestra.run.smithi139.stdout: "metadata": { 2024-09-19T08:39:54.343 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T08:39:54.343 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T08:39:54.343 INFO:teuthology.orchestra.run.smithi139.stdout: "entity_id": "1", 2024-09-19T08:39:54.343 INFO:teuthology.orchestra.run.smithi139.stdout: "hostname": "smithi139", 2024-09-19T08:39:54.343 INFO:teuthology.orchestra.run.smithi139.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-19T08:39:54.343 INFO:teuthology.orchestra.run.smithi139.stdout: "pid": "35408", 2024-09-19T08:39:54.343 INFO:teuthology.orchestra.run.smithi139.stdout: "root": "/" 2024-09-19T08:39:54.343 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:39:54.344 INFO:teuthology.orchestra.run.smithi139.stdout: "dentry_count": 0, 2024-09-19T08:39:54.344 INFO:teuthology.orchestra.run.smithi139.stdout: "dentry_pinned_count": 0, 2024-09-19T08:39:54.344 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 5116, 2024-09-19T08:39:54.344 INFO:teuthology.orchestra.run.smithi139.stdout: "inst": { 2024-09-19T08:39:54.344 INFO:teuthology.orchestra.run.smithi139.stdout: "name": { 2024-09-19T08:39:54.344 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "client", 2024-09-19T08:39:54.344 INFO:teuthology.orchestra.run.smithi139.stdout: "num": 5116 2024-09-19T08:39:54.344 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:39:54.344 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": { 2024-09-19T08:39:54.344 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-19T08:39:54.344 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:39:54.344 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1654456460 2024-09-19T08:39:54.345 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-19T08:39:54.345 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:39:54.345 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": { 2024-09-19T08:39:54.345 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-19T08:39:54.345 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:39:54.345 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1654456460 2024-09-19T08:39:54.345 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:39:54.345 INFO:teuthology.orchestra.run.smithi139.stdout: "inst_str": "client.5116 192.168.0.1:0/1654456460", 2024-09-19T08:39:54.345 INFO:teuthology.orchestra.run.smithi139.stdout: "addr_str": "192.168.0.1:0/1654456460", 2024-09-19T08:39:54.345 INFO:teuthology.orchestra.run.smithi139.stdout: "inode_count": 1, 2024-09-19T08:39:54.345 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_epoch": 23, 2024-09-19T08:39:54.345 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_epoch": 43, 2024-09-19T08:39:54.345 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_epoch_barrier": 0, 2024-09-19T08:39:54.345 INFO:teuthology.orchestra.run.smithi139.stdout: "blocklisted": false, 2024-09-19T08:39:54.346 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs" 2024-09-19T08:39:54.346 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-19T08:39:54.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:54.356 DEBUG:teuthology.orchestra.run.smithi139:> stat --file-system '--printf=%T 2024-09-19T08:39:54.356 DEBUG:teuthology.orchestra.run.smithi139:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-19T08:39:54.362 INFO:teuthology.orchestra.run.smithi139.stdout:fuseblk 2024-09-19T08:39:54.363 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-19T08:39:54.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:54.363 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-19T08:39:54.421 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-19T08:39:54.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.559+0000 7fb5da5ec640 1 Processor -- start 2024-09-19T08:39:54.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.563+0000 7fb5da5ec640 1 -- start start 2024-09-19T08:39:54.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.563+0000 7fb5da5ec640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb5d40cf700 0x7fb5d415ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:54.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.563+0000 7fb5da5ec640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb5d406d4c0 0x7fb5d406d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:54.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.563+0000 7fb5da5ec640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5d41608c0 con 0x7fb5d415c290 2024-09-19T08:39:54.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.563+0000 7fb5da5ec640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fb5d406de70 con 0x7fb5d406d4c0 2024-09-19T08:39:54.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.563+0000 7fb5da5ec640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb5d4160e80 con 0x7fb5d40cf700 2024-09-19T08:39:54.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.563+0000 7fb5d8b89640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb5d406d4c0 0x7fb5d406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:54.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.563+0000 7fb5d8b89640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb5d406d4c0 0x7fb5d406d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57150/0 (socket says 172.21.15.98:57150) 2024-09-19T08:39:54.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.563+0000 7fb5d8b89640 1 -- 172.21.15.98:0/1107293297 learned_addr learned my addr 172.21.15.98:0/1107293297 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:54.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.563+0000 7fb5d37fe640 1 --2- 172.21.15.98:0/1107293297 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb5d40cf700 0x7fb5d415ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:54.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.563+0000 7fb5d8b89640 1 -- 172.21.15.98:0/1107293297 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb5d40cf700 msgr2=0x7fb5d415ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:54.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.563+0000 7fb5d8b89640 1 --2- 172.21.15.98:0/1107293297 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb5d40cf700 0x7fb5d415ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:54.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.563+0000 7fb5d8b89640 1 -- 172.21.15.98:0/1107293297 >> v1:172.21.15.139:6789/0 conn(0x7fb5d415c290 legacy=0x7fb5d40cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:54.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.563+0000 7fb5d8b89640 1 -- 172.21.15.98:0/1107293297 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d4160ba0 con 0x7fb5d406d4c0 2024-09-19T08:39:54.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.563+0000 7fb5d8b89640 1 --2- 172.21.15.98:0/1107293297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5d406d4c0 0x7fb5d406d930 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc009e50 tx=0x7fb5cc01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=172367694e273a62 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:54.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.567+0000 7fb5d2ffd640 1 -- 172.21.15.98:0/1107293297 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc0a5670 con 0x7fb5d406d4c0 2024-09-19T08:39:54.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.567+0000 7fb5d2ffd640 1 -- 172.21.15.98:0/1107293297 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5cc0a5810 con 0x7fb5d406d4c0 2024-09-19T08:39:54.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.567+0000 7fb5d2ffd640 1 -- 172.21.15.98:0/1107293297 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc0a5be0 con 0x7fb5d406d4c0 2024-09-19T08:39:54.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.567+0000 7fb5da5ec640 1 -- 172.21.15.98:0/1107293297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5d406d4c0 msgr2=0x7fb5d406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:54.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.567+0000 7fb5da5ec640 1 --2- 172.21.15.98:0/1107293297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5d406d4c0 0x7fb5d406d930 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc009e50 tx=0x7fb5cc01c9f0 comp rx=0 tx=0).stop 2024-09-19T08:39:54.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.567+0000 7fb5da5ec640 1 -- 172.21.15.98:0/1107293297 shutdown_connections 2024-09-19T08:39:54.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.567+0000 7fb5da5ec640 1 --2- 172.21.15.98:0/1107293297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5d406d4c0 0x7fb5d406d930 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:54.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.567+0000 7fb5da5ec640 1 --2- 172.21.15.98:0/1107293297 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb5d40cf700 0x7fb5d415ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:54.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.567+0000 7fb5da5ec640 1 -- 172.21.15.98:0/1107293297 >> 172.21.15.98:0/1107293297 conn(0x7fb5d406c090 msgr2=0x7fb5d406c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:54.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.567+0000 7fb5da5ec640 1 -- 172.21.15.98:0/1107293297 shutdown_connections 2024-09-19T08:39:54.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.567+0000 7fb5da5ec640 1 -- 172.21.15.98:0/1107293297 wait complete. 2024-09-19T08:39:54.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.567+0000 7fb5da5ec640 1 Processor -- start 2024-09-19T08:39:54.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.567+0000 7fb5da5ec640 1 -- start start 2024-09-19T08:39:54.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.567+0000 7fb5da5ec640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5d406d4c0 0x7fb5d41f72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:54.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.567+0000 7fb5da5ec640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5d40cf700 0x7fb5d415db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:54.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.567+0000 7fb5da5ec640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb5d415c290 0x7fb5d4158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:54.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.567+0000 7fb5da5ec640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d40b5580 con 0x7fb5d406d4c0 2024-09-19T08:39:54.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.567+0000 7fb5da5ec640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d40b5400 con 0x7fb5d40cf700 2024-09-19T08:39:54.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.567+0000 7fb5da5ec640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb5d41690c0 con 0x7fb5d415c290 2024-09-19T08:39:54.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.567+0000 7fb5d3fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5d406d4c0 0x7fb5d41f72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:54.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.567+0000 7fb5d3fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5d406d4c0 0x7fb5d41f72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57156/0 (socket says 172.21.15.98:57156) 2024-09-19T08:39:54.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.567+0000 7fb5d3fff640 1 -- 172.21.15.98:0/2480017984 learned_addr learned my addr 172.21.15.98:0/2480017984 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:54.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.567+0000 7fb5d8b89640 1 --2- 172.21.15.98:0/2480017984 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb5d415c290 0x7fb5d4158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:54.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.567+0000 7fb5d3fff640 1 -- 172.21.15.98:0/2480017984 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb5d415c290 msgr2=0x7fb5d4158dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:54.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.567+0000 7fb5d37fe640 1 --2- 172.21.15.98:0/2480017984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5d40cf700 0x7fb5d415db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:54.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.567+0000 7fb5d3fff640 1 --2- 172.21.15.98:0/2480017984 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb5d415c290 0x7fb5d4158dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:54.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.567+0000 7fb5d3fff640 1 -- 172.21.15.98:0/2480017984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5d40cf700 msgr2=0x7fb5d415db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:54.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.567+0000 7fb5d3fff640 1 --2- 172.21.15.98:0/2480017984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5d40cf700 0x7fb5d415db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:54.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.567+0000 7fb5d3fff640 1 -- 172.21.15.98:0/2480017984 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d40720e0 con 0x7fb5d406d4c0 2024-09-19T08:39:54.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.567+0000 7fb5d8b89640 1 --2- 172.21.15.98:0/2480017984 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb5d415c290 0x7fb5d4158dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:54.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.567+0000 7fb5d3fff640 1 --2- 172.21.15.98:0/2480017984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5d406d4c0 0x7fb5d41f72f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb5c800b410 tx=0x7fb5c800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:54.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.567+0000 7fb5d0ff9640 1 -- 172.21.15.98:0/2480017984 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8015030 con 0x7fb5d406d4c0 2024-09-19T08:39:54.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.567+0000 7fb5d0ff9640 1 -- 172.21.15.98:0/2480017984 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5c8004020 con 0x7fb5d406d4c0 2024-09-19T08:39:54.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.567+0000 7fb5da5ec640 1 -- 172.21.15.98:0/2480017984 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5d40ad270 con 0x7fb5d406d4c0 2024-09-19T08:39:54.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.567+0000 7fb5d0ff9640 1 -- 172.21.15.98:0/2480017984 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8004450 con 0x7fb5d406d4c0 2024-09-19T08:39:54.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.567+0000 7fb5da5ec640 1 -- 172.21.15.98:0/2480017984 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5d40ad830 con 0x7fb5d406d4c0 2024-09-19T08:39:54.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.571+0000 7fb5da5ec640 1 -- 172.21.15.98:0/2480017984 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5d406d930 con 0x7fb5d406d4c0 2024-09-19T08:39:54.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.571+0000 7fb5d0ff9640 1 -- 172.21.15.98:0/2480017984 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fb5c8018070 con 0x7fb5d406d4c0 2024-09-19T08:39:54.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.571+0000 7fb5d0ff9640 1 --2- 172.21.15.98:0/2480017984 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb5c0067a70 0x7fb5c0069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:54.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.571+0000 7fb5d0ff9640 1 -- 172.21.15.98:0/2480017984 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fb5c8085ac0 con 0x7fb5d406d4c0 2024-09-19T08:39:54.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.571+0000 7fb5d37fe640 1 --2- 172.21.15.98:0/2480017984 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb5c0067a70 0x7fb5c0069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:54.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.571+0000 7fb5d37fe640 1 --2- 172.21.15.98:0/2480017984 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb5c0067a70 0x7fb5c0069f30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb5d40af750 tx=0x7fb5c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:54.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.575+0000 7fb5d0ff9640 1 -- 172.21.15.98:0/2480017984 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb5c8056360 con 0x7fb5d406d4c0 2024-09-19T08:39:54.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:54.775+0000 7fb5da5ec640 1 -- 172.21.15.98:0/2480017984 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fb5d40b1c00 con 0x7fb5d406d4c0 2024-09-19T08:39:55.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.183+0000 7fb5d0ff9640 1 -- 172.21.15.98:0/2480017984 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v24) ==== 136+0+0 (secure 0 0 0) 0x7fb5c8055c60 con 0x7fb5d406d4c0 2024-09-19T08:39:55.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.183+0000 7fb5da5ec640 1 -- 172.21.15.98:0/2480017984 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb5c0067a70 msgr2=0x7fb5c0069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:55.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.183+0000 7fb5da5ec640 1 --2- 172.21.15.98:0/2480017984 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb5c0067a70 0x7fb5c0069f30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb5d40af750 tx=0x7fb5c4008040 comp rx=0 tx=0).stop 2024-09-19T08:39:55.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.183+0000 7fb5da5ec640 1 -- 172.21.15.98:0/2480017984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5d406d4c0 msgr2=0x7fb5d41f72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:55.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.183+0000 7fb5da5ec640 1 --2- 172.21.15.98:0/2480017984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5d406d4c0 0x7fb5d41f72f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb5c800b410 tx=0x7fb5c800b8d0 comp rx=0 tx=0).stop 2024-09-19T08:39:55.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.183+0000 7fb5da5ec640 1 -- 172.21.15.98:0/2480017984 shutdown_connections 2024-09-19T08:39:55.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.183+0000 7fb5da5ec640 1 --2- 172.21.15.98:0/2480017984 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb5c0067a70 0x7fb5c0069f30 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:55.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.183+0000 7fb5da5ec640 1 --2- 172.21.15.98:0/2480017984 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb5d415c290 0x7fb5d4158dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:55.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.183+0000 7fb5da5ec640 1 --2- 172.21.15.98:0/2480017984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5d40cf700 0x7fb5d415db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:55.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.183+0000 7fb5da5ec640 1 --2- 172.21.15.98:0/2480017984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb5d406d4c0 0x7fb5d41f72f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:55.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.187+0000 7fb5da5ec640 1 -- 172.21.15.98:0/2480017984 >> 172.21.15.98:0/2480017984 conn(0x7fb5d406c090 msgr2=0x7fb5d40cfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:55.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.187+0000 7fb5da5ec640 1 -- 172.21.15.98:0/2480017984 shutdown_connections 2024-09-19T08:39:55.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.187+0000 7fb5da5ec640 1 -- 172.21.15.98:0/2480017984 wait complete. 2024-09-19T08:39:55.204 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-19T08:39:55.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.327+0000 7f3486689640 1 Processor -- start 2024-09-19T08:39:55.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.327+0000 7f3486689640 1 -- start start 2024-09-19T08:39:55.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.327+0000 7f3486689640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3480158fd0 0x7f34800b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:55.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.327+0000 7f3486689640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f348006c3a0 0x7f34800ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:55.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.327+0000 7f3486689640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34800ccc60 con 0x7f34800b4360 2024-09-19T08:39:55.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.327+0000 7f3486689640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f34801593b0 con 0x7f348006c3a0 2024-09-19T08:39:55.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.327+0000 7f3486689640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f34800ccf40 con 0x7f3480158fd0 2024-09-19T08:39:55.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.327+0000 7f347ffff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3480158fd0 0x7f34800b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:55.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.327+0000 7f347ffff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3480158fd0 0x7f34800b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:33884/0 (socket says 172.21.15.98:33884) 2024-09-19T08:39:55.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.327+0000 7f347ffff640 1 -- 172.21.15.98:0/1519385108 learned_addr learned my addr 172.21.15.98:0/1519385108 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:55.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.327+0000 7f3484c26640 1 --2- 172.21.15.98:0/1519385108 >> v2:172.21.15.139:3300/0 conn(0x7f348006c3a0 0x7f34800ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:55.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.327+0000 7f347ffff640 1 -- 172.21.15.98:0/1519385108 >> v2:172.21.15.139:3300/0 conn(0x7f348006c3a0 msgr2=0x7f34800ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:55.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.327+0000 7f347ffff640 1 --2- 172.21.15.98:0/1519385108 >> v2:172.21.15.139:3300/0 conn(0x7f348006c3a0 0x7f34800ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:55.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.327+0000 7f347ffff640 1 -- 172.21.15.98:0/1519385108 >> v1:172.21.15.139:6789/0 conn(0x7f34800b4360 legacy=0x7f348006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:55.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.327+0000 7f347ffff640 1 -- 172.21.15.98:0/1519385108 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34800cd220 con 0x7f3480158fd0 2024-09-19T08:39:55.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.331+0000 7f347ffff640 1 --2- 172.21.15.98:0/1519385108 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3480158fd0 0x7f34800b3e20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3470009870 tx=0x7f347001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7405ec4e91cfd1b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:55.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.331+0000 7f347f7fe640 1 -- 172.21.15.98:0/1519385108 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34700aa600 con 0x7f3480158fd0 2024-09-19T08:39:55.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.331+0000 7f347f7fe640 1 -- 172.21.15.98:0/1519385108 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34700af070 con 0x7f3480158fd0 2024-09-19T08:39:55.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.331+0000 7f347f7fe640 1 -- 172.21.15.98:0/1519385108 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34700aa9c0 con 0x7f3480158fd0 2024-09-19T08:39:55.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.331+0000 7f3486689640 1 -- 172.21.15.98:0/1519385108 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3480158fd0 msgr2=0x7f34800b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:55.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.331+0000 7f3486689640 1 --2- 172.21.15.98:0/1519385108 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3480158fd0 0x7f34800b3e20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3470009870 tx=0x7f347001c4a0 comp rx=0 tx=0).stop 2024-09-19T08:39:55.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.331+0000 7f3486689640 1 -- 172.21.15.98:0/1519385108 shutdown_connections 2024-09-19T08:39:55.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.331+0000 7f3486689640 1 --2- 172.21.15.98:0/1519385108 >> v2:172.21.15.139:3300/0 conn(0x7f348006c3a0 0x7f34800ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:55.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.331+0000 7f3486689640 1 --2- 172.21.15.98:0/1519385108 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3480158fd0 0x7f34800b3e20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:55.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.331+0000 7f3486689640 1 -- 172.21.15.98:0/1519385108 >> 172.21.15.98:0/1519385108 conn(0x7f34800b2010 msgr2=0x7f34800b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:55.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.331+0000 7f3486689640 1 -- 172.21.15.98:0/1519385108 shutdown_connections 2024-09-19T08:39:55.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.331+0000 7f3486689640 1 -- 172.21.15.98:0/1519385108 wait complete. 2024-09-19T08:39:55.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.331+0000 7f3486689640 1 Processor -- start 2024-09-19T08:39:55.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.331+0000 7f3486689640 1 -- start start 2024-09-19T08:39:55.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.335+0000 7f3486689640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f348006c3a0 0x7f3480157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:55.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.335+0000 7f3486689640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f34800b4360 0x7f3480159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:55.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.335+0000 7f3486689640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3480158fd0 0x7f3480144f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:55.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.335+0000 7f3486689640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f34800cf710 con 0x7f3480158fd0 2024-09-19T08:39:55.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.335+0000 7f3486689640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f34800cf590 con 0x7f348006c3a0 2024-09-19T08:39:55.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.335+0000 7f3486689640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f34800cf890 con 0x7f34800b4360 2024-09-19T08:39:55.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.335+0000 7f3484c26640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3480158fd0 0x7f3480144f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:55.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.335+0000 7f3484c26640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3480158fd0 0x7f3480144f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57158/0 (socket says 172.21.15.98:57158) 2024-09-19T08:39:55.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.335+0000 7f3484c26640 1 -- 172.21.15.98:0/624163430 learned_addr learned my addr 172.21.15.98:0/624163430 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:55.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.335+0000 7f347ffff640 1 --2- 172.21.15.98:0/624163430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f348006c3a0 0x7f3480157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:55.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.335+0000 7f34777fe640 1 --2- 172.21.15.98:0/624163430 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f34800b4360 0x7f3480159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:55.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.335+0000 7f3484c26640 1 -- 172.21.15.98:0/624163430 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f34800b4360 msgr2=0x7f3480159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:55.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.335+0000 7f3484c26640 1 --2- 172.21.15.98:0/624163430 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f34800b4360 0x7f3480159f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:55.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.335+0000 7f3484c26640 1 -- 172.21.15.98:0/624163430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f348006c3a0 msgr2=0x7f3480157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:55.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.335+0000 7f3484c26640 1 --2- 172.21.15.98:0/624163430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f348006c3a0 0x7f3480157c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:55.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.335+0000 7f3484c26640 1 -- 172.21.15.98:0/624163430 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f348015a4b0 con 0x7f3480158fd0 2024-09-19T08:39:55.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.335+0000 7f3484c26640 1 --2- 172.21.15.98:0/624163430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3480158fd0 0x7f3480144f20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3478009dd0 tx=0x7f347800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:55.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.335+0000 7f347d7fa640 1 -- 172.21.15.98:0/624163430 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3478018020 con 0x7f3480158fd0 2024-09-19T08:39:55.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.335+0000 7f347d7fa640 1 -- 172.21.15.98:0/624163430 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3478014480 con 0x7f3480158fd0 2024-09-19T08:39:55.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.335+0000 7f3486689640 1 -- 172.21.15.98:0/624163430 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3480145460 con 0x7f3480158fd0 2024-09-19T08:39:55.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.335+0000 7f347d7fa640 1 -- 172.21.15.98:0/624163430 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3478014870 con 0x7f3480158fd0 2024-09-19T08:39:55.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.335+0000 7f3486689640 1 -- 172.21.15.98:0/624163430 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f348015aa30 con 0x7f3480158fd0 2024-09-19T08:39:55.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.335+0000 7f3486689640 1 -- 172.21.15.98:0/624163430 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f348006dfc0 con 0x7f3480158fd0 2024-09-19T08:39:55.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.335+0000 7f347d7fa640 1 -- 172.21.15.98:0/624163430 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f347801b020 con 0x7f3480158fd0 2024-09-19T08:39:55.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.335+0000 7f347d7fa640 1 --2- 172.21.15.98:0/624163430 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3460067a70 0x7f3460069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:55.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.335+0000 7f347ffff640 1 --2- 172.21.15.98:0/624163430 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3460067a70 0x7f3460069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:55.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.335+0000 7f347d7fa640 1 -- 172.21.15.98:0/624163430 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f3478087180 con 0x7f3480158fd0 2024-09-19T08:39:55.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.339+0000 7f347ffff640 1 --2- 172.21.15.98:0/624163430 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3460067a70 0x7f3460069f30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f34700ae000 tx=0x7f34700a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:55.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.339+0000 7f347d7fa640 1 -- 172.21.15.98:0/624163430 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3478022070 con 0x7f3480158fd0 2024-09-19T08:39:55.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.523+0000 7f3486689640 1 -- 172.21.15.98:0/624163430 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f348015ad70 con 0x7f3460067a70 2024-09-19T08:39:55.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.527+0000 7f347d7fa640 1 -- 172.21.15.98:0/624163430 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f348015ad70 con 0x7f3460067a70 2024-09-19T08:39:55.527 INFO:teuthology.orchestra.run.smithi098.stdout:[ 2024-09-19T08:39:55.527 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:39:55.527 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "cephfs" 2024-09-19T08:39:55.527 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:39:55.527 INFO:teuthology.orchestra.run.smithi098.stdout:] 2024-09-19T08:39:55.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.531+0000 7f3486689640 1 -- 172.21.15.98:0/624163430 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3460067a70 msgr2=0x7f3460069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:55.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.531+0000 7f3486689640 1 --2- 172.21.15.98:0/624163430 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3460067a70 0x7f3460069f30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f34700ae000 tx=0x7f34700a7000 comp rx=0 tx=0).stop 2024-09-19T08:39:55.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.531+0000 7f3486689640 1 -- 172.21.15.98:0/624163430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3480158fd0 msgr2=0x7f3480144f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:55.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.531+0000 7f3486689640 1 --2- 172.21.15.98:0/624163430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3480158fd0 0x7f3480144f20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3478009dd0 tx=0x7f347800c490 comp rx=0 tx=0).stop 2024-09-19T08:39:55.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.531+0000 7f3486689640 1 -- 172.21.15.98:0/624163430 shutdown_connections 2024-09-19T08:39:55.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.531+0000 7f3486689640 1 --2- 172.21.15.98:0/624163430 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3460067a70 0x7f3460069f30 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:55.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.531+0000 7f3486689640 1 --2- 172.21.15.98:0/624163430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3480158fd0 0x7f3480144f20 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:55.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.531+0000 7f3486689640 1 --2- 172.21.15.98:0/624163430 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f34800b4360 0x7f3480159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:55.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.531+0000 7f3486689640 1 --2- 172.21.15.98:0/624163430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f348006c3a0 0x7f3480157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:55.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.531+0000 7f3486689640 1 -- 172.21.15.98:0/624163430 >> 172.21.15.98:0/624163430 conn(0x7f34800b2010 msgr2=0x7f348015bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:55.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.531+0000 7f3486689640 1 -- 172.21.15.98:0/624163430 shutdown_connections 2024-09-19T08:39:55.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.531+0000 7f3486689640 1 -- 172.21.15.98:0/624163430 wait complete. 2024-09-19T08:39:55.546 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T08:39:55.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.671+0000 7f273694b640 1 Processor -- start 2024-09-19T08:39:55.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.671+0000 7f273694b640 1 -- start start 2024-09-19T08:39:55.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.671+0000 7f273694b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2730144d10 0x7f27300722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:55.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.671+0000 7f273694b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2730072820 0x7f273006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:55.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.671+0000 7f273694b640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2730072ea0 con 0x7f273006c1d0 2024-09-19T08:39:55.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.671+0000 7f273694b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f2730072c60 con 0x7f2730144d10 2024-09-19T08:39:55.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.671+0000 7f273694b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2730073ab0 con 0x7f2730072820 2024-09-19T08:39:55.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.671+0000 7f2734ee8640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f273006c1d0 0x7f27300b4520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:35842/0 (socket says 172.21.15.98:35842) 2024-09-19T08:39:55.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.671+0000 7f2734ee8640 1 -- 172.21.15.98:0/2730686428 learned_addr learned my addr 172.21.15.98:0/2730686428 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:55.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.671+0000 7f272ffff640 1 --2- 172.21.15.98:0/2730686428 >> v2:172.21.15.139:3300/0 conn(0x7f2730144d10 0x7f27300722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:55.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.671+0000 7f2727fff640 1 --2- 172.21.15.98:0/2730686428 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2730072820 0x7f273006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:55.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.671+0000 7f272f7fe640 1 -- 172.21.15.98:0/2730686428 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 39731491 0 0) 0x7f2730072ea0 con 0x7f273006c1d0 2024-09-19T08:39:55.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.671+0000 7f272f7fe640 1 -- 172.21.15.98:0/2730686428 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f271c003530 con 0x7f273006c1d0 2024-09-19T08:39:55.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.671+0000 7f272f7fe640 1 -- 172.21.15.98:0/2730686428 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4051141552 0 0) 0x7f271c003530 con 0x7f273006c1d0 2024-09-19T08:39:55.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.671+0000 7f272f7fe640 1 -- 172.21.15.98:0/2730686428 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2730072820 msgr2=0x7f273006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:55.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.671+0000 7f272f7fe640 1 --2- 172.21.15.98:0/2730686428 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2730072820 0x7f273006bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:55.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.671+0000 7f272f7fe640 1 -- 172.21.15.98:0/2730686428 >> v2:172.21.15.139:3300/0 conn(0x7f2730144d10 msgr2=0x7f27300722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:55.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.671+0000 7f272f7fe640 1 --2- 172.21.15.98:0/2730686428 >> v2:172.21.15.139:3300/0 conn(0x7f2730144d10 0x7f27300722e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:55.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.671+0000 7f272f7fe640 1 -- 172.21.15.98:0/2730686428 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2730073180 con 0x7f273006c1d0 2024-09-19T08:39:55.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.671+0000 7f272f7fe640 1 -- 172.21.15.98:0/2730686428 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f2728001a00 con 0x7f273006c1d0 2024-09-19T08:39:55.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.671+0000 7f272f7fe640 1 -- 172.21.15.98:0/2730686428 >> v1:172.21.15.98:6789/0 conn(0x7f273006c1d0 legacy=0x7f27300b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:55.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.671+0000 7f272f7fe640 1 --2- 172.21.15.98:0/2730686428 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2730072820 0x7f271c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:55.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.671+0000 7f272f7fe640 1 --2- 172.21.15.98:0/2730686428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f271c003f40 0x7f271c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:55.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.671+0000 7f272f7fe640 1 --2- 172.21.15.98:0/2730686428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f271c004d50 0x7f271c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:55.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.671+0000 7f272f7fe640 1 -- 172.21.15.98:0/2730686428 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2730072c60 con 0x7f271c003f40 2024-09-19T08:39:55.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.671+0000 7f272f7fe640 1 -- 172.21.15.98:0/2730686428 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2730073ab0 con 0x7f271c004d50 2024-09-19T08:39:55.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.671+0000 7f272f7fe640 1 -- 172.21.15.98:0/2730686428 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f271c00b9d0 con 0x7f2730072820 2024-09-19T08:39:55.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.671+0000 7f2727fff640 1 --2- 172.21.15.98:0/2730686428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f271c003f40 0x7f271c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:55.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.671+0000 7f272ffff640 1 --2- 172.21.15.98:0/2730686428 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2730072820 0x7f271c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:55.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.671+0000 7f2734ee8640 1 --2- 172.21.15.98:0/2730686428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f271c004d50 0x7f271c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:55.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.671+0000 7f2727fff640 1 -- 172.21.15.98:0/2730686428 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2730072820 msgr2=0x7f271c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:55.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.671+0000 7f2727fff640 1 --2- 172.21.15.98:0/2730686428 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2730072820 0x7f271c0039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:55.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.671+0000 7f2727fff640 1 -- 172.21.15.98:0/2730686428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f271c004d50 msgr2=0x7f271c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:55.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.671+0000 7f2727fff640 1 --2- 172.21.15.98:0/2730686428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f271c004d50 0x7f271c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:55.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.671+0000 7f2727fff640 1 -- 172.21.15.98:0/2730686428 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f271c003530 con 0x7f271c003f40 2024-09-19T08:39:55.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.671+0000 7f2734ee8640 1 --2- 172.21.15.98:0/2730686428 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f271c004d50 0x7f271c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:55.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.675+0000 7f2727fff640 1 --2- 172.21.15.98:0/2730686428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f271c003f40 0x7f271c004810 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2718009ab0 tx=0x7f271801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10dd5f9fa7ec958e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:55.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.675+0000 7f272f7fe640 1 -- 172.21.15.98:0/2730686428 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27180b2070 con 0x7f271c003f40 2024-09-19T08:39:55.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.675+0000 7f272f7fe640 1 -- 172.21.15.98:0/2730686428 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27180a5540 con 0x7f271c003f40 2024-09-19T08:39:55.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.675+0000 7f272f7fe640 1 -- 172.21.15.98:0/2730686428 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27180096b0 con 0x7f271c003f40 2024-09-19T08:39:55.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.675+0000 7f273694b640 1 -- 172.21.15.98:0/2730686428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f271c003f40 msgr2=0x7f271c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:55.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.675+0000 7f273694b640 1 --2- 172.21.15.98:0/2730686428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f271c003f40 0x7f271c004810 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2718009ab0 tx=0x7f271801c9f0 comp rx=0 tx=0).stop 2024-09-19T08:39:55.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.675+0000 7f2734ee8640 1 -- 172.21.15.98:0/2730686428 reap_dead start 2024-09-19T08:39:55.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.675+0000 7f273694b640 1 -- 172.21.15.98:0/2730686428 shutdown_connections 2024-09-19T08:39:55.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.675+0000 7f273694b640 1 -- 172.21.15.98:0/2730686428 >> 172.21.15.98:0/2730686428 conn(0x7f27300b2010 msgr2=0x7f27300b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:55.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.675+0000 7f273694b640 1 -- 172.21.15.98:0/2730686428 shutdown_connections 2024-09-19T08:39:55.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.675+0000 7f273694b640 1 -- 172.21.15.98:0/2730686428 wait complete. 2024-09-19T08:39:55.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.675+0000 7f273694b640 1 Processor -- start 2024-09-19T08:39:55.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.675+0000 7f273694b640 1 -- start start 2024-09-19T08:39:55.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.679+0000 7f273694b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27301e68f0 0x7f27301f4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:55.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.679+0000 7f273694b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27301f4870 0x7f27301e55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:55.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.679+0000 7f273694b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f27301e5ae0 0x7f27301e6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:55.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.679+0000 7f273694b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2730073740 con 0x7f27301e68f0 2024-09-19T08:39:55.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.679+0000 7f273694b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f27301450f0 con 0x7f27301f4870 2024-09-19T08:39:55.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.679+0000 7f273694b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f27300af8b0 con 0x7f27301e5ae0 2024-09-19T08:39:55.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.679+0000 7f272ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27301e68f0 0x7f27301f4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:55.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.679+0000 7f272ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27301e68f0 0x7f27301f4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57182/0 (socket says 172.21.15.98:57182) 2024-09-19T08:39:55.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.679+0000 7f272ffff640 1 -- 172.21.15.98:0/2795372515 learned_addr learned my addr 172.21.15.98:0/2795372515 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:55.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.679+0000 7f2734ee8640 1 --2- 172.21.15.98:0/2795372515 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f27301e5ae0 0x7f27301e6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:55.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.679+0000 7f2727fff640 1 --2- 172.21.15.98:0/2795372515 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27301f4870 0x7f27301e55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:55.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.679+0000 7f272ffff640 1 -- 172.21.15.98:0/2795372515 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f27301e5ae0 msgr2=0x7f27301e6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:55.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.679+0000 7f272ffff640 1 --2- 172.21.15.98:0/2795372515 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f27301e5ae0 0x7f27301e6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:55.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.679+0000 7f272ffff640 1 -- 172.21.15.98:0/2795372515 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27301f4870 msgr2=0x7f27301e55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:55.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.679+0000 7f272ffff640 1 --2- 172.21.15.98:0/2795372515 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27301f4870 0x7f27301e55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:55.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.679+0000 7f272ffff640 1 -- 172.21.15.98:0/2795372515 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27301e5f70 con 0x7f27301e68f0 2024-09-19T08:39:55.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.679+0000 7f272ffff640 1 --2- 172.21.15.98:0/2795372515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27301e68f0 0x7f27301f4330 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2720010900 tx=0x7f2720010dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:55.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.679+0000 7f272d7fa640 1 -- 172.21.15.98:0/2795372515 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272000f040 con 0x7f27301e68f0 2024-09-19T08:39:55.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.679+0000 7f272d7fa640 1 -- 172.21.15.98:0/2795372515 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f272000ec20 con 0x7f27301e68f0 2024-09-19T08:39:55.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.679+0000 7f273694b640 1 -- 172.21.15.98:0/2795372515 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27301e4500 con 0x7f27301e68f0 2024-09-19T08:39:55.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.679+0000 7f272d7fa640 1 -- 172.21.15.98:0/2795372515 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272001c400 con 0x7f27301e68f0 2024-09-19T08:39:55.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.679+0000 7f273694b640 1 -- 172.21.15.98:0/2795372515 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27301e4ac0 con 0x7f27301e68f0 2024-09-19T08:39:55.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.679+0000 7f273694b640 1 -- 172.21.15.98:0/2795372515 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27300722e0 con 0x7f27301e68f0 2024-09-19T08:39:55.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.679+0000 7f272d7fa640 1 -- 172.21.15.98:0/2795372515 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f272001c5a0 con 0x7f27301e68f0 2024-09-19T08:39:55.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.679+0000 7f272d7fa640 1 --2- 172.21.15.98:0/2795372515 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2710067a70 0x7f2710069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:55.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.679+0000 7f272d7fa640 1 -- 172.21.15.98:0/2795372515 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f27200866d0 con 0x7f27301e68f0 2024-09-19T08:39:55.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.679+0000 7f2727fff640 1 --2- 172.21.15.98:0/2795372515 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2710067a70 0x7f2710069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:55.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.679+0000 7f2727fff640 1 --2- 172.21.15.98:0/2795372515 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2710067a70 0x7f2710069f30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f27180af000 tx=0x7f2718008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:55.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.683+0000 7f272d7fa640 1 -- 172.21.15.98:0/2795372515 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2720018030 con 0x7f27301e68f0 2024-09-19T08:39:55.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.831+0000 7f273694b640 1 -- 172.21.15.98:0/2795372515 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f2730074260 con 0x7f27301e68f0 2024-09-19T08:39:55.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.835+0000 7f272d7fa640 1 -- 172.21.15.98:0/2795372515 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f2720056f90 con 0x7f27301e68f0 2024-09-19T08:39:55.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.835+0000 7f273694b640 1 -- 172.21.15.98:0/2795372515 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2710067a70 msgr2=0x7f2710069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:55.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.835+0000 7f273694b640 1 --2- 172.21.15.98:0/2795372515 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2710067a70 0x7f2710069f30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f27180af000 tx=0x7f2718008040 comp rx=0 tx=0).stop 2024-09-19T08:39:55.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.835+0000 7f273694b640 1 -- 172.21.15.98:0/2795372515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27301e68f0 msgr2=0x7f27301f4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:55.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.835+0000 7f273694b640 1 --2- 172.21.15.98:0/2795372515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27301e68f0 0x7f27301f4330 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2720010900 tx=0x7f2720010dc0 comp rx=0 tx=0).stop 2024-09-19T08:39:55.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.835+0000 7f273694b640 1 -- 172.21.15.98:0/2795372515 shutdown_connections 2024-09-19T08:39:55.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.835+0000 7f273694b640 1 --2- 172.21.15.98:0/2795372515 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2710067a70 0x7f2710069f30 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:55.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.835+0000 7f273694b640 1 --2- 172.21.15.98:0/2795372515 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f27301e5ae0 0x7f27301e6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:55.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.835+0000 7f273694b640 1 --2- 172.21.15.98:0/2795372515 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27301f4870 0x7f27301e55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:55.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.835+0000 7f273694b640 1 --2- 172.21.15.98:0/2795372515 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27301e68f0 0x7f27301f4330 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:55.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.835+0000 7f273694b640 1 -- 172.21.15.98:0/2795372515 >> 172.21.15.98:0/2795372515 conn(0x7f27300b2010 msgr2=0x7f273006daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:55.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.835+0000 7f273694b640 1 -- 172.21.15.98:0/2795372515 shutdown_connections 2024-09-19T08:39:55.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:55.835+0000 7f273694b640 1 -- 172.21.15.98:0/2795372515 wait complete. 2024-09-19T08:39:55.850 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:39:55.850 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:39:55.855 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T08:39:56.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.015+0000 7ff75bad1640 1 Processor -- start 2024-09-19T08:39:56.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.015+0000 7ff75bad1640 1 -- start start 2024-09-19T08:39:56.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.015+0000 7ff75bad1640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff75406bc90 0x7ff75406c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:56.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.015+0000 7ff75bad1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7540ce840 0x7ff7540cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:56.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.015+0000 7ff75bad1640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7540ccc60 con 0x7ff754158fd0 2024-09-19T08:39:56.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.015+0000 7ff75bad1640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7ff7541593b0 con 0x7ff75406bc90 2024-09-19T08:39:56.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.015+0000 7ff75bad1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff7540ccf40 con 0x7ff7540ce840 2024-09-19T08:39:56.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.015+0000 7ff75906c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff75406bc90 0x7ff75406c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:56.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.015+0000 7ff75906c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff75406bc90 0x7ff75406c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57190/0 (socket says 172.21.15.98:57190) 2024-09-19T08:39:56.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.015+0000 7ff75906c640 1 -- 172.21.15.98:0/3972453278 learned_addr learned my addr 172.21.15.98:0/3972453278 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:56.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.015+0000 7ff75a06e640 1 --2- 172.21.15.98:0/3972453278 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7540ce840 0x7ff7540cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:56.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.015+0000 7ff75906c640 1 -- 172.21.15.98:0/3972453278 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7540ce840 msgr2=0x7ff7540cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:56.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.015+0000 7ff75906c640 1 --2- 172.21.15.98:0/3972453278 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7540ce840 0x7ff7540cec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:56.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.015+0000 7ff75906c640 1 -- 172.21.15.98:0/3972453278 >> v1:172.21.15.139:6789/0 conn(0x7ff754158fd0 legacy=0x7ff7540b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:56.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.015+0000 7ff75906c640 1 -- 172.21.15.98:0/3972453278 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7540cd220 con 0x7ff75406bc90 2024-09-19T08:39:56.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.019+0000 7ff75906c640 1 --2- 172.21.15.98:0/3972453278 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff75406bc90 0x7ff75406c120 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff744009870 tx=0x7ff74401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b316f8a705e41a2a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:56.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.019+0000 7ff75886b640 1 -- 172.21.15.98:0/3972453278 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7440b0070 con 0x7ff75406bc90 2024-09-19T08:39:56.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.019+0000 7ff75886b640 1 -- 172.21.15.98:0/3972453278 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7440ab600 con 0x7ff75406bc90 2024-09-19T08:39:56.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.019+0000 7ff75886b640 1 -- 172.21.15.98:0/3972453278 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7440ab9d0 con 0x7ff75406bc90 2024-09-19T08:39:56.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.019+0000 7ff75bad1640 1 -- 172.21.15.98:0/3972453278 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff75406bc90 msgr2=0x7ff75406c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:56.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.019+0000 7ff75bad1640 1 --2- 172.21.15.98:0/3972453278 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff75406bc90 0x7ff75406c120 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff744009870 tx=0x7ff74401c490 comp rx=0 tx=0).stop 2024-09-19T08:39:56.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.019+0000 7ff75bad1640 1 -- 172.21.15.98:0/3972453278 shutdown_connections 2024-09-19T08:39:56.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.019+0000 7ff75bad1640 1 --2- 172.21.15.98:0/3972453278 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7540ce840 0x7ff7540cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:56.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.019+0000 7ff75bad1640 1 --2- 172.21.15.98:0/3972453278 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff75406bc90 0x7ff75406c120 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:56.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.019+0000 7ff75bad1640 1 -- 172.21.15.98:0/3972453278 >> 172.21.15.98:0/3972453278 conn(0x7ff7540b2010 msgr2=0x7ff7540b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:56.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.019+0000 7ff75bad1640 1 -- 172.21.15.98:0/3972453278 shutdown_connections 2024-09-19T08:39:56.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.019+0000 7ff75bad1640 1 -- 172.21.15.98:0/3972453278 wait complete. 2024-09-19T08:39:56.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.019+0000 7ff75bad1640 1 Processor -- start 2024-09-19T08:39:56.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.019+0000 7ff75bad1640 1 -- start start 2024-09-19T08:39:56.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.023+0000 7ff75bad1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff75406bc90 0x7ff7540cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:56.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.023+0000 7ff75bad1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7540ce840 0x7ff754149900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:56.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.023+0000 7ff75bad1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff754158fd0 0x7ff754149e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:56.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.023+0000 7ff75bad1640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff75415c920 con 0x7ff75406bc90 2024-09-19T08:39:56.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.023+0000 7ff75bad1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff75415c7a0 con 0x7ff754158fd0 2024-09-19T08:39:56.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.023+0000 7ff75bad1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff754168110 con 0x7ff7540ce840 2024-09-19T08:39:56.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.023+0000 7ff75986d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff75406bc90 0x7ff7540cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:56.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.023+0000 7ff75986d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff75406bc90 0x7ff7540cc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57204/0 (socket says 172.21.15.98:57204) 2024-09-19T08:39:56.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.023+0000 7ff75986d640 1 -- 172.21.15.98:0/451475233 learned_addr learned my addr 172.21.15.98:0/451475233 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:56.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.023+0000 7ff75906c640 1 --2- 172.21.15.98:0/451475233 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7540ce840 0x7ff754149900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:56.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.023+0000 7ff75a06e640 1 --2- 172.21.15.98:0/451475233 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff754158fd0 0x7ff754149e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:56.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.023+0000 7ff75986d640 1 -- 172.21.15.98:0/451475233 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7540ce840 msgr2=0x7ff754149900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:56.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.023+0000 7ff75986d640 1 --2- 172.21.15.98:0/451475233 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7540ce840 0x7ff754149900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:56.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.023+0000 7ff75986d640 1 -- 172.21.15.98:0/451475233 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff754158fd0 msgr2=0x7ff754149e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:56.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.023+0000 7ff75986d640 1 --2- 172.21.15.98:0/451475233 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff754158fd0 0x7ff754149e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:56.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.023+0000 7ff75986d640 1 -- 172.21.15.98:0/451475233 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff75414ae40 con 0x7ff75406bc90 2024-09-19T08:39:56.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.023+0000 7ff75a06e640 1 --2- 172.21.15.98:0/451475233 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff754158fd0 0x7ff754149e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:56.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.023+0000 7ff75906c640 1 --2- 172.21.15.98:0/451475233 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7540ce840 0x7ff754149900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:39:56.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.023+0000 7ff75986d640 1 --2- 172.21.15.98:0/451475233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff75406bc90 0x7ff7540cc500 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff75000d990 tx=0x7ff75000de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:56.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.023+0000 7ff74a7fc640 1 -- 172.21.15.98:0/451475233 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75001b070 con 0x7ff75406bc90 2024-09-19T08:39:56.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.023+0000 7ff74a7fc640 1 -- 172.21.15.98:0/451475233 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff75000edc0 con 0x7ff75406bc90 2024-09-19T08:39:56.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.023+0000 7ff75bad1640 1 -- 172.21.15.98:0/451475233 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff754071db0 con 0x7ff75406bc90 2024-09-19T08:39:56.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.023+0000 7ff74a7fc640 1 -- 172.21.15.98:0/451475233 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff750016680 con 0x7ff75406bc90 2024-09-19T08:39:56.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.023+0000 7ff75bad1640 1 -- 172.21.15.98:0/451475233 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff754072370 con 0x7ff75406bc90 2024-09-19T08:39:56.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.023+0000 7ff75bad1640 1 -- 172.21.15.98:0/451475233 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff75406c120 con 0x7ff75406bc90 2024-09-19T08:39:56.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.023+0000 7ff74a7fc640 1 -- 172.21.15.98:0/451475233 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7ff75001e070 con 0x7ff75406bc90 2024-09-19T08:39:56.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.023+0000 7ff74a7fc640 1 --2- 172.21.15.98:0/451475233 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff738067a70 0x7ff738069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:56.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.023+0000 7ff74a7fc640 1 -- 172.21.15.98:0/451475233 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7ff750087c60 con 0x7ff75406bc90 2024-09-19T08:39:56.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.023+0000 7ff75906c640 1 --2- 172.21.15.98:0/451475233 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff738067a70 0x7ff738069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:56.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.023+0000 7ff75906c640 1 --2- 172.21.15.98:0/451475233 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff738067a70 0x7ff738069f30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff75406e310 tx=0x7ff744008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:56.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.027+0000 7ff74a7fc640 1 -- 172.21.15.98:0/451475233 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff7500585b0 con 0x7ff75406bc90 2024-09-19T08:39:56.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.199+0000 7ff75bad1640 1 -- 172.21.15.98:0/451475233 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff7540b41a0 con 0x7ff75406bc90 2024-09-19T08:39:56.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.199+0000 7ff74a7fc640 1 -- 172.21.15.98:0/451475233 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v43) ==== 80+0+0 (secure 0 0 0) 0x7ff750057eb0 con 0x7ff75406bc90 2024-09-19T08:39:56.199 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-19T08:39:56.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.203+0000 7ff75bad1640 1 -- 172.21.15.98:0/451475233 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff738067a70 msgr2=0x7ff738069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:56.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.203+0000 7ff75bad1640 1 --2- 172.21.15.98:0/451475233 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff738067a70 0x7ff738069f30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff75406e310 tx=0x7ff744008040 comp rx=0 tx=0).stop 2024-09-19T08:39:56.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.203+0000 7ff75bad1640 1 -- 172.21.15.98:0/451475233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff75406bc90 msgr2=0x7ff7540cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:56.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.203+0000 7ff75bad1640 1 --2- 172.21.15.98:0/451475233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff75406bc90 0x7ff7540cc500 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff75000d990 tx=0x7ff75000de50 comp rx=0 tx=0).stop 2024-09-19T08:39:56.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.203+0000 7ff75bad1640 1 -- 172.21.15.98:0/451475233 shutdown_connections 2024-09-19T08:39:56.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.203+0000 7ff75bad1640 1 --2- 172.21.15.98:0/451475233 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff738067a70 0x7ff738069f30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:56.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.203+0000 7ff75bad1640 1 --2- 172.21.15.98:0/451475233 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff754158fd0 0x7ff754149e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:56.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.203+0000 7ff75bad1640 1 --2- 172.21.15.98:0/451475233 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7540ce840 0x7ff754149900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:56.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.203+0000 7ff75bad1640 1 --2- 172.21.15.98:0/451475233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff75406bc90 0x7ff7540cc500 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:56.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.203+0000 7ff75bad1640 1 -- 172.21.15.98:0/451475233 >> 172.21.15.98:0/451475233 conn(0x7ff7540b2010 msgr2=0x7ff7540b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:56.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.203+0000 7ff75bad1640 1 -- 172.21.15.98:0/451475233 shutdown_connections 2024-09-19T08:39:56.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.203+0000 7ff75bad1640 1 -- 172.21.15.98:0/451475233 wait complete. 2024-09-19T08:39:56.217 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:39:56.217 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:39:56.222 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T08:39:56.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.383+0000 7fdea83f5640 1 Processor -- start 2024-09-19T08:39:56.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.383+0000 7fdea83f5640 1 -- start start 2024-09-19T08:39:56.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.383+0000 7fdea83f5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdea006c3a0 0x7fdea00cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:56.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.383+0000 7fdea83f5640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdea006d310 con 0x7fdea015c550 2024-09-19T08:39:56.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.383+0000 7fdea83f5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdea006d5f0 con 0x7fdea00cf900 2024-09-19T08:39:56.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.383+0000 7fdea83f5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fdea006d8d0 con 0x7fdea006c3a0 2024-09-19T08:39:56.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.383+0000 7fdea6191640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fdea015c550 0x7fdea006bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:35854/0 (socket says 172.21.15.98:35854) 2024-09-19T08:39:56.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.383+0000 7fdea6191640 1 -- 172.21.15.98:0/1579609518 learned_addr learned my addr 172.21.15.98:0/1579609518 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:39:56.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.383+0000 7fdea5990640 1 --2- 172.21.15.98:0/1579609518 >> v2:172.21.15.139:3300/0 conn(0x7fdea006c3a0 0x7fdea00cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:56.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.383+0000 7fdea518f640 1 -- 172.21.15.98:0/1579609518 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1746382923 0 0) 0x7fdea006d310 con 0x7fdea015c550 2024-09-19T08:39:56.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.383+0000 7fdea518f640 1 -- 172.21.15.98:0/1579609518 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde88003530 con 0x7fdea015c550 2024-09-19T08:39:56.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.383+0000 7fdea518f640 1 -- 172.21.15.98:0/1579609518 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1964297062 0 0) 0x7fde88003530 con 0x7fdea015c550 2024-09-19T08:39:56.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.383+0000 7fdea518f640 1 -- 172.21.15.98:0/1579609518 >> v2:172.21.15.139:3300/0 conn(0x7fdea006c3a0 msgr2=0x7fdea00cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:56.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.383+0000 7fdea518f640 1 --2- 172.21.15.98:0/1579609518 >> v2:172.21.15.139:3300/0 conn(0x7fdea006c3a0 0x7fdea00cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:56.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.383+0000 7fdea518f640 1 -- 172.21.15.98:0/1579609518 >> v1:172.21.15.139:6789/0 conn(0x7fdea00cf900 legacy=0x7fdea00723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:56.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.383+0000 7fdea518f640 1 -- 172.21.15.98:0/1579609518 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea006dbb0 con 0x7fdea015c550 2024-09-19T08:39:56.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.383+0000 7fdea518f640 1 -- 172.21.15.98:0/1579609518 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fde90001a00 con 0x7fdea015c550 2024-09-19T08:39:56.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.383+0000 7fdea518f640 1 -- 172.21.15.98:0/1579609518 >> v1:172.21.15.98:6789/0 conn(0x7fdea015c550 legacy=0x7fdea006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:56.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.383+0000 7fdea518f640 1 --2- 172.21.15.98:0/1579609518 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde88003a10 0x7fde88003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:56.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.383+0000 7fdea518f640 1 --2- 172.21.15.98:0/1579609518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde88004810 0x7fde88004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:56.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.383+0000 7fdea518f640 1 --2- 172.21.15.98:0/1579609518 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fde880051d0 0x7fde8800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:56.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.383+0000 7fdea518f640 1 -- 172.21.15.98:0/1579609518 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdea006d8d0 con 0x7fde88004810 2024-09-19T08:39:56.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.383+0000 7fdea518f640 1 -- 172.21.15.98:0/1579609518 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fde8800c0d0 con 0x7fde88003a10 2024-09-19T08:39:56.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.387+0000 7fdea518f640 1 -- 172.21.15.98:0/1579609518 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fde8800c2d0 con 0x7fde880051d0 2024-09-19T08:39:56.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.387+0000 7fdea5990640 1 --2- 172.21.15.98:0/1579609518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde88004810 0x7fde88004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:56.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.387+0000 7fdea6191640 1 --2- 172.21.15.98:0/1579609518 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde88003a10 0x7fde88003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:56.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.387+0000 7fdea5990640 1 -- 172.21.15.98:0/1579609518 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fde880051d0 msgr2=0x7fde8800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:56.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.387+0000 7fdea5990640 1 --2- 172.21.15.98:0/1579609518 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fde880051d0 0x7fde8800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:56.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.387+0000 7fdea5990640 1 -- 172.21.15.98:0/1579609518 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde88003a10 msgr2=0x7fde88003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:39:56.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.387+0000 7fdea5990640 1 --2- 172.21.15.98:0/1579609518 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde88003a10 0x7fde88003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:56.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.387+0000 7fdea5990640 1 -- 172.21.15.98:0/1579609518 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde88003530 con 0x7fde88004810 2024-09-19T08:39:56.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.387+0000 7fdea6992640 1 -- 172.21.15.98:0/1579609518 reap_dead start 2024-09-19T08:39:56.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.387+0000 7fdea5990640 1 --2- 172.21.15.98:0/1579609518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde88004810 0x7fde88004c90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fde94009ae0 tx=0x7fde9401ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c61005dcf3a52489 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:56.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.387+0000 7fdea518f640 1 -- 172.21.15.98:0/1579609518 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde940b2070 con 0x7fde88004810 2024-09-19T08:39:56.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.387+0000 7fdea518f640 1 -- 172.21.15.98:0/1579609518 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde940a5670 con 0x7fde88004810 2024-09-19T08:39:56.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.387+0000 7fdea518f640 1 -- 172.21.15.98:0/1579609518 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde940096b0 con 0x7fde88004810 2024-09-19T08:39:56.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.387+0000 7fdea83f5640 1 -- 172.21.15.98:0/1579609518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde88004810 msgr2=0x7fde88004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:56.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.387+0000 7fdea83f5640 1 --2- 172.21.15.98:0/1579609518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde88004810 0x7fde88004c90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fde94009ae0 tx=0x7fde9401ca20 comp rx=0 tx=0).stop 2024-09-19T08:39:56.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.387+0000 7fdea83f5640 1 -- 172.21.15.98:0/1579609518 shutdown_connections 2024-09-19T08:39:56.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.387+0000 7fdea83f5640 1 --2- 172.21.15.98:0/1579609518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fde88004810 0x7fde88004c90 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:56.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.387+0000 7fdea83f5640 1 -- 172.21.15.98:0/1579609518 >> 172.21.15.98:0/1579609518 conn(0x7fdea00b2010 msgr2=0x7fdea00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:56.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.387+0000 7fdea83f5640 1 -- 172.21.15.98:0/1579609518 shutdown_connections 2024-09-19T08:39:56.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.387+0000 7fdea83f5640 1 -- 172.21.15.98:0/1579609518 wait complete. 2024-09-19T08:39:56.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.391+0000 7fdea83f5640 1 Processor -- start 2024-09-19T08:39:56.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.391+0000 7fdea83f5640 1 -- start start 2024-09-19T08:39:56.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.391+0000 7fdea83f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde88004810 0x7fdea014dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:56.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.391+0000 7fdea83f5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdea015d6d0 0x7fdea01e7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:56.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.391+0000 7fdea83f5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdea015ec70 0x7fdea0072940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:56.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.391+0000 7fdea83f5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdea015cab0 con 0x7fdea015ec70 2024-09-19T08:39:56.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.391+0000 7fdea83f5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdea015c930 con 0x7fde88004810 2024-09-19T08:39:56.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.391+0000 7fdea83f5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdea00cf3c0 con 0x7fdea015d6d0 2024-09-19T08:39:56.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.391+0000 7fdea6992640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdea015ec70 0x7fdea0072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:56.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.391+0000 7fdea6992640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdea015ec70 0x7fdea0072940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:57220/0 (socket says 172.21.15.98:57220) 2024-09-19T08:39:56.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.391+0000 7fdea6992640 1 -- 172.21.15.98:0/552116352 learned_addr learned my addr 172.21.15.98:0/552116352 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:39:56.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.391+0000 7fdea5990640 1 --2- 172.21.15.98:0/552116352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdea015d6d0 0x7fdea01e7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:56.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.391+0000 7fdea6191640 1 --2- 172.21.15.98:0/552116352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde88004810 0x7fdea014dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:56.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.391+0000 7fdea6992640 1 -- 172.21.15.98:0/552116352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdea015d6d0 msgr2=0x7fdea01e7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:56.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.391+0000 7fdea6992640 1 --2- 172.21.15.98:0/552116352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdea015d6d0 0x7fdea01e7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:56.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.391+0000 7fdea6992640 1 -- 172.21.15.98:0/552116352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde88004810 msgr2=0x7fdea014dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:56.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.391+0000 7fdea6992640 1 --2- 172.21.15.98:0/552116352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde88004810 0x7fdea014dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:56.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.391+0000 7fdea6992640 1 -- 172.21.15.98:0/552116352 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea015dbd0 con 0x7fdea015ec70 2024-09-19T08:39:56.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.391+0000 7fdea6992640 1 --2- 172.21.15.98:0/552116352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdea015ec70 0x7fdea0072940 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fdea0073b20 tx=0x7fde9800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:56.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.391+0000 7fde9effd640 1 -- 172.21.15.98:0/552116352 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde98004220 con 0x7fdea015ec70 2024-09-19T08:39:56.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.391+0000 7fde9effd640 1 -- 172.21.15.98:0/552116352 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde980043e0 con 0x7fdea015ec70 2024-09-19T08:39:56.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.391+0000 7fdea83f5640 1 -- 172.21.15.98:0/552116352 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdea015deb0 con 0x7fdea015ec70 2024-09-19T08:39:56.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.391+0000 7fde9effd640 1 -- 172.21.15.98:0/552116352 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde98007310 con 0x7fdea015ec70 2024-09-19T08:39:56.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.391+0000 7fdea83f5640 1 -- 172.21.15.98:0/552116352 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdea015e470 con 0x7fdea015ec70 2024-09-19T08:39:56.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.391+0000 7fdea83f5640 1 -- 172.21.15.98:0/552116352 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdea00cf6c0 con 0x7fdea015ec70 2024-09-19T08:39:56.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.391+0000 7fde9effd640 1 -- 172.21.15.98:0/552116352 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fde980074b0 con 0x7fdea015ec70 2024-09-19T08:39:56.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.391+0000 7fde9effd640 1 --2- 172.21.15.98:0/552116352 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fde8c067a70 0x7fde8c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:39:56.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.395+0000 7fde9effd640 1 -- 172.21.15.98:0/552116352 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fde98085080 con 0x7fdea015ec70 2024-09-19T08:39:56.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.395+0000 7fdea6191640 1 --2- 172.21.15.98:0/552116352 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fde8c067a70 0x7fde8c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:39:56.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.395+0000 7fdea6191640 1 --2- 172.21.15.98:0/552116352 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fde8c067a70 0x7fde8c069f30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fde900013e0 tx=0x7fde90009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:39:56.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.399+0000 7fde9effd640 1 -- 172.21.15.98:0/552116352 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fde980559d0 con 0x7fdea015ec70 2024-09-19T08:39:56.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.567+0000 7fdea83f5640 1 -- 172.21.15.98:0/552116352 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdea00cdd40 con 0x7fdea015ec70 2024-09-19T08:39:56.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.567+0000 7fde9effd640 1 -- 172.21.15.98:0/552116352 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v43) ==== 80+0+0 (secure 0 0 0) 0x7fde980552d0 con 0x7fdea015ec70 2024-09-19T08:39:56.566 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-19T08:39:56.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.567+0000 7fdea83f5640 1 -- 172.21.15.98:0/552116352 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fde8c067a70 msgr2=0x7fde8c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:56.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.567+0000 7fdea83f5640 1 --2- 172.21.15.98:0/552116352 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fde8c067a70 0x7fde8c069f30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fde900013e0 tx=0x7fde90009040 comp rx=0 tx=0).stop 2024-09-19T08:39:56.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.567+0000 7fdea83f5640 1 -- 172.21.15.98:0/552116352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdea015ec70 msgr2=0x7fdea0072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:39:56.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.567+0000 7fdea83f5640 1 --2- 172.21.15.98:0/552116352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdea015ec70 0x7fdea0072940 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fdea0073b20 tx=0x7fde9800c820 comp rx=0 tx=0).stop 2024-09-19T08:39:56.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.567+0000 7fdea83f5640 1 -- 172.21.15.98:0/552116352 shutdown_connections 2024-09-19T08:39:56.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.567+0000 7fdea83f5640 1 --2- 172.21.15.98:0/552116352 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fde8c067a70 0x7fde8c069f30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:56.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.567+0000 7fdea83f5640 1 --2- 172.21.15.98:0/552116352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdea015ec70 0x7fdea0072940 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:56.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.567+0000 7fdea83f5640 1 --2- 172.21.15.98:0/552116352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdea015d6d0 0x7fdea01e7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:56.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.567+0000 7fdea83f5640 1 --2- 172.21.15.98:0/552116352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fde88004810 0x7fdea014dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:39:56.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.567+0000 7fdea83f5640 1 -- 172.21.15.98:0/552116352 >> 172.21.15.98:0/552116352 conn(0x7fdea00b2010 msgr2=0x7fdea014fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:39:56.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.567+0000 7fdea83f5640 1 -- 172.21.15.98:0/552116352 shutdown_connections 2024-09-19T08:39:56.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:39:56.567+0000 7fdea83f5640 1 -- 172.21.15.98:0/552116352 wait complete. 2024-09-19T08:39:56.583 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi098.front.sepia.ceph.com... 2024-09-19T08:39:56.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:39:56.583 DEBUG:teuthology.orchestra.run.smithi098:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T08:39:56.593 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:ceph-fuse[39257]: fuse finished with error 0 and tester_r 0 2024-09-19T08:39:56.594 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:40:02.696 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:40:02.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:40:02.697 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T08:40:02.704 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:40:02.705 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:40:02.712 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T08:40:02.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.883+0000 7fdfdc275640 1 Processor -- start 2024-09-19T08:40:02.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.883+0000 7fdfdc275640 1 -- start start 2024-09-19T08:40:02.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.883+0000 7fdfdc275640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdfd4140d30 0x7fdfd406d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:02.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.883+0000 7fdfdc275640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fdfd4141110 0x7fdfd406d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:02.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.883+0000 7fdfdc275640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfd415ea10 con 0x7fdfd40b3310 2024-09-19T08:40:02.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.883+0000 7fdfdc275640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fdfd415e7c0 con 0x7fdfd4141110 2024-09-19T08:40:02.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.883+0000 7fdfdc275640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd415ecf0 con 0x7fdfd4140d30 2024-09-19T08:40:02.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.883+0000 7fdfd9810640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fdfd4141110 0x7fdfd406d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:02.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.883+0000 7fdfd9810640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fdfd4141110 0x7fdfd406d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34836/0 (socket says 172.21.15.98:34836) 2024-09-19T08:40:02.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.883+0000 7fdfd9810640 1 -- 172.21.15.98:0/3488515219 learned_addr learned my addr 172.21.15.98:0/3488515219 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:40:02.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.883+0000 7fdfda011640 1 --2- 172.21.15.98:0/3488515219 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdfd4140d30 0x7fdfd406d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:02.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.883+0000 7fdfd9810640 1 -- 172.21.15.98:0/3488515219 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdfd4140d30 msgr2=0x7fdfd406d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:40:02.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.883+0000 7fdfd9810640 1 --2- 172.21.15.98:0/3488515219 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdfd4140d30 0x7fdfd406d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:02.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.883+0000 7fdfd9810640 1 -- 172.21.15.98:0/3488515219 >> v1:172.21.15.139:6789/0 conn(0x7fdfd40b3310 legacy=0x7fdfd40b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:40:02.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.883+0000 7fdfd9810640 1 -- 172.21.15.98:0/3488515219 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd415efd0 con 0x7fdfd4141110 2024-09-19T08:40:02.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.887+0000 7fdfda011640 1 --2- 172.21.15.98:0/3488515219 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdfd4140d30 0x7fdfd406d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:40:02.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.887+0000 7fdfd9810640 1 --2- 172.21.15.98:0/3488515219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfd4141110 0x7fdfd406d6b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fdfc400bb10 tx=0x7fdfc401ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=848678501e720a9b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:02.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.887+0000 7fdfd900f640 1 -- 172.21.15.98:0/3488515219 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc40b4070 con 0x7fdfd4141110 2024-09-19T08:40:02.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.887+0000 7fdfd900f640 1 -- 172.21.15.98:0/3488515219 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfc40a7c20 con 0x7fdfd4141110 2024-09-19T08:40:02.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.887+0000 7fdfd900f640 1 -- 172.21.15.98:0/3488515219 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc40af440 con 0x7fdfd4141110 2024-09-19T08:40:02.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.887+0000 7fdfdc275640 1 -- 172.21.15.98:0/3488515219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfd4141110 msgr2=0x7fdfd406d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:02.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.887+0000 7fdfdc275640 1 --2- 172.21.15.98:0/3488515219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfd4141110 0x7fdfd406d6b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fdfc400bb10 tx=0x7fdfc401ebf0 comp rx=0 tx=0).stop 2024-09-19T08:40:02.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.887+0000 7fdfdc275640 1 -- 172.21.15.98:0/3488515219 shutdown_connections 2024-09-19T08:40:02.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.887+0000 7fdfdc275640 1 --2- 172.21.15.98:0/3488515219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfd4141110 0x7fdfd406d6b0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:02.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.887+0000 7fdfdc275640 1 --2- 172.21.15.98:0/3488515219 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdfd4140d30 0x7fdfd406d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:02.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.887+0000 7fdfdc275640 1 -- 172.21.15.98:0/3488515219 >> 172.21.15.98:0/3488515219 conn(0x7fdfd40b2010 msgr2=0x7fdfd40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:40:02.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.887+0000 7fdfdc275640 1 -- 172.21.15.98:0/3488515219 shutdown_connections 2024-09-19T08:40:02.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.887+0000 7fdfdc275640 1 -- 172.21.15.98:0/3488515219 wait complete. 2024-09-19T08:40:02.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.891+0000 7fdfdc275640 1 Processor -- start 2024-09-19T08:40:02.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.891+0000 7fdfdc275640 1 -- start start 2024-09-19T08:40:02.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.891+0000 7fdfdc275640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfd40b3310 0x7fdfd4147c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:02.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.891+0000 7fdfdc275640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfd4140d30 0x7fdfd41481a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:02.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.891+0000 7fdfdc275640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdfd4141110 0x7fdfd41622c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:02.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.891+0000 7fdfdc275640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd415b600 con 0x7fdfd4140d30 2024-09-19T08:40:02.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.891+0000 7fdfdc275640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd415b480 con 0x7fdfd40b3310 2024-09-19T08:40:02.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.891+0000 7fdfdc275640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd415b780 con 0x7fdfd4141110 2024-09-19T08:40:02.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.891+0000 7fdfd9810640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfd4140d30 0x7fdfd41481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:02.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.891+0000 7fdfd9810640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfd4140d30 0x7fdfd41481a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34846/0 (socket says 172.21.15.98:34846) 2024-09-19T08:40:02.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.891+0000 7fdfd9810640 1 -- 172.21.15.98:0/1164279282 learned_addr learned my addr 172.21.15.98:0/1164279282 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:40:02.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.891+0000 7fdfd9810640 1 -- 172.21.15.98:0/1164279282 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdfd4141110 msgr2=0x7fdfd41622c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:02.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.891+0000 7fdfd9810640 1 --2- 172.21.15.98:0/1164279282 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdfd4141110 0x7fdfd41622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:02.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.891+0000 7fdfd9810640 1 -- 172.21.15.98:0/1164279282 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfd40b3310 msgr2=0x7fdfd4147c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:02.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.891+0000 7fdfd9810640 1 --2- 172.21.15.98:0/1164279282 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfd40b3310 0x7fdfd4147c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:02.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.891+0000 7fdfd9810640 1 -- 172.21.15.98:0/1164279282 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd4073230 con 0x7fdfd4140d30 2024-09-19T08:40:02.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.891+0000 7fdfd9810640 1 --2- 172.21.15.98:0/1164279282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfd4140d30 0x7fdfd41481a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fdfc400b2c0 tx=0x7fdfc401b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:02.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.891+0000 7fdfcaffd640 1 -- 172.21.15.98:0/1164279282 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc40b4040 con 0x7fdfd4140d30 2024-09-19T08:40:02.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.891+0000 7fdfcaffd640 1 -- 172.21.15.98:0/1164279282 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfc40bf070 con 0x7fdfd4140d30 2024-09-19T08:40:02.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.891+0000 7fdfcaffd640 1 -- 172.21.15.98:0/1164279282 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc40ba6a0 con 0x7fdfd4140d30 2024-09-19T08:40:02.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.891+0000 7fdfdc275640 1 -- 172.21.15.98:0/1164279282 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfd4073510 con 0x7fdfd4140d30 2024-09-19T08:40:02.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.891+0000 7fdfdc275640 1 -- 172.21.15.98:0/1164279282 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfd4156860 con 0x7fdfd4140d30 2024-09-19T08:40:02.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.891+0000 7fdfdc275640 1 -- 172.21.15.98:0/1164279282 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfd40b3760 con 0x7fdfd4140d30 2024-09-19T08:40:02.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.891+0000 7fdfcaffd640 1 -- 172.21.15.98:0/1164279282 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fdfc40ba840 con 0x7fdfd4140d30 2024-09-19T08:40:02.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.895+0000 7fdfcaffd640 1 --2- 172.21.15.98:0/1164279282 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdfb8067a70 0x7fdfb8069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:02.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.895+0000 7fdfda011640 1 --2- 172.21.15.98:0/1164279282 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdfb8067a70 0x7fdfb8069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:02.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.895+0000 7fdfcaffd640 1 -- 172.21.15.98:0/1164279282 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7fdfc4123460 con 0x7fdfd4140d30 2024-09-19T08:40:02.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.895+0000 7fdfda011640 1 --2- 172.21.15.98:0/1164279282 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdfb8067a70 0x7fdfb8069f30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdfd000a8d0 tx=0x7fdfd0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:02.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:02.899+0000 7fdfcaffd640 1 -- 172.21.15.98:0/1164279282 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fdfc40187e0 con 0x7fdfd4140d30 2024-09-19T08:40:03.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.071+0000 7fdfdc275640 1 -- 172.21.15.98:0/1164279282 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdfd415e0a0 con 0x7fdfd4140d30 2024-09-19T08:40:03.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.075+0000 7fdfcaffd640 1 -- 172.21.15.98:0/1164279282 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v45) ==== 80+0+0 (secure 0 0 0) 0x7fdfc40f3df0 con 0x7fdfd4140d30 2024-09-19T08:40:03.073 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-19T08:40:03.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.075+0000 7fdfdc275640 1 -- 172.21.15.98:0/1164279282 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdfb8067a70 msgr2=0x7fdfb8069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:03.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.075+0000 7fdfdc275640 1 --2- 172.21.15.98:0/1164279282 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdfb8067a70 0x7fdfb8069f30 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdfd000a8d0 tx=0x7fdfd0008040 comp rx=0 tx=0).stop 2024-09-19T08:40:03.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.075+0000 7fdfdc275640 1 -- 172.21.15.98:0/1164279282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfd4140d30 msgr2=0x7fdfd41481a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:03.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.075+0000 7fdfdc275640 1 --2- 172.21.15.98:0/1164279282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfd4140d30 0x7fdfd41481a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fdfc400b2c0 tx=0x7fdfc401b990 comp rx=0 tx=0).stop 2024-09-19T08:40:03.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.075+0000 7fdfdc275640 1 -- 172.21.15.98:0/1164279282 shutdown_connections 2024-09-19T08:40:03.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.075+0000 7fdfdc275640 1 --2- 172.21.15.98:0/1164279282 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdfb8067a70 0x7fdfb8069f30 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:03.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.075+0000 7fdfdc275640 1 --2- 172.21.15.98:0/1164279282 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdfd4141110 0x7fdfd41622c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:03.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.075+0000 7fdfdc275640 1 --2- 172.21.15.98:0/1164279282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfd4140d30 0x7fdfd41481a0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:03.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.075+0000 7fdfdc275640 1 --2- 172.21.15.98:0/1164279282 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfd40b3310 0x7fdfd4147c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:03.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.075+0000 7fdfdc275640 1 -- 172.21.15.98:0/1164279282 >> 172.21.15.98:0/1164279282 conn(0x7fdfd40b2010 msgr2=0x7fdfd40ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:40:03.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.075+0000 7fdfdc275640 1 -- 172.21.15.98:0/1164279282 shutdown_connections 2024-09-19T08:40:03.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.075+0000 7fdfdc275640 1 -- 172.21.15.98:0/1164279282 wait complete. 2024-09-19T08:40:03.091 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:40:03.091 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:40:03.098 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T08:40:03.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.219+0000 7fe07d04c640 1 Processor -- start 2024-09-19T08:40:03.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.219+0000 7fe07d04c640 1 -- start start 2024-09-19T08:40:03.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.219+0000 7fe07d04c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe0780b7f80 0x7fe07815f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:03.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.219+0000 7fe07d04c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe0780d4820 0x7fe0781621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:03.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.219+0000 7fe07d04c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe078068440 con 0x7fe0780b9970 2024-09-19T08:40:03.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.219+0000 7fe07d04c640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fe078163050 con 0x7fe0780d4820 2024-09-19T08:40:03.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.219+0000 7fe07d04c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe078162a90 con 0x7fe0780b7f80 2024-09-19T08:40:03.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.219+0000 7fe06ffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe0780d4820 0x7fe0781621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:03.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.219+0000 7fe06ffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe0780d4820 0x7fe0781621b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34850/0 (socket says 172.21.15.98:34850) 2024-09-19T08:40:03.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.219+0000 7fe06ffff640 1 -- 172.21.15.98:0/1937182707 learned_addr learned my addr 172.21.15.98:0/1937182707 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:40:03.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.219+0000 7fe06ffff640 1 -- 172.21.15.98:0/1937182707 >> v2:172.21.15.139:3300/0 conn(0x7fe0780b7f80 msgr2=0x7fe07815f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:40:03.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.219+0000 7fe06ffff640 1 --2- 172.21.15.98:0/1937182707 >> v2:172.21.15.139:3300/0 conn(0x7fe0780b7f80 0x7fe07815f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:03.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.219+0000 7fe06ffff640 1 -- 172.21.15.98:0/1937182707 >> v1:172.21.15.139:6789/0 conn(0x7fe0780b9970 legacy=0x7fe0780b9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:40:03.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.219+0000 7fe06ffff640 1 -- 172.21.15.98:0/1937182707 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe078162d70 con 0x7fe0780d4820 2024-09-19T08:40:03.221 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.223+0000 7fe06ffff640 1 --2- 172.21.15.98:0/1937182707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0780d4820 0x7fe0781621b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe0780d0dd0 tx=0x7fe06001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a3431354846807 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:03.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.223+0000 7fe07659c640 1 -- 172.21.15.98:0/1937182707 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0600b0070 con 0x7fe0780d4820 2024-09-19T08:40:03.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.223+0000 7fe07659c640 1 -- 172.21.15.98:0/1937182707 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0600ab600 con 0x7fe0780d4820 2024-09-19T08:40:03.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.223+0000 7fe07659c640 1 -- 172.21.15.98:0/1937182707 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0600ab9d0 con 0x7fe0780d4820 2024-09-19T08:40:03.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.223+0000 7fe07d04c640 1 -- 172.21.15.98:0/1937182707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0780d4820 msgr2=0x7fe0781621b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:03.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.223+0000 7fe07d04c640 1 --2- 172.21.15.98:0/1937182707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0780d4820 0x7fe0781621b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe0780d0dd0 tx=0x7fe06001c450 comp rx=0 tx=0).stop 2024-09-19T08:40:03.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.223+0000 7fe07d04c640 1 -- 172.21.15.98:0/1937182707 shutdown_connections 2024-09-19T08:40:03.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.223+0000 7fe07d04c640 1 --2- 172.21.15.98:0/1937182707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0780d4820 0x7fe0781621b0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:03.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.223+0000 7fe07d04c640 1 --2- 172.21.15.98:0/1937182707 >> v2:172.21.15.139:3300/0 conn(0x7fe0780b7f80 0x7fe07815f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:03.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.223+0000 7fe07d04c640 1 -- 172.21.15.98:0/1937182707 >> 172.21.15.98:0/1937182707 conn(0x7fe078074e40 msgr2=0x7fe078075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:40:03.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.223+0000 7fe07d04c640 1 -- 172.21.15.98:0/1937182707 shutdown_connections 2024-09-19T08:40:03.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.223+0000 7fe07d04c640 1 -- 172.21.15.98:0/1937182707 wait complete. 2024-09-19T08:40:03.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.223+0000 7fe07d04c640 1 Processor -- start 2024-09-19T08:40:03.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.223+0000 7fe07d04c640 1 -- start start 2024-09-19T08:40:03.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.223+0000 7fe07d04c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0780b7f80 0x7fe078158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:03.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.223+0000 7fe07d04c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0780b9970 0x7fe07813eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:03.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.223+0000 7fe07d04c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0780d4820 0x7fe078150ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:03.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.223+0000 7fe07d04c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe0780b84e0 con 0x7fe0780b9970 2024-09-19T08:40:03.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.223+0000 7fe07d04c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe0780b8360 con 0x7fe0780d4820 2024-09-19T08:40:03.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.223+0000 7fe07d04c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe078144590 con 0x7fe0780b7f80 2024-09-19T08:40:03.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.223+0000 7fe06ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0780b9970 0x7fe07813eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:03.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.223+0000 7fe06ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0780b9970 0x7fe07813eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34854/0 (socket says 172.21.15.98:34854) 2024-09-19T08:40:03.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.223+0000 7fe06ffff640 1 -- 172.21.15.98:0/2494781377 learned_addr learned my addr 172.21.15.98:0/2494781377 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:40:03.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.223+0000 7fe06ffff640 1 -- 172.21.15.98:0/2494781377 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0780b7f80 msgr2=0x7fe078158200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:03.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.223+0000 7fe06ffff640 1 --2- 172.21.15.98:0/2494781377 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0780b7f80 0x7fe078158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:03.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.223+0000 7fe06ffff640 1 -- 172.21.15.98:0/2494781377 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0780d4820 msgr2=0x7fe078150ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:03.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.223+0000 7fe07759e640 1 --2- 172.21.15.98:0/2494781377 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0780d4820 0x7fe078150ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:03.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.223+0000 7fe06ffff640 1 --2- 172.21.15.98:0/2494781377 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0780d4820 0x7fe078150ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:03.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.223+0000 7fe06ffff640 1 -- 172.21.15.98:0/2494781377 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe078157160 con 0x7fe0780b9970 2024-09-19T08:40:03.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.223+0000 7fe07759e640 1 --2- 172.21.15.98:0/2494781377 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0780d4820 0x7fe078150ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:40:03.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.227+0000 7fe06ffff640 1 --2- 172.21.15.98:0/2494781377 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0780b9970 0x7fe07813eb80 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe0600ae040 tx=0x7fe06001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:03.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.227+0000 7fe06f7fe640 1 -- 172.21.15.98:0/2494781377 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0600ba020 con 0x7fe0780b9970 2024-09-19T08:40:03.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.227+0000 7fe06f7fe640 1 -- 172.21.15.98:0/2494781377 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0600b6640 con 0x7fe0780b9970 2024-09-19T08:40:03.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.227+0000 7fe07d04c640 1 -- 172.21.15.98:0/2494781377 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe078157440 con 0x7fe0780b9970 2024-09-19T08:40:03.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.227+0000 7fe06f7fe640 1 -- 172.21.15.98:0/2494781377 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0600b0040 con 0x7fe0780b9970 2024-09-19T08:40:03.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.227+0000 7fe07d04c640 1 -- 172.21.15.98:0/2494781377 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe078157a00 con 0x7fe0780b9970 2024-09-19T08:40:03.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.227+0000 7fe07d04c640 1 -- 172.21.15.98:0/2494781377 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe078069c40 con 0x7fe0780b9970 2024-09-19T08:40:03.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.227+0000 7fe06f7fe640 1 -- 172.21.15.98:0/2494781377 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fe0600bd020 con 0x7fe0780b9970 2024-09-19T08:40:03.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.227+0000 7fe06f7fe640 1 --2- 172.21.15.98:0/2494781377 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe058067a70 0x7fe058069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:03.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.227+0000 7fe06f7fe640 1 -- 172.21.15.98:0/2494781377 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7fe0601207d0 con 0x7fe0780b9970 2024-09-19T08:40:03.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.227+0000 7fe076d9d640 1 --2- 172.21.15.98:0/2494781377 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe058067a70 0x7fe058069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:03.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.227+0000 7fe076d9d640 1 --2- 172.21.15.98:0/2494781377 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe058067a70 0x7fe058069f30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe068009690 tx=0x7fe068006c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:03.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.231+0000 7fe06f7fe640 1 -- 172.21.15.98:0/2494781377 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe0600c4070 con 0x7fe0780b9970 2024-09-19T08:40:03.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.407+0000 7fe07d04c640 1 -- 172.21.15.98:0/2494781377 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe0781448c0 con 0x7fe0780b9970 2024-09-19T08:40:03.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.407+0000 7fe06f7fe640 1 -- 172.21.15.98:0/2494781377 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v45) ==== 80+0+0 (secure 0 0 0) 0x7fe0600160d0 con 0x7fe0780b9970 2024-09-19T08:40:03.406 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-19T08:40:03.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.407+0000 7fe07d04c640 1 -- 172.21.15.98:0/2494781377 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe058067a70 msgr2=0x7fe058069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:03.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.407+0000 7fe07d04c640 1 --2- 172.21.15.98:0/2494781377 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe058067a70 0x7fe058069f30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe068009690 tx=0x7fe068006c00 comp rx=0 tx=0).stop 2024-09-19T08:40:03.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.407+0000 7fe07d04c640 1 -- 172.21.15.98:0/2494781377 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0780b9970 msgr2=0x7fe07813eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:03.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.407+0000 7fe07d04c640 1 --2- 172.21.15.98:0/2494781377 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0780b9970 0x7fe07813eb80 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe0600ae040 tx=0x7fe06001ac90 comp rx=0 tx=0).stop 2024-09-19T08:40:03.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.407+0000 7fe07d04c640 1 -- 172.21.15.98:0/2494781377 shutdown_connections 2024-09-19T08:40:03.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.407+0000 7fe07d04c640 1 --2- 172.21.15.98:0/2494781377 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe058067a70 0x7fe058069f30 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:03.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.407+0000 7fe07d04c640 1 --2- 172.21.15.98:0/2494781377 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0780d4820 0x7fe078150ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:03.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.407+0000 7fe07d04c640 1 --2- 172.21.15.98:0/2494781377 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0780b9970 0x7fe07813eb80 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:03.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.407+0000 7fe07d04c640 1 --2- 172.21.15.98:0/2494781377 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0780b7f80 0x7fe078158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:03.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.411+0000 7fe07d04c640 1 -- 172.21.15.98:0/2494781377 >> 172.21.15.98:0/2494781377 conn(0x7fe078074e40 msgr2=0x7fe0780d0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:40:03.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.411+0000 7fe07d04c640 1 -- 172.21.15.98:0/2494781377 shutdown_connections 2024-09-19T08:40:03.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:03.411+0000 7fe07d04c640 1 -- 172.21.15.98:0/2494781377 wait complete. 2024-09-19T08:40:03.425 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi139.front.sepia.ceph.com... 2024-09-19T08:40:03.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:40:03.425 DEBUG:teuthology.orchestra.run.smithi139:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-19T08:40:03.439 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[35408]: fuse finished with error 0 and tester_r 0 2024-09-19T08:40:03.440 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:40:09.542 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:40:09.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:40:09.543 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-19T08:40:09.551 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:40:09.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.683+0000 7fbca5b99640 1 Processor -- start 2024-09-19T08:40:09.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.687+0000 7fbca5b99640 1 -- start start 2024-09-19T08:40:09.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.687+0000 7fbca5b99640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbca006bc90 0x7fbca006c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:09.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.687+0000 7fbca5b99640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbca00ce840 0x7fbca00cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:09.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.687+0000 7fbca5b99640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbca00ccc60 con 0x7fbca0158fd0 2024-09-19T08:40:09.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.687+0000 7fbca5b99640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fbca01593b0 con 0x7fbca006bc90 2024-09-19T08:40:09.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.687+0000 7fbca5b99640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbca00ccf40 con 0x7fbca00ce840 2024-09-19T08:40:09.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.687+0000 7fbc96ffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbca006bc90 0x7fbca006c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:09.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.687+0000 7fbc96ffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbca006bc90 0x7fbca006c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34368/0 (socket says 172.21.15.98:34368) 2024-09-19T08:40:09.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.687+0000 7fbc96ffd640 1 -- 172.21.15.98:0/579722992 learned_addr learned my addr 172.21.15.98:0/579722992 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:40:09.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.687+0000 7fbc9ffff640 1 --2- 172.21.15.98:0/579722992 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbca00ce840 0x7fbca00cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:09.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.687+0000 7fbc96ffd640 1 -- 172.21.15.98:0/579722992 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbca00ce840 msgr2=0x7fbca00cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:40:09.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.687+0000 7fbc96ffd640 1 --2- 172.21.15.98:0/579722992 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbca00ce840 0x7fbca00cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:09.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.687+0000 7fbc96ffd640 1 -- 172.21.15.98:0/579722992 >> v1:172.21.15.139:6789/0 conn(0x7fbca0158fd0 legacy=0x7fbca00b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:09.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.687+0000 7fbc96ffd640 1 -- 172.21.15.98:0/579722992 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca00cd220 con 0x7fbca006bc90 2024-09-19T08:40:09.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.687+0000 7fbc96ffd640 1 --2- 172.21.15.98:0/579722992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbca006bc90 0x7fbca006c120 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fbc88009870 tx=0x7fbc8801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43de0da6bf2d71d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:09.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.691+0000 7fbc9effd640 1 -- 172.21.15.98:0/579722992 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc880b1070 con 0x7fbca006bc90 2024-09-19T08:40:09.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.691+0000 7fbc9effd640 1 -- 172.21.15.98:0/579722992 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc880ac620 con 0x7fbca006bc90 2024-09-19T08:40:09.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.691+0000 7fbc9effd640 1 -- 172.21.15.98:0/579722992 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc880ac9f0 con 0x7fbca006bc90 2024-09-19T08:40:09.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.691+0000 7fbca5b99640 1 -- 172.21.15.98:0/579722992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbca006bc90 msgr2=0x7fbca006c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:09.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.691+0000 7fbca5b99640 1 --2- 172.21.15.98:0/579722992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbca006bc90 0x7fbca006c120 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fbc88009870 tx=0x7fbc8801c4d0 comp rx=0 tx=0).stop 2024-09-19T08:40:09.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.691+0000 7fbca5b99640 1 -- 172.21.15.98:0/579722992 shutdown_connections 2024-09-19T08:40:09.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.691+0000 7fbca5b99640 1 --2- 172.21.15.98:0/579722992 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbca00ce840 0x7fbca00cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:09.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.691+0000 7fbca5b99640 1 --2- 172.21.15.98:0/579722992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbca006bc90 0x7fbca006c120 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:09.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.691+0000 7fbca5b99640 1 -- 172.21.15.98:0/579722992 >> 172.21.15.98:0/579722992 conn(0x7fbca00b2010 msgr2=0x7fbca00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:40:09.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.691+0000 7fbca5b99640 1 -- 172.21.15.98:0/579722992 shutdown_connections 2024-09-19T08:40:09.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.691+0000 7fbca5b99640 1 -- 172.21.15.98:0/579722992 wait complete. 2024-09-19T08:40:09.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.691+0000 7fbca5b99640 1 Processor -- start 2024-09-19T08:40:09.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.691+0000 7fbca5b99640 1 -- start start 2024-09-19T08:40:09.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.691+0000 7fbca5b99640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbca006bc90 0x7fbca0073890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:09.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.691+0000 7fbca5b99640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbca00ce840 0x7fbca01600d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:09.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.691+0000 7fbca5b99640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbca0158fd0 0x7fbca01549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:09.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.691+0000 7fbca5b99640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbca015c920 con 0x7fbca0158fd0 2024-09-19T08:40:09.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.691+0000 7fbca5b99640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbca015c7a0 con 0x7fbca00ce840 2024-09-19T08:40:09.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.691+0000 7fbca5b99640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbca0168110 con 0x7fbca006bc90 2024-09-19T08:40:09.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.691+0000 7fbc9ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbca0158fd0 0x7fbca01549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:09.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.691+0000 7fbc9ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbca0158fd0 0x7fbca01549b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34378/0 (socket says 172.21.15.98:34378) 2024-09-19T08:40:09.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.691+0000 7fbc9ffff640 1 -- 172.21.15.98:0/3479775500 learned_addr learned my addr 172.21.15.98:0/3479775500 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:40:09.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.691+0000 7fbc96ffd640 1 --2- 172.21.15.98:0/3479775500 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbca00ce840 0x7fbca01600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:09.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.691+0000 7fbc9f7fe640 1 --2- 172.21.15.98:0/3479775500 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbca006bc90 0x7fbca0073890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:09.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.691+0000 7fbc9ffff640 1 -- 172.21.15.98:0/3479775500 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbca006bc90 msgr2=0x7fbca0073890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:09.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.691+0000 7fbc9ffff640 1 --2- 172.21.15.98:0/3479775500 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbca006bc90 0x7fbca0073890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:09.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.691+0000 7fbc9ffff640 1 -- 172.21.15.98:0/3479775500 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbca00ce840 msgr2=0x7fbca01600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:09.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.691+0000 7fbc9ffff640 1 --2- 172.21.15.98:0/3479775500 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbca00ce840 0x7fbca01600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:09.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.691+0000 7fbc9ffff640 1 -- 172.21.15.98:0/3479775500 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca015d5b0 con 0x7fbca0158fd0 2024-09-19T08:40:09.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.691+0000 7fbc9ffff640 1 --2- 172.21.15.98:0/3479775500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbca0158fd0 0x7fbca01549b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fbc98007e40 tx=0x7fbc9800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:09.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.691+0000 7fbc9cff9640 1 -- 172.21.15.98:0/3479775500 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc98014620 con 0x7fbca0158fd0 2024-09-19T08:40:09.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.691+0000 7fbc9cff9640 1 -- 172.21.15.98:0/3479775500 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc98019070 con 0x7fbca0158fd0 2024-09-19T08:40:09.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.691+0000 7fbca5b99640 1 -- 172.21.15.98:0/3479775500 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbca015d820 con 0x7fbca0158fd0 2024-09-19T08:40:09.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.691+0000 7fbc9cff9640 1 -- 172.21.15.98:0/3479775500 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc98014a20 con 0x7fbca0158fd0 2024-09-19T08:40:09.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.691+0000 7fbca5b99640 1 -- 172.21.15.98:0/3479775500 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbca015dde0 con 0x7fbca0158fd0 2024-09-19T08:40:09.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.695+0000 7fbca5b99640 1 -- 172.21.15.98:0/3479775500 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbca015dfe0 con 0x7fbca0158fd0 2024-09-19T08:40:09.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.695+0000 7fbc9cff9640 1 -- 172.21.15.98:0/3479775500 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7fbc98014bc0 con 0x7fbca0158fd0 2024-09-19T08:40:09.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.695+0000 7fbc9cff9640 1 --2- 172.21.15.98:0/3479775500 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fbc80067a70 0x7fbc80069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:09.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.695+0000 7fbc9cff9640 1 -- 172.21.15.98:0/3479775500 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7fbc98086900 con 0x7fbca0158fd0 2024-09-19T08:40:09.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.695+0000 7fbc9f7fe640 1 --2- 172.21.15.98:0/3479775500 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fbc80067a70 0x7fbc80069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:09.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.695+0000 7fbc9f7fe640 1 --2- 172.21.15.98:0/3479775500 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fbc80067a70 0x7fbc80069f30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbc90007fd0 tx=0x7fbc90009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:09.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.699+0000 7fbc9cff9640 1 -- 172.21.15.98:0/3479775500 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fbca015dfe0 con 0x7fbca0158fd0 2024-09-19T08:40:09.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.887+0000 7fbca5b99640 1 -- 172.21.15.98:0/3479775500 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbca006c120 con 0x7fbca0158fd0 2024-09-19T08:40:09.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.887+0000 7fbc9cff9640 1 -- 172.21.15.98:0/3479775500 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5229 (secure 0 0 0) 0x7fbc98057170 con 0x7fbca0158fd0 2024-09-19T08:40:09.888 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:40:09.888 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"btime":"2024-09-19T08:39:55:181532+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/2874086421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":2874086421},{"type":"v1","addr":"172.21.15.139:6833","nonce":2874086421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4996,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/4178563848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":4178563848},{"type":"v1","addr":"172.21.15.139:6835","nonce":4178563848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3584568343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3584568343},{"type":"v1","addr":"172.21.15.98:6835","nonce":3584568343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:39:38.262938+0000","modified":"2024-09-19T08:39:41.276985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/3085042202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3085042202},{"type":"v1","addr":"172.21.15.98:6837","nonce":3085042202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244]},"id":3}]} 2024-09-19T08:40:09.889 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 24 2024-09-19T08:40:09.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.891+0000 7fbca5b99640 1 -- 172.21.15.98:0/3479775500 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fbc80067a70 msgr2=0x7fbc80069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:09.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.891+0000 7fbca5b99640 1 --2- 172.21.15.98:0/3479775500 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fbc80067a70 0x7fbc80069f30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbc90007fd0 tx=0x7fbc90009040 comp rx=0 tx=0).stop 2024-09-19T08:40:09.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.891+0000 7fbca5b99640 1 -- 172.21.15.98:0/3479775500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbca0158fd0 msgr2=0x7fbca01549b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:09.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.891+0000 7fbca5b99640 1 --2- 172.21.15.98:0/3479775500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbca0158fd0 0x7fbca01549b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fbc98007e40 tx=0x7fbc9800e4b0 comp rx=0 tx=0).stop 2024-09-19T08:40:09.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.891+0000 7fbca5b99640 1 -- 172.21.15.98:0/3479775500 shutdown_connections 2024-09-19T08:40:09.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.891+0000 7fbca5b99640 1 --2- 172.21.15.98:0/3479775500 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fbc80067a70 0x7fbc80069f30 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:09.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.891+0000 7fbca5b99640 1 --2- 172.21.15.98:0/3479775500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbca0158fd0 0x7fbca01549b0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:09.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.891+0000 7fbca5b99640 1 --2- 172.21.15.98:0/3479775500 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbca00ce840 0x7fbca01600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:09.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.891+0000 7fbca5b99640 1 --2- 172.21.15.98:0/3479775500 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbca006bc90 0x7fbca0073890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:09.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.891+0000 7fbca5b99640 1 -- 172.21.15.98:0/3479775500 >> 172.21.15.98:0/3479775500 conn(0x7fbca00b2010 msgr2=0x7fbca00b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:40:09.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.891+0000 7fbca5b99640 1 -- 172.21.15.98:0/3479775500 shutdown_connections 2024-09-19T08:40:09.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:09.891+0000 7fbca5b99640 1 -- 172.21.15.98:0/3479775500 wait complete. 2024-09-19T08:40:09.906 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-19T08:40:10.044 INFO:teuthology.orchestra.run.smithi098.stdout:[] 2024-09-19T08:40:10.045 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-09-19T08:40:10.045 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol58510 2024-09-19T08:40:10.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.163+0000 7f2fda12c640 1 Processor -- start 2024-09-19T08:40:10.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.167+0000 7f2fda12c640 1 -- start start 2024-09-19T08:40:10.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.167+0000 7f2fda12c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2fd406caf0 0x7f2fd40b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:10.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.167+0000 7f2fda12c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fd40b3770 0x7f2fd40cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:10.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.167+0000 7f2fda12c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fd40b08c0 con 0x7f2fd415c730 2024-09-19T08:40:10.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.167+0000 7f2fda12c640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f2fd40b0ba0 con 0x7f2fd406caf0 2024-09-19T08:40:10.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.167+0000 7f2fda12c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2fd40b0e80 con 0x7f2fd40b3770 2024-09-19T08:40:10.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.167+0000 7f2fd37fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2fd406caf0 0x7f2fd40b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:10.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.167+0000 7f2fd37fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2fd406caf0 0x7f2fd40b02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34386/0 (socket says 172.21.15.98:34386) 2024-09-19T08:40:10.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.167+0000 7f2fd37fe640 1 -- 172.21.15.98:0/939131633 learned_addr learned my addr 172.21.15.98:0/939131633 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:40:10.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.167+0000 7f2fd37fe640 1 -- 172.21.15.98:0/939131633 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fd40b3770 msgr2=0x7f2fd40cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:40:10.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.167+0000 7f2fd3fff640 1 --2- 172.21.15.98:0/939131633 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fd40b3770 0x7f2fd40cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:10.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.167+0000 7f2fd37fe640 1 --2- 172.21.15.98:0/939131633 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fd40b3770 0x7f2fd40cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:10.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.167+0000 7f2fd37fe640 1 -- 172.21.15.98:0/939131633 >> v1:172.21.15.139:6789/0 conn(0x7f2fd415c730 legacy=0x7f2fd406ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:40:10.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.167+0000 7f2fd37fe640 1 -- 172.21.15.98:0/939131633 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd40b1160 con 0x7f2fd406caf0 2024-09-19T08:40:10.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.167+0000 7f2fd37fe640 1 --2- 172.21.15.98:0/939131633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fd406caf0 0x7f2fd40b02e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc009e50 tx=0x7f2fcc01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dcca7f2571b0a69 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:10.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.167+0000 7f2fd27fc640 1 -- 172.21.15.98:0/939131633 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc0a5860 con 0x7f2fd406caf0 2024-09-19T08:40:10.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.171+0000 7f2fd27fc640 1 -- 172.21.15.98:0/939131633 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fcc0a5a00 con 0x7f2fd406caf0 2024-09-19T08:40:10.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.171+0000 7f2fd27fc640 1 -- 172.21.15.98:0/939131633 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc0a5dd0 con 0x7f2fd406caf0 2024-09-19T08:40:10.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.171+0000 7f2fda12c640 1 -- 172.21.15.98:0/939131633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fd406caf0 msgr2=0x7f2fd40b02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:10.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.171+0000 7f2fda12c640 1 --2- 172.21.15.98:0/939131633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fd406caf0 0x7f2fd40b02e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc009e50 tx=0x7f2fcc01c9c0 comp rx=0 tx=0).stop 2024-09-19T08:40:10.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.171+0000 7f2fda12c640 1 -- 172.21.15.98:0/939131633 shutdown_connections 2024-09-19T08:40:10.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.171+0000 7f2fda12c640 1 --2- 172.21.15.98:0/939131633 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fd40b3770 0x7f2fd40cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:10.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.171+0000 7f2fda12c640 1 --2- 172.21.15.98:0/939131633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fd406caf0 0x7f2fd40b02e0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:10.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.171+0000 7f2fda12c640 1 -- 172.21.15.98:0/939131633 >> 172.21.15.98:0/939131633 conn(0x7f2fd40b5f50 msgr2=0x7f2fd40b6650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:40:10.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.171+0000 7f2fda12c640 1 -- 172.21.15.98:0/939131633 shutdown_connections 2024-09-19T08:40:10.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.171+0000 7f2fda12c640 1 -- 172.21.15.98:0/939131633 wait complete. 2024-09-19T08:40:10.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.171+0000 7f2fda12c640 1 Processor -- start 2024-09-19T08:40:10.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.171+0000 7f2fda12c640 1 -- start start 2024-09-19T08:40:10.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.171+0000 7f2fda12c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fd406caf0 0x7f2fd415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:10.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.171+0000 7f2fda12c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fd40b3770 0x7f2fd414d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:10.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.171+0000 7f2fda12c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fd415c730 0x7f2fd414c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:10.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.171+0000 7f2fda12c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd4169120 con 0x7f2fd40b3770 2024-09-19T08:40:10.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.171+0000 7f2fda12c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd40cc350 con 0x7f2fd415c730 2024-09-19T08:40:10.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.171+0000 7f2fda12c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2fd41692a0 con 0x7f2fd406caf0 2024-09-19T08:40:10.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.171+0000 7f2fd2ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fd40b3770 0x7f2fd414d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:10.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.171+0000 7f2fd2ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fd40b3770 0x7f2fd414d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34388/0 (socket says 172.21.15.98:34388) 2024-09-19T08:40:10.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.171+0000 7f2fd2ffd640 1 -- 172.21.15.98:0/1246527245 learned_addr learned my addr 172.21.15.98:0/1246527245 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:40:10.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.171+0000 7f2fd37fe640 1 --2- 172.21.15.98:0/1246527245 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fd406caf0 0x7f2fd415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:10.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.171+0000 7f2fd3fff640 1 --2- 172.21.15.98:0/1246527245 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fd415c730 0x7f2fd414c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:10.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.171+0000 7f2fd2ffd640 1 -- 172.21.15.98:0/1246527245 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fd406caf0 msgr2=0x7f2fd415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:10.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.171+0000 7f2fd2ffd640 1 --2- 172.21.15.98:0/1246527245 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fd406caf0 0x7f2fd415b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:10.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.171+0000 7f2fd2ffd640 1 -- 172.21.15.98:0/1246527245 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fd415c730 msgr2=0x7f2fd414c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:10.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.171+0000 7f2fd2ffd640 1 --2- 172.21.15.98:0/1246527245 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fd415c730 0x7f2fd414c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:10.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.171+0000 7f2fd2ffd640 1 -- 172.21.15.98:0/1246527245 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd414de10 con 0x7f2fd40b3770 2024-09-19T08:40:10.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.171+0000 7f2fd2ffd640 1 --2- 172.21.15.98:0/1246527245 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fd40b3770 0x7f2fd414d390 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc00c450 tx=0x7f2fbc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:10.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.171+0000 7f2fcbfff640 1 -- 172.21.15.98:0/1246527245 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc012830 con 0x7f2fd40b3770 2024-09-19T08:40:10.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.171+0000 7f2fcbfff640 1 -- 172.21.15.98:0/1246527245 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fbc017070 con 0x7f2fd40b3770 2024-09-19T08:40:10.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.171+0000 7f2fda12c640 1 -- 172.21.15.98:0/1246527245 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fd414e0f0 con 0x7f2fd40b3770 2024-09-19T08:40:10.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.171+0000 7f2fcbfff640 1 -- 172.21.15.98:0/1246527245 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc012c30 con 0x7f2fd40b3770 2024-09-19T08:40:10.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.171+0000 7f2fda12c640 1 -- 172.21.15.98:0/1246527245 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fd414eee0 con 0x7f2fd40b3770 2024-09-19T08:40:10.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.175+0000 7f2fda12c640 1 -- 172.21.15.98:0/1246527245 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fd414f0e0 con 0x7f2fd40b3770 2024-09-19T08:40:10.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.175+0000 7f2fcbfff640 1 -- 172.21.15.98:0/1246527245 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f2fbc01c460 con 0x7f2fd40b3770 2024-09-19T08:40:10.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.175+0000 7f2fcbfff640 1 --2- 172.21.15.98:0/1246527245 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2fb4067a70 0x7f2fb4069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:10.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.175+0000 7f2fd37fe640 1 --2- 172.21.15.98:0/1246527245 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2fb4067a70 0x7f2fb4069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:10.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.175+0000 7f2fcbfff640 1 -- 172.21.15.98:0/1246527245 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f2fbc085dc0 con 0x7f2fd40b3770 2024-09-19T08:40:10.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.175+0000 7f2fd37fe640 1 --2- 172.21.15.98:0/1246527245 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2fb4067a70 0x7f2fb4069f30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc0ae000 tx=0x7f2fcc0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:10.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.179+0000 7f2fcbfff640 1 -- 172.21.15.98:0/1246527245 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2fd414f0e0 con 0x7f2fd40b3770 2024-09-19T08:40:10.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.367+0000 7f2fda12c640 1 -- 172.21.15.98:0/1246527245 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol58510", "target": ["mon-mgr", ""]}) -- 0x7f2fd40b02e0 con 0x7f2fb4067a70 2024-09-19T08:40:10.372 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:40:10.375+0000 7f36e9c04640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:40:10.373 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:40:10.375+0000 7f36e9c04640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:40:10.373 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:40:10.375+0000 7f36e9c04640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:40:10.373 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:40:10.375+0000 7f36e9c04640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:40:10.373 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:40:10.375+0000 7f36e9c04640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:40:10.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.411+0000 7f2fcbfff640 1 -- 172.21.15.98:0/1246527245 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2fd40b02e0 con 0x7f2fb4067a70 2024-09-19T08:40:10.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.415+0000 7f2fda12c640 1 -- 172.21.15.98:0/1246527245 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2fb4067a70 msgr2=0x7f2fb4069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:10.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.415+0000 7f2fda12c640 1 --2- 172.21.15.98:0/1246527245 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2fb4067a70 0x7f2fb4069f30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc0ae000 tx=0x7f2fcc0a7070 comp rx=0 tx=0).stop 2024-09-19T08:40:10.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.415+0000 7f2fda12c640 1 -- 172.21.15.98:0/1246527245 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fd40b3770 msgr2=0x7f2fd414d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:10.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.415+0000 7f2fda12c640 1 --2- 172.21.15.98:0/1246527245 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fd40b3770 0x7f2fd414d390 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc00c450 tx=0x7f2fbc00c910 comp rx=0 tx=0).stop 2024-09-19T08:40:10.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.415+0000 7f2fda12c640 1 -- 172.21.15.98:0/1246527245 shutdown_connections 2024-09-19T08:40:10.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.415+0000 7f2fda12c640 1 --2- 172.21.15.98:0/1246527245 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2fb4067a70 0x7f2fb4069f30 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:10.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.415+0000 7f2fda12c640 1 --2- 172.21.15.98:0/1246527245 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fd415c730 0x7f2fd414c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:10.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.415+0000 7f2fda12c640 1 --2- 172.21.15.98:0/1246527245 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fd40b3770 0x7f2fd414d390 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:10.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.415+0000 7f2fda12c640 1 --2- 172.21.15.98:0/1246527245 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fd406caf0 0x7f2fd415b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:10.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.415+0000 7f2fda12c640 1 -- 172.21.15.98:0/1246527245 >> 172.21.15.98:0/1246527245 conn(0x7f2fd40b5f50 msgr2=0x7f2fd415a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:40:10.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.415+0000 7f2fda12c640 1 -- 172.21.15.98:0/1246527245 shutdown_connections 2024-09-19T08:40:10.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.415+0000 7f2fda12c640 1 -- 172.21.15.98:0/1246527245 wait complete. 2024-09-19T08:40:10.428 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:40:10.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.539+0000 7ff79dfa2640 1 Processor -- start 2024-09-19T08:40:10.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.539+0000 7ff79dfa2640 1 -- start start 2024-09-19T08:40:10.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.539+0000 7ff79dfa2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff79806d880 0x7ff7980b3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:10.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.539+0000 7ff79dfa2640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff798141110 0x7ff7980b3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:10.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.539+0000 7ff79dfa2640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff79815ea10 con 0x7ff798140d30 2024-09-19T08:40:10.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.539+0000 7ff79dfa2640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff79815e7c0 con 0x7ff798141110 2024-09-19T08:40:10.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.539+0000 7ff79dfa2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff79815ecf0 con 0x7ff79806d880 2024-09-19T08:40:10.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.539+0000 7ff797fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff798141110 0x7ff7980b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:10.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.539+0000 7ff796ffd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff79806d880 0x7ff7980b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:10.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.539+0000 7ff797fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff798141110 0x7ff7980b3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:39134/0 (socket says 172.21.15.98:39134) 2024-09-19T08:40:10.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.539+0000 7ff797fff640 1 -- 172.21.15.98:0/2586369184 learned_addr learned my addr 172.21.15.98:0/2586369184 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:40:10.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.539+0000 7ff7977fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ff798140d30 0x7ff79806d170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:41084/0 (socket says 172.21.15.98:41084) 2024-09-19T08:40:10.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.539+0000 7ff797fff640 1 -- 172.21.15.98:0/2586369184 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff79806d880 msgr2=0x7ff7980b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:40:10.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.539+0000 7ff797fff640 1 --2- 172.21.15.98:0/2586369184 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff79806d880 0x7ff7980b3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:10.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.539+0000 7ff797fff640 1 -- 172.21.15.98:0/2586369184 >> v1:172.21.15.139:6789/0 conn(0x7ff798140d30 legacy=0x7ff79806d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:10.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.539+0000 7ff797fff640 1 -- 172.21.15.98:0/2586369184 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff79815efd0 con 0x7ff798141110 2024-09-19T08:40:10.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.543+0000 7ff797fff640 1 --2- 172.21.15.98:0/2586369184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff798141110 0x7ff7980b3850 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff79815e940 tx=0x7ff79001ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2abc750e4a60af2e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:10.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.543+0000 7ff7967fc640 1 -- 172.21.15.98:0/2586369184 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7900a5880 con 0x7ff798141110 2024-09-19T08:40:10.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.543+0000 7ff7967fc640 1 -- 172.21.15.98:0/2586369184 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7900a5a20 con 0x7ff798141110 2024-09-19T08:40:10.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.543+0000 7ff7967fc640 1 -- 172.21.15.98:0/2586369184 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7900a5df0 con 0x7ff798141110 2024-09-19T08:40:10.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.543+0000 7ff79dfa2640 1 -- 172.21.15.98:0/2586369184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff798141110 msgr2=0x7ff7980b3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:10.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.543+0000 7ff79dfa2640 1 --2- 172.21.15.98:0/2586369184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff798141110 0x7ff7980b3850 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff79815e940 tx=0x7ff79001ca10 comp rx=0 tx=0).stop 2024-09-19T08:40:10.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.543+0000 7ff79dfa2640 1 -- 172.21.15.98:0/2586369184 shutdown_connections 2024-09-19T08:40:10.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.543+0000 7ff79dfa2640 1 --2- 172.21.15.98:0/2586369184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff798141110 0x7ff7980b3850 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:10.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.543+0000 7ff79dfa2640 1 --2- 172.21.15.98:0/2586369184 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff79806d880 0x7ff7980b3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:10.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.543+0000 7ff79dfa2640 1 -- 172.21.15.98:0/2586369184 >> 172.21.15.98:0/2586369184 conn(0x7ff7980b2010 msgr2=0x7ff7980b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:40:10.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.543+0000 7ff79dfa2640 1 -- 172.21.15.98:0/2586369184 shutdown_connections 2024-09-19T08:40:10.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.543+0000 7ff79dfa2640 1 -- 172.21.15.98:0/2586369184 wait complete. 2024-09-19T08:40:10.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.543+0000 7ff79dfa2640 1 Processor -- start 2024-09-19T08:40:10.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.543+0000 7ff79dfa2640 1 -- start start 2024-09-19T08:40:10.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.543+0000 7ff79dfa2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff79806d880 0x7ff798156900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:10.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.543+0000 7ff79dfa2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff798140d30 0x7ff7980ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:10.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.543+0000 7ff79dfa2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff798141110 0x7ff7980aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:10.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.543+0000 7ff79dfa2640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff79815b580 con 0x7ff79806d880 2024-09-19T08:40:10.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.543+0000 7ff79dfa2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff79815b400 con 0x7ff798141110 2024-09-19T08:40:10.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.543+0000 7ff79dfa2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff79815b700 con 0x7ff798140d30 2024-09-19T08:40:10.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.543+0000 7ff7977fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff79806d880 0x7ff798156900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:10.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.543+0000 7ff7977fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff79806d880 0x7ff798156900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34434/0 (socket says 172.21.15.98:34434) 2024-09-19T08:40:10.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.543+0000 7ff7977fe640 1 -- 172.21.15.98:0/76390196 learned_addr learned my addr 172.21.15.98:0/76390196 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:40:10.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.543+0000 7ff797fff640 1 --2- 172.21.15.98:0/76390196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff798141110 0x7ff7980aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:10.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.547+0000 7ff796ffd640 1 --2- 172.21.15.98:0/76390196 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff798140d30 0x7ff7980ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:10.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.547+0000 7ff7977fe640 1 -- 172.21.15.98:0/76390196 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff798140d30 msgr2=0x7ff7980ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:10.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.547+0000 7ff7977fe640 1 --2- 172.21.15.98:0/76390196 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff798140d30 0x7ff7980ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:10.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.547+0000 7ff7977fe640 1 -- 172.21.15.98:0/76390196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff798141110 msgr2=0x7ff7980aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:10.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.547+0000 7ff7977fe640 1 --2- 172.21.15.98:0/76390196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff798141110 0x7ff7980aedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:10.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.547+0000 7ff7977fe640 1 -- 172.21.15.98:0/76390196 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff798071e30 con 0x7ff79806d880 2024-09-19T08:40:10.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.547+0000 7ff796ffd640 1 --2- 172.21.15.98:0/76390196 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff798140d30 0x7ff7980ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:40:10.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.547+0000 7ff797fff640 1 --2- 172.21.15.98:0/76390196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff798141110 0x7ff7980aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:40:10.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.547+0000 7ff7977fe640 1 --2- 172.21.15.98:0/76390196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff79806d880 0x7ff798156900 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff78800c420 tx=0x7ff78800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:10.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.547+0000 7ff77ffff640 1 -- 172.21.15.98:0/76390196 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7880074e0 con 0x7ff79806d880 2024-09-19T08:40:10.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.547+0000 7ff77ffff640 1 -- 172.21.15.98:0/76390196 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff788007680 con 0x7ff79806d880 2024-09-19T08:40:10.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.547+0000 7ff79dfa2640 1 -- 172.21.15.98:0/76390196 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7980737f0 con 0x7ff79806d880 2024-09-19T08:40:10.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.547+0000 7ff77ffff640 1 -- 172.21.15.98:0/76390196 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788004a30 con 0x7ff79806d880 2024-09-19T08:40:10.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.547+0000 7ff79dfa2640 1 -- 172.21.15.98:0/76390196 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff798073db0 con 0x7ff79806d880 2024-09-19T08:40:10.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.547+0000 7ff79dfa2640 1 -- 172.21.15.98:0/76390196 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff798073fb0 con 0x7ff79806d880 2024-09-19T08:40:10.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.547+0000 7ff77ffff640 1 -- 172.21.15.98:0/76390196 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7ff78801a020 con 0x7ff79806d880 2024-09-19T08:40:10.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.547+0000 7ff77ffff640 1 --2- 172.21.15.98:0/76390196 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff784067a70 0x7ff784069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:10.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.547+0000 7ff77ffff640 1 -- 172.21.15.98:0/76390196 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7ff788085820 con 0x7ff79806d880 2024-09-19T08:40:10.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.547+0000 7ff796ffd640 1 --2- 172.21.15.98:0/76390196 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff784067a70 0x7ff784069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:10.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.547+0000 7ff796ffd640 1 --2- 172.21.15.98:0/76390196 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff784067a70 0x7ff784069f30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff7981fa600 tx=0x7ff78c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:10.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.551+0000 7ff77ffff640 1 -- 172.21.15.98:0/76390196 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff798073fb0 con 0x7ff79806d880 2024-09-19T08:40:10.744 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.743+0000 7ff79dfa2640 1 -- 172.21.15.98:0/76390196 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7980cc830 con 0x7ff79806d880 2024-09-19T08:40:10.744 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.743+0000 7ff77ffff640 1 -- 172.21.15.98:0/76390196 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5229 (secure 0 0 0) 0x7ff7880560a0 con 0x7ff79806d880 2024-09-19T08:40:10.744 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:40:10.745 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"btime":"2024-09-19T08:39:55:181532+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/2874086421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":2874086421},{"type":"v1","addr":"172.21.15.139:6833","nonce":2874086421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4996,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/4178563848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":4178563848},{"type":"v1","addr":"172.21.15.139:6835","nonce":4178563848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3584568343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3584568343},{"type":"v1","addr":"172.21.15.98:6835","nonce":3584568343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:39:38.262938+0000","modified":"2024-09-19T08:39:41.276985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/3085042202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3085042202},{"type":"v1","addr":"172.21.15.98:6837","nonce":3085042202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244]},"id":3}]} 2024-09-19T08:40:10.745 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 24 2024-09-19T08:40:10.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.747+0000 7ff79dfa2640 1 -- 172.21.15.98:0/76390196 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff784067a70 msgr2=0x7ff784069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:10.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.747+0000 7ff79dfa2640 1 --2- 172.21.15.98:0/76390196 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff784067a70 0x7ff784069f30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff7981fa600 tx=0x7ff78c008040 comp rx=0 tx=0).stop 2024-09-19T08:40:10.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.747+0000 7ff79dfa2640 1 -- 172.21.15.98:0/76390196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff79806d880 msgr2=0x7ff798156900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:10.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.747+0000 7ff79dfa2640 1 --2- 172.21.15.98:0/76390196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff79806d880 0x7ff798156900 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff78800c420 tx=0x7ff78800c8e0 comp rx=0 tx=0).stop 2024-09-19T08:40:10.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.747+0000 7ff79dfa2640 1 -- 172.21.15.98:0/76390196 shutdown_connections 2024-09-19T08:40:10.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.747+0000 7ff79dfa2640 1 --2- 172.21.15.98:0/76390196 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff784067a70 0x7ff784069f30 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:10.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.747+0000 7ff79dfa2640 1 --2- 172.21.15.98:0/76390196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff798141110 0x7ff7980aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:10.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.747+0000 7ff79dfa2640 1 --2- 172.21.15.98:0/76390196 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff798140d30 0x7ff7980ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:10.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.747+0000 7ff79dfa2640 1 --2- 172.21.15.98:0/76390196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff79806d880 0x7ff798156900 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:10.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.747+0000 7ff79dfa2640 1 -- 172.21.15.98:0/76390196 >> 172.21.15.98:0/76390196 conn(0x7ff7980b2010 msgr2=0x7ff7981586b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:40:10.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.747+0000 7ff79dfa2640 1 -- 172.21.15.98:0/76390196 shutdown_connections 2024-09-19T08:40:10.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:10.747+0000 7ff79dfa2640 1 -- 172.21.15.98:0/76390196 wait complete. 2024-09-19T08:40:10.762 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-19T08:40:10.902 INFO:teuthology.orchestra.run.smithi098.stdout:[{"id":5490,"entity":{"name":{"type":"client","num":5490},"addr":{"type":"any","addr":"172.21.15.98:0","nonce":379128374}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":0.51199558599999995,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.974774896012725,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5490 172.21.15.98:0/379128374","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"y","hostname":"smithi098","pid":"28958","root":"/"}}] 2024-09-19T08:40:10.903 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi098", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28958", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.98:0", "nonce": 379128374, "type": "any" }, "name": { "num": 5490, "type": "client" } }, "id": 5490, "inst": "client.5490 172.21.15.98:0/379128374", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.974774896012725 }, "state": "open", "uptime": 0.511995586 } ] 2024-09-19T08:40:10.903 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:40:11.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.015+0000 7f2fb8c9d640 1 Processor -- start 2024-09-19T08:40:11.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.015+0000 7f2fb8c9d640 1 -- start start 2024-09-19T08:40:11.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.019+0000 7f2fb8c9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2fb406bc90 0x7f2fb406c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:11.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.019+0000 7f2fb8c9d640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fb40ccc60 con 0x7f2fb4158fd0 2024-09-19T08:40:11.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.019+0000 7f2fb8c9d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fb40ccf40 con 0x7f2fb40ce840 2024-09-19T08:40:11.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.019+0000 7f2fb8c9d640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f2fb41593b0 con 0x7f2fb406bc90 2024-09-19T08:40:11.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.019+0000 7f2fb259c640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f2fb4158fd0 0x7f2fb40b3e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36884/0 (socket says 172.21.15.98:36884) 2024-09-19T08:40:11.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.019+0000 7f2fb259c640 1 -- 172.21.15.98:0/1171782039 learned_addr learned my addr 172.21.15.98:0/1171782039 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:40:11.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.019+0000 7f2fb1d9b640 1 --2- 172.21.15.98:0/1171782039 >> v2:172.21.15.98:3300/0 conn(0x7f2fb406bc90 0x7f2fb406c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:11.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.019+0000 7f2fb159a640 1 -- 172.21.15.98:0/1171782039 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 676662341 0 0) 0x7f2fb40ccc60 con 0x7f2fb4158fd0 2024-09-19T08:40:11.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.019+0000 7f2fb159a640 1 -- 172.21.15.98:0/1171782039 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fa0003530 con 0x7f2fb4158fd0 2024-09-19T08:40:11.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.019+0000 7f2fb1d9b640 1 -- 172.21.15.98:0/1171782039 >> v1:172.21.15.139:6789/0 conn(0x7f2fb40ce840 legacy=0x7f2fb40cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:11.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.019+0000 7f2fb1d9b640 1 -- 172.21.15.98:0/1171782039 >> v1:172.21.15.98:6789/0 conn(0x7f2fb4158fd0 legacy=0x7f2fb40b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:11.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.019+0000 7f2fb1d9b640 1 -- 172.21.15.98:0/1171782039 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb40cd220 con 0x7f2fb406bc90 2024-09-19T08:40:11.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.019+0000 7f2fb1d9b640 1 --2- 172.21.15.98:0/1171782039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fb406bc90 0x7f2fb406c120 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c0097c0 tx=0x7f2f9c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c1e521ce4fb978 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:11.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.019+0000 7f2fb159a640 1 -- 172.21.15.98:0/1171782039 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c0b0070 con 0x7f2fb406bc90 2024-09-19T08:40:11.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.019+0000 7f2fb159a640 1 -- 172.21.15.98:0/1171782039 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f9c0ab600 con 0x7f2fb406bc90 2024-09-19T08:40:11.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.019+0000 7f2fb159a640 1 -- 172.21.15.98:0/1171782039 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c0ab9b0 con 0x7f2fb406bc90 2024-09-19T08:40:11.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.023+0000 7f2fb8c9d640 1 -- 172.21.15.98:0/1171782039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fb406bc90 msgr2=0x7f2fb406c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:11.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.023+0000 7f2fb8c9d640 1 --2- 172.21.15.98:0/1171782039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fb406bc90 0x7f2fb406c120 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c0097c0 tx=0x7f2f9c01c410 comp rx=0 tx=0).stop 2024-09-19T08:40:11.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.023+0000 7f2fb8c9d640 1 -- 172.21.15.98:0/1171782039 shutdown_connections 2024-09-19T08:40:11.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.023+0000 7f2fb8c9d640 1 --2- 172.21.15.98:0/1171782039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fb406bc90 0x7f2fb406c120 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:11.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.023+0000 7f2fb8c9d640 1 -- 172.21.15.98:0/1171782039 >> 172.21.15.98:0/1171782039 conn(0x7f2fb40b2010 msgr2=0x7f2fb40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:40:11.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.023+0000 7f2fb8c9d640 1 -- 172.21.15.98:0/1171782039 shutdown_connections 2024-09-19T08:40:11.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.023+0000 7f2fb8c9d640 1 -- 172.21.15.98:0/1171782039 wait complete. 2024-09-19T08:40:11.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.023+0000 7f2fb8c9d640 1 Processor -- start 2024-09-19T08:40:11.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.023+0000 7f2fb8c9d640 1 -- start start 2024-09-19T08:40:11.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.023+0000 7f2fb8c9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fb406bc90 0x7f2fb415f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:11.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.023+0000 7f2fb8c9d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fb40ce840 0x7f2fb4155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:11.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.023+0000 7f2fb8c9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fb4158fd0 0x7f2fb41fa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:11.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.023+0000 7f2fb8c9d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb415c920 con 0x7f2fb4158fd0 2024-09-19T08:40:11.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.023+0000 7f2fb8c9d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb415c7a0 con 0x7f2fb406bc90 2024-09-19T08:40:11.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.023+0000 7f2fb8c9d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2fb4169120 con 0x7f2fb40ce840 2024-09-19T08:40:11.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.023+0000 7f2fb2d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fb4158fd0 0x7f2fb41fa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:11.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.023+0000 7f2fb2d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fb4158fd0 0x7f2fb41fa3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34456/0 (socket says 172.21.15.98:34456) 2024-09-19T08:40:11.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.023+0000 7f2fb2d9d640 1 -- 172.21.15.98:0/640119356 learned_addr learned my addr 172.21.15.98:0/640119356 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:40:11.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.023+0000 7f2fb259c640 1 --2- 172.21.15.98:0/640119356 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fb406bc90 0x7f2fb415f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:11.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.023+0000 7f2fb1d9b640 1 --2- 172.21.15.98:0/640119356 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fb40ce840 0x7f2fb4155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:11.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.023+0000 7f2fb2d9d640 1 -- 172.21.15.98:0/640119356 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fb40ce840 msgr2=0x7f2fb4155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:11.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.023+0000 7f2fb2d9d640 1 --2- 172.21.15.98:0/640119356 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fb40ce840 0x7f2fb4155c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:11.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.023+0000 7f2fb2d9d640 1 -- 172.21.15.98:0/640119356 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fb406bc90 msgr2=0x7f2fb415f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:11.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.023+0000 7f2fb2d9d640 1 --2- 172.21.15.98:0/640119356 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fb406bc90 0x7f2fb415f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:11.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.023+0000 7f2fb2d9d640 1 -- 172.21.15.98:0/640119356 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb415e110 con 0x7f2fb4158fd0 2024-09-19T08:40:11.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.023+0000 7f2fb259c640 1 --2- 172.21.15.98:0/640119356 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fb406bc90 0x7f2fb415f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:40:11.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.023+0000 7f2fb2d9d640 1 --2- 172.21.15.98:0/640119356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fb4158fd0 0x7f2fb41fa3b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400c390 tx=0x7f2fa400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:11.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.023+0000 7f2faaffd640 1 -- 172.21.15.98:0/640119356 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa40052d0 con 0x7f2fb4158fd0 2024-09-19T08:40:11.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.023+0000 7f2faaffd640 1 -- 172.21.15.98:0/640119356 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fa4005470 con 0x7f2fb4158fd0 2024-09-19T08:40:11.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.023+0000 7f2fb8c9d640 1 -- 172.21.15.98:0/640119356 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb415d070 con 0x7f2fb4158fd0 2024-09-19T08:40:11.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.023+0000 7f2faaffd640 1 -- 172.21.15.98:0/640119356 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4007240 con 0x7f2fb4158fd0 2024-09-19T08:40:11.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.023+0000 7f2fb8c9d640 1 -- 172.21.15.98:0/640119356 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb415d630 con 0x7f2fb4158fd0 2024-09-19T08:40:11.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.023+0000 7f2fb8c9d640 1 -- 172.21.15.98:0/640119356 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fb406c120 con 0x7f2fb4158fd0 2024-09-19T08:40:11.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.027+0000 7f2faaffd640 1 -- 172.21.15.98:0/640119356 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 7) ==== 87598+0+0 (secure 0 0 0) 0x7f2fa40073e0 con 0x7f2fb4158fd0 2024-09-19T08:40:11.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.027+0000 7f2faaffd640 1 --2- 172.21.15.98:0/640119356 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2f94067a70 0x7f2f94069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:11.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.027+0000 7f2faaffd640 1 -- 172.21.15.98:0/640119356 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f2fa4085470 con 0x7f2fb4158fd0 2024-09-19T08:40:11.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.027+0000 7f2fb259c640 1 --2- 172.21.15.98:0/640119356 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2f94067a70 0x7f2f94069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:11.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.027+0000 7f2fb259c640 1 --2- 172.21.15.98:0/640119356 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2f94067a70 0x7f2f94069f30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2fac001570 tx=0x7f2fac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:11.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.031+0000 7f2faaffd640 1 -- 172.21.15.98:0/640119356 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2fa4055cf0 con 0x7f2fb4158fd0 2024-09-19T08:40:11.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.211+0000 7f2fb8c9d640 1 -- 172.21.15.98:0/640119356 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2fb40757a0 con 0x7f2fb4158fd0 2024-09-19T08:40:11.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.211+0000 7f2faaffd640 1 -- 172.21.15.98:0/640119356 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5229 (secure 0 0 0) 0x7f2fa40555f0 con 0x7f2fb4158fd0 2024-09-19T08:40:11.212 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:40:11.212 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"btime":"2024-09-19T08:39:55:181532+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/2874086421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":2874086421},{"type":"v1","addr":"172.21.15.139:6833","nonce":2874086421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4996,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/4178563848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":4178563848},{"type":"v1","addr":"172.21.15.139:6835","nonce":4178563848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3584568343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3584568343},{"type":"v1","addr":"172.21.15.98:6835","nonce":3584568343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:39:38.262938+0000","modified":"2024-09-19T08:39:41.276985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/3085042202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3085042202},{"type":"v1","addr":"172.21.15.98:6837","nonce":3085042202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244]},"id":3}]} 2024-09-19T08:40:11.212 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 24 2024-09-19T08:40:11.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.215+0000 7f2fb8c9d640 1 -- 172.21.15.98:0/640119356 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2f94067a70 msgr2=0x7f2f94069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:11.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.215+0000 7f2fb8c9d640 1 --2- 172.21.15.98:0/640119356 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2f94067a70 0x7f2f94069f30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2fac001570 tx=0x7f2fac009040 comp rx=0 tx=0).stop 2024-09-19T08:40:11.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.215+0000 7f2fb8c9d640 1 -- 172.21.15.98:0/640119356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fb4158fd0 msgr2=0x7f2fb41fa3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:11.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.215+0000 7f2fb8c9d640 1 --2- 172.21.15.98:0/640119356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fb4158fd0 0x7f2fb41fa3b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400c390 tx=0x7f2fa400c850 comp rx=0 tx=0).stop 2024-09-19T08:40:11.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.215+0000 7f2fb8c9d640 1 -- 172.21.15.98:0/640119356 shutdown_connections 2024-09-19T08:40:11.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.215+0000 7f2fb8c9d640 1 --2- 172.21.15.98:0/640119356 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2f94067a70 0x7f2f94069f30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:11.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.215+0000 7f2fb8c9d640 1 --2- 172.21.15.98:0/640119356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fb4158fd0 0x7f2fb41fa3b0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:11.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.215+0000 7f2fb8c9d640 1 --2- 172.21.15.98:0/640119356 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fb40ce840 0x7f2fb4155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:11.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.215+0000 7f2fb8c9d640 1 --2- 172.21.15.98:0/640119356 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fb406bc90 0x7f2fb415f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:11.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.215+0000 7f2fb8c9d640 1 -- 172.21.15.98:0/640119356 >> 172.21.15.98:0/640119356 conn(0x7f2fb40b2010 msgr2=0x7f2fb40b3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:40:11.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.215+0000 7f2fb8c9d640 1 -- 172.21.15.98:0/640119356 shutdown_connections 2024-09-19T08:40:11.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:11.215+0000 7f2fb8c9d640 1 -- 172.21.15.98:0/640119356 wait complete. 2024-09-19T08:40:11.229 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-19T08:40:11.355 INFO:teuthology.orchestra.run.smithi098.stdout:[{"id":5490,"entity":{"name":{"type":"client","num":5490},"addr":{"type":"any","addr":"172.21.15.98:0","nonce":379128374}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":0.96799165600000003,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.951635022521874,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5490 172.21.15.98:0/379128374","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"y","hostname":"smithi098","pid":"28958","root":"/"}}] 2024-09-19T08:40:11.356 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi098", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28958", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.98:0", "nonce": 379128374, "type": "any" }, "name": { "num": 5490, "type": "client" } }, "id": 5490, "inst": "client.5490 172.21.15.98:0/379128374", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.951635022521874 }, "state": "open", "uptime": 0.967991656 } ] 2024-09-19T08:40:11.356 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-19T08:40:16.357 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:40:16.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.491+0000 7f312be2b640 1 Processor -- start 2024-09-19T08:40:16.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.491+0000 7f312be2b640 1 -- start start 2024-09-19T08:40:16.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.491+0000 7f312be2b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3124158fd0 0x7f31240b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:16.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.491+0000 7f312be2b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f31240b4360 0x7f312406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:16.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.491+0000 7f312be2b640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31240ccc60 con 0x7f312406c1d0 2024-09-19T08:40:16.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.491+0000 7f312be2b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f312406c620 con 0x7f31240b4360 2024-09-19T08:40:16.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.491+0000 7f312be2b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f31241593b0 con 0x7f3124158fd0 2024-09-19T08:40:16.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.491+0000 7f312a3c8640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f312406c1d0 0x7f312415efc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36890/0 (socket says 172.21.15.98:36890) 2024-09-19T08:40:16.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.491+0000 7f312a3c8640 1 -- 172.21.15.98:0/2293881440 learned_addr learned my addr 172.21.15.98:0/2293881440 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:40:16.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.491+0000 7f31293c6640 1 --2- 172.21.15.98:0/2293881440 >> v2:172.21.15.139:3300/0 conn(0x7f31240b4360 0x7f312406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:16.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.491+0000 7f3129bc7640 1 --2- 172.21.15.98:0/2293881440 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3124158fd0 0x7f31240b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:16.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.491+0000 7f3128bc5640 1 -- 172.21.15.98:0/2293881440 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 821418391 0 0) 0x7f31240ccc60 con 0x7f312406c1d0 2024-09-19T08:40:16.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.491+0000 7f3128bc5640 1 -- 172.21.15.98:0/2293881440 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3110003530 con 0x7f312406c1d0 2024-09-19T08:40:16.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.491+0000 7f3128bc5640 1 -- 172.21.15.98:0/2293881440 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 802530958 0 0) 0x7f3110003530 con 0x7f312406c1d0 2024-09-19T08:40:16.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.491+0000 7f3128bc5640 1 -- 172.21.15.98:0/2293881440 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3124158fd0 msgr2=0x7f31240b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:40:16.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.491+0000 7f3128bc5640 1 --2- 172.21.15.98:0/2293881440 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3124158fd0 0x7f31240b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:16.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.491+0000 7f3128bc5640 1 -- 172.21.15.98:0/2293881440 >> v2:172.21.15.139:3300/0 conn(0x7f31240b4360 msgr2=0x7f312406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:40:16.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.491+0000 7f3128bc5640 1 --2- 172.21.15.98:0/2293881440 >> v2:172.21.15.139:3300/0 conn(0x7f31240b4360 0x7f312406bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:16.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.491+0000 7f3128bc5640 1 -- 172.21.15.98:0/2293881440 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31240ccf40 con 0x7f312406c1d0 2024-09-19T08:40:16.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.491+0000 7f3128bc5640 1 -- 172.21.15.98:0/2293881440 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f311c001a00 con 0x7f312406c1d0 2024-09-19T08:40:16.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.491+0000 7f3128bc5640 1 -- 172.21.15.98:0/2293881440 >> v1:172.21.15.98:6789/0 conn(0x7f312406c1d0 legacy=0x7f312415efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:16.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.491+0000 7f3128bc5640 1 --2- 172.21.15.98:0/2293881440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31100039c0 0x7f3110003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:16.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.491+0000 7f3128bc5640 1 --2- 172.21.15.98:0/2293881440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3110004810 0x7f3110004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:16.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.491+0000 7f3128bc5640 1 --2- 172.21.15.98:0/2293881440 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3124158fd0 0x7f31100051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:16.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.491+0000 7f3128bc5640 1 -- 172.21.15.98:0/2293881440 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f312406c620 con 0x7f3110004810 2024-09-19T08:40:16.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.491+0000 7f3128bc5640 1 -- 172.21.15.98:0/2293881440 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f31241593b0 con 0x7f31100039c0 2024-09-19T08:40:16.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.491+0000 7f3128bc5640 1 -- 172.21.15.98:0/2293881440 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f311000b9d0 con 0x7f3124158fd0 2024-09-19T08:40:16.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.491+0000 7f31293c6640 1 --2- 172.21.15.98:0/2293881440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3110004810 0x7f3110004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:16.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.495+0000 7f3129bc7640 1 --2- 172.21.15.98:0/2293881440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31100039c0 0x7f3110003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:16.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.495+0000 7f312a3c8640 1 --2- 172.21.15.98:0/2293881440 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3124158fd0 0x7f31100051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:16.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.495+0000 7f31293c6640 1 -- 172.21.15.98:0/2293881440 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3124158fd0 msgr2=0x7f31100051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:40:16.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.495+0000 7f31293c6640 1 --2- 172.21.15.98:0/2293881440 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3124158fd0 0x7f31100051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:16.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.495+0000 7f31293c6640 1 -- 172.21.15.98:0/2293881440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31100039c0 msgr2=0x7f3110003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:40:16.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.495+0000 7f31293c6640 1 --2- 172.21.15.98:0/2293881440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31100039c0 0x7f3110003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:16.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.495+0000 7f31293c6640 1 -- 172.21.15.98:0/2293881440 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3110003530 con 0x7f3110004810 2024-09-19T08:40:16.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.495+0000 7f3129bc7640 1 --2- 172.21.15.98:0/2293881440 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f31100039c0 0x7f3110003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:40:16.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.495+0000 7f312a3c8640 1 --2- 172.21.15.98:0/2293881440 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3124158fd0 0x7f31100051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:40:16.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.495+0000 7f31293c6640 1 --2- 172.21.15.98:0/2293881440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3110004810 0x7f3110004c90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3114009ae0 tx=0x7f311401ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc0c6c4b5720c28 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:16.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.495+0000 7f3128bc5640 1 -- 172.21.15.98:0/2293881440 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31140b2070 con 0x7f3110004810 2024-09-19T08:40:16.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.495+0000 7f3128bc5640 1 -- 172.21.15.98:0/2293881440 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31140a5530 con 0x7f3110004810 2024-09-19T08:40:16.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.495+0000 7f3128bc5640 1 -- 172.21.15.98:0/2293881440 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31140096b0 con 0x7f3110004810 2024-09-19T08:40:16.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.495+0000 7f312be2b640 1 -- 172.21.15.98:0/2293881440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3110004810 msgr2=0x7f3110004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:16.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.495+0000 7f312be2b640 1 --2- 172.21.15.98:0/2293881440 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3110004810 0x7f3110004c90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3114009ae0 tx=0x7f311401ca40 comp rx=0 tx=0).stop 2024-09-19T08:40:16.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.495+0000 7f312a3c8640 1 -- 172.21.15.98:0/2293881440 reap_dead start 2024-09-19T08:40:16.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.495+0000 7f312be2b640 1 -- 172.21.15.98:0/2293881440 shutdown_connections 2024-09-19T08:40:16.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.495+0000 7f312be2b640 1 -- 172.21.15.98:0/2293881440 >> 172.21.15.98:0/2293881440 conn(0x7f31240b2010 msgr2=0x7f31240b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:40:16.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.495+0000 7f312be2b640 1 -- 172.21.15.98:0/2293881440 shutdown_connections 2024-09-19T08:40:16.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.495+0000 7f312be2b640 1 -- 172.21.15.98:0/2293881440 wait complete. 2024-09-19T08:40:16.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.499+0000 7f312be2b640 1 Processor -- start 2024-09-19T08:40:16.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.499+0000 7f312be2b640 1 -- start start 2024-09-19T08:40:16.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.499+0000 7f312be2b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3110004810 0x7f3124167320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:16.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.499+0000 7f312be2b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f312415dd00 0x7f3124154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:16.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.499+0000 7f312be2b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31241689a0 0x7f31240ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:16.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.499+0000 7f312be2b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f312415c920 con 0x7f31241689a0 2024-09-19T08:40:16.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.499+0000 7f312be2b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f312415c7a0 con 0x7f312415dd00 2024-09-19T08:40:16.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.499+0000 7f312be2b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3124166100 con 0x7f3110004810 2024-09-19T08:40:16.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.499+0000 7f312a3c8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31241689a0 0x7f31240ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:16.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.499+0000 7f312a3c8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31241689a0 0x7f31240ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34484/0 (socket says 172.21.15.98:34484) 2024-09-19T08:40:16.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.499+0000 7f312a3c8640 1 -- 172.21.15.98:0/1511968261 learned_addr learned my addr 172.21.15.98:0/1511968261 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:40:16.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.499+0000 7f3129bc7640 1 --2- 172.21.15.98:0/1511968261 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3110004810 0x7f3124167320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:16.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.499+0000 7f31293c6640 1 --2- 172.21.15.98:0/1511968261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f312415dd00 0x7f3124154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:16.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.499+0000 7f312a3c8640 1 -- 172.21.15.98:0/1511968261 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3110004810 msgr2=0x7f3124167320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:16.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.499+0000 7f312a3c8640 1 --2- 172.21.15.98:0/1511968261 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3110004810 0x7f3124167320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:16.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.499+0000 7f312a3c8640 1 -- 172.21.15.98:0/1511968261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f312415dd00 msgr2=0x7f3124154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:16.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.499+0000 7f312a3c8640 1 --2- 172.21.15.98:0/1511968261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f312415dd00 0x7f3124154010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:16.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.499+0000 7f312a3c8640 1 -- 172.21.15.98:0/1511968261 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3124154550 con 0x7f31241689a0 2024-09-19T08:40:16.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.499+0000 7f312a3c8640 1 --2- 172.21.15.98:0/1511968261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31241689a0 0x7f31240ae8b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3124070e60 tx=0x7f311c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:16.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.499+0000 7f311a7fc640 1 -- 172.21.15.98:0/1511968261 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c004d70 con 0x7f31241689a0 2024-09-19T08:40:16.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.499+0000 7f311a7fc640 1 -- 172.21.15.98:0/1511968261 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f311c00bb20 con 0x7f31241689a0 2024-09-19T08:40:16.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.499+0000 7f312be2b640 1 -- 172.21.15.98:0/1511968261 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31240cc7b0 con 0x7f31241689a0 2024-09-19T08:40:16.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.499+0000 7f311a7fc640 1 -- 172.21.15.98:0/1511968261 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311c007070 con 0x7f31241689a0 2024-09-19T08:40:16.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.499+0000 7f312be2b640 1 -- 172.21.15.98:0/1511968261 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3124167be0 con 0x7f31241689a0 2024-09-19T08:40:16.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.499+0000 7f312be2b640 1 -- 172.21.15.98:0/1511968261 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f312406bc90 con 0x7f31241689a0 2024-09-19T08:40:16.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.499+0000 7f311a7fc640 1 -- 172.21.15.98:0/1511968261 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7f311c01b020 con 0x7f31241689a0 2024-09-19T08:40:16.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.503+0000 7f311a7fc640 1 --2- 172.21.15.98:0/1511968261 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3108067b30 0x7f3108069ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:16.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.503+0000 7f311a7fc640 1 -- 172.21.15.98:0/1511968261 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f311c086290 con 0x7f31241689a0 2024-09-19T08:40:16.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.503+0000 7f3129bc7640 1 --2- 172.21.15.98:0/1511968261 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3108067b30 0x7f3108069ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:16.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.503+0000 7f3129bc7640 1 --2- 172.21.15.98:0/1511968261 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3108067b30 0x7f3108069ff0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f312415cd00 tx=0x7f3120008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:16.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.507+0000 7f311a7fc640 1 -- 172.21.15.98:0/1511968261 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f311c0569b0 con 0x7f31241689a0 2024-09-19T08:40:16.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.703+0000 7f312be2b640 1 -- 172.21.15.98:0/1511968261 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31240d0f20 con 0x7f31241689a0 2024-09-19T08:40:16.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.703+0000 7f311a7fc640 1 -- 172.21.15.98:0/1511968261 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5229 (secure 0 0 0) 0x7f311c0562b0 con 0x7f31241689a0 2024-09-19T08:40:16.704 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:40:16.704 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"btime":"2024-09-19T08:39:55:181532+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/2874086421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":2874086421},{"type":"v1","addr":"172.21.15.139:6833","nonce":2874086421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4996,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/4178563848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":4178563848},{"type":"v1","addr":"172.21.15.139:6835","nonce":4178563848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3584568343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3584568343},{"type":"v1","addr":"172.21.15.98:6835","nonce":3584568343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:39:38.262938+0000","modified":"2024-09-19T08:39:41.276985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/3085042202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3085042202},{"type":"v1","addr":"172.21.15.98:6837","nonce":3085042202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244]},"id":3}]} 2024-09-19T08:40:16.704 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 24 2024-09-19T08:40:16.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.707+0000 7f312be2b640 1 -- 172.21.15.98:0/1511968261 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3108067b30 msgr2=0x7f3108069ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:16.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.707+0000 7f312be2b640 1 --2- 172.21.15.98:0/1511968261 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3108067b30 0x7f3108069ff0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f312415cd00 tx=0x7f3120008040 comp rx=0 tx=0).stop 2024-09-19T08:40:16.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.707+0000 7f312be2b640 1 -- 172.21.15.98:0/1511968261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31241689a0 msgr2=0x7f31240ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:16.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.707+0000 7f312be2b640 1 --2- 172.21.15.98:0/1511968261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31241689a0 0x7f31240ae8b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3124070e60 tx=0x7f311c00b4a0 comp rx=0 tx=0).stop 2024-09-19T08:40:16.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.707+0000 7f312be2b640 1 -- 172.21.15.98:0/1511968261 shutdown_connections 2024-09-19T08:40:16.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.707+0000 7f312be2b640 1 --2- 172.21.15.98:0/1511968261 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3108067b30 0x7f3108069ff0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:16.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.707+0000 7f312be2b640 1 --2- 172.21.15.98:0/1511968261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31241689a0 0x7f31240ae8b0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:16.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.707+0000 7f312be2b640 1 --2- 172.21.15.98:0/1511968261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f312415dd00 0x7f3124154010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:16.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.707+0000 7f312be2b640 1 --2- 172.21.15.98:0/1511968261 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3110004810 0x7f3124167320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:16.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.707+0000 7f312be2b640 1 -- 172.21.15.98:0/1511968261 >> 172.21.15.98:0/1511968261 conn(0x7f31240b2010 msgr2=0x7f31240b0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:40:16.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.707+0000 7f312be2b640 1 -- 172.21.15.98:0/1511968261 shutdown_connections 2024-09-19T08:40:16.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:16.707+0000 7f312be2b640 1 -- 172.21.15.98:0/1511968261 wait complete. 2024-09-19T08:40:16.723 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-19T08:40:16.867 INFO:teuthology.orchestra.run.smithi098.stdout:[{"id":5490,"entity":{"name":{"type":"client","num":5490},"addr":{"type":"any","addr":"172.21.15.98:0","nonce":379128374}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":13,"uptime":6.4799441189999998,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.673846665231334,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5490 172.21.15.98:0/379128374","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"y","hostname":"smithi098","pid":"28958","root":"/"}}] 2024-09-19T08:40:16.868 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi098", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28958", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.98:0", "nonce": 379128374, "type": "any" }, "name": { "num": 5490, "type": "client" } }, "id": 5490, "inst": "client.5490 172.21.15.98:0/379128374", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 13, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.673846665231334 }, "state": "open", "uptime": 6.479944119 } ] 2024-09-19T08:40:16.868 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-19T08:40:17.309 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:40:17.312 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:40:21.870 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:40:22.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.002+0000 7fe667f47640 1 Processor -- start 2024-09-19T08:40:22.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.002+0000 7fe667f47640 1 -- start start 2024-09-19T08:40:22.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.002+0000 7fe667f47640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe6600b9970 0x7fe6600b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:22.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.002+0000 7fe667f47640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe660068440 con 0x7fe6600d4820 2024-09-19T08:40:22.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.002+0000 7fe667f47640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe660162a90 con 0x7fe6600b7f80 2024-09-19T08:40:22.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.002+0000 7fe667f47640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fe660163050 con 0x7fe6600b9970 2024-09-19T08:40:22.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.002+0000 7fe6664e4640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe6600b9970 0x7fe6600b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:22.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.002+0000 7fe6654e2640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fe6600d4820 0x7fe6601621b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:60456/0 (socket says 172.21.15.98:60456) 2024-09-19T08:40:22.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.002+0000 7fe6654e2640 1 -- 172.21.15.98:0/865622053 learned_addr learned my addr 172.21.15.98:0/865622053 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:40:22.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.002+0000 7fe6664e4640 1 --2- 172.21.15.98:0/865622053 >> v2:172.21.15.98:3300/0 conn(0x7fe6600b9970 0x7fe6600b9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48394/0 (socket says 172.21.15.98:48394) 2024-09-19T08:40:22.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.002+0000 7fe664ce1640 1 -- 172.21.15.98:0/865622053 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4164578175 0 0) 0x7fe660068440 con 0x7fe6600d4820 2024-09-19T08:40:22.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.002+0000 7fe664ce1640 1 -- 172.21.15.98:0/865622053 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe64c003530 con 0x7fe6600d4820 2024-09-19T08:40:22.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.002+0000 7fe6664e4640 1 -- 172.21.15.98:0/865622053 >> v1:172.21.15.139:6789/0 conn(0x7fe6600b7f80 legacy=0x7fe66015f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:22.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.002+0000 7fe6664e4640 1 -- 172.21.15.98:0/865622053 >> v1:172.21.15.98:6789/0 conn(0x7fe6600d4820 legacy=0x7fe6601621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:22.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.002+0000 7fe6664e4640 1 -- 172.21.15.98:0/865622053 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe660162d70 con 0x7fe6600b9970 2024-09-19T08:40:22.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.006+0000 7fe6664e4640 1 --2- 172.21.15.98:0/865622053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6600b9970 0x7fe6600b9de0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe658009d90 tx=0x7fe65801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c59490d7a3137ff3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:22.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.006+0000 7fe664ce1640 1 -- 172.21.15.98:0/865622053 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6580b2070 con 0x7fe6600b9970 2024-09-19T08:40:22.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.006+0000 7fe664ce1640 1 -- 172.21.15.98:0/865622053 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6580a5830 con 0x7fe6600b9970 2024-09-19T08:40:22.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.006+0000 7fe664ce1640 1 -- 172.21.15.98:0/865622053 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6580a5be0 con 0x7fe6600b9970 2024-09-19T08:40:22.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.006+0000 7fe667f47640 1 -- 172.21.15.98:0/865622053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6600b9970 msgr2=0x7fe6600b9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:22.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.006+0000 7fe667f47640 1 --2- 172.21.15.98:0/865622053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6600b9970 0x7fe6600b9de0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe658009d90 tx=0x7fe65801c970 comp rx=0 tx=0).stop 2024-09-19T08:40:22.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.006+0000 7fe667f47640 1 -- 172.21.15.98:0/865622053 shutdown_connections 2024-09-19T08:40:22.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.006+0000 7fe667f47640 1 --2- 172.21.15.98:0/865622053 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6600b9970 0x7fe6600b9de0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:22.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.006+0000 7fe667f47640 1 -- 172.21.15.98:0/865622053 >> 172.21.15.98:0/865622053 conn(0x7fe660074e40 msgr2=0x7fe660075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:40:22.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.006+0000 7fe667f47640 1 -- 172.21.15.98:0/865622053 shutdown_connections 2024-09-19T08:40:22.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.006+0000 7fe667f47640 1 -- 172.21.15.98:0/865622053 wait complete. 2024-09-19T08:40:22.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.006+0000 7fe667f47640 1 Processor -- start 2024-09-19T08:40:22.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.006+0000 7fe667f47640 1 -- start start 2024-09-19T08:40:22.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.006+0000 7fe667f47640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6600b7f80 0x7fe660140e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:22.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.006+0000 7fe667f47640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6600b9970 0x7fe6600cca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:22.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.006+0000 7fe667f47640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6600d4820 0x7fe66013e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:22.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.006+0000 7fe667f47640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe660144710 con 0x7fe6600d4820 2024-09-19T08:40:22.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.006+0000 7fe667f47640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe660144590 con 0x7fe6600b7f80 2024-09-19T08:40:22.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.006+0000 7fe667f47640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe660144890 con 0x7fe6600b9970 2024-09-19T08:40:22.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.006+0000 7fe6664e4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6600d4820 0x7fe66013e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:22.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.006+0000 7fe6664e4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6600d4820 0x7fe66013e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48406/0 (socket says 172.21.15.98:48406) 2024-09-19T08:40:22.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.006+0000 7fe6664e4640 1 -- 172.21.15.98:0/1455385361 learned_addr learned my addr 172.21.15.98:0/1455385361 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:40:22.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.006+0000 7fe6664e4640 1 -- 172.21.15.98:0/1455385361 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6600b9970 msgr2=0x7fe6600cca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:22.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.006+0000 7fe6654e2640 1 --2- 172.21.15.98:0/1455385361 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6600b9970 0x7fe6600cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:22.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.006+0000 7fe665ce3640 1 --2- 172.21.15.98:0/1455385361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6600b7f80 0x7fe660140e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:22.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.006+0000 7fe6664e4640 1 --2- 172.21.15.98:0/1455385361 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6600b9970 0x7fe6600cca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:22.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.006+0000 7fe6664e4640 1 -- 172.21.15.98:0/1455385361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6600b7f80 msgr2=0x7fe660140e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:22.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.006+0000 7fe6664e4640 1 --2- 172.21.15.98:0/1455385361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6600b7f80 0x7fe660140e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:22.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.006+0000 7fe6664e4640 1 -- 172.21.15.98:0/1455385361 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6600cd180 con 0x7fe6600d4820 2024-09-19T08:40:22.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.006+0000 7fe665ce3640 1 --2- 172.21.15.98:0/1455385361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6600b7f80 0x7fe660140e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:40:22.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.006+0000 7fe6664e4640 1 --2- 172.21.15.98:0/1455385361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6600d4820 0x7fe66013e1b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe658009d90 tx=0x7fe658019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:22.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.010+0000 7fe6567fc640 1 -- 172.21.15.98:0/1455385361 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6580b2050 con 0x7fe6600d4820 2024-09-19T08:40:22.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.010+0000 7fe6567fc640 1 -- 172.21.15.98:0/1455385361 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6580b3040 con 0x7fe6600d4820 2024-09-19T08:40:22.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.010+0000 7fe6567fc640 1 -- 172.21.15.98:0/1455385361 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe658003ba0 con 0x7fe6600d4820 2024-09-19T08:40:22.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.010+0000 7fe667f47640 1 -- 172.21.15.98:0/1455385361 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6600cd460 con 0x7fe6600d4820 2024-09-19T08:40:22.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.010+0000 7fe667f47640 1 -- 172.21.15.98:0/1455385361 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6600cda20 con 0x7fe6600d4820 2024-09-19T08:40:22.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.010+0000 7fe667f47640 1 -- 172.21.15.98:0/1455385361 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6600b9de0 con 0x7fe6600d4820 2024-09-19T08:40:22.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.010+0000 7fe6567fc640 1 -- 172.21.15.98:0/1455385361 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7fe6580c1060 con 0x7fe6600d4820 2024-09-19T08:40:22.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.010+0000 7fe6567fc640 1 --2- 172.21.15.98:0/1455385361 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe644067b00 0x7fe644069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:22.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.010+0000 7fe6567fc640 1 -- 172.21.15.98:0/1455385361 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7fe658120430 con 0x7fe6600d4820 2024-09-19T08:40:22.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.010+0000 7fe665ce3640 1 --2- 172.21.15.98:0/1455385361 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe644067b00 0x7fe644069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:22.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.010+0000 7fe665ce3640 1 --2- 172.21.15.98:0/1455385361 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe644067b00 0x7fe644069fc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe660164820 tx=0x7fe65c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:22.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.014+0000 7fe6567fc640 1 -- 172.21.15.98:0/1455385361 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe658016560 con 0x7fe6600d4820 2024-09-19T08:40:22.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.206+0000 7fe667f47640 1 -- 172.21.15.98:0/1455385361 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe660068050 con 0x7fe6600d4820 2024-09-19T08:40:22.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.206+0000 7fe6567fc640 1 -- 172.21.15.98:0/1455385361 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5229 (secure 0 0 0) 0x7fe6580f0b30 con 0x7fe6600d4820 2024-09-19T08:40:22.209 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:40:22.209 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"btime":"2024-09-19T08:39:55:181532+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/2874086421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":2874086421},{"type":"v1","addr":"172.21.15.139:6833","nonce":2874086421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4996,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/4178563848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":4178563848},{"type":"v1","addr":"172.21.15.139:6835","nonce":4178563848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3584568343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3584568343},{"type":"v1","addr":"172.21.15.98:6835","nonce":3584568343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:39:38.262938+0000","modified":"2024-09-19T08:39:41.276985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/3085042202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3085042202},{"type":"v1","addr":"172.21.15.98:6837","nonce":3085042202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244]},"id":3}]} 2024-09-19T08:40:22.209 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 24 2024-09-19T08:40:22.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.210+0000 7fe667f47640 1 -- 172.21.15.98:0/1455385361 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe644067b00 msgr2=0x7fe644069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:22.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.210+0000 7fe667f47640 1 --2- 172.21.15.98:0/1455385361 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe644067b00 0x7fe644069fc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe660164820 tx=0x7fe65c009040 comp rx=0 tx=0).stop 2024-09-19T08:40:22.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.210+0000 7fe667f47640 1 -- 172.21.15.98:0/1455385361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6600d4820 msgr2=0x7fe66013e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:22.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.210+0000 7fe667f47640 1 --2- 172.21.15.98:0/1455385361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6600d4820 0x7fe66013e1b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe658009d90 tx=0x7fe658019710 comp rx=0 tx=0).stop 2024-09-19T08:40:22.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.210+0000 7fe667f47640 1 -- 172.21.15.98:0/1455385361 shutdown_connections 2024-09-19T08:40:22.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.210+0000 7fe667f47640 1 --2- 172.21.15.98:0/1455385361 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe644067b00 0x7fe644069fc0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:22.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.210+0000 7fe667f47640 1 --2- 172.21.15.98:0/1455385361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6600d4820 0x7fe66013e1b0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:22.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.210+0000 7fe667f47640 1 --2- 172.21.15.98:0/1455385361 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6600b9970 0x7fe6600cca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:22.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.210+0000 7fe667f47640 1 --2- 172.21.15.98:0/1455385361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6600b7f80 0x7fe660140e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:22.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.210+0000 7fe667f47640 1 -- 172.21.15.98:0/1455385361 >> 172.21.15.98:0/1455385361 conn(0x7fe660074e40 msgr2=0x7fe66015f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:40:22.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.210+0000 7fe667f47640 1 -- 172.21.15.98:0/1455385361 shutdown_connections 2024-09-19T08:40:22.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:22.210+0000 7fe667f47640 1 -- 172.21.15.98:0/1455385361 wait complete. 2024-09-19T08:40:22.226 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-19T08:40:22.350 INFO:teuthology.orchestra.run.smithi098.stdout:[{"id":5490,"entity":{"name":{"type":"client","num":5490},"addr":{"type":"any","addr":"172.21.15.98:0","nonce":379128374}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":13,"uptime":11.963896783999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.400958051499437,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5490 172.21.15.98:0/379128374","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"y","hostname":"smithi098","pid":"28958","root":"/"}}] 2024-09-19T08:40:22.351 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi098", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28958", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.98:0", "nonce": 379128374, "type": "any" }, "name": { "num": 5490, "type": "client" } }, "id": 5490, "inst": "client.5490 172.21.15.98:0/379128374", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 13, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.400958051499437 }, "state": "open", "uptime": 11.963896784 } ] 2024-09-19T08:40:22.351 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-19T08:40:27.352 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:40:27.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.482+0000 7f731ddc9640 1 Processor -- start 2024-09-19T08:40:27.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.482+0000 7f731ddc9640 1 -- start start 2024-09-19T08:40:27.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.486+0000 7f731ddc9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f731815c550 0x7f731806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:27.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.486+0000 7f731ddc9640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f731806c1d0 0x7f73180cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:27.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.486+0000 7f731ddc9640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f731806d310 con 0x7f73180cf900 2024-09-19T08:40:27.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.486+0000 7f731ddc9640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f731806c610 con 0x7f731806c1d0 2024-09-19T08:40:27.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.486+0000 7f731ddc9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f731806d5f0 con 0x7f731815c550 2024-09-19T08:40:27.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.486+0000 7f730effd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f731806c1d0 0x7f73180cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:27.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.486+0000 7f7317fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f73180cf900 0x7f73180ce1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:60470/0 (socket says 172.21.15.98:60470) 2024-09-19T08:40:27.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.486+0000 7f7317fff640 1 -- 172.21.15.98:0/1866964928 learned_addr learned my addr 172.21.15.98:0/1866964928 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:40:27.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.486+0000 7f73177fe640 1 --2- 172.21.15.98:0/1866964928 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f731815c550 0x7f731806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:27.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.486+0000 7f730effd640 1 -- 172.21.15.98:0/1866964928 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f731815c550 msgr2=0x7f731806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:40:27.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.486+0000 7f730effd640 1 --2- 172.21.15.98:0/1866964928 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f731815c550 0x7f731806bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:27.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.486+0000 7f730effd640 1 -- 172.21.15.98:0/1866964928 >> v1:172.21.15.98:6789/0 conn(0x7f73180cf900 legacy=0x7f73180ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:27.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.486+0000 7f730effd640 1 -- 172.21.15.98:0/1866964928 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f731806d8d0 con 0x7f731806c1d0 2024-09-19T08:40:27.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.486+0000 7f7316ffd640 1 -- 172.21.15.98:0/1866964928 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2647059290 0 0) 0x7f731806d310 con 0x7f73180cf900 2024-09-19T08:40:27.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.486+0000 7f730effd640 1 --2- 172.21.15.98:0/1866964928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f731806c1d0 0x7f73180cf3c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f73000097c0 tx=0x7f730001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d1146d73e44c49 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:27.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.486+0000 7f7316ffd640 1 -- 172.21.15.98:0/1866964928 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73000b0070 con 0x7f731806c1d0 2024-09-19T08:40:27.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.486+0000 7f7316ffd640 1 -- 172.21.15.98:0/1866964928 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73000ab600 con 0x7f731806c1d0 2024-09-19T08:40:27.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.486+0000 7f7316ffd640 1 -- 172.21.15.98:0/1866964928 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73000ab9d0 con 0x7f731806c1d0 2024-09-19T08:40:27.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.490+0000 7f731ddc9640 1 -- 172.21.15.98:0/1866964928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f731806c1d0 msgr2=0x7f73180cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:27.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.490+0000 7f731ddc9640 1 --2- 172.21.15.98:0/1866964928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f731806c1d0 0x7f73180cf3c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f73000097c0 tx=0x7f730001c4a0 comp rx=0 tx=0).stop 2024-09-19T08:40:27.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.490+0000 7f731ddc9640 1 -- 172.21.15.98:0/1866964928 shutdown_connections 2024-09-19T08:40:27.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.490+0000 7f731ddc9640 1 --2- 172.21.15.98:0/1866964928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f731806c1d0 0x7f73180cf3c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:27.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.490+0000 7f731ddc9640 1 --2- 172.21.15.98:0/1866964928 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f731815c550 0x7f731806bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:27.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.490+0000 7f731ddc9640 1 -- 172.21.15.98:0/1866964928 >> 172.21.15.98:0/1866964928 conn(0x7f73180b2010 msgr2=0x7f73180b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:40:27.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.490+0000 7f731ddc9640 1 -- 172.21.15.98:0/1866964928 shutdown_connections 2024-09-19T08:40:27.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.490+0000 7f731ddc9640 1 -- 172.21.15.98:0/1866964928 wait complete. 2024-09-19T08:40:27.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.490+0000 7f731ddc9640 1 Processor -- start 2024-09-19T08:40:27.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.490+0000 7f731ddc9640 1 -- start start 2024-09-19T08:40:27.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.490+0000 7f731ddc9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f731806c1d0 0x7f73181526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:27.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.490+0000 7f731ddc9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73180cf900 0x7f731814a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:27.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.490+0000 7f731ddc9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f731815c550 0x7f731814a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:27.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.490+0000 7f731ddc9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7318169230 con 0x7f731815c550 2024-09-19T08:40:27.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.490+0000 7f731ddc9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f731815cb40 con 0x7f73180cf900 2024-09-19T08:40:27.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.490+0000 7f731ddc9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f73181693b0 con 0x7f731806c1d0 2024-09-19T08:40:27.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.490+0000 7f7317fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f731815c550 0x7f731814a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:27.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.490+0000 7f7317fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f731815c550 0x7f731814a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48422/0 (socket says 172.21.15.98:48422) 2024-09-19T08:40:27.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.490+0000 7f7317fff640 1 -- 172.21.15.98:0/890422058 learned_addr learned my addr 172.21.15.98:0/890422058 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:40:27.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.490+0000 7f730effd640 1 --2- 172.21.15.98:0/890422058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73180cf900 0x7f731814a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:27.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.490+0000 7f73177fe640 1 --2- 172.21.15.98:0/890422058 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f731806c1d0 0x7f73181526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:27.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.490+0000 7f7317fff640 1 -- 172.21.15.98:0/890422058 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f731806c1d0 msgr2=0x7f73181526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:27.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.490+0000 7f7317fff640 1 --2- 172.21.15.98:0/890422058 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f731806c1d0 0x7f73181526a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:27.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.490+0000 7f7317fff640 1 -- 172.21.15.98:0/890422058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73180cf900 msgr2=0x7f731814a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:27.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.490+0000 7f7317fff640 1 --2- 172.21.15.98:0/890422058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73180cf900 0x7f731814a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:27.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.490+0000 7f7317fff640 1 -- 172.21.15.98:0/890422058 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73180b27c0 con 0x7f731815c550 2024-09-19T08:40:27.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.490+0000 7f7317fff640 1 --2- 172.21.15.98:0/890422058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f731815c550 0x7f731814a830 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f731000c7a0 tx=0x7f731000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:27.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.490+0000 7f7314ff9640 1 -- 172.21.15.98:0/890422058 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7310018070 con 0x7f731815c550 2024-09-19T08:40:27.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.494+0000 7f7314ff9640 1 -- 172.21.15.98:0/890422058 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f731000a640 con 0x7f731815c550 2024-09-19T08:40:27.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.494+0000 7f731ddc9640 1 -- 172.21.15.98:0/890422058 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7318150c60 con 0x7f731815c550 2024-09-19T08:40:27.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.494+0000 7f7314ff9640 1 -- 172.21.15.98:0/890422058 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731000aab0 con 0x7f731815c550 2024-09-19T08:40:27.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.494+0000 7f731ddc9640 1 -- 172.21.15.98:0/890422058 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7318151220 con 0x7f731815c550 2024-09-19T08:40:27.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.494+0000 7f731ddc9640 1 -- 172.21.15.98:0/890422058 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73180cf3c0 con 0x7f731815c550 2024-09-19T08:40:27.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.494+0000 7f7314ff9640 1 -- 172.21.15.98:0/890422058 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7f731001b070 con 0x7f731815c550 2024-09-19T08:40:27.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.494+0000 7f7314ff9640 1 --2- 172.21.15.98:0/890422058 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f72f8067b00 0x7f72f8069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:27.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.494+0000 7f7314ff9640 1 -- 172.21.15.98:0/890422058 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f7310085640 con 0x7f731815c550 2024-09-19T08:40:27.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.494+0000 7f73177fe640 1 --2- 172.21.15.98:0/890422058 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f72f8067b00 0x7f72f8069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:27.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.494+0000 7f73177fe640 1 --2- 172.21.15.98:0/890422058 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f72f8067b00 0x7f72f8069fc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f73180cf690 tx=0x7f7308008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:27.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.498+0000 7f7314ff9640 1 -- 172.21.15.98:0/890422058 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f7310055e70 con 0x7f731815c550 2024-09-19T08:40:27.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.690+0000 7f731ddc9640 1 -- 172.21.15.98:0/890422058 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73180757a0 con 0x7f731815c550 2024-09-19T08:40:27.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.690+0000 7f7314ff9640 1 -- 172.21.15.98:0/890422058 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5229 (secure 0 0 0) 0x7f7310055770 con 0x7f731815c550 2024-09-19T08:40:27.692 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:40:27.693 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"btime":"2024-09-19T08:39:55:181532+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/2874086421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":2874086421},{"type":"v1","addr":"172.21.15.139:6833","nonce":2874086421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4996,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/4178563848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":4178563848},{"type":"v1","addr":"172.21.15.139:6835","nonce":4178563848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3584568343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3584568343},{"type":"v1","addr":"172.21.15.98:6835","nonce":3584568343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:39:38.262938+0000","modified":"2024-09-19T08:39:41.276985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/3085042202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3085042202},{"type":"v1","addr":"172.21.15.98:6837","nonce":3085042202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244]},"id":3}]} 2024-09-19T08:40:27.693 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 24 2024-09-19T08:40:27.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.694+0000 7f731ddc9640 1 -- 172.21.15.98:0/890422058 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f72f8067b00 msgr2=0x7f72f8069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:27.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.694+0000 7f731ddc9640 1 --2- 172.21.15.98:0/890422058 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f72f8067b00 0x7f72f8069fc0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f73180cf690 tx=0x7f7308008040 comp rx=0 tx=0).stop 2024-09-19T08:40:27.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.694+0000 7f731ddc9640 1 -- 172.21.15.98:0/890422058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f731815c550 msgr2=0x7f731814a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:27.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.694+0000 7f731ddc9640 1 --2- 172.21.15.98:0/890422058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f731815c550 0x7f731814a830 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f731000c7a0 tx=0x7f731000cc60 comp rx=0 tx=0).stop 2024-09-19T08:40:27.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.694+0000 7f731ddc9640 1 -- 172.21.15.98:0/890422058 shutdown_connections 2024-09-19T08:40:27.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.694+0000 7f731ddc9640 1 --2- 172.21.15.98:0/890422058 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f72f8067b00 0x7f72f8069fc0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:27.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.694+0000 7f731ddc9640 1 --2- 172.21.15.98:0/890422058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f731815c550 0x7f731814a830 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:27.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.694+0000 7f731ddc9640 1 --2- 172.21.15.98:0/890422058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73180cf900 0x7f731814a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:27.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.694+0000 7f731ddc9640 1 --2- 172.21.15.98:0/890422058 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f731806c1d0 0x7f73181526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:27.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.694+0000 7f731ddc9640 1 -- 172.21.15.98:0/890422058 >> 172.21.15.98:0/890422058 conn(0x7f73180b2010 msgr2=0x7f731814c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:40:27.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.694+0000 7f731ddc9640 1 -- 172.21.15.98:0/890422058 shutdown_connections 2024-09-19T08:40:27.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:27.694+0000 7f731ddc9640 1 -- 172.21.15.98:0/890422058 wait complete. 2024-09-19T08:40:27.710 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-19T08:40:27.849 INFO:teuthology.orchestra.run.smithi098.stdout:[{"id":5490,"entity":{"name":{"type":"client","num":5490},"addr":{"type":"any","addr":"172.21.15.98:0","nonce":379128374}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":12,"uptime":17.459849292000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.130919418871599,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5490 172.21.15.98:0/379128374","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"y","hostname":"smithi098","pid":"28958","root":"/"}}] 2024-09-19T08:40:27.850 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi098", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28958", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.98:0", "nonce": 379128374, "type": "any" }, "name": { "num": 5490, "type": "client" } }, "id": 5490, "inst": "client.5490 172.21.15.98:0/379128374", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 12, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.1309194188716 }, "state": "open", "uptime": 17.459849292 } ] 2024-09-19T08:40:27.850 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-19T08:40:32.851 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:40:32.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.982+0000 7f458e4e5640 1 Processor -- start 2024-09-19T08:40:32.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.982+0000 7f458e4e5640 1 -- start start 2024-09-19T08:40:32.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.982+0000 7f458e4e5640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f458815c550 0x7f458806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:32.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.982+0000 7f458e4e5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f458806c1d0 0x7f45880cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:32.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.986+0000 7f458e4e5640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f458806d310 con 0x7f45880cf900 2024-09-19T08:40:32.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.986+0000 7f458e4e5640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f458806c640 con 0x7f458815c550 2024-09-19T08:40:32.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.986+0000 7f458e4e5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f458806d5f0 con 0x7f458806c1d0 2024-09-19T08:40:32.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.986+0000 7f4587fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f458815c550 0x7f458806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:32.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.986+0000 7f458ca82640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f45880cf900 0x7f45880ce1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:54002/0 (socket says 172.21.15.98:54002) 2024-09-19T08:40:32.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.986+0000 7f458ca82640 1 -- 172.21.15.98:0/3827646149 learned_addr learned my addr 172.21.15.98:0/3827646149 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:40:32.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.986+0000 7f45877fe640 1 --2- 172.21.15.98:0/3827646149 >> v2:172.21.15.139:3300/0 conn(0x7f458806c1d0 0x7f45880cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:32.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.986+0000 7f4587fff640 1 -- 172.21.15.98:0/3827646149 >> v2:172.21.15.139:3300/0 conn(0x7f458806c1d0 msgr2=0x7f45880cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:40:32.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.986+0000 7f4587fff640 1 --2- 172.21.15.98:0/3827646149 >> v2:172.21.15.139:3300/0 conn(0x7f458806c1d0 0x7f45880cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:32.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.986+0000 7f4587fff640 1 -- 172.21.15.98:0/3827646149 >> v1:172.21.15.98:6789/0 conn(0x7f45880cf900 legacy=0x7f45880ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:32.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.986+0000 7f4587fff640 1 -- 172.21.15.98:0/3827646149 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458806d8d0 con 0x7f458815c550 2024-09-19T08:40:32.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.986+0000 7f4586ffd640 1 -- 172.21.15.98:0/3827646149 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1396850538 0 0) 0x7f458806d310 con 0x7f45880cf900 2024-09-19T08:40:32.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.986+0000 7f4587fff640 1 --2- 172.21.15.98:0/3827646149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f458815c550 0x7f458806bc90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f45880ccc60 tx=0x7f457801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86adba215479063a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:32.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.986+0000 7f4586ffd640 1 -- 172.21.15.98:0/3827646149 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45780b2070 con 0x7f458815c550 2024-09-19T08:40:32.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.986+0000 7f4586ffd640 1 -- 172.21.15.98:0/3827646149 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45780a57f0 con 0x7f458815c550 2024-09-19T08:40:32.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.986+0000 7f4586ffd640 1 -- 172.21.15.98:0/3827646149 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45780a5bc0 con 0x7f458815c550 2024-09-19T08:40:32.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.986+0000 7f458e4e5640 1 -- 172.21.15.98:0/3827646149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f458815c550 msgr2=0x7f458806bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:32.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.986+0000 7f458e4e5640 1 --2- 172.21.15.98:0/3827646149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f458815c550 0x7f458806bc90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f45880ccc60 tx=0x7f457801c930 comp rx=0 tx=0).stop 2024-09-19T08:40:32.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.986+0000 7f458e4e5640 1 -- 172.21.15.98:0/3827646149 shutdown_connections 2024-09-19T08:40:32.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.986+0000 7f458e4e5640 1 --2- 172.21.15.98:0/3827646149 >> v2:172.21.15.139:3300/0 conn(0x7f458806c1d0 0x7f45880cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:32.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.986+0000 7f458e4e5640 1 --2- 172.21.15.98:0/3827646149 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f458815c550 0x7f458806bc90 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:32.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.986+0000 7f458e4e5640 1 -- 172.21.15.98:0/3827646149 >> 172.21.15.98:0/3827646149 conn(0x7f45880b2010 msgr2=0x7f45880b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:40:32.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.986+0000 7f458e4e5640 1 -- 172.21.15.98:0/3827646149 shutdown_connections 2024-09-19T08:40:32.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.986+0000 7f458e4e5640 1 -- 172.21.15.98:0/3827646149 wait complete. 2024-09-19T08:40:32.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.990+0000 7f458e4e5640 1 Processor -- start 2024-09-19T08:40:32.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.990+0000 7f458e4e5640 1 -- start start 2024-09-19T08:40:32.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.990+0000 7f458e4e5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f458806c1d0 0x7f458814f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:32.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.990+0000 7f458e4e5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45880cf900 0x7f45880ac4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:32.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.990+0000 7f458e4e5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f458815c550 0x7f4588159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:32.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.990+0000 7f458e4e5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f45881693b0 con 0x7f45880cf900 2024-09-19T08:40:32.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.990+0000 7f458e4e5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4588169230 con 0x7f458806c1d0 2024-09-19T08:40:32.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.990+0000 7f458e4e5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4588169530 con 0x7f458815c550 2024-09-19T08:40:32.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.990+0000 7f45877fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45880cf900 0x7f45880ac4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:32.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.990+0000 7f45877fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45880cf900 0x7f45880ac4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33450/0 (socket says 172.21.15.98:33450) 2024-09-19T08:40:32.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.990+0000 7f45877fe640 1 -- 172.21.15.98:0/2855387310 learned_addr learned my addr 172.21.15.98:0/2855387310 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:40:32.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.990+0000 7f458ca82640 1 --2- 172.21.15.98:0/2855387310 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f458815c550 0x7f4588159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:32.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.990+0000 7f4587fff640 1 --2- 172.21.15.98:0/2855387310 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f458806c1d0 0x7f458814f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:32.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.990+0000 7f45877fe640 1 -- 172.21.15.98:0/2855387310 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f458815c550 msgr2=0x7f4588159e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:32.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.990+0000 7f45877fe640 1 --2- 172.21.15.98:0/2855387310 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f458815c550 0x7f4588159e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:32.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.990+0000 7f45877fe640 1 -- 172.21.15.98:0/2855387310 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f458806c1d0 msgr2=0x7f458814f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:32.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.990+0000 7f45877fe640 1 --2- 172.21.15.98:0/2855387310 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f458806c1d0 0x7f458814f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:32.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.990+0000 7f45877fe640 1 -- 172.21.15.98:0/2855387310 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458814fb60 con 0x7f45880cf900 2024-09-19T08:40:32.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.990+0000 7f45877fe640 1 --2- 172.21.15.98:0/2855387310 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45880cf900 0x7f45880ac4b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f457c00d620 tx=0x7f457c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:32.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.990+0000 7f4584ff9640 1 -- 172.21.15.98:0/2855387310 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c00b620 con 0x7f45880cf900 2024-09-19T08:40:32.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.990+0000 7f4584ff9640 1 -- 172.21.15.98:0/2855387310 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f457c019070 con 0x7f45880cf900 2024-09-19T08:40:32.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.990+0000 7f4584ff9640 1 -- 172.21.15.98:0/2855387310 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c00ba20 con 0x7f45880cf900 2024-09-19T08:40:32.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.990+0000 7f458e4e5640 1 -- 172.21.15.98:0/2855387310 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f458814fe40 con 0x7f45880cf900 2024-09-19T08:40:32.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.990+0000 7f458e4e5640 1 -- 172.21.15.98:0/2855387310 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4588150400 con 0x7f45880cf900 2024-09-19T08:40:32.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.990+0000 7f458e4e5640 1 -- 172.21.15.98:0/2855387310 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45880cf3c0 con 0x7f45880cf900 2024-09-19T08:40:32.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.994+0000 7f4584ff9640 1 -- 172.21.15.98:0/2855387310 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7f457c00bbc0 con 0x7f45880cf900 2024-09-19T08:40:32.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.994+0000 7f4584ff9640 1 --2- 172.21.15.98:0/2855387310 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4574067b00 0x7f4574069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:32.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.994+0000 7f4587fff640 1 --2- 172.21.15.98:0/2855387310 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4574067b00 0x7f4574069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:32.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.994+0000 7f4584ff9640 1 -- 172.21.15.98:0/2855387310 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f457c0863b0 con 0x7f45880cf900 2024-09-19T08:40:32.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.994+0000 7f4587fff640 1 --2- 172.21.15.98:0/2855387310 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4574067b00 0x7f4574069fc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f45780af000 tx=0x7f45780a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:32.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:32.998+0000 7f4584ff9640 1 -- 172.21.15.98:0/2855387310 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f457c056590 con 0x7f45880cf900 2024-09-19T08:40:33.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:33.194+0000 7f458e4e5640 1 -- 172.21.15.98:0/2855387310 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f458806bf40 con 0x7f45880cf900 2024-09-19T08:40:33.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:33.198+0000 7f4584ff9640 1 -- 172.21.15.98:0/2855387310 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5229 (secure 0 0 0) 0x7f457c05a890 con 0x7f45880cf900 2024-09-19T08:40:33.200 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:40:33.200 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"btime":"2024-09-19T08:39:55:181532+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/2874086421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":2874086421},{"type":"v1","addr":"172.21.15.139:6833","nonce":2874086421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4996,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/4178563848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":4178563848},{"type":"v1","addr":"172.21.15.139:6835","nonce":4178563848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3584568343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3584568343},{"type":"v1","addr":"172.21.15.98:6835","nonce":3584568343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:39:38.262938+0000","modified":"2024-09-19T08:39:41.276985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/3085042202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3085042202},{"type":"v1","addr":"172.21.15.98:6837","nonce":3085042202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244]},"id":3}]} 2024-09-19T08:40:33.201 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 24 2024-09-19T08:40:33.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:33.198+0000 7f458e4e5640 1 -- 172.21.15.98:0/2855387310 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4574067b00 msgr2=0x7f4574069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:33.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:33.198+0000 7f458e4e5640 1 --2- 172.21.15.98:0/2855387310 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4574067b00 0x7f4574069fc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f45780af000 tx=0x7f45780a7070 comp rx=0 tx=0).stop 2024-09-19T08:40:33.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:33.198+0000 7f458e4e5640 1 -- 172.21.15.98:0/2855387310 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45880cf900 msgr2=0x7f45880ac4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:33.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:33.198+0000 7f458e4e5640 1 --2- 172.21.15.98:0/2855387310 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45880cf900 0x7f45880ac4b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f457c00d620 tx=0x7f457c00dae0 comp rx=0 tx=0).stop 2024-09-19T08:40:33.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:33.198+0000 7f458e4e5640 1 -- 172.21.15.98:0/2855387310 shutdown_connections 2024-09-19T08:40:33.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:33.198+0000 7f458e4e5640 1 --2- 172.21.15.98:0/2855387310 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4574067b00 0x7f4574069fc0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:33.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:33.198+0000 7f458e4e5640 1 --2- 172.21.15.98:0/2855387310 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f458815c550 0x7f4588159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:33.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:33.198+0000 7f458e4e5640 1 --2- 172.21.15.98:0/2855387310 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45880cf900 0x7f45880ac4b0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:33.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:33.198+0000 7f458e4e5640 1 --2- 172.21.15.98:0/2855387310 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f458806c1d0 0x7f458814f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:33.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:33.198+0000 7f458e4e5640 1 -- 172.21.15.98:0/2855387310 >> 172.21.15.98:0/2855387310 conn(0x7f45880b2010 msgr2=0x7f45880aff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:40:33.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:33.198+0000 7f458e4e5640 1 -- 172.21.15.98:0/2855387310 shutdown_connections 2024-09-19T08:40:33.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:33.198+0000 7f458e4e5640 1 -- 172.21.15.98:0/2855387310 wait complete. 2024-09-19T08:40:33.216 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-19T08:40:33.358 INFO:teuthology.orchestra.run.smithi098.stdout:[{"id":5490,"entity":{"name":{"type":"client","num":5490},"addr":{"type":"any","addr":"172.21.15.98:0","nonce":379128374}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":11,"uptime":22.967801650999998,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.863709684969848,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5490 172.21.15.98:0/379128374","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"y","hostname":"smithi098","pid":"28958","root":"/"}}] 2024-09-19T08:40:33.359 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi098", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28958", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.98:0", "nonce": 379128374, "type": "any" }, "name": { "num": 5490, "type": "client" } }, "id": 5490, "inst": "client.5490 172.21.15.98:0/379128374", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 11, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.863709684969848 }, "state": "open", "uptime": 22.967801651 } ] 2024-09-19T08:40:33.359 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-19T08:40:38.360 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:40:38.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.494+0000 7f6ed9ff3640 1 Processor -- start 2024-09-19T08:40:38.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.494+0000 7f6ed9ff3640 1 -- start start 2024-09-19T08:40:38.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.494+0000 7f6ed9ff3640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6ed415c550 0x7f6ed406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:38.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.494+0000 7f6ed9ff3640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ed406c1d0 0x7f6ed40cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:38.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.494+0000 7f6ed9ff3640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6ed40cf900 0x7f6ed40ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:38.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.494+0000 7f6ed9ff3640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f6ed406c610 con 0x7f6ed415c550 2024-09-19T08:40:38.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.494+0000 7f6ed9ff3640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f6ed406d310 con 0x7f6ed40cf900 2024-09-19T08:40:38.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.494+0000 7f6ed9ff3640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6ed406d5f0 con 0x7f6ed406c1d0 2024-09-19T08:40:38.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.494+0000 7f6ed37fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6ed415c550 0x7f6ed406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:38.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.494+0000 7f6ed37fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6ed415c550 0x7f6ed406bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:52942/0 (socket says 172.21.15.98:52942) 2024-09-19T08:40:38.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.494+0000 7f6ed37fe640 1 -- 172.21.15.98:0/3804314954 learned_addr learned my addr 172.21.15.98:0/3804314954 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:40:38.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.494+0000 7f6ed3fff640 1 --2- 172.21.15.98:0/3804314954 >> v2:172.21.15.139:3300/0 conn(0x7f6ed40cf900 0x7f6ed40ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:38.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.494+0000 7f6ecaffd640 1 --2- 172.21.15.98:0/3804314954 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ed406c1d0 0x7f6ed40cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:38.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.494+0000 7f6ed37fe640 1 -- 172.21.15.98:0/3804314954 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ed406c1d0 msgr2=0x7f6ed40cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:40:38.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.494+0000 7f6ed37fe640 1 --2- 172.21.15.98:0/3804314954 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ed406c1d0 0x7f6ed40cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:38.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.494+0000 7f6ed37fe640 1 -- 172.21.15.98:0/3804314954 >> v2:172.21.15.139:3300/0 conn(0x7f6ed40cf900 msgr2=0x7f6ed40ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:40:38.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.494+0000 7f6ed37fe640 1 --2- 172.21.15.98:0/3804314954 >> v2:172.21.15.139:3300/0 conn(0x7f6ed40cf900 0x7f6ed40ce1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:38.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.494+0000 7f6ed37fe640 1 -- 172.21.15.98:0/3804314954 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed406d8d0 con 0x7f6ed415c550 2024-09-19T08:40:38.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.498+0000 7f6ed37fe640 1 --2- 172.21.15.98:0/3804314954 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ed415c550 0x7f6ed406bc90 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4074090 tx=0x7f6ec401c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e8a16497d526eb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:38.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.498+0000 7f6ed2ffd640 1 -- 172.21.15.98:0/3804314954 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec40b0070 con 0x7f6ed415c550 2024-09-19T08:40:38.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.498+0000 7f6ed2ffd640 1 -- 172.21.15.98:0/3804314954 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ec40ab600 con 0x7f6ed415c550 2024-09-19T08:40:38.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.498+0000 7f6ed2ffd640 1 -- 172.21.15.98:0/3804314954 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec40ab9d0 con 0x7f6ed415c550 2024-09-19T08:40:38.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.498+0000 7f6ed9ff3640 1 -- 172.21.15.98:0/3804314954 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ed415c550 msgr2=0x7f6ed406bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:38.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.498+0000 7f6ed9ff3640 1 --2- 172.21.15.98:0/3804314954 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ed415c550 0x7f6ed406bc90 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4074090 tx=0x7f6ec401c260 comp rx=0 tx=0).stop 2024-09-19T08:40:38.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.498+0000 7f6ed9ff3640 1 -- 172.21.15.98:0/3804314954 shutdown_connections 2024-09-19T08:40:38.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.498+0000 7f6ed9ff3640 1 --2- 172.21.15.98:0/3804314954 >> v2:172.21.15.139:3300/0 conn(0x7f6ed40cf900 0x7f6ed40ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:38.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.498+0000 7f6ed9ff3640 1 --2- 172.21.15.98:0/3804314954 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ed406c1d0 0x7f6ed40cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:38.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.498+0000 7f6ed9ff3640 1 --2- 172.21.15.98:0/3804314954 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ed415c550 0x7f6ed406bc90 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:38.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.498+0000 7f6ed9ff3640 1 -- 172.21.15.98:0/3804314954 >> 172.21.15.98:0/3804314954 conn(0x7f6ed40b2010 msgr2=0x7f6ed40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:40:38.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.498+0000 7f6ed9ff3640 1 -- 172.21.15.98:0/3804314954 shutdown_connections 2024-09-19T08:40:38.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.498+0000 7f6ed9ff3640 1 -- 172.21.15.98:0/3804314954 wait complete. 2024-09-19T08:40:38.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.498+0000 7f6ed9ff3640 1 Processor -- start 2024-09-19T08:40:38.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.498+0000 7f6ed9ff3640 1 -- start start 2024-09-19T08:40:38.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.498+0000 7f6ed9ff3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ed406c1d0 0x7f6ed40c7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:38.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.498+0000 7f6ed9ff3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ed40cf900 0x7f6ed4146f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:38.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.498+0000 7f6ed9ff3640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ed415c550 0x7f6ed40ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:38.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.498+0000 7f6ed9ff3640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed40723a0 con 0x7f6ed40cf900 2024-09-19T08:40:38.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.498+0000 7f6ed9ff3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed415cb40 con 0x7f6ed406c1d0 2024-09-19T08:40:38.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.498+0000 7f6ed9ff3640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6ed4072520 con 0x7f6ed415c550 2024-09-19T08:40:38.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.498+0000 7f6ecaffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ed40cf900 0x7f6ed4146f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:38.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.498+0000 7f6ecaffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ed40cf900 0x7f6ed4146f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:52952/0 (socket says 172.21.15.98:52952) 2024-09-19T08:40:38.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.498+0000 7f6ecaffd640 1 -- 172.21.15.98:0/556876346 learned_addr learned my addr 172.21.15.98:0/556876346 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:40:38.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.498+0000 7f6ed37fe640 1 --2- 172.21.15.98:0/556876346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ed406c1d0 0x7f6ed40c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:38.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.498+0000 7f6ed3fff640 1 --2- 172.21.15.98:0/556876346 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ed415c550 0x7f6ed40ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:38.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.498+0000 7f6ecaffd640 1 -- 172.21.15.98:0/556876346 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ed415c550 msgr2=0x7f6ed40ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:38.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.498+0000 7f6ecaffd640 1 --2- 172.21.15.98:0/556876346 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ed415c550 0x7f6ed40ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:38.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.498+0000 7f6ecaffd640 1 -- 172.21.15.98:0/556876346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ed406c1d0 msgr2=0x7f6ed40c7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:38.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.498+0000 7f6ecaffd640 1 --2- 172.21.15.98:0/556876346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ed406c1d0 0x7f6ed40c7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:38.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.498+0000 7f6ecaffd640 1 -- 172.21.15.98:0/556876346 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed40c78a0 con 0x7f6ed40cf900 2024-09-19T08:40:38.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.498+0000 7f6ed3fff640 1 --2- 172.21.15.98:0/556876346 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ed415c550 0x7f6ed40ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:40:38.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.498+0000 7f6ecaffd640 1 --2- 172.21.15.98:0/556876346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ed40cf900 0x7f6ed4146f40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc00d800 tx=0x7f6ebc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:38.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.502+0000 7f6ed0ff9640 1 -- 172.21.15.98:0/556876346 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc018020 con 0x7f6ed40cf900 2024-09-19T08:40:38.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.502+0000 7f6ed0ff9640 1 -- 172.21.15.98:0/556876346 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ebc0096a0 con 0x7f6ed40cf900 2024-09-19T08:40:38.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.502+0000 7f6ed9ff3640 1 -- 172.21.15.98:0/556876346 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed4147480 con 0x7f6ed40cf900 2024-09-19T08:40:38.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.502+0000 7f6ed0ff9640 1 -- 172.21.15.98:0/556876346 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc009a50 con 0x7f6ed40cf900 2024-09-19T08:40:38.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.502+0000 7f6ed9ff3640 1 -- 172.21.15.98:0/556876346 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed40b3bf0 con 0x7f6ed40cf900 2024-09-19T08:40:38.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.502+0000 7f6ed9ff3640 1 -- 172.21.15.98:0/556876346 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ed40ce1a0 con 0x7f6ed40cf900 2024-09-19T08:40:38.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.502+0000 7f6ed0ff9640 1 -- 172.21.15.98:0/556876346 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7f6ebc01b020 con 0x7f6ed40cf900 2024-09-19T08:40:38.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.502+0000 7f6ed0ff9640 1 --2- 172.21.15.98:0/556876346 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6eb4067b00 0x7f6eb4069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:38.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.502+0000 7f6ed37fe640 1 --2- 172.21.15.98:0/556876346 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6eb4067b00 0x7f6eb4069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:38.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.502+0000 7f6ed0ff9640 1 -- 172.21.15.98:0/556876346 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f6ebc0865e0 con 0x7f6ed40cf900 2024-09-19T08:40:38.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.502+0000 7f6ed37fe640 1 --2- 172.21.15.98:0/556876346 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6eb4067b00 0x7f6eb4069fc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40ad000 tx=0x7f6ec4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:38.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.506+0000 7f6ed0ff9640 1 -- 172.21.15.98:0/556876346 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6ebc022020 con 0x7f6ed40cf900 2024-09-19T08:40:38.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.702+0000 7f6ed9ff3640 1 -- 172.21.15.98:0/556876346 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ed40b1940 con 0x7f6ed40cf900 2024-09-19T08:40:38.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.702+0000 7f6ed0ff9640 1 -- 172.21.15.98:0/556876346 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5229 (secure 0 0 0) 0x7f6ebc056e10 con 0x7f6ed40cf900 2024-09-19T08:40:38.706 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:40:38.706 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"btime":"2024-09-19T08:39:55:181532+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/2874086421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":2874086421},{"type":"v1","addr":"172.21.15.139:6833","nonce":2874086421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4996,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/4178563848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":4178563848},{"type":"v1","addr":"172.21.15.139:6835","nonce":4178563848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3584568343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3584568343},{"type":"v1","addr":"172.21.15.98:6835","nonce":3584568343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:39:38.262938+0000","modified":"2024-09-19T08:39:41.276985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/3085042202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3085042202},{"type":"v1","addr":"172.21.15.98:6837","nonce":3085042202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244]},"id":3}]} 2024-09-19T08:40:38.707 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 24 2024-09-19T08:40:38.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.706+0000 7f6ed9ff3640 1 -- 172.21.15.98:0/556876346 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6eb4067b00 msgr2=0x7f6eb4069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:38.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.706+0000 7f6ed9ff3640 1 --2- 172.21.15.98:0/556876346 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6eb4067b00 0x7f6eb4069fc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40ad000 tx=0x7f6ec4008040 comp rx=0 tx=0).stop 2024-09-19T08:40:38.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.706+0000 7f6ed9ff3640 1 -- 172.21.15.98:0/556876346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ed40cf900 msgr2=0x7f6ed4146f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:38.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.706+0000 7f6ed9ff3640 1 --2- 172.21.15.98:0/556876346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ed40cf900 0x7f6ed4146f40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc00d800 tx=0x7f6ebc00dcc0 comp rx=0 tx=0).stop 2024-09-19T08:40:38.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.706+0000 7f6ed9ff3640 1 -- 172.21.15.98:0/556876346 shutdown_connections 2024-09-19T08:40:38.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.706+0000 7f6ed9ff3640 1 --2- 172.21.15.98:0/556876346 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6eb4067b00 0x7f6eb4069fc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:38.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.706+0000 7f6ed9ff3640 1 --2- 172.21.15.98:0/556876346 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ed415c550 0x7f6ed40ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:38.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.706+0000 7f6ed9ff3640 1 --2- 172.21.15.98:0/556876346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ed40cf900 0x7f6ed4146f40 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:38.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.706+0000 7f6ed9ff3640 1 --2- 172.21.15.98:0/556876346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ed406c1d0 0x7f6ed40c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:38.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.706+0000 7f6ed9ff3640 1 -- 172.21.15.98:0/556876346 >> 172.21.15.98:0/556876346 conn(0x7f6ed40b2010 msgr2=0x7f6ed4067a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:40:38.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.706+0000 7f6ed9ff3640 1 -- 172.21.15.98:0/556876346 shutdown_connections 2024-09-19T08:40:38.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:38.706+0000 7f6ed9ff3640 1 -- 172.21.15.98:0/556876346 wait complete. 2024-09-19T08:40:38.723 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-19T08:40:38.861 INFO:teuthology.orchestra.run.smithi098.stdout:[{"id":5490,"entity":{"name":{"type":"client","num":5490},"addr":{"type":"any","addr":"172.21.15.98:0","nonce":379128374}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":10,"uptime":28.471754007000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.600069319552116,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5490 172.21.15.98:0/379128374","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"y","hostname":"smithi098","pid":"28958","root":"/"}}] 2024-09-19T08:40:38.861 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi098", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28958", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.98:0", "nonce": 379128374, "type": "any" }, "name": { "num": 5490, "type": "client" } }, "id": 5490, "inst": "client.5490 172.21.15.98:0/379128374", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 10, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.600069319552116 }, "state": "open", "uptime": 28.471754007 } ] 2024-09-19T08:40:38.862 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-19T08:40:43.863 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:40:43.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:43.994+0000 7fa572478640 1 Processor -- start 2024-09-19T08:40:43.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:43.994+0000 7fa572478640 1 -- start start 2024-09-19T08:40:43.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:43.998+0000 7fa572478640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa56c158fd0 0x7fa56c0b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:43.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:43.998+0000 7fa572478640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa56c06c3a0 0x7fa56c0ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:43.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:43.998+0000 7fa572478640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa56c0ccc60 con 0x7fa56c0b4360 2024-09-19T08:40:43.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:43.998+0000 7fa572478640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fa56c1593b0 con 0x7fa56c06c3a0 2024-09-19T08:40:43.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:43.998+0000 7fa572478640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fa56c0ccf40 con 0x7fa56c158fd0 2024-09-19T08:40:43.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:43.998+0000 7fa570a15640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa56c06c3a0 0x7fa56c0ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:43.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:43.998+0000 7fa570a15640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa56c06c3a0 0x7fa56c0ce840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:52962/0 (socket says 172.21.15.98:52962) 2024-09-19T08:40:43.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:43.998+0000 7fa570a15640 1 -- 172.21.15.98:0/428248185 learned_addr learned my addr 172.21.15.98:0/428248185 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:40:43.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:43.998+0000 7fa56bfff640 1 --2- 172.21.15.98:0/428248185 >> v2:172.21.15.139:3300/0 conn(0x7fa56c158fd0 0x7fa56c0b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:43.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:43.998+0000 7fa570a15640 1 -- 172.21.15.98:0/428248185 >> v2:172.21.15.139:3300/0 conn(0x7fa56c158fd0 msgr2=0x7fa56c0b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:40:43.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:43.998+0000 7fa570a15640 1 --2- 172.21.15.98:0/428248185 >> v2:172.21.15.139:3300/0 conn(0x7fa56c158fd0 0x7fa56c0b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:43.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:43.998+0000 7fa570a15640 1 -- 172.21.15.98:0/428248185 >> v1:172.21.15.139:6789/0 conn(0x7fa56c0b4360 legacy=0x7fa56c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:40:43.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:43.998+0000 7fa570a15640 1 -- 172.21.15.98:0/428248185 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56c0cd220 con 0x7fa56c06c3a0 2024-09-19T08:40:44.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:43.998+0000 7fa570a15640 1 --2- 172.21.15.98:0/428248185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa56c06c3a0 0x7fa56c0ce840 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa56c06e310 tx=0x7fa56401c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84eaca5c91a485a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:44.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:43.998+0000 7fa56affd640 1 -- 172.21.15.98:0/428248185 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5640a57b0 con 0x7fa56c06c3a0 2024-09-19T08:40:44.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:43.998+0000 7fa56affd640 1 -- 172.21.15.98:0/428248185 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5640a5950 con 0x7fa56c06c3a0 2024-09-19T08:40:44.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:43.998+0000 7fa56affd640 1 -- 172.21.15.98:0/428248185 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5640a5d20 con 0x7fa56c06c3a0 2024-09-19T08:40:44.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:43.998+0000 7fa572478640 1 -- 172.21.15.98:0/428248185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa56c06c3a0 msgr2=0x7fa56c0ce840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:44.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:43.998+0000 7fa572478640 1 --2- 172.21.15.98:0/428248185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa56c06c3a0 0x7fa56c0ce840 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa56c06e310 tx=0x7fa56401c8f0 comp rx=0 tx=0).stop 2024-09-19T08:40:44.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.002+0000 7fa572478640 1 -- 172.21.15.98:0/428248185 shutdown_connections 2024-09-19T08:40:44.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.002+0000 7fa572478640 1 --2- 172.21.15.98:0/428248185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa56c06c3a0 0x7fa56c0ce840 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:44.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.002+0000 7fa572478640 1 --2- 172.21.15.98:0/428248185 >> v2:172.21.15.139:3300/0 conn(0x7fa56c158fd0 0x7fa56c0b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:44.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.002+0000 7fa572478640 1 -- 172.21.15.98:0/428248185 >> 172.21.15.98:0/428248185 conn(0x7fa56c0b2010 msgr2=0x7fa56c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:40:44.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.002+0000 7fa572478640 1 -- 172.21.15.98:0/428248185 shutdown_connections 2024-09-19T08:40:44.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.002+0000 7fa572478640 1 -- 172.21.15.98:0/428248185 wait complete. 2024-09-19T08:40:44.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.002+0000 7fa572478640 1 Processor -- start 2024-09-19T08:40:44.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.002+0000 7fa572478640 1 -- start start 2024-09-19T08:40:44.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.002+0000 7fa572478640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa56c06c3a0 0x7fa56c14cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:44.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.002+0000 7fa572478640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa56c0b4360 0x7fa56c14c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:44.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.002+0000 7fa572478640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa56c158fd0 0x7fa56c158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:44.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.002+0000 7fa572478640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa56c0cf710 con 0x7fa56c158fd0 2024-09-19T08:40:44.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.002+0000 7fa572478640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa56c0cf590 con 0x7fa56c06c3a0 2024-09-19T08:40:44.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.002+0000 7fa572478640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa56c0cf890 con 0x7fa56c0b4360 2024-09-19T08:40:44.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.002+0000 7fa570a15640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa56c158fd0 0x7fa56c158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:44.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.002+0000 7fa570a15640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa56c158fd0 0x7fa56c158360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:52968/0 (socket says 172.21.15.98:52968) 2024-09-19T08:40:44.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.002+0000 7fa570a15640 1 -- 172.21.15.98:0/1014220812 learned_addr learned my addr 172.21.15.98:0/1014220812 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:40:44.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.002+0000 7fa56bfff640 1 --2- 172.21.15.98:0/1014220812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa56c06c3a0 0x7fa56c14cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:44.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.002+0000 7fa56b7fe640 1 --2- 172.21.15.98:0/1014220812 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa56c0b4360 0x7fa56c14c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:44.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.002+0000 7fa570a15640 1 -- 172.21.15.98:0/1014220812 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa56c0b4360 msgr2=0x7fa56c14c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:44.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.002+0000 7fa570a15640 1 --2- 172.21.15.98:0/1014220812 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa56c0b4360 0x7fa56c14c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:44.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.002+0000 7fa570a15640 1 -- 172.21.15.98:0/1014220812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa56c06c3a0 msgr2=0x7fa56c14cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:44.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.002+0000 7fa570a15640 1 --2- 172.21.15.98:0/1014220812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa56c06c3a0 0x7fa56c14cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:44.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.002+0000 7fa570a15640 1 -- 172.21.15.98:0/1014220812 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56c0b09a0 con 0x7fa56c158fd0 2024-09-19T08:40:44.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.002+0000 7fa56b7fe640 1 --2- 172.21.15.98:0/1014220812 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa56c0b4360 0x7fa56c14c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:40:44.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.002+0000 7fa570a15640 1 --2- 172.21.15.98:0/1014220812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa56c158fd0 0x7fa56c158360 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa5640af040 tx=0x7fa564019690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:44.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.002+0000 7fa568ff9640 1 -- 172.21.15.98:0/1014220812 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5640bc020 con 0x7fa56c158fd0 2024-09-19T08:40:44.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.002+0000 7fa568ff9640 1 -- 172.21.15.98:0/1014220812 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5640b8670 con 0x7fa56c158fd0 2024-09-19T08:40:44.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.002+0000 7fa572478640 1 -- 172.21.15.98:0/1014220812 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa56c0b0c80 con 0x7fa56c158fd0 2024-09-19T08:40:44.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.002+0000 7fa568ff9640 1 -- 172.21.15.98:0/1014220812 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5640b8a60 con 0x7fa56c158fd0 2024-09-19T08:40:44.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.002+0000 7fa572478640 1 -- 172.21.15.98:0/1014220812 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa56c1fa8d0 con 0x7fa56c158fd0 2024-09-19T08:40:44.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.002+0000 7fa572478640 1 -- 172.21.15.98:0/1014220812 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa56c1faaa0 con 0x7fa56c158fd0 2024-09-19T08:40:44.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.006+0000 7fa568ff9640 1 -- 172.21.15.98:0/1014220812 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7fa5640bf020 con 0x7fa56c158fd0 2024-09-19T08:40:44.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.006+0000 7fa568ff9640 1 --2- 172.21.15.98:0/1014220812 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa54c067b00 0x7fa54c069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:44.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.006+0000 7fa568ff9640 1 -- 172.21.15.98:0/1014220812 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7fa5641211b0 con 0x7fa56c158fd0 2024-09-19T08:40:44.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.006+0000 7fa56bfff640 1 --2- 172.21.15.98:0/1014220812 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa54c067b00 0x7fa54c069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:44.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.006+0000 7fa56bfff640 1 --2- 172.21.15.98:0/1014220812 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa54c067b00 0x7fa54c069fc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa56c0cc740 tx=0x7fa55c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:44.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.010+0000 7fa568ff9640 1 -- 172.21.15.98:0/1014220812 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa5640c6030 con 0x7fa56c158fd0 2024-09-19T08:40:44.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.206+0000 7fa572478640 1 -- 172.21.15.98:0/1014220812 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa56c15bd60 con 0x7fa56c158fd0 2024-09-19T08:40:44.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.206+0000 7fa568ff9640 1 -- 172.21.15.98:0/1014220812 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5229 (secure 0 0 0) 0x7fa5640165a0 con 0x7fa56c158fd0 2024-09-19T08:40:44.208 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:40:44.208 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"btime":"2024-09-19T08:39:55:181532+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/2874086421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":2874086421},{"type":"v1","addr":"172.21.15.139:6833","nonce":2874086421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4996,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/4178563848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":4178563848},{"type":"v1","addr":"172.21.15.139:6835","nonce":4178563848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3584568343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3584568343},{"type":"v1","addr":"172.21.15.98:6835","nonce":3584568343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:39:38.262938+0000","modified":"2024-09-19T08:39:41.276985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/3085042202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3085042202},{"type":"v1","addr":"172.21.15.98:6837","nonce":3085042202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244]},"id":3}]} 2024-09-19T08:40:44.208 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 24 2024-09-19T08:40:44.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.210+0000 7fa572478640 1 -- 172.21.15.98:0/1014220812 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa54c067b00 msgr2=0x7fa54c069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:44.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.210+0000 7fa572478640 1 --2- 172.21.15.98:0/1014220812 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa54c067b00 0x7fa54c069fc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa56c0cc740 tx=0x7fa55c008040 comp rx=0 tx=0).stop 2024-09-19T08:40:44.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.210+0000 7fa572478640 1 -- 172.21.15.98:0/1014220812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa56c158fd0 msgr2=0x7fa56c158360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:44.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.210+0000 7fa572478640 1 --2- 172.21.15.98:0/1014220812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa56c158fd0 0x7fa56c158360 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa5640af040 tx=0x7fa564019690 comp rx=0 tx=0).stop 2024-09-19T08:40:44.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.210+0000 7fa572478640 1 -- 172.21.15.98:0/1014220812 shutdown_connections 2024-09-19T08:40:44.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.210+0000 7fa572478640 1 --2- 172.21.15.98:0/1014220812 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa54c067b00 0x7fa54c069fc0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:44.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.210+0000 7fa572478640 1 --2- 172.21.15.98:0/1014220812 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa56c158fd0 0x7fa56c158360 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:44.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.210+0000 7fa572478640 1 --2- 172.21.15.98:0/1014220812 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa56c0b4360 0x7fa56c14c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:44.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.210+0000 7fa572478640 1 --2- 172.21.15.98:0/1014220812 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa56c06c3a0 0x7fa56c14cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:44.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.210+0000 7fa572478640 1 -- 172.21.15.98:0/1014220812 >> 172.21.15.98:0/1014220812 conn(0x7fa56c0b2010 msgr2=0x7fa56c06bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:40:44.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.210+0000 7fa572478640 1 -- 172.21.15.98:0/1014220812 shutdown_connections 2024-09-19T08:40:44.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:44.210+0000 7fa572478640 1 -- 172.21.15.98:0/1014220812 wait complete. 2024-09-19T08:40:44.227 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-19T08:40:44.358 INFO:teuthology.orchestra.run.smithi098.stdout:[{"id":5490,"entity":{"name":{"type":"client","num":5490},"addr":{"type":"any","addr":"172.21.15.98:0","nonce":379128374}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":10,"uptime":33.967706380000003,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.340136357582491,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5490 172.21.15.98:0/379128374","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"y","hostname":"smithi098","pid":"28958","root":"/"}}] 2024-09-19T08:40:44.359 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi098", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28958", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.98:0", "nonce": 379128374, "type": "any" }, "name": { "num": 5490, "type": "client" } }, "id": 5490, "inst": "client.5490 172.21.15.98:0/379128374", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 10, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.34013635758249 }, "state": "open", "uptime": 33.96770638 } ] 2024-09-19T08:40:44.359 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-19T08:40:47.425 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:40:47.428 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:40:49.360 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:40:49.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.498+0000 7f5276b27640 1 Processor -- start 2024-09-19T08:40:49.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.498+0000 7f5276b27640 1 -- start start 2024-09-19T08:40:49.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.498+0000 7f5276b27640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5270158e40 0x7f527006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:49.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.498+0000 7f5276b27640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f52700ce8b0 0x7f52700b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:49.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.498+0000 7f5276b27640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f527015fcf0 con 0x7f527006c1d0 2024-09-19T08:40:49.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.498+0000 7f5276b27640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f527006c640 con 0x7f5270158e40 2024-09-19T08:40:49.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.498+0000 7f5276b27640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f527015ef30 con 0x7f52700ce8b0 2024-09-19T08:40:49.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.502+0000 7f52748c3640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5270158e40 0x7f527006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:49.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.502+0000 7f526ffff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f527006c1d0 0x7f52700ce1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:59906/0 (socket says 172.21.15.98:59906) 2024-09-19T08:40:49.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.502+0000 7f526ffff640 1 -- 172.21.15.98:0/3893928166 learned_addr learned my addr 172.21.15.98:0/3893928166 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:40:49.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.502+0000 7f52748c3640 1 -- 172.21.15.98:0/3893928166 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f52700ce8b0 msgr2=0x7f52700b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:40:49.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.502+0000 7f52748c3640 1 --2- 172.21.15.98:0/3893928166 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f52700ce8b0 0x7f52700b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:49.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.502+0000 7f52748c3640 1 -- 172.21.15.98:0/3893928166 >> v1:172.21.15.98:6789/0 conn(0x7f527006c1d0 legacy=0x7f52700ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:49.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.502+0000 7f52748c3640 1 -- 172.21.15.98:0/3893928166 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f527015f210 con 0x7f5270158e40 2024-09-19T08:40:49.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.502+0000 7f526f7fe640 1 -- 172.21.15.98:0/3893928166 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2142751987 0 0) 0x7f527015fcf0 con 0x7f527006c1d0 2024-09-19T08:40:49.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.502+0000 7f52748c3640 1 --2- 172.21.15.98:0/3893928166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5270158e40 0x7f527006bc90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f52600097c0 tx=0x7f526001ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c507044d8b3e417e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:49.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.502+0000 7f526f7fe640 1 -- 172.21.15.98:0/3893928166 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52600b5070 con 0x7f5270158e40 2024-09-19T08:40:49.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.502+0000 7f526f7fe640 1 -- 172.21.15.98:0/3893928166 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52600a8990 con 0x7f5270158e40 2024-09-19T08:40:49.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.502+0000 7f526f7fe640 1 -- 172.21.15.98:0/3893928166 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52600a8d60 con 0x7f5270158e40 2024-09-19T08:40:49.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.502+0000 7f5276b27640 1 -- 172.21.15.98:0/3893928166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5270158e40 msgr2=0x7f527006bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:49.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.502+0000 7f5276b27640 1 --2- 172.21.15.98:0/3893928166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5270158e40 0x7f527006bc90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f52600097c0 tx=0x7f526001ec90 comp rx=0 tx=0).stop 2024-09-19T08:40:49.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.502+0000 7f5276b27640 1 -- 172.21.15.98:0/3893928166 shutdown_connections 2024-09-19T08:40:49.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.502+0000 7f5276b27640 1 --2- 172.21.15.98:0/3893928166 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f52700ce8b0 0x7f52700b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:49.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.502+0000 7f5276b27640 1 --2- 172.21.15.98:0/3893928166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5270158e40 0x7f527006bc90 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:49.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.502+0000 7f5276b27640 1 -- 172.21.15.98:0/3893928166 >> 172.21.15.98:0/3893928166 conn(0x7f52700b2010 msgr2=0x7f52700b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:40:49.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.502+0000 7f5276b27640 1 -- 172.21.15.98:0/3893928166 shutdown_connections 2024-09-19T08:40:49.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.502+0000 7f5276b27640 1 -- 172.21.15.98:0/3893928166 wait complete. 2024-09-19T08:40:49.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.506+0000 7f5276b27640 1 Processor -- start 2024-09-19T08:40:49.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.506+0000 7f5276b27640 1 -- start start 2024-09-19T08:40:49.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.506+0000 7f5276b27640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f527006c1d0 0x7f5270148c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:49.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.506+0000 7f5276b27640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f52700ce8b0 0x7f527015d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:49.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.506+0000 7f5276b27640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5270158e40 0x7f527015d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:49.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.506+0000 7f5276b27640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5270159220 con 0x7f527006c1d0 2024-09-19T08:40:49.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.506+0000 7f5276b27640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f527015f7d0 con 0x7f52700ce8b0 2024-09-19T08:40:49.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.506+0000 7f5276b27640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f52700cee20 con 0x7f5270158e40 2024-09-19T08:40:49.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.506+0000 7f52748c3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f527006c1d0 0x7f5270148c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:49.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.506+0000 7f52748c3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f527006c1d0 0x7f5270148c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:43122/0 (socket says 172.21.15.98:43122) 2024-09-19T08:40:49.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.506+0000 7f52748c3640 1 -- 172.21.15.98:0/3118841058 learned_addr learned my addr 172.21.15.98:0/3118841058 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:40:49.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.506+0000 7f52750c4640 1 --2- 172.21.15.98:0/3118841058 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5270158e40 0x7f527015d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:49.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.506+0000 7f52748c3640 1 -- 172.21.15.98:0/3118841058 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5270158e40 msgr2=0x7f527015d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:49.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.506+0000 7f52748c3640 1 --2- 172.21.15.98:0/3118841058 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5270158e40 0x7f527015d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:49.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.506+0000 7f526ffff640 1 --2- 172.21.15.98:0/3118841058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f52700ce8b0 0x7f527015d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:49.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.506+0000 7f52748c3640 1 -- 172.21.15.98:0/3118841058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f52700ce8b0 msgr2=0x7f527015d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:49.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.506+0000 7f52748c3640 1 --2- 172.21.15.98:0/3118841058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f52700ce8b0 0x7f527015d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:49.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.506+0000 7f52748c3640 1 -- 172.21.15.98:0/3118841058 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f527014d660 con 0x7f527006c1d0 2024-09-19T08:40:49.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.506+0000 7f52750c4640 1 --2- 172.21.15.98:0/3118841058 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5270158e40 0x7f527015d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:40:49.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.506+0000 7f526ffff640 1 --2- 172.21.15.98:0/3118841058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f52700ce8b0 0x7f527015d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:40:49.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.506+0000 7f52748c3640 1 --2- 172.21.15.98:0/3118841058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f527006c1d0 0x7f5270148c10 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f52600b3040 tx=0x7f526001d4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:49.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.506+0000 7f526d7fa640 1 -- 172.21.15.98:0/3118841058 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52600c1070 con 0x7f527006c1d0 2024-09-19T08:40:49.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.506+0000 7f526d7fa640 1 -- 172.21.15.98:0/3118841058 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52600b6040 con 0x7f527006c1d0 2024-09-19T08:40:49.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.506+0000 7f5276b27640 1 -- 172.21.15.98:0/3118841058 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f527014d940 con 0x7f527006c1d0 2024-09-19T08:40:49.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.506+0000 7f526d7fa640 1 -- 172.21.15.98:0/3118841058 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52600b5050 con 0x7f527006c1d0 2024-09-19T08:40:49.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.506+0000 7f5276b27640 1 -- 172.21.15.98:0/3118841058 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5270159430 con 0x7f527006c1d0 2024-09-19T08:40:49.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.506+0000 7f5276b27640 1 -- 172.21.15.98:0/3118841058 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f527006bc90 con 0x7f527006c1d0 2024-09-19T08:40:49.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.510+0000 7f526d7fa640 1 -- 172.21.15.98:0/3118841058 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7f52600c4070 con 0x7f527006c1d0 2024-09-19T08:40:49.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.510+0000 7f526d7fa640 1 --2- 172.21.15.98:0/3118841058 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f525c067b00 0x7f525c069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:49.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.510+0000 7f526d7fa640 1 -- 172.21.15.98:0/3118841058 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f5260123570 con 0x7f527006c1d0 2024-09-19T08:40:49.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.510+0000 7f526ffff640 1 --2- 172.21.15.98:0/3118841058 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f525c067b00 0x7f525c069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:49.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.510+0000 7f526ffff640 1 --2- 172.21.15.98:0/3118841058 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f525c067b00 0x7f525c069fc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5264007fd0 tx=0x7f5264009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:49.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.514+0000 7f526d7fa640 1 -- 172.21.15.98:0/3118841058 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f52600187f0 con 0x7f527006c1d0 2024-09-19T08:40:49.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.706+0000 7f5276b27640 1 -- 172.21.15.98:0/3118841058 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52700b1d80 con 0x7f527006c1d0 2024-09-19T08:40:49.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.706+0000 7f526d7fa640 1 -- 172.21.15.98:0/3118841058 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5229 (secure 0 0 0) 0x7f52600f3da0 con 0x7f527006c1d0 2024-09-19T08:40:49.708 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:40:49.708 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"btime":"2024-09-19T08:39:55:181532+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/2874086421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":2874086421},{"type":"v1","addr":"172.21.15.139:6833","nonce":2874086421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4996,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/4178563848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":4178563848},{"type":"v1","addr":"172.21.15.139:6835","nonce":4178563848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3584568343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3584568343},{"type":"v1","addr":"172.21.15.98:6835","nonce":3584568343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:39:38.262938+0000","modified":"2024-09-19T08:39:41.276985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/3085042202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3085042202},{"type":"v1","addr":"172.21.15.98:6837","nonce":3085042202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244]},"id":3}]} 2024-09-19T08:40:49.708 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 24 2024-09-19T08:40:49.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.710+0000 7f5276b27640 1 -- 172.21.15.98:0/3118841058 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f525c067b00 msgr2=0x7f525c069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:49.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.710+0000 7f5276b27640 1 --2- 172.21.15.98:0/3118841058 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f525c067b00 0x7f525c069fc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5264007fd0 tx=0x7f5264009040 comp rx=0 tx=0).stop 2024-09-19T08:40:49.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.710+0000 7f5276b27640 1 -- 172.21.15.98:0/3118841058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f527006c1d0 msgr2=0x7f5270148c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:49.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.710+0000 7f5276b27640 1 --2- 172.21.15.98:0/3118841058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f527006c1d0 0x7f5270148c10 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f52600b3040 tx=0x7f526001d4d0 comp rx=0 tx=0).stop 2024-09-19T08:40:49.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.710+0000 7f5276b27640 1 -- 172.21.15.98:0/3118841058 shutdown_connections 2024-09-19T08:40:49.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.710+0000 7f5276b27640 1 --2- 172.21.15.98:0/3118841058 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f525c067b00 0x7f525c069fc0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:49.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.710+0000 7f5276b27640 1 --2- 172.21.15.98:0/3118841058 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5270158e40 0x7f527015d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:49.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.710+0000 7f5276b27640 1 --2- 172.21.15.98:0/3118841058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f52700ce8b0 0x7f527015d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:49.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.710+0000 7f5276b27640 1 --2- 172.21.15.98:0/3118841058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f527006c1d0 0x7f5270148c10 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:49.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.710+0000 7f5276b27640 1 -- 172.21.15.98:0/3118841058 >> 172.21.15.98:0/3118841058 conn(0x7f52700b2010 msgr2=0x7f52700ccc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:40:49.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.710+0000 7f5276b27640 1 -- 172.21.15.98:0/3118841058 shutdown_connections 2024-09-19T08:40:49.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:49.710+0000 7f5276b27640 1 -- 172.21.15.98:0/3118841058 wait complete. 2024-09-19T08:40:49.725 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-19T08:40:49.859 INFO:teuthology.orchestra.run.smithi098.stdout:[{"id":5490,"entity":{"name":{"type":"client","num":5490},"addr":{"type":"any","addr":"172.21.15.98:0","nonce":379128374}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":9,"uptime":39.471658650000002,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.083112032586026,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5490 172.21.15.98:0/379128374","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"y","hostname":"smithi098","pid":"28958","root":"/"}}] 2024-09-19T08:40:49.860 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi098", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28958", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.98:0", "nonce": 379128374, "type": "any" }, "name": { "num": 5490, "type": "client" } }, "id": 5490, "inst": "client.5490 172.21.15.98:0/379128374", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 9, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.083112032586026 }, "state": "open", "uptime": 39.47165865 } ] 2024-09-19T08:40:49.860 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-19T08:40:54.861 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:40:54.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.994+0000 7eff51e52640 1 Processor -- start 2024-09-19T08:40:54.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.994+0000 7eff51e52640 1 -- start start 2024-09-19T08:40:54.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.994+0000 7eff51e52640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7eff4c06c3a0 0x7eff4c0cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:54.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.994+0000 7eff51e52640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff4c06d310 con 0x7eff4c0cf900 2024-09-19T08:40:54.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.994+0000 7eff51e52640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff4c06d5f0 con 0x7eff4c15c550 2024-09-19T08:40:54.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.994+0000 7eff51e52640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7eff4c06d8d0 con 0x7eff4c06c3a0 2024-09-19T08:40:54.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.994+0000 7eff4bfff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7eff4c0cf900 0x7eff4c0723a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:59910/0 (socket says 172.21.15.98:59910) 2024-09-19T08:40:54.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.994+0000 7eff4bfff640 1 -- 172.21.15.98:0/486457884 learned_addr learned my addr 172.21.15.98:0/486457884 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:40:54.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.994+0000 7eff4affd640 1 --2- 172.21.15.98:0/486457884 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7eff4c06c3a0 0x7eff4c0cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:54.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.994+0000 7eff4a7fc640 1 -- 172.21.15.98:0/486457884 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3634509210 0 0) 0x7eff4c06d310 con 0x7eff4c0cf900 2024-09-19T08:40:54.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.994+0000 7eff4a7fc640 1 -- 172.21.15.98:0/486457884 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7eff34003530 con 0x7eff4c0cf900 2024-09-19T08:40:54.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.998+0000 7eff4a7fc640 1 -- 172.21.15.98:0/486457884 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 491943020 0 0) 0x7eff34003530 con 0x7eff4c0cf900 2024-09-19T08:40:54.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.998+0000 7eff4a7fc640 1 -- 172.21.15.98:0/486457884 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7eff4c06c3a0 msgr2=0x7eff4c0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:40:54.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.998+0000 7eff4a7fc640 1 --2- 172.21.15.98:0/486457884 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7eff4c06c3a0 0x7eff4c0cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:54.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.998+0000 7eff4a7fc640 1 -- 172.21.15.98:0/486457884 >> v1:172.21.15.139:6789/0 conn(0x7eff4c15c550 legacy=0x7eff4c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:54.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.998+0000 7eff4a7fc640 1 -- 172.21.15.98:0/486457884 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff4c06dbb0 con 0x7eff4c0cf900 2024-09-19T08:40:54.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.998+0000 7eff4a7fc640 1 -- 172.21.15.98:0/486457884 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7eff44001a00 con 0x7eff4c0cf900 2024-09-19T08:40:54.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.998+0000 7eff4a7fc640 1 -- 172.21.15.98:0/486457884 >> v1:172.21.15.98:6789/0 conn(0x7eff4c0cf900 legacy=0x7eff4c0723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:54.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.998+0000 7eff4a7fc640 1 --2- 172.21.15.98:0/486457884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff340039f0 0x7eff34003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:54.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.998+0000 7eff4a7fc640 1 --2- 172.21.15.98:0/486457884 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7eff4c06c3a0 0x7eff34004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:54.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.998+0000 7eff4a7fc640 1 --2- 172.21.15.98:0/486457884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff34004d50 0x7eff340051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:54.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.998+0000 7eff4a7fc640 1 -- 172.21.15.98:0/486457884 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7eff4c06d8d0 con 0x7eff34004d50 2024-09-19T08:40:54.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.998+0000 7eff4a7fc640 1 -- 172.21.15.98:0/486457884 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7eff3400ba80 con 0x7eff340039f0 2024-09-19T08:40:54.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.998+0000 7eff4a7fc640 1 -- 172.21.15.98:0/486457884 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7eff3400bc80 con 0x7eff4c06c3a0 2024-09-19T08:40:54.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.998+0000 7eff4bfff640 1 --2- 172.21.15.98:0/486457884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff34004d50 0x7eff340051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:54.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.998+0000 7eff4b7fe640 1 --2- 172.21.15.98:0/486457884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff340039f0 0x7eff34003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:54.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.998+0000 7eff4affd640 1 --2- 172.21.15.98:0/486457884 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7eff4c06c3a0 0x7eff34004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:54.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.998+0000 7eff4bfff640 1 -- 172.21.15.98:0/486457884 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7eff4c06c3a0 msgr2=0x7eff34004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:40:54.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.998+0000 7eff4bfff640 1 --2- 172.21.15.98:0/486457884 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7eff4c06c3a0 0x7eff34004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:54.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.998+0000 7eff4bfff640 1 -- 172.21.15.98:0/486457884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff340039f0 msgr2=0x7eff34003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:40:54.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.998+0000 7eff4bfff640 1 --2- 172.21.15.98:0/486457884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff340039f0 0x7eff34003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:54.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.998+0000 7eff4bfff640 1 -- 172.21.15.98:0/486457884 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff34003530 con 0x7eff34004d50 2024-09-19T08:40:55.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.998+0000 7eff4bfff640 1 --2- 172.21.15.98:0/486457884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff34004d50 0x7eff340051d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7eff4400a810 tx=0x7eff4401d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=230019032ad6785 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:55.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.998+0000 7eff4a7fc640 1 -- 172.21.15.98:0/486457884 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4400a460 con 0x7eff34004d50 2024-09-19T08:40:55.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.998+0000 7eff4a7fc640 1 -- 172.21.15.98:0/486457884 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff440ad650 con 0x7eff34004d50 2024-09-19T08:40:55.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.998+0000 7eff4a7fc640 1 -- 172.21.15.98:0/486457884 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff440ada40 con 0x7eff34004d50 2024-09-19T08:40:55.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.998+0000 7eff51e52640 1 -- 172.21.15.98:0/486457884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff34004d50 msgr2=0x7eff340051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:55.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.998+0000 7eff51e52640 1 --2- 172.21.15.98:0/486457884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff34004d50 0x7eff340051d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7eff4400a810 tx=0x7eff4401d340 comp rx=0 tx=0).stop 2024-09-19T08:40:55.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.998+0000 7eff4bfff640 1 -- 172.21.15.98:0/486457884 reap_dead start 2024-09-19T08:40:55.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.998+0000 7eff51e52640 1 -- 172.21.15.98:0/486457884 shutdown_connections 2024-09-19T08:40:55.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.998+0000 7eff51e52640 1 -- 172.21.15.98:0/486457884 >> 172.21.15.98:0/486457884 conn(0x7eff4c0b2010 msgr2=0x7eff4c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:40:55.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.998+0000 7eff51e52640 1 -- 172.21.15.98:0/486457884 shutdown_connections 2024-09-19T08:40:55.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:54.998+0000 7eff51e52640 1 -- 172.21.15.98:0/486457884 wait complete. 2024-09-19T08:40:55.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.002+0000 7eff51e52640 1 Processor -- start 2024-09-19T08:40:55.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.002+0000 7eff51e52640 1 -- start start 2024-09-19T08:40:55.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.002+0000 7eff51e52640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff4c140640 0x7eff4c15d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:55.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.002+0000 7eff51e52640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7eff4c1e4b00 0x7eff4c1e4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:55.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.002+0000 7eff51e52640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff4c142d80 0x7eff4c143200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:55.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.002+0000 7eff51e52640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7eff4c15cab0 con 0x7eff4c140640 2024-09-19T08:40:55.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.002+0000 7eff51e52640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7eff4c15c930 con 0x7eff4c142d80 2024-09-19T08:40:55.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.002+0000 7eff51e52640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7eff4c0cf3c0 con 0x7eff4c1e4b00 2024-09-19T08:40:55.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.002+0000 7eff4b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff4c140640 0x7eff4c15d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:55.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.002+0000 7eff4b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff4c140640 0x7eff4c15d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:43128/0 (socket says 172.21.15.98:43128) 2024-09-19T08:40:55.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.002+0000 7eff4b7fe640 1 -- 172.21.15.98:0/4047785156 learned_addr learned my addr 172.21.15.98:0/4047785156 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:40:55.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.002+0000 7eff4affd640 1 --2- 172.21.15.98:0/4047785156 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7eff4c1e4b00 0x7eff4c1e4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:55.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.002+0000 7eff4bfff640 1 --2- 172.21.15.98:0/4047785156 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff4c142d80 0x7eff4c143200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:55.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.002+0000 7eff4b7fe640 1 -- 172.21.15.98:0/4047785156 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7eff4c1e4b00 msgr2=0x7eff4c1e4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:55.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.002+0000 7eff4b7fe640 1 --2- 172.21.15.98:0/4047785156 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7eff4c1e4b00 0x7eff4c1e4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:55.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.002+0000 7eff4b7fe640 1 -- 172.21.15.98:0/4047785156 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff4c142d80 msgr2=0x7eff4c143200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:55.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.002+0000 7eff4b7fe640 1 --2- 172.21.15.98:0/4047785156 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff4c142d80 0x7eff4c143200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:55.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.002+0000 7eff4b7fe640 1 -- 172.21.15.98:0/4047785156 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff4c140a20 con 0x7eff4c140640 2024-09-19T08:40:55.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.002+0000 7eff4b7fe640 1 --2- 172.21.15.98:0/4047785156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff4c140640 0x7eff4c15d120 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7eff3c00e580 tx=0x7eff3c00ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:55.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.002+0000 7eff33fff640 1 -- 172.21.15.98:0/4047785156 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c0052e0 con 0x7eff4c140640 2024-09-19T08:40:55.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.002+0000 7eff33fff640 1 -- 172.21.15.98:0/4047785156 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff3c018070 con 0x7eff4c140640 2024-09-19T08:40:55.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.002+0000 7eff51e52640 1 -- 172.21.15.98:0/4047785156 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff4c1f2360 con 0x7eff4c140640 2024-09-19T08:40:55.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.002+0000 7eff33fff640 1 -- 172.21.15.98:0/4047785156 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c003820 con 0x7eff4c140640 2024-09-19T08:40:55.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.002+0000 7eff51e52640 1 -- 172.21.15.98:0/4047785156 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff4c141100 con 0x7eff4c140640 2024-09-19T08:40:55.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.006+0000 7eff51e52640 1 -- 172.21.15.98:0/4047785156 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff4c069460 con 0x7eff4c140640 2024-09-19T08:40:55.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.006+0000 7eff33fff640 1 -- 172.21.15.98:0/4047785156 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7eff3c0039c0 con 0x7eff4c140640 2024-09-19T08:40:55.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.006+0000 7eff33fff640 1 --2- 172.21.15.98:0/4047785156 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7eff38067b00 0x7eff38069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:40:55.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.006+0000 7eff4affd640 1 --2- 172.21.15.98:0/4047785156 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7eff38067b00 0x7eff38069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:40:55.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.006+0000 7eff33fff640 1 -- 172.21.15.98:0/4047785156 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7eff3c085d80 con 0x7eff4c140640 2024-09-19T08:40:55.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.006+0000 7eff4affd640 1 --2- 172.21.15.98:0/4047785156 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7eff38067b00 0x7eff38069fc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7eff4c156630 tx=0x7eff40008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:40:55.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.010+0000 7eff33fff640 1 -- 172.21.15.98:0/4047785156 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7eff3c056530 con 0x7eff4c140640 2024-09-19T08:40:55.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.210+0000 7eff51e52640 1 -- 172.21.15.98:0/4047785156 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff4c0cf6e0 con 0x7eff4c140640 2024-09-19T08:40:55.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.210+0000 7eff33fff640 1 -- 172.21.15.98:0/4047785156 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5229 (secure 0 0 0) 0x7eff3c005480 con 0x7eff4c140640 2024-09-19T08:40:55.211 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:40:55.211 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"btime":"2024-09-19T08:39:55:181532+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/2874086421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":2874086421},{"type":"v1","addr":"172.21.15.139:6833","nonce":2874086421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4996,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/4178563848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":4178563848},{"type":"v1","addr":"172.21.15.139:6835","nonce":4178563848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3584568343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3584568343},{"type":"v1","addr":"172.21.15.98:6835","nonce":3584568343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:39:38.262938+0000","modified":"2024-09-19T08:39:41.276985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/3085042202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3085042202},{"type":"v1","addr":"172.21.15.98:6837","nonce":3085042202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244]},"id":3}]} 2024-09-19T08:40:55.211 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 24 2024-09-19T08:40:55.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.214+0000 7eff51e52640 1 -- 172.21.15.98:0/4047785156 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7eff38067b00 msgr2=0x7eff38069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:55.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.214+0000 7eff51e52640 1 --2- 172.21.15.98:0/4047785156 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7eff38067b00 0x7eff38069fc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7eff4c156630 tx=0x7eff40008040 comp rx=0 tx=0).stop 2024-09-19T08:40:55.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.214+0000 7eff51e52640 1 -- 172.21.15.98:0/4047785156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff4c140640 msgr2=0x7eff4c15d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:40:55.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.214+0000 7eff51e52640 1 --2- 172.21.15.98:0/4047785156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff4c140640 0x7eff4c15d120 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7eff3c00e580 tx=0x7eff3c00ea40 comp rx=0 tx=0).stop 2024-09-19T08:40:55.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.214+0000 7eff51e52640 1 -- 172.21.15.98:0/4047785156 shutdown_connections 2024-09-19T08:40:55.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.214+0000 7eff51e52640 1 --2- 172.21.15.98:0/4047785156 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7eff38067b00 0x7eff38069fc0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:55.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.214+0000 7eff51e52640 1 --2- 172.21.15.98:0/4047785156 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff4c142d80 0x7eff4c143200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:55.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.214+0000 7eff51e52640 1 --2- 172.21.15.98:0/4047785156 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7eff4c1e4b00 0x7eff4c1e4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:55.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.214+0000 7eff51e52640 1 --2- 172.21.15.98:0/4047785156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff4c140640 0x7eff4c15d120 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:40:55.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.214+0000 7eff51e52640 1 -- 172.21.15.98:0/4047785156 >> 172.21.15.98:0/4047785156 conn(0x7eff4c0b2010 msgr2=0x7eff4c072b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:40:55.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.214+0000 7eff51e52640 1 -- 172.21.15.98:0/4047785156 shutdown_connections 2024-09-19T08:40:55.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:40:55.214+0000 7eff51e52640 1 -- 172.21.15.98:0/4047785156 wait complete. 2024-09-19T08:40:55.230 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-19T08:40:55.372 INFO:teuthology.orchestra.run.smithi098.stdout:[{"id":5490,"entity":{"name":{"type":"client","num":5490},"addr":{"type":"any","addr":"172.21.15.98:0","nonce":379128374}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":8,"uptime":44.983610808999998,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.828969031877634,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5490 172.21.15.98:0/379128374","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"y","hostname":"smithi098","pid":"28958","root":"/"}}] 2024-09-19T08:40:55.372 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi098", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28958", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.98:0", "nonce": 379128374, "type": "any" }, "name": { "num": 5490, "type": "client" } }, "id": 5490, "inst": "client.5490 172.21.15.98:0/379128374", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.828969031877634 }, "state": "open", "uptime": 44.983610809 } ] 2024-09-19T08:40:55.372 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-19T08:41:00.373 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:41:00.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.498+0000 7ff1fb289640 1 Processor -- start 2024-09-19T08:41:00.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.498+0000 7ff1fb289640 1 -- start start 2024-09-19T08:41:00.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.498+0000 7ff1fb289640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff1f40b0280 0x7ff1f406d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:00.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.498+0000 7ff1fb289640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff1f4074c70 0x7ff1f40b6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:00.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.498+0000 7ff1fb289640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1f40b6d90 0x7ff1f40d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:00.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.498+0000 7ff1fb289640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7ff1f40b7200 con 0x7ff1f40b0280 2024-09-19T08:41:00.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.498+0000 7ff1fb289640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff1f4160350 con 0x7ff1f4074c70 2024-09-19T08:41:00.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.498+0000 7ff1fb289640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff1f4160630 con 0x7ff1f40b6d90 2024-09-19T08:41:00.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.498+0000 7ff1f9025640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff1f40b0280 0x7ff1f406d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:00.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.498+0000 7ff1f9025640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff1f40b0280 0x7ff1f406d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46522/0 (socket says 172.21.15.98:46522) 2024-09-19T08:41:00.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.498+0000 7ff1f9025640 1 -- 172.21.15.98:0/2136978245 learned_addr learned my addr 172.21.15.98:0/2136978245 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:00.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.498+0000 7ff1f9826640 1 --2- 172.21.15.98:0/2136978245 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1f40b6d90 0x7ff1f40d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:00.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.498+0000 7ff1f8824640 1 --2- 172.21.15.98:0/2136978245 >> v2:172.21.15.139:3300/0 conn(0x7ff1f4074c70 0x7ff1f40b6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:00.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.498+0000 7ff1f9025640 1 -- 172.21.15.98:0/2136978245 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1f40b6d90 msgr2=0x7ff1f40d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:00.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.498+0000 7ff1f9025640 1 --2- 172.21.15.98:0/2136978245 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1f40b6d90 0x7ff1f40d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:00.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.498+0000 7ff1f9025640 1 -- 172.21.15.98:0/2136978245 >> v2:172.21.15.139:3300/0 conn(0x7ff1f4074c70 msgr2=0x7ff1f40b6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:00.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.498+0000 7ff1f9025640 1 --2- 172.21.15.98:0/2136978245 >> v2:172.21.15.139:3300/0 conn(0x7ff1f4074c70 0x7ff1f40b6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:00.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.498+0000 7ff1f9025640 1 -- 172.21.15.98:0/2136978245 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f4160910 con 0x7ff1f40b0280 2024-09-19T08:41:00.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.502+0000 7ff1f9025640 1 --2- 172.21.15.98:0/2136978245 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1f40b0280 0x7ff1f406d1d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff1f406cf40 tx=0x7ff1f001c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d616951b70d756f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:00.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.502+0000 7ff1ebfff640 1 -- 172.21.15.98:0/2136978245 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f00b0070 con 0x7ff1f40b0280 2024-09-19T08:41:00.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.502+0000 7ff1ebfff640 1 -- 172.21.15.98:0/2136978245 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1f00ab600 con 0x7ff1f40b0280 2024-09-19T08:41:00.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.502+0000 7ff1ebfff640 1 -- 172.21.15.98:0/2136978245 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f00ab9d0 con 0x7ff1f40b0280 2024-09-19T08:41:00.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.502+0000 7ff1fb289640 1 -- 172.21.15.98:0/2136978245 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1f40b0280 msgr2=0x7ff1f406d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:00.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.502+0000 7ff1fb289640 1 --2- 172.21.15.98:0/2136978245 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1f40b0280 0x7ff1f406d1d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff1f406cf40 tx=0x7ff1f001c1d0 comp rx=0 tx=0).stop 2024-09-19T08:41:00.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.502+0000 7ff1fb289640 1 -- 172.21.15.98:0/2136978245 shutdown_connections 2024-09-19T08:41:00.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.502+0000 7ff1fb289640 1 --2- 172.21.15.98:0/2136978245 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1f40b6d90 0x7ff1f40d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:00.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.502+0000 7ff1fb289640 1 --2- 172.21.15.98:0/2136978245 >> v2:172.21.15.139:3300/0 conn(0x7ff1f4074c70 0x7ff1f40b6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:00.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.502+0000 7ff1fb289640 1 --2- 172.21.15.98:0/2136978245 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1f40b0280 0x7ff1f406d1d0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:00.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.502+0000 7ff1fb289640 1 -- 172.21.15.98:0/2136978245 >> 172.21.15.98:0/2136978245 conn(0x7ff1f4073170 msgr2=0x7ff1f4073580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:00.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.502+0000 7ff1fb289640 1 -- 172.21.15.98:0/2136978245 shutdown_connections 2024-09-19T08:41:00.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.502+0000 7ff1fb289640 1 -- 172.21.15.98:0/2136978245 wait complete. 2024-09-19T08:41:00.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.502+0000 7ff1fb289640 1 Processor -- start 2024-09-19T08:41:00.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.502+0000 7ff1fb289640 1 -- start start 2024-09-19T08:41:00.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.502+0000 7ff1fb289640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1f4074c70 0x7ff1f409a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:00.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.502+0000 7ff1fb289640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1f40b0280 0x7ff1f40b0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:00.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.502+0000 7ff1fb289640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1f40b6d90 0x7ff1f4157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:00.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.502+0000 7ff1fb289640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f40ae4c0 con 0x7ff1f40b0280 2024-09-19T08:41:00.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.502+0000 7ff1fb289640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f40ae340 con 0x7ff1f40b6d90 2024-09-19T08:41:00.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.502+0000 7ff1fb289640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff1f40ae640 con 0x7ff1f4074c70 2024-09-19T08:41:00.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.506+0000 7ff1f8824640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1f40b0280 0x7ff1f40b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:00.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.506+0000 7ff1f8824640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1f40b0280 0x7ff1f40b0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46530/0 (socket says 172.21.15.98:46530) 2024-09-19T08:41:00.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.506+0000 7ff1f8824640 1 -- 172.21.15.98:0/296602104 learned_addr learned my addr 172.21.15.98:0/296602104 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:00.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.506+0000 7ff1f9025640 1 --2- 172.21.15.98:0/296602104 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1f4074c70 0x7ff1f409a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:00.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.506+0000 7ff1f9826640 1 --2- 172.21.15.98:0/296602104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1f40b6d90 0x7ff1f4157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:00.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.506+0000 7ff1f8824640 1 -- 172.21.15.98:0/296602104 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1f4074c70 msgr2=0x7ff1f409a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:00.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.506+0000 7ff1f8824640 1 --2- 172.21.15.98:0/296602104 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1f4074c70 0x7ff1f409a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:00.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.506+0000 7ff1f8824640 1 -- 172.21.15.98:0/296602104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1f40b6d90 msgr2=0x7ff1f4157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:00.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.506+0000 7ff1f8824640 1 --2- 172.21.15.98:0/296602104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1f40b6d90 0x7ff1f4157640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:00.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.506+0000 7ff1f8824640 1 -- 172.21.15.98:0/296602104 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f406d190 con 0x7ff1f40b0280 2024-09-19T08:41:00.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.506+0000 7ff1f8824640 1 --2- 172.21.15.98:0/296602104 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1f40b0280 0x7ff1f40b0660 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff1e400d800 tx=0x7ff1e400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:00.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.506+0000 7ff1e9ffb640 1 -- 172.21.15.98:0/296602104 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4018020 con 0x7ff1f40b0280 2024-09-19T08:41:00.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.506+0000 7ff1e9ffb640 1 -- 172.21.15.98:0/296602104 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1e40096a0 con 0x7ff1f40b0280 2024-09-19T08:41:00.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.506+0000 7ff1fb289640 1 -- 172.21.15.98:0/296602104 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f4154df0 con 0x7ff1f40b0280 2024-09-19T08:41:00.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.506+0000 7ff1e9ffb640 1 -- 172.21.15.98:0/296602104 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4009a50 con 0x7ff1f40b0280 2024-09-19T08:41:00.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.506+0000 7ff1fb289640 1 -- 172.21.15.98:0/296602104 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f41553b0 con 0x7ff1f40b0280 2024-09-19T08:41:00.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.506+0000 7ff1fb289640 1 -- 172.21.15.98:0/296602104 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1f40b6850 con 0x7ff1f40b0280 2024-09-19T08:41:00.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.506+0000 7ff1e9ffb640 1 -- 172.21.15.98:0/296602104 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7ff1e401b020 con 0x7ff1f40b0280 2024-09-19T08:41:00.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.506+0000 7ff1e9ffb640 1 --2- 172.21.15.98:0/296602104 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff1d8067b00 0x7ff1d8069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:00.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.506+0000 7ff1e9ffb640 1 -- 172.21.15.98:0/296602104 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7ff1e4086670 con 0x7ff1f40b0280 2024-09-19T08:41:00.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.506+0000 7ff1f9025640 1 --2- 172.21.15.98:0/296602104 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff1d8067b00 0x7ff1d8069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:00.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.506+0000 7ff1f9025640 1 --2- 172.21.15.98:0/296602104 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff1d8067b00 0x7ff1d8069fc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff1f00ad000 tx=0x7ff1f0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:00.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.510+0000 7ff1e9ffb640 1 -- 172.21.15.98:0/296602104 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff1e4022020 con 0x7ff1f40b0280 2024-09-19T08:41:00.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.710+0000 7ff1fb289640 1 -- 172.21.15.98:0/296602104 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1f406efd0 con 0x7ff1f40b0280 2024-09-19T08:41:00.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.710+0000 7ff1e9ffb640 1 -- 172.21.15.98:0/296602104 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5229 (secure 0 0 0) 0x7ff1e4056e10 con 0x7ff1f40b0280 2024-09-19T08:41:00.710 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:41:00.710 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"btime":"2024-09-19T08:39:55:181532+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/2874086421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":2874086421},{"type":"v1","addr":"172.21.15.139:6833","nonce":2874086421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4996,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/4178563848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":4178563848},{"type":"v1","addr":"172.21.15.139:6835","nonce":4178563848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3584568343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3584568343},{"type":"v1","addr":"172.21.15.98:6835","nonce":3584568343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:39:38.262938+0000","modified":"2024-09-19T08:39:41.276985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/3085042202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3085042202},{"type":"v1","addr":"172.21.15.98:6837","nonce":3085042202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244]},"id":3}]} 2024-09-19T08:41:00.710 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 24 2024-09-19T08:41:00.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.710+0000 7ff1fb289640 1 -- 172.21.15.98:0/296602104 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff1d8067b00 msgr2=0x7ff1d8069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:00.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.710+0000 7ff1fb289640 1 --2- 172.21.15.98:0/296602104 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff1d8067b00 0x7ff1d8069fc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff1f00ad000 tx=0x7ff1f0008040 comp rx=0 tx=0).stop 2024-09-19T08:41:00.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.710+0000 7ff1fb289640 1 -- 172.21.15.98:0/296602104 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1f40b0280 msgr2=0x7ff1f40b0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:00.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.710+0000 7ff1fb289640 1 --2- 172.21.15.98:0/296602104 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1f40b0280 0x7ff1f40b0660 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff1e400d800 tx=0x7ff1e400dcc0 comp rx=0 tx=0).stop 2024-09-19T08:41:00.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.710+0000 7ff1fb289640 1 -- 172.21.15.98:0/296602104 shutdown_connections 2024-09-19T08:41:00.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.710+0000 7ff1fb289640 1 --2- 172.21.15.98:0/296602104 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff1d8067b00 0x7ff1d8069fc0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:00.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.710+0000 7ff1fb289640 1 --2- 172.21.15.98:0/296602104 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1f40b6d90 0x7ff1f4157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:00.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.710+0000 7ff1fb289640 1 --2- 172.21.15.98:0/296602104 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1f40b0280 0x7ff1f40b0660 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:00.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.710+0000 7ff1fb289640 1 --2- 172.21.15.98:0/296602104 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1f4074c70 0x7ff1f409a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:00.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.710+0000 7ff1fb289640 1 -- 172.21.15.98:0/296602104 >> 172.21.15.98:0/296602104 conn(0x7ff1f4073170 msgr2=0x7ff1f415f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:00.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.710+0000 7ff1fb289640 1 -- 172.21.15.98:0/296602104 shutdown_connections 2024-09-19T08:41:00.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:00.710+0000 7ff1fb289640 1 -- 172.21.15.98:0/296602104 wait complete. 2024-09-19T08:41:00.728 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-19T08:41:00.868 INFO:teuthology.orchestra.run.smithi098.stdout:[{"id":5490,"entity":{"name":{"type":"client","num":5490},"addr":{"type":"any","addr":"172.21.15.98:0","nonce":379128374}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":8,"uptime":50.479563059999997,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.578765867042701,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5490 172.21.15.98:0/379128374","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"y","hostname":"smithi098","pid":"28958","root":"/"}}] 2024-09-19T08:41:00.869 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi098", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28958", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.98:0", "nonce": 379128374, "type": "any" }, "name": { "num": 5490, "type": "client" } }, "id": 5490, "inst": "client.5490 172.21.15.98:0/379128374", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.5787658670427 }, "state": "open", "uptime": 50.47956306 } ] 2024-09-19T08:41:00.869 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-19T08:41:05.870 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:41:06.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.002+0000 7fa7a540d640 1 Processor -- start 2024-09-19T08:41:06.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.002+0000 7fa7a540d640 1 -- start start 2024-09-19T08:41:06.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.002+0000 7fa7a540d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7a0147530 0x7fa7a00d1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:06.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.002+0000 7fa7a540d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa7a00b6200 0x7fa7a00b6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:06.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.002+0000 7fa7a540d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7a00b8060 con 0x7fa7a00d1d70 2024-09-19T08:41:06.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.002+0000 7fa7a540d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fa7a00b6bb0 con 0x7fa7a00b6200 2024-09-19T08:41:06.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.002+0000 7fa7a540d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa7a00d2230 con 0x7fa7a0147530 2024-09-19T08:41:06.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.002+0000 7fa79effd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7a0147530 0x7fa7a00d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:06.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.002+0000 7fa79f7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa7a00b6200 0x7fa7a00b6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:06.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.002+0000 7fa79e7fc640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fa7a00d1d70 0x7fa7a015ec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:48024/0 (socket says 172.21.15.98:48024) 2024-09-19T08:41:06.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.002+0000 7fa79f7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa7a00b6200 0x7fa7a00b6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:53394/0 (socket says 172.21.15.98:53394) 2024-09-19T08:41:06.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.002+0000 7fa79effd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7a0147530 0x7fa7a00d1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:48884/0 (socket says 172.21.15.98:48884) 2024-09-19T08:41:06.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.002+0000 7fa79e7fc640 1 -- 172.21.15.98:0/43516272 learned_addr learned my addr 172.21.15.98:0/43516272 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:41:06.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.002+0000 7fa79f7fe640 1 -- 172.21.15.98:0/43516272 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7a0147530 msgr2=0x7fa7a00d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:06.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.002+0000 7fa79f7fe640 1 --2- 172.21.15.98:0/43516272 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7a0147530 0x7fa7a00d1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:06.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.002+0000 7fa79f7fe640 1 -- 172.21.15.98:0/43516272 >> v1:172.21.15.139:6789/0 conn(0x7fa7a00d1d70 legacy=0x7fa7a015ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:06.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.002+0000 7fa79f7fe640 1 -- 172.21.15.98:0/43516272 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a00b8340 con 0x7fa7a00b6200 2024-09-19T08:41:06.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.002+0000 7fa79effd640 1 --2- 172.21.15.98:0/43516272 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7a0147530 0x7fa7a00d1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:41:06.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.006+0000 7fa79f7fe640 1 --2- 172.21.15.98:0/43516272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7a00b6200 0x7fa7a00b6670 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa798009d90 tx=0x7fa79801c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f59ecc634297db2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:06.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.006+0000 7fa79dffb640 1 -- 172.21.15.98:0/43516272 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7980b2070 con 0x7fa7a00b6200 2024-09-19T08:41:06.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.006+0000 7fa79dffb640 1 -- 172.21.15.98:0/43516272 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7980a5a00 con 0x7fa7a00b6200 2024-09-19T08:41:06.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.006+0000 7fa79dffb640 1 -- 172.21.15.98:0/43516272 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7980a5dd0 con 0x7fa7a00b6200 2024-09-19T08:41:06.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.006+0000 7fa7a540d640 1 -- 172.21.15.98:0/43516272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7a00b6200 msgr2=0x7fa7a00b6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:06.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.006+0000 7fa7a540d640 1 --2- 172.21.15.98:0/43516272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7a00b6200 0x7fa7a00b6670 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa798009d90 tx=0x7fa79801c940 comp rx=0 tx=0).stop 2024-09-19T08:41:06.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.006+0000 7fa7a540d640 1 -- 172.21.15.98:0/43516272 shutdown_connections 2024-09-19T08:41:06.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.006+0000 7fa7a540d640 1 --2- 172.21.15.98:0/43516272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7a00b6200 0x7fa7a00b6670 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:06.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.006+0000 7fa7a540d640 1 --2- 172.21.15.98:0/43516272 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7a0147530 0x7fa7a00d1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:06.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.006+0000 7fa7a540d640 1 -- 172.21.15.98:0/43516272 >> 172.21.15.98:0/43516272 conn(0x7fa7a0075070 msgr2=0x7fa7a00722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:06.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.006+0000 7fa7a540d640 1 -- 172.21.15.98:0/43516272 shutdown_connections 2024-09-19T08:41:06.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.006+0000 7fa7a540d640 1 -- 172.21.15.98:0/43516272 wait complete. 2024-09-19T08:41:06.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.006+0000 7fa7a540d640 1 Processor -- start 2024-09-19T08:41:06.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.006+0000 7fa7a540d640 1 -- start start 2024-09-19T08:41:06.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.010+0000 7fa7a540d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7a00b6200 0x7fa7a013c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:06.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.010+0000 7fa7a540d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7a00d1d70 0x7fa7a015d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:06.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.010+0000 7fa7a540d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7a0147530 0x7fa7a0160570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:06.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.010+0000 7fa7a540d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a0147a90 con 0x7fa7a0147530 2024-09-19T08:41:06.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.010+0000 7fa7a540d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a0147910 con 0x7fa7a00b6200 2024-09-19T08:41:06.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.010+0000 7fa7a540d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa7a00b9630 con 0x7fa7a00d1d70 2024-09-19T08:41:06.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.010+0000 7fa79f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7a0147530 0x7fa7a0160570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:06.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.010+0000 7fa79f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7a0147530 0x7fa7a0160570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46534/0 (socket says 172.21.15.98:46534) 2024-09-19T08:41:06.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.010+0000 7fa79f7fe640 1 -- 172.21.15.98:0/475100599 learned_addr learned my addr 172.21.15.98:0/475100599 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:06.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.010+0000 7fa79effd640 1 --2- 172.21.15.98:0/475100599 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7a00b6200 0x7fa7a013c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:06.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.010+0000 7fa79e7fc640 1 --2- 172.21.15.98:0/475100599 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7a00d1d70 0x7fa7a015d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:06.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.010+0000 7fa79f7fe640 1 -- 172.21.15.98:0/475100599 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7a00d1d70 msgr2=0x7fa7a015d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:06.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.010+0000 7fa79f7fe640 1 --2- 172.21.15.98:0/475100599 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7a00d1d70 0x7fa7a015d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:06.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.010+0000 7fa79f7fe640 1 -- 172.21.15.98:0/475100599 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7a00b6200 msgr2=0x7fa7a013c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:06.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.010+0000 7fa79f7fe640 1 --2- 172.21.15.98:0/475100599 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7a00b6200 0x7fa7a013c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:06.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.010+0000 7fa79f7fe640 1 -- 172.21.15.98:0/475100599 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a015e1a0 con 0x7fa7a0147530 2024-09-19T08:41:06.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.010+0000 7fa79effd640 1 --2- 172.21.15.98:0/475100599 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7a00b6200 0x7fa7a013c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:41:06.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.010+0000 7fa79f7fe640 1 --2- 172.21.15.98:0/475100599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7a0147530 0x7fa7a0160570 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa7980b0040 tx=0x7fa7980196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:06.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.010+0000 7fa7877fe640 1 -- 172.21.15.98:0/475100599 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7980b2050 con 0x7fa7a0147530 2024-09-19T08:41:06.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.010+0000 7fa7877fe640 1 -- 172.21.15.98:0/475100599 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7980be070 con 0x7fa7a0147530 2024-09-19T08:41:06.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.010+0000 7fa7a540d640 1 -- 172.21.15.98:0/475100599 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7a013cb30 con 0x7fa7a0147530 2024-09-19T08:41:06.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.010+0000 7fa7877fe640 1 -- 172.21.15.98:0/475100599 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7980adb10 con 0x7fa7a0147530 2024-09-19T08:41:06.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.010+0000 7fa7a540d640 1 -- 172.21.15.98:0/475100599 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7a013bb10 con 0x7fa7a0147530 2024-09-19T08:41:06.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.010+0000 7fa7a540d640 1 -- 172.21.15.98:0/475100599 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7a00b6670 con 0x7fa7a0147530 2024-09-19T08:41:06.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.010+0000 7fa7877fe640 1 -- 172.21.15.98:0/475100599 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7fa7980adcb0 con 0x7fa7a0147530 2024-09-19T08:41:06.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.010+0000 7fa7877fe640 1 --2- 172.21.15.98:0/475100599 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa78c067b30 0x7fa78c069ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:06.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.010+0000 7fa79effd640 1 --2- 172.21.15.98:0/475100599 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa78c067b30 0x7fa78c069ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:06.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.010+0000 7fa7877fe640 1 -- 172.21.15.98:0/475100599 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7fa798121490 con 0x7fa7a0147530 2024-09-19T08:41:06.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.010+0000 7fa79effd640 1 --2- 172.21.15.98:0/475100599 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa78c067b30 0x7fa78c069ff0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0161180 tx=0x7fa790008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:06.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.018+0000 7fa7877fe640 1 -- 172.21.15.98:0/475100599 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa7a015e1a0 con 0x7fa7a0147530 2024-09-19T08:41:06.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.206+0000 7fa7a540d640 1 -- 172.21.15.98:0/475100599 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7a00d1790 con 0x7fa7a0147530 2024-09-19T08:41:06.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.210+0000 7fa7877fe640 1 -- 172.21.15.98:0/475100599 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5229 (secure 0 0 0) 0x7fa798016560 con 0x7fa7a0147530 2024-09-19T08:41:06.209 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:41:06.209 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"btime":"2024-09-19T08:39:55:181532+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/2874086421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":2874086421},{"type":"v1","addr":"172.21.15.139:6833","nonce":2874086421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4996,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/4178563848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":4178563848},{"type":"v1","addr":"172.21.15.139:6835","nonce":4178563848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3584568343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3584568343},{"type":"v1","addr":"172.21.15.98:6835","nonce":3584568343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:39:38.262938+0000","modified":"2024-09-19T08:39:41.276985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/3085042202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3085042202},{"type":"v1","addr":"172.21.15.98:6837","nonce":3085042202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244]},"id":3}]} 2024-09-19T08:41:06.209 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 24 2024-09-19T08:41:06.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.210+0000 7fa7a540d640 1 -- 172.21.15.98:0/475100599 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa78c067b30 msgr2=0x7fa78c069ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:06.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.210+0000 7fa7a540d640 1 --2- 172.21.15.98:0/475100599 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa78c067b30 0x7fa78c069ff0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0161180 tx=0x7fa790008040 comp rx=0 tx=0).stop 2024-09-19T08:41:06.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.210+0000 7fa7a540d640 1 -- 172.21.15.98:0/475100599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7a0147530 msgr2=0x7fa7a0160570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:06.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.210+0000 7fa7a540d640 1 --2- 172.21.15.98:0/475100599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7a0147530 0x7fa7a0160570 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa7980b0040 tx=0x7fa7980196e0 comp rx=0 tx=0).stop 2024-09-19T08:41:06.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.210+0000 7fa7a540d640 1 -- 172.21.15.98:0/475100599 shutdown_connections 2024-09-19T08:41:06.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.210+0000 7fa7a540d640 1 --2- 172.21.15.98:0/475100599 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa78c067b30 0x7fa78c069ff0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:06.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.210+0000 7fa7a540d640 1 --2- 172.21.15.98:0/475100599 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7a0147530 0x7fa7a0160570 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:06.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.210+0000 7fa7a540d640 1 --2- 172.21.15.98:0/475100599 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7a00d1d70 0x7fa7a015d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:06.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.210+0000 7fa7a540d640 1 --2- 172.21.15.98:0/475100599 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7a00b6200 0x7fa7a013c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:06.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.210+0000 7fa7a540d640 1 -- 172.21.15.98:0/475100599 >> 172.21.15.98:0/475100599 conn(0x7fa7a0075070 msgr2=0x7fa7a0146610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:06.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.210+0000 7fa7a540d640 1 -- 172.21.15.98:0/475100599 shutdown_connections 2024-09-19T08:41:06.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:06.210+0000 7fa7a540d640 1 -- 172.21.15.98:0/475100599 wait complete. 2024-09-19T08:41:06.227 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-19T08:41:06.367 INFO:teuthology.orchestra.run.smithi098.stdout:[{"id":5490,"entity":{"name":{"type":"client","num":5490},"addr":{"type":"any","addr":"172.21.15.98:0","nonce":379128374}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":7,"uptime":55.979515241000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.331541122231862,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5490 172.21.15.98:0/379128374","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"y","hostname":"smithi098","pid":"28958","root":"/"}}] 2024-09-19T08:41:06.368 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi098", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28958", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.98:0", "nonce": 379128374, "type": "any" }, "name": { "num": 5490, "type": "client" } }, "id": 5490, "inst": "client.5490 172.21.15.98:0/379128374", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 7, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.33154112223186 }, "state": "open", "uptime": 55.979515241 } ] 2024-09-19T08:41:06.368 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-19T08:41:11.369 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:41:11.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.502+0000 7fdc770c1640 1 Processor -- start 2024-09-19T08:41:11.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.502+0000 7fdc770c1640 1 -- start start 2024-09-19T08:41:11.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.502+0000 7fdc770c1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdc70160740 0x7fdc70163170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:11.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.502+0000 7fdc770c1640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fdc7006c3a0 0x7fdc700ceb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:11.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.502+0000 7fdc770c1640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc70163bf0 con 0x7fdc701636b0 2024-09-19T08:41:11.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.502+0000 7fdc770c1640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fdc70163ed0 con 0x7fdc7006c3a0 2024-09-19T08:41:11.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.502+0000 7fdc770c1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fdc700ce320 con 0x7fdc70160740 2024-09-19T08:41:11.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.502+0000 7fdc7565e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fdc7006c3a0 0x7fdc700ceb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:11.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.502+0000 7fdc7565e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fdc7006c3a0 0x7fdc700ceb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:43724/0 (socket says 172.21.15.98:43724) 2024-09-19T08:41:11.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.502+0000 7fdc7565e640 1 -- 172.21.15.98:0/3995260576 learned_addr learned my addr 172.21.15.98:0/3995260576 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:11.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.502+0000 7fdc74e5d640 1 --2- 172.21.15.98:0/3995260576 >> v2:172.21.15.139:3300/0 conn(0x7fdc70160740 0x7fdc70163170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:11.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.502+0000 7fdc7565e640 1 -- 172.21.15.98:0/3995260576 >> v2:172.21.15.139:3300/0 conn(0x7fdc70160740 msgr2=0x7fdc70163170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:11.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.502+0000 7fdc7565e640 1 --2- 172.21.15.98:0/3995260576 >> v2:172.21.15.139:3300/0 conn(0x7fdc70160740 0x7fdc70163170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:11.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.502+0000 7fdc7565e640 1 -- 172.21.15.98:0/3995260576 >> v1:172.21.15.139:6789/0 conn(0x7fdc701636b0 legacy=0x7fdc7006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:11.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.502+0000 7fdc7565e640 1 -- 172.21.15.98:0/3995260576 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc700ce600 con 0x7fdc7006c3a0 2024-09-19T08:41:11.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.506+0000 7fdc7565e640 1 --2- 172.21.15.98:0/3995260576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc7006c3a0 0x7fdc700ceb20 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fdc7006e1e0 tx=0x7fdc6801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b746a7d62251cf0b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:11.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.506+0000 7fdc6f7fe640 1 -- 172.21.15.98:0/3995260576 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc680b0070 con 0x7fdc7006c3a0 2024-09-19T08:41:11.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.506+0000 7fdc6f7fe640 1 -- 172.21.15.98:0/3995260576 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc680ab600 con 0x7fdc7006c3a0 2024-09-19T08:41:11.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.506+0000 7fdc6f7fe640 1 -- 172.21.15.98:0/3995260576 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc680ab9d0 con 0x7fdc7006c3a0 2024-09-19T08:41:11.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.506+0000 7fdc770c1640 1 -- 172.21.15.98:0/3995260576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc7006c3a0 msgr2=0x7fdc700ceb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:11.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.506+0000 7fdc770c1640 1 --2- 172.21.15.98:0/3995260576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc7006c3a0 0x7fdc700ceb20 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fdc7006e1e0 tx=0x7fdc6801c520 comp rx=0 tx=0).stop 2024-09-19T08:41:11.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.506+0000 7fdc770c1640 1 -- 172.21.15.98:0/3995260576 shutdown_connections 2024-09-19T08:41:11.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.506+0000 7fdc770c1640 1 --2- 172.21.15.98:0/3995260576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc7006c3a0 0x7fdc700ceb20 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:11.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.506+0000 7fdc770c1640 1 --2- 172.21.15.98:0/3995260576 >> v2:172.21.15.139:3300/0 conn(0x7fdc70160740 0x7fdc70163170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:11.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.506+0000 7fdc770c1640 1 -- 172.21.15.98:0/3995260576 >> 172.21.15.98:0/3995260576 conn(0x7fdc700b2010 msgr2=0x7fdc700b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:11.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.506+0000 7fdc770c1640 1 -- 172.21.15.98:0/3995260576 shutdown_connections 2024-09-19T08:41:11.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.506+0000 7fdc770c1640 1 -- 172.21.15.98:0/3995260576 wait complete. 2024-09-19T08:41:11.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.506+0000 7fdc770c1640 1 Processor -- start 2024-09-19T08:41:11.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.506+0000 7fdc770c1640 1 -- start start 2024-09-19T08:41:11.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.506+0000 7fdc770c1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc7006c3a0 0x7fdc70152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:11.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.506+0000 7fdc770c1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc70160740 0x7fdc7013f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:11.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.506+0000 7fdc770c1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc701636b0 0x7fdc7015da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:11.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.506+0000 7fdc770c1640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdc700cfc00 con 0x7fdc7006c3a0 2024-09-19T08:41:11.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.506+0000 7fdc770c1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdc700cfa80 con 0x7fdc701636b0 2024-09-19T08:41:11.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.506+0000 7fdc770c1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdc700cfd80 con 0x7fdc70160740 2024-09-19T08:41:11.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.506+0000 7fdc74e5d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc7006c3a0 0x7fdc70152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:11.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.506+0000 7fdc74e5d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc7006c3a0 0x7fdc70152120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:43736/0 (socket says 172.21.15.98:43736) 2024-09-19T08:41:11.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.506+0000 7fdc74e5d640 1 -- 172.21.15.98:0/1105477491 learned_addr learned my addr 172.21.15.98:0/1105477491 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:11.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.506+0000 7fdc6ffff640 1 --2- 172.21.15.98:0/1105477491 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc70160740 0x7fdc7013f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:11.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.506+0000 7fdc7565e640 1 --2- 172.21.15.98:0/1105477491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc701636b0 0x7fdc7015da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:11.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.506+0000 7fdc74e5d640 1 -- 172.21.15.98:0/1105477491 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc70160740 msgr2=0x7fdc7013f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:11.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.506+0000 7fdc74e5d640 1 --2- 172.21.15.98:0/1105477491 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc70160740 0x7fdc7013f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:11.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.506+0000 7fdc74e5d640 1 -- 172.21.15.98:0/1105477491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc701636b0 msgr2=0x7fdc7015da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:11.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.506+0000 7fdc74e5d640 1 --2- 172.21.15.98:0/1105477491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc701636b0 0x7fdc7015da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:11.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.506+0000 7fdc74e5d640 1 -- 172.21.15.98:0/1105477491 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc7013fbd0 con 0x7fdc7006c3a0 2024-09-19T08:41:11.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.510+0000 7fdc74e5d640 1 --2- 172.21.15.98:0/1105477491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc7006c3a0 0x7fdc70152120 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fdc60007f20 tx=0x7fdc6000e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:11.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.510+0000 7fdc6d7fa640 1 -- 172.21.15.98:0/1105477491 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc60014620 con 0x7fdc7006c3a0 2024-09-19T08:41:11.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.510+0000 7fdc6d7fa640 1 -- 172.21.15.98:0/1105477491 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc60019070 con 0x7fdc7006c3a0 2024-09-19T08:41:11.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.510+0000 7fdc6d7fa640 1 -- 172.21.15.98:0/1105477491 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc60014a00 con 0x7fdc7006c3a0 2024-09-19T08:41:11.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.510+0000 7fdc770c1640 1 -- 172.21.15.98:0/1105477491 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc7013feb0 con 0x7fdc7006c3a0 2024-09-19T08:41:11.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.510+0000 7fdc770c1640 1 -- 172.21.15.98:0/1105477491 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc7015d270 con 0x7fdc7006c3a0 2024-09-19T08:41:11.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.510+0000 7fdc770c1640 1 -- 172.21.15.98:0/1105477491 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc700ceb20 con 0x7fdc7006c3a0 2024-09-19T08:41:11.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.510+0000 7fdc6d7fa640 1 -- 172.21.15.98:0/1105477491 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7fdc60014ba0 con 0x7fdc7006c3a0 2024-09-19T08:41:11.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.510+0000 7fdc6d7fa640 1 --2- 172.21.15.98:0/1105477491 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdc50067b00 0x7fdc50069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:11.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.510+0000 7fdc6d7fa640 1 -- 172.21.15.98:0/1105477491 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7fdc60086a10 con 0x7fdc7006c3a0 2024-09-19T08:41:11.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.510+0000 7fdc6ffff640 1 --2- 172.21.15.98:0/1105477491 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdc50067b00 0x7fdc50069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:11.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.510+0000 7fdc6ffff640 1 --2- 172.21.15.98:0/1105477491 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdc50067b00 0x7fdc50069fc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdc7015ef00 tx=0x7fdc58009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:11.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.514+0000 7fdc6d7fa640 1 -- 172.21.15.98:0/1105477491 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fdc60057280 con 0x7fdc7006c3a0 2024-09-19T08:41:11.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.702+0000 7fdc770c1640 1 -- 172.21.15.98:0/1105477491 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc700742b0 con 0x7fdc7006c3a0 2024-09-19T08:41:11.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.702+0000 7fdc6d7fa640 1 -- 172.21.15.98:0/1105477491 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5229 (secure 0 0 0) 0x7fdc60056b80 con 0x7fdc7006c3a0 2024-09-19T08:41:11.704 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:41:11.704 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"btime":"2024-09-19T08:39:55:181532+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/2874086421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":2874086421},{"type":"v1","addr":"172.21.15.139:6833","nonce":2874086421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4996,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/4178563848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":4178563848},{"type":"v1","addr":"172.21.15.139:6835","nonce":4178563848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3584568343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3584568343},{"type":"v1","addr":"172.21.15.98:6835","nonce":3584568343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:39:38.262938+0000","modified":"2024-09-19T08:39:41.276985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/3085042202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3085042202},{"type":"v1","addr":"172.21.15.98:6837","nonce":3085042202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244]},"id":3}]} 2024-09-19T08:41:11.704 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 24 2024-09-19T08:41:11.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.706+0000 7fdc770c1640 1 -- 172.21.15.98:0/1105477491 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdc50067b00 msgr2=0x7fdc50069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:11.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.706+0000 7fdc770c1640 1 --2- 172.21.15.98:0/1105477491 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdc50067b00 0x7fdc50069fc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdc7015ef00 tx=0x7fdc58009040 comp rx=0 tx=0).stop 2024-09-19T08:41:11.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.706+0000 7fdc770c1640 1 -- 172.21.15.98:0/1105477491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc7006c3a0 msgr2=0x7fdc70152120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:11.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.706+0000 7fdc770c1640 1 --2- 172.21.15.98:0/1105477491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc7006c3a0 0x7fdc70152120 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fdc60007f20 tx=0x7fdc6000e570 comp rx=0 tx=0).stop 2024-09-19T08:41:11.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.706+0000 7fdc770c1640 1 -- 172.21.15.98:0/1105477491 shutdown_connections 2024-09-19T08:41:11.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.706+0000 7fdc770c1640 1 --2- 172.21.15.98:0/1105477491 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdc50067b00 0x7fdc50069fc0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:11.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.706+0000 7fdc770c1640 1 --2- 172.21.15.98:0/1105477491 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc701636b0 0x7fdc7015da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:11.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.706+0000 7fdc770c1640 1 --2- 172.21.15.98:0/1105477491 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc70160740 0x7fdc7013f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:11.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.706+0000 7fdc770c1640 1 --2- 172.21.15.98:0/1105477491 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc7006c3a0 0x7fdc70152120 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:11.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.706+0000 7fdc770c1640 1 -- 172.21.15.98:0/1105477491 >> 172.21.15.98:0/1105477491 conn(0x7fdc700b2010 msgr2=0x7fdc701615b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:11.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.706+0000 7fdc770c1640 1 -- 172.21.15.98:0/1105477491 shutdown_connections 2024-09-19T08:41:11.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:11.706+0000 7fdc770c1640 1 -- 172.21.15.98:0/1105477491 wait complete. 2024-09-19T08:41:11.722 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-19T08:41:11.861 INFO:teuthology.orchestra.run.smithi098.stdout:[{"id":5490,"entity":{"name":{"type":"client","num":5490},"addr":{"type":"any","addr":"172.21.15.98:0","nonce":379128374}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":7,"uptime":61.471467451999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.087790943301819,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5490 172.21.15.98:0/379128374","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"y","hostname":"smithi098","pid":"28958","root":"/"}}] 2024-09-19T08:41:11.861 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi098", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28958", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.98:0", "nonce": 379128374, "type": "any" }, "name": { "num": 5490, "type": "client" } }, "id": 5490, "inst": "client.5490 172.21.15.98:0/379128374", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 7, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.08779094330182 }, "state": "open", "uptime": 61.471467452 } ] 2024-09-19T08:41:11.862 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-19T08:41:16.862 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:41:16.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:16.994+0000 7f7be1895640 1 Processor -- start 2024-09-19T08:41:16.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:16.998+0000 7f7be1895640 1 -- start start 2024-09-19T08:41:16.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:16.998+0000 7f7be1895640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7bdc0d4820 0x7f7bdc1621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:16.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:16.998+0000 7f7be1895640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7bdc0b9970 0x7f7bdc0b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:16.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:16.998+0000 7f7be1895640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bdc068440 con 0x7f7bdc0b7f80 2024-09-19T08:41:16.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:16.998+0000 7f7be1895640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f7bdc163050 con 0x7f7bdc0b9970 2024-09-19T08:41:16.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:16.998+0000 7f7be1895640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f7bdc162a90 con 0x7f7bdc0d4820 2024-09-19T08:41:16.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:16.998+0000 7f7bdb7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7bdc0b9970 0x7f7bdc0b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:16.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:16.998+0000 7f7bdb7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7bdc0b9970 0x7f7bdc0b9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:43752/0 (socket says 172.21.15.98:43752) 2024-09-19T08:41:16.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:16.998+0000 7f7bdb7fe640 1 -- 172.21.15.98:0/2539293578 learned_addr learned my addr 172.21.15.98:0/2539293578 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:16.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:16.998+0000 7f7bdb7fe640 1 -- 172.21.15.98:0/2539293578 >> v2:172.21.15.139:3300/0 conn(0x7f7bdc0d4820 msgr2=0x7f7bdc1621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:16.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:16.998+0000 7f7bd3fff640 1 --2- 172.21.15.98:0/2539293578 >> v2:172.21.15.139:3300/0 conn(0x7f7bdc0d4820 0x7f7bdc1621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:16.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:16.998+0000 7f7bdb7fe640 1 --2- 172.21.15.98:0/2539293578 >> v2:172.21.15.139:3300/0 conn(0x7f7bdc0d4820 0x7f7bdc1621b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:16.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:16.998+0000 7f7bdb7fe640 1 -- 172.21.15.98:0/2539293578 >> v1:172.21.15.139:6789/0 conn(0x7f7bdc0b7f80 legacy=0x7f7bdc15f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:16.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:16.998+0000 7f7bdb7fe640 1 -- 172.21.15.98:0/2539293578 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bdc162d70 con 0x7f7bdc0b9970 2024-09-19T08:41:17.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:16.998+0000 7f7bdb7fe640 1 --2- 172.21.15.98:0/2539293578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bdc0b9970 0x7f7bdc0b9de0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc0ceb00 tx=0x7f7bd401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7415d5abbaa8870a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:17.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.002+0000 7f7bda7fc640 1 -- 172.21.15.98:0/2539293578 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd40a57d0 con 0x7f7bdc0b9970 2024-09-19T08:41:17.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.002+0000 7f7bda7fc640 1 -- 172.21.15.98:0/2539293578 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bd40a5970 con 0x7f7bdc0b9970 2024-09-19T08:41:17.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.002+0000 7f7bda7fc640 1 -- 172.21.15.98:0/2539293578 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd40a5d40 con 0x7f7bdc0b9970 2024-09-19T08:41:17.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.002+0000 7f7be1895640 1 -- 172.21.15.98:0/2539293578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bdc0b9970 msgr2=0x7f7bdc0b9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:17.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.002+0000 7f7be1895640 1 --2- 172.21.15.98:0/2539293578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bdc0b9970 0x7f7bdc0b9de0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc0ceb00 tx=0x7f7bd401c9f0 comp rx=0 tx=0).stop 2024-09-19T08:41:17.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.002+0000 7f7be1895640 1 -- 172.21.15.98:0/2539293578 shutdown_connections 2024-09-19T08:41:17.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.002+0000 7f7be1895640 1 --2- 172.21.15.98:0/2539293578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bdc0b9970 0x7f7bdc0b9de0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:17.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.002+0000 7f7be1895640 1 --2- 172.21.15.98:0/2539293578 >> v2:172.21.15.139:3300/0 conn(0x7f7bdc0d4820 0x7f7bdc1621b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:17.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.002+0000 7f7be1895640 1 -- 172.21.15.98:0/2539293578 >> 172.21.15.98:0/2539293578 conn(0x7f7bdc074e40 msgr2=0x7f7bdc075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:17.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.002+0000 7f7be1895640 1 -- 172.21.15.98:0/2539293578 shutdown_connections 2024-09-19T08:41:17.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.002+0000 7f7be1895640 1 -- 172.21.15.98:0/2539293578 wait complete. 2024-09-19T08:41:17.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.002+0000 7f7be1895640 1 Processor -- start 2024-09-19T08:41:17.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.002+0000 7f7be1895640 1 -- start start 2024-09-19T08:41:17.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.002+0000 7f7be1895640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7bdc0b7f80 0x7f7bdc167da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:17.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.002+0000 7f7be1895640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7bdc0b9970 0x7f7bdc13d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:17.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.002+0000 7f7be1895640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bdc0d4820 0x7f7bdc1654e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:17.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.002+0000 7f7be1895640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc0b84e0 con 0x7f7bdc0d4820 2024-09-19T08:41:17.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.002+0000 7f7be1895640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7bdc0b8360 con 0x7f7bdc0b9970 2024-09-19T08:41:17.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.002+0000 7f7be1895640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7bdc144590 con 0x7f7bdc0b7f80 2024-09-19T08:41:17.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.002+0000 7f7bdb7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bdc0d4820 0x7f7bdc1654e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:17.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.002+0000 7f7bdb7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bdc0d4820 0x7f7bdc1654e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:43758/0 (socket says 172.21.15.98:43758) 2024-09-19T08:41:17.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.002+0000 7f7bdb7fe640 1 -- 172.21.15.98:0/1853775505 learned_addr learned my addr 172.21.15.98:0/1853775505 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:17.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.002+0000 7f7bd3fff640 1 --2- 172.21.15.98:0/1853775505 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7bdc0b9970 0x7f7bdc13d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:17.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.002+0000 7f7bdaffd640 1 --2- 172.21.15.98:0/1853775505 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7bdc0b7f80 0x7f7bdc167da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:17.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.002+0000 7f7bdb7fe640 1 -- 172.21.15.98:0/1853775505 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7bdc0b7f80 msgr2=0x7f7bdc167da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:17.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.002+0000 7f7bdb7fe640 1 --2- 172.21.15.98:0/1853775505 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7bdc0b7f80 0x7f7bdc167da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:17.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.002+0000 7f7bdb7fe640 1 -- 172.21.15.98:0/1853775505 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7bdc0b9970 msgr2=0x7f7bdc13d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:17.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.002+0000 7f7bdb7fe640 1 --2- 172.21.15.98:0/1853775505 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7bdc0b9970 0x7f7bdc13d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:17.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.002+0000 7f7bdb7fe640 1 -- 172.21.15.98:0/1853775505 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bdc1684a0 con 0x7f7bdc0d4820 2024-09-19T08:41:17.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.002+0000 7f7bdaffd640 1 --2- 172.21.15.98:0/1853775505 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7bdc0b7f80 0x7f7bdc167da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:17.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.002+0000 7f7bd3fff640 1 --2- 172.21.15.98:0/1853775505 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7bdc0b9970 0x7f7bdc13d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:17.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.002+0000 7f7bdb7fe640 1 --2- 172.21.15.98:0/1853775505 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bdc0d4820 0x7f7bdc1654e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7bd40af040 tx=0x7f7bd4019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:17.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.002+0000 7f7bd8ff9640 1 -- 172.21.15.98:0/1853775505 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd40bc020 con 0x7f7bdc0d4820 2024-09-19T08:41:17.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.002+0000 7f7bd8ff9640 1 -- 172.21.15.98:0/1853775505 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bd40b8670 con 0x7f7bdc0d4820 2024-09-19T08:41:17.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.002+0000 7f7bd8ff9640 1 -- 172.21.15.98:0/1853775505 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd40b8a60 con 0x7f7bdc0d4820 2024-09-19T08:41:17.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.006+0000 7f7be1895640 1 -- 172.21.15.98:0/1853775505 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bdc168780 con 0x7f7bdc0d4820 2024-09-19T08:41:17.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.006+0000 7f7be1895640 1 -- 172.21.15.98:0/1853775505 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bdc13afd0 con 0x7f7bdc0d4820 2024-09-19T08:41:17.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.006+0000 7f7be1895640 1 -- 172.21.15.98:0/1853775505 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bdc068f50 con 0x7f7bdc0d4820 2024-09-19T08:41:17.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.006+0000 7f7bd8ff9640 1 -- 172.21.15.98:0/1853775505 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7f7bd40bf020 con 0x7f7bdc0d4820 2024-09-19T08:41:17.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.006+0000 7f7bd8ff9640 1 --2- 172.21.15.98:0/1853775505 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7bbc067b00 0x7f7bbc069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:17.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.006+0000 7f7bd8ff9640 1 -- 172.21.15.98:0/1853775505 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f7bd41213e0 con 0x7f7bdc0d4820 2024-09-19T08:41:17.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.006+0000 7f7bdaffd640 1 --2- 172.21.15.98:0/1853775505 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7bbc067b00 0x7f7bbc069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:17.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.006+0000 7f7bdaffd640 1 --2- 172.21.15.98:0/1853775505 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7bbc067b00 0x7f7bbc069fc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc15e5a0 tx=0x7f7bcc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:17.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.010+0000 7f7bd8ff9640 1 -- 172.21.15.98:0/1853775505 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f7bd40c6030 con 0x7f7bdc0d4820 2024-09-19T08:41:17.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.206+0000 7f7be1895640 1 -- 172.21.15.98:0/1853775505 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7bdc1448c0 con 0x7f7bdc0d4820 2024-09-19T08:41:17.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.206+0000 7f7bd8ff9640 1 -- 172.21.15.98:0/1853775505 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5229 (secure 0 0 0) 0x7f7bd40165e0 con 0x7f7bdc0d4820 2024-09-19T08:41:17.208 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:41:17.208 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"btime":"2024-09-19T08:39:55:181532+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/2874086421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":2874086421},{"type":"v1","addr":"172.21.15.139:6833","nonce":2874086421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4996,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/4178563848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":4178563848},{"type":"v1","addr":"172.21.15.139:6835","nonce":4178563848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3584568343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3584568343},{"type":"v1","addr":"172.21.15.98:6835","nonce":3584568343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:39:38.262938+0000","modified":"2024-09-19T08:39:41.276985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/3085042202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3085042202},{"type":"v1","addr":"172.21.15.98:6837","nonce":3085042202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244]},"id":3}]} 2024-09-19T08:41:17.208 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 24 2024-09-19T08:41:17.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.210+0000 7f7be1895640 1 -- 172.21.15.98:0/1853775505 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7bbc067b00 msgr2=0x7f7bbc069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:17.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.210+0000 7f7be1895640 1 --2- 172.21.15.98:0/1853775505 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7bbc067b00 0x7f7bbc069fc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc15e5a0 tx=0x7f7bcc009040 comp rx=0 tx=0).stop 2024-09-19T08:41:17.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.210+0000 7f7be1895640 1 -- 172.21.15.98:0/1853775505 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bdc0d4820 msgr2=0x7f7bdc1654e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:17.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.210+0000 7f7be1895640 1 --2- 172.21.15.98:0/1853775505 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bdc0d4820 0x7f7bdc1654e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7bd40af040 tx=0x7f7bd4019790 comp rx=0 tx=0).stop 2024-09-19T08:41:17.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.210+0000 7f7be1895640 1 -- 172.21.15.98:0/1853775505 shutdown_connections 2024-09-19T08:41:17.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.210+0000 7f7be1895640 1 --2- 172.21.15.98:0/1853775505 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7bbc067b00 0x7f7bbc069fc0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:17.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.210+0000 7f7be1895640 1 --2- 172.21.15.98:0/1853775505 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7bdc0d4820 0x7f7bdc1654e0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:17.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.210+0000 7f7be1895640 1 --2- 172.21.15.98:0/1853775505 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7bdc0b9970 0x7f7bdc13d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:17.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.210+0000 7f7be1895640 1 --2- 172.21.15.98:0/1853775505 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7bdc0b7f80 0x7f7bdc167da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:17.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.210+0000 7f7be1895640 1 -- 172.21.15.98:0/1853775505 >> 172.21.15.98:0/1853775505 conn(0x7f7bdc074e40 msgr2=0x7f7bdc067a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:17.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.210+0000 7f7be1895640 1 -- 172.21.15.98:0/1853775505 shutdown_connections 2024-09-19T08:41:17.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:17.210+0000 7f7be1895640 1 -- 172.21.15.98:0/1853775505 wait complete. 2024-09-19T08:41:17.227 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-19T08:41:17.354 INFO:teuthology.orchestra.run.smithi098.stdout:[{"id":5490,"entity":{"name":{"type":"client","num":5490},"addr":{"type":"any","addr":"172.21.15.98:0","nonce":379128374}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":6,"uptime":66.963419625,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.847114196610917,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5490 172.21.15.98:0/379128374","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"y","hostname":"smithi098","pid":"28958","root":"/"}}] 2024-09-19T08:41:17.355 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi098", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28958", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.98:0", "nonce": 379128374, "type": "any" }, "name": { "num": 5490, "type": "client" } }, "id": 5490, "inst": "client.5490 172.21.15.98:0/379128374", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 6, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.847114196610917 }, "state": "open", "uptime": 66.963419625 } ] 2024-09-19T08:41:17.355 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-19T08:41:17.549 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:41:17.552 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:41:22.356 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:41:22.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.486+0000 7fa8fe853640 1 Processor -- start 2024-09-19T08:41:22.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.486+0000 7fa8fe853640 1 -- start start 2024-09-19T08:41:22.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.490+0000 7fa8fe853640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa8f815c550 0x7fa8f806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:22.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.490+0000 7fa8fe853640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa8f806c1d0 0x7fa8f80cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:22.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.490+0000 7fa8fe853640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8f80cf900 0x7fa8f80ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:22.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.490+0000 7fa8fe853640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fa8f806c640 con 0x7fa8f815c550 2024-09-19T08:41:22.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.490+0000 7fa8fe853640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fa8f806d310 con 0x7fa8f806c1d0 2024-09-19T08:41:22.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.490+0000 7fa8fe853640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa8f806d5f0 con 0x7fa8f80cf900 2024-09-19T08:41:22.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.490+0000 7fa8f7fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa8f815c550 0x7fa8f806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:22.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.490+0000 7fa8f7fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa8f815c550 0x7fa8f806bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39258/0 (socket says 172.21.15.98:39258) 2024-09-19T08:41:22.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.490+0000 7fa8f7fff640 1 -- 172.21.15.98:0/2062808541 learned_addr learned my addr 172.21.15.98:0/2062808541 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:22.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.490+0000 7fa8f77fe640 1 --2- 172.21.15.98:0/2062808541 >> v2:172.21.15.139:3300/0 conn(0x7fa8f806c1d0 0x7fa8f80cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:22.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.490+0000 7fa8f7fff640 1 -- 172.21.15.98:0/2062808541 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8f80cf900 msgr2=0x7fa8f80ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:22.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.490+0000 7fa8f7fff640 1 --2- 172.21.15.98:0/2062808541 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8f80cf900 0x7fa8f80ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:22.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.490+0000 7fa8f7fff640 1 -- 172.21.15.98:0/2062808541 >> v2:172.21.15.139:3300/0 conn(0x7fa8f806c1d0 msgr2=0x7fa8f80cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:22.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.490+0000 7fa8f7fff640 1 --2- 172.21.15.98:0/2062808541 >> v2:172.21.15.139:3300/0 conn(0x7fa8f806c1d0 0x7fa8f80cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:22.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.490+0000 7fa8f7fff640 1 -- 172.21.15.98:0/2062808541 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f806d8d0 con 0x7fa8f815c550 2024-09-19T08:41:22.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.490+0000 7fa8f77fe640 1 --2- 172.21.15.98:0/2062808541 >> v2:172.21.15.139:3300/0 conn(0x7fa8f806c1d0 0x7fa8f80cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:22.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.490+0000 7fa8f7fff640 1 --2- 172.21.15.98:0/2062808541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8f815c550 0x7fa8f806bc90 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80ccc60 tx=0x7fa8e801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827b44a1cbc2cc91 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:22.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.490+0000 7fa8f6ffd640 1 -- 172.21.15.98:0/2062808541 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e80b0070 con 0x7fa8f815c550 2024-09-19T08:41:22.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.490+0000 7fa8f6ffd640 1 -- 172.21.15.98:0/2062808541 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8e80ab600 con 0x7fa8f815c550 2024-09-19T08:41:22.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.490+0000 7fa8f6ffd640 1 -- 172.21.15.98:0/2062808541 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e80ab9d0 con 0x7fa8f815c550 2024-09-19T08:41:22.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.494+0000 7fa8fe853640 1 -- 172.21.15.98:0/2062808541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8f815c550 msgr2=0x7fa8f806bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:22.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.494+0000 7fa8fe853640 1 --2- 172.21.15.98:0/2062808541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8f815c550 0x7fa8f806bc90 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80ccc60 tx=0x7fa8e801c260 comp rx=0 tx=0).stop 2024-09-19T08:41:22.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.494+0000 7fa8fe853640 1 -- 172.21.15.98:0/2062808541 shutdown_connections 2024-09-19T08:41:22.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.494+0000 7fa8fe853640 1 --2- 172.21.15.98:0/2062808541 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8f80cf900 0x7fa8f80ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:22.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.494+0000 7fa8fe853640 1 --2- 172.21.15.98:0/2062808541 >> v2:172.21.15.139:3300/0 conn(0x7fa8f806c1d0 0x7fa8f80cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:22.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.494+0000 7fa8fe853640 1 --2- 172.21.15.98:0/2062808541 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8f815c550 0x7fa8f806bc90 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:22.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.494+0000 7fa8fe853640 1 -- 172.21.15.98:0/2062808541 >> 172.21.15.98:0/2062808541 conn(0x7fa8f80b2010 msgr2=0x7fa8f80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:22.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.494+0000 7fa8fe853640 1 -- 172.21.15.98:0/2062808541 shutdown_connections 2024-09-19T08:41:22.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.494+0000 7fa8fe853640 1 -- 172.21.15.98:0/2062808541 wait complete. 2024-09-19T08:41:22.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.494+0000 7fa8fe853640 1 Processor -- start 2024-09-19T08:41:22.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.494+0000 7fa8fe853640 1 -- start start 2024-09-19T08:41:22.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.494+0000 7fa8fe853640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8f806c1d0 0x7fa8f8156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:22.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.494+0000 7fa8fe853640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8f80cf900 0x7fa8f80abc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:22.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.494+0000 7fa8fe853640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8f815c550 0x7fa8f80aa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:22.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.494+0000 7fa8fe853640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f8072520 con 0x7fa8f806c1d0 2024-09-19T08:41:22.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.494+0000 7fa8fe853640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f80723a0 con 0x7fa8f80cf900 2024-09-19T08:41:22.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.494+0000 7fa8fe853640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa8f80726a0 con 0x7fa8f815c550 2024-09-19T08:41:22.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.494+0000 7fa8f7fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8f806c1d0 0x7fa8f8156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:22.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.494+0000 7fa8f7fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8f806c1d0 0x7fa8f8156830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39268/0 (socket says 172.21.15.98:39268) 2024-09-19T08:41:22.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.494+0000 7fa8f7fff640 1 -- 172.21.15.98:0/4112198352 learned_addr learned my addr 172.21.15.98:0/4112198352 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:22.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.494+0000 7fa8f7fff640 1 -- 172.21.15.98:0/4112198352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8f815c550 msgr2=0x7fa8f80aa4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:22.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.494+0000 7fa8f7fff640 1 --2- 172.21.15.98:0/4112198352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8f815c550 0x7fa8f80aa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:22.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.494+0000 7fa8f7fff640 1 -- 172.21.15.98:0/4112198352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8f80cf900 msgr2=0x7fa8f80abc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:22.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.494+0000 7fa8f7fff640 1 --2- 172.21.15.98:0/4112198352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8f80cf900 0x7fa8f80abc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:22.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.494+0000 7fa8f7fff640 1 -- 172.21.15.98:0/4112198352 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f80aca60 con 0x7fa8f806c1d0 2024-09-19T08:41:22.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.494+0000 7fa8f7fff640 1 --2- 172.21.15.98:0/4112198352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8f806c1d0 0x7fa8f8156830 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa8e80ae040 tx=0x7fa8e8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:22.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.494+0000 7fa8f4ff9640 1 -- 172.21.15.98:0/4112198352 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e80b0040 con 0x7fa8f806c1d0 2024-09-19T08:41:22.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.494+0000 7fa8f4ff9640 1 -- 172.21.15.98:0/4112198352 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8e80b9070 con 0x7fa8f806c1d0 2024-09-19T08:41:22.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.494+0000 7fa8f4ff9640 1 -- 172.21.15.98:0/4112198352 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e80b48a0 con 0x7fa8f806c1d0 2024-09-19T08:41:22.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.494+0000 7fa8fe853640 1 -- 172.21.15.98:0/4112198352 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8f80acd40 con 0x7fa8f806c1d0 2024-09-19T08:41:22.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.494+0000 7fa8fe853640 1 -- 172.21.15.98:0/4112198352 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8f80ad300 con 0x7fa8f806c1d0 2024-09-19T08:41:22.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.498+0000 7fa8fe853640 1 -- 172.21.15.98:0/4112198352 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8f80ce1a0 con 0x7fa8f806c1d0 2024-09-19T08:41:22.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.498+0000 7fa8f4ff9640 1 -- 172.21.15.98:0/4112198352 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7fa8e80ab4c0 con 0x7fa8f806c1d0 2024-09-19T08:41:22.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.498+0000 7fa8f4ff9640 1 --2- 172.21.15.98:0/4112198352 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa8d8067b00 0x7fa8d8069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:22.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.498+0000 7fa8f4ff9640 1 -- 172.21.15.98:0/4112198352 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7fa8e811feb0 con 0x7fa8f806c1d0 2024-09-19T08:41:22.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.498+0000 7fa8f77fe640 1 --2- 172.21.15.98:0/4112198352 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa8d8067b00 0x7fa8d8069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:22.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.498+0000 7fa8f77fe640 1 --2- 172.21.15.98:0/4112198352 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa8d8067b00 0x7fa8d8069fc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8067f20 tx=0x7fa8e00083d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:22.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.502+0000 7fa8f4ff9640 1 -- 172.21.15.98:0/4112198352 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa8e8016110 con 0x7fa8f806c1d0 2024-09-19T08:41:22.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.694+0000 7fa8fe853640 1 -- 172.21.15.98:0/4112198352 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8f8156310 con 0x7fa8f806c1d0 2024-09-19T08:41:22.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.694+0000 7fa8f4ff9640 1 -- 172.21.15.98:0/4112198352 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5229 (secure 0 0 0) 0x7fa8e80f06a0 con 0x7fa8f806c1d0 2024-09-19T08:41:22.694 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:41:22.694 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"btime":"2024-09-19T08:39:55:181532+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/2874086421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":2874086421},{"type":"v1","addr":"172.21.15.139:6833","nonce":2874086421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4996,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/4178563848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":4178563848},{"type":"v1","addr":"172.21.15.139:6835","nonce":4178563848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3584568343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3584568343},{"type":"v1","addr":"172.21.15.98:6835","nonce":3584568343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:39:38.262938+0000","modified":"2024-09-19T08:39:41.276985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/3085042202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3085042202},{"type":"v1","addr":"172.21.15.98:6837","nonce":3085042202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244]},"id":3}]} 2024-09-19T08:41:22.694 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 24 2024-09-19T08:41:22.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.694+0000 7fa8fe853640 1 -- 172.21.15.98:0/4112198352 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa8d8067b00 msgr2=0x7fa8d8069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:22.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.694+0000 7fa8fe853640 1 --2- 172.21.15.98:0/4112198352 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa8d8067b00 0x7fa8d8069fc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8067f20 tx=0x7fa8e00083d0 comp rx=0 tx=0).stop 2024-09-19T08:41:22.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.694+0000 7fa8fe853640 1 -- 172.21.15.98:0/4112198352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8f806c1d0 msgr2=0x7fa8f8156830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:22.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.694+0000 7fa8fe853640 1 --2- 172.21.15.98:0/4112198352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8f806c1d0 0x7fa8f8156830 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa8e80ae040 tx=0x7fa8e8019000 comp rx=0 tx=0).stop 2024-09-19T08:41:22.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.694+0000 7fa8fe853640 1 -- 172.21.15.98:0/4112198352 shutdown_connections 2024-09-19T08:41:22.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.694+0000 7fa8fe853640 1 --2- 172.21.15.98:0/4112198352 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa8d8067b00 0x7fa8d8069fc0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:22.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.694+0000 7fa8fe853640 1 --2- 172.21.15.98:0/4112198352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8f815c550 0x7fa8f80aa4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:22.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.694+0000 7fa8fe853640 1 --2- 172.21.15.98:0/4112198352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8f80cf900 0x7fa8f80abc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:22.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.694+0000 7fa8fe853640 1 --2- 172.21.15.98:0/4112198352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8f806c1d0 0x7fa8f8156830 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:22.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.694+0000 7fa8fe853640 1 -- 172.21.15.98:0/4112198352 >> 172.21.15.98:0/4112198352 conn(0x7fa8f80b2010 msgr2=0x7fa8f80cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:22.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.694+0000 7fa8fe853640 1 -- 172.21.15.98:0/4112198352 shutdown_connections 2024-09-19T08:41:22.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:22.694+0000 7fa8fe853640 1 -- 172.21.15.98:0/4112198352 wait complete. 2024-09-19T08:41:22.711 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-19T08:41:22.851 INFO:teuthology.orchestra.run.smithi098.stdout:[{"id":5490,"entity":{"name":{"type":"client","num":5490},"addr":{"type":"any","addr":"172.21.15.98:0","nonce":379128374}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":6,"uptime":72.459371724999997,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.609300143662573,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5490 172.21.15.98:0/379128374","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"y","hostname":"smithi098","pid":"28958","root":"/"}}] 2024-09-19T08:41:22.852 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi098", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28958", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.98:0", "nonce": 379128374, "type": "any" }, "name": { "num": 5490, "type": "client" } }, "id": 5490, "inst": "client.5490 172.21.15.98:0/379128374", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 6, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.609300143662573 }, "state": "open", "uptime": 72.459371725 } ] 2024-09-19T08:41:22.852 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-19T08:41:27.853 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:41:27.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.982+0000 7f3915c15640 1 Processor -- start 2024-09-19T08:41:27.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.986+0000 7f3915c15640 1 -- start start 2024-09-19T08:41:27.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.986+0000 7f3915c15640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f391006bc90 0x7f391006c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:27.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.986+0000 7f3915c15640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39100ccc60 con 0x7f3910158fd0 2024-09-19T08:41:27.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.986+0000 7f3915c15640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39100ccf40 con 0x7f39100ce840 2024-09-19T08:41:27.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.986+0000 7f3915c15640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f39101593b0 con 0x7f391006bc90 2024-09-19T08:41:27.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.986+0000 7f390f7fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f3910158fd0 0x7f39100b3e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:58270/0 (socket says 172.21.15.98:58270) 2024-09-19T08:41:27.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.986+0000 7f390f7fe640 1 -- 172.21.15.98:0/1538012479 learned_addr learned my addr 172.21.15.98:0/1538012479 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:41:27.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.986+0000 7f390e7fc640 1 -- 172.21.15.98:0/1538012479 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3195736640 0 0) 0x7f39100ccc60 con 0x7f3910158fd0 2024-09-19T08:41:27.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.986+0000 7f390e7fc640 1 -- 172.21.15.98:0/1538012479 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f38fc003530 con 0x7f3910158fd0 2024-09-19T08:41:27.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.986+0000 7f390effd640 1 --2- 172.21.15.98:0/1538012479 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f391006bc90 0x7f391006c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:27.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.986+0000 7f390e7fc640 1 -- 172.21.15.98:0/1538012479 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2548899498 0 0) 0x7f38fc003530 con 0x7f3910158fd0 2024-09-19T08:41:27.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.986+0000 7f390e7fc640 1 -- 172.21.15.98:0/1538012479 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f391006bc90 msgr2=0x7f391006c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:27.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.986+0000 7f390e7fc640 1 --2- 172.21.15.98:0/1538012479 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f391006bc90 0x7f391006c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:27.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.986+0000 7f390e7fc640 1 -- 172.21.15.98:0/1538012479 >> v1:172.21.15.139:6789/0 conn(0x7f39100ce840 legacy=0x7f39100cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:27.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.986+0000 7f390e7fc640 1 -- 172.21.15.98:0/1538012479 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39100cd220 con 0x7f3910158fd0 2024-09-19T08:41:27.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.986+0000 7f390e7fc640 1 -- 172.21.15.98:0/1538012479 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f3900001680 con 0x7f3910158fd0 2024-09-19T08:41:27.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.986+0000 7f390e7fc640 1 -- 172.21.15.98:0/1538012479 >> v1:172.21.15.98:6789/0 conn(0x7f3910158fd0 legacy=0x7f39100b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:27.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.986+0000 7f390e7fc640 1 --2- 172.21.15.98:0/1538012479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38fc0039f0 0x7f38fc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:27.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.986+0000 7f390e7fc640 1 --2- 172.21.15.98:0/1538012479 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f391006bc90 0x7f38fc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:27.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.986+0000 7f390e7fc640 1 --2- 172.21.15.98:0/1538012479 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38fc004d50 0x7f38fc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:27.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.986+0000 7f390e7fc640 1 -- 172.21.15.98:0/1538012479 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f39101593b0 con 0x7f38fc004d50 2024-09-19T08:41:27.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.986+0000 7f390e7fc640 1 -- 172.21.15.98:0/1538012479 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc00ddf0 con 0x7f38fc0039f0 2024-09-19T08:41:27.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.986+0000 7f390e7fc640 1 -- 172.21.15.98:0/1538012479 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f38fc00dff0 con 0x7f391006bc90 2024-09-19T08:41:27.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.986+0000 7f390ffff640 1 --2- 172.21.15.98:0/1538012479 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38fc004d50 0x7f38fc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:27.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.986+0000 7f390f7fe640 1 --2- 172.21.15.98:0/1538012479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38fc0039f0 0x7f38fc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:27.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.986+0000 7f390effd640 1 --2- 172.21.15.98:0/1538012479 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f391006bc90 0x7f38fc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:27.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.986+0000 7f390ffff640 1 -- 172.21.15.98:0/1538012479 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f391006bc90 msgr2=0x7f38fc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:27.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.986+0000 7f390ffff640 1 --2- 172.21.15.98:0/1538012479 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f391006bc90 0x7f38fc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:27.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.986+0000 7f390ffff640 1 -- 172.21.15.98:0/1538012479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38fc0039f0 msgr2=0x7f38fc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:27.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.986+0000 7f390ffff640 1 --2- 172.21.15.98:0/1538012479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38fc0039f0 0x7f38fc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:27.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.986+0000 7f390ffff640 1 -- 172.21.15.98:0/1538012479 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38fc003530 con 0x7f38fc004d50 2024-09-19T08:41:27.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.986+0000 7f390f7fe640 1 --2- 172.21.15.98:0/1538012479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38fc0039f0 0x7f38fc003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:41:27.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.990+0000 7f390ffff640 1 --2- 172.21.15.98:0/1538012479 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38fc004d50 0x7f38fc0051d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f3908001e70 tx=0x7f390801c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a7d9f7730eafe57 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:27.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.990+0000 7f390e7fc640 1 -- 172.21.15.98:0/1538012479 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3908001a90 con 0x7f38fc004d50 2024-09-19T08:41:27.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.990+0000 7f390e7fc640 1 -- 172.21.15.98:0/1538012479 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3908003b80 con 0x7f38fc004d50 2024-09-19T08:41:27.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.990+0000 7f390e7fc640 1 -- 172.21.15.98:0/1538012479 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3908005080 con 0x7f38fc004d50 2024-09-19T08:41:27.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.990+0000 7f3915c15640 1 -- 172.21.15.98:0/1538012479 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38fc004d50 msgr2=0x7f38fc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:27.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.990+0000 7f3915c15640 1 --2- 172.21.15.98:0/1538012479 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38fc004d50 0x7f38fc0051d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f3908001e70 tx=0x7f390801c780 comp rx=0 tx=0).stop 2024-09-19T08:41:27.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.990+0000 7f390ffff640 1 -- 172.21.15.98:0/1538012479 reap_dead start 2024-09-19T08:41:27.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.990+0000 7f3915c15640 1 -- 172.21.15.98:0/1538012479 shutdown_connections 2024-09-19T08:41:27.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.990+0000 7f3915c15640 1 -- 172.21.15.98:0/1538012479 >> 172.21.15.98:0/1538012479 conn(0x7f39100b2010 msgr2=0x7f39100b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:27.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.990+0000 7f3915c15640 1 -- 172.21.15.98:0/1538012479 shutdown_connections 2024-09-19T08:41:27.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.990+0000 7f3915c15640 1 -- 172.21.15.98:0/1538012479 wait complete. 2024-09-19T08:41:27.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.990+0000 7f3915c15640 1 Processor -- start 2024-09-19T08:41:27.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.990+0000 7f3915c15640 1 -- start start 2024-09-19T08:41:27.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.990+0000 7f3915c15640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38fc004d50 0x7f3910072100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:27.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.990+0000 7f3915c15640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39101545f0 0x7f3910151f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:27.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.990+0000 7f3915c15640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3910072950 0x7f39100b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:27.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.990+0000 7f3915c15640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f39100b4530 con 0x7f38fc004d50 2024-09-19T08:41:27.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.990+0000 7f3915c15640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f39100cd7e0 con 0x7f39101545f0 2024-09-19T08:41:27.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.990+0000 7f3915c15640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f391015c7a0 con 0x7f3910072950 2024-09-19T08:41:27.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.990+0000 7f390f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38fc004d50 0x7f3910072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:27.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.990+0000 7f390f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38fc004d50 0x7f3910072100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37586/0 (socket says 172.21.15.98:37586) 2024-09-19T08:41:27.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.990+0000 7f390f7fe640 1 -- 172.21.15.98:0/2326864255 learned_addr learned my addr 172.21.15.98:0/2326864255 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:27.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.990+0000 7f390effd640 1 --2- 172.21.15.98:0/2326864255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39101545f0 0x7f3910151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:27.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.994+0000 7f390ffff640 1 --2- 172.21.15.98:0/2326864255 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3910072950 0x7f39100b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:27.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.994+0000 7f390f7fe640 1 -- 172.21.15.98:0/2326864255 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3910072950 msgr2=0x7f39100b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:27.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.994+0000 7f390f7fe640 1 --2- 172.21.15.98:0/2326864255 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3910072950 0x7f39100b10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:27.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.994+0000 7f390f7fe640 1 -- 172.21.15.98:0/2326864255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39101545f0 msgr2=0x7f3910151f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:27.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.994+0000 7f390f7fe640 1 --2- 172.21.15.98:0/2326864255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39101545f0 0x7f3910151f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:27.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.994+0000 7f390f7fe640 1 -- 172.21.15.98:0/2326864255 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3910071490 con 0x7f38fc004d50 2024-09-19T08:41:27.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.994+0000 7f390f7fe640 1 --2- 172.21.15.98:0/2326864255 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38fc004d50 0x7f3910072100 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f39000058b0 tx=0x7f3900011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:27.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.994+0000 7f3907fff640 1 -- 172.21.15.98:0/2326864255 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3900010040 con 0x7f38fc004d50 2024-09-19T08:41:27.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.994+0000 7f3907fff640 1 -- 172.21.15.98:0/2326864255 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f390001a070 con 0x7f38fc004d50 2024-09-19T08:41:27.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.994+0000 7f3915c15640 1 -- 172.21.15.98:0/2326864255 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39101f4470 con 0x7f38fc004d50 2024-09-19T08:41:27.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.994+0000 7f3907fff640 1 -- 172.21.15.98:0/2326864255 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390000fd60 con 0x7f38fc004d50 2024-09-19T08:41:27.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.994+0000 7f3915c15640 1 -- 172.21.15.98:0/2326864255 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39101f4a30 con 0x7f38fc004d50 2024-09-19T08:41:27.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.994+0000 7f3915c15640 1 -- 172.21.15.98:0/2326864255 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f391006c120 con 0x7f38fc004d50 2024-09-19T08:41:27.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.994+0000 7f3907fff640 1 -- 172.21.15.98:0/2326864255 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 8) ==== 87649+0+0 (secure 0 0 0) 0x7f390001f460 con 0x7f38fc004d50 2024-09-19T08:41:27.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.994+0000 7f3907fff640 1 --2- 172.21.15.98:0/2326864255 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f38f0067b00 0x7f38f0069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:27.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.994+0000 7f3907fff640 1 -- 172.21.15.98:0/2326864255 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f3900089230 con 0x7f38fc004d50 2024-09-19T08:41:27.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.994+0000 7f390effd640 1 --2- 172.21.15.98:0/2326864255 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f38f0067b00 0x7f38f0069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:27.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.994+0000 7f390effd640 1 --2- 172.21.15.98:0/2326864255 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f38f0067b00 0x7f38f0069fc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f391006d0c0 tx=0x7f38f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:27.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:27.998+0000 7f3907fff640 1 -- 172.21.15.98:0/2326864255 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3900059a60 con 0x7f38fc004d50 2024-09-19T08:41:28.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:28.198+0000 7f3915c15640 1 -- 172.21.15.98:0/2326864255 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f391015c170 con 0x7f38fc004d50 2024-09-19T08:41:28.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:28.198+0000 7f3907fff640 1 -- 172.21.15.98:0/2326864255 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5229 (secure 0 0 0) 0x7f3900059360 con 0x7f38fc004d50 2024-09-19T08:41:28.200 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:41:28.200 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"btime":"2024-09-19T08:39:55:181532+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/2874086421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":2874086421},{"type":"v1","addr":"172.21.15.139:6833","nonce":2874086421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4996,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/4178563848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":4178563848},{"type":"v1","addr":"172.21.15.139:6835","nonce":4178563848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3584568343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3584568343},{"type":"v1","addr":"172.21.15.98:6835","nonce":3584568343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:39:38.262938+0000","modified":"2024-09-19T08:39:41.276985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/3085042202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3085042202},{"type":"v1","addr":"172.21.15.98:6837","nonce":3085042202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244]},"id":3}]} 2024-09-19T08:41:28.200 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 24 2024-09-19T08:41:28.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:28.202+0000 7f3915c15640 1 -- 172.21.15.98:0/2326864255 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f38f0067b00 msgr2=0x7f38f0069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:28.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:28.202+0000 7f3915c15640 1 --2- 172.21.15.98:0/2326864255 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f38f0067b00 0x7f38f0069fc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f391006d0c0 tx=0x7f38f8008040 comp rx=0 tx=0).stop 2024-09-19T08:41:28.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:28.202+0000 7f3915c15640 1 -- 172.21.15.98:0/2326864255 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38fc004d50 msgr2=0x7f3910072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:28.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:28.202+0000 7f3915c15640 1 --2- 172.21.15.98:0/2326864255 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38fc004d50 0x7f3910072100 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f39000058b0 tx=0x7f3900011d20 comp rx=0 tx=0).stop 2024-09-19T08:41:28.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:28.202+0000 7f3915c15640 1 -- 172.21.15.98:0/2326864255 shutdown_connections 2024-09-19T08:41:28.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:28.202+0000 7f3915c15640 1 --2- 172.21.15.98:0/2326864255 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f38f0067b00 0x7f38f0069fc0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:28.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:28.202+0000 7f3915c15640 1 --2- 172.21.15.98:0/2326864255 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3910072950 0x7f39100b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:28.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:28.202+0000 7f3915c15640 1 --2- 172.21.15.98:0/2326864255 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39101545f0 0x7f3910151f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:28.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:28.202+0000 7f3915c15640 1 --2- 172.21.15.98:0/2326864255 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f38fc004d50 0x7f3910072100 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:28.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:28.202+0000 7f3915c15640 1 -- 172.21.15.98:0/2326864255 >> 172.21.15.98:0/2326864255 conn(0x7f39100b2010 msgr2=0x7f39100b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:28.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:28.202+0000 7f3915c15640 1 -- 172.21.15.98:0/2326864255 shutdown_connections 2024-09-19T08:41:28.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:28.202+0000 7f3915c15640 1 -- 172.21.15.98:0/2326864255 wait complete. 2024-09-19T08:41:28.219 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-19T08:41:28.359 INFO:teuthology.orchestra.run.smithi098.stdout:[{"id":5490,"entity":{"name":{"type":"client","num":5490},"addr":{"type":"any","addr":"172.21.15.98:0","nonce":379128374}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":6,"uptime":77.967323687000004,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.373977407118101,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5490 172.21.15.98:0/379128374","completed_requests":[{"tid":14,"created_ino":"0x0"},{"tid":15,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"y","hostname":"smithi098","pid":"28958","root":"/"}}] 2024-09-19T08:41:28.359 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi098", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28958", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 14 }, { "created_ino": "0x100000001f9", "tid": 15 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.98:0", "nonce": 379128374, "type": "any" }, "name": { "num": 5490, "type": "client" } }, "id": 5490, "inst": "client.5490 172.21.15.98:0/379128374", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 6, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.3739774071181 }, "state": "open", "uptime": 77.967323687 } ] 2024-09-19T08:41:28.360 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-19T08:41:33.361 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:41:33.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.490+0000 7fabf483d640 1 Processor -- start 2024-09-19T08:41:33.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.490+0000 7fabf483d640 1 -- start start 2024-09-19T08:41:33.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.494+0000 7fabf483d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fabf00ce8b0 0x7fabf01693e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:33.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.494+0000 7fabf483d640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabf015fcf0 con 0x7fabf006c3a0 2024-09-19T08:41:33.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.494+0000 7fabf483d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabf015ef30 con 0x7fabf0158e40 2024-09-19T08:41:33.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.494+0000 7fabf483d640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fabf015f210 con 0x7fabf00ce8b0 2024-09-19T08:41:33.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.494+0000 7fabeed9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fabf00ce8b0 0x7fabf01693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:33.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.494+0000 7fabeed9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fabf00ce8b0 0x7fabf01693e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37596/0 (socket says 172.21.15.98:37596) 2024-09-19T08:41:33.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.494+0000 7fabeed9d640 1 -- 172.21.15.98:0/2236498720 learned_addr learned my addr 172.21.15.98:0/2236498720 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:33.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.494+0000 7fabeed9d640 1 -- 172.21.15.98:0/2236498720 >> v1:172.21.15.139:6789/0 conn(0x7fabf0158e40 legacy=0x7fabf006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:33.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.494+0000 7fabeed9d640 1 -- 172.21.15.98:0/2236498720 >> v1:172.21.15.98:6789/0 conn(0x7fabf006c3a0 legacy=0x7fabf00ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:33.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.494+0000 7fabeed9d640 1 -- 172.21.15.98:0/2236498720 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf015f4f0 con 0x7fabf00ce8b0 2024-09-19T08:41:33.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.494+0000 7fabeed9d640 1 --2- 172.21.15.98:0/2236498720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fabf00ce8b0 0x7fabf01693e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fabe8009e50 tx=0x7fabe801f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d70086cd6b258cee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:33.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.494+0000 7fabedd9b640 1 -- 172.21.15.98:0/2236498720 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe80b4070 con 0x7fabf00ce8b0 2024-09-19T08:41:33.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.494+0000 7fabedd9b640 1 -- 172.21.15.98:0/2236498720 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabe80af400 con 0x7fabf00ce8b0 2024-09-19T08:41:33.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.494+0000 7fabedd9b640 1 -- 172.21.15.98:0/2236498720 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe80af7d0 con 0x7fabf00ce8b0 2024-09-19T08:41:33.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.498+0000 7fabf483d640 1 -- 172.21.15.98:0/2236498720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fabf00ce8b0 msgr2=0x7fabf01693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:33.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.498+0000 7fabf483d640 1 --2- 172.21.15.98:0/2236498720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fabf00ce8b0 0x7fabf01693e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fabe8009e50 tx=0x7fabe801f1e0 comp rx=0 tx=0).stop 2024-09-19T08:41:33.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.498+0000 7fabf483d640 1 -- 172.21.15.98:0/2236498720 shutdown_connections 2024-09-19T08:41:33.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.498+0000 7fabf483d640 1 --2- 172.21.15.98:0/2236498720 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fabf00ce8b0 0x7fabf01693e0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:33.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.498+0000 7fabf483d640 1 -- 172.21.15.98:0/2236498720 >> 172.21.15.98:0/2236498720 conn(0x7fabf00b2010 msgr2=0x7fabf00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:33.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.498+0000 7fabf483d640 1 -- 172.21.15.98:0/2236498720 shutdown_connections 2024-09-19T08:41:33.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.498+0000 7fabf483d640 1 -- 172.21.15.98:0/2236498720 wait complete. 2024-09-19T08:41:33.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.498+0000 7fabf483d640 1 Processor -- start 2024-09-19T08:41:33.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.498+0000 7fabf483d640 1 -- start start 2024-09-19T08:41:33.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.498+0000 7fabf483d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fabf006c3a0 0x7fabf0149da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:33.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.498+0000 7fabf483d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabf00ce8b0 0x7fabf015af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:33.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.498+0000 7fabf483d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fabf0158e40 0x7fabf0072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:33.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.498+0000 7fabf483d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fabf00cee20 con 0x7fabf006c3a0 2024-09-19T08:41:33.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.498+0000 7fabf483d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fabf0159220 con 0x7fabf00ce8b0 2024-09-19T08:41:33.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.498+0000 7fabf483d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fabf00b4570 con 0x7fabf0158e40 2024-09-19T08:41:33.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.498+0000 7fabee59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fabf006c3a0 0x7fabf0149da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:33.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.498+0000 7fabee59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fabf006c3a0 0x7fabf0149da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37602/0 (socket says 172.21.15.98:37602) 2024-09-19T08:41:33.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.498+0000 7fabee59c640 1 -- 172.21.15.98:0/1390631897 learned_addr learned my addr 172.21.15.98:0/1390631897 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:33.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.498+0000 7fabee59c640 1 -- 172.21.15.98:0/1390631897 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fabf0158e40 msgr2=0x7fabf0072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:33.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.498+0000 7fabee59c640 1 --2- 172.21.15.98:0/1390631897 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fabf0158e40 0x7fabf0072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:33.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.498+0000 7fabee59c640 1 -- 172.21.15.98:0/1390631897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabf00ce8b0 msgr2=0x7fabf015af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:33.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.498+0000 7fabe5d9b640 1 --2- 172.21.15.98:0/1390631897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabf00ce8b0 0x7fabf015af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:33.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.498+0000 7fabee59c640 1 --2- 172.21.15.98:0/1390631897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabf00ce8b0 0x7fabf015af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:33.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.498+0000 7fabee59c640 1 -- 172.21.15.98:0/1390631897 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf01519b0 con 0x7fabf006c3a0 2024-09-19T08:41:33.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.498+0000 7fabee59c640 1 --2- 172.21.15.98:0/1390631897 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fabf006c3a0 0x7fabf0149da0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fabe000c450 tx=0x7fabe000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:33.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.498+0000 7fabe77fe640 1 -- 172.21.15.98:0/1390631897 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe0012830 con 0x7fabf006c3a0 2024-09-19T08:41:33.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.498+0000 7fabe77fe640 1 -- 172.21.15.98:0/1390631897 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabe0017070 con 0x7fabf006c3a0 2024-09-19T08:41:33.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.498+0000 7fabf483d640 1 -- 172.21.15.98:0/1390631897 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabf014a2e0 con 0x7fabf006c3a0 2024-09-19T08:41:33.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.498+0000 7fabe77fe640 1 -- 172.21.15.98:0/1390631897 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe0012c30 con 0x7fabf006c3a0 2024-09-19T08:41:33.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.498+0000 7fabf483d640 1 -- 172.21.15.98:0/1390631897 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabf015bd00 con 0x7fabf006c3a0 2024-09-19T08:41:33.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.502+0000 7fabf483d640 1 -- 172.21.15.98:0/1390631897 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabf00ce1a0 con 0x7fabf006c3a0 2024-09-19T08:41:33.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.502+0000 7fabe77fe640 1 -- 172.21.15.98:0/1390631897 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fabe001c460 con 0x7fabf006c3a0 2024-09-19T08:41:33.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.502+0000 7fabe77fe640 1 --2- 172.21.15.98:0/1390631897 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fabd0067a70 0x7fabd0069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:33.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.502+0000 7fabe5d9b640 1 --2- 172.21.15.98:0/1390631897 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fabd0067a70 0x7fabd0069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:33.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.502+0000 7fabe77fe640 1 -- 172.21.15.98:0/1390631897 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7fabe0085d60 con 0x7fabf006c3a0 2024-09-19T08:41:33.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.502+0000 7fabe5d9b640 1 --2- 172.21.15.98:0/1390631897 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fabd0067a70 0x7fabd0069f30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fabf0151760 tx=0x7fabd8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:33.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.510+0000 7fabe77fe640 1 -- 172.21.15.98:0/1390631897 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fabe00565e0 con 0x7fabf006c3a0 2024-09-19T08:41:33.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.702+0000 7fabf483d640 1 -- 172.21.15.98:0/1390631897 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fabf00b0a10 con 0x7fabf006c3a0 2024-09-19T08:41:33.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.702+0000 7fabe77fe640 1 -- 172.21.15.98:0/1390631897 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5229 (secure 0 0 0) 0x7fabe0055ee0 con 0x7fabf006c3a0 2024-09-19T08:41:33.702 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:41:33.702 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"btime":"2024-09-19T08:39:55:181532+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/2874086421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":2874086421},{"type":"v1","addr":"172.21.15.139:6833","nonce":2874086421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4996,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/4178563848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":4178563848},{"type":"v1","addr":"172.21.15.139:6835","nonce":4178563848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3584568343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3584568343},{"type":"v1","addr":"172.21.15.98:6835","nonce":3584568343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:39:38.262938+0000","modified":"2024-09-19T08:39:41.276985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/3085042202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3085042202},{"type":"v1","addr":"172.21.15.98:6837","nonce":3085042202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244]},"id":3}]} 2024-09-19T08:41:33.702 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 24 2024-09-19T08:41:33.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.702+0000 7fabf483d640 1 -- 172.21.15.98:0/1390631897 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fabd0067a70 msgr2=0x7fabd0069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:33.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.702+0000 7fabf483d640 1 --2- 172.21.15.98:0/1390631897 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fabd0067a70 0x7fabd0069f30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fabf0151760 tx=0x7fabd8009040 comp rx=0 tx=0).stop 2024-09-19T08:41:33.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.702+0000 7fabf483d640 1 -- 172.21.15.98:0/1390631897 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fabf006c3a0 msgr2=0x7fabf0149da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:33.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.702+0000 7fabf483d640 1 --2- 172.21.15.98:0/1390631897 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fabf006c3a0 0x7fabf0149da0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fabe000c450 tx=0x7fabe000c910 comp rx=0 tx=0).stop 2024-09-19T08:41:33.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.702+0000 7fabf483d640 1 -- 172.21.15.98:0/1390631897 shutdown_connections 2024-09-19T08:41:33.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.702+0000 7fabf483d640 1 --2- 172.21.15.98:0/1390631897 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fabd0067a70 0x7fabd0069f30 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:33.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.702+0000 7fabf483d640 1 --2- 172.21.15.98:0/1390631897 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fabf0158e40 0x7fabf0072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:33.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.702+0000 7fabf483d640 1 --2- 172.21.15.98:0/1390631897 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fabf00ce8b0 0x7fabf015af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:33.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.702+0000 7fabf483d640 1 --2- 172.21.15.98:0/1390631897 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fabf006c3a0 0x7fabf0149da0 secure :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fabe000c450 tx=0x7fabe000c910 comp rx=0 tx=0).stop 2024-09-19T08:41:33.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.702+0000 7fabf483d640 1 -- 172.21.15.98:0/1390631897 >> 172.21.15.98:0/1390631897 conn(0x7fabf00b2010 msgr2=0x7fabf00af640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:33.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.702+0000 7fabf483d640 1 -- 172.21.15.98:0/1390631897 shutdown_connections 2024-09-19T08:41:33.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:33.702+0000 7fabf483d640 1 -- 172.21.15.98:0/1390631897 wait complete. 2024-09-19T08:41:33.719 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-19T08:41:33.857 INFO:teuthology.orchestra.run.smithi098.stdout:[] 2024-09-19T08:41:33.857 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-09-19T08:41:33.858 DEBUG:tasks.ceph_test_case:wait_until_true: success in 75s and 0 retries 2024-09-19T08:41:33.858 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T08:41:33.858 DEBUG:teuthology.orchestra.run.smithi098:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:41:33.918 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:41:33.995 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T08:41:33.996 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:41:33.996 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:41:33.996 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:41:34.002 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15846 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T08:41:34.002 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:41:34.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:41:34.002 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T08:41:34.049 INFO:teuthology.orchestra.run.smithi098.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-19T08:41:34.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:41:34.050 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:41:34.050 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:41:34.050 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:41:34.057 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T08:41:34.057 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:41:34.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:41:34.057 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-19T08:41:34.105 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-09-19T08:41:34.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:41:34.106 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:41:34.106 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:41:34.106 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:41:34.112 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15846 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T08:41:34.113 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:41:34.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:41:34.113 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-19T08:41:34.161 INFO:teuthology.orchestra.run.smithi098.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-19T08:41:34.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:41:34.162 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:41:34.162 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:41:34.163 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:41:34.168 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T08:41:34.168 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:41:34.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:41:34.169 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-19T08:41:34.214 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-19T08:41:34.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:41:34.215 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:41:34.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.346+0000 7f1b441b2640 1 Processor -- start 2024-09-19T08:41:34.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.346+0000 7f1b441b2640 1 -- start start 2024-09-19T08:41:34.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.346+0000 7f1b441b2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b3c06c3a0 0x7f1b3c0cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:34.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.346+0000 7f1b441b2640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1b3c0cf900 0x7f1b3c0723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:34.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.346+0000 7f1b441b2640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b3c06d310 con 0x7f1b3c15c550 2024-09-19T08:41:34.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.346+0000 7f1b441b2640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f1b3c06d5f0 con 0x7f1b3c0cf900 2024-09-19T08:41:34.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.346+0000 7f1b441b2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1b3c06d8d0 con 0x7f1b3c06c3a0 2024-09-19T08:41:34.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.346+0000 7f1b41f4e640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f1b3c15c550 0x7f1b3c06bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:58280/0 (socket says 172.21.15.98:58280) 2024-09-19T08:41:34.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.346+0000 7f1b41f4e640 1 -- 172.21.15.98:0/1546338984 learned_addr learned my addr 172.21.15.98:0/1546338984 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:41:34.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.346+0000 7f1b4274f640 1 --2- 172.21.15.98:0/1546338984 >> v2:172.21.15.139:3300/0 conn(0x7f1b3c0cf900 0x7f1b3c0723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:34.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.346+0000 7f1b4174d640 1 --2- 172.21.15.98:0/1546338984 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b3c06c3a0 0x7f1b3c0cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:34.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.346+0000 7f1b40f4c640 1 -- 172.21.15.98:0/1546338984 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3378739109 0 0) 0x7f1b3c06d310 con 0x7f1b3c15c550 2024-09-19T08:41:34.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.346+0000 7f1b40f4c640 1 -- 172.21.15.98:0/1546338984 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b28003530 con 0x7f1b3c15c550 2024-09-19T08:41:34.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.350+0000 7f1b40f4c640 1 -- 172.21.15.98:0/1546338984 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 975348460 0 0) 0x7f1b28003530 con 0x7f1b3c15c550 2024-09-19T08:41:34.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.350+0000 7f1b40f4c640 1 -- 172.21.15.98:0/1546338984 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b3c06c3a0 msgr2=0x7f1b3c0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:34.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.350+0000 7f1b40f4c640 1 --2- 172.21.15.98:0/1546338984 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b3c06c3a0 0x7f1b3c0cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:34.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.350+0000 7f1b40f4c640 1 -- 172.21.15.98:0/1546338984 >> v2:172.21.15.139:3300/0 conn(0x7f1b3c0cf900 msgr2=0x7f1b3c0723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:34.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.350+0000 7f1b40f4c640 1 --2- 172.21.15.98:0/1546338984 >> v2:172.21.15.139:3300/0 conn(0x7f1b3c0cf900 0x7f1b3c0723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:34.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.350+0000 7f1b40f4c640 1 -- 172.21.15.98:0/1546338984 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3c06dbb0 con 0x7f1b3c15c550 2024-09-19T08:41:34.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.350+0000 7f1b40f4c640 1 -- 172.21.15.98:0/1546338984 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f1b38001680 con 0x7f1b3c15c550 2024-09-19T08:41:34.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.350+0000 7f1b40f4c640 1 -- 172.21.15.98:0/1546338984 >> v1:172.21.15.98:6789/0 conn(0x7f1b3c15c550 legacy=0x7f1b3c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:34.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.350+0000 7f1b40f4c640 1 --2- 172.21.15.98:0/1546338984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b280039c0 0x7f1b28003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:34.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.350+0000 7f1b40f4c640 1 --2- 172.21.15.98:0/1546338984 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b3c06c3a0 0x7f1b28004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:34.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.350+0000 7f1b40f4c640 1 --2- 172.21.15.98:0/1546338984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b28004d50 0x7f1b280051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:34.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.350+0000 7f1b40f4c640 1 -- 172.21.15.98:0/1546338984 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c06d5f0 con 0x7f1b280039c0 2024-09-19T08:41:34.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.350+0000 7f1b40f4c640 1 -- 172.21.15.98:0/1546338984 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c06d8d0 con 0x7f1b28004d50 2024-09-19T08:41:34.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.350+0000 7f1b40f4c640 1 -- 172.21.15.98:0/1546338984 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1b2800b9d0 con 0x7f1b3c06c3a0 2024-09-19T08:41:34.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.350+0000 7f1b41f4e640 1 --2- 172.21.15.98:0/1546338984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b280039c0 0x7f1b28003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:34.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.350+0000 7f1b4174d640 1 --2- 172.21.15.98:0/1546338984 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b3c06c3a0 0x7f1b28004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:34.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.350+0000 7f1b4274f640 1 --2- 172.21.15.98:0/1546338984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b28004d50 0x7f1b280051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:34.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.350+0000 7f1b41f4e640 1 -- 172.21.15.98:0/1546338984 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b3c06c3a0 msgr2=0x7f1b28004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:34.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.350+0000 7f1b41f4e640 1 --2- 172.21.15.98:0/1546338984 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b3c06c3a0 0x7f1b28004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:34.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.350+0000 7f1b41f4e640 1 -- 172.21.15.98:0/1546338984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b28004d50 msgr2=0x7f1b280051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:34.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.350+0000 7f1b41f4e640 1 --2- 172.21.15.98:0/1546338984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b28004d50 0x7f1b280051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:34.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.350+0000 7f1b41f4e640 1 -- 172.21.15.98:0/1546338984 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b28003530 con 0x7f1b280039c0 2024-09-19T08:41:34.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.350+0000 7f1b4274f640 1 --2- 172.21.15.98:0/1546338984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b28004d50 0x7f1b280051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:34.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.350+0000 7f1b41f4e640 1 --2- 172.21.15.98:0/1546338984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b280039c0 0x7f1b28003de0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800c240 tx=0x7f1b3801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a242e0faa077eb8a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:34.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.350+0000 7f1b40f4c640 1 -- 172.21.15.98:0/1546338984 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b38007d40 con 0x7f1b280039c0 2024-09-19T08:41:34.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.350+0000 7f1b40f4c640 1 -- 172.21.15.98:0/1546338984 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b380a9020 con 0x7f1b280039c0 2024-09-19T08:41:34.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.350+0000 7f1b40f4c640 1 -- 172.21.15.98:0/1546338984 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b38005ae0 con 0x7f1b280039c0 2024-09-19T08:41:34.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.350+0000 7f1b441b2640 1 -- 172.21.15.98:0/1546338984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b280039c0 msgr2=0x7f1b28003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:34.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.350+0000 7f1b441b2640 1 --2- 172.21.15.98:0/1546338984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b280039c0 0x7f1b28003de0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800c240 tx=0x7f1b3801cca0 comp rx=0 tx=0).stop 2024-09-19T08:41:34.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.350+0000 7f1b4274f640 1 -- 172.21.15.98:0/1546338984 reap_dead start 2024-09-19T08:41:34.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.350+0000 7f1b441b2640 1 -- 172.21.15.98:0/1546338984 shutdown_connections 2024-09-19T08:41:34.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.350+0000 7f1b441b2640 1 -- 172.21.15.98:0/1546338984 >> 172.21.15.98:0/1546338984 conn(0x7f1b3c0b2010 msgr2=0x7f1b3c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:34.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.350+0000 7f1b441b2640 1 -- 172.21.15.98:0/1546338984 shutdown_connections 2024-09-19T08:41:34.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.350+0000 7f1b441b2640 1 -- 172.21.15.98:0/1546338984 wait complete. 2024-09-19T08:41:34.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.354+0000 7f1b441b2640 1 Processor -- start 2024-09-19T08:41:34.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.354+0000 7f1b441b2640 1 -- start start 2024-09-19T08:41:34.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.354+0000 7f1b441b2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b280039c0 0x7f1b3c1507c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:34.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.354+0000 7f1b441b2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b3c15eef0 0x7f1b3c0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:34.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.354+0000 7f1b441b2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b3c0b3950 0x7f1b3c0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:34.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.354+0000 7f1b441b2640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c073300 con 0x7f1b3c0b3950 2024-09-19T08:41:34.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.354+0000 7f1b441b2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c15cb40 con 0x7f1b3c15eef0 2024-09-19T08:41:34.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.354+0000 7f1b441b2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1b3c073480 con 0x7f1b280039c0 2024-09-19T08:41:34.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.354+0000 7f1b4274f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b3c0b3950 0x7f1b3c0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:34.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.354+0000 7f1b4274f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b3c0b3950 0x7f1b3c0ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37628/0 (socket says 172.21.15.98:37628) 2024-09-19T08:41:34.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.354+0000 7f1b4274f640 1 -- 172.21.15.98:0/1491273628 learned_addr learned my addr 172.21.15.98:0/1491273628 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:34.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.354+0000 7f1b41f4e640 1 --2- 172.21.15.98:0/1491273628 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b280039c0 0x7f1b3c1507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:34.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.354+0000 7f1b4274f640 1 -- 172.21.15.98:0/1491273628 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b280039c0 msgr2=0x7f1b3c1507c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:34.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.354+0000 7f1b4274f640 1 --2- 172.21.15.98:0/1491273628 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b280039c0 0x7f1b3c1507c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:34.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.354+0000 7f1b4274f640 1 -- 172.21.15.98:0/1491273628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b3c15eef0 msgr2=0x7f1b3c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:34.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.354+0000 7f1b4274f640 1 --2- 172.21.15.98:0/1491273628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b3c15eef0 0x7f1b3c0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:34.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.354+0000 7f1b4274f640 1 -- 172.21.15.98:0/1491273628 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3c150d00 con 0x7f1b3c0b3950 2024-09-19T08:41:34.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.354+0000 7f1b4274f640 1 --2- 172.21.15.98:0/1491273628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b3c0b3950 0x7f1b3c0ae8b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c073b20 tx=0x7f1b3400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:34.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.354+0000 7f1b327fc640 1 -- 172.21.15.98:0/1491273628 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b34016620 con 0x7f1b3c0b3950 2024-09-19T08:41:34.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.354+0000 7f1b327fc640 1 -- 172.21.15.98:0/1491273628 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b3401b070 con 0x7f1b3c0b3950 2024-09-19T08:41:34.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.354+0000 7f1b441b2640 1 -- 172.21.15.98:0/1491273628 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b3c15e450 con 0x7f1b3c0b3950 2024-09-19T08:41:34.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.354+0000 7f1b327fc640 1 -- 172.21.15.98:0/1491273628 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b340169e0 con 0x7f1b3c0b3950 2024-09-19T08:41:34.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.354+0000 7f1b441b2640 1 -- 172.21.15.98:0/1491273628 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b3c0b4170 con 0x7f1b3c0b3950 2024-09-19T08:41:34.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.354+0000 7f1b441b2640 1 -- 172.21.15.98:0/1491273628 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b3c0723a0 con 0x7f1b3c0b3950 2024-09-19T08:41:34.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.358+0000 7f1b327fc640 1 -- 172.21.15.98:0/1491273628 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f1b34016b80 con 0x7f1b3c0b3950 2024-09-19T08:41:34.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.358+0000 7f1b327fc640 1 --2- 172.21.15.98:0/1491273628 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f1b20067aa0 0x7f1b20069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:34.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.358+0000 7f1b327fc640 1 -- 172.21.15.98:0/1491273628 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f1b34086c90 con 0x7f1b3c0b3950 2024-09-19T08:41:34.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.358+0000 7f1b41f4e640 1 --2- 172.21.15.98:0/1491273628 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f1b20067aa0 0x7f1b20069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:34.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.358+0000 7f1b41f4e640 1 --2- 172.21.15.98:0/1491273628 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f1b20067aa0 0x7f1b20069f60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1b380aa000 tx=0x7f1b3c06d4e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:34.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.362+0000 7f1b327fc640 1 -- 172.21.15.98:0/1491273628 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f1b34057490 con 0x7f1b3c0b3950 2024-09-19T08:41:34.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.558+0000 7f1b441b2640 1 -- 172.21.15.98:0/1491273628 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b3c157bb0 con 0x7f1b3c0b3950 2024-09-19T08:41:34.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.558+0000 7f1b327fc640 1 -- 172.21.15.98:0/1491273628 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5229 (secure 0 0 0) 0x7f1b34056d90 con 0x7f1b3c0b3950 2024-09-19T08:41:34.559 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:41:34.559 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"btime":"2024-09-19T08:39:55:181532+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/2874086421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":2874086421},{"type":"v1","addr":"172.21.15.139:6833","nonce":2874086421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4996,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/4178563848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":4178563848},{"type":"v1","addr":"172.21.15.139:6835","nonce":4178563848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3584568343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3584568343},{"type":"v1","addr":"172.21.15.98:6835","nonce":3584568343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:39:38.262938+0000","modified":"2024-09-19T08:39:41.276985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/3085042202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3085042202},{"type":"v1","addr":"172.21.15.98:6837","nonce":3085042202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244]},"id":3}]} 2024-09-19T08:41:34.559 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 24 2024-09-19T08:41:34.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.562+0000 7f1b441b2640 1 -- 172.21.15.98:0/1491273628 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f1b20067aa0 msgr2=0x7f1b20069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:34.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.562+0000 7f1b441b2640 1 --2- 172.21.15.98:0/1491273628 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f1b20067aa0 0x7f1b20069f60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1b380aa000 tx=0x7f1b3c06d4e0 comp rx=0 tx=0).stop 2024-09-19T08:41:34.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.562+0000 7f1b441b2640 1 -- 172.21.15.98:0/1491273628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b3c0b3950 msgr2=0x7f1b3c0ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:34.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.562+0000 7f1b441b2640 1 --2- 172.21.15.98:0/1491273628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b3c0b3950 0x7f1b3c0ae8b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c073b20 tx=0x7f1b3400f520 comp rx=0 tx=0).stop 2024-09-19T08:41:34.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.562+0000 7f1b441b2640 1 -- 172.21.15.98:0/1491273628 shutdown_connections 2024-09-19T08:41:34.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.562+0000 7f1b441b2640 1 --2- 172.21.15.98:0/1491273628 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f1b20067aa0 0x7f1b20069f60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:34.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.562+0000 7f1b441b2640 1 --2- 172.21.15.98:0/1491273628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b3c0b3950 0x7f1b3c0ae8b0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:34.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.562+0000 7f1b441b2640 1 --2- 172.21.15.98:0/1491273628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b3c15eef0 0x7f1b3c0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:34.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.562+0000 7f1b441b2640 1 --2- 172.21.15.98:0/1491273628 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b280039c0 0x7f1b3c1507c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:34.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.562+0000 7f1b441b2640 1 -- 172.21.15.98:0/1491273628 >> 172.21.15.98:0/1491273628 conn(0x7f1b3c0b2010 msgr2=0x7f1b3c073c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:34.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.562+0000 7f1b441b2640 1 -- 172.21.15.98:0/1491273628 shutdown_connections 2024-09-19T08:41:34.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.562+0000 7f1b441b2640 1 -- 172.21.15.98:0/1491273628 wait complete. 2024-09-19T08:41:34.576 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:41:34.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.682+0000 7f21108b2640 1 Processor -- start 2024-09-19T08:41:34.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.682+0000 7f21108b2640 1 -- start start 2024-09-19T08:41:34.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.686+0000 7f21108b2640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f210c0d4820 0x7f210c1621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:34.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.686+0000 7f21108b2640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f210c0b9970 0x7f210c0b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:34.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.686+0000 7f21108b2640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f210c068440 con 0x7f210c0b7f80 2024-09-19T08:41:34.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.686+0000 7f21108b2640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f210c163050 con 0x7f210c0b9970 2024-09-19T08:41:34.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.686+0000 7f21108b2640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f210c162a90 con 0x7f210c0d4820 2024-09-19T08:41:34.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.686+0000 7f210a59c640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f210c0b7f80 0x7f210c15f560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:58284/0 (socket says 172.21.15.98:58284) 2024-09-19T08:41:34.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.686+0000 7f210a59c640 1 -- 172.21.15.98:0/3586181519 learned_addr learned my addr 172.21.15.98:0/3586181519 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:41:34.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.686+0000 7f210ad9d640 1 --2- 172.21.15.98:0/3586181519 >> v2:172.21.15.98:3300/0 conn(0x7f210c0b9970 0x7f210c0b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:34.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.686+0000 7f2109d9b640 1 --2- 172.21.15.98:0/3586181519 >> v2:172.21.15.139:3300/0 conn(0x7f210c0d4820 0x7f210c1621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:34.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.686+0000 7f210ad9d640 1 -- 172.21.15.98:0/3586181519 >> v2:172.21.15.139:3300/0 conn(0x7f210c0d4820 msgr2=0x7f210c1621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:34.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.686+0000 7f210ad9d640 1 --2- 172.21.15.98:0/3586181519 >> v2:172.21.15.139:3300/0 conn(0x7f210c0d4820 0x7f210c1621b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:34.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.686+0000 7f210ad9d640 1 -- 172.21.15.98:0/3586181519 >> v1:172.21.15.98:6789/0 conn(0x7f210c0b7f80 legacy=0x7f210c15f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:34.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.686+0000 7f210959a640 1 -- 172.21.15.98:0/3586181519 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2346200177 0 0) 0x7f210c068440 con 0x7f210c0b7f80 2024-09-19T08:41:34.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.686+0000 7f210ad9d640 1 -- 172.21.15.98:0/3586181519 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f210c162d70 con 0x7f210c0b9970 2024-09-19T08:41:34.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.686+0000 7f2109d9b640 1 --2- 172.21.15.98:0/3586181519 >> v2:172.21.15.139:3300/0 conn(0x7f210c0d4820 0x7f210c1621b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:34.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.686+0000 7f210ad9d640 1 --2- 172.21.15.98:0/3586181519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f210c0b9970 0x7f210c0b9de0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f210c0ceb00 tx=0x7f20fc01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7380742e1eccf9e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:34.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.686+0000 7f210959a640 1 -- 172.21.15.98:0/3586181519 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20fc0b2070 con 0x7f210c0b9970 2024-09-19T08:41:34.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.686+0000 7f210959a640 1 -- 172.21.15.98:0/3586181519 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20fc0a5900 con 0x7f210c0b9970 2024-09-19T08:41:34.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.686+0000 7f210959a640 1 -- 172.21.15.98:0/3586181519 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20fc0a5cd0 con 0x7f210c0b9970 2024-09-19T08:41:34.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.686+0000 7f21108b2640 1 -- 172.21.15.98:0/3586181519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f210c0b9970 msgr2=0x7f210c0b9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:34.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.686+0000 7f21108b2640 1 --2- 172.21.15.98:0/3586181519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f210c0b9970 0x7f210c0b9de0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f210c0ceb00 tx=0x7f20fc01c930 comp rx=0 tx=0).stop 2024-09-19T08:41:34.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.690+0000 7f21108b2640 1 -- 172.21.15.98:0/3586181519 shutdown_connections 2024-09-19T08:41:34.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.690+0000 7f21108b2640 1 --2- 172.21.15.98:0/3586181519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f210c0b9970 0x7f210c0b9de0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:34.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.690+0000 7f21108b2640 1 --2- 172.21.15.98:0/3586181519 >> v2:172.21.15.139:3300/0 conn(0x7f210c0d4820 0x7f210c1621b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:34.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.690+0000 7f21108b2640 1 -- 172.21.15.98:0/3586181519 >> 172.21.15.98:0/3586181519 conn(0x7f210c074e40 msgr2=0x7f210c075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:34.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.690+0000 7f21108b2640 1 -- 172.21.15.98:0/3586181519 shutdown_connections 2024-09-19T08:41:34.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.690+0000 7f21108b2640 1 -- 172.21.15.98:0/3586181519 wait complete. 2024-09-19T08:41:34.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.690+0000 7f21108b2640 1 Processor -- start 2024-09-19T08:41:34.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.690+0000 7f21108b2640 1 -- start start 2024-09-19T08:41:34.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.690+0000 7f21108b2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f210c0b7f80 0x7f210c139610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:34.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.690+0000 7f21108b2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f210c0b9970 0x7f210c153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:34.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.690+0000 7f21108b2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f210c0d4820 0x7f210c0d3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:34.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.690+0000 7f21108b2640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f210c0b84e0 con 0x7f210c0d4820 2024-09-19T08:41:34.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.690+0000 7f21108b2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f210c0b8360 con 0x7f210c0b9970 2024-09-19T08:41:34.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.690+0000 7f21108b2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f210c144590 con 0x7f210c0b7f80 2024-09-19T08:41:34.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.690+0000 7f210ad9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f210c0d4820 0x7f210c0d3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:34.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.690+0000 7f210ad9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f210c0d4820 0x7f210c0d3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37642/0 (socket says 172.21.15.98:37642) 2024-09-19T08:41:34.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.690+0000 7f210ad9d640 1 -- 172.21.15.98:0/2162163811 learned_addr learned my addr 172.21.15.98:0/2162163811 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:34.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.690+0000 7f2109d9b640 1 --2- 172.21.15.98:0/2162163811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f210c0b9970 0x7f210c153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:34.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.690+0000 7f210ad9d640 1 -- 172.21.15.98:0/2162163811 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f210c0b7f80 msgr2=0x7f210c139610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:34.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.690+0000 7f210a59c640 1 --2- 172.21.15.98:0/2162163811 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f210c0b7f80 0x7f210c139610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:34.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.690+0000 7f210ad9d640 1 --2- 172.21.15.98:0/2162163811 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f210c0b7f80 0x7f210c139610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:34.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.690+0000 7f210ad9d640 1 -- 172.21.15.98:0/2162163811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f210c0b9970 msgr2=0x7f210c153fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:34.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.690+0000 7f210ad9d640 1 --2- 172.21.15.98:0/2162163811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f210c0b9970 0x7f210c153fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:34.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.690+0000 7f210ad9d640 1 -- 172.21.15.98:0/2162163811 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f210c154aa0 con 0x7f210c0d4820 2024-09-19T08:41:34.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.690+0000 7f2109d9b640 1 --2- 172.21.15.98:0/2162163811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f210c0b9970 0x7f210c153fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:34.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.690+0000 7f210ad9d640 1 --2- 172.21.15.98:0/2162163811 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f210c0d4820 0x7f210c0d3770 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f20fc019540 tx=0x7f20fc0196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:34.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.690+0000 7f2102ffd640 1 -- 172.21.15.98:0/2162163811 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20fc0b2050 con 0x7f210c0d4820 2024-09-19T08:41:34.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.690+0000 7f2102ffd640 1 -- 172.21.15.98:0/2162163811 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20fc0be070 con 0x7f210c0d4820 2024-09-19T08:41:34.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.690+0000 7f2102ffd640 1 -- 172.21.15.98:0/2162163811 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20fc0adb10 con 0x7f210c0d4820 2024-09-19T08:41:34.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.690+0000 7f21108b2640 1 -- 172.21.15.98:0/2162163811 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f210c0d26d0 con 0x7f210c0d4820 2024-09-19T08:41:34.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.690+0000 7f21108b2640 1 -- 172.21.15.98:0/2162163811 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f210c0d2c90 con 0x7f210c0d4820 2024-09-19T08:41:34.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.690+0000 7f21108b2640 1 -- 172.21.15.98:0/2162163811 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f210c068f50 con 0x7f210c0d4820 2024-09-19T08:41:34.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.694+0000 7f2102ffd640 1 -- 172.21.15.98:0/2162163811 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f20fc0adcb0 con 0x7f210c0d4820 2024-09-19T08:41:34.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.694+0000 7f2102ffd640 1 --2- 172.21.15.98:0/2162163811 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f20ec067a70 0x7f20ec069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:34.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.694+0000 7f2102ffd640 1 -- 172.21.15.98:0/2162163811 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f20fc121380 con 0x7f210c0d4820 2024-09-19T08:41:34.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.694+0000 7f210a59c640 1 --2- 172.21.15.98:0/2162163811 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f20ec067a70 0x7f20ec069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:34.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.694+0000 7f210a59c640 1 --2- 172.21.15.98:0/2162163811 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f20ec067a70 0x7f20ec069f30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2104000f70 tx=0x7f2104009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:34.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.698+0000 7f2102ffd640 1 -- 172.21.15.98:0/2162163811 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f210c154aa0 con 0x7f210c0d4820 2024-09-19T08:41:34.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.878+0000 7f21108b2640 1 -- 172.21.15.98:0/2162163811 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f210c0d4ea0 con 0x7f210c0d4820 2024-09-19T08:41:34.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.878+0000 7f2102ffd640 1 -- 172.21.15.98:0/2162163811 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5229 (secure 0 0 0) 0x7f20fc016520 con 0x7f210c0d4820 2024-09-19T08:41:34.879 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:41:34.879 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"btime":"2024-09-19T08:39:55:181532+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/2874086421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":2874086421},{"type":"v1","addr":"172.21.15.139:6833","nonce":2874086421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4996,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/4178563848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":4178563848},{"type":"v1","addr":"172.21.15.139:6835","nonce":4178563848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3584568343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3584568343},{"type":"v1","addr":"172.21.15.98:6835","nonce":3584568343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:39:38.262938+0000","modified":"2024-09-19T08:39:41.276985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/3085042202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3085042202},{"type":"v1","addr":"172.21.15.98:6837","nonce":3085042202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244]},"id":3}]} 2024-09-19T08:41:34.879 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 24 2024-09-19T08:41:34.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.882+0000 7f21108b2640 1 -- 172.21.15.98:0/2162163811 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f20ec067a70 msgr2=0x7f20ec069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:34.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.882+0000 7f21108b2640 1 --2- 172.21.15.98:0/2162163811 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f20ec067a70 0x7f20ec069f30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2104000f70 tx=0x7f2104009040 comp rx=0 tx=0).stop 2024-09-19T08:41:34.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.882+0000 7f21108b2640 1 -- 172.21.15.98:0/2162163811 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f210c0d4820 msgr2=0x7f210c0d3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:34.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.882+0000 7f21108b2640 1 --2- 172.21.15.98:0/2162163811 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f210c0d4820 0x7f210c0d3770 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f20fc019540 tx=0x7f20fc0196d0 comp rx=0 tx=0).stop 2024-09-19T08:41:34.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.882+0000 7f21108b2640 1 -- 172.21.15.98:0/2162163811 shutdown_connections 2024-09-19T08:41:34.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.882+0000 7f21108b2640 1 --2- 172.21.15.98:0/2162163811 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f20ec067a70 0x7f20ec069f30 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:34.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.882+0000 7f21108b2640 1 --2- 172.21.15.98:0/2162163811 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f210c0d4820 0x7f210c0d3770 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:34.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.882+0000 7f21108b2640 1 --2- 172.21.15.98:0/2162163811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f210c0b9970 0x7f210c153fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:34.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.882+0000 7f21108b2640 1 --2- 172.21.15.98:0/2162163811 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f210c0b7f80 0x7f210c139610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:34.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.882+0000 7f21108b2640 1 -- 172.21.15.98:0/2162163811 >> 172.21.15.98:0/2162163811 conn(0x7f210c074e40 msgr2=0x7f210c15fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:34.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.882+0000 7f21108b2640 1 -- 172.21.15.98:0/2162163811 shutdown_connections 2024-09-19T08:41:34.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:34.882+0000 7f21108b2640 1 -- 172.21.15.98:0/2162163811 wait complete. 2024-09-19T08:41:34.897 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:41:35.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.002+0000 7fa37fa6c640 1 Processor -- start 2024-09-19T08:41:35.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.002+0000 7fa37fa6c640 1 -- start start 2024-09-19T08:41:35.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.006+0000 7fa37fa6c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa37806c120 0x7fa3780cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:35.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.006+0000 7fa37fa6c640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3780ccc60 con 0x7fa37806bc90 2024-09-19T08:41:35.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.006+0000 7fa37fa6c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3780ccf40 con 0x7fa378158fd0 2024-09-19T08:41:35.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.006+0000 7fa37fa6c640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fa3781593b0 con 0x7fa37806c120 2024-09-19T08:41:35.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.006+0000 7fa37e009640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa37806c120 0x7fa3780cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:35.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.006+0000 7fa37d007640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fa37806bc90 0x7fa3780ce840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:58300/0 (socket says 172.21.15.98:58300) 2024-09-19T08:41:35.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.006+0000 7fa37d007640 1 -- 172.21.15.98:0/3632102487 learned_addr learned my addr 172.21.15.98:0/3632102487 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:41:35.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.006+0000 7fa37e009640 1 -- 172.21.15.98:0/3632102487 >> v1:172.21.15.139:6789/0 conn(0x7fa378158fd0 legacy=0x7fa3780b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:35.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.006+0000 7fa37e009640 1 -- 172.21.15.98:0/3632102487 >> v1:172.21.15.98:6789/0 conn(0x7fa37806bc90 legacy=0x7fa3780ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:35.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.006+0000 7fa37e009640 1 -- 172.21.15.98:0/3632102487 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3780cd220 con 0x7fa37806c120 2024-09-19T08:41:35.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.006+0000 7fa37e009640 1 --2- 172.21.15.98:0/3632102487 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa37806c120 0x7fa3780cef50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa370009d90 tx=0x7fa37001c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb0fa753954f5d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:35.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.006+0000 7fa37c806640 1 -- 172.21.15.98:0/3632102487 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3700b2070 con 0x7fa37806c120 2024-09-19T08:41:35.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.006+0000 7fa37c806640 1 -- 172.21.15.98:0/3632102487 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3700a5930 con 0x7fa37806c120 2024-09-19T08:41:35.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.006+0000 7fa37c806640 1 -- 172.21.15.98:0/3632102487 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3700a5d00 con 0x7fa37806c120 2024-09-19T08:41:35.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.006+0000 7fa37fa6c640 1 -- 172.21.15.98:0/3632102487 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa37806c120 msgr2=0x7fa3780cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:35.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.006+0000 7fa37fa6c640 1 --2- 172.21.15.98:0/3632102487 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa37806c120 0x7fa3780cef50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa370009d90 tx=0x7fa37001c870 comp rx=0 tx=0).stop 2024-09-19T08:41:35.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.006+0000 7fa37fa6c640 1 -- 172.21.15.98:0/3632102487 shutdown_connections 2024-09-19T08:41:35.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.006+0000 7fa37fa6c640 1 --2- 172.21.15.98:0/3632102487 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa37806c120 0x7fa3780cef50 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:35.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.006+0000 7fa37fa6c640 1 -- 172.21.15.98:0/3632102487 >> 172.21.15.98:0/3632102487 conn(0x7fa3780b2010 msgr2=0x7fa3780b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:35.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.010+0000 7fa37fa6c640 1 -- 172.21.15.98:0/3632102487 shutdown_connections 2024-09-19T08:41:35.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.010+0000 7fa37fa6c640 1 -- 172.21.15.98:0/3632102487 wait complete. 2024-09-19T08:41:35.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.010+0000 7fa37fa6c640 1 Processor -- start 2024-09-19T08:41:35.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.010+0000 7fa37fa6c640 1 -- start start 2024-09-19T08:41:35.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.010+0000 7fa37fa6c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa37806bc90 0x7fa37815ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:35.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.010+0000 7fa37fa6c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa37806c120 0x7fa37814adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:35.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.010+0000 7fa37fa6c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa378158fd0 0x7fa378150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:35.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.010+0000 7fa37fa6c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa37815c7a0 con 0x7fa37806c120 2024-09-19T08:41:35.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.010+0000 7fa37fa6c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa3780b4530 con 0x7fa37806bc90 2024-09-19T08:41:35.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.010+0000 7fa37fa6c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa37815c920 con 0x7fa378158fd0 2024-09-19T08:41:35.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.010+0000 7fa37d007640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa37806c120 0x7fa37814adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:35.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.010+0000 7fa37d007640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa37806c120 0x7fa37814adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37660/0 (socket says 172.21.15.98:37660) 2024-09-19T08:41:35.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.010+0000 7fa37d007640 1 -- 172.21.15.98:0/1434949697 learned_addr learned my addr 172.21.15.98:0/1434949697 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:35.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.010+0000 7fa37d808640 1 --2- 172.21.15.98:0/1434949697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa37806bc90 0x7fa37815ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:35.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.010+0000 7fa37d007640 1 -- 172.21.15.98:0/1434949697 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa378158fd0 msgr2=0x7fa378150530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:35.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.010+0000 7fa37d007640 1 --2- 172.21.15.98:0/1434949697 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa378158fd0 0x7fa378150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:35.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.010+0000 7fa37d007640 1 -- 172.21.15.98:0/1434949697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa37806bc90 msgr2=0x7fa37815ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:35.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.010+0000 7fa37d007640 1 --2- 172.21.15.98:0/1434949697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa37806bc90 0x7fa37815ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:35.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.010+0000 7fa37d007640 1 -- 172.21.15.98:0/1434949697 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa37815f380 con 0x7fa37806c120 2024-09-19T08:41:35.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.010+0000 7fa37d007640 1 --2- 172.21.15.98:0/1434949697 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa37806c120 0x7fa37814adf0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa36800c450 tx=0x7fa36800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:35.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.010+0000 7fa36e7fc640 1 -- 172.21.15.98:0/1434949697 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa368017070 con 0x7fa37806c120 2024-09-19T08:41:35.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.010+0000 7fa36e7fc640 1 -- 172.21.15.98:0/1434949697 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa368005cf0 con 0x7fa37806c120 2024-09-19T08:41:35.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.010+0000 7fa37fa6c640 1 -- 172.21.15.98:0/1434949697 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa37815f660 con 0x7fa37806c120 2024-09-19T08:41:35.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.010+0000 7fa36e7fc640 1 -- 172.21.15.98:0/1434949697 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa368007290 con 0x7fa37806c120 2024-09-19T08:41:35.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.010+0000 7fa37fa6c640 1 -- 172.21.15.98:0/1434949697 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa37815f940 con 0x7fa37806c120 2024-09-19T08:41:35.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.010+0000 7fa37fa6c640 1 -- 172.21.15.98:0/1434949697 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa378069bf0 con 0x7fa37806c120 2024-09-19T08:41:35.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.014+0000 7fa36e7fc640 1 -- 172.21.15.98:0/1434949697 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fa36801a070 con 0x7fa37806c120 2024-09-19T08:41:35.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.014+0000 7fa36e7fc640 1 --2- 172.21.15.98:0/1434949697 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa35c067aa0 0x7fa35c069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:35.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.014+0000 7fa36e7fc640 1 -- 172.21.15.98:0/1434949697 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7fa368085240 con 0x7fa37806c120 2024-09-19T08:41:35.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.014+0000 7fa37d808640 1 --2- 172.21.15.98:0/1434949697 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa35c067aa0 0x7fa35c069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:35.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.014+0000 7fa37d808640 1 --2- 172.21.15.98:0/1434949697 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa35c067aa0 0x7fa35c069f60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa37814e3f0 tx=0x7fa374008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:35.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.018+0000 7fa36e7fc640 1 -- 172.21.15.98:0/1434949697 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa368055b50 con 0x7fa37806c120 2024-09-19T08:41:35.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.178+0000 7fa37fa6c640 1 -- 172.21.15.98:0/1434949697 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa378151ce0 con 0x7fa37806c120 2024-09-19T08:41:35.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.178+0000 7fa36e7fc640 1 -- 172.21.15.98:0/1434949697 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16825 (secure 0 0 0) 0x7fa368055450 con 0x7fa37806c120 2024-09-19T08:41:35.180 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:41:35.180 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":45,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:40:01.212861+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:39:36.829570+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-19T08:39:37.237199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.30","mappings":[{"from":0,"to":5}]},{"pgid":"7.32","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:41:35.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.182+0000 7fa37fa6c640 1 -- 172.21.15.98:0/1434949697 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa35c067aa0 msgr2=0x7fa35c069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:35.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.182+0000 7fa37fa6c640 1 --2- 172.21.15.98:0/1434949697 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa35c067aa0 0x7fa35c069f60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa37814e3f0 tx=0x7fa374008040 comp rx=0 tx=0).stop 2024-09-19T08:41:35.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.182+0000 7fa37fa6c640 1 -- 172.21.15.98:0/1434949697 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa37806c120 msgr2=0x7fa37814adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:35.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.182+0000 7fa37fa6c640 1 --2- 172.21.15.98:0/1434949697 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa37806c120 0x7fa37814adf0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa36800c450 tx=0x7fa36800c910 comp rx=0 tx=0).stop 2024-09-19T08:41:35.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.182+0000 7fa37fa6c640 1 -- 172.21.15.98:0/1434949697 shutdown_connections 2024-09-19T08:41:35.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.182+0000 7fa37fa6c640 1 --2- 172.21.15.98:0/1434949697 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa35c067aa0 0x7fa35c069f60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:35.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.182+0000 7fa37fa6c640 1 --2- 172.21.15.98:0/1434949697 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa378158fd0 0x7fa378150530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:35.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.182+0000 7fa37fa6c640 1 --2- 172.21.15.98:0/1434949697 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa37806c120 0x7fa37814adf0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:35.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.182+0000 7fa37fa6c640 1 --2- 172.21.15.98:0/1434949697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa37806bc90 0x7fa37815ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:35.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.182+0000 7fa37fa6c640 1 -- 172.21.15.98:0/1434949697 >> 172.21.15.98:0/1434949697 conn(0x7fa3780b2010 msgr2=0x7fa3780b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:35.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.182+0000 7fa37fa6c640 1 -- 172.21.15.98:0/1434949697 shutdown_connections 2024-09-19T08:41:35.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.182+0000 7fa37fa6c640 1 -- 172.21.15.98:0/1434949697 wait complete. 2024-09-19T08:41:35.199 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T08:41:35.199 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:41:35.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.306+0000 7f80f602e640 1 Processor -- start 2024-09-19T08:41:35.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.306+0000 7f80f602e640 1 -- start start 2024-09-19T08:41:35.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.306+0000 7f80f602e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80f006c3a0 0x7f80f00cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:35.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.306+0000 7f80f602e640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80f006d310 con 0x7f80f015c550 2024-09-19T08:41:35.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.306+0000 7f80f602e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80f006d5f0 con 0x7f80f00cf900 2024-09-19T08:41:35.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.306+0000 7f80f602e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f80f006d8d0 con 0x7f80f006c3a0 2024-09-19T08:41:35.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.306+0000 7f80ef7fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f80f015c550 0x7f80f006bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:58302/0 (socket says 172.21.15.98:58302) 2024-09-19T08:41:35.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.306+0000 7f80ef7fe640 1 -- 172.21.15.98:0/3076630820 learned_addr learned my addr 172.21.15.98:0/3076630820 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:41:35.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.306+0000 7f80ee7fc640 1 -- 172.21.15.98:0/3076630820 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2959801863 0 0) 0x7f80f006d310 con 0x7f80f015c550 2024-09-19T08:41:35.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.306+0000 7f80ee7fc640 1 -- 172.21.15.98:0/3076630820 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80d0003530 con 0x7f80f015c550 2024-09-19T08:41:35.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.306+0000 7f80eeffd640 1 --2- 172.21.15.98:0/3076630820 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80f006c3a0 0x7f80f00cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:35.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.306+0000 7f80ee7fc640 1 -- 172.21.15.98:0/3076630820 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1584815354 0 0) 0x7f80d0003530 con 0x7f80f015c550 2024-09-19T08:41:35.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.306+0000 7f80ee7fc640 1 -- 172.21.15.98:0/3076630820 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80f006c3a0 msgr2=0x7f80f00cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:35.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.306+0000 7f80ee7fc640 1 --2- 172.21.15.98:0/3076630820 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80f006c3a0 0x7f80f00cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:35.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.306+0000 7f80ee7fc640 1 -- 172.21.15.98:0/3076630820 >> v1:172.21.15.139:6789/0 conn(0x7f80f00cf900 legacy=0x7f80f00723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:35.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.306+0000 7f80ee7fc640 1 -- 172.21.15.98:0/3076630820 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f006dbb0 con 0x7f80f015c550 2024-09-19T08:41:35.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.306+0000 7f80ee7fc640 1 -- 172.21.15.98:0/3076630820 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f80e0001680 con 0x7f80f015c550 2024-09-19T08:41:35.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.306+0000 7f80ee7fc640 1 -- 172.21.15.98:0/3076630820 >> v1:172.21.15.98:6789/0 conn(0x7f80f015c550 legacy=0x7f80f006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:35.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.306+0000 7f80ee7fc640 1 --2- 172.21.15.98:0/3076630820 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80d00039f0 0x7f80d0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:35.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.306+0000 7f80ee7fc640 1 --2- 172.21.15.98:0/3076630820 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80f006c3a0 0x7f80d0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:35.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.306+0000 7f80ee7fc640 1 --2- 172.21.15.98:0/3076630820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80d0004d50 0x7f80d00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:35.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.306+0000 7f80ee7fc640 1 -- 172.21.15.98:0/3076630820 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f80f006d8d0 con 0x7f80d0004d50 2024-09-19T08:41:35.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.306+0000 7f80ee7fc640 1 -- 172.21.15.98:0/3076630820 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f80d000bb60 con 0x7f80d00039f0 2024-09-19T08:41:35.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.306+0000 7f80ee7fc640 1 -- 172.21.15.98:0/3076630820 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f80d000bd30 con 0x7f80f006c3a0 2024-09-19T08:41:35.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.306+0000 7f80effff640 1 --2- 172.21.15.98:0/3076630820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80d0004d50 0x7f80d00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:35.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.306+0000 7f80ef7fe640 1 --2- 172.21.15.98:0/3076630820 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80d00039f0 0x7f80d0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:35.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.306+0000 7f80eeffd640 1 --2- 172.21.15.98:0/3076630820 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80f006c3a0 0x7f80d0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:35.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.306+0000 7f80effff640 1 -- 172.21.15.98:0/3076630820 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80f006c3a0 msgr2=0x7f80d0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:35.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.306+0000 7f80effff640 1 --2- 172.21.15.98:0/3076630820 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80f006c3a0 0x7f80d0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:35.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.306+0000 7f80effff640 1 -- 172.21.15.98:0/3076630820 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80d00039f0 msgr2=0x7f80d0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:35.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.306+0000 7f80effff640 1 --2- 172.21.15.98:0/3076630820 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80d00039f0 0x7f80d0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:35.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.306+0000 7f80effff640 1 -- 172.21.15.98:0/3076630820 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d0003530 con 0x7f80d0004d50 2024-09-19T08:41:35.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.306+0000 7f80ef7fe640 1 --2- 172.21.15.98:0/3076630820 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80d00039f0 0x7f80d0003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:35.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.306+0000 7f80eeffd640 1 --2- 172.21.15.98:0/3076630820 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80f006c3a0 0x7f80d0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:35.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.310+0000 7f80effff640 1 --2- 172.21.15.98:0/3076630820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80d0004d50 0x7f80d00051d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f80e80017b0 tx=0x7f80e801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7dcfd3b5b79c18d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:35.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.310+0000 7f80ee7fc640 1 -- 172.21.15.98:0/3076630820 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e80013a0 con 0x7f80d0004d50 2024-09-19T08:41:35.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.310+0000 7f80ee7fc640 1 -- 172.21.15.98:0/3076630820 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80e80052f0 con 0x7f80d0004d50 2024-09-19T08:41:35.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.310+0000 7f80ee7fc640 1 -- 172.21.15.98:0/3076630820 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e8003840 con 0x7f80d0004d50 2024-09-19T08:41:35.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.310+0000 7f80f602e640 1 -- 172.21.15.98:0/3076630820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80d0004d50 msgr2=0x7f80d00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:35.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.310+0000 7f80f602e640 1 --2- 172.21.15.98:0/3076630820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80d0004d50 0x7f80d00051d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f80e80017b0 tx=0x7f80e801c400 comp rx=0 tx=0).stop 2024-09-19T08:41:35.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.310+0000 7f80effff640 1 -- 172.21.15.98:0/3076630820 reap_dead start 2024-09-19T08:41:35.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.310+0000 7f80f602e640 1 -- 172.21.15.98:0/3076630820 shutdown_connections 2024-09-19T08:41:35.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.310+0000 7f80f602e640 1 -- 172.21.15.98:0/3076630820 >> 172.21.15.98:0/3076630820 conn(0x7f80f00b2010 msgr2=0x7f80f00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:35.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.310+0000 7f80f602e640 1 -- 172.21.15.98:0/3076630820 shutdown_connections 2024-09-19T08:41:35.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.310+0000 7f80f602e640 1 -- 172.21.15.98:0/3076630820 wait complete. 2024-09-19T08:41:35.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.310+0000 7f80f602e640 1 Processor -- start 2024-09-19T08:41:35.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.310+0000 7f80f602e640 1 -- start start 2024-09-19T08:41:35.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.310+0000 7f80f602e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80f014b220 0x7f80f0152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:35.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.310+0000 7f80f602e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80f0152c20 0x7f80f00b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:35.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.310+0000 7f80f602e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80f015f9c0 0x7f80f00b4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:35.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.310+0000 7f80f602e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f80f015cab0 con 0x7f80f014b220 2024-09-19T08:41:35.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.310+0000 7f80f602e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f80f015c930 con 0x7f80f0152c20 2024-09-19T08:41:35.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.310+0000 7f80f602e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f80f00cf3c0 con 0x7f80f015f9c0 2024-09-19T08:41:35.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.310+0000 7f80ef7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80f014b220 0x7f80f0152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:35.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.310+0000 7f80ef7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80f014b220 0x7f80f0152010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37688/0 (socket says 172.21.15.98:37688) 2024-09-19T08:41:35.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.310+0000 7f80ef7fe640 1 -- 172.21.15.98:0/1572026811 learned_addr learned my addr 172.21.15.98:0/1572026811 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:35.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.310+0000 7f80eeffd640 1 --2- 172.21.15.98:0/1572026811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80f0152c20 0x7f80f00b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:35.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.310+0000 7f80effff640 1 --2- 172.21.15.98:0/1572026811 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80f015f9c0 0x7f80f00b4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:35.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.310+0000 7f80ef7fe640 1 -- 172.21.15.98:0/1572026811 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80f015f9c0 msgr2=0x7f80f00b4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:35.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.310+0000 7f80ef7fe640 1 --2- 172.21.15.98:0/1572026811 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80f015f9c0 0x7f80f00b4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:35.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.310+0000 7f80ef7fe640 1 -- 172.21.15.98:0/1572026811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80f0152c20 msgr2=0x7f80f00b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:35.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.310+0000 7f80ef7fe640 1 --2- 172.21.15.98:0/1572026811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80f0152c20 0x7f80f00b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:35.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.310+0000 7f80ef7fe640 1 -- 172.21.15.98:0/1572026811 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f01530b0 con 0x7f80f014b220 2024-09-19T08:41:35.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.310+0000 7f80eeffd640 1 --2- 172.21.15.98:0/1572026811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80f0152c20 0x7f80f00b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:35.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.310+0000 7f80ef7fe640 1 --2- 172.21.15.98:0/1572026811 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80f014b220 0x7f80f0152010 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f80e000dae0 tx=0x7f80e000dee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:35.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.310+0000 7f80e7fff640 1 -- 172.21.15.98:0/1572026811 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e0019070 con 0x7f80f014b220 2024-09-19T08:41:35.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.314+0000 7f80e7fff640 1 -- 172.21.15.98:0/1572026811 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80e0004cc0 con 0x7f80f014b220 2024-09-19T08:41:35.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.314+0000 7f80e7fff640 1 -- 172.21.15.98:0/1572026811 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e00072d0 con 0x7f80f014b220 2024-09-19T08:41:35.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.314+0000 7f80f602e640 1 -- 172.21.15.98:0/1572026811 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80f0153390 con 0x7f80f014b220 2024-09-19T08:41:35.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.314+0000 7f80f602e640 1 -- 172.21.15.98:0/1572026811 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80f0153950 con 0x7f80f014b220 2024-09-19T08:41:35.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.314+0000 7f80f602e640 1 -- 172.21.15.98:0/1572026811 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80f00cf690 con 0x7f80f014b220 2024-09-19T08:41:35.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.314+0000 7f80e7fff640 1 -- 172.21.15.98:0/1572026811 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f80e001c070 con 0x7f80f014b220 2024-09-19T08:41:35.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.314+0000 7f80e7fff640 1 --2- 172.21.15.98:0/1572026811 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f80d4067aa0 0x7f80d4069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:35.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.314+0000 7f80eeffd640 1 --2- 172.21.15.98:0/1572026811 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f80d4067aa0 0x7f80d4069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:35.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.314+0000 7f80e7fff640 1 -- 172.21.15.98:0/1572026811 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f80e0085bd0 con 0x7f80f014b220 2024-09-19T08:41:35.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.314+0000 7f80eeffd640 1 --2- 172.21.15.98:0/1572026811 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f80d4067aa0 0x7f80d4069f60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f80d8009540 tx=0x7f80d800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:35.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.318+0000 7f80e7fff640 1 -- 172.21.15.98:0/1572026811 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f80e00563d0 con 0x7f80f014b220 2024-09-19T08:41:35.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.510+0000 7f80f602e640 1 -- 172.21.15.98:0/1572026811 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80f00d0620 con 0x7f80f014b220 2024-09-19T08:41:35.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.510+0000 7f80e7fff640 1 -- 172.21.15.98:0/1572026811 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5229 (secure 0 0 0) 0x7f80e0055cd0 con 0x7f80f014b220 2024-09-19T08:41:35.510 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:41:35.510 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"btime":"2024-09-19T08:39:55:181532+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/2874086421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":2874086421},{"type":"v1","addr":"172.21.15.139:6833","nonce":2874086421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4996,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/4178563848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":4178563848},{"type":"v1","addr":"172.21.15.139:6835","nonce":4178563848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3584568343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3584568343},{"type":"v1","addr":"172.21.15.98:6835","nonce":3584568343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:39:38.262938+0000","modified":"2024-09-19T08:39:41.276985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/3085042202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3085042202},{"type":"v1","addr":"172.21.15.98:6837","nonce":3085042202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244]},"id":3}]} 2024-09-19T08:41:35.510 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 24 2024-09-19T08:41:35.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.510+0000 7f80f602e640 1 -- 172.21.15.98:0/1572026811 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f80d4067aa0 msgr2=0x7f80d4069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:35.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.510+0000 7f80f602e640 1 --2- 172.21.15.98:0/1572026811 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f80d4067aa0 0x7f80d4069f60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f80d8009540 tx=0x7f80d800b040 comp rx=0 tx=0).stop 2024-09-19T08:41:35.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.510+0000 7f80f602e640 1 -- 172.21.15.98:0/1572026811 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80f014b220 msgr2=0x7f80f0152010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:35.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.510+0000 7f80f602e640 1 --2- 172.21.15.98:0/1572026811 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80f014b220 0x7f80f0152010 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f80e000dae0 tx=0x7f80e000dee0 comp rx=0 tx=0).stop 2024-09-19T08:41:35.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.510+0000 7f80f602e640 1 -- 172.21.15.98:0/1572026811 shutdown_connections 2024-09-19T08:41:35.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.510+0000 7f80f602e640 1 --2- 172.21.15.98:0/1572026811 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f80d4067aa0 0x7f80d4069f60 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:35.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.510+0000 7f80f602e640 1 --2- 172.21.15.98:0/1572026811 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80f015f9c0 0x7f80f00b4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:35.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.510+0000 7f80f602e640 1 --2- 172.21.15.98:0/1572026811 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80f0152c20 0x7f80f00b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:35.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.510+0000 7f80f602e640 1 --2- 172.21.15.98:0/1572026811 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80f014b220 0x7f80f0152010 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:35.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.510+0000 7f80f602e640 1 -- 172.21.15.98:0/1572026811 >> 172.21.15.98:0/1572026811 conn(0x7f80f00b2010 msgr2=0x7f80f00ae8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:35.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.510+0000 7f80f602e640 1 -- 172.21.15.98:0/1572026811 shutdown_connections 2024-09-19T08:41:35.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.514+0000 7f80f602e640 1 -- 172.21.15.98:0/1572026811 wait complete. 2024-09-19T08:41:35.528 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:41:35.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.646+0000 7fdfa7c70640 1 Processor -- start 2024-09-19T08:41:35.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.650+0000 7fdfa7c70640 1 -- start start 2024-09-19T08:41:35.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.650+0000 7fdfa7c70640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fdfa0158fd0 0x7fdfa00b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:35.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.650+0000 7fdfa7c70640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdfa006c3a0 0x7fdfa00ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:35.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.650+0000 7fdfa7c70640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfa00ccc60 con 0x7fdfa00b4360 2024-09-19T08:41:35.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.650+0000 7fdfa7c70640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fdfa01593b0 con 0x7fdfa0158fd0 2024-09-19T08:41:35.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.650+0000 7fdfa7c70640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdfa00ccf40 con 0x7fdfa006c3a0 2024-09-19T08:41:35.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.650+0000 7fdfa520b640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fdfa00b4360 0x7fdfa006bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:58310/0 (socket says 172.21.15.98:58310) 2024-09-19T08:41:35.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.650+0000 7fdfa520b640 1 -- 172.21.15.98:0/1043137480 learned_addr learned my addr 172.21.15.98:0/1043137480 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:41:35.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.650+0000 7fdfa5a0c640 1 --2- 172.21.15.98:0/1043137480 >> v2:172.21.15.98:3300/0 conn(0x7fdfa0158fd0 0x7fdfa00b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:35.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.650+0000 7fdfa620d640 1 --2- 172.21.15.98:0/1043137480 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdfa006c3a0 0x7fdfa00ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:35.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.650+0000 7fdfa4a0a640 1 -- 172.21.15.98:0/1043137480 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3713791310 0 0) 0x7fdfa00ccc60 con 0x7fdfa00b4360 2024-09-19T08:41:35.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.650+0000 7fdfa4a0a640 1 -- 172.21.15.98:0/1043137480 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf88003530 con 0x7fdfa00b4360 2024-09-19T08:41:35.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.650+0000 7fdfa5a0c640 1 -- 172.21.15.98:0/1043137480 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdfa006c3a0 msgr2=0x7fdfa00ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:35.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.650+0000 7fdfa5a0c640 1 --2- 172.21.15.98:0/1043137480 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdfa006c3a0 0x7fdfa00ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:35.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.650+0000 7fdfa5a0c640 1 -- 172.21.15.98:0/1043137480 >> v1:172.21.15.98:6789/0 conn(0x7fdfa00b4360 legacy=0x7fdfa006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:35.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.650+0000 7fdfa5a0c640 1 -- 172.21.15.98:0/1043137480 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa00cd220 con 0x7fdfa0158fd0 2024-09-19T08:41:35.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.650+0000 7fdfa5a0c640 1 --2- 172.21.15.98:0/1043137480 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfa0158fd0 0x7fdfa00b3e20 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdf980097c0 tx=0x7fdf9801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766adfa7eb596046 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:35.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.650+0000 7fdfa4a0a640 1 -- 172.21.15.98:0/1043137480 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf980b1070 con 0x7fdfa0158fd0 2024-09-19T08:41:35.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.650+0000 7fdfa4a0a640 1 -- 172.21.15.98:0/1043137480 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf980a8050 con 0x7fdfa0158fd0 2024-09-19T08:41:35.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.650+0000 7fdfa4a0a640 1 -- 172.21.15.98:0/1043137480 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf980ac860 con 0x7fdfa0158fd0 2024-09-19T08:41:35.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.654+0000 7fdfa7c70640 1 -- 172.21.15.98:0/1043137480 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfa0158fd0 msgr2=0x7fdfa00b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:35.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.654+0000 7fdfa7c70640 1 --2- 172.21.15.98:0/1043137480 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfa0158fd0 0x7fdfa00b3e20 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdf980097c0 tx=0x7fdf9801c3e0 comp rx=0 tx=0).stop 2024-09-19T08:41:35.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.654+0000 7fdfa7c70640 1 -- 172.21.15.98:0/1043137480 shutdown_connections 2024-09-19T08:41:35.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.654+0000 7fdfa7c70640 1 --2- 172.21.15.98:0/1043137480 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdfa006c3a0 0x7fdfa00ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:35.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.654+0000 7fdfa7c70640 1 --2- 172.21.15.98:0/1043137480 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfa0158fd0 0x7fdfa00b3e20 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:35.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.654+0000 7fdfa7c70640 1 -- 172.21.15.98:0/1043137480 >> 172.21.15.98:0/1043137480 conn(0x7fdfa00b2010 msgr2=0x7fdfa00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:35.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.654+0000 7fdfa7c70640 1 -- 172.21.15.98:0/1043137480 shutdown_connections 2024-09-19T08:41:35.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.654+0000 7fdfa7c70640 1 -- 172.21.15.98:0/1043137480 wait complete. 2024-09-19T08:41:35.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.654+0000 7fdfa7c70640 1 Processor -- start 2024-09-19T08:41:35.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.654+0000 7fdfa7c70640 1 -- start start 2024-09-19T08:41:35.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.654+0000 7fdfa7c70640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfa006c3a0 0x7fdfa0156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:35.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.654+0000 7fdfa7c70640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdfa00b4360 0x7fdfa0154c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:35.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.654+0000 7fdfa7c70640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfa0158fd0 0x7fdfa01f7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:35.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.654+0000 7fdfa7c70640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa00cf710 con 0x7fdfa0158fd0 2024-09-19T08:41:35.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.654+0000 7fdfa7c70640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa00cf590 con 0x7fdfa006c3a0 2024-09-19T08:41:35.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.654+0000 7fdfa7c70640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdfa00cf890 con 0x7fdfa00b4360 2024-09-19T08:41:35.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.654+0000 7fdfa620d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfa0158fd0 0x7fdfa01f7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:35.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.654+0000 7fdfa620d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfa0158fd0 0x7fdfa01f7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37706/0 (socket says 172.21.15.98:37706) 2024-09-19T08:41:35.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.654+0000 7fdfa620d640 1 -- 172.21.15.98:0/2433140966 learned_addr learned my addr 172.21.15.98:0/2433140966 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:35.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.654+0000 7fdfa520b640 1 --2- 172.21.15.98:0/2433140966 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdfa00b4360 0x7fdfa0154c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:35.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.654+0000 7fdfa5a0c640 1 --2- 172.21.15.98:0/2433140966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfa006c3a0 0x7fdfa0156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:35.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.654+0000 7fdfa620d640 1 -- 172.21.15.98:0/2433140966 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdfa00b4360 msgr2=0x7fdfa0154c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:35.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.654+0000 7fdfa620d640 1 --2- 172.21.15.98:0/2433140966 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdfa00b4360 0x7fdfa0154c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:35.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.654+0000 7fdfa620d640 1 -- 172.21.15.98:0/2433140966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfa006c3a0 msgr2=0x7fdfa0156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:35.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.654+0000 7fdfa620d640 1 --2- 172.21.15.98:0/2433140966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfa006c3a0 0x7fdfa0156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:35.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.654+0000 7fdfa620d640 1 -- 172.21.15.98:0/2433140966 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa0156790 con 0x7fdfa0158fd0 2024-09-19T08:41:35.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.654+0000 7fdfa5a0c640 1 --2- 172.21.15.98:0/2433140966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfa006c3a0 0x7fdfa0156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:35.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.654+0000 7fdfa620d640 1 --2- 172.21.15.98:0/2433140966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfa0158fd0 0x7fdfa01f7240 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdf90007e60 tx=0x7fdf9000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:35.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.654+0000 7fdf9e7fc640 1 -- 172.21.15.98:0/2433140966 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf90014420 con 0x7fdfa0158fd0 2024-09-19T08:41:35.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.654+0000 7fdf9e7fc640 1 -- 172.21.15.98:0/2433140966 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf90019070 con 0x7fdfa0158fd0 2024-09-19T08:41:35.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.654+0000 7fdfa7c70640 1 -- 172.21.15.98:0/2433140966 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfa01571d0 con 0x7fdfa0158fd0 2024-09-19T08:41:35.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.654+0000 7fdf9e7fc640 1 -- 172.21.15.98:0/2433140966 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf90014820 con 0x7fdfa0158fd0 2024-09-19T08:41:35.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.654+0000 7fdfa7c70640 1 -- 172.21.15.98:0/2433140966 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfa0155490 con 0x7fdfa0158fd0 2024-09-19T08:41:35.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.658+0000 7fdfa7c70640 1 -- 172.21.15.98:0/2433140966 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfa0071220 con 0x7fdfa0158fd0 2024-09-19T08:41:35.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.658+0000 7fdf9e7fc640 1 -- 172.21.15.98:0/2433140966 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fdf900149c0 con 0x7fdfa0158fd0 2024-09-19T08:41:35.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.658+0000 7fdf9e7fc640 1 --2- 172.21.15.98:0/2433140966 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdf8c067a70 0x7fdf8c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:35.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.658+0000 7fdf9e7fc640 1 -- 172.21.15.98:0/2433140966 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7fdf900868f0 con 0x7fdfa0158fd0 2024-09-19T08:41:35.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.658+0000 7fdfa5a0c640 1 --2- 172.21.15.98:0/2433140966 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdf8c067a70 0x7fdf8c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:35.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.658+0000 7fdfa5a0c640 1 --2- 172.21.15.98:0/2433140966 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdf8c067a70 0x7fdf8c069f30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdf980ae000 tx=0x7fdf98008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:35.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.662+0000 7fdf9e7fc640 1 -- 172.21.15.98:0/2433140966 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fdf900570f0 con 0x7fdfa0158fd0 2024-09-19T08:41:35.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.858+0000 7fdfa7c70640 1 -- 172.21.15.98:0/2433140966 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdfa00ce7c0 con 0x7fdfa0158fd0 2024-09-19T08:41:35.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.862+0000 7fdf9e7fc640 1 -- 172.21.15.98:0/2433140966 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5229 (secure 0 0 0) 0x7fdf900569f0 con 0x7fdfa0158fd0 2024-09-19T08:41:35.861 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:41:35.861 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":24,"btime":"2024-09-19T08:39:55:181532+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/2874086421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":2874086421},{"type":"v1","addr":"172.21.15.139:6833","nonce":2874086421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4996,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/4178563848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":4178563848},{"type":"v1","addr":"172.21.15.139:6835","nonce":4178563848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3584568343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3584568343},{"type":"v1","addr":"172.21.15.98:6835","nonce":3584568343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:39:38.262938+0000","modified":"2024-09-19T08:39:41.276985+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5244},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5244":{"gid":5244,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/3085042202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3085042202},{"type":"v1","addr":"172.21.15.98:6837","nonce":3085042202}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5244,"qdb_cluster":[5244]},"id":3}]} 2024-09-19T08:41:35.862 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 24 2024-09-19T08:41:35.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.862+0000 7fdfa7c70640 1 -- 172.21.15.98:0/2433140966 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdf8c067a70 msgr2=0x7fdf8c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:35.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.862+0000 7fdfa7c70640 1 --2- 172.21.15.98:0/2433140966 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdf8c067a70 0x7fdf8c069f30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdf980ae000 tx=0x7fdf98008040 comp rx=0 tx=0).stop 2024-09-19T08:41:35.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.862+0000 7fdfa7c70640 1 -- 172.21.15.98:0/2433140966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfa0158fd0 msgr2=0x7fdfa01f7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:35.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.862+0000 7fdfa7c70640 1 --2- 172.21.15.98:0/2433140966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfa0158fd0 0x7fdfa01f7240 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdf90007e60 tx=0x7fdf9000e4b0 comp rx=0 tx=0).stop 2024-09-19T08:41:35.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.862+0000 7fdfa7c70640 1 -- 172.21.15.98:0/2433140966 shutdown_connections 2024-09-19T08:41:35.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.862+0000 7fdfa7c70640 1 --2- 172.21.15.98:0/2433140966 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdf8c067a70 0x7fdf8c069f30 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:35.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.862+0000 7fdfa7c70640 1 --2- 172.21.15.98:0/2433140966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdfa0158fd0 0x7fdfa01f7240 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:35.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.862+0000 7fdfa7c70640 1 --2- 172.21.15.98:0/2433140966 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdfa00b4360 0x7fdfa0154c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:35.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.862+0000 7fdfa7c70640 1 --2- 172.21.15.98:0/2433140966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfa006c3a0 0x7fdfa0156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:35.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.862+0000 7fdfa7c70640 1 -- 172.21.15.98:0/2433140966 >> 172.21.15.98:0/2433140966 conn(0x7fdfa00b2010 msgr2=0x7fdfa00b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:35.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.862+0000 7fdfa7c70640 1 -- 172.21.15.98:0/2433140966 shutdown_connections 2024-09-19T08:41:35.865 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.862+0000 7fdfa7c70640 1 -- 172.21.15.98:0/2433140966 wait complete. 2024-09-19T08:41:35.880 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:41:35.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.994+0000 7f66e5d92640 1 Processor -- start 2024-09-19T08:41:35.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.994+0000 7f66e5d92640 1 -- start start 2024-09-19T08:41:35.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.998+0000 7f66e5d92640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f66e006d880 0x7f66e00b3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:35.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.998+0000 7f66e5d92640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f66e0141110 0x7f66e00b3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:35.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.998+0000 7f66e5d92640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66e015ea10 con 0x7f66e0140d30 2024-09-19T08:41:35.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.998+0000 7f66e5d92640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f66e015e7c0 con 0x7f66e0141110 2024-09-19T08:41:35.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.998+0000 7f66e5d92640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f66e015ecf0 con 0x7f66e006d880 2024-09-19T08:41:35.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.998+0000 7f66dffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f66e0141110 0x7f66e00b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:35.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.998+0000 7f66dffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f66e0141110 0x7f66e00b3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37714/0 (socket says 172.21.15.98:37714) 2024-09-19T08:41:35.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.998+0000 7f66dffff640 1 -- 172.21.15.98:0/796896818 learned_addr learned my addr 172.21.15.98:0/796896818 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:35.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.998+0000 7f66deffd640 1 --2- 172.21.15.98:0/796896818 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f66e006d880 0x7f66e00b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:35.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.998+0000 7f66dffff640 1 -- 172.21.15.98:0/796896818 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f66e006d880 msgr2=0x7f66e00b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:35.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.998+0000 7f66dffff640 1 --2- 172.21.15.98:0/796896818 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f66e006d880 0x7f66e00b3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:35.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.998+0000 7f66dffff640 1 -- 172.21.15.98:0/796896818 >> v1:172.21.15.139:6789/0 conn(0x7f66e0140d30 legacy=0x7f66e006d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:35.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.998+0000 7f66dffff640 1 -- 172.21.15.98:0/796896818 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e015efd0 con 0x7f66e0141110 2024-09-19T08:41:36.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.998+0000 7f66dffff640 1 --2- 172.21.15.98:0/796896818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66e0141110 0x7f66e00b3850 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f66e015e940 tx=0x7f66d801ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84f319fcd4d0917 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:36.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.998+0000 7f66de7fc640 1 -- 172.21.15.98:0/796896818 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d80a5900 con 0x7f66e0141110 2024-09-19T08:41:36.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.998+0000 7f66de7fc640 1 -- 172.21.15.98:0/796896818 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66d80a5aa0 con 0x7f66e0141110 2024-09-19T08:41:36.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:35.998+0000 7f66de7fc640 1 -- 172.21.15.98:0/796896818 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d80ad3f0 con 0x7f66e0141110 2024-09-19T08:41:36.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.002+0000 7f66e5d92640 1 -- 172.21.15.98:0/796896818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66e0141110 msgr2=0x7f66e00b3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:36.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.002+0000 7f66e5d92640 1 --2- 172.21.15.98:0/796896818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66e0141110 0x7f66e00b3850 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f66e015e940 tx=0x7f66d801ca10 comp rx=0 tx=0).stop 2024-09-19T08:41:36.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.002+0000 7f66e5d92640 1 -- 172.21.15.98:0/796896818 shutdown_connections 2024-09-19T08:41:36.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.002+0000 7f66e5d92640 1 --2- 172.21.15.98:0/796896818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66e0141110 0x7f66e00b3850 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:36.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.002+0000 7f66e5d92640 1 --2- 172.21.15.98:0/796896818 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f66e006d880 0x7f66e00b3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:36.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.002+0000 7f66e5d92640 1 -- 172.21.15.98:0/796896818 >> 172.21.15.98:0/796896818 conn(0x7f66e00b2010 msgr2=0x7f66e00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:36.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.002+0000 7f66e5d92640 1 -- 172.21.15.98:0/796896818 shutdown_connections 2024-09-19T08:41:36.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.002+0000 7f66e5d92640 1 -- 172.21.15.98:0/796896818 wait complete. 2024-09-19T08:41:36.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.002+0000 7f66e5d92640 1 Processor -- start 2024-09-19T08:41:36.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.002+0000 7f66e5d92640 1 -- start start 2024-09-19T08:41:36.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.002+0000 7f66e5d92640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66e006d880 0x7f66e00c7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:36.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.002+0000 7f66e5d92640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66e0140d30 0x7f66e0140060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:36.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.002+0000 7f66e5d92640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f66e0141110 0x7f66e0163d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:36.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.002+0000 7f66e5d92640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f66e015bd90 con 0x7f66e006d880 2024-09-19T08:41:36.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.002+0000 7f66e5d92640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f66e015bc10 con 0x7f66e0140d30 2024-09-19T08:41:36.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.002+0000 7f66e5d92640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f66e015bf10 con 0x7f66e0141110 2024-09-19T08:41:36.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.002+0000 7f66df7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66e006d880 0x7f66e00c7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:36.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.002+0000 7f66df7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66e006d880 0x7f66e00c7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37728/0 (socket says 172.21.15.98:37728) 2024-09-19T08:41:36.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.002+0000 7f66df7fe640 1 -- 172.21.15.98:0/1493877610 learned_addr learned my addr 172.21.15.98:0/1493877610 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:36.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.002+0000 7f66deffd640 1 --2- 172.21.15.98:0/1493877610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66e0140d30 0x7f66e0140060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:36.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.002+0000 7f66dffff640 1 --2- 172.21.15.98:0/1493877610 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f66e0141110 0x7f66e0163d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:36.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.002+0000 7f66df7fe640 1 -- 172.21.15.98:0/1493877610 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f66e0141110 msgr2=0x7f66e0163d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:36.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.002+0000 7f66df7fe640 1 --2- 172.21.15.98:0/1493877610 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f66e0141110 0x7f66e0163d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:36.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.002+0000 7f66df7fe640 1 -- 172.21.15.98:0/1493877610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66e0140d30 msgr2=0x7f66e0140060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:36.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.002+0000 7f66df7fe640 1 --2- 172.21.15.98:0/1493877610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66e0140d30 0x7f66e0140060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:36.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.002+0000 7f66df7fe640 1 -- 172.21.15.98:0/1493877610 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e01405a0 con 0x7f66e006d880 2024-09-19T08:41:36.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.002+0000 7f66df7fe640 1 --2- 172.21.15.98:0/1493877610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66e006d880 0x7f66e00c7390 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f66d000c420 tx=0x7f66d000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:36.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.002+0000 7f66d7fff640 1 -- 172.21.15.98:0/1493877610 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d00052d0 con 0x7f66e006d880 2024-09-19T08:41:36.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.002+0000 7f66d7fff640 1 -- 172.21.15.98:0/1493877610 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66d0005470 con 0x7f66e006d880 2024-09-19T08:41:36.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.002+0000 7f66e5d92640 1 -- 172.21.15.98:0/1493877610 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66e00b1970 con 0x7f66e006d880 2024-09-19T08:41:36.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.002+0000 7f66d7fff640 1 -- 172.21.15.98:0/1493877610 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d0007260 con 0x7f66e006d880 2024-09-19T08:41:36.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.002+0000 7f66e5d92640 1 -- 172.21.15.98:0/1493877610 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66e0161b50 con 0x7f66e006d880 2024-09-19T08:41:36.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.002+0000 7f66e5d92640 1 -- 172.21.15.98:0/1493877610 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66e0161900 con 0x7f66e006d880 2024-09-19T08:41:36.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.006+0000 7f66d7fff640 1 -- 172.21.15.98:0/1493877610 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f66d0007400 con 0x7f66e006d880 2024-09-19T08:41:36.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.006+0000 7f66d7fff640 1 --2- 172.21.15.98:0/1493877610 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f66c0067a70 0x7f66c0069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:36.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.006+0000 7f66d7fff640 1 -- 172.21.15.98:0/1493877610 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f66d0085520 con 0x7f66e006d880 2024-09-19T08:41:36.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.006+0000 7f66deffd640 1 --2- 172.21.15.98:0/1493877610 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f66c0067a70 0x7f66c0069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:36.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.006+0000 7f66deffd640 1 --2- 172.21.15.98:0/1493877610 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f66c0067a70 0x7f66c0069f30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f66e00cc510 tx=0x7f66c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:36.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.010+0000 7f66d7fff640 1 -- 172.21.15.98:0/1493877610 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f66e01405a0 con 0x7f66e006d880 2024-09-19T08:41:36.162 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.162+0000 7f66e5d92640 1 -- 172.21.15.98:0/1493877610 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f66e0161f30 con 0x7f66e006d880 2024-09-19T08:41:36.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.166+0000 7f66d7fff640 1 -- 172.21.15.98:0/1493877610 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16825 (secure 0 0 0) 0x7f66d0055da0 con 0x7f66e006d880 2024-09-19T08:41:36.165 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:41:36.165 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":45,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:40:01.212861+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:39:36.829570+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-19T08:39:37.237199+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.30","mappings":[{"from":0,"to":5}]},{"pgid":"7.32","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:41:36.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.166+0000 7f66e5d92640 1 -- 172.21.15.98:0/1493877610 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f66c0067a70 msgr2=0x7f66c0069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:36.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.166+0000 7f66e5d92640 1 --2- 172.21.15.98:0/1493877610 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f66c0067a70 0x7f66c0069f30 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f66e00cc510 tx=0x7f66c8008040 comp rx=0 tx=0).stop 2024-09-19T08:41:36.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.166+0000 7f66e5d92640 1 -- 172.21.15.98:0/1493877610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66e006d880 msgr2=0x7f66e00c7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:36.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.166+0000 7f66e5d92640 1 --2- 172.21.15.98:0/1493877610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66e006d880 0x7f66e00c7390 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f66d000c420 tx=0x7f66d000c8e0 comp rx=0 tx=0).stop 2024-09-19T08:41:36.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.166+0000 7f66e5d92640 1 -- 172.21.15.98:0/1493877610 shutdown_connections 2024-09-19T08:41:36.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.166+0000 7f66e5d92640 1 --2- 172.21.15.98:0/1493877610 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f66c0067a70 0x7f66c0069f30 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:36.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.166+0000 7f66e5d92640 1 --2- 172.21.15.98:0/1493877610 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f66e0141110 0x7f66e0163d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:36.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.166+0000 7f66e5d92640 1 --2- 172.21.15.98:0/1493877610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66e0140d30 0x7f66e0140060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:36.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.166+0000 7f66e5d92640 1 --2- 172.21.15.98:0/1493877610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66e006d880 0x7f66e00c7390 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:36.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.166+0000 7f66e5d92640 1 -- 172.21.15.98:0/1493877610 >> 172.21.15.98:0/1493877610 conn(0x7f66e00b2010 msgr2=0x7f66e0158e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:36.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.166+0000 7f66e5d92640 1 -- 172.21.15.98:0/1493877610 shutdown_connections 2024-09-19T08:41:36.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.166+0000 7f66e5d92640 1 -- 172.21.15.98:0/1493877610 wait complete. 2024-09-19T08:41:36.182 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-19T08:41:36.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.298+0000 7f7b5a6fd640 1 Processor -- start 2024-09-19T08:41:36.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.298+0000 7f7b5a6fd640 1 -- start start 2024-09-19T08:41:36.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.298+0000 7f7b5a6fd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7b540b7f80 0x7f7b5415f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:36.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.298+0000 7f7b5a6fd640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b54068440 con 0x7f7b540b9970 2024-09-19T08:41:36.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.298+0000 7f7b5a6fd640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b54162a90 con 0x7f7b540d4820 2024-09-19T08:41:36.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.298+0000 7f7b5a6fd640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f7b54163050 con 0x7f7b540b7f80 2024-09-19T08:41:36.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.298+0000 7f7b53fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7b540b7f80 0x7f7b5415f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:36.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.298+0000 7f7b53fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7b540b7f80 0x7f7b5415f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37732/0 (socket says 172.21.15.98:37732) 2024-09-19T08:41:36.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.298+0000 7f7b58c9a640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f7b540b9970 0x7f7b540b9de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:58316/0 (socket says 172.21.15.98:58316) 2024-09-19T08:41:36.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.298+0000 7f7b53fff640 1 -- 172.21.15.98:0/1505210483 learned_addr learned my addr 172.21.15.98:0/1505210483 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:36.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.298+0000 7f7b53fff640 1 -- 172.21.15.98:0/1505210483 >> v1:172.21.15.139:6789/0 conn(0x7f7b540d4820 legacy=0x7f7b541621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:36.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.298+0000 7f7b53fff640 1 -- 172.21.15.98:0/1505210483 >> v1:172.21.15.98:6789/0 conn(0x7f7b540b9970 legacy=0x7f7b540b9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:36.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.298+0000 7f7b53fff640 1 -- 172.21.15.98:0/1505210483 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b54162d70 con 0x7f7b540b7f80 2024-09-19T08:41:36.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.298+0000 7f7b52ffd640 1 -- 172.21.15.98:0/1505210483 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4239505173 0 0) 0x7f7b54068440 con 0x7f7b540b9970 2024-09-19T08:41:36.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.302+0000 7f7b53fff640 1 --2- 172.21.15.98:0/1505210483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b540b7f80 0x7f7b5415f560 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c009870 tx=0x7f7b4c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13147f788127d78e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:36.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.302+0000 7f7b52ffd640 1 -- 172.21.15.98:0/1505210483 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c0b0070 con 0x7f7b540b7f80 2024-09-19T08:41:36.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.302+0000 7f7b52ffd640 1 -- 172.21.15.98:0/1505210483 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b4c0ab600 con 0x7f7b540b7f80 2024-09-19T08:41:36.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.302+0000 7f7b52ffd640 1 -- 172.21.15.98:0/1505210483 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c0ab9d0 con 0x7f7b540b7f80 2024-09-19T08:41:36.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.302+0000 7f7b5a6fd640 1 -- 172.21.15.98:0/1505210483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b540b7f80 msgr2=0x7f7b5415f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:36.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.302+0000 7f7b5a6fd640 1 --2- 172.21.15.98:0/1505210483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b540b7f80 0x7f7b5415f560 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c009870 tx=0x7f7b4c01c490 comp rx=0 tx=0).stop 2024-09-19T08:41:36.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.302+0000 7f7b5a6fd640 1 -- 172.21.15.98:0/1505210483 shutdown_connections 2024-09-19T08:41:36.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.302+0000 7f7b5a6fd640 1 --2- 172.21.15.98:0/1505210483 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b540b7f80 0x7f7b5415f560 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:36.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.302+0000 7f7b5a6fd640 1 -- 172.21.15.98:0/1505210483 >> 172.21.15.98:0/1505210483 conn(0x7f7b54074e40 msgr2=0x7f7b54075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:36.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.302+0000 7f7b5a6fd640 1 -- 172.21.15.98:0/1505210483 shutdown_connections 2024-09-19T08:41:36.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.302+0000 7f7b5a6fd640 1 -- 172.21.15.98:0/1505210483 wait complete. 2024-09-19T08:41:36.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.302+0000 7f7b5a6fd640 1 Processor -- start 2024-09-19T08:41:36.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.302+0000 7f7b5a6fd640 1 -- start start 2024-09-19T08:41:36.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.302+0000 7f7b5a6fd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b540b7f80 0x7f7b5413e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:36.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.302+0000 7f7b5a6fd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b540b9970 0x7f7b5413c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:36.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.302+0000 7f7b5a6fd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b540d4820 0x7f7b5413ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:36.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.302+0000 7f7b5a6fd640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7b54144710 con 0x7f7b540d4820 2024-09-19T08:41:36.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.302+0000 7f7b5a6fd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7b54144590 con 0x7f7b540b7f80 2024-09-19T08:41:36.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.302+0000 7f7b5a6fd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7b54144890 con 0x7f7b540b9970 2024-09-19T08:41:36.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.302+0000 7f7b58c9a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b540d4820 0x7f7b5413ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:36.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.302+0000 7f7b58c9a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b540d4820 0x7f7b5413ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37744/0 (socket says 172.21.15.98:37744) 2024-09-19T08:41:36.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.302+0000 7f7b58c9a640 1 -- 172.21.15.98:0/3627785644 learned_addr learned my addr 172.21.15.98:0/3627785644 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:36.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.302+0000 7f7b53fff640 1 --2- 172.21.15.98:0/3627785644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b540b7f80 0x7f7b5413e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:36.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.302+0000 7f7b58c9a640 1 -- 172.21.15.98:0/3627785644 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b540b9970 msgr2=0x7f7b5413c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:36.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.302+0000 7f7b537fe640 1 --2- 172.21.15.98:0/3627785644 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b540b9970 0x7f7b5413c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:36.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.302+0000 7f7b58c9a640 1 --2- 172.21.15.98:0/3627785644 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b540b9970 0x7f7b5413c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:36.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.302+0000 7f7b58c9a640 1 -- 172.21.15.98:0/3627785644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b540b7f80 msgr2=0x7f7b5413e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:36.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.302+0000 7f7b58c9a640 1 --2- 172.21.15.98:0/3627785644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b540b7f80 0x7f7b5413e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:36.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.302+0000 7f7b58c9a640 1 -- 172.21.15.98:0/3627785644 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b540b7070 con 0x7f7b540d4820 2024-09-19T08:41:36.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.302+0000 7f7b53fff640 1 --2- 172.21.15.98:0/3627785644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b540b7f80 0x7f7b5413e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:36.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.302+0000 7f7b58c9a640 1 --2- 172.21.15.98:0/3627785644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b540d4820 0x7f7b5413ba70 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f7b4400c3c0 tx=0x7f7b4400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:36.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.302+0000 7f7b50ff9640 1 -- 172.21.15.98:0/3627785644 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b440074e0 con 0x7f7b540d4820 2024-09-19T08:41:36.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.302+0000 7f7b50ff9640 1 -- 172.21.15.98:0/3627785644 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b440047f0 con 0x7f7b540d4820 2024-09-19T08:41:36.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.302+0000 7f7b5a6fd640 1 -- 172.21.15.98:0/3627785644 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b5413cb20 con 0x7f7b540d4820 2024-09-19T08:41:36.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.302+0000 7f7b50ff9640 1 -- 172.21.15.98:0/3627785644 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b44007680 con 0x7f7b540d4820 2024-09-19T08:41:36.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.306+0000 7f7b5a6fd640 1 -- 172.21.15.98:0/3627785644 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b54156eb0 con 0x7f7b540d4820 2024-09-19T08:41:36.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.306+0000 7f7b5a6fd640 1 -- 172.21.15.98:0/3627785644 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b5413d040 con 0x7f7b540d4820 2024-09-19T08:41:36.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.306+0000 7f7b50ff9640 1 -- 172.21.15.98:0/3627785644 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f7b44011070 con 0x7f7b540d4820 2024-09-19T08:41:36.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.306+0000 7f7b50ff9640 1 --2- 172.21.15.98:0/3627785644 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7b34067a70 0x7f7b34069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:36.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.306+0000 7f7b50ff9640 1 -- 172.21.15.98:0/3627785644 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f7b44085b30 con 0x7f7b540d4820 2024-09-19T08:41:36.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.306+0000 7f7b53fff640 1 --2- 172.21.15.98:0/3627785644 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7b34067a70 0x7f7b34069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:36.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.306+0000 7f7b53fff640 1 --2- 172.21.15.98:0/3627785644 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7b34067a70 0x7f7b34069f30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c0ad000 tx=0x7f7b4c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:36.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.310+0000 7f7b50ff9640 1 -- 172.21.15.98:0/3627785644 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f7b44056440 con 0x7f7b540d4820 2024-09-19T08:41:36.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.494+0000 7f7b5a6fd640 1 -- 172.21.15.98:0/3627785644 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7b54156ca0 con 0x7f7b540d4820 2024-09-19T08:41:36.493 INFO:tasks.ceph.mon.a.smithi098.stderr:2024-09-19T08:41:36.494+0000 7f5e49017640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T08:41:36.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.494+0000 7f7b50ff9640 1 -- 172.21.15.98:0/3627785644 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) ==== 161+0+0 (secure 0 0 0) 0x7f7b44055d40 con 0x7f7b540d4820 2024-09-19T08:41:36.496 INFO:teuthology.orchestra.run.smithi098.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T08:41:36.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.498+0000 7f7b5a6fd640 1 -- 172.21.15.98:0/3627785644 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7b34067a70 msgr2=0x7f7b34069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:36.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.498+0000 7f7b5a6fd640 1 --2- 172.21.15.98:0/3627785644 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7b34067a70 0x7f7b34069f30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c0ad000 tx=0x7f7b4c008040 comp rx=0 tx=0).stop 2024-09-19T08:41:36.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.498+0000 7f7b5a6fd640 1 -- 172.21.15.98:0/3627785644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b540d4820 msgr2=0x7f7b5413ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:36.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.498+0000 7f7b5a6fd640 1 --2- 172.21.15.98:0/3627785644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b540d4820 0x7f7b5413ba70 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f7b4400c3c0 tx=0x7f7b4400c880 comp rx=0 tx=0).stop 2024-09-19T08:41:36.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.498+0000 7f7b5a6fd640 1 -- 172.21.15.98:0/3627785644 shutdown_connections 2024-09-19T08:41:36.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.498+0000 7f7b5a6fd640 1 --2- 172.21.15.98:0/3627785644 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7b34067a70 0x7f7b34069f30 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:36.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.498+0000 7f7b5a6fd640 1 --2- 172.21.15.98:0/3627785644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b540d4820 0x7f7b5413ba70 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:36.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.498+0000 7f7b5a6fd640 1 --2- 172.21.15.98:0/3627785644 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b540b9970 0x7f7b5413c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:36.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.498+0000 7f7b5a6fd640 1 --2- 172.21.15.98:0/3627785644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b540b7f80 0x7f7b5413e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:36.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.498+0000 7f7b5a6fd640 1 -- 172.21.15.98:0/3627785644 >> 172.21.15.98:0/3627785644 conn(0x7f7b54074e40 msgr2=0x7f7b540d2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:36.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.498+0000 7f7b5a6fd640 1 -- 172.21.15.98:0/3627785644 shutdown_connections 2024-09-19T08:41:36.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.498+0000 7f7b5a6fd640 1 -- 172.21.15.98:0/3627785644 wait complete. 2024-09-19T08:41:36.517 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T08:41:36.548 INFO:tasks.ceph.mds.c.smithi098.stdout:starting mds.c at 2024-09-19T08:41:36.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.630+0000 7f8d7fc14640 1 Processor -- start 2024-09-19T08:41:36.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.630+0000 7f8d7fc14640 1 -- start start 2024-09-19T08:41:36.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.630+0000 7f8d7fc14640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8d780d3a70 0x7f8d78169100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:36.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.630+0000 7f8d7fc14640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d7815eae0 con 0x7f8d780d27c0 2024-09-19T08:41:36.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.630+0000 7f8d7fc14640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d7815edc0 con 0x7f8d781415a0 2024-09-19T08:41:36.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.630+0000 7f8d7fc14640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8d7815f0a0 con 0x7f8d780d3a70 2024-09-19T08:41:36.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.630+0000 7f8d7d1af640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f8d780d27c0 0x7f8d780d3360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:58336/0 (socket says 172.21.15.98:58336) 2024-09-19T08:41:36.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.630+0000 7f8d7d1af640 1 -- 172.21.15.98:0/1825959531 learned_addr learned my addr 172.21.15.98:0/1825959531 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:41:36.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.630+0000 7f8d7e1b1640 1 --2- 172.21.15.98:0/1825959531 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8d780d3a70 0x7f8d78169100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:36.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.630+0000 7f8d7c9ae640 1 -- 172.21.15.98:0/1825959531 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4265421298 0 0) 0x7f8d7815eae0 con 0x7f8d780d27c0 2024-09-19T08:41:36.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.630+0000 7f8d7c9ae640 1 -- 172.21.15.98:0/1825959531 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d64003530 con 0x7f8d780d27c0 2024-09-19T08:41:36.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.630+0000 7f8d7c9ae640 1 -- 172.21.15.98:0/1825959531 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2884529059 0 0) 0x7f8d64003530 con 0x7f8d780d27c0 2024-09-19T08:41:36.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.630+0000 7f8d7c9ae640 1 -- 172.21.15.98:0/1825959531 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8d780d3a70 msgr2=0x7f8d78169100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:36.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.630+0000 7f8d7c9ae640 1 --2- 172.21.15.98:0/1825959531 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8d780d3a70 0x7f8d78169100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:36.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.630+0000 7f8d7c9ae640 1 -- 172.21.15.98:0/1825959531 >> v1:172.21.15.139:6789/0 conn(0x7f8d781415a0 legacy=0x7f8d780d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:36.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.630+0000 7f8d7c9ae640 1 -- 172.21.15.98:0/1825959531 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d7815f380 con 0x7f8d780d27c0 2024-09-19T08:41:36.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.630+0000 7f8d7c9ae640 1 -- 172.21.15.98:0/1825959531 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f8d68001680 con 0x7f8d780d27c0 2024-09-19T08:41:36.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.630+0000 7f8d7c9ae640 1 -- 172.21.15.98:0/1825959531 >> v1:172.21.15.98:6789/0 conn(0x7f8d780d27c0 legacy=0x7f8d780d3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:36.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.630+0000 7f8d7c9ae640 1 --2- 172.21.15.98:0/1825959531 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8d780d3a70 0x7f8d640039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:36.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.630+0000 7f8d7c9ae640 1 --2- 172.21.15.98:0/1825959531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8d64003f70 0x7f8d64004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:36.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.630+0000 7f8d7c9ae640 1 --2- 172.21.15.98:0/1825959531 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d64004d50 0x7f8d640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:36.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.630+0000 7f8d7c9ae640 1 -- 172.21.15.98:0/1825959531 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7815f0a0 con 0x7f8d64003f70 2024-09-19T08:41:36.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.630+0000 7f8d7c9ae640 1 -- 172.21.15.98:0/1825959531 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6400bab0 con 0x7f8d64004d50 2024-09-19T08:41:36.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.630+0000 7f8d7c9ae640 1 -- 172.21.15.98:0/1825959531 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8d6400bcb0 con 0x7f8d780d3a70 2024-09-19T08:41:36.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.630+0000 7f8d7d1af640 1 --2- 172.21.15.98:0/1825959531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8d64003f70 0x7f8d64004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:36.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.630+0000 7f8d7e1b1640 1 --2- 172.21.15.98:0/1825959531 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d64004d50 0x7f8d640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:36.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.630+0000 7f8d7d1af640 1 -- 172.21.15.98:0/1825959531 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8d780d3a70 msgr2=0x7f8d640039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:36.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.630+0000 7f8d7d9b0640 1 --2- 172.21.15.98:0/1825959531 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8d780d3a70 0x7f8d640039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:36.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.630+0000 7f8d7d1af640 1 --2- 172.21.15.98:0/1825959531 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8d780d3a70 0x7f8d640039f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:36.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.630+0000 7f8d7d1af640 1 -- 172.21.15.98:0/1825959531 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d64004d50 msgr2=0x7f8d640051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:36.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.630+0000 7f8d7d1af640 1 --2- 172.21.15.98:0/1825959531 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d64004d50 0x7f8d640051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:36.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.630+0000 7f8d7d1af640 1 -- 172.21.15.98:0/1825959531 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d64003530 con 0x7f8d64003f70 2024-09-19T08:41:36.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.630+0000 7f8d7e1b1640 1 --2- 172.21.15.98:0/1825959531 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d64004d50 0x7f8d640051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:36.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.634+0000 7f8d7d1af640 1 --2- 172.21.15.98:0/1825959531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8d64003f70 0x7f8d64004810 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f8d6800c210 tx=0x7f8d6801f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec8a859fc26f035f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:36.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.634+0000 7f8d7c9ae640 1 -- 172.21.15.98:0/1825959531 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d68007d40 con 0x7f8d64003f70 2024-09-19T08:41:36.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.634+0000 7f8d7c9ae640 1 -- 172.21.15.98:0/1825959531 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d68005b00 con 0x7f8d64003f70 2024-09-19T08:41:36.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.634+0000 7f8d7c9ae640 1 -- 172.21.15.98:0/1825959531 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d68007070 con 0x7f8d64003f70 2024-09-19T08:41:36.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.634+0000 7f8d7fc14640 1 -- 172.21.15.98:0/1825959531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8d64003f70 msgr2=0x7f8d64004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:36.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.634+0000 7f8d7fc14640 1 --2- 172.21.15.98:0/1825959531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8d64003f70 0x7f8d64004810 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f8d6800c210 tx=0x7f8d6801f3f0 comp rx=0 tx=0).stop 2024-09-19T08:41:36.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.634+0000 7f8d7e1b1640 1 -- 172.21.15.98:0/1825959531 reap_dead start 2024-09-19T08:41:36.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.634+0000 7f8d7fc14640 1 -- 172.21.15.98:0/1825959531 shutdown_connections 2024-09-19T08:41:36.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.634+0000 7f8d7fc14640 1 -- 172.21.15.98:0/1825959531 >> 172.21.15.98:0/1825959531 conn(0x7f8d78074c70 msgr2=0x7f8d78075080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:36.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.634+0000 7f8d7fc14640 1 -- 172.21.15.98:0/1825959531 shutdown_connections 2024-09-19T08:41:36.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.634+0000 7f8d7fc14640 1 -- 172.21.15.98:0/1825959531 wait complete. 2024-09-19T08:41:36.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.638+0000 7f8d7fc14640 1 Processor -- start 2024-09-19T08:41:36.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.638+0000 7f8d7fc14640 1 -- start start 2024-09-19T08:41:36.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.638+0000 7f8d7fc14640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8d781585f0 0x7f8d781615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:36.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.638+0000 7f8d7fc14640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8d78161ba0 0x7f8d780cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:36.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.638+0000 7f8d7fc14640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d780d0110 0x7f8d7815fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:36.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.638+0000 7f8d7fc14640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8d780d4120 con 0x7f8d781585f0 2024-09-19T08:41:36.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.638+0000 7f8d7fc14640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7815f8a0 con 0x7f8d780d0110 2024-09-19T08:41:36.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.638+0000 7f8d7fc14640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8d780d42a0 con 0x7f8d78161ba0 2024-09-19T08:41:36.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.638+0000 7f8d7d9b0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8d781585f0 0x7f8d781615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:36.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.638+0000 7f8d7d9b0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8d781585f0 0x7f8d781615e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37782/0 (socket says 172.21.15.98:37782) 2024-09-19T08:41:36.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.638+0000 7f8d7d9b0640 1 -- 172.21.15.98:0/387177439 learned_addr learned my addr 172.21.15.98:0/387177439 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:36.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.638+0000 7f8d7d1af640 1 --2- 172.21.15.98:0/387177439 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8d78161ba0 0x7f8d780cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:36.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.638+0000 7f8d7d9b0640 1 -- 172.21.15.98:0/387177439 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8d78161ba0 msgr2=0x7f8d780cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:36.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.638+0000 7f8d7d9b0640 1 --2- 172.21.15.98:0/387177439 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8d78161ba0 0x7f8d780cfbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:36.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.638+0000 7f8d7d9b0640 1 -- 172.21.15.98:0/387177439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d780d0110 msgr2=0x7f8d7815fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:36.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.638+0000 7f8d7d9b0640 1 --2- 172.21.15.98:0/387177439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d780d0110 0x7f8d7815fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:36.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.638+0000 7f8d7d9b0640 1 -- 172.21.15.98:0/387177439 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d78162210 con 0x7f8d781585f0 2024-09-19T08:41:36.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.638+0000 7f8d7d9b0640 1 --2- 172.21.15.98:0/387177439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8d781585f0 0x7f8d781615e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8d7400d770 tx=0x7f8d7400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:36.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.638+0000 7f8d6e7fc640 1 -- 172.21.15.98:0/387177439 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d74005270 con 0x7f8d781585f0 2024-09-19T08:41:36.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.638+0000 7f8d6e7fc640 1 -- 172.21.15.98:0/387177439 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d74017070 con 0x7f8d781585f0 2024-09-19T08:41:36.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.638+0000 7f8d7fc14640 1 -- 172.21.15.98:0/387177439 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d781624f0 con 0x7f8d781585f0 2024-09-19T08:41:36.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.638+0000 7f8d6e7fc640 1 -- 172.21.15.98:0/387177439 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d74003620 con 0x7f8d781585f0 2024-09-19T08:41:36.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.638+0000 7f8d7fc14640 1 -- 172.21.15.98:0/387177439 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d78162ab0 con 0x7f8d781585f0 2024-09-19T08:41:36.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.638+0000 7f8d7fc14640 1 -- 172.21.15.98:0/387177439 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d78161f80 con 0x7f8d781585f0 2024-09-19T08:41:36.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.638+0000 7f8d6e7fc640 1 -- 172.21.15.98:0/387177439 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f8d740037c0 con 0x7f8d781585f0 2024-09-19T08:41:36.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.638+0000 7f8d6e7fc640 1 --2- 172.21.15.98:0/387177439 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8d5c067a70 0x7f8d5c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:36.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.638+0000 7f8d6e7fc640 1 -- 172.21.15.98:0/387177439 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5853+0+0 (secure 0 0 0) 0x7f8d74086a80 con 0x7f8d781585f0 2024-09-19T08:41:36.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.638+0000 7f8d7d1af640 1 --2- 172.21.15.98:0/387177439 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8d5c067a70 0x7f8d5c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:36.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.642+0000 7f8d7d1af640 1 --2- 172.21.15.98:0/387177439 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8d5c067a70 0x7f8d5c069f30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8d680af000 tx=0x7f8d680aa000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:36.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.642+0000 7f8d6e7fc640 1 -- 172.21.15.98:0/387177439 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f8d74057180 con 0x7f8d781585f0 2024-09-19T08:41:36.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:36.842+0000 7f8d7fc14640 1 -- 172.21.15.98:0/387177439 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f8d780d3360 con 0x7f8d781585f0 2024-09-19T08:41:37.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.522+0000 7f8d6e7fc640 1 -- 172.21.15.98:0/387177439 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) ==== 104+0+0 (secure 0 0 0) 0x7f8d74056a80 con 0x7f8d781585f0 2024-09-19T08:41:37.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.526+0000 7f8d7fc14640 1 -- 172.21.15.98:0/387177439 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8d5c067a70 msgr2=0x7f8d5c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:37.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.526+0000 7f8d7fc14640 1 --2- 172.21.15.98:0/387177439 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8d5c067a70 0x7f8d5c069f30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8d680af000 tx=0x7f8d680aa000 comp rx=0 tx=0).stop 2024-09-19T08:41:37.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.526+0000 7f8d7fc14640 1 -- 172.21.15.98:0/387177439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8d781585f0 msgr2=0x7f8d781615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:37.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.526+0000 7f8d7fc14640 1 --2- 172.21.15.98:0/387177439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8d781585f0 0x7f8d781615e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8d7400d770 tx=0x7f8d7400dc30 comp rx=0 tx=0).stop 2024-09-19T08:41:37.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.526+0000 7f8d7fc14640 1 -- 172.21.15.98:0/387177439 shutdown_connections 2024-09-19T08:41:37.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.526+0000 7f8d7fc14640 1 --2- 172.21.15.98:0/387177439 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8d5c067a70 0x7f8d5c069f30 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:37.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.526+0000 7f8d7fc14640 1 --2- 172.21.15.98:0/387177439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8d780d0110 0x7f8d7815fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:37.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.526+0000 7f8d7fc14640 1 --2- 172.21.15.98:0/387177439 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8d78161ba0 0x7f8d780cfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:37.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.526+0000 7f8d7fc14640 1 --2- 172.21.15.98:0/387177439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8d781585f0 0x7f8d781615e0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:37.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.526+0000 7f8d7fc14640 1 -- 172.21.15.98:0/387177439 >> 172.21.15.98:0/387177439 conn(0x7f8d78074c70 msgr2=0x7f8d780ca2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:37.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.526+0000 7f8d7fc14640 1 -- 172.21.15.98:0/387177439 shutdown_connections 2024-09-19T08:41:37.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.526+0000 7f8d7fc14640 1 -- 172.21.15.98:0/387177439 wait complete. 2024-09-19T08:41:37.549 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T08:41:37.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.718+0000 7f07c4e6b640 1 Processor -- start 2024-09-19T08:41:37.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.718+0000 7f07c4e6b640 1 -- start start 2024-09-19T08:41:37.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.718+0000 7f07c4e6b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07c00b7770 0x7f07c00b7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:37.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.718+0000 7f07c4e6b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f07c00d3a60 0x7f07c00d3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:37.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.718+0000 7f07c4e6b640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07c00cfd20 con 0x7f07c015e720 2024-09-19T08:41:37.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.718+0000 7f07c4e6b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f07c00d29c0 con 0x7f07c00d3a60 2024-09-19T08:41:37.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.718+0000 7f07c4e6b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f07c00d2ca0 con 0x7f07c00b7770 2024-09-19T08:41:37.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.718+0000 7f07be59c640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f07c015e720 0x7f07c00cf610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:37092/0 (socket says 172.21.15.98:37092) 2024-09-19T08:41:37.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.718+0000 7f07be59c640 1 -- 172.21.15.98:0/505543786 learned_addr learned my addr 172.21.15.98:0/505543786 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:41:37.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.718+0000 7f07bed9d640 1 --2- 172.21.15.98:0/505543786 >> v2:172.21.15.139:3300/0 conn(0x7f07c00d3a60 0x7f07c00d3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:37.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.718+0000 7f07bd59a640 1 -- 172.21.15.98:0/505543786 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 923696589 0 0) 0x7f07c00cfd20 con 0x7f07c015e720 2024-09-19T08:41:37.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.718+0000 7f07bdd9b640 1 --2- 172.21.15.98:0/505543786 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07c00b7770 0x7f07c00b7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:37.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.718+0000 7f07bd59a640 1 -- 172.21.15.98:0/505543786 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07ac003530 con 0x7f07c015e720 2024-09-19T08:41:37.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.718+0000 7f07bd59a640 1 -- 172.21.15.98:0/505543786 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1026303198 0 0) 0x7f07ac003530 con 0x7f07c015e720 2024-09-19T08:41:37.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.718+0000 7f07bd59a640 1 -- 172.21.15.98:0/505543786 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07c00b7770 msgr2=0x7f07c00b7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:37.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.718+0000 7f07bd59a640 1 --2- 172.21.15.98:0/505543786 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07c00b7770 0x7f07c00b7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:37.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.718+0000 7f07bd59a640 1 -- 172.21.15.98:0/505543786 >> v2:172.21.15.139:3300/0 conn(0x7f07c00d3a60 msgr2=0x7f07c00d3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:37.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.718+0000 7f07bd59a640 1 --2- 172.21.15.98:0/505543786 >> v2:172.21.15.139:3300/0 conn(0x7f07c00d3a60 0x7f07c00d3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:37.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.718+0000 7f07bd59a640 1 -- 172.21.15.98:0/505543786 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c00d2f80 con 0x7f07c015e720 2024-09-19T08:41:37.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.718+0000 7f07bd59a640 1 -- 172.21.15.98:0/505543786 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f07b0001680 con 0x7f07c015e720 2024-09-19T08:41:37.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.718+0000 7f07bd59a640 1 -- 172.21.15.98:0/505543786 >> v1:172.21.15.98:6789/0 conn(0x7f07c015e720 legacy=0x7f07c00cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:37.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.722+0000 7f07bd59a640 1 --2- 172.21.15.98:0/505543786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07ac0039c0 0x7f07ac003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:37.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.722+0000 7f07bd59a640 1 --2- 172.21.15.98:0/505543786 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07c00b7770 0x7f07ac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:37.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.722+0000 7f07bd59a640 1 --2- 172.21.15.98:0/505543786 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07ac004d50 0x7f07ac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:37.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.722+0000 7f07bd59a640 1 -- 172.21.15.98:0/505543786 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f07c00d29c0 con 0x7f07ac004d50 2024-09-19T08:41:37.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.722+0000 7f07bd59a640 1 -- 172.21.15.98:0/505543786 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f07c00d2ca0 con 0x7f07ac0039c0 2024-09-19T08:41:37.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.722+0000 7f07bd59a640 1 -- 172.21.15.98:0/505543786 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f07ac00ba00 con 0x7f07c00b7770 2024-09-19T08:41:37.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.722+0000 7f07bed9d640 1 --2- 172.21.15.98:0/505543786 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07ac004d50 0x7f07ac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:37.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.722+0000 7f07be59c640 1 --2- 172.21.15.98:0/505543786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07ac0039c0 0x7f07ac003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:37.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.722+0000 7f07bed9d640 1 -- 172.21.15.98:0/505543786 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07c00b7770 msgr2=0x7f07ac004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:37.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.722+0000 7f07bdd9b640 1 --2- 172.21.15.98:0/505543786 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07c00b7770 0x7f07ac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:37.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.722+0000 7f07bed9d640 1 --2- 172.21.15.98:0/505543786 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07c00b7770 0x7f07ac004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:37.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.722+0000 7f07bed9d640 1 -- 172.21.15.98:0/505543786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07ac0039c0 msgr2=0x7f07ac003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:37.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.722+0000 7f07bed9d640 1 --2- 172.21.15.98:0/505543786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07ac0039c0 0x7f07ac003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:37.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.722+0000 7f07bed9d640 1 -- 172.21.15.98:0/505543786 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07ac003530 con 0x7f07ac004d50 2024-09-19T08:41:37.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.722+0000 7f07be59c640 1 --2- 172.21.15.98:0/505543786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07ac0039c0 0x7f07ac003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:37.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.722+0000 7f07bed9d640 1 --2- 172.21.15.98:0/505543786 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07ac004d50 0x7f07ac0051d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f07b8009ae0 tx=0x7f07b801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982fbfcfecf749c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:37.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.722+0000 7f07bd59a640 1 -- 172.21.15.98:0/505543786 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b80b2070 con 0x7f07ac004d50 2024-09-19T08:41:37.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.722+0000 7f07bd59a640 1 -- 172.21.15.98:0/505543786 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07b80a56a0 con 0x7f07ac004d50 2024-09-19T08:41:37.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.722+0000 7f07bd59a640 1 -- 172.21.15.98:0/505543786 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b80096b0 con 0x7f07ac004d50 2024-09-19T08:41:37.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.722+0000 7f07c4e6b640 1 -- 172.21.15.98:0/505543786 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07ac004d50 msgr2=0x7f07ac0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:37.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.722+0000 7f07c4e6b640 1 --2- 172.21.15.98:0/505543786 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07ac004d50 0x7f07ac0051d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f07b8009ae0 tx=0x7f07b801cab0 comp rx=0 tx=0).stop 2024-09-19T08:41:37.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.722+0000 7f07bed9d640 1 -- 172.21.15.98:0/505543786 reap_dead start 2024-09-19T08:41:37.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.722+0000 7f07c4e6b640 1 -- 172.21.15.98:0/505543786 shutdown_connections 2024-09-19T08:41:37.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.722+0000 7f07c4e6b640 1 -- 172.21.15.98:0/505543786 >> 172.21.15.98:0/505543786 conn(0x7f07c00d4640 msgr2=0x7f07c00d4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:37.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.722+0000 7f07c4e6b640 1 -- 172.21.15.98:0/505543786 shutdown_connections 2024-09-19T08:41:37.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.722+0000 7f07c4e6b640 1 -- 172.21.15.98:0/505543786 wait complete. 2024-09-19T08:41:37.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.726+0000 7f07c4e6b640 1 Processor -- start 2024-09-19T08:41:37.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.726+0000 7f07c4e6b640 1 -- start start 2024-09-19T08:41:37.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.726+0000 7f07c4e6b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07c015bad0 0x7f07c01612e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:37.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.726+0000 7f07c4e6b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07c015beb0 0x7f07c01f4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:37.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.726+0000 7f07c4e6b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07c00cb710 0x7f07c00b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:37.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.726+0000 7f07c4e6b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f07c01630a0 con 0x7f07c015bad0 2024-09-19T08:41:37.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.726+0000 7f07c4e6b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f07c00d3540 con 0x7f07c00cb710 2024-09-19T08:41:37.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.726+0000 7f07c4e6b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f07c0163220 con 0x7f07c015beb0 2024-09-19T08:41:37.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.726+0000 7f07be59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07c015bad0 0x7f07c01612e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:37.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.726+0000 7f07be59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07c015bad0 0x7f07c01612e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40846/0 (socket says 172.21.15.98:40846) 2024-09-19T08:41:37.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.726+0000 7f07be59c640 1 -- 172.21.15.98:0/3705142958 learned_addr learned my addr 172.21.15.98:0/3705142958 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:37.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.726+0000 7f07bed9d640 1 --2- 172.21.15.98:0/3705142958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07c00cb710 0x7f07c00b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:37.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.726+0000 7f07bdd9b640 1 --2- 172.21.15.98:0/3705142958 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07c015beb0 0x7f07c01f4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:37.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.726+0000 7f07be59c640 1 -- 172.21.15.98:0/3705142958 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07c015beb0 msgr2=0x7f07c01f4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:37.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.726+0000 7f07be59c640 1 --2- 172.21.15.98:0/3705142958 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07c015beb0 0x7f07c01f4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:37.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.726+0000 7f07be59c640 1 -- 172.21.15.98:0/3705142958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07c00cb710 msgr2=0x7f07c00b26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:37.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.726+0000 7f07be59c640 1 --2- 172.21.15.98:0/3705142958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07c00cb710 0x7f07c00b26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:37.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.726+0000 7f07be59c640 1 -- 172.21.15.98:0/3705142958 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c00cbcb0 con 0x7f07c015bad0 2024-09-19T08:41:37.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.726+0000 7f07be59c640 1 --2- 172.21.15.98:0/3705142958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07c015bad0 0x7f07c01612e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f07b0001000 tx=0x7f07b000dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:37.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.726+0000 7f07b6ffd640 1 -- 172.21.15.98:0/3705142958 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b0019070 con 0x7f07c015bad0 2024-09-19T08:41:37.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.726+0000 7f07b6ffd640 1 -- 172.21.15.98:0/3705142958 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07b0004cc0 con 0x7f07c015bad0 2024-09-19T08:41:37.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.726+0000 7f07c4e6b640 1 -- 172.21.15.98:0/3705142958 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07c00cbf90 con 0x7f07c015bad0 2024-09-19T08:41:37.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.726+0000 7f07b6ffd640 1 -- 172.21.15.98:0/3705142958 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b00072f0 con 0x7f07c015bad0 2024-09-19T08:41:37.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.726+0000 7f07c4e6b640 1 -- 172.21.15.98:0/3705142958 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07c00cc830 con 0x7f07c015bad0 2024-09-19T08:41:37.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.726+0000 7f07c4e6b640 1 -- 172.21.15.98:0/3705142958 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07c00cf1e0 con 0x7f07c015bad0 2024-09-19T08:41:37.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.726+0000 7f07b6ffd640 1 -- 172.21.15.98:0/3705142958 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f07b001c070 con 0x7f07c015bad0 2024-09-19T08:41:37.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.730+0000 7f07b6ffd640 1 --2- 172.21.15.98:0/3705142958 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f07a0067a70 0x7f07a0069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:37.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.730+0000 7f07b6ffd640 1 -- 172.21.15.98:0/3705142958 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5853+0+0 (secure 0 0 0) 0x7f07b0085d90 con 0x7f07c015bad0 2024-09-19T08:41:37.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.730+0000 7f07bdd9b640 1 --2- 172.21.15.98:0/3705142958 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f07a0067a70 0x7f07a0069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:37.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.730+0000 7f07bdd9b640 1 --2- 172.21.15.98:0/3705142958 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f07a0067a70 0x7f07a0069f30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f07c015c290 tx=0x7f07a800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:37.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.734+0000 7f07b6ffd640 1 -- 172.21.15.98:0/3705142958 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f07b0056410 con 0x7f07c015bad0 2024-09-19T08:41:37.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:37.886+0000 7f07c4e6b640 1 -- 172.21.15.98:0/3705142958 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f07c00d54e0 con 0x7f07c015bad0 2024-09-19T08:41:38.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.534+0000 7f07b6ffd640 1 -- 172.21.15.98:0/3705142958 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v47) ==== 181+0+0 (secure 0 0 0) 0x7f07b0055d10 con 0x7f07c015bad0 2024-09-19T08:41:38.533 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_metadata' removed 2024-09-19T08:41:38.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.538+0000 7f07c4e6b640 1 -- 172.21.15.98:0/3705142958 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f07a0067a70 msgr2=0x7f07a0069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:38.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.538+0000 7f07c4e6b640 1 --2- 172.21.15.98:0/3705142958 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f07a0067a70 0x7f07a0069f30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f07c015c290 tx=0x7f07a800b040 comp rx=0 tx=0).stop 2024-09-19T08:41:38.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.538+0000 7f07c4e6b640 1 -- 172.21.15.98:0/3705142958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07c015bad0 msgr2=0x7f07c01612e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:38.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.538+0000 7f07c4e6b640 1 --2- 172.21.15.98:0/3705142958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07c015bad0 0x7f07c01612e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f07b0001000 tx=0x7f07b000dd60 comp rx=0 tx=0).stop 2024-09-19T08:41:38.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.538+0000 7f07c4e6b640 1 -- 172.21.15.98:0/3705142958 shutdown_connections 2024-09-19T08:41:38.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.538+0000 7f07c4e6b640 1 --2- 172.21.15.98:0/3705142958 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f07a0067a70 0x7f07a0069f30 secure :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f07c015c290 tx=0x7f07a800b040 comp rx=0 tx=0).stop 2024-09-19T08:41:38.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.538+0000 7f07c4e6b640 1 --2- 172.21.15.98:0/3705142958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07c00cb710 0x7f07c00b26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:38.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.538+0000 7f07c4e6b640 1 --2- 172.21.15.98:0/3705142958 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07c015beb0 0x7f07c01f4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:38.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.538+0000 7f07c4e6b640 1 --2- 172.21.15.98:0/3705142958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07c015bad0 0x7f07c01612e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:38.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.538+0000 7f07c4e6b640 1 -- 172.21.15.98:0/3705142958 >> 172.21.15.98:0/3705142958 conn(0x7f07c00d4640 msgr2=0x7f07c00b6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:38.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.538+0000 7f07c4e6b640 1 -- 172.21.15.98:0/3705142958 shutdown_connections 2024-09-19T08:41:38.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.538+0000 7f07c4e6b640 1 -- 172.21.15.98:0/3705142958 wait complete. 2024-09-19T08:41:38.561 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T08:41:38.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.686+0000 7f6463f91640 1 Processor -- start 2024-09-19T08:41:38.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.686+0000 7f6463f91640 1 -- start start 2024-09-19T08:41:38.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.690+0000 7f6463f91640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f645c0b02f0 0x7f645c0b0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:38.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.690+0000 7f6463f91640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f645c0ae140 0x7f645c0ae550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:38.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.690+0000 7f6463f91640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f645c0b6060 con 0x7f645c073290 2024-09-19T08:41:38.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.690+0000 7f6463f91640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f645c0b2600 con 0x7f645c0ae140 2024-09-19T08:41:38.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.690+0000 7f6463f91640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f645c0b28e0 con 0x7f645c0b02f0 2024-09-19T08:41:38.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.690+0000 7f646252e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f645c0ae140 0x7f645c0ae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:38.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.690+0000 7f646252e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f645c0ae140 0x7f645c0ae550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40856/0 (socket says 172.21.15.98:40856) 2024-09-19T08:41:38.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.690+0000 7f646252e640 1 -- 172.21.15.98:0/4272157548 learned_addr learned my addr 172.21.15.98:0/4272157548 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:38.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.690+0000 7f646252e640 1 -- 172.21.15.98:0/4272157548 >> v2:172.21.15.139:3300/0 conn(0x7f645c0b02f0 msgr2=0x7f645c0b0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:38.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.690+0000 7f646252e640 1 --2- 172.21.15.98:0/4272157548 >> v2:172.21.15.139:3300/0 conn(0x7f645c0b02f0 0x7f645c0b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:38.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.690+0000 7f646252e640 1 -- 172.21.15.98:0/4272157548 >> v1:172.21.15.139:6789/0 conn(0x7f645c073290 legacy=0x7f645c073670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:38.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.690+0000 7f646252e640 1 -- 172.21.15.98:0/4272157548 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645c0b6300 con 0x7f645c0ae140 2024-09-19T08:41:38.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.690+0000 7f646252e640 1 --2- 172.21.15.98:0/4272157548 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f645c0ae140 0x7f645c0ae550 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f645c071e30 tx=0x7f645401f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40c43cbe23d011f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:38.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.690+0000 7f6460d2b640 1 -- 172.21.15.98:0/4272157548 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645400b2d0 con 0x7f645c0ae140 2024-09-19T08:41:38.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.690+0000 7f6460d2b640 1 -- 172.21.15.98:0/4272157548 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f645400b470 con 0x7f645c0ae140 2024-09-19T08:41:38.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.690+0000 7f6460d2b640 1 -- 172.21.15.98:0/4272157548 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64540a7cc0 con 0x7f645c0ae140 2024-09-19T08:41:38.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.690+0000 7f6463f91640 1 -- 172.21.15.98:0/4272157548 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f645c0ae140 msgr2=0x7f645c0ae550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:38.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.690+0000 7f6463f91640 1 --2- 172.21.15.98:0/4272157548 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f645c0ae140 0x7f645c0ae550 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f645c071e30 tx=0x7f645401f610 comp rx=0 tx=0).stop 2024-09-19T08:41:38.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.690+0000 7f6463f91640 1 -- 172.21.15.98:0/4272157548 shutdown_connections 2024-09-19T08:41:38.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.690+0000 7f6463f91640 1 --2- 172.21.15.98:0/4272157548 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f645c0ae140 0x7f645c0ae550 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:38.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.690+0000 7f6463f91640 1 --2- 172.21.15.98:0/4272157548 >> v2:172.21.15.139:3300/0 conn(0x7f645c0b02f0 0x7f645c0b0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:38.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.690+0000 7f6463f91640 1 -- 172.21.15.98:0/4272157548 >> 172.21.15.98:0/4272157548 conn(0x7f645c0b2e60 msgr2=0x7f645c0b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:38.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.690+0000 7f6463f91640 1 -- 172.21.15.98:0/4272157548 shutdown_connections 2024-09-19T08:41:38.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.690+0000 7f6463f91640 1 -- 172.21.15.98:0/4272157548 wait complete. 2024-09-19T08:41:38.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.694+0000 7f6463f91640 1 Processor -- start 2024-09-19T08:41:38.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.694+0000 7f6463f91640 1 -- start start 2024-09-19T08:41:38.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.694+0000 7f6463f91640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f645c073290 0x7f645c0ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:38.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.694+0000 7f6463f91640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f645c0ae140 0x7f645c155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:38.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.694+0000 7f6463f91640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f645c0b02f0 0x7f645c0c7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:38.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.694+0000 7f6463f91640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f645c074df0 con 0x7f645c073290 2024-09-19T08:41:38.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.694+0000 7f6463f91640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f645c074c70 con 0x7f645c0b02f0 2024-09-19T08:41:38.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.694+0000 7f6463f91640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f645c074f70 con 0x7f645c0ae140 2024-09-19T08:41:38.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.694+0000 7f6461d2d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f645c073290 0x7f645c0ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:38.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.694+0000 7f6461d2d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f645c073290 0x7f645c0ca430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40858/0 (socket says 172.21.15.98:40858) 2024-09-19T08:41:38.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.694+0000 7f6461d2d640 1 -- 172.21.15.98:0/3081205195 learned_addr learned my addr 172.21.15.98:0/3081205195 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:38.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.694+0000 7f646152c640 1 --2- 172.21.15.98:0/3081205195 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f645c0ae140 0x7f645c155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:38.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.694+0000 7f646252e640 1 --2- 172.21.15.98:0/3081205195 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f645c0b02f0 0x7f645c0c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:38.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.694+0000 7f6461d2d640 1 -- 172.21.15.98:0/3081205195 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f645c0ae140 msgr2=0x7f645c155150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:38.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.694+0000 7f6461d2d640 1 --2- 172.21.15.98:0/3081205195 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f645c0ae140 0x7f645c155150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:38.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.694+0000 7f6461d2d640 1 -- 172.21.15.98:0/3081205195 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f645c0b02f0 msgr2=0x7f645c0c7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:38.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.694+0000 7f6461d2d640 1 --2- 172.21.15.98:0/3081205195 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f645c0b02f0 0x7f645c0c7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:38.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.694+0000 7f6461d2d640 1 -- 172.21.15.98:0/3081205195 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645c156860 con 0x7f645c073290 2024-09-19T08:41:38.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.694+0000 7f646152c640 1 --2- 172.21.15.98:0/3081205195 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f645c0ae140 0x7f645c155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:38.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.694+0000 7f646252e640 1 --2- 172.21.15.98:0/3081205195 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f645c0b02f0 0x7f645c0c7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:38.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.694+0000 7f6461d2d640 1 --2- 172.21.15.98:0/3081205195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f645c073290 0x7f645c0ca430 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f645800c490 tx=0x7f6458001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:38.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.694+0000 7f64527fc640 1 -- 172.21.15.98:0/3081205195 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6458017070 con 0x7f645c073290 2024-09-19T08:41:38.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.694+0000 7f64527fc640 1 -- 172.21.15.98:0/3081205195 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6458003600 con 0x7f645c073290 2024-09-19T08:41:38.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.694+0000 7f6463f91640 1 -- 172.21.15.98:0/3081205195 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f645c0caa80 con 0x7f645c073290 2024-09-19T08:41:38.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.694+0000 7f64527fc640 1 -- 172.21.15.98:0/3081205195 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64580039f0 con 0x7f645c073290 2024-09-19T08:41:38.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.694+0000 7f6463f91640 1 -- 172.21.15.98:0/3081205195 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f645c0cb040 con 0x7f645c073290 2024-09-19T08:41:38.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.694+0000 7f6463f91640 1 -- 172.21.15.98:0/3081205195 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f645c0ae550 con 0x7f645c073290 2024-09-19T08:41:38.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.694+0000 7f64527fc640 1 -- 172.21.15.98:0/3081205195 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f645801a070 con 0x7f645c073290 2024-09-19T08:41:38.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.698+0000 7f64527fc640 1 --2- 172.21.15.98:0/3081205195 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6440067a70 0x7f6440069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:38.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.698+0000 7f64527fc640 1 -- 172.21.15.98:0/3081205195 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5415+0+0 (secure 0 0 0) 0x7f6458085290 con 0x7f645c073290 2024-09-19T08:41:38.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.698+0000 7f646152c640 1 --2- 172.21.15.98:0/3081205195 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6440067a70 0x7f6440069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:38.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.698+0000 7f646152c640 1 --2- 172.21.15.98:0/3081205195 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6440067a70 0x7f6440069f30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f645c1565d0 tx=0x7f644c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:38.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.702+0000 7f64527fc640 1 -- 172.21.15.98:0/3081205195 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6458055c70 con 0x7f645c073290 2024-09-19T08:41:38.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:38.862+0000 7f6463f91640 1 -- 172.21.15.98:0/3081205195 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f645c06e3a0 con 0x7f645c073290 2024-09-19T08:41:39.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.546+0000 7f64527fc640 1 -- 172.21.15.98:0/3081205195 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v48) ==== 169+0+0 (secure 0 0 0) 0x7f6458055570 con 0x7f645c073290 2024-09-19T08:41:39.546 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' removed 2024-09-19T08:41:39.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.554+0000 7f6463f91640 1 -- 172.21.15.98:0/3081205195 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6440067a70 msgr2=0x7f6440069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:39.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.554+0000 7f6463f91640 1 --2- 172.21.15.98:0/3081205195 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6440067a70 0x7f6440069f30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f645c1565d0 tx=0x7f644c008040 comp rx=0 tx=0).stop 2024-09-19T08:41:39.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.554+0000 7f6463f91640 1 -- 172.21.15.98:0/3081205195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f645c073290 msgr2=0x7f645c0ca430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:39.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.554+0000 7f6463f91640 1 --2- 172.21.15.98:0/3081205195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f645c073290 0x7f645c0ca430 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f645800c490 tx=0x7f6458001e80 comp rx=0 tx=0).stop 2024-09-19T08:41:39.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.554+0000 7f6463f91640 1 -- 172.21.15.98:0/3081205195 shutdown_connections 2024-09-19T08:41:39.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.554+0000 7f6463f91640 1 --2- 172.21.15.98:0/3081205195 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6440067a70 0x7f6440069f30 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:39.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.554+0000 7f6463f91640 1 --2- 172.21.15.98:0/3081205195 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f645c0b02f0 0x7f645c0c7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:39.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.554+0000 7f6463f91640 1 --2- 172.21.15.98:0/3081205195 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f645c0ae140 0x7f645c155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:39.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.554+0000 7f6463f91640 1 --2- 172.21.15.98:0/3081205195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f645c073290 0x7f645c0ca430 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:39.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.554+0000 7f6463f91640 1 -- 172.21.15.98:0/3081205195 >> 172.21.15.98:0/3081205195 conn(0x7f645c0b2e60 msgr2=0x7f645c0cc910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:39.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.554+0000 7f6463f91640 1 -- 172.21.15.98:0/3081205195 shutdown_connections 2024-09-19T08:41:39.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.554+0000 7f6463f91640 1 -- 172.21.15.98:0/3081205195 wait complete. 2024-09-19T08:41:39.575 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-19T08:41:39.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.698+0000 7f4d258e1640 1 Processor -- start 2024-09-19T08:41:39.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.698+0000 7f4d258e1640 1 -- start start 2024-09-19T08:41:39.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.698+0000 7f4d258e1640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4d2006c3a0 0x7f4d20162200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:39.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.698+0000 7f4d258e1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4d20162740 0x7f4d200b3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:39.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.698+0000 7f4d258e1640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d20162f90 con 0x7f4d20147cb0 2024-09-19T08:41:39.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.698+0000 7f4d258e1640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f4d200cee10 con 0x7f4d2006c3a0 2024-09-19T08:41:39.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.698+0000 7f4d258e1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f4d200cf040 con 0x7f4d20162740 2024-09-19T08:41:39.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.698+0000 7f4d1e7fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4d2006c3a0 0x7f4d20162200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:39.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.698+0000 7f4d1effd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f4d20147cb0 0x7f4d2006bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:37096/0 (socket says 172.21.15.98:37096) 2024-09-19T08:41:39.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.698+0000 7f4d1effd640 1 -- 172.21.15.98:0/1516596394 learned_addr learned my addr 172.21.15.98:0/1516596394 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:41:39.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.698+0000 7f4d1f7fe640 1 --2- 172.21.15.98:0/1516596394 >> v2:172.21.15.139:3300/0 conn(0x7f4d20162740 0x7f4d200b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:39.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.698+0000 7f4d1e7fc640 1 -- 172.21.15.98:0/1516596394 >> v2:172.21.15.139:3300/0 conn(0x7f4d20162740 msgr2=0x7f4d200b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:39.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.698+0000 7f4d1e7fc640 1 --2- 172.21.15.98:0/1516596394 >> v2:172.21.15.139:3300/0 conn(0x7f4d20162740 0x7f4d200b3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:39.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.698+0000 7f4d1e7fc640 1 -- 172.21.15.98:0/1516596394 >> v1:172.21.15.98:6789/0 conn(0x7f4d20147cb0 legacy=0x7f4d2006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:39.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.698+0000 7f4d1dffb640 1 -- 172.21.15.98:0/1516596394 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2056320658 0 0) 0x7f4d20162f90 con 0x7f4d20147cb0 2024-09-19T08:41:39.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.698+0000 7f4d1e7fc640 1 -- 172.21.15.98:0/1516596394 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d200cf2e0 con 0x7f4d2006c3a0 2024-09-19T08:41:39.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.698+0000 7f4d1f7fe640 1 --2- 172.21.15.98:0/1516596394 >> v2:172.21.15.139:3300/0 conn(0x7f4d20162740 0x7f4d200b3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:39.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.702+0000 7f4d1e7fc640 1 --2- 172.21.15.98:0/1516596394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d2006c3a0 0x7f4d20162200 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f4d20074220 tx=0x7f4d0801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8efae2b5b68ac72b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:39.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.702+0000 7f4d1dffb640 1 -- 172.21.15.98:0/1516596394 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d080b1070 con 0x7f4d2006c3a0 2024-09-19T08:41:39.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.702+0000 7f4d1dffb640 1 -- 172.21.15.98:0/1516596394 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d080ac440 con 0x7f4d2006c3a0 2024-09-19T08:41:39.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.702+0000 7f4d1dffb640 1 -- 172.21.15.98:0/1516596394 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d080ac810 con 0x7f4d2006c3a0 2024-09-19T08:41:39.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.702+0000 7f4d258e1640 1 -- 172.21.15.98:0/1516596394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d2006c3a0 msgr2=0x7f4d20162200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:39.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.702+0000 7f4d258e1640 1 --2- 172.21.15.98:0/1516596394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d2006c3a0 0x7f4d20162200 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f4d20074220 tx=0x7f4d0801c3f0 comp rx=0 tx=0).stop 2024-09-19T08:41:39.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.702+0000 7f4d258e1640 1 -- 172.21.15.98:0/1516596394 shutdown_connections 2024-09-19T08:41:39.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.702+0000 7f4d258e1640 1 --2- 172.21.15.98:0/1516596394 >> v2:172.21.15.139:3300/0 conn(0x7f4d20162740 0x7f4d200b3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:39.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.702+0000 7f4d258e1640 1 --2- 172.21.15.98:0/1516596394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d2006c3a0 0x7f4d20162200 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:39.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.702+0000 7f4d258e1640 1 -- 172.21.15.98:0/1516596394 >> 172.21.15.98:0/1516596394 conn(0x7f4d200b2010 msgr2=0x7f4d200b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:39.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.702+0000 7f4d258e1640 1 -- 172.21.15.98:0/1516596394 shutdown_connections 2024-09-19T08:41:39.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.702+0000 7f4d258e1640 1 -- 172.21.15.98:0/1516596394 wait complete. 2024-09-19T08:41:39.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.706+0000 7f4d258e1640 1 Processor -- start 2024-09-19T08:41:39.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.706+0000 7f4d258e1640 1 -- start start 2024-09-19T08:41:39.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.706+0000 7f4d258e1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d2006c3a0 0x7f4d2013c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:39.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.706+0000 7f4d258e1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d20147cb0 0x7f4d20073af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:39.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.706+0000 7f4d258e1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d20162740 0x7f4d20071b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:39.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.706+0000 7f4d258e1640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4d200cfdc0 con 0x7f4d20147cb0 2024-09-19T08:41:39.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.706+0000 7f4d258e1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4d200cff40 con 0x7f4d20162740 2024-09-19T08:41:39.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.706+0000 7f4d258e1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4d2016b3d0 con 0x7f4d2006c3a0 2024-09-19T08:41:39.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.706+0000 7f4d1e7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d20147cb0 0x7f4d20073af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:39.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.706+0000 7f4d1e7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d20147cb0 0x7f4d20073af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40874/0 (socket says 172.21.15.98:40874) 2024-09-19T08:41:39.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.706+0000 7f4d1e7fc640 1 -- 172.21.15.98:0/3519636154 learned_addr learned my addr 172.21.15.98:0/3519636154 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:39.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.706+0000 7f4d1f7fe640 1 --2- 172.21.15.98:0/3519636154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d20162740 0x7f4d20071b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:39.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.706+0000 7f4d1effd640 1 --2- 172.21.15.98:0/3519636154 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d2006c3a0 0x7f4d2013c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:39.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.706+0000 7f4d1e7fc640 1 -- 172.21.15.98:0/3519636154 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d2006c3a0 msgr2=0x7f4d2013c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:39.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.706+0000 7f4d1e7fc640 1 --2- 172.21.15.98:0/3519636154 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d2006c3a0 0x7f4d2013c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:39.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.706+0000 7f4d1e7fc640 1 -- 172.21.15.98:0/3519636154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d20162740 msgr2=0x7f4d20071b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:39.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.706+0000 7f4d1e7fc640 1 --2- 172.21.15.98:0/3519636154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d20162740 0x7f4d20071b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:39.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.706+0000 7f4d1e7fc640 1 -- 172.21.15.98:0/3519636154 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d2013d0e0 con 0x7f4d20147cb0 2024-09-19T08:41:39.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.706+0000 7f4d1effd640 1 --2- 172.21.15.98:0/3519636154 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d2006c3a0 0x7f4d2013c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:39.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.706+0000 7f4d1e7fc640 1 --2- 172.21.15.98:0/3519636154 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d20147cb0 0x7f4d20073af0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f4d08019000 tx=0x7f4d08019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:39.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.706+0000 7f4d17fff640 1 -- 172.21.15.98:0/3519636154 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d080b1040 con 0x7f4d20147cb0 2024-09-19T08:41:39.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.706+0000 7f4d17fff640 1 -- 172.21.15.98:0/3519636154 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d08004020 con 0x7f4d20147cb0 2024-09-19T08:41:39.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.706+0000 7f4d258e1640 1 -- 172.21.15.98:0/3519636154 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d2013d3c0 con 0x7f4d20147cb0 2024-09-19T08:41:39.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.706+0000 7f4d17fff640 1 -- 172.21.15.98:0/3519636154 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d080043d0 con 0x7f4d20147cb0 2024-09-19T08:41:39.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.706+0000 7f4d258e1640 1 -- 172.21.15.98:0/3519636154 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d2013de70 con 0x7f4d20147cb0 2024-09-19T08:41:39.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.706+0000 7f4d258e1640 1 -- 172.21.15.98:0/3519636154 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d2013dc90 con 0x7f4d20147cb0 2024-09-19T08:41:39.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.706+0000 7f4d17fff640 1 -- 172.21.15.98:0/3519636154 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f4d08003620 con 0x7f4d20147cb0 2024-09-19T08:41:39.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.706+0000 7f4d17fff640 1 --2- 172.21.15.98:0/3519636154 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4d04067a70 0x7f4d04069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:39.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.706+0000 7f4d17fff640 1 -- 172.21.15.98:0/3519636154 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4943+0+0 (secure 0 0 0) 0x7f4d0811fba0 con 0x7f4d20147cb0 2024-09-19T08:41:39.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.706+0000 7f4d1effd640 1 --2- 172.21.15.98:0/3519636154 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4d04067a70 0x7f4d04069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:39.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.710+0000 7f4d1effd640 1 --2- 172.21.15.98:0/3519636154 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4d04067a70 0x7f4d04069f30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4d18007fd0 tx=0x7f4d18009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:39.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.714+0000 7f4d17fff640 1 -- 172.21.15.98:0/3519636154 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f4d2013dc90 con 0x7f4d20147cb0 2024-09-19T08:41:39.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.890+0000 7f4d258e1640 1 -- 172.21.15.98:0/3519636154 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f4d20162200 con 0x7f4d20147cb0 2024-09-19T08:41:39.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.894+0000 7f4d17fff640 1 -- 172.21.15.98:0/3519636154 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7f4d080c2030 con 0x7f4d20147cb0 2024-09-19T08:41:39.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.898+0000 7f4d258e1640 1 -- 172.21.15.98:0/3519636154 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4d04067a70 msgr2=0x7f4d04069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:39.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.898+0000 7f4d258e1640 1 --2- 172.21.15.98:0/3519636154 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4d04067a70 0x7f4d04069f30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4d18007fd0 tx=0x7f4d18009040 comp rx=0 tx=0).stop 2024-09-19T08:41:39.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.898+0000 7f4d258e1640 1 -- 172.21.15.98:0/3519636154 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d20147cb0 msgr2=0x7f4d20073af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:39.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.898+0000 7f4d258e1640 1 --2- 172.21.15.98:0/3519636154 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d20147cb0 0x7f4d20073af0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f4d08019000 tx=0x7f4d08019190 comp rx=0 tx=0).stop 2024-09-19T08:41:39.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.898+0000 7f4d258e1640 1 -- 172.21.15.98:0/3519636154 shutdown_connections 2024-09-19T08:41:39.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.898+0000 7f4d258e1640 1 --2- 172.21.15.98:0/3519636154 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4d04067a70 0x7f4d04069f30 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:39.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.898+0000 7f4d258e1640 1 --2- 172.21.15.98:0/3519636154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d20162740 0x7f4d20071b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:39.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.898+0000 7f4d258e1640 1 --2- 172.21.15.98:0/3519636154 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d20147cb0 0x7f4d20073af0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:39.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.898+0000 7f4d258e1640 1 --2- 172.21.15.98:0/3519636154 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d2006c3a0 0x7f4d2013c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:39.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.898+0000 7f4d258e1640 1 -- 172.21.15.98:0/3519636154 >> 172.21.15.98:0/3519636154 conn(0x7f4d200b2010 msgr2=0x7f4d20142340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:39.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.898+0000 7f4d258e1640 1 -- 172.21.15.98:0/3519636154 shutdown_connections 2024-09-19T08:41:39.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:39.898+0000 7f4d258e1640 1 -- 172.21.15.98:0/3519636154 wait complete. 2024-09-19T08:41:39.915 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2024-09-19T08:41:40.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.038+0000 7fc8d808a640 1 Processor -- start 2024-09-19T08:41:40.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.038+0000 7fc8d808a640 1 -- start start 2024-09-19T08:41:40.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.038+0000 7fc8d808a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8d00d0ff0 0x7fc8d00d1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:40.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.038+0000 7fc8d808a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc8d00b6f40 0x7fc8d00b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:40.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.038+0000 7fc8d808a640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8d00740b0 con 0x7fc8d015c220 2024-09-19T08:41:40.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.038+0000 7fc8d808a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fc8d00d1970 con 0x7fc8d00b6f40 2024-09-19T08:41:40.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.038+0000 7fc8d808a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc8d00b7900 con 0x7fc8d00d0ff0 2024-09-19T08:41:40.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.038+0000 7fc8d5e26640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fc8d015c220 0x7fc8d00733f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:37112/0 (socket says 172.21.15.98:37112) 2024-09-19T08:41:40.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.038+0000 7fc8d5e26640 1 -- 172.21.15.98:0/2497128062 learned_addr learned my addr 172.21.15.98:0/2497128062 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:41:40.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.038+0000 7fc8d4e24640 1 -- 172.21.15.98:0/2497128062 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3923957948 0 0) 0x7fc8d00740b0 con 0x7fc8d015c220 2024-09-19T08:41:40.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.038+0000 7fc8d5625640 1 --2- 172.21.15.98:0/2497128062 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8d00d0ff0 0x7fc8d00d1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:40.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.038+0000 7fc8d6627640 1 --2- 172.21.15.98:0/2497128062 >> v2:172.21.15.139:3300/0 conn(0x7fc8d00b6f40 0x7fc8d00b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:40.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.038+0000 7fc8d4e24640 1 -- 172.21.15.98:0/2497128062 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc8b8003530 con 0x7fc8d015c220 2024-09-19T08:41:40.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.038+0000 7fc8d4e24640 1 -- 172.21.15.98:0/2497128062 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4140025320 0 0) 0x7fc8b8003530 con 0x7fc8d015c220 2024-09-19T08:41:40.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.038+0000 7fc8d4e24640 1 -- 172.21.15.98:0/2497128062 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8d00d0ff0 msgr2=0x7fc8d00d1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:40.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.038+0000 7fc8d4e24640 1 --2- 172.21.15.98:0/2497128062 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8d00d0ff0 0x7fc8d00d1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:40.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.038+0000 7fc8d4e24640 1 -- 172.21.15.98:0/2497128062 >> v2:172.21.15.139:3300/0 conn(0x7fc8d00b6f40 msgr2=0x7fc8d00b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:40.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.038+0000 7fc8d4e24640 1 --2- 172.21.15.98:0/2497128062 >> v2:172.21.15.139:3300/0 conn(0x7fc8d00b6f40 0x7fc8d00b73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:40.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.038+0000 7fc8d4e24640 1 -- 172.21.15.98:0/2497128062 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8d0073b00 con 0x7fc8d015c220 2024-09-19T08:41:40.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.038+0000 7fc8d4e24640 1 -- 172.21.15.98:0/2497128062 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fc8c8001a00 con 0x7fc8d015c220 2024-09-19T08:41:40.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.038+0000 7fc8d4e24640 1 -- 172.21.15.98:0/2497128062 >> v1:172.21.15.98:6789/0 conn(0x7fc8d015c220 legacy=0x7fc8d00733f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:40.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.038+0000 7fc8d4e24640 1 --2- 172.21.15.98:0/2497128062 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8d00d0ff0 0x7fc8b80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:40.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.038+0000 7fc8d4e24640 1 --2- 172.21.15.98:0/2497128062 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8b8003fb0 0x7fc8b8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:40.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.038+0000 7fc8d4e24640 1 --2- 172.21.15.98:0/2497128062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8b8004d50 0x7fc8b80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:40.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.038+0000 7fc8d4e24640 1 -- 172.21.15.98:0/2497128062 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d00d1970 con 0x7fc8b8004d50 2024-09-19T08:41:40.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.038+0000 7fc8d4e24640 1 -- 172.21.15.98:0/2497128062 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d00b7900 con 0x7fc8b8003fb0 2024-09-19T08:41:40.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.038+0000 7fc8d4e24640 1 -- 172.21.15.98:0/2497128062 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc8b800ba00 con 0x7fc8d00d0ff0 2024-09-19T08:41:40.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.038+0000 7fc8d6627640 1 --2- 172.21.15.98:0/2497128062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8b8004d50 0x7fc8b80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:40.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.038+0000 7fc8d5625640 1 --2- 172.21.15.98:0/2497128062 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8b8003fb0 0x7fc8b8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:40.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.038+0000 7fc8d5e26640 1 --2- 172.21.15.98:0/2497128062 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8d00d0ff0 0x7fc8b80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:40.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.038+0000 7fc8d6627640 1 -- 172.21.15.98:0/2497128062 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8d00d0ff0 msgr2=0x7fc8b80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:40.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.038+0000 7fc8d6627640 1 --2- 172.21.15.98:0/2497128062 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8d00d0ff0 0x7fc8b80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:40.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.038+0000 7fc8d6627640 1 -- 172.21.15.98:0/2497128062 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8b8003fb0 msgr2=0x7fc8b8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:40.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.038+0000 7fc8d6627640 1 --2- 172.21.15.98:0/2497128062 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8b8003fb0 0x7fc8b8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:40.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.038+0000 7fc8d6627640 1 -- 172.21.15.98:0/2497128062 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b8003530 con 0x7fc8b8004d50 2024-09-19T08:41:40.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.042+0000 7fc8d6627640 1 --2- 172.21.15.98:0/2497128062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8b8004d50 0x7fc8b80051d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0009ae0 tx=0x7fc8c001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86a9e581dd8ff19b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:40.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.042+0000 7fc8d4e24640 1 -- 172.21.15.98:0/2497128062 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c00b2070 con 0x7fc8b8004d50 2024-09-19T08:41:40.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.042+0000 7fc8d4e24640 1 -- 172.21.15.98:0/2497128062 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8c00a56a0 con 0x7fc8b8004d50 2024-09-19T08:41:40.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.042+0000 7fc8d4e24640 1 -- 172.21.15.98:0/2497128062 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c00096b0 con 0x7fc8b8004d50 2024-09-19T08:41:40.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.042+0000 7fc8d808a640 1 -- 172.21.15.98:0/2497128062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8b8004d50 msgr2=0x7fc8b80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:40.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.042+0000 7fc8d808a640 1 --2- 172.21.15.98:0/2497128062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8b8004d50 0x7fc8b80051d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0009ae0 tx=0x7fc8c001cab0 comp rx=0 tx=0).stop 2024-09-19T08:41:40.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.042+0000 7fc8d6627640 1 -- 172.21.15.98:0/2497128062 reap_dead start 2024-09-19T08:41:40.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.042+0000 7fc8d808a640 1 -- 172.21.15.98:0/2497128062 shutdown_connections 2024-09-19T08:41:40.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.042+0000 7fc8d808a640 1 -- 172.21.15.98:0/2497128062 >> 172.21.15.98:0/2497128062 conn(0x7fc8d00b40a0 msgr2=0x7fc8d00b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:40.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.042+0000 7fc8d808a640 1 -- 172.21.15.98:0/2497128062 shutdown_connections 2024-09-19T08:41:40.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.042+0000 7fc8d808a640 1 -- 172.21.15.98:0/2497128062 wait complete. 2024-09-19T08:41:40.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.042+0000 7fc8d808a640 1 Processor -- start 2024-09-19T08:41:40.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.042+0000 7fc8d808a640 1 -- start start 2024-09-19T08:41:40.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.046+0000 7fc8d808a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8d01f3c50 0x7fc8d01f4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:40.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.046+0000 7fc8d808a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8d00b1770 0x7fc8d01f4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:40.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.046+0000 7fc8d808a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8d01f4570 0x7fc8d01f7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:40.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.046+0000 7fc8d808a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d00b6740 con 0x7fc8d00b1770 2024-09-19T08:41:40.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.046+0000 7fc8d808a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d00b6460 con 0x7fc8d01f3c50 2024-09-19T08:41:40.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.046+0000 7fc8d808a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc8d00b6a20 con 0x7fc8d01f4570 2024-09-19T08:41:40.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.046+0000 7fc8d5625640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8d00b1770 0x7fc8d01f4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:40.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.046+0000 7fc8d5625640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8d00b1770 0x7fc8d01f4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40896/0 (socket says 172.21.15.98:40896) 2024-09-19T08:41:40.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.046+0000 7fc8d5625640 1 -- 172.21.15.98:0/983208874 learned_addr learned my addr 172.21.15.98:0/983208874 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:40.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.046+0000 7fc8d5e26640 1 --2- 172.21.15.98:0/983208874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8d01f3c50 0x7fc8d01f4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:40.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.046+0000 7fc8d6627640 1 --2- 172.21.15.98:0/983208874 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8d01f4570 0x7fc8d01f7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:40.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.046+0000 7fc8d5625640 1 -- 172.21.15.98:0/983208874 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8d01f4570 msgr2=0x7fc8d01f7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:40.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.046+0000 7fc8d5625640 1 --2- 172.21.15.98:0/983208874 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8d01f4570 0x7fc8d01f7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:40.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.046+0000 7fc8d5625640 1 -- 172.21.15.98:0/983208874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8d01f3c50 msgr2=0x7fc8d01f4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:40.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.046+0000 7fc8d5625640 1 --2- 172.21.15.98:0/983208874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8d01f3c50 0x7fc8d01f4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:40.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.046+0000 7fc8d5625640 1 -- 172.21.15.98:0/983208874 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8d006c740 con 0x7fc8d00b1770 2024-09-19T08:41:40.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.046+0000 7fc8d6627640 1 --2- 172.21.15.98:0/983208874 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8d01f4570 0x7fc8d01f7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:40.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.046+0000 7fc8d5625640 1 --2- 172.21.15.98:0/983208874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8d00b1770 0x7fc8d01f4030 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc011570 tx=0x7fc8cc011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:40.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.046+0000 7fc8c67fc640 1 -- 172.21.15.98:0/983208874 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc015930 con 0x7fc8d00b1770 2024-09-19T08:41:40.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.046+0000 7fc8c67fc640 1 -- 172.21.15.98:0/983208874 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8cc015ad0 con 0x7fc8d00b1770 2024-09-19T08:41:40.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.046+0000 7fc8d808a640 1 -- 172.21.15.98:0/983208874 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8d006ca20 con 0x7fc8d00b1770 2024-09-19T08:41:40.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.046+0000 7fc8c67fc640 1 -- 172.21.15.98:0/983208874 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc01d400 con 0x7fc8d00b1770 2024-09-19T08:41:40.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.046+0000 7fc8d808a640 1 -- 172.21.15.98:0/983208874 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8d006cfe0 con 0x7fc8d00b1770 2024-09-19T08:41:40.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.046+0000 7fc8d808a640 1 -- 172.21.15.98:0/983208874 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8d00d0c30 con 0x7fc8d00b1770 2024-09-19T08:41:40.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.046+0000 7fc8c67fc640 1 -- 172.21.15.98:0/983208874 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fc8cc01d5a0 con 0x7fc8d00b1770 2024-09-19T08:41:40.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.046+0000 7fc8c67fc640 1 --2- 172.21.15.98:0/983208874 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc8bc067a70 0x7fc8bc069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:40.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.046+0000 7fc8c67fc640 1 -- 172.21.15.98:0/983208874 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4943+0+0 (secure 0 0 0) 0x7fc8cc0864e0 con 0x7fc8d00b1770 2024-09-19T08:41:40.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.046+0000 7fc8d5e26640 1 --2- 172.21.15.98:0/983208874 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc8bc067a70 0x7fc8bc069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:40.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.046+0000 7fc8d5e26640 1 --2- 172.21.15.98:0/983208874 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc8bc067a70 0x7fc8bc069f30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8002240 tx=0x7fc8c8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:40.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.050+0000 7fc8c67fc640 1 -- 172.21.15.98:0/983208874 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc8cc019030 con 0x7fc8d00b1770 2024-09-19T08:41:40.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.274+0000 7fc8d808a640 1 -- 172.21.15.98:0/983208874 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) -- 0x7fc8d00d1640 con 0x7fc8d00b1770 2024-09-19T08:41:40.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.558+0000 7fc8c67fc640 1 -- 172.21.15.98:0/983208874 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v175) ==== 140+0+0 (secure 0 0 0) 0x7fc8cc057130 con 0x7fc8d00b1770 2024-09-19T08:41:40.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.562+0000 7fc8d808a640 1 -- 172.21.15.98:0/983208874 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc8bc067a70 msgr2=0x7fc8bc069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:40.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.562+0000 7fc8d808a640 1 --2- 172.21.15.98:0/983208874 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc8bc067a70 0x7fc8bc069f30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8002240 tx=0x7fc8c8009040 comp rx=0 tx=0).stop 2024-09-19T08:41:40.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.562+0000 7fc8d808a640 1 -- 172.21.15.98:0/983208874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8d00b1770 msgr2=0x7fc8d01f4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:40.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.562+0000 7fc8d808a640 1 --2- 172.21.15.98:0/983208874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8d00b1770 0x7fc8d01f4030 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc011570 tx=0x7fc8cc011a30 comp rx=0 tx=0).stop 2024-09-19T08:41:40.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.562+0000 7fc8d808a640 1 -- 172.21.15.98:0/983208874 shutdown_connections 2024-09-19T08:41:40.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.562+0000 7fc8d808a640 1 --2- 172.21.15.98:0/983208874 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc8bc067a70 0x7fc8bc069f30 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:40.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.562+0000 7fc8d808a640 1 --2- 172.21.15.98:0/983208874 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8d01f4570 0x7fc8d01f7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:40.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.562+0000 7fc8d808a640 1 --2- 172.21.15.98:0/983208874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8d00b1770 0x7fc8d01f4030 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:40.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.562+0000 7fc8d808a640 1 --2- 172.21.15.98:0/983208874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8d01f3c50 0x7fc8d01f4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:40.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.562+0000 7fc8d808a640 1 -- 172.21.15.98:0/983208874 >> 172.21.15.98:0/983208874 conn(0x7fc8d00b40a0 msgr2=0x7fc8d0070bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:40.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.562+0000 7fc8d808a640 1 -- 172.21.15.98:0/983208874 shutdown_connections 2024-09-19T08:41:40.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.562+0000 7fc8d808a640 1 -- 172.21.15.98:0/983208874 wait complete. 2024-09-19T08:41:40.582 INFO:tasks.cephfs_test_runner:test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) ... ok 2024-09-19T08:41:40.582 INFO:tasks.cephfs_test_runner:Starting test: test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate handcrafted .meta file on legacy subvol root doesn't break the system 2024-09-19T08:41:40.582 INFO:tasks.cephfs_test_runner:test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-09-19T08:41:40.583 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:41:40.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.706+0000 7fc4caccc640 1 Processor -- start 2024-09-19T08:41:40.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.706+0000 7fc4caccc640 1 -- start start 2024-09-19T08:41:40.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.710+0000 7fc4caccc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc4c4158fd0 0x7fc4c40b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:40.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.710+0000 7fc4caccc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc4c40b4360 0x7fc4c406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:40.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.710+0000 7fc4caccc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4c406c1d0 0x7fc4c415efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:40.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.710+0000 7fc4caccc640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fc4c406c640 con 0x7fc4c40b4360 2024-09-19T08:41:40.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.710+0000 7fc4caccc640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fc4c41593b0 con 0x7fc4c4158fd0 2024-09-19T08:41:40.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.710+0000 7fc4caccc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc4c40ccc60 con 0x7fc4c406c1d0 2024-09-19T08:41:40.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.710+0000 7fc4c3fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc4c40b4360 0x7fc4c406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:40.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.710+0000 7fc4c8a68640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc4c4158fd0 0x7fc4c40b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:40.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.710+0000 7fc4c3fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc4c40b4360 0x7fc4c406bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40908/0 (socket says 172.21.15.98:40908) 2024-09-19T08:41:40.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.710+0000 7fc4c3fff640 1 -- 172.21.15.98:0/1694903045 learned_addr learned my addr 172.21.15.98:0/1694903045 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:40.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.710+0000 7fc4c9269640 1 --2- 172.21.15.98:0/1694903045 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4c406c1d0 0x7fc4c415efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:40.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.710+0000 7fc4c3fff640 1 -- 172.21.15.98:0/1694903045 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4c406c1d0 msgr2=0x7fc4c415efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:40.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.710+0000 7fc4c3fff640 1 --2- 172.21.15.98:0/1694903045 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4c406c1d0 0x7fc4c415efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:40.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.710+0000 7fc4c3fff640 1 -- 172.21.15.98:0/1694903045 >> v2:172.21.15.139:3300/0 conn(0x7fc4c4158fd0 msgr2=0x7fc4c40b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:40.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.710+0000 7fc4c3fff640 1 --2- 172.21.15.98:0/1694903045 >> v2:172.21.15.139:3300/0 conn(0x7fc4c4158fd0 0x7fc4c40b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:40.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.710+0000 7fc4c3fff640 1 -- 172.21.15.98:0/1694903045 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c40ccf40 con 0x7fc4c40b4360 2024-09-19T08:41:40.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.710+0000 7fc4c3fff640 1 --2- 172.21.15.98:0/1694903045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4c40b4360 0x7fc4c406bc90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc4c406e310 tx=0x7fc4b801c1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948972a0c565e972 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:40.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.710+0000 7fc4c37fe640 1 -- 172.21.15.98:0/1694903045 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b80b0070 con 0x7fc4c40b4360 2024-09-19T08:41:40.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.710+0000 7fc4c37fe640 1 -- 172.21.15.98:0/1694903045 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4b80ab600 con 0x7fc4c40b4360 2024-09-19T08:41:40.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.710+0000 7fc4c37fe640 1 -- 172.21.15.98:0/1694903045 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b80ab9d0 con 0x7fc4c40b4360 2024-09-19T08:41:40.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.714+0000 7fc4caccc640 1 -- 172.21.15.98:0/1694903045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4c40b4360 msgr2=0x7fc4c406bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:40.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.714+0000 7fc4caccc640 1 --2- 172.21.15.98:0/1694903045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4c40b4360 0x7fc4c406bc90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc4c406e310 tx=0x7fc4b801c1a0 comp rx=0 tx=0).stop 2024-09-19T08:41:40.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.714+0000 7fc4caccc640 1 -- 172.21.15.98:0/1694903045 shutdown_connections 2024-09-19T08:41:40.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.714+0000 7fc4caccc640 1 --2- 172.21.15.98:0/1694903045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4c40b4360 0x7fc4c406bc90 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:40.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.714+0000 7fc4caccc640 1 --2- 172.21.15.98:0/1694903045 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4c406c1d0 0x7fc4c415efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:40.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.714+0000 7fc4caccc640 1 --2- 172.21.15.98:0/1694903045 >> v2:172.21.15.139:3300/0 conn(0x7fc4c4158fd0 0x7fc4c40b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:40.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.714+0000 7fc4caccc640 1 -- 172.21.15.98:0/1694903045 >> 172.21.15.98:0/1694903045 conn(0x7fc4c40b2010 msgr2=0x7fc4c40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:40.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.714+0000 7fc4caccc640 1 -- 172.21.15.98:0/1694903045 shutdown_connections 2024-09-19T08:41:40.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.714+0000 7fc4caccc640 1 -- 172.21.15.98:0/1694903045 wait complete. 2024-09-19T08:41:40.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.714+0000 7fc4caccc640 1 Processor -- start 2024-09-19T08:41:40.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.714+0000 7fc4caccc640 1 -- start start 2024-09-19T08:41:40.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.714+0000 7fc4caccc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4c406c1d0 0x7fc4c4167d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:40.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.714+0000 7fc4caccc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4c40b4360 0x7fc4c4147c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:40.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.714+0000 7fc4caccc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4c4158fd0 0x7fc4c41644e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:40.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.714+0000 7fc4caccc640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c415c920 con 0x7fc4c4158fd0 2024-09-19T08:41:40.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.714+0000 7fc4caccc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c415c7a0 con 0x7fc4c406c1d0 2024-09-19T08:41:40.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.714+0000 7fc4caccc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc4c406d170 con 0x7fc4c40b4360 2024-09-19T08:41:40.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.714+0000 7fc4c9269640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4c4158fd0 0x7fc4c41644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:40.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.714+0000 7fc4c9269640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4c4158fd0 0x7fc4c41644e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40916/0 (socket says 172.21.15.98:40916) 2024-09-19T08:41:40.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.714+0000 7fc4c9269640 1 -- 172.21.15.98:0/4081483928 learned_addr learned my addr 172.21.15.98:0/4081483928 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:40.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.714+0000 7fc4c8a68640 1 --2- 172.21.15.98:0/4081483928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4c406c1d0 0x7fc4c4167d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:40.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.714+0000 7fc4c3fff640 1 --2- 172.21.15.98:0/4081483928 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4c40b4360 0x7fc4c4147c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:40.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.714+0000 7fc4c9269640 1 -- 172.21.15.98:0/4081483928 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4c40b4360 msgr2=0x7fc4c4147c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:40.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.714+0000 7fc4c9269640 1 --2- 172.21.15.98:0/4081483928 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4c40b4360 0x7fc4c4147c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:40.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.714+0000 7fc4c9269640 1 -- 172.21.15.98:0/4081483928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4c406c1d0 msgr2=0x7fc4c4167d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:40.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.714+0000 7fc4c9269640 1 --2- 172.21.15.98:0/4081483928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4c406c1d0 0x7fc4c4167d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:40.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.714+0000 7fc4c9269640 1 -- 172.21.15.98:0/4081483928 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c4166cf0 con 0x7fc4c4158fd0 2024-09-19T08:41:40.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.714+0000 7fc4c9269640 1 --2- 172.21.15.98:0/4081483928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4c4158fd0 0x7fc4c41644e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc007e60 tx=0x7fc4bc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:40.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.714+0000 7fc4c17fa640 1 -- 172.21.15.98:0/4081483928 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc014620 con 0x7fc4c4158fd0 2024-09-19T08:41:40.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.714+0000 7fc4c17fa640 1 -- 172.21.15.98:0/4081483928 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4bc019070 con 0x7fc4c4158fd0 2024-09-19T08:41:40.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.718+0000 7fc4caccc640 1 -- 172.21.15.98:0/4081483928 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c4166fd0 con 0x7fc4c4158fd0 2024-09-19T08:41:40.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.718+0000 7fc4c17fa640 1 -- 172.21.15.98:0/4081483928 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc014a20 con 0x7fc4c4158fd0 2024-09-19T08:41:40.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.718+0000 7fc4caccc640 1 -- 172.21.15.98:0/4081483928 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c4167590 con 0x7fc4c4158fd0 2024-09-19T08:41:40.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.718+0000 7fc4caccc640 1 -- 172.21.15.98:0/4081483928 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4c406bc90 con 0x7fc4c4158fd0 2024-09-19T08:41:40.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.718+0000 7fc4c17fa640 1 -- 172.21.15.98:0/4081483928 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fc4bc014bc0 con 0x7fc4c4158fd0 2024-09-19T08:41:40.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.718+0000 7fc4c17fa640 1 --2- 172.21.15.98:0/4081483928 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc4b0067a70 0x7fc4b0069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:40.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.718+0000 7fc4c17fa640 1 -- 172.21.15.98:0/4081483928 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4943+0+0 (secure 0 0 0) 0x7fc4bc086520 con 0x7fc4c4158fd0 2024-09-19T08:41:40.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.718+0000 7fc4c8a68640 1 --2- 172.21.15.98:0/4081483928 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc4b0067a70 0x7fc4b0069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:40.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.718+0000 7fc4c8a68640 1 --2- 172.21.15.98:0/4081483928 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc4b0067a70 0x7fc4b0069f30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4009a50 tx=0x7fc4b4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:40.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.722+0000 7fc4c17fa640 1 -- 172.21.15.98:0/4081483928 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc4bc0570e0 con 0x7fc4c4158fd0 2024-09-19T08:41:40.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.886+0000 7fc4caccc640 1 -- 172.21.15.98:0/4081483928 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc4c40b1510 con 0x7fc4c4158fd0 2024-09-19T08:41:40.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.886+0000 7fc4c17fa640 1 -- 172.21.15.98:0/4081483928 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+13032 (secure 0 0 0) 0x7fc4bc0569e0 con 0x7fc4c4158fd0 2024-09-19T08:41:40.888 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:41:40.888 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":48,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:41:39.546130+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.98:6837/3085042202":"2024-09-20T08:41:36.496247+0000","172.21.15.98:6836/3085042202":"2024-09-20T08:41:36.496247+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:41:40.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.890+0000 7fc4caccc640 1 -- 172.21.15.98:0/4081483928 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc4b0067a70 msgr2=0x7fc4b0069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:40.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.890+0000 7fc4caccc640 1 --2- 172.21.15.98:0/4081483928 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc4b0067a70 0x7fc4b0069f30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4009a50 tx=0x7fc4b4008040 comp rx=0 tx=0).stop 2024-09-19T08:41:40.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.890+0000 7fc4caccc640 1 -- 172.21.15.98:0/4081483928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4c4158fd0 msgr2=0x7fc4c41644e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:40.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.890+0000 7fc4caccc640 1 --2- 172.21.15.98:0/4081483928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4c4158fd0 0x7fc4c41644e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc007e60 tx=0x7fc4bc00e4b0 comp rx=0 tx=0).stop 2024-09-19T08:41:40.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.890+0000 7fc4caccc640 1 -- 172.21.15.98:0/4081483928 shutdown_connections 2024-09-19T08:41:40.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.890+0000 7fc4caccc640 1 --2- 172.21.15.98:0/4081483928 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc4b0067a70 0x7fc4b0069f30 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:40.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.890+0000 7fc4caccc640 1 --2- 172.21.15.98:0/4081483928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc4c4158fd0 0x7fc4c41644e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:40.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.890+0000 7fc4caccc640 1 --2- 172.21.15.98:0/4081483928 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc4c40b4360 0x7fc4c4147c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:40.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.890+0000 7fc4caccc640 1 --2- 172.21.15.98:0/4081483928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc4c406c1d0 0x7fc4c4167d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:40.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.890+0000 7fc4caccc640 1 -- 172.21.15.98:0/4081483928 >> 172.21.15.98:0/4081483928 conn(0x7fc4c40b2010 msgr2=0x7fc4c40b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:40.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.890+0000 7fc4caccc640 1 -- 172.21.15.98:0/4081483928 shutdown_connections 2024-09-19T08:41:40.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:40.890+0000 7fc4caccc640 1 -- 172.21.15.98:0/4081483928 wait complete. 2024-09-19T08:41:40.908 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T08:38:08.105684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T08:41:40.909 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T08:41:41.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.030+0000 7f8966fdd640 1 Processor -- start 2024-09-19T08:41:41.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.030+0000 7f8966fdd640 1 -- start start 2024-09-19T08:41:41.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.030+0000 7f8966fdd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8960144d10 0x7f89600722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:41.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.030+0000 7f8966fdd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8960072820 0x7f896006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:41.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.030+0000 7f8966fdd640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8960072ea0 con 0x7f896006c1d0 2024-09-19T08:41:41.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.030+0000 7f8966fdd640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f8960072c60 con 0x7f8960072820 2024-09-19T08:41:41.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.030+0000 7f8966fdd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8960073ab0 con 0x7f8960144d10 2024-09-19T08:41:41.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.030+0000 7f896557a640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f896006c1d0 0x7f89600b4520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:37122/0 (socket says 172.21.15.98:37122) 2024-09-19T08:41:41.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.030+0000 7f896557a640 1 -- 172.21.15.98:0/3855902315 learned_addr learned my addr 172.21.15.98:0/3855902315 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:41:41.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.030+0000 7f895ffff640 1 --2- 172.21.15.98:0/3855902315 >> v2:172.21.15.139:3300/0 conn(0x7f8960072820 0x7f896006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:41.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.030+0000 7f895f7fe640 1 -- 172.21.15.98:0/3855902315 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3468978030 0 0) 0x7f8960072ea0 con 0x7f896006c1d0 2024-09-19T08:41:41.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.030+0000 7f8964d79640 1 --2- 172.21.15.98:0/3855902315 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8960144d10 0x7f89600722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:41.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.030+0000 7f895f7fe640 1 -- 172.21.15.98:0/3855902315 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8948003530 con 0x7f896006c1d0 2024-09-19T08:41:41.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.030+0000 7f895f7fe640 1 -- 172.21.15.98:0/3855902315 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2595355903 0 0) 0x7f8948003530 con 0x7f896006c1d0 2024-09-19T08:41:41.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.030+0000 7f895f7fe640 1 -- 172.21.15.98:0/3855902315 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8960144d10 msgr2=0x7f89600722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:41.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.030+0000 7f895f7fe640 1 --2- 172.21.15.98:0/3855902315 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8960144d10 0x7f89600722e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:41.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.030+0000 7f895f7fe640 1 -- 172.21.15.98:0/3855902315 >> v2:172.21.15.139:3300/0 conn(0x7f8960072820 msgr2=0x7f896006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:41.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.030+0000 7f895f7fe640 1 --2- 172.21.15.98:0/3855902315 >> v2:172.21.15.139:3300/0 conn(0x7f8960072820 0x7f896006bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:41.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.030+0000 7f895f7fe640 1 -- 172.21.15.98:0/3855902315 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8960073180 con 0x7f896006c1d0 2024-09-19T08:41:41.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.030+0000 7f895f7fe640 1 -- 172.21.15.98:0/3855902315 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f8958001a00 con 0x7f896006c1d0 2024-09-19T08:41:41.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.030+0000 7f895f7fe640 1 -- 172.21.15.98:0/3855902315 >> v1:172.21.15.98:6789/0 conn(0x7f896006c1d0 legacy=0x7f89600b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:41.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.030+0000 7f895f7fe640 1 --2- 172.21.15.98:0/3855902315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89480039c0 0x7f8948003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:41.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.030+0000 7f895f7fe640 1 --2- 172.21.15.98:0/3855902315 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8948004810 0x7f8948004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:41.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.030+0000 7f895f7fe640 1 --2- 172.21.15.98:0/3855902315 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8960144d10 0x7f89480051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:41.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.030+0000 7f895f7fe640 1 -- 172.21.15.98:0/3855902315 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8960072c60 con 0x7f89480039c0 2024-09-19T08:41:41.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.030+0000 7f895f7fe640 1 -- 172.21.15.98:0/3855902315 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8960073ab0 con 0x7f8948004810 2024-09-19T08:41:41.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.030+0000 7f895f7fe640 1 -- 172.21.15.98:0/3855902315 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f894800b9d0 con 0x7f8960144d10 2024-09-19T08:41:41.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.030+0000 7f8964d79640 1 --2- 172.21.15.98:0/3855902315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89480039c0 0x7f8948003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:41.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.034+0000 7f895ffff640 1 --2- 172.21.15.98:0/3855902315 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8948004810 0x7f8948004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:41.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.034+0000 7f896557a640 1 --2- 172.21.15.98:0/3855902315 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8960144d10 0x7f89480051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:41.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.034+0000 7f8964d79640 1 -- 172.21.15.98:0/3855902315 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8960144d10 msgr2=0x7f89480051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:41.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.034+0000 7f8964d79640 1 --2- 172.21.15.98:0/3855902315 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8960144d10 0x7f89480051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:41.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.034+0000 7f8964d79640 1 -- 172.21.15.98:0/3855902315 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8948004810 msgr2=0x7f8948004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:41.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.034+0000 7f8964d79640 1 --2- 172.21.15.98:0/3855902315 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8948004810 0x7f8948004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:41.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.034+0000 7f8964d79640 1 -- 172.21.15.98:0/3855902315 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8948003530 con 0x7f89480039c0 2024-09-19T08:41:41.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.034+0000 7f8964d79640 1 --2- 172.21.15.98:0/3855902315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89480039c0 0x7f8948003de0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f8950009ae0 tx=0x7f895001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f451f3b491bfb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:41.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.034+0000 7f895f7fe640 1 -- 172.21.15.98:0/3855902315 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89500096b0 con 0x7f89480039c0 2024-09-19T08:41:41.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.034+0000 7f895f7fe640 1 -- 172.21.15.98:0/3855902315 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89500a9470 con 0x7f89480039c0 2024-09-19T08:41:41.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.034+0000 7f895f7fe640 1 -- 172.21.15.98:0/3855902315 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89500a9860 con 0x7f89480039c0 2024-09-19T08:41:41.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.034+0000 7f8966fdd640 1 -- 172.21.15.98:0/3855902315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89480039c0 msgr2=0x7f8948003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:41.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.034+0000 7f8966fdd640 1 --2- 172.21.15.98:0/3855902315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89480039c0 0x7f8948003de0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f8950009ae0 tx=0x7f895001ca70 comp rx=0 tx=0).stop 2024-09-19T08:41:41.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.034+0000 7f896557a640 1 -- 172.21.15.98:0/3855902315 reap_dead start 2024-09-19T08:41:41.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.034+0000 7f8966fdd640 1 -- 172.21.15.98:0/3855902315 shutdown_connections 2024-09-19T08:41:41.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.034+0000 7f8966fdd640 1 -- 172.21.15.98:0/3855902315 >> 172.21.15.98:0/3855902315 conn(0x7f89600b2010 msgr2=0x7f89600b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:41.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.034+0000 7f8966fdd640 1 -- 172.21.15.98:0/3855902315 shutdown_connections 2024-09-19T08:41:41.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.034+0000 7f8966fdd640 1 -- 172.21.15.98:0/3855902315 wait complete. 2024-09-19T08:41:41.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.038+0000 7f8966fdd640 1 Processor -- start 2024-09-19T08:41:41.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.038+0000 7f8966fdd640 1 -- start start 2024-09-19T08:41:41.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.038+0000 7f8966fdd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89480039c0 0x7f89601407d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:41.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.038+0000 7f8966fdd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8960159ee0 0x7f896013d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:41.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.038+0000 7f8966fdd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f896013e200 0x7f896013e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:41.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.038+0000 7f8966fdd640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f89600b3590 con 0x7f89480039c0 2024-09-19T08:41:41.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.038+0000 7f8966fdd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f89600b3410 con 0x7f896013e200 2024-09-19T08:41:41.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.038+0000 7f8966fdd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f89600b3710 con 0x7f8960159ee0 2024-09-19T08:41:41.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.038+0000 7f8964d79640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89480039c0 0x7f89601407d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:41.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.038+0000 7f8964d79640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89480039c0 0x7f89601407d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40928/0 (socket says 172.21.15.98:40928) 2024-09-19T08:41:41.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.038+0000 7f8964d79640 1 -- 172.21.15.98:0/1504378997 learned_addr learned my addr 172.21.15.98:0/1504378997 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:41.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.038+0000 7f896557a640 1 --2- 172.21.15.98:0/1504378997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f896013e200 0x7f896013e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:41.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.038+0000 7f895ffff640 1 --2- 172.21.15.98:0/1504378997 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8960159ee0 0x7f896013d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:41.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.038+0000 7f8964d79640 1 -- 172.21.15.98:0/1504378997 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8960159ee0 msgr2=0x7f896013d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:41.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.038+0000 7f8964d79640 1 --2- 172.21.15.98:0/1504378997 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8960159ee0 0x7f896013d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:41.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.038+0000 7f8964d79640 1 -- 172.21.15.98:0/1504378997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f896013e200 msgr2=0x7f896013e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:41.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.038+0000 7f8964d79640 1 --2- 172.21.15.98:0/1504378997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f896013e200 0x7f896013e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:41.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.038+0000 7f8964d79640 1 -- 172.21.15.98:0/1504378997 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f896013de50 con 0x7f89480039c0 2024-09-19T08:41:41.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.038+0000 7f895ffff640 1 --2- 172.21.15.98:0/1504378997 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8960159ee0 0x7f896013d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:41.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.038+0000 7f8964d79640 1 --2- 172.21.15.98:0/1504378997 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89480039c0 0x7f89601407d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8950009ae0 tx=0x7f8948005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:41.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.038+0000 7f895d7fa640 1 -- 172.21.15.98:0/1504378997 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89500c1070 con 0x7f89480039c0 2024-09-19T08:41:41.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.038+0000 7f895d7fa640 1 -- 172.21.15.98:0/1504378997 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f895000f000 con 0x7f89480039c0 2024-09-19T08:41:41.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.038+0000 7f8966fdd640 1 -- 172.21.15.98:0/1504378997 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f896015a450 con 0x7f89480039c0 2024-09-19T08:41:41.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.038+0000 7f895d7fa640 1 -- 172.21.15.98:0/1504378997 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89500a9e40 con 0x7f89480039c0 2024-09-19T08:41:41.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.038+0000 7f8966fdd640 1 -- 172.21.15.98:0/1504378997 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f896015aa10 con 0x7f89480039c0 2024-09-19T08:41:41.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.038+0000 7f8966fdd640 1 -- 172.21.15.98:0/1504378997 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8960072ea0 con 0x7f89480039c0 2024-09-19T08:41:41.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.038+0000 7f895d7fa640 1 -- 172.21.15.98:0/1504378997 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f89500c4070 con 0x7f89480039c0 2024-09-19T08:41:41.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.038+0000 7f895d7fa640 1 --2- 172.21.15.98:0/1504378997 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f894c067a40 0x7f894c069f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:41.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.038+0000 7f895d7fa640 1 -- 172.21.15.98:0/1504378997 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4943+0+0 (secure 0 0 0) 0x7f895011ff10 con 0x7f89480039c0 2024-09-19T08:41:41.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.038+0000 7f895ffff640 1 --2- 172.21.15.98:0/1504378997 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f894c067a40 0x7f894c069f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:41.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.042+0000 7f895ffff640 1 --2- 172.21.15.98:0/1504378997 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f894c067a40 0x7f894c069f00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f896015b560 tx=0x7f8954008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:41.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.042+0000 7f895d7fa640 1 -- 172.21.15.98:0/1504378997 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f89500ad020 con 0x7f89480039c0 2024-09-19T08:41:41.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.194+0000 7f8966fdd640 1 -- 172.21.15.98:0/1504378997 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f89601434f0 con 0x7f89480039c0 2024-09-19T08:41:41.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.194+0000 7f895d7fa640 1 -- 172.21.15.98:0/1504378997 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) ==== 93+0+10 (secure 0 0 0) 0x7f8950016780 con 0x7f89480039c0 2024-09-19T08:41:41.195 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2024-09-19T08:41:41.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.198+0000 7f8966fdd640 1 -- 172.21.15.98:0/1504378997 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f894c067a40 msgr2=0x7f894c069f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:41.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.198+0000 7f8966fdd640 1 --2- 172.21.15.98:0/1504378997 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f894c067a40 0x7f894c069f00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f896015b560 tx=0x7f8954008040 comp rx=0 tx=0).stop 2024-09-19T08:41:41.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.198+0000 7f8966fdd640 1 -- 172.21.15.98:0/1504378997 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89480039c0 msgr2=0x7f89601407d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:41.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.198+0000 7f8966fdd640 1 --2- 172.21.15.98:0/1504378997 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89480039c0 0x7f89601407d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8950009ae0 tx=0x7f8948005840 comp rx=0 tx=0).stop 2024-09-19T08:41:41.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.198+0000 7f8966fdd640 1 -- 172.21.15.98:0/1504378997 shutdown_connections 2024-09-19T08:41:41.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.198+0000 7f8966fdd640 1 --2- 172.21.15.98:0/1504378997 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f894c067a40 0x7f894c069f00 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:41.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.198+0000 7f8966fdd640 1 --2- 172.21.15.98:0/1504378997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f896013e200 0x7f896013e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:41.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.198+0000 7f8966fdd640 1 --2- 172.21.15.98:0/1504378997 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8960159ee0 0x7f896013d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:41.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.198+0000 7f8966fdd640 1 --2- 172.21.15.98:0/1504378997 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89480039c0 0x7f89601407d0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:41.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.198+0000 7f8966fdd640 1 -- 172.21.15.98:0/1504378997 >> 172.21.15.98:0/1504378997 conn(0x7f89600b2010 msgr2=0x7f89600cf7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:41.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.198+0000 7f8966fdd640 1 -- 172.21.15.98:0/1504378997 shutdown_connections 2024-09-19T08:41:41.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.198+0000 7f8966fdd640 1 -- 172.21.15.98:0/1504378997 wait complete. 2024-09-19T08:41:41.213 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade' 2024-09-19T08:41:41.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.322+0000 7fa729e53640 1 Processor -- start 2024-09-19T08:41:41.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.322+0000 7fa729e53640 1 -- start start 2024-09-19T08:41:41.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.326+0000 7fa729e53640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa72415dee0 0x7fa7240afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:41.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.326+0000 7fa729e53640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7240cef60 0x7fa724071520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:41.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.326+0000 7fa729e53640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7240cffe0 con 0x7fa724071a60 2024-09-19T08:41:41.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.326+0000 7fa729e53640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fa7240cf460 con 0x7fa72415dee0 2024-09-19T08:41:41.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.326+0000 7fa729e53640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa7240cf740 con 0x7fa7240cef60 2024-09-19T08:41:41.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.326+0000 7fa7237fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa72415dee0 0x7fa7240afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:41.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.326+0000 7fa723fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fa724071a60 0x7fa72415e440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:37130/0 (socket says 172.21.15.98:37130) 2024-09-19T08:41:41.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.326+0000 7fa723fff640 1 -- 172.21.15.98:0/250774118 learned_addr learned my addr 172.21.15.98:0/250774118 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:41:41.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.326+0000 7fa7237fe640 1 -- 172.21.15.98:0/250774118 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7240cef60 msgr2=0x7fa724071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:41.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.326+0000 7fa722ffd640 1 --2- 172.21.15.98:0/250774118 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7240cef60 0x7fa724071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:41.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.326+0000 7fa7227fc640 1 -- 172.21.15.98:0/250774118 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1563312349 0 0) 0x7fa7240cffe0 con 0x7fa724071a60 2024-09-19T08:41:41.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.326+0000 7fa7237fe640 1 --2- 172.21.15.98:0/250774118 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7240cef60 0x7fa724071520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:41.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.326+0000 7fa7237fe640 1 -- 172.21.15.98:0/250774118 >> v1:172.21.15.98:6789/0 conn(0x7fa724071a60 legacy=0x7fa72415e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:41.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.326+0000 7fa7237fe640 1 -- 172.21.15.98:0/250774118 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7240cfa20 con 0x7fa72415dee0 2024-09-19T08:41:41.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.326+0000 7fa722ffd640 1 --2- 172.21.15.98:0/250774118 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7240cef60 0x7fa724071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:41.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.326+0000 7fa7237fe640 1 --2- 172.21.15.98:0/250774118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa72415dee0 0x7fa7240afaa0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa7140097c0 tx=0x7fa71401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6376f16e9e19956 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:41.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.326+0000 7fa7227fc640 1 -- 172.21.15.98:0/250774118 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7140b0070 con 0x7fa72415dee0 2024-09-19T08:41:41.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.326+0000 7fa7227fc640 1 -- 172.21.15.98:0/250774118 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7140ab600 con 0x7fa72415dee0 2024-09-19T08:41:41.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.326+0000 7fa7227fc640 1 -- 172.21.15.98:0/250774118 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7140ab9d0 con 0x7fa72415dee0 2024-09-19T08:41:41.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.330+0000 7fa729e53640 1 -- 172.21.15.98:0/250774118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa72415dee0 msgr2=0x7fa7240afaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:41.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.330+0000 7fa729e53640 1 --2- 172.21.15.98:0/250774118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa72415dee0 0x7fa7240afaa0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa7140097c0 tx=0x7fa71401c4a0 comp rx=0 tx=0).stop 2024-09-19T08:41:41.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.330+0000 7fa729e53640 1 -- 172.21.15.98:0/250774118 shutdown_connections 2024-09-19T08:41:41.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.330+0000 7fa729e53640 1 --2- 172.21.15.98:0/250774118 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7240cef60 0x7fa724071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:41.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.330+0000 7fa729e53640 1 --2- 172.21.15.98:0/250774118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa72415dee0 0x7fa7240afaa0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:41.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.330+0000 7fa729e53640 1 -- 172.21.15.98:0/250774118 >> 172.21.15.98:0/250774118 conn(0x7fa7240d0980 msgr2=0x7fa7240d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:41.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.330+0000 7fa729e53640 1 -- 172.21.15.98:0/250774118 shutdown_connections 2024-09-19T08:41:41.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.330+0000 7fa729e53640 1 -- 172.21.15.98:0/250774118 wait complete. 2024-09-19T08:41:41.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.330+0000 7fa729e53640 1 Processor -- start 2024-09-19T08:41:41.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.330+0000 7fa729e53640 1 -- start start 2024-09-19T08:41:41.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.330+0000 7fa729e53640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa724071a60 0x7fa72415f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:41.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.330+0000 7fa729e53640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7240cef60 0x7fa7240caa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:41.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.330+0000 7fa729e53640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa72415dee0 0x7fa7240c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:41.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.330+0000 7fa729e53640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa7240685b0 con 0x7fa72415dee0 2024-09-19T08:41:41.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.330+0000 7fa729e53640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa724068430 con 0x7fa724071a60 2024-09-19T08:41:41.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.330+0000 7fa729e53640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa724068730 con 0x7fa7240cef60 2024-09-19T08:41:41.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.330+0000 7fa723fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa72415dee0 0x7fa7240c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:41.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.330+0000 7fa723fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa72415dee0 0x7fa7240c8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40938/0 (socket says 172.21.15.98:40938) 2024-09-19T08:41:41.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.330+0000 7fa723fff640 1 -- 172.21.15.98:0/3690138494 learned_addr learned my addr 172.21.15.98:0/3690138494 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:41.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.330+0000 7fa722ffd640 1 --2- 172.21.15.98:0/3690138494 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7240cef60 0x7fa7240caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:41.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.330+0000 7fa7237fe640 1 --2- 172.21.15.98:0/3690138494 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa724071a60 0x7fa72415f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:41.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.330+0000 7fa723fff640 1 -- 172.21.15.98:0/3690138494 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7240cef60 msgr2=0x7fa7240caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:41.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.330+0000 7fa723fff640 1 --2- 172.21.15.98:0/3690138494 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7240cef60 0x7fa7240caa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:41.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.330+0000 7fa723fff640 1 -- 172.21.15.98:0/3690138494 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa724071a60 msgr2=0x7fa72415f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:41.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.330+0000 7fa723fff640 1 --2- 172.21.15.98:0/3690138494 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa724071a60 0x7fa72415f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:41.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.330+0000 7fa723fff640 1 -- 172.21.15.98:0/3690138494 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa72414d2d0 con 0x7fa72415dee0 2024-09-19T08:41:41.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.330+0000 7fa723fff640 1 --2- 172.21.15.98:0/3690138494 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa72415dee0 0x7fa7240c8260 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa71c00c780 tx=0x7fa71c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:41.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.330+0000 7fa70bfff640 1 -- 172.21.15.98:0/3690138494 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71c018070 con 0x7fa72415dee0 2024-09-19T08:41:41.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.330+0000 7fa70bfff640 1 -- 172.21.15.98:0/3690138494 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa71c00a640 con 0x7fa72415dee0 2024-09-19T08:41:41.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.330+0000 7fa729e53640 1 -- 172.21.15.98:0/3690138494 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa72414d570 con 0x7fa72415dee0 2024-09-19T08:41:41.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.330+0000 7fa70bfff640 1 -- 172.21.15.98:0/3690138494 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71c00aa30 con 0x7fa72415dee0 2024-09-19T08:41:41.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.330+0000 7fa729e53640 1 -- 172.21.15.98:0/3690138494 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa72414db30 con 0x7fa72415dee0 2024-09-19T08:41:41.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.330+0000 7fa729e53640 1 -- 172.21.15.98:0/3690138494 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa724071520 con 0x7fa72415dee0 2024-09-19T08:41:41.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.334+0000 7fa70bfff640 1 -- 172.21.15.98:0/3690138494 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fa71c01b070 con 0x7fa72415dee0 2024-09-19T08:41:41.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.334+0000 7fa70bfff640 1 --2- 172.21.15.98:0/3690138494 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa710067aa0 0x7fa710069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:41.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.334+0000 7fa7237fe640 1 --2- 172.21.15.98:0/3690138494 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa710067aa0 0x7fa710069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:41.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.334+0000 7fa70bfff640 1 -- 172.21.15.98:0/3690138494 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4943+0+0 (secure 0 0 0) 0x7fa71c085220 con 0x7fa72415dee0 2024-09-19T08:41:41.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.334+0000 7fa7237fe640 1 --2- 172.21.15.98:0/3690138494 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa710067aa0 0x7fa710069f60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa7140ad000 tx=0x7fa7140a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:41.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.338+0000 7fa70bfff640 1 -- 172.21.15.98:0/3690138494 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa71c055de0 con 0x7fa72415dee0 2024-09-19T08:41:41.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.554+0000 7fa729e53640 1 -- 172.21.15.98:0/3690138494 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7fa7240ceb20 con 0x7fa72415dee0 2024-09-19T08:41:41.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.562+0000 7fa70bfff640 1 -- 172.21.15.98:0/3690138494 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]}]=0 v176) ==== 164+0+0 (secure 0 0 0) 0x7fa71c0556e0 con 0x7fa72415dee0 2024-09-19T08:41:41.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.562+0000 7fa729e53640 1 -- 172.21.15.98:0/3690138494 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa710067aa0 msgr2=0x7fa710069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:41.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.562+0000 7fa729e53640 1 --2- 172.21.15.98:0/3690138494 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa710067aa0 0x7fa710069f60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa7140ad000 tx=0x7fa7140a6000 comp rx=0 tx=0).stop 2024-09-19T08:41:41.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.562+0000 7fa729e53640 1 -- 172.21.15.98:0/3690138494 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa72415dee0 msgr2=0x7fa7240c8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:41.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.562+0000 7fa729e53640 1 --2- 172.21.15.98:0/3690138494 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa72415dee0 0x7fa7240c8260 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa71c00c780 tx=0x7fa71c00cc40 comp rx=0 tx=0).stop 2024-09-19T08:41:41.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.562+0000 7fa729e53640 1 -- 172.21.15.98:0/3690138494 shutdown_connections 2024-09-19T08:41:41.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.562+0000 7fa729e53640 1 --2- 172.21.15.98:0/3690138494 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa710067aa0 0x7fa710069f60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:41.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.562+0000 7fa729e53640 1 --2- 172.21.15.98:0/3690138494 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa72415dee0 0x7fa7240c8260 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:41.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.562+0000 7fa729e53640 1 --2- 172.21.15.98:0/3690138494 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7240cef60 0x7fa7240caa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:41.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.562+0000 7fa729e53640 1 --2- 172.21.15.98:0/3690138494 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa724071a60 0x7fa72415f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:41.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.562+0000 7fa729e53640 1 -- 172.21.15.98:0/3690138494 >> 172.21.15.98:0/3690138494 conn(0x7fa7240d0980 msgr2=0x7fa72406f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:41.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.562+0000 7fa729e53640 1 -- 172.21.15.98:0/3690138494 shutdown_connections 2024-09-19T08:41:41.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.562+0000 7fa729e53640 1 -- 172.21.15.98:0/3690138494 wait complete. 2024-09-19T08:41:41.579 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T08:41:41.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.702+0000 7f64e1fbd640 1 Processor -- start 2024-09-19T08:41:41.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.702+0000 7f64e1fbd640 1 -- start start 2024-09-19T08:41:41.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.702+0000 7f64e1fbd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f64dc144d10 0x7f64dc0722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:41.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.702+0000 7f64e1fbd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f64dc072820 0x7f64dc06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:41.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.702+0000 7f64e1fbd640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64dc072ea0 con 0x7f64dc06c1d0 2024-09-19T08:41:41.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.702+0000 7f64e1fbd640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f64dc06c640 con 0x7f64dc144d10 2024-09-19T08:41:41.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.702+0000 7f64e1fbd640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f64dc072c60 con 0x7f64dc072820 2024-09-19T08:41:41.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.702+0000 7f64dbfff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f64dc06c1d0 0x7f64dc0b4520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:37142/0 (socket says 172.21.15.98:37142) 2024-09-19T08:41:41.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.702+0000 7f64dbfff640 1 -- 172.21.15.98:0/2240412420 learned_addr learned my addr 172.21.15.98:0/2240412420 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:41:41.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.702+0000 7f64db7fe640 1 --2- 172.21.15.98:0/2240412420 >> v2:172.21.15.98:3300/0 conn(0x7f64dc144d10 0x7f64dc0722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:41.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.702+0000 7f64daffd640 1 --2- 172.21.15.98:0/2240412420 >> v2:172.21.15.139:3300/0 conn(0x7f64dc072820 0x7f64dc06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:41.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.702+0000 7f64da7fc640 1 -- 172.21.15.98:0/2240412420 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3139396706 0 0) 0x7f64dc072ea0 con 0x7f64dc06c1d0 2024-09-19T08:41:41.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.702+0000 7f64da7fc640 1 -- 172.21.15.98:0/2240412420 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64c4003530 con 0x7f64dc06c1d0 2024-09-19T08:41:41.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.702+0000 7f64db7fe640 1 -- 172.21.15.98:0/2240412420 >> v2:172.21.15.139:3300/0 conn(0x7f64dc072820 msgr2=0x7f64dc06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:41.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.702+0000 7f64db7fe640 1 --2- 172.21.15.98:0/2240412420 >> v2:172.21.15.139:3300/0 conn(0x7f64dc072820 0x7f64dc06bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:41.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.702+0000 7f64da7fc640 1 -- 172.21.15.98:0/2240412420 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1652779548 0 0) 0x7f64c4003530 con 0x7f64dc06c1d0 2024-09-19T08:41:41.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.702+0000 7f64db7fe640 1 -- 172.21.15.98:0/2240412420 >> v1:172.21.15.98:6789/0 conn(0x7f64dc06c1d0 legacy=0x7f64dc0b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:41.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.702+0000 7f64db7fe640 1 -- 172.21.15.98:0/2240412420 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64dc073180 con 0x7f64dc144d10 2024-09-19T08:41:41.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.706+0000 7f64db7fe640 1 --2- 172.21.15.98:0/2240412420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64dc144d10 0x7f64dc0722e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f64dc0aff50 tx=0x7f64cc01c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16cf0bac8b0ae5d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:41.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.706+0000 7f64da7fc640 1 -- 172.21.15.98:0/2240412420 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64cc0b0070 con 0x7f64dc144d10 2024-09-19T08:41:41.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.706+0000 7f64da7fc640 1 -- 172.21.15.98:0/2240412420 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64cc0ab400 con 0x7f64dc144d10 2024-09-19T08:41:41.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.706+0000 7f64da7fc640 1 -- 172.21.15.98:0/2240412420 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d4001a00 con 0x7f64dc144d10 2024-09-19T08:41:41.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.706+0000 7f64e1fbd640 1 -- 172.21.15.98:0/2240412420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64dc144d10 msgr2=0x7f64dc0722e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:41.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.706+0000 7f64e1fbd640 1 --2- 172.21.15.98:0/2240412420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64dc144d10 0x7f64dc0722e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f64dc0aff50 tx=0x7f64cc01c3a0 comp rx=0 tx=0).stop 2024-09-19T08:41:41.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.706+0000 7f64e1fbd640 1 -- 172.21.15.98:0/2240412420 shutdown_connections 2024-09-19T08:41:41.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.706+0000 7f64e1fbd640 1 --2- 172.21.15.98:0/2240412420 >> v2:172.21.15.139:3300/0 conn(0x7f64dc072820 0x7f64dc06bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:41.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.706+0000 7f64e1fbd640 1 --2- 172.21.15.98:0/2240412420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64dc144d10 0x7f64dc0722e0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:41.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.706+0000 7f64e1fbd640 1 -- 172.21.15.98:0/2240412420 >> 172.21.15.98:0/2240412420 conn(0x7f64dc0b2010 msgr2=0x7f64dc0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:41.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.706+0000 7f64e1fbd640 1 -- 172.21.15.98:0/2240412420 shutdown_connections 2024-09-19T08:41:41.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.706+0000 7f64e1fbd640 1 -- 172.21.15.98:0/2240412420 wait complete. 2024-09-19T08:41:41.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.706+0000 7f64e1fbd640 1 Processor -- start 2024-09-19T08:41:41.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.706+0000 7f64e1fbd640 1 -- start start 2024-09-19T08:41:41.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.710+0000 7f64e1fbd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64dc06c1d0 0x7f64dc149100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:41.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.710+0000 7f64e1fbd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64dc072820 0x7f64dc0b3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:41.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.710+0000 7f64e1fbd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64dc144d10 0x7f64dc13ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:41.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.710+0000 7f64e1fbd640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f64dc1680e0 con 0x7f64dc072820 2024-09-19T08:41:41.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.710+0000 7f64e1fbd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f64dc167f60 con 0x7f64dc144d10 2024-09-19T08:41:41.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.710+0000 7f64e1fbd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f64dc168260 con 0x7f64dc06c1d0 2024-09-19T08:41:41.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.710+0000 7f64daffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64dc072820 0x7f64dc0b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:41.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.710+0000 7f64daffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64dc072820 0x7f64dc0b3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40970/0 (socket says 172.21.15.98:40970) 2024-09-19T08:41:41.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.710+0000 7f64daffd640 1 -- 172.21.15.98:0/2426171776 learned_addr learned my addr 172.21.15.98:0/2426171776 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:41.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.710+0000 7f64db7fe640 1 --2- 172.21.15.98:0/2426171776 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64dc06c1d0 0x7f64dc149100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:41.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.710+0000 7f64dbfff640 1 --2- 172.21.15.98:0/2426171776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64dc144d10 0x7f64dc13ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:41.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.710+0000 7f64daffd640 1 -- 172.21.15.98:0/2426171776 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64dc06c1d0 msgr2=0x7f64dc149100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:41.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.710+0000 7f64daffd640 1 --2- 172.21.15.98:0/2426171776 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64dc06c1d0 0x7f64dc149100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:41.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.710+0000 7f64daffd640 1 -- 172.21.15.98:0/2426171776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64dc144d10 msgr2=0x7f64dc13ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:41.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.710+0000 7f64daffd640 1 --2- 172.21.15.98:0/2426171776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64dc144d10 0x7f64dc13ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:41.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.710+0000 7f64daffd640 1 -- 172.21.15.98:0/2426171776 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64dc149640 con 0x7f64dc072820 2024-09-19T08:41:41.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.710+0000 7f64dbfff640 1 --2- 172.21.15.98:0/2426171776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64dc144d10 0x7f64dc13ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:41.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.710+0000 7f64daffd640 1 --2- 172.21.15.98:0/2426171776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64dc072820 0x7f64dc0b3670 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f64d000e930 tx=0x7f64d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:41.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.710+0000 7f64c3fff640 1 -- 172.21.15.98:0/2426171776 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d0017070 con 0x7f64dc072820 2024-09-19T08:41:41.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.710+0000 7f64c3fff640 1 -- 172.21.15.98:0/2426171776 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64d000ca00 con 0x7f64dc072820 2024-09-19T08:41:41.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.710+0000 7f64e1fbd640 1 -- 172.21.15.98:0/2426171776 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64dc145630 con 0x7f64dc072820 2024-09-19T08:41:41.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.710+0000 7f64c3fff640 1 -- 172.21.15.98:0/2426171776 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d000cdb0 con 0x7f64dc072820 2024-09-19T08:41:41.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.710+0000 7f64e1fbd640 1 -- 172.21.15.98:0/2426171776 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64dc145bf0 con 0x7f64dc072820 2024-09-19T08:41:41.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.710+0000 7f64e1fbd640 1 -- 172.21.15.98:0/2426171776 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64dc06bc90 con 0x7f64dc072820 2024-09-19T08:41:41.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.710+0000 7f64c3fff640 1 -- 172.21.15.98:0/2426171776 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f64d001a070 con 0x7f64dc072820 2024-09-19T08:41:41.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.710+0000 7f64c3fff640 1 --2- 172.21.15.98:0/2426171776 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f64c8067a70 0x7f64c8069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:41.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.710+0000 7f64db7fe640 1 --2- 172.21.15.98:0/2426171776 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f64c8067a70 0x7f64c8069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:41.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.710+0000 7f64c3fff640 1 -- 172.21.15.98:0/2426171776 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4943+0+0 (secure 0 0 0) 0x7f64d0085c50 con 0x7f64dc072820 2024-09-19T08:41:41.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.710+0000 7f64db7fe640 1 --2- 172.21.15.98:0/2426171776 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f64c8067a70 0x7f64c8069f30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f64cc0ad000 tx=0x7f64cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:41.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.714+0000 7f64c3fff640 1 -- 172.21.15.98:0/2426171776 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f64d008b020 con 0x7f64dc072820 2024-09-19T08:41:41.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.878+0000 7f64e1fbd640 1 -- 172.21.15.98:0/2426171776 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f64dc13e5d0 con 0x7f64dc072820 2024-09-19T08:41:41.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.882+0000 7f64c3fff640 1 -- 172.21.15.98:0/2426171776 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7f64dc149640 con 0x7f64dc072820 2024-09-19T08:41:41.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.886+0000 7f64e1fbd640 1 -- 172.21.15.98:0/2426171776 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f64c8067a70 msgr2=0x7f64c8069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:41.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.886+0000 7f64e1fbd640 1 --2- 172.21.15.98:0/2426171776 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f64c8067a70 0x7f64c8069f30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f64cc0ad000 tx=0x7f64cc008040 comp rx=0 tx=0).stop 2024-09-19T08:41:41.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.886+0000 7f64e1fbd640 1 -- 172.21.15.98:0/2426171776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64dc072820 msgr2=0x7f64dc0b3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:41.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.886+0000 7f64e1fbd640 1 --2- 172.21.15.98:0/2426171776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64dc072820 0x7f64dc0b3670 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f64d000e930 tx=0x7f64d000edf0 comp rx=0 tx=0).stop 2024-09-19T08:41:41.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.886+0000 7f64e1fbd640 1 -- 172.21.15.98:0/2426171776 shutdown_connections 2024-09-19T08:41:41.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.886+0000 7f64e1fbd640 1 --2- 172.21.15.98:0/2426171776 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f64c8067a70 0x7f64c8069f30 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:41.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.886+0000 7f64e1fbd640 1 --2- 172.21.15.98:0/2426171776 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64dc144d10 0x7f64dc13ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:41.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.886+0000 7f64e1fbd640 1 --2- 172.21.15.98:0/2426171776 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64dc072820 0x7f64dc0b3670 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:41.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.886+0000 7f64e1fbd640 1 --2- 172.21.15.98:0/2426171776 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64dc06c1d0 0x7f64dc149100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:41.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.886+0000 7f64e1fbd640 1 -- 172.21.15.98:0/2426171776 >> 172.21.15.98:0/2426171776 conn(0x7f64dc0b2010 msgr2=0x7f64dc13db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:41.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.886+0000 7f64e1fbd640 1 -- 172.21.15.98:0/2426171776 shutdown_connections 2024-09-19T08:41:41.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:41.886+0000 7f64e1fbd640 1 -- 172.21.15.98:0/2426171776 wait complete. 2024-09-19T08:41:41.901 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T08:41:41.901 DEBUG:teuthology.orchestra.run.smithi098:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:41:41.973 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:41:42.051 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T08:41:42.051 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:41:42.051 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:41:42.057 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15846 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T08:41:42.058 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:41:42.058 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:41:42.063 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T08:41:42.063 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:41:42.063 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:41:42.105 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15846 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T08:41:42.105 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:41:42.106 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:41:42.111 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T08:41:42.111 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-19T08:41:42.111 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:41:42.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.278+0000 7f9f33851640 1 Processor -- start 2024-09-19T08:41:42.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.278+0000 7f9f33851640 1 -- start start 2024-09-19T08:41:42.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.278+0000 7f9f33851640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9f2c0733b0 0x7f9f2c0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:42.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.278+0000 7f9f33851640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f2c06d310 con 0x7f9f2c15c550 2024-09-19T08:41:42.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.278+0000 7f9f33851640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f2c06d5f0 con 0x7f9f2c06c3a0 2024-09-19T08:41:42.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.278+0000 7f9f33851640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f9f2c06d8d0 con 0x7f9f2c0733b0 2024-09-19T08:41:42.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.278+0000 7f9f315ed640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f9f2c15c550 0x7f9f2c06bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:37150/0 (socket says 172.21.15.98:37150) 2024-09-19T08:41:42.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.278+0000 7f9f315ed640 1 -- 172.21.15.98:0/1746195300 learned_addr learned my addr 172.21.15.98:0/1746195300 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:41:42.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.278+0000 7f9f31dee640 1 --2- 172.21.15.98:0/1746195300 >> v2:172.21.15.98:3300/0 conn(0x7f9f2c0733b0 0x7f9f2c0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:42.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.278+0000 7f9f23fff640 1 -- 172.21.15.98:0/1746195300 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1470665270 0 0) 0x7f9f2c06d310 con 0x7f9f2c15c550 2024-09-19T08:41:42.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.278+0000 7f9f23fff640 1 -- 172.21.15.98:0/1746195300 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f18003530 con 0x7f9f2c15c550 2024-09-19T08:41:42.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.278+0000 7f9f31dee640 1 -- 172.21.15.98:0/1746195300 >> v1:172.21.15.139:6789/0 conn(0x7f9f2c06c3a0 legacy=0x7f9f2c0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:42.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.278+0000 7f9f31dee640 1 -- 172.21.15.98:0/1746195300 >> v1:172.21.15.98:6789/0 conn(0x7f9f2c15c550 legacy=0x7f9f2c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:42.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.278+0000 7f9f31dee640 1 -- 172.21.15.98:0/1746195300 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f2c06dbb0 con 0x7f9f2c0733b0 2024-09-19T08:41:42.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.278+0000 7f9f31dee640 1 --2- 172.21.15.98:0/1746195300 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f2c0733b0 0x7f9f2c0cfad0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9f24009d90 tx=0x7f9f2401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4f0972ecaec3be9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:42.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.282+0000 7f9f23fff640 1 -- 172.21.15.98:0/1746195300 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f240b2070 con 0x7f9f2c0733b0 2024-09-19T08:41:42.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.282+0000 7f9f23fff640 1 -- 172.21.15.98:0/1746195300 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f240a57f0 con 0x7f9f2c0733b0 2024-09-19T08:41:42.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.282+0000 7f9f23fff640 1 -- 172.21.15.98:0/1746195300 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f240a5bc0 con 0x7f9f2c0733b0 2024-09-19T08:41:42.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.282+0000 7f9f33851640 1 -- 172.21.15.98:0/1746195300 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f2c0733b0 msgr2=0x7f9f2c0cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:42.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.282+0000 7f9f33851640 1 --2- 172.21.15.98:0/1746195300 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f2c0733b0 0x7f9f2c0cfad0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9f24009d90 tx=0x7f9f2401c930 comp rx=0 tx=0).stop 2024-09-19T08:41:42.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.282+0000 7f9f33851640 1 -- 172.21.15.98:0/1746195300 shutdown_connections 2024-09-19T08:41:42.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.282+0000 7f9f33851640 1 --2- 172.21.15.98:0/1746195300 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f2c0733b0 0x7f9f2c0cfad0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:42.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.282+0000 7f9f33851640 1 -- 172.21.15.98:0/1746195300 >> 172.21.15.98:0/1746195300 conn(0x7f9f2c0b2010 msgr2=0x7f9f2c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:42.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.282+0000 7f9f33851640 1 -- 172.21.15.98:0/1746195300 shutdown_connections 2024-09-19T08:41:42.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.282+0000 7f9f33851640 1 -- 172.21.15.98:0/1746195300 wait complete. 2024-09-19T08:41:42.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.282+0000 7f9f33851640 1 Processor -- start 2024-09-19T08:41:42.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.282+0000 7f9f33851640 1 -- start start 2024-09-19T08:41:42.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.282+0000 7f9f33851640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f2c06c3a0 0x7f9f2c0afe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:42.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.282+0000 7f9f33851640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f2c0733b0 0x7f9f2c0b03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:42.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.282+0000 7f9f33851640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f2c15c550 0x7f9f2c14a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:42.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.282+0000 7f9f33851640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c169dd0 con 0x7f9f2c15c550 2024-09-19T08:41:42.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.282+0000 7f9f33851640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c169c50 con 0x7f9f2c06c3a0 2024-09-19T08:41:42.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.282+0000 7f9f33851640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9f2c169f50 con 0x7f9f2c0733b0 2024-09-19T08:41:42.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.282+0000 7f9f31dee640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f2c15c550 0x7f9f2c14a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:42.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.282+0000 7f9f31dee640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f2c15c550 0x7f9f2c14a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40994/0 (socket says 172.21.15.98:40994) 2024-09-19T08:41:42.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.282+0000 7f9f31dee640 1 -- 172.21.15.98:0/1489233087 learned_addr learned my addr 172.21.15.98:0/1489233087 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:42.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.282+0000 7f9f315ed640 1 --2- 172.21.15.98:0/1489233087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f2c06c3a0 0x7f9f2c0afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:42.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.282+0000 7f9f31dee640 1 -- 172.21.15.98:0/1489233087 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f2c0733b0 msgr2=0x7f9f2c0b03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:42.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.282+0000 7f9f31dee640 1 --2- 172.21.15.98:0/1489233087 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f2c0733b0 0x7f9f2c0b03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:42.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.282+0000 7f9f31dee640 1 -- 172.21.15.98:0/1489233087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f2c06c3a0 msgr2=0x7f9f2c0afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:42.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.282+0000 7f9f31dee640 1 --2- 172.21.15.98:0/1489233087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f2c06c3a0 0x7f9f2c0afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:42.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.282+0000 7f9f31dee640 1 -- 172.21.15.98:0/1489233087 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f2c14ec20 con 0x7f9f2c15c550 2024-09-19T08:41:42.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.282+0000 7f9f315ed640 1 --2- 172.21.15.98:0/1489233087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f2c06c3a0 0x7f9f2c0afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:42.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.282+0000 7f9f31dee640 1 --2- 172.21.15.98:0/1489233087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f2c15c550 0x7f9f2c14a230 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9f240b0040 tx=0x7f9f240196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:42.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.282+0000 7f9f21ffb640 1 -- 172.21.15.98:0/1489233087 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f240b2050 con 0x7f9f2c15c550 2024-09-19T08:41:42.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.282+0000 7f9f21ffb640 1 -- 172.21.15.98:0/1489233087 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f240be070 con 0x7f9f2c15c550 2024-09-19T08:41:42.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.282+0000 7f9f33851640 1 -- 172.21.15.98:0/1489233087 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f2c14eec0 con 0x7f9f2c15c550 2024-09-19T08:41:42.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.282+0000 7f9f21ffb640 1 -- 172.21.15.98:0/1489233087 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24009450 con 0x7f9f2c15c550 2024-09-19T08:41:42.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.282+0000 7f9f33851640 1 -- 172.21.15.98:0/1489233087 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f2c15fb30 con 0x7f9f2c15c550 2024-09-19T08:41:42.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.286+0000 7f9f33851640 1 -- 172.21.15.98:0/1489233087 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f2c06bc90 con 0x7f9f2c15c550 2024-09-19T08:41:42.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.286+0000 7f9f21ffb640 1 -- 172.21.15.98:0/1489233087 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f9f240adc40 con 0x7f9f2c15c550 2024-09-19T08:41:42.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.286+0000 7f9f21ffb640 1 --2- 172.21.15.98:0/1489233087 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9f10067aa0 0x7f9f10069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:42.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.286+0000 7f9f21ffb640 1 -- 172.21.15.98:0/1489233087 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4943+0+0 (secure 0 0 0) 0x7f9f24121270 con 0x7f9f2c15c550 2024-09-19T08:41:42.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.286+0000 7f9f315ed640 1 --2- 172.21.15.98:0/1489233087 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9f10067aa0 0x7f9f10069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:42.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.286+0000 7f9f315ed640 1 --2- 172.21.15.98:0/1489233087 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9f10067aa0 0x7f9f10069f60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9f28005890 tx=0x7f9f28009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:42.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.290+0000 7f9f21ffb640 1 -- 172.21.15.98:0/1489233087 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9f240c7020 con 0x7f9f2c15c550 2024-09-19T08:41:42.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.470+0000 7f9f33851640 1 -- 172.21.15.98:0/1489233087 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f2c0d0e90 con 0x7f9f2c15c550 2024-09-19T08:41:42.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.470+0000 7f9f21ffb640 1 -- 172.21.15.98:0/1489233087 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+3917 (secure 0 0 0) 0x7f9f24016520 con 0x7f9f2c15c550 2024-09-19T08:41:42.471 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:41:42.471 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":26,"btime":"2024-09-19T08:41:37:524394+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/2874086421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":2874086421},{"type":"v1","addr":"172.21.15.139:6833","nonce":2874086421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4996,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/4178563848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":4178563848},{"type":"v1","addr":"172.21.15.139:6835","nonce":4178563848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5241,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3584568343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3584568343},{"type":"v1","addr":"172.21.15.98:6835","nonce":3584568343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5667,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/3832204573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3832204573},{"type":"v1","addr":"172.21.15.98:6837","nonce":3832204573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[]} 2024-09-19T08:41:42.471 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 26 2024-09-19T08:41:42.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.474+0000 7f9f33851640 1 -- 172.21.15.98:0/1489233087 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9f10067aa0 msgr2=0x7f9f10069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:42.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.474+0000 7f9f33851640 1 --2- 172.21.15.98:0/1489233087 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9f10067aa0 0x7f9f10069f60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9f28005890 tx=0x7f9f28009040 comp rx=0 tx=0).stop 2024-09-19T08:41:42.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.474+0000 7f9f33851640 1 -- 172.21.15.98:0/1489233087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f2c15c550 msgr2=0x7f9f2c14a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:42.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.474+0000 7f9f33851640 1 --2- 172.21.15.98:0/1489233087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f2c15c550 0x7f9f2c14a230 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9f240b0040 tx=0x7f9f240196d0 comp rx=0 tx=0).stop 2024-09-19T08:41:42.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.474+0000 7f9f33851640 1 -- 172.21.15.98:0/1489233087 shutdown_connections 2024-09-19T08:41:42.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.474+0000 7f9f33851640 1 --2- 172.21.15.98:0/1489233087 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9f10067aa0 0x7f9f10069f60 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:42.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.474+0000 7f9f33851640 1 --2- 172.21.15.98:0/1489233087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f2c15c550 0x7f9f2c14a230 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:42.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.474+0000 7f9f33851640 1 --2- 172.21.15.98:0/1489233087 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f2c0733b0 0x7f9f2c0b03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:42.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.474+0000 7f9f33851640 1 --2- 172.21.15.98:0/1489233087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f2c06c3a0 0x7f9f2c0afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:42.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.474+0000 7f9f33851640 1 -- 172.21.15.98:0/1489233087 >> 172.21.15.98:0/1489233087 conn(0x7f9f2c0b2010 msgr2=0x7f9f2c0cf3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:42.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.474+0000 7f9f33851640 1 -- 172.21.15.98:0/1489233087 shutdown_connections 2024-09-19T08:41:42.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.474+0000 7f9f33851640 1 -- 172.21.15.98:0/1489233087 wait complete. 2024-09-19T08:41:42.490 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-19T08:41:42.490 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-19T08:41:42.490 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-19T08:41:42.490 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:41:42.490 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-19T08:41:42.491 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-19T08:41:42.491 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-19T08:41:42.491 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:41:42.491 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-19T08:41:42.491 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-19T08:41:42.491 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-19T08:41:42.491 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:41:42.492 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-19T08:41:42.492 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-19T08:41:42.492 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-19T08:41:42.492 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:41:42.500 INFO:tasks.ceph.mds.c:Stopped 2024-09-19T08:41:42.500 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-19T08:41:42.502 INFO:tasks.ceph.mds.a:Stopped 2024-09-19T08:41:42.502 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-19T08:41:42.507 INFO:tasks.ceph.mds.d:Stopped 2024-09-19T08:41:42.507 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-19T08:41:42.508 INFO:tasks.ceph.mds.b:Stopped 2024-09-19T08:41:42.508 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-19T08:41:42.545 INFO:tasks.ceph.mds.c:Started 2024-09-19T08:41:42.545 DEBUG:teuthology.parallel:result is None 2024-09-19T08:41:42.546 INFO:tasks.ceph.mds.a:Started 2024-09-19T08:41:42.546 DEBUG:teuthology.parallel:result is None 2024-09-19T08:41:42.554 INFO:tasks.ceph.mds.d:Started 2024-09-19T08:41:42.555 DEBUG:teuthology.parallel:result is None 2024-09-19T08:41:42.556 INFO:tasks.ceph.mds.b:Started 2024-09-19T08:41:42.556 DEBUG:teuthology.parallel:result is None 2024-09-19T08:41:42.557 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-19T08:41:42.609 INFO:tasks.ceph.mds.c.smithi098.stdout:starting mds.c at 2024-09-19T08:41:42.611 INFO:tasks.ceph.mds.a.smithi098.stdout:starting mds.a at 2024-09-19T08:41:42.627 INFO:tasks.ceph.mds.b.smithi139.stdout:starting mds.b at 2024-09-19T08:41:42.627 INFO:tasks.ceph.mds.d.smithi139.stdout:starting mds.d at 2024-09-19T08:41:42.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.666+0000 7f59aea1f640 1 Processor -- start 2024-09-19T08:41:42.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.666+0000 7f59aea1f640 1 -- start start 2024-09-19T08:41:42.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.670+0000 7f59aea1f640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f59a80b0000 0x7f59a815d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:42.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.670+0000 7f59aea1f640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59a806eb00 con 0x7f59a80732a0 2024-09-19T08:41:42.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.670+0000 7f59aea1f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59a806fe40 con 0x7f59a80d0140 2024-09-19T08:41:42.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.670+0000 7f59aea1f640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f59a806eda0 con 0x7f59a80b0000 2024-09-19T08:41:42.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.670+0000 7f59a7fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f59a80b0000 0x7f59a815d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:42.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.670+0000 7f59acfbc640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f59a80732a0 0x7f59a80ce550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:37164/0 (socket says 172.21.15.98:37164) 2024-09-19T08:41:42.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.670+0000 7f59acfbc640 1 -- 172.21.15.98:0/742215763 learned_addr learned my addr 172.21.15.98:0/742215763 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:41:42.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.670+0000 7f59a7fff640 1 -- 172.21.15.98:0/742215763 >> v1:172.21.15.139:6789/0 conn(0x7f59a80d0140 legacy=0x7f59a8072b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:42.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.670+0000 7f59a7fff640 1 -- 172.21.15.98:0/742215763 >> v1:172.21.15.98:6789/0 conn(0x7f59a80732a0 legacy=0x7f59a80ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:42.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.670+0000 7f59a77fe640 1 -- 172.21.15.98:0/742215763 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3224938295 0 0) 0x7f59a806eb00 con 0x7f59a80732a0 2024-09-19T08:41:42.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.670+0000 7f59a7fff640 1 -- 172.21.15.98:0/742215763 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a806f080 con 0x7f59a80b0000 2024-09-19T08:41:42.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.670+0000 7f59a7fff640 1 --2- 172.21.15.98:0/742215763 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59a80b0000 0x7f59a815d250 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f59980097f0 tx=0x7f599801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a4687ff2ffd7826 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:42.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.670+0000 7f59a77fe640 1 -- 172.21.15.98:0/742215763 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59980b0070 con 0x7f59a80b0000 2024-09-19T08:41:42.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.670+0000 7f59a77fe640 1 -- 172.21.15.98:0/742215763 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5998004230 con 0x7f59a80b0000 2024-09-19T08:41:42.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.670+0000 7f59a77fe640 1 -- 172.21.15.98:0/742215763 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5998003620 con 0x7f59a80b0000 2024-09-19T08:41:42.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.670+0000 7f59aea1f640 1 -- 172.21.15.98:0/742215763 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59a80b0000 msgr2=0x7f59a815d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:42.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.674+0000 7f59aea1f640 1 --2- 172.21.15.98:0/742215763 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59a80b0000 0x7f59a815d250 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f59980097f0 tx=0x7f599801c460 comp rx=0 tx=0).stop 2024-09-19T08:41:42.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.674+0000 7f59aea1f640 1 -- 172.21.15.98:0/742215763 shutdown_connections 2024-09-19T08:41:42.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.674+0000 7f59aea1f640 1 --2- 172.21.15.98:0/742215763 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59a80b0000 0x7f59a815d250 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:42.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.674+0000 7f59aea1f640 1 -- 172.21.15.98:0/742215763 >> 172.21.15.98:0/742215763 conn(0x7f59a80d0980 msgr2=0x7f59a80d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:42.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.674+0000 7f59aea1f640 1 -- 172.21.15.98:0/742215763 shutdown_connections 2024-09-19T08:41:42.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.674+0000 7f59aea1f640 1 -- 172.21.15.98:0/742215763 wait complete. 2024-09-19T08:41:42.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.674+0000 7f59aea1f640 1 Processor -- start 2024-09-19T08:41:42.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.674+0000 7f59aea1f640 1 -- start start 2024-09-19T08:41:42.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.674+0000 7f59aea1f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f59a80732a0 0x7f59a814b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:42.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.674+0000 7f59aea1f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59a80b0000 0x7f59a80b4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:42.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.674+0000 7f59aea1f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59a80d0140 0x7f59a80cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:42.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.674+0000 7f59aea1f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f59a8169f80 con 0x7f59a80b0000 2024-09-19T08:41:42.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.674+0000 7f59aea1f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f59a8169e00 con 0x7f59a80d0140 2024-09-19T08:41:42.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.674+0000 7f59aea1f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f59a816a100 con 0x7f59a80732a0 2024-09-19T08:41:42.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.674+0000 7f599f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59a80b0000 0x7f59a80b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:42.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.674+0000 7f599f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59a80b0000 0x7f59a80b4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41042/0 (socket says 172.21.15.98:41042) 2024-09-19T08:41:42.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.674+0000 7f599f7fe640 1 -- 172.21.15.98:0/394695052 learned_addr learned my addr 172.21.15.98:0/394695052 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:42.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.674+0000 7f59a7fff640 1 --2- 172.21.15.98:0/394695052 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f59a80732a0 0x7f59a814b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:42.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.674+0000 7f59acfbc640 1 --2- 172.21.15.98:0/394695052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59a80d0140 0x7f59a80cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:42.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.674+0000 7f599f7fe640 1 -- 172.21.15.98:0/394695052 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f59a80732a0 msgr2=0x7f59a814b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:42.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.674+0000 7f599f7fe640 1 --2- 172.21.15.98:0/394695052 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f59a80732a0 0x7f59a814b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:42.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.674+0000 7f599f7fe640 1 -- 172.21.15.98:0/394695052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59a80d0140 msgr2=0x7f59a80cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:42.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.674+0000 7f599f7fe640 1 --2- 172.21.15.98:0/394695052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59a80d0140 0x7f59a80cbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:42.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.674+0000 7f599f7fe640 1 -- 172.21.15.98:0/394695052 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a80cb190 con 0x7f59a80b0000 2024-09-19T08:41:42.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.674+0000 7f59acfbc640 1 --2- 172.21.15.98:0/394695052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59a80d0140 0x7f59a80cbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:41:42.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.674+0000 7f599f7fe640 1 --2- 172.21.15.98:0/394695052 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59a80b0000 0x7f59a80b4230 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f599000b3e0 tx=0x7f599000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:42.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.674+0000 7f59a57fa640 1 -- 172.21.15.98:0/394695052 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5990015020 con 0x7f59a80b0000 2024-09-19T08:41:42.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.674+0000 7f59a57fa640 1 -- 172.21.15.98:0/394695052 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5990011850 con 0x7f59a80b0000 2024-09-19T08:41:42.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.674+0000 7f59a57fa640 1 -- 172.21.15.98:0/394695052 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5990011c40 con 0x7f59a80b0000 2024-09-19T08:41:42.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.674+0000 7f59aea1f640 1 -- 172.21.15.98:0/394695052 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59a80cb470 con 0x7f59a80b0000 2024-09-19T08:41:42.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.674+0000 7f59aea1f640 1 -- 172.21.15.98:0/394695052 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59a80b6390 con 0x7f59a80b0000 2024-09-19T08:41:42.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.674+0000 7f59aea1f640 1 -- 172.21.15.98:0/394695052 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59a815d250 con 0x7f59a80b0000 2024-09-19T08:41:42.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.678+0000 7f59a57fa640 1 -- 172.21.15.98:0/394695052 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f5990018020 con 0x7f59a80b0000 2024-09-19T08:41:42.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.678+0000 7f59a57fa640 1 --2- 172.21.15.98:0/394695052 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5988067a70 0x7f5988069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:42.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.678+0000 7f59a7fff640 1 --2- 172.21.15.98:0/394695052 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5988067a70 0x7f5988069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:42.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.678+0000 7f59a57fa640 1 -- 172.21.15.98:0/394695052 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4943+0+0 (secure 0 0 0) 0x7f5990085b80 con 0x7f59a80b0000 2024-09-19T08:41:42.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.678+0000 7f59a7fff640 1 --2- 172.21.15.98:0/394695052 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5988067a70 0x7f5988069f30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f59980ad000 tx=0x7f5998008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:42.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.682+0000 7f59a57fa640 1 -- 172.21.15.98:0/394695052 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f599001f020 con 0x7f59a80b0000 2024-09-19T08:41:42.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:42.854+0000 7f59aea1f640 1 -- 172.21.15.98:0/394695052 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f59a8152740 con 0x7f59a80b0000 2024-09-19T08:41:43.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:43.914+0000 7f59a57fa640 1 -- 172.21.15.98:0/394695052 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v53) ==== 97+0+0 (secure 0 0 0) 0x7f5990056740 con 0x7f59a80b0000 2024-09-19T08:41:43.913 INFO:teuthology.orchestra.run.smithi098.stderr: removed all blocklist entries 2024-09-19T08:41:43.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:43.914+0000 7f59aea1f640 1 -- 172.21.15.98:0/394695052 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5988067a70 msgr2=0x7f5988069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:43.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:43.914+0000 7f59aea1f640 1 --2- 172.21.15.98:0/394695052 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5988067a70 0x7f5988069f30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f59980ad000 tx=0x7f5998008040 comp rx=0 tx=0).stop 2024-09-19T08:41:43.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:43.914+0000 7f59aea1f640 1 -- 172.21.15.98:0/394695052 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59a80b0000 msgr2=0x7f59a80b4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:43.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:43.914+0000 7f59aea1f640 1 --2- 172.21.15.98:0/394695052 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59a80b0000 0x7f59a80b4230 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f599000b3e0 tx=0x7f599000b8a0 comp rx=0 tx=0).stop 2024-09-19T08:41:43.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:43.914+0000 7f59aea1f640 1 -- 172.21.15.98:0/394695052 shutdown_connections 2024-09-19T08:41:43.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:43.914+0000 7f59aea1f640 1 --2- 172.21.15.98:0/394695052 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5988067a70 0x7f5988069f30 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:43.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:43.914+0000 7f59aea1f640 1 --2- 172.21.15.98:0/394695052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59a80d0140 0x7f59a80cbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:43.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:43.914+0000 7f59aea1f640 1 --2- 172.21.15.98:0/394695052 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f59a80b0000 0x7f59a80b4230 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:43.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:43.914+0000 7f59aea1f640 1 --2- 172.21.15.98:0/394695052 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f59a80732a0 0x7f59a814b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:43.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:43.914+0000 7f59aea1f640 1 -- 172.21.15.98:0/394695052 >> 172.21.15.98:0/394695052 conn(0x7f59a80d0980 msgr2=0x7f59a80d0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:43.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:43.914+0000 7f59aea1f640 1 -- 172.21.15.98:0/394695052 shutdown_connections 2024-09-19T08:41:43.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:43.914+0000 7f59aea1f640 1 -- 172.21.15.98:0/394695052 wait complete. 2024-09-19T08:41:43.938 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-19T08:41:44.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.062+0000 7f14db185640 1 Processor -- start 2024-09-19T08:41:44.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.062+0000 7f14db185640 1 -- start start 2024-09-19T08:41:44.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.062+0000 7f14db185640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14d415c550 0x7f14d406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:44.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.062+0000 7f14db185640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f14d406c1d0 0x7f14d40cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:44.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.062+0000 7f14db185640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14d406d310 con 0x7f14d40cf900 2024-09-19T08:41:44.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.062+0000 7f14db185640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f14d406c610 con 0x7f14d406c1d0 2024-09-19T08:41:44.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.062+0000 7f14db185640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f14d406d5f0 con 0x7f14d415c550 2024-09-19T08:41:44.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.062+0000 7f14d3fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f14d406c1d0 0x7f14d40cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:44.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.062+0000 7f14d9722640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f14d40cf900 0x7f14d40ce1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:37172/0 (socket says 172.21.15.98:37172) 2024-09-19T08:41:44.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.062+0000 7f14d9722640 1 -- 172.21.15.98:0/2246316560 learned_addr learned my addr 172.21.15.98:0/2246316560 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:41:44.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.062+0000 7f14d8f21640 1 --2- 172.21.15.98:0/2246316560 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14d415c550 0x7f14d406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:44.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.062+0000 7f14d3fff640 1 -- 172.21.15.98:0/2246316560 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14d415c550 msgr2=0x7f14d406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:44.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.062+0000 7f14d3fff640 1 --2- 172.21.15.98:0/2246316560 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14d415c550 0x7f14d406bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:44.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.062+0000 7f14d3fff640 1 -- 172.21.15.98:0/2246316560 >> v1:172.21.15.98:6789/0 conn(0x7f14d40cf900 legacy=0x7f14d40ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:44.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.062+0000 7f14d3fff640 1 -- 172.21.15.98:0/2246316560 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d406d8d0 con 0x7f14d406c1d0 2024-09-19T08:41:44.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.062+0000 7f14d8f21640 1 --2- 172.21.15.98:0/2246316560 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14d415c550 0x7f14d406bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:44.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.062+0000 7f14d37fe640 1 -- 172.21.15.98:0/2246316560 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 782965732 0 0) 0x7f14d406d310 con 0x7f14d40cf900 2024-09-19T08:41:44.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.066+0000 7f14d3fff640 1 --2- 172.21.15.98:0/2246316560 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14d406c1d0 0x7f14d40cf3c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f14c80097c0 tx=0x7f14c801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4904bdeb088f786d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:44.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.066+0000 7f14d37fe640 1 -- 172.21.15.98:0/2246316560 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c80b0070 con 0x7f14d406c1d0 2024-09-19T08:41:44.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.066+0000 7f14d37fe640 1 -- 172.21.15.98:0/2246316560 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14c80ab600 con 0x7f14d406c1d0 2024-09-19T08:41:44.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.066+0000 7f14d37fe640 1 -- 172.21.15.98:0/2246316560 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c80ab9d0 con 0x7f14d406c1d0 2024-09-19T08:41:44.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.066+0000 7f14db185640 1 -- 172.21.15.98:0/2246316560 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14d406c1d0 msgr2=0x7f14d40cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:44.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.066+0000 7f14db185640 1 --2- 172.21.15.98:0/2246316560 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14d406c1d0 0x7f14d40cf3c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f14c80097c0 tx=0x7f14c801c4a0 comp rx=0 tx=0).stop 2024-09-19T08:41:44.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.066+0000 7f14db185640 1 -- 172.21.15.98:0/2246316560 shutdown_connections 2024-09-19T08:41:44.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.066+0000 7f14db185640 1 --2- 172.21.15.98:0/2246316560 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14d406c1d0 0x7f14d40cf3c0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:44.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.066+0000 7f14db185640 1 --2- 172.21.15.98:0/2246316560 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14d415c550 0x7f14d406bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:44.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.066+0000 7f14db185640 1 -- 172.21.15.98:0/2246316560 >> 172.21.15.98:0/2246316560 conn(0x7f14d40b2010 msgr2=0x7f14d40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:44.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.066+0000 7f14db185640 1 -- 172.21.15.98:0/2246316560 shutdown_connections 2024-09-19T08:41:44.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.066+0000 7f14db185640 1 -- 172.21.15.98:0/2246316560 wait complete. 2024-09-19T08:41:44.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.066+0000 7f14db185640 1 Processor -- start 2024-09-19T08:41:44.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.066+0000 7f14db185640 1 -- start start 2024-09-19T08:41:44.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.066+0000 7f14db185640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14d406c1d0 0x7f14d4149a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:44.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.066+0000 7f14db185640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14d40cf900 0x7f14d4151d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:44.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.066+0000 7f14db185640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14d415c550 0x7f14d4158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:44.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.066+0000 7f14db185640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f14d4169230 con 0x7f14d40cf900 2024-09-19T08:41:44.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.066+0000 7f14db185640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f14d415cb40 con 0x7f14d406c1d0 2024-09-19T08:41:44.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.066+0000 7f14db185640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f14d41693b0 con 0x7f14d415c550 2024-09-19T08:41:44.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.066+0000 7f14d3fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14d40cf900 0x7f14d4151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:44.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.066+0000 7f14d3fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14d40cf900 0x7f14d4151d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41068/0 (socket says 172.21.15.98:41068) 2024-09-19T08:41:44.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.066+0000 7f14d3fff640 1 -- 172.21.15.98:0/3729661247 learned_addr learned my addr 172.21.15.98:0/3729661247 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:44.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.066+0000 7f14d3fff640 1 -- 172.21.15.98:0/3729661247 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14d415c550 msgr2=0x7f14d4158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:44.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.066+0000 7f14d3fff640 1 --2- 172.21.15.98:0/3729661247 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14d415c550 0x7f14d4158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:44.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.066+0000 7f14d3fff640 1 -- 172.21.15.98:0/3729661247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14d406c1d0 msgr2=0x7f14d4149a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:44.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.066+0000 7f14d3fff640 1 --2- 172.21.15.98:0/3729661247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14d406c1d0 0x7f14d4149a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:44.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.066+0000 7f14d3fff640 1 -- 172.21.15.98:0/3729661247 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d414e630 con 0x7f14d40cf900 2024-09-19T08:41:44.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.070+0000 7f14d3fff640 1 --2- 172.21.15.98:0/3729661247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14d40cf900 0x7f14d4151d20 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f14c801a0d0 tx=0x7f14c801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:44.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.070+0000 7f14d17fa640 1 -- 172.21.15.98:0/3729661247 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c80b0050 con 0x7f14d40cf900 2024-09-19T08:41:44.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.070+0000 7f14d17fa640 1 -- 172.21.15.98:0/3729661247 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14c80bc070 con 0x7f14d40cf900 2024-09-19T08:41:44.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.070+0000 7f14db185640 1 -- 172.21.15.98:0/3729661247 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14d414e910 con 0x7f14d40cf900 2024-09-19T08:41:44.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.070+0000 7f14d17fa640 1 -- 172.21.15.98:0/3729661247 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c80b5a70 con 0x7f14d40cf900 2024-09-19T08:41:44.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.070+0000 7f14db185640 1 -- 172.21.15.98:0/3729661247 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14d414eed0 con 0x7f14d40cf900 2024-09-19T08:41:44.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.070+0000 7f14db185640 1 -- 172.21.15.98:0/3729661247 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14d414f2b0 con 0x7f14d40cf900 2024-09-19T08:41:44.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.070+0000 7f14d17fa640 1 -- 172.21.15.98:0/3729661247 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f14c80ab480 con 0x7f14d40cf900 2024-09-19T08:41:44.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.070+0000 7f14d17fa640 1 --2- 172.21.15.98:0/3729661247 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f14c0067a70 0x7f14c0069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:44.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.070+0000 7f14d8f21640 1 --2- 172.21.15.98:0/3729661247 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f14c0067a70 0x7f14c0069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:44.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.070+0000 7f14d17fa640 1 -- 172.21.15.98:0/3729661247 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4857+0+0 (secure 0 0 0) 0x7f14c8120760 con 0x7f14d40cf900 2024-09-19T08:41:44.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.070+0000 7f14d8f21640 1 --2- 172.21.15.98:0/3729661247 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f14c0067a70 0x7f14c0069f30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f14c400a890 tx=0x7f14c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:44.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.074+0000 7f14d17fa640 1 -- 172.21.15.98:0/3729661247 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f14c8016090 con 0x7f14d40cf900 2024-09-19T08:41:44.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.290+0000 7f14db185640 1 -- 172.21.15.98:0/3729661247 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f14d40cf3c0 con 0x7f14d40cf900 2024-09-19T08:41:44.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.294+0000 7f14d17fa640 1 -- 172.21.15.98:0/3729661247 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7f14c80f1260 con 0x7f14d40cf900 2024-09-19T08:41:44.293 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:41:44.293 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:41:44.293 INFO:teuthology.orchestra.run.smithi098.stdout: "auth_dump": [ 2024-09-19T08:41:44.293 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:41:44.293 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.a", 2024-09-19T08:41:44.293 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmuujGBRAAa2k/kW68NJI6nUKx+W+G2Q==", 2024-09-19T08:41:44.293 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:41:44.294 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:41:44.294 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:41:44.294 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:41:44.294 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:41:44.294 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:41:44.294 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:41:44.294 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:41:44.294 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.b", 2024-09-19T08:41:44.294 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmFoojFBAAaOZPpa41fdjBPEn3CGsJnA==", 2024-09-19T08:41:44.294 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:41:44.294 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:41:44.294 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:41:44.294 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:41:44.295 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:41:44.295 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:41:44.295 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:41:44.295 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:41:44.295 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.c", 2024-09-19T08:41:44.295 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmcQcDDhAAF0QJMlljMH+DiHfWW+/Sog==", 2024-09-19T08:41:44.295 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:41:44.295 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:41:44.295 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:41:44.295 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:41:44.295 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:41:44.295 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:41:44.295 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:41:44.296 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:41:44.296 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.d", 2024-09-19T08:41:44.296 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmwWmqHBAAf15VcrdeHf8AzQjGsN9LoA==", 2024-09-19T08:41:44.296 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:41:44.296 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:41:44.296 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:41:44.296 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:41:44.296 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:41:44.296 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:41:44.296 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:41:44.296 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:41:44.296 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.0", 2024-09-19T08:41:44.297 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDC4utm9PjNKxAAa6SG9AE4smPiu4azRAFKvg==", 2024-09-19T08:41:44.297 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:41:44.297 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:41:44.297 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:41:44.297 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:41:44.297 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:41:44.297 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:41:44.297 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:41:44.297 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.1", 2024-09-19T08:41:44.297 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDG4utmHy6UCxAAN6hpG4DJEp81iG/KoloTvQ==", 2024-09-19T08:41:44.297 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:41:44.297 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:41:44.297 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:41:44.298 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:41:44.298 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:41:44.298 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:41:44.298 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:41:44.298 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.2", 2024-09-19T08:41:44.298 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDJ4utmUVP4GhAAJNAL30sEAseoDG2fE6YbZQ==", 2024-09-19T08:41:44.298 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:41:44.298 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:41:44.298 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:41:44.298 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:41:44.298 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:41:44.298 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:41:44.298 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:41:44.298 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.3", 2024-09-19T08:41:44.299 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDM4utmTKgoLRAAiCrPRLML+wvjOt6RhKay3w==", 2024-09-19T08:41:44.299 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:41:44.299 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:41:44.299 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:41:44.299 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:41:44.299 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:41:44.299 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:41:44.299 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:41:44.299 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.4", 2024-09-19T08:41:44.299 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDS4utmAf6bMhAAI3uENuFoXLicUeuspgFjYA==", 2024-09-19T08:41:44.299 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:41:44.299 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:41:44.299 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:41:44.300 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:41:44.300 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:41:44.300 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:41:44.300 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:41:44.300 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.5", 2024-09-19T08:41:44.300 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDW4utmRjxwEhAAF1A2GtrZXVx9BlmjHjhUDQ==", 2024-09-19T08:41:44.300 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:41:44.300 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:41:44.300 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:41:44.300 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:41:44.300 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:41:44.300 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:41:44.300 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:41:44.301 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.6", 2024-09-19T08:41:44.301 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDZ4utmSk/XIRAAROSfzGL7smR88ZkDSx5jAg==", 2024-09-19T08:41:44.301 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:41:44.301 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:41:44.301 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:41:44.301 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:41:44.301 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:41:44.301 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:41:44.301 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:41:44.301 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.7", 2024-09-19T08:41:44.301 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDc4utmXfpbMxAAqEM4I/ACFvfu/sMA8bMcrQ==", 2024-09-19T08:41:44.301 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:41:44.302 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:41:44.302 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:41:44.302 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:41:44.302 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:41:44.302 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:41:44.302 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:41:44.302 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.0", 2024-09-19T08:41:44.302 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utm0vNpIhAA4S1E8dFZ0+ibmSKhmZ+bIA==", 2024-09-19T08:41:44.302 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:41:44.302 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:41:44.302 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow r", 2024-09-19T08:41:44.302 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-19T08:41:44.302 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:41:44.302 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:41:44.303 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:41:44.303 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.1", 2024-09-19T08:41:44.303 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmx7xyKhAAC84lykPEb01MiypXKQVREQ==", 2024-09-19T08:41:44.303 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:41:44.303 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:41:44.303 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-19T08:41:44.303 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-19T08:41:44.303 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-19T08:41:44.303 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:41:44.303 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:41:44.303 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:41:44.303 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.2", 2024-09-19T08:41:44.303 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmteOQJxAAtTF2Py21ZE2WRQjeQSrkKQ==", 2024-09-19T08:41:44.304 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:41:44.304 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:41:44.304 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-19T08:41:44.304 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-19T08:41:44.304 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-19T08:41:44.304 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:41:44.304 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:41:44.304 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:41:44.304 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.3", 2024-09-19T08:41:44.304 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmKN92LxAAMqraZOWRT6IIfx5AuDWS2g==", 2024-09-19T08:41:44.304 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:41:44.304 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:41:44.304 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-19T08:41:44.304 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-19T08:41:44.305 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-19T08:41:44.305 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:41:44.305 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:41:44.305 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:41:44.305 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.admin", 2024-09-19T08:41:44.305 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC+4utmLH97KhAA1DTYW5bTrGUG+7cb8+d7gA==", 2024-09-19T08:41:44.305 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:41:44.305 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-09-19T08:41:44.305 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:41:44.305 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:41:44.305 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:41:44.306 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:41:44.306 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:41:44.306 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:41:44.306 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mgr.x", 2024-09-19T08:41:44.306 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmbKuyAhAA0/XjHUIPOIV/YvNEhB8rEQ==", 2024-09-19T08:41:44.306 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:41:44.306 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-09-19T08:41:44.306 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile mgr", 2024-09-19T08:41:44.306 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:41:44.306 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:41:44.306 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:41:44.306 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:41:44.306 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mgr.y", 2024-09-19T08:41:44.306 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC+4utmU+ZrOxAAsvcwINH6qyxfN7N+rDrkqg==", 2024-09-19T08:41:44.306 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:41:44.307 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-09-19T08:41:44.307 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile mgr", 2024-09-19T08:41:44.307 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:41:44.307 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:41:44.307 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:41:44.307 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-19T08:41:44.307 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:41:44.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.294+0000 7f14db185640 1 -- 172.21.15.98:0/3729661247 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f14c0067a70 msgr2=0x7f14c0069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:44.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.294+0000 7f14db185640 1 --2- 172.21.15.98:0/3729661247 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f14c0067a70 0x7f14c0069f30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f14c400a890 tx=0x7f14c4008040 comp rx=0 tx=0).stop 2024-09-19T08:41:44.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.294+0000 7f14db185640 1 -- 172.21.15.98:0/3729661247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14d40cf900 msgr2=0x7f14d4151d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:44.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.294+0000 7f14db185640 1 --2- 172.21.15.98:0/3729661247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14d40cf900 0x7f14d4151d20 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f14c801a0d0 tx=0x7f14c801ace0 comp rx=0 tx=0).stop 2024-09-19T08:41:44.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.294+0000 7f14db185640 1 -- 172.21.15.98:0/3729661247 shutdown_connections 2024-09-19T08:41:44.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.294+0000 7f14db185640 1 --2- 172.21.15.98:0/3729661247 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f14c0067a70 0x7f14c0069f30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:44.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.294+0000 7f14db185640 1 --2- 172.21.15.98:0/3729661247 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14d415c550 0x7f14d4158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:44.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.294+0000 7f14db185640 1 --2- 172.21.15.98:0/3729661247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14d40cf900 0x7f14d4151d20 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:44.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.294+0000 7f14db185640 1 --2- 172.21.15.98:0/3729661247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14d406c1d0 0x7f14d4149a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:44.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.294+0000 7f14db185640 1 -- 172.21.15.98:0/3729661247 >> 172.21.15.98:0/3729661247 conn(0x7f14d40b2010 msgr2=0x7f14d4159f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:44.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.294+0000 7f14db185640 1 -- 172.21.15.98:0/3729661247 shutdown_connections 2024-09-19T08:41:44.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.294+0000 7f14db185640 1 -- 172.21.15.98:0/3729661247 wait complete. 2024-09-19T08:41:44.310 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-19T08:41:44.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.430+0000 7f26967cf640 1 Processor -- start 2024-09-19T08:41:44.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.430+0000 7f26967cf640 1 -- start start 2024-09-19T08:41:44.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.434+0000 7f26967cf640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f269015e750 0x7f2690073c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:44.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.434+0000 7f26967cf640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2690075380 0x7f26900cf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:44.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.434+0000 7f26967cf640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2690075b80 con 0x7f26900b28e0 2024-09-19T08:41:44.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.434+0000 7f26967cf640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f26900d1bd0 con 0x7f269015e750 2024-09-19T08:41:44.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.434+0000 7f26967cf640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f26900d1eb0 con 0x7f2690075380 2024-09-19T08:41:44.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.434+0000 7f268ffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f269015e750 0x7f2690073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:44.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.434+0000 7f268f7fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f26900b28e0 0x7f2690074c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:37180/0 (socket says 172.21.15.98:37180) 2024-09-19T08:41:44.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.434+0000 7f268ffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f269015e750 0x7f2690073c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41070/0 (socket says 172.21.15.98:41070) 2024-09-19T08:41:44.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.434+0000 7f268f7fe640 1 -- 172.21.15.98:0/373430988 learned_addr learned my addr 172.21.15.98:0/373430988 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:41:44.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.434+0000 7f2694d6c640 1 --2- 172.21.15.98:0/373430988 >> v2:172.21.15.139:3300/0 conn(0x7f2690075380 0x7f26900cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:44.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.434+0000 7f268ffff640 1 -- 172.21.15.98:0/373430988 >> v2:172.21.15.139:3300/0 conn(0x7f2690075380 msgr2=0x7f26900cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:44.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.434+0000 7f268ffff640 1 --2- 172.21.15.98:0/373430988 >> v2:172.21.15.139:3300/0 conn(0x7f2690075380 0x7f26900cf520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:44.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.434+0000 7f268ffff640 1 -- 172.21.15.98:0/373430988 >> v1:172.21.15.98:6789/0 conn(0x7f26900b28e0 legacy=0x7f2690074c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:44.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.434+0000 7f268ffff640 1 -- 172.21.15.98:0/373430988 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26900cf000 con 0x7f269015e750 2024-09-19T08:41:44.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.434+0000 7f2694d6c640 1 --2- 172.21.15.98:0/373430988 >> v2:172.21.15.139:3300/0 conn(0x7f2690075380 0x7f26900cf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:44.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.434+0000 7f268ffff640 1 --2- 172.21.15.98:0/373430988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f269015e750 0x7f2690073c90 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f26900cb870 tx=0x7f268001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab1defbe485843a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:44.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.434+0000 7f268effd640 1 -- 172.21.15.98:0/373430988 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26800ab610 con 0x7f269015e750 2024-09-19T08:41:44.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.434+0000 7f268effd640 1 -- 172.21.15.98:0/373430988 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26800b0070 con 0x7f269015e750 2024-09-19T08:41:44.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.434+0000 7f268effd640 1 -- 172.21.15.98:0/373430988 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26800ab9d0 con 0x7f269015e750 2024-09-19T08:41:44.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.438+0000 7f26967cf640 1 -- 172.21.15.98:0/373430988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f269015e750 msgr2=0x7f2690073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:44.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.438+0000 7f26967cf640 1 --2- 172.21.15.98:0/373430988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f269015e750 0x7f2690073c90 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f26900cb870 tx=0x7f268001c3f0 comp rx=0 tx=0).stop 2024-09-19T08:41:44.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.438+0000 7f26967cf640 1 -- 172.21.15.98:0/373430988 shutdown_connections 2024-09-19T08:41:44.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.438+0000 7f26967cf640 1 --2- 172.21.15.98:0/373430988 >> v2:172.21.15.139:3300/0 conn(0x7f2690075380 0x7f26900cf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:44.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.438+0000 7f26967cf640 1 --2- 172.21.15.98:0/373430988 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f269015e750 0x7f2690073c90 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:44.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.438+0000 7f26967cf640 1 -- 172.21.15.98:0/373430988 >> 172.21.15.98:0/373430988 conn(0x7f2690073170 msgr2=0x7f2690073580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:44.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.438+0000 7f26967cf640 1 -- 172.21.15.98:0/373430988 shutdown_connections 2024-09-19T08:41:44.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.438+0000 7f26967cf640 1 -- 172.21.15.98:0/373430988 wait complete. 2024-09-19T08:41:44.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.438+0000 7f26967cf640 1 Processor -- start 2024-09-19T08:41:44.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.438+0000 7f26967cf640 1 -- start start 2024-09-19T08:41:44.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.438+0000 7f26967cf640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2690075380 0x7f2690155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:44.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.438+0000 7f26967cf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26900b28e0 0x7f2690099e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:44.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.438+0000 7f26967cf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f269015e750 0x7f269015b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:44.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.438+0000 7f26967cf640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f26900d0600 con 0x7f269015e750 2024-09-19T08:41:44.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.438+0000 7f26967cf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f26900d0480 con 0x7f26900b28e0 2024-09-19T08:41:44.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.438+0000 7f26967cf640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f26900d0780 con 0x7f2690075380 2024-09-19T08:41:44.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.438+0000 7f2694d6c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f269015e750 0x7f269015b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:44.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.438+0000 7f2694d6c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f269015e750 0x7f269015b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41076/0 (socket says 172.21.15.98:41076) 2024-09-19T08:41:44.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.438+0000 7f2694d6c640 1 -- 172.21.15.98:0/964856610 learned_addr learned my addr 172.21.15.98:0/964856610 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:44.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.438+0000 7f268ffff640 1 --2- 172.21.15.98:0/964856610 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2690075380 0x7f2690155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:44.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.438+0000 7f268f7fe640 1 --2- 172.21.15.98:0/964856610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26900b28e0 0x7f2690099e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:44.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.438+0000 7f2694d6c640 1 -- 172.21.15.98:0/964856610 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2690075380 msgr2=0x7f2690155690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:44.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.438+0000 7f2694d6c640 1 --2- 172.21.15.98:0/964856610 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2690075380 0x7f2690155690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:44.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.438+0000 7f2694d6c640 1 -- 172.21.15.98:0/964856610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26900b28e0 msgr2=0x7f2690099e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:44.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.438+0000 7f2694d6c640 1 --2- 172.21.15.98:0/964856610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26900b28e0 0x7f2690099e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:44.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.438+0000 7f2694d6c640 1 -- 172.21.15.98:0/964856610 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f269009a3a0 con 0x7f269015e750 2024-09-19T08:41:44.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.438+0000 7f268ffff640 1 --2- 172.21.15.98:0/964856610 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2690075380 0x7f2690155690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:44.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.438+0000 7f268f7fe640 1 --2- 172.21.15.98:0/964856610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26900b28e0 0x7f2690099e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:44.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.438+0000 7f2694d6c640 1 --2- 172.21.15.98:0/964856610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f269015e750 0x7f269015b910 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f268800cef0 tx=0x7f268800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:44.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.438+0000 7f268cff9640 1 -- 172.21.15.98:0/964856610 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268801a070 con 0x7f269015e750 2024-09-19T08:41:44.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.438+0000 7f268cff9640 1 -- 172.21.15.98:0/964856610 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f268800d860 con 0x7f269015e750 2024-09-19T08:41:44.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.438+0000 7f268cff9640 1 -- 172.21.15.98:0/964856610 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268800dc50 con 0x7f269015e750 2024-09-19T08:41:44.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.438+0000 7f26967cf640 1 -- 172.21.15.98:0/964856610 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f269009a640 con 0x7f269015e750 2024-09-19T08:41:44.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.438+0000 7f26967cf640 1 -- 172.21.15.98:0/964856610 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f269015b110 con 0x7f269015e750 2024-09-19T08:41:44.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.442+0000 7f26967cf640 1 -- 172.21.15.98:0/964856610 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26900cf520 con 0x7f269015e750 2024-09-19T08:41:44.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.442+0000 7f268cff9640 1 -- 172.21.15.98:0/964856610 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f268801d070 con 0x7f269015e750 2024-09-19T08:41:44.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.442+0000 7f268cff9640 1 --2- 172.21.15.98:0/964856610 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f267c067a70 0x7f267c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:44.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.442+0000 7f268ffff640 1 --2- 172.21.15.98:0/964856610 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f267c067a70 0x7f267c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:44.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.442+0000 7f268cff9640 1 -- 172.21.15.98:0/964856610 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4857+0+0 (secure 0 0 0) 0x7f2688086420 con 0x7f269015e750 2024-09-19T08:41:44.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.442+0000 7f268ffff640 1 --2- 172.21.15.98:0/964856610 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f267c067a70 0x7f267c069f30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f26800af000 tx=0x7f2680008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:44.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.446+0000 7f268cff9640 1 -- 172.21.15.98:0/964856610 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2688057180 con 0x7f269015e750 2024-09-19T08:41:44.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.610+0000 7f26967cf640 1 -- 172.21.15.98:0/964856610 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f269015da60 con 0x7f269015e750 2024-09-19T08:41:44.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.610+0000 7f268cff9640 1 -- 172.21.15.98:0/964856610 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v53) ==== 84+0+65 (secure 0 0 0) 0x7f2688056a80 con 0x7f269015e750 2024-09-19T08:41:44.612 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:41:44.612 INFO:teuthology.orchestra.run.smithi098.stdout:[ 2024-09-19T08:41:44.612 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:41:44.612 INFO:teuthology.orchestra.run.smithi098.stdout: "poolnum": 1, 2024-09-19T08:41:44.613 INFO:teuthology.orchestra.run.smithi098.stdout: "poolname": ".mgr" 2024-09-19T08:41:44.613 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:41:44.613 INFO:teuthology.orchestra.run.smithi098.stdout:] 2024-09-19T08:41:44.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.614+0000 7f26967cf640 1 -- 172.21.15.98:0/964856610 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f267c067a70 msgr2=0x7f267c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:44.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.614+0000 7f26967cf640 1 --2- 172.21.15.98:0/964856610 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f267c067a70 0x7f267c069f30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f26800af000 tx=0x7f2680008040 comp rx=0 tx=0).stop 2024-09-19T08:41:44.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.614+0000 7f26967cf640 1 -- 172.21.15.98:0/964856610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f269015e750 msgr2=0x7f269015b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:44.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.614+0000 7f26967cf640 1 --2- 172.21.15.98:0/964856610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f269015e750 0x7f269015b910 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f268800cef0 tx=0x7f268800a710 comp rx=0 tx=0).stop 2024-09-19T08:41:44.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.614+0000 7f26967cf640 1 -- 172.21.15.98:0/964856610 shutdown_connections 2024-09-19T08:41:44.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.614+0000 7f26967cf640 1 --2- 172.21.15.98:0/964856610 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f267c067a70 0x7f267c069f30 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:44.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.614+0000 7f26967cf640 1 --2- 172.21.15.98:0/964856610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f269015e750 0x7f269015b910 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:44.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.614+0000 7f26967cf640 1 --2- 172.21.15.98:0/964856610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f26900b28e0 0x7f2690099e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:44.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.614+0000 7f26967cf640 1 --2- 172.21.15.98:0/964856610 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2690075380 0x7f2690155690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:44.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.614+0000 7f26967cf640 1 -- 172.21.15.98:0/964856610 >> 172.21.15.98:0/964856610 conn(0x7f2690073170 msgr2=0x7f269015c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:44.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.614+0000 7f26967cf640 1 -- 172.21.15.98:0/964856610 shutdown_connections 2024-09-19T08:41:44.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.614+0000 7f26967cf640 1 -- 172.21.15.98:0/964856610 wait complete. 2024-09-19T08:41:44.631 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T08:41:44.632 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T08:41:44.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.758+0000 7f0ac824f640 1 Processor -- start 2024-09-19T08:41:44.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.758+0000 7f0ac824f640 1 -- start start 2024-09-19T08:41:44.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.758+0000 7f0ac824f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ac006dbd0 0x7f0ac0169280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:44.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.758+0000 7f0ac824f640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ac015fac0 con 0x7f0ac0144200 2024-09-19T08:41:44.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.758+0000 7f0ac824f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ac01622a0 con 0x7f0ac0071030 2024-09-19T08:41:44.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.758+0000 7f0ac824f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0ac0141940 con 0x7f0ac006dbd0 2024-09-19T08:41:44.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.758+0000 7f0ac5feb640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f0ac0144200 0x7f0ac01445e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:37192/0 (socket says 172.21.15.98:37192) 2024-09-19T08:41:44.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.758+0000 7f0ac5feb640 1 -- 172.21.15.98:0/2350440011 learned_addr learned my addr 172.21.15.98:0/2350440011 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:41:44.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.758+0000 7f0ac67ec640 1 --2- 172.21.15.98:0/2350440011 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ac006dbd0 0x7f0ac0169280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:44.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.758+0000 7f0ac4fe9640 1 -- 172.21.15.98:0/2350440011 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2479335290 0 0) 0x7f0ac015fac0 con 0x7f0ac0144200 2024-09-19T08:41:44.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.758+0000 7f0ac4fe9640 1 -- 172.21.15.98:0/2350440011 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0aa8003530 con 0x7f0ac0144200 2024-09-19T08:41:44.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.758+0000 7f0ac4fe9640 1 -- 172.21.15.98:0/2350440011 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2345357997 0 0) 0x7f0aa8003530 con 0x7f0ac0144200 2024-09-19T08:41:44.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.758+0000 7f0ac4fe9640 1 -- 172.21.15.98:0/2350440011 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ac006dbd0 msgr2=0x7f0ac0169280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:44.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.758+0000 7f0ac4fe9640 1 --2- 172.21.15.98:0/2350440011 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ac006dbd0 0x7f0ac0169280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:44.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.758+0000 7f0ac4fe9640 1 -- 172.21.15.98:0/2350440011 >> v1:172.21.15.139:6789/0 conn(0x7f0ac0071030 legacy=0x7f0ac006d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:44.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.758+0000 7f0ac4fe9640 1 -- 172.21.15.98:0/2350440011 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac0162580 con 0x7f0ac0144200 2024-09-19T08:41:44.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.758+0000 7f0ac4fe9640 1 -- 172.21.15.98:0/2350440011 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f0ab8001680 con 0x7f0ac0144200 2024-09-19T08:41:44.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.758+0000 7f0ac4fe9640 1 -- 172.21.15.98:0/2350440011 >> v1:172.21.15.98:6789/0 conn(0x7f0ac0144200 legacy=0x7f0ac01445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:44.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.758+0000 7f0ac4fe9640 1 --2- 172.21.15.98:0/2350440011 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ac006dbd0 0x7f0aa80039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:44.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.758+0000 7f0ac4fe9640 1 --2- 172.21.15.98:0/2350440011 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0aa8003f70 0x7f0aa8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:44.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.758+0000 7f0ac4fe9640 1 --2- 172.21.15.98:0/2350440011 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0aa8004d50 0x7f0aa80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:44.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.758+0000 7f0ac4fe9640 1 -- 172.21.15.98:0/2350440011 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac0141940 con 0x7f0aa8004d50 2024-09-19T08:41:44.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.758+0000 7f0ac4fe9640 1 -- 172.21.15.98:0/2350440011 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa800bab0 con 0x7f0aa8003f70 2024-09-19T08:41:44.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.758+0000 7f0ac4fe9640 1 -- 172.21.15.98:0/2350440011 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0aa800bcb0 con 0x7f0ac006dbd0 2024-09-19T08:41:44.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.758+0000 7f0ac67ec640 1 --2- 172.21.15.98:0/2350440011 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0aa8004d50 0x7f0aa80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:44.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.758+0000 7f0ac5feb640 1 --2- 172.21.15.98:0/2350440011 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ac006dbd0 0x7f0aa80039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:44.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.758+0000 7f0ac57ea640 1 --2- 172.21.15.98:0/2350440011 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0aa8003f70 0x7f0aa8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:44.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.758+0000 7f0ac67ec640 1 -- 172.21.15.98:0/2350440011 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ac006dbd0 msgr2=0x7f0aa80039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:44.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.758+0000 7f0ac67ec640 1 --2- 172.21.15.98:0/2350440011 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ac006dbd0 0x7f0aa80039f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:44.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.758+0000 7f0ac67ec640 1 -- 172.21.15.98:0/2350440011 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0aa8003f70 msgr2=0x7f0aa8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:44.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.758+0000 7f0ac67ec640 1 --2- 172.21.15.98:0/2350440011 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0aa8003f70 0x7f0aa8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:44.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.758+0000 7f0ac67ec640 1 -- 172.21.15.98:0/2350440011 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa8003530 con 0x7f0aa8004d50 2024-09-19T08:41:44.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.758+0000 7f0ac57ea640 1 --2- 172.21.15.98:0/2350440011 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0aa8003f70 0x7f0aa8004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:44.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.762+0000 7f0ac67ec640 1 --2- 172.21.15.98:0/2350440011 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0aa8004d50 0x7f0aa80051d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0009ae0 tx=0x7f0ab001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86cc75dc176bfb68 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:44.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.762+0000 7f0ac4fe9640 1 -- 172.21.15.98:0/2350440011 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab00b2070 con 0x7f0aa8004d50 2024-09-19T08:41:44.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.762+0000 7f0ac4fe9640 1 -- 172.21.15.98:0/2350440011 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ab00a5660 con 0x7f0aa8004d50 2024-09-19T08:41:44.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.762+0000 7f0ac4fe9640 1 -- 172.21.15.98:0/2350440011 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab00096b0 con 0x7f0aa8004d50 2024-09-19T08:41:44.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.762+0000 7f0ac824f640 1 -- 172.21.15.98:0/2350440011 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0aa8004d50 msgr2=0x7f0aa80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:44.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.762+0000 7f0ac824f640 1 --2- 172.21.15.98:0/2350440011 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0aa8004d50 0x7f0aa80051d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0009ae0 tx=0x7f0ab001ca70 comp rx=0 tx=0).stop 2024-09-19T08:41:44.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.762+0000 7f0ac67ec640 1 -- 172.21.15.98:0/2350440011 reap_dead start 2024-09-19T08:41:44.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.762+0000 7f0ac824f640 1 -- 172.21.15.98:0/2350440011 shutdown_connections 2024-09-19T08:41:44.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.762+0000 7f0ac824f640 1 -- 172.21.15.98:0/2350440011 >> 172.21.15.98:0/2350440011 conn(0x7f0ac006c090 msgr2=0x7f0ac006c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:44.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.762+0000 7f0ac824f640 1 -- 172.21.15.98:0/2350440011 shutdown_connections 2024-09-19T08:41:44.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.762+0000 7f0ac824f640 1 -- 172.21.15.98:0/2350440011 wait complete. 2024-09-19T08:41:44.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.762+0000 7f0ac824f640 1 Processor -- start 2024-09-19T08:41:44.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.762+0000 7f0ac824f640 1 -- start start 2024-09-19T08:41:44.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.766+0000 7f0ac824f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ac013a3a0 0x7f0ac013abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:44.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.766+0000 7f0ac824f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ac013a780 0x7f0ac013b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:44.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.766+0000 7f0ac824f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ac013d0a0 0x7f0ac013e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:44.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.766+0000 7f0ac824f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac00718f0 con 0x7f0ac013d0a0 2024-09-19T08:41:44.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.766+0000 7f0ac824f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac0163060 con 0x7f0ac013a3a0 2024-09-19T08:41:44.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.766+0000 7f0ac824f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0ac0162b40 con 0x7f0ac013a780 2024-09-19T08:41:44.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.766+0000 7f0ac67ec640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ac013d0a0 0x7f0ac013e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:44.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.766+0000 7f0ac67ec640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ac013d0a0 0x7f0ac013e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41098/0 (socket says 172.21.15.98:41098) 2024-09-19T08:41:44.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.766+0000 7f0ac67ec640 1 -- 172.21.15.98:0/2792892251 learned_addr learned my addr 172.21.15.98:0/2792892251 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:44.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.766+0000 7f0ac5feb640 1 --2- 172.21.15.98:0/2792892251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ac013a3a0 0x7f0ac013abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:44.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.766+0000 7f0ac57ea640 1 --2- 172.21.15.98:0/2792892251 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ac013a780 0x7f0ac013b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:44.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.766+0000 7f0ac67ec640 1 -- 172.21.15.98:0/2792892251 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ac013a780 msgr2=0x7f0ac013b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:44.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.766+0000 7f0ac67ec640 1 --2- 172.21.15.98:0/2792892251 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ac013a780 0x7f0ac013b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:44.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.766+0000 7f0ac67ec640 1 -- 172.21.15.98:0/2792892251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ac013a3a0 msgr2=0x7f0ac013abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:44.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.766+0000 7f0ac67ec640 1 --2- 172.21.15.98:0/2792892251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ac013a3a0 0x7f0ac013abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:44.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.766+0000 7f0ac67ec640 1 -- 172.21.15.98:0/2792892251 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac01439d0 con 0x7f0ac013d0a0 2024-09-19T08:41:44.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.766+0000 7f0ac67ec640 1 --2- 172.21.15.98:0/2792892251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ac013d0a0 0x7f0ac013e290 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0ab00b0040 tx=0x7f0ab0019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:44.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.766+0000 7f0ab67fc640 1 -- 172.21.15.98:0/2792892251 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab00b2040 con 0x7f0ac013d0a0 2024-09-19T08:41:44.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.766+0000 7f0ab67fc640 1 -- 172.21.15.98:0/2792892251 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ab0004020 con 0x7f0ac013d0a0 2024-09-19T08:41:44.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.766+0000 7f0ac824f640 1 -- 172.21.15.98:0/2792892251 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ac013c000 con 0x7f0ac013d0a0 2024-09-19T08:41:44.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.766+0000 7f0ab67fc640 1 -- 172.21.15.98:0/2792892251 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab00096b0 con 0x7f0ac013d0a0 2024-09-19T08:41:44.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.766+0000 7f0ac824f640 1 -- 172.21.15.98:0/2792892251 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ac013c5c0 con 0x7f0ac013d0a0 2024-09-19T08:41:44.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.766+0000 7f0ac824f640 1 -- 172.21.15.98:0/2792892251 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ac00714a0 con 0x7f0ac013d0a0 2024-09-19T08:41:44.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.766+0000 7f0ab67fc640 1 -- 172.21.15.98:0/2792892251 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f0ab00bf060 con 0x7f0ac013d0a0 2024-09-19T08:41:44.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.766+0000 7f0ab67fc640 1 --2- 172.21.15.98:0/2792892251 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0aac067aa0 0x7f0aac069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:44.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.766+0000 7f0ab67fc640 1 -- 172.21.15.98:0/2792892251 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4857+0+0 (secure 0 0 0) 0x7f0ab011f430 con 0x7f0ac013d0a0 2024-09-19T08:41:44.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.766+0000 7f0ac5feb640 1 --2- 172.21.15.98:0/2792892251 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0aac067aa0 0x7f0aac069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:44.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.766+0000 7f0ac5feb640 1 --2- 172.21.15.98:0/2792892251 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0aac067aa0 0x7f0aac069f60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8001ec0 tx=0x7f0ab8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:44.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.770+0000 7f0ab67fc640 1 -- 172.21.15.98:0/2792892251 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f0ab0016780 con 0x7f0ac013d0a0 2024-09-19T08:41:44.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:44.922+0000 7f0ac824f640 1 -- 172.21.15.98:0/2792892251 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f0ac00b6250 con 0x7f0ac013d0a0 2024-09-19T08:41:45.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:45.918+0000 7f0ab67fc640 1 -- 172.21.15.98:0/2792892251 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v54) ==== 138+0+0 (secure 0 0 0) 0x7f0ab00f0040 con 0x7f0ac013d0a0 2024-09-19T08:41:45.917 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_metadata' created 2024-09-19T08:41:45.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:45.922+0000 7f0ac824f640 1 -- 172.21.15.98:0/2792892251 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0aac067aa0 msgr2=0x7f0aac069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:45.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:45.922+0000 7f0ac824f640 1 --2- 172.21.15.98:0/2792892251 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0aac067aa0 0x7f0aac069f60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8001ec0 tx=0x7f0ab8009040 comp rx=0 tx=0).stop 2024-09-19T08:41:45.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:45.922+0000 7f0ac824f640 1 -- 172.21.15.98:0/2792892251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ac013d0a0 msgr2=0x7f0ac013e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:45.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:45.922+0000 7f0ac824f640 1 --2- 172.21.15.98:0/2792892251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ac013d0a0 0x7f0ac013e290 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0ab00b0040 tx=0x7f0ab0019810 comp rx=0 tx=0).stop 2024-09-19T08:41:45.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:45.922+0000 7f0ac824f640 1 -- 172.21.15.98:0/2792892251 shutdown_connections 2024-09-19T08:41:45.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:45.922+0000 7f0ac824f640 1 --2- 172.21.15.98:0/2792892251 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0aac067aa0 0x7f0aac069f60 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:45.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:45.922+0000 7f0ac824f640 1 --2- 172.21.15.98:0/2792892251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ac013d0a0 0x7f0ac013e290 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:45.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:45.922+0000 7f0ac824f640 1 --2- 172.21.15.98:0/2792892251 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ac013a780 0x7f0ac013b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:45.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:45.922+0000 7f0ac824f640 1 --2- 172.21.15.98:0/2792892251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ac013a3a0 0x7f0ac013abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:45.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:45.922+0000 7f0ac824f640 1 -- 172.21.15.98:0/2792892251 >> 172.21.15.98:0/2792892251 conn(0x7f0ac006c090 msgr2=0x7f0ac0143090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:45.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:45.922+0000 7f0ac824f640 1 -- 172.21.15.98:0/2792892251 shutdown_connections 2024-09-19T08:41:45.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:45.922+0000 7f0ac824f640 1 -- 172.21.15.98:0/2792892251 wait complete. 2024-09-19T08:41:45.948 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T08:41:46.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.074+0000 7f874463a640 1 Processor -- start 2024-09-19T08:41:46.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.074+0000 7f874463a640 1 -- start start 2024-09-19T08:41:46.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.074+0000 7f874463a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f873c145b70 0x7f873c06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:46.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.074+0000 7f874463a640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f873c06c5f0 con 0x7f873c06c1d0 2024-09-19T08:41:46.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.074+0000 7f874463a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f873c073530 con 0x7f873c071ac0 2024-09-19T08:41:46.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.074+0000 7f874463a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f873c071f20 con 0x7f873c145b70 2024-09-19T08:41:46.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.074+0000 7f8741bd5640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f873c06c1d0 0x7f873c0713b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:37206/0 (socket says 172.21.15.98:37206) 2024-09-19T08:41:46.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.074+0000 7f8741bd5640 1 -- 172.21.15.98:0/4073890727 learned_addr learned my addr 172.21.15.98:0/4073890727 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:41:46.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.078+0000 7f87423d6640 1 --2- 172.21.15.98:0/4073890727 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f873c145b70 0x7f873c06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:46.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.078+0000 7f87413d4640 1 -- 172.21.15.98:0/4073890727 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1915696021 0 0) 0x7f873c06c5f0 con 0x7f873c06c1d0 2024-09-19T08:41:46.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.078+0000 7f87413d4640 1 -- 172.21.15.98:0/4073890727 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8728003530 con 0x7f873c06c1d0 2024-09-19T08:41:46.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.078+0000 7f87413d4640 1 -- 172.21.15.98:0/4073890727 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3482708880 0 0) 0x7f8728003530 con 0x7f873c06c1d0 2024-09-19T08:41:46.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.078+0000 7f87413d4640 1 -- 172.21.15.98:0/4073890727 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f873c145b70 msgr2=0x7f873c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:46.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.078+0000 7f87413d4640 1 --2- 172.21.15.98:0/4073890727 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f873c145b70 0x7f873c06bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:46.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.078+0000 7f87413d4640 1 -- 172.21.15.98:0/4073890727 >> v1:172.21.15.139:6789/0 conn(0x7f873c071ac0 legacy=0x7f873c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:46.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.078+0000 7f87413d4640 1 -- 172.21.15.98:0/4073890727 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f873c073810 con 0x7f873c06c1d0 2024-09-19T08:41:46.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.078+0000 7f87413d4640 1 -- 172.21.15.98:0/4073890727 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f872c001680 con 0x7f873c06c1d0 2024-09-19T08:41:46.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.078+0000 7f87413d4640 1 -- 172.21.15.98:0/4073890727 >> v1:172.21.15.98:6789/0 conn(0x7f873c06c1d0 legacy=0x7f873c0713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:46.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.078+0000 7f87413d4640 1 --2- 172.21.15.98:0/4073890727 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f873c145b70 0x7f87280039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:46.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.078+0000 7f87413d4640 1 --2- 172.21.15.98:0/4073890727 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8728003f70 0x7f8728004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:46.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.078+0000 7f87413d4640 1 --2- 172.21.15.98:0/4073890727 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8728004d50 0x7f87280051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:46.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.078+0000 7f87413d4640 1 -- 172.21.15.98:0/4073890727 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f873c071f20 con 0x7f8728004d50 2024-09-19T08:41:46.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.078+0000 7f87413d4640 1 -- 172.21.15.98:0/4073890727 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f872800bb60 con 0x7f8728003f70 2024-09-19T08:41:46.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.078+0000 7f87413d4640 1 -- 172.21.15.98:0/4073890727 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f872800bd60 con 0x7f873c145b70 2024-09-19T08:41:46.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.078+0000 7f8742bd7640 1 --2- 172.21.15.98:0/4073890727 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8728004d50 0x7f87280051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:46.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.078+0000 7f8741bd5640 1 --2- 172.21.15.98:0/4073890727 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8728003f70 0x7f8728004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:46.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.078+0000 7f87423d6640 1 --2- 172.21.15.98:0/4073890727 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f873c145b70 0x7f87280039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:46.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.078+0000 7f8742bd7640 1 -- 172.21.15.98:0/4073890727 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f873c145b70 msgr2=0x7f87280039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:46.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.078+0000 7f8742bd7640 1 --2- 172.21.15.98:0/4073890727 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f873c145b70 0x7f87280039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:46.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.078+0000 7f8742bd7640 1 -- 172.21.15.98:0/4073890727 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8728003f70 msgr2=0x7f8728004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:46.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.078+0000 7f8742bd7640 1 --2- 172.21.15.98:0/4073890727 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8728003f70 0x7f8728004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:46.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.078+0000 7f8742bd7640 1 -- 172.21.15.98:0/4073890727 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8728003530 con 0x7f8728004d50 2024-09-19T08:41:46.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.078+0000 7f8742bd7640 1 --2- 172.21.15.98:0/4073890727 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8728004d50 0x7f87280051d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f8734001e70 tx=0x7f873401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd90d9266b014edc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:46.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.078+0000 7f87413d4640 1 -- 172.21.15.98:0/4073890727 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8734001a90 con 0x7f8728004d50 2024-09-19T08:41:46.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.078+0000 7f87413d4640 1 -- 172.21.15.98:0/4073890727 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8734003b80 con 0x7f8728004d50 2024-09-19T08:41:46.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.078+0000 7f87413d4640 1 -- 172.21.15.98:0/4073890727 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8734005080 con 0x7f8728004d50 2024-09-19T08:41:46.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.082+0000 7f874463a640 1 -- 172.21.15.98:0/4073890727 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8728004d50 msgr2=0x7f87280051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:46.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.082+0000 7f874463a640 1 --2- 172.21.15.98:0/4073890727 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8728004d50 0x7f87280051d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f8734001e70 tx=0x7f873401c780 comp rx=0 tx=0).stop 2024-09-19T08:41:46.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.082+0000 7f8742bd7640 1 -- 172.21.15.98:0/4073890727 reap_dead start 2024-09-19T08:41:46.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.082+0000 7f874463a640 1 -- 172.21.15.98:0/4073890727 shutdown_connections 2024-09-19T08:41:46.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.082+0000 7f874463a640 1 -- 172.21.15.98:0/4073890727 >> 172.21.15.98:0/4073890727 conn(0x7f873c0b2010 msgr2=0x7f873c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:46.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.082+0000 7f874463a640 1 -- 172.21.15.98:0/4073890727 shutdown_connections 2024-09-19T08:41:46.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.082+0000 7f874463a640 1 -- 172.21.15.98:0/4073890727 wait complete. 2024-09-19T08:41:46.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.082+0000 7f874463a640 1 Processor -- start 2024-09-19T08:41:46.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.082+0000 7f874463a640 1 -- start start 2024-09-19T08:41:46.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.082+0000 7f874463a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8728004d50 0x7f873c1ef2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:46.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.082+0000 7f874463a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f873c1f5f70 0x7f873c13d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:46.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.082+0000 7f874463a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f873c13d720 0x7f873c1f5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:46.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.082+0000 7f874463a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f873c145f50 con 0x7f873c13d720 2024-09-19T08:41:46.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.082+0000 7f874463a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f873c073dd0 con 0x7f8728004d50 2024-09-19T08:41:46.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.082+0000 7f874463a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f873c1460d0 con 0x7f873c1f5f70 2024-09-19T08:41:46.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.082+0000 7f8742bd7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f873c13d720 0x7f873c1f5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:46.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.082+0000 7f8742bd7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f873c13d720 0x7f873c1f5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41112/0 (socket says 172.21.15.98:41112) 2024-09-19T08:41:46.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.082+0000 7f8742bd7640 1 -- 172.21.15.98:0/2352304598 learned_addr learned my addr 172.21.15.98:0/2352304598 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:46.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.082+0000 7f87423d6640 1 --2- 172.21.15.98:0/2352304598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8728004d50 0x7f873c1ef2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:46.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.082+0000 7f8741bd5640 1 --2- 172.21.15.98:0/2352304598 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f873c1f5f70 0x7f873c13d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:46.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.082+0000 7f8742bd7640 1 -- 172.21.15.98:0/2352304598 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f873c1f5f70 msgr2=0x7f873c13d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:46.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.082+0000 7f8742bd7640 1 --2- 172.21.15.98:0/2352304598 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f873c1f5f70 0x7f873c13d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:46.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.082+0000 7f8742bd7640 1 -- 172.21.15.98:0/2352304598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8728004d50 msgr2=0x7f873c1ef2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:46.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.082+0000 7f8742bd7640 1 --2- 172.21.15.98:0/2352304598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8728004d50 0x7f873c1ef2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:46.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.082+0000 7f8742bd7640 1 -- 172.21.15.98:0/2352304598 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f873c13dc30 con 0x7f873c13d720 2024-09-19T08:41:46.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.082+0000 7f8742bd7640 1 --2- 172.21.15.98:0/2352304598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f873c13d720 0x7f873c1f5390 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8734098190 tx=0x7f8728005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:46.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.082+0000 7f8732ffd640 1 -- 172.21.15.98:0/2352304598 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87340bb070 con 0x7f873c13d720 2024-09-19T08:41:46.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.082+0000 7f8732ffd640 1 -- 172.21.15.98:0/2352304598 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8734005610 con 0x7f873c13d720 2024-09-19T08:41:46.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.082+0000 7f874463a640 1 -- 172.21.15.98:0/2352304598 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f873c13df10 con 0x7f873c13d720 2024-09-19T08:41:46.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.082+0000 7f8732ffd640 1 -- 172.21.15.98:0/2352304598 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8734001a90 con 0x7f873c13d720 2024-09-19T08:41:46.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.082+0000 7f874463a640 1 -- 172.21.15.98:0/2352304598 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f873c13e4d0 con 0x7f873c13d720 2024-09-19T08:41:46.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.086+0000 7f874463a640 1 -- 172.21.15.98:0/2352304598 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f873c073320 con 0x7f873c13d720 2024-09-19T08:41:46.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.086+0000 7f8732ffd640 1 -- 172.21.15.98:0/2352304598 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f87340be070 con 0x7f873c13d720 2024-09-19T08:41:46.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.086+0000 7f8732ffd640 1 --2- 172.21.15.98:0/2352304598 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8720067aa0 0x7f8720069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:46.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.086+0000 7f8732ffd640 1 -- 172.21.15.98:0/2352304598 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5227+0+0 (secure 0 0 0) 0x7f873411f730 con 0x7f873c13d720 2024-09-19T08:41:46.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.086+0000 7f87423d6640 1 --2- 172.21.15.98:0/2352304598 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8720067aa0 0x7f8720069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:46.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.086+0000 7f87423d6640 1 --2- 172.21.15.98:0/2352304598 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8720067aa0 0x7f8720069f60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8738009b20 tx=0x7f8738008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:46.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.090+0000 7f8732ffd640 1 -- 172.21.15.98:0/2352304598 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f87340ac020 con 0x7f873c13d720 2024-09-19T08:41:46.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.254+0000 7f874463a640 1 -- 172.21.15.98:0/2352304598 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f873c0b02a0 con 0x7f873c13d720 2024-09-19T08:41:46.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.926+0000 7f8732ffd640 1 -- 172.21.15.98:0/2352304598 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v55) ==== 170+0+0 (secure 0 0 0) 0x7f8734016490 con 0x7f873c13d720 2024-09-19T08:41:46.929 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' created 2024-09-19T08:41:46.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.930+0000 7f874463a640 1 -- 172.21.15.98:0/2352304598 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8720067aa0 msgr2=0x7f8720069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:46.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.930+0000 7f874463a640 1 --2- 172.21.15.98:0/2352304598 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8720067aa0 0x7f8720069f60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8738009b20 tx=0x7f8738008040 comp rx=0 tx=0).stop 2024-09-19T08:41:46.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.930+0000 7f874463a640 1 -- 172.21.15.98:0/2352304598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f873c13d720 msgr2=0x7f873c1f5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:46.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.930+0000 7f874463a640 1 --2- 172.21.15.98:0/2352304598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f873c13d720 0x7f873c1f5390 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8734098190 tx=0x7f8728005840 comp rx=0 tx=0).stop 2024-09-19T08:41:46.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.930+0000 7f874463a640 1 -- 172.21.15.98:0/2352304598 shutdown_connections 2024-09-19T08:41:46.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.930+0000 7f874463a640 1 --2- 172.21.15.98:0/2352304598 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8720067aa0 0x7f8720069f60 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:46.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.930+0000 7f874463a640 1 --2- 172.21.15.98:0/2352304598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f873c13d720 0x7f873c1f5390 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:46.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.930+0000 7f874463a640 1 --2- 172.21.15.98:0/2352304598 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f873c1f5f70 0x7f873c13d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:46.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.930+0000 7f874463a640 1 --2- 172.21.15.98:0/2352304598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8728004d50 0x7f873c1ef2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:46.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.930+0000 7f874463a640 1 -- 172.21.15.98:0/2352304598 >> 172.21.15.98:0/2352304598 conn(0x7f873c0b2010 msgr2=0x7f873c0b1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:46.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.930+0000 7f874463a640 1 -- 172.21.15.98:0/2352304598 shutdown_connections 2024-09-19T08:41:46.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:46.930+0000 7f874463a640 1 -- 172.21.15.98:0/2352304598 wait complete. 2024-09-19T08:41:46.967 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T08:41:47.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.090+0000 7f9cd7288640 1 Processor -- start 2024-09-19T08:41:47.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.090+0000 7f9cd7288640 1 -- start start 2024-09-19T08:41:47.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.090+0000 7f9cd7288640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9cd01415a0 0x7f9cd00d20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:47.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.090+0000 7f9cd7288640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9cd00d25f0 0x7f9cd00d3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:47.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.090+0000 7f9cd7288640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cd015eae0 con 0x7f9cd00d38a0 2024-09-19T08:41:47.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.090+0000 7f9cd7288640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f9cd00d2a30 con 0x7f9cd00d25f0 2024-09-19T08:41:47.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.090+0000 7f9cd7288640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9cd015edc0 con 0x7f9cd01415a0 2024-09-19T08:41:47.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.090+0000 7f9cd5825640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f9cd00d38a0 0x7f9cd0161fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:37214/0 (socket says 172.21.15.98:37214) 2024-09-19T08:41:47.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.090+0000 7f9cd5825640 1 -- 172.21.15.98:0/500015436 learned_addr learned my addr 172.21.15.98:0/500015436 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:41:47.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.090+0000 7f9cd4823640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9cd00d25f0 0x7f9cd00d3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:47.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.090+0000 7f9cd5024640 1 --2- 172.21.15.98:0/500015436 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9cd01415a0 0x7f9cd00d20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:47.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.090+0000 7f9cc7fff640 1 -- 172.21.15.98:0/500015436 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 140091276 0 0) 0x7f9cd015eae0 con 0x7f9cd00d38a0 2024-09-19T08:41:47.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.090+0000 7f9cc7fff640 1 -- 172.21.15.98:0/500015436 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9cbc003530 con 0x7f9cd00d38a0 2024-09-19T08:41:47.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.090+0000 7f9cd4823640 1 -- 172.21.15.98:0/500015436 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9cd01415a0 msgr2=0x7f9cd00d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:47.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.090+0000 7f9cd4823640 1 --2- 172.21.15.98:0/500015436 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9cd01415a0 0x7f9cd00d20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:47.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.090+0000 7f9cd4823640 1 -- 172.21.15.98:0/500015436 >> v1:172.21.15.98:6789/0 conn(0x7f9cd00d38a0 legacy=0x7f9cd0161fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:47.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.090+0000 7f9cd4823640 1 -- 172.21.15.98:0/500015436 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd015f0a0 con 0x7f9cd00d25f0 2024-09-19T08:41:47.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.094+0000 7f9cd4823640 1 --2- 172.21.15.98:0/500015436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9cd00d25f0 0x7f9cd00d3360 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00097c0 tx=0x7f9cc001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11d7ff34b0359c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:47.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.094+0000 7f9cc7fff640 1 -- 172.21.15.98:0/500015436 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc00b0070 con 0x7f9cd00d25f0 2024-09-19T08:41:47.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.094+0000 7f9cc7fff640 1 -- 172.21.15.98:0/500015436 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cc00ab600 con 0x7f9cd00d25f0 2024-09-19T08:41:47.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.094+0000 7f9cc7fff640 1 -- 172.21.15.98:0/500015436 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc00ab9d0 con 0x7f9cd00d25f0 2024-09-19T08:41:47.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.094+0000 7f9cd7288640 1 -- 172.21.15.98:0/500015436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9cd00d25f0 msgr2=0x7f9cd00d3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:47.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.094+0000 7f9cd7288640 1 --2- 172.21.15.98:0/500015436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9cd00d25f0 0x7f9cd00d3360 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00097c0 tx=0x7f9cc001c470 comp rx=0 tx=0).stop 2024-09-19T08:41:47.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.094+0000 7f9cd7288640 1 -- 172.21.15.98:0/500015436 shutdown_connections 2024-09-19T08:41:47.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.094+0000 7f9cd7288640 1 --2- 172.21.15.98:0/500015436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9cd00d25f0 0x7f9cd00d3360 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:47.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.094+0000 7f9cd7288640 1 --2- 172.21.15.98:0/500015436 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9cd01415a0 0x7f9cd00d20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:47.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.094+0000 7f9cd7288640 1 -- 172.21.15.98:0/500015436 >> 172.21.15.98:0/500015436 conn(0x7f9cd0074c70 msgr2=0x7f9cd0075080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:47.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.094+0000 7f9cd7288640 1 -- 172.21.15.98:0/500015436 shutdown_connections 2024-09-19T08:41:47.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.094+0000 7f9cd7288640 1 -- 172.21.15.98:0/500015436 wait complete. 2024-09-19T08:41:47.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.094+0000 7f9cd7288640 1 Processor -- start 2024-09-19T08:41:47.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.094+0000 7f9cd7288640 1 -- start start 2024-09-19T08:41:47.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.094+0000 7f9cd7288640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9cd00d25f0 0x7f9cd013e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:47.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.094+0000 7f9cd7288640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9cd00d38a0 0x7f9cd00d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:47.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.094+0000 7f9cd7288640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9cd01415a0 0x7f9cd00d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:47.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.094+0000 7f9cd7288640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd00d42a0 con 0x7f9cd00d25f0 2024-09-19T08:41:47.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.094+0000 7f9cd7288640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd00d4120 con 0x7f9cd01415a0 2024-09-19T08:41:47.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.094+0000 7f9cd7288640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9cd00d3cd0 con 0x7f9cd00d38a0 2024-09-19T08:41:47.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.094+0000 7f9cd5024640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9cd00d25f0 0x7f9cd013e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:47.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.094+0000 7f9cd5024640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9cd00d25f0 0x7f9cd013e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41140/0 (socket says 172.21.15.98:41140) 2024-09-19T08:41:47.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.094+0000 7f9cd5024640 1 -- 172.21.15.98:0/140052000 learned_addr learned my addr 172.21.15.98:0/140052000 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:47.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.094+0000 7f9cd5825640 1 --2- 172.21.15.98:0/140052000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9cd01415a0 0x7f9cd00d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:47.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.098+0000 7f9cd5024640 1 -- 172.21.15.98:0/140052000 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9cd00d38a0 msgr2=0x7f9cd00d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:47.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.098+0000 7f9cd5024640 1 --2- 172.21.15.98:0/140052000 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9cd00d38a0 0x7f9cd00d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:47.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.098+0000 7f9cd5024640 1 -- 172.21.15.98:0/140052000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9cd01415a0 msgr2=0x7f9cd00d13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:47.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.098+0000 7f9cd5024640 1 --2- 172.21.15.98:0/140052000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9cd01415a0 0x7f9cd00d13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:47.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.098+0000 7f9cd5024640 1 -- 172.21.15.98:0/140052000 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd00cf030 con 0x7f9cd00d25f0 2024-09-19T08:41:47.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.098+0000 7f9cd5825640 1 --2- 172.21.15.98:0/140052000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9cd01415a0 0x7f9cd00d13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:47.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.098+0000 7f9cd5024640 1 --2- 172.21.15.98:0/140052000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9cd00d25f0 0x7f9cd013e8b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc00d830 tx=0x7f9ccc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:47.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.098+0000 7f9cc5ffb640 1 -- 172.21.15.98:0/140052000 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc00b6c0 con 0x7f9cd00d25f0 2024-09-19T08:41:47.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.098+0000 7f9cc5ffb640 1 -- 172.21.15.98:0/140052000 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ccc019070 con 0x7f9cd00d25f0 2024-09-19T08:41:47.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.098+0000 7f9cd7288640 1 -- 172.21.15.98:0/140052000 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cd013f0f0 con 0x7f9cd00d25f0 2024-09-19T08:41:47.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.098+0000 7f9cc5ffb640 1 -- 172.21.15.98:0/140052000 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc00ba60 con 0x7f9cd00d25f0 2024-09-19T08:41:47.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.098+0000 7f9cd7288640 1 -- 172.21.15.98:0/140052000 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cd00cf5b0 con 0x7f9cd00d25f0 2024-09-19T08:41:47.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.098+0000 7f9cd7288640 1 -- 172.21.15.98:0/140052000 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cd00d3360 con 0x7f9cd00d25f0 2024-09-19T08:41:47.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.098+0000 7f9cc5ffb640 1 -- 172.21.15.98:0/140052000 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f9ccc01e460 con 0x7f9cd00d25f0 2024-09-19T08:41:47.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.098+0000 7f9cc5ffb640 1 --2- 172.21.15.98:0/140052000 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9cb4067a70 0x7f9cb4069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:47.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.098+0000 7f9cd4823640 1 --2- 172.21.15.98:0/140052000 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9cb4067a70 0x7f9cb4069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:47.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.098+0000 7f9cc5ffb640 1 -- 172.21.15.98:0/140052000 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5609+0+0 (secure 0 0 0) 0x7f9ccc0865b0 con 0x7f9cd00d25f0 2024-09-19T08:41:47.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.098+0000 7f9cd4823640 1 --2- 172.21.15.98:0/140052000 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9cb4067a70 0x7f9cb4069f30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00ad000 tx=0x7f9cc0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:47.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.102+0000 7f9cc5ffb640 1 -- 172.21.15.98:0/140052000 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9ccc056f60 con 0x7f9cd00d25f0 2024-09-19T08:41:47.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.298+0000 7f9cd7288640 1 -- 172.21.15.98:0/140052000 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9cd01565b0 con 0x7f9cd00d25f0 2024-09-19T08:41:47.665 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:41:47.668 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:41:47.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.970+0000 7f9cc5ffb640 1 -- 172.21.15.98:0/140052000 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v28) ==== 172+0+0 (secure 0 0 0) 0x7f9ccc056860 con 0x7f9cd00d25f0 2024-09-19T08:41:47.970 INFO:teuthology.orchestra.run.smithi098.stderr:new fs with metadata pool 8 and data pool 9 2024-09-19T08:41:47.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.978+0000 7f9cd7288640 1 -- 172.21.15.98:0/140052000 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9cb4067a70 msgr2=0x7f9cb4069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:47.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.978+0000 7f9cd7288640 1 --2- 172.21.15.98:0/140052000 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9cb4067a70 0x7f9cb4069f30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00ad000 tx=0x7f9cc0008040 comp rx=0 tx=0).stop 2024-09-19T08:41:47.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.978+0000 7f9cd7288640 1 -- 172.21.15.98:0/140052000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9cd00d25f0 msgr2=0x7f9cd013e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:47.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.978+0000 7f9cd7288640 1 --2- 172.21.15.98:0/140052000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9cd00d25f0 0x7f9cd013e8b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc00d830 tx=0x7f9ccc00dcf0 comp rx=0 tx=0).stop 2024-09-19T08:41:47.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.978+0000 7f9cd7288640 1 -- 172.21.15.98:0/140052000 shutdown_connections 2024-09-19T08:41:47.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.978+0000 7f9cd7288640 1 --2- 172.21.15.98:0/140052000 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9cb4067a70 0x7f9cb4069f30 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:47.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.978+0000 7f9cd7288640 1 --2- 172.21.15.98:0/140052000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9cd01415a0 0x7f9cd00d13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:47.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.978+0000 7f9cd7288640 1 --2- 172.21.15.98:0/140052000 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9cd00d38a0 0x7f9cd00d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:47.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.978+0000 7f9cd7288640 1 --2- 172.21.15.98:0/140052000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9cd00d25f0 0x7f9cd013e8b0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:47.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.978+0000 7f9cd7288640 1 -- 172.21.15.98:0/140052000 >> 172.21.15.98:0/140052000 conn(0x7f9cd0074c70 msgr2=0x7f9cd015fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:47.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.978+0000 7f9cd7288640 1 -- 172.21.15.98:0/140052000 shutdown_connections 2024-09-19T08:41:47.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:47.978+0000 7f9cd7288640 1 -- 172.21.15.98:0/140052000 wait complete. 2024-09-19T08:41:48.006 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:41:48.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.122+0000 7fc79a8b9640 1 Processor -- start 2024-09-19T08:41:48.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.122+0000 7fc79a8b9640 1 -- start start 2024-09-19T08:41:48.122 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.122+0000 7fc79a8b9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7940b7f80 0x7fc79415f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:48.122 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.122+0000 7fc79a8b9640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc794068440 con 0x7fc7940d4820 2024-09-19T08:41:48.122 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.122+0000 7fc79a8b9640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc794162a90 con 0x7fc7940b9970 2024-09-19T08:41:48.122 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.122+0000 7fc79a8b9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc794163050 con 0x7fc7940b7f80 2024-09-19T08:41:48.122 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.122+0000 7fc78bfff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fc7940d4820 0x7fc7941621b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40016/0 (socket says 172.21.15.98:40016) 2024-09-19T08:41:48.122 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.122+0000 7fc78bfff640 1 -- 172.21.15.98:0/2178676892 learned_addr learned my addr 172.21.15.98:0/2178676892 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:41:48.124 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.122+0000 7fc793fff640 1 --2- 172.21.15.98:0/2178676892 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7940b7f80 0x7fc79415f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:48.124 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.122+0000 7fc7937fe640 1 -- 172.21.15.98:0/2178676892 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2339280123 0 0) 0x7fc794068440 con 0x7fc7940d4820 2024-09-19T08:41:48.124 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.122+0000 7fc7937fe640 1 -- 172.21.15.98:0/2178676892 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc774003530 con 0x7fc7940d4820 2024-09-19T08:41:48.124 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.122+0000 7fc7937fe640 1 -- 172.21.15.98:0/2178676892 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1533421871 0 0) 0x7fc774003530 con 0x7fc7940d4820 2024-09-19T08:41:48.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.122+0000 7fc7937fe640 1 -- 172.21.15.98:0/2178676892 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7940b7f80 msgr2=0x7fc79415f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:48.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.122+0000 7fc7937fe640 1 --2- 172.21.15.98:0/2178676892 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7940b7f80 0x7fc79415f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:48.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.122+0000 7fc7937fe640 1 -- 172.21.15.98:0/2178676892 >> v1:172.21.15.139:6789/0 conn(0x7fc7940b9970 legacy=0x7fc7940b9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:48.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.122+0000 7fc7937fe640 1 -- 172.21.15.98:0/2178676892 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc794162d70 con 0x7fc7940d4820 2024-09-19T08:41:48.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.122+0000 7fc7937fe640 1 -- 172.21.15.98:0/2178676892 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fc77c001680 con 0x7fc7940d4820 2024-09-19T08:41:48.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.122+0000 7fc7937fe640 1 -- 172.21.15.98:0/2178676892 >> v1:172.21.15.98:6789/0 conn(0x7fc7940d4820 legacy=0x7fc7941621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:48.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.122+0000 7fc7937fe640 1 --2- 172.21.15.98:0/2178676892 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7940b7f80 0x7fc7740039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:48.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.122+0000 7fc7937fe640 1 --2- 172.21.15.98:0/2178676892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc774004020 0x7fc774004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:48.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.122+0000 7fc7937fe640 1 --2- 172.21.15.98:0/2178676892 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc774004d50 0x7fc7740051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:48.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.122+0000 7fc7937fe640 1 -- 172.21.15.98:0/2178676892 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc794163050 con 0x7fc774004d50 2024-09-19T08:41:48.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.122+0000 7fc7937fe640 1 -- 172.21.15.98:0/2178676892 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc77400bb90 con 0x7fc774004020 2024-09-19T08:41:48.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.122+0000 7fc7937fe640 1 -- 172.21.15.98:0/2178676892 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc77400bd90 con 0x7fc7940b7f80 2024-09-19T08:41:48.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.122+0000 7fc798e56640 1 --2- 172.21.15.98:0/2178676892 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc774004d50 0x7fc7740051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:48.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.122+0000 7fc793fff640 1 --2- 172.21.15.98:0/2178676892 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7940b7f80 0x7fc7740039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:48.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.122+0000 7fc78bfff640 1 --2- 172.21.15.98:0/2178676892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc774004020 0x7fc774004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:48.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.122+0000 7fc798e56640 1 -- 172.21.15.98:0/2178676892 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7940b7f80 msgr2=0x7fc7740039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:48.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.122+0000 7fc798e56640 1 --2- 172.21.15.98:0/2178676892 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7940b7f80 0x7fc7740039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:48.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.122+0000 7fc798e56640 1 -- 172.21.15.98:0/2178676892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc774004020 msgr2=0x7fc774004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:48.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.122+0000 7fc798e56640 1 --2- 172.21.15.98:0/2178676892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc774004020 0x7fc774004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:48.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.122+0000 7fc798e56640 1 -- 172.21.15.98:0/2178676892 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc774003530 con 0x7fc774004d50 2024-09-19T08:41:48.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.122+0000 7fc793fff640 1 --2- 172.21.15.98:0/2178676892 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7940b7f80 0x7fc7740039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:41:48.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.126+0000 7fc798e56640 1 --2- 172.21.15.98:0/2178676892 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc774004d50 0x7fc7740051d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc78c001e70 tx=0x7fc78c01c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c9433c8395700d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:48.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.126+0000 7fc7937fe640 1 -- 172.21.15.98:0/2178676892 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c001a90 con 0x7fc774004d50 2024-09-19T08:41:48.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.126+0000 7fc7937fe640 1 -- 172.21.15.98:0/2178676892 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc78c003b80 con 0x7fc774004d50 2024-09-19T08:41:48.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.126+0000 7fc7937fe640 1 -- 172.21.15.98:0/2178676892 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78c005080 con 0x7fc774004d50 2024-09-19T08:41:48.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.126+0000 7fc79a8b9640 1 -- 172.21.15.98:0/2178676892 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc774004d50 msgr2=0x7fc7740051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:48.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.126+0000 7fc79a8b9640 1 --2- 172.21.15.98:0/2178676892 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc774004d50 0x7fc7740051d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc78c001e70 tx=0x7fc78c01c780 comp rx=0 tx=0).stop 2024-09-19T08:41:48.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.126+0000 7fc798e56640 1 -- 172.21.15.98:0/2178676892 reap_dead start 2024-09-19T08:41:48.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.126+0000 7fc79a8b9640 1 -- 172.21.15.98:0/2178676892 shutdown_connections 2024-09-19T08:41:48.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.126+0000 7fc79a8b9640 1 -- 172.21.15.98:0/2178676892 >> 172.21.15.98:0/2178676892 conn(0x7fc794074e40 msgr2=0x7fc794075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:48.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.126+0000 7fc79a8b9640 1 -- 172.21.15.98:0/2178676892 shutdown_connections 2024-09-19T08:41:48.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.126+0000 7fc79a8b9640 1 -- 172.21.15.98:0/2178676892 wait complete. 2024-09-19T08:41:48.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.126+0000 7fc79a8b9640 1 Processor -- start 2024-09-19T08:41:48.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.126+0000 7fc79a8b9640 1 -- start start 2024-09-19T08:41:48.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.126+0000 7fc79a8b9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc794153b90 0x7fc794157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:48.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.126+0000 7fc79a8b9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc7940ca180 0x7fc79413e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:48.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.126+0000 7fc79a8b9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7940ca600 0x7fc79413eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:48.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.126+0000 7fc79a8b9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc7940b84e0 con 0x7fc7940ca180 2024-09-19T08:41:48.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.126+0000 7fc79a8b9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc7940b8360 con 0x7fc7940ca600 2024-09-19T08:41:48.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.126+0000 7fc79a8b9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc79415f560 con 0x7fc794153b90 2024-09-19T08:41:48.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.130+0000 7fc78bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc7940ca180 0x7fc79413e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:48.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.130+0000 7fc78bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc7940ca180 0x7fc79413e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46842/0 (socket says 172.21.15.98:46842) 2024-09-19T08:41:48.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.130+0000 7fc78bfff640 1 -- 172.21.15.98:0/1442502659 learned_addr learned my addr 172.21.15.98:0/1442502659 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:48.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.130+0000 7fc793fff640 1 --2- 172.21.15.98:0/1442502659 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc794153b90 0x7fc794157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:48.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.130+0000 7fc798e56640 1 --2- 172.21.15.98:0/1442502659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7940ca600 0x7fc79413eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:48.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.130+0000 7fc78bfff640 1 -- 172.21.15.98:0/1442502659 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc794153b90 msgr2=0x7fc794157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:48.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.130+0000 7fc78bfff640 1 --2- 172.21.15.98:0/1442502659 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc794153b90 0x7fc794157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:48.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.130+0000 7fc78bfff640 1 -- 172.21.15.98:0/1442502659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7940ca600 msgr2=0x7fc79413eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:48.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.130+0000 7fc78bfff640 1 --2- 172.21.15.98:0/1442502659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7940ca600 0x7fc79413eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:48.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.130+0000 7fc78bfff640 1 -- 172.21.15.98:0/1442502659 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7940cae40 con 0x7fc7940ca180 2024-09-19T08:41:48.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.130+0000 7fc798e56640 1 --2- 172.21.15.98:0/1442502659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7940ca600 0x7fc79413eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:48.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.130+0000 7fc78bfff640 1 --2- 172.21.15.98:0/1442502659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc7940ca180 0x7fc79413e970 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00e7f0 tx=0x7fc77c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:48.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.130+0000 7fc7917fa640 1 -- 172.21.15.98:0/1442502659 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c00c8d0 con 0x7fc7940ca180 2024-09-19T08:41:48.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.130+0000 7fc7917fa640 1 -- 172.21.15.98:0/1442502659 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc77c018070 con 0x7fc7940ca180 2024-09-19T08:41:48.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.130+0000 7fc7917fa640 1 -- 172.21.15.98:0/1442502659 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c00ccb0 con 0x7fc7940ca180 2024-09-19T08:41:48.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.130+0000 7fc79a8b9640 1 -- 172.21.15.98:0/1442502659 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7940cc2b0 con 0x7fc7940ca180 2024-09-19T08:41:48.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.130+0000 7fc79a8b9640 1 -- 172.21.15.98:0/1442502659 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7940cc550 con 0x7fc7940ca180 2024-09-19T08:41:48.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.130+0000 7fc79a8b9640 1 -- 172.21.15.98:0/1442502659 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc79415f8b0 con 0x7fc7940ca180 2024-09-19T08:41:48.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.130+0000 7fc7917fa640 1 -- 172.21.15.98:0/1442502659 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fc77c01d460 con 0x7fc7940ca180 2024-09-19T08:41:48.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.130+0000 7fc7917fa640 1 --2- 172.21.15.98:0/1442502659 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc778067a70 0x7fc778069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:48.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.130+0000 7fc793fff640 1 --2- 172.21.15.98:0/1442502659 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc778067a70 0x7fc778069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:48.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.130+0000 7fc7917fa640 1 -- 172.21.15.98:0/1442502659 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fc77c085f90 con 0x7fc7940ca180 2024-09-19T08:41:48.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.130+0000 7fc793fff640 1 --2- 172.21.15.98:0/1442502659 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc778067a70 0x7fc778069f30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc7840038b0 tx=0x7fc784008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:48.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.134+0000 7fc7917fa640 1 -- 172.21.15.98:0/1442502659 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc77c0567b0 con 0x7fc7940ca180 2024-09-19T08:41:48.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.298+0000 7fc79a8b9640 1 -- 172.21.15.98:0/1442502659 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc7940d5050 con 0x7fc7940ca180 2024-09-19T08:41:48.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.302+0000 7fc7917fa640 1 -- 172.21.15.98:0/1442502659 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+16738 (secure 0 0 0) 0x7fc77c0560b0 con 0x7fc7940ca180 2024-09-19T08:41:48.301 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:41:48.301 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":56,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:41:47.303480+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:41:44.964773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-19T08:41:46.289740+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:41:48.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.302+0000 7fc79a8b9640 1 -- 172.21.15.98:0/1442502659 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc778067a70 msgr2=0x7fc778069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:48.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.302+0000 7fc79a8b9640 1 --2- 172.21.15.98:0/1442502659 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc778067a70 0x7fc778069f30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc7840038b0 tx=0x7fc784008040 comp rx=0 tx=0).stop 2024-09-19T08:41:48.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.302+0000 7fc79a8b9640 1 -- 172.21.15.98:0/1442502659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc7940ca180 msgr2=0x7fc79413e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:48.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.302+0000 7fc79a8b9640 1 --2- 172.21.15.98:0/1442502659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc7940ca180 0x7fc79413e970 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00e7f0 tx=0x7fc77c00ecb0 comp rx=0 tx=0).stop 2024-09-19T08:41:48.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.302+0000 7fc79a8b9640 1 -- 172.21.15.98:0/1442502659 shutdown_connections 2024-09-19T08:41:48.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.302+0000 7fc79a8b9640 1 --2- 172.21.15.98:0/1442502659 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc778067a70 0x7fc778069f30 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:48.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.302+0000 7fc79a8b9640 1 --2- 172.21.15.98:0/1442502659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7940ca600 0x7fc79413eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:48.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.302+0000 7fc79a8b9640 1 --2- 172.21.15.98:0/1442502659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc7940ca180 0x7fc79413e970 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:48.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.302+0000 7fc79a8b9640 1 --2- 172.21.15.98:0/1442502659 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc794153b90 0x7fc794157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:48.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.302+0000 7fc79a8b9640 1 -- 172.21.15.98:0/1442502659 >> 172.21.15.98:0/1442502659 conn(0x7fc794074e40 msgr2=0x7fc7940cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:48.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.302+0000 7fc79a8b9640 1 -- 172.21.15.98:0/1442502659 shutdown_connections 2024-09-19T08:41:48.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.302+0000 7fc79a8b9640 1 -- 172.21.15.98:0/1442502659 wait complete. 2024-09-19T08:41:48.320 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:41:48.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.442+0000 7f5ab3d6c640 1 Processor -- start 2024-09-19T08:41:48.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.442+0000 7f5ab3d6c640 1 -- start start 2024-09-19T08:41:48.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.442+0000 7f5ab3d6c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5aac06bc90 0x7f5aac06c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:48.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.442+0000 7f5ab3d6c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5aac0ce840 0x7f5aac0cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:48.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.442+0000 7f5ab3d6c640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5aac0ccc60 con 0x7f5aac158fd0 2024-09-19T08:41:48.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.442+0000 7f5ab3d6c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f5aac1593b0 con 0x7f5aac0ce840 2024-09-19T08:41:48.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.442+0000 7f5ab3d6c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5aac0ccf40 con 0x7f5aac06bc90 2024-09-19T08:41:48.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.442+0000 7f5ab1b08640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f5aac158fd0 0x7f5aac0b3e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40026/0 (socket says 172.21.15.98:40026) 2024-09-19T08:41:48.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.442+0000 7f5ab1b08640 1 -- 172.21.15.98:0/931674256 learned_addr learned my addr 172.21.15.98:0/931674256 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:41:48.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.442+0000 7f5ab1307640 1 --2- 172.21.15.98:0/931674256 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5aac06bc90 0x7f5aac06c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:48.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.442+0000 7f5ab2309640 1 --2- 172.21.15.98:0/931674256 >> v2:172.21.15.139:3300/0 conn(0x7f5aac0ce840 0x7f5aac0cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:48.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.442+0000 7f5ab0b06640 1 -- 172.21.15.98:0/931674256 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2115198968 0 0) 0x7f5aac0ccc60 con 0x7f5aac158fd0 2024-09-19T08:41:48.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.442+0000 7f5ab0b06640 1 -- 172.21.15.98:0/931674256 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a98003530 con 0x7f5aac158fd0 2024-09-19T08:41:48.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.442+0000 7f5ab0b06640 1 -- 172.21.15.98:0/931674256 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1377505018 0 0) 0x7f5a98003530 con 0x7f5aac158fd0 2024-09-19T08:41:48.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.442+0000 7f5ab0b06640 1 -- 172.21.15.98:0/931674256 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5aac06bc90 msgr2=0x7f5aac06c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:48.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.442+0000 7f5ab0b06640 1 --2- 172.21.15.98:0/931674256 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5aac06bc90 0x7f5aac06c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:48.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.442+0000 7f5ab0b06640 1 -- 172.21.15.98:0/931674256 >> v2:172.21.15.139:3300/0 conn(0x7f5aac0ce840 msgr2=0x7f5aac0cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:48.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.442+0000 7f5ab0b06640 1 --2- 172.21.15.98:0/931674256 >> v2:172.21.15.139:3300/0 conn(0x7f5aac0ce840 0x7f5aac0cec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:48.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.442+0000 7f5ab0b06640 1 -- 172.21.15.98:0/931674256 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aac0cd220 con 0x7f5aac158fd0 2024-09-19T08:41:48.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.442+0000 7f5ab0b06640 1 -- 172.21.15.98:0/931674256 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f5aa8001680 con 0x7f5aac158fd0 2024-09-19T08:41:48.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.442+0000 7f5ab0b06640 1 -- 172.21.15.98:0/931674256 >> v1:172.21.15.98:6789/0 conn(0x7f5aac158fd0 legacy=0x7f5aac0b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:48.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.442+0000 7f5ab0b06640 1 --2- 172.21.15.98:0/931674256 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5aac06bc90 0x7f5a980039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:48.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.442+0000 7f5ab0b06640 1 --2- 172.21.15.98:0/931674256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a98003f40 0x7f5a98004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:48.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.442+0000 7f5ab0b06640 1 --2- 172.21.15.98:0/931674256 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a98004d50 0x7f5a980051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:48.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.442+0000 7f5ab0b06640 1 -- 172.21.15.98:0/931674256 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac1593b0 con 0x7f5a98003f40 2024-09-19T08:41:48.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.442+0000 7f5ab0b06640 1 -- 172.21.15.98:0/931674256 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac0ccf40 con 0x7f5a98004d50 2024-09-19T08:41:48.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.442+0000 7f5ab0b06640 1 -- 172.21.15.98:0/931674256 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5a9800ba20 con 0x7f5aac06bc90 2024-09-19T08:41:48.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.442+0000 7f5ab1307640 1 --2- 172.21.15.98:0/931674256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a98003f40 0x7f5a98004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:48.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.442+0000 7f5ab1b08640 1 --2- 172.21.15.98:0/931674256 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5aac06bc90 0x7f5a980039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:48.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.442+0000 7f5ab2309640 1 --2- 172.21.15.98:0/931674256 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a98004d50 0x7f5a980051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:48.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.442+0000 7f5ab1307640 1 -- 172.21.15.98:0/931674256 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5aac06bc90 msgr2=0x7f5a980039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:48.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.442+0000 7f5ab1307640 1 --2- 172.21.15.98:0/931674256 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5aac06bc90 0x7f5a980039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:48.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.442+0000 7f5ab1307640 1 -- 172.21.15.98:0/931674256 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a98004d50 msgr2=0x7f5a980051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:48.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.442+0000 7f5ab1307640 1 --2- 172.21.15.98:0/931674256 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a98004d50 0x7f5a980051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:48.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.442+0000 7f5ab1307640 1 -- 172.21.15.98:0/931674256 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a98003530 con 0x7f5a98003f40 2024-09-19T08:41:48.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.446+0000 7f5ab1307640 1 --2- 172.21.15.98:0/931674256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a98003f40 0x7f5a98004810 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c009b10 tx=0x7f5a9c01ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=475906df2e41693e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:48.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.446+0000 7f5ab0b06640 1 -- 172.21.15.98:0/931674256 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c0b2070 con 0x7f5a98003f40 2024-09-19T08:41:48.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.446+0000 7f5ab0b06640 1 -- 172.21.15.98:0/931674256 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a9c0a55b0 con 0x7f5a98003f40 2024-09-19T08:41:48.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.446+0000 7f5ab0b06640 1 -- 172.21.15.98:0/931674256 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c0096b0 con 0x7f5a98003f40 2024-09-19T08:41:48.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.446+0000 7f5ab3d6c640 1 -- 172.21.15.98:0/931674256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a98003f40 msgr2=0x7f5a98004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:48.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.446+0000 7f5ab3d6c640 1 --2- 172.21.15.98:0/931674256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a98003f40 0x7f5a98004810 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c009b10 tx=0x7f5a9c01ca40 comp rx=0 tx=0).stop 2024-09-19T08:41:48.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.446+0000 7f5ab2309640 1 -- 172.21.15.98:0/931674256 reap_dead start 2024-09-19T08:41:48.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.446+0000 7f5ab3d6c640 1 -- 172.21.15.98:0/931674256 shutdown_connections 2024-09-19T08:41:48.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.446+0000 7f5ab3d6c640 1 -- 172.21.15.98:0/931674256 >> 172.21.15.98:0/931674256 conn(0x7f5aac0b2010 msgr2=0x7f5aac0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:48.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.446+0000 7f5ab3d6c640 1 -- 172.21.15.98:0/931674256 shutdown_connections 2024-09-19T08:41:48.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.446+0000 7f5ab3d6c640 1 -- 172.21.15.98:0/931674256 wait complete. 2024-09-19T08:41:48.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.450+0000 7f5ab3d6c640 1 Processor -- start 2024-09-19T08:41:48.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.450+0000 7f5ab3d6c640 1 -- start start 2024-09-19T08:41:48.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.450+0000 7f5ab3d6c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a98003f40 0x7f5aac147e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:48.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.450+0000 7f5ab3d6c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5aac072f30 0x7f5aac0733b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:48.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.450+0000 7f5ab3d6c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5aac071dc0 0x7f5aac072240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:48.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.450+0000 7f5ab3d6c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac06c600 con 0x7f5aac071dc0 2024-09-19T08:41:48.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.450+0000 7f5ab3d6c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac06c480 con 0x7f5a98003f40 2024-09-19T08:41:48.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.450+0000 7f5ab3d6c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5aac15c7a0 con 0x7f5aac072f30 2024-09-19T08:41:48.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.450+0000 7f5ab2309640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5aac071dc0 0x7f5aac072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:48.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.450+0000 7f5ab2309640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5aac071dc0 0x7f5aac072240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46858/0 (socket says 172.21.15.98:46858) 2024-09-19T08:41:48.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.450+0000 7f5ab2309640 1 -- 172.21.15.98:0/730124559 learned_addr learned my addr 172.21.15.98:0/730124559 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:48.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.450+0000 7f5ab1307640 1 --2- 172.21.15.98:0/730124559 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5aac072f30 0x7f5aac0733b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:48.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.450+0000 7f5ab1b08640 1 --2- 172.21.15.98:0/730124559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a98003f40 0x7f5aac147e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:48.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.450+0000 7f5ab2309640 1 -- 172.21.15.98:0/730124559 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5aac072f30 msgr2=0x7f5aac0733b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:48.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.450+0000 7f5ab2309640 1 --2- 172.21.15.98:0/730124559 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5aac072f30 0x7f5aac0733b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:48.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.450+0000 7f5ab2309640 1 -- 172.21.15.98:0/730124559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a98003f40 msgr2=0x7f5aac147e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:48.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.450+0000 7f5ab2309640 1 --2- 172.21.15.98:0/730124559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a98003f40 0x7f5aac147e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:48.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.450+0000 7f5ab2309640 1 -- 172.21.15.98:0/730124559 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aac072b30 con 0x7f5aac071dc0 2024-09-19T08:41:48.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.450+0000 7f5ab2309640 1 --2- 172.21.15.98:0/730124559 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5aac071dc0 0x7f5aac072240 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5aac15c770 tx=0x7f5aa4012590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:48.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.450+0000 7f5aa27fc640 1 -- 172.21.15.98:0/730124559 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa40039a0 con 0x7f5aac071dc0 2024-09-19T08:41:48.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.450+0000 7f5aa27fc640 1 -- 172.21.15.98:0/730124559 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5aa401d070 con 0x7f5aac071dc0 2024-09-19T08:41:48.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.450+0000 7f5aa27fc640 1 -- 172.21.15.98:0/730124559 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4003d80 con 0x7f5aac071dc0 2024-09-19T08:41:48.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.450+0000 7f5ab3d6c640 1 -- 172.21.15.98:0/730124559 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aac15cd10 con 0x7f5aac071dc0 2024-09-19T08:41:48.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.450+0000 7f5ab3d6c640 1 -- 172.21.15.98:0/730124559 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aac15d2d0 con 0x7f5aac071dc0 2024-09-19T08:41:48.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.450+0000 7f5ab3d6c640 1 -- 172.21.15.98:0/730124559 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5aac0cec90 con 0x7f5aac071dc0 2024-09-19T08:41:48.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.450+0000 7f5aa27fc640 1 -- 172.21.15.98:0/730124559 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f5aa4004030 con 0x7f5aac071dc0 2024-09-19T08:41:48.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.450+0000 7f5aa27fc640 1 --2- 172.21.15.98:0/730124559 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5a90067a70 0x7f5a90069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:48.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.450+0000 7f5aa27fc640 1 -- 172.21.15.98:0/730124559 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f5aa4088d90 con 0x7f5aac071dc0 2024-09-19T08:41:48.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.454+0000 7f5ab1b08640 1 --2- 172.21.15.98:0/730124559 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5a90067a70 0x7f5a90069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:48.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.454+0000 7f5ab1b08640 1 --2- 172.21.15.98:0/730124559 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5a90067a70 0x7f5a90069f30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8000c10 tx=0x7f5aa8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:48.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.458+0000 7f5aa27fc640 1 -- 172.21.15.98:0/730124559 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f5aa4027020 con 0x7f5aac071dc0 2024-09-19T08:41:48.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.622+0000 7f5ab3d6c640 1 -- 172.21.15.98:0/730124559 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5aac1618f0 con 0x7f5aac071dc0 2024-09-19T08:41:48.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.622+0000 7f5aa27fc640 1 -- 172.21.15.98:0/730124559 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+16738 (secure 0 0 0) 0x7f5aa4059540 con 0x7f5aac071dc0 2024-09-19T08:41:48.622 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:41:48.623 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":56,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:41:47.303480+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:41:44.964773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-19T08:41:46.289740+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:41:48.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.626+0000 7f5ab3d6c640 1 -- 172.21.15.98:0/730124559 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5a90067a70 msgr2=0x7f5a90069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:48.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.626+0000 7f5ab3d6c640 1 --2- 172.21.15.98:0/730124559 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5a90067a70 0x7f5a90069f30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8000c10 tx=0x7f5aa8009040 comp rx=0 tx=0).stop 2024-09-19T08:41:48.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.626+0000 7f5ab3d6c640 1 -- 172.21.15.98:0/730124559 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5aac071dc0 msgr2=0x7f5aac072240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:48.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.626+0000 7f5ab3d6c640 1 --2- 172.21.15.98:0/730124559 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5aac071dc0 0x7f5aac072240 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5aac15c770 tx=0x7f5aa4012590 comp rx=0 tx=0).stop 2024-09-19T08:41:48.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.626+0000 7f5ab3d6c640 1 -- 172.21.15.98:0/730124559 shutdown_connections 2024-09-19T08:41:48.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.626+0000 7f5ab3d6c640 1 --2- 172.21.15.98:0/730124559 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5a90067a70 0x7f5a90069f30 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:48.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.626+0000 7f5ab3d6c640 1 --2- 172.21.15.98:0/730124559 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5aac071dc0 0x7f5aac072240 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:48.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.626+0000 7f5ab3d6c640 1 --2- 172.21.15.98:0/730124559 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5aac072f30 0x7f5aac0733b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:48.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.626+0000 7f5ab3d6c640 1 --2- 172.21.15.98:0/730124559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a98003f40 0x7f5aac147e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:48.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.626+0000 7f5ab3d6c640 1 -- 172.21.15.98:0/730124559 >> 172.21.15.98:0/730124559 conn(0x7f5aac0b2010 msgr2=0x7f5aac155c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:48.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.626+0000 7f5ab3d6c640 1 -- 172.21.15.98:0/730124559 shutdown_connections 2024-09-19T08:41:48.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.626+0000 7f5ab3d6c640 1 -- 172.21.15.98:0/730124559 wait complete. 2024-09-19T08:41:48.641 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-19T08:41:48.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.754+0000 7f9f51c4b640 1 Processor -- start 2024-09-19T08:41:48.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.754+0000 7f9f51c4b640 1 -- start start 2024-09-19T08:41:48.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.754+0000 7f9f51c4b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9f4c15c9b0 0x7f9f4c15ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:48.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.754+0000 7f9f51c4b640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f4c0d1360 con 0x7f9f4c06bc90 2024-09-19T08:41:48.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.754+0000 7f9f51c4b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f4c06d590 con 0x7f9f4c0d1e20 2024-09-19T08:41:48.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.754+0000 7f9f51c4b640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f9f4c0d1620 con 0x7f9f4c15c9b0 2024-09-19T08:41:48.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.754+0000 7f9f4bfff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9f4c15c9b0 0x7f9f4c15ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:48.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.754+0000 7f9f4b7fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f9f4c06bc90 0x7f9f4c15c270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40040/0 (socket says 172.21.15.98:40040) 2024-09-19T08:41:48.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.754+0000 7f9f4b7fe640 1 -- 172.21.15.98:0/2335377416 learned_addr learned my addr 172.21.15.98:0/2335377416 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:41:48.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.754+0000 7f9f4bfff640 1 -- 172.21.15.98:0/2335377416 >> v1:172.21.15.139:6789/0 conn(0x7f9f4c0d1e20 legacy=0x7f9f4c06c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:48.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.754+0000 7f9f4bfff640 1 -- 172.21.15.98:0/2335377416 >> v1:172.21.15.98:6789/0 conn(0x7f9f4c06bc90 legacy=0x7f9f4c15c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:48.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.754+0000 7f9f4bfff640 1 -- 172.21.15.98:0/2335377416 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f4c0d1900 con 0x7f9f4c15c9b0 2024-09-19T08:41:48.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.754+0000 7f9f4bfff640 1 --2- 172.21.15.98:0/2335377416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f4c15c9b0 0x7f9f4c15ce20 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0097c0 tx=0x7f9f3c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5bd4135a11977e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:48.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.754+0000 7f9f4a7fc640 1 -- 172.21.15.98:0/2335377416 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c0b0070 con 0x7f9f4c15c9b0 2024-09-19T08:41:48.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.754+0000 7f9f4a7fc640 1 -- 172.21.15.98:0/2335377416 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f3c0ab600 con 0x7f9f4c15c9b0 2024-09-19T08:41:48.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.754+0000 7f9f4a7fc640 1 -- 172.21.15.98:0/2335377416 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c0ab9d0 con 0x7f9f4c15c9b0 2024-09-19T08:41:48.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.758+0000 7f9f51c4b640 1 -- 172.21.15.98:0/2335377416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f4c15c9b0 msgr2=0x7f9f4c15ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:48.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.758+0000 7f9f51c4b640 1 --2- 172.21.15.98:0/2335377416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f4c15c9b0 0x7f9f4c15ce20 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0097c0 tx=0x7f9f3c01c460 comp rx=0 tx=0).stop 2024-09-19T08:41:48.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.758+0000 7f9f51c4b640 1 -- 172.21.15.98:0/2335377416 shutdown_connections 2024-09-19T08:41:48.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.758+0000 7f9f51c4b640 1 --2- 172.21.15.98:0/2335377416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f4c15c9b0 0x7f9f4c15ce20 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:48.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.758+0000 7f9f51c4b640 1 -- 172.21.15.98:0/2335377416 >> 172.21.15.98:0/2335377416 conn(0x7f9f4c0b4d60 msgr2=0x7f9f4c0b5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:48.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.758+0000 7f9f51c4b640 1 -- 172.21.15.98:0/2335377416 shutdown_connections 2024-09-19T08:41:48.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.758+0000 7f9f51c4b640 1 -- 172.21.15.98:0/2335377416 wait complete. 2024-09-19T08:41:48.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.758+0000 7f9f51c4b640 1 Processor -- start 2024-09-19T08:41:48.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.758+0000 7f9f51c4b640 1 -- start start 2024-09-19T08:41:48.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.758+0000 7f9f51c4b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f4c06bc90 0x7f9f4c149170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:48.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.758+0000 7f9f51c4b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f4c0d1e20 0x7f9f4c148b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:48.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.758+0000 7f9f51c4b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f4c15c9b0 0x7f9f4c147460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:48.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.758+0000 7f9f51c4b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c1697d0 con 0x7f9f4c06bc90 2024-09-19T08:41:48.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.758+0000 7f9f51c4b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c169650 con 0x7f9f4c0d1e20 2024-09-19T08:41:48.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.758+0000 7f9f51c4b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9f4c169950 con 0x7f9f4c15c9b0 2024-09-19T08:41:48.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.758+0000 7f9f4b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f4c06bc90 0x7f9f4c149170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:48.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.758+0000 7f9f4b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f4c06bc90 0x7f9f4c149170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46884/0 (socket says 172.21.15.98:46884) 2024-09-19T08:41:48.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.758+0000 7f9f4b7fe640 1 -- 172.21.15.98:0/660632384 learned_addr learned my addr 172.21.15.98:0/660632384 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:48.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.758+0000 7f9f4b7fe640 1 -- 172.21.15.98:0/660632384 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f4c15c9b0 msgr2=0x7f9f4c147460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:48.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.758+0000 7f9f4b7fe640 1 --2- 172.21.15.98:0/660632384 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f4c15c9b0 0x7f9f4c147460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:48.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.758+0000 7f9f4b7fe640 1 -- 172.21.15.98:0/660632384 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f4c0d1e20 msgr2=0x7f9f4c148b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:48.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.758+0000 7f9f4b7fe640 1 --2- 172.21.15.98:0/660632384 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f4c0d1e20 0x7f9f4c148b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:48.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.758+0000 7f9f4b7fe640 1 -- 172.21.15.98:0/660632384 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f4c147dd0 con 0x7f9f4c06bc90 2024-09-19T08:41:48.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.758+0000 7f9f4b7fe640 1 --2- 172.21.15.98:0/660632384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f4c06bc90 0x7f9f4c149170 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400cb40 tx=0x7f9f4400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:48.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.758+0000 7f9f43fff640 1 -- 172.21.15.98:0/660632384 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f44017070 con 0x7f9f4c06bc90 2024-09-19T08:41:48.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.758+0000 7f9f43fff640 1 -- 172.21.15.98:0/660632384 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f4400aca0 con 0x7f9f4c06bc90 2024-09-19T08:41:48.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.758+0000 7f9f51c4b640 1 -- 172.21.15.98:0/660632384 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f4c1480b0 con 0x7f9f4c06bc90 2024-09-19T08:41:48.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.758+0000 7f9f43fff640 1 -- 172.21.15.98:0/660632384 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f44012620 con 0x7f9f4c06bc90 2024-09-19T08:41:48.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.758+0000 7f9f51c4b640 1 -- 172.21.15.98:0/660632384 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f4c1479a0 con 0x7f9f4c06bc90 2024-09-19T08:41:48.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.758+0000 7f9f51c4b640 1 -- 172.21.15.98:0/660632384 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f4c15ce20 con 0x7f9f4c06bc90 2024-09-19T08:41:48.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.762+0000 7f9f43fff640 1 -- 172.21.15.98:0/660632384 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f9f4401a070 con 0x7f9f4c06bc90 2024-09-19T08:41:48.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.762+0000 7f9f43fff640 1 --2- 172.21.15.98:0/660632384 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9f2c067a70 0x7f9f2c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:48.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.762+0000 7f9f43fff640 1 -- 172.21.15.98:0/660632384 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f9f44085e90 con 0x7f9f4c06bc90 2024-09-19T08:41:48.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.762+0000 7f9f4affd640 1 --2- 172.21.15.98:0/660632384 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9f2c067a70 0x7f9f2c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:48.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.762+0000 7f9f4affd640 1 --2- 172.21.15.98:0/660632384 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9f2c067a70 0x7f9f2c069f30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9f34007fd0 tx=0x7f9f340079a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:48.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.766+0000 7f9f43fff640 1 -- 172.21.15.98:0/660632384 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9f440566c0 con 0x7f9f4c06bc90 2024-09-19T08:41:48.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.950+0000 7f9f51c4b640 1 -- 172.21.15.98:0/660632384 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f9f4c0b1840 con 0x7f9f4c06bc90 2024-09-19T08:41:48.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.962+0000 7f9f43fff640 1 -- 172.21.15.98:0/660632384 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v29) ==== 118+0+0 (secure 0 0 0) 0x7f9f44055fc0 con 0x7f9f4c06bc90 2024-09-19T08:41:48.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.966+0000 7f9f51c4b640 1 -- 172.21.15.98:0/660632384 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9f2c067a70 msgr2=0x7f9f2c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:48.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.966+0000 7f9f51c4b640 1 --2- 172.21.15.98:0/660632384 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9f2c067a70 0x7f9f2c069f30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9f34007fd0 tx=0x7f9f340079a0 comp rx=0 tx=0).stop 2024-09-19T08:41:48.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.966+0000 7f9f51c4b640 1 -- 172.21.15.98:0/660632384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f4c06bc90 msgr2=0x7f9f4c149170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:48.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.966+0000 7f9f51c4b640 1 --2- 172.21.15.98:0/660632384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f4c06bc90 0x7f9f4c149170 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400cb40 tx=0x7f9f4400a060 comp rx=0 tx=0).stop 2024-09-19T08:41:48.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.966+0000 7f9f51c4b640 1 -- 172.21.15.98:0/660632384 shutdown_connections 2024-09-19T08:41:48.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.966+0000 7f9f51c4b640 1 --2- 172.21.15.98:0/660632384 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9f2c067a70 0x7f9f2c069f30 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:48.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.966+0000 7f9f51c4b640 1 --2- 172.21.15.98:0/660632384 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f4c15c9b0 0x7f9f4c147460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:48.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.966+0000 7f9f51c4b640 1 --2- 172.21.15.98:0/660632384 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f4c0d1e20 0x7f9f4c148b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:48.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.966+0000 7f9f51c4b640 1 --2- 172.21.15.98:0/660632384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f4c06bc90 0x7f9f4c149170 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:48.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.966+0000 7f9f51c4b640 1 -- 172.21.15.98:0/660632384 >> 172.21.15.98:0/660632384 conn(0x7f9f4c0b4d60 msgr2=0x7f9f4c073290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:48.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.966+0000 7f9f51c4b640 1 -- 172.21.15.98:0/660632384 shutdown_connections 2024-09-19T08:41:48.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:48.966+0000 7f9f51c4b640 1 -- 172.21.15.98:0/660632384 wait complete. 2024-09-19T08:41:48.983 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-19T08:41:48.983 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-19T08:41:49.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.098+0000 7fdba44b6640 1 Processor -- start 2024-09-19T08:41:49.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.098+0000 7fdba44b6640 1 -- start start 2024-09-19T08:41:49.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.098+0000 7fdba44b6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb9c144d10 0x7fdb9c0722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:49.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.098+0000 7fdba44b6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdb9c06c3a0 0x7fdb9c0cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:49.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.098+0000 7fdba44b6640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb9c072ea0 con 0x7fdb9c072820 2024-09-19T08:41:49.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.098+0000 7fdba44b6640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fdb9c072c60 con 0x7fdb9c06c3a0 2024-09-19T08:41:49.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.098+0000 7fdba44b6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdb9c073ab0 con 0x7fdb9c144d10 2024-09-19T08:41:49.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.098+0000 7fdba2a53640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdb9c06c3a0 0x7fdb9c0cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:49.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.098+0000 7fdba2252640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb9c144d10 0x7fdb9c0722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:49.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.098+0000 7fdba1a51640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fdb9c072820 0x7fdb9c06bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:53718/0 (socket says 172.21.15.98:53718) 2024-09-19T08:41:49.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.098+0000 7fdba1a51640 1 -- 172.21.15.98:0/1000565100 learned_addr learned my addr 172.21.15.98:0/1000565100 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:41:49.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.098+0000 7fdba2a53640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdb9c06c3a0 0x7fdb9c0cd250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:51906/0 (socket says 172.21.15.98:51906) 2024-09-19T08:41:49.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.098+0000 7fdba2a53640 1 -- 172.21.15.98:0/1000565100 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb9c144d10 msgr2=0x7fdb9c0722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:49.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.098+0000 7fdba2a53640 1 --2- 172.21.15.98:0/1000565100 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb9c144d10 0x7fdb9c0722e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:49.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.098+0000 7fdba2a53640 1 -- 172.21.15.98:0/1000565100 >> v1:172.21.15.139:6789/0 conn(0x7fdb9c072820 legacy=0x7fdb9c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:49.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.098+0000 7fdba2a53640 1 -- 172.21.15.98:0/1000565100 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9c073180 con 0x7fdb9c06c3a0 2024-09-19T08:41:49.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.098+0000 7fdba2252640 1 --2- 172.21.15.98:0/1000565100 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb9c144d10 0x7fdb9c0722e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:41:49.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.102+0000 7fdba2a53640 1 --2- 172.21.15.98:0/1000565100 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb9c06c3a0 0x7fdb9c0cd250 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdb940097c0 tx=0x7fdb9401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9bf8b9995d1a8c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:49.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.102+0000 7fdba1250640 1 -- 172.21.15.98:0/1000565100 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb940b2070 con 0x7fdb9c06c3a0 2024-09-19T08:41:49.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.102+0000 7fdba1250640 1 -- 172.21.15.98:0/1000565100 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb940ad640 con 0x7fdb9c06c3a0 2024-09-19T08:41:49.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.102+0000 7fdba1250640 1 -- 172.21.15.98:0/1000565100 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb940ada10 con 0x7fdb9c06c3a0 2024-09-19T08:41:49.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.102+0000 7fdba44b6640 1 -- 172.21.15.98:0/1000565100 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb9c06c3a0 msgr2=0x7fdb9c0cd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:49.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.102+0000 7fdba44b6640 1 --2- 172.21.15.98:0/1000565100 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb9c06c3a0 0x7fdb9c0cd250 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdb940097c0 tx=0x7fdb9401c4a0 comp rx=0 tx=0).stop 2024-09-19T08:41:49.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.102+0000 7fdba44b6640 1 -- 172.21.15.98:0/1000565100 shutdown_connections 2024-09-19T08:41:49.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.102+0000 7fdba44b6640 1 --2- 172.21.15.98:0/1000565100 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb9c06c3a0 0x7fdb9c0cd250 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:49.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.102+0000 7fdba44b6640 1 --2- 172.21.15.98:0/1000565100 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb9c144d10 0x7fdb9c0722e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:49.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.102+0000 7fdba44b6640 1 -- 172.21.15.98:0/1000565100 >> 172.21.15.98:0/1000565100 conn(0x7fdb9c0b2010 msgr2=0x7fdb9c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:49.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.102+0000 7fdba44b6640 1 -- 172.21.15.98:0/1000565100 shutdown_connections 2024-09-19T08:41:49.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.102+0000 7fdba44b6640 1 -- 172.21.15.98:0/1000565100 wait complete. 2024-09-19T08:41:49.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.102+0000 7fdba44b6640 1 Processor -- start 2024-09-19T08:41:49.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.102+0000 7fdba44b6640 1 -- start start 2024-09-19T08:41:49.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.102+0000 7fdba44b6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb9c06c3a0 0x7fdb9c0ab530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:49.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.102+0000 7fdba44b6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb9c072820 0x7fdb9c149860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:49.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.102+0000 7fdba44b6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb9c144d10 0x7fdb9c1429b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:49.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.102+0000 7fdba44b6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c0ce120 con 0x7fdb9c06c3a0 2024-09-19T08:41:49.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.102+0000 7fdba44b6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c0cdfa0 con 0x7fdb9c144d10 2024-09-19T08:41:49.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.102+0000 7fdba44b6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdb9c0ce2a0 con 0x7fdb9c072820 2024-09-19T08:41:49.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.102+0000 7fdba2252640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb9c06c3a0 0x7fdb9c0ab530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:49.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.102+0000 7fdba2252640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb9c06c3a0 0x7fdb9c0ab530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46888/0 (socket says 172.21.15.98:46888) 2024-09-19T08:41:49.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.102+0000 7fdba2252640 1 -- 172.21.15.98:0/3212183871 learned_addr learned my addr 172.21.15.98:0/3212183871 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:49.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.106+0000 7fdba1a51640 1 --2- 172.21.15.98:0/3212183871 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb9c072820 0x7fdb9c149860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:49.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.106+0000 7fdba2a53640 1 --2- 172.21.15.98:0/3212183871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb9c144d10 0x7fdb9c1429b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:49.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.106+0000 7fdba2252640 1 -- 172.21.15.98:0/3212183871 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb9c072820 msgr2=0x7fdb9c149860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:49.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.106+0000 7fdba2252640 1 --2- 172.21.15.98:0/3212183871 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb9c072820 0x7fdb9c149860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:49.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.106+0000 7fdba2252640 1 -- 172.21.15.98:0/3212183871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb9c144d10 msgr2=0x7fdb9c1429b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:49.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.106+0000 7fdba2252640 1 --2- 172.21.15.98:0/3212183871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb9c144d10 0x7fdb9c1429b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:49.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.106+0000 7fdba2252640 1 -- 172.21.15.98:0/3212183871 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9c1f69d0 con 0x7fdb9c06c3a0 2024-09-19T08:41:49.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.106+0000 7fdba2252640 1 --2- 172.21.15.98:0/3212183871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb9c06c3a0 0x7fdb9c0ab530 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00efc0 tx=0x7fdb8c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:49.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.106+0000 7fdb9affd640 1 -- 172.21.15.98:0/3212183871 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c018020 con 0x7fdb9c06c3a0 2024-09-19T08:41:49.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.106+0000 7fdb9affd640 1 -- 172.21.15.98:0/3212183871 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb8c014680 con 0x7fdb9c06c3a0 2024-09-19T08:41:49.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.106+0000 7fdba44b6640 1 -- 172.21.15.98:0/3212183871 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb9c1f6cb0 con 0x7fdb9c06c3a0 2024-09-19T08:41:49.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.106+0000 7fdb9affd640 1 -- 172.21.15.98:0/3212183871 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c014a50 con 0x7fdb9c06c3a0 2024-09-19T08:41:49.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.106+0000 7fdba44b6640 1 -- 172.21.15.98:0/3212183871 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb9c1f7550 con 0x7fdb9c06c3a0 2024-09-19T08:41:49.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.106+0000 7fdba44b6640 1 -- 172.21.15.98:0/3212183871 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb9c1f70a0 con 0x7fdb9c06c3a0 2024-09-19T08:41:49.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.106+0000 7fdb9affd640 1 -- 172.21.15.98:0/3212183871 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fdb8c01b020 con 0x7fdb9c06c3a0 2024-09-19T08:41:49.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.106+0000 7fdb9affd640 1 --2- 172.21.15.98:0/3212183871 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdb88067a70 0x7fdb88069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:49.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.106+0000 7fdba1a51640 1 --2- 172.21.15.98:0/3212183871 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdb88067a70 0x7fdb88069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:49.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.106+0000 7fdb9affd640 1 -- 172.21.15.98:0/3212183871 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fdb8c086f20 con 0x7fdb9c06c3a0 2024-09-19T08:41:49.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.106+0000 7fdba1a51640 1 --2- 172.21.15.98:0/3212183871 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdb88067a70 0x7fdb88069f30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c15aeb0 tx=0x7fdb90009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:49.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.110+0000 7fdb9affd640 1 -- 172.21.15.98:0/3212183871 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fdb8c022020 con 0x7fdb9c06c3a0 2024-09-19T08:41:49.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.302+0000 7fdba44b6640 1 -- 172.21.15.98:0/3212183871 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fdb9c0cd250 con 0x7fdb9c06c3a0 2024-09-19T08:41:49.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.966+0000 7fdb9affd640 1 -- 172.21.15.98:0/3212183871 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v30) ==== 122+0+0 (secure 0 0 0) 0x7fdb8c010040 con 0x7fdb9c06c3a0 2024-09-19T08:41:49.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.970+0000 7fdba44b6640 1 -- 172.21.15.98:0/3212183871 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdb88067a70 msgr2=0x7fdb88069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:49.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.970+0000 7fdba44b6640 1 --2- 172.21.15.98:0/3212183871 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdb88067a70 0x7fdb88069f30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c15aeb0 tx=0x7fdb90009040 comp rx=0 tx=0).stop 2024-09-19T08:41:49.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.970+0000 7fdba44b6640 1 -- 172.21.15.98:0/3212183871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb9c06c3a0 msgr2=0x7fdb9c0ab530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:49.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.970+0000 7fdba44b6640 1 --2- 172.21.15.98:0/3212183871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb9c06c3a0 0x7fdb9c0ab530 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00efc0 tx=0x7fdb8c00c490 comp rx=0 tx=0).stop 2024-09-19T08:41:49.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.970+0000 7fdba44b6640 1 -- 172.21.15.98:0/3212183871 shutdown_connections 2024-09-19T08:41:49.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.970+0000 7fdba44b6640 1 --2- 172.21.15.98:0/3212183871 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fdb88067a70 0x7fdb88069f30 secure :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c15aeb0 tx=0x7fdb90009040 comp rx=0 tx=0).stop 2024-09-19T08:41:49.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.970+0000 7fdba44b6640 1 --2- 172.21.15.98:0/3212183871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdb9c144d10 0x7fdb9c1429b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:49.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.970+0000 7fdba44b6640 1 --2- 172.21.15.98:0/3212183871 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdb9c072820 0x7fdb9c149860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:49.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.970+0000 7fdba44b6640 1 --2- 172.21.15.98:0/3212183871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdb9c06c3a0 0x7fdb9c0ab530 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:49.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.970+0000 7fdba44b6640 1 -- 172.21.15.98:0/3212183871 >> 172.21.15.98:0/3212183871 conn(0x7fdb9c0b2010 msgr2=0x7fdb9c0b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:49.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.970+0000 7fdba44b6640 1 -- 172.21.15.98:0/3212183871 shutdown_connections 2024-09-19T08:41:49.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:49.970+0000 7fdba44b6640 1 -- 172.21.15.98:0/3212183871 wait complete. 2024-09-19T08:41:49.991 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-19T08:41:49.991 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:41:50.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.106+0000 7f6eabf80640 1 Processor -- start 2024-09-19T08:41:50.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.106+0000 7f6eabf80640 1 -- start start 2024-09-19T08:41:50.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.106+0000 7f6eabf80640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6ea406efa0 0x7f6ea40ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:50.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.106+0000 7f6eabf80640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ea40af4a0 con 0x7f6ea406f380 2024-09-19T08:41:50.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.106+0000 7f6eabf80640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ea40af780 con 0x7f6ea40aefd0 2024-09-19T08:41:50.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.106+0000 7f6eabf80640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f6ea415ed00 con 0x7f6ea406efa0 2024-09-19T08:41:50.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.106+0000 7f6ea951b640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f6ea406f380 0x7f6ea40ae8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40044/0 (socket says 172.21.15.98:40044) 2024-09-19T08:41:50.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.106+0000 7f6ea951b640 1 -- 172.21.15.98:0/3199218597 learned_addr learned my addr 172.21.15.98:0/3199218597 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:41:50.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.106+0000 7f6ea9d1c640 1 --2- 172.21.15.98:0/3199218597 >> v2:172.21.15.139:3300/0 conn(0x7f6ea406efa0 0x7f6ea40ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:50.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.106+0000 7f6ea8d1a640 1 -- 172.21.15.98:0/3199218597 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3449468369 0 0) 0x7f6ea40af4a0 con 0x7f6ea406f380 2024-09-19T08:41:50.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.106+0000 7f6ea8d1a640 1 -- 172.21.15.98:0/3199218597 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e90003530 con 0x7f6ea406f380 2024-09-19T08:41:50.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.106+0000 7f6ea8d1a640 1 -- 172.21.15.98:0/3199218597 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3878232599 0 0) 0x7f6e90003530 con 0x7f6ea406f380 2024-09-19T08:41:50.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.106+0000 7f6ea8d1a640 1 -- 172.21.15.98:0/3199218597 >> v2:172.21.15.139:3300/0 conn(0x7f6ea406efa0 msgr2=0x7f6ea40ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:50.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.106+0000 7f6ea8d1a640 1 --2- 172.21.15.98:0/3199218597 >> v2:172.21.15.139:3300/0 conn(0x7f6ea406efa0 0x7f6ea40ce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:50.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.106+0000 7f6ea8d1a640 1 -- 172.21.15.98:0/3199218597 >> v1:172.21.15.139:6789/0 conn(0x7f6ea40aefd0 legacy=0x7f6ea415b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:50.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.106+0000 7f6ea8d1a640 1 -- 172.21.15.98:0/3199218597 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea40afa60 con 0x7f6ea406f380 2024-09-19T08:41:50.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.106+0000 7f6ea8d1a640 1 -- 172.21.15.98:0/3199218597 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f6e94001680 con 0x7f6ea406f380 2024-09-19T08:41:50.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.106+0000 7f6ea8d1a640 1 -- 172.21.15.98:0/3199218597 >> v1:172.21.15.98:6789/0 conn(0x7f6ea406f380 legacy=0x7f6ea40ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:50.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.106+0000 7f6ea8d1a640 1 --2- 172.21.15.98:0/3199218597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e900039c0 0x7f6e90003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:50.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.106+0000 7f6ea8d1a640 1 --2- 172.21.15.98:0/3199218597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e90004810 0x7f6e90004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:50.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.106+0000 7f6ea8d1a640 1 --2- 172.21.15.98:0/3199218597 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6e900052f0 0x7f6e9000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:50.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.106+0000 7f6ea8d1a640 1 -- 172.21.15.98:0/3199218597 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea415ed00 con 0x7f6e900039c0 2024-09-19T08:41:50.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.106+0000 7f6ea8d1a640 1 -- 172.21.15.98:0/3199218597 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6e9000c1c0 con 0x7f6e90004810 2024-09-19T08:41:50.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.106+0000 7f6ea8d1a640 1 -- 172.21.15.98:0/3199218597 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6e9000c3c0 con 0x7f6e900052f0 2024-09-19T08:41:50.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.106+0000 7f6ea9d1c640 1 --2- 172.21.15.98:0/3199218597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e900039c0 0x7f6e90003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:50.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.106+0000 7f6ea951b640 1 --2- 172.21.15.98:0/3199218597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e90004810 0x7f6e90004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:50.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.106+0000 7f6ea9d1c640 1 -- 172.21.15.98:0/3199218597 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6e900052f0 msgr2=0x7f6e9000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:50.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.106+0000 7f6ea9d1c640 1 --2- 172.21.15.98:0/3199218597 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6e900052f0 0x7f6e9000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:50.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.106+0000 7f6ea9d1c640 1 -- 172.21.15.98:0/3199218597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e90004810 msgr2=0x7f6e90004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:50.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.106+0000 7f6ea9d1c640 1 --2- 172.21.15.98:0/3199218597 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e90004810 0x7f6e90004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:50.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.106+0000 7f6ea9d1c640 1 -- 172.21.15.98:0/3199218597 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e90003530 con 0x7f6e900039c0 2024-09-19T08:41:50.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.106+0000 7f6eaa51d640 1 -- 172.21.15.98:0/3199218597 reap_dead start 2024-09-19T08:41:50.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.110+0000 7f6ea9d1c640 1 --2- 172.21.15.98:0/3199218597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e900039c0 0x7f6e90003de0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0009ab0 tx=0x7f6ea001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb52aea9754a06d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:50.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.110+0000 7f6ea8d1a640 1 -- 172.21.15.98:0/3199218597 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea00096b0 con 0x7f6e900039c0 2024-09-19T08:41:50.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.110+0000 7f6ea8d1a640 1 -- 172.21.15.98:0/3199218597 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ea00a6430 con 0x7f6e900039c0 2024-09-19T08:41:50.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.110+0000 7f6ea8d1a640 1 -- 172.21.15.98:0/3199218597 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ea00a6820 con 0x7f6e900039c0 2024-09-19T08:41:50.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.110+0000 7f6eabf80640 1 -- 172.21.15.98:0/3199218597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e900039c0 msgr2=0x7f6e90003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:50.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.110+0000 7f6eabf80640 1 --2- 172.21.15.98:0/3199218597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e900039c0 0x7f6e90003de0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0009ab0 tx=0x7f6ea001ca30 comp rx=0 tx=0).stop 2024-09-19T08:41:50.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.110+0000 7f6eabf80640 1 -- 172.21.15.98:0/3199218597 shutdown_connections 2024-09-19T08:41:50.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.110+0000 7f6eabf80640 1 --2- 172.21.15.98:0/3199218597 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e900039c0 0x7f6e90003de0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:50.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.110+0000 7f6eabf80640 1 -- 172.21.15.98:0/3199218597 >> 172.21.15.98:0/3199218597 conn(0x7f6ea40d0980 msgr2=0x7f6ea40d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:50.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.110+0000 7f6eabf80640 1 -- 172.21.15.98:0/3199218597 shutdown_connections 2024-09-19T08:41:50.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.110+0000 7f6eabf80640 1 -- 172.21.15.98:0/3199218597 wait complete. 2024-09-19T08:41:50.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.110+0000 7f6eabf80640 1 Processor -- start 2024-09-19T08:41:50.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.110+0000 7f6eabf80640 1 -- start start 2024-09-19T08:41:50.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.114+0000 7f6eabf80640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e900039c0 0x7f6ea4150da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:50.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.114+0000 7f6eabf80640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ea40ca570 0x7f6ea4159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:50.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.114+0000 7f6eabf80640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ea40ca9f0 0x7f6ea4159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:50.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.114+0000 7f6eabf80640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea415b970 con 0x7f6ea40ca9f0 2024-09-19T08:41:50.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.114+0000 7f6eabf80640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea415b7f0 con 0x7f6e900039c0 2024-09-19T08:41:50.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.114+0000 7f6eabf80640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6ea415baf0 con 0x7f6ea40ca570 2024-09-19T08:41:50.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.114+0000 7f6eaa51d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ea40ca9f0 0x7f6ea4159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:50.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.114+0000 7f6eaa51d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ea40ca9f0 0x7f6ea4159e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46902/0 (socket says 172.21.15.98:46902) 2024-09-19T08:41:50.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.114+0000 7f6eaa51d640 1 -- 172.21.15.98:0/1055941760 learned_addr learned my addr 172.21.15.98:0/1055941760 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:50.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.114+0000 7f6ea9d1c640 1 --2- 172.21.15.98:0/1055941760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e900039c0 0x7f6ea4150da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:50.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.114+0000 7f6ea951b640 1 --2- 172.21.15.98:0/1055941760 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ea40ca570 0x7f6ea4159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:50.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.114+0000 7f6eaa51d640 1 -- 172.21.15.98:0/1055941760 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ea40ca570 msgr2=0x7f6ea4159920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:50.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.114+0000 7f6eaa51d640 1 --2- 172.21.15.98:0/1055941760 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ea40ca570 0x7f6ea4159920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:50.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.114+0000 7f6eaa51d640 1 -- 172.21.15.98:0/1055941760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e900039c0 msgr2=0x7f6ea4150da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:50.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.114+0000 7f6eaa51d640 1 --2- 172.21.15.98:0/1055941760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e900039c0 0x7f6ea4150da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:50.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.114+0000 7f6eaa51d640 1 -- 172.21.15.98:0/1055941760 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea40b3850 con 0x7f6ea40ca9f0 2024-09-19T08:41:50.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.114+0000 7f6ea9d1c640 1 --2- 172.21.15.98:0/1055941760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e900039c0 0x7f6ea4150da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:41:50.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.114+0000 7f6eaa51d640 1 --2- 172.21.15.98:0/1055941760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ea40ca9f0 0x7f6ea4159e60 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4068240 tx=0x7f6e9c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:50.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.114+0000 7f6e9a7fc640 1 -- 172.21.15.98:0/1055941760 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c005d50 con 0x7f6ea40ca9f0 2024-09-19T08:41:50.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.114+0000 7f6e9a7fc640 1 -- 172.21.15.98:0/1055941760 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e9c003980 con 0x7f6ea40ca9f0 2024-09-19T08:41:50.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.114+0000 7f6eabf80640 1 -- 172.21.15.98:0/1055941760 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ea41fb530 con 0x7f6ea40ca9f0 2024-09-19T08:41:50.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.114+0000 7f6e9a7fc640 1 -- 172.21.15.98:0/1055941760 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c003d70 con 0x7f6ea40ca9f0 2024-09-19T08:41:50.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.114+0000 7f6eabf80640 1 -- 172.21.15.98:0/1055941760 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ea41fbaf0 con 0x7f6ea40ca9f0 2024-09-19T08:41:50.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.114+0000 7f6eabf80640 1 -- 172.21.15.98:0/1055941760 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ea415be80 con 0x7f6ea40ca9f0 2024-09-19T08:41:50.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.114+0000 7f6e9a7fc640 1 -- 172.21.15.98:0/1055941760 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f6e9c00a640 con 0x7f6ea40ca9f0 2024-09-19T08:41:50.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.114+0000 7f6e9a7fc640 1 --2- 172.21.15.98:0/1055941760 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6e88067aa0 0x7f6e88069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:50.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.114+0000 7f6ea9d1c640 1 --2- 172.21.15.98:0/1055941760 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6e88067aa0 0x7f6e88069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:50.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.114+0000 7f6e9a7fc640 1 -- 172.21.15.98:0/1055941760 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f6e9c085310 con 0x7f6ea40ca9f0 2024-09-19T08:41:50.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.114+0000 7f6ea9d1c640 1 --2- 172.21.15.98:0/1055941760 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6e88067aa0 0x7f6e88069f60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6ea00ad000 tx=0x7f6ea00046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:50.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.118+0000 7f6e9a7fc640 1 -- 172.21.15.98:0/1055941760 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6e9c055bd0 con 0x7f6ea40ca9f0 2024-09-19T08:41:50.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.318+0000 7f6eabf80640 1 -- 172.21.15.98:0/1055941760 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ea4163cd0 con 0x7f6ea40ca9f0 2024-09-19T08:41:50.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.318+0000 7f6e9a7fc640 1 -- 172.21.15.98:0/1055941760 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+5229 (secure 0 0 0) 0x7f6e9c0554d0 con 0x7f6ea40ca9f0 2024-09-19T08:41:50.317 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:41:50.317 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":30,"btime":"2024-09-19T08:41:49:966866+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4880,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/1549394645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":1549394645},{"type":"v1","addr":"172.21.15.139:6834","nonce":1549394645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":4883,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1847525681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6833","nonce":1847525681},{"type":"v1","addr":"172.21.15.139:6835","nonce":1847525681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5745,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/1677812726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1677812726},{"type":"v1","addr":"172.21.15.98:6835","nonce":1677812726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:41:47.304224+0000","modified":"2024-09-19T08:41:49.306408+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/3548234111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3548234111},{"type":"v1","addr":"172.21.15.98:6837","nonce":3548234111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5748,"qdb_cluster":[5748]},"id":4}]} 2024-09-19T08:41:50.318 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 30 2024-09-19T08:41:50.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.318+0000 7f6eabf80640 1 -- 172.21.15.98:0/1055941760 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6e88067aa0 msgr2=0x7f6e88069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:50.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.318+0000 7f6eabf80640 1 --2- 172.21.15.98:0/1055941760 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6e88067aa0 0x7f6e88069f60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6ea00ad000 tx=0x7f6ea00046d0 comp rx=0 tx=0).stop 2024-09-19T08:41:50.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.318+0000 7f6eabf80640 1 -- 172.21.15.98:0/1055941760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ea40ca9f0 msgr2=0x7f6ea4159e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:50.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.318+0000 7f6eabf80640 1 --2- 172.21.15.98:0/1055941760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ea40ca9f0 0x7f6ea4159e60 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4068240 tx=0x7f6e9c00cb90 comp rx=0 tx=0).stop 2024-09-19T08:41:50.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.318+0000 7f6eabf80640 1 -- 172.21.15.98:0/1055941760 shutdown_connections 2024-09-19T08:41:50.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.318+0000 7f6eabf80640 1 --2- 172.21.15.98:0/1055941760 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6e88067aa0 0x7f6e88069f60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:50.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.318+0000 7f6eabf80640 1 --2- 172.21.15.98:0/1055941760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ea40ca9f0 0x7f6ea4159e60 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:50.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.318+0000 7f6eabf80640 1 --2- 172.21.15.98:0/1055941760 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ea40ca570 0x7f6ea4159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:50.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.318+0000 7f6eabf80640 1 --2- 172.21.15.98:0/1055941760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e900039c0 0x7f6ea4150da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:50.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.318+0000 7f6eabf80640 1 -- 172.21.15.98:0/1055941760 >> 172.21.15.98:0/1055941760 conn(0x7f6ea40d0980 msgr2=0x7f6ea40b1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:50.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.318+0000 7f6eabf80640 1 -- 172.21.15.98:0/1055941760 shutdown_connections 2024-09-19T08:41:50.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.318+0000 7f6eabf80640 1 -- 172.21.15.98:0/1055941760 wait complete. 2024-09-19T08:41:50.335 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:41:50.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.450+0000 7f0c55d1e640 1 Processor -- start 2024-09-19T08:41:50.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.450+0000 7f0c55d1e640 1 -- start start 2024-09-19T08:41:50.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.450+0000 7f0c55d1e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0c50140d30 0x7f0c5006d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:50.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.450+0000 7f0c55d1e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0c500b3a20 0x7f0c5006d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:50.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.450+0000 7f0c55d1e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c5015ea10 con 0x7f0c50141110 2024-09-19T08:41:50.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.450+0000 7f0c55d1e640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f0c5015e7c0 con 0x7f0c500b3a20 2024-09-19T08:41:50.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.450+0000 7f0c55d1e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f0c5015ecf0 con 0x7f0c50140d30 2024-09-19T08:41:50.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.450+0000 7f0c4ffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0c500b3a20 0x7f0c5006d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:50.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.450+0000 7f0c4ffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0c500b3a20 0x7f0c5006d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46916/0 (socket says 172.21.15.98:46916) 2024-09-19T08:41:50.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.450+0000 7f0c4ffff640 1 -- 172.21.15.98:0/3894523573 learned_addr learned my addr 172.21.15.98:0/3894523573 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:50.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.454+0000 7f0c4f7fe640 1 --2- 172.21.15.98:0/3894523573 >> v2:172.21.15.139:3300/0 conn(0x7f0c50140d30 0x7f0c5006d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:50.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.454+0000 7f0c4ffff640 1 -- 172.21.15.98:0/3894523573 >> v2:172.21.15.139:3300/0 conn(0x7f0c50140d30 msgr2=0x7f0c5006d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:50.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.454+0000 7f0c4ffff640 1 --2- 172.21.15.98:0/3894523573 >> v2:172.21.15.139:3300/0 conn(0x7f0c50140d30 0x7f0c5006d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:50.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.454+0000 7f0c4ffff640 1 -- 172.21.15.98:0/3894523573 >> v1:172.21.15.139:6789/0 conn(0x7f0c50141110 legacy=0x7f0c500b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:50.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.454+0000 7f0c4ffff640 1 -- 172.21.15.98:0/3894523573 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c5015efd0 con 0x7f0c500b3a20 2024-09-19T08:41:50.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.454+0000 7f0c4ffff640 1 --2- 172.21.15.98:0/3894523573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c500b3a20 0x7f0c5006d6b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0c5015e940 tx=0x7f0c4801c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c574121178ebff9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:50.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.454+0000 7f0c4e7fc640 1 -- 172.21.15.98:0/3894523573 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c480a5810 con 0x7f0c500b3a20 2024-09-19T08:41:50.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.454+0000 7f0c4e7fc640 1 -- 172.21.15.98:0/3894523573 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c480a59b0 con 0x7f0c500b3a20 2024-09-19T08:41:50.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.454+0000 7f0c4e7fc640 1 -- 172.21.15.98:0/3894523573 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c480a5d80 con 0x7f0c500b3a20 2024-09-19T08:41:50.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.454+0000 7f0c55d1e640 1 -- 172.21.15.98:0/3894523573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c500b3a20 msgr2=0x7f0c5006d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:50.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.454+0000 7f0c55d1e640 1 --2- 172.21.15.98:0/3894523573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c500b3a20 0x7f0c5006d6b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f0c5015e940 tx=0x7f0c4801c9b0 comp rx=0 tx=0).stop 2024-09-19T08:41:50.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.454+0000 7f0c55d1e640 1 -- 172.21.15.98:0/3894523573 shutdown_connections 2024-09-19T08:41:50.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.454+0000 7f0c55d1e640 1 --2- 172.21.15.98:0/3894523573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c500b3a20 0x7f0c5006d6b0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:50.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.454+0000 7f0c55d1e640 1 --2- 172.21.15.98:0/3894523573 >> v2:172.21.15.139:3300/0 conn(0x7f0c50140d30 0x7f0c5006d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:50.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.454+0000 7f0c55d1e640 1 -- 172.21.15.98:0/3894523573 >> 172.21.15.98:0/3894523573 conn(0x7f0c500b2010 msgr2=0x7f0c500b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:50.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.454+0000 7f0c55d1e640 1 -- 172.21.15.98:0/3894523573 shutdown_connections 2024-09-19T08:41:50.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.454+0000 7f0c55d1e640 1 -- 172.21.15.98:0/3894523573 wait complete. 2024-09-19T08:41:50.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.458+0000 7f0c55d1e640 1 Processor -- start 2024-09-19T08:41:50.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.458+0000 7f0c55d1e640 1 -- start start 2024-09-19T08:41:50.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.458+0000 7f0c55d1e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c500b3a20 0x7f0c5013c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:50.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.458+0000 7f0c55d1e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c50140d30 0x7f0c50154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:50.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.458+0000 7f0c55d1e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0c50141110 0x7f0c50154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:50.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.458+0000 7f0c55d1e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0c50165d10 con 0x7f0c500b3a20 2024-09-19T08:41:50.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.458+0000 7f0c55d1e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0c50165b90 con 0x7f0c50140d30 2024-09-19T08:41:50.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.458+0000 7f0c55d1e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0c50165e90 con 0x7f0c50141110 2024-09-19T08:41:50.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.458+0000 7f0c4f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c500b3a20 0x7f0c5013c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:50.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.458+0000 7f0c4f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c500b3a20 0x7f0c5013c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46932/0 (socket says 172.21.15.98:46932) 2024-09-19T08:41:50.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.458+0000 7f0c4f7fe640 1 -- 172.21.15.98:0/1945622433 learned_addr learned my addr 172.21.15.98:0/1945622433 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:50.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.458+0000 7f0c4effd640 1 --2- 172.21.15.98:0/1945622433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c50140d30 0x7f0c50154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:50.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.458+0000 7f0c4ffff640 1 --2- 172.21.15.98:0/1945622433 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0c50141110 0x7f0c50154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:50.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.458+0000 7f0c4f7fe640 1 -- 172.21.15.98:0/1945622433 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0c50141110 msgr2=0x7f0c50154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:50.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.458+0000 7f0c4f7fe640 1 --2- 172.21.15.98:0/1945622433 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0c50141110 0x7f0c50154680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:50.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.458+0000 7f0c4f7fe640 1 -- 172.21.15.98:0/1945622433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c50140d30 msgr2=0x7f0c50154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:50.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.458+0000 7f0c4f7fe640 1 --2- 172.21.15.98:0/1945622433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c50140d30 0x7f0c50154140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:50.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.458+0000 7f0c4f7fe640 1 -- 172.21.15.98:0/1945622433 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c5013cd70 con 0x7f0c500b3a20 2024-09-19T08:41:50.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.458+0000 7f0c4f7fe640 1 --2- 172.21.15.98:0/1945622433 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c500b3a20 0x7f0c5013c830 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0c4000d800 tx=0x7f0c4000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:50.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.458+0000 7f0c37fff640 1 -- 172.21.15.98:0/1945622433 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4000b6c0 con 0x7f0c500b3a20 2024-09-19T08:41:50.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.458+0000 7f0c37fff640 1 -- 172.21.15.98:0/1945622433 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c40019070 con 0x7f0c500b3a20 2024-09-19T08:41:50.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.458+0000 7f0c55d1e640 1 -- 172.21.15.98:0/1945622433 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c5015b250 con 0x7f0c500b3a20 2024-09-19T08:41:50.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.458+0000 7f0c37fff640 1 -- 172.21.15.98:0/1945622433 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4000ba60 con 0x7f0c500b3a20 2024-09-19T08:41:50.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.458+0000 7f0c55d1e640 1 -- 172.21.15.98:0/1945622433 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c5015a770 con 0x7f0c500b3a20 2024-09-19T08:41:50.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.458+0000 7f0c55d1e640 1 -- 172.21.15.98:0/1945622433 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c500b3310 con 0x7f0c500b3a20 2024-09-19T08:41:50.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.458+0000 7f0c37fff640 1 -- 172.21.15.98:0/1945622433 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f0c4000bc00 con 0x7f0c500b3a20 2024-09-19T08:41:50.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.458+0000 7f0c37fff640 1 --2- 172.21.15.98:0/1945622433 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0c3c067a70 0x7f0c3c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:50.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.462+0000 7f0c4effd640 1 --2- 172.21.15.98:0/1945622433 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0c3c067a70 0x7f0c3c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:50.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.462+0000 7f0c37fff640 1 -- 172.21.15.98:0/1945622433 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f0c40086460 con 0x7f0c500b3a20 2024-09-19T08:41:50.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.462+0000 7f0c4effd640 1 --2- 172.21.15.98:0/1945622433 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0c3c067a70 0x7f0c3c069f30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0c500cc620 tx=0x7f0c44008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:50.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.466+0000 7f0c37fff640 1 -- 172.21.15.98:0/1945622433 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f0c40056d20 con 0x7f0c500b3a20 2024-09-19T08:41:50.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.622+0000 7f0c55d1e640 1 -- 172.21.15.98:0/1945622433 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0c5015b7d0 con 0x7f0c500b3a20 2024-09-19T08:41:50.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.626+0000 7f0c37fff640 1 -- 172.21.15.98:0/1945622433 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+16725 (secure 0 0 0) 0x7f0c40056620 con 0x7f0c500b3a20 2024-09-19T08:41:50.625 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:41:50.625 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":57,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:41:48.960009+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:41:44.964773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-19T08:41:46.289740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:41:50.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.626+0000 7f0c55d1e640 1 -- 172.21.15.98:0/1945622433 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0c3c067a70 msgr2=0x7f0c3c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:50.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.626+0000 7f0c55d1e640 1 --2- 172.21.15.98:0/1945622433 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0c3c067a70 0x7f0c3c069f30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0c500cc620 tx=0x7f0c44008040 comp rx=0 tx=0).stop 2024-09-19T08:41:50.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.626+0000 7f0c55d1e640 1 -- 172.21.15.98:0/1945622433 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c500b3a20 msgr2=0x7f0c5013c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:50.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.626+0000 7f0c55d1e640 1 --2- 172.21.15.98:0/1945622433 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c500b3a20 0x7f0c5013c830 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0c4000d800 tx=0x7f0c4000dcc0 comp rx=0 tx=0).stop 2024-09-19T08:41:50.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.626+0000 7f0c55d1e640 1 -- 172.21.15.98:0/1945622433 shutdown_connections 2024-09-19T08:41:50.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.626+0000 7f0c55d1e640 1 --2- 172.21.15.98:0/1945622433 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0c3c067a70 0x7f0c3c069f30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:50.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.626+0000 7f0c55d1e640 1 --2- 172.21.15.98:0/1945622433 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0c50141110 0x7f0c50154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:50.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.626+0000 7f0c55d1e640 1 --2- 172.21.15.98:0/1945622433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c50140d30 0x7f0c50154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:50.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.626+0000 7f0c55d1e640 1 --2- 172.21.15.98:0/1945622433 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c500b3a20 0x7f0c5013c830 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:50.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.626+0000 7f0c55d1e640 1 -- 172.21.15.98:0/1945622433 >> 172.21.15.98:0/1945622433 conn(0x7f0c500b2010 msgr2=0x7f0c500ceb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:50.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.626+0000 7f0c55d1e640 1 -- 172.21.15.98:0/1945622433 shutdown_connections 2024-09-19T08:41:50.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.626+0000 7f0c55d1e640 1 -- 172.21.15.98:0/1945622433 wait complete. 2024-09-19T08:41:50.643 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T08:41:50.643 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:41:50.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.762+0000 7f3062285640 1 Processor -- start 2024-09-19T08:41:50.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.762+0000 7f3062285640 1 -- start start 2024-09-19T08:41:50.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.766+0000 7f3062285640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f305c158e40 0x7f305c06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:50.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.766+0000 7f3062285640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f305c0ce8b0 0x7f305c0b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:50.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.766+0000 7f3062285640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f305c15fcf0 con 0x7f305c06c1d0 2024-09-19T08:41:50.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.766+0000 7f3062285640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f305c06c640 con 0x7f305c0ce8b0 2024-09-19T08:41:50.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.766+0000 7f3062285640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f305c15ef30 con 0x7f305c158e40 2024-09-19T08:41:50.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.766+0000 7f3060822640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f305c0ce8b0 0x7f305c0b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:50.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.766+0000 7f3060822640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f305c0ce8b0 0x7f305c0b3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46948/0 (socket says 172.21.15.98:46948) 2024-09-19T08:41:50.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.766+0000 7f3060822640 1 -- 172.21.15.98:0/2659953485 learned_addr learned my addr 172.21.15.98:0/2659953485 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:50.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.766+0000 7f305bfff640 1 --2- 172.21.15.98:0/2659953485 >> v2:172.21.15.139:3300/0 conn(0x7f305c158e40 0x7f305c06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:50.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.766+0000 7f3060822640 1 -- 172.21.15.98:0/2659953485 >> v2:172.21.15.139:3300/0 conn(0x7f305c158e40 msgr2=0x7f305c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:50.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.766+0000 7f3060822640 1 --2- 172.21.15.98:0/2659953485 >> v2:172.21.15.139:3300/0 conn(0x7f305c158e40 0x7f305c06bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:50.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.766+0000 7f3060822640 1 -- 172.21.15.98:0/2659953485 >> v1:172.21.15.139:6789/0 conn(0x7f305c06c1d0 legacy=0x7f305c0ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:50.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.766+0000 7f3060822640 1 -- 172.21.15.98:0/2659953485 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f305c15f210 con 0x7f305c0ce8b0 2024-09-19T08:41:50.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.766+0000 7f3060822640 1 --2- 172.21.15.98:0/2659953485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f305c0ce8b0 0x7f305c0b3560 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f305c06e1e0 tx=0x7f305401f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ce1787cf9ed249 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:50.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.766+0000 7f305affd640 1 -- 172.21.15.98:0/2659953485 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30540b4070 con 0x7f305c0ce8b0 2024-09-19T08:41:50.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.766+0000 7f305affd640 1 -- 172.21.15.98:0/2659953485 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30540af400 con 0x7f305c0ce8b0 2024-09-19T08:41:50.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.766+0000 7f305affd640 1 -- 172.21.15.98:0/2659953485 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30540af7d0 con 0x7f305c0ce8b0 2024-09-19T08:41:50.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.770+0000 7f3062285640 1 -- 172.21.15.98:0/2659953485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f305c0ce8b0 msgr2=0x7f305c0b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:50.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.770+0000 7f3062285640 1 --2- 172.21.15.98:0/2659953485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f305c0ce8b0 0x7f305c0b3560 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f305c06e1e0 tx=0x7f305401f1e0 comp rx=0 tx=0).stop 2024-09-19T08:41:50.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.770+0000 7f3062285640 1 -- 172.21.15.98:0/2659953485 shutdown_connections 2024-09-19T08:41:50.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.770+0000 7f3062285640 1 --2- 172.21.15.98:0/2659953485 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f305c0ce8b0 0x7f305c0b3560 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:50.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.770+0000 7f3062285640 1 --2- 172.21.15.98:0/2659953485 >> v2:172.21.15.139:3300/0 conn(0x7f305c158e40 0x7f305c06bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:50.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.770+0000 7f3062285640 1 -- 172.21.15.98:0/2659953485 >> 172.21.15.98:0/2659953485 conn(0x7f305c0b2010 msgr2=0x7f305c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:50.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.770+0000 7f3062285640 1 -- 172.21.15.98:0/2659953485 shutdown_connections 2024-09-19T08:41:50.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.770+0000 7f3062285640 1 -- 172.21.15.98:0/2659953485 wait complete. 2024-09-19T08:41:50.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.770+0000 7f3062285640 1 Processor -- start 2024-09-19T08:41:50.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.770+0000 7f3062285640 1 -- start start 2024-09-19T08:41:50.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.770+0000 7f3062285640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f305c06c1d0 0x7f305c14bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:50.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.770+0000 7f3062285640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f305c0ce8b0 0x7f305c071890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:50.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.770+0000 7f3062285640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f305c158e40 0x7f305c0b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:50.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.770+0000 7f3062285640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f305c159220 con 0x7f305c0ce8b0 2024-09-19T08:41:50.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.770+0000 7f3062285640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f305c15f7d0 con 0x7f305c158e40 2024-09-19T08:41:50.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.770+0000 7f3062285640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f305c0cee20 con 0x7f305c06c1d0 2024-09-19T08:41:50.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.770+0000 7f305b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f305c0ce8b0 0x7f305c071890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:50.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.770+0000 7f305b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f305c0ce8b0 0x7f305c071890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46950/0 (socket says 172.21.15.98:46950) 2024-09-19T08:41:50.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.770+0000 7f305b7fe640 1 -- 172.21.15.98:0/4268944722 learned_addr learned my addr 172.21.15.98:0/4268944722 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:50.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.770+0000 7f305bfff640 1 --2- 172.21.15.98:0/4268944722 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f305c06c1d0 0x7f305c14bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:50.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.770+0000 7f3060822640 1 --2- 172.21.15.98:0/4268944722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f305c158e40 0x7f305c0b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:50.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.770+0000 7f305b7fe640 1 -- 172.21.15.98:0/4268944722 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f305c06c1d0 msgr2=0x7f305c14bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:50.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.770+0000 7f305b7fe640 1 --2- 172.21.15.98:0/4268944722 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f305c06c1d0 0x7f305c14bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:50.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.770+0000 7f305b7fe640 1 -- 172.21.15.98:0/4268944722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f305c158e40 msgr2=0x7f305c0b0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:50.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.770+0000 7f305b7fe640 1 --2- 172.21.15.98:0/4268944722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f305c158e40 0x7f305c0b0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:50.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.770+0000 7f305b7fe640 1 -- 172.21.15.98:0/4268944722 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f305c072380 con 0x7f305c0ce8b0 2024-09-19T08:41:50.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.770+0000 7f305b7fe640 1 --2- 172.21.15.98:0/4268944722 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f305c0ce8b0 0x7f305c071890 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f305000c530 tx=0x7f305000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:50.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.770+0000 7f3058ff9640 1 -- 172.21.15.98:0/4268944722 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3050016020 con 0x7f305c0ce8b0 2024-09-19T08:41:50.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.770+0000 7f3058ff9640 1 -- 172.21.15.98:0/4268944722 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3050005220 con 0x7f305c0ce8b0 2024-09-19T08:41:50.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.770+0000 7f3062285640 1 -- 172.21.15.98:0/4268944722 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f305c072660 con 0x7f305c0ce8b0 2024-09-19T08:41:50.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.770+0000 7f3058ff9640 1 -- 172.21.15.98:0/4268944722 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3050007030 con 0x7f305c0ce8b0 2024-09-19T08:41:50.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.770+0000 7f3062285640 1 -- 172.21.15.98:0/4268944722 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f305c15a8e0 con 0x7f305c0ce8b0 2024-09-19T08:41:50.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.774+0000 7f3062285640 1 -- 172.21.15.98:0/4268944722 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f305c072830 con 0x7f305c0ce8b0 2024-09-19T08:41:50.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.774+0000 7f3058ff9640 1 -- 172.21.15.98:0/4268944722 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f30500071d0 con 0x7f305c0ce8b0 2024-09-19T08:41:50.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.774+0000 7f3058ff9640 1 --2- 172.21.15.98:0/4268944722 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3048067a70 0x7f3048069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:50.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.774+0000 7f3058ff9640 1 -- 172.21.15.98:0/4268944722 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f30500858d0 con 0x7f305c0ce8b0 2024-09-19T08:41:50.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.774+0000 7f305bfff640 1 --2- 172.21.15.98:0/4268944722 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3048067a70 0x7f3048069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:50.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.774+0000 7f305bfff640 1 --2- 172.21.15.98:0/4268944722 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3048067a70 0x7f3048069f30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f304c00b930 tx=0x7f304c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:50.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.778+0000 7f3058ff9640 1 -- 172.21.15.98:0/4268944722 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f305c072830 con 0x7f305c0ce8b0 2024-09-19T08:41:50.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.938+0000 7f3062285640 1 -- 172.21.15.98:0/4268944722 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f305c06bc90 con 0x7f3048067a70 2024-09-19T08:41:50.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.954+0000 7f3058ff9640 1 -- 172.21.15.98:0/4268944722 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251655 (secure 0 0 0) 0x7f305c06bc90 con 0x7f3048067a70 2024-09-19T08:41:50.958 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:41:50.960 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:41:50.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.958+0000 7f3062285640 1 -- 172.21.15.98:0/4268944722 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3048067a70 msgr2=0x7f3048069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:50.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.958+0000 7f3062285640 1 --2- 172.21.15.98:0/4268944722 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3048067a70 0x7f3048069f30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f304c00b930 tx=0x7f304c008040 comp rx=0 tx=0).stop 2024-09-19T08:41:50.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.958+0000 7f3062285640 1 -- 172.21.15.98:0/4268944722 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f305c0ce8b0 msgr2=0x7f305c071890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:50.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.958+0000 7f3062285640 1 --2- 172.21.15.98:0/4268944722 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f305c0ce8b0 0x7f305c071890 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f305000c530 tx=0x7f305000c9f0 comp rx=0 tx=0).stop 2024-09-19T08:41:50.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.958+0000 7f3062285640 1 -- 172.21.15.98:0/4268944722 shutdown_connections 2024-09-19T08:41:50.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.958+0000 7f3062285640 1 --2- 172.21.15.98:0/4268944722 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3048067a70 0x7f3048069f30 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:50.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.958+0000 7f3062285640 1 --2- 172.21.15.98:0/4268944722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f305c158e40 0x7f305c0b0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:50.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.958+0000 7f3062285640 1 --2- 172.21.15.98:0/4268944722 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f305c0ce8b0 0x7f305c071890 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:50.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.958+0000 7f3062285640 1 --2- 172.21.15.98:0/4268944722 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f305c06c1d0 0x7f305c14bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:50.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.958+0000 7f3062285640 1 -- 172.21.15.98:0/4268944722 >> 172.21.15.98:0/4268944722 conn(0x7f305c0b2010 msgr2=0x7f305c0ce1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:50.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.962+0000 7f3062285640 1 -- 172.21.15.98:0/4268944722 shutdown_connections 2024-09-19T08:41:50.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:50.962+0000 7f3062285640 1 -- 172.21.15.98:0/4268944722 wait complete. 2024-09-19T08:41:50.978 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-09-19T08:41:50.096986+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":101,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":247596,"kb_used_data":6376,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754727124,"statfs":{"total":773094113280,"available":772840574976,"internally_reserved":0,"allocated":6529024,"data_stored":3357678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.175662"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.985587+0000","last_change":"2024-09-19T08:41:47.980122+0000","last_active":"2024-09-19T08:41:47.985587+0000","last_peered":"2024-09-19T08:41:47.985587+0000","last_clean":"2024-09-19T08:41:47.985587+0000","last_became_active":"2024-09-19T08:41:47.976398+0000","last_became_peered":"2024-09-19T08:41:47.976398+0000","last_unstale":"2024-09-19T08:41:47.985587+0000","last_undegraded":"2024-09-19T08:41:47.985587+0000","last_fullsized":"2024-09-19T08:41:47.985587+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:25.855696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.976086+0000","last_change":"2024-09-19T08:41:47.973338+0000","last_active":"2024-09-19T08:41:47.976086+0000","last_peered":"2024-09-19T08:41:47.976086+0000","last_clean":"2024-09-19T08:41:47.976086+0000","last_became_active":"2024-09-19T08:41:47.973065+0000","last_became_peered":"2024-09-19T08:41:47.973065+0000","last_unstale":"2024-09-19T08:41:47.976086+0000","last_undegraded":"2024-09-19T08:41:47.976086+0000","last_fullsized":"2024-09-19T08:41:47.976086+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:07.544296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.982165+0000","last_change":"2024-09-19T08:41:47.966283+0000","last_active":"2024-09-19T08:41:47.982165+0000","last_peered":"2024-09-19T08:41:47.982165+0000","last_clean":"2024-09-19T08:41:47.982165+0000","last_became_active":"2024-09-19T08:41:47.965563+0000","last_became_peered":"2024-09-19T08:41:47.965563+0000","last_unstale":"2024-09-19T08:41:47.982165+0000","last_undegraded":"2024-09-19T08:41:47.982165+0000","last_fullsized":"2024-09-19T08:41:47.982165+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:39:24.749819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.993880+0000","last_change":"2024-09-19T08:41:47.978137+0000","last_active":"2024-09-19T08:41:47.993880+0000","last_peered":"2024-09-19T08:41:47.993880+0000","last_clean":"2024-09-19T08:41:47.993880+0000","last_became_active":"2024-09-19T08:41:47.976050+0000","last_became_peered":"2024-09-19T08:41:47.976050+0000","last_unstale":"2024-09-19T08:41:47.993880+0000","last_undegraded":"2024-09-19T08:41:47.993880+0000","last_fullsized":"2024-09-19T08:41:47.993880+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:56:44.612767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.994306+0000","last_change":"2024-09-19T08:41:47.993040+0000","last_active":"2024-09-19T08:41:47.994306+0000","last_peered":"2024-09-19T08:41:47.994306+0000","last_clean":"2024-09-19T08:41:47.994306+0000","last_became_active":"2024-09-19T08:41:47.992767+0000","last_became_peered":"2024-09-19T08:41:47.992767+0000","last_unstale":"2024-09-19T08:41:47.994306+0000","last_undegraded":"2024-09-19T08:41:47.994306+0000","last_fullsized":"2024-09-19T08:41:47.994306+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:54.206986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.964412+0000","last_change":"2024-09-19T08:41:47.961838+0000","last_active":"2024-09-19T08:41:47.964412+0000","last_peered":"2024-09-19T08:41:47.964412+0000","last_clean":"2024-09-19T08:41:47.964412+0000","last_became_active":"2024-09-19T08:41:47.961116+0000","last_became_peered":"2024-09-19T08:41:47.961116+0000","last_unstale":"2024-09-19T08:41:47.964412+0000","last_undegraded":"2024-09-19T08:41:47.964412+0000","last_fullsized":"2024-09-19T08:41:47.964412+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:41.615523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971387+0000","last_change":"2024-09-19T08:41:47.966503+0000","last_active":"2024-09-19T08:41:48.971387+0000","last_peered":"2024-09-19T08:41:48.971387+0000","last_clean":"2024-09-19T08:41:48.971387+0000","last_became_active":"2024-09-19T08:41:47.965805+0000","last_became_peered":"2024-09-19T08:41:47.965805+0000","last_unstale":"2024-09-19T08:41:48.971387+0000","last_undegraded":"2024-09-19T08:41:48.971387+0000","last_fullsized":"2024-09-19T08:41:48.971387+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:41.615523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.971097+0000","last_change":"2024-09-19T08:41:47.969144+0000","last_active":"2024-09-19T08:41:47.971097+0000","last_peered":"2024-09-19T08:41:47.971097+0000","last_clean":"2024-09-19T08:41:47.971097+0000","last_became_active":"2024-09-19T08:41:47.968766+0000","last_became_peered":"2024-09-19T08:41:47.968766+0000","last_unstale":"2024-09-19T08:41:47.971097+0000","last_undegraded":"2024-09-19T08:41:47.971097+0000","last_fullsized":"2024-09-19T08:41:47.971097+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:20:19.600052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971951+0000","last_change":"2024-09-19T08:41:47.954679+0000","last_active":"2024-09-19T08:41:48.971951+0000","last_peered":"2024-09-19T08:41:48.971951+0000","last_clean":"2024-09-19T08:41:48.971951+0000","last_became_active":"2024-09-19T08:41:47.951685+0000","last_became_peered":"2024-09-19T08:41:47.951685+0000","last_unstale":"2024-09-19T08:41:48.971951+0000","last_undegraded":"2024-09-19T08:41:48.971951+0000","last_fullsized":"2024-09-19T08:41:48.971951+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:17.405379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971133+0000","last_change":"2024-09-19T08:41:47.977064+0000","last_active":"2024-09-19T08:41:48.971133+0000","last_peered":"2024-09-19T08:41:48.971133+0000","last_clean":"2024-09-19T08:41:48.971133+0000","last_became_active":"2024-09-19T08:41:47.976747+0000","last_became_peered":"2024-09-19T08:41:47.976747+0000","last_unstale":"2024-09-19T08:41:48.971133+0000","last_undegraded":"2024-09-19T08:41:48.971133+0000","last_fullsized":"2024-09-19T08:41:48.971133+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:37.366531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971332+0000","last_change":"2024-09-19T08:41:47.971592+0000","last_active":"2024-09-19T08:41:48.971332+0000","last_peered":"2024-09-19T08:41:48.971332+0000","last_clean":"2024-09-19T08:41:48.971332+0000","last_became_active":"2024-09-19T08:41:47.969075+0000","last_became_peered":"2024-09-19T08:41:47.969075+0000","last_unstale":"2024-09-19T08:41:48.971332+0000","last_undegraded":"2024-09-19T08:41:48.971332+0000","last_fullsized":"2024-09-19T08:41:48.971332+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:29.817497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.992937+0000","last_change":"2024-09-19T08:41:47.968996+0000","last_active":"2024-09-19T08:41:47.992937+0000","last_peered":"2024-09-19T08:41:47.992937+0000","last_clean":"2024-09-19T08:41:47.992937+0000","last_became_active":"2024-09-19T08:41:47.968254+0000","last_became_peered":"2024-09-19T08:41:47.968254+0000","last_unstale":"2024-09-19T08:41:47.992937+0000","last_undegraded":"2024-09-19T08:41:47.992937+0000","last_fullsized":"2024-09-19T08:41:47.992937+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:12.552026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-19T08:41:46.952023+0000","last_change":"2024-09-19T08:41:46.952023+0000","last_active":"2024-09-19T08:41:46.920839+0000","last_peered":"2024-09-19T08:41:46.920839+0000","last_clean":"2024-09-19T08:41:46.920839+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:41:46.952023+0000","last_undegraded":"2024-09-19T08:41:46.952023+0000","last_fullsized":"2024-09-19T08:41:46.952023+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971528+0000","last_change":"2024-09-19T08:41:47.962145+0000","last_active":"2024-09-19T08:41:48.971528+0000","last_peered":"2024-09-19T08:41:48.971528+0000","last_clean":"2024-09-19T08:41:48.971528+0000","last_became_active":"2024-09-19T08:41:47.961449+0000","last_became_peered":"2024-09-19T08:41:47.961449+0000","last_unstale":"2024-09-19T08:41:48.971528+0000","last_undegraded":"2024-09-19T08:41:48.971528+0000","last_fullsized":"2024-09-19T08:41:48.971528+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:09.481675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.972047+0000","last_change":"2024-09-19T08:41:47.960027+0000","last_active":"2024-09-19T08:41:48.972047+0000","last_peered":"2024-09-19T08:41:48.972047+0000","last_clean":"2024-09-19T08:41:48.972047+0000","last_became_active":"2024-09-19T08:41:47.959242+0000","last_became_peered":"2024-09-19T08:41:47.959242+0000","last_unstale":"2024-09-19T08:41:48.972047+0000","last_undegraded":"2024-09-19T08:41:48.972047+0000","last_fullsized":"2024-09-19T08:41:48.972047+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:56:44.612767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.984338+0000","last_change":"2024-09-19T08:41:47.979371+0000","last_active":"2024-09-19T08:41:47.984338+0000","last_peered":"2024-09-19T08:41:47.984338+0000","last_clean":"2024-09-19T08:41:47.984338+0000","last_became_active":"2024-09-19T08:41:47.978819+0000","last_became_peered":"2024-09-19T08:41:47.978819+0000","last_unstale":"2024-09-19T08:41:47.984338+0000","last_undegraded":"2024-09-19T08:41:47.984338+0000","last_fullsized":"2024-09-19T08:41:47.984338+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:34.451800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.981562+0000","last_change":"2024-09-19T08:41:47.977880+0000","last_active":"2024-09-19T08:41:47.981562+0000","last_peered":"2024-09-19T08:41:47.981562+0000","last_clean":"2024-09-19T08:41:47.981562+0000","last_became_active":"2024-09-19T08:41:47.977590+0000","last_became_peered":"2024-09-19T08:41:47.977590+0000","last_unstale":"2024-09-19T08:41:47.981562+0000","last_undegraded":"2024-09-19T08:41:47.981562+0000","last_fullsized":"2024-09-19T08:41:47.981562+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:43:07.334696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.982300+0000","last_change":"2024-09-19T08:41:47.968677+0000","last_active":"2024-09-19T08:41:47.982300+0000","last_peered":"2024-09-19T08:41:47.982300+0000","last_clean":"2024-09-19T08:41:47.982300+0000","last_became_active":"2024-09-19T08:41:47.967937+0000","last_became_peered":"2024-09-19T08:41:47.967937+0000","last_unstale":"2024-09-19T08:41:47.982300+0000","last_undegraded":"2024-09-19T08:41:47.982300+0000","last_fullsized":"2024-09-19T08:41:47.982300+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:32:44.673319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.981344+0000","last_change":"2024-09-19T08:41:47.975831+0000","last_active":"2024-09-19T08:41:47.981344+0000","last_peered":"2024-09-19T08:41:47.981344+0000","last_clean":"2024-09-19T08:41:47.981344+0000","last_became_active":"2024-09-19T08:41:47.975518+0000","last_became_peered":"2024-09-19T08:41:47.975518+0000","last_unstale":"2024-09-19T08:41:47.981344+0000","last_undegraded":"2024-09-19T08:41:47.981344+0000","last_fullsized":"2024-09-19T08:41:47.981344+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:23:47.530098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.976135+0000","last_change":"2024-09-19T08:41:47.973832+0000","last_active":"2024-09-19T08:41:47.976135+0000","last_peered":"2024-09-19T08:41:47.976135+0000","last_clean":"2024-09-19T08:41:47.976135+0000","last_became_active":"2024-09-19T08:41:47.973516+0000","last_became_peered":"2024-09-19T08:41:47.973516+0000","last_unstale":"2024-09-19T08:41:47.976135+0000","last_undegraded":"2024-09-19T08:41:47.976135+0000","last_fullsized":"2024-09-19T08:41:47.976135+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:43:59.145791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.994338+0000","last_change":"2024-09-19T08:41:47.979480+0000","last_active":"2024-09-19T08:41:47.994338+0000","last_peered":"2024-09-19T08:41:47.994338+0000","last_clean":"2024-09-19T08:41:47.994338+0000","last_became_active":"2024-09-19T08:41:47.977765+0000","last_became_peered":"2024-09-19T08:41:47.977765+0000","last_unstale":"2024-09-19T08:41:47.994338+0000","last_undegraded":"2024-09-19T08:41:47.994338+0000","last_fullsized":"2024-09-19T08:41:47.994338+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:41.615523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.996753+0000","last_change":"2024-09-19T08:41:47.981514+0000","last_active":"2024-09-19T08:41:47.996753+0000","last_peered":"2024-09-19T08:41:47.996753+0000","last_clean":"2024-09-19T08:41:47.996753+0000","last_became_active":"2024-09-19T08:41:47.981044+0000","last_became_peered":"2024-09-19T08:41:47.981044+0000","last_unstale":"2024-09-19T08:41:47.996753+0000","last_undegraded":"2024-09-19T08:41:47.996753+0000","last_fullsized":"2024-09-19T08:41:47.996753+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:07.796396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.993777+0000","last_change":"2024-09-19T08:41:47.970880+0000","last_active":"2024-09-19T08:41:47.993777+0000","last_peered":"2024-09-19T08:41:47.993777+0000","last_clean":"2024-09-19T08:41:47.993777+0000","last_became_active":"2024-09-19T08:41:47.970145+0000","last_became_peered":"2024-09-19T08:41:47.970145+0000","last_unstale":"2024-09-19T08:41:47.993777+0000","last_undegraded":"2024-09-19T08:41:47.993777+0000","last_fullsized":"2024-09-19T08:41:47.993777+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:52:41.224293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-19T08:41:46.949524+0000","last_change":"2024-09-19T08:41:46.949524+0000","last_active":"2024-09-19T08:41:46.920839+0000","last_peered":"2024-09-19T08:41:46.920839+0000","last_clean":"2024-09-19T08:41:46.920839+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:41:46.949524+0000","last_undegraded":"2024-09-19T08:41:46.949524+0000","last_fullsized":"2024-09-19T08:41:46.949524+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.985564+0000","last_change":"2024-09-19T08:41:47.977025+0000","last_active":"2024-09-19T08:41:47.985564+0000","last_peered":"2024-09-19T08:41:47.985564+0000","last_clean":"2024-09-19T08:41:47.985564+0000","last_became_active":"2024-09-19T08:41:47.975707+0000","last_became_peered":"2024-09-19T08:41:47.975707+0000","last_unstale":"2024-09-19T08:41:47.985564+0000","last_undegraded":"2024-09-19T08:41:47.985564+0000","last_fullsized":"2024-09-19T08:41:47.985564+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:17.908127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.990912+0000","last_change":"2024-09-19T08:41:47.966128+0000","last_active":"2024-09-19T08:41:47.990912+0000","last_peered":"2024-09-19T08:41:47.990912+0000","last_clean":"2024-09-19T08:41:47.990912+0000","last_became_active":"2024-09-19T08:41:47.965403+0000","last_became_peered":"2024-09-19T08:41:47.965403+0000","last_unstale":"2024-09-19T08:41:47.990912+0000","last_undegraded":"2024-09-19T08:41:47.990912+0000","last_fullsized":"2024-09-19T08:41:47.990912+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:43.544741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.981730+0000","last_change":"2024-09-19T08:41:47.979320+0000","last_active":"2024-09-19T08:41:47.981730+0000","last_peered":"2024-09-19T08:41:47.981730+0000","last_clean":"2024-09-19T08:41:47.981730+0000","last_became_active":"2024-09-19T08:41:47.979010+0000","last_became_peered":"2024-09-19T08:41:47.979010+0000","last_unstale":"2024-09-19T08:41:47.981730+0000","last_undegraded":"2024-09-19T08:41:47.981730+0000","last_fullsized":"2024-09-19T08:41:47.981730+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:44:04.680315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.975042+0000","last_change":"2024-09-19T08:41:47.966338+0000","last_active":"2024-09-19T08:41:47.975042+0000","last_peered":"2024-09-19T08:41:47.975042+0000","last_clean":"2024-09-19T08:41:47.975042+0000","last_became_active":"2024-09-19T08:41:47.966024+0000","last_became_peered":"2024-09-19T08:41:47.966024+0000","last_unstale":"2024-09-19T08:41:47.975042+0000","last_undegraded":"2024-09-19T08:41:47.975042+0000","last_fullsized":"2024-09-19T08:41:47.975042+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:45.927550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.997659+0000","last_change":"2024-09-19T08:41:47.996502+0000","last_active":"2024-09-19T08:41:47.997659+0000","last_peered":"2024-09-19T08:41:47.997659+0000","last_clean":"2024-09-19T08:41:47.997659+0000","last_became_active":"2024-09-19T08:41:47.996162+0000","last_became_peered":"2024-09-19T08:41:47.996162+0000","last_unstale":"2024-09-19T08:41:47.997659+0000","last_undegraded":"2024-09-19T08:41:47.997659+0000","last_fullsized":"2024-09-19T08:41:47.997659+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:13.341860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.982174+0000","last_change":"2024-09-19T08:41:47.966508+0000","last_active":"2024-09-19T08:41:47.982174+0000","last_peered":"2024-09-19T08:41:47.982174+0000","last_clean":"2024-09-19T08:41:47.982174+0000","last_became_active":"2024-09-19T08:41:47.962098+0000","last_became_peered":"2024-09-19T08:41:47.962098+0000","last_unstale":"2024-09-19T08:41:47.982174+0000","last_undegraded":"2024-09-19T08:41:47.982174+0000","last_fullsized":"2024-09-19T08:41:47.982174+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:23.999718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.981287+0000","last_change":"2024-09-19T08:41:47.974969+0000","last_active":"2024-09-19T08:41:47.981287+0000","last_peered":"2024-09-19T08:41:47.981287+0000","last_clean":"2024-09-19T08:41:47.981287+0000","last_became_active":"2024-09-19T08:41:47.974440+0000","last_became_peered":"2024-09-19T08:41:47.974440+0000","last_unstale":"2024-09-19T08:41:47.981287+0000","last_undegraded":"2024-09-19T08:41:47.981287+0000","last_fullsized":"2024-09-19T08:41:47.981287+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:35:19.862729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.978537+0000","last_change":"2024-09-19T08:41:47.972260+0000","last_active":"2024-09-19T08:41:47.978537+0000","last_peered":"2024-09-19T08:41:47.978537+0000","last_clean":"2024-09-19T08:41:47.978537+0000","last_became_active":"2024-09-19T08:41:47.971683+0000","last_became_peered":"2024-09-19T08:41:47.971683+0000","last_unstale":"2024-09-19T08:41:47.978537+0000","last_undegraded":"2024-09-19T08:41:47.978537+0000","last_fullsized":"2024-09-19T08:41:47.978537+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:56:44.612767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.964666+0000","last_change":"2024-09-19T08:41:47.962374+0000","last_active":"2024-09-19T08:41:47.964666+0000","last_peered":"2024-09-19T08:41:47.964666+0000","last_clean":"2024-09-19T08:41:47.964666+0000","last_became_active":"2024-09-19T08:41:47.961305+0000","last_became_peered":"2024-09-19T08:41:47.961305+0000","last_unstale":"2024-09-19T08:41:47.964666+0000","last_undegraded":"2024-09-19T08:41:47.964666+0000","last_fullsized":"2024-09-19T08:41:47.964666+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:09.481675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"56'4","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.001343+0000","last_change":"2024-09-19T08:41:46.976847+0000","last_active":"2024-09-19T08:41:48.001343+0000","last_peered":"2024-09-19T08:41:48.001343+0000","last_clean":"2024-09-19T08:41:48.001343+0000","last_became_active":"2024-09-19T08:41:46.976405+0000","last_became_peered":"2024-09-19T08:41:46.976405+0000","last_unstale":"2024-09-19T08:41:48.001343+0000","last_undegraded":"2024-09-19T08:41:48.001343+0000","last_fullsized":"2024-09-19T08:41:48.001343+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:52:40.220622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971359+0000","last_change":"2024-09-19T08:41:47.971091+0000","last_active":"2024-09-19T08:41:48.971359+0000","last_peered":"2024-09-19T08:41:48.971359+0000","last_clean":"2024-09-19T08:41:48.971359+0000","last_became_active":"2024-09-19T08:41:47.958633+0000","last_became_peered":"2024-09-19T08:41:47.958633+0000","last_unstale":"2024-09-19T08:41:48.971359+0000","last_undegraded":"2024-09-19T08:41:48.971359+0000","last_fullsized":"2024-09-19T08:41:48.971359+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:38.634694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"56'1","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.997195+0000","last_change":"2024-09-19T08:41:46.953976+0000","last_active":"2024-09-19T08:41:47.997195+0000","last_peered":"2024-09-19T08:41:47.997195+0000","last_clean":"2024-09-19T08:41:47.997195+0000","last_became_active":"2024-09-19T08:41:46.953613+0000","last_became_peered":"2024-09-19T08:41:46.953613+0000","last_unstale":"2024-09-19T08:41:47.997195+0000","last_undegraded":"2024-09-19T08:41:47.997195+0000","last_fullsized":"2024-09-19T08:41:47.997195+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:24.976526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.964324+0000","last_change":"2024-09-19T08:41:47.961793+0000","last_active":"2024-09-19T08:41:47.964324+0000","last_peered":"2024-09-19T08:41:47.964324+0000","last_clean":"2024-09-19T08:41:47.964324+0000","last_became_active":"2024-09-19T08:41:47.961008+0000","last_became_peered":"2024-09-19T08:41:47.961008+0000","last_unstale":"2024-09-19T08:41:47.964324+0000","last_undegraded":"2024-09-19T08:41:47.964324+0000","last_fullsized":"2024-09-19T08:41:47.964324+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:42.104552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"56'2","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.000690+0000","last_change":"2024-09-19T08:41:46.950525+0000","last_active":"2024-09-19T08:41:48.000690+0000","last_peered":"2024-09-19T08:41:48.000690+0000","last_clean":"2024-09-19T08:41:48.000690+0000","last_became_active":"2024-09-19T08:41:46.950185+0000","last_became_peered":"2024-09-19T08:41:46.950185+0000","last_unstale":"2024-09-19T08:41:48.000690+0000","last_undegraded":"2024-09-19T08:41:48.000690+0000","last_fullsized":"2024-09-19T08:41:48.000690+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:11.548355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.999907+0000","last_change":"2024-09-19T08:41:47.997038+0000","last_active":"2024-09-19T08:41:47.999907+0000","last_peered":"2024-09-19T08:41:47.999907+0000","last_clean":"2024-09-19T08:41:47.999907+0000","last_became_active":"2024-09-19T08:41:47.996654+0000","last_became_peered":"2024-09-19T08:41:47.996654+0000","last_unstale":"2024-09-19T08:41:47.999907+0000","last_undegraded":"2024-09-19T08:41:47.999907+0000","last_fullsized":"2024-09-19T08:41:47.999907+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:25:54.925434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"56'2","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.001947+0000","last_change":"2024-09-19T08:41:46.976094+0000","last_active":"2024-09-19T08:41:48.001947+0000","last_peered":"2024-09-19T08:41:48.001947+0000","last_clean":"2024-09-19T08:41:48.001947+0000","last_became_active":"2024-09-19T08:41:46.975349+0000","last_became_peered":"2024-09-19T08:41:46.975349+0000","last_unstale":"2024-09-19T08:41:48.001947+0000","last_undegraded":"2024-09-19T08:41:48.001947+0000","last_fullsized":"2024-09-19T08:41:48.001947+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:22.996047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.984243+0000","last_change":"2024-09-19T08:41:47.981348+0000","last_active":"2024-09-19T08:41:47.984243+0000","last_peered":"2024-09-19T08:41:47.984243+0000","last_clean":"2024-09-19T08:41:47.984243+0000","last_became_active":"2024-09-19T08:41:47.980623+0000","last_became_peered":"2024-09-19T08:41:47.980623+0000","last_unstale":"2024-09-19T08:41:47.984243+0000","last_undegraded":"2024-09-19T08:41:47.984243+0000","last_fullsized":"2024-09-19T08:41:47.984243+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:21:26.704824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971966+0000","last_change":"2024-09-19T08:41:46.959290+0000","last_active":"2024-09-19T08:41:48.971966+0000","last_peered":"2024-09-19T08:41:48.971966+0000","last_clean":"2024-09-19T08:41:48.971966+0000","last_became_active":"2024-09-19T08:41:46.958454+0000","last_became_peered":"2024-09-19T08:41:46.958454+0000","last_unstale":"2024-09-19T08:41:48.971966+0000","last_undegraded":"2024-09-19T08:41:48.971966+0000","last_fullsized":"2024-09-19T08:41:48.971966+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:39:23.746148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.971630+0000","last_change":"2024-09-19T08:41:47.969748+0000","last_active":"2024-09-19T08:41:47.971630+0000","last_peered":"2024-09-19T08:41:47.971630+0000","last_clean":"2024-09-19T08:41:47.971630+0000","last_became_active":"2024-09-19T08:41:47.969443+0000","last_became_peered":"2024-09-19T08:41:47.969443+0000","last_unstale":"2024-09-19T08:41:47.971630+0000","last_undegraded":"2024-09-19T08:41:47.971630+0000","last_fullsized":"2024-09-19T08:41:47.971630+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:47:48.598741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"56'1","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.999759+0000","last_change":"2024-09-19T08:41:46.956247+0000","last_active":"2024-09-19T08:41:47.999759+0000","last_peered":"2024-09-19T08:41:47.999759+0000","last_clean":"2024-09-19T08:41:47.999759+0000","last_became_active":"2024-09-19T08:41:46.955941+0000","last_became_peered":"2024-09-19T08:41:46.955941+0000","last_unstale":"2024-09-19T08:41:47.999759+0000","last_undegraded":"2024-09-19T08:41:47.999759+0000","last_fullsized":"2024-09-19T08:41:47.999759+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:22.996047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-19T08:41:46.948430+0000","last_change":"2024-09-19T08:41:46.948430+0000","last_active":"2024-09-19T08:41:46.920839+0000","last_peered":"2024-09-19T08:41:46.920839+0000","last_clean":"2024-09-19T08:41:46.920839+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:41:46.948430+0000","last_undegraded":"2024-09-19T08:41:46.948430+0000","last_fullsized":"2024-09-19T08:41:46.948430+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.950542+0000","last_change":"2024-09-19T08:41:46.976878+0000","last_active":"2024-09-19T08:41:47.950542+0000","last_peered":"2024-09-19T08:41:47.950542+0000","last_clean":"2024-09-19T08:41:47.950542+0000","last_became_active":"2024-09-19T08:41:46.975427+0000","last_became_peered":"2024-09-19T08:41:46.975427+0000","last_unstale":"2024-09-19T08:41:47.950542+0000","last_undegraded":"2024-09-19T08:41:47.950542+0000","last_fullsized":"2024-09-19T08:41:47.950542+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:41:25.735590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.979456+0000","last_change":"2024-09-19T08:41:47.977608+0000","last_active":"2024-09-19T08:41:47.979456+0000","last_peered":"2024-09-19T08:41:47.979456+0000","last_clean":"2024-09-19T08:41:47.979456+0000","last_became_active":"2024-09-19T08:41:47.977328+0000","last_became_peered":"2024-09-19T08:41:47.977328+0000","last_unstale":"2024-09-19T08:41:47.979456+0000","last_undegraded":"2024-09-19T08:41:47.979456+0000","last_fullsized":"2024-09-19T08:41:47.979456+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:38.632932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971395+0000","last_change":"2024-09-19T08:41:46.951849+0000","last_active":"2024-09-19T08:41:48.971395+0000","last_peered":"2024-09-19T08:41:48.971395+0000","last_clean":"2024-09-19T08:41:48.971395+0000","last_became_active":"2024-09-19T08:41:46.951049+0000","last_became_peered":"2024-09-19T08:41:46.951049+0000","last_unstale":"2024-09-19T08:41:48.971395+0000","last_undegraded":"2024-09-19T08:41:48.971395+0000","last_fullsized":"2024-09-19T08:41:48.971395+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:37.631023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.972453+0000","last_change":"2024-09-19T08:41:47.959107+0000","last_active":"2024-09-19T08:41:48.972453+0000","last_peered":"2024-09-19T08:41:48.972453+0000","last_clean":"2024-09-19T08:41:48.972453+0000","last_became_active":"2024-09-19T08:41:47.958257+0000","last_became_peered":"2024-09-19T08:41:47.958257+0000","last_unstale":"2024-09-19T08:41:48.972453+0000","last_undegraded":"2024-09-19T08:41:48.972453+0000","last_fullsized":"2024-09-19T08:41:48.972453+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:04:26.313098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.949274+0000","last_change":"2024-09-19T08:41:46.960020+0000","last_active":"2024-09-19T08:41:47.949274+0000","last_peered":"2024-09-19T08:41:47.949274+0000","last_clean":"2024-09-19T08:41:47.949274+0000","last_became_active":"2024-09-19T08:41:46.959232+0000","last_became_peered":"2024-09-19T08:41:46.959232+0000","last_unstale":"2024-09-19T08:41:47.949274+0000","last_undegraded":"2024-09-19T08:41:47.949274+0000","last_fullsized":"2024-09-19T08:41:47.949274+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:28.813825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.993989+0000","last_change":"2024-09-19T08:41:47.973711+0000","last_active":"2024-09-19T08:41:47.993989+0000","last_peered":"2024-09-19T08:41:47.993989+0000","last_clean":"2024-09-19T08:41:47.993989+0000","last_became_active":"2024-09-19T08:41:47.972968+0000","last_became_peered":"2024-09-19T08:41:47.972968+0000","last_unstale":"2024-09-19T08:41:47.993989+0000","last_undegraded":"2024-09-19T08:41:47.993989+0000","last_fullsized":"2024-09-19T08:41:47.993989+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:12.552026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.964270+0000","last_change":"2024-09-19T08:41:46.957881+0000","last_active":"2024-09-19T08:41:47.964270+0000","last_peered":"2024-09-19T08:41:47.964270+0000","last_clean":"2024-09-19T08:41:47.964270+0000","last_became_active":"2024-09-19T08:41:46.957503+0000","last_became_peered":"2024-09-19T08:41:46.957503+0000","last_unstale":"2024-09-19T08:41:47.964270+0000","last_undegraded":"2024-09-19T08:41:47.964270+0000","last_fullsized":"2024-09-19T08:41:47.964270+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:02:24.036512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971244+0000","last_change":"2024-09-19T08:41:47.968178+0000","last_active":"2024-09-19T08:41:48.971244+0000","last_peered":"2024-09-19T08:41:48.971244+0000","last_clean":"2024-09-19T08:41:48.971244+0000","last_became_active":"2024-09-19T08:41:47.967773+0000","last_became_peered":"2024-09-19T08:41:47.967773+0000","last_unstale":"2024-09-19T08:41:48.971244+0000","last_undegraded":"2024-09-19T08:41:48.971244+0000","last_fullsized":"2024-09-19T08:41:48.971244+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:21.843717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"56'3","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.002671+0000","last_change":"2024-09-19T08:41:46.957368+0000","last_active":"2024-09-19T08:41:48.002671+0000","last_peered":"2024-09-19T08:41:48.002671+0000","last_clean":"2024-09-19T08:41:48.002671+0000","last_became_active":"2024-09-19T08:41:46.956670+0000","last_became_peered":"2024-09-19T08:41:46.956670+0000","last_unstale":"2024-09-19T08:41:48.002671+0000","last_undegraded":"2024-09-19T08:41:48.002671+0000","last_fullsized":"2024-09-19T08:41:48.002671+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:37.631023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.002229+0000","last_change":"2024-09-19T08:41:47.977022+0000","last_active":"2024-09-19T08:41:48.002229+0000","last_peered":"2024-09-19T08:41:48.002229+0000","last_clean":"2024-09-19T08:41:48.002229+0000","last_became_active":"2024-09-19T08:41:47.973384+0000","last_became_peered":"2024-09-19T08:41:47.973384+0000","last_unstale":"2024-09-19T08:41:48.002229+0000","last_undegraded":"2024-09-19T08:41:48.002229+0000","last_fullsized":"2024-09-19T08:41:48.002229+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:42.104552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"56'6","reported_seq":20,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.972332+0000","last_change":"2024-09-19T08:41:46.953997+0000","last_active":"2024-09-19T08:41:48.972332+0000","last_peered":"2024-09-19T08:41:48.972332+0000","last_clean":"2024-09-19T08:41:48.972332+0000","last_became_active":"2024-09-19T08:41:46.953261+0000","last_became_peered":"2024-09-19T08:41:46.953261+0000","last_unstale":"2024-09-19T08:41:48.972332+0000","last_undegraded":"2024-09-19T08:41:48.972332+0000","last_fullsized":"2024-09-19T08:41:48.972332+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:41:25.735590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.991181+0000","last_change":"2024-09-19T08:41:47.973617+0000","last_active":"2024-09-19T08:41:47.991181+0000","last_peered":"2024-09-19T08:41:47.991181+0000","last_clean":"2024-09-19T08:41:47.991181+0000","last_became_active":"2024-09-19T08:41:47.973096+0000","last_became_peered":"2024-09-19T08:41:47.973096+0000","last_unstale":"2024-09-19T08:41:47.991181+0000","last_undegraded":"2024-09-19T08:41:47.991181+0000","last_fullsized":"2024-09-19T08:41:47.991181+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:42.104552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"56'2","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.996894+0000","last_change":"2024-09-19T08:41:46.958209+0000","last_active":"2024-09-19T08:41:47.996894+0000","last_peered":"2024-09-19T08:41:47.996894+0000","last_clean":"2024-09-19T08:41:47.996894+0000","last_became_active":"2024-09-19T08:41:46.957655+0000","last_became_peered":"2024-09-19T08:41:46.957655+0000","last_unstale":"2024-09-19T08:41:47.996894+0000","last_undegraded":"2024-09-19T08:41:47.996894+0000","last_fullsized":"2024-09-19T08:41:47.996894+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:06.027506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.994435+0000","last_change":"2024-09-19T08:41:47.980616+0000","last_active":"2024-09-19T08:41:47.994435+0000","last_peered":"2024-09-19T08:41:47.994435+0000","last_clean":"2024-09-19T08:41:47.994435+0000","last_became_active":"2024-09-19T08:41:47.978716+0000","last_became_peered":"2024-09-19T08:41:47.978716+0000","last_unstale":"2024-09-19T08:41:47.994435+0000","last_undegraded":"2024-09-19T08:41:47.994435+0000","last_fullsized":"2024-09-19T08:41:47.994435+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:56:44.612767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-19T08:41:46.966764+0000","last_change":"2024-09-19T08:41:46.963078+0000","last_active":"2024-09-19T08:41:46.966764+0000","last_peered":"2024-09-19T08:41:46.966764+0000","last_clean":"2024-09-19T08:41:46.966764+0000","last_became_active":"2024-09-19T08:41:46.962049+0000","last_became_peered":"2024-09-19T08:41:46.962049+0000","last_unstale":"2024-09-19T08:41:46.966764+0000","last_undegraded":"2024-09-19T08:41:46.966764+0000","last_fullsized":"2024-09-19T08:41:46.966764+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:18.215821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971307+0000","last_change":"2024-09-19T08:41:47.985774+0000","last_active":"2024-09-19T08:41:48.971307+0000","last_peered":"2024-09-19T08:41:48.971307+0000","last_clean":"2024-09-19T08:41:48.971307+0000","last_became_active":"2024-09-19T08:41:47.984953+0000","last_became_peered":"2024-09-19T08:41:47.984953+0000","last_unstale":"2024-09-19T08:41:48.971307+0000","last_undegraded":"2024-09-19T08:41:48.971307+0000","last_fullsized":"2024-09-19T08:41:48.971307+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:42.104552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.950609+0000","last_change":"2024-09-19T08:41:46.973134+0000","last_active":"2024-09-19T08:41:47.950609+0000","last_peered":"2024-09-19T08:41:47.950609+0000","last_clean":"2024-09-19T08:41:47.950609+0000","last_became_active":"2024-09-19T08:41:46.971993+0000","last_became_peered":"2024-09-19T08:41:46.971993+0000","last_unstale":"2024-09-19T08:41:47.950609+0000","last_undegraded":"2024-09-19T08:41:47.950609+0000","last_fullsized":"2024-09-19T08:41:47.950609+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:39:23.746148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.993733+0000","last_change":"2024-09-19T08:41:47.970744+0000","last_active":"2024-09-19T08:41:47.993733+0000","last_peered":"2024-09-19T08:41:47.993733+0000","last_clean":"2024-09-19T08:41:47.993733+0000","last_became_active":"2024-09-19T08:41:47.969645+0000","last_became_peered":"2024-09-19T08:41:47.969645+0000","last_unstale":"2024-09-19T08:41:47.993733+0000","last_undegraded":"2024-09-19T08:41:47.993733+0000","last_fullsized":"2024-09-19T08:41:47.993733+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:52.726130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"56'8","reported_seq":20,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.002402+0000","last_change":"2024-09-19T08:41:46.971824+0000","last_active":"2024-09-19T08:41:48.002402+0000","last_peered":"2024-09-19T08:41:48.002402+0000","last_clean":"2024-09-19T08:41:48.002402+0000","last_became_active":"2024-09-19T08:41:46.971096+0000","last_became_peered":"2024-09-19T08:41:46.971096+0000","last_unstale":"2024-09-19T08:41:48.002402+0000","last_undegraded":"2024-09-19T08:41:48.002402+0000","last_fullsized":"2024-09-19T08:41:48.002402+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:41:25.735590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.994127+0000","last_change":"2024-09-19T08:41:47.972240+0000","last_active":"2024-09-19T08:41:47.994127+0000","last_peered":"2024-09-19T08:41:47.994127+0000","last_clean":"2024-09-19T08:41:47.994127+0000","last_became_active":"2024-09-19T08:41:47.971284+0000","last_became_peered":"2024-09-19T08:41:47.971284+0000","last_unstale":"2024-09-19T08:41:47.994127+0000","last_undegraded":"2024-09-19T08:41:47.994127+0000","last_fullsized":"2024-09-19T08:41:47.994127+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:43.544741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"56'1","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.989173+0000","last_change":"2024-09-19T08:41:46.956816+0000","last_active":"2024-09-19T08:41:47.989173+0000","last_peered":"2024-09-19T08:41:47.989173+0000","last_clean":"2024-09-19T08:41:47.989173+0000","last_became_active":"2024-09-19T08:41:46.956468+0000","last_became_peered":"2024-09-19T08:41:46.956468+0000","last_unstale":"2024-09-19T08:41:47.989173+0000","last_undegraded":"2024-09-19T08:41:47.989173+0000","last_fullsized":"2024-09-19T08:41:47.989173+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:11.860203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.991287+0000","last_change":"2024-09-19T08:41:47.974760+0000","last_active":"2024-09-19T08:41:47.991287+0000","last_peered":"2024-09-19T08:41:47.991287+0000","last_clean":"2024-09-19T08:41:47.991287+0000","last_became_active":"2024-09-19T08:41:47.972626+0000","last_became_peered":"2024-09-19T08:41:47.972626+0000","last_unstale":"2024-09-19T08:41:47.991287+0000","last_undegraded":"2024-09-19T08:41:47.991287+0000","last_fullsized":"2024-09-19T08:41:47.991287+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:09.481675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"56'2","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.000795+0000","last_change":"2024-09-19T08:41:46.956922+0000","last_active":"2024-09-19T08:41:48.000795+0000","last_peered":"2024-09-19T08:41:48.000795+0000","last_clean":"2024-09-19T08:41:48.000795+0000","last_became_active":"2024-09-19T08:41:46.956049+0000","last_became_peered":"2024-09-19T08:41:46.956049+0000","last_unstale":"2024-09-19T08:41:48.000795+0000","last_undegraded":"2024-09-19T08:41:48.000795+0000","last_fullsized":"2024-09-19T08:41:48.000795+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:51.722459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.988936+0000","last_change":"2024-09-19T08:41:47.987128+0000","last_active":"2024-09-19T08:41:47.988936+0000","last_peered":"2024-09-19T08:41:47.988936+0000","last_clean":"2024-09-19T08:41:47.988936+0000","last_became_active":"2024-09-19T08:41:47.986850+0000","last_became_peered":"2024-09-19T08:41:47.986850+0000","last_unstale":"2024-09-19T08:41:47.988936+0000","last_undegraded":"2024-09-19T08:41:47.988936+0000","last_fullsized":"2024-09-19T08:41:47.988936+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:13.337414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"56'1","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.002911+0000","last_change":"2024-09-19T08:41:46.971406+0000","last_active":"2024-09-19T08:41:48.002911+0000","last_peered":"2024-09-19T08:41:48.002911+0000","last_clean":"2024-09-19T08:41:48.002911+0000","last_became_active":"2024-09-19T08:41:46.970580+0000","last_became_peered":"2024-09-19T08:41:46.970580+0000","last_unstale":"2024-09-19T08:41:48.002911+0000","last_undegraded":"2024-09-19T08:41:48.002911+0000","last_fullsized":"2024-09-19T08:41:48.002911+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:52:40.220622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.002262+0000","last_change":"2024-09-19T08:41:47.979548+0000","last_active":"2024-09-19T08:41:48.002262+0000","last_peered":"2024-09-19T08:41:48.002262+0000","last_clean":"2024-09-19T08:41:48.002262+0000","last_became_active":"2024-09-19T08:41:47.978787+0000","last_became_peered":"2024-09-19T08:41:47.978787+0000","last_unstale":"2024-09-19T08:41:48.002262+0000","last_undegraded":"2024-09-19T08:41:48.002262+0000","last_fullsized":"2024-09-19T08:41:48.002262+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:09.481675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.939420+0000","last_change":"2024-09-19T08:41:46.958787+0000","last_active":"2024-09-19T08:41:47.939420+0000","last_peered":"2024-09-19T08:41:47.939420+0000","last_clean":"2024-09-19T08:41:47.939420+0000","last_became_active":"2024-09-19T08:41:46.958288+0000","last_became_peered":"2024-09-19T08:41:46.958288+0000","last_unstale":"2024-09-19T08:41:47.939420+0000","last_undegraded":"2024-09-19T08:41:47.939420+0000","last_fullsized":"2024-09-19T08:41:47.939420+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:32:43.669648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.991121+0000","last_change":"2024-09-19T08:41:47.959094+0000","last_active":"2024-09-19T08:41:47.991121+0000","last_peered":"2024-09-19T08:41:47.991121+0000","last_clean":"2024-09-19T08:41:47.991121+0000","last_became_active":"2024-09-19T08:41:47.958309+0000","last_became_peered":"2024-09-19T08:41:47.958309+0000","last_unstale":"2024-09-19T08:41:47.991121+0000","last_undegraded":"2024-09-19T08:41:47.991121+0000","last_fullsized":"2024-09-19T08:41:47.991121+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:52.726130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"56'2","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.996240+0000","last_change":"2024-09-19T08:41:46.955663+0000","last_active":"2024-09-19T08:41:47.996240+0000","last_peered":"2024-09-19T08:41:47.996240+0000","last_clean":"2024-09-19T08:41:47.996240+0000","last_became_active":"2024-09-19T08:41:46.955237+0000","last_became_peered":"2024-09-19T08:41:46.955237+0000","last_unstale":"2024-09-19T08:41:47.996240+0000","last_undegraded":"2024-09-19T08:41:47.996240+0000","last_fullsized":"2024-09-19T08:41:47.996240+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:49:41.535703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.981832+0000","last_change":"2024-09-19T08:41:47.979199+0000","last_active":"2024-09-19T08:41:47.981832+0000","last_peered":"2024-09-19T08:41:47.981832+0000","last_clean":"2024-09-19T08:41:47.981832+0000","last_became_active":"2024-09-19T08:41:47.978858+0000","last_became_peered":"2024-09-19T08:41:47.978858+0000","last_unstale":"2024-09-19T08:41:47.981832+0000","last_undegraded":"2024-09-19T08:41:47.981832+0000","last_fullsized":"2024-09-19T08:41:47.981832+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:08:18.593800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.970964+0000","last_change":"2024-09-19T08:41:46.957163+0000","last_active":"2024-09-19T08:41:48.970964+0000","last_peered":"2024-09-19T08:41:48.970964+0000","last_clean":"2024-09-19T08:41:48.970964+0000","last_became_active":"2024-09-19T08:41:46.956745+0000","last_became_peered":"2024-09-19T08:41:46.956745+0000","last_unstale":"2024-09-19T08:41:48.970964+0000","last_undegraded":"2024-09-19T08:41:48.970964+0000","last_fullsized":"2024-09-19T08:41:48.970964+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:51:54.055870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.965010+0000","last_change":"2024-09-19T08:41:47.962932+0000","last_active":"2024-09-19T08:41:47.965010+0000","last_peered":"2024-09-19T08:41:47.965010+0000","last_clean":"2024-09-19T08:41:47.965010+0000","last_became_active":"2024-09-19T08:41:47.962192+0000","last_became_peered":"2024-09-19T08:41:47.962192+0000","last_unstale":"2024-09-19T08:41:47.965010+0000","last_undegraded":"2024-09-19T08:41:47.965010+0000","last_fullsized":"2024-09-19T08:41:47.965010+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:17.405379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.949954+0000","last_change":"2024-09-19T08:41:46.970587+0000","last_active":"2024-09-19T08:41:47.949954+0000","last_peered":"2024-09-19T08:41:47.949954+0000","last_clean":"2024-09-19T08:41:47.949954+0000","last_became_active":"2024-09-19T08:41:46.968910+0000","last_became_peered":"2024-09-19T08:41:46.968910+0000","last_unstale":"2024-09-19T08:41:47.949954+0000","last_undegraded":"2024-09-19T08:41:47.949954+0000","last_fullsized":"2024-09-19T08:41:47.949954+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:37.631023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.993141+0000","last_change":"2024-09-19T08:41:47.975389+0000","last_active":"2024-09-19T08:41:47.993141+0000","last_peered":"2024-09-19T08:41:47.993141+0000","last_clean":"2024-09-19T08:41:47.993141+0000","last_became_active":"2024-09-19T08:41:47.974667+0000","last_became_peered":"2024-09-19T08:41:47.974667+0000","last_unstale":"2024-09-19T08:41:47.993141+0000","last_undegraded":"2024-09-19T08:41:47.993141+0000","last_fullsized":"2024-09-19T08:41:47.993141+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:17.405379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971357+0000","last_change":"2024-09-19T08:41:46.949050+0000","last_active":"2024-09-19T08:41:48.971357+0000","last_peered":"2024-09-19T08:41:48.971357+0000","last_clean":"2024-09-19T08:41:48.971357+0000","last_became_active":"2024-09-19T08:41:46.948768+0000","last_became_peered":"2024-09-19T08:41:46.948768+0000","last_unstale":"2024-09-19T08:41:48.971357+0000","last_undegraded":"2024-09-19T08:41:48.971357+0000","last_fullsized":"2024-09-19T08:41:48.971357+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:25.916697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.991230+0000","last_change":"2024-09-19T08:41:47.974663+0000","last_active":"2024-09-19T08:41:47.991230+0000","last_peered":"2024-09-19T08:41:47.991230+0000","last_clean":"2024-09-19T08:41:47.991230+0000","last_became_active":"2024-09-19T08:41:47.972510+0000","last_became_peered":"2024-09-19T08:41:47.972510+0000","last_unstale":"2024-09-19T08:41:47.991230+0000","last_undegraded":"2024-09-19T08:41:47.991230+0000","last_fullsized":"2024-09-19T08:41:47.991230+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:41.615523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.940482+0000","last_change":"2024-09-19T08:41:46.958587+0000","last_active":"2024-09-19T08:41:47.940482+0000","last_peered":"2024-09-19T08:41:47.940482+0000","last_clean":"2024-09-19T08:41:47.940482+0000","last_became_active":"2024-09-19T08:41:46.958146+0000","last_became_peered":"2024-09-19T08:41:46.958146+0000","last_unstale":"2024-09-19T08:41:47.940482+0000","last_undegraded":"2024-09-19T08:41:47.940482+0000","last_fullsized":"2024-09-19T08:41:47.940482+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:42.541069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971266+0000","last_change":"2024-09-19T08:41:47.967496+0000","last_active":"2024-09-19T08:41:48.971266+0000","last_peered":"2024-09-19T08:41:48.971266+0000","last_clean":"2024-09-19T08:41:48.971266+0000","last_became_active":"2024-09-19T08:41:47.966684+0000","last_became_peered":"2024-09-19T08:41:47.966684+0000","last_unstale":"2024-09-19T08:41:48.971266+0000","last_undegraded":"2024-09-19T08:41:48.971266+0000","last_fullsized":"2024-09-19T08:41:48.971266+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:44:15.639843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-19T08:41:46.960637+0000","last_change":"2024-09-19T08:41:46.957237+0000","last_active":"2024-09-19T08:41:46.960637+0000","last_peered":"2024-09-19T08:41:46.960637+0000","last_clean":"2024-09-19T08:41:46.960637+0000","last_became_active":"2024-09-19T08:41:46.956480+0000","last_became_peered":"2024-09-19T08:41:46.956480+0000","last_unstale":"2024-09-19T08:41:46.960637+0000","last_undegraded":"2024-09-19T08:41:46.960637+0000","last_fullsized":"2024-09-19T08:41:46.960637+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:11.952600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.979216+0000","last_change":"2024-09-19T08:41:47.972895+0000","last_active":"2024-09-19T08:41:47.979216+0000","last_peered":"2024-09-19T08:41:47.979216+0000","last_clean":"2024-09-19T08:41:47.979216+0000","last_became_active":"2024-09-19T08:41:47.972572+0000","last_became_peered":"2024-09-19T08:41:47.972572+0000","last_unstale":"2024-09-19T08:41:47.979216+0000","last_undegraded":"2024-09-19T08:41:47.979216+0000","last_fullsized":"2024-09-19T08:41:47.979216+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:04:26.313098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.955154+0000","last_change":"2024-09-19T08:41:46.976718+0000","last_active":"2024-09-19T08:41:47.955154+0000","last_peered":"2024-09-19T08:41:47.955154+0000","last_clean":"2024-09-19T08:41:47.955154+0000","last_became_active":"2024-09-19T08:41:46.975905+0000","last_became_peered":"2024-09-19T08:41:46.975905+0000","last_unstale":"2024-09-19T08:41:47.955154+0000","last_undegraded":"2024-09-19T08:41:47.955154+0000","last_fullsized":"2024-09-19T08:41:47.955154+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:32:43.669648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.978612+0000","last_change":"2024-09-19T08:41:47.976156+0000","last_active":"2024-09-19T08:41:47.978612+0000","last_peered":"2024-09-19T08:41:47.978612+0000","last_clean":"2024-09-19T08:41:47.978612+0000","last_became_active":"2024-09-19T08:41:47.975738+0000","last_became_peered":"2024-09-19T08:41:47.975738+0000","last_unstale":"2024-09-19T08:41:47.978612+0000","last_undegraded":"2024-09-19T08:41:47.978612+0000","last_fullsized":"2024-09-19T08:41:47.978612+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:04:25.609497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.970965+0000","last_change":"2024-09-19T08:41:46.976793+0000","last_active":"2024-09-19T08:41:48.970965+0000","last_peered":"2024-09-19T08:41:48.970965+0000","last_clean":"2024-09-19T08:41:48.970965+0000","last_became_active":"2024-09-19T08:41:46.976404+0000","last_became_peered":"2024-09-19T08:41:46.976404+0000","last_unstale":"2024-09-19T08:41:48.970965+0000","last_undegraded":"2024-09-19T08:41:48.970965+0000","last_fullsized":"2024-09-19T08:41:48.970965+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:05.984736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.994385+0000","last_change":"2024-09-19T08:41:47.974129+0000","last_active":"2024-09-19T08:41:47.994385+0000","last_peered":"2024-09-19T08:41:47.994385+0000","last_clean":"2024-09-19T08:41:47.994385+0000","last_became_active":"2024-09-19T08:41:47.973452+0000","last_became_peered":"2024-09-19T08:41:47.973452+0000","last_unstale":"2024-09-19T08:41:47.994385+0000","last_undegraded":"2024-09-19T08:41:47.994385+0000","last_fullsized":"2024-09-19T08:41:47.994385+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:17.405379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971581+0000","last_change":"2024-09-19T08:41:46.958803+0000","last_active":"2024-09-19T08:41:48.971581+0000","last_peered":"2024-09-19T08:41:48.971581+0000","last_clean":"2024-09-19T08:41:48.971581+0000","last_became_active":"2024-09-19T08:41:46.957741+0000","last_became_peered":"2024-09-19T08:41:46.957741+0000","last_unstale":"2024-09-19T08:41:48.971581+0000","last_undegraded":"2024-09-19T08:41:48.971581+0000","last_fullsized":"2024-09-19T08:41:48.971581+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:28.813825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.970699+0000","last_change":"2024-09-19T08:41:47.977309+0000","last_active":"2024-09-19T08:41:48.970699+0000","last_peered":"2024-09-19T08:41:48.970699+0000","last_clean":"2024-09-19T08:41:48.970699+0000","last_became_active":"2024-09-19T08:41:47.976991+0000","last_became_peered":"2024-09-19T08:41:47.976991+0000","last_unstale":"2024-09-19T08:41:48.970699+0000","last_undegraded":"2024-09-19T08:41:48.970699+0000","last_fullsized":"2024-09-19T08:41:48.970699+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:56:40.725742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.970759+0000","last_change":"2024-09-19T08:41:46.954225+0000","last_active":"2024-09-19T08:41:48.970759+0000","last_peered":"2024-09-19T08:41:48.970759+0000","last_clean":"2024-09-19T08:41:48.970759+0000","last_became_active":"2024-09-19T08:41:46.953891+0000","last_became_peered":"2024-09-19T08:41:46.953891+0000","last_unstale":"2024-09-19T08:41:48.970759+0000","last_undegraded":"2024-09-19T08:41:48.970759+0000","last_fullsized":"2024-09-19T08:41:48.970759+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:59:22.248802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.984239+0000","last_change":"2024-09-19T08:41:47.978333+0000","last_active":"2024-09-19T08:41:47.984239+0000","last_peered":"2024-09-19T08:41:47.984239+0000","last_clean":"2024-09-19T08:41:47.984239+0000","last_became_active":"2024-09-19T08:41:47.977872+0000","last_became_peered":"2024-09-19T08:41:47.977872+0000","last_unstale":"2024-09-19T08:41:47.984239+0000","last_undegraded":"2024-09-19T08:41:47.984239+0000","last_fullsized":"2024-09-19T08:41:47.984239+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:05:29.016642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.965580+0000","last_change":"2024-09-19T08:41:46.971660+0000","last_active":"2024-09-19T08:41:47.965580+0000","last_peered":"2024-09-19T08:41:47.965580+0000","last_clean":"2024-09-19T08:41:47.965580+0000","last_became_active":"2024-09-19T08:41:46.971322+0000","last_became_peered":"2024-09-19T08:41:46.971322+0000","last_unstale":"2024-09-19T08:41:47.965580+0000","last_undegraded":"2024-09-19T08:41:47.965580+0000","last_fullsized":"2024-09-19T08:41:47.965580+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:42:15.264769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.971766+0000","last_change":"2024-09-19T08:41:47.970806+0000","last_active":"2024-09-19T08:41:47.971766+0000","last_peered":"2024-09-19T08:41:47.971766+0000","last_clean":"2024-09-19T08:41:47.971766+0000","last_became_active":"2024-09-19T08:41:47.970054+0000","last_became_peered":"2024-09-19T08:41:47.970054+0000","last_unstale":"2024-09-19T08:41:47.971766+0000","last_undegraded":"2024-09-19T08:41:47.971766+0000","last_fullsized":"2024-09-19T08:41:47.971766+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:15:45.598587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-19T08:41:46.981008+0000","last_change":"2024-09-19T08:41:46.978092+0000","last_active":"2024-09-19T08:41:46.981008+0000","last_peered":"2024-09-19T08:41:46.981008+0000","last_clean":"2024-09-19T08:41:46.981008+0000","last_became_active":"2024-09-19T08:41:46.977408+0000","last_became_peered":"2024-09-19T08:41:46.977408+0000","last_unstale":"2024-09-19T08:41:46.981008+0000","last_undegraded":"2024-09-19T08:41:46.981008+0000","last_fullsized":"2024-09-19T08:41:46.981008+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:58.851715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":231,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.958190+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:41:47.958190+0000","last_peered":"2024-09-19T08:41:47.958190+0000","last_clean":"2024-09-19T08:41:47.958190+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:41:47.958190+0000","last_undegraded":"2024-09-19T08:41:47.958190+0000","last_fullsized":"2024-09-19T08:41:47.958190+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509487,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1265664,"data_stored":863744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509489,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27688,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":671744,"data_stored":273868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509489,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27692,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":675840,"data_stored":274828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509486,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31844,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":671744,"data_stored":273272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509488,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31844,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":667648,"data_stored":273306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509486,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1273856,"data_stored":865778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574895,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31828,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340012,"statfs":{"total":96636764160,"available":96604172288,"internally_reserved":0,"allocated":651264,"data_stored":266441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542191,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31828,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340012,"statfs":{"total":96636764160,"available":96604172288,"internally_reserved":0,"allocated":651264,"data_stored":266441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:41:50.983 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:41:51.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.134+0000 7f61317f5640 1 Processor -- start 2024-09-19T08:41:51.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.134+0000 7f61317f5640 1 -- start start 2024-09-19T08:41:51.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.138+0000 7f61317f5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f612c06c3a0 0x7f612c0cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:51.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.138+0000 7f61317f5640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f612c0cf900 0x7f612c0723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:51.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.138+0000 7f61317f5640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f612c06d310 con 0x7f612c15c550 2024-09-19T08:41:51.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.138+0000 7f61317f5640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f612c06d5f0 con 0x7f612c0cf900 2024-09-19T08:41:51.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.138+0000 7f61317f5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f612c06d8d0 con 0x7f612c06c3a0 2024-09-19T08:41:51.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.138+0000 7f612b7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f612c0cf900 0x7f612c0723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:51.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.138+0000 7f612affd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f612c15c550 0x7f612c06bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40056/0 (socket says 172.21.15.98:40056) 2024-09-19T08:41:51.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.138+0000 7f612affd640 1 -- 172.21.15.98:0/4018355906 learned_addr learned my addr 172.21.15.98:0/4018355906 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:41:51.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.138+0000 7f612a7fc640 1 --2- 172.21.15.98:0/4018355906 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f612c06c3a0 0x7f612c0cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:51.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.138+0000 7f612b7fe640 1 -- 172.21.15.98:0/4018355906 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f612c06c3a0 msgr2=0x7f612c0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:51.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.138+0000 7f612b7fe640 1 --2- 172.21.15.98:0/4018355906 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f612c06c3a0 0x7f612c0cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:51.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.138+0000 7f612b7fe640 1 -- 172.21.15.98:0/4018355906 >> v1:172.21.15.98:6789/0 conn(0x7f612c15c550 legacy=0x7f612c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:51.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.138+0000 7f612b7fe640 1 -- 172.21.15.98:0/4018355906 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f612c06dbb0 con 0x7f612c0cf900 2024-09-19T08:41:51.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.138+0000 7f612a7fc640 1 --2- 172.21.15.98:0/4018355906 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f612c06c3a0 0x7f612c0cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:51.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.138+0000 7f612b7fe640 1 --2- 172.21.15.98:0/4018355906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f612c0cf900 0x7f612c0723a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f6124009d90 tx=0x7f612401c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c7549743e7e3805 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:51.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.138+0000 7f6129ffb640 1 -- 172.21.15.98:0/4018355906 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61240b2070 con 0x7f612c0cf900 2024-09-19T08:41:51.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.138+0000 7f6129ffb640 1 -- 172.21.15.98:0/4018355906 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61240a5710 con 0x7f612c0cf900 2024-09-19T08:41:51.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.138+0000 7f6129ffb640 1 -- 172.21.15.98:0/4018355906 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61240a5ae0 con 0x7f612c0cf900 2024-09-19T08:41:51.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.138+0000 7f61317f5640 1 -- 172.21.15.98:0/4018355906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f612c0cf900 msgr2=0x7f612c0723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:51.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.138+0000 7f61317f5640 1 --2- 172.21.15.98:0/4018355906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f612c0cf900 0x7f612c0723a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f6124009d90 tx=0x7f612401c800 comp rx=0 tx=0).stop 2024-09-19T08:41:51.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.138+0000 7f61317f5640 1 -- 172.21.15.98:0/4018355906 shutdown_connections 2024-09-19T08:41:51.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.138+0000 7f61317f5640 1 --2- 172.21.15.98:0/4018355906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f612c0cf900 0x7f612c0723a0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:51.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.138+0000 7f61317f5640 1 --2- 172.21.15.98:0/4018355906 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f612c06c3a0 0x7f612c0cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:51.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.138+0000 7f61317f5640 1 -- 172.21.15.98:0/4018355906 >> 172.21.15.98:0/4018355906 conn(0x7f612c0b2010 msgr2=0x7f612c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:51.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.142+0000 7f61317f5640 1 -- 172.21.15.98:0/4018355906 shutdown_connections 2024-09-19T08:41:51.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.142+0000 7f61317f5640 1 -- 172.21.15.98:0/4018355906 wait complete. 2024-09-19T08:41:51.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.142+0000 7f61317f5640 1 Processor -- start 2024-09-19T08:41:51.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.142+0000 7f61317f5640 1 -- start start 2024-09-19T08:41:51.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.142+0000 7f61317f5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f612c06c3a0 0x7f612c14ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:51.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.142+0000 7f61317f5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f612c0cf900 0x7f612c14d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:51.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.142+0000 7f61317f5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f612c15c550 0x7f612c1497e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:51.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.142+0000 7f61317f5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f612c073480 con 0x7f612c06c3a0 2024-09-19T08:41:51.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.142+0000 7f61317f5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f612c073300 con 0x7f612c0cf900 2024-09-19T08:41:51.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.142+0000 7f61317f5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f612c073600 con 0x7f612c15c550 2024-09-19T08:41:51.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.142+0000 7f612affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f612c06c3a0 0x7f612c14ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:51.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.142+0000 7f612affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f612c06c3a0 0x7f612c14ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46960/0 (socket says 172.21.15.98:46960) 2024-09-19T08:41:51.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.142+0000 7f612affd640 1 -- 172.21.15.98:0/2401004257 learned_addr learned my addr 172.21.15.98:0/2401004257 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:51.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.142+0000 7f612affd640 1 -- 172.21.15.98:0/2401004257 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f612c15c550 msgr2=0x7f612c1497e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:51.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.142+0000 7f612affd640 1 --2- 172.21.15.98:0/2401004257 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f612c15c550 0x7f612c1497e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:51.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.142+0000 7f612affd640 1 -- 172.21.15.98:0/2401004257 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f612c0cf900 msgr2=0x7f612c14d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:51.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.142+0000 7f612affd640 1 --2- 172.21.15.98:0/2401004257 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f612c0cf900 0x7f612c14d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:51.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.142+0000 7f612affd640 1 -- 172.21.15.98:0/2401004257 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f612c159270 con 0x7f612c06c3a0 2024-09-19T08:41:51.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.142+0000 7f612affd640 1 --2- 172.21.15.98:0/2401004257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f612c06c3a0 0x7f612c14ba00 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f611c00c7b0 tx=0x7f611c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:51.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.142+0000 7f61137fe640 1 -- 172.21.15.98:0/2401004257 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c018070 con 0x7f612c06c3a0 2024-09-19T08:41:51.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.142+0000 7f61137fe640 1 -- 172.21.15.98:0/2401004257 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f611c0072d0 con 0x7f612c06c3a0 2024-09-19T08:41:51.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.142+0000 7f61317f5640 1 -- 172.21.15.98:0/2401004257 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f612c159550 con 0x7f612c06c3a0 2024-09-19T08:41:51.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.142+0000 7f61137fe640 1 -- 172.21.15.98:0/2401004257 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c0047f0 con 0x7f612c06c3a0 2024-09-19T08:41:51.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.142+0000 7f61317f5640 1 -- 172.21.15.98:0/2401004257 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f612c159b10 con 0x7f612c06c3a0 2024-09-19T08:41:51.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.142+0000 7f61317f5640 1 -- 172.21.15.98:0/2401004257 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f612c0cf3c0 con 0x7f612c06c3a0 2024-09-19T08:41:51.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.146+0000 7f61137fe640 1 -- 172.21.15.98:0/2401004257 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f611c01b070 con 0x7f612c06c3a0 2024-09-19T08:41:51.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.146+0000 7f61137fe640 1 --2- 172.21.15.98:0/2401004257 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6118067a70 0x7f6118069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:51.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.146+0000 7f61137fe640 1 -- 172.21.15.98:0/2401004257 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f611c085400 con 0x7f612c06c3a0 2024-09-19T08:41:51.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.146+0000 7f612a7fc640 1 --2- 172.21.15.98:0/2401004257 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6118067a70 0x7f6118069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:51.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.146+0000 7f612a7fc640 1 --2- 172.21.15.98:0/2401004257 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6118067a70 0x7f6118069f30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f612c14a820 tx=0x7f6120008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:51.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.150+0000 7f61137fe640 1 -- 172.21.15.98:0/2401004257 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f611c055d50 con 0x7f612c06c3a0 2024-09-19T08:41:51.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.294+0000 7f61317f5640 1 -- 172.21.15.98:0/2401004257 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f612c159020 con 0x7f6118067a70 2024-09-19T08:41:51.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.302+0000 7f61137fe640 1 -- 172.21.15.98:0/2401004257 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251655 (secure 0 0 0) 0x7f612c159020 con 0x7f6118067a70 2024-09-19T08:41:51.307 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:41:51.309 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:41:51.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.306+0000 7f61317f5640 1 -- 172.21.15.98:0/2401004257 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6118067a70 msgr2=0x7f6118069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:51.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.306+0000 7f61317f5640 1 --2- 172.21.15.98:0/2401004257 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6118067a70 0x7f6118069f30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f612c14a820 tx=0x7f6120008040 comp rx=0 tx=0).stop 2024-09-19T08:41:51.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.306+0000 7f61317f5640 1 -- 172.21.15.98:0/2401004257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f612c06c3a0 msgr2=0x7f612c14ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:51.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.306+0000 7f61317f5640 1 --2- 172.21.15.98:0/2401004257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f612c06c3a0 0x7f612c14ba00 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f611c00c7b0 tx=0x7f611c00cc70 comp rx=0 tx=0).stop 2024-09-19T08:41:51.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.306+0000 7f61317f5640 1 -- 172.21.15.98:0/2401004257 shutdown_connections 2024-09-19T08:41:51.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.306+0000 7f61317f5640 1 --2- 172.21.15.98:0/2401004257 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6118067a70 0x7f6118069f30 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:51.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.306+0000 7f61317f5640 1 --2- 172.21.15.98:0/2401004257 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f612c15c550 0x7f612c1497e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:51.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.306+0000 7f61317f5640 1 --2- 172.21.15.98:0/2401004257 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f612c0cf900 0x7f612c14d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:51.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.306+0000 7f61317f5640 1 --2- 172.21.15.98:0/2401004257 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f612c06c3a0 0x7f612c14ba00 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:51.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.306+0000 7f61317f5640 1 -- 172.21.15.98:0/2401004257 >> 172.21.15.98:0/2401004257 conn(0x7f612c0b2010 msgr2=0x7f612c0afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:51.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.306+0000 7f61317f5640 1 -- 172.21.15.98:0/2401004257 shutdown_connections 2024-09-19T08:41:51.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.306+0000 7f61317f5640 1 -- 172.21.15.98:0/2401004257 wait complete. 2024-09-19T08:41:51.322 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-09-19T08:41:50.096986+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":101,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":247596,"kb_used_data":6376,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754727124,"statfs":{"total":773094113280,"available":772840574976,"internally_reserved":0,"allocated":6529024,"data_stored":3357678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.175662"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.985587+0000","last_change":"2024-09-19T08:41:47.980122+0000","last_active":"2024-09-19T08:41:47.985587+0000","last_peered":"2024-09-19T08:41:47.985587+0000","last_clean":"2024-09-19T08:41:47.985587+0000","last_became_active":"2024-09-19T08:41:47.976398+0000","last_became_peered":"2024-09-19T08:41:47.976398+0000","last_unstale":"2024-09-19T08:41:47.985587+0000","last_undegraded":"2024-09-19T08:41:47.985587+0000","last_fullsized":"2024-09-19T08:41:47.985587+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:25.855696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.976086+0000","last_change":"2024-09-19T08:41:47.973338+0000","last_active":"2024-09-19T08:41:47.976086+0000","last_peered":"2024-09-19T08:41:47.976086+0000","last_clean":"2024-09-19T08:41:47.976086+0000","last_became_active":"2024-09-19T08:41:47.973065+0000","last_became_peered":"2024-09-19T08:41:47.973065+0000","last_unstale":"2024-09-19T08:41:47.976086+0000","last_undegraded":"2024-09-19T08:41:47.976086+0000","last_fullsized":"2024-09-19T08:41:47.976086+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:07.544296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.982165+0000","last_change":"2024-09-19T08:41:47.966283+0000","last_active":"2024-09-19T08:41:47.982165+0000","last_peered":"2024-09-19T08:41:47.982165+0000","last_clean":"2024-09-19T08:41:47.982165+0000","last_became_active":"2024-09-19T08:41:47.965563+0000","last_became_peered":"2024-09-19T08:41:47.965563+0000","last_unstale":"2024-09-19T08:41:47.982165+0000","last_undegraded":"2024-09-19T08:41:47.982165+0000","last_fullsized":"2024-09-19T08:41:47.982165+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:39:24.749819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.993880+0000","last_change":"2024-09-19T08:41:47.978137+0000","last_active":"2024-09-19T08:41:47.993880+0000","last_peered":"2024-09-19T08:41:47.993880+0000","last_clean":"2024-09-19T08:41:47.993880+0000","last_became_active":"2024-09-19T08:41:47.976050+0000","last_became_peered":"2024-09-19T08:41:47.976050+0000","last_unstale":"2024-09-19T08:41:47.993880+0000","last_undegraded":"2024-09-19T08:41:47.993880+0000","last_fullsized":"2024-09-19T08:41:47.993880+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:56:44.612767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.994306+0000","last_change":"2024-09-19T08:41:47.993040+0000","last_active":"2024-09-19T08:41:47.994306+0000","last_peered":"2024-09-19T08:41:47.994306+0000","last_clean":"2024-09-19T08:41:47.994306+0000","last_became_active":"2024-09-19T08:41:47.992767+0000","last_became_peered":"2024-09-19T08:41:47.992767+0000","last_unstale":"2024-09-19T08:41:47.994306+0000","last_undegraded":"2024-09-19T08:41:47.994306+0000","last_fullsized":"2024-09-19T08:41:47.994306+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:54.206986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.964412+0000","last_change":"2024-09-19T08:41:47.961838+0000","last_active":"2024-09-19T08:41:47.964412+0000","last_peered":"2024-09-19T08:41:47.964412+0000","last_clean":"2024-09-19T08:41:47.964412+0000","last_became_active":"2024-09-19T08:41:47.961116+0000","last_became_peered":"2024-09-19T08:41:47.961116+0000","last_unstale":"2024-09-19T08:41:47.964412+0000","last_undegraded":"2024-09-19T08:41:47.964412+0000","last_fullsized":"2024-09-19T08:41:47.964412+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:41.615523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971387+0000","last_change":"2024-09-19T08:41:47.966503+0000","last_active":"2024-09-19T08:41:48.971387+0000","last_peered":"2024-09-19T08:41:48.971387+0000","last_clean":"2024-09-19T08:41:48.971387+0000","last_became_active":"2024-09-19T08:41:47.965805+0000","last_became_peered":"2024-09-19T08:41:47.965805+0000","last_unstale":"2024-09-19T08:41:48.971387+0000","last_undegraded":"2024-09-19T08:41:48.971387+0000","last_fullsized":"2024-09-19T08:41:48.971387+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:41.615523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.971097+0000","last_change":"2024-09-19T08:41:47.969144+0000","last_active":"2024-09-19T08:41:47.971097+0000","last_peered":"2024-09-19T08:41:47.971097+0000","last_clean":"2024-09-19T08:41:47.971097+0000","last_became_active":"2024-09-19T08:41:47.968766+0000","last_became_peered":"2024-09-19T08:41:47.968766+0000","last_unstale":"2024-09-19T08:41:47.971097+0000","last_undegraded":"2024-09-19T08:41:47.971097+0000","last_fullsized":"2024-09-19T08:41:47.971097+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:20:19.600052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971951+0000","last_change":"2024-09-19T08:41:47.954679+0000","last_active":"2024-09-19T08:41:48.971951+0000","last_peered":"2024-09-19T08:41:48.971951+0000","last_clean":"2024-09-19T08:41:48.971951+0000","last_became_active":"2024-09-19T08:41:47.951685+0000","last_became_peered":"2024-09-19T08:41:47.951685+0000","last_unstale":"2024-09-19T08:41:48.971951+0000","last_undegraded":"2024-09-19T08:41:48.971951+0000","last_fullsized":"2024-09-19T08:41:48.971951+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:17.405379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971133+0000","last_change":"2024-09-19T08:41:47.977064+0000","last_active":"2024-09-19T08:41:48.971133+0000","last_peered":"2024-09-19T08:41:48.971133+0000","last_clean":"2024-09-19T08:41:48.971133+0000","last_became_active":"2024-09-19T08:41:47.976747+0000","last_became_peered":"2024-09-19T08:41:47.976747+0000","last_unstale":"2024-09-19T08:41:48.971133+0000","last_undegraded":"2024-09-19T08:41:48.971133+0000","last_fullsized":"2024-09-19T08:41:48.971133+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:37.366531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971332+0000","last_change":"2024-09-19T08:41:47.971592+0000","last_active":"2024-09-19T08:41:48.971332+0000","last_peered":"2024-09-19T08:41:48.971332+0000","last_clean":"2024-09-19T08:41:48.971332+0000","last_became_active":"2024-09-19T08:41:47.969075+0000","last_became_peered":"2024-09-19T08:41:47.969075+0000","last_unstale":"2024-09-19T08:41:48.971332+0000","last_undegraded":"2024-09-19T08:41:48.971332+0000","last_fullsized":"2024-09-19T08:41:48.971332+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:29.817497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.992937+0000","last_change":"2024-09-19T08:41:47.968996+0000","last_active":"2024-09-19T08:41:47.992937+0000","last_peered":"2024-09-19T08:41:47.992937+0000","last_clean":"2024-09-19T08:41:47.992937+0000","last_became_active":"2024-09-19T08:41:47.968254+0000","last_became_peered":"2024-09-19T08:41:47.968254+0000","last_unstale":"2024-09-19T08:41:47.992937+0000","last_undegraded":"2024-09-19T08:41:47.992937+0000","last_fullsized":"2024-09-19T08:41:47.992937+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:12.552026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-19T08:41:46.952023+0000","last_change":"2024-09-19T08:41:46.952023+0000","last_active":"2024-09-19T08:41:46.920839+0000","last_peered":"2024-09-19T08:41:46.920839+0000","last_clean":"2024-09-19T08:41:46.920839+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:41:46.952023+0000","last_undegraded":"2024-09-19T08:41:46.952023+0000","last_fullsized":"2024-09-19T08:41:46.952023+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971528+0000","last_change":"2024-09-19T08:41:47.962145+0000","last_active":"2024-09-19T08:41:48.971528+0000","last_peered":"2024-09-19T08:41:48.971528+0000","last_clean":"2024-09-19T08:41:48.971528+0000","last_became_active":"2024-09-19T08:41:47.961449+0000","last_became_peered":"2024-09-19T08:41:47.961449+0000","last_unstale":"2024-09-19T08:41:48.971528+0000","last_undegraded":"2024-09-19T08:41:48.971528+0000","last_fullsized":"2024-09-19T08:41:48.971528+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:09.481675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.972047+0000","last_change":"2024-09-19T08:41:47.960027+0000","last_active":"2024-09-19T08:41:48.972047+0000","last_peered":"2024-09-19T08:41:48.972047+0000","last_clean":"2024-09-19T08:41:48.972047+0000","last_became_active":"2024-09-19T08:41:47.959242+0000","last_became_peered":"2024-09-19T08:41:47.959242+0000","last_unstale":"2024-09-19T08:41:48.972047+0000","last_undegraded":"2024-09-19T08:41:48.972047+0000","last_fullsized":"2024-09-19T08:41:48.972047+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:56:44.612767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.984338+0000","last_change":"2024-09-19T08:41:47.979371+0000","last_active":"2024-09-19T08:41:47.984338+0000","last_peered":"2024-09-19T08:41:47.984338+0000","last_clean":"2024-09-19T08:41:47.984338+0000","last_became_active":"2024-09-19T08:41:47.978819+0000","last_became_peered":"2024-09-19T08:41:47.978819+0000","last_unstale":"2024-09-19T08:41:47.984338+0000","last_undegraded":"2024-09-19T08:41:47.984338+0000","last_fullsized":"2024-09-19T08:41:47.984338+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:34.451800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.981562+0000","last_change":"2024-09-19T08:41:47.977880+0000","last_active":"2024-09-19T08:41:47.981562+0000","last_peered":"2024-09-19T08:41:47.981562+0000","last_clean":"2024-09-19T08:41:47.981562+0000","last_became_active":"2024-09-19T08:41:47.977590+0000","last_became_peered":"2024-09-19T08:41:47.977590+0000","last_unstale":"2024-09-19T08:41:47.981562+0000","last_undegraded":"2024-09-19T08:41:47.981562+0000","last_fullsized":"2024-09-19T08:41:47.981562+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:43:07.334696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.982300+0000","last_change":"2024-09-19T08:41:47.968677+0000","last_active":"2024-09-19T08:41:47.982300+0000","last_peered":"2024-09-19T08:41:47.982300+0000","last_clean":"2024-09-19T08:41:47.982300+0000","last_became_active":"2024-09-19T08:41:47.967937+0000","last_became_peered":"2024-09-19T08:41:47.967937+0000","last_unstale":"2024-09-19T08:41:47.982300+0000","last_undegraded":"2024-09-19T08:41:47.982300+0000","last_fullsized":"2024-09-19T08:41:47.982300+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:32:44.673319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.981344+0000","last_change":"2024-09-19T08:41:47.975831+0000","last_active":"2024-09-19T08:41:47.981344+0000","last_peered":"2024-09-19T08:41:47.981344+0000","last_clean":"2024-09-19T08:41:47.981344+0000","last_became_active":"2024-09-19T08:41:47.975518+0000","last_became_peered":"2024-09-19T08:41:47.975518+0000","last_unstale":"2024-09-19T08:41:47.981344+0000","last_undegraded":"2024-09-19T08:41:47.981344+0000","last_fullsized":"2024-09-19T08:41:47.981344+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:23:47.530098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.976135+0000","last_change":"2024-09-19T08:41:47.973832+0000","last_active":"2024-09-19T08:41:47.976135+0000","last_peered":"2024-09-19T08:41:47.976135+0000","last_clean":"2024-09-19T08:41:47.976135+0000","last_became_active":"2024-09-19T08:41:47.973516+0000","last_became_peered":"2024-09-19T08:41:47.973516+0000","last_unstale":"2024-09-19T08:41:47.976135+0000","last_undegraded":"2024-09-19T08:41:47.976135+0000","last_fullsized":"2024-09-19T08:41:47.976135+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:43:59.145791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.994338+0000","last_change":"2024-09-19T08:41:47.979480+0000","last_active":"2024-09-19T08:41:47.994338+0000","last_peered":"2024-09-19T08:41:47.994338+0000","last_clean":"2024-09-19T08:41:47.994338+0000","last_became_active":"2024-09-19T08:41:47.977765+0000","last_became_peered":"2024-09-19T08:41:47.977765+0000","last_unstale":"2024-09-19T08:41:47.994338+0000","last_undegraded":"2024-09-19T08:41:47.994338+0000","last_fullsized":"2024-09-19T08:41:47.994338+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:41.615523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.996753+0000","last_change":"2024-09-19T08:41:47.981514+0000","last_active":"2024-09-19T08:41:47.996753+0000","last_peered":"2024-09-19T08:41:47.996753+0000","last_clean":"2024-09-19T08:41:47.996753+0000","last_became_active":"2024-09-19T08:41:47.981044+0000","last_became_peered":"2024-09-19T08:41:47.981044+0000","last_unstale":"2024-09-19T08:41:47.996753+0000","last_undegraded":"2024-09-19T08:41:47.996753+0000","last_fullsized":"2024-09-19T08:41:47.996753+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:07.796396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.993777+0000","last_change":"2024-09-19T08:41:47.970880+0000","last_active":"2024-09-19T08:41:47.993777+0000","last_peered":"2024-09-19T08:41:47.993777+0000","last_clean":"2024-09-19T08:41:47.993777+0000","last_became_active":"2024-09-19T08:41:47.970145+0000","last_became_peered":"2024-09-19T08:41:47.970145+0000","last_unstale":"2024-09-19T08:41:47.993777+0000","last_undegraded":"2024-09-19T08:41:47.993777+0000","last_fullsized":"2024-09-19T08:41:47.993777+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:52:41.224293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-19T08:41:46.949524+0000","last_change":"2024-09-19T08:41:46.949524+0000","last_active":"2024-09-19T08:41:46.920839+0000","last_peered":"2024-09-19T08:41:46.920839+0000","last_clean":"2024-09-19T08:41:46.920839+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:41:46.949524+0000","last_undegraded":"2024-09-19T08:41:46.949524+0000","last_fullsized":"2024-09-19T08:41:46.949524+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.985564+0000","last_change":"2024-09-19T08:41:47.977025+0000","last_active":"2024-09-19T08:41:47.985564+0000","last_peered":"2024-09-19T08:41:47.985564+0000","last_clean":"2024-09-19T08:41:47.985564+0000","last_became_active":"2024-09-19T08:41:47.975707+0000","last_became_peered":"2024-09-19T08:41:47.975707+0000","last_unstale":"2024-09-19T08:41:47.985564+0000","last_undegraded":"2024-09-19T08:41:47.985564+0000","last_fullsized":"2024-09-19T08:41:47.985564+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:17.908127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.990912+0000","last_change":"2024-09-19T08:41:47.966128+0000","last_active":"2024-09-19T08:41:47.990912+0000","last_peered":"2024-09-19T08:41:47.990912+0000","last_clean":"2024-09-19T08:41:47.990912+0000","last_became_active":"2024-09-19T08:41:47.965403+0000","last_became_peered":"2024-09-19T08:41:47.965403+0000","last_unstale":"2024-09-19T08:41:47.990912+0000","last_undegraded":"2024-09-19T08:41:47.990912+0000","last_fullsized":"2024-09-19T08:41:47.990912+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:43.544741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.981730+0000","last_change":"2024-09-19T08:41:47.979320+0000","last_active":"2024-09-19T08:41:47.981730+0000","last_peered":"2024-09-19T08:41:47.981730+0000","last_clean":"2024-09-19T08:41:47.981730+0000","last_became_active":"2024-09-19T08:41:47.979010+0000","last_became_peered":"2024-09-19T08:41:47.979010+0000","last_unstale":"2024-09-19T08:41:47.981730+0000","last_undegraded":"2024-09-19T08:41:47.981730+0000","last_fullsized":"2024-09-19T08:41:47.981730+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:44:04.680315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.975042+0000","last_change":"2024-09-19T08:41:47.966338+0000","last_active":"2024-09-19T08:41:47.975042+0000","last_peered":"2024-09-19T08:41:47.975042+0000","last_clean":"2024-09-19T08:41:47.975042+0000","last_became_active":"2024-09-19T08:41:47.966024+0000","last_became_peered":"2024-09-19T08:41:47.966024+0000","last_unstale":"2024-09-19T08:41:47.975042+0000","last_undegraded":"2024-09-19T08:41:47.975042+0000","last_fullsized":"2024-09-19T08:41:47.975042+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:45.927550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.997659+0000","last_change":"2024-09-19T08:41:47.996502+0000","last_active":"2024-09-19T08:41:47.997659+0000","last_peered":"2024-09-19T08:41:47.997659+0000","last_clean":"2024-09-19T08:41:47.997659+0000","last_became_active":"2024-09-19T08:41:47.996162+0000","last_became_peered":"2024-09-19T08:41:47.996162+0000","last_unstale":"2024-09-19T08:41:47.997659+0000","last_undegraded":"2024-09-19T08:41:47.997659+0000","last_fullsized":"2024-09-19T08:41:47.997659+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:13.341860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.982174+0000","last_change":"2024-09-19T08:41:47.966508+0000","last_active":"2024-09-19T08:41:47.982174+0000","last_peered":"2024-09-19T08:41:47.982174+0000","last_clean":"2024-09-19T08:41:47.982174+0000","last_became_active":"2024-09-19T08:41:47.962098+0000","last_became_peered":"2024-09-19T08:41:47.962098+0000","last_unstale":"2024-09-19T08:41:47.982174+0000","last_undegraded":"2024-09-19T08:41:47.982174+0000","last_fullsized":"2024-09-19T08:41:47.982174+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:23.999718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.981287+0000","last_change":"2024-09-19T08:41:47.974969+0000","last_active":"2024-09-19T08:41:47.981287+0000","last_peered":"2024-09-19T08:41:47.981287+0000","last_clean":"2024-09-19T08:41:47.981287+0000","last_became_active":"2024-09-19T08:41:47.974440+0000","last_became_peered":"2024-09-19T08:41:47.974440+0000","last_unstale":"2024-09-19T08:41:47.981287+0000","last_undegraded":"2024-09-19T08:41:47.981287+0000","last_fullsized":"2024-09-19T08:41:47.981287+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:35:19.862729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.978537+0000","last_change":"2024-09-19T08:41:47.972260+0000","last_active":"2024-09-19T08:41:47.978537+0000","last_peered":"2024-09-19T08:41:47.978537+0000","last_clean":"2024-09-19T08:41:47.978537+0000","last_became_active":"2024-09-19T08:41:47.971683+0000","last_became_peered":"2024-09-19T08:41:47.971683+0000","last_unstale":"2024-09-19T08:41:47.978537+0000","last_undegraded":"2024-09-19T08:41:47.978537+0000","last_fullsized":"2024-09-19T08:41:47.978537+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:56:44.612767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.964666+0000","last_change":"2024-09-19T08:41:47.962374+0000","last_active":"2024-09-19T08:41:47.964666+0000","last_peered":"2024-09-19T08:41:47.964666+0000","last_clean":"2024-09-19T08:41:47.964666+0000","last_became_active":"2024-09-19T08:41:47.961305+0000","last_became_peered":"2024-09-19T08:41:47.961305+0000","last_unstale":"2024-09-19T08:41:47.964666+0000","last_undegraded":"2024-09-19T08:41:47.964666+0000","last_fullsized":"2024-09-19T08:41:47.964666+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:09.481675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"56'4","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.001343+0000","last_change":"2024-09-19T08:41:46.976847+0000","last_active":"2024-09-19T08:41:48.001343+0000","last_peered":"2024-09-19T08:41:48.001343+0000","last_clean":"2024-09-19T08:41:48.001343+0000","last_became_active":"2024-09-19T08:41:46.976405+0000","last_became_peered":"2024-09-19T08:41:46.976405+0000","last_unstale":"2024-09-19T08:41:48.001343+0000","last_undegraded":"2024-09-19T08:41:48.001343+0000","last_fullsized":"2024-09-19T08:41:48.001343+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:52:40.220622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971359+0000","last_change":"2024-09-19T08:41:47.971091+0000","last_active":"2024-09-19T08:41:48.971359+0000","last_peered":"2024-09-19T08:41:48.971359+0000","last_clean":"2024-09-19T08:41:48.971359+0000","last_became_active":"2024-09-19T08:41:47.958633+0000","last_became_peered":"2024-09-19T08:41:47.958633+0000","last_unstale":"2024-09-19T08:41:48.971359+0000","last_undegraded":"2024-09-19T08:41:48.971359+0000","last_fullsized":"2024-09-19T08:41:48.971359+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:38.634694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"56'1","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.997195+0000","last_change":"2024-09-19T08:41:46.953976+0000","last_active":"2024-09-19T08:41:47.997195+0000","last_peered":"2024-09-19T08:41:47.997195+0000","last_clean":"2024-09-19T08:41:47.997195+0000","last_became_active":"2024-09-19T08:41:46.953613+0000","last_became_peered":"2024-09-19T08:41:46.953613+0000","last_unstale":"2024-09-19T08:41:47.997195+0000","last_undegraded":"2024-09-19T08:41:47.997195+0000","last_fullsized":"2024-09-19T08:41:47.997195+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:24.976526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.964324+0000","last_change":"2024-09-19T08:41:47.961793+0000","last_active":"2024-09-19T08:41:47.964324+0000","last_peered":"2024-09-19T08:41:47.964324+0000","last_clean":"2024-09-19T08:41:47.964324+0000","last_became_active":"2024-09-19T08:41:47.961008+0000","last_became_peered":"2024-09-19T08:41:47.961008+0000","last_unstale":"2024-09-19T08:41:47.964324+0000","last_undegraded":"2024-09-19T08:41:47.964324+0000","last_fullsized":"2024-09-19T08:41:47.964324+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:42.104552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"56'2","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.000690+0000","last_change":"2024-09-19T08:41:46.950525+0000","last_active":"2024-09-19T08:41:48.000690+0000","last_peered":"2024-09-19T08:41:48.000690+0000","last_clean":"2024-09-19T08:41:48.000690+0000","last_became_active":"2024-09-19T08:41:46.950185+0000","last_became_peered":"2024-09-19T08:41:46.950185+0000","last_unstale":"2024-09-19T08:41:48.000690+0000","last_undegraded":"2024-09-19T08:41:48.000690+0000","last_fullsized":"2024-09-19T08:41:48.000690+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:11.548355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.999907+0000","last_change":"2024-09-19T08:41:47.997038+0000","last_active":"2024-09-19T08:41:47.999907+0000","last_peered":"2024-09-19T08:41:47.999907+0000","last_clean":"2024-09-19T08:41:47.999907+0000","last_became_active":"2024-09-19T08:41:47.996654+0000","last_became_peered":"2024-09-19T08:41:47.996654+0000","last_unstale":"2024-09-19T08:41:47.999907+0000","last_undegraded":"2024-09-19T08:41:47.999907+0000","last_fullsized":"2024-09-19T08:41:47.999907+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:25:54.925434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"56'2","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.001947+0000","last_change":"2024-09-19T08:41:46.976094+0000","last_active":"2024-09-19T08:41:48.001947+0000","last_peered":"2024-09-19T08:41:48.001947+0000","last_clean":"2024-09-19T08:41:48.001947+0000","last_became_active":"2024-09-19T08:41:46.975349+0000","last_became_peered":"2024-09-19T08:41:46.975349+0000","last_unstale":"2024-09-19T08:41:48.001947+0000","last_undegraded":"2024-09-19T08:41:48.001947+0000","last_fullsized":"2024-09-19T08:41:48.001947+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:22.996047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.984243+0000","last_change":"2024-09-19T08:41:47.981348+0000","last_active":"2024-09-19T08:41:47.984243+0000","last_peered":"2024-09-19T08:41:47.984243+0000","last_clean":"2024-09-19T08:41:47.984243+0000","last_became_active":"2024-09-19T08:41:47.980623+0000","last_became_peered":"2024-09-19T08:41:47.980623+0000","last_unstale":"2024-09-19T08:41:47.984243+0000","last_undegraded":"2024-09-19T08:41:47.984243+0000","last_fullsized":"2024-09-19T08:41:47.984243+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:21:26.704824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971966+0000","last_change":"2024-09-19T08:41:46.959290+0000","last_active":"2024-09-19T08:41:48.971966+0000","last_peered":"2024-09-19T08:41:48.971966+0000","last_clean":"2024-09-19T08:41:48.971966+0000","last_became_active":"2024-09-19T08:41:46.958454+0000","last_became_peered":"2024-09-19T08:41:46.958454+0000","last_unstale":"2024-09-19T08:41:48.971966+0000","last_undegraded":"2024-09-19T08:41:48.971966+0000","last_fullsized":"2024-09-19T08:41:48.971966+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:39:23.746148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.971630+0000","last_change":"2024-09-19T08:41:47.969748+0000","last_active":"2024-09-19T08:41:47.971630+0000","last_peered":"2024-09-19T08:41:47.971630+0000","last_clean":"2024-09-19T08:41:47.971630+0000","last_became_active":"2024-09-19T08:41:47.969443+0000","last_became_peered":"2024-09-19T08:41:47.969443+0000","last_unstale":"2024-09-19T08:41:47.971630+0000","last_undegraded":"2024-09-19T08:41:47.971630+0000","last_fullsized":"2024-09-19T08:41:47.971630+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:47:48.598741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"56'1","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.999759+0000","last_change":"2024-09-19T08:41:46.956247+0000","last_active":"2024-09-19T08:41:47.999759+0000","last_peered":"2024-09-19T08:41:47.999759+0000","last_clean":"2024-09-19T08:41:47.999759+0000","last_became_active":"2024-09-19T08:41:46.955941+0000","last_became_peered":"2024-09-19T08:41:46.955941+0000","last_unstale":"2024-09-19T08:41:47.999759+0000","last_undegraded":"2024-09-19T08:41:47.999759+0000","last_fullsized":"2024-09-19T08:41:47.999759+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:22.996047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-19T08:41:46.948430+0000","last_change":"2024-09-19T08:41:46.948430+0000","last_active":"2024-09-19T08:41:46.920839+0000","last_peered":"2024-09-19T08:41:46.920839+0000","last_clean":"2024-09-19T08:41:46.920839+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:41:46.948430+0000","last_undegraded":"2024-09-19T08:41:46.948430+0000","last_fullsized":"2024-09-19T08:41:46.948430+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.950542+0000","last_change":"2024-09-19T08:41:46.976878+0000","last_active":"2024-09-19T08:41:47.950542+0000","last_peered":"2024-09-19T08:41:47.950542+0000","last_clean":"2024-09-19T08:41:47.950542+0000","last_became_active":"2024-09-19T08:41:46.975427+0000","last_became_peered":"2024-09-19T08:41:46.975427+0000","last_unstale":"2024-09-19T08:41:47.950542+0000","last_undegraded":"2024-09-19T08:41:47.950542+0000","last_fullsized":"2024-09-19T08:41:47.950542+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:41:25.735590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.979456+0000","last_change":"2024-09-19T08:41:47.977608+0000","last_active":"2024-09-19T08:41:47.979456+0000","last_peered":"2024-09-19T08:41:47.979456+0000","last_clean":"2024-09-19T08:41:47.979456+0000","last_became_active":"2024-09-19T08:41:47.977328+0000","last_became_peered":"2024-09-19T08:41:47.977328+0000","last_unstale":"2024-09-19T08:41:47.979456+0000","last_undegraded":"2024-09-19T08:41:47.979456+0000","last_fullsized":"2024-09-19T08:41:47.979456+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:38.632932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971395+0000","last_change":"2024-09-19T08:41:46.951849+0000","last_active":"2024-09-19T08:41:48.971395+0000","last_peered":"2024-09-19T08:41:48.971395+0000","last_clean":"2024-09-19T08:41:48.971395+0000","last_became_active":"2024-09-19T08:41:46.951049+0000","last_became_peered":"2024-09-19T08:41:46.951049+0000","last_unstale":"2024-09-19T08:41:48.971395+0000","last_undegraded":"2024-09-19T08:41:48.971395+0000","last_fullsized":"2024-09-19T08:41:48.971395+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:37.631023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.972453+0000","last_change":"2024-09-19T08:41:47.959107+0000","last_active":"2024-09-19T08:41:48.972453+0000","last_peered":"2024-09-19T08:41:48.972453+0000","last_clean":"2024-09-19T08:41:48.972453+0000","last_became_active":"2024-09-19T08:41:47.958257+0000","last_became_peered":"2024-09-19T08:41:47.958257+0000","last_unstale":"2024-09-19T08:41:48.972453+0000","last_undegraded":"2024-09-19T08:41:48.972453+0000","last_fullsized":"2024-09-19T08:41:48.972453+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:04:26.313098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.949274+0000","last_change":"2024-09-19T08:41:46.960020+0000","last_active":"2024-09-19T08:41:47.949274+0000","last_peered":"2024-09-19T08:41:47.949274+0000","last_clean":"2024-09-19T08:41:47.949274+0000","last_became_active":"2024-09-19T08:41:46.959232+0000","last_became_peered":"2024-09-19T08:41:46.959232+0000","last_unstale":"2024-09-19T08:41:47.949274+0000","last_undegraded":"2024-09-19T08:41:47.949274+0000","last_fullsized":"2024-09-19T08:41:47.949274+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:28.813825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.993989+0000","last_change":"2024-09-19T08:41:47.973711+0000","last_active":"2024-09-19T08:41:47.993989+0000","last_peered":"2024-09-19T08:41:47.993989+0000","last_clean":"2024-09-19T08:41:47.993989+0000","last_became_active":"2024-09-19T08:41:47.972968+0000","last_became_peered":"2024-09-19T08:41:47.972968+0000","last_unstale":"2024-09-19T08:41:47.993989+0000","last_undegraded":"2024-09-19T08:41:47.993989+0000","last_fullsized":"2024-09-19T08:41:47.993989+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:12.552026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.964270+0000","last_change":"2024-09-19T08:41:46.957881+0000","last_active":"2024-09-19T08:41:47.964270+0000","last_peered":"2024-09-19T08:41:47.964270+0000","last_clean":"2024-09-19T08:41:47.964270+0000","last_became_active":"2024-09-19T08:41:46.957503+0000","last_became_peered":"2024-09-19T08:41:46.957503+0000","last_unstale":"2024-09-19T08:41:47.964270+0000","last_undegraded":"2024-09-19T08:41:47.964270+0000","last_fullsized":"2024-09-19T08:41:47.964270+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:02:24.036512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971244+0000","last_change":"2024-09-19T08:41:47.968178+0000","last_active":"2024-09-19T08:41:48.971244+0000","last_peered":"2024-09-19T08:41:48.971244+0000","last_clean":"2024-09-19T08:41:48.971244+0000","last_became_active":"2024-09-19T08:41:47.967773+0000","last_became_peered":"2024-09-19T08:41:47.967773+0000","last_unstale":"2024-09-19T08:41:48.971244+0000","last_undegraded":"2024-09-19T08:41:48.971244+0000","last_fullsized":"2024-09-19T08:41:48.971244+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:21.843717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"56'3","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.002671+0000","last_change":"2024-09-19T08:41:46.957368+0000","last_active":"2024-09-19T08:41:48.002671+0000","last_peered":"2024-09-19T08:41:48.002671+0000","last_clean":"2024-09-19T08:41:48.002671+0000","last_became_active":"2024-09-19T08:41:46.956670+0000","last_became_peered":"2024-09-19T08:41:46.956670+0000","last_unstale":"2024-09-19T08:41:48.002671+0000","last_undegraded":"2024-09-19T08:41:48.002671+0000","last_fullsized":"2024-09-19T08:41:48.002671+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:37.631023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.002229+0000","last_change":"2024-09-19T08:41:47.977022+0000","last_active":"2024-09-19T08:41:48.002229+0000","last_peered":"2024-09-19T08:41:48.002229+0000","last_clean":"2024-09-19T08:41:48.002229+0000","last_became_active":"2024-09-19T08:41:47.973384+0000","last_became_peered":"2024-09-19T08:41:47.973384+0000","last_unstale":"2024-09-19T08:41:48.002229+0000","last_undegraded":"2024-09-19T08:41:48.002229+0000","last_fullsized":"2024-09-19T08:41:48.002229+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:42.104552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"56'6","reported_seq":20,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.972332+0000","last_change":"2024-09-19T08:41:46.953997+0000","last_active":"2024-09-19T08:41:48.972332+0000","last_peered":"2024-09-19T08:41:48.972332+0000","last_clean":"2024-09-19T08:41:48.972332+0000","last_became_active":"2024-09-19T08:41:46.953261+0000","last_became_peered":"2024-09-19T08:41:46.953261+0000","last_unstale":"2024-09-19T08:41:48.972332+0000","last_undegraded":"2024-09-19T08:41:48.972332+0000","last_fullsized":"2024-09-19T08:41:48.972332+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:41:25.735590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.991181+0000","last_change":"2024-09-19T08:41:47.973617+0000","last_active":"2024-09-19T08:41:47.991181+0000","last_peered":"2024-09-19T08:41:47.991181+0000","last_clean":"2024-09-19T08:41:47.991181+0000","last_became_active":"2024-09-19T08:41:47.973096+0000","last_became_peered":"2024-09-19T08:41:47.973096+0000","last_unstale":"2024-09-19T08:41:47.991181+0000","last_undegraded":"2024-09-19T08:41:47.991181+0000","last_fullsized":"2024-09-19T08:41:47.991181+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:42.104552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"56'2","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.996894+0000","last_change":"2024-09-19T08:41:46.958209+0000","last_active":"2024-09-19T08:41:47.996894+0000","last_peered":"2024-09-19T08:41:47.996894+0000","last_clean":"2024-09-19T08:41:47.996894+0000","last_became_active":"2024-09-19T08:41:46.957655+0000","last_became_peered":"2024-09-19T08:41:46.957655+0000","last_unstale":"2024-09-19T08:41:47.996894+0000","last_undegraded":"2024-09-19T08:41:47.996894+0000","last_fullsized":"2024-09-19T08:41:47.996894+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:06.027506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.994435+0000","last_change":"2024-09-19T08:41:47.980616+0000","last_active":"2024-09-19T08:41:47.994435+0000","last_peered":"2024-09-19T08:41:47.994435+0000","last_clean":"2024-09-19T08:41:47.994435+0000","last_became_active":"2024-09-19T08:41:47.978716+0000","last_became_peered":"2024-09-19T08:41:47.978716+0000","last_unstale":"2024-09-19T08:41:47.994435+0000","last_undegraded":"2024-09-19T08:41:47.994435+0000","last_fullsized":"2024-09-19T08:41:47.994435+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:56:44.612767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-19T08:41:46.966764+0000","last_change":"2024-09-19T08:41:46.963078+0000","last_active":"2024-09-19T08:41:46.966764+0000","last_peered":"2024-09-19T08:41:46.966764+0000","last_clean":"2024-09-19T08:41:46.966764+0000","last_became_active":"2024-09-19T08:41:46.962049+0000","last_became_peered":"2024-09-19T08:41:46.962049+0000","last_unstale":"2024-09-19T08:41:46.966764+0000","last_undegraded":"2024-09-19T08:41:46.966764+0000","last_fullsized":"2024-09-19T08:41:46.966764+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:18.215821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971307+0000","last_change":"2024-09-19T08:41:47.985774+0000","last_active":"2024-09-19T08:41:48.971307+0000","last_peered":"2024-09-19T08:41:48.971307+0000","last_clean":"2024-09-19T08:41:48.971307+0000","last_became_active":"2024-09-19T08:41:47.984953+0000","last_became_peered":"2024-09-19T08:41:47.984953+0000","last_unstale":"2024-09-19T08:41:48.971307+0000","last_undegraded":"2024-09-19T08:41:48.971307+0000","last_fullsized":"2024-09-19T08:41:48.971307+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:42.104552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.950609+0000","last_change":"2024-09-19T08:41:46.973134+0000","last_active":"2024-09-19T08:41:47.950609+0000","last_peered":"2024-09-19T08:41:47.950609+0000","last_clean":"2024-09-19T08:41:47.950609+0000","last_became_active":"2024-09-19T08:41:46.971993+0000","last_became_peered":"2024-09-19T08:41:46.971993+0000","last_unstale":"2024-09-19T08:41:47.950609+0000","last_undegraded":"2024-09-19T08:41:47.950609+0000","last_fullsized":"2024-09-19T08:41:47.950609+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:39:23.746148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.993733+0000","last_change":"2024-09-19T08:41:47.970744+0000","last_active":"2024-09-19T08:41:47.993733+0000","last_peered":"2024-09-19T08:41:47.993733+0000","last_clean":"2024-09-19T08:41:47.993733+0000","last_became_active":"2024-09-19T08:41:47.969645+0000","last_became_peered":"2024-09-19T08:41:47.969645+0000","last_unstale":"2024-09-19T08:41:47.993733+0000","last_undegraded":"2024-09-19T08:41:47.993733+0000","last_fullsized":"2024-09-19T08:41:47.993733+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:52.726130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"56'8","reported_seq":20,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.002402+0000","last_change":"2024-09-19T08:41:46.971824+0000","last_active":"2024-09-19T08:41:48.002402+0000","last_peered":"2024-09-19T08:41:48.002402+0000","last_clean":"2024-09-19T08:41:48.002402+0000","last_became_active":"2024-09-19T08:41:46.971096+0000","last_became_peered":"2024-09-19T08:41:46.971096+0000","last_unstale":"2024-09-19T08:41:48.002402+0000","last_undegraded":"2024-09-19T08:41:48.002402+0000","last_fullsized":"2024-09-19T08:41:48.002402+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:41:25.735590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.994127+0000","last_change":"2024-09-19T08:41:47.972240+0000","last_active":"2024-09-19T08:41:47.994127+0000","last_peered":"2024-09-19T08:41:47.994127+0000","last_clean":"2024-09-19T08:41:47.994127+0000","last_became_active":"2024-09-19T08:41:47.971284+0000","last_became_peered":"2024-09-19T08:41:47.971284+0000","last_unstale":"2024-09-19T08:41:47.994127+0000","last_undegraded":"2024-09-19T08:41:47.994127+0000","last_fullsized":"2024-09-19T08:41:47.994127+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:43.544741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"56'1","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.989173+0000","last_change":"2024-09-19T08:41:46.956816+0000","last_active":"2024-09-19T08:41:47.989173+0000","last_peered":"2024-09-19T08:41:47.989173+0000","last_clean":"2024-09-19T08:41:47.989173+0000","last_became_active":"2024-09-19T08:41:46.956468+0000","last_became_peered":"2024-09-19T08:41:46.956468+0000","last_unstale":"2024-09-19T08:41:47.989173+0000","last_undegraded":"2024-09-19T08:41:47.989173+0000","last_fullsized":"2024-09-19T08:41:47.989173+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:11.860203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.991287+0000","last_change":"2024-09-19T08:41:47.974760+0000","last_active":"2024-09-19T08:41:47.991287+0000","last_peered":"2024-09-19T08:41:47.991287+0000","last_clean":"2024-09-19T08:41:47.991287+0000","last_became_active":"2024-09-19T08:41:47.972626+0000","last_became_peered":"2024-09-19T08:41:47.972626+0000","last_unstale":"2024-09-19T08:41:47.991287+0000","last_undegraded":"2024-09-19T08:41:47.991287+0000","last_fullsized":"2024-09-19T08:41:47.991287+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:09.481675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"56'2","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.000795+0000","last_change":"2024-09-19T08:41:46.956922+0000","last_active":"2024-09-19T08:41:48.000795+0000","last_peered":"2024-09-19T08:41:48.000795+0000","last_clean":"2024-09-19T08:41:48.000795+0000","last_became_active":"2024-09-19T08:41:46.956049+0000","last_became_peered":"2024-09-19T08:41:46.956049+0000","last_unstale":"2024-09-19T08:41:48.000795+0000","last_undegraded":"2024-09-19T08:41:48.000795+0000","last_fullsized":"2024-09-19T08:41:48.000795+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:51.722459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.988936+0000","last_change":"2024-09-19T08:41:47.987128+0000","last_active":"2024-09-19T08:41:47.988936+0000","last_peered":"2024-09-19T08:41:47.988936+0000","last_clean":"2024-09-19T08:41:47.988936+0000","last_became_active":"2024-09-19T08:41:47.986850+0000","last_became_peered":"2024-09-19T08:41:47.986850+0000","last_unstale":"2024-09-19T08:41:47.988936+0000","last_undegraded":"2024-09-19T08:41:47.988936+0000","last_fullsized":"2024-09-19T08:41:47.988936+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:13.337414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"56'1","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.002911+0000","last_change":"2024-09-19T08:41:46.971406+0000","last_active":"2024-09-19T08:41:48.002911+0000","last_peered":"2024-09-19T08:41:48.002911+0000","last_clean":"2024-09-19T08:41:48.002911+0000","last_became_active":"2024-09-19T08:41:46.970580+0000","last_became_peered":"2024-09-19T08:41:46.970580+0000","last_unstale":"2024-09-19T08:41:48.002911+0000","last_undegraded":"2024-09-19T08:41:48.002911+0000","last_fullsized":"2024-09-19T08:41:48.002911+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:52:40.220622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.002262+0000","last_change":"2024-09-19T08:41:47.979548+0000","last_active":"2024-09-19T08:41:48.002262+0000","last_peered":"2024-09-19T08:41:48.002262+0000","last_clean":"2024-09-19T08:41:48.002262+0000","last_became_active":"2024-09-19T08:41:47.978787+0000","last_became_peered":"2024-09-19T08:41:47.978787+0000","last_unstale":"2024-09-19T08:41:48.002262+0000","last_undegraded":"2024-09-19T08:41:48.002262+0000","last_fullsized":"2024-09-19T08:41:48.002262+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:09.481675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.939420+0000","last_change":"2024-09-19T08:41:46.958787+0000","last_active":"2024-09-19T08:41:47.939420+0000","last_peered":"2024-09-19T08:41:47.939420+0000","last_clean":"2024-09-19T08:41:47.939420+0000","last_became_active":"2024-09-19T08:41:46.958288+0000","last_became_peered":"2024-09-19T08:41:46.958288+0000","last_unstale":"2024-09-19T08:41:47.939420+0000","last_undegraded":"2024-09-19T08:41:47.939420+0000","last_fullsized":"2024-09-19T08:41:47.939420+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:32:43.669648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.991121+0000","last_change":"2024-09-19T08:41:47.959094+0000","last_active":"2024-09-19T08:41:47.991121+0000","last_peered":"2024-09-19T08:41:47.991121+0000","last_clean":"2024-09-19T08:41:47.991121+0000","last_became_active":"2024-09-19T08:41:47.958309+0000","last_became_peered":"2024-09-19T08:41:47.958309+0000","last_unstale":"2024-09-19T08:41:47.991121+0000","last_undegraded":"2024-09-19T08:41:47.991121+0000","last_fullsized":"2024-09-19T08:41:47.991121+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:52.726130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"56'2","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.996240+0000","last_change":"2024-09-19T08:41:46.955663+0000","last_active":"2024-09-19T08:41:47.996240+0000","last_peered":"2024-09-19T08:41:47.996240+0000","last_clean":"2024-09-19T08:41:47.996240+0000","last_became_active":"2024-09-19T08:41:46.955237+0000","last_became_peered":"2024-09-19T08:41:46.955237+0000","last_unstale":"2024-09-19T08:41:47.996240+0000","last_undegraded":"2024-09-19T08:41:47.996240+0000","last_fullsized":"2024-09-19T08:41:47.996240+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:49:41.535703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.981832+0000","last_change":"2024-09-19T08:41:47.979199+0000","last_active":"2024-09-19T08:41:47.981832+0000","last_peered":"2024-09-19T08:41:47.981832+0000","last_clean":"2024-09-19T08:41:47.981832+0000","last_became_active":"2024-09-19T08:41:47.978858+0000","last_became_peered":"2024-09-19T08:41:47.978858+0000","last_unstale":"2024-09-19T08:41:47.981832+0000","last_undegraded":"2024-09-19T08:41:47.981832+0000","last_fullsized":"2024-09-19T08:41:47.981832+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:08:18.593800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.970964+0000","last_change":"2024-09-19T08:41:46.957163+0000","last_active":"2024-09-19T08:41:48.970964+0000","last_peered":"2024-09-19T08:41:48.970964+0000","last_clean":"2024-09-19T08:41:48.970964+0000","last_became_active":"2024-09-19T08:41:46.956745+0000","last_became_peered":"2024-09-19T08:41:46.956745+0000","last_unstale":"2024-09-19T08:41:48.970964+0000","last_undegraded":"2024-09-19T08:41:48.970964+0000","last_fullsized":"2024-09-19T08:41:48.970964+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:51:54.055870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.965010+0000","last_change":"2024-09-19T08:41:47.962932+0000","last_active":"2024-09-19T08:41:47.965010+0000","last_peered":"2024-09-19T08:41:47.965010+0000","last_clean":"2024-09-19T08:41:47.965010+0000","last_became_active":"2024-09-19T08:41:47.962192+0000","last_became_peered":"2024-09-19T08:41:47.962192+0000","last_unstale":"2024-09-19T08:41:47.965010+0000","last_undegraded":"2024-09-19T08:41:47.965010+0000","last_fullsized":"2024-09-19T08:41:47.965010+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:17.405379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.949954+0000","last_change":"2024-09-19T08:41:46.970587+0000","last_active":"2024-09-19T08:41:47.949954+0000","last_peered":"2024-09-19T08:41:47.949954+0000","last_clean":"2024-09-19T08:41:47.949954+0000","last_became_active":"2024-09-19T08:41:46.968910+0000","last_became_peered":"2024-09-19T08:41:46.968910+0000","last_unstale":"2024-09-19T08:41:47.949954+0000","last_undegraded":"2024-09-19T08:41:47.949954+0000","last_fullsized":"2024-09-19T08:41:47.949954+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:37.631023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.993141+0000","last_change":"2024-09-19T08:41:47.975389+0000","last_active":"2024-09-19T08:41:47.993141+0000","last_peered":"2024-09-19T08:41:47.993141+0000","last_clean":"2024-09-19T08:41:47.993141+0000","last_became_active":"2024-09-19T08:41:47.974667+0000","last_became_peered":"2024-09-19T08:41:47.974667+0000","last_unstale":"2024-09-19T08:41:47.993141+0000","last_undegraded":"2024-09-19T08:41:47.993141+0000","last_fullsized":"2024-09-19T08:41:47.993141+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:17.405379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971357+0000","last_change":"2024-09-19T08:41:46.949050+0000","last_active":"2024-09-19T08:41:48.971357+0000","last_peered":"2024-09-19T08:41:48.971357+0000","last_clean":"2024-09-19T08:41:48.971357+0000","last_became_active":"2024-09-19T08:41:46.948768+0000","last_became_peered":"2024-09-19T08:41:46.948768+0000","last_unstale":"2024-09-19T08:41:48.971357+0000","last_undegraded":"2024-09-19T08:41:48.971357+0000","last_fullsized":"2024-09-19T08:41:48.971357+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:25.916697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.991230+0000","last_change":"2024-09-19T08:41:47.974663+0000","last_active":"2024-09-19T08:41:47.991230+0000","last_peered":"2024-09-19T08:41:47.991230+0000","last_clean":"2024-09-19T08:41:47.991230+0000","last_became_active":"2024-09-19T08:41:47.972510+0000","last_became_peered":"2024-09-19T08:41:47.972510+0000","last_unstale":"2024-09-19T08:41:47.991230+0000","last_undegraded":"2024-09-19T08:41:47.991230+0000","last_fullsized":"2024-09-19T08:41:47.991230+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:41.615523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.940482+0000","last_change":"2024-09-19T08:41:46.958587+0000","last_active":"2024-09-19T08:41:47.940482+0000","last_peered":"2024-09-19T08:41:47.940482+0000","last_clean":"2024-09-19T08:41:47.940482+0000","last_became_active":"2024-09-19T08:41:46.958146+0000","last_became_peered":"2024-09-19T08:41:46.958146+0000","last_unstale":"2024-09-19T08:41:47.940482+0000","last_undegraded":"2024-09-19T08:41:47.940482+0000","last_fullsized":"2024-09-19T08:41:47.940482+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:42.541069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971266+0000","last_change":"2024-09-19T08:41:47.967496+0000","last_active":"2024-09-19T08:41:48.971266+0000","last_peered":"2024-09-19T08:41:48.971266+0000","last_clean":"2024-09-19T08:41:48.971266+0000","last_became_active":"2024-09-19T08:41:47.966684+0000","last_became_peered":"2024-09-19T08:41:47.966684+0000","last_unstale":"2024-09-19T08:41:48.971266+0000","last_undegraded":"2024-09-19T08:41:48.971266+0000","last_fullsized":"2024-09-19T08:41:48.971266+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:44:15.639843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-19T08:41:46.960637+0000","last_change":"2024-09-19T08:41:46.957237+0000","last_active":"2024-09-19T08:41:46.960637+0000","last_peered":"2024-09-19T08:41:46.960637+0000","last_clean":"2024-09-19T08:41:46.960637+0000","last_became_active":"2024-09-19T08:41:46.956480+0000","last_became_peered":"2024-09-19T08:41:46.956480+0000","last_unstale":"2024-09-19T08:41:46.960637+0000","last_undegraded":"2024-09-19T08:41:46.960637+0000","last_fullsized":"2024-09-19T08:41:46.960637+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:11.952600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.979216+0000","last_change":"2024-09-19T08:41:47.972895+0000","last_active":"2024-09-19T08:41:47.979216+0000","last_peered":"2024-09-19T08:41:47.979216+0000","last_clean":"2024-09-19T08:41:47.979216+0000","last_became_active":"2024-09-19T08:41:47.972572+0000","last_became_peered":"2024-09-19T08:41:47.972572+0000","last_unstale":"2024-09-19T08:41:47.979216+0000","last_undegraded":"2024-09-19T08:41:47.979216+0000","last_fullsized":"2024-09-19T08:41:47.979216+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:04:26.313098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.955154+0000","last_change":"2024-09-19T08:41:46.976718+0000","last_active":"2024-09-19T08:41:47.955154+0000","last_peered":"2024-09-19T08:41:47.955154+0000","last_clean":"2024-09-19T08:41:47.955154+0000","last_became_active":"2024-09-19T08:41:46.975905+0000","last_became_peered":"2024-09-19T08:41:46.975905+0000","last_unstale":"2024-09-19T08:41:47.955154+0000","last_undegraded":"2024-09-19T08:41:47.955154+0000","last_fullsized":"2024-09-19T08:41:47.955154+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:32:43.669648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.978612+0000","last_change":"2024-09-19T08:41:47.976156+0000","last_active":"2024-09-19T08:41:47.978612+0000","last_peered":"2024-09-19T08:41:47.978612+0000","last_clean":"2024-09-19T08:41:47.978612+0000","last_became_active":"2024-09-19T08:41:47.975738+0000","last_became_peered":"2024-09-19T08:41:47.975738+0000","last_unstale":"2024-09-19T08:41:47.978612+0000","last_undegraded":"2024-09-19T08:41:47.978612+0000","last_fullsized":"2024-09-19T08:41:47.978612+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:04:25.609497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.970965+0000","last_change":"2024-09-19T08:41:46.976793+0000","last_active":"2024-09-19T08:41:48.970965+0000","last_peered":"2024-09-19T08:41:48.970965+0000","last_clean":"2024-09-19T08:41:48.970965+0000","last_became_active":"2024-09-19T08:41:46.976404+0000","last_became_peered":"2024-09-19T08:41:46.976404+0000","last_unstale":"2024-09-19T08:41:48.970965+0000","last_undegraded":"2024-09-19T08:41:48.970965+0000","last_fullsized":"2024-09-19T08:41:48.970965+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:05.984736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.994385+0000","last_change":"2024-09-19T08:41:47.974129+0000","last_active":"2024-09-19T08:41:47.994385+0000","last_peered":"2024-09-19T08:41:47.994385+0000","last_clean":"2024-09-19T08:41:47.994385+0000","last_became_active":"2024-09-19T08:41:47.973452+0000","last_became_peered":"2024-09-19T08:41:47.973452+0000","last_unstale":"2024-09-19T08:41:47.994385+0000","last_undegraded":"2024-09-19T08:41:47.994385+0000","last_fullsized":"2024-09-19T08:41:47.994385+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:17.405379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971581+0000","last_change":"2024-09-19T08:41:46.958803+0000","last_active":"2024-09-19T08:41:48.971581+0000","last_peered":"2024-09-19T08:41:48.971581+0000","last_clean":"2024-09-19T08:41:48.971581+0000","last_became_active":"2024-09-19T08:41:46.957741+0000","last_became_peered":"2024-09-19T08:41:46.957741+0000","last_unstale":"2024-09-19T08:41:48.971581+0000","last_undegraded":"2024-09-19T08:41:48.971581+0000","last_fullsized":"2024-09-19T08:41:48.971581+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:28.813825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.970699+0000","last_change":"2024-09-19T08:41:47.977309+0000","last_active":"2024-09-19T08:41:48.970699+0000","last_peered":"2024-09-19T08:41:48.970699+0000","last_clean":"2024-09-19T08:41:48.970699+0000","last_became_active":"2024-09-19T08:41:47.976991+0000","last_became_peered":"2024-09-19T08:41:47.976991+0000","last_unstale":"2024-09-19T08:41:48.970699+0000","last_undegraded":"2024-09-19T08:41:48.970699+0000","last_fullsized":"2024-09-19T08:41:48.970699+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:56:40.725742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.970759+0000","last_change":"2024-09-19T08:41:46.954225+0000","last_active":"2024-09-19T08:41:48.970759+0000","last_peered":"2024-09-19T08:41:48.970759+0000","last_clean":"2024-09-19T08:41:48.970759+0000","last_became_active":"2024-09-19T08:41:46.953891+0000","last_became_peered":"2024-09-19T08:41:46.953891+0000","last_unstale":"2024-09-19T08:41:48.970759+0000","last_undegraded":"2024-09-19T08:41:48.970759+0000","last_fullsized":"2024-09-19T08:41:48.970759+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:59:22.248802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.984239+0000","last_change":"2024-09-19T08:41:47.978333+0000","last_active":"2024-09-19T08:41:47.984239+0000","last_peered":"2024-09-19T08:41:47.984239+0000","last_clean":"2024-09-19T08:41:47.984239+0000","last_became_active":"2024-09-19T08:41:47.977872+0000","last_became_peered":"2024-09-19T08:41:47.977872+0000","last_unstale":"2024-09-19T08:41:47.984239+0000","last_undegraded":"2024-09-19T08:41:47.984239+0000","last_fullsized":"2024-09-19T08:41:47.984239+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:05:29.016642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.965580+0000","last_change":"2024-09-19T08:41:46.971660+0000","last_active":"2024-09-19T08:41:47.965580+0000","last_peered":"2024-09-19T08:41:47.965580+0000","last_clean":"2024-09-19T08:41:47.965580+0000","last_became_active":"2024-09-19T08:41:46.971322+0000","last_became_peered":"2024-09-19T08:41:46.971322+0000","last_unstale":"2024-09-19T08:41:47.965580+0000","last_undegraded":"2024-09-19T08:41:47.965580+0000","last_fullsized":"2024-09-19T08:41:47.965580+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:42:15.264769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.971766+0000","last_change":"2024-09-19T08:41:47.970806+0000","last_active":"2024-09-19T08:41:47.971766+0000","last_peered":"2024-09-19T08:41:47.971766+0000","last_clean":"2024-09-19T08:41:47.971766+0000","last_became_active":"2024-09-19T08:41:47.970054+0000","last_became_peered":"2024-09-19T08:41:47.970054+0000","last_unstale":"2024-09-19T08:41:47.971766+0000","last_undegraded":"2024-09-19T08:41:47.971766+0000","last_fullsized":"2024-09-19T08:41:47.971766+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:15:45.598587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-19T08:41:46.981008+0000","last_change":"2024-09-19T08:41:46.978092+0000","last_active":"2024-09-19T08:41:46.981008+0000","last_peered":"2024-09-19T08:41:46.981008+0000","last_clean":"2024-09-19T08:41:46.981008+0000","last_became_active":"2024-09-19T08:41:46.977408+0000","last_became_peered":"2024-09-19T08:41:46.977408+0000","last_unstale":"2024-09-19T08:41:46.981008+0000","last_undegraded":"2024-09-19T08:41:46.981008+0000","last_fullsized":"2024-09-19T08:41:46.981008+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:58.851715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":231,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.958190+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:41:47.958190+0000","last_peered":"2024-09-19T08:41:47.958190+0000","last_clean":"2024-09-19T08:41:47.958190+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:41:47.958190+0000","last_undegraded":"2024-09-19T08:41:47.958190+0000","last_fullsized":"2024-09-19T08:41:47.958190+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509487,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1265664,"data_stored":863744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509489,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27688,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":671744,"data_stored":273868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509489,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27692,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":675840,"data_stored":274828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509486,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31844,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":671744,"data_stored":273272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509488,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31844,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":667648,"data_stored":273306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509486,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1273856,"data_stored":865778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574895,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31828,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340012,"statfs":{"total":96636764160,"available":96604172288,"internally_reserved":0,"allocated":651264,"data_stored":266441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542191,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31828,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340012,"statfs":{"total":96636764160,"available":96604172288,"internally_reserved":0,"allocated":651264,"data_stored":266441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:41:51.326 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:41:51.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.482+0000 7fb59a157640 1 Processor -- start 2024-09-19T08:41:51.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.482+0000 7fb59a157640 1 -- start start 2024-09-19T08:41:51.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.482+0000 7fb59a157640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb59415c550 0x7fb59406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:51.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.482+0000 7fb59a157640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb59406d310 con 0x7fb59406c1d0 2024-09-19T08:41:51.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.482+0000 7fb59a157640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb59406d5f0 con 0x7fb5940cfad0 2024-09-19T08:41:51.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.482+0000 7fb59a157640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb59406c610 con 0x7fb59415c550 2024-09-19T08:41:51.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.482+0000 7fb58affd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fb59406c1d0 0x7fb5940cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40072/0 (socket says 172.21.15.98:40072) 2024-09-19T08:41:51.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.482+0000 7fb58affd640 1 -- 172.21.15.98:0/1570917673 learned_addr learned my addr 172.21.15.98:0/1570917673 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:41:51.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.482+0000 7fb5937fe640 1 --2- 172.21.15.98:0/1570917673 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb59415c550 0x7fb59406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:51.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.482+0000 7fb592ffd640 1 -- 172.21.15.98:0/1570917673 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3044172931 0 0) 0x7fb59406d310 con 0x7fb59406c1d0 2024-09-19T08:41:51.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.482+0000 7fb592ffd640 1 -- 172.21.15.98:0/1570917673 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb580003530 con 0x7fb59406c1d0 2024-09-19T08:41:51.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.482+0000 7fb592ffd640 1 -- 172.21.15.98:0/1570917673 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 933332739 0 0) 0x7fb580003530 con 0x7fb59406c1d0 2024-09-19T08:41:51.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.482+0000 7fb592ffd640 1 -- 172.21.15.98:0/1570917673 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb59415c550 msgr2=0x7fb59406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:51.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.482+0000 7fb592ffd640 1 --2- 172.21.15.98:0/1570917673 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb59415c550 0x7fb59406bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:51.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.482+0000 7fb592ffd640 1 -- 172.21.15.98:0/1570917673 >> v1:172.21.15.139:6789/0 conn(0x7fb5940cfad0 legacy=0x7fb5940723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:51.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.482+0000 7fb592ffd640 1 -- 172.21.15.98:0/1570917673 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb59406d8d0 con 0x7fb59406c1d0 2024-09-19T08:41:51.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.482+0000 7fb592ffd640 1 -- 172.21.15.98:0/1570917673 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fb57c001680 con 0x7fb59406c1d0 2024-09-19T08:41:51.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.482+0000 7fb592ffd640 1 -- 172.21.15.98:0/1570917673 >> v1:172.21.15.98:6789/0 conn(0x7fb59406c1d0 legacy=0x7fb5940cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:51.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.482+0000 7fb592ffd640 1 --2- 172.21.15.98:0/1570917673 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb59415c550 0x7fb5800039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:51.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.482+0000 7fb592ffd640 1 --2- 172.21.15.98:0/1570917673 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb580003f70 0x7fb580004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:51.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.482+0000 7fb592ffd640 1 --2- 172.21.15.98:0/1570917673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb580004d50 0x7fb5800051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:51.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.482+0000 7fb592ffd640 1 -- 172.21.15.98:0/1570917673 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb59406c610 con 0x7fb580004d50 2024-09-19T08:41:51.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.482+0000 7fb592ffd640 1 -- 172.21.15.98:0/1570917673 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb58000bab0 con 0x7fb580003f70 2024-09-19T08:41:51.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.482+0000 7fb592ffd640 1 -- 172.21.15.98:0/1570917673 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb58000bcb0 con 0x7fb59415c550 2024-09-19T08:41:51.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.482+0000 7fb593fff640 1 --2- 172.21.15.98:0/1570917673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb580004d50 0x7fb5800051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:51.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.482+0000 7fb58affd640 1 --2- 172.21.15.98:0/1570917673 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb580003f70 0x7fb580004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:51.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.482+0000 7fb5937fe640 1 --2- 172.21.15.98:0/1570917673 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb59415c550 0x7fb5800039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:51.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.482+0000 7fb593fff640 1 -- 172.21.15.98:0/1570917673 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb59415c550 msgr2=0x7fb5800039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:51.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.482+0000 7fb593fff640 1 --2- 172.21.15.98:0/1570917673 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb59415c550 0x7fb5800039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:51.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.482+0000 7fb593fff640 1 -- 172.21.15.98:0/1570917673 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb580003f70 msgr2=0x7fb580004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:51.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.482+0000 7fb593fff640 1 --2- 172.21.15.98:0/1570917673 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb580003f70 0x7fb580004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:51.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.482+0000 7fb593fff640 1 -- 172.21.15.98:0/1570917673 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb580003530 con 0x7fb580004d50 2024-09-19T08:41:51.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.486+0000 7fb593fff640 1 --2- 172.21.15.98:0/1570917673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb580004d50 0x7fb5800051d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb58c001e70 tx=0x7fb58c01c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91adfe81a58a36e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:51.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.486+0000 7fb592ffd640 1 -- 172.21.15.98:0/1570917673 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c001a90 con 0x7fb580004d50 2024-09-19T08:41:51.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.486+0000 7fb592ffd640 1 -- 172.21.15.98:0/1570917673 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb58c003b80 con 0x7fb580004d50 2024-09-19T08:41:51.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.486+0000 7fb592ffd640 1 -- 172.21.15.98:0/1570917673 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58c005080 con 0x7fb580004d50 2024-09-19T08:41:51.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.486+0000 7fb59a157640 1 -- 172.21.15.98:0/1570917673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb580004d50 msgr2=0x7fb5800051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:51.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.486+0000 7fb59a157640 1 --2- 172.21.15.98:0/1570917673 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb580004d50 0x7fb5800051d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb58c001e70 tx=0x7fb58c01c780 comp rx=0 tx=0).stop 2024-09-19T08:41:51.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.486+0000 7fb593fff640 1 -- 172.21.15.98:0/1570917673 reap_dead start 2024-09-19T08:41:51.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.486+0000 7fb59a157640 1 -- 172.21.15.98:0/1570917673 shutdown_connections 2024-09-19T08:41:51.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.486+0000 7fb59a157640 1 -- 172.21.15.98:0/1570917673 >> 172.21.15.98:0/1570917673 conn(0x7fb5940b2010 msgr2=0x7fb5940b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:51.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.486+0000 7fb59a157640 1 -- 172.21.15.98:0/1570917673 shutdown_connections 2024-09-19T08:41:51.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.486+0000 7fb59a157640 1 -- 172.21.15.98:0/1570917673 wait complete. 2024-09-19T08:41:51.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.486+0000 7fb59a157640 1 Processor -- start 2024-09-19T08:41:51.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.486+0000 7fb59a157640 1 -- start start 2024-09-19T08:41:51.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.486+0000 7fb59a157640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb580004d50 0x7fb5941f2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:51.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.486+0000 7fb59a157640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb59415ccc0 0x7fb59414d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:51.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.486+0000 7fb59a157640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb594151620 0x7fb594151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:51.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.486+0000 7fb59a157640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb59415cab0 con 0x7fb59415ccc0 2024-09-19T08:41:51.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.486+0000 7fb59a157640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb59415c930 con 0x7fb580004d50 2024-09-19T08:41:51.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.486+0000 7fb59a157640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb59406bc90 con 0x7fb594151620 2024-09-19T08:41:51.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.490+0000 7fb58affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb59415ccc0 0x7fb59414d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:51.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.490+0000 7fb58affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb59415ccc0 0x7fb59414d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46986/0 (socket says 172.21.15.98:46986) 2024-09-19T08:41:51.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.490+0000 7fb58affd640 1 -- 172.21.15.98:0/4161450089 learned_addr learned my addr 172.21.15.98:0/4161450089 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:51.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.490+0000 7fb5937fe640 1 --2- 172.21.15.98:0/4161450089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb580004d50 0x7fb5941f2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:51.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.490+0000 7fb593fff640 1 --2- 172.21.15.98:0/4161450089 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb594151620 0x7fb594151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:51.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.490+0000 7fb58affd640 1 -- 172.21.15.98:0/4161450089 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb594151620 msgr2=0x7fb594151aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:51.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.490+0000 7fb58affd640 1 --2- 172.21.15.98:0/4161450089 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb594151620 0x7fb594151aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:51.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.490+0000 7fb58affd640 1 -- 172.21.15.98:0/4161450089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb580004d50 msgr2=0x7fb5941f2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:51.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.490+0000 7fb58affd640 1 --2- 172.21.15.98:0/4161450089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb580004d50 0x7fb5941f2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:51.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.490+0000 7fb58affd640 1 -- 172.21.15.98:0/4161450089 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb59415d4f0 con 0x7fb59415ccc0 2024-09-19T08:41:51.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.490+0000 7fb593fff640 1 --2- 172.21.15.98:0/4161450089 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb594151620 0x7fb594151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:51.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.490+0000 7fb58affd640 1 --2- 172.21.15.98:0/4161450089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb59415ccc0 0x7fb59414d940 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00e7f0 tx=0x7fb57c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:51.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.490+0000 7fb590ff9640 1 -- 172.21.15.98:0/4161450089 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c00c870 con 0x7fb59415ccc0 2024-09-19T08:41:51.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.490+0000 7fb590ff9640 1 -- 172.21.15.98:0/4161450089 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb57c018070 con 0x7fb59415ccc0 2024-09-19T08:41:51.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.490+0000 7fb590ff9640 1 -- 172.21.15.98:0/4161450089 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c00cc50 con 0x7fb59415ccc0 2024-09-19T08:41:51.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.490+0000 7fb59a157640 1 -- 172.21.15.98:0/4161450089 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb59415d7d0 con 0x7fb59415ccc0 2024-09-19T08:41:51.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.490+0000 7fb59a157640 1 -- 172.21.15.98:0/4161450089 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb594150e20 con 0x7fb59415ccc0 2024-09-19T08:41:51.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.490+0000 7fb59a157640 1 -- 172.21.15.98:0/4161450089 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5940cff80 con 0x7fb59415ccc0 2024-09-19T08:41:51.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.490+0000 7fb590ff9640 1 -- 172.21.15.98:0/4161450089 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fb57c01d460 con 0x7fb59415ccc0 2024-09-19T08:41:51.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.490+0000 7fb590ff9640 1 --2- 172.21.15.98:0/4161450089 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb574067aa0 0x7fb574069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:51.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.490+0000 7fb590ff9640 1 -- 172.21.15.98:0/4161450089 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fb57c085d40 con 0x7fb59415ccc0 2024-09-19T08:41:51.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.490+0000 7fb5937fe640 1 --2- 172.21.15.98:0/4161450089 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb574067aa0 0x7fb574069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:51.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.490+0000 7fb5937fe640 1 --2- 172.21.15.98:0/4161450089 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb574067aa0 0x7fb574069f60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb594153a80 tx=0x7fb584008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:51.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.494+0000 7fb590ff9640 1 -- 172.21.15.98:0/4161450089 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb57c056600 con 0x7fb59415ccc0 2024-09-19T08:41:51.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.638+0000 7fb59a157640 1 -- 172.21.15.98:0/4161450089 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb5940d0c40 con 0x7fb574067aa0 2024-09-19T08:41:51.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.658+0000 7fb590ff9640 1 -- 172.21.15.98:0/4161450089 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251655 (secure 0 0 0) 0x7fb5940d0c40 con 0x7fb574067aa0 2024-09-19T08:41:51.660 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:41:51.661 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:41:51.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.662+0000 7fb59a157640 1 -- 172.21.15.98:0/4161450089 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb574067aa0 msgr2=0x7fb574069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:51.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.662+0000 7fb59a157640 1 --2- 172.21.15.98:0/4161450089 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb574067aa0 0x7fb574069f60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb594153a80 tx=0x7fb584008040 comp rx=0 tx=0).stop 2024-09-19T08:41:51.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.662+0000 7fb59a157640 1 -- 172.21.15.98:0/4161450089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb59415ccc0 msgr2=0x7fb59414d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:51.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.662+0000 7fb59a157640 1 --2- 172.21.15.98:0/4161450089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb59415ccc0 0x7fb59414d940 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00e7f0 tx=0x7fb57c00ecb0 comp rx=0 tx=0).stop 2024-09-19T08:41:51.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.662+0000 7fb59a157640 1 -- 172.21.15.98:0/4161450089 shutdown_connections 2024-09-19T08:41:51.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.662+0000 7fb59a157640 1 --2- 172.21.15.98:0/4161450089 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb574067aa0 0x7fb574069f60 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:51.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.662+0000 7fb59a157640 1 --2- 172.21.15.98:0/4161450089 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb594151620 0x7fb594151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:51.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.662+0000 7fb59a157640 1 --2- 172.21.15.98:0/4161450089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb59415ccc0 0x7fb59414d940 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:51.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.662+0000 7fb59a157640 1 --2- 172.21.15.98:0/4161450089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb580004d50 0x7fb5941f2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:51.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.662+0000 7fb59a157640 1 -- 172.21.15.98:0/4161450089 >> 172.21.15.98:0/4161450089 conn(0x7fb5940b2010 msgr2=0x7fb5941599c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:51.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.662+0000 7fb59a157640 1 -- 172.21.15.98:0/4161450089 shutdown_connections 2024-09-19T08:41:51.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.662+0000 7fb59a157640 1 -- 172.21.15.98:0/4161450089 wait complete. 2024-09-19T08:41:51.677 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-09-19T08:41:50.096986+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":101,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":247596,"kb_used_data":6376,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754727124,"statfs":{"total":773094113280,"available":772840574976,"internally_reserved":0,"allocated":6529024,"data_stored":3357678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.175662"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.985587+0000","last_change":"2024-09-19T08:41:47.980122+0000","last_active":"2024-09-19T08:41:47.985587+0000","last_peered":"2024-09-19T08:41:47.985587+0000","last_clean":"2024-09-19T08:41:47.985587+0000","last_became_active":"2024-09-19T08:41:47.976398+0000","last_became_peered":"2024-09-19T08:41:47.976398+0000","last_unstale":"2024-09-19T08:41:47.985587+0000","last_undegraded":"2024-09-19T08:41:47.985587+0000","last_fullsized":"2024-09-19T08:41:47.985587+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:25.855696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.976086+0000","last_change":"2024-09-19T08:41:47.973338+0000","last_active":"2024-09-19T08:41:47.976086+0000","last_peered":"2024-09-19T08:41:47.976086+0000","last_clean":"2024-09-19T08:41:47.976086+0000","last_became_active":"2024-09-19T08:41:47.973065+0000","last_became_peered":"2024-09-19T08:41:47.973065+0000","last_unstale":"2024-09-19T08:41:47.976086+0000","last_undegraded":"2024-09-19T08:41:47.976086+0000","last_fullsized":"2024-09-19T08:41:47.976086+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:07.544296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.982165+0000","last_change":"2024-09-19T08:41:47.966283+0000","last_active":"2024-09-19T08:41:47.982165+0000","last_peered":"2024-09-19T08:41:47.982165+0000","last_clean":"2024-09-19T08:41:47.982165+0000","last_became_active":"2024-09-19T08:41:47.965563+0000","last_became_peered":"2024-09-19T08:41:47.965563+0000","last_unstale":"2024-09-19T08:41:47.982165+0000","last_undegraded":"2024-09-19T08:41:47.982165+0000","last_fullsized":"2024-09-19T08:41:47.982165+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:39:24.749819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.993880+0000","last_change":"2024-09-19T08:41:47.978137+0000","last_active":"2024-09-19T08:41:47.993880+0000","last_peered":"2024-09-19T08:41:47.993880+0000","last_clean":"2024-09-19T08:41:47.993880+0000","last_became_active":"2024-09-19T08:41:47.976050+0000","last_became_peered":"2024-09-19T08:41:47.976050+0000","last_unstale":"2024-09-19T08:41:47.993880+0000","last_undegraded":"2024-09-19T08:41:47.993880+0000","last_fullsized":"2024-09-19T08:41:47.993880+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:56:44.612767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.994306+0000","last_change":"2024-09-19T08:41:47.993040+0000","last_active":"2024-09-19T08:41:47.994306+0000","last_peered":"2024-09-19T08:41:47.994306+0000","last_clean":"2024-09-19T08:41:47.994306+0000","last_became_active":"2024-09-19T08:41:47.992767+0000","last_became_peered":"2024-09-19T08:41:47.992767+0000","last_unstale":"2024-09-19T08:41:47.994306+0000","last_undegraded":"2024-09-19T08:41:47.994306+0000","last_fullsized":"2024-09-19T08:41:47.994306+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:54.206986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.964412+0000","last_change":"2024-09-19T08:41:47.961838+0000","last_active":"2024-09-19T08:41:47.964412+0000","last_peered":"2024-09-19T08:41:47.964412+0000","last_clean":"2024-09-19T08:41:47.964412+0000","last_became_active":"2024-09-19T08:41:47.961116+0000","last_became_peered":"2024-09-19T08:41:47.961116+0000","last_unstale":"2024-09-19T08:41:47.964412+0000","last_undegraded":"2024-09-19T08:41:47.964412+0000","last_fullsized":"2024-09-19T08:41:47.964412+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:41.615523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971387+0000","last_change":"2024-09-19T08:41:47.966503+0000","last_active":"2024-09-19T08:41:48.971387+0000","last_peered":"2024-09-19T08:41:48.971387+0000","last_clean":"2024-09-19T08:41:48.971387+0000","last_became_active":"2024-09-19T08:41:47.965805+0000","last_became_peered":"2024-09-19T08:41:47.965805+0000","last_unstale":"2024-09-19T08:41:48.971387+0000","last_undegraded":"2024-09-19T08:41:48.971387+0000","last_fullsized":"2024-09-19T08:41:48.971387+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:41.615523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.971097+0000","last_change":"2024-09-19T08:41:47.969144+0000","last_active":"2024-09-19T08:41:47.971097+0000","last_peered":"2024-09-19T08:41:47.971097+0000","last_clean":"2024-09-19T08:41:47.971097+0000","last_became_active":"2024-09-19T08:41:47.968766+0000","last_became_peered":"2024-09-19T08:41:47.968766+0000","last_unstale":"2024-09-19T08:41:47.971097+0000","last_undegraded":"2024-09-19T08:41:47.971097+0000","last_fullsized":"2024-09-19T08:41:47.971097+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:20:19.600052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971951+0000","last_change":"2024-09-19T08:41:47.954679+0000","last_active":"2024-09-19T08:41:48.971951+0000","last_peered":"2024-09-19T08:41:48.971951+0000","last_clean":"2024-09-19T08:41:48.971951+0000","last_became_active":"2024-09-19T08:41:47.951685+0000","last_became_peered":"2024-09-19T08:41:47.951685+0000","last_unstale":"2024-09-19T08:41:48.971951+0000","last_undegraded":"2024-09-19T08:41:48.971951+0000","last_fullsized":"2024-09-19T08:41:48.971951+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:17.405379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971133+0000","last_change":"2024-09-19T08:41:47.977064+0000","last_active":"2024-09-19T08:41:48.971133+0000","last_peered":"2024-09-19T08:41:48.971133+0000","last_clean":"2024-09-19T08:41:48.971133+0000","last_became_active":"2024-09-19T08:41:47.976747+0000","last_became_peered":"2024-09-19T08:41:47.976747+0000","last_unstale":"2024-09-19T08:41:48.971133+0000","last_undegraded":"2024-09-19T08:41:48.971133+0000","last_fullsized":"2024-09-19T08:41:48.971133+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:37.366531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971332+0000","last_change":"2024-09-19T08:41:47.971592+0000","last_active":"2024-09-19T08:41:48.971332+0000","last_peered":"2024-09-19T08:41:48.971332+0000","last_clean":"2024-09-19T08:41:48.971332+0000","last_became_active":"2024-09-19T08:41:47.969075+0000","last_became_peered":"2024-09-19T08:41:47.969075+0000","last_unstale":"2024-09-19T08:41:48.971332+0000","last_undegraded":"2024-09-19T08:41:48.971332+0000","last_fullsized":"2024-09-19T08:41:48.971332+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:29.817497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.992937+0000","last_change":"2024-09-19T08:41:47.968996+0000","last_active":"2024-09-19T08:41:47.992937+0000","last_peered":"2024-09-19T08:41:47.992937+0000","last_clean":"2024-09-19T08:41:47.992937+0000","last_became_active":"2024-09-19T08:41:47.968254+0000","last_became_peered":"2024-09-19T08:41:47.968254+0000","last_unstale":"2024-09-19T08:41:47.992937+0000","last_undegraded":"2024-09-19T08:41:47.992937+0000","last_fullsized":"2024-09-19T08:41:47.992937+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:12.552026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-19T08:41:46.952023+0000","last_change":"2024-09-19T08:41:46.952023+0000","last_active":"2024-09-19T08:41:46.920839+0000","last_peered":"2024-09-19T08:41:46.920839+0000","last_clean":"2024-09-19T08:41:46.920839+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:41:46.952023+0000","last_undegraded":"2024-09-19T08:41:46.952023+0000","last_fullsized":"2024-09-19T08:41:46.952023+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971528+0000","last_change":"2024-09-19T08:41:47.962145+0000","last_active":"2024-09-19T08:41:48.971528+0000","last_peered":"2024-09-19T08:41:48.971528+0000","last_clean":"2024-09-19T08:41:48.971528+0000","last_became_active":"2024-09-19T08:41:47.961449+0000","last_became_peered":"2024-09-19T08:41:47.961449+0000","last_unstale":"2024-09-19T08:41:48.971528+0000","last_undegraded":"2024-09-19T08:41:48.971528+0000","last_fullsized":"2024-09-19T08:41:48.971528+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:09.481675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.972047+0000","last_change":"2024-09-19T08:41:47.960027+0000","last_active":"2024-09-19T08:41:48.972047+0000","last_peered":"2024-09-19T08:41:48.972047+0000","last_clean":"2024-09-19T08:41:48.972047+0000","last_became_active":"2024-09-19T08:41:47.959242+0000","last_became_peered":"2024-09-19T08:41:47.959242+0000","last_unstale":"2024-09-19T08:41:48.972047+0000","last_undegraded":"2024-09-19T08:41:48.972047+0000","last_fullsized":"2024-09-19T08:41:48.972047+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:56:44.612767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.984338+0000","last_change":"2024-09-19T08:41:47.979371+0000","last_active":"2024-09-19T08:41:47.984338+0000","last_peered":"2024-09-19T08:41:47.984338+0000","last_clean":"2024-09-19T08:41:47.984338+0000","last_became_active":"2024-09-19T08:41:47.978819+0000","last_became_peered":"2024-09-19T08:41:47.978819+0000","last_unstale":"2024-09-19T08:41:47.984338+0000","last_undegraded":"2024-09-19T08:41:47.984338+0000","last_fullsized":"2024-09-19T08:41:47.984338+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:34.451800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.981562+0000","last_change":"2024-09-19T08:41:47.977880+0000","last_active":"2024-09-19T08:41:47.981562+0000","last_peered":"2024-09-19T08:41:47.981562+0000","last_clean":"2024-09-19T08:41:47.981562+0000","last_became_active":"2024-09-19T08:41:47.977590+0000","last_became_peered":"2024-09-19T08:41:47.977590+0000","last_unstale":"2024-09-19T08:41:47.981562+0000","last_undegraded":"2024-09-19T08:41:47.981562+0000","last_fullsized":"2024-09-19T08:41:47.981562+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:43:07.334696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.982300+0000","last_change":"2024-09-19T08:41:47.968677+0000","last_active":"2024-09-19T08:41:47.982300+0000","last_peered":"2024-09-19T08:41:47.982300+0000","last_clean":"2024-09-19T08:41:47.982300+0000","last_became_active":"2024-09-19T08:41:47.967937+0000","last_became_peered":"2024-09-19T08:41:47.967937+0000","last_unstale":"2024-09-19T08:41:47.982300+0000","last_undegraded":"2024-09-19T08:41:47.982300+0000","last_fullsized":"2024-09-19T08:41:47.982300+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:32:44.673319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.981344+0000","last_change":"2024-09-19T08:41:47.975831+0000","last_active":"2024-09-19T08:41:47.981344+0000","last_peered":"2024-09-19T08:41:47.981344+0000","last_clean":"2024-09-19T08:41:47.981344+0000","last_became_active":"2024-09-19T08:41:47.975518+0000","last_became_peered":"2024-09-19T08:41:47.975518+0000","last_unstale":"2024-09-19T08:41:47.981344+0000","last_undegraded":"2024-09-19T08:41:47.981344+0000","last_fullsized":"2024-09-19T08:41:47.981344+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:23:47.530098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.976135+0000","last_change":"2024-09-19T08:41:47.973832+0000","last_active":"2024-09-19T08:41:47.976135+0000","last_peered":"2024-09-19T08:41:47.976135+0000","last_clean":"2024-09-19T08:41:47.976135+0000","last_became_active":"2024-09-19T08:41:47.973516+0000","last_became_peered":"2024-09-19T08:41:47.973516+0000","last_unstale":"2024-09-19T08:41:47.976135+0000","last_undegraded":"2024-09-19T08:41:47.976135+0000","last_fullsized":"2024-09-19T08:41:47.976135+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:43:59.145791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.994338+0000","last_change":"2024-09-19T08:41:47.979480+0000","last_active":"2024-09-19T08:41:47.994338+0000","last_peered":"2024-09-19T08:41:47.994338+0000","last_clean":"2024-09-19T08:41:47.994338+0000","last_became_active":"2024-09-19T08:41:47.977765+0000","last_became_peered":"2024-09-19T08:41:47.977765+0000","last_unstale":"2024-09-19T08:41:47.994338+0000","last_undegraded":"2024-09-19T08:41:47.994338+0000","last_fullsized":"2024-09-19T08:41:47.994338+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:41.615523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.996753+0000","last_change":"2024-09-19T08:41:47.981514+0000","last_active":"2024-09-19T08:41:47.996753+0000","last_peered":"2024-09-19T08:41:47.996753+0000","last_clean":"2024-09-19T08:41:47.996753+0000","last_became_active":"2024-09-19T08:41:47.981044+0000","last_became_peered":"2024-09-19T08:41:47.981044+0000","last_unstale":"2024-09-19T08:41:47.996753+0000","last_undegraded":"2024-09-19T08:41:47.996753+0000","last_fullsized":"2024-09-19T08:41:47.996753+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:07.796396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.993777+0000","last_change":"2024-09-19T08:41:47.970880+0000","last_active":"2024-09-19T08:41:47.993777+0000","last_peered":"2024-09-19T08:41:47.993777+0000","last_clean":"2024-09-19T08:41:47.993777+0000","last_became_active":"2024-09-19T08:41:47.970145+0000","last_became_peered":"2024-09-19T08:41:47.970145+0000","last_unstale":"2024-09-19T08:41:47.993777+0000","last_undegraded":"2024-09-19T08:41:47.993777+0000","last_fullsized":"2024-09-19T08:41:47.993777+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:52:41.224293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-19T08:41:46.949524+0000","last_change":"2024-09-19T08:41:46.949524+0000","last_active":"2024-09-19T08:41:46.920839+0000","last_peered":"2024-09-19T08:41:46.920839+0000","last_clean":"2024-09-19T08:41:46.920839+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:41:46.949524+0000","last_undegraded":"2024-09-19T08:41:46.949524+0000","last_fullsized":"2024-09-19T08:41:46.949524+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.985564+0000","last_change":"2024-09-19T08:41:47.977025+0000","last_active":"2024-09-19T08:41:47.985564+0000","last_peered":"2024-09-19T08:41:47.985564+0000","last_clean":"2024-09-19T08:41:47.985564+0000","last_became_active":"2024-09-19T08:41:47.975707+0000","last_became_peered":"2024-09-19T08:41:47.975707+0000","last_unstale":"2024-09-19T08:41:47.985564+0000","last_undegraded":"2024-09-19T08:41:47.985564+0000","last_fullsized":"2024-09-19T08:41:47.985564+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:17.908127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.990912+0000","last_change":"2024-09-19T08:41:47.966128+0000","last_active":"2024-09-19T08:41:47.990912+0000","last_peered":"2024-09-19T08:41:47.990912+0000","last_clean":"2024-09-19T08:41:47.990912+0000","last_became_active":"2024-09-19T08:41:47.965403+0000","last_became_peered":"2024-09-19T08:41:47.965403+0000","last_unstale":"2024-09-19T08:41:47.990912+0000","last_undegraded":"2024-09-19T08:41:47.990912+0000","last_fullsized":"2024-09-19T08:41:47.990912+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:43.544741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.981730+0000","last_change":"2024-09-19T08:41:47.979320+0000","last_active":"2024-09-19T08:41:47.981730+0000","last_peered":"2024-09-19T08:41:47.981730+0000","last_clean":"2024-09-19T08:41:47.981730+0000","last_became_active":"2024-09-19T08:41:47.979010+0000","last_became_peered":"2024-09-19T08:41:47.979010+0000","last_unstale":"2024-09-19T08:41:47.981730+0000","last_undegraded":"2024-09-19T08:41:47.981730+0000","last_fullsized":"2024-09-19T08:41:47.981730+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:44:04.680315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.975042+0000","last_change":"2024-09-19T08:41:47.966338+0000","last_active":"2024-09-19T08:41:47.975042+0000","last_peered":"2024-09-19T08:41:47.975042+0000","last_clean":"2024-09-19T08:41:47.975042+0000","last_became_active":"2024-09-19T08:41:47.966024+0000","last_became_peered":"2024-09-19T08:41:47.966024+0000","last_unstale":"2024-09-19T08:41:47.975042+0000","last_undegraded":"2024-09-19T08:41:47.975042+0000","last_fullsized":"2024-09-19T08:41:47.975042+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:45.927550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.997659+0000","last_change":"2024-09-19T08:41:47.996502+0000","last_active":"2024-09-19T08:41:47.997659+0000","last_peered":"2024-09-19T08:41:47.997659+0000","last_clean":"2024-09-19T08:41:47.997659+0000","last_became_active":"2024-09-19T08:41:47.996162+0000","last_became_peered":"2024-09-19T08:41:47.996162+0000","last_unstale":"2024-09-19T08:41:47.997659+0000","last_undegraded":"2024-09-19T08:41:47.997659+0000","last_fullsized":"2024-09-19T08:41:47.997659+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:13.341860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.982174+0000","last_change":"2024-09-19T08:41:47.966508+0000","last_active":"2024-09-19T08:41:47.982174+0000","last_peered":"2024-09-19T08:41:47.982174+0000","last_clean":"2024-09-19T08:41:47.982174+0000","last_became_active":"2024-09-19T08:41:47.962098+0000","last_became_peered":"2024-09-19T08:41:47.962098+0000","last_unstale":"2024-09-19T08:41:47.982174+0000","last_undegraded":"2024-09-19T08:41:47.982174+0000","last_fullsized":"2024-09-19T08:41:47.982174+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:23.999718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.981287+0000","last_change":"2024-09-19T08:41:47.974969+0000","last_active":"2024-09-19T08:41:47.981287+0000","last_peered":"2024-09-19T08:41:47.981287+0000","last_clean":"2024-09-19T08:41:47.981287+0000","last_became_active":"2024-09-19T08:41:47.974440+0000","last_became_peered":"2024-09-19T08:41:47.974440+0000","last_unstale":"2024-09-19T08:41:47.981287+0000","last_undegraded":"2024-09-19T08:41:47.981287+0000","last_fullsized":"2024-09-19T08:41:47.981287+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:35:19.862729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.978537+0000","last_change":"2024-09-19T08:41:47.972260+0000","last_active":"2024-09-19T08:41:47.978537+0000","last_peered":"2024-09-19T08:41:47.978537+0000","last_clean":"2024-09-19T08:41:47.978537+0000","last_became_active":"2024-09-19T08:41:47.971683+0000","last_became_peered":"2024-09-19T08:41:47.971683+0000","last_unstale":"2024-09-19T08:41:47.978537+0000","last_undegraded":"2024-09-19T08:41:47.978537+0000","last_fullsized":"2024-09-19T08:41:47.978537+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:56:44.612767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.964666+0000","last_change":"2024-09-19T08:41:47.962374+0000","last_active":"2024-09-19T08:41:47.964666+0000","last_peered":"2024-09-19T08:41:47.964666+0000","last_clean":"2024-09-19T08:41:47.964666+0000","last_became_active":"2024-09-19T08:41:47.961305+0000","last_became_peered":"2024-09-19T08:41:47.961305+0000","last_unstale":"2024-09-19T08:41:47.964666+0000","last_undegraded":"2024-09-19T08:41:47.964666+0000","last_fullsized":"2024-09-19T08:41:47.964666+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:09.481675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"56'4","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.001343+0000","last_change":"2024-09-19T08:41:46.976847+0000","last_active":"2024-09-19T08:41:48.001343+0000","last_peered":"2024-09-19T08:41:48.001343+0000","last_clean":"2024-09-19T08:41:48.001343+0000","last_became_active":"2024-09-19T08:41:46.976405+0000","last_became_peered":"2024-09-19T08:41:46.976405+0000","last_unstale":"2024-09-19T08:41:48.001343+0000","last_undegraded":"2024-09-19T08:41:48.001343+0000","last_fullsized":"2024-09-19T08:41:48.001343+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:52:40.220622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971359+0000","last_change":"2024-09-19T08:41:47.971091+0000","last_active":"2024-09-19T08:41:48.971359+0000","last_peered":"2024-09-19T08:41:48.971359+0000","last_clean":"2024-09-19T08:41:48.971359+0000","last_became_active":"2024-09-19T08:41:47.958633+0000","last_became_peered":"2024-09-19T08:41:47.958633+0000","last_unstale":"2024-09-19T08:41:48.971359+0000","last_undegraded":"2024-09-19T08:41:48.971359+0000","last_fullsized":"2024-09-19T08:41:48.971359+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:38.634694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"56'1","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.997195+0000","last_change":"2024-09-19T08:41:46.953976+0000","last_active":"2024-09-19T08:41:47.997195+0000","last_peered":"2024-09-19T08:41:47.997195+0000","last_clean":"2024-09-19T08:41:47.997195+0000","last_became_active":"2024-09-19T08:41:46.953613+0000","last_became_peered":"2024-09-19T08:41:46.953613+0000","last_unstale":"2024-09-19T08:41:47.997195+0000","last_undegraded":"2024-09-19T08:41:47.997195+0000","last_fullsized":"2024-09-19T08:41:47.997195+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:24.976526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.964324+0000","last_change":"2024-09-19T08:41:47.961793+0000","last_active":"2024-09-19T08:41:47.964324+0000","last_peered":"2024-09-19T08:41:47.964324+0000","last_clean":"2024-09-19T08:41:47.964324+0000","last_became_active":"2024-09-19T08:41:47.961008+0000","last_became_peered":"2024-09-19T08:41:47.961008+0000","last_unstale":"2024-09-19T08:41:47.964324+0000","last_undegraded":"2024-09-19T08:41:47.964324+0000","last_fullsized":"2024-09-19T08:41:47.964324+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:42.104552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"56'2","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.000690+0000","last_change":"2024-09-19T08:41:46.950525+0000","last_active":"2024-09-19T08:41:48.000690+0000","last_peered":"2024-09-19T08:41:48.000690+0000","last_clean":"2024-09-19T08:41:48.000690+0000","last_became_active":"2024-09-19T08:41:46.950185+0000","last_became_peered":"2024-09-19T08:41:46.950185+0000","last_unstale":"2024-09-19T08:41:48.000690+0000","last_undegraded":"2024-09-19T08:41:48.000690+0000","last_fullsized":"2024-09-19T08:41:48.000690+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:11.548355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.999907+0000","last_change":"2024-09-19T08:41:47.997038+0000","last_active":"2024-09-19T08:41:47.999907+0000","last_peered":"2024-09-19T08:41:47.999907+0000","last_clean":"2024-09-19T08:41:47.999907+0000","last_became_active":"2024-09-19T08:41:47.996654+0000","last_became_peered":"2024-09-19T08:41:47.996654+0000","last_unstale":"2024-09-19T08:41:47.999907+0000","last_undegraded":"2024-09-19T08:41:47.999907+0000","last_fullsized":"2024-09-19T08:41:47.999907+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:25:54.925434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"56'2","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.001947+0000","last_change":"2024-09-19T08:41:46.976094+0000","last_active":"2024-09-19T08:41:48.001947+0000","last_peered":"2024-09-19T08:41:48.001947+0000","last_clean":"2024-09-19T08:41:48.001947+0000","last_became_active":"2024-09-19T08:41:46.975349+0000","last_became_peered":"2024-09-19T08:41:46.975349+0000","last_unstale":"2024-09-19T08:41:48.001947+0000","last_undegraded":"2024-09-19T08:41:48.001947+0000","last_fullsized":"2024-09-19T08:41:48.001947+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:22.996047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.984243+0000","last_change":"2024-09-19T08:41:47.981348+0000","last_active":"2024-09-19T08:41:47.984243+0000","last_peered":"2024-09-19T08:41:47.984243+0000","last_clean":"2024-09-19T08:41:47.984243+0000","last_became_active":"2024-09-19T08:41:47.980623+0000","last_became_peered":"2024-09-19T08:41:47.980623+0000","last_unstale":"2024-09-19T08:41:47.984243+0000","last_undegraded":"2024-09-19T08:41:47.984243+0000","last_fullsized":"2024-09-19T08:41:47.984243+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:21:26.704824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971966+0000","last_change":"2024-09-19T08:41:46.959290+0000","last_active":"2024-09-19T08:41:48.971966+0000","last_peered":"2024-09-19T08:41:48.971966+0000","last_clean":"2024-09-19T08:41:48.971966+0000","last_became_active":"2024-09-19T08:41:46.958454+0000","last_became_peered":"2024-09-19T08:41:46.958454+0000","last_unstale":"2024-09-19T08:41:48.971966+0000","last_undegraded":"2024-09-19T08:41:48.971966+0000","last_fullsized":"2024-09-19T08:41:48.971966+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:39:23.746148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.971630+0000","last_change":"2024-09-19T08:41:47.969748+0000","last_active":"2024-09-19T08:41:47.971630+0000","last_peered":"2024-09-19T08:41:47.971630+0000","last_clean":"2024-09-19T08:41:47.971630+0000","last_became_active":"2024-09-19T08:41:47.969443+0000","last_became_peered":"2024-09-19T08:41:47.969443+0000","last_unstale":"2024-09-19T08:41:47.971630+0000","last_undegraded":"2024-09-19T08:41:47.971630+0000","last_fullsized":"2024-09-19T08:41:47.971630+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:47:48.598741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"56'1","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.999759+0000","last_change":"2024-09-19T08:41:46.956247+0000","last_active":"2024-09-19T08:41:47.999759+0000","last_peered":"2024-09-19T08:41:47.999759+0000","last_clean":"2024-09-19T08:41:47.999759+0000","last_became_active":"2024-09-19T08:41:46.955941+0000","last_became_peered":"2024-09-19T08:41:46.955941+0000","last_unstale":"2024-09-19T08:41:47.999759+0000","last_undegraded":"2024-09-19T08:41:47.999759+0000","last_fullsized":"2024-09-19T08:41:47.999759+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:22.996047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-19T08:41:46.948430+0000","last_change":"2024-09-19T08:41:46.948430+0000","last_active":"2024-09-19T08:41:46.920839+0000","last_peered":"2024-09-19T08:41:46.920839+0000","last_clean":"2024-09-19T08:41:46.920839+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:41:46.948430+0000","last_undegraded":"2024-09-19T08:41:46.948430+0000","last_fullsized":"2024-09-19T08:41:46.948430+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.950542+0000","last_change":"2024-09-19T08:41:46.976878+0000","last_active":"2024-09-19T08:41:47.950542+0000","last_peered":"2024-09-19T08:41:47.950542+0000","last_clean":"2024-09-19T08:41:47.950542+0000","last_became_active":"2024-09-19T08:41:46.975427+0000","last_became_peered":"2024-09-19T08:41:46.975427+0000","last_unstale":"2024-09-19T08:41:47.950542+0000","last_undegraded":"2024-09-19T08:41:47.950542+0000","last_fullsized":"2024-09-19T08:41:47.950542+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:41:25.735590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.979456+0000","last_change":"2024-09-19T08:41:47.977608+0000","last_active":"2024-09-19T08:41:47.979456+0000","last_peered":"2024-09-19T08:41:47.979456+0000","last_clean":"2024-09-19T08:41:47.979456+0000","last_became_active":"2024-09-19T08:41:47.977328+0000","last_became_peered":"2024-09-19T08:41:47.977328+0000","last_unstale":"2024-09-19T08:41:47.979456+0000","last_undegraded":"2024-09-19T08:41:47.979456+0000","last_fullsized":"2024-09-19T08:41:47.979456+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:38.632932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971395+0000","last_change":"2024-09-19T08:41:46.951849+0000","last_active":"2024-09-19T08:41:48.971395+0000","last_peered":"2024-09-19T08:41:48.971395+0000","last_clean":"2024-09-19T08:41:48.971395+0000","last_became_active":"2024-09-19T08:41:46.951049+0000","last_became_peered":"2024-09-19T08:41:46.951049+0000","last_unstale":"2024-09-19T08:41:48.971395+0000","last_undegraded":"2024-09-19T08:41:48.971395+0000","last_fullsized":"2024-09-19T08:41:48.971395+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:37.631023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.972453+0000","last_change":"2024-09-19T08:41:47.959107+0000","last_active":"2024-09-19T08:41:48.972453+0000","last_peered":"2024-09-19T08:41:48.972453+0000","last_clean":"2024-09-19T08:41:48.972453+0000","last_became_active":"2024-09-19T08:41:47.958257+0000","last_became_peered":"2024-09-19T08:41:47.958257+0000","last_unstale":"2024-09-19T08:41:48.972453+0000","last_undegraded":"2024-09-19T08:41:48.972453+0000","last_fullsized":"2024-09-19T08:41:48.972453+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:04:26.313098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.949274+0000","last_change":"2024-09-19T08:41:46.960020+0000","last_active":"2024-09-19T08:41:47.949274+0000","last_peered":"2024-09-19T08:41:47.949274+0000","last_clean":"2024-09-19T08:41:47.949274+0000","last_became_active":"2024-09-19T08:41:46.959232+0000","last_became_peered":"2024-09-19T08:41:46.959232+0000","last_unstale":"2024-09-19T08:41:47.949274+0000","last_undegraded":"2024-09-19T08:41:47.949274+0000","last_fullsized":"2024-09-19T08:41:47.949274+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:28.813825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.993989+0000","last_change":"2024-09-19T08:41:47.973711+0000","last_active":"2024-09-19T08:41:47.993989+0000","last_peered":"2024-09-19T08:41:47.993989+0000","last_clean":"2024-09-19T08:41:47.993989+0000","last_became_active":"2024-09-19T08:41:47.972968+0000","last_became_peered":"2024-09-19T08:41:47.972968+0000","last_unstale":"2024-09-19T08:41:47.993989+0000","last_undegraded":"2024-09-19T08:41:47.993989+0000","last_fullsized":"2024-09-19T08:41:47.993989+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:12.552026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.964270+0000","last_change":"2024-09-19T08:41:46.957881+0000","last_active":"2024-09-19T08:41:47.964270+0000","last_peered":"2024-09-19T08:41:47.964270+0000","last_clean":"2024-09-19T08:41:47.964270+0000","last_became_active":"2024-09-19T08:41:46.957503+0000","last_became_peered":"2024-09-19T08:41:46.957503+0000","last_unstale":"2024-09-19T08:41:47.964270+0000","last_undegraded":"2024-09-19T08:41:47.964270+0000","last_fullsized":"2024-09-19T08:41:47.964270+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:02:24.036512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971244+0000","last_change":"2024-09-19T08:41:47.968178+0000","last_active":"2024-09-19T08:41:48.971244+0000","last_peered":"2024-09-19T08:41:48.971244+0000","last_clean":"2024-09-19T08:41:48.971244+0000","last_became_active":"2024-09-19T08:41:47.967773+0000","last_became_peered":"2024-09-19T08:41:47.967773+0000","last_unstale":"2024-09-19T08:41:48.971244+0000","last_undegraded":"2024-09-19T08:41:48.971244+0000","last_fullsized":"2024-09-19T08:41:48.971244+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:21.843717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"56'3","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.002671+0000","last_change":"2024-09-19T08:41:46.957368+0000","last_active":"2024-09-19T08:41:48.002671+0000","last_peered":"2024-09-19T08:41:48.002671+0000","last_clean":"2024-09-19T08:41:48.002671+0000","last_became_active":"2024-09-19T08:41:46.956670+0000","last_became_peered":"2024-09-19T08:41:46.956670+0000","last_unstale":"2024-09-19T08:41:48.002671+0000","last_undegraded":"2024-09-19T08:41:48.002671+0000","last_fullsized":"2024-09-19T08:41:48.002671+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:37.631023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.002229+0000","last_change":"2024-09-19T08:41:47.977022+0000","last_active":"2024-09-19T08:41:48.002229+0000","last_peered":"2024-09-19T08:41:48.002229+0000","last_clean":"2024-09-19T08:41:48.002229+0000","last_became_active":"2024-09-19T08:41:47.973384+0000","last_became_peered":"2024-09-19T08:41:47.973384+0000","last_unstale":"2024-09-19T08:41:48.002229+0000","last_undegraded":"2024-09-19T08:41:48.002229+0000","last_fullsized":"2024-09-19T08:41:48.002229+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:42.104552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"56'6","reported_seq":20,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.972332+0000","last_change":"2024-09-19T08:41:46.953997+0000","last_active":"2024-09-19T08:41:48.972332+0000","last_peered":"2024-09-19T08:41:48.972332+0000","last_clean":"2024-09-19T08:41:48.972332+0000","last_became_active":"2024-09-19T08:41:46.953261+0000","last_became_peered":"2024-09-19T08:41:46.953261+0000","last_unstale":"2024-09-19T08:41:48.972332+0000","last_undegraded":"2024-09-19T08:41:48.972332+0000","last_fullsized":"2024-09-19T08:41:48.972332+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:41:25.735590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.991181+0000","last_change":"2024-09-19T08:41:47.973617+0000","last_active":"2024-09-19T08:41:47.991181+0000","last_peered":"2024-09-19T08:41:47.991181+0000","last_clean":"2024-09-19T08:41:47.991181+0000","last_became_active":"2024-09-19T08:41:47.973096+0000","last_became_peered":"2024-09-19T08:41:47.973096+0000","last_unstale":"2024-09-19T08:41:47.991181+0000","last_undegraded":"2024-09-19T08:41:47.991181+0000","last_fullsized":"2024-09-19T08:41:47.991181+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:42.104552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"56'2","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.996894+0000","last_change":"2024-09-19T08:41:46.958209+0000","last_active":"2024-09-19T08:41:47.996894+0000","last_peered":"2024-09-19T08:41:47.996894+0000","last_clean":"2024-09-19T08:41:47.996894+0000","last_became_active":"2024-09-19T08:41:46.957655+0000","last_became_peered":"2024-09-19T08:41:46.957655+0000","last_unstale":"2024-09-19T08:41:47.996894+0000","last_undegraded":"2024-09-19T08:41:47.996894+0000","last_fullsized":"2024-09-19T08:41:47.996894+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:06.027506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.994435+0000","last_change":"2024-09-19T08:41:47.980616+0000","last_active":"2024-09-19T08:41:47.994435+0000","last_peered":"2024-09-19T08:41:47.994435+0000","last_clean":"2024-09-19T08:41:47.994435+0000","last_became_active":"2024-09-19T08:41:47.978716+0000","last_became_peered":"2024-09-19T08:41:47.978716+0000","last_unstale":"2024-09-19T08:41:47.994435+0000","last_undegraded":"2024-09-19T08:41:47.994435+0000","last_fullsized":"2024-09-19T08:41:47.994435+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:56:44.612767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-19T08:41:46.966764+0000","last_change":"2024-09-19T08:41:46.963078+0000","last_active":"2024-09-19T08:41:46.966764+0000","last_peered":"2024-09-19T08:41:46.966764+0000","last_clean":"2024-09-19T08:41:46.966764+0000","last_became_active":"2024-09-19T08:41:46.962049+0000","last_became_peered":"2024-09-19T08:41:46.962049+0000","last_unstale":"2024-09-19T08:41:46.966764+0000","last_undegraded":"2024-09-19T08:41:46.966764+0000","last_fullsized":"2024-09-19T08:41:46.966764+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:18.215821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971307+0000","last_change":"2024-09-19T08:41:47.985774+0000","last_active":"2024-09-19T08:41:48.971307+0000","last_peered":"2024-09-19T08:41:48.971307+0000","last_clean":"2024-09-19T08:41:48.971307+0000","last_became_active":"2024-09-19T08:41:47.984953+0000","last_became_peered":"2024-09-19T08:41:47.984953+0000","last_unstale":"2024-09-19T08:41:48.971307+0000","last_undegraded":"2024-09-19T08:41:48.971307+0000","last_fullsized":"2024-09-19T08:41:48.971307+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:42.104552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.950609+0000","last_change":"2024-09-19T08:41:46.973134+0000","last_active":"2024-09-19T08:41:47.950609+0000","last_peered":"2024-09-19T08:41:47.950609+0000","last_clean":"2024-09-19T08:41:47.950609+0000","last_became_active":"2024-09-19T08:41:46.971993+0000","last_became_peered":"2024-09-19T08:41:46.971993+0000","last_unstale":"2024-09-19T08:41:47.950609+0000","last_undegraded":"2024-09-19T08:41:47.950609+0000","last_fullsized":"2024-09-19T08:41:47.950609+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:39:23.746148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.993733+0000","last_change":"2024-09-19T08:41:47.970744+0000","last_active":"2024-09-19T08:41:47.993733+0000","last_peered":"2024-09-19T08:41:47.993733+0000","last_clean":"2024-09-19T08:41:47.993733+0000","last_became_active":"2024-09-19T08:41:47.969645+0000","last_became_peered":"2024-09-19T08:41:47.969645+0000","last_unstale":"2024-09-19T08:41:47.993733+0000","last_undegraded":"2024-09-19T08:41:47.993733+0000","last_fullsized":"2024-09-19T08:41:47.993733+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:52.726130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"56'8","reported_seq":20,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.002402+0000","last_change":"2024-09-19T08:41:46.971824+0000","last_active":"2024-09-19T08:41:48.002402+0000","last_peered":"2024-09-19T08:41:48.002402+0000","last_clean":"2024-09-19T08:41:48.002402+0000","last_became_active":"2024-09-19T08:41:46.971096+0000","last_became_peered":"2024-09-19T08:41:46.971096+0000","last_unstale":"2024-09-19T08:41:48.002402+0000","last_undegraded":"2024-09-19T08:41:48.002402+0000","last_fullsized":"2024-09-19T08:41:48.002402+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:41:25.735590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.994127+0000","last_change":"2024-09-19T08:41:47.972240+0000","last_active":"2024-09-19T08:41:47.994127+0000","last_peered":"2024-09-19T08:41:47.994127+0000","last_clean":"2024-09-19T08:41:47.994127+0000","last_became_active":"2024-09-19T08:41:47.971284+0000","last_became_peered":"2024-09-19T08:41:47.971284+0000","last_unstale":"2024-09-19T08:41:47.994127+0000","last_undegraded":"2024-09-19T08:41:47.994127+0000","last_fullsized":"2024-09-19T08:41:47.994127+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:43.544741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"56'1","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.989173+0000","last_change":"2024-09-19T08:41:46.956816+0000","last_active":"2024-09-19T08:41:47.989173+0000","last_peered":"2024-09-19T08:41:47.989173+0000","last_clean":"2024-09-19T08:41:47.989173+0000","last_became_active":"2024-09-19T08:41:46.956468+0000","last_became_peered":"2024-09-19T08:41:46.956468+0000","last_unstale":"2024-09-19T08:41:47.989173+0000","last_undegraded":"2024-09-19T08:41:47.989173+0000","last_fullsized":"2024-09-19T08:41:47.989173+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:11.860203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.991287+0000","last_change":"2024-09-19T08:41:47.974760+0000","last_active":"2024-09-19T08:41:47.991287+0000","last_peered":"2024-09-19T08:41:47.991287+0000","last_clean":"2024-09-19T08:41:47.991287+0000","last_became_active":"2024-09-19T08:41:47.972626+0000","last_became_peered":"2024-09-19T08:41:47.972626+0000","last_unstale":"2024-09-19T08:41:47.991287+0000","last_undegraded":"2024-09-19T08:41:47.991287+0000","last_fullsized":"2024-09-19T08:41:47.991287+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:09.481675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"56'2","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.000795+0000","last_change":"2024-09-19T08:41:46.956922+0000","last_active":"2024-09-19T08:41:48.000795+0000","last_peered":"2024-09-19T08:41:48.000795+0000","last_clean":"2024-09-19T08:41:48.000795+0000","last_became_active":"2024-09-19T08:41:46.956049+0000","last_became_peered":"2024-09-19T08:41:46.956049+0000","last_unstale":"2024-09-19T08:41:48.000795+0000","last_undegraded":"2024-09-19T08:41:48.000795+0000","last_fullsized":"2024-09-19T08:41:48.000795+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:51.722459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.988936+0000","last_change":"2024-09-19T08:41:47.987128+0000","last_active":"2024-09-19T08:41:47.988936+0000","last_peered":"2024-09-19T08:41:47.988936+0000","last_clean":"2024-09-19T08:41:47.988936+0000","last_became_active":"2024-09-19T08:41:47.986850+0000","last_became_peered":"2024-09-19T08:41:47.986850+0000","last_unstale":"2024-09-19T08:41:47.988936+0000","last_undegraded":"2024-09-19T08:41:47.988936+0000","last_fullsized":"2024-09-19T08:41:47.988936+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:13.337414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"56'1","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.002911+0000","last_change":"2024-09-19T08:41:46.971406+0000","last_active":"2024-09-19T08:41:48.002911+0000","last_peered":"2024-09-19T08:41:48.002911+0000","last_clean":"2024-09-19T08:41:48.002911+0000","last_became_active":"2024-09-19T08:41:46.970580+0000","last_became_peered":"2024-09-19T08:41:46.970580+0000","last_unstale":"2024-09-19T08:41:48.002911+0000","last_undegraded":"2024-09-19T08:41:48.002911+0000","last_fullsized":"2024-09-19T08:41:48.002911+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:52:40.220622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.002262+0000","last_change":"2024-09-19T08:41:47.979548+0000","last_active":"2024-09-19T08:41:48.002262+0000","last_peered":"2024-09-19T08:41:48.002262+0000","last_clean":"2024-09-19T08:41:48.002262+0000","last_became_active":"2024-09-19T08:41:47.978787+0000","last_became_peered":"2024-09-19T08:41:47.978787+0000","last_unstale":"2024-09-19T08:41:48.002262+0000","last_undegraded":"2024-09-19T08:41:48.002262+0000","last_fullsized":"2024-09-19T08:41:48.002262+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:09.481675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.939420+0000","last_change":"2024-09-19T08:41:46.958787+0000","last_active":"2024-09-19T08:41:47.939420+0000","last_peered":"2024-09-19T08:41:47.939420+0000","last_clean":"2024-09-19T08:41:47.939420+0000","last_became_active":"2024-09-19T08:41:46.958288+0000","last_became_peered":"2024-09-19T08:41:46.958288+0000","last_unstale":"2024-09-19T08:41:47.939420+0000","last_undegraded":"2024-09-19T08:41:47.939420+0000","last_fullsized":"2024-09-19T08:41:47.939420+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:32:43.669648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.991121+0000","last_change":"2024-09-19T08:41:47.959094+0000","last_active":"2024-09-19T08:41:47.991121+0000","last_peered":"2024-09-19T08:41:47.991121+0000","last_clean":"2024-09-19T08:41:47.991121+0000","last_became_active":"2024-09-19T08:41:47.958309+0000","last_became_peered":"2024-09-19T08:41:47.958309+0000","last_unstale":"2024-09-19T08:41:47.991121+0000","last_undegraded":"2024-09-19T08:41:47.991121+0000","last_fullsized":"2024-09-19T08:41:47.991121+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:52.726130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"56'2","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.996240+0000","last_change":"2024-09-19T08:41:46.955663+0000","last_active":"2024-09-19T08:41:47.996240+0000","last_peered":"2024-09-19T08:41:47.996240+0000","last_clean":"2024-09-19T08:41:47.996240+0000","last_became_active":"2024-09-19T08:41:46.955237+0000","last_became_peered":"2024-09-19T08:41:46.955237+0000","last_unstale":"2024-09-19T08:41:47.996240+0000","last_undegraded":"2024-09-19T08:41:47.996240+0000","last_fullsized":"2024-09-19T08:41:47.996240+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:49:41.535703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.981832+0000","last_change":"2024-09-19T08:41:47.979199+0000","last_active":"2024-09-19T08:41:47.981832+0000","last_peered":"2024-09-19T08:41:47.981832+0000","last_clean":"2024-09-19T08:41:47.981832+0000","last_became_active":"2024-09-19T08:41:47.978858+0000","last_became_peered":"2024-09-19T08:41:47.978858+0000","last_unstale":"2024-09-19T08:41:47.981832+0000","last_undegraded":"2024-09-19T08:41:47.981832+0000","last_fullsized":"2024-09-19T08:41:47.981832+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:08:18.593800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.970964+0000","last_change":"2024-09-19T08:41:46.957163+0000","last_active":"2024-09-19T08:41:48.970964+0000","last_peered":"2024-09-19T08:41:48.970964+0000","last_clean":"2024-09-19T08:41:48.970964+0000","last_became_active":"2024-09-19T08:41:46.956745+0000","last_became_peered":"2024-09-19T08:41:46.956745+0000","last_unstale":"2024-09-19T08:41:48.970964+0000","last_undegraded":"2024-09-19T08:41:48.970964+0000","last_fullsized":"2024-09-19T08:41:48.970964+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:51:54.055870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.965010+0000","last_change":"2024-09-19T08:41:47.962932+0000","last_active":"2024-09-19T08:41:47.965010+0000","last_peered":"2024-09-19T08:41:47.965010+0000","last_clean":"2024-09-19T08:41:47.965010+0000","last_became_active":"2024-09-19T08:41:47.962192+0000","last_became_peered":"2024-09-19T08:41:47.962192+0000","last_unstale":"2024-09-19T08:41:47.965010+0000","last_undegraded":"2024-09-19T08:41:47.965010+0000","last_fullsized":"2024-09-19T08:41:47.965010+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:17.405379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.949954+0000","last_change":"2024-09-19T08:41:46.970587+0000","last_active":"2024-09-19T08:41:47.949954+0000","last_peered":"2024-09-19T08:41:47.949954+0000","last_clean":"2024-09-19T08:41:47.949954+0000","last_became_active":"2024-09-19T08:41:46.968910+0000","last_became_peered":"2024-09-19T08:41:46.968910+0000","last_unstale":"2024-09-19T08:41:47.949954+0000","last_undegraded":"2024-09-19T08:41:47.949954+0000","last_fullsized":"2024-09-19T08:41:47.949954+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:37.631023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.993141+0000","last_change":"2024-09-19T08:41:47.975389+0000","last_active":"2024-09-19T08:41:47.993141+0000","last_peered":"2024-09-19T08:41:47.993141+0000","last_clean":"2024-09-19T08:41:47.993141+0000","last_became_active":"2024-09-19T08:41:47.974667+0000","last_became_peered":"2024-09-19T08:41:47.974667+0000","last_unstale":"2024-09-19T08:41:47.993141+0000","last_undegraded":"2024-09-19T08:41:47.993141+0000","last_fullsized":"2024-09-19T08:41:47.993141+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:17.405379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971357+0000","last_change":"2024-09-19T08:41:46.949050+0000","last_active":"2024-09-19T08:41:48.971357+0000","last_peered":"2024-09-19T08:41:48.971357+0000","last_clean":"2024-09-19T08:41:48.971357+0000","last_became_active":"2024-09-19T08:41:46.948768+0000","last_became_peered":"2024-09-19T08:41:46.948768+0000","last_unstale":"2024-09-19T08:41:48.971357+0000","last_undegraded":"2024-09-19T08:41:48.971357+0000","last_fullsized":"2024-09-19T08:41:48.971357+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:25.916697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.991230+0000","last_change":"2024-09-19T08:41:47.974663+0000","last_active":"2024-09-19T08:41:47.991230+0000","last_peered":"2024-09-19T08:41:47.991230+0000","last_clean":"2024-09-19T08:41:47.991230+0000","last_became_active":"2024-09-19T08:41:47.972510+0000","last_became_peered":"2024-09-19T08:41:47.972510+0000","last_unstale":"2024-09-19T08:41:47.991230+0000","last_undegraded":"2024-09-19T08:41:47.991230+0000","last_fullsized":"2024-09-19T08:41:47.991230+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:41.615523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.940482+0000","last_change":"2024-09-19T08:41:46.958587+0000","last_active":"2024-09-19T08:41:47.940482+0000","last_peered":"2024-09-19T08:41:47.940482+0000","last_clean":"2024-09-19T08:41:47.940482+0000","last_became_active":"2024-09-19T08:41:46.958146+0000","last_became_peered":"2024-09-19T08:41:46.958146+0000","last_unstale":"2024-09-19T08:41:47.940482+0000","last_undegraded":"2024-09-19T08:41:47.940482+0000","last_fullsized":"2024-09-19T08:41:47.940482+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:42.541069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971266+0000","last_change":"2024-09-19T08:41:47.967496+0000","last_active":"2024-09-19T08:41:48.971266+0000","last_peered":"2024-09-19T08:41:48.971266+0000","last_clean":"2024-09-19T08:41:48.971266+0000","last_became_active":"2024-09-19T08:41:47.966684+0000","last_became_peered":"2024-09-19T08:41:47.966684+0000","last_unstale":"2024-09-19T08:41:48.971266+0000","last_undegraded":"2024-09-19T08:41:48.971266+0000","last_fullsized":"2024-09-19T08:41:48.971266+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:44:15.639843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-19T08:41:46.960637+0000","last_change":"2024-09-19T08:41:46.957237+0000","last_active":"2024-09-19T08:41:46.960637+0000","last_peered":"2024-09-19T08:41:46.960637+0000","last_clean":"2024-09-19T08:41:46.960637+0000","last_became_active":"2024-09-19T08:41:46.956480+0000","last_became_peered":"2024-09-19T08:41:46.956480+0000","last_unstale":"2024-09-19T08:41:46.960637+0000","last_undegraded":"2024-09-19T08:41:46.960637+0000","last_fullsized":"2024-09-19T08:41:46.960637+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:11.952600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.979216+0000","last_change":"2024-09-19T08:41:47.972895+0000","last_active":"2024-09-19T08:41:47.979216+0000","last_peered":"2024-09-19T08:41:47.979216+0000","last_clean":"2024-09-19T08:41:47.979216+0000","last_became_active":"2024-09-19T08:41:47.972572+0000","last_became_peered":"2024-09-19T08:41:47.972572+0000","last_unstale":"2024-09-19T08:41:47.979216+0000","last_undegraded":"2024-09-19T08:41:47.979216+0000","last_fullsized":"2024-09-19T08:41:47.979216+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:04:26.313098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.955154+0000","last_change":"2024-09-19T08:41:46.976718+0000","last_active":"2024-09-19T08:41:47.955154+0000","last_peered":"2024-09-19T08:41:47.955154+0000","last_clean":"2024-09-19T08:41:47.955154+0000","last_became_active":"2024-09-19T08:41:46.975905+0000","last_became_peered":"2024-09-19T08:41:46.975905+0000","last_unstale":"2024-09-19T08:41:47.955154+0000","last_undegraded":"2024-09-19T08:41:47.955154+0000","last_fullsized":"2024-09-19T08:41:47.955154+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:32:43.669648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.978612+0000","last_change":"2024-09-19T08:41:47.976156+0000","last_active":"2024-09-19T08:41:47.978612+0000","last_peered":"2024-09-19T08:41:47.978612+0000","last_clean":"2024-09-19T08:41:47.978612+0000","last_became_active":"2024-09-19T08:41:47.975738+0000","last_became_peered":"2024-09-19T08:41:47.975738+0000","last_unstale":"2024-09-19T08:41:47.978612+0000","last_undegraded":"2024-09-19T08:41:47.978612+0000","last_fullsized":"2024-09-19T08:41:47.978612+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:04:25.609497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.970965+0000","last_change":"2024-09-19T08:41:46.976793+0000","last_active":"2024-09-19T08:41:48.970965+0000","last_peered":"2024-09-19T08:41:48.970965+0000","last_clean":"2024-09-19T08:41:48.970965+0000","last_became_active":"2024-09-19T08:41:46.976404+0000","last_became_peered":"2024-09-19T08:41:46.976404+0000","last_unstale":"2024-09-19T08:41:48.970965+0000","last_undegraded":"2024-09-19T08:41:48.970965+0000","last_fullsized":"2024-09-19T08:41:48.970965+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:05.984736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.994385+0000","last_change":"2024-09-19T08:41:47.974129+0000","last_active":"2024-09-19T08:41:47.994385+0000","last_peered":"2024-09-19T08:41:47.994385+0000","last_clean":"2024-09-19T08:41:47.994385+0000","last_became_active":"2024-09-19T08:41:47.973452+0000","last_became_peered":"2024-09-19T08:41:47.973452+0000","last_unstale":"2024-09-19T08:41:47.994385+0000","last_undegraded":"2024-09-19T08:41:47.994385+0000","last_fullsized":"2024-09-19T08:41:47.994385+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:17.405379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971581+0000","last_change":"2024-09-19T08:41:46.958803+0000","last_active":"2024-09-19T08:41:48.971581+0000","last_peered":"2024-09-19T08:41:48.971581+0000","last_clean":"2024-09-19T08:41:48.971581+0000","last_became_active":"2024-09-19T08:41:46.957741+0000","last_became_peered":"2024-09-19T08:41:46.957741+0000","last_unstale":"2024-09-19T08:41:48.971581+0000","last_undegraded":"2024-09-19T08:41:48.971581+0000","last_fullsized":"2024-09-19T08:41:48.971581+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:28.813825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.970699+0000","last_change":"2024-09-19T08:41:47.977309+0000","last_active":"2024-09-19T08:41:48.970699+0000","last_peered":"2024-09-19T08:41:48.970699+0000","last_clean":"2024-09-19T08:41:48.970699+0000","last_became_active":"2024-09-19T08:41:47.976991+0000","last_became_peered":"2024-09-19T08:41:47.976991+0000","last_unstale":"2024-09-19T08:41:48.970699+0000","last_undegraded":"2024-09-19T08:41:48.970699+0000","last_fullsized":"2024-09-19T08:41:48.970699+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:56:40.725742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.970759+0000","last_change":"2024-09-19T08:41:46.954225+0000","last_active":"2024-09-19T08:41:48.970759+0000","last_peered":"2024-09-19T08:41:48.970759+0000","last_clean":"2024-09-19T08:41:48.970759+0000","last_became_active":"2024-09-19T08:41:46.953891+0000","last_became_peered":"2024-09-19T08:41:46.953891+0000","last_unstale":"2024-09-19T08:41:48.970759+0000","last_undegraded":"2024-09-19T08:41:48.970759+0000","last_fullsized":"2024-09-19T08:41:48.970759+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:59:22.248802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.984239+0000","last_change":"2024-09-19T08:41:47.978333+0000","last_active":"2024-09-19T08:41:47.984239+0000","last_peered":"2024-09-19T08:41:47.984239+0000","last_clean":"2024-09-19T08:41:47.984239+0000","last_became_active":"2024-09-19T08:41:47.977872+0000","last_became_peered":"2024-09-19T08:41:47.977872+0000","last_unstale":"2024-09-19T08:41:47.984239+0000","last_undegraded":"2024-09-19T08:41:47.984239+0000","last_fullsized":"2024-09-19T08:41:47.984239+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:05:29.016642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.965580+0000","last_change":"2024-09-19T08:41:46.971660+0000","last_active":"2024-09-19T08:41:47.965580+0000","last_peered":"2024-09-19T08:41:47.965580+0000","last_clean":"2024-09-19T08:41:47.965580+0000","last_became_active":"2024-09-19T08:41:46.971322+0000","last_became_peered":"2024-09-19T08:41:46.971322+0000","last_unstale":"2024-09-19T08:41:47.965580+0000","last_undegraded":"2024-09-19T08:41:47.965580+0000","last_fullsized":"2024-09-19T08:41:47.965580+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:42:15.264769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.971766+0000","last_change":"2024-09-19T08:41:47.970806+0000","last_active":"2024-09-19T08:41:47.971766+0000","last_peered":"2024-09-19T08:41:47.971766+0000","last_clean":"2024-09-19T08:41:47.971766+0000","last_became_active":"2024-09-19T08:41:47.970054+0000","last_became_peered":"2024-09-19T08:41:47.970054+0000","last_unstale":"2024-09-19T08:41:47.971766+0000","last_undegraded":"2024-09-19T08:41:47.971766+0000","last_fullsized":"2024-09-19T08:41:47.971766+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:15:45.598587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-19T08:41:46.981008+0000","last_change":"2024-09-19T08:41:46.978092+0000","last_active":"2024-09-19T08:41:46.981008+0000","last_peered":"2024-09-19T08:41:46.981008+0000","last_clean":"2024-09-19T08:41:46.981008+0000","last_became_active":"2024-09-19T08:41:46.977408+0000","last_became_peered":"2024-09-19T08:41:46.977408+0000","last_unstale":"2024-09-19T08:41:46.981008+0000","last_undegraded":"2024-09-19T08:41:46.981008+0000","last_fullsized":"2024-09-19T08:41:46.981008+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:58.851715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":231,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.958190+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:41:47.958190+0000","last_peered":"2024-09-19T08:41:47.958190+0000","last_clean":"2024-09-19T08:41:47.958190+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:41:47.958190+0000","last_undegraded":"2024-09-19T08:41:47.958190+0000","last_fullsized":"2024-09-19T08:41:47.958190+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509487,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1265664,"data_stored":863744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509489,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27688,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":671744,"data_stored":273868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509489,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27692,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":675840,"data_stored":274828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509486,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31844,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":671744,"data_stored":273272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509488,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31844,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":667648,"data_stored":273306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509486,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1273856,"data_stored":865778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574895,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31828,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340012,"statfs":{"total":96636764160,"available":96604172288,"internally_reserved":0,"allocated":651264,"data_stored":266441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542191,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31828,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340012,"statfs":{"total":96636764160,"available":96604172288,"internally_reserved":0,"allocated":651264,"data_stored":266441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:41:51.680 INFO:tasks.ceph.ceph_manager.ceph:PG 9.33 is not active+clean 2024-09-19T08:41:51.681 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:41:46.952023+0000', 'last_change': '2024-09-19T08:41:46.952023+0000', 'last_active': '2024-09-19T08:41:46.920839+0000', 'last_peered': '2024-09-19T08:41:46.920839+0000', 'last_clean': '2024-09-19T08:41:46.920839+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:41:46.952023+0000', 'last_undegraded': '2024-09-19T08:41:46.952023+0000', 'last_fullsized': '2024-09-19T08:41:46.952023+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:41:46.920839+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:41:46.920839+0000', 'last_clean_scrub_stamp': '2024-09-19T08:41:46.920839+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-19T08:41:51.681 INFO:tasks.ceph.ceph_manager.ceph:PG 9.28 is not active+clean 2024-09-19T08:41:51.681 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:41:46.949524+0000', 'last_change': '2024-09-19T08:41:46.949524+0000', 'last_active': '2024-09-19T08:41:46.920839+0000', 'last_peered': '2024-09-19T08:41:46.920839+0000', 'last_clean': '2024-09-19T08:41:46.920839+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:41:46.949524+0000', 'last_undegraded': '2024-09-19T08:41:46.949524+0000', 'last_fullsized': '2024-09-19T08:41:46.949524+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:41:46.920839+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:41:46.920839+0000', 'last_clean_scrub_stamp': '2024-09-19T08:41:46.920839+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-19T08:41:51.681 INFO:tasks.ceph.ceph_manager.ceph:PG 9.18 is not active+clean 2024-09-19T08:41:51.681 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '9.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:41:46.948430+0000', 'last_change': '2024-09-19T08:41:46.948430+0000', 'last_active': '2024-09-19T08:41:46.920839+0000', 'last_peered': '2024-09-19T08:41:46.920839+0000', 'last_clean': '2024-09-19T08:41:46.920839+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:41:46.948430+0000', 'last_undegraded': '2024-09-19T08:41:46.948430+0000', 'last_fullsized': '2024-09-19T08:41:46.948430+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:41:46.920839+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:41:46.920839+0000', 'last_clean_scrub_stamp': '2024-09-19T08:41:46.920839+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-19T08:41:51.681 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T08:41:51.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.838+0000 7f02755f2640 1 Processor -- start 2024-09-19T08:41:51.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.838+0000 7f02755f2640 1 -- start start 2024-09-19T08:41:51.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.838+0000 7f02755f2640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f02700b2800 0x7f02700cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:51.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.838+0000 7f02755f2640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f027015b120 con 0x7f0270154f70 2024-09-19T08:41:51.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.838+0000 7f02755f2640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f027015b3e0 con 0x7f02700cfe00 2024-09-19T08:41:51.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.838+0000 7f02755f2640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f027015b6c0 con 0x7f02700b2800 2024-09-19T08:41:51.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.842+0000 7f026f7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f02700b2800 0x7f02700cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:51.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.842+0000 7f026effd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f0270154f70 0x7f02700d0d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40082/0 (socket says 172.21.15.98:40082) 2024-09-19T08:41:51.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.842+0000 7f026effd640 1 -- 172.21.15.98:0/3558371999 learned_addr learned my addr 172.21.15.98:0/3558371999 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:41:51.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.842+0000 7f026f7fe640 1 -- 172.21.15.98:0/3558371999 >> v1:172.21.15.139:6789/0 conn(0x7f02700cfe00 legacy=0x7f02700b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:51.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.842+0000 7f026f7fe640 1 -- 172.21.15.98:0/3558371999 >> v1:172.21.15.98:6789/0 conn(0x7f0270154f70 legacy=0x7f02700d0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:51.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.842+0000 7f026f7fe640 1 -- 172.21.15.98:0/3558371999 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f027015b9a0 con 0x7f02700b2800 2024-09-19T08:41:51.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.842+0000 7f026f7fe640 1 --2- 172.21.15.98:0/3558371999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02700b2800 0x7f02700cd0c0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0268009d90 tx=0x7f026801da40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0f83b42ce2e51e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:51.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.842+0000 7f026dffb640 1 -- 172.21.15.98:0/3558371999 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02680b3070 con 0x7f02700b2800 2024-09-19T08:41:51.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.842+0000 7f026dffb640 1 -- 172.21.15.98:0/3558371999 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02680a6ac0 con 0x7f02700b2800 2024-09-19T08:41:51.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.842+0000 7f026dffb640 1 -- 172.21.15.98:0/3558371999 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02680ae440 con 0x7f02700b2800 2024-09-19T08:41:51.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.842+0000 7f02755f2640 1 -- 172.21.15.98:0/3558371999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02700b2800 msgr2=0x7f02700cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:51.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.842+0000 7f02755f2640 1 --2- 172.21.15.98:0/3558371999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02700b2800 0x7f02700cd0c0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0268009d90 tx=0x7f026801da40 comp rx=0 tx=0).stop 2024-09-19T08:41:51.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.842+0000 7f02755f2640 1 -- 172.21.15.98:0/3558371999 shutdown_connections 2024-09-19T08:41:51.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.842+0000 7f02755f2640 1 --2- 172.21.15.98:0/3558371999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02700b2800 0x7f02700cd0c0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:51.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.842+0000 7f02755f2640 1 -- 172.21.15.98:0/3558371999 >> 172.21.15.98:0/3558371999 conn(0x7f027006bc90 msgr2=0x7f027006c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:51.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.842+0000 7f02755f2640 1 -- 172.21.15.98:0/3558371999 shutdown_connections 2024-09-19T08:41:51.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.842+0000 7f02755f2640 1 -- 172.21.15.98:0/3558371999 wait complete. 2024-09-19T08:41:51.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.846+0000 7f02755f2640 1 Processor -- start 2024-09-19T08:41:51.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.846+0000 7f02755f2640 1 -- start start 2024-09-19T08:41:51.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.846+0000 7f02755f2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02700b2800 0x7f0270150d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:51.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.846+0000 7f02755f2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02700cfe00 0x7f027014b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:51.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.846+0000 7f02755f2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0270154f70 0x7f02701526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:51.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.846+0000 7f02755f2640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f02700b5040 con 0x7f02700cfe00 2024-09-19T08:41:51.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.846+0000 7f02755f2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f02700b4ec0 con 0x7f02700b2800 2024-09-19T08:41:51.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.846+0000 7f02755f2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f02700b51c0 con 0x7f0270154f70 2024-09-19T08:41:51.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.846+0000 7f026e7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02700cfe00 0x7f027014b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:51.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.846+0000 7f026e7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02700cfe00 0x7f027014b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47008/0 (socket says 172.21.15.98:47008) 2024-09-19T08:41:51.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.846+0000 7f026e7fc640 1 -- 172.21.15.98:0/1512889997 learned_addr learned my addr 172.21.15.98:0/1512889997 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:51.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.846+0000 7f026f7fe640 1 --2- 172.21.15.98:0/1512889997 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0270154f70 0x7f02701526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:51.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.846+0000 7f026effd640 1 --2- 172.21.15.98:0/1512889997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02700b2800 0x7f0270150d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:51.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.846+0000 7f026e7fc640 1 -- 172.21.15.98:0/1512889997 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0270154f70 msgr2=0x7f02701526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:51.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.846+0000 7f026e7fc640 1 --2- 172.21.15.98:0/1512889997 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0270154f70 0x7f02701526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:51.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.846+0000 7f026e7fc640 1 -- 172.21.15.98:0/1512889997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02700b2800 msgr2=0x7f0270150d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:51.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.846+0000 7f026e7fc640 1 --2- 172.21.15.98:0/1512889997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02700b2800 0x7f0270150d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:51.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.846+0000 7f026e7fc640 1 -- 172.21.15.98:0/1512889997 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f027014ba70 con 0x7f02700cfe00 2024-09-19T08:41:51.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.846+0000 7f026f7fe640 1 --2- 172.21.15.98:0/1512889997 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0270154f70 0x7f02701526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:51.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.846+0000 7f026effd640 1 --2- 172.21.15.98:0/1512889997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02700b2800 0x7f0270150d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:51.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.846+0000 7f026e7fc640 1 --2- 172.21.15.98:0/1512889997 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02700cfe00 0x7f027014b530 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f026000c490 tx=0x7f0260001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:51.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.846+0000 7f02677fe640 1 -- 172.21.15.98:0/1512889997 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0260017070 con 0x7f02700cfe00 2024-09-19T08:41:51.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.846+0000 7f02677fe640 1 -- 172.21.15.98:0/1512889997 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0260003600 con 0x7f02700cfe00 2024-09-19T08:41:51.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.846+0000 7f02677fe640 1 -- 172.21.15.98:0/1512889997 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0260003a30 con 0x7f02700cfe00 2024-09-19T08:41:51.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.846+0000 7f02755f2640 1 -- 172.21.15.98:0/1512889997 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f027014fc80 con 0x7f02700cfe00 2024-09-19T08:41:51.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.846+0000 7f02755f2640 1 -- 172.21.15.98:0/1512889997 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0270150240 con 0x7f02700cfe00 2024-09-19T08:41:51.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.846+0000 7f02755f2640 1 -- 172.21.15.98:0/1512889997 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02700cd0c0 con 0x7f02700cfe00 2024-09-19T08:41:51.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.850+0000 7f02677fe640 1 -- 172.21.15.98:0/1512889997 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f0260003bd0 con 0x7f02700cfe00 2024-09-19T08:41:51.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.850+0000 7f02677fe640 1 --2- 172.21.15.98:0/1512889997 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0250067a70 0x7f0250069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:51.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.850+0000 7f02677fe640 1 -- 172.21.15.98:0/1512889997 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f02600851f0 con 0x7f02700cfe00 2024-09-19T08:41:51.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.850+0000 7f026effd640 1 --2- 172.21.15.98:0/1512889997 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0250067a70 0x7f0250069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:51.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.850+0000 7f026effd640 1 --2- 172.21.15.98:0/1512889997 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0250067a70 0x7f0250069f30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0258007fd0 tx=0x7f0258009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:51.853 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:51.854+0000 7f02677fe640 1 -- 172.21.15.98:0/1512889997 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f0260055b40 con 0x7f02700cfe00 2024-09-19T08:41:52.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.070+0000 7f02755f2640 1 -- 172.21.15.98:0/1512889997 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f027014c720 con 0x7f02700cfe00 2024-09-19T08:41:52.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.070+0000 7f02677fe640 1 -- 172.21.15.98:0/1512889997 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1142 (secure 0 0 0) 0x7f0260055440 con 0x7f02700cfe00 2024-09-19T08:41:52.070 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:41:52.070 INFO:teuthology.orchestra.run.smithi098.stdout:{"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":236,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":8,"num_up_osds":8,"osd_up_since":1726735088,"num_in_osds":8,"osd_in_since":1726735077,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":94},{"state_name":"creating+peering","count":3}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592778,"bytes_used":253538304,"bytes_avail":772840574976,"bytes_total":773094113280,"inactive_pgs_ratio":0.030927835032343864,"write_bytes_sec":3433,"read_op_per_sec":0,"write_op_per_sec":10},"fsmap":{"epoch":30,"btime":"2024-09-19T08:41:49:966866+0000","id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"a","status":"up:active","gid":5748}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-19T08:40:46.073688+0000","services":{}},"progress_events":{}} 2024-09-19T08:41:52.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.070+0000 7f02755f2640 1 -- 172.21.15.98:0/1512889997 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0250067a70 msgr2=0x7f0250069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:52.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.070+0000 7f02755f2640 1 --2- 172.21.15.98:0/1512889997 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0250067a70 0x7f0250069f30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0258007fd0 tx=0x7f0258009040 comp rx=0 tx=0).stop 2024-09-19T08:41:52.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.070+0000 7f02755f2640 1 -- 172.21.15.98:0/1512889997 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02700cfe00 msgr2=0x7f027014b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:52.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.070+0000 7f02755f2640 1 --2- 172.21.15.98:0/1512889997 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02700cfe00 0x7f027014b530 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f026000c490 tx=0x7f0260001e80 comp rx=0 tx=0).stop 2024-09-19T08:41:52.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.070+0000 7f02755f2640 1 -- 172.21.15.98:0/1512889997 shutdown_connections 2024-09-19T08:41:52.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.070+0000 7f02755f2640 1 --2- 172.21.15.98:0/1512889997 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0250067a70 0x7f0250069f30 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:52.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.070+0000 7f02755f2640 1 --2- 172.21.15.98:0/1512889997 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0270154f70 0x7f02701526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:52.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.070+0000 7f02755f2640 1 --2- 172.21.15.98:0/1512889997 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02700cfe00 0x7f027014b530 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:52.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.070+0000 7f02755f2640 1 --2- 172.21.15.98:0/1512889997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02700b2800 0x7f0270150d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:52.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.070+0000 7f02755f2640 1 -- 172.21.15.98:0/1512889997 >> 172.21.15.98:0/1512889997 conn(0x7f027006bc90 msgr2=0x7f0270158d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:52.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.074+0000 7f02755f2640 1 -- 172.21.15.98:0/1512889997 shutdown_connections 2024-09-19T08:41:52.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.074+0000 7f02755f2640 1 -- 172.21.15.98:0/1512889997 wait complete. 2024-09-19T08:41:52.087 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-19T08:41:52.087 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:41:52.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.198+0000 7fc02a4ca640 1 Processor -- start 2024-09-19T08:41:52.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.198+0000 7fc02a4ca640 1 -- start start 2024-09-19T08:41:52.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.198+0000 7fc02a4ca640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc0240b9970 0x7fc0240b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:52.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.198+0000 7fc02a4ca640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc024068440 con 0x7fc0240b7f80 2024-09-19T08:41:52.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.198+0000 7fc02a4ca640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc024162a90 con 0x7fc0240d4820 2024-09-19T08:41:52.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.198+0000 7fc02a4ca640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fc024163050 con 0x7fc0240b9970 2024-09-19T08:41:52.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.198+0000 7fc023fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fc0240b7f80 0x7fc02415f560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40092/0 (socket says 172.21.15.98:40092) 2024-09-19T08:41:52.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.198+0000 7fc023fff640 1 -- 172.21.15.98:0/2131337195 learned_addr learned my addr 172.21.15.98:0/2131337195 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:41:52.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.198+0000 7fc022ffd640 1 -- 172.21.15.98:0/2131337195 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 743085229 0 0) 0x7fc024068440 con 0x7fc0240b7f80 2024-09-19T08:41:52.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.198+0000 7fc022ffd640 1 -- 172.21.15.98:0/2131337195 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc00c003530 con 0x7fc0240b7f80 2024-09-19T08:41:52.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.198+0000 7fc028a67640 1 --2- 172.21.15.98:0/2131337195 >> v2:172.21.15.139:3300/0 conn(0x7fc0240b9970 0x7fc0240b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:52.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.202+0000 7fc022ffd640 1 -- 172.21.15.98:0/2131337195 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2418221750 0 0) 0x7fc00c003530 con 0x7fc0240b7f80 2024-09-19T08:41:52.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.202+0000 7fc022ffd640 1 -- 172.21.15.98:0/2131337195 >> v2:172.21.15.139:3300/0 conn(0x7fc0240b9970 msgr2=0x7fc0240b9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:52.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.202+0000 7fc022ffd640 1 --2- 172.21.15.98:0/2131337195 >> v2:172.21.15.139:3300/0 conn(0x7fc0240b9970 0x7fc0240b9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:52.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.202+0000 7fc022ffd640 1 -- 172.21.15.98:0/2131337195 >> v1:172.21.15.139:6789/0 conn(0x7fc0240d4820 legacy=0x7fc0241621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:52.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.202+0000 7fc022ffd640 1 -- 172.21.15.98:0/2131337195 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc024162d70 con 0x7fc0240b7f80 2024-09-19T08:41:52.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.202+0000 7fc022ffd640 1 -- 172.21.15.98:0/2131337195 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fc014001680 con 0x7fc0240b7f80 2024-09-19T08:41:52.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.202+0000 7fc022ffd640 1 -- 172.21.15.98:0/2131337195 >> v1:172.21.15.98:6789/0 conn(0x7fc0240b7f80 legacy=0x7fc02415f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:52.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.202+0000 7fc022ffd640 1 --2- 172.21.15.98:0/2131337195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc00c0039c0 0x7fc00c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:52.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.202+0000 7fc022ffd640 1 --2- 172.21.15.98:0/2131337195 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc00c004810 0x7fc00c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:52.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.202+0000 7fc022ffd640 1 --2- 172.21.15.98:0/2131337195 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc00c0051d0 0x7fc00c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:52.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.202+0000 7fc022ffd640 1 -- 172.21.15.98:0/2131337195 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc024163050 con 0x7fc00c0039c0 2024-09-19T08:41:52.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.202+0000 7fc022ffd640 1 -- 172.21.15.98:0/2131337195 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c00c120 con 0x7fc00c0051d0 2024-09-19T08:41:52.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.202+0000 7fc022ffd640 1 -- 172.21.15.98:0/2131337195 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc00c00c320 con 0x7fc00c004810 2024-09-19T08:41:52.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.202+0000 7fc023fff640 1 --2- 172.21.15.98:0/2131337195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc00c0039c0 0x7fc00c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:52.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.202+0000 7fc028a67640 1 --2- 172.21.15.98:0/2131337195 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc00c0051d0 0x7fc00c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:52.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.202+0000 7fc0237fe640 1 --2- 172.21.15.98:0/2131337195 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc00c004810 0x7fc00c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:52.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.202+0000 7fc023fff640 1 -- 172.21.15.98:0/2131337195 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc00c004810 msgr2=0x7fc00c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:52.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.202+0000 7fc023fff640 1 --2- 172.21.15.98:0/2131337195 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc00c004810 0x7fc00c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:52.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.202+0000 7fc023fff640 1 -- 172.21.15.98:0/2131337195 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc00c0051d0 msgr2=0x7fc00c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:52.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.202+0000 7fc023fff640 1 --2- 172.21.15.98:0/2131337195 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc00c0051d0 0x7fc00c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:52.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.202+0000 7fc028a67640 1 -- 172.21.15.98:0/2131337195 reap_dead start 2024-09-19T08:41:52.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.202+0000 7fc023fff640 1 -- 172.21.15.98:0/2131337195 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc00c003530 con 0x7fc00c0039c0 2024-09-19T08:41:52.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.206+0000 7fc023fff640 1 --2- 172.21.15.98:0/2131337195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc00c0039c0 0x7fc00c003de0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc01400c1c0 tx=0x7fc01401cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20bc4c13515c5b5e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:52.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.206+0000 7fc022ffd640 1 -- 172.21.15.98:0/2131337195 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc014007d00 con 0x7fc00c0039c0 2024-09-19T08:41:52.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.206+0000 7fc022ffd640 1 -- 172.21.15.98:0/2131337195 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0140a9020 con 0x7fc00c0039c0 2024-09-19T08:41:52.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.206+0000 7fc022ffd640 1 -- 172.21.15.98:0/2131337195 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc014005b00 con 0x7fc00c0039c0 2024-09-19T08:41:52.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.206+0000 7fc02a4ca640 1 -- 172.21.15.98:0/2131337195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc00c0039c0 msgr2=0x7fc00c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:52.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.206+0000 7fc02a4ca640 1 --2- 172.21.15.98:0/2131337195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc00c0039c0 0x7fc00c003de0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc01400c1c0 tx=0x7fc01401cc20 comp rx=0 tx=0).stop 2024-09-19T08:41:52.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.206+0000 7fc02a4ca640 1 -- 172.21.15.98:0/2131337195 shutdown_connections 2024-09-19T08:41:52.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.206+0000 7fc02a4ca640 1 --2- 172.21.15.98:0/2131337195 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc00c0039c0 0x7fc00c003de0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:52.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.206+0000 7fc02a4ca640 1 -- 172.21.15.98:0/2131337195 >> 172.21.15.98:0/2131337195 conn(0x7fc024074e40 msgr2=0x7fc024075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:52.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.206+0000 7fc02a4ca640 1 -- 172.21.15.98:0/2131337195 shutdown_connections 2024-09-19T08:41:52.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.206+0000 7fc02a4ca640 1 -- 172.21.15.98:0/2131337195 wait complete. 2024-09-19T08:41:52.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.210+0000 7fc02a4ca640 1 Processor -- start 2024-09-19T08:41:52.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.210+0000 7fc02a4ca640 1 -- start start 2024-09-19T08:41:52.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.210+0000 7fc02a4ca640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc024156f60 0x7fc024158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:52.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.210+0000 7fc02a4ca640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc024157340 0x7fc02415c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:52.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.210+0000 7fc02a4ca640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc024130570 0x7fc024130a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:52.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.210+0000 7fc02a4ca640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc0240ba190 con 0x7fc024157340 2024-09-19T08:41:52.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.210+0000 7fc02a4ca640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc0240ba310 con 0x7fc024130570 2024-09-19T08:41:52.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.210+0000 7fc02a4ca640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc024143d80 con 0x7fc024156f60 2024-09-19T08:41:52.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.210+0000 7fc0237fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc024157340 0x7fc02415c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:52.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.210+0000 7fc0237fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc024157340 0x7fc02415c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47026/0 (socket says 172.21.15.98:47026) 2024-09-19T08:41:52.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.210+0000 7fc0237fe640 1 -- 172.21.15.98:0/2959619314 learned_addr learned my addr 172.21.15.98:0/2959619314 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:52.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.210+0000 7fc023fff640 1 --2- 172.21.15.98:0/2959619314 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc024156f60 0x7fc024158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:52.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.210+0000 7fc028a67640 1 --2- 172.21.15.98:0/2959619314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc024130570 0x7fc024130a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:52.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.210+0000 7fc0237fe640 1 -- 172.21.15.98:0/2959619314 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc024156f60 msgr2=0x7fc024158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:52.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.210+0000 7fc0237fe640 1 --2- 172.21.15.98:0/2959619314 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc024156f60 0x7fc024158780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:52.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.210+0000 7fc0237fe640 1 -- 172.21.15.98:0/2959619314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc024130570 msgr2=0x7fc024130a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:52.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.210+0000 7fc0237fe640 1 --2- 172.21.15.98:0/2959619314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc024130570 0x7fc024130a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:52.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.210+0000 7fc0237fe640 1 -- 172.21.15.98:0/2959619314 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc02415cc40 con 0x7fc024157340 2024-09-19T08:41:52.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.210+0000 7fc0237fe640 1 --2- 172.21.15.98:0/2959619314 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc024157340 0x7fc02415c700 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc01800f610 tx=0x7fc01800fad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:52.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.210+0000 7fc020ff9640 1 -- 172.21.15.98:0/2959619314 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0180052e0 con 0x7fc024157340 2024-09-19T08:41:52.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.210+0000 7fc020ff9640 1 -- 172.21.15.98:0/2959619314 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc018018070 con 0x7fc024157340 2024-09-19T08:41:52.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.210+0000 7fc02a4ca640 1 -- 172.21.15.98:0/2959619314 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc02415cee0 con 0x7fc024157340 2024-09-19T08:41:52.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.210+0000 7fc020ff9640 1 -- 172.21.15.98:0/2959619314 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc018003840 con 0x7fc024157340 2024-09-19T08:41:52.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.210+0000 7fc02a4ca640 1 -- 172.21.15.98:0/2959619314 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc02415d4a0 con 0x7fc024157340 2024-09-19T08:41:52.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.210+0000 7fc02a4ca640 1 -- 172.21.15.98:0/2959619314 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc024144110 con 0x7fc024157340 2024-09-19T08:41:52.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.210+0000 7fc020ff9640 1 -- 172.21.15.98:0/2959619314 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7fc0180039e0 con 0x7fc024157340 2024-09-19T08:41:52.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.214+0000 7fc020ff9640 1 --2- 172.21.15.98:0/2959619314 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc010067a70 0x7fc010069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:52.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.214+0000 7fc023fff640 1 --2- 172.21.15.98:0/2959619314 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc010067a70 0x7fc010069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:52.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.214+0000 7fc020ff9640 1 -- 172.21.15.98:0/2959619314 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fc018085da0 con 0x7fc024157340 2024-09-19T08:41:52.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.214+0000 7fc023fff640 1 --2- 172.21.15.98:0/2959619314 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc010067a70 0x7fc010069f30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc0140aa000 tx=0x7fc02415f040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:52.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.218+0000 7fc020ff9640 1 -- 172.21.15.98:0/2959619314 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc018056660 con 0x7fc024157340 2024-09-19T08:41:52.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.362+0000 7fc02a4ca640 1 -- 172.21.15.98:0/2959619314 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc0241ed040 con 0x7fc010067a70 2024-09-19T08:41:52.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.370+0000 7fc020ff9640 1 -- 172.21.15.98:0/2959619314 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251902 (secure 0 0 0) 0x7fc0241ed040 con 0x7fc010067a70 2024-09-19T08:41:52.372 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:41:52.374 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:41:52.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.374+0000 7fc02a4ca640 1 -- 172.21.15.98:0/2959619314 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc010067a70 msgr2=0x7fc010069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:52.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.374+0000 7fc02a4ca640 1 --2- 172.21.15.98:0/2959619314 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc010067a70 0x7fc010069f30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc0140aa000 tx=0x7fc02415f040 comp rx=0 tx=0).stop 2024-09-19T08:41:52.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.374+0000 7fc02a4ca640 1 -- 172.21.15.98:0/2959619314 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc024157340 msgr2=0x7fc02415c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:52.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.374+0000 7fc02a4ca640 1 --2- 172.21.15.98:0/2959619314 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc024157340 0x7fc02415c700 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc01800f610 tx=0x7fc01800fad0 comp rx=0 tx=0).stop 2024-09-19T08:41:52.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.374+0000 7fc02a4ca640 1 -- 172.21.15.98:0/2959619314 shutdown_connections 2024-09-19T08:41:52.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.374+0000 7fc02a4ca640 1 --2- 172.21.15.98:0/2959619314 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fc010067a70 0x7fc010069f30 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:52.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.374+0000 7fc02a4ca640 1 --2- 172.21.15.98:0/2959619314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc024130570 0x7fc024130a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:52.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.374+0000 7fc02a4ca640 1 --2- 172.21.15.98:0/2959619314 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc024157340 0x7fc02415c700 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:52.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.374+0000 7fc02a4ca640 1 --2- 172.21.15.98:0/2959619314 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc024156f60 0x7fc024158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:52.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.374+0000 7fc02a4ca640 1 -- 172.21.15.98:0/2959619314 >> 172.21.15.98:0/2959619314 conn(0x7fc024074e40 msgr2=0x7fc0240d2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:52.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.374+0000 7fc02a4ca640 1 -- 172.21.15.98:0/2959619314 shutdown_connections 2024-09-19T08:41:52.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:52.374+0000 7fc02a4ca640 1 -- 172.21.15.98:0/2959619314 wait complete. 2024-09-19T08:41:52.389 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2024-09-19T08:41:52.097697+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":110,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":247616,"kb_used_data":6400,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754727104,"statfs":{"total":773094113280,"available":772840554496,"internally_reserved":0,"allocated":6553600,"data_stored":3370753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":47,"apply_latency_ms":47,"commit_latency_ns":47000000,"apply_latency_ns":47000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001622"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.985587+0000","last_change":"2024-09-19T08:41:47.980122+0000","last_active":"2024-09-19T08:41:47.985587+0000","last_peered":"2024-09-19T08:41:47.985587+0000","last_clean":"2024-09-19T08:41:47.985587+0000","last_became_active":"2024-09-19T08:41:47.976398+0000","last_became_peered":"2024-09-19T08:41:47.976398+0000","last_unstale":"2024-09-19T08:41:47.985587+0000","last_undegraded":"2024-09-19T08:41:47.985587+0000","last_fullsized":"2024-09-19T08:41:47.985587+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:25.855696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.976086+0000","last_change":"2024-09-19T08:41:47.973338+0000","last_active":"2024-09-19T08:41:47.976086+0000","last_peered":"2024-09-19T08:41:47.976086+0000","last_clean":"2024-09-19T08:41:47.976086+0000","last_became_active":"2024-09-19T08:41:47.973065+0000","last_became_peered":"2024-09-19T08:41:47.973065+0000","last_unstale":"2024-09-19T08:41:47.976086+0000","last_undegraded":"2024-09-19T08:41:47.976086+0000","last_fullsized":"2024-09-19T08:41:47.976086+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:07.544296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.982165+0000","last_change":"2024-09-19T08:41:47.966283+0000","last_active":"2024-09-19T08:41:47.982165+0000","last_peered":"2024-09-19T08:41:47.982165+0000","last_clean":"2024-09-19T08:41:47.982165+0000","last_became_active":"2024-09-19T08:41:47.965563+0000","last_became_peered":"2024-09-19T08:41:47.965563+0000","last_unstale":"2024-09-19T08:41:47.982165+0000","last_undegraded":"2024-09-19T08:41:47.982165+0000","last_fullsized":"2024-09-19T08:41:47.982165+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:39:24.749819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.993880+0000","last_change":"2024-09-19T08:41:47.978137+0000","last_active":"2024-09-19T08:41:47.993880+0000","last_peered":"2024-09-19T08:41:47.993880+0000","last_clean":"2024-09-19T08:41:47.993880+0000","last_became_active":"2024-09-19T08:41:47.976050+0000","last_became_peered":"2024-09-19T08:41:47.976050+0000","last_unstale":"2024-09-19T08:41:47.993880+0000","last_undegraded":"2024-09-19T08:41:47.993880+0000","last_fullsized":"2024-09-19T08:41:47.993880+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:56:44.612767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.994306+0000","last_change":"2024-09-19T08:41:47.993040+0000","last_active":"2024-09-19T08:41:47.994306+0000","last_peered":"2024-09-19T08:41:47.994306+0000","last_clean":"2024-09-19T08:41:47.994306+0000","last_became_active":"2024-09-19T08:41:47.992767+0000","last_became_peered":"2024-09-19T08:41:47.992767+0000","last_unstale":"2024-09-19T08:41:47.994306+0000","last_undegraded":"2024-09-19T08:41:47.994306+0000","last_fullsized":"2024-09-19T08:41:47.994306+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:52:54.206986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.964412+0000","last_change":"2024-09-19T08:41:47.961838+0000","last_active":"2024-09-19T08:41:47.964412+0000","last_peered":"2024-09-19T08:41:47.964412+0000","last_clean":"2024-09-19T08:41:47.964412+0000","last_became_active":"2024-09-19T08:41:47.961116+0000","last_became_peered":"2024-09-19T08:41:47.961116+0000","last_unstale":"2024-09-19T08:41:47.964412+0000","last_undegraded":"2024-09-19T08:41:47.964412+0000","last_fullsized":"2024-09-19T08:41:47.964412+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:41.615523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971387+0000","last_change":"2024-09-19T08:41:47.966503+0000","last_active":"2024-09-19T08:41:48.971387+0000","last_peered":"2024-09-19T08:41:48.971387+0000","last_clean":"2024-09-19T08:41:48.971387+0000","last_became_active":"2024-09-19T08:41:47.965805+0000","last_became_peered":"2024-09-19T08:41:47.965805+0000","last_unstale":"2024-09-19T08:41:48.971387+0000","last_undegraded":"2024-09-19T08:41:48.971387+0000","last_fullsized":"2024-09-19T08:41:48.971387+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:41.615523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.971097+0000","last_change":"2024-09-19T08:41:47.969144+0000","last_active":"2024-09-19T08:41:47.971097+0000","last_peered":"2024-09-19T08:41:47.971097+0000","last_clean":"2024-09-19T08:41:47.971097+0000","last_became_active":"2024-09-19T08:41:47.968766+0000","last_became_peered":"2024-09-19T08:41:47.968766+0000","last_unstale":"2024-09-19T08:41:47.971097+0000","last_undegraded":"2024-09-19T08:41:47.971097+0000","last_fullsized":"2024-09-19T08:41:47.971097+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:20:19.600052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971951+0000","last_change":"2024-09-19T08:41:47.954679+0000","last_active":"2024-09-19T08:41:48.971951+0000","last_peered":"2024-09-19T08:41:48.971951+0000","last_clean":"2024-09-19T08:41:48.971951+0000","last_became_active":"2024-09-19T08:41:47.951685+0000","last_became_peered":"2024-09-19T08:41:47.951685+0000","last_unstale":"2024-09-19T08:41:48.971951+0000","last_undegraded":"2024-09-19T08:41:48.971951+0000","last_fullsized":"2024-09-19T08:41:48.971951+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:17.405379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971133+0000","last_change":"2024-09-19T08:41:47.977064+0000","last_active":"2024-09-19T08:41:48.971133+0000","last_peered":"2024-09-19T08:41:48.971133+0000","last_clean":"2024-09-19T08:41:48.971133+0000","last_became_active":"2024-09-19T08:41:47.976747+0000","last_became_peered":"2024-09-19T08:41:47.976747+0000","last_unstale":"2024-09-19T08:41:48.971133+0000","last_undegraded":"2024-09-19T08:41:48.971133+0000","last_fullsized":"2024-09-19T08:41:48.971133+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:37.366531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971332+0000","last_change":"2024-09-19T08:41:47.971592+0000","last_active":"2024-09-19T08:41:48.971332+0000","last_peered":"2024-09-19T08:41:48.971332+0000","last_clean":"2024-09-19T08:41:48.971332+0000","last_became_active":"2024-09-19T08:41:47.969075+0000","last_became_peered":"2024-09-19T08:41:47.969075+0000","last_unstale":"2024-09-19T08:41:48.971332+0000","last_undegraded":"2024-09-19T08:41:48.971332+0000","last_fullsized":"2024-09-19T08:41:48.971332+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:29.817497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.992937+0000","last_change":"2024-09-19T08:41:47.968996+0000","last_active":"2024-09-19T08:41:47.992937+0000","last_peered":"2024-09-19T08:41:47.992937+0000","last_clean":"2024-09-19T08:41:47.992937+0000","last_became_active":"2024-09-19T08:41:47.968254+0000","last_became_peered":"2024-09-19T08:41:47.968254+0000","last_unstale":"2024-09-19T08:41:47.992937+0000","last_undegraded":"2024-09-19T08:41:47.992937+0000","last_fullsized":"2024-09-19T08:41:47.992937+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:12.552026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.972167+0000","last_change":"2024-09-19T08:41:47.975212+0000","last_active":"2024-09-19T08:41:49.972167+0000","last_peered":"2024-09-19T08:41:49.972167+0000","last_clean":"2024-09-19T08:41:49.972167+0000","last_became_active":"2024-09-19T08:41:47.974446+0000","last_became_peered":"2024-09-19T08:41:47.974446+0000","last_unstale":"2024-09-19T08:41:49.972167+0000","last_undegraded":"2024-09-19T08:41:49.972167+0000","last_fullsized":"2024-09-19T08:41:49.972167+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:15:16.250986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971528+0000","last_change":"2024-09-19T08:41:47.962145+0000","last_active":"2024-09-19T08:41:48.971528+0000","last_peered":"2024-09-19T08:41:48.971528+0000","last_clean":"2024-09-19T08:41:48.971528+0000","last_became_active":"2024-09-19T08:41:47.961449+0000","last_became_peered":"2024-09-19T08:41:47.961449+0000","last_unstale":"2024-09-19T08:41:48.971528+0000","last_undegraded":"2024-09-19T08:41:48.971528+0000","last_fullsized":"2024-09-19T08:41:48.971528+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:09.481675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.972047+0000","last_change":"2024-09-19T08:41:47.960027+0000","last_active":"2024-09-19T08:41:48.972047+0000","last_peered":"2024-09-19T08:41:48.972047+0000","last_clean":"2024-09-19T08:41:48.972047+0000","last_became_active":"2024-09-19T08:41:47.959242+0000","last_became_peered":"2024-09-19T08:41:47.959242+0000","last_unstale":"2024-09-19T08:41:48.972047+0000","last_undegraded":"2024-09-19T08:41:48.972047+0000","last_fullsized":"2024-09-19T08:41:48.972047+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:56:44.612767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.984338+0000","last_change":"2024-09-19T08:41:47.979371+0000","last_active":"2024-09-19T08:41:47.984338+0000","last_peered":"2024-09-19T08:41:47.984338+0000","last_clean":"2024-09-19T08:41:47.984338+0000","last_became_active":"2024-09-19T08:41:47.978819+0000","last_became_peered":"2024-09-19T08:41:47.978819+0000","last_unstale":"2024-09-19T08:41:47.984338+0000","last_undegraded":"2024-09-19T08:41:47.984338+0000","last_fullsized":"2024-09-19T08:41:47.984338+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:34.451800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.981562+0000","last_change":"2024-09-19T08:41:47.977880+0000","last_active":"2024-09-19T08:41:47.981562+0000","last_peered":"2024-09-19T08:41:47.981562+0000","last_clean":"2024-09-19T08:41:47.981562+0000","last_became_active":"2024-09-19T08:41:47.977590+0000","last_became_peered":"2024-09-19T08:41:47.977590+0000","last_unstale":"2024-09-19T08:41:47.981562+0000","last_undegraded":"2024-09-19T08:41:47.981562+0000","last_fullsized":"2024-09-19T08:41:47.981562+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:43:07.334696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.982300+0000","last_change":"2024-09-19T08:41:47.968677+0000","last_active":"2024-09-19T08:41:47.982300+0000","last_peered":"2024-09-19T08:41:47.982300+0000","last_clean":"2024-09-19T08:41:47.982300+0000","last_became_active":"2024-09-19T08:41:47.967937+0000","last_became_peered":"2024-09-19T08:41:47.967937+0000","last_unstale":"2024-09-19T08:41:47.982300+0000","last_undegraded":"2024-09-19T08:41:47.982300+0000","last_fullsized":"2024-09-19T08:41:47.982300+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:32:44.673319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.981344+0000","last_change":"2024-09-19T08:41:47.975831+0000","last_active":"2024-09-19T08:41:47.981344+0000","last_peered":"2024-09-19T08:41:47.981344+0000","last_clean":"2024-09-19T08:41:47.981344+0000","last_became_active":"2024-09-19T08:41:47.975518+0000","last_became_peered":"2024-09-19T08:41:47.975518+0000","last_unstale":"2024-09-19T08:41:47.981344+0000","last_undegraded":"2024-09-19T08:41:47.981344+0000","last_fullsized":"2024-09-19T08:41:47.981344+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:23:47.530098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.976135+0000","last_change":"2024-09-19T08:41:47.973832+0000","last_active":"2024-09-19T08:41:47.976135+0000","last_peered":"2024-09-19T08:41:47.976135+0000","last_clean":"2024-09-19T08:41:47.976135+0000","last_became_active":"2024-09-19T08:41:47.973516+0000","last_became_peered":"2024-09-19T08:41:47.973516+0000","last_unstale":"2024-09-19T08:41:47.976135+0000","last_undegraded":"2024-09-19T08:41:47.976135+0000","last_fullsized":"2024-09-19T08:41:47.976135+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:43:59.145791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.994338+0000","last_change":"2024-09-19T08:41:47.979480+0000","last_active":"2024-09-19T08:41:47.994338+0000","last_peered":"2024-09-19T08:41:47.994338+0000","last_clean":"2024-09-19T08:41:47.994338+0000","last_became_active":"2024-09-19T08:41:47.977765+0000","last_became_peered":"2024-09-19T08:41:47.977765+0000","last_unstale":"2024-09-19T08:41:47.994338+0000","last_undegraded":"2024-09-19T08:41:47.994338+0000","last_fullsized":"2024-09-19T08:41:47.994338+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:41.615523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.996753+0000","last_change":"2024-09-19T08:41:47.981514+0000","last_active":"2024-09-19T08:41:47.996753+0000","last_peered":"2024-09-19T08:41:47.996753+0000","last_clean":"2024-09-19T08:41:47.996753+0000","last_became_active":"2024-09-19T08:41:47.981044+0000","last_became_peered":"2024-09-19T08:41:47.981044+0000","last_unstale":"2024-09-19T08:41:47.996753+0000","last_undegraded":"2024-09-19T08:41:47.996753+0000","last_fullsized":"2024-09-19T08:41:47.996753+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:07.796396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.993777+0000","last_change":"2024-09-19T08:41:47.970880+0000","last_active":"2024-09-19T08:41:47.993777+0000","last_peered":"2024-09-19T08:41:47.993777+0000","last_clean":"2024-09-19T08:41:47.993777+0000","last_became_active":"2024-09-19T08:41:47.970145+0000","last_became_peered":"2024-09-19T08:41:47.970145+0000","last_unstale":"2024-09-19T08:41:47.993777+0000","last_undegraded":"2024-09-19T08:41:47.993777+0000","last_fullsized":"2024-09-19T08:41:47.993777+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:52:41.224293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.971590+0000","last_change":"2024-09-19T08:41:47.976061+0000","last_active":"2024-09-19T08:41:49.971590+0000","last_peered":"2024-09-19T08:41:49.971590+0000","last_clean":"2024-09-19T08:41:49.971590+0000","last_became_active":"2024-09-19T08:41:47.975280+0000","last_became_peered":"2024-09-19T08:41:47.975280+0000","last_unstale":"2024-09-19T08:41:49.971590+0000","last_undegraded":"2024-09-19T08:41:49.971590+0000","last_fullsized":"2024-09-19T08:41:49.971590+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:12:23.981537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.985564+0000","last_change":"2024-09-19T08:41:47.977025+0000","last_active":"2024-09-19T08:41:47.985564+0000","last_peered":"2024-09-19T08:41:47.985564+0000","last_clean":"2024-09-19T08:41:47.985564+0000","last_became_active":"2024-09-19T08:41:47.975707+0000","last_became_peered":"2024-09-19T08:41:47.975707+0000","last_unstale":"2024-09-19T08:41:47.985564+0000","last_undegraded":"2024-09-19T08:41:47.985564+0000","last_fullsized":"2024-09-19T08:41:47.985564+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:17.908127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.990912+0000","last_change":"2024-09-19T08:41:47.966128+0000","last_active":"2024-09-19T08:41:47.990912+0000","last_peered":"2024-09-19T08:41:47.990912+0000","last_clean":"2024-09-19T08:41:47.990912+0000","last_became_active":"2024-09-19T08:41:47.965403+0000","last_became_peered":"2024-09-19T08:41:47.965403+0000","last_unstale":"2024-09-19T08:41:47.990912+0000","last_undegraded":"2024-09-19T08:41:47.990912+0000","last_fullsized":"2024-09-19T08:41:47.990912+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:43.544741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.981730+0000","last_change":"2024-09-19T08:41:47.979320+0000","last_active":"2024-09-19T08:41:47.981730+0000","last_peered":"2024-09-19T08:41:47.981730+0000","last_clean":"2024-09-19T08:41:47.981730+0000","last_became_active":"2024-09-19T08:41:47.979010+0000","last_became_peered":"2024-09-19T08:41:47.979010+0000","last_unstale":"2024-09-19T08:41:47.981730+0000","last_undegraded":"2024-09-19T08:41:47.981730+0000","last_fullsized":"2024-09-19T08:41:47.981730+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:44:04.680315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.975042+0000","last_change":"2024-09-19T08:41:47.966338+0000","last_active":"2024-09-19T08:41:47.975042+0000","last_peered":"2024-09-19T08:41:47.975042+0000","last_clean":"2024-09-19T08:41:47.975042+0000","last_became_active":"2024-09-19T08:41:47.966024+0000","last_became_peered":"2024-09-19T08:41:47.966024+0000","last_unstale":"2024-09-19T08:41:47.975042+0000","last_undegraded":"2024-09-19T08:41:47.975042+0000","last_fullsized":"2024-09-19T08:41:47.975042+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:45.927550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.997659+0000","last_change":"2024-09-19T08:41:47.996502+0000","last_active":"2024-09-19T08:41:47.997659+0000","last_peered":"2024-09-19T08:41:47.997659+0000","last_clean":"2024-09-19T08:41:47.997659+0000","last_became_active":"2024-09-19T08:41:47.996162+0000","last_became_peered":"2024-09-19T08:41:47.996162+0000","last_unstale":"2024-09-19T08:41:47.997659+0000","last_undegraded":"2024-09-19T08:41:47.997659+0000","last_fullsized":"2024-09-19T08:41:47.997659+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:13.341860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.982174+0000","last_change":"2024-09-19T08:41:47.966508+0000","last_active":"2024-09-19T08:41:47.982174+0000","last_peered":"2024-09-19T08:41:47.982174+0000","last_clean":"2024-09-19T08:41:47.982174+0000","last_became_active":"2024-09-19T08:41:47.962098+0000","last_became_peered":"2024-09-19T08:41:47.962098+0000","last_unstale":"2024-09-19T08:41:47.982174+0000","last_undegraded":"2024-09-19T08:41:47.982174+0000","last_fullsized":"2024-09-19T08:41:47.982174+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:23.999718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.981287+0000","last_change":"2024-09-19T08:41:47.974969+0000","last_active":"2024-09-19T08:41:47.981287+0000","last_peered":"2024-09-19T08:41:47.981287+0000","last_clean":"2024-09-19T08:41:47.981287+0000","last_became_active":"2024-09-19T08:41:47.974440+0000","last_became_peered":"2024-09-19T08:41:47.974440+0000","last_unstale":"2024-09-19T08:41:47.981287+0000","last_undegraded":"2024-09-19T08:41:47.981287+0000","last_fullsized":"2024-09-19T08:41:47.981287+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:35:19.862729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.978537+0000","last_change":"2024-09-19T08:41:47.972260+0000","last_active":"2024-09-19T08:41:47.978537+0000","last_peered":"2024-09-19T08:41:47.978537+0000","last_clean":"2024-09-19T08:41:47.978537+0000","last_became_active":"2024-09-19T08:41:47.971683+0000","last_became_peered":"2024-09-19T08:41:47.971683+0000","last_unstale":"2024-09-19T08:41:47.978537+0000","last_undegraded":"2024-09-19T08:41:47.978537+0000","last_fullsized":"2024-09-19T08:41:47.978537+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:56:44.612767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.964666+0000","last_change":"2024-09-19T08:41:47.962374+0000","last_active":"2024-09-19T08:41:47.964666+0000","last_peered":"2024-09-19T08:41:47.964666+0000","last_clean":"2024-09-19T08:41:47.964666+0000","last_became_active":"2024-09-19T08:41:47.961305+0000","last_became_peered":"2024-09-19T08:41:47.961305+0000","last_unstale":"2024-09-19T08:41:47.964666+0000","last_undegraded":"2024-09-19T08:41:47.964666+0000","last_fullsized":"2024-09-19T08:41:47.964666+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:09.481675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"56'4","reported_seq":16,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.001343+0000","last_change":"2024-09-19T08:41:46.976847+0000","last_active":"2024-09-19T08:41:48.001343+0000","last_peered":"2024-09-19T08:41:48.001343+0000","last_clean":"2024-09-19T08:41:48.001343+0000","last_became_active":"2024-09-19T08:41:46.976405+0000","last_became_peered":"2024-09-19T08:41:46.976405+0000","last_unstale":"2024-09-19T08:41:48.001343+0000","last_undegraded":"2024-09-19T08:41:48.001343+0000","last_fullsized":"2024-09-19T08:41:48.001343+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:52:40.220622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971359+0000","last_change":"2024-09-19T08:41:47.971091+0000","last_active":"2024-09-19T08:41:48.971359+0000","last_peered":"2024-09-19T08:41:48.971359+0000","last_clean":"2024-09-19T08:41:48.971359+0000","last_became_active":"2024-09-19T08:41:47.958633+0000","last_became_peered":"2024-09-19T08:41:47.958633+0000","last_unstale":"2024-09-19T08:41:48.971359+0000","last_undegraded":"2024-09-19T08:41:48.971359+0000","last_fullsized":"2024-09-19T08:41:48.971359+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:38.634694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"56'1","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.997195+0000","last_change":"2024-09-19T08:41:46.953976+0000","last_active":"2024-09-19T08:41:47.997195+0000","last_peered":"2024-09-19T08:41:47.997195+0000","last_clean":"2024-09-19T08:41:47.997195+0000","last_became_active":"2024-09-19T08:41:46.953613+0000","last_became_peered":"2024-09-19T08:41:46.953613+0000","last_unstale":"2024-09-19T08:41:47.997195+0000","last_undegraded":"2024-09-19T08:41:47.997195+0000","last_fullsized":"2024-09-19T08:41:47.997195+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:24.976526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.964324+0000","last_change":"2024-09-19T08:41:47.961793+0000","last_active":"2024-09-19T08:41:47.964324+0000","last_peered":"2024-09-19T08:41:47.964324+0000","last_clean":"2024-09-19T08:41:47.964324+0000","last_became_active":"2024-09-19T08:41:47.961008+0000","last_became_peered":"2024-09-19T08:41:47.961008+0000","last_unstale":"2024-09-19T08:41:47.964324+0000","last_undegraded":"2024-09-19T08:41:47.964324+0000","last_fullsized":"2024-09-19T08:41:47.964324+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:42.104552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"56'2","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.000690+0000","last_change":"2024-09-19T08:41:46.950525+0000","last_active":"2024-09-19T08:41:48.000690+0000","last_peered":"2024-09-19T08:41:48.000690+0000","last_clean":"2024-09-19T08:41:48.000690+0000","last_became_active":"2024-09-19T08:41:46.950185+0000","last_became_peered":"2024-09-19T08:41:46.950185+0000","last_unstale":"2024-09-19T08:41:48.000690+0000","last_undegraded":"2024-09-19T08:41:48.000690+0000","last_fullsized":"2024-09-19T08:41:48.000690+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:11.548355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.999907+0000","last_change":"2024-09-19T08:41:47.997038+0000","last_active":"2024-09-19T08:41:47.999907+0000","last_peered":"2024-09-19T08:41:47.999907+0000","last_clean":"2024-09-19T08:41:47.999907+0000","last_became_active":"2024-09-19T08:41:47.996654+0000","last_became_peered":"2024-09-19T08:41:47.996654+0000","last_unstale":"2024-09-19T08:41:47.999907+0000","last_undegraded":"2024-09-19T08:41:47.999907+0000","last_fullsized":"2024-09-19T08:41:47.999907+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:25:54.925434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"56'2","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.001947+0000","last_change":"2024-09-19T08:41:46.976094+0000","last_active":"2024-09-19T08:41:48.001947+0000","last_peered":"2024-09-19T08:41:48.001947+0000","last_clean":"2024-09-19T08:41:48.001947+0000","last_became_active":"2024-09-19T08:41:46.975349+0000","last_became_peered":"2024-09-19T08:41:46.975349+0000","last_unstale":"2024-09-19T08:41:48.001947+0000","last_undegraded":"2024-09-19T08:41:48.001947+0000","last_fullsized":"2024-09-19T08:41:48.001947+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:22.996047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.984243+0000","last_change":"2024-09-19T08:41:47.981348+0000","last_active":"2024-09-19T08:41:47.984243+0000","last_peered":"2024-09-19T08:41:47.984243+0000","last_clean":"2024-09-19T08:41:47.984243+0000","last_became_active":"2024-09-19T08:41:47.980623+0000","last_became_peered":"2024-09-19T08:41:47.980623+0000","last_unstale":"2024-09-19T08:41:47.984243+0000","last_undegraded":"2024-09-19T08:41:47.984243+0000","last_fullsized":"2024-09-19T08:41:47.984243+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:21:26.704824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971966+0000","last_change":"2024-09-19T08:41:46.959290+0000","last_active":"2024-09-19T08:41:48.971966+0000","last_peered":"2024-09-19T08:41:48.971966+0000","last_clean":"2024-09-19T08:41:48.971966+0000","last_became_active":"2024-09-19T08:41:46.958454+0000","last_became_peered":"2024-09-19T08:41:46.958454+0000","last_unstale":"2024-09-19T08:41:48.971966+0000","last_undegraded":"2024-09-19T08:41:48.971966+0000","last_fullsized":"2024-09-19T08:41:48.971966+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:39:23.746148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.971630+0000","last_change":"2024-09-19T08:41:47.969748+0000","last_active":"2024-09-19T08:41:47.971630+0000","last_peered":"2024-09-19T08:41:47.971630+0000","last_clean":"2024-09-19T08:41:47.971630+0000","last_became_active":"2024-09-19T08:41:47.969443+0000","last_became_peered":"2024-09-19T08:41:47.969443+0000","last_unstale":"2024-09-19T08:41:47.971630+0000","last_undegraded":"2024-09-19T08:41:47.971630+0000","last_fullsized":"2024-09-19T08:41:47.971630+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:47:48.598741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"56'1","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.999759+0000","last_change":"2024-09-19T08:41:46.956247+0000","last_active":"2024-09-19T08:41:47.999759+0000","last_peered":"2024-09-19T08:41:47.999759+0000","last_clean":"2024-09-19T08:41:47.999759+0000","last_became_active":"2024-09-19T08:41:46.955941+0000","last_became_peered":"2024-09-19T08:41:46.955941+0000","last_unstale":"2024-09-19T08:41:47.999759+0000","last_undegraded":"2024-09-19T08:41:47.999759+0000","last_fullsized":"2024-09-19T08:41:47.999759+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:22.996047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.971708+0000","last_change":"2024-09-19T08:41:47.984017+0000","last_active":"2024-09-19T08:41:49.971708+0000","last_peered":"2024-09-19T08:41:49.971708+0000","last_clean":"2024-09-19T08:41:49.971708+0000","last_became_active":"2024-09-19T08:41:47.983240+0000","last_became_peered":"2024-09-19T08:41:47.983240+0000","last_unstale":"2024-09-19T08:41:49.971708+0000","last_undegraded":"2024-09-19T08:41:49.971708+0000","last_fullsized":"2024-09-19T08:41:49.971708+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:22.655726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.950542+0000","last_change":"2024-09-19T08:41:46.976878+0000","last_active":"2024-09-19T08:41:47.950542+0000","last_peered":"2024-09-19T08:41:47.950542+0000","last_clean":"2024-09-19T08:41:47.950542+0000","last_became_active":"2024-09-19T08:41:46.975427+0000","last_became_peered":"2024-09-19T08:41:46.975427+0000","last_unstale":"2024-09-19T08:41:47.950542+0000","last_undegraded":"2024-09-19T08:41:47.950542+0000","last_fullsized":"2024-09-19T08:41:47.950542+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:41:25.735590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.979456+0000","last_change":"2024-09-19T08:41:47.977608+0000","last_active":"2024-09-19T08:41:47.979456+0000","last_peered":"2024-09-19T08:41:47.979456+0000","last_clean":"2024-09-19T08:41:47.979456+0000","last_became_active":"2024-09-19T08:41:47.977328+0000","last_became_peered":"2024-09-19T08:41:47.977328+0000","last_unstale":"2024-09-19T08:41:47.979456+0000","last_undegraded":"2024-09-19T08:41:47.979456+0000","last_fullsized":"2024-09-19T08:41:47.979456+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:11:38.632932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971395+0000","last_change":"2024-09-19T08:41:46.951849+0000","last_active":"2024-09-19T08:41:48.971395+0000","last_peered":"2024-09-19T08:41:48.971395+0000","last_clean":"2024-09-19T08:41:48.971395+0000","last_became_active":"2024-09-19T08:41:46.951049+0000","last_became_peered":"2024-09-19T08:41:46.951049+0000","last_unstale":"2024-09-19T08:41:48.971395+0000","last_undegraded":"2024-09-19T08:41:48.971395+0000","last_fullsized":"2024-09-19T08:41:48.971395+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:37.631023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.972453+0000","last_change":"2024-09-19T08:41:47.959107+0000","last_active":"2024-09-19T08:41:48.972453+0000","last_peered":"2024-09-19T08:41:48.972453+0000","last_clean":"2024-09-19T08:41:48.972453+0000","last_became_active":"2024-09-19T08:41:47.958257+0000","last_became_peered":"2024-09-19T08:41:47.958257+0000","last_unstale":"2024-09-19T08:41:48.972453+0000","last_undegraded":"2024-09-19T08:41:48.972453+0000","last_fullsized":"2024-09-19T08:41:48.972453+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:04:26.313098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.949274+0000","last_change":"2024-09-19T08:41:46.960020+0000","last_active":"2024-09-19T08:41:47.949274+0000","last_peered":"2024-09-19T08:41:47.949274+0000","last_clean":"2024-09-19T08:41:47.949274+0000","last_became_active":"2024-09-19T08:41:46.959232+0000","last_became_peered":"2024-09-19T08:41:46.959232+0000","last_unstale":"2024-09-19T08:41:47.949274+0000","last_undegraded":"2024-09-19T08:41:47.949274+0000","last_fullsized":"2024-09-19T08:41:47.949274+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:28.813825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.993989+0000","last_change":"2024-09-19T08:41:47.973711+0000","last_active":"2024-09-19T08:41:47.993989+0000","last_peered":"2024-09-19T08:41:47.993989+0000","last_clean":"2024-09-19T08:41:47.993989+0000","last_became_active":"2024-09-19T08:41:47.972968+0000","last_became_peered":"2024-09-19T08:41:47.972968+0000","last_unstale":"2024-09-19T08:41:47.993989+0000","last_undegraded":"2024-09-19T08:41:47.993989+0000","last_fullsized":"2024-09-19T08:41:47.993989+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:12.552026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.964270+0000","last_change":"2024-09-19T08:41:46.957881+0000","last_active":"2024-09-19T08:41:47.964270+0000","last_peered":"2024-09-19T08:41:47.964270+0000","last_clean":"2024-09-19T08:41:47.964270+0000","last_became_active":"2024-09-19T08:41:46.957503+0000","last_became_peered":"2024-09-19T08:41:46.957503+0000","last_unstale":"2024-09-19T08:41:47.964270+0000","last_undegraded":"2024-09-19T08:41:47.964270+0000","last_fullsized":"2024-09-19T08:41:47.964270+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:02:24.036512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971244+0000","last_change":"2024-09-19T08:41:47.968178+0000","last_active":"2024-09-19T08:41:48.971244+0000","last_peered":"2024-09-19T08:41:48.971244+0000","last_clean":"2024-09-19T08:41:48.971244+0000","last_became_active":"2024-09-19T08:41:47.967773+0000","last_became_peered":"2024-09-19T08:41:47.967773+0000","last_unstale":"2024-09-19T08:41:48.971244+0000","last_undegraded":"2024-09-19T08:41:48.971244+0000","last_fullsized":"2024-09-19T08:41:48.971244+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:21.843717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"56'3","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.002671+0000","last_change":"2024-09-19T08:41:46.957368+0000","last_active":"2024-09-19T08:41:48.002671+0000","last_peered":"2024-09-19T08:41:48.002671+0000","last_clean":"2024-09-19T08:41:48.002671+0000","last_became_active":"2024-09-19T08:41:46.956670+0000","last_became_peered":"2024-09-19T08:41:46.956670+0000","last_unstale":"2024-09-19T08:41:48.002671+0000","last_undegraded":"2024-09-19T08:41:48.002671+0000","last_fullsized":"2024-09-19T08:41:48.002671+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:37.631023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.002229+0000","last_change":"2024-09-19T08:41:47.977022+0000","last_active":"2024-09-19T08:41:48.002229+0000","last_peered":"2024-09-19T08:41:48.002229+0000","last_clean":"2024-09-19T08:41:48.002229+0000","last_became_active":"2024-09-19T08:41:47.973384+0000","last_became_peered":"2024-09-19T08:41:47.973384+0000","last_unstale":"2024-09-19T08:41:48.002229+0000","last_undegraded":"2024-09-19T08:41:48.002229+0000","last_fullsized":"2024-09-19T08:41:48.002229+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:42.104552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"56'6","reported_seq":20,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.972332+0000","last_change":"2024-09-19T08:41:46.953997+0000","last_active":"2024-09-19T08:41:48.972332+0000","last_peered":"2024-09-19T08:41:48.972332+0000","last_clean":"2024-09-19T08:41:48.972332+0000","last_became_active":"2024-09-19T08:41:46.953261+0000","last_became_peered":"2024-09-19T08:41:46.953261+0000","last_unstale":"2024-09-19T08:41:48.972332+0000","last_undegraded":"2024-09-19T08:41:48.972332+0000","last_fullsized":"2024-09-19T08:41:48.972332+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:41:25.735590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.991181+0000","last_change":"2024-09-19T08:41:47.973617+0000","last_active":"2024-09-19T08:41:47.991181+0000","last_peered":"2024-09-19T08:41:47.991181+0000","last_clean":"2024-09-19T08:41:47.991181+0000","last_became_active":"2024-09-19T08:41:47.973096+0000","last_became_peered":"2024-09-19T08:41:47.973096+0000","last_unstale":"2024-09-19T08:41:47.991181+0000","last_undegraded":"2024-09-19T08:41:47.991181+0000","last_fullsized":"2024-09-19T08:41:47.991181+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:42.104552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"56'2","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.996894+0000","last_change":"2024-09-19T08:41:46.958209+0000","last_active":"2024-09-19T08:41:47.996894+0000","last_peered":"2024-09-19T08:41:47.996894+0000","last_clean":"2024-09-19T08:41:47.996894+0000","last_became_active":"2024-09-19T08:41:46.957655+0000","last_became_peered":"2024-09-19T08:41:46.957655+0000","last_unstale":"2024-09-19T08:41:47.996894+0000","last_undegraded":"2024-09-19T08:41:47.996894+0000","last_fullsized":"2024-09-19T08:41:47.996894+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:06.027506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.994435+0000","last_change":"2024-09-19T08:41:47.980616+0000","last_active":"2024-09-19T08:41:47.994435+0000","last_peered":"2024-09-19T08:41:47.994435+0000","last_clean":"2024-09-19T08:41:47.994435+0000","last_became_active":"2024-09-19T08:41:47.978716+0000","last_became_peered":"2024-09-19T08:41:47.978716+0000","last_unstale":"2024-09-19T08:41:47.994435+0000","last_undegraded":"2024-09-19T08:41:47.994435+0000","last_fullsized":"2024-09-19T08:41:47.994435+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:56:44.612767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.971654+0000","last_change":"2024-09-19T08:41:46.963078+0000","last_active":"2024-09-19T08:41:49.971654+0000","last_peered":"2024-09-19T08:41:49.971654+0000","last_clean":"2024-09-19T08:41:49.971654+0000","last_became_active":"2024-09-19T08:41:46.962049+0000","last_became_peered":"2024-09-19T08:41:46.962049+0000","last_unstale":"2024-09-19T08:41:49.971654+0000","last_undegraded":"2024-09-19T08:41:49.971654+0000","last_fullsized":"2024-09-19T08:41:49.971654+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:54.008032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971307+0000","last_change":"2024-09-19T08:41:47.985774+0000","last_active":"2024-09-19T08:41:48.971307+0000","last_peered":"2024-09-19T08:41:48.971307+0000","last_clean":"2024-09-19T08:41:48.971307+0000","last_became_active":"2024-09-19T08:41:47.984953+0000","last_became_peered":"2024-09-19T08:41:47.984953+0000","last_unstale":"2024-09-19T08:41:48.971307+0000","last_undegraded":"2024-09-19T08:41:48.971307+0000","last_fullsized":"2024-09-19T08:41:48.971307+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:42.104552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.950609+0000","last_change":"2024-09-19T08:41:46.973134+0000","last_active":"2024-09-19T08:41:47.950609+0000","last_peered":"2024-09-19T08:41:47.950609+0000","last_clean":"2024-09-19T08:41:47.950609+0000","last_became_active":"2024-09-19T08:41:46.971993+0000","last_became_peered":"2024-09-19T08:41:46.971993+0000","last_unstale":"2024-09-19T08:41:47.950609+0000","last_undegraded":"2024-09-19T08:41:47.950609+0000","last_fullsized":"2024-09-19T08:41:47.950609+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:39:23.746148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.993733+0000","last_change":"2024-09-19T08:41:47.970744+0000","last_active":"2024-09-19T08:41:47.993733+0000","last_peered":"2024-09-19T08:41:47.993733+0000","last_clean":"2024-09-19T08:41:47.993733+0000","last_became_active":"2024-09-19T08:41:47.969645+0000","last_became_peered":"2024-09-19T08:41:47.969645+0000","last_unstale":"2024-09-19T08:41:47.993733+0000","last_undegraded":"2024-09-19T08:41:47.993733+0000","last_fullsized":"2024-09-19T08:41:47.993733+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:52.726130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"56'8","reported_seq":20,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.002402+0000","last_change":"2024-09-19T08:41:46.971824+0000","last_active":"2024-09-19T08:41:48.002402+0000","last_peered":"2024-09-19T08:41:48.002402+0000","last_clean":"2024-09-19T08:41:48.002402+0000","last_became_active":"2024-09-19T08:41:46.971096+0000","last_became_peered":"2024-09-19T08:41:46.971096+0000","last_unstale":"2024-09-19T08:41:48.002402+0000","last_undegraded":"2024-09-19T08:41:48.002402+0000","last_fullsized":"2024-09-19T08:41:48.002402+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:41:25.735590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.994127+0000","last_change":"2024-09-19T08:41:47.972240+0000","last_active":"2024-09-19T08:41:47.994127+0000","last_peered":"2024-09-19T08:41:47.994127+0000","last_clean":"2024-09-19T08:41:47.994127+0000","last_became_active":"2024-09-19T08:41:47.971284+0000","last_became_peered":"2024-09-19T08:41:47.971284+0000","last_unstale":"2024-09-19T08:41:47.994127+0000","last_undegraded":"2024-09-19T08:41:47.994127+0000","last_fullsized":"2024-09-19T08:41:47.994127+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:43.544741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"56'1","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.989173+0000","last_change":"2024-09-19T08:41:46.956816+0000","last_active":"2024-09-19T08:41:47.989173+0000","last_peered":"2024-09-19T08:41:47.989173+0000","last_clean":"2024-09-19T08:41:47.989173+0000","last_became_active":"2024-09-19T08:41:46.956468+0000","last_became_peered":"2024-09-19T08:41:46.956468+0000","last_unstale":"2024-09-19T08:41:47.989173+0000","last_undegraded":"2024-09-19T08:41:47.989173+0000","last_fullsized":"2024-09-19T08:41:47.989173+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:11.860203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.991287+0000","last_change":"2024-09-19T08:41:47.974760+0000","last_active":"2024-09-19T08:41:47.991287+0000","last_peered":"2024-09-19T08:41:47.991287+0000","last_clean":"2024-09-19T08:41:47.991287+0000","last_became_active":"2024-09-19T08:41:47.972626+0000","last_became_peered":"2024-09-19T08:41:47.972626+0000","last_unstale":"2024-09-19T08:41:47.991287+0000","last_undegraded":"2024-09-19T08:41:47.991287+0000","last_fullsized":"2024-09-19T08:41:47.991287+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:09.481675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"56'2","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.000795+0000","last_change":"2024-09-19T08:41:46.956922+0000","last_active":"2024-09-19T08:41:48.000795+0000","last_peered":"2024-09-19T08:41:48.000795+0000","last_clean":"2024-09-19T08:41:48.000795+0000","last_became_active":"2024-09-19T08:41:46.956049+0000","last_became_peered":"2024-09-19T08:41:46.956049+0000","last_unstale":"2024-09-19T08:41:48.000795+0000","last_undegraded":"2024-09-19T08:41:48.000795+0000","last_fullsized":"2024-09-19T08:41:48.000795+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:51.722459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.988936+0000","last_change":"2024-09-19T08:41:47.987128+0000","last_active":"2024-09-19T08:41:47.988936+0000","last_peered":"2024-09-19T08:41:47.988936+0000","last_clean":"2024-09-19T08:41:47.988936+0000","last_became_active":"2024-09-19T08:41:47.986850+0000","last_became_peered":"2024-09-19T08:41:47.986850+0000","last_unstale":"2024-09-19T08:41:47.988936+0000","last_undegraded":"2024-09-19T08:41:47.988936+0000","last_fullsized":"2024-09-19T08:41:47.988936+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:13.337414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"56'1","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.002911+0000","last_change":"2024-09-19T08:41:46.971406+0000","last_active":"2024-09-19T08:41:48.002911+0000","last_peered":"2024-09-19T08:41:48.002911+0000","last_clean":"2024-09-19T08:41:48.002911+0000","last_became_active":"2024-09-19T08:41:46.970580+0000","last_became_peered":"2024-09-19T08:41:46.970580+0000","last_unstale":"2024-09-19T08:41:48.002911+0000","last_undegraded":"2024-09-19T08:41:48.002911+0000","last_fullsized":"2024-09-19T08:41:48.002911+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:52:40.220622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.002262+0000","last_change":"2024-09-19T08:41:47.979548+0000","last_active":"2024-09-19T08:41:48.002262+0000","last_peered":"2024-09-19T08:41:48.002262+0000","last_clean":"2024-09-19T08:41:48.002262+0000","last_became_active":"2024-09-19T08:41:47.978787+0000","last_became_peered":"2024-09-19T08:41:47.978787+0000","last_unstale":"2024-09-19T08:41:48.002262+0000","last_undegraded":"2024-09-19T08:41:48.002262+0000","last_fullsized":"2024-09-19T08:41:48.002262+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:09.481675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.939420+0000","last_change":"2024-09-19T08:41:46.958787+0000","last_active":"2024-09-19T08:41:47.939420+0000","last_peered":"2024-09-19T08:41:47.939420+0000","last_clean":"2024-09-19T08:41:47.939420+0000","last_became_active":"2024-09-19T08:41:46.958288+0000","last_became_peered":"2024-09-19T08:41:46.958288+0000","last_unstale":"2024-09-19T08:41:47.939420+0000","last_undegraded":"2024-09-19T08:41:47.939420+0000","last_fullsized":"2024-09-19T08:41:47.939420+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:32:43.669648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.991121+0000","last_change":"2024-09-19T08:41:47.959094+0000","last_active":"2024-09-19T08:41:47.991121+0000","last_peered":"2024-09-19T08:41:47.991121+0000","last_clean":"2024-09-19T08:41:47.991121+0000","last_became_active":"2024-09-19T08:41:47.958309+0000","last_became_peered":"2024-09-19T08:41:47.958309+0000","last_unstale":"2024-09-19T08:41:47.991121+0000","last_undegraded":"2024-09-19T08:41:47.991121+0000","last_fullsized":"2024-09-19T08:41:47.991121+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:52.726130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"56'2","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.996240+0000","last_change":"2024-09-19T08:41:46.955663+0000","last_active":"2024-09-19T08:41:47.996240+0000","last_peered":"2024-09-19T08:41:47.996240+0000","last_clean":"2024-09-19T08:41:47.996240+0000","last_became_active":"2024-09-19T08:41:46.955237+0000","last_became_peered":"2024-09-19T08:41:46.955237+0000","last_unstale":"2024-09-19T08:41:47.996240+0000","last_undegraded":"2024-09-19T08:41:47.996240+0000","last_fullsized":"2024-09-19T08:41:47.996240+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:49:41.535703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.981832+0000","last_change":"2024-09-19T08:41:47.979199+0000","last_active":"2024-09-19T08:41:47.981832+0000","last_peered":"2024-09-19T08:41:47.981832+0000","last_clean":"2024-09-19T08:41:47.981832+0000","last_became_active":"2024-09-19T08:41:47.978858+0000","last_became_peered":"2024-09-19T08:41:47.978858+0000","last_unstale":"2024-09-19T08:41:47.981832+0000","last_undegraded":"2024-09-19T08:41:47.981832+0000","last_fullsized":"2024-09-19T08:41:47.981832+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:08:18.593800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.970964+0000","last_change":"2024-09-19T08:41:46.957163+0000","last_active":"2024-09-19T08:41:48.970964+0000","last_peered":"2024-09-19T08:41:48.970964+0000","last_clean":"2024-09-19T08:41:48.970964+0000","last_became_active":"2024-09-19T08:41:46.956745+0000","last_became_peered":"2024-09-19T08:41:46.956745+0000","last_unstale":"2024-09-19T08:41:48.970964+0000","last_undegraded":"2024-09-19T08:41:48.970964+0000","last_fullsized":"2024-09-19T08:41:48.970964+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:51:54.055870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.965010+0000","last_change":"2024-09-19T08:41:47.962932+0000","last_active":"2024-09-19T08:41:47.965010+0000","last_peered":"2024-09-19T08:41:47.965010+0000","last_clean":"2024-09-19T08:41:47.965010+0000","last_became_active":"2024-09-19T08:41:47.962192+0000","last_became_peered":"2024-09-19T08:41:47.962192+0000","last_unstale":"2024-09-19T08:41:47.965010+0000","last_undegraded":"2024-09-19T08:41:47.965010+0000","last_fullsized":"2024-09-19T08:41:47.965010+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:17.405379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.949954+0000","last_change":"2024-09-19T08:41:46.970587+0000","last_active":"2024-09-19T08:41:47.949954+0000","last_peered":"2024-09-19T08:41:47.949954+0000","last_clean":"2024-09-19T08:41:47.949954+0000","last_became_active":"2024-09-19T08:41:46.968910+0000","last_became_peered":"2024-09-19T08:41:46.968910+0000","last_unstale":"2024-09-19T08:41:47.949954+0000","last_undegraded":"2024-09-19T08:41:47.949954+0000","last_fullsized":"2024-09-19T08:41:47.949954+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:37.631023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.993141+0000","last_change":"2024-09-19T08:41:47.975389+0000","last_active":"2024-09-19T08:41:47.993141+0000","last_peered":"2024-09-19T08:41:47.993141+0000","last_clean":"2024-09-19T08:41:47.993141+0000","last_became_active":"2024-09-19T08:41:47.974667+0000","last_became_peered":"2024-09-19T08:41:47.974667+0000","last_unstale":"2024-09-19T08:41:47.993141+0000","last_undegraded":"2024-09-19T08:41:47.993141+0000","last_fullsized":"2024-09-19T08:41:47.993141+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:17.405379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971357+0000","last_change":"2024-09-19T08:41:46.949050+0000","last_active":"2024-09-19T08:41:48.971357+0000","last_peered":"2024-09-19T08:41:48.971357+0000","last_clean":"2024-09-19T08:41:48.971357+0000","last_became_active":"2024-09-19T08:41:46.948768+0000","last_became_peered":"2024-09-19T08:41:46.948768+0000","last_unstale":"2024-09-19T08:41:48.971357+0000","last_undegraded":"2024-09-19T08:41:48.971357+0000","last_fullsized":"2024-09-19T08:41:48.971357+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:25.916697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.991230+0000","last_change":"2024-09-19T08:41:47.974663+0000","last_active":"2024-09-19T08:41:47.991230+0000","last_peered":"2024-09-19T08:41:47.991230+0000","last_clean":"2024-09-19T08:41:47.991230+0000","last_became_active":"2024-09-19T08:41:47.972510+0000","last_became_peered":"2024-09-19T08:41:47.972510+0000","last_unstale":"2024-09-19T08:41:47.991230+0000","last_undegraded":"2024-09-19T08:41:47.991230+0000","last_fullsized":"2024-09-19T08:41:47.991230+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:41.615523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.940482+0000","last_change":"2024-09-19T08:41:46.958587+0000","last_active":"2024-09-19T08:41:47.940482+0000","last_peered":"2024-09-19T08:41:47.940482+0000","last_clean":"2024-09-19T08:41:47.940482+0000","last_became_active":"2024-09-19T08:41:46.958146+0000","last_became_peered":"2024-09-19T08:41:46.958146+0000","last_unstale":"2024-09-19T08:41:47.940482+0000","last_undegraded":"2024-09-19T08:41:47.940482+0000","last_fullsized":"2024-09-19T08:41:47.940482+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:42.541069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971266+0000","last_change":"2024-09-19T08:41:47.967496+0000","last_active":"2024-09-19T08:41:48.971266+0000","last_peered":"2024-09-19T08:41:48.971266+0000","last_clean":"2024-09-19T08:41:48.971266+0000","last_became_active":"2024-09-19T08:41:47.966684+0000","last_became_peered":"2024-09-19T08:41:47.966684+0000","last_unstale":"2024-09-19T08:41:48.971266+0000","last_undegraded":"2024-09-19T08:41:48.971266+0000","last_fullsized":"2024-09-19T08:41:48.971266+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:44:15.639843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.972072+0000","last_change":"2024-09-19T08:41:46.957237+0000","last_active":"2024-09-19T08:41:49.972072+0000","last_peered":"2024-09-19T08:41:49.972072+0000","last_clean":"2024-09-19T08:41:49.972072+0000","last_became_active":"2024-09-19T08:41:46.956480+0000","last_became_peered":"2024-09-19T08:41:46.956480+0000","last_unstale":"2024-09-19T08:41:49.972072+0000","last_undegraded":"2024-09-19T08:41:49.972072+0000","last_fullsized":"2024-09-19T08:41:49.972072+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:14.215825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.979216+0000","last_change":"2024-09-19T08:41:47.972895+0000","last_active":"2024-09-19T08:41:47.979216+0000","last_peered":"2024-09-19T08:41:47.979216+0000","last_clean":"2024-09-19T08:41:47.979216+0000","last_became_active":"2024-09-19T08:41:47.972572+0000","last_became_peered":"2024-09-19T08:41:47.972572+0000","last_unstale":"2024-09-19T08:41:47.979216+0000","last_undegraded":"2024-09-19T08:41:47.979216+0000","last_fullsized":"2024-09-19T08:41:47.979216+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:04:26.313098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.955154+0000","last_change":"2024-09-19T08:41:46.976718+0000","last_active":"2024-09-19T08:41:47.955154+0000","last_peered":"2024-09-19T08:41:47.955154+0000","last_clean":"2024-09-19T08:41:47.955154+0000","last_became_active":"2024-09-19T08:41:46.975905+0000","last_became_peered":"2024-09-19T08:41:46.975905+0000","last_unstale":"2024-09-19T08:41:47.955154+0000","last_undegraded":"2024-09-19T08:41:47.955154+0000","last_fullsized":"2024-09-19T08:41:47.955154+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:32:43.669648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.978612+0000","last_change":"2024-09-19T08:41:47.976156+0000","last_active":"2024-09-19T08:41:47.978612+0000","last_peered":"2024-09-19T08:41:47.978612+0000","last_clean":"2024-09-19T08:41:47.978612+0000","last_became_active":"2024-09-19T08:41:47.975738+0000","last_became_peered":"2024-09-19T08:41:47.975738+0000","last_unstale":"2024-09-19T08:41:47.978612+0000","last_undegraded":"2024-09-19T08:41:47.978612+0000","last_fullsized":"2024-09-19T08:41:47.978612+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:04:25.609497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.970965+0000","last_change":"2024-09-19T08:41:46.976793+0000","last_active":"2024-09-19T08:41:48.970965+0000","last_peered":"2024-09-19T08:41:48.970965+0000","last_clean":"2024-09-19T08:41:48.970965+0000","last_became_active":"2024-09-19T08:41:46.976404+0000","last_became_peered":"2024-09-19T08:41:46.976404+0000","last_unstale":"2024-09-19T08:41:48.970965+0000","last_undegraded":"2024-09-19T08:41:48.970965+0000","last_fullsized":"2024-09-19T08:41:48.970965+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:05.984736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.994385+0000","last_change":"2024-09-19T08:41:47.974129+0000","last_active":"2024-09-19T08:41:47.994385+0000","last_peered":"2024-09-19T08:41:47.994385+0000","last_clean":"2024-09-19T08:41:47.994385+0000","last_became_active":"2024-09-19T08:41:47.973452+0000","last_became_peered":"2024-09-19T08:41:47.973452+0000","last_unstale":"2024-09-19T08:41:47.994385+0000","last_undegraded":"2024-09-19T08:41:47.994385+0000","last_fullsized":"2024-09-19T08:41:47.994385+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:17.405379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971581+0000","last_change":"2024-09-19T08:41:46.958803+0000","last_active":"2024-09-19T08:41:48.971581+0000","last_peered":"2024-09-19T08:41:48.971581+0000","last_clean":"2024-09-19T08:41:48.971581+0000","last_became_active":"2024-09-19T08:41:46.957741+0000","last_became_peered":"2024-09-19T08:41:46.957741+0000","last_unstale":"2024-09-19T08:41:48.971581+0000","last_undegraded":"2024-09-19T08:41:48.971581+0000","last_fullsized":"2024-09-19T08:41:48.971581+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:28.813825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.970699+0000","last_change":"2024-09-19T08:41:47.977309+0000","last_active":"2024-09-19T08:41:48.970699+0000","last_peered":"2024-09-19T08:41:48.970699+0000","last_clean":"2024-09-19T08:41:48.970699+0000","last_became_active":"2024-09-19T08:41:47.976991+0000","last_became_peered":"2024-09-19T08:41:47.976991+0000","last_unstale":"2024-09-19T08:41:48.970699+0000","last_undegraded":"2024-09-19T08:41:48.970699+0000","last_fullsized":"2024-09-19T08:41:48.970699+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:56:40.725742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.970759+0000","last_change":"2024-09-19T08:41:46.954225+0000","last_active":"2024-09-19T08:41:48.970759+0000","last_peered":"2024-09-19T08:41:48.970759+0000","last_clean":"2024-09-19T08:41:48.970759+0000","last_became_active":"2024-09-19T08:41:46.953891+0000","last_became_peered":"2024-09-19T08:41:46.953891+0000","last_unstale":"2024-09-19T08:41:48.970759+0000","last_undegraded":"2024-09-19T08:41:48.970759+0000","last_fullsized":"2024-09-19T08:41:48.970759+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:59:22.248802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.984239+0000","last_change":"2024-09-19T08:41:47.978333+0000","last_active":"2024-09-19T08:41:47.984239+0000","last_peered":"2024-09-19T08:41:47.984239+0000","last_clean":"2024-09-19T08:41:47.984239+0000","last_became_active":"2024-09-19T08:41:47.977872+0000","last_became_peered":"2024-09-19T08:41:47.977872+0000","last_unstale":"2024-09-19T08:41:47.984239+0000","last_undegraded":"2024-09-19T08:41:47.984239+0000","last_fullsized":"2024-09-19T08:41:47.984239+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:05:29.016642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.965580+0000","last_change":"2024-09-19T08:41:46.971660+0000","last_active":"2024-09-19T08:41:47.965580+0000","last_peered":"2024-09-19T08:41:47.965580+0000","last_clean":"2024-09-19T08:41:47.965580+0000","last_became_active":"2024-09-19T08:41:46.971322+0000","last_became_peered":"2024-09-19T08:41:46.971322+0000","last_unstale":"2024-09-19T08:41:47.965580+0000","last_undegraded":"2024-09-19T08:41:47.965580+0000","last_fullsized":"2024-09-19T08:41:47.965580+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:42:15.264769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.971766+0000","last_change":"2024-09-19T08:41:47.970806+0000","last_active":"2024-09-19T08:41:47.971766+0000","last_peered":"2024-09-19T08:41:47.971766+0000","last_clean":"2024-09-19T08:41:47.971766+0000","last_became_active":"2024-09-19T08:41:47.970054+0000","last_became_peered":"2024-09-19T08:41:47.970054+0000","last_unstale":"2024-09-19T08:41:47.971766+0000","last_undegraded":"2024-09-19T08:41:47.971766+0000","last_fullsized":"2024-09-19T08:41:47.971766+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:15:45.598587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.971704+0000","last_change":"2024-09-19T08:41:46.978092+0000","last_active":"2024-09-19T08:41:49.971704+0000","last_peered":"2024-09-19T08:41:49.971704+0000","last_clean":"2024-09-19T08:41:49.971704+0000","last_became_active":"2024-09-19T08:41:46.977408+0000","last_became_peered":"2024-09-19T08:41:46.977408+0000","last_unstale":"2024-09-19T08:41:49.971704+0000","last_undegraded":"2024-09-19T08:41:49.971704+0000","last_fullsized":"2024-09-19T08:41:49.971704+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:11:43.892869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":231,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.958190+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:41:47.958190+0000","last_peered":"2024-09-19T08:41:47.958190+0000","last_clean":"2024-09-19T08:41:47.958190+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:41:47.958190+0000","last_undegraded":"2024-09-19T08:41:47.958190+0000","last_fullsized":"2024-09-19T08:41:47.958190+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509487,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1265664,"data_stored":863744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509489,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27688,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":671744,"data_stored":273868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509489,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27692,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":675840,"data_stored":274828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509486,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31844,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":671744,"data_stored":273272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509488,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31844,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":667648,"data_stored":273306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509486,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1273856,"data_stored":865778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574896,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31848,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339992,"statfs":{"total":96636764160,"available":96604151808,"internally_reserved":0,"allocated":675840,"data_stored":279516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542191,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31828,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340012,"statfs":{"total":96636764160,"available":96604172288,"internally_reserved":0,"allocated":651264,"data_stored":266441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:41:53.585 INFO:tasks.ceph.mds.a.smithi098.stderr:2024-09-19T08:41:53.586+0000 7feba77d8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T08:41:55.393 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:41:55.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.526+0000 7f2e59206640 1 Processor -- start 2024-09-19T08:41:55.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.526+0000 7f2e59206640 1 -- start start 2024-09-19T08:41:55.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.526+0000 7f2e59206640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2e54158fd0 0x7f2e540b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:55.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.526+0000 7f2e59206640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e540b4360 0x7f2e5406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:55.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.526+0000 7f2e59206640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e540ccc60 con 0x7f2e5406c1d0 2024-09-19T08:41:55.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.526+0000 7f2e59206640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f2e541593b0 con 0x7f2e54158fd0 2024-09-19T08:41:55.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.526+0000 7f2e59206640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2e5406c600 con 0x7f2e540b4360 2024-09-19T08:41:55.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.526+0000 7f2e52d9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2e54158fd0 0x7f2e540b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:55.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.526+0000 7f2e52d9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2e54158fd0 0x7f2e540b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47036/0 (socket says 172.21.15.98:47036) 2024-09-19T08:41:55.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.526+0000 7f2e52d9d640 1 -- 172.21.15.98:0/3468832091 learned_addr learned my addr 172.21.15.98:0/3468832091 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:55.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.526+0000 7f2e4bfff640 1 --2- 172.21.15.98:0/3468832091 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e540b4360 0x7f2e5406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:55.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.526+0000 7f2e52d9d640 1 -- 172.21.15.98:0/3468832091 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e540b4360 msgr2=0x7f2e5406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:55.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.526+0000 7f2e52d9d640 1 --2- 172.21.15.98:0/3468832091 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e540b4360 0x7f2e5406bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:55.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.526+0000 7f2e52d9d640 1 -- 172.21.15.98:0/3468832091 >> v1:172.21.15.139:6789/0 conn(0x7f2e5406c1d0 legacy=0x7f2e5415efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:55.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.526+0000 7f2e52d9d640 1 -- 172.21.15.98:0/3468832091 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e540ccf40 con 0x7f2e54158fd0 2024-09-19T08:41:55.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.526+0000 7f2e4bfff640 1 --2- 172.21.15.98:0/3468832091 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e540b4360 0x7f2e5406bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:55.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.530+0000 7f2e52d9d640 1 --2- 172.21.15.98:0/3468832091 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e54158fd0 0x7f2e540b3e20 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c009870 tx=0x7f2e3c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d42afc50d38dcc89 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:55.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.530+0000 7f2e5259c640 1 -- 172.21.15.98:0/3468832091 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c0ab620 con 0x7f2e54158fd0 2024-09-19T08:41:55.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.530+0000 7f2e5259c640 1 -- 172.21.15.98:0/3468832091 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e3c0b0070 con 0x7f2e54158fd0 2024-09-19T08:41:55.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.530+0000 7f2e5259c640 1 -- 172.21.15.98:0/3468832091 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c0ab9e0 con 0x7f2e54158fd0 2024-09-19T08:41:55.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.530+0000 7f2e59206640 1 -- 172.21.15.98:0/3468832091 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e54158fd0 msgr2=0x7f2e540b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:55.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.530+0000 7f2e59206640 1 --2- 172.21.15.98:0/3468832091 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e54158fd0 0x7f2e540b3e20 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c009870 tx=0x7f2e3c01c460 comp rx=0 tx=0).stop 2024-09-19T08:41:55.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.530+0000 7f2e59206640 1 -- 172.21.15.98:0/3468832091 shutdown_connections 2024-09-19T08:41:55.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.530+0000 7f2e59206640 1 --2- 172.21.15.98:0/3468832091 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e540b4360 0x7f2e5406bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:55.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.530+0000 7f2e59206640 1 --2- 172.21.15.98:0/3468832091 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e54158fd0 0x7f2e540b3e20 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:55.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.530+0000 7f2e59206640 1 -- 172.21.15.98:0/3468832091 >> 172.21.15.98:0/3468832091 conn(0x7f2e540b2010 msgr2=0x7f2e540b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:55.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.530+0000 7f2e59206640 1 -- 172.21.15.98:0/3468832091 shutdown_connections 2024-09-19T08:41:55.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.530+0000 7f2e59206640 1 -- 172.21.15.98:0/3468832091 wait complete. 2024-09-19T08:41:55.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.530+0000 7f2e59206640 1 Processor -- start 2024-09-19T08:41:55.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.530+0000 7f2e59206640 1 -- start start 2024-09-19T08:41:55.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.530+0000 7f2e59206640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e5406c1d0 0x7f2e5414fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:55.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.530+0000 7f2e59206640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e540b4360 0x7f2e541644e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:55.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.530+0000 7f2e59206640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e54158fd0 0x7f2e54164a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:55.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.530+0000 7f2e59206640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5415c920 con 0x7f2e5406c1d0 2024-09-19T08:41:55.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.530+0000 7f2e59206640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2e5415c7a0 con 0x7f2e540b4360 2024-09-19T08:41:55.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.530+0000 7f2e59206640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2e54166100 con 0x7f2e54158fd0 2024-09-19T08:41:55.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.530+0000 7f2e52d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e5406c1d0 0x7f2e5414fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:55.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.530+0000 7f2e52d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e5406c1d0 0x7f2e5414fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47046/0 (socket says 172.21.15.98:47046) 2024-09-19T08:41:55.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.530+0000 7f2e52d9d640 1 -- 172.21.15.98:0/1060233850 learned_addr learned my addr 172.21.15.98:0/1060233850 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:55.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.530+0000 7f2e4bfff640 1 --2- 172.21.15.98:0/1060233850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e540b4360 0x7f2e541644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:55.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.530+0000 7f2e5359e640 1 --2- 172.21.15.98:0/1060233850 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e54158fd0 0x7f2e54164a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:55.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.530+0000 7f2e52d9d640 1 -- 172.21.15.98:0/1060233850 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e54158fd0 msgr2=0x7f2e54164a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:55.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.530+0000 7f2e52d9d640 1 --2- 172.21.15.98:0/1060233850 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e54158fd0 0x7f2e54164a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:55.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.530+0000 7f2e52d9d640 1 -- 172.21.15.98:0/1060233850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e540b4360 msgr2=0x7f2e541644e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:55.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.530+0000 7f2e52d9d640 1 --2- 172.21.15.98:0/1060233850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e540b4360 0x7f2e541644e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:55.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.530+0000 7f2e52d9d640 1 -- 172.21.15.98:0/1060233850 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e54152a20 con 0x7f2e5406c1d0 2024-09-19T08:41:55.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.530+0000 7f2e4bfff640 1 --2- 172.21.15.98:0/1060233850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e540b4360 0x7f2e541644e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:55.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.534+0000 7f2e52d9d640 1 --2- 172.21.15.98:0/1060233850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e5406c1d0 0x7f2e5414fa80 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c01ab10 tx=0x7f2e3c01aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:55.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.534+0000 7f2e4b7fe640 1 -- 172.21.15.98:0/1060233850 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c0be3f0 con 0x7f2e5406c1d0 2024-09-19T08:41:55.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.534+0000 7f2e4b7fe640 1 -- 172.21.15.98:0/1060233850 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e3c0be590 con 0x7f2e5406c1d0 2024-09-19T08:41:55.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.534+0000 7f2e59206640 1 -- 172.21.15.98:0/1060233850 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e54152d00 con 0x7f2e5406c1d0 2024-09-19T08:41:55.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.534+0000 7f2e4b7fe640 1 -- 172.21.15.98:0/1060233850 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c0be980 con 0x7f2e5406c1d0 2024-09-19T08:41:55.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.534+0000 7f2e59206640 1 -- 172.21.15.98:0/1060233850 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e541526b0 con 0x7f2e5406c1d0 2024-09-19T08:41:55.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.534+0000 7f2e59206640 1 -- 172.21.15.98:0/1060233850 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e5406bc90 con 0x7f2e5406c1d0 2024-09-19T08:41:55.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.534+0000 7f2e4b7fe640 1 -- 172.21.15.98:0/1060233850 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f2e3c0b43f0 con 0x7f2e5406c1d0 2024-09-19T08:41:55.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.534+0000 7f2e4b7fe640 1 --2- 172.21.15.98:0/1060233850 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2e34067a70 0x7f2e34069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:55.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.534+0000 7f2e4b7fe640 1 -- 172.21.15.98:0/1060233850 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f2e3c120520 con 0x7f2e5406c1d0 2024-09-19T08:41:55.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.534+0000 7f2e4bfff640 1 --2- 172.21.15.98:0/1060233850 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2e34067a70 0x7f2e34069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:55.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.534+0000 7f2e4bfff640 1 --2- 172.21.15.98:0/1060233850 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2e34067a70 0x7f2e34069f30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2e5415b0b0 tx=0x7f2e44008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:55.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.538+0000 7f2e4b7fe640 1 -- 172.21.15.98:0/1060233850 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2e3c0c4060 con 0x7f2e5406c1d0 2024-09-19T08:41:55.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.698+0000 7f2e59206640 1 -- 172.21.15.98:0/1060233850 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2e5406d730 con 0x7f2e34067a70 2024-09-19T08:41:55.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.706+0000 7f2e4b7fe640 1 -- 172.21.15.98:0/1060233850 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251923 (secure 0 0 0) 0x7f2e5406d730 con 0x7f2e34067a70 2024-09-19T08:41:55.707 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:41:55.709 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:41:55.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.710+0000 7f2e59206640 1 -- 172.21.15.98:0/1060233850 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2e34067a70 msgr2=0x7f2e34069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:55.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.710+0000 7f2e59206640 1 --2- 172.21.15.98:0/1060233850 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2e34067a70 0x7f2e34069f30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2e5415b0b0 tx=0x7f2e44008040 comp rx=0 tx=0).stop 2024-09-19T08:41:55.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.710+0000 7f2e59206640 1 -- 172.21.15.98:0/1060233850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e5406c1d0 msgr2=0x7f2e5414fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:55.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.710+0000 7f2e59206640 1 --2- 172.21.15.98:0/1060233850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e5406c1d0 0x7f2e5414fa80 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c01ab10 tx=0x7f2e3c01aca0 comp rx=0 tx=0).stop 2024-09-19T08:41:55.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.710+0000 7f2e59206640 1 -- 172.21.15.98:0/1060233850 shutdown_connections 2024-09-19T08:41:55.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.710+0000 7f2e59206640 1 --2- 172.21.15.98:0/1060233850 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2e34067a70 0x7f2e34069f30 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:55.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.710+0000 7f2e59206640 1 --2- 172.21.15.98:0/1060233850 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e54158fd0 0x7f2e54164a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:55.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.710+0000 7f2e59206640 1 --2- 172.21.15.98:0/1060233850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e540b4360 0x7f2e541644e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:55.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.710+0000 7f2e59206640 1 --2- 172.21.15.98:0/1060233850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e5406c1d0 0x7f2e5414fa80 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:55.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.710+0000 7f2e59206640 1 -- 172.21.15.98:0/1060233850 >> 172.21.15.98:0/1060233850 conn(0x7f2e540b2010 msgr2=0x7f2e54165670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:55.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.710+0000 7f2e59206640 1 -- 172.21.15.98:0/1060233850 shutdown_connections 2024-09-19T08:41:55.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.710+0000 7f2e59206640 1 -- 172.21.15.98:0/1060233850 wait complete. 2024-09-19T08:41:55.727 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2024-09-19T08:41:54.098020+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":247696,"kb_used_data":6492,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754727024,"statfs":{"total":773094113280,"available":772840472576,"internally_reserved":0,"allocated":6647808,"data_stored":3415384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.166112"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:50.156190+0000","last_change":"2024-09-19T08:41:47.980122+0000","last_active":"2024-09-19T08:41:50.156190+0000","last_peered":"2024-09-19T08:41:50.156190+0000","last_clean":"2024-09-19T08:41:50.156190+0000","last_became_active":"2024-09-19T08:41:47.976398+0000","last_became_peered":"2024-09-19T08:41:47.976398+0000","last_unstale":"2024-09-19T08:41:50.156190+0000","last_undegraded":"2024-09-19T08:41:50.156190+0000","last_fullsized":"2024-09-19T08:41:50.156190+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:50:03.711796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.078718+0000","last_change":"2024-09-19T08:41:47.973338+0000","last_active":"2024-09-19T08:41:49.078718+0000","last_peered":"2024-09-19T08:41:49.078718+0000","last_clean":"2024-09-19T08:41:49.078718+0000","last_became_active":"2024-09-19T08:41:47.973065+0000","last_became_peered":"2024-09-19T08:41:47.973065+0000","last_unstale":"2024-09-19T08:41:49.078718+0000","last_undegraded":"2024-09-19T08:41:49.078718+0000","last_fullsized":"2024-09-19T08:41:49.078718+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:52.968350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:50.158024+0000","last_change":"2024-09-19T08:41:47.966283+0000","last_active":"2024-09-19T08:41:50.158024+0000","last_peered":"2024-09-19T08:41:50.158024+0000","last_clean":"2024-09-19T08:41:50.158024+0000","last_became_active":"2024-09-19T08:41:47.965563+0000","last_became_peered":"2024-09-19T08:41:47.965563+0000","last_unstale":"2024-09-19T08:41:50.158024+0000","last_undegraded":"2024-09-19T08:41:50.158024+0000","last_fullsized":"2024-09-19T08:41:50.158024+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:37.475720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.993880+0000","last_change":"2024-09-19T08:41:47.978137+0000","last_active":"2024-09-19T08:41:47.993880+0000","last_peered":"2024-09-19T08:41:47.993880+0000","last_clean":"2024-09-19T08:41:47.993880+0000","last_became_active":"2024-09-19T08:41:47.976050+0000","last_became_peered":"2024-09-19T08:41:47.976050+0000","last_unstale":"2024-09-19T08:41:47.993880+0000","last_undegraded":"2024-09-19T08:41:47.993880+0000","last_fullsized":"2024-09-19T08:41:47.993880+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:56:44.612767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:50.156123+0000","last_change":"2024-09-19T08:41:47.993040+0000","last_active":"2024-09-19T08:41:50.156123+0000","last_peered":"2024-09-19T08:41:50.156123+0000","last_clean":"2024-09-19T08:41:50.156123+0000","last_became_active":"2024-09-19T08:41:47.992767+0000","last_became_peered":"2024-09-19T08:41:47.992767+0000","last_unstale":"2024-09-19T08:41:50.156123+0000","last_undegraded":"2024-09-19T08:41:50.156123+0000","last_fullsized":"2024-09-19T08:41:50.156123+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:25:39.371805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.615325+0000","last_change":"2024-09-19T08:41:47.961838+0000","last_active":"2024-09-19T08:41:49.615325+0000","last_peered":"2024-09-19T08:41:49.615325+0000","last_clean":"2024-09-19T08:41:49.615325+0000","last_became_active":"2024-09-19T08:41:47.961116+0000","last_became_peered":"2024-09-19T08:41:47.961116+0000","last_unstale":"2024-09-19T08:41:49.615325+0000","last_undegraded":"2024-09-19T08:41:49.615325+0000","last_fullsized":"2024-09-19T08:41:49.615325+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:23:23.686083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971387+0000","last_change":"2024-09-19T08:41:47.966503+0000","last_active":"2024-09-19T08:41:48.971387+0000","last_peered":"2024-09-19T08:41:48.971387+0000","last_clean":"2024-09-19T08:41:48.971387+0000","last_became_active":"2024-09-19T08:41:47.965805+0000","last_became_peered":"2024-09-19T08:41:47.965805+0000","last_unstale":"2024-09-19T08:41:48.971387+0000","last_undegraded":"2024-09-19T08:41:48.971387+0000","last_fullsized":"2024-09-19T08:41:48.971387+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:41.615523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.078486+0000","last_change":"2024-09-19T08:41:47.969144+0000","last_active":"2024-09-19T08:41:49.078486+0000","last_peered":"2024-09-19T08:41:49.078486+0000","last_clean":"2024-09-19T08:41:49.078486+0000","last_became_active":"2024-09-19T08:41:47.968766+0000","last_became_peered":"2024-09-19T08:41:47.968766+0000","last_unstale":"2024-09-19T08:41:49.078486+0000","last_undegraded":"2024-09-19T08:41:49.078486+0000","last_fullsized":"2024-09-19T08:41:49.078486+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:08:01.156689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971951+0000","last_change":"2024-09-19T08:41:47.954679+0000","last_active":"2024-09-19T08:41:48.971951+0000","last_peered":"2024-09-19T08:41:48.971951+0000","last_clean":"2024-09-19T08:41:48.971951+0000","last_became_active":"2024-09-19T08:41:47.951685+0000","last_became_peered":"2024-09-19T08:41:47.951685+0000","last_unstale":"2024-09-19T08:41:48.971951+0000","last_undegraded":"2024-09-19T08:41:48.971951+0000","last_fullsized":"2024-09-19T08:41:48.971951+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:17.405379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971133+0000","last_change":"2024-09-19T08:41:47.977064+0000","last_active":"2024-09-19T08:41:48.971133+0000","last_peered":"2024-09-19T08:41:48.971133+0000","last_clean":"2024-09-19T08:41:48.971133+0000","last_became_active":"2024-09-19T08:41:47.976747+0000","last_became_peered":"2024-09-19T08:41:47.976747+0000","last_unstale":"2024-09-19T08:41:48.971133+0000","last_undegraded":"2024-09-19T08:41:48.971133+0000","last_fullsized":"2024-09-19T08:41:48.971133+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:37.366531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971332+0000","last_change":"2024-09-19T08:41:47.971592+0000","last_active":"2024-09-19T08:41:48.971332+0000","last_peered":"2024-09-19T08:41:48.971332+0000","last_clean":"2024-09-19T08:41:48.971332+0000","last_became_active":"2024-09-19T08:41:47.969075+0000","last_became_peered":"2024-09-19T08:41:47.969075+0000","last_unstale":"2024-09-19T08:41:48.971332+0000","last_undegraded":"2024-09-19T08:41:48.971332+0000","last_fullsized":"2024-09-19T08:41:48.971332+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:29.817497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.992937+0000","last_change":"2024-09-19T08:41:47.968996+0000","last_active":"2024-09-19T08:41:47.992937+0000","last_peered":"2024-09-19T08:41:47.992937+0000","last_clean":"2024-09-19T08:41:47.992937+0000","last_became_active":"2024-09-19T08:41:47.968254+0000","last_became_peered":"2024-09-19T08:41:47.968254+0000","last_unstale":"2024-09-19T08:41:47.992937+0000","last_undegraded":"2024-09-19T08:41:47.992937+0000","last_fullsized":"2024-09-19T08:41:47.992937+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:12.552026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.972167+0000","last_change":"2024-09-19T08:41:47.975212+0000","last_active":"2024-09-19T08:41:49.972167+0000","last_peered":"2024-09-19T08:41:49.972167+0000","last_clean":"2024-09-19T08:41:49.972167+0000","last_became_active":"2024-09-19T08:41:47.974446+0000","last_became_peered":"2024-09-19T08:41:47.974446+0000","last_unstale":"2024-09-19T08:41:49.972167+0000","last_undegraded":"2024-09-19T08:41:49.972167+0000","last_fullsized":"2024-09-19T08:41:49.972167+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:15:16.250986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971528+0000","last_change":"2024-09-19T08:41:47.962145+0000","last_active":"2024-09-19T08:41:48.971528+0000","last_peered":"2024-09-19T08:41:48.971528+0000","last_clean":"2024-09-19T08:41:48.971528+0000","last_became_active":"2024-09-19T08:41:47.961449+0000","last_became_peered":"2024-09-19T08:41:47.961449+0000","last_unstale":"2024-09-19T08:41:48.971528+0000","last_undegraded":"2024-09-19T08:41:48.971528+0000","last_fullsized":"2024-09-19T08:41:48.971528+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:09.481675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.972047+0000","last_change":"2024-09-19T08:41:47.960027+0000","last_active":"2024-09-19T08:41:48.972047+0000","last_peered":"2024-09-19T08:41:48.972047+0000","last_clean":"2024-09-19T08:41:48.972047+0000","last_became_active":"2024-09-19T08:41:47.959242+0000","last_became_peered":"2024-09-19T08:41:47.959242+0000","last_unstale":"2024-09-19T08:41:48.972047+0000","last_undegraded":"2024-09-19T08:41:48.972047+0000","last_fullsized":"2024-09-19T08:41:48.972047+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:56:44.612767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.615122+0000","last_change":"2024-09-19T08:41:47.979371+0000","last_active":"2024-09-19T08:41:49.615122+0000","last_peered":"2024-09-19T08:41:49.615122+0000","last_clean":"2024-09-19T08:41:49.615122+0000","last_became_active":"2024-09-19T08:41:47.978819+0000","last_became_peered":"2024-09-19T08:41:47.978819+0000","last_unstale":"2024-09-19T08:41:49.615122+0000","last_undegraded":"2024-09-19T08:41:49.615122+0000","last_fullsized":"2024-09-19T08:41:49.615122+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:16.379976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.078898+0000","last_change":"2024-09-19T08:41:47.977880+0000","last_active":"2024-09-19T08:41:49.078898+0000","last_peered":"2024-09-19T08:41:49.078898+0000","last_clean":"2024-09-19T08:41:49.078898+0000","last_became_active":"2024-09-19T08:41:47.977590+0000","last_became_peered":"2024-09-19T08:41:47.977590+0000","last_unstale":"2024-09-19T08:41:49.078898+0000","last_undegraded":"2024-09-19T08:41:49.078898+0000","last_fullsized":"2024-09-19T08:41:49.078898+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:38:13.315138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:50.157965+0000","last_change":"2024-09-19T08:41:47.968677+0000","last_active":"2024-09-19T08:41:50.157965+0000","last_peered":"2024-09-19T08:41:50.157965+0000","last_clean":"2024-09-19T08:41:50.157965+0000","last_became_active":"2024-09-19T08:41:47.967937+0000","last_became_peered":"2024-09-19T08:41:47.967937+0000","last_unstale":"2024-09-19T08:41:50.157965+0000","last_undegraded":"2024-09-19T08:41:50.157965+0000","last_fullsized":"2024-09-19T08:41:50.157965+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:16.379976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:50.156057+0000","last_change":"2024-09-19T08:41:47.975831+0000","last_active":"2024-09-19T08:41:50.156057+0000","last_peered":"2024-09-19T08:41:50.156057+0000","last_clean":"2024-09-19T08:41:50.156057+0000","last_became_active":"2024-09-19T08:41:47.975518+0000","last_became_peered":"2024-09-19T08:41:47.975518+0000","last_unstale":"2024-09-19T08:41:50.156057+0000","last_undegraded":"2024-09-19T08:41:50.156057+0000","last_fullsized":"2024-09-19T08:41:50.156057+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:11.850211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.078450+0000","last_change":"2024-09-19T08:41:47.973832+0000","last_active":"2024-09-19T08:41:49.078450+0000","last_peered":"2024-09-19T08:41:49.078450+0000","last_clean":"2024-09-19T08:41:49.078450+0000","last_became_active":"2024-09-19T08:41:47.973516+0000","last_became_peered":"2024-09-19T08:41:47.973516+0000","last_unstale":"2024-09-19T08:41:49.078450+0000","last_undegraded":"2024-09-19T08:41:49.078450+0000","last_fullsized":"2024-09-19T08:41:49.078450+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:13:42.871517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:50.157455+0000","last_change":"2024-09-19T08:41:47.979480+0000","last_active":"2024-09-19T08:41:50.157455+0000","last_peered":"2024-09-19T08:41:50.157455+0000","last_clean":"2024-09-19T08:41:50.157455+0000","last_became_active":"2024-09-19T08:41:47.977765+0000","last_became_peered":"2024-09-19T08:41:47.977765+0000","last_unstale":"2024-09-19T08:41:50.157455+0000","last_undegraded":"2024-09-19T08:41:50.157455+0000","last_fullsized":"2024-09-19T08:41:50.157455+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:25:54.925434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.078155+0000","last_change":"2024-09-19T08:41:47.981514+0000","last_active":"2024-09-19T08:41:49.078155+0000","last_peered":"2024-09-19T08:41:49.078155+0000","last_clean":"2024-09-19T08:41:49.078155+0000","last_became_active":"2024-09-19T08:41:47.981044+0000","last_became_peered":"2024-09-19T08:41:47.981044+0000","last_unstale":"2024-09-19T08:41:49.078155+0000","last_undegraded":"2024-09-19T08:41:49.078155+0000","last_fullsized":"2024-09-19T08:41:49.078155+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:50.621757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:50.156845+0000","last_change":"2024-09-19T08:41:47.970880+0000","last_active":"2024-09-19T08:41:50.156845+0000","last_peered":"2024-09-19T08:41:50.156845+0000","last_clean":"2024-09-19T08:41:50.156845+0000","last_became_active":"2024-09-19T08:41:47.970145+0000","last_became_peered":"2024-09-19T08:41:47.970145+0000","last_unstale":"2024-09-19T08:41:50.156845+0000","last_undegraded":"2024-09-19T08:41:50.156845+0000","last_fullsized":"2024-09-19T08:41:50.156845+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:08:18.593800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.971590+0000","last_change":"2024-09-19T08:41:47.976061+0000","last_active":"2024-09-19T08:41:49.971590+0000","last_peered":"2024-09-19T08:41:49.971590+0000","last_clean":"2024-09-19T08:41:49.971590+0000","last_became_active":"2024-09-19T08:41:47.975280+0000","last_became_peered":"2024-09-19T08:41:47.975280+0000","last_unstale":"2024-09-19T08:41:49.971590+0000","last_undegraded":"2024-09-19T08:41:49.971590+0000","last_fullsized":"2024-09-19T08:41:49.971590+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:12:23.981537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:50.156266+0000","last_change":"2024-09-19T08:41:47.977025+0000","last_active":"2024-09-19T08:41:50.156266+0000","last_peered":"2024-09-19T08:41:50.156266+0000","last_clean":"2024-09-19T08:41:50.156266+0000","last_became_active":"2024-09-19T08:41:47.975707+0000","last_became_peered":"2024-09-19T08:41:47.975707+0000","last_unstale":"2024-09-19T08:41:50.156266+0000","last_undegraded":"2024-09-19T08:41:50.156266+0000","last_fullsized":"2024-09-19T08:41:50.156266+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:37:41.853182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.990912+0000","last_change":"2024-09-19T08:41:47.966128+0000","last_active":"2024-09-19T08:41:47.990912+0000","last_peered":"2024-09-19T08:41:47.990912+0000","last_clean":"2024-09-19T08:41:47.990912+0000","last_became_active":"2024-09-19T08:41:47.965403+0000","last_became_peered":"2024-09-19T08:41:47.965403+0000","last_unstale":"2024-09-19T08:41:47.990912+0000","last_undegraded":"2024-09-19T08:41:47.990912+0000","last_fullsized":"2024-09-19T08:41:47.990912+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:43.544741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.615765+0000","last_change":"2024-09-19T08:41:47.979320+0000","last_active":"2024-09-19T08:41:49.615765+0000","last_peered":"2024-09-19T08:41:49.615765+0000","last_clean":"2024-09-19T08:41:49.615765+0000","last_became_active":"2024-09-19T08:41:47.979010+0000","last_became_peered":"2024-09-19T08:41:47.979010+0000","last_unstale":"2024-09-19T08:41:49.615765+0000","last_undegraded":"2024-09-19T08:41:49.615765+0000","last_fullsized":"2024-09-19T08:41:49.615765+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:20.635245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:50.156059+0000","last_change":"2024-09-19T08:41:47.966338+0000","last_active":"2024-09-19T08:41:50.156059+0000","last_peered":"2024-09-19T08:41:50.156059+0000","last_clean":"2024-09-19T08:41:50.156059+0000","last_became_active":"2024-09-19T08:41:47.966024+0000","last_became_peered":"2024-09-19T08:41:47.966024+0000","last_unstale":"2024-09-19T08:41:50.156059+0000","last_undegraded":"2024-09-19T08:41:50.156059+0000","last_fullsized":"2024-09-19T08:41:50.156059+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:38:58.020278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.078835+0000","last_change":"2024-09-19T08:41:47.996502+0000","last_active":"2024-09-19T08:41:49.078835+0000","last_peered":"2024-09-19T08:41:49.078835+0000","last_clean":"2024-09-19T08:41:49.078835+0000","last_became_active":"2024-09-19T08:41:47.996162+0000","last_became_peered":"2024-09-19T08:41:47.996162+0000","last_unstale":"2024-09-19T08:41:49.078835+0000","last_undegraded":"2024-09-19T08:41:49.078835+0000","last_fullsized":"2024-09-19T08:41:49.078835+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:11:18.482389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:50.157269+0000","last_change":"2024-09-19T08:41:47.966508+0000","last_active":"2024-09-19T08:41:50.157269+0000","last_peered":"2024-09-19T08:41:50.157269+0000","last_clean":"2024-09-19T08:41:50.157269+0000","last_became_active":"2024-09-19T08:41:47.962098+0000","last_became_peered":"2024-09-19T08:41:47.962098+0000","last_unstale":"2024-09-19T08:41:50.157269+0000","last_undegraded":"2024-09-19T08:41:50.157269+0000","last_fullsized":"2024-09-19T08:41:50.157269+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:14:34.451800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:50.156065+0000","last_change":"2024-09-19T08:41:47.974969+0000","last_active":"2024-09-19T08:41:50.156065+0000","last_peered":"2024-09-19T08:41:50.156065+0000","last_clean":"2024-09-19T08:41:50.156065+0000","last_became_active":"2024-09-19T08:41:47.974440+0000","last_became_peered":"2024-09-19T08:41:47.974440+0000","last_unstale":"2024-09-19T08:41:50.156065+0000","last_undegraded":"2024-09-19T08:41:50.156065+0000","last_fullsized":"2024-09-19T08:41:50.156065+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:21.834940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.615370+0000","last_change":"2024-09-19T08:41:47.972260+0000","last_active":"2024-09-19T08:41:49.615370+0000","last_peered":"2024-09-19T08:41:49.615370+0000","last_clean":"2024-09-19T08:41:49.615370+0000","last_became_active":"2024-09-19T08:41:47.971683+0000","last_became_peered":"2024-09-19T08:41:47.971683+0000","last_unstale":"2024-09-19T08:41:49.615370+0000","last_undegraded":"2024-09-19T08:41:49.615370+0000","last_fullsized":"2024-09-19T08:41:49.615370+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:31.250022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.616515+0000","last_change":"2024-09-19T08:41:47.962374+0000","last_active":"2024-09-19T08:41:49.616515+0000","last_peered":"2024-09-19T08:41:49.616515+0000","last_clean":"2024-09-19T08:41:49.616515+0000","last_became_active":"2024-09-19T08:41:47.961305+0000","last_became_peered":"2024-09-19T08:41:47.961305+0000","last_unstale":"2024-09-19T08:41:49.616515+0000","last_undegraded":"2024-09-19T08:41:49.616515+0000","last_fullsized":"2024-09-19T08:41:49.616515+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:06:53.773907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"56'4","reported_seq":18,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.616048+0000","last_change":"2024-09-19T08:41:46.976847+0000","last_active":"2024-09-19T08:41:49.616048+0000","last_peered":"2024-09-19T08:41:49.616048+0000","last_clean":"2024-09-19T08:41:49.616048+0000","last_became_active":"2024-09-19T08:41:46.976405+0000","last_became_peered":"2024-09-19T08:41:46.976405+0000","last_unstale":"2024-09-19T08:41:49.616048+0000","last_undegraded":"2024-09-19T08:41:49.616048+0000","last_fullsized":"2024-09-19T08:41:49.616048+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:52:40.220622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971359+0000","last_change":"2024-09-19T08:41:47.971091+0000","last_active":"2024-09-19T08:41:48.971359+0000","last_peered":"2024-09-19T08:41:48.971359+0000","last_clean":"2024-09-19T08:41:48.971359+0000","last_became_active":"2024-09-19T08:41:47.958633+0000","last_became_peered":"2024-09-19T08:41:47.958633+0000","last_unstale":"2024-09-19T08:41:48.971359+0000","last_undegraded":"2024-09-19T08:41:48.971359+0000","last_fullsized":"2024-09-19T08:41:48.971359+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:38.634694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.078635+0000","last_change":"2024-09-19T08:41:46.953976+0000","last_active":"2024-09-19T08:41:49.078635+0000","last_peered":"2024-09-19T08:41:49.078635+0000","last_clean":"2024-09-19T08:41:49.078635+0000","last_became_active":"2024-09-19T08:41:46.953613+0000","last_became_peered":"2024-09-19T08:41:46.953613+0000","last_unstale":"2024-09-19T08:41:49.078635+0000","last_undegraded":"2024-09-19T08:41:49.078635+0000","last_fullsized":"2024-09-19T08:41:49.078635+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:24.976526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.615937+0000","last_change":"2024-09-19T08:41:47.961793+0000","last_active":"2024-09-19T08:41:49.615937+0000","last_peered":"2024-09-19T08:41:49.615937+0000","last_clean":"2024-09-19T08:41:49.615937+0000","last_became_active":"2024-09-19T08:41:47.961008+0000","last_became_peered":"2024-09-19T08:41:47.961008+0000","last_unstale":"2024-09-19T08:41:49.615937+0000","last_undegraded":"2024-09-19T08:41:49.615937+0000","last_fullsized":"2024-09-19T08:41:49.615937+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:33.679610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.616113+0000","last_change":"2024-09-19T08:41:46.950525+0000","last_active":"2024-09-19T08:41:49.616113+0000","last_peered":"2024-09-19T08:41:49.616113+0000","last_clean":"2024-09-19T08:41:49.616113+0000","last_became_active":"2024-09-19T08:41:46.950185+0000","last_became_peered":"2024-09-19T08:41:46.950185+0000","last_unstale":"2024-09-19T08:41:49.616113+0000","last_undegraded":"2024-09-19T08:41:49.616113+0000","last_fullsized":"2024-09-19T08:41:49.616113+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:11.548355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.615957+0000","last_change":"2024-09-19T08:41:47.997038+0000","last_active":"2024-09-19T08:41:49.615957+0000","last_peered":"2024-09-19T08:41:49.615957+0000","last_clean":"2024-09-19T08:41:49.615957+0000","last_became_active":"2024-09-19T08:41:47.996654+0000","last_became_peered":"2024-09-19T08:41:47.996654+0000","last_unstale":"2024-09-19T08:41:49.615957+0000","last_undegraded":"2024-09-19T08:41:49.615957+0000","last_fullsized":"2024-09-19T08:41:49.615957+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:04:25.054924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"56'2","reported_seq":14,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.001947+0000","last_change":"2024-09-19T08:41:46.976094+0000","last_active":"2024-09-19T08:41:48.001947+0000","last_peered":"2024-09-19T08:41:48.001947+0000","last_clean":"2024-09-19T08:41:48.001947+0000","last_became_active":"2024-09-19T08:41:46.975349+0000","last_became_peered":"2024-09-19T08:41:46.975349+0000","last_unstale":"2024-09-19T08:41:48.001947+0000","last_undegraded":"2024-09-19T08:41:48.001947+0000","last_fullsized":"2024-09-19T08:41:48.001947+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:22.996047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.078515+0000","last_change":"2024-09-19T08:41:47.981348+0000","last_active":"2024-09-19T08:41:49.078515+0000","last_peered":"2024-09-19T08:41:49.078515+0000","last_clean":"2024-09-19T08:41:49.078515+0000","last_became_active":"2024-09-19T08:41:47.980623+0000","last_became_peered":"2024-09-19T08:41:47.980623+0000","last_unstale":"2024-09-19T08:41:49.078515+0000","last_undegraded":"2024-09-19T08:41:49.078515+0000","last_fullsized":"2024-09-19T08:41:49.078515+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:51:19.953620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971966+0000","last_change":"2024-09-19T08:41:46.959290+0000","last_active":"2024-09-19T08:41:48.971966+0000","last_peered":"2024-09-19T08:41:48.971966+0000","last_clean":"2024-09-19T08:41:48.971966+0000","last_became_active":"2024-09-19T08:41:46.958454+0000","last_became_peered":"2024-09-19T08:41:46.958454+0000","last_unstale":"2024-09-19T08:41:48.971966+0000","last_undegraded":"2024-09-19T08:41:48.971966+0000","last_fullsized":"2024-09-19T08:41:48.971966+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:39:23.746148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.078557+0000","last_change":"2024-09-19T08:41:47.969748+0000","last_active":"2024-09-19T08:41:49.078557+0000","last_peered":"2024-09-19T08:41:49.078557+0000","last_clean":"2024-09-19T08:41:49.078557+0000","last_became_active":"2024-09-19T08:41:47.969443+0000","last_became_peered":"2024-09-19T08:41:47.969443+0000","last_unstale":"2024-09-19T08:41:49.078557+0000","last_undegraded":"2024-09-19T08:41:49.078557+0000","last_fullsized":"2024-09-19T08:41:49.078557+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:07:03.287267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.615355+0000","last_change":"2024-09-19T08:41:46.956247+0000","last_active":"2024-09-19T08:41:49.615355+0000","last_peered":"2024-09-19T08:41:49.615355+0000","last_clean":"2024-09-19T08:41:49.615355+0000","last_became_active":"2024-09-19T08:41:46.955941+0000","last_became_peered":"2024-09-19T08:41:46.955941+0000","last_unstale":"2024-09-19T08:41:49.615355+0000","last_undegraded":"2024-09-19T08:41:49.615355+0000","last_fullsized":"2024-09-19T08:41:49.615355+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:22.996047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.971708+0000","last_change":"2024-09-19T08:41:47.984017+0000","last_active":"2024-09-19T08:41:49.971708+0000","last_peered":"2024-09-19T08:41:49.971708+0000","last_clean":"2024-09-19T08:41:49.971708+0000","last_became_active":"2024-09-19T08:41:47.983240+0000","last_became_peered":"2024-09-19T08:41:47.983240+0000","last_unstale":"2024-09-19T08:41:49.971708+0000","last_undegraded":"2024-09-19T08:41:49.971708+0000","last_fullsized":"2024-09-19T08:41:49.971708+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:22.655726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.950542+0000","last_change":"2024-09-19T08:41:46.976878+0000","last_active":"2024-09-19T08:41:47.950542+0000","last_peered":"2024-09-19T08:41:47.950542+0000","last_clean":"2024-09-19T08:41:47.950542+0000","last_became_active":"2024-09-19T08:41:46.975427+0000","last_became_peered":"2024-09-19T08:41:46.975427+0000","last_unstale":"2024-09-19T08:41:47.950542+0000","last_undegraded":"2024-09-19T08:41:47.950542+0000","last_fullsized":"2024-09-19T08:41:47.950542+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:41:25.735590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.078104+0000","last_change":"2024-09-19T08:41:47.977608+0000","last_active":"2024-09-19T08:41:49.078104+0000","last_peered":"2024-09-19T08:41:49.078104+0000","last_clean":"2024-09-19T08:41:49.078104+0000","last_became_active":"2024-09-19T08:41:47.977328+0000","last_became_peered":"2024-09-19T08:41:47.977328+0000","last_unstale":"2024-09-19T08:41:49.078104+0000","last_undegraded":"2024-09-19T08:41:49.078104+0000","last_fullsized":"2024-09-19T08:41:49.078104+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:18:51.975758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971395+0000","last_change":"2024-09-19T08:41:46.951849+0000","last_active":"2024-09-19T08:41:48.971395+0000","last_peered":"2024-09-19T08:41:48.971395+0000","last_clean":"2024-09-19T08:41:48.971395+0000","last_became_active":"2024-09-19T08:41:46.951049+0000","last_became_peered":"2024-09-19T08:41:46.951049+0000","last_unstale":"2024-09-19T08:41:48.971395+0000","last_undegraded":"2024-09-19T08:41:48.971395+0000","last_fullsized":"2024-09-19T08:41:48.971395+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:37.631023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.972453+0000","last_change":"2024-09-19T08:41:47.959107+0000","last_active":"2024-09-19T08:41:48.972453+0000","last_peered":"2024-09-19T08:41:48.972453+0000","last_clean":"2024-09-19T08:41:48.972453+0000","last_became_active":"2024-09-19T08:41:47.958257+0000","last_became_peered":"2024-09-19T08:41:47.958257+0000","last_unstale":"2024-09-19T08:41:48.972453+0000","last_undegraded":"2024-09-19T08:41:48.972453+0000","last_fullsized":"2024-09-19T08:41:48.972453+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:04:26.313098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:50.158454+0000","last_change":"2024-09-19T08:41:46.960020+0000","last_active":"2024-09-19T08:41:50.158454+0000","last_peered":"2024-09-19T08:41:50.158454+0000","last_clean":"2024-09-19T08:41:50.158454+0000","last_became_active":"2024-09-19T08:41:46.959232+0000","last_became_peered":"2024-09-19T08:41:46.959232+0000","last_unstale":"2024-09-19T08:41:50.158454+0000","last_undegraded":"2024-09-19T08:41:50.158454+0000","last_fullsized":"2024-09-19T08:41:50.158454+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:28.813825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:50.158537+0000","last_change":"2024-09-19T08:41:47.973711+0000","last_active":"2024-09-19T08:41:50.158537+0000","last_peered":"2024-09-19T08:41:50.158537+0000","last_clean":"2024-09-19T08:41:50.158537+0000","last_became_active":"2024-09-19T08:41:47.972968+0000","last_became_peered":"2024-09-19T08:41:47.972968+0000","last_unstale":"2024-09-19T08:41:50.158537+0000","last_undegraded":"2024-09-19T08:41:50.158537+0000","last_fullsized":"2024-09-19T08:41:50.158537+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:33.679610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.078806+0000","last_change":"2024-09-19T08:41:46.957881+0000","last_active":"2024-09-19T08:41:49.078806+0000","last_peered":"2024-09-19T08:41:49.078806+0000","last_clean":"2024-09-19T08:41:49.078806+0000","last_became_active":"2024-09-19T08:41:46.957503+0000","last_became_peered":"2024-09-19T08:41:46.957503+0000","last_unstale":"2024-09-19T08:41:49.078806+0000","last_undegraded":"2024-09-19T08:41:49.078806+0000","last_fullsized":"2024-09-19T08:41:49.078806+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:02:24.036512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971244+0000","last_change":"2024-09-19T08:41:47.968178+0000","last_active":"2024-09-19T08:41:48.971244+0000","last_peered":"2024-09-19T08:41:48.971244+0000","last_clean":"2024-09-19T08:41:48.971244+0000","last_became_active":"2024-09-19T08:41:47.967773+0000","last_became_peered":"2024-09-19T08:41:47.967773+0000","last_unstale":"2024-09-19T08:41:48.971244+0000","last_undegraded":"2024-09-19T08:41:48.971244+0000","last_fullsized":"2024-09-19T08:41:48.971244+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:21.843717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"56'3","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:50.158834+0000","last_change":"2024-09-19T08:41:46.957368+0000","last_active":"2024-09-19T08:41:50.158834+0000","last_peered":"2024-09-19T08:41:50.158834+0000","last_clean":"2024-09-19T08:41:50.158834+0000","last_became_active":"2024-09-19T08:41:46.956670+0000","last_became_peered":"2024-09-19T08:41:46.956670+0000","last_unstale":"2024-09-19T08:41:50.158834+0000","last_undegraded":"2024-09-19T08:41:50.158834+0000","last_fullsized":"2024-09-19T08:41:50.158834+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:37.631023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:50.158436+0000","last_change":"2024-09-19T08:41:47.977022+0000","last_active":"2024-09-19T08:41:50.158436+0000","last_peered":"2024-09-19T08:41:50.158436+0000","last_clean":"2024-09-19T08:41:50.158436+0000","last_became_active":"2024-09-19T08:41:47.973384+0000","last_became_peered":"2024-09-19T08:41:47.973384+0000","last_unstale":"2024-09-19T08:41:50.158436+0000","last_undegraded":"2024-09-19T08:41:50.158436+0000","last_fullsized":"2024-09-19T08:41:50.158436+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:20.635245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"56'6","reported_seq":20,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.972332+0000","last_change":"2024-09-19T08:41:46.953997+0000","last_active":"2024-09-19T08:41:48.972332+0000","last_peered":"2024-09-19T08:41:48.972332+0000","last_clean":"2024-09-19T08:41:48.972332+0000","last_became_active":"2024-09-19T08:41:46.953261+0000","last_became_peered":"2024-09-19T08:41:46.953261+0000","last_unstale":"2024-09-19T08:41:48.972332+0000","last_undegraded":"2024-09-19T08:41:48.972332+0000","last_fullsized":"2024-09-19T08:41:48.972332+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:41:25.735590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.991181+0000","last_change":"2024-09-19T08:41:47.973617+0000","last_active":"2024-09-19T08:41:47.991181+0000","last_peered":"2024-09-19T08:41:47.991181+0000","last_clean":"2024-09-19T08:41:47.991181+0000","last_became_active":"2024-09-19T08:41:47.973096+0000","last_became_peered":"2024-09-19T08:41:47.973096+0000","last_unstale":"2024-09-19T08:41:47.991181+0000","last_undegraded":"2024-09-19T08:41:47.991181+0000","last_fullsized":"2024-09-19T08:41:47.991181+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:42.104552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.078685+0000","last_change":"2024-09-19T08:41:46.958209+0000","last_active":"2024-09-19T08:41:49.078685+0000","last_peered":"2024-09-19T08:41:49.078685+0000","last_clean":"2024-09-19T08:41:49.078685+0000","last_became_active":"2024-09-19T08:41:46.957655+0000","last_became_peered":"2024-09-19T08:41:46.957655+0000","last_unstale":"2024-09-19T08:41:49.078685+0000","last_undegraded":"2024-09-19T08:41:49.078685+0000","last_fullsized":"2024-09-19T08:41:49.078685+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:06.027506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:50.157789+0000","last_change":"2024-09-19T08:41:47.980616+0000","last_active":"2024-09-19T08:41:50.157789+0000","last_peered":"2024-09-19T08:41:50.157789+0000","last_clean":"2024-09-19T08:41:50.157789+0000","last_became_active":"2024-09-19T08:41:47.978716+0000","last_became_peered":"2024-09-19T08:41:47.978716+0000","last_unstale":"2024-09-19T08:41:50.157789+0000","last_undegraded":"2024-09-19T08:41:50.157789+0000","last_fullsized":"2024-09-19T08:41:50.157789+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:23:44.679864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.971654+0000","last_change":"2024-09-19T08:41:46.963078+0000","last_active":"2024-09-19T08:41:49.971654+0000","last_peered":"2024-09-19T08:41:49.971654+0000","last_clean":"2024-09-19T08:41:49.971654+0000","last_became_active":"2024-09-19T08:41:46.962049+0000","last_became_peered":"2024-09-19T08:41:46.962049+0000","last_unstale":"2024-09-19T08:41:49.971654+0000","last_undegraded":"2024-09-19T08:41:49.971654+0000","last_fullsized":"2024-09-19T08:41:49.971654+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:54.008032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971307+0000","last_change":"2024-09-19T08:41:47.985774+0000","last_active":"2024-09-19T08:41:48.971307+0000","last_peered":"2024-09-19T08:41:48.971307+0000","last_clean":"2024-09-19T08:41:48.971307+0000","last_became_active":"2024-09-19T08:41:47.984953+0000","last_became_peered":"2024-09-19T08:41:47.984953+0000","last_unstale":"2024-09-19T08:41:48.971307+0000","last_undegraded":"2024-09-19T08:41:48.971307+0000","last_fullsized":"2024-09-19T08:41:48.971307+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:42:42.104552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.950609+0000","last_change":"2024-09-19T08:41:46.973134+0000","last_active":"2024-09-19T08:41:47.950609+0000","last_peered":"2024-09-19T08:41:47.950609+0000","last_clean":"2024-09-19T08:41:47.950609+0000","last_became_active":"2024-09-19T08:41:46.971993+0000","last_became_peered":"2024-09-19T08:41:46.971993+0000","last_unstale":"2024-09-19T08:41:47.950609+0000","last_undegraded":"2024-09-19T08:41:47.950609+0000","last_fullsized":"2024-09-19T08:41:47.950609+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:39:23.746148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:50.156957+0000","last_change":"2024-09-19T08:41:47.970744+0000","last_active":"2024-09-19T08:41:50.156957+0000","last_peered":"2024-09-19T08:41:50.156957+0000","last_clean":"2024-09-19T08:41:50.156957+0000","last_became_active":"2024-09-19T08:41:47.969645+0000","last_became_peered":"2024-09-19T08:41:47.969645+0000","last_unstale":"2024-09-19T08:41:50.156957+0000","last_undegraded":"2024-09-19T08:41:50.156957+0000","last_fullsized":"2024-09-19T08:41:50.156957+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:44:04.680315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"56'8","reported_seq":22,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:50.156617+0000","last_change":"2024-09-19T08:41:46.971824+0000","last_active":"2024-09-19T08:41:50.156617+0000","last_peered":"2024-09-19T08:41:50.156617+0000","last_clean":"2024-09-19T08:41:50.156617+0000","last_became_active":"2024-09-19T08:41:46.971096+0000","last_became_peered":"2024-09-19T08:41:46.971096+0000","last_unstale":"2024-09-19T08:41:50.156617+0000","last_undegraded":"2024-09-19T08:41:50.156617+0000","last_fullsized":"2024-09-19T08:41:50.156617+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:41:25.735590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:50.158064+0000","last_change":"2024-09-19T08:41:47.972240+0000","last_active":"2024-09-19T08:41:50.158064+0000","last_peered":"2024-09-19T08:41:50.158064+0000","last_clean":"2024-09-19T08:41:50.158064+0000","last_became_active":"2024-09-19T08:41:47.971284+0000","last_became_peered":"2024-09-19T08:41:47.971284+0000","last_unstale":"2024-09-19T08:41:50.158064+0000","last_undegraded":"2024-09-19T08:41:50.158064+0000","last_fullsized":"2024-09-19T08:41:50.158064+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:23:23.686083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"56'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.079256+0000","last_change":"2024-09-19T08:41:46.956816+0000","last_active":"2024-09-19T08:41:49.079256+0000","last_peered":"2024-09-19T08:41:49.079256+0000","last_clean":"2024-09-19T08:41:49.079256+0000","last_became_active":"2024-09-19T08:41:46.956468+0000","last_became_peered":"2024-09-19T08:41:46.956468+0000","last_unstale":"2024-09-19T08:41:49.079256+0000","last_undegraded":"2024-09-19T08:41:49.079256+0000","last_fullsized":"2024-09-19T08:41:49.079256+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:11.860203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.991287+0000","last_change":"2024-09-19T08:41:47.974760+0000","last_active":"2024-09-19T08:41:47.991287+0000","last_peered":"2024-09-19T08:41:47.991287+0000","last_clean":"2024-09-19T08:41:47.991287+0000","last_became_active":"2024-09-19T08:41:47.972626+0000","last_became_peered":"2024-09-19T08:41:47.972626+0000","last_unstale":"2024-09-19T08:41:47.991287+0000","last_undegraded":"2024-09-19T08:41:47.991287+0000","last_fullsized":"2024-09-19T08:41:47.991287+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:09.481675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.616342+0000","last_change":"2024-09-19T08:41:46.956922+0000","last_active":"2024-09-19T08:41:49.616342+0000","last_peered":"2024-09-19T08:41:49.616342+0000","last_clean":"2024-09-19T08:41:49.616342+0000","last_became_active":"2024-09-19T08:41:46.956049+0000","last_became_peered":"2024-09-19T08:41:46.956049+0000","last_unstale":"2024-09-19T08:41:49.616342+0000","last_undegraded":"2024-09-19T08:41:49.616342+0000","last_fullsized":"2024-09-19T08:41:49.616342+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:51.722459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.078600+0000","last_change":"2024-09-19T08:41:47.987128+0000","last_active":"2024-09-19T08:41:49.078600+0000","last_peered":"2024-09-19T08:41:49.078600+0000","last_clean":"2024-09-19T08:41:49.078600+0000","last_became_active":"2024-09-19T08:41:47.986850+0000","last_became_peered":"2024-09-19T08:41:47.986850+0000","last_unstale":"2024-09-19T08:41:49.078600+0000","last_undegraded":"2024-09-19T08:41:49.078600+0000","last_fullsized":"2024-09-19T08:41:49.078600+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:20.230154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"56'1","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.002911+0000","last_change":"2024-09-19T08:41:46.971406+0000","last_active":"2024-09-19T08:41:48.002911+0000","last_peered":"2024-09-19T08:41:48.002911+0000","last_clean":"2024-09-19T08:41:48.002911+0000","last_became_active":"2024-09-19T08:41:46.970580+0000","last_became_peered":"2024-09-19T08:41:46.970580+0000","last_unstale":"2024-09-19T08:41:48.002911+0000","last_undegraded":"2024-09-19T08:41:48.002911+0000","last_fullsized":"2024-09-19T08:41:48.002911+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:52:40.220622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:50.158704+0000","last_change":"2024-09-19T08:41:47.979548+0000","last_active":"2024-09-19T08:41:50.158704+0000","last_peered":"2024-09-19T08:41:50.158704+0000","last_clean":"2024-09-19T08:41:50.158704+0000","last_became_active":"2024-09-19T08:41:47.978787+0000","last_became_peered":"2024-09-19T08:41:47.978787+0000","last_unstale":"2024-09-19T08:41:50.158704+0000","last_undegraded":"2024-09-19T08:41:50.158704+0000","last_fullsized":"2024-09-19T08:41:50.158704+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:04:25.054924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.616098+0000","last_change":"2024-09-19T08:41:46.958787+0000","last_active":"2024-09-19T08:41:49.616098+0000","last_peered":"2024-09-19T08:41:49.616098+0000","last_clean":"2024-09-19T08:41:49.616098+0000","last_became_active":"2024-09-19T08:41:46.958288+0000","last_became_peered":"2024-09-19T08:41:46.958288+0000","last_unstale":"2024-09-19T08:41:49.616098+0000","last_undegraded":"2024-09-19T08:41:49.616098+0000","last_fullsized":"2024-09-19T08:41:49.616098+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:32:43.669648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.991121+0000","last_change":"2024-09-19T08:41:47.959094+0000","last_active":"2024-09-19T08:41:47.991121+0000","last_peered":"2024-09-19T08:41:47.991121+0000","last_clean":"2024-09-19T08:41:47.991121+0000","last_became_active":"2024-09-19T08:41:47.958309+0000","last_became_peered":"2024-09-19T08:41:47.958309+0000","last_unstale":"2024-09-19T08:41:47.991121+0000","last_undegraded":"2024-09-19T08:41:47.991121+0000","last_fullsized":"2024-09-19T08:41:47.991121+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:52.726130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.078874+0000","last_change":"2024-09-19T08:41:46.955663+0000","last_active":"2024-09-19T08:41:49.078874+0000","last_peered":"2024-09-19T08:41:49.078874+0000","last_clean":"2024-09-19T08:41:49.078874+0000","last_became_active":"2024-09-19T08:41:46.955237+0000","last_became_peered":"2024-09-19T08:41:46.955237+0000","last_unstale":"2024-09-19T08:41:49.078874+0000","last_undegraded":"2024-09-19T08:41:49.078874+0000","last_fullsized":"2024-09-19T08:41:49.078874+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:49:41.535703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.615998+0000","last_change":"2024-09-19T08:41:47.979199+0000","last_active":"2024-09-19T08:41:49.615998+0000","last_peered":"2024-09-19T08:41:49.615998+0000","last_clean":"2024-09-19T08:41:49.615998+0000","last_became_active":"2024-09-19T08:41:47.978858+0000","last_became_peered":"2024-09-19T08:41:47.978858+0000","last_unstale":"2024-09-19T08:41:49.615998+0000","last_undegraded":"2024-09-19T08:41:49.615998+0000","last_fullsized":"2024-09-19T08:41:49.615998+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:15:11.080936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.970964+0000","last_change":"2024-09-19T08:41:46.957163+0000","last_active":"2024-09-19T08:41:48.970964+0000","last_peered":"2024-09-19T08:41:48.970964+0000","last_clean":"2024-09-19T08:41:48.970964+0000","last_became_active":"2024-09-19T08:41:46.956745+0000","last_became_peered":"2024-09-19T08:41:46.956745+0000","last_unstale":"2024-09-19T08:41:48.970964+0000","last_undegraded":"2024-09-19T08:41:48.970964+0000","last_fullsized":"2024-09-19T08:41:48.970964+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:51:54.055870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.615306+0000","last_change":"2024-09-19T08:41:47.962932+0000","last_active":"2024-09-19T08:41:49.615306+0000","last_peered":"2024-09-19T08:41:49.615306+0000","last_clean":"2024-09-19T08:41:49.615306+0000","last_became_active":"2024-09-19T08:41:47.962192+0000","last_became_peered":"2024-09-19T08:41:47.962192+0000","last_unstale":"2024-09-19T08:41:49.615306+0000","last_undegraded":"2024-09-19T08:41:49.615306+0000","last_fullsized":"2024-09-19T08:41:49.615306+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:50:37.475720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.949954+0000","last_change":"2024-09-19T08:41:46.970587+0000","last_active":"2024-09-19T08:41:47.949954+0000","last_peered":"2024-09-19T08:41:47.949954+0000","last_clean":"2024-09-19T08:41:47.949954+0000","last_became_active":"2024-09-19T08:41:46.968910+0000","last_became_peered":"2024-09-19T08:41:46.968910+0000","last_unstale":"2024-09-19T08:41:47.949954+0000","last_undegraded":"2024-09-19T08:41:47.949954+0000","last_fullsized":"2024-09-19T08:41:47.949954+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:58:37.631023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.993141+0000","last_change":"2024-09-19T08:41:47.975389+0000","last_active":"2024-09-19T08:41:47.993141+0000","last_peered":"2024-09-19T08:41:47.993141+0000","last_clean":"2024-09-19T08:41:47.993141+0000","last_became_active":"2024-09-19T08:41:47.974667+0000","last_became_peered":"2024-09-19T08:41:47.974667+0000","last_unstale":"2024-09-19T08:41:47.993141+0000","last_undegraded":"2024-09-19T08:41:47.993141+0000","last_fullsized":"2024-09-19T08:41:47.993141+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:17.405379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"56'2","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971357+0000","last_change":"2024-09-19T08:41:46.949050+0000","last_active":"2024-09-19T08:41:48.971357+0000","last_peered":"2024-09-19T08:41:48.971357+0000","last_clean":"2024-09-19T08:41:48.971357+0000","last_became_active":"2024-09-19T08:41:46.948768+0000","last_became_peered":"2024-09-19T08:41:46.948768+0000","last_unstale":"2024-09-19T08:41:48.971357+0000","last_undegraded":"2024-09-19T08:41:48.971357+0000","last_fullsized":"2024-09-19T08:41:48.971357+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:25.916697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.991230+0000","last_change":"2024-09-19T08:41:47.974663+0000","last_active":"2024-09-19T08:41:47.991230+0000","last_peered":"2024-09-19T08:41:47.991230+0000","last_clean":"2024-09-19T08:41:47.991230+0000","last_became_active":"2024-09-19T08:41:47.972510+0000","last_became_peered":"2024-09-19T08:41:47.972510+0000","last_unstale":"2024-09-19T08:41:47.991230+0000","last_undegraded":"2024-09-19T08:41:47.991230+0000","last_fullsized":"2024-09-19T08:41:47.991230+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:41.615523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"57'1","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.616533+0000","last_change":"2024-09-19T08:41:46.958587+0000","last_active":"2024-09-19T08:41:49.616533+0000","last_peered":"2024-09-19T08:41:49.616533+0000","last_clean":"2024-09-19T08:41:49.616533+0000","last_became_active":"2024-09-19T08:41:46.958146+0000","last_became_peered":"2024-09-19T08:41:46.958146+0000","last_unstale":"2024-09-19T08:41:49.616533+0000","last_undegraded":"2024-09-19T08:41:49.616533+0000","last_fullsized":"2024-09-19T08:41:49.616533+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:52:42.541069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971266+0000","last_change":"2024-09-19T08:41:47.967496+0000","last_active":"2024-09-19T08:41:48.971266+0000","last_peered":"2024-09-19T08:41:48.971266+0000","last_clean":"2024-09-19T08:41:48.971266+0000","last_became_active":"2024-09-19T08:41:47.966684+0000","last_became_peered":"2024-09-19T08:41:47.966684+0000","last_unstale":"2024-09-19T08:41:48.971266+0000","last_undegraded":"2024-09-19T08:41:48.971266+0000","last_fullsized":"2024-09-19T08:41:48.971266+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:44:15.639843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.972072+0000","last_change":"2024-09-19T08:41:46.957237+0000","last_active":"2024-09-19T08:41:49.972072+0000","last_peered":"2024-09-19T08:41:49.972072+0000","last_clean":"2024-09-19T08:41:49.972072+0000","last_became_active":"2024-09-19T08:41:46.956480+0000","last_became_peered":"2024-09-19T08:41:46.956480+0000","last_unstale":"2024-09-19T08:41:49.972072+0000","last_undegraded":"2024-09-19T08:41:49.972072+0000","last_fullsized":"2024-09-19T08:41:49.972072+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:14.215825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.616167+0000","last_change":"2024-09-19T08:41:47.972895+0000","last_active":"2024-09-19T08:41:49.616167+0000","last_peered":"2024-09-19T08:41:49.616167+0000","last_clean":"2024-09-19T08:41:49.616167+0000","last_became_active":"2024-09-19T08:41:47.972572+0000","last_became_peered":"2024-09-19T08:41:47.972572+0000","last_unstale":"2024-09-19T08:41:49.616167+0000","last_undegraded":"2024-09-19T08:41:49.616167+0000","last_fullsized":"2024-09-19T08:41:49.616167+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:42:08.602488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-19T08:41:47.955154+0000","last_change":"2024-09-19T08:41:46.976718+0000","last_active":"2024-09-19T08:41:47.955154+0000","last_peered":"2024-09-19T08:41:47.955154+0000","last_clean":"2024-09-19T08:41:47.955154+0000","last_became_active":"2024-09-19T08:41:46.975905+0000","last_became_peered":"2024-09-19T08:41:46.975905+0000","last_unstale":"2024-09-19T08:41:47.955154+0000","last_undegraded":"2024-09-19T08:41:47.955154+0000","last_fullsized":"2024-09-19T08:41:47.955154+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:32:43.669648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.078992+0000","last_change":"2024-09-19T08:41:47.976156+0000","last_active":"2024-09-19T08:41:49.078992+0000","last_peered":"2024-09-19T08:41:49.078992+0000","last_clean":"2024-09-19T08:41:49.078992+0000","last_became_active":"2024-09-19T08:41:47.975738+0000","last_became_peered":"2024-09-19T08:41:47.975738+0000","last_unstale":"2024-09-19T08:41:49.078992+0000","last_undegraded":"2024-09-19T08:41:49.078992+0000","last_fullsized":"2024-09-19T08:41:49.078992+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:01:13.078668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.970965+0000","last_change":"2024-09-19T08:41:46.976793+0000","last_active":"2024-09-19T08:41:48.970965+0000","last_peered":"2024-09-19T08:41:48.970965+0000","last_clean":"2024-09-19T08:41:48.970965+0000","last_became_active":"2024-09-19T08:41:46.976404+0000","last_became_peered":"2024-09-19T08:41:46.976404+0000","last_unstale":"2024-09-19T08:41:48.970965+0000","last_undegraded":"2024-09-19T08:41:48.970965+0000","last_fullsized":"2024-09-19T08:41:48.970965+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:05.984736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:50.158157+0000","last_change":"2024-09-19T08:41:47.974129+0000","last_active":"2024-09-19T08:41:50.158157+0000","last_peered":"2024-09-19T08:41:50.158157+0000","last_clean":"2024-09-19T08:41:50.158157+0000","last_became_active":"2024-09-19T08:41:47.973452+0000","last_became_peered":"2024-09-19T08:41:47.973452+0000","last_unstale":"2024-09-19T08:41:50.158157+0000","last_undegraded":"2024-09-19T08:41:50.158157+0000","last_fullsized":"2024-09-19T08:41:50.158157+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:31.250022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.971581+0000","last_change":"2024-09-19T08:41:46.958803+0000","last_active":"2024-09-19T08:41:48.971581+0000","last_peered":"2024-09-19T08:41:48.971581+0000","last_clean":"2024-09-19T08:41:48.971581+0000","last_became_active":"2024-09-19T08:41:46.957741+0000","last_became_peered":"2024-09-19T08:41:46.957741+0000","last_unstale":"2024-09-19T08:41:48.971581+0000","last_undegraded":"2024-09-19T08:41:48.971581+0000","last_fullsized":"2024-09-19T08:41:48.971581+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:28.813825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.970699+0000","last_change":"2024-09-19T08:41:47.977309+0000","last_active":"2024-09-19T08:41:48.970699+0000","last_peered":"2024-09-19T08:41:48.970699+0000","last_clean":"2024-09-19T08:41:48.970699+0000","last_became_active":"2024-09-19T08:41:47.976991+0000","last_became_peered":"2024-09-19T08:41:47.976991+0000","last_unstale":"2024-09-19T08:41:48.970699+0000","last_undegraded":"2024-09-19T08:41:48.970699+0000","last_fullsized":"2024-09-19T08:41:48.970699+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:56:40.725742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:48.970759+0000","last_change":"2024-09-19T08:41:46.954225+0000","last_active":"2024-09-19T08:41:48.970759+0000","last_peered":"2024-09-19T08:41:48.970759+0000","last_clean":"2024-09-19T08:41:48.970759+0000","last_became_active":"2024-09-19T08:41:46.953891+0000","last_became_peered":"2024-09-19T08:41:46.953891+0000","last_unstale":"2024-09-19T08:41:48.970759+0000","last_undegraded":"2024-09-19T08:41:48.970759+0000","last_fullsized":"2024-09-19T08:41:48.970759+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:59:22.248802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.078268+0000","last_change":"2024-09-19T08:41:47.978333+0000","last_active":"2024-09-19T08:41:49.078268+0000","last_peered":"2024-09-19T08:41:49.078268+0000","last_clean":"2024-09-19T08:41:49.078268+0000","last_became_active":"2024-09-19T08:41:47.977872+0000","last_became_peered":"2024-09-19T08:41:47.977872+0000","last_unstale":"2024-09-19T08:41:49.078268+0000","last_undegraded":"2024-09-19T08:41:49.078268+0000","last_fullsized":"2024-09-19T08:41:49.078268+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:42:45.253320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.078773+0000","last_change":"2024-09-19T08:41:46.971660+0000","last_active":"2024-09-19T08:41:49.078773+0000","last_peered":"2024-09-19T08:41:49.078773+0000","last_clean":"2024-09-19T08:41:49.078773+0000","last_became_active":"2024-09-19T08:41:46.971322+0000","last_became_peered":"2024-09-19T08:41:46.971322+0000","last_unstale":"2024-09-19T08:41:49.078773+0000","last_undegraded":"2024-09-19T08:41:49.078773+0000","last_fullsized":"2024-09-19T08:41:49.078773+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:42:15.264769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.078589+0000","last_change":"2024-09-19T08:41:47.970806+0000","last_active":"2024-09-19T08:41:49.078589+0000","last_peered":"2024-09-19T08:41:49.078589+0000","last_clean":"2024-09-19T08:41:49.078589+0000","last_became_active":"2024-09-19T08:41:47.970054+0000","last_became_peered":"2024-09-19T08:41:47.970054+0000","last_unstale":"2024-09-19T08:41:49.078589+0000","last_undegraded":"2024-09-19T08:41:49.078589+0000","last_fullsized":"2024-09-19T08:41:49.078589+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:46.920839+0000","last_clean_scrub_stamp":"2024-09-19T08:41:46.920839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:06:34.255760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:49.971704+0000","last_change":"2024-09-19T08:41:46.978092+0000","last_active":"2024-09-19T08:41:49.971704+0000","last_peered":"2024-09-19T08:41:49.971704+0000","last_clean":"2024-09-19T08:41:49.971704+0000","last_became_active":"2024-09-19T08:41:46.977408+0000","last_became_peered":"2024-09-19T08:41:46.977408+0000","last_unstale":"2024-09-19T08:41:49.971704+0000","last_undegraded":"2024-09-19T08:41:49.971704+0000","last_fullsized":"2024-09-19T08:41:49.971704+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:41:45.917168+0000","last_clean_scrub_stamp":"2024-09-19T08:41:45.917168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:11:43.892869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":233,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-19T08:41:50.156661+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:41:50.156661+0000","last_peered":"2024-09-19T08:41:50.156661+0000","last_clean":"2024-09-19T08:41:50.156661+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:41:50.156661+0000","last_undegraded":"2024-09-19T08:41:50.156661+0000","last_fullsized":"2024-09-19T08:41:50.156661+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509488,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1277952,"data_stored":870044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509489,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27688,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":671744,"data_stored":273868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509490,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":688128,"data_stored":281128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509487,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31856,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339984,"statfs":{"total":96636764160,"available":96604143616,"internally_reserved":0,"allocated":684032,"data_stored":279572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509489,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":679936,"data_stored":279606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509487,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1286144,"data_stored":872078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574896,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31848,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339992,"statfs":{"total":96636764160,"available":96604151808,"internally_reserved":0,"allocated":675840,"data_stored":279516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542192,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31856,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339984,"statfs":{"total":96636764160,"available":96604143616,"internally_reserved":0,"allocated":684032,"data_stored":279572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:41:55.731 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T08:41:55.732 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-19T08:41:55.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.898+0000 7f3cf0b2c640 1 Processor -- start 2024-09-19T08:41:55.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.898+0000 7f3cf0b2c640 1 -- start start 2024-09-19T08:41:55.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.898+0000 7f3cf0b2c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3cec0b71a0 0x7f3cec06d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:55.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.898+0000 7f3cf0b2c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3cec0b5640 0x7f3cec161840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:55.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.898+0000 7f3cf0b2c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3cec0b7580 0x7f3cec161d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:55.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.898+0000 7f3cf0b2c640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f3cec073270 con 0x7f3cec0b7580 2024-09-19T08:41:55.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.898+0000 7f3cf0b2c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f3cec073550 con 0x7f3cec0b5640 2024-09-19T08:41:55.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.898+0000 7f3cf0b2c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3cec073830 con 0x7f3cec0b71a0 2024-09-19T08:41:55.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.898+0000 7f3cead9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3cec0b7580 0x7f3cec161d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:55.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.898+0000 7f3cead9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3cec0b7580 0x7f3cec161d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47062/0 (socket says 172.21.15.98:47062) 2024-09-19T08:41:55.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.898+0000 7f3cead9d640 1 -- 172.21.15.98:0/2438100859 learned_addr learned my addr 172.21.15.98:0/2438100859 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:55.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.898+0000 7f3cea59c640 1 --2- 172.21.15.98:0/2438100859 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3cec0b71a0 0x7f3cec06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:55.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.898+0000 7f3ce9d9b640 1 --2- 172.21.15.98:0/2438100859 >> v2:172.21.15.139:3300/0 conn(0x7f3cec0b5640 0x7f3cec161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:55.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.898+0000 7f3cead9d640 1 -- 172.21.15.98:0/2438100859 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3cec0b71a0 msgr2=0x7f3cec06d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:55.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.898+0000 7f3cead9d640 1 --2- 172.21.15.98:0/2438100859 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3cec0b71a0 0x7f3cec06d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:55.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.898+0000 7f3cead9d640 1 -- 172.21.15.98:0/2438100859 >> v2:172.21.15.139:3300/0 conn(0x7f3cec0b5640 msgr2=0x7f3cec161840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:55.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.898+0000 7f3cead9d640 1 --2- 172.21.15.98:0/2438100859 >> v2:172.21.15.139:3300/0 conn(0x7f3cec0b5640 0x7f3cec161840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:55.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.898+0000 7f3cead9d640 1 -- 172.21.15.98:0/2438100859 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cec073b10 con 0x7f3cec0b7580 2024-09-19T08:41:55.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.898+0000 7f3cea59c640 1 --2- 172.21.15.98:0/2438100859 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3cec0b71a0 0x7f3cec06d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:55.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.902+0000 7f3cead9d640 1 --2- 172.21.15.98:0/2438100859 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cec0b7580 0x7f3cec161d80 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3cec072df0 tx=0x7f3ce401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c921b7b964c666ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:55.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.902+0000 7f3ce959a640 1 -- 172.21.15.98:0/2438100859 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce40a5690 con 0x7f3cec0b7580 2024-09-19T08:41:55.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.902+0000 7f3ce959a640 1 -- 172.21.15.98:0/2438100859 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ce40a5830 con 0x7f3cec0b7580 2024-09-19T08:41:55.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.902+0000 7f3ce959a640 1 -- 172.21.15.98:0/2438100859 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce40a5c00 con 0x7f3cec0b7580 2024-09-19T08:41:55.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.902+0000 7f3cf0b2c640 1 -- 172.21.15.98:0/2438100859 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cec0b7580 msgr2=0x7f3cec161d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:55.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.902+0000 7f3cf0b2c640 1 --2- 172.21.15.98:0/2438100859 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cec0b7580 0x7f3cec161d80 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3cec072df0 tx=0x7f3ce401c9f0 comp rx=0 tx=0).stop 2024-09-19T08:41:55.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.902+0000 7f3cf0b2c640 1 -- 172.21.15.98:0/2438100859 shutdown_connections 2024-09-19T08:41:55.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.902+0000 7f3cf0b2c640 1 --2- 172.21.15.98:0/2438100859 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cec0b7580 0x7f3cec161d80 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:55.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.902+0000 7f3cf0b2c640 1 --2- 172.21.15.98:0/2438100859 >> v2:172.21.15.139:3300/0 conn(0x7f3cec0b5640 0x7f3cec161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:55.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.902+0000 7f3cf0b2c640 1 --2- 172.21.15.98:0/2438100859 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3cec0b71a0 0x7f3cec06d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:55.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.902+0000 7f3cf0b2c640 1 -- 172.21.15.98:0/2438100859 >> 172.21.15.98:0/2438100859 conn(0x7f3cec0b4260 msgr2=0x7f3cec0b4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:55.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.902+0000 7f3cf0b2c640 1 -- 172.21.15.98:0/2438100859 shutdown_connections 2024-09-19T08:41:55.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.902+0000 7f3cf0b2c640 1 -- 172.21.15.98:0/2438100859 wait complete. 2024-09-19T08:41:55.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.902+0000 7f3cf0b2c640 1 Processor -- start 2024-09-19T08:41:55.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.902+0000 7f3cf0b2c640 1 -- start start 2024-09-19T08:41:55.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.906+0000 7f3cf0b2c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cec0b5640 0x7f3cec1438a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:55.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.906+0000 7f3cf0b2c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cec0b71a0 0x7f3cec15e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:55.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.906+0000 7f3cf0b2c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3cec0b7580 0x7f3cec160810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:55.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.906+0000 7f3cf0b2c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec0714c0 con 0x7f3cec0b71a0 2024-09-19T08:41:55.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.906+0000 7f3cf0b2c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec071340 con 0x7f3cec0b5640 2024-09-19T08:41:55.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.906+0000 7f3cf0b2c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3cec071640 con 0x7f3cec0b7580 2024-09-19T08:41:55.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.906+0000 7f3ce9d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cec0b71a0 0x7f3cec15e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:55.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.906+0000 7f3ce9d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cec0b71a0 0x7f3cec15e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47066/0 (socket says 172.21.15.98:47066) 2024-09-19T08:41:55.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.906+0000 7f3ce9d9b640 1 -- 172.21.15.98:0/3955377793 learned_addr learned my addr 172.21.15.98:0/3955377793 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:55.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.906+0000 7f3cead9d640 1 --2- 172.21.15.98:0/3955377793 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3cec0b7580 0x7f3cec160810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:55.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.906+0000 7f3cea59c640 1 --2- 172.21.15.98:0/3955377793 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cec0b5640 0x7f3cec1438a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:55.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.906+0000 7f3ce9d9b640 1 -- 172.21.15.98:0/3955377793 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3cec0b7580 msgr2=0x7f3cec160810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:55.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.906+0000 7f3ce9d9b640 1 --2- 172.21.15.98:0/3955377793 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3cec0b7580 0x7f3cec160810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:55.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.906+0000 7f3ce9d9b640 1 -- 172.21.15.98:0/3955377793 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cec0b5640 msgr2=0x7f3cec1438a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:55.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.906+0000 7f3ce9d9b640 1 --2- 172.21.15.98:0/3955377793 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cec0b5640 0x7f3cec1438a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:55.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.906+0000 7f3ce9d9b640 1 -- 172.21.15.98:0/3955377793 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cec06cd50 con 0x7f3cec0b71a0 2024-09-19T08:41:55.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.906+0000 7f3ce9d9b640 1 --2- 172.21.15.98:0/3955377793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cec0b71a0 0x7f3cec15e500 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc00c7f0 tx=0x7f3cdc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:55.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.906+0000 7f3ce2ffd640 1 -- 172.21.15.98:0/3955377793 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc004230 con 0x7f3cec0b71a0 2024-09-19T08:41:55.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.906+0000 7f3ce2ffd640 1 -- 172.21.15.98:0/3955377793 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cdc0043d0 con 0x7f3cec0b71a0 2024-09-19T08:41:55.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.906+0000 7f3cf0b2c640 1 -- 172.21.15.98:0/3955377793 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cec143de0 con 0x7f3cec0b71a0 2024-09-19T08:41:55.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.906+0000 7f3ce2ffd640 1 -- 172.21.15.98:0/3955377793 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc003860 con 0x7f3cec0b71a0 2024-09-19T08:41:55.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.906+0000 7f3cf0b2c640 1 -- 172.21.15.98:0/3955377793 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cec15ea40 con 0x7f3cec0b71a0 2024-09-19T08:41:55.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.906+0000 7f3cf0b2c640 1 -- 172.21.15.98:0/3955377793 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cec15cd10 con 0x7f3cec0b71a0 2024-09-19T08:41:55.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.906+0000 7f3ce2ffd640 1 -- 172.21.15.98:0/3955377793 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f3cdc003c10 con 0x7f3cec0b71a0 2024-09-19T08:41:55.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.906+0000 7f3ce2ffd640 1 --2- 172.21.15.98:0/3955377793 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3ccc067aa0 0x7f3ccc069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:55.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.906+0000 7f3ce2ffd640 1 -- 172.21.15.98:0/3955377793 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f3cdc086510 con 0x7f3cec0b71a0 2024-09-19T08:41:55.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.906+0000 7f3cea59c640 1 --2- 172.21.15.98:0/3955377793 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3ccc067aa0 0x7f3ccc069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:55.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.910+0000 7f3cea59c640 1 --2- 172.21.15.98:0/3955377793 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3ccc067aa0 0x7f3ccc069f60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3cec14c720 tx=0x7f3cd4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:55.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:55.910+0000 7f3ce2ffd640 1 -- 172.21.15.98:0/3955377793 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3cdc015020 con 0x7f3cec0b71a0 2024-09-19T08:41:56.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.138+0000 7f3cf0b2c640 1 -- 172.21.15.98:0/3955377793 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f3cec15ae10 con 0x7f3cec0b71a0 2024-09-19T08:41:56.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.142+0000 7f3ce2ffd640 1 -- 172.21.15.98:0/3955377793 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7f3cdc056e30 con 0x7f3cec0b71a0 2024-09-19T08:41:56.144 INFO:teuthology.orchestra.run.smithi098.stdout:[client.0] 2024-09-19T08:41:56.144 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQC/4utm0vNpIhAA4S1E8dFZ0+ibmSKhmZ+bIA== 2024-09-19T08:41:56.144 INFO:teuthology.orchestra.run.smithi098.stdout: caps mds = "allow" 2024-09-19T08:41:56.144 INFO:teuthology.orchestra.run.smithi098.stdout: caps mon = "allow r" 2024-09-19T08:41:56.145 INFO:teuthology.orchestra.run.smithi098.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-19T08:41:56.145 INFO:teuthology.orchestra.run.smithi098.stderr:updated caps for client.0 2024-09-19T08:41:56.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.146+0000 7f3cf0b2c640 1 -- 172.21.15.98:0/3955377793 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3ccc067aa0 msgr2=0x7f3ccc069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:56.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.146+0000 7f3cf0b2c640 1 --2- 172.21.15.98:0/3955377793 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3ccc067aa0 0x7f3ccc069f60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3cec14c720 tx=0x7f3cd4008040 comp rx=0 tx=0).stop 2024-09-19T08:41:56.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.146+0000 7f3cf0b2c640 1 -- 172.21.15.98:0/3955377793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cec0b71a0 msgr2=0x7f3cec15e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:56.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.146+0000 7f3cf0b2c640 1 --2- 172.21.15.98:0/3955377793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cec0b71a0 0x7f3cec15e500 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc00c7f0 tx=0x7f3cdc00ccb0 comp rx=0 tx=0).stop 2024-09-19T08:41:56.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.146+0000 7f3cf0b2c640 1 -- 172.21.15.98:0/3955377793 shutdown_connections 2024-09-19T08:41:56.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.146+0000 7f3cf0b2c640 1 --2- 172.21.15.98:0/3955377793 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3ccc067aa0 0x7f3ccc069f60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:56.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.146+0000 7f3cf0b2c640 1 --2- 172.21.15.98:0/3955377793 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3cec0b7580 0x7f3cec160810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:56.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.146+0000 7f3cf0b2c640 1 --2- 172.21.15.98:0/3955377793 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3cec0b71a0 0x7f3cec15e500 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:56.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.146+0000 7f3cf0b2c640 1 --2- 172.21.15.98:0/3955377793 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3cec0b5640 0x7f3cec1438a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:56.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.146+0000 7f3cf0b2c640 1 -- 172.21.15.98:0/3955377793 >> 172.21.15.98:0/3955377793 conn(0x7f3cec0b4260 msgr2=0x7f3cec161610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:56.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.146+0000 7f3cf0b2c640 1 -- 172.21.15.98:0/3955377793 shutdown_connections 2024-09-19T08:41:56.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.146+0000 7f3cf0b2c640 1 -- 172.21.15.98:0/3955377793 wait complete. 2024-09-19T08:41:56.164 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:41:56.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.274+0000 7feb65a22640 1 Processor -- start 2024-09-19T08:41:56.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.274+0000 7feb65a22640 1 -- start start 2024-09-19T08:41:56.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.274+0000 7feb65a22640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7feb6006c3a0 0x7feb600cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:56.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.274+0000 7feb65a22640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb600cf900 0x7feb600723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:56.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.274+0000 7feb65a22640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb6006d310 con 0x7feb6015c550 2024-09-19T08:41:56.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.274+0000 7feb65a22640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7feb6006d5f0 con 0x7feb6006c3a0 2024-09-19T08:41:56.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.274+0000 7feb65a22640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7feb6006d8d0 con 0x7feb600cf900 2024-09-19T08:41:56.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.274+0000 7feb5e7fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7feb6006c3a0 0x7feb600cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:56.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.274+0000 7feb5e7fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7feb6006c3a0 0x7feb600cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47080/0 (socket says 172.21.15.98:47080) 2024-09-19T08:41:56.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.274+0000 7feb5e7fc640 1 -- 172.21.15.98:0/2532284169 learned_addr learned my addr 172.21.15.98:0/2532284169 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:56.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.274+0000 7feb5f7fe640 1 --2- 172.21.15.98:0/2532284169 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb600cf900 0x7feb600723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:56.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.274+0000 7feb5e7fc640 1 -- 172.21.15.98:0/2532284169 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb600cf900 msgr2=0x7feb600723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:56.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.274+0000 7feb5e7fc640 1 --2- 172.21.15.98:0/2532284169 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb600cf900 0x7feb600723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:56.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.274+0000 7feb5e7fc640 1 -- 172.21.15.98:0/2532284169 >> v1:172.21.15.139:6789/0 conn(0x7feb6015c550 legacy=0x7feb6006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:56.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.274+0000 7feb5e7fc640 1 -- 172.21.15.98:0/2532284169 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6006dbb0 con 0x7feb6006c3a0 2024-09-19T08:41:56.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.274+0000 7feb5f7fe640 1 --2- 172.21.15.98:0/2532284169 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb600cf900 0x7feb600723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:56.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.278+0000 7feb5e7fc640 1 --2- 172.21.15.98:0/2532284169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb6006c3a0 0x7feb600cf3c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7feb48009870 tx=0x7feb4801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8248d016eb89fb2f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:56.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.278+0000 7feb5dffb640 1 -- 172.21.15.98:0/2532284169 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb480ab5f0 con 0x7feb6006c3a0 2024-09-19T08:41:56.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.278+0000 7feb5dffb640 1 -- 172.21.15.98:0/2532284169 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb480b0070 con 0x7feb6006c3a0 2024-09-19T08:41:56.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.278+0000 7feb5dffb640 1 -- 172.21.15.98:0/2532284169 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb480ab9b0 con 0x7feb6006c3a0 2024-09-19T08:41:56.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.278+0000 7feb65a22640 1 -- 172.21.15.98:0/2532284169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb6006c3a0 msgr2=0x7feb600cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:56.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.278+0000 7feb65a22640 1 --2- 172.21.15.98:0/2532284169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb6006c3a0 0x7feb600cf3c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7feb48009870 tx=0x7feb4801c520 comp rx=0 tx=0).stop 2024-09-19T08:41:56.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.278+0000 7feb65a22640 1 -- 172.21.15.98:0/2532284169 shutdown_connections 2024-09-19T08:41:56.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.278+0000 7feb65a22640 1 --2- 172.21.15.98:0/2532284169 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb600cf900 0x7feb600723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:56.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.278+0000 7feb65a22640 1 --2- 172.21.15.98:0/2532284169 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb6006c3a0 0x7feb600cf3c0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:56.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.278+0000 7feb65a22640 1 -- 172.21.15.98:0/2532284169 >> 172.21.15.98:0/2532284169 conn(0x7feb600b2010 msgr2=0x7feb600b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:56.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.278+0000 7feb65a22640 1 -- 172.21.15.98:0/2532284169 shutdown_connections 2024-09-19T08:41:56.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.278+0000 7feb65a22640 1 -- 172.21.15.98:0/2532284169 wait complete. 2024-09-19T08:41:56.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.278+0000 7feb65a22640 1 Processor -- start 2024-09-19T08:41:56.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.278+0000 7feb65a22640 1 -- start start 2024-09-19T08:41:56.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.278+0000 7feb65a22640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb6006c3a0 0x7feb6015a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:56.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.278+0000 7feb65a22640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb600cf900 0x7feb6015f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:56.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.278+0000 7feb65a22640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb6015c550 0x7feb600ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:56.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.278+0000 7feb65a22640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7feb60073500 con 0x7feb6015c550 2024-09-19T08:41:56.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.278+0000 7feb65a22640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feb60073380 con 0x7feb6006c3a0 2024-09-19T08:41:56.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.278+0000 7feb65a22640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7feb60073680 con 0x7feb600cf900 2024-09-19T08:41:56.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.278+0000 7feb5f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb6015c550 0x7feb600ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:56.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.278+0000 7feb5f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb6015c550 0x7feb600ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47096/0 (socket says 172.21.15.98:47096) 2024-09-19T08:41:56.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.278+0000 7feb5f7fe640 1 -- 172.21.15.98:0/252092506 learned_addr learned my addr 172.21.15.98:0/252092506 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:56.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.278+0000 7feb5e7fc640 1 --2- 172.21.15.98:0/252092506 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb600cf900 0x7feb6015f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:56.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.278+0000 7feb5effd640 1 --2- 172.21.15.98:0/252092506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb6006c3a0 0x7feb6015a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:56.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.278+0000 7feb5f7fe640 1 -- 172.21.15.98:0/252092506 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb600cf900 msgr2=0x7feb6015f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:56.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.278+0000 7feb5f7fe640 1 --2- 172.21.15.98:0/252092506 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb600cf900 0x7feb6015f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:56.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.278+0000 7feb5f7fe640 1 -- 172.21.15.98:0/252092506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb6006c3a0 msgr2=0x7feb6015a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:56.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.278+0000 7feb5f7fe640 1 --2- 172.21.15.98:0/252092506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb6006c3a0 0x7feb6015a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:56.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.278+0000 7feb5f7fe640 1 -- 172.21.15.98:0/252092506 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb60157760 con 0x7feb6015c550 2024-09-19T08:41:56.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.278+0000 7feb5e7fc640 1 --2- 172.21.15.98:0/252092506 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb600cf900 0x7feb6015f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:41:56.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.278+0000 7feb5f7fe640 1 --2- 172.21.15.98:0/252092506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb6015c550 0x7feb600ae8b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7feb5800cd50 tx=0x7feb5800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:56.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.282+0000 7feb57fff640 1 -- 172.21.15.98:0/252092506 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5800f680 con 0x7feb6015c550 2024-09-19T08:41:56.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.282+0000 7feb57fff640 1 -- 172.21.15.98:0/252092506 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb5800f840 con 0x7feb6015c550 2024-09-19T08:41:56.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.282+0000 7feb57fff640 1 -- 172.21.15.98:0/252092506 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5800fc30 con 0x7feb6015c550 2024-09-19T08:41:56.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.282+0000 7feb65a22640 1 -- 172.21.15.98:0/252092506 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb60157a40 con 0x7feb6015c550 2024-09-19T08:41:56.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.282+0000 7feb65a22640 1 -- 172.21.15.98:0/252092506 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb60158000 con 0x7feb6015c550 2024-09-19T08:41:56.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.282+0000 7feb65a22640 1 -- 172.21.15.98:0/252092506 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb600cf3c0 con 0x7feb6015c550 2024-09-19T08:41:56.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.282+0000 7feb57fff640 1 -- 172.21.15.98:0/252092506 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7feb5800e3e0 con 0x7feb6015c550 2024-09-19T08:41:56.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.282+0000 7feb57fff640 1 --2- 172.21.15.98:0/252092506 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7feb40067a70 0x7feb40069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:56.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.282+0000 7feb57fff640 1 -- 172.21.15.98:0/252092506 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7feb58086e80 con 0x7feb6015c550 2024-09-19T08:41:56.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.282+0000 7feb5effd640 1 --2- 172.21.15.98:0/252092506 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7feb40067a70 0x7feb40069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:56.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.282+0000 7feb5effd640 1 --2- 172.21.15.98:0/252092506 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7feb40067a70 0x7feb40069f30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7feb5000b810 tx=0x7feb50008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:56.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.286+0000 7feb57fff640 1 -- 172.21.15.98:0/252092506 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7feb58014030 con 0x7feb6015c550 2024-09-19T08:41:56.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.482+0000 7feb65a22640 1 -- 172.21.15.98:0/252092506 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb60151ba0 con 0x7feb6015c550 2024-09-19T08:41:56.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.482+0000 7feb57fff640 1 -- 172.21.15.98:0/252092506 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+5229 (secure 0 0 0) 0x7feb58057630 con 0x7feb6015c550 2024-09-19T08:41:56.483 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:41:56.483 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":30,"btime":"2024-09-19T08:41:49:966866+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4880,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/1549394645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":1549394645},{"type":"v1","addr":"172.21.15.139:6834","nonce":1549394645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":4883,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1847525681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6833","nonce":1847525681},{"type":"v1","addr":"172.21.15.139:6835","nonce":1847525681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5745,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/1677812726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1677812726},{"type":"v1","addr":"172.21.15.98:6835","nonce":1677812726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:41:47.304224+0000","modified":"2024-09-19T08:41:49.306408+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/3548234111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3548234111},{"type":"v1","addr":"172.21.15.98:6837","nonce":3548234111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5748,"qdb_cluster":[5748]},"id":4}]} 2024-09-19T08:41:56.484 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 30 2024-09-19T08:41:56.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.486+0000 7feb65a22640 1 -- 172.21.15.98:0/252092506 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7feb40067a70 msgr2=0x7feb40069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:56.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.486+0000 7feb65a22640 1 --2- 172.21.15.98:0/252092506 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7feb40067a70 0x7feb40069f30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7feb5000b810 tx=0x7feb50008040 comp rx=0 tx=0).stop 2024-09-19T08:41:56.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.486+0000 7feb65a22640 1 -- 172.21.15.98:0/252092506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb6015c550 msgr2=0x7feb600ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:56.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.486+0000 7feb65a22640 1 --2- 172.21.15.98:0/252092506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb6015c550 0x7feb600ae8b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7feb5800cd50 tx=0x7feb5800a650 comp rx=0 tx=0).stop 2024-09-19T08:41:56.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.486+0000 7feb65a22640 1 -- 172.21.15.98:0/252092506 shutdown_connections 2024-09-19T08:41:56.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.486+0000 7feb65a22640 1 --2- 172.21.15.98:0/252092506 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7feb40067a70 0x7feb40069f30 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:56.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.486+0000 7feb65a22640 1 --2- 172.21.15.98:0/252092506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb6015c550 0x7feb600ae8b0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:56.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.486+0000 7feb65a22640 1 --2- 172.21.15.98:0/252092506 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb600cf900 0x7feb6015f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:56.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.486+0000 7feb65a22640 1 --2- 172.21.15.98:0/252092506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb6006c3a0 0x7feb6015a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:56.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.486+0000 7feb65a22640 1 -- 172.21.15.98:0/252092506 >> 172.21.15.98:0/252092506 conn(0x7feb600b2010 msgr2=0x7feb60067a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:56.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.486+0000 7feb65a22640 1 -- 172.21.15.98:0/252092506 shutdown_connections 2024-09-19T08:41:56.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:56.486+0000 7feb65a22640 1 -- 172.21.15.98:0/252092506 wait complete. 2024-09-19T08:41:56.904 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T08:41:47.304224+0000', 'modified': '2024-09-19T08:41:49.306408+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5748}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5748': {'gid': 5748, 'name': 'a', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.98:6837/3548234111', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6836', 'nonce': 3548234111}, {'type': 'v1', 'addr': '172.21.15.98:6837', 'nonce': 3548234111}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5748, 'qdb_cluster': [5748]} 2024-09-19T08:41:56.905 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T08:41:56.905 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-19T08:41:57.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.038+0000 7f9549bbf640 1 Processor -- start 2024-09-19T08:41:57.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.038+0000 7f9549bbf640 1 -- start start 2024-09-19T08:41:57.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.038+0000 7f9549bbf640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9544158fd0 0x7f95440b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:57.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.038+0000 7f9549bbf640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f95440b4360 0x7f954406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:57.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.038+0000 7f9549bbf640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95440ccc60 con 0x7f954406c1d0 2024-09-19T08:41:57.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.038+0000 7f9549bbf640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f954406c640 con 0x7f9544158fd0 2024-09-19T08:41:57.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.038+0000 7f9549bbf640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f95441593b0 con 0x7f95440b4360 2024-09-19T08:41:57.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.038+0000 7f95437fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9544158fd0 0x7f95440b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:57.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.038+0000 7f95437fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9544158fd0 0x7f95440b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47108/0 (socket says 172.21.15.98:47108) 2024-09-19T08:41:57.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.038+0000 7f95437fe640 1 -- 172.21.15.98:0/801347957 learned_addr learned my addr 172.21.15.98:0/801347957 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:57.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.038+0000 7f9542ffd640 1 --2- 172.21.15.98:0/801347957 >> v2:172.21.15.139:3300/0 conn(0x7f95440b4360 0x7f954406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:57.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.038+0000 7f95437fe640 1 -- 172.21.15.98:0/801347957 >> v2:172.21.15.139:3300/0 conn(0x7f95440b4360 msgr2=0x7f954406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:57.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.038+0000 7f95437fe640 1 --2- 172.21.15.98:0/801347957 >> v2:172.21.15.139:3300/0 conn(0x7f95440b4360 0x7f954406bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.038+0000 7f95437fe640 1 -- 172.21.15.98:0/801347957 >> v1:172.21.15.139:6789/0 conn(0x7f954406c1d0 legacy=0x7f954415efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:57.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.038+0000 7f95437fe640 1 -- 172.21.15.98:0/801347957 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95440ccf40 con 0x7f9544158fd0 2024-09-19T08:41:57.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.042+0000 7f95437fe640 1 --2- 172.21.15.98:0/801347957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9544158fd0 0x7f95440b3e20 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f95440cdef0 tx=0x7f952c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c8d6d53c0074f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:57.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.042+0000 7f95427fc640 1 -- 172.21.15.98:0/801347957 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c0ab660 con 0x7f9544158fd0 2024-09-19T08:41:57.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.042+0000 7f95427fc640 1 -- 172.21.15.98:0/801347957 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f952c0b0070 con 0x7f9544158fd0 2024-09-19T08:41:57.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.042+0000 7f95427fc640 1 -- 172.21.15.98:0/801347957 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c0aba20 con 0x7f9544158fd0 2024-09-19T08:41:57.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.042+0000 7f9549bbf640 1 -- 172.21.15.98:0/801347957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9544158fd0 msgr2=0x7f95440b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:57.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.042+0000 7f9549bbf640 1 --2- 172.21.15.98:0/801347957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9544158fd0 0x7f95440b3e20 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f95440cdef0 tx=0x7f952c01c460 comp rx=0 tx=0).stop 2024-09-19T08:41:57.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.042+0000 7f9549bbf640 1 -- 172.21.15.98:0/801347957 shutdown_connections 2024-09-19T08:41:57.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.042+0000 7f9549bbf640 1 --2- 172.21.15.98:0/801347957 >> v2:172.21.15.139:3300/0 conn(0x7f95440b4360 0x7f954406bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.042+0000 7f9549bbf640 1 --2- 172.21.15.98:0/801347957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9544158fd0 0x7f95440b3e20 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.042+0000 7f9549bbf640 1 -- 172.21.15.98:0/801347957 >> 172.21.15.98:0/801347957 conn(0x7f95440b2010 msgr2=0x7f95440b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:57.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.042+0000 7f9549bbf640 1 -- 172.21.15.98:0/801347957 shutdown_connections 2024-09-19T08:41:57.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.042+0000 7f9549bbf640 1 -- 172.21.15.98:0/801347957 wait complete. 2024-09-19T08:41:57.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.042+0000 7f9549bbf640 1 Processor -- start 2024-09-19T08:41:57.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.042+0000 7f9549bbf640 1 -- start start 2024-09-19T08:41:57.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.046+0000 7f9549bbf640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f954406c1d0 0x7f9544153790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:57.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.046+0000 7f9549bbf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95440b4360 0x7f95441655e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:57.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.046+0000 7f9549bbf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9544158fd0 0x7f954414dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:57.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.046+0000 7f9549bbf640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f954415c920 con 0x7f95440b4360 2024-09-19T08:41:57.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.046+0000 7f9549bbf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f954415c7a0 con 0x7f9544158fd0 2024-09-19T08:41:57.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.046+0000 7f9549bbf640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9544168110 con 0x7f954406c1d0 2024-09-19T08:41:57.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.046+0000 7f9542ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95440b4360 0x7f95441655e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:57.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.046+0000 7f9542ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95440b4360 0x7f95441655e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47120/0 (socket says 172.21.15.98:47120) 2024-09-19T08:41:57.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.046+0000 7f9542ffd640 1 -- 172.21.15.98:0/1030249957 learned_addr learned my addr 172.21.15.98:0/1030249957 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:57.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.046+0000 7f9543fff640 1 --2- 172.21.15.98:0/1030249957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9544158fd0 0x7f954414dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:57.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.046+0000 7f95437fe640 1 --2- 172.21.15.98:0/1030249957 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f954406c1d0 0x7f9544153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:57.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.046+0000 7f9542ffd640 1 -- 172.21.15.98:0/1030249957 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f954406c1d0 msgr2=0x7f9544153790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:57.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.046+0000 7f9542ffd640 1 --2- 172.21.15.98:0/1030249957 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f954406c1d0 0x7f9544153790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.046+0000 7f9542ffd640 1 -- 172.21.15.98:0/1030249957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9544158fd0 msgr2=0x7f954414dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:57.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.046+0000 7f9542ffd640 1 --2- 172.21.15.98:0/1030249957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9544158fd0 0x7f954414dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.046+0000 7f9542ffd640 1 -- 172.21.15.98:0/1030249957 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9544151100 con 0x7f95440b4360 2024-09-19T08:41:57.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.046+0000 7f9543fff640 1 --2- 172.21.15.98:0/1030249957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9544158fd0 0x7f954414dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:41:57.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.046+0000 7f9542ffd640 1 --2- 172.21.15.98:0/1030249957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95440b4360 0x7f95441655e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f953400d830 tx=0x7f953400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:57.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.046+0000 7f953bfff640 1 -- 172.21.15.98:0/1030249957 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953400b6c0 con 0x7f95440b4360 2024-09-19T08:41:57.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.046+0000 7f953bfff640 1 -- 172.21.15.98:0/1030249957 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9534019070 con 0x7f95440b4360 2024-09-19T08:41:57.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.046+0000 7f9549bbf640 1 -- 172.21.15.98:0/1030249957 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9544150340 con 0x7f95440b4360 2024-09-19T08:41:57.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.046+0000 7f953bfff640 1 -- 172.21.15.98:0/1030249957 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953400bae0 con 0x7f95440b4360 2024-09-19T08:41:57.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.046+0000 7f9549bbf640 1 -- 172.21.15.98:0/1030249957 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9544150900 con 0x7f95440b4360 2024-09-19T08:41:57.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.046+0000 7f953bfff640 1 -- 172.21.15.98:0/1030249957 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f953401e460 con 0x7f95440b4360 2024-09-19T08:41:57.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.046+0000 7f953bfff640 1 --2- 172.21.15.98:0/1030249957 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9524067ac0 0x7f9524069f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:57.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f95437fe640 1 --2- 172.21.15.98:0/1030249957 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9524067ac0 0x7f9524069f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:57.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f953bfff640 1 -- 172.21.15.98:0/1030249957 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f95340866d0 con 0x7f95440b4360 2024-09-19T08:41:57.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f95437fe640 1 --2- 172.21.15.98:0/1030249957 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9524067ac0 0x7f9524069f80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f952c0af000 tx=0x7f952c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:57.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f954cc7b000 1 Processor -- start 2024-09-19T08:41:57.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f954cc7b000 1 -- start start 2024-09-19T08:41:57.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f954cc7b000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x560d9bcf5b20 0x560d9bcf5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:57.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f954cc7b000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d9bcf6440 0x560d9bcabd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:57.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f954cc7b000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d9bcac290 0x560d9bcac670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:57.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f954cc7b000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560d9bb93b60 con 0x560d9bcf6440 2024-09-19T08:41:57.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f954cc7b000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560d9bafac70 con 0x560d9bcac290 2024-09-19T08:41:57.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f954cc7b000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x560d9baa0ed0 con 0x560d9bcf5b20 2024-09-19T08:41:57.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f9542ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d9bcf6440 0x560d9bcabd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:57.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f9542ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d9bcf6440 0x560d9bcabd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47126/0 (socket says 172.21.15.98:47126) 2024-09-19T08:41:57.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f9542ffd640 1 -- 172.21.15.98:0/2530656632 learned_addr learned my addr 172.21.15.98:0/2530656632 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:57.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f95437fe640 1 --2- 172.21.15.98:0/2530656632 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x560d9bcf5b20 0x560d9bcf5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:57.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f9542ffd640 1 -- 172.21.15.98:0/2530656632 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x560d9bcf5b20 msgr2=0x560d9bcf5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:57.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f9543fff640 1 --2- 172.21.15.98:0/2530656632 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d9bcac290 0x560d9bcac670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:57.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f9542ffd640 1 --2- 172.21.15.98:0/2530656632 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x560d9bcf5b20 0x560d9bcf5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f9542ffd640 1 -- 172.21.15.98:0/2530656632 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d9bcac290 msgr2=0x560d9bcac670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:57.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f9542ffd640 1 --2- 172.21.15.98:0/2530656632 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d9bcac290 0x560d9bcac670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f9542ffd640 1 -- 172.21.15.98:0/2530656632 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d9bab2e50 con 0x560d9bcf6440 2024-09-19T08:41:57.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f95437fe640 1 --2- 172.21.15.98:0/2530656632 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x560d9bcf5b20 0x560d9bcf5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:57.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f9542ffd640 1 --2- 172.21.15.98:0/2530656632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d9bcf6440 0x560d9bcabd50 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f9534052d50 tx=0x7f95340570b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd92e39c76f1b062 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:57.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f95397fa640 1 -- 172.21.15.98:0/2530656632 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953405f4d0 con 0x560d9bcf6440 2024-09-19T08:41:57.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f95397fa640 1 -- 172.21.15.98:0/2530656632 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f953405d070 con 0x560d9bcf6440 2024-09-19T08:41:57.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f95397fa640 1 -- 172.21.15.98:0/2530656632 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9534084bc0 con 0x560d9bcf6440 2024-09-19T08:41:57.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f954cc7b000 1 -- 172.21.15.98:0/2530656632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d9bcf6440 msgr2=0x560d9bcabd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:57.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f954cc7b000 1 --2- 172.21.15.98:0/2530656632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d9bcf6440 0x560d9bcabd50 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f9534052d50 tx=0x7f95340570b0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f954cc7b000 1 -- 172.21.15.98:0/2530656632 shutdown_connections 2024-09-19T08:41:57.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f954cc7b000 1 --2- 172.21.15.98:0/2530656632 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d9bcac290 0x560d9bcac670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f954cc7b000 1 --2- 172.21.15.98:0/2530656632 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d9bcf6440 0x560d9bcabd50 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f954cc7b000 1 --2- 172.21.15.98:0/2530656632 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x560d9bcf5b20 0x560d9bcf5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f954cc7b000 1 -- 172.21.15.98:0/2530656632 >> 172.21.15.98:0/2530656632 conn(0x560d9bcb2600 msgr2=0x560d9bcaa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:57.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f954cc7b000 1 -- 172.21.15.98:0/2530656632 shutdown_connections 2024-09-19T08:41:57.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f954cc7b000 1 -- 172.21.15.98:0/2530656632 wait complete. 2024-09-19T08:41:57.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f954cc7b000 1 -- start start 2024-09-19T08:41:57.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.050+0000 7f954cc7b000 1 Processor -- start 2024-09-19T08:41:57.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f954cc7b000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x560d9bcf5b20 0x560d9bcb0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:57.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f954cc7b000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d9bcf6440 0x560d9bcb6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:57.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f954cc7b000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d9bcb1390 0x560d9bcb7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:57.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f954cc7b000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560d9bb5cb00 con 0x560d9bcb1390 2024-09-19T08:41:57.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f954cc7b000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560d9bad5360 con 0x560d9bcf6440 2024-09-19T08:41:57.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f954cc7b000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x560d9bad22d0 con 0x560d9bcf5b20 2024-09-19T08:41:57.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f954cc7b000 10 client.0 _ll_register_callbacks cb 0x560d9bc97820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:41:57.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f9543fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d9bcb1390 0x560d9bcb7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:57.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f9543fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d9bcb1390 0x560d9bcb7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47134/0 (socket says 172.21.15.98:47134) 2024-09-19T08:41:57.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f9543fff640 1 -- 172.21.15.98:0/3169039666 learned_addr learned my addr 172.21.15.98:0/3169039666 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:57.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f95437fe640 1 --2- 172.21.15.98:0/3169039666 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x560d9bcf5b20 0x560d9bcb0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:57.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f9542ffd640 1 --2- 172.21.15.98:0/3169039666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d9bcf6440 0x560d9bcb6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:57.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f9543fff640 1 -- 172.21.15.98:0/3169039666 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x560d9bcf5b20 msgr2=0x560d9bcb0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:57.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f9543fff640 1 --2- 172.21.15.98:0/3169039666 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x560d9bcf5b20 0x560d9bcb0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f9543fff640 1 -- 172.21.15.98:0/3169039666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d9bcf6440 msgr2=0x560d9bcb6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:57.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f9543fff640 1 --2- 172.21.15.98:0/3169039666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d9bcf6440 0x560d9bcb6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f9543fff640 1 -- 172.21.15.98:0/3169039666 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d9baa1120 con 0x560d9bcb1390 2024-09-19T08:41:57.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f95437fe640 1 --2- 172.21.15.98:0/3169039666 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x560d9bcf5b20 0x560d9bcb0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:41:57.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f9543fff640 1 --2- 172.21.15.98:0/3169039666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d9bcb1390 0x560d9bcb7230 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f953c00f4f0 tx=0x7f953c00f9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=516da91ea73ed870 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:57.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f95227fc640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-19T08:41:57.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f95227fc640 1 -- 172.21.15.98:0/3169039666 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c017020 con 0x560d9bcb1390 2024-09-19T08:41:57.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f95227fc640 1 -- 172.21.15.98:0/3169039666 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f953c003b90 con 0x560d9bcb1390 2024-09-19T08:41:57.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f954cc7b000 1 -- 172.21.15.98:0/3169039666 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f953405d070 con 0x560d9bcb1390 2024-09-19T08:41:57.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f95227fc640 1 -- 172.21.15.98:0/3169039666 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f953c021050 con 0x560d9bcb1390 2024-09-19T08:41:57.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f954cc7b000 10 client.5910 fetch_fsmap learned FSMap version 30 2024-09-19T08:41:57.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f954cc7b000 1 -- 172.21.15.98:0/3169039666 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) -- 0x560d9ba9a450 con 0x560d9bcb1390 2024-09-19T08:41:57.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f95227fc640 1 -- 172.21.15.98:0/3169039666 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f953405d070 con 0x560d9bcb1390 2024-09-19T08:41:57.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f95227fc640 1 -- 172.21.15.98:0/3169039666 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) ==== 3502+0+0 (secure 0 0 0) 0x7f953c024020 con 0x560d9bcb1390 2024-09-19T08:41:57.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f954cc7b000 10 client.5910 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:41:57.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f954cc7b000 10 client.5910 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T08:41:57.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f954cc7b000 20 client.5910 populate_metadata read hostname 'smithi098' 2024-09-19T08:41:57.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f954cc7b000 1 --2- 172.21.15.98:0/3169039666 >> [v2:172.21.15.98:6836/3548234111,v1:172.21.15.98:6837/3548234111] conn(0x560d9bd04280 0x560d9bd06670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:57.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f954cc7b000 4 client.5910 mds_command: new command op to 5748 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:41:57.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f954cc7b000 1 -- 172.21.15.98:0/3169039666 --> [v2:172.21.15.98:6836/3548234111,v1:172.21.15.98:6837/3548234111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d9ba94c00 con 0x560d9bd04280 2024-09-19T08:41:57.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f95437fe640 1 --2- 172.21.15.98:0/3169039666 >> [v2:172.21.15.98:6836/3548234111,v1:172.21.15.98:6837/3548234111] conn(0x560d9bd04280 0x560d9bd06670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:57.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f95437fe640 1 --2- 172.21.15.98:0/3169039666 >> [v2:172.21.15.98:6836/3548234111,v1:172.21.15.98:6837/3548234111] conn(0x560d9bd04280 0x560d9bd06670 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f534d8b06c2cc22d server_cookie=323fae769cafc898 in_seq=0 out_seq=0 2024-09-19T08:41:57.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f95227fc640 10 client.5910 ms_handle_connect on v2:172.21.15.98:6836/3548234111 2024-09-19T08:41:57.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f95227fc640 1 -- 172.21.15.98:0/3169039666 <== mds.0 v2:172.21.15.98:6836/3548234111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560d9ba94c00 con 0x560d9bd04280 2024-09-19T08:41:57.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f95227fc640 10 client.5910 handle_command_reply: tid=0 2024-09-19T08:41:57.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f954cc7b000 1 client.5910 shutdown 2024-09-19T08:41:57.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f954cc7b000 1 -- 172.21.15.98:0/3169039666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d9bcb1390 msgr2=0x560d9bcb7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:57.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f954cc7b000 1 --2- 172.21.15.98:0/3169039666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d9bcb1390 0x560d9bcb7230 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f953c00f4f0 tx=0x7f953c00f9b0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f954cc7b000 1 -- 172.21.15.98:0/3169039666 shutdown_connections 2024-09-19T08:41:57.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f954cc7b000 1 --2- 172.21.15.98:0/3169039666 >> [v2:172.21.15.98:6836/3548234111,v1:172.21.15.98:6837/3548234111] conn(0x560d9bd04280 0x560d9bd06670 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f954cc7b000 1 --2- 172.21.15.98:0/3169039666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d9bcb1390 0x560d9bcb7230 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f954cc7b000 1 --2- 172.21.15.98:0/3169039666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d9bcf6440 0x560d9bcb6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f95227fc640 0 client.5910 ms_handle_reset on v2:172.21.15.98:6836/3548234111 2024-09-19T08:41:57.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f954cc7b000 1 --2- 172.21.15.98:0/3169039666 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x560d9bcf5b20 0x560d9bcb0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f954cc7b000 1 -- 172.21.15.98:0/3169039666 >> 172.21.15.98:0/3169039666 conn(0x560d9bcac290 msgr2=0x560d9bcaad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:57.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f954cc7b000 1 -- 172.21.15.98:0/3169039666 shutdown_connections 2024-09-19T08:41:57.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f954cc7b000 1 -- 172.21.15.98:0/3169039666 wait complete. 2024-09-19T08:41:57.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.054+0000 7f954cc7b000 20 client.5910 trim_cache size 0 max 16384 2024-09-19T08:41:57.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.074+0000 7f954cc7b000 1 Processor -- start 2024-09-19T08:41:57.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.074+0000 7f954cc7b000 1 -- start start 2024-09-19T08:41:57.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.074+0000 7f954cc7b000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x560d9bcb1390 0x560d9bcb0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:57.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.074+0000 7f954cc7b000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d9bcf5b20 0x560d9bcb0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:57.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.074+0000 7f954cc7b000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d9bcf6440 0x560d9bd06670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:57.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.074+0000 7f954cc7b000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f954415a8e0 con 0x560d9bcf6440 2024-09-19T08:41:57.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.074+0000 7f954cc7b000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9544152a20 con 0x560d9bcf5b20 2024-09-19T08:41:57.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.074+0000 7f954cc7b000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9544072000 con 0x560d9bcb1390 2024-09-19T08:41:57.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.074+0000 7f9543fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d9bcf6440 0x560d9bd06670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:57.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.074+0000 7f9543fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d9bcf6440 0x560d9bd06670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47140/0 (socket says 172.21.15.98:47140) 2024-09-19T08:41:57.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.074+0000 7f9543fff640 1 -- 172.21.15.98:0/3457351005 learned_addr learned my addr 172.21.15.98:0/3457351005 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:57.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.074+0000 7f9543fff640 1 -- 172.21.15.98:0/3457351005 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x560d9bcb1390 msgr2=0x560d9bcb0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:57.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.074+0000 7f95437fe640 1 --2- 172.21.15.98:0/3457351005 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x560d9bcb1390 0x560d9bcb0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:57.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.074+0000 7f9543fff640 1 --2- 172.21.15.98:0/3457351005 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x560d9bcb1390 0x560d9bcb0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.074+0000 7f9543fff640 1 -- 172.21.15.98:0/3457351005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d9bcf5b20 msgr2=0x560d9bcb0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:57.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.074+0000 7f9542ffd640 1 --2- 172.21.15.98:0/3457351005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d9bcf5b20 0x560d9bcb0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:57.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.074+0000 7f9543fff640 1 --2- 172.21.15.98:0/3457351005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d9bcf5b20 0x560d9bcb0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.074+0000 7f9543fff640 1 -- 172.21.15.98:0/3457351005 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d9bd018e0 con 0x560d9bcf6440 2024-09-19T08:41:57.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.074+0000 7f9543fff640 1 --2- 172.21.15.98:0/3457351005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d9bcf6440 0x560d9bd06670 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f953c004fd0 tx=0x7f953c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9219d4e8736ce61d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:57.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.074+0000 7f9538ff9640 1 -- 172.21.15.98:0/3457351005 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c017030 con 0x560d9bcf6440 2024-09-19T08:41:57.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.074+0000 7f9538ff9640 1 -- 172.21.15.98:0/3457351005 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f953c006b60 con 0x560d9bcf6440 2024-09-19T08:41:57.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.074+0000 7f9538ff9640 1 -- 172.21.15.98:0/3457351005 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c005330 con 0x560d9bcf6440 2024-09-19T08:41:57.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.074+0000 7f954cc7b000 1 -- 172.21.15.98:0/3457351005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d9bcf6440 msgr2=0x560d9bd06670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:57.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.074+0000 7f954cc7b000 1 --2- 172.21.15.98:0/3457351005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d9bcf6440 0x560d9bd06670 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f953c004fd0 tx=0x7f953c013000 comp rx=0 tx=0).stop 2024-09-19T08:41:57.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.074+0000 7f954cc7b000 1 -- 172.21.15.98:0/3457351005 shutdown_connections 2024-09-19T08:41:57.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.074+0000 7f954cc7b000 1 --2- 172.21.15.98:0/3457351005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d9bcf6440 0x560d9bd06670 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.074+0000 7f954cc7b000 1 --2- 172.21.15.98:0/3457351005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d9bcf5b20 0x560d9bcb0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.074+0000 7f954cc7b000 1 --2- 172.21.15.98:0/3457351005 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x560d9bcb1390 0x560d9bcb0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.074+0000 7f954cc7b000 1 -- 172.21.15.98:0/3457351005 >> 172.21.15.98:0/3457351005 conn(0x560d9bcac290 msgr2=0x560d9bd098c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:57.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.074+0000 7f954cc7b000 1 -- 172.21.15.98:0/3457351005 shutdown_connections 2024-09-19T08:41:57.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.074+0000 7f954cc7b000 1 -- 172.21.15.98:0/3457351005 wait complete. 2024-09-19T08:41:57.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.074+0000 7f954cc7b000 1 -- start start 2024-09-19T08:41:57.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.074+0000 7f954cc7b000 1 Processor -- start 2024-09-19T08:41:57.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f954cc7b000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d9bcb1390 0x560d9bd08b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:57.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f954cc7b000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x560d9bcf5b20 0x560d9bd090d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:57.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f954cc7b000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d9bcf6440 0x560d9bd50e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:57.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f954cc7b000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x560d9baf9f80 con 0x560d9bcf6440 2024-09-19T08:41:57.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f954cc7b000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x560d9bab2ca0 con 0x560d9bcb1390 2024-09-19T08:41:57.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f954cc7b000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x560d9baff2a0 con 0x560d9bcf5b20 2024-09-19T08:41:57.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f954cc7b000 10 client.0 _ll_register_callbacks cb 0x7f953c001700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:41:57.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f9543fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d9bcf6440 0x560d9bd50e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:57.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f9543fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d9bcf6440 0x560d9bd50e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47146/0 (socket says 172.21.15.98:47146) 2024-09-19T08:41:57.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f9543fff640 1 -- 172.21.15.98:0/4210816956 learned_addr learned my addr 172.21.15.98:0/4210816956 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:57.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f9542ffd640 1 --2- 172.21.15.98:0/4210816956 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x560d9bcf5b20 0x560d9bd090d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:57.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f95437fe640 1 --2- 172.21.15.98:0/4210816956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d9bcb1390 0x560d9bd08b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:57.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f9543fff640 1 -- 172.21.15.98:0/4210816956 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x560d9bcf5b20 msgr2=0x560d9bd090d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:57.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f9543fff640 1 --2- 172.21.15.98:0/4210816956 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x560d9bcf5b20 0x560d9bd090d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f9543fff640 1 -- 172.21.15.98:0/4210816956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d9bcb1390 msgr2=0x560d9bd08b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:57.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f9543fff640 1 --2- 172.21.15.98:0/4210816956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d9bcb1390 0x560d9bd08b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f9543fff640 1 -- 172.21.15.98:0/4210816956 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d9bcf6270 con 0x560d9bcf6440 2024-09-19T08:41:57.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f9542ffd640 1 --2- 172.21.15.98:0/4210816956 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x560d9bcf5b20 0x560d9bd090d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:57.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f9543fff640 1 --2- 172.21.15.98:0/4210816956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d9bcf6440 0x560d9bd50e30 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f953c027040 tx=0x7f953c001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd7f4f018c4a0bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:57.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f95227fc640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-19T08:41:57.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f95227fc640 1 -- 172.21.15.98:0/4210816956 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c005680 con 0x560d9bcf6440 2024-09-19T08:41:57.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f95227fc640 1 -- 172.21.15.98:0/4210816956 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f953c02e480 con 0x560d9bcf6440 2024-09-19T08:41:57.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f954cc7b000 1 -- 172.21.15.98:0/4210816956 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f953c006b60 con 0x560d9bcf6440 2024-09-19T08:41:57.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f95227fc640 1 -- 172.21.15.98:0/4210816956 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f953c039050 con 0x560d9bcf6440 2024-09-19T08:41:57.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f954cc7b000 10 client.5916 fetch_fsmap learned FSMap version 30 2024-09-19T08:41:57.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f954cc7b000 1 -- 172.21.15.98:0/4210816956 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) -- 0x560d9ba81fd0 con 0x560d9bcf6440 2024-09-19T08:41:57.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f95227fc640 1 -- 172.21.15.98:0/4210816956 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f953c006b60 con 0x560d9bcf6440 2024-09-19T08:41:57.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f95227fc640 1 -- 172.21.15.98:0/4210816956 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 30) ==== 3502+0+0 (secure 0 0 0) 0x7f953c024070 con 0x560d9bcf6440 2024-09-19T08:41:57.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f954cc7b000 10 client.5916 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T08:41:57.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f954cc7b000 10 client.5916 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T08:41:57.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f954cc7b000 20 client.5916 populate_metadata read hostname 'smithi098' 2024-09-19T08:41:57.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f954cc7b000 1 --2- 172.21.15.98:0/4210816956 >> [v2:172.21.15.98:6836/3548234111,v1:172.21.15.98:6837/3548234111] conn(0x560d9bd517c0 0x560d9bd691c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:57.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f954cc7b000 4 client.5916 mds_command: new command op to 5748 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T08:41:57.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f954cc7b000 1 -- 172.21.15.98:0/4210816956 --> [v2:172.21.15.98:6836/3548234111,v1:172.21.15.98:6837/3548234111] -- command(tid 0: {"prefix": "status"}) -- 0x560d9bb083a0 con 0x560d9bd517c0 2024-09-19T08:41:57.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f95437fe640 1 --2- 172.21.15.98:0/4210816956 >> [v2:172.21.15.98:6836/3548234111,v1:172.21.15.98:6837/3548234111] conn(0x560d9bd517c0 0x560d9bd691c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:57.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f95437fe640 1 --2- 172.21.15.98:0/4210816956 >> [v2:172.21.15.98:6836/3548234111,v1:172.21.15.98:6837/3548234111] conn(0x560d9bd517c0 0x560d9bd691c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a139774e02ac191d server_cookie=4e978b0943f522d0 in_seq=0 out_seq=0 2024-09-19T08:41:57.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f95227fc640 10 client.5916 ms_handle_connect on v2:172.21.15.98:6836/3548234111 2024-09-19T08:41:57.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f95227fc640 1 -- 172.21.15.98:0/4210816956 <== mds.0 v2:172.21.15.98:6836/3548234111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x560d9bb083a0 con 0x560d9bd517c0 2024-09-19T08:41:57.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f95227fc640 10 client.5916 handle_command_reply: tid=0 2024-09-19T08:41:57.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f954cc7b000 1 client.5916 shutdown 2024-09-19T08:41:57.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f954cc7b000 1 -- 172.21.15.98:0/4210816956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d9bcf6440 msgr2=0x560d9bd50e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:57.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f954cc7b000 1 --2- 172.21.15.98:0/4210816956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d9bcf6440 0x560d9bd50e30 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f953c027040 tx=0x7f953c001260 comp rx=0 tx=0).stop 2024-09-19T08:41:57.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f954cc7b000 1 -- 172.21.15.98:0/4210816956 shutdown_connections 2024-09-19T08:41:57.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f954cc7b000 1 --2- 172.21.15.98:0/4210816956 >> [v2:172.21.15.98:6836/3548234111,v1:172.21.15.98:6837/3548234111] conn(0x560d9bd517c0 0x560d9bd691c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f954cc7b000 1 --2- 172.21.15.98:0/4210816956 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x560d9bcf6440 0x560d9bd50e30 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f954cc7b000 1 --2- 172.21.15.98:0/4210816956 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x560d9bcf5b20 0x560d9bd090d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f954cc7b000 1 --2- 172.21.15.98:0/4210816956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x560d9bcb1390 0x560d9bd08b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f95227fc640 0 client.5916 ms_handle_reset on v2:172.21.15.98:6836/3548234111 2024-09-19T08:41:57.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f954cc7b000 1 -- 172.21.15.98:0/4210816956 >> 172.21.15.98:0/4210816956 conn(0x560d9bcac290 msgr2=0x560d9bcb0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:57.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f954cc7b000 1 -- 172.21.15.98:0/4210816956 shutdown_connections 2024-09-19T08:41:57.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f954cc7b000 1 -- 172.21.15.98:0/4210816956 wait complete. 2024-09-19T08:41:57.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f954cc7b000 20 client.5916 trim_cache size 0 max 16384 2024-09-19T08:41:57.087 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:41:57.088 INFO:teuthology.orchestra.run.smithi098.stdout: "cluster_fsid": "552bf0ee-1f4f-4644-ba7c-d0931e55a451", 2024-09-19T08:41:57.088 INFO:teuthology.orchestra.run.smithi098.stdout: "whoami": 0, 2024-09-19T08:41:57.088 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 5748, 2024-09-19T08:41:57.088 INFO:teuthology.orchestra.run.smithi098.stdout: "want_state": "up:active", 2024-09-19T08:41:57.088 INFO:teuthology.orchestra.run.smithi098.stdout: "state": "up:active", 2024-09-19T08:41:57.088 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "cephfs", 2024-09-19T08:41:57.088 INFO:teuthology.orchestra.run.smithi098.stdout: "rank_uptime": 9.1196606419999995, 2024-09-19T08:41:57.088 INFO:teuthology.orchestra.run.smithi098.stdout: "mdsmap_epoch": 30, 2024-09-19T08:41:57.088 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch": 57, 2024-09-19T08:41:57.088 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch_barrier": 57, 2024-09-19T08:41:57.088 INFO:teuthology.orchestra.run.smithi098.stdout: "uptime": 14.465622615999999 2024-09-19T08:41:57.088 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:41:57.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f9549bbf640 1 -- 172.21.15.98:0/1030249957 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9524067ac0 msgr2=0x7f9524069f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:57.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f9549bbf640 1 --2- 172.21.15.98:0/1030249957 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9524067ac0 0x7f9524069f80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f952c0af000 tx=0x7f952c0a7000 comp rx=0 tx=0).stop 2024-09-19T08:41:57.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f9549bbf640 1 -- 172.21.15.98:0/1030249957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95440b4360 msgr2=0x7f95441655e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:57.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f9549bbf640 1 --2- 172.21.15.98:0/1030249957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95440b4360 0x7f95441655e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f953400d830 tx=0x7f953400dcf0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f9549bbf640 1 -- 172.21.15.98:0/1030249957 shutdown_connections 2024-09-19T08:41:57.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f9549bbf640 1 --2- 172.21.15.98:0/1030249957 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9524067ac0 0x7f9524069f80 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f9549bbf640 1 --2- 172.21.15.98:0/1030249957 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9544158fd0 0x7f954414dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f9549bbf640 1 --2- 172.21.15.98:0/1030249957 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f95440b4360 0x7f95441655e0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f9549bbf640 1 --2- 172.21.15.98:0/1030249957 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f954406c1d0 0x7f9544153790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f9549bbf640 1 -- 172.21.15.98:0/1030249957 >> 172.21.15.98:0/1030249957 conn(0x7f95440b2010 msgr2=0x7f95440b3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:57.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f9549bbf640 1 -- 172.21.15.98:0/1030249957 shutdown_connections 2024-09-19T08:41:57.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.078+0000 7f9549bbf640 1 -- 172.21.15.98:0/1030249957 wait complete. 2024-09-19T08:41:57.095 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T08:41:57.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.198+0000 7f8f0acd3640 1 Processor -- start 2024-09-19T08:41:57.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.198+0000 7f8f0acd3640 1 -- start start 2024-09-19T08:41:57.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.198+0000 7f8f0acd3640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8f0415bf30 0x7f8f04073170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:57.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.198+0000 7f8f0acd3640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f0406f960 0x7f8f0415ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:57.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.198+0000 7f8f0acd3640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f0415c310 con 0x7f8f0406d4c0 2024-09-19T08:41:57.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.198+0000 7f8f0acd3640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f8f04162e90 con 0x7f8f0415bf30 2024-09-19T08:41:57.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.198+0000 7f8f0acd3640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8f04162310 con 0x7f8f0406f960 2024-09-19T08:41:57.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.198+0000 7f8f08a6f640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8f0415bf30 0x7f8f04073170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:57.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.198+0000 7f8f08a6f640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8f0415bf30 0x7f8f04073170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47156/0 (socket says 172.21.15.98:47156) 2024-09-19T08:41:57.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.198+0000 7f8f08a6f640 1 -- 172.21.15.98:0/4243565410 learned_addr learned my addr 172.21.15.98:0/4243565410 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:57.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.198+0000 7f8f08a6f640 1 -- 172.21.15.98:0/4243565410 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f0406f960 msgr2=0x7f8f0415ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:57.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.198+0000 7f8f09270640 1 --2- 172.21.15.98:0/4243565410 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f0406f960 0x7f8f0415ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:57.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.198+0000 7f8f08a6f640 1 --2- 172.21.15.98:0/4243565410 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f0406f960 0x7f8f0415ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.198+0000 7f8f08a6f640 1 -- 172.21.15.98:0/4243565410 >> v1:172.21.15.139:6789/0 conn(0x7f8f0406d4c0 legacy=0x7f8f0406f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:57.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.198+0000 7f8f08a6f640 1 -- 172.21.15.98:0/4243565410 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f041625f0 con 0x7f8f0415bf30 2024-09-19T08:41:57.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.198+0000 7f8f09270640 1 --2- 172.21.15.98:0/4243565410 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f0406f960 0x7f8f0415ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:41:57.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.198+0000 7f8f08a6f640 1 --2- 172.21.15.98:0/4243565410 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f0415bf30 0x7f8f04073170 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8009870 tx=0x7f8ef801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657bc89aad70bc80 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:57.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.198+0000 7f8f037fe640 1 -- 172.21.15.98:0/4243565410 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef80b1070 con 0x7f8f0415bf30 2024-09-19T08:41:57.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.198+0000 7f8f037fe640 1 -- 172.21.15.98:0/4243565410 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ef80ac630 con 0x7f8f0415bf30 2024-09-19T08:41:57.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.198+0000 7f8f037fe640 1 -- 172.21.15.98:0/4243565410 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef80a8050 con 0x7f8f0415bf30 2024-09-19T08:41:57.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.202+0000 7f8f0acd3640 1 -- 172.21.15.98:0/4243565410 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f0415bf30 msgr2=0x7f8f04073170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:57.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.202+0000 7f8f0acd3640 1 --2- 172.21.15.98:0/4243565410 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f0415bf30 0x7f8f04073170 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8009870 tx=0x7f8ef801c520 comp rx=0 tx=0).stop 2024-09-19T08:41:57.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.202+0000 7f8f0acd3640 1 -- 172.21.15.98:0/4243565410 shutdown_connections 2024-09-19T08:41:57.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.202+0000 7f8f0acd3640 1 --2- 172.21.15.98:0/4243565410 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f0406f960 0x7f8f0415ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.202+0000 7f8f0acd3640 1 --2- 172.21.15.98:0/4243565410 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f0415bf30 0x7f8f04073170 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.202+0000 7f8f0acd3640 1 -- 172.21.15.98:0/4243565410 >> 172.21.15.98:0/4243565410 conn(0x7f8f040b40a0 msgr2=0x7f8f040b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:57.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.202+0000 7f8f0acd3640 1 -- 172.21.15.98:0/4243565410 shutdown_connections 2024-09-19T08:41:57.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.202+0000 7f8f0acd3640 1 -- 172.21.15.98:0/4243565410 wait complete. 2024-09-19T08:41:57.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.202+0000 7f8f0acd3640 1 Processor -- start 2024-09-19T08:41:57.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.202+0000 7f8f0acd3640 1 -- start start 2024-09-19T08:41:57.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.202+0000 7f8f0acd3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f0406d4c0 0x7f8f040ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:57.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.202+0000 7f8f0acd3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f0406f960 0x7f8f0415b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:57.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.202+0000 7f8f0acd3640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f0415bf30 0x7f8f041566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:57.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.202+0000 7f8f0acd3640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8f041620f0 con 0x7f8f0406f960 2024-09-19T08:41:57.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.202+0000 7f8f0acd3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8f04161f70 con 0x7f8f0406d4c0 2024-09-19T08:41:57.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.202+0000 7f8f0acd3640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8f0415f6e0 con 0x7f8f0415bf30 2024-09-19T08:41:57.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.202+0000 7f8f03fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f0406f960 0x7f8f0415b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:57.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.202+0000 7f8f03fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f0406f960 0x7f8f0415b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47172/0 (socket says 172.21.15.98:47172) 2024-09-19T08:41:57.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.202+0000 7f8f03fff640 1 -- 172.21.15.98:0/3890133152 learned_addr learned my addr 172.21.15.98:0/3890133152 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:57.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.202+0000 7f8f09270640 1 --2- 172.21.15.98:0/3890133152 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f0415bf30 0x7f8f041566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:57.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.202+0000 7f8f08a6f640 1 --2- 172.21.15.98:0/3890133152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f0406d4c0 0x7f8f040ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:57.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.202+0000 7f8f03fff640 1 -- 172.21.15.98:0/3890133152 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f0415bf30 msgr2=0x7f8f041566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:57.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.202+0000 7f8f03fff640 1 --2- 172.21.15.98:0/3890133152 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f0415bf30 0x7f8f041566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.202+0000 7f8f03fff640 1 -- 172.21.15.98:0/3890133152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f0406d4c0 msgr2=0x7f8f040ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:57.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.202+0000 7f8f03fff640 1 --2- 172.21.15.98:0/3890133152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f0406d4c0 0x7f8f040ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.202+0000 7f8f03fff640 1 -- 172.21.15.98:0/3890133152 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f040aee90 con 0x7f8f0406f960 2024-09-19T08:41:57.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.202+0000 7f8f03fff640 1 --2- 172.21.15.98:0/3890133152 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f0406f960 0x7f8f0415b0b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f8ef000c460 tx=0x7f8ef000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:57.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.202+0000 7f8f017fa640 1 -- 172.21.15.98:0/3890133152 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0016020 con 0x7f8f0406f960 2024-09-19T08:41:57.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.202+0000 7f8f017fa640 1 -- 172.21.15.98:0/3890133152 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ef0012870 con 0x7f8f0406f960 2024-09-19T08:41:57.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.202+0000 7f8f0acd3640 1 -- 172.21.15.98:0/3890133152 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f040af170 con 0x7f8f0406f960 2024-09-19T08:41:57.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.202+0000 7f8f017fa640 1 -- 172.21.15.98:0/3890133152 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0012c60 con 0x7f8f0406f960 2024-09-19T08:41:57.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.202+0000 7f8f0acd3640 1 -- 172.21.15.98:0/3890133152 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f040af730 con 0x7f8f0406f960 2024-09-19T08:41:57.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.202+0000 7f8f0acd3640 1 -- 172.21.15.98:0/3890133152 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f0415ed90 con 0x7f8f0406f960 2024-09-19T08:41:57.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.206+0000 7f8f017fa640 1 -- 172.21.15.98:0/3890133152 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f8ef0019020 con 0x7f8f0406f960 2024-09-19T08:41:57.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.206+0000 7f8f017fa640 1 --2- 172.21.15.98:0/3890133152 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8ee8067a70 0x7f8ee8069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:57.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.206+0000 7f8f017fa640 1 -- 172.21.15.98:0/3890133152 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f8ef0085d20 con 0x7f8f0406f960 2024-09-19T08:41:57.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.206+0000 7f8f08a6f640 1 --2- 172.21.15.98:0/3890133152 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8ee8067a70 0x7f8ee8069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:57.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.206+0000 7f8f08a6f640 1 --2- 172.21.15.98:0/3890133152 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8ee8067a70 0x7f8ee8069f30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8ef80ae000 tx=0x7f8ef8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:57.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.210+0000 7f8f017fa640 1 -- 172.21.15.98:0/3890133152 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f8ef00212a0 con 0x7f8f0406f960 2024-09-19T08:41:57.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.390+0000 7f8f0acd3640 1 -- 172.21.15.98:0/3890133152 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8f040732c0 con 0x7f8f0406f960 2024-09-19T08:41:57.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.390+0000 7f8f017fa640 1 -- 172.21.15.98:0/3890133152 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v30) ==== 53+0+73 (secure 0 0 0) 0x7f8ef00565e0 con 0x7f8f0406f960 2024-09-19T08:41:57.392 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T08:41:57.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.394+0000 7f8f0acd3640 1 -- 172.21.15.98:0/3890133152 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8ee8067a70 msgr2=0x7f8ee8069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:57.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.394+0000 7f8f0acd3640 1 --2- 172.21.15.98:0/3890133152 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8ee8067a70 0x7f8ee8069f30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8ef80ae000 tx=0x7f8ef8008040 comp rx=0 tx=0).stop 2024-09-19T08:41:57.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.394+0000 7f8f0acd3640 1 -- 172.21.15.98:0/3890133152 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f0406f960 msgr2=0x7f8f0415b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:57.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.394+0000 7f8f0acd3640 1 --2- 172.21.15.98:0/3890133152 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f0406f960 0x7f8f0415b0b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f8ef000c460 tx=0x7f8ef000c920 comp rx=0 tx=0).stop 2024-09-19T08:41:57.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.394+0000 7f8f0acd3640 1 -- 172.21.15.98:0/3890133152 shutdown_connections 2024-09-19T08:41:57.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.394+0000 7f8f0acd3640 1 --2- 172.21.15.98:0/3890133152 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8ee8067a70 0x7f8ee8069f30 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.394+0000 7f8f0acd3640 1 --2- 172.21.15.98:0/3890133152 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f0415bf30 0x7f8f041566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.394+0000 7f8f0acd3640 1 --2- 172.21.15.98:0/3890133152 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f0406f960 0x7f8f0415b0b0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.394+0000 7f8f0acd3640 1 --2- 172.21.15.98:0/3890133152 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f0406d4c0 0x7f8f040ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:57.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.394+0000 7f8f0acd3640 1 -- 172.21.15.98:0/3890133152 >> 172.21.15.98:0/3890133152 conn(0x7f8f040b40a0 msgr2=0x7f8f040738c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:57.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.394+0000 7f8f0acd3640 1 -- 172.21.15.98:0/3890133152 shutdown_connections 2024-09-19T08:41:57.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57.394+0000 7f8f0acd3640 1 -- 172.21.15.98:0/3890133152 wait complete. 2024-09-19T08:41:57.409 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T08:41:57.410 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T08:41:57.410 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi098.front.sepia.ceph.com 2024-09-19T08:41:57.410 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T08:41:57.410 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T08:41:57.410 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T08:41:57.410 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T08:41:57.410 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T08:41:57.410 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:41:57.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:41:57.411 DEBUG:teuthology.orchestra.run.smithi098:> ip addr 2024-09-19T08:41:57.417 INFO:teuthology.orchestra.run.smithi098.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T08:41:57.417 INFO:teuthology.orchestra.run.smithi098.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T08:41:57.417 INFO:teuthology.orchestra.run.smithi098.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T08:41:57.417 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:41:57.417 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 ::1/128 scope host 2024-09-19T08:41:57.417 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:41:57.417 INFO:teuthology.orchestra.run.smithi098.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:41:57.417 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b8 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:41:57.417 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f0 2024-09-19T08:41:57.417 INFO:teuthology.orchestra.run.smithi098.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:41:57.417 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b9 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:41:57.418 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f1 2024-09-19T08:41:57.418 INFO:teuthology.orchestra.run.smithi098.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:41:57.418 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6a brd ff:ff:ff:ff:ff:ff 2024-09-19T08:41:57.418 INFO:teuthology.orchestra.run.smithi098.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T08:41:57.418 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6b brd ff:ff:ff:ff:ff:ff 2024-09-19T08:41:57.418 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.21.15.98/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T08:41:57.418 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft 42291sec preferred_lft 42291sec 2024-09-19T08:41:57.418 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::ec4:7aff:fe8f:d26b/64 scope link 2024-09-19T08:41:57.418 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:41:57.418 INFO:teuthology.orchestra.run.smithi098.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T08:41:57.418 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 02:42:98:dd:32:2d brd ff:ff:ff:ff:ff:ff 2024-09-19T08:41:57.418 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T08:41:57.418 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:41:57.419 INFO:teuthology.orchestra.run.smithi098.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T08:41:57.419 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-19T08:41:57.419 INFO:teuthology.orchestra.run.smithi098.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T08:41:57.419 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:41:57.419 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-19T08:41:57.419 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:41:57.419 INFO:teuthology.orchestra.run.smithi098.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:41:57.419 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:41:57.419 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-19T08:41:57.419 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:41:57.419 INFO:teuthology.orchestra.run.smithi098.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:41:57.419 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:41:57.419 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-19T08:41:57.419 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:41:57.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:41:57.420 DEBUG:teuthology.orchestra.run.smithi098:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T08:41:57.468 INFO:teuthology.orchestra.run.smithi098.stdout:1 2024-09-19T08:41:57.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:41:57.469 DEBUG:teuthology.orchestra.run.smithi098:> route 2024-09-19T08:41:57.520 INFO:teuthology.orchestra.run.smithi098.stdout:Kernel IP routing table 2024-09-19T08:41:57.520 INFO:teuthology.orchestra.run.smithi098.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T08:41:57.520 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T08:41:57.521 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T08:41:57.521 INFO:teuthology.orchestra.run.smithi098.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T08:41:57.521 INFO:teuthology.orchestra.run.smithi098.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T08:41:57.521 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T08:41:57.521 INFO:teuthology.orchestra.run.smithi098.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:41:57.521 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:41:57.521 INFO:teuthology.orchestra.run.smithi098.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:41:57.521 INFO:teuthology.orchestra.run.smithi098.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T08:41:57.521 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:41:57.521 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-09-19T08:41:57.522 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T08:41:57.522 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T08:41:57.522 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T08:41:57.522 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-09-19T08:41:57.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:41:57.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:41:57.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:41:57.644 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2024-09-19T08:41:57.693 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-19T08:41:57.693 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T08:41:57.693 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:41:57.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:41:57.694 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set brx.0 up 2024-09-19T08:41:57.749 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-19T08:41:57.750 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:41:57.750 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T08:41:57.801 INFO:teuthology.orchestra.run.smithi098.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:41:57.801 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:41:57.801 DEBUG:teuthology.orchestra.run.smithi098:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T08:41:57.848 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:41:57.906 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:41:57.954 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:41:57.954 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:41:57.954 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:41:57.954 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:41:57.954 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:41:57.954 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:41:57.954 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:41:57.954 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:41:57.954 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:41:57.955 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:41:57.955 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:41:57.955 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:41:57.955 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:41:57.955 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:41:57.955 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:41:57.955 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:41:57.955 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:41:57.955 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:41:57.955 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:41:57.955 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:41:57.955 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:41:57.955 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:41:57.956 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:41:57.956 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:41:57.956 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:41:57.956 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:41:57.956 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:41:57.956 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:41:57.956 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:41:57.956 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:41:57.956 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:41:57.956 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:41:57.956 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:41:57.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:41:57.957 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:41:58.005 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T08:41:58.005 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-19T08:41:58.050 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:41:58.065 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:41:58.112 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:41:58.112 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:41:58.113 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:41:58.113 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:41:58.113 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:41:58.113 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:41:58.113 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:41:58.113 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:41:58.113 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:41:58.113 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:41:58.113 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:41:58.113 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:41:58.113 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:41:58.113 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:41:58.113 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:41:58.114 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:41:58.114 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:41:58.114 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:41:58.114 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:41:58.114 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:41:58.114 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:41:58.114 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:41:58.114 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:41:58.114 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:41:58.114 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:41:58.114 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:41:58.114 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:41:58.114 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:41:58.114 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:41:58.115 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:41:58.115 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:41:58.115 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:41:58.115 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:41:58.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:41:58.116 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:41:58.130 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:2024-09-19T08:41:58.130+0000 7f4c83399580 -1 init, newargv = 0x555c537be2b0 newargc=15 2024-09-19T08:41:58.130 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:ceph-fuse[41283]: starting ceph client 2024-09-19T08:41:58.136 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:ceph-fuse[41283]: starting fuse 2024-09-19T08:41:59.235 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:41:59.252 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:41:59.302 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:41:59.303 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:41:59.303 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:41:59.303 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:41:59.303 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:41:59.303 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:41:59.304 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:41:59.304 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:41:59.304 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:41:59.304 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:41:59.304 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:41:59.304 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:41:59.305 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:41:59.305 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:41:59.305 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:41:59.305 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:41:59.305 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:41:59.305 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:41:59.306 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:41:59.306 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:41:59.306 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:41:59.306 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:41:59.306 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:41:59.306 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:41:59.307 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:41:59.307 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:41:59.307 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:41:59.307 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:41:59.307 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:41:59.308 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:41:59.308 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:41:59.308 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:41:59.308 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:41:59.309 INFO:teuthology.orchestra.run.smithi098.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-19T08:41:59.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:41:59.310 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:41:59.357 INFO:teuthology.orchestra.run.smithi098.stdout:45 2024-09-19T08:41:59.357 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-19T08:41:59.358 DEBUG:teuthology.orchestra.run.smithi098:> sudo stdin-killer -- python3 -c ' 2024-09-19T08:41:59.358 DEBUG:teuthology.orchestra.run.smithi098:> import glob 2024-09-19T08:41:59.358 DEBUG:teuthology.orchestra.run.smithi098:> import re 2024-09-19T08:41:59.358 DEBUG:teuthology.orchestra.run.smithi098:> import os 2024-09-19T08:41:59.358 DEBUG:teuthology.orchestra.run.smithi098:> import subprocess 2024-09-19T08:41:59.358 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:41:59.358 DEBUG:teuthology.orchestra.run.smithi098:> def _find_admin_socket(client_name): 2024-09-19T08:41:59.358 DEBUG:teuthology.orchestra.run.smithi098:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T08:41:59.358 DEBUG:teuthology.orchestra.run.smithi098:> files = glob.glob(asok_path) 2024-09-19T08:41:59.358 DEBUG:teuthology.orchestra.run.smithi098:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T08:41:59.358 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:41:59.359 DEBUG:teuthology.orchestra.run.smithi098:> # Given a non-glob path, it better be there 2024-09-19T08:41:59.359 DEBUG:teuthology.orchestra.run.smithi098:> if "*" not in asok_path: 2024-09-19T08:41:59.359 DEBUG:teuthology.orchestra.run.smithi098:> assert(len(files) == 1) 2024-09-19T08:41:59.359 DEBUG:teuthology.orchestra.run.smithi098:> return files[0] 2024-09-19T08:41:59.359 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:41:59.359 DEBUG:teuthology.orchestra.run.smithi098:> for f in files: 2024-09-19T08:41:59.359 DEBUG:teuthology.orchestra.run.smithi098:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T08:41:59.359 DEBUG:teuthology.orchestra.run.smithi098:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T08:41:59.359 DEBUG:teuthology.orchestra.run.smithi098:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T08:41:59.359 DEBUG:teuthology.orchestra.run.smithi098:> contents = proc_f.read() 2024-09-19T08:41:59.359 DEBUG:teuthology.orchestra.run.smithi098:> if mountpoint in contents: 2024-09-19T08:41:59.359 DEBUG:teuthology.orchestra.run.smithi098:> return f 2024-09-19T08:41:59.359 DEBUG:teuthology.orchestra.run.smithi098:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T08:41:59.359 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:41:59.360 DEBUG:teuthology.orchestra.run.smithi098:> print(_find_admin_socket("client.0")) 2024-09-19T08:41:59.360 DEBUG:teuthology.orchestra.run.smithi098:> ' 2024-09-19T08:41:59.482 INFO:teuthology.orchestra.run.smithi098.stdout:/var/run/ceph/ceph-client.0.41283.asok 2024-09-19T08:41:59.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:41:59.494 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.41283.asok 2024-09-19T08:41:59.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:41:59.494 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.41283.asok status 2024-09-19T08:41:59.649 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:41:59.649 INFO:teuthology.orchestra.run.smithi098.stdout: "metadata": { 2024-09-19T08:41:59.649 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T08:41:59.650 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T08:41:59.650 INFO:teuthology.orchestra.run.smithi098.stdout: "entity_id": "0", 2024-09-19T08:41:59.650 INFO:teuthology.orchestra.run.smithi098.stdout: "hostname": "smithi098", 2024-09-19T08:41:59.650 INFO:teuthology.orchestra.run.smithi098.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T08:41:59.650 INFO:teuthology.orchestra.run.smithi098.stdout: "pid": "41283", 2024-09-19T08:41:59.650 INFO:teuthology.orchestra.run.smithi098.stdout: "root": "/" 2024-09-19T08:41:59.650 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:41:59.650 INFO:teuthology.orchestra.run.smithi098.stdout: "dentry_count": 0, 2024-09-19T08:41:59.650 INFO:teuthology.orchestra.run.smithi098.stdout: "dentry_pinned_count": 0, 2024-09-19T08:41:59.650 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 5928, 2024-09-19T08:41:59.650 INFO:teuthology.orchestra.run.smithi098.stdout: "inst": { 2024-09-19T08:41:59.650 INFO:teuthology.orchestra.run.smithi098.stdout: "name": { 2024-09-19T08:41:59.651 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "client", 2024-09-19T08:41:59.651 INFO:teuthology.orchestra.run.smithi098.stdout: "num": 5928 2024-09-19T08:41:59.651 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:41:59.651 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": { 2024-09-19T08:41:59.651 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v1", 2024-09-19T08:41:59.651 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:41:59.651 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 1199674616 2024-09-19T08:41:59.651 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:41:59.651 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:41:59.651 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": { 2024-09-19T08:41:59.651 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v1", 2024-09-19T08:41:59.651 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:41:59.652 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 1199674616 2024-09-19T08:41:59.652 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:41:59.652 INFO:teuthology.orchestra.run.smithi098.stdout: "inst_str": "client.5928 192.168.0.1:0/1199674616", 2024-09-19T08:41:59.652 INFO:teuthology.orchestra.run.smithi098.stdout: "addr_str": "192.168.0.1:0/1199674616", 2024-09-19T08:41:59.652 INFO:teuthology.orchestra.run.smithi098.stdout: "inode_count": 1, 2024-09-19T08:41:59.652 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_epoch": 30, 2024-09-19T08:41:59.652 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_epoch": 57, 2024-09-19T08:41:59.652 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_epoch_barrier": 0, 2024-09-19T08:41:59.652 INFO:teuthology.orchestra.run.smithi098.stdout: "blocklisted": false, 2024-09-19T08:41:59.652 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "cephfs" 2024-09-19T08:41:59.652 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:41:59.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:41:59.664 DEBUG:teuthology.orchestra.run.smithi098:> stat --file-system '--printf=%T 2024-09-19T08:41:59.664 DEBUG:teuthology.orchestra.run.smithi098:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T08:41:59.671 INFO:teuthology.orchestra.run.smithi098.stdout:fuseblk 2024-09-19T08:41:59.671 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T08:41:59.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:41:59.671 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T08:41:59.726 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T08:41:59.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.894+0000 7ff2b2b06640 1 Processor -- start 2024-09-19T08:41:59.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.894+0000 7ff2b2b06640 1 -- start start 2024-09-19T08:41:59.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.894+0000 7ff2b2b06640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff2ac15c550 0x7ff2ac06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:59.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.894+0000 7ff2b2b06640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2ac0cfad0 0x7ff2ac0723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:59.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.894+0000 7ff2b2b06640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2ac06d310 con 0x7ff2ac06c1d0 2024-09-19T08:41:59.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.894+0000 7ff2b2b06640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff2ac06c640 con 0x7ff2ac15c550 2024-09-19T08:41:59.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.894+0000 7ff2b2b06640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff2ac06d5f0 con 0x7ff2ac0cfad0 2024-09-19T08:41:59.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.894+0000 7ff2b10a3640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2ac0cfad0 0x7ff2ac0723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:59.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.894+0000 7ff2b08a2640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff2ac15c550 0x7ff2ac06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:59.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.894+0000 7ff2b10a3640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2ac0cfad0 0x7ff2ac0723a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:34396/0 (socket says 172.21.15.98:34396) 2024-09-19T08:41:59.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.894+0000 7ff2b10a3640 1 -- 172.21.15.98:0/3175604251 learned_addr learned my addr 172.21.15.98:0/3175604251 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:59.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.894+0000 7ff2b10a3640 1 -- 172.21.15.98:0/3175604251 >> v2:172.21.15.139:3300/0 conn(0x7ff2ac15c550 msgr2=0x7ff2ac06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:41:59.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.894+0000 7ff2b10a3640 1 --2- 172.21.15.98:0/3175604251 >> v2:172.21.15.139:3300/0 conn(0x7ff2ac15c550 0x7ff2ac06bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:59.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.894+0000 7ff2b10a3640 1 -- 172.21.15.98:0/3175604251 >> v1:172.21.15.139:6789/0 conn(0x7ff2ac06c1d0 legacy=0x7ff2ac0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:59.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.894+0000 7ff2b10a3640 1 -- 172.21.15.98:0/3175604251 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2ac06d8d0 con 0x7ff2ac0cfad0 2024-09-19T08:41:59.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.894+0000 7ff2b08a2640 1 --2- 172.21.15.98:0/3175604251 >> v2:172.21.15.139:3300/0 conn(0x7ff2ac15c550 0x7ff2ac06bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:41:59.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.898+0000 7ff2b10a3640 1 --2- 172.21.15.98:0/3175604251 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2ac0cfad0 0x7ff2ac0723a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4009e50 tx=0x7ff2a401ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96d5c28ffc5c53e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:59.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.898+0000 7ff2ab7fe640 1 -- 172.21.15.98:0/3175604251 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a40a5810 con 0x7ff2ac0cfad0 2024-09-19T08:41:59.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.898+0000 7ff2ab7fe640 1 -- 172.21.15.98:0/3175604251 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2a40a59b0 con 0x7ff2ac0cfad0 2024-09-19T08:41:59.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.898+0000 7ff2ab7fe640 1 -- 172.21.15.98:0/3175604251 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a40a5d80 con 0x7ff2ac0cfad0 2024-09-19T08:41:59.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.898+0000 7ff2b2b06640 1 -- 172.21.15.98:0/3175604251 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2ac0cfad0 msgr2=0x7ff2ac0723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:59.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.898+0000 7ff2b2b06640 1 --2- 172.21.15.98:0/3175604251 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2ac0cfad0 0x7ff2ac0723a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4009e50 tx=0x7ff2a401ca30 comp rx=0 tx=0).stop 2024-09-19T08:41:59.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.898+0000 7ff2b2b06640 1 -- 172.21.15.98:0/3175604251 shutdown_connections 2024-09-19T08:41:59.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.898+0000 7ff2b2b06640 1 --2- 172.21.15.98:0/3175604251 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2ac0cfad0 0x7ff2ac0723a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:59.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.898+0000 7ff2b2b06640 1 --2- 172.21.15.98:0/3175604251 >> v2:172.21.15.139:3300/0 conn(0x7ff2ac15c550 0x7ff2ac06bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:59.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.898+0000 7ff2b2b06640 1 -- 172.21.15.98:0/3175604251 >> 172.21.15.98:0/3175604251 conn(0x7ff2ac0b2010 msgr2=0x7ff2ac0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:41:59.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.898+0000 7ff2b2b06640 1 -- 172.21.15.98:0/3175604251 shutdown_connections 2024-09-19T08:41:59.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.898+0000 7ff2b2b06640 1 -- 172.21.15.98:0/3175604251 wait complete. 2024-09-19T08:41:59.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.898+0000 7ff2b2b06640 1 Processor -- start 2024-09-19T08:41:59.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.898+0000 7ff2b2b06640 1 -- start start 2024-09-19T08:41:59.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.902+0000 7ff2b2b06640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2ac06c1d0 0x7ff2ac1544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:59.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.902+0000 7ff2b2b06640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2ac0cfad0 0x7ff2ac14d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:59.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.902+0000 7ff2b2b06640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2ac15c550 0x7ff2ac1526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:59.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.902+0000 7ff2b2b06640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ac073500 con 0x7ff2ac15c550 2024-09-19T08:41:59.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.902+0000 7ff2b2b06640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ac073380 con 0x7ff2ac0cfad0 2024-09-19T08:41:59.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.902+0000 7ff2b2b06640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff2ac073680 con 0x7ff2ac06c1d0 2024-09-19T08:41:59.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.902+0000 7ff2b10a3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2ac15c550 0x7ff2ac1526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:59.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.902+0000 7ff2b10a3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2ac15c550 0x7ff2ac1526d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:43916/0 (socket says 172.21.15.98:43916) 2024-09-19T08:41:59.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.902+0000 7ff2b10a3640 1 -- 172.21.15.98:0/2556021283 learned_addr learned my addr 172.21.15.98:0/2556021283 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:41:59.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.902+0000 7ff2b08a2640 1 --2- 172.21.15.98:0/2556021283 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2ac06c1d0 0x7ff2ac1544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:59.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.902+0000 7ff2abfff640 1 --2- 172.21.15.98:0/2556021283 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2ac0cfad0 0x7ff2ac14d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:59.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.902+0000 7ff2b10a3640 1 -- 172.21.15.98:0/2556021283 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2ac06c1d0 msgr2=0x7ff2ac1544b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:59.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.902+0000 7ff2b10a3640 1 --2- 172.21.15.98:0/2556021283 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2ac06c1d0 0x7ff2ac1544b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:59.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.902+0000 7ff2b10a3640 1 -- 172.21.15.98:0/2556021283 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2ac0cfad0 msgr2=0x7ff2ac14d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:41:59.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.902+0000 7ff2b10a3640 1 --2- 172.21.15.98:0/2556021283 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2ac0cfad0 0x7ff2ac14d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:41:59.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.902+0000 7ff2b10a3640 1 -- 172.21.15.98:0/2556021283 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2ac1549f0 con 0x7ff2ac15c550 2024-09-19T08:41:59.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.902+0000 7ff2b10a3640 1 --2- 172.21.15.98:0/2556021283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2ac15c550 0x7ff2ac1526d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff2a401a660 tx=0x7ff2a40197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:59.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.902+0000 7ff2a97fa640 1 -- 172.21.15.98:0/2556021283 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a40b8670 con 0x7ff2ac15c550 2024-09-19T08:41:59.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.902+0000 7ff2a97fa640 1 -- 172.21.15.98:0/2556021283 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2a40be070 con 0x7ff2ac15c550 2024-09-19T08:41:59.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.902+0000 7ff2b2b06640 1 -- 172.21.15.98:0/2556021283 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2ac14beb0 con 0x7ff2ac15c550 2024-09-19T08:41:59.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.902+0000 7ff2a97fa640 1 -- 172.21.15.98:0/2556021283 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a40b8a90 con 0x7ff2ac15c550 2024-09-19T08:41:59.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.902+0000 7ff2b2b06640 1 -- 172.21.15.98:0/2556021283 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2ac14c470 con 0x7ff2ac15c550 2024-09-19T08:41:59.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.902+0000 7ff2b2b06640 1 -- 172.21.15.98:0/2556021283 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2ac0723a0 con 0x7ff2ac15c550 2024-09-19T08:41:59.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.902+0000 7ff2a97fa640 1 -- 172.21.15.98:0/2556021283 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7ff2a40b8c30 con 0x7ff2ac15c550 2024-09-19T08:41:59.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.902+0000 7ff2a97fa640 1 --2- 172.21.15.98:0/2556021283 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff298067a70 0x7ff298069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:41:59.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.902+0000 7ff2a97fa640 1 -- 172.21.15.98:0/2556021283 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7ff2a41211d0 con 0x7ff2ac15c550 2024-09-19T08:41:59.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.902+0000 7ff2b08a2640 1 --2- 172.21.15.98:0/2556021283 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff298067a70 0x7ff298069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:41:59.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.906+0000 7ff2b08a2640 1 --2- 172.21.15.98:0/2556021283 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff298067a70 0x7ff298069f30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0009580 tx=0x7ff2a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:41:59.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:41:59.910+0000 7ff2a97fa640 1 -- 172.21.15.98:0/2556021283 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff2a4016620 con 0x7ff2ac15c550 2024-09-19T08:42:00.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:00.206+0000 7ff2b2b06640 1 -- 172.21.15.98:0/2556021283 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff2ac06cba0 con 0x7ff2ac15c550 2024-09-19T08:42:00.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:00.206+0000 7ff2a97fa640 1 -- 172.21.15.98:0/2556021283 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v30) ==== 53+0+73 (secure 0 0 0) 0x7ff2a40f1aa0 con 0x7ff2ac15c550 2024-09-19T08:42:00.206 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T08:42:00.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:00.206+0000 7ff2b2b06640 1 -- 172.21.15.98:0/2556021283 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff298067a70 msgr2=0x7ff298069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:00.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:00.206+0000 7ff2b2b06640 1 --2- 172.21.15.98:0/2556021283 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff298067a70 0x7ff298069f30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0009580 tx=0x7ff2a0008040 comp rx=0 tx=0).stop 2024-09-19T08:42:00.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:00.206+0000 7ff2b2b06640 1 -- 172.21.15.98:0/2556021283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2ac15c550 msgr2=0x7ff2ac1526d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:00.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:00.206+0000 7ff2b2b06640 1 --2- 172.21.15.98:0/2556021283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2ac15c550 0x7ff2ac1526d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff2a401a660 tx=0x7ff2a40197d0 comp rx=0 tx=0).stop 2024-09-19T08:42:00.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:00.206+0000 7ff2b2b06640 1 -- 172.21.15.98:0/2556021283 shutdown_connections 2024-09-19T08:42:00.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:00.206+0000 7ff2b2b06640 1 --2- 172.21.15.98:0/2556021283 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff298067a70 0x7ff298069f30 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:00.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:00.206+0000 7ff2b2b06640 1 --2- 172.21.15.98:0/2556021283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2ac15c550 0x7ff2ac1526d0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:00.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:00.206+0000 7ff2b2b06640 1 --2- 172.21.15.98:0/2556021283 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2ac0cfad0 0x7ff2ac14d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:00.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:00.206+0000 7ff2b2b06640 1 --2- 172.21.15.98:0/2556021283 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2ac06c1d0 0x7ff2ac1544b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:00.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:00.206+0000 7ff2b2b06640 1 -- 172.21.15.98:0/2556021283 >> 172.21.15.98:0/2556021283 conn(0x7ff2ac0b2010 msgr2=0x7ff2ac154d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:00.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:00.206+0000 7ff2b2b06640 1 -- 172.21.15.98:0/2556021283 shutdown_connections 2024-09-19T08:42:00.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:00.206+0000 7ff2b2b06640 1 -- 172.21.15.98:0/2556021283 wait complete. 2024-09-19T08:42:00.223 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T08:42:00.223 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T08:42:00.223 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2024-09-19T08:42:00.223 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-19T08:42:00.223 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-19T08:42:00.223 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T08:42:00.223 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T08:42:00.223 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T08:42:00.223 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T08:42:00.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:00.224 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2024-09-19T08:42:00.230 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T08:42:00.231 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T08:42:00.231 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T08:42:00.231 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:42:00.231 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-09-19T08:42:00.231 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:42:00.231 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:42:00.231 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:42:00.231 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2024-09-19T08:42:00.231 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:42:00.231 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:42:00.231 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2024-09-19T08:42:00.231 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:42:00.232 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:42:00.232 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T08:42:00.232 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:42:00.232 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T08:42:00.232 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 42300sec preferred_lft 42300sec 2024-09-19T08:42:00.232 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::ec4:7aff:fe88:7f59/64 scope link 2024-09-19T08:42:00.232 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:42:00.232 INFO:teuthology.orchestra.run.smithi139.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T08:42:00.232 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 02:42:74:ed:e4:60 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:42:00.232 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T08:42:00.232 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:42:00.232 INFO:teuthology.orchestra.run.smithi139.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T08:42:00.232 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-19T08:42:00.233 INFO:teuthology.orchestra.run.smithi139.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T08:42:00.233 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:42:00.233 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-19T08:42:00.233 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:42:00.233 INFO:teuthology.orchestra.run.smithi139.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:42:00.233 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:42:00.233 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-19T08:42:00.233 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:42:00.233 INFO:teuthology.orchestra.run.smithi139.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:42:00.233 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:42:00.233 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-19T08:42:00.233 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:42:00.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:00.234 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T08:42:00.287 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-09-19T08:42:00.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:00.289 DEBUG:teuthology.orchestra.run.smithi139:> route 2024-09-19T08:42:00.342 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2024-09-19T08:42:00.342 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T08:42:00.342 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T08:42:00.342 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T08:42:00.342 INFO:teuthology.orchestra.run.smithi139.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T08:42:00.342 INFO:teuthology.orchestra.run.smithi139.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T08:42:00.343 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T08:42:00.343 INFO:teuthology.orchestra.run.smithi139.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:42:00.343 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:42:00.343 INFO:teuthology.orchestra.run.smithi139.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:42:00.343 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T08:42:00.343 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:42:00.343 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-09-19T08:42:00.344 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T08:42:00.344 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T08:42:00.344 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T08:42:00.344 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-09-19T08:42:00.431 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:42:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:42:00.459 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:42:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:42:00.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:00.467 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-09-19T08:42:00.472 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-19T08:42:00.472 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-19T08:42:00.473 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T08:42:00.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:00.473 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.0 up 2024-09-19T08:42:00.525 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-19T08:42:00.525 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:42:00.525 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-19T08:42:00.574 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-19T08:42:00.574 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:42:00.574 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-19T08:42:00.622 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:42:00.679 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:42:00.728 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:42:00.728 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:42:00.728 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:42:00.728 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:42:00.728 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:42:00.729 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:42:00.729 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:42:00.729 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:42:00.729 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:42:00.729 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:42:00.729 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:42:00.729 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:42:00.729 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:42:00.729 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:42:00.729 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:42:00.729 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:42:00.729 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:42:00.729 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:42:00.729 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:42:00.730 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:42:00.730 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:42:00.730 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:42:00.730 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:42:00.730 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:42:00.730 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:42:00.730 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:42:00.730 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:42:00.730 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:42:00.730 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:42:00.730 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:42:00.730 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:42:00.730 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:42:00.730 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:42:00.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:00.731 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:42:00.778 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T08:42:00.778 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-19T08:42:00.823 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:42:00.837 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:42:00.886 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:42:00.887 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:42:00.887 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:42:00.887 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:42:00.887 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:42:00.887 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:42:00.887 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:42:00.887 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:42:00.887 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:42:00.887 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:42:00.887 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:42:00.887 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:42:00.887 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:42:00.887 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:42:00.888 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:42:00.888 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:42:00.888 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:42:00.888 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:42:00.888 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:42:00.888 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:42:00.888 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:42:00.888 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:42:00.888 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:42:00.888 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:42:00.888 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:42:00.888 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:42:00.888 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:42:00.889 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:42:00.889 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:42:00.889 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:42:00.889 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:42:00.889 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:42:00.889 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:42:00.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:00.890 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:42:00.903 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:2024-09-19T08:42:00.900+0000 7ff670cbe580 -1 init, newargv = 0x55c5beb5b370 newargc=15 2024-09-19T08:42:00.903 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[35574]: starting ceph client 2024-09-19T08:42:00.912 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[35574]: starting fuse 2024-09-19T08:42:02.009 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:42:02.025 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:42:02.077 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:42:02.077 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:42:02.077 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:42:02.077 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:42:02.077 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:42:02.077 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:42:02.077 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:42:02.077 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:42:02.077 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:42:02.077 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:42:02.077 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:42:02.078 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:42:02.078 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:42:02.078 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:42:02.078 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:42:02.078 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:42:02.078 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:42:02.078 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:42:02.078 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:42:02.078 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:42:02.078 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:42:02.078 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:42:02.078 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:42:02.078 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:42:02.079 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:42:02.079 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:42:02.079 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:42:02.079 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:42:02.079 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:42:02.079 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:42:02.079 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:42:02.079 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:42:02.079 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:42:02.079 INFO:teuthology.orchestra.run.smithi139.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-19T08:42:02.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:02.080 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:42:02.126 INFO:teuthology.orchestra.run.smithi139.stdout:45 2024-09-19T08:42:02.126 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-19T08:42:02.126 DEBUG:teuthology.orchestra.run.smithi139:> sudo stdin-killer -- python3 -c ' 2024-09-19T08:42:02.127 DEBUG:teuthology.orchestra.run.smithi139:> import glob 2024-09-19T08:42:02.127 DEBUG:teuthology.orchestra.run.smithi139:> import re 2024-09-19T08:42:02.127 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-09-19T08:42:02.127 DEBUG:teuthology.orchestra.run.smithi139:> import subprocess 2024-09-19T08:42:02.127 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:42:02.127 DEBUG:teuthology.orchestra.run.smithi139:> def _find_admin_socket(client_name): 2024-09-19T08:42:02.127 DEBUG:teuthology.orchestra.run.smithi139:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-19T08:42:02.127 DEBUG:teuthology.orchestra.run.smithi139:> files = glob.glob(asok_path) 2024-09-19T08:42:02.127 DEBUG:teuthology.orchestra.run.smithi139:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-19T08:42:02.127 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:42:02.127 DEBUG:teuthology.orchestra.run.smithi139:> # Given a non-glob path, it better be there 2024-09-19T08:42:02.127 DEBUG:teuthology.orchestra.run.smithi139:> if "*" not in asok_path: 2024-09-19T08:42:02.127 DEBUG:teuthology.orchestra.run.smithi139:> assert(len(files) == 1) 2024-09-19T08:42:02.127 DEBUG:teuthology.orchestra.run.smithi139:> return files[0] 2024-09-19T08:42:02.127 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:42:02.128 DEBUG:teuthology.orchestra.run.smithi139:> for f in files: 2024-09-19T08:42:02.128 DEBUG:teuthology.orchestra.run.smithi139:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T08:42:02.128 DEBUG:teuthology.orchestra.run.smithi139:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T08:42:02.128 DEBUG:teuthology.orchestra.run.smithi139:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T08:42:02.128 DEBUG:teuthology.orchestra.run.smithi139:> contents = proc_f.read() 2024-09-19T08:42:02.128 DEBUG:teuthology.orchestra.run.smithi139:> if mountpoint in contents: 2024-09-19T08:42:02.128 DEBUG:teuthology.orchestra.run.smithi139:> return f 2024-09-19T08:42:02.128 DEBUG:teuthology.orchestra.run.smithi139:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T08:42:02.128 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:42:02.128 DEBUG:teuthology.orchestra.run.smithi139:> print(_find_admin_socket("client.1")) 2024-09-19T08:42:02.128 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-09-19T08:42:02.245 INFO:teuthology.orchestra.run.smithi139.stdout:/var/run/ceph/ceph-client.1.35574.asok 2024-09-19T08:42:02.248 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:42:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:42:02.256 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.35574.asok 2024-09-19T08:42:02.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:02.256 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.35574.asok status 2024-09-19T08:42:02.411 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-19T08:42:02.411 INFO:teuthology.orchestra.run.smithi139.stdout: "metadata": { 2024-09-19T08:42:02.411 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T08:42:02.411 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T08:42:02.411 INFO:teuthology.orchestra.run.smithi139.stdout: "entity_id": "1", 2024-09-19T08:42:02.411 INFO:teuthology.orchestra.run.smithi139.stdout: "hostname": "smithi139", 2024-09-19T08:42:02.412 INFO:teuthology.orchestra.run.smithi139.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-19T08:42:02.412 INFO:teuthology.orchestra.run.smithi139.stdout: "pid": "35574", 2024-09-19T08:42:02.412 INFO:teuthology.orchestra.run.smithi139.stdout: "root": "/" 2024-09-19T08:42:02.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:42:02.412 INFO:teuthology.orchestra.run.smithi139.stdout: "dentry_count": 0, 2024-09-19T08:42:02.412 INFO:teuthology.orchestra.run.smithi139.stdout: "dentry_pinned_count": 0, 2024-09-19T08:42:02.412 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 5401, 2024-09-19T08:42:02.412 INFO:teuthology.orchestra.run.smithi139.stdout: "inst": { 2024-09-19T08:42:02.412 INFO:teuthology.orchestra.run.smithi139.stdout: "name": { 2024-09-19T08:42:02.412 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "client", 2024-09-19T08:42:02.412 INFO:teuthology.orchestra.run.smithi139.stdout: "num": 5401 2024-09-19T08:42:02.412 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:42:02.412 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": { 2024-09-19T08:42:02.412 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-19T08:42:02.412 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:42:02.413 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3041004904 2024-09-19T08:42:02.413 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-19T08:42:02.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:42:02.413 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": { 2024-09-19T08:42:02.413 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-19T08:42:02.413 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:42:02.413 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3041004904 2024-09-19T08:42:02.413 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:42:02.413 INFO:teuthology.orchestra.run.smithi139.stdout: "inst_str": "client.5401 192.168.0.1:0/3041004904", 2024-09-19T08:42:02.413 INFO:teuthology.orchestra.run.smithi139.stdout: "addr_str": "192.168.0.1:0/3041004904", 2024-09-19T08:42:02.413 INFO:teuthology.orchestra.run.smithi139.stdout: "inode_count": 1, 2024-09-19T08:42:02.413 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_epoch": 30, 2024-09-19T08:42:02.413 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_epoch": 57, 2024-09-19T08:42:02.414 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_epoch_barrier": 0, 2024-09-19T08:42:02.414 INFO:teuthology.orchestra.run.smithi139.stdout: "blocklisted": false, 2024-09-19T08:42:02.414 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs" 2024-09-19T08:42:02.414 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-19T08:42:02.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:02.424 DEBUG:teuthology.orchestra.run.smithi139:> stat --file-system '--printf=%T 2024-09-19T08:42:02.425 DEBUG:teuthology.orchestra.run.smithi139:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-19T08:42:02.430 INFO:teuthology.orchestra.run.smithi139.stdout:fuseblk 2024-09-19T08:42:02.431 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-19T08:42:02.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:02.431 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-19T08:42:02.486 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-19T08:42:02.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.622+0000 7f06a22dd640 1 Processor -- start 2024-09-19T08:42:02.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.622+0000 7f06a22dd640 1 -- start start 2024-09-19T08:42:02.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.622+0000 7f06a22dd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f069c15e720 0x7f069c0cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:02.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.622+0000 7f06a22dd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f069c0cfb50 0x7f069c0b7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:02.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.622+0000 7f06a22dd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f069c0b7cb0 0x7f069c0d3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:02.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.622+0000 7f06a22dd640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f069c0d29c0 con 0x7f069c0b7cb0 2024-09-19T08:42:02.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.622+0000 7f06a22dd640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f069c0d2ca0 con 0x7f069c0cfb50 2024-09-19T08:42:02.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.622+0000 7f06a22dd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f069c0d2f80 con 0x7f069c15e720 2024-09-19T08:42:02.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.626+0000 7f06a087a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f069c0b7cb0 0x7f069c0d3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:02.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.626+0000 7f069bfff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f069c15e720 0x7f069c0cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:02.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.626+0000 7f06a087a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f069c0b7cb0 0x7f069c0d3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:43926/0 (socket says 172.21.15.98:43926) 2024-09-19T08:42:02.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.626+0000 7f06a087a640 1 -- 172.21.15.98:0/1151976394 learned_addr learned my addr 172.21.15.98:0/1151976394 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:02.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.626+0000 7f069b7fe640 1 --2- 172.21.15.98:0/1151976394 >> v2:172.21.15.139:3300/0 conn(0x7f069c0cfb50 0x7f069c0b7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:02.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.626+0000 7f06a087a640 1 -- 172.21.15.98:0/1151976394 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f069c15e720 msgr2=0x7f069c0cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:02.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.626+0000 7f06a087a640 1 --2- 172.21.15.98:0/1151976394 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f069c15e720 0x7f069c0cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:02.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.626+0000 7f06a087a640 1 -- 172.21.15.98:0/1151976394 >> v2:172.21.15.139:3300/0 conn(0x7f069c0cfb50 msgr2=0x7f069c0b7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:02.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.626+0000 7f06a087a640 1 --2- 172.21.15.98:0/1151976394 >> v2:172.21.15.139:3300/0 conn(0x7f069c0cfb50 0x7f069c0b7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:02.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.626+0000 7f06a087a640 1 -- 172.21.15.98:0/1151976394 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f069c0d3260 con 0x7f069c0b7cb0 2024-09-19T08:42:02.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.626+0000 7f069b7fe640 1 --2- 172.21.15.98:0/1151976394 >> v2:172.21.15.139:3300/0 conn(0x7f069c0cfb50 0x7f069c0b7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:42:02.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.626+0000 7f06a087a640 1 --2- 172.21.15.98:0/1151976394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f069c0b7cb0 0x7f069c0d3a60 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f069c0b7230 tx=0x7f069401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f6673dde0e97037 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:02.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.626+0000 7f069affd640 1 -- 172.21.15.98:0/1151976394 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06940a5670 con 0x7f069c0b7cb0 2024-09-19T08:42:02.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.626+0000 7f069affd640 1 -- 172.21.15.98:0/1151976394 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06940a5810 con 0x7f069c0b7cb0 2024-09-19T08:42:02.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.626+0000 7f069affd640 1 -- 172.21.15.98:0/1151976394 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06940a5be0 con 0x7f069c0b7cb0 2024-09-19T08:42:02.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.626+0000 7f06a22dd640 1 -- 172.21.15.98:0/1151976394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f069c0b7cb0 msgr2=0x7f069c0d3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:02.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.626+0000 7f06a22dd640 1 --2- 172.21.15.98:0/1151976394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f069c0b7cb0 0x7f069c0d3a60 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f069c0b7230 tx=0x7f069401c9f0 comp rx=0 tx=0).stop 2024-09-19T08:42:02.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.626+0000 7f06a22dd640 1 -- 172.21.15.98:0/1151976394 shutdown_connections 2024-09-19T08:42:02.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.626+0000 7f06a22dd640 1 --2- 172.21.15.98:0/1151976394 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f069c0b7cb0 0x7f069c0d3a60 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:02.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.626+0000 7f06a22dd640 1 --2- 172.21.15.98:0/1151976394 >> v2:172.21.15.139:3300/0 conn(0x7f069c0cfb50 0x7f069c0b7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:02.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.626+0000 7f06a22dd640 1 --2- 172.21.15.98:0/1151976394 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f069c15e720 0x7f069c0cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:02.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.626+0000 7f06a22dd640 1 -- 172.21.15.98:0/1151976394 >> 172.21.15.98:0/1151976394 conn(0x7f069c0d4640 msgr2=0x7f069c0d4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:02.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.626+0000 7f06a22dd640 1 -- 172.21.15.98:0/1151976394 shutdown_connections 2024-09-19T08:42:02.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.626+0000 7f06a22dd640 1 -- 172.21.15.98:0/1151976394 wait complete. 2024-09-19T08:42:02.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.630+0000 7f06a22dd640 1 Processor -- start 2024-09-19T08:42:02.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.630+0000 7f06a22dd640 1 -- start start 2024-09-19T08:42:02.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.630+0000 7f06a22dd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f069c0b7cb0 0x7f069c156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:02.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.630+0000 7f06a22dd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f069c0cfb50 0x7f069c1573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:02.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.630+0000 7f06a22dd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f069c15e720 0x7f069c1578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:02.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.630+0000 7f06a22dd640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f069c0722b0 con 0x7f069c0b7cb0 2024-09-19T08:42:02.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.630+0000 7f06a22dd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f069c15ed10 con 0x7f069c15e720 2024-09-19T08:42:02.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.630+0000 7f06a22dd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f069c072430 con 0x7f069c0cfb50 2024-09-19T08:42:02.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.630+0000 7f069bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f069c0b7cb0 0x7f069c156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:02.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.630+0000 7f069bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f069c0b7cb0 0x7f069c156ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:43938/0 (socket says 172.21.15.98:43938) 2024-09-19T08:42:02.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.630+0000 7f069bfff640 1 -- 172.21.15.98:0/3803124896 learned_addr learned my addr 172.21.15.98:0/3803124896 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:02.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.630+0000 7f069b7fe640 1 --2- 172.21.15.98:0/3803124896 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f069c0cfb50 0x7f069c1573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:02.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.630+0000 7f069bfff640 1 -- 172.21.15.98:0/3803124896 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f069c0cfb50 msgr2=0x7f069c1573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:02.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.630+0000 7f06a087a640 1 --2- 172.21.15.98:0/3803124896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f069c15e720 0x7f069c1578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:02.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.630+0000 7f069bfff640 1 --2- 172.21.15.98:0/3803124896 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f069c0cfb50 0x7f069c1573b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:02.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.630+0000 7f069bfff640 1 -- 172.21.15.98:0/3803124896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f069c15e720 msgr2=0x7f069c1578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:02.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.630+0000 7f069bfff640 1 --2- 172.21.15.98:0/3803124896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f069c15e720 0x7f069c1578f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:02.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.630+0000 7f069bfff640 1 -- 172.21.15.98:0/3803124896 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f069c15c9f0 con 0x7f069c0b7cb0 2024-09-19T08:42:02.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.630+0000 7f069bfff640 1 --2- 172.21.15.98:0/3803124896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f069c0b7cb0 0x7f069c156ad0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f068c00e8a0 tx=0x7f068c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:02.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.630+0000 7f0698ff9640 1 -- 172.21.15.98:0/3803124896 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c017070 con 0x7f069c0b7cb0 2024-09-19T08:42:02.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.630+0000 7f0698ff9640 1 -- 172.21.15.98:0/3803124896 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f068c00c940 con 0x7f069c0b7cb0 2024-09-19T08:42:02.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.630+0000 7f06a22dd640 1 -- 172.21.15.98:0/3803124896 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f069c160e80 con 0x7f069c0b7cb0 2024-09-19T08:42:02.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.630+0000 7f0698ff9640 1 -- 172.21.15.98:0/3803124896 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c00ccf0 con 0x7f069c0b7cb0 2024-09-19T08:42:02.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.630+0000 7f06a22dd640 1 -- 172.21.15.98:0/3803124896 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f069c161440 con 0x7f069c0b7cb0 2024-09-19T08:42:02.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.630+0000 7f06a22dd640 1 -- 172.21.15.98:0/3803124896 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f069c0b7770 con 0x7f069c0b7cb0 2024-09-19T08:42:02.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.634+0000 7f0698ff9640 1 -- 172.21.15.98:0/3803124896 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f068c01a070 con 0x7f069c0b7cb0 2024-09-19T08:42:02.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.634+0000 7f0698ff9640 1 --2- 172.21.15.98:0/3803124896 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0688067a70 0x7f0688069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:02.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.634+0000 7f0698ff9640 1 -- 172.21.15.98:0/3803124896 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f068c086190 con 0x7f069c0b7cb0 2024-09-19T08:42:02.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.634+0000 7f069b7fe640 1 --2- 172.21.15.98:0/3803124896 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0688067a70 0x7f0688069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:02.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.634+0000 7f069b7fe640 1 --2- 172.21.15.98:0/3803124896 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0688067a70 0x7f0688069f30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f069c08dc60 tx=0x7f0690008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:02.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.638+0000 7f0698ff9640 1 -- 172.21.15.98:0/3803124896 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f068c056ae0 con 0x7f069c0b7cb0 2024-09-19T08:42:02.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:02.838+0000 7f06a22dd640 1 -- 172.21.15.98:0/3803124896 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f069c1694c0 con 0x7f069c0b7cb0 2024-09-19T08:42:03.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.178+0000 7f0698ff9640 1 -- 172.21.15.98:0/3803124896 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v31) ==== 136+0+0 (secure 0 0 0) 0x7f068c0563e0 con 0x7f069c0b7cb0 2024-09-19T08:42:03.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.182+0000 7f06a22dd640 1 -- 172.21.15.98:0/3803124896 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0688067a70 msgr2=0x7f0688069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:03.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.182+0000 7f06a22dd640 1 --2- 172.21.15.98:0/3803124896 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0688067a70 0x7f0688069f30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f069c08dc60 tx=0x7f0690008040 comp rx=0 tx=0).stop 2024-09-19T08:42:03.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.182+0000 7f06a22dd640 1 -- 172.21.15.98:0/3803124896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f069c0b7cb0 msgr2=0x7f069c156ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:03.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.182+0000 7f06a22dd640 1 --2- 172.21.15.98:0/3803124896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f069c0b7cb0 0x7f069c156ad0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f068c00e8a0 tx=0x7f068c00eca0 comp rx=0 tx=0).stop 2024-09-19T08:42:03.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.182+0000 7f06a22dd640 1 -- 172.21.15.98:0/3803124896 shutdown_connections 2024-09-19T08:42:03.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.182+0000 7f06a22dd640 1 --2- 172.21.15.98:0/3803124896 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0688067a70 0x7f0688069f30 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:03.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.182+0000 7f06a22dd640 1 --2- 172.21.15.98:0/3803124896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f069c15e720 0x7f069c1578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:03.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.182+0000 7f06a22dd640 1 --2- 172.21.15.98:0/3803124896 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f069c0cfb50 0x7f069c1573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:03.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.182+0000 7f06a22dd640 1 --2- 172.21.15.98:0/3803124896 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f069c0b7cb0 0x7f069c156ad0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:03.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.182+0000 7f06a22dd640 1 -- 172.21.15.98:0/3803124896 >> 172.21.15.98:0/3803124896 conn(0x7f069c0d4640 msgr2=0x7f069c0b4080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:03.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.182+0000 7f06a22dd640 1 -- 172.21.15.98:0/3803124896 shutdown_connections 2024-09-19T08:42:03.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.182+0000 7f06a22dd640 1 -- 172.21.15.98:0/3803124896 wait complete. 2024-09-19T08:42:03.203 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-19T08:42:03.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.326+0000 7f2c52ca7640 1 Processor -- start 2024-09-19T08:42:03.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.326+0000 7f2c52ca7640 1 -- start start 2024-09-19T08:42:03.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.326+0000 7f2c52ca7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c4c0733b0 0x7f2c4c0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:03.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.326+0000 7f2c52ca7640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c4c06d310 con 0x7f2c4c15c550 2024-09-19T08:42:03.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.326+0000 7f2c52ca7640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c4c06d5f0 con 0x7f2c4c06c3a0 2024-09-19T08:42:03.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.326+0000 7f2c52ca7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2c4c06d8d0 con 0x7f2c4c0733b0 2024-09-19T08:42:03.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.326+0000 7f2c50a43640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f2c4c15c550 0x7f2c4c06bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56448/0 (socket says 172.21.15.98:56448) 2024-09-19T08:42:03.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.326+0000 7f2c50a43640 1 -- 172.21.15.98:0/3044434241 learned_addr learned my addr 172.21.15.98:0/3044434241 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:03.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.326+0000 7f2c51244640 1 --2- 172.21.15.98:0/3044434241 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c4c0733b0 0x7f2c4c0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:03.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.326+0000 7f2c4b7fe640 1 -- 172.21.15.98:0/3044434241 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 414856400 0 0) 0x7f2c4c06d310 con 0x7f2c4c15c550 2024-09-19T08:42:03.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.326+0000 7f2c4b7fe640 1 -- 172.21.15.98:0/3044434241 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c38003530 con 0x7f2c4c15c550 2024-09-19T08:42:03.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.326+0000 7f2c4b7fe640 1 -- 172.21.15.98:0/3044434241 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1320169616 0 0) 0x7f2c38003530 con 0x7f2c4c15c550 2024-09-19T08:42:03.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.326+0000 7f2c4b7fe640 1 -- 172.21.15.98:0/3044434241 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c4c0733b0 msgr2=0x7f2c4c0cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:03.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.326+0000 7f2c4b7fe640 1 --2- 172.21.15.98:0/3044434241 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c4c0733b0 0x7f2c4c0cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:03.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.326+0000 7f2c4b7fe640 1 -- 172.21.15.98:0/3044434241 >> v1:172.21.15.139:6789/0 conn(0x7f2c4c06c3a0 legacy=0x7f2c4c0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:03.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.330+0000 7f2c4b7fe640 1 -- 172.21.15.98:0/3044434241 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4c06dbb0 con 0x7f2c4c15c550 2024-09-19T08:42:03.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.330+0000 7f2c4b7fe640 1 -- 172.21.15.98:0/3044434241 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f2c3c001a00 con 0x7f2c4c15c550 2024-09-19T08:42:03.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.330+0000 7f2c4b7fe640 1 -- 172.21.15.98:0/3044434241 >> v1:172.21.15.98:6789/0 conn(0x7f2c4c15c550 legacy=0x7f2c4c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:03.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.330+0000 7f2c4b7fe640 1 --2- 172.21.15.98:0/3044434241 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c380039f0 0x7f2c38003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:03.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.330+0000 7f2c4b7fe640 1 --2- 172.21.15.98:0/3044434241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c380043a0 0x7f2c38004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:03.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.330+0000 7f2c4b7fe640 1 --2- 172.21.15.98:0/3044434241 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c4c0733b0 0x7f2c380050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:03.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.330+0000 7f2c4b7fe640 1 -- 172.21.15.98:0/3044434241 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4c06d8d0 con 0x7f2c380043a0 2024-09-19T08:42:03.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.330+0000 7f2c4b7fe640 1 -- 172.21.15.98:0/3044434241 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3800b9d0 con 0x7f2c380039f0 2024-09-19T08:42:03.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.330+0000 7f2c4b7fe640 1 -- 172.21.15.98:0/3044434241 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2c3800bbd0 con 0x7f2c4c0733b0 2024-09-19T08:42:03.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.330+0000 7f2c4bfff640 1 --2- 172.21.15.98:0/3044434241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c380043a0 0x7f2c38004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:03.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.330+0000 7f2c50a43640 1 --2- 172.21.15.98:0/3044434241 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c380039f0 0x7f2c38003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:03.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.330+0000 7f2c51244640 1 --2- 172.21.15.98:0/3044434241 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c4c0733b0 0x7f2c380050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:03.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.330+0000 7f2c4bfff640 1 -- 172.21.15.98:0/3044434241 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c4c0733b0 msgr2=0x7f2c380050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:03.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.330+0000 7f2c4bfff640 1 --2- 172.21.15.98:0/3044434241 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c4c0733b0 0x7f2c380050d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:03.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.330+0000 7f2c4bfff640 1 -- 172.21.15.98:0/3044434241 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c380039f0 msgr2=0x7f2c38003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:03.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.330+0000 7f2c4bfff640 1 --2- 172.21.15.98:0/3044434241 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c380039f0 0x7f2c38003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:03.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.330+0000 7f2c4bfff640 1 -- 172.21.15.98:0/3044434241 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c38003530 con 0x7f2c380043a0 2024-09-19T08:42:03.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.330+0000 7f2c51244640 1 --2- 172.21.15.98:0/3044434241 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c4c0733b0 0x7f2c380050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:42:03.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.330+0000 7f2c4bfff640 1 --2- 172.21.15.98:0/3044434241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c380043a0 0x7f2c38004870 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2c34001860 tx=0x7f2c3401ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5909d7efc6951bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:03.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.330+0000 7f2c4b7fe640 1 -- 172.21.15.98:0/3044434241 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c340b5070 con 0x7f2c380043a0 2024-09-19T08:42:03.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.330+0000 7f2c4b7fe640 1 -- 172.21.15.98:0/3044434241 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c340a8890 con 0x7f2c380043a0 2024-09-19T08:42:03.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.330+0000 7f2c4b7fe640 1 -- 172.21.15.98:0/3044434241 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34001460 con 0x7f2c380043a0 2024-09-19T08:42:03.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.330+0000 7f2c52ca7640 1 -- 172.21.15.98:0/3044434241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c380043a0 msgr2=0x7f2c38004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:03.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.330+0000 7f2c52ca7640 1 --2- 172.21.15.98:0/3044434241 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c380043a0 0x7f2c38004870 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2c34001860 tx=0x7f2c3401ec70 comp rx=0 tx=0).stop 2024-09-19T08:42:03.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.330+0000 7f2c51244640 1 -- 172.21.15.98:0/3044434241 reap_dead start 2024-09-19T08:42:03.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.330+0000 7f2c52ca7640 1 -- 172.21.15.98:0/3044434241 shutdown_connections 2024-09-19T08:42:03.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.330+0000 7f2c52ca7640 1 -- 172.21.15.98:0/3044434241 >> 172.21.15.98:0/3044434241 conn(0x7f2c4c0b2010 msgr2=0x7f2c4c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:03.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.330+0000 7f2c52ca7640 1 -- 172.21.15.98:0/3044434241 shutdown_connections 2024-09-19T08:42:03.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.330+0000 7f2c52ca7640 1 -- 172.21.15.98:0/3044434241 wait complete. 2024-09-19T08:42:03.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.334+0000 7f2c52ca7640 1 Processor -- start 2024-09-19T08:42:03.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.334+0000 7f2c52ca7640 1 -- start start 2024-09-19T08:42:03.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.334+0000 7f2c52ca7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c380043a0 0x7f2c4c1f3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:03.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.334+0000 7f2c52ca7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c4c152f50 0x7f2c4c145cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:03.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.334+0000 7f2c52ca7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c4c154040 0x7f2c4c155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:03.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.334+0000 7f2c52ca7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4c0cfc50 con 0x7f2c4c152f50 2024-09-19T08:42:03.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.334+0000 7f2c52ca7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2c4c0cfad0 con 0x7f2c380043a0 2024-09-19T08:42:03.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.334+0000 7f2c52ca7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2c4c0cfdd0 con 0x7f2c4c154040 2024-09-19T08:42:03.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.334+0000 7f2c4bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c4c152f50 0x7f2c4c145cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:03.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.334+0000 7f2c4bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c4c152f50 0x7f2c4c145cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:43958/0 (socket says 172.21.15.98:43958) 2024-09-19T08:42:03.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.334+0000 7f2c4bfff640 1 -- 172.21.15.98:0/4167395146 learned_addr learned my addr 172.21.15.98:0/4167395146 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:03.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.334+0000 7f2c51244640 1 --2- 172.21.15.98:0/4167395146 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c4c154040 0x7f2c4c155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:03.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.334+0000 7f2c50a43640 1 --2- 172.21.15.98:0/4167395146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c380043a0 0x7f2c4c1f3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:03.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.334+0000 7f2c4bfff640 1 -- 172.21.15.98:0/4167395146 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c4c154040 msgr2=0x7f2c4c155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:03.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.334+0000 7f2c4bfff640 1 --2- 172.21.15.98:0/4167395146 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c4c154040 0x7f2c4c155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:03.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.334+0000 7f2c4bfff640 1 -- 172.21.15.98:0/4167395146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c380043a0 msgr2=0x7f2c4c1f3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:03.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.334+0000 7f2c4bfff640 1 --2- 172.21.15.98:0/4167395146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c380043a0 0x7f2c4c1f3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:03.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.334+0000 7f2c4bfff640 1 -- 172.21.15.98:0/4167395146 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4c154c70 con 0x7f2c4c152f50 2024-09-19T08:42:03.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.334+0000 7f2c50a43640 1 --2- 172.21.15.98:0/4167395146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c380043a0 0x7f2c4c1f3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:42:03.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.334+0000 7f2c4bfff640 1 --2- 172.21.15.98:0/4167395146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c4c152f50 0x7f2c4c145cf0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f2c340b3040 tx=0x7f2c38005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:03.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.334+0000 7f2c497fa640 1 -- 172.21.15.98:0/4167395146 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c340b5040 con 0x7f2c4c152f50 2024-09-19T08:42:03.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.334+0000 7f2c497fa640 1 -- 172.21.15.98:0/4167395146 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c34005270 con 0x7f2c4c152f50 2024-09-19T08:42:03.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.334+0000 7f2c497fa640 1 -- 172.21.15.98:0/4167395146 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c340067e0 con 0x7f2c4c152f50 2024-09-19T08:42:03.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.334+0000 7f2c52ca7640 1 -- 172.21.15.98:0/4167395146 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c4c150230 con 0x7f2c4c152f50 2024-09-19T08:42:03.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.334+0000 7f2c52ca7640 1 -- 172.21.15.98:0/4167395146 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c4c154990 con 0x7f2c4c152f50 2024-09-19T08:42:03.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.334+0000 7f2c52ca7640 1 -- 172.21.15.98:0/4167395146 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c4c0cf3c0 con 0x7f2c4c152f50 2024-09-19T08:42:03.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.334+0000 7f2c497fa640 1 -- 172.21.15.98:0/4167395146 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f2c340b8070 con 0x7f2c4c152f50 2024-09-19T08:42:03.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.338+0000 7f2c497fa640 1 --2- 172.21.15.98:0/4167395146 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2c2c067aa0 0x7f2c2c069f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:03.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.338+0000 7f2c497fa640 1 -- 172.21.15.98:0/4167395146 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f2c34123710 con 0x7f2c4c152f50 2024-09-19T08:42:03.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.338+0000 7f2c50a43640 1 --2- 172.21.15.98:0/4167395146 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2c2c067aa0 0x7f2c2c069f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:03.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.338+0000 7f2c50a43640 1 --2- 172.21.15.98:0/4167395146 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2c2c067aa0 0x7f2c2c069f60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c005c40 tx=0x7f2c3c0017c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:03.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.342+0000 7f2c497fa640 1 -- 172.21.15.98:0/4167395146 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2c34018980 con 0x7f2c4c152f50 2024-09-19T08:42:03.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.526+0000 7f2c52ca7640 1 -- 172.21.15.98:0/4167395146 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f2c4c0ccb60 con 0x7f2c2c067aa0 2024-09-19T08:42:03.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.526+0000 7f2c497fa640 1 -- 172.21.15.98:0/4167395146 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f2c4c0ccb60 con 0x7f2c2c067aa0 2024-09-19T08:42:03.528 INFO:teuthology.orchestra.run.smithi098.stdout:[ 2024-09-19T08:42:03.528 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:42:03.528 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "cephfs" 2024-09-19T08:42:03.528 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:42:03.528 INFO:teuthology.orchestra.run.smithi098.stdout:] 2024-09-19T08:42:03.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.530+0000 7f2c52ca7640 1 -- 172.21.15.98:0/4167395146 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2c2c067aa0 msgr2=0x7f2c2c069f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:03.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.530+0000 7f2c52ca7640 1 --2- 172.21.15.98:0/4167395146 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2c2c067aa0 0x7f2c2c069f60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c005c40 tx=0x7f2c3c0017c0 comp rx=0 tx=0).stop 2024-09-19T08:42:03.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.530+0000 7f2c52ca7640 1 -- 172.21.15.98:0/4167395146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c4c152f50 msgr2=0x7f2c4c145cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:03.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.530+0000 7f2c52ca7640 1 --2- 172.21.15.98:0/4167395146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c4c152f50 0x7f2c4c145cf0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f2c340b3040 tx=0x7f2c38005840 comp rx=0 tx=0).stop 2024-09-19T08:42:03.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.530+0000 7f2c52ca7640 1 -- 172.21.15.98:0/4167395146 shutdown_connections 2024-09-19T08:42:03.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.530+0000 7f2c52ca7640 1 --2- 172.21.15.98:0/4167395146 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2c2c067aa0 0x7f2c2c069f60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:03.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.530+0000 7f2c52ca7640 1 --2- 172.21.15.98:0/4167395146 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c4c154040 0x7f2c4c155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:03.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.530+0000 7f2c52ca7640 1 --2- 172.21.15.98:0/4167395146 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c4c152f50 0x7f2c4c145cf0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:03.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.530+0000 7f2c52ca7640 1 --2- 172.21.15.98:0/4167395146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c380043a0 0x7f2c4c1f3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:03.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.530+0000 7f2c52ca7640 1 -- 172.21.15.98:0/4167395146 >> 172.21.15.98:0/4167395146 conn(0x7f2c4c0b2010 msgr2=0x7f2c4c0d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:03.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.530+0000 7f2c52ca7640 1 -- 172.21.15.98:0/4167395146 shutdown_connections 2024-09-19T08:42:03.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.530+0000 7f2c52ca7640 1 -- 172.21.15.98:0/4167395146 wait complete. 2024-09-19T08:42:03.547 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T08:42:03.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.670+0000 7f39ef728640 1 Processor -- start 2024-09-19T08:42:03.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.670+0000 7f39ef728640 1 -- start start 2024-09-19T08:42:03.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.674+0000 7f39ef728640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f39e806c3a0 0x7f39e80713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:03.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.674+0000 7f39ef728640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39e8073530 con 0x7f39e80718f0 2024-09-19T08:42:03.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.674+0000 7f39ef728640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39e8073810 con 0x7f39e8145b70 2024-09-19T08:42:03.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.674+0000 7f39ef728640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f39e8073af0 con 0x7f39e806c3a0 2024-09-19T08:42:03.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.674+0000 7f39edcc5640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f39e80718f0 0x7f39e80b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56460/0 (socket says 172.21.15.98:56460) 2024-09-19T08:42:03.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.674+0000 7f39edcc5640 1 -- 172.21.15.98:0/2532676401 learned_addr learned my addr 172.21.15.98:0/2532676401 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:03.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.674+0000 7f39eccc3640 1 --2- 172.21.15.98:0/2532676401 >> v2:172.21.15.139:3300/0 conn(0x7f39e806c3a0 0x7f39e80713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:03.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.674+0000 7f39dffff640 1 -- 172.21.15.98:0/2532676401 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4048444670 0 0) 0x7f39e8073530 con 0x7f39e80718f0 2024-09-19T08:42:03.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.674+0000 7f39dffff640 1 -- 172.21.15.98:0/2532676401 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39d4003530 con 0x7f39e80718f0 2024-09-19T08:42:03.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.674+0000 7f39dffff640 1 -- 172.21.15.98:0/2532676401 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 887366063 0 0) 0x7f39d4003530 con 0x7f39e80718f0 2024-09-19T08:42:03.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.674+0000 7f39dffff640 1 -- 172.21.15.98:0/2532676401 >> v2:172.21.15.139:3300/0 conn(0x7f39e806c3a0 msgr2=0x7f39e80713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:03.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.674+0000 7f39dffff640 1 --2- 172.21.15.98:0/2532676401 >> v2:172.21.15.139:3300/0 conn(0x7f39e806c3a0 0x7f39e80713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:03.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.674+0000 7f39dffff640 1 -- 172.21.15.98:0/2532676401 >> v1:172.21.15.139:6789/0 conn(0x7f39e8145b70 legacy=0x7f39e806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:03.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.674+0000 7f39dffff640 1 -- 172.21.15.98:0/2532676401 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e8073dd0 con 0x7f39e80718f0 2024-09-19T08:42:03.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.674+0000 7f39dffff640 1 -- 172.21.15.98:0/2532676401 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f39e0001a00 con 0x7f39e80718f0 2024-09-19T08:42:03.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.674+0000 7f39dffff640 1 -- 172.21.15.98:0/2532676401 >> v1:172.21.15.98:6789/0 conn(0x7f39e80718f0 legacy=0x7f39e80b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:03.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.674+0000 7f39dffff640 1 --2- 172.21.15.98:0/2532676401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39d40039c0 0x7f39d4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:03.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.674+0000 7f39dffff640 1 --2- 172.21.15.98:0/2532676401 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39d4004810 0x7f39d4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:03.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.674+0000 7f39dffff640 1 --2- 172.21.15.98:0/2532676401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39d40051d0 0x7f39d400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:03.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.674+0000 7f39dffff640 1 -- 172.21.15.98:0/2532676401 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f39e8073af0 con 0x7f39d40051d0 2024-09-19T08:42:03.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.674+0000 7f39dffff640 1 -- 172.21.15.98:0/2532676401 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f39d400c170 con 0x7f39d40039c0 2024-09-19T08:42:03.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.674+0000 7f39dffff640 1 -- 172.21.15.98:0/2532676401 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f39d400c370 con 0x7f39d4004810 2024-09-19T08:42:03.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.674+0000 7f39edcc5640 1 --2- 172.21.15.98:0/2532676401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39d40051d0 0x7f39d400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:03.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.674+0000 7f39ed4c4640 1 --2- 172.21.15.98:0/2532676401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39d40039c0 0x7f39d4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:03.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.674+0000 7f39eccc3640 1 --2- 172.21.15.98:0/2532676401 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39d4004810 0x7f39d4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:03.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.674+0000 7f39edcc5640 1 -- 172.21.15.98:0/2532676401 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39d4004810 msgr2=0x7f39d4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:03.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.674+0000 7f39edcc5640 1 --2- 172.21.15.98:0/2532676401 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39d4004810 0x7f39d4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:03.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.674+0000 7f39edcc5640 1 -- 172.21.15.98:0/2532676401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39d40039c0 msgr2=0x7f39d4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:03.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.674+0000 7f39edcc5640 1 --2- 172.21.15.98:0/2532676401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39d40039c0 0x7f39d4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:03.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.674+0000 7f39edcc5640 1 -- 172.21.15.98:0/2532676401 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d4003530 con 0x7f39d40051d0 2024-09-19T08:42:03.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.678+0000 7f39edcc5640 1 -- 172.21.15.98:0/2532676401 reap_dead start 2024-09-19T08:42:03.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.678+0000 7f39edcc5640 1 --2- 172.21.15.98:0/2532676401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39d40051d0 0x7f39d400b9d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f39e000a860 tx=0x7f39e001fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=448899b3b6249548 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:03.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.678+0000 7f39dffff640 1 -- 172.21.15.98:0/2532676401 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e000a480 con 0x7f39d40051d0 2024-09-19T08:42:03.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.678+0000 7f39dffff640 1 -- 172.21.15.98:0/2532676401 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39e00aa510 con 0x7f39d40051d0 2024-09-19T08:42:03.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.678+0000 7f39dffff640 1 -- 172.21.15.98:0/2532676401 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e00aa900 con 0x7f39d40051d0 2024-09-19T08:42:03.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.678+0000 7f39ef728640 1 -- 172.21.15.98:0/2532676401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39d40051d0 msgr2=0x7f39d400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:03.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.678+0000 7f39ef728640 1 --2- 172.21.15.98:0/2532676401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39d40051d0 0x7f39d400b9d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f39e000a860 tx=0x7f39e001fae0 comp rx=0 tx=0).stop 2024-09-19T08:42:03.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.678+0000 7f39ef728640 1 -- 172.21.15.98:0/2532676401 shutdown_connections 2024-09-19T08:42:03.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.678+0000 7f39ef728640 1 --2- 172.21.15.98:0/2532676401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39d40051d0 0x7f39d400b9d0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:03.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.678+0000 7f39ef728640 1 -- 172.21.15.98:0/2532676401 >> 172.21.15.98:0/2532676401 conn(0x7f39e80b2010 msgr2=0x7f39e80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:03.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.678+0000 7f39ef728640 1 -- 172.21.15.98:0/2532676401 shutdown_connections 2024-09-19T08:42:03.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.678+0000 7f39ef728640 1 -- 172.21.15.98:0/2532676401 wait complete. 2024-09-19T08:42:03.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.678+0000 7f39ef728640 1 Processor -- start 2024-09-19T08:42:03.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.678+0000 7f39ef728640 1 -- start start 2024-09-19T08:42:03.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.678+0000 7f39ef728640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39d40051d0 0x7f39e8159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:03.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.678+0000 7f39ef728640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39e80adb30 0x7f39e80adf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:03.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.678+0000 7f39ef728640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39e815f8a0 0x7f39e815fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:03.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.678+0000 7f39ef728640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f39e8071ec0 con 0x7f39e815f8a0 2024-09-19T08:42:03.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.678+0000 7f39ef728640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f39e8071d40 con 0x7f39d40051d0 2024-09-19T08:42:03.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.678+0000 7f39ef728640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f39e8145f50 con 0x7f39e80adb30 2024-09-19T08:42:03.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.678+0000 7f39edcc5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39e815f8a0 0x7f39e815fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:03.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.678+0000 7f39edcc5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39e815f8a0 0x7f39e815fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:43984/0 (socket says 172.21.15.98:43984) 2024-09-19T08:42:03.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.678+0000 7f39edcc5640 1 -- 172.21.15.98:0/1463947412 learned_addr learned my addr 172.21.15.98:0/1463947412 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:03.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.678+0000 7f39ed4c4640 1 --2- 172.21.15.98:0/1463947412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39d40051d0 0x7f39e8159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:03.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.678+0000 7f39eccc3640 1 --2- 172.21.15.98:0/1463947412 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39e80adb30 0x7f39e80adf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:03.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.678+0000 7f39edcc5640 1 -- 172.21.15.98:0/1463947412 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39e80adb30 msgr2=0x7f39e80adf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:03.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.678+0000 7f39edcc5640 1 --2- 172.21.15.98:0/1463947412 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39e80adb30 0x7f39e80adf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:03.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.678+0000 7f39edcc5640 1 -- 172.21.15.98:0/1463947412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39d40051d0 msgr2=0x7f39e8159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:03.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.678+0000 7f39edcc5640 1 --2- 172.21.15.98:0/1463947412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39d40051d0 0x7f39e8159db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:03.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.678+0000 7f39edcc5640 1 -- 172.21.15.98:0/1463947412 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e80ab320 con 0x7f39e815f8a0 2024-09-19T08:42:03.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.678+0000 7f39ed4c4640 1 --2- 172.21.15.98:0/1463947412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39d40051d0 0x7f39e8159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:42:03.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.678+0000 7f39edcc5640 1 --2- 172.21.15.98:0/1463947412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39e815f8a0 0x7f39e815fd20 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f39e001f910 tx=0x7f39d4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:03.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.678+0000 7f39ddffb640 1 -- 172.21.15.98:0/1463947412 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e000a480 con 0x7f39e815f8a0 2024-09-19T08:42:03.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.682+0000 7f39ddffb640 1 -- 172.21.15.98:0/1463947412 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39e00bc070 con 0x7f39e815f8a0 2024-09-19T08:42:03.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.682+0000 7f39ef728640 1 -- 172.21.15.98:0/1463947412 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39e80ab600 con 0x7f39e815f8a0 2024-09-19T08:42:03.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.682+0000 7f39ddffb640 1 -- 172.21.15.98:0/1463947412 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e00aa8e0 con 0x7f39e815f8a0 2024-09-19T08:42:03.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.682+0000 7f39ef728640 1 -- 172.21.15.98:0/1463947412 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39e815a2f0 con 0x7f39e815f8a0 2024-09-19T08:42:03.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.682+0000 7f39ef728640 1 -- 172.21.15.98:0/1463947412 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39e80713b0 con 0x7f39e815f8a0 2024-09-19T08:42:03.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.682+0000 7f39ddffb640 1 -- 172.21.15.98:0/1463947412 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7f39e00b3040 con 0x7f39e815f8a0 2024-09-19T08:42:03.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.682+0000 7f39ddffb640 1 --2- 172.21.15.98:0/1463947412 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f39cc067a70 0x7f39cc069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:03.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.682+0000 7f39ddffb640 1 -- 172.21.15.98:0/1463947412 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f39e0123f40 con 0x7f39e815f8a0 2024-09-19T08:42:03.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.682+0000 7f39ed4c4640 1 --2- 172.21.15.98:0/1463947412 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f39cc067a70 0x7f39cc069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:03.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.682+0000 7f39ed4c4640 1 --2- 172.21.15.98:0/1463947412 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f39cc067a70 0x7f39cc069f30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f39e4000c10 tx=0x7f39e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:03.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.686+0000 7f39ddffb640 1 -- 172.21.15.98:0/1463947412 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f39e00197f0 con 0x7f39e815f8a0 2024-09-19T08:42:03.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.846+0000 7f39ef728640 1 -- 172.21.15.98:0/1463947412 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f39e80b4690 con 0x7f39e815f8a0 2024-09-19T08:42:03.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.846+0000 7f39ddffb640 1 -- 172.21.15.98:0/1463947412 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7f39e00f47d0 con 0x7f39e815f8a0 2024-09-19T08:42:03.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.850+0000 7f39ef728640 1 -- 172.21.15.98:0/1463947412 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f39cc067a70 msgr2=0x7f39cc069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:03.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.850+0000 7f39ef728640 1 --2- 172.21.15.98:0/1463947412 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f39cc067a70 0x7f39cc069f30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f39e4000c10 tx=0x7f39e4009040 comp rx=0 tx=0).stop 2024-09-19T08:42:03.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.850+0000 7f39ef728640 1 -- 172.21.15.98:0/1463947412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39e815f8a0 msgr2=0x7f39e815fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:03.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.850+0000 7f39ef728640 1 --2- 172.21.15.98:0/1463947412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39e815f8a0 0x7f39e815fd20 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f39e001f910 tx=0x7f39d4005840 comp rx=0 tx=0).stop 2024-09-19T08:42:03.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.850+0000 7f39ef728640 1 -- 172.21.15.98:0/1463947412 shutdown_connections 2024-09-19T08:42:03.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.850+0000 7f39ef728640 1 --2- 172.21.15.98:0/1463947412 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f39cc067a70 0x7f39cc069f30 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:03.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.850+0000 7f39ef728640 1 --2- 172.21.15.98:0/1463947412 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39e815f8a0 0x7f39e815fd20 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:03.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.850+0000 7f39ef728640 1 --2- 172.21.15.98:0/1463947412 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f39e80adb30 0x7f39e80adf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:03.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.850+0000 7f39ef728640 1 --2- 172.21.15.98:0/1463947412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39d40051d0 0x7f39e8159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:03.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.850+0000 7f39ef728640 1 -- 172.21.15.98:0/1463947412 >> 172.21.15.98:0/1463947412 conn(0x7f39e80b2010 msgr2=0x7f39e80b4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:03.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.850+0000 7f39ef728640 1 -- 172.21.15.98:0/1463947412 shutdown_connections 2024-09-19T08:42:03.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:03.850+0000 7f39ef728640 1 -- 172.21.15.98:0/1463947412 wait complete. 2024-09-19T08:42:03.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:03.867 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/_nogroup/subvol98020) 2024-09-19T08:42:03.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:03.894 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-19T08:42:03.944 INFO:teuthology.orchestra.run.smithi098.stdout:cephfs_data 2024-09-19T08:42:03.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:03.944 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/_nogroup/subvol98020) 2024-09-19T08:42:04.009 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol27721 2024-09-19T08:42:04.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.182+0000 7feec3141640 1 Processor -- start 2024-09-19T08:42:04.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.182+0000 7feec3141640 1 -- start start 2024-09-19T08:42:04.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.186+0000 7feec3141640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feebc145b70 0x7feebc06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:04.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.186+0000 7feec3141640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7feebc071ac0 0x7feebc0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:04.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.186+0000 7feec3141640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feebc06c5f0 con 0x7feebc06c1d0 2024-09-19T08:42:04.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.186+0000 7feec3141640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7feebc071f20 con 0x7feebc071ac0 2024-09-19T08:42:04.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.186+0000 7feec3141640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7feebc073530 con 0x7feebc145b70 2024-09-19T08:42:04.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.186+0000 7feec0edd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feebc145b70 0x7feebc06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:04.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.186+0000 7feec0edd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feebc145b70 0x7feebc06bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:34476/0 (socket says 172.21.15.98:34476) 2024-09-19T08:42:04.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.186+0000 7feec0edd640 1 -- 172.21.15.98:0/2933680517 learned_addr learned my addr 172.21.15.98:0/2933680517 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:04.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.186+0000 7feec16de640 1 --2- 172.21.15.98:0/2933680517 >> v2:172.21.15.139:3300/0 conn(0x7feebc071ac0 0x7feebc0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:04.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.186+0000 7feec0edd640 1 -- 172.21.15.98:0/2933680517 >> v2:172.21.15.139:3300/0 conn(0x7feebc071ac0 msgr2=0x7feebc0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:04.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.186+0000 7feec0edd640 1 --2- 172.21.15.98:0/2933680517 >> v2:172.21.15.139:3300/0 conn(0x7feebc071ac0 0x7feebc0b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:04.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.186+0000 7feec0edd640 1 -- 172.21.15.98:0/2933680517 >> v1:172.21.15.139:6789/0 conn(0x7feebc06c1d0 legacy=0x7feebc0713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:04.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.186+0000 7feec0edd640 1 -- 172.21.15.98:0/2933680517 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feebc073810 con 0x7feebc145b70 2024-09-19T08:42:04.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.190+0000 7feec0edd640 1 --2- 172.21.15.98:0/2933680517 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feebc145b70 0x7feebc06bc90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feeac009870 tx=0x7feeac01c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b8ee59cb6ce24b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:04.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.194+0000 7feebb7fe640 1 -- 172.21.15.98:0/2933680517 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac0b0070 con 0x7feebc145b70 2024-09-19T08:42:04.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.194+0000 7feebb7fe640 1 -- 172.21.15.98:0/2933680517 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeac0ab600 con 0x7feebc145b70 2024-09-19T08:42:04.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.194+0000 7feebb7fe640 1 -- 172.21.15.98:0/2933680517 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac0ab9d0 con 0x7feebc145b70 2024-09-19T08:42:04.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.194+0000 7feec3141640 1 -- 172.21.15.98:0/2933680517 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feebc145b70 msgr2=0x7feebc06bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:04.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.194+0000 7feec3141640 1 --2- 172.21.15.98:0/2933680517 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feebc145b70 0x7feebc06bc90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feeac009870 tx=0x7feeac01c4d0 comp rx=0 tx=0).stop 2024-09-19T08:42:04.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.194+0000 7feec3141640 1 -- 172.21.15.98:0/2933680517 shutdown_connections 2024-09-19T08:42:04.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.194+0000 7feec3141640 1 --2- 172.21.15.98:0/2933680517 >> v2:172.21.15.139:3300/0 conn(0x7feebc071ac0 0x7feebc0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:04.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.194+0000 7feec3141640 1 --2- 172.21.15.98:0/2933680517 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feebc145b70 0x7feebc06bc90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:04.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.194+0000 7feec3141640 1 -- 172.21.15.98:0/2933680517 >> 172.21.15.98:0/2933680517 conn(0x7feebc0b2010 msgr2=0x7feebc0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:04.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.194+0000 7feec3141640 1 -- 172.21.15.98:0/2933680517 shutdown_connections 2024-09-19T08:42:04.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.194+0000 7feec3141640 1 -- 172.21.15.98:0/2933680517 wait complete. 2024-09-19T08:42:04.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.194+0000 7feec3141640 1 Processor -- start 2024-09-19T08:42:04.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.194+0000 7feec3141640 1 -- start start 2024-09-19T08:42:04.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.194+0000 7feec3141640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feebc06c1d0 0x7feebc166490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:04.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.194+0000 7feec3141640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feebc071ac0 0x7feebc0aa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:04.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.194+0000 7feec3141640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feebc145b70 0x7feebc13b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:04.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.194+0000 7feec3141640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7feebc0b42e0 con 0x7feebc06c1d0 2024-09-19T08:42:04.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.194+0000 7feec3141640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feebc0b4160 con 0x7feebc071ac0 2024-09-19T08:42:04.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.194+0000 7feec3141640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7feebc0b4460 con 0x7feebc145b70 2024-09-19T08:42:04.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.194+0000 7feec0edd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feebc06c1d0 0x7feebc166490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:04.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.194+0000 7feec0edd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feebc06c1d0 0x7feebc166490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:43992/0 (socket says 172.21.15.98:43992) 2024-09-19T08:42:04.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.194+0000 7feec0edd640 1 -- 172.21.15.98:0/1999300451 learned_addr learned my addr 172.21.15.98:0/1999300451 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:04.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.198+0000 7feec16de640 1 --2- 172.21.15.98:0/1999300451 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feebc145b70 0x7feebc13b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:04.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.198+0000 7feebbfff640 1 --2- 172.21.15.98:0/1999300451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feebc071ac0 0x7feebc0aa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:04.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.198+0000 7feec0edd640 1 -- 172.21.15.98:0/1999300451 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feebc145b70 msgr2=0x7feebc13b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:04.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.198+0000 7feec0edd640 1 --2- 172.21.15.98:0/1999300451 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feebc145b70 0x7feebc13b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:04.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.198+0000 7feec0edd640 1 -- 172.21.15.98:0/1999300451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feebc071ac0 msgr2=0x7feebc0aa4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:04.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.198+0000 7feec0edd640 1 --2- 172.21.15.98:0/1999300451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feebc071ac0 0x7feebc0aa4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:04.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.198+0000 7feec0edd640 1 -- 172.21.15.98:0/1999300451 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feebc0aae80 con 0x7feebc06c1d0 2024-09-19T08:42:04.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.198+0000 7feec0edd640 1 --2- 172.21.15.98:0/1999300451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feebc06c1d0 0x7feebc166490 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7feeac0ae040 tx=0x7feeac019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:04.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.198+0000 7feeb97fa640 1 -- 172.21.15.98:0/1999300451 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac0b0050 con 0x7feebc06c1d0 2024-09-19T08:42:04.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.198+0000 7feeb97fa640 1 -- 172.21.15.98:0/1999300451 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeac0b1040 con 0x7feebc06c1d0 2024-09-19T08:42:04.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.198+0000 7feec3141640 1 -- 172.21.15.98:0/1999300451 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feebc0ab160 con 0x7feebc06c1d0 2024-09-19T08:42:04.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.198+0000 7feeb97fa640 1 -- 172.21.15.98:0/1999300451 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac0b5610 con 0x7feebc06c1d0 2024-09-19T08:42:04.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.198+0000 7feec3141640 1 -- 172.21.15.98:0/1999300451 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feebc0ab720 con 0x7feebc06c1d0 2024-09-19T08:42:04.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.198+0000 7feec3141640 1 -- 172.21.15.98:0/1999300451 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feebc0b3410 con 0x7feebc06c1d0 2024-09-19T08:42:04.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.198+0000 7feeb97fa640 1 -- 172.21.15.98:0/1999300451 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 9) ==== 87598+0+0 (secure 0 0 0) 0x7feeac0be020 con 0x7feebc06c1d0 2024-09-19T08:42:04.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.202+0000 7feeb97fa640 1 --2- 172.21.15.98:0/1999300451 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7feea8067a70 0x7feea8069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:04.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.202+0000 7feeb97fa640 1 -- 172.21.15.98:0/1999300451 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7feeac11fd00 con 0x7feebc06c1d0 2024-09-19T08:42:04.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.202+0000 7feebbfff640 1 --2- 172.21.15.98:0/1999300451 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7feea8067a70 0x7feea8069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:04.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.202+0000 7feebbfff640 1 --2- 172.21.15.98:0/1999300451 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7feea8067a70 0x7feea8069f30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7feebc0af510 tx=0x7feeb0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:04.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.206+0000 7feeb97fa640 1 -- 172.21.15.98:0/1999300451 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7feeac0c5020 con 0x7feebc06c1d0 2024-09-19T08:42:04.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.390+0000 7feec3141640 1 -- 172.21.15.98:0/1999300451 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol27721", "target": ["mon-mgr", ""]}) -- 0x7feebc1453a0 con 0x7feea8067a70 2024-09-19T08:42:04.398 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:42:04.398+0000 7f36e9c04640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:42:04.398 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:42:04.398+0000 7f36e9c04640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:42:04.398 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:42:04.398+0000 7f36e9c04640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:42:04.398 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:42:04.398+0000 7f36e9c04640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:42:04.398 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:42:04.398+0000 7f36e9c04640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:42:04.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.438+0000 7feeb97fa640 1 -- 172.21.15.98:0/1999300451 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7feebc1453a0 con 0x7feea8067a70 2024-09-19T08:42:04.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.442+0000 7feec3141640 1 -- 172.21.15.98:0/1999300451 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7feea8067a70 msgr2=0x7feea8069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:04.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.442+0000 7feec3141640 1 --2- 172.21.15.98:0/1999300451 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7feea8067a70 0x7feea8069f30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7feebc0af510 tx=0x7feeb0009040 comp rx=0 tx=0).stop 2024-09-19T08:42:04.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.442+0000 7feec3141640 1 -- 172.21.15.98:0/1999300451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feebc06c1d0 msgr2=0x7feebc166490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:04.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.442+0000 7feec3141640 1 --2- 172.21.15.98:0/1999300451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feebc06c1d0 0x7feebc166490 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7feeac0ae040 tx=0x7feeac019270 comp rx=0 tx=0).stop 2024-09-19T08:42:04.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.442+0000 7feec3141640 1 -- 172.21.15.98:0/1999300451 shutdown_connections 2024-09-19T08:42:04.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.442+0000 7feec3141640 1 --2- 172.21.15.98:0/1999300451 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7feea8067a70 0x7feea8069f30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:04.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.442+0000 7feec3141640 1 --2- 172.21.15.98:0/1999300451 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feebc145b70 0x7feebc13b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:04.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.442+0000 7feec3141640 1 --2- 172.21.15.98:0/1999300451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feebc071ac0 0x7feebc0aa4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:04.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.442+0000 7feec3141640 1 --2- 172.21.15.98:0/1999300451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feebc06c1d0 0x7feebc166490 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:04.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.442+0000 7feec3141640 1 -- 172.21.15.98:0/1999300451 >> 172.21.15.98:0/1999300451 conn(0x7feebc0b2010 msgr2=0x7feebc0aff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:04.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.442+0000 7feec3141640 1 -- 172.21.15.98:0/1999300451 shutdown_connections 2024-09-19T08:42:04.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:04.442+0000 7feec3141640 1 -- 172.21.15.98:0/1999300451 wait complete. 2024-09-19T08:42:04.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:04.460 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cp ./volumes/_nogroup/subvol27721/.meta ./volumes/_nogroup/subvol98020) 2024-09-19T08:42:07.638 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol98020 2024-09-19T08:42:07.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.770+0000 7f8060f1f640 1 Processor -- start 2024-09-19T08:42:07.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.770+0000 7f8060f1f640 1 -- start start 2024-09-19T08:42:07.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.770+0000 7f8060f1f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f805c0d27c0 0x7f805c0d3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:07.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.770+0000 7f8060f1f640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f805c0d38a0 0x7f805c0b8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:07.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.770+0000 7f8060f1f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f805c15eae0 con 0x7f805c1415a0 2024-09-19T08:42:07.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.770+0000 7f8060f1f640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f805c15edc0 con 0x7f805c0d38a0 2024-09-19T08:42:07.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.770+0000 7f8060f1f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f805c15f0a0 con 0x7f805c0d27c0 2024-09-19T08:42:07.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.770+0000 7f805ad9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f805c0d38a0 0x7f805c0b8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:07.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.770+0000 7f805ad9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f805c0d38a0 0x7f805c0b8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:45956/0 (socket says 172.21.15.98:45956) 2024-09-19T08:42:07.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.770+0000 7f805ad9d640 1 -- 172.21.15.98:0/1427592575 learned_addr learned my addr 172.21.15.98:0/1427592575 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:07.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.770+0000 7f8059d9b640 1 --2- 172.21.15.98:0/1427592575 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f805c0d27c0 0x7f805c0d3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:07.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.770+0000 7f805ad9d640 1 -- 172.21.15.98:0/1427592575 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f805c0d27c0 msgr2=0x7f805c0d3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:07.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.770+0000 7f805ad9d640 1 --2- 172.21.15.98:0/1427592575 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f805c0d27c0 0x7f805c0d3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:07.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.770+0000 7f805ad9d640 1 -- 172.21.15.98:0/1427592575 >> v1:172.21.15.139:6789/0 conn(0x7f805c1415a0 legacy=0x7f805c0d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:07.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.770+0000 7f805ad9d640 1 -- 172.21.15.98:0/1427592575 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f805c15f380 con 0x7f805c0d38a0 2024-09-19T08:42:07.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.770+0000 7f8059d9b640 1 --2- 172.21.15.98:0/1427592575 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f805c0d27c0 0x7f805c0d3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:42:07.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.774+0000 7f805ad9d640 1 --2- 172.21.15.98:0/1427592575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f805c0d38a0 0x7f805c0b8790 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f8054009870 tx=0x7f805401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e206f3e964d67655 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:07.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.774+0000 7f805959a640 1 -- 172.21.15.98:0/1427592575 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80540a7070 con 0x7f805c0d38a0 2024-09-19T08:42:07.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.774+0000 7f805959a640 1 -- 172.21.15.98:0/1427592575 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80540b0070 con 0x7f805c0d38a0 2024-09-19T08:42:07.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.774+0000 7f805959a640 1 -- 172.21.15.98:0/1427592575 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80540ab890 con 0x7f805c0d38a0 2024-09-19T08:42:07.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.774+0000 7f8060f1f640 1 -- 172.21.15.98:0/1427592575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f805c0d38a0 msgr2=0x7f805c0b8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:07.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.774+0000 7f8060f1f640 1 --2- 172.21.15.98:0/1427592575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f805c0d38a0 0x7f805c0b8790 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f8054009870 tx=0x7f805401c490 comp rx=0 tx=0).stop 2024-09-19T08:42:07.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.774+0000 7f8060f1f640 1 -- 172.21.15.98:0/1427592575 shutdown_connections 2024-09-19T08:42:07.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.774+0000 7f8060f1f640 1 --2- 172.21.15.98:0/1427592575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f805c0d38a0 0x7f805c0b8790 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:07.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.774+0000 7f8060f1f640 1 --2- 172.21.15.98:0/1427592575 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f805c0d27c0 0x7f805c0d3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:07.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.774+0000 7f8060f1f640 1 -- 172.21.15.98:0/1427592575 >> 172.21.15.98:0/1427592575 conn(0x7f805c074c70 msgr2=0x7f805c075080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:07.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.774+0000 7f8060f1f640 1 -- 172.21.15.98:0/1427592575 shutdown_connections 2024-09-19T08:42:07.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.774+0000 7f8060f1f640 1 -- 172.21.15.98:0/1427592575 wait complete. 2024-09-19T08:42:07.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.774+0000 7f8060f1f640 1 Processor -- start 2024-09-19T08:42:07.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.774+0000 7f8060f1f640 1 -- start start 2024-09-19T08:42:07.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.774+0000 7f8060f1f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f805c0d27c0 0x7f805c155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:07.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.774+0000 7f8060f1f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f805c0d38a0 0x7f805c13e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:07.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.774+0000 7f8060f1f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f805c1415a0 0x7f805c1571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:07.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.774+0000 7f8060f1f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f805c0b9660 con 0x7f805c1415a0 2024-09-19T08:42:07.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.774+0000 7f8060f1f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f805c0b94e0 con 0x7f805c0d38a0 2024-09-19T08:42:07.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.774+0000 7f8060f1f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f805c0b97e0 con 0x7f805c0d27c0 2024-09-19T08:42:07.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.774+0000 7f805ad9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f805c1415a0 0x7f805c1571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:07.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.774+0000 7f805ad9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f805c1415a0 0x7f805c1571e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:45970/0 (socket says 172.21.15.98:45970) 2024-09-19T08:42:07.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.774+0000 7f805ad9d640 1 -- 172.21.15.98:0/707158155 learned_addr learned my addr 172.21.15.98:0/707158155 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:07.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.774+0000 7f805a59c640 1 --2- 172.21.15.98:0/707158155 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f805c0d27c0 0x7f805c155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:07.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.774+0000 7f8059d9b640 1 --2- 172.21.15.98:0/707158155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f805c0d38a0 0x7f805c13e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:07.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.774+0000 7f805ad9d640 1 -- 172.21.15.98:0/707158155 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f805c0d27c0 msgr2=0x7f805c155af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:07.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.774+0000 7f805ad9d640 1 --2- 172.21.15.98:0/707158155 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f805c0d27c0 0x7f805c155af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:07.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.774+0000 7f805ad9d640 1 -- 172.21.15.98:0/707158155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f805c0d38a0 msgr2=0x7f805c13e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:07.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.778+0000 7f805ad9d640 1 --2- 172.21.15.98:0/707158155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f805c0d38a0 0x7f805c13e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:07.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.778+0000 7f805ad9d640 1 -- 172.21.15.98:0/707158155 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f805c162bd0 con 0x7f805c1415a0 2024-09-19T08:42:07.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.778+0000 7f805a59c640 1 --2- 172.21.15.98:0/707158155 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f805c0d27c0 0x7f805c155af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:42:07.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.778+0000 7f805ad9d640 1 --2- 172.21.15.98:0/707158155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f805c1415a0 0x7f805c1571e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f805401ab40 tx=0x7f805401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:07.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.778+0000 7f804affd640 1 -- 172.21.15.98:0/707158155 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80540b4800 con 0x7f805c1415a0 2024-09-19T08:42:07.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.778+0000 7f804affd640 1 -- 172.21.15.98:0/707158155 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80540b49c0 con 0x7f805c1415a0 2024-09-19T08:42:07.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.778+0000 7f8060f1f640 1 -- 172.21.15.98:0/707158155 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f805c156140 con 0x7f805c1415a0 2024-09-19T08:42:07.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.778+0000 7f804affd640 1 -- 172.21.15.98:0/707158155 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80540b4db0 con 0x7f805c1415a0 2024-09-19T08:42:07.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.778+0000 7f8060f1f640 1 -- 172.21.15.98:0/707158155 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f805c1569e0 con 0x7f805c1415a0 2024-09-19T08:42:07.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.778+0000 7f8060f1f640 1 -- 172.21.15.98:0/707158155 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f805c0d3360 con 0x7f805c1415a0 2024-09-19T08:42:07.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.778+0000 7f804affd640 1 -- 172.21.15.98:0/707158155 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 10) ==== 87649+0+0 (secure 0 0 0) 0x7f80540b3380 con 0x7f805c1415a0 2024-09-19T08:42:07.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.778+0000 7f804affd640 1 --2- 172.21.15.98:0/707158155 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f803c067b00 0x7f803c069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:07.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.778+0000 7f804affd640 1 -- 172.21.15.98:0/707158155 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f8054120760 con 0x7f805c1415a0 2024-09-19T08:42:07.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.778+0000 7f805a59c640 1 --2- 172.21.15.98:0/707158155 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f803c067b00 0x7f803c069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:07.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.778+0000 7f805a59c640 1 --2- 172.21.15.98:0/707158155 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f803c067b00 0x7f803c069fc0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f804c007fd0 tx=0x7f804c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:07.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.782+0000 7f804affd640 1 -- 172.21.15.98:0/707158155 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f80540c4060 con 0x7f805c1415a0 2024-09-19T08:42:07.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:07.974+0000 7f8060f1f640 1 -- 172.21.15.98:0/707158155 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol98020", "target": ["mon-mgr", ""]}) -- 0x7f805c1564d0 con 0x7f803c067b00 2024-09-19T08:42:08.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.018+0000 7f804affd640 1 -- 172.21.15.98:0/707158155 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7f805c1564d0 con 0x7f803c067b00 2024-09-19T08:42:08.017 INFO:teuthology.orchestra.run.smithi098.stdout:/volumes/_nogroup/subvol98020 2024-09-19T08:42:08.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.018+0000 7f8060f1f640 1 -- 172.21.15.98:0/707158155 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f803c067b00 msgr2=0x7f803c069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:08.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.018+0000 7f8060f1f640 1 --2- 172.21.15.98:0/707158155 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f803c067b00 0x7f803c069fc0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f804c007fd0 tx=0x7f804c009040 comp rx=0 tx=0).stop 2024-09-19T08:42:08.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.018+0000 7f8060f1f640 1 -- 172.21.15.98:0/707158155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f805c1415a0 msgr2=0x7f805c1571e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:08.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.018+0000 7f8060f1f640 1 --2- 172.21.15.98:0/707158155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f805c1415a0 0x7f805c1571e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f805401ab40 tx=0x7f805401acd0 comp rx=0 tx=0).stop 2024-09-19T08:42:08.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.018+0000 7f8060f1f640 1 -- 172.21.15.98:0/707158155 shutdown_connections 2024-09-19T08:42:08.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.018+0000 7f8060f1f640 1 --2- 172.21.15.98:0/707158155 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f803c067b00 0x7f803c069fc0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:08.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.018+0000 7f8060f1f640 1 --2- 172.21.15.98:0/707158155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f805c1415a0 0x7f805c1571e0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:08.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.018+0000 7f8060f1f640 1 --2- 172.21.15.98:0/707158155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f805c0d38a0 0x7f805c13e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:08.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.018+0000 7f8060f1f640 1 --2- 172.21.15.98:0/707158155 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f805c0d27c0 0x7f805c155af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:08.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.018+0000 7f8060f1f640 1 -- 172.21.15.98:0/707158155 >> 172.21.15.98:0/707158155 conn(0x7f805c074c70 msgr2=0x7f805c13fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:08.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.018+0000 7f8060f1f640 1 -- 172.21.15.98:0/707158155 shutdown_connections 2024-09-19T08:42:08.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.018+0000 7f8060f1f640 1 -- 172.21.15.98:0/707158155 wait complete. 2024-09-19T08:42:08.037 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol98020 2024-09-19T08:42:08.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.158+0000 7f3bea896640 1 Processor -- start 2024-09-19T08:42:08.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.158+0000 7f3bea896640 1 -- start start 2024-09-19T08:42:08.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.162+0000 7f3bea896640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3be415a8b0 0x7f3be40c5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:08.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.162+0000 7f3bea896640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3be4073700 0x7f3be40cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:08.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.162+0000 7f3bea896640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3be40cfc70 con 0x7f3be4073190 2024-09-19T08:42:08.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.162+0000 7f3bea896640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f3be4071c40 con 0x7f3be4073700 2024-09-19T08:42:08.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.162+0000 7f3bea896640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3be40cf6b0 con 0x7f3be415a8b0 2024-09-19T08:42:08.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.162+0000 7f3be8e33640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f3be4073190 0x7f3be40720d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:35914/0 (socket says 172.21.15.98:35914) 2024-09-19T08:42:08.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.162+0000 7f3be8e33640 1 -- 172.21.15.98:0/4266609288 learned_addr learned my addr 172.21.15.98:0/4266609288 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:08.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.162+0000 7f3be3fff640 1 --2- 172.21.15.98:0/4266609288 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3be415a8b0 0x7f3be40c5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:08.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.162+0000 7f3be37fe640 1 --2- 172.21.15.98:0/4266609288 >> v2:172.21.15.139:3300/0 conn(0x7f3be4073700 0x7f3be40cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:08.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.162+0000 7f3be2ffd640 1 -- 172.21.15.98:0/4266609288 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 844827078 0 0) 0x7f3be40cfc70 con 0x7f3be4073190 2024-09-19T08:42:08.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.162+0000 7f3be2ffd640 1 -- 172.21.15.98:0/4266609288 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3bcc003530 con 0x7f3be4073190 2024-09-19T08:42:08.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.162+0000 7f3be2ffd640 1 -- 172.21.15.98:0/4266609288 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3494199642 0 0) 0x7f3bcc003530 con 0x7f3be4073190 2024-09-19T08:42:08.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.162+0000 7f3be2ffd640 1 -- 172.21.15.98:0/4266609288 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3be415a8b0 msgr2=0x7f3be40c5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:08.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.162+0000 7f3be2ffd640 1 --2- 172.21.15.98:0/4266609288 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3be415a8b0 0x7f3be40c5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:08.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.162+0000 7f3be2ffd640 1 -- 172.21.15.98:0/4266609288 >> v2:172.21.15.139:3300/0 conn(0x7f3be4073700 msgr2=0x7f3be40cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:08.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.162+0000 7f3be2ffd640 1 --2- 172.21.15.98:0/4266609288 >> v2:172.21.15.139:3300/0 conn(0x7f3be4073700 0x7f3be40cb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:08.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.162+0000 7f3be2ffd640 1 -- 172.21.15.98:0/4266609288 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be40cf990 con 0x7f3be4073190 2024-09-19T08:42:08.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.162+0000 7f3be2ffd640 1 -- 172.21.15.98:0/4266609288 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f3bdc001a00 con 0x7f3be4073190 2024-09-19T08:42:08.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.162+0000 7f3be2ffd640 1 -- 172.21.15.98:0/4266609288 >> v1:172.21.15.98:6789/0 conn(0x7f3be4073190 legacy=0x7f3be40720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:08.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.162+0000 7f3be2ffd640 1 --2- 172.21.15.98:0/4266609288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bcc0039c0 0x7f3bcc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:08.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.162+0000 7f3be2ffd640 1 --2- 172.21.15.98:0/4266609288 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3be415a8b0 0x7f3bcc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:08.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.162+0000 7f3be2ffd640 1 --2- 172.21.15.98:0/4266609288 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bcc004d50 0x7f3bcc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:08.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.162+0000 7f3be2ffd640 1 -- 172.21.15.98:0/4266609288 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3be4071c40 con 0x7f3bcc004d50 2024-09-19T08:42:08.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.162+0000 7f3be2ffd640 1 -- 172.21.15.98:0/4266609288 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3be40cf6b0 con 0x7f3bcc0039c0 2024-09-19T08:42:08.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.162+0000 7f3be2ffd640 1 -- 172.21.15.98:0/4266609288 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3bcc00cae0 con 0x7f3be415a8b0 2024-09-19T08:42:08.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.162+0000 7f3be8e33640 1 --2- 172.21.15.98:0/4266609288 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bcc004d50 0x7f3bcc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:08.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.162+0000 7f3be37fe640 1 --2- 172.21.15.98:0/4266609288 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3be415a8b0 0x7f3bcc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:08.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.162+0000 7f3be8e33640 1 -- 172.21.15.98:0/4266609288 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3be415a8b0 msgr2=0x7f3bcc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:08.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.162+0000 7f3be8e33640 1 --2- 172.21.15.98:0/4266609288 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3be415a8b0 0x7f3bcc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:08.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.162+0000 7f3be8e33640 1 -- 172.21.15.98:0/4266609288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bcc0039c0 msgr2=0x7f3bcc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:08.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.162+0000 7f3be8e33640 1 --2- 172.21.15.98:0/4266609288 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bcc0039c0 0x7f3bcc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:08.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.162+0000 7f3be8e33640 1 -- 172.21.15.98:0/4266609288 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bcc003530 con 0x7f3bcc004d50 2024-09-19T08:42:08.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.162+0000 7f3be37fe640 1 --2- 172.21.15.98:0/4266609288 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3be415a8b0 0x7f3bcc004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:42:08.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.162+0000 7f3be8e33640 1 --2- 172.21.15.98:0/4266609288 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bcc004d50 0x7f3bcc0051d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00a880 tx=0x7f3bdc01d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f84cc87910e1fe4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:08.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.166+0000 7f3be2ffd640 1 -- 172.21.15.98:0/4266609288 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc00a4a0 con 0x7f3bcc004d50 2024-09-19T08:42:08.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.166+0000 7f3be2ffd640 1 -- 172.21.15.98:0/4266609288 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bdc0ad630 con 0x7f3bcc004d50 2024-09-19T08:42:08.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.166+0000 7f3be2ffd640 1 -- 172.21.15.98:0/4266609288 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc0ada20 con 0x7f3bcc004d50 2024-09-19T08:42:08.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.166+0000 7f3bea896640 1 -- 172.21.15.98:0/4266609288 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bcc004d50 msgr2=0x7f3bcc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:08.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.166+0000 7f3bea896640 1 --2- 172.21.15.98:0/4266609288 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bcc004d50 0x7f3bcc0051d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00a880 tx=0x7f3bdc01d3a0 comp rx=0 tx=0).stop 2024-09-19T08:42:08.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.166+0000 7f3be8e33640 1 -- 172.21.15.98:0/4266609288 reap_dead start 2024-09-19T08:42:08.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.166+0000 7f3bea896640 1 -- 172.21.15.98:0/4266609288 shutdown_connections 2024-09-19T08:42:08.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.166+0000 7f3bea896640 1 -- 172.21.15.98:0/4266609288 >> 172.21.15.98:0/4266609288 conn(0x7f3be40d0a50 msgr2=0x7f3be40d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:08.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.166+0000 7f3bea896640 1 -- 172.21.15.98:0/4266609288 shutdown_connections 2024-09-19T08:42:08.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.166+0000 7f3bea896640 1 -- 172.21.15.98:0/4266609288 wait complete. 2024-09-19T08:42:08.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.166+0000 7f3bea896640 1 Processor -- start 2024-09-19T08:42:08.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.166+0000 7f3bea896640 1 -- start start 2024-09-19T08:42:08.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.166+0000 7f3bea896640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3be40b6410 0x7f3be41e35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:08.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.166+0000 7f3bea896640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3be41e3ae0 0x7f3be41e0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:08.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.166+0000 7f3bea896640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3be41e1260 0x7f3be41e20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:08.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.166+0000 7f3bea896640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3be415ac90 con 0x7f3be41e1260 2024-09-19T08:42:08.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.166+0000 7f3bea896640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3be40cf340 con 0x7f3be40b6410 2024-09-19T08:42:08.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.166+0000 7f3bea896640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3be415ae10 con 0x7f3be41e3ae0 2024-09-19T08:42:08.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.166+0000 7f3be8e33640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3be41e1260 0x7f3be41e20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:08.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.166+0000 7f3be8e33640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3be41e1260 0x7f3be41e20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:45988/0 (socket says 172.21.15.98:45988) 2024-09-19T08:42:08.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.166+0000 7f3be8e33640 1 -- 172.21.15.98:0/3543082079 learned_addr learned my addr 172.21.15.98:0/3543082079 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:08.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.166+0000 7f3be3fff640 1 --2- 172.21.15.98:0/3543082079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3be40b6410 0x7f3be41e35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:08.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.166+0000 7f3be37fe640 1 --2- 172.21.15.98:0/3543082079 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3be41e3ae0 0x7f3be41e0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:08.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.166+0000 7f3be8e33640 1 -- 172.21.15.98:0/3543082079 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3be41e3ae0 msgr2=0x7f3be41e0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:08.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.166+0000 7f3be8e33640 1 --2- 172.21.15.98:0/3543082079 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3be41e3ae0 0x7f3be41e0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:08.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.166+0000 7f3be8e33640 1 -- 172.21.15.98:0/3543082079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3be40b6410 msgr2=0x7f3be41e35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:08.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.166+0000 7f3be8e33640 1 --2- 172.21.15.98:0/3543082079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3be40b6410 0x7f3be41e35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:08.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.166+0000 7f3be8e33640 1 -- 172.21.15.98:0/3543082079 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be40c8520 con 0x7f3be41e1260 2024-09-19T08:42:08.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.166+0000 7f3be37fe640 1 --2- 172.21.15.98:0/3543082079 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3be41e3ae0 0x7f3be41e0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:42:08.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.166+0000 7f3be3fff640 1 --2- 172.21.15.98:0/3543082079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3be40b6410 0x7f3be41e35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:42:08.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.166+0000 7f3be8e33640 1 --2- 172.21.15.98:0/3543082079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3be41e1260 0x7f3be41e20e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc098dc0 tx=0x7f3bcc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:08.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.166+0000 7f3be0ff9640 1 -- 172.21.15.98:0/3543082079 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc0ba070 con 0x7f3be41e1260 2024-09-19T08:42:08.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.166+0000 7f3be0ff9640 1 -- 172.21.15.98:0/3543082079 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bdc0019d0 con 0x7f3be41e1260 2024-09-19T08:42:08.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.166+0000 7f3bea896640 1 -- 172.21.15.98:0/3543082079 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3be41e40d0 con 0x7f3be41e1260 2024-09-19T08:42:08.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.166+0000 7f3be0ff9640 1 -- 172.21.15.98:0/3543082079 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc00a4a0 con 0x7f3be41e1260 2024-09-19T08:42:08.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.166+0000 7f3bea896640 1 -- 172.21.15.98:0/3543082079 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3be41e4690 con 0x7f3be41e1260 2024-09-19T08:42:08.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.170+0000 7f3bea896640 1 -- 172.21.15.98:0/3543082079 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3be41e16e0 con 0x7f3be41e1260 2024-09-19T08:42:08.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.170+0000 7f3be0ff9640 1 -- 172.21.15.98:0/3543082079 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 10) ==== 87649+0+0 (secure 0 0 0) 0x7f3bdc0bd070 con 0x7f3be41e1260 2024-09-19T08:42:08.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.170+0000 7f3be0ff9640 1 --2- 172.21.15.98:0/3543082079 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3bd0067a40 0x7f3bd0069f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:08.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.170+0000 7f3be0ff9640 1 -- 172.21.15.98:0/3543082079 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f3bdc11fbf0 con 0x7f3be41e1260 2024-09-19T08:42:08.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.170+0000 7f3be3fff640 1 --2- 172.21.15.98:0/3543082079 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3bd0067a40 0x7f3bd0069f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:08.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.170+0000 7f3be3fff640 1 --2- 172.21.15.98:0/3543082079 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3bd0067a40 0x7f3bd0069f00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3bd40047c0 tx=0x7f3bd4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:08.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.174+0000 7f3be0ff9640 1 -- 172.21.15.98:0/3543082079 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3bdc0170b0 con 0x7f3be41e1260 2024-09-19T08:42:08.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.362+0000 7f3bea896640 1 -- 172.21.15.98:0/3543082079 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol98020", "target": ["mon-mgr", ""]}) -- 0x7f3be40b19e0 con 0x7f3bd0067a40 2024-09-19T08:42:08.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.370+0000 7f3be0ff9640 1 -- 172.21.15.98:0/3543082079 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7f3be40b19e0 con 0x7f3bd0067a40 2024-09-19T08:42:08.371 INFO:teuthology.orchestra.run.smithi098.stdout:/volumes/_nogroup/subvol98020 2024-09-19T08:42:08.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.374+0000 7f3bea896640 1 -- 172.21.15.98:0/3543082079 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3bd0067a40 msgr2=0x7f3bd0069f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:08.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.374+0000 7f3bea896640 1 --2- 172.21.15.98:0/3543082079 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3bd0067a40 0x7f3bd0069f00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3bd40047c0 tx=0x7f3bd4008040 comp rx=0 tx=0).stop 2024-09-19T08:42:08.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.374+0000 7f3bea896640 1 -- 172.21.15.98:0/3543082079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3be41e1260 msgr2=0x7f3be41e20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:08.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.374+0000 7f3bea896640 1 --2- 172.21.15.98:0/3543082079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3be41e1260 0x7f3be41e20e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc098dc0 tx=0x7f3bcc005840 comp rx=0 tx=0).stop 2024-09-19T08:42:08.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.374+0000 7f3bea896640 1 -- 172.21.15.98:0/3543082079 shutdown_connections 2024-09-19T08:42:08.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.374+0000 7f3bea896640 1 --2- 172.21.15.98:0/3543082079 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3bd0067a40 0x7f3bd0069f00 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:08.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.374+0000 7f3bea896640 1 --2- 172.21.15.98:0/3543082079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3be41e1260 0x7f3be41e20e0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:08.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.374+0000 7f3bea896640 1 --2- 172.21.15.98:0/3543082079 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3be41e3ae0 0x7f3be41e0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:08.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.374+0000 7f3bea896640 1 --2- 172.21.15.98:0/3543082079 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3be40b6410 0x7f3be41e35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:08.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.374+0000 7f3bea896640 1 -- 172.21.15.98:0/3543082079 >> 172.21.15.98:0/3543082079 conn(0x7f3be40d0a50 msgr2=0x7f3be40c6c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:08.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.374+0000 7f3bea896640 1 -- 172.21.15.98:0/3543082079 shutdown_connections 2024-09-19T08:42:08.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:08.374+0000 7f3bea896640 1 -- 172.21.15.98:0/3543082079 wait complete. 2024-09-19T08:42:08.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:08.390 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/37a428aeb421f12b4cfa26fa20792fca.meta) 2024-09-19T08:42:12.627 INFO:teuthology.orchestra.run.smithi098.stdout:[GLOBAL] 2024-09-19T08:42:12.627 INFO:teuthology.orchestra.run.smithi098.stdout:version = 1 2024-09-19T08:42:12.627 INFO:teuthology.orchestra.run.smithi098.stdout:type = subvolume 2024-09-19T08:42:12.628 INFO:teuthology.orchestra.run.smithi098.stdout:path = /volumes/_nogroup/subvol98020 2024-09-19T08:42:12.628 INFO:teuthology.orchestra.run.smithi098.stdout:state = complete 2024-09-19T08:42:12.628 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:42:12.630 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume authorize cephfs subvol98020 alice 2024-09-19T08:42:12.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.762+0000 7f5a1ec6a640 1 Processor -- start 2024-09-19T08:42:12.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.762+0000 7f5a1ec6a640 1 -- start start 2024-09-19T08:42:12.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.762+0000 7f5a1ec6a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5a1815e720 0x7f5a180cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:12.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.762+0000 7f5a1ec6a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5a180cfb50 0x7f5a180b7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:12.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.762+0000 7f5a1ec6a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a180d29c0 con 0x7f5a180b7cb0 2024-09-19T08:42:12.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.762+0000 7f5a1ec6a640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f5a180b80c0 con 0x7f5a1815e720 2024-09-19T08:42:12.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.762+0000 7f5a1ec6a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f5a180d2ca0 con 0x7f5a180cfb50 2024-09-19T08:42:12.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.762+0000 7f5a1ca06640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5a1815e720 0x7f5a180cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:12.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.766+0000 7f5a1ca06640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5a1815e720 0x7f5a180cf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:45998/0 (socket says 172.21.15.98:45998) 2024-09-19T08:42:12.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.766+0000 7f5a1ca06640 1 -- 172.21.15.98:0/3103028550 learned_addr learned my addr 172.21.15.98:0/3103028550 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:12.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.766+0000 7f5a17fff640 1 --2- 172.21.15.98:0/3103028550 >> v2:172.21.15.139:3300/0 conn(0x7f5a180cfb50 0x7f5a180b7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:12.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.766+0000 7f5a1ca06640 1 -- 172.21.15.98:0/3103028550 >> v2:172.21.15.139:3300/0 conn(0x7f5a180cfb50 msgr2=0x7f5a180b7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:12.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.766+0000 7f5a1ca06640 1 --2- 172.21.15.98:0/3103028550 >> v2:172.21.15.139:3300/0 conn(0x7f5a180cfb50 0x7f5a180b7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:12.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.766+0000 7f5a1ca06640 1 -- 172.21.15.98:0/3103028550 >> v1:172.21.15.139:6789/0 conn(0x7f5a180b7cb0 legacy=0x7f5a180d3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:12.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.766+0000 7f5a1ca06640 1 -- 172.21.15.98:0/3103028550 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a180d2f80 con 0x7f5a1815e720 2024-09-19T08:42:12.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.766+0000 7f5a1ca06640 1 --2- 172.21.15.98:0/3103028550 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a1815e720 0x7f5a180cf610 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f5a180d51f0 tx=0x7f5a0001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a03d51fa32af51d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:12.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.766+0000 7f5a177fe640 1 -- 172.21.15.98:0/3103028550 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a000aa610 con 0x7f5a1815e720 2024-09-19T08:42:12.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.766+0000 7f5a177fe640 1 -- 172.21.15.98:0/3103028550 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a000af070 con 0x7f5a1815e720 2024-09-19T08:42:12.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.766+0000 7f5a177fe640 1 -- 172.21.15.98:0/3103028550 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a000aa9d0 con 0x7f5a1815e720 2024-09-19T08:42:12.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.766+0000 7f5a1ec6a640 1 -- 172.21.15.98:0/3103028550 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a1815e720 msgr2=0x7f5a180cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:12.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.766+0000 7f5a1ec6a640 1 --2- 172.21.15.98:0/3103028550 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a1815e720 0x7f5a180cf610 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f5a180d51f0 tx=0x7f5a0001c4b0 comp rx=0 tx=0).stop 2024-09-19T08:42:12.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.766+0000 7f5a1ec6a640 1 -- 172.21.15.98:0/3103028550 shutdown_connections 2024-09-19T08:42:12.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.766+0000 7f5a1ec6a640 1 --2- 172.21.15.98:0/3103028550 >> v2:172.21.15.139:3300/0 conn(0x7f5a180cfb50 0x7f5a180b7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:12.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.766+0000 7f5a1ec6a640 1 --2- 172.21.15.98:0/3103028550 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a1815e720 0x7f5a180cf610 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:12.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.766+0000 7f5a1ec6a640 1 -- 172.21.15.98:0/3103028550 >> 172.21.15.98:0/3103028550 conn(0x7f5a180d4640 msgr2=0x7f5a180d4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:12.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.766+0000 7f5a1ec6a640 1 -- 172.21.15.98:0/3103028550 shutdown_connections 2024-09-19T08:42:12.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.766+0000 7f5a1ec6a640 1 -- 172.21.15.98:0/3103028550 wait complete. 2024-09-19T08:42:12.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.770+0000 7f5a1ec6a640 1 Processor -- start 2024-09-19T08:42:12.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.770+0000 7f5a1ec6a640 1 -- start start 2024-09-19T08:42:12.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.770+0000 7f5a1ec6a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a180b7cb0 0x7f5a181f8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:12.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.770+0000 7f5a1ec6a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a180cfb50 0x7f5a1815c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:12.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.770+0000 7f5a1ec6a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a1815e720 0x7f5a1815cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:12.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.770+0000 7f5a1ec6a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1815ec80 con 0x7f5a1815e720 2024-09-19T08:42:12.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.770+0000 7f5a1ec6a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1815eb00 con 0x7f5a180cfb50 2024-09-19T08:42:12.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.770+0000 7f5a1ec6a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5a180d4170 con 0x7f5a180b7cb0 2024-09-19T08:42:12.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.770+0000 7f5a1d207640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a1815e720 0x7f5a1815cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:12.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.770+0000 7f5a1d207640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a1815e720 0x7f5a1815cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46004/0 (socket says 172.21.15.98:46004) 2024-09-19T08:42:12.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.770+0000 7f5a1d207640 1 -- 172.21.15.98:0/436112371 learned_addr learned my addr 172.21.15.98:0/436112371 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:12.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.770+0000 7f5a17fff640 1 --2- 172.21.15.98:0/436112371 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a180cfb50 0x7f5a1815c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:12.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.770+0000 7f5a1ca06640 1 --2- 172.21.15.98:0/436112371 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a180b7cb0 0x7f5a181f8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:12.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.770+0000 7f5a1d207640 1 -- 172.21.15.98:0/436112371 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a180b7cb0 msgr2=0x7f5a181f8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:12.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.770+0000 7f5a1d207640 1 --2- 172.21.15.98:0/436112371 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a180b7cb0 0x7f5a181f8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:12.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.770+0000 7f5a1d207640 1 -- 172.21.15.98:0/436112371 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a180cfb50 msgr2=0x7f5a1815c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:12.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.770+0000 7f5a1d207640 1 --2- 172.21.15.98:0/436112371 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a180cfb50 0x7f5a1815c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:12.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.770+0000 7f5a1d207640 1 -- 172.21.15.98:0/436112371 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a18153dc0 con 0x7f5a1815e720 2024-09-19T08:42:12.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.770+0000 7f5a1d207640 1 --2- 172.21.15.98:0/436112371 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a1815e720 0x7f5a1815cc50 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5a1000b9e0 tx=0x7f5a1000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:12.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.770+0000 7f5a157fa640 1 -- 172.21.15.98:0/436112371 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a10003b80 con 0x7f5a1815e720 2024-09-19T08:42:12.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.770+0000 7f5a157fa640 1 -- 172.21.15.98:0/436112371 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a10003d20 con 0x7f5a1815e720 2024-09-19T08:42:12.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.770+0000 7f5a1ec6a640 1 -- 172.21.15.98:0/436112371 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a181fb520 con 0x7f5a1815e720 2024-09-19T08:42:12.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.770+0000 7f5a157fa640 1 -- 172.21.15.98:0/436112371 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1000cae0 con 0x7f5a1815e720 2024-09-19T08:42:12.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.770+0000 7f5a1ec6a640 1 -- 172.21.15.98:0/436112371 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a181fbae0 con 0x7f5a1815e720 2024-09-19T08:42:12.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.770+0000 7f5a1ec6a640 1 -- 172.21.15.98:0/436112371 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a180b7770 con 0x7f5a1815e720 2024-09-19T08:42:12.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.774+0000 7f5a157fa640 1 -- 172.21.15.98:0/436112371 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 10) ==== 87649+0+0 (secure 0 0 0) 0x7f5a10006d10 con 0x7f5a1815e720 2024-09-19T08:42:12.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.774+0000 7f5a157fa640 1 --2- 172.21.15.98:0/436112371 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f59f8067b00 0x7f59f8069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:12.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.774+0000 7f5a1ca06640 1 --2- 172.21.15.98:0/436112371 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f59f8067b00 0x7f59f8069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:12.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.774+0000 7f5a157fa640 1 -- 172.21.15.98:0/436112371 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f5a10085b50 con 0x7f5a1815e720 2024-09-19T08:42:12.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.774+0000 7f5a1ca06640 1 --2- 172.21.15.98:0/436112371 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f59f8067b00 0x7f59f8069fc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5a000ae000 tx=0x7f5a000a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:12.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.778+0000 7f5a157fa640 1 -- 172.21.15.98:0/436112371 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f5a100563c0 con 0x7f5a1815e720 2024-09-19T08:42:12.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:12.966+0000 7f5a1ec6a640 1 -- 172.21.15.98:0/436112371 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "fs subvolume authorize", "vol_name": "cephfs", "sub_name": "subvol98020", "auth_id": "alice", "target": ["mon-mgr", ""]}) -- 0x7f5a180cf1e0 con 0x7f59f8067b00 2024-09-19T08:42:12.981 INFO:tasks.ceph.mgr.y.smithi098.stderr:Creating meta for ID alice with tenant None 2024-09-19T08:42:13.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.030+0000 7f5a157fa640 1 -- 172.21.15.98:0/436112371 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5a180cf1e0 con 0x7f59f8067b00 2024-09-19T08:42:13.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.034+0000 7f5a1ec6a640 1 -- 172.21.15.98:0/436112371 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f59f8067b00 msgr2=0x7f59f8069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:13.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.034+0000 7f5a1ec6a640 1 --2- 172.21.15.98:0/436112371 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f59f8067b00 0x7f59f8069fc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5a000ae000 tx=0x7f5a000a7000 comp rx=0 tx=0).stop 2024-09-19T08:42:13.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.034+0000 7f5a1ec6a640 1 -- 172.21.15.98:0/436112371 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a1815e720 msgr2=0x7f5a1815cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:13.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.034+0000 7f5a1ec6a640 1 --2- 172.21.15.98:0/436112371 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a1815e720 0x7f5a1815cc50 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5a1000b9e0 tx=0x7f5a1000bea0 comp rx=0 tx=0).stop 2024-09-19T08:42:13.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.034+0000 7f5a1ec6a640 1 -- 172.21.15.98:0/436112371 shutdown_connections 2024-09-19T08:42:13.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.034+0000 7f5a1ec6a640 1 --2- 172.21.15.98:0/436112371 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f59f8067b00 0x7f59f8069fc0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:13.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.034+0000 7f5a1ec6a640 1 --2- 172.21.15.98:0/436112371 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5a1815e720 0x7f5a1815cc50 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:13.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.034+0000 7f5a1ec6a640 1 --2- 172.21.15.98:0/436112371 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5a180cfb50 0x7f5a1815c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:13.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.034+0000 7f5a1ec6a640 1 --2- 172.21.15.98:0/436112371 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5a180b7cb0 0x7f5a181f8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:13.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.034+0000 7f5a1ec6a640 1 -- 172.21.15.98:0/436112371 >> 172.21.15.98:0/436112371 conn(0x7f5a180d4640 msgr2=0x7f5a180b6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:13.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.034+0000 7f5a1ec6a640 1 -- 172.21.15.98:0/436112371 shutdown_connections 2024-09-19T08:42:13.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.034+0000 7f5a1ec6a640 1 -- 172.21.15.98:0/436112371 wait complete. 2024-09-19T08:42:13.052 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.alice --format=json-pretty 2024-09-19T08:42:13.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.178+0000 7ff06e0a1640 1 Processor -- start 2024-09-19T08:42:13.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.178+0000 7ff06e0a1640 1 -- start start 2024-09-19T08:42:13.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.178+0000 7ff06e0a1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff06815c550 0x7ff06806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:13.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.178+0000 7ff06e0a1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff06806c1d0 0x7ff0680cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:13.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.178+0000 7ff06e0a1640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff06806d310 con 0x7ff0680cf900 2024-09-19T08:42:13.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.178+0000 7ff06e0a1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff06806c610 con 0x7ff06815c550 2024-09-19T08:42:13.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.178+0000 7ff06e0a1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff06806d5f0 con 0x7ff06806c1d0 2024-09-19T08:42:13.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.178+0000 7ff067fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7ff0680cf900 0x7ff0680ce1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:35920/0 (socket says 172.21.15.98:35920) 2024-09-19T08:42:13.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.178+0000 7ff067fff640 1 -- 172.21.15.98:0/3063218925 learned_addr learned my addr 172.21.15.98:0/3063218925 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:13.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.178+0000 7ff0677fe640 1 --2- 172.21.15.98:0/3063218925 >> v2:172.21.15.139:3300/0 conn(0x7ff06815c550 0x7ff06806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:13.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.178+0000 7ff066ffd640 1 --2- 172.21.15.98:0/3063218925 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff06806c1d0 0x7ff0680cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:13.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.178+0000 7ff0667fc640 1 -- 172.21.15.98:0/3063218925 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3679725743 0 0) 0x7ff06806d310 con 0x7ff0680cf900 2024-09-19T08:42:13.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.178+0000 7ff0667fc640 1 -- 172.21.15.98:0/3063218925 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff054003530 con 0x7ff0680cf900 2024-09-19T08:42:13.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.178+0000 7ff0667fc640 1 -- 172.21.15.98:0/3063218925 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1568844561 0 0) 0x7ff054003530 con 0x7ff0680cf900 2024-09-19T08:42:13.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.178+0000 7ff0667fc640 1 -- 172.21.15.98:0/3063218925 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff06806c1d0 msgr2=0x7ff0680cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:13.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.178+0000 7ff0667fc640 1 --2- 172.21.15.98:0/3063218925 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff06806c1d0 0x7ff0680cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:13.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.178+0000 7ff0667fc640 1 -- 172.21.15.98:0/3063218925 >> v2:172.21.15.139:3300/0 conn(0x7ff06815c550 msgr2=0x7ff06806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:13.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.178+0000 7ff0667fc640 1 --2- 172.21.15.98:0/3063218925 >> v2:172.21.15.139:3300/0 conn(0x7ff06815c550 0x7ff06806bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:13.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.178+0000 7ff0667fc640 1 -- 172.21.15.98:0/3063218925 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff06806d8d0 con 0x7ff0680cf900 2024-09-19T08:42:13.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.178+0000 7ff0667fc640 1 -- 172.21.15.98:0/3063218925 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7ff060001a00 con 0x7ff0680cf900 2024-09-19T08:42:13.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.178+0000 7ff0667fc640 1 -- 172.21.15.98:0/3063218925 >> v1:172.21.15.98:6789/0 conn(0x7ff0680cf900 legacy=0x7ff0680ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:13.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.178+0000 7ff0667fc640 1 --2- 172.21.15.98:0/3063218925 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0540039c0 0x7ff054003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:13.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.178+0000 7ff0667fc640 1 --2- 172.21.15.98:0/3063218925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff054004810 0x7ff054004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:13.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.178+0000 7ff0667fc640 1 --2- 172.21.15.98:0/3063218925 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff06806c1d0 0x7ff0540051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:13.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.178+0000 7ff0667fc640 1 -- 172.21.15.98:0/3063218925 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff06806c610 con 0x7ff0540039c0 2024-09-19T08:42:13.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.178+0000 7ff0667fc640 1 -- 172.21.15.98:0/3063218925 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff06806d5f0 con 0x7ff054004810 2024-09-19T08:42:13.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.178+0000 7ff0667fc640 1 -- 172.21.15.98:0/3063218925 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff05400b9d0 con 0x7ff06806c1d0 2024-09-19T08:42:13.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.182+0000 7ff0677fe640 1 --2- 172.21.15.98:0/3063218925 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0540039c0 0x7ff054003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:13.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.182+0000 7ff066ffd640 1 --2- 172.21.15.98:0/3063218925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff054004810 0x7ff054004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:13.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.182+0000 7ff0677fe640 1 -- 172.21.15.98:0/3063218925 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff06806c1d0 msgr2=0x7ff0540051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:13.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.182+0000 7ff0677fe640 1 --2- 172.21.15.98:0/3063218925 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff06806c1d0 0x7ff0540051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:13.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.182+0000 7ff0677fe640 1 -- 172.21.15.98:0/3063218925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff054004810 msgr2=0x7ff054004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:13.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.182+0000 7ff0677fe640 1 --2- 172.21.15.98:0/3063218925 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff054004810 0x7ff054004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:13.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.182+0000 7ff0677fe640 1 -- 172.21.15.98:0/3063218925 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff054003530 con 0x7ff0540039c0 2024-09-19T08:42:13.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.182+0000 7ff0677fe640 1 --2- 172.21.15.98:0/3063218925 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0540039c0 0x7ff054003de0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff058009ae0 tx=0x7ff05801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8267b680c4941db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:13.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.182+0000 7ff0667fc640 1 -- 172.21.15.98:0/3063218925 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0580096b0 con 0x7ff0540039c0 2024-09-19T08:42:13.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.182+0000 7ff0667fc640 1 -- 172.21.15.98:0/3063218925 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0580a8410 con 0x7ff0540039c0 2024-09-19T08:42:13.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.182+0000 7ff0667fc640 1 -- 172.21.15.98:0/3063218925 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0580a8800 con 0x7ff0540039c0 2024-09-19T08:42:13.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.182+0000 7ff06e0a1640 1 -- 172.21.15.98:0/3063218925 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0540039c0 msgr2=0x7ff054003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:13.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.182+0000 7ff06e0a1640 1 --2- 172.21.15.98:0/3063218925 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0540039c0 0x7ff054003de0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff058009ae0 tx=0x7ff05801cab0 comp rx=0 tx=0).stop 2024-09-19T08:42:13.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.182+0000 7ff067fff640 1 -- 172.21.15.98:0/3063218925 reap_dead start 2024-09-19T08:42:13.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.182+0000 7ff06e0a1640 1 -- 172.21.15.98:0/3063218925 shutdown_connections 2024-09-19T08:42:13.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.182+0000 7ff06e0a1640 1 -- 172.21.15.98:0/3063218925 >> 172.21.15.98:0/3063218925 conn(0x7ff0680b2010 msgr2=0x7ff0680b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:13.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.182+0000 7ff06e0a1640 1 -- 172.21.15.98:0/3063218925 shutdown_connections 2024-09-19T08:42:13.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.182+0000 7ff06e0a1640 1 -- 172.21.15.98:0/3063218925 wait complete. 2024-09-19T08:42:13.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.186+0000 7ff06e0a1640 1 Processor -- start 2024-09-19T08:42:13.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.186+0000 7ff06e0a1640 1 -- start start 2024-09-19T08:42:13.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.186+0000 7ff06e0a1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0540039c0 0x7ff068154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:13.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.186+0000 7ff06e0a1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff068155e00 0x7ff068152420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:13.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.186+0000 7ff06e0a1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff068156280 0x7ff068152960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:13.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.186+0000 7ff06e0a1640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff068167220 con 0x7ff0540039c0 2024-09-19T08:42:13.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.186+0000 7ff06e0a1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff06815cb40 con 0x7ff068155e00 2024-09-19T08:42:13.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.186+0000 7ff06e0a1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff0681673a0 con 0x7ff068156280 2024-09-19T08:42:13.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.186+0000 7ff0677fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0540039c0 0x7ff068154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:13.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.186+0000 7ff0677fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0540039c0 0x7ff068154dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46024/0 (socket says 172.21.15.98:46024) 2024-09-19T08:42:13.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.186+0000 7ff0677fe640 1 -- 172.21.15.98:0/2285612713 learned_addr learned my addr 172.21.15.98:0/2285612713 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:13.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.186+0000 7ff066ffd640 1 --2- 172.21.15.98:0/2285612713 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff068155e00 0x7ff068152420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:13.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.186+0000 7ff067fff640 1 --2- 172.21.15.98:0/2285612713 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff068156280 0x7ff068152960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:13.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.186+0000 7ff0677fe640 1 -- 172.21.15.98:0/2285612713 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff068156280 msgr2=0x7ff068152960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:13.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.186+0000 7ff0677fe640 1 --2- 172.21.15.98:0/2285612713 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff068156280 0x7ff068152960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:13.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.186+0000 7ff0677fe640 1 -- 172.21.15.98:0/2285612713 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff068155e00 msgr2=0x7ff068152420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:13.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.186+0000 7ff0677fe640 1 --2- 172.21.15.98:0/2285612713 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff068155e00 0x7ff068152420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:13.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.186+0000 7ff0677fe640 1 -- 172.21.15.98:0/2285612713 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff068155310 con 0x7ff0540039c0 2024-09-19T08:42:13.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.186+0000 7ff0677fe640 1 --2- 172.21.15.98:0/2285612713 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0540039c0 0x7ff068154dd0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff05801c8e0 tx=0x7ff058019850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:13.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.186+0000 7ff05ffff640 1 -- 172.21.15.98:0/2285612713 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05800f000 con 0x7ff0540039c0 2024-09-19T08:42:13.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.186+0000 7ff05ffff640 1 -- 172.21.15.98:0/2285612713 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0580a8bc0 con 0x7ff0540039c0 2024-09-19T08:42:13.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.186+0000 7ff06e0a1640 1 -- 172.21.15.98:0/2285612713 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff068161400 con 0x7ff0540039c0 2024-09-19T08:42:13.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.186+0000 7ff05ffff640 1 -- 172.21.15.98:0/2285612713 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff058004020 con 0x7ff0540039c0 2024-09-19T08:42:13.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.186+0000 7ff06e0a1640 1 -- 172.21.15.98:0/2285612713 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0681619c0 con 0x7ff0540039c0 2024-09-19T08:42:13.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.186+0000 7ff06e0a1640 1 -- 172.21.15.98:0/2285612713 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff06806bc90 con 0x7ff0540039c0 2024-09-19T08:42:13.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.186+0000 7ff05ffff640 1 -- 172.21.15.98:0/2285612713 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 10) ==== 87649+0+0 (secure 0 0 0) 0x7ff0580c1060 con 0x7ff0540039c0 2024-09-19T08:42:13.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.186+0000 7ff05ffff640 1 --2- 172.21.15.98:0/2285612713 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff048067b00 0x7ff048069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:13.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.186+0000 7ff05ffff640 1 -- 172.21.15.98:0/2285612713 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7ff05811ffe0 con 0x7ff0540039c0 2024-09-19T08:42:13.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.186+0000 7ff066ffd640 1 --2- 172.21.15.98:0/2285612713 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff048067b00 0x7ff048069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:13.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.190+0000 7ff066ffd640 1 --2- 172.21.15.98:0/2285612713 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff048067b00 0x7ff048069fc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff050009a30 tx=0x7ff050008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:13.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.194+0000 7ff05ffff640 1 -- 172.21.15.98:0/2285612713 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff068155310 con 0x7ff0540039c0 2024-09-19T08:42:13.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.414+0000 7ff06e0a1640 1 -- 172.21.15.98:0/2285612713 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.alice", "format": "json-pretty"} v 0) -- 0x7ff0680cdcb0 con 0x7ff0540039c0 2024-09-19T08:42:13.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.414+0000 7ff05ffff640 1 -- 172.21.15.98:0/2285612713 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.alice", "format": "json-pretty"}]=0 v16) ==== 107+0+280 (secure 0 0 0) 0x7ff0580ac030 con 0x7ff0540039c0 2024-09-19T08:42:13.414 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:42:13.414 INFO:teuthology.orchestra.run.smithi098.stdout:[ 2024-09-19T08:42:13.414 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:42:13.414 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.alice", 2024-09-19T08:42:13.414 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDl4+tmaRNMABAA7Zkv2UbmJKSZ8W9z9oI5pA==", 2024-09-19T08:42:13.415 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:42:13.415 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow rw path=/volumes/_nogroup/subvol98020", 2024-09-19T08:42:13.415 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow r", 2024-09-19T08:42:13.415 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rw pool=cephfs_data" 2024-09-19T08:42:13.415 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:42:13.415 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:42:13.415 INFO:teuthology.orchestra.run.smithi098.stdout:] 2024-09-19T08:42:13.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.414+0000 7ff06e0a1640 1 -- 172.21.15.98:0/2285612713 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff048067b00 msgr2=0x7ff048069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:13.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.414+0000 7ff06e0a1640 1 --2- 172.21.15.98:0/2285612713 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff048067b00 0x7ff048069fc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff050009a30 tx=0x7ff050008040 comp rx=0 tx=0).stop 2024-09-19T08:42:13.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.414+0000 7ff06e0a1640 1 -- 172.21.15.98:0/2285612713 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0540039c0 msgr2=0x7ff068154dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:13.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.414+0000 7ff06e0a1640 1 --2- 172.21.15.98:0/2285612713 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0540039c0 0x7ff068154dd0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff05801c8e0 tx=0x7ff058019850 comp rx=0 tx=0).stop 2024-09-19T08:42:13.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.418+0000 7ff06e0a1640 1 -- 172.21.15.98:0/2285612713 shutdown_connections 2024-09-19T08:42:13.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.418+0000 7ff06e0a1640 1 --2- 172.21.15.98:0/2285612713 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff048067b00 0x7ff048069fc0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:13.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.418+0000 7ff06e0a1640 1 --2- 172.21.15.98:0/2285612713 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff068156280 0x7ff068152960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:13.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.418+0000 7ff06e0a1640 1 --2- 172.21.15.98:0/2285612713 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff068155e00 0x7ff068152420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:13.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.418+0000 7ff06e0a1640 1 --2- 172.21.15.98:0/2285612713 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0540039c0 0x7ff068154dd0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:13.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.418+0000 7ff06e0a1640 1 -- 172.21.15.98:0/2285612713 >> 172.21.15.98:0/2285612713 conn(0x7ff0680b2010 msgr2=0x7ff0680cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:13.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.418+0000 7ff06e0a1640 1 -- 172.21.15.98:0/2285612713 shutdown_connections 2024-09-19T08:42:13.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.418+0000 7ff06e0a1640 1 -- 172.21.15.98:0/2285612713 wait complete. 2024-09-19T08:42:13.433 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol98020 2024-09-19T08:42:13.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.554+0000 7fbc31174640 1 Processor -- start 2024-09-19T08:42:13.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.554+0000 7fbc31174640 1 -- start start 2024-09-19T08:42:13.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.554+0000 7fbc31174640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbc2c1415a0 0x7fbc2c0d20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:13.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.554+0000 7fbc31174640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc2c0d25f0 0x7fbc2c0d3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:13.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.554+0000 7fbc31174640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc2c15eae0 con 0x7fbc2c0d38a0 2024-09-19T08:42:13.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.554+0000 7fbc31174640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fbc2c0d2a30 con 0x7fbc2c1415a0 2024-09-19T08:42:13.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.554+0000 7fbc31174640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbc2c15edc0 con 0x7fbc2c0d25f0 2024-09-19T08:42:13.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.554+0000 7fbc2ad9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbc2c1415a0 0x7fbc2c0d20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:13.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.554+0000 7fbc2ad9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbc2c1415a0 0x7fbc2c0d20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46026/0 (socket says 172.21.15.98:46026) 2024-09-19T08:42:13.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.554+0000 7fbc2ad9d640 1 -- 172.21.15.98:0/2088927058 learned_addr learned my addr 172.21.15.98:0/2088927058 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:13.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.554+0000 7fbc2ad9d640 1 -- 172.21.15.98:0/2088927058 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc2c0d25f0 msgr2=0x7fbc2c0d3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:13.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.554+0000 7fbc2a59c640 1 --2- 172.21.15.98:0/2088927058 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc2c0d25f0 0x7fbc2c0d3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:13.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.554+0000 7fbc2ad9d640 1 --2- 172.21.15.98:0/2088927058 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc2c0d25f0 0x7fbc2c0d3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:13.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.554+0000 7fbc2ad9d640 1 -- 172.21.15.98:0/2088927058 >> v1:172.21.15.139:6789/0 conn(0x7fbc2c0d38a0 legacy=0x7fbc2c161fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:13.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.554+0000 7fbc2ad9d640 1 -- 172.21.15.98:0/2088927058 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc2c15f0a0 con 0x7fbc2c1415a0 2024-09-19T08:42:13.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.558+0000 7fbc2ad9d640 1 --2- 172.21.15.98:0/2088927058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc2c1415a0 0x7fbc2c0d20b0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c009870 tx=0x7fbc1c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e34ba7fb6adc9a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:13.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.558+0000 7fbc29d9b640 1 -- 172.21.15.98:0/2088927058 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c0b0070 con 0x7fbc2c1415a0 2024-09-19T08:42:13.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.558+0000 7fbc29d9b640 1 -- 172.21.15.98:0/2088927058 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc1c0ab600 con 0x7fbc2c1415a0 2024-09-19T08:42:13.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.558+0000 7fbc29d9b640 1 -- 172.21.15.98:0/2088927058 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c0ab9d0 con 0x7fbc2c1415a0 2024-09-19T08:42:13.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.558+0000 7fbc31174640 1 -- 172.21.15.98:0/2088927058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc2c1415a0 msgr2=0x7fbc2c0d20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:13.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.558+0000 7fbc31174640 1 --2- 172.21.15.98:0/2088927058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc2c1415a0 0x7fbc2c0d20b0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c009870 tx=0x7fbc1c01c4f0 comp rx=0 tx=0).stop 2024-09-19T08:42:13.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.558+0000 7fbc31174640 1 -- 172.21.15.98:0/2088927058 shutdown_connections 2024-09-19T08:42:13.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.558+0000 7fbc31174640 1 --2- 172.21.15.98:0/2088927058 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc2c0d25f0 0x7fbc2c0d3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:13.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.558+0000 7fbc31174640 1 --2- 172.21.15.98:0/2088927058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc2c1415a0 0x7fbc2c0d20b0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:13.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.558+0000 7fbc31174640 1 -- 172.21.15.98:0/2088927058 >> 172.21.15.98:0/2088927058 conn(0x7fbc2c074c70 msgr2=0x7fbc2c075080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:13.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.558+0000 7fbc31174640 1 -- 172.21.15.98:0/2088927058 shutdown_connections 2024-09-19T08:42:13.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.558+0000 7fbc31174640 1 -- 172.21.15.98:0/2088927058 wait complete. 2024-09-19T08:42:13.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.558+0000 7fbc31174640 1 Processor -- start 2024-09-19T08:42:13.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.558+0000 7fbc31174640 1 -- start start 2024-09-19T08:42:13.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.558+0000 7fbc31174640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc2c0d25f0 0x7fbc2c13f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:13.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.558+0000 7fbc31174640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc2c0d38a0 0x7fbc2c142050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:13.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.558+0000 7fbc31174640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc2c1415a0 0x7fbc2c0cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:13.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.558+0000 7fbc31174640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c0d3eb0 con 0x7fbc2c1415a0 2024-09-19T08:42:13.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.558+0000 7fbc31174640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c0d3d30 con 0x7fbc2c0d38a0 2024-09-19T08:42:13.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.558+0000 7fbc31174640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbc2c0d4120 con 0x7fbc2c0d25f0 2024-09-19T08:42:13.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.558+0000 7fbc2b59e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc2c1415a0 0x7fbc2c0cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:13.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.562+0000 7fbc2b59e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc2c1415a0 0x7fbc2c0cf1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46028/0 (socket says 172.21.15.98:46028) 2024-09-19T08:42:13.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.562+0000 7fbc2b59e640 1 -- 172.21.15.98:0/934578229 learned_addr learned my addr 172.21.15.98:0/934578229 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:13.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.562+0000 7fbc2ad9d640 1 --2- 172.21.15.98:0/934578229 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc2c0d25f0 0x7fbc2c13f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:13.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.562+0000 7fbc2a59c640 1 --2- 172.21.15.98:0/934578229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc2c0d38a0 0x7fbc2c142050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:13.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.562+0000 7fbc2b59e640 1 -- 172.21.15.98:0/934578229 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc2c0d25f0 msgr2=0x7fbc2c13f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:13.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.562+0000 7fbc2b59e640 1 --2- 172.21.15.98:0/934578229 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc2c0d25f0 0x7fbc2c13f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:13.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.562+0000 7fbc2b59e640 1 -- 172.21.15.98:0/934578229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc2c0d38a0 msgr2=0x7fbc2c142050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:13.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.562+0000 7fbc2b59e640 1 --2- 172.21.15.98:0/934578229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc2c0d38a0 0x7fbc2c142050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:13.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.562+0000 7fbc2b59e640 1 -- 172.21.15.98:0/934578229 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc2c0cd370 con 0x7fbc2c1415a0 2024-09-19T08:42:13.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.562+0000 7fbc2a59c640 1 --2- 172.21.15.98:0/934578229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc2c0d38a0 0x7fbc2c142050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:42:13.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.562+0000 7fbc2b59e640 1 --2- 172.21.15.98:0/934578229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc2c1415a0 0x7fbc2c0cf1f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fbc2400c7b0 tx=0x7fbc2400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:13.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.562+0000 7fbc137fe640 1 -- 172.21.15.98:0/934578229 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc24004b70 con 0x7fbc2c1415a0 2024-09-19T08:42:13.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.562+0000 7fbc137fe640 1 -- 172.21.15.98:0/934578229 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc24004d10 con 0x7fbc2c1415a0 2024-09-19T08:42:13.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.562+0000 7fbc31174640 1 -- 172.21.15.98:0/934578229 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc2c0cd650 con 0x7fbc2c1415a0 2024-09-19T08:42:13.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.562+0000 7fbc137fe640 1 -- 172.21.15.98:0/934578229 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2400aa90 con 0x7fbc2c1415a0 2024-09-19T08:42:13.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.562+0000 7fbc31174640 1 -- 172.21.15.98:0/934578229 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc2c0ce1c0 con 0x7fbc2c1415a0 2024-09-19T08:42:13.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.562+0000 7fbc31174640 1 -- 172.21.15.98:0/934578229 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc2c13fb20 con 0x7fbc2c1415a0 2024-09-19T08:42:13.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.562+0000 7fbc137fe640 1 -- 172.21.15.98:0/934578229 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 10) ==== 87649+0+0 (secure 0 0 0) 0x7fbc2400ac30 con 0x7fbc2c1415a0 2024-09-19T08:42:13.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.562+0000 7fbc137fe640 1 --2- 172.21.15.98:0/934578229 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fbc18067b30 0x7fbc18069ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:13.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.562+0000 7fbc2ad9d640 1 --2- 172.21.15.98:0/934578229 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fbc18067b30 0x7fbc18069ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:13.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.562+0000 7fbc137fe640 1 -- 172.21.15.98:0/934578229 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fbc240862f0 con 0x7fbc2c1415a0 2024-09-19T08:42:13.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.562+0000 7fbc2ad9d640 1 --2- 172.21.15.98:0/934578229 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fbc18067b30 0x7fbc18069ff0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c0ad000 tx=0x7fbc1c0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:13.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.566+0000 7fbc137fe640 1 -- 172.21.15.98:0/934578229 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fbc24020070 con 0x7fbc2c1415a0 2024-09-19T08:42:13.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.754+0000 7fbc31174640 1 -- 172.21.15.98:0/934578229 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol98020", "target": ["mon-mgr", ""]}) -- 0x7fbc2c0cd120 con 0x7fbc18067b30 2024-09-19T08:42:13.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.770+0000 7fbc137fe640 1 -- 172.21.15.98:0/934578229 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbc2c0cd120 con 0x7fbc18067b30 2024-09-19T08:42:13.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.774+0000 7fbc31174640 1 -- 172.21.15.98:0/934578229 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fbc18067b30 msgr2=0x7fbc18069ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:13.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.774+0000 7fbc31174640 1 --2- 172.21.15.98:0/934578229 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fbc18067b30 0x7fbc18069ff0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c0ad000 tx=0x7fbc1c0a6000 comp rx=0 tx=0).stop 2024-09-19T08:42:13.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.774+0000 7fbc31174640 1 -- 172.21.15.98:0/934578229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc2c1415a0 msgr2=0x7fbc2c0cf1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:13.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.774+0000 7fbc31174640 1 --2- 172.21.15.98:0/934578229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc2c1415a0 0x7fbc2c0cf1f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fbc2400c7b0 tx=0x7fbc2400cc70 comp rx=0 tx=0).stop 2024-09-19T08:42:13.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.774+0000 7fbc31174640 1 -- 172.21.15.98:0/934578229 shutdown_connections 2024-09-19T08:42:13.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.774+0000 7fbc31174640 1 --2- 172.21.15.98:0/934578229 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fbc18067b30 0x7fbc18069ff0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:13.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.774+0000 7fbc31174640 1 --2- 172.21.15.98:0/934578229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc2c1415a0 0x7fbc2c0cf1f0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:13.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.774+0000 7fbc31174640 1 --2- 172.21.15.98:0/934578229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc2c0d38a0 0x7fbc2c142050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:13.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.774+0000 7fbc31174640 1 --2- 172.21.15.98:0/934578229 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc2c0d25f0 0x7fbc2c13f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:13.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.774+0000 7fbc31174640 1 -- 172.21.15.98:0/934578229 >> 172.21.15.98:0/934578229 conn(0x7fbc2c074c70 msgr2=0x7fbc2c1602d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:13.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.774+0000 7fbc31174640 1 -- 172.21.15.98:0/934578229 shutdown_connections 2024-09-19T08:42:13.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.774+0000 7fbc31174640 1 -- 172.21.15.98:0/934578229 wait complete. 2024-09-19T08:42:13.777 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:42:13.778+0000 7f36ec409640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:42:13.777 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:42:13.778+0000 7f36ec409640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:42:13.777 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:42:13.778+0000 7f36ec409640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:42:13.777 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:42:13.778+0000 7f36ec409640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:42:13.777 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:42:13.778+0000 7f36ec409640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:42:13.792 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:42:13.794+0000 7f36ecc0a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:42:13.792 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:42:13.794+0000 7f36ecc0a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:42:13.792 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:42:13.794+0000 7f36ecc0a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:42:13.792 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:42:13.794+0000 7f36ecc0a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:42:13.793 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:42:13.794+0000 7f36ecc0a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:42:13.795 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol27721 2024-09-19T08:42:13.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.914+0000 7fa150393640 1 Processor -- start 2024-09-19T08:42:13.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.914+0000 7fa150393640 1 -- start start 2024-09-19T08:42:13.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.914+0000 7fa150393640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa148145b70 0x7fa14806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:13.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.914+0000 7fa150393640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa148071ac0 0x7fa1480b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:13.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.914+0000 7fa150393640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa148073530 con 0x7fa14806c1d0 2024-09-19T08:42:13.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.914+0000 7fa150393640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fa148071f10 con 0x7fa148071ac0 2024-09-19T08:42:13.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.914+0000 7fa150393640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fa14806c620 con 0x7fa148145b70 2024-09-19T08:42:13.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.914+0000 7fa14e930640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa148071ac0 0x7fa1480b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:13.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.914+0000 7fa14d92e640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fa14806c1d0 0x7fa1480713b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:35924/0 (socket says 172.21.15.98:35924) 2024-09-19T08:42:13.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.914+0000 7fa14e930640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa148071ac0 0x7fa1480b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46072/0 (socket says 172.21.15.98:46072) 2024-09-19T08:42:13.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.914+0000 7fa14d92e640 1 -- 172.21.15.98:0/1597295497 learned_addr learned my addr 172.21.15.98:0/1597295497 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:13.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.914+0000 7fa14e12f640 1 --2- 172.21.15.98:0/1597295497 >> v2:172.21.15.139:3300/0 conn(0x7fa148145b70 0x7fa14806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:13.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.914+0000 7fa14e930640 1 -- 172.21.15.98:0/1597295497 >> v2:172.21.15.139:3300/0 conn(0x7fa148145b70 msgr2=0x7fa14806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:13.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.914+0000 7fa14e930640 1 --2- 172.21.15.98:0/1597295497 >> v2:172.21.15.139:3300/0 conn(0x7fa148145b70 0x7fa14806bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:13.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.914+0000 7fa14e930640 1 -- 172.21.15.98:0/1597295497 >> v1:172.21.15.98:6789/0 conn(0x7fa14806c1d0 legacy=0x7fa1480713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:13.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.914+0000 7fa14e930640 1 -- 172.21.15.98:0/1597295497 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa148073810 con 0x7fa148071ac0 2024-09-19T08:42:13.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.914+0000 7fa14d12d640 1 -- 172.21.15.98:0/1597295497 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1160168771 0 0) 0x7fa148073530 con 0x7fa14806c1d0 2024-09-19T08:42:13.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa14e930640 1 --2- 172.21.15.98:0/1597295497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa148071ac0 0x7fa1480b3410 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa1480b2d30 tx=0x7fa14001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd564effa5634987 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:13.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa14d12d640 1 -- 172.21.15.98:0/1597295497 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1400b2070 con 0x7fa148071ac0 2024-09-19T08:42:13.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa14d12d640 1 -- 172.21.15.98:0/1597295497 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1400a5760 con 0x7fa148071ac0 2024-09-19T08:42:13.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa14d12d640 1 -- 172.21.15.98:0/1597295497 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1400a5b30 con 0x7fa148071ac0 2024-09-19T08:42:13.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa150393640 1 -- 172.21.15.98:0/1597295497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa148071ac0 msgr2=0x7fa1480b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:13.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa150393640 1 --2- 172.21.15.98:0/1597295497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa148071ac0 0x7fa1480b3410 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa1480b2d30 tx=0x7fa14001c8a0 comp rx=0 tx=0).stop 2024-09-19T08:42:13.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa150393640 1 -- 172.21.15.98:0/1597295497 shutdown_connections 2024-09-19T08:42:13.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa150393640 1 --2- 172.21.15.98:0/1597295497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa148071ac0 0x7fa1480b3410 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:13.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa150393640 1 --2- 172.21.15.98:0/1597295497 >> v2:172.21.15.139:3300/0 conn(0x7fa148145b70 0x7fa14806bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:13.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa150393640 1 -- 172.21.15.98:0/1597295497 >> 172.21.15.98:0/1597295497 conn(0x7fa1480b2010 msgr2=0x7fa1480b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:13.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa150393640 1 -- 172.21.15.98:0/1597295497 shutdown_connections 2024-09-19T08:42:13.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa150393640 1 -- 172.21.15.98:0/1597295497 wait complete. 2024-09-19T08:42:13.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa150393640 1 Processor -- start 2024-09-19T08:42:13.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa150393640 1 -- start start 2024-09-19T08:42:13.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa150393640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa14806c1d0 0x7fa1480cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:13.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa150393640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa148071ac0 0x7fa148159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:13.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa150393640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa148145b70 0x7fa148165420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:13.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa150393640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa1480b42e0 con 0x7fa148071ac0 2024-09-19T08:42:13.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa150393640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa1480b4160 con 0x7fa14806c1d0 2024-09-19T08:42:13.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa150393640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa1480b4460 con 0x7fa148145b70 2024-09-19T08:42:13.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa14d92e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa148071ac0 0x7fa148159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:13.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa14d92e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa148071ac0 0x7fa148159f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46082/0 (socket says 172.21.15.98:46082) 2024-09-19T08:42:13.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa14d92e640 1 -- 172.21.15.98:0/2448453666 learned_addr learned my addr 172.21.15.98:0/2448453666 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:13.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa14e12f640 1 --2- 172.21.15.98:0/2448453666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa14806c1d0 0x7fa1480cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:13.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa14e930640 1 --2- 172.21.15.98:0/2448453666 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa148145b70 0x7fa148165420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:13.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa14d92e640 1 -- 172.21.15.98:0/2448453666 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa148145b70 msgr2=0x7fa148165420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:13.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa14d92e640 1 --2- 172.21.15.98:0/2448453666 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa148145b70 0x7fa148165420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:13.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa14d92e640 1 -- 172.21.15.98:0/2448453666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa14806c1d0 msgr2=0x7fa1480cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:13.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa14d92e640 1 --2- 172.21.15.98:0/2448453666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa14806c1d0 0x7fa1480cdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:13.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa14d92e640 1 -- 172.21.15.98:0/2448453666 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa14815af20 con 0x7fa148071ac0 2024-09-19T08:42:13.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa14e12f640 1 --2- 172.21.15.98:0/2448453666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa14806c1d0 0x7fa1480cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:42:13.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa14e930640 1 --2- 172.21.15.98:0/2448453666 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa148145b70 0x7fa148165420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:42:13.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa14d92e640 1 --2- 172.21.15.98:0/2448453666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa148071ac0 0x7fa148159f00 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa13800c4d0 tx=0x7fa13800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:13.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa13effd640 1 -- 172.21.15.98:0/2448453666 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa138018070 con 0x7fa148071ac0 2024-09-19T08:42:13.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa13effd640 1 -- 172.21.15.98:0/2448453666 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1380074e0 con 0x7fa148071ac0 2024-09-19T08:42:13.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa13effd640 1 -- 172.21.15.98:0/2448453666 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa138004a30 con 0x7fa148071ac0 2024-09-19T08:42:13.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa150393640 1 -- 172.21.15.98:0/2448453666 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1480cd660 con 0x7fa148071ac0 2024-09-19T08:42:13.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.918+0000 7fa150393640 1 -- 172.21.15.98:0/2448453666 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1481fb060 con 0x7fa148071ac0 2024-09-19T08:42:13.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.922+0000 7fa150393640 1 -- 172.21.15.98:0/2448453666 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1480b3410 con 0x7fa148071ac0 2024-09-19T08:42:13.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.922+0000 7fa13effd640 1 -- 172.21.15.98:0/2448453666 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 10) ==== 87649+0+0 (secure 0 0 0) 0x7fa13801b070 con 0x7fa148071ac0 2024-09-19T08:42:13.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.922+0000 7fa13effd640 1 --2- 172.21.15.98:0/2448453666 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa12c067b30 0x7fa12c069ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:13.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.922+0000 7fa13effd640 1 -- 172.21.15.98:0/2448453666 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fa138085380 con 0x7fa148071ac0 2024-09-19T08:42:13.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.922+0000 7fa14e12f640 1 --2- 172.21.15.98:0/2448453666 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa12c067b30 0x7fa12c069ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:13.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.922+0000 7fa14e12f640 1 --2- 172.21.15.98:0/2448453666 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa12c067b30 0x7fa12c069ff0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa1440097a0 tx=0x7fa144008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:13.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:13.926+0000 7fa13effd640 1 -- 172.21.15.98:0/2448453666 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa14815af20 con 0x7fa148071ac0 2024-09-19T08:42:14.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.102+0000 7fa150393640 1 -- 172.21.15.98:0/2448453666 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol27721", "target": ["mon-mgr", ""]}) -- 0x7fa148166e20 con 0x7fa12c067b30 2024-09-19T08:42:14.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.130+0000 7fa13effd640 1 -- 172.21.15.98:0/2448453666 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa148166e20 con 0x7fa12c067b30 2024-09-19T08:42:14.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.130+0000 7fa150393640 1 -- 172.21.15.98:0/2448453666 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa12c067b30 msgr2=0x7fa12c069ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:14.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.130+0000 7fa150393640 1 --2- 172.21.15.98:0/2448453666 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa12c067b30 0x7fa12c069ff0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa1440097a0 tx=0x7fa144008040 comp rx=0 tx=0).stop 2024-09-19T08:42:14.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.130+0000 7fa150393640 1 -- 172.21.15.98:0/2448453666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa148071ac0 msgr2=0x7fa148159f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:14.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.130+0000 7fa150393640 1 --2- 172.21.15.98:0/2448453666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa148071ac0 0x7fa148159f00 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa13800c4d0 tx=0x7fa13800c990 comp rx=0 tx=0).stop 2024-09-19T08:42:14.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.130+0000 7fa150393640 1 -- 172.21.15.98:0/2448453666 shutdown_connections 2024-09-19T08:42:14.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.130+0000 7fa150393640 1 --2- 172.21.15.98:0/2448453666 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa12c067b30 0x7fa12c069ff0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:14.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.130+0000 7fa150393640 1 --2- 172.21.15.98:0/2448453666 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa148145b70 0x7fa148165420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:14.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.130+0000 7fa150393640 1 --2- 172.21.15.98:0/2448453666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa148071ac0 0x7fa148159f00 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:14.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.130+0000 7fa150393640 1 --2- 172.21.15.98:0/2448453666 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa14806c1d0 0x7fa1480cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:14.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.130+0000 7fa150393640 1 -- 172.21.15.98:0/2448453666 >> 172.21.15.98:0/2448453666 conn(0x7fa1480b2010 msgr2=0x7fa1480b1870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:14.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.130+0000 7fa150393640 1 -- 172.21.15.98:0/2448453666 shutdown_connections 2024-09-19T08:42:14.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.130+0000 7fa150393640 1 -- 172.21.15.98:0/2448453666 wait complete. 2024-09-19T08:42:14.149 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-19T08:42:14.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T08:42:14.196 INFO:teuthology.orchestra.run.smithi098.stdout:2 2024-09-19T08:42:14.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:42:14.202 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T08:42:14.202 DEBUG:teuthology.orchestra.run.smithi098:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:42:14.313 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:42:14.387 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T08:42:14.387 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:42:14.387 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:42:14.387 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:42:14.393 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T08:42:14.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.558+0000 7f8f061f9640 1 Processor -- start 2024-09-19T08:42:14.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.558+0000 7f8f061f9640 1 -- start start 2024-09-19T08:42:14.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.558+0000 7f8f061f9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8f00158fd0 0x7f8f000b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:14.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.558+0000 7f8f061f9640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8f000b4360 0x7f8f0006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:14.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.558+0000 7f8f061f9640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f000ccc60 con 0x7f8f0006c1d0 2024-09-19T08:42:14.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.558+0000 7f8f061f9640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f8f0006c640 con 0x7f8f000b4360 2024-09-19T08:42:14.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.558+0000 7f8f061f9640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f8f001593b0 con 0x7f8f00158fd0 2024-09-19T08:42:14.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.558+0000 7f8ef6ffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8f000b4360 0x7f8f0006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:14.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.558+0000 7f8efffff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f8f0006c1d0 0x7f8f0015efc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:35936/0 (socket says 172.21.15.98:35936) 2024-09-19T08:42:14.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.558+0000 7f8efffff640 1 -- 172.21.15.98:0/2598340623 learned_addr learned my addr 172.21.15.98:0/2598340623 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:14.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.558+0000 7f8eff7fe640 1 --2- 172.21.15.98:0/2598340623 >> v2:172.21.15.139:3300/0 conn(0x7f8f00158fd0 0x7f8f000b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:14.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.558+0000 7f8ef6ffd640 1 -- 172.21.15.98:0/2598340623 >> v2:172.21.15.139:3300/0 conn(0x7f8f00158fd0 msgr2=0x7f8f000b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:14.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.558+0000 7f8ef6ffd640 1 --2- 172.21.15.98:0/2598340623 >> v2:172.21.15.139:3300/0 conn(0x7f8f00158fd0 0x7f8f000b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:14.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.558+0000 7f8ef6ffd640 1 -- 172.21.15.98:0/2598340623 >> v1:172.21.15.98:6789/0 conn(0x7f8f0006c1d0 legacy=0x7f8f0015efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:14.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.558+0000 7f8ef6ffd640 1 -- 172.21.15.98:0/2598340623 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f000ccf40 con 0x7f8f000b4360 2024-09-19T08:42:14.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.558+0000 7f8efeffd640 1 -- 172.21.15.98:0/2598340623 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3134689544 0 0) 0x7f8f000ccc60 con 0x7f8f0006c1d0 2024-09-19T08:42:14.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.562+0000 7f8ef6ffd640 1 --2- 172.21.15.98:0/2598340623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f000b4360 0x7f8f0006bc90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f8f0006e310 tx=0x7f8ee801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c923f1fb8ae787 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:14.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.562+0000 7f8efeffd640 1 -- 172.21.15.98:0/2598340623 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee80b1070 con 0x7f8f000b4360 2024-09-19T08:42:14.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.562+0000 7f8efeffd640 1 -- 172.21.15.98:0/2598340623 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ee80ac620 con 0x7f8f000b4360 2024-09-19T08:42:14.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.562+0000 7f8efeffd640 1 -- 172.21.15.98:0/2598340623 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee80ac9f0 con 0x7f8f000b4360 2024-09-19T08:42:14.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.562+0000 7f8f061f9640 1 -- 172.21.15.98:0/2598340623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f000b4360 msgr2=0x7f8f0006bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:14.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.562+0000 7f8f061f9640 1 --2- 172.21.15.98:0/2598340623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f000b4360 0x7f8f0006bc90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f8f0006e310 tx=0x7f8ee801c3c0 comp rx=0 tx=0).stop 2024-09-19T08:42:14.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.562+0000 7f8f061f9640 1 -- 172.21.15.98:0/2598340623 shutdown_connections 2024-09-19T08:42:14.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.562+0000 7f8f061f9640 1 --2- 172.21.15.98:0/2598340623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f000b4360 0x7f8f0006bc90 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:14.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.562+0000 7f8f061f9640 1 --2- 172.21.15.98:0/2598340623 >> v2:172.21.15.139:3300/0 conn(0x7f8f00158fd0 0x7f8f000b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:14.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.562+0000 7f8f061f9640 1 -- 172.21.15.98:0/2598340623 >> 172.21.15.98:0/2598340623 conn(0x7f8f000b2010 msgr2=0x7f8f000b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:14.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.562+0000 7f8f061f9640 1 -- 172.21.15.98:0/2598340623 shutdown_connections 2024-09-19T08:42:14.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.562+0000 7f8f061f9640 1 -- 172.21.15.98:0/2598340623 wait complete. 2024-09-19T08:42:14.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.562+0000 7f8f061f9640 1 Processor -- start 2024-09-19T08:42:14.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.562+0000 7f8f061f9640 1 -- start start 2024-09-19T08:42:14.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.566+0000 7f8f061f9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f0006c1d0 0x7f8f000ada20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:14.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.566+0000 7f8f061f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f000b4360 0x7f8f00155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:14.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.566+0000 7f8f061f9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f00158fd0 0x7f8f001600d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:14.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.566+0000 7f8f061f9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0015c920 con 0x7f8f00158fd0 2024-09-19T08:42:14.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.566+0000 7f8f061f9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0015c7a0 con 0x7f8f000b4360 2024-09-19T08:42:14.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.566+0000 7f8f061f9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8f00168110 con 0x7f8f0006c1d0 2024-09-19T08:42:14.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.566+0000 7f8efffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f00158fd0 0x7f8f001600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:14.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.566+0000 7f8efffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f00158fd0 0x7f8f001600d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46104/0 (socket says 172.21.15.98:46104) 2024-09-19T08:42:14.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.566+0000 7f8efffff640 1 -- 172.21.15.98:0/2224543636 learned_addr learned my addr 172.21.15.98:0/2224543636 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:14.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.566+0000 7f8eff7fe640 1 --2- 172.21.15.98:0/2224543636 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f0006c1d0 0x7f8f000ada20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:14.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.566+0000 7f8ef6ffd640 1 --2- 172.21.15.98:0/2224543636 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f000b4360 0x7f8f00155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:14.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.566+0000 7f8efffff640 1 -- 172.21.15.98:0/2224543636 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f0006c1d0 msgr2=0x7f8f000ada20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:14.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.566+0000 7f8efffff640 1 --2- 172.21.15.98:0/2224543636 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f0006c1d0 0x7f8f000ada20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:14.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.566+0000 7f8efffff640 1 -- 172.21.15.98:0/2224543636 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f000b4360 msgr2=0x7f8f00155450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:14.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.566+0000 7f8efffff640 1 --2- 172.21.15.98:0/2224543636 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f000b4360 0x7f8f00155450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:14.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.566+0000 7f8efffff640 1 -- 172.21.15.98:0/2224543636 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f00069b80 con 0x7f8f00158fd0 2024-09-19T08:42:14.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.566+0000 7f8efffff640 1 --2- 172.21.15.98:0/2224543636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f00158fd0 0x7f8f001600d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800c7d0 tx=0x7f8ef800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:14.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.566+0000 7f8efcff9640 1 -- 172.21.15.98:0/2224543636 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef8018070 con 0x7f8f00158fd0 2024-09-19T08:42:14.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.566+0000 7f8efcff9640 1 -- 172.21.15.98:0/2224543636 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ef800a640 con 0x7f8f00158fd0 2024-09-19T08:42:14.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.566+0000 7f8f061f9640 1 -- 172.21.15.98:0/2224543636 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f000ae1a0 con 0x7f8f00158fd0 2024-09-19T08:42:14.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.566+0000 7f8efcff9640 1 -- 172.21.15.98:0/2224543636 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef800aa70 con 0x7f8f00158fd0 2024-09-19T08:42:14.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.566+0000 7f8f061f9640 1 -- 172.21.15.98:0/2224543636 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f000ab0f0 con 0x7f8f00158fd0 2024-09-19T08:42:14.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.566+0000 7f8f061f9640 1 -- 172.21.15.98:0/2224543636 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f0006bc90 con 0x7f8f00158fd0 2024-09-19T08:42:14.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.566+0000 7f8efcff9640 1 -- 172.21.15.98:0/2224543636 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 10) ==== 87649+0+0 (secure 0 0 0) 0x7f8ef801b070 con 0x7f8f00158fd0 2024-09-19T08:42:14.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.566+0000 7f8efcff9640 1 --2- 172.21.15.98:0/2224543636 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8ee0067b00 0x7f8ee0069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:14.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.566+0000 7f8efcff9640 1 -- 172.21.15.98:0/2224543636 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f8ef80856b0 con 0x7f8f00158fd0 2024-09-19T08:42:14.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.566+0000 7f8eff7fe640 1 --2- 172.21.15.98:0/2224543636 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8ee0067b00 0x7f8ee0069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:14.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.566+0000 7f8eff7fe640 1 --2- 172.21.15.98:0/2224543636 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8ee0067b00 0x7f8ee0069fc0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8ef00096f0 tx=0x7f8ef0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:14.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.570+0000 7f8efcff9640 1 -- 172.21.15.98:0/2224543636 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f8ef8055e90 con 0x7f8f00158fd0 2024-09-19T08:42:14.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.746+0000 7f8f061f9640 1 -- 172.21.15.98:0/2224543636 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8f000757a0 con 0x7f8f00158fd0 2024-09-19T08:42:14.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.746+0000 7f8efcff9640 1 -- 172.21.15.98:0/2224543636 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v57) ==== 80+0+0 (secure 0 0 0) 0x7f8ef8055790 con 0x7f8f00158fd0 2024-09-19T08:42:14.748 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-19T08:42:14.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.750+0000 7f8f061f9640 1 -- 172.21.15.98:0/2224543636 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8ee0067b00 msgr2=0x7f8ee0069fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:14.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.750+0000 7f8f061f9640 1 --2- 172.21.15.98:0/2224543636 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8ee0067b00 0x7f8ee0069fc0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8ef00096f0 tx=0x7f8ef0008040 comp rx=0 tx=0).stop 2024-09-19T08:42:14.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.750+0000 7f8f061f9640 1 -- 172.21.15.98:0/2224543636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f00158fd0 msgr2=0x7f8f001600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:14.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.750+0000 7f8f061f9640 1 --2- 172.21.15.98:0/2224543636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f00158fd0 0x7f8f001600d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800c7d0 tx=0x7f8ef800cc90 comp rx=0 tx=0).stop 2024-09-19T08:42:14.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.750+0000 7f8f061f9640 1 -- 172.21.15.98:0/2224543636 shutdown_connections 2024-09-19T08:42:14.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.750+0000 7f8f061f9640 1 --2- 172.21.15.98:0/2224543636 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f8ee0067b00 0x7f8ee0069fc0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:14.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.750+0000 7f8f061f9640 1 --2- 172.21.15.98:0/2224543636 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8f00158fd0 0x7f8f001600d0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:14.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.750+0000 7f8f061f9640 1 --2- 172.21.15.98:0/2224543636 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8f000b4360 0x7f8f00155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:14.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.750+0000 7f8f061f9640 1 --2- 172.21.15.98:0/2224543636 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8f0006c1d0 0x7f8f000ada20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:14.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.750+0000 7f8f061f9640 1 -- 172.21.15.98:0/2224543636 >> 172.21.15.98:0/2224543636 conn(0x7f8f000b2010 msgr2=0x7f8f000b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:14.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.750+0000 7f8f061f9640 1 -- 172.21.15.98:0/2224543636 shutdown_connections 2024-09-19T08:42:14.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:14.750+0000 7f8f061f9640 1 -- 172.21.15.98:0/2224543636 wait complete. 2024-09-19T08:42:14.767 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi098.front.sepia.ceph.com... 2024-09-19T08:42:14.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:14.767 DEBUG:teuthology.orchestra.run.smithi098:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T08:42:14.777 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:ceph-fuse[41283]: fuse finished with error 0 and tester_r 0 2024-09-19T08:42:14.779 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:42:14.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:14.779 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T08:42:14.942 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:42:14.943 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:42:14.943 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:42:14.950 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T08:42:15.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.110+0000 7f5c484f9640 1 Processor -- start 2024-09-19T08:42:15.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.110+0000 7f5c484f9640 1 -- start start 2024-09-19T08:42:15.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.110+0000 7f5c484f9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c40141110 0x7f5c40165940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:15.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.110+0000 7f5c484f9640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c4015ea10 con 0x7f5c40140d30 2024-09-19T08:42:15.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.110+0000 7f5c484f9640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c4015ecf0 con 0x7f5c4006d880 2024-09-19T08:42:15.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.110+0000 7f5c484f9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5c4015e7c0 con 0x7f5c40141110 2024-09-19T08:42:15.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.110+0000 7f5c46295640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f5c40140d30 0x7f5c4006d170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:35944/0 (socket says 172.21.15.98:35944) 2024-09-19T08:42:15.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.110+0000 7f5c46295640 1 -- 172.21.15.98:0/1201916183 learned_addr learned my addr 172.21.15.98:0/1201916183 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:15.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.110+0000 7f5c46a96640 1 --2- 172.21.15.98:0/1201916183 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c40141110 0x7f5c40165940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:15.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.110+0000 7f5c45293640 1 -- 172.21.15.98:0/1201916183 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1810860733 0 0) 0x7f5c4015ea10 con 0x7f5c40140d30 2024-09-19T08:42:15.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.110+0000 7f5c45293640 1 -- 172.21.15.98:0/1201916183 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c2c003530 con 0x7f5c40140d30 2024-09-19T08:42:15.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.110+0000 7f5c45293640 1 -- 172.21.15.98:0/1201916183 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4170095451 0 0) 0x7f5c4015ecf0 con 0x7f5c4006d880 2024-09-19T08:42:15.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.110+0000 7f5c45293640 1 -- 172.21.15.98:0/1201916183 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c4015ea10 con 0x7f5c4006d880 2024-09-19T08:42:15.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.110+0000 7f5c45293640 1 -- 172.21.15.98:0/1201916183 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2590460900 0 0) 0x7f5c2c003530 con 0x7f5c40140d30 2024-09-19T08:42:15.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.110+0000 7f5c45293640 1 -- 172.21.15.98:0/1201916183 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c40141110 msgr2=0x7f5c40165940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:15.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.110+0000 7f5c45293640 1 --2- 172.21.15.98:0/1201916183 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c40141110 0x7f5c40165940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:15.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.110+0000 7f5c45293640 1 -- 172.21.15.98:0/1201916183 >> v1:172.21.15.139:6789/0 conn(0x7f5c4006d880 legacy=0x7f5c400b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:15.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.110+0000 7f5c45293640 1 -- 172.21.15.98:0/1201916183 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c4015efd0 con 0x7f5c40140d30 2024-09-19T08:42:15.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.110+0000 7f5c45293640 1 -- 172.21.15.98:0/1201916183 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f5c3c0016a0 con 0x7f5c40140d30 2024-09-19T08:42:15.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.110+0000 7f5c45293640 1 -- 172.21.15.98:0/1201916183 >> v1:172.21.15.98:6789/0 conn(0x7f5c40140d30 legacy=0x7f5c4006d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:15.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.110+0000 7f5c45293640 1 --2- 172.21.15.98:0/1201916183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c2c003b80 0x7f5c2c003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:15.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.110+0000 7f5c45293640 1 --2- 172.21.15.98:0/1201916183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c2c004810 0x7f5c2c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:15.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.110+0000 7f5c45293640 1 --2- 172.21.15.98:0/1201916183 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c40141110 0x7f5c2c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:15.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.110+0000 7f5c45293640 1 -- 172.21.15.98:0/1201916183 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4015e7c0 con 0x7f5c2c003b80 2024-09-19T08:42:15.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.110+0000 7f5c45293640 1 -- 172.21.15.98:0/1201916183 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c00ba80 con 0x7f5c2c004810 2024-09-19T08:42:15.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.110+0000 7f5c45293640 1 -- 172.21.15.98:0/1201916183 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5c2c00bc80 con 0x7f5c40141110 2024-09-19T08:42:15.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.110+0000 7f5c46295640 1 --2- 172.21.15.98:0/1201916183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c2c003b80 0x7f5c2c003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:15.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.110+0000 7f5c45a94640 1 --2- 172.21.15.98:0/1201916183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c2c004810 0x7f5c2c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:15.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.110+0000 7f5c46a96640 1 --2- 172.21.15.98:0/1201916183 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c40141110 0x7f5c2c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:15.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.110+0000 7f5c46295640 1 -- 172.21.15.98:0/1201916183 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c40141110 msgr2=0x7f5c2c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:15.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.110+0000 7f5c46295640 1 --2- 172.21.15.98:0/1201916183 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c40141110 0x7f5c2c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:15.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.110+0000 7f5c46295640 1 -- 172.21.15.98:0/1201916183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c2c004810 msgr2=0x7f5c2c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:15.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.110+0000 7f5c46295640 1 --2- 172.21.15.98:0/1201916183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c2c004810 0x7f5c2c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:15.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.110+0000 7f5c46295640 1 -- 172.21.15.98:0/1201916183 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c2c003530 con 0x7f5c2c003b80 2024-09-19T08:42:15.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.114+0000 7f5c46295640 1 --2- 172.21.15.98:0/1201916183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c2c003b80 0x7f5c2c003f80 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00c0e0 tx=0x7f5c3c01cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccdc82890e8d393a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:15.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.114+0000 7f5c45293640 1 -- 172.21.15.98:0/1201916183 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c007c30 con 0x7f5c2c003b80 2024-09-19T08:42:15.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.114+0000 7f5c45293640 1 -- 172.21.15.98:0/1201916183 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c3c0a9020 con 0x7f5c2c003b80 2024-09-19T08:42:15.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.114+0000 7f5c45293640 1 -- 172.21.15.98:0/1201916183 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c005b20 con 0x7f5c2c003b80 2024-09-19T08:42:15.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.114+0000 7f5c484f9640 1 -- 172.21.15.98:0/1201916183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c2c003b80 msgr2=0x7f5c2c003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:15.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.114+0000 7f5c484f9640 1 --2- 172.21.15.98:0/1201916183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c2c003b80 0x7f5c2c003f80 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00c0e0 tx=0x7f5c3c01cb20 comp rx=0 tx=0).stop 2024-09-19T08:42:15.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.114+0000 7f5c46a96640 1 -- 172.21.15.98:0/1201916183 reap_dead start 2024-09-19T08:42:15.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.114+0000 7f5c484f9640 1 -- 172.21.15.98:0/1201916183 shutdown_connections 2024-09-19T08:42:15.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.114+0000 7f5c484f9640 1 -- 172.21.15.98:0/1201916183 >> 172.21.15.98:0/1201916183 conn(0x7f5c400b2010 msgr2=0x7f5c400b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:15.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.114+0000 7f5c484f9640 1 -- 172.21.15.98:0/1201916183 shutdown_connections 2024-09-19T08:42:15.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.114+0000 7f5c484f9640 1 -- 172.21.15.98:0/1201916183 wait complete. 2024-09-19T08:42:15.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.118+0000 7f5c484f9640 1 Processor -- start 2024-09-19T08:42:15.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.118+0000 7f5c484f9640 1 -- start start 2024-09-19T08:42:15.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.118+0000 7f5c484f9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c2c003b80 0x7f5c4013fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:15.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.118+0000 7f5c484f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c40168730 0x7f5c400aacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:15.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.118+0000 7f5c484f9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c4013e760 0x7f5c4013ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:15.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.118+0000 7f5c484f9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5c40165d50 con 0x7f5c4013e760 2024-09-19T08:42:15.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.118+0000 7f5c484f9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5c40165bd0 con 0x7f5c40168730 2024-09-19T08:42:15.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.118+0000 7f5c484f9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5c40165ed0 con 0x7f5c2c003b80 2024-09-19T08:42:15.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.118+0000 7f5c46a96640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c4013e760 0x7f5c4013ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:15.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.118+0000 7f5c46a96640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c4013e760 0x7f5c4013ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46132/0 (socket says 172.21.15.98:46132) 2024-09-19T08:42:15.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.118+0000 7f5c46a96640 1 -- 172.21.15.98:0/2423789652 learned_addr learned my addr 172.21.15.98:0/2423789652 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:15.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.118+0000 7f5c46295640 1 --2- 172.21.15.98:0/2423789652 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c2c003b80 0x7f5c4013fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:15.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.118+0000 7f5c45a94640 1 --2- 172.21.15.98:0/2423789652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c40168730 0x7f5c400aacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:15.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.118+0000 7f5c46a96640 1 -- 172.21.15.98:0/2423789652 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c2c003b80 msgr2=0x7f5c4013fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:15.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.118+0000 7f5c46a96640 1 --2- 172.21.15.98:0/2423789652 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c2c003b80 0x7f5c4013fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:15.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.118+0000 7f5c46a96640 1 -- 172.21.15.98:0/2423789652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c40168730 msgr2=0x7f5c400aacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:15.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.118+0000 7f5c46a96640 1 --2- 172.21.15.98:0/2423789652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c40168730 0x7f5c400aacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:15.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.118+0000 7f5c46a96640 1 -- 172.21.15.98:0/2423789652 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c40168ea0 con 0x7f5c4013e760 2024-09-19T08:42:15.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.118+0000 7f5c46a96640 1 --2- 172.21.15.98:0/2423789652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c4013e760 0x7f5c4013ebe0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f5c4006cd60 tx=0x7f5c380115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:15.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.118+0000 7f5c36ffd640 1 -- 172.21.15.98:0/2423789652 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c380039a0 con 0x7f5c4013e760 2024-09-19T08:42:15.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.118+0000 7f5c36ffd640 1 -- 172.21.15.98:0/2423789652 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c3801b070 con 0x7f5c4013e760 2024-09-19T08:42:15.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.118+0000 7f5c36ffd640 1 -- 172.21.15.98:0/2423789652 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c38003b80 con 0x7f5c4013e760 2024-09-19T08:42:15.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.118+0000 7f5c484f9640 1 -- 172.21.15.98:0/2423789652 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c40169180 con 0x7f5c4013e760 2024-09-19T08:42:15.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.118+0000 7f5c484f9640 1 -- 172.21.15.98:0/2423789652 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c40169740 con 0x7f5c4013e760 2024-09-19T08:42:15.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.118+0000 7f5c484f9640 1 -- 172.21.15.98:0/2423789652 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c40169a70 con 0x7f5c4013e760 2024-09-19T08:42:15.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.122+0000 7f5c36ffd640 1 -- 172.21.15.98:0/2423789652 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 10) ==== 87649+0+0 (secure 0 0 0) 0x7f5c38004030 con 0x7f5c4013e760 2024-09-19T08:42:15.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.122+0000 7f5c36ffd640 1 --2- 172.21.15.98:0/2423789652 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5c24067b30 0x7f5c24069ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:15.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.122+0000 7f5c36ffd640 1 -- 172.21.15.98:0/2423789652 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f5c38086a80 con 0x7f5c4013e760 2024-09-19T08:42:15.122 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.122+0000 7f5c46295640 1 --2- 172.21.15.98:0/2423789652 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5c24067b30 0x7f5c24069ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:15.122 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.122+0000 7f5c46295640 1 --2- 172.21.15.98:0/2423789652 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5c24067b30 0x7f5c24069ff0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c0aa000 tx=0x7f5c4015ebe0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:15.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.126+0000 7f5c36ffd640 1 -- 172.21.15.98:0/2423789652 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f5c38057150 con 0x7f5c4013e760 2024-09-19T08:42:15.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.266+0000 7f5c36ffd640 1 -- 172.21.15.98:0/2423789652 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f5c380530c0 con 0x7f5c4013e760 2024-09-19T08:42:15.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.298+0000 7f5c484f9640 1 -- 172.21.15.98:0/2423789652 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5c400b3310 con 0x7f5c4013e760 2024-09-19T08:42:15.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.302+0000 7f5c36ffd640 1 -- 172.21.15.98:0/2423789652 <== mon.0 v2:172.21.15.98:3300/0 8 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v57) ==== 80+0+0 (secure 0 0 0) 0x7f5c38084650 con 0x7f5c4013e760 2024-09-19T08:42:15.301 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-19T08:42:15.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.302+0000 7f5c484f9640 1 -- 172.21.15.98:0/2423789652 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5c24067b30 msgr2=0x7f5c24069ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:15.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.302+0000 7f5c484f9640 1 --2- 172.21.15.98:0/2423789652 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5c24067b30 0x7f5c24069ff0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c0aa000 tx=0x7f5c4015ebe0 comp rx=0 tx=0).stop 2024-09-19T08:42:15.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.302+0000 7f5c484f9640 1 -- 172.21.15.98:0/2423789652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c4013e760 msgr2=0x7f5c4013ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:15.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.302+0000 7f5c484f9640 1 --2- 172.21.15.98:0/2423789652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c4013e760 0x7f5c4013ebe0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f5c4006cd60 tx=0x7f5c380115a0 comp rx=0 tx=0).stop 2024-09-19T08:42:15.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.302+0000 7f5c484f9640 1 -- 172.21.15.98:0/2423789652 shutdown_connections 2024-09-19T08:42:15.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.302+0000 7f5c484f9640 1 --2- 172.21.15.98:0/2423789652 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5c24067b30 0x7f5c24069ff0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:15.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.302+0000 7f5c484f9640 1 --2- 172.21.15.98:0/2423789652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c4013e760 0x7f5c4013ebe0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:15.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.302+0000 7f5c484f9640 1 --2- 172.21.15.98:0/2423789652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c40168730 0x7f5c400aacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:15.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.302+0000 7f5c484f9640 1 --2- 172.21.15.98:0/2423789652 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c2c003b80 0x7f5c4013fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:15.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.302+0000 7f5c484f9640 1 -- 172.21.15.98:0/2423789652 >> 172.21.15.98:0/2423789652 conn(0x7f5c400b2010 msgr2=0x7f5c400b0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:15.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.302+0000 7f5c484f9640 1 -- 172.21.15.98:0/2423789652 shutdown_connections 2024-09-19T08:42:15.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.302+0000 7f5c484f9640 1 -- 172.21.15.98:0/2423789652 wait complete. 2024-09-19T08:42:15.318 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi139.front.sepia.ceph.com... 2024-09-19T08:42:15.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:15.319 DEBUG:teuthology.orchestra.run.smithi139:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-19T08:42:15.332 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[35574]: fuse finished with error 0 and tester_r 0 2024-09-19T08:42:15.334 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:42:15.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:15.334 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-19T08:42:15.504 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:42:15.504 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:42:15.504 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:42:15.511 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15846 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T08:42:15.511 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:42:15.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:15.511 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-19T08:42:15.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:42:15.558 INFO:teuthology.orchestra.run.smithi098.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-19T08:42:15.558 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:42:15.558 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:42:15.559 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:42:15.565 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T08:42:15.565 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:42:15.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:15.565 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-19T08:42:15.614 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-19T08:42:15.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:42:15.616 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:42:15.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.745+0000 7fb10905d640 1 Processor -- start 2024-09-19T08:42:15.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.745+0000 7fb10905d640 1 -- start start 2024-09-19T08:42:15.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.745+0000 7fb10905d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb10406c3a0 0x7fb1040cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:15.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.745+0000 7fb10905d640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb10406d310 con 0x7fb10415c550 2024-09-19T08:42:15.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.745+0000 7fb10905d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb10406d5f0 con 0x7fb1040cf900 2024-09-19T08:42:15.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.745+0000 7fb10905d640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fb10406d8d0 con 0x7fb10406c3a0 2024-09-19T08:42:15.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.745+0000 7fb102d9d640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fb10415c550 0x7fb10406bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:35958/0 (socket says 172.21.15.98:35958) 2024-09-19T08:42:15.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.745+0000 7fb102d9d640 1 -- 172.21.15.98:0/950627829 learned_addr learned my addr 172.21.15.98:0/950627829 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:15.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.745+0000 7fb10259c640 1 --2- 172.21.15.98:0/950627829 >> v2:172.21.15.98:3300/0 conn(0x7fb10406c3a0 0x7fb1040cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:15.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.745+0000 7fb10259c640 1 -- 172.21.15.98:0/950627829 >> v1:172.21.15.139:6789/0 conn(0x7fb1040cf900 legacy=0x7fb1040723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:15.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.745+0000 7fb10259c640 1 -- 172.21.15.98:0/950627829 >> v1:172.21.15.98:6789/0 conn(0x7fb10415c550 legacy=0x7fb10406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:15.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.745+0000 7fb10259c640 1 -- 172.21.15.98:0/950627829 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb10406dbb0 con 0x7fb10406c3a0 2024-09-19T08:42:15.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.745+0000 7fb101d9b640 1 -- 172.21.15.98:0/950627829 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1476503060 0 0) 0x7fb10406d310 con 0x7fb10415c550 2024-09-19T08:42:15.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.749+0000 7fb10259c640 1 --2- 172.21.15.98:0/950627829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb10406c3a0 0x7fb1040cf3c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8009d90 tx=0x7fb0f801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f2bd1cf0f23f89 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:15.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.749+0000 7fb101d9b640 1 -- 172.21.15.98:0/950627829 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f80b2070 con 0x7fb10406c3a0 2024-09-19T08:42:15.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.749+0000 7fb101d9b640 1 -- 172.21.15.98:0/950627829 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0f80a5900 con 0x7fb10406c3a0 2024-09-19T08:42:15.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.749+0000 7fb101d9b640 1 -- 172.21.15.98:0/950627829 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f80a5cd0 con 0x7fb10406c3a0 2024-09-19T08:42:15.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.749+0000 7fb10905d640 1 -- 172.21.15.98:0/950627829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb10406c3a0 msgr2=0x7fb1040cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:15.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.749+0000 7fb10905d640 1 --2- 172.21.15.98:0/950627829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb10406c3a0 0x7fb1040cf3c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8009d90 tx=0x7fb0f801c930 comp rx=0 tx=0).stop 2024-09-19T08:42:15.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.749+0000 7fb10905d640 1 -- 172.21.15.98:0/950627829 shutdown_connections 2024-09-19T08:42:15.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.749+0000 7fb10905d640 1 --2- 172.21.15.98:0/950627829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb10406c3a0 0x7fb1040cf3c0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:15.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.749+0000 7fb10905d640 1 -- 172.21.15.98:0/950627829 >> 172.21.15.98:0/950627829 conn(0x7fb1040b2010 msgr2=0x7fb1040b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:15.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.749+0000 7fb10905d640 1 -- 172.21.15.98:0/950627829 shutdown_connections 2024-09-19T08:42:15.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.749+0000 7fb10905d640 1 -- 172.21.15.98:0/950627829 wait complete. 2024-09-19T08:42:15.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.749+0000 7fb10905d640 1 Processor -- start 2024-09-19T08:42:15.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.749+0000 7fb10905d640 1 -- start start 2024-09-19T08:42:15.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.749+0000 7fb10905d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb10406c3a0 0x7fb104154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:15.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.749+0000 7fb10905d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1040cf900 0x7fb10415f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:15.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.749+0000 7fb10905d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb10415c550 0x7fb10414c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:15.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.749+0000 7fb10905d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb1040736d0 con 0x7fb1040cf900 2024-09-19T08:42:15.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.749+0000 7fb10905d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb104073550 con 0x7fb10406c3a0 2024-09-19T08:42:15.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.749+0000 7fb10905d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb10416a240 con 0x7fb10415c550 2024-09-19T08:42:15.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.749+0000 7fb10259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1040cf900 0x7fb10415f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:15.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.749+0000 7fb10259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1040cf900 0x7fb10415f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46142/0 (socket says 172.21.15.98:46142) 2024-09-19T08:42:15.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.749+0000 7fb10259c640 1 -- 172.21.15.98:0/946257417 learned_addr learned my addr 172.21.15.98:0/946257417 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:15.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.749+0000 7fb102d9d640 1 --2- 172.21.15.98:0/946257417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb10406c3a0 0x7fb104154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:15.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.749+0000 7fb10359e640 1 --2- 172.21.15.98:0/946257417 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb10415c550 0x7fb10414c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:15.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.753+0000 7fb10259c640 1 -- 172.21.15.98:0/946257417 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb10415c550 msgr2=0x7fb10414c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:15.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.753+0000 7fb10259c640 1 --2- 172.21.15.98:0/946257417 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb10415c550 0x7fb10414c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:15.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.753+0000 7fb10259c640 1 -- 172.21.15.98:0/946257417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb10406c3a0 msgr2=0x7fb104154160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:15.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.753+0000 7fb10259c640 1 --2- 172.21.15.98:0/946257417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb10406c3a0 0x7fb104154160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:15.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.753+0000 7fb10259c640 1 -- 172.21.15.98:0/946257417 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1041511c0 con 0x7fb1040cf900 2024-09-19T08:42:15.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.753+0000 7fb10259c640 1 --2- 172.21.15.98:0/946257417 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1040cf900 0x7fb10415f5b0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb0f80b0040 tx=0x7fb0f80196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:15.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.753+0000 7fb0eb7fe640 1 -- 172.21.15.98:0/946257417 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f80b2050 con 0x7fb1040cf900 2024-09-19T08:42:15.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.753+0000 7fb0eb7fe640 1 -- 172.21.15.98:0/946257417 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0f80be070 con 0x7fb1040cf900 2024-09-19T08:42:15.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.753+0000 7fb10905d640 1 -- 172.21.15.98:0/946257417 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1041514a0 con 0x7fb1040cf900 2024-09-19T08:42:15.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.753+0000 7fb0eb7fe640 1 -- 172.21.15.98:0/946257417 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f80adb10 con 0x7fb1040cf900 2024-09-19T08:42:15.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.753+0000 7fb10905d640 1 -- 172.21.15.98:0/946257417 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb104151a60 con 0x7fb1040cf900 2024-09-19T08:42:15.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.753+0000 7fb10905d640 1 -- 172.21.15.98:0/946257417 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1040cf3c0 con 0x7fb1040cf900 2024-09-19T08:42:15.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.753+0000 7fb0eb7fe640 1 -- 172.21.15.98:0/946257417 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fb0f80adcb0 con 0x7fb1040cf900 2024-09-19T08:42:15.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.753+0000 7fb0eb7fe640 1 --2- 172.21.15.98:0/946257417 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb0f0067c20 0x7fb0f006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:15.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.753+0000 7fb0eb7fe640 1 -- 172.21.15.98:0/946257417 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fb0f8121410 con 0x7fb1040cf900 2024-09-19T08:42:15.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.753+0000 7fb102d9d640 1 --2- 172.21.15.98:0/946257417 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb0f0067c20 0x7fb0f006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:15.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.753+0000 7fb102d9d640 1 --2- 172.21.15.98:0/946257417 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb0f0067c20 0x7fb0f006a0e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc000f70 tx=0x7fb0fc007df0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:15.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.757+0000 7fb0eb7fe640 1 -- 172.21.15.98:0/946257417 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb0f8016520 con 0x7fb1040cf900 2024-09-19T08:42:15.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.957+0000 7fb10905d640 1 -- 172.21.15.98:0/946257417 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb10414fd00 con 0x7fb1040cf900 2024-09-19T08:42:15.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.957+0000 7fb0eb7fe640 1 -- 172.21.15.98:0/946257417 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+5229 (secure 0 0 0) 0x7fb0f80f1500 con 0x7fb1040cf900 2024-09-19T08:42:15.959 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:42:15.959 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":31,"btime":"2024-09-19T08:42:03:179377+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4880,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/1549394645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":1549394645},{"type":"v1","addr":"172.21.15.139:6834","nonce":1549394645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":4883,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1847525681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6833","nonce":1847525681},{"type":"v1","addr":"172.21.15.139:6835","nonce":1847525681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5745,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/1677812726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1677812726},{"type":"v1","addr":"172.21.15.98:6835","nonce":1677812726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:41:47.304224+0000","modified":"2024-09-19T08:41:49.306408+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/3548234111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3548234111},{"type":"v1","addr":"172.21.15.98:6837","nonce":3548234111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5748,"qdb_cluster":[5748]},"id":4}]} 2024-09-19T08:42:15.960 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 31 2024-09-19T08:42:15.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.961+0000 7fb10905d640 1 -- 172.21.15.98:0/946257417 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb0f0067c20 msgr2=0x7fb0f006a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:15.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.961+0000 7fb10905d640 1 --2- 172.21.15.98:0/946257417 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb0f0067c20 0x7fb0f006a0e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc000f70 tx=0x7fb0fc007df0 comp rx=0 tx=0).stop 2024-09-19T08:42:15.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.961+0000 7fb10905d640 1 -- 172.21.15.98:0/946257417 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1040cf900 msgr2=0x7fb10415f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:15.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.961+0000 7fb10905d640 1 --2- 172.21.15.98:0/946257417 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1040cf900 0x7fb10415f5b0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb0f80b0040 tx=0x7fb0f80196d0 comp rx=0 tx=0).stop 2024-09-19T08:42:15.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.961+0000 7fb10905d640 1 -- 172.21.15.98:0/946257417 shutdown_connections 2024-09-19T08:42:15.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.961+0000 7fb10905d640 1 --2- 172.21.15.98:0/946257417 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb0f0067c20 0x7fb0f006a0e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:15.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.961+0000 7fb10905d640 1 --2- 172.21.15.98:0/946257417 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb10415c550 0x7fb10414c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:15.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.961+0000 7fb10905d640 1 --2- 172.21.15.98:0/946257417 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1040cf900 0x7fb10415f5b0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:15.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.961+0000 7fb10905d640 1 --2- 172.21.15.98:0/946257417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb10406c3a0 0x7fb104154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:15.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.961+0000 7fb10905d640 1 -- 172.21.15.98:0/946257417 >> 172.21.15.98:0/946257417 conn(0x7fb1040b2010 msgr2=0x7fb10406bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:15.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.961+0000 7fb10905d640 1 -- 172.21.15.98:0/946257417 shutdown_connections 2024-09-19T08:42:15.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:15.961+0000 7fb10905d640 1 -- 172.21.15.98:0/946257417 wait complete. 2024-09-19T08:42:15.979 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:42:16.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.097+0000 7f388c74c640 1 Processor -- start 2024-09-19T08:42:16.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.097+0000 7f388c74c640 1 -- start start 2024-09-19T08:42:16.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.097+0000 7f388c74c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3884158e40 0x7f388406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:16.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.097+0000 7f388c74c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f388406c1d0 0x7f38840ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:16.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.097+0000 7f388c74c640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38840ceb70 con 0x7f38840ce6e0 2024-09-19T08:42:16.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.097+0000 7f388c74c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f388406c610 con 0x7f3884158e40 2024-09-19T08:42:16.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.097+0000 7f388c74c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f388415fcf0 con 0x7f388406c1d0 2024-09-19T08:42:16.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.097+0000 7f388ace9640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f38840ce6e0 0x7f388406d170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:35974/0 (socket says 172.21.15.98:35974) 2024-09-19T08:42:16.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.097+0000 7f388ace9640 1 -- 172.21.15.98:0/3148446604 learned_addr learned my addr 172.21.15.98:0/3148446604 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:16.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.097+0000 7f388a4e8640 1 --2- 172.21.15.98:0/3148446604 >> v2:172.21.15.139:3300/0 conn(0x7f3884158e40 0x7f388406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:16.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.097+0000 7f3889ce7640 1 --2- 172.21.15.98:0/3148446604 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f388406c1d0 0x7f38840ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:16.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.097+0000 7f38894e6640 1 -- 172.21.15.98:0/3148446604 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 61601164 0 0) 0x7f38840ceb70 con 0x7f38840ce6e0 2024-09-19T08:42:16.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.097+0000 7f38894e6640 1 -- 172.21.15.98:0/3148446604 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3870003530 con 0x7f38840ce6e0 2024-09-19T08:42:16.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.097+0000 7f38894e6640 1 -- 172.21.15.98:0/3148446604 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3521816935 0 0) 0x7f3870003530 con 0x7f38840ce6e0 2024-09-19T08:42:16.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.097+0000 7f38894e6640 1 -- 172.21.15.98:0/3148446604 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f388406c1d0 msgr2=0x7f38840ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:16.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.097+0000 7f38894e6640 1 --2- 172.21.15.98:0/3148446604 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f388406c1d0 0x7f38840ce1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:16.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.097+0000 7f38894e6640 1 -- 172.21.15.98:0/3148446604 >> v2:172.21.15.139:3300/0 conn(0x7f3884158e40 msgr2=0x7f388406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:16.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.097+0000 7f38894e6640 1 --2- 172.21.15.98:0/3148446604 >> v2:172.21.15.139:3300/0 conn(0x7f3884158e40 0x7f388406bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:16.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.097+0000 7f38894e6640 1 -- 172.21.15.98:0/3148446604 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388415ef30 con 0x7f38840ce6e0 2024-09-19T08:42:16.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.097+0000 7f38894e6640 1 -- 172.21.15.98:0/3148446604 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f387c001a00 con 0x7f38840ce6e0 2024-09-19T08:42:16.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.097+0000 7f38894e6640 1 -- 172.21.15.98:0/3148446604 >> v1:172.21.15.98:6789/0 conn(0x7f38840ce6e0 legacy=0x7f388406d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:16.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.097+0000 7f38894e6640 1 --2- 172.21.15.98:0/3148446604 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f388406c1d0 0x7f38700039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:16.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.097+0000 7f38894e6640 1 --2- 172.21.15.98:0/3148446604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3870003f90 0x7f3870004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:16.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.097+0000 7f38894e6640 1 --2- 172.21.15.98:0/3148446604 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3870004d50 0x7f38700051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:16.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.097+0000 7f38894e6640 1 -- 172.21.15.98:0/3148446604 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f388406c610 con 0x7f3870003f90 2024-09-19T08:42:16.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.097+0000 7f38894e6640 1 -- 172.21.15.98:0/3148446604 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f388415fcf0 con 0x7f3870004d50 2024-09-19T08:42:16.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.097+0000 7f38894e6640 1 -- 172.21.15.98:0/3148446604 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f387000ba50 con 0x7f388406c1d0 2024-09-19T08:42:16.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.097+0000 7f3889ce7640 1 --2- 172.21.15.98:0/3148446604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3870003f90 0x7f3870004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:16.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.097+0000 7f388a4e8640 1 --2- 172.21.15.98:0/3148446604 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f388406c1d0 0x7f38700039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:16.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.097+0000 7f388ace9640 1 --2- 172.21.15.98:0/3148446604 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3870004d50 0x7f38700051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:16.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.097+0000 7f3889ce7640 1 -- 172.21.15.98:0/3148446604 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f388406c1d0 msgr2=0x7f38700039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:16.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.097+0000 7f3889ce7640 1 --2- 172.21.15.98:0/3148446604 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f388406c1d0 0x7f38700039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:16.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.097+0000 7f3889ce7640 1 -- 172.21.15.98:0/3148446604 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3870004d50 msgr2=0x7f38700051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:16.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.097+0000 7f3889ce7640 1 --2- 172.21.15.98:0/3148446604 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3870004d50 0x7f38700051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:16.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.097+0000 7f3889ce7640 1 -- 172.21.15.98:0/3148446604 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3870003530 con 0x7f3870003f90 2024-09-19T08:42:16.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.101+0000 7f3889ce7640 1 --2- 172.21.15.98:0/3148446604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3870003f90 0x7f3870004810 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3874009ae0 tx=0x7f387401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f570e695b8aeb44 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:16.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.101+0000 7f38894e6640 1 -- 172.21.15.98:0/3148446604 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38740b2070 con 0x7f3870003f90 2024-09-19T08:42:16.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.101+0000 7f38894e6640 1 -- 172.21.15.98:0/3148446604 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38740a5660 con 0x7f3870003f90 2024-09-19T08:42:16.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.101+0000 7f38894e6640 1 -- 172.21.15.98:0/3148446604 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38740096b0 con 0x7f3870003f90 2024-09-19T08:42:16.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.101+0000 7f388c74c640 1 -- 172.21.15.98:0/3148446604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3870003f90 msgr2=0x7f3870004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:16.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.101+0000 7f388c74c640 1 --2- 172.21.15.98:0/3148446604 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3870003f90 0x7f3870004810 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3874009ae0 tx=0x7f387401ca70 comp rx=0 tx=0).stop 2024-09-19T08:42:16.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.101+0000 7f388ace9640 1 -- 172.21.15.98:0/3148446604 reap_dead start 2024-09-19T08:42:16.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.101+0000 7f388c74c640 1 -- 172.21.15.98:0/3148446604 shutdown_connections 2024-09-19T08:42:16.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.101+0000 7f388c74c640 1 -- 172.21.15.98:0/3148446604 >> 172.21.15.98:0/3148446604 conn(0x7f38840b2010 msgr2=0x7f38840b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:16.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.101+0000 7f388c74c640 1 -- 172.21.15.98:0/3148446604 shutdown_connections 2024-09-19T08:42:16.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.101+0000 7f388c74c640 1 -- 172.21.15.98:0/3148446604 wait complete. 2024-09-19T08:42:16.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.101+0000 7f388c74c640 1 Processor -- start 2024-09-19T08:42:16.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.101+0000 7f388c74c640 1 -- start start 2024-09-19T08:42:16.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.105+0000 7f388c74c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3870003f90 0x7f388414d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:16.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.105+0000 7f388c74c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3884157680 0x7f388414f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:16.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.105+0000 7f388c74c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3884157b00 0x7f388414f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:16.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.105+0000 7f388c74c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f388415f7d0 con 0x7f3870003f90 2024-09-19T08:42:16.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.105+0000 7f388c74c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f388415f4f0 con 0x7f3884157b00 2024-09-19T08:42:16.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.105+0000 7f388c74c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3884159220 con 0x7f3884157680 2024-09-19T08:42:16.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.105+0000 7f388a4e8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3870003f90 0x7f388414d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:16.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.105+0000 7f388a4e8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3870003f90 0x7f388414d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46150/0 (socket says 172.21.15.98:46150) 2024-09-19T08:42:16.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.105+0000 7f388a4e8640 1 -- 172.21.15.98:0/2925865347 learned_addr learned my addr 172.21.15.98:0/2925865347 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:16.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.105+0000 7f3889ce7640 1 --2- 172.21.15.98:0/2925865347 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3884157680 0x7f388414f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:16.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.105+0000 7f388ace9640 1 --2- 172.21.15.98:0/2925865347 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3884157b00 0x7f388414f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:16.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.105+0000 7f388a4e8640 1 -- 172.21.15.98:0/2925865347 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3884157680 msgr2=0x7f388414f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:16.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.105+0000 7f388a4e8640 1 --2- 172.21.15.98:0/2925865347 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3884157680 0x7f388414f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:16.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.105+0000 7f388a4e8640 1 -- 172.21.15.98:0/2925865347 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3884157b00 msgr2=0x7f388414f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:16.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.105+0000 7f388a4e8640 1 --2- 172.21.15.98:0/2925865347 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3884157b00 0x7f388414f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:16.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.105+0000 7f388a4e8640 1 -- 172.21.15.98:0/2925865347 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38841f4680 con 0x7f3870003f90 2024-09-19T08:42:16.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.105+0000 7f388a4e8640 1 --2- 172.21.15.98:0/2925865347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3870003f90 0x7f388414d5e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3880011660 tx=0x7f3880011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:16.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.105+0000 7f387affd640 1 -- 172.21.15.98:0/2925865347 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388000f9c0 con 0x7f3870003f90 2024-09-19T08:42:16.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.105+0000 7f387affd640 1 -- 172.21.15.98:0/2925865347 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f388000fb60 con 0x7f3870003f90 2024-09-19T08:42:16.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.105+0000 7f388c74c640 1 -- 172.21.15.98:0/2925865347 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38841f4960 con 0x7f3870003f90 2024-09-19T08:42:16.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.105+0000 7f387affd640 1 -- 172.21.15.98:0/2925865347 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3880015460 con 0x7f3870003f90 2024-09-19T08:42:16.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.105+0000 7f388c74c640 1 -- 172.21.15.98:0/2925865347 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38841f4f20 con 0x7f3870003f90 2024-09-19T08:42:16.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.105+0000 7f388c74c640 1 -- 172.21.15.98:0/2925865347 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38840cee20 con 0x7f3870003f90 2024-09-19T08:42:16.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.105+0000 7f387affd640 1 -- 172.21.15.98:0/2925865347 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f3880015600 con 0x7f3870003f90 2024-09-19T08:42:16.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.105+0000 7f387affd640 1 --2- 172.21.15.98:0/2925865347 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3868067c50 0x7f386806a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:16.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.105+0000 7f387affd640 1 -- 172.21.15.98:0/2925865347 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f3880086b00 con 0x7f3870003f90 2024-09-19T08:42:16.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.105+0000 7f3889ce7640 1 --2- 172.21.15.98:0/2925865347 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3868067c50 0x7f386806a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:16.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.105+0000 7f3889ce7640 1 --2- 172.21.15.98:0/2925865347 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3868067c50 0x7f386806a110 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f38740af000 tx=0x7f38740046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:16.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.109+0000 7f387affd640 1 -- 172.21.15.98:0/2925865347 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3880019030 con 0x7f3870003f90 2024-09-19T08:42:16.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.305+0000 7f388c74c640 1 -- 172.21.15.98:0/2925865347 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38840cfc10 con 0x7f3870003f90 2024-09-19T08:42:16.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.305+0000 7f387affd640 1 -- 172.21.15.98:0/2925865347 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+5229 (secure 0 0 0) 0x7f38800572c0 con 0x7f3870003f90 2024-09-19T08:42:16.307 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:42:16.307 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":31,"btime":"2024-09-19T08:42:03:179377+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4880,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/1549394645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":1549394645},{"type":"v1","addr":"172.21.15.139:6834","nonce":1549394645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":4883,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1847525681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6833","nonce":1847525681},{"type":"v1","addr":"172.21.15.139:6835","nonce":1847525681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5745,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/1677812726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1677812726},{"type":"v1","addr":"172.21.15.98:6835","nonce":1677812726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:41:47.304224+0000","modified":"2024-09-19T08:41:49.306408+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/3548234111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3548234111},{"type":"v1","addr":"172.21.15.98:6837","nonce":3548234111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5748,"qdb_cluster":[5748]},"id":4}]} 2024-09-19T08:42:16.307 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 31 2024-09-19T08:42:16.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.309+0000 7f388c74c640 1 -- 172.21.15.98:0/2925865347 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3868067c50 msgr2=0x7f386806a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:16.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.309+0000 7f388c74c640 1 --2- 172.21.15.98:0/2925865347 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3868067c50 0x7f386806a110 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f38740af000 tx=0x7f38740046d0 comp rx=0 tx=0).stop 2024-09-19T08:42:16.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.309+0000 7f388c74c640 1 -- 172.21.15.98:0/2925865347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3870003f90 msgr2=0x7f388414d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:16.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.309+0000 7f388c74c640 1 --2- 172.21.15.98:0/2925865347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3870003f90 0x7f388414d5e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f3880011660 tx=0x7f3880011b20 comp rx=0 tx=0).stop 2024-09-19T08:42:16.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.309+0000 7f388c74c640 1 -- 172.21.15.98:0/2925865347 shutdown_connections 2024-09-19T08:42:16.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.309+0000 7f388c74c640 1 --2- 172.21.15.98:0/2925865347 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3868067c50 0x7f386806a110 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:16.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.309+0000 7f388c74c640 1 --2- 172.21.15.98:0/2925865347 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3884157b00 0x7f388414f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:16.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.309+0000 7f388c74c640 1 --2- 172.21.15.98:0/2925865347 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3884157680 0x7f388414f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:16.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.309+0000 7f388c74c640 1 --2- 172.21.15.98:0/2925865347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3870003f90 0x7f388414d5e0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:16.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.309+0000 7f388c74c640 1 -- 172.21.15.98:0/2925865347 >> 172.21.15.98:0/2925865347 conn(0x7f38840b2010 msgr2=0x7f388415e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:16.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.309+0000 7f388c74c640 1 -- 172.21.15.98:0/2925865347 shutdown_connections 2024-09-19T08:42:16.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.309+0000 7f388c74c640 1 -- 172.21.15.98:0/2925865347 wait complete. 2024-09-19T08:42:16.327 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:42:16.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.445+0000 7f6a3a9aa640 1 Processor -- start 2024-09-19T08:42:16.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.445+0000 7f6a3a9aa640 1 -- start start 2024-09-19T08:42:16.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.445+0000 7f6a3a9aa640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6a3406c3a0 0x7f6a340cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:16.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.445+0000 7f6a3a9aa640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a340cf900 0x7f6a340723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:16.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.445+0000 7f6a3a9aa640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a3406d310 con 0x7f6a3415c550 2024-09-19T08:42:16.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.445+0000 7f6a3a9aa640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f6a3406d5f0 con 0x7f6a3406c3a0 2024-09-19T08:42:16.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.445+0000 7f6a3a9aa640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6a3406d8d0 con 0x7f6a340cf900 2024-09-19T08:42:16.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.445+0000 7f6a33fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f6a3415c550 0x7f6a3406bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:35986/0 (socket says 172.21.15.98:35986) 2024-09-19T08:42:16.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.445+0000 7f6a33fff640 1 -- 172.21.15.98:0/4068846040 learned_addr learned my addr 172.21.15.98:0/4068846040 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:16.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.445+0000 7f6a2bfff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6a3406c3a0 0x7f6a340cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:16.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.445+0000 7f6a38f47640 1 --2- 172.21.15.98:0/4068846040 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a340cf900 0x7f6a340723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:16.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.445+0000 7f6a2bfff640 1 -- 172.21.15.98:0/4068846040 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a340cf900 msgr2=0x7f6a340723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:16.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.445+0000 7f6a2bfff640 1 --2- 172.21.15.98:0/4068846040 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a340cf900 0x7f6a340723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:16.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.445+0000 7f6a2bfff640 1 -- 172.21.15.98:0/4068846040 >> v1:172.21.15.98:6789/0 conn(0x7f6a3415c550 legacy=0x7f6a3406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:16.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.445+0000 7f6a2bfff640 1 -- 172.21.15.98:0/4068846040 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a3406dbb0 con 0x7f6a3406c3a0 2024-09-19T08:42:16.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.445+0000 7f6a38f47640 1 --2- 172.21.15.98:0/4068846040 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a340cf900 0x7f6a340723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:42:16.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.449+0000 7f6a2bfff640 1 --2- 172.21.15.98:0/4068846040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a3406c3a0 0x7f6a340cf3c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c0097c0 tx=0x7f6a1c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d2ce81949c137fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:16.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.449+0000 7f6a337fe640 1 -- 172.21.15.98:0/4068846040 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c0b1070 con 0x7f6a3406c3a0 2024-09-19T08:42:16.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.449+0000 7f6a337fe640 1 -- 172.21.15.98:0/4068846040 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a1c0ac410 con 0x7f6a3406c3a0 2024-09-19T08:42:16.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.449+0000 7f6a337fe640 1 -- 172.21.15.98:0/4068846040 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c0ac7e0 con 0x7f6a3406c3a0 2024-09-19T08:42:16.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.449+0000 7f6a3a9aa640 1 -- 172.21.15.98:0/4068846040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a3406c3a0 msgr2=0x7f6a340cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:16.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.449+0000 7f6a3a9aa640 1 --2- 172.21.15.98:0/4068846040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a3406c3a0 0x7f6a340cf3c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c0097c0 tx=0x7f6a1c01c4a0 comp rx=0 tx=0).stop 2024-09-19T08:42:16.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.449+0000 7f6a3a9aa640 1 -- 172.21.15.98:0/4068846040 shutdown_connections 2024-09-19T08:42:16.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.449+0000 7f6a3a9aa640 1 --2- 172.21.15.98:0/4068846040 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a340cf900 0x7f6a340723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:16.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.449+0000 7f6a3a9aa640 1 --2- 172.21.15.98:0/4068846040 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a3406c3a0 0x7f6a340cf3c0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:16.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.449+0000 7f6a3a9aa640 1 -- 172.21.15.98:0/4068846040 >> 172.21.15.98:0/4068846040 conn(0x7f6a340b2010 msgr2=0x7f6a340b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:16.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.449+0000 7f6a3a9aa640 1 -- 172.21.15.98:0/4068846040 shutdown_connections 2024-09-19T08:42:16.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.449+0000 7f6a3a9aa640 1 -- 172.21.15.98:0/4068846040 wait complete. 2024-09-19T08:42:16.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.449+0000 7f6a3a9aa640 1 Processor -- start 2024-09-19T08:42:16.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.449+0000 7f6a3a9aa640 1 -- start start 2024-09-19T08:42:16.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.449+0000 7f6a3a9aa640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a3406c3a0 0x7f6a3414a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:16.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.449+0000 7f6a3a9aa640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a340cf900 0x7f6a341516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:16.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.449+0000 7f6a3a9aa640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6a3415c550 0x7f6a34151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:16.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.449+0000 7f6a3a9aa640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6a34073500 con 0x7f6a340cf900 2024-09-19T08:42:16.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.449+0000 7f6a3a9aa640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6a34073380 con 0x7f6a3415c550 2024-09-19T08:42:16.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.449+0000 7f6a3a9aa640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6a34073680 con 0x7f6a3406c3a0 2024-09-19T08:42:16.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.449+0000 7f6a2bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a340cf900 0x7f6a341516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:16.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.449+0000 7f6a2bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a340cf900 0x7f6a341516e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46176/0 (socket says 172.21.15.98:46176) 2024-09-19T08:42:16.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.449+0000 7f6a2bfff640 1 -- 172.21.15.98:0/4233425747 learned_addr learned my addr 172.21.15.98:0/4233425747 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:16.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.449+0000 7f6a2bfff640 1 -- 172.21.15.98:0/4233425747 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a3406c3a0 msgr2=0x7f6a3414a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:16.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.449+0000 7f6a2bfff640 1 --2- 172.21.15.98:0/4233425747 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a3406c3a0 0x7f6a3414a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:16.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.449+0000 7f6a2bfff640 1 -- 172.21.15.98:0/4233425747 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6a3415c550 msgr2=0x7f6a34151c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:16.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.449+0000 7f6a2bfff640 1 --2- 172.21.15.98:0/4233425747 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6a3415c550 0x7f6a34151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:16.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.449+0000 7f6a2bfff640 1 -- 172.21.15.98:0/4233425747 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a340b4b30 con 0x7f6a340cf900 2024-09-19T08:42:16.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.449+0000 7f6a2bfff640 1 --2- 172.21.15.98:0/4233425747 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a340cf900 0x7f6a341516e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c01a0d0 tx=0x7f6a1c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:16.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.449+0000 7f6a317fa640 1 -- 172.21.15.98:0/4233425747 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c0b1040 con 0x7f6a340cf900 2024-09-19T08:42:16.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.449+0000 7f6a317fa640 1 -- 172.21.15.98:0/4233425747 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a1c0bb070 con 0x7f6a340cf900 2024-09-19T08:42:16.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.449+0000 7f6a3a9aa640 1 -- 172.21.15.98:0/4233425747 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a340b4e10 con 0x7f6a340cf900 2024-09-19T08:42:16.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.449+0000 7f6a317fa640 1 -- 172.21.15.98:0/4233425747 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c0b5630 con 0x7f6a340cf900 2024-09-19T08:42:16.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.449+0000 7f6a3a9aa640 1 -- 172.21.15.98:0/4233425747 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a3414f170 con 0x7f6a340cf900 2024-09-19T08:42:16.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.453+0000 7f6a3a9aa640 1 -- 172.21.15.98:0/4233425747 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a340cf3c0 con 0x7f6a340cf900 2024-09-19T08:42:16.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.453+0000 7f6a317fa640 1 -- 172.21.15.98:0/4233425747 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f6a1c0b57d0 con 0x7f6a340cf900 2024-09-19T08:42:16.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.453+0000 7f6a317fa640 1 --2- 172.21.15.98:0/4233425747 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6a14067c20 0x7f6a1406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:16.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.453+0000 7f6a317fa640 1 -- 172.21.15.98:0/4233425747 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f6a1c11ff20 con 0x7f6a340cf900 2024-09-19T08:42:16.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.453+0000 7f6a33fff640 1 --2- 172.21.15.98:0/4233425747 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6a14067c20 0x7f6a1406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:16.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.453+0000 7f6a33fff640 1 --2- 172.21.15.98:0/4233425747 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6a14067c20 0x7f6a1406a0e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6a24000f70 tx=0x7f6a24007a90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:16.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.457+0000 7f6a317fa640 1 -- 172.21.15.98:0/4233425747 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6a1c016090 con 0x7f6a340cf900 2024-09-19T08:42:16.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.621+0000 7f6a3a9aa640 1 -- 172.21.15.98:0/4233425747 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6a34154e20 con 0x7f6a340cf900 2024-09-19T08:42:16.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.625+0000 7f6a317fa640 1 -- 172.21.15.98:0/4233425747 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+16725 (secure 0 0 0) 0x7f6a1c0ace20 con 0x7f6a340cf900 2024-09-19T08:42:16.626 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:42:16.626 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":57,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:41:48.960009+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:41:44.964773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-19T08:41:46.289740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:42:16.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.625+0000 7f6a3a9aa640 1 -- 172.21.15.98:0/4233425747 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6a14067c20 msgr2=0x7f6a1406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:16.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.625+0000 7f6a3a9aa640 1 --2- 172.21.15.98:0/4233425747 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6a14067c20 0x7f6a1406a0e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6a24000f70 tx=0x7f6a24007a90 comp rx=0 tx=0).stop 2024-09-19T08:42:16.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.625+0000 7f6a3a9aa640 1 -- 172.21.15.98:0/4233425747 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a340cf900 msgr2=0x7f6a341516e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:16.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.625+0000 7f6a3a9aa640 1 --2- 172.21.15.98:0/4233425747 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a340cf900 0x7f6a341516e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c01a0d0 tx=0x7f6a1c019240 comp rx=0 tx=0).stop 2024-09-19T08:42:16.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.625+0000 7f6a3a9aa640 1 -- 172.21.15.98:0/4233425747 shutdown_connections 2024-09-19T08:42:16.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.625+0000 7f6a3a9aa640 1 --2- 172.21.15.98:0/4233425747 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6a14067c20 0x7f6a1406a0e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:16.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.625+0000 7f6a3a9aa640 1 --2- 172.21.15.98:0/4233425747 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6a3415c550 0x7f6a34151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:16.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.625+0000 7f6a3a9aa640 1 --2- 172.21.15.98:0/4233425747 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a340cf900 0x7f6a341516e0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:16.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.625+0000 7f6a3a9aa640 1 --2- 172.21.15.98:0/4233425747 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a3406c3a0 0x7f6a3414a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:16.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.625+0000 7f6a3a9aa640 1 -- 172.21.15.98:0/4233425747 >> 172.21.15.98:0/4233425747 conn(0x7f6a340b2010 msgr2=0x7f6a340d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:16.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.629+0000 7f6a3a9aa640 1 -- 172.21.15.98:0/4233425747 shutdown_connections 2024-09-19T08:42:16.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.629+0000 7f6a3a9aa640 1 -- 172.21.15.98:0/4233425747 wait complete. 2024-09-19T08:42:16.645 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T08:42:16.645 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:42:16.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.761+0000 7f64e6a87640 1 Processor -- start 2024-09-19T08:42:16.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.761+0000 7f64e6a87640 1 -- start start 2024-09-19T08:42:16.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.765+0000 7f64e6a87640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f64e00b7f80 0x7f64e015f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:16.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.765+0000 7f64e6a87640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f64e00b9970 0x7f64e00b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:16.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.765+0000 7f64e6a87640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64e0068440 con 0x7f64e00d4820 2024-09-19T08:42:16.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.765+0000 7f64e6a87640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f64e0163050 con 0x7f64e00b7f80 2024-09-19T08:42:16.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.765+0000 7f64e6a87640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f64e0162a90 con 0x7f64e00b9970 2024-09-19T08:42:16.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.765+0000 7f64e4823640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f64e00b7f80 0x7f64e015f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:16.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.765+0000 7f64e4823640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f64e00b7f80 0x7f64e015f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46190/0 (socket says 172.21.15.98:46190) 2024-09-19T08:42:16.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.765+0000 7f64e4823640 1 -- 172.21.15.98:0/2571571414 learned_addr learned my addr 172.21.15.98:0/2571571414 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:16.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.765+0000 7f64e5024640 1 --2- 172.21.15.98:0/2571571414 >> v2:172.21.15.139:3300/0 conn(0x7f64e00b9970 0x7f64e00b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:16.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.765+0000 7f64e4823640 1 -- 172.21.15.98:0/2571571414 >> v2:172.21.15.139:3300/0 conn(0x7f64e00b9970 msgr2=0x7f64e00b9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:16.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.765+0000 7f64e4823640 1 --2- 172.21.15.98:0/2571571414 >> v2:172.21.15.139:3300/0 conn(0x7f64e00b9970 0x7f64e00b9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:16.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.765+0000 7f64e4823640 1 -- 172.21.15.98:0/2571571414 >> v1:172.21.15.139:6789/0 conn(0x7f64e00d4820 legacy=0x7f64e01621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:16.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.765+0000 7f64e4823640 1 -- 172.21.15.98:0/2571571414 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e0162d70 con 0x7f64e00b7f80 2024-09-19T08:42:16.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.765+0000 7f64e4823640 1 --2- 172.21.15.98:0/2571571414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64e00b7f80 0x7f64e015f560 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f64e00b69c0 tx=0x7f64d401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e31ded69f727db7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:16.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.769+0000 7f64df7fe640 1 -- 172.21.15.98:0/2571571414 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d40b0070 con 0x7f64e00b7f80 2024-09-19T08:42:16.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.769+0000 7f64df7fe640 1 -- 172.21.15.98:0/2571571414 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64d40ab600 con 0x7f64e00b7f80 2024-09-19T08:42:16.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.769+0000 7f64df7fe640 1 -- 172.21.15.98:0/2571571414 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d40ab9d0 con 0x7f64e00b7f80 2024-09-19T08:42:16.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.769+0000 7f64e6a87640 1 -- 172.21.15.98:0/2571571414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64e00b7f80 msgr2=0x7f64e015f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:16.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.769+0000 7f64e6a87640 1 --2- 172.21.15.98:0/2571571414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64e00b7f80 0x7f64e015f560 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f64e00b69c0 tx=0x7f64d401c4e0 comp rx=0 tx=0).stop 2024-09-19T08:42:16.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.769+0000 7f64e6a87640 1 -- 172.21.15.98:0/2571571414 shutdown_connections 2024-09-19T08:42:16.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.769+0000 7f64e6a87640 1 --2- 172.21.15.98:0/2571571414 >> v2:172.21.15.139:3300/0 conn(0x7f64e00b9970 0x7f64e00b9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:16.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.769+0000 7f64e6a87640 1 --2- 172.21.15.98:0/2571571414 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64e00b7f80 0x7f64e015f560 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:16.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.769+0000 7f64e6a87640 1 -- 172.21.15.98:0/2571571414 >> 172.21.15.98:0/2571571414 conn(0x7f64e0074e40 msgr2=0x7f64e0075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:16.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.769+0000 7f64e6a87640 1 -- 172.21.15.98:0/2571571414 shutdown_connections 2024-09-19T08:42:16.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.769+0000 7f64e6a87640 1 -- 172.21.15.98:0/2571571414 wait complete. 2024-09-19T08:42:16.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.769+0000 7f64e6a87640 1 Processor -- start 2024-09-19T08:42:16.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.769+0000 7f64e6a87640 1 -- start start 2024-09-19T08:42:16.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.769+0000 7f64e6a87640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64e00b7f80 0x7f64e01504e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:16.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.769+0000 7f64e6a87640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64e00b9970 0x7f64e013bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:16.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.769+0000 7f64e6a87640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64e00d4820 0x7f64e01611d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:16.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.769+0000 7f64e6a87640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f64e00b84e0 con 0x7f64e00b7f80 2024-09-19T08:42:16.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.769+0000 7f64e6a87640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f64e00b8360 con 0x7f64e00d4820 2024-09-19T08:42:16.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.769+0000 7f64e6a87640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f64e0144590 con 0x7f64e00b9970 2024-09-19T08:42:16.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.769+0000 7f64e4823640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64e00b7f80 0x7f64e01504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:16.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.769+0000 7f64e4823640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64e00b7f80 0x7f64e01504e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46200/0 (socket says 172.21.15.98:46200) 2024-09-19T08:42:16.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.769+0000 7f64e4823640 1 -- 172.21.15.98:0/2768912267 learned_addr learned my addr 172.21.15.98:0/2768912267 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:16.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.769+0000 7f64e4823640 1 -- 172.21.15.98:0/2768912267 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64e00b9970 msgr2=0x7f64e013bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:16.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.769+0000 7f64e4823640 1 --2- 172.21.15.98:0/2768912267 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64e00b9970 0x7f64e013bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:16.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.769+0000 7f64e4823640 1 -- 172.21.15.98:0/2768912267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64e00d4820 msgr2=0x7f64e01611d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:16.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.769+0000 7f64e5024640 1 --2- 172.21.15.98:0/2768912267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64e00d4820 0x7f64e01611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:16.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.769+0000 7f64e4823640 1 --2- 172.21.15.98:0/2768912267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64e00d4820 0x7f64e01611d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:16.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.769+0000 7f64e4823640 1 -- 172.21.15.98:0/2768912267 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e0143500 con 0x7f64e00b7f80 2024-09-19T08:42:16.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.769+0000 7f64e4823640 1 --2- 172.21.15.98:0/2768912267 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64e00b7f80 0x7f64e01504e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f64d40ae040 tx=0x7f64d4019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:16.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.769+0000 7f64dd7fa640 1 -- 172.21.15.98:0/2768912267 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d40b0040 con 0x7f64e00b7f80 2024-09-19T08:42:16.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.769+0000 7f64dd7fa640 1 -- 172.21.15.98:0/2768912267 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64d40b9070 con 0x7f64e00b7f80 2024-09-19T08:42:16.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.769+0000 7f64dd7fa640 1 -- 172.21.15.98:0/2768912267 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d40b46a0 con 0x7f64e00b7f80 2024-09-19T08:42:16.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.769+0000 7f64e6a87640 1 -- 172.21.15.98:0/2768912267 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64e01437a0 con 0x7f64e00b7f80 2024-09-19T08:42:16.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.769+0000 7f64e6a87640 1 -- 172.21.15.98:0/2768912267 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64e013ef90 con 0x7f64e00b7f80 2024-09-19T08:42:16.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.773+0000 7f64e6a87640 1 -- 172.21.15.98:0/2768912267 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64e01448a0 con 0x7f64e00b7f80 2024-09-19T08:42:16.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.773+0000 7f64dd7fa640 1 -- 172.21.15.98:0/2768912267 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f64d40b4840 con 0x7f64e00b7f80 2024-09-19T08:42:16.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.773+0000 7f64dd7fa640 1 --2- 172.21.15.98:0/2768912267 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f64c4067c20 0x7f64c406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:16.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.773+0000 7f64dd7fa640 1 -- 172.21.15.98:0/2768912267 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f64d411fed0 con 0x7f64e00b7f80 2024-09-19T08:42:16.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.773+0000 7f64dffff640 1 --2- 172.21.15.98:0/2768912267 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f64c4067c20 0x7f64c406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:16.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.773+0000 7f64dffff640 1 --2- 172.21.15.98:0/2768912267 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f64c4067c20 0x7f64c406a0e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f64e0138780 tx=0x7f64cc0079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:16.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.777+0000 7f64dd7fa640 1 -- 172.21.15.98:0/2768912267 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f64d40160d0 con 0x7f64e00b7f80 2024-09-19T08:42:16.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.973+0000 7f64e6a87640 1 -- 172.21.15.98:0/2768912267 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64e013ac80 con 0x7f64e00b7f80 2024-09-19T08:42:16.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.977+0000 7f64dd7fa640 1 -- 172.21.15.98:0/2768912267 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+5229 (secure 0 0 0) 0x7f64d40f0580 con 0x7f64e00b7f80 2024-09-19T08:42:16.977 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:42:16.977 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":31,"btime":"2024-09-19T08:42:03:179377+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4880,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/1549394645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":1549394645},{"type":"v1","addr":"172.21.15.139:6834","nonce":1549394645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":4883,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1847525681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6833","nonce":1847525681},{"type":"v1","addr":"172.21.15.139:6835","nonce":1847525681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5745,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/1677812726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1677812726},{"type":"v1","addr":"172.21.15.98:6835","nonce":1677812726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:41:47.304224+0000","modified":"2024-09-19T08:41:49.306408+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/3548234111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3548234111},{"type":"v1","addr":"172.21.15.98:6837","nonce":3548234111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5748,"qdb_cluster":[5748]},"id":4}]} 2024-09-19T08:42:16.977 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 31 2024-09-19T08:42:16.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.977+0000 7f64e6a87640 1 -- 172.21.15.98:0/2768912267 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f64c4067c20 msgr2=0x7f64c406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:16.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.977+0000 7f64e6a87640 1 --2- 172.21.15.98:0/2768912267 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f64c4067c20 0x7f64c406a0e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f64e0138780 tx=0x7f64cc0079c0 comp rx=0 tx=0).stop 2024-09-19T08:42:16.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.977+0000 7f64e6a87640 1 -- 172.21.15.98:0/2768912267 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64e00b7f80 msgr2=0x7f64e01504e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:16.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.977+0000 7f64e6a87640 1 --2- 172.21.15.98:0/2768912267 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64e00b7f80 0x7f64e01504e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f64d40ae040 tx=0x7f64d4019280 comp rx=0 tx=0).stop 2024-09-19T08:42:16.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.977+0000 7f64e6a87640 1 -- 172.21.15.98:0/2768912267 shutdown_connections 2024-09-19T08:42:16.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.977+0000 7f64e6a87640 1 --2- 172.21.15.98:0/2768912267 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f64c4067c20 0x7f64c406a0e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:16.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.977+0000 7f64e6a87640 1 --2- 172.21.15.98:0/2768912267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64e00d4820 0x7f64e01611d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:16.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.977+0000 7f64e6a87640 1 --2- 172.21.15.98:0/2768912267 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64e00b9970 0x7f64e013bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:16.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.977+0000 7f64e6a87640 1 --2- 172.21.15.98:0/2768912267 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64e00b7f80 0x7f64e01504e0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:16.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.977+0000 7f64e6a87640 1 -- 172.21.15.98:0/2768912267 >> 172.21.15.98:0/2768912267 conn(0x7f64e0074e40 msgr2=0x7f64e00d14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:16.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.977+0000 7f64e6a87640 1 -- 172.21.15.98:0/2768912267 shutdown_connections 2024-09-19T08:42:16.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:16.977+0000 7f64e6a87640 1 -- 172.21.15.98:0/2768912267 wait complete. 2024-09-19T08:42:16.995 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:42:17.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.109+0000 7f7c5506d640 1 Processor -- start 2024-09-19T08:42:17.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.109+0000 7f7c5506d640 1 -- start start 2024-09-19T08:42:17.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.109+0000 7f7c5506d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7c50158fd0 0x7f7c500b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:17.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.109+0000 7f7c5506d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c500b4360 0x7f7c5006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:17.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.109+0000 7f7c5506d640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c500ccc60 con 0x7f7c5006c1d0 2024-09-19T08:42:17.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.109+0000 7f7c5506d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f7c501593b0 con 0x7f7c50158fd0 2024-09-19T08:42:17.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.109+0000 7f7c5506d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7c5006c600 con 0x7f7c500b4360 2024-09-19T08:42:17.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.109+0000 7f7c4f59e640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f7c5006c1d0 0x7f7c5015efc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:35990/0 (socket says 172.21.15.98:35990) 2024-09-19T08:42:17.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.109+0000 7f7c4f59e640 1 -- 172.21.15.98:0/2188414701 learned_addr learned my addr 172.21.15.98:0/2188414701 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:17.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.109+0000 7f7c4e59c640 1 --2- 172.21.15.98:0/2188414701 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c500b4360 0x7f7c5006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:17.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.109+0000 7f7c4ed9d640 1 --2- 172.21.15.98:0/2188414701 >> v2:172.21.15.139:3300/0 conn(0x7f7c50158fd0 0x7f7c500b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:17.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.109+0000 7f7c4dd9b640 1 -- 172.21.15.98:0/2188414701 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3922542061 0 0) 0x7f7c500ccc60 con 0x7f7c5006c1d0 2024-09-19T08:42:17.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.109+0000 7f7c4dd9b640 1 -- 172.21.15.98:0/2188414701 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c38003530 con 0x7f7c5006c1d0 2024-09-19T08:42:17.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.109+0000 7f7c4dd9b640 1 -- 172.21.15.98:0/2188414701 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1610804830 0 0) 0x7f7c38003530 con 0x7f7c5006c1d0 2024-09-19T08:42:17.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.109+0000 7f7c4dd9b640 1 -- 172.21.15.98:0/2188414701 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c500b4360 msgr2=0x7f7c5006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:17.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.109+0000 7f7c4dd9b640 1 --2- 172.21.15.98:0/2188414701 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c500b4360 0x7f7c5006bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:17.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.109+0000 7f7c4dd9b640 1 -- 172.21.15.98:0/2188414701 >> v2:172.21.15.139:3300/0 conn(0x7f7c50158fd0 msgr2=0x7f7c500b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:17.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.109+0000 7f7c4dd9b640 1 --2- 172.21.15.98:0/2188414701 >> v2:172.21.15.139:3300/0 conn(0x7f7c50158fd0 0x7f7c500b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:17.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.109+0000 7f7c4dd9b640 1 -- 172.21.15.98:0/2188414701 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c500ccf40 con 0x7f7c5006c1d0 2024-09-19T08:42:17.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.109+0000 7f7c4dd9b640 1 -- 172.21.15.98:0/2188414701 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f7c48001a00 con 0x7f7c5006c1d0 2024-09-19T08:42:17.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.109+0000 7f7c4dd9b640 1 -- 172.21.15.98:0/2188414701 >> v1:172.21.15.98:6789/0 conn(0x7f7c5006c1d0 legacy=0x7f7c5015efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:17.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.109+0000 7f7c4dd9b640 1 --2- 172.21.15.98:0/2188414701 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c500b4360 0x7f7c380039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:17.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.109+0000 7f7c4dd9b640 1 --2- 172.21.15.98:0/2188414701 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c38003f40 0x7f7c38004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:17.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.109+0000 7f7c4dd9b640 1 --2- 172.21.15.98:0/2188414701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c38004d50 0x7f7c380051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:17.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.109+0000 7f7c4dd9b640 1 -- 172.21.15.98:0/2188414701 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7c501593b0 con 0x7f7c38004d50 2024-09-19T08:42:17.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.109+0000 7f7c4dd9b640 1 -- 172.21.15.98:0/2188414701 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5006c600 con 0x7f7c38003f40 2024-09-19T08:42:17.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.109+0000 7f7c4dd9b640 1 -- 172.21.15.98:0/2188414701 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7c3800ba20 con 0x7f7c500b4360 2024-09-19T08:42:17.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.109+0000 7f7c4f59e640 1 --2- 172.21.15.98:0/2188414701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c38004d50 0x7f7c380051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:17.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.109+0000 7f7c4e59c640 1 --2- 172.21.15.98:0/2188414701 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c38003f40 0x7f7c38004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:17.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.109+0000 7f7c4ed9d640 1 --2- 172.21.15.98:0/2188414701 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c500b4360 0x7f7c380039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:17.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.109+0000 7f7c4f59e640 1 -- 172.21.15.98:0/2188414701 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c500b4360 msgr2=0x7f7c380039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:17.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.109+0000 7f7c4f59e640 1 --2- 172.21.15.98:0/2188414701 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c500b4360 0x7f7c380039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:17.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.109+0000 7f7c4f59e640 1 -- 172.21.15.98:0/2188414701 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c38003f40 msgr2=0x7f7c38004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:17.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.109+0000 7f7c4f59e640 1 --2- 172.21.15.98:0/2188414701 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c38003f40 0x7f7c38004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:17.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.109+0000 7f7c4f59e640 1 -- 172.21.15.98:0/2188414701 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c38003530 con 0x7f7c38004d50 2024-09-19T08:42:17.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.109+0000 7f7c4ed9d640 1 --2- 172.21.15.98:0/2188414701 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c500b4360 0x7f7c380039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:42:17.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.113+0000 7f7c4f59e640 1 --2- 172.21.15.98:0/2188414701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c38004d50 0x7f7c380051d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f7c4800a480 tx=0x7f7c48021d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=814c809d05efd48e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:17.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.113+0000 7f7c4dd9b640 1 -- 172.21.15.98:0/2188414701 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4800a4c0 con 0x7f7c38004d50 2024-09-19T08:42:17.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.113+0000 7f7c4dd9b640 1 -- 172.21.15.98:0/2188414701 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c480af020 con 0x7f7c38004d50 2024-09-19T08:42:17.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.113+0000 7f7c4dd9b640 1 -- 172.21.15.98:0/2188414701 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c480ab970 con 0x7f7c38004d50 2024-09-19T08:42:17.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.113+0000 7f7c5506d640 1 -- 172.21.15.98:0/2188414701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c38004d50 msgr2=0x7f7c380051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:17.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.113+0000 7f7c5506d640 1 --2- 172.21.15.98:0/2188414701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c38004d50 0x7f7c380051d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f7c4800a480 tx=0x7f7c48021d10 comp rx=0 tx=0).stop 2024-09-19T08:42:17.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.113+0000 7f7c4f59e640 1 -- 172.21.15.98:0/2188414701 reap_dead start 2024-09-19T08:42:17.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.113+0000 7f7c5506d640 1 -- 172.21.15.98:0/2188414701 shutdown_connections 2024-09-19T08:42:17.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.113+0000 7f7c5506d640 1 -- 172.21.15.98:0/2188414701 >> 172.21.15.98:0/2188414701 conn(0x7f7c500b2010 msgr2=0x7f7c500b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:17.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.113+0000 7f7c5506d640 1 -- 172.21.15.98:0/2188414701 shutdown_connections 2024-09-19T08:42:17.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.113+0000 7f7c5506d640 1 -- 172.21.15.98:0/2188414701 wait complete. 2024-09-19T08:42:17.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.113+0000 7f7c5506d640 1 Processor -- start 2024-09-19T08:42:17.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.113+0000 7f7c5506d640 1 -- start start 2024-09-19T08:42:17.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.113+0000 7f7c5506d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c38004d50 0x7f7c501f3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:17.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.117+0000 7f7c5506d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c501515c0 0x7f7c501ef930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:17.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.117+0000 7f7c5506d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c501efe70 0x7f7c5015eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:17.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.117+0000 7f7c5506d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7c500cd7e0 con 0x7f7c38004d50 2024-09-19T08:42:17.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.117+0000 7f7c5506d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7c500cd500 con 0x7f7c501efe70 2024-09-19T08:42:17.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.117+0000 7f7c5506d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7c5015c7a0 con 0x7f7c501515c0 2024-09-19T08:42:17.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.117+0000 7f7c4ed9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c38004d50 0x7f7c501f3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:17.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.117+0000 7f7c4ed9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c38004d50 0x7f7c501f3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46226/0 (socket says 172.21.15.98:46226) 2024-09-19T08:42:17.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.117+0000 7f7c4ed9d640 1 -- 172.21.15.98:0/2053456190 learned_addr learned my addr 172.21.15.98:0/2053456190 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:17.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.117+0000 7f7c4ed9d640 1 -- 172.21.15.98:0/2053456190 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c501515c0 msgr2=0x7f7c501ef930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:17.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.117+0000 7f7c4e59c640 1 --2- 172.21.15.98:0/2053456190 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c501515c0 0x7f7c501ef930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:17.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.117+0000 7f7c4f59e640 1 --2- 172.21.15.98:0/2053456190 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c501efe70 0x7f7c5015eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:17.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.117+0000 7f7c4ed9d640 1 --2- 172.21.15.98:0/2053456190 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c501515c0 0x7f7c501ef930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:17.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.117+0000 7f7c4ed9d640 1 -- 172.21.15.98:0/2053456190 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c501efe70 msgr2=0x7f7c5015eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:17.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.117+0000 7f7c4ed9d640 1 --2- 172.21.15.98:0/2053456190 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c501efe70 0x7f7c5015eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:17.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.117+0000 7f7c4ed9d640 1 -- 172.21.15.98:0/2053456190 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c5015d5f0 con 0x7f7c38004d50 2024-09-19T08:42:17.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.117+0000 7f7c4ed9d640 1 --2- 172.21.15.98:0/2053456190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c38004d50 0x7f7c501f3ae0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f7c40010950 tx=0x7f7c40010e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:17.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.117+0000 7f7c377fe640 1 -- 172.21.15.98:0/2053456190 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4000f040 con 0x7f7c38004d50 2024-09-19T08:42:17.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.117+0000 7f7c377fe640 1 -- 172.21.15.98:0/2053456190 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c4000ec20 con 0x7f7c38004d50 2024-09-19T08:42:17.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.117+0000 7f7c5506d640 1 -- 172.21.15.98:0/2053456190 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c501f0830 con 0x7f7c38004d50 2024-09-19T08:42:17.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.117+0000 7f7c377fe640 1 -- 172.21.15.98:0/2053456190 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c40004020 con 0x7f7c38004d50 2024-09-19T08:42:17.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.117+0000 7f7c5506d640 1 -- 172.21.15.98:0/2053456190 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c501f3000 con 0x7f7c38004d50 2024-09-19T08:42:17.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.117+0000 7f7c5506d640 1 -- 172.21.15.98:0/2053456190 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c50157670 con 0x7f7c38004d50 2024-09-19T08:42:17.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.117+0000 7f7c377fe640 1 -- 172.21.15.98:0/2053456190 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f7c400041c0 con 0x7f7c38004d50 2024-09-19T08:42:17.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.117+0000 7f7c377fe640 1 --2- 172.21.15.98:0/2053456190 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7c3c067c20 0x7f7c3c06a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:17.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.117+0000 7f7c4e59c640 1 --2- 172.21.15.98:0/2053456190 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7c3c067c20 0x7f7c3c06a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:17.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.117+0000 7f7c377fe640 1 -- 172.21.15.98:0/2053456190 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f7c40086300 con 0x7f7c38004d50 2024-09-19T08:42:17.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.117+0000 7f7c4e59c640 1 --2- 172.21.15.98:0/2053456190 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7c3c067c20 0x7f7c3c06a0e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7c440098f0 tx=0x7f7c44008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:17.123 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.121+0000 7f7c377fe640 1 -- 172.21.15.98:0/2053456190 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f7c40018030 con 0x7f7c38004d50 2024-09-19T08:42:17.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.317+0000 7f7c5506d640 1 -- 172.21.15.98:0/2053456190 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c500b3d10 con 0x7f7c38004d50 2024-09-19T08:42:17.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.317+0000 7f7c377fe640 1 -- 172.21.15.98:0/2053456190 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+5229 (secure 0 0 0) 0x7f7c400044b0 con 0x7f7c38004d50 2024-09-19T08:42:17.318 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:42:17.319 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":31,"btime":"2024-09-19T08:42:03:179377+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4880,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/1549394645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":1549394645},{"type":"v1","addr":"172.21.15.139:6834","nonce":1549394645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":4883,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1847525681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6833","nonce":1847525681},{"type":"v1","addr":"172.21.15.139:6835","nonce":1847525681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5745,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/1677812726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1677812726},{"type":"v1","addr":"172.21.15.98:6835","nonce":1677812726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:41:47.304224+0000","modified":"2024-09-19T08:41:49.306408+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5748},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5748":{"gid":5748,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/3548234111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":3548234111},{"type":"v1","addr":"172.21.15.98:6837","nonce":3548234111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5748,"qdb_cluster":[5748]},"id":4}]} 2024-09-19T08:42:17.319 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 31 2024-09-19T08:42:17.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.321+0000 7f7c5506d640 1 -- 172.21.15.98:0/2053456190 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7c3c067c20 msgr2=0x7f7c3c06a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:17.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.321+0000 7f7c5506d640 1 --2- 172.21.15.98:0/2053456190 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7c3c067c20 0x7f7c3c06a0e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7c440098f0 tx=0x7f7c44008040 comp rx=0 tx=0).stop 2024-09-19T08:42:17.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.321+0000 7f7c5506d640 1 -- 172.21.15.98:0/2053456190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c38004d50 msgr2=0x7f7c501f3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:17.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.321+0000 7f7c5506d640 1 --2- 172.21.15.98:0/2053456190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c38004d50 0x7f7c501f3ae0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f7c40010950 tx=0x7f7c40010e10 comp rx=0 tx=0).stop 2024-09-19T08:42:17.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.321+0000 7f7c5506d640 1 -- 172.21.15.98:0/2053456190 shutdown_connections 2024-09-19T08:42:17.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.321+0000 7f7c5506d640 1 --2- 172.21.15.98:0/2053456190 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7c3c067c20 0x7f7c3c06a0e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:17.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.321+0000 7f7c5506d640 1 --2- 172.21.15.98:0/2053456190 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c501efe70 0x7f7c5015eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:17.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.321+0000 7f7c5506d640 1 --2- 172.21.15.98:0/2053456190 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c501515c0 0x7f7c501ef930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:17.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.321+0000 7f7c5506d640 1 --2- 172.21.15.98:0/2053456190 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c38004d50 0x7f7c501f3ae0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:17.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.321+0000 7f7c5506d640 1 -- 172.21.15.98:0/2053456190 >> 172.21.15.98:0/2053456190 conn(0x7f7c500b2010 msgr2=0x7f7c500b33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:17.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.321+0000 7f7c5506d640 1 -- 172.21.15.98:0/2053456190 shutdown_connections 2024-09-19T08:42:17.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.321+0000 7f7c5506d640 1 -- 172.21.15.98:0/2053456190 wait complete. 2024-09-19T08:42:17.336 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:42:17.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.453+0000 7ffbc9f55640 1 Processor -- start 2024-09-19T08:42:17.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.453+0000 7ffbc9f55640 1 -- start start 2024-09-19T08:42:17.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.453+0000 7ffbc9f55640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ffbc406c120 0x7ffbc40cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:17.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.453+0000 7ffbc9f55640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbc40ccc60 con 0x7ffbc406bc90 2024-09-19T08:42:17.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.453+0000 7ffbc9f55640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbc40ccf40 con 0x7ffbc4158fd0 2024-09-19T08:42:17.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.453+0000 7ffbc9f55640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7ffbc41593b0 con 0x7ffbc406c120 2024-09-19T08:42:17.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.453+0000 7ffbc3fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ffbc406c120 0x7ffbc40cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:17.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.453+0000 7ffbc2ffd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7ffbc406bc90 0x7ffbc40ce840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:35998/0 (socket says 172.21.15.98:35998) 2024-09-19T08:42:17.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.453+0000 7ffbc3fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ffbc406c120 0x7ffbc40cef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46234/0 (socket says 172.21.15.98:46234) 2024-09-19T08:42:17.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.453+0000 7ffbc2ffd640 1 -- 172.21.15.98:0/2916955075 learned_addr learned my addr 172.21.15.98:0/2916955075 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:17.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.453+0000 7ffbc3fff640 1 -- 172.21.15.98:0/2916955075 >> v1:172.21.15.139:6789/0 conn(0x7ffbc4158fd0 legacy=0x7ffbc40b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:17.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.453+0000 7ffbc3fff640 1 -- 172.21.15.98:0/2916955075 >> v1:172.21.15.98:6789/0 conn(0x7ffbc406bc90 legacy=0x7ffbc40ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:17.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.453+0000 7ffbc3fff640 1 -- 172.21.15.98:0/2916955075 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc40cd220 con 0x7ffbc406c120 2024-09-19T08:42:17.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.457+0000 7ffbc3fff640 1 --2- 172.21.15.98:0/2916955075 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbc406c120 0x7ffbc40cef50 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4009d90 tx=0x7ffbb401c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ab77d3c3e59847 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:17.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.457+0000 7ffbc27fc640 1 -- 172.21.15.98:0/2916955075 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb40b2070 con 0x7ffbc406c120 2024-09-19T08:42:17.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.457+0000 7ffbc27fc640 1 -- 172.21.15.98:0/2916955075 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbb40a5930 con 0x7ffbc406c120 2024-09-19T08:42:17.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.457+0000 7ffbc27fc640 1 -- 172.21.15.98:0/2916955075 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb40a5d00 con 0x7ffbc406c120 2024-09-19T08:42:17.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.457+0000 7ffbc9f55640 1 -- 172.21.15.98:0/2916955075 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbc406c120 msgr2=0x7ffbc40cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:17.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.457+0000 7ffbc9f55640 1 --2- 172.21.15.98:0/2916955075 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbc406c120 0x7ffbc40cef50 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4009d90 tx=0x7ffbb401c870 comp rx=0 tx=0).stop 2024-09-19T08:42:17.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.457+0000 7ffbc9f55640 1 -- 172.21.15.98:0/2916955075 shutdown_connections 2024-09-19T08:42:17.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.457+0000 7ffbc9f55640 1 --2- 172.21.15.98:0/2916955075 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbc406c120 0x7ffbc40cef50 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:17.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.457+0000 7ffbc9f55640 1 -- 172.21.15.98:0/2916955075 >> 172.21.15.98:0/2916955075 conn(0x7ffbc40b2010 msgr2=0x7ffbc40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:17.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.457+0000 7ffbc9f55640 1 -- 172.21.15.98:0/2916955075 shutdown_connections 2024-09-19T08:42:17.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.457+0000 7ffbc9f55640 1 -- 172.21.15.98:0/2916955075 wait complete. 2024-09-19T08:42:17.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.457+0000 7ffbc9f55640 1 Processor -- start 2024-09-19T08:42:17.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.457+0000 7ffbc9f55640 1 -- start start 2024-09-19T08:42:17.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.457+0000 7ffbc9f55640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbc406bc90 0x7ffbc4073e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:17.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.457+0000 7ffbc9f55640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbc406c120 0x7ffbc40715e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:17.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.457+0000 7ffbc9f55640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffbc4158fd0 0x7ffbc414cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:17.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.457+0000 7ffbc9f55640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc415c7a0 con 0x7ffbc406c120 2024-09-19T08:42:17.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.457+0000 7ffbc9f55640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc40b4530 con 0x7ffbc406bc90 2024-09-19T08:42:17.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.457+0000 7ffbc9f55640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ffbc415c920 con 0x7ffbc4158fd0 2024-09-19T08:42:17.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.457+0000 7ffbc2ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbc406c120 0x7ffbc40715e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:17.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.457+0000 7ffbc2ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbc406c120 0x7ffbc40715e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46242/0 (socket says 172.21.15.98:46242) 2024-09-19T08:42:17.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.457+0000 7ffbc2ffd640 1 -- 172.21.15.98:0/539858125 learned_addr learned my addr 172.21.15.98:0/539858125 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:17.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.457+0000 7ffbc37fe640 1 --2- 172.21.15.98:0/539858125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbc406bc90 0x7ffbc4073e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:17.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.457+0000 7ffbc3fff640 1 --2- 172.21.15.98:0/539858125 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffbc4158fd0 0x7ffbc414cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:17.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.461+0000 7ffbc2ffd640 1 -- 172.21.15.98:0/539858125 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffbc4158fd0 msgr2=0x7ffbc414cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:17.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.461+0000 7ffbc2ffd640 1 --2- 172.21.15.98:0/539858125 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffbc4158fd0 0x7ffbc414cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:17.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.461+0000 7ffbc2ffd640 1 -- 172.21.15.98:0/539858125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbc406bc90 msgr2=0x7ffbc4073e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:17.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.461+0000 7ffbc2ffd640 1 --2- 172.21.15.98:0/539858125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbc406bc90 0x7ffbc4073e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:17.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.461+0000 7ffbc2ffd640 1 -- 172.21.15.98:0/539858125 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc4134370 con 0x7ffbc406c120 2024-09-19T08:42:17.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.461+0000 7ffbc2ffd640 1 --2- 172.21.15.98:0/539858125 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbc406c120 0x7ffbc40715e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ffbb800c420 tx=0x7ffbb800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:17.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.461+0000 7ffbabfff640 1 -- 172.21.15.98:0/539858125 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb8017070 con 0x7ffbc406c120 2024-09-19T08:42:17.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.461+0000 7ffbabfff640 1 -- 172.21.15.98:0/539858125 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbb8005cf0 con 0x7ffbc406c120 2024-09-19T08:42:17.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.461+0000 7ffbc9f55640 1 -- 172.21.15.98:0/539858125 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbc4073360 con 0x7ffbc406c120 2024-09-19T08:42:17.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.461+0000 7ffbabfff640 1 -- 172.21.15.98:0/539858125 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb80072f0 con 0x7ffbc406c120 2024-09-19T08:42:17.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.461+0000 7ffbc9f55640 1 -- 172.21.15.98:0/539858125 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbc4073920 con 0x7ffbc406c120 2024-09-19T08:42:17.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.461+0000 7ffbc9f55640 1 -- 172.21.15.98:0/539858125 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbc4150c40 con 0x7ffbc406c120 2024-09-19T08:42:17.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.461+0000 7ffbabfff640 1 -- 172.21.15.98:0/539858125 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7ffbb801a070 con 0x7ffbc406c120 2024-09-19T08:42:17.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.461+0000 7ffbabfff640 1 --2- 172.21.15.98:0/539858125 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ffbb0067c20 0x7ffbb006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:17.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.461+0000 7ffbabfff640 1 -- 172.21.15.98:0/539858125 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7ffbb8085520 con 0x7ffbc406c120 2024-09-19T08:42:17.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.461+0000 7ffbc37fe640 1 --2- 172.21.15.98:0/539858125 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ffbb0067c20 0x7ffbb006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:17.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.461+0000 7ffbc37fe640 1 --2- 172.21.15.98:0/539858125 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ffbb0067c20 0x7ffbb006a0e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ffbc41314f0 tx=0x7ffbbc009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:17.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.465+0000 7ffbabfff640 1 -- 172.21.15.98:0/539858125 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ffbb8055600 con 0x7ffbc406c120 2024-09-19T08:42:17.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.617+0000 7ffbc9f55640 1 -- 172.21.15.98:0/539858125 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffbc40cc750 con 0x7ffbc406c120 2024-09-19T08:42:17.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.621+0000 7ffbabfff640 1 -- 172.21.15.98:0/539858125 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+16725 (secure 0 0 0) 0x7ffbb8059900 con 0x7ffbc406c120 2024-09-19T08:42:17.621 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:42:17.621 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":57,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:41:48.960009+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:41:44.964773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-19T08:41:46.289740+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:42:17.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.621+0000 7ffbc9f55640 1 -- 172.21.15.98:0/539858125 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ffbb0067c20 msgr2=0x7ffbb006a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:17.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.621+0000 7ffbc9f55640 1 --2- 172.21.15.98:0/539858125 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ffbb0067c20 0x7ffbb006a0e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ffbc41314f0 tx=0x7ffbbc009060 comp rx=0 tx=0).stop 2024-09-19T08:42:17.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.621+0000 7ffbc9f55640 1 -- 172.21.15.98:0/539858125 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbc406c120 msgr2=0x7ffbc40715e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:17.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.621+0000 7ffbc9f55640 1 --2- 172.21.15.98:0/539858125 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbc406c120 0x7ffbc40715e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ffbb800c420 tx=0x7ffbb800c8e0 comp rx=0 tx=0).stop 2024-09-19T08:42:17.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.621+0000 7ffbc9f55640 1 -- 172.21.15.98:0/539858125 shutdown_connections 2024-09-19T08:42:17.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.621+0000 7ffbc9f55640 1 --2- 172.21.15.98:0/539858125 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ffbb0067c20 0x7ffbb006a0e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:17.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.621+0000 7ffbc9f55640 1 --2- 172.21.15.98:0/539858125 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffbc4158fd0 0x7ffbc414cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:17.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.621+0000 7ffbc9f55640 1 --2- 172.21.15.98:0/539858125 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbc406c120 0x7ffbc40715e0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:17.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.621+0000 7ffbc9f55640 1 --2- 172.21.15.98:0/539858125 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbc406bc90 0x7ffbc4073e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:17.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.621+0000 7ffbc9f55640 1 -- 172.21.15.98:0/539858125 >> 172.21.15.98:0/539858125 conn(0x7ffbc40b2010 msgr2=0x7ffbc40b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:17.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.621+0000 7ffbc9f55640 1 -- 172.21.15.98:0/539858125 shutdown_connections 2024-09-19T08:42:17.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.621+0000 7ffbc9f55640 1 -- 172.21.15.98:0/539858125 wait complete. 2024-09-19T08:42:17.639 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-19T08:42:17.723 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:42:17.725 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:42:17.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.761+0000 7fb6aee7b640 1 Processor -- start 2024-09-19T08:42:17.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.761+0000 7fb6aee7b640 1 -- start start 2024-09-19T08:42:17.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.761+0000 7fb6aee7b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb6a80b7f80 0x7fb6a815f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:17.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.761+0000 7fb6aee7b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb6a80b9970 0x7fb6a80b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:17.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.761+0000 7fb6aee7b640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6a8068440 con 0x7fb6a80d4820 2024-09-19T08:42:17.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.761+0000 7fb6aee7b640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fb6a8163050 con 0x7fb6a80b7f80 2024-09-19T08:42:17.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.761+0000 7fb6aee7b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fb6a8162a90 con 0x7fb6a80b9970 2024-09-19T08:42:17.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.761+0000 7fb6acc17640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb6a80b7f80 0x7fb6a815f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:17.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.761+0000 7fb6a7fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fb6a80d4820 0x7fb6a81621b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:58948/0 (socket says 172.21.15.98:58948) 2024-09-19T08:42:17.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.761+0000 7fb6a7fff640 1 -- 172.21.15.98:0/4126962041 learned_addr learned my addr 172.21.15.98:0/4126962041 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:17.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.761+0000 7fb6acc17640 1 -- 172.21.15.98:0/4126962041 >> v2:172.21.15.139:3300/0 conn(0x7fb6a80b9970 msgr2=0x7fb6a80b9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:17.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.761+0000 7fb6ad418640 1 --2- 172.21.15.98:0/4126962041 >> v2:172.21.15.139:3300/0 conn(0x7fb6a80b9970 0x7fb6a80b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:17.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.761+0000 7fb6acc17640 1 --2- 172.21.15.98:0/4126962041 >> v2:172.21.15.139:3300/0 conn(0x7fb6a80b9970 0x7fb6a80b9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:17.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.761+0000 7fb6a77fe640 1 -- 172.21.15.98:0/4126962041 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 311944749 0 0) 0x7fb6a8068440 con 0x7fb6a80d4820 2024-09-19T08:42:17.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.761+0000 7fb6acc17640 1 -- 172.21.15.98:0/4126962041 >> v1:172.21.15.98:6789/0 conn(0x7fb6a80d4820 legacy=0x7fb6a81621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:17.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.761+0000 7fb6acc17640 1 -- 172.21.15.98:0/4126962041 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6a8162d70 con 0x7fb6a80b7f80 2024-09-19T08:42:17.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.761+0000 7fb6ad418640 1 --2- 172.21.15.98:0/4126962041 >> v2:172.21.15.139:3300/0 conn(0x7fb6a80b9970 0x7fb6a80b9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:42:17.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.765+0000 7fb6acc17640 1 --2- 172.21.15.98:0/4126962041 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6a80b7f80 0x7fb6a815f560 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb6a80b69c0 tx=0x7fb69801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86dce58e85549c92 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:17.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.765+0000 7fb6a77fe640 1 -- 172.21.15.98:0/4126962041 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6980b0070 con 0x7fb6a80b7f80 2024-09-19T08:42:17.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.765+0000 7fb6a77fe640 1 -- 172.21.15.98:0/4126962041 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6980ab600 con 0x7fb6a80b7f80 2024-09-19T08:42:17.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.765+0000 7fb6a77fe640 1 -- 172.21.15.98:0/4126962041 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6980ab9d0 con 0x7fb6a80b7f80 2024-09-19T08:42:17.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.765+0000 7fb6aee7b640 1 -- 172.21.15.98:0/4126962041 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6a80b7f80 msgr2=0x7fb6a815f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:17.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.765+0000 7fb6aee7b640 1 --2- 172.21.15.98:0/4126962041 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6a80b7f80 0x7fb6a815f560 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb6a80b69c0 tx=0x7fb69801c480 comp rx=0 tx=0).stop 2024-09-19T08:42:17.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.765+0000 7fb6aee7b640 1 -- 172.21.15.98:0/4126962041 shutdown_connections 2024-09-19T08:42:17.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.765+0000 7fb6aee7b640 1 --2- 172.21.15.98:0/4126962041 >> v2:172.21.15.139:3300/0 conn(0x7fb6a80b9970 0x7fb6a80b9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:17.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.765+0000 7fb6aee7b640 1 --2- 172.21.15.98:0/4126962041 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6a80b7f80 0x7fb6a815f560 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:17.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.765+0000 7fb6aee7b640 1 -- 172.21.15.98:0/4126962041 >> 172.21.15.98:0/4126962041 conn(0x7fb6a8074e40 msgr2=0x7fb6a8075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:17.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.765+0000 7fb6aee7b640 1 -- 172.21.15.98:0/4126962041 shutdown_connections 2024-09-19T08:42:17.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.765+0000 7fb6aee7b640 1 -- 172.21.15.98:0/4126962041 wait complete. 2024-09-19T08:42:17.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.765+0000 7fb6aee7b640 1 Processor -- start 2024-09-19T08:42:17.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.765+0000 7fb6aee7b640 1 -- start start 2024-09-19T08:42:17.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.765+0000 7fb6aee7b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6a80b7f80 0x7fb6a813b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:17.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.765+0000 7fb6aee7b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6a80b9970 0x7fb6a8152a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:17.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.765+0000 7fb6aee7b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6a80d4820 0x7fb6a813cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:17.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.765+0000 7fb6aee7b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a80b84e0 con 0x7fb6a80b7f80 2024-09-19T08:42:17.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.765+0000 7fb6aee7b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a80b8360 con 0x7fb6a80d4820 2024-09-19T08:42:17.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.765+0000 7fb6aee7b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb6a8144590 con 0x7fb6a80b9970 2024-09-19T08:42:17.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.765+0000 7fb6acc17640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6a80b7f80 0x7fb6a813b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:17.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.765+0000 7fb6acc17640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6a80b7f80 0x7fb6a813b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54290/0 (socket says 172.21.15.98:54290) 2024-09-19T08:42:17.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.765+0000 7fb6acc17640 1 -- 172.21.15.98:0/204549098 learned_addr learned my addr 172.21.15.98:0/204549098 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:17.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.765+0000 7fb6ad418640 1 --2- 172.21.15.98:0/204549098 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6a80d4820 0x7fb6a813cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:17.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.765+0000 7fb6a7fff640 1 --2- 172.21.15.98:0/204549098 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6a80b9970 0x7fb6a8152a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:17.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.769+0000 7fb6acc17640 1 -- 172.21.15.98:0/204549098 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6a80b9970 msgr2=0x7fb6a8152a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:17.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.769+0000 7fb6acc17640 1 --2- 172.21.15.98:0/204549098 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6a80b9970 0x7fb6a8152a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:17.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.769+0000 7fb6acc17640 1 -- 172.21.15.98:0/204549098 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6a80d4820 msgr2=0x7fb6a813cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:17.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.769+0000 7fb6acc17640 1 --2- 172.21.15.98:0/204549098 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6a80d4820 0x7fb6a813cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:17.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.769+0000 7fb6acc17640 1 -- 172.21.15.98:0/204549098 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6a80d01b0 con 0x7fb6a80b7f80 2024-09-19T08:42:17.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.769+0000 7fb6a7fff640 1 --2- 172.21.15.98:0/204549098 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6a80b9970 0x7fb6a8152a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:42:17.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.769+0000 7fb6acc17640 1 --2- 172.21.15.98:0/204549098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6a80b7f80 0x7fb6a813b490 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb69801a0b0 tx=0x7fb69801acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:17.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.769+0000 7fb6a57fa640 1 -- 172.21.15.98:0/204549098 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6980b0050 con 0x7fb6a80b7f80 2024-09-19T08:42:17.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.769+0000 7fb6a57fa640 1 -- 172.21.15.98:0/204549098 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6980b1040 con 0x7fb6a80b7f80 2024-09-19T08:42:17.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.769+0000 7fb6aee7b640 1 -- 172.21.15.98:0/204549098 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6a80d0490 con 0x7fb6a80b7f80 2024-09-19T08:42:17.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.769+0000 7fb6a57fa640 1 -- 172.21.15.98:0/204549098 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb698004220 con 0x7fb6a80b7f80 2024-09-19T08:42:17.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.769+0000 7fb6aee7b640 1 -- 172.21.15.98:0/204549098 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6a8153640 con 0x7fb6a80b7f80 2024-09-19T08:42:17.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.769+0000 7fb6aee7b640 1 -- 172.21.15.98:0/204549098 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6a815f560 con 0x7fb6a80b7f80 2024-09-19T08:42:17.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.769+0000 7fb6a57fa640 1 -- 172.21.15.98:0/204549098 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fb6980be060 con 0x7fb6a80b7f80 2024-09-19T08:42:17.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.769+0000 7fb6a57fa640 1 --2- 172.21.15.98:0/204549098 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb694067c20 0x7fb69406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:17.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.769+0000 7fb6a57fa640 1 -- 172.21.15.98:0/204549098 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fb69811fe10 con 0x7fb6a80b7f80 2024-09-19T08:42:17.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.769+0000 7fb6a7fff640 1 --2- 172.21.15.98:0/204549098 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb694067c20 0x7fb69406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:17.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.769+0000 7fb6a7fff640 1 --2- 172.21.15.98:0/204549098 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb694067c20 0x7fb69406a0e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb6a80d3f30 tx=0x7fb69c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:17.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.773+0000 7fb6a57fa640 1 -- 172.21.15.98:0/204549098 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb698016070 con 0x7fb6a80b7f80 2024-09-19T08:42:17.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.973+0000 7fb6aee7b640 1 -- 172.21.15.98:0/204549098 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb6a8075220 con 0x7fb6a80b7f80 2024-09-19T08:42:17.974 INFO:tasks.ceph.mon.a.smithi098.stderr:2024-09-19T08:42:17.973+0000 7f5e49017640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T08:42:17.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.977+0000 7fb6a57fa640 1 -- 172.21.15.98:0/204549098 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v32) ==== 161+0+0 (secure 0 0 0) 0x7fb6980f0430 con 0x7fb6a80b7f80 2024-09-19T08:42:17.976 INFO:teuthology.orchestra.run.smithi098.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T08:42:17.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.981+0000 7fb6aee7b640 1 -- 172.21.15.98:0/204549098 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb694067c20 msgr2=0x7fb69406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:17.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.981+0000 7fb6aee7b640 1 --2- 172.21.15.98:0/204549098 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb694067c20 0x7fb69406a0e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb6a80d3f30 tx=0x7fb69c009040 comp rx=0 tx=0).stop 2024-09-19T08:42:17.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.981+0000 7fb6aee7b640 1 -- 172.21.15.98:0/204549098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6a80b7f80 msgr2=0x7fb6a813b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:17.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.981+0000 7fb6aee7b640 1 --2- 172.21.15.98:0/204549098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6a80b7f80 0x7fb6a813b490 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb69801a0b0 tx=0x7fb69801acc0 comp rx=0 tx=0).stop 2024-09-19T08:42:17.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.981+0000 7fb6aee7b640 1 -- 172.21.15.98:0/204549098 shutdown_connections 2024-09-19T08:42:17.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.981+0000 7fb6aee7b640 1 --2- 172.21.15.98:0/204549098 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb694067c20 0x7fb69406a0e0 secure :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb6a80d3f30 tx=0x7fb69c009040 comp rx=0 tx=0).stop 2024-09-19T08:42:17.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.981+0000 7fb6aee7b640 1 --2- 172.21.15.98:0/204549098 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6a80d4820 0x7fb6a813cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:17.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.981+0000 7fb6aee7b640 1 --2- 172.21.15.98:0/204549098 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6a80b9970 0x7fb6a8152a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:17.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.981+0000 7fb6aee7b640 1 --2- 172.21.15.98:0/204549098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6a80b7f80 0x7fb6a813b490 secure :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb69801a0b0 tx=0x7fb69801acc0 comp rx=0 tx=0).stop 2024-09-19T08:42:17.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.981+0000 7fb6aee7b640 1 -- 172.21.15.98:0/204549098 >> 172.21.15.98:0/204549098 conn(0x7fb6a8074e40 msgr2=0x7fb6a80d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:17.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.981+0000 7fb6aee7b640 1 -- 172.21.15.98:0/204549098 shutdown_connections 2024-09-19T08:42:17.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:17.981+0000 7fb6aee7b640 1 -- 172.21.15.98:0/204549098 wait complete. 2024-09-19T08:42:17.999 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T08:42:18.018 INFO:tasks.ceph.mds.a.smithi098.stdout:starting mds.a at 2024-09-19T08:42:18.123 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.121+0000 7f734453f640 1 Processor -- start 2024-09-19T08:42:18.123 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.121+0000 7f734453f640 1 -- start start 2024-09-19T08:42:18.124 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.121+0000 7f734453f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f733c145b70 0x7f733c06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:18.124 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.121+0000 7f734453f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f733c071ac0 0x7f733c0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:18.124 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.121+0000 7f734453f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f733c073530 con 0x7f733c06c1d0 2024-09-19T08:42:18.124 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.121+0000 7f734453f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f733c06c620 con 0x7f733c145b70 2024-09-19T08:42:18.124 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.121+0000 7f734453f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f733c073810 con 0x7f733c071ac0 2024-09-19T08:42:18.124 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.125+0000 7f7341ada640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f733c06c1d0 0x7f733c0713b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:34118/0 (socket says 172.21.15.98:34118) 2024-09-19T08:42:18.124 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.125+0000 7f7341ada640 1 -- 172.21.15.98:0/4280024227 learned_addr learned my addr 172.21.15.98:0/4280024227 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:18.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.125+0000 7f7342adc640 1 --2- 172.21.15.98:0/4280024227 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f733c071ac0 0x7f733c0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:18.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.125+0000 7f73422db640 1 --2- 172.21.15.98:0/4280024227 >> v2:172.21.15.139:3300/0 conn(0x7f733c145b70 0x7f733c06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:18.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.125+0000 7f73412d9640 1 -- 172.21.15.98:0/4280024227 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2410302460 0 0) 0x7f733c073530 con 0x7f733c06c1d0 2024-09-19T08:42:18.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.125+0000 7f73412d9640 1 -- 172.21.15.98:0/4280024227 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7328003530 con 0x7f733c06c1d0 2024-09-19T08:42:18.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.125+0000 7f73422db640 1 -- 172.21.15.98:0/4280024227 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f733c071ac0 msgr2=0x7f733c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:18.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.125+0000 7f73422db640 1 --2- 172.21.15.98:0/4280024227 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f733c071ac0 0x7f733c0b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:18.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.125+0000 7f73422db640 1 -- 172.21.15.98:0/4280024227 >> v1:172.21.15.139:6789/0 conn(0x7f733c06c1d0 legacy=0x7f733c0713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:18.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.125+0000 7f73422db640 1 -- 172.21.15.98:0/4280024227 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f733c073af0 con 0x7f733c145b70 2024-09-19T08:42:18.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.125+0000 7f7342adc640 1 --2- 172.21.15.98:0/4280024227 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f733c071ac0 0x7f733c0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:42:18.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.125+0000 7f73422db640 1 --2- 172.21.15.98:0/4280024227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f733c145b70 0x7f733c06bc90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f73340097c0 tx=0x7f733401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=397af49008b11fe2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:18.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.125+0000 7f73412d9640 1 -- 172.21.15.98:0/4280024227 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73340b0070 con 0x7f733c145b70 2024-09-19T08:42:18.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.129+0000 7f73412d9640 1 -- 172.21.15.98:0/4280024227 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73340ab600 con 0x7f733c145b70 2024-09-19T08:42:18.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.129+0000 7f73412d9640 1 -- 172.21.15.98:0/4280024227 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73340ab9b0 con 0x7f733c145b70 2024-09-19T08:42:18.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.129+0000 7f734453f640 1 -- 172.21.15.98:0/4280024227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f733c145b70 msgr2=0x7f733c06bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:18.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.129+0000 7f734453f640 1 --2- 172.21.15.98:0/4280024227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f733c145b70 0x7f733c06bc90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f73340097c0 tx=0x7f733401c410 comp rx=0 tx=0).stop 2024-09-19T08:42:18.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.129+0000 7f734453f640 1 -- 172.21.15.98:0/4280024227 shutdown_connections 2024-09-19T08:42:18.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.129+0000 7f734453f640 1 --2- 172.21.15.98:0/4280024227 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f733c071ac0 0x7f733c0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:18.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.129+0000 7f734453f640 1 --2- 172.21.15.98:0/4280024227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f733c145b70 0x7f733c06bc90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:18.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.129+0000 7f734453f640 1 -- 172.21.15.98:0/4280024227 >> 172.21.15.98:0/4280024227 conn(0x7f733c0b2010 msgr2=0x7f733c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:18.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.129+0000 7f734453f640 1 -- 172.21.15.98:0/4280024227 shutdown_connections 2024-09-19T08:42:18.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.129+0000 7f734453f640 1 -- 172.21.15.98:0/4280024227 wait complete. 2024-09-19T08:42:18.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.129+0000 7f734453f640 1 Processor -- start 2024-09-19T08:42:18.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.129+0000 7f734453f640 1 -- start start 2024-09-19T08:42:18.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.129+0000 7f734453f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f733c06c1d0 0x7f733c159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:18.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.129+0000 7f734453f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f733c071ac0 0x7f733c140a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:18.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.129+0000 7f734453f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f733c15f240 0x7f733c140fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:18.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.129+0000 7f734453f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f733c0b44f0 con 0x7f733c06c1d0 2024-09-19T08:42:18.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.129+0000 7f734453f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f733c0b4370 con 0x7f733c071ac0 2024-09-19T08:42:18.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.129+0000 7f734453f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f733c0b4670 con 0x7f733c15f240 2024-09-19T08:42:18.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.129+0000 7f73422db640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f733c06c1d0 0x7f733c159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:18.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.129+0000 7f73422db640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f733c06c1d0 0x7f733c159460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54326/0 (socket says 172.21.15.98:54326) 2024-09-19T08:42:18.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.129+0000 7f73422db640 1 -- 172.21.15.98:0/86887 learned_addr learned my addr 172.21.15.98:0/86887 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:18.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.129+0000 7f73422db640 1 -- 172.21.15.98:0/86887 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f733c15f240 msgr2=0x7f733c140fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:18.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.129+0000 7f73422db640 1 --2- 172.21.15.98:0/86887 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f733c15f240 0x7f733c140fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:18.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.129+0000 7f73422db640 1 -- 172.21.15.98:0/86887 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f733c071ac0 msgr2=0x7f733c140a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:18.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.129+0000 7f73422db640 1 --2- 172.21.15.98:0/86887 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f733c071ac0 0x7f733c140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:18.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.129+0000 7f73422db640 1 -- 172.21.15.98:0/86887 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f733c15e1a0 con 0x7f733c06c1d0 2024-09-19T08:42:18.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.129+0000 7f73422db640 1 --2- 172.21.15.98:0/86887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f733c06c1d0 0x7f733c159460 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f7334000c10 tx=0x7f73340191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:18.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.129+0000 7f7332ffd640 1 -- 172.21.15.98:0/86887 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73340ba070 con 0x7f733c06c1d0 2024-09-19T08:42:18.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.129+0000 7f7332ffd640 1 -- 172.21.15.98:0/86887 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73340ab9b0 con 0x7f733c06c1d0 2024-09-19T08:42:18.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.129+0000 7f734453f640 1 -- 172.21.15.98:0/86887 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f733c15e480 con 0x7f733c06c1d0 2024-09-19T08:42:18.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.129+0000 7f7332ffd640 1 -- 172.21.15.98:0/86887 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73340b0040 con 0x7f733c06c1d0 2024-09-19T08:42:18.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.129+0000 7f734453f640 1 -- 172.21.15.98:0/86887 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f733c15ea40 con 0x7f733c06c1d0 2024-09-19T08:42:18.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.133+0000 7f734453f640 1 -- 172.21.15.98:0/86887 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f733c06bc90 con 0x7f733c06c1d0 2024-09-19T08:42:18.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.133+0000 7f7332ffd640 1 -- 172.21.15.98:0/86887 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f73340bd070 con 0x7f733c06c1d0 2024-09-19T08:42:18.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.133+0000 7f7332ffd640 1 --2- 172.21.15.98:0/86887 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7320067c20 0x7f732006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:18.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.133+0000 7f7332ffd640 1 -- 172.21.15.98:0/86887 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5795+0+0 (secure 0 0 0) 0x7f733411f3a0 con 0x7f733c06c1d0 2024-09-19T08:42:18.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.133+0000 7f7341ada640 1 --2- 172.21.15.98:0/86887 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7320067c20 0x7f732006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:18.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.133+0000 7f7341ada640 1 --2- 172.21.15.98:0/86887 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7320067c20 0x7f732006a0e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f732c007fd0 tx=0x7f732c007940 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:18.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.137+0000 7f7332ffd640 1 -- 172.21.15.98:0/86887 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f7334016090 con 0x7f733c06c1d0 2024-09-19T08:42:18.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.337+0000 7f734453f640 1 -- 172.21.15.98:0/86887 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f733c0b1a10 con 0x7f733c06c1d0 2024-09-19T08:42:18.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.985+0000 7f7332ffd640 1 -- 172.21.15.98:0/86887 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v33) ==== 104+0+0 (secure 0 0 0) 0x7f73340ef960 con 0x7f733c06c1d0 2024-09-19T08:42:18.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.989+0000 7f734453f640 1 -- 172.21.15.98:0/86887 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7320067c20 msgr2=0x7f732006a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:18.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.989+0000 7f734453f640 1 --2- 172.21.15.98:0/86887 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7320067c20 0x7f732006a0e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f732c007fd0 tx=0x7f732c007940 comp rx=0 tx=0).stop 2024-09-19T08:42:18.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.989+0000 7f734453f640 1 -- 172.21.15.98:0/86887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f733c06c1d0 msgr2=0x7f733c159460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:18.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.989+0000 7f734453f640 1 --2- 172.21.15.98:0/86887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f733c06c1d0 0x7f733c159460 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f7334000c10 tx=0x7f73340191b0 comp rx=0 tx=0).stop 2024-09-19T08:42:18.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.989+0000 7f734453f640 1 -- 172.21.15.98:0/86887 shutdown_connections 2024-09-19T08:42:18.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.989+0000 7f734453f640 1 --2- 172.21.15.98:0/86887 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7320067c20 0x7f732006a0e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:18.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.989+0000 7f734453f640 1 --2- 172.21.15.98:0/86887 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f733c15f240 0x7f733c140fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:18.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.989+0000 7f734453f640 1 --2- 172.21.15.98:0/86887 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f733c071ac0 0x7f733c140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:18.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.989+0000 7f734453f640 1 --2- 172.21.15.98:0/86887 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f733c06c1d0 0x7f733c159460 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:18.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.989+0000 7f734453f640 1 -- 172.21.15.98:0/86887 >> 172.21.15.98:0/86887 conn(0x7f733c0b2010 msgr2=0x7f733c160610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:18.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.989+0000 7f734453f640 1 -- 172.21.15.98:0/86887 shutdown_connections 2024-09-19T08:42:18.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:18.989+0000 7f734453f640 1 -- 172.21.15.98:0/86887 wait complete. 2024-09-19T08:42:19.011 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T08:42:19.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.185+0000 7f6f91b48640 1 Processor -- start 2024-09-19T08:42:19.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.185+0000 7f6f91b48640 1 -- start start 2024-09-19T08:42:19.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.185+0000 7f6f91b48640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f8c0cf700 0x7f6f8c15ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:19.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.185+0000 7f6f91b48640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f8c1608c0 con 0x7f6f8c06d4c0 2024-09-19T08:42:19.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.185+0000 7f6f91b48640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f8c160ba0 con 0x7f6f8c15c290 2024-09-19T08:42:19.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.185+0000 7f6f91b48640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6f8c160e80 con 0x7f6f8c0cf700 2024-09-19T08:42:19.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.185+0000 7f6f8bfff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f6f8c06d4c0 0x7f6f8c0b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:58956/0 (socket says 172.21.15.98:58956) 2024-09-19T08:42:19.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.185+0000 7f6f8bfff640 1 -- 172.21.15.98:0/2104829284 learned_addr learned my addr 172.21.15.98:0/2104829284 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:19.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.185+0000 7f6f8affd640 1 --2- 172.21.15.98:0/2104829284 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f8c0cf700 0x7f6f8c15ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:19.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.185+0000 7f6f8a7fc640 1 -- 172.21.15.98:0/2104829284 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2526087659 0 0) 0x7f6f8c1608c0 con 0x7f6f8c06d4c0 2024-09-19T08:42:19.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.185+0000 7f6f8a7fc640 1 -- 172.21.15.98:0/2104829284 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f78003530 con 0x7f6f8c06d4c0 2024-09-19T08:42:19.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.185+0000 7f6f8a7fc640 1 -- 172.21.15.98:0/2104829284 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3439291525 0 0) 0x7f6f78003530 con 0x7f6f8c06d4c0 2024-09-19T08:42:19.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.185+0000 7f6f8a7fc640 1 -- 172.21.15.98:0/2104829284 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f8c0cf700 msgr2=0x7f6f8c15ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:19.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.185+0000 7f6f8a7fc640 1 --2- 172.21.15.98:0/2104829284 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f8c0cf700 0x7f6f8c15ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:19.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.185+0000 7f6f8a7fc640 1 -- 172.21.15.98:0/2104829284 >> v1:172.21.15.139:6789/0 conn(0x7f6f8c15c290 legacy=0x7f6f8c0cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:19.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.185+0000 7f6f8a7fc640 1 -- 172.21.15.98:0/2104829284 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8c0cd1a0 con 0x7f6f8c06d4c0 2024-09-19T08:42:19.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.185+0000 7f6f8a7fc640 1 -- 172.21.15.98:0/2104829284 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f6f84001a00 con 0x7f6f8c06d4c0 2024-09-19T08:42:19.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.185+0000 7f6f8a7fc640 1 -- 172.21.15.98:0/2104829284 >> v1:172.21.15.98:6789/0 conn(0x7f6f8c06d4c0 legacy=0x7f6f8c0b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:19.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.185+0000 7f6f8a7fc640 1 --2- 172.21.15.98:0/2104829284 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f8c0cf700 0x7f6f780039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:19.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.185+0000 7f6f8a7fc640 1 --2- 172.21.15.98:0/2104829284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f78003f70 0x7f6f78004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:19.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.185+0000 7f6f8a7fc640 1 --2- 172.21.15.98:0/2104829284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f78004d50 0x7f6f780051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:19.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.185+0000 7f6f8a7fc640 1 -- 172.21.15.98:0/2104829284 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7800ba80 con 0x7f6f78003f70 2024-09-19T08:42:19.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.185+0000 7f6f8a7fc640 1 -- 172.21.15.98:0/2104829284 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7800bc30 con 0x7f6f78004d50 2024-09-19T08:42:19.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.185+0000 7f6f8a7fc640 1 -- 172.21.15.98:0/2104829284 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6f7800be30 con 0x7f6f8c0cf700 2024-09-19T08:42:19.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.185+0000 7f6f8affd640 1 --2- 172.21.15.98:0/2104829284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f78003f70 0x7f6f78004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:19.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.185+0000 7f6f8b7fe640 1 --2- 172.21.15.98:0/2104829284 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f8c0cf700 0x7f6f780039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:19.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.185+0000 7f6f8bfff640 1 --2- 172.21.15.98:0/2104829284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f78004d50 0x7f6f780051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:19.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.185+0000 7f6f8affd640 1 -- 172.21.15.98:0/2104829284 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f8c0cf700 msgr2=0x7f6f780039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:19.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.185+0000 7f6f8affd640 1 --2- 172.21.15.98:0/2104829284 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f8c0cf700 0x7f6f780039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:19.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.185+0000 7f6f8affd640 1 -- 172.21.15.98:0/2104829284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f78004d50 msgr2=0x7f6f780051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:19.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.185+0000 7f6f8affd640 1 --2- 172.21.15.98:0/2104829284 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f78004d50 0x7f6f780051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:19.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.185+0000 7f6f8affd640 1 -- 172.21.15.98:0/2104829284 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f78003530 con 0x7f6f78003f70 2024-09-19T08:42:19.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.189+0000 7f6f8affd640 1 --2- 172.21.15.98:0/2104829284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f78003f70 0x7f6f78004810 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f6f74009a90 tx=0x7f6f7401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de90baa6655c9c6a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:19.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.189+0000 7f6f8a7fc640 1 -- 172.21.15.98:0/2104829284 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f740b2070 con 0x7f6f78003f70 2024-09-19T08:42:19.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.189+0000 7f6f8a7fc640 1 -- 172.21.15.98:0/2104829284 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f740a5620 con 0x7f6f78003f70 2024-09-19T08:42:19.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.189+0000 7f6f8a7fc640 1 -- 172.21.15.98:0/2104829284 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f74009690 con 0x7f6f78003f70 2024-09-19T08:42:19.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.189+0000 7f6f91b48640 1 -- 172.21.15.98:0/2104829284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f78003f70 msgr2=0x7f6f78004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:19.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.189+0000 7f6f91b48640 1 --2- 172.21.15.98:0/2104829284 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f78003f70 0x7f6f78004810 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f6f74009a90 tx=0x7f6f7401ca30 comp rx=0 tx=0).stop 2024-09-19T08:42:19.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.189+0000 7f6f8bfff640 1 -- 172.21.15.98:0/2104829284 reap_dead start 2024-09-19T08:42:19.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.189+0000 7f6f91b48640 1 -- 172.21.15.98:0/2104829284 shutdown_connections 2024-09-19T08:42:19.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.189+0000 7f6f91b48640 1 -- 172.21.15.98:0/2104829284 >> 172.21.15.98:0/2104829284 conn(0x7f6f8c06c090 msgr2=0x7f6f8c06c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:19.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.189+0000 7f6f91b48640 1 -- 172.21.15.98:0/2104829284 shutdown_connections 2024-09-19T08:42:19.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.189+0000 7f6f91b48640 1 -- 172.21.15.98:0/2104829284 wait complete. 2024-09-19T08:42:19.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.189+0000 7f6f91b48640 1 Processor -- start 2024-09-19T08:42:19.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.189+0000 7f6f91b48640 1 -- start start 2024-09-19T08:42:19.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.193+0000 7f6f91b48640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f78003f70 0x7f6f8c155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:19.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.193+0000 7f6f91b48640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f8c15db50 0x7f6f8c15dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:19.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.193+0000 7f6f91b48640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f8c1f2150 0x7f6f8c1f25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:19.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.193+0000 7f6f91b48640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8c0b5300 con 0x7f6f78003f70 2024-09-19T08:42:19.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.193+0000 7f6f91b48640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8c0cd510 con 0x7f6f8c15db50 2024-09-19T08:42:19.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.193+0000 7f6f91b48640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6f8c0b5480 con 0x7f6f8c1f2150 2024-09-19T08:42:19.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.193+0000 7f6f8b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f78003f70 0x7f6f8c155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:19.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.193+0000 7f6f8b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f78003f70 0x7f6f8c155220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54338/0 (socket says 172.21.15.98:54338) 2024-09-19T08:42:19.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.193+0000 7f6f8b7fe640 1 -- 172.21.15.98:0/2079368184 learned_addr learned my addr 172.21.15.98:0/2079368184 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:19.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.193+0000 7f6f8affd640 1 --2- 172.21.15.98:0/2079368184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f8c15db50 0x7f6f8c15dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:19.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.193+0000 7f6f8bfff640 1 --2- 172.21.15.98:0/2079368184 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f8c1f2150 0x7f6f8c1f25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:19.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.193+0000 7f6f8b7fe640 1 -- 172.21.15.98:0/2079368184 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f8c1f2150 msgr2=0x7f6f8c1f25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:19.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.193+0000 7f6f8b7fe640 1 --2- 172.21.15.98:0/2079368184 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f8c1f2150 0x7f6f8c1f25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:19.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.193+0000 7f6f8b7fe640 1 -- 172.21.15.98:0/2079368184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f8c15db50 msgr2=0x7f6f8c15dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:19.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.193+0000 7f6f8b7fe640 1 --2- 172.21.15.98:0/2079368184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f8c15db50 0x7f6f8c15dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:19.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.193+0000 7f6f8b7fe640 1 -- 172.21.15.98:0/2079368184 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8c155a80 con 0x7f6f78003f70 2024-09-19T08:42:19.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.193+0000 7f6f8b7fe640 1 --2- 172.21.15.98:0/2079368184 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f78003f70 0x7f6f8c155220 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c00e550 tx=0x7f6f7c00ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:19.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.193+0000 7f6f83fff640 1 -- 172.21.15.98:0/2079368184 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c0052e0 con 0x7f6f78003f70 2024-09-19T08:42:19.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.193+0000 7f6f83fff640 1 -- 172.21.15.98:0/2079368184 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f7c018070 con 0x7f6f78003f70 2024-09-19T08:42:19.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.193+0000 7f6f91b48640 1 -- 172.21.15.98:0/2079368184 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f8c155d60 con 0x7f6f78003f70 2024-09-19T08:42:19.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.193+0000 7f6f83fff640 1 -- 172.21.15.98:0/2079368184 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c003820 con 0x7f6f78003f70 2024-09-19T08:42:19.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.193+0000 7f6f91b48640 1 -- 172.21.15.98:0/2079368184 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f8c1f2b10 con 0x7f6f78003f70 2024-09-19T08:42:19.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.193+0000 7f6f83fff640 1 -- 172.21.15.98:0/2079368184 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f6f7c0039c0 con 0x7f6f78003f70 2024-09-19T08:42:19.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.193+0000 7f6f91b48640 1 -- 172.21.15.98:0/2079368184 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f8c06d930 con 0x7f6f78003f70 2024-09-19T08:42:19.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.193+0000 7f6f83fff640 1 --2- 172.21.15.98:0/2079368184 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6f6c067c50 0x7f6f6c06a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:19.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.193+0000 7f6f8affd640 1 --2- 172.21.15.98:0/2079368184 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6f6c067c50 0x7f6f6c06a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:19.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.193+0000 7f6f83fff640 1 -- 172.21.15.98:0/2079368184 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5795+0+0 (secure 0 0 0) 0x7f6f7c085f00 con 0x7f6f78003f70 2024-09-19T08:42:19.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.197+0000 7f6f8affd640 1 --2- 172.21.15.98:0/2079368184 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6f6c067c50 0x7f6f6c06a110 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6f740af000 tx=0x7f6f740046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:19.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.201+0000 7f6f83fff640 1 -- 172.21.15.98:0/2079368184 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6f7c056620 con 0x7f6f78003f70 2024-09-19T08:42:19.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:19.361+0000 7f6f91b48640 1 -- 172.21.15.98:0/2079368184 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6f8c0b4f70 con 0x7f6f78003f70 2024-09-19T08:42:20.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.297+0000 7f6f83fff640 1 -- 172.21.15.98:0/2079368184 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v59) ==== 181+0+0 (secure 0 0 0) 0x7f6f7c005480 con 0x7f6f78003f70 2024-09-19T08:42:20.297 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_metadata' removed 2024-09-19T08:42:20.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.297+0000 7f6f91b48640 1 -- 172.21.15.98:0/2079368184 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6f6c067c50 msgr2=0x7f6f6c06a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:20.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.297+0000 7f6f91b48640 1 --2- 172.21.15.98:0/2079368184 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6f6c067c50 0x7f6f6c06a110 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6f740af000 tx=0x7f6f740046d0 comp rx=0 tx=0).stop 2024-09-19T08:42:20.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.297+0000 7f6f91b48640 1 -- 172.21.15.98:0/2079368184 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f78003f70 msgr2=0x7f6f8c155220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:20.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.297+0000 7f6f91b48640 1 --2- 172.21.15.98:0/2079368184 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f78003f70 0x7f6f8c155220 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c00e550 tx=0x7f6f7c00ea10 comp rx=0 tx=0).stop 2024-09-19T08:42:20.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.297+0000 7f6f91b48640 1 -- 172.21.15.98:0/2079368184 shutdown_connections 2024-09-19T08:42:20.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.297+0000 7f6f91b48640 1 --2- 172.21.15.98:0/2079368184 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6f6c067c50 0x7f6f6c06a110 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:20.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.297+0000 7f6f91b48640 1 --2- 172.21.15.98:0/2079368184 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f8c1f2150 0x7f6f8c1f25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:20.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.297+0000 7f6f91b48640 1 --2- 172.21.15.98:0/2079368184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f8c15db50 0x7f6f8c15dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:20.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.297+0000 7f6f91b48640 1 --2- 172.21.15.98:0/2079368184 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f78003f70 0x7f6f8c155220 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:20.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.297+0000 7f6f91b48640 1 -- 172.21.15.98:0/2079368184 >> 172.21.15.98:0/2079368184 conn(0x7f6f8c06c090 msgr2=0x7f6f8c0cfc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:20.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.297+0000 7f6f91b48640 1 -- 172.21.15.98:0/2079368184 shutdown_connections 2024-09-19T08:42:20.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.297+0000 7f6f91b48640 1 -- 172.21.15.98:0/2079368184 wait complete. 2024-09-19T08:42:20.326 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T08:42:20.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.453+0000 7f7d8908f640 1 Processor -- start 2024-09-19T08:42:20.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.453+0000 7f7d8908f640 1 -- start start 2024-09-19T08:42:20.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.453+0000 7f7d8908f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7d8415c290 0x7f7d840cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:20.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.453+0000 7f7d8908f640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7d840cf700 0x7f7d8415ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:20.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.453+0000 7f7d8908f640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d841608c0 con 0x7f7d8406d4c0 2024-09-19T08:42:20.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.453+0000 7f7d8908f640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f7d84160e80 con 0x7f7d840cf700 2024-09-19T08:42:20.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.453+0000 7f7d8908f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f7d840cd510 con 0x7f7d8415c290 2024-09-19T08:42:20.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.453+0000 7f7d8359e640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f7d8406d4c0 0x7f7d8406d8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:58970/0 (socket says 172.21.15.98:58970) 2024-09-19T08:42:20.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.453+0000 7f7d8259c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7d840cf700 0x7f7d8415ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:20.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.453+0000 7f7d8359e640 1 -- 172.21.15.98:0/3134859283 learned_addr learned my addr 172.21.15.98:0/3134859283 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:20.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.453+0000 7f7d82d9d640 1 --2- 172.21.15.98:0/3134859283 >> v2:172.21.15.139:3300/0 conn(0x7f7d8415c290 0x7f7d840cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:20.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.453+0000 7f7d81d9b640 1 -- 172.21.15.98:0/3134859283 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2495131086 0 0) 0x7f7d841608c0 con 0x7f7d8406d4c0 2024-09-19T08:42:20.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.453+0000 7f7d81d9b640 1 -- 172.21.15.98:0/3134859283 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d6c003530 con 0x7f7d8406d4c0 2024-09-19T08:42:20.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.457+0000 7f7d8259c640 1 -- 172.21.15.98:0/3134859283 >> v2:172.21.15.139:3300/0 conn(0x7f7d8415c290 msgr2=0x7f7d840cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:20.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.457+0000 7f7d8259c640 1 --2- 172.21.15.98:0/3134859283 >> v2:172.21.15.139:3300/0 conn(0x7f7d8415c290 0x7f7d840cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:20.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.457+0000 7f7d8259c640 1 -- 172.21.15.98:0/3134859283 >> v1:172.21.15.98:6789/0 conn(0x7f7d8406d4c0 legacy=0x7f7d8406d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:20.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.457+0000 7f7d8259c640 1 -- 172.21.15.98:0/3134859283 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d84160ba0 con 0x7f7d840cf700 2024-09-19T08:42:20.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.457+0000 7f7d8259c640 1 --2- 172.21.15.98:0/3134859283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d840cf700 0x7f7d8415ff50 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f7d840742e0 tx=0x7f7d7801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87283434a48faef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:20.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.457+0000 7f7d81d9b640 1 -- 172.21.15.98:0/3134859283 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d780b0070 con 0x7f7d840cf700 2024-09-19T08:42:20.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.457+0000 7f7d81d9b640 1 -- 172.21.15.98:0/3134859283 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d780ab600 con 0x7f7d840cf700 2024-09-19T08:42:20.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.457+0000 7f7d81d9b640 1 -- 172.21.15.98:0/3134859283 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d780ab9d0 con 0x7f7d840cf700 2024-09-19T08:42:20.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.461+0000 7f7d8908f640 1 -- 172.21.15.98:0/3134859283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d840cf700 msgr2=0x7f7d8415ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:20.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.461+0000 7f7d8908f640 1 --2- 172.21.15.98:0/3134859283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d840cf700 0x7f7d8415ff50 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f7d840742e0 tx=0x7f7d7801c3f0 comp rx=0 tx=0).stop 2024-09-19T08:42:20.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.461+0000 7f7d8908f640 1 -- 172.21.15.98:0/3134859283 shutdown_connections 2024-09-19T08:42:20.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.461+0000 7f7d8908f640 1 --2- 172.21.15.98:0/3134859283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d840cf700 0x7f7d8415ff50 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:20.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.461+0000 7f7d8908f640 1 --2- 172.21.15.98:0/3134859283 >> v2:172.21.15.139:3300/0 conn(0x7f7d8415c290 0x7f7d840cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:20.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.461+0000 7f7d8908f640 1 -- 172.21.15.98:0/3134859283 >> 172.21.15.98:0/3134859283 conn(0x7f7d8406c090 msgr2=0x7f7d8406c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:20.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.461+0000 7f7d8908f640 1 -- 172.21.15.98:0/3134859283 shutdown_connections 2024-09-19T08:42:20.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.461+0000 7f7d8908f640 1 -- 172.21.15.98:0/3134859283 wait complete. 2024-09-19T08:42:20.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.461+0000 7f7d8908f640 1 Processor -- start 2024-09-19T08:42:20.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.461+0000 7f7d8908f640 1 -- start start 2024-09-19T08:42:20.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.461+0000 7f7d8908f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d8406d4c0 0x7f7d840af790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:20.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.461+0000 7f7d8908f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7d840cf700 0x7f7d8415aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:20.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.461+0000 7f7d8908f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d8415c290 0x7f7d841592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:20.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.461+0000 7f7d8908f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7d84160610 con 0x7f7d8415c290 2024-09-19T08:42:20.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.461+0000 7f7d8908f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7d84160490 con 0x7f7d8406d4c0 2024-09-19T08:42:20.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.461+0000 7f7d8908f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7d840b5400 con 0x7f7d840cf700 2024-09-19T08:42:20.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.461+0000 7f7d8359e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d8415c290 0x7f7d841592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:20.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.461+0000 7f7d8359e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d8415c290 0x7f7d841592f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54362/0 (socket says 172.21.15.98:54362) 2024-09-19T08:42:20.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.461+0000 7f7d8359e640 1 -- 172.21.15.98:0/2792780757 learned_addr learned my addr 172.21.15.98:0/2792780757 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:20.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.461+0000 7f7d8259c640 1 --2- 172.21.15.98:0/2792780757 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7d840cf700 0x7f7d8415aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:20.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.461+0000 7f7d82d9d640 1 --2- 172.21.15.98:0/2792780757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d8406d4c0 0x7f7d840af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:20.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.461+0000 7f7d8359e640 1 -- 172.21.15.98:0/2792780757 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7d840cf700 msgr2=0x7f7d8415aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:20.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.461+0000 7f7d8359e640 1 --2- 172.21.15.98:0/2792780757 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7d840cf700 0x7f7d8415aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:20.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.461+0000 7f7d8359e640 1 -- 172.21.15.98:0/2792780757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d8406d4c0 msgr2=0x7f7d840af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:20.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.461+0000 7f7d8359e640 1 --2- 172.21.15.98:0/2792780757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d8406d4c0 0x7f7d840af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:20.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.461+0000 7f7d8359e640 1 -- 172.21.15.98:0/2792780757 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d841641f0 con 0x7f7d8415c290 2024-09-19T08:42:20.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.461+0000 7f7d8359e640 1 --2- 172.21.15.98:0/2792780757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d8415c290 0x7f7d841592f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c00cb40 tx=0x7f7d7c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:20.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.465+0000 7f7d6b7fe640 1 -- 172.21.15.98:0/2792780757 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c017070 con 0x7f7d8415c290 2024-09-19T08:42:20.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.465+0000 7f7d6b7fe640 1 -- 172.21.15.98:0/2792780757 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d7c00aca0 con 0x7f7d8415c290 2024-09-19T08:42:20.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.465+0000 7f7d8908f640 1 -- 172.21.15.98:0/2792780757 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d8415b430 con 0x7f7d8415c290 2024-09-19T08:42:20.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.465+0000 7f7d6b7fe640 1 -- 172.21.15.98:0/2792780757 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c012620 con 0x7f7d8415c290 2024-09-19T08:42:20.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.465+0000 7f7d8908f640 1 -- 172.21.15.98:0/2792780757 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d8415ac10 con 0x7f7d8415c290 2024-09-19T08:42:20.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.465+0000 7f7d8908f640 1 -- 172.21.15.98:0/2792780757 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d841643c0 con 0x7f7d8415c290 2024-09-19T08:42:20.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.465+0000 7f7d6b7fe640 1 -- 172.21.15.98:0/2792780757 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f7d7c01a070 con 0x7f7d8415c290 2024-09-19T08:42:20.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.465+0000 7f7d6b7fe640 1 --2- 172.21.15.98:0/2792780757 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7d70067c20 0x7f7d7006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:20.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.465+0000 7f7d6b7fe640 1 -- 172.21.15.98:0/2792780757 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5357+0+0 (secure 0 0 0) 0x7f7d7c085e20 con 0x7f7d8415c290 2024-09-19T08:42:20.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.465+0000 7f7d82d9d640 1 --2- 172.21.15.98:0/2792780757 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7d70067c20 0x7f7d7006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:20.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.465+0000 7f7d82d9d640 1 --2- 172.21.15.98:0/2792780757 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7d70067c20 0x7f7d7006a0e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400b930 tx=0x7f7d74008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:20.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.469+0000 7f7d6b7fe640 1 -- 172.21.15.98:0/2792780757 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f7d7c056740 con 0x7f7d8415c290 2024-09-19T08:42:20.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:20.633+0000 7f7d8908f640 1 -- 172.21.15.98:0/2792780757 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7d8415a6b0 con 0x7f7d8415c290 2024-09-19T08:42:21.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.297+0000 7f7d6b7fe640 1 -- 172.21.15.98:0/2792780757 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v60) ==== 169+0+0 (secure 0 0 0) 0x7f7d8415a6b0 con 0x7f7d8415c290 2024-09-19T08:42:21.299 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' removed 2024-09-19T08:42:21.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.305+0000 7f7d8908f640 1 -- 172.21.15.98:0/2792780757 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7d70067c20 msgr2=0x7f7d7006a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:21.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.305+0000 7f7d8908f640 1 --2- 172.21.15.98:0/2792780757 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7d70067c20 0x7f7d7006a0e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400b930 tx=0x7f7d74008040 comp rx=0 tx=0).stop 2024-09-19T08:42:21.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.305+0000 7f7d8908f640 1 -- 172.21.15.98:0/2792780757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d8415c290 msgr2=0x7f7d841592f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:21.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.305+0000 7f7d8908f640 1 --2- 172.21.15.98:0/2792780757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d8415c290 0x7f7d841592f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c00cb40 tx=0x7f7d7c00a060 comp rx=0 tx=0).stop 2024-09-19T08:42:21.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.305+0000 7f7d8908f640 1 -- 172.21.15.98:0/2792780757 shutdown_connections 2024-09-19T08:42:21.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.305+0000 7f7d8908f640 1 --2- 172.21.15.98:0/2792780757 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f7d70067c20 0x7f7d7006a0e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:21.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.305+0000 7f7d8908f640 1 --2- 172.21.15.98:0/2792780757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d8415c290 0x7f7d841592f0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:21.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.305+0000 7f7d8908f640 1 --2- 172.21.15.98:0/2792780757 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7d840cf700 0x7f7d8415aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:21.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.305+0000 7f7d8908f640 1 --2- 172.21.15.98:0/2792780757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d8406d4c0 0x7f7d840af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:21.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.305+0000 7f7d8908f640 1 -- 172.21.15.98:0/2792780757 >> 172.21.15.98:0/2792780757 conn(0x7f7d8406c090 msgr2=0x7f7d840b0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:21.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.305+0000 7f7d8908f640 1 -- 172.21.15.98:0/2792780757 shutdown_connections 2024-09-19T08:42:21.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.305+0000 7f7d8908f640 1 -- 172.21.15.98:0/2792780757 wait complete. 2024-09-19T08:42:21.330 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-19T08:42:21.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.453+0000 7f9a75bea640 1 Processor -- start 2024-09-19T08:42:21.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.453+0000 7f9a75bea640 1 -- start start 2024-09-19T08:42:21.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.453+0000 7f9a75bea640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9a70158e90 0x7f9a700d0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:21.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.453+0000 7f9a75bea640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a700d1190 0x7f9a7006e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:21.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.453+0000 7f9a75bea640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a700cfeb0 con 0x7f9a7006e9c0 2024-09-19T08:42:21.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.453+0000 7f9a75bea640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f9a7015c7b0 con 0x7f9a70158e90 2024-09-19T08:42:21.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.453+0000 7f9a75bea640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9a700d0170 con 0x7f9a700d1190 2024-09-19T08:42:21.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.453+0000 7f9a6f7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9a70158e90 0x7f9a700d0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:21.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.453+0000 7f9a6f7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9a70158e90 0x7f9a700d0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54364/0 (socket says 172.21.15.98:54364) 2024-09-19T08:42:21.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.453+0000 7f9a6f7fe640 1 -- 172.21.15.98:0/3937325415 learned_addr learned my addr 172.21.15.98:0/3937325415 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:21.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.453+0000 7f9a6f7fe640 1 -- 172.21.15.98:0/3937325415 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a700d1190 msgr2=0x7f9a7006e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:21.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.453+0000 7f9a6f7fe640 1 --2- 172.21.15.98:0/3937325415 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a700d1190 0x7f9a7006e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:21.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.453+0000 7f9a6f7fe640 1 -- 172.21.15.98:0/3937325415 >> v1:172.21.15.98:6789/0 conn(0x7f9a7006e9c0 legacy=0x7f9a7015ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:21.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.453+0000 7f9a6f7fe640 1 -- 172.21.15.98:0/3937325415 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a700d0450 con 0x7f9a70158e90 2024-09-19T08:42:21.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.457+0000 7f9a6f7fe640 1 --2- 172.21.15.98:0/3937325415 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a70158e90 0x7f9a700d0c50 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9a68009e50 tx=0x7f9a6801c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6fb474aba2169b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:21.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.457+0000 7f9a6e7fc640 1 -- 172.21.15.98:0/3937325415 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a680a5650 con 0x7f9a70158e90 2024-09-19T08:42:21.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.457+0000 7f9a6e7fc640 1 -- 172.21.15.98:0/3937325415 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a680a57f0 con 0x7f9a70158e90 2024-09-19T08:42:21.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.457+0000 7f9a6e7fc640 1 -- 172.21.15.98:0/3937325415 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a680a5bc0 con 0x7f9a70158e90 2024-09-19T08:42:21.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.457+0000 7f9a75bea640 1 -- 172.21.15.98:0/3937325415 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a70158e90 msgr2=0x7f9a700d0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:21.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.457+0000 7f9a75bea640 1 --2- 172.21.15.98:0/3937325415 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a70158e90 0x7f9a700d0c50 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9a68009e50 tx=0x7f9a6801c830 comp rx=0 tx=0).stop 2024-09-19T08:42:21.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.457+0000 7f9a75bea640 1 -- 172.21.15.98:0/3937325415 shutdown_connections 2024-09-19T08:42:21.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.457+0000 7f9a75bea640 1 --2- 172.21.15.98:0/3937325415 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a700d1190 0x7f9a7006e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:21.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.457+0000 7f9a75bea640 1 --2- 172.21.15.98:0/3937325415 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a70158e90 0x7f9a700d0c50 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:21.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.457+0000 7f9a75bea640 1 -- 172.21.15.98:0/3937325415 >> 172.21.15.98:0/3937325415 conn(0x7f9a70075270 msgr2=0x7f9a7006bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:21.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.457+0000 7f9a75bea640 1 -- 172.21.15.98:0/3937325415 shutdown_connections 2024-09-19T08:42:21.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.457+0000 7f9a75bea640 1 -- 172.21.15.98:0/3937325415 wait complete. 2024-09-19T08:42:21.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.461+0000 7f9a75bea640 1 Processor -- start 2024-09-19T08:42:21.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.461+0000 7f9a75bea640 1 -- start start 2024-09-19T08:42:21.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.461+0000 7f9a75bea640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a7006e9c0 0x7f9a701502c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:21.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.461+0000 7f9a75bea640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a700d1190 0x7f9a7015e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:21.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.461+0000 7f9a75bea640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a70158e90 0x7f9a701535a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:21.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.461+0000 7f9a75bea640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7015c310 con 0x7f9a700d1190 2024-09-19T08:42:21.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.461+0000 7f9a75bea640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7015c190 con 0x7f9a70158e90 2024-09-19T08:42:21.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.461+0000 7f9a75bea640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9a7015c490 con 0x7f9a7006e9c0 2024-09-19T08:42:21.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.461+0000 7f9a6effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a700d1190 0x7f9a7015e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:21.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.461+0000 7f9a6effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a700d1190 0x7f9a7015e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54370/0 (socket says 172.21.15.98:54370) 2024-09-19T08:42:21.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.461+0000 7f9a6effd640 1 -- 172.21.15.98:0/3742903662 learned_addr learned my addr 172.21.15.98:0/3742903662 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:21.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.461+0000 7f9a6f7fe640 1 --2- 172.21.15.98:0/3742903662 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a7006e9c0 0x7f9a701502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:21.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.461+0000 7f9a6ffff640 1 --2- 172.21.15.98:0/3742903662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a70158e90 0x7f9a701535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:21.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.461+0000 7f9a6effd640 1 -- 172.21.15.98:0/3742903662 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a7006e9c0 msgr2=0x7f9a701502c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:21.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.461+0000 7f9a6effd640 1 --2- 172.21.15.98:0/3742903662 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a7006e9c0 0x7f9a701502c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:21.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.461+0000 7f9a6effd640 1 -- 172.21.15.98:0/3742903662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a70158e90 msgr2=0x7f9a701535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:21.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.461+0000 7f9a6effd640 1 --2- 172.21.15.98:0/3742903662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a70158e90 0x7f9a701535a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:21.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.461+0000 7f9a6effd640 1 -- 172.21.15.98:0/3742903662 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a7015d4c0 con 0x7f9a700d1190 2024-09-19T08:42:21.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.461+0000 7f9a6effd640 1 --2- 172.21.15.98:0/3742903662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a700d1190 0x7f9a7015e560 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f9a6400b3f0 tx=0x7f9a6400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:21.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.461+0000 7f9a57fff640 1 -- 172.21.15.98:0/3742903662 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a64015020 con 0x7f9a700d1190 2024-09-19T08:42:21.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.461+0000 7f9a57fff640 1 -- 172.21.15.98:0/3742903662 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a64011850 con 0x7f9a700d1190 2024-09-19T08:42:21.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.461+0000 7f9a75bea640 1 -- 172.21.15.98:0/3742903662 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a7015d7a0 con 0x7f9a700d1190 2024-09-19T08:42:21.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.461+0000 7f9a57fff640 1 -- 172.21.15.98:0/3742903662 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a64011c40 con 0x7f9a700d1190 2024-09-19T08:42:21.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.461+0000 7f9a75bea640 1 -- 172.21.15.98:0/3742903662 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a7015dd60 con 0x7f9a700d1190 2024-09-19T08:42:21.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.461+0000 7f9a75bea640 1 -- 172.21.15.98:0/3742903662 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a7006e480 con 0x7f9a700d1190 2024-09-19T08:42:21.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.461+0000 7f9a57fff640 1 -- 172.21.15.98:0/3742903662 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f9a64018020 con 0x7f9a700d1190 2024-09-19T08:42:21.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.461+0000 7f9a57fff640 1 --2- 172.21.15.98:0/3742903662 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9a5c067c20 0x7f9a5c06a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:21.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.465+0000 7f9a6f7fe640 1 --2- 172.21.15.98:0/3742903662 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9a5c067c20 0x7f9a5c06a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:21.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.465+0000 7f9a57fff640 1 -- 172.21.15.98:0/3742903662 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 4943+0+0 (secure 0 0 0) 0x7f9a64085e40 con 0x7f9a700d1190 2024-09-19T08:42:21.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.465+0000 7f9a6f7fe640 1 --2- 172.21.15.98:0/3742903662 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9a5c067c20 0x7f9a5c06a0e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9a680ae000 tx=0x7f9a680a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:21.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.469+0000 7f9a57fff640 1 -- 172.21.15.98:0/3742903662 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9a6401f020 con 0x7f9a700d1190 2024-09-19T08:42:21.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.641+0000 7f9a75bea640 1 -- 172.21.15.98:0/3742903662 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f9a700b1020 con 0x7f9a700d1190 2024-09-19T08:42:21.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.649+0000 7f9a57fff640 1 -- 172.21.15.98:0/3742903662 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7f9a640172c0 con 0x7f9a700d1190 2024-09-19T08:42:21.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.649+0000 7f9a75bea640 1 -- 172.21.15.98:0/3742903662 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9a5c067c20 msgr2=0x7f9a5c06a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:21.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.649+0000 7f9a75bea640 1 --2- 172.21.15.98:0/3742903662 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9a5c067c20 0x7f9a5c06a0e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9a680ae000 tx=0x7f9a680a7070 comp rx=0 tx=0).stop 2024-09-19T08:42:21.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.649+0000 7f9a75bea640 1 -- 172.21.15.98:0/3742903662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a700d1190 msgr2=0x7f9a7015e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:21.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.649+0000 7f9a75bea640 1 --2- 172.21.15.98:0/3742903662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a700d1190 0x7f9a7015e560 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f9a6400b3f0 tx=0x7f9a6400b8b0 comp rx=0 tx=0).stop 2024-09-19T08:42:21.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.649+0000 7f9a75bea640 1 -- 172.21.15.98:0/3742903662 shutdown_connections 2024-09-19T08:42:21.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.649+0000 7f9a75bea640 1 --2- 172.21.15.98:0/3742903662 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9a5c067c20 0x7f9a5c06a0e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:21.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.649+0000 7f9a75bea640 1 --2- 172.21.15.98:0/3742903662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a70158e90 0x7f9a701535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:21.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.649+0000 7f9a75bea640 1 --2- 172.21.15.98:0/3742903662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9a700d1190 0x7f9a7015e560 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:21.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.649+0000 7f9a75bea640 1 --2- 172.21.15.98:0/3742903662 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9a7006e9c0 0x7f9a701502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:21.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.649+0000 7f9a75bea640 1 -- 172.21.15.98:0/3742903662 >> 172.21.15.98:0/3742903662 conn(0x7f9a70075270 msgr2=0x7f9a700b1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:21.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.649+0000 7f9a75bea640 1 -- 172.21.15.98:0/3742903662 shutdown_connections 2024-09-19T08:42:21.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.649+0000 7f9a75bea640 1 -- 172.21.15.98:0/3742903662 wait complete. 2024-09-19T08:42:21.669 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade' 2024-09-19T08:42:21.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.789+0000 7f337e700640 1 Processor -- start 2024-09-19T08:42:21.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.789+0000 7f337e700640 1 -- start start 2024-09-19T08:42:21.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.789+0000 7f337e700640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f337815c220 0x7f33780733f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:21.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.789+0000 7f337e700640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33780740b0 con 0x7f3378073930 2024-09-19T08:42:21.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.789+0000 7f337e700640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3378073dd0 con 0x7f33780d1700 2024-09-19T08:42:21.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.789+0000 7f337e700640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f33780b6460 con 0x7f337815c220 2024-09-19T08:42:21.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.789+0000 7f336f7fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f3378073930 0x7f33780d0ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:59000/0 (socket says 172.21.15.98:59000) 2024-09-19T08:42:21.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.789+0000 7f3377fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f337815c220 0x7f33780733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:21.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.789+0000 7f336f7fe640 1 -- 172.21.15.98:0/583537191 learned_addr learned my addr 172.21.15.98:0/583537191 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:21.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.789+0000 7f33777fe640 1 -- 172.21.15.98:0/583537191 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3408217354 0 0) 0x7f33780740b0 con 0x7f3378073930 2024-09-19T08:42:21.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.789+0000 7f33777fe640 1 -- 172.21.15.98:0/583537191 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3364003530 con 0x7f3378073930 2024-09-19T08:42:21.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.789+0000 7f3377fff640 1 -- 172.21.15.98:0/583537191 >> v1:172.21.15.139:6789/0 conn(0x7f33780d1700 legacy=0x7f33780b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:21.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.789+0000 7f3377fff640 1 -- 172.21.15.98:0/583537191 >> v1:172.21.15.98:6789/0 conn(0x7f3378073930 legacy=0x7f33780d0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:21.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.789+0000 7f3377fff640 1 -- 172.21.15.98:0/583537191 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33780b6740 con 0x7f337815c220 2024-09-19T08:42:21.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.793+0000 7f3377fff640 1 --2- 172.21.15.98:0/583537191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f337815c220 0x7f33780733f0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f33680097c0 tx=0x7f336801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a034874932e590 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:21.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.793+0000 7f33777fe640 1 -- 172.21.15.98:0/583537191 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33680ab650 con 0x7f337815c220 2024-09-19T08:42:21.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.793+0000 7f33777fe640 1 -- 172.21.15.98:0/583537191 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33680b0070 con 0x7f337815c220 2024-09-19T08:42:21.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.793+0000 7f33777fe640 1 -- 172.21.15.98:0/583537191 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33680aba10 con 0x7f337815c220 2024-09-19T08:42:21.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.793+0000 7f337e700640 1 -- 172.21.15.98:0/583537191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f337815c220 msgr2=0x7f33780733f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:21.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.793+0000 7f337e700640 1 --2- 172.21.15.98:0/583537191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f337815c220 0x7f33780733f0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f33680097c0 tx=0x7f336801c430 comp rx=0 tx=0).stop 2024-09-19T08:42:21.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.793+0000 7f337e700640 1 -- 172.21.15.98:0/583537191 shutdown_connections 2024-09-19T08:42:21.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.793+0000 7f337e700640 1 --2- 172.21.15.98:0/583537191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f337815c220 0x7f33780733f0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:21.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.793+0000 7f337e700640 1 -- 172.21.15.98:0/583537191 >> 172.21.15.98:0/583537191 conn(0x7f33780b40a0 msgr2=0x7f33780b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:21.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.793+0000 7f337e700640 1 -- 172.21.15.98:0/583537191 shutdown_connections 2024-09-19T08:42:21.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.793+0000 7f337e700640 1 -- 172.21.15.98:0/583537191 wait complete. 2024-09-19T08:42:21.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.793+0000 7f337e700640 1 Processor -- start 2024-09-19T08:42:21.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.793+0000 7f337e700640 1 -- start start 2024-09-19T08:42:21.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.797+0000 7f337e700640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3378073930 0x7f337814efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:21.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.797+0000 7f337e700640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33780d1700 0x7f33780ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:21.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.797+0000 7f337e700640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f337815c220 0x7f337815e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:21.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.797+0000 7f337e700640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3378169eb0 con 0x7f33780d1700 2024-09-19T08:42:21.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.797+0000 7f337e700640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f337815d0a0 con 0x7f3378073930 2024-09-19T08:42:21.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.797+0000 7f337e700640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f337816a030 con 0x7f337815c220 2024-09-19T08:42:21.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.797+0000 7f336f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33780d1700 0x7f33780ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:21.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.797+0000 7f336f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33780d1700 0x7f33780ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54396/0 (socket says 172.21.15.98:54396) 2024-09-19T08:42:21.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.797+0000 7f336f7fe640 1 -- 172.21.15.98:0/945565051 learned_addr learned my addr 172.21.15.98:0/945565051 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:21.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.797+0000 7f3377fff640 1 --2- 172.21.15.98:0/945565051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3378073930 0x7f337814efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:21.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.797+0000 7f337cc9d640 1 --2- 172.21.15.98:0/945565051 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f337815c220 0x7f337815e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:21.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.797+0000 7f336f7fe640 1 -- 172.21.15.98:0/945565051 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f337815c220 msgr2=0x7f337815e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:21.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.797+0000 7f336f7fe640 1 --2- 172.21.15.98:0/945565051 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f337815c220 0x7f337815e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:21.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.797+0000 7f336f7fe640 1 -- 172.21.15.98:0/945565051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3378073930 msgr2=0x7f337814efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:21.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.797+0000 7f336f7fe640 1 --2- 172.21.15.98:0/945565051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3378073930 0x7f337814efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:21.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.797+0000 7f336f7fe640 1 -- 172.21.15.98:0/945565051 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33780af1b0 con 0x7f33780d1700 2024-09-19T08:42:21.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.797+0000 7f3377fff640 1 --2- 172.21.15.98:0/945565051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3378073930 0x7f337814efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:42:21.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.797+0000 7f337cc9d640 1 --2- 172.21.15.98:0/945565051 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f337815c220 0x7f337815e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:42:21.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.797+0000 7f336f7fe640 1 --2- 172.21.15.98:0/945565051 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33780d1700 0x7f33780ae8b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f336000c450 tx=0x7f336000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:21.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.797+0000 7f33757fa640 1 -- 172.21.15.98:0/945565051 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360017070 con 0x7f33780d1700 2024-09-19T08:42:21.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.797+0000 7f33757fa640 1 -- 172.21.15.98:0/945565051 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3360005cf0 con 0x7f33780d1700 2024-09-19T08:42:21.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.797+0000 7f337e700640 1 -- 172.21.15.98:0/945565051 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33780af490 con 0x7f33780d1700 2024-09-19T08:42:21.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.797+0000 7f33757fa640 1 -- 172.21.15.98:0/945565051 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360007290 con 0x7f33780d1700 2024-09-19T08:42:21.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.797+0000 7f337e700640 1 -- 172.21.15.98:0/945565051 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33780afa50 con 0x7f33780d1700 2024-09-19T08:42:21.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.797+0000 7f337e700640 1 -- 172.21.15.98:0/945565051 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33780733f0 con 0x7f33780d1700 2024-09-19T08:42:21.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.797+0000 7f33757fa640 1 -- 172.21.15.98:0/945565051 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f336001a070 con 0x7f33780d1700 2024-09-19T08:42:21.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.797+0000 7f33757fa640 1 --2- 172.21.15.98:0/945565051 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3358067c50 0x7f335806a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:21.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.797+0000 7f3377fff640 1 --2- 172.21.15.98:0/945565051 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3358067c50 0x7f335806a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:21.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.797+0000 7f33757fa640 1 -- 172.21.15.98:0/945565051 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 4943+0+0 (secure 0 0 0) 0x7f3360085250 con 0x7f33780d1700 2024-09-19T08:42:21.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.801+0000 7f3377fff640 1 --2- 172.21.15.98:0/945565051 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3358067c50 0x7f335806a110 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f33680af000 tx=0x7f3368008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:21.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:21.805+0000 7f33757fa640 1 -- 172.21.15.98:0/945565051 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3360055c80 con 0x7f33780d1700 2024-09-19T08:42:22.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.029+0000 7f337e700640 1 -- 172.21.15.98:0/945565051 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7f3378069030 con 0x7f33780d1700 2024-09-19T08:42:22.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.309+0000 7f33757fa640 1 -- 172.21.15.98:0/945565051 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]}]=0 v210) ==== 161+0+0 (secure 0 0 0) 0x7f3360055580 con 0x7f33780d1700 2024-09-19T08:42:22.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.313+0000 7f337e700640 1 -- 172.21.15.98:0/945565051 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3358067c50 msgr2=0x7f335806a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:22.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.313+0000 7f337e700640 1 --2- 172.21.15.98:0/945565051 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3358067c50 0x7f335806a110 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f33680af000 tx=0x7f3368008040 comp rx=0 tx=0).stop 2024-09-19T08:42:22.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.313+0000 7f337e700640 1 -- 172.21.15.98:0/945565051 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33780d1700 msgr2=0x7f33780ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:22.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.313+0000 7f337e700640 1 --2- 172.21.15.98:0/945565051 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33780d1700 0x7f33780ae8b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f336000c450 tx=0x7f336000c910 comp rx=0 tx=0).stop 2024-09-19T08:42:22.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.313+0000 7f337e700640 1 -- 172.21.15.98:0/945565051 shutdown_connections 2024-09-19T08:42:22.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.313+0000 7f337e700640 1 --2- 172.21.15.98:0/945565051 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3358067c50 0x7f335806a110 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:22.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.313+0000 7f337e700640 1 --2- 172.21.15.98:0/945565051 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f337815c220 0x7f337815e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:22.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.313+0000 7f337e700640 1 --2- 172.21.15.98:0/945565051 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33780d1700 0x7f33780ae8b0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:22.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.313+0000 7f337e700640 1 --2- 172.21.15.98:0/945565051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3378073930 0x7f337814efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:22.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.313+0000 7f337e700640 1 -- 172.21.15.98:0/945565051 >> 172.21.15.98:0/945565051 conn(0x7f33780b40a0 msgr2=0x7f3378159cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:22.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.313+0000 7f337e700640 1 -- 172.21.15.98:0/945565051 shutdown_connections 2024-09-19T08:42:22.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.313+0000 7f337e700640 1 -- 172.21.15.98:0/945565051 wait complete. 2024-09-19T08:42:22.333 INFO:tasks.cephfs_test_runner:Validate handcrafted .meta file on legacy subvol root doesn't break the system ... ok 2024-09-19T08:42:22.334 INFO:tasks.cephfs_test_runner:Starting test: test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc) 2024-09-19T08:42:22.334 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:42:22.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.457+0000 7fe6b3be6640 1 Processor -- start 2024-09-19T08:42:22.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.457+0000 7fe6b3be6640 1 -- start start 2024-09-19T08:42:22.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.457+0000 7fe6b3be6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe6ac06bc90 0x7fe6ac06c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:22.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.457+0000 7fe6b3be6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6ac0ce840 0x7fe6ac0cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:22.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.457+0000 7fe6b3be6640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6ac0ccc60 con 0x7fe6ac158fd0 2024-09-19T08:42:22.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.457+0000 7fe6b3be6640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe6ac1593b0 con 0x7fe6ac06bc90 2024-09-19T08:42:22.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.457+0000 7fe6b3be6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe6ac0ccf40 con 0x7fe6ac0ce840 2024-09-19T08:42:22.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.457+0000 7fe6b1982640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fe6ac158fd0 0x7fe6ac0b3e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:59006/0 (socket says 172.21.15.98:59006) 2024-09-19T08:42:22.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.457+0000 7fe6b1982640 1 -- 172.21.15.98:0/961901906 learned_addr learned my addr 172.21.15.98:0/961901906 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:22.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.457+0000 7fe6b2183640 1 --2- 172.21.15.98:0/961901906 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6ac0ce840 0x7fe6ac0cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:22.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.457+0000 7fe6b1181640 1 --2- 172.21.15.98:0/961901906 >> v2:172.21.15.139:3300/0 conn(0x7fe6ac06bc90 0x7fe6ac06c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:22.460 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.457+0000 7fe6b0980640 1 -- 172.21.15.98:0/961901906 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1461478470 0 0) 0x7fe6ac0ccc60 con 0x7fe6ac158fd0 2024-09-19T08:42:22.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.457+0000 7fe6b0980640 1 -- 172.21.15.98:0/961901906 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe698003530 con 0x7fe6ac158fd0 2024-09-19T08:42:22.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.457+0000 7fe6b0980640 1 -- 172.21.15.98:0/961901906 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2437543735 0 0) 0x7fe698003530 con 0x7fe6ac158fd0 2024-09-19T08:42:22.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.457+0000 7fe6b0980640 1 -- 172.21.15.98:0/961901906 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6ac0ce840 msgr2=0x7fe6ac0cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:22.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.457+0000 7fe6b0980640 1 --2- 172.21.15.98:0/961901906 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6ac0ce840 0x7fe6ac0cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:22.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.457+0000 7fe6b0980640 1 -- 172.21.15.98:0/961901906 >> v2:172.21.15.139:3300/0 conn(0x7fe6ac06bc90 msgr2=0x7fe6ac06c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:22.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.457+0000 7fe6b0980640 1 --2- 172.21.15.98:0/961901906 >> v2:172.21.15.139:3300/0 conn(0x7fe6ac06bc90 0x7fe6ac06c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:22.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.457+0000 7fe6b0980640 1 -- 172.21.15.98:0/961901906 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6ac0cd220 con 0x7fe6ac158fd0 2024-09-19T08:42:22.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.457+0000 7fe6b0980640 1 -- 172.21.15.98:0/961901906 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fe6a8001680 con 0x7fe6ac158fd0 2024-09-19T08:42:22.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.457+0000 7fe6b0980640 1 -- 172.21.15.98:0/961901906 >> v1:172.21.15.98:6789/0 conn(0x7fe6ac158fd0 legacy=0x7fe6ac0b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:22.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.457+0000 7fe6b0980640 1 --2- 172.21.15.98:0/961901906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6980039c0 0x7fe698003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:22.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.457+0000 7fe6b0980640 1 --2- 172.21.15.98:0/961901906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe698004810 0x7fe698004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:22.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.457+0000 7fe6b0980640 1 --2- 172.21.15.98:0/961901906 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6ac0ce840 0x7fe6980051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:22.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.457+0000 7fe6b0980640 1 -- 172.21.15.98:0/961901906 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ac1593b0 con 0x7fe698004810 2024-09-19T08:42:22.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.457+0000 7fe6b0980640 1 -- 172.21.15.98:0/961901906 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ac0ccf40 con 0x7fe6980039c0 2024-09-19T08:42:22.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.457+0000 7fe6b0980640 1 -- 172.21.15.98:0/961901906 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe69800ba00 con 0x7fe6ac0ce840 2024-09-19T08:42:22.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.457+0000 7fe6b1181640 1 --2- 172.21.15.98:0/961901906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe698004810 0x7fe698004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:22.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.457+0000 7fe6b2183640 1 --2- 172.21.15.98:0/961901906 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6ac0ce840 0x7fe6980051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:22.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.457+0000 7fe6b1982640 1 --2- 172.21.15.98:0/961901906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6980039c0 0x7fe698003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:22.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.457+0000 7fe6b1181640 1 -- 172.21.15.98:0/961901906 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6ac0ce840 msgr2=0x7fe6980051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:22.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.457+0000 7fe6b1181640 1 --2- 172.21.15.98:0/961901906 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6ac0ce840 0x7fe6980051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:22.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.457+0000 7fe6b1181640 1 -- 172.21.15.98:0/961901906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6980039c0 msgr2=0x7fe698003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:22.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.457+0000 7fe6b1181640 1 --2- 172.21.15.98:0/961901906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6980039c0 0x7fe698003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:22.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.457+0000 7fe6b1181640 1 -- 172.21.15.98:0/961901906 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe698003530 con 0x7fe698004810 2024-09-19T08:42:22.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.457+0000 7fe6b1982640 1 --2- 172.21.15.98:0/961901906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6980039c0 0x7fe698003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:42:22.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.461+0000 7fe6b1181640 1 --2- 172.21.15.98:0/961901906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe698004810 0x7fe698004c90 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0009670 tx=0x7fe6a001ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f7c092d928d933f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:22.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.461+0000 7fe6b0980640 1 -- 172.21.15.98:0/961901906 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a00b4070 con 0x7fe698004810 2024-09-19T08:42:22.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.461+0000 7fe6b0980640 1 -- 172.21.15.98:0/961901906 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6a00a76d0 con 0x7fe698004810 2024-09-19T08:42:22.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.461+0000 7fe6b0980640 1 -- 172.21.15.98:0/961901906 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a00096b0 con 0x7fe698004810 2024-09-19T08:42:22.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.461+0000 7fe6b3be6640 1 -- 172.21.15.98:0/961901906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe698004810 msgr2=0x7fe698004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:22.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.461+0000 7fe6b3be6640 1 --2- 172.21.15.98:0/961901906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe698004810 0x7fe698004c90 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0009670 tx=0x7fe6a001ec40 comp rx=0 tx=0).stop 2024-09-19T08:42:22.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.461+0000 7fe6b2183640 1 -- 172.21.15.98:0/961901906 reap_dead start 2024-09-19T08:42:22.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.461+0000 7fe6b3be6640 1 -- 172.21.15.98:0/961901906 shutdown_connections 2024-09-19T08:42:22.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.461+0000 7fe6b3be6640 1 -- 172.21.15.98:0/961901906 >> 172.21.15.98:0/961901906 conn(0x7fe6ac0b2010 msgr2=0x7fe6ac0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:22.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.461+0000 7fe6b3be6640 1 -- 172.21.15.98:0/961901906 shutdown_connections 2024-09-19T08:42:22.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.461+0000 7fe6b3be6640 1 -- 172.21.15.98:0/961901906 wait complete. 2024-09-19T08:42:22.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.461+0000 7fe6b3be6640 1 Processor -- start 2024-09-19T08:42:22.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.461+0000 7fe6b3be6640 1 -- start start 2024-09-19T08:42:22.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.461+0000 7fe6b3be6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe698004810 0x7fe6ac157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:22.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.461+0000 7fe6b3be6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6ac15aae0 0x7fe6ac072130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:22.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.461+0000 7fe6b3be6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6ac14d3d0 0x7fe6ac152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:22.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.461+0000 7fe6b3be6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ac15c920 con 0x7fe6ac15aae0 2024-09-19T08:42:22.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.461+0000 7fe6b3be6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ac15c7a0 con 0x7fe698004810 2024-09-19T08:42:22.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.461+0000 7fe6b3be6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe6ac0cf460 con 0x7fe6ac14d3d0 2024-09-19T08:42:22.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.461+0000 7fe6b1181640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6ac15aae0 0x7fe6ac072130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:22.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.461+0000 7fe6b1181640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6ac15aae0 0x7fe6ac072130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54422/0 (socket says 172.21.15.98:54422) 2024-09-19T08:42:22.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.461+0000 7fe6b1181640 1 -- 172.21.15.98:0/197430891 learned_addr learned my addr 172.21.15.98:0/197430891 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:22.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.465+0000 7fe6b2183640 1 --2- 172.21.15.98:0/197430891 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6ac14d3d0 0x7fe6ac152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:22.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.465+0000 7fe6b1982640 1 --2- 172.21.15.98:0/197430891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe698004810 0x7fe6ac157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:22.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.465+0000 7fe6b1181640 1 -- 172.21.15.98:0/197430891 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6ac14d3d0 msgr2=0x7fe6ac152c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:22.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.465+0000 7fe6b1181640 1 --2- 172.21.15.98:0/197430891 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6ac14d3d0 0x7fe6ac152c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:22.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.465+0000 7fe6b1181640 1 -- 172.21.15.98:0/197430891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe698004810 msgr2=0x7fe6ac157ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:22.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.465+0000 7fe6b1181640 1 --2- 172.21.15.98:0/197430891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe698004810 0x7fe6ac157ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:22.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.465+0000 7fe6b1181640 1 -- 172.21.15.98:0/197430891 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6ac14d860 con 0x7fe6ac15aae0 2024-09-19T08:42:22.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.465+0000 7fe6b1181640 1 --2- 172.21.15.98:0/197430891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6ac15aae0 0x7fe6ac072130 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fe6a00b2040 tx=0x7fe698005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:22.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.465+0000 7fe69e7fc640 1 -- 172.21.15.98:0/197430891 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a00be030 con 0x7fe6ac15aae0 2024-09-19T08:42:22.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.465+0000 7fe69e7fc640 1 -- 172.21.15.98:0/197430891 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6a000f000 con 0x7fe6ac15aae0 2024-09-19T08:42:22.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.465+0000 7fe69e7fc640 1 -- 172.21.15.98:0/197430891 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a00b4040 con 0x7fe6ac15aae0 2024-09-19T08:42:22.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.465+0000 7fe6b3be6640 1 -- 172.21.15.98:0/197430891 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6ac0b1680 con 0x7fe6ac15aae0 2024-09-19T08:42:22.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.465+0000 7fe6b3be6640 1 -- 172.21.15.98:0/197430891 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6ac14dcc0 con 0x7fe6ac15aae0 2024-09-19T08:42:22.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.465+0000 7fe6b3be6640 1 -- 172.21.15.98:0/197430891 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6ac0d0940 con 0x7fe6ac15aae0 2024-09-19T08:42:22.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.465+0000 7fe69e7fc640 1 -- 172.21.15.98:0/197430891 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fe6a00c1070 con 0x7fe6ac15aae0 2024-09-19T08:42:22.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.465+0000 7fe69e7fc640 1 --2- 172.21.15.98:0/197430891 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe690067c50 0x7fe69006a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:22.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.465+0000 7fe6b1982640 1 --2- 172.21.15.98:0/197430891 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe690067c50 0x7fe69006a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:22.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.465+0000 7fe69e7fc640 1 -- 172.21.15.98:0/197430891 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 4943+0+0 (secure 0 0 0) 0x7fe6a0121bf0 con 0x7fe6ac15aae0 2024-09-19T08:42:22.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.465+0000 7fe6b1982640 1 --2- 172.21.15.98:0/197430891 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe690067c50 0x7fe69006a110 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8000f70 tx=0x7fe6a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:22.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.469+0000 7fe69e7fc640 1 -- 172.21.15.98:0/197430891 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe6a00c9020 con 0x7fe6ac15aae0 2024-09-19T08:42:22.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.633+0000 7fe6b3be6640 1 -- 172.21.15.98:0/197430891 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6ac1553a0 con 0x7fe6ac15aae0 2024-09-19T08:42:22.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.637+0000 7fe69e7fc640 1 -- 172.21.15.98:0/197430891 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+13032 (secure 0 0 0) 0x7fe6a00189f0 con 0x7fe6ac15aae0 2024-09-19T08:42:22.637 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:42:22.637 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":60,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:42:21.299517+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.98:6837/3548234111":"2024-09-20T08:42:17.977264+0000","172.21.15.98:6836/3548234111":"2024-09-20T08:42:17.977264+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:42:22.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.637+0000 7fe6b3be6640 1 -- 172.21.15.98:0/197430891 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe690067c50 msgr2=0x7fe69006a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:22.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.637+0000 7fe6b3be6640 1 --2- 172.21.15.98:0/197430891 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe690067c50 0x7fe69006a110 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8000f70 tx=0x7fe6a8009040 comp rx=0 tx=0).stop 2024-09-19T08:42:22.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.637+0000 7fe6b3be6640 1 -- 172.21.15.98:0/197430891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6ac15aae0 msgr2=0x7fe6ac072130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:22.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.637+0000 7fe6b3be6640 1 --2- 172.21.15.98:0/197430891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6ac15aae0 0x7fe6ac072130 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fe6a00b2040 tx=0x7fe698005840 comp rx=0 tx=0).stop 2024-09-19T08:42:22.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.637+0000 7fe6b3be6640 1 -- 172.21.15.98:0/197430891 shutdown_connections 2024-09-19T08:42:22.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.637+0000 7fe6b3be6640 1 --2- 172.21.15.98:0/197430891 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe690067c50 0x7fe69006a110 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:22.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.637+0000 7fe6b3be6640 1 --2- 172.21.15.98:0/197430891 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6ac14d3d0 0x7fe6ac152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:22.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.637+0000 7fe6b3be6640 1 --2- 172.21.15.98:0/197430891 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6ac15aae0 0x7fe6ac072130 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:22.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.637+0000 7fe6b3be6640 1 --2- 172.21.15.98:0/197430891 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe698004810 0x7fe6ac157ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:22.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.637+0000 7fe6b3be6640 1 -- 172.21.15.98:0/197430891 >> 172.21.15.98:0/197430891 conn(0x7fe6ac0b2010 msgr2=0x7fe6ac155ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:22.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.637+0000 7fe6b3be6640 1 -- 172.21.15.98:0/197430891 shutdown_connections 2024-09-19T08:42:22.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.637+0000 7fe6b3be6640 1 -- 172.21.15.98:0/197430891 wait complete. 2024-09-19T08:42:22.657 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T08:38:08.105684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T08:42:22.657 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T08:42:22.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.781+0000 7f0350e61640 1 Processor -- start 2024-09-19T08:42:22.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.781+0000 7f0350e61640 1 -- start start 2024-09-19T08:42:22.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.781+0000 7f0350e61640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f034c144d10 0x7f034c0722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:22.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.781+0000 7f0350e61640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f034c072820 0x7f034c06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:22.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.781+0000 7f0350e61640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f034c06c1d0 0x7f034c0b4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:22.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.781+0000 7f0350e61640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f034c072c60 con 0x7f034c06c1d0 2024-09-19T08:42:22.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.781+0000 7f0350e61640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f034c073ab0 con 0x7f034c072820 2024-09-19T08:42:22.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.781+0000 7f0350e61640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f034c1450f0 con 0x7f034c144d10 2024-09-19T08:42:22.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.781+0000 7f034ad9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f034c06c1d0 0x7f034c0b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:22.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.781+0000 7f034ad9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f034c06c1d0 0x7f034c0b4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54430/0 (socket says 172.21.15.98:54430) 2024-09-19T08:42:22.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.781+0000 7f034ad9d640 1 -- 172.21.15.98:0/362831526 learned_addr learned my addr 172.21.15.98:0/362831526 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:22.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.781+0000 7f034a59c640 1 --2- 172.21.15.98:0/362831526 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f034c144d10 0x7f034c0722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:22.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.781+0000 7f0349d9b640 1 --2- 172.21.15.98:0/362831526 >> v2:172.21.15.139:3300/0 conn(0x7f034c072820 0x7f034c06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:22.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.781+0000 7f034ad9d640 1 -- 172.21.15.98:0/362831526 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f034c144d10 msgr2=0x7f034c0722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:22.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.781+0000 7f034ad9d640 1 --2- 172.21.15.98:0/362831526 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f034c144d10 0x7f034c0722e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:22.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.781+0000 7f034ad9d640 1 -- 172.21.15.98:0/362831526 >> v2:172.21.15.139:3300/0 conn(0x7f034c072820 msgr2=0x7f034c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:22.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.781+0000 7f034ad9d640 1 --2- 172.21.15.98:0/362831526 >> v2:172.21.15.139:3300/0 conn(0x7f034c072820 0x7f034c06bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:22.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.781+0000 7f034ad9d640 1 -- 172.21.15.98:0/362831526 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f034c072ea0 con 0x7f034c06c1d0 2024-09-19T08:42:22.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.781+0000 7f034ad9d640 1 --2- 172.21.15.98:0/362831526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f034c06c1d0 0x7f034c0b4520 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f034c06f100 tx=0x7f034401e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc87ba5133a8314 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:22.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.785+0000 7f034959a640 1 -- 172.21.15.98:0/362831526 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034400b2d0 con 0x7f034c06c1d0 2024-09-19T08:42:22.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.785+0000 7f034959a640 1 -- 172.21.15.98:0/362831526 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f034400b470 con 0x7f034c06c1d0 2024-09-19T08:42:22.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.785+0000 7f034959a640 1 -- 172.21.15.98:0/362831526 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03440a78d0 con 0x7f034c06c1d0 2024-09-19T08:42:22.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.785+0000 7f0350e61640 1 -- 172.21.15.98:0/362831526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f034c06c1d0 msgr2=0x7f034c0b4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:22.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.785+0000 7f0350e61640 1 --2- 172.21.15.98:0/362831526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f034c06c1d0 0x7f034c0b4520 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f034c06f100 tx=0x7f034401e930 comp rx=0 tx=0).stop 2024-09-19T08:42:22.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.785+0000 7f0350e61640 1 -- 172.21.15.98:0/362831526 shutdown_connections 2024-09-19T08:42:22.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.785+0000 7f0350e61640 1 --2- 172.21.15.98:0/362831526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f034c06c1d0 0x7f034c0b4520 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:22.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.785+0000 7f0350e61640 1 --2- 172.21.15.98:0/362831526 >> v2:172.21.15.139:3300/0 conn(0x7f034c072820 0x7f034c06bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:22.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.785+0000 7f0350e61640 1 --2- 172.21.15.98:0/362831526 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f034c144d10 0x7f034c0722e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:22.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.785+0000 7f0350e61640 1 -- 172.21.15.98:0/362831526 >> 172.21.15.98:0/362831526 conn(0x7f034c0b2010 msgr2=0x7f034c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:22.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.785+0000 7f0350e61640 1 -- 172.21.15.98:0/362831526 shutdown_connections 2024-09-19T08:42:22.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.785+0000 7f0350e61640 1 -- 172.21.15.98:0/362831526 wait complete. 2024-09-19T08:42:22.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.785+0000 7f0350e61640 1 Processor -- start 2024-09-19T08:42:22.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.785+0000 7f0350e61640 1 -- start start 2024-09-19T08:42:22.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.785+0000 7f0350e61640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f034c06c1d0 0x7f034c15a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:22.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.785+0000 7f0350e61640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f034c072820 0x7f034c15b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:22.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.785+0000 7f0350e61640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f034c144d10 0x7f034c13d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:22.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.785+0000 7f0350e61640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f034c0afa30 con 0x7f034c072820 2024-09-19T08:42:22.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.785+0000 7f0350e61640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f034c0af8b0 con 0x7f034c144d10 2024-09-19T08:42:22.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.785+0000 7f0350e61640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f034c0b3410 con 0x7f034c06c1d0 2024-09-19T08:42:22.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.785+0000 7f0349d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f034c072820 0x7f034c15b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:22.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.785+0000 7f0349d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f034c072820 0x7f034c15b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54434/0 (socket says 172.21.15.98:54434) 2024-09-19T08:42:22.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.785+0000 7f0349d9b640 1 -- 172.21.15.98:0/751920482 learned_addr learned my addr 172.21.15.98:0/751920482 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:22.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.785+0000 7f034a59c640 1 --2- 172.21.15.98:0/751920482 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f034c06c1d0 0x7f034c15a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:22.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.785+0000 7f0349d9b640 1 -- 172.21.15.98:0/751920482 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f034c06c1d0 msgr2=0x7f034c15a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:22.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.785+0000 7f034ad9d640 1 --2- 172.21.15.98:0/751920482 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f034c144d10 0x7f034c13d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:22.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.785+0000 7f0349d9b640 1 --2- 172.21.15.98:0/751920482 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f034c06c1d0 0x7f034c15a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:22.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.785+0000 7f0349d9b640 1 -- 172.21.15.98:0/751920482 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f034c144d10 msgr2=0x7f034c13d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:22.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.785+0000 7f0349d9b640 1 --2- 172.21.15.98:0/751920482 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f034c144d10 0x7f034c13d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:22.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.785+0000 7f0349d9b640 1 -- 172.21.15.98:0/751920482 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f034c134150 con 0x7f034c072820 2024-09-19T08:42:22.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.785+0000 7f0349d9b640 1 --2- 172.21.15.98:0/751920482 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f034c072820 0x7f034c15b840 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f034000d7d0 tx=0x7f034000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:22.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.785+0000 7f033affd640 1 -- 172.21.15.98:0/751920482 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340004020 con 0x7f034c072820 2024-09-19T08:42:22.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.785+0000 7f033affd640 1 -- 172.21.15.98:0/751920482 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03400041c0 con 0x7f034c072820 2024-09-19T08:42:22.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.785+0000 7f0350e61640 1 -- 172.21.15.98:0/751920482 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f034c15e540 con 0x7f034c072820 2024-09-19T08:42:22.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.785+0000 7f033affd640 1 -- 172.21.15.98:0/751920482 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03400043b0 con 0x7f034c072820 2024-09-19T08:42:22.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.785+0000 7f0350e61640 1 -- 172.21.15.98:0/751920482 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f034c136630 con 0x7f034c072820 2024-09-19T08:42:22.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.789+0000 7f0350e61640 1 -- 172.21.15.98:0/751920482 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f034c069b80 con 0x7f034c072820 2024-09-19T08:42:22.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.789+0000 7f033affd640 1 -- 172.21.15.98:0/751920482 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f034001b020 con 0x7f034c072820 2024-09-19T08:42:22.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.789+0000 7f033affd640 1 --2- 172.21.15.98:0/751920482 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f032c067c50 0x7f032c06a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:22.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.789+0000 7f033affd640 1 -- 172.21.15.98:0/751920482 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 4943+0+0 (secure 0 0 0) 0x7f0340085cc0 con 0x7f034c072820 2024-09-19T08:42:22.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.789+0000 7f034a59c640 1 --2- 172.21.15.98:0/751920482 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f032c067c50 0x7f032c06a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:22.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.789+0000 7f034a59c640 1 --2- 172.21.15.98:0/751920482 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f032c067c50 0x7f032c06a110 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f034c13a750 tx=0x7f033c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:22.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.793+0000 7f033affd640 1 -- 172.21.15.98:0/751920482 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f03400566f0 con 0x7f034c072820 2024-09-19T08:42:22.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.945+0000 7f0350e61640 1 -- 172.21.15.98:0/751920482 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f034c0b3f00 con 0x7f034c072820 2024-09-19T08:42:22.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.945+0000 7f033affd640 1 -- 172.21.15.98:0/751920482 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) ==== 93+0+10 (secure 0 0 0) 0x7f0340055ff0 con 0x7f034c072820 2024-09-19T08:42:22.946 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2024-09-19T08:42:22.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.949+0000 7f0350e61640 1 -- 172.21.15.98:0/751920482 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f032c067c50 msgr2=0x7f032c06a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:22.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.949+0000 7f0350e61640 1 --2- 172.21.15.98:0/751920482 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f032c067c50 0x7f032c06a110 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f034c13a750 tx=0x7f033c008040 comp rx=0 tx=0).stop 2024-09-19T08:42:22.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.949+0000 7f0350e61640 1 -- 172.21.15.98:0/751920482 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f034c072820 msgr2=0x7f034c15b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:22.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.949+0000 7f0350e61640 1 --2- 172.21.15.98:0/751920482 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f034c072820 0x7f034c15b840 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f034000d7d0 tx=0x7f034000dc90 comp rx=0 tx=0).stop 2024-09-19T08:42:22.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.949+0000 7f0350e61640 1 -- 172.21.15.98:0/751920482 shutdown_connections 2024-09-19T08:42:22.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.949+0000 7f0350e61640 1 --2- 172.21.15.98:0/751920482 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f032c067c50 0x7f032c06a110 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:22.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.949+0000 7f0350e61640 1 --2- 172.21.15.98:0/751920482 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f034c144d10 0x7f034c13d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:22.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.949+0000 7f0350e61640 1 --2- 172.21.15.98:0/751920482 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f034c072820 0x7f034c15b840 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:22.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.949+0000 7f0350e61640 1 --2- 172.21.15.98:0/751920482 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f034c06c1d0 0x7f034c15a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:22.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.949+0000 7f0350e61640 1 -- 172.21.15.98:0/751920482 >> 172.21.15.98:0/751920482 conn(0x7f034c0b2010 msgr2=0x7f034c0b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:22.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.949+0000 7f0350e61640 1 -- 172.21.15.98:0/751920482 shutdown_connections 2024-09-19T08:42:22.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:22.949+0000 7f0350e61640 1 -- 172.21.15.98:0/751920482 wait complete. 2024-09-19T08:42:22.964 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction' 2024-09-19T08:42:23.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.085+0000 7ff5eb893640 1 Processor -- start 2024-09-19T08:42:23.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.085+0000 7ff5eb893640 1 -- start start 2024-09-19T08:42:23.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.085+0000 7ff5eb893640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff5e40733b0 0x7ff5e40cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:23.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.085+0000 7ff5eb893640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5e406d310 con 0x7ff5e406c3a0 2024-09-19T08:42:23.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.085+0000 7ff5eb893640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5e406d5f0 con 0x7ff5e415c550 2024-09-19T08:42:23.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.085+0000 7ff5eb893640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff5e406d8d0 con 0x7ff5e40733b0 2024-09-19T08:42:23.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.085+0000 7ff5e8e2e640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7ff5e406c3a0 0x7ff5e40cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:59018/0 (socket says 172.21.15.98:59018) 2024-09-19T08:42:23.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.085+0000 7ff5e8e2e640 1 -- 172.21.15.98:0/2824670263 learned_addr learned my addr 172.21.15.98:0/2824670263 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:23.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.085+0000 7ff5e9e30640 1 --2- 172.21.15.98:0/2824670263 >> v2:172.21.15.139:3300/0 conn(0x7ff5e40733b0 0x7ff5e40cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:23.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.085+0000 7ff5dbfff640 1 -- 172.21.15.98:0/2824670263 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 421718676 0 0) 0x7ff5e406d310 con 0x7ff5e406c3a0 2024-09-19T08:42:23.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.085+0000 7ff5dbfff640 1 -- 172.21.15.98:0/2824670263 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5d0003530 con 0x7ff5e406c3a0 2024-09-19T08:42:23.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.085+0000 7ff5dbfff640 1 -- 172.21.15.98:0/2824670263 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 41980740 0 0) 0x7ff5d0003530 con 0x7ff5e406c3a0 2024-09-19T08:42:23.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.085+0000 7ff5dbfff640 1 -- 172.21.15.98:0/2824670263 >> v2:172.21.15.139:3300/0 conn(0x7ff5e40733b0 msgr2=0x7ff5e40cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:23.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.085+0000 7ff5dbfff640 1 --2- 172.21.15.98:0/2824670263 >> v2:172.21.15.139:3300/0 conn(0x7ff5e40733b0 0x7ff5e40cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:23.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.085+0000 7ff5dbfff640 1 -- 172.21.15.98:0/2824670263 >> v1:172.21.15.139:6789/0 conn(0x7ff5e415c550 legacy=0x7ff5e406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:23.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.085+0000 7ff5dbfff640 1 -- 172.21.15.98:0/2824670263 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e406dbb0 con 0x7ff5e406c3a0 2024-09-19T08:42:23.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.085+0000 7ff5dbfff640 1 -- 172.21.15.98:0/2824670263 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7ff5d4001680 con 0x7ff5e406c3a0 2024-09-19T08:42:23.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.085+0000 7ff5dbfff640 1 -- 172.21.15.98:0/2824670263 >> v1:172.21.15.98:6789/0 conn(0x7ff5e406c3a0 legacy=0x7ff5e40cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:23.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.085+0000 7ff5dbfff640 1 --2- 172.21.15.98:0/2824670263 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5d00039c0 0x7ff5d0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:23.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.085+0000 7ff5dbfff640 1 --2- 172.21.15.98:0/2824670263 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5d0004810 0x7ff5d0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:23.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.085+0000 7ff5dbfff640 1 --2- 172.21.15.98:0/2824670263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5d00051d0 0x7ff5d000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:23.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.085+0000 7ff5dbfff640 1 -- 172.21.15.98:0/2824670263 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e406d8d0 con 0x7ff5d00051d0 2024-09-19T08:42:23.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.085+0000 7ff5dbfff640 1 -- 172.21.15.98:0/2824670263 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d000c070 con 0x7ff5d0004810 2024-09-19T08:42:23.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.085+0000 7ff5dbfff640 1 -- 172.21.15.98:0/2824670263 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff5d000c270 con 0x7ff5d00039c0 2024-09-19T08:42:23.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.085+0000 7ff5e9e30640 1 --2- 172.21.15.98:0/2824670263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5d00051d0 0x7ff5d000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:23.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.085+0000 7ff5e8e2e640 1 --2- 172.21.15.98:0/2824670263 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5d0004810 0x7ff5d0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:23.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.085+0000 7ff5e962f640 1 --2- 172.21.15.98:0/2824670263 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5d00039c0 0x7ff5d0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:23.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.085+0000 7ff5e9e30640 1 -- 172.21.15.98:0/2824670263 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5d00039c0 msgr2=0x7ff5d0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:23.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.085+0000 7ff5e9e30640 1 --2- 172.21.15.98:0/2824670263 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5d00039c0 0x7ff5d0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:23.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.085+0000 7ff5e9e30640 1 -- 172.21.15.98:0/2824670263 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5d0004810 msgr2=0x7ff5d0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:23.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.085+0000 7ff5e9e30640 1 --2- 172.21.15.98:0/2824670263 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5d0004810 0x7ff5d0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:23.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.085+0000 7ff5e9e30640 1 -- 172.21.15.98:0/2824670263 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d0003530 con 0x7ff5d00051d0 2024-09-19T08:42:23.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.085+0000 7ff5e962f640 1 --2- 172.21.15.98:0/2824670263 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5d00039c0 0x7ff5d0003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:42:23.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.089+0000 7ff5e9e30640 1 -- 172.21.15.98:0/2824670263 reap_dead start 2024-09-19T08:42:23.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.089+0000 7ff5e9e30640 1 --2- 172.21.15.98:0/2824670263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5d00051d0 0x7ff5d000b9d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc009fc0 tx=0x7ff5dc01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=188d16b54b67cbcd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:23.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.089+0000 7ff5dbfff640 1 -- 172.21.15.98:0/2824670263 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc009c10 con 0x7ff5d00051d0 2024-09-19T08:42:23.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.089+0000 7ff5dbfff640 1 -- 172.21.15.98:0/2824670263 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5dc0a5d10 con 0x7ff5d00051d0 2024-09-19T08:42:23.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.089+0000 7ff5dbfff640 1 -- 172.21.15.98:0/2824670263 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5dc003be0 con 0x7ff5d00051d0 2024-09-19T08:42:23.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.089+0000 7ff5eb893640 1 -- 172.21.15.98:0/2824670263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5d00051d0 msgr2=0x7ff5d000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:23.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.089+0000 7ff5eb893640 1 --2- 172.21.15.98:0/2824670263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5d00051d0 0x7ff5d000b9d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc009fc0 tx=0x7ff5dc01ccc0 comp rx=0 tx=0).stop 2024-09-19T08:42:23.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.089+0000 7ff5eb893640 1 -- 172.21.15.98:0/2824670263 shutdown_connections 2024-09-19T08:42:23.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.089+0000 7ff5eb893640 1 --2- 172.21.15.98:0/2824670263 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5d00051d0 0x7ff5d000b9d0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:23.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.089+0000 7ff5eb893640 1 -- 172.21.15.98:0/2824670263 >> 172.21.15.98:0/2824670263 conn(0x7ff5e40b2010 msgr2=0x7ff5e40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:23.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.089+0000 7ff5eb893640 1 -- 172.21.15.98:0/2824670263 shutdown_connections 2024-09-19T08:42:23.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.089+0000 7ff5eb893640 1 -- 172.21.15.98:0/2824670263 wait complete. 2024-09-19T08:42:23.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.089+0000 7ff5eb893640 1 Processor -- start 2024-09-19T08:42:23.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.089+0000 7ff5eb893640 1 -- start start 2024-09-19T08:42:23.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.093+0000 7ff5eb893640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5d00051d0 0x7ff5e4154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:23.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.093+0000 7ff5eb893640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5e4151c90 0x7ff5e4145d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:23.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.093+0000 7ff5eb893640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5e4159ac0 0x7ff5e4159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:23.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.093+0000 7ff5eb893640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e40d0050 con 0x7ff5e4151c90 2024-09-19T08:42:23.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.093+0000 7ff5eb893640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e415cb40 con 0x7ff5d00051d0 2024-09-19T08:42:23.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.093+0000 7ff5eb893640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff5e406bc90 con 0x7ff5e4159ac0 2024-09-19T08:42:23.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.093+0000 7ff5e8e2e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5e4151c90 0x7ff5e4145d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:23.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.093+0000 7ff5e8e2e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5e4151c90 0x7ff5e4145d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54452/0 (socket says 172.21.15.98:54452) 2024-09-19T08:42:23.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.093+0000 7ff5e8e2e640 1 -- 172.21.15.98:0/1267809352 learned_addr learned my addr 172.21.15.98:0/1267809352 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:23.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.093+0000 7ff5e962f640 1 --2- 172.21.15.98:0/1267809352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5d00051d0 0x7ff5e4154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:23.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.093+0000 7ff5e9e30640 1 --2- 172.21.15.98:0/1267809352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5e4159ac0 0x7ff5e4159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:23.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.093+0000 7ff5e8e2e640 1 -- 172.21.15.98:0/1267809352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5e4159ac0 msgr2=0x7ff5e4159f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:23.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.093+0000 7ff5e8e2e640 1 --2- 172.21.15.98:0/1267809352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5e4159ac0 0x7ff5e4159f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:23.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.093+0000 7ff5e8e2e640 1 -- 172.21.15.98:0/1267809352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5d00051d0 msgr2=0x7ff5e4154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:23.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.093+0000 7ff5e8e2e640 1 --2- 172.21.15.98:0/1267809352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5d00051d0 0x7ff5e4154e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:23.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.093+0000 7ff5e8e2e640 1 -- 172.21.15.98:0/1267809352 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e4152400 con 0x7ff5e4151c90 2024-09-19T08:42:23.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.093+0000 7ff5e8e2e640 1 --2- 172.21.15.98:0/1267809352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5e4151c90 0x7ff5e4145d60 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ff5d400e580 tx=0x7ff5d400ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:23.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.093+0000 7ff5d9ffb640 1 -- 172.21.15.98:0/1267809352 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4012590 con 0x7ff5e4151c90 2024-09-19T08:42:23.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.093+0000 7ff5d9ffb640 1 -- 172.21.15.98:0/1267809352 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5d4018070 con 0x7ff5e4151c90 2024-09-19T08:42:23.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.093+0000 7ff5eb893640 1 -- 172.21.15.98:0/1267809352 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5e41526e0 con 0x7ff5e4151c90 2024-09-19T08:42:23.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.093+0000 7ff5d9ffb640 1 -- 172.21.15.98:0/1267809352 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4012990 con 0x7ff5e4151c90 2024-09-19T08:42:23.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.093+0000 7ff5eb893640 1 -- 172.21.15.98:0/1267809352 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5e4152ca0 con 0x7ff5e4151c90 2024-09-19T08:42:23.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.093+0000 7ff5eb893640 1 -- 172.21.15.98:0/1267809352 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5e415a480 con 0x7ff5e4151c90 2024-09-19T08:42:23.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.093+0000 7ff5d9ffb640 1 -- 172.21.15.98:0/1267809352 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7ff5d4012b30 con 0x7ff5e4151c90 2024-09-19T08:42:23.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.093+0000 7ff5d9ffb640 1 --2- 172.21.15.98:0/1267809352 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff5c8067c20 0x7ff5c806a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:23.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.093+0000 7ff5e962f640 1 --2- 172.21.15.98:0/1267809352 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff5c8067c20 0x7ff5c806a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:23.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.093+0000 7ff5d9ffb640 1 -- 172.21.15.98:0/1267809352 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 4943+0+0 (secure 0 0 0) 0x7ff5d4085cd0 con 0x7ff5e4151c90 2024-09-19T08:42:23.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.093+0000 7ff5e962f640 1 --2- 172.21.15.98:0/1267809352 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff5c8067c20 0x7ff5c806a0e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0007fd0 tx=0x7ff5e0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:23.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.097+0000 7ff5d9ffb640 1 -- 172.21.15.98:0/1267809352 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff5d408b020 con 0x7ff5e4151c90 2024-09-19T08:42:23.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:23.321+0000 7ff5eb893640 1 -- 172.21.15.98:0/1267809352 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]} v 0) -- 0x7ff5e406c020 con 0x7ff5e4151c90 2024-09-19T08:42:24.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.313+0000 7ff5d9ffb640 1 -- 172.21.15.98:0/1267809352 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]}]=0 v212) ==== 134+0+0 (secure 0 0 0) 0x7ff5d40560c0 con 0x7ff5e4151c90 2024-09-19T08:42:24.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.317+0000 7ff5eb893640 1 -- 172.21.15.98:0/1267809352 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff5c8067c20 msgr2=0x7ff5c806a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:24.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.317+0000 7ff5eb893640 1 --2- 172.21.15.98:0/1267809352 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff5c8067c20 0x7ff5c806a0e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0007fd0 tx=0x7ff5e0009040 comp rx=0 tx=0).stop 2024-09-19T08:42:24.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.317+0000 7ff5eb893640 1 -- 172.21.15.98:0/1267809352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5e4151c90 msgr2=0x7ff5e4145d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:24.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.317+0000 7ff5eb893640 1 --2- 172.21.15.98:0/1267809352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5e4151c90 0x7ff5e4145d60 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ff5d400e580 tx=0x7ff5d400ea40 comp rx=0 tx=0).stop 2024-09-19T08:42:24.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.317+0000 7ff5eb893640 1 -- 172.21.15.98:0/1267809352 shutdown_connections 2024-09-19T08:42:24.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.317+0000 7ff5eb893640 1 --2- 172.21.15.98:0/1267809352 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff5c8067c20 0x7ff5c806a0e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:24.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.317+0000 7ff5eb893640 1 --2- 172.21.15.98:0/1267809352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff5e4159ac0 0x7ff5e4159f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:24.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.317+0000 7ff5eb893640 1 --2- 172.21.15.98:0/1267809352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff5e4151c90 0x7ff5e4145d60 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:24.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.317+0000 7ff5eb893640 1 --2- 172.21.15.98:0/1267809352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff5d00051d0 0x7ff5e4154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:24.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.317+0000 7ff5eb893640 1 -- 172.21.15.98:0/1267809352 >> 172.21.15.98:0/1267809352 conn(0x7ff5e40b2010 msgr2=0x7ff5e415b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:24.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.317+0000 7ff5eb893640 1 -- 172.21.15.98:0/1267809352 shutdown_connections 2024-09-19T08:42:24.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.317+0000 7ff5eb893640 1 -- 172.21.15.98:0/1267809352 wait complete. 2024-09-19T08:42:24.340 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T08:42:24.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.469+0000 7fe6f9da5640 1 Processor -- start 2024-09-19T08:42:24.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.469+0000 7fe6f9da5640 1 -- start start 2024-09-19T08:42:24.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.469+0000 7fe6f9da5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6f40cea90 0x7fe6f40cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:24.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.469+0000 7fe6f9da5640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe6f415b7e0 0x7fe6f415e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:24.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.469+0000 7fe6f9da5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6f4158870 con 0x7fe6f415ea10 2024-09-19T08:42:24.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.469+0000 7fe6f9da5640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fe6f40cd600 con 0x7fe6f415b7e0 2024-09-19T08:42:24.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.469+0000 7fe6f9da5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe6f40ccd10 con 0x7fe6f40cea90 2024-09-19T08:42:24.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.469+0000 7fe6ebfff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe6f415b7e0 0x7fe6f415e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:24.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.469+0000 7fe6ebfff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe6f415b7e0 0x7fe6f415e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54456/0 (socket says 172.21.15.98:54456) 2024-09-19T08:42:24.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.469+0000 7fe6ebfff640 1 -- 172.21.15.98:0/829208425 learned_addr learned my addr 172.21.15.98:0/829208425 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:24.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.469+0000 7fe6f37fe640 1 --2- 172.21.15.98:0/829208425 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6f40cea90 0x7fe6f40cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:24.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.469+0000 7fe6ebfff640 1 -- 172.21.15.98:0/829208425 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6f40cea90 msgr2=0x7fe6f40cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:24.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.469+0000 7fe6ebfff640 1 --2- 172.21.15.98:0/829208425 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6f40cea90 0x7fe6f40cee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:24.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.469+0000 7fe6ebfff640 1 -- 172.21.15.98:0/829208425 >> v1:172.21.15.139:6789/0 conn(0x7fe6f415ea10 legacy=0x7fe6f40b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:24.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.469+0000 7fe6ebfff640 1 -- 172.21.15.98:0/829208425 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6f40cf3b0 con 0x7fe6f415b7e0 2024-09-19T08:42:24.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.473+0000 7fe6ebfff640 1 --2- 172.21.15.98:0/829208425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6f415b7e0 0x7fe6f415e4d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc009870 tx=0x7fe6dc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bcb51ce33a3c65c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:24.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.473+0000 7fe6f2ffd640 1 -- 172.21.15.98:0/829208425 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc0aa430 con 0x7fe6f415b7e0 2024-09-19T08:42:24.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.473+0000 7fe6f2ffd640 1 -- 172.21.15.98:0/829208425 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6dc0af070 con 0x7fe6f415b7e0 2024-09-19T08:42:24.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.473+0000 7fe6f2ffd640 1 -- 172.21.15.98:0/829208425 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc0aa7f0 con 0x7fe6f415b7e0 2024-09-19T08:42:24.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.473+0000 7fe6f9da5640 1 -- 172.21.15.98:0/829208425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6f415b7e0 msgr2=0x7fe6f415e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:24.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.473+0000 7fe6f9da5640 1 --2- 172.21.15.98:0/829208425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6f415b7e0 0x7fe6f415e4d0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc009870 tx=0x7fe6dc01c4f0 comp rx=0 tx=0).stop 2024-09-19T08:42:24.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.473+0000 7fe6f9da5640 1 -- 172.21.15.98:0/829208425 shutdown_connections 2024-09-19T08:42:24.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.473+0000 7fe6f9da5640 1 --2- 172.21.15.98:0/829208425 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6f415b7e0 0x7fe6f415e4d0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:24.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.473+0000 7fe6f9da5640 1 --2- 172.21.15.98:0/829208425 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6f40cea90 0x7fe6f40cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:24.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.473+0000 7fe6f9da5640 1 -- 172.21.15.98:0/829208425 >> 172.21.15.98:0/829208425 conn(0x7fe6f40b2010 msgr2=0x7fe6f40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:24.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.473+0000 7fe6f9da5640 1 -- 172.21.15.98:0/829208425 shutdown_connections 2024-09-19T08:42:24.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.473+0000 7fe6f9da5640 1 -- 172.21.15.98:0/829208425 wait complete. 2024-09-19T08:42:24.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.473+0000 7fe6f9da5640 1 Processor -- start 2024-09-19T08:42:24.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.473+0000 7fe6f9da5640 1 -- start start 2024-09-19T08:42:24.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.473+0000 7fe6f9da5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6f40cea90 0x7fe6f40b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:24.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.473+0000 7fe6f9da5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6f415b7e0 0x7fe6f4155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:24.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.473+0000 7fe6f9da5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6f415ea10 0x7fe6f415a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:24.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.473+0000 7fe6f9da5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f40ce4c0 con 0x7fe6f415b7e0 2024-09-19T08:42:24.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.473+0000 7fe6f9da5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f406cba0 con 0x7fe6f415ea10 2024-09-19T08:42:24.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.473+0000 7fe6f9da5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe6f40cda70 con 0x7fe6f40cea90 2024-09-19T08:42:24.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.473+0000 7fe6ebfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6f415b7e0 0x7fe6f4155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:24.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.473+0000 7fe6ebfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6f415b7e0 0x7fe6f4155eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54470/0 (socket says 172.21.15.98:54470) 2024-09-19T08:42:24.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.473+0000 7fe6ebfff640 1 -- 172.21.15.98:0/3399268680 learned_addr learned my addr 172.21.15.98:0/3399268680 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:24.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.473+0000 7fe6f37fe640 1 --2- 172.21.15.98:0/3399268680 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6f40cea90 0x7fe6f40b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:24.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.473+0000 7fe6ebfff640 1 -- 172.21.15.98:0/3399268680 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6f40cea90 msgr2=0x7fe6f40b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:24.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.473+0000 7fe6f3fff640 1 --2- 172.21.15.98:0/3399268680 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6f415ea10 0x7fe6f415a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:24.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.473+0000 7fe6ebfff640 1 --2- 172.21.15.98:0/3399268680 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6f40cea90 0x7fe6f40b0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:24.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.473+0000 7fe6ebfff640 1 -- 172.21.15.98:0/3399268680 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6f415ea10 msgr2=0x7fe6f415a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:24.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.473+0000 7fe6ebfff640 1 --2- 172.21.15.98:0/3399268680 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6f415ea10 0x7fe6f415a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:24.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.473+0000 7fe6ebfff640 1 -- 172.21.15.98:0/3399268680 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6f4160070 con 0x7fe6f415b7e0 2024-09-19T08:42:24.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.473+0000 7fe6ebfff640 1 --2- 172.21.15.98:0/3399268680 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6f415b7e0 0x7fe6f4155eb0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc0ac040 tx=0x7fe6dc01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:24.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.473+0000 7fe6f0ff9640 1 -- 172.21.15.98:0/3399268680 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc0bd5f0 con 0x7fe6f415b7e0 2024-09-19T08:42:24.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.473+0000 7fe6f0ff9640 1 -- 172.21.15.98:0/3399268680 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6dc0bd790 con 0x7fe6f415b7e0 2024-09-19T08:42:24.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.473+0000 7fe6f9da5640 1 -- 172.21.15.98:0/3399268680 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6f4073d60 con 0x7fe6f415b7e0 2024-09-19T08:42:24.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.473+0000 7fe6f0ff9640 1 -- 172.21.15.98:0/3399268680 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc0bdb80 con 0x7fe6f415b7e0 2024-09-19T08:42:24.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.473+0000 7fe6f9da5640 1 -- 172.21.15.98:0/3399268680 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6f415f830 con 0x7fe6f415b7e0 2024-09-19T08:42:24.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.477+0000 7fe6f9da5640 1 -- 172.21.15.98:0/3399268680 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6f4160550 con 0x7fe6f415b7e0 2024-09-19T08:42:24.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.477+0000 7fe6f0ff9640 1 -- 172.21.15.98:0/3399268680 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fe6dc0b33f0 con 0x7fe6f415b7e0 2024-09-19T08:42:24.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.477+0000 7fe6f0ff9640 1 --2- 172.21.15.98:0/3399268680 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe6d4067c20 0x7fe6d406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:24.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.477+0000 7fe6f0ff9640 1 -- 172.21.15.98:0/3399268680 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 4943+0+0 (secure 0 0 0) 0x7fe6dc120ba0 con 0x7fe6f415b7e0 2024-09-19T08:42:24.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.477+0000 7fe6f37fe640 1 --2- 172.21.15.98:0/3399268680 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe6d4067c20 0x7fe6d406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:24.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.477+0000 7fe6f37fe640 1 --2- 172.21.15.98:0/3399268680 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe6d4067c20 0x7fe6d406a0e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400b9d0 tx=0x7fe6e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:24.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.481+0000 7fe6f0ff9640 1 -- 172.21.15.98:0/3399268680 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe6dc0c3060 con 0x7fe6f415b7e0 2024-09-19T08:42:24.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.641+0000 7fe6f9da5640 1 -- 172.21.15.98:0/3399268680 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe6f406c730 con 0x7fe6f415b7e0 2024-09-19T08:42:24.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.649+0000 7fe6f0ff9640 1 -- 172.21.15.98:0/3399268680 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7fe6dc016110 con 0x7fe6f415b7e0 2024-09-19T08:42:24.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.649+0000 7fe6f9da5640 1 -- 172.21.15.98:0/3399268680 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe6d4067c20 msgr2=0x7fe6d406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:24.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.649+0000 7fe6f9da5640 1 --2- 172.21.15.98:0/3399268680 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe6d4067c20 0x7fe6d406a0e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400b9d0 tx=0x7fe6e4008040 comp rx=0 tx=0).stop 2024-09-19T08:42:24.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.649+0000 7fe6f9da5640 1 -- 172.21.15.98:0/3399268680 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6f415b7e0 msgr2=0x7fe6f4155eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:24.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.649+0000 7fe6f9da5640 1 --2- 172.21.15.98:0/3399268680 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6f415b7e0 0x7fe6f4155eb0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc0ac040 tx=0x7fe6dc01ad30 comp rx=0 tx=0).stop 2024-09-19T08:42:24.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.653+0000 7fe6f9da5640 1 -- 172.21.15.98:0/3399268680 shutdown_connections 2024-09-19T08:42:24.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.653+0000 7fe6f9da5640 1 --2- 172.21.15.98:0/3399268680 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe6d4067c20 0x7fe6d406a0e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:24.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.653+0000 7fe6f9da5640 1 --2- 172.21.15.98:0/3399268680 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6f415ea10 0x7fe6f415a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:24.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.653+0000 7fe6f9da5640 1 --2- 172.21.15.98:0/3399268680 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6f415b7e0 0x7fe6f4155eb0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:24.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.653+0000 7fe6f9da5640 1 --2- 172.21.15.98:0/3399268680 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6f40cea90 0x7fe6f40b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:24.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.653+0000 7fe6f9da5640 1 -- 172.21.15.98:0/3399268680 >> 172.21.15.98:0/3399268680 conn(0x7fe6f40b2010 msgr2=0x7fe6f406bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:24.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.653+0000 7fe6f9da5640 1 -- 172.21.15.98:0/3399268680 shutdown_connections 2024-09-19T08:42:24.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:24.653+0000 7fe6f9da5640 1 -- 172.21.15.98:0/3399268680 wait complete. 2024-09-19T08:42:24.668 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T08:42:24.668 DEBUG:teuthology.orchestra.run.smithi098:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:42:24.738 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:42:24.807 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T08:42:24.807 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:42:24.807 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:42:24.813 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15846 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T08:42:24.813 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:42:24.813 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:42:24.819 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T08:42:24.819 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:42:24.819 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:42:24.861 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15846 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T08:42:24.861 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:42:24.861 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:42:24.867 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T08:42:24.867 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-19T08:42:24.868 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:42:25.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.025+0000 7f48d2d21640 1 Processor -- start 2024-09-19T08:42:25.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.025+0000 7f48d2d21640 1 -- start start 2024-09-19T08:42:25.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.025+0000 7f48d2d21640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f48cc147530 0x7f48cc0d1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:25.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.025+0000 7f48d2d21640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48cc0b8060 con 0x7f48cc0d1d70 2024-09-19T08:42:25.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.025+0000 7f48d2d21640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48cc0b8340 con 0x7f48cc0b6200 2024-09-19T08:42:25.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.025+0000 7f48d2d21640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f48cc0d2230 con 0x7f48cc147530 2024-09-19T08:42:25.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.025+0000 7f48d0abd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f48cc147530 0x7f48cc0d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:25.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.025+0000 7f48cbfff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f48cc0d1d70 0x7f48cc15ec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:59030/0 (socket says 172.21.15.98:59030) 2024-09-19T08:42:25.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.025+0000 7f48cbfff640 1 -- 172.21.15.98:0/700153993 learned_addr learned my addr 172.21.15.98:0/700153993 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:25.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.025+0000 7f48d0abd640 1 -- 172.21.15.98:0/700153993 >> v1:172.21.15.139:6789/0 conn(0x7f48cc0b6200 legacy=0x7f48cc0b8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:25.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.025+0000 7f48cb7fe640 1 -- 172.21.15.98:0/700153993 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2312354080 0 0) 0x7f48cc0b8060 con 0x7f48cc0d1d70 2024-09-19T08:42:25.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.025+0000 7f48d0abd640 1 -- 172.21.15.98:0/700153993 >> v1:172.21.15.98:6789/0 conn(0x7f48cc0d1d70 legacy=0x7f48cc15ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:25.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.025+0000 7f48d0abd640 1 -- 172.21.15.98:0/700153993 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48cc0b8620 con 0x7f48cc147530 2024-09-19T08:42:25.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.029+0000 7f48d0abd640 1 --2- 172.21.15.98:0/700153993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48cc147530 0x7f48cc0d1510 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f48c40097c0 tx=0x7f48c401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f110e688d4de8803 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:25.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.029+0000 7f48cb7fe640 1 -- 172.21.15.98:0/700153993 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c40b1070 con 0x7f48cc147530 2024-09-19T08:42:25.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.029+0000 7f48cb7fe640 1 -- 172.21.15.98:0/700153993 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48c40ac630 con 0x7f48cc147530 2024-09-19T08:42:25.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.029+0000 7f48cb7fe640 1 -- 172.21.15.98:0/700153993 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c40a8050 con 0x7f48cc147530 2024-09-19T08:42:25.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.029+0000 7f48d2d21640 1 -- 172.21.15.98:0/700153993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48cc147530 msgr2=0x7f48cc0d1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:25.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.029+0000 7f48d2d21640 1 --2- 172.21.15.98:0/700153993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48cc147530 0x7f48cc0d1510 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f48c40097c0 tx=0x7f48c401c430 comp rx=0 tx=0).stop 2024-09-19T08:42:25.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.029+0000 7f48d2d21640 1 -- 172.21.15.98:0/700153993 shutdown_connections 2024-09-19T08:42:25.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.029+0000 7f48d2d21640 1 --2- 172.21.15.98:0/700153993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48cc147530 0x7f48cc0d1510 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:25.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.029+0000 7f48d2d21640 1 -- 172.21.15.98:0/700153993 >> 172.21.15.98:0/700153993 conn(0x7f48cc075070 msgr2=0x7f48cc0722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:25.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.029+0000 7f48d2d21640 1 -- 172.21.15.98:0/700153993 shutdown_connections 2024-09-19T08:42:25.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.029+0000 7f48d2d21640 1 -- 172.21.15.98:0/700153993 wait complete. 2024-09-19T08:42:25.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.029+0000 7f48d2d21640 1 Processor -- start 2024-09-19T08:42:25.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.029+0000 7f48d2d21640 1 -- start start 2024-09-19T08:42:25.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.029+0000 7f48d2d21640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48cc0b6200 0x7f48cc142840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:25.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.029+0000 7f48d2d21640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48cc0d1d70 0x7f48cc15b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:25.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.029+0000 7f48d2d21640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48cc147530 0x7f48cc13f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:25.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.029+0000 7f48d2d21640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc147a90 con 0x7f48cc147530 2024-09-19T08:42:25.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.029+0000 7f48d2d21640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc147910 con 0x7f48cc0b6200 2024-09-19T08:42:25.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.029+0000 7f48d2d21640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f48cc0b9940 con 0x7f48cc0d1d70 2024-09-19T08:42:25.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.033+0000 7f48d12be640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48cc147530 0x7f48cc13f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:25.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.033+0000 7f48d12be640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48cc147530 0x7f48cc13f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54496/0 (socket says 172.21.15.98:54496) 2024-09-19T08:42:25.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.033+0000 7f48d12be640 1 -- 172.21.15.98:0/173821686 learned_addr learned my addr 172.21.15.98:0/173821686 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:25.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.033+0000 7f48cbfff640 1 --2- 172.21.15.98:0/173821686 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48cc0d1d70 0x7f48cc15b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:25.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.033+0000 7f48d0abd640 1 --2- 172.21.15.98:0/173821686 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48cc0b6200 0x7f48cc142840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:25.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.033+0000 7f48d12be640 1 -- 172.21.15.98:0/173821686 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48cc0d1d70 msgr2=0x7f48cc15b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:25.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.033+0000 7f48d12be640 1 --2- 172.21.15.98:0/173821686 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48cc0d1d70 0x7f48cc15b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:25.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.033+0000 7f48d12be640 1 -- 172.21.15.98:0/173821686 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48cc0b6200 msgr2=0x7f48cc142840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:25.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.033+0000 7f48d12be640 1 --2- 172.21.15.98:0/173821686 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48cc0b6200 0x7f48cc142840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:25.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.033+0000 7f48d12be640 1 -- 172.21.15.98:0/173821686 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48cc140b60 con 0x7f48cc147530 2024-09-19T08:42:25.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.033+0000 7f48cbfff640 1 --2- 172.21.15.98:0/173821686 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48cc0d1d70 0x7f48cc15b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:42:25.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.033+0000 7f48d12be640 1 --2- 172.21.15.98:0/173821686 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48cc147530 0x7f48cc13f110 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00b410 tx=0x7f48bc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:25.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.033+0000 7f48c97fa640 1 -- 172.21.15.98:0/173821686 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc015020 con 0x7f48cc147530 2024-09-19T08:42:25.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.033+0000 7f48c97fa640 1 -- 172.21.15.98:0/173821686 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48bc011870 con 0x7f48cc147530 2024-09-19T08:42:25.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.033+0000 7f48c97fa640 1 -- 172.21.15.98:0/173821686 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc011c60 con 0x7f48cc147530 2024-09-19T08:42:25.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.033+0000 7f48d2d21640 1 -- 172.21.15.98:0/173821686 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48cc140de0 con 0x7f48cc147530 2024-09-19T08:42:25.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.033+0000 7f48d2d21640 1 -- 172.21.15.98:0/173821686 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48cc1433e0 con 0x7f48cc147530 2024-09-19T08:42:25.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.033+0000 7f48d2d21640 1 -- 172.21.15.98:0/173821686 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48cc0d1510 con 0x7f48cc147530 2024-09-19T08:42:25.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.033+0000 7f48c97fa640 1 -- 172.21.15.98:0/173821686 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f48bc018020 con 0x7f48cc147530 2024-09-19T08:42:25.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.033+0000 7f48c97fa640 1 --2- 172.21.15.98:0/173821686 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f48b8067c20 0x7f48b806a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:25.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.033+0000 7f48d0abd640 1 --2- 172.21.15.98:0/173821686 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f48b8067c20 0x7f48b806a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:25.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.033+0000 7f48c97fa640 1 -- 172.21.15.98:0/173821686 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 4943+0+0 (secure 0 0 0) 0x7f48bc085da0 con 0x7f48cc147530 2024-09-19T08:42:25.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.033+0000 7f48d0abd640 1 --2- 172.21.15.98:0/173821686 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f48b8067c20 0x7f48b806a0e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f48c40ae000 tx=0x7f48c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:25.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.037+0000 7f48c97fa640 1 -- 172.21.15.98:0/173821686 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f48bc01f020 con 0x7f48cc147530 2024-09-19T08:42:25.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.229+0000 7f48d2d21640 1 -- 172.21.15.98:0/173821686 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48cc13d1a0 con 0x7f48cc147530 2024-09-19T08:42:25.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.229+0000 7f48c97fa640 1 -- 172.21.15.98:0/173821686 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+3914 (secure 0 0 0) 0x7f48cc140b60 con 0x7f48cc147530 2024-09-19T08:42:25.230 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:42:25.230 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":33,"btime":"2024-09-19T08:42:18:980157+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4880,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/1549394645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":1549394645},{"type":"v1","addr":"172.21.15.139:6834","nonce":1549394645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":4883,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1847525681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6833","nonce":1847525681},{"type":"v1","addr":"172.21.15.139:6835","nonce":1847525681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5745,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/1677812726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":1677812726},{"type":"v1","addr":"172.21.15.98:6835","nonce":1677812726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":6087,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6837/107618854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6836","nonce":107618854},{"type":"v1","addr":"172.21.15.98:6837","nonce":107618854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":33}],"filesystems":[]} 2024-09-19T08:42:25.231 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 33 2024-09-19T08:42:25.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.233+0000 7f48d2d21640 1 -- 172.21.15.98:0/173821686 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f48b8067c20 msgr2=0x7f48b806a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:25.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.233+0000 7f48d2d21640 1 --2- 172.21.15.98:0/173821686 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f48b8067c20 0x7f48b806a0e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f48c40ae000 tx=0x7f48c4008040 comp rx=0 tx=0).stop 2024-09-19T08:42:25.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.233+0000 7f48d2d21640 1 -- 172.21.15.98:0/173821686 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48cc147530 msgr2=0x7f48cc13f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:25.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.233+0000 7f48d2d21640 1 --2- 172.21.15.98:0/173821686 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48cc147530 0x7f48cc13f110 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00b410 tx=0x7f48bc00b8d0 comp rx=0 tx=0).stop 2024-09-19T08:42:25.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.233+0000 7f48d2d21640 1 -- 172.21.15.98:0/173821686 shutdown_connections 2024-09-19T08:42:25.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.233+0000 7f48d2d21640 1 --2- 172.21.15.98:0/173821686 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f48b8067c20 0x7f48b806a0e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:25.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.233+0000 7f48d2d21640 1 --2- 172.21.15.98:0/173821686 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f48cc147530 0x7f48cc13f110 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:25.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.233+0000 7f48d2d21640 1 --2- 172.21.15.98:0/173821686 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f48cc0d1d70 0x7f48cc15b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:25.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.233+0000 7f48d2d21640 1 --2- 172.21.15.98:0/173821686 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48cc0b6200 0x7f48cc142840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:25.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.233+0000 7f48d2d21640 1 -- 172.21.15.98:0/173821686 >> 172.21.15.98:0/173821686 conn(0x7f48cc075070 msgr2=0x7f48cc0722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:25.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.233+0000 7f48d2d21640 1 -- 172.21.15.98:0/173821686 shutdown_connections 2024-09-19T08:42:25.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.233+0000 7f48d2d21640 1 -- 172.21.15.98:0/173821686 wait complete. 2024-09-19T08:42:25.248 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-19T08:42:25.248 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-19T08:42:25.248 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-19T08:42:25.248 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:42:25.249 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-19T08:42:25.249 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-19T08:42:25.249 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-19T08:42:25.250 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:42:25.250 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-19T08:42:25.250 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-19T08:42:25.250 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-19T08:42:25.250 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:42:25.250 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-19T08:42:25.250 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-19T08:42:25.251 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-19T08:42:25.251 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:42:25.258 INFO:tasks.ceph.mds.c:Stopped 2024-09-19T08:42:25.259 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-19T08:42:25.260 INFO:tasks.ceph.mds.a:Stopped 2024-09-19T08:42:25.260 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-19T08:42:25.268 INFO:tasks.ceph.mds.b:Stopped 2024-09-19T08:42:25.268 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-19T08:42:25.270 INFO:tasks.ceph.mds.d:Stopped 2024-09-19T08:42:25.270 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-19T08:42:25.301 INFO:tasks.ceph.mds.c:Started 2024-09-19T08:42:25.302 DEBUG:teuthology.parallel:result is None 2024-09-19T08:42:25.302 INFO:tasks.ceph.mds.a:Started 2024-09-19T08:42:25.302 DEBUG:teuthology.parallel:result is None 2024-09-19T08:42:25.314 INFO:tasks.ceph.mds.b:Started 2024-09-19T08:42:25.314 DEBUG:teuthology.parallel:result is None 2024-09-19T08:42:25.316 INFO:tasks.ceph.mds.d:Started 2024-09-19T08:42:25.316 DEBUG:teuthology.parallel:result is None 2024-09-19T08:42:25.316 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-19T08:42:25.361 INFO:tasks.ceph.mds.c.smithi098.stdout:starting mds.c at 2024-09-19T08:42:25.361 INFO:tasks.ceph.mds.a.smithi098.stdout:starting mds.a at 2024-09-19T08:42:25.388 INFO:tasks.ceph.mds.d.smithi139.stdout:starting mds.d at 2024-09-19T08:42:25.389 INFO:tasks.ceph.mds.b.smithi139.stdout:starting mds.b at 2024-09-19T08:42:25.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.421+0000 7f0044986640 1 Processor -- start 2024-09-19T08:42:25.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.421+0000 7f0044986640 1 -- start start 2024-09-19T08:42:25.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.425+0000 7f0044986640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f004015c550 0x7f004006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:25.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.425+0000 7f0044986640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f004006d310 con 0x7f004006c1d0 2024-09-19T08:42:25.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.425+0000 7f0044986640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f004006d5f0 con 0x7f00400cfad0 2024-09-19T08:42:25.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.425+0000 7f0044986640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f004006c640 con 0x7f004015c550 2024-09-19T08:42:25.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.425+0000 7f003dd9b640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f004006c1d0 0x7f00400cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:59050/0 (socket says 172.21.15.98:59050) 2024-09-19T08:42:25.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.425+0000 7f003dd9b640 1 -- 172.21.15.98:0/1441297305 learned_addr learned my addr 172.21.15.98:0/1441297305 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:25.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.425+0000 7f003e59c640 1 --2- 172.21.15.98:0/1441297305 >> v2:172.21.15.139:3300/0 conn(0x7f004015c550 0x7f004006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:25.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.425+0000 7f003d59a640 1 -- 172.21.15.98:0/1441297305 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1352086079 0 0) 0x7f004006d310 con 0x7f004006c1d0 2024-09-19T08:42:25.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.425+0000 7f003d59a640 1 -- 172.21.15.98:0/1441297305 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f002c003530 con 0x7f004006c1d0 2024-09-19T08:42:25.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.425+0000 7f003e59c640 1 -- 172.21.15.98:0/1441297305 >> v1:172.21.15.139:6789/0 conn(0x7f00400cfad0 legacy=0x7f00400723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:25.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.425+0000 7f003e59c640 1 -- 172.21.15.98:0/1441297305 >> v1:172.21.15.98:6789/0 conn(0x7f004006c1d0 legacy=0x7f00400cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:25.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.425+0000 7f003e59c640 1 -- 172.21.15.98:0/1441297305 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f004006d8d0 con 0x7f004015c550 2024-09-19T08:42:25.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.425+0000 7f003e59c640 1 --2- 172.21.15.98:0/1441297305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f004015c550 0x7f004006bc90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f00300097c0 tx=0x7f003001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4325d4b5c4c9a870 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:25.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.425+0000 7f003d59a640 1 -- 172.21.15.98:0/1441297305 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00300b0070 con 0x7f004015c550 2024-09-19T08:42:25.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.425+0000 7f003d59a640 1 -- 172.21.15.98:0/1441297305 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00300ab600 con 0x7f004015c550 2024-09-19T08:42:25.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.425+0000 7f003d59a640 1 -- 172.21.15.98:0/1441297305 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00300ab9d0 con 0x7f004015c550 2024-09-19T08:42:25.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.429+0000 7f0044986640 1 -- 172.21.15.98:0/1441297305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f004015c550 msgr2=0x7f004006bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:25.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.429+0000 7f0044986640 1 --2- 172.21.15.98:0/1441297305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f004015c550 0x7f004006bc90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f00300097c0 tx=0x7f003001c4a0 comp rx=0 tx=0).stop 2024-09-19T08:42:25.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.429+0000 7f0044986640 1 -- 172.21.15.98:0/1441297305 shutdown_connections 2024-09-19T08:42:25.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.429+0000 7f0044986640 1 --2- 172.21.15.98:0/1441297305 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f004015c550 0x7f004006bc90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:25.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.429+0000 7f0044986640 1 -- 172.21.15.98:0/1441297305 >> 172.21.15.98:0/1441297305 conn(0x7f00400b2010 msgr2=0x7f00400b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:25.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.429+0000 7f0044986640 1 -- 172.21.15.98:0/1441297305 shutdown_connections 2024-09-19T08:42:25.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.429+0000 7f0044986640 1 -- 172.21.15.98:0/1441297305 wait complete. 2024-09-19T08:42:25.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.429+0000 7f0044986640 1 Processor -- start 2024-09-19T08:42:25.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.429+0000 7f0044986640 1 -- start start 2024-09-19T08:42:25.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.429+0000 7f0044986640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f004006c1d0 0x7f004014c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:25.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.429+0000 7f0044986640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00400cfad0 0x7f0040159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:25.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.429+0000 7f0044986640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f004015c550 0x7f00400d0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:25.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.429+0000 7f0044986640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f00400736d0 con 0x7f004006c1d0 2024-09-19T08:42:25.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.429+0000 7f0044986640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0040073550 con 0x7f00400cfad0 2024-09-19T08:42:25.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.429+0000 7f0044986640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f004016a240 con 0x7f004015c550 2024-09-19T08:42:25.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.429+0000 7f003e59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f004006c1d0 0x7f004014c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:25.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.429+0000 7f003e59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f004006c1d0 0x7f004014c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54542/0 (socket says 172.21.15.98:54542) 2024-09-19T08:42:25.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.429+0000 7f003e59c640 1 -- 172.21.15.98:0/4272904366 learned_addr learned my addr 172.21.15.98:0/4272904366 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:25.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.429+0000 7f003ed9d640 1 --2- 172.21.15.98:0/4272904366 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f004015c550 0x7f00400d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:25.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.429+0000 7f003dd9b640 1 --2- 172.21.15.98:0/4272904366 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00400cfad0 0x7f0040159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:25.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.429+0000 7f003e59c640 1 -- 172.21.15.98:0/4272904366 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f004015c550 msgr2=0x7f00400d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:25.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.429+0000 7f003e59c640 1 --2- 172.21.15.98:0/4272904366 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f004015c550 0x7f00400d0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:25.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.429+0000 7f003e59c640 1 -- 172.21.15.98:0/4272904366 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00400cfad0 msgr2=0x7f0040159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:25.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.429+0000 7f003e59c640 1 --2- 172.21.15.98:0/4272904366 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00400cfad0 0x7f0040159260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:25.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.429+0000 7f003e59c640 1 -- 172.21.15.98:0/4272904366 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00400b5420 con 0x7f004006c1d0 2024-09-19T08:42:25.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.429+0000 7f003e59c640 1 --2- 172.21.15.98:0/4272904366 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f004006c1d0 0x7f004014c090 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f00300190b0 tx=0x7f003001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:25.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.429+0000 7f0036ffd640 1 -- 172.21.15.98:0/4272904366 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00300b0050 con 0x7f004006c1d0 2024-09-19T08:42:25.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.429+0000 7f0036ffd640 1 -- 172.21.15.98:0/4272904366 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00300bc070 con 0x7f004006c1d0 2024-09-19T08:42:25.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.429+0000 7f0036ffd640 1 -- 172.21.15.98:0/4272904366 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00300b5830 con 0x7f004006c1d0 2024-09-19T08:42:25.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.429+0000 7f0044986640 1 -- 172.21.15.98:0/4272904366 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0040156d70 con 0x7f004006c1d0 2024-09-19T08:42:25.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.429+0000 7f0044986640 1 -- 172.21.15.98:0/4272904366 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0040157330 con 0x7f004006c1d0 2024-09-19T08:42:25.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.429+0000 7f0044986640 1 -- 172.21.15.98:0/4272904366 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f004006bc90 con 0x7f004006c1d0 2024-09-19T08:42:25.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.433+0000 7f0036ffd640 1 -- 172.21.15.98:0/4272904366 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f00300b59d0 con 0x7f004006c1d0 2024-09-19T08:42:25.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.433+0000 7f0036ffd640 1 --2- 172.21.15.98:0/4272904366 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0020067c20 0x7f002006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:25.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.433+0000 7f0036ffd640 1 -- 172.21.15.98:0/4272904366 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 4943+0+0 (secure 0 0 0) 0x7f0030120710 con 0x7f004006c1d0 2024-09-19T08:42:25.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.433+0000 7f003dd9b640 1 --2- 172.21.15.98:0/4272904366 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0020067c20 0x7f002006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:25.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.433+0000 7f003dd9b640 1 --2- 172.21.15.98:0/4272904366 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0020067c20 0x7f002006a0e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f00401fa9a0 tx=0x7f0028009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:25.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.437+0000 7f0036ffd640 1 -- 172.21.15.98:0/4272904366 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f0030016090 con 0x7f004006c1d0 2024-09-19T08:42:25.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:25.609+0000 7f0044986640 1 -- 172.21.15.98:0/4272904366 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f0040152210 con 0x7f004006c1d0 2024-09-19T08:42:26.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.333+0000 7f0036ffd640 1 -- 172.21.15.98:0/4272904366 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v62) ==== 97+0+0 (secure 0 0 0) 0x7f00300f1030 con 0x7f004006c1d0 2024-09-19T08:42:26.334 INFO:teuthology.orchestra.run.smithi098.stderr: removed all blocklist entries 2024-09-19T08:42:26.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.337+0000 7f0044986640 1 -- 172.21.15.98:0/4272904366 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0020067c20 msgr2=0x7f002006a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:26.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.337+0000 7f0044986640 1 --2- 172.21.15.98:0/4272904366 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0020067c20 0x7f002006a0e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f00401fa9a0 tx=0x7f0028009040 comp rx=0 tx=0).stop 2024-09-19T08:42:26.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.337+0000 7f0044986640 1 -- 172.21.15.98:0/4272904366 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f004006c1d0 msgr2=0x7f004014c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:26.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.337+0000 7f0044986640 1 --2- 172.21.15.98:0/4272904366 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f004006c1d0 0x7f004014c090 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f00300190b0 tx=0x7f003001ace0 comp rx=0 tx=0).stop 2024-09-19T08:42:26.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.337+0000 7f0044986640 1 -- 172.21.15.98:0/4272904366 shutdown_connections 2024-09-19T08:42:26.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.337+0000 7f0044986640 1 --2- 172.21.15.98:0/4272904366 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f0020067c20 0x7f002006a0e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:26.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.337+0000 7f0044986640 1 --2- 172.21.15.98:0/4272904366 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f004015c550 0x7f00400d0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:26.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.337+0000 7f0044986640 1 --2- 172.21.15.98:0/4272904366 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00400cfad0 0x7f0040159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:26.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.337+0000 7f0044986640 1 --2- 172.21.15.98:0/4272904366 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f004006c1d0 0x7f004014c090 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:26.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.337+0000 7f0044986640 1 -- 172.21.15.98:0/4272904366 >> 172.21.15.98:0/4272904366 conn(0x7f00400b2010 msgr2=0x7f00400afe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:26.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.337+0000 7f0044986640 1 -- 172.21.15.98:0/4272904366 shutdown_connections 2024-09-19T08:42:26.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.337+0000 7f0044986640 1 -- 172.21.15.98:0/4272904366 wait complete. 2024-09-19T08:42:26.357 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-19T08:42:26.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.473+0000 7f4f77487640 1 Processor -- start 2024-09-19T08:42:26.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.473+0000 7f4f77487640 1 -- start start 2024-09-19T08:42:26.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.477+0000 7f4f77487640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4f7006c120 0x7f4f700cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:26.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.477+0000 7f4f77487640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f700ccc60 con 0x7f4f7006bc90 2024-09-19T08:42:26.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.477+0000 7f4f77487640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f700ccf40 con 0x7f4f70158fd0 2024-09-19T08:42:26.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.477+0000 7f4f77487640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f4f701593b0 con 0x7f4f7006c120 2024-09-19T08:42:26.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.477+0000 7f4f74a22640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f4f7006bc90 0x7f4f700ce840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:59052/0 (socket says 172.21.15.98:59052) 2024-09-19T08:42:26.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.477+0000 7f4f74a22640 1 -- 172.21.15.98:0/269379063 learned_addr learned my addr 172.21.15.98:0/269379063 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:26.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.477+0000 7f4f67fff640 1 -- 172.21.15.98:0/269379063 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4252197377 0 0) 0x7f4f700ccc60 con 0x7f4f7006bc90 2024-09-19T08:42:26.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.477+0000 7f4f67fff640 1 -- 172.21.15.98:0/269379063 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f5c003530 con 0x7f4f7006bc90 2024-09-19T08:42:26.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.477+0000 7f4f75a24640 1 --2- 172.21.15.98:0/269379063 >> v2:172.21.15.139:3300/0 conn(0x7f4f7006c120 0x7f4f700cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:26.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.477+0000 7f4f67fff640 1 -- 172.21.15.98:0/269379063 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 930298887 0 0) 0x7f4f5c003530 con 0x7f4f7006bc90 2024-09-19T08:42:26.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.477+0000 7f4f67fff640 1 -- 172.21.15.98:0/269379063 >> v2:172.21.15.139:3300/0 conn(0x7f4f7006c120 msgr2=0x7f4f700cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:26.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.477+0000 7f4f67fff640 1 --2- 172.21.15.98:0/269379063 >> v2:172.21.15.139:3300/0 conn(0x7f4f7006c120 0x7f4f700cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:26.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.477+0000 7f4f67fff640 1 -- 172.21.15.98:0/269379063 >> v1:172.21.15.139:6789/0 conn(0x7f4f70158fd0 legacy=0x7f4f700b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:26.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.477+0000 7f4f67fff640 1 -- 172.21.15.98:0/269379063 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f700cd220 con 0x7f4f7006bc90 2024-09-19T08:42:26.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.477+0000 7f4f67fff640 1 -- 172.21.15.98:0/269379063 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f4f60001680 con 0x7f4f7006bc90 2024-09-19T08:42:26.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.477+0000 7f4f67fff640 1 -- 172.21.15.98:0/269379063 >> v1:172.21.15.98:6789/0 conn(0x7f4f7006bc90 legacy=0x7f4f700ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:26.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.477+0000 7f4f67fff640 1 --2- 172.21.15.98:0/269379063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f5c0039c0 0x7f4f5c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:26.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.477+0000 7f4f67fff640 1 --2- 172.21.15.98:0/269379063 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f5c004810 0x7f4f5c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:26.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.477+0000 7f4f67fff640 1 --2- 172.21.15.98:0/269379063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f5c0052f0 0x7f4f5c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:26.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.477+0000 7f4f67fff640 1 -- 172.21.15.98:0/269379063 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4f701593b0 con 0x7f4f5c0039c0 2024-09-19T08:42:26.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.477+0000 7f4f67fff640 1 -- 172.21.15.98:0/269379063 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5c00c1c0 con 0x7f4f5c0052f0 2024-09-19T08:42:26.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.477+0000 7f4f67fff640 1 -- 172.21.15.98:0/269379063 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4f5c00c3c0 con 0x7f4f5c004810 2024-09-19T08:42:26.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.477+0000 7f4f75223640 1 --2- 172.21.15.98:0/269379063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f5c0039c0 0x7f4f5c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:26.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.477+0000 7f4f75223640 1 -- 172.21.15.98:0/269379063 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f5c004810 msgr2=0x7f4f5c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:26.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.477+0000 7f4f75223640 1 --2- 172.21.15.98:0/269379063 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f5c004810 0x7f4f5c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:26.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.477+0000 7f4f75223640 1 -- 172.21.15.98:0/269379063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f5c0052f0 msgr2=0x7f4f5c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:26.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.477+0000 7f4f75223640 1 --2- 172.21.15.98:0/269379063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f5c0052f0 0x7f4f5c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:26.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.477+0000 7f4f75223640 1 -- 172.21.15.98:0/269379063 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f5c003530 con 0x7f4f5c0039c0 2024-09-19T08:42:26.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.477+0000 7f4f75a24640 1 -- 172.21.15.98:0/269379063 reap_dead start 2024-09-19T08:42:26.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.477+0000 7f4f75223640 1 --2- 172.21.15.98:0/269379063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f5c0039c0 0x7f4f5c003de0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c001750 tx=0x7f4f6c01c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=832565988bceb9a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:26.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.481+0000 7f4f67fff640 1 -- 172.21.15.98:0/269379063 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c0013a0 con 0x7f4f5c0039c0 2024-09-19T08:42:26.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.481+0000 7f4f67fff640 1 -- 172.21.15.98:0/269379063 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f6c0052f0 con 0x7f4f5c0039c0 2024-09-19T08:42:26.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.481+0000 7f4f67fff640 1 -- 172.21.15.98:0/269379063 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c003840 con 0x7f4f5c0039c0 2024-09-19T08:42:26.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.481+0000 7f4f77487640 1 -- 172.21.15.98:0/269379063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f5c0039c0 msgr2=0x7f4f5c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:26.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.481+0000 7f4f77487640 1 --2- 172.21.15.98:0/269379063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f5c0039c0 0x7f4f5c003de0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c001750 tx=0x7f4f6c01c3b0 comp rx=0 tx=0).stop 2024-09-19T08:42:26.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.481+0000 7f4f77487640 1 -- 172.21.15.98:0/269379063 shutdown_connections 2024-09-19T08:42:26.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.481+0000 7f4f77487640 1 --2- 172.21.15.98:0/269379063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f5c0039c0 0x7f4f5c003de0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:26.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.481+0000 7f4f77487640 1 -- 172.21.15.98:0/269379063 >> 172.21.15.98:0/269379063 conn(0x7f4f700b2010 msgr2=0x7f4f700b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:26.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.481+0000 7f4f77487640 1 -- 172.21.15.98:0/269379063 shutdown_connections 2024-09-19T08:42:26.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.481+0000 7f4f77487640 1 -- 172.21.15.98:0/269379063 wait complete. 2024-09-19T08:42:26.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.481+0000 7f4f77487640 1 Processor -- start 2024-09-19T08:42:26.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.481+0000 7f4f77487640 1 -- start start 2024-09-19T08:42:26.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.481+0000 7f4f77487640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f5c0039c0 0x7f4f70156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:26.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.481+0000 7f4f77487640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f7014a000 0x7f4f70071970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:26.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.481+0000 7f4f77487640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f70073db0 0x7f4f70071eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:26.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.481+0000 7f4f77487640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7015c920 con 0x7f4f5c0039c0 2024-09-19T08:42:26.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.481+0000 7f4f77487640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7015c7a0 con 0x7f4f70073db0 2024-09-19T08:42:26.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.481+0000 7f4f77487640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4f7015efc0 con 0x7f4f7014a000 2024-09-19T08:42:26.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.481+0000 7f4f75223640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f5c0039c0 0x7f4f70156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:26.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.481+0000 7f4f75223640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f5c0039c0 0x7f4f70156e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54546/0 (socket says 172.21.15.98:54546) 2024-09-19T08:42:26.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.481+0000 7f4f75223640 1 -- 172.21.15.98:0/241026370 learned_addr learned my addr 172.21.15.98:0/241026370 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:26.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.481+0000 7f4f74a22640 1 --2- 172.21.15.98:0/241026370 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f7014a000 0x7f4f70071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:26.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.481+0000 7f4f75a24640 1 --2- 172.21.15.98:0/241026370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f70073db0 0x7f4f70071eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:26.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.481+0000 7f4f75223640 1 -- 172.21.15.98:0/241026370 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f7014a000 msgr2=0x7f4f70071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:26.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.481+0000 7f4f75223640 1 --2- 172.21.15.98:0/241026370 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f7014a000 0x7f4f70071970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:26.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.481+0000 7f4f75223640 1 -- 172.21.15.98:0/241026370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f70073db0 msgr2=0x7f4f70071eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:26.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.481+0000 7f4f75223640 1 --2- 172.21.15.98:0/241026370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f70073db0 0x7f4f70071eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:26.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.481+0000 7f4f75223640 1 -- 172.21.15.98:0/241026370 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f70157400 con 0x7f4f5c0039c0 2024-09-19T08:42:26.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.481+0000 7f4f75a24640 1 --2- 172.21.15.98:0/241026370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f70073db0 0x7f4f70071eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:42:26.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.481+0000 7f4f75223640 1 --2- 172.21.15.98:0/241026370 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f5c0039c0 0x7f4f70156e20 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c005040 tx=0x7f4f5c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:26.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.481+0000 7f4f65ffb640 1 -- 172.21.15.98:0/241026370 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c0b8020 con 0x7f4f5c0039c0 2024-09-19T08:42:26.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.481+0000 7f4f65ffb640 1 -- 172.21.15.98:0/241026370 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f6c0054a0 con 0x7f4f5c0039c0 2024-09-19T08:42:26.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.481+0000 7f4f77487640 1 -- 172.21.15.98:0/241026370 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f701576e0 con 0x7f4f5c0039c0 2024-09-19T08:42:26.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.481+0000 7f4f65ffb640 1 -- 172.21.15.98:0/241026370 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c0013a0 con 0x7f4f5c0039c0 2024-09-19T08:42:26.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.481+0000 7f4f77487640 1 -- 172.21.15.98:0/241026370 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f700726d0 con 0x7f4f5c0039c0 2024-09-19T08:42:26.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.485+0000 7f4f77487640 1 -- 172.21.15.98:0/241026370 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f7015f2f0 con 0x7f4f5c0039c0 2024-09-19T08:42:26.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.485+0000 7f4f65ffb640 1 -- 172.21.15.98:0/241026370 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f4f6c0bb020 con 0x7f4f5c0039c0 2024-09-19T08:42:26.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.485+0000 7f4f65ffb640 1 --2- 172.21.15.98:0/241026370 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4f54067c50 0x7f4f5406a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:26.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.485+0000 7f4f65ffb640 1 -- 172.21.15.98:0/241026370 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 4857+0+0 (secure 0 0 0) 0x7f4f6c11fcb0 con 0x7f4f5c0039c0 2024-09-19T08:42:26.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.485+0000 7f4f74a22640 1 --2- 172.21.15.98:0/241026370 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4f54067c50 0x7f4f5406a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:26.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.485+0000 7f4f74a22640 1 --2- 172.21.15.98:0/241026370 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4f54067c50 0x7f4f5406a110 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4f60001ec0 tx=0x7f4f60009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:26.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.489+0000 7f4f65ffb640 1 -- 172.21.15.98:0/241026370 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f4f6c0c2030 con 0x7f4f5c0039c0 2024-09-19T08:42:26.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.705+0000 7f4f77487640 1 -- 172.21.15.98:0/241026370 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f4f700aeed0 con 0x7f4f5c0039c0 2024-09-19T08:42:26.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.709+0000 7f4f65ffb640 1 -- 172.21.15.98:0/241026370 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5617 (secure 0 0 0) 0x7f4f6c0160f0 con 0x7f4f5c0039c0 2024-09-19T08:42:26.708 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:42:26.708 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:42:26.709 INFO:teuthology.orchestra.run.smithi098.stdout: "auth_dump": [ 2024-09-19T08:42:26.709 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:42:26.709 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.a", 2024-09-19T08:42:26.709 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmuujGBRAAa2k/kW68NJI6nUKx+W+G2Q==", 2024-09-19T08:42:26.709 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:42:26.709 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:42:26.709 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:42:26.710 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:42:26.710 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:42:26.710 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:42:26.710 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:42:26.710 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:42:26.710 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.b", 2024-09-19T08:42:26.710 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmFoojFBAAaOZPpa41fdjBPEn3CGsJnA==", 2024-09-19T08:42:26.710 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:42:26.710 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:42:26.710 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:42:26.710 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:42:26.710 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:42:26.710 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:42:26.711 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:42:26.711 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:42:26.711 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.c", 2024-09-19T08:42:26.711 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmcQcDDhAAF0QJMlljMH+DiHfWW+/Sog==", 2024-09-19T08:42:26.711 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:42:26.711 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:42:26.711 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:42:26.711 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:42:26.711 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:42:26.711 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:42:26.711 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:42:26.711 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:42:26.711 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.d", 2024-09-19T08:42:26.712 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmwWmqHBAAf15VcrdeHf8AzQjGsN9LoA==", 2024-09-19T08:42:26.712 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:42:26.712 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:42:26.712 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:42:26.712 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:42:26.712 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:42:26.712 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:42:26.712 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:42:26.712 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:42:26.712 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.0", 2024-09-19T08:42:26.712 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDC4utm9PjNKxAAa6SG9AE4smPiu4azRAFKvg==", 2024-09-19T08:42:26.713 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:42:26.713 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:42:26.713 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:42:26.713 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:42:26.713 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:42:26.713 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:42:26.713 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:42:26.713 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.1", 2024-09-19T08:42:26.713 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDG4utmHy6UCxAAN6hpG4DJEp81iG/KoloTvQ==", 2024-09-19T08:42:26.713 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:42:26.713 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:42:26.713 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:42:26.714 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:42:26.714 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:42:26.714 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:42:26.714 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:42:26.714 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.2", 2024-09-19T08:42:26.714 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDJ4utmUVP4GhAAJNAL30sEAseoDG2fE6YbZQ==", 2024-09-19T08:42:26.714 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:42:26.714 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:42:26.714 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:42:26.714 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:42:26.714 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:42:26.714 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:42:26.714 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:42:26.715 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.3", 2024-09-19T08:42:26.715 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDM4utmTKgoLRAAiCrPRLML+wvjOt6RhKay3w==", 2024-09-19T08:42:26.715 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:42:26.715 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:42:26.715 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:42:26.715 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:42:26.715 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:42:26.715 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:42:26.715 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:42:26.715 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.4", 2024-09-19T08:42:26.715 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDS4utmAf6bMhAAI3uENuFoXLicUeuspgFjYA==", 2024-09-19T08:42:26.715 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:42:26.715 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:42:26.716 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:42:26.716 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:42:26.716 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:42:26.716 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:42:26.716 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:42:26.716 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.5", 2024-09-19T08:42:26.716 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDW4utmRjxwEhAAF1A2GtrZXVx9BlmjHjhUDQ==", 2024-09-19T08:42:26.716 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:42:26.716 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:42:26.716 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:42:26.716 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:42:26.716 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:42:26.716 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:42:26.717 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:42:26.717 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.6", 2024-09-19T08:42:26.717 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDZ4utmSk/XIRAAROSfzGL7smR88ZkDSx5jAg==", 2024-09-19T08:42:26.717 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:42:26.717 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:42:26.717 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:42:26.717 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:42:26.717 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:42:26.717 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:42:26.717 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:42:26.717 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.7", 2024-09-19T08:42:26.717 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDc4utmXfpbMxAAqEM4I/ACFvfu/sMA8bMcrQ==", 2024-09-19T08:42:26.717 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:42:26.718 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:42:26.718 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:42:26.718 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:42:26.718 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:42:26.718 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:42:26.718 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:42:26.718 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.0", 2024-09-19T08:42:26.718 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utm0vNpIhAA4S1E8dFZ0+ibmSKhmZ+bIA==", 2024-09-19T08:42:26.718 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:42:26.718 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:42:26.718 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow r", 2024-09-19T08:42:26.718 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-19T08:42:26.719 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:42:26.719 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:42:26.719 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:42:26.719 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.1", 2024-09-19T08:42:26.719 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmx7xyKhAAC84lykPEb01MiypXKQVREQ==", 2024-09-19T08:42:26.719 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:42:26.719 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:42:26.719 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-19T08:42:26.719 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-19T08:42:26.719 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-19T08:42:26.719 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:42:26.719 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:42:26.719 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:42:26.719 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.2", 2024-09-19T08:42:26.720 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmteOQJxAAtTF2Py21ZE2WRQjeQSrkKQ==", 2024-09-19T08:42:26.720 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:42:26.720 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:42:26.720 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-19T08:42:26.720 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-19T08:42:26.720 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-19T08:42:26.720 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:42:26.720 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:42:26.720 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:42:26.720 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.3", 2024-09-19T08:42:26.720 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmKN92LxAAMqraZOWRT6IIfx5AuDWS2g==", 2024-09-19T08:42:26.720 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:42:26.721 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:42:26.721 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-19T08:42:26.721 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-19T08:42:26.721 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-19T08:42:26.721 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:42:26.721 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:42:26.721 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:42:26.721 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.admin", 2024-09-19T08:42:26.721 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC+4utmLH97KhAA1DTYW5bTrGUG+7cb8+d7gA==", 2024-09-19T08:42:26.721 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:42:26.722 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-09-19T08:42:26.722 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:42:26.722 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:42:26.722 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:42:26.722 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:42:26.722 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:42:26.722 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:42:26.722 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.alice", 2024-09-19T08:42:26.722 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDl4+tmaRNMABAA7Zkv2UbmJKSZ8W9z9oI5pA==", 2024-09-19T08:42:26.722 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:42:26.722 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow rw path=/volumes/_nogroup/subvol98020", 2024-09-19T08:42:26.722 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow r", 2024-09-19T08:42:26.722 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rw pool=cephfs_data" 2024-09-19T08:42:26.723 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:42:26.723 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:42:26.723 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:42:26.723 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mgr.x", 2024-09-19T08:42:26.723 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmbKuyAhAA0/XjHUIPOIV/YvNEhB8rEQ==", 2024-09-19T08:42:26.723 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:42:26.723 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-09-19T08:42:26.723 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile mgr", 2024-09-19T08:42:26.723 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:42:26.723 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:42:26.723 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:42:26.723 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:42:26.723 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mgr.y", 2024-09-19T08:42:26.723 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC+4utmU+ZrOxAAsvcwINH6qyxfN7N+rDrkqg==", 2024-09-19T08:42:26.724 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:42:26.724 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-09-19T08:42:26.724 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile mgr", 2024-09-19T08:42:26.724 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:42:26.724 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:42:26.724 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:42:26.724 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-19T08:42:26.724 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:42:26.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.709+0000 7f4f77487640 1 -- 172.21.15.98:0/241026370 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4f54067c50 msgr2=0x7f4f5406a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:26.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.709+0000 7f4f77487640 1 --2- 172.21.15.98:0/241026370 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4f54067c50 0x7f4f5406a110 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4f60001ec0 tx=0x7f4f60009040 comp rx=0 tx=0).stop 2024-09-19T08:42:26.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.709+0000 7f4f77487640 1 -- 172.21.15.98:0/241026370 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f5c0039c0 msgr2=0x7f4f70156e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:26.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.709+0000 7f4f77487640 1 --2- 172.21.15.98:0/241026370 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f5c0039c0 0x7f4f70156e20 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c005040 tx=0x7f4f5c005840 comp rx=0 tx=0).stop 2024-09-19T08:42:26.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.709+0000 7f4f77487640 1 -- 172.21.15.98:0/241026370 shutdown_connections 2024-09-19T08:42:26.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.709+0000 7f4f77487640 1 --2- 172.21.15.98:0/241026370 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f4f54067c50 0x7f4f5406a110 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:26.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.709+0000 7f4f77487640 1 --2- 172.21.15.98:0/241026370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4f70073db0 0x7f4f70071eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:26.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.709+0000 7f4f77487640 1 --2- 172.21.15.98:0/241026370 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4f7014a000 0x7f4f70071970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:26.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.709+0000 7f4f77487640 1 --2- 172.21.15.98:0/241026370 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4f5c0039c0 0x7f4f70156e20 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:26.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.709+0000 7f4f77487640 1 -- 172.21.15.98:0/241026370 >> 172.21.15.98:0/241026370 conn(0x7f4f700b2010 msgr2=0x7f4f70159b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:26.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.709+0000 7f4f77487640 1 -- 172.21.15.98:0/241026370 shutdown_connections 2024-09-19T08:42:26.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.709+0000 7f4f77487640 1 -- 172.21.15.98:0/241026370 wait complete. 2024-09-19T08:42:26.727 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.alice 2024-09-19T08:42:26.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.841+0000 7ff0ee420640 1 Processor -- start 2024-09-19T08:42:26.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.841+0000 7ff0ee420640 1 -- start start 2024-09-19T08:42:26.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.841+0000 7ff0ee420640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0e815c220 0x7ff0e80733f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:26.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.841+0000 7ff0ee420640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff0e8073930 0x7ff0e80d0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:26.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.841+0000 7ff0ee420640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0e80740b0 con 0x7ff0e80d1530 2024-09-19T08:42:26.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.841+0000 7ff0ee420640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff0e8073dd0 con 0x7ff0e8073930 2024-09-19T08:42:26.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.841+0000 7ff0ee420640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff0e80b6460 con 0x7ff0e815c220 2024-09-19T08:42:26.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.841+0000 7ff0e7fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0e815c220 0x7ff0e80733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:26.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.841+0000 7ff0e77fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff0e8073930 0x7ff0e80d0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:26.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.841+0000 7ff0e7fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0e815c220 0x7ff0e80733f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:59802/0 (socket says 172.21.15.98:59802) 2024-09-19T08:42:26.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.841+0000 7ff0e7fff640 1 -- 172.21.15.98:0/4266177075 learned_addr learned my addr 172.21.15.98:0/4266177075 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:26.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.841+0000 7ff0e7fff640 1 -- 172.21.15.98:0/4266177075 >> v2:172.21.15.139:3300/0 conn(0x7ff0e8073930 msgr2=0x7ff0e80d0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:26.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.841+0000 7ff0e7fff640 1 --2- 172.21.15.98:0/4266177075 >> v2:172.21.15.139:3300/0 conn(0x7ff0e8073930 0x7ff0e80d0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:26.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.841+0000 7ff0e7fff640 1 -- 172.21.15.98:0/4266177075 >> v1:172.21.15.139:6789/0 conn(0x7ff0e80d1530 legacy=0x7ff0e80b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:26.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.841+0000 7ff0e7fff640 1 -- 172.21.15.98:0/4266177075 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0e80b6740 con 0x7ff0e815c220 2024-09-19T08:42:26.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.845+0000 7ff0e7fff640 1 --2- 172.21.15.98:0/4266177075 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0e815c220 0x7ff0e80733f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0009e50 tx=0x7ff0e001ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28997b39c66cb810 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:26.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.845+0000 7ff0e6ffd640 1 -- 172.21.15.98:0/4266177075 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e00a5810 con 0x7ff0e815c220 2024-09-19T08:42:26.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.845+0000 7ff0e6ffd640 1 -- 172.21.15.98:0/4266177075 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0e00a59b0 con 0x7ff0e815c220 2024-09-19T08:42:26.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.845+0000 7ff0e6ffd640 1 -- 172.21.15.98:0/4266177075 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e00a5d80 con 0x7ff0e815c220 2024-09-19T08:42:26.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.845+0000 7ff0ee420640 1 -- 172.21.15.98:0/4266177075 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0e815c220 msgr2=0x7ff0e80733f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:26.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.845+0000 7ff0ee420640 1 --2- 172.21.15.98:0/4266177075 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0e815c220 0x7ff0e80733f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0009e50 tx=0x7ff0e001ca30 comp rx=0 tx=0).stop 2024-09-19T08:42:26.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.845+0000 7ff0ee420640 1 -- 172.21.15.98:0/4266177075 shutdown_connections 2024-09-19T08:42:26.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.845+0000 7ff0ee420640 1 --2- 172.21.15.98:0/4266177075 >> v2:172.21.15.139:3300/0 conn(0x7ff0e8073930 0x7ff0e80d0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:26.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.845+0000 7ff0ee420640 1 --2- 172.21.15.98:0/4266177075 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0e815c220 0x7ff0e80733f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:26.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.845+0000 7ff0ee420640 1 -- 172.21.15.98:0/4266177075 >> 172.21.15.98:0/4266177075 conn(0x7ff0e80b40a0 msgr2=0x7ff0e80b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:26.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.845+0000 7ff0ee420640 1 -- 172.21.15.98:0/4266177075 shutdown_connections 2024-09-19T08:42:26.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.845+0000 7ff0ee420640 1 -- 172.21.15.98:0/4266177075 wait complete. 2024-09-19T08:42:26.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.845+0000 7ff0ee420640 1 Processor -- start 2024-09-19T08:42:26.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.845+0000 7ff0ee420640 1 -- start start 2024-09-19T08:42:26.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.849+0000 7ff0ee420640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0e8073930 0x7ff0e80b2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:26.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.849+0000 7ff0ee420640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0e80d1530 0x7ff0e814b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:26.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.849+0000 7ff0ee420640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0e815c220 0x7ff0e814ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:26.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.849+0000 7ff0ee420640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e8168ea0 con 0x7ff0e8073930 2024-09-19T08:42:26.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.849+0000 7ff0ee420640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e815d0a0 con 0x7ff0e815c220 2024-09-19T08:42:26.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.849+0000 7ff0ee420640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff0e8169020 con 0x7ff0e80d1530 2024-09-19T08:42:26.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.849+0000 7ff0e7fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0e8073930 0x7ff0e80b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:26.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.849+0000 7ff0e7fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0e8073930 0x7ff0e80b2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54552/0 (socket says 172.21.15.98:54552) 2024-09-19T08:42:26.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.849+0000 7ff0e7fff640 1 -- 172.21.15.98:0/2114800865 learned_addr learned my addr 172.21.15.98:0/2114800865 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:26.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.849+0000 7ff0e77fe640 1 --2- 172.21.15.98:0/2114800865 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0e80d1530 0x7ff0e814b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:26.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.849+0000 7ff0ec9bd640 1 --2- 172.21.15.98:0/2114800865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0e815c220 0x7ff0e814ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:26.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.849+0000 7ff0e7fff640 1 -- 172.21.15.98:0/2114800865 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0e80d1530 msgr2=0x7ff0e814b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:26.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.849+0000 7ff0e7fff640 1 --2- 172.21.15.98:0/2114800865 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0e80d1530 0x7ff0e814b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:26.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.849+0000 7ff0e7fff640 1 -- 172.21.15.98:0/2114800865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0e815c220 msgr2=0x7ff0e814ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:26.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.849+0000 7ff0e7fff640 1 --2- 172.21.15.98:0/2114800865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0e815c220 0x7ff0e814ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:26.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.849+0000 7ff0e7fff640 1 -- 172.21.15.98:0/2114800865 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0e80b2ad0 con 0x7ff0e8073930 2024-09-19T08:42:26.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.849+0000 7ff0e7fff640 1 --2- 172.21.15.98:0/2114800865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0e8073930 0x7ff0e80b2010 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7ff0e00af040 tx=0x7ff0e00197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:26.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.849+0000 7ff0e4ff9640 1 -- 172.21.15.98:0/2114800865 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e00bc070 con 0x7ff0e8073930 2024-09-19T08:42:26.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.849+0000 7ff0e4ff9640 1 -- 172.21.15.98:0/2114800865 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0e00b8640 con 0x7ff0e8073930 2024-09-19T08:42:26.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.849+0000 7ff0e4ff9640 1 -- 172.21.15.98:0/2114800865 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e00b89f0 con 0x7ff0e8073930 2024-09-19T08:42:26.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.849+0000 7ff0ee420640 1 -- 172.21.15.98:0/2114800865 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0e80b2db0 con 0x7ff0e8073930 2024-09-19T08:42:26.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.849+0000 7ff0ee420640 1 -- 172.21.15.98:0/2114800865 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0e8159280 con 0x7ff0e8073930 2024-09-19T08:42:26.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.849+0000 7ff0ee420640 1 -- 172.21.15.98:0/2114800865 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0e80b31a0 con 0x7ff0e8073930 2024-09-19T08:42:26.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.849+0000 7ff0e4ff9640 1 -- 172.21.15.98:0/2114800865 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7ff0e00b8b90 con 0x7ff0e8073930 2024-09-19T08:42:26.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.849+0000 7ff0e4ff9640 1 --2- 172.21.15.98:0/2114800865 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff0d4067c20 0x7ff0d406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:26.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.849+0000 7ff0e77fe640 1 --2- 172.21.15.98:0/2114800865 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff0d4067c20 0x7ff0d406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:26.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.849+0000 7ff0e4ff9640 1 -- 172.21.15.98:0/2114800865 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 4857+0+0 (secure 0 0 0) 0x7ff0e0120ee0 con 0x7ff0e8073930 2024-09-19T08:42:26.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.849+0000 7ff0e77fe640 1 --2- 172.21.15.98:0/2114800865 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff0d4067c20 0x7ff0d406a0e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff0e815b2c0 tx=0x7ff0dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:26.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:26.853+0000 7ff0e4ff9640 1 -- 172.21.15.98:0/2114800865 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff0e0016620 con 0x7ff0e8073930 2024-09-19T08:42:27.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.057+0000 7ff0ee420640 1 -- 172.21.15.98:0/2114800865 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.alice"} v 0) -- 0x7ff0e80d0ff0 con 0x7ff0e8073930 2024-09-19T08:42:27.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.061+0000 7ff0e4ff9640 1 -- 172.21.15.98:0/2114800865 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.alice"}]=0 v17) ==== 82+0+0 (secure 0 0 0) 0x7ff0e00f19f0 con 0x7ff0e8073930 2024-09-19T08:42:27.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.061+0000 7ff0ee420640 1 -- 172.21.15.98:0/2114800865 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff0d4067c20 msgr2=0x7ff0d406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:27.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.061+0000 7ff0ee420640 1 --2- 172.21.15.98:0/2114800865 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff0d4067c20 0x7ff0d406a0e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff0e815b2c0 tx=0x7ff0dc008040 comp rx=0 tx=0).stop 2024-09-19T08:42:27.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.061+0000 7ff0ee420640 1 -- 172.21.15.98:0/2114800865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0e8073930 msgr2=0x7ff0e80b2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:27.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.061+0000 7ff0ee420640 1 --2- 172.21.15.98:0/2114800865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0e8073930 0x7ff0e80b2010 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7ff0e00af040 tx=0x7ff0e00197d0 comp rx=0 tx=0).stop 2024-09-19T08:42:27.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.061+0000 7ff0ee420640 1 -- 172.21.15.98:0/2114800865 shutdown_connections 2024-09-19T08:42:27.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.061+0000 7ff0ee420640 1 --2- 172.21.15.98:0/2114800865 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff0d4067c20 0x7ff0d406a0e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:27.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.061+0000 7ff0ee420640 1 --2- 172.21.15.98:0/2114800865 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0e815c220 0x7ff0e814ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:27.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.061+0000 7ff0ee420640 1 --2- 172.21.15.98:0/2114800865 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0e80d1530 0x7ff0e814b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:27.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.061+0000 7ff0ee420640 1 --2- 172.21.15.98:0/2114800865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff0e8073930 0x7ff0e80b2010 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:27.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.061+0000 7ff0ee420640 1 -- 172.21.15.98:0/2114800865 >> 172.21.15.98:0/2114800865 conn(0x7ff0e80b40a0 msgr2=0x7ff0e80b6f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:27.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.061+0000 7ff0ee420640 1 -- 172.21.15.98:0/2114800865 shutdown_connections 2024-09-19T08:42:27.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.061+0000 7ff0ee420640 1 -- 172.21.15.98:0/2114800865 wait complete. 2024-09-19T08:42:27.080 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-19T08:42:27.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.197+0000 7f70fc983640 1 Processor -- start 2024-09-19T08:42:27.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.197+0000 7f70fc983640 1 -- start start 2024-09-19T08:42:27.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.197+0000 7f70fc983640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f70f80cd350 0x7f70f80b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:27.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.197+0000 7f70fc983640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f70f80ce380 0x7f70f80cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:27.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.197+0000 7f70fc983640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70f80cd730 con 0x7f70f80b6f50 2024-09-19T08:42:27.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.197+0000 7f70fc983640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f70f80b5f60 con 0x7f70f80cd350 2024-09-19T08:42:27.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.197+0000 7f70fc983640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f70f80b6190 con 0x7f70f80ce380 2024-09-19T08:42:27.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.197+0000 7f70f659c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f70f80cd350 0x7f70f80b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:27.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.197+0000 7f70f659c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f70f80cd350 0x7f70f80b24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54560/0 (socket says 172.21.15.98:54560) 2024-09-19T08:42:27.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.197+0000 7f70f659c640 1 -- 172.21.15.98:0/3696065969 learned_addr learned my addr 172.21.15.98:0/3696065969 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:27.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.197+0000 7f70f6d9d640 1 --2- 172.21.15.98:0/3696065969 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f70f80ce380 0x7f70f80cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:27.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.197+0000 7f70f659c640 1 -- 172.21.15.98:0/3696065969 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f70f80ce380 msgr2=0x7f70f80cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:27.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.197+0000 7f70f659c640 1 --2- 172.21.15.98:0/3696065969 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f70f80ce380 0x7f70f80cb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:27.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.197+0000 7f70f659c640 1 -- 172.21.15.98:0/3696065969 >> v1:172.21.15.139:6789/0 conn(0x7f70f80b6f50 legacy=0x7f70f80cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:27.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.197+0000 7f70f659c640 1 -- 172.21.15.98:0/3696065969 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70f80b6470 con 0x7f70f80cd350 2024-09-19T08:42:27.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.201+0000 7f70f659c640 1 --2- 172.21.15.98:0/3696065969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70f80cd350 0x7f70f80b24b0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f70e0009870 tx=0x7f70e001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93da2ea1e2f0c408 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:27.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.201+0000 7f70f559a640 1 -- 172.21.15.98:0/3696065969 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e00aa610 con 0x7f70f80cd350 2024-09-19T08:42:27.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.201+0000 7f70f559a640 1 -- 172.21.15.98:0/3696065969 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70e00af070 con 0x7f70f80cd350 2024-09-19T08:42:27.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.201+0000 7f70f559a640 1 -- 172.21.15.98:0/3696065969 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e00aa9d0 con 0x7f70f80cd350 2024-09-19T08:42:27.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.201+0000 7f70fc983640 1 -- 172.21.15.98:0/3696065969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70f80cd350 msgr2=0x7f70f80b24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:27.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.201+0000 7f70fc983640 1 --2- 172.21.15.98:0/3696065969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70f80cd350 0x7f70f80b24b0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f70e0009870 tx=0x7f70e001c4f0 comp rx=0 tx=0).stop 2024-09-19T08:42:27.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.201+0000 7f70fc983640 1 -- 172.21.15.98:0/3696065969 shutdown_connections 2024-09-19T08:42:27.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.201+0000 7f70fc983640 1 --2- 172.21.15.98:0/3696065969 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f70f80ce380 0x7f70f80cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:27.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.201+0000 7f70fc983640 1 --2- 172.21.15.98:0/3696065969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70f80cd350 0x7f70f80b24b0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:27.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.201+0000 7f70fc983640 1 -- 172.21.15.98:0/3696065969 >> 172.21.15.98:0/3696065969 conn(0x7f70f8073170 msgr2=0x7f70f8073580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:27.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.201+0000 7f70fc983640 1 -- 172.21.15.98:0/3696065969 shutdown_connections 2024-09-19T08:42:27.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.201+0000 7f70fc983640 1 -- 172.21.15.98:0/3696065969 wait complete. 2024-09-19T08:42:27.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.201+0000 7f70fc983640 1 Processor -- start 2024-09-19T08:42:27.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.201+0000 7f70fc983640 1 -- start start 2024-09-19T08:42:27.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.201+0000 7f70fc983640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70f80b6f50 0x7f70f80c4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:27.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.201+0000 7f70fc983640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f70f80cd350 0x7f70f8156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:27.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.201+0000 7f70fc983640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70f80ce380 0x7f70f80cf820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:27.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.201+0000 7f70fc983640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f70f8168190 con 0x7f70f80ce380 2024-09-19T08:42:27.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.201+0000 7f70fc983640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f70f8168010 con 0x7f70f80b6f50 2024-09-19T08:42:27.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.201+0000 7f70fc983640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f70f8168310 con 0x7f70f80cd350 2024-09-19T08:42:27.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.201+0000 7f70f6d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70f80ce380 0x7f70f80cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:27.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.201+0000 7f70f6d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70f80ce380 0x7f70f80cf820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54564/0 (socket says 172.21.15.98:54564) 2024-09-19T08:42:27.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.201+0000 7f70f6d9d640 1 -- 172.21.15.98:0/1300846221 learned_addr learned my addr 172.21.15.98:0/1300846221 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:27.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.205+0000 7f70f5d9b640 1 --2- 172.21.15.98:0/1300846221 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f70f80cd350 0x7f70f8156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:27.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.205+0000 7f70f659c640 1 --2- 172.21.15.98:0/1300846221 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70f80b6f50 0x7f70f80c4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:27.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.205+0000 7f70f6d9d640 1 -- 172.21.15.98:0/1300846221 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f70f80cd350 msgr2=0x7f70f8156510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:27.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.205+0000 7f70f6d9d640 1 --2- 172.21.15.98:0/1300846221 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f70f80cd350 0x7f70f8156510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:27.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.205+0000 7f70f6d9d640 1 -- 172.21.15.98:0/1300846221 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70f80b6f50 msgr2=0x7f70f80c4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:27.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.205+0000 7f70f6d9d640 1 --2- 172.21.15.98:0/1300846221 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70f80b6f50 0x7f70f80c4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:27.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.205+0000 7f70f6d9d640 1 -- 172.21.15.98:0/1300846221 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70f806c4b0 con 0x7f70f80ce380 2024-09-19T08:42:27.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.205+0000 7f70f6d9d640 1 --2- 172.21.15.98:0/1300846221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70f80ce380 0x7f70f80cf820 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f70f0009fd0 tx=0x7f70f000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:27.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.205+0000 7f70eeffd640 1 -- 172.21.15.98:0/1300846221 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f0014420 con 0x7f70f80ce380 2024-09-19T08:42:27.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.205+0000 7f70eeffd640 1 -- 172.21.15.98:0/1300846221 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70f0019070 con 0x7f70f80ce380 2024-09-19T08:42:27.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.205+0000 7f70fc983640 1 -- 172.21.15.98:0/1300846221 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70f80d19f0 con 0x7f70f80ce380 2024-09-19T08:42:27.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.205+0000 7f70eeffd640 1 -- 172.21.15.98:0/1300846221 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f0014800 con 0x7f70f80ce380 2024-09-19T08:42:27.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.205+0000 7f70fc983640 1 -- 172.21.15.98:0/1300846221 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70f80d16a0 con 0x7f70f80ce380 2024-09-19T08:42:27.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.205+0000 7f70fc983640 1 -- 172.21.15.98:0/1300846221 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70f80b24b0 con 0x7f70f80ce380 2024-09-19T08:42:27.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.205+0000 7f70eeffd640 1 -- 172.21.15.98:0/1300846221 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f70f00149a0 con 0x7f70f80ce380 2024-09-19T08:42:27.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.205+0000 7f70eeffd640 1 --2- 172.21.15.98:0/1300846221 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f70d8067c50 0x7f70d806a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:27.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.205+0000 7f70eeffd640 1 -- 172.21.15.98:0/1300846221 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 4857+0+0 (secure 0 0 0) 0x7f70f00865c0 con 0x7f70f80ce380 2024-09-19T08:42:27.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.205+0000 7f70f659c640 1 --2- 172.21.15.98:0/1300846221 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f70d8067c50 0x7f70d806a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:27.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.205+0000 7f70f659c640 1 --2- 172.21.15.98:0/1300846221 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f70d8067c50 0x7f70d806a110 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f70e00ae000 tx=0x7f70e00a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:27.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.209+0000 7f70eeffd640 1 -- 172.21.15.98:0/1300846221 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f70f0057090 con 0x7f70f80ce380 2024-09-19T08:42:27.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.377+0000 7f70fc983640 1 -- 172.21.15.98:0/1300846221 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f70f815de00 con 0x7f70f80ce380 2024-09-19T08:42:27.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.377+0000 7f70eeffd640 1 -- 172.21.15.98:0/1300846221 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v64) ==== 84+0+65 (secure 0 0 0) 0x7f70f0056990 con 0x7f70f80ce380 2024-09-19T08:42:27.377 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:42:27.378 INFO:teuthology.orchestra.run.smithi098.stdout:[ 2024-09-19T08:42:27.378 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:42:27.378 INFO:teuthology.orchestra.run.smithi098.stdout: "poolnum": 1, 2024-09-19T08:42:27.378 INFO:teuthology.orchestra.run.smithi098.stdout: "poolname": ".mgr" 2024-09-19T08:42:27.378 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:42:27.378 INFO:teuthology.orchestra.run.smithi098.stdout:] 2024-09-19T08:42:27.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.377+0000 7f70fc983640 1 -- 172.21.15.98:0/1300846221 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f70d8067c50 msgr2=0x7f70d806a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:27.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.377+0000 7f70fc983640 1 --2- 172.21.15.98:0/1300846221 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f70d8067c50 0x7f70d806a110 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f70e00ae000 tx=0x7f70e00a7000 comp rx=0 tx=0).stop 2024-09-19T08:42:27.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.377+0000 7f70fc983640 1 -- 172.21.15.98:0/1300846221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70f80ce380 msgr2=0x7f70f80cf820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:27.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.377+0000 7f70fc983640 1 --2- 172.21.15.98:0/1300846221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70f80ce380 0x7f70f80cf820 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f70f0009fd0 tx=0x7f70f000e4b0 comp rx=0 tx=0).stop 2024-09-19T08:42:27.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.381+0000 7f70fc983640 1 -- 172.21.15.98:0/1300846221 shutdown_connections 2024-09-19T08:42:27.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.381+0000 7f70fc983640 1 --2- 172.21.15.98:0/1300846221 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f70d8067c50 0x7f70d806a110 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:27.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.381+0000 7f70fc983640 1 --2- 172.21.15.98:0/1300846221 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f70f80ce380 0x7f70f80cf820 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:27.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.381+0000 7f70fc983640 1 --2- 172.21.15.98:0/1300846221 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f70f80cd350 0x7f70f8156510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:27.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.381+0000 7f70fc983640 1 --2- 172.21.15.98:0/1300846221 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f70f80b6f50 0x7f70f80c4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:27.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.381+0000 7f70fc983640 1 -- 172.21.15.98:0/1300846221 >> 172.21.15.98:0/1300846221 conn(0x7f70f8073170 msgr2=0x7f70f806e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:27.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.381+0000 7f70fc983640 1 -- 172.21.15.98:0/1300846221 shutdown_connections 2024-09-19T08:42:27.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.381+0000 7f70fc983640 1 -- 172.21.15.98:0/1300846221 wait complete. 2024-09-19T08:42:27.397 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T08:42:27.398 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T08:42:27.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.517+0000 7fda38ef1640 1 Processor -- start 2024-09-19T08:42:27.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.517+0000 7fda38ef1640 1 -- start start 2024-09-19T08:42:27.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.517+0000 7fda38ef1640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fda3415f6e0 0x7fda3406d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:27.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.517+0000 7fda38ef1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda3415fac0 0x7fda3406da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:27.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.517+0000 7fda38ef1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fda34073680 0x7fda34073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:27.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.517+0000 7fda38ef1640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fda3415c8b0 con 0x7fda3415f6e0 2024-09-19T08:42:27.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.517+0000 7fda38ef1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fda340742d0 con 0x7fda34073680 2024-09-19T08:42:27.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.517+0000 7fda38ef1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fda3415b890 con 0x7fda3415fac0 2024-09-19T08:42:27.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.517+0000 7fda3259c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fda3415f6e0 0x7fda3406d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:27.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.517+0000 7fda3259c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fda3415f6e0 0x7fda3406d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34216/0 (socket says 172.21.15.98:34216) 2024-09-19T08:42:27.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.517+0000 7fda3259c640 1 -- 172.21.15.98:0/4189202025 learned_addr learned my addr 172.21.15.98:0/4189202025 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:27.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.517+0000 7fda32d9d640 1 --2- 172.21.15.98:0/4189202025 >> v2:172.21.15.139:3300/0 conn(0x7fda34073680 0x7fda34073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:27.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.517+0000 7fda31d9b640 1 --2- 172.21.15.98:0/4189202025 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda3415fac0 0x7fda3406da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:27.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.517+0000 7fda3259c640 1 -- 172.21.15.98:0/4189202025 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda3415fac0 msgr2=0x7fda3406da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:27.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.517+0000 7fda3259c640 1 --2- 172.21.15.98:0/4189202025 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda3415fac0 0x7fda3406da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:27.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.517+0000 7fda3259c640 1 -- 172.21.15.98:0/4189202025 >> v2:172.21.15.139:3300/0 conn(0x7fda34073680 msgr2=0x7fda34073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:27.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.517+0000 7fda3259c640 1 --2- 172.21.15.98:0/4189202025 >> v2:172.21.15.139:3300/0 conn(0x7fda34073680 0x7fda34073ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:27.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.517+0000 7fda3259c640 1 -- 172.21.15.98:0/4189202025 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3415baf0 con 0x7fda3415f6e0 2024-09-19T08:42:27.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.521+0000 7fda3259c640 1 --2- 172.21.15.98:0/4189202025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda3415f6e0 0x7fda3406d4c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fda340b02b0 tx=0x7fda2401eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4955867bf68cac5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:27.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.521+0000 7fda3159a640 1 -- 172.21.15.98:0/4189202025 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda240b4070 con 0x7fda3415f6e0 2024-09-19T08:42:27.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.521+0000 7fda3159a640 1 -- 172.21.15.98:0/4189202025 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda240a7ad0 con 0x7fda3415f6e0 2024-09-19T08:42:27.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.521+0000 7fda3159a640 1 -- 172.21.15.98:0/4189202025 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda240af440 con 0x7fda3415f6e0 2024-09-19T08:42:27.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.521+0000 7fda38ef1640 1 -- 172.21.15.98:0/4189202025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda3415f6e0 msgr2=0x7fda3406d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:27.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.521+0000 7fda38ef1640 1 --2- 172.21.15.98:0/4189202025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda3415f6e0 0x7fda3406d4c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fda340b02b0 tx=0x7fda2401eb30 comp rx=0 tx=0).stop 2024-09-19T08:42:27.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.521+0000 7fda38ef1640 1 -- 172.21.15.98:0/4189202025 shutdown_connections 2024-09-19T08:42:27.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.521+0000 7fda38ef1640 1 --2- 172.21.15.98:0/4189202025 >> v2:172.21.15.139:3300/0 conn(0x7fda34073680 0x7fda34073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:27.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.521+0000 7fda38ef1640 1 --2- 172.21.15.98:0/4189202025 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda3415fac0 0x7fda3406da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:27.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.521+0000 7fda38ef1640 1 --2- 172.21.15.98:0/4189202025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda3415f6e0 0x7fda3406d4c0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:27.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.521+0000 7fda38ef1640 1 -- 172.21.15.98:0/4189202025 >> 172.21.15.98:0/4189202025 conn(0x7fda3406c090 msgr2=0x7fda3406c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:27.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.521+0000 7fda38ef1640 1 -- 172.21.15.98:0/4189202025 shutdown_connections 2024-09-19T08:42:27.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.521+0000 7fda38ef1640 1 -- 172.21.15.98:0/4189202025 wait complete. 2024-09-19T08:42:27.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.525+0000 7fda38ef1640 1 Processor -- start 2024-09-19T08:42:27.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.525+0000 7fda38ef1640 1 -- start start 2024-09-19T08:42:27.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.525+0000 7fda38ef1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda34073680 0x7fda3413e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:27.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.525+0000 7fda38ef1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda3415f6e0 0x7fda34141a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:27.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.525+0000 7fda38ef1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda3415fac0 0x7fda341e6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:27.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.525+0000 7fda38ef1640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fda340cec30 con 0x7fda34073680 2024-09-19T08:42:27.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.525+0000 7fda38ef1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fda34073ff0 con 0x7fda3415fac0 2024-09-19T08:42:27.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.525+0000 7fda38ef1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fda340cedb0 con 0x7fda3415f6e0 2024-09-19T08:42:27.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.525+0000 7fda3259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda34073680 0x7fda3413e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:27.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.525+0000 7fda3259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda34073680 0x7fda3413e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34232/0 (socket says 172.21.15.98:34232) 2024-09-19T08:42:27.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.525+0000 7fda3259c640 1 -- 172.21.15.98:0/1611741213 learned_addr learned my addr 172.21.15.98:0/1611741213 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:27.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.525+0000 7fda32d9d640 1 --2- 172.21.15.98:0/1611741213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda3415fac0 0x7fda341e6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:27.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.525+0000 7fda31d9b640 1 --2- 172.21.15.98:0/1611741213 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda3415f6e0 0x7fda34141a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:27.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.525+0000 7fda3259c640 1 -- 172.21.15.98:0/1611741213 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda3415f6e0 msgr2=0x7fda34141a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:27.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.525+0000 7fda3259c640 1 --2- 172.21.15.98:0/1611741213 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda3415f6e0 0x7fda34141a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:27.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.525+0000 7fda3259c640 1 -- 172.21.15.98:0/1611741213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda3415fac0 msgr2=0x7fda341e6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:27.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.525+0000 7fda3259c640 1 --2- 172.21.15.98:0/1611741213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda3415fac0 0x7fda341e6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:27.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.525+0000 7fda3259c640 1 -- 172.21.15.98:0/1611741213 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda34141f60 con 0x7fda34073680 2024-09-19T08:42:27.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.525+0000 7fda3259c640 1 --2- 172.21.15.98:0/1611741213 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda34073680 0x7fda3413e7b0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fda24000c10 tx=0x7fda2401b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:27.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.525+0000 7fda22ffd640 1 -- 172.21.15.98:0/1611741213 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda240b4030 con 0x7fda34073680 2024-09-19T08:42:27.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.525+0000 7fda22ffd640 1 -- 172.21.15.98:0/1611741213 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda240b6070 con 0x7fda34073680 2024-09-19T08:42:27.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.525+0000 7fda38ef1640 1 -- 172.21.15.98:0/1611741213 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda3413f3d0 con 0x7fda34073680 2024-09-19T08:42:27.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.525+0000 7fda22ffd640 1 -- 172.21.15.98:0/1611741213 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda240ad550 con 0x7fda34073680 2024-09-19T08:42:27.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.525+0000 7fda38ef1640 1 -- 172.21.15.98:0/1611741213 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda3413f990 con 0x7fda34073680 2024-09-19T08:42:27.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.525+0000 7fda38ef1640 1 -- 172.21.15.98:0/1611741213 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda34073ab0 con 0x7fda34073680 2024-09-19T08:42:27.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.525+0000 7fda22ffd640 1 -- 172.21.15.98:0/1611741213 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fda240c1060 con 0x7fda34073680 2024-09-19T08:42:27.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.529+0000 7fda22ffd640 1 --2- 172.21.15.98:0/1611741213 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fda14067c50 0x7fda1406a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:27.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.529+0000 7fda31d9b640 1 --2- 172.21.15.98:0/1611741213 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fda14067c50 0x7fda1406a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:27.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.529+0000 7fda22ffd640 1 -- 172.21.15.98:0/1611741213 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 4857+0+0 (secure 0 0 0) 0x7fda24120320 con 0x7fda34073680 2024-09-19T08:42:27.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.529+0000 7fda31d9b640 1 --2- 172.21.15.98:0/1611741213 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fda14067c50 0x7fda1406a110 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fda34069c30 tx=0x7fda28008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:27.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.533+0000 7fda22ffd640 1 -- 172.21.15.98:0/1611741213 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fda34141f60 con 0x7fda34073680 2024-09-19T08:42:27.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:27.693+0000 7fda38ef1640 1 -- 172.21.15.98:0/1611741213 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fda341ecfc0 con 0x7fda34073680 2024-09-19T08:42:28.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.341+0000 7fda22ffd640 1 -- 172.21.15.98:0/1611741213 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v65) ==== 138+0+0 (secure 0 0 0) 0x7fda240187e0 con 0x7fda34073680 2024-09-19T08:42:28.341 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_metadata' created 2024-09-19T08:42:28.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.341+0000 7fda38ef1640 1 -- 172.21.15.98:0/1611741213 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fda14067c50 msgr2=0x7fda1406a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:28.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.341+0000 7fda38ef1640 1 --2- 172.21.15.98:0/1611741213 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fda14067c50 0x7fda1406a110 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fda34069c30 tx=0x7fda28008040 comp rx=0 tx=0).stop 2024-09-19T08:42:28.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.341+0000 7fda38ef1640 1 -- 172.21.15.98:0/1611741213 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda34073680 msgr2=0x7fda3413e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:28.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.341+0000 7fda38ef1640 1 --2- 172.21.15.98:0/1611741213 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda34073680 0x7fda3413e7b0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fda24000c10 tx=0x7fda2401b8d0 comp rx=0 tx=0).stop 2024-09-19T08:42:28.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.341+0000 7fda38ef1640 1 -- 172.21.15.98:0/1611741213 shutdown_connections 2024-09-19T08:42:28.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.341+0000 7fda38ef1640 1 --2- 172.21.15.98:0/1611741213 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fda14067c50 0x7fda1406a110 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:28.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.341+0000 7fda38ef1640 1 --2- 172.21.15.98:0/1611741213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda3415fac0 0x7fda341e6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:28.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.341+0000 7fda38ef1640 1 --2- 172.21.15.98:0/1611741213 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda3415f6e0 0x7fda34141a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:28.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.341+0000 7fda38ef1640 1 --2- 172.21.15.98:0/1611741213 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda34073680 0x7fda3413e7b0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:28.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.345+0000 7fda38ef1640 1 -- 172.21.15.98:0/1611741213 >> 172.21.15.98:0/1611741213 conn(0x7fda3406c090 msgr2=0x7fda340b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:28.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.345+0000 7fda38ef1640 1 -- 172.21.15.98:0/1611741213 shutdown_connections 2024-09-19T08:42:28.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.345+0000 7fda38ef1640 1 -- 172.21.15.98:0/1611741213 wait complete. 2024-09-19T08:42:28.368 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T08:42:28.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.497+0000 7ff069001640 1 Processor -- start 2024-09-19T08:42:28.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.497+0000 7ff069001640 1 -- start start 2024-09-19T08:42:28.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.497+0000 7ff069001640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff064145b70 0x7ff06406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:28.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.497+0000 7ff069001640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff064071ac0 0x7ff0640b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:28.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.497+0000 7ff069001640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff064073530 con 0x7ff06406c1d0 2024-09-19T08:42:28.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.497+0000 7ff069001640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff06406c620 con 0x7ff064145b70 2024-09-19T08:42:28.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.497+0000 7ff069001640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff064073810 con 0x7ff064071ac0 2024-09-19T08:42:28.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.497+0000 7ff06259c640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7ff06406c1d0 0x7ff0640713b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41262/0 (socket says 172.21.15.98:41262) 2024-09-19T08:42:28.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.497+0000 7ff06259c640 1 -- 172.21.15.98:0/1130088233 learned_addr learned my addr 172.21.15.98:0/1130088233 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:28.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.497+0000 7ff06359e640 1 --2- 172.21.15.98:0/1130088233 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff064071ac0 0x7ff0640b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:28.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.497+0000 7ff062d9d640 1 --2- 172.21.15.98:0/1130088233 >> v2:172.21.15.139:3300/0 conn(0x7ff064145b70 0x7ff06406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:28.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.497+0000 7ff061d9b640 1 -- 172.21.15.98:0/1130088233 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1817278511 0 0) 0x7ff064073530 con 0x7ff06406c1d0 2024-09-19T08:42:28.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.497+0000 7ff061d9b640 1 -- 172.21.15.98:0/1130088233 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff04c003530 con 0x7ff06406c1d0 2024-09-19T08:42:28.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.497+0000 7ff061d9b640 1 -- 172.21.15.98:0/1130088233 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2167924150 0 0) 0x7ff04c003530 con 0x7ff06406c1d0 2024-09-19T08:42:28.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.497+0000 7ff061d9b640 1 -- 172.21.15.98:0/1130088233 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff064071ac0 msgr2=0x7ff0640b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:28.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.497+0000 7ff061d9b640 1 --2- 172.21.15.98:0/1130088233 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff064071ac0 0x7ff0640b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:28.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.497+0000 7ff061d9b640 1 -- 172.21.15.98:0/1130088233 >> v2:172.21.15.139:3300/0 conn(0x7ff064145b70 msgr2=0x7ff06406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:28.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.497+0000 7ff061d9b640 1 --2- 172.21.15.98:0/1130088233 >> v2:172.21.15.139:3300/0 conn(0x7ff064145b70 0x7ff06406bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:28.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.497+0000 7ff061d9b640 1 -- 172.21.15.98:0/1130088233 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff064073af0 con 0x7ff06406c1d0 2024-09-19T08:42:28.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.497+0000 7ff061d9b640 1 -- 172.21.15.98:0/1130088233 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7ff058001680 con 0x7ff06406c1d0 2024-09-19T08:42:28.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.497+0000 7ff061d9b640 1 -- 172.21.15.98:0/1130088233 >> v1:172.21.15.98:6789/0 conn(0x7ff06406c1d0 legacy=0x7ff0640713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:28.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.497+0000 7ff061d9b640 1 --2- 172.21.15.98:0/1130088233 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff064071ac0 0x7ff04c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:28.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.497+0000 7ff061d9b640 1 --2- 172.21.15.98:0/1130088233 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff04c003f40 0x7ff04c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:28.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.497+0000 7ff061d9b640 1 --2- 172.21.15.98:0/1130088233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff04c004d50 0x7ff04c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:28.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.497+0000 7ff061d9b640 1 -- 172.21.15.98:0/1130088233 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff06406c620 con 0x7ff04c004d50 2024-09-19T08:42:28.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.497+0000 7ff061d9b640 1 -- 172.21.15.98:0/1130088233 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff064073810 con 0x7ff04c003f40 2024-09-19T08:42:28.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.497+0000 7ff061d9b640 1 -- 172.21.15.98:0/1130088233 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff04c00bad0 con 0x7ff064071ac0 2024-09-19T08:42:28.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.497+0000 7ff062d9d640 1 --2- 172.21.15.98:0/1130088233 >> v2:172.21.15.139:3300/0 conn(0x7ff064145b70 0x7ff06406bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:42:28.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.497+0000 7ff06359e640 1 --2- 172.21.15.98:0/1130088233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff04c004d50 0x7ff04c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:28.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.497+0000 7ff06259c640 1 --2- 172.21.15.98:0/1130088233 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff04c003f40 0x7ff04c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:28.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.501+0000 7ff06359e640 1 -- 172.21.15.98:0/1130088233 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff064071ac0 msgr2=0x7ff04c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:28.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.501+0000 7ff06359e640 1 --2- 172.21.15.98:0/1130088233 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff064071ac0 0x7ff04c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:28.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.501+0000 7ff06359e640 1 -- 172.21.15.98:0/1130088233 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff04c003f40 msgr2=0x7ff04c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:28.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.501+0000 7ff06359e640 1 --2- 172.21.15.98:0/1130088233 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff04c003f40 0x7ff04c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:28.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.501+0000 7ff06359e640 1 -- 172.21.15.98:0/1130088233 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff04c003530 con 0x7ff04c004d50 2024-09-19T08:42:28.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.501+0000 7ff06359e640 1 --2- 172.21.15.98:0/1130088233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff04c004d50 0x7ff04c0051d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ff05c00e170 tx=0x7ff05c01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b7efae78d1ab3ed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:28.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.501+0000 7ff061d9b640 1 -- 172.21.15.98:0/1130088233 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c0b2070 con 0x7ff04c004d50 2024-09-19T08:42:28.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.501+0000 7ff061d9b640 1 -- 172.21.15.98:0/1130088233 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff05c0a5ba0 con 0x7ff04c004d50 2024-09-19T08:42:28.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.501+0000 7ff061d9b640 1 -- 172.21.15.98:0/1130088233 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c009c10 con 0x7ff04c004d50 2024-09-19T08:42:28.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.501+0000 7ff069001640 1 -- 172.21.15.98:0/1130088233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff04c004d50 msgr2=0x7ff04c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:28.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.501+0000 7ff069001640 1 --2- 172.21.15.98:0/1130088233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff04c004d50 0x7ff04c0051d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ff05c00e170 tx=0x7ff05c01cca0 comp rx=0 tx=0).stop 2024-09-19T08:42:28.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.501+0000 7ff06359e640 1 -- 172.21.15.98:0/1130088233 reap_dead start 2024-09-19T08:42:28.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.501+0000 7ff069001640 1 -- 172.21.15.98:0/1130088233 shutdown_connections 2024-09-19T08:42:28.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.501+0000 7ff069001640 1 -- 172.21.15.98:0/1130088233 >> 172.21.15.98:0/1130088233 conn(0x7ff0640b2010 msgr2=0x7ff0640b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:28.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.501+0000 7ff069001640 1 -- 172.21.15.98:0/1130088233 shutdown_connections 2024-09-19T08:42:28.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.501+0000 7ff069001640 1 -- 172.21.15.98:0/1130088233 wait complete. 2024-09-19T08:42:28.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.505+0000 7ff069001640 1 Processor -- start 2024-09-19T08:42:28.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.505+0000 7ff069001640 1 -- start start 2024-09-19T08:42:28.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.505+0000 7ff069001640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff04c004d50 0x7ff0640aad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:28.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.505+0000 7ff069001640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0640acb50 0x7ff06415da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:28.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.505+0000 7ff069001640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0640acfd0 0x7ff06413d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:28.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.505+0000 7ff069001640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff0641460d0 con 0x7ff04c004d50 2024-09-19T08:42:28.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.505+0000 7ff069001640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff064145f50 con 0x7ff0640acfd0 2024-09-19T08:42:28.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.505+0000 7ff069001640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff0640b3410 con 0x7ff0640acb50 2024-09-19T08:42:28.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.505+0000 7ff062d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff04c004d50 0x7ff0640aad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:28.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.505+0000 7ff062d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff04c004d50 0x7ff0640aad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34242/0 (socket says 172.21.15.98:34242) 2024-09-19T08:42:28.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.505+0000 7ff062d9d640 1 -- 172.21.15.98:0/3331535588 learned_addr learned my addr 172.21.15.98:0/3331535588 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:28.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.505+0000 7ff06359e640 1 --2- 172.21.15.98:0/3331535588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0640acfd0 0x7ff06413d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:28.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.505+0000 7ff062d9d640 1 -- 172.21.15.98:0/3331535588 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0640acb50 msgr2=0x7ff06415da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:28.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.505+0000 7ff062d9d640 1 --2- 172.21.15.98:0/3331535588 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0640acb50 0x7ff06415da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:28.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.505+0000 7ff062d9d640 1 -- 172.21.15.98:0/3331535588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0640acfd0 msgr2=0x7ff06413d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:28.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.505+0000 7ff062d9d640 1 --2- 172.21.15.98:0/3331535588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0640acfd0 0x7ff06413d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:28.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.505+0000 7ff062d9d640 1 -- 172.21.15.98:0/3331535588 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff06415ef00 con 0x7ff04c004d50 2024-09-19T08:42:28.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.505+0000 7ff06359e640 1 --2- 172.21.15.98:0/3331535588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0640acfd0 0x7ff06413d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:42:28.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.505+0000 7ff062d9d640 1 --2- 172.21.15.98:0/3331535588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff04c004d50 0x7ff0640aad80 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7ff05400e810 tx=0x7ff05400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:28.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.505+0000 7ff04b7fe640 1 -- 172.21.15.98:0/3331535588 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff054017070 con 0x7ff04c004d50 2024-09-19T08:42:28.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.505+0000 7ff04b7fe640 1 -- 172.21.15.98:0/3331535588 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff054004020 con 0x7ff04c004d50 2024-09-19T08:42:28.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.505+0000 7ff069001640 1 -- 172.21.15.98:0/3331535588 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff06415f1e0 con 0x7ff04c004d50 2024-09-19T08:42:28.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.505+0000 7ff04b7fe640 1 -- 172.21.15.98:0/3331535588 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05400f040 con 0x7ff04c004d50 2024-09-19T08:42:28.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.505+0000 7ff069001640 1 -- 172.21.15.98:0/3331535588 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff06415f7a0 con 0x7ff04c004d50 2024-09-19T08:42:28.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.505+0000 7ff069001640 1 -- 172.21.15.98:0/3331535588 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff064073320 con 0x7ff04c004d50 2024-09-19T08:42:28.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.505+0000 7ff04b7fe640 1 -- 172.21.15.98:0/3331535588 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7ff05400c6d0 con 0x7ff04c004d50 2024-09-19T08:42:28.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.505+0000 7ff04b7fe640 1 --2- 172.21.15.98:0/3331535588 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff050067c50 0x7ff05006a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:28.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.505+0000 7ff04b7fe640 1 -- 172.21.15.98:0/3331535588 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5227+0+0 (secure 0 0 0) 0x7ff0540860a0 con 0x7ff04c004d50 2024-09-19T08:42:28.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.505+0000 7ff06259c640 1 --2- 172.21.15.98:0/3331535588 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff050067c50 0x7ff05006a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:28.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.509+0000 7ff06259c640 1 --2- 172.21.15.98:0/3331535588 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff050067c50 0x7ff05006a110 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff058004880 tx=0x7ff058001ca0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:28.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.509+0000 7ff04b7fe640 1 -- 172.21.15.98:0/3331535588 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff0540569a0 con 0x7ff04c004d50 2024-09-19T08:42:28.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:28.677+0000 7ff069001640 1 -- 172.21.15.98:0/3331535588 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff0641402e0 con 0x7ff04c004d50 2024-09-19T08:42:29.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.345+0000 7ff04b7fe640 1 -- 172.21.15.98:0/3331535588 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v66) ==== 170+0+0 (secure 0 0 0) 0x7ff0540562a0 con 0x7ff04c004d50 2024-09-19T08:42:29.345 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' created 2024-09-19T08:42:29.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.349+0000 7ff069001640 1 -- 172.21.15.98:0/3331535588 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff050067c50 msgr2=0x7ff05006a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:29.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.349+0000 7ff069001640 1 --2- 172.21.15.98:0/3331535588 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff050067c50 0x7ff05006a110 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff058004880 tx=0x7ff058001ca0 comp rx=0 tx=0).stop 2024-09-19T08:42:29.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.349+0000 7ff069001640 1 -- 172.21.15.98:0/3331535588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff04c004d50 msgr2=0x7ff0640aad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:29.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.349+0000 7ff069001640 1 --2- 172.21.15.98:0/3331535588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff04c004d50 0x7ff0640aad80 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7ff05400e810 tx=0x7ff05400ecd0 comp rx=0 tx=0).stop 2024-09-19T08:42:29.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.349+0000 7ff069001640 1 -- 172.21.15.98:0/3331535588 shutdown_connections 2024-09-19T08:42:29.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.349+0000 7ff069001640 1 --2- 172.21.15.98:0/3331535588 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff050067c50 0x7ff05006a110 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:29.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.349+0000 7ff069001640 1 --2- 172.21.15.98:0/3331535588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff0640acfd0 0x7ff06413d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:29.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.349+0000 7ff069001640 1 --2- 172.21.15.98:0/3331535588 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff0640acb50 0x7ff06415da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:29.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.349+0000 7ff069001640 1 --2- 172.21.15.98:0/3331535588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff04c004d50 0x7ff0640aad80 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:29.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.349+0000 7ff069001640 1 -- 172.21.15.98:0/3331535588 >> 172.21.15.98:0/3331535588 conn(0x7ff0640b2010 msgr2=0x7ff064141030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:29.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.349+0000 7ff069001640 1 -- 172.21.15.98:0/3331535588 shutdown_connections 2024-09-19T08:42:29.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.349+0000 7ff069001640 1 -- 172.21.15.98:0/3331535588 wait complete. 2024-09-19T08:42:29.379 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T08:42:29.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.501+0000 7f1dc1065640 1 Processor -- start 2024-09-19T08:42:29.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.501+0000 7f1dc1065640 1 -- start start 2024-09-19T08:42:29.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.501+0000 7f1dc1065640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1dbc145b70 0x7f1dbc06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:29.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.501+0000 7f1dc1065640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f1dbc06c1d0 0x7f1dbc0713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:29.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.501+0000 7f1dc1065640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dbc06c5f0 con 0x7f1dbc0718f0 2024-09-19T08:42:29.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.501+0000 7f1dc1065640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f1dbc073530 con 0x7f1dbc06c1d0 2024-09-19T08:42:29.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.501+0000 7f1dc1065640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1dbc073810 con 0x7f1dbc145b70 2024-09-19T08:42:29.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.501+0000 7f1db259c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f1dbc06c1d0 0x7f1dbc0713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:29.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.501+0000 7f1dbb59e640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f1dbc0718f0 0x7f1dbc0cef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41276/0 (socket says 172.21.15.98:41276) 2024-09-19T08:42:29.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.501+0000 7f1dbb59e640 1 -- 172.21.15.98:0/3609789919 learned_addr learned my addr 172.21.15.98:0/3609789919 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:29.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.501+0000 7f1dbad9d640 1 --2- 172.21.15.98:0/3609789919 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1dbc145b70 0x7f1dbc06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:29.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.501+0000 7f1db259c640 1 -- 172.21.15.98:0/3609789919 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1dbc145b70 msgr2=0x7f1dbc06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:29.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.501+0000 7f1db259c640 1 --2- 172.21.15.98:0/3609789919 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1dbc145b70 0x7f1dbc06bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:29.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.501+0000 7f1db259c640 1 -- 172.21.15.98:0/3609789919 >> v1:172.21.15.98:6789/0 conn(0x7f1dbc0718f0 legacy=0x7f1dbc0cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:29.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.501+0000 7f1db259c640 1 -- 172.21.15.98:0/3609789919 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dbc073af0 con 0x7f1dbc06c1d0 2024-09-19T08:42:29.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.501+0000 7f1db259c640 1 --2- 172.21.15.98:0/3609789919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dbc06c1d0 0x7f1dbc0713b0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f1dac0097c0 tx=0x7f1dac01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34cf9510fd498362 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:29.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.505+0000 7f1dba59c640 1 -- 172.21.15.98:0/3609789919 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac0b0070 con 0x7f1dbc06c1d0 2024-09-19T08:42:29.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.505+0000 7f1dba59c640 1 -- 172.21.15.98:0/3609789919 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dac0ab600 con 0x7f1dbc06c1d0 2024-09-19T08:42:29.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.505+0000 7f1dba59c640 1 -- 172.21.15.98:0/3609789919 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac0ab9d0 con 0x7f1dbc06c1d0 2024-09-19T08:42:29.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.505+0000 7f1dc1065640 1 -- 172.21.15.98:0/3609789919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dbc06c1d0 msgr2=0x7f1dbc0713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:29.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.505+0000 7f1dc1065640 1 --2- 172.21.15.98:0/3609789919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dbc06c1d0 0x7f1dbc0713b0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f1dac0097c0 tx=0x7f1dac01c410 comp rx=0 tx=0).stop 2024-09-19T08:42:29.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.505+0000 7f1dc1065640 1 -- 172.21.15.98:0/3609789919 shutdown_connections 2024-09-19T08:42:29.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.505+0000 7f1dc1065640 1 --2- 172.21.15.98:0/3609789919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dbc06c1d0 0x7f1dbc0713b0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:29.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.505+0000 7f1dc1065640 1 --2- 172.21.15.98:0/3609789919 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1dbc145b70 0x7f1dbc06bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:29.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.505+0000 7f1dc1065640 1 -- 172.21.15.98:0/3609789919 >> 172.21.15.98:0/3609789919 conn(0x7f1dbc0b2010 msgr2=0x7f1dbc0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:29.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.505+0000 7f1dc1065640 1 -- 172.21.15.98:0/3609789919 shutdown_connections 2024-09-19T08:42:29.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.505+0000 7f1dc1065640 1 -- 172.21.15.98:0/3609789919 wait complete. 2024-09-19T08:42:29.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.505+0000 7f1dc1065640 1 Processor -- start 2024-09-19T08:42:29.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.505+0000 7f1dc1065640 1 -- start start 2024-09-19T08:42:29.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.505+0000 7f1dc1065640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1dbc06c1d0 0x7f1dbc1609e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:29.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.505+0000 7f1dc1065640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1dbc0718f0 0x7f1dbc13e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:29.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.505+0000 7f1dc1065640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dbc145b70 0x7f1dbc15ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:29.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.505+0000 7f1dc1065640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc169070 con 0x7f1dbc145b70 2024-09-19T08:42:29.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.505+0000 7f1dc1065640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc071f20 con 0x7f1dbc0718f0 2024-09-19T08:42:29.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.505+0000 7f1dc1065640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1dbc1691f0 con 0x7f1dbc06c1d0 2024-09-19T08:42:29.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.505+0000 7f1dbb59e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dbc145b70 0x7f1dbc15ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:29.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.505+0000 7f1dbb59e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dbc145b70 0x7f1dbc15ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34258/0 (socket says 172.21.15.98:34258) 2024-09-19T08:42:29.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.505+0000 7f1dbb59e640 1 -- 172.21.15.98:0/1530496010 learned_addr learned my addr 172.21.15.98:0/1530496010 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:29.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.505+0000 7f1dbad9d640 1 --2- 172.21.15.98:0/1530496010 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1dbc06c1d0 0x7f1dbc1609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:29.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.505+0000 7f1db259c640 1 --2- 172.21.15.98:0/1530496010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1dbc0718f0 0x7f1dbc13e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:29.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.505+0000 7f1dbb59e640 1 -- 172.21.15.98:0/1530496010 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1dbc06c1d0 msgr2=0x7f1dbc1609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:29.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.505+0000 7f1dbb59e640 1 --2- 172.21.15.98:0/1530496010 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1dbc06c1d0 0x7f1dbc1609e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:29.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.505+0000 7f1dbb59e640 1 -- 172.21.15.98:0/1530496010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1dbc0718f0 msgr2=0x7f1dbc13e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:29.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.505+0000 7f1dbb59e640 1 --2- 172.21.15.98:0/1530496010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1dbc0718f0 0x7f1dbc13e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:29.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.505+0000 7f1dbb59e640 1 -- 172.21.15.98:0/1530496010 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dbc160220 con 0x7f1dbc145b70 2024-09-19T08:42:29.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.505+0000 7f1db259c640 1 --2- 172.21.15.98:0/1530496010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1dbc0718f0 0x7f1dbc13e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:42:29.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.505+0000 7f1dbb59e640 1 --2- 172.21.15.98:0/1530496010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dbc145b70 0x7f1dbc15ba30 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f1db400caa0 tx=0x7f1db400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:29.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.505+0000 7f1db3fff640 1 -- 172.21.15.98:0/1530496010 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db4017070 con 0x7f1dbc145b70 2024-09-19T08:42:29.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.505+0000 7f1db3fff640 1 -- 172.21.15.98:0/1530496010 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db400ac00 con 0x7f1dbc145b70 2024-09-19T08:42:29.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.505+0000 7f1db3fff640 1 -- 172.21.15.98:0/1530496010 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db4012400 con 0x7f1dbc145b70 2024-09-19T08:42:29.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.509+0000 7f1dc1065640 1 -- 172.21.15.98:0/1530496010 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dbc13eda0 con 0x7f1dbc145b70 2024-09-19T08:42:29.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.509+0000 7f1dc1065640 1 -- 172.21.15.98:0/1530496010 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dbc13f670 con 0x7f1dbc145b70 2024-09-19T08:42:29.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.509+0000 7f1dc1065640 1 -- 172.21.15.98:0/1530496010 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dbc0713b0 con 0x7f1dbc145b70 2024-09-19T08:42:29.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.509+0000 7f1db3fff640 1 -- 172.21.15.98:0/1530496010 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f1db401a070 con 0x7f1dbc145b70 2024-09-19T08:42:29.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.509+0000 7f1db3fff640 1 --2- 172.21.15.98:0/1530496010 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f1d9c067c20 0x7f1d9c06a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:29.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.509+0000 7f1db3fff640 1 -- 172.21.15.98:0/1530496010 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5609+0+0 (secure 0 0 0) 0x7f1db4085e70 con 0x7f1dbc145b70 2024-09-19T08:42:29.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.509+0000 7f1dbad9d640 1 --2- 172.21.15.98:0/1530496010 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f1d9c067c20 0x7f1d9c06a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:29.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.509+0000 7f1dbad9d640 1 --2- 172.21.15.98:0/1530496010 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f1d9c067c20 0x7f1d9c06a0e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1da400b930 tx=0x7f1da4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:29.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.513+0000 7f1db3fff640 1 -- 172.21.15.98:0/1530496010 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f1db4056690 con 0x7f1dbc145b70 2024-09-19T08:42:29.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:29.713+0000 7f1dc1065640 1 -- 172.21.15.98:0/1530496010 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f1dbc0cc960 con 0x7f1dbc145b70 2024-09-19T08:42:30.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.409+0000 7f1db3fff640 1 -- 172.21.15.98:0/1530496010 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v36) ==== 174+0+0 (secure 0 0 0) 0x7f1db4055f90 con 0x7f1dbc145b70 2024-09-19T08:42:30.420 INFO:teuthology.orchestra.run.smithi098.stderr:new fs with metadata pool 10 and data pool 11 2024-09-19T08:42:30.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.425+0000 7f1dc1065640 1 -- 172.21.15.98:0/1530496010 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f1d9c067c20 msgr2=0x7f1d9c06a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:30.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.425+0000 7f1dc1065640 1 --2- 172.21.15.98:0/1530496010 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f1d9c067c20 0x7f1d9c06a0e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1da400b930 tx=0x7f1da4008040 comp rx=0 tx=0).stop 2024-09-19T08:42:30.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.425+0000 7f1dc1065640 1 -- 172.21.15.98:0/1530496010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dbc145b70 msgr2=0x7f1dbc15ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:30.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.425+0000 7f1dc1065640 1 --2- 172.21.15.98:0/1530496010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dbc145b70 0x7f1dbc15ba30 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f1db400caa0 tx=0x7f1db400cf60 comp rx=0 tx=0).stop 2024-09-19T08:42:30.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.425+0000 7f1dc1065640 1 -- 172.21.15.98:0/1530496010 shutdown_connections 2024-09-19T08:42:30.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.425+0000 7f1dc1065640 1 --2- 172.21.15.98:0/1530496010 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f1d9c067c20 0x7f1d9c06a0e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:30.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.425+0000 7f1dc1065640 1 --2- 172.21.15.98:0/1530496010 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dbc145b70 0x7f1dbc15ba30 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:30.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.425+0000 7f1dc1065640 1 --2- 172.21.15.98:0/1530496010 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1dbc0718f0 0x7f1dbc13e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:30.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.425+0000 7f1dc1065640 1 --2- 172.21.15.98:0/1530496010 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1dbc06c1d0 0x7f1dbc1609e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:30.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.425+0000 7f1dc1065640 1 -- 172.21.15.98:0/1530496010 >> 172.21.15.98:0/1530496010 conn(0x7f1dbc0b2010 msgr2=0x7f1dbc142840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:30.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.425+0000 7f1dc1065640 1 -- 172.21.15.98:0/1530496010 shutdown_connections 2024-09-19T08:42:30.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.425+0000 7f1dc1065640 1 -- 172.21.15.98:0/1530496010 wait complete. 2024-09-19T08:42:30.452 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:42:30.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.565+0000 7f6f740e5640 1 Processor -- start 2024-09-19T08:42:30.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.565+0000 7f6f740e5640 1 -- start start 2024-09-19T08:42:30.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.569+0000 7f6f740e5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6f6c15c550 0x7f6f6c06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:30.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.569+0000 7f6f740e5640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6f6c06c1d0 0x7f6f6c0cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:30.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.569+0000 7f6f740e5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f6c06d310 con 0x7f6f6c0cf900 2024-09-19T08:42:30.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.569+0000 7f6f740e5640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f6f6c06c640 con 0x7f6f6c06c1d0 2024-09-19T08:42:30.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.569+0000 7f6f740e5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f6f6c06d5f0 con 0x7f6f6c15c550 2024-09-19T08:42:30.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.569+0000 7f6f71680640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6f6c06c1d0 0x7f6f6c0cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:30.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.569+0000 7f6f71680640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6f6c06c1d0 0x7f6f6c0cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34260/0 (socket says 172.21.15.98:34260) 2024-09-19T08:42:30.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.569+0000 7f6f71680640 1 -- 172.21.15.98:0/2122377882 learned_addr learned my addr 172.21.15.98:0/2122377882 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:30.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.569+0000 7f6f71e81640 1 --2- 172.21.15.98:0/2122377882 >> v2:172.21.15.139:3300/0 conn(0x7f6f6c15c550 0x7f6f6c06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:30.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.569+0000 7f6f71680640 1 -- 172.21.15.98:0/2122377882 >> v2:172.21.15.139:3300/0 conn(0x7f6f6c15c550 msgr2=0x7f6f6c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:30.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.569+0000 7f6f71680640 1 --2- 172.21.15.98:0/2122377882 >> v2:172.21.15.139:3300/0 conn(0x7f6f6c15c550 0x7f6f6c06bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:30.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.569+0000 7f6f71680640 1 -- 172.21.15.98:0/2122377882 >> v1:172.21.15.139:6789/0 conn(0x7f6f6c0cf900 legacy=0x7f6f6c0ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:30.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.569+0000 7f6f71680640 1 -- 172.21.15.98:0/2122377882 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f6c06d8d0 con 0x7f6f6c06c1d0 2024-09-19T08:42:30.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.569+0000 7f6f71680640 1 --2- 172.21.15.98:0/2122377882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f6c06c1d0 0x7f6f6c0cf3c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c0b56b0 tx=0x7f6f6801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de2db04b9a84559 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:30.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.569+0000 7f6f70e7f640 1 -- 172.21.15.98:0/2122377882 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f680b0070 con 0x7f6f6c06c1d0 2024-09-19T08:42:30.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.569+0000 7f6f70e7f640 1 -- 172.21.15.98:0/2122377882 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f680ab600 con 0x7f6f6c06c1d0 2024-09-19T08:42:30.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.569+0000 7f6f70e7f640 1 -- 172.21.15.98:0/2122377882 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f680ab9d0 con 0x7f6f6c06c1d0 2024-09-19T08:42:30.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.573+0000 7f6f740e5640 1 -- 172.21.15.98:0/2122377882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f6c06c1d0 msgr2=0x7f6f6c0cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:30.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.573+0000 7f6f740e5640 1 --2- 172.21.15.98:0/2122377882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f6c06c1d0 0x7f6f6c0cf3c0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c0b56b0 tx=0x7f6f6801c4e0 comp rx=0 tx=0).stop 2024-09-19T08:42:30.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.573+0000 7f6f740e5640 1 -- 172.21.15.98:0/2122377882 shutdown_connections 2024-09-19T08:42:30.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.573+0000 7f6f740e5640 1 --2- 172.21.15.98:0/2122377882 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f6c06c1d0 0x7f6f6c0cf3c0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:30.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.573+0000 7f6f740e5640 1 --2- 172.21.15.98:0/2122377882 >> v2:172.21.15.139:3300/0 conn(0x7f6f6c15c550 0x7f6f6c06bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:30.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.573+0000 7f6f740e5640 1 -- 172.21.15.98:0/2122377882 >> 172.21.15.98:0/2122377882 conn(0x7f6f6c0b2010 msgr2=0x7f6f6c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:30.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.573+0000 7f6f740e5640 1 -- 172.21.15.98:0/2122377882 shutdown_connections 2024-09-19T08:42:30.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.573+0000 7f6f740e5640 1 -- 172.21.15.98:0/2122377882 wait complete. 2024-09-19T08:42:30.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.573+0000 7f6f740e5640 1 Processor -- start 2024-09-19T08:42:30.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.573+0000 7f6f740e5640 1 -- start start 2024-09-19T08:42:30.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.573+0000 7f6f740e5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f6c06c1d0 0x7f6f6c1516f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:30.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.573+0000 7f6f740e5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f6c0cf900 0x7f6f6c0ae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:30.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.573+0000 7f6f740e5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f6c15c550 0x7f6f6c1545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:30.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.573+0000 7f6f740e5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c1693b0 con 0x7f6f6c06c1d0 2024-09-19T08:42:30.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.573+0000 7f6f740e5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c169230 con 0x7f6f6c0cf900 2024-09-19T08:42:30.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.573+0000 7f6f740e5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6f6c169530 con 0x7f6f6c15c550 2024-09-19T08:42:30.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.573+0000 7f6f71e81640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f6c06c1d0 0x7f6f6c1516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:30.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.573+0000 7f6f71e81640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f6c06c1d0 0x7f6f6c1516f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34270/0 (socket says 172.21.15.98:34270) 2024-09-19T08:42:30.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.573+0000 7f6f71e81640 1 -- 172.21.15.98:0/126669238 learned_addr learned my addr 172.21.15.98:0/126669238 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:30.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.573+0000 7f6f71680640 1 --2- 172.21.15.98:0/126669238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f6c0cf900 0x7f6f6c0ae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:30.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.573+0000 7f6f72682640 1 --2- 172.21.15.98:0/126669238 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f6c15c550 0x7f6f6c1545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:30.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.573+0000 7f6f71e81640 1 -- 172.21.15.98:0/126669238 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f6c15c550 msgr2=0x7f6f6c1545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:30.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.573+0000 7f6f71e81640 1 --2- 172.21.15.98:0/126669238 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f6c15c550 0x7f6f6c1545f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:30.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.573+0000 7f6f71e81640 1 -- 172.21.15.98:0/126669238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f6c0cf900 msgr2=0x7f6f6c0ae1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:30.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.573+0000 7f6f71e81640 1 --2- 172.21.15.98:0/126669238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f6c0cf900 0x7f6f6c0ae1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:30.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.573+0000 7f6f71e81640 1 -- 172.21.15.98:0/126669238 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f6c151f40 con 0x7f6f6c06c1d0 2024-09-19T08:42:30.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.573+0000 7f6f71e81640 1 --2- 172.21.15.98:0/126669238 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f6c06c1d0 0x7f6f6c1516f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c00c7f0 tx=0x7f6f5c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:30.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.573+0000 7f6f627fc640 1 -- 172.21.15.98:0/126669238 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c015070 con 0x7f6f6c06c1d0 2024-09-19T08:42:30.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.573+0000 7f6f627fc640 1 -- 172.21.15.98:0/126669238 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f5c004240 con 0x7f6f6c06c1d0 2024-09-19T08:42:30.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.573+0000 7f6f740e5640 1 -- 172.21.15.98:0/126669238 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f6c152220 con 0x7f6f6c06c1d0 2024-09-19T08:42:30.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.573+0000 7f6f627fc640 1 -- 172.21.15.98:0/126669238 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c003620 con 0x7f6f6c06c1d0 2024-09-19T08:42:30.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.573+0000 7f6f740e5640 1 -- 172.21.15.98:0/126669238 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f6c1527e0 con 0x7f6f6c06c1d0 2024-09-19T08:42:30.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.577+0000 7f6f740e5640 1 -- 172.21.15.98:0/126669238 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f6c0cf3c0 con 0x7f6f6c06c1d0 2024-09-19T08:42:30.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.577+0000 7f6f627fc640 1 -- 172.21.15.98:0/126669238 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f6f5c0039d0 con 0x7f6f6c06c1d0 2024-09-19T08:42:30.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.577+0000 7f6f627fc640 1 --2- 172.21.15.98:0/126669238 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6f50067c20 0x7f6f5006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:30.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.577+0000 7f6f627fc640 1 -- 172.21.15.98:0/126669238 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f6f5c0863b0 con 0x7f6f6c06c1d0 2024-09-19T08:42:30.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.577+0000 7f6f71680640 1 --2- 172.21.15.98:0/126669238 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6f50067c20 0x7f6f5006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:30.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.577+0000 7f6f71680640 1 --2- 172.21.15.98:0/126669238 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6f50067c20 0x7f6f5006a0e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6f680ad000 tx=0x7f6f68008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:30.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.581+0000 7f6f627fc640 1 -- 172.21.15.98:0/126669238 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6f5c0569d0 con 0x7f6f6c06c1d0 2024-09-19T08:42:30.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.745+0000 7f6f740e5640 1 -- 172.21.15.98:0/126669238 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6f6c0ce4e0 con 0x7f6f6c06c1d0 2024-09-19T08:42:30.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.745+0000 7f6f627fc640 1 -- 172.21.15.98:0/126669238 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) ==== 74+0+16741 (secure 0 0 0) 0x7f6f5c0562d0 con 0x7f6f6c06c1d0 2024-09-19T08:42:30.747 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:42:30.747 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":67,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:42:29.720108+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:42:27.733851+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-19T08:42:28.717566+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:42:30.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.749+0000 7f6f740e5640 1 -- 172.21.15.98:0/126669238 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6f50067c20 msgr2=0x7f6f5006a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:30.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.749+0000 7f6f740e5640 1 --2- 172.21.15.98:0/126669238 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6f50067c20 0x7f6f5006a0e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6f680ad000 tx=0x7f6f68008040 comp rx=0 tx=0).stop 2024-09-19T08:42:30.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.749+0000 7f6f740e5640 1 -- 172.21.15.98:0/126669238 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f6c06c1d0 msgr2=0x7f6f6c1516f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:30.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.749+0000 7f6f740e5640 1 --2- 172.21.15.98:0/126669238 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f6c06c1d0 0x7f6f6c1516f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c00c7f0 tx=0x7f6f5c00ccb0 comp rx=0 tx=0).stop 2024-09-19T08:42:30.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.749+0000 7f6f740e5640 1 -- 172.21.15.98:0/126669238 shutdown_connections 2024-09-19T08:42:30.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.749+0000 7f6f740e5640 1 --2- 172.21.15.98:0/126669238 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6f50067c20 0x7f6f5006a0e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:30.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.749+0000 7f6f740e5640 1 --2- 172.21.15.98:0/126669238 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f6c15c550 0x7f6f6c1545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:30.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.749+0000 7f6f740e5640 1 --2- 172.21.15.98:0/126669238 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f6c0cf900 0x7f6f6c0ae1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:30.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.749+0000 7f6f740e5640 1 --2- 172.21.15.98:0/126669238 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f6c06c1d0 0x7f6f6c1516f0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:30.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.749+0000 7f6f740e5640 1 -- 172.21.15.98:0/126669238 >> 172.21.15.98:0/126669238 conn(0x7f6f6c0b2010 msgr2=0x7f6f6c0b0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:30.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.749+0000 7f6f740e5640 1 -- 172.21.15.98:0/126669238 shutdown_connections 2024-09-19T08:42:30.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.749+0000 7f6f740e5640 1 -- 172.21.15.98:0/126669238 wait complete. 2024-09-19T08:42:30.766 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:42:30.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.869+0000 7f9486f57640 1 Processor -- start 2024-09-19T08:42:30.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.869+0000 7f9486f57640 1 -- start start 2024-09-19T08:42:30.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.873+0000 7f9486f57640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f94800b7f80 0x7f948015f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:30.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.873+0000 7f9486f57640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94800b9970 0x7f94800b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:30.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.873+0000 7f9486f57640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9480068440 con 0x7f94800d4820 2024-09-19T08:42:30.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.873+0000 7f9486f57640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f9480163050 con 0x7f94800b7f80 2024-09-19T08:42:30.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.873+0000 7f9486f57640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9480162a90 con 0x7f94800b9970 2024-09-19T08:42:30.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.873+0000 7f94854f4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94800b9970 0x7f94800b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:30.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.873+0000 7f9484cf3640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f94800b7f80 0x7f948015f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:30.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.873+0000 7f94854f4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94800b9970 0x7f94800b9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:46306/0 (socket says 172.21.15.98:46306) 2024-09-19T08:42:30.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.873+0000 7f94854f4640 1 -- 172.21.15.98:0/3854708915 learned_addr learned my addr 172.21.15.98:0/3854708915 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:30.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.873+0000 7f947ffff640 1 --1- 172.21.15.98:0/3854708915 >> v1:172.21.15.139:6789/0 conn(0x7f94800d4820 0x7f94801621b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:52736/0 (socket says 172.21.15.98:52736) 2024-09-19T08:42:30.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.873+0000 7f94854f4640 1 -- 172.21.15.98:0/3854708915 >> v2:172.21.15.139:3300/0 conn(0x7f94800b7f80 msgr2=0x7f948015f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:30.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.873+0000 7f94854f4640 1 --2- 172.21.15.98:0/3854708915 >> v2:172.21.15.139:3300/0 conn(0x7f94800b7f80 0x7f948015f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:30.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.873+0000 7f94854f4640 1 -- 172.21.15.98:0/3854708915 >> v1:172.21.15.139:6789/0 conn(0x7f94800d4820 legacy=0x7f94801621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:30.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.873+0000 7f94854f4640 1 -- 172.21.15.98:0/3854708915 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9480162d70 con 0x7f94800b9970 2024-09-19T08:42:30.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.873+0000 7f94854f4640 1 --2- 172.21.15.98:0/3854708915 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94800b9970 0x7f94800b9de0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9470009870 tx=0x7f947001c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c48a6f57a7014fe2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:30.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.873+0000 7f947f7fe640 1 -- 172.21.15.98:0/3854708915 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94700b2070 con 0x7f94800b9970 2024-09-19T08:42:30.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.873+0000 7f947f7fe640 1 -- 172.21.15.98:0/3854708915 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94700a9070 con 0x7f94800b9970 2024-09-19T08:42:30.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.877+0000 7f947f7fe640 1 -- 172.21.15.98:0/3854708915 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94700ad880 con 0x7f94800b9970 2024-09-19T08:42:30.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.877+0000 7f9486f57640 1 -- 172.21.15.98:0/3854708915 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94800b9970 msgr2=0x7f94800b9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:30.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.877+0000 7f9486f57640 1 --2- 172.21.15.98:0/3854708915 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94800b9970 0x7f94800b9de0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9470009870 tx=0x7f947001c560 comp rx=0 tx=0).stop 2024-09-19T08:42:30.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.877+0000 7f9486f57640 1 -- 172.21.15.98:0/3854708915 shutdown_connections 2024-09-19T08:42:30.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.877+0000 7f9486f57640 1 --2- 172.21.15.98:0/3854708915 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94800b9970 0x7f94800b9de0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:30.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.877+0000 7f9486f57640 1 --2- 172.21.15.98:0/3854708915 >> v2:172.21.15.139:3300/0 conn(0x7f94800b7f80 0x7f948015f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:30.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.877+0000 7f9486f57640 1 -- 172.21.15.98:0/3854708915 >> 172.21.15.98:0/3854708915 conn(0x7f9480074e40 msgr2=0x7f9480075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:30.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.877+0000 7f9486f57640 1 -- 172.21.15.98:0/3854708915 shutdown_connections 2024-09-19T08:42:30.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.877+0000 7f9486f57640 1 -- 172.21.15.98:0/3854708915 wait complete. 2024-09-19T08:42:30.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.877+0000 7f9486f57640 1 Processor -- start 2024-09-19T08:42:30.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.877+0000 7f9486f57640 1 -- start start 2024-09-19T08:42:30.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.877+0000 7f9486f57640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94800b7f80 0x7f948013f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:30.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.877+0000 7f9486f57640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94800b9970 0x7f9480151250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:30.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.877+0000 7f9486f57640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94800d4820 0x7f9480151790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:30.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.877+0000 7f9486f57640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f94800b84e0 con 0x7f94800b9970 2024-09-19T08:42:30.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.877+0000 7f9486f57640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f94800b8360 con 0x7f94800b7f80 2024-09-19T08:42:30.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.877+0000 7f9486f57640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9480144590 con 0x7f94800d4820 2024-09-19T08:42:30.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.877+0000 7f947ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94800b9970 0x7f9480151250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:30.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.877+0000 7f947ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94800b9970 0x7f9480151250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34284/0 (socket says 172.21.15.98:34284) 2024-09-19T08:42:30.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.877+0000 7f947ffff640 1 -- 172.21.15.98:0/853420085 learned_addr learned my addr 172.21.15.98:0/853420085 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:30.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.877+0000 7f9484cf3640 1 --2- 172.21.15.98:0/853420085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94800b7f80 0x7f948013f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:30.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.877+0000 7f94854f4640 1 --2- 172.21.15.98:0/853420085 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94800d4820 0x7f9480151790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:30.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.877+0000 7f947ffff640 1 -- 172.21.15.98:0/853420085 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94800d4820 msgr2=0x7f9480151790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:30.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.877+0000 7f947ffff640 1 --2- 172.21.15.98:0/853420085 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94800d4820 0x7f9480151790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:30.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.877+0000 7f947ffff640 1 -- 172.21.15.98:0/853420085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94800b7f80 msgr2=0x7f948013f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:30.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.877+0000 7f947ffff640 1 --2- 172.21.15.98:0/853420085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94800b7f80 0x7f948013f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:30.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.877+0000 7f947ffff640 1 -- 172.21.15.98:0/853420085 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9480140300 con 0x7f94800b9970 2024-09-19T08:42:30.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.877+0000 7f9484cf3640 1 --2- 172.21.15.98:0/853420085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94800b7f80 0x7f948013f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:42:30.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.877+0000 7f947ffff640 1 --2- 172.21.15.98:0/853420085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94800b9970 0x7f9480151250 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f947400c790 tx=0x7f947400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:30.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.877+0000 7f947d7fa640 1 -- 172.21.15.98:0/853420085 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9474017070 con 0x7f94800b9970 2024-09-19T08:42:30.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.877+0000 7f947d7fa640 1 -- 172.21.15.98:0/853420085 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f947400a640 con 0x7f94800b9970 2024-09-19T08:42:30.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.877+0000 7f9486f57640 1 -- 172.21.15.98:0/853420085 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94801405e0 con 0x7f94800b9970 2024-09-19T08:42:30.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.877+0000 7f947d7fa640 1 -- 172.21.15.98:0/853420085 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947400aab0 con 0x7f94800b9970 2024-09-19T08:42:30.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.877+0000 7f9486f57640 1 -- 172.21.15.98:0/853420085 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f948013fe30 con 0x7f94800b9970 2024-09-19T08:42:30.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.881+0000 7f9486f57640 1 -- 172.21.15.98:0/853420085 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9480068f50 con 0x7f94800b9970 2024-09-19T08:42:30.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.881+0000 7f947d7fa640 1 -- 172.21.15.98:0/853420085 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f947401a070 con 0x7f94800b9970 2024-09-19T08:42:30.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.881+0000 7f947d7fa640 1 --2- 172.21.15.98:0/853420085 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f946c067c20 0x7f946c06a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:30.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.881+0000 7f947d7fa640 1 -- 172.21.15.98:0/853420085 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f9474085c30 con 0x7f94800b9970 2024-09-19T08:42:30.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.881+0000 7f9484cf3640 1 --2- 172.21.15.98:0/853420085 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f946c067c20 0x7f946c06a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:30.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.881+0000 7f9484cf3640 1 --2- 172.21.15.98:0/853420085 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f946c067c20 0x7f946c06a0e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9478009a40 tx=0x7f9478008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:30.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:30.885+0000 7f947d7fa640 1 -- 172.21.15.98:0/853420085 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9474021020 con 0x7f94800b9970 2024-09-19T08:42:31.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.045+0000 7f9486f57640 1 -- 172.21.15.98:0/853420085 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f94801448c0 con 0x7f94800b9970 2024-09-19T08:42:31.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.049+0000 7f947d7fa640 1 -- 172.21.15.98:0/853420085 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) ==== 74+0+16741 (secure 0 0 0) 0x7f94740563e0 con 0x7f94800b9970 2024-09-19T08:42:31.050 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:42:31.050 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":67,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:42:29.720108+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:42:27.733851+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-19T08:42:28.717566+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:42:31.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.049+0000 7f9486f57640 1 -- 172.21.15.98:0/853420085 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f946c067c20 msgr2=0x7f946c06a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:31.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.049+0000 7f9486f57640 1 --2- 172.21.15.98:0/853420085 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f946c067c20 0x7f946c06a0e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9478009a40 tx=0x7f9478008040 comp rx=0 tx=0).stop 2024-09-19T08:42:31.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.049+0000 7f9486f57640 1 -- 172.21.15.98:0/853420085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94800b9970 msgr2=0x7f9480151250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:31.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.049+0000 7f9486f57640 1 --2- 172.21.15.98:0/853420085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94800b9970 0x7f9480151250 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f947400c790 tx=0x7f947400cc50 comp rx=0 tx=0).stop 2024-09-19T08:42:31.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.049+0000 7f9486f57640 1 -- 172.21.15.98:0/853420085 shutdown_connections 2024-09-19T08:42:31.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.049+0000 7f9486f57640 1 --2- 172.21.15.98:0/853420085 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f946c067c20 0x7f946c06a0e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:31.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.049+0000 7f9486f57640 1 --2- 172.21.15.98:0/853420085 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f94800d4820 0x7f9480151790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:31.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.049+0000 7f9486f57640 1 --2- 172.21.15.98:0/853420085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f94800b9970 0x7f9480151250 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:31.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.049+0000 7f9486f57640 1 --2- 172.21.15.98:0/853420085 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f94800b7f80 0x7f948013f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:31.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.049+0000 7f9486f57640 1 -- 172.21.15.98:0/853420085 >> 172.21.15.98:0/853420085 conn(0x7f9480074e40 msgr2=0x7f94800d3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:31.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.049+0000 7f9486f57640 1 -- 172.21.15.98:0/853420085 shutdown_connections 2024-09-19T08:42:31.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.053+0000 7f9486f57640 1 -- 172.21.15.98:0/853420085 wait complete. 2024-09-19T08:42:31.068 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-19T08:42:31.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.181+0000 7fa225122640 1 Processor -- start 2024-09-19T08:42:31.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.181+0000 7fa225122640 1 -- start start 2024-09-19T08:42:31.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.181+0000 7fa225122640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa2200ceaa0 0x7fa220142630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:31.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.181+0000 7fa225122640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa22006cca0 0x7fa22006d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:31.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.181+0000 7fa225122640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa220141b70 con 0x7fa220145b50 2024-09-19T08:42:31.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.181+0000 7fa225122640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fa22013f860 con 0x7fa2200ceaa0 2024-09-19T08:42:31.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.181+0000 7fa225122640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa2200d0e40 con 0x7fa22006cca0 2024-09-19T08:42:31.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.181+0000 7fa21f59e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa22006cca0 0x7fa22006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:31.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.181+0000 7fa21ed9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa2200ceaa0 0x7fa220142630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:31.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.181+0000 7fa21f59e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa22006cca0 0x7fa22006d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:46324/0 (socket says 172.21.15.98:46324) 2024-09-19T08:42:31.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.181+0000 7fa21f59e640 1 -- 172.21.15.98:0/187708823 learned_addr learned my addr 172.21.15.98:0/187708823 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:31.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.181+0000 7fa21f59e640 1 -- 172.21.15.98:0/187708823 >> v2:172.21.15.139:3300/0 conn(0x7fa2200ceaa0 msgr2=0x7fa220142630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:31.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.181+0000 7fa21f59e640 1 --2- 172.21.15.98:0/187708823 >> v2:172.21.15.139:3300/0 conn(0x7fa2200ceaa0 0x7fa220142630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:31.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.181+0000 7fa21f59e640 1 -- 172.21.15.98:0/187708823 >> v1:172.21.15.139:6789/0 conn(0x7fa220145b50 legacy=0x7fa2200d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:31.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.181+0000 7fa21f59e640 1 -- 172.21.15.98:0/187708823 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2200d1120 con 0x7fa22006cca0 2024-09-19T08:42:31.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.185+0000 7fa21f59e640 1 --2- 172.21.15.98:0/187708823 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa22006cca0 0x7fa22006d150 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa218009e50 tx=0x7fa21801ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=989debdc6d5a6b54 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:31.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.185+0000 7fa21dd9b640 1 -- 172.21.15.98:0/187708823 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2180a57f0 con 0x7fa22006cca0 2024-09-19T08:42:31.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.185+0000 7fa21dd9b640 1 -- 172.21.15.98:0/187708823 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2180a5990 con 0x7fa22006cca0 2024-09-19T08:42:31.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.185+0000 7fa21dd9b640 1 -- 172.21.15.98:0/187708823 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2180a5d60 con 0x7fa22006cca0 2024-09-19T08:42:31.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.185+0000 7fa225122640 1 -- 172.21.15.98:0/187708823 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa22006cca0 msgr2=0x7fa22006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:31.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.185+0000 7fa225122640 1 --2- 172.21.15.98:0/187708823 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa22006cca0 0x7fa22006d150 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa218009e50 tx=0x7fa21801ca30 comp rx=0 tx=0).stop 2024-09-19T08:42:31.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.185+0000 7fa225122640 1 -- 172.21.15.98:0/187708823 shutdown_connections 2024-09-19T08:42:31.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.185+0000 7fa225122640 1 --2- 172.21.15.98:0/187708823 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa22006cca0 0x7fa22006d150 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:31.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.185+0000 7fa225122640 1 --2- 172.21.15.98:0/187708823 >> v2:172.21.15.139:3300/0 conn(0x7fa2200ceaa0 0x7fa220142630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:31.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.185+0000 7fa225122640 1 -- 172.21.15.98:0/187708823 >> 172.21.15.98:0/187708823 conn(0x7fa2200720a0 msgr2=0x7fa2200724b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:31.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.185+0000 7fa225122640 1 -- 172.21.15.98:0/187708823 shutdown_connections 2024-09-19T08:42:31.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.185+0000 7fa225122640 1 -- 172.21.15.98:0/187708823 wait complete. 2024-09-19T08:42:31.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.185+0000 7fa225122640 1 Processor -- start 2024-09-19T08:42:31.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.185+0000 7fa225122640 1 -- start start 2024-09-19T08:42:31.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.185+0000 7fa225122640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa22006cca0 0x7fa220144c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:31.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.185+0000 7fa225122640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2200ceaa0 0x7fa22013d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:31.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.185+0000 7fa225122640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa220145b50 0x7fa2201594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:31.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.185+0000 7fa225122640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa2200cf000 con 0x7fa2200ceaa0 2024-09-19T08:42:31.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.185+0000 7fa225122640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa2200cee80 con 0x7fa220145b50 2024-09-19T08:42:31.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.185+0000 7fa225122640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa2201606d0 con 0x7fa22006cca0 2024-09-19T08:42:31.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.185+0000 7fa21e59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2200ceaa0 0x7fa22013d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:31.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.185+0000 7fa21e59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2200ceaa0 0x7fa22013d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34296/0 (socket says 172.21.15.98:34296) 2024-09-19T08:42:31.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.185+0000 7fa21e59c640 1 -- 172.21.15.98:0/2423272945 learned_addr learned my addr 172.21.15.98:0/2423272945 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:31.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.185+0000 7fa21ed9d640 1 --2- 172.21.15.98:0/2423272945 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa22006cca0 0x7fa220144c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:31.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.185+0000 7fa21f59e640 1 --2- 172.21.15.98:0/2423272945 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa220145b50 0x7fa2201594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:31.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.185+0000 7fa21e59c640 1 -- 172.21.15.98:0/2423272945 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa22006cca0 msgr2=0x7fa220144c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:31.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.185+0000 7fa21e59c640 1 --2- 172.21.15.98:0/2423272945 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa22006cca0 0x7fa220144c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:31.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.185+0000 7fa21e59c640 1 -- 172.21.15.98:0/2423272945 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa220145b50 msgr2=0x7fa2201594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:31.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.185+0000 7fa21e59c640 1 --2- 172.21.15.98:0/2423272945 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa220145b50 0x7fa2201594c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:31.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.185+0000 7fa21e59c640 1 -- 172.21.15.98:0/2423272945 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa220158320 con 0x7fa2200ceaa0 2024-09-19T08:42:31.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.189+0000 7fa21e59c640 1 --2- 172.21.15.98:0/2423272945 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2200ceaa0 0x7fa22013d6b0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa21400c3a0 tx=0x7fa21400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:31.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.189+0000 7fa2077fe640 1 -- 172.21.15.98:0/2423272945 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2140052d0 con 0x7fa2200ceaa0 2024-09-19T08:42:31.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.189+0000 7fa2077fe640 1 -- 172.21.15.98:0/2423272945 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa214005470 con 0x7fa2200ceaa0 2024-09-19T08:42:31.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.189+0000 7fa225122640 1 -- 172.21.15.98:0/2423272945 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa220158600 con 0x7fa2200ceaa0 2024-09-19T08:42:31.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.189+0000 7fa2077fe640 1 -- 172.21.15.98:0/2423272945 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa214007260 con 0x7fa2200ceaa0 2024-09-19T08:42:31.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.189+0000 7fa225122640 1 -- 172.21.15.98:0/2423272945 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa220158bc0 con 0x7fa2200ceaa0 2024-09-19T08:42:31.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.189+0000 7fa225122640 1 -- 172.21.15.98:0/2423272945 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa220160ae0 con 0x7fa2200ceaa0 2024-09-19T08:42:31.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.189+0000 7fa2077fe640 1 -- 172.21.15.98:0/2423272945 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fa214007400 con 0x7fa2200ceaa0 2024-09-19T08:42:31.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.189+0000 7fa2077fe640 1 --2- 172.21.15.98:0/2423272945 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa20c067c20 0x7fa20c06a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:31.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.189+0000 7fa21ed9d640 1 --2- 172.21.15.98:0/2423272945 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa20c067c20 0x7fa20c06a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:31.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.189+0000 7fa2077fe640 1 -- 172.21.15.98:0/2423272945 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7fa214085630 con 0x7fa2200ceaa0 2024-09-19T08:42:31.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.189+0000 7fa21ed9d640 1 --2- 172.21.15.98:0/2423272945 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa20c067c20 0x7fa20c06a0e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa210009af0 tx=0x7fa210008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:31.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.193+0000 7fa2077fe640 1 -- 172.21.15.98:0/2423272945 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa214055d70 con 0x7fa2200ceaa0 2024-09-19T08:42:31.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.377+0000 7fa225122640 1 -- 172.21.15.98:0/2423272945 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa22013f2f0 con 0x7fa2200ceaa0 2024-09-19T08:42:31.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.409+0000 7fa2077fe640 1 -- 172.21.15.98:0/2423272945 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v37) ==== 118+0+0 (secure 0 0 0) 0x7fa214055670 con 0x7fa2200ceaa0 2024-09-19T08:42:31.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.413+0000 7fa225122640 1 -- 172.21.15.98:0/2423272945 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa20c067c20 msgr2=0x7fa20c06a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:31.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.413+0000 7fa225122640 1 --2- 172.21.15.98:0/2423272945 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa20c067c20 0x7fa20c06a0e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa210009af0 tx=0x7fa210008040 comp rx=0 tx=0).stop 2024-09-19T08:42:31.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.413+0000 7fa225122640 1 -- 172.21.15.98:0/2423272945 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2200ceaa0 msgr2=0x7fa22013d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:31.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.413+0000 7fa225122640 1 --2- 172.21.15.98:0/2423272945 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2200ceaa0 0x7fa22013d6b0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa21400c3a0 tx=0x7fa21400c860 comp rx=0 tx=0).stop 2024-09-19T08:42:31.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.413+0000 7fa225122640 1 -- 172.21.15.98:0/2423272945 shutdown_connections 2024-09-19T08:42:31.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.413+0000 7fa225122640 1 --2- 172.21.15.98:0/2423272945 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fa20c067c20 0x7fa20c06a0e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:31.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.413+0000 7fa225122640 1 --2- 172.21.15.98:0/2423272945 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa220145b50 0x7fa2201594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:31.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.413+0000 7fa225122640 1 --2- 172.21.15.98:0/2423272945 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2200ceaa0 0x7fa22013d6b0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:31.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.413+0000 7fa225122640 1 --2- 172.21.15.98:0/2423272945 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa22006cca0 0x7fa220144c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:31.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.413+0000 7fa225122640 1 -- 172.21.15.98:0/2423272945 >> 172.21.15.98:0/2423272945 conn(0x7fa2200720a0 msgr2=0x7fa22006f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:31.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.413+0000 7fa225122640 1 -- 172.21.15.98:0/2423272945 shutdown_connections 2024-09-19T08:42:31.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.413+0000 7fa225122640 1 -- 172.21.15.98:0/2423272945 wait complete. 2024-09-19T08:42:31.431 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-19T08:42:31.431 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-19T08:42:31.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.545+0000 7f06f728e640 1 Processor -- start 2024-09-19T08:42:31.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.545+0000 7f06f728e640 1 -- start start 2024-09-19T08:42:31.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.545+0000 7f06f728e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06f0145b70 0x7f06f006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:31.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.545+0000 7f06f728e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f06f006c1d0 0x7f06f00713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:31.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.545+0000 7f06f728e640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06f006c5f0 con 0x7f06f00718f0 2024-09-19T08:42:31.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.545+0000 7f06f728e640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f06f0073530 con 0x7f06f006c1d0 2024-09-19T08:42:31.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.545+0000 7f06f728e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f06f0073810 con 0x7f06f0145b70 2024-09-19T08:42:31.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.545+0000 7f06f4829640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f06f006c1d0 0x7f06f00713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:31.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.545+0000 7f06f582b640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f06f00718f0 0x7f06f00cef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41282/0 (socket says 172.21.15.98:41282) 2024-09-19T08:42:31.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.545+0000 7f06f582b640 1 -- 172.21.15.98:0/1346205692 learned_addr learned my addr 172.21.15.98:0/1346205692 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:31.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.545+0000 7f06f502a640 1 --2- 172.21.15.98:0/1346205692 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06f0145b70 0x7f06f006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:31.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.545+0000 7f06f4829640 1 -- 172.21.15.98:0/1346205692 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06f0145b70 msgr2=0x7f06f006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:31.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.545+0000 7f06f4829640 1 --2- 172.21.15.98:0/1346205692 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06f0145b70 0x7f06f006bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:31.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.545+0000 7f06f4829640 1 -- 172.21.15.98:0/1346205692 >> v1:172.21.15.98:6789/0 conn(0x7f06f00718f0 legacy=0x7f06f00cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:31.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.545+0000 7f06f4829640 1 -- 172.21.15.98:0/1346205692 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f0073af0 con 0x7f06f006c1d0 2024-09-19T08:42:31.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.549+0000 7f06f4829640 1 --2- 172.21.15.98:0/1346205692 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06f006c1d0 0x7f06f00713b0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f06e00097c0 tx=0x7f06e001c2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8dffdf3efd1f2a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:31.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.549+0000 7f06e7fff640 1 -- 172.21.15.98:0/1346205692 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e00b0070 con 0x7f06f006c1d0 2024-09-19T08:42:31.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.549+0000 7f06e7fff640 1 -- 172.21.15.98:0/1346205692 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06e00ab600 con 0x7f06f006c1d0 2024-09-19T08:42:31.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.549+0000 7f06e7fff640 1 -- 172.21.15.98:0/1346205692 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e00ab9d0 con 0x7f06f006c1d0 2024-09-19T08:42:31.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.549+0000 7f06f728e640 1 -- 172.21.15.98:0/1346205692 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06f006c1d0 msgr2=0x7f06f00713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:31.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.549+0000 7f06f728e640 1 --2- 172.21.15.98:0/1346205692 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06f006c1d0 0x7f06f00713b0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f06e00097c0 tx=0x7f06e001c2a0 comp rx=0 tx=0).stop 2024-09-19T08:42:31.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.549+0000 7f06f728e640 1 -- 172.21.15.98:0/1346205692 shutdown_connections 2024-09-19T08:42:31.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.549+0000 7f06f728e640 1 --2- 172.21.15.98:0/1346205692 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06f006c1d0 0x7f06f00713b0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:31.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.549+0000 7f06f728e640 1 --2- 172.21.15.98:0/1346205692 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06f0145b70 0x7f06f006bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:31.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.549+0000 7f06f728e640 1 -- 172.21.15.98:0/1346205692 >> 172.21.15.98:0/1346205692 conn(0x7f06f00b2010 msgr2=0x7f06f00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:31.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.549+0000 7f06f728e640 1 -- 172.21.15.98:0/1346205692 shutdown_connections 2024-09-19T08:42:31.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.549+0000 7f06f728e640 1 -- 172.21.15.98:0/1346205692 wait complete. 2024-09-19T08:42:31.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.549+0000 7f06f728e640 1 Processor -- start 2024-09-19T08:42:31.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.549+0000 7f06f728e640 1 -- start start 2024-09-19T08:42:31.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.553+0000 7f06f728e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06f006c1d0 0x7f06f0154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:31.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.553+0000 7f06f728e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06f00718f0 0x7f06f0141200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:31.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.553+0000 7f06f728e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06f0145b70 0x7f06f0141740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:31.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.553+0000 7f06f728e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f06f0071ed0 con 0x7f06f006c1d0 2024-09-19T08:42:31.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.553+0000 7f06f728e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f06f0071d50 con 0x7f06f0145b70 2024-09-19T08:42:31.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.553+0000 7f06f728e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f06f0145f50 con 0x7f06f00718f0 2024-09-19T08:42:31.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.553+0000 7f06f502a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06f006c1d0 0x7f06f0154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:31.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.553+0000 7f06f502a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06f006c1d0 0x7f06f0154d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34308/0 (socket says 172.21.15.98:34308) 2024-09-19T08:42:31.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.553+0000 7f06f502a640 1 -- 172.21.15.98:0/189825630 learned_addr learned my addr 172.21.15.98:0/189825630 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:31.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.553+0000 7f06f502a640 1 -- 172.21.15.98:0/189825630 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06f00718f0 msgr2=0x7f06f0141200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:31.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.553+0000 7f06f502a640 1 --2- 172.21.15.98:0/189825630 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06f00718f0 0x7f06f0141200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:31.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.553+0000 7f06f502a640 1 -- 172.21.15.98:0/189825630 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06f0145b70 msgr2=0x7f06f0141740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:31.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.553+0000 7f06f502a640 1 --2- 172.21.15.98:0/189825630 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06f0145b70 0x7f06f0141740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:31.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.553+0000 7f06f502a640 1 -- 172.21.15.98:0/189825630 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f013c810 con 0x7f06f006c1d0 2024-09-19T08:42:31.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.553+0000 7f06f502a640 1 --2- 172.21.15.98:0/189825630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06f006c1d0 0x7f06f0154d60 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f06ec00d620 tx=0x7f06ec00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:31.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.553+0000 7f06e5ffb640 1 -- 172.21.15.98:0/189825630 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ec00b620 con 0x7f06f006c1d0 2024-09-19T08:42:31.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.553+0000 7f06e5ffb640 1 -- 172.21.15.98:0/189825630 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06ec019070 con 0x7f06f006c1d0 2024-09-19T08:42:31.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.553+0000 7f06f728e640 1 -- 172.21.15.98:0/189825630 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06f0158400 con 0x7f06f006c1d0 2024-09-19T08:42:31.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.553+0000 7f06e5ffb640 1 -- 172.21.15.98:0/189825630 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ec00b9e0 con 0x7f06f006c1d0 2024-09-19T08:42:31.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.553+0000 7f06f728e640 1 -- 172.21.15.98:0/189825630 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06f01552a0 con 0x7f06f006c1d0 2024-09-19T08:42:31.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.553+0000 7f06f728e640 1 -- 172.21.15.98:0/189825630 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06f00713b0 con 0x7f06f006c1d0 2024-09-19T08:42:31.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.553+0000 7f06e5ffb640 1 -- 172.21.15.98:0/189825630 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f06ec00bb80 con 0x7f06f006c1d0 2024-09-19T08:42:31.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.553+0000 7f06e5ffb640 1 --2- 172.21.15.98:0/189825630 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f06d4067c20 0x7f06d406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:31.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.553+0000 7f06f4829640 1 --2- 172.21.15.98:0/189825630 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f06d4067c20 0x7f06d406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:31.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.553+0000 7f06e5ffb640 1 -- 172.21.15.98:0/189825630 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f06ec0866a0 con 0x7f06f006c1d0 2024-09-19T08:42:31.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.553+0000 7f06f4829640 1 --2- 172.21.15.98:0/189825630 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f06d4067c20 0x7f06d406a0e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f06e00ad000 tx=0x7f06e0004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:31.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.557+0000 7f06e5ffb640 1 -- 172.21.15.98:0/189825630 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f06ec056d40 con 0x7f06f006c1d0 2024-09-19T08:42:31.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:31.737+0000 7f06f728e640 1 -- 172.21.15.98:0/189825630 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f06f006ddd0 con 0x7f06f006c1d0 2024-09-19T08:42:32.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.413+0000 7f06e5ffb640 1 -- 172.21.15.98:0/189825630 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v38) ==== 122+0+0 (secure 0 0 0) 0x7f06ec056640 con 0x7f06f006c1d0 2024-09-19T08:42:32.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.417+0000 7f06f728e640 1 -- 172.21.15.98:0/189825630 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f06d4067c20 msgr2=0x7f06d406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:32.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.417+0000 7f06f728e640 1 --2- 172.21.15.98:0/189825630 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f06d4067c20 0x7f06d406a0e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f06e00ad000 tx=0x7f06e0004740 comp rx=0 tx=0).stop 2024-09-19T08:42:32.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.417+0000 7f06f728e640 1 -- 172.21.15.98:0/189825630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06f006c1d0 msgr2=0x7f06f0154d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:32.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.417+0000 7f06f728e640 1 --2- 172.21.15.98:0/189825630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06f006c1d0 0x7f06f0154d60 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f06ec00d620 tx=0x7f06ec00dae0 comp rx=0 tx=0).stop 2024-09-19T08:42:32.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.417+0000 7f06f728e640 1 -- 172.21.15.98:0/189825630 shutdown_connections 2024-09-19T08:42:32.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.417+0000 7f06f728e640 1 --2- 172.21.15.98:0/189825630 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f06d4067c20 0x7f06d406a0e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:32.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.417+0000 7f06f728e640 1 --2- 172.21.15.98:0/189825630 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f06f0145b70 0x7f06f0141740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:32.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.417+0000 7f06f728e640 1 --2- 172.21.15.98:0/189825630 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f06f00718f0 0x7f06f0141200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:32.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.417+0000 7f06f728e640 1 --2- 172.21.15.98:0/189825630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06f006c1d0 0x7f06f0154d60 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:32.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.417+0000 7f06f728e640 1 -- 172.21.15.98:0/189825630 >> 172.21.15.98:0/189825630 conn(0x7f06f00b2010 msgr2=0x7f06f0143830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:32.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.417+0000 7f06f728e640 1 -- 172.21.15.98:0/189825630 shutdown_connections 2024-09-19T08:42:32.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.417+0000 7f06f728e640 1 -- 172.21.15.98:0/189825630 wait complete. 2024-09-19T08:42:32.438 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-19T08:42:32.439 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:42:32.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.565+0000 7f5edec70640 1 Processor -- start 2024-09-19T08:42:32.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.565+0000 7f5edec70640 1 -- start start 2024-09-19T08:42:32.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.565+0000 7f5edec70640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5ed8158fd0 0x7f5ed80b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:32.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.565+0000 7f5edec70640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5ed806c3a0 0x7f5ed80ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:32.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.565+0000 7f5edec70640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ed80ccc60 con 0x7f5ed80b4360 2024-09-19T08:42:32.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.565+0000 7f5edec70640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f5ed81593b0 con 0x7f5ed806c3a0 2024-09-19T08:42:32.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.565+0000 7f5edec70640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f5ed80ccf40 con 0x7f5ed8158fd0 2024-09-19T08:42:32.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.565+0000 7f5edd20d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5ed806c3a0 0x7f5ed80ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:32.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.565+0000 7f5ed7fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f5ed80b4360 0x7f5ed806bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41298/0 (socket says 172.21.15.98:41298) 2024-09-19T08:42:32.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.565+0000 7f5ed7fff640 1 -- 172.21.15.98:0/3045303212 learned_addr learned my addr 172.21.15.98:0/3045303212 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:32.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.565+0000 7f5edd20d640 1 -- 172.21.15.98:0/3045303212 >> v2:172.21.15.139:3300/0 conn(0x7f5ed8158fd0 msgr2=0x7f5ed80b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:32.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.565+0000 7f5edca0c640 1 --2- 172.21.15.98:0/3045303212 >> v2:172.21.15.139:3300/0 conn(0x7f5ed8158fd0 0x7f5ed80b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:32.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.565+0000 7f5edd20d640 1 --2- 172.21.15.98:0/3045303212 >> v2:172.21.15.139:3300/0 conn(0x7f5ed8158fd0 0x7f5ed80b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:32.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.565+0000 7f5ed77fe640 1 -- 172.21.15.98:0/3045303212 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2091082332 0 0) 0x7f5ed80ccc60 con 0x7f5ed80b4360 2024-09-19T08:42:32.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.565+0000 7f5edd20d640 1 -- 172.21.15.98:0/3045303212 >> v1:172.21.15.98:6789/0 conn(0x7f5ed80b4360 legacy=0x7f5ed806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:32.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.565+0000 7f5edd20d640 1 -- 172.21.15.98:0/3045303212 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed80cd220 con 0x7f5ed806c3a0 2024-09-19T08:42:32.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.569+0000 7f5edd20d640 1 --2- 172.21.15.98:0/3045303212 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ed806c3a0 0x7f5ed80ce840 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f5ed806e310 tx=0x7f5ed001c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2463890ef3e8f3f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:32.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.569+0000 7f5ed77fe640 1 -- 172.21.15.98:0/3045303212 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed00b2070 con 0x7f5ed806c3a0 2024-09-19T08:42:32.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.569+0000 7f5ed77fe640 1 -- 172.21.15.98:0/3045303212 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ed00a5930 con 0x7f5ed806c3a0 2024-09-19T08:42:32.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.569+0000 7f5ed77fe640 1 -- 172.21.15.98:0/3045303212 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed00a5d00 con 0x7f5ed806c3a0 2024-09-19T08:42:32.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.569+0000 7f5edec70640 1 -- 172.21.15.98:0/3045303212 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ed806c3a0 msgr2=0x7f5ed80ce840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:32.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.569+0000 7f5edec70640 1 --2- 172.21.15.98:0/3045303212 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ed806c3a0 0x7f5ed80ce840 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f5ed806e310 tx=0x7f5ed001c870 comp rx=0 tx=0).stop 2024-09-19T08:42:32.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.569+0000 7f5edec70640 1 -- 172.21.15.98:0/3045303212 shutdown_connections 2024-09-19T08:42:32.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.569+0000 7f5edec70640 1 --2- 172.21.15.98:0/3045303212 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ed806c3a0 0x7f5ed80ce840 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:32.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.569+0000 7f5edec70640 1 --2- 172.21.15.98:0/3045303212 >> v2:172.21.15.139:3300/0 conn(0x7f5ed8158fd0 0x7f5ed80b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:32.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.569+0000 7f5edec70640 1 -- 172.21.15.98:0/3045303212 >> 172.21.15.98:0/3045303212 conn(0x7f5ed80b2010 msgr2=0x7f5ed80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:32.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.569+0000 7f5edec70640 1 -- 172.21.15.98:0/3045303212 shutdown_connections 2024-09-19T08:42:32.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.569+0000 7f5edec70640 1 -- 172.21.15.98:0/3045303212 wait complete. 2024-09-19T08:42:32.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.569+0000 7f5edec70640 1 Processor -- start 2024-09-19T08:42:32.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.569+0000 7f5edec70640 1 -- start start 2024-09-19T08:42:32.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.569+0000 7f5edec70640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ed806c3a0 0x7f5ed8157d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:32.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.569+0000 7f5edec70640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ed80b4360 0x7f5ed81408c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:32.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.569+0000 7f5edec70640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ed8158fd0 0x7f5ed8140e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:32.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.569+0000 7f5edec70640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed80cf710 con 0x7f5ed8158fd0 2024-09-19T08:42:32.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.569+0000 7f5edec70640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed80cf590 con 0x7f5ed80b4360 2024-09-19T08:42:32.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.569+0000 7f5edec70640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5ed80cf890 con 0x7f5ed806c3a0 2024-09-19T08:42:32.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.569+0000 7f5edd20d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ed8158fd0 0x7f5ed8140e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:32.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.569+0000 7f5edd20d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ed8158fd0 0x7f5ed8140e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34318/0 (socket says 172.21.15.98:34318) 2024-09-19T08:42:32.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.569+0000 7f5edd20d640 1 -- 172.21.15.98:0/324694481 learned_addr learned my addr 172.21.15.98:0/324694481 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:32.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.569+0000 7f5edd20d640 1 -- 172.21.15.98:0/324694481 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ed806c3a0 msgr2=0x7f5ed8157d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:32.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.569+0000 7f5edd20d640 1 --2- 172.21.15.98:0/324694481 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ed806c3a0 0x7f5ed8157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:32.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.569+0000 7f5ed7fff640 1 --2- 172.21.15.98:0/324694481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ed80b4360 0x7f5ed81408c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:32.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.569+0000 7f5edd20d640 1 -- 172.21.15.98:0/324694481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ed80b4360 msgr2=0x7f5ed81408c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:32.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.569+0000 7f5edd20d640 1 --2- 172.21.15.98:0/324694481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ed80b4360 0x7f5ed81408c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:32.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.569+0000 7f5edd20d640 1 -- 172.21.15.98:0/324694481 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed815eba0 con 0x7f5ed8158fd0 2024-09-19T08:42:32.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.569+0000 7f5edd20d640 1 --2- 172.21.15.98:0/324694481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ed8158fd0 0x7f5ed8140e00 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f5ed001a4a0 tx=0x7f5ed0019610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:32.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.569+0000 7f5ed57fa640 1 -- 172.21.15.98:0/324694481 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed00b2050 con 0x7f5ed8158fd0 2024-09-19T08:42:32.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.569+0000 7f5ed57fa640 1 -- 172.21.15.98:0/324694481 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ed00be070 con 0x7f5ed8158fd0 2024-09-19T08:42:32.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.569+0000 7f5edec70640 1 -- 172.21.15.98:0/324694481 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ed815ee80 con 0x7f5ed8158fd0 2024-09-19T08:42:32.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.569+0000 7f5ed57fa640 1 -- 172.21.15.98:0/324694481 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed00adad0 con 0x7f5ed8158fd0 2024-09-19T08:42:32.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.569+0000 7f5edec70640 1 -- 172.21.15.98:0/324694481 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ed8071350 con 0x7f5ed8158fd0 2024-09-19T08:42:32.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.573+0000 7f5edec70640 1 -- 172.21.15.98:0/324694481 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ed80b3c00 con 0x7f5ed8158fd0 2024-09-19T08:42:32.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.573+0000 7f5ed57fa640 1 -- 172.21.15.98:0/324694481 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f5ed00adc70 con 0x7f5ed8158fd0 2024-09-19T08:42:32.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.573+0000 7f5ed57fa640 1 --2- 172.21.15.98:0/324694481 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5ec4067c20 0x7f5ec406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:32.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.573+0000 7f5ed57fa640 1 -- 172.21.15.98:0/324694481 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f5ed0121580 con 0x7f5ed8158fd0 2024-09-19T08:42:32.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.573+0000 7f5edca0c640 1 --2- 172.21.15.98:0/324694481 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5ec4067c20 0x7f5ec406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:32.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.573+0000 7f5edca0c640 1 --2- 172.21.15.98:0/324694481 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5ec4067c20 0x7f5ec406a0e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5ec800a810 tx=0x7f5ec8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:32.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.577+0000 7f5ed57fa640 1 -- 172.21.15.98:0/324694481 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f5ed0016520 con 0x7f5ed8158fd0 2024-09-19T08:42:32.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.773+0000 7f5edec70640 1 -- 172.21.15.98:0/324694481 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ed806ddb0 con 0x7f5ed8158fd0 2024-09-19T08:42:32.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.777+0000 7f5ed57fa640 1 -- 172.21.15.98:0/324694481 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5228 (secure 0 0 0) 0x7f5ed00f1d40 con 0x7f5ed8158fd0 2024-09-19T08:42:32.776 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:42:32.777 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":38,"btime":"2024-09-19T08:42:32:414567+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5548,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3092385040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3092385040},{"type":"v1","addr":"172.21.15.139:6833","nonce":3092385040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35},{"gid":5551,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3517671290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3517671290},{"type":"v1","addr":"172.21.15.139:6835","nonce":3517671290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35},{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6836/965821274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":965821274},{"type":"v1","addr":"172.21.15.98:6836","nonce":965821274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:42:29.720825+0000","modified":"2024-09-19T08:42:31.743275+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6171":{"gid":6171,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/1767881083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6835","nonce":1767881083},{"type":"v1","addr":"172.21.15.98:6837","nonce":1767881083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6171,"qdb_cluster":[6171]},"id":5}]} 2024-09-19T08:42:32.777 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 38 2024-09-19T08:42:32.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.777+0000 7f5edec70640 1 -- 172.21.15.98:0/324694481 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5ec4067c20 msgr2=0x7f5ec406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:32.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.777+0000 7f5edec70640 1 --2- 172.21.15.98:0/324694481 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5ec4067c20 0x7f5ec406a0e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5ec800a810 tx=0x7f5ec8008040 comp rx=0 tx=0).stop 2024-09-19T08:42:32.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.777+0000 7f5edec70640 1 -- 172.21.15.98:0/324694481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ed8158fd0 msgr2=0x7f5ed8140e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:32.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.777+0000 7f5edec70640 1 --2- 172.21.15.98:0/324694481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ed8158fd0 0x7f5ed8140e00 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f5ed001a4a0 tx=0x7f5ed0019610 comp rx=0 tx=0).stop 2024-09-19T08:42:32.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.777+0000 7f5edec70640 1 -- 172.21.15.98:0/324694481 shutdown_connections 2024-09-19T08:42:32.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.777+0000 7f5edec70640 1 --2- 172.21.15.98:0/324694481 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5ec4067c20 0x7f5ec406a0e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:32.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.777+0000 7f5edec70640 1 --2- 172.21.15.98:0/324694481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ed8158fd0 0x7f5ed8140e00 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:32.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.777+0000 7f5edec70640 1 --2- 172.21.15.98:0/324694481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ed80b4360 0x7f5ed81408c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:32.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.777+0000 7f5edec70640 1 --2- 172.21.15.98:0/324694481 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ed806c3a0 0x7f5ed8157d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:32.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.777+0000 7f5edec70640 1 -- 172.21.15.98:0/324694481 >> 172.21.15.98:0/324694481 conn(0x7f5ed80b2010 msgr2=0x7f5ed806bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:32.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.777+0000 7f5edec70640 1 -- 172.21.15.98:0/324694481 shutdown_connections 2024-09-19T08:42:32.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.777+0000 7f5edec70640 1 -- 172.21.15.98:0/324694481 wait complete. 2024-09-19T08:42:32.796 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:42:32.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.913+0000 7faca2f55640 1 Processor -- start 2024-09-19T08:42:32.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.913+0000 7faca2f55640 1 -- start start 2024-09-19T08:42:32.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.917+0000 7faca2f55640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fac9c0b7520 0x7fac9c0b7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:32.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.917+0000 7faca2f55640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac9c0b6200 0x7fac9c0b6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:32.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.917+0000 7faca2f55640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac9c0b99a0 con 0x7fac9c147770 2024-09-19T08:42:32.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.917+0000 7faca2f55640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fac9c0b6ba0 con 0x7fac9c0b7520 2024-09-19T08:42:32.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.917+0000 7faca2f55640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fac9c0b85a0 con 0x7fac9c0b6200 2024-09-19T08:42:32.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.917+0000 7fac9bfff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fac9c0b7520 0x7fac9c0b7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:32.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.917+0000 7fac9bfff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fac9c0b7520 0x7fac9c0b7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34332/0 (socket says 172.21.15.98:34332) 2024-09-19T08:42:32.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.917+0000 7fac9bfff640 1 -- 172.21.15.98:0/1752409951 learned_addr learned my addr 172.21.15.98:0/1752409951 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:32.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.917+0000 7faca0cf1640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fac9c147770 0x7fac9c0b9290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41312/0 (socket says 172.21.15.98:41312) 2024-09-19T08:42:32.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.917+0000 7fac9bfff640 1 -- 172.21.15.98:0/1752409951 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac9c0b6200 msgr2=0x7fac9c0b6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:32.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.917+0000 7faca14f2640 1 --2- 172.21.15.98:0/1752409951 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac9c0b6200 0x7fac9c0b6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:32.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.917+0000 7fac9bfff640 1 --2- 172.21.15.98:0/1752409951 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac9c0b6200 0x7fac9c0b6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:32.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.917+0000 7fac9bfff640 1 -- 172.21.15.98:0/1752409951 >> v1:172.21.15.98:6789/0 conn(0x7fac9c147770 legacy=0x7fac9c0b9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:32.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.917+0000 7fac9bfff640 1 -- 172.21.15.98:0/1752409951 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9c0b7fe0 con 0x7fac9c0b7520 2024-09-19T08:42:32.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.917+0000 7faca14f2640 1 --2- 172.21.15.98:0/1752409951 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac9c0b6200 0x7fac9c0b6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:42:32.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.917+0000 7fac9bfff640 1 --2- 172.21.15.98:0/1752409951 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac9c0b7520 0x7fac9c0b7960 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fac90009870 tx=0x7fac9001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e3f842bdf27eff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:32.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.917+0000 7fac9b7fe640 1 -- 172.21.15.98:0/1752409951 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac900aa600 con 0x7fac9c0b7520 2024-09-19T08:42:32.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.917+0000 7fac9b7fe640 1 -- 172.21.15.98:0/1752409951 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac900af070 con 0x7fac9c0b7520 2024-09-19T08:42:32.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.917+0000 7fac9b7fe640 1 -- 172.21.15.98:0/1752409951 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac900aa9c0 con 0x7fac9c0b7520 2024-09-19T08:42:32.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.917+0000 7faca2f55640 1 -- 172.21.15.98:0/1752409951 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac9c0b7520 msgr2=0x7fac9c0b7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:32.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.917+0000 7faca2f55640 1 --2- 172.21.15.98:0/1752409951 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac9c0b7520 0x7fac9c0b7960 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fac90009870 tx=0x7fac9001c4a0 comp rx=0 tx=0).stop 2024-09-19T08:42:32.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.917+0000 7faca2f55640 1 -- 172.21.15.98:0/1752409951 shutdown_connections 2024-09-19T08:42:32.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.917+0000 7faca2f55640 1 --2- 172.21.15.98:0/1752409951 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac9c0b6200 0x7fac9c0b6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:32.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.917+0000 7faca2f55640 1 --2- 172.21.15.98:0/1752409951 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac9c0b7520 0x7fac9c0b7960 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:32.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.917+0000 7faca2f55640 1 -- 172.21.15.98:0/1752409951 >> 172.21.15.98:0/1752409951 conn(0x7fac9c074e40 msgr2=0x7fac9c0722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:32.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.917+0000 7faca2f55640 1 -- 172.21.15.98:0/1752409951 shutdown_connections 2024-09-19T08:42:32.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.917+0000 7faca2f55640 1 -- 172.21.15.98:0/1752409951 wait complete. 2024-09-19T08:42:32.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.921+0000 7faca2f55640 1 Processor -- start 2024-09-19T08:42:32.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.921+0000 7faca2f55640 1 -- start start 2024-09-19T08:42:32.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.921+0000 7faca2f55640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac9c0b6200 0x7fac9c141d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:32.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.921+0000 7faca2f55640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac9c0b7520 0x7fac9c160d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:32.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.921+0000 7faca2f55640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac9c147770 0x7fac9c15d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:32.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.921+0000 7faca2f55640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c147b50 con 0x7fac9c147770 2024-09-19T08:42:32.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.921+0000 7faca2f55640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c1482a0 con 0x7fac9c0b6200 2024-09-19T08:42:32.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.921+0000 7faca2f55640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fac9c147cd0 con 0x7fac9c0b7520 2024-09-19T08:42:32.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.921+0000 7faca14f2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac9c147770 0x7fac9c15d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:32.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.921+0000 7faca14f2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac9c147770 0x7fac9c15d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34344/0 (socket says 172.21.15.98:34344) 2024-09-19T08:42:32.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.921+0000 7faca14f2640 1 -- 172.21.15.98:0/2955627761 learned_addr learned my addr 172.21.15.98:0/2955627761 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:32.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.921+0000 7faca0cf1640 1 --2- 172.21.15.98:0/2955627761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac9c0b6200 0x7fac9c141d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:32.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.921+0000 7faca14f2640 1 -- 172.21.15.98:0/2955627761 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac9c0b7520 msgr2=0x7fac9c160d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:32.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.921+0000 7fac9bfff640 1 --2- 172.21.15.98:0/2955627761 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac9c0b7520 0x7fac9c160d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:32.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.921+0000 7faca14f2640 1 --2- 172.21.15.98:0/2955627761 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac9c0b7520 0x7fac9c160d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:32.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.921+0000 7faca14f2640 1 -- 172.21.15.98:0/2955627761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac9c0b6200 msgr2=0x7fac9c141d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:32.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.921+0000 7faca14f2640 1 --2- 172.21.15.98:0/2955627761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac9c0b6200 0x7fac9c141d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:32.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.921+0000 7faca14f2640 1 -- 172.21.15.98:0/2955627761 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9c161650 con 0x7fac9c147770 2024-09-19T08:42:32.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.921+0000 7faca0cf1640 1 --2- 172.21.15.98:0/2955627761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac9c0b6200 0x7fac9c141d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:42:32.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.921+0000 7faca14f2640 1 --2- 172.21.15.98:0/2955627761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac9c147770 0x7fac9c15d9f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fac9400cef0 tx=0x7fac9400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:32.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.921+0000 7fac997fa640 1 -- 172.21.15.98:0/2955627761 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9401a070 con 0x7fac9c147770 2024-09-19T08:42:32.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.921+0000 7fac997fa640 1 -- 172.21.15.98:0/2955627761 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac9400d860 con 0x7fac9c147770 2024-09-19T08:42:32.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.921+0000 7faca2f55640 1 -- 172.21.15.98:0/2955627761 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac9c143e70 con 0x7fac9c147770 2024-09-19T08:42:32.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.921+0000 7fac997fa640 1 -- 172.21.15.98:0/2955627761 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9400dc50 con 0x7fac9c147770 2024-09-19T08:42:32.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.921+0000 7faca2f55640 1 -- 172.21.15.98:0/2955627761 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac9c161290 con 0x7fac9c147770 2024-09-19T08:42:32.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.921+0000 7faca2f55640 1 -- 172.21.15.98:0/2955627761 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac9c0b6660 con 0x7fac9c147770 2024-09-19T08:42:32.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.925+0000 7fac997fa640 1 -- 172.21.15.98:0/2955627761 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fac9401d070 con 0x7fac9c147770 2024-09-19T08:42:32.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.925+0000 7fac997fa640 1 --2- 172.21.15.98:0/2955627761 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fac88067c20 0x7fac8806a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:32.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.925+0000 7fac997fa640 1 -- 172.21.15.98:0/2955627761 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7fac94086ac0 con 0x7fac9c147770 2024-09-19T08:42:32.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.925+0000 7faca0cf1640 1 --2- 172.21.15.98:0/2955627761 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fac88067c20 0x7fac8806a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:32.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.925+0000 7faca0cf1640 1 --2- 172.21.15.98:0/2955627761 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fac88067c20 0x7fac8806a0e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fac8c007fd0 tx=0x7fac8c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:32.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:32.929+0000 7fac997fa640 1 -- 172.21.15.98:0/2955627761 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fac940571f0 con 0x7fac9c147770 2024-09-19T08:42:33.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.089+0000 7faca2f55640 1 -- 172.21.15.98:0/2955627761 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fac9c158f50 con 0x7fac9c147770 2024-09-19T08:42:33.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.093+0000 7fac997fa640 1 -- 172.21.15.98:0/2955627761 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+16728 (secure 0 0 0) 0x7fac94086e20 con 0x7fac9c147770 2024-09-19T08:42:33.094 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:42:33.094 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":68,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:42:31.388275+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:42:27.733851+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-19T08:42:28.717566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:42:33.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.093+0000 7faca2f55640 1 -- 172.21.15.98:0/2955627761 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fac88067c20 msgr2=0x7fac8806a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:33.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.093+0000 7faca2f55640 1 --2- 172.21.15.98:0/2955627761 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fac88067c20 0x7fac8806a0e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fac8c007fd0 tx=0x7fac8c009040 comp rx=0 tx=0).stop 2024-09-19T08:42:33.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.093+0000 7faca2f55640 1 -- 172.21.15.98:0/2955627761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac9c147770 msgr2=0x7fac9c15d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:33.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.093+0000 7faca2f55640 1 --2- 172.21.15.98:0/2955627761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac9c147770 0x7fac9c15d9f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fac9400cef0 tx=0x7fac9400a710 comp rx=0 tx=0).stop 2024-09-19T08:42:33.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.093+0000 7faca2f55640 1 -- 172.21.15.98:0/2955627761 shutdown_connections 2024-09-19T08:42:33.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.093+0000 7faca2f55640 1 --2- 172.21.15.98:0/2955627761 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fac88067c20 0x7fac8806a0e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:33.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.093+0000 7faca2f55640 1 --2- 172.21.15.98:0/2955627761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac9c147770 0x7fac9c15d9f0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:33.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.093+0000 7faca2f55640 1 --2- 172.21.15.98:0/2955627761 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac9c0b7520 0x7fac9c160d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:33.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.093+0000 7faca2f55640 1 --2- 172.21.15.98:0/2955627761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac9c0b6200 0x7fac9c141d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:33.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.093+0000 7faca2f55640 1 -- 172.21.15.98:0/2955627761 >> 172.21.15.98:0/2955627761 conn(0x7fac9c074e40 msgr2=0x7fac9c0d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:33.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.093+0000 7faca2f55640 1 -- 172.21.15.98:0/2955627761 shutdown_connections 2024-09-19T08:42:33.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.093+0000 7faca2f55640 1 -- 172.21.15.98:0/2955627761 wait complete. 2024-09-19T08:42:33.113 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T08:42:33.113 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:42:33.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.229+0000 7fd3f7c7a640 1 Processor -- start 2024-09-19T08:42:33.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.229+0000 7fd3f7c7a640 1 -- start start 2024-09-19T08:42:33.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.233+0000 7fd3f7c7a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd3f00d0ff0 0x7fd3f00d1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:33.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.233+0000 7fd3f7c7a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd3f00b6f40 0x7fd3f00b73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:33.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.233+0000 7fd3f7c7a640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3f00b78e0 con 0x7fd3f015c220 2024-09-19T08:42:33.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.233+0000 7fd3f7c7a640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fd3f00d19a0 con 0x7fd3f00b6f40 2024-09-19T08:42:33.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.233+0000 7fd3f7c7a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fd3f00740b0 con 0x7fd3f00d0ff0 2024-09-19T08:42:33.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.233+0000 7fd3f5a16640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fd3f015c220 0x7fd3f00733f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41328/0 (socket says 172.21.15.98:41328) 2024-09-19T08:42:33.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.233+0000 7fd3f5a16640 1 -- 172.21.15.98:0/2901702411 learned_addr learned my addr 172.21.15.98:0/2901702411 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:33.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.233+0000 7fd3f6217640 1 --2- 172.21.15.98:0/2901702411 >> v2:172.21.15.98:3300/0 conn(0x7fd3f00b6f40 0x7fd3f00b73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:33.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.233+0000 7fd3f4a14640 1 -- 172.21.15.98:0/2901702411 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3178924566 0 0) 0x7fd3ec001120 con 0x7fd3f015c220 2024-09-19T08:42:33.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.233+0000 7fd3f4a14640 1 -- 172.21.15.98:0/2901702411 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd3dc003530 con 0x7fd3f015c220 2024-09-19T08:42:33.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.233+0000 7fd3f5215640 1 --2- 172.21.15.98:0/2901702411 >> v2:172.21.15.139:3300/0 conn(0x7fd3f00d0ff0 0x7fd3f00d1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:33.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.233+0000 7fd3f6217640 1 -- 172.21.15.98:0/2901702411 >> v2:172.21.15.139:3300/0 conn(0x7fd3f00d0ff0 msgr2=0x7fd3f00d1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:33.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.233+0000 7fd3f6217640 1 --2- 172.21.15.98:0/2901702411 >> v2:172.21.15.139:3300/0 conn(0x7fd3f00d0ff0 0x7fd3f00d1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:33.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.233+0000 7fd3f6217640 1 -- 172.21.15.98:0/2901702411 >> v1:172.21.15.98:6789/0 conn(0x7fd3f015c220 legacy=0x7fd3f00733f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:33.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.233+0000 7fd3f6217640 1 -- 172.21.15.98:0/2901702411 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3f0073b00 con 0x7fd3f00b6f40 2024-09-19T08:42:33.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.233+0000 7fd3f5215640 1 --2- 172.21.15.98:0/2901702411 >> v2:172.21.15.139:3300/0 conn(0x7fd3f00d0ff0 0x7fd3f00d1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:42:33.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.237+0000 7fd3f6217640 1 --2- 172.21.15.98:0/2901702411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3f00b6f40 0x7fd3f00b73a0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0073090 tx=0x7fd3e801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9edf0e0890ab181c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:33.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.237+0000 7fd3f4a14640 1 -- 172.21.15.98:0/2901702411 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e80b2070 con 0x7fd3f00b6f40 2024-09-19T08:42:33.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.237+0000 7fd3f4a14640 1 -- 172.21.15.98:0/2901702411 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3e80a59c0 con 0x7fd3f00b6f40 2024-09-19T08:42:33.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.237+0000 7fd3f4a14640 1 -- 172.21.15.98:0/2901702411 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e80a5d70 con 0x7fd3f00b6f40 2024-09-19T08:42:33.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.237+0000 7fd3f7c7a640 1 -- 172.21.15.98:0/2901702411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3f00b6f40 msgr2=0x7fd3f00b73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:33.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.237+0000 7fd3f7c7a640 1 --2- 172.21.15.98:0/2901702411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3f00b6f40 0x7fd3f00b73a0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0073090 tx=0x7fd3e801c900 comp rx=0 tx=0).stop 2024-09-19T08:42:33.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.237+0000 7fd3f7c7a640 1 -- 172.21.15.98:0/2901702411 shutdown_connections 2024-09-19T08:42:33.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.237+0000 7fd3f7c7a640 1 --2- 172.21.15.98:0/2901702411 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3f00b6f40 0x7fd3f00b73a0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:33.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.237+0000 7fd3f7c7a640 1 --2- 172.21.15.98:0/2901702411 >> v2:172.21.15.139:3300/0 conn(0x7fd3f00d0ff0 0x7fd3f00d1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:33.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.237+0000 7fd3f7c7a640 1 -- 172.21.15.98:0/2901702411 >> 172.21.15.98:0/2901702411 conn(0x7fd3f00b40a0 msgr2=0x7fd3f00b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:33.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.237+0000 7fd3f7c7a640 1 -- 172.21.15.98:0/2901702411 shutdown_connections 2024-09-19T08:42:33.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.237+0000 7fd3f7c7a640 1 -- 172.21.15.98:0/2901702411 wait complete. 2024-09-19T08:42:33.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.237+0000 7fd3f7c7a640 1 Processor -- start 2024-09-19T08:42:33.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.237+0000 7fd3f7c7a640 1 -- start start 2024-09-19T08:42:33.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.237+0000 7fd3f7c7a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3f00b6f40 0x7fd3f0157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:33.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.237+0000 7fd3f7c7a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3f00d0ff0 0x7fd3f014e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:33.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.237+0000 7fd3f7c7a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3f015c220 0x7fd3f014e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:33.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.237+0000 7fd3f7c7a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f00d0c30 con 0x7fd3f00d0ff0 2024-09-19T08:42:33.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.237+0000 7fd3f7c7a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f00b6a20 con 0x7fd3f015c220 2024-09-19T08:42:33.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.237+0000 7fd3f7c7a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd3f00d0db0 con 0x7fd3f00b6f40 2024-09-19T08:42:33.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.237+0000 7fd3f5215640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3f00d0ff0 0x7fd3f014e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:33.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.237+0000 7fd3f5215640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3f00d0ff0 0x7fd3f014e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34348/0 (socket says 172.21.15.98:34348) 2024-09-19T08:42:33.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.237+0000 7fd3f5215640 1 -- 172.21.15.98:0/1859938131 learned_addr learned my addr 172.21.15.98:0/1859938131 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:33.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.237+0000 7fd3f5a16640 1 --2- 172.21.15.98:0/1859938131 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3f00b6f40 0x7fd3f0157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:33.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.237+0000 7fd3f6217640 1 --2- 172.21.15.98:0/1859938131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3f015c220 0x7fd3f014e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:33.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.237+0000 7fd3f5215640 1 -- 172.21.15.98:0/1859938131 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3f00b6f40 msgr2=0x7fd3f0157ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:33.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.237+0000 7fd3f5215640 1 --2- 172.21.15.98:0/1859938131 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3f00b6f40 0x7fd3f0157ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:33.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.237+0000 7fd3f5215640 1 -- 172.21.15.98:0/1859938131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3f015c220 msgr2=0x7fd3f014e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:33.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.237+0000 7fd3f5215640 1 --2- 172.21.15.98:0/1859938131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3f015c220 0x7fd3f014e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:33.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.237+0000 7fd3f5215640 1 -- 172.21.15.98:0/1859938131 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3f01550b0 con 0x7fd3f00d0ff0 2024-09-19T08:42:33.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.237+0000 7fd3f5215640 1 --2- 172.21.15.98:0/1859938131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3f00d0ff0 0x7fd3f014e400 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd3e000c7f0 tx=0x7fd3e000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:33.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.241+0000 7fd3e67fc640 1 -- 172.21.15.98:0/1859938131 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e0007a30 con 0x7fd3f00d0ff0 2024-09-19T08:42:33.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.241+0000 7fd3e67fc640 1 -- 172.21.15.98:0/1859938131 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3e0007bd0 con 0x7fd3f00d0ff0 2024-09-19T08:42:33.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.241+0000 7fd3f7c7a640 1 -- 172.21.15.98:0/1859938131 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3f0155390 con 0x7fd3f00d0ff0 2024-09-19T08:42:33.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.241+0000 7fd3e67fc640 1 -- 172.21.15.98:0/1859938131 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e00113f0 con 0x7fd3f00d0ff0 2024-09-19T08:42:33.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.241+0000 7fd3f7c7a640 1 -- 172.21.15.98:0/1859938131 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3f01580e0 con 0x7fd3f00d0ff0 2024-09-19T08:42:33.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.241+0000 7fd3f7c7a640 1 -- 172.21.15.98:0/1859938131 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3f0150ae0 con 0x7fd3f00d0ff0 2024-09-19T08:42:33.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.241+0000 7fd3e67fc640 1 -- 172.21.15.98:0/1859938131 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fd3e0011590 con 0x7fd3f00d0ff0 2024-09-19T08:42:33.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.241+0000 7fd3e67fc640 1 --2- 172.21.15.98:0/1859938131 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd3d4067c20 0x7fd3d406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:33.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.241+0000 7fd3e67fc640 1 -- 172.21.15.98:0/1859938131 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7fd3e00864e0 con 0x7fd3f00d0ff0 2024-09-19T08:42:33.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.241+0000 7fd3f5a16640 1 --2- 172.21.15.98:0/1859938131 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd3d4067c20 0x7fd3d406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:33.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.241+0000 7fd3f5a16640 1 --2- 172.21.15.98:0/1859938131 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd3d4067c20 0x7fd3d406a0e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec005890 tx=0x7fd3ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:33.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.245+0000 7fd3e67fc640 1 -- 172.21.15.98:0/1859938131 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd3e0015020 con 0x7fd3f00d0ff0 2024-09-19T08:42:33.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.405+0000 7fd3f7c7a640 1 -- 172.21.15.98:0/1859938131 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd3f015ce10 con 0x7fd3d4067c20 2024-09-19T08:42:33.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.417+0000 7fd3e67fc640 1 -- 172.21.15.98:0/1859938131 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241995 (secure 0 0 0) 0x7fd3f015ce10 con 0x7fd3d4067c20 2024-09-19T08:42:33.424 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:42:33.425 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:42:33.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.421+0000 7fd3f7c7a640 1 -- 172.21.15.98:0/1859938131 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd3d4067c20 msgr2=0x7fd3d406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:33.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.421+0000 7fd3f7c7a640 1 --2- 172.21.15.98:0/1859938131 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd3d4067c20 0x7fd3d406a0e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec005890 tx=0x7fd3ec009040 comp rx=0 tx=0).stop 2024-09-19T08:42:33.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.421+0000 7fd3f7c7a640 1 -- 172.21.15.98:0/1859938131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3f00d0ff0 msgr2=0x7fd3f014e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:33.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.421+0000 7fd3f7c7a640 1 --2- 172.21.15.98:0/1859938131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3f00d0ff0 0x7fd3f014e400 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd3e000c7f0 tx=0x7fd3e000ccb0 comp rx=0 tx=0).stop 2024-09-19T08:42:33.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.421+0000 7fd3f7c7a640 1 -- 172.21.15.98:0/1859938131 shutdown_connections 2024-09-19T08:42:33.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.421+0000 7fd3f7c7a640 1 --2- 172.21.15.98:0/1859938131 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fd3d4067c20 0x7fd3d406a0e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:33.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.421+0000 7fd3f7c7a640 1 --2- 172.21.15.98:0/1859938131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd3f015c220 0x7fd3f014e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:33.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.421+0000 7fd3f7c7a640 1 --2- 172.21.15.98:0/1859938131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3f00d0ff0 0x7fd3f014e400 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:33.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.421+0000 7fd3f7c7a640 1 --2- 172.21.15.98:0/1859938131 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3f00b6f40 0x7fd3f0157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:33.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.421+0000 7fd3f7c7a640 1 -- 172.21.15.98:0/1859938131 >> 172.21.15.98:0/1859938131 conn(0x7fd3f00b40a0 msgr2=0x7fd3f015b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:33.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.421+0000 7fd3f7c7a640 1 -- 172.21.15.98:0/1859938131 shutdown_connections 2024-09-19T08:42:33.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.421+0000 7fd3f7c7a640 1 -- 172.21.15.98:0/1859938131 wait complete. 2024-09-19T08:42:33.439 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2024-09-19T08:42:32.110399+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590460,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":168,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126,"ondisk_log_size":126,"up":80,"acting":80,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":256832,"kb_used_data":7292,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717888,"statfs":{"total":773094113280,"available":772831117312,"internally_reserved":0,"allocated":7467008,"data_stored":3775472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":92,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001134"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402870+0000","last_change":"2024-09-19T08:42:30.446888+0000","last_active":"2024-09-19T08:42:31.402870+0000","last_peered":"2024-09-19T08:42:31.402870+0000","last_clean":"2024-09-19T08:42:31.402870+0000","last_became_active":"2024-09-19T08:42:30.438820+0000","last_became_peered":"2024-09-19T08:42:30.438820+0000","last_unstale":"2024-09-19T08:42:31.402870+0000","last_undegraded":"2024-09-19T08:42:31.402870+0000","last_fullsized":"2024-09-19T08:42:31.402870+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:52:02.376361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401471+0000","last_change":"2024-09-19T08:42:30.431823+0000","last_active":"2024-09-19T08:42:31.401471+0000","last_peered":"2024-09-19T08:42:31.401471+0000","last_clean":"2024-09-19T08:42:31.401471+0000","last_became_active":"2024-09-19T08:42:30.430962+0000","last_became_peered":"2024-09-19T08:42:30.430962+0000","last_unstale":"2024-09-19T08:42:31.401471+0000","last_undegraded":"2024-09-19T08:42:31.401471+0000","last_fullsized":"2024-09-19T08:42:31.401471+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:55.760155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401494+0000","last_change":"2024-09-19T08:42:30.438133+0000","last_active":"2024-09-19T08:42:31.401494+0000","last_peered":"2024-09-19T08:42:31.401494+0000","last_clean":"2024-09-19T08:42:31.401494+0000","last_became_active":"2024-09-19T08:42:30.437186+0000","last_became_peered":"2024-09-19T08:42:30.437186+0000","last_unstale":"2024-09-19T08:42:31.401494+0000","last_undegraded":"2024-09-19T08:42:31.401494+0000","last_fullsized":"2024-09-19T08:42:31.401494+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:55.764601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402257+0000","last_change":"2024-09-19T08:42:30.439106+0000","last_active":"2024-09-19T08:42:31.402257+0000","last_peered":"2024-09-19T08:42:31.402257+0000","last_clean":"2024-09-19T08:42:31.402257+0000","last_became_active":"2024-09-19T08:42:30.438367+0000","last_became_peered":"2024-09-19T08:42:30.438367+0000","last_unstale":"2024-09-19T08:42:31.402257+0000","last_undegraded":"2024-09-19T08:42:31.402257+0000","last_fullsized":"2024-09-19T08:42:31.402257+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:14:25.294258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401507+0000","last_change":"2024-09-19T08:42:30.434665+0000","last_active":"2024-09-19T08:42:31.401507+0000","last_peered":"2024-09-19T08:42:31.401507+0000","last_clean":"2024-09-19T08:42:31.401507+0000","last_became_active":"2024-09-19T08:42:30.433735+0000","last_became_peered":"2024-09-19T08:42:30.433735+0000","last_unstale":"2024-09-19T08:42:31.401507+0000","last_undegraded":"2024-09-19T08:42:31.401507+0000","last_fullsized":"2024-09-19T08:42:31.401507+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:37:26.942476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.356233+0000","last_change":"2024-09-19T08:42:28.356233+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.356233+0000","last_undegraded":"2024-09-19T08:42:28.356233+0000","last_fullsized":"2024-09-19T08:42:28.356233+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.349928+0000","last_change":"2024-09-19T08:42:28.349928+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.349928+0000","last_undegraded":"2024-09-19T08:42:28.349928+0000","last_fullsized":"2024-09-19T08:42:28.349928+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402198+0000","last_change":"2024-09-19T08:42:30.458129+0000","last_active":"2024-09-19T08:42:31.402198+0000","last_peered":"2024-09-19T08:42:31.402198+0000","last_clean":"2024-09-19T08:42:31.402198+0000","last_became_active":"2024-09-19T08:42:30.456984+0000","last_became_peered":"2024-09-19T08:42:30.456984+0000","last_unstale":"2024-09-19T08:42:31.402198+0000","last_undegraded":"2024-09-19T08:42:31.402198+0000","last_fullsized":"2024-09-19T08:42:31.402198+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:43:27.676061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.351206+0000","last_change":"2024-09-19T08:42:28.351206+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.351206+0000","last_undegraded":"2024-09-19T08:42:28.351206+0000","last_fullsized":"2024-09-19T08:42:28.351206+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402459+0000","last_change":"2024-09-19T08:42:29.395380+0000","last_active":"2024-09-19T08:42:31.402459+0000","last_peered":"2024-09-19T08:42:31.402459+0000","last_clean":"2024-09-19T08:42:31.402459+0000","last_became_active":"2024-09-19T08:42:29.394641+0000","last_became_peered":"2024-09-19T08:42:29.394641+0000","last_unstale":"2024-09-19T08:42:31.402459+0000","last_undegraded":"2024-09-19T08:42:31.402459+0000","last_fullsized":"2024-09-19T08:42:31.402459+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:26.586550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:28.334315+0000","last_change":"2024-09-19T08:42:28.334315+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.334315+0000","last_undegraded":"2024-09-19T08:42:28.334315+0000","last_fullsized":"2024-09-19T08:42:28.334315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401528+0000","last_change":"2024-09-19T08:42:29.399398+0000","last_active":"2024-09-19T08:42:31.401528+0000","last_peered":"2024-09-19T08:42:31.401528+0000","last_clean":"2024-09-19T08:42:31.401528+0000","last_became_active":"2024-09-19T08:42:29.396716+0000","last_became_peered":"2024-09-19T08:42:29.396716+0000","last_unstale":"2024-09-19T08:42:31.401528+0000","last_undegraded":"2024-09-19T08:42:31.401528+0000","last_fullsized":"2024-09-19T08:42:31.401528+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:50:23.952850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.355932+0000","last_change":"2024-09-19T08:42:28.355932+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.355932+0000","last_undegraded":"2024-09-19T08:42:28.355932+0000","last_fullsized":"2024-09-19T08:42:28.355932+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.353311+0000","last_change":"2024-09-19T08:42:28.353311+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.353311+0000","last_undegraded":"2024-09-19T08:42:28.353311+0000","last_fullsized":"2024-09-19T08:42:28.353311+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.349295+0000","last_change":"2024-09-19T08:42:28.349295+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.349295+0000","last_undegraded":"2024-09-19T08:42:28.349295+0000","last_fullsized":"2024-09-19T08:42:28.349295+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402803+0000","last_change":"2024-09-19T08:42:29.399190+0000","last_active":"2024-09-19T08:42:31.402803+0000","last_peered":"2024-09-19T08:42:31.402803+0000","last_clean":"2024-09-19T08:42:31.402803+0000","last_became_active":"2024-09-19T08:42:29.396183+0000","last_became_peered":"2024-09-19T08:42:29.396183+0000","last_unstale":"2024-09-19T08:42:31.402803+0000","last_undegraded":"2024-09-19T08:42:31.402803+0000","last_fullsized":"2024-09-19T08:42:31.402803+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:21:01.013528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402155+0000","last_change":"2024-09-19T08:42:30.451725+0000","last_active":"2024-09-19T08:42:31.402155+0000","last_peered":"2024-09-19T08:42:31.402155+0000","last_clean":"2024-09-19T08:42:31.402155+0000","last_became_active":"2024-09-19T08:42:30.437086+0000","last_became_peered":"2024-09-19T08:42:30.437086+0000","last_unstale":"2024-09-19T08:42:31.402155+0000","last_undegraded":"2024-09-19T08:42:31.402155+0000","last_fullsized":"2024-09-19T08:42:31.402155+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:34.398500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:28.334315+0000","last_change":"2024-09-19T08:42:28.334315+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.334315+0000","last_undegraded":"2024-09-19T08:42:28.334315+0000","last_fullsized":"2024-09-19T08:42:28.334315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.355876+0000","last_change":"2024-09-19T08:42:28.355876+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.355876+0000","last_undegraded":"2024-09-19T08:42:28.355876+0000","last_fullsized":"2024-09-19T08:42:28.355876+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.355949+0000","last_change":"2024-09-19T08:42:28.355949+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.355949+0000","last_undegraded":"2024-09-19T08:42:28.355949+0000","last_fullsized":"2024-09-19T08:42:28.355949+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401738+0000","last_change":"2024-09-19T08:42:29.399130+0000","last_active":"2024-09-19T08:42:31.401738+0000","last_peered":"2024-09-19T08:42:31.401738+0000","last_clean":"2024-09-19T08:42:31.401738+0000","last_became_active":"2024-09-19T08:42:29.396039+0000","last_became_peered":"2024-09-19T08:42:29.396039+0000","last_unstale":"2024-09-19T08:42:31.401738+0000","last_undegraded":"2024-09-19T08:42:31.401738+0000","last_fullsized":"2024-09-19T08:42:31.401738+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:54.277351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.361851+0000","last_change":"2024-09-19T08:42:28.361851+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.361851+0000","last_undegraded":"2024-09-19T08:42:28.361851+0000","last_fullsized":"2024-09-19T08:42:28.361851+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.348977+0000","last_change":"2024-09-19T08:42:28.348977+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.348977+0000","last_undegraded":"2024-09-19T08:42:28.348977+0000","last_fullsized":"2024-09-19T08:42:28.348977+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402598+0000","last_change":"2024-09-19T08:42:29.400466+0000","last_active":"2024-09-19T08:42:31.402598+0000","last_peered":"2024-09-19T08:42:31.402598+0000","last_clean":"2024-09-19T08:42:31.402598+0000","last_became_active":"2024-09-19T08:42:29.395739+0000","last_became_peered":"2024-09-19T08:42:29.395739+0000","last_unstale":"2024-09-19T08:42:31.402598+0000","last_undegraded":"2024-09-19T08:42:31.402598+0000","last_fullsized":"2024-09-19T08:42:31.402598+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:41:59.302829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.349238+0000","last_change":"2024-09-19T08:42:28.349238+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.349238+0000","last_undegraded":"2024-09-19T08:42:28.349238+0000","last_fullsized":"2024-09-19T08:42:28.349238+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402346+0000","last_change":"2024-09-19T08:42:29.399885+0000","last_active":"2024-09-19T08:42:31.402346+0000","last_peered":"2024-09-19T08:42:31.402346+0000","last_clean":"2024-09-19T08:42:31.402346+0000","last_became_active":"2024-09-19T08:42:29.396798+0000","last_became_peered":"2024-09-19T08:42:29.396798+0000","last_unstale":"2024-09-19T08:42:31.402346+0000","last_undegraded":"2024-09-19T08:42:31.402346+0000","last_fullsized":"2024-09-19T08:42:31.402346+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:07.393673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.356804+0000","last_change":"2024-09-19T08:42:28.356804+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.356804+0000","last_undegraded":"2024-09-19T08:42:28.356804+0000","last_fullsized":"2024-09-19T08:42:28.356804+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401789+0000","last_change":"2024-09-19T08:42:29.395444+0000","last_active":"2024-09-19T08:42:31.401789+0000","last_peered":"2024-09-19T08:42:31.401789+0000","last_clean":"2024-09-19T08:42:31.401789+0000","last_became_active":"2024-09-19T08:42:29.394674+0000","last_became_peered":"2024-09-19T08:42:29.394674+0000","last_unstale":"2024-09-19T08:42:31.401789+0000","last_undegraded":"2024-09-19T08:42:31.401789+0000","last_fullsized":"2024-09-19T08:42:31.401789+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:42:57.681916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.360740+0000","last_change":"2024-09-19T08:42:28.360740+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.360740+0000","last_undegraded":"2024-09-19T08:42:28.360740+0000","last_fullsized":"2024-09-19T08:42:28.360740+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.349868+0000","last_change":"2024-09-19T08:42:28.349868+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.349868+0000","last_undegraded":"2024-09-19T08:42:28.349868+0000","last_fullsized":"2024-09-19T08:42:28.349868+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402779+0000","last_change":"2024-09-19T08:42:29.396548+0000","last_active":"2024-09-19T08:42:31.402779+0000","last_peered":"2024-09-19T08:42:31.402779+0000","last_clean":"2024-09-19T08:42:31.402779+0000","last_became_active":"2024-09-19T08:42:29.394896+0000","last_became_peered":"2024-09-19T08:42:29.394896+0000","last_unstale":"2024-09-19T08:42:31.402779+0000","last_undegraded":"2024-09-19T08:42:31.402779+0000","last_fullsized":"2024-09-19T08:42:31.402779+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:27:20.145977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.363524+0000","last_change":"2024-09-19T08:42:28.363524+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.363524+0000","last_undegraded":"2024-09-19T08:42:28.363524+0000","last_fullsized":"2024-09-19T08:42:28.363524+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.354289+0000","last_change":"2024-09-19T08:42:28.354289+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.354289+0000","last_undegraded":"2024-09-19T08:42:28.354289+0000","last_fullsized":"2024-09-19T08:42:28.354289+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.349910+0000","last_change":"2024-09-19T08:42:28.349910+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.349910+0000","last_undegraded":"2024-09-19T08:42:28.349910+0000","last_fullsized":"2024-09-19T08:42:28.349910+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.353322+0000","last_change":"2024-09-19T08:42:28.353322+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.353322+0000","last_undegraded":"2024-09-19T08:42:28.353322+0000","last_fullsized":"2024-09-19T08:42:28.353322+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401692+0000","last_change":"2024-09-19T08:42:30.459642+0000","last_active":"2024-09-19T08:42:31.401692+0000","last_peered":"2024-09-19T08:42:31.401692+0000","last_clean":"2024-09-19T08:42:31.401692+0000","last_became_active":"2024-09-19T08:42:30.459010+0000","last_became_peered":"2024-09-19T08:42:30.459010+0000","last_unstale":"2024-09-19T08:42:31.401692+0000","last_undegraded":"2024-09-19T08:42:31.401692+0000","last_fullsized":"2024-09-19T08:42:31.401692+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:03:33.044498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.356091+0000","last_change":"2024-09-19T08:42:28.356091+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.356091+0000","last_undegraded":"2024-09-19T08:42:28.356091+0000","last_fullsized":"2024-09-19T08:42:28.356091+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.349168+0000","last_change":"2024-09-19T08:42:28.349168+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.349168+0000","last_undegraded":"2024-09-19T08:42:28.349168+0000","last_fullsized":"2024-09-19T08:42:28.349168+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401444+0000","last_change":"2024-09-19T08:42:30.431276+0000","last_active":"2024-09-19T08:42:31.401444+0000","last_peered":"2024-09-19T08:42:31.401444+0000","last_clean":"2024-09-19T08:42:31.401444+0000","last_became_active":"2024-09-19T08:42:30.430462+0000","last_became_peered":"2024-09-19T08:42:30.430462+0000","last_unstale":"2024-09-19T08:42:31.401444+0000","last_undegraded":"2024-09-19T08:42:31.401444+0000","last_fullsized":"2024-09-19T08:42:31.401444+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:50.219137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401735+0000","last_change":"2024-09-19T08:42:29.373808+0000","last_active":"2024-09-19T08:42:31.401735+0000","last_peered":"2024-09-19T08:42:31.401735+0000","last_clean":"2024-09-19T08:42:31.401735+0000","last_became_active":"2024-09-19T08:42:29.373099+0000","last_became_peered":"2024-09-19T08:42:29.373099+0000","last_unstale":"2024-09-19T08:42:31.401735+0000","last_undegraded":"2024-09-19T08:42:31.401735+0000","last_fullsized":"2024-09-19T08:42:31.401735+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:48.444653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401741+0000","last_change":"2024-09-19T08:42:29.399988+0000","last_active":"2024-09-19T08:42:31.401741+0000","last_peered":"2024-09-19T08:42:31.401741+0000","last_clean":"2024-09-19T08:42:31.401741+0000","last_became_active":"2024-09-19T08:42:29.399260+0000","last_became_peered":"2024-09-19T08:42:29.399260+0000","last_unstale":"2024-09-19T08:42:31.401741+0000","last_undegraded":"2024-09-19T08:42:31.401741+0000","last_fullsized":"2024-09-19T08:42:31.401741+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:06.453660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":248,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T08:42:28.359131+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:42:28.359131+0000","last_peered":"2024-09-19T08:42:28.359131+0000","last_clean":"2024-09-19T08:42:28.359131+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:42:28.359131+0000","last_undegraded":"2024-09-19T08:42:28.359131+0000","last_fullsized":"2024-09-19T08:42:28.359131+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":1},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":92,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":60,"acting":60,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509495,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1376256,"data_stored":913734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509497,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":782336,"data_stored":323366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509497,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":782336,"data_stored":323366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509494,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":782336,"data_stored":323366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509496,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":782336,"data_stored":323366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509494,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1363968,"data_stored":907917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574904,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":827392,"data_stored":342808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542199,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31940,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339900,"statfs":{"total":96636764160,"available":96604057600,"internally_reserved":0,"allocated":770048,"data_stored":317549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:42:33.444 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:42:33.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.589+0000 7efeaf6f9640 1 Processor -- start 2024-09-19T08:42:33.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.589+0000 7efeaf6f9640 1 -- start start 2024-09-19T08:42:33.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.589+0000 7efeaf6f9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7efea806bc90 0x7efea806c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:33.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.589+0000 7efeaf6f9640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7efea80ce840 0x7efea80cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:33.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.589+0000 7efeaf6f9640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efea80ccc60 con 0x7efea8158fd0 2024-09-19T08:42:33.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.589+0000 7efeaf6f9640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7efea81593b0 con 0x7efea80ce840 2024-09-19T08:42:33.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.589+0000 7efeaf6f9640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7efea80ccf40 con 0x7efea806bc90 2024-09-19T08:42:33.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.589+0000 7efeadc96640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7efea80ce840 0x7efea80cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:33.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.589+0000 7efeadc96640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7efea80ce840 0x7efea80cec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34362/0 (socket says 172.21.15.98:34362) 2024-09-19T08:42:33.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.589+0000 7efeadc96640 1 -- 172.21.15.98:0/2645393595 learned_addr learned my addr 172.21.15.98:0/2645393595 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:33.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.589+0000 7efeacc94640 1 --2- 172.21.15.98:0/2645393595 >> v2:172.21.15.139:3300/0 conn(0x7efea806bc90 0x7efea806c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:33.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.589+0000 7efeadc96640 1 -- 172.21.15.98:0/2645393595 >> v2:172.21.15.139:3300/0 conn(0x7efea806bc90 msgr2=0x7efea806c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:33.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.589+0000 7efeadc96640 1 --2- 172.21.15.98:0/2645393595 >> v2:172.21.15.139:3300/0 conn(0x7efea806bc90 0x7efea806c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:33.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.589+0000 7efeadc96640 1 -- 172.21.15.98:0/2645393595 >> v1:172.21.15.139:6789/0 conn(0x7efea8158fd0 legacy=0x7efea80b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:33.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.589+0000 7efeadc96640 1 -- 172.21.15.98:0/2645393595 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efea80cd220 con 0x7efea80ce840 2024-09-19T08:42:33.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.593+0000 7efeadc96640 1 --2- 172.21.15.98:0/2645393595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efea80ce840 0x7efea80cec70 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7efea80cdef0 tx=0x7efea001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddfc483fc3e916b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:33.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.593+0000 7efe9ffff640 1 -- 172.21.15.98:0/2645393595 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea00a56d0 con 0x7efea80ce840 2024-09-19T08:42:33.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.593+0000 7efe9ffff640 1 -- 172.21.15.98:0/2645393595 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efea00a5870 con 0x7efea80ce840 2024-09-19T08:42:33.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.593+0000 7efe9ffff640 1 -- 172.21.15.98:0/2645393595 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea00a5c40 con 0x7efea80ce840 2024-09-19T08:42:33.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.593+0000 7efeaf6f9640 1 -- 172.21.15.98:0/2645393595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efea80ce840 msgr2=0x7efea80cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:33.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.593+0000 7efeaf6f9640 1 --2- 172.21.15.98:0/2645393595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efea80ce840 0x7efea80cec70 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7efea80cdef0 tx=0x7efea001c920 comp rx=0 tx=0).stop 2024-09-19T08:42:33.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.593+0000 7efeaf6f9640 1 -- 172.21.15.98:0/2645393595 shutdown_connections 2024-09-19T08:42:33.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.593+0000 7efeaf6f9640 1 --2- 172.21.15.98:0/2645393595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efea80ce840 0x7efea80cec70 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:33.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.593+0000 7efeaf6f9640 1 --2- 172.21.15.98:0/2645393595 >> v2:172.21.15.139:3300/0 conn(0x7efea806bc90 0x7efea806c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:33.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.593+0000 7efeaf6f9640 1 -- 172.21.15.98:0/2645393595 >> 172.21.15.98:0/2645393595 conn(0x7efea80b2010 msgr2=0x7efea80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:33.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.593+0000 7efeaf6f9640 1 -- 172.21.15.98:0/2645393595 shutdown_connections 2024-09-19T08:42:33.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.593+0000 7efeaf6f9640 1 -- 172.21.15.98:0/2645393595 wait complete. 2024-09-19T08:42:33.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.593+0000 7efeaf6f9640 1 Processor -- start 2024-09-19T08:42:33.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.593+0000 7efeaf6f9640 1 -- start start 2024-09-19T08:42:33.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.593+0000 7efeaf6f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efea806bc90 0x7efea8163750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:33.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.593+0000 7efeaf6f9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efea80ce840 0x7efea8157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:33.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.593+0000 7efeaf6f9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efea8158fd0 0x7efea815a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:33.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.593+0000 7efeaf6f9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efea815c920 con 0x7efea80ce840 2024-09-19T08:42:33.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.593+0000 7efeaf6f9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efea815c7a0 con 0x7efea806bc90 2024-09-19T08:42:33.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.593+0000 7efeaf6f9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7efea8168110 con 0x7efea8158fd0 2024-09-19T08:42:33.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.593+0000 7efeacc94640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efea80ce840 0x7efea8157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:33.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.593+0000 7efeacc94640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efea80ce840 0x7efea8157410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34368/0 (socket says 172.21.15.98:34368) 2024-09-19T08:42:33.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.593+0000 7efeacc94640 1 -- 172.21.15.98:0/320018687 learned_addr learned my addr 172.21.15.98:0/320018687 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:33.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.593+0000 7efead495640 1 --2- 172.21.15.98:0/320018687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efea806bc90 0x7efea8163750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:33.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.593+0000 7efeadc96640 1 --2- 172.21.15.98:0/320018687 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efea8158fd0 0x7efea815a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:33.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.593+0000 7efeacc94640 1 -- 172.21.15.98:0/320018687 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efea8158fd0 msgr2=0x7efea815a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:33.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.593+0000 7efeacc94640 1 --2- 172.21.15.98:0/320018687 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efea8158fd0 0x7efea815a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:33.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.593+0000 7efeacc94640 1 -- 172.21.15.98:0/320018687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efea806bc90 msgr2=0x7efea8163750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:33.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.593+0000 7efeacc94640 1 --2- 172.21.15.98:0/320018687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efea806bc90 0x7efea8163750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:33.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.593+0000 7efeacc94640 1 -- 172.21.15.98:0/320018687 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efea8141c90 con 0x7efea80ce840 2024-09-19T08:42:33.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.593+0000 7efeacc94640 1 --2- 172.21.15.98:0/320018687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efea80ce840 0x7efea8157410 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7efe9800c820 tx=0x7efe9800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:33.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.593+0000 7efe9dffb640 1 -- 172.21.15.98:0/320018687 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe98015070 con 0x7efea80ce840 2024-09-19T08:42:33.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.597+0000 7efe9dffb640 1 -- 172.21.15.98:0/320018687 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe98004020 con 0x7efea80ce840 2024-09-19T08:42:33.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.597+0000 7efeaf6f9640 1 -- 172.21.15.98:0/320018687 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efea8141f70 con 0x7efea80ce840 2024-09-19T08:42:33.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.597+0000 7efe9dffb640 1 -- 172.21.15.98:0/320018687 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe98004410 con 0x7efea80ce840 2024-09-19T08:42:33.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.597+0000 7efeaf6f9640 1 -- 172.21.15.98:0/320018687 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efea8142530 con 0x7efea80ce840 2024-09-19T08:42:33.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.597+0000 7efeaf6f9640 1 -- 172.21.15.98:0/320018687 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efea806c120 con 0x7efea80ce840 2024-09-19T08:42:33.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.597+0000 7efe9dffb640 1 -- 172.21.15.98:0/320018687 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7efe98007830 con 0x7efea80ce840 2024-09-19T08:42:33.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.597+0000 7efe9dffb640 1 --2- 172.21.15.98:0/320018687 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7efe8c067c20 0x7efe8c06a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:33.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.597+0000 7efe9dffb640 1 -- 172.21.15.98:0/320018687 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7efe98086170 con 0x7efea80ce840 2024-09-19T08:42:33.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.597+0000 7efead495640 1 --2- 172.21.15.98:0/320018687 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7efe8c067c20 0x7efe8c06a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:33.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.597+0000 7efead495640 1 --2- 172.21.15.98:0/320018687 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7efe8c067c20 0x7efe8c06a0e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7efea40016c0 tx=0x7efea4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:33.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.601+0000 7efe9dffb640 1 -- 172.21.15.98:0/320018687 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7efe98056700 con 0x7efea80ce840 2024-09-19T08:42:33.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.745+0000 7efeaf6f9640 1 -- 172.21.15.98:0/320018687 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efea8157f90 con 0x7efe8c067c20 2024-09-19T08:42:33.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.757+0000 7efe9dffb640 1 -- 172.21.15.98:0/320018687 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241995 (secure 0 0 0) 0x7efea8157f90 con 0x7efe8c067c20 2024-09-19T08:42:33.760 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:42:33.761 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:42:33.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.761+0000 7efeaf6f9640 1 -- 172.21.15.98:0/320018687 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7efe8c067c20 msgr2=0x7efe8c06a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:33.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.761+0000 7efeaf6f9640 1 --2- 172.21.15.98:0/320018687 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7efe8c067c20 0x7efe8c06a0e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7efea40016c0 tx=0x7efea4009040 comp rx=0 tx=0).stop 2024-09-19T08:42:33.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.761+0000 7efeaf6f9640 1 -- 172.21.15.98:0/320018687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efea80ce840 msgr2=0x7efea8157410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:33.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.761+0000 7efeaf6f9640 1 --2- 172.21.15.98:0/320018687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efea80ce840 0x7efea8157410 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7efe9800c820 tx=0x7efe9800cce0 comp rx=0 tx=0).stop 2024-09-19T08:42:33.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.761+0000 7efeaf6f9640 1 -- 172.21.15.98:0/320018687 shutdown_connections 2024-09-19T08:42:33.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.761+0000 7efeaf6f9640 1 --2- 172.21.15.98:0/320018687 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7efe8c067c20 0x7efe8c06a0e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:33.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.761+0000 7efeaf6f9640 1 --2- 172.21.15.98:0/320018687 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efea8158fd0 0x7efea815a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:33.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.761+0000 7efeaf6f9640 1 --2- 172.21.15.98:0/320018687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efea80ce840 0x7efea8157410 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:33.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.761+0000 7efeaf6f9640 1 --2- 172.21.15.98:0/320018687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efea806bc90 0x7efea8163750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:33.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.761+0000 7efeaf6f9640 1 -- 172.21.15.98:0/320018687 >> 172.21.15.98:0/320018687 conn(0x7efea80b2010 msgr2=0x7efea80b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:33.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.761+0000 7efeaf6f9640 1 -- 172.21.15.98:0/320018687 shutdown_connections 2024-09-19T08:42:33.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.761+0000 7efeaf6f9640 1 -- 172.21.15.98:0/320018687 wait complete. 2024-09-19T08:42:33.777 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2024-09-19T08:42:32.110399+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590460,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":168,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126,"ondisk_log_size":126,"up":80,"acting":80,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":256832,"kb_used_data":7292,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717888,"statfs":{"total":773094113280,"available":772831117312,"internally_reserved":0,"allocated":7467008,"data_stored":3775472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":92,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001134"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402870+0000","last_change":"2024-09-19T08:42:30.446888+0000","last_active":"2024-09-19T08:42:31.402870+0000","last_peered":"2024-09-19T08:42:31.402870+0000","last_clean":"2024-09-19T08:42:31.402870+0000","last_became_active":"2024-09-19T08:42:30.438820+0000","last_became_peered":"2024-09-19T08:42:30.438820+0000","last_unstale":"2024-09-19T08:42:31.402870+0000","last_undegraded":"2024-09-19T08:42:31.402870+0000","last_fullsized":"2024-09-19T08:42:31.402870+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:52:02.376361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401471+0000","last_change":"2024-09-19T08:42:30.431823+0000","last_active":"2024-09-19T08:42:31.401471+0000","last_peered":"2024-09-19T08:42:31.401471+0000","last_clean":"2024-09-19T08:42:31.401471+0000","last_became_active":"2024-09-19T08:42:30.430962+0000","last_became_peered":"2024-09-19T08:42:30.430962+0000","last_unstale":"2024-09-19T08:42:31.401471+0000","last_undegraded":"2024-09-19T08:42:31.401471+0000","last_fullsized":"2024-09-19T08:42:31.401471+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:55.760155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401494+0000","last_change":"2024-09-19T08:42:30.438133+0000","last_active":"2024-09-19T08:42:31.401494+0000","last_peered":"2024-09-19T08:42:31.401494+0000","last_clean":"2024-09-19T08:42:31.401494+0000","last_became_active":"2024-09-19T08:42:30.437186+0000","last_became_peered":"2024-09-19T08:42:30.437186+0000","last_unstale":"2024-09-19T08:42:31.401494+0000","last_undegraded":"2024-09-19T08:42:31.401494+0000","last_fullsized":"2024-09-19T08:42:31.401494+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:55.764601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402257+0000","last_change":"2024-09-19T08:42:30.439106+0000","last_active":"2024-09-19T08:42:31.402257+0000","last_peered":"2024-09-19T08:42:31.402257+0000","last_clean":"2024-09-19T08:42:31.402257+0000","last_became_active":"2024-09-19T08:42:30.438367+0000","last_became_peered":"2024-09-19T08:42:30.438367+0000","last_unstale":"2024-09-19T08:42:31.402257+0000","last_undegraded":"2024-09-19T08:42:31.402257+0000","last_fullsized":"2024-09-19T08:42:31.402257+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:14:25.294258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401507+0000","last_change":"2024-09-19T08:42:30.434665+0000","last_active":"2024-09-19T08:42:31.401507+0000","last_peered":"2024-09-19T08:42:31.401507+0000","last_clean":"2024-09-19T08:42:31.401507+0000","last_became_active":"2024-09-19T08:42:30.433735+0000","last_became_peered":"2024-09-19T08:42:30.433735+0000","last_unstale":"2024-09-19T08:42:31.401507+0000","last_undegraded":"2024-09-19T08:42:31.401507+0000","last_fullsized":"2024-09-19T08:42:31.401507+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:37:26.942476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.356233+0000","last_change":"2024-09-19T08:42:28.356233+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.356233+0000","last_undegraded":"2024-09-19T08:42:28.356233+0000","last_fullsized":"2024-09-19T08:42:28.356233+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.349928+0000","last_change":"2024-09-19T08:42:28.349928+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.349928+0000","last_undegraded":"2024-09-19T08:42:28.349928+0000","last_fullsized":"2024-09-19T08:42:28.349928+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402198+0000","last_change":"2024-09-19T08:42:30.458129+0000","last_active":"2024-09-19T08:42:31.402198+0000","last_peered":"2024-09-19T08:42:31.402198+0000","last_clean":"2024-09-19T08:42:31.402198+0000","last_became_active":"2024-09-19T08:42:30.456984+0000","last_became_peered":"2024-09-19T08:42:30.456984+0000","last_unstale":"2024-09-19T08:42:31.402198+0000","last_undegraded":"2024-09-19T08:42:31.402198+0000","last_fullsized":"2024-09-19T08:42:31.402198+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:43:27.676061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.351206+0000","last_change":"2024-09-19T08:42:28.351206+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.351206+0000","last_undegraded":"2024-09-19T08:42:28.351206+0000","last_fullsized":"2024-09-19T08:42:28.351206+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402459+0000","last_change":"2024-09-19T08:42:29.395380+0000","last_active":"2024-09-19T08:42:31.402459+0000","last_peered":"2024-09-19T08:42:31.402459+0000","last_clean":"2024-09-19T08:42:31.402459+0000","last_became_active":"2024-09-19T08:42:29.394641+0000","last_became_peered":"2024-09-19T08:42:29.394641+0000","last_unstale":"2024-09-19T08:42:31.402459+0000","last_undegraded":"2024-09-19T08:42:31.402459+0000","last_fullsized":"2024-09-19T08:42:31.402459+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:26.586550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:28.334315+0000","last_change":"2024-09-19T08:42:28.334315+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.334315+0000","last_undegraded":"2024-09-19T08:42:28.334315+0000","last_fullsized":"2024-09-19T08:42:28.334315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401528+0000","last_change":"2024-09-19T08:42:29.399398+0000","last_active":"2024-09-19T08:42:31.401528+0000","last_peered":"2024-09-19T08:42:31.401528+0000","last_clean":"2024-09-19T08:42:31.401528+0000","last_became_active":"2024-09-19T08:42:29.396716+0000","last_became_peered":"2024-09-19T08:42:29.396716+0000","last_unstale":"2024-09-19T08:42:31.401528+0000","last_undegraded":"2024-09-19T08:42:31.401528+0000","last_fullsized":"2024-09-19T08:42:31.401528+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:50:23.952850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.355932+0000","last_change":"2024-09-19T08:42:28.355932+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.355932+0000","last_undegraded":"2024-09-19T08:42:28.355932+0000","last_fullsized":"2024-09-19T08:42:28.355932+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.353311+0000","last_change":"2024-09-19T08:42:28.353311+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.353311+0000","last_undegraded":"2024-09-19T08:42:28.353311+0000","last_fullsized":"2024-09-19T08:42:28.353311+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.349295+0000","last_change":"2024-09-19T08:42:28.349295+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.349295+0000","last_undegraded":"2024-09-19T08:42:28.349295+0000","last_fullsized":"2024-09-19T08:42:28.349295+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402803+0000","last_change":"2024-09-19T08:42:29.399190+0000","last_active":"2024-09-19T08:42:31.402803+0000","last_peered":"2024-09-19T08:42:31.402803+0000","last_clean":"2024-09-19T08:42:31.402803+0000","last_became_active":"2024-09-19T08:42:29.396183+0000","last_became_peered":"2024-09-19T08:42:29.396183+0000","last_unstale":"2024-09-19T08:42:31.402803+0000","last_undegraded":"2024-09-19T08:42:31.402803+0000","last_fullsized":"2024-09-19T08:42:31.402803+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:21:01.013528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402155+0000","last_change":"2024-09-19T08:42:30.451725+0000","last_active":"2024-09-19T08:42:31.402155+0000","last_peered":"2024-09-19T08:42:31.402155+0000","last_clean":"2024-09-19T08:42:31.402155+0000","last_became_active":"2024-09-19T08:42:30.437086+0000","last_became_peered":"2024-09-19T08:42:30.437086+0000","last_unstale":"2024-09-19T08:42:31.402155+0000","last_undegraded":"2024-09-19T08:42:31.402155+0000","last_fullsized":"2024-09-19T08:42:31.402155+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:34.398500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:28.334315+0000","last_change":"2024-09-19T08:42:28.334315+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.334315+0000","last_undegraded":"2024-09-19T08:42:28.334315+0000","last_fullsized":"2024-09-19T08:42:28.334315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.355876+0000","last_change":"2024-09-19T08:42:28.355876+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.355876+0000","last_undegraded":"2024-09-19T08:42:28.355876+0000","last_fullsized":"2024-09-19T08:42:28.355876+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.355949+0000","last_change":"2024-09-19T08:42:28.355949+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.355949+0000","last_undegraded":"2024-09-19T08:42:28.355949+0000","last_fullsized":"2024-09-19T08:42:28.355949+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401738+0000","last_change":"2024-09-19T08:42:29.399130+0000","last_active":"2024-09-19T08:42:31.401738+0000","last_peered":"2024-09-19T08:42:31.401738+0000","last_clean":"2024-09-19T08:42:31.401738+0000","last_became_active":"2024-09-19T08:42:29.396039+0000","last_became_peered":"2024-09-19T08:42:29.396039+0000","last_unstale":"2024-09-19T08:42:31.401738+0000","last_undegraded":"2024-09-19T08:42:31.401738+0000","last_fullsized":"2024-09-19T08:42:31.401738+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:54.277351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.361851+0000","last_change":"2024-09-19T08:42:28.361851+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.361851+0000","last_undegraded":"2024-09-19T08:42:28.361851+0000","last_fullsized":"2024-09-19T08:42:28.361851+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.348977+0000","last_change":"2024-09-19T08:42:28.348977+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.348977+0000","last_undegraded":"2024-09-19T08:42:28.348977+0000","last_fullsized":"2024-09-19T08:42:28.348977+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402598+0000","last_change":"2024-09-19T08:42:29.400466+0000","last_active":"2024-09-19T08:42:31.402598+0000","last_peered":"2024-09-19T08:42:31.402598+0000","last_clean":"2024-09-19T08:42:31.402598+0000","last_became_active":"2024-09-19T08:42:29.395739+0000","last_became_peered":"2024-09-19T08:42:29.395739+0000","last_unstale":"2024-09-19T08:42:31.402598+0000","last_undegraded":"2024-09-19T08:42:31.402598+0000","last_fullsized":"2024-09-19T08:42:31.402598+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:41:59.302829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.349238+0000","last_change":"2024-09-19T08:42:28.349238+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.349238+0000","last_undegraded":"2024-09-19T08:42:28.349238+0000","last_fullsized":"2024-09-19T08:42:28.349238+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402346+0000","last_change":"2024-09-19T08:42:29.399885+0000","last_active":"2024-09-19T08:42:31.402346+0000","last_peered":"2024-09-19T08:42:31.402346+0000","last_clean":"2024-09-19T08:42:31.402346+0000","last_became_active":"2024-09-19T08:42:29.396798+0000","last_became_peered":"2024-09-19T08:42:29.396798+0000","last_unstale":"2024-09-19T08:42:31.402346+0000","last_undegraded":"2024-09-19T08:42:31.402346+0000","last_fullsized":"2024-09-19T08:42:31.402346+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:07.393673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.356804+0000","last_change":"2024-09-19T08:42:28.356804+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.356804+0000","last_undegraded":"2024-09-19T08:42:28.356804+0000","last_fullsized":"2024-09-19T08:42:28.356804+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401789+0000","last_change":"2024-09-19T08:42:29.395444+0000","last_active":"2024-09-19T08:42:31.401789+0000","last_peered":"2024-09-19T08:42:31.401789+0000","last_clean":"2024-09-19T08:42:31.401789+0000","last_became_active":"2024-09-19T08:42:29.394674+0000","last_became_peered":"2024-09-19T08:42:29.394674+0000","last_unstale":"2024-09-19T08:42:31.401789+0000","last_undegraded":"2024-09-19T08:42:31.401789+0000","last_fullsized":"2024-09-19T08:42:31.401789+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:42:57.681916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.360740+0000","last_change":"2024-09-19T08:42:28.360740+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.360740+0000","last_undegraded":"2024-09-19T08:42:28.360740+0000","last_fullsized":"2024-09-19T08:42:28.360740+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.349868+0000","last_change":"2024-09-19T08:42:28.349868+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.349868+0000","last_undegraded":"2024-09-19T08:42:28.349868+0000","last_fullsized":"2024-09-19T08:42:28.349868+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402779+0000","last_change":"2024-09-19T08:42:29.396548+0000","last_active":"2024-09-19T08:42:31.402779+0000","last_peered":"2024-09-19T08:42:31.402779+0000","last_clean":"2024-09-19T08:42:31.402779+0000","last_became_active":"2024-09-19T08:42:29.394896+0000","last_became_peered":"2024-09-19T08:42:29.394896+0000","last_unstale":"2024-09-19T08:42:31.402779+0000","last_undegraded":"2024-09-19T08:42:31.402779+0000","last_fullsized":"2024-09-19T08:42:31.402779+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:27:20.145977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.363524+0000","last_change":"2024-09-19T08:42:28.363524+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.363524+0000","last_undegraded":"2024-09-19T08:42:28.363524+0000","last_fullsized":"2024-09-19T08:42:28.363524+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.354289+0000","last_change":"2024-09-19T08:42:28.354289+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.354289+0000","last_undegraded":"2024-09-19T08:42:28.354289+0000","last_fullsized":"2024-09-19T08:42:28.354289+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.349910+0000","last_change":"2024-09-19T08:42:28.349910+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.349910+0000","last_undegraded":"2024-09-19T08:42:28.349910+0000","last_fullsized":"2024-09-19T08:42:28.349910+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.353322+0000","last_change":"2024-09-19T08:42:28.353322+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.353322+0000","last_undegraded":"2024-09-19T08:42:28.353322+0000","last_fullsized":"2024-09-19T08:42:28.353322+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401692+0000","last_change":"2024-09-19T08:42:30.459642+0000","last_active":"2024-09-19T08:42:31.401692+0000","last_peered":"2024-09-19T08:42:31.401692+0000","last_clean":"2024-09-19T08:42:31.401692+0000","last_became_active":"2024-09-19T08:42:30.459010+0000","last_became_peered":"2024-09-19T08:42:30.459010+0000","last_unstale":"2024-09-19T08:42:31.401692+0000","last_undegraded":"2024-09-19T08:42:31.401692+0000","last_fullsized":"2024-09-19T08:42:31.401692+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:03:33.044498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.356091+0000","last_change":"2024-09-19T08:42:28.356091+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.356091+0000","last_undegraded":"2024-09-19T08:42:28.356091+0000","last_fullsized":"2024-09-19T08:42:28.356091+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.349168+0000","last_change":"2024-09-19T08:42:28.349168+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.349168+0000","last_undegraded":"2024-09-19T08:42:28.349168+0000","last_fullsized":"2024-09-19T08:42:28.349168+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401444+0000","last_change":"2024-09-19T08:42:30.431276+0000","last_active":"2024-09-19T08:42:31.401444+0000","last_peered":"2024-09-19T08:42:31.401444+0000","last_clean":"2024-09-19T08:42:31.401444+0000","last_became_active":"2024-09-19T08:42:30.430462+0000","last_became_peered":"2024-09-19T08:42:30.430462+0000","last_unstale":"2024-09-19T08:42:31.401444+0000","last_undegraded":"2024-09-19T08:42:31.401444+0000","last_fullsized":"2024-09-19T08:42:31.401444+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:50.219137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401735+0000","last_change":"2024-09-19T08:42:29.373808+0000","last_active":"2024-09-19T08:42:31.401735+0000","last_peered":"2024-09-19T08:42:31.401735+0000","last_clean":"2024-09-19T08:42:31.401735+0000","last_became_active":"2024-09-19T08:42:29.373099+0000","last_became_peered":"2024-09-19T08:42:29.373099+0000","last_unstale":"2024-09-19T08:42:31.401735+0000","last_undegraded":"2024-09-19T08:42:31.401735+0000","last_fullsized":"2024-09-19T08:42:31.401735+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:48.444653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401741+0000","last_change":"2024-09-19T08:42:29.399988+0000","last_active":"2024-09-19T08:42:31.401741+0000","last_peered":"2024-09-19T08:42:31.401741+0000","last_clean":"2024-09-19T08:42:31.401741+0000","last_became_active":"2024-09-19T08:42:29.399260+0000","last_became_peered":"2024-09-19T08:42:29.399260+0000","last_unstale":"2024-09-19T08:42:31.401741+0000","last_undegraded":"2024-09-19T08:42:31.401741+0000","last_fullsized":"2024-09-19T08:42:31.401741+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:06.453660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":248,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T08:42:28.359131+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:42:28.359131+0000","last_peered":"2024-09-19T08:42:28.359131+0000","last_clean":"2024-09-19T08:42:28.359131+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:42:28.359131+0000","last_undegraded":"2024-09-19T08:42:28.359131+0000","last_fullsized":"2024-09-19T08:42:28.359131+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":1},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":92,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":60,"acting":60,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509495,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1376256,"data_stored":913734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509497,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":782336,"data_stored":323366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509497,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":782336,"data_stored":323366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509494,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":782336,"data_stored":323366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509496,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":782336,"data_stored":323366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509494,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1363968,"data_stored":907917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574904,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":827392,"data_stored":342808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542199,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31940,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339900,"statfs":{"total":96636764160,"available":96604057600,"internally_reserved":0,"allocated":770048,"data_stored":317549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:42:33.780 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:42:33.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.929+0000 7f75e9525640 1 Processor -- start 2024-09-19T08:42:33.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.929+0000 7f75e9525640 1 -- start start 2024-09-19T08:42:33.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.933+0000 7f75e9525640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75e4158fd0 0x7f75e40b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:33.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.933+0000 7f75e9525640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f75e406c3a0 0x7f75e40ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:33.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.933+0000 7f75e9525640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75e40ccc60 con 0x7f75e40b4360 2024-09-19T08:42:33.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.933+0000 7f75e9525640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f75e41593b0 con 0x7f75e406c3a0 2024-09-19T08:42:33.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.933+0000 7f75e9525640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f75e40ccf40 con 0x7f75e4158fd0 2024-09-19T08:42:33.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.933+0000 7f75e27fc640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f75e40b4360 0x7f75e406bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41338/0 (socket says 172.21.15.98:41338) 2024-09-19T08:42:33.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.933+0000 7f75e27fc640 1 -- 172.21.15.98:0/2655848416 learned_addr learned my addr 172.21.15.98:0/2655848416 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:33.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.933+0000 7f75e2ffd640 1 --2- 172.21.15.98:0/2655848416 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75e4158fd0 0x7f75e40b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:33.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.933+0000 7f75e37fe640 1 --2- 172.21.15.98:0/2655848416 >> v2:172.21.15.139:3300/0 conn(0x7f75e406c3a0 0x7f75e40ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:33.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.933+0000 7f75e1ffb640 1 -- 172.21.15.98:0/2655848416 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1412417354 0 0) 0x7f75e40ccc60 con 0x7f75e40b4360 2024-09-19T08:42:33.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.933+0000 7f75e1ffb640 1 -- 172.21.15.98:0/2655848416 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75cc003530 con 0x7f75e40b4360 2024-09-19T08:42:33.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.933+0000 7f75e1ffb640 1 -- 172.21.15.98:0/2655848416 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 624298554 0 0) 0x7f75cc003530 con 0x7f75e40b4360 2024-09-19T08:42:33.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.933+0000 7f75e1ffb640 1 -- 172.21.15.98:0/2655848416 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75e4158fd0 msgr2=0x7f75e40b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:33.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.933+0000 7f75e1ffb640 1 --2- 172.21.15.98:0/2655848416 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75e4158fd0 0x7f75e40b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:33.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.933+0000 7f75e1ffb640 1 -- 172.21.15.98:0/2655848416 >> v2:172.21.15.139:3300/0 conn(0x7f75e406c3a0 msgr2=0x7f75e40ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:33.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.933+0000 7f75e1ffb640 1 --2- 172.21.15.98:0/2655848416 >> v2:172.21.15.139:3300/0 conn(0x7f75e406c3a0 0x7f75e40ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:33.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.933+0000 7f75e1ffb640 1 -- 172.21.15.98:0/2655848416 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75e40cd220 con 0x7f75e40b4360 2024-09-19T08:42:33.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.933+0000 7f75e1ffb640 1 -- 172.21.15.98:0/2655848416 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f75d8001680 con 0x7f75e40b4360 2024-09-19T08:42:33.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.933+0000 7f75e1ffb640 1 -- 172.21.15.98:0/2655848416 >> v1:172.21.15.98:6789/0 conn(0x7f75e40b4360 legacy=0x7f75e406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:33.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.933+0000 7f75e1ffb640 1 --2- 172.21.15.98:0/2655848416 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75e4158fd0 0x7f75cc0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:33.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.933+0000 7f75e1ffb640 1 --2- 172.21.15.98:0/2655848416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75cc003f40 0x7f75cc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:33.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.933+0000 7f75e1ffb640 1 --2- 172.21.15.98:0/2655848416 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75cc004d50 0x7f75cc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:33.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.933+0000 7f75e1ffb640 1 -- 172.21.15.98:0/2655848416 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f75e41593b0 con 0x7f75cc003f40 2024-09-19T08:42:33.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.933+0000 7f75e1ffb640 1 -- 172.21.15.98:0/2655848416 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f75e40ccf40 con 0x7f75cc004d50 2024-09-19T08:42:33.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.933+0000 7f75e1ffb640 1 -- 172.21.15.98:0/2655848416 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f75cc00ba20 con 0x7f75e4158fd0 2024-09-19T08:42:33.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.933+0000 7f75e27fc640 1 --2- 172.21.15.98:0/2655848416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75cc003f40 0x7f75cc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:33.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.933+0000 7f75e2ffd640 1 --2- 172.21.15.98:0/2655848416 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75e4158fd0 0x7f75cc0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:33.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.933+0000 7f75e37fe640 1 --2- 172.21.15.98:0/2655848416 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75cc004d50 0x7f75cc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:33.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.933+0000 7f75e27fc640 1 -- 172.21.15.98:0/2655848416 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75e4158fd0 msgr2=0x7f75cc0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:33.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.933+0000 7f75e27fc640 1 --2- 172.21.15.98:0/2655848416 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75e4158fd0 0x7f75cc0039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:33.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.933+0000 7f75e27fc640 1 -- 172.21.15.98:0/2655848416 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75cc004d50 msgr2=0x7f75cc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:33.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.933+0000 7f75e27fc640 1 --2- 172.21.15.98:0/2655848416 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75cc004d50 0x7f75cc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:33.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.933+0000 7f75e27fc640 1 -- 172.21.15.98:0/2655848416 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75cc003530 con 0x7f75cc003f40 2024-09-19T08:42:33.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.933+0000 7f75e37fe640 1 --2- 172.21.15.98:0/2655848416 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75cc004d50 0x7f75cc0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:42:33.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.937+0000 7f75e27fc640 1 --2- 172.21.15.98:0/2655848416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75cc003f40 0x7f75cc004810 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f75d800c210 tx=0x7f75d801cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926034b18ac8717f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:33.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.937+0000 7f75e1ffb640 1 -- 172.21.15.98:0/2655848416 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d8007d60 con 0x7f75cc003f40 2024-09-19T08:42:33.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.937+0000 7f75e1ffb640 1 -- 172.21.15.98:0/2655848416 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75d80a9020 con 0x7f75cc003f40 2024-09-19T08:42:33.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.937+0000 7f75e1ffb640 1 -- 172.21.15.98:0/2655848416 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d8005b20 con 0x7f75cc003f40 2024-09-19T08:42:33.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.937+0000 7f75e9525640 1 -- 172.21.15.98:0/2655848416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75cc003f40 msgr2=0x7f75cc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:33.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.937+0000 7f75e9525640 1 --2- 172.21.15.98:0/2655848416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75cc003f40 0x7f75cc004810 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f75d800c210 tx=0x7f75d801cc30 comp rx=0 tx=0).stop 2024-09-19T08:42:33.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.937+0000 7f75e37fe640 1 -- 172.21.15.98:0/2655848416 reap_dead start 2024-09-19T08:42:33.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.937+0000 7f75e9525640 1 -- 172.21.15.98:0/2655848416 shutdown_connections 2024-09-19T08:42:33.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.937+0000 7f75e9525640 1 -- 172.21.15.98:0/2655848416 >> 172.21.15.98:0/2655848416 conn(0x7f75e40b2010 msgr2=0x7f75e40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:33.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.937+0000 7f75e9525640 1 -- 172.21.15.98:0/2655848416 shutdown_connections 2024-09-19T08:42:33.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.937+0000 7f75e9525640 1 -- 172.21.15.98:0/2655848416 wait complete. 2024-09-19T08:42:33.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.937+0000 7f75e9525640 1 Processor -- start 2024-09-19T08:42:33.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.937+0000 7f75e9525640 1 -- start start 2024-09-19T08:42:33.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.937+0000 7f75e9525640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75cc003f40 0x7f75e40b1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:33.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.937+0000 7f75e9525640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75e4151f00 0x7f75e40ad0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:33.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.937+0000 7f75e9525640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75e4151940 0x7f75e4154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:33.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.937+0000 7f75e9525640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f75e40cf710 con 0x7f75cc003f40 2024-09-19T08:42:33.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.937+0000 7f75e9525640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f75e40cf590 con 0x7f75e4151f00 2024-09-19T08:42:33.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.937+0000 7f75e9525640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f75e40cf890 con 0x7f75e4151940 2024-09-19T08:42:33.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.937+0000 7f75e2ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75cc003f40 0x7f75e40b1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:33.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.937+0000 7f75e2ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75cc003f40 0x7f75e40b1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34396/0 (socket says 172.21.15.98:34396) 2024-09-19T08:42:33.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.937+0000 7f75e2ffd640 1 -- 172.21.15.98:0/1682792635 learned_addr learned my addr 172.21.15.98:0/1682792635 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:33.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.937+0000 7f75e27fc640 1 --2- 172.21.15.98:0/1682792635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75e4151f00 0x7f75e40ad0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:33.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.937+0000 7f75e37fe640 1 --2- 172.21.15.98:0/1682792635 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75e4151940 0x7f75e4154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:33.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.937+0000 7f75e2ffd640 1 -- 172.21.15.98:0/1682792635 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75e4151940 msgr2=0x7f75e4154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:33.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.937+0000 7f75e2ffd640 1 --2- 172.21.15.98:0/1682792635 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75e4151940 0x7f75e4154d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:33.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.937+0000 7f75e2ffd640 1 -- 172.21.15.98:0/1682792635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75e4151f00 msgr2=0x7f75e40ad0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:33.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.937+0000 7f75e2ffd640 1 --2- 172.21.15.98:0/1682792635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75e4151f00 0x7f75e40ad0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:33.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.937+0000 7f75e2ffd640 1 -- 172.21.15.98:0/1682792635 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75e41f9b80 con 0x7f75cc003f40 2024-09-19T08:42:33.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.937+0000 7f75e37fe640 1 --2- 172.21.15.98:0/1682792635 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75e4151940 0x7f75e4154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:42:33.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.941+0000 7f75e2ffd640 1 --2- 172.21.15.98:0/1682792635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75cc003f40 0x7f75e40b1190 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f75d400f350 tx=0x7f75d400f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:33.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.941+0000 7f75cb7fe640 1 -- 172.21.15.98:0/1682792635 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d401c070 con 0x7f75cc003f40 2024-09-19T08:42:33.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.941+0000 7f75cb7fe640 1 -- 172.21.15.98:0/1682792635 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75d4017620 con 0x7f75cc003f40 2024-09-19T08:42:33.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.941+0000 7f75e9525640 1 -- 172.21.15.98:0/1682792635 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75e40ac310 con 0x7f75cc003f40 2024-09-19T08:42:33.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.941+0000 7f75cb7fe640 1 -- 172.21.15.98:0/1682792635 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d4017a10 con 0x7f75cc003f40 2024-09-19T08:42:33.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.941+0000 7f75e9525640 1 -- 172.21.15.98:0/1682792635 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75e40ac8d0 con 0x7f75cc003f40 2024-09-19T08:42:33.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.941+0000 7f75e9525640 1 -- 172.21.15.98:0/1682792635 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75e40ce840 con 0x7f75cc003f40 2024-09-19T08:42:33.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.941+0000 7f75cb7fe640 1 -- 172.21.15.98:0/1682792635 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f75d401f070 con 0x7f75cc003f40 2024-09-19T08:42:33.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.941+0000 7f75cb7fe640 1 --2- 172.21.15.98:0/1682792635 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f75d0067c50 0x7f75d006a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:33.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.941+0000 7f75cb7fe640 1 -- 172.21.15.98:0/1682792635 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f75d4086ec0 con 0x7f75cc003f40 2024-09-19T08:42:33.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.941+0000 7f75e27fc640 1 --2- 172.21.15.98:0/1682792635 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f75d0067c50 0x7f75d006a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:33.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.941+0000 7f75e27fc640 1 --2- 172.21.15.98:0/1682792635 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f75d0067c50 0x7f75d006a110 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f75d80aa000 tx=0x7f75d8000ff0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:33.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:33.945+0000 7f75cb7fe640 1 -- 172.21.15.98:0/1682792635 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f75d4026020 con 0x7f75cc003f40 2024-09-19T08:42:34.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.101+0000 7f75e9525640 1 -- 172.21.15.98:0/1682792635 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f75e40b3580 con 0x7f75d0067c50 2024-09-19T08:42:34.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.117+0000 7f75cb7fe640 1 -- 172.21.15.98:0/1682792635 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241995 (secure 0 0 0) 0x7f75e40b3580 con 0x7f75d0067c50 2024-09-19T08:42:34.122 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:42:34.124 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:42:34.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.121+0000 7f75e9525640 1 -- 172.21.15.98:0/1682792635 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f75d0067c50 msgr2=0x7f75d006a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:34.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.121+0000 7f75e9525640 1 --2- 172.21.15.98:0/1682792635 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f75d0067c50 0x7f75d006a110 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f75d80aa000 tx=0x7f75d8000ff0 comp rx=0 tx=0).stop 2024-09-19T08:42:34.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.125+0000 7f75e9525640 1 -- 172.21.15.98:0/1682792635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75cc003f40 msgr2=0x7f75e40b1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:34.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.125+0000 7f75e9525640 1 --2- 172.21.15.98:0/1682792635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75cc003f40 0x7f75e40b1190 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f75d400f350 tx=0x7f75d400f810 comp rx=0 tx=0).stop 2024-09-19T08:42:34.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.125+0000 7f75e9525640 1 -- 172.21.15.98:0/1682792635 shutdown_connections 2024-09-19T08:42:34.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.125+0000 7f75e9525640 1 --2- 172.21.15.98:0/1682792635 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f75d0067c50 0x7f75d006a110 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:34.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.125+0000 7f75e9525640 1 --2- 172.21.15.98:0/1682792635 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75e4151940 0x7f75e4154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:34.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.125+0000 7f75e9525640 1 --2- 172.21.15.98:0/1682792635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75e4151f00 0x7f75e40ad0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:34.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.125+0000 7f75e9525640 1 --2- 172.21.15.98:0/1682792635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75cc003f40 0x7f75e40b1190 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:34.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.125+0000 7f75e9525640 1 -- 172.21.15.98:0/1682792635 >> 172.21.15.98:0/1682792635 conn(0x7f75e40b2010 msgr2=0x7f75e40d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:34.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.125+0000 7f75e9525640 1 -- 172.21.15.98:0/1682792635 shutdown_connections 2024-09-19T08:42:34.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.125+0000 7f75e9525640 1 -- 172.21.15.98:0/1682792635 wait complete. 2024-09-19T08:42:34.141 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2024-09-19T08:42:32.110399+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590460,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":168,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126,"ondisk_log_size":126,"up":80,"acting":80,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":256832,"kb_used_data":7292,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717888,"statfs":{"total":773094113280,"available":772831117312,"internally_reserved":0,"allocated":7467008,"data_stored":3775472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":92,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001134"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402870+0000","last_change":"2024-09-19T08:42:30.446888+0000","last_active":"2024-09-19T08:42:31.402870+0000","last_peered":"2024-09-19T08:42:31.402870+0000","last_clean":"2024-09-19T08:42:31.402870+0000","last_became_active":"2024-09-19T08:42:30.438820+0000","last_became_peered":"2024-09-19T08:42:30.438820+0000","last_unstale":"2024-09-19T08:42:31.402870+0000","last_undegraded":"2024-09-19T08:42:31.402870+0000","last_fullsized":"2024-09-19T08:42:31.402870+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:52:02.376361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401471+0000","last_change":"2024-09-19T08:42:30.431823+0000","last_active":"2024-09-19T08:42:31.401471+0000","last_peered":"2024-09-19T08:42:31.401471+0000","last_clean":"2024-09-19T08:42:31.401471+0000","last_became_active":"2024-09-19T08:42:30.430962+0000","last_became_peered":"2024-09-19T08:42:30.430962+0000","last_unstale":"2024-09-19T08:42:31.401471+0000","last_undegraded":"2024-09-19T08:42:31.401471+0000","last_fullsized":"2024-09-19T08:42:31.401471+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:55.760155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401494+0000","last_change":"2024-09-19T08:42:30.438133+0000","last_active":"2024-09-19T08:42:31.401494+0000","last_peered":"2024-09-19T08:42:31.401494+0000","last_clean":"2024-09-19T08:42:31.401494+0000","last_became_active":"2024-09-19T08:42:30.437186+0000","last_became_peered":"2024-09-19T08:42:30.437186+0000","last_unstale":"2024-09-19T08:42:31.401494+0000","last_undegraded":"2024-09-19T08:42:31.401494+0000","last_fullsized":"2024-09-19T08:42:31.401494+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:55.764601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402257+0000","last_change":"2024-09-19T08:42:30.439106+0000","last_active":"2024-09-19T08:42:31.402257+0000","last_peered":"2024-09-19T08:42:31.402257+0000","last_clean":"2024-09-19T08:42:31.402257+0000","last_became_active":"2024-09-19T08:42:30.438367+0000","last_became_peered":"2024-09-19T08:42:30.438367+0000","last_unstale":"2024-09-19T08:42:31.402257+0000","last_undegraded":"2024-09-19T08:42:31.402257+0000","last_fullsized":"2024-09-19T08:42:31.402257+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:14:25.294258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401507+0000","last_change":"2024-09-19T08:42:30.434665+0000","last_active":"2024-09-19T08:42:31.401507+0000","last_peered":"2024-09-19T08:42:31.401507+0000","last_clean":"2024-09-19T08:42:31.401507+0000","last_became_active":"2024-09-19T08:42:30.433735+0000","last_became_peered":"2024-09-19T08:42:30.433735+0000","last_unstale":"2024-09-19T08:42:31.401507+0000","last_undegraded":"2024-09-19T08:42:31.401507+0000","last_fullsized":"2024-09-19T08:42:31.401507+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:37:26.942476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.356233+0000","last_change":"2024-09-19T08:42:28.356233+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.356233+0000","last_undegraded":"2024-09-19T08:42:28.356233+0000","last_fullsized":"2024-09-19T08:42:28.356233+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.349928+0000","last_change":"2024-09-19T08:42:28.349928+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.349928+0000","last_undegraded":"2024-09-19T08:42:28.349928+0000","last_fullsized":"2024-09-19T08:42:28.349928+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402198+0000","last_change":"2024-09-19T08:42:30.458129+0000","last_active":"2024-09-19T08:42:31.402198+0000","last_peered":"2024-09-19T08:42:31.402198+0000","last_clean":"2024-09-19T08:42:31.402198+0000","last_became_active":"2024-09-19T08:42:30.456984+0000","last_became_peered":"2024-09-19T08:42:30.456984+0000","last_unstale":"2024-09-19T08:42:31.402198+0000","last_undegraded":"2024-09-19T08:42:31.402198+0000","last_fullsized":"2024-09-19T08:42:31.402198+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:43:27.676061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.351206+0000","last_change":"2024-09-19T08:42:28.351206+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.351206+0000","last_undegraded":"2024-09-19T08:42:28.351206+0000","last_fullsized":"2024-09-19T08:42:28.351206+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402459+0000","last_change":"2024-09-19T08:42:29.395380+0000","last_active":"2024-09-19T08:42:31.402459+0000","last_peered":"2024-09-19T08:42:31.402459+0000","last_clean":"2024-09-19T08:42:31.402459+0000","last_became_active":"2024-09-19T08:42:29.394641+0000","last_became_peered":"2024-09-19T08:42:29.394641+0000","last_unstale":"2024-09-19T08:42:31.402459+0000","last_undegraded":"2024-09-19T08:42:31.402459+0000","last_fullsized":"2024-09-19T08:42:31.402459+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:26.586550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:28.334315+0000","last_change":"2024-09-19T08:42:28.334315+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.334315+0000","last_undegraded":"2024-09-19T08:42:28.334315+0000","last_fullsized":"2024-09-19T08:42:28.334315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401528+0000","last_change":"2024-09-19T08:42:29.399398+0000","last_active":"2024-09-19T08:42:31.401528+0000","last_peered":"2024-09-19T08:42:31.401528+0000","last_clean":"2024-09-19T08:42:31.401528+0000","last_became_active":"2024-09-19T08:42:29.396716+0000","last_became_peered":"2024-09-19T08:42:29.396716+0000","last_unstale":"2024-09-19T08:42:31.401528+0000","last_undegraded":"2024-09-19T08:42:31.401528+0000","last_fullsized":"2024-09-19T08:42:31.401528+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:50:23.952850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.355932+0000","last_change":"2024-09-19T08:42:28.355932+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.355932+0000","last_undegraded":"2024-09-19T08:42:28.355932+0000","last_fullsized":"2024-09-19T08:42:28.355932+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.353311+0000","last_change":"2024-09-19T08:42:28.353311+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.353311+0000","last_undegraded":"2024-09-19T08:42:28.353311+0000","last_fullsized":"2024-09-19T08:42:28.353311+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.349295+0000","last_change":"2024-09-19T08:42:28.349295+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.349295+0000","last_undegraded":"2024-09-19T08:42:28.349295+0000","last_fullsized":"2024-09-19T08:42:28.349295+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402803+0000","last_change":"2024-09-19T08:42:29.399190+0000","last_active":"2024-09-19T08:42:31.402803+0000","last_peered":"2024-09-19T08:42:31.402803+0000","last_clean":"2024-09-19T08:42:31.402803+0000","last_became_active":"2024-09-19T08:42:29.396183+0000","last_became_peered":"2024-09-19T08:42:29.396183+0000","last_unstale":"2024-09-19T08:42:31.402803+0000","last_undegraded":"2024-09-19T08:42:31.402803+0000","last_fullsized":"2024-09-19T08:42:31.402803+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:21:01.013528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402155+0000","last_change":"2024-09-19T08:42:30.451725+0000","last_active":"2024-09-19T08:42:31.402155+0000","last_peered":"2024-09-19T08:42:31.402155+0000","last_clean":"2024-09-19T08:42:31.402155+0000","last_became_active":"2024-09-19T08:42:30.437086+0000","last_became_peered":"2024-09-19T08:42:30.437086+0000","last_unstale":"2024-09-19T08:42:31.402155+0000","last_undegraded":"2024-09-19T08:42:31.402155+0000","last_fullsized":"2024-09-19T08:42:31.402155+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:34.398500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:28.334315+0000","last_change":"2024-09-19T08:42:28.334315+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.334315+0000","last_undegraded":"2024-09-19T08:42:28.334315+0000","last_fullsized":"2024-09-19T08:42:28.334315+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.355876+0000","last_change":"2024-09-19T08:42:28.355876+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.355876+0000","last_undegraded":"2024-09-19T08:42:28.355876+0000","last_fullsized":"2024-09-19T08:42:28.355876+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.355949+0000","last_change":"2024-09-19T08:42:28.355949+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.355949+0000","last_undegraded":"2024-09-19T08:42:28.355949+0000","last_fullsized":"2024-09-19T08:42:28.355949+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401738+0000","last_change":"2024-09-19T08:42:29.399130+0000","last_active":"2024-09-19T08:42:31.401738+0000","last_peered":"2024-09-19T08:42:31.401738+0000","last_clean":"2024-09-19T08:42:31.401738+0000","last_became_active":"2024-09-19T08:42:29.396039+0000","last_became_peered":"2024-09-19T08:42:29.396039+0000","last_unstale":"2024-09-19T08:42:31.401738+0000","last_undegraded":"2024-09-19T08:42:31.401738+0000","last_fullsized":"2024-09-19T08:42:31.401738+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:54.277351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.361851+0000","last_change":"2024-09-19T08:42:28.361851+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.361851+0000","last_undegraded":"2024-09-19T08:42:28.361851+0000","last_fullsized":"2024-09-19T08:42:28.361851+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.348977+0000","last_change":"2024-09-19T08:42:28.348977+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.348977+0000","last_undegraded":"2024-09-19T08:42:28.348977+0000","last_fullsized":"2024-09-19T08:42:28.348977+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402598+0000","last_change":"2024-09-19T08:42:29.400466+0000","last_active":"2024-09-19T08:42:31.402598+0000","last_peered":"2024-09-19T08:42:31.402598+0000","last_clean":"2024-09-19T08:42:31.402598+0000","last_became_active":"2024-09-19T08:42:29.395739+0000","last_became_peered":"2024-09-19T08:42:29.395739+0000","last_unstale":"2024-09-19T08:42:31.402598+0000","last_undegraded":"2024-09-19T08:42:31.402598+0000","last_fullsized":"2024-09-19T08:42:31.402598+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:41:59.302829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.349238+0000","last_change":"2024-09-19T08:42:28.349238+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.349238+0000","last_undegraded":"2024-09-19T08:42:28.349238+0000","last_fullsized":"2024-09-19T08:42:28.349238+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402346+0000","last_change":"2024-09-19T08:42:29.399885+0000","last_active":"2024-09-19T08:42:31.402346+0000","last_peered":"2024-09-19T08:42:31.402346+0000","last_clean":"2024-09-19T08:42:31.402346+0000","last_became_active":"2024-09-19T08:42:29.396798+0000","last_became_peered":"2024-09-19T08:42:29.396798+0000","last_unstale":"2024-09-19T08:42:31.402346+0000","last_undegraded":"2024-09-19T08:42:31.402346+0000","last_fullsized":"2024-09-19T08:42:31.402346+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:07.393673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.356804+0000","last_change":"2024-09-19T08:42:28.356804+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.356804+0000","last_undegraded":"2024-09-19T08:42:28.356804+0000","last_fullsized":"2024-09-19T08:42:28.356804+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401789+0000","last_change":"2024-09-19T08:42:29.395444+0000","last_active":"2024-09-19T08:42:31.401789+0000","last_peered":"2024-09-19T08:42:31.401789+0000","last_clean":"2024-09-19T08:42:31.401789+0000","last_became_active":"2024-09-19T08:42:29.394674+0000","last_became_peered":"2024-09-19T08:42:29.394674+0000","last_unstale":"2024-09-19T08:42:31.401789+0000","last_undegraded":"2024-09-19T08:42:31.401789+0000","last_fullsized":"2024-09-19T08:42:31.401789+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:42:57.681916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.360740+0000","last_change":"2024-09-19T08:42:28.360740+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.360740+0000","last_undegraded":"2024-09-19T08:42:28.360740+0000","last_fullsized":"2024-09-19T08:42:28.360740+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.349868+0000","last_change":"2024-09-19T08:42:28.349868+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.349868+0000","last_undegraded":"2024-09-19T08:42:28.349868+0000","last_fullsized":"2024-09-19T08:42:28.349868+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402779+0000","last_change":"2024-09-19T08:42:29.396548+0000","last_active":"2024-09-19T08:42:31.402779+0000","last_peered":"2024-09-19T08:42:31.402779+0000","last_clean":"2024-09-19T08:42:31.402779+0000","last_became_active":"2024-09-19T08:42:29.394896+0000","last_became_peered":"2024-09-19T08:42:29.394896+0000","last_unstale":"2024-09-19T08:42:31.402779+0000","last_undegraded":"2024-09-19T08:42:31.402779+0000","last_fullsized":"2024-09-19T08:42:31.402779+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:27:20.145977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.363524+0000","last_change":"2024-09-19T08:42:28.363524+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.363524+0000","last_undegraded":"2024-09-19T08:42:28.363524+0000","last_fullsized":"2024-09-19T08:42:28.363524+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.354289+0000","last_change":"2024-09-19T08:42:28.354289+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.354289+0000","last_undegraded":"2024-09-19T08:42:28.354289+0000","last_fullsized":"2024-09-19T08:42:28.354289+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.349910+0000","last_change":"2024-09-19T08:42:28.349910+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.349910+0000","last_undegraded":"2024-09-19T08:42:28.349910+0000","last_fullsized":"2024-09-19T08:42:28.349910+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.353322+0000","last_change":"2024-09-19T08:42:28.353322+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.353322+0000","last_undegraded":"2024-09-19T08:42:28.353322+0000","last_fullsized":"2024-09-19T08:42:28.353322+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401692+0000","last_change":"2024-09-19T08:42:30.459642+0000","last_active":"2024-09-19T08:42:31.401692+0000","last_peered":"2024-09-19T08:42:31.401692+0000","last_clean":"2024-09-19T08:42:31.401692+0000","last_became_active":"2024-09-19T08:42:30.459010+0000","last_became_peered":"2024-09-19T08:42:30.459010+0000","last_unstale":"2024-09-19T08:42:31.401692+0000","last_undegraded":"2024-09-19T08:42:31.401692+0000","last_fullsized":"2024-09-19T08:42:31.401692+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:03:33.044498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.356091+0000","last_change":"2024-09-19T08:42:28.356091+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.356091+0000","last_undegraded":"2024-09-19T08:42:28.356091+0000","last_fullsized":"2024-09-19T08:42:28.356091+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T08:42:28.349168+0000","last_change":"2024-09-19T08:42:28.349168+0000","last_active":"2024-09-19T08:42:28.334315+0000","last_peered":"2024-09-19T08:42:28.334315+0000","last_clean":"2024-09-19T08:42:28.334315+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:28.349168+0000","last_undegraded":"2024-09-19T08:42:28.349168+0000","last_fullsized":"2024-09-19T08:42:28.349168+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401444+0000","last_change":"2024-09-19T08:42:30.431276+0000","last_active":"2024-09-19T08:42:31.401444+0000","last_peered":"2024-09-19T08:42:31.401444+0000","last_clean":"2024-09-19T08:42:31.401444+0000","last_became_active":"2024-09-19T08:42:30.430462+0000","last_became_peered":"2024-09-19T08:42:30.430462+0000","last_unstale":"2024-09-19T08:42:31.401444+0000","last_undegraded":"2024-09-19T08:42:31.401444+0000","last_fullsized":"2024-09-19T08:42:31.401444+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:50.219137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401735+0000","last_change":"2024-09-19T08:42:29.373808+0000","last_active":"2024-09-19T08:42:31.401735+0000","last_peered":"2024-09-19T08:42:31.401735+0000","last_clean":"2024-09-19T08:42:31.401735+0000","last_became_active":"2024-09-19T08:42:29.373099+0000","last_became_peered":"2024-09-19T08:42:29.373099+0000","last_unstale":"2024-09-19T08:42:31.401735+0000","last_undegraded":"2024-09-19T08:42:31.401735+0000","last_fullsized":"2024-09-19T08:42:31.401735+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:48.444653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:42:29.343580+0000","last_change":"2024-09-19T08:42:29.343580+0000","last_active":"2024-09-19T08:42:29.343580+0000","last_peered":"2024-09-19T08:42:29.343580+0000","last_clean":"2024-09-19T08:42:29.343580+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:42:29.343580+0000","last_undegraded":"2024-09-19T08:42:29.343580+0000","last_fullsized":"2024-09-19T08:42:29.343580+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401741+0000","last_change":"2024-09-19T08:42:29.399988+0000","last_active":"2024-09-19T08:42:31.401741+0000","last_peered":"2024-09-19T08:42:31.401741+0000","last_clean":"2024-09-19T08:42:31.401741+0000","last_became_active":"2024-09-19T08:42:29.399260+0000","last_became_peered":"2024-09-19T08:42:29.399260+0000","last_unstale":"2024-09-19T08:42:31.401741+0000","last_undegraded":"2024-09-19T08:42:31.401741+0000","last_fullsized":"2024-09-19T08:42:31.401741+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:06.453660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":248,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T08:42:28.359131+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:42:28.359131+0000","last_peered":"2024-09-19T08:42:28.359131+0000","last_clean":"2024-09-19T08:42:28.359131+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:42:28.359131+0000","last_undegraded":"2024-09-19T08:42:28.359131+0000","last_fullsized":"2024-09-19T08:42:28.359131+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":1},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":92,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":60,"acting":60,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509495,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1376256,"data_stored":913734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509497,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":782336,"data_stored":323366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509497,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":782336,"data_stored":323366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509494,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":782336,"data_stored":323366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509496,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":782336,"data_stored":323366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509494,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1363968,"data_stored":907917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574904,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":827392,"data_stored":342808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542199,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31940,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339900,"statfs":{"total":96636764160,"available":96604057600,"internally_reserved":0,"allocated":770048,"data_stored":317549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:42:34.145 INFO:tasks.ceph.ceph_manager.ceph:PG 11.3f is not active+clean 2024-09-19T08:42:34.145 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.145 INFO:tasks.ceph.ceph_manager.ceph:PG 11.3d is not active+clean 2024-09-19T08:42:34.145 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.145 INFO:tasks.ceph.ceph_manager.ceph:PG 11.3c is not active+clean 2024-09-19T08:42:34.146 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.146 INFO:tasks.ceph.ceph_manager.ceph:PG 11.3b is not active+clean 2024-09-19T08:42:34.146 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.146 INFO:tasks.ceph.ceph_manager.ceph:PG 11.39 is not active+clean 2024-09-19T08:42:34.146 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.146 INFO:tasks.ceph.ceph_manager.ceph:PG 11.37 is not active+clean 2024-09-19T08:42:34.146 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.146 INFO:tasks.ceph.ceph_manager.ceph:PG 11.36 is not active+clean 2024-09-19T08:42:34.146 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.146 INFO:tasks.ceph.ceph_manager.ceph:PG 11.35 is not active+clean 2024-09-19T08:42:34.147 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.147 INFO:tasks.ceph.ceph_manager.ceph:PG 11.34 is not active+clean 2024-09-19T08:42:34.147 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.147 INFO:tasks.ceph.ceph_manager.ceph:PG 11.33 is not active+clean 2024-09-19T08:42:34.147 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.147 INFO:tasks.ceph.ceph_manager.ceph:PG 11.32 is not active+clean 2024-09-19T08:42:34.147 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.147 INFO:tasks.ceph.ceph_manager.ceph:PG 11.31 is not active+clean 2024-09-19T08:42:34.147 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.147 INFO:tasks.ceph.ceph_manager.ceph:PG 11.30 is not active+clean 2024-09-19T08:42:34.147 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.148 INFO:tasks.ceph.ceph_manager.ceph:PG 11.2f is not active+clean 2024-09-19T08:42:34.148 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.148 INFO:tasks.ceph.ceph_manager.ceph:PG 11.2e is not active+clean 2024-09-19T08:42:34.148 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.148 INFO:tasks.ceph.ceph_manager.ceph:PG 11.2c is not active+clean 2024-09-19T08:42:34.148 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.148 INFO:tasks.ceph.ceph_manager.ceph:PG 11.2b is not active+clean 2024-09-19T08:42:34.148 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.148 INFO:tasks.ceph.ceph_manager.ceph:PG 11.2a is not active+clean 2024-09-19T08:42:34.148 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.148 INFO:tasks.ceph.ceph_manager.ceph:PG 11.29 is not active+clean 2024-09-19T08:42:34.149 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.149 INFO:tasks.ceph.ceph_manager.ceph:PG 11.28 is not active+clean 2024-09-19T08:42:34.149 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.149 INFO:tasks.ceph.ceph_manager.ceph:PG 11.27 is not active+clean 2024-09-19T08:42:34.149 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.149 INFO:tasks.ceph.ceph_manager.ceph:PG 11.26 is not active+clean 2024-09-19T08:42:34.149 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.149 INFO:tasks.ceph.ceph_manager.ceph:PG 11.25 is not active+clean 2024-09-19T08:42:34.149 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.149 INFO:tasks.ceph.ceph_manager.ceph:PG 11.24 is not active+clean 2024-09-19T08:42:34.150 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.150 INFO:tasks.ceph.ceph_manager.ceph:PG 11.23 is not active+clean 2024-09-19T08:42:34.150 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.150 INFO:tasks.ceph.ceph_manager.ceph:PG 11.22 is not active+clean 2024-09-19T08:42:34.150 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.150 INFO:tasks.ceph.ceph_manager.ceph:PG 11.21 is not active+clean 2024-09-19T08:42:34.150 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.150 INFO:tasks.ceph.ceph_manager.ceph:PG 11.1e is not active+clean 2024-09-19T08:42:34.150 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.150 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1f is not active+clean 2024-09-19T08:42:34.150 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:42:28.356233+0000', 'last_change': '2024-09-19T08:42:28.356233+0000', 'last_active': '2024-09-19T08:42:28.334315+0000', 'last_peered': '2024-09-19T08:42:28.334315+0000', 'last_clean': '2024-09-19T08:42:28.334315+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:28.356233+0000', 'last_undegraded': '2024-09-19T08:42:28.356233+0000', 'last_fullsized': '2024-09-19T08:42:28.356233+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T08:42:34.151 INFO:tasks.ceph.ceph_manager.ceph:PG 11.1f is not active+clean 2024-09-19T08:42:34.151 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.151 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1e is not active+clean 2024-09-19T08:42:34.151 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:42:28.349928+0000', 'last_change': '2024-09-19T08:42:28.349928+0000', 'last_active': '2024-09-19T08:42:28.334315+0000', 'last_peered': '2024-09-19T08:42:28.334315+0000', 'last_clean': '2024-09-19T08:42:28.334315+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:28.349928+0000', 'last_undegraded': '2024-09-19T08:42:28.349928+0000', 'last_fullsized': '2024-09-19T08:42:28.349928+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T08:42:34.151 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1d is not active+clean 2024-09-19T08:42:34.151 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:42:28.351206+0000', 'last_change': '2024-09-19T08:42:28.351206+0000', 'last_active': '2024-09-19T08:42:28.334315+0000', 'last_peered': '2024-09-19T08:42:28.334315+0000', 'last_clean': '2024-09-19T08:42:28.334315+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:28.351206+0000', 'last_undegraded': '2024-09-19T08:42:28.351206+0000', 'last_fullsized': '2024-09-19T08:42:28.351206+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T08:42:34.151 INFO:tasks.ceph.ceph_manager.ceph:PG 11.1d is not active+clean 2024-09-19T08:42:34.151 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.151 INFO:tasks.ceph.ceph_manager.ceph:PG 11.1a is not active+clean 2024-09-19T08:42:34.151 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.152 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1b is not active+clean 2024-09-19T08:42:34.152 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:28.334315+0000', 'last_change': '2024-09-19T08:42:28.334315+0000', 'last_active': '2024-09-19T08:42:28.334315+0000', 'last_peered': '2024-09-19T08:42:28.334315+0000', 'last_clean': '2024-09-19T08:42:28.334315+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:28.334315+0000', 'last_undegraded': '2024-09-19T08:42:28.334315+0000', 'last_fullsized': '2024-09-19T08:42:28.334315+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.152 INFO:tasks.ceph.ceph_manager.ceph:PG 11.1b is not active+clean 2024-09-19T08:42:34.152 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.152 INFO:tasks.ceph.ceph_manager.ceph:PG 11.18 is not active+clean 2024-09-19T08:42:34.152 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.152 INFO:tasks.ceph.ceph_manager.ceph:PG 10.19 is not active+clean 2024-09-19T08:42:34.152 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:42:28.355932+0000', 'last_change': '2024-09-19T08:42:28.355932+0000', 'last_active': '2024-09-19T08:42:28.334315+0000', 'last_peered': '2024-09-19T08:42:28.334315+0000', 'last_clean': '2024-09-19T08:42:28.334315+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:28.355932+0000', 'last_undegraded': '2024-09-19T08:42:28.355932+0000', 'last_fullsized': '2024-09-19T08:42:28.355932+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T08:42:34.152 INFO:tasks.ceph.ceph_manager.ceph:PG 11.19 is not active+clean 2024-09-19T08:42:34.153 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.153 INFO:tasks.ceph.ceph_manager.ceph:PG 10.18 is not active+clean 2024-09-19T08:42:34.153 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:42:28.353311+0000', 'last_change': '2024-09-19T08:42:28.353311+0000', 'last_active': '2024-09-19T08:42:28.334315+0000', 'last_peered': '2024-09-19T08:42:28.334315+0000', 'last_clean': '2024-09-19T08:42:28.334315+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:28.353311+0000', 'last_undegraded': '2024-09-19T08:42:28.353311+0000', 'last_fullsized': '2024-09-19T08:42:28.353311+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-19T08:42:34.153 INFO:tasks.ceph.ceph_manager.ceph:PG 11.16 is not active+clean 2024-09-19T08:42:34.153 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.153 INFO:tasks.ceph.ceph_manager.ceph:PG 10.17 is not active+clean 2024-09-19T08:42:34.153 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:42:28.349295+0000', 'last_change': '2024-09-19T08:42:28.349295+0000', 'last_active': '2024-09-19T08:42:28.334315+0000', 'last_peered': '2024-09-19T08:42:28.334315+0000', 'last_clean': '2024-09-19T08:42:28.334315+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:28.349295+0000', 'last_undegraded': '2024-09-19T08:42:28.349295+0000', 'last_fullsized': '2024-09-19T08:42:28.349295+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T08:42:34.153 INFO:tasks.ceph.ceph_manager.ceph:PG 11.17 is not active+clean 2024-09-19T08:42:34.153 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.153 INFO:tasks.ceph.ceph_manager.ceph:PG 10.15 is not active+clean 2024-09-19T08:42:34.153 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:28.334315+0000', 'last_change': '2024-09-19T08:42:28.334315+0000', 'last_active': '2024-09-19T08:42:28.334315+0000', 'last_peered': '2024-09-19T08:42:28.334315+0000', 'last_clean': '2024-09-19T08:42:28.334315+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:28.334315+0000', 'last_undegraded': '2024-09-19T08:42:28.334315+0000', 'last_fullsized': '2024-09-19T08:42:28.334315+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.154 INFO:tasks.ceph.ceph_manager.ceph:PG 11.15 is not active+clean 2024-09-19T08:42:34.154 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.154 INFO:tasks.ceph.ceph_manager.ceph:PG 10.14 is not active+clean 2024-09-19T08:42:34.154 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:42:28.355876+0000', 'last_change': '2024-09-19T08:42:28.355876+0000', 'last_active': '2024-09-19T08:42:28.334315+0000', 'last_peered': '2024-09-19T08:42:28.334315+0000', 'last_clean': '2024-09-19T08:42:28.334315+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:28.355876+0000', 'last_undegraded': '2024-09-19T08:42:28.355876+0000', 'last_fullsized': '2024-09-19T08:42:28.355876+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T08:42:34.154 INFO:tasks.ceph.ceph_manager.ceph:PG 11.12 is not active+clean 2024-09-19T08:42:34.154 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.154 INFO:tasks.ceph.ceph_manager.ceph:PG 10.13 is not active+clean 2024-09-19T08:42:34.154 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:42:28.355949+0000', 'last_change': '2024-09-19T08:42:28.355949+0000', 'last_active': '2024-09-19T08:42:28.334315+0000', 'last_peered': '2024-09-19T08:42:28.334315+0000', 'last_clean': '2024-09-19T08:42:28.334315+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:28.355949+0000', 'last_undegraded': '2024-09-19T08:42:28.355949+0000', 'last_fullsized': '2024-09-19T08:42:28.355949+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T08:42:34.154 INFO:tasks.ceph.ceph_manager.ceph:PG 11.13 is not active+clean 2024-09-19T08:42:34.154 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.155 INFO:tasks.ceph.ceph_manager.ceph:PG 11.10 is not active+clean 2024-09-19T08:42:34.155 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.155 INFO:tasks.ceph.ceph_manager.ceph:PG 10.11 is not active+clean 2024-09-19T08:42:34.155 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:42:28.361851+0000', 'last_change': '2024-09-19T08:42:28.361851+0000', 'last_active': '2024-09-19T08:42:28.334315+0000', 'last_peered': '2024-09-19T08:42:28.334315+0000', 'last_clean': '2024-09-19T08:42:28.334315+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:28.361851+0000', 'last_undegraded': '2024-09-19T08:42:28.361851+0000', 'last_fullsized': '2024-09-19T08:42:28.361851+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T08:42:34.155 INFO:tasks.ceph.ceph_manager.ceph:PG 11.11 is not active+clean 2024-09-19T08:42:34.155 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.155 INFO:tasks.ceph.ceph_manager.ceph:PG 10.10 is not active+clean 2024-09-19T08:42:34.156 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:42:28.348977+0000', 'last_change': '2024-09-19T08:42:28.348977+0000', 'last_active': '2024-09-19T08:42:28.334315+0000', 'last_peered': '2024-09-19T08:42:28.334315+0000', 'last_clean': '2024-09-19T08:42:28.334315+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:28.348977+0000', 'last_undegraded': '2024-09-19T08:42:28.348977+0000', 'last_fullsized': '2024-09-19T08:42:28.348977+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T08:42:34.156 INFO:tasks.ceph.ceph_manager.ceph:PG 11.e is not active+clean 2024-09-19T08:42:34.156 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.156 INFO:tasks.ceph.ceph_manager.ceph:PG 11.f is not active+clean 2024-09-19T08:42:34.156 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.156 INFO:tasks.ceph.ceph_manager.ceph:PG 10.e is not active+clean 2024-09-19T08:42:34.156 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:42:28.349238+0000', 'last_change': '2024-09-19T08:42:28.349238+0000', 'last_active': '2024-09-19T08:42:28.334315+0000', 'last_peered': '2024-09-19T08:42:28.334315+0000', 'last_clean': '2024-09-19T08:42:28.334315+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:28.349238+0000', 'last_undegraded': '2024-09-19T08:42:28.349238+0000', 'last_fullsized': '2024-09-19T08:42:28.349238+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T08:42:34.156 INFO:tasks.ceph.ceph_manager.ceph:PG 11.c is not active+clean 2024-09-19T08:42:34.156 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.156 INFO:tasks.ceph.ceph_manager.ceph:PG 11.d is not active+clean 2024-09-19T08:42:34.157 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.157 INFO:tasks.ceph.ceph_manager.ceph:PG 10.c is not active+clean 2024-09-19T08:42:34.157 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:42:28.356804+0000', 'last_change': '2024-09-19T08:42:28.356804+0000', 'last_active': '2024-09-19T08:42:28.334315+0000', 'last_peered': '2024-09-19T08:42:28.334315+0000', 'last_clean': '2024-09-19T08:42:28.334315+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:28.356804+0000', 'last_undegraded': '2024-09-19T08:42:28.356804+0000', 'last_fullsized': '2024-09-19T08:42:28.356804+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T08:42:34.157 INFO:tasks.ceph.ceph_manager.ceph:PG 11.b is not active+clean 2024-09-19T08:42:34.157 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.157 INFO:tasks.ceph.ceph_manager.ceph:PG 11.8 is not active+clean 2024-09-19T08:42:34.157 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.157 INFO:tasks.ceph.ceph_manager.ceph:PG 10.9 is not active+clean 2024-09-19T08:42:34.157 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:42:28.360740+0000', 'last_change': '2024-09-19T08:42:28.360740+0000', 'last_active': '2024-09-19T08:42:28.334315+0000', 'last_peered': '2024-09-19T08:42:28.334315+0000', 'last_clean': '2024-09-19T08:42:28.334315+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:28.360740+0000', 'last_undegraded': '2024-09-19T08:42:28.360740+0000', 'last_fullsized': '2024-09-19T08:42:28.360740+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T08:42:34.157 INFO:tasks.ceph.ceph_manager.ceph:PG 11.9 is not active+clean 2024-09-19T08:42:34.157 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.158 INFO:tasks.ceph.ceph_manager.ceph:PG 10.8 is not active+clean 2024-09-19T08:42:34.158 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:42:28.349868+0000', 'last_change': '2024-09-19T08:42:28.349868+0000', 'last_active': '2024-09-19T08:42:28.334315+0000', 'last_peered': '2024-09-19T08:42:28.334315+0000', 'last_clean': '2024-09-19T08:42:28.334315+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:28.349868+0000', 'last_undegraded': '2024-09-19T08:42:28.349868+0000', 'last_fullsized': '2024-09-19T08:42:28.349868+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T08:42:34.158 INFO:tasks.ceph.ceph_manager.ceph:PG 11.6 is not active+clean 2024-09-19T08:42:34.158 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.158 INFO:tasks.ceph.ceph_manager.ceph:PG 11.7 is not active+clean 2024-09-19T08:42:34.158 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.158 INFO:tasks.ceph.ceph_manager.ceph:PG 10.6 is not active+clean 2024-09-19T08:42:34.158 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:42:28.363524+0000', 'last_change': '2024-09-19T08:42:28.363524+0000', 'last_active': '2024-09-19T08:42:28.334315+0000', 'last_peered': '2024-09-19T08:42:28.334315+0000', 'last_clean': '2024-09-19T08:42:28.334315+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:28.363524+0000', 'last_undegraded': '2024-09-19T08:42:28.363524+0000', 'last_fullsized': '2024-09-19T08:42:28.363524+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T08:42:34.158 INFO:tasks.ceph.ceph_manager.ceph:PG 11.4 is not active+clean 2024-09-19T08:42:34.158 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.158 INFO:tasks.ceph.ceph_manager.ceph:PG 10.5 is not active+clean 2024-09-19T08:42:34.158 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:42:28.354289+0000', 'last_change': '2024-09-19T08:42:28.354289+0000', 'last_active': '2024-09-19T08:42:28.334315+0000', 'last_peered': '2024-09-19T08:42:28.334315+0000', 'last_clean': '2024-09-19T08:42:28.334315+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:28.354289+0000', 'last_undegraded': '2024-09-19T08:42:28.354289+0000', 'last_fullsized': '2024-09-19T08:42:28.354289+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-19T08:42:34.159 INFO:tasks.ceph.ceph_manager.ceph:PG 11.5 is not active+clean 2024-09-19T08:42:34.159 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.159 INFO:tasks.ceph.ceph_manager.ceph:PG 10.4 is not active+clean 2024-09-19T08:42:34.159 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:42:28.349910+0000', 'last_change': '2024-09-19T08:42:28.349910+0000', 'last_active': '2024-09-19T08:42:28.334315+0000', 'last_peered': '2024-09-19T08:42:28.334315+0000', 'last_clean': '2024-09-19T08:42:28.334315+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:28.349910+0000', 'last_undegraded': '2024-09-19T08:42:28.349910+0000', 'last_fullsized': '2024-09-19T08:42:28.349910+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T08:42:34.159 INFO:tasks.ceph.ceph_manager.ceph:PG 11.2 is not active+clean 2024-09-19T08:42:34.159 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.159 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3 is not active+clean 2024-09-19T08:42:34.159 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:42:28.353322+0000', 'last_change': '2024-09-19T08:42:28.353322+0000', 'last_active': '2024-09-19T08:42:28.334315+0000', 'last_peered': '2024-09-19T08:42:28.334315+0000', 'last_clean': '2024-09-19T08:42:28.334315+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:28.353322+0000', 'last_undegraded': '2024-09-19T08:42:28.353322+0000', 'last_fullsized': '2024-09-19T08:42:28.353322+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-19T08:42:34.159 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2 is not active+clean 2024-09-19T08:42:34.159 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:42:28.356091+0000', 'last_change': '2024-09-19T08:42:28.356091+0000', 'last_active': '2024-09-19T08:42:28.334315+0000', 'last_peered': '2024-09-19T08:42:28.334315+0000', 'last_clean': '2024-09-19T08:42:28.334315+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:28.356091+0000', 'last_undegraded': '2024-09-19T08:42:28.356091+0000', 'last_fullsized': '2024-09-19T08:42:28.356091+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T08:42:34.160 INFO:tasks.ceph.ceph_manager.ceph:PG 11.0 is not active+clean 2024-09-19T08:42:34.160 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.160 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1 is not active+clean 2024-09-19T08:42:34.160 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T08:42:28.349168+0000', 'last_change': '2024-09-19T08:42:28.349168+0000', 'last_active': '2024-09-19T08:42:28.334315+0000', 'last_peered': '2024-09-19T08:42:28.334315+0000', 'last_clean': '2024-09-19T08:42:28.334315+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:28.349168+0000', 'last_undegraded': '2024-09-19T08:42:28.349168+0000', 'last_fullsized': '2024-09-19T08:42:28.349168+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:28.334315+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T08:42:34.160 INFO:tasks.ceph.ceph_manager.ceph:PG 11.a is not active+clean 2024-09-19T08:42:34.160 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '11.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-19T08:42:29.343580+0000', 'last_change': '2024-09-19T08:42:29.343580+0000', 'last_active': '2024-09-19T08:42:29.343580+0000', 'last_peered': '2024-09-19T08:42:29.343580+0000', 'last_clean': '2024-09-19T08:42:29.343580+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T08:42:29.343580+0000', 'last_undegraded': '2024-09-19T08:42:29.343580+0000', 'last_fullsized': '2024-09-19T08:42:29.343580+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'last_clean_scrub_stamp': '2024-09-19T08:42:29.343580+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-19T08:42:34.160 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T08:42:34.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.305+0000 7f02a6b54640 1 Processor -- start 2024-09-19T08:42:34.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.305+0000 7f02a6b54640 1 -- start start 2024-09-19T08:42:34.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.305+0000 7f02a6b54640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f02a00d05a0 0x7f02a0160be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:34.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.305+0000 7f02a6b54640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f02a015bed0 0x7f02a015c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:34.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.305+0000 7f02a6b54640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02a00afec0 con 0x7f02a00d0980 2024-09-19T08:42:34.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.305+0000 7f02a6b54640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f02a00b3fd0 con 0x7f02a015bed0 2024-09-19T08:42:34.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.305+0000 7f02a6b54640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f02a0160100 con 0x7f02a00d05a0 2024-09-19T08:42:34.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.305+0000 7f02a50f1640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f02a015bed0 0x7f02a015c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:34.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.305+0000 7f02a50f1640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f02a015bed0 0x7f02a015c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34406/0 (socket says 172.21.15.98:34406) 2024-09-19T08:42:34.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.305+0000 7f02a50f1640 1 -- 172.21.15.98:0/3096148428 learned_addr learned my addr 172.21.15.98:0/3096148428 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:34.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.305+0000 7f029ffff640 1 --1- 172.21.15.98:0/3096148428 >> v1:172.21.15.98:6789/0 conn(0x7f02a00d0980 0x7f02a00b3460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41348/0 (socket says 172.21.15.98:41348) 2024-09-19T08:42:34.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.305+0000 7f02a48f0640 1 --2- 172.21.15.98:0/3096148428 >> v2:172.21.15.139:3300/0 conn(0x7f02a00d05a0 0x7f02a0160be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:34.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.305+0000 7f02a50f1640 1 -- 172.21.15.98:0/3096148428 >> v2:172.21.15.139:3300/0 conn(0x7f02a00d05a0 msgr2=0x7f02a0160be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:34.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.305+0000 7f02a50f1640 1 --2- 172.21.15.98:0/3096148428 >> v2:172.21.15.139:3300/0 conn(0x7f02a00d05a0 0x7f02a0160be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:34.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.305+0000 7f02a50f1640 1 -- 172.21.15.98:0/3096148428 >> v1:172.21.15.98:6789/0 conn(0x7f02a00d0980 legacy=0x7f02a00b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:34.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.305+0000 7f029f7fe640 1 -- 172.21.15.98:0/3096148428 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3187163745 0 0) 0x7f02a00afec0 con 0x7f02a00d0980 2024-09-19T08:42:34.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.305+0000 7f02a50f1640 1 -- 172.21.15.98:0/3096148428 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a01603e0 con 0x7f02a015bed0 2024-09-19T08:42:34.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.309+0000 7f02a50f1640 1 --2- 172.21.15.98:0/3096148428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02a015bed0 0x7f02a015c320 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f02a00b3200 tx=0x7f029801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d475d1e5b7f39b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:34.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.309+0000 7f029f7fe640 1 -- 172.21.15.98:0/3096148428 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02980a5730 con 0x7f02a015bed0 2024-09-19T08:42:34.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.309+0000 7f029f7fe640 1 -- 172.21.15.98:0/3096148428 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02980a58d0 con 0x7f02a015bed0 2024-09-19T08:42:34.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.309+0000 7f029f7fe640 1 -- 172.21.15.98:0/3096148428 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02980a5ca0 con 0x7f02a015bed0 2024-09-19T08:42:34.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.309+0000 7f02a6b54640 1 -- 172.21.15.98:0/3096148428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02a015bed0 msgr2=0x7f02a015c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:34.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.309+0000 7f02a6b54640 1 --2- 172.21.15.98:0/3096148428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02a015bed0 0x7f02a015c320 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f02a00b3200 tx=0x7f029801c960 comp rx=0 tx=0).stop 2024-09-19T08:42:34.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.309+0000 7f02a6b54640 1 -- 172.21.15.98:0/3096148428 shutdown_connections 2024-09-19T08:42:34.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.309+0000 7f02a6b54640 1 --2- 172.21.15.98:0/3096148428 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02a015bed0 0x7f02a015c320 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:34.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.309+0000 7f02a6b54640 1 --2- 172.21.15.98:0/3096148428 >> v2:172.21.15.139:3300/0 conn(0x7f02a00d05a0 0x7f02a0160be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:34.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.309+0000 7f02a6b54640 1 -- 172.21.15.98:0/3096148428 >> 172.21.15.98:0/3096148428 conn(0x7f02a006bc90 msgr2=0x7f02a006c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:34.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.309+0000 7f02a6b54640 1 -- 172.21.15.98:0/3096148428 shutdown_connections 2024-09-19T08:42:34.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.309+0000 7f02a6b54640 1 -- 172.21.15.98:0/3096148428 wait complete. 2024-09-19T08:42:34.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.309+0000 7f02a6b54640 1 Processor -- start 2024-09-19T08:42:34.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.309+0000 7f02a6b54640 1 -- start start 2024-09-19T08:42:34.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.313+0000 7f02a6b54640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f02a00d05a0 0x7f02a0158740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:34.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.313+0000 7f02a6b54640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02a00d0980 0x7f02a0160e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:34.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.313+0000 7f02a6b54640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02a015bed0 0x7f02a0142920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:34.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.313+0000 7f02a6b54640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f02a00b4f90 con 0x7f02a00d0980 2024-09-19T08:42:34.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.313+0000 7f02a6b54640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f02a00b3d80 con 0x7f02a015bed0 2024-09-19T08:42:34.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.313+0000 7f02a6b54640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f02a00b5110 con 0x7f02a00d05a0 2024-09-19T08:42:34.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.313+0000 7f029ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02a00d0980 0x7f02a0160e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:34.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.313+0000 7f029ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02a00d0980 0x7f02a0160e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34420/0 (socket says 172.21.15.98:34420) 2024-09-19T08:42:34.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.313+0000 7f029ffff640 1 -- 172.21.15.98:0/1014732360 learned_addr learned my addr 172.21.15.98:0/1014732360 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:34.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.313+0000 7f02a50f1640 1 --2- 172.21.15.98:0/1014732360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02a015bed0 0x7f02a0142920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:34.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.313+0000 7f02a48f0640 1 --2- 172.21.15.98:0/1014732360 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f02a00d05a0 0x7f02a0158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:34.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.313+0000 7f029ffff640 1 -- 172.21.15.98:0/1014732360 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f02a00d05a0 msgr2=0x7f02a0158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:34.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.313+0000 7f029ffff640 1 --2- 172.21.15.98:0/1014732360 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f02a00d05a0 0x7f02a0158740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:34.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.313+0000 7f029ffff640 1 -- 172.21.15.98:0/1014732360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02a015bed0 msgr2=0x7f02a0142920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:34.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.313+0000 7f029ffff640 1 --2- 172.21.15.98:0/1014732360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02a015bed0 0x7f02a0142920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:34.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.313+0000 7f029ffff640 1 -- 172.21.15.98:0/1014732360 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a0146070 con 0x7f02a00d0980 2024-09-19T08:42:34.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.313+0000 7f029ffff640 1 --2- 172.21.15.98:0/1014732360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02a00d0980 0x7f02a0160e60 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f02940011f0 tx=0x7f029400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:34.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.313+0000 7f029d7fa640 1 -- 172.21.15.98:0/1014732360 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0294018070 con 0x7f02a00d0980 2024-09-19T08:42:34.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.313+0000 7f029d7fa640 1 -- 172.21.15.98:0/1014732360 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02940074e0 con 0x7f02a00d0980 2024-09-19T08:42:34.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.313+0000 7f02a6b54640 1 -- 172.21.15.98:0/1014732360 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02a00af990 con 0x7f02a00d0980 2024-09-19T08:42:34.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.313+0000 7f029d7fa640 1 -- 172.21.15.98:0/1014732360 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0294004a10 con 0x7f02a00d0980 2024-09-19T08:42:34.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.313+0000 7f02a6b54640 1 -- 172.21.15.98:0/1014732360 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02a00ab050 con 0x7f02a00d0980 2024-09-19T08:42:34.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.313+0000 7f02a6b54640 1 -- 172.21.15.98:0/1014732360 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02a00afeb0 con 0x7f02a00d0980 2024-09-19T08:42:34.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.313+0000 7f029d7fa640 1 -- 172.21.15.98:0/1014732360 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f029401b070 con 0x7f02a00d0980 2024-09-19T08:42:34.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.313+0000 7f029d7fa640 1 --2- 172.21.15.98:0/1014732360 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f028c067c20 0x7f028c06a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:34.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.313+0000 7f029d7fa640 1 -- 172.21.15.98:0/1014732360 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f0294085670 con 0x7f02a00d0980 2024-09-19T08:42:34.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.313+0000 7f02a48f0640 1 --2- 172.21.15.98:0/1014732360 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f028c067c20 0x7f028c06a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:34.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.313+0000 7f02a48f0640 1 --2- 172.21.15.98:0/1014732360 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f028c067c20 0x7f028c06a0e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f02a015c6c0 tx=0x7f0290008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:34.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.317+0000 7f029d7fa640 1 -- 172.21.15.98:0/1014732360 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f0294055ca0 con 0x7f02a00d0980 2024-09-19T08:42:34.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.541+0000 7f02a6b54640 1 -- 172.21.15.98:0/1014732360 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f02a00aae40 con 0x7f02a00d0980 2024-09-19T08:42:34.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.541+0000 7f029d7fa640 1 -- 172.21.15.98:0/1014732360 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1474 (secure 0 0 0) 0x7f02940555a0 con 0x7f02a00d0980 2024-09-19T08:42:34.544 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:42:34.544 INFO:teuthology.orchestra.run.smithi098.stdout:{"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":279,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":68,"num_osds":8,"num_up_osds":8,"osd_up_since":1726735088,"num_in_osds":8,"osd_in_since":1726735077,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":263327744,"bytes_avail":772830785536,"bytes_total":773094113280,"write_bytes_sec":2481,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":38,"btime":"2024-09-19T08:42:32:414567+0000","id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"a","status":"up:active","gid":6171}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-09-19T08:42:18.106402+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-19T08:42:34.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.545+0000 7f02a6b54640 1 -- 172.21.15.98:0/1014732360 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f028c067c20 msgr2=0x7f028c06a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:34.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.545+0000 7f02a6b54640 1 --2- 172.21.15.98:0/1014732360 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f028c067c20 0x7f028c06a0e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f02a015c6c0 tx=0x7f0290008040 comp rx=0 tx=0).stop 2024-09-19T08:42:34.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.545+0000 7f02a6b54640 1 -- 172.21.15.98:0/1014732360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02a00d0980 msgr2=0x7f02a0160e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:34.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.545+0000 7f02a6b54640 1 --2- 172.21.15.98:0/1014732360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02a00d0980 0x7f02a0160e60 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f02940011f0 tx=0x7f029400c9b0 comp rx=0 tx=0).stop 2024-09-19T08:42:34.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.545+0000 7f02a6b54640 1 -- 172.21.15.98:0/1014732360 shutdown_connections 2024-09-19T08:42:34.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.545+0000 7f02a6b54640 1 --2- 172.21.15.98:0/1014732360 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f028c067c20 0x7f028c06a0e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:34.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.545+0000 7f02a6b54640 1 --2- 172.21.15.98:0/1014732360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02a015bed0 0x7f02a0142920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:34.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.545+0000 7f02a6b54640 1 --2- 172.21.15.98:0/1014732360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02a00d0980 0x7f02a0160e60 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:34.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.545+0000 7f02a6b54640 1 --2- 172.21.15.98:0/1014732360 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f02a00d05a0 0x7f02a0158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:34.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.545+0000 7f02a6b54640 1 -- 172.21.15.98:0/1014732360 >> 172.21.15.98:0/1014732360 conn(0x7f02a006bc90 msgr2=0x7f02a00cf6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:34.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.545+0000 7f02a6b54640 1 -- 172.21.15.98:0/1014732360 shutdown_connections 2024-09-19T08:42:34.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.545+0000 7f02a6b54640 1 -- 172.21.15.98:0/1014732360 wait complete. 2024-09-19T08:42:34.562 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-19T08:42:34.563 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:42:34.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.685+0000 7fafdd462640 1 Processor -- start 2024-09-19T08:42:34.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.685+0000 7fafdd462640 1 -- start start 2024-09-19T08:42:34.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.685+0000 7fafdd462640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafd815c550 0x7fafd806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:34.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.685+0000 7fafdd462640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fafd80cfad0 0x7fafd80723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:34.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.685+0000 7fafdd462640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafd806d310 con 0x7fafd806c1d0 2024-09-19T08:42:34.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.685+0000 7fafdd462640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fafd806c610 con 0x7fafd80cfad0 2024-09-19T08:42:34.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.685+0000 7fafdd462640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fafd806d5f0 con 0x7fafd815c550 2024-09-19T08:42:34.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.685+0000 7fafd77fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fafd80cfad0 0x7fafd80723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:34.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.685+0000 7fafd6ffd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafd815c550 0x7fafd806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:34.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.685+0000 7fafce7fc640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fafd806c1d0 0x7fafd80cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:52756/0 (socket says 172.21.15.98:52756) 2024-09-19T08:42:34.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.685+0000 7fafce7fc640 1 -- 172.21.15.98:0/3156460082 learned_addr learned my addr 172.21.15.98:0/3156460082 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:34.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.685+0000 7fafd6ffd640 1 -- 172.21.15.98:0/3156460082 >> v2:172.21.15.139:3300/0 conn(0x7fafd80cfad0 msgr2=0x7fafd80723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:34.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.685+0000 7fafd6ffd640 1 --2- 172.21.15.98:0/3156460082 >> v2:172.21.15.139:3300/0 conn(0x7fafd80cfad0 0x7fafd80723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:34.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.685+0000 7fafd6ffd640 1 -- 172.21.15.98:0/3156460082 >> v1:172.21.15.139:6789/0 conn(0x7fafd806c1d0 legacy=0x7fafd80cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:34.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.685+0000 7fafd6ffd640 1 -- 172.21.15.98:0/3156460082 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd806d8d0 con 0x7fafd815c550 2024-09-19T08:42:34.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.685+0000 7fafd77fe640 1 --2- 172.21.15.98:0/3156460082 >> v2:172.21.15.139:3300/0 conn(0x7fafd80cfad0 0x7fafd80723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:42:34.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.689+0000 7fafd6ffd640 1 --2- 172.21.15.98:0/3156460082 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafd815c550 0x7fafd806bc90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fafc80097c0 tx=0x7fafc801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19ffef697b7e8d51 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:34.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.689+0000 7fafd67fc640 1 -- 172.21.15.98:0/3156460082 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc80b0070 con 0x7fafd815c550 2024-09-19T08:42:34.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.689+0000 7fafd67fc640 1 -- 172.21.15.98:0/3156460082 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafc80ab600 con 0x7fafd815c550 2024-09-19T08:42:34.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.689+0000 7fafd67fc640 1 -- 172.21.15.98:0/3156460082 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc80ab9d0 con 0x7fafd815c550 2024-09-19T08:42:34.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.689+0000 7fafdd462640 1 -- 172.21.15.98:0/3156460082 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafd815c550 msgr2=0x7fafd806bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:34.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.689+0000 7fafdd462640 1 --2- 172.21.15.98:0/3156460082 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafd815c550 0x7fafd806bc90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fafc80097c0 tx=0x7fafc801c4a0 comp rx=0 tx=0).stop 2024-09-19T08:42:34.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.689+0000 7fafdd462640 1 -- 172.21.15.98:0/3156460082 shutdown_connections 2024-09-19T08:42:34.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.689+0000 7fafdd462640 1 --2- 172.21.15.98:0/3156460082 >> v2:172.21.15.139:3300/0 conn(0x7fafd80cfad0 0x7fafd80723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:34.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.689+0000 7fafdd462640 1 --2- 172.21.15.98:0/3156460082 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafd815c550 0x7fafd806bc90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:34.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.689+0000 7fafdd462640 1 -- 172.21.15.98:0/3156460082 >> 172.21.15.98:0/3156460082 conn(0x7fafd80b2010 msgr2=0x7fafd80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:34.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.689+0000 7fafdd462640 1 -- 172.21.15.98:0/3156460082 shutdown_connections 2024-09-19T08:42:34.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.689+0000 7fafdd462640 1 -- 172.21.15.98:0/3156460082 wait complete. 2024-09-19T08:42:34.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.689+0000 7fafdd462640 1 Processor -- start 2024-09-19T08:42:34.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.689+0000 7fafdd462640 1 -- start start 2024-09-19T08:42:34.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.693+0000 7fafdd462640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafd806c1d0 0x7fafd815dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:34.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.693+0000 7fafdd462640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafd80cfad0 0x7fafd8157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:34.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.693+0000 7fafdd462640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fafd815c550 0x7fafd8158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:34.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.693+0000 7fafdd462640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fafd8073300 con 0x7fafd80cfad0 2024-09-19T08:42:34.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.693+0000 7fafdd462640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fafd815cb40 con 0x7fafd815c550 2024-09-19T08:42:34.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.693+0000 7fafdd462640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fafd8073480 con 0x7fafd806c1d0 2024-09-19T08:42:34.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.693+0000 7fafce7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafd80cfad0 0x7fafd8157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:34.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.693+0000 7fafce7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafd80cfad0 0x7fafd8157be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34422/0 (socket says 172.21.15.98:34422) 2024-09-19T08:42:34.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.693+0000 7fafce7fc640 1 -- 172.21.15.98:0/1978457731 learned_addr learned my addr 172.21.15.98:0/1978457731 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:34.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.693+0000 7fafd77fe640 1 --2- 172.21.15.98:0/1978457731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fafd815c550 0x7fafd8158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:34.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.693+0000 7fafd6ffd640 1 --2- 172.21.15.98:0/1978457731 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafd806c1d0 0x7fafd815dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:34.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.693+0000 7fafce7fc640 1 -- 172.21.15.98:0/1978457731 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafd806c1d0 msgr2=0x7fafd815dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:34.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.693+0000 7fafce7fc640 1 --2- 172.21.15.98:0/1978457731 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafd806c1d0 0x7fafd815dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:34.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.693+0000 7fafce7fc640 1 -- 172.21.15.98:0/1978457731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fafd815c550 msgr2=0x7fafd8158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:34.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.693+0000 7fafce7fc640 1 --2- 172.21.15.98:0/1978457731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fafd815c550 0x7fafd8158120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:34.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.693+0000 7fafce7fc640 1 -- 172.21.15.98:0/1978457731 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd8152250 con 0x7fafd80cfad0 2024-09-19T08:42:34.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.693+0000 7fafce7fc640 1 --2- 172.21.15.98:0/1978457731 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafd80cfad0 0x7fafd8157be0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fafc000c340 tx=0x7fafc000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:34.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.693+0000 7fafcffff640 1 -- 172.21.15.98:0/1978457731 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc0017070 con 0x7fafd80cfad0 2024-09-19T08:42:34.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.693+0000 7fafdd462640 1 -- 172.21.15.98:0/1978457731 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd8152530 con 0x7fafd80cfad0 2024-09-19T08:42:34.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.693+0000 7fafcffff640 1 -- 172.21.15.98:0/1978457731 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafc0005cf0 con 0x7fafd80cfad0 2024-09-19T08:42:34.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.693+0000 7fafcffff640 1 -- 172.21.15.98:0/1978457731 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc00072f0 con 0x7fafd80cfad0 2024-09-19T08:42:34.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.693+0000 7fafdd462640 1 -- 172.21.15.98:0/1978457731 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd8152af0 con 0x7fafd80cfad0 2024-09-19T08:42:34.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.693+0000 7fafdd462640 1 -- 172.21.15.98:0/1978457731 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafd80723a0 con 0x7fafd80cfad0 2024-09-19T08:42:34.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.693+0000 7fafcffff640 1 -- 172.21.15.98:0/1978457731 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fafc001a070 con 0x7fafd80cfad0 2024-09-19T08:42:34.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.693+0000 7fafcffff640 1 --2- 172.21.15.98:0/1978457731 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fafb8067c20 0x7fafb806a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:34.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.693+0000 7fafcffff640 1 -- 172.21.15.98:0/1978457731 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7fafc0085580 con 0x7fafd80cfad0 2024-09-19T08:42:34.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.693+0000 7fafd6ffd640 1 --2- 172.21.15.98:0/1978457731 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fafb8067c20 0x7fafb806a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:34.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.693+0000 7fafd6ffd640 1 --2- 172.21.15.98:0/1978457731 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fafb8067c20 0x7fafb806a0e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fafc80ad000 tx=0x7fafc80a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:34.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.697+0000 7fafcffff640 1 -- 172.21.15.98:0/1978457731 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fafc0055c30 con 0x7fafd80cfad0 2024-09-19T08:42:34.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.845+0000 7fafdd462640 1 -- 172.21.15.98:0/1978457731 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fafd8156b10 con 0x7fafb8067c20 2024-09-19T08:42:34.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.861+0000 7fafcffff640 1 -- 172.21.15.98:0/1978457731 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252079 (secure 0 0 0) 0x7fafd8156b10 con 0x7fafb8067c20 2024-09-19T08:42:34.865 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:42:34.866 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:42:34.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.865+0000 7fafdd462640 1 -- 172.21.15.98:0/1978457731 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fafb8067c20 msgr2=0x7fafb806a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:34.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.865+0000 7fafdd462640 1 --2- 172.21.15.98:0/1978457731 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fafb8067c20 0x7fafb806a0e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fafc80ad000 tx=0x7fafc80a6000 comp rx=0 tx=0).stop 2024-09-19T08:42:34.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.865+0000 7fafdd462640 1 -- 172.21.15.98:0/1978457731 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafd80cfad0 msgr2=0x7fafd8157be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:34.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.865+0000 7fafdd462640 1 --2- 172.21.15.98:0/1978457731 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafd80cfad0 0x7fafd8157be0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fafc000c340 tx=0x7fafc000c800 comp rx=0 tx=0).stop 2024-09-19T08:42:34.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.865+0000 7fafdd462640 1 -- 172.21.15.98:0/1978457731 shutdown_connections 2024-09-19T08:42:34.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.865+0000 7fafdd462640 1 --2- 172.21.15.98:0/1978457731 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fafb8067c20 0x7fafb806a0e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:34.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.865+0000 7fafdd462640 1 --2- 172.21.15.98:0/1978457731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fafd815c550 0x7fafd8158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:34.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.865+0000 7fafdd462640 1 --2- 172.21.15.98:0/1978457731 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafd80cfad0 0x7fafd8157be0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:34.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.865+0000 7fafdd462640 1 --2- 172.21.15.98:0/1978457731 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafd806c1d0 0x7fafd815dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:34.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.865+0000 7fafdd462640 1 -- 172.21.15.98:0/1978457731 >> 172.21.15.98:0/1978457731 conn(0x7fafd80b2010 msgr2=0x7fafd8159ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:34.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.865+0000 7fafdd462640 1 -- 172.21.15.98:0/1978457731 shutdown_connections 2024-09-19T08:42:34.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:34.865+0000 7fafdd462640 1 -- 172.21.15.98:0/1978457731 wait complete. 2024-09-19T08:42:34.881 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":194,"stamp":"2024-09-19T08:42:34.122907+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":137,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257156,"kb_used_data":7624,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717564,"statfs":{"total":773094113280,"available":772830785536,"internally_reserved":0,"allocated":7806976,"data_stored":3927284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":51,"apply_latency_ms":51,"commit_latency_ns":51000000,"apply_latency_ns":51000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.778095"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.458414+0000","last_change":"2024-09-19T08:42:30.432422+0000","last_active":"2024-09-19T08:42:31.458414+0000","last_peered":"2024-09-19T08:42:31.458414+0000","last_clean":"2024-09-19T08:42:31.458414+0000","last_became_active":"2024-09-19T08:42:30.431696+0000","last_became_peered":"2024-09-19T08:42:30.431696+0000","last_unstale":"2024-09-19T08:42:31.458414+0000","last_undegraded":"2024-09-19T08:42:31.458414+0000","last_fullsized":"2024-09-19T08:42:31.458414+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:33.925297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402870+0000","last_change":"2024-09-19T08:42:30.446888+0000","last_active":"2024-09-19T08:42:31.402870+0000","last_peered":"2024-09-19T08:42:31.402870+0000","last_clean":"2024-09-19T08:42:31.402870+0000","last_became_active":"2024-09-19T08:42:30.438820+0000","last_became_peered":"2024-09-19T08:42:30.438820+0000","last_unstale":"2024-09-19T08:42:31.402870+0000","last_undegraded":"2024-09-19T08:42:31.402870+0000","last_fullsized":"2024-09-19T08:42:31.402870+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:52:02.376361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.456309+0000","last_change":"2024-09-19T08:42:30.417593+0000","last_active":"2024-09-19T08:42:31.456309+0000","last_peered":"2024-09-19T08:42:31.456309+0000","last_clean":"2024-09-19T08:42:31.456309+0000","last_became_active":"2024-09-19T08:42:30.417241+0000","last_became_peered":"2024-09-19T08:42:30.417241+0000","last_unstale":"2024-09-19T08:42:31.456309+0000","last_undegraded":"2024-09-19T08:42:31.456309+0000","last_fullsized":"2024-09-19T08:42:31.456309+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:30:42.663654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.455352+0000","last_change":"2024-09-19T08:42:30.454395+0000","last_active":"2024-09-19T08:42:31.455352+0000","last_peered":"2024-09-19T08:42:31.455352+0000","last_clean":"2024-09-19T08:42:31.455352+0000","last_became_active":"2024-09-19T08:42:30.454076+0000","last_became_peered":"2024-09-19T08:42:30.454076+0000","last_unstale":"2024-09-19T08:42:31.455352+0000","last_undegraded":"2024-09-19T08:42:31.455352+0000","last_fullsized":"2024-09-19T08:42:31.455352+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:42:51.025229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.398464+0000","last_change":"2024-09-19T08:42:30.431886+0000","last_active":"2024-09-19T08:42:31.398464+0000","last_peered":"2024-09-19T08:42:31.398464+0000","last_clean":"2024-09-19T08:42:31.398464+0000","last_became_active":"2024-09-19T08:42:30.429749+0000","last_became_peered":"2024-09-19T08:42:30.429749+0000","last_unstale":"2024-09-19T08:42:31.398464+0000","last_undegraded":"2024-09-19T08:42:31.398464+0000","last_fullsized":"2024-09-19T08:42:31.398464+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:46:06.422459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401471+0000","last_change":"2024-09-19T08:42:30.431823+0000","last_active":"2024-09-19T08:42:31.401471+0000","last_peered":"2024-09-19T08:42:31.401471+0000","last_clean":"2024-09-19T08:42:31.401471+0000","last_became_active":"2024-09-19T08:42:30.430962+0000","last_became_peered":"2024-09-19T08:42:30.430962+0000","last_unstale":"2024-09-19T08:42:31.401471+0000","last_undegraded":"2024-09-19T08:42:31.401471+0000","last_fullsized":"2024-09-19T08:42:31.401471+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:55.760155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.425773+0000","last_change":"2024-09-19T08:42:30.449238+0000","last_active":"2024-09-19T08:42:31.425773+0000","last_peered":"2024-09-19T08:42:31.425773+0000","last_clean":"2024-09-19T08:42:31.425773+0000","last_became_active":"2024-09-19T08:42:30.448544+0000","last_became_peered":"2024-09-19T08:42:30.448544+0000","last_unstale":"2024-09-19T08:42:31.425773+0000","last_undegraded":"2024-09-19T08:42:31.425773+0000","last_fullsized":"2024-09-19T08:42:31.425773+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:19.467184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401494+0000","last_change":"2024-09-19T08:42:30.438133+0000","last_active":"2024-09-19T08:42:31.401494+0000","last_peered":"2024-09-19T08:42:31.401494+0000","last_clean":"2024-09-19T08:42:31.401494+0000","last_became_active":"2024-09-19T08:42:30.437186+0000","last_became_peered":"2024-09-19T08:42:30.437186+0000","last_unstale":"2024-09-19T08:42:31.401494+0000","last_undegraded":"2024-09-19T08:42:31.401494+0000","last_fullsized":"2024-09-19T08:42:31.401494+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:55.764601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.398650+0000","last_change":"2024-09-19T08:42:30.421194+0000","last_active":"2024-09-19T08:42:31.398650+0000","last_peered":"2024-09-19T08:42:31.398650+0000","last_clean":"2024-09-19T08:42:31.398650+0000","last_became_active":"2024-09-19T08:42:30.420916+0000","last_became_peered":"2024-09-19T08:42:30.420916+0000","last_unstale":"2024-09-19T08:42:31.398650+0000","last_undegraded":"2024-09-19T08:42:31.398650+0000","last_fullsized":"2024-09-19T08:42:31.398650+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:23.647034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.458288+0000","last_change":"2024-09-19T08:42:30.423732+0000","last_active":"2024-09-19T08:42:31.458288+0000","last_peered":"2024-09-19T08:42:31.458288+0000","last_clean":"2024-09-19T08:42:31.458288+0000","last_became_active":"2024-09-19T08:42:30.422050+0000","last_became_peered":"2024-09-19T08:42:30.422050+0000","last_unstale":"2024-09-19T08:42:31.458288+0000","last_undegraded":"2024-09-19T08:42:31.458288+0000","last_fullsized":"2024-09-19T08:42:31.458288+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:27:04.608943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.458324+0000","last_change":"2024-09-19T08:42:30.427962+0000","last_active":"2024-09-19T08:42:31.458324+0000","last_peered":"2024-09-19T08:42:31.458324+0000","last_clean":"2024-09-19T08:42:31.458324+0000","last_became_active":"2024-09-19T08:42:30.424985+0000","last_became_peered":"2024-09-19T08:42:30.424985+0000","last_unstale":"2024-09-19T08:42:31.458324+0000","last_undegraded":"2024-09-19T08:42:31.458324+0000","last_fullsized":"2024-09-19T08:42:31.458324+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:03:03.502166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.455535+0000","last_change":"2024-09-19T08:42:30.420899+0000","last_active":"2024-09-19T08:42:31.455535+0000","last_peered":"2024-09-19T08:42:31.455535+0000","last_clean":"2024-09-19T08:42:31.455535+0000","last_became_active":"2024-09-19T08:42:30.420530+0000","last_became_peered":"2024-09-19T08:42:30.420530+0000","last_unstale":"2024-09-19T08:42:31.455535+0000","last_undegraded":"2024-09-19T08:42:31.455535+0000","last_fullsized":"2024-09-19T08:42:31.455535+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:07:36.196648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.429614+0000","last_change":"2024-09-19T08:42:30.440000+0000","last_active":"2024-09-19T08:42:31.429614+0000","last_peered":"2024-09-19T08:42:31.429614+0000","last_clean":"2024-09-19T08:42:31.429614+0000","last_became_active":"2024-09-19T08:42:30.439710+0000","last_became_peered":"2024-09-19T08:42:30.439710+0000","last_unstale":"2024-09-19T08:42:31.429614+0000","last_undegraded":"2024-09-19T08:42:31.429614+0000","last_fullsized":"2024-09-19T08:42:31.429614+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:41:07.896587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.398338+0000","last_change":"2024-09-19T08:42:30.447680+0000","last_active":"2024-09-19T08:42:31.398338+0000","last_peered":"2024-09-19T08:42:31.398338+0000","last_clean":"2024-09-19T08:42:31.398338+0000","last_became_active":"2024-09-19T08:42:30.447314+0000","last_became_peered":"2024-09-19T08:42:30.447314+0000","last_unstale":"2024-09-19T08:42:31.398338+0000","last_undegraded":"2024-09-19T08:42:31.398338+0000","last_fullsized":"2024-09-19T08:42:31.398338+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:40:07.172560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.429559+0000","last_change":"2024-09-19T08:42:30.420747+0000","last_active":"2024-09-19T08:42:31.429559+0000","last_peered":"2024-09-19T08:42:31.429559+0000","last_clean":"2024-09-19T08:42:31.429559+0000","last_became_active":"2024-09-19T08:42:30.420371+0000","last_became_peered":"2024-09-19T08:42:30.420371+0000","last_unstale":"2024-09-19T08:42:31.429559+0000","last_undegraded":"2024-09-19T08:42:31.429559+0000","last_fullsized":"2024-09-19T08:42:31.429559+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:02:41.820517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.456504+0000","last_change":"2024-09-19T08:42:30.422404+0000","last_active":"2024-09-19T08:42:31.456504+0000","last_peered":"2024-09-19T08:42:31.456504+0000","last_clean":"2024-09-19T08:42:31.456504+0000","last_became_active":"2024-09-19T08:42:30.419609+0000","last_became_peered":"2024-09-19T08:42:30.419609+0000","last_unstale":"2024-09-19T08:42:31.456504+0000","last_undegraded":"2024-09-19T08:42:31.456504+0000","last_fullsized":"2024-09-19T08:42:31.456504+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:47.618846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.398779+0000","last_change":"2024-09-19T08:42:30.417780+0000","last_active":"2024-09-19T08:42:31.398779+0000","last_peered":"2024-09-19T08:42:31.398779+0000","last_clean":"2024-09-19T08:42:31.398779+0000","last_became_active":"2024-09-19T08:42:30.417309+0000","last_became_peered":"2024-09-19T08:42:30.417309+0000","last_unstale":"2024-09-19T08:42:31.398779+0000","last_undegraded":"2024-09-19T08:42:31.398779+0000","last_fullsized":"2024-09-19T08:42:31.398779+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:25.967482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.455739+0000","last_change":"2024-09-19T08:42:30.455358+0000","last_active":"2024-09-19T08:42:31.455739+0000","last_peered":"2024-09-19T08:42:31.455739+0000","last_clean":"2024-09-19T08:42:31.455739+0000","last_became_active":"2024-09-19T08:42:30.455033+0000","last_became_peered":"2024-09-19T08:42:30.455033+0000","last_unstale":"2024-09-19T08:42:31.455739+0000","last_undegraded":"2024-09-19T08:42:31.455739+0000","last_fullsized":"2024-09-19T08:42:31.455739+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:17:36.400315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402257+0000","last_change":"2024-09-19T08:42:30.439106+0000","last_active":"2024-09-19T08:42:31.402257+0000","last_peered":"2024-09-19T08:42:31.402257+0000","last_clean":"2024-09-19T08:42:31.402257+0000","last_became_active":"2024-09-19T08:42:30.438367+0000","last_became_peered":"2024-09-19T08:42:30.438367+0000","last_unstale":"2024-09-19T08:42:31.402257+0000","last_undegraded":"2024-09-19T08:42:31.402257+0000","last_fullsized":"2024-09-19T08:42:31.402257+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:14:25.294258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.429592+0000","last_change":"2024-09-19T08:42:30.449020+0000","last_active":"2024-09-19T08:42:31.429592+0000","last_peered":"2024-09-19T08:42:31.429592+0000","last_clean":"2024-09-19T08:42:31.429592+0000","last_became_active":"2024-09-19T08:42:30.448683+0000","last_became_peered":"2024-09-19T08:42:30.448683+0000","last_unstale":"2024-09-19T08:42:31.429592+0000","last_undegraded":"2024-09-19T08:42:31.429592+0000","last_fullsized":"2024-09-19T08:42:31.429592+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:57:27.072229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.457374+0000","last_change":"2024-09-19T08:42:30.424713+0000","last_active":"2024-09-19T08:42:31.457374+0000","last_peered":"2024-09-19T08:42:31.457374+0000","last_clean":"2024-09-19T08:42:31.457374+0000","last_became_active":"2024-09-19T08:42:30.423957+0000","last_became_peered":"2024-09-19T08:42:30.423957+0000","last_unstale":"2024-09-19T08:42:31.457374+0000","last_undegraded":"2024-09-19T08:42:31.457374+0000","last_fullsized":"2024-09-19T08:42:31.457374+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:03:35.149826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.456851+0000","last_change":"2024-09-19T08:42:30.421062+0000","last_active":"2024-09-19T08:42:31.456851+0000","last_peered":"2024-09-19T08:42:31.456851+0000","last_clean":"2024-09-19T08:42:31.456851+0000","last_became_active":"2024-09-19T08:42:30.420358+0000","last_became_peered":"2024-09-19T08:42:30.420358+0000","last_unstale":"2024-09-19T08:42:31.456851+0000","last_undegraded":"2024-09-19T08:42:31.456851+0000","last_fullsized":"2024-09-19T08:42:31.456851+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:59:11.462609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.458255+0000","last_change":"2024-09-19T08:42:30.452202+0000","last_active":"2024-09-19T08:42:31.458255+0000","last_peered":"2024-09-19T08:42:31.458255+0000","last_clean":"2024-09-19T08:42:31.458255+0000","last_became_active":"2024-09-19T08:42:30.451574+0000","last_became_peered":"2024-09-19T08:42:30.451574+0000","last_unstale":"2024-09-19T08:42:31.458255+0000","last_undegraded":"2024-09-19T08:42:31.458255+0000","last_fullsized":"2024-09-19T08:42:31.458255+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:41:07.896587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.457408+0000","last_change":"2024-09-19T08:42:30.458493+0000","last_active":"2024-09-19T08:42:31.457408+0000","last_peered":"2024-09-19T08:42:31.457408+0000","last_clean":"2024-09-19T08:42:31.457408+0000","last_became_active":"2024-09-19T08:42:30.457803+0000","last_became_peered":"2024-09-19T08:42:30.457803+0000","last_unstale":"2024-09-19T08:42:31.457408+0000","last_undegraded":"2024-09-19T08:42:31.457408+0000","last_fullsized":"2024-09-19T08:42:31.457408+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:07:43.736470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.458092+0000","last_change":"2024-09-19T08:42:30.429459+0000","last_active":"2024-09-19T08:42:31.458092+0000","last_peered":"2024-09-19T08:42:31.458092+0000","last_clean":"2024-09-19T08:42:31.458092+0000","last_became_active":"2024-09-19T08:42:30.428684+0000","last_became_peered":"2024-09-19T08:42:30.428684+0000","last_unstale":"2024-09-19T08:42:31.458092+0000","last_undegraded":"2024-09-19T08:42:31.458092+0000","last_fullsized":"2024-09-19T08:42:31.458092+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:18.296523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.426036+0000","last_change":"2024-09-19T08:42:30.456942+0000","last_active":"2024-09-19T08:42:31.426036+0000","last_peered":"2024-09-19T08:42:31.426036+0000","last_clean":"2024-09-19T08:42:31.426036+0000","last_became_active":"2024-09-19T08:42:30.456357+0000","last_became_peered":"2024-09-19T08:42:30.456357+0000","last_unstale":"2024-09-19T08:42:31.426036+0000","last_undegraded":"2024-09-19T08:42:31.426036+0000","last_fullsized":"2024-09-19T08:42:31.426036+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:18.296523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.456428+0000","last_change":"2024-09-19T08:42:30.426146+0000","last_active":"2024-09-19T08:42:31.456428+0000","last_peered":"2024-09-19T08:42:31.456428+0000","last_clean":"2024-09-19T08:42:31.456428+0000","last_became_active":"2024-09-19T08:42:30.425677+0000","last_became_peered":"2024-09-19T08:42:30.425677+0000","last_unstale":"2024-09-19T08:42:31.456428+0000","last_undegraded":"2024-09-19T08:42:31.456428+0000","last_fullsized":"2024-09-19T08:42:31.456428+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:46:40.343564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.457322+0000","last_change":"2024-09-19T08:42:30.423051+0000","last_active":"2024-09-19T08:42:31.457322+0000","last_peered":"2024-09-19T08:42:31.457322+0000","last_clean":"2024-09-19T08:42:31.457322+0000","last_became_active":"2024-09-19T08:42:30.422249+0000","last_became_peered":"2024-09-19T08:42:30.422249+0000","last_unstale":"2024-09-19T08:42:31.457322+0000","last_undegraded":"2024-09-19T08:42:31.457322+0000","last_fullsized":"2024-09-19T08:42:31.457322+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:20.537518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.455552+0000","last_change":"2024-09-19T08:42:30.449478+0000","last_active":"2024-09-19T08:42:31.455552+0000","last_peered":"2024-09-19T08:42:31.455552+0000","last_clean":"2024-09-19T08:42:31.455552+0000","last_became_active":"2024-09-19T08:42:30.449078+0000","last_became_peered":"2024-09-19T08:42:30.449078+0000","last_unstale":"2024-09-19T08:42:31.455552+0000","last_undegraded":"2024-09-19T08:42:31.455552+0000","last_fullsized":"2024-09-19T08:42:31.455552+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:27:14.185735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.426078+0000","last_change":"2024-09-19T08:42:30.455857+0000","last_active":"2024-09-19T08:42:31.426078+0000","last_peered":"2024-09-19T08:42:31.426078+0000","last_clean":"2024-09-19T08:42:31.426078+0000","last_became_active":"2024-09-19T08:42:30.455048+0000","last_became_peered":"2024-09-19T08:42:30.455048+0000","last_unstale":"2024-09-19T08:42:31.426078+0000","last_undegraded":"2024-09-19T08:42:31.426078+0000","last_fullsized":"2024-09-19T08:42:31.426078+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:27:04.608943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.426187+0000","last_change":"2024-09-19T08:42:30.449376+0000","last_active":"2024-09-19T08:42:31.426187+0000","last_peered":"2024-09-19T08:42:31.426187+0000","last_clean":"2024-09-19T08:42:31.426187+0000","last_became_active":"2024-09-19T08:42:30.448718+0000","last_became_peered":"2024-09-19T08:42:30.448718+0000","last_unstale":"2024-09-19T08:42:31.426187+0000","last_undegraded":"2024-09-19T08:42:31.426187+0000","last_fullsized":"2024-09-19T08:42:31.426187+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:10:55.061767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401507+0000","last_change":"2024-09-19T08:42:30.434665+0000","last_active":"2024-09-19T08:42:31.401507+0000","last_peered":"2024-09-19T08:42:31.401507+0000","last_clean":"2024-09-19T08:42:31.401507+0000","last_became_active":"2024-09-19T08:42:30.433735+0000","last_became_peered":"2024-09-19T08:42:30.433735+0000","last_unstale":"2024-09-19T08:42:31.401507+0000","last_undegraded":"2024-09-19T08:42:31.401507+0000","last_fullsized":"2024-09-19T08:42:31.401507+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:37:26.942476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.458805+0000","last_change":"2024-09-19T08:42:30.459349+0000","last_active":"2024-09-19T08:42:31.458805+0000","last_peered":"2024-09-19T08:42:31.458805+0000","last_clean":"2024-09-19T08:42:31.458805+0000","last_became_active":"2024-09-19T08:42:30.458221+0000","last_became_peered":"2024-09-19T08:42:30.458221+0000","last_unstale":"2024-09-19T08:42:31.458805+0000","last_undegraded":"2024-09-19T08:42:31.458805+0000","last_fullsized":"2024-09-19T08:42:31.458805+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:47:20.596787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"67'4","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.458564+0000","last_change":"2024-09-19T08:42:29.380992+0000","last_active":"2024-09-19T08:42:31.458564+0000","last_peered":"2024-09-19T08:42:31.458564+0000","last_clean":"2024-09-19T08:42:31.458564+0000","last_became_active":"2024-09-19T08:42:29.380057+0000","last_became_peered":"2024-09-19T08:42:29.380057+0000","last_unstale":"2024-09-19T08:42:31.458564+0000","last_undegraded":"2024-09-19T08:42:31.458564+0000","last_fullsized":"2024-09-19T08:42:31.458564+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:32:38.780363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.398896+0000","last_change":"2024-09-19T08:42:30.417560+0000","last_active":"2024-09-19T08:42:31.398896+0000","last_peered":"2024-09-19T08:42:31.398896+0000","last_clean":"2024-09-19T08:42:31.398896+0000","last_became_active":"2024-09-19T08:42:30.416895+0000","last_became_peered":"2024-09-19T08:42:30.416895+0000","last_unstale":"2024-09-19T08:42:31.398896+0000","last_undegraded":"2024-09-19T08:42:31.398896+0000","last_fullsized":"2024-09-19T08:42:31.398896+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:43:24.527293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.429724+0000","last_change":"2024-09-19T08:42:29.373006+0000","last_active":"2024-09-19T08:42:31.429724+0000","last_peered":"2024-09-19T08:42:31.429724+0000","last_clean":"2024-09-19T08:42:31.429724+0000","last_became_active":"2024-09-19T08:42:29.371937+0000","last_became_peered":"2024-09-19T08:42:29.371937+0000","last_unstale":"2024-09-19T08:42:31.429724+0000","last_undegraded":"2024-09-19T08:42:31.429724+0000","last_fullsized":"2024-09-19T08:42:31.429724+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:19.528253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402198+0000","last_change":"2024-09-19T08:42:30.458129+0000","last_active":"2024-09-19T08:42:31.402198+0000","last_peered":"2024-09-19T08:42:31.402198+0000","last_clean":"2024-09-19T08:42:31.402198+0000","last_became_active":"2024-09-19T08:42:30.456984+0000","last_became_peered":"2024-09-19T08:42:30.456984+0000","last_unstale":"2024-09-19T08:42:31.402198+0000","last_undegraded":"2024-09-19T08:42:31.402198+0000","last_fullsized":"2024-09-19T08:42:31.402198+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:43:27.676061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.457712+0000","last_change":"2024-09-19T08:42:29.399152+0000","last_active":"2024-09-19T08:42:31.457712+0000","last_peered":"2024-09-19T08:42:31.457712+0000","last_clean":"2024-09-19T08:42:31.457712+0000","last_became_active":"2024-09-19T08:42:29.398078+0000","last_became_peered":"2024-09-19T08:42:29.398078+0000","last_unstale":"2024-09-19T08:42:31.457712+0000","last_undegraded":"2024-09-19T08:42:31.457712+0000","last_fullsized":"2024-09-19T08:42:31.457712+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:03:18.808677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.458619+0000","last_change":"2024-09-19T08:42:30.428464+0000","last_active":"2024-09-19T08:42:31.458619+0000","last_peered":"2024-09-19T08:42:31.458619+0000","last_clean":"2024-09-19T08:42:31.458619+0000","last_became_active":"2024-09-19T08:42:30.425762+0000","last_became_peered":"2024-09-19T08:42:30.425762+0000","last_unstale":"2024-09-19T08:42:31.458619+0000","last_undegraded":"2024-09-19T08:42:31.458619+0000","last_fullsized":"2024-09-19T08:42:31.458619+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:20:16.897649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402459+0000","last_change":"2024-09-19T08:42:29.395380+0000","last_active":"2024-09-19T08:42:31.402459+0000","last_peered":"2024-09-19T08:42:31.402459+0000","last_clean":"2024-09-19T08:42:31.402459+0000","last_became_active":"2024-09-19T08:42:29.394641+0000","last_became_peered":"2024-09-19T08:42:29.394641+0000","last_unstale":"2024-09-19T08:42:31.402459+0000","last_undegraded":"2024-09-19T08:42:31.402459+0000","last_fullsized":"2024-09-19T08:42:31.402459+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:26.586550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.426219+0000","last_change":"2024-09-19T08:42:30.443041+0000","last_active":"2024-09-19T08:42:31.426219+0000","last_peered":"2024-09-19T08:42:31.426219+0000","last_clean":"2024-09-19T08:42:31.426219+0000","last_became_active":"2024-09-19T08:42:30.426497+0000","last_became_peered":"2024-09-19T08:42:30.426497+0000","last_unstale":"2024-09-19T08:42:31.426219+0000","last_undegraded":"2024-09-19T08:42:31.426219+0000","last_fullsized":"2024-09-19T08:42:31.426219+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:48.938145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.455772+0000","last_change":"2024-09-19T08:42:29.385378+0000","last_active":"2024-09-19T08:42:31.455772+0000","last_peered":"2024-09-19T08:42:31.455772+0000","last_clean":"2024-09-19T08:42:31.455772+0000","last_became_active":"2024-09-19T08:42:29.385116+0000","last_became_peered":"2024-09-19T08:42:29.385116+0000","last_unstale":"2024-09-19T08:42:31.455772+0000","last_undegraded":"2024-09-19T08:42:31.455772+0000","last_fullsized":"2024-09-19T08:42:31.455772+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:53.241533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.457409+0000","last_change":"2024-09-19T08:42:30.431019+0000","last_active":"2024-09-19T08:42:31.457409+0000","last_peered":"2024-09-19T08:42:31.457409+0000","last_clean":"2024-09-19T08:42:31.457409+0000","last_became_active":"2024-09-19T08:42:30.430173+0000","last_became_peered":"2024-09-19T08:42:30.430173+0000","last_unstale":"2024-09-19T08:42:31.457409+0000","last_undegraded":"2024-09-19T08:42:31.457409+0000","last_fullsized":"2024-09-19T08:42:31.457409+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:19.467184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401528+0000","last_change":"2024-09-19T08:42:29.399398+0000","last_active":"2024-09-19T08:42:31.401528+0000","last_peered":"2024-09-19T08:42:31.401528+0000","last_clean":"2024-09-19T08:42:31.401528+0000","last_became_active":"2024-09-19T08:42:29.396716+0000","last_became_peered":"2024-09-19T08:42:29.396716+0000","last_unstale":"2024-09-19T08:42:31.401528+0000","last_undegraded":"2024-09-19T08:42:31.401528+0000","last_fullsized":"2024-09-19T08:42:31.401528+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:50:23.952850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.398010+0000","last_change":"2024-09-19T08:42:30.411536+0000","last_active":"2024-09-19T08:42:31.398010+0000","last_peered":"2024-09-19T08:42:31.398010+0000","last_clean":"2024-09-19T08:42:31.398010+0000","last_became_active":"2024-09-19T08:42:30.410812+0000","last_became_peered":"2024-09-19T08:42:30.410812+0000","last_unstale":"2024-09-19T08:42:31.398010+0000","last_undegraded":"2024-09-19T08:42:31.398010+0000","last_fullsized":"2024-09-19T08:42:31.398010+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:42:09.162003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.455444+0000","last_change":"2024-09-19T08:42:29.387234+0000","last_active":"2024-09-19T08:42:31.455444+0000","last_peered":"2024-09-19T08:42:31.455444+0000","last_clean":"2024-09-19T08:42:31.455444+0000","last_became_active":"2024-09-19T08:42:29.386856+0000","last_became_peered":"2024-09-19T08:42:29.386856+0000","last_unstale":"2024-09-19T08:42:31.455444+0000","last_undegraded":"2024-09-19T08:42:31.455444+0000","last_fullsized":"2024-09-19T08:42:31.455444+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:26.093340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.429774+0000","last_change":"2024-09-19T08:42:30.426900+0000","last_active":"2024-09-19T08:42:31.429774+0000","last_peered":"2024-09-19T08:42:31.429774+0000","last_clean":"2024-09-19T08:42:31.429774+0000","last_became_active":"2024-09-19T08:42:30.426646+0000","last_became_peered":"2024-09-19T08:42:30.426646+0000","last_unstale":"2024-09-19T08:42:31.429774+0000","last_undegraded":"2024-09-19T08:42:31.429774+0000","last_fullsized":"2024-09-19T08:42:31.429774+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:33.925297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.398066+0000","last_change":"2024-09-19T08:42:29.368732+0000","last_active":"2024-09-19T08:42:31.398066+0000","last_peered":"2024-09-19T08:42:31.398066+0000","last_clean":"2024-09-19T08:42:31.398066+0000","last_became_active":"2024-09-19T08:42:29.368426+0000","last_became_peered":"2024-09-19T08:42:29.368426+0000","last_unstale":"2024-09-19T08:42:31.398066+0000","last_undegraded":"2024-09-19T08:42:31.398066+0000","last_fullsized":"2024-09-19T08:42:31.398066+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:33:10.262511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.426339+0000","last_change":"2024-09-19T08:42:30.449116+0000","last_active":"2024-09-19T08:42:31.426339+0000","last_peered":"2024-09-19T08:42:31.426339+0000","last_clean":"2024-09-19T08:42:31.426339+0000","last_became_active":"2024-09-19T08:42:30.448401+0000","last_became_peered":"2024-09-19T08:42:30.448401+0000","last_unstale":"2024-09-19T08:42:31.426339+0000","last_undegraded":"2024-09-19T08:42:31.426339+0000","last_fullsized":"2024-09-19T08:42:31.426339+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:02:41.820517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.458224+0000","last_change":"2024-09-19T08:42:29.388802+0000","last_active":"2024-09-19T08:42:31.458224+0000","last_peered":"2024-09-19T08:42:31.458224+0000","last_clean":"2024-09-19T08:42:31.458224+0000","last_became_active":"2024-09-19T08:42:29.388077+0000","last_became_peered":"2024-09-19T08:42:29.388077+0000","last_unstale":"2024-09-19T08:42:31.458224+0000","last_undegraded":"2024-09-19T08:42:31.458224+0000","last_fullsized":"2024-09-19T08:42:31.458224+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:24:19.025320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.458479+0000","last_change":"2024-09-19T08:42:30.458537+0000","last_active":"2024-09-19T08:42:31.458479+0000","last_peered":"2024-09-19T08:42:31.458479+0000","last_clean":"2024-09-19T08:42:31.458479+0000","last_became_active":"2024-09-19T08:42:30.457874+0000","last_became_peered":"2024-09-19T08:42:30.457874+0000","last_unstale":"2024-09-19T08:42:31.458479+0000","last_undegraded":"2024-09-19T08:42:31.458479+0000","last_fullsized":"2024-09-19T08:42:31.458479+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:16:14.953301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402803+0000","last_change":"2024-09-19T08:42:29.399190+0000","last_active":"2024-09-19T08:42:31.402803+0000","last_peered":"2024-09-19T08:42:31.402803+0000","last_clean":"2024-09-19T08:42:31.402803+0000","last_became_active":"2024-09-19T08:42:29.396183+0000","last_became_peered":"2024-09-19T08:42:29.396183+0000","last_unstale":"2024-09-19T08:42:31.402803+0000","last_undegraded":"2024-09-19T08:42:31.402803+0000","last_fullsized":"2024-09-19T08:42:31.402803+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:21:01.013528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402155+0000","last_change":"2024-09-19T08:42:30.451725+0000","last_active":"2024-09-19T08:42:31.402155+0000","last_peered":"2024-09-19T08:42:31.402155+0000","last_clean":"2024-09-19T08:42:31.402155+0000","last_became_active":"2024-09-19T08:42:30.437086+0000","last_became_peered":"2024-09-19T08:42:30.437086+0000","last_unstale":"2024-09-19T08:42:31.402155+0000","last_undegraded":"2024-09-19T08:42:31.402155+0000","last_fullsized":"2024-09-19T08:42:31.402155+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:34.398500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"67'3","reported_seq":17,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.456414+0000","last_change":"2024-09-19T08:42:29.375916+0000","last_active":"2024-09-19T08:42:31.456414+0000","last_peered":"2024-09-19T08:42:31.456414+0000","last_clean":"2024-09-19T08:42:31.456414+0000","last_became_active":"2024-09-19T08:42:29.375398+0000","last_became_peered":"2024-09-19T08:42:29.375398+0000","last_unstale":"2024-09-19T08:42:31.456414+0000","last_undegraded":"2024-09-19T08:42:31.456414+0000","last_fullsized":"2024-09-19T08:42:31.456414+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:40:41.566855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.398886+0000","last_change":"2024-09-19T08:42:30.416450+0000","last_active":"2024-09-19T08:42:31.398886+0000","last_peered":"2024-09-19T08:42:31.398886+0000","last_clean":"2024-09-19T08:42:31.398886+0000","last_became_active":"2024-09-19T08:42:30.415819+0000","last_became_peered":"2024-09-19T08:42:30.415819+0000","last_unstale":"2024-09-19T08:42:31.398886+0000","last_undegraded":"2024-09-19T08:42:31.398886+0000","last_fullsized":"2024-09-19T08:42:31.398886+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:54.974767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"67'6","reported_seq":20,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.456098+0000","last_change":"2024-09-19T08:42:29.368802+0000","last_active":"2024-09-19T08:42:31.456098+0000","last_peered":"2024-09-19T08:42:31.456098+0000","last_clean":"2024-09-19T08:42:31.456098+0000","last_became_active":"2024-09-19T08:42:29.368414+0000","last_became_peered":"2024-09-19T08:42:29.368414+0000","last_unstale":"2024-09-19T08:42:31.456098+0000","last_undegraded":"2024-09-19T08:42:31.456098+0000","last_fullsized":"2024-09-19T08:42:31.456098+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:18.889196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.457614+0000","last_change":"2024-09-19T08:42:30.428859+0000","last_active":"2024-09-19T08:42:31.457614+0000","last_peered":"2024-09-19T08:42:31.457614+0000","last_clean":"2024-09-19T08:42:31.457614+0000","last_became_active":"2024-09-19T08:42:30.426388+0000","last_became_peered":"2024-09-19T08:42:30.426388+0000","last_unstale":"2024-09-19T08:42:31.457614+0000","last_undegraded":"2024-09-19T08:42:31.457614+0000","last_fullsized":"2024-09-19T08:42:31.457614+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:02:41.820517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.455549+0000","last_change":"2024-09-19T08:42:29.369719+0000","last_active":"2024-09-19T08:42:31.455549+0000","last_peered":"2024-09-19T08:42:31.455549+0000","last_clean":"2024-09-19T08:42:31.455549+0000","last_became_active":"2024-09-19T08:42:29.369306+0000","last_became_peered":"2024-09-19T08:42:29.369306+0000","last_unstale":"2024-09-19T08:42:31.455549+0000","last_undegraded":"2024-09-19T08:42:31.455549+0000","last_fullsized":"2024-09-19T08:42:31.455549+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:57.793452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.426092+0000","last_change":"2024-09-19T08:42:30.443062+0000","last_active":"2024-09-19T08:42:31.426092+0000","last_peered":"2024-09-19T08:42:31.426092+0000","last_clean":"2024-09-19T08:42:31.426092+0000","last_became_active":"2024-09-19T08:42:30.442440+0000","last_became_peered":"2024-09-19T08:42:30.442440+0000","last_unstale":"2024-09-19T08:42:31.426092+0000","last_undegraded":"2024-09-19T08:42:31.426092+0000","last_fullsized":"2024-09-19T08:42:31.426092+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:52:10.022065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401738+0000","last_change":"2024-09-19T08:42:29.399130+0000","last_active":"2024-09-19T08:42:31.401738+0000","last_peered":"2024-09-19T08:42:31.401738+0000","last_clean":"2024-09-19T08:42:31.401738+0000","last_became_active":"2024-09-19T08:42:29.396039+0000","last_became_peered":"2024-09-19T08:42:29.396039+0000","last_unstale":"2024-09-19T08:42:31.401738+0000","last_undegraded":"2024-09-19T08:42:31.401738+0000","last_fullsized":"2024-09-19T08:42:31.401738+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:54.277351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.429445+0000","last_change":"2024-09-19T08:42:30.428966+0000","last_active":"2024-09-19T08:42:31.429445+0000","last_peered":"2024-09-19T08:42:31.429445+0000","last_clean":"2024-09-19T08:42:31.429445+0000","last_became_active":"2024-09-19T08:42:30.428673+0000","last_became_peered":"2024-09-19T08:42:30.428673+0000","last_unstale":"2024-09-19T08:42:31.429445+0000","last_undegraded":"2024-09-19T08:42:31.429445+0000","last_fullsized":"2024-09-19T08:42:31.429445+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:07:43.736470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.425646+0000","last_change":"2024-09-19T08:42:29.377410+0000","last_active":"2024-09-19T08:42:31.425646+0000","last_peered":"2024-09-19T08:42:31.425646+0000","last_clean":"2024-09-19T08:42:31.425646+0000","last_became_active":"2024-09-19T08:42:29.376673+0000","last_became_peered":"2024-09-19T08:42:29.376673+0000","last_unstale":"2024-09-19T08:42:31.425646+0000","last_undegraded":"2024-09-19T08:42:31.425646+0000","last_fullsized":"2024-09-19T08:42:31.425646+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:09.382166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.398460+0000","last_change":"2024-09-19T08:42:30.420451+0000","last_active":"2024-09-19T08:42:31.398460+0000","last_peered":"2024-09-19T08:42:31.398460+0000","last_clean":"2024-09-19T08:42:31.398460+0000","last_became_active":"2024-09-19T08:42:30.420221+0000","last_became_peered":"2024-09-19T08:42:30.420221+0000","last_unstale":"2024-09-19T08:42:31.398460+0000","last_undegraded":"2024-09-19T08:42:31.398460+0000","last_fullsized":"2024-09-19T08:42:31.398460+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:33:27.096060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"67'8","reported_seq":22,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.456394+0000","last_change":"2024-09-19T08:42:29.385268+0000","last_active":"2024-09-19T08:42:31.456394+0000","last_peered":"2024-09-19T08:42:31.456394+0000","last_clean":"2024-09-19T08:42:31.456394+0000","last_became_active":"2024-09-19T08:42:29.384547+0000","last_became_peered":"2024-09-19T08:42:29.384547+0000","last_unstale":"2024-09-19T08:42:31.456394+0000","last_undegraded":"2024-09-19T08:42:31.456394+0000","last_fullsized":"2024-09-19T08:42:31.456394+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:15.005825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.458668+0000","last_change":"2024-09-19T08:42:30.424337+0000","last_active":"2024-09-19T08:42:31.458668+0000","last_peered":"2024-09-19T08:42:31.458668+0000","last_clean":"2024-09-19T08:42:31.458668+0000","last_became_active":"2024-09-19T08:42:30.423013+0000","last_became_peered":"2024-09-19T08:42:30.423013+0000","last_unstale":"2024-09-19T08:42:31.458668+0000","last_undegraded":"2024-09-19T08:42:31.458668+0000","last_fullsized":"2024-09-19T08:42:31.458668+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:52:10.022065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402598+0000","last_change":"2024-09-19T08:42:29.400466+0000","last_active":"2024-09-19T08:42:31.402598+0000","last_peered":"2024-09-19T08:42:31.402598+0000","last_clean":"2024-09-19T08:42:31.402598+0000","last_became_active":"2024-09-19T08:42:29.395739+0000","last_became_peered":"2024-09-19T08:42:29.395739+0000","last_unstale":"2024-09-19T08:42:31.402598+0000","last_undegraded":"2024-09-19T08:42:31.402598+0000","last_fullsized":"2024-09-19T08:42:31.402598+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:41:59.302829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.458847+0000","last_change":"2024-09-19T08:42:30.425830+0000","last_active":"2024-09-19T08:42:31.458847+0000","last_peered":"2024-09-19T08:42:31.458847+0000","last_clean":"2024-09-19T08:42:31.458847+0000","last_became_active":"2024-09-19T08:42:30.425130+0000","last_became_peered":"2024-09-19T08:42:30.425130+0000","last_unstale":"2024-09-19T08:42:31.458847+0000","last_undegraded":"2024-09-19T08:42:31.458847+0000","last_fullsized":"2024-09-19T08:42:31.458847+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:59.839015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.458802+0000","last_change":"2024-09-19T08:42:29.398636+0000","last_active":"2024-09-19T08:42:31.458802+0000","last_peered":"2024-09-19T08:42:31.458802+0000","last_clean":"2024-09-19T08:42:31.458802+0000","last_became_active":"2024-09-19T08:42:29.398016+0000","last_became_peered":"2024-09-19T08:42:29.398016+0000","last_unstale":"2024-09-19T08:42:31.458802+0000","last_undegraded":"2024-09-19T08:42:31.458802+0000","last_fullsized":"2024-09-19T08:42:31.458802+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:12.758285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.456032+0000","last_change":"2024-09-19T08:42:30.420773+0000","last_active":"2024-09-19T08:42:31.456032+0000","last_peered":"2024-09-19T08:42:31.456032+0000","last_clean":"2024-09-19T08:42:31.456032+0000","last_became_active":"2024-09-19T08:42:30.420139+0000","last_became_peered":"2024-09-19T08:42:30.420139+0000","last_unstale":"2024-09-19T08:42:31.456032+0000","last_undegraded":"2024-09-19T08:42:31.456032+0000","last_fullsized":"2024-09-19T08:42:31.456032+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:59:42.739064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402346+0000","last_change":"2024-09-19T08:42:29.399885+0000","last_active":"2024-09-19T08:42:31.402346+0000","last_peered":"2024-09-19T08:42:31.402346+0000","last_clean":"2024-09-19T08:42:31.402346+0000","last_became_active":"2024-09-19T08:42:29.396798+0000","last_became_peered":"2024-09-19T08:42:29.396798+0000","last_unstale":"2024-09-19T08:42:31.402346+0000","last_undegraded":"2024-09-19T08:42:31.402346+0000","last_fullsized":"2024-09-19T08:42:31.402346+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:07.393673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.399251+0000","last_change":"2024-09-19T08:42:30.411732+0000","last_active":"2024-09-19T08:42:31.399251+0000","last_peered":"2024-09-19T08:42:31.399251+0000","last_clean":"2024-09-19T08:42:31.399251+0000","last_became_active":"2024-09-19T08:42:30.411033+0000","last_became_peered":"2024-09-19T08:42:30.411033+0000","last_unstale":"2024-09-19T08:42:31.399251+0000","last_undegraded":"2024-09-19T08:42:31.399251+0000","last_fullsized":"2024-09-19T08:42:31.399251+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:03:24.038264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.458041+0000","last_change":"2024-09-19T08:42:29.389208+0000","last_active":"2024-09-19T08:42:31.458041+0000","last_peered":"2024-09-19T08:42:31.458041+0000","last_clean":"2024-09-19T08:42:31.458041+0000","last_became_active":"2024-09-19T08:42:29.388479+0000","last_became_peered":"2024-09-19T08:42:29.388479+0000","last_unstale":"2024-09-19T08:42:31.458041+0000","last_undegraded":"2024-09-19T08:42:31.458041+0000","last_fullsized":"2024-09-19T08:42:31.458041+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:07.217342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.455688+0000","last_change":"2024-09-19T08:42:30.418094+0000","last_active":"2024-09-19T08:42:31.455688+0000","last_peered":"2024-09-19T08:42:31.455688+0000","last_clean":"2024-09-19T08:42:31.455688+0000","last_became_active":"2024-09-19T08:42:30.415422+0000","last_became_peered":"2024-09-19T08:42:30.415422+0000","last_unstale":"2024-09-19T08:42:31.455688+0000","last_undegraded":"2024-09-19T08:42:31.455688+0000","last_fullsized":"2024-09-19T08:42:31.455688+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:31:25.534663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401789+0000","last_change":"2024-09-19T08:42:29.395444+0000","last_active":"2024-09-19T08:42:31.401789+0000","last_peered":"2024-09-19T08:42:31.401789+0000","last_clean":"2024-09-19T08:42:31.401789+0000","last_became_active":"2024-09-19T08:42:29.394674+0000","last_became_peered":"2024-09-19T08:42:29.394674+0000","last_unstale":"2024-09-19T08:42:31.401789+0000","last_undegraded":"2024-09-19T08:42:31.401789+0000","last_fullsized":"2024-09-19T08:42:31.401789+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:42:57.681916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.455278+0000","last_change":"2024-09-19T08:42:30.438215+0000","last_active":"2024-09-19T08:42:31.455278+0000","last_peered":"2024-09-19T08:42:31.455278+0000","last_clean":"2024-09-19T08:42:31.455278+0000","last_became_active":"2024-09-19T08:42:30.437782+0000","last_became_peered":"2024-09-19T08:42:30.437782+0000","last_unstale":"2024-09-19T08:42:31.455278+0000","last_undegraded":"2024-09-19T08:42:31.455278+0000","last_fullsized":"2024-09-19T08:42:31.455278+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:08.138224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.426283+0000","last_change":"2024-09-19T08:42:29.375051+0000","last_active":"2024-09-19T08:42:31.426283+0000","last_peered":"2024-09-19T08:42:31.426283+0000","last_clean":"2024-09-19T08:42:31.426283+0000","last_became_active":"2024-09-19T08:42:29.374145+0000","last_became_peered":"2024-09-19T08:42:29.374145+0000","last_unstale":"2024-09-19T08:42:31.426283+0000","last_undegraded":"2024-09-19T08:42:31.426283+0000","last_fullsized":"2024-09-19T08:42:31.426283+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:44.656487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.455364+0000","last_change":"2024-09-19T08:42:30.439052+0000","last_active":"2024-09-19T08:42:31.455364+0000","last_peered":"2024-09-19T08:42:31.455364+0000","last_clean":"2024-09-19T08:42:31.455364+0000","last_became_active":"2024-09-19T08:42:30.438753+0000","last_became_peered":"2024-09-19T08:42:30.438753+0000","last_unstale":"2024-09-19T08:42:31.455364+0000","last_undegraded":"2024-09-19T08:42:31.455364+0000","last_fullsized":"2024-09-19T08:42:31.455364+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:01:53.973114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.429311+0000","last_change":"2024-09-19T08:42:29.372223+0000","last_active":"2024-09-19T08:42:31.429311+0000","last_peered":"2024-09-19T08:42:31.429311+0000","last_clean":"2024-09-19T08:42:31.429311+0000","last_became_active":"2024-09-19T08:42:29.371947+0000","last_became_peered":"2024-09-19T08:42:29.371947+0000","last_unstale":"2024-09-19T08:42:31.429311+0000","last_undegraded":"2024-09-19T08:42:31.429311+0000","last_fullsized":"2024-09-19T08:42:31.429311+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:59:10.453344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.429694+0000","last_change":"2024-09-19T08:42:30.419025+0000","last_active":"2024-09-19T08:42:31.429694+0000","last_peered":"2024-09-19T08:42:31.429694+0000","last_clean":"2024-09-19T08:42:31.429694+0000","last_became_active":"2024-09-19T08:42:30.418674+0000","last_became_peered":"2024-09-19T08:42:30.418674+0000","last_unstale":"2024-09-19T08:42:31.429694+0000","last_undegraded":"2024-09-19T08:42:31.429694+0000","last_fullsized":"2024-09-19T08:42:31.429694+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:03:03.502166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402779+0000","last_change":"2024-09-19T08:42:29.396548+0000","last_active":"2024-09-19T08:42:31.402779+0000","last_peered":"2024-09-19T08:42:31.402779+0000","last_clean":"2024-09-19T08:42:31.402779+0000","last_became_active":"2024-09-19T08:42:29.394896+0000","last_became_peered":"2024-09-19T08:42:29.394896+0000","last_unstale":"2024-09-19T08:42:31.402779+0000","last_undegraded":"2024-09-19T08:42:31.402779+0000","last_fullsized":"2024-09-19T08:42:31.402779+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:27:20.145977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.426300+0000","last_change":"2024-09-19T08:42:30.447256+0000","last_active":"2024-09-19T08:42:31.426300+0000","last_peered":"2024-09-19T08:42:31.426300+0000","last_clean":"2024-09-19T08:42:31.426300+0000","last_became_active":"2024-09-19T08:42:30.446395+0000","last_became_peered":"2024-09-19T08:42:30.446395+0000","last_unstale":"2024-09-19T08:42:31.426300+0000","last_undegraded":"2024-09-19T08:42:31.426300+0000","last_fullsized":"2024-09-19T08:42:31.426300+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:07:43.736470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"68'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.426480+0000","last_change":"2024-09-19T08:42:29.385128+0000","last_active":"2024-09-19T08:42:31.426480+0000","last_peered":"2024-09-19T08:42:31.426480+0000","last_clean":"2024-09-19T08:42:31.426480+0000","last_became_active":"2024-09-19T08:42:29.384397+0000","last_became_peered":"2024-09-19T08:42:29.384397+0000","last_unstale":"2024-09-19T08:42:31.426480+0000","last_undegraded":"2024-09-19T08:42:31.426480+0000","last_fullsized":"2024-09-19T08:42:31.426480+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:14:13.112131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.458245+0000","last_change":"2024-09-19T08:42:30.459124+0000","last_active":"2024-09-19T08:42:31.458245+0000","last_peered":"2024-09-19T08:42:31.458245+0000","last_clean":"2024-09-19T08:42:31.458245+0000","last_became_active":"2024-09-19T08:42:30.458447+0000","last_became_peered":"2024-09-19T08:42:30.458447+0000","last_unstale":"2024-09-19T08:42:31.458245+0000","last_undegraded":"2024-09-19T08:42:31.458245+0000","last_fullsized":"2024-09-19T08:42:31.458245+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:57:27.072229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.399366+0000","last_change":"2024-09-19T08:42:29.375037+0000","last_active":"2024-09-19T08:42:31.399366+0000","last_peered":"2024-09-19T08:42:31.399366+0000","last_clean":"2024-09-19T08:42:31.399366+0000","last_became_active":"2024-09-19T08:42:29.374723+0000","last_became_peered":"2024-09-19T08:42:29.374723+0000","last_unstale":"2024-09-19T08:42:31.399366+0000","last_undegraded":"2024-09-19T08:42:31.399366+0000","last_fullsized":"2024-09-19T08:42:31.399366+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:07:10.884601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.458931+0000","last_change":"2024-09-19T08:42:30.449648+0000","last_active":"2024-09-19T08:42:31.458931+0000","last_peered":"2024-09-19T08:42:31.458931+0000","last_clean":"2024-09-19T08:42:31.458931+0000","last_became_active":"2024-09-19T08:42:30.449054+0000","last_became_peered":"2024-09-19T08:42:30.449054+0000","last_unstale":"2024-09-19T08:42:31.458931+0000","last_undegraded":"2024-09-19T08:42:31.458931+0000","last_fullsized":"2024-09-19T08:42:31.458931+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:41:07.372011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.429507+0000","last_change":"2024-09-19T08:42:29.384202+0000","last_active":"2024-09-19T08:42:31.429507+0000","last_peered":"2024-09-19T08:42:31.429507+0000","last_clean":"2024-09-19T08:42:31.429507+0000","last_became_active":"2024-09-19T08:42:29.383866+0000","last_became_peered":"2024-09-19T08:42:29.383866+0000","last_unstale":"2024-09-19T08:42:31.429507+0000","last_undegraded":"2024-09-19T08:42:31.429507+0000","last_fullsized":"2024-09-19T08:42:31.429507+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:03:34.140561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.398524+0000","last_change":"2024-09-19T08:42:30.416404+0000","last_active":"2024-09-19T08:42:31.398524+0000","last_peered":"2024-09-19T08:42:31.398524+0000","last_clean":"2024-09-19T08:42:31.398524+0000","last_became_active":"2024-09-19T08:42:30.415848+0000","last_became_peered":"2024-09-19T08:42:30.415848+0000","last_unstale":"2024-09-19T08:42:31.398524+0000","last_undegraded":"2024-09-19T08:42:31.398524+0000","last_fullsized":"2024-09-19T08:42:31.398524+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:35.148871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.398338+0000","last_change":"2024-09-19T08:42:29.381869+0000","last_active":"2024-09-19T08:42:31.398338+0000","last_peered":"2024-09-19T08:42:31.398338+0000","last_clean":"2024-09-19T08:42:31.398338+0000","last_became_active":"2024-09-19T08:42:29.381590+0000","last_became_peered":"2024-09-19T08:42:29.381590+0000","last_unstale":"2024-09-19T08:42:31.398338+0000","last_undegraded":"2024-09-19T08:42:31.398338+0000","last_fullsized":"2024-09-19T08:42:31.398338+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:22:08.333844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401692+0000","last_change":"2024-09-19T08:42:30.459642+0000","last_active":"2024-09-19T08:42:31.401692+0000","last_peered":"2024-09-19T08:42:31.401692+0000","last_clean":"2024-09-19T08:42:31.401692+0000","last_became_active":"2024-09-19T08:42:30.459010+0000","last_became_peered":"2024-09-19T08:42:30.459010+0000","last_unstale":"2024-09-19T08:42:31.401692+0000","last_undegraded":"2024-09-19T08:42:31.401692+0000","last_fullsized":"2024-09-19T08:42:31.401692+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:03:33.044498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.455327+0000","last_change":"2024-09-19T08:42:29.387334+0000","last_active":"2024-09-19T08:42:31.455327+0000","last_peered":"2024-09-19T08:42:31.455327+0000","last_clean":"2024-09-19T08:42:31.455327+0000","last_became_active":"2024-09-19T08:42:29.386999+0000","last_became_peered":"2024-09-19T08:42:29.386999+0000","last_unstale":"2024-09-19T08:42:31.455327+0000","last_undegraded":"2024-09-19T08:42:31.455327+0000","last_fullsized":"2024-09-19T08:42:31.455327+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:26:36.338910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.455271+0000","last_change":"2024-09-19T08:42:30.411643+0000","last_active":"2024-09-19T08:42:31.455271+0000","last_peered":"2024-09-19T08:42:31.455271+0000","last_clean":"2024-09-19T08:42:31.455271+0000","last_became_active":"2024-09-19T08:42:30.410592+0000","last_became_peered":"2024-09-19T08:42:30.410592+0000","last_unstale":"2024-09-19T08:42:31.455271+0000","last_undegraded":"2024-09-19T08:42:31.455271+0000","last_fullsized":"2024-09-19T08:42:31.455271+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:15:53.503677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.456427+0000","last_change":"2024-09-19T08:42:29.382377+0000","last_active":"2024-09-19T08:42:31.456427+0000","last_peered":"2024-09-19T08:42:31.456427+0000","last_clean":"2024-09-19T08:42:31.456427+0000","last_became_active":"2024-09-19T08:42:29.381685+0000","last_became_peered":"2024-09-19T08:42:29.381685+0000","last_unstale":"2024-09-19T08:42:31.456427+0000","last_undegraded":"2024-09-19T08:42:31.456427+0000","last_fullsized":"2024-09-19T08:42:31.456427+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:15:13.220029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401444+0000","last_change":"2024-09-19T08:42:30.431276+0000","last_active":"2024-09-19T08:42:31.401444+0000","last_peered":"2024-09-19T08:42:31.401444+0000","last_clean":"2024-09-19T08:42:31.401444+0000","last_became_active":"2024-09-19T08:42:30.430462+0000","last_became_peered":"2024-09-19T08:42:30.430462+0000","last_unstale":"2024-09-19T08:42:31.401444+0000","last_undegraded":"2024-09-19T08:42:31.401444+0000","last_fullsized":"2024-09-19T08:42:31.401444+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:50.219137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401735+0000","last_change":"2024-09-19T08:42:29.373808+0000","last_active":"2024-09-19T08:42:31.401735+0000","last_peered":"2024-09-19T08:42:31.401735+0000","last_clean":"2024-09-19T08:42:31.401735+0000","last_became_active":"2024-09-19T08:42:29.373099+0000","last_became_peered":"2024-09-19T08:42:29.373099+0000","last_unstale":"2024-09-19T08:42:31.401735+0000","last_undegraded":"2024-09-19T08:42:31.401735+0000","last_fullsized":"2024-09-19T08:42:31.401735+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:48.444653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.455409+0000","last_change":"2024-09-19T08:42:30.458262+0000","last_active":"2024-09-19T08:42:31.455409+0000","last_peered":"2024-09-19T08:42:31.455409+0000","last_clean":"2024-09-19T08:42:31.455409+0000","last_became_active":"2024-09-19T08:42:30.457905+0000","last_became_peered":"2024-09-19T08:42:30.457905+0000","last_unstale":"2024-09-19T08:42:31.455409+0000","last_undegraded":"2024-09-19T08:42:31.455409+0000","last_fullsized":"2024-09-19T08:42:31.455409+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:16.269375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401741+0000","last_change":"2024-09-19T08:42:29.399988+0000","last_active":"2024-09-19T08:42:31.401741+0000","last_peered":"2024-09-19T08:42:31.401741+0000","last_clean":"2024-09-19T08:42:31.401741+0000","last_became_active":"2024-09-19T08:42:29.399260+0000","last_became_peered":"2024-09-19T08:42:29.399260+0000","last_unstale":"2024-09-19T08:42:31.401741+0000","last_undegraded":"2024-09-19T08:42:31.401741+0000","last_fullsized":"2024-09-19T08:42:31.401741+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:06.453660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":254,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.457218+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:42:31.457218+0000","last_peered":"2024-09-19T08:42:31.457218+0000","last_clean":"2024-09-19T08:42:31.457218+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:42:31.457218+0000","last_undegraded":"2024-09-19T08:42:31.457218+0000","last_fullsized":"2024-09-19T08:42:31.457218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509496,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1421312,"data_stored":933700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509498,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":823296,"data_stored":342806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509498,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31992,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339848,"statfs":{"total":96636764160,"available":96604004352,"internally_reserved":0,"allocated":823296,"data_stored":342786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509495,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":831488,"data_stored":344328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509497,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":839680,"data_stored":344944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509495,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1413120,"data_stored":933084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574904,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":827392,"data_stored":342808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542200,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":827392,"data_stored":342828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:42:36.340 INFO:tasks.ceph.mds.a.smithi098.stderr:2024-09-19T08:42:36.341+0000 7f909d3b3640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T08:42:37.887 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:42:38.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.017+0000 7fb96344a640 1 Processor -- start 2024-09-19T08:42:38.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.017+0000 7fb96344a640 1 -- start start 2024-09-19T08:42:38.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.017+0000 7fb96344a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb95c06c3a0 0x7fb95c0cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:38.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.021+0000 7fb96344a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb95c0cf900 0x7fb95c0723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:38.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.021+0000 7fb96344a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb95c06d310 con 0x7fb95c15c550 2024-09-19T08:42:38.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.021+0000 7fb96344a640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fb95c06d5f0 con 0x7fb95c0cf900 2024-09-19T08:42:38.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.021+0000 7fb96344a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb95c06d8d0 con 0x7fb95c06c3a0 2024-09-19T08:42:38.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.021+0000 7fb9619e7640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb95c0cf900 0x7fb95c0723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:38.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.021+0000 7fb9619e7640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb95c0cf900 0x7fb95c0723a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36612/0 (socket says 172.21.15.98:36612) 2024-09-19T08:42:38.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.021+0000 7fb9619e7640 1 -- 172.21.15.98:0/3987408108 learned_addr learned my addr 172.21.15.98:0/3987408108 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:38.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.021+0000 7fb9609e5640 1 --2- 172.21.15.98:0/3987408108 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb95c06c3a0 0x7fb95c0cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:38.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.021+0000 7fb9619e7640 1 -- 172.21.15.98:0/3987408108 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb95c06c3a0 msgr2=0x7fb95c0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:38.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.021+0000 7fb9619e7640 1 --2- 172.21.15.98:0/3987408108 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb95c06c3a0 0x7fb95c0cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:38.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.021+0000 7fb9619e7640 1 -- 172.21.15.98:0/3987408108 >> v1:172.21.15.139:6789/0 conn(0x7fb95c15c550 legacy=0x7fb95c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:38.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.021+0000 7fb9619e7640 1 -- 172.21.15.98:0/3987408108 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb95c06dbb0 con 0x7fb95c0cf900 2024-09-19T08:42:38.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.021+0000 7fb9619e7640 1 --2- 172.21.15.98:0/3987408108 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb95c0cf900 0x7fb95c0723a0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fb954009e50 tx=0x7fb95401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65083e9e6c6ee532 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:38.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.021+0000 7fb953fff640 1 -- 172.21.15.98:0/3987408108 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9540a5780 con 0x7fb95c0cf900 2024-09-19T08:42:38.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.021+0000 7fb953fff640 1 -- 172.21.15.98:0/3987408108 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9540a5920 con 0x7fb95c0cf900 2024-09-19T08:42:38.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.021+0000 7fb953fff640 1 -- 172.21.15.98:0/3987408108 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9540a5cf0 con 0x7fb95c0cf900 2024-09-19T08:42:38.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.021+0000 7fb96344a640 1 -- 172.21.15.98:0/3987408108 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb95c0cf900 msgr2=0x7fb95c0723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:38.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.021+0000 7fb96344a640 1 --2- 172.21.15.98:0/3987408108 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb95c0cf900 0x7fb95c0723a0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fb954009e50 tx=0x7fb95401c9f0 comp rx=0 tx=0).stop 2024-09-19T08:42:38.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.021+0000 7fb96344a640 1 -- 172.21.15.98:0/3987408108 shutdown_connections 2024-09-19T08:42:38.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.021+0000 7fb96344a640 1 --2- 172.21.15.98:0/3987408108 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb95c0cf900 0x7fb95c0723a0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:38.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.021+0000 7fb96344a640 1 --2- 172.21.15.98:0/3987408108 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb95c06c3a0 0x7fb95c0cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:38.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.021+0000 7fb96344a640 1 -- 172.21.15.98:0/3987408108 >> 172.21.15.98:0/3987408108 conn(0x7fb95c0b2010 msgr2=0x7fb95c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:38.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.021+0000 7fb96344a640 1 -- 172.21.15.98:0/3987408108 shutdown_connections 2024-09-19T08:42:38.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.021+0000 7fb96344a640 1 -- 172.21.15.98:0/3987408108 wait complete. 2024-09-19T08:42:38.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.025+0000 7fb96344a640 1 Processor -- start 2024-09-19T08:42:38.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.025+0000 7fb96344a640 1 -- start start 2024-09-19T08:42:38.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.025+0000 7fb96344a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb95c06c3a0 0x7fb95c13dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:38.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.025+0000 7fb96344a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb95c0cf900 0x7fb95c153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:38.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.025+0000 7fb96344a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb95c15c550 0x7fb95c153540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:38.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.025+0000 7fb96344a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c073480 con 0x7fb95c15c550 2024-09-19T08:42:38.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.025+0000 7fb96344a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c073300 con 0x7fb95c0cf900 2024-09-19T08:42:38.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.025+0000 7fb96344a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb95c073600 con 0x7fb95c06c3a0 2024-09-19T08:42:38.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.025+0000 7fb9619e7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb95c15c550 0x7fb95c153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:38.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.025+0000 7fb9619e7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb95c15c550 0x7fb95c153540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36620/0 (socket says 172.21.15.98:36620) 2024-09-19T08:42:38.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.025+0000 7fb9619e7640 1 -- 172.21.15.98:0/265397352 learned_addr learned my addr 172.21.15.98:0/265397352 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:38.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.025+0000 7fb9609e5640 1 --2- 172.21.15.98:0/265397352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb95c0cf900 0x7fb95c153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:38.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.025+0000 7fb9611e6640 1 --2- 172.21.15.98:0/265397352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb95c06c3a0 0x7fb95c13dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:38.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.025+0000 7fb9619e7640 1 -- 172.21.15.98:0/265397352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb95c06c3a0 msgr2=0x7fb95c13dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:38.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.025+0000 7fb9619e7640 1 --2- 172.21.15.98:0/265397352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb95c06c3a0 0x7fb95c13dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:38.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.025+0000 7fb9619e7640 1 -- 172.21.15.98:0/265397352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb95c0cf900 msgr2=0x7fb95c153000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:38.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.025+0000 7fb9619e7640 1 --2- 172.21.15.98:0/265397352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb95c0cf900 0x7fb95c153000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:38.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.025+0000 7fb9619e7640 1 -- 172.21.15.98:0/265397352 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb95c0cb820 con 0x7fb95c15c550 2024-09-19T08:42:38.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.025+0000 7fb9619e7640 1 --2- 172.21.15.98:0/265397352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb95c15c550 0x7fb95c153540 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fb9540af040 tx=0x7fb954019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:38.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.025+0000 7fb951ffb640 1 -- 172.21.15.98:0/265397352 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9540b8670 con 0x7fb95c15c550 2024-09-19T08:42:38.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.025+0000 7fb951ffb640 1 -- 172.21.15.98:0/265397352 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9540be070 con 0x7fb95c15c550 2024-09-19T08:42:38.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.025+0000 7fb951ffb640 1 -- 172.21.15.98:0/265397352 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9540b8a90 con 0x7fb95c15c550 2024-09-19T08:42:38.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.025+0000 7fb96344a640 1 -- 172.21.15.98:0/265397352 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb95c0cbb00 con 0x7fb95c15c550 2024-09-19T08:42:38.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.025+0000 7fb96344a640 1 -- 172.21.15.98:0/265397352 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb95c0cc0c0 con 0x7fb95c15c550 2024-09-19T08:42:38.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.029+0000 7fb96344a640 1 -- 172.21.15.98:0/265397352 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb95c0cf3c0 con 0x7fb95c15c550 2024-09-19T08:42:38.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.029+0000 7fb951ffb640 1 -- 172.21.15.98:0/265397352 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fb9540b8c30 con 0x7fb95c15c550 2024-09-19T08:42:38.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.029+0000 7fb951ffb640 1 --2- 172.21.15.98:0/265397352 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb940067c20 0x7fb94006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:38.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.029+0000 7fb951ffb640 1 -- 172.21.15.98:0/265397352 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7fb9541213c0 con 0x7fb95c15c550 2024-09-19T08:42:38.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.029+0000 7fb9611e6640 1 --2- 172.21.15.98:0/265397352 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb940067c20 0x7fb94006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:38.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.029+0000 7fb9611e6640 1 --2- 172.21.15.98:0/265397352 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb940067c20 0x7fb94006a0e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb958004e30 tx=0x7fb958009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:38.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.033+0000 7fb951ffb640 1 -- 172.21.15.98:0/265397352 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb9540165e0 con 0x7fb95c15c550 2024-09-19T08:42:38.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.189+0000 7fb96344a640 1 -- 172.21.15.98:0/265397352 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb95c0ccb20 con 0x7fb940067c20 2024-09-19T08:42:38.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.197+0000 7fb951ffb640 1 -- 172.21.15.98:0/265397352 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252053 (secure 0 0 0) 0x7fb95c0ccb20 con 0x7fb940067c20 2024-09-19T08:42:38.202 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:42:38.203 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:42:38.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.201+0000 7fb96344a640 1 -- 172.21.15.98:0/265397352 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb940067c20 msgr2=0x7fb94006a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:38.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.201+0000 7fb96344a640 1 --2- 172.21.15.98:0/265397352 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb940067c20 0x7fb94006a0e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb958004e30 tx=0x7fb958009040 comp rx=0 tx=0).stop 2024-09-19T08:42:38.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.201+0000 7fb96344a640 1 -- 172.21.15.98:0/265397352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb95c15c550 msgr2=0x7fb95c153540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:38.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.201+0000 7fb96344a640 1 --2- 172.21.15.98:0/265397352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb95c15c550 0x7fb95c153540 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fb9540af040 tx=0x7fb954019790 comp rx=0 tx=0).stop 2024-09-19T08:42:38.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.201+0000 7fb96344a640 1 -- 172.21.15.98:0/265397352 shutdown_connections 2024-09-19T08:42:38.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.201+0000 7fb96344a640 1 --2- 172.21.15.98:0/265397352 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb940067c20 0x7fb94006a0e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:38.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.201+0000 7fb96344a640 1 --2- 172.21.15.98:0/265397352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb95c15c550 0x7fb95c153540 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:38.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.201+0000 7fb96344a640 1 --2- 172.21.15.98:0/265397352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb95c0cf900 0x7fb95c153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:38.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.201+0000 7fb96344a640 1 --2- 172.21.15.98:0/265397352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb95c06c3a0 0x7fb95c13dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:38.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.201+0000 7fb96344a640 1 -- 172.21.15.98:0/265397352 >> 172.21.15.98:0/265397352 conn(0x7fb95c0b2010 msgr2=0x7fb95c157880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:38.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.201+0000 7fb96344a640 1 -- 172.21.15.98:0/265397352 shutdown_connections 2024-09-19T08:42:38.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.201+0000 7fb96344a640 1 -- 172.21.15.98:0/265397352 wait complete. 2024-09-19T08:42:38.221 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":196,"stamp":"2024-09-19T08:42:38.124220+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257156,"kb_used_data":7624,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717564,"statfs":{"total":773094113280,"available":772830785536,"internally_reserved":0,"allocated":7806976,"data_stored":3927284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.014434"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.458414+0000","last_change":"2024-09-19T08:42:30.432422+0000","last_active":"2024-09-19T08:42:31.458414+0000","last_peered":"2024-09-19T08:42:31.458414+0000","last_clean":"2024-09-19T08:42:31.458414+0000","last_became_active":"2024-09-19T08:42:30.431696+0000","last_became_peered":"2024-09-19T08:42:30.431696+0000","last_unstale":"2024-09-19T08:42:31.458414+0000","last_undegraded":"2024-09-19T08:42:31.458414+0000","last_fullsized":"2024-09-19T08:42:31.458414+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:33.925297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402870+0000","last_change":"2024-09-19T08:42:30.446888+0000","last_active":"2024-09-19T08:42:31.402870+0000","last_peered":"2024-09-19T08:42:31.402870+0000","last_clean":"2024-09-19T08:42:31.402870+0000","last_became_active":"2024-09-19T08:42:30.438820+0000","last_became_peered":"2024-09-19T08:42:30.438820+0000","last_unstale":"2024-09-19T08:42:31.402870+0000","last_undegraded":"2024-09-19T08:42:31.402870+0000","last_fullsized":"2024-09-19T08:42:31.402870+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:52:02.376361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.456309+0000","last_change":"2024-09-19T08:42:30.417593+0000","last_active":"2024-09-19T08:42:31.456309+0000","last_peered":"2024-09-19T08:42:31.456309+0000","last_clean":"2024-09-19T08:42:31.456309+0000","last_became_active":"2024-09-19T08:42:30.417241+0000","last_became_peered":"2024-09-19T08:42:30.417241+0000","last_unstale":"2024-09-19T08:42:31.456309+0000","last_undegraded":"2024-09-19T08:42:31.456309+0000","last_fullsized":"2024-09-19T08:42:31.456309+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:30:42.663654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.455352+0000","last_change":"2024-09-19T08:42:30.454395+0000","last_active":"2024-09-19T08:42:31.455352+0000","last_peered":"2024-09-19T08:42:31.455352+0000","last_clean":"2024-09-19T08:42:31.455352+0000","last_became_active":"2024-09-19T08:42:30.454076+0000","last_became_peered":"2024-09-19T08:42:30.454076+0000","last_unstale":"2024-09-19T08:42:31.455352+0000","last_undegraded":"2024-09-19T08:42:31.455352+0000","last_fullsized":"2024-09-19T08:42:31.455352+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:42:51.025229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.398464+0000","last_change":"2024-09-19T08:42:30.431886+0000","last_active":"2024-09-19T08:42:31.398464+0000","last_peered":"2024-09-19T08:42:31.398464+0000","last_clean":"2024-09-19T08:42:31.398464+0000","last_became_active":"2024-09-19T08:42:30.429749+0000","last_became_peered":"2024-09-19T08:42:30.429749+0000","last_unstale":"2024-09-19T08:42:31.398464+0000","last_undegraded":"2024-09-19T08:42:31.398464+0000","last_fullsized":"2024-09-19T08:42:31.398464+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:46:06.422459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401471+0000","last_change":"2024-09-19T08:42:30.431823+0000","last_active":"2024-09-19T08:42:31.401471+0000","last_peered":"2024-09-19T08:42:31.401471+0000","last_clean":"2024-09-19T08:42:31.401471+0000","last_became_active":"2024-09-19T08:42:30.430962+0000","last_became_peered":"2024-09-19T08:42:30.430962+0000","last_unstale":"2024-09-19T08:42:31.401471+0000","last_undegraded":"2024-09-19T08:42:31.401471+0000","last_fullsized":"2024-09-19T08:42:31.401471+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:55.760155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.425773+0000","last_change":"2024-09-19T08:42:30.449238+0000","last_active":"2024-09-19T08:42:31.425773+0000","last_peered":"2024-09-19T08:42:31.425773+0000","last_clean":"2024-09-19T08:42:31.425773+0000","last_became_active":"2024-09-19T08:42:30.448544+0000","last_became_peered":"2024-09-19T08:42:30.448544+0000","last_unstale":"2024-09-19T08:42:31.425773+0000","last_undegraded":"2024-09-19T08:42:31.425773+0000","last_fullsized":"2024-09-19T08:42:31.425773+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:19.467184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401494+0000","last_change":"2024-09-19T08:42:30.438133+0000","last_active":"2024-09-19T08:42:31.401494+0000","last_peered":"2024-09-19T08:42:31.401494+0000","last_clean":"2024-09-19T08:42:31.401494+0000","last_became_active":"2024-09-19T08:42:30.437186+0000","last_became_peered":"2024-09-19T08:42:30.437186+0000","last_unstale":"2024-09-19T08:42:31.401494+0000","last_undegraded":"2024-09-19T08:42:31.401494+0000","last_fullsized":"2024-09-19T08:42:31.401494+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:55.764601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.398650+0000","last_change":"2024-09-19T08:42:30.421194+0000","last_active":"2024-09-19T08:42:31.398650+0000","last_peered":"2024-09-19T08:42:31.398650+0000","last_clean":"2024-09-19T08:42:31.398650+0000","last_became_active":"2024-09-19T08:42:30.420916+0000","last_became_peered":"2024-09-19T08:42:30.420916+0000","last_unstale":"2024-09-19T08:42:31.398650+0000","last_undegraded":"2024-09-19T08:42:31.398650+0000","last_fullsized":"2024-09-19T08:42:31.398650+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:23.647034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.458288+0000","last_change":"2024-09-19T08:42:30.423732+0000","last_active":"2024-09-19T08:42:31.458288+0000","last_peered":"2024-09-19T08:42:31.458288+0000","last_clean":"2024-09-19T08:42:31.458288+0000","last_became_active":"2024-09-19T08:42:30.422050+0000","last_became_peered":"2024-09-19T08:42:30.422050+0000","last_unstale":"2024-09-19T08:42:31.458288+0000","last_undegraded":"2024-09-19T08:42:31.458288+0000","last_fullsized":"2024-09-19T08:42:31.458288+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:27:04.608943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.458324+0000","last_change":"2024-09-19T08:42:30.427962+0000","last_active":"2024-09-19T08:42:31.458324+0000","last_peered":"2024-09-19T08:42:31.458324+0000","last_clean":"2024-09-19T08:42:31.458324+0000","last_became_active":"2024-09-19T08:42:30.424985+0000","last_became_peered":"2024-09-19T08:42:30.424985+0000","last_unstale":"2024-09-19T08:42:31.458324+0000","last_undegraded":"2024-09-19T08:42:31.458324+0000","last_fullsized":"2024-09-19T08:42:31.458324+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:03:03.502166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.455535+0000","last_change":"2024-09-19T08:42:30.420899+0000","last_active":"2024-09-19T08:42:31.455535+0000","last_peered":"2024-09-19T08:42:31.455535+0000","last_clean":"2024-09-19T08:42:31.455535+0000","last_became_active":"2024-09-19T08:42:30.420530+0000","last_became_peered":"2024-09-19T08:42:30.420530+0000","last_unstale":"2024-09-19T08:42:31.455535+0000","last_undegraded":"2024-09-19T08:42:31.455535+0000","last_fullsized":"2024-09-19T08:42:31.455535+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:07:36.196648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.429614+0000","last_change":"2024-09-19T08:42:30.440000+0000","last_active":"2024-09-19T08:42:31.429614+0000","last_peered":"2024-09-19T08:42:31.429614+0000","last_clean":"2024-09-19T08:42:31.429614+0000","last_became_active":"2024-09-19T08:42:30.439710+0000","last_became_peered":"2024-09-19T08:42:30.439710+0000","last_unstale":"2024-09-19T08:42:31.429614+0000","last_undegraded":"2024-09-19T08:42:31.429614+0000","last_fullsized":"2024-09-19T08:42:31.429614+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:41:07.896587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.398338+0000","last_change":"2024-09-19T08:42:30.447680+0000","last_active":"2024-09-19T08:42:31.398338+0000","last_peered":"2024-09-19T08:42:31.398338+0000","last_clean":"2024-09-19T08:42:31.398338+0000","last_became_active":"2024-09-19T08:42:30.447314+0000","last_became_peered":"2024-09-19T08:42:30.447314+0000","last_unstale":"2024-09-19T08:42:31.398338+0000","last_undegraded":"2024-09-19T08:42:31.398338+0000","last_fullsized":"2024-09-19T08:42:31.398338+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:40:07.172560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.429559+0000","last_change":"2024-09-19T08:42:30.420747+0000","last_active":"2024-09-19T08:42:31.429559+0000","last_peered":"2024-09-19T08:42:31.429559+0000","last_clean":"2024-09-19T08:42:31.429559+0000","last_became_active":"2024-09-19T08:42:30.420371+0000","last_became_peered":"2024-09-19T08:42:30.420371+0000","last_unstale":"2024-09-19T08:42:31.429559+0000","last_undegraded":"2024-09-19T08:42:31.429559+0000","last_fullsized":"2024-09-19T08:42:31.429559+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:02:41.820517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.456504+0000","last_change":"2024-09-19T08:42:30.422404+0000","last_active":"2024-09-19T08:42:31.456504+0000","last_peered":"2024-09-19T08:42:31.456504+0000","last_clean":"2024-09-19T08:42:31.456504+0000","last_became_active":"2024-09-19T08:42:30.419609+0000","last_became_peered":"2024-09-19T08:42:30.419609+0000","last_unstale":"2024-09-19T08:42:31.456504+0000","last_undegraded":"2024-09-19T08:42:31.456504+0000","last_fullsized":"2024-09-19T08:42:31.456504+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:47.618846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.398779+0000","last_change":"2024-09-19T08:42:30.417780+0000","last_active":"2024-09-19T08:42:31.398779+0000","last_peered":"2024-09-19T08:42:31.398779+0000","last_clean":"2024-09-19T08:42:31.398779+0000","last_became_active":"2024-09-19T08:42:30.417309+0000","last_became_peered":"2024-09-19T08:42:30.417309+0000","last_unstale":"2024-09-19T08:42:31.398779+0000","last_undegraded":"2024-09-19T08:42:31.398779+0000","last_fullsized":"2024-09-19T08:42:31.398779+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:53:25.967482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.455739+0000","last_change":"2024-09-19T08:42:30.455358+0000","last_active":"2024-09-19T08:42:31.455739+0000","last_peered":"2024-09-19T08:42:31.455739+0000","last_clean":"2024-09-19T08:42:31.455739+0000","last_became_active":"2024-09-19T08:42:30.455033+0000","last_became_peered":"2024-09-19T08:42:30.455033+0000","last_unstale":"2024-09-19T08:42:31.455739+0000","last_undegraded":"2024-09-19T08:42:31.455739+0000","last_fullsized":"2024-09-19T08:42:31.455739+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:17:36.400315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402257+0000","last_change":"2024-09-19T08:42:30.439106+0000","last_active":"2024-09-19T08:42:31.402257+0000","last_peered":"2024-09-19T08:42:31.402257+0000","last_clean":"2024-09-19T08:42:31.402257+0000","last_became_active":"2024-09-19T08:42:30.438367+0000","last_became_peered":"2024-09-19T08:42:30.438367+0000","last_unstale":"2024-09-19T08:42:31.402257+0000","last_undegraded":"2024-09-19T08:42:31.402257+0000","last_fullsized":"2024-09-19T08:42:31.402257+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:14:25.294258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.429592+0000","last_change":"2024-09-19T08:42:30.449020+0000","last_active":"2024-09-19T08:42:31.429592+0000","last_peered":"2024-09-19T08:42:31.429592+0000","last_clean":"2024-09-19T08:42:31.429592+0000","last_became_active":"2024-09-19T08:42:30.448683+0000","last_became_peered":"2024-09-19T08:42:30.448683+0000","last_unstale":"2024-09-19T08:42:31.429592+0000","last_undegraded":"2024-09-19T08:42:31.429592+0000","last_fullsized":"2024-09-19T08:42:31.429592+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:57:27.072229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.457374+0000","last_change":"2024-09-19T08:42:30.424713+0000","last_active":"2024-09-19T08:42:31.457374+0000","last_peered":"2024-09-19T08:42:31.457374+0000","last_clean":"2024-09-19T08:42:31.457374+0000","last_became_active":"2024-09-19T08:42:30.423957+0000","last_became_peered":"2024-09-19T08:42:30.423957+0000","last_unstale":"2024-09-19T08:42:31.457374+0000","last_undegraded":"2024-09-19T08:42:31.457374+0000","last_fullsized":"2024-09-19T08:42:31.457374+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:03:35.149826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.456851+0000","last_change":"2024-09-19T08:42:30.421062+0000","last_active":"2024-09-19T08:42:31.456851+0000","last_peered":"2024-09-19T08:42:31.456851+0000","last_clean":"2024-09-19T08:42:31.456851+0000","last_became_active":"2024-09-19T08:42:30.420358+0000","last_became_peered":"2024-09-19T08:42:30.420358+0000","last_unstale":"2024-09-19T08:42:31.456851+0000","last_undegraded":"2024-09-19T08:42:31.456851+0000","last_fullsized":"2024-09-19T08:42:31.456851+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:59:11.462609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.458255+0000","last_change":"2024-09-19T08:42:30.452202+0000","last_active":"2024-09-19T08:42:31.458255+0000","last_peered":"2024-09-19T08:42:31.458255+0000","last_clean":"2024-09-19T08:42:31.458255+0000","last_became_active":"2024-09-19T08:42:30.451574+0000","last_became_peered":"2024-09-19T08:42:30.451574+0000","last_unstale":"2024-09-19T08:42:31.458255+0000","last_undegraded":"2024-09-19T08:42:31.458255+0000","last_fullsized":"2024-09-19T08:42:31.458255+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:41:07.896587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.457408+0000","last_change":"2024-09-19T08:42:30.458493+0000","last_active":"2024-09-19T08:42:31.457408+0000","last_peered":"2024-09-19T08:42:31.457408+0000","last_clean":"2024-09-19T08:42:31.457408+0000","last_became_active":"2024-09-19T08:42:30.457803+0000","last_became_peered":"2024-09-19T08:42:30.457803+0000","last_unstale":"2024-09-19T08:42:31.457408+0000","last_undegraded":"2024-09-19T08:42:31.457408+0000","last_fullsized":"2024-09-19T08:42:31.457408+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:07:43.736470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.458092+0000","last_change":"2024-09-19T08:42:30.429459+0000","last_active":"2024-09-19T08:42:31.458092+0000","last_peered":"2024-09-19T08:42:31.458092+0000","last_clean":"2024-09-19T08:42:31.458092+0000","last_became_active":"2024-09-19T08:42:30.428684+0000","last_became_peered":"2024-09-19T08:42:30.428684+0000","last_unstale":"2024-09-19T08:42:31.458092+0000","last_undegraded":"2024-09-19T08:42:31.458092+0000","last_fullsized":"2024-09-19T08:42:31.458092+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:18.296523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.426036+0000","last_change":"2024-09-19T08:42:30.456942+0000","last_active":"2024-09-19T08:42:31.426036+0000","last_peered":"2024-09-19T08:42:31.426036+0000","last_clean":"2024-09-19T08:42:31.426036+0000","last_became_active":"2024-09-19T08:42:30.456357+0000","last_became_peered":"2024-09-19T08:42:30.456357+0000","last_unstale":"2024-09-19T08:42:31.426036+0000","last_undegraded":"2024-09-19T08:42:31.426036+0000","last_fullsized":"2024-09-19T08:42:31.426036+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:18.296523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.456428+0000","last_change":"2024-09-19T08:42:30.426146+0000","last_active":"2024-09-19T08:42:31.456428+0000","last_peered":"2024-09-19T08:42:31.456428+0000","last_clean":"2024-09-19T08:42:31.456428+0000","last_became_active":"2024-09-19T08:42:30.425677+0000","last_became_peered":"2024-09-19T08:42:30.425677+0000","last_unstale":"2024-09-19T08:42:31.456428+0000","last_undegraded":"2024-09-19T08:42:31.456428+0000","last_fullsized":"2024-09-19T08:42:31.456428+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:46:40.343564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.457322+0000","last_change":"2024-09-19T08:42:30.423051+0000","last_active":"2024-09-19T08:42:31.457322+0000","last_peered":"2024-09-19T08:42:31.457322+0000","last_clean":"2024-09-19T08:42:31.457322+0000","last_became_active":"2024-09-19T08:42:30.422249+0000","last_became_peered":"2024-09-19T08:42:30.422249+0000","last_unstale":"2024-09-19T08:42:31.457322+0000","last_undegraded":"2024-09-19T08:42:31.457322+0000","last_fullsized":"2024-09-19T08:42:31.457322+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:20.537518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.455552+0000","last_change":"2024-09-19T08:42:30.449478+0000","last_active":"2024-09-19T08:42:31.455552+0000","last_peered":"2024-09-19T08:42:31.455552+0000","last_clean":"2024-09-19T08:42:31.455552+0000","last_became_active":"2024-09-19T08:42:30.449078+0000","last_became_peered":"2024-09-19T08:42:30.449078+0000","last_unstale":"2024-09-19T08:42:31.455552+0000","last_undegraded":"2024-09-19T08:42:31.455552+0000","last_fullsized":"2024-09-19T08:42:31.455552+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:27:14.185735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.426078+0000","last_change":"2024-09-19T08:42:30.455857+0000","last_active":"2024-09-19T08:42:31.426078+0000","last_peered":"2024-09-19T08:42:31.426078+0000","last_clean":"2024-09-19T08:42:31.426078+0000","last_became_active":"2024-09-19T08:42:30.455048+0000","last_became_peered":"2024-09-19T08:42:30.455048+0000","last_unstale":"2024-09-19T08:42:31.426078+0000","last_undegraded":"2024-09-19T08:42:31.426078+0000","last_fullsized":"2024-09-19T08:42:31.426078+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:27:04.608943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.426187+0000","last_change":"2024-09-19T08:42:30.449376+0000","last_active":"2024-09-19T08:42:31.426187+0000","last_peered":"2024-09-19T08:42:31.426187+0000","last_clean":"2024-09-19T08:42:31.426187+0000","last_became_active":"2024-09-19T08:42:30.448718+0000","last_became_peered":"2024-09-19T08:42:30.448718+0000","last_unstale":"2024-09-19T08:42:31.426187+0000","last_undegraded":"2024-09-19T08:42:31.426187+0000","last_fullsized":"2024-09-19T08:42:31.426187+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:10:55.061767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401507+0000","last_change":"2024-09-19T08:42:30.434665+0000","last_active":"2024-09-19T08:42:31.401507+0000","last_peered":"2024-09-19T08:42:31.401507+0000","last_clean":"2024-09-19T08:42:31.401507+0000","last_became_active":"2024-09-19T08:42:30.433735+0000","last_became_peered":"2024-09-19T08:42:30.433735+0000","last_unstale":"2024-09-19T08:42:31.401507+0000","last_undegraded":"2024-09-19T08:42:31.401507+0000","last_fullsized":"2024-09-19T08:42:31.401507+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:37:26.942476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.458805+0000","last_change":"2024-09-19T08:42:30.459349+0000","last_active":"2024-09-19T08:42:31.458805+0000","last_peered":"2024-09-19T08:42:31.458805+0000","last_clean":"2024-09-19T08:42:31.458805+0000","last_became_active":"2024-09-19T08:42:30.458221+0000","last_became_peered":"2024-09-19T08:42:30.458221+0000","last_unstale":"2024-09-19T08:42:31.458805+0000","last_undegraded":"2024-09-19T08:42:31.458805+0000","last_fullsized":"2024-09-19T08:42:31.458805+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:47:20.596787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"67'4","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.458564+0000","last_change":"2024-09-19T08:42:29.380992+0000","last_active":"2024-09-19T08:42:31.458564+0000","last_peered":"2024-09-19T08:42:31.458564+0000","last_clean":"2024-09-19T08:42:31.458564+0000","last_became_active":"2024-09-19T08:42:29.380057+0000","last_became_peered":"2024-09-19T08:42:29.380057+0000","last_unstale":"2024-09-19T08:42:31.458564+0000","last_undegraded":"2024-09-19T08:42:31.458564+0000","last_fullsized":"2024-09-19T08:42:31.458564+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:32:38.780363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.398896+0000","last_change":"2024-09-19T08:42:30.417560+0000","last_active":"2024-09-19T08:42:31.398896+0000","last_peered":"2024-09-19T08:42:31.398896+0000","last_clean":"2024-09-19T08:42:31.398896+0000","last_became_active":"2024-09-19T08:42:30.416895+0000","last_became_peered":"2024-09-19T08:42:30.416895+0000","last_unstale":"2024-09-19T08:42:31.398896+0000","last_undegraded":"2024-09-19T08:42:31.398896+0000","last_fullsized":"2024-09-19T08:42:31.398896+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:43:24.527293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.429724+0000","last_change":"2024-09-19T08:42:29.373006+0000","last_active":"2024-09-19T08:42:31.429724+0000","last_peered":"2024-09-19T08:42:31.429724+0000","last_clean":"2024-09-19T08:42:31.429724+0000","last_became_active":"2024-09-19T08:42:29.371937+0000","last_became_peered":"2024-09-19T08:42:29.371937+0000","last_unstale":"2024-09-19T08:42:31.429724+0000","last_undegraded":"2024-09-19T08:42:31.429724+0000","last_fullsized":"2024-09-19T08:42:31.429724+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:19.528253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402198+0000","last_change":"2024-09-19T08:42:30.458129+0000","last_active":"2024-09-19T08:42:31.402198+0000","last_peered":"2024-09-19T08:42:31.402198+0000","last_clean":"2024-09-19T08:42:31.402198+0000","last_became_active":"2024-09-19T08:42:30.456984+0000","last_became_peered":"2024-09-19T08:42:30.456984+0000","last_unstale":"2024-09-19T08:42:31.402198+0000","last_undegraded":"2024-09-19T08:42:31.402198+0000","last_fullsized":"2024-09-19T08:42:31.402198+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:43:27.676061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.457712+0000","last_change":"2024-09-19T08:42:29.399152+0000","last_active":"2024-09-19T08:42:31.457712+0000","last_peered":"2024-09-19T08:42:31.457712+0000","last_clean":"2024-09-19T08:42:31.457712+0000","last_became_active":"2024-09-19T08:42:29.398078+0000","last_became_peered":"2024-09-19T08:42:29.398078+0000","last_unstale":"2024-09-19T08:42:31.457712+0000","last_undegraded":"2024-09-19T08:42:31.457712+0000","last_fullsized":"2024-09-19T08:42:31.457712+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:03:18.808677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.458619+0000","last_change":"2024-09-19T08:42:30.428464+0000","last_active":"2024-09-19T08:42:31.458619+0000","last_peered":"2024-09-19T08:42:31.458619+0000","last_clean":"2024-09-19T08:42:31.458619+0000","last_became_active":"2024-09-19T08:42:30.425762+0000","last_became_peered":"2024-09-19T08:42:30.425762+0000","last_unstale":"2024-09-19T08:42:31.458619+0000","last_undegraded":"2024-09-19T08:42:31.458619+0000","last_fullsized":"2024-09-19T08:42:31.458619+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:20:16.897649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402459+0000","last_change":"2024-09-19T08:42:29.395380+0000","last_active":"2024-09-19T08:42:31.402459+0000","last_peered":"2024-09-19T08:42:31.402459+0000","last_clean":"2024-09-19T08:42:31.402459+0000","last_became_active":"2024-09-19T08:42:29.394641+0000","last_became_peered":"2024-09-19T08:42:29.394641+0000","last_unstale":"2024-09-19T08:42:31.402459+0000","last_undegraded":"2024-09-19T08:42:31.402459+0000","last_fullsized":"2024-09-19T08:42:31.402459+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:26.586550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.426219+0000","last_change":"2024-09-19T08:42:30.443041+0000","last_active":"2024-09-19T08:42:31.426219+0000","last_peered":"2024-09-19T08:42:31.426219+0000","last_clean":"2024-09-19T08:42:31.426219+0000","last_became_active":"2024-09-19T08:42:30.426497+0000","last_became_peered":"2024-09-19T08:42:30.426497+0000","last_unstale":"2024-09-19T08:42:31.426219+0000","last_undegraded":"2024-09-19T08:42:31.426219+0000","last_fullsized":"2024-09-19T08:42:31.426219+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:48.938145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.455772+0000","last_change":"2024-09-19T08:42:29.385378+0000","last_active":"2024-09-19T08:42:31.455772+0000","last_peered":"2024-09-19T08:42:31.455772+0000","last_clean":"2024-09-19T08:42:31.455772+0000","last_became_active":"2024-09-19T08:42:29.385116+0000","last_became_peered":"2024-09-19T08:42:29.385116+0000","last_unstale":"2024-09-19T08:42:31.455772+0000","last_undegraded":"2024-09-19T08:42:31.455772+0000","last_fullsized":"2024-09-19T08:42:31.455772+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:53.241533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.457409+0000","last_change":"2024-09-19T08:42:30.431019+0000","last_active":"2024-09-19T08:42:31.457409+0000","last_peered":"2024-09-19T08:42:31.457409+0000","last_clean":"2024-09-19T08:42:31.457409+0000","last_became_active":"2024-09-19T08:42:30.430173+0000","last_became_peered":"2024-09-19T08:42:30.430173+0000","last_unstale":"2024-09-19T08:42:31.457409+0000","last_undegraded":"2024-09-19T08:42:31.457409+0000","last_fullsized":"2024-09-19T08:42:31.457409+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:19.467184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401528+0000","last_change":"2024-09-19T08:42:29.399398+0000","last_active":"2024-09-19T08:42:31.401528+0000","last_peered":"2024-09-19T08:42:31.401528+0000","last_clean":"2024-09-19T08:42:31.401528+0000","last_became_active":"2024-09-19T08:42:29.396716+0000","last_became_peered":"2024-09-19T08:42:29.396716+0000","last_unstale":"2024-09-19T08:42:31.401528+0000","last_undegraded":"2024-09-19T08:42:31.401528+0000","last_fullsized":"2024-09-19T08:42:31.401528+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:50:23.952850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.398010+0000","last_change":"2024-09-19T08:42:30.411536+0000","last_active":"2024-09-19T08:42:31.398010+0000","last_peered":"2024-09-19T08:42:31.398010+0000","last_clean":"2024-09-19T08:42:31.398010+0000","last_became_active":"2024-09-19T08:42:30.410812+0000","last_became_peered":"2024-09-19T08:42:30.410812+0000","last_unstale":"2024-09-19T08:42:31.398010+0000","last_undegraded":"2024-09-19T08:42:31.398010+0000","last_fullsized":"2024-09-19T08:42:31.398010+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:42:09.162003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.455444+0000","last_change":"2024-09-19T08:42:29.387234+0000","last_active":"2024-09-19T08:42:31.455444+0000","last_peered":"2024-09-19T08:42:31.455444+0000","last_clean":"2024-09-19T08:42:31.455444+0000","last_became_active":"2024-09-19T08:42:29.386856+0000","last_became_peered":"2024-09-19T08:42:29.386856+0000","last_unstale":"2024-09-19T08:42:31.455444+0000","last_undegraded":"2024-09-19T08:42:31.455444+0000","last_fullsized":"2024-09-19T08:42:31.455444+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:26.093340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.429774+0000","last_change":"2024-09-19T08:42:30.426900+0000","last_active":"2024-09-19T08:42:31.429774+0000","last_peered":"2024-09-19T08:42:31.429774+0000","last_clean":"2024-09-19T08:42:31.429774+0000","last_became_active":"2024-09-19T08:42:30.426646+0000","last_became_peered":"2024-09-19T08:42:30.426646+0000","last_unstale":"2024-09-19T08:42:31.429774+0000","last_undegraded":"2024-09-19T08:42:31.429774+0000","last_fullsized":"2024-09-19T08:42:31.429774+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:33.925297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.398066+0000","last_change":"2024-09-19T08:42:29.368732+0000","last_active":"2024-09-19T08:42:31.398066+0000","last_peered":"2024-09-19T08:42:31.398066+0000","last_clean":"2024-09-19T08:42:31.398066+0000","last_became_active":"2024-09-19T08:42:29.368426+0000","last_became_peered":"2024-09-19T08:42:29.368426+0000","last_unstale":"2024-09-19T08:42:31.398066+0000","last_undegraded":"2024-09-19T08:42:31.398066+0000","last_fullsized":"2024-09-19T08:42:31.398066+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:33:10.262511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.426339+0000","last_change":"2024-09-19T08:42:30.449116+0000","last_active":"2024-09-19T08:42:31.426339+0000","last_peered":"2024-09-19T08:42:31.426339+0000","last_clean":"2024-09-19T08:42:31.426339+0000","last_became_active":"2024-09-19T08:42:30.448401+0000","last_became_peered":"2024-09-19T08:42:30.448401+0000","last_unstale":"2024-09-19T08:42:31.426339+0000","last_undegraded":"2024-09-19T08:42:31.426339+0000","last_fullsized":"2024-09-19T08:42:31.426339+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:02:41.820517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.458224+0000","last_change":"2024-09-19T08:42:29.388802+0000","last_active":"2024-09-19T08:42:31.458224+0000","last_peered":"2024-09-19T08:42:31.458224+0000","last_clean":"2024-09-19T08:42:31.458224+0000","last_became_active":"2024-09-19T08:42:29.388077+0000","last_became_peered":"2024-09-19T08:42:29.388077+0000","last_unstale":"2024-09-19T08:42:31.458224+0000","last_undegraded":"2024-09-19T08:42:31.458224+0000","last_fullsized":"2024-09-19T08:42:31.458224+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:24:19.025320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.458479+0000","last_change":"2024-09-19T08:42:30.458537+0000","last_active":"2024-09-19T08:42:31.458479+0000","last_peered":"2024-09-19T08:42:31.458479+0000","last_clean":"2024-09-19T08:42:31.458479+0000","last_became_active":"2024-09-19T08:42:30.457874+0000","last_became_peered":"2024-09-19T08:42:30.457874+0000","last_unstale":"2024-09-19T08:42:31.458479+0000","last_undegraded":"2024-09-19T08:42:31.458479+0000","last_fullsized":"2024-09-19T08:42:31.458479+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:16:14.953301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402803+0000","last_change":"2024-09-19T08:42:29.399190+0000","last_active":"2024-09-19T08:42:31.402803+0000","last_peered":"2024-09-19T08:42:31.402803+0000","last_clean":"2024-09-19T08:42:31.402803+0000","last_became_active":"2024-09-19T08:42:29.396183+0000","last_became_peered":"2024-09-19T08:42:29.396183+0000","last_unstale":"2024-09-19T08:42:31.402803+0000","last_undegraded":"2024-09-19T08:42:31.402803+0000","last_fullsized":"2024-09-19T08:42:31.402803+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:21:01.013528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402155+0000","last_change":"2024-09-19T08:42:30.451725+0000","last_active":"2024-09-19T08:42:31.402155+0000","last_peered":"2024-09-19T08:42:31.402155+0000","last_clean":"2024-09-19T08:42:31.402155+0000","last_became_active":"2024-09-19T08:42:30.437086+0000","last_became_peered":"2024-09-19T08:42:30.437086+0000","last_unstale":"2024-09-19T08:42:31.402155+0000","last_undegraded":"2024-09-19T08:42:31.402155+0000","last_fullsized":"2024-09-19T08:42:31.402155+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:34.398500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"67'3","reported_seq":17,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.456414+0000","last_change":"2024-09-19T08:42:29.375916+0000","last_active":"2024-09-19T08:42:31.456414+0000","last_peered":"2024-09-19T08:42:31.456414+0000","last_clean":"2024-09-19T08:42:31.456414+0000","last_became_active":"2024-09-19T08:42:29.375398+0000","last_became_peered":"2024-09-19T08:42:29.375398+0000","last_unstale":"2024-09-19T08:42:31.456414+0000","last_undegraded":"2024-09-19T08:42:31.456414+0000","last_fullsized":"2024-09-19T08:42:31.456414+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:40:41.566855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.398886+0000","last_change":"2024-09-19T08:42:30.416450+0000","last_active":"2024-09-19T08:42:31.398886+0000","last_peered":"2024-09-19T08:42:31.398886+0000","last_clean":"2024-09-19T08:42:31.398886+0000","last_became_active":"2024-09-19T08:42:30.415819+0000","last_became_peered":"2024-09-19T08:42:30.415819+0000","last_unstale":"2024-09-19T08:42:31.398886+0000","last_undegraded":"2024-09-19T08:42:31.398886+0000","last_fullsized":"2024-09-19T08:42:31.398886+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:54.974767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"67'6","reported_seq":20,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.456098+0000","last_change":"2024-09-19T08:42:29.368802+0000","last_active":"2024-09-19T08:42:31.456098+0000","last_peered":"2024-09-19T08:42:31.456098+0000","last_clean":"2024-09-19T08:42:31.456098+0000","last_became_active":"2024-09-19T08:42:29.368414+0000","last_became_peered":"2024-09-19T08:42:29.368414+0000","last_unstale":"2024-09-19T08:42:31.456098+0000","last_undegraded":"2024-09-19T08:42:31.456098+0000","last_fullsized":"2024-09-19T08:42:31.456098+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:18.889196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.457614+0000","last_change":"2024-09-19T08:42:30.428859+0000","last_active":"2024-09-19T08:42:31.457614+0000","last_peered":"2024-09-19T08:42:31.457614+0000","last_clean":"2024-09-19T08:42:31.457614+0000","last_became_active":"2024-09-19T08:42:30.426388+0000","last_became_peered":"2024-09-19T08:42:30.426388+0000","last_unstale":"2024-09-19T08:42:31.457614+0000","last_undegraded":"2024-09-19T08:42:31.457614+0000","last_fullsized":"2024-09-19T08:42:31.457614+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:02:41.820517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.455549+0000","last_change":"2024-09-19T08:42:29.369719+0000","last_active":"2024-09-19T08:42:31.455549+0000","last_peered":"2024-09-19T08:42:31.455549+0000","last_clean":"2024-09-19T08:42:31.455549+0000","last_became_active":"2024-09-19T08:42:29.369306+0000","last_became_peered":"2024-09-19T08:42:29.369306+0000","last_unstale":"2024-09-19T08:42:31.455549+0000","last_undegraded":"2024-09-19T08:42:31.455549+0000","last_fullsized":"2024-09-19T08:42:31.455549+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:57.793452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.426092+0000","last_change":"2024-09-19T08:42:30.443062+0000","last_active":"2024-09-19T08:42:31.426092+0000","last_peered":"2024-09-19T08:42:31.426092+0000","last_clean":"2024-09-19T08:42:31.426092+0000","last_became_active":"2024-09-19T08:42:30.442440+0000","last_became_peered":"2024-09-19T08:42:30.442440+0000","last_unstale":"2024-09-19T08:42:31.426092+0000","last_undegraded":"2024-09-19T08:42:31.426092+0000","last_fullsized":"2024-09-19T08:42:31.426092+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:52:10.022065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401738+0000","last_change":"2024-09-19T08:42:29.399130+0000","last_active":"2024-09-19T08:42:31.401738+0000","last_peered":"2024-09-19T08:42:31.401738+0000","last_clean":"2024-09-19T08:42:31.401738+0000","last_became_active":"2024-09-19T08:42:29.396039+0000","last_became_peered":"2024-09-19T08:42:29.396039+0000","last_unstale":"2024-09-19T08:42:31.401738+0000","last_undegraded":"2024-09-19T08:42:31.401738+0000","last_fullsized":"2024-09-19T08:42:31.401738+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:54.277351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.429445+0000","last_change":"2024-09-19T08:42:30.428966+0000","last_active":"2024-09-19T08:42:31.429445+0000","last_peered":"2024-09-19T08:42:31.429445+0000","last_clean":"2024-09-19T08:42:31.429445+0000","last_became_active":"2024-09-19T08:42:30.428673+0000","last_became_peered":"2024-09-19T08:42:30.428673+0000","last_unstale":"2024-09-19T08:42:31.429445+0000","last_undegraded":"2024-09-19T08:42:31.429445+0000","last_fullsized":"2024-09-19T08:42:31.429445+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:07:43.736470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.425646+0000","last_change":"2024-09-19T08:42:29.377410+0000","last_active":"2024-09-19T08:42:31.425646+0000","last_peered":"2024-09-19T08:42:31.425646+0000","last_clean":"2024-09-19T08:42:31.425646+0000","last_became_active":"2024-09-19T08:42:29.376673+0000","last_became_peered":"2024-09-19T08:42:29.376673+0000","last_unstale":"2024-09-19T08:42:31.425646+0000","last_undegraded":"2024-09-19T08:42:31.425646+0000","last_fullsized":"2024-09-19T08:42:31.425646+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:09.382166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.398460+0000","last_change":"2024-09-19T08:42:30.420451+0000","last_active":"2024-09-19T08:42:31.398460+0000","last_peered":"2024-09-19T08:42:31.398460+0000","last_clean":"2024-09-19T08:42:31.398460+0000","last_became_active":"2024-09-19T08:42:30.420221+0000","last_became_peered":"2024-09-19T08:42:30.420221+0000","last_unstale":"2024-09-19T08:42:31.398460+0000","last_undegraded":"2024-09-19T08:42:31.398460+0000","last_fullsized":"2024-09-19T08:42:31.398460+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:33:27.096060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"67'8","reported_seq":22,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.456394+0000","last_change":"2024-09-19T08:42:29.385268+0000","last_active":"2024-09-19T08:42:31.456394+0000","last_peered":"2024-09-19T08:42:31.456394+0000","last_clean":"2024-09-19T08:42:31.456394+0000","last_became_active":"2024-09-19T08:42:29.384547+0000","last_became_peered":"2024-09-19T08:42:29.384547+0000","last_unstale":"2024-09-19T08:42:31.456394+0000","last_undegraded":"2024-09-19T08:42:31.456394+0000","last_fullsized":"2024-09-19T08:42:31.456394+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:15.005825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.458668+0000","last_change":"2024-09-19T08:42:30.424337+0000","last_active":"2024-09-19T08:42:31.458668+0000","last_peered":"2024-09-19T08:42:31.458668+0000","last_clean":"2024-09-19T08:42:31.458668+0000","last_became_active":"2024-09-19T08:42:30.423013+0000","last_became_peered":"2024-09-19T08:42:30.423013+0000","last_unstale":"2024-09-19T08:42:31.458668+0000","last_undegraded":"2024-09-19T08:42:31.458668+0000","last_fullsized":"2024-09-19T08:42:31.458668+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:52:10.022065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402598+0000","last_change":"2024-09-19T08:42:29.400466+0000","last_active":"2024-09-19T08:42:31.402598+0000","last_peered":"2024-09-19T08:42:31.402598+0000","last_clean":"2024-09-19T08:42:31.402598+0000","last_became_active":"2024-09-19T08:42:29.395739+0000","last_became_peered":"2024-09-19T08:42:29.395739+0000","last_unstale":"2024-09-19T08:42:31.402598+0000","last_undegraded":"2024-09-19T08:42:31.402598+0000","last_fullsized":"2024-09-19T08:42:31.402598+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:41:59.302829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.458847+0000","last_change":"2024-09-19T08:42:30.425830+0000","last_active":"2024-09-19T08:42:31.458847+0000","last_peered":"2024-09-19T08:42:31.458847+0000","last_clean":"2024-09-19T08:42:31.458847+0000","last_became_active":"2024-09-19T08:42:30.425130+0000","last_became_peered":"2024-09-19T08:42:30.425130+0000","last_unstale":"2024-09-19T08:42:31.458847+0000","last_undegraded":"2024-09-19T08:42:31.458847+0000","last_fullsized":"2024-09-19T08:42:31.458847+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:59.839015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.458802+0000","last_change":"2024-09-19T08:42:29.398636+0000","last_active":"2024-09-19T08:42:31.458802+0000","last_peered":"2024-09-19T08:42:31.458802+0000","last_clean":"2024-09-19T08:42:31.458802+0000","last_became_active":"2024-09-19T08:42:29.398016+0000","last_became_peered":"2024-09-19T08:42:29.398016+0000","last_unstale":"2024-09-19T08:42:31.458802+0000","last_undegraded":"2024-09-19T08:42:31.458802+0000","last_fullsized":"2024-09-19T08:42:31.458802+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:12.758285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.456032+0000","last_change":"2024-09-19T08:42:30.420773+0000","last_active":"2024-09-19T08:42:31.456032+0000","last_peered":"2024-09-19T08:42:31.456032+0000","last_clean":"2024-09-19T08:42:31.456032+0000","last_became_active":"2024-09-19T08:42:30.420139+0000","last_became_peered":"2024-09-19T08:42:30.420139+0000","last_unstale":"2024-09-19T08:42:31.456032+0000","last_undegraded":"2024-09-19T08:42:31.456032+0000","last_fullsized":"2024-09-19T08:42:31.456032+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:59:42.739064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402346+0000","last_change":"2024-09-19T08:42:29.399885+0000","last_active":"2024-09-19T08:42:31.402346+0000","last_peered":"2024-09-19T08:42:31.402346+0000","last_clean":"2024-09-19T08:42:31.402346+0000","last_became_active":"2024-09-19T08:42:29.396798+0000","last_became_peered":"2024-09-19T08:42:29.396798+0000","last_unstale":"2024-09-19T08:42:31.402346+0000","last_undegraded":"2024-09-19T08:42:31.402346+0000","last_fullsized":"2024-09-19T08:42:31.402346+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:07.393673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.399251+0000","last_change":"2024-09-19T08:42:30.411732+0000","last_active":"2024-09-19T08:42:31.399251+0000","last_peered":"2024-09-19T08:42:31.399251+0000","last_clean":"2024-09-19T08:42:31.399251+0000","last_became_active":"2024-09-19T08:42:30.411033+0000","last_became_peered":"2024-09-19T08:42:30.411033+0000","last_unstale":"2024-09-19T08:42:31.399251+0000","last_undegraded":"2024-09-19T08:42:31.399251+0000","last_fullsized":"2024-09-19T08:42:31.399251+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:03:24.038264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.458041+0000","last_change":"2024-09-19T08:42:29.389208+0000","last_active":"2024-09-19T08:42:31.458041+0000","last_peered":"2024-09-19T08:42:31.458041+0000","last_clean":"2024-09-19T08:42:31.458041+0000","last_became_active":"2024-09-19T08:42:29.388479+0000","last_became_peered":"2024-09-19T08:42:29.388479+0000","last_unstale":"2024-09-19T08:42:31.458041+0000","last_undegraded":"2024-09-19T08:42:31.458041+0000","last_fullsized":"2024-09-19T08:42:31.458041+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:08:07.217342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.455688+0000","last_change":"2024-09-19T08:42:30.418094+0000","last_active":"2024-09-19T08:42:31.455688+0000","last_peered":"2024-09-19T08:42:31.455688+0000","last_clean":"2024-09-19T08:42:31.455688+0000","last_became_active":"2024-09-19T08:42:30.415422+0000","last_became_peered":"2024-09-19T08:42:30.415422+0000","last_unstale":"2024-09-19T08:42:31.455688+0000","last_undegraded":"2024-09-19T08:42:31.455688+0000","last_fullsized":"2024-09-19T08:42:31.455688+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:31:25.534663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401789+0000","last_change":"2024-09-19T08:42:29.395444+0000","last_active":"2024-09-19T08:42:31.401789+0000","last_peered":"2024-09-19T08:42:31.401789+0000","last_clean":"2024-09-19T08:42:31.401789+0000","last_became_active":"2024-09-19T08:42:29.394674+0000","last_became_peered":"2024-09-19T08:42:29.394674+0000","last_unstale":"2024-09-19T08:42:31.401789+0000","last_undegraded":"2024-09-19T08:42:31.401789+0000","last_fullsized":"2024-09-19T08:42:31.401789+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:42:57.681916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.455278+0000","last_change":"2024-09-19T08:42:30.438215+0000","last_active":"2024-09-19T08:42:31.455278+0000","last_peered":"2024-09-19T08:42:31.455278+0000","last_clean":"2024-09-19T08:42:31.455278+0000","last_became_active":"2024-09-19T08:42:30.437782+0000","last_became_peered":"2024-09-19T08:42:30.437782+0000","last_unstale":"2024-09-19T08:42:31.455278+0000","last_undegraded":"2024-09-19T08:42:31.455278+0000","last_fullsized":"2024-09-19T08:42:31.455278+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:08.138224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.426283+0000","last_change":"2024-09-19T08:42:29.375051+0000","last_active":"2024-09-19T08:42:31.426283+0000","last_peered":"2024-09-19T08:42:31.426283+0000","last_clean":"2024-09-19T08:42:31.426283+0000","last_became_active":"2024-09-19T08:42:29.374145+0000","last_became_peered":"2024-09-19T08:42:29.374145+0000","last_unstale":"2024-09-19T08:42:31.426283+0000","last_undegraded":"2024-09-19T08:42:31.426283+0000","last_fullsized":"2024-09-19T08:42:31.426283+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:44.656487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.455364+0000","last_change":"2024-09-19T08:42:30.439052+0000","last_active":"2024-09-19T08:42:31.455364+0000","last_peered":"2024-09-19T08:42:31.455364+0000","last_clean":"2024-09-19T08:42:31.455364+0000","last_became_active":"2024-09-19T08:42:30.438753+0000","last_became_peered":"2024-09-19T08:42:30.438753+0000","last_unstale":"2024-09-19T08:42:31.455364+0000","last_undegraded":"2024-09-19T08:42:31.455364+0000","last_fullsized":"2024-09-19T08:42:31.455364+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:01:53.973114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.429311+0000","last_change":"2024-09-19T08:42:29.372223+0000","last_active":"2024-09-19T08:42:31.429311+0000","last_peered":"2024-09-19T08:42:31.429311+0000","last_clean":"2024-09-19T08:42:31.429311+0000","last_became_active":"2024-09-19T08:42:29.371947+0000","last_became_peered":"2024-09-19T08:42:29.371947+0000","last_unstale":"2024-09-19T08:42:31.429311+0000","last_undegraded":"2024-09-19T08:42:31.429311+0000","last_fullsized":"2024-09-19T08:42:31.429311+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:59:10.453344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.429694+0000","last_change":"2024-09-19T08:42:30.419025+0000","last_active":"2024-09-19T08:42:31.429694+0000","last_peered":"2024-09-19T08:42:31.429694+0000","last_clean":"2024-09-19T08:42:31.429694+0000","last_became_active":"2024-09-19T08:42:30.418674+0000","last_became_peered":"2024-09-19T08:42:30.418674+0000","last_unstale":"2024-09-19T08:42:31.429694+0000","last_undegraded":"2024-09-19T08:42:31.429694+0000","last_fullsized":"2024-09-19T08:42:31.429694+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:03:03.502166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.402779+0000","last_change":"2024-09-19T08:42:29.396548+0000","last_active":"2024-09-19T08:42:31.402779+0000","last_peered":"2024-09-19T08:42:31.402779+0000","last_clean":"2024-09-19T08:42:31.402779+0000","last_became_active":"2024-09-19T08:42:29.394896+0000","last_became_peered":"2024-09-19T08:42:29.394896+0000","last_unstale":"2024-09-19T08:42:31.402779+0000","last_undegraded":"2024-09-19T08:42:31.402779+0000","last_fullsized":"2024-09-19T08:42:31.402779+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:27:20.145977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.426300+0000","last_change":"2024-09-19T08:42:30.447256+0000","last_active":"2024-09-19T08:42:31.426300+0000","last_peered":"2024-09-19T08:42:31.426300+0000","last_clean":"2024-09-19T08:42:31.426300+0000","last_became_active":"2024-09-19T08:42:30.446395+0000","last_became_peered":"2024-09-19T08:42:30.446395+0000","last_unstale":"2024-09-19T08:42:31.426300+0000","last_undegraded":"2024-09-19T08:42:31.426300+0000","last_fullsized":"2024-09-19T08:42:31.426300+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:07:43.736470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"68'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.426480+0000","last_change":"2024-09-19T08:42:29.385128+0000","last_active":"2024-09-19T08:42:31.426480+0000","last_peered":"2024-09-19T08:42:31.426480+0000","last_clean":"2024-09-19T08:42:31.426480+0000","last_became_active":"2024-09-19T08:42:29.384397+0000","last_became_peered":"2024-09-19T08:42:29.384397+0000","last_unstale":"2024-09-19T08:42:31.426480+0000","last_undegraded":"2024-09-19T08:42:31.426480+0000","last_fullsized":"2024-09-19T08:42:31.426480+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:14:13.112131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.458245+0000","last_change":"2024-09-19T08:42:30.459124+0000","last_active":"2024-09-19T08:42:31.458245+0000","last_peered":"2024-09-19T08:42:31.458245+0000","last_clean":"2024-09-19T08:42:31.458245+0000","last_became_active":"2024-09-19T08:42:30.458447+0000","last_became_peered":"2024-09-19T08:42:30.458447+0000","last_unstale":"2024-09-19T08:42:31.458245+0000","last_undegraded":"2024-09-19T08:42:31.458245+0000","last_fullsized":"2024-09-19T08:42:31.458245+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:57:27.072229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.399366+0000","last_change":"2024-09-19T08:42:29.375037+0000","last_active":"2024-09-19T08:42:31.399366+0000","last_peered":"2024-09-19T08:42:31.399366+0000","last_clean":"2024-09-19T08:42:31.399366+0000","last_became_active":"2024-09-19T08:42:29.374723+0000","last_became_peered":"2024-09-19T08:42:29.374723+0000","last_unstale":"2024-09-19T08:42:31.399366+0000","last_undegraded":"2024-09-19T08:42:31.399366+0000","last_fullsized":"2024-09-19T08:42:31.399366+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:07:10.884601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.458931+0000","last_change":"2024-09-19T08:42:30.449648+0000","last_active":"2024-09-19T08:42:31.458931+0000","last_peered":"2024-09-19T08:42:31.458931+0000","last_clean":"2024-09-19T08:42:31.458931+0000","last_became_active":"2024-09-19T08:42:30.449054+0000","last_became_peered":"2024-09-19T08:42:30.449054+0000","last_unstale":"2024-09-19T08:42:31.458931+0000","last_undegraded":"2024-09-19T08:42:31.458931+0000","last_fullsized":"2024-09-19T08:42:31.458931+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:41:07.372011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.429507+0000","last_change":"2024-09-19T08:42:29.384202+0000","last_active":"2024-09-19T08:42:31.429507+0000","last_peered":"2024-09-19T08:42:31.429507+0000","last_clean":"2024-09-19T08:42:31.429507+0000","last_became_active":"2024-09-19T08:42:29.383866+0000","last_became_peered":"2024-09-19T08:42:29.383866+0000","last_unstale":"2024-09-19T08:42:31.429507+0000","last_undegraded":"2024-09-19T08:42:31.429507+0000","last_fullsized":"2024-09-19T08:42:31.429507+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:03:34.140561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.398524+0000","last_change":"2024-09-19T08:42:30.416404+0000","last_active":"2024-09-19T08:42:31.398524+0000","last_peered":"2024-09-19T08:42:31.398524+0000","last_clean":"2024-09-19T08:42:31.398524+0000","last_became_active":"2024-09-19T08:42:30.415848+0000","last_became_peered":"2024-09-19T08:42:30.415848+0000","last_unstale":"2024-09-19T08:42:31.398524+0000","last_undegraded":"2024-09-19T08:42:31.398524+0000","last_fullsized":"2024-09-19T08:42:31.398524+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:51:35.148871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.398338+0000","last_change":"2024-09-19T08:42:29.381869+0000","last_active":"2024-09-19T08:42:31.398338+0000","last_peered":"2024-09-19T08:42:31.398338+0000","last_clean":"2024-09-19T08:42:31.398338+0000","last_became_active":"2024-09-19T08:42:29.381590+0000","last_became_peered":"2024-09-19T08:42:29.381590+0000","last_unstale":"2024-09-19T08:42:31.398338+0000","last_undegraded":"2024-09-19T08:42:31.398338+0000","last_fullsized":"2024-09-19T08:42:31.398338+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:22:08.333844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401692+0000","last_change":"2024-09-19T08:42:30.459642+0000","last_active":"2024-09-19T08:42:31.401692+0000","last_peered":"2024-09-19T08:42:31.401692+0000","last_clean":"2024-09-19T08:42:31.401692+0000","last_became_active":"2024-09-19T08:42:30.459010+0000","last_became_peered":"2024-09-19T08:42:30.459010+0000","last_unstale":"2024-09-19T08:42:31.401692+0000","last_undegraded":"2024-09-19T08:42:31.401692+0000","last_fullsized":"2024-09-19T08:42:31.401692+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:03:33.044498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.455327+0000","last_change":"2024-09-19T08:42:29.387334+0000","last_active":"2024-09-19T08:42:31.455327+0000","last_peered":"2024-09-19T08:42:31.455327+0000","last_clean":"2024-09-19T08:42:31.455327+0000","last_became_active":"2024-09-19T08:42:29.386999+0000","last_became_peered":"2024-09-19T08:42:29.386999+0000","last_unstale":"2024-09-19T08:42:31.455327+0000","last_undegraded":"2024-09-19T08:42:31.455327+0000","last_fullsized":"2024-09-19T08:42:31.455327+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:26:36.338910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.455271+0000","last_change":"2024-09-19T08:42:30.411643+0000","last_active":"2024-09-19T08:42:31.455271+0000","last_peered":"2024-09-19T08:42:31.455271+0000","last_clean":"2024-09-19T08:42:31.455271+0000","last_became_active":"2024-09-19T08:42:30.410592+0000","last_became_peered":"2024-09-19T08:42:30.410592+0000","last_unstale":"2024-09-19T08:42:31.455271+0000","last_undegraded":"2024-09-19T08:42:31.455271+0000","last_fullsized":"2024-09-19T08:42:31.455271+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:15:53.503677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.456427+0000","last_change":"2024-09-19T08:42:29.382377+0000","last_active":"2024-09-19T08:42:31.456427+0000","last_peered":"2024-09-19T08:42:31.456427+0000","last_clean":"2024-09-19T08:42:31.456427+0000","last_became_active":"2024-09-19T08:42:29.381685+0000","last_became_peered":"2024-09-19T08:42:29.381685+0000","last_unstale":"2024-09-19T08:42:31.456427+0000","last_undegraded":"2024-09-19T08:42:31.456427+0000","last_fullsized":"2024-09-19T08:42:31.456427+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:15:13.220029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401444+0000","last_change":"2024-09-19T08:42:30.431276+0000","last_active":"2024-09-19T08:42:31.401444+0000","last_peered":"2024-09-19T08:42:31.401444+0000","last_clean":"2024-09-19T08:42:31.401444+0000","last_became_active":"2024-09-19T08:42:30.430462+0000","last_became_peered":"2024-09-19T08:42:30.430462+0000","last_unstale":"2024-09-19T08:42:31.401444+0000","last_undegraded":"2024-09-19T08:42:31.401444+0000","last_fullsized":"2024-09-19T08:42:31.401444+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:50.219137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401735+0000","last_change":"2024-09-19T08:42:29.373808+0000","last_active":"2024-09-19T08:42:31.401735+0000","last_peered":"2024-09-19T08:42:31.401735+0000","last_clean":"2024-09-19T08:42:31.401735+0000","last_became_active":"2024-09-19T08:42:29.373099+0000","last_became_peered":"2024-09-19T08:42:29.373099+0000","last_unstale":"2024-09-19T08:42:31.401735+0000","last_undegraded":"2024-09-19T08:42:31.401735+0000","last_fullsized":"2024-09-19T08:42:31.401735+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:48.444653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.455409+0000","last_change":"2024-09-19T08:42:30.458262+0000","last_active":"2024-09-19T08:42:31.455409+0000","last_peered":"2024-09-19T08:42:31.455409+0000","last_clean":"2024-09-19T08:42:31.455409+0000","last_became_active":"2024-09-19T08:42:30.457905+0000","last_became_peered":"2024-09-19T08:42:30.457905+0000","last_unstale":"2024-09-19T08:42:31.455409+0000","last_undegraded":"2024-09-19T08:42:31.455409+0000","last_fullsized":"2024-09-19T08:42:31.455409+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:29.343580+0000","last_clean_scrub_stamp":"2024-09-19T08:42:29.343580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:16.269375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.401741+0000","last_change":"2024-09-19T08:42:29.399988+0000","last_active":"2024-09-19T08:42:31.401741+0000","last_peered":"2024-09-19T08:42:31.401741+0000","last_clean":"2024-09-19T08:42:31.401741+0000","last_became_active":"2024-09-19T08:42:29.399260+0000","last_became_peered":"2024-09-19T08:42:29.399260+0000","last_unstale":"2024-09-19T08:42:31.401741+0000","last_undegraded":"2024-09-19T08:42:31.401741+0000","last_fullsized":"2024-09-19T08:42:31.401741+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:42:28.334315+0000","last_clean_scrub_stamp":"2024-09-19T08:42:28.334315+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:06.453660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":254,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-19T08:42:31.457218+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:42:31.457218+0000","last_peered":"2024-09-19T08:42:31.457218+0000","last_clean":"2024-09-19T08:42:31.457218+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:42:31.457218+0000","last_undegraded":"2024-09-19T08:42:31.457218+0000","last_fullsized":"2024-09-19T08:42:31.457218+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509496,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1421312,"data_stored":933700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509498,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":823296,"data_stored":342806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509498,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31992,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339848,"statfs":{"total":96636764160,"available":96604004352,"internally_reserved":0,"allocated":823296,"data_stored":342786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509495,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":831488,"data_stored":344328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509497,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":839680,"data_stored":344944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509495,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1413120,"data_stored":933084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574905,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":827392,"data_stored":342808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542201,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":827392,"data_stored":342828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:42:38.224 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T08:42:38.225 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-19T08:42:38.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.389+0000 7fbd53522640 1 Processor -- start 2024-09-19T08:42:38.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.389+0000 7fbd53522640 1 -- start start 2024-09-19T08:42:38.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.389+0000 7fbd53522640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd4c06f6c0 0x7fbd4c06fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:38.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.389+0000 7fbd53522640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd4c148350 con 0x7fbd4c06cca0 2024-09-19T08:42:38.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.389+0000 7fbd53522640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd4c06e620 con 0x7fbd4c15fc40 2024-09-19T08:42:38.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.389+0000 7fbd53522640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbd4c06e900 con 0x7fbd4c06f6c0 2024-09-19T08:42:38.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.389+0000 7fbd512be640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fbd4c06cca0 0x7fbd4c06d080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:39730/0 (socket says 172.21.15.98:39730) 2024-09-19T08:42:38.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.389+0000 7fbd512be640 1 -- 172.21.15.98:0/2307688566 learned_addr learned my addr 172.21.15.98:0/2307688566 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:38.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.389+0000 7fbd43fff640 1 -- 172.21.15.98:0/2307688566 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2034964411 0 0) 0x7fbd48001120 con 0x7fbd4c06cca0 2024-09-19T08:42:38.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.389+0000 7fbd43fff640 1 -- 172.21.15.98:0/2307688566 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd38003530 con 0x7fbd4c06cca0 2024-09-19T08:42:38.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.389+0000 7fbd50abd640 1 --2- 172.21.15.98:0/2307688566 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd4c06f6c0 0x7fbd4c06fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:38.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.389+0000 7fbd43fff640 1 -- 172.21.15.98:0/2307688566 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 421502675 0 0) 0x7fbd38003530 con 0x7fbd4c06cca0 2024-09-19T08:42:38.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.389+0000 7fbd43fff640 1 -- 172.21.15.98:0/2307688566 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd4c06f6c0 msgr2=0x7fbd4c06fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:38.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.389+0000 7fbd43fff640 1 --2- 172.21.15.98:0/2307688566 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd4c06f6c0 0x7fbd4c06fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:38.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.389+0000 7fbd43fff640 1 -- 172.21.15.98:0/2307688566 >> v1:172.21.15.139:6789/0 conn(0x7fbd4c15fc40 legacy=0x7fbd4c0b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:38.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.389+0000 7fbd43fff640 1 -- 172.21.15.98:0/2307688566 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd4c06ebe0 con 0x7fbd4c06cca0 2024-09-19T08:42:38.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.389+0000 7fbd43fff640 1 -- 172.21.15.98:0/2307688566 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fbd48001850 con 0x7fbd4c06cca0 2024-09-19T08:42:38.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.389+0000 7fbd43fff640 1 -- 172.21.15.98:0/2307688566 >> v1:172.21.15.98:6789/0 conn(0x7fbd4c06cca0 legacy=0x7fbd4c06d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:38.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.389+0000 7fbd43fff640 1 --2- 172.21.15.98:0/2307688566 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd4c06f6c0 0x7fbd380039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:38.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.389+0000 7fbd43fff640 1 --2- 172.21.15.98:0/2307688566 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd38004020 0x7fbd38004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:38.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.389+0000 7fbd43fff640 1 --2- 172.21.15.98:0/2307688566 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd38004d50 0x7fbd380051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:38.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.389+0000 7fbd43fff640 1 -- 172.21.15.98:0/2307688566 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c06e900 con 0x7fbd38004d50 2024-09-19T08:42:38.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.389+0000 7fbd43fff640 1 -- 172.21.15.98:0/2307688566 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3800bb90 con 0x7fbd38004020 2024-09-19T08:42:38.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.389+0000 7fbd43fff640 1 -- 172.21.15.98:0/2307688566 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbd3800bd90 con 0x7fbd4c06f6c0 2024-09-19T08:42:38.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.389+0000 7fbd51abf640 1 --2- 172.21.15.98:0/2307688566 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd38004d50 0x7fbd380051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:38.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.389+0000 7fbd50abd640 1 --2- 172.21.15.98:0/2307688566 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd38004020 0x7fbd38004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:38.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.389+0000 7fbd512be640 1 --2- 172.21.15.98:0/2307688566 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd4c06f6c0 0x7fbd380039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:38.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.389+0000 7fbd51abf640 1 -- 172.21.15.98:0/2307688566 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd4c06f6c0 msgr2=0x7fbd380039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:38.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.389+0000 7fbd51abf640 1 --2- 172.21.15.98:0/2307688566 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd4c06f6c0 0x7fbd380039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:38.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.389+0000 7fbd51abf640 1 -- 172.21.15.98:0/2307688566 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd38004020 msgr2=0x7fbd38004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:38.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.389+0000 7fbd51abf640 1 --2- 172.21.15.98:0/2307688566 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd38004020 0x7fbd38004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:38.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.389+0000 7fbd51abf640 1 -- 172.21.15.98:0/2307688566 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd38003530 con 0x7fbd38004d50 2024-09-19T08:42:38.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.393+0000 7fbd51abf640 1 --2- 172.21.15.98:0/2307688566 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd38004d50 0x7fbd380051d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fbd44001e70 tx=0x7fbd4401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce2414520cdcddf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:38.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.393+0000 7fbd43fff640 1 -- 172.21.15.98:0/2307688566 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd44001a90 con 0x7fbd38004d50 2024-09-19T08:42:38.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.393+0000 7fbd43fff640 1 -- 172.21.15.98:0/2307688566 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd44003b80 con 0x7fbd38004d50 2024-09-19T08:42:38.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.393+0000 7fbd43fff640 1 -- 172.21.15.98:0/2307688566 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd44005080 con 0x7fbd38004d50 2024-09-19T08:42:38.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.393+0000 7fbd53522640 1 -- 172.21.15.98:0/2307688566 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd38004d50 msgr2=0x7fbd380051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:38.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.393+0000 7fbd53522640 1 --2- 172.21.15.98:0/2307688566 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd38004d50 0x7fbd380051d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fbd44001e70 tx=0x7fbd4401c780 comp rx=0 tx=0).stop 2024-09-19T08:42:38.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.393+0000 7fbd51abf640 1 -- 172.21.15.98:0/2307688566 reap_dead start 2024-09-19T08:42:38.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.393+0000 7fbd53522640 1 -- 172.21.15.98:0/2307688566 shutdown_connections 2024-09-19T08:42:38.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.393+0000 7fbd53522640 1 -- 172.21.15.98:0/2307688566 >> 172.21.15.98:0/2307688566 conn(0x7fbd4c0720a0 msgr2=0x7fbd4c0724b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:38.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.393+0000 7fbd53522640 1 -- 172.21.15.98:0/2307688566 shutdown_connections 2024-09-19T08:42:38.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.393+0000 7fbd53522640 1 -- 172.21.15.98:0/2307688566 wait complete. 2024-09-19T08:42:38.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.393+0000 7fbd53522640 1 Processor -- start 2024-09-19T08:42:38.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.393+0000 7fbd53522640 1 -- start start 2024-09-19T08:42:38.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.393+0000 7fbd53522640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd4c1583e0 0x7fbd4c144330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:38.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.393+0000 7fbd53522640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd4c139140 0x7fbd4c1591a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:38.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.393+0000 7fbd53522640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd4c145790 0x7fbd4c145c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:38.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.393+0000 7fbd53522640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c0b5400 con 0x7fbd4c145790 2024-09-19T08:42:38.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.393+0000 7fbd53522640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c06f1a0 con 0x7fbd4c1583e0 2024-09-19T08:42:38.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.393+0000 7fbd53522640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbd4c0b5580 con 0x7fbd4c139140 2024-09-19T08:42:38.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.397+0000 7fbd51abf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd4c145790 0x7fbd4c145c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:38.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.397+0000 7fbd51abf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd4c145790 0x7fbd4c145c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36628/0 (socket says 172.21.15.98:36628) 2024-09-19T08:42:38.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.397+0000 7fbd51abf640 1 -- 172.21.15.98:0/3993782058 learned_addr learned my addr 172.21.15.98:0/3993782058 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:38.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.397+0000 7fbd512be640 1 --2- 172.21.15.98:0/3993782058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd4c1583e0 0x7fbd4c144330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:38.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.397+0000 7fbd50abd640 1 --2- 172.21.15.98:0/3993782058 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd4c139140 0x7fbd4c1591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:38.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.397+0000 7fbd51abf640 1 -- 172.21.15.98:0/3993782058 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd4c139140 msgr2=0x7fbd4c1591a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:38.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.397+0000 7fbd51abf640 1 --2- 172.21.15.98:0/3993782058 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd4c139140 0x7fbd4c1591a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:38.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.397+0000 7fbd51abf640 1 -- 172.21.15.98:0/3993782058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd4c1583e0 msgr2=0x7fbd4c144330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:38.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.397+0000 7fbd51abf640 1 --2- 172.21.15.98:0/3993782058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd4c1583e0 0x7fbd4c144330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:38.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.397+0000 7fbd51abf640 1 -- 172.21.15.98:0/3993782058 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd4c139600 con 0x7fbd4c145790 2024-09-19T08:42:38.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.397+0000 7fbd50abd640 1 --2- 172.21.15.98:0/3993782058 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd4c139140 0x7fbd4c1591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:42:38.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.397+0000 7fbd51abf640 1 --2- 172.21.15.98:0/3993782058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd4c145790 0x7fbd4c145c20 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fbd44098190 tx=0x7fbd38005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:38.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.397+0000 7fbd41ffb640 1 -- 172.21.15.98:0/3993782058 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd440b8070 con 0x7fbd4c145790 2024-09-19T08:42:38.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.397+0000 7fbd41ffb640 1 -- 172.21.15.98:0/3993782058 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd44005820 con 0x7fbd4c145790 2024-09-19T08:42:38.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.397+0000 7fbd53522640 1 -- 172.21.15.98:0/3993782058 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd4c1398e0 con 0x7fbd4c145790 2024-09-19T08:42:38.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.397+0000 7fbd41ffb640 1 -- 172.21.15.98:0/3993782058 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd44007dc0 con 0x7fbd4c145790 2024-09-19T08:42:38.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.397+0000 7fbd53522640 1 -- 172.21.15.98:0/3993782058 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd4c139ea0 con 0x7fbd4c145790 2024-09-19T08:42:38.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.397+0000 7fbd53522640 1 -- 172.21.15.98:0/3993782058 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd4c06fb20 con 0x7fbd4c145790 2024-09-19T08:42:38.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.397+0000 7fbd41ffb640 1 -- 172.21.15.98:0/3993782058 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fbd440bb070 con 0x7fbd4c145790 2024-09-19T08:42:38.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.397+0000 7fbd41ffb640 1 --2- 172.21.15.98:0/3993782058 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fbd30067c50 0x7fbd3006a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:38.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.397+0000 7fbd512be640 1 --2- 172.21.15.98:0/3993782058 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fbd30067c50 0x7fbd3006a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:38.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.397+0000 7fbd41ffb640 1 -- 172.21.15.98:0/3993782058 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7fbd4411f5f0 con 0x7fbd4c145790 2024-09-19T08:42:38.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.397+0000 7fbd512be640 1 --2- 172.21.15.98:0/3993782058 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fbd30067c50 0x7fbd3006a110 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbd48004850 tx=0x7fbd48009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:38.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.401+0000 7fbd41ffb640 1 -- 172.21.15.98:0/3993782058 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fbd440ac020 con 0x7fbd4c145790 2024-09-19T08:42:38.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.613+0000 7fbd53522640 1 -- 172.21.15.98:0/3993782058 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fbd4c06e2f0 con 0x7fbd4c145790 2024-09-19T08:42:38.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.617+0000 7fbd41ffb640 1 -- 172.21.15.98:0/3993782058 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) ==== 190+0+147 (secure 0 0 0) 0x7fbd44016490 con 0x7fbd4c145790 2024-09-19T08:42:38.619 INFO:teuthology.orchestra.run.smithi098.stdout:[client.0] 2024-09-19T08:42:38.619 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQC/4utm0vNpIhAA4S1E8dFZ0+ibmSKhmZ+bIA== 2024-09-19T08:42:38.619 INFO:teuthology.orchestra.run.smithi098.stdout: caps mds = "allow" 2024-09-19T08:42:38.619 INFO:teuthology.orchestra.run.smithi098.stdout: caps mon = "allow r" 2024-09-19T08:42:38.619 INFO:teuthology.orchestra.run.smithi098.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-19T08:42:38.619 INFO:teuthology.orchestra.run.smithi098.stderr:updated caps for client.0 2024-09-19T08:42:38.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.621+0000 7fbd53522640 1 -- 172.21.15.98:0/3993782058 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fbd30067c50 msgr2=0x7fbd3006a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:38.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.621+0000 7fbd53522640 1 --2- 172.21.15.98:0/3993782058 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fbd30067c50 0x7fbd3006a110 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbd48004850 tx=0x7fbd48009040 comp rx=0 tx=0).stop 2024-09-19T08:42:38.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.621+0000 7fbd53522640 1 -- 172.21.15.98:0/3993782058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd4c145790 msgr2=0x7fbd4c145c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:38.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.621+0000 7fbd53522640 1 --2- 172.21.15.98:0/3993782058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd4c145790 0x7fbd4c145c20 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fbd44098190 tx=0x7fbd38005840 comp rx=0 tx=0).stop 2024-09-19T08:42:38.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.621+0000 7fbd53522640 1 -- 172.21.15.98:0/3993782058 shutdown_connections 2024-09-19T08:42:38.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.621+0000 7fbd53522640 1 --2- 172.21.15.98:0/3993782058 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fbd30067c50 0x7fbd3006a110 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:38.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.621+0000 7fbd53522640 1 --2- 172.21.15.98:0/3993782058 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd4c145790 0x7fbd4c145c20 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:38.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.621+0000 7fbd53522640 1 --2- 172.21.15.98:0/3993782058 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd4c139140 0x7fbd4c1591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:38.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.621+0000 7fbd53522640 1 --2- 172.21.15.98:0/3993782058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd4c1583e0 0x7fbd4c144330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:38.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.621+0000 7fbd53522640 1 -- 172.21.15.98:0/3993782058 >> 172.21.15.98:0/3993782058 conn(0x7fbd4c0720a0 msgr2=0x7fbd4c15f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:38.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.621+0000 7fbd53522640 1 -- 172.21.15.98:0/3993782058 shutdown_connections 2024-09-19T08:42:38.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.621+0000 7fbd53522640 1 -- 172.21.15.98:0/3993782058 wait complete. 2024-09-19T08:42:38.637 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:42:38.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.745+0000 7f2c9383e640 1 Processor -- start 2024-09-19T08:42:38.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.745+0000 7f2c9383e640 1 -- start start 2024-09-19T08:42:38.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.745+0000 7f2c9383e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2c8c15e420 0x7f2c8c15f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:38.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.745+0000 7f2c9383e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c8c15e800 0x7f2c8c06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:38.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.745+0000 7f2c9383e640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c8c06c6e0 con 0x7f2c8c06c1d0 2024-09-19T08:42:38.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.745+0000 7f2c9383e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f2c8c06c9c0 con 0x7f2c8c15e420 2024-09-19T08:42:38.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.745+0000 7f2c9383e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2c8c15ed60 con 0x7f2c8c15e800 2024-09-19T08:42:38.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.745+0000 7f2c91ddb640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f2c8c06c1d0 0x7f2c8c0b78b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:39736/0 (socket says 172.21.15.98:39736) 2024-09-19T08:42:38.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.745+0000 7f2c91ddb640 1 -- 172.21.15.98:0/1978054500 learned_addr learned my addr 172.21.15.98:0/1978054500 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:38.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.745+0000 7f2c90dd9640 1 --2- 172.21.15.98:0/1978054500 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c8c15e800 0x7f2c8c06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:38.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.745+0000 7f2c915da640 1 --2- 172.21.15.98:0/1978054500 >> v2:172.21.15.139:3300/0 conn(0x7f2c8c15e420 0x7f2c8c15f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:38.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.745+0000 7f2c83fff640 1 -- 172.21.15.98:0/1978054500 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3893224861 0 0) 0x7f2c8c06c6e0 con 0x7f2c8c06c1d0 2024-09-19T08:42:38.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.745+0000 7f2c83fff640 1 -- 172.21.15.98:0/1978054500 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c78003530 con 0x7f2c8c06c1d0 2024-09-19T08:42:38.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.745+0000 7f2c83fff640 1 -- 172.21.15.98:0/1978054500 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4078485032 0 0) 0x7f2c78003530 con 0x7f2c8c06c1d0 2024-09-19T08:42:38.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.745+0000 7f2c83fff640 1 -- 172.21.15.98:0/1978054500 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c8c15e800 msgr2=0x7f2c8c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:38.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.745+0000 7f2c83fff640 1 --2- 172.21.15.98:0/1978054500 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c8c15e800 0x7f2c8c06bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:38.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.745+0000 7f2c83fff640 1 -- 172.21.15.98:0/1978054500 >> v2:172.21.15.139:3300/0 conn(0x7f2c8c15e420 msgr2=0x7f2c8c15f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:38.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.745+0000 7f2c83fff640 1 --2- 172.21.15.98:0/1978054500 >> v2:172.21.15.139:3300/0 conn(0x7f2c8c15e420 0x7f2c8c15f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:38.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.745+0000 7f2c83fff640 1 -- 172.21.15.98:0/1978054500 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c8c15f040 con 0x7f2c8c06c1d0 2024-09-19T08:42:38.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.745+0000 7f2c83fff640 1 -- 172.21.15.98:0/1978054500 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f2c84001a00 con 0x7f2c8c06c1d0 2024-09-19T08:42:38.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.745+0000 7f2c83fff640 1 -- 172.21.15.98:0/1978054500 >> v1:172.21.15.98:6789/0 conn(0x7f2c8c06c1d0 legacy=0x7f2c8c0b78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:38.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.745+0000 7f2c83fff640 1 --2- 172.21.15.98:0/1978054500 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c8c15e800 0x7f2c780039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:38.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.745+0000 7f2c83fff640 1 --2- 172.21.15.98:0/1978054500 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c78003f40 0x7f2c78004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:38.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.745+0000 7f2c83fff640 1 --2- 172.21.15.98:0/1978054500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c78004d50 0x7f2c780051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:38.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.745+0000 7f2c83fff640 1 -- 172.21.15.98:0/1978054500 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8c06c9c0 con 0x7f2c78004d50 2024-09-19T08:42:38.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.745+0000 7f2c83fff640 1 -- 172.21.15.98:0/1978054500 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8c15ed60 con 0x7f2c78003f40 2024-09-19T08:42:38.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.745+0000 7f2c83fff640 1 -- 172.21.15.98:0/1978054500 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2c7800b9d0 con 0x7f2c8c15e800 2024-09-19T08:42:38.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.745+0000 7f2c91ddb640 1 --2- 172.21.15.98:0/1978054500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c78004d50 0x7f2c780051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:38.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.745+0000 7f2c90dd9640 1 --2- 172.21.15.98:0/1978054500 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c78003f40 0x7f2c78004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:38.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.745+0000 7f2c915da640 1 --2- 172.21.15.98:0/1978054500 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c8c15e800 0x7f2c780039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:38.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.745+0000 7f2c91ddb640 1 -- 172.21.15.98:0/1978054500 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c8c15e800 msgr2=0x7f2c780039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:38.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.745+0000 7f2c91ddb640 1 --2- 172.21.15.98:0/1978054500 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c8c15e800 0x7f2c780039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:38.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.745+0000 7f2c91ddb640 1 -- 172.21.15.98:0/1978054500 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c78003f40 msgr2=0x7f2c78004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:38.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.745+0000 7f2c91ddb640 1 --2- 172.21.15.98:0/1978054500 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c78003f40 0x7f2c78004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:38.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.745+0000 7f2c91ddb640 1 -- 172.21.15.98:0/1978054500 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c78003530 con 0x7f2c78004d50 2024-09-19T08:42:38.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.749+0000 7f2c91ddb640 1 --2- 172.21.15.98:0/1978054500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c78004d50 0x7f2c780051d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f2c8400a910 tx=0x7f2c8401d3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9dfe88f4efb42 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:38.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.749+0000 7f2c83fff640 1 -- 172.21.15.98:0/1978054500 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8400a4e0 con 0x7f2c78004d50 2024-09-19T08:42:38.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.749+0000 7f2c83fff640 1 -- 172.21.15.98:0/1978054500 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c840072d0 con 0x7f2c78004d50 2024-09-19T08:42:38.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.749+0000 7f2c83fff640 1 -- 172.21.15.98:0/1978054500 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c840076c0 con 0x7f2c78004d50 2024-09-19T08:42:38.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.749+0000 7f2c9383e640 1 -- 172.21.15.98:0/1978054500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c78004d50 msgr2=0x7f2c780051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:38.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.749+0000 7f2c9383e640 1 --2- 172.21.15.98:0/1978054500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c78004d50 0x7f2c780051d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f2c8400a910 tx=0x7f2c8401d3e0 comp rx=0 tx=0).stop 2024-09-19T08:42:38.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.749+0000 7f2c91ddb640 1 -- 172.21.15.98:0/1978054500 reap_dead start 2024-09-19T08:42:38.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.749+0000 7f2c9383e640 1 -- 172.21.15.98:0/1978054500 shutdown_connections 2024-09-19T08:42:38.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.749+0000 7f2c9383e640 1 -- 172.21.15.98:0/1978054500 >> 172.21.15.98:0/1978054500 conn(0x7f2c8c0b40a0 msgr2=0x7f2c8c0b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:38.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.749+0000 7f2c9383e640 1 -- 172.21.15.98:0/1978054500 shutdown_connections 2024-09-19T08:42:38.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.749+0000 7f2c9383e640 1 -- 172.21.15.98:0/1978054500 wait complete. 2024-09-19T08:42:38.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.749+0000 7f2c9383e640 1 Processor -- start 2024-09-19T08:42:38.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.749+0000 7f2c9383e640 1 -- start start 2024-09-19T08:42:38.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.753+0000 7f2c9383e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c8c152f70 0x7f2c8c0b1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:38.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.753+0000 7f2c9383e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c8c161410 0x7f2c8c0b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:38.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.753+0000 7f2c9383e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c8c161890 0x7f2c8c0b3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:38.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.753+0000 7f2c9383e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8c0b8140 con 0x7f2c8c161410 2024-09-19T08:42:38.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.753+0000 7f2c9383e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8c0b7fc0 con 0x7f2c8c161890 2024-09-19T08:42:38.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.753+0000 7f2c9383e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2c8c06bc90 con 0x7f2c8c152f70 2024-09-19T08:42:38.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.753+0000 7f2c90dd9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c8c161410 0x7f2c8c0b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:38.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.753+0000 7f2c90dd9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c8c161410 0x7f2c8c0b33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36642/0 (socket says 172.21.15.98:36642) 2024-09-19T08:42:38.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.753+0000 7f2c90dd9640 1 -- 172.21.15.98:0/2605672568 learned_addr learned my addr 172.21.15.98:0/2605672568 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:38.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.753+0000 7f2c91ddb640 1 --2- 172.21.15.98:0/2605672568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c8c161890 0x7f2c8c0b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:38.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.753+0000 7f2c915da640 1 --2- 172.21.15.98:0/2605672568 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c8c152f70 0x7f2c8c0b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:38.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.753+0000 7f2c90dd9640 1 -- 172.21.15.98:0/2605672568 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c8c152f70 msgr2=0x7f2c8c0b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:38.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.753+0000 7f2c90dd9640 1 --2- 172.21.15.98:0/2605672568 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c8c152f70 0x7f2c8c0b1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:38.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.753+0000 7f2c90dd9640 1 -- 172.21.15.98:0/2605672568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c8c161890 msgr2=0x7f2c8c0b3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:38.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.753+0000 7f2c90dd9640 1 --2- 172.21.15.98:0/2605672568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c8c161890 0x7f2c8c0b3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:38.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.753+0000 7f2c90dd9640 1 -- 172.21.15.98:0/2605672568 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c8c162040 con 0x7f2c8c161410 2024-09-19T08:42:38.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.753+0000 7f2c90dd9640 1 --2- 172.21.15.98:0/2605672568 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c8c161410 0x7f2c8c0b33d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c0119d0 tx=0x7f2c7c011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:38.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.753+0000 7f2c81ffb640 1 -- 172.21.15.98:0/2605672568 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c00fb50 con 0x7f2c8c161410 2024-09-19T08:42:38.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.753+0000 7f2c81ffb640 1 -- 172.21.15.98:0/2605672568 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c7c00fcf0 con 0x7f2c8c161410 2024-09-19T08:42:38.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.753+0000 7f2c81ffb640 1 -- 172.21.15.98:0/2605672568 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c00fea0 con 0x7f2c8c161410 2024-09-19T08:42:38.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.753+0000 7f2c9383e640 1 -- 172.21.15.98:0/2605672568 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c8c161d60 con 0x7f2c8c161410 2024-09-19T08:42:38.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.753+0000 7f2c9383e640 1 -- 172.21.15.98:0/2605672568 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c8c0aec30 con 0x7f2c8c161410 2024-09-19T08:42:38.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.753+0000 7f2c9383e640 1 -- 172.21.15.98:0/2605672568 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c8c06c640 con 0x7f2c8c161410 2024-09-19T08:42:38.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.753+0000 7f2c81ffb640 1 -- 172.21.15.98:0/2605672568 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f2c7c012040 con 0x7f2c8c161410 2024-09-19T08:42:38.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.753+0000 7f2c81ffb640 1 --2- 172.21.15.98:0/2605672568 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2c70067c20 0x7f2c7006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:38.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.753+0000 7f2c915da640 1 --2- 172.21.15.98:0/2605672568 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2c70067c20 0x7f2c7006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:38.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.753+0000 7f2c81ffb640 1 -- 172.21.15.98:0/2605672568 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f2c7c086830 con 0x7f2c8c161410 2024-09-19T08:42:38.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.757+0000 7f2c915da640 1 --2- 172.21.15.98:0/2605672568 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2c70067c20 0x7f2c7006a0e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2c88009ba0 tx=0x7f2c88008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:38.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.761+0000 7f2c81ffb640 1 -- 172.21.15.98:0/2605672568 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2c7c019030 con 0x7f2c8c161410 2024-09-19T08:42:38.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.949+0000 7f2c9383e640 1 -- 172.21.15.98:0/2605672568 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c8c06dc80 con 0x7f2c8c161410 2024-09-19T08:42:38.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.949+0000 7f2c81ffb640 1 -- 172.21.15.98:0/2605672568 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5228 (secure 0 0 0) 0x7f2c7c0570b0 con 0x7f2c8c161410 2024-09-19T08:42:38.952 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:42:38.952 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":38,"btime":"2024-09-19T08:42:32:414567+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5548,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3092385040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3092385040},{"type":"v1","addr":"172.21.15.139:6833","nonce":3092385040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35},{"gid":5551,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3517671290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3517671290},{"type":"v1","addr":"172.21.15.139:6835","nonce":3517671290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35},{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6836/965821274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":965821274},{"type":"v1","addr":"172.21.15.98:6836","nonce":965821274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:42:29.720825+0000","modified":"2024-09-19T08:42:31.743275+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6171":{"gid":6171,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/1767881083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6835","nonce":1767881083},{"type":"v1","addr":"172.21.15.98:6837","nonce":1767881083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6171,"qdb_cluster":[6171]},"id":5}]} 2024-09-19T08:42:38.952 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 38 2024-09-19T08:42:38.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.953+0000 7f2c9383e640 1 -- 172.21.15.98:0/2605672568 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2c70067c20 msgr2=0x7f2c7006a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:38.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.953+0000 7f2c9383e640 1 --2- 172.21.15.98:0/2605672568 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2c70067c20 0x7f2c7006a0e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2c88009ba0 tx=0x7f2c88008040 comp rx=0 tx=0).stop 2024-09-19T08:42:38.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.953+0000 7f2c9383e640 1 -- 172.21.15.98:0/2605672568 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c8c161410 msgr2=0x7f2c8c0b33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:38.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.953+0000 7f2c9383e640 1 --2- 172.21.15.98:0/2605672568 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c8c161410 0x7f2c8c0b33d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c0119d0 tx=0x7f2c7c011e90 comp rx=0 tx=0).stop 2024-09-19T08:42:38.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.953+0000 7f2c9383e640 1 -- 172.21.15.98:0/2605672568 shutdown_connections 2024-09-19T08:42:38.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.953+0000 7f2c9383e640 1 --2- 172.21.15.98:0/2605672568 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f2c70067c20 0x7f2c7006a0e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:38.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.953+0000 7f2c9383e640 1 --2- 172.21.15.98:0/2605672568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c8c161890 0x7f2c8c0b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:38.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.953+0000 7f2c9383e640 1 --2- 172.21.15.98:0/2605672568 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c8c161410 0x7f2c8c0b33d0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:38.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.953+0000 7f2c9383e640 1 --2- 172.21.15.98:0/2605672568 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c8c152f70 0x7f2c8c0b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:38.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.953+0000 7f2c9383e640 1 -- 172.21.15.98:0/2605672568 >> 172.21.15.98:0/2605672568 conn(0x7f2c8c0b40a0 msgr2=0x7f2c8c1592b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:38.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.953+0000 7f2c9383e640 1 -- 172.21.15.98:0/2605672568 shutdown_connections 2024-09-19T08:42:38.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:38.953+0000 7f2c9383e640 1 -- 172.21.15.98:0/2605672568 wait complete. 2024-09-19T08:42:39.374 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 38, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T08:42:29.720825+0000', 'modified': '2024-09-19T08:42:31.743275+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6171}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6171': {'gid': 6171, 'name': 'a', 'rank': 0, 'incarnation': 36, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.98:6837/1767881083', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6835', 'nonce': 1767881083}, {'type': 'v1', 'addr': '172.21.15.98:6837', 'nonce': 1767881083}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6171, 'qdb_cluster': [6171]} 2024-09-19T08:42:39.374 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T08:42:39.374 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-19T08:42:39.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.505+0000 7fe3d60a7640 1 Processor -- start 2024-09-19T08:42:39.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.505+0000 7fe3d60a7640 1 -- start start 2024-09-19T08:42:39.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.505+0000 7fe3d60a7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe3d015c550 0x7fe3d006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:39.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.505+0000 7fe3d60a7640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe3d00cfad0 0x7fe3d00723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:39.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.505+0000 7fe3d60a7640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3d006d310 con 0x7fe3d006c1d0 2024-09-19T08:42:39.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.505+0000 7fe3d60a7640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fe3d006c640 con 0x7fe3d00cfad0 2024-09-19T08:42:39.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.505+0000 7fe3d60a7640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe3d006d5f0 con 0x7fe3d015c550 2024-09-19T08:42:39.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.505+0000 7fe3cffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe3d00cfad0 0x7fe3d00723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:39.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.505+0000 7fe3cffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe3d00cfad0 0x7fe3d00723a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36644/0 (socket says 172.21.15.98:36644) 2024-09-19T08:42:39.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.505+0000 7fe3cffff640 1 -- 172.21.15.98:0/3244674937 learned_addr learned my addr 172.21.15.98:0/3244674937 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:39.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.505+0000 7fe3cf7fe640 1 --2- 172.21.15.98:0/3244674937 >> v2:172.21.15.139:3300/0 conn(0x7fe3d015c550 0x7fe3d006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:39.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.505+0000 7fe3cffff640 1 -- 172.21.15.98:0/3244674937 >> v2:172.21.15.139:3300/0 conn(0x7fe3d015c550 msgr2=0x7fe3d006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:39.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.505+0000 7fe3cffff640 1 --2- 172.21.15.98:0/3244674937 >> v2:172.21.15.139:3300/0 conn(0x7fe3d015c550 0x7fe3d006bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.505+0000 7fe3cffff640 1 -- 172.21.15.98:0/3244674937 >> v1:172.21.15.98:6789/0 conn(0x7fe3d006c1d0 legacy=0x7fe3d00cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:39.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.505+0000 7fe3cffff640 1 -- 172.21.15.98:0/3244674937 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d006d8d0 con 0x7fe3d00cfad0 2024-09-19T08:42:39.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.509+0000 7fe3cffff640 1 --2- 172.21.15.98:0/3244674937 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d00cfad0 0x7fe3d00723a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fe3d00b56b0 tx=0x7fe3c801c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2692fbc9dd558f9c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:39.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.509+0000 7fe3ce7fc640 1 -- 172.21.15.98:0/3244674937 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c80a56e0 con 0x7fe3d00cfad0 2024-09-19T08:42:39.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.509+0000 7fe3ce7fc640 1 -- 172.21.15.98:0/3244674937 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3c80a5880 con 0x7fe3d00cfad0 2024-09-19T08:42:39.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.509+0000 7fe3ce7fc640 1 -- 172.21.15.98:0/3244674937 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c80a5c50 con 0x7fe3d00cfad0 2024-09-19T08:42:39.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.509+0000 7fe3d60a7640 1 -- 172.21.15.98:0/3244674937 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d00cfad0 msgr2=0x7fe3d00723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:39.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.509+0000 7fe3d60a7640 1 --2- 172.21.15.98:0/3244674937 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d00cfad0 0x7fe3d00723a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fe3d00b56b0 tx=0x7fe3c801c880 comp rx=0 tx=0).stop 2024-09-19T08:42:39.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.509+0000 7fe3d60a7640 1 -- 172.21.15.98:0/3244674937 shutdown_connections 2024-09-19T08:42:39.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.509+0000 7fe3d60a7640 1 --2- 172.21.15.98:0/3244674937 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d00cfad0 0x7fe3d00723a0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.509+0000 7fe3d60a7640 1 --2- 172.21.15.98:0/3244674937 >> v2:172.21.15.139:3300/0 conn(0x7fe3d015c550 0x7fe3d006bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.509+0000 7fe3d60a7640 1 -- 172.21.15.98:0/3244674937 >> 172.21.15.98:0/3244674937 conn(0x7fe3d00b2010 msgr2=0x7fe3d00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:39.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.509+0000 7fe3d60a7640 1 -- 172.21.15.98:0/3244674937 shutdown_connections 2024-09-19T08:42:39.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.509+0000 7fe3d60a7640 1 -- 172.21.15.98:0/3244674937 wait complete. 2024-09-19T08:42:39.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.509+0000 7fe3d60a7640 1 Processor -- start 2024-09-19T08:42:39.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.509+0000 7fe3d60a7640 1 -- start start 2024-09-19T08:42:39.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.513+0000 7fe3d60a7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d006c1d0 0x7fe3d0143720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:39.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.513+0000 7fe3d60a7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3d00cfad0 0x7fe3d01403e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:39.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.513+0000 7fe3d60a7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3d015c550 0x7fe3d0150ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:39.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.513+0000 7fe3d60a7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d015cab0 con 0x7fe3d006c1d0 2024-09-19T08:42:39.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.513+0000 7fe3d60a7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d015c930 con 0x7fe3d015c550 2024-09-19T08:42:39.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.513+0000 7fe3d60a7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe3d00728e0 con 0x7fe3d00cfad0 2024-09-19T08:42:39.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.513+0000 7fe3cf7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d006c1d0 0x7fe3d0143720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:39.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.513+0000 7fe3cf7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d006c1d0 0x7fe3d0143720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36648/0 (socket says 172.21.15.98:36648) 2024-09-19T08:42:39.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.513+0000 7fe3cf7fe640 1 -- 172.21.15.98:0/1409613012 learned_addr learned my addr 172.21.15.98:0/1409613012 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:39.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.513+0000 7fe3cffff640 1 --2- 172.21.15.98:0/1409613012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3d015c550 0x7fe3d0150ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:39.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.513+0000 7fe3cf7fe640 1 -- 172.21.15.98:0/1409613012 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3d00cfad0 msgr2=0x7fe3d01403e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:39.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.513+0000 7fe3ceffd640 1 --2- 172.21.15.98:0/1409613012 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3d00cfad0 0x7fe3d01403e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:39.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.513+0000 7fe3cf7fe640 1 --2- 172.21.15.98:0/1409613012 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3d00cfad0 0x7fe3d01403e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.513+0000 7fe3cf7fe640 1 -- 172.21.15.98:0/1409613012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3d015c550 msgr2=0x7fe3d0150ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:39.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.513+0000 7fe3cf7fe640 1 --2- 172.21.15.98:0/1409613012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3d015c550 0x7fe3d0150ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.513+0000 7fe3cf7fe640 1 -- 172.21.15.98:0/1409613012 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d0143c60 con 0x7fe3d006c1d0 2024-09-19T08:42:39.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.513+0000 7fe3cffff640 1 --2- 172.21.15.98:0/1409613012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3d015c550 0x7fe3d0150ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:42:39.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.513+0000 7fe3cf7fe640 1 --2- 172.21.15.98:0/1409613012 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d006c1d0 0x7fe3d0143720 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000c7f0 tx=0x7fe3c000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:39.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.513+0000 7fe3b7fff640 1 -- 172.21.15.98:0/1409613012 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0015070 con 0x7fe3d006c1d0 2024-09-19T08:42:39.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.513+0000 7fe3b7fff640 1 -- 172.21.15.98:0/1409613012 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3c0007820 con 0x7fe3d006c1d0 2024-09-19T08:42:39.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.513+0000 7fe3d60a7640 1 -- 172.21.15.98:0/1409613012 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3d01539d0 con 0x7fe3d006c1d0 2024-09-19T08:42:39.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.513+0000 7fe3b7fff640 1 -- 172.21.15.98:0/1409613012 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0007c10 con 0x7fe3d006c1d0 2024-09-19T08:42:39.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.513+0000 7fe3d60a7640 1 -- 172.21.15.98:0/1409613012 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3d0154270 con 0x7fe3d006c1d0 2024-09-19T08:42:39.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.513+0000 7fe3b7fff640 1 -- 172.21.15.98:0/1409613012 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fe3c0018070 con 0x7fe3d006c1d0 2024-09-19T08:42:39.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.513+0000 7fe3b7fff640 1 --2- 172.21.15.98:0/1409613012 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe3bc067c50 0x7fe3bc06a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:39.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.513+0000 7fe3b7fff640 1 -- 172.21.15.98:0/1409613012 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7fe3c00862d0 con 0x7fe3d006c1d0 2024-09-19T08:42:39.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.513+0000 7fe3ceffd640 1 --2- 172.21.15.98:0/1409613012 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe3bc067c50 0x7fe3bc06a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:39.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.513+0000 7fe3ceffd640 1 --2- 172.21.15.98:0/1409613012 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe3bc067c50 0x7fe3bc06a110 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4001ec0 tx=0x7fe3c400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:39.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3d9163000 1 Processor -- start 2024-09-19T08:42:39.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3d9163000 1 -- start start 2024-09-19T08:42:39.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3d9163000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561839e79b20 0x561839e79f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:39.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3d9163000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561839e7a440 0x561839e2fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:39.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3d9163000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561839e30290 0x561839e30670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:39.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3d9163000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561839d17b60 con 0x561839e7a440 2024-09-19T08:42:39.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3d9163000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561839c7ec70 con 0x561839e30290 2024-09-19T08:42:39.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3d9163000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x561839c24ed0 con 0x561839e79b20 2024-09-19T08:42:39.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3ceffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561839e7a440 0x561839e2fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:39.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3ceffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561839e7a440 0x561839e2fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36656/0 (socket says 172.21.15.98:36656) 2024-09-19T08:42:39.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3ceffd640 1 -- 172.21.15.98:0/1253997760 learned_addr learned my addr 172.21.15.98:0/1253997760 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:39.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3cf7fe640 1 --2- 172.21.15.98:0/1253997760 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561839e79b20 0x561839e79f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:39.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3cffff640 1 --2- 172.21.15.98:0/1253997760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561839e30290 0x561839e30670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:39.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3ceffd640 1 -- 172.21.15.98:0/1253997760 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561839e79b20 msgr2=0x561839e79f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:39.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3ceffd640 1 --2- 172.21.15.98:0/1253997760 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561839e79b20 0x561839e79f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3ceffd640 1 -- 172.21.15.98:0/1253997760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561839e30290 msgr2=0x561839e30670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:39.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3ceffd640 1 --2- 172.21.15.98:0/1253997760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561839e30290 0x561839e30670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3ceffd640 1 -- 172.21.15.98:0/1253997760 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561839c36e50 con 0x561839e7a440 2024-09-19T08:42:39.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3cffff640 1 --2- 172.21.15.98:0/1253997760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561839e30290 0x561839e30670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:42:39.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3ceffd640 1 --2- 172.21.15.98:0/1253997760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561839e7a440 0x561839e2fd50 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fe3c40166e0 tx=0x7fe3c4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=242701d069349e81 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:39.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3b57fa640 1 -- 172.21.15.98:0/1253997760 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4026070 con 0x561839e7a440 2024-09-19T08:42:39.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3b57fa640 1 -- 172.21.15.98:0/1253997760 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3c4014920 con 0x561839e7a440 2024-09-19T08:42:39.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3b57fa640 1 -- 172.21.15.98:0/1253997760 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4014cd0 con 0x561839e7a440 2024-09-19T08:42:39.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3d9163000 1 -- 172.21.15.98:0/1253997760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561839e7a440 msgr2=0x561839e2fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:39.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3d9163000 1 --2- 172.21.15.98:0/1253997760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561839e7a440 0x561839e2fd50 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fe3c40166e0 tx=0x7fe3c4012000 comp rx=0 tx=0).stop 2024-09-19T08:42:39.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3d9163000 1 -- 172.21.15.98:0/1253997760 shutdown_connections 2024-09-19T08:42:39.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3d9163000 1 --2- 172.21.15.98:0/1253997760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561839e30290 0x561839e30670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3d9163000 1 --2- 172.21.15.98:0/1253997760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561839e7a440 0x561839e2fd50 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3d9163000 1 --2- 172.21.15.98:0/1253997760 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561839e79b20 0x561839e79f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3d9163000 1 -- 172.21.15.98:0/1253997760 >> 172.21.15.98:0/1253997760 conn(0x561839e36600 msgr2=0x561839e2e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:39.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3d9163000 1 -- 172.21.15.98:0/1253997760 shutdown_connections 2024-09-19T08:42:39.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3d9163000 1 -- 172.21.15.98:0/1253997760 wait complete. 2024-09-19T08:42:39.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3d9163000 1 -- start start 2024-09-19T08:42:39.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3d9163000 1 Processor -- start 2024-09-19T08:42:39.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3d9163000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561839e79b20 0x561839e34e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:39.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3d9163000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561839e7a440 0x561839e3acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:39.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3d9163000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561839e35390 0x561839e3b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:39.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3d9163000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561839ce0b00 con 0x561839e79b20 2024-09-19T08:42:39.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3d9163000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561839c59360 con 0x561839e35390 2024-09-19T08:42:39.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3d9163000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x561839c562d0 con 0x561839e7a440 2024-09-19T08:42:39.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.517+0000 7fe3d9163000 10 client.0 _ll_register_callbacks cb 0x561839e1b820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:42:39.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3cf7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561839e79b20 0x561839e34e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:39.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3cf7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561839e79b20 0x561839e34e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36668/0 (socket says 172.21.15.98:36668) 2024-09-19T08:42:39.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3cf7fe640 1 -- 172.21.15.98:0/1976966434 learned_addr learned my addr 172.21.15.98:0/1976966434 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:39.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3cffff640 1 --2- 172.21.15.98:0/1976966434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561839e35390 0x561839e3b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:39.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3ceffd640 1 --2- 172.21.15.98:0/1976966434 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561839e7a440 0x561839e3acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:39.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3cf7fe640 1 -- 172.21.15.98:0/1976966434 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561839e7a440 msgr2=0x561839e3acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:39.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3cf7fe640 1 --2- 172.21.15.98:0/1976966434 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561839e7a440 0x561839e3acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3cf7fe640 1 -- 172.21.15.98:0/1976966434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561839e35390 msgr2=0x561839e3b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:39.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3cf7fe640 1 --2- 172.21.15.98:0/1976966434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561839e35390 0x561839e3b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3cf7fe640 1 -- 172.21.15.98:0/1976966434 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561839c25120 con 0x561839e79b20 2024-09-19T08:42:39.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3cf7fe640 1 --2- 172.21.15.98:0/1976966434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561839e79b20 0x561839e34e50 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000afd0 tx=0x7fe3c00568e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2feecd428e111130 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:39.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3ae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-19T08:42:39.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3ae7fc640 1 -- 172.21.15.98:0/1976966434 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c006e020 con 0x561839e79b20 2024-09-19T08:42:39.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3ae7fc640 1 -- 172.21.15.98:0/1976966434 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7fe3c005a110 con 0x561839e79b20 2024-09-19T08:42:39.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3d9163000 1 -- 172.21.15.98:0/1976966434 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3c4014920 con 0x561839e79b20 2024-09-19T08:42:39.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3ae7fc640 1 -- 172.21.15.98:0/1976966434 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fe3c001f040 con 0x561839e79b20 2024-09-19T08:42:39.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3d9163000 10 client.6324 fetch_fsmap learned FSMap version 38 2024-09-19T08:42:39.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3d9163000 1 -- 172.21.15.98:0/1976966434 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) -- 0x561839c1e450 con 0x561839e79b20 2024-09-19T08:42:39.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3ae7fc640 1 -- 172.21.15.98:0/1976966434 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3c4014920 con 0x561839e79b20 2024-09-19T08:42:39.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3ae7fc640 1 -- 172.21.15.98:0/1976966434 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 38) ==== 3502+0+0 (secure 0 0 0) 0x7fe3c00554c0 con 0x561839e79b20 2024-09-19T08:42:39.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3d9163000 10 client.6324 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T08:42:39.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3d9163000 10 client.6324 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T08:42:39.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3d9163000 20 client.6324 populate_metadata read hostname 'smithi098' 2024-09-19T08:42:39.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3d9163000 1 --2- 172.21.15.98:0/1976966434 >> [v2:172.21.15.98:6835/1767881083,v1:172.21.15.98:6837/1767881083] conn(0x561839e88280 0x561839e8a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:39.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3d9163000 4 client.6324 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:42:39.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3d9163000 1 -- 172.21.15.98:0/1976966434 --> [v2:172.21.15.98:6835/1767881083,v1:172.21.15.98:6837/1767881083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561839c18c00 con 0x561839e88280 2024-09-19T08:42:39.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3ceffd640 1 --2- 172.21.15.98:0/1976966434 >> [v2:172.21.15.98:6835/1767881083,v1:172.21.15.98:6837/1767881083] conn(0x561839e88280 0x561839e8a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:39.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3ceffd640 1 --2- 172.21.15.98:0/1976966434 >> [v2:172.21.15.98:6835/1767881083,v1:172.21.15.98:6837/1767881083] conn(0x561839e88280 0x561839e8a670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94b3dc4785b8a73e server_cookie=f6610272dc4f5d1a in_seq=0 out_seq=0 2024-09-19T08:42:39.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3ae7fc640 10 client.6324 ms_handle_connect on v2:172.21.15.98:6835/1767881083 2024-09-19T08:42:39.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3ae7fc640 1 -- 172.21.15.98:0/1976966434 <== mds.0 v2:172.21.15.98:6835/1767881083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561839c18c00 con 0x561839e88280 2024-09-19T08:42:39.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3ae7fc640 10 client.6324 handle_command_reply: tid=0 2024-09-19T08:42:39.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3d9163000 1 client.6324 shutdown 2024-09-19T08:42:39.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3d9163000 1 -- 172.21.15.98:0/1976966434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561839e79b20 msgr2=0x561839e34e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:39.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3d9163000 1 --2- 172.21.15.98:0/1976966434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561839e79b20 0x561839e34e50 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000afd0 tx=0x7fe3c00568e0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3d9163000 1 -- 172.21.15.98:0/1976966434 shutdown_connections 2024-09-19T08:42:39.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3d9163000 1 --2- 172.21.15.98:0/1976966434 >> [v2:172.21.15.98:6835/1767881083,v1:172.21.15.98:6837/1767881083] conn(0x561839e88280 0x561839e8a670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3d9163000 1 --2- 172.21.15.98:0/1976966434 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561839e35390 0x561839e3b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3d9163000 1 --2- 172.21.15.98:0/1976966434 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561839e7a440 0x561839e3acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3d9163000 1 --2- 172.21.15.98:0/1976966434 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561839e79b20 0x561839e34e50 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3ae7fc640 0 client.6324 ms_handle_reset on v2:172.21.15.98:6835/1767881083 2024-09-19T08:42:39.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3d9163000 1 -- 172.21.15.98:0/1976966434 >> 172.21.15.98:0/1976966434 conn(0x561839e30290 msgr2=0x561839e2ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:39.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3d9163000 1 -- 172.21.15.98:0/1976966434 shutdown_connections 2024-09-19T08:42:39.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3d9163000 1 -- 172.21.15.98:0/1976966434 wait complete. 2024-09-19T08:42:39.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.521+0000 7fe3d9163000 20 client.6324 trim_cache size 0 max 16384 2024-09-19T08:42:39.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3d9163000 1 Processor -- start 2024-09-19T08:42:39.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3d9163000 1 -- start start 2024-09-19T08:42:39.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3d9163000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561839e35390 0x561839e78c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:39.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3d9163000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561839e79b20 0x561839e791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:39.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3d9163000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561839e7a440 0x561839e8a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:39.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3d9163000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d01fd960 con 0x561839e35390 2024-09-19T08:42:39.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3d9163000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d01592b0 con 0x561839e79b20 2024-09-19T08:42:39.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3d9163000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe3d0141850 con 0x561839e7a440 2024-09-19T08:42:39.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3cf7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561839e35390 0x561839e78c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:39.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3cf7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561839e35390 0x561839e78c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36682/0 (socket says 172.21.15.98:36682) 2024-09-19T08:42:39.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3cf7fe640 1 -- 172.21.15.98:0/968046929 learned_addr learned my addr 172.21.15.98:0/968046929 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:39.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3cffff640 1 --2- 172.21.15.98:0/968046929 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561839e7a440 0x561839e8a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:39.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3ceffd640 1 --2- 172.21.15.98:0/968046929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561839e79b20 0x561839e791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:39.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3cf7fe640 1 -- 172.21.15.98:0/968046929 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561839e7a440 msgr2=0x561839e8a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:39.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3cf7fe640 1 --2- 172.21.15.98:0/968046929 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561839e7a440 0x561839e8a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3cf7fe640 1 -- 172.21.15.98:0/968046929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561839e79b20 msgr2=0x561839e791d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:39.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3cf7fe640 1 --2- 172.21.15.98:0/968046929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561839e79b20 0x561839e791d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3cf7fe640 1 -- 172.21.15.98:0/968046929 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561839c05fd0 con 0x561839e35390 2024-09-19T08:42:39.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3cf7fe640 1 --2- 172.21.15.98:0/968046929 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561839e35390 0x561839e78c90 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0000c10 tx=0x7fe3c0059000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896fa5bd5c1839cc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:39.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3b4ff9640 1 -- 172.21.15.98:0/968046929 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c006e030 con 0x561839e35390 2024-09-19T08:42:39.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3b4ff9640 1 -- 172.21.15.98:0/968046929 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3c005a110 con 0x561839e35390 2024-09-19T08:42:39.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3b4ff9640 1 -- 172.21.15.98:0/968046929 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0051dc0 con 0x561839e35390 2024-09-19T08:42:39.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3d9163000 1 -- 172.21.15.98:0/968046929 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561839e35390 msgr2=0x561839e78c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:39.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3d9163000 1 --2- 172.21.15.98:0/968046929 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561839e35390 0x561839e78c90 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0000c10 tx=0x7fe3c0059000 comp rx=0 tx=0).stop 2024-09-19T08:42:39.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3d9163000 1 -- 172.21.15.98:0/968046929 shutdown_connections 2024-09-19T08:42:39.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3d9163000 1 --2- 172.21.15.98:0/968046929 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561839e7a440 0x561839e8a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3d9163000 1 --2- 172.21.15.98:0/968046929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561839e79b20 0x561839e791d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3d9163000 1 --2- 172.21.15.98:0/968046929 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561839e35390 0x561839e78c90 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3d9163000 1 -- 172.21.15.98:0/968046929 >> 172.21.15.98:0/968046929 conn(0x561839e30290 msgr2=0x561839e34650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:39.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3d9163000 1 -- 172.21.15.98:0/968046929 shutdown_connections 2024-09-19T08:42:39.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3d9163000 1 -- 172.21.15.98:0/968046929 wait complete. 2024-09-19T08:42:39.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3d9163000 1 -- start start 2024-09-19T08:42:39.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3d9163000 1 Processor -- start 2024-09-19T08:42:39.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3d9163000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561839e35390 0x561839e8d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:39.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3d9163000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561839e79b20 0x561839ea8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:39.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3d9163000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561839e7a440 0x561839ea8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:39.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3d9163000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561839c7df80 con 0x561839e7a440 2024-09-19T08:42:39.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3d9163000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561839c36ca0 con 0x561839e79b20 2024-09-19T08:42:39.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3d9163000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x561839c832a0 con 0x561839e35390 2024-09-19T08:42:39.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3d9163000 10 client.0 _ll_register_callbacks cb 0x7fe3c0052330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:42:39.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3cffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561839e7a440 0x561839ea8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:39.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3cffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561839e7a440 0x561839ea8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36684/0 (socket says 172.21.15.98:36684) 2024-09-19T08:42:39.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.541+0000 7fe3cffff640 1 -- 172.21.15.98:0/2270224718 learned_addr learned my addr 172.21.15.98:0/2270224718 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:39.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3cffff640 1 -- 172.21.15.98:0/2270224718 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561839e35390 msgr2=0x561839e8d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:39.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3cffff640 1 --2- 172.21.15.98:0/2270224718 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561839e35390 0x561839e8d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3cffff640 1 -- 172.21.15.98:0/2270224718 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561839e79b20 msgr2=0x561839ea8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:39.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3cffff640 1 --2- 172.21.15.98:0/2270224718 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561839e79b20 0x561839ea8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3cffff640 1 -- 172.21.15.98:0/2270224718 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561839d01750 con 0x561839e7a440 2024-09-19T08:42:39.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3cffff640 1 --2- 172.21.15.98:0/2270224718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561839e7a440 0x561839ea8ea0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800c090 tx=0x7fe3c801b0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f04a9f3ba73ec3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:39.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3ae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-19T08:42:39.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3ae7fc640 1 -- 172.21.15.98:0/2270224718 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c80a5c70 con 0x561839e7a440 2024-09-19T08:42:39.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3ae7fc640 1 -- 172.21.15.98:0/2270224718 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7fe3c80c0ac0 con 0x561839e7a440 2024-09-19T08:42:39.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3d9163000 1 -- 172.21.15.98:0/2270224718 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3c005a110 con 0x561839e7a440 2024-09-19T08:42:39.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3ae7fc640 1 -- 172.21.15.98:0/2270224718 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fe3c80a5e10 con 0x561839e7a440 2024-09-19T08:42:39.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3d9163000 10 client.6330 fetch_fsmap learned FSMap version 38 2024-09-19T08:42:39.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3d9163000 1 -- 172.21.15.98:0/2270224718 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) -- 0x561839c0ca30 con 0x561839e7a440 2024-09-19T08:42:39.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3ae7fc640 1 -- 172.21.15.98:0/2270224718 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3c005a110 con 0x561839e7a440 2024-09-19T08:42:39.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3ae7fc640 1 -- 172.21.15.98:0/2270224718 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 38) ==== 3502+0+0 (secure 0 0 0) 0x7fe3c8008040 con 0x561839e7a440 2024-09-19T08:42:39.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3d9163000 10 client.6330 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T08:42:39.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3d9163000 10 client.6330 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T08:42:39.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3d9163000 20 client.6330 populate_metadata read hostname 'smithi098' 2024-09-19T08:42:39.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3d9163000 1 --2- 172.21.15.98:0/2270224718 >> [v2:172.21.15.98:6835/1767881083,v1:172.21.15.98:6837/1767881083] conn(0x561839ea93e0 0x561839eecde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:39.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3d9163000 4 client.6330 mds_command: new command op to 6171 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T08:42:39.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3d9163000 1 -- 172.21.15.98:0/2270224718 --> [v2:172.21.15.98:6835/1767881083,v1:172.21.15.98:6837/1767881083] -- command(tid 0: {"prefix": "status"}) -- 0x561839c8c3a0 con 0x561839ea93e0 2024-09-19T08:42:39.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3cf7fe640 1 --2- 172.21.15.98:0/2270224718 >> [v2:172.21.15.98:6835/1767881083,v1:172.21.15.98:6837/1767881083] conn(0x561839ea93e0 0x561839eecde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:39.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3ae7fc640 10 client.6330 ms_handle_connect on v2:172.21.15.98:6835/1767881083 2024-09-19T08:42:39.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3cf7fe640 1 --2- 172.21.15.98:0/2270224718 >> [v2:172.21.15.98:6835/1767881083,v1:172.21.15.98:6837/1767881083] conn(0x561839ea93e0 0x561839eecde0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87d1201c45e3f2f3 server_cookie=b431db6099ae916b in_seq=0 out_seq=0 2024-09-19T08:42:39.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3ae7fc640 1 -- 172.21.15.98:0/2270224718 <== mds.0 v2:172.21.15.98:6835/1767881083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x561839c8c3a0 con 0x561839ea93e0 2024-09-19T08:42:39.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3ae7fc640 10 client.6330 handle_command_reply: tid=0 2024-09-19T08:42:39.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3d9163000 1 client.6330 shutdown 2024-09-19T08:42:39.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3d9163000 1 -- 172.21.15.98:0/2270224718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561839e7a440 msgr2=0x561839ea8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:39.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3d9163000 1 --2- 172.21.15.98:0/2270224718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561839e7a440 0x561839ea8ea0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800c090 tx=0x7fe3c801b0c0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3d9163000 1 -- 172.21.15.98:0/2270224718 shutdown_connections 2024-09-19T08:42:39.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3d9163000 1 --2- 172.21.15.98:0/2270224718 >> [v2:172.21.15.98:6835/1767881083,v1:172.21.15.98:6837/1767881083] conn(0x561839ea93e0 0x561839eecde0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3d9163000 1 --2- 172.21.15.98:0/2270224718 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561839e7a440 0x561839ea8ea0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3ae7fc640 0 client.6330 ms_handle_reset on v2:172.21.15.98:6835/1767881083 2024-09-19T08:42:39.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3d9163000 1 --2- 172.21.15.98:0/2270224718 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561839e79b20 0x561839ea8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3d9163000 1 --2- 172.21.15.98:0/2270224718 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561839e35390 0x561839e8d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3d9163000 1 -- 172.21.15.98:0/2270224718 >> 172.21.15.98:0/2270224718 conn(0x561839e30290 msgr2=0x561839e77c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:39.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3d9163000 1 -- 172.21.15.98:0/2270224718 shutdown_connections 2024-09-19T08:42:39.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3d9163000 1 -- 172.21.15.98:0/2270224718 wait complete. 2024-09-19T08:42:39.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3d9163000 20 client.6330 trim_cache size 0 max 16384 2024-09-19T08:42:39.552 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:42:39.552 INFO:teuthology.orchestra.run.smithi098.stdout: "cluster_fsid": "552bf0ee-1f4f-4644-ba7c-d0931e55a451", 2024-09-19T08:42:39.552 INFO:teuthology.orchestra.run.smithi098.stdout: "whoami": 0, 2024-09-19T08:42:39.552 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 6171, 2024-09-19T08:42:39.552 INFO:teuthology.orchestra.run.smithi098.stdout: "want_state": "up:active", 2024-09-19T08:42:39.553 INFO:teuthology.orchestra.run.smithi098.stdout: "state": "up:active", 2024-09-19T08:42:39.553 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "cephfs", 2024-09-19T08:42:39.553 INFO:teuthology.orchestra.run.smithi098.stdout: "rank_uptime": 9.1323210479999997, 2024-09-19T08:42:39.553 INFO:teuthology.orchestra.run.smithi098.stdout: "mdsmap_epoch": 38, 2024-09-19T08:42:39.553 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch": 68, 2024-09-19T08:42:39.553 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch_barrier": 68, 2024-09-19T08:42:39.553 INFO:teuthology.orchestra.run.smithi098.stdout: "uptime": 14.183110055 2024-09-19T08:42:39.553 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:42:39.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3d60a7640 1 -- 172.21.15.98:0/1409613012 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe3bc067c50 msgr2=0x7fe3bc06a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:39.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3d60a7640 1 --2- 172.21.15.98:0/1409613012 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe3bc067c50 0x7fe3bc06a110 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4001ec0 tx=0x7fe3c400b040 comp rx=0 tx=0).stop 2024-09-19T08:42:39.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3d60a7640 1 -- 172.21.15.98:0/1409613012 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d006c1d0 msgr2=0x7fe3d0143720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:39.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3d60a7640 1 --2- 172.21.15.98:0/1409613012 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d006c1d0 0x7fe3d0143720 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fe3c000c7f0 tx=0x7fe3c000ccb0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3d60a7640 1 -- 172.21.15.98:0/1409613012 shutdown_connections 2024-09-19T08:42:39.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3d60a7640 1 --2- 172.21.15.98:0/1409613012 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe3bc067c50 0x7fe3bc06a110 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3d60a7640 1 --2- 172.21.15.98:0/1409613012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe3d015c550 0x7fe3d0150ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3d60a7640 1 --2- 172.21.15.98:0/1409613012 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe3d00cfad0 0x7fe3d01403e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3d60a7640 1 --2- 172.21.15.98:0/1409613012 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe3d006c1d0 0x7fe3d0143720 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3d60a7640 1 -- 172.21.15.98:0/1409613012 >> 172.21.15.98:0/1409613012 conn(0x7fe3d00b2010 msgr2=0x7fe3d00cd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:39.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3d60a7640 1 -- 172.21.15.98:0/1409613012 shutdown_connections 2024-09-19T08:42:39.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.545+0000 7fe3d60a7640 1 -- 172.21.15.98:0/1409613012 wait complete. 2024-09-19T08:42:39.562 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T08:42:39.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.669+0000 7f607d801640 1 Processor -- start 2024-09-19T08:42:39.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.673+0000 7f607d801640 1 -- start start 2024-09-19T08:42:39.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.673+0000 7f607d801640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f607806bc90 0x7f607806c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:39.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.673+0000 7f607d801640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f60780ce840 0x7f60780cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:39.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.673+0000 7f607d801640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60780ccc60 con 0x7f6078158fd0 2024-09-19T08:42:39.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.673+0000 7f607d801640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f60781593b0 con 0x7f607806bc90 2024-09-19T08:42:39.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.673+0000 7f607d801640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f60780ccf40 con 0x7f60780ce840 2024-09-19T08:42:39.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.673+0000 7f6076ffd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f6078158fd0 0x7f60780b3e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:39766/0 (socket says 172.21.15.98:39766) 2024-09-19T08:42:39.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.673+0000 7f6076ffd640 1 -- 172.21.15.98:0/3456679095 learned_addr learned my addr 172.21.15.98:0/3456679095 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:39.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.673+0000 7f606e7fc640 1 --2- 172.21.15.98:0/3456679095 >> v2:172.21.15.98:3300/0 conn(0x7f607806bc90 0x7f607806c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:39.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.673+0000 7f60777fe640 1 --2- 172.21.15.98:0/3456679095 >> v2:172.21.15.139:3300/0 conn(0x7f60780ce840 0x7f60780cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:39.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.673+0000 7f60767fc640 1 -- 172.21.15.98:0/3456679095 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 511765659 0 0) 0x7f60780ccc60 con 0x7f6078158fd0 2024-09-19T08:42:39.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.673+0000 7f60767fc640 1 -- 172.21.15.98:0/3456679095 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6064003530 con 0x7f6078158fd0 2024-09-19T08:42:39.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.673+0000 7f606e7fc640 1 -- 172.21.15.98:0/3456679095 >> v2:172.21.15.139:3300/0 conn(0x7f60780ce840 msgr2=0x7f60780cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:39.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.673+0000 7f606e7fc640 1 --2- 172.21.15.98:0/3456679095 >> v2:172.21.15.139:3300/0 conn(0x7f60780ce840 0x7f60780cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.673+0000 7f606e7fc640 1 -- 172.21.15.98:0/3456679095 >> v1:172.21.15.98:6789/0 conn(0x7f6078158fd0 legacy=0x7f60780b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:39.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.673+0000 7f606e7fc640 1 -- 172.21.15.98:0/3456679095 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60780cd220 con 0x7f607806bc90 2024-09-19T08:42:39.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.673+0000 7f60777fe640 1 --2- 172.21.15.98:0/3456679095 >> v2:172.21.15.139:3300/0 conn(0x7f60780ce840 0x7f60780cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:42:39.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.677+0000 7f606e7fc640 1 --2- 172.21.15.98:0/3456679095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f607806bc90 0x7f607806c120 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f60780afe30 tx=0x7f606001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91339e9ce31890de server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:39.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.677+0000 7f60767fc640 1 -- 172.21.15.98:0/3456679095 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60600b0070 con 0x7f607806bc90 2024-09-19T08:42:39.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.677+0000 7f60767fc640 1 -- 172.21.15.98:0/3456679095 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60600ab600 con 0x7f607806bc90 2024-09-19T08:42:39.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.677+0000 7f60767fc640 1 -- 172.21.15.98:0/3456679095 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60600ab9b0 con 0x7f607806bc90 2024-09-19T08:42:39.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.677+0000 7f607d801640 1 -- 172.21.15.98:0/3456679095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f607806bc90 msgr2=0x7f607806c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:39.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.677+0000 7f607d801640 1 --2- 172.21.15.98:0/3456679095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f607806bc90 0x7f607806c120 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f60780afe30 tx=0x7f606001c3f0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.677+0000 7f607d801640 1 -- 172.21.15.98:0/3456679095 shutdown_connections 2024-09-19T08:42:39.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.677+0000 7f607d801640 1 --2- 172.21.15.98:0/3456679095 >> v2:172.21.15.139:3300/0 conn(0x7f60780ce840 0x7f60780cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.677+0000 7f607d801640 1 --2- 172.21.15.98:0/3456679095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f607806bc90 0x7f607806c120 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.677+0000 7f607d801640 1 -- 172.21.15.98:0/3456679095 >> 172.21.15.98:0/3456679095 conn(0x7f60780b2010 msgr2=0x7f60780b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:39.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.677+0000 7f607d801640 1 -- 172.21.15.98:0/3456679095 shutdown_connections 2024-09-19T08:42:39.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.677+0000 7f607d801640 1 -- 172.21.15.98:0/3456679095 wait complete. 2024-09-19T08:42:39.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.677+0000 7f607d801640 1 Processor -- start 2024-09-19T08:42:39.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.677+0000 7f607d801640 1 -- start start 2024-09-19T08:42:39.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.677+0000 7f607d801640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f607806bc90 0x7f60780ac4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:39.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.677+0000 7f607d801640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60780ce840 0x7f607815f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:39.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.677+0000 7f607d801640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6078158fd0 0x7f6078144c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:39.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.677+0000 7f607d801640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f607815c920 con 0x7f607806bc90 2024-09-19T08:42:39.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.677+0000 7f607d801640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f607815c7a0 con 0x7f60780ce840 2024-09-19T08:42:39.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.677+0000 7f607d801640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6078168110 con 0x7f6078158fd0 2024-09-19T08:42:39.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.677+0000 7f6076ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f607806bc90 0x7f60780ac4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:39.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.677+0000 7f6076ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f607806bc90 0x7f60780ac4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36704/0 (socket says 172.21.15.98:36704) 2024-09-19T08:42:39.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.677+0000 7f6076ffd640 1 -- 172.21.15.98:0/2127409682 learned_addr learned my addr 172.21.15.98:0/2127409682 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:39.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.681+0000 7f606e7fc640 1 --2- 172.21.15.98:0/2127409682 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60780ce840 0x7f607815f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:39.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.681+0000 7f6076ffd640 1 -- 172.21.15.98:0/2127409682 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6078158fd0 msgr2=0x7f6078144c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:39.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.681+0000 7f6076ffd640 1 --2- 172.21.15.98:0/2127409682 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6078158fd0 0x7f6078144c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.681+0000 7f6076ffd640 1 -- 172.21.15.98:0/2127409682 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60780ce840 msgr2=0x7f607815f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:39.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.681+0000 7f6076ffd640 1 --2- 172.21.15.98:0/2127409682 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60780ce840 0x7f607815f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.681+0000 7f6076ffd640 1 -- 172.21.15.98:0/2127409682 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60780acc30 con 0x7f607806bc90 2024-09-19T08:42:39.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.681+0000 7f6076ffd640 1 --2- 172.21.15.98:0/2127409682 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f607806bc90 0x7f60780ac4c0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f606800c3c0 tx=0x7f606800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:39.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.681+0000 7f606ffff640 1 -- 172.21.15.98:0/2127409682 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6068017070 con 0x7f607806bc90 2024-09-19T08:42:39.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.681+0000 7f606ffff640 1 -- 172.21.15.98:0/2127409682 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6068005cf0 con 0x7f607806bc90 2024-09-19T08:42:39.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.681+0000 7f607d801640 1 -- 172.21.15.98:0/2127409682 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60780acf10 con 0x7f607806bc90 2024-09-19T08:42:39.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.681+0000 7f606ffff640 1 -- 172.21.15.98:0/2127409682 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60680072f0 con 0x7f607806bc90 2024-09-19T08:42:39.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.681+0000 7f607d801640 1 -- 172.21.15.98:0/2127409682 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60780ad4d0 con 0x7f607806bc90 2024-09-19T08:42:39.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.681+0000 7f607d801640 1 -- 172.21.15.98:0/2127409682 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f607806c120 con 0x7f607806bc90 2024-09-19T08:42:39.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.681+0000 7f606ffff640 1 -- 172.21.15.98:0/2127409682 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f606801a070 con 0x7f607806bc90 2024-09-19T08:42:39.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.681+0000 7f606ffff640 1 --2- 172.21.15.98:0/2127409682 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6058067c50 0x7f605806a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:39.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.681+0000 7f606ffff640 1 -- 172.21.15.98:0/2127409682 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f6068085670 con 0x7f607806bc90 2024-09-19T08:42:39.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.681+0000 7f606e7fc640 1 --2- 172.21.15.98:0/2127409682 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6058067c50 0x7f605806a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:39.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.681+0000 7f606e7fc640 1 --2- 172.21.15.98:0/2127409682 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6058067c50 0x7f605806a110 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f60600ad000 tx=0x7f6060008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:39.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.685+0000 7f606ffff640 1 -- 172.21.15.98:0/2127409682 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6068055d10 con 0x7f607806bc90 2024-09-19T08:42:39.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.885+0000 7f607d801640 1 -- 172.21.15.98:0/2127409682 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f60780b1220 con 0x7f607806bc90 2024-09-19T08:42:39.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.889+0000 7f606ffff640 1 -- 172.21.15.98:0/2127409682 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v38) ==== 53+0+73 (secure 0 0 0) 0x7f6068055610 con 0x7f607806bc90 2024-09-19T08:42:39.889 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T08:42:39.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.889+0000 7f607d801640 1 -- 172.21.15.98:0/2127409682 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6058067c50 msgr2=0x7f605806a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:39.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.889+0000 7f607d801640 1 --2- 172.21.15.98:0/2127409682 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6058067c50 0x7f605806a110 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f60600ad000 tx=0x7f6060008040 comp rx=0 tx=0).stop 2024-09-19T08:42:39.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.889+0000 7f607d801640 1 -- 172.21.15.98:0/2127409682 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f607806bc90 msgr2=0x7f60780ac4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:39.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.889+0000 7f607d801640 1 --2- 172.21.15.98:0/2127409682 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f607806bc90 0x7f60780ac4c0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f606800c3c0 tx=0x7f606800c880 comp rx=0 tx=0).stop 2024-09-19T08:42:39.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.889+0000 7f607d801640 1 -- 172.21.15.98:0/2127409682 shutdown_connections 2024-09-19T08:42:39.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.889+0000 7f607d801640 1 --2- 172.21.15.98:0/2127409682 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f6058067c50 0x7f605806a110 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.889+0000 7f607d801640 1 --2- 172.21.15.98:0/2127409682 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6078158fd0 0x7f6078144c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.889+0000 7f607d801640 1 --2- 172.21.15.98:0/2127409682 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f60780ce840 0x7f607815f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.889+0000 7f607d801640 1 --2- 172.21.15.98:0/2127409682 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f607806bc90 0x7f60780ac4c0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:39.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.889+0000 7f607d801640 1 -- 172.21.15.98:0/2127409682 >> 172.21.15.98:0/2127409682 conn(0x7f60780b2010 msgr2=0x7f60780b1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:39.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.889+0000 7f607d801640 1 -- 172.21.15.98:0/2127409682 shutdown_connections 2024-09-19T08:42:39.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:39.889+0000 7f607d801640 1 -- 172.21.15.98:0/2127409682 wait complete. 2024-09-19T08:42:39.907 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T08:42:39.907 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T08:42:39.907 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi098.front.sepia.ceph.com 2024-09-19T08:42:39.907 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T08:42:39.907 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T08:42:39.907 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T08:42:39.908 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T08:42:39.908 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T08:42:39.908 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:42:39.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:39.908 DEBUG:teuthology.orchestra.run.smithi098:> ip addr 2024-09-19T08:42:39.914 INFO:teuthology.orchestra.run.smithi098.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T08:42:39.914 INFO:teuthology.orchestra.run.smithi098.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T08:42:39.915 INFO:teuthology.orchestra.run.smithi098.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T08:42:39.915 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:42:39.915 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 ::1/128 scope host 2024-09-19T08:42:39.915 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:42:39.915 INFO:teuthology.orchestra.run.smithi098.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:42:39.915 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b8 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:42:39.915 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f0 2024-09-19T08:42:39.915 INFO:teuthology.orchestra.run.smithi098.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:42:39.915 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b9 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:42:39.915 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f1 2024-09-19T08:42:39.915 INFO:teuthology.orchestra.run.smithi098.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:42:39.916 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6a brd ff:ff:ff:ff:ff:ff 2024-09-19T08:42:39.916 INFO:teuthology.orchestra.run.smithi098.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T08:42:39.916 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6b brd ff:ff:ff:ff:ff:ff 2024-09-19T08:42:39.916 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.21.15.98/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T08:42:39.916 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft 42248sec preferred_lft 42248sec 2024-09-19T08:42:39.916 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::ec4:7aff:fe8f:d26b/64 scope link 2024-09-19T08:42:39.916 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:42:39.916 INFO:teuthology.orchestra.run.smithi098.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T08:42:39.916 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 02:42:98:dd:32:2d brd ff:ff:ff:ff:ff:ff 2024-09-19T08:42:39.916 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T08:42:39.916 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:42:39.916 INFO:teuthology.orchestra.run.smithi098.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T08:42:39.916 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-19T08:42:39.916 INFO:teuthology.orchestra.run.smithi098.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T08:42:39.917 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:42:39.917 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-19T08:42:39.917 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:42:39.917 INFO:teuthology.orchestra.run.smithi098.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:42:39.917 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:42:39.917 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-19T08:42:39.917 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:42:39.917 INFO:teuthology.orchestra.run.smithi098.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:42:39.917 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:42:39.917 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-19T08:42:39.918 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:42:39.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:39.918 DEBUG:teuthology.orchestra.run.smithi098:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T08:42:39.966 INFO:teuthology.orchestra.run.smithi098.stdout:1 2024-09-19T08:42:39.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:39.968 DEBUG:teuthology.orchestra.run.smithi098:> route 2024-09-19T08:42:40.020 INFO:teuthology.orchestra.run.smithi098.stdout:Kernel IP routing table 2024-09-19T08:42:40.020 INFO:teuthology.orchestra.run.smithi098.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T08:42:40.020 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T08:42:40.020 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T08:42:40.020 INFO:teuthology.orchestra.run.smithi098.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T08:42:40.020 INFO:teuthology.orchestra.run.smithi098.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T08:42:40.020 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T08:42:40.020 INFO:teuthology.orchestra.run.smithi098.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:42:40.020 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:42:40.021 INFO:teuthology.orchestra.run.smithi098.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:42:40.021 INFO:teuthology.orchestra.run.smithi098.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T08:42:40.021 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:42:40.021 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-09-19T08:42:40.021 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T08:42:40.021 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T08:42:40.021 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T08:42:40.021 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-09-19T08:42:40.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:42:40.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:42:40.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:40.145 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2024-09-19T08:42:40.150 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-19T08:42:40.151 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T08:42:40.151 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:42:40.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:40.151 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set brx.0 up 2024-09-19T08:42:40.203 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-19T08:42:40.204 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:42:40.204 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T08:42:40.257 INFO:teuthology.orchestra.run.smithi098.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:42:40.257 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:42:40.257 DEBUG:teuthology.orchestra.run.smithi098:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T08:42:40.304 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:42:40.362 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:42:40.409 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:42:40.410 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:42:40.410 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:42:40.410 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:42:40.410 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:42:40.410 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:42:40.410 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:42:40.410 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:42:40.410 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:42:40.410 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:42:40.410 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:42:40.410 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:42:40.410 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:42:40.410 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:42:40.411 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:42:40.411 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:42:40.411 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:42:40.411 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:42:40.411 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:42:40.411 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:42:40.411 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:42:40.411 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:42:40.411 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:42:40.411 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:42:40.411 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:42:40.411 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:42:40.411 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:42:40.411 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:42:40.411 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:42:40.412 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:42:40.412 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:42:40.412 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:42:40.412 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:42:40.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:40.413 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:42:40.465 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T08:42:40.465 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-19T08:42:40.510 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:42:40.525 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:42:40.573 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:42:40.573 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:42:40.573 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:42:40.573 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:42:40.573 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:42:40.573 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:42:40.573 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:42:40.573 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:42:40.574 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:42:40.574 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:42:40.574 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:42:40.574 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:42:40.574 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:42:40.574 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:42:40.574 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:42:40.574 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:42:40.574 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:42:40.574 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:42:40.574 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:42:40.574 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:42:40.574 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:42:40.575 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:42:40.575 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:42:40.575 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:42:40.575 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:42:40.575 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:42:40.575 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:42:40.575 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:42:40.575 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:42:40.575 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:42:40.575 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:42:40.575 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:42:40.575 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:42:40.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:40.576 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:42:40.590 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:2024-09-19T08:42:40.589+0000 7f1b91204580 -1 init, newargv = 0x55e8d20c6230 newargc=15 2024-09-19T08:42:40.590 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:ceph-fuse[42929]: starting ceph client 2024-09-19T08:42:40.595 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:ceph-fuse[42929]: starting fuse 2024-09-19T08:42:41.696 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:42:41.711 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:42:41.759 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:42:41.759 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:42:41.759 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:42:41.760 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:42:41.760 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:42:41.760 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:42:41.760 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:42:41.760 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:42:41.760 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:42:41.760 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:42:41.760 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:42:41.760 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:42:41.760 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:42:41.760 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:42:41.760 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:42:41.760 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:42:41.761 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:42:41.761 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:42:41.761 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:42:41.761 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:42:41.761 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:42:41.761 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:42:41.761 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:42:41.761 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:42:41.761 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:42:41.761 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:42:41.761 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:42:41.761 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:42:41.761 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:42:41.762 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:42:41.762 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:42:41.762 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:42:41.762 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:42:41.762 INFO:teuthology.orchestra.run.smithi098.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-19T08:42:41.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:41.763 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:42:41.809 INFO:teuthology.orchestra.run.smithi098.stdout:45 2024-09-19T08:42:41.809 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-19T08:42:41.809 DEBUG:teuthology.orchestra.run.smithi098:> sudo stdin-killer -- python3 -c ' 2024-09-19T08:42:41.809 DEBUG:teuthology.orchestra.run.smithi098:> import glob 2024-09-19T08:42:41.809 DEBUG:teuthology.orchestra.run.smithi098:> import re 2024-09-19T08:42:41.809 DEBUG:teuthology.orchestra.run.smithi098:> import os 2024-09-19T08:42:41.810 DEBUG:teuthology.orchestra.run.smithi098:> import subprocess 2024-09-19T08:42:41.810 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:42:41.810 DEBUG:teuthology.orchestra.run.smithi098:> def _find_admin_socket(client_name): 2024-09-19T08:42:41.810 DEBUG:teuthology.orchestra.run.smithi098:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T08:42:41.810 DEBUG:teuthology.orchestra.run.smithi098:> files = glob.glob(asok_path) 2024-09-19T08:42:41.810 DEBUG:teuthology.orchestra.run.smithi098:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T08:42:41.810 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:42:41.810 DEBUG:teuthology.orchestra.run.smithi098:> # Given a non-glob path, it better be there 2024-09-19T08:42:41.810 DEBUG:teuthology.orchestra.run.smithi098:> if "*" not in asok_path: 2024-09-19T08:42:41.810 DEBUG:teuthology.orchestra.run.smithi098:> assert(len(files) == 1) 2024-09-19T08:42:41.810 DEBUG:teuthology.orchestra.run.smithi098:> return files[0] 2024-09-19T08:42:41.810 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:42:41.810 DEBUG:teuthology.orchestra.run.smithi098:> for f in files: 2024-09-19T08:42:41.810 DEBUG:teuthology.orchestra.run.smithi098:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T08:42:41.811 DEBUG:teuthology.orchestra.run.smithi098:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T08:42:41.811 DEBUG:teuthology.orchestra.run.smithi098:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T08:42:41.811 DEBUG:teuthology.orchestra.run.smithi098:> contents = proc_f.read() 2024-09-19T08:42:41.811 DEBUG:teuthology.orchestra.run.smithi098:> if mountpoint in contents: 2024-09-19T08:42:41.811 DEBUG:teuthology.orchestra.run.smithi098:> return f 2024-09-19T08:42:41.811 DEBUG:teuthology.orchestra.run.smithi098:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T08:42:41.811 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:42:41.811 DEBUG:teuthology.orchestra.run.smithi098:> print(_find_admin_socket("client.0")) 2024-09-19T08:42:41.811 DEBUG:teuthology.orchestra.run.smithi098:> ' 2024-09-19T08:42:41.934 INFO:teuthology.orchestra.run.smithi098.stdout:/var/run/ceph/ceph-client.0.42929.asok 2024-09-19T08:42:41.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:42:41.946 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.42929.asok 2024-09-19T08:42:41.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:41.946 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.42929.asok status 2024-09-19T08:42:42.095 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:42:42.095 INFO:teuthology.orchestra.run.smithi098.stdout: "metadata": { 2024-09-19T08:42:42.096 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T08:42:42.096 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T08:42:42.096 INFO:teuthology.orchestra.run.smithi098.stdout: "entity_id": "0", 2024-09-19T08:42:42.096 INFO:teuthology.orchestra.run.smithi098.stdout: "hostname": "smithi098", 2024-09-19T08:42:42.096 INFO:teuthology.orchestra.run.smithi098.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T08:42:42.096 INFO:teuthology.orchestra.run.smithi098.stdout: "pid": "42929", 2024-09-19T08:42:42.096 INFO:teuthology.orchestra.run.smithi098.stdout: "root": "/" 2024-09-19T08:42:42.096 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:42:42.096 INFO:teuthology.orchestra.run.smithi098.stdout: "dentry_count": 0, 2024-09-19T08:42:42.096 INFO:teuthology.orchestra.run.smithi098.stdout: "dentry_pinned_count": 0, 2024-09-19T08:42:42.096 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 6345, 2024-09-19T08:42:42.096 INFO:teuthology.orchestra.run.smithi098.stdout: "inst": { 2024-09-19T08:42:42.097 INFO:teuthology.orchestra.run.smithi098.stdout: "name": { 2024-09-19T08:42:42.097 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "client", 2024-09-19T08:42:42.097 INFO:teuthology.orchestra.run.smithi098.stdout: "num": 6345 2024-09-19T08:42:42.097 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:42:42.097 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": { 2024-09-19T08:42:42.097 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v1", 2024-09-19T08:42:42.097 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:42:42.097 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 1862922580 2024-09-19T08:42:42.097 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:42:42.097 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:42:42.097 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": { 2024-09-19T08:42:42.097 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v1", 2024-09-19T08:42:42.097 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:42:42.097 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 1862922580 2024-09-19T08:42:42.098 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:42:42.098 INFO:teuthology.orchestra.run.smithi098.stdout: "inst_str": "client.6345 192.168.0.1:0/1862922580", 2024-09-19T08:42:42.098 INFO:teuthology.orchestra.run.smithi098.stdout: "addr_str": "192.168.0.1:0/1862922580", 2024-09-19T08:42:42.098 INFO:teuthology.orchestra.run.smithi098.stdout: "inode_count": 1, 2024-09-19T08:42:42.098 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_epoch": 38, 2024-09-19T08:42:42.098 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_epoch": 68, 2024-09-19T08:42:42.098 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_epoch_barrier": 0, 2024-09-19T08:42:42.098 INFO:teuthology.orchestra.run.smithi098.stdout: "blocklisted": false, 2024-09-19T08:42:42.098 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "cephfs" 2024-09-19T08:42:42.098 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:42:42.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:42.110 DEBUG:teuthology.orchestra.run.smithi098:> stat --file-system '--printf=%T 2024-09-19T08:42:42.110 DEBUG:teuthology.orchestra.run.smithi098:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T08:42:42.116 INFO:teuthology.orchestra.run.smithi098.stdout:fuseblk 2024-09-19T08:42:42.116 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T08:42:42.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:42.116 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T08:42:42.173 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T08:42:42.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.333+0000 7fb12f570640 1 Processor -- start 2024-09-19T08:42:42.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.333+0000 7fb12f570640 1 -- start start 2024-09-19T08:42:42.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.337+0000 7fb12f570640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1280b7520 0x7fb1280b7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:42.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.337+0000 7fb12f570640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1280b99a0 con 0x7fb1280b6200 2024-09-19T08:42:42.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.337+0000 7fb12f570640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1280b7fe0 con 0x7fb128147770 2024-09-19T08:42:42.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.337+0000 7fb12f570640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb1280b85a0 con 0x7fb1280b7520 2024-09-19T08:42:42.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.337+0000 7fb12db0d640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fb1280b6200 0x7fb12815ebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:39772/0 (socket says 172.21.15.98:39772) 2024-09-19T08:42:42.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.337+0000 7fb12db0d640 1 -- 172.21.15.98:0/4083575984 learned_addr learned my addr 172.21.15.98:0/4083575984 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:42.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.337+0000 7fb12cb0b640 1 --2- 172.21.15.98:0/4083575984 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1280b7520 0x7fb1280b7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:42.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.337+0000 7fb11ffff640 1 -- 172.21.15.98:0/4083575984 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1866387017 0 0) 0x7fb1280b99a0 con 0x7fb1280b6200 2024-09-19T08:42:42.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.337+0000 7fb11ffff640 1 -- 172.21.15.98:0/4083575984 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb114003530 con 0x7fb1280b6200 2024-09-19T08:42:42.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.337+0000 7fb11ffff640 1 -- 172.21.15.98:0/4083575984 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1504878080 0 0) 0x7fb114003530 con 0x7fb1280b6200 2024-09-19T08:42:42.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.337+0000 7fb11ffff640 1 -- 172.21.15.98:0/4083575984 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1280b7520 msgr2=0x7fb1280b7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:42.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.337+0000 7fb11ffff640 1 --2- 172.21.15.98:0/4083575984 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1280b7520 0x7fb1280b7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:42.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.337+0000 7fb11ffff640 1 -- 172.21.15.98:0/4083575984 >> v1:172.21.15.139:6789/0 conn(0x7fb128147770 legacy=0x7fb1280b9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:42.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.337+0000 7fb11ffff640 1 -- 172.21.15.98:0/4083575984 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1280b82c0 con 0x7fb1280b6200 2024-09-19T08:42:42.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.337+0000 7fb11ffff640 1 -- 172.21.15.98:0/4083575984 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fb120001a00 con 0x7fb1280b6200 2024-09-19T08:42:42.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.337+0000 7fb11ffff640 1 -- 172.21.15.98:0/4083575984 >> v1:172.21.15.98:6789/0 conn(0x7fb1280b6200 legacy=0x7fb12815ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:42.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.337+0000 7fb11ffff640 1 --2- 172.21.15.98:0/4083575984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1140039f0 0x7fb114003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:42.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.337+0000 7fb11ffff640 1 --2- 172.21.15.98:0/4083575984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb114004810 0x7fb114004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:42.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.337+0000 7fb11ffff640 1 --2- 172.21.15.98:0/4083575984 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1280b7520 0x7fb114005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:42.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.337+0000 7fb11ffff640 1 -- 172.21.15.98:0/4083575984 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb1280b85a0 con 0x7fb1140039f0 2024-09-19T08:42:42.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.337+0000 7fb11ffff640 1 -- 172.21.15.98:0/4083575984 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb11400bb00 con 0x7fb114004810 2024-09-19T08:42:42.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.337+0000 7fb11ffff640 1 -- 172.21.15.98:0/4083575984 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb11400bd00 con 0x7fb1280b7520 2024-09-19T08:42:42.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.337+0000 7fb12d30c640 1 --2- 172.21.15.98:0/4083575984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1140039f0 0x7fb114003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:42.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.337+0000 7fb12cb0b640 1 --2- 172.21.15.98:0/4083575984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb114004810 0x7fb114004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:42.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.337+0000 7fb12d30c640 1 -- 172.21.15.98:0/4083575984 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1280b7520 msgr2=0x7fb114005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:42.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.337+0000 7fb12d30c640 1 --2- 172.21.15.98:0/4083575984 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1280b7520 0x7fb114005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:42.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.337+0000 7fb12d30c640 1 -- 172.21.15.98:0/4083575984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb114004810 msgr2=0x7fb114004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:42.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.337+0000 7fb12d30c640 1 --2- 172.21.15.98:0/4083575984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb114004810 0x7fb114004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:42.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.337+0000 7fb12d30c640 1 -- 172.21.15.98:0/4083575984 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb114003530 con 0x7fb1140039f0 2024-09-19T08:42:42.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.341+0000 7fb12d30c640 1 --2- 172.21.15.98:0/4083575984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1140039f0 0x7fb114003e10 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fb124001780 tx=0x7fb12401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b9bd8823ad9c862 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:42.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.341+0000 7fb11ffff640 1 -- 172.21.15.98:0/4083575984 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1240013a0 con 0x7fb1140039f0 2024-09-19T08:42:42.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.341+0000 7fb11ffff640 1 -- 172.21.15.98:0/4083575984 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1240052f0 con 0x7fb1140039f0 2024-09-19T08:42:42.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.341+0000 7fb11ffff640 1 -- 172.21.15.98:0/4083575984 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb124003840 con 0x7fb1140039f0 2024-09-19T08:42:42.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.341+0000 7fb12f570640 1 -- 172.21.15.98:0/4083575984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1140039f0 msgr2=0x7fb114003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:42.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.341+0000 7fb12f570640 1 --2- 172.21.15.98:0/4083575984 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1140039f0 0x7fb114003e10 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fb124001780 tx=0x7fb12401c3f0 comp rx=0 tx=0).stop 2024-09-19T08:42:42.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.341+0000 7fb12db0d640 1 -- 172.21.15.98:0/4083575984 reap_dead start 2024-09-19T08:42:42.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.341+0000 7fb12f570640 1 -- 172.21.15.98:0/4083575984 shutdown_connections 2024-09-19T08:42:42.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.341+0000 7fb12f570640 1 -- 172.21.15.98:0/4083575984 >> 172.21.15.98:0/4083575984 conn(0x7fb128074e40 msgr2=0x7fb1280722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:42.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.341+0000 7fb12f570640 1 -- 172.21.15.98:0/4083575984 shutdown_connections 2024-09-19T08:42:42.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.341+0000 7fb12f570640 1 -- 172.21.15.98:0/4083575984 wait complete. 2024-09-19T08:42:42.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.341+0000 7fb12f570640 1 Processor -- start 2024-09-19T08:42:42.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.341+0000 7fb12f570640 1 -- start start 2024-09-19T08:42:42.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.341+0000 7fb12f570640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1140039f0 0x7fb128141a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:42.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.341+0000 7fb12f570640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb128161480 0x7fb1281618e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:42.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.341+0000 7fb12f570640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb128162110 0x7fb128162590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:42.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.341+0000 7fb12f570640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb128147b50 con 0x7fb128161480 2024-09-19T08:42:42.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.341+0000 7fb12f570640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb1281482a0 con 0x7fb1140039f0 2024-09-19T08:42:42.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.341+0000 7fb12f570640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb128147cd0 con 0x7fb128162110 2024-09-19T08:42:42.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.341+0000 7fb12cb0b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb128161480 0x7fb1281618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:42.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.341+0000 7fb12cb0b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb128161480 0x7fb1281618e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36714/0 (socket says 172.21.15.98:36714) 2024-09-19T08:42:42.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.341+0000 7fb12cb0b640 1 -- 172.21.15.98:0/2262110471 learned_addr learned my addr 172.21.15.98:0/2262110471 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:42.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.341+0000 7fb12d30c640 1 --2- 172.21.15.98:0/2262110471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1140039f0 0x7fb128141a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:42.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.341+0000 7fb12db0d640 1 --2- 172.21.15.98:0/2262110471 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb128162110 0x7fb128162590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:42.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.341+0000 7fb12cb0b640 1 -- 172.21.15.98:0/2262110471 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb128162110 msgr2=0x7fb128162590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:42.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.341+0000 7fb12cb0b640 1 --2- 172.21.15.98:0/2262110471 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb128162110 0x7fb128162590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:42.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.341+0000 7fb12cb0b640 1 -- 172.21.15.98:0/2262110471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1140039f0 msgr2=0x7fb128141a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:42.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.341+0000 7fb12cb0b640 1 --2- 172.21.15.98:0/2262110471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1140039f0 0x7fb128141a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:42.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.341+0000 7fb12cb0b640 1 -- 172.21.15.98:0/2262110471 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb128162e20 con 0x7fb128161480 2024-09-19T08:42:42.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.341+0000 7fb12cb0b640 1 --2- 172.21.15.98:0/2262110471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb128161480 0x7fb1281618e0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fb1180115e0 tx=0x7fb118011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:42.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.341+0000 7fb11dffb640 1 -- 172.21.15.98:0/2262110471 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb118015870 con 0x7fb128161480 2024-09-19T08:42:42.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.341+0000 7fb11dffb640 1 -- 172.21.15.98:0/2262110471 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb118015a10 con 0x7fb128161480 2024-09-19T08:42:42.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.341+0000 7fb12f570640 1 -- 172.21.15.98:0/2262110471 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb128163100 con 0x7fb128161480 2024-09-19T08:42:42.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.341+0000 7fb11dffb640 1 -- 172.21.15.98:0/2262110471 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb118015dc0 con 0x7fb128161480 2024-09-19T08:42:42.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.341+0000 7fb12f570640 1 -- 172.21.15.98:0/2262110471 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb12815ba20 con 0x7fb128161480 2024-09-19T08:42:42.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.345+0000 7fb12f570640 1 -- 172.21.15.98:0/2262110471 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1280b6630 con 0x7fb128161480 2024-09-19T08:42:42.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.345+0000 7fb11dffb640 1 -- 172.21.15.98:0/2262110471 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fb11800f480 con 0x7fb128161480 2024-09-19T08:42:42.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.345+0000 7fb11dffb640 1 --2- 172.21.15.98:0/2262110471 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb10c067c50 0x7fb10c06a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:42.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.345+0000 7fb11dffb640 1 -- 172.21.15.98:0/2262110471 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7fb118086a50 con 0x7fb128161480 2024-09-19T08:42:42.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.345+0000 7fb12d30c640 1 --2- 172.21.15.98:0/2262110471 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb10c067c50 0x7fb10c06a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:42.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.345+0000 7fb12d30c640 1 --2- 172.21.15.98:0/2262110471 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb10c067c50 0x7fb10c06a110 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb124001750 tx=0x7fb1240011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:42.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.349+0000 7fb11dffb640 1 -- 172.21.15.98:0/2262110471 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb118019030 con 0x7fb128161480 2024-09-19T08:42:42.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.537+0000 7fb12f570640 1 -- 172.21.15.98:0/2262110471 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb1280d3790 con 0x7fb128161480 2024-09-19T08:42:42.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.541+0000 7fb11dffb640 1 -- 172.21.15.98:0/2262110471 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v38) ==== 53+0+73 (secure 0 0 0) 0x7fb118057180 con 0x7fb128161480 2024-09-19T08:42:42.541 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T08:42:42.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.541+0000 7fb12f570640 1 -- 172.21.15.98:0/2262110471 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb10c067c50 msgr2=0x7fb10c06a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:42.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.541+0000 7fb12f570640 1 --2- 172.21.15.98:0/2262110471 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb10c067c50 0x7fb10c06a110 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb124001750 tx=0x7fb1240011a0 comp rx=0 tx=0).stop 2024-09-19T08:42:42.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.541+0000 7fb12f570640 1 -- 172.21.15.98:0/2262110471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb128161480 msgr2=0x7fb1281618e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:42.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.541+0000 7fb12f570640 1 --2- 172.21.15.98:0/2262110471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb128161480 0x7fb1281618e0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fb1180115e0 tx=0x7fb118011aa0 comp rx=0 tx=0).stop 2024-09-19T08:42:42.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.541+0000 7fb12f570640 1 -- 172.21.15.98:0/2262110471 shutdown_connections 2024-09-19T08:42:42.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.541+0000 7fb12f570640 1 --2- 172.21.15.98:0/2262110471 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb10c067c50 0x7fb10c06a110 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:42.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.541+0000 7fb12f570640 1 --2- 172.21.15.98:0/2262110471 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb128162110 0x7fb128162590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:42.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.541+0000 7fb12f570640 1 --2- 172.21.15.98:0/2262110471 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb128161480 0x7fb1281618e0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:42.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.541+0000 7fb12f570640 1 --2- 172.21.15.98:0/2262110471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1140039f0 0x7fb128141a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:42.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.541+0000 7fb12f570640 1 -- 172.21.15.98:0/2262110471 >> 172.21.15.98:0/2262110471 conn(0x7fb128074e40 msgr2=0x7fb128067a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:42.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.541+0000 7fb12f570640 1 -- 172.21.15.98:0/2262110471 shutdown_connections 2024-09-19T08:42:42.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:42.541+0000 7fb12f570640 1 -- 172.21.15.98:0/2262110471 wait complete. 2024-09-19T08:42:42.558 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T08:42:42.558 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T08:42:42.558 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2024-09-19T08:42:42.558 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-19T08:42:42.559 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-19T08:42:42.559 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T08:42:42.559 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T08:42:42.559 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T08:42:42.559 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T08:42:42.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:42.559 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2024-09-19T08:42:42.566 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T08:42:42.566 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T08:42:42.566 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T08:42:42.566 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:42:42.566 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-09-19T08:42:42.566 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:42:42.567 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:42:42.567 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:42:42.567 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2024-09-19T08:42:42.567 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:42:42.567 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:42:42.567 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2024-09-19T08:42:42.567 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:42:42.567 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:42:42.567 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T08:42:42.567 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:42:42.567 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T08:42:42.567 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 42257sec preferred_lft 42257sec 2024-09-19T08:42:42.567 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::ec4:7aff:fe88:7f59/64 scope link 2024-09-19T08:42:42.568 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:42:42.568 INFO:teuthology.orchestra.run.smithi139.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T08:42:42.568 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 02:42:74:ed:e4:60 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:42:42.568 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T08:42:42.568 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:42:42.568 INFO:teuthology.orchestra.run.smithi139.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T08:42:42.568 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-19T08:42:42.568 INFO:teuthology.orchestra.run.smithi139.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T08:42:42.568 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:42:42.568 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-19T08:42:42.568 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:42:42.568 INFO:teuthology.orchestra.run.smithi139.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:42:42.568 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:42:42.568 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-19T08:42:42.569 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:42:42.569 INFO:teuthology.orchestra.run.smithi139.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:42:42.569 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:42:42.569 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-19T08:42:42.569 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:42:42.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:42.569 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T08:42:42.621 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-09-19T08:42:42.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:42.623 DEBUG:teuthology.orchestra.run.smithi139:> route 2024-09-19T08:42:42.674 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2024-09-19T08:42:42.675 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T08:42:42.675 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T08:42:42.675 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T08:42:42.675 INFO:teuthology.orchestra.run.smithi139.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T08:42:42.675 INFO:teuthology.orchestra.run.smithi139.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T08:42:42.675 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T08:42:42.675 INFO:teuthology.orchestra.run.smithi139.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:42:42.675 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:42:42.675 INFO:teuthology.orchestra.run.smithi139.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:42:42.675 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T08:42:42.675 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:42:42.676 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-09-19T08:42:42.676 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T08:42:42.676 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T08:42:42.676 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T08:42:42.676 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-09-19T08:42:42.763 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:42:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:42:42.791 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:42:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:42:42.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:42.799 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-09-19T08:42:42.804 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-19T08:42:42.804 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-19T08:42:42.804 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T08:42:42.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:42.804 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.0 up 2024-09-19T08:42:42.857 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-19T08:42:42.857 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:42:42.857 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-19T08:42:42.906 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-19T08:42:42.906 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:42:42.906 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-19T08:42:42.953 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:42:43.012 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:42:43.063 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:42:43.063 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:42:43.063 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:42:43.063 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:42:43.063 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:42:43.063 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:42:43.063 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:42:43.063 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:42:43.063 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:42:43.063 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:42:43.063 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:42:43.064 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:42:43.064 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:42:43.064 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:42:43.064 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:42:43.064 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:42:43.064 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:42:43.064 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:42:43.064 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:42:43.064 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:42:43.064 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:42:43.064 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:42:43.064 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:42:43.064 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:42:43.064 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:42:43.065 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:42:43.065 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:42:43.065 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:42:43.065 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:42:43.065 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:42:43.065 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:42:43.065 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:42:43.065 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:42:43.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:43.066 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:42:43.113 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T08:42:43.114 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-19T08:42:43.159 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:42:43.175 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:42:43.226 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:42:43.226 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:42:43.226 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:42:43.226 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:42:43.226 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:42:43.226 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:42:43.226 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:42:43.226 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:42:43.227 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:42:43.227 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:42:43.227 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:42:43.227 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:42:43.227 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:42:43.227 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:42:43.227 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:42:43.227 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:42:43.227 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:42:43.227 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:42:43.227 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:42:43.228 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:42:43.228 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:42:43.228 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:42:43.228 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:42:43.228 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:42:43.228 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:42:43.228 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:42:43.228 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:42:43.228 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:42:43.228 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:42:43.228 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:42:43.229 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:42:43.229 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:42:43.229 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:42:43.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:43.230 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:42:43.239 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:2024-09-19T08:42:43.236+0000 7fd01cb84580 -1 init, newargv = 0x5620d890af60 newargc=15 2024-09-19T08:42:43.239 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[35728]: starting ceph client 2024-09-19T08:42:43.247 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[35728]: starting fuse 2024-09-19T08:42:44.344 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:42:44.360 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:42:44.406 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:42:44.407 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:42:44.407 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:42:44.407 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:42:44.407 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:42:44.407 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:42:44.407 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:42:44.407 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:42:44.407 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:42:44.407 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:42:44.407 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:42:44.407 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:42:44.407 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:42:44.407 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:42:44.408 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:42:44.408 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:42:44.408 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:42:44.408 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:42:44.408 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:42:44.408 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:42:44.408 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:42:44.408 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:42:44.408 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:42:44.408 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:42:44.408 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:42:44.408 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:42:44.408 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:42:44.409 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:42:44.409 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:42:44.409 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:42:44.409 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:42:44.409 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:42:44.409 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:42:44.409 INFO:teuthology.orchestra.run.smithi139.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-19T08:42:44.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:44.410 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:42:44.458 INFO:teuthology.orchestra.run.smithi139.stdout:45 2024-09-19T08:42:44.458 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-19T08:42:44.458 DEBUG:teuthology.orchestra.run.smithi139:> sudo stdin-killer -- python3 -c ' 2024-09-19T08:42:44.458 DEBUG:teuthology.orchestra.run.smithi139:> import glob 2024-09-19T08:42:44.459 DEBUG:teuthology.orchestra.run.smithi139:> import re 2024-09-19T08:42:44.459 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-09-19T08:42:44.459 DEBUG:teuthology.orchestra.run.smithi139:> import subprocess 2024-09-19T08:42:44.459 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:42:44.459 DEBUG:teuthology.orchestra.run.smithi139:> def _find_admin_socket(client_name): 2024-09-19T08:42:44.459 DEBUG:teuthology.orchestra.run.smithi139:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-19T08:42:44.459 DEBUG:teuthology.orchestra.run.smithi139:> files = glob.glob(asok_path) 2024-09-19T08:42:44.459 DEBUG:teuthology.orchestra.run.smithi139:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-19T08:42:44.459 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:42:44.459 DEBUG:teuthology.orchestra.run.smithi139:> # Given a non-glob path, it better be there 2024-09-19T08:42:44.459 DEBUG:teuthology.orchestra.run.smithi139:> if "*" not in asok_path: 2024-09-19T08:42:44.459 DEBUG:teuthology.orchestra.run.smithi139:> assert(len(files) == 1) 2024-09-19T08:42:44.459 DEBUG:teuthology.orchestra.run.smithi139:> return files[0] 2024-09-19T08:42:44.459 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:42:44.459 DEBUG:teuthology.orchestra.run.smithi139:> for f in files: 2024-09-19T08:42:44.460 DEBUG:teuthology.orchestra.run.smithi139:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T08:42:44.460 DEBUG:teuthology.orchestra.run.smithi139:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T08:42:44.460 DEBUG:teuthology.orchestra.run.smithi139:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T08:42:44.460 DEBUG:teuthology.orchestra.run.smithi139:> contents = proc_f.read() 2024-09-19T08:42:44.460 DEBUG:teuthology.orchestra.run.smithi139:> if mountpoint in contents: 2024-09-19T08:42:44.460 DEBUG:teuthology.orchestra.run.smithi139:> return f 2024-09-19T08:42:44.460 DEBUG:teuthology.orchestra.run.smithi139:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T08:42:44.460 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:42:44.460 DEBUG:teuthology.orchestra.run.smithi139:> print(_find_admin_socket("client.1")) 2024-09-19T08:42:44.460 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-09-19T08:42:44.584 INFO:teuthology.orchestra.run.smithi139.stdout:/var/run/ceph/ceph-client.1.35728.asok 2024-09-19T08:42:44.588 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:42:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:42:44.596 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.35728.asok 2024-09-19T08:42:44.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:44.597 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.35728.asok status 2024-09-19T08:42:44.745 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-19T08:42:44.746 INFO:teuthology.orchestra.run.smithi139.stdout: "metadata": { 2024-09-19T08:42:44.746 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T08:42:44.746 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T08:42:44.746 INFO:teuthology.orchestra.run.smithi139.stdout: "entity_id": "1", 2024-09-19T08:42:44.746 INFO:teuthology.orchestra.run.smithi139.stdout: "hostname": "smithi139", 2024-09-19T08:42:44.746 INFO:teuthology.orchestra.run.smithi139.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-19T08:42:44.746 INFO:teuthology.orchestra.run.smithi139.stdout: "pid": "35728", 2024-09-19T08:42:44.746 INFO:teuthology.orchestra.run.smithi139.stdout: "root": "/" 2024-09-19T08:42:44.746 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:42:44.746 INFO:teuthology.orchestra.run.smithi139.stdout: "dentry_count": 0, 2024-09-19T08:42:44.746 INFO:teuthology.orchestra.run.smithi139.stdout: "dentry_pinned_count": 0, 2024-09-19T08:42:44.747 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 5168, 2024-09-19T08:42:44.747 INFO:teuthology.orchestra.run.smithi139.stdout: "inst": { 2024-09-19T08:42:44.747 INFO:teuthology.orchestra.run.smithi139.stdout: "name": { 2024-09-19T08:42:44.747 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "client", 2024-09-19T08:42:44.747 INFO:teuthology.orchestra.run.smithi139.stdout: "num": 5168 2024-09-19T08:42:44.747 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:42:44.747 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": { 2024-09-19T08:42:44.747 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-19T08:42:44.747 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:42:44.747 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1475543375 2024-09-19T08:42:44.747 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-19T08:42:44.747 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:42:44.747 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": { 2024-09-19T08:42:44.747 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-19T08:42:44.748 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:42:44.748 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1475543375 2024-09-19T08:42:44.748 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:42:44.748 INFO:teuthology.orchestra.run.smithi139.stdout: "inst_str": "client.5168 192.168.0.1:0/1475543375", 2024-09-19T08:42:44.748 INFO:teuthology.orchestra.run.smithi139.stdout: "addr_str": "192.168.0.1:0/1475543375", 2024-09-19T08:42:44.748 INFO:teuthology.orchestra.run.smithi139.stdout: "inode_count": 1, 2024-09-19T08:42:44.748 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_epoch": 38, 2024-09-19T08:42:44.748 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_epoch": 68, 2024-09-19T08:42:44.748 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_epoch_barrier": 0, 2024-09-19T08:42:44.748 INFO:teuthology.orchestra.run.smithi139.stdout: "blocklisted": false, 2024-09-19T08:42:44.748 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs" 2024-09-19T08:42:44.748 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-19T08:42:44.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:44.760 DEBUG:teuthology.orchestra.run.smithi139:> stat --file-system '--printf=%T 2024-09-19T08:42:44.760 DEBUG:teuthology.orchestra.run.smithi139:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-19T08:42:44.767 INFO:teuthology.orchestra.run.smithi139.stdout:fuseblk 2024-09-19T08:42:44.767 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-19T08:42:44.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:44.767 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-19T08:42:44.820 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-19T08:42:44.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.957+0000 7f5df931e640 1 Processor -- start 2024-09-19T08:42:44.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.957+0000 7f5df931e640 1 -- start start 2024-09-19T08:42:44.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.957+0000 7f5df931e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5df40cb2c0 0x7f5df406d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:44.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.957+0000 7f5df931e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5df40b6b90 0x7f5df40c9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:44.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.957+0000 7f5df931e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5df40c83e0 con 0x7f5df4075680 2024-09-19T08:42:44.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.957+0000 7f5df931e640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f5df40c86c0 con 0x7f5df40b6b90 2024-09-19T08:42:44.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.957+0000 7f5df931e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5df40c89a0 con 0x7f5df40cb2c0 2024-09-19T08:42:44.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.957+0000 7f5df37fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5df40b6b90 0x7f5df40c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:44.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.957+0000 7f5df37fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5df40b6b90 0x7f5df40c9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36720/0 (socket says 172.21.15.98:36720) 2024-09-19T08:42:44.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.957+0000 7f5df37fe640 1 -- 172.21.15.98:0/1774040003 learned_addr learned my addr 172.21.15.98:0/1774040003 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:44.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.957+0000 7f5df2ffd640 1 --2- 172.21.15.98:0/1774040003 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5df40cb2c0 0x7f5df406d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:44.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.957+0000 7f5df37fe640 1 -- 172.21.15.98:0/1774040003 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5df40cb2c0 msgr2=0x7f5df406d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:44.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.957+0000 7f5df37fe640 1 --2- 172.21.15.98:0/1774040003 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5df40cb2c0 0x7f5df406d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:44.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.957+0000 7f5df37fe640 1 -- 172.21.15.98:0/1774040003 >> v1:172.21.15.139:6789/0 conn(0x7f5df4075680 legacy=0x7f5df40b6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:44.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.957+0000 7f5df37fe640 1 -- 172.21.15.98:0/1774040003 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df40c8c80 con 0x7f5df40b6b90 2024-09-19T08:42:44.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.961+0000 7f5df37fe640 1 --2- 172.21.15.98:0/1774040003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5df40b6b90 0x7f5df40c9480 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5de4009870 tx=0x7f5de401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=348d62e18a78b2e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:44.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.961+0000 7f5df1ffb640 1 -- 172.21.15.98:0/1774040003 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de40b0070 con 0x7f5df40b6b90 2024-09-19T08:42:44.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.961+0000 7f5df1ffb640 1 -- 172.21.15.98:0/1774040003 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5de40ab600 con 0x7f5df40b6b90 2024-09-19T08:42:44.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.961+0000 7f5df1ffb640 1 -- 172.21.15.98:0/1774040003 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de40ab9d0 con 0x7f5df40b6b90 2024-09-19T08:42:44.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.961+0000 7f5df931e640 1 -- 172.21.15.98:0/1774040003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5df40b6b90 msgr2=0x7f5df40c9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:44.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.961+0000 7f5df931e640 1 --2- 172.21.15.98:0/1774040003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5df40b6b90 0x7f5df40c9480 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5de4009870 tx=0x7f5de401c520 comp rx=0 tx=0).stop 2024-09-19T08:42:44.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.961+0000 7f5df931e640 1 -- 172.21.15.98:0/1774040003 shutdown_connections 2024-09-19T08:42:44.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.961+0000 7f5df931e640 1 --2- 172.21.15.98:0/1774040003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5df40b6b90 0x7f5df40c9480 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:44.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.961+0000 7f5df931e640 1 --2- 172.21.15.98:0/1774040003 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5df40cb2c0 0x7f5df406d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:44.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.961+0000 7f5df931e640 1 -- 172.21.15.98:0/1774040003 >> 172.21.15.98:0/1774040003 conn(0x7f5df40b2e60 msgr2=0x7f5df40b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:44.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.961+0000 7f5df931e640 1 -- 172.21.15.98:0/1774040003 shutdown_connections 2024-09-19T08:42:44.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.961+0000 7f5df931e640 1 -- 172.21.15.98:0/1774040003 wait complete. 2024-09-19T08:42:44.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.961+0000 7f5df931e640 1 Processor -- start 2024-09-19T08:42:44.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.961+0000 7f5df931e640 1 -- start start 2024-09-19T08:42:44.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.965+0000 7f5df931e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5df4075680 0x7f5df40c5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:44.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.965+0000 7f5df931e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5df40b6b90 0x7f5df40c5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:44.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.965+0000 7f5df931e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5df40cb2c0 0x7f5df4099ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:44.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.965+0000 7f5df931e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5df40ca560 con 0x7f5df40cb2c0 2024-09-19T08:42:44.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.965+0000 7f5df931e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5df40ca3e0 con 0x7f5df40b6b90 2024-09-19T08:42:44.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.965+0000 7f5df931e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5df40ca6e0 con 0x7f5df4075680 2024-09-19T08:42:44.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.965+0000 7f5df37fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5df40cb2c0 0x7f5df4099ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:44.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.965+0000 7f5df37fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5df40cb2c0 0x7f5df4099ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36724/0 (socket says 172.21.15.98:36724) 2024-09-19T08:42:44.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.965+0000 7f5df37fe640 1 -- 172.21.15.98:0/2691322749 learned_addr learned my addr 172.21.15.98:0/2691322749 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:44.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.965+0000 7f5df27fc640 1 --2- 172.21.15.98:0/2691322749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5df40b6b90 0x7f5df40c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:44.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.965+0000 7f5df2ffd640 1 --2- 172.21.15.98:0/2691322749 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5df4075680 0x7f5df40c5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:44.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.965+0000 7f5df37fe640 1 -- 172.21.15.98:0/2691322749 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5df4075680 msgr2=0x7f5df40c5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:44.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.965+0000 7f5df37fe640 1 --2- 172.21.15.98:0/2691322749 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5df4075680 0x7f5df40c5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:44.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.965+0000 7f5df37fe640 1 -- 172.21.15.98:0/2691322749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5df40b6b90 msgr2=0x7f5df40c5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:44.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.965+0000 7f5df37fe640 1 --2- 172.21.15.98:0/2691322749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5df40b6b90 0x7f5df40c5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:44.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.965+0000 7f5df37fe640 1 -- 172.21.15.98:0/2691322749 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df415c160 con 0x7f5df40cb2c0 2024-09-19T08:42:44.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.965+0000 7f5df27fc640 1 --2- 172.21.15.98:0/2691322749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5df40b6b90 0x7f5df40c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:42:44.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.965+0000 7f5df37fe640 1 --2- 172.21.15.98:0/2691322749 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5df40cb2c0 0x7f5df4099ec0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5de40ae040 tx=0x7f5de40192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:44.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.965+0000 7f5deb7fe640 1 -- 172.21.15.98:0/2691322749 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de40b0040 con 0x7f5df40cb2c0 2024-09-19T08:42:44.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.965+0000 7f5deb7fe640 1 -- 172.21.15.98:0/2691322749 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5de40b4430 con 0x7f5df40cb2c0 2024-09-19T08:42:44.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.965+0000 7f5deb7fe640 1 -- 172.21.15.98:0/2691322749 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de40b4800 con 0x7f5df40cb2c0 2024-09-19T08:42:44.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.965+0000 7f5df931e640 1 -- 172.21.15.98:0/2691322749 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df40c5c10 con 0x7f5df40cb2c0 2024-09-19T08:42:44.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.965+0000 7f5df931e640 1 -- 172.21.15.98:0/2691322749 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df415c400 con 0x7f5df40cb2c0 2024-09-19T08:42:44.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.965+0000 7f5df931e640 1 -- 172.21.15.98:0/2691322749 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5df40c5de0 con 0x7f5df40cb2c0 2024-09-19T08:42:44.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.965+0000 7f5deb7fe640 1 -- 172.21.15.98:0/2691322749 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f5de40b49a0 con 0x7f5df40cb2c0 2024-09-19T08:42:44.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.965+0000 7f5deb7fe640 1 --2- 172.21.15.98:0/2691322749 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5dd4067c20 0x7f5dd406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:44.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.965+0000 7f5df2ffd640 1 --2- 172.21.15.98:0/2691322749 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5dd4067c20 0x7f5dd406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:44.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.965+0000 7f5deb7fe640 1 -- 172.21.15.98:0/2691322749 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f5de411fd80 con 0x7f5df40cb2c0 2024-09-19T08:42:44.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.965+0000 7f5df2ffd640 1 --2- 172.21.15.98:0/2691322749 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5dd4067c20 0x7f5dd406a0e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5df4150970 tx=0x7f5dec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:44.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:44.969+0000 7f5deb7fe640 1 -- 172.21.15.98:0/2691322749 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f5de4016110 con 0x7f5df40cb2c0 2024-09-19T08:42:45.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.169+0000 7f5df931e640 1 -- 172.21.15.98:0/2691322749 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f5df40c9480 con 0x7f5df40cb2c0 2024-09-19T08:42:45.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.441+0000 7f5deb7fe640 1 -- 172.21.15.98:0/2691322749 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v39) ==== 136+0+0 (secure 0 0 0) 0x7f5de40f03f0 con 0x7f5df40cb2c0 2024-09-19T08:42:45.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.445+0000 7f5df931e640 1 -- 172.21.15.98:0/2691322749 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5dd4067c20 msgr2=0x7f5dd406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:45.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.445+0000 7f5df931e640 1 --2- 172.21.15.98:0/2691322749 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5dd4067c20 0x7f5dd406a0e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5df4150970 tx=0x7f5dec008040 comp rx=0 tx=0).stop 2024-09-19T08:42:45.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.445+0000 7f5df931e640 1 -- 172.21.15.98:0/2691322749 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5df40cb2c0 msgr2=0x7f5df4099ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:45.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.445+0000 7f5df931e640 1 --2- 172.21.15.98:0/2691322749 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5df40cb2c0 0x7f5df4099ec0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5de40ae040 tx=0x7f5de40192c0 comp rx=0 tx=0).stop 2024-09-19T08:42:45.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.445+0000 7f5df931e640 1 -- 172.21.15.98:0/2691322749 shutdown_connections 2024-09-19T08:42:45.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.445+0000 7f5df931e640 1 --2- 172.21.15.98:0/2691322749 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5dd4067c20 0x7f5dd406a0e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:45.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.445+0000 7f5df931e640 1 --2- 172.21.15.98:0/2691322749 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5df40cb2c0 0x7f5df4099ec0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:45.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.445+0000 7f5df931e640 1 --2- 172.21.15.98:0/2691322749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5df40b6b90 0x7f5df40c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:45.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.445+0000 7f5df931e640 1 --2- 172.21.15.98:0/2691322749 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5df4075680 0x7f5df40c5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:45.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.445+0000 7f5df931e640 1 -- 172.21.15.98:0/2691322749 >> 172.21.15.98:0/2691322749 conn(0x7f5df40b2e60 msgr2=0x7f5df40d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:45.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.445+0000 7f5df931e640 1 -- 172.21.15.98:0/2691322749 shutdown_connections 2024-09-19T08:42:45.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.445+0000 7f5df931e640 1 -- 172.21.15.98:0/2691322749 wait complete. 2024-09-19T08:42:45.467 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-19T08:42:45.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.589+0000 7fb866bff640 1 Processor -- start 2024-09-19T08:42:45.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.589+0000 7fb866bff640 1 -- start start 2024-09-19T08:42:45.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.589+0000 7fb866bff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb86015c550 0x7fb86006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:45.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.589+0000 7fb866bff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb86006c1d0 0x7fb8600cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:45.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.589+0000 7fb866bff640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb86006d310 con 0x7fb8600cf900 2024-09-19T08:42:45.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.589+0000 7fb866bff640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fb86006c610 con 0x7fb86015c550 2024-09-19T08:42:45.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.589+0000 7fb866bff640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb86006d5f0 con 0x7fb86006c1d0 2024-09-19T08:42:45.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.589+0000 7fb86499b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb86015c550 0x7fb86006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:45.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.589+0000 7fb86519c640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fb8600cf900 0x7fb8600ce1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:39778/0 (socket says 172.21.15.98:39778) 2024-09-19T08:42:45.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.589+0000 7fb86519c640 1 -- 172.21.15.98:0/3125953197 learned_addr learned my addr 172.21.15.98:0/3125953197 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:45.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.589+0000 7fb857fff640 1 --2- 172.21.15.98:0/3125953197 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb86006c1d0 0x7fb8600cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:45.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.589+0000 7fb86499b640 1 -- 172.21.15.98:0/3125953197 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb86006c1d0 msgr2=0x7fb8600cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:45.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.589+0000 7fb86499b640 1 --2- 172.21.15.98:0/3125953197 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb86006c1d0 0x7fb8600cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:45.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.589+0000 7fb86499b640 1 -- 172.21.15.98:0/3125953197 >> v1:172.21.15.98:6789/0 conn(0x7fb8600cf900 legacy=0x7fb8600ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:45.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.589+0000 7fb8577fe640 1 -- 172.21.15.98:0/3125953197 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1904698 0 0) 0x7fb86006d310 con 0x7fb8600cf900 2024-09-19T08:42:45.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.589+0000 7fb86499b640 1 -- 172.21.15.98:0/3125953197 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86006d8d0 con 0x7fb86015c550 2024-09-19T08:42:45.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.589+0000 7fb86499b640 1 --2- 172.21.15.98:0/3125953197 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb86015c550 0x7fb86006bc90 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb85c0097c0 tx=0x7fb85c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e5bd297c38b4c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:45.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.593+0000 7fb8577fe640 1 -- 172.21.15.98:0/3125953197 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c0b0070 con 0x7fb86015c550 2024-09-19T08:42:45.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.593+0000 7fb8577fe640 1 -- 172.21.15.98:0/3125953197 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb85c0ab600 con 0x7fb86015c550 2024-09-19T08:42:45.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.593+0000 7fb8577fe640 1 -- 172.21.15.98:0/3125953197 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c0ab9d0 con 0x7fb86015c550 2024-09-19T08:42:45.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.593+0000 7fb866bff640 1 -- 172.21.15.98:0/3125953197 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb86015c550 msgr2=0x7fb86006bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:45.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.593+0000 7fb866bff640 1 --2- 172.21.15.98:0/3125953197 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb86015c550 0x7fb86006bc90 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb85c0097c0 tx=0x7fb85c01c4a0 comp rx=0 tx=0).stop 2024-09-19T08:42:45.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.593+0000 7fb866bff640 1 -- 172.21.15.98:0/3125953197 shutdown_connections 2024-09-19T08:42:45.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.593+0000 7fb866bff640 1 --2- 172.21.15.98:0/3125953197 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb86006c1d0 0x7fb8600cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:45.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.593+0000 7fb866bff640 1 --2- 172.21.15.98:0/3125953197 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb86015c550 0x7fb86006bc90 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:45.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.593+0000 7fb866bff640 1 -- 172.21.15.98:0/3125953197 >> 172.21.15.98:0/3125953197 conn(0x7fb8600b2010 msgr2=0x7fb8600b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:45.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.593+0000 7fb866bff640 1 -- 172.21.15.98:0/3125953197 shutdown_connections 2024-09-19T08:42:45.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.593+0000 7fb866bff640 1 -- 172.21.15.98:0/3125953197 wait complete. 2024-09-19T08:42:45.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.593+0000 7fb866bff640 1 Processor -- start 2024-09-19T08:42:45.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.593+0000 7fb866bff640 1 -- start start 2024-09-19T08:42:45.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.593+0000 7fb866bff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb86006c1d0 0x7fb8600af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:45.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.593+0000 7fb866bff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8600cf900 0x7fb8601f79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:45.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.593+0000 7fb866bff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb86015c550 0x7fb860159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:45.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.593+0000 7fb866bff640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb860169230 con 0x7fb86015c550 2024-09-19T08:42:45.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.593+0000 7fb866bff640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb86015cb40 con 0x7fb8600cf900 2024-09-19T08:42:45.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.593+0000 7fb866bff640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb8601693b0 con 0x7fb86006c1d0 2024-09-19T08:42:45.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.593+0000 7fb86519c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb86015c550 0x7fb860159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:45.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.593+0000 7fb86519c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb86015c550 0x7fb860159650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36748/0 (socket says 172.21.15.98:36748) 2024-09-19T08:42:45.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.593+0000 7fb86519c640 1 -- 172.21.15.98:0/678728786 learned_addr learned my addr 172.21.15.98:0/678728786 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:45.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.593+0000 7fb86499b640 1 --2- 172.21.15.98:0/678728786 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb86006c1d0 0x7fb8600af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:45.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.593+0000 7fb857fff640 1 --2- 172.21.15.98:0/678728786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8600cf900 0x7fb8601f79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:45.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.593+0000 7fb86519c640 1 -- 172.21.15.98:0/678728786 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb86006c1d0 msgr2=0x7fb8600af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:45.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.593+0000 7fb86519c640 1 --2- 172.21.15.98:0/678728786 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb86006c1d0 0x7fb8600af2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:45.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.593+0000 7fb86519c640 1 -- 172.21.15.98:0/678728786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8600cf900 msgr2=0x7fb8601f79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:45.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.593+0000 7fb86519c640 1 --2- 172.21.15.98:0/678728786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8600cf900 0x7fb8601f79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:45.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.593+0000 7fb86519c640 1 -- 172.21.15.98:0/678728786 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86014def0 con 0x7fb86015c550 2024-09-19T08:42:45.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.593+0000 7fb857fff640 1 --2- 172.21.15.98:0/678728786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8600cf900 0x7fb8601f79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:42:45.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.593+0000 7fb86519c640 1 --2- 172.21.15.98:0/678728786 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb86015c550 0x7fb860159650 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fb858001570 tx=0x7fb85800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:45.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.593+0000 7fb8557fa640 1 -- 172.21.15.98:0/678728786 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb858018070 con 0x7fb86015c550 2024-09-19T08:42:45.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.593+0000 7fb8557fa640 1 -- 172.21.15.98:0/678728786 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb85800a640 con 0x7fb86015c550 2024-09-19T08:42:45.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.593+0000 7fb866bff640 1 -- 172.21.15.98:0/678728786 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb86014e1d0 con 0x7fb86015c550 2024-09-19T08:42:45.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.593+0000 7fb8557fa640 1 -- 172.21.15.98:0/678728786 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85800aa30 con 0x7fb86015c550 2024-09-19T08:42:45.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.593+0000 7fb866bff640 1 -- 172.21.15.98:0/678728786 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8600b3980 con 0x7fb86015c550 2024-09-19T08:42:45.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.597+0000 7fb866bff640 1 -- 172.21.15.98:0/678728786 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8600cf3c0 con 0x7fb86015c550 2024-09-19T08:42:45.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.597+0000 7fb8557fa640 1 -- 172.21.15.98:0/678728786 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fb85801b070 con 0x7fb86015c550 2024-09-19T08:42:45.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.597+0000 7fb8557fa640 1 --2- 172.21.15.98:0/678728786 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb84c067c20 0x7fb84c06a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:45.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.597+0000 7fb8557fa640 1 -- 172.21.15.98:0/678728786 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7fb858085970 con 0x7fb86015c550 2024-09-19T08:42:45.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.597+0000 7fb86499b640 1 --2- 172.21.15.98:0/678728786 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb84c067c20 0x7fb84c06a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:45.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.597+0000 7fb86499b640 1 --2- 172.21.15.98:0/678728786 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb84c067c20 0x7fb84c06a0e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb85c0ad000 tx=0x7fb85c0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:45.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.601+0000 7fb8557fa640 1 -- 172.21.15.98:0/678728786 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb858055fa0 con 0x7fb86015c550 2024-09-19T08:42:45.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.777+0000 7fb866bff640 1 -- 172.21.15.98:0/678728786 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fb860166fd0 con 0x7fb84c067c20 2024-09-19T08:42:45.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.781+0000 7fb8557fa640 1 -- 172.21.15.98:0/678728786 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fb860166fd0 con 0x7fb84c067c20 2024-09-19T08:42:45.781 INFO:teuthology.orchestra.run.smithi098.stdout:[ 2024-09-19T08:42:45.781 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:42:45.781 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "cephfs" 2024-09-19T08:42:45.781 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:42:45.781 INFO:teuthology.orchestra.run.smithi098.stdout:] 2024-09-19T08:42:45.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.781+0000 7fb866bff640 1 -- 172.21.15.98:0/678728786 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb84c067c20 msgr2=0x7fb84c06a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:45.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.781+0000 7fb866bff640 1 --2- 172.21.15.98:0/678728786 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb84c067c20 0x7fb84c06a0e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb85c0ad000 tx=0x7fb85c0a6000 comp rx=0 tx=0).stop 2024-09-19T08:42:45.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.781+0000 7fb866bff640 1 -- 172.21.15.98:0/678728786 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb86015c550 msgr2=0x7fb860159650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:45.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.781+0000 7fb866bff640 1 --2- 172.21.15.98:0/678728786 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb86015c550 0x7fb860159650 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fb858001570 tx=0x7fb85800cca0 comp rx=0 tx=0).stop 2024-09-19T08:42:45.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.781+0000 7fb866bff640 1 -- 172.21.15.98:0/678728786 shutdown_connections 2024-09-19T08:42:45.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.781+0000 7fb866bff640 1 --2- 172.21.15.98:0/678728786 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fb84c067c20 0x7fb84c06a0e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:45.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.781+0000 7fb866bff640 1 --2- 172.21.15.98:0/678728786 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb86015c550 0x7fb860159650 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:45.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.781+0000 7fb866bff640 1 --2- 172.21.15.98:0/678728786 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8600cf900 0x7fb8601f79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:45.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.781+0000 7fb866bff640 1 --2- 172.21.15.98:0/678728786 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb86006c1d0 0x7fb8600af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:45.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.781+0000 7fb866bff640 1 -- 172.21.15.98:0/678728786 >> 172.21.15.98:0/678728786 conn(0x7fb8600b2010 msgr2=0x7fb8600b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:45.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.781+0000 7fb866bff640 1 -- 172.21.15.98:0/678728786 shutdown_connections 2024-09-19T08:42:45.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.781+0000 7fb866bff640 1 -- 172.21.15.98:0/678728786 wait complete. 2024-09-19T08:42:45.797 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T08:42:45.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.909+0000 7f3482b1b640 1 Processor -- start 2024-09-19T08:42:45.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.909+0000 7f3482b1b640 1 -- start start 2024-09-19T08:42:45.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.909+0000 7f3482b1b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f347c06c3a0 0x7f347c0713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:45.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.909+0000 7f3482b1b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f347c0718f0 0x7f347c0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:45.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.909+0000 7f3482b1b640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f347c073530 con 0x7f347c145b70 2024-09-19T08:42:45.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.909+0000 7f3482b1b640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f347c073810 con 0x7f347c06c3a0 2024-09-19T08:42:45.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.909+0000 7f3482b1b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f347c073af0 con 0x7f347c0718f0 2024-09-19T08:42:45.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.909+0000 7f347bfff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f347c06c3a0 0x7f347c0713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:45.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.909+0000 7f34808b7640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f347c145b70 0x7f347c06bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:39790/0 (socket says 172.21.15.98:39790) 2024-09-19T08:42:45.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.909+0000 7f34808b7640 1 -- 172.21.15.98:0/2388368102 learned_addr learned my addr 172.21.15.98:0/2388368102 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:45.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.909+0000 7f34810b8640 1 --2- 172.21.15.98:0/2388368102 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f347c0718f0 0x7f347c0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:45.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.909+0000 7f347bfff640 1 -- 172.21.15.98:0/2388368102 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f347c0718f0 msgr2=0x7f347c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:45.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.909+0000 7f347bfff640 1 --2- 172.21.15.98:0/2388368102 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f347c0718f0 0x7f347c0b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:45.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.909+0000 7f347bfff640 1 -- 172.21.15.98:0/2388368102 >> v1:172.21.15.98:6789/0 conn(0x7f347c145b70 legacy=0x7f347c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:45.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.909+0000 7f347bfff640 1 -- 172.21.15.98:0/2388368102 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347c073dd0 con 0x7f347c06c3a0 2024-09-19T08:42:45.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.913+0000 7f347bfff640 1 --2- 172.21.15.98:0/2388368102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f347c06c3a0 0x7f347c0713b0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f34700097c0 tx=0x7f347001c2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c49fbc5293a4d44 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:45.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.913+0000 7f347b7fe640 1 -- 172.21.15.98:0/2388368102 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34700b0070 con 0x7f347c06c3a0 2024-09-19T08:42:45.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.913+0000 7f347b7fe640 1 -- 172.21.15.98:0/2388368102 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34700ab600 con 0x7f347c06c3a0 2024-09-19T08:42:45.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.913+0000 7f347b7fe640 1 -- 172.21.15.98:0/2388368102 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34700ab9d0 con 0x7f347c06c3a0 2024-09-19T08:42:45.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.913+0000 7f3482b1b640 1 -- 172.21.15.98:0/2388368102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f347c06c3a0 msgr2=0x7f347c0713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:45.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.913+0000 7f3482b1b640 1 --2- 172.21.15.98:0/2388368102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f347c06c3a0 0x7f347c0713b0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f34700097c0 tx=0x7f347001c2a0 comp rx=0 tx=0).stop 2024-09-19T08:42:45.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.913+0000 7f3482b1b640 1 -- 172.21.15.98:0/2388368102 shutdown_connections 2024-09-19T08:42:45.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.913+0000 7f3482b1b640 1 --2- 172.21.15.98:0/2388368102 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f347c0718f0 0x7f347c0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:45.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.913+0000 7f3482b1b640 1 --2- 172.21.15.98:0/2388368102 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f347c06c3a0 0x7f347c0713b0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:45.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.913+0000 7f3482b1b640 1 -- 172.21.15.98:0/2388368102 >> 172.21.15.98:0/2388368102 conn(0x7f347c0b2010 msgr2=0x7f347c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:45.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.913+0000 7f3482b1b640 1 -- 172.21.15.98:0/2388368102 shutdown_connections 2024-09-19T08:42:45.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.913+0000 7f3482b1b640 1 -- 172.21.15.98:0/2388368102 wait complete. 2024-09-19T08:42:45.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.913+0000 7f3482b1b640 1 Processor -- start 2024-09-19T08:42:45.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.913+0000 7f3482b1b640 1 -- start start 2024-09-19T08:42:45.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.913+0000 7f3482b1b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f347c06c3a0 0x7f347c13b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:45.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.913+0000 7f3482b1b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f347c0718f0 0x7f347c140d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:45.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.913+0000 7f3482b1b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f347c145b70 0x7f347c1412b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:45.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.913+0000 7f3482b1b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f347c0b44f0 con 0x7f347c0718f0 2024-09-19T08:42:45.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.913+0000 7f3482b1b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f347c0b4370 con 0x7f347c145b70 2024-09-19T08:42:45.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.913+0000 7f3482b1b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f347c0b4670 con 0x7f347c06c3a0 2024-09-19T08:42:45.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.913+0000 7f347bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f347c0718f0 0x7f347c140d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:45.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.913+0000 7f347bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f347c0718f0 0x7f347c140d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36772/0 (socket says 172.21.15.98:36772) 2024-09-19T08:42:45.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.913+0000 7f347bfff640 1 -- 172.21.15.98:0/1547990701 learned_addr learned my addr 172.21.15.98:0/1547990701 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:45.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.913+0000 7f347bfff640 1 -- 172.21.15.98:0/1547990701 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f347c06c3a0 msgr2=0x7f347c13b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:45.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.913+0000 7f347bfff640 1 --2- 172.21.15.98:0/1547990701 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f347c06c3a0 0x7f347c13b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:45.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.913+0000 7f347bfff640 1 -- 172.21.15.98:0/1547990701 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f347c145b70 msgr2=0x7f347c1412b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:45.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.913+0000 7f347bfff640 1 --2- 172.21.15.98:0/1547990701 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f347c145b70 0x7f347c1412b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:45.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.913+0000 7f347bfff640 1 -- 172.21.15.98:0/1547990701 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347c0cdea0 con 0x7f347c0718f0 2024-09-19T08:42:45.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.913+0000 7f347bfff640 1 --2- 172.21.15.98:0/1547990701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f347c0718f0 0x7f347c140d70 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f347001a950 tx=0x7f3470019040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:45.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.913+0000 7f34797fa640 1 -- 172.21.15.98:0/1547990701 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34700b8020 con 0x7f347c0718f0 2024-09-19T08:42:45.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.913+0000 7f34797fa640 1 -- 172.21.15.98:0/1547990701 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34700b4630 con 0x7f347c0718f0 2024-09-19T08:42:45.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.913+0000 7f3482b1b640 1 -- 172.21.15.98:0/1547990701 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f347c0ce180 con 0x7f347c0718f0 2024-09-19T08:42:45.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.913+0000 7f34797fa640 1 -- 172.21.15.98:0/1547990701 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34700b0040 con 0x7f347c0718f0 2024-09-19T08:42:45.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.913+0000 7f3482b1b640 1 -- 172.21.15.98:0/1547990701 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f347c0ce740 con 0x7f347c0718f0 2024-09-19T08:42:45.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.917+0000 7f3482b1b640 1 -- 172.21.15.98:0/1547990701 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f347c166e40 con 0x7f347c0718f0 2024-09-19T08:42:45.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.917+0000 7f34797fa640 1 -- 172.21.15.98:0/1547990701 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f34700bb020 con 0x7f347c0718f0 2024-09-19T08:42:45.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.917+0000 7f34797fa640 1 --2- 172.21.15.98:0/1547990701 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3468067c20 0x7f346806a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:45.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.917+0000 7f34808b7640 1 --2- 172.21.15.98:0/1547990701 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3468067c20 0x7f346806a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:45.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.917+0000 7f34797fa640 1 -- 172.21.15.98:0/1547990701 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f34701200a0 con 0x7f347c0718f0 2024-09-19T08:42:45.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.917+0000 7f34808b7640 1 --2- 172.21.15.98:0/1547990701 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3468067c20 0x7f346806a0e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f347c0b15b0 tx=0x7f346c0079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:45.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:45.921+0000 7f34797fa640 1 -- 172.21.15.98:0/1547990701 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f34700c2030 con 0x7f347c0718f0 2024-09-19T08:42:46.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.073+0000 7f3482b1b640 1 -- 172.21.15.98:0/1547990701 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f347c0b00a0 con 0x7f347c0718f0 2024-09-19T08:42:46.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.073+0000 7f34797fa640 1 -- 172.21.15.98:0/1547990701 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f3470015fb0 con 0x7f347c0718f0 2024-09-19T08:42:46.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.073+0000 7f3482b1b640 1 -- 172.21.15.98:0/1547990701 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3468067c20 msgr2=0x7f346806a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:46.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.073+0000 7f3482b1b640 1 --2- 172.21.15.98:0/1547990701 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3468067c20 0x7f346806a0e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f347c0b15b0 tx=0x7f346c0079b0 comp rx=0 tx=0).stop 2024-09-19T08:42:46.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.073+0000 7f3482b1b640 1 -- 172.21.15.98:0/1547990701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f347c0718f0 msgr2=0x7f347c140d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:46.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.073+0000 7f3482b1b640 1 --2- 172.21.15.98:0/1547990701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f347c0718f0 0x7f347c140d70 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f347001a950 tx=0x7f3470019040 comp rx=0 tx=0).stop 2024-09-19T08:42:46.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.073+0000 7f3482b1b640 1 -- 172.21.15.98:0/1547990701 shutdown_connections 2024-09-19T08:42:46.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.073+0000 7f3482b1b640 1 --2- 172.21.15.98:0/1547990701 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3468067c20 0x7f346806a0e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:46.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.073+0000 7f3482b1b640 1 --2- 172.21.15.98:0/1547990701 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f347c145b70 0x7f347c1412b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:46.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.073+0000 7f3482b1b640 1 --2- 172.21.15.98:0/1547990701 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f347c0718f0 0x7f347c140d70 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:46.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.073+0000 7f3482b1b640 1 --2- 172.21.15.98:0/1547990701 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f347c06c3a0 0x7f347c13b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:46.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.073+0000 7f3482b1b640 1 -- 172.21.15.98:0/1547990701 >> 172.21.15.98:0/1547990701 conn(0x7f347c0b2010 msgr2=0x7f347c166590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:46.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.073+0000 7f3482b1b640 1 -- 172.21.15.98:0/1547990701 shutdown_connections 2024-09-19T08:42:46.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.073+0000 7f3482b1b640 1 -- 172.21.15.98:0/1547990701 wait complete. 2024-09-19T08:42:46.091 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:42:46.091 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:42:46.096 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T08:42:46.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.257+0000 7f1a6791e640 1 Processor -- start 2024-09-19T08:42:46.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.257+0000 7f1a6791e640 1 -- start start 2024-09-19T08:42:46.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.257+0000 7f1a6791e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f1a6006c3a0 0x7f1a600cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:46.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.257+0000 7f1a6791e640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a6006d310 con 0x7f1a6015c550 2024-09-19T08:42:46.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.257+0000 7f1a6791e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a6006d5f0 con 0x7f1a600cf900 2024-09-19T08:42:46.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.257+0000 7f1a6791e640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f1a6006d8d0 con 0x7f1a6006c3a0 2024-09-19T08:42:46.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.257+0000 7f1a64eb9640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f1a6006c3a0 0x7f1a600cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:46.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.257+0000 7f1a64eb9640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f1a6006c3a0 0x7f1a600cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36776/0 (socket says 172.21.15.98:36776) 2024-09-19T08:42:46.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.257+0000 7f1a64eb9640 1 -- 172.21.15.98:0/4044790340 learned_addr learned my addr 172.21.15.98:0/4044790340 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:46.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.257+0000 7f1a656ba640 1 --1- 172.21.15.98:0/4044790340 >> v1:172.21.15.98:6789/0 conn(0x7f1a6015c550 0x7f1a6006bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:39800/0 (socket says 172.21.15.98:39800) 2024-09-19T08:42:46.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.257+0000 7f1a64eb9640 1 -- 172.21.15.98:0/4044790340 >> v1:172.21.15.139:6789/0 conn(0x7f1a600cf900 legacy=0x7f1a600723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:46.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.257+0000 7f1a64eb9640 1 -- 172.21.15.98:0/4044790340 >> v1:172.21.15.98:6789/0 conn(0x7f1a6015c550 legacy=0x7f1a6006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:46.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.257+0000 7f1a57fff640 1 -- 172.21.15.98:0/4044790340 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2721997267 0 0) 0x7f1a6006d310 con 0x7f1a6015c550 2024-09-19T08:42:46.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.257+0000 7f1a64eb9640 1 -- 172.21.15.98:0/4044790340 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a6006dbb0 con 0x7f1a6006c3a0 2024-09-19T08:42:46.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.261+0000 7f1a64eb9640 1 --2- 172.21.15.98:0/4044790340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a6006c3a0 0x7f1a600cf3c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f1a50009870 tx=0x7f1a5001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab52579fa8a6af87 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:46.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.261+0000 7f1a57fff640 1 -- 172.21.15.98:0/4044790340 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a500ab430 con 0x7f1a6006c3a0 2024-09-19T08:42:46.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.261+0000 7f1a57fff640 1 -- 172.21.15.98:0/4044790340 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a500b0070 con 0x7f1a6006c3a0 2024-09-19T08:42:46.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.261+0000 7f1a57fff640 1 -- 172.21.15.98:0/4044790340 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a500ab7f0 con 0x7f1a6006c3a0 2024-09-19T08:42:46.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.261+0000 7f1a6791e640 1 -- 172.21.15.98:0/4044790340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a6006c3a0 msgr2=0x7f1a600cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:46.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.261+0000 7f1a6791e640 1 --2- 172.21.15.98:0/4044790340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a6006c3a0 0x7f1a600cf3c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f1a50009870 tx=0x7f1a5001c520 comp rx=0 tx=0).stop 2024-09-19T08:42:46.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.261+0000 7f1a6791e640 1 -- 172.21.15.98:0/4044790340 shutdown_connections 2024-09-19T08:42:46.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.261+0000 7f1a6791e640 1 --2- 172.21.15.98:0/4044790340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a6006c3a0 0x7f1a600cf3c0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:46.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.261+0000 7f1a6791e640 1 -- 172.21.15.98:0/4044790340 >> 172.21.15.98:0/4044790340 conn(0x7f1a600b2010 msgr2=0x7f1a600b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:46.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.261+0000 7f1a6791e640 1 -- 172.21.15.98:0/4044790340 shutdown_connections 2024-09-19T08:42:46.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.261+0000 7f1a6791e640 1 -- 172.21.15.98:0/4044790340 wait complete. 2024-09-19T08:42:46.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.265+0000 7f1a6791e640 1 Processor -- start 2024-09-19T08:42:46.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.265+0000 7f1a6791e640 1 -- start start 2024-09-19T08:42:46.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.265+0000 7f1a6791e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a6006c3a0 0x7f1a6014a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:46.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.265+0000 7f1a6791e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1a600cf900 0x7f1a60156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:46.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.265+0000 7f1a6791e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a6015c550 0x7f1a60156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:46.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.265+0000 7f1a6791e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1a600736d0 con 0x7f1a6015c550 2024-09-19T08:42:46.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.265+0000 7f1a6791e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1a60073550 con 0x7f1a6006c3a0 2024-09-19T08:42:46.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.265+0000 7f1a6791e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1a6016a240 con 0x7f1a600cf900 2024-09-19T08:42:46.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.265+0000 7f1a65ebb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a6015c550 0x7f1a60156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:46.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.265+0000 7f1a65ebb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a6015c550 0x7f1a60156b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36790/0 (socket says 172.21.15.98:36790) 2024-09-19T08:42:46.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.265+0000 7f1a65ebb640 1 -- 172.21.15.98:0/4294525201 learned_addr learned my addr 172.21.15.98:0/4294525201 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:46.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.265+0000 7f1a64eb9640 1 --2- 172.21.15.98:0/4294525201 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1a600cf900 0x7f1a60156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:46.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.265+0000 7f1a656ba640 1 --2- 172.21.15.98:0/4294525201 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a6006c3a0 0x7f1a6014a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:46.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.265+0000 7f1a65ebb640 1 -- 172.21.15.98:0/4294525201 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1a600cf900 msgr2=0x7f1a60156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:46.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.265+0000 7f1a65ebb640 1 --2- 172.21.15.98:0/4294525201 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1a600cf900 0x7f1a60156620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:46.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.265+0000 7f1a65ebb640 1 -- 172.21.15.98:0/4294525201 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a6006c3a0 msgr2=0x7f1a6014a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:46.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.265+0000 7f1a65ebb640 1 --2- 172.21.15.98:0/4294525201 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a6006c3a0 0x7f1a6014a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:46.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.265+0000 7f1a65ebb640 1 -- 172.21.15.98:0/4294525201 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a60157150 con 0x7f1a6015c550 2024-09-19T08:42:46.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.265+0000 7f1a64eb9640 1 --2- 172.21.15.98:0/4294525201 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1a600cf900 0x7f1a60156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:42:46.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.265+0000 7f1a65ebb640 1 --2- 172.21.15.98:0/4294525201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a6015c550 0x7f1a60156b60 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800c7b0 tx=0x7f1a5800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:46.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.265+0000 7f1a55ffb640 1 -- 172.21.15.98:0/4294525201 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a58004b70 con 0x7f1a6015c550 2024-09-19T08:42:46.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.265+0000 7f1a55ffb640 1 -- 172.21.15.98:0/4294525201 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a58004d10 con 0x7f1a6015c550 2024-09-19T08:42:46.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.265+0000 7f1a6791e640 1 -- 172.21.15.98:0/4294525201 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a6014a700 con 0x7f1a6015c550 2024-09-19T08:42:46.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.265+0000 7f1a55ffb640 1 -- 172.21.15.98:0/4294525201 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5800aa90 con 0x7f1a6015c550 2024-09-19T08:42:46.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.265+0000 7f1a6791e640 1 -- 172.21.15.98:0/4294525201 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a6014acc0 con 0x7f1a6015c550 2024-09-19T08:42:46.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.265+0000 7f1a6791e640 1 -- 172.21.15.98:0/4294525201 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a600cf3c0 con 0x7f1a6015c550 2024-09-19T08:42:46.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.265+0000 7f1a55ffb640 1 -- 172.21.15.98:0/4294525201 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f1a5800ac30 con 0x7f1a6015c550 2024-09-19T08:42:46.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.265+0000 7f1a55ffb640 1 --2- 172.21.15.98:0/4294525201 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f1a44067c50 0x7f1a4406a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:46.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.265+0000 7f1a55ffb640 1 -- 172.21.15.98:0/4294525201 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f1a58086460 con 0x7f1a6015c550 2024-09-19T08:42:46.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.269+0000 7f1a656ba640 1 --2- 172.21.15.98:0/4294525201 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f1a44067c50 0x7f1a4406a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:46.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.269+0000 7f1a656ba640 1 --2- 172.21.15.98:0/4294525201 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f1a44067c50 0x7f1a4406a110 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c000f70 tx=0x7f1a5c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:46.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.273+0000 7f1a55ffb640 1 -- 172.21.15.98:0/4294525201 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f1a58020070 con 0x7f1a6015c550 2024-09-19T08:42:46.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.437+0000 7f1a6791e640 1 -- 172.21.15.98:0/4294525201 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1a6014fca0 con 0x7f1a6015c550 2024-09-19T08:42:46.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.437+0000 7f1a55ffb640 1 -- 172.21.15.98:0/4294525201 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v68) ==== 80+0+0 (secure 0 0 0) 0x7f1a58056b20 con 0x7f1a6015c550 2024-09-19T08:42:46.438 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-19T08:42:46.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.437+0000 7f1a6791e640 1 -- 172.21.15.98:0/4294525201 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f1a44067c50 msgr2=0x7f1a4406a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:46.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.437+0000 7f1a6791e640 1 --2- 172.21.15.98:0/4294525201 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f1a44067c50 0x7f1a4406a110 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c000f70 tx=0x7f1a5c009040 comp rx=0 tx=0).stop 2024-09-19T08:42:46.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.437+0000 7f1a6791e640 1 -- 172.21.15.98:0/4294525201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a6015c550 msgr2=0x7f1a60156b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:46.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.437+0000 7f1a6791e640 1 --2- 172.21.15.98:0/4294525201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a6015c550 0x7f1a60156b60 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800c7b0 tx=0x7f1a5800cc70 comp rx=0 tx=0).stop 2024-09-19T08:42:46.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.437+0000 7f1a6791e640 1 -- 172.21.15.98:0/4294525201 shutdown_connections 2024-09-19T08:42:46.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.437+0000 7f1a6791e640 1 --2- 172.21.15.98:0/4294525201 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f1a44067c50 0x7f1a4406a110 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:46.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.437+0000 7f1a6791e640 1 --2- 172.21.15.98:0/4294525201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a6015c550 0x7f1a60156b60 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:46.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.437+0000 7f1a6791e640 1 --2- 172.21.15.98:0/4294525201 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1a600cf900 0x7f1a60156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:46.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.437+0000 7f1a6791e640 1 --2- 172.21.15.98:0/4294525201 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a6006c3a0 0x7f1a6014a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:46.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.437+0000 7f1a6791e640 1 -- 172.21.15.98:0/4294525201 >> 172.21.15.98:0/4294525201 conn(0x7f1a600b2010 msgr2=0x7f1a601504a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:46.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.441+0000 7f1a6791e640 1 -- 172.21.15.98:0/4294525201 shutdown_connections 2024-09-19T08:42:46.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.441+0000 7f1a6791e640 1 -- 172.21.15.98:0/4294525201 wait complete. 2024-09-19T08:42:46.457 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:42:46.457 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:42:46.462 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T08:42:46.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.621+0000 7f39256f0640 1 Processor -- start 2024-09-19T08:42:46.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.621+0000 7f39256f0640 1 -- start start 2024-09-19T08:42:46.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.621+0000 7f39256f0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f39200cec60 0x7f392015b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:46.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.621+0000 7f39256f0640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39200af4a0 con 0x7f392006f380 2024-09-19T08:42:46.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.621+0000 7f39256f0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39200af780 con 0x7f392006efa0 2024-09-19T08:42:46.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.621+0000 7f39256f0640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f392015ed00 con 0x7f39200cec60 2024-09-19T08:42:46.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.621+0000 7f391e7fc640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f392006f380 0x7f39200ae8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:39806/0 (socket says 172.21.15.98:39806) 2024-09-19T08:42:46.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.621+0000 7f391e7fc640 1 -- 172.21.15.98:0/3601896685 learned_addr learned my addr 172.21.15.98:0/3601896685 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:46.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.621+0000 7f391f7fe640 1 --2- 172.21.15.98:0/3601896685 >> v2:172.21.15.139:3300/0 conn(0x7f39200cec60 0x7f392015b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:46.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.621+0000 7f391dffb640 1 -- 172.21.15.98:0/3601896685 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1644563653 0 0) 0x7f39200af4a0 con 0x7f392006f380 2024-09-19T08:42:46.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.621+0000 7f391dffb640 1 -- 172.21.15.98:0/3601896685 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f390c003530 con 0x7f392006f380 2024-09-19T08:42:46.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.621+0000 7f391dffb640 1 -- 172.21.15.98:0/3601896685 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3348338910 0 0) 0x7f390c003530 con 0x7f392006f380 2024-09-19T08:42:46.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.621+0000 7f391dffb640 1 -- 172.21.15.98:0/3601896685 >> v2:172.21.15.139:3300/0 conn(0x7f39200cec60 msgr2=0x7f392015b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:46.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.621+0000 7f391dffb640 1 --2- 172.21.15.98:0/3601896685 >> v2:172.21.15.139:3300/0 conn(0x7f39200cec60 0x7f392015b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:46.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.621+0000 7f391dffb640 1 -- 172.21.15.98:0/3601896685 >> v1:172.21.15.139:6789/0 conn(0x7f392006efa0 legacy=0x7f39200ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:46.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.621+0000 7f391dffb640 1 -- 172.21.15.98:0/3601896685 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39200afa60 con 0x7f392006f380 2024-09-19T08:42:46.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.621+0000 7f391dffb640 1 -- 172.21.15.98:0/3601896685 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f3908001a00 con 0x7f392006f380 2024-09-19T08:42:46.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.621+0000 7f391dffb640 1 -- 172.21.15.98:0/3601896685 >> v1:172.21.15.98:6789/0 conn(0x7f392006f380 legacy=0x7f39200ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:46.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.621+0000 7f391dffb640 1 --2- 172.21.15.98:0/3601896685 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f390c0039c0 0x7f390c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:46.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.621+0000 7f391dffb640 1 --2- 172.21.15.98:0/3601896685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f390c004810 0x7f390c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:46.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.621+0000 7f391dffb640 1 --2- 172.21.15.98:0/3601896685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f390c0053d0 0x7f390c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:46.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.621+0000 7f391dffb640 1 -- 172.21.15.98:0/3601896685 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f392015ed00 con 0x7f390c0053d0 2024-09-19T08:42:46.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.621+0000 7f391dffb640 1 -- 172.21.15.98:0/3601896685 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f390c00c260 con 0x7f390c004810 2024-09-19T08:42:46.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.621+0000 7f391dffb640 1 -- 172.21.15.98:0/3601896685 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f390c00c460 con 0x7f390c0039c0 2024-09-19T08:42:46.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.621+0000 7f391f7fe640 1 --2- 172.21.15.98:0/3601896685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f390c0053d0 0x7f390c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:46.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.621+0000 7f391e7fc640 1 --2- 172.21.15.98:0/3601896685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f390c004810 0x7f390c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:46.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.621+0000 7f391effd640 1 --2- 172.21.15.98:0/3601896685 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f390c0039c0 0x7f390c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:46.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.621+0000 7f391f7fe640 1 -- 172.21.15.98:0/3601896685 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f390c0039c0 msgr2=0x7f390c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:46.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.621+0000 7f391f7fe640 1 --2- 172.21.15.98:0/3601896685 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f390c0039c0 0x7f390c003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:46.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.621+0000 7f391f7fe640 1 -- 172.21.15.98:0/3601896685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f390c004810 msgr2=0x7f390c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:46.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.621+0000 7f391f7fe640 1 --2- 172.21.15.98:0/3601896685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f390c004810 0x7f390c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:46.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.621+0000 7f391f7fe640 1 -- 172.21.15.98:0/3601896685 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f390c003530 con 0x7f390c0053d0 2024-09-19T08:42:46.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.625+0000 7f391f7fe640 1 -- 172.21.15.98:0/3601896685 reap_dead start 2024-09-19T08:42:46.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.625+0000 7f391f7fe640 1 --2- 172.21.15.98:0/3601896685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f390c0053d0 0x7f390c00b9d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f391000ba30 tx=0x7f391001c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930ab973fe0259e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:46.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.625+0000 7f391dffb640 1 -- 172.21.15.98:0/3601896685 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39100b0020 con 0x7f390c0053d0 2024-09-19T08:42:46.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.625+0000 7f391dffb640 1 -- 172.21.15.98:0/3601896685 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3910004220 con 0x7f390c0053d0 2024-09-19T08:42:46.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.625+0000 7f391dffb640 1 -- 172.21.15.98:0/3601896685 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391000b600 con 0x7f390c0053d0 2024-09-19T08:42:46.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.625+0000 7f39256f0640 1 -- 172.21.15.98:0/3601896685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f390c0053d0 msgr2=0x7f390c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:46.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.625+0000 7f39256f0640 1 --2- 172.21.15.98:0/3601896685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f390c0053d0 0x7f390c00b9d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f391000ba30 tx=0x7f391001c6e0 comp rx=0 tx=0).stop 2024-09-19T08:42:46.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.625+0000 7f39256f0640 1 -- 172.21.15.98:0/3601896685 shutdown_connections 2024-09-19T08:42:46.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.625+0000 7f39256f0640 1 --2- 172.21.15.98:0/3601896685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f390c0053d0 0x7f390c00b9d0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:46.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.625+0000 7f39256f0640 1 -- 172.21.15.98:0/3601896685 >> 172.21.15.98:0/3601896685 conn(0x7f39200d0980 msgr2=0x7f39200d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:46.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.625+0000 7f39256f0640 1 -- 172.21.15.98:0/3601896685 shutdown_connections 2024-09-19T08:42:46.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.625+0000 7f39256f0640 1 -- 172.21.15.98:0/3601896685 wait complete. 2024-09-19T08:42:46.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.625+0000 7f39256f0640 1 Processor -- start 2024-09-19T08:42:46.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.625+0000 7f39256f0640 1 -- start start 2024-09-19T08:42:46.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.629+0000 7f39256f0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f390c0053d0 0x7f392014df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:46.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.629+0000 7f39256f0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3920071420 0x7f3920156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:46.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.629+0000 7f39256f0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3920157300 0x7f3920160b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:46.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.629+0000 7f39256f0640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f392015b970 con 0x7f3920157300 2024-09-19T08:42:46.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.629+0000 7f39256f0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f392015b7f0 con 0x7f390c0053d0 2024-09-19T08:42:46.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.629+0000 7f39256f0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f39200ce550 con 0x7f3920071420 2024-09-19T08:42:46.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.629+0000 7f391f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3920157300 0x7f3920160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:46.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.629+0000 7f391f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3920157300 0x7f3920160b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36808/0 (socket says 172.21.15.98:36808) 2024-09-19T08:42:46.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.629+0000 7f391f7fe640 1 -- 172.21.15.98:0/1345600127 learned_addr learned my addr 172.21.15.98:0/1345600127 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:46.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.629+0000 7f391e7fc640 1 --2- 172.21.15.98:0/1345600127 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3920071420 0x7f3920156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:46.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.629+0000 7f391effd640 1 --2- 172.21.15.98:0/1345600127 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f390c0053d0 0x7f392014df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:46.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.629+0000 7f391f7fe640 1 -- 172.21.15.98:0/1345600127 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3920071420 msgr2=0x7f3920156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:46.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.629+0000 7f391f7fe640 1 --2- 172.21.15.98:0/1345600127 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3920071420 0x7f3920156dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:46.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.629+0000 7f391f7fe640 1 -- 172.21.15.98:0/1345600127 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f390c0053d0 msgr2=0x7f392014df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:46.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.629+0000 7f391f7fe640 1 --2- 172.21.15.98:0/1345600127 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f390c0053d0 0x7f392014df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:46.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.629+0000 7f391f7fe640 1 -- 172.21.15.98:0/1345600127 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39200718b0 con 0x7f3920157300 2024-09-19T08:42:46.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.629+0000 7f391f7fe640 1 --2- 172.21.15.98:0/1345600127 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3920157300 0x7f3920160b90 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f391001cbc0 tx=0x7f390c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:46.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.629+0000 7f39177fe640 1 -- 172.21.15.98:0/1345600127 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39100b0040 con 0x7f3920157300 2024-09-19T08:42:46.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.629+0000 7f39177fe640 1 -- 172.21.15.98:0/1345600127 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39100be070 con 0x7f3920157300 2024-09-19T08:42:46.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.629+0000 7f39177fe640 1 -- 172.21.15.98:0/1345600127 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391000b600 con 0x7f3920157300 2024-09-19T08:42:46.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.629+0000 7f39256f0640 1 -- 172.21.15.98:0/1345600127 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3920071b90 con 0x7f3920157300 2024-09-19T08:42:46.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.629+0000 7f39256f0640 1 -- 172.21.15.98:0/1345600127 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3920072150 con 0x7f3920157300 2024-09-19T08:42:46.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.629+0000 7f39256f0640 1 -- 172.21.15.98:0/1345600127 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39200af3e0 con 0x7f3920157300 2024-09-19T08:42:46.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.629+0000 7f39177fe640 1 -- 172.21.15.98:0/1345600127 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f39100aa050 con 0x7f3920157300 2024-09-19T08:42:46.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.629+0000 7f39177fe640 1 --2- 172.21.15.98:0/1345600127 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3900067c20 0x7f390006a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:46.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.629+0000 7f391effd640 1 --2- 172.21.15.98:0/1345600127 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3900067c20 0x7f390006a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:46.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.629+0000 7f39177fe640 1 -- 172.21.15.98:0/1345600127 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f39101204d0 con 0x7f3920157300 2024-09-19T08:42:46.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.629+0000 7f391effd640 1 --2- 172.21.15.98:0/1345600127 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3900067c20 0x7f390006a0e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3918000f70 tx=0x7f3918009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:46.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.633+0000 7f39177fe640 1 -- 172.21.15.98:0/1345600127 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f39100163f0 con 0x7f3920157300 2024-09-19T08:42:46.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.809+0000 7f39256f0640 1 -- 172.21.15.98:0/1345600127 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f392006fee0 con 0x7f3920157300 2024-09-19T08:42:46.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.809+0000 7f39177fe640 1 -- 172.21.15.98:0/1345600127 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v68) ==== 80+0+0 (secure 0 0 0) 0x7f39100f0af0 con 0x7f3920157300 2024-09-19T08:42:46.809 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-19T08:42:46.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.809+0000 7f39256f0640 1 -- 172.21.15.98:0/1345600127 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3900067c20 msgr2=0x7f390006a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:46.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.809+0000 7f39256f0640 1 --2- 172.21.15.98:0/1345600127 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3900067c20 0x7f390006a0e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3918000f70 tx=0x7f3918009040 comp rx=0 tx=0).stop 2024-09-19T08:42:46.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.809+0000 7f39256f0640 1 -- 172.21.15.98:0/1345600127 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3920157300 msgr2=0x7f3920160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:46.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.809+0000 7f39256f0640 1 --2- 172.21.15.98:0/1345600127 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3920157300 0x7f3920160b90 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f391001cbc0 tx=0x7f390c005840 comp rx=0 tx=0).stop 2024-09-19T08:42:46.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.809+0000 7f39256f0640 1 -- 172.21.15.98:0/1345600127 shutdown_connections 2024-09-19T08:42:46.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.809+0000 7f39256f0640 1 --2- 172.21.15.98:0/1345600127 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f3900067c20 0x7f390006a0e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:46.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.809+0000 7f39256f0640 1 --2- 172.21.15.98:0/1345600127 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3920157300 0x7f3920160b90 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:46.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.809+0000 7f39256f0640 1 --2- 172.21.15.98:0/1345600127 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3920071420 0x7f3920156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:46.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.809+0000 7f39256f0640 1 --2- 172.21.15.98:0/1345600127 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f390c0053d0 0x7f392014df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:46.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.809+0000 7f39256f0640 1 -- 172.21.15.98:0/1345600127 >> 172.21.15.98:0/1345600127 conn(0x7f39200d0980 msgr2=0x7f392015c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:46.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.813+0000 7f39256f0640 1 -- 172.21.15.98:0/1345600127 shutdown_connections 2024-09-19T08:42:46.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:46.813+0000 7f39256f0640 1 -- 172.21.15.98:0/1345600127 wait complete. 2024-09-19T08:42:46.828 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi098.front.sepia.ceph.com... 2024-09-19T08:42:46.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:46.829 DEBUG:teuthology.orchestra.run.smithi098:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T08:42:46.840 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:ceph-fuse[42929]: fuse finished with error 0 and tester_r 0 2024-09-19T08:42:46.841 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:42:47.848 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:42:47.851 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:42:52.942 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:42:52.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:52.943 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T08:42:52.951 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:42:52.951 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:42:52.958 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T08:42:53.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.129+0000 7fe7b2970640 1 Processor -- start 2024-09-19T08:42:53.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.129+0000 7fe7b2970640 1 -- start start 2024-09-19T08:42:53.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.129+0000 7fe7b2970640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe7ac158fd0 0x7fe7ac0b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:53.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.129+0000 7fe7b2970640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7ac06c3a0 0x7fe7ac0ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:53.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.129+0000 7fe7b2970640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7ac0ccc60 con 0x7fe7ac0b4360 2024-09-19T08:42:53.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.129+0000 7fe7b2970640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fe7ac1593b0 con 0x7fe7ac158fd0 2024-09-19T08:42:53.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.129+0000 7fe7b2970640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe7ac0ccf40 con 0x7fe7ac06c3a0 2024-09-19T08:42:53.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.129+0000 7fe7abfff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe7ac158fd0 0x7fe7ac0b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:53.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.129+0000 7fe7abfff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe7ac158fd0 0x7fe7ac0b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48870/0 (socket says 172.21.15.98:48870) 2024-09-19T08:42:53.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.129+0000 7fe7abfff640 1 -- 172.21.15.98:0/1041869973 learned_addr learned my addr 172.21.15.98:0/1041869973 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:53.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.129+0000 7fe7abfff640 1 -- 172.21.15.98:0/1041869973 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7ac06c3a0 msgr2=0x7fe7ac0ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:53.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.129+0000 7fe7abfff640 1 --2- 172.21.15.98:0/1041869973 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7ac06c3a0 0x7fe7ac0ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:53.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.129+0000 7fe7abfff640 1 -- 172.21.15.98:0/1041869973 >> v1:172.21.15.98:6789/0 conn(0x7fe7ac0b4360 legacy=0x7fe7ac06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:53.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.129+0000 7fe7abfff640 1 -- 172.21.15.98:0/1041869973 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ac0cd220 con 0x7fe7ac158fd0 2024-09-19T08:42:53.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.133+0000 7fe7abfff640 1 --2- 172.21.15.98:0/1041869973 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7ac158fd0 0x7fe7ac0b3e20 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00bb10 tx=0x7fe79c01eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56bec4b3b1ce579c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:53.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.133+0000 7fe7aaffd640 1 -- 172.21.15.98:0/1041869973 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c0b4070 con 0x7fe7ac158fd0 2024-09-19T08:42:53.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.133+0000 7fe7aaffd640 1 -- 172.21.15.98:0/1041869973 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe79c0a7c40 con 0x7fe7ac158fd0 2024-09-19T08:42:53.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.133+0000 7fe7aaffd640 1 -- 172.21.15.98:0/1041869973 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c0af440 con 0x7fe7ac158fd0 2024-09-19T08:42:53.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.133+0000 7fe7b2970640 1 -- 172.21.15.98:0/1041869973 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7ac158fd0 msgr2=0x7fe7ac0b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:53.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.133+0000 7fe7b2970640 1 --2- 172.21.15.98:0/1041869973 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7ac158fd0 0x7fe7ac0b3e20 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00bb10 tx=0x7fe79c01eb70 comp rx=0 tx=0).stop 2024-09-19T08:42:53.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.133+0000 7fe7b2970640 1 -- 172.21.15.98:0/1041869973 shutdown_connections 2024-09-19T08:42:53.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.133+0000 7fe7b2970640 1 --2- 172.21.15.98:0/1041869973 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7ac06c3a0 0x7fe7ac0ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:53.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.133+0000 7fe7b2970640 1 --2- 172.21.15.98:0/1041869973 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7ac158fd0 0x7fe7ac0b3e20 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:53.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.133+0000 7fe7b2970640 1 -- 172.21.15.98:0/1041869973 >> 172.21.15.98:0/1041869973 conn(0x7fe7ac0b2010 msgr2=0x7fe7ac0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:53.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.133+0000 7fe7b2970640 1 -- 172.21.15.98:0/1041869973 shutdown_connections 2024-09-19T08:42:53.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.133+0000 7fe7b2970640 1 -- 172.21.15.98:0/1041869973 wait complete. 2024-09-19T08:42:53.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.137+0000 7fe7b2970640 1 Processor -- start 2024-09-19T08:42:53.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.137+0000 7fe7b2970640 1 -- start start 2024-09-19T08:42:53.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.137+0000 7fe7b2970640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7ac06c3a0 0x7fe7ac15f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:53.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.137+0000 7fe7b2970640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7ac0b4360 0x7fe7ac14bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:53.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.137+0000 7fe7b2970640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7ac158fd0 0x7fe7ac1644e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:53.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.137+0000 7fe7b2970640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ac0cf710 con 0x7fe7ac158fd0 2024-09-19T08:42:53.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.137+0000 7fe7b2970640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ac0cf590 con 0x7fe7ac0b4360 2024-09-19T08:42:53.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.137+0000 7fe7b2970640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe7ac0cf890 con 0x7fe7ac06c3a0 2024-09-19T08:42:53.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.137+0000 7fe7b0f0d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7ac158fd0 0x7fe7ac1644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:53.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.137+0000 7fe7b0f0d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7ac158fd0 0x7fe7ac1644e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48874/0 (socket says 172.21.15.98:48874) 2024-09-19T08:42:53.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.137+0000 7fe7b0f0d640 1 -- 172.21.15.98:0/1741880640 learned_addr learned my addr 172.21.15.98:0/1741880640 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:53.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.137+0000 7fe7abfff640 1 --2- 172.21.15.98:0/1741880640 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7ac06c3a0 0x7fe7ac15f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:53.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.137+0000 7fe7ab7fe640 1 --2- 172.21.15.98:0/1741880640 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7ac0b4360 0x7fe7ac14bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:53.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.137+0000 7fe7b0f0d640 1 -- 172.21.15.98:0/1741880640 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7ac06c3a0 msgr2=0x7fe7ac15f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:53.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.137+0000 7fe7b0f0d640 1 --2- 172.21.15.98:0/1741880640 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7ac06c3a0 0x7fe7ac15f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:53.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.137+0000 7fe7b0f0d640 1 -- 172.21.15.98:0/1741880640 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7ac0b4360 msgr2=0x7fe7ac14bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:53.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.137+0000 7fe7b0f0d640 1 --2- 172.21.15.98:0/1741880640 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7ac0b4360 0x7fe7ac14bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:53.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.137+0000 7fe7b0f0d640 1 -- 172.21.15.98:0/1741880640 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ac158280 con 0x7fe7ac158fd0 2024-09-19T08:42:53.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.137+0000 7fe7b0f0d640 1 --2- 172.21.15.98:0/1741880640 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7ac158fd0 0x7fe7ac1644e0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fe7a400b970 tx=0x7fe7a400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:53.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.137+0000 7fe7a8ff9640 1 -- 172.21.15.98:0/1741880640 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a4019070 con 0x7fe7ac158fd0 2024-09-19T08:42:53.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.137+0000 7fe7a8ff9640 1 -- 172.21.15.98:0/1741880640 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7a400cb60 con 0x7fe7ac158fd0 2024-09-19T08:42:53.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.137+0000 7fe7b2970640 1 -- 172.21.15.98:0/1741880640 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7ac15aee0 con 0x7fe7ac158fd0 2024-09-19T08:42:53.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.137+0000 7fe7a8ff9640 1 -- 172.21.15.98:0/1741880640 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a4014440 con 0x7fe7ac158fd0 2024-09-19T08:42:53.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.137+0000 7fe7b2970640 1 -- 172.21.15.98:0/1741880640 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7ac0ab330 con 0x7fe7ac158fd0 2024-09-19T08:42:53.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.137+0000 7fe7b2970640 1 -- 172.21.15.98:0/1741880640 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7ac0ce840 con 0x7fe7ac158fd0 2024-09-19T08:42:53.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.137+0000 7fe7a8ff9640 1 -- 172.21.15.98:0/1741880640 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fe7a401c070 con 0x7fe7ac158fd0 2024-09-19T08:42:53.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.137+0000 7fe7a8ff9640 1 --2- 172.21.15.98:0/1741880640 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe798067c20 0x7fe79806a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:53.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.137+0000 7fe7a8ff9640 1 -- 172.21.15.98:0/1741880640 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7fe7a4085f00 con 0x7fe7ac158fd0 2024-09-19T08:42:53.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.137+0000 7fe7abfff640 1 --2- 172.21.15.98:0/1741880640 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe798067c20 0x7fe79806a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:53.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.141+0000 7fe7abfff640 1 --2- 172.21.15.98:0/1741880640 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe798067c20 0x7fe79806a0e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe79c0b1000 tx=0x7fe79c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:53.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.145+0000 7fe7a8ff9640 1 -- 172.21.15.98:0/1741880640 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe7a4056630 con 0x7fe7ac158fd0 2024-09-19T08:42:53.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.317+0000 7fe7b2970640 1 -- 172.21.15.98:0/1741880640 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe7ac06bd30 con 0x7fe7ac158fd0 2024-09-19T08:42:53.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.321+0000 7fe7a8ff9640 1 -- 172.21.15.98:0/1741880640 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v68) ==== 80+0+0 (secure 0 0 0) 0x7fe7a4055f30 con 0x7fe7ac158fd0 2024-09-19T08:42:53.320 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-19T08:42:53.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.321+0000 7fe7b2970640 1 -- 172.21.15.98:0/1741880640 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe798067c20 msgr2=0x7fe79806a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:53.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.321+0000 7fe7b2970640 1 --2- 172.21.15.98:0/1741880640 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe798067c20 0x7fe79806a0e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe79c0b1000 tx=0x7fe79c008040 comp rx=0 tx=0).stop 2024-09-19T08:42:53.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.321+0000 7fe7b2970640 1 -- 172.21.15.98:0/1741880640 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7ac158fd0 msgr2=0x7fe7ac1644e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:53.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.321+0000 7fe7b2970640 1 --2- 172.21.15.98:0/1741880640 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7ac158fd0 0x7fe7ac1644e0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fe7a400b970 tx=0x7fe7a400be30 comp rx=0 tx=0).stop 2024-09-19T08:42:53.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.321+0000 7fe7b2970640 1 -- 172.21.15.98:0/1741880640 shutdown_connections 2024-09-19T08:42:53.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.321+0000 7fe7b2970640 1 --2- 172.21.15.98:0/1741880640 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fe798067c20 0x7fe79806a0e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:53.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.321+0000 7fe7b2970640 1 --2- 172.21.15.98:0/1741880640 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7ac158fd0 0x7fe7ac1644e0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:53.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.321+0000 7fe7b2970640 1 --2- 172.21.15.98:0/1741880640 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7ac0b4360 0x7fe7ac14bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:53.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.321+0000 7fe7b2970640 1 --2- 172.21.15.98:0/1741880640 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7ac06c3a0 0x7fe7ac15f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:53.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.321+0000 7fe7b2970640 1 -- 172.21.15.98:0/1741880640 >> 172.21.15.98:0/1741880640 conn(0x7fe7ac0b2010 msgr2=0x7fe7ac0b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:53.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.321+0000 7fe7b2970640 1 -- 172.21.15.98:0/1741880640 shutdown_connections 2024-09-19T08:42:53.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.321+0000 7fe7b2970640 1 -- 172.21.15.98:0/1741880640 wait complete. 2024-09-19T08:42:53.341 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:42:53.341 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:42:53.347 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T08:42:53.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.473+0000 7f9d30dda640 1 Processor -- start 2024-09-19T08:42:53.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.473+0000 7f9d30dda640 1 -- start start 2024-09-19T08:42:53.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.473+0000 7f9d30dda640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9d2c0b9970 0x7f9d2c0b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:53.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.473+0000 7f9d30dda640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d2c068440 con 0x7f9d2c0d4820 2024-09-19T08:42:53.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.473+0000 7f9d30dda640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d2c162a90 con 0x7f9d2c0b7f80 2024-09-19T08:42:53.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.473+0000 7f9d30dda640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f9d2c163050 con 0x7f9d2c0b9970 2024-09-19T08:42:53.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.473+0000 7f9d21d9b640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f9d2c0d4820 0x7f9d2c1621b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:48240/0 (socket says 172.21.15.98:48240) 2024-09-19T08:42:53.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.473+0000 7f9d21d9b640 1 -- 172.21.15.98:0/3207743183 learned_addr learned my addr 172.21.15.98:0/3207743183 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:42:53.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.473+0000 7f9d2ad9d640 1 --2- 172.21.15.98:0/3207743183 >> v2:172.21.15.98:3300/0 conn(0x7f9d2c0b9970 0x7f9d2c0b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:53.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.473+0000 7f9d29d9b640 1 -- 172.21.15.98:0/3207743183 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4246575646 0 0) 0x7f9d2c068440 con 0x7f9d2c0d4820 2024-09-19T08:42:53.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.473+0000 7f9d29d9b640 1 -- 172.21.15.98:0/3207743183 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d18003530 con 0x7f9d2c0d4820 2024-09-19T08:42:53.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.473+0000 7f9d2ad9d640 1 -- 172.21.15.98:0/3207743183 >> v1:172.21.15.139:6789/0 conn(0x7f9d2c0b7f80 legacy=0x7f9d2c15f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:53.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.473+0000 7f9d2ad9d640 1 -- 172.21.15.98:0/3207743183 >> v1:172.21.15.98:6789/0 conn(0x7f9d2c0d4820 legacy=0x7f9d2c1621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:53.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.473+0000 7f9d2ad9d640 1 -- 172.21.15.98:0/3207743183 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d2c162d70 con 0x7f9d2c0b9970 2024-09-19T08:42:53.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.477+0000 7f9d2ad9d640 1 --2- 172.21.15.98:0/3207743183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d2c0b9970 0x7f9d2c0b9de0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f9d24009d90 tx=0x7f9d2401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca9c1d82758b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:53.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.477+0000 7f9d29d9b640 1 -- 172.21.15.98:0/3207743183 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d240b2070 con 0x7f9d2c0b9970 2024-09-19T08:42:53.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.477+0000 7f9d29d9b640 1 -- 172.21.15.98:0/3207743183 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d240a5830 con 0x7f9d2c0b9970 2024-09-19T08:42:53.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.477+0000 7f9d29d9b640 1 -- 172.21.15.98:0/3207743183 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d240a5c00 con 0x7f9d2c0b9970 2024-09-19T08:42:53.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.477+0000 7f9d30dda640 1 -- 172.21.15.98:0/3207743183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d2c0b9970 msgr2=0x7f9d2c0b9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:53.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.477+0000 7f9d30dda640 1 --2- 172.21.15.98:0/3207743183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d2c0b9970 0x7f9d2c0b9de0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f9d24009d90 tx=0x7f9d2401c970 comp rx=0 tx=0).stop 2024-09-19T08:42:53.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.477+0000 7f9d30dda640 1 -- 172.21.15.98:0/3207743183 shutdown_connections 2024-09-19T08:42:53.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.477+0000 7f9d30dda640 1 --2- 172.21.15.98:0/3207743183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d2c0b9970 0x7f9d2c0b9de0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:53.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.477+0000 7f9d30dda640 1 -- 172.21.15.98:0/3207743183 >> 172.21.15.98:0/3207743183 conn(0x7f9d2c074e40 msgr2=0x7f9d2c075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:53.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.477+0000 7f9d30dda640 1 -- 172.21.15.98:0/3207743183 shutdown_connections 2024-09-19T08:42:53.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.477+0000 7f9d30dda640 1 -- 172.21.15.98:0/3207743183 wait complete. 2024-09-19T08:42:53.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.477+0000 7f9d30dda640 1 Processor -- start 2024-09-19T08:42:53.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.477+0000 7f9d30dda640 1 -- start start 2024-09-19T08:42:53.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.481+0000 7f9d30dda640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9d2c0b7f80 0x7f9d2c140c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:53.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.481+0000 7f9d30dda640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d2c0b9970 0x7f9d2c1544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:53.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.481+0000 7f9d30dda640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d2c0d4820 0x7f9d2c13e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:53.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.481+0000 7f9d30dda640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9d2c144710 con 0x7f9d2c0d4820 2024-09-19T08:42:53.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.481+0000 7f9d30dda640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9d2c144590 con 0x7f9d2c0b9970 2024-09-19T08:42:53.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.481+0000 7f9d30dda640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9d2c144890 con 0x7f9d2c0b7f80 2024-09-19T08:42:53.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.481+0000 7f9d2ad9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d2c0d4820 0x7f9d2c13e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:53.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.481+0000 7f9d2ad9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d2c0d4820 0x7f9d2c13e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48894/0 (socket says 172.21.15.98:48894) 2024-09-19T08:42:53.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.481+0000 7f9d2ad9d640 1 -- 172.21.15.98:0/2239439908 learned_addr learned my addr 172.21.15.98:0/2239439908 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:53.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.481+0000 7f9d2a59c640 1 --2- 172.21.15.98:0/2239439908 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9d2c0b7f80 0x7f9d2c140c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:53.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.481+0000 7f9d21d9b640 1 --2- 172.21.15.98:0/2239439908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d2c0b9970 0x7f9d2c1544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:53.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.481+0000 7f9d2ad9d640 1 -- 172.21.15.98:0/2239439908 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9d2c0b7f80 msgr2=0x7f9d2c140c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:53.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.481+0000 7f9d2ad9d640 1 --2- 172.21.15.98:0/2239439908 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9d2c0b7f80 0x7f9d2c140c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:53.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.481+0000 7f9d2ad9d640 1 -- 172.21.15.98:0/2239439908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d2c0b9970 msgr2=0x7f9d2c1544a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:53.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.481+0000 7f9d2ad9d640 1 --2- 172.21.15.98:0/2239439908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d2c0b9970 0x7f9d2c1544a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:53.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.481+0000 7f9d2ad9d640 1 -- 172.21.15.98:0/2239439908 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d2c138530 con 0x7f9d2c0d4820 2024-09-19T08:42:53.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.481+0000 7f9d2ad9d640 1 --2- 172.21.15.98:0/2239439908 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d2c0d4820 0x7f9d2c13e3f0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f9d240b0040 tx=0x7f9d24019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:53.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.481+0000 7f9d237fe640 1 -- 172.21.15.98:0/2239439908 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d240b2050 con 0x7f9d2c0d4820 2024-09-19T08:42:53.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.481+0000 7f9d237fe640 1 -- 172.21.15.98:0/2239439908 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d240be070 con 0x7f9d2c0d4820 2024-09-19T08:42:53.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.481+0000 7f9d30dda640 1 -- 172.21.15.98:0/2239439908 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d2c138810 con 0x7f9d2c0d4820 2024-09-19T08:42:53.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.481+0000 7f9d237fe640 1 -- 172.21.15.98:0/2239439908 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d240ad870 con 0x7f9d2c0d4820 2024-09-19T08:42:53.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.481+0000 7f9d30dda640 1 -- 172.21.15.98:0/2239439908 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d2c138dd0 con 0x7f9d2c0d4820 2024-09-19T08:42:53.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.481+0000 7f9d30dda640 1 -- 172.21.15.98:0/2239439908 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d2c0b9de0 con 0x7f9d2c0d4820 2024-09-19T08:42:53.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.481+0000 7f9d237fe640 1 -- 172.21.15.98:0/2239439908 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f9d240ada10 con 0x7f9d2c0d4820 2024-09-19T08:42:53.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.481+0000 7f9d237fe640 1 --2- 172.21.15.98:0/2239439908 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9d0c067c20 0x7f9d0c06a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:53.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.481+0000 7f9d237fe640 1 -- 172.21.15.98:0/2239439908 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f9d24121300 con 0x7f9d2c0d4820 2024-09-19T08:42:53.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.481+0000 7f9d2a59c640 1 --2- 172.21.15.98:0/2239439908 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9d0c067c20 0x7f9d0c06a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:53.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.481+0000 7f9d2a59c640 1 --2- 172.21.15.98:0/2239439908 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9d0c067c20 0x7f9d0c06a0e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9d140015b0 tx=0x7f9d14009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:53.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.485+0000 7f9d237fe640 1 -- 172.21.15.98:0/2239439908 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9d24016560 con 0x7f9d2c0d4820 2024-09-19T08:42:53.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.657+0000 7f9d30dda640 1 -- 172.21.15.98:0/2239439908 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9d2c1391f0 con 0x7f9d2c0d4820 2024-09-19T08:42:53.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.657+0000 7f9d237fe640 1 -- 172.21.15.98:0/2239439908 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v68) ==== 80+0+0 (secure 0 0 0) 0x7f9d240add00 con 0x7f9d2c0d4820 2024-09-19T08:42:53.658 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-19T08:42:53.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.661+0000 7f9d30dda640 1 -- 172.21.15.98:0/2239439908 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9d0c067c20 msgr2=0x7f9d0c06a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:53.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.661+0000 7f9d30dda640 1 --2- 172.21.15.98:0/2239439908 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9d0c067c20 0x7f9d0c06a0e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9d140015b0 tx=0x7f9d14009040 comp rx=0 tx=0).stop 2024-09-19T08:42:53.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.661+0000 7f9d30dda640 1 -- 172.21.15.98:0/2239439908 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d2c0d4820 msgr2=0x7f9d2c13e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:53.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.661+0000 7f9d30dda640 1 --2- 172.21.15.98:0/2239439908 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d2c0d4820 0x7f9d2c13e3f0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f9d240b0040 tx=0x7f9d24019710 comp rx=0 tx=0).stop 2024-09-19T08:42:53.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.661+0000 7f9d30dda640 1 -- 172.21.15.98:0/2239439908 shutdown_connections 2024-09-19T08:42:53.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.661+0000 7f9d30dda640 1 --2- 172.21.15.98:0/2239439908 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f9d0c067c20 0x7f9d0c06a0e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:53.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.661+0000 7f9d30dda640 1 --2- 172.21.15.98:0/2239439908 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d2c0d4820 0x7f9d2c13e3f0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:53.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.661+0000 7f9d30dda640 1 --2- 172.21.15.98:0/2239439908 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d2c0b9970 0x7f9d2c1544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:53.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.661+0000 7f9d30dda640 1 --2- 172.21.15.98:0/2239439908 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9d2c0b7f80 0x7f9d2c140c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:53.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.661+0000 7f9d30dda640 1 -- 172.21.15.98:0/2239439908 >> 172.21.15.98:0/2239439908 conn(0x7f9d2c074e40 msgr2=0x7f9d2c0cff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:53.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.661+0000 7f9d30dda640 1 -- 172.21.15.98:0/2239439908 shutdown_connections 2024-09-19T08:42:53.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:53.661+0000 7f9d30dda640 1 -- 172.21.15.98:0/2239439908 wait complete. 2024-09-19T08:42:53.676 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi139.front.sepia.ceph.com... 2024-09-19T08:42:53.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:53.676 DEBUG:teuthology.orchestra.run.smithi139:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-19T08:42:53.689 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[35728]: fuse finished with error 0 and tester_r 0 2024-09-19T08:42:53.690 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:42:59.791 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:42:59.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:42:59.792 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-19T08:42:59.800 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:42:59.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.929+0000 7ff2be554640 1 Processor -- start 2024-09-19T08:42:59.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.929+0000 7ff2be554640 1 -- start start 2024-09-19T08:42:59.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.933+0000 7ff2be554640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff2b815c550 0x7ff2b806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:59.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.933+0000 7ff2be554640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff2b806c1d0 0x7ff2b80cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:59.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.933+0000 7ff2be554640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2b806d310 con 0x7ff2b80cf900 2024-09-19T08:42:59.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.933+0000 7ff2be554640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7ff2b806c640 con 0x7ff2b815c550 2024-09-19T08:42:59.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.933+0000 7ff2be554640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff2b806d5f0 con 0x7ff2b806c1d0 2024-09-19T08:42:59.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.933+0000 7ff2b7fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff2b815c550 0x7ff2b806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:59.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.933+0000 7ff2b7fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff2b815c550 0x7ff2b806bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53270/0 (socket says 172.21.15.98:53270) 2024-09-19T08:42:59.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.933+0000 7ff2b7fff640 1 -- 172.21.15.98:0/218172383 learned_addr learned my addr 172.21.15.98:0/218172383 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:59.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.933+0000 7ff2b77fe640 1 --2- 172.21.15.98:0/218172383 >> v2:172.21.15.139:3300/0 conn(0x7ff2b806c1d0 0x7ff2b80cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:59.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.933+0000 7ff2b7fff640 1 -- 172.21.15.98:0/218172383 >> v2:172.21.15.139:3300/0 conn(0x7ff2b806c1d0 msgr2=0x7ff2b80cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:42:59.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.933+0000 7ff2b7fff640 1 --2- 172.21.15.98:0/218172383 >> v2:172.21.15.139:3300/0 conn(0x7ff2b806c1d0 0x7ff2b80cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:59.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.933+0000 7ff2b7fff640 1 -- 172.21.15.98:0/218172383 >> v1:172.21.15.139:6789/0 conn(0x7ff2b80cf900 legacy=0x7ff2b80ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:59.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.933+0000 7ff2b7fff640 1 -- 172.21.15.98:0/218172383 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b806d8d0 con 0x7ff2b815c550 2024-09-19T08:42:59.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.933+0000 7ff2b77fe640 1 --2- 172.21.15.98:0/218172383 >> v2:172.21.15.139:3300/0 conn(0x7ff2b806c1d0 0x7ff2b80cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:42:59.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.933+0000 7ff2b7fff640 1 --2- 172.21.15.98:0/218172383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2b815c550 0x7ff2b806bc90 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80ccc60 tx=0x7ff2a801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b42ae7ee092c182 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:59.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.933+0000 7ff2b6ffd640 1 -- 172.21.15.98:0/218172383 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a80b0070 con 0x7ff2b815c550 2024-09-19T08:42:59.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.933+0000 7ff2b6ffd640 1 -- 172.21.15.98:0/218172383 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2a80ab600 con 0x7ff2b815c550 2024-09-19T08:42:59.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.933+0000 7ff2b6ffd640 1 -- 172.21.15.98:0/218172383 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a80ab9d0 con 0x7ff2b815c550 2024-09-19T08:42:59.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.937+0000 7ff2be554640 1 -- 172.21.15.98:0/218172383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2b815c550 msgr2=0x7ff2b806bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:59.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.937+0000 7ff2be554640 1 --2- 172.21.15.98:0/218172383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2b815c550 0x7ff2b806bc90 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80ccc60 tx=0x7ff2a801c520 comp rx=0 tx=0).stop 2024-09-19T08:42:59.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.937+0000 7ff2be554640 1 -- 172.21.15.98:0/218172383 shutdown_connections 2024-09-19T08:42:59.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.937+0000 7ff2be554640 1 --2- 172.21.15.98:0/218172383 >> v2:172.21.15.139:3300/0 conn(0x7ff2b806c1d0 0x7ff2b80cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:59.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.937+0000 7ff2be554640 1 --2- 172.21.15.98:0/218172383 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2b815c550 0x7ff2b806bc90 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:59.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.937+0000 7ff2be554640 1 -- 172.21.15.98:0/218172383 >> 172.21.15.98:0/218172383 conn(0x7ff2b80b2010 msgr2=0x7ff2b80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:42:59.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.937+0000 7ff2be554640 1 -- 172.21.15.98:0/218172383 shutdown_connections 2024-09-19T08:42:59.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.937+0000 7ff2be554640 1 -- 172.21.15.98:0/218172383 wait complete. 2024-09-19T08:42:59.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.937+0000 7ff2be554640 1 Processor -- start 2024-09-19T08:42:59.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.937+0000 7ff2be554640 1 -- start start 2024-09-19T08:42:59.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.937+0000 7ff2be554640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2b806c1d0 0x7ff2b81663e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:59.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.937+0000 7ff2be554640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2b80cf900 0x7ff2b81582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:59.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.937+0000 7ff2be554640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2b815c550 0x7ff2b80ae910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:59.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.937+0000 7ff2be554640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b81693b0 con 0x7ff2b80cf900 2024-09-19T08:42:59.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.937+0000 7ff2be554640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b8169230 con 0x7ff2b806c1d0 2024-09-19T08:42:59.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.937+0000 7ff2be554640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff2b8169530 con 0x7ff2b815c550 2024-09-19T08:42:59.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.937+0000 7ff2b77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2b80cf900 0x7ff2b81582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:59.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.937+0000 7ff2b77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2b80cf900 0x7ff2b81582f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53280/0 (socket says 172.21.15.98:53280) 2024-09-19T08:42:59.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.937+0000 7ff2b77fe640 1 -- 172.21.15.98:0/894842185 learned_addr learned my addr 172.21.15.98:0/894842185 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:42:59.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.937+0000 7ff2b7fff640 1 --2- 172.21.15.98:0/894842185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2b806c1d0 0x7ff2b81663e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:59.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.937+0000 7ff2bcaf1640 1 --2- 172.21.15.98:0/894842185 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2b815c550 0x7ff2b80ae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:59.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.937+0000 7ff2b77fe640 1 -- 172.21.15.98:0/894842185 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2b815c550 msgr2=0x7ff2b80ae910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:59.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.937+0000 7ff2b77fe640 1 --2- 172.21.15.98:0/894842185 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2b815c550 0x7ff2b80ae910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:59.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.937+0000 7ff2b77fe640 1 -- 172.21.15.98:0/894842185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2b806c1d0 msgr2=0x7ff2b81663e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:42:59.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.937+0000 7ff2b77fe640 1 --2- 172.21.15.98:0/894842185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2b806c1d0 0x7ff2b81663e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:42:59.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.937+0000 7ff2b77fe640 1 -- 172.21.15.98:0/894842185 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b81589b0 con 0x7ff2b80cf900 2024-09-19T08:42:59.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.937+0000 7ff2b77fe640 1 --2- 172.21.15.98:0/894842185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2b80cf900 0x7ff2b81582f0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac00d800 tx=0x7ff2ac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:59.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.937+0000 7ff2b4ff9640 1 -- 172.21.15.98:0/894842185 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac018020 con 0x7ff2b80cf900 2024-09-19T08:42:59.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.937+0000 7ff2b4ff9640 1 -- 172.21.15.98:0/894842185 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2ac0096c0 con 0x7ff2b80cf900 2024-09-19T08:42:59.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.937+0000 7ff2be554640 1 -- 172.21.15.98:0/894842185 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b8158c90 con 0x7ff2b80cf900 2024-09-19T08:42:59.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.937+0000 7ff2b4ff9640 1 -- 172.21.15.98:0/894842185 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac009ab0 con 0x7ff2b80cf900 2024-09-19T08:42:59.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.937+0000 7ff2be554640 1 -- 172.21.15.98:0/894842185 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b8159250 con 0x7ff2b80cf900 2024-09-19T08:42:59.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.941+0000 7ff2be554640 1 -- 172.21.15.98:0/894842185 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2b80cf3c0 con 0x7ff2b80cf900 2024-09-19T08:42:59.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.941+0000 7ff2b4ff9640 1 -- 172.21.15.98:0/894842185 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7ff2ac01b020 con 0x7ff2b80cf900 2024-09-19T08:42:59.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.941+0000 7ff2b4ff9640 1 --2- 172.21.15.98:0/894842185 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff2a4067c20 0x7ff2a406a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:42:59.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.941+0000 7ff2b4ff9640 1 -- 172.21.15.98:0/894842185 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7ff2ac086670 con 0x7ff2b80cf900 2024-09-19T08:42:59.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.941+0000 7ff2b7fff640 1 --2- 172.21.15.98:0/894842185 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff2a4067c20 0x7ff2a406a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:42:59.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.941+0000 7ff2b7fff640 1 --2- 172.21.15.98:0/894842185 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff2a4067c20 0x7ff2a406a0e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff2a80ad000 tx=0x7ff2a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:42:59.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:42:59.945+0000 7ff2b4ff9640 1 -- 172.21.15.98:0/894842185 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff2ac022020 con 0x7ff2b80cf900 2024-09-19T08:43:00.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.137+0000 7ff2be554640 1 -- 172.21.15.98:0/894842185 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2b806bd30 con 0x7ff2b80cf900 2024-09-19T08:43:00.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.137+0000 7ff2b4ff9640 1 -- 172.21.15.98:0/894842185 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5228 (secure 0 0 0) 0x7ff2ac056e30 con 0x7ff2b80cf900 2024-09-19T08:43:00.139 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:00.140 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":39,"btime":"2024-09-19T08:42:45:442311+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5548,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3092385040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3092385040},{"type":"v1","addr":"172.21.15.139:6833","nonce":3092385040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35},{"gid":5551,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3517671290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3517671290},{"type":"v1","addr":"172.21.15.139:6835","nonce":3517671290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35},{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6836/965821274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":965821274},{"type":"v1","addr":"172.21.15.98:6836","nonce":965821274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:42:29.720825+0000","modified":"2024-09-19T08:42:31.743275+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6171":{"gid":6171,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/1767881083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6835","nonce":1767881083},{"type":"v1","addr":"172.21.15.98:6837","nonce":1767881083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6171,"qdb_cluster":[6171]},"id":5}]} 2024-09-19T08:43:00.140 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 39 2024-09-19T08:43:00.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.141+0000 7ff2be554640 1 -- 172.21.15.98:0/894842185 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff2a4067c20 msgr2=0x7ff2a406a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:00.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.141+0000 7ff2be554640 1 --2- 172.21.15.98:0/894842185 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff2a4067c20 0x7ff2a406a0e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff2a80ad000 tx=0x7ff2a8008040 comp rx=0 tx=0).stop 2024-09-19T08:43:00.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.141+0000 7ff2be554640 1 -- 172.21.15.98:0/894842185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2b80cf900 msgr2=0x7ff2b81582f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:00.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.141+0000 7ff2be554640 1 --2- 172.21.15.98:0/894842185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2b80cf900 0x7ff2b81582f0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac00d800 tx=0x7ff2ac00dcc0 comp rx=0 tx=0).stop 2024-09-19T08:43:00.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.141+0000 7ff2be554640 1 -- 172.21.15.98:0/894842185 shutdown_connections 2024-09-19T08:43:00.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.141+0000 7ff2be554640 1 --2- 172.21.15.98:0/894842185 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7ff2a4067c20 0x7ff2a406a0e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:00.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.141+0000 7ff2be554640 1 --2- 172.21.15.98:0/894842185 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff2b815c550 0x7ff2b80ae910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:00.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.141+0000 7ff2be554640 1 --2- 172.21.15.98:0/894842185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff2b80cf900 0x7ff2b81582f0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:00.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.141+0000 7ff2be554640 1 --2- 172.21.15.98:0/894842185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2b806c1d0 0x7ff2b81663e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:00.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.141+0000 7ff2be554640 1 -- 172.21.15.98:0/894842185 >> 172.21.15.98:0/894842185 conn(0x7ff2b80b2010 msgr2=0x7ff2b80cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:00.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.141+0000 7ff2be554640 1 -- 172.21.15.98:0/894842185 shutdown_connections 2024-09-19T08:43:00.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.141+0000 7ff2be554640 1 -- 172.21.15.98:0/894842185 wait complete. 2024-09-19T08:43:00.158 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-19T08:43:00.298 INFO:teuthology.orchestra.run.smithi098.stdout:[] 2024-09-19T08:43:00.298 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-09-19T08:43:00.298 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol95540 2024-09-19T08:43:00.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.421+0000 7f14bd7e8640 1 Processor -- start 2024-09-19T08:43:00.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.421+0000 7f14bd7e8640 1 -- start start 2024-09-19T08:43:00.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.425+0000 7f14bd7e8640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f14b815e4b0 0x7f14b8068220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:00.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.425+0000 7f14bd7e8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14b80ce5e0 0x7f14b806d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:00.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.425+0000 7f14bd7e8640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14b8071e20 con 0x7f14b80c9d20 2024-09-19T08:43:00.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.425+0000 7f14bd7e8640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f14b8068760 con 0x7f14b815e4b0 2024-09-19T08:43:00.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.425+0000 7f14bd7e8640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f14b80720a0 con 0x7f14b80ce5e0 2024-09-19T08:43:00.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.425+0000 7f14b6ffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f14b815e4b0 0x7f14b8068220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:00.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.425+0000 7f14b6ffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f14b815e4b0 0x7f14b8068220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53290/0 (socket says 172.21.15.98:53290) 2024-09-19T08:43:00.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.425+0000 7f14b6ffd640 1 -- 172.21.15.98:0/2658131974 learned_addr learned my addr 172.21.15.98:0/2658131974 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:00.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.425+0000 7f14b6ffd640 1 -- 172.21.15.98:0/2658131974 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14b80ce5e0 msgr2=0x7f14b806d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:00.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.425+0000 7f14b67fc640 1 --2- 172.21.15.98:0/2658131974 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14b80ce5e0 0x7f14b806d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:00.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.425+0000 7f14b6ffd640 1 --2- 172.21.15.98:0/2658131974 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14b80ce5e0 0x7f14b806d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:00.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.425+0000 7f14b6ffd640 1 -- 172.21.15.98:0/2658131974 >> v1:172.21.15.139:6789/0 conn(0x7f14b80c9d20 legacy=0x7f14b8072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:00.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.425+0000 7f14b6ffd640 1 -- 172.21.15.98:0/2658131974 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b8072360 con 0x7f14b815e4b0 2024-09-19T08:43:00.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.425+0000 7f14b6ffd640 1 --2- 172.21.15.98:0/2658131974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14b815e4b0 0x7f14b8068220 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f14a8009870 tx=0x7f14a801c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c23c7a791831d7e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:00.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.425+0000 7f14b5ffb640 1 -- 172.21.15.98:0/2658131974 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a80b0070 con 0x7f14b815e4b0 2024-09-19T08:43:00.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.425+0000 7f14b5ffb640 1 -- 172.21.15.98:0/2658131974 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14a80ab600 con 0x7f14b815e4b0 2024-09-19T08:43:00.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.425+0000 7f14b5ffb640 1 -- 172.21.15.98:0/2658131974 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a80ab9d0 con 0x7f14b815e4b0 2024-09-19T08:43:00.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.425+0000 7f14bd7e8640 1 -- 172.21.15.98:0/2658131974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14b815e4b0 msgr2=0x7f14b8068220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:00.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.425+0000 7f14bd7e8640 1 --2- 172.21.15.98:0/2658131974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14b815e4b0 0x7f14b8068220 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f14a8009870 tx=0x7f14a801c340 comp rx=0 tx=0).stop 2024-09-19T08:43:00.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.429+0000 7f14bd7e8640 1 -- 172.21.15.98:0/2658131974 shutdown_connections 2024-09-19T08:43:00.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.429+0000 7f14bd7e8640 1 --2- 172.21.15.98:0/2658131974 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14b80ce5e0 0x7f14b806d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:00.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.429+0000 7f14bd7e8640 1 --2- 172.21.15.98:0/2658131974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14b815e4b0 0x7f14b8068220 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:00.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.429+0000 7f14bd7e8640 1 -- 172.21.15.98:0/2658131974 >> 172.21.15.98:0/2658131974 conn(0x7f14b80d0a10 msgr2=0x7f14b80d0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:00.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.429+0000 7f14bd7e8640 1 -- 172.21.15.98:0/2658131974 shutdown_connections 2024-09-19T08:43:00.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.429+0000 7f14bd7e8640 1 -- 172.21.15.98:0/2658131974 wait complete. 2024-09-19T08:43:00.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.429+0000 7f14bd7e8640 1 Processor -- start 2024-09-19T08:43:00.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.429+0000 7f14bd7e8640 1 -- start start 2024-09-19T08:43:00.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.429+0000 7f14bd7e8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14b80c9d20 0x7f14b814ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:00.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.429+0000 7f14bd7e8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14b80ce5e0 0x7f14b8151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:00.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.429+0000 7f14bd7e8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14b815e4b0 0x7f14b8147bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:00.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.429+0000 7f14bd7e8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f14b806dbe0 con 0x7f14b80ce5e0 2024-09-19T08:43:00.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.429+0000 7f14bd7e8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f14b8073960 con 0x7f14b80c9d20 2024-09-19T08:43:00.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.429+0000 7f14bd7e8640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f14b80ca5c0 con 0x7f14b815e4b0 2024-09-19T08:43:00.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.429+0000 7f14b67fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14b80ce5e0 0x7f14b8151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:00.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.429+0000 7f14b67fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14b80ce5e0 0x7f14b8151350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53292/0 (socket says 172.21.15.98:53292) 2024-09-19T08:43:00.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.429+0000 7f14b67fc640 1 -- 172.21.15.98:0/2972919929 learned_addr learned my addr 172.21.15.98:0/2972919929 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:00.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.429+0000 7f14b6ffd640 1 --2- 172.21.15.98:0/2972919929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14b80c9d20 0x7f14b814ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:00.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.429+0000 7f14b77fe640 1 --2- 172.21.15.98:0/2972919929 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14b815e4b0 0x7f14b8147bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:00.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.429+0000 7f14b67fc640 1 -- 172.21.15.98:0/2972919929 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14b815e4b0 msgr2=0x7f14b8147bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:00.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.429+0000 7f14b67fc640 1 --2- 172.21.15.98:0/2972919929 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14b815e4b0 0x7f14b8147bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:00.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.429+0000 7f14b67fc640 1 -- 172.21.15.98:0/2972919929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14b80c9d20 msgr2=0x7f14b814ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:00.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.429+0000 7f14b67fc640 1 --2- 172.21.15.98:0/2972919929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14b80c9d20 0x7f14b814ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:00.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.429+0000 7f14b67fc640 1 -- 172.21.15.98:0/2972919929 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b814f180 con 0x7f14b80ce5e0 2024-09-19T08:43:00.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.429+0000 7f14b67fc640 1 --2- 172.21.15.98:0/2972919929 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14b80ce5e0 0x7f14b8151350 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00c7f0 tx=0x7f14ac00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:00.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.429+0000 7f149f7fe640 1 -- 172.21.15.98:0/2972919929 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac015070 con 0x7f14b80ce5e0 2024-09-19T08:43:00.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.429+0000 7f149f7fe640 1 -- 172.21.15.98:0/2972919929 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14ac007820 con 0x7f14b80ce5e0 2024-09-19T08:43:00.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.429+0000 7f14bd7e8640 1 -- 172.21.15.98:0/2972919929 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14b814f460 con 0x7f14b80ce5e0 2024-09-19T08:43:00.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.429+0000 7f149f7fe640 1 -- 172.21.15.98:0/2972919929 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac007c50 con 0x7f14b80ce5e0 2024-09-19T08:43:00.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.429+0000 7f14bd7e8640 1 -- 172.21.15.98:0/2972919929 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14b8150e30 con 0x7f14b80ce5e0 2024-09-19T08:43:00.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.429+0000 7f14bd7e8640 1 -- 172.21.15.98:0/2972919929 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14b8151030 con 0x7f14b80ce5e0 2024-09-19T08:43:00.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.433+0000 7f149f7fe640 1 -- 172.21.15.98:0/2972919929 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f14ac018070 con 0x7f14b80ce5e0 2024-09-19T08:43:00.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.433+0000 7f149f7fe640 1 --2- 172.21.15.98:0/2972919929 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f14a4067ca0 0x7f14a406a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:00.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.433+0000 7f14b6ffd640 1 --2- 172.21.15.98:0/2972919929 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f14a4067ca0 0x7f14a406a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:00.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.433+0000 7f149f7fe640 1 -- 172.21.15.98:0/2972919929 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f14ac0863f0 con 0x7f14b80ce5e0 2024-09-19T08:43:00.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.433+0000 7f14b6ffd640 1 --2- 172.21.15.98:0/2972919929 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f14a4067ca0 0x7f14a406a160 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f14a80ad000 tx=0x7f14a80a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:00.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.437+0000 7f149f7fe640 1 -- 172.21.15.98:0/2972919929 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f14b8151030 con 0x7f14b80ce5e0 2024-09-19T08:43:00.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.629+0000 7f14bd7e8640 1 -- 172.21.15.98:0/2972919929 --> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol95540", "target": ["mon-mgr", ""]}) -- 0x7f14b806d6a0 con 0x7f14a4067ca0 2024-09-19T08:43:00.636 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:43:00.633+0000 7f36e9c04640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:43:00.636 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:43:00.633+0000 7f36e9c04640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:43:00.636 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:43:00.633+0000 7f36e9c04640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:43:00.636 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:43:00.633+0000 7f36e9c04640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:43:00.636 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:43:00.633+0000 7f36e9c04640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:43:00.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.669+0000 7f149f7fe640 1 -- 172.21.15.98:0/2972919929 <== mgr.4101 v2:172.21.15.98:6832/2040310606 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f14b806d6a0 con 0x7f14a4067ca0 2024-09-19T08:43:00.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.673+0000 7f14bd7e8640 1 -- 172.21.15.98:0/2972919929 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f14a4067ca0 msgr2=0x7f14a406a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:00.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.673+0000 7f14bd7e8640 1 --2- 172.21.15.98:0/2972919929 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f14a4067ca0 0x7f14a406a160 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f14a80ad000 tx=0x7f14a80a6000 comp rx=0 tx=0).stop 2024-09-19T08:43:00.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.673+0000 7f14bd7e8640 1 -- 172.21.15.98:0/2972919929 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14b80ce5e0 msgr2=0x7f14b8151350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:00.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.673+0000 7f14bd7e8640 1 --2- 172.21.15.98:0/2972919929 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14b80ce5e0 0x7f14b8151350 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00c7f0 tx=0x7f14ac00ccb0 comp rx=0 tx=0).stop 2024-09-19T08:43:00.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.673+0000 7f14bd7e8640 1 -- 172.21.15.98:0/2972919929 shutdown_connections 2024-09-19T08:43:00.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.673+0000 7f14bd7e8640 1 --2- 172.21.15.98:0/2972919929 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f14a4067ca0 0x7f14a406a160 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:00.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.673+0000 7f14bd7e8640 1 --2- 172.21.15.98:0/2972919929 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14b815e4b0 0x7f14b8147bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:00.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.673+0000 7f14bd7e8640 1 --2- 172.21.15.98:0/2972919929 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14b80ce5e0 0x7f14b8151350 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:00.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.673+0000 7f14bd7e8640 1 --2- 172.21.15.98:0/2972919929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14b80c9d20 0x7f14b814ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:00.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.673+0000 7f14bd7e8640 1 -- 172.21.15.98:0/2972919929 >> 172.21.15.98:0/2972919929 conn(0x7f14b80d0a10 msgr2=0x7f14b8072e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:00.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.673+0000 7f14bd7e8640 1 -- 172.21.15.98:0/2972919929 shutdown_connections 2024-09-19T08:43:00.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.673+0000 7f14bd7e8640 1 -- 172.21.15.98:0/2972919929 wait complete. 2024-09-19T08:43:00.689 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:43:00.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.805+0000 7f56b61e8640 1 Processor -- start 2024-09-19T08:43:00.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.805+0000 7f56b61e8640 1 -- start start 2024-09-19T08:43:00.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.805+0000 7f56b61e8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f56b00b7fc0 0x7f56b015f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:00.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.805+0000 7f56b61e8640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56b006c6e0 con 0x7f56b015e420 2024-09-19T08:43:00.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.805+0000 7f56b61e8640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56b006c9c0 con 0x7f56b015e800 2024-09-19T08:43:00.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.805+0000 7f56b61e8640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f56b015ed60 con 0x7f56b00b7fc0 2024-09-19T08:43:00.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.805+0000 7f56af7fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f56b015e420 0x7f56b006bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:37462/0 (socket says 172.21.15.98:37462) 2024-09-19T08:43:00.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.805+0000 7f56af7fe640 1 -- 172.21.15.98:0/430327356 learned_addr learned my addr 172.21.15.98:0/430327356 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:00.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.805+0000 7f56affff640 1 --2- 172.21.15.98:0/430327356 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f56b00b7fc0 0x7f56b015f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:00.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.805+0000 7f56aeffd640 1 -- 172.21.15.98:0/430327356 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2039497708 0 0) 0x7f56b006c6e0 con 0x7f56b015e420 2024-09-19T08:43:00.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.805+0000 7f56aeffd640 1 -- 172.21.15.98:0/430327356 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5690003530 con 0x7f56b015e420 2024-09-19T08:43:00.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.805+0000 7f56aeffd640 1 -- 172.21.15.98:0/430327356 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 566634840 0 0) 0x7f5690003530 con 0x7f56b015e420 2024-09-19T08:43:00.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.805+0000 7f56aeffd640 1 -- 172.21.15.98:0/430327356 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f56b00b7fc0 msgr2=0x7f56b015f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:00.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.805+0000 7f56aeffd640 1 --2- 172.21.15.98:0/430327356 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f56b00b7fc0 0x7f56b015f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:00.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.805+0000 7f56aeffd640 1 -- 172.21.15.98:0/430327356 >> v1:172.21.15.139:6789/0 conn(0x7f56b015e800 legacy=0x7f56b00b78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:00.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.805+0000 7f56aeffd640 1 -- 172.21.15.98:0/430327356 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b015f040 con 0x7f56b015e420 2024-09-19T08:43:00.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.805+0000 7f56aeffd640 1 -- 172.21.15.98:0/430327356 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f5698001680 con 0x7f56b015e420 2024-09-19T08:43:00.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.805+0000 7f56aeffd640 1 -- 172.21.15.98:0/430327356 >> v1:172.21.15.98:6789/0 conn(0x7f56b015e420 legacy=0x7f56b006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:00.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.805+0000 7f56aeffd640 1 --2- 172.21.15.98:0/430327356 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56900039f0 0x7f5690003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:00.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.805+0000 7f56aeffd640 1 --2- 172.21.15.98:0/430327356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5690004810 0x7f5690004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:00.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.805+0000 7f56aeffd640 1 --2- 172.21.15.98:0/430327356 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f56b00b7fc0 0x7f56900051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:00.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.805+0000 7f56aeffd640 1 -- 172.21.15.98:0/430327356 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f56b015ed60 con 0x7f5690004810 2024-09-19T08:43:00.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.805+0000 7f56aeffd640 1 -- 172.21.15.98:0/430327356 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f569000bab0 con 0x7f56900039f0 2024-09-19T08:43:00.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.805+0000 7f56aeffd640 1 -- 172.21.15.98:0/430327356 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f569000bcb0 con 0x7f56b00b7fc0 2024-09-19T08:43:00.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.805+0000 7f56a6ffd640 1 --2- 172.21.15.98:0/430327356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5690004810 0x7f5690004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:00.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.805+0000 7f56af7fe640 1 --2- 172.21.15.98:0/430327356 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56900039f0 0x7f5690003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:00.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.805+0000 7f56affff640 1 --2- 172.21.15.98:0/430327356 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f56b00b7fc0 0x7f56900051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:00.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.805+0000 7f56a6ffd640 1 -- 172.21.15.98:0/430327356 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f56b00b7fc0 msgr2=0x7f56900051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:00.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.805+0000 7f56a6ffd640 1 --2- 172.21.15.98:0/430327356 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f56b00b7fc0 0x7f56900051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:00.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.805+0000 7f56a6ffd640 1 -- 172.21.15.98:0/430327356 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56900039f0 msgr2=0x7f5690003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:00.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.805+0000 7f56a6ffd640 1 --2- 172.21.15.98:0/430327356 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56900039f0 0x7f5690003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:00.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.805+0000 7f56a6ffd640 1 -- 172.21.15.98:0/430327356 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5690003530 con 0x7f5690004810 2024-09-19T08:43:00.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.809+0000 7f56a6ffd640 1 --2- 172.21.15.98:0/430327356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5690004810 0x7f5690004c90 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f56a0001750 tx=0x7f56a001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9d5d2e622d0f1a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:00.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.809+0000 7f56aeffd640 1 -- 172.21.15.98:0/430327356 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a00013a0 con 0x7f5690004810 2024-09-19T08:43:00.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.809+0000 7f56aeffd640 1 -- 172.21.15.98:0/430327356 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56a00052f0 con 0x7f5690004810 2024-09-19T08:43:00.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.809+0000 7f56aeffd640 1 -- 172.21.15.98:0/430327356 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a0003840 con 0x7f5690004810 2024-09-19T08:43:00.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.809+0000 7f56b61e8640 1 -- 172.21.15.98:0/430327356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5690004810 msgr2=0x7f5690004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:00.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.809+0000 7f56b61e8640 1 --2- 172.21.15.98:0/430327356 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5690004810 0x7f5690004c90 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f56a0001750 tx=0x7f56a001c3c0 comp rx=0 tx=0).stop 2024-09-19T08:43:00.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.809+0000 7f56affff640 1 -- 172.21.15.98:0/430327356 reap_dead start 2024-09-19T08:43:00.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.809+0000 7f56b61e8640 1 -- 172.21.15.98:0/430327356 shutdown_connections 2024-09-19T08:43:00.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.809+0000 7f56b61e8640 1 -- 172.21.15.98:0/430327356 >> 172.21.15.98:0/430327356 conn(0x7f56b00b40a0 msgr2=0x7f56b00b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:00.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.809+0000 7f56b61e8640 1 -- 172.21.15.98:0/430327356 shutdown_connections 2024-09-19T08:43:00.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.809+0000 7f56b61e8640 1 -- 172.21.15.98:0/430327356 wait complete. 2024-09-19T08:43:00.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.813+0000 7f56b61e8640 1 Processor -- start 2024-09-19T08:43:00.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.813+0000 7f56b61e8640 1 -- start start 2024-09-19T08:43:00.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.813+0000 7f56b61e8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5690004810 0x7f56b00b0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:00.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.813+0000 7f56b61e8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56b00c7430 0x7f56b014fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:00.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.813+0000 7f56b61e8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56b00ae9d0 0x7f56b01f4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:00.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.813+0000 7f56b61e8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f56b0073750 con 0x7f56b00ae9d0 2024-09-19T08:43:00.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.813+0000 7f56b61e8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f56b00735d0 con 0x7f56b00c7430 2024-09-19T08:43:00.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.813+0000 7f56b61e8640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f56b015f560 con 0x7f5690004810 2024-09-19T08:43:00.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.813+0000 7f56affff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56b00ae9d0 0x7f56b01f4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:00.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.813+0000 7f56affff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56b00ae9d0 0x7f56b01f4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53326/0 (socket says 172.21.15.98:53326) 2024-09-19T08:43:00.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.813+0000 7f56affff640 1 -- 172.21.15.98:0/1983552364 learned_addr learned my addr 172.21.15.98:0/1983552364 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:00.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.813+0000 7f56a6ffd640 1 --2- 172.21.15.98:0/1983552364 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56b00c7430 0x7f56b014fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:00.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.813+0000 7f56affff640 1 -- 172.21.15.98:0/1983552364 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5690004810 msgr2=0x7f56b00b0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:00.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.813+0000 7f56af7fe640 1 --2- 172.21.15.98:0/1983552364 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5690004810 0x7f56b00b0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:00.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.813+0000 7f56affff640 1 --2- 172.21.15.98:0/1983552364 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5690004810 0x7f56b00b0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:00.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.813+0000 7f56affff640 1 -- 172.21.15.98:0/1983552364 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56b00c7430 msgr2=0x7f56b014fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:00.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.813+0000 7f56affff640 1 --2- 172.21.15.98:0/1983552364 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56b00c7430 0x7f56b014fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:00.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.813+0000 7f56affff640 1 -- 172.21.15.98:0/1983552364 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b00af3b0 con 0x7f56b00ae9d0 2024-09-19T08:43:00.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.813+0000 7f56a6ffd640 1 --2- 172.21.15.98:0/1983552364 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56b00c7430 0x7f56b014fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:00.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.813+0000 7f56affff640 1 --2- 172.21.15.98:0/1983552364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56b00ae9d0 0x7f56b01f4510 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f56b0070320 tx=0x7f56a800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:00.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.813+0000 7f56acff9640 1 -- 172.21.15.98:0/1983552364 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a801c070 con 0x7f56b00ae9d0 2024-09-19T08:43:00.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.813+0000 7f56acff9640 1 -- 172.21.15.98:0/1983552364 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56a8017400 con 0x7f56b00ae9d0 2024-09-19T08:43:00.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.813+0000 7f56acff9640 1 -- 172.21.15.98:0/1983552364 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a80177f0 con 0x7f56b00ae9d0 2024-09-19T08:43:00.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.813+0000 7f56b61e8640 1 -- 172.21.15.98:0/1983552364 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56b00af690 con 0x7f56b00ae9d0 2024-09-19T08:43:00.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.813+0000 7f56b61e8640 1 -- 172.21.15.98:0/1983552364 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56b00afc50 con 0x7f56b00ae9d0 2024-09-19T08:43:00.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.813+0000 7f56b61e8640 1 -- 172.21.15.98:0/1983552364 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56b015f920 con 0x7f56b00ae9d0 2024-09-19T08:43:00.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.817+0000 7f56acff9640 1 -- 172.21.15.98:0/1983552364 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7f56a801f070 con 0x7f56b00ae9d0 2024-09-19T08:43:00.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.817+0000 7f56acff9640 1 --2- 172.21.15.98:0/1983552364 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5694067c50 0x7f569406a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:00.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.817+0000 7f56acff9640 1 -- 172.21.15.98:0/1983552364 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5825+0+0 (secure 0 0 0) 0x7f56a80864f0 con 0x7f56b00ae9d0 2024-09-19T08:43:00.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.817+0000 7f56af7fe640 1 --2- 172.21.15.98:0/1983552364 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5694067c50 0x7f569406a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:00.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.817+0000 7f56af7fe640 1 --2- 172.21.15.98:0/1983552364 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5694067c50 0x7f569406a110 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5698001210 tx=0x7f5698001140 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:00.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:00.821+0000 7f56acff9640 1 -- 172.21.15.98:0/1983552364 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f56a8012070 con 0x7f56b00ae9d0 2024-09-19T08:43:01.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.017+0000 7f56b61e8640 1 -- 172.21.15.98:0/1983552364 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56b006dc80 con 0x7f56b00ae9d0 2024-09-19T08:43:01.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.017+0000 7f56acff9640 1 -- 172.21.15.98:0/1983552364 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5228 (secure 0 0 0) 0x7f56a8056400 con 0x7f56b00ae9d0 2024-09-19T08:43:01.019 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:01.019 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":39,"btime":"2024-09-19T08:42:45:442311+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5548,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3092385040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3092385040},{"type":"v1","addr":"172.21.15.139:6833","nonce":3092385040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35},{"gid":5551,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3517671290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3517671290},{"type":"v1","addr":"172.21.15.139:6835","nonce":3517671290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35},{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6836/965821274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":965821274},{"type":"v1","addr":"172.21.15.98:6836","nonce":965821274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:42:29.720825+0000","modified":"2024-09-19T08:42:31.743275+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6171":{"gid":6171,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/1767881083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6835","nonce":1767881083},{"type":"v1","addr":"172.21.15.98:6837","nonce":1767881083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6171,"qdb_cluster":[6171]},"id":5}]} 2024-09-19T08:43:01.019 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 39 2024-09-19T08:43:01.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.021+0000 7f56b61e8640 1 -- 172.21.15.98:0/1983552364 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5694067c50 msgr2=0x7f569406a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:01.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.021+0000 7f56b61e8640 1 --2- 172.21.15.98:0/1983552364 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5694067c50 0x7f569406a110 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5698001210 tx=0x7f5698001140 comp rx=0 tx=0).stop 2024-09-19T08:43:01.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.021+0000 7f56b61e8640 1 -- 172.21.15.98:0/1983552364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56b00ae9d0 msgr2=0x7f56b01f4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:01.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.021+0000 7f56b61e8640 1 --2- 172.21.15.98:0/1983552364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56b00ae9d0 0x7f56b01f4510 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f56b0070320 tx=0x7f56a800f4a0 comp rx=0 tx=0).stop 2024-09-19T08:43:01.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.021+0000 7f56b61e8640 1 -- 172.21.15.98:0/1983552364 shutdown_connections 2024-09-19T08:43:01.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.021+0000 7f56b61e8640 1 --2- 172.21.15.98:0/1983552364 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f5694067c50 0x7f569406a110 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:01.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.021+0000 7f56b61e8640 1 --2- 172.21.15.98:0/1983552364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56b00ae9d0 0x7f56b01f4510 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:01.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.021+0000 7f56b61e8640 1 --2- 172.21.15.98:0/1983552364 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56b00c7430 0x7f56b014fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:01.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.021+0000 7f56b61e8640 1 --2- 172.21.15.98:0/1983552364 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5690004810 0x7f56b00b0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:01.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.021+0000 7f56b61e8640 1 -- 172.21.15.98:0/1983552364 >> 172.21.15.98:0/1983552364 conn(0x7f56b00b40a0 msgr2=0x7f56b00b67c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:01.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.021+0000 7f56b61e8640 1 -- 172.21.15.98:0/1983552364 shutdown_connections 2024-09-19T08:43:01.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.021+0000 7f56b61e8640 1 -- 172.21.15.98:0/1983552364 wait complete. 2024-09-19T08:43:01.038 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-19T08:43:01.179 INFO:teuthology.orchestra.run.smithi098.stdout:[{"id":6426,"entity":{"name":{"type":"client","num":6426},"addr":{"type":"any","addr":"172.21.15.98:0","nonce":1268825931}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":0.52799323499999995,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.973759586919773,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":14,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6426 172.21.15.98:0/1268825931","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"y","hostname":"smithi098","pid":"28958","root":"/"}}] 2024-09-19T08:43:01.179 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi098", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28958", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.98:0", "nonce": 1268825931, "type": "any" }, "name": { "num": 6426, "type": "client" } }, "id": 6426, "inst": "client.6426 172.21.15.98:0/1268825931", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 14, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.973759586919773 }, "state": "open", "uptime": 0.527993235 } ] 2024-09-19T08:43:01.179 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-19T08:43:01.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.293+0000 7fcd66e6d640 1 Processor -- start 2024-09-19T08:43:01.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.293+0000 7fcd66e6d640 1 -- start start 2024-09-19T08:43:01.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.293+0000 7fcd66e6d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcd6015c550 0x7fcd6006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:01.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.293+0000 7fcd66e6d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd600cfad0 0x7fcd600723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:01.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.293+0000 7fcd66e6d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd6006d310 con 0x7fcd6006c1d0 2024-09-19T08:43:01.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.293+0000 7fcd66e6d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fcd6006c640 con 0x7fcd6015c550 2024-09-19T08:43:01.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.293+0000 7fcd66e6d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcd6006d5f0 con 0x7fcd600cfad0 2024-09-19T08:43:01.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.297+0000 7fcd64c09640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcd6015c550 0x7fcd6006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:01.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.297+0000 7fcd6540a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd600cfad0 0x7fcd600723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:01.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.297+0000 7fcd5ffff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fcd6006c1d0 0x7fcd600cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:38424/0 (socket says 172.21.15.98:38424) 2024-09-19T08:43:01.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.297+0000 7fcd64c09640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcd6015c550 0x7fcd6006bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:48572/0 (socket says 172.21.15.98:48572) 2024-09-19T08:43:01.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.297+0000 7fcd5ffff640 1 -- 172.21.15.98:0/4169279708 learned_addr learned my addr 172.21.15.98:0/4169279708 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:01.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.297+0000 7fcd64c09640 1 -- 172.21.15.98:0/4169279708 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd600cfad0 msgr2=0x7fcd600723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:01.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.297+0000 7fcd64c09640 1 --2- 172.21.15.98:0/4169279708 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd600cfad0 0x7fcd600723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:01.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.297+0000 7fcd64c09640 1 -- 172.21.15.98:0/4169279708 >> v1:172.21.15.139:6789/0 conn(0x7fcd6006c1d0 legacy=0x7fcd600cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:01.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.297+0000 7fcd64c09640 1 -- 172.21.15.98:0/4169279708 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd6006d8d0 con 0x7fcd6015c550 2024-09-19T08:43:01.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.301+0000 7fcd64c09640 1 --2- 172.21.15.98:0/4169279708 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd6015c550 0x7fcd6006bc90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcd500097c0 tx=0x7fcd5001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b3ad0adf7c9b9e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:01.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.301+0000 7fcd5f7fe640 1 -- 172.21.15.98:0/4169279708 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd500b0070 con 0x7fcd6015c550 2024-09-19T08:43:01.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.301+0000 7fcd5f7fe640 1 -- 172.21.15.98:0/4169279708 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd500ab600 con 0x7fcd6015c550 2024-09-19T08:43:01.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.301+0000 7fcd5f7fe640 1 -- 172.21.15.98:0/4169279708 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd500ab9d0 con 0x7fcd6015c550 2024-09-19T08:43:01.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.301+0000 7fcd66e6d640 1 -- 172.21.15.98:0/4169279708 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd6015c550 msgr2=0x7fcd6006bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:01.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.301+0000 7fcd66e6d640 1 --2- 172.21.15.98:0/4169279708 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd6015c550 0x7fcd6006bc90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcd500097c0 tx=0x7fcd5001c4a0 comp rx=0 tx=0).stop 2024-09-19T08:43:01.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.301+0000 7fcd66e6d640 1 -- 172.21.15.98:0/4169279708 shutdown_connections 2024-09-19T08:43:01.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.301+0000 7fcd66e6d640 1 --2- 172.21.15.98:0/4169279708 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd600cfad0 0x7fcd600723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:01.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.301+0000 7fcd66e6d640 1 --2- 172.21.15.98:0/4169279708 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd6015c550 0x7fcd6006bc90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:01.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.301+0000 7fcd66e6d640 1 -- 172.21.15.98:0/4169279708 >> 172.21.15.98:0/4169279708 conn(0x7fcd600b2010 msgr2=0x7fcd600b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:01.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.301+0000 7fcd66e6d640 1 -- 172.21.15.98:0/4169279708 shutdown_connections 2024-09-19T08:43:01.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.301+0000 7fcd66e6d640 1 -- 172.21.15.98:0/4169279708 wait complete. 2024-09-19T08:43:01.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.301+0000 7fcd66e6d640 1 Processor -- start 2024-09-19T08:43:01.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.301+0000 7fcd66e6d640 1 -- start start 2024-09-19T08:43:01.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.301+0000 7fcd66e6d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd6006c1d0 0x7fcd6014cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:01.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.301+0000 7fcd66e6d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd600cfad0 0x7fcd6014e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:01.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.301+0000 7fcd66e6d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd6015c550 0x7fcd6014e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:01.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.301+0000 7fcd66e6d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcd60073380 con 0x7fcd6006c1d0 2024-09-19T08:43:01.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.301+0000 7fcd66e6d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6015cb40 con 0x7fcd6015c550 2024-09-19T08:43:01.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.301+0000 7fcd66e6d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcd60073500 con 0x7fcd600cfad0 2024-09-19T08:43:01.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.301+0000 7fcd64c09640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd6006c1d0 0x7fcd6014cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:01.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.301+0000 7fcd64c09640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd6006c1d0 0x7fcd6014cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53342/0 (socket says 172.21.15.98:53342) 2024-09-19T08:43:01.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.301+0000 7fcd64c09640 1 -- 172.21.15.98:0/3855413466 learned_addr learned my addr 172.21.15.98:0/3855413466 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:01.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.301+0000 7fcd6540a640 1 --2- 172.21.15.98:0/3855413466 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd6015c550 0x7fcd6014e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:01.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.301+0000 7fcd5ffff640 1 --2- 172.21.15.98:0/3855413466 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd600cfad0 0x7fcd6014e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:01.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.301+0000 7fcd64c09640 1 -- 172.21.15.98:0/3855413466 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd600cfad0 msgr2=0x7fcd6014e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:01.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.301+0000 7fcd64c09640 1 --2- 172.21.15.98:0/3855413466 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd600cfad0 0x7fcd6014e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:01.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.301+0000 7fcd64c09640 1 -- 172.21.15.98:0/3855413466 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd6015c550 msgr2=0x7fcd6014e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:01.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.301+0000 7fcd64c09640 1 --2- 172.21.15.98:0/3855413466 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd6015c550 0x7fcd6014e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:01.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.301+0000 7fcd64c09640 1 -- 172.21.15.98:0/3855413466 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd60151c80 con 0x7fcd6006c1d0 2024-09-19T08:43:01.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.301+0000 7fcd64c09640 1 --2- 172.21.15.98:0/3855413466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd6006c1d0 0x7fcd6014cd80 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fcd5001a0d0 tx=0x7fcd50019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:01.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.301+0000 7fcd5d7fa640 1 -- 172.21.15.98:0/3855413466 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd500b8020 con 0x7fcd6006c1d0 2024-09-19T08:43:01.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.301+0000 7fcd5d7fa640 1 -- 172.21.15.98:0/3855413466 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd500b4410 con 0x7fcd6006c1d0 2024-09-19T08:43:01.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.301+0000 7fcd66e6d640 1 -- 172.21.15.98:0/3855413466 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd60151f60 con 0x7fcd6006c1d0 2024-09-19T08:43:01.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.301+0000 7fcd5d7fa640 1 -- 172.21.15.98:0/3855413466 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd500b0040 con 0x7fcd6006c1d0 2024-09-19T08:43:01.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.305+0000 7fcd66e6d640 1 -- 172.21.15.98:0/3855413466 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd6014d2c0 con 0x7fcd6006c1d0 2024-09-19T08:43:01.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.305+0000 7fcd66e6d640 1 -- 172.21.15.98:0/3855413466 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd600723a0 con 0x7fcd6006c1d0 2024-09-19T08:43:01.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.305+0000 7fcd5d7fa640 1 -- 172.21.15.98:0/3855413466 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 11) ==== 87751+0+0 (secure 0 0 0) 0x7fcd500bb020 con 0x7fcd6006c1d0 2024-09-19T08:43:01.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.305+0000 7fcd5d7fa640 1 --2- 172.21.15.98:0/3855413466 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fcd4c067c20 0x7fcd4c06a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:01.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.305+0000 7fcd5d7fa640 1 -- 172.21.15.98:0/3855413466 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5825+0+0 (secure 0 0 0) 0x7fcd501201b0 con 0x7fcd6006c1d0 2024-09-19T08:43:01.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.305+0000 7fcd5ffff640 1 --2- 172.21.15.98:0/3855413466 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fcd4c067c20 0x7fcd4c06a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:01.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.305+0000 7fcd5ffff640 1 --2- 172.21.15.98:0/3855413466 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fcd4c067c20 0x7fcd4c06a0e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcd60158500 tx=0x7fcd54009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:01.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.309+0000 7fcd5d7fa640 1 -- 172.21.15.98:0/3855413466 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fcd500c2030 con 0x7fcd6006c1d0 2024-09-19T08:43:01.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.509+0000 7fcd5d7fa640 1 -- 172.21.15.98:0/3855413466 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mgrmap(e 12) ==== 87802+0+0 (secure 0 0 0) 0x7fcd500ec850 con 0x7fcd6006c1d0 2024-09-19T08:43:01.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.521+0000 7fcd66e6d640 1 -- 172.21.15.98:0/3855413466 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fcd600cc7e0 con 0x7fcd6006c1d0 2024-09-19T08:43:01.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.537+0000 7fcd5d7fa640 1 -- 172.21.15.98:0/3855413466 <== mon.0 v2:172.21.15.98:3300/0 8 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v12) ==== 81+0+453825 (secure 0 0 0) 0x7fcd50016090 con 0x7fcd6006c1d0 2024-09-19T08:43:01.542 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:01.542 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:43:01.542 INFO:teuthology.orchestra.run.smithi098.stdout: "epoch": 12, 2024-09-19T08:43:01.542 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.542 INFO:teuthology.orchestra.run.smithi098.stdout: "active_gid": 4101, 2024-09-19T08:43:01.543 INFO:teuthology.orchestra.run.smithi098.stdout: "active_name": "y", 2024-09-19T08:43:01.543 INFO:teuthology.orchestra.run.smithi098.stdout: "active_addrs": { 2024-09-19T08:43:01.543 INFO:teuthology.orchestra.run.smithi098.stdout: "addrvec": [ 2024-09-19T08:43:01.543 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:01.543 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v2", 2024-09-19T08:43:01.543 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "172.21.15.98:6832", 2024-09-19T08:43:01.543 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 2040310606 2024-09-19T08:43:01.543 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.543 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:01.543 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v1", 2024-09-19T08:43:01.543 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "172.21.15.98:6833", 2024-09-19T08:43:01.543 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 2040310606 2024-09-19T08:43:01.543 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.543 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-19T08:43:01.544 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.544 INFO:teuthology.orchestra.run.smithi098.stdout: "active_addr": "172.21.15.98:6833/2040310606", 2024-09-19T08:43:01.544 INFO:teuthology.orchestra.run.smithi098.stdout: "active_change": "2024-09-19T08:38:00.016702+0000", 2024-09-19T08:43:01.544 INFO:teuthology.orchestra.run.smithi098.stdout: "active_mgr_features": 4540701547738038271, 2024-09-19T08:43:01.544 INFO:teuthology.orchestra.run.smithi098.stdout: "available": true, 2024-09-19T08:43:01.544 INFO:teuthology.orchestra.run.smithi098.stdout: "standbys": [ 2024-09-19T08:43:01.544 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:01.544 INFO:teuthology.orchestra.run.smithi098.stdout: "gid": 4106, 2024-09-19T08:43:01.544 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "x", 2024-09-19T08:43:01.544 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr_features": 4540701547738038271, 2024-09-19T08:43:01.544 INFO:teuthology.orchestra.run.smithi098.stdout: "available_modules": [ 2024-09-19T08:43:01.544 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:01.544 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "alerts", 2024-09-19T08:43:01.545 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:01.545 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:01.545 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:01.545 INFO:teuthology.orchestra.run.smithi098.stdout: "interval": { 2024-09-19T08:43:01.545 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "interval", 2024-09-19T08:43:01.545 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:01.545 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.545 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.545 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "60", 2024-09-19T08:43:01.545 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.545 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.545 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.545 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T08:43:01.545 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.546 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.546 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.546 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.546 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:01.546 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:01.546 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.546 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.546 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.546 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.546 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.546 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.546 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.546 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.546 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.547 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.547 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.547 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.547 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.547 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.547 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.547 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.547 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.547 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.547 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.547 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:01.547 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:01.547 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.548 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.548 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.548 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.548 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.548 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.548 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.548 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.548 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.548 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.548 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.548 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.548 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:01.548 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:01.548 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.549 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.549 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.549 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:01.549 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.549 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.549 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.549 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.549 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.549 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.549 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.549 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.549 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.549 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.549 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.550 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.550 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.550 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.550 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.550 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:01.550 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:01.550 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.550 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.550 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.550 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.550 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.550 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.550 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.550 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.551 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.551 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.551 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.551 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.551 INFO:teuthology.orchestra.run.smithi098.stdout: "smtp_destination": { 2024-09-19T08:43:01.551 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "smtp_destination", 2024-09-19T08:43:01.551 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.551 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.551 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.551 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.551 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.551 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.551 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.552 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Email address to send alerts to", 2024-09-19T08:43:01.552 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.552 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.552 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.552 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.552 INFO:teuthology.orchestra.run.smithi098.stdout: "smtp_from_name": { 2024-09-19T08:43:01.552 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "smtp_from_name", 2024-09-19T08:43:01.552 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.552 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.552 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.552 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "Ceph", 2024-09-19T08:43:01.552 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.552 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.553 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.553 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Email From: name", 2024-09-19T08:43:01.553 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.553 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.553 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.553 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.553 INFO:teuthology.orchestra.run.smithi098.stdout: "smtp_host": { 2024-09-19T08:43:01.553 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "smtp_host", 2024-09-19T08:43:01.553 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.553 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.554 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.554 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.554 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.554 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.554 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.555 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "SMTP server", 2024-09-19T08:43:01.555 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.555 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.555 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.555 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.555 INFO:teuthology.orchestra.run.smithi098.stdout: "smtp_password": { 2024-09-19T08:43:01.555 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "smtp_password", 2024-09-19T08:43:01.555 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.555 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.556 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.556 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.556 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.556 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.556 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.556 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Password to authenticate with", 2024-09-19T08:43:01.556 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.556 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.556 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.556 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.556 INFO:teuthology.orchestra.run.smithi098.stdout: "smtp_port": { 2024-09-19T08:43:01.556 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "smtp_port", 2024-09-19T08:43:01.556 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.557 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.557 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.557 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "465", 2024-09-19T08:43:01.557 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.557 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.557 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.557 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "SMTP port", 2024-09-19T08:43:01.557 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.557 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.557 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.558 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.558 INFO:teuthology.orchestra.run.smithi098.stdout: "smtp_sender": { 2024-09-19T08:43:01.558 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "smtp_sender", 2024-09-19T08:43:01.558 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.558 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.558 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.558 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.558 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.558 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.558 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.559 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "SMTP envelope sender", 2024-09-19T08:43:01.559 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.559 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.559 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.559 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.559 INFO:teuthology.orchestra.run.smithi098.stdout: "smtp_ssl": { 2024-09-19T08:43:01.559 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "smtp_ssl", 2024-09-19T08:43:01.559 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.559 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.559 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.559 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.559 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.560 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.560 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.560 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T08:43:01.560 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.560 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.560 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.560 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.560 INFO:teuthology.orchestra.run.smithi098.stdout: "smtp_user": { 2024-09-19T08:43:01.560 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "smtp_user", 2024-09-19T08:43:01.560 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.560 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.560 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.560 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.561 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.561 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.561 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.561 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "User to authenticate as", 2024-09-19T08:43:01.561 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.561 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.561 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.561 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.561 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.561 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.561 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:01.561 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "balancer", 2024-09-19T08:43:01.561 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:01.561 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:01.562 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:01.562 INFO:teuthology.orchestra.run.smithi098.stdout: "active": { 2024-09-19T08:43:01.562 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "active", 2024-09-19T08:43:01.562 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.562 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.562 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.562 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.562 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.562 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.562 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.562 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T08:43:01.562 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.562 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.563 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.563 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.563 INFO:teuthology.orchestra.run.smithi098.stdout: "begin_time": { 2024-09-19T08:43:01.563 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "begin_time", 2024-09-19T08:43:01.563 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.563 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.563 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.563 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "0000", 2024-09-19T08:43:01.563 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.563 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.563 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.563 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T08:43:01.563 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T08:43:01.563 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.564 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.564 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.564 INFO:teuthology.orchestra.run.smithi098.stdout: "begin_weekday": { 2024-09-19T08:43:01.564 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "begin_weekday", 2024-09-19T08:43:01.564 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "uint", 2024-09-19T08:43:01.564 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.564 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.564 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "0", 2024-09-19T08:43:01.564 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "0", 2024-09-19T08:43:01.564 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "6", 2024-09-19T08:43:01.564 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.564 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T08:43:01.565 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T08:43:01.565 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.565 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.565 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.565 INFO:teuthology.orchestra.run.smithi098.stdout: "crush_compat_max_iterations": { 2024-09-19T08:43:01.565 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "crush_compat_max_iterations", 2024-09-19T08:43:01.565 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "uint", 2024-09-19T08:43:01.565 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.565 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.565 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "25", 2024-09-19T08:43:01.565 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "1", 2024-09-19T08:43:01.565 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "250", 2024-09-19T08:43:01.565 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.566 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T08:43:01.566 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.566 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.566 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.566 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.566 INFO:teuthology.orchestra.run.smithi098.stdout: "crush_compat_metrics": { 2024-09-19T08:43:01.566 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "crush_compat_metrics", 2024-09-19T08:43:01.566 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.566 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.566 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.566 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T08:43:01.566 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.566 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.566 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.567 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T08:43:01.567 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T08:43:01.567 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.567 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.567 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.567 INFO:teuthology.orchestra.run.smithi098.stdout: "crush_compat_step": { 2024-09-19T08:43:01.567 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "crush_compat_step", 2024-09-19T08:43:01.567 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "float", 2024-09-19T08:43:01.567 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.567 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.567 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "0.5", 2024-09-19T08:43:01.567 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "0.001", 2024-09-19T08:43:01.567 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "0.999", 2024-09-19T08:43:01.568 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.568 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "aggressiveness of optimization", 2024-09-19T08:43:01.568 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T08:43:01.568 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.568 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.568 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.568 INFO:teuthology.orchestra.run.smithi098.stdout: "end_time": { 2024-09-19T08:43:01.568 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "end_time", 2024-09-19T08:43:01.568 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.568 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.568 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.568 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "2359", 2024-09-19T08:43:01.568 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.568 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.569 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.569 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T08:43:01.569 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T08:43:01.569 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.569 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.569 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.569 INFO:teuthology.orchestra.run.smithi098.stdout: "end_weekday": { 2024-09-19T08:43:01.569 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "end_weekday", 2024-09-19T08:43:01.569 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "uint", 2024-09-19T08:43:01.569 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.569 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.569 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "0", 2024-09-19T08:43:01.569 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "0", 2024-09-19T08:43:01.569 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "6", 2024-09-19T08:43:01.570 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.570 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T08:43:01.570 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T08:43:01.570 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.570 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.570 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.570 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:01.570 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:01.570 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.570 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.570 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.570 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.570 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.571 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.571 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.571 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.571 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.571 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.571 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.571 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.571 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.571 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.571 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.571 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.571 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.572 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.572 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.572 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:01.572 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:01.572 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.572 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.572 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.572 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.572 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.572 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.572 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.572 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.573 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.573 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.573 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.573 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.573 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:01.573 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:01.573 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.573 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.573 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.573 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:01.573 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.573 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.573 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.573 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.574 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.574 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.574 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.574 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.574 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.574 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.574 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.574 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.574 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.574 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.574 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.574 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:01.574 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:01.574 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.575 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.575 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.575 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.575 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.575 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.575 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.575 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.575 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.575 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.575 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.575 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.575 INFO:teuthology.orchestra.run.smithi098.stdout: "min_score": { 2024-09-19T08:43:01.575 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "min_score", 2024-09-19T08:43:01.576 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "float", 2024-09-19T08:43:01.576 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.576 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.576 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "0", 2024-09-19T08:43:01.576 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.576 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.576 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.576 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T08:43:01.576 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.576 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.576 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.576 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.576 INFO:teuthology.orchestra.run.smithi098.stdout: "mode": { 2024-09-19T08:43:01.577 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "mode", 2024-09-19T08:43:01.577 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.577 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.577 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.577 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "upmap", 2024-09-19T08:43:01.577 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.577 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.577 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.577 INFO:teuthology.orchestra.run.smithi098.stdout: "crush-compat", 2024-09-19T08:43:01.577 INFO:teuthology.orchestra.run.smithi098.stdout: "none", 2024-09-19T08:43:01.577 INFO:teuthology.orchestra.run.smithi098.stdout: "read", 2024-09-19T08:43:01.577 INFO:teuthology.orchestra.run.smithi098.stdout: "upmap", 2024-09-19T08:43:01.577 INFO:teuthology.orchestra.run.smithi098.stdout: "upmap-read" 2024-09-19T08:43:01.577 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.578 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Balancer mode", 2024-09-19T08:43:01.578 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.578 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.578 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.578 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.578 INFO:teuthology.orchestra.run.smithi098.stdout: "pool_ids": { 2024-09-19T08:43:01.578 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "pool_ids", 2024-09-19T08:43:01.578 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.578 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.578 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.578 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.578 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.578 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.579 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.579 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T08:43:01.579 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.579 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.579 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.579 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.579 INFO:teuthology.orchestra.run.smithi098.stdout: "sleep_interval": { 2024-09-19T08:43:01.579 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "sleep_interval", 2024-09-19T08:43:01.579 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:01.579 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.579 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.579 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "60", 2024-09-19T08:43:01.579 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.579 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.580 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.580 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T08:43:01.580 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.580 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.580 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.580 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.580 INFO:teuthology.orchestra.run.smithi098.stdout: "upmap_max_deviation": { 2024-09-19T08:43:01.580 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "upmap_max_deviation", 2024-09-19T08:43:01.580 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.580 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.580 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.580 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "5", 2024-09-19T08:43:01.580 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "1", 2024-09-19T08:43:01.580 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.581 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.581 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T08:43:01.581 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T08:43:01.581 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.581 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.581 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.581 INFO:teuthology.orchestra.run.smithi098.stdout: "upmap_max_optimizations": { 2024-09-19T08:43:01.581 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "upmap_max_optimizations", 2024-09-19T08:43:01.581 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "uint", 2024-09-19T08:43:01.581 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.581 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.581 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "10", 2024-09-19T08:43:01.581 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.582 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.582 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.582 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T08:43:01.582 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.582 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.582 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.582 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.582 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.582 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.582 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:01.582 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "cephadm", 2024-09-19T08:43:01.582 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:01.582 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:01.582 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:01.583 INFO:teuthology.orchestra.run.smithi098.stdout: "agent_down_multiplier": { 2024-09-19T08:43:01.583 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "agent_down_multiplier", 2024-09-19T08:43:01.583 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "float", 2024-09-19T08:43:01.583 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.583 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.583 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "3.0", 2024-09-19T08:43:01.583 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.583 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.583 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.583 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T08:43:01.583 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.583 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.583 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.583 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.584 INFO:teuthology.orchestra.run.smithi098.stdout: "agent_refresh_rate": { 2024-09-19T08:43:01.584 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "agent_refresh_rate", 2024-09-19T08:43:01.584 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:01.584 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.584 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.584 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "20", 2024-09-19T08:43:01.584 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.584 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.584 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.584 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T08:43:01.584 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.584 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.584 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.584 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.585 INFO:teuthology.orchestra.run.smithi098.stdout: "agent_starting_port": { 2024-09-19T08:43:01.585 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "agent_starting_port", 2024-09-19T08:43:01.585 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.585 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.585 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.585 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "4721", 2024-09-19T08:43:01.585 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.585 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.585 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.585 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T08:43:01.585 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.585 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.585 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.585 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.586 INFO:teuthology.orchestra.run.smithi098.stdout: "allow_ptrace": { 2024-09-19T08:43:01.586 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "allow_ptrace", 2024-09-19T08:43:01.586 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.586 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.586 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.586 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.586 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.586 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.586 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.586 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T08:43:01.586 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T08:43:01.586 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.586 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.587 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.587 INFO:teuthology.orchestra.run.smithi098.stdout: "autotune_interval": { 2024-09-19T08:43:01.587 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "autotune_interval", 2024-09-19T08:43:01.587 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:01.587 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.587 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.587 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "600", 2024-09-19T08:43:01.587 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.587 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.587 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.587 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T08:43:01.587 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.587 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.588 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.588 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.588 INFO:teuthology.orchestra.run.smithi098.stdout: "autotune_memory_target_ratio": { 2024-09-19T08:43:01.588 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T08:43:01.588 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "float", 2024-09-19T08:43:01.588 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.588 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.588 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "0.7", 2024-09-19T08:43:01.588 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.588 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.588 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.588 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T08:43:01.588 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.588 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.589 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.589 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.589 INFO:teuthology.orchestra.run.smithi098.stdout: "cephadm_log_destination": { 2024-09-19T08:43:01.589 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "cephadm_log_destination", 2024-09-19T08:43:01.589 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.589 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.589 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.589 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.589 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.589 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.589 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.589 INFO:teuthology.orchestra.run.smithi098.stdout: "file", 2024-09-19T08:43:01.589 INFO:teuthology.orchestra.run.smithi098.stdout: "file,syslog", 2024-09-19T08:43:01.590 INFO:teuthology.orchestra.run.smithi098.stdout: "syslog" 2024-09-19T08:43:01.590 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.590 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T08:43:01.590 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.590 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.590 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.590 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.590 INFO:teuthology.orchestra.run.smithi098.stdout: "cgroups_split": { 2024-09-19T08:43:01.590 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "cgroups_split", 2024-09-19T08:43:01.590 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.590 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.590 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.590 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.590 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.591 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.591 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.591 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T08:43:01.591 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.591 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.591 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.591 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.591 INFO:teuthology.orchestra.run.smithi098.stdout: "config_checks_enabled": { 2024-09-19T08:43:01.591 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "config_checks_enabled", 2024-09-19T08:43:01.591 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.591 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.591 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.591 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.592 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.592 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.592 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.592 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T08:43:01.592 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.592 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.592 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.592 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.592 INFO:teuthology.orchestra.run.smithi098.stdout: "config_dashboard": { 2024-09-19T08:43:01.592 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "config_dashboard", 2024-09-19T08:43:01.592 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.592 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.592 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.593 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.593 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.593 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.593 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.593 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T08:43:01.593 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.593 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.593 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.593 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.593 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_alertmanager": { 2024-09-19T08:43:01.593 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_alertmanager", 2024-09-19T08:43:01.593 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.593 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.593 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.594 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T08:43:01.594 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.594 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.594 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.594 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Prometheus container image", 2024-09-19T08:43:01.594 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.594 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.594 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.594 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.594 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_base": { 2024-09-19T08:43:01.594 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_base", 2024-09-19T08:43:01.594 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.594 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.594 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.595 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T08:43:01.595 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.595 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.595 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.595 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Container image name, without the tag", 2024-09-19T08:43:01.595 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.595 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.595 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.595 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.595 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_elasticsearch": { 2024-09-19T08:43:01.595 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_elasticsearch", 2024-09-19T08:43:01.595 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.595 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.596 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.596 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T08:43:01.596 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.596 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.596 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.596 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "elasticsearch container image", 2024-09-19T08:43:01.596 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.596 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.596 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.596 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.596 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_grafana": { 2024-09-19T08:43:01.596 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_grafana", 2024-09-19T08:43:01.596 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.596 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.597 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.597 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T08:43:01.597 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.597 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.597 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.597 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Prometheus container image", 2024-09-19T08:43:01.597 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.597 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.597 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.597 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.597 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_haproxy": { 2024-09-19T08:43:01.597 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_haproxy", 2024-09-19T08:43:01.597 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.598 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.598 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.598 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T08:43:01.598 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.598 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.598 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.598 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "HAproxy container image", 2024-09-19T08:43:01.598 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.598 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.598 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.598 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.598 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_jaeger_agent": { 2024-09-19T08:43:01.598 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_jaeger_agent", 2024-09-19T08:43:01.599 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.599 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.599 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.599 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T08:43:01.599 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.599 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.599 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.599 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Jaeger agent container image", 2024-09-19T08:43:01.599 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.599 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.599 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.599 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.599 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_jaeger_collector": { 2024-09-19T08:43:01.600 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_jaeger_collector", 2024-09-19T08:43:01.600 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.600 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.600 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.600 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T08:43:01.600 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.600 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.600 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.600 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Jaeger collector container image", 2024-09-19T08:43:01.600 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.600 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.600 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.600 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.600 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_jaeger_query": { 2024-09-19T08:43:01.601 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_jaeger_query", 2024-09-19T08:43:01.601 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.601 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.601 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.601 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T08:43:01.601 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.601 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.601 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.601 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Jaeger query container image", 2024-09-19T08:43:01.601 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.601 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.601 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.601 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.602 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_keepalived": { 2024-09-19T08:43:01.602 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_keepalived", 2024-09-19T08:43:01.602 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.602 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.602 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.602 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T08:43:01.602 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.602 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.602 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.602 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Keepalived container image", 2024-09-19T08:43:01.602 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.602 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.602 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.603 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.603 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_loki": { 2024-09-19T08:43:01.603 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_loki", 2024-09-19T08:43:01.603 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.603 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.603 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.603 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T08:43:01.603 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.603 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.603 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.603 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Loki container image", 2024-09-19T08:43:01.603 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.603 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.603 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.604 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.604 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_node_exporter": { 2024-09-19T08:43:01.604 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_node_exporter", 2024-09-19T08:43:01.604 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.604 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.604 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.604 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T08:43:01.604 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.604 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.604 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.604 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Prometheus container image", 2024-09-19T08:43:01.604 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.604 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.605 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.605 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.605 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_nvmeof": { 2024-09-19T08:43:01.605 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_nvmeof", 2024-09-19T08:43:01.605 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.605 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.605 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.605 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T08:43:01.605 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.605 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.605 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.605 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Nvme-of container image", 2024-09-19T08:43:01.605 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.605 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.606 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.606 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.606 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_prometheus": { 2024-09-19T08:43:01.606 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_prometheus", 2024-09-19T08:43:01.606 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.606 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.606 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.606 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T08:43:01.606 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.606 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.606 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.606 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Prometheus container image", 2024-09-19T08:43:01.606 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.606 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.607 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.607 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.607 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_promtail": { 2024-09-19T08:43:01.607 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_promtail", 2024-09-19T08:43:01.607 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.607 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.607 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.607 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T08:43:01.607 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.607 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.607 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.607 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Promtail container image", 2024-09-19T08:43:01.607 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.607 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.608 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.608 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.608 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_samba": { 2024-09-19T08:43:01.608 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_samba", 2024-09-19T08:43:01.608 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.608 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.608 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.608 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T08:43:01.608 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.608 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.608 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.608 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Samba/SMB container image", 2024-09-19T08:43:01.608 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.609 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.609 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.609 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.609 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_snmp_gateway": { 2024-09-19T08:43:01.609 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_snmp_gateway", 2024-09-19T08:43:01.609 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.609 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.609 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.609 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T08:43:01.609 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.609 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.609 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.609 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "SNMP Gateway container image", 2024-09-19T08:43:01.609 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.610 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.610 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.610 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.610 INFO:teuthology.orchestra.run.smithi098.stdout: "container_init": { 2024-09-19T08:43:01.610 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_init", 2024-09-19T08:43:01.610 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.610 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.610 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.610 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.610 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.610 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.610 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.610 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T08:43:01.611 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.611 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.611 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.611 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.611 INFO:teuthology.orchestra.run.smithi098.stdout: "daemon_cache_timeout": { 2024-09-19T08:43:01.611 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "daemon_cache_timeout", 2024-09-19T08:43:01.611 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:01.611 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.611 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.611 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "600", 2024-09-19T08:43:01.611 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.611 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.611 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.611 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T08:43:01.612 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.612 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.612 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.612 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.612 INFO:teuthology.orchestra.run.smithi098.stdout: "default_cephadm_command_timeout": { 2024-09-19T08:43:01.612 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T08:43:01.612 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.612 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.612 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.612 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "900", 2024-09-19T08:43:01.612 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.612 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.612 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.613 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T08:43:01.613 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.613 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.613 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.613 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.613 INFO:teuthology.orchestra.run.smithi098.stdout: "default_registry": { 2024-09-19T08:43:01.613 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "default_registry", 2024-09-19T08:43:01.613 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.613 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.613 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.613 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "docker.io", 2024-09-19T08:43:01.613 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.613 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.613 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.614 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T08:43:01.614 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.614 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.614 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.614 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.614 INFO:teuthology.orchestra.run.smithi098.stdout: "device_cache_timeout": { 2024-09-19T08:43:01.614 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "device_cache_timeout", 2024-09-19T08:43:01.614 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:01.614 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.614 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.614 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "1800", 2024-09-19T08:43:01.614 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.614 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.614 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.615 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "seconds to cache device inventory", 2024-09-19T08:43:01.615 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.615 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.615 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.615 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.615 INFO:teuthology.orchestra.run.smithi098.stdout: "device_enhanced_scan": { 2024-09-19T08:43:01.615 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "device_enhanced_scan", 2024-09-19T08:43:01.615 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.615 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.615 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.615 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.615 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.615 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.615 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.616 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T08:43:01.616 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.616 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.616 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.616 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.616 INFO:teuthology.orchestra.run.smithi098.stdout: "facts_cache_timeout": { 2024-09-19T08:43:01.616 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "facts_cache_timeout", 2024-09-19T08:43:01.616 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:01.616 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.616 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.616 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "60", 2024-09-19T08:43:01.616 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.617 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.617 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.617 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "seconds to cache host facts data", 2024-09-19T08:43:01.617 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.617 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.617 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.617 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.617 INFO:teuthology.orchestra.run.smithi098.stdout: "grafana_dashboards_path": { 2024-09-19T08:43:01.617 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "grafana_dashboards_path", 2024-09-19T08:43:01.617 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.617 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.617 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.617 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T08:43:01.618 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.618 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.618 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.618 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T08:43:01.618 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.618 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.618 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.618 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.618 INFO:teuthology.orchestra.run.smithi098.stdout: "host_check_interval": { 2024-09-19T08:43:01.618 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "host_check_interval", 2024-09-19T08:43:01.618 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:01.618 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.618 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.618 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "600", 2024-09-19T08:43:01.619 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.619 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.619 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.619 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "how frequently to perform a host check", 2024-09-19T08:43:01.619 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.619 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.619 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.619 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.619 INFO:teuthology.orchestra.run.smithi098.stdout: "hw_monitoring": { 2024-09-19T08:43:01.619 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "hw_monitoring", 2024-09-19T08:43:01.619 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.619 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.619 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.619 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.620 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.620 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.620 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.620 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T08:43:01.620 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.620 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.620 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.620 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.620 INFO:teuthology.orchestra.run.smithi098.stdout: "inventory_list_all": { 2024-09-19T08:43:01.620 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "inventory_list_all", 2024-09-19T08:43:01.620 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.620 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.620 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.620 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.621 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.621 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.621 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.621 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T08:43:01.621 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.621 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.621 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.621 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.621 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:01.621 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:01.621 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.621 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.621 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.621 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.622 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.622 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.622 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.622 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.622 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.622 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.622 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.622 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.622 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.622 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.622 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.622 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.622 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.622 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.623 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.623 INFO:teuthology.orchestra.run.smithi098.stdout: "log_refresh_metadata": { 2024-09-19T08:43:01.623 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_refresh_metadata", 2024-09-19T08:43:01.623 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.623 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.623 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.623 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.623 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.623 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.623 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.623 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T08:43:01.623 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.623 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.624 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.624 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.624 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:01.624 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:01.624 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.624 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.624 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.624 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.624 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.624 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.624 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.624 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T08:43:01.624 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.625 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.625 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.625 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.625 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:01.625 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:01.625 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.625 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.625 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.625 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:01.625 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.625 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.625 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.625 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.625 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.626 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.626 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.626 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.626 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.626 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.626 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.626 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.626 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.626 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.626 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.626 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:01.626 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:01.626 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.626 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.627 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.627 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.627 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.627 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.627 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.627 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.627 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.627 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.627 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.627 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.627 INFO:teuthology.orchestra.run.smithi098.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T08:43:01.627 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T08:43:01.627 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.628 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.628 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.628 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.628 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.628 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.628 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.628 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T08:43:01.628 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.628 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.628 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.628 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.628 INFO:teuthology.orchestra.run.smithi098.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T08:43:01.628 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T08:43:01.629 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.629 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.629 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.629 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "*", 2024-09-19T08:43:01.629 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.629 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.629 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.629 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T08:43:01.629 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.629 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.629 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.629 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.629 INFO:teuthology.orchestra.run.smithi098.stdout: "max_count_per_host": { 2024-09-19T08:43:01.629 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "max_count_per_host", 2024-09-19T08:43:01.630 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.630 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.630 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.630 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "10", 2024-09-19T08:43:01.630 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.630 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.630 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.630 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "max number of daemons per service per host", 2024-09-19T08:43:01.630 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.630 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.630 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.630 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.630 INFO:teuthology.orchestra.run.smithi098.stdout: "max_osd_draining_count": { 2024-09-19T08:43:01.630 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "max_osd_draining_count", 2024-09-19T08:43:01.631 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.631 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.631 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.631 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "10", 2024-09-19T08:43:01.631 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.631 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.631 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.631 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T08:43:01.631 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.631 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.631 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.631 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.631 INFO:teuthology.orchestra.run.smithi098.stdout: "migration_current": { 2024-09-19T08:43:01.632 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "migration_current", 2024-09-19T08:43:01.632 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.632 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.632 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.632 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.632 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.632 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.632 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.632 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "internal - do not modify", 2024-09-19T08:43:01.632 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.632 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.632 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.632 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.633 INFO:teuthology.orchestra.run.smithi098.stdout: "mode": { 2024-09-19T08:43:01.633 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "mode", 2024-09-19T08:43:01.633 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.633 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.633 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.633 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "root", 2024-09-19T08:43:01.633 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.633 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.633 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.633 INFO:teuthology.orchestra.run.smithi098.stdout: "cephadm-package", 2024-09-19T08:43:01.633 INFO:teuthology.orchestra.run.smithi098.stdout: "root" 2024-09-19T08:43:01.633 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.633 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T08:43:01.633 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.634 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.634 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.634 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.634 INFO:teuthology.orchestra.run.smithi098.stdout: "oob_default_addr": { 2024-09-19T08:43:01.634 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "oob_default_addr", 2024-09-19T08:43:01.634 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.634 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.634 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.634 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "169.254.1.1", 2024-09-19T08:43:01.634 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.634 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.634 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.635 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T08:43:01.635 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.635 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.635 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.635 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.635 INFO:teuthology.orchestra.run.smithi098.stdout: "prometheus_alerts_path": { 2024-09-19T08:43:01.635 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "prometheus_alerts_path", 2024-09-19T08:43:01.635 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.635 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.635 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.635 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T08:43:01.635 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.635 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.636 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.636 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T08:43:01.636 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.636 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.636 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.636 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.636 INFO:teuthology.orchestra.run.smithi098.stdout: "registry_insecure": { 2024-09-19T08:43:01.636 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "registry_insecure", 2024-09-19T08:43:01.636 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.636 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.636 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.636 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.636 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.637 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.637 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.637 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T08:43:01.637 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.637 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.637 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.637 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.637 INFO:teuthology.orchestra.run.smithi098.stdout: "registry_password": { 2024-09-19T08:43:01.637 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "registry_password", 2024-09-19T08:43:01.637 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.637 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.637 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.637 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.638 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.638 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.638 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.638 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T08:43:01.638 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.638 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.638 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.638 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.638 INFO:teuthology.orchestra.run.smithi098.stdout: "registry_url": { 2024-09-19T08:43:01.638 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "registry_url", 2024-09-19T08:43:01.638 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.638 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.638 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.638 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.639 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.639 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.639 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.639 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T08:43:01.639 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.639 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.639 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.639 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.639 INFO:teuthology.orchestra.run.smithi098.stdout: "registry_username": { 2024-09-19T08:43:01.639 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "registry_username", 2024-09-19T08:43:01.639 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.639 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.639 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.639 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.640 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.640 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.640 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.640 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T08:43:01.640 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.640 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.640 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.640 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.640 INFO:teuthology.orchestra.run.smithi098.stdout: "secure_monitoring_stack": { 2024-09-19T08:43:01.640 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "secure_monitoring_stack", 2024-09-19T08:43:01.640 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.640 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.640 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.641 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.641 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.641 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.641 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.641 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T08:43:01.641 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.641 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.641 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.641 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.641 INFO:teuthology.orchestra.run.smithi098.stdout: "service_discovery_port": { 2024-09-19T08:43:01.641 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "service_discovery_port", 2024-09-19T08:43:01.641 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.641 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.641 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.642 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "8765", 2024-09-19T08:43:01.642 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.642 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.642 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.642 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "cephadm service discovery port", 2024-09-19T08:43:01.642 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.642 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.642 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.642 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.642 INFO:teuthology.orchestra.run.smithi098.stdout: "ssh_config_file": { 2024-09-19T08:43:01.642 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ssh_config_file", 2024-09-19T08:43:01.642 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.642 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.642 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.643 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.643 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.643 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.643 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.643 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T08:43:01.643 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.643 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.643 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.643 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.643 INFO:teuthology.orchestra.run.smithi098.stdout: "use_agent": { 2024-09-19T08:43:01.643 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "use_agent", 2024-09-19T08:43:01.643 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.643 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.643 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.644 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.644 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.644 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.644 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.644 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T08:43:01.644 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.644 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.644 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.644 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.644 INFO:teuthology.orchestra.run.smithi098.stdout: "use_repo_digest": { 2024-09-19T08:43:01.644 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "use_repo_digest", 2024-09-19T08:43:01.644 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.644 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.644 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.645 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.645 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.645 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.645 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.645 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T08:43:01.645 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.645 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.645 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.645 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.645 INFO:teuthology.orchestra.run.smithi098.stdout: "warn_on_failed_host_check": { 2024-09-19T08:43:01.645 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "warn_on_failed_host_check", 2024-09-19T08:43:01.645 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.645 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.646 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.646 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.646 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.646 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.646 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.646 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T08:43:01.646 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.646 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.646 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.646 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.646 INFO:teuthology.orchestra.run.smithi098.stdout: "warn_on_stray_daemons": { 2024-09-19T08:43:01.646 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "warn_on_stray_daemons", 2024-09-19T08:43:01.646 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.646 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.647 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.647 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.647 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.647 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.647 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.647 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T08:43:01.647 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.647 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.647 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.647 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.647 INFO:teuthology.orchestra.run.smithi098.stdout: "warn_on_stray_hosts": { 2024-09-19T08:43:01.647 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "warn_on_stray_hosts", 2024-09-19T08:43:01.647 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.647 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.648 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.648 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.648 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.648 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.648 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.648 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T08:43:01.648 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.648 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.648 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.648 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.648 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.648 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.648 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:01.648 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "crash", 2024-09-19T08:43:01.649 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:01.649 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:01.649 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:01.649 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:01.649 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:01.649 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.649 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.649 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.649 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.649 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.649 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.649 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.649 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.650 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.650 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.650 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.650 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.650 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.650 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.650 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.650 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.650 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.650 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.650 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.650 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:01.650 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:01.650 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.651 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.651 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.651 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.651 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.651 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.651 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.651 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.651 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.651 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.651 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.651 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.651 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:01.651 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:01.651 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.652 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.652 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.652 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:01.652 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.652 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.652 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.652 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.652 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.652 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.653 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.653 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.653 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.653 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.653 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.653 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.653 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.653 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.653 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.654 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:01.654 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:01.654 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.654 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.654 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.654 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.654 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.654 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.654 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.654 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.654 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.654 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.654 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.654 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.655 INFO:teuthology.orchestra.run.smithi098.stdout: "retain_interval": { 2024-09-19T08:43:01.655 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "retain_interval", 2024-09-19T08:43:01.655 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:01.655 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.655 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.655 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "31536000", 2024-09-19T08:43:01.655 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.655 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.655 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.655 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T08:43:01.655 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.655 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.655 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.656 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.656 INFO:teuthology.orchestra.run.smithi098.stdout: "warn_recent_interval": { 2024-09-19T08:43:01.656 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "warn_recent_interval", 2024-09-19T08:43:01.656 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:01.656 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.656 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.656 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "1209600", 2024-09-19T08:43:01.656 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.656 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.656 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.656 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T08:43:01.656 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.656 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.657 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.657 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.657 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.657 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.657 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:01.657 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "dashboard", 2024-09-19T08:43:01.657 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:01.657 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:01.657 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:01.657 INFO:teuthology.orchestra.run.smithi098.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T08:43:01.657 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T08:43:01.657 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.657 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.657 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.658 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "10", 2024-09-19T08:43:01.658 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.658 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.658 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.658 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.658 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.658 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.658 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.658 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.658 INFO:teuthology.orchestra.run.smithi098.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T08:43:01.658 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T08:43:01.658 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.658 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.658 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.659 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.659 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.659 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.659 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.659 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.659 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.659 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.659 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.659 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.659 INFO:teuthology.orchestra.run.smithi098.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T08:43:01.659 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T08:43:01.659 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.659 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.659 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.660 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.660 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.660 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.660 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.660 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.660 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.660 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.660 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.660 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.660 INFO:teuthology.orchestra.run.smithi098.stdout: "AUDIT_API_ENABLED": { 2024-09-19T08:43:01.660 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T08:43:01.660 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.660 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.660 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.661 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.661 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.661 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.661 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.661 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.661 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.661 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.661 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.661 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.661 INFO:teuthology.orchestra.run.smithi098.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T08:43:01.661 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T08:43:01.661 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.661 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.661 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.662 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.662 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.662 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.662 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.662 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.662 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.662 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.662 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.662 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.662 INFO:teuthology.orchestra.run.smithi098.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T08:43:01.662 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T08:43:01.662 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.662 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.663 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.663 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.663 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.663 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.663 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.663 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.663 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.663 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.663 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.663 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.663 INFO:teuthology.orchestra.run.smithi098.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T08:43:01.663 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T08:43:01.663 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.664 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.664 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.664 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.664 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.664 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.664 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.664 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.664 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.664 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.664 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.664 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.664 INFO:teuthology.orchestra.run.smithi098.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T08:43:01.664 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T08:43:01.664 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.665 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.665 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.665 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.665 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.665 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.665 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.665 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.665 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.665 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.665 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.665 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.665 INFO:teuthology.orchestra.run.smithi098.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T08:43:01.665 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T08:43:01.666 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.666 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.666 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.666 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.666 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.666 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.666 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.666 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.666 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.666 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.666 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.666 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.666 INFO:teuthology.orchestra.run.smithi098.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T08:43:01.666 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T08:43:01.667 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.667 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.667 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.667 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.667 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.667 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.667 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.667 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.667 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.667 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.667 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.667 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.667 INFO:teuthology.orchestra.run.smithi098.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T08:43:01.667 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T08:43:01.668 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.668 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.668 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.668 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.668 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.668 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.668 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.668 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.668 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.668 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.668 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.668 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.668 INFO:teuthology.orchestra.run.smithi098.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T08:43:01.669 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T08:43:01.669 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.669 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.669 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.669 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.669 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.669 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.669 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.669 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.669 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.669 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.669 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.669 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.669 INFO:teuthology.orchestra.run.smithi098.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T08:43:01.670 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T08:43:01.670 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.670 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.670 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.670 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.670 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.670 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.670 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.670 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.670 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.671 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.671 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.671 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.671 INFO:teuthology.orchestra.run.smithi098.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T08:43:01.671 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T08:43:01.671 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.671 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.671 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.672 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.672 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.672 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.672 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.672 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.672 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.672 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.672 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.672 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.672 INFO:teuthology.orchestra.run.smithi098.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T08:43:01.672 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T08:43:01.672 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.673 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.673 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.673 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "admin", 2024-09-19T08:43:01.673 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.673 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.673 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.673 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.673 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.673 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.673 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.673 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.673 INFO:teuthology.orchestra.run.smithi098.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T08:43:01.673 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T08:43:01.673 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.674 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.674 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.674 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.674 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.674 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.674 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.674 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.674 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.674 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.674 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.674 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.674 INFO:teuthology.orchestra.run.smithi098.stdout: "GRAFANA_API_URL": { 2024-09-19T08:43:01.674 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "GRAFANA_API_URL", 2024-09-19T08:43:01.674 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.675 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.675 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.675 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.675 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.675 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.675 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.675 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.675 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.675 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.675 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.675 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.675 INFO:teuthology.orchestra.run.smithi098.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T08:43:01.675 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T08:43:01.675 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.676 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.676 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.676 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "admin", 2024-09-19T08:43:01.676 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.676 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.676 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.676 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.676 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.676 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.676 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.676 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.676 INFO:teuthology.orchestra.run.smithi098.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T08:43:01.676 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T08:43:01.677 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.677 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.677 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.677 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.677 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.677 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.677 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.677 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.677 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.677 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.677 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.677 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.677 INFO:teuthology.orchestra.run.smithi098.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T08:43:01.678 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T08:43:01.678 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.678 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.678 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.678 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.678 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.678 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.678 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.678 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.678 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.678 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.678 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.678 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.678 INFO:teuthology.orchestra.run.smithi098.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T08:43:01.679 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T08:43:01.679 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.679 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.679 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.679 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.679 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.679 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.679 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.679 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.679 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.679 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.679 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.679 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.679 INFO:teuthology.orchestra.run.smithi098.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T08:43:01.680 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T08:43:01.680 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.680 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.680 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.680 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.680 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.680 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.680 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.680 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.680 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.680 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.680 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.680 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.681 INFO:teuthology.orchestra.run.smithi098.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T08:43:01.681 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T08:43:01.681 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.681 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.681 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.681 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.681 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.681 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.681 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.681 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.681 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.681 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.681 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.682 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.682 INFO:teuthology.orchestra.run.smithi098.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T08:43:01.682 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T08:43:01.682 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.682 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.682 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.682 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.682 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.682 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.682 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.682 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.682 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.682 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.682 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.683 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.683 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T08:43:01.683 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T08:43:01.683 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.683 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.683 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.683 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.683 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.683 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.683 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.683 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.683 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.683 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.683 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.684 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.684 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T08:43:01.684 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T08:43:01.684 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.684 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.684 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.684 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.684 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.684 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.684 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.684 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.684 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.684 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.685 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.685 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.685 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T08:43:01.685 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T08:43:01.685 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.685 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.685 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.685 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.685 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.685 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.685 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.685 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.685 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.685 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.686 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.686 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.686 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T08:43:01.686 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T08:43:01.686 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.686 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.686 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.686 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.686 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.686 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.686 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.686 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.686 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.686 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.687 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.687 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.687 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T08:43:01.687 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T08:43:01.687 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.687 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.687 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.687 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.687 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.687 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.687 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.687 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.687 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.687 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.688 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.688 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.688 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T08:43:01.688 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T08:43:01.688 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.688 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.688 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.688 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.688 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.688 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.688 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.688 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.689 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.689 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.689 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.689 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.689 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T08:43:01.689 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T08:43:01.689 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.689 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.689 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.689 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.689 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.689 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.689 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.689 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.690 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.690 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.690 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.690 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.690 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T08:43:01.690 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T08:43:01.690 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.690 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.690 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.690 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.690 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.690 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.690 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.690 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.691 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.691 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.691 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.691 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.691 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T08:43:01.691 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T08:43:01.691 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.691 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.691 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.691 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T08:43:01.691 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.691 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.691 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.692 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.692 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.692 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.692 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.692 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.692 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T08:43:01.692 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T08:43:01.692 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.692 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.692 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.692 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "10", 2024-09-19T08:43:01.692 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.692 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.692 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.693 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.693 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.693 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.693 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.693 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.693 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T08:43:01.693 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T08:43:01.693 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.693 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.693 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.693 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "8", 2024-09-19T08:43:01.693 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.693 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.694 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.694 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.694 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.694 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.694 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.694 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.694 INFO:teuthology.orchestra.run.smithi098.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T08:43:01.694 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T08:43:01.694 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.694 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.694 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.694 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "45", 2024-09-19T08:43:01.694 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.695 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.695 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.695 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.695 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.695 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.695 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.695 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.695 INFO:teuthology.orchestra.run.smithi098.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T08:43:01.695 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T08:43:01.695 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.695 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.695 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.695 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.695 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.696 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.696 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.696 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.696 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.696 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.696 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.696 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.696 INFO:teuthology.orchestra.run.smithi098.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T08:43:01.696 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T08:43:01.696 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.696 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.696 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.696 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "admin", 2024-09-19T08:43:01.697 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.697 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.697 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.697 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.697 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.697 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.697 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.697 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.697 INFO:teuthology.orchestra.run.smithi098.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T08:43:01.697 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T08:43:01.697 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.697 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.697 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.697 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.698 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.698 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.698 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.698 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.698 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.698 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.698 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.698 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.698 INFO:teuthology.orchestra.run.smithi098.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T08:43:01.698 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T08:43:01.698 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.698 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.698 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.698 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.699 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.699 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.699 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.699 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.699 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.699 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.699 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.699 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.699 INFO:teuthology.orchestra.run.smithi098.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T08:43:01.699 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T08:43:01.699 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.699 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.699 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.699 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "0", 2024-09-19T08:43:01.700 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.700 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.700 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.700 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.700 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.700 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.700 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.700 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.700 INFO:teuthology.orchestra.run.smithi098.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T08:43:01.700 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T08:43:01.700 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.700 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.700 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.701 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "10", 2024-09-19T08:43:01.701 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.701 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.701 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.701 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.701 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.701 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.701 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.701 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.701 INFO:teuthology.orchestra.run.smithi098.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T08:43:01.701 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T08:43:01.701 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.701 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.701 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.702 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "5", 2024-09-19T08:43:01.702 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.702 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.702 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.702 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.702 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.702 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.702 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.702 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.702 INFO:teuthology.orchestra.run.smithi098.stdout: "cross_origin_url": { 2024-09-19T08:43:01.702 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "cross_origin_url", 2024-09-19T08:43:01.702 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.702 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.702 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.702 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.703 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.703 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.703 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.703 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.703 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.703 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.703 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.703 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.703 INFO:teuthology.orchestra.run.smithi098.stdout: "crt_file": { 2024-09-19T08:43:01.703 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "crt_file", 2024-09-19T08:43:01.703 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.703 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.703 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.703 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.704 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.704 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.704 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.704 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.704 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.704 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.704 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.704 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.704 INFO:teuthology.orchestra.run.smithi098.stdout: "debug": { 2024-09-19T08:43:01.704 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "debug", 2024-09-19T08:43:01.704 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.704 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.704 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.705 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.705 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.705 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.705 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.705 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Enable/disable debug options", 2024-09-19T08:43:01.705 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.705 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.705 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.705 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.705 INFO:teuthology.orchestra.run.smithi098.stdout: "jwt_token_ttl": { 2024-09-19T08:43:01.705 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "jwt_token_ttl", 2024-09-19T08:43:01.705 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.705 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.705 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.706 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "28800", 2024-09-19T08:43:01.706 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.706 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.706 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.706 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.706 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.706 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.706 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.706 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.706 INFO:teuthology.orchestra.run.smithi098.stdout: "key_file": { 2024-09-19T08:43:01.706 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "key_file", 2024-09-19T08:43:01.706 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.706 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.707 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.707 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.707 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.707 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.707 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.707 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.707 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.707 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.707 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.707 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.707 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:01.707 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:01.707 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.707 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.708 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.708 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.708 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.708 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.708 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.708 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.708 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.708 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.708 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.708 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.708 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.708 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.708 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.709 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.709 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.709 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.709 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.709 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:01.709 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:01.709 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.709 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.709 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.709 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.709 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.709 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.709 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.709 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.710 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.710 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.710 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.710 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.710 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:01.710 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:01.710 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.710 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.710 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.710 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:01.710 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.710 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.710 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.710 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.711 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.711 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.711 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.711 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.711 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.711 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.711 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.711 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.711 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.711 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.711 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.711 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:01.711 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:01.711 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.712 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.712 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.712 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.712 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.712 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.712 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.712 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.712 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.712 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.712 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.712 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.712 INFO:teuthology.orchestra.run.smithi098.stdout: "motd": { 2024-09-19T08:43:01.712 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "motd", 2024-09-19T08:43:01.712 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.713 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.713 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.713 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.713 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.713 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.713 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.713 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "The message of the day", 2024-09-19T08:43:01.713 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.713 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.713 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.713 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.713 INFO:teuthology.orchestra.run.smithi098.stdout: "redirect_resolve_ip_addr": { 2024-09-19T08:43:01.713 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T08:43:01.713 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.714 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.714 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.714 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.714 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.714 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.714 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.714 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.714 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.714 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.714 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.714 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.714 INFO:teuthology.orchestra.run.smithi098.stdout: "server_addr": { 2024-09-19T08:43:01.714 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "server_addr", 2024-09-19T08:43:01.715 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.715 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.715 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.715 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "::", 2024-09-19T08:43:01.715 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.715 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.715 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.715 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.715 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.715 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.715 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.715 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.715 INFO:teuthology.orchestra.run.smithi098.stdout: "server_port": { 2024-09-19T08:43:01.715 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "server_port", 2024-09-19T08:43:01.716 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.716 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.716 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.716 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "8080", 2024-09-19T08:43:01.716 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.716 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.716 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.716 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.716 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.716 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.716 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.716 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.716 INFO:teuthology.orchestra.run.smithi098.stdout: "ssl": { 2024-09-19T08:43:01.717 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ssl", 2024-09-19T08:43:01.717 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.717 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.717 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.717 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.717 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.717 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.717 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.717 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.717 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.717 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.717 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.717 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.717 INFO:teuthology.orchestra.run.smithi098.stdout: "ssl_server_port": { 2024-09-19T08:43:01.717 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ssl_server_port", 2024-09-19T08:43:01.718 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.718 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.718 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.718 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "8443", 2024-09-19T08:43:01.718 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.718 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.718 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.718 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.718 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.718 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.718 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.718 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.719 INFO:teuthology.orchestra.run.smithi098.stdout: "standby_behaviour": { 2024-09-19T08:43:01.719 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "standby_behaviour", 2024-09-19T08:43:01.719 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.719 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.719 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.719 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "redirect", 2024-09-19T08:43:01.719 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.719 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.719 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.719 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.719 INFO:teuthology.orchestra.run.smithi098.stdout: "redirect" 2024-09-19T08:43:01.719 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.719 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.719 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.720 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.720 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.720 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.720 INFO:teuthology.orchestra.run.smithi098.stdout: "standby_error_status_code": { 2024-09-19T08:43:01.720 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "standby_error_status_code", 2024-09-19T08:43:01.720 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.720 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.720 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.720 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "500", 2024-09-19T08:43:01.720 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "400", 2024-09-19T08:43:01.720 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "599", 2024-09-19T08:43:01.720 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.720 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.720 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.721 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.721 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.721 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.721 INFO:teuthology.orchestra.run.smithi098.stdout: "url_prefix": { 2024-09-19T08:43:01.721 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "url_prefix", 2024-09-19T08:43:01.721 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.721 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.721 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.721 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.721 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.721 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.721 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.721 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.721 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.722 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.722 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.722 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.722 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.722 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.722 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:01.722 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "devicehealth", 2024-09-19T08:43:01.722 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:01.722 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:01.722 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:01.722 INFO:teuthology.orchestra.run.smithi098.stdout: "enable_monitoring": { 2024-09-19T08:43:01.722 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "enable_monitoring", 2024-09-19T08:43:01.722 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.722 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.723 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.723 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.723 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.723 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.723 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.723 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "monitor device health metrics", 2024-09-19T08:43:01.723 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.723 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.723 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.723 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.723 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:01.723 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:01.723 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.723 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.724 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.724 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.724 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.724 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.724 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.724 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.724 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.724 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.724 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.724 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.724 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.724 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.724 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.725 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.725 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.725 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.725 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.725 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:01.725 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:01.725 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.725 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.725 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.725 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.725 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.725 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.725 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.725 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.726 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.726 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.726 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.726 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.726 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:01.726 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:01.726 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.726 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.726 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.726 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:01.726 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.726 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.726 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.726 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.727 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.727 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.727 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.727 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.727 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.727 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.727 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.727 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.727 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.727 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.727 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.727 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:01.727 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:01.727 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.728 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.728 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.728 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.728 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.728 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.728 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.728 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.728 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.728 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.728 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.728 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.728 INFO:teuthology.orchestra.run.smithi098.stdout: "mark_out_threshold": { 2024-09-19T08:43:01.729 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "mark_out_threshold", 2024-09-19T08:43:01.729 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:01.729 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.729 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.729 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "2419200", 2024-09-19T08:43:01.729 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.729 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.729 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.729 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T08:43:01.729 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.729 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.729 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.729 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.729 INFO:teuthology.orchestra.run.smithi098.stdout: "pool_name": { 2024-09-19T08:43:01.730 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "pool_name", 2024-09-19T08:43:01.730 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.730 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.730 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.730 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "device_health_metrics", 2024-09-19T08:43:01.730 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.730 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.730 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.730 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T08:43:01.730 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.730 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.730 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.730 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.730 INFO:teuthology.orchestra.run.smithi098.stdout: "retention_period": { 2024-09-19T08:43:01.731 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "retention_period", 2024-09-19T08:43:01.731 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:01.731 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.731 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.731 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "15552000", 2024-09-19T08:43:01.731 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.731 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.731 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.731 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "how long to retain device health metrics", 2024-09-19T08:43:01.731 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.731 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.731 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.731 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.731 INFO:teuthology.orchestra.run.smithi098.stdout: "scrape_frequency": { 2024-09-19T08:43:01.732 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "scrape_frequency", 2024-09-19T08:43:01.732 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:01.732 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.732 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.732 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "86400", 2024-09-19T08:43:01.732 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.732 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.732 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.732 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T08:43:01.732 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.732 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.732 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.732 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.733 INFO:teuthology.orchestra.run.smithi098.stdout: "self_heal": { 2024-09-19T08:43:01.733 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "self_heal", 2024-09-19T08:43:01.733 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.733 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.733 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.733 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.733 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.733 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.733 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.733 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T08:43:01.733 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.733 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.733 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.734 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.734 INFO:teuthology.orchestra.run.smithi098.stdout: "sleep_interval": { 2024-09-19T08:43:01.734 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "sleep_interval", 2024-09-19T08:43:01.734 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:01.734 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.734 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.734 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "600", 2024-09-19T08:43:01.734 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.734 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.734 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.734 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T08:43:01.734 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.735 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.735 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.735 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.735 INFO:teuthology.orchestra.run.smithi098.stdout: "warn_threshold": { 2024-09-19T08:43:01.735 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "warn_threshold", 2024-09-19T08:43:01.735 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:01.735 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.735 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.735 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "7257600", 2024-09-19T08:43:01.735 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.735 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.735 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.735 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T08:43:01.736 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.736 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.736 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.736 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.736 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.736 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.736 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:01.736 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "diskprediction_local", 2024-09-19T08:43:01.736 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:01.736 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:01.736 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:01.736 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:01.736 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:01.736 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.737 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.737 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.737 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.737 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.737 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.737 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.737 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.737 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.737 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.737 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.737 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.737 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.737 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.737 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.738 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.738 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.738 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.738 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.738 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:01.738 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:01.738 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.738 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.738 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.738 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.738 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.738 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.738 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.738 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.739 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.739 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.739 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.739 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.739 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:01.739 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:01.739 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.739 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.739 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.739 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:01.739 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.739 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.739 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.739 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.740 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.740 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.740 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.740 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.740 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.740 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.740 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.740 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.740 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.740 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.740 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.740 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:01.740 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:01.740 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: "predict_interval": { 2024-09-19T08:43:01.741 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "predict_interval", 2024-09-19T08:43:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "86400", 2024-09-19T08:43:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: "predictor_model": { 2024-09-19T08:43:01.742 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "predictor_model", 2024-09-19T08:43:01.743 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.743 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.743 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.743 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "prophetstor", 2024-09-19T08:43:01.743 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.743 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.743 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.743 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.743 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.743 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.743 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.743 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.743 INFO:teuthology.orchestra.run.smithi098.stdout: "sleep_interval": { 2024-09-19T08:43:01.744 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "sleep_interval", 2024-09-19T08:43:01.744 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.744 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.744 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.744 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "600", 2024-09-19T08:43:01.744 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.744 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.744 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.744 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.744 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.744 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.744 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.744 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.744 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.745 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.745 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:01.745 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "influx", 2024-09-19T08:43:01.745 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": false, 2024-09-19T08:43:01.745 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "influxdb python module not found", 2024-09-19T08:43:01.745 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:01.745 INFO:teuthology.orchestra.run.smithi098.stdout: "batch_size": { 2024-09-19T08:43:01.745 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "batch_size", 2024-09-19T08:43:01.745 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.745 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.745 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.745 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "5000", 2024-09-19T08:43:01.745 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.746 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.746 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.746 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T08:43:01.746 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.746 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.746 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.746 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.746 INFO:teuthology.orchestra.run.smithi098.stdout: "database": { 2024-09-19T08:43:01.746 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "database", 2024-09-19T08:43:01.746 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.746 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.746 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.746 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "ceph", 2024-09-19T08:43:01.747 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.747 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.747 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.747 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T08:43:01.747 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.747 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.747 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.747 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.747 INFO:teuthology.orchestra.run.smithi098.stdout: "hostname": { 2024-09-19T08:43:01.747 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "hostname", 2024-09-19T08:43:01.747 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.747 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.747 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.747 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.748 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.748 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.748 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.748 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "InfluxDB server hostname", 2024-09-19T08:43:01.748 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.748 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.748 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.748 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.748 INFO:teuthology.orchestra.run.smithi098.stdout: "interval": { 2024-09-19T08:43:01.748 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "interval", 2024-09-19T08:43:01.748 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:01.748 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.748 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.748 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "30", 2024-09-19T08:43:01.749 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "5", 2024-09-19T08:43:01.749 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.749 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.749 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T08:43:01.749 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.749 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.749 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.749 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.749 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:01.749 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:01.749 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.749 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.749 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.750 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.750 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.750 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.750 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.750 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.750 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.750 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.750 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.750 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.750 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.750 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.750 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.750 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.750 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.751 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.751 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.751 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:01.751 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:01.751 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.751 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.751 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.751 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.751 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.751 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.751 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.751 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.751 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.751 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.752 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.752 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.752 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:01.752 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:01.752 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.752 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.752 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.752 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:01.752 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.752 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.752 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.752 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.752 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.752 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.753 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.753 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.753 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.753 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.753 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.753 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.753 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.753 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.753 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.753 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:01.753 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:01.753 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.753 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.754 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.754 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.754 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.754 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.754 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.754 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.754 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.754 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.754 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.754 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.754 INFO:teuthology.orchestra.run.smithi098.stdout: "password": { 2024-09-19T08:43:01.754 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "password", 2024-09-19T08:43:01.754 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.754 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.755 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.755 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.755 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.755 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.755 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.755 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "password of InfluxDB server user", 2024-09-19T08:43:01.755 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.755 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.755 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.755 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.755 INFO:teuthology.orchestra.run.smithi098.stdout: "port": { 2024-09-19T08:43:01.755 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "port", 2024-09-19T08:43:01.755 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.755 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.756 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.756 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "8086", 2024-09-19T08:43:01.756 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.756 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.756 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.756 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "InfluxDB server port", 2024-09-19T08:43:01.756 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.756 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.756 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.756 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.756 INFO:teuthology.orchestra.run.smithi098.stdout: "ssl": { 2024-09-19T08:43:01.756 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ssl", 2024-09-19T08:43:01.756 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.756 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.757 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.757 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "false", 2024-09-19T08:43:01.757 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.757 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.757 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.757 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T08:43:01.757 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.757 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.757 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.757 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.757 INFO:teuthology.orchestra.run.smithi098.stdout: "threads": { 2024-09-19T08:43:01.757 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "threads", 2024-09-19T08:43:01.757 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.757 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.758 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.758 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "5", 2024-09-19T08:43:01.758 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "1", 2024-09-19T08:43:01.758 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "32", 2024-09-19T08:43:01.758 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.758 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T08:43:01.758 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.758 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.758 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.758 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.758 INFO:teuthology.orchestra.run.smithi098.stdout: "username": { 2024-09-19T08:43:01.758 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "username", 2024-09-19T08:43:01.758 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.759 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.759 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.759 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.759 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.759 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.759 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.759 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "username of InfluxDB server user", 2024-09-19T08:43:01.759 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.759 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.759 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.759 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.759 INFO:teuthology.orchestra.run.smithi098.stdout: "verify_ssl": { 2024-09-19T08:43:01.759 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "verify_ssl", 2024-09-19T08:43:01.760 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.760 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.760 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.760 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "true", 2024-09-19T08:43:01.760 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.760 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.760 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.760 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T08:43:01.760 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.760 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.760 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.760 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.760 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.760 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.761 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:01.761 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "insights", 2024-09-19T08:43:01.761 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:01.761 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:01.761 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:01.761 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:01.761 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:01.761 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.761 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.761 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.761 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.761 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.761 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.762 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.762 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.762 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.762 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.762 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.762 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.762 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.762 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.762 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.762 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.762 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.762 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.762 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.762 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:01.762 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:01.763 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.763 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.763 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.763 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.763 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.763 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.763 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.763 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.763 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.763 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.763 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.763 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.763 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:01.764 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:01.764 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.764 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.764 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.764 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:01.764 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.764 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.764 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.764 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.764 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.764 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.764 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.764 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.764 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.765 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.765 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.765 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.765 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.765 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.765 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.765 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:01.765 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:01.765 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.765 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.765 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.765 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.765 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.765 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.766 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.766 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.766 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.766 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.766 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.766 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.766 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.766 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.766 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:01.766 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "iostat", 2024-09-19T08:43:01.766 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:01.766 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:01.766 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:01.766 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:01.767 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:01.767 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.767 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.767 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.767 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.767 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.767 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.767 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.767 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.767 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.767 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.767 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.767 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.767 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.768 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.768 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.768 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.768 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.768 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.768 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.768 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:01.768 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:01.768 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.768 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.768 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.768 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.768 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.769 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.769 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.769 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.769 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.769 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.769 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.769 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.769 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:01.769 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:01.769 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.769 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.769 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.769 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:01.769 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.770 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.770 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.770 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.770 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.770 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.770 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.770 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.770 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.770 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.770 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.770 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.770 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.770 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.770 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.771 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:01.771 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:01.771 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.771 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.771 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.771 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.771 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.771 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.771 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.771 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.771 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.771 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.771 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.772 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.772 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.772 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.772 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:01.772 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "localpool", 2024-09-19T08:43:01.772 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:01.772 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:01.772 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:01.772 INFO:teuthology.orchestra.run.smithi098.stdout: "failure_domain": { 2024-09-19T08:43:01.772 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "failure_domain", 2024-09-19T08:43:01.772 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.772 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.773 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.773 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "host", 2024-09-19T08:43:01.773 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.773 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.773 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.773 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "failure domain for any created local pool", 2024-09-19T08:43:01.773 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T08:43:01.773 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.773 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.773 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.773 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:01.773 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:01.773 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.773 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.774 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.774 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.774 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.774 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.774 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.774 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.774 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.774 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.774 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.774 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.774 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.774 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.775 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.775 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.775 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.775 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.775 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.775 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:01.775 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:01.775 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.775 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.775 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.775 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.775 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.775 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.775 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.776 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.776 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.776 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.776 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.776 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.776 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:01.776 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:01.776 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.776 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.776 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.776 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:01.776 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.776 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.776 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.777 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.777 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.777 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.777 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.777 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.777 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.777 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.777 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.777 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.777 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.777 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.777 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.777 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:01.778 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:01.778 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.778 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.778 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.778 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.778 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.778 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.778 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.778 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.778 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.778 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.778 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.778 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.778 INFO:teuthology.orchestra.run.smithi098.stdout: "min_size": { 2024-09-19T08:43:01.779 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "min_size", 2024-09-19T08:43:01.779 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.779 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.779 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.779 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.779 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.779 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.779 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.779 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "default min_size for any created local pool", 2024-09-19T08:43:01.779 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T08:43:01.779 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.779 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.779 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.779 INFO:teuthology.orchestra.run.smithi098.stdout: "num_rep": { 2024-09-19T08:43:01.780 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "num_rep", 2024-09-19T08:43:01.780 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.780 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.780 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.780 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "3", 2024-09-19T08:43:01.780 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.780 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.780 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.780 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "default replica count for any created local pool", 2024-09-19T08:43:01.780 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.780 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.780 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.780 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.780 INFO:teuthology.orchestra.run.smithi098.stdout: "pg_num": { 2024-09-19T08:43:01.781 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "pg_num", 2024-09-19T08:43:01.781 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.781 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.781 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.781 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "128", 2024-09-19T08:43:01.781 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.781 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.781 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.781 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T08:43:01.781 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.781 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.781 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.781 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.781 INFO:teuthology.orchestra.run.smithi098.stdout: "prefix": { 2024-09-19T08:43:01.782 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "prefix", 2024-09-19T08:43:01.782 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.782 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.782 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.782 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.782 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.782 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.782 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.782 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "name prefix for any created local pool", 2024-09-19T08:43:01.782 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.782 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.782 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.782 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.783 INFO:teuthology.orchestra.run.smithi098.stdout: "subtree": { 2024-09-19T08:43:01.783 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "subtree", 2024-09-19T08:43:01.783 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.783 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.783 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.783 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "rack", 2024-09-19T08:43:01.783 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.783 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.783 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.783 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T08:43:01.783 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T08:43:01.783 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.783 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.783 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.783 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.784 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.784 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:01.784 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "mirroring", 2024-09-19T08:43:01.784 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:01.784 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:01.784 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:01.784 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:01.784 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:01.784 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.784 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.784 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.784 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.784 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.785 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.785 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.785 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.785 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.785 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.785 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.785 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.785 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.785 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.785 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.785 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.785 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.785 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.785 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.786 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:01.786 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:01.786 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.786 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.786 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.786 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.786 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.786 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.786 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.786 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.786 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.786 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.786 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.786 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.787 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:01.787 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:01.787 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.787 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.787 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.787 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:01.787 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.787 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.787 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.787 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.787 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.787 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.787 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.788 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.788 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.788 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.788 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.788 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.788 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.788 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.788 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.788 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:01.788 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:01.788 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.789 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.789 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.789 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.789 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.789 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.789 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.789 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.789 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.789 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.789 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.789 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.789 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.789 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.789 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:01.790 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "nfs", 2024-09-19T08:43:01.790 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:01.790 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:01.790 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:01.790 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:01.790 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:01.790 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.790 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.790 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.790 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.790 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.790 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.790 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.791 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.791 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.791 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.791 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.791 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.791 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.791 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.791 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.791 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.791 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.791 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.791 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.791 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:01.791 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:01.792 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.792 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.792 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.792 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.792 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.792 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.792 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.792 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.792 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.792 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.792 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.792 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.792 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:01.792 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:01.793 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.793 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.793 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.793 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:01.793 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.793 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.793 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.793 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.793 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.793 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.793 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.793 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.793 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.793 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.794 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.794 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.794 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.794 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.794 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.794 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:01.794 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:01.794 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.794 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.794 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.794 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.794 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.794 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.794 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.795 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.795 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.795 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.795 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.795 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.795 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.795 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.795 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:01.795 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "orchestrator", 2024-09-19T08:43:01.795 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:01.795 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:01.795 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:01.795 INFO:teuthology.orchestra.run.smithi098.stdout: "fail_fs": { 2024-09-19T08:43:01.796 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "fail_fs", 2024-09-19T08:43:01.796 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.796 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.796 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.796 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.796 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.796 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.796 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.796 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T08:43:01.796 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.796 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.796 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.796 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.796 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:01.797 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:01.797 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.797 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.797 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.797 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.797 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.797 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.797 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.797 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.797 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.797 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.797 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.797 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.797 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.798 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.798 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.798 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.798 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.798 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.798 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.798 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:01.798 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:01.798 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.798 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.798 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.798 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.798 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.798 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.799 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.799 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.799 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.799 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.799 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.799 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.799 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:01.799 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:01.799 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.799 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.799 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.799 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:01.799 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.799 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.800 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.800 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.800 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.800 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.800 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.800 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.800 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.800 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.800 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.800 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.800 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.800 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.800 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.801 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:01.801 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:01.801 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.801 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.801 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.801 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.801 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.801 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.801 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.801 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.801 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.801 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.801 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.801 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.802 INFO:teuthology.orchestra.run.smithi098.stdout: "orchestrator": { 2024-09-19T08:43:01.802 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "orchestrator", 2024-09-19T08:43:01.802 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.802 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.802 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.802 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.802 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.802 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.802 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.802 INFO:teuthology.orchestra.run.smithi098.stdout: "cephadm", 2024-09-19T08:43:01.802 INFO:teuthology.orchestra.run.smithi098.stdout: "rook", 2024-09-19T08:43:01.802 INFO:teuthology.orchestra.run.smithi098.stdout: "test_orchestrator" 2024-09-19T08:43:01.802 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.803 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Orchestrator backend", 2024-09-19T08:43:01.803 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.803 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.803 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.803 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.803 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.803 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.803 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:01.803 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "osd_perf_query", 2024-09-19T08:43:01.803 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:01.803 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:01.803 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:01.803 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:01.804 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:01.804 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.804 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.804 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.804 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.804 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.804 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.804 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.804 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.804 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.804 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.804 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.804 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.804 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.805 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.805 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.805 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.805 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.805 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.805 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.805 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:01.805 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:01.805 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.805 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.805 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.805 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.805 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.806 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.806 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.806 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.806 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.806 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.806 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.806 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.806 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:01.806 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:01.806 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.806 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.806 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.806 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:01.806 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.807 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.807 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.807 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.807 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.807 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.807 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.807 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.807 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.807 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.807 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.807 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.807 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.807 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.807 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.808 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:01.808 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:01.808 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.808 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.808 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.808 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.808 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.808 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.808 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.808 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.808 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.808 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.808 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.809 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.809 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.809 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.809 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:01.809 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "osd_support", 2024-09-19T08:43:01.809 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:01.809 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:01.809 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:01.809 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:01.809 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:01.809 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.809 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.809 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.809 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.810 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.810 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.810 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.810 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.810 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.810 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.810 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.810 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.810 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.810 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.810 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.810 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.810 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.810 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.811 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.811 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:01.811 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:01.811 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.811 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.811 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.811 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.811 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.811 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.811 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.811 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.811 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.811 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.811 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.812 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.812 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:01.812 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:01.812 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.812 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.812 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.812 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:01.812 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.812 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.812 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.812 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.812 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.812 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.813 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.813 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.813 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.813 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.813 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.813 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.813 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.813 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.813 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.813 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:01.813 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:01.813 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.813 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.813 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.814 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.814 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.814 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.814 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.814 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.814 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.814 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.814 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.814 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.814 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.814 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.814 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:01.814 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "pg_autoscaler", 2024-09-19T08:43:01.814 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:01.815 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:01.815 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:01.815 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:01.815 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:01.815 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.815 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.815 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.815 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.815 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.815 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.815 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.815 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.815 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.816 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.816 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.816 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.816 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.816 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.816 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.816 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.816 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.816 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.816 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.816 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:01.816 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:01.816 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.816 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.817 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.817 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.817 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.817 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.817 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.817 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.817 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.821 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.821 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.821 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.821 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:01.821 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:01.821 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.821 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.821 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.821 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:01.821 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.822 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.822 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.822 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.822 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.822 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.822 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.822 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.822 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.822 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.822 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.822 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.822 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.822 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.823 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.823 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:01.823 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:01.823 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.823 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.823 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.823 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.823 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.823 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.823 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.823 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.823 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.823 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.823 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.824 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.824 INFO:teuthology.orchestra.run.smithi098.stdout: "sleep_interval": { 2024-09-19T08:43:01.824 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "sleep_interval", 2024-09-19T08:43:01.824 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:01.824 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.824 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.824 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "60", 2024-09-19T08:43:01.824 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.824 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.824 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.824 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.824 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.824 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.825 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.825 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.825 INFO:teuthology.orchestra.run.smithi098.stdout: "threshold": { 2024-09-19T08:43:01.825 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "threshold", 2024-09-19T08:43:01.825 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "float", 2024-09-19T08:43:01.825 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.825 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.825 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "3.0", 2024-09-19T08:43:01.825 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "1.0", 2024-09-19T08:43:01.825 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.825 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.825 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "scaling threshold", 2024-09-19T08:43:01.825 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T08:43:01.825 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.826 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.826 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.826 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.826 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.826 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:01.826 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "progress", 2024-09-19T08:43:01.826 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:01.826 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:01.826 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:01.826 INFO:teuthology.orchestra.run.smithi098.stdout: "allow_pg_recovery_event": { 2024-09-19T08:43:01.826 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "allow_pg_recovery_event", 2024-09-19T08:43:01.826 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.826 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.826 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.827 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.827 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.827 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.827 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.827 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T08:43:01.827 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.827 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.827 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.827 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.827 INFO:teuthology.orchestra.run.smithi098.stdout: "enabled": { 2024-09-19T08:43:01.827 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "enabled", 2024-09-19T08:43:01.827 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.827 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.827 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.828 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.828 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.828 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.828 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.828 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.828 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.828 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.828 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.828 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.828 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:01.828 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:01.828 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.828 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.829 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.829 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.829 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.829 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.829 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.829 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.829 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.829 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.829 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.829 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.829 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.829 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.829 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.830 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.830 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.830 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.830 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.830 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:01.830 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:01.830 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.830 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.830 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.830 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.830 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.830 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.830 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.830 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.831 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.831 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.831 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.831 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.831 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:01.831 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:01.831 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.831 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.831 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.831 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:01.831 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.831 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.831 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.832 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.832 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.832 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.832 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.832 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.832 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.832 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.832 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.832 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.832 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.832 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.832 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.832 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:01.832 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:01.833 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.833 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.833 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.833 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.833 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.833 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.833 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.833 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.833 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.833 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.833 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.833 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.833 INFO:teuthology.orchestra.run.smithi098.stdout: "max_completed_events": { 2024-09-19T08:43:01.833 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "max_completed_events", 2024-09-19T08:43:01.834 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.834 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.834 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.834 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "50", 2024-09-19T08:43:01.834 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.834 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.834 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.834 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "number of past completed events to remember", 2024-09-19T08:43:01.834 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.834 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.834 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.834 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.834 INFO:teuthology.orchestra.run.smithi098.stdout: "sleep_interval": { 2024-09-19T08:43:01.835 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "sleep_interval", 2024-09-19T08:43:01.835 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:01.835 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.835 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.835 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "5", 2024-09-19T08:43:01.835 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.835 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.835 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.835 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "how long the module is going to sleep", 2024-09-19T08:43:01.835 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.835 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.835 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.835 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.836 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.836 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.836 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:01.836 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "prometheus", 2024-09-19T08:43:01.836 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:01.836 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:01.836 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:01.836 INFO:teuthology.orchestra.run.smithi098.stdout: "cache": { 2024-09-19T08:43:01.836 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "cache", 2024-09-19T08:43:01.836 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.836 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.836 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.836 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.836 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.837 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.837 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.837 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.837 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.837 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.837 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.837 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.837 INFO:teuthology.orchestra.run.smithi098.stdout: "exclude_perf_counters": { 2024-09-19T08:43:01.837 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "exclude_perf_counters", 2024-09-19T08:43:01.837 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.837 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.837 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.837 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.838 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.838 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.838 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.838 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T08:43:01.838 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T08:43:01.838 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.838 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.838 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.838 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:01.838 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:01.838 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.838 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.838 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.838 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.839 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.839 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.839 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.839 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.839 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.839 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.839 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.839 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.839 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.839 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.839 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.839 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.839 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.839 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.840 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.840 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:01.840 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:01.840 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.840 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.840 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.840 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.840 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.840 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.840 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.840 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.840 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.840 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.841 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.841 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.841 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:01.841 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:01.841 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.841 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.841 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.841 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:01.841 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.841 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.841 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.841 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.841 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.841 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.842 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.842 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.842 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.842 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.842 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.842 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.842 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.842 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.842 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.842 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:01.842 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:01.842 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.842 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.842 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.843 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.843 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.843 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.843 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.843 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.843 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.843 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.843 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.843 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.843 INFO:teuthology.orchestra.run.smithi098.stdout: "rbd_stats_pools": { 2024-09-19T08:43:01.843 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rbd_stats_pools", 2024-09-19T08:43:01.843 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.843 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.843 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.844 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.844 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.844 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.844 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.844 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.844 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.844 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.844 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.844 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.844 INFO:teuthology.orchestra.run.smithi098.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T08:43:01.844 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T08:43:01.844 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.844 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.845 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.845 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "300", 2024-09-19T08:43:01.845 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.845 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.845 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.845 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.845 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.845 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.845 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.845 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.845 INFO:teuthology.orchestra.run.smithi098.stdout: "scrape_interval": { 2024-09-19T08:43:01.845 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "scrape_interval", 2024-09-19T08:43:01.845 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "float", 2024-09-19T08:43:01.845 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.846 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.846 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "15.0", 2024-09-19T08:43:01.846 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.846 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.846 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.846 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.846 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.846 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.846 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.846 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.846 INFO:teuthology.orchestra.run.smithi098.stdout: "server_addr": { 2024-09-19T08:43:01.846 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "server_addr", 2024-09-19T08:43:01.846 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.846 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.847 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.847 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "::", 2024-09-19T08:43:01.847 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.847 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.847 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.847 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T08:43:01.847 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.847 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.847 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.847 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.847 INFO:teuthology.orchestra.run.smithi098.stdout: "server_port": { 2024-09-19T08:43:01.847 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "server_port", 2024-09-19T08:43:01.847 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.847 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.848 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.848 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "9283", 2024-09-19T08:43:01.848 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.848 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.848 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.848 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T08:43:01.848 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.848 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.848 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.848 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.848 INFO:teuthology.orchestra.run.smithi098.stdout: "stale_cache_strategy": { 2024-09-19T08:43:01.848 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "stale_cache_strategy", 2024-09-19T08:43:01.848 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.849 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.849 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.849 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "log", 2024-09-19T08:43:01.849 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.849 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.849 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.849 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.849 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.849 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.849 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.849 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.849 INFO:teuthology.orchestra.run.smithi098.stdout: "standby_behaviour": { 2024-09-19T08:43:01.849 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "standby_behaviour", 2024-09-19T08:43:01.849 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.850 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.850 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.850 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "default", 2024-09-19T08:43:01.850 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.850 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.850 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.850 INFO:teuthology.orchestra.run.smithi098.stdout: "default", 2024-09-19T08:43:01.850 INFO:teuthology.orchestra.run.smithi098.stdout: "error" 2024-09-19T08:43:01.850 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.850 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.850 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.850 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.850 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.850 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.851 INFO:teuthology.orchestra.run.smithi098.stdout: "standby_error_status_code": { 2024-09-19T08:43:01.851 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "standby_error_status_code", 2024-09-19T08:43:01.851 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.851 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.851 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.851 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "500", 2024-09-19T08:43:01.851 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "400", 2024-09-19T08:43:01.851 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "599", 2024-09-19T08:43:01.851 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.851 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.851 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.851 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.851 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.851 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.852 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.852 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.852 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:01.852 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rbd_support", 2024-09-19T08:43:01.852 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:01.852 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:01.852 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:01.852 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:01.852 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:01.852 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.852 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.852 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.852 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.853 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.853 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.853 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.853 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.853 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.853 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.853 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.853 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.853 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.853 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.853 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.853 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.853 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.854 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.854 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.854 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:01.854 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:01.854 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.854 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.854 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.854 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.854 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.854 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.854 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.854 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.854 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.854 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.855 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.855 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.855 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:01.855 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:01.855 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.855 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.855 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.855 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:01.855 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.855 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.855 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.855 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.855 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.855 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.856 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.856 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.856 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.856 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.856 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.856 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.856 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.856 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.856 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.856 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:01.856 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:01.856 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.856 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.857 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.857 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.857 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.857 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.857 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.857 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.857 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.857 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.857 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.857 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.857 INFO:teuthology.orchestra.run.smithi098.stdout: "max_concurrent_snap_create": { 2024-09-19T08:43:01.857 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "max_concurrent_snap_create", 2024-09-19T08:43:01.857 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.857 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.858 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.858 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "10", 2024-09-19T08:43:01.858 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.858 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.858 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.858 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.858 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.858 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.858 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.858 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.858 INFO:teuthology.orchestra.run.smithi098.stdout: "mirror_snapshot_schedule": { 2024-09-19T08:43:01.858 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T08:43:01.858 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.858 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.859 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.859 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.859 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.859 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.859 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.859 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.859 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.859 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.859 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.859 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.859 INFO:teuthology.orchestra.run.smithi098.stdout: "trash_purge_schedule": { 2024-09-19T08:43:01.859 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "trash_purge_schedule", 2024-09-19T08:43:01.859 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.860 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.860 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.860 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.860 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.860 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.860 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.860 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.860 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.860 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.860 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.860 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.860 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.860 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.860 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:01.861 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "restful", 2024-09-19T08:43:01.861 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:01.861 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:01.861 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:01.861 INFO:teuthology.orchestra.run.smithi098.stdout: "enable_auth": { 2024-09-19T08:43:01.861 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "enable_auth", 2024-09-19T08:43:01.861 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.861 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.861 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.861 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.861 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.861 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.861 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.861 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.862 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.862 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.862 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.862 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.862 INFO:teuthology.orchestra.run.smithi098.stdout: "key_file": { 2024-09-19T08:43:01.862 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "key_file", 2024-09-19T08:43:01.862 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.862 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.862 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.862 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.863 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.863 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.863 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.863 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.863 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.863 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.863 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.863 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.863 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:01.863 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:01.863 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.863 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.863 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.864 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.864 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.864 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.864 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.864 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.864 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.864 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.864 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.864 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.864 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.864 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.864 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.864 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.864 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.865 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.865 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.865 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:01.865 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:01.865 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.865 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.865 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.865 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.865 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.865 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.865 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.865 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.865 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.865 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.866 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.866 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.866 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:01.866 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:01.866 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.866 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.866 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.866 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:01.866 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.866 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.866 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.866 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.866 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.867 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.867 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.867 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.867 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.867 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.867 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.867 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.867 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.867 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.867 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.867 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:01.867 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:01.867 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.867 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.868 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.868 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.868 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.868 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.868 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.868 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.868 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.868 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.868 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.868 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.868 INFO:teuthology.orchestra.run.smithi098.stdout: "server_addr": { 2024-09-19T08:43:01.868 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "server_addr", 2024-09-19T08:43:01.868 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.869 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.869 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.869 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.869 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.869 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.869 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.869 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.869 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.869 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.869 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.869 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.869 INFO:teuthology.orchestra.run.smithi098.stdout: "server_port": { 2024-09-19T08:43:01.869 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "server_port", 2024-09-19T08:43:01.869 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.869 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.870 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.870 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.870 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.870 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.870 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.870 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.870 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.870 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.870 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.870 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.870 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.870 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.870 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:01.870 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "selftest", 2024-09-19T08:43:01.871 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:01.871 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:01.871 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:01.871 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:01.871 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:01.871 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.871 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.871 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.871 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.871 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.871 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.871 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.871 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.872 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.872 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.872 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.872 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.872 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.872 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.872 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.872 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.872 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.872 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.872 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.872 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:01.872 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:01.873 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.873 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.873 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.873 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.873 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.873 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.873 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.873 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.873 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.873 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.873 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.873 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.873 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:01.874 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:01.874 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.874 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.874 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.874 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:01.874 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.874 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.874 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.874 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.874 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.874 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.874 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.874 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.874 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.875 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.875 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.875 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.875 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.875 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.875 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.875 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:01.875 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:01.875 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.875 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.875 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.875 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.875 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.875 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.876 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.876 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.876 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.876 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.876 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.876 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.876 INFO:teuthology.orchestra.run.smithi098.stdout: "roption1": { 2024-09-19T08:43:01.876 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "roption1", 2024-09-19T08:43:01.876 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.876 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.876 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.876 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.877 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.877 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.877 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.877 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.877 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.877 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.877 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.877 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.877 INFO:teuthology.orchestra.run.smithi098.stdout: "roption2": { 2024-09-19T08:43:01.877 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "roption2", 2024-09-19T08:43:01.877 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.877 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.877 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.878 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "xyz", 2024-09-19T08:43:01.878 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.878 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.878 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.878 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.878 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.878 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.878 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.878 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.878 INFO:teuthology.orchestra.run.smithi098.stdout: "rwoption1": { 2024-09-19T08:43:01.878 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rwoption1", 2024-09-19T08:43:01.878 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.878 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.878 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.879 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.879 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.879 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.879 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.879 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.879 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.879 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.879 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.879 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.879 INFO:teuthology.orchestra.run.smithi098.stdout: "rwoption2": { 2024-09-19T08:43:01.879 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rwoption2", 2024-09-19T08:43:01.879 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.879 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.879 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.880 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.880 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.880 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.880 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.880 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.880 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.880 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.880 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.880 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.880 INFO:teuthology.orchestra.run.smithi098.stdout: "rwoption3": { 2024-09-19T08:43:01.880 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rwoption3", 2024-09-19T08:43:01.880 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "float", 2024-09-19T08:43:01.880 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.881 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.881 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.881 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.881 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.881 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.881 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.881 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.881 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.881 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.881 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.881 INFO:teuthology.orchestra.run.smithi098.stdout: "rwoption4": { 2024-09-19T08:43:01.881 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rwoption4", 2024-09-19T08:43:01.881 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.881 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.882 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.882 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.882 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.882 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.882 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.882 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.882 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.882 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.882 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.882 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.882 INFO:teuthology.orchestra.run.smithi098.stdout: "rwoption5": { 2024-09-19T08:43:01.882 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rwoption5", 2024-09-19T08:43:01.882 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.883 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.883 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.883 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.883 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.883 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.883 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.883 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.883 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.883 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.883 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.883 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.883 INFO:teuthology.orchestra.run.smithi098.stdout: "rwoption6": { 2024-09-19T08:43:01.883 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rwoption6", 2024-09-19T08:43:01.883 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.884 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.884 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.884 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.884 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.884 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.884 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.884 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.884 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.884 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.884 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.884 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.884 INFO:teuthology.orchestra.run.smithi098.stdout: "rwoption7": { 2024-09-19T08:43:01.884 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rwoption7", 2024-09-19T08:43:01.884 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.885 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.885 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.885 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.885 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "1", 2024-09-19T08:43:01.885 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "42", 2024-09-19T08:43:01.885 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.885 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.885 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.885 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.885 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.885 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.885 INFO:teuthology.orchestra.run.smithi098.stdout: "testkey": { 2024-09-19T08:43:01.885 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "testkey", 2024-09-19T08:43:01.885 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.886 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.886 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.886 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.886 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.886 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.886 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.886 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.886 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.886 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.886 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.886 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.886 INFO:teuthology.orchestra.run.smithi098.stdout: "testlkey": { 2024-09-19T08:43:01.886 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "testlkey", 2024-09-19T08:43:01.886 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.887 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.887 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.887 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.887 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.887 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.887 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.887 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.887 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.887 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.887 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.887 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.887 INFO:teuthology.orchestra.run.smithi098.stdout: "testnewline": { 2024-09-19T08:43:01.887 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "testnewline", 2024-09-19T08:43:01.887 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.888 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.888 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.888 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.888 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.888 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.888 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.888 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.888 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.888 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.888 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.888 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.888 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.888 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.888 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:01.889 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "snap_schedule", 2024-09-19T08:43:01.889 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:01.889 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:01.889 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:01.889 INFO:teuthology.orchestra.run.smithi098.stdout: "allow_m_granularity": { 2024-09-19T08:43:01.889 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "allow_m_granularity", 2024-09-19T08:43:01.889 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.889 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.889 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.889 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.889 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.889 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.889 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.890 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T08:43:01.890 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.890 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.890 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.890 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.890 INFO:teuthology.orchestra.run.smithi098.stdout: "dump_on_update": { 2024-09-19T08:43:01.890 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "dump_on_update", 2024-09-19T08:43:01.890 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.890 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.890 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.890 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.890 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.890 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.890 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.891 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "dump database to debug log on update", 2024-09-19T08:43:01.891 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.891 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.891 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.891 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.891 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:01.891 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:01.891 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.891 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.891 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.891 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.891 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.891 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.892 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.892 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.892 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.892 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.892 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.892 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.892 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.892 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.892 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.892 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.892 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.892 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.892 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.893 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:01.893 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:01.893 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.893 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.893 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.893 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.893 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.893 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.893 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.893 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.893 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.893 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.893 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.893 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.894 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:01.894 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:01.894 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.894 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.894 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.894 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:01.894 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.894 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.894 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.894 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.894 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.894 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.894 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.894 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.895 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.895 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.895 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.895 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.895 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.895 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.895 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.895 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:01.895 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:01.895 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.895 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.895 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.895 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.895 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.896 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.896 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.896 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.896 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.896 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.896 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.896 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.896 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.896 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.896 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:01.896 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "stats", 2024-09-19T08:43:01.896 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:01.896 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:01.897 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:01.897 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:01.897 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:01.897 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.897 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.897 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.897 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.897 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.897 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.897 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.897 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.897 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.897 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.898 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.898 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.898 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.898 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.898 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.898 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.898 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.898 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.898 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.898 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:01.898 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:01.898 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.898 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.898 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.899 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.899 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.899 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.899 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.899 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.899 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.899 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.899 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.899 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.899 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:01.899 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:01.899 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.899 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.899 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.899 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:01.900 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.900 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.900 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.900 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.900 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.900 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.900 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.900 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.900 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.900 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.900 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.900 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.900 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.901 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.901 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.901 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:01.901 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:01.901 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.901 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.901 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.901 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.901 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.901 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.901 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.901 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.901 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.901 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.902 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.902 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.902 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.902 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.902 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:01.902 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "status", 2024-09-19T08:43:01.902 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:01.902 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:01.902 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:01.902 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:01.902 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:01.902 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.902 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.902 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.903 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.903 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.903 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.903 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.903 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.903 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.903 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.903 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.903 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.903 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.903 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.903 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.903 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.903 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.904 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.904 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.904 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:01.904 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:01.904 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.904 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.904 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.904 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.904 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.904 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.904 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.904 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.904 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.905 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.905 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.905 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.905 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:01.905 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:01.905 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.905 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.905 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.905 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:01.905 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.905 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.905 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.905 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.905 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.906 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.906 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.906 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.906 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.906 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.906 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.906 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.906 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.906 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.906 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.906 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:01.906 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:01.906 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.906 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.907 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.907 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.907 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.907 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.907 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.907 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.907 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.907 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.907 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.907 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.907 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.907 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.907 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:01.907 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "telegraf", 2024-09-19T08:43:01.908 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:01.908 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:01.908 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:01.908 INFO:teuthology.orchestra.run.smithi098.stdout: "address": { 2024-09-19T08:43:01.908 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "address", 2024-09-19T08:43:01.908 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.908 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.908 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.908 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T08:43:01.908 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.908 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.908 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.908 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.909 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.909 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.909 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.909 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.909 INFO:teuthology.orchestra.run.smithi098.stdout: "interval": { 2024-09-19T08:43:01.909 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "interval", 2024-09-19T08:43:01.909 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:01.909 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.909 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.909 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "15", 2024-09-19T08:43:01.909 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.909 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.909 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.909 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.910 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.910 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.910 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.910 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.910 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:01.910 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:01.910 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.910 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.910 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.910 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.910 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.910 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.910 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.910 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.911 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.911 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.911 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.911 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.911 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.911 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.911 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.911 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.911 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.911 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.911 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.911 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:01.911 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:01.912 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.912 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.912 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.912 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.912 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.912 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.912 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.912 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.912 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.912 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.912 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.912 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.912 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:01.912 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:01.913 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.913 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.913 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.913 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:01.913 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.913 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.913 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.913 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.913 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.913 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.913 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.913 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.913 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.913 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.914 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.914 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.914 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.914 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.914 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.914 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:01.914 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:01.914 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.914 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.914 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.914 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.914 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.914 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.915 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.915 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.915 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.915 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.915 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.915 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.915 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.915 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.915 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:01.915 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "telemetry", 2024-09-19T08:43:01.915 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:01.915 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:01.915 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:01.915 INFO:teuthology.orchestra.run.smithi098.stdout: "channel_basic": { 2024-09-19T08:43:01.916 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "channel_basic", 2024-09-19T08:43:01.916 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.916 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.916 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.916 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.916 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.916 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.916 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.916 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T08:43:01.916 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.916 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.916 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.916 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.917 INFO:teuthology.orchestra.run.smithi098.stdout: "channel_crash": { 2024-09-19T08:43:01.917 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "channel_crash", 2024-09-19T08:43:01.917 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.917 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.917 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.917 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.917 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.917 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.917 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.917 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T08:43:01.917 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.917 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.917 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.917 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.918 INFO:teuthology.orchestra.run.smithi098.stdout: "channel_device": { 2024-09-19T08:43:01.918 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "channel_device", 2024-09-19T08:43:01.918 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.918 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.918 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.918 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.918 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.918 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.918 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.918 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T08:43:01.918 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.918 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.918 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.918 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.919 INFO:teuthology.orchestra.run.smithi098.stdout: "channel_ident": { 2024-09-19T08:43:01.919 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "channel_ident", 2024-09-19T08:43:01.919 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.919 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.919 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.919 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.919 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.919 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.919 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.919 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T08:43:01.919 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.919 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.919 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.920 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.920 INFO:teuthology.orchestra.run.smithi098.stdout: "channel_perf": { 2024-09-19T08:43:01.920 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "channel_perf", 2024-09-19T08:43:01.920 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.920 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.920 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.920 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.920 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.920 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.920 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.920 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T08:43:01.920 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.920 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.921 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.921 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.921 INFO:teuthology.orchestra.run.smithi098.stdout: "contact": { 2024-09-19T08:43:01.921 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "contact", 2024-09-19T08:43:01.921 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.921 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.921 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.921 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.921 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.921 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.921 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.921 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.921 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.922 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.922 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.922 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.922 INFO:teuthology.orchestra.run.smithi098.stdout: "description": { 2024-09-19T08:43:01.922 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "description", 2024-09-19T08:43:01.922 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.922 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.922 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.922 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.922 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.922 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.922 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.922 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.922 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.923 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.923 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.923 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.923 INFO:teuthology.orchestra.run.smithi098.stdout: "device_url": { 2024-09-19T08:43:01.923 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "device_url", 2024-09-19T08:43:01.923 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.923 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.923 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.923 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T08:43:01.923 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.923 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.923 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.923 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.924 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.924 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.924 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.924 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.924 INFO:teuthology.orchestra.run.smithi098.stdout: "enabled": { 2024-09-19T08:43:01.924 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "enabled", 2024-09-19T08:43:01.924 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.924 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.924 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.924 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.924 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.924 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.924 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.924 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.925 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.925 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.925 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.925 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.925 INFO:teuthology.orchestra.run.smithi098.stdout: "interval": { 2024-09-19T08:43:01.925 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "interval", 2024-09-19T08:43:01.925 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.925 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.925 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.925 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "24", 2024-09-19T08:43:01.925 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "8", 2024-09-19T08:43:01.925 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.925 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.925 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.926 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.926 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.926 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.926 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.926 INFO:teuthology.orchestra.run.smithi098.stdout: "last_opt_revision": { 2024-09-19T08:43:01.926 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "last_opt_revision", 2024-09-19T08:43:01.926 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.926 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.926 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.926 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "1", 2024-09-19T08:43:01.926 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.926 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.926 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.927 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.927 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.927 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.927 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.927 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.927 INFO:teuthology.orchestra.run.smithi098.stdout: "leaderboard": { 2024-09-19T08:43:01.927 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "leaderboard", 2024-09-19T08:43:01.927 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.927 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.927 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.927 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.927 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.927 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.927 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.928 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.928 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.928 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.928 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.928 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.928 INFO:teuthology.orchestra.run.smithi098.stdout: "leaderboard_description": { 2024-09-19T08:43:01.928 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "leaderboard_description", 2024-09-19T08:43:01.928 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.928 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.928 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.928 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.928 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.928 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.929 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.929 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.929 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.929 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.929 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.929 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.929 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:01.929 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:01.929 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.929 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.929 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.929 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.929 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.930 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.930 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.930 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.930 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.930 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.930 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.930 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.930 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.930 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.930 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.930 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.930 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.930 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.930 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.931 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:01.931 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:01.931 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.931 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.931 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.931 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.931 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.931 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.931 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.931 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.931 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.931 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.931 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.931 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.932 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:01.932 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:01.932 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.932 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.932 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.932 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:01.932 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.932 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.932 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.932 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.932 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.932 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.932 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.933 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.933 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.933 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.933 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.933 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.933 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.933 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.933 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.933 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:01.933 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:01.933 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.933 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.933 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.933 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.934 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.934 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.934 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.934 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.934 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.934 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.934 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.934 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.934 INFO:teuthology.orchestra.run.smithi098.stdout: "organization": { 2024-09-19T08:43:01.934 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "organization", 2024-09-19T08:43:01.934 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.934 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.934 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.935 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.935 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.935 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.935 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.935 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.935 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.935 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.935 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.935 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.935 INFO:teuthology.orchestra.run.smithi098.stdout: "proxy": { 2024-09-19T08:43:01.935 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "proxy", 2024-09-19T08:43:01.935 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.935 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.935 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.936 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.936 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.936 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.936 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.936 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.936 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.936 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.936 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.936 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.936 INFO:teuthology.orchestra.run.smithi098.stdout: "url": { 2024-09-19T08:43:01.936 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "url", 2024-09-19T08:43:01.937 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.937 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.937 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.937 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T08:43:01.937 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.937 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.937 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.937 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.937 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.937 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.938 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.938 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.938 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.938 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.938 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:01.938 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "test_orchestrator", 2024-09-19T08:43:01.938 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:01.938 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:01.938 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:01.938 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:01.938 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:01.939 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.939 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.939 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.939 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.939 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.939 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.939 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.939 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.939 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.939 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.939 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.939 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.940 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.940 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.940 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.940 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.940 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.940 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.940 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.940 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:01.940 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:01.940 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.940 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.940 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.941 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.941 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.941 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.941 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.941 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.941 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.941 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.941 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.941 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.941 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:01.941 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:01.941 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.942 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.942 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.942 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:01.942 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.942 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.942 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.942 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.942 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.942 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.942 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.942 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.942 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.943 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.943 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.943 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.943 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.943 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.943 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.943 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:01.943 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:01.943 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.943 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.943 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.943 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.944 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.944 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.944 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.944 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.944 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.944 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.944 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.944 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.944 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.944 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.944 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:01.945 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "volumes", 2024-09-19T08:43:01.945 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:01.945 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:01.945 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:01.945 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:01.945 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:01.945 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.945 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.945 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.945 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.945 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.946 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.946 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.946 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.946 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.946 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.946 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.946 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.946 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.946 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.946 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.946 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.946 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.947 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.947 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.947 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:01.947 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:01.947 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.947 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.947 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.947 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.947 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.947 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.947 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.948 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.948 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.948 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.948 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.948 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.948 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:01.948 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:01.948 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.948 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.948 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.948 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:01.948 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.949 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.949 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.949 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.949 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.949 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.949 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.949 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.949 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.949 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.949 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.949 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.949 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.950 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.950 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.950 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:01.950 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:01.950 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.950 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.950 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.950 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.950 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.950 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.950 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.951 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.951 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.951 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.951 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.951 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.951 INFO:teuthology.orchestra.run.smithi098.stdout: "max_concurrent_clones": { 2024-09-19T08:43:01.951 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "max_concurrent_clones", 2024-09-19T08:43:01.951 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.951 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.951 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.951 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "4", 2024-09-19T08:43:01.952 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.952 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.952 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.952 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T08:43:01.952 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.952 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.952 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.952 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.952 INFO:teuthology.orchestra.run.smithi098.stdout: "periodic_async_work": { 2024-09-19T08:43:01.952 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "periodic_async_work", 2024-09-19T08:43:01.952 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.952 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.953 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.953 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.953 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.953 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.953 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.953 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Periodically check for async work", 2024-09-19T08:43:01.953 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.953 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.953 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.953 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.953 INFO:teuthology.orchestra.run.smithi098.stdout: "snapshot_clone_delay": { 2024-09-19T08:43:01.953 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "snapshot_clone_delay", 2024-09-19T08:43:01.954 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.954 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.954 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.954 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "0", 2024-09-19T08:43:01.954 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.954 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.954 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.954 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T08:43:01.954 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.954 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.954 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.955 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.955 INFO:teuthology.orchestra.run.smithi098.stdout: "snapshot_clone_no_wait": { 2024-09-19T08:43:01.955 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T08:43:01.955 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.955 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.955 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.955 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.955 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.955 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.955 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.955 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T08:43:01.956 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.956 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.956 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.956 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.956 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.956 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.956 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:01.956 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "zabbix", 2024-09-19T08:43:01.956 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:01.956 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:01.956 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:01.957 INFO:teuthology.orchestra.run.smithi098.stdout: "discovery_interval": { 2024-09-19T08:43:01.957 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "discovery_interval", 2024-09-19T08:43:01.957 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "uint", 2024-09-19T08:43:01.957 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.957 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.957 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "100", 2024-09-19T08:43:01.957 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.957 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.957 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.957 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.957 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.957 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.958 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.958 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.958 INFO:teuthology.orchestra.run.smithi098.stdout: "identifier": { 2024-09-19T08:43:01.958 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "identifier", 2024-09-19T08:43:01.958 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.958 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.958 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.958 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.958 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.958 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.958 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.959 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.959 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.959 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.959 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.959 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.959 INFO:teuthology.orchestra.run.smithi098.stdout: "interval": { 2024-09-19T08:43:01.959 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "interval", 2024-09-19T08:43:01.959 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:01.959 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.959 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.960 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "60", 2024-09-19T08:43:01.960 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.960 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.960 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.960 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.960 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.960 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.960 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.960 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.960 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:01.960 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:01.961 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.961 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.961 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.961 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.961 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.961 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.961 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.961 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.961 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.961 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.961 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.961 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.962 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.962 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.962 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.962 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.962 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.962 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.962 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.962 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:01.962 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:01.962 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.962 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.963 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.963 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.963 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.963 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.963 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.963 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.963 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.963 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.963 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.963 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.963 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:01.963 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:01.964 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.964 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.964 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.964 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:01.964 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.964 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.964 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.964 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.964 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.964 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.964 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.964 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.965 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.965 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.965 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.965 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.965 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.965 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.965 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.965 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:01.965 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:01.965 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.965 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.965 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.966 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.966 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.966 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.966 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.966 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.966 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.966 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.966 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.966 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.966 INFO:teuthology.orchestra.run.smithi098.stdout: "zabbix_host": { 2024-09-19T08:43:01.967 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "zabbix_host", 2024-09-19T08:43:01.967 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.967 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.967 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.967 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.967 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.967 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.967 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.967 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.967 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.967 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.967 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.968 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.968 INFO:teuthology.orchestra.run.smithi098.stdout: "zabbix_port": { 2024-09-19T08:43:01.968 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "zabbix_port", 2024-09-19T08:43:01.968 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.968 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.968 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.968 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "10051", 2024-09-19T08:43:01.968 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.968 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.968 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.969 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.969 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.969 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.969 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.969 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.969 INFO:teuthology.orchestra.run.smithi098.stdout: "zabbix_sender": { 2024-09-19T08:43:01.969 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "zabbix_sender", 2024-09-19T08:43:01.969 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.969 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.969 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:01.969 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T08:43:01.970 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.970 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.970 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.970 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.970 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.970 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.970 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.970 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.970 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.970 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.970 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-19T08:43:01.970 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.970 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.971 INFO:teuthology.orchestra.run.smithi098.stdout: "modules": [ 2024-09-19T08:43:01.971 INFO:teuthology.orchestra.run.smithi098.stdout: "iostat", 2024-09-19T08:43:01.971 INFO:teuthology.orchestra.run.smithi098.stdout: "nfs", 2024-09-19T08:43:01.971 INFO:teuthology.orchestra.run.smithi098.stdout: "restful" 2024-09-19T08:43:01.971 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.971 INFO:teuthology.orchestra.run.smithi098.stdout: "available_modules": [ 2024-09-19T08:43:01.971 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:01.971 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "alerts", 2024-09-19T08:43:01.971 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:01.971 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:01.971 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:01.972 INFO:teuthology.orchestra.run.smithi098.stdout: "interval": { 2024-09-19T08:43:01.972 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "interval", 2024-09-19T08:43:01.972 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:01.972 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.972 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.972 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "60", 2024-09-19T08:43:01.972 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.972 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.972 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.972 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T08:43:01.972 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.973 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.973 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.973 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.973 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:01.973 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:01.973 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.973 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.973 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.973 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.973 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.973 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.974 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.974 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.974 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.974 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.974 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.974 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.974 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.974 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.974 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.974 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.974 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.975 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.975 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.975 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:01.975 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:01.975 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.975 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.975 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.975 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.975 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.975 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.975 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.976 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.976 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.976 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.976 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.976 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.976 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:01.976 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:01.976 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.976 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.976 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.976 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:01.977 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.977 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.977 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:01.977 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:01.977 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:01.977 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:01.977 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:01.977 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:01.977 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:01.977 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:01.977 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.977 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.978 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.978 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.978 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.978 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:01.978 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:01.978 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.978 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.978 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.978 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:01.978 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.978 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.978 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.979 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:01.979 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.979 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.979 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.979 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.979 INFO:teuthology.orchestra.run.smithi098.stdout: "smtp_destination": { 2024-09-19T08:43:01.979 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "smtp_destination", 2024-09-19T08:43:01.979 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.979 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.979 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.979 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.980 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.980 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.980 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.980 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Email address to send alerts to", 2024-09-19T08:43:01.980 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.980 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.980 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.980 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.980 INFO:teuthology.orchestra.run.smithi098.stdout: "smtp_from_name": { 2024-09-19T08:43:01.980 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "smtp_from_name", 2024-09-19T08:43:01.980 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.981 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.981 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.981 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "Ceph", 2024-09-19T08:43:01.981 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.981 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.981 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.981 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Email From: name", 2024-09-19T08:43:01.981 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.981 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.981 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.982 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.982 INFO:teuthology.orchestra.run.smithi098.stdout: "smtp_host": { 2024-09-19T08:43:01.982 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "smtp_host", 2024-09-19T08:43:01.982 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.982 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.982 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.982 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.982 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.982 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.982 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.982 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "SMTP server", 2024-09-19T08:43:01.982 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.983 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.983 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.983 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.983 INFO:teuthology.orchestra.run.smithi098.stdout: "smtp_password": { 2024-09-19T08:43:01.983 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "smtp_password", 2024-09-19T08:43:01.983 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.983 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.983 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.983 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.983 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.983 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.983 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.984 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Password to authenticate with", 2024-09-19T08:43:01.984 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.984 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.984 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.984 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.984 INFO:teuthology.orchestra.run.smithi098.stdout: "smtp_port": { 2024-09-19T08:43:01.984 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "smtp_port", 2024-09-19T08:43:01.984 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:01.984 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.984 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.984 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "465", 2024-09-19T08:43:01.984 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.985 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.985 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.985 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "SMTP port", 2024-09-19T08:43:01.985 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.985 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.985 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.985 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.985 INFO:teuthology.orchestra.run.smithi098.stdout: "smtp_sender": { 2024-09-19T08:43:01.985 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "smtp_sender", 2024-09-19T08:43:01.985 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.985 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.985 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.986 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.986 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.986 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.986 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.986 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "SMTP envelope sender", 2024-09-19T08:43:01.986 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.986 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.986 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.986 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.986 INFO:teuthology.orchestra.run.smithi098.stdout: "smtp_ssl": { 2024-09-19T08:43:01.986 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "smtp_ssl", 2024-09-19T08:43:01.987 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.987 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.987 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.987 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.987 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.987 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.987 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.987 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T08:43:01.987 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.987 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.987 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.988 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.988 INFO:teuthology.orchestra.run.smithi098.stdout: "smtp_user": { 2024-09-19T08:43:01.988 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "smtp_user", 2024-09-19T08:43:01.988 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.988 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.988 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.988 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:01.988 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.988 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.988 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.988 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "User to authenticate as", 2024-09-19T08:43:01.988 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.988 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.989 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.989 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.989 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:01.989 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.989 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:01.989 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "balancer", 2024-09-19T08:43:01.989 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:01.989 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:01.989 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:01.989 INFO:teuthology.orchestra.run.smithi098.stdout: "active": { 2024-09-19T08:43:01.990 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "active", 2024-09-19T08:43:01.990 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:01.990 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.990 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.990 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:01.990 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.990 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.990 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.990 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T08:43:01.990 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.990 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.990 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.991 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.991 INFO:teuthology.orchestra.run.smithi098.stdout: "begin_time": { 2024-09-19T08:43:01.991 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "begin_time", 2024-09-19T08:43:01.991 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.991 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.991 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.991 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "0000", 2024-09-19T08:43:01.991 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.991 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.991 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.992 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T08:43:01.992 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T08:43:01.992 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.992 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.992 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.992 INFO:teuthology.orchestra.run.smithi098.stdout: "begin_weekday": { 2024-09-19T08:43:01.992 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "begin_weekday", 2024-09-19T08:43:01.992 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "uint", 2024-09-19T08:43:01.992 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.992 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.992 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "0", 2024-09-19T08:43:01.993 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "0", 2024-09-19T08:43:01.993 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "6", 2024-09-19T08:43:01.993 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.993 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T08:43:01.993 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T08:43:01.993 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.993 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.993 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.993 INFO:teuthology.orchestra.run.smithi098.stdout: "crush_compat_max_iterations": { 2024-09-19T08:43:01.993 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "crush_compat_max_iterations", 2024-09-19T08:43:01.993 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "uint", 2024-09-19T08:43:01.994 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.994 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.994 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "25", 2024-09-19T08:43:01.994 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "1", 2024-09-19T08:43:01.994 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "250", 2024-09-19T08:43:01.994 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.994 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T08:43:01.994 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:01.994 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.994 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.994 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.994 INFO:teuthology.orchestra.run.smithi098.stdout: "crush_compat_metrics": { 2024-09-19T08:43:01.995 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "crush_compat_metrics", 2024-09-19T08:43:01.995 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.995 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.995 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.995 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T08:43:01.995 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.995 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.995 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.995 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T08:43:01.995 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T08:43:01.996 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.996 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.996 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.996 INFO:teuthology.orchestra.run.smithi098.stdout: "crush_compat_step": { 2024-09-19T08:43:01.996 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "crush_compat_step", 2024-09-19T08:43:01.996 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "float", 2024-09-19T08:43:01.996 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.996 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.996 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "0.5", 2024-09-19T08:43:01.996 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "0.001", 2024-09-19T08:43:01.997 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "0.999", 2024-09-19T08:43:01.997 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.997 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "aggressiveness of optimization", 2024-09-19T08:43:01.997 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T08:43:01.997 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.997 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.997 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.997 INFO:teuthology.orchestra.run.smithi098.stdout: "end_time": { 2024-09-19T08:43:01.997 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "end_time", 2024-09-19T08:43:01.997 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:01.997 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.998 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.998 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "2359", 2024-09-19T08:43:01.998 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:01.998 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:01.998 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.998 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T08:43:01.998 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T08:43:01.998 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.998 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.998 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.998 INFO:teuthology.orchestra.run.smithi098.stdout: "end_weekday": { 2024-09-19T08:43:01.998 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "end_weekday", 2024-09-19T08:43:01.998 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "uint", 2024-09-19T08:43:01.999 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:01.999 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:01.999 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "0", 2024-09-19T08:43:01.999 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "0", 2024-09-19T08:43:01.999 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "6", 2024-09-19T08:43:01.999 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:01.999 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T08:43:01.999 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T08:43:01.999 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:01.999 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:01.999 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:01.999 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:02.000 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:02.000 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.000 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.000 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.000 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.000 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.000 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.000 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.000 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.000 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.000 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.001 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.001 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.001 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.001 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.001 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.001 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.001 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.001 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.001 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.001 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:02.001 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:02.002 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.002 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.002 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.002 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.002 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.002 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.002 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.002 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.002 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.002 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.003 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.003 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.003 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:02.003 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:02.003 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.003 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.003 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.003 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:02.003 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.003 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.003 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.003 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.004 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.004 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.004 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.004 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.004 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.004 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.004 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.004 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.004 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.004 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.004 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.005 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:02.005 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:02.005 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.005 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.005 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.005 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.005 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.005 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.005 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.005 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.005 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.005 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.006 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.006 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.006 INFO:teuthology.orchestra.run.smithi098.stdout: "min_score": { 2024-09-19T08:43:02.006 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "min_score", 2024-09-19T08:43:02.006 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "float", 2024-09-19T08:43:02.006 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.006 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.006 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "0", 2024-09-19T08:43:02.006 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.006 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.007 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.007 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T08:43:02.007 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.007 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.007 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.007 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.007 INFO:teuthology.orchestra.run.smithi098.stdout: "mode": { 2024-09-19T08:43:02.007 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "mode", 2024-09-19T08:43:02.007 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.007 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.007 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.007 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "upmap", 2024-09-19T08:43:02.008 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.008 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.008 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.008 INFO:teuthology.orchestra.run.smithi098.stdout: "crush-compat", 2024-09-19T08:43:02.008 INFO:teuthology.orchestra.run.smithi098.stdout: "none", 2024-09-19T08:43:02.008 INFO:teuthology.orchestra.run.smithi098.stdout: "read", 2024-09-19T08:43:02.008 INFO:teuthology.orchestra.run.smithi098.stdout: "upmap", 2024-09-19T08:43:02.008 INFO:teuthology.orchestra.run.smithi098.stdout: "upmap-read" 2024-09-19T08:43:02.008 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.008 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Balancer mode", 2024-09-19T08:43:02.008 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.009 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.009 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.009 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.009 INFO:teuthology.orchestra.run.smithi098.stdout: "pool_ids": { 2024-09-19T08:43:02.009 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "pool_ids", 2024-09-19T08:43:02.009 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.009 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.009 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.009 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.009 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.009 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.010 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.010 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T08:43:02.010 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.010 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.010 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.010 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.010 INFO:teuthology.orchestra.run.smithi098.stdout: "sleep_interval": { 2024-09-19T08:43:02.010 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "sleep_interval", 2024-09-19T08:43:02.010 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:02.010 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.010 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.010 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "60", 2024-09-19T08:43:02.011 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.011 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.011 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.011 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T08:43:02.011 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.011 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.011 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.011 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.011 INFO:teuthology.orchestra.run.smithi098.stdout: "upmap_max_deviation": { 2024-09-19T08:43:02.011 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "upmap_max_deviation", 2024-09-19T08:43:02.011 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:02.011 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.012 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.012 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "5", 2024-09-19T08:43:02.012 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "1", 2024-09-19T08:43:02.012 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.012 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.012 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T08:43:02.012 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T08:43:02.012 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.012 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.012 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.012 INFO:teuthology.orchestra.run.smithi098.stdout: "upmap_max_optimizations": { 2024-09-19T08:43:02.013 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "upmap_max_optimizations", 2024-09-19T08:43:02.013 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "uint", 2024-09-19T08:43:02.013 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.013 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.013 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "10", 2024-09-19T08:43:02.013 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.013 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.013 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.013 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T08:43:02.013 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.013 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.014 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.014 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.014 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.014 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.014 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.014 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "cephadm", 2024-09-19T08:43:02.014 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:02.014 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:02.014 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:02.014 INFO:teuthology.orchestra.run.smithi098.stdout: "agent_down_multiplier": { 2024-09-19T08:43:02.014 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "agent_down_multiplier", 2024-09-19T08:43:02.014 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "float", 2024-09-19T08:43:02.015 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.015 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.015 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "3.0", 2024-09-19T08:43:02.015 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.015 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.015 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.015 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T08:43:02.015 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.015 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.015 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.015 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.016 INFO:teuthology.orchestra.run.smithi098.stdout: "agent_refresh_rate": { 2024-09-19T08:43:02.016 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "agent_refresh_rate", 2024-09-19T08:43:02.016 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:02.016 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.016 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.016 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "20", 2024-09-19T08:43:02.016 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.016 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.016 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.016 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T08:43:02.017 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.017 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.017 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.017 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.017 INFO:teuthology.orchestra.run.smithi098.stdout: "agent_starting_port": { 2024-09-19T08:43:02.017 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "agent_starting_port", 2024-09-19T08:43:02.017 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:02.017 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.017 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.017 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "4721", 2024-09-19T08:43:02.017 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.017 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.018 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.018 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T08:43:02.018 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.018 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.018 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.018 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.018 INFO:teuthology.orchestra.run.smithi098.stdout: "allow_ptrace": { 2024-09-19T08:43:02.018 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "allow_ptrace", 2024-09-19T08:43:02.018 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.018 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.018 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.019 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.019 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.019 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.019 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.019 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T08:43:02.019 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T08:43:02.019 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.019 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.019 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.019 INFO:teuthology.orchestra.run.smithi098.stdout: "autotune_interval": { 2024-09-19T08:43:02.019 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "autotune_interval", 2024-09-19T08:43:02.020 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:02.020 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.020 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.020 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "600", 2024-09-19T08:43:02.020 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.020 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.020 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.020 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T08:43:02.020 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.020 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.020 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.020 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.021 INFO:teuthology.orchestra.run.smithi098.stdout: "autotune_memory_target_ratio": { 2024-09-19T08:43:02.021 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T08:43:02.021 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "float", 2024-09-19T08:43:02.021 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.021 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.021 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "0.7", 2024-09-19T08:43:02.021 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.021 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.021 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.021 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T08:43:02.021 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.022 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.022 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.022 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.022 INFO:teuthology.orchestra.run.smithi098.stdout: "cephadm_log_destination": { 2024-09-19T08:43:02.022 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "cephadm_log_destination", 2024-09-19T08:43:02.022 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.022 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.022 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.022 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.022 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.022 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.022 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.023 INFO:teuthology.orchestra.run.smithi098.stdout: "file", 2024-09-19T08:43:02.023 INFO:teuthology.orchestra.run.smithi098.stdout: "file,syslog", 2024-09-19T08:43:02.023 INFO:teuthology.orchestra.run.smithi098.stdout: "syslog" 2024-09-19T08:43:02.023 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.023 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T08:43:02.025 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.025 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.025 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.026 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.026 INFO:teuthology.orchestra.run.smithi098.stdout: "cgroups_split": { 2024-09-19T08:43:02.026 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "cgroups_split", 2024-09-19T08:43:02.026 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.026 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.026 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.026 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.026 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.026 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.026 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.026 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T08:43:02.026 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.027 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.027 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.027 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.027 INFO:teuthology.orchestra.run.smithi098.stdout: "config_checks_enabled": { 2024-09-19T08:43:02.027 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "config_checks_enabled", 2024-09-19T08:43:02.027 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.027 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.027 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.027 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.027 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.027 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.028 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.028 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T08:43:02.028 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.028 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.028 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.028 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.028 INFO:teuthology.orchestra.run.smithi098.stdout: "config_dashboard": { 2024-09-19T08:43:02.028 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "config_dashboard", 2024-09-19T08:43:02.028 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.028 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.028 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.029 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.029 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.029 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.029 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.029 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T08:43:02.029 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.029 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.029 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.029 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.029 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_alertmanager": { 2024-09-19T08:43:02.029 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_alertmanager", 2024-09-19T08:43:02.029 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.030 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.030 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.030 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T08:43:02.030 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.030 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.030 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.030 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Prometheus container image", 2024-09-19T08:43:02.030 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.030 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.030 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.030 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.030 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_base": { 2024-09-19T08:43:02.031 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_base", 2024-09-19T08:43:02.031 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.031 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.031 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.031 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T08:43:02.031 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.031 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.031 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.031 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Container image name, without the tag", 2024-09-19T08:43:02.031 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.031 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.032 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.032 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.032 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_elasticsearch": { 2024-09-19T08:43:02.032 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_elasticsearch", 2024-09-19T08:43:02.032 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.032 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.032 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.032 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T08:43:02.032 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.032 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.032 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.033 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "elasticsearch container image", 2024-09-19T08:43:02.033 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.033 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.033 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.033 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.033 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_grafana": { 2024-09-19T08:43:02.033 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_grafana", 2024-09-19T08:43:02.033 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.033 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.033 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.033 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T08:43:02.033 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.034 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.034 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.034 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Prometheus container image", 2024-09-19T08:43:02.034 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.034 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.034 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.034 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.034 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_haproxy": { 2024-09-19T08:43:02.034 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_haproxy", 2024-09-19T08:43:02.034 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.034 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.034 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.035 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T08:43:02.035 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.035 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.035 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.035 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "HAproxy container image", 2024-09-19T08:43:02.035 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.035 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.035 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.035 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.035 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_jaeger_agent": { 2024-09-19T08:43:02.035 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_jaeger_agent", 2024-09-19T08:43:02.036 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.036 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.036 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.036 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T08:43:02.036 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.036 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.036 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.036 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Jaeger agent container image", 2024-09-19T08:43:02.036 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.036 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.036 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.037 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.037 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_jaeger_collector": { 2024-09-19T08:43:02.037 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_jaeger_collector", 2024-09-19T08:43:02.037 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.037 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.037 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.037 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T08:43:02.037 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.037 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.037 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.037 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Jaeger collector container image", 2024-09-19T08:43:02.037 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.038 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.038 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.038 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.038 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_jaeger_query": { 2024-09-19T08:43:02.038 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_jaeger_query", 2024-09-19T08:43:02.038 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.038 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.038 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.038 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T08:43:02.038 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.038 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.038 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.039 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Jaeger query container image", 2024-09-19T08:43:02.039 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.039 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.039 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.039 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.039 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_keepalived": { 2024-09-19T08:43:02.039 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_keepalived", 2024-09-19T08:43:02.039 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.039 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.039 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.039 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T08:43:02.039 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.040 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.040 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.040 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Keepalived container image", 2024-09-19T08:43:02.040 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.040 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.040 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.040 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.040 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_loki": { 2024-09-19T08:43:02.040 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_loki", 2024-09-19T08:43:02.040 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.041 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.041 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.041 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T08:43:02.041 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.041 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.041 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.041 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Loki container image", 2024-09-19T08:43:02.041 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.041 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.041 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.041 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.041 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_node_exporter": { 2024-09-19T08:43:02.042 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_node_exporter", 2024-09-19T08:43:02.042 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.042 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.042 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.042 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T08:43:02.042 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.042 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.042 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.042 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Prometheus container image", 2024-09-19T08:43:02.042 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.042 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.043 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.043 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.043 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_nvmeof": { 2024-09-19T08:43:02.043 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_nvmeof", 2024-09-19T08:43:02.043 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.043 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.043 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.043 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T08:43:02.043 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.043 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.043 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.044 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Nvme-of container image", 2024-09-19T08:43:02.044 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.044 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.044 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.044 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.044 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_prometheus": { 2024-09-19T08:43:02.044 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_prometheus", 2024-09-19T08:43:02.044 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.044 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.044 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.044 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T08:43:02.044 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.045 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.045 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.045 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Prometheus container image", 2024-09-19T08:43:02.045 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.045 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.045 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.045 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.045 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_promtail": { 2024-09-19T08:43:02.045 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_promtail", 2024-09-19T08:43:02.045 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.045 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.046 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.046 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T08:43:02.046 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.046 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.046 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.046 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Promtail container image", 2024-09-19T08:43:02.046 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.046 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.046 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.046 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.046 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_samba": { 2024-09-19T08:43:02.046 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_samba", 2024-09-19T08:43:02.047 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.047 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.047 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.047 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T08:43:02.047 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.047 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.047 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.047 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Samba/SMB container image", 2024-09-19T08:43:02.047 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.047 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.048 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.048 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.048 INFO:teuthology.orchestra.run.smithi098.stdout: "container_image_snmp_gateway": { 2024-09-19T08:43:02.048 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_image_snmp_gateway", 2024-09-19T08:43:02.048 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.048 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.048 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.048 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T08:43:02.048 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.048 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.048 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.048 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "SNMP Gateway container image", 2024-09-19T08:43:02.049 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.049 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.049 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.049 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.049 INFO:teuthology.orchestra.run.smithi098.stdout: "container_init": { 2024-09-19T08:43:02.049 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "container_init", 2024-09-19T08:43:02.049 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.049 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.049 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.049 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.049 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.050 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.050 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.050 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T08:43:02.050 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.050 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.050 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.050 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.050 INFO:teuthology.orchestra.run.smithi098.stdout: "daemon_cache_timeout": { 2024-09-19T08:43:02.050 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "daemon_cache_timeout", 2024-09-19T08:43:02.050 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:02.050 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.050 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.050 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "600", 2024-09-19T08:43:02.051 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.051 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.051 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.051 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T08:43:02.051 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.051 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.051 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.051 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.051 INFO:teuthology.orchestra.run.smithi098.stdout: "default_cephadm_command_timeout": { 2024-09-19T08:43:02.051 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T08:43:02.051 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:02.051 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.051 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.052 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "900", 2024-09-19T08:43:02.052 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.052 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.052 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.052 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T08:43:02.052 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.052 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.052 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.052 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.052 INFO:teuthology.orchestra.run.smithi098.stdout: "default_registry": { 2024-09-19T08:43:02.052 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "default_registry", 2024-09-19T08:43:02.052 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.053 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.053 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.053 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "docker.io", 2024-09-19T08:43:02.053 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.053 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.053 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.053 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T08:43:02.053 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.053 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.053 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.053 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.053 INFO:teuthology.orchestra.run.smithi098.stdout: "device_cache_timeout": { 2024-09-19T08:43:02.054 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "device_cache_timeout", 2024-09-19T08:43:02.054 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:02.054 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.054 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.054 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "1800", 2024-09-19T08:43:02.054 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.054 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.054 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.054 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "seconds to cache device inventory", 2024-09-19T08:43:02.054 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.054 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.055 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.055 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.055 INFO:teuthology.orchestra.run.smithi098.stdout: "device_enhanced_scan": { 2024-09-19T08:43:02.055 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "device_enhanced_scan", 2024-09-19T08:43:02.055 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.055 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.055 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.055 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.055 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.055 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.056 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.056 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T08:43:02.056 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.056 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.056 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.056 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.056 INFO:teuthology.orchestra.run.smithi098.stdout: "facts_cache_timeout": { 2024-09-19T08:43:02.056 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "facts_cache_timeout", 2024-09-19T08:43:02.056 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:02.056 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.056 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.057 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "60", 2024-09-19T08:43:02.057 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.057 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.057 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.057 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "seconds to cache host facts data", 2024-09-19T08:43:02.057 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.057 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.057 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.057 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.057 INFO:teuthology.orchestra.run.smithi098.stdout: "grafana_dashboards_path": { 2024-09-19T08:43:02.057 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "grafana_dashboards_path", 2024-09-19T08:43:02.058 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.058 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.058 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.058 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T08:43:02.058 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.058 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.058 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.058 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T08:43:02.058 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.058 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.058 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.059 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.059 INFO:teuthology.orchestra.run.smithi098.stdout: "host_check_interval": { 2024-09-19T08:43:02.059 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "host_check_interval", 2024-09-19T08:43:02.059 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:02.059 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.059 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.059 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "600", 2024-09-19T08:43:02.059 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.059 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.059 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.059 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "how frequently to perform a host check", 2024-09-19T08:43:02.059 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.060 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.060 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.060 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.060 INFO:teuthology.orchestra.run.smithi098.stdout: "hw_monitoring": { 2024-09-19T08:43:02.060 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "hw_monitoring", 2024-09-19T08:43:02.060 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.060 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.060 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.060 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.060 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.060 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.061 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.061 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T08:43:02.061 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.061 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.061 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.061 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.061 INFO:teuthology.orchestra.run.smithi098.stdout: "inventory_list_all": { 2024-09-19T08:43:02.061 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "inventory_list_all", 2024-09-19T08:43:02.061 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.061 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.061 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.061 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.061 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.062 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.062 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.062 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T08:43:02.062 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.062 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.062 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.062 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.062 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:02.062 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:02.062 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.062 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.063 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.063 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.063 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.063 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.063 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.063 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.063 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.063 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.063 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.063 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.063 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.063 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.063 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.064 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.064 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.064 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.064 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.064 INFO:teuthology.orchestra.run.smithi098.stdout: "log_refresh_metadata": { 2024-09-19T08:43:02.064 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_refresh_metadata", 2024-09-19T08:43:02.064 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.064 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.064 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.064 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.064 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.064 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.064 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.065 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T08:43:02.065 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.065 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.065 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.065 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.065 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:02.065 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:02.065 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.065 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.065 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.066 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.066 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.066 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.066 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.066 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T08:43:02.066 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.066 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.066 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.066 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.066 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:02.067 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:02.067 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.067 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.067 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.067 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:02.067 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.067 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.067 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.067 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.067 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.067 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.068 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.068 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.068 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.068 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.068 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.068 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.068 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.068 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.068 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.068 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:02.068 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:02.069 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.069 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.069 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.069 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.069 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.069 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.069 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.069 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.069 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.069 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.069 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.069 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.070 INFO:teuthology.orchestra.run.smithi098.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T08:43:02.070 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T08:43:02.070 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.070 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.070 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.070 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.070 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.070 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.070 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.070 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T08:43:02.070 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.071 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.071 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.071 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.071 INFO:teuthology.orchestra.run.smithi098.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T08:43:02.071 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T08:43:02.071 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.071 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.071 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.071 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "*", 2024-09-19T08:43:02.071 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.071 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.072 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.072 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T08:43:02.072 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.072 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.072 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.072 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.072 INFO:teuthology.orchestra.run.smithi098.stdout: "max_count_per_host": { 2024-09-19T08:43:02.072 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "max_count_per_host", 2024-09-19T08:43:02.072 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:02.072 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.072 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.073 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "10", 2024-09-19T08:43:02.073 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.073 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.073 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.073 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "max number of daemons per service per host", 2024-09-19T08:43:02.073 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.073 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.073 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.073 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.073 INFO:teuthology.orchestra.run.smithi098.stdout: "max_osd_draining_count": { 2024-09-19T08:43:02.073 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "max_osd_draining_count", 2024-09-19T08:43:02.074 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:02.074 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.074 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.074 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "10", 2024-09-19T08:43:02.074 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.074 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.074 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.074 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T08:43:02.074 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.074 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.074 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.075 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.075 INFO:teuthology.orchestra.run.smithi098.stdout: "migration_current": { 2024-09-19T08:43:02.075 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "migration_current", 2024-09-19T08:43:02.075 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:02.075 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.075 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.075 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.075 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.075 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.075 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.075 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "internal - do not modify", 2024-09-19T08:43:02.076 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.076 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.076 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.076 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.076 INFO:teuthology.orchestra.run.smithi098.stdout: "mode": { 2024-09-19T08:43:02.076 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "mode", 2024-09-19T08:43:02.076 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.076 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.076 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.076 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "root", 2024-09-19T08:43:02.076 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.077 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.077 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.077 INFO:teuthology.orchestra.run.smithi098.stdout: "cephadm-package", 2024-09-19T08:43:02.077 INFO:teuthology.orchestra.run.smithi098.stdout: "root" 2024-09-19T08:43:02.077 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.077 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T08:43:02.077 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.077 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.077 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.077 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.077 INFO:teuthology.orchestra.run.smithi098.stdout: "oob_default_addr": { 2024-09-19T08:43:02.078 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "oob_default_addr", 2024-09-19T08:43:02.078 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.078 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.078 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.078 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "169.254.1.1", 2024-09-19T08:43:02.078 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.078 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.078 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.078 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T08:43:02.078 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.078 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.079 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.079 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.079 INFO:teuthology.orchestra.run.smithi098.stdout: "prometheus_alerts_path": { 2024-09-19T08:43:02.079 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "prometheus_alerts_path", 2024-09-19T08:43:02.079 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.079 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.079 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.079 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T08:43:02.079 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.079 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.079 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.079 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T08:43:02.080 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.080 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.080 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.080 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.080 INFO:teuthology.orchestra.run.smithi098.stdout: "registry_insecure": { 2024-09-19T08:43:02.080 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "registry_insecure", 2024-09-19T08:43:02.080 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.080 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.080 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.080 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.080 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.081 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.081 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.081 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T08:43:02.081 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.081 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.081 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.081 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.081 INFO:teuthology.orchestra.run.smithi098.stdout: "registry_password": { 2024-09-19T08:43:02.081 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "registry_password", 2024-09-19T08:43:02.081 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.081 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.081 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.082 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.082 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.082 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.082 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.082 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T08:43:02.082 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.082 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.082 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.082 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.082 INFO:teuthology.orchestra.run.smithi098.stdout: "registry_url": { 2024-09-19T08:43:02.082 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "registry_url", 2024-09-19T08:43:02.082 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.082 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.083 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.083 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.083 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.083 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.083 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.083 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T08:43:02.083 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.083 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.083 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.083 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.083 INFO:teuthology.orchestra.run.smithi098.stdout: "registry_username": { 2024-09-19T08:43:02.084 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "registry_username", 2024-09-19T08:43:02.084 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.084 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.084 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.084 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.084 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.084 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.084 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.084 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T08:43:02.084 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.084 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.084 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.085 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.085 INFO:teuthology.orchestra.run.smithi098.stdout: "secure_monitoring_stack": { 2024-09-19T08:43:02.085 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "secure_monitoring_stack", 2024-09-19T08:43:02.085 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.085 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.085 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.085 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.085 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.085 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.085 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.085 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T08:43:02.086 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.086 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.086 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.086 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.086 INFO:teuthology.orchestra.run.smithi098.stdout: "service_discovery_port": { 2024-09-19T08:43:02.086 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "service_discovery_port", 2024-09-19T08:43:02.086 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:02.086 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.086 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.086 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "8765", 2024-09-19T08:43:02.086 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.087 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.087 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.087 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "cephadm service discovery port", 2024-09-19T08:43:02.087 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.087 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.087 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.087 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.087 INFO:teuthology.orchestra.run.smithi098.stdout: "ssh_config_file": { 2024-09-19T08:43:02.087 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ssh_config_file", 2024-09-19T08:43:02.087 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.087 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.087 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.088 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.088 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.088 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.088 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.088 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T08:43:02.088 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.088 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.088 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.088 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.088 INFO:teuthology.orchestra.run.smithi098.stdout: "use_agent": { 2024-09-19T08:43:02.088 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "use_agent", 2024-09-19T08:43:02.088 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.089 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.089 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.089 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.089 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.089 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.089 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.089 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T08:43:02.089 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.089 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.089 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.089 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.089 INFO:teuthology.orchestra.run.smithi098.stdout: "use_repo_digest": { 2024-09-19T08:43:02.089 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "use_repo_digest", 2024-09-19T08:43:02.090 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.090 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.090 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.090 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.090 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.090 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.090 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.090 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T08:43:02.090 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.090 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.090 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.091 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.091 INFO:teuthology.orchestra.run.smithi098.stdout: "warn_on_failed_host_check": { 2024-09-19T08:43:02.091 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "warn_on_failed_host_check", 2024-09-19T08:43:02.091 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.091 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.091 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.091 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.091 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.091 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.091 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.091 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T08:43:02.091 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.092 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.092 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.092 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.092 INFO:teuthology.orchestra.run.smithi098.stdout: "warn_on_stray_daemons": { 2024-09-19T08:43:02.092 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "warn_on_stray_daemons", 2024-09-19T08:43:02.092 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.092 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.092 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.092 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.092 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.092 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.093 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.093 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T08:43:02.093 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.093 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.093 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.093 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.093 INFO:teuthology.orchestra.run.smithi098.stdout: "warn_on_stray_hosts": { 2024-09-19T08:43:02.093 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "warn_on_stray_hosts", 2024-09-19T08:43:02.093 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.093 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.093 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.093 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.094 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.094 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.094 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.094 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T08:43:02.094 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.094 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.094 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.094 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.094 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.094 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.095 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.095 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "crash", 2024-09-19T08:43:02.095 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:02.095 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:02.095 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:02.095 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:02.095 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:02.095 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.095 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.095 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.095 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.095 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.096 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.096 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.096 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.096 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.096 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.096 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.096 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.096 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.096 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.096 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.097 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.097 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.097 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.097 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.097 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:02.097 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:02.097 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.097 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.097 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.097 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.097 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.098 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.098 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.098 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.098 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.098 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.098 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.098 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.098 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:02.098 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:02.098 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.098 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.098 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.098 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:02.099 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.099 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.099 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.099 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.099 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.099 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.099 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.099 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.099 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.099 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.099 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.100 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.100 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.100 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.100 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.100 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:02.100 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:02.100 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.100 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.100 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.100 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.100 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.100 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.101 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.101 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.101 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.101 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.101 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.101 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.101 INFO:teuthology.orchestra.run.smithi098.stdout: "retain_interval": { 2024-09-19T08:43:02.101 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "retain_interval", 2024-09-19T08:43:02.101 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:02.101 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.101 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.101 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "31536000", 2024-09-19T08:43:02.102 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.102 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.102 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.102 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T08:43:02.102 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.102 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.102 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.102 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.102 INFO:teuthology.orchestra.run.smithi098.stdout: "warn_recent_interval": { 2024-09-19T08:43:02.102 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "warn_recent_interval", 2024-09-19T08:43:02.103 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:02.103 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.103 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.103 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "1209600", 2024-09-19T08:43:02.103 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.103 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.103 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.103 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T08:43:02.103 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.103 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.103 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.103 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.104 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.104 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.104 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.104 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "dashboard", 2024-09-19T08:43:02.104 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:02.104 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:02.104 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:02.104 INFO:teuthology.orchestra.run.smithi098.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T08:43:02.104 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T08:43:02.104 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:02.105 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.105 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.105 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "10", 2024-09-19T08:43:02.105 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.105 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.105 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.105 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.105 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.105 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.105 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.106 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.106 INFO:teuthology.orchestra.run.smithi098.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T08:43:02.106 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T08:43:02.106 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.106 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.106 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.106 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.106 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.106 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.106 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.106 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.107 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.107 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.107 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.107 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.107 INFO:teuthology.orchestra.run.smithi098.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T08:43:02.107 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T08:43:02.107 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.107 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.107 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.107 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.107 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.108 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.108 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.108 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.108 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.108 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.108 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.108 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.108 INFO:teuthology.orchestra.run.smithi098.stdout: "AUDIT_API_ENABLED": { 2024-09-19T08:43:02.108 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T08:43:02.108 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.108 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.109 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.109 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.109 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.109 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.109 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.109 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.109 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.109 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.109 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.109 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.110 INFO:teuthology.orchestra.run.smithi098.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T08:43:02.110 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T08:43:02.110 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.110 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.110 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.110 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.110 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.110 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.110 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.110 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.110 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.111 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.111 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.111 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.111 INFO:teuthology.orchestra.run.smithi098.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T08:43:02.111 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T08:43:02.111 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.111 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.111 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.111 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.111 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.111 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.111 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.112 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.112 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.112 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.112 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.112 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.112 INFO:teuthology.orchestra.run.smithi098.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T08:43:02.112 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T08:43:02.112 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.112 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.112 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.112 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.113 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.113 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.113 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.113 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.113 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.113 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.113 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.113 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.113 INFO:teuthology.orchestra.run.smithi098.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T08:43:02.113 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T08:43:02.113 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.114 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.114 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.114 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.114 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.114 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.114 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.114 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.114 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.114 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.114 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.114 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.115 INFO:teuthology.orchestra.run.smithi098.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T08:43:02.115 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T08:43:02.115 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.115 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.115 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.115 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.115 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.115 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.115 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.115 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.115 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.116 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.116 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.116 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.116 INFO:teuthology.orchestra.run.smithi098.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T08:43:02.116 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T08:43:02.116 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.116 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.116 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.116 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.116 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.116 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.117 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.117 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.117 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.117 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.117 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.117 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.117 INFO:teuthology.orchestra.run.smithi098.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T08:43:02.117 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T08:43:02.117 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.117 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.117 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.118 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.118 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.118 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.118 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.118 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.118 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.118 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.118 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.118 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.118 INFO:teuthology.orchestra.run.smithi098.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T08:43:02.118 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T08:43:02.119 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.119 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.119 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.119 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.119 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.119 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.119 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.119 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.119 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.119 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.119 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.119 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.120 INFO:teuthology.orchestra.run.smithi098.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T08:43:02.120 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T08:43:02.120 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.120 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.120 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.120 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.120 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.120 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.120 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.120 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.121 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.121 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.121 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.121 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.121 INFO:teuthology.orchestra.run.smithi098.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T08:43:02.121 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T08:43:02.121 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.121 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.121 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.121 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.121 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.121 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.122 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.122 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.122 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.122 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.122 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.122 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.122 INFO:teuthology.orchestra.run.smithi098.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T08:43:02.122 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T08:43:02.122 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.122 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.122 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.123 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "admin", 2024-09-19T08:43:02.123 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.123 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.123 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.123 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.123 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.123 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.123 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.123 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.123 INFO:teuthology.orchestra.run.smithi098.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T08:43:02.123 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T08:43:02.124 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.124 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.124 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.124 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.124 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.124 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.124 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.124 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.124 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.124 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.124 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.124 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.124 INFO:teuthology.orchestra.run.smithi098.stdout: "GRAFANA_API_URL": { 2024-09-19T08:43:02.125 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "GRAFANA_API_URL", 2024-09-19T08:43:02.125 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.125 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.125 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.125 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.125 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.125 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.125 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.125 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.125 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.125 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.126 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.126 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.126 INFO:teuthology.orchestra.run.smithi098.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T08:43:02.126 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T08:43:02.126 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.126 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.126 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.126 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "admin", 2024-09-19T08:43:02.126 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.126 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.126 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.126 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.127 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.127 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.127 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.127 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.127 INFO:teuthology.orchestra.run.smithi098.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T08:43:02.127 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T08:43:02.127 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.127 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.127 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.127 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.127 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.127 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.128 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.128 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.128 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.128 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.128 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.128 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.128 INFO:teuthology.orchestra.run.smithi098.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T08:43:02.128 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T08:43:02.128 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.128 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.128 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.129 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.129 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.129 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.129 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.129 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.129 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.129 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.129 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.129 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.129 INFO:teuthology.orchestra.run.smithi098.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T08:43:02.129 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T08:43:02.129 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.130 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.130 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.130 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.130 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.130 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.130 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.130 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.130 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.130 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.130 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.130 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.131 INFO:teuthology.orchestra.run.smithi098.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T08:43:02.131 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T08:43:02.131 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.131 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.131 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.131 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.131 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.131 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.131 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.131 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.131 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.131 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.132 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.132 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.132 INFO:teuthology.orchestra.run.smithi098.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T08:43:02.132 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T08:43:02.132 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.132 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.132 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.132 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.132 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.132 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.132 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.132 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.133 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.133 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.133 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.133 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.133 INFO:teuthology.orchestra.run.smithi098.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T08:43:02.133 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T08:43:02.133 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.133 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.133 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.133 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.133 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.133 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.134 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.134 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.134 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.134 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.134 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.134 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.134 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T08:43:02.134 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T08:43:02.134 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.134 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.135 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.135 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.135 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.135 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.135 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.135 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.135 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.135 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.135 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.135 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.135 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T08:43:02.136 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T08:43:02.136 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.136 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.136 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.136 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.136 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.136 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.136 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.136 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.136 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.136 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.137 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.137 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.137 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T08:43:02.137 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T08:43:02.137 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.137 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.137 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.137 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.137 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.137 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.137 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.138 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.138 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.138 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.138 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.138 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.138 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T08:43:02.138 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T08:43:02.138 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.138 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.138 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.138 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.138 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.139 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.139 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.139 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.139 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.139 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.139 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.139 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.139 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T08:43:02.139 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T08:43:02.139 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.139 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.140 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.140 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.140 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.140 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.140 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.140 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.140 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.140 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.140 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.140 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.140 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T08:43:02.140 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T08:43:02.141 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.141 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.141 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.141 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.141 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.141 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.141 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.141 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.141 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.141 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.141 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.141 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.142 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T08:43:02.142 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T08:43:02.142 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.142 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.142 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.142 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.142 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.142 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.142 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.142 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.142 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.142 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.143 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.143 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.143 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T08:43:02.143 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T08:43:02.143 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.143 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.143 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.143 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.143 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.143 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.143 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.144 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.144 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.144 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.144 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.144 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.144 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T08:43:02.144 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T08:43:02.144 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.144 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.144 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.144 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T08:43:02.145 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.145 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.145 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.145 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.145 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.145 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.145 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.145 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.145 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T08:43:02.145 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T08:43:02.145 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:02.146 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.146 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.146 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "10", 2024-09-19T08:43:02.146 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.146 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.146 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.146 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.146 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.146 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.146 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.146 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.146 INFO:teuthology.orchestra.run.smithi098.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T08:43:02.147 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T08:43:02.147 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:02.147 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.147 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.147 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "8", 2024-09-19T08:43:02.147 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.147 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.147 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.147 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.147 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.147 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.147 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.148 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.148 INFO:teuthology.orchestra.run.smithi098.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T08:43:02.148 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T08:43:02.148 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:02.148 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.148 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.148 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "45", 2024-09-19T08:43:02.148 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.148 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.148 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.148 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.149 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.149 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.149 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.149 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.149 INFO:teuthology.orchestra.run.smithi098.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T08:43:02.149 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T08:43:02.149 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.149 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.149 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.149 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.149 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.149 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.150 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.150 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.150 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.150 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.150 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.150 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.150 INFO:teuthology.orchestra.run.smithi098.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T08:43:02.150 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T08:43:02.150 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.150 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.150 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.150 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "admin", 2024-09-19T08:43:02.151 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.151 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.151 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.151 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.151 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.151 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.151 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.151 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.151 INFO:teuthology.orchestra.run.smithi098.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T08:43:02.151 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T08:43:02.151 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.151 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.152 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.152 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.152 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.152 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.152 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.152 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.152 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.152 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.152 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.152 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.152 INFO:teuthology.orchestra.run.smithi098.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T08:43:02.153 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T08:43:02.153 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.153 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.153 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.153 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.153 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.153 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.153 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.153 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.153 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.153 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.153 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.153 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.154 INFO:teuthology.orchestra.run.smithi098.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T08:43:02.154 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T08:43:02.154 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:02.154 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.154 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.154 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "0", 2024-09-19T08:43:02.154 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.154 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.154 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.154 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.154 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.154 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.155 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.155 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.155 INFO:teuthology.orchestra.run.smithi098.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T08:43:02.155 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T08:43:02.155 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:02.155 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.155 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.155 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "10", 2024-09-19T08:43:02.155 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.155 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.155 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.155 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.155 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.156 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.156 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.156 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.156 INFO:teuthology.orchestra.run.smithi098.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T08:43:02.156 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T08:43:02.156 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:02.156 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.156 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.156 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "5", 2024-09-19T08:43:02.156 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.156 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.157 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.157 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.157 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.157 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.157 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.157 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.157 INFO:teuthology.orchestra.run.smithi098.stdout: "cross_origin_url": { 2024-09-19T08:43:02.157 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "cross_origin_url", 2024-09-19T08:43:02.157 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.157 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.157 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.157 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.158 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.158 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.158 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.158 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.158 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.158 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.158 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.158 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.158 INFO:teuthology.orchestra.run.smithi098.stdout: "crt_file": { 2024-09-19T08:43:02.158 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "crt_file", 2024-09-19T08:43:02.158 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.159 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.159 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.159 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.159 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.159 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.159 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.159 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.159 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.159 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.159 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.159 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.160 INFO:teuthology.orchestra.run.smithi098.stdout: "debug": { 2024-09-19T08:43:02.160 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "debug", 2024-09-19T08:43:02.160 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.160 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.160 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.160 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.160 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.160 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.160 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.160 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Enable/disable debug options", 2024-09-19T08:43:02.160 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.160 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.161 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.161 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.161 INFO:teuthology.orchestra.run.smithi098.stdout: "jwt_token_ttl": { 2024-09-19T08:43:02.161 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "jwt_token_ttl", 2024-09-19T08:43:02.161 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:02.161 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.161 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.161 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "28800", 2024-09-19T08:43:02.161 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.161 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.161 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.162 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.162 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.162 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.162 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.162 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.162 INFO:teuthology.orchestra.run.smithi098.stdout: "key_file": { 2024-09-19T08:43:02.162 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "key_file", 2024-09-19T08:43:02.162 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.162 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.162 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.162 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.162 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.163 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.163 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.163 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.163 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.163 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.163 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.163 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.163 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:02.163 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:02.163 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.163 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.164 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.164 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.164 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.164 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.164 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.164 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.164 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.164 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.164 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.164 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.164 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.165 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.165 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.165 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.165 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.165 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.165 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.165 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:02.165 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:02.165 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.165 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.165 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.166 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.166 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.166 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.166 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.166 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.166 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.166 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.166 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.166 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.166 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:02.166 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:02.167 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.167 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.167 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.167 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:02.167 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.167 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.167 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.167 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.167 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.167 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.167 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.168 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.168 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.168 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.168 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.168 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.168 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.168 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.168 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.168 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:02.168 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:02.169 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.169 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.169 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.169 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.169 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.169 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.169 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.169 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.169 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.169 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.169 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.169 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.170 INFO:teuthology.orchestra.run.smithi098.stdout: "motd": { 2024-09-19T08:43:02.170 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "motd", 2024-09-19T08:43:02.170 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.170 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.170 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.170 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.170 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.170 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.170 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.170 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "The message of the day", 2024-09-19T08:43:02.170 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.171 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.171 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.171 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.171 INFO:teuthology.orchestra.run.smithi098.stdout: "redirect_resolve_ip_addr": { 2024-09-19T08:43:02.171 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T08:43:02.171 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.171 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.171 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.171 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.171 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.171 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.172 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.172 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.172 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.172 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.172 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.172 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.172 INFO:teuthology.orchestra.run.smithi098.stdout: "server_addr": { 2024-09-19T08:43:02.172 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "server_addr", 2024-09-19T08:43:02.172 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.172 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.172 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.173 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "::", 2024-09-19T08:43:02.173 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.173 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.173 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.173 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.173 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.173 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.173 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.173 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.173 INFO:teuthology.orchestra.run.smithi098.stdout: "server_port": { 2024-09-19T08:43:02.173 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "server_port", 2024-09-19T08:43:02.174 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:02.174 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.174 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.174 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "8080", 2024-09-19T08:43:02.174 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.174 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.174 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.174 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.174 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.174 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.174 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.175 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.175 INFO:teuthology.orchestra.run.smithi098.stdout: "ssl": { 2024-09-19T08:43:02.175 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ssl", 2024-09-19T08:43:02.175 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.175 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.175 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.175 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.175 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.175 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.175 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.175 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.175 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.176 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.176 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.176 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.176 INFO:teuthology.orchestra.run.smithi098.stdout: "ssl_server_port": { 2024-09-19T08:43:02.176 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ssl_server_port", 2024-09-19T08:43:02.176 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:02.176 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.176 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.176 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "8443", 2024-09-19T08:43:02.176 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.176 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.177 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.177 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.177 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.177 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.177 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.177 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.177 INFO:teuthology.orchestra.run.smithi098.stdout: "standby_behaviour": { 2024-09-19T08:43:02.177 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "standby_behaviour", 2024-09-19T08:43:02.177 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.177 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.177 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.177 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "redirect", 2024-09-19T08:43:02.178 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.178 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.178 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.178 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.178 INFO:teuthology.orchestra.run.smithi098.stdout: "redirect" 2024-09-19T08:43:02.178 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.178 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.178 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.178 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.178 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.178 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.178 INFO:teuthology.orchestra.run.smithi098.stdout: "standby_error_status_code": { 2024-09-19T08:43:02.178 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "standby_error_status_code", 2024-09-19T08:43:02.179 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:02.179 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.179 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.179 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "500", 2024-09-19T08:43:02.179 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "400", 2024-09-19T08:43:02.179 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "599", 2024-09-19T08:43:02.179 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.179 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.179 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.179 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.179 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.180 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.180 INFO:teuthology.orchestra.run.smithi098.stdout: "url_prefix": { 2024-09-19T08:43:02.180 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "url_prefix", 2024-09-19T08:43:02.180 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.180 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.180 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.180 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.180 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.180 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.180 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.180 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.180 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.181 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.181 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.181 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.181 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.181 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.181 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.181 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "devicehealth", 2024-09-19T08:43:02.181 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:02.181 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:02.181 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:02.181 INFO:teuthology.orchestra.run.smithi098.stdout: "enable_monitoring": { 2024-09-19T08:43:02.181 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "enable_monitoring", 2024-09-19T08:43:02.182 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.182 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.182 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.182 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.182 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.182 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.182 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.182 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "monitor device health metrics", 2024-09-19T08:43:02.182 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.182 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.182 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.182 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.183 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:02.183 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:02.183 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.183 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.183 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.183 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.183 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.183 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.183 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.183 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.183 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.183 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.183 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.184 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.184 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.184 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.184 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.184 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.184 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.184 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.184 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.184 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:02.184 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:02.185 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.185 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.185 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.185 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.185 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.185 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.185 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.185 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.185 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.185 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.185 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.185 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.186 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:02.186 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:02.186 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.186 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.186 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.186 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:02.186 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.186 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.186 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.186 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.186 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.187 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.187 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.187 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.187 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.187 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.187 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.187 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.187 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.187 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.187 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.187 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:02.187 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:02.187 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.188 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.188 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.188 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.188 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.188 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.188 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.188 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.188 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.188 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.188 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.188 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.188 INFO:teuthology.orchestra.run.smithi098.stdout: "mark_out_threshold": { 2024-09-19T08:43:02.189 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "mark_out_threshold", 2024-09-19T08:43:02.189 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:02.189 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.189 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.189 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "2419200", 2024-09-19T08:43:02.189 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.189 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.189 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.189 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T08:43:02.189 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.189 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.189 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.190 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.190 INFO:teuthology.orchestra.run.smithi098.stdout: "pool_name": { 2024-09-19T08:43:02.190 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "pool_name", 2024-09-19T08:43:02.190 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.190 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.190 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.190 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "device_health_metrics", 2024-09-19T08:43:02.190 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.190 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.190 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.190 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T08:43:02.190 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.191 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.191 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.191 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.191 INFO:teuthology.orchestra.run.smithi098.stdout: "retention_period": { 2024-09-19T08:43:02.191 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "retention_period", 2024-09-19T08:43:02.191 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:02.191 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.191 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.191 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "15552000", 2024-09-19T08:43:02.191 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.191 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.191 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.191 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "how long to retain device health metrics", 2024-09-19T08:43:02.192 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.192 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.192 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.192 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.192 INFO:teuthology.orchestra.run.smithi098.stdout: "scrape_frequency": { 2024-09-19T08:43:02.192 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "scrape_frequency", 2024-09-19T08:43:02.192 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:02.192 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.192 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.192 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "86400", 2024-09-19T08:43:02.193 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.193 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.193 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.193 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T08:43:02.193 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.193 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.193 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.193 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.193 INFO:teuthology.orchestra.run.smithi098.stdout: "self_heal": { 2024-09-19T08:43:02.193 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "self_heal", 2024-09-19T08:43:02.194 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.194 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.194 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.194 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.194 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.194 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.194 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.194 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T08:43:02.194 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.194 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.194 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.194 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.195 INFO:teuthology.orchestra.run.smithi098.stdout: "sleep_interval": { 2024-09-19T08:43:02.195 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "sleep_interval", 2024-09-19T08:43:02.195 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:02.195 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.195 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.195 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "600", 2024-09-19T08:43:02.195 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.195 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.195 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.195 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T08:43:02.195 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.196 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.196 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.196 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.196 INFO:teuthology.orchestra.run.smithi098.stdout: "warn_threshold": { 2024-09-19T08:43:02.196 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "warn_threshold", 2024-09-19T08:43:02.196 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:02.196 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.196 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.196 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "7257600", 2024-09-19T08:43:02.196 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.196 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.197 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.197 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T08:43:02.197 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.197 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.197 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.197 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.197 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.197 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.197 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.197 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "diskprediction_local", 2024-09-19T08:43:02.197 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:02.198 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:02.198 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:02.198 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:02.198 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:02.198 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.198 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.198 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.198 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.198 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.198 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.198 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.199 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.199 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.199 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.199 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.199 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.199 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.199 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.199 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.199 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.199 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.199 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.200 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.200 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:02.200 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:02.200 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.200 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.200 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.200 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.200 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.200 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.200 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.200 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.201 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.201 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.201 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.201 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.201 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:02.201 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:02.201 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.201 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.201 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.201 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:02.201 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.201 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.202 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.202 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.202 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.202 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.202 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.202 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.202 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.202 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.202 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.202 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.202 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.202 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.203 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.203 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:02.203 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:02.203 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.203 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.203 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.203 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.203 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.203 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.203 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.203 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.203 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.204 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.204 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.204 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.204 INFO:teuthology.orchestra.run.smithi098.stdout: "predict_interval": { 2024-09-19T08:43:02.204 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "predict_interval", 2024-09-19T08:43:02.204 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.204 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.204 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.204 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "86400", 2024-09-19T08:43:02.204 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.204 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.204 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.204 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.205 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.205 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.205 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.205 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.205 INFO:teuthology.orchestra.run.smithi098.stdout: "predictor_model": { 2024-09-19T08:43:02.205 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "predictor_model", 2024-09-19T08:43:02.205 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.205 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.205 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.205 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "prophetstor", 2024-09-19T08:43:02.205 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.206 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.206 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.206 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.206 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.206 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.206 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.206 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.206 INFO:teuthology.orchestra.run.smithi098.stdout: "sleep_interval": { 2024-09-19T08:43:02.206 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "sleep_interval", 2024-09-19T08:43:02.206 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.206 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.206 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.207 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "600", 2024-09-19T08:43:02.207 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.207 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.207 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.207 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.207 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.207 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.207 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.207 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.207 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.207 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.208 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.208 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "influx", 2024-09-19T08:43:02.208 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": false, 2024-09-19T08:43:02.208 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "influxdb python module not found", 2024-09-19T08:43:02.208 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:02.208 INFO:teuthology.orchestra.run.smithi098.stdout: "batch_size": { 2024-09-19T08:43:02.208 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "batch_size", 2024-09-19T08:43:02.208 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:02.208 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.208 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.208 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "5000", 2024-09-19T08:43:02.208 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.209 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.209 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.209 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T08:43:02.209 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.209 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.209 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.209 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.209 INFO:teuthology.orchestra.run.smithi098.stdout: "database": { 2024-09-19T08:43:02.209 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "database", 2024-09-19T08:43:02.209 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.209 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.209 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.210 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "ceph", 2024-09-19T08:43:02.210 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.210 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.210 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.210 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T08:43:02.210 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.210 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.210 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.210 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.210 INFO:teuthology.orchestra.run.smithi098.stdout: "hostname": { 2024-09-19T08:43:02.210 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "hostname", 2024-09-19T08:43:02.211 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.211 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.211 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.211 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.211 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.211 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.211 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.211 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "InfluxDB server hostname", 2024-09-19T08:43:02.211 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.211 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.211 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.211 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.212 INFO:teuthology.orchestra.run.smithi098.stdout: "interval": { 2024-09-19T08:43:02.212 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "interval", 2024-09-19T08:43:02.212 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:02.212 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.212 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.212 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "30", 2024-09-19T08:43:02.212 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "5", 2024-09-19T08:43:02.212 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.212 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.212 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T08:43:02.213 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.213 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.213 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.213 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.213 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:02.213 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:02.213 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.213 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.213 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.213 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.213 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.213 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.214 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.214 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.214 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.214 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.214 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.214 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.214 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.214 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.214 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.214 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.214 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.214 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.215 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.215 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:02.215 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:02.215 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.215 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.215 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.215 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.215 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.215 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.215 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.215 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.216 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.216 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.216 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.216 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.216 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:02.216 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:02.216 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.216 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.216 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.216 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:02.216 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.217 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.217 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.217 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.217 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.217 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.217 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.217 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.217 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.217 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.217 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.217 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.218 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.218 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.218 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.218 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:02.218 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:02.218 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.218 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.218 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.218 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.218 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.218 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.218 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.219 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.219 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.219 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.219 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.219 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.219 INFO:teuthology.orchestra.run.smithi098.stdout: "password": { 2024-09-19T08:43:02.219 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "password", 2024-09-19T08:43:02.219 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.219 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.219 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.219 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.220 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.220 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.220 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.220 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "password of InfluxDB server user", 2024-09-19T08:43:02.220 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.220 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.220 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.220 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.220 INFO:teuthology.orchestra.run.smithi098.stdout: "port": { 2024-09-19T08:43:02.220 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "port", 2024-09-19T08:43:02.220 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:02.221 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.221 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.221 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "8086", 2024-09-19T08:43:02.221 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.221 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.221 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.221 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "InfluxDB server port", 2024-09-19T08:43:02.221 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.221 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.221 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.221 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.222 INFO:teuthology.orchestra.run.smithi098.stdout: "ssl": { 2024-09-19T08:43:02.222 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "ssl", 2024-09-19T08:43:02.222 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.222 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.222 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.222 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "false", 2024-09-19T08:43:02.222 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.222 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.222 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.222 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T08:43:02.222 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.222 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.223 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.223 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.223 INFO:teuthology.orchestra.run.smithi098.stdout: "threads": { 2024-09-19T08:43:02.223 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "threads", 2024-09-19T08:43:02.223 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:02.223 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.223 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.223 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "5", 2024-09-19T08:43:02.223 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "1", 2024-09-19T08:43:02.223 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "32", 2024-09-19T08:43:02.223 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.224 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T08:43:02.224 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.224 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.224 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.224 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.224 INFO:teuthology.orchestra.run.smithi098.stdout: "username": { 2024-09-19T08:43:02.224 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "username", 2024-09-19T08:43:02.224 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.224 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.224 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.224 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.224 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.225 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.225 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.225 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "username of InfluxDB server user", 2024-09-19T08:43:02.225 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.225 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.225 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.225 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.225 INFO:teuthology.orchestra.run.smithi098.stdout: "verify_ssl": { 2024-09-19T08:43:02.225 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "verify_ssl", 2024-09-19T08:43:02.228 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.228 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.228 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.228 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "true", 2024-09-19T08:43:02.229 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.229 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.229 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.229 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T08:43:02.229 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.229 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.229 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.229 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.229 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.229 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.229 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.229 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "insights", 2024-09-19T08:43:02.230 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:02.230 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:02.230 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:02.230 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:02.230 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:02.230 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.230 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.230 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.230 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.230 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.230 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.231 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.231 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.231 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.231 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.231 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.231 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.231 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.231 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.231 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.231 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.231 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.231 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.232 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.232 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:02.232 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:02.232 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.232 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.232 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.232 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.232 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.232 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.232 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.232 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.232 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.233 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.233 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.233 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.233 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:02.233 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:02.233 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.233 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.233 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.233 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:02.233 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.233 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.234 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.234 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.234 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.234 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.234 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.234 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.234 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.234 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.234 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.234 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.235 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.235 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.235 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.235 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:02.235 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:02.235 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.235 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.235 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.235 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.235 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.235 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.236 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.236 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.236 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.236 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.236 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.236 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.236 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.236 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.236 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.236 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "iostat", 2024-09-19T08:43:02.237 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:02.237 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:02.237 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:02.237 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:02.237 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:02.237 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.237 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.237 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.237 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.237 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.237 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.238 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.238 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.238 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.238 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.238 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.238 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.238 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.238 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.238 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.238 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.238 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.239 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.239 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.239 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:02.239 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:02.239 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.239 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.239 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.239 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.239 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.239 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.239 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.239 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.240 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.240 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.240 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.240 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.240 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:02.240 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:02.240 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.240 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.240 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.240 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:02.240 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.240 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.241 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.241 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.241 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.241 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.241 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.241 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.241 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.241 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.241 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.241 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.241 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.241 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.242 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.242 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:02.242 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:02.242 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.242 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.242 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.242 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.242 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.242 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.242 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.242 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.242 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.243 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.243 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.243 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.243 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.243 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.243 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.243 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "localpool", 2024-09-19T08:43:02.243 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:02.243 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:02.243 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:02.243 INFO:teuthology.orchestra.run.smithi098.stdout: "failure_domain": { 2024-09-19T08:43:02.244 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "failure_domain", 2024-09-19T08:43:02.244 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.244 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.244 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.244 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "host", 2024-09-19T08:43:02.244 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.244 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.244 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.244 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "failure domain for any created local pool", 2024-09-19T08:43:02.244 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T08:43:02.244 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.245 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.245 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.245 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:02.245 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:02.245 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.245 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.245 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.245 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.245 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.245 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.245 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.245 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.246 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.246 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.246 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.246 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.246 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.246 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.246 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.246 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.246 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.246 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.246 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.247 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:02.247 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:02.247 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.247 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.247 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.247 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.247 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.247 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.247 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.247 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.247 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.248 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.248 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.248 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.248 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:02.248 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:02.248 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.248 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.248 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.248 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:02.248 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.248 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.248 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.249 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.249 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.249 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.249 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.249 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.249 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.249 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.249 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.249 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.249 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.249 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.249 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.250 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:02.250 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:02.250 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.250 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.250 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.250 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.250 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.250 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.250 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.250 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.250 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.251 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.251 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.251 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.251 INFO:teuthology.orchestra.run.smithi098.stdout: "min_size": { 2024-09-19T08:43:02.251 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "min_size", 2024-09-19T08:43:02.251 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:02.251 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.251 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.251 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.251 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.251 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.252 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.252 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "default min_size for any created local pool", 2024-09-19T08:43:02.252 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T08:43:02.252 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.252 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.252 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.252 INFO:teuthology.orchestra.run.smithi098.stdout: "num_rep": { 2024-09-19T08:43:02.252 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "num_rep", 2024-09-19T08:43:02.252 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:02.252 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.252 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.252 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "3", 2024-09-19T08:43:02.253 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.253 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.253 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.253 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "default replica count for any created local pool", 2024-09-19T08:43:02.253 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.253 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.253 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.253 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.253 INFO:teuthology.orchestra.run.smithi098.stdout: "pg_num": { 2024-09-19T08:43:02.253 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "pg_num", 2024-09-19T08:43:02.253 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:02.254 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.254 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.254 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "128", 2024-09-19T08:43:02.254 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.254 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.254 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.254 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T08:43:02.254 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.254 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.254 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.254 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.255 INFO:teuthology.orchestra.run.smithi098.stdout: "prefix": { 2024-09-19T08:43:02.255 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "prefix", 2024-09-19T08:43:02.255 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.255 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.255 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.255 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.255 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.255 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.255 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.255 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "name prefix for any created local pool", 2024-09-19T08:43:02.255 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.256 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.256 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.256 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.256 INFO:teuthology.orchestra.run.smithi098.stdout: "subtree": { 2024-09-19T08:43:02.256 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "subtree", 2024-09-19T08:43:02.256 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.256 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.256 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.256 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "rack", 2024-09-19T08:43:02.256 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.256 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.257 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.257 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T08:43:02.257 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T08:43:02.257 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.257 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.257 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.257 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.257 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.257 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.257 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "mirroring", 2024-09-19T08:43:02.257 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:02.257 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:02.258 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:02.258 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:02.258 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:02.258 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.258 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.258 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.258 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.258 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.258 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.258 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.258 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.259 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.259 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.259 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.259 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.259 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.259 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.259 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.259 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.259 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.259 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.260 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.260 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:02.260 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:02.260 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.260 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.260 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.260 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.260 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.260 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.260 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.261 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.261 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.261 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.261 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.261 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.261 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:02.261 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:02.261 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.261 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.261 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.261 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:02.261 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.262 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.262 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.262 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.262 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.262 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.262 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.262 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.262 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.262 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.262 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.262 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.262 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.263 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.263 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.263 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:02.263 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:02.263 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.263 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.263 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.263 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.263 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.263 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.263 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.263 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.264 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.264 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.264 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.264 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.264 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.264 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.264 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.264 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "nfs", 2024-09-19T08:43:02.264 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:02.264 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:02.264 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:02.264 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:02.265 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:02.265 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.265 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.265 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.265 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.265 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.265 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.265 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.265 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.265 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.265 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.265 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.266 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.266 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.266 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.266 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.266 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.266 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.266 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.266 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.266 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:02.266 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:02.266 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.267 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.267 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.267 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.267 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.267 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.267 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.267 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.267 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.267 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.267 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.268 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.268 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:02.268 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:02.268 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.268 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.268 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.268 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:02.268 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.268 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.268 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.268 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.269 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.269 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.269 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.269 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.269 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.269 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.269 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.269 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.269 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.269 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.269 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.270 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:02.270 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:02.270 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.270 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.270 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.270 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.270 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.270 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.270 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.270 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.270 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.270 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.271 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.271 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.271 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.271 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.271 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.271 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "orchestrator", 2024-09-19T08:43:02.271 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:02.271 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:02.271 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:02.271 INFO:teuthology.orchestra.run.smithi098.stdout: "fail_fs": { 2024-09-19T08:43:02.271 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "fail_fs", 2024-09-19T08:43:02.272 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.272 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.272 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.272 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.272 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.272 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.272 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.272 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T08:43:02.272 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.272 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.272 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.273 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.273 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:02.273 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:02.273 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.273 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.273 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.273 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.273 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.273 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.273 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.273 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.274 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.274 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.274 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.274 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.274 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.274 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.274 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.274 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.274 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.274 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.275 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.275 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:02.275 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:02.275 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.275 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.275 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.275 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.275 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.275 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.275 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.275 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.275 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.276 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.276 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.276 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.276 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:02.276 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:02.276 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.276 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.276 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.276 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:02.276 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.276 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.277 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.277 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.277 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.277 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.277 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.277 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.277 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.277 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.277 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.277 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.277 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.278 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.278 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.278 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:02.278 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:02.278 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.278 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.278 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.278 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.278 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.278 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.278 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.278 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.279 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.279 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.279 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.279 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.279 INFO:teuthology.orchestra.run.smithi098.stdout: "orchestrator": { 2024-09-19T08:43:02.279 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "orchestrator", 2024-09-19T08:43:02.279 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.279 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.279 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.279 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.279 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.279 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.280 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.280 INFO:teuthology.orchestra.run.smithi098.stdout: "cephadm", 2024-09-19T08:43:02.280 INFO:teuthology.orchestra.run.smithi098.stdout: "rook", 2024-09-19T08:43:02.280 INFO:teuthology.orchestra.run.smithi098.stdout: "test_orchestrator" 2024-09-19T08:43:02.280 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.280 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Orchestrator backend", 2024-09-19T08:43:02.280 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.280 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.280 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.280 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.280 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.281 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.281 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.281 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "osd_perf_query", 2024-09-19T08:43:02.281 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:02.281 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:02.281 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:02.281 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:02.281 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:02.281 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.281 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.281 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.281 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.281 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.282 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.282 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.282 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.282 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.282 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.282 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.282 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.282 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.282 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.282 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.282 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.282 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.282 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.283 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.283 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:02.283 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:02.283 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.283 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.283 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.283 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.283 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.283 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.283 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.283 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.283 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.283 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.284 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.284 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.284 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:02.284 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:02.284 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.284 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.284 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.284 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:02.284 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.284 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.284 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.284 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.285 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.285 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.285 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.285 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.285 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.285 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.285 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.285 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.285 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.285 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.285 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.286 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:02.286 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:02.286 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.286 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.286 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.286 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.286 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.286 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.286 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.286 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.287 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.287 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.287 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.287 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.287 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.287 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.287 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.287 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "osd_support", 2024-09-19T08:43:02.287 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:02.287 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:02.287 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:02.287 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:02.288 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:02.288 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.288 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.288 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.288 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.288 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.288 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.288 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.288 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.288 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.288 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.288 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.289 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.289 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.289 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.289 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.289 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.289 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.289 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.289 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.289 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:02.289 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:02.289 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.289 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.290 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.290 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.290 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.290 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.290 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.290 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.290 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.290 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.290 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.290 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.290 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:02.290 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:02.291 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.291 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.291 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.291 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:02.291 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.291 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.291 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.291 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.291 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.291 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.291 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.291 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.292 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.292 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.292 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.292 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.292 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.292 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.292 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.292 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:02.292 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:02.292 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.292 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.292 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.292 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.293 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.293 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.293 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.293 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.293 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.293 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.293 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.293 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.293 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.293 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.293 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.294 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "pg_autoscaler", 2024-09-19T08:43:02.294 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:02.294 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:02.294 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:02.294 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:02.294 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:02.294 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.294 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.294 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.294 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.294 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.294 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.295 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.295 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.295 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.295 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.295 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.295 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.295 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.295 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.295 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.295 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.295 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.295 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.296 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.296 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:02.296 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:02.296 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.296 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.296 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.296 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.296 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.296 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.296 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.296 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.296 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.297 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.297 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.297 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.297 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:02.297 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:02.297 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.297 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.297 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.297 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:02.297 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.297 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.298 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.298 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.298 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.298 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.298 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.298 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.298 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.298 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.298 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.298 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.298 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.298 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.299 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.299 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:02.299 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:02.299 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.299 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.299 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.299 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.299 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.299 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.299 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.299 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.299 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.300 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.300 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.300 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.300 INFO:teuthology.orchestra.run.smithi098.stdout: "sleep_interval": { 2024-09-19T08:43:02.300 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "sleep_interval", 2024-09-19T08:43:02.300 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:02.300 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.300 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.300 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "60", 2024-09-19T08:43:02.300 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.300 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.301 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.301 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.301 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.301 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.301 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.301 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.301 INFO:teuthology.orchestra.run.smithi098.stdout: "threshold": { 2024-09-19T08:43:02.301 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "threshold", 2024-09-19T08:43:02.301 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "float", 2024-09-19T08:43:02.301 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.301 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.301 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "3.0", 2024-09-19T08:43:02.302 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "1.0", 2024-09-19T08:43:02.302 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.302 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.302 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "scaling threshold", 2024-09-19T08:43:02.302 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T08:43:02.302 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.302 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.302 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.302 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.302 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.302 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.303 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "progress", 2024-09-19T08:43:02.303 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:02.303 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:02.303 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:02.303 INFO:teuthology.orchestra.run.smithi098.stdout: "allow_pg_recovery_event": { 2024-09-19T08:43:02.303 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "allow_pg_recovery_event", 2024-09-19T08:43:02.303 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.303 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.303 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.303 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.303 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.303 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.304 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.304 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T08:43:02.304 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.304 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.304 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.304 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.304 INFO:teuthology.orchestra.run.smithi098.stdout: "enabled": { 2024-09-19T08:43:02.304 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "enabled", 2024-09-19T08:43:02.304 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.304 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.304 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.304 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.305 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.305 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.305 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.305 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.305 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.305 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.305 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.305 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.305 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:02.305 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:02.305 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.305 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.306 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.306 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.306 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.306 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.306 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.306 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.306 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.306 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.306 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.306 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.306 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.307 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.307 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.307 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.307 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.307 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.307 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.307 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:02.307 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:02.307 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.307 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.307 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.307 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.308 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.308 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.308 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.308 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.308 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.308 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.308 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.308 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.308 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:02.308 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:02.308 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.309 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.309 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.309 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:02.309 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.309 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.309 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.309 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.309 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.309 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.309 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.309 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.309 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.310 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.310 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.310 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.310 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.310 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.310 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.310 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:02.310 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:02.310 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.310 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.311 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.311 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.311 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.311 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.311 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.311 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.311 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.311 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.311 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.311 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.311 INFO:teuthology.orchestra.run.smithi098.stdout: "max_completed_events": { 2024-09-19T08:43:02.311 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "max_completed_events", 2024-09-19T08:43:02.312 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:02.312 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.312 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.312 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "50", 2024-09-19T08:43:02.312 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.312 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.312 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.312 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "number of past completed events to remember", 2024-09-19T08:43:02.312 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.312 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.312 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.312 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.313 INFO:teuthology.orchestra.run.smithi098.stdout: "sleep_interval": { 2024-09-19T08:43:02.313 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "sleep_interval", 2024-09-19T08:43:02.313 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:02.313 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.313 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.313 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "5", 2024-09-19T08:43:02.313 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.313 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.313 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.313 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "how long the module is going to sleep", 2024-09-19T08:43:02.313 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.313 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.314 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.314 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.314 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.314 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.314 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.314 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "prometheus", 2024-09-19T08:43:02.314 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:02.314 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:02.314 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:02.314 INFO:teuthology.orchestra.run.smithi098.stdout: "cache": { 2024-09-19T08:43:02.314 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "cache", 2024-09-19T08:43:02.315 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.315 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.315 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.315 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.315 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.315 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.315 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.315 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.315 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.315 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.315 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.315 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.316 INFO:teuthology.orchestra.run.smithi098.stdout: "exclude_perf_counters": { 2024-09-19T08:43:02.316 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "exclude_perf_counters", 2024-09-19T08:43:02.316 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.316 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.316 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.316 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.316 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.316 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.316 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.316 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T08:43:02.316 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T08:43:02.316 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.317 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.317 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.317 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:02.317 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:02.317 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.317 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.317 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.317 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.317 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.317 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.317 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.317 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.318 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.318 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.318 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.318 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.318 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.318 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.318 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.318 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.318 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.318 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.318 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.318 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:02.319 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:02.319 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.319 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.319 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.319 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.319 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.319 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.319 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.319 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.319 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.319 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.320 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.320 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.320 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:02.320 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:02.320 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.320 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.320 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.320 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:02.320 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.320 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.320 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.321 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.321 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.321 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.321 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.321 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.321 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.321 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.321 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.321 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.321 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.321 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.322 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.322 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:02.322 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:02.322 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.322 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.322 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.322 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.322 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.322 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.322 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.322 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.322 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.323 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.323 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.323 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.323 INFO:teuthology.orchestra.run.smithi098.stdout: "rbd_stats_pools": { 2024-09-19T08:43:02.323 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rbd_stats_pools", 2024-09-19T08:43:02.323 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.323 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.323 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.323 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.323 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.323 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.324 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.324 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.324 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.324 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.324 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.324 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.324 INFO:teuthology.orchestra.run.smithi098.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T08:43:02.324 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T08:43:02.324 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:02.324 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.324 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.325 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "300", 2024-09-19T08:43:02.325 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.325 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.325 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.325 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.325 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.325 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.325 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.325 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.325 INFO:teuthology.orchestra.run.smithi098.stdout: "scrape_interval": { 2024-09-19T08:43:02.325 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "scrape_interval", 2024-09-19T08:43:02.326 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "float", 2024-09-19T08:43:02.326 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.326 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.326 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "15.0", 2024-09-19T08:43:02.326 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.326 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.326 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.326 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.326 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.326 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.326 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.326 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.327 INFO:teuthology.orchestra.run.smithi098.stdout: "server_addr": { 2024-09-19T08:43:02.327 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "server_addr", 2024-09-19T08:43:02.327 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.327 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.327 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.327 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "::", 2024-09-19T08:43:02.327 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.327 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.327 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.327 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T08:43:02.327 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.327 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.328 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.328 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.328 INFO:teuthology.orchestra.run.smithi098.stdout: "server_port": { 2024-09-19T08:43:02.328 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "server_port", 2024-09-19T08:43:02.328 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:02.328 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.328 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.328 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "9283", 2024-09-19T08:43:02.328 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.328 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.328 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.328 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T08:43:02.329 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.329 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.329 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.329 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.329 INFO:teuthology.orchestra.run.smithi098.stdout: "stale_cache_strategy": { 2024-09-19T08:43:02.329 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "stale_cache_strategy", 2024-09-19T08:43:02.329 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.329 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.329 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.329 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "log", 2024-09-19T08:43:02.329 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.330 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.330 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.330 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.330 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.330 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.330 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.330 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.330 INFO:teuthology.orchestra.run.smithi098.stdout: "standby_behaviour": { 2024-09-19T08:43:02.330 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "standby_behaviour", 2024-09-19T08:43:02.330 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.330 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.330 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.330 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "default", 2024-09-19T08:43:02.331 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.331 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.331 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.331 INFO:teuthology.orchestra.run.smithi098.stdout: "default", 2024-09-19T08:43:02.331 INFO:teuthology.orchestra.run.smithi098.stdout: "error" 2024-09-19T08:43:02.331 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.331 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.331 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.331 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.331 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.331 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.332 INFO:teuthology.orchestra.run.smithi098.stdout: "standby_error_status_code": { 2024-09-19T08:43:02.332 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "standby_error_status_code", 2024-09-19T08:43:02.332 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:02.332 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.332 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.332 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "500", 2024-09-19T08:43:02.332 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "400", 2024-09-19T08:43:02.332 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "599", 2024-09-19T08:43:02.332 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.332 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.332 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.332 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.333 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.333 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.333 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.333 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.333 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.333 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rbd_support", 2024-09-19T08:43:02.333 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:02.333 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:02.333 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:02.333 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:02.333 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:02.333 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.334 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.334 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.334 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.334 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.334 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.334 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.334 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.334 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.334 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.334 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.334 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.335 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.335 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.335 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.335 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.335 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.335 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.335 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.335 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:02.335 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:02.335 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.335 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.335 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.336 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.336 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.336 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.336 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.336 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.336 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.336 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.336 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.336 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.336 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:02.336 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:02.336 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.337 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.337 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.337 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:02.337 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.337 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.337 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.337 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.337 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.337 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.337 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.337 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.337 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.338 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.338 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.338 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.338 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.338 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.338 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.338 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:02.338 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:02.338 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.338 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.338 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.338 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.339 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.339 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.339 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.339 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.339 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.339 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.339 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.339 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.339 INFO:teuthology.orchestra.run.smithi098.stdout: "max_concurrent_snap_create": { 2024-09-19T08:43:02.339 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "max_concurrent_snap_create", 2024-09-19T08:43:02.339 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:02.340 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.340 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.340 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "10", 2024-09-19T08:43:02.340 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.340 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.340 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.340 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.340 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.340 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.340 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.340 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.340 INFO:teuthology.orchestra.run.smithi098.stdout: "mirror_snapshot_schedule": { 2024-09-19T08:43:02.341 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T08:43:02.341 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.341 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.341 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.341 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.341 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.341 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.341 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.341 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.341 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.341 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.342 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.342 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.342 INFO:teuthology.orchestra.run.smithi098.stdout: "trash_purge_schedule": { 2024-09-19T08:43:02.342 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "trash_purge_schedule", 2024-09-19T08:43:02.342 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.342 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.342 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.342 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.342 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.342 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.342 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.342 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.342 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.343 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.343 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.343 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.343 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.343 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.343 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.343 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "restful", 2024-09-19T08:43:02.343 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:02.343 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:02.343 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:02.343 INFO:teuthology.orchestra.run.smithi098.stdout: "enable_auth": { 2024-09-19T08:43:02.344 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "enable_auth", 2024-09-19T08:43:02.344 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.344 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.344 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.344 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.344 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.344 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.344 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.344 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.345 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.345 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.345 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.345 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.345 INFO:teuthology.orchestra.run.smithi098.stdout: "key_file": { 2024-09-19T08:43:02.345 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "key_file", 2024-09-19T08:43:02.345 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.345 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.345 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.345 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.345 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.346 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.346 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.346 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.346 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.346 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.346 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.346 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.346 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:02.346 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:02.346 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.346 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.346 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.347 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.347 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.347 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.347 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.347 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.347 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.347 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.347 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.347 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.347 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.347 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.348 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.348 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.348 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.348 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.348 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.348 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:02.348 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:02.348 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.348 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.348 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.348 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.348 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.349 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.349 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.349 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.349 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.349 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.349 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.349 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.349 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:02.349 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:02.349 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.349 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.350 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.350 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:02.350 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.350 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.350 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.350 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.350 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.350 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.350 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.350 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.350 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.350 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.351 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.351 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.351 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.351 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.351 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.351 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:02.351 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:02.351 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.351 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.351 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.352 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.352 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.352 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.352 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.352 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.352 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.352 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.352 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.352 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.352 INFO:teuthology.orchestra.run.smithi098.stdout: "server_addr": { 2024-09-19T08:43:02.352 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "server_addr", 2024-09-19T08:43:02.352 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.353 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.353 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.353 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.353 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.353 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.353 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.353 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.353 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.353 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.353 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.353 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.354 INFO:teuthology.orchestra.run.smithi098.stdout: "server_port": { 2024-09-19T08:43:02.354 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "server_port", 2024-09-19T08:43:02.354 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.354 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.354 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.354 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.354 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.354 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.354 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.354 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.354 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.355 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.355 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.355 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.355 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.355 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.355 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.355 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "selftest", 2024-09-19T08:43:02.355 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:02.355 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:02.355 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:02.355 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:02.355 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:02.356 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.356 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.356 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.356 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.356 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.356 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.356 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.356 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.356 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.356 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.356 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.357 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.357 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.357 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.357 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.357 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.357 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.357 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.357 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.357 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:02.357 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:02.357 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.357 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.357 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.358 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.358 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.358 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.358 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.358 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.358 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.358 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.358 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.358 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.358 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:02.358 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:02.358 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.359 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.359 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.359 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:02.359 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.359 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.359 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.359 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.359 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.359 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.359 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.360 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.360 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.360 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.360 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.360 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.360 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.360 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.360 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.360 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:02.360 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:02.360 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.361 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.361 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.361 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.361 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.361 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.361 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.361 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.361 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.361 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.361 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.361 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.362 INFO:teuthology.orchestra.run.smithi098.stdout: "roption1": { 2024-09-19T08:43:02.362 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "roption1", 2024-09-19T08:43:02.362 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.362 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.362 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.362 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.362 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.362 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.362 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.362 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.362 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.362 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.363 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.363 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.363 INFO:teuthology.orchestra.run.smithi098.stdout: "roption2": { 2024-09-19T08:43:02.363 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "roption2", 2024-09-19T08:43:02.363 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.363 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.363 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.363 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "xyz", 2024-09-19T08:43:02.363 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.363 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.363 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.364 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.364 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.364 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.364 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.364 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.364 INFO:teuthology.orchestra.run.smithi098.stdout: "rwoption1": { 2024-09-19T08:43:02.364 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rwoption1", 2024-09-19T08:43:02.364 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.364 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.364 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.364 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.364 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.365 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.365 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.365 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.365 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.365 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.365 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.365 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.365 INFO:teuthology.orchestra.run.smithi098.stdout: "rwoption2": { 2024-09-19T08:43:02.365 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rwoption2", 2024-09-19T08:43:02.365 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:02.365 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.366 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.366 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.366 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.366 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.366 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.366 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.366 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.366 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.366 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.366 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.367 INFO:teuthology.orchestra.run.smithi098.stdout: "rwoption3": { 2024-09-19T08:43:02.367 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rwoption3", 2024-09-19T08:43:02.367 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "float", 2024-09-19T08:43:02.367 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.367 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.367 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.367 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.367 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.367 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.367 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.367 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.368 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.368 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.368 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.368 INFO:teuthology.orchestra.run.smithi098.stdout: "rwoption4": { 2024-09-19T08:43:02.368 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rwoption4", 2024-09-19T08:43:02.368 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.368 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.368 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.368 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.368 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.368 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.369 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.369 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.369 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.369 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.369 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.369 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.369 INFO:teuthology.orchestra.run.smithi098.stdout: "rwoption5": { 2024-09-19T08:43:02.369 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rwoption5", 2024-09-19T08:43:02.369 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.369 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.369 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.370 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.370 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.370 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.370 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.370 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.370 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.370 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.370 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.370 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.370 INFO:teuthology.orchestra.run.smithi098.stdout: "rwoption6": { 2024-09-19T08:43:02.370 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rwoption6", 2024-09-19T08:43:02.370 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.371 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.371 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.371 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.371 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.371 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.371 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.371 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.371 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.371 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.371 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.371 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.371 INFO:teuthology.orchestra.run.smithi098.stdout: "rwoption7": { 2024-09-19T08:43:02.372 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rwoption7", 2024-09-19T08:43:02.372 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:02.372 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.372 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.372 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.372 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "1", 2024-09-19T08:43:02.372 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "42", 2024-09-19T08:43:02.372 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.372 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.372 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.372 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.373 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.373 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.373 INFO:teuthology.orchestra.run.smithi098.stdout: "testkey": { 2024-09-19T08:43:02.373 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "testkey", 2024-09-19T08:43:02.373 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.373 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.373 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.373 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.373 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.373 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.373 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.373 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.374 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.374 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.374 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.374 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.374 INFO:teuthology.orchestra.run.smithi098.stdout: "testlkey": { 2024-09-19T08:43:02.374 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "testlkey", 2024-09-19T08:43:02.374 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.374 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.374 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.374 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.374 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.374 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.375 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.375 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.375 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.375 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.375 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.375 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.375 INFO:teuthology.orchestra.run.smithi098.stdout: "testnewline": { 2024-09-19T08:43:02.375 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "testnewline", 2024-09-19T08:43:02.375 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.375 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.375 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.376 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.376 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.376 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.376 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.376 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.376 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.376 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.376 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.376 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.376 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.376 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.377 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.377 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "snap_schedule", 2024-09-19T08:43:02.377 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:02.377 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:02.377 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:02.377 INFO:teuthology.orchestra.run.smithi098.stdout: "allow_m_granularity": { 2024-09-19T08:43:02.377 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "allow_m_granularity", 2024-09-19T08:43:02.377 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.377 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.377 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.377 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.377 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.378 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.378 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.378 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T08:43:02.378 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.378 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.378 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.378 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.378 INFO:teuthology.orchestra.run.smithi098.stdout: "dump_on_update": { 2024-09-19T08:43:02.378 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "dump_on_update", 2024-09-19T08:43:02.378 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.378 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.378 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.379 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.379 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.379 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.379 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.379 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "dump database to debug log on update", 2024-09-19T08:43:02.379 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.379 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.379 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.379 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.379 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:02.379 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:02.380 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.380 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.380 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.380 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.380 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.380 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.380 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.380 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.380 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.380 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.380 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.380 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.381 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.381 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.381 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.381 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.381 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.381 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.381 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.381 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:02.381 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:02.381 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.382 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.382 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.382 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.382 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.382 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.382 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.382 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.382 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.382 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.382 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.382 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.383 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:02.383 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:02.383 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.383 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.383 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.383 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:02.383 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.383 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.383 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.383 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.383 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.383 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.384 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.384 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.384 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.384 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.384 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.384 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.384 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.384 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.384 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.384 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:02.384 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:02.385 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.385 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.385 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.385 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.385 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.385 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.385 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.385 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.385 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.385 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.385 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.386 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.386 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.386 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.386 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.386 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "stats", 2024-09-19T08:43:02.386 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:02.386 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:02.386 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:02.386 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:02.386 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:02.386 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.386 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.387 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.387 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.387 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.387 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.387 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.387 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.387 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.387 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.387 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.387 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.387 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.387 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.388 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.388 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.388 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.388 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.388 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.388 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:02.388 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:02.388 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.388 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.388 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.388 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.389 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.389 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.389 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.389 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.389 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.389 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.389 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.389 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.389 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:02.389 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:02.390 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.390 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.390 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.390 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:02.390 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.390 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.390 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.390 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.390 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.390 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.390 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.391 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.391 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.391 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.391 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.391 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.391 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.391 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.391 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.391 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:02.391 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:02.391 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.392 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.392 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.392 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.392 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.392 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.392 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.392 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.392 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.392 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.392 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.392 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.392 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.393 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.393 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.393 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "status", 2024-09-19T08:43:02.393 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:02.393 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:02.393 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:02.393 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:02.393 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:02.393 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.393 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.393 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.394 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.394 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.394 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.394 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.394 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.394 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.394 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.394 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.394 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.394 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.394 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.394 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.394 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.395 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.395 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.395 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.395 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:02.395 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:02.395 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.395 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.395 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.395 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.395 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.395 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.396 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.396 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.396 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.396 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.396 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.396 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.396 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:02.396 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:02.396 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.396 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.396 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.397 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:02.397 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.397 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.397 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.397 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.397 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.397 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.397 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.397 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.397 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.397 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.397 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.398 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.398 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.398 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.398 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.398 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:02.398 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:02.398 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.398 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.398 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.398 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.398 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.399 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.399 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.399 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.399 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.399 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.399 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.399 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.399 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.399 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.399 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.399 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "telegraf", 2024-09-19T08:43:02.400 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:02.400 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:02.400 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:02.400 INFO:teuthology.orchestra.run.smithi098.stdout: "address": { 2024-09-19T08:43:02.400 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "address", 2024-09-19T08:43:02.400 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.400 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.400 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.400 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T08:43:02.400 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.400 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.400 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.401 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.401 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.401 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.401 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.401 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.401 INFO:teuthology.orchestra.run.smithi098.stdout: "interval": { 2024-09-19T08:43:02.401 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "interval", 2024-09-19T08:43:02.401 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:02.401 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.401 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.401 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "15", 2024-09-19T08:43:02.401 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.402 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.402 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.402 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.402 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.402 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.402 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.402 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.402 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:02.402 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:02.402 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.402 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.403 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.403 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.403 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.403 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.403 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.403 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.403 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.403 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.403 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.403 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.403 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.403 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.403 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.403 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.404 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.404 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.404 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.404 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:02.404 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:02.404 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.404 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.404 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.404 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.404 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.404 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.404 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.405 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.405 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.405 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.405 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.405 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.405 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:02.405 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:02.405 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.405 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.405 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.405 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:02.405 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.406 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.406 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.406 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.406 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.406 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.406 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.406 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.406 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.406 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.406 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.406 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.407 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.407 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.407 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.407 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:02.407 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:02.407 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.407 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.407 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.407 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.407 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.407 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.407 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.408 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.408 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.408 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.408 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.408 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.408 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.408 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.408 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.408 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "telemetry", 2024-09-19T08:43:02.408 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:02.408 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:02.409 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:02.409 INFO:teuthology.orchestra.run.smithi098.stdout: "channel_basic": { 2024-09-19T08:43:02.409 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "channel_basic", 2024-09-19T08:43:02.409 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.409 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.409 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.409 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.409 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.409 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.409 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.409 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T08:43:02.410 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.410 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.410 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.410 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.410 INFO:teuthology.orchestra.run.smithi098.stdout: "channel_crash": { 2024-09-19T08:43:02.410 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "channel_crash", 2024-09-19T08:43:02.410 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.410 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.410 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.410 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.411 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.411 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.411 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.411 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T08:43:02.411 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.411 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.411 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.411 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.411 INFO:teuthology.orchestra.run.smithi098.stdout: "channel_device": { 2024-09-19T08:43:02.411 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "channel_device", 2024-09-19T08:43:02.411 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.411 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.412 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.412 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.412 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.412 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.412 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.412 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T08:43:02.412 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.412 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.412 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.412 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.412 INFO:teuthology.orchestra.run.smithi098.stdout: "channel_ident": { 2024-09-19T08:43:02.413 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "channel_ident", 2024-09-19T08:43:02.413 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.413 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.413 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.413 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.413 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.413 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.413 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.413 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T08:43:02.413 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.413 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.413 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.414 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.414 INFO:teuthology.orchestra.run.smithi098.stdout: "channel_perf": { 2024-09-19T08:43:02.414 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "channel_perf", 2024-09-19T08:43:02.414 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.414 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.414 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.414 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.414 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.414 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.414 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.414 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T08:43:02.414 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.415 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.415 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.415 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.415 INFO:teuthology.orchestra.run.smithi098.stdout: "contact": { 2024-09-19T08:43:02.415 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "contact", 2024-09-19T08:43:02.415 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.415 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.415 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.415 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.415 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.415 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.416 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.416 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.416 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.416 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.416 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.416 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.416 INFO:teuthology.orchestra.run.smithi098.stdout: "description": { 2024-09-19T08:43:02.416 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "description", 2024-09-19T08:43:02.416 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.416 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.416 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.416 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.417 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.417 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.417 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.417 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.417 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.417 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.417 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.417 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.417 INFO:teuthology.orchestra.run.smithi098.stdout: "device_url": { 2024-09-19T08:43:02.417 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "device_url", 2024-09-19T08:43:02.417 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.418 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.418 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.418 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T08:43:02.418 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.418 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.418 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.418 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.418 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.418 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.418 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.418 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.418 INFO:teuthology.orchestra.run.smithi098.stdout: "enabled": { 2024-09-19T08:43:02.419 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "enabled", 2024-09-19T08:43:02.419 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.419 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.419 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.419 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.419 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.419 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.419 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.419 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.419 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.419 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.419 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.420 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.420 INFO:teuthology.orchestra.run.smithi098.stdout: "interval": { 2024-09-19T08:43:02.420 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "interval", 2024-09-19T08:43:02.420 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:02.420 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.420 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.420 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "24", 2024-09-19T08:43:02.420 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "8", 2024-09-19T08:43:02.420 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.420 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.420 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.421 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.421 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.421 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.421 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.421 INFO:teuthology.orchestra.run.smithi098.stdout: "last_opt_revision": { 2024-09-19T08:43:02.421 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "last_opt_revision", 2024-09-19T08:43:02.421 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:02.421 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.421 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.421 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "1", 2024-09-19T08:43:02.421 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.421 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.422 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.422 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.422 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.422 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.422 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.422 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.422 INFO:teuthology.orchestra.run.smithi098.stdout: "leaderboard": { 2024-09-19T08:43:02.422 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "leaderboard", 2024-09-19T08:43:02.422 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.422 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.422 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.422 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.422 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.423 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.423 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.423 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.423 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.423 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.423 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.423 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.423 INFO:teuthology.orchestra.run.smithi098.stdout: "leaderboard_description": { 2024-09-19T08:43:02.423 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "leaderboard_description", 2024-09-19T08:43:02.423 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.423 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.423 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.423 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.424 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.424 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.424 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.424 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.424 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.424 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.424 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.424 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.424 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:02.424 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:02.424 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.424 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.425 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.425 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.425 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.425 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.425 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.425 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.425 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.425 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.425 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.425 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.425 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.425 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.426 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.426 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.426 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.426 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.426 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.426 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:02.426 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:02.426 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.426 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.426 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.427 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.427 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.427 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.427 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.427 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.427 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.427 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.427 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.427 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.427 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:02.427 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:02.427 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.428 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.428 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.428 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:02.428 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.428 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.428 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.428 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.428 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.428 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.428 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.428 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.429 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.429 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.429 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.429 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.429 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.429 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.429 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.429 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:02.429 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:02.429 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.429 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.429 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.430 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.430 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.430 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.430 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.430 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.430 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.430 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.430 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.430 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.430 INFO:teuthology.orchestra.run.smithi098.stdout: "organization": { 2024-09-19T08:43:02.430 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "organization", 2024-09-19T08:43:02.431 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.431 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.431 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.431 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.431 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.431 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.431 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.431 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.432 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.432 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.432 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.432 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.432 INFO:teuthology.orchestra.run.smithi098.stdout: "proxy": { 2024-09-19T08:43:02.432 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "proxy", 2024-09-19T08:43:02.432 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.433 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.433 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.433 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.433 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.433 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.433 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.433 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.433 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.433 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.433 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.433 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.433 INFO:teuthology.orchestra.run.smithi098.stdout: "url": { 2024-09-19T08:43:02.434 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "url", 2024-09-19T08:43:02.434 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.434 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.434 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.434 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T08:43:02.434 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.434 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.434 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.434 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.434 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.434 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.435 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.435 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.435 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.435 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.435 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.435 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "test_orchestrator", 2024-09-19T08:43:02.435 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:02.435 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:02.435 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:02.435 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:02.435 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:02.436 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.436 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.436 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.436 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.436 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.436 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.436 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.436 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.436 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.436 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.436 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.436 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.437 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.437 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.437 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.437 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.437 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.437 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.437 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.437 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:02.437 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:02.437 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.437 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.438 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.438 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.438 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.438 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.438 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.438 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.438 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.438 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.438 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.438 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.439 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:02.439 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:02.439 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.439 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.439 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.439 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:02.439 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.439 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.439 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.439 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.439 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.440 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.440 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.440 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.440 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.440 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.440 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.440 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.440 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.440 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.440 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.441 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:02.441 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:02.441 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.441 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.441 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.441 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.441 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.441 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.441 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.441 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.441 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.442 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.442 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.442 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.442 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.442 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.442 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.442 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "volumes", 2024-09-19T08:43:02.442 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:02.442 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:02.442 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:02.442 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:02.442 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:02.443 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.443 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.443 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.443 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.443 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.443 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.443 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.443 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.443 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.443 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.443 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.443 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.444 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.444 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.444 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.444 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.444 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.444 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.444 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.444 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:02.444 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:02.444 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.444 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.445 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.445 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.445 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.445 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.445 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.445 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.445 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.445 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.445 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.445 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.445 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:02.445 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:02.445 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.446 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.446 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.446 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:02.446 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.446 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.446 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.446 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.446 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.446 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.446 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.446 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.446 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.447 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.447 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.447 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.447 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.447 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.447 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.447 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:02.447 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:02.447 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.447 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.447 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.447 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.448 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.448 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.448 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.448 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.448 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.448 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.448 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.448 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.448 INFO:teuthology.orchestra.run.smithi098.stdout: "max_concurrent_clones": { 2024-09-19T08:43:02.448 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "max_concurrent_clones", 2024-09-19T08:43:02.448 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:02.448 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.449 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.449 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "4", 2024-09-19T08:43:02.449 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.449 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.449 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.449 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T08:43:02.449 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.449 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.449 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.449 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.449 INFO:teuthology.orchestra.run.smithi098.stdout: "periodic_async_work": { 2024-09-19T08:43:02.450 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "periodic_async_work", 2024-09-19T08:43:02.450 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.450 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.450 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.450 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.450 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.450 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.450 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.450 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Periodically check for async work", 2024-09-19T08:43:02.450 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.450 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.450 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.451 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.451 INFO:teuthology.orchestra.run.smithi098.stdout: "snapshot_clone_delay": { 2024-09-19T08:43:02.451 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "snapshot_clone_delay", 2024-09-19T08:43:02.451 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:02.451 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.451 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.451 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "0", 2024-09-19T08:43:02.451 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.451 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.451 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.451 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T08:43:02.452 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.452 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.452 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.452 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.452 INFO:teuthology.orchestra.run.smithi098.stdout: "snapshot_clone_no_wait": { 2024-09-19T08:43:02.452 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T08:43:02.452 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.452 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.452 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.452 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "True", 2024-09-19T08:43:02.452 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.453 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.453 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.453 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T08:43:02.453 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.453 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.453 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.453 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.453 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.453 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.453 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.454 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "zabbix", 2024-09-19T08:43:02.454 INFO:teuthology.orchestra.run.smithi098.stdout: "can_run": true, 2024-09-19T08:43:02.454 INFO:teuthology.orchestra.run.smithi098.stdout: "error_string": "", 2024-09-19T08:43:02.454 INFO:teuthology.orchestra.run.smithi098.stdout: "module_options": { 2024-09-19T08:43:02.454 INFO:teuthology.orchestra.run.smithi098.stdout: "discovery_interval": { 2024-09-19T08:43:02.454 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "discovery_interval", 2024-09-19T08:43:02.454 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "uint", 2024-09-19T08:43:02.454 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.454 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.454 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "100", 2024-09-19T08:43:02.454 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.454 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.455 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.455 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.455 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.455 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.455 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.455 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.455 INFO:teuthology.orchestra.run.smithi098.stdout: "identifier": { 2024-09-19T08:43:02.455 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "identifier", 2024-09-19T08:43:02.455 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.455 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.455 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.456 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.456 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.456 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.456 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.456 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.456 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.456 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.456 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.456 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.456 INFO:teuthology.orchestra.run.smithi098.stdout: "interval": { 2024-09-19T08:43:02.456 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "interval", 2024-09-19T08:43:02.457 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "secs", 2024-09-19T08:43:02.457 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.457 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.457 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "60", 2024-09-19T08:43:02.457 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.457 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.457 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.457 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.457 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.457 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.457 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.458 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.458 INFO:teuthology.orchestra.run.smithi098.stdout: "log_level": { 2024-09-19T08:43:02.458 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_level", 2024-09-19T08:43:02.458 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.458 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.458 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.458 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.458 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.458 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.458 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.458 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.458 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.459 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.459 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.459 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.459 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.459 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.459 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.459 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.459 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.459 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.459 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.459 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster": { 2024-09-19T08:43:02.460 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster", 2024-09-19T08:43:02.460 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.460 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.460 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.460 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.460 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.460 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.460 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.460 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.460 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.460 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.461 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.461 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.461 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_cluster_level": { 2024-09-19T08:43:02.461 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_cluster_level", 2024-09-19T08:43:02.461 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.461 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.461 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.461 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "info", 2024-09-19T08:43:02.461 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.461 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.461 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [ 2024-09-19T08:43:02.462 INFO:teuthology.orchestra.run.smithi098.stdout: "", 2024-09-19T08:43:02.462 INFO:teuthology.orchestra.run.smithi098.stdout: "critical", 2024-09-19T08:43:02.462 INFO:teuthology.orchestra.run.smithi098.stdout: "debug", 2024-09-19T08:43:02.462 INFO:teuthology.orchestra.run.smithi098.stdout: "error", 2024-09-19T08:43:02.462 INFO:teuthology.orchestra.run.smithi098.stdout: "info", 2024-09-19T08:43:02.462 INFO:teuthology.orchestra.run.smithi098.stdout: "warning" 2024-09-19T08:43:02.462 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.462 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.462 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.462 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.462 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.462 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.463 INFO:teuthology.orchestra.run.smithi098.stdout: "log_to_file": { 2024-09-19T08:43:02.463 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "log_to_file", 2024-09-19T08:43:02.463 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "bool", 2024-09-19T08:43:02.463 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.463 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 1, 2024-09-19T08:43:02.463 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "False", 2024-09-19T08:43:02.463 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.463 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.463 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.463 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.463 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.463 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.464 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.464 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.464 INFO:teuthology.orchestra.run.smithi098.stdout: "zabbix_host": { 2024-09-19T08:43:02.464 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "zabbix_host", 2024-09-19T08:43:02.464 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.464 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.464 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.464 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "", 2024-09-19T08:43:02.464 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.464 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.464 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.465 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.465 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.465 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.465 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.465 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.465 INFO:teuthology.orchestra.run.smithi098.stdout: "zabbix_port": { 2024-09-19T08:43:02.465 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "zabbix_port", 2024-09-19T08:43:02.465 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "int", 2024-09-19T08:43:02.465 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.465 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.465 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "10051", 2024-09-19T08:43:02.465 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.466 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.466 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.466 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.466 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.466 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.466 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.466 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.466 INFO:teuthology.orchestra.run.smithi098.stdout: "zabbix_sender": { 2024-09-19T08:43:02.466 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "zabbix_sender", 2024-09-19T08:43:02.466 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "str", 2024-09-19T08:43:02.466 INFO:teuthology.orchestra.run.smithi098.stdout: "level": "advanced", 2024-09-19T08:43:02.467 INFO:teuthology.orchestra.run.smithi098.stdout: "flags": 0, 2024-09-19T08:43:02.467 INFO:teuthology.orchestra.run.smithi098.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T08:43:02.467 INFO:teuthology.orchestra.run.smithi098.stdout: "min": "", 2024-09-19T08:43:02.467 INFO:teuthology.orchestra.run.smithi098.stdout: "max": "", 2024-09-19T08:43:02.467 INFO:teuthology.orchestra.run.smithi098.stdout: "enum_allowed": [], 2024-09-19T08:43:02.467 INFO:teuthology.orchestra.run.smithi098.stdout: "desc": "", 2024-09-19T08:43:02.467 INFO:teuthology.orchestra.run.smithi098.stdout: "long_desc": "", 2024-09-19T08:43:02.467 INFO:teuthology.orchestra.run.smithi098.stdout: "tags": [], 2024-09-19T08:43:02.467 INFO:teuthology.orchestra.run.smithi098.stdout: "see_also": [] 2024-09-19T08:43:02.467 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.467 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.467 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.468 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.468 INFO:teuthology.orchestra.run.smithi098.stdout: "services": {}, 2024-09-19T08:43:02.468 INFO:teuthology.orchestra.run.smithi098.stdout: "always_on_modules": { 2024-09-19T08:43:02.468 INFO:teuthology.orchestra.run.smithi098.stdout: "octopus": [ 2024-09-19T08:43:02.468 INFO:teuthology.orchestra.run.smithi098.stdout: "balancer", 2024-09-19T08:43:02.468 INFO:teuthology.orchestra.run.smithi098.stdout: "crash", 2024-09-19T08:43:02.468 INFO:teuthology.orchestra.run.smithi098.stdout: "devicehealth", 2024-09-19T08:43:02.468 INFO:teuthology.orchestra.run.smithi098.stdout: "orchestrator", 2024-09-19T08:43:02.468 INFO:teuthology.orchestra.run.smithi098.stdout: "pg_autoscaler", 2024-09-19T08:43:02.468 INFO:teuthology.orchestra.run.smithi098.stdout: "progress", 2024-09-19T08:43:02.468 INFO:teuthology.orchestra.run.smithi098.stdout: "rbd_support", 2024-09-19T08:43:02.469 INFO:teuthology.orchestra.run.smithi098.stdout: "status", 2024-09-19T08:43:02.469 INFO:teuthology.orchestra.run.smithi098.stdout: "telemetry", 2024-09-19T08:43:02.469 INFO:teuthology.orchestra.run.smithi098.stdout: "volumes" 2024-09-19T08:43:02.469 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.469 INFO:teuthology.orchestra.run.smithi098.stdout: "pacific": [ 2024-09-19T08:43:02.469 INFO:teuthology.orchestra.run.smithi098.stdout: "balancer", 2024-09-19T08:43:02.469 INFO:teuthology.orchestra.run.smithi098.stdout: "crash", 2024-09-19T08:43:02.469 INFO:teuthology.orchestra.run.smithi098.stdout: "devicehealth", 2024-09-19T08:43:02.469 INFO:teuthology.orchestra.run.smithi098.stdout: "orchestrator", 2024-09-19T08:43:02.469 INFO:teuthology.orchestra.run.smithi098.stdout: "pg_autoscaler", 2024-09-19T08:43:02.469 INFO:teuthology.orchestra.run.smithi098.stdout: "progress", 2024-09-19T08:43:02.469 INFO:teuthology.orchestra.run.smithi098.stdout: "rbd_support", 2024-09-19T08:43:02.470 INFO:teuthology.orchestra.run.smithi098.stdout: "status", 2024-09-19T08:43:02.470 INFO:teuthology.orchestra.run.smithi098.stdout: "telemetry", 2024-09-19T08:43:02.470 INFO:teuthology.orchestra.run.smithi098.stdout: "volumes" 2024-09-19T08:43:02.470 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.470 INFO:teuthology.orchestra.run.smithi098.stdout: "quincy": [ 2024-09-19T08:43:02.470 INFO:teuthology.orchestra.run.smithi098.stdout: "balancer", 2024-09-19T08:43:02.470 INFO:teuthology.orchestra.run.smithi098.stdout: "crash", 2024-09-19T08:43:02.470 INFO:teuthology.orchestra.run.smithi098.stdout: "devicehealth", 2024-09-19T08:43:02.470 INFO:teuthology.orchestra.run.smithi098.stdout: "orchestrator", 2024-09-19T08:43:02.470 INFO:teuthology.orchestra.run.smithi098.stdout: "pg_autoscaler", 2024-09-19T08:43:02.470 INFO:teuthology.orchestra.run.smithi098.stdout: "progress", 2024-09-19T08:43:02.471 INFO:teuthology.orchestra.run.smithi098.stdout: "rbd_support", 2024-09-19T08:43:02.471 INFO:teuthology.orchestra.run.smithi098.stdout: "status", 2024-09-19T08:43:02.471 INFO:teuthology.orchestra.run.smithi098.stdout: "telemetry", 2024-09-19T08:43:02.471 INFO:teuthology.orchestra.run.smithi098.stdout: "volumes" 2024-09-19T08:43:02.471 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.471 INFO:teuthology.orchestra.run.smithi098.stdout: "reef": [ 2024-09-19T08:43:02.471 INFO:teuthology.orchestra.run.smithi098.stdout: "balancer", 2024-09-19T08:43:02.471 INFO:teuthology.orchestra.run.smithi098.stdout: "crash", 2024-09-19T08:43:02.471 INFO:teuthology.orchestra.run.smithi098.stdout: "devicehealth", 2024-09-19T08:43:02.471 INFO:teuthology.orchestra.run.smithi098.stdout: "orchestrator", 2024-09-19T08:43:02.471 INFO:teuthology.orchestra.run.smithi098.stdout: "pg_autoscaler", 2024-09-19T08:43:02.472 INFO:teuthology.orchestra.run.smithi098.stdout: "progress", 2024-09-19T08:43:02.472 INFO:teuthology.orchestra.run.smithi098.stdout: "rbd_support", 2024-09-19T08:43:02.472 INFO:teuthology.orchestra.run.smithi098.stdout: "status", 2024-09-19T08:43:02.472 INFO:teuthology.orchestra.run.smithi098.stdout: "telemetry", 2024-09-19T08:43:02.472 INFO:teuthology.orchestra.run.smithi098.stdout: "volumes" 2024-09-19T08:43:02.472 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:43:02.472 INFO:teuthology.orchestra.run.smithi098.stdout: "squid": [ 2024-09-19T08:43:02.472 INFO:teuthology.orchestra.run.smithi098.stdout: "balancer", 2024-09-19T08:43:02.472 INFO:teuthology.orchestra.run.smithi098.stdout: "crash", 2024-09-19T08:43:02.472 INFO:teuthology.orchestra.run.smithi098.stdout: "devicehealth", 2024-09-19T08:43:02.472 INFO:teuthology.orchestra.run.smithi098.stdout: "orchestrator", 2024-09-19T08:43:02.473 INFO:teuthology.orchestra.run.smithi098.stdout: "pg_autoscaler", 2024-09-19T08:43:02.473 INFO:teuthology.orchestra.run.smithi098.stdout: "progress", 2024-09-19T08:43:02.473 INFO:teuthology.orchestra.run.smithi098.stdout: "rbd_support", 2024-09-19T08:43:02.473 INFO:teuthology.orchestra.run.smithi098.stdout: "status", 2024-09-19T08:43:02.473 INFO:teuthology.orchestra.run.smithi098.stdout: "telemetry", 2024-09-19T08:43:02.473 INFO:teuthology.orchestra.run.smithi098.stdout: "volumes" 2024-09-19T08:43:02.473 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-19T08:43:02.473 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.473 INFO:teuthology.orchestra.run.smithi098.stdout: "last_failure_osd_epoch": 0, 2024-09-19T08:43:02.473 INFO:teuthology.orchestra.run.smithi098.stdout: "active_clients": [ 2024-09-19T08:43:02.473 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.473 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "devicehealth", 2024-09-19T08:43:02.474 INFO:teuthology.orchestra.run.smithi098.stdout: "addrvec": [ 2024-09-19T08:43:02.474 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.474 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v2", 2024-09-19T08:43:02.474 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "172.21.15.98:0", 2024-09-19T08:43:02.474 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 2131240070 2024-09-19T08:43:02.474 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.474 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-19T08:43:02.474 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.474 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.474 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "libcephsqlite", 2024-09-19T08:43:02.474 INFO:teuthology.orchestra.run.smithi098.stdout: "addrvec": [ 2024-09-19T08:43:02.474 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.475 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v2", 2024-09-19T08:43:02.475 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "172.21.15.98:0", 2024-09-19T08:43:02.475 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 1465974372 2024-09-19T08:43:02.475 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.475 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-19T08:43:02.475 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.475 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.475 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "rbd_support", 2024-09-19T08:43:02.475 INFO:teuthology.orchestra.run.smithi098.stdout: "addrvec": [ 2024-09-19T08:43:02.475 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.476 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v2", 2024-09-19T08:43:02.476 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "172.21.15.98:0", 2024-09-19T08:43:02.476 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 1109131925 2024-09-19T08:43:02.476 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.476 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-19T08:43:02.476 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.476 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.476 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "volumes", 2024-09-19T08:43:02.476 INFO:teuthology.orchestra.run.smithi098.stdout: "addrvec": [ 2024-09-19T08:43:02.476 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.476 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v2", 2024-09-19T08:43:02.477 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "172.21.15.98:0", 2024-09-19T08:43:02.477 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 3519546751 2024-09-19T08:43:02.477 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.477 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-19T08:43:02.477 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.477 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.477 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "volumes", 2024-09-19T08:43:02.477 INFO:teuthology.orchestra.run.smithi098.stdout: "addrvec": [ 2024-09-19T08:43:02.477 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.477 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v2", 2024-09-19T08:43:02.477 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "172.21.15.98:0", 2024-09-19T08:43:02.478 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 3957109968 2024-09-19T08:43:02.478 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.478 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-19T08:43:02.478 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.478 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.478 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "volumes", 2024-09-19T08:43:02.478 INFO:teuthology.orchestra.run.smithi098.stdout: "addrvec": [ 2024-09-19T08:43:02.478 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.478 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v2", 2024-09-19T08:43:02.478 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "172.21.15.98:0", 2024-09-19T08:43:02.478 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 1028111778 2024-09-19T08:43:02.479 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.479 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-19T08:43:02.479 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.479 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.479 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "volumes", 2024-09-19T08:43:02.479 INFO:teuthology.orchestra.run.smithi098.stdout: "addrvec": [ 2024-09-19T08:43:02.479 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.479 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v2", 2024-09-19T08:43:02.479 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "172.21.15.98:0", 2024-09-19T08:43:02.479 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 1688754871 2024-09-19T08:43:02.479 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.479 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-19T08:43:02.479 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.480 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.480 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "volumes", 2024-09-19T08:43:02.480 INFO:teuthology.orchestra.run.smithi098.stdout: "addrvec": [ 2024-09-19T08:43:02.480 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.480 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v2", 2024-09-19T08:43:02.480 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "172.21.15.98:0", 2024-09-19T08:43:02.480 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 1649589780 2024-09-19T08:43:02.480 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.480 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-19T08:43:02.480 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.480 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.480 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "volumes", 2024-09-19T08:43:02.481 INFO:teuthology.orchestra.run.smithi098.stdout: "addrvec": [ 2024-09-19T08:43:02.481 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.481 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v2", 2024-09-19T08:43:02.481 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "172.21.15.98:0", 2024-09-19T08:43:02.481 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 1973723249 2024-09-19T08:43:02.481 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.481 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-19T08:43:02.481 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.481 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.481 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "volumes", 2024-09-19T08:43:02.482 INFO:teuthology.orchestra.run.smithi098.stdout: "addrvec": [ 2024-09-19T08:43:02.482 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.482 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v2", 2024-09-19T08:43:02.482 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "172.21.15.98:0", 2024-09-19T08:43:02.482 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 2085575004 2024-09-19T08:43:02.482 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.482 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-19T08:43:02.482 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:02.482 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.482 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "volumes", 2024-09-19T08:43:02.482 INFO:teuthology.orchestra.run.smithi098.stdout: "addrvec": [ 2024-09-19T08:43:02.482 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:02.483 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v2", 2024-09-19T08:43:02.483 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "172.21.15.98:0", 2024-09-19T08:43:02.483 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 1268825931 2024-09-19T08:43:02.483 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.483 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-19T08:43:02.483 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:02.483 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-19T08:43:02.483 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:43:02.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.541+0000 7fcd66e6d640 1 -- 172.21.15.98:0/3855413466 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fcd4c067c20 msgr2=0x7fcd4c06a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:02.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.541+0000 7fcd66e6d640 1 --2- 172.21.15.98:0/3855413466 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fcd4c067c20 0x7fcd4c06a0e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcd60158500 tx=0x7fcd54009040 comp rx=0 tx=0).stop 2024-09-19T08:43:02.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.541+0000 7fcd66e6d640 1 -- 172.21.15.98:0/3855413466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd6006c1d0 msgr2=0x7fcd6014cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:02.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.541+0000 7fcd66e6d640 1 --2- 172.21.15.98:0/3855413466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd6006c1d0 0x7fcd6014cd80 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fcd5001a0d0 tx=0x7fcd50019240 comp rx=0 tx=0).stop 2024-09-19T08:43:02.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.541+0000 7fcd66e6d640 1 -- 172.21.15.98:0/3855413466 shutdown_connections 2024-09-19T08:43:02.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.541+0000 7fcd66e6d640 1 --2- 172.21.15.98:0/3855413466 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7fcd4c067c20 0x7fcd4c06a0e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:02.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.541+0000 7fcd66e6d640 1 --2- 172.21.15.98:0/3855413466 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd6015c550 0x7fcd6014e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:02.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.541+0000 7fcd66e6d640 1 --2- 172.21.15.98:0/3855413466 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd600cfad0 0x7fcd6014e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:02.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.541+0000 7fcd66e6d640 1 --2- 172.21.15.98:0/3855413466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd6006c1d0 0x7fcd6014cd80 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:02.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.541+0000 7fcd66e6d640 1 -- 172.21.15.98:0/3855413466 >> 172.21.15.98:0/3855413466 conn(0x7fcd600b2010 msgr2=0x7fcd6015a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:02.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.541+0000 7fcd66e6d640 1 -- 172.21.15.98:0/3855413466 shutdown_connections 2024-09-19T08:43:02.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:01.541+0000 7fcd66e6d640 1 -- 172.21.15.98:0/3855413466 wait complete. 2024-09-19T08:43:02.491 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr fail y 2024-09-19T08:43:02.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.625+0000 7f20ff4b5640 1 Processor -- start 2024-09-19T08:43:02.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.625+0000 7f20ff4b5640 1 -- start start 2024-09-19T08:43:02.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.625+0000 7f20ff4b5640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f20f806c3a0 0x7f20f80cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:02.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.625+0000 7f20ff4b5640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20f806d310 con 0x7f20f80cf900 2024-09-19T08:43:02.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.625+0000 7f20ff4b5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20f806d5f0 con 0x7f20f815c550 2024-09-19T08:43:02.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.625+0000 7f20ff4b5640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f20f806d8d0 con 0x7f20f806c3a0 2024-09-19T08:43:02.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.625+0000 7f20fca50640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f20f806c3a0 0x7f20f80cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:02.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.625+0000 7f20fda52640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f20f80cf900 0x7f20f80723a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:37468/0 (socket says 172.21.15.98:37468) 2024-09-19T08:43:02.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.625+0000 7f20fda52640 1 -- 172.21.15.98:0/1096594657 learned_addr learned my addr 172.21.15.98:0/1096594657 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:02.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.625+0000 7f20fca50640 1 -- 172.21.15.98:0/1096594657 >> v1:172.21.15.139:6789/0 conn(0x7f20f815c550 legacy=0x7f20f806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:02.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.625+0000 7f20fca50640 1 -- 172.21.15.98:0/1096594657 >> v1:172.21.15.98:6789/0 conn(0x7f20f80cf900 legacy=0x7f20f80723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:02.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.625+0000 7f20fca50640 1 -- 172.21.15.98:0/1096594657 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f806dbb0 con 0x7f20f806c3a0 2024-09-19T08:43:02.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.629+0000 7f20fca50640 1 --2- 172.21.15.98:0/1096594657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f20f806c3a0 0x7f20f80cf3c0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f20e80097c0 tx=0x7f20e801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4240d2da52a5d8d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:02.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.629+0000 7f20effff640 1 -- 172.21.15.98:0/1096594657 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e80b0070 con 0x7f20f806c3a0 2024-09-19T08:43:02.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.629+0000 7f20effff640 1 -- 172.21.15.98:0/1096594657 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20e80ab600 con 0x7f20f806c3a0 2024-09-19T08:43:02.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.629+0000 7f20effff640 1 -- 172.21.15.98:0/1096594657 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e80ab9d0 con 0x7f20f806c3a0 2024-09-19T08:43:02.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.629+0000 7f20ff4b5640 1 -- 172.21.15.98:0/1096594657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f20f806c3a0 msgr2=0x7f20f80cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:02.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.629+0000 7f20ff4b5640 1 --2- 172.21.15.98:0/1096594657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f20f806c3a0 0x7f20f80cf3c0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f20e80097c0 tx=0x7f20e801c460 comp rx=0 tx=0).stop 2024-09-19T08:43:02.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.629+0000 7f20ff4b5640 1 -- 172.21.15.98:0/1096594657 shutdown_connections 2024-09-19T08:43:02.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.629+0000 7f20ff4b5640 1 --2- 172.21.15.98:0/1096594657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f20f806c3a0 0x7f20f80cf3c0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:02.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.629+0000 7f20ff4b5640 1 -- 172.21.15.98:0/1096594657 >> 172.21.15.98:0/1096594657 conn(0x7f20f80b2010 msgr2=0x7f20f80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:02.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.629+0000 7f20ff4b5640 1 -- 172.21.15.98:0/1096594657 shutdown_connections 2024-09-19T08:43:02.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.629+0000 7f20ff4b5640 1 -- 172.21.15.98:0/1096594657 wait complete. 2024-09-19T08:43:02.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.629+0000 7f20ff4b5640 1 Processor -- start 2024-09-19T08:43:02.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.629+0000 7f20ff4b5640 1 -- start start 2024-09-19T08:43:02.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.633+0000 7f20ff4b5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f20f806c3a0 0x7f20f814bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:02.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.633+0000 7f20ff4b5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20f80cf900 0x7f20f8158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:02.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.633+0000 7f20ff4b5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f20f815c550 0x7f20f8147ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:02.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.633+0000 7f20ff4b5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f20f80736d0 con 0x7f20f806c3a0 2024-09-19T08:43:02.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.633+0000 7f20ff4b5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f20f8073550 con 0x7f20f80cf900 2024-09-19T08:43:02.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.633+0000 7f20ff4b5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f20f816a240 con 0x7f20f815c550 2024-09-19T08:43:02.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.633+0000 7f20fd251640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f20f806c3a0 0x7f20f814bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:02.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.633+0000 7f20fd251640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f20f806c3a0 0x7f20f814bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53370/0 (socket says 172.21.15.98:53370) 2024-09-19T08:43:02.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.633+0000 7f20fd251640 1 -- 172.21.15.98:0/1307015041 learned_addr learned my addr 172.21.15.98:0/1307015041 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:02.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.633+0000 7f20fd251640 1 -- 172.21.15.98:0/1307015041 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f20f815c550 msgr2=0x7f20f8147ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:02.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.633+0000 7f20fd251640 1 --2- 172.21.15.98:0/1307015041 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f20f815c550 0x7f20f8147ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:02.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.633+0000 7f20fd251640 1 -- 172.21.15.98:0/1307015041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20f80cf900 msgr2=0x7f20f8158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:02.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.633+0000 7f20fca50640 1 --2- 172.21.15.98:0/1307015041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20f80cf900 0x7f20f8158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:02.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.633+0000 7f20fd251640 1 --2- 172.21.15.98:0/1307015041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20f80cf900 0x7f20f8158870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:02.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.633+0000 7f20fd251640 1 -- 172.21.15.98:0/1307015041 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f80cc960 con 0x7f20f806c3a0 2024-09-19T08:43:02.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.633+0000 7f20fd251640 1 --2- 172.21.15.98:0/1307015041 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f20f806c3a0 0x7f20f814bfc0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f20f400c4c0 tx=0x7f20f4001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:02.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.633+0000 7f20edffb640 1 -- 172.21.15.98:0/1307015041 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f4017070 con 0x7f20f806c3a0 2024-09-19T08:43:02.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.633+0000 7f20edffb640 1 -- 172.21.15.98:0/1307015041 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20f4003600 con 0x7f20f806c3a0 2024-09-19T08:43:02.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.633+0000 7f20edffb640 1 -- 172.21.15.98:0/1307015041 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f40039f0 con 0x7f20f806c3a0 2024-09-19T08:43:02.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.633+0000 7f20ff4b5640 1 -- 172.21.15.98:0/1307015041 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20f8156770 con 0x7f20f806c3a0 2024-09-19T08:43:02.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.633+0000 7f20ff4b5640 1 -- 172.21.15.98:0/1307015041 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20f8156d30 con 0x7f20f806c3a0 2024-09-19T08:43:02.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.633+0000 7f20ff4b5640 1 -- 172.21.15.98:0/1307015041 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20f80cf3c0 con 0x7f20f806c3a0 2024-09-19T08:43:02.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.633+0000 7f20edffb640 1 -- 172.21.15.98:0/1307015041 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 12) ==== 87802+0+0 (secure 0 0 0) 0x7f20f401a070 con 0x7f20f806c3a0 2024-09-19T08:43:02.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.633+0000 7f20edffb640 1 --2- 172.21.15.98:0/1307015041 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f20dc067cb0 0x7f20dc06a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:02.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.633+0000 7f20edffb640 1 -- 172.21.15.98:0/1307015041 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5825+0+0 (secure 0 0 0) 0x7f20f4085670 con 0x7f20f806c3a0 2024-09-19T08:43:02.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.633+0000 7f20fca50640 1 --2- 172.21.15.98:0/1307015041 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f20dc067cb0 0x7f20dc06a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:02.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.633+0000 7f20fca50640 1 --2- 172.21.15.98:0/1307015041 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f20dc067cb0 0x7f20dc06a170 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f20e80ad000 tx=0x7f20e8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:02.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.637+0000 7f20edffb640 1 -- 172.21.15.98:0/1307015041 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f20f4055ce0 con 0x7f20f806c3a0 2024-09-19T08:43:02.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.837+0000 7f20ff4b5640 1 -- 172.21.15.98:0/1307015041 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "mgr fail", "who": "y"} v 0) -- 0x7f20f80cdc90 con 0x7f20f806c3a0 2024-09-19T08:43:02.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.845+0000 7f20edffb640 1 -- 172.21.15.98:0/1307015041 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "mgr fail", "who": "y"}]=0 v13) ==== 68+0+0 (secure 0 0 0) 0x7f20f40555e0 con 0x7f20f806c3a0 2024-09-19T08:43:02.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.849+0000 7f20ff4b5640 1 -- 172.21.15.98:0/1307015041 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f20dc067cb0 msgr2=0x7f20dc06a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:02.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.849+0000 7f20ff4b5640 1 --2- 172.21.15.98:0/1307015041 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f20dc067cb0 0x7f20dc06a170 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f20e80ad000 tx=0x7f20e8008040 comp rx=0 tx=0).stop 2024-09-19T08:43:02.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.849+0000 7f20ff4b5640 1 -- 172.21.15.98:0/1307015041 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f20f806c3a0 msgr2=0x7f20f814bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:02.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.849+0000 7f20edffb640 1 -- 172.21.15.98:0/1307015041 <== mon.0 v2:172.21.15.98:3300/0 8 ==== mgrmap(e 13) ==== 43764+0+0 (secure 0 0 0) 0x7f20f4051c50 con 0x7f20f806c3a0 2024-09-19T08:43:02.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.849+0000 7f20ff4b5640 1 --2- 172.21.15.98:0/1307015041 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f20f806c3a0 0x7f20f814bfc0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f20f400c4c0 tx=0x7f20f4001e80 comp rx=0 tx=0).stop 2024-09-19T08:43:02.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.849+0000 7f20ff4b5640 1 -- 172.21.15.98:0/1307015041 shutdown_connections 2024-09-19T08:43:02.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.849+0000 7f20ff4b5640 1 --2- 172.21.15.98:0/1307015041 >> [v2:172.21.15.98:6832/2040310606,v1:172.21.15.98:6833/2040310606] conn(0x7f20dc067cb0 0x7f20dc06a170 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:02.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.849+0000 7f20ff4b5640 1 --2- 172.21.15.98:0/1307015041 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f20f815c550 0x7f20f8147ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:02.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.849+0000 7f20ff4b5640 1 --2- 172.21.15.98:0/1307015041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20f80cf900 0x7f20f8158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:02.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.849+0000 7f20ff4b5640 1 --2- 172.21.15.98:0/1307015041 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f20f806c3a0 0x7f20f814bfc0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:02.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.849+0000 7f20ff4b5640 1 -- 172.21.15.98:0/1307015041 >> 172.21.15.98:0/1307015041 conn(0x7f20f80b2010 msgr2=0x7f20f8067a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:02.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.849+0000 7f20ff4b5640 1 -- 172.21.15.98:0/1307015041 shutdown_connections 2024-09-19T08:43:02.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:02.849+0000 7f20ff4b5640 1 -- 172.21.15.98:0/1307015041 wait complete. 2024-09-19T08:43:02.880 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:43:02.960 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:43:02.956+0000 7ff7406a8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:43:02.960 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:43:02.956+0000 7ff7406a8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:43:02.961 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:43:02.956+0000 7ff7406a8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:43:02.961 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:43:02.956+0000 7ff7406a8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:43:02.961 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:43:02.956+0000 7ff7406a8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:43:02.961 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:43:02.956+0000 7ff73cea1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:43:02.961 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:43:02.956+0000 7ff73cea1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:43:02.961 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:43:02.956+0000 7ff73cea1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:43:02.961 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:43:02.956+0000 7ff73cea1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:43:02.961 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:43:02.956+0000 7ff73cea1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:43:03.008 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:43:03.009+0000 7f67f6cae180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T08:43:03.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.013+0000 7fd10085b640 1 Processor -- start 2024-09-19T08:43:03.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.013+0000 7fd10085b640 1 -- start start 2024-09-19T08:43:03.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.013+0000 7fd10085b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd0fc0b28e0 0x7fd0fc073c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:03.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.013+0000 7fd10085b640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0fc075b80 con 0x7fd0fc15e750 2024-09-19T08:43:03.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.013+0000 7fd10085b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0fc0d1bd0 con 0x7fd0fc075380 2024-09-19T08:43:03.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.013+0000 7fd10085b640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fd0fc0d1eb0 con 0x7fd0fc0b28e0 2024-09-19T08:43:03.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.013+0000 7fd0f9d9b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd0fc0b28e0 0x7fd0fc073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:03.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.013+0000 7fd0f9d9b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd0fc0b28e0 0x7fd0fc073c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53388/0 (socket says 172.21.15.98:53388) 2024-09-19T08:43:03.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.013+0000 7fd0f9d9b640 1 -- 172.21.15.98:0/999373623 learned_addr learned my addr 172.21.15.98:0/999373623 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:03.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.013+0000 7fd0f9d9b640 1 -- 172.21.15.98:0/999373623 >> v1:172.21.15.139:6789/0 conn(0x7fd0fc075380 legacy=0x7fd0fc0cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:03.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.013+0000 7fd0f9d9b640 1 -- 172.21.15.98:0/999373623 >> v1:172.21.15.98:6789/0 conn(0x7fd0fc15e750 legacy=0x7fd0fc074c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:03.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.013+0000 7fd0f9d9b640 1 -- 172.21.15.98:0/999373623 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0fc0cf000 con 0x7fd0fc0b28e0 2024-09-19T08:43:03.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.017+0000 7fd0f9d9b640 1 --2- 172.21.15.98:0/999373623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0fc0b28e0 0x7fd0fc073c90 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec009870 tx=0x7fd0ec01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29263756735df003 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:03.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.017+0000 7fd0f959a640 1 -- 172.21.15.98:0/999373623 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec0aa610 con 0x7fd0fc0b28e0 2024-09-19T08:43:03.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.017+0000 7fd0f959a640 1 -- 172.21.15.98:0/999373623 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0ec0af070 con 0x7fd0fc0b28e0 2024-09-19T08:43:03.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.017+0000 7fd0f959a640 1 -- 172.21.15.98:0/999373623 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec0aa9d0 con 0x7fd0fc0b28e0 2024-09-19T08:43:03.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.017+0000 7fd10085b640 1 -- 172.21.15.98:0/999373623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0fc0b28e0 msgr2=0x7fd0fc073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:03.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.017+0000 7fd10085b640 1 --2- 172.21.15.98:0/999373623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0fc0b28e0 0x7fd0fc073c90 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec009870 tx=0x7fd0ec01c490 comp rx=0 tx=0).stop 2024-09-19T08:43:03.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.017+0000 7fd10085b640 1 -- 172.21.15.98:0/999373623 shutdown_connections 2024-09-19T08:43:03.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.017+0000 7fd10085b640 1 --2- 172.21.15.98:0/999373623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0fc0b28e0 0x7fd0fc073c90 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:03.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.017+0000 7fd10085b640 1 -- 172.21.15.98:0/999373623 >> 172.21.15.98:0/999373623 conn(0x7fd0fc073170 msgr2=0x7fd0fc073580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:03.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.017+0000 7fd10085b640 1 -- 172.21.15.98:0/999373623 shutdown_connections 2024-09-19T08:43:03.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.017+0000 7fd10085b640 1 -- 172.21.15.98:0/999373623 wait complete. 2024-09-19T08:43:03.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.017+0000 7fd10085b640 1 Processor -- start 2024-09-19T08:43:03.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.017+0000 7fd10085b640 1 -- start start 2024-09-19T08:43:03.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.017+0000 7fd10085b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0fc075380 0x7fd0fc0cb270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:03.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.017+0000 7fd10085b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0fc0b28e0 0x7fd0fc09aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:03.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.017+0000 7fd10085b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0fc15e750 0x7fd0fc15b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:03.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.017+0000 7fd10085b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc0d0810 con 0x7fd0fc15e750 2024-09-19T08:43:03.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.017+0000 7fd10085b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd0fc0d0690 con 0x7fd0fc0b28e0 2024-09-19T08:43:03.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.017+0000 7fd10085b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd0fc16c180 con 0x7fd0fc075380 2024-09-19T08:43:03.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.017+0000 7fd0fad9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0fc15e750 0x7fd0fc15b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:03.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.017+0000 7fd0fad9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0fc15e750 0x7fd0fc15b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53400/0 (socket says 172.21.15.98:53400) 2024-09-19T08:43:03.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.017+0000 7fd0fad9d640 1 -- 172.21.15.98:0/4056429165 learned_addr learned my addr 172.21.15.98:0/4056429165 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:03.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.021+0000 7fd0f9d9b640 1 --2- 172.21.15.98:0/4056429165 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0fc0b28e0 0x7fd0fc09aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:03.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.021+0000 7fd0fad9d640 1 -- 172.21.15.98:0/4056429165 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0fc075380 msgr2=0x7fd0fc0cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:03.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.021+0000 7fd0fa59c640 1 --2- 172.21.15.98:0/4056429165 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0fc075380 0x7fd0fc0cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:03.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.021+0000 7fd0fad9d640 1 --2- 172.21.15.98:0/4056429165 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0fc075380 0x7fd0fc0cb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:03.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.021+0000 7fd0fad9d640 1 -- 172.21.15.98:0/4056429165 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0fc0b28e0 msgr2=0x7fd0fc09aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:03.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.021+0000 7fd0fad9d640 1 --2- 172.21.15.98:0/4056429165 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0fc0b28e0 0x7fd0fc09aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:03.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.021+0000 7fd0fad9d640 1 -- 172.21.15.98:0/4056429165 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0fc15a1b0 con 0x7fd0fc15e750 2024-09-19T08:43:03.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.021+0000 7fd0fa59c640 1 --2- 172.21.15.98:0/4056429165 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0fc075380 0x7fd0fc0cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:03.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.021+0000 7fd0fad9d640 1 --2- 172.21.15.98:0/4056429165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0fc15e750 0x7fd0fc15b250 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fd0f400c7b0 tx=0x7fd0f400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:03.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.021+0000 7fd0f2ffd640 1 -- 172.21.15.98:0/4056429165 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4004b70 con 0x7fd0fc15e750 2024-09-19T08:43:03.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.021+0000 7fd0f2ffd640 1 -- 172.21.15.98:0/4056429165 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0f4004d10 con 0x7fd0fc15e750 2024-09-19T08:43:03.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.021+0000 7fd0f2ffd640 1 -- 172.21.15.98:0/4056429165 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f400aa90 con 0x7fd0fc15e750 2024-09-19T08:43:03.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.021+0000 7fd10085b640 1 -- 172.21.15.98:0/4056429165 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0fc15a490 con 0x7fd0fc15e750 2024-09-19T08:43:03.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.021+0000 7fd10085b640 1 -- 172.21.15.98:0/4056429165 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0fc15aa50 con 0x7fd0fc15e750 2024-09-19T08:43:03.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.021+0000 7fd0f2ffd640 1 -- 172.21.15.98:0/4056429165 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 13) ==== 43764+0+0 (secure 0 0 0) 0x7fd0f400ac30 con 0x7fd0fc15e750 2024-09-19T08:43:03.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.021+0000 7fd10085b640 1 -- 172.21.15.98:0/4056429165 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0fc073c90 con 0x7fd0fc15e750 2024-09-19T08:43:03.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.021+0000 7fd0f2ffd640 1 -- 172.21.15.98:0/4056429165 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6384+0+0 (secure 0 0 0) 0x7fd0f4048fe0 con 0x7fd0fc15e750 2024-09-19T08:43:03.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.025+0000 7fd0f2ffd640 1 -- 172.21.15.98:0/4056429165 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd0f4020070 con 0x7fd0fc15e750 2024-09-19T08:43:03.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.213+0000 7fd10085b640 1 -- 172.21.15.98:0/4056429165 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0fc068e10 con 0x7fd0fc15e750 2024-09-19T08:43:03.216 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.217+0000 7fd0f2ffd640 1 -- 172.21.15.98:0/4056429165 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5228 (secure 0 0 0) 0x7fd0f401a770 con 0x7fd0fc15e750 2024-09-19T08:43:03.216 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:03.216 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":39,"btime":"2024-09-19T08:42:45:442311+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5548,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3092385040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3092385040},{"type":"v1","addr":"172.21.15.139:6833","nonce":3092385040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35},{"gid":5551,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3517671290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3517671290},{"type":"v1","addr":"172.21.15.139:6835","nonce":3517671290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35},{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6836/965821274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":965821274},{"type":"v1","addr":"172.21.15.98:6836","nonce":965821274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:42:29.720825+0000","modified":"2024-09-19T08:42:31.743275+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6171":{"gid":6171,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/1767881083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6835","nonce":1767881083},{"type":"v1","addr":"172.21.15.98:6837","nonce":1767881083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6171,"qdb_cluster":[6171]},"id":5}]} 2024-09-19T08:43:03.217 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 39 2024-09-19T08:43:03.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.217+0000 7fd10085b640 1 -- 172.21.15.98:0/4056429165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0fc15e750 msgr2=0x7fd0fc15b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:03.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.217+0000 7fd10085b640 1 --2- 172.21.15.98:0/4056429165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0fc15e750 0x7fd0fc15b250 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fd0f400c7b0 tx=0x7fd0f400cc70 comp rx=0 tx=0).stop 2024-09-19T08:43:03.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.217+0000 7fd10085b640 1 -- 172.21.15.98:0/4056429165 shutdown_connections 2024-09-19T08:43:03.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.217+0000 7fd10085b640 1 --2- 172.21.15.98:0/4056429165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0fc15e750 0x7fd0fc15b250 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:03.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.217+0000 7fd10085b640 1 --2- 172.21.15.98:0/4056429165 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0fc0b28e0 0x7fd0fc09aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:03.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.217+0000 7fd10085b640 1 --2- 172.21.15.98:0/4056429165 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0fc075380 0x7fd0fc0cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:03.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.217+0000 7fd10085b640 1 -- 172.21.15.98:0/4056429165 >> 172.21.15.98:0/4056429165 conn(0x7fd0fc073170 msgr2=0x7fd0fc0b5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:03.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.217+0000 7fd10085b640 1 -- 172.21.15.98:0/4056429165 shutdown_connections 2024-09-19T08:43:03.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.217+0000 7fd10085b640 1 -- 172.21.15.98:0/4056429165 wait complete. 2024-09-19T08:43:03.234 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-19T08:43:03.367 INFO:teuthology.orchestra.run.smithi098.stdout:[{"id":5728,"entity":{"name":{"type":"client","num":5728},"addr":{"type":"any","addr":"172.21.15.139:0","nonce":203050147}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":2,"uptime":0.39199499199999999,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.9982078855883927,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5728 172.21.15.139:0/203050147","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"x","hostname":"smithi139","pid":"28889","root":"/"}},{"id":5725,"entity":{"name":{"type":"client","num":5725},"addr":{"type":"any","addr":"172.21.15.139:0","nonce":2138523125}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":2,"uptime":0.39199499199999999,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.9982078855883927,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5725 172.21.15.139:0/2138523125","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"2f599a46ad10e37c929e5c43da2f6907ddcd1961","ceph_version":"ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)","entity_id":"x","hostname":"smithi139","pid":"28889","root":"/"}}] 2024-09-19T08:43:03.368 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi139", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28889", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.139:0", "nonce": 203050147, "type": "any" }, "name": { "num": 5728, "type": "client" } }, "id": 5728, "inst": "client.5728 172.21.15.139:0/203050147", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.9982078855883927 }, "state": "open", "uptime": 0.391994992 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi139", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28889", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.139:0", "nonce": 2138523125, "type": "any" }, "name": { "num": 5725, "type": "client" } }, "id": 5725, "inst": "client.5725 172.21.15.139:0/2138523125", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.9982078855883927 }, "state": "open", "uptime": 0.391994992 } ] 2024-09-19T08:43:03.368 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-19T08:43:03.368 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T08:43:03.368 DEBUG:teuthology.orchestra.run.smithi098:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:43:03.382 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:43:03.381+0000 7f67f6cae180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T08:43:03.449 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:43:03.512 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:43:03.509+0000 7f67f6cae180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T08:43:03.523 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T08:43:03.523 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:43:03.523 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:43:03.523 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:43:03.529 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15846 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T08:43:03.529 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:43:03.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:43:03.529 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T08:43:03.576 INFO:teuthology.orchestra.run.smithi098.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-19T08:43:03.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:43:03.578 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:43:03.578 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:43:03.578 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:43:03.584 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T08:43:03.584 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:43:03.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:43:03.584 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-19T08:43:03.629 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-09-19T08:43:03.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:43:03.630 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:43:03.630 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:43:03.631 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:43:03.636 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15846 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T08:43:03.636 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:43:03.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:43:03.636 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-19T08:43:03.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:43:03.685 INFO:teuthology.orchestra.run.smithi098.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-19T08:43:03.685 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:43:03.685 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:43:03.685 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:43:03.691 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T08:43:03.691 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:43:03.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:43:03.691 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-19T08:43:03.737 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-19T08:43:03.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:43:03.738 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:43:03.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.849+0000 7f70a3c9e640 1 Processor -- start 2024-09-19T08:43:03.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.849+0000 7f70a3c9e640 1 -- start start 2024-09-19T08:43:03.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.849+0000 7f70a3c9e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f709c0b7650 0x7f709c0d1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:03.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.849+0000 7f70a3c9e640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f709c0740b0 con 0x7f709c0d0ff0 2024-09-19T08:43:03.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.849+0000 7f70a3c9e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f709c073b00 con 0x7f709c15c220 2024-09-19T08:43:03.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.849+0000 7f70a3c9e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f709c0d19a0 con 0x7f709c0b7650 2024-09-19T08:43:03.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.849+0000 7f70a1239640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f709c0d0ff0 0x7f709c0b6f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:37494/0 (socket says 172.21.15.98:37494) 2024-09-19T08:43:03.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.849+0000 7f70a1239640 1 -- 172.21.15.98:0/1559854390 learned_addr learned my addr 172.21.15.98:0/1559854390 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:03.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.849+0000 7f70a223b640 1 --2- 172.21.15.98:0/1559854390 >> v2:172.21.15.139:3300/0 conn(0x7f709c0b7650 0x7f709c0d1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:03.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.849+0000 7f70a223b640 1 -- 172.21.15.98:0/1559854390 >> v1:172.21.15.139:6789/0 conn(0x7f709c15c220 legacy=0x7f709c0733f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:03.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.849+0000 7f70a223b640 1 -- 172.21.15.98:0/1559854390 >> v1:172.21.15.98:6789/0 conn(0x7f709c0d0ff0 legacy=0x7f709c0b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:03.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.849+0000 7f70a223b640 1 -- 172.21.15.98:0/1559854390 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f709c073dd0 con 0x7f709c0b7650 2024-09-19T08:43:03.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.853+0000 7f70a223b640 1 --2- 172.21.15.98:0/1559854390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f709c0b7650 0x7f709c0d1460 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7094009d90 tx=0x7f709401c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8612b5b2d465322a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:03.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.857+0000 7f70a0a38640 1 -- 172.21.15.98:0/1559854390 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70940b2070 con 0x7f709c0b7650 2024-09-19T08:43:03.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.857+0000 7f70a0a38640 1 -- 172.21.15.98:0/1559854390 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70940a5a00 con 0x7f709c0b7650 2024-09-19T08:43:03.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.857+0000 7f70a0a38640 1 -- 172.21.15.98:0/1559854390 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70940a5dd0 con 0x7f709c0b7650 2024-09-19T08:43:03.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.861+0000 7f70a3c9e640 1 -- 172.21.15.98:0/1559854390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f709c0b7650 msgr2=0x7f709c0d1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:03.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.861+0000 7f70a3c9e640 1 --2- 172.21.15.98:0/1559854390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f709c0b7650 0x7f709c0d1460 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7094009d90 tx=0x7f709401c940 comp rx=0 tx=0).stop 2024-09-19T08:43:03.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.861+0000 7f70a3c9e640 1 -- 172.21.15.98:0/1559854390 shutdown_connections 2024-09-19T08:43:03.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.861+0000 7f70a3c9e640 1 --2- 172.21.15.98:0/1559854390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f709c0b7650 0x7f709c0d1460 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:03.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.861+0000 7f70a3c9e640 1 -- 172.21.15.98:0/1559854390 >> 172.21.15.98:0/1559854390 conn(0x7f709c0b40a0 msgr2=0x7f709c0b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:03.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.861+0000 7f70a3c9e640 1 -- 172.21.15.98:0/1559854390 shutdown_connections 2024-09-19T08:43:03.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.861+0000 7f70a3c9e640 1 -- 172.21.15.98:0/1559854390 wait complete. 2024-09-19T08:43:03.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.861+0000 7f70a3c9e640 1 Processor -- start 2024-09-19T08:43:03.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.861+0000 7f70a3c9e640 1 -- start start 2024-09-19T08:43:03.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.861+0000 7f70a3c9e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f709c0b7650 0x7f709c15e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:03.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.861+0000 7f70a3c9e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f709c0d0ff0 0x7f709c150a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:03.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.861+0000 7f70a3c9e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f709c15c220 0x7f709c150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:03.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.861+0000 7f70a3c9e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f709c0d0db0 con 0x7f709c0b7650 2024-09-19T08:43:03.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.861+0000 7f70a3c9e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f709c0d0c30 con 0x7f709c15c220 2024-09-19T08:43:03.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.861+0000 7f70a3c9e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f709c15ce10 con 0x7f709c0d0ff0 2024-09-19T08:43:03.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.861+0000 7f70a1a3a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f709c0b7650 0x7f709c15e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:03.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.861+0000 7f70a1a3a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f709c0b7650 0x7f709c15e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53408/0 (socket says 172.21.15.98:53408) 2024-09-19T08:43:03.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.861+0000 7f70a1a3a640 1 -- 172.21.15.98:0/761954067 learned_addr learned my addr 172.21.15.98:0/761954067 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:03.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.861+0000 7f70a1239640 1 --2- 172.21.15.98:0/761954067 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f709c0d0ff0 0x7f709c150a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:03.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.861+0000 7f70a1a3a640 1 -- 172.21.15.98:0/761954067 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f709c0d0ff0 msgr2=0x7f709c150a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:03.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.861+0000 7f70a1a3a640 1 --2- 172.21.15.98:0/761954067 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f709c0d0ff0 0x7f709c150a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:03.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.861+0000 7f70a223b640 1 --2- 172.21.15.98:0/761954067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f709c15c220 0x7f709c150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:03.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.861+0000 7f70a1a3a640 1 -- 172.21.15.98:0/761954067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f709c15c220 msgr2=0x7f709c150f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:03.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.861+0000 7f70a1a3a640 1 --2- 172.21.15.98:0/761954067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f709c15c220 0x7f709c150f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:03.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.861+0000 7f70a1a3a640 1 -- 172.21.15.98:0/761954067 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f709c0cf670 con 0x7f709c0b7650 2024-09-19T08:43:03.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.861+0000 7f70a1a3a640 1 --2- 172.21.15.98:0/761954067 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f709c0b7650 0x7f709c15e670 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f709800c370 tx=0x7f709800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:03.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.861+0000 7f70927fc640 1 -- 172.21.15.98:0/761954067 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70980052d0 con 0x7f709c0b7650 2024-09-19T08:43:03.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.861+0000 7f70927fc640 1 -- 172.21.15.98:0/761954067 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7098005470 con 0x7f709c0b7650 2024-09-19T08:43:03.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.861+0000 7f70a3c9e640 1 -- 172.21.15.98:0/761954067 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f709c14e970 con 0x7f709c0b7650 2024-09-19T08:43:03.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.861+0000 7f70927fc640 1 -- 172.21.15.98:0/761954067 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7098007260 con 0x7f709c0b7650 2024-09-19T08:43:03.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.861+0000 7f70a3c9e640 1 -- 172.21.15.98:0/761954067 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f709c14f210 con 0x7f709c0b7650 2024-09-19T08:43:03.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.865+0000 7f70927fc640 1 -- 172.21.15.98:0/761954067 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 14) ==== 44099+0+0 (secure 0 0 0) 0x7f7098007400 con 0x7f709c0b7650 2024-09-19T08:43:03.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.865+0000 7f70a3c9e640 1 -- 172.21.15.98:0/761954067 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f709c14eb40 con 0x7f709c0b7650 2024-09-19T08:43:03.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.865+0000 7f70927fc640 1 --2- 172.21.15.98:0/761954067 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7080035f50 0x7f7080038410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:03.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.865+0000 7f70927fc640 1 -- 172.21.15.98:0/761954067 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6384+0+0 (secure 0 0 0) 0x7f7098048790 con 0x7f709c0b7650 2024-09-19T08:43:03.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.865+0000 7f70a1239640 1 --2- 172.21.15.98:0/761954067 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7080035f50 0x7f7080038410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:03.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.865+0000 7f70a1239640 1 --2- 172.21.15.98:0/761954067 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7080035f50 0x7f7080038410 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f709c15f0e0 tx=0x7f708c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:03.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:03.869+0000 7f70927fc640 1 -- 172.21.15.98:0/761954067 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f7098012a60 con 0x7f709c0b7650 2024-09-19T08:43:03.945 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:43:03.945+0000 7f67f6cae180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T08:43:03.997 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:43:03.997+0000 7f67f6cae180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T08:43:04.046 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:43:04.045+0000 7f67f6cae180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T08:43:04.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.053+0000 7f70a3c9e640 1 -- 172.21.15.98:0/761954067 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f709c0d1460 con 0x7f709c0b7650 2024-09-19T08:43:04.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.053+0000 7f70927fc640 1 -- 172.21.15.98:0/761954067 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5228 (secure 0 0 0) 0x7f7098012d20 con 0x7f709c0b7650 2024-09-19T08:43:04.054 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:04.054 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":39,"btime":"2024-09-19T08:42:45:442311+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5548,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3092385040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3092385040},{"type":"v1","addr":"172.21.15.139:6833","nonce":3092385040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35},{"gid":5551,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3517671290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3517671290},{"type":"v1","addr":"172.21.15.139:6835","nonce":3517671290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35},{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6836/965821274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":965821274},{"type":"v1","addr":"172.21.15.98:6836","nonce":965821274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:42:29.720825+0000","modified":"2024-09-19T08:42:31.743275+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6171":{"gid":6171,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/1767881083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6835","nonce":1767881083},{"type":"v1","addr":"172.21.15.98:6837","nonce":1767881083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6171,"qdb_cluster":[6171]},"id":5}]} 2024-09-19T08:43:04.055 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 39 2024-09-19T08:43:04.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.057+0000 7f70a3c9e640 1 -- 172.21.15.98:0/761954067 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7080035f50 msgr2=0x7f7080038410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:04.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.057+0000 7f70a3c9e640 1 --2- 172.21.15.98:0/761954067 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7080035f50 0x7f7080038410 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f709c15f0e0 tx=0x7f708c009040 comp rx=0 tx=0).stop 2024-09-19T08:43:04.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.057+0000 7f70a3c9e640 1 -- 172.21.15.98:0/761954067 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f709c0b7650 msgr2=0x7f709c15e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:04.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.057+0000 7f70a3c9e640 1 --2- 172.21.15.98:0/761954067 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f709c0b7650 0x7f709c15e670 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f709800c370 tx=0x7f709800c830 comp rx=0 tx=0).stop 2024-09-19T08:43:04.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.057+0000 7f70a3c9e640 1 -- 172.21.15.98:0/761954067 shutdown_connections 2024-09-19T08:43:04.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.057+0000 7f70a3c9e640 1 --2- 172.21.15.98:0/761954067 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7080035f50 0x7f7080038410 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.057+0000 7f70a3c9e640 1 --2- 172.21.15.98:0/761954067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f709c15c220 0x7f709c150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.057+0000 7f70a3c9e640 1 --2- 172.21.15.98:0/761954067 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f709c0d0ff0 0x7f709c150a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.057+0000 7f70a3c9e640 1 --2- 172.21.15.98:0/761954067 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f709c0b7650 0x7f709c15e670 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.057+0000 7f70a3c9e640 1 -- 172.21.15.98:0/761954067 >> 172.21.15.98:0/761954067 conn(0x7f709c0b40a0 msgr2=0x7f709c14fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:04.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.057+0000 7f70a3c9e640 1 -- 172.21.15.98:0/761954067 shutdown_connections 2024-09-19T08:43:04.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.057+0000 7f70a3c9e640 1 -- 172.21.15.98:0/761954067 wait complete. 2024-09-19T08:43:04.074 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:43:04.124 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:43:04.125+0000 7f67f6cae180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T08:43:04.175 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:43:04.173+0000 7f67f6cae180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T08:43:04.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.189+0000 7f789e1f0640 1 Processor -- start 2024-09-19T08:43:04.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.189+0000 7f789e1f0640 1 -- start start 2024-09-19T08:43:04.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.189+0000 7f789e1f0640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f78980d01b0 0x7f7898071c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:04.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.189+0000 7f789e1f0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7898163d90 0x7f7898166180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:04.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.189+0000 7f789e1f0640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f789806ff30 con 0x7f789815cc10 2024-09-19T08:43:04.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.189+0000 7f789e1f0640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f789815d3a0 con 0x7f78980d01b0 2024-09-19T08:43:04.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.189+0000 7f789e1f0640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f7898073080 con 0x7f7898163d90 2024-09-19T08:43:04.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.189+0000 7f7896ffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f78980d01b0 0x7f7898071c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:04.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.189+0000 7f7896ffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f78980d01b0 0x7f7898071c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53424/0 (socket says 172.21.15.98:53424) 2024-09-19T08:43:04.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.189+0000 7f78977fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f789815cc10 0x7f7898071520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:37500/0 (socket says 172.21.15.98:37500) 2024-09-19T08:43:04.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.189+0000 7f7896ffd640 1 -- 172.21.15.98:0/882063378 learned_addr learned my addr 172.21.15.98:0/882063378 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:04.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.189+0000 7f7897fff640 1 --2- 172.21.15.98:0/882063378 >> v2:172.21.15.139:3300/0 conn(0x7f7898163d90 0x7f7898166180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:04.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.189+0000 7f7896ffd640 1 -- 172.21.15.98:0/882063378 >> v2:172.21.15.139:3300/0 conn(0x7f7898163d90 msgr2=0x7f7898166180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:04.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.189+0000 7f78967fc640 1 -- 172.21.15.98:0/882063378 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 437455630 0 0) 0x7f789806ff30 con 0x7f789815cc10 2024-09-19T08:43:04.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.189+0000 7f7896ffd640 1 --2- 172.21.15.98:0/882063378 >> v2:172.21.15.139:3300/0 conn(0x7f7898163d90 0x7f7898166180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.189+0000 7f7896ffd640 1 -- 172.21.15.98:0/882063378 >> v1:172.21.15.98:6789/0 conn(0x7f789815cc10 legacy=0x7f7898071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:04.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.189+0000 7f7896ffd640 1 -- 172.21.15.98:0/882063378 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7898073360 con 0x7f78980d01b0 2024-09-19T08:43:04.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.189+0000 7f7897fff640 1 --2- 172.21.15.98:0/882063378 >> v2:172.21.15.139:3300/0 conn(0x7f7898163d90 0x7f7898166180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:04.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.189+0000 7f7896ffd640 1 --2- 172.21.15.98:0/882063378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78980d01b0 0x7f7898071c30 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f789806e4f0 tx=0x7f788c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=463c101f899e163f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:04.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.189+0000 7f78967fc640 1 -- 172.21.15.98:0/882063378 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788c0ab680 con 0x7f78980d01b0 2024-09-19T08:43:04.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.193+0000 7f78967fc640 1 -- 172.21.15.98:0/882063378 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f788c0b0070 con 0x7f78980d01b0 2024-09-19T08:43:04.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.193+0000 7f78967fc640 1 -- 172.21.15.98:0/882063378 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788c0aba40 con 0x7f78980d01b0 2024-09-19T08:43:04.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.193+0000 7f789e1f0640 1 -- 172.21.15.98:0/882063378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78980d01b0 msgr2=0x7f7898071c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:04.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.193+0000 7f789e1f0640 1 --2- 172.21.15.98:0/882063378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78980d01b0 0x7f7898071c30 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f789806e4f0 tx=0x7f788c01c4f0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.193+0000 7f789e1f0640 1 -- 172.21.15.98:0/882063378 shutdown_connections 2024-09-19T08:43:04.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.193+0000 7f789e1f0640 1 --2- 172.21.15.98:0/882063378 >> v2:172.21.15.139:3300/0 conn(0x7f7898163d90 0x7f7898166180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.193+0000 7f789e1f0640 1 --2- 172.21.15.98:0/882063378 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f78980d01b0 0x7f7898071c30 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.193+0000 7f789e1f0640 1 -- 172.21.15.98:0/882063378 >> 172.21.15.98:0/882063378 conn(0x7f78980d0980 msgr2=0x7f78980d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:04.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.193+0000 7f789e1f0640 1 -- 172.21.15.98:0/882063378 shutdown_connections 2024-09-19T08:43:04.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.193+0000 7f789e1f0640 1 -- 172.21.15.98:0/882063378 wait complete. 2024-09-19T08:43:04.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.193+0000 7f789e1f0640 1 Processor -- start 2024-09-19T08:43:04.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.193+0000 7f789e1f0640 1 -- start start 2024-09-19T08:43:04.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.193+0000 7f789e1f0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78980d01b0 0x7f78980b62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:04.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.193+0000 7f789e1f0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f789815cc10 0x7f78980b59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:04.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.193+0000 7f789e1f0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7898163d90 0x7f78981580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:04.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.193+0000 7f789e1f0640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f789815cff0 con 0x7f7898163d90 2024-09-19T08:43:04.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.193+0000 7f789e1f0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f78980ce550 con 0x7f78980d01b0 2024-09-19T08:43:04.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.193+0000 7f789e1f0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7898156940 con 0x7f789815cc10 2024-09-19T08:43:04.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.193+0000 7f7897fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7898163d90 0x7f78981580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:04.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.197+0000 7f7897fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7898163d90 0x7f78981580d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53440/0 (socket says 172.21.15.98:53440) 2024-09-19T08:43:04.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.197+0000 7f7897fff640 1 -- 172.21.15.98:0/2998955451 learned_addr learned my addr 172.21.15.98:0/2998955451 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:04.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.197+0000 7f7896ffd640 1 --2- 172.21.15.98:0/2998955451 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f789815cc10 0x7f78980b59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:04.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.197+0000 7f7897fff640 1 -- 172.21.15.98:0/2998955451 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f789815cc10 msgr2=0x7f78980b59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:04.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.197+0000 7f7897fff640 1 --2- 172.21.15.98:0/2998955451 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f789815cc10 0x7f78980b59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.197+0000 7f7897fff640 1 -- 172.21.15.98:0/2998955451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78980d01b0 msgr2=0x7f78980b62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:04.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.197+0000 7f7897fff640 1 --2- 172.21.15.98:0/2998955451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78980d01b0 0x7f78980b62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.197+0000 7f7897fff640 1 -- 172.21.15.98:0/2998955451 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7898154fa0 con 0x7f7898163d90 2024-09-19T08:43:04.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.197+0000 7f7897fff640 1 --2- 172.21.15.98:0/2998955451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7898163d90 0x7f78981580d0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f789000cd50 tx=0x7f789000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:04.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.197+0000 7f787ffff640 1 -- 172.21.15.98:0/2998955451 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789000f680 con 0x7f7898163d90 2024-09-19T08:43:04.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.197+0000 7f787ffff640 1 -- 172.21.15.98:0/2998955451 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f789000f840 con 0x7f7898163d90 2024-09-19T08:43:04.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.197+0000 7f789e1f0640 1 -- 172.21.15.98:0/2998955451 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78980b45c0 con 0x7f7898163d90 2024-09-19T08:43:04.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.197+0000 7f787ffff640 1 -- 172.21.15.98:0/2998955451 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789000fc30 con 0x7f7898163d90 2024-09-19T08:43:04.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.197+0000 7f789e1f0640 1 -- 172.21.15.98:0/2998955451 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78980b4860 con 0x7f7898163d90 2024-09-19T08:43:04.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.197+0000 7f787ffff640 1 -- 172.21.15.98:0/2998955451 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 14) ==== 44099+0+0 (secure 0 0 0) 0x7f78900213f0 con 0x7f7898163d90 2024-09-19T08:43:04.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.197+0000 7f787ffff640 1 --2- 172.21.15.98:0/2998955451 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7884035f50 0x7f7884038410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:04.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.197+0000 7f789e1f0640 1 -- 172.21.15.98:0/2998955451 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78980b4aa0 con 0x7f7898163d90 2024-09-19T08:43:04.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.197+0000 7f787ffff640 1 -- 172.21.15.98:0/2998955451 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6384+0+0 (secure 0 0 0) 0x7f789004a1f0 con 0x7f7898163d90 2024-09-19T08:43:04.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.197+0000 7f78977fe640 1 --2- 172.21.15.98:0/2998955451 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7884035f50 0x7f7884038410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:04.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.197+0000 7f78977fe640 1 --2- 172.21.15.98:0/2998955451 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7884035f50 0x7f7884038410 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f78880011f0 tx=0x7f7888007930 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:04.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.201+0000 7f787ffff640 1 -- 172.21.15.98:0/2998955451 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f7890014030 con 0x7f7898163d90 2024-09-19T08:43:04.253 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:43:04.253+0000 7f67f6cae180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T08:43:04.304 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:43:04.301+0000 7f67f6cae180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T08:43:04.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.389+0000 7f789e1f0640 1 -- 172.21.15.98:0/2998955451 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7898071520 con 0x7f7898163d90 2024-09-19T08:43:04.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.389+0000 7f787ffff640 1 -- 172.21.15.98:0/2998955451 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5228 (secure 0 0 0) 0x7f789000d370 con 0x7f7898163d90 2024-09-19T08:43:04.389 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:04.389 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":39,"btime":"2024-09-19T08:42:45:442311+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5548,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3092385040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3092385040},{"type":"v1","addr":"172.21.15.139:6833","nonce":3092385040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35},{"gid":5551,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3517671290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3517671290},{"type":"v1","addr":"172.21.15.139:6835","nonce":3517671290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35},{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6836/965821274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":965821274},{"type":"v1","addr":"172.21.15.98:6836","nonce":965821274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:42:29.720825+0000","modified":"2024-09-19T08:42:31.743275+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6171":{"gid":6171,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/1767881083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6835","nonce":1767881083},{"type":"v1","addr":"172.21.15.98:6837","nonce":1767881083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6171,"qdb_cluster":[6171]},"id":5}]} 2024-09-19T08:43:04.390 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 39 2024-09-19T08:43:04.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.389+0000 7f789e1f0640 1 -- 172.21.15.98:0/2998955451 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7884035f50 msgr2=0x7f7884038410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:04.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.389+0000 7f789e1f0640 1 --2- 172.21.15.98:0/2998955451 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7884035f50 0x7f7884038410 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f78880011f0 tx=0x7f7888007930 comp rx=0 tx=0).stop 2024-09-19T08:43:04.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.389+0000 7f789e1f0640 1 -- 172.21.15.98:0/2998955451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7898163d90 msgr2=0x7f78981580d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:04.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.389+0000 7f789e1f0640 1 --2- 172.21.15.98:0/2998955451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7898163d90 0x7f78981580d0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f789000cd50 tx=0x7f789000a650 comp rx=0 tx=0).stop 2024-09-19T08:43:04.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.389+0000 7f789e1f0640 1 -- 172.21.15.98:0/2998955451 shutdown_connections 2024-09-19T08:43:04.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.389+0000 7f789e1f0640 1 --2- 172.21.15.98:0/2998955451 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7884035f50 0x7f7884038410 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.389+0000 7f789e1f0640 1 --2- 172.21.15.98:0/2998955451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7898163d90 0x7f78981580d0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.389+0000 7f789e1f0640 1 --2- 172.21.15.98:0/2998955451 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f789815cc10 0x7f78980b59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.389+0000 7f789e1f0640 1 --2- 172.21.15.98:0/2998955451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f78980d01b0 0x7f78980b62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.389+0000 7f789e1f0640 1 -- 172.21.15.98:0/2998955451 >> 172.21.15.98:0/2998955451 conn(0x7f78980d0980 msgr2=0x7f789815ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:04.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.389+0000 7f789e1f0640 1 -- 172.21.15.98:0/2998955451 shutdown_connections 2024-09-19T08:43:04.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.389+0000 7f789e1f0640 1 -- 172.21.15.98:0/2998955451 wait complete. 2024-09-19T08:43:04.407 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:43:04.447 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:43:04.445+0000 7f67f6cae180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T08:43:04.499 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:43:04.497+0000 7f67f6cae180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T08:43:04.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.517+0000 7f29b479d640 1 Processor -- start 2024-09-19T08:43:04.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.517+0000 7f29b479d640 1 -- start start 2024-09-19T08:43:04.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.517+0000 7f29b479d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29ac0b7f80 0x7f29ac15f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:04.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.517+0000 7f29b479d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f29ac0b9970 0x7f29ac0b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:04.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.517+0000 7f29b479d640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29ac068440 con 0x7f29ac0d4820 2024-09-19T08:43:04.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.517+0000 7f29b479d640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f29ac163050 con 0x7f29ac0b9970 2024-09-19T08:43:04.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.517+0000 7f29b479d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f29ac0ba360 con 0x7f29ac0b7f80 2024-09-19T08:43:04.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.517+0000 7f29b1d38640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f29ac0d4820 0x7f29ac1621b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:37506/0 (socket says 172.21.15.98:37506) 2024-09-19T08:43:04.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.517+0000 7f29b1d38640 1 -- 172.21.15.98:0/1153842612 learned_addr learned my addr 172.21.15.98:0/1153842612 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:04.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.517+0000 7f29b2d3a640 1 --2- 172.21.15.98:0/1153842612 >> v2:172.21.15.98:3300/0 conn(0x7f29ac0b9970 0x7f29ac0b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:04.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.517+0000 7f29b1537640 1 -- 172.21.15.98:0/1153842612 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 676294313 0 0) 0x7f29ac068440 con 0x7f29ac0d4820 2024-09-19T08:43:04.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.517+0000 7f29b2539640 1 --2- 172.21.15.98:0/1153842612 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29ac0b7f80 0x7f29ac15f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:04.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.517+0000 7f29b1537640 1 -- 172.21.15.98:0/1153842612 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2998003530 con 0x7f29ac0d4820 2024-09-19T08:43:04.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.517+0000 7f29b2d3a640 1 -- 172.21.15.98:0/1153842612 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29ac0b7f80 msgr2=0x7f29ac15f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:04.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.517+0000 7f29b2d3a640 1 --2- 172.21.15.98:0/1153842612 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29ac0b7f80 0x7f29ac15f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.517+0000 7f29b2d3a640 1 -- 172.21.15.98:0/1153842612 >> v1:172.21.15.98:6789/0 conn(0x7f29ac0d4820 legacy=0x7f29ac1621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:04.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.517+0000 7f29b2d3a640 1 -- 172.21.15.98:0/1153842612 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29ac162a90 con 0x7f29ac0b9970 2024-09-19T08:43:04.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.521+0000 7f29b2d3a640 1 --2- 172.21.15.98:0/1153842612 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29ac0b9970 0x7f29ac0b9e20 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f29a4009d90 tx=0x7f29a401c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a038857abca6f672 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:04.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.521+0000 7f29b1537640 1 -- 172.21.15.98:0/1153842612 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a8001680 con 0x7f29ac0b9970 2024-09-19T08:43:04.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.521+0000 7f29b1537640 1 -- 172.21.15.98:0/1153842612 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29a40a8460 con 0x7f29ac0b9970 2024-09-19T08:43:04.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.521+0000 7f29b1537640 1 -- 172.21.15.98:0/1153842612 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a40a8830 con 0x7f29ac0b9970 2024-09-19T08:43:04.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.521+0000 7f29b479d640 1 -- 172.21.15.98:0/1153842612 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29ac0b9970 msgr2=0x7f29ac0b9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:04.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.521+0000 7f29b479d640 1 --2- 172.21.15.98:0/1153842612 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29ac0b9970 0x7f29ac0b9e20 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f29a4009d90 tx=0x7f29a401c7b0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.521+0000 7f29b479d640 1 -- 172.21.15.98:0/1153842612 shutdown_connections 2024-09-19T08:43:04.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.521+0000 7f29b479d640 1 --2- 172.21.15.98:0/1153842612 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29ac0b9970 0x7f29ac0b9e20 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.521+0000 7f29b479d640 1 --2- 172.21.15.98:0/1153842612 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29ac0b7f80 0x7f29ac15f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.521+0000 7f29b479d640 1 -- 172.21.15.98:0/1153842612 >> 172.21.15.98:0/1153842612 conn(0x7f29ac074e40 msgr2=0x7f29ac075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:04.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.521+0000 7f29b479d640 1 -- 172.21.15.98:0/1153842612 shutdown_connections 2024-09-19T08:43:04.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.521+0000 7f29b479d640 1 -- 172.21.15.98:0/1153842612 wait complete. 2024-09-19T08:43:04.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.521+0000 7f29b479d640 1 Processor -- start 2024-09-19T08:43:04.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.521+0000 7f29b479d640 1 -- start start 2024-09-19T08:43:04.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.521+0000 7f29b479d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29ac0b7f80 0x7f29ac152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:04.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.521+0000 7f29b479d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29ac0b9970 0x7f29ac1396b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:04.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.521+0000 7f29b479d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29ac0d4820 0x7f29ac139bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:04.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.521+0000 7f29b479d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f29ac0b84e0 con 0x7f29ac0b9970 2024-09-19T08:43:04.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.521+0000 7f29b479d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f29ac0b8360 con 0x7f29ac0b7f80 2024-09-19T08:43:04.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.521+0000 7f29b479d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f29ac144590 con 0x7f29ac0d4820 2024-09-19T08:43:04.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.521+0000 7f29b1d38640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29ac0b9970 0x7f29ac1396b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:04.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.521+0000 7f29b1d38640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29ac0b9970 0x7f29ac1396b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53450/0 (socket says 172.21.15.98:53450) 2024-09-19T08:43:04.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.521+0000 7f29b1d38640 1 -- 172.21.15.98:0/2343780989 learned_addr learned my addr 172.21.15.98:0/2343780989 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:04.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.521+0000 7f29b2539640 1 --2- 172.21.15.98:0/2343780989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29ac0b7f80 0x7f29ac152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:04.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.521+0000 7f29b2d3a640 1 --2- 172.21.15.98:0/2343780989 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29ac0d4820 0x7f29ac139bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:04.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.521+0000 7f29b1d38640 1 -- 172.21.15.98:0/2343780989 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29ac0d4820 msgr2=0x7f29ac139bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:04.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.521+0000 7f29b1d38640 1 --2- 172.21.15.98:0/2343780989 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29ac0d4820 0x7f29ac139bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.521+0000 7f29b1d38640 1 -- 172.21.15.98:0/2343780989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29ac0b7f80 msgr2=0x7f29ac152c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:04.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.521+0000 7f29b1d38640 1 --2- 172.21.15.98:0/2343780989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29ac0b7f80 0x7f29ac152c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.521+0000 7f29b1d38640 1 -- 172.21.15.98:0/2343780989 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29ac13e360 con 0x7f29ac0b9970 2024-09-19T08:43:04.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.521+0000 7f29b2539640 1 --2- 172.21.15.98:0/2343780989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29ac0b7f80 0x7f29ac152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:04.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.521+0000 7f29b2d3a640 1 --2- 172.21.15.98:0/2343780989 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29ac0d4820 0x7f29ac139bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:04.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.521+0000 7f29b1d38640 1 --2- 172.21.15.98:0/2343780989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29ac0b9970 0x7f29ac1396b0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f29a800cac0 tx=0x7f29a800cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:04.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.521+0000 7f299effd640 1 -- 172.21.15.98:0/2343780989 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a8005cf0 con 0x7f29ac0b9970 2024-09-19T08:43:04.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.521+0000 7f299effd640 1 -- 172.21.15.98:0/2343780989 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29a8007070 con 0x7f29ac0b9970 2024-09-19T08:43:04.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.525+0000 7f29b479d640 1 -- 172.21.15.98:0/2343780989 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29ac13d7e0 con 0x7f29ac0b9970 2024-09-19T08:43:04.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.525+0000 7f299effd640 1 -- 172.21.15.98:0/2343780989 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a8007460 con 0x7f29ac0b9970 2024-09-19T08:43:04.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.525+0000 7f29b479d640 1 -- 172.21.15.98:0/2343780989 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29ac13dda0 con 0x7f29ac0b9970 2024-09-19T08:43:04.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.525+0000 7f299effd640 1 -- 172.21.15.98:0/2343780989 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 14) ==== 44099+0+0 (secure 0 0 0) 0x7f29a8007600 con 0x7f29ac0b9970 2024-09-19T08:43:04.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.525+0000 7f299effd640 1 --2- 172.21.15.98:0/2343780989 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2990035f80 0x7f2990038440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:04.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.525+0000 7f299effd640 1 -- 172.21.15.98:0/2343780989 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6384+0+0 (secure 0 0 0) 0x7f29a80492f0 con 0x7f29ac0b9970 2024-09-19T08:43:04.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.525+0000 7f29b479d640 1 -- 172.21.15.98:0/2343780989 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29ac15f560 con 0x7f29ac0b9970 2024-09-19T08:43:04.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.525+0000 7f29b2539640 1 --2- 172.21.15.98:0/2343780989 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2990035f80 0x7f2990038440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:04.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.525+0000 7f29b2539640 1 --2- 172.21.15.98:0/2343780989 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2990035f80 0x7f2990038440 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f29ac1fa4b0 tx=0x7f29a0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:04.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.529+0000 7f299effd640 1 -- 172.21.15.98:0/2343780989 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f29a800a5e0 con 0x7f29ac0b9970 2024-09-19T08:43:04.723 INFO:tasks.ceph.mgr.y.smithi098.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-19T08:43:04.723 INFO:tasks.ceph.mgr.y.smithi098.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-19T08:43:04.723 INFO:tasks.ceph.mgr.y.smithi098.stderr: from numpy import show_config as show_numpy_config 2024-09-19T08:43:04.731 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:43:04.729+0000 7f67f6cae180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T08:43:04.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.733+0000 7f29b479d640 1 -- 172.21.15.98:0/2343780989 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f29ac13ee60 con 0x7f29ac0b9970 2024-09-19T08:43:04.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.733+0000 7f299effd640 1 -- 172.21.15.98:0/2343780989 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+17732 (secure 0 0 0) 0x7f29a80490d0 con 0x7f29ac0b9970 2024-09-19T08:43:04.734 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:04.735 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":71,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:43:02.844105+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:42:27.733851+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-19T08:42:28.717566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.4","mappings":[{"from":6,"to":5}]},{"pgid":"11.10","mappings":[{"from":6,"to":1}]},{"pgid":"11.1e","mappings":[{"from":6,"to":0}]},{"pgid":"11.3a","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.98:0/3957109968":"2024-09-20T08:43:02.843944+0000","172.21.15.98:0/1688754871":"2024-09-20T08:43:02.843944+0000","172.21.15.98:0/1109131925":"2024-09-20T08:43:02.843944+0000","172.21.15.98:0/2131240070":"2024-09-20T08:43:02.843944+0000","172.21.15.98:0/1973723249":"2024-09-20T08:43:02.843944+0000","172.21.15.98:0/1465974372":"2024-09-20T08:43:02.843944+0000","172.21.15.98:0/2085575004":"2024-09-20T08:43:02.843944+0000","172.21.15.98:0/1028111778":"2024-09-20T08:43:02.843944+0000","172.21.15.98:0/3519546751":"2024-09-20T08:43:02.843944+0000","172.21.15.98:6832/2040310606":"2024-09-20T08:43:02.843944+0000","172.21.15.98:0/1268825931":"2024-09-20T08:43:02.843944+0000","172.21.15.98:6833/2040310606":"2024-09-20T08:43:02.843944+0000","172.21.15.98:0/1649589780":"2024-09-20T08:43:02.843944+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:43:04.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.737+0000 7f29b479d640 1 -- 172.21.15.98:0/2343780989 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2990035f80 msgr2=0x7f2990038440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:04.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.737+0000 7f29b479d640 1 --2- 172.21.15.98:0/2343780989 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2990035f80 0x7f2990038440 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f29ac1fa4b0 tx=0x7f29a0008040 comp rx=0 tx=0).stop 2024-09-19T08:43:04.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.737+0000 7f29b479d640 1 -- 172.21.15.98:0/2343780989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29ac0b9970 msgr2=0x7f29ac1396b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:04.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.737+0000 7f29b479d640 1 --2- 172.21.15.98:0/2343780989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29ac0b9970 0x7f29ac1396b0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f29a800cac0 tx=0x7f29a800cf80 comp rx=0 tx=0).stop 2024-09-19T08:43:04.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.737+0000 7f29b479d640 1 -- 172.21.15.98:0/2343780989 shutdown_connections 2024-09-19T08:43:04.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.737+0000 7f29b479d640 1 --2- 172.21.15.98:0/2343780989 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2990035f80 0x7f2990038440 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.737+0000 7f29b479d640 1 --2- 172.21.15.98:0/2343780989 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29ac0d4820 0x7f29ac139bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.737+0000 7f29b479d640 1 --2- 172.21.15.98:0/2343780989 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f29ac0b9970 0x7f29ac1396b0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.737+0000 7f29b479d640 1 --2- 172.21.15.98:0/2343780989 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29ac0b7f80 0x7f29ac152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.737+0000 7f29b479d640 1 -- 172.21.15.98:0/2343780989 >> 172.21.15.98:0/2343780989 conn(0x7f29ac074e40 msgr2=0x7f29ac0cf1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:04.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.737+0000 7f29b479d640 1 -- 172.21.15.98:0/2343780989 shutdown_connections 2024-09-19T08:43:04.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.737+0000 7f29b479d640 1 -- 172.21.15.98:0/2343780989 wait complete. 2024-09-19T08:43:04.752 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T08:43:04.753 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:43:04.870 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:43:04.869+0000 7f67f6cae180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:43:04.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.905+0000 7ffb53889640 1 Processor -- start 2024-09-19T08:43:04.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.905+0000 7ffb53889640 1 -- start start 2024-09-19T08:43:04.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.909+0000 7ffb53889640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ffb4c15e750 0x7ffb4c073c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:04.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.909+0000 7ffb53889640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ffb4c075380 0x7ffb4c0cf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:04.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.909+0000 7ffb53889640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb4c075b80 con 0x7ffb4c0b28e0 2024-09-19T08:43:04.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.909+0000 7ffb53889640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7ffb4c0d1bd0 con 0x7ffb4c075380 2024-09-19T08:43:04.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.909+0000 7ffb53889640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ffb4c0d1eb0 con 0x7ffb4c15e750 2024-09-19T08:43:04.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.909+0000 7ffb51e26640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ffb4c075380 0x7ffb4c0cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:04.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.909+0000 7ffb51e26640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ffb4c075380 0x7ffb4c0cf520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53464/0 (socket says 172.21.15.98:53464) 2024-09-19T08:43:04.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.909+0000 7ffb51e26640 1 -- 172.21.15.98:0/3107064552 learned_addr learned my addr 172.21.15.98:0/3107064552 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:04.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.909+0000 7ffb51625640 1 --2- 172.21.15.98:0/3107064552 >> v2:172.21.15.139:3300/0 conn(0x7ffb4c15e750 0x7ffb4c073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:04.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.909+0000 7ffb51e26640 1 -- 172.21.15.98:0/3107064552 >> v2:172.21.15.139:3300/0 conn(0x7ffb4c15e750 msgr2=0x7ffb4c073c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:04.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.909+0000 7ffb51e26640 1 --2- 172.21.15.98:0/3107064552 >> v2:172.21.15.139:3300/0 conn(0x7ffb4c15e750 0x7ffb4c073c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.909+0000 7ffb51e26640 1 -- 172.21.15.98:0/3107064552 >> v1:172.21.15.139:6789/0 conn(0x7ffb4c0b28e0 legacy=0x7ffb4c074c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:04.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.909+0000 7ffb51e26640 1 -- 172.21.15.98:0/3107064552 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb4c0cf000 con 0x7ffb4c075380 2024-09-19T08:43:04.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.909+0000 7ffb51e26640 1 --2- 172.21.15.98:0/3107064552 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb4c075380 0x7ffb4c0cf520 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c069910 tx=0x7ffb4401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e46d818c1110c00 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:04.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.909+0000 7ffb43fff640 1 -- 172.21.15.98:0/3107064552 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb440a5800 con 0x7ffb4c075380 2024-09-19T08:43:04.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.909+0000 7ffb43fff640 1 -- 172.21.15.98:0/3107064552 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb440a59a0 con 0x7ffb4c075380 2024-09-19T08:43:04.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.909+0000 7ffb43fff640 1 -- 172.21.15.98:0/3107064552 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb440a5d70 con 0x7ffb4c075380 2024-09-19T08:43:04.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.913+0000 7ffb53889640 1 -- 172.21.15.98:0/3107064552 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb4c075380 msgr2=0x7ffb4c0cf520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:04.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.913+0000 7ffb53889640 1 --2- 172.21.15.98:0/3107064552 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb4c075380 0x7ffb4c0cf520 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c069910 tx=0x7ffb4401c920 comp rx=0 tx=0).stop 2024-09-19T08:43:04.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.913+0000 7ffb53889640 1 -- 172.21.15.98:0/3107064552 shutdown_connections 2024-09-19T08:43:04.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.913+0000 7ffb53889640 1 --2- 172.21.15.98:0/3107064552 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb4c075380 0x7ffb4c0cf520 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.913+0000 7ffb53889640 1 --2- 172.21.15.98:0/3107064552 >> v2:172.21.15.139:3300/0 conn(0x7ffb4c15e750 0x7ffb4c073c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.913+0000 7ffb53889640 1 -- 172.21.15.98:0/3107064552 >> 172.21.15.98:0/3107064552 conn(0x7ffb4c073170 msgr2=0x7ffb4c073580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:04.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.913+0000 7ffb53889640 1 -- 172.21.15.98:0/3107064552 shutdown_connections 2024-09-19T08:43:04.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.913+0000 7ffb53889640 1 -- 172.21.15.98:0/3107064552 wait complete. 2024-09-19T08:43:04.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.913+0000 7ffb53889640 1 Processor -- start 2024-09-19T08:43:04.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.913+0000 7ffb53889640 1 -- start start 2024-09-19T08:43:04.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.913+0000 7ffb53889640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb4c075380 0x7ffb4c09aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:04.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.913+0000 7ffb53889640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb4c0b28e0 0x7ffb4c1568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:04.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.913+0000 7ffb53889640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffb4c15e750 0x7ffb4c15b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:04.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.913+0000 7ffb53889640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4c0d0600 con 0x7ffb4c075380 2024-09-19T08:43:04.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.913+0000 7ffb53889640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4c0d0480 con 0x7ffb4c0b28e0 2024-09-19T08:43:04.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.913+0000 7ffb53889640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ffb4c0d0780 con 0x7ffb4c15e750 2024-09-19T08:43:04.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.913+0000 7ffb51625640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb4c075380 0x7ffb4c09aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:04.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.913+0000 7ffb51625640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb4c075380 0x7ffb4c09aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53474/0 (socket says 172.21.15.98:53474) 2024-09-19T08:43:04.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.913+0000 7ffb51625640 1 -- 172.21.15.98:0/216785346 learned_addr learned my addr 172.21.15.98:0/216785346 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:04.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.913+0000 7ffb51625640 1 -- 172.21.15.98:0/216785346 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffb4c15e750 msgr2=0x7ffb4c15b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:04.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.913+0000 7ffb51e26640 1 --2- 172.21.15.98:0/216785346 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffb4c15e750 0x7ffb4c15b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:04.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.913+0000 7ffb50e24640 1 --2- 172.21.15.98:0/216785346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb4c0b28e0 0x7ffb4c1568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:04.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.913+0000 7ffb51625640 1 --2- 172.21.15.98:0/216785346 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffb4c15e750 0x7ffb4c15b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.913+0000 7ffb51625640 1 -- 172.21.15.98:0/216785346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb4c0b28e0 msgr2=0x7ffb4c1568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:04.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.913+0000 7ffb51625640 1 --2- 172.21.15.98:0/216785346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb4c0b28e0 0x7ffb4c1568f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:04.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.913+0000 7ffb51625640 1 -- 172.21.15.98:0/216785346 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb4c1f8f80 con 0x7ffb4c075380 2024-09-19T08:43:04.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.917+0000 7ffb51625640 1 --2- 172.21.15.98:0/216785346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb4c075380 0x7ffb4c09aa80 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ffb4800d620 tx=0x7ffb4800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:04.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.917+0000 7ffb41ffb640 1 -- 172.21.15.98:0/216785346 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4800b620 con 0x7ffb4c075380 2024-09-19T08:43:04.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.917+0000 7ffb41ffb640 1 -- 172.21.15.98:0/216785346 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb48019070 con 0x7ffb4c075380 2024-09-19T08:43:04.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.917+0000 7ffb53889640 1 -- 172.21.15.98:0/216785346 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb4c1f9260 con 0x7ffb4c075380 2024-09-19T08:43:04.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.917+0000 7ffb41ffb640 1 -- 172.21.15.98:0/216785346 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4800ba20 con 0x7ffb4c075380 2024-09-19T08:43:04.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.917+0000 7ffb53889640 1 -- 172.21.15.98:0/216785346 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb4c1f9820 con 0x7ffb4c075380 2024-09-19T08:43:04.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.917+0000 7ffb41ffb640 1 -- 172.21.15.98:0/216785346 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 14) ==== 44099+0+0 (secure 0 0 0) 0x7ffb4800bbc0 con 0x7ffb4c075380 2024-09-19T08:43:04.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.917+0000 7ffb53889640 1 -- 172.21.15.98:0/216785346 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb4c0cf520 con 0x7ffb4c075380 2024-09-19T08:43:04.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.917+0000 7ffb41ffb640 1 --2- 172.21.15.98:0/216785346 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ffb30035f50 0x7ffb30038410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:04.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.917+0000 7ffb41ffb640 1 -- 172.21.15.98:0/216785346 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6384+0+0 (secure 0 0 0) 0x7ffb480497f0 con 0x7ffb4c075380 2024-09-19T08:43:04.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.917+0000 7ffb50e24640 1 --2- 172.21.15.98:0/216785346 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ffb30035f50 0x7ffb30038410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:04.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.917+0000 7ffb50e24640 1 --2- 172.21.15.98:0/216785346 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ffb30035f50 0x7ffb30038410 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c007fd0 tx=0x7ffb3c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:04.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:04.921+0000 7ffb41ffb640 1 -- 172.21.15.98:0/216785346 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ffb4804bc30 con 0x7ffb4c075380 2024-09-19T08:43:04.986 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:43:04.985+0000 7f67f6cae180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T08:43:05.049 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:43:05.049+0000 7f67f6cae180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T08:43:05.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.109+0000 7ffb53889640 1 -- 172.21.15.98:0/216785346 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb4c0c7370 con 0x7ffb4c075380 2024-09-19T08:43:05.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.113+0000 7ffb41ffb640 1 -- 172.21.15.98:0/216785346 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5228 (secure 0 0 0) 0x7ffb4801d3c0 con 0x7ffb4c075380 2024-09-19T08:43:05.113 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:05.113 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":39,"btime":"2024-09-19T08:42:45:442311+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5548,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3092385040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3092385040},{"type":"v1","addr":"172.21.15.139:6833","nonce":3092385040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35},{"gid":5551,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3517671290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3517671290},{"type":"v1","addr":"172.21.15.139:6835","nonce":3517671290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35},{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6836/965821274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":965821274},{"type":"v1","addr":"172.21.15.98:6836","nonce":965821274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:42:29.720825+0000","modified":"2024-09-19T08:42:31.743275+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6171":{"gid":6171,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/1767881083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6835","nonce":1767881083},{"type":"v1","addr":"172.21.15.98:6837","nonce":1767881083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6171,"qdb_cluster":[6171]},"id":5}]} 2024-09-19T08:43:05.113 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 39 2024-09-19T08:43:05.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.113+0000 7ffb53889640 1 -- 172.21.15.98:0/216785346 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ffb30035f50 msgr2=0x7ffb30038410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:05.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.113+0000 7ffb53889640 1 --2- 172.21.15.98:0/216785346 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ffb30035f50 0x7ffb30038410 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c007fd0 tx=0x7ffb3c009040 comp rx=0 tx=0).stop 2024-09-19T08:43:05.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.113+0000 7ffb53889640 1 -- 172.21.15.98:0/216785346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb4c075380 msgr2=0x7ffb4c09aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:05.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.113+0000 7ffb53889640 1 --2- 172.21.15.98:0/216785346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb4c075380 0x7ffb4c09aa80 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ffb4800d620 tx=0x7ffb4800dae0 comp rx=0 tx=0).stop 2024-09-19T08:43:05.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.113+0000 7ffb53889640 1 -- 172.21.15.98:0/216785346 shutdown_connections 2024-09-19T08:43:05.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.113+0000 7ffb53889640 1 --2- 172.21.15.98:0/216785346 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ffb30035f50 0x7ffb30038410 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:05.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.113+0000 7ffb53889640 1 --2- 172.21.15.98:0/216785346 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffb4c15e750 0x7ffb4c15b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:05.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.113+0000 7ffb53889640 1 --2- 172.21.15.98:0/216785346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb4c0b28e0 0x7ffb4c1568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:05.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.113+0000 7ffb53889640 1 --2- 172.21.15.98:0/216785346 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb4c075380 0x7ffb4c09aa80 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:05.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.113+0000 7ffb53889640 1 -- 172.21.15.98:0/216785346 >> 172.21.15.98:0/216785346 conn(0x7ffb4c073170 msgr2=0x7ffb4c06e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:05.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.113+0000 7ffb53889640 1 -- 172.21.15.98:0/216785346 shutdown_connections 2024-09-19T08:43:05.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.113+0000 7ffb53889640 1 -- 172.21.15.98:0/216785346 wait complete. 2024-09-19T08:43:05.131 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:43:05.187 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:43:05.185+0000 7f67f6cae180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T08:43:05.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.241+0000 7fa851e97640 1 Processor -- start 2024-09-19T08:43:05.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.241+0000 7fa851e97640 1 -- start start 2024-09-19T08:43:05.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.241+0000 7fa851e97640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa84c0d01b0 0x7fa84c071c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:05.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.241+0000 7fa851e97640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa84c06ff30 con 0x7fa84c15cc10 2024-09-19T08:43:05.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.241+0000 7fa851e97640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa84c15d3a0 con 0x7fa84c163d90 2024-09-19T08:43:05.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.241+0000 7fa851e97640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fa84c073080 con 0x7fa84c0d01b0 2024-09-19T08:43:05.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.241+0000 7fa843fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa84c0d01b0 0x7fa84c071c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:05.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.241+0000 7fa84b7fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fa84c15cc10 0x7fa84c071520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:37514/0 (socket says 172.21.15.98:37514) 2024-09-19T08:43:05.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.241+0000 7fa84b7fe640 1 -- 172.21.15.98:0/866059556 learned_addr learned my addr 172.21.15.98:0/866059556 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:05.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.241+0000 7fa843fff640 1 -- 172.21.15.98:0/866059556 >> v1:172.21.15.139:6789/0 conn(0x7fa84c163d90 legacy=0x7fa84c166180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:05.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.241+0000 7fa843fff640 1 -- 172.21.15.98:0/866059556 >> v1:172.21.15.98:6789/0 conn(0x7fa84c15cc10 legacy=0x7fa84c071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:05.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.241+0000 7fa843fff640 1 -- 172.21.15.98:0/866059556 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84c073360 con 0x7fa84c0d01b0 2024-09-19T08:43:05.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.241+0000 7fa84affd640 1 -- 172.21.15.98:0/866059556 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 886434163 0 0) 0x7fa84c06ff30 con 0x7fa84c15cc10 2024-09-19T08:43:05.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.245+0000 7fa843fff640 1 --2- 172.21.15.98:0/866059556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa84c0d01b0 0x7fa84c071c30 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fa8340097c0 tx=0x7fa83401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=961c490edd9230db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:05.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.245+0000 7fa84affd640 1 -- 172.21.15.98:0/866059556 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8340b1070 con 0x7fa84c0d01b0 2024-09-19T08:43:05.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.245+0000 7fa84affd640 1 -- 172.21.15.98:0/866059556 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8340ac620 con 0x7fa84c0d01b0 2024-09-19T08:43:05.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.245+0000 7fa84affd640 1 -- 172.21.15.98:0/866059556 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8340ac9f0 con 0x7fa84c0d01b0 2024-09-19T08:43:05.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.245+0000 7fa851e97640 1 -- 172.21.15.98:0/866059556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa84c0d01b0 msgr2=0x7fa84c071c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:05.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.245+0000 7fa851e97640 1 --2- 172.21.15.98:0/866059556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa84c0d01b0 0x7fa84c071c30 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fa8340097c0 tx=0x7fa83401c430 comp rx=0 tx=0).stop 2024-09-19T08:43:05.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.245+0000 7fa851e97640 1 -- 172.21.15.98:0/866059556 shutdown_connections 2024-09-19T08:43:05.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.245+0000 7fa851e97640 1 --2- 172.21.15.98:0/866059556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa84c0d01b0 0x7fa84c071c30 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:05.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.245+0000 7fa851e97640 1 -- 172.21.15.98:0/866059556 >> 172.21.15.98:0/866059556 conn(0x7fa84c0d0980 msgr2=0x7fa84c0d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:05.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.245+0000 7fa851e97640 1 -- 172.21.15.98:0/866059556 shutdown_connections 2024-09-19T08:43:05.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.245+0000 7fa851e97640 1 -- 172.21.15.98:0/866059556 wait complete. 2024-09-19T08:43:05.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.245+0000 7fa851e97640 1 Processor -- start 2024-09-19T08:43:05.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.245+0000 7fa851e97640 1 -- start start 2024-09-19T08:43:05.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.245+0000 7fa851e97640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa84c0d01b0 0x7fa84c154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:05.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.245+0000 7fa851e97640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa84c15cc10 0x7fa84c0b6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:05.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.245+0000 7fa851e97640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa84c163d90 0x7fa84c0ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:05.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.245+0000 7fa851e97640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c15cff0 con 0x7fa84c0d01b0 2024-09-19T08:43:05.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.245+0000 7fa851e97640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c0ce550 con 0x7fa84c163d90 2024-09-19T08:43:05.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.245+0000 7fa851e97640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa84c0b5d60 con 0x7fa84c15cc10 2024-09-19T08:43:05.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.245+0000 7fa84b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa84c0d01b0 0x7fa84c154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:05.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.245+0000 7fa84b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa84c0d01b0 0x7fa84c154160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53496/0 (socket says 172.21.15.98:53496) 2024-09-19T08:43:05.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.245+0000 7fa84b7fe640 1 -- 172.21.15.98:0/205862306 learned_addr learned my addr 172.21.15.98:0/205862306 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:05.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.245+0000 7fa84b7fe640 1 -- 172.21.15.98:0/205862306 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa84c15cc10 msgr2=0x7fa84c0b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:05.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.245+0000 7fa84b7fe640 1 --2- 172.21.15.98:0/205862306 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa84c15cc10 0x7fa84c0b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:05.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.245+0000 7fa84b7fe640 1 -- 172.21.15.98:0/205862306 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa84c163d90 msgr2=0x7fa84c0ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:05.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.245+0000 7fa84b7fe640 1 --2- 172.21.15.98:0/205862306 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa84c163d90 0x7fa84c0ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:05.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.245+0000 7fa84b7fe640 1 -- 172.21.15.98:0/205862306 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84c0b57a0 con 0x7fa84c0d01b0 2024-09-19T08:43:05.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.245+0000 7fa84b7fe640 1 --2- 172.21.15.98:0/205862306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa84c0d01b0 0x7fa84c154160 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fa83c00c520 tx=0x7fa83c00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:05.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.245+0000 7fa848ff9640 1 -- 172.21.15.98:0/205862306 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c018070 con 0x7fa84c0d01b0 2024-09-19T08:43:05.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.245+0000 7fa848ff9640 1 -- 172.21.15.98:0/205862306 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa83c0074e0 con 0x7fa84c0d01b0 2024-09-19T08:43:05.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.245+0000 7fa848ff9640 1 -- 172.21.15.98:0/205862306 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c004a10 con 0x7fa84c0d01b0 2024-09-19T08:43:05.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.245+0000 7fa851e97640 1 -- 172.21.15.98:0/205862306 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa84c0b5a80 con 0x7fa84c0d01b0 2024-09-19T08:43:05.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.245+0000 7fa851e97640 1 -- 172.21.15.98:0/205862306 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa84c1546a0 con 0x7fa84c0d01b0 2024-09-19T08:43:05.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.249+0000 7fa848ff9640 1 -- 172.21.15.98:0/205862306 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 14) ==== 44099+0+0 (secure 0 0 0) 0x7fa83c004bb0 con 0x7fa84c0d01b0 2024-09-19T08:43:05.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.249+0000 7fa851e97640 1 -- 172.21.15.98:0/205862306 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa84c072200 con 0x7fa84c0d01b0 2024-09-19T08:43:05.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.249+0000 7fa848ff9640 1 --2- 172.21.15.98:0/205862306 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa82c035f50 0x7fa82c038410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:05.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.249+0000 7fa848ff9640 1 -- 172.21.15.98:0/205862306 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6384+0+0 (secure 0 0 0) 0x7fa83c048630 con 0x7fa84c0d01b0 2024-09-19T08:43:05.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.249+0000 7fa843fff640 1 --2- 172.21.15.98:0/205862306 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa82c035f50 0x7fa82c038410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:05.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.249+0000 7fa843fff640 1 --2- 172.21.15.98:0/205862306 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa82c035f50 0x7fa82c038410 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa8340ae000 tx=0x7fa834008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:05.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.253+0000 7fa848ff9640 1 -- 172.21.15.98:0/205862306 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa83c011520 con 0x7fa84c0d01b0 2024-09-19T08:43:05.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.437+0000 7fa851e97640 1 -- 172.21.15.98:0/205862306 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa84c157c90 con 0x7fa84c0d01b0 2024-09-19T08:43:05.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.437+0000 7fa848ff9640 1 -- 172.21.15.98:0/205862306 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5228 (secure 0 0 0) 0x7fa83c01bcf0 con 0x7fa84c0d01b0 2024-09-19T08:43:05.440 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:05.440 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":39,"btime":"2024-09-19T08:42:45:442311+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5548,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3092385040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3092385040},{"type":"v1","addr":"172.21.15.139:6833","nonce":3092385040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35},{"gid":5551,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3517671290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3517671290},{"type":"v1","addr":"172.21.15.139:6835","nonce":3517671290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35},{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6836/965821274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":965821274},{"type":"v1","addr":"172.21.15.98:6836","nonce":965821274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:42:29.720825+0000","modified":"2024-09-19T08:42:31.743275+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6171},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6171":{"gid":6171,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6837/1767881083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6835","nonce":1767881083},{"type":"v1","addr":"172.21.15.98:6837","nonce":1767881083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6171,"qdb_cluster":[6171]},"id":5}]} 2024-09-19T08:43:05.440 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 39 2024-09-19T08:43:05.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.441+0000 7fa851e97640 1 -- 172.21.15.98:0/205862306 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa82c035f50 msgr2=0x7fa82c038410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:05.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.441+0000 7fa851e97640 1 --2- 172.21.15.98:0/205862306 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa82c035f50 0x7fa82c038410 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa8340ae000 tx=0x7fa834008040 comp rx=0 tx=0).stop 2024-09-19T08:43:05.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.441+0000 7fa851e97640 1 -- 172.21.15.98:0/205862306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa84c0d01b0 msgr2=0x7fa84c154160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:05.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.441+0000 7fa851e97640 1 --2- 172.21.15.98:0/205862306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa84c0d01b0 0x7fa84c154160 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fa83c00c520 tx=0x7fa83c00c9e0 comp rx=0 tx=0).stop 2024-09-19T08:43:05.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.441+0000 7fa851e97640 1 -- 172.21.15.98:0/205862306 shutdown_connections 2024-09-19T08:43:05.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.441+0000 7fa851e97640 1 --2- 172.21.15.98:0/205862306 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa82c035f50 0x7fa82c038410 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:05.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.441+0000 7fa851e97640 1 --2- 172.21.15.98:0/205862306 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa84c163d90 0x7fa84c0ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:05.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.441+0000 7fa851e97640 1 --2- 172.21.15.98:0/205862306 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa84c15cc10 0x7fa84c0b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:05.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.441+0000 7fa851e97640 1 --2- 172.21.15.98:0/205862306 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa84c0d01b0 0x7fa84c154160 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:05.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.441+0000 7fa851e97640 1 -- 172.21.15.98:0/205862306 >> 172.21.15.98:0/205862306 conn(0x7fa84c0d0980 msgr2=0x7fa84c15f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:05.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.441+0000 7fa851e97640 1 -- 172.21.15.98:0/205862306 shutdown_connections 2024-09-19T08:43:05.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.441+0000 7fa851e97640 1 -- 172.21.15.98:0/205862306 wait complete. 2024-09-19T08:43:05.458 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:43:05.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.573+0000 7f591fcd4640 1 Processor -- start 2024-09-19T08:43:05.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.573+0000 7f591fcd4640 1 -- start start 2024-09-19T08:43:05.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.573+0000 7f591fcd4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f591815cc10 0x7f5918071520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:05.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.573+0000 7f591fcd4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f59180d01b0 0x7f5918071a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:05.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.573+0000 7f591fcd4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f591815d3a0 con 0x7f5918071fa0 2024-09-19T08:43:05.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.573+0000 7f591fcd4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f5918073080 con 0x7f59180d01b0 2024-09-19T08:43:05.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.573+0000 7f591fcd4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5918073360 con 0x7f591815cc10 2024-09-19T08:43:05.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.573+0000 7f591d26f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f59180d01b0 0x7f5918071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:05.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.573+0000 7f591da70640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f591815cc10 0x7f5918071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:05.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.573+0000 7f591e271640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f5918071fa0 0x7f591806ef20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:38500/0 (socket says 172.21.15.98:38500) 2024-09-19T08:43:05.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.573+0000 7f591e271640 1 -- 172.21.15.98:0/4259380920 learned_addr learned my addr 172.21.15.98:0/4259380920 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:05.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.573+0000 7f591ca6e640 1 -- 172.21.15.98:0/4259380920 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3580930703 0 0) 0x7f591815d3a0 con 0x7f5918071fa0 2024-09-19T08:43:05.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.573+0000 7f591ca6e640 1 -- 172.21.15.98:0/4259380920 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5904003530 con 0x7f5918071fa0 2024-09-19T08:43:05.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.573+0000 7f591da70640 1 -- 172.21.15.98:0/4259380920 >> v2:172.21.15.139:3300/0 conn(0x7f59180d01b0 msgr2=0x7f5918071a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:05.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.573+0000 7f591da70640 1 --2- 172.21.15.98:0/4259380920 >> v2:172.21.15.139:3300/0 conn(0x7f59180d01b0 0x7f5918071a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:05.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.573+0000 7f591da70640 1 -- 172.21.15.98:0/4259380920 >> v1:172.21.15.139:6789/0 conn(0x7f5918071fa0 legacy=0x7f591806ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:05.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.573+0000 7f591da70640 1 -- 172.21.15.98:0/4259380920 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5918073640 con 0x7f591815cc10 2024-09-19T08:43:05.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.573+0000 7f591d26f640 1 --2- 172.21.15.98:0/4259380920 >> v2:172.21.15.139:3300/0 conn(0x7f59180d01b0 0x7f5918071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:43:05.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.577+0000 7f591da70640 1 --2- 172.21.15.98:0/4259380920 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f591815cc10 0x7f5918071520 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f59000097c0 tx=0x7f590001c150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe2a552721d3aac4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:05.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.577+0000 7f591ca6e640 1 -- 172.21.15.98:0/4259380920 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59000b0070 con 0x7f591815cc10 2024-09-19T08:43:05.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.577+0000 7f591ca6e640 1 -- 172.21.15.98:0/4259380920 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59000ab600 con 0x7f591815cc10 2024-09-19T08:43:05.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.577+0000 7f591ca6e640 1 -- 172.21.15.98:0/4259380920 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59000ab9b0 con 0x7f591815cc10 2024-09-19T08:43:05.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.577+0000 7f591fcd4640 1 -- 172.21.15.98:0/4259380920 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f591815cc10 msgr2=0x7f5918071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:05.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.577+0000 7f591fcd4640 1 --2- 172.21.15.98:0/4259380920 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f591815cc10 0x7f5918071520 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f59000097c0 tx=0x7f590001c150 comp rx=0 tx=0).stop 2024-09-19T08:43:05.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.577+0000 7f591fcd4640 1 -- 172.21.15.98:0/4259380920 shutdown_connections 2024-09-19T08:43:05.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.577+0000 7f591fcd4640 1 --2- 172.21.15.98:0/4259380920 >> v2:172.21.15.139:3300/0 conn(0x7f59180d01b0 0x7f5918071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:05.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.577+0000 7f591fcd4640 1 --2- 172.21.15.98:0/4259380920 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f591815cc10 0x7f5918071520 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:05.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.577+0000 7f591fcd4640 1 -- 172.21.15.98:0/4259380920 >> 172.21.15.98:0/4259380920 conn(0x7f59180d0980 msgr2=0x7f59180d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:05.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.577+0000 7f591fcd4640 1 -- 172.21.15.98:0/4259380920 shutdown_connections 2024-09-19T08:43:05.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.577+0000 7f591fcd4640 1 -- 172.21.15.98:0/4259380920 wait complete. 2024-09-19T08:43:05.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.577+0000 7f591fcd4640 1 Processor -- start 2024-09-19T08:43:05.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.577+0000 7f591fcd4640 1 -- start start 2024-09-19T08:43:05.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.581+0000 7f591fcd4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5918071fa0 0x7f5918158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:05.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.581+0000 7f591fcd4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f59180d01b0 0x7f5918153810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:05.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.581+0000 7f591fcd4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f591815cc10 0x7f5918152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:05.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.581+0000 7f591fcd4640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f591815cff0 con 0x7f591815cc10 2024-09-19T08:43:05.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.581+0000 7f591fcd4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5918168dc0 con 0x7f5918071fa0 2024-09-19T08:43:05.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.581+0000 7f591fcd4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5918155130 con 0x7f59180d01b0 2024-09-19T08:43:05.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.581+0000 7f591e271640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f591815cc10 0x7f5918152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:05.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.581+0000 7f591e271640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f591815cc10 0x7f5918152720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53510/0 (socket says 172.21.15.98:53510) 2024-09-19T08:43:05.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.581+0000 7f591e271640 1 -- 172.21.15.98:0/254991761 learned_addr learned my addr 172.21.15.98:0/254991761 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:05.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.581+0000 7f591d26f640 1 --2- 172.21.15.98:0/254991761 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f59180d01b0 0x7f5918153810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:05.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.581+0000 7f591da70640 1 --2- 172.21.15.98:0/254991761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5918071fa0 0x7f5918158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:05.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.581+0000 7f591e271640 1 -- 172.21.15.98:0/254991761 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f59180d01b0 msgr2=0x7f5918153810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:05.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.581+0000 7f591e271640 1 --2- 172.21.15.98:0/254991761 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f59180d01b0 0x7f5918153810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:05.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.581+0000 7f591e271640 1 -- 172.21.15.98:0/254991761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5918071fa0 msgr2=0x7f5918158a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:05.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.581+0000 7f591e271640 1 --2- 172.21.15.98:0/254991761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5918071fa0 0x7f5918158a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:05.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.581+0000 7f591e271640 1 -- 172.21.15.98:0/254991761 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59181531b0 con 0x7f591815cc10 2024-09-19T08:43:05.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.581+0000 7f591e271640 1 --2- 172.21.15.98:0/254991761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f591815cc10 0x7f5918152720 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f591000c3f0 tx=0x7f591000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:05.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.581+0000 7f59167fc640 1 -- 172.21.15.98:0/254991761 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5910017070 con 0x7f591815cc10 2024-09-19T08:43:05.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.581+0000 7f59167fc640 1 -- 172.21.15.98:0/254991761 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5910005cf0 con 0x7f591815cc10 2024-09-19T08:43:05.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.581+0000 7f591fcd4640 1 -- 172.21.15.98:0/254991761 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5918152c60 con 0x7f591815cc10 2024-09-19T08:43:05.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.581+0000 7f59167fc640 1 -- 172.21.15.98:0/254991761 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5910007310 con 0x7f591815cc10 2024-09-19T08:43:05.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.581+0000 7f591fcd4640 1 -- 172.21.15.98:0/254991761 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5918155690 con 0x7f591815cc10 2024-09-19T08:43:05.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.581+0000 7f59167fc640 1 -- 172.21.15.98:0/254991761 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 14) ==== 44099+0+0 (secure 0 0 0) 0x7f591001a070 con 0x7f591815cc10 2024-09-19T08:43:05.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.581+0000 7f591fcd4640 1 -- 172.21.15.98:0/254991761 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f591806ef20 con 0x7f591815cc10 2024-09-19T08:43:05.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.581+0000 7f59167fc640 1 --2- 172.21.15.98:0/254991761 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f58f8035f50 0x7f58f8038410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:05.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.581+0000 7f59167fc640 1 -- 172.21.15.98:0/254991761 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6384+0+0 (secure 0 0 0) 0x7f5910048630 con 0x7f591815cc10 2024-09-19T08:43:05.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.581+0000 7f591da70640 1 --2- 172.21.15.98:0/254991761 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f58f8035f50 0x7f58f8038410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:05.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.581+0000 7f591da70640 1 --2- 172.21.15.98:0/254991761 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f58f8035f50 0x7f58f8038410 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f59000ad000 tx=0x7f59000a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:05.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.585+0000 7f59167fc640 1 -- 172.21.15.98:0/254991761 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f59181531b0 con 0x7f591815cc10 2024-09-19T08:43:05.681 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:43:05.681+0000 7f67f6cae180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T08:43:05.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.749+0000 7f591fcd4640 1 -- 172.21.15.98:0/254991761 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f59180b1b10 con 0x7f591815cc10 2024-09-19T08:43:05.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.749+0000 7f59167fc640 1 -- 172.21.15.98:0/254991761 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+17732 (secure 0 0 0) 0x7f5910007740 con 0x7f591815cc10 2024-09-19T08:43:05.752 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:05.752 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":71,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:43:02.844105+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:42:27.733851+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-19T08:42:28.717566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.4","mappings":[{"from":6,"to":5}]},{"pgid":"11.10","mappings":[{"from":6,"to":1}]},{"pgid":"11.1e","mappings":[{"from":6,"to":0}]},{"pgid":"11.3a","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.98:0/3957109968":"2024-09-20T08:43:02.843944+0000","172.21.15.98:0/1688754871":"2024-09-20T08:43:02.843944+0000","172.21.15.98:0/1109131925":"2024-09-20T08:43:02.843944+0000","172.21.15.98:0/2131240070":"2024-09-20T08:43:02.843944+0000","172.21.15.98:0/1973723249":"2024-09-20T08:43:02.843944+0000","172.21.15.98:0/1465974372":"2024-09-20T08:43:02.843944+0000","172.21.15.98:0/2085575004":"2024-09-20T08:43:02.843944+0000","172.21.15.98:0/1028111778":"2024-09-20T08:43:02.843944+0000","172.21.15.98:0/3519546751":"2024-09-20T08:43:02.843944+0000","172.21.15.98:6832/2040310606":"2024-09-20T08:43:02.843944+0000","172.21.15.98:0/1268825931":"2024-09-20T08:43:02.843944+0000","172.21.15.98:6833/2040310606":"2024-09-20T08:43:02.843944+0000","172.21.15.98:0/1649589780":"2024-09-20T08:43:02.843944+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:43:05.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.753+0000 7f591fcd4640 1 -- 172.21.15.98:0/254991761 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f58f8035f50 msgr2=0x7f58f8038410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:05.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.753+0000 7f591fcd4640 1 --2- 172.21.15.98:0/254991761 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f58f8035f50 0x7f58f8038410 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f59000ad000 tx=0x7f59000a6000 comp rx=0 tx=0).stop 2024-09-19T08:43:05.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.753+0000 7f591fcd4640 1 -- 172.21.15.98:0/254991761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f591815cc10 msgr2=0x7f5918152720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:05.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.753+0000 7f591fcd4640 1 --2- 172.21.15.98:0/254991761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f591815cc10 0x7f5918152720 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f591000c3f0 tx=0x7f591000c8b0 comp rx=0 tx=0).stop 2024-09-19T08:43:05.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.753+0000 7f591fcd4640 1 -- 172.21.15.98:0/254991761 shutdown_connections 2024-09-19T08:43:05.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.753+0000 7f591fcd4640 1 --2- 172.21.15.98:0/254991761 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f58f8035f50 0x7f58f8038410 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:05.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.753+0000 7f591fcd4640 1 --2- 172.21.15.98:0/254991761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f591815cc10 0x7f5918152720 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:05.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.753+0000 7f591fcd4640 1 --2- 172.21.15.98:0/254991761 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f59180d01b0 0x7f5918153810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:05.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.753+0000 7f591fcd4640 1 --2- 172.21.15.98:0/254991761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5918071fa0 0x7f5918158a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:05.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.753+0000 7f591fcd4640 1 -- 172.21.15.98:0/254991761 >> 172.21.15.98:0/254991761 conn(0x7f59180d0980 msgr2=0x7f5918156340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:05.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.753+0000 7f591fcd4640 1 -- 172.21.15.98:0/254991761 shutdown_connections 2024-09-19T08:43:05.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.753+0000 7f591fcd4640 1 -- 172.21.15.98:0/254991761 wait complete. 2024-09-19T08:43:05.756 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:43:05.753+0000 7f67f6cae180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T08:43:05.770 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-19T08:43:05.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.897+0000 7fef97c8b640 1 Processor -- start 2024-09-19T08:43:05.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.897+0000 7fef97c8b640 1 -- start start 2024-09-19T08:43:05.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.897+0000 7fef97c8b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fef9006f380 0x7fef900ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:05.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.897+0000 7fef97c8b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fef900cec60 0x7fef900aee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:05.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.897+0000 7fef97c8b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef900af4a0 con 0x7fef9006efa0 2024-09-19T08:43:05.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.897+0000 7fef97c8b640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fef9015ed00 con 0x7fef900cec60 2024-09-19T08:43:05.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.897+0000 7fef97c8b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fef900af780 con 0x7fef9006f380 2024-09-19T08:43:05.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.897+0000 7fef96228640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fef900cec60 0x7fef900aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:05.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.897+0000 7fef96228640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fef900cec60 0x7fef900aee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53524/0 (socket says 172.21.15.98:53524) 2024-09-19T08:43:05.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.897+0000 7fef96228640 1 -- 172.21.15.98:0/3636579498 learned_addr learned my addr 172.21.15.98:0/3636579498 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:05.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.897+0000 7fef95226640 1 --2- 172.21.15.98:0/3636579498 >> v2:172.21.15.139:3300/0 conn(0x7fef9006f380 0x7fef900ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:05.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.897+0000 7fef96228640 1 -- 172.21.15.98:0/3636579498 >> v2:172.21.15.139:3300/0 conn(0x7fef9006f380 msgr2=0x7fef900ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:05.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.897+0000 7fef96228640 1 --2- 172.21.15.98:0/3636579498 >> v2:172.21.15.139:3300/0 conn(0x7fef9006f380 0x7fef900ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:05.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.897+0000 7fef96228640 1 -- 172.21.15.98:0/3636579498 >> v1:172.21.15.139:6789/0 conn(0x7fef9006efa0 legacy=0x7fef900ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:05.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.897+0000 7fef96228640 1 -- 172.21.15.98:0/3636579498 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef900afa60 con 0x7fef900cec60 2024-09-19T08:43:05.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.901+0000 7fef96228640 1 --2- 172.21.15.98:0/3636579498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef900cec60 0x7fef900aee00 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fef80009e50 tx=0x7fef8001c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ed9d213c180406 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:05.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.901+0000 7fef94a25640 1 -- 172.21.15.98:0/3636579498 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef800a5660 con 0x7fef900cec60 2024-09-19T08:43:05.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.901+0000 7fef94a25640 1 -- 172.21.15.98:0/3636579498 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef800a5800 con 0x7fef900cec60 2024-09-19T08:43:05.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.901+0000 7fef94a25640 1 -- 172.21.15.98:0/3636579498 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef800a5bd0 con 0x7fef900cec60 2024-09-19T08:43:05.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.901+0000 7fef97c8b640 1 -- 172.21.15.98:0/3636579498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef900cec60 msgr2=0x7fef900aee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:05.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.901+0000 7fef97c8b640 1 --2- 172.21.15.98:0/3636579498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef900cec60 0x7fef900aee00 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fef80009e50 tx=0x7fef8001c890 comp rx=0 tx=0).stop 2024-09-19T08:43:05.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.901+0000 7fef97c8b640 1 -- 172.21.15.98:0/3636579498 shutdown_connections 2024-09-19T08:43:05.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.901+0000 7fef97c8b640 1 --2- 172.21.15.98:0/3636579498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef900cec60 0x7fef900aee00 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:05.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.901+0000 7fef97c8b640 1 --2- 172.21.15.98:0/3636579498 >> v2:172.21.15.139:3300/0 conn(0x7fef9006f380 0x7fef900ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:05.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.901+0000 7fef97c8b640 1 -- 172.21.15.98:0/3636579498 >> 172.21.15.98:0/3636579498 conn(0x7fef900d0980 msgr2=0x7fef900d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:05.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.901+0000 7fef97c8b640 1 -- 172.21.15.98:0/3636579498 shutdown_connections 2024-09-19T08:43:05.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.901+0000 7fef97c8b640 1 -- 172.21.15.98:0/3636579498 wait complete. 2024-09-19T08:43:05.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.901+0000 7fef97c8b640 1 Processor -- start 2024-09-19T08:43:05.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.901+0000 7fef97c8b640 1 -- start start 2024-09-19T08:43:05.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.905+0000 7fef97c8b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef9006efa0 0x7fef9014f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:05.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.905+0000 7fef97c8b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef900cec60 0x7fef90153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:05.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.905+0000 7fef97c8b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef90157f60 0x7fef9015fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:05.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.905+0000 7fef97c8b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fef9015c180 con 0x7fef9006efa0 2024-09-19T08:43:05.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.905+0000 7fef97c8b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fef9015c000 con 0x7fef900cec60 2024-09-19T08:43:05.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.905+0000 7fef97c8b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fef9015c300 con 0x7fef90157f60 2024-09-19T08:43:05.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.905+0000 7fef95a27640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef9006efa0 0x7fef9014f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:05.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.905+0000 7fef95a27640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef9006efa0 0x7fef9014f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53534/0 (socket says 172.21.15.98:53534) 2024-09-19T08:43:05.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.905+0000 7fef95a27640 1 -- 172.21.15.98:0/3699662035 learned_addr learned my addr 172.21.15.98:0/3699662035 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:05.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.905+0000 7fef95226640 1 --2- 172.21.15.98:0/3699662035 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef900cec60 0x7fef90153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:05.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.905+0000 7fef95a27640 1 -- 172.21.15.98:0/3699662035 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef90157f60 msgr2=0x7fef9015fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:05.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.905+0000 7fef95a27640 1 --2- 172.21.15.98:0/3699662035 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef90157f60 0x7fef9015fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:05.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.905+0000 7fef95a27640 1 -- 172.21.15.98:0/3699662035 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef900cec60 msgr2=0x7fef90153300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:05.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.905+0000 7fef95a27640 1 --2- 172.21.15.98:0/3699662035 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef900cec60 0x7fef90153300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:05.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.905+0000 7fef95a27640 1 -- 172.21.15.98:0/3699662035 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef901570d0 con 0x7fef9006efa0 2024-09-19T08:43:05.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.905+0000 7fef95a27640 1 --2- 172.21.15.98:0/3699662035 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef9006efa0 0x7fef9014f180 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fef8800c480 tx=0x7fef8800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:05.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.905+0000 7fef8e7fc640 1 -- 172.21.15.98:0/3699662035 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88012830 con 0x7fef9006efa0 2024-09-19T08:43:05.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.905+0000 7fef8e7fc640 1 -- 172.21.15.98:0/3699662035 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef88017070 con 0x7fef9006efa0 2024-09-19T08:43:05.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.905+0000 7fef97c8b640 1 -- 172.21.15.98:0/3699662035 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef90152b40 con 0x7fef9006efa0 2024-09-19T08:43:05.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.905+0000 7fef8e7fc640 1 -- 172.21.15.98:0/3699662035 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88012c30 con 0x7fef9006efa0 2024-09-19T08:43:05.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.905+0000 7fef97c8b640 1 -- 172.21.15.98:0/3699662035 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef90156df0 con 0x7fef9006efa0 2024-09-19T08:43:05.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.905+0000 7fef8e7fc640 1 -- 172.21.15.98:0/3699662035 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 15) ==== 44155+0+0 (secure 0 0 0) 0x7fef8801c460 con 0x7fef9006efa0 2024-09-19T08:43:05.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.905+0000 7fef97c8b640 1 -- 172.21.15.98:0/3699662035 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef900ce550 con 0x7fef9006efa0 2024-09-19T08:43:05.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.905+0000 7fef8e7fc640 1 --2- 172.21.15.98:0/3699662035 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fef7c035fe0 0x7fef7c0384a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:05.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.905+0000 7fef8e7fc640 1 -- 172.21.15.98:0/3699662035 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6384+0+0 (secure 0 0 0) 0x7fef88049130 con 0x7fef9006efa0 2024-09-19T08:43:05.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.905+0000 7fef95226640 1 --2- 172.21.15.98:0/3699662035 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fef7c035fe0 0x7fef7c0384a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:05.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.905+0000 7fef95226640 1 --2- 172.21.15.98:0/3699662035 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fef7c035fe0 0x7fef7c0384a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fef90161ed0 tx=0x7fef84008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:05.911 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:43:05.909+0000 7f67f6cae180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T08:43:05.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:05.909+0000 7fef8e7fc640 1 -- 172.21.15.98:0/3699662035 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fef8801a520 con 0x7fef9006efa0 2024-09-19T08:43:05.986 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:43:05.985+0000 7f67f6cae180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T08:43:06.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.109+0000 7fef97c8b640 1 -- 172.21.15.98:0/3699662035 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fef9006fee0 con 0x7fef9006efa0 2024-09-19T08:43:06.109 INFO:tasks.ceph.mon.a.smithi098.stderr:2024-09-19T08:43:06.109+0000 7f5e49017640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T08:43:06.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.109+0000 7fef8e7fc640 1 -- 172.21.15.98:0/3699662035 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v40) ==== 161+0+0 (secure 0 0 0) 0x7fef8801a780 con 0x7fef9006efa0 2024-09-19T08:43:06.112 INFO:teuthology.orchestra.run.smithi098.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T08:43:06.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.113+0000 7fef97c8b640 1 -- 172.21.15.98:0/3699662035 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fef7c035fe0 msgr2=0x7fef7c0384a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:06.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.113+0000 7fef97c8b640 1 --2- 172.21.15.98:0/3699662035 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fef7c035fe0 0x7fef7c0384a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fef90161ed0 tx=0x7fef84008040 comp rx=0 tx=0).stop 2024-09-19T08:43:06.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.113+0000 7fef97c8b640 1 -- 172.21.15.98:0/3699662035 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef9006efa0 msgr2=0x7fef9014f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:06.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.113+0000 7fef97c8b640 1 --2- 172.21.15.98:0/3699662035 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef9006efa0 0x7fef9014f180 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fef8800c480 tx=0x7fef8800c940 comp rx=0 tx=0).stop 2024-09-19T08:43:06.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.113+0000 7fef97c8b640 1 -- 172.21.15.98:0/3699662035 shutdown_connections 2024-09-19T08:43:06.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.113+0000 7fef97c8b640 1 --2- 172.21.15.98:0/3699662035 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fef7c035fe0 0x7fef7c0384a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:06.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.113+0000 7fef97c8b640 1 --2- 172.21.15.98:0/3699662035 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef90157f60 0x7fef9015fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:06.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.113+0000 7fef97c8b640 1 --2- 172.21.15.98:0/3699662035 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef900cec60 0x7fef90153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:06.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.113+0000 7fef97c8b640 1 --2- 172.21.15.98:0/3699662035 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef9006efa0 0x7fef9014f180 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:06.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.113+0000 7fef97c8b640 1 -- 172.21.15.98:0/3699662035 >> 172.21.15.98:0/3699662035 conn(0x7fef900d0980 msgr2=0x7fef900b1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:06.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.113+0000 7fef97c8b640 1 -- 172.21.15.98:0/3699662035 shutdown_connections 2024-09-19T08:43:06.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.113+0000 7fef97c8b640 1 -- 172.21.15.98:0/3699662035 wait complete. 2024-09-19T08:43:06.127 INFO:tasks.ceph.mds.a.smithi098.stderr: -6727> 2024-09-19T08:42:36.341+0000 7f909d3b3640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T08:43:06.133 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T08:43:06.164 INFO:tasks.ceph.mds.a.smithi098.stdout:starting mds.a at 2024-09-19T08:43:06.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.245+0000 7fbc20db1640 1 Processor -- start 2024-09-19T08:43:06.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.249+0000 7fbc20db1640 1 -- start start 2024-09-19T08:43:06.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.249+0000 7fbc20db1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc1c15e680 0x7fbc1c0d0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:06.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.249+0000 7fbc20db1640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbc1c06fa90 0x7fbc1c06c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:06.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.249+0000 7fbc20db1640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc1c0d12a0 con 0x7fbc1c06bc90 2024-09-19T08:43:06.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.249+0000 7fbc20db1640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fbc1c15ea60 con 0x7fbc1c06fa90 2024-09-19T08:43:06.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.249+0000 7fbc20db1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbc1c0d08d0 con 0x7fbc1c15e680 2024-09-19T08:43:06.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.249+0000 7fbc1ad9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbc1c06fa90 0x7fbc1c06c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:06.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.249+0000 7fbc1ad9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbc1c06fa90 0x7fbc1c06c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53552/0 (socket says 172.21.15.98:53552) 2024-09-19T08:43:06.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.249+0000 7fbc1ad9d640 1 -- 172.21.15.98:0/3964233509 learned_addr learned my addr 172.21.15.98:0/3964233509 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:06.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.249+0000 7fbc1a59c640 1 --2- 172.21.15.98:0/3964233509 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc1c15e680 0x7fbc1c0d0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:06.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.249+0000 7fbc1ad9d640 1 -- 172.21.15.98:0/3964233509 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc1c15e680 msgr2=0x7fbc1c0d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:06.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.249+0000 7fbc1ad9d640 1 --2- 172.21.15.98:0/3964233509 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc1c15e680 0x7fbc1c0d0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:06.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.249+0000 7fbc1ad9d640 1 -- 172.21.15.98:0/3964233509 >> v1:172.21.15.139:6789/0 conn(0x7fbc1c06bc90 legacy=0x7fbc1c06f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:06.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.249+0000 7fbc1ad9d640 1 -- 172.21.15.98:0/3964233509 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc1c06c800 con 0x7fbc1c06fa90 2024-09-19T08:43:06.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.249+0000 7fbc1ad9d640 1 --2- 172.21.15.98:0/3964233509 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc1c06fa90 0x7fbc1c06c0d0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fbc14009e50 tx=0x7fbc1401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9044e81cf6e549d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:06.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.249+0000 7fbc19d9b640 1 -- 172.21.15.98:0/3964233509 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc140a5760 con 0x7fbc1c06fa90 2024-09-19T08:43:06.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.249+0000 7fbc19d9b640 1 -- 172.21.15.98:0/3964233509 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc140a5900 con 0x7fbc1c06fa90 2024-09-19T08:43:06.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.249+0000 7fbc19d9b640 1 -- 172.21.15.98:0/3964233509 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc140a5cd0 con 0x7fbc1c06fa90 2024-09-19T08:43:06.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.253+0000 7fbc20db1640 1 -- 172.21.15.98:0/3964233509 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc1c06fa90 msgr2=0x7fbc1c06c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:06.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.253+0000 7fbc20db1640 1 --2- 172.21.15.98:0/3964233509 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc1c06fa90 0x7fbc1c06c0d0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fbc14009e50 tx=0x7fbc1401c9f0 comp rx=0 tx=0).stop 2024-09-19T08:43:06.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.253+0000 7fbc20db1640 1 -- 172.21.15.98:0/3964233509 shutdown_connections 2024-09-19T08:43:06.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.253+0000 7fbc20db1640 1 --2- 172.21.15.98:0/3964233509 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc1c06fa90 0x7fbc1c06c0d0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:06.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.253+0000 7fbc20db1640 1 --2- 172.21.15.98:0/3964233509 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc1c15e680 0x7fbc1c0d0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:06.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.253+0000 7fbc20db1640 1 -- 172.21.15.98:0/3964233509 >> 172.21.15.98:0/3964233509 conn(0x7fbc1c0d23d0 msgr2=0x7fbc1c0d27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:06.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.253+0000 7fbc20db1640 1 -- 172.21.15.98:0/3964233509 shutdown_connections 2024-09-19T08:43:06.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.253+0000 7fbc20db1640 1 -- 172.21.15.98:0/3964233509 wait complete. 2024-09-19T08:43:06.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.253+0000 7fbc20db1640 1 Processor -- start 2024-09-19T08:43:06.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.253+0000 7fbc20db1640 1 -- start start 2024-09-19T08:43:06.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.253+0000 7fbc20db1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc1c06bc90 0x7fbc1c159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:06.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.253+0000 7fbc20db1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc1c06fa90 0x7fbc1c0b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:06.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.253+0000 7fbc20db1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc1c15e680 0x7fbc1c149ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:06.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.253+0000 7fbc20db1640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c0737a0 con 0x7fbc1c15e680 2024-09-19T08:43:06.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.253+0000 7fbc20db1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c070970 con 0x7fbc1c06bc90 2024-09-19T08:43:06.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.253+0000 7fbc20db1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbc1c073920 con 0x7fbc1c06fa90 2024-09-19T08:43:06.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.253+0000 7fbc1ad9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc1c15e680 0x7fbc1c149ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:06.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.253+0000 7fbc1ad9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc1c15e680 0x7fbc1c149ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53556/0 (socket says 172.21.15.98:53556) 2024-09-19T08:43:06.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.253+0000 7fbc1ad9d640 1 -- 172.21.15.98:0/3832274278 learned_addr learned my addr 172.21.15.98:0/3832274278 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:06.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.253+0000 7fbc1ad9d640 1 -- 172.21.15.98:0/3832274278 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc1c06fa90 msgr2=0x7fbc1c0b5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:06.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.253+0000 7fbc1ad9d640 1 --2- 172.21.15.98:0/3832274278 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc1c06fa90 0x7fbc1c0b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:06.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.253+0000 7fbc1ad9d640 1 -- 172.21.15.98:0/3832274278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc1c06bc90 msgr2=0x7fbc1c159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:06.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.253+0000 7fbc1ad9d640 1 --2- 172.21.15.98:0/3832274278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc1c06bc90 0x7fbc1c159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:06.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.253+0000 7fbc1ad9d640 1 -- 172.21.15.98:0/3832274278 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc1c14a350 con 0x7fbc1c15e680 2024-09-19T08:43:06.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.253+0000 7fbc1ad9d640 1 --2- 172.21.15.98:0/3832274278 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc1c15e680 0x7fbc1c149ba0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fbc140af040 tx=0x7fbc14019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:06.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.253+0000 7fbc12ffd640 1 -- 172.21.15.98:0/3832274278 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc140b8670 con 0x7fbc1c15e680 2024-09-19T08:43:06.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.253+0000 7fbc12ffd640 1 -- 172.21.15.98:0/3832274278 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc140be070 con 0x7fbc1c15e680 2024-09-19T08:43:06.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.253+0000 7fbc20db1640 1 -- 172.21.15.98:0/3832274278 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc1c14ea90 con 0x7fbc1c15e680 2024-09-19T08:43:06.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.253+0000 7fbc12ffd640 1 -- 172.21.15.98:0/3832274278 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc140b8a70 con 0x7fbc1c15e680 2024-09-19T08:43:06.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.253+0000 7fbc20db1640 1 -- 172.21.15.98:0/3832274278 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc1c14f050 con 0x7fbc1c15e680 2024-09-19T08:43:06.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.253+0000 7fbc12ffd640 1 -- 172.21.15.98:0/3832274278 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 15) ==== 44155+0+0 (secure 0 0 0) 0x7fbc140b8c10 con 0x7fbc1c15e680 2024-09-19T08:43:06.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.253+0000 7fbc20db1640 1 -- 172.21.15.98:0/3832274278 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc1c06c0d0 con 0x7fbc1c15e680 2024-09-19T08:43:06.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.257+0000 7fbc12ffd640 1 --2- 172.21.15.98:0/3832274278 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbbfc035fe0 0x7fbbfc0384a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:06.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.257+0000 7fbc12ffd640 1 -- 172.21.15.98:0/3832274278 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6470+0+0 (secure 0 0 0) 0x7fbc140e47b0 con 0x7fbc1c15e680 2024-09-19T08:43:06.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.257+0000 7fbc1a59c640 1 --2- 172.21.15.98:0/3832274278 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbbfc035fe0 0x7fbbfc0384a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:06.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.257+0000 7fbc1a59c640 1 --2- 172.21.15.98:0/3832274278 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbbfc035fe0 0x7fbbfc0384a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbc0400b850 tx=0x7fbc04008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:06.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.261+0000 7fbc12ffd640 1 -- 172.21.15.98:0/3832274278 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fbc140165e0 con 0x7fbc1c15e680 2024-09-19T08:43:06.285 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:43:06.285+0000 7f67f6cae180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T08:43:06.340 INFO:tasks.ceph.mgr.y.smithi098.stderr:2024-09-19T08:43:06.341+0000 7f67f6cae180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T08:43:06.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:06.457+0000 7fbc20db1640 1 -- 172.21.15.98:0/3832274278 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fbc1c159cd0 con 0x7fbc1c15e680 2024-09-19T08:43:07.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.113+0000 7fbc12ffd640 1 -- 172.21.15.98:0/3832274278 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v41) ==== 104+0+0 (secure 0 0 0) 0x7fbc140b7590 con 0x7fbc1c15e680 2024-09-19T08:43:07.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.117+0000 7fbc20db1640 1 -- 172.21.15.98:0/3832274278 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbbfc035fe0 msgr2=0x7fbbfc0384a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:07.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.117+0000 7fbc20db1640 1 --2- 172.21.15.98:0/3832274278 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbbfc035fe0 0x7fbbfc0384a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbc0400b850 tx=0x7fbc04008040 comp rx=0 tx=0).stop 2024-09-19T08:43:07.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.117+0000 7fbc20db1640 1 -- 172.21.15.98:0/3832274278 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc1c15e680 msgr2=0x7fbc1c149ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:07.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.117+0000 7fbc20db1640 1 --2- 172.21.15.98:0/3832274278 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc1c15e680 0x7fbc1c149ba0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fbc140af040 tx=0x7fbc14019790 comp rx=0 tx=0).stop 2024-09-19T08:43:07.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.117+0000 7fbc20db1640 1 -- 172.21.15.98:0/3832274278 shutdown_connections 2024-09-19T08:43:07.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.117+0000 7fbc20db1640 1 --2- 172.21.15.98:0/3832274278 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbbfc035fe0 0x7fbbfc0384a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:07.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.117+0000 7fbc20db1640 1 --2- 172.21.15.98:0/3832274278 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc1c15e680 0x7fbc1c149ba0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:07.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.117+0000 7fbc20db1640 1 --2- 172.21.15.98:0/3832274278 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc1c06fa90 0x7fbc1c0b5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:07.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.117+0000 7fbc20db1640 1 --2- 172.21.15.98:0/3832274278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc1c06bc90 0x7fbc1c159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:07.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.117+0000 7fbc20db1640 1 -- 172.21.15.98:0/3832274278 >> 172.21.15.98:0/3832274278 conn(0x7fbc1c0d23d0 msgr2=0x7fbc1c06f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:07.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.117+0000 7fbc20db1640 1 -- 172.21.15.98:0/3832274278 shutdown_connections 2024-09-19T08:43:07.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.117+0000 7fbc20db1640 1 -- 172.21.15.98:0/3832274278 wait complete. 2024-09-19T08:43:07.141 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T08:43:07.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.309+0000 7fbe59920640 1 Processor -- start 2024-09-19T08:43:07.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.309+0000 7fbe59920640 1 -- start start 2024-09-19T08:43:07.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.313+0000 7fbe59920640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbe5415e3c0 0x7fbe54073140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:07.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.313+0000 7fbe59920640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe5406e7e0 0x7fbe540b5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:07.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.313+0000 7fbe59920640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe5406ec40 con 0x7fbe54073680 2024-09-19T08:43:07.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.313+0000 7fbe59920640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fbe54073af0 con 0x7fbe5415e3c0 2024-09-19T08:43:07.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.313+0000 7fbe59920640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbe5406be30 con 0x7fbe5406e7e0 2024-09-19T08:43:07.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.313+0000 7fbe527fc640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fbe54073680 0x7fbe5406e0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:37524/0 (socket says 172.21.15.98:37524) 2024-09-19T08:43:07.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.313+0000 7fbe52ffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbe5415e3c0 0x7fbe54073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:07.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.313+0000 7fbe527fc640 1 -- 172.21.15.98:0/3060682578 learned_addr learned my addr 172.21.15.98:0/3060682578 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:07.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.313+0000 7fbe51ffb640 1 -- 172.21.15.98:0/3060682578 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 785811843 0 0) 0x7fbe5406ec40 con 0x7fbe54073680 2024-09-19T08:43:07.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.313+0000 7fbe51ffb640 1 -- 172.21.15.98:0/3060682578 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe3c003530 con 0x7fbe54073680 2024-09-19T08:43:07.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.313+0000 7fbe537fe640 1 --2- 172.21.15.98:0/3060682578 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe5406e7e0 0x7fbe540b5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:07.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.313+0000 7fbe52ffd640 1 -- 172.21.15.98:0/3060682578 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe5406e7e0 msgr2=0x7fbe540b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:07.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.313+0000 7fbe52ffd640 1 --2- 172.21.15.98:0/3060682578 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe5406e7e0 0x7fbe540b5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:07.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.313+0000 7fbe52ffd640 1 -- 172.21.15.98:0/3060682578 >> v1:172.21.15.98:6789/0 conn(0x7fbe54073680 legacy=0x7fbe5406e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:07.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.313+0000 7fbe52ffd640 1 -- 172.21.15.98:0/3060682578 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe5406c110 con 0x7fbe5415e3c0 2024-09-19T08:43:07.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.313+0000 7fbe52ffd640 1 --2- 172.21.15.98:0/3060682578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe5415e3c0 0x7fbe54073140 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fbe440097c0 tx=0x7fbe4401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ac8922b00fb40f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:07.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.313+0000 7fbe51ffb640 1 -- 172.21.15.98:0/3060682578 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe440b2070 con 0x7fbe5415e3c0 2024-09-19T08:43:07.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.313+0000 7fbe51ffb640 1 -- 172.21.15.98:0/3060682578 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe440ad640 con 0x7fbe5415e3c0 2024-09-19T08:43:07.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.313+0000 7fbe51ffb640 1 -- 172.21.15.98:0/3060682578 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe440ad9f0 con 0x7fbe5415e3c0 2024-09-19T08:43:07.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.317+0000 7fbe59920640 1 -- 172.21.15.98:0/3060682578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe5415e3c0 msgr2=0x7fbe54073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:07.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.317+0000 7fbe59920640 1 --2- 172.21.15.98:0/3060682578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe5415e3c0 0x7fbe54073140 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fbe440097c0 tx=0x7fbe4401c4a0 comp rx=0 tx=0).stop 2024-09-19T08:43:07.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.317+0000 7fbe59920640 1 -- 172.21.15.98:0/3060682578 shutdown_connections 2024-09-19T08:43:07.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.317+0000 7fbe59920640 1 --2- 172.21.15.98:0/3060682578 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe5406e7e0 0x7fbe540b5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:07.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.317+0000 7fbe59920640 1 --2- 172.21.15.98:0/3060682578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe5415e3c0 0x7fbe54073140 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:07.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.317+0000 7fbe59920640 1 -- 172.21.15.98:0/3060682578 >> 172.21.15.98:0/3060682578 conn(0x7fbe540b4180 msgr2=0x7fbe540b4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:07.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.317+0000 7fbe59920640 1 -- 172.21.15.98:0/3060682578 shutdown_connections 2024-09-19T08:43:07.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.317+0000 7fbe59920640 1 -- 172.21.15.98:0/3060682578 wait complete. 2024-09-19T08:43:07.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.317+0000 7fbe59920640 1 Processor -- start 2024-09-19T08:43:07.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.317+0000 7fbe59920640 1 -- start start 2024-09-19T08:43:07.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.317+0000 7fbe59920640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe5406e7e0 0x7fbe5414ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:07.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.317+0000 7fbe59920640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe54073680 0x7fbe5415a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:07.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.317+0000 7fbe59920640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe5415e3c0 0x7fbe540b3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:07.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.317+0000 7fbe59920640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5415e920 con 0x7fbe54073680 2024-09-19T08:43:07.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.317+0000 7fbe59920640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5415e7a0 con 0x7fbe5406e7e0 2024-09-19T08:43:07.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.317+0000 7fbe59920640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbe540b6860 con 0x7fbe5415e3c0 2024-09-19T08:43:07.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.317+0000 7fbe527fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe54073680 0x7fbe5415a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:07.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.317+0000 7fbe527fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe54073680 0x7fbe5415a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53570/0 (socket says 172.21.15.98:53570) 2024-09-19T08:43:07.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.317+0000 7fbe527fc640 1 -- 172.21.15.98:0/2474620754 learned_addr learned my addr 172.21.15.98:0/2474620754 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:07.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.317+0000 7fbe52ffd640 1 --2- 172.21.15.98:0/2474620754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe5406e7e0 0x7fbe5414ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:07.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.317+0000 7fbe537fe640 1 --2- 172.21.15.98:0/2474620754 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe5415e3c0 0x7fbe540b3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:07.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.317+0000 7fbe527fc640 1 -- 172.21.15.98:0/2474620754 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe5415e3c0 msgr2=0x7fbe540b3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:07.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.317+0000 7fbe527fc640 1 --2- 172.21.15.98:0/2474620754 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe5415e3c0 0x7fbe540b3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:07.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.317+0000 7fbe527fc640 1 -- 172.21.15.98:0/2474620754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe5406e7e0 msgr2=0x7fbe5414ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:07.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.317+0000 7fbe527fc640 1 --2- 172.21.15.98:0/2474620754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe5406e7e0 0x7fbe5414ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:07.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.317+0000 7fbe527fc640 1 -- 172.21.15.98:0/2474620754 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe541504b0 con 0x7fbe54073680 2024-09-19T08:43:07.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.317+0000 7fbe537fe640 1 --2- 172.21.15.98:0/2474620754 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe5415e3c0 0x7fbe540b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:07.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.317+0000 7fbe527fc640 1 --2- 172.21.15.98:0/2474620754 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe54073680 0x7fbe5415a9a0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fbe4800c480 tx=0x7fbe4800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:07.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.317+0000 7fbe3bfff640 1 -- 172.21.15.98:0/2474620754 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe48017070 con 0x7fbe54073680 2024-09-19T08:43:07.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.317+0000 7fbe3bfff640 1 -- 172.21.15.98:0/2474620754 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe48005cf0 con 0x7fbe54073680 2024-09-19T08:43:07.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.317+0000 7fbe59920640 1 -- 172.21.15.98:0/2474620754 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe54159290 con 0x7fbe54073680 2024-09-19T08:43:07.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.317+0000 7fbe3bfff640 1 -- 172.21.15.98:0/2474620754 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe48007290 con 0x7fbe54073680 2024-09-19T08:43:07.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.317+0000 7fbe59920640 1 -- 172.21.15.98:0/2474620754 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe54159850 con 0x7fbe54073680 2024-09-19T08:43:07.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.317+0000 7fbe59920640 1 -- 172.21.15.98:0/2474620754 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe54069460 con 0x7fbe54073680 2024-09-19T08:43:07.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.321+0000 7fbe3bfff640 1 -- 172.21.15.98:0/2474620754 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fbe4801a070 con 0x7fbe54073680 2024-09-19T08:43:07.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.321+0000 7fbe3bfff640 1 --2- 172.21.15.98:0/2474620754 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbe400679e0 0x7fbe40069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:07.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.321+0000 7fbe3bfff640 1 -- 172.21.15.98:0/2474620754 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6470+0+0 (secure 0 0 0) 0x7fbe48085490 con 0x7fbe54073680 2024-09-19T08:43:07.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.321+0000 7fbe52ffd640 1 --2- 172.21.15.98:0/2474620754 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbe400679e0 0x7fbe40069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:07.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.321+0000 7fbe52ffd640 1 --2- 172.21.15.98:0/2474620754 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbe400679e0 0x7fbe40069ea0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbe440af000 tx=0x7fbe44008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:07.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.325+0000 7fbe3bfff640 1 -- 172.21.15.98:0/2474620754 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fbe48055c50 con 0x7fbe54073680 2024-09-19T08:43:07.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:07.489+0000 7fbe59920640 1 -- 172.21.15.98:0/2474620754 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbe540b6c00 con 0x7fbe54073680 2024-09-19T08:43:08.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.125+0000 7fbe3bfff640 1 -- 172.21.15.98:0/2474620754 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v73) ==== 181+0+0 (secure 0 0 0) 0x7fbe48055550 con 0x7fbe54073680 2024-09-19T08:43:08.125 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_metadata' removed 2024-09-19T08:43:08.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.129+0000 7fbe59920640 1 -- 172.21.15.98:0/2474620754 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbe400679e0 msgr2=0x7fbe40069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:08.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.129+0000 7fbe59920640 1 --2- 172.21.15.98:0/2474620754 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbe400679e0 0x7fbe40069ea0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbe440af000 tx=0x7fbe44008040 comp rx=0 tx=0).stop 2024-09-19T08:43:08.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.129+0000 7fbe59920640 1 -- 172.21.15.98:0/2474620754 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe54073680 msgr2=0x7fbe5415a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:08.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.129+0000 7fbe59920640 1 --2- 172.21.15.98:0/2474620754 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe54073680 0x7fbe5415a9a0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fbe4800c480 tx=0x7fbe4800c940 comp rx=0 tx=0).stop 2024-09-19T08:43:08.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.129+0000 7fbe59920640 1 -- 172.21.15.98:0/2474620754 shutdown_connections 2024-09-19T08:43:08.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.129+0000 7fbe59920640 1 --2- 172.21.15.98:0/2474620754 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbe400679e0 0x7fbe40069ea0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:08.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.129+0000 7fbe59920640 1 --2- 172.21.15.98:0/2474620754 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe5415e3c0 0x7fbe540b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:08.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.129+0000 7fbe59920640 1 --2- 172.21.15.98:0/2474620754 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe54073680 0x7fbe5415a9a0 secure :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fbe4800c480 tx=0x7fbe4800c940 comp rx=0 tx=0).stop 2024-09-19T08:43:08.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.129+0000 7fbe59920640 1 --2- 172.21.15.98:0/2474620754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe5406e7e0 0x7fbe5414ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:08.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.129+0000 7fbe59920640 1 -- 172.21.15.98:0/2474620754 >> 172.21.15.98:0/2474620754 conn(0x7fbe540b4180 msgr2=0x7fbe5415cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:08.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.129+0000 7fbe59920640 1 -- 172.21.15.98:0/2474620754 shutdown_connections 2024-09-19T08:43:08.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.129+0000 7fbe59920640 1 -- 172.21.15.98:0/2474620754 wait complete. 2024-09-19T08:43:08.151 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T08:43:08.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.281+0000 7fc1571a8640 1 Processor -- start 2024-09-19T08:43:08.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.281+0000 7fc1571a8640 1 -- start start 2024-09-19T08:43:08.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.281+0000 7fc1571a8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc15006fa90 0x7fc15006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:08.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.281+0000 7fc1571a8640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1500737b0 con 0x7fc150074000 2024-09-19T08:43:08.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.281+0000 7fc1571a8640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1500b79f0 con 0x7fc150162b10 2024-09-19T08:43:08.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.281+0000 7fc1571a8640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc150073230 con 0x7fc15006fa90 2024-09-19T08:43:08.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.281+0000 7fc155745640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fc150074000 0x7fc15006c1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41886/0 (socket says 172.21.15.98:41886) 2024-09-19T08:43:08.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.281+0000 7fc155745640 1 -- 172.21.15.98:0/3876574775 learned_addr learned my addr 172.21.15.98:0/3876574775 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:08.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.281+0000 7fc14ffff640 1 --2- 172.21.15.98:0/3876574775 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc15006fa90 0x7fc15006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:08.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.281+0000 7fc14f7fe640 1 -- 172.21.15.98:0/3876574775 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1839227055 0 0) 0x7fc1500737b0 con 0x7fc150074000 2024-09-19T08:43:08.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.281+0000 7fc14f7fe640 1 -- 172.21.15.98:0/3876574775 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc138003530 con 0x7fc150074000 2024-09-19T08:43:08.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.281+0000 7fc14f7fe640 1 -- 172.21.15.98:0/3876574775 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1491883283 0 0) 0x7fc138003530 con 0x7fc150074000 2024-09-19T08:43:08.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.281+0000 7fc14f7fe640 1 -- 172.21.15.98:0/3876574775 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc15006fa90 msgr2=0x7fc15006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:08.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.281+0000 7fc14f7fe640 1 --2- 172.21.15.98:0/3876574775 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc15006fa90 0x7fc15006bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:08.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.281+0000 7fc14f7fe640 1 -- 172.21.15.98:0/3876574775 >> v1:172.21.15.139:6789/0 conn(0x7fc150162b10 legacy=0x7fc15006f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:08.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.281+0000 7fc14f7fe640 1 -- 172.21.15.98:0/3876574775 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1500b7cd0 con 0x7fc150074000 2024-09-19T08:43:08.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.281+0000 7fc14f7fe640 1 -- 172.21.15.98:0/3876574775 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fc140001680 con 0x7fc150074000 2024-09-19T08:43:08.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.281+0000 7fc14f7fe640 1 -- 172.21.15.98:0/3876574775 >> v1:172.21.15.98:6789/0 conn(0x7fc150074000 legacy=0x7fc15006c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:08.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.281+0000 7fc14f7fe640 1 --2- 172.21.15.98:0/3876574775 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc1380039f0 0x7fc138003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:08.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.281+0000 7fc14f7fe640 1 --2- 172.21.15.98:0/3876574775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc138004810 0x7fc138004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:08.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.281+0000 7fc14f7fe640 1 --2- 172.21.15.98:0/3876574775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1380051d0 0x7fc13800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:08.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.281+0000 7fc14f7fe640 1 -- 172.21.15.98:0/3876574775 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc13800c0a0 con 0x7fc138004810 2024-09-19T08:43:08.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.281+0000 7fc14f7fe640 1 -- 172.21.15.98:0/3876574775 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc13800c220 con 0x7fc1380051d0 2024-09-19T08:43:08.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.281+0000 7fc14f7fe640 1 -- 172.21.15.98:0/3876574775 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc13800c420 con 0x7fc1380039f0 2024-09-19T08:43:08.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.281+0000 7fc14ffff640 1 --2- 172.21.15.98:0/3876574775 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc15006fa90 0x7fc15006bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:43:08.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.281+0000 7fc14ffff640 1 --2- 172.21.15.98:0/3876574775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc138004810 0x7fc138004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:08.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.281+0000 7fc154f44640 1 --2- 172.21.15.98:0/3876574775 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc1380039f0 0x7fc138003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:08.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.281+0000 7fc155745640 1 --2- 172.21.15.98:0/3876574775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1380051d0 0x7fc13800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:08.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.281+0000 7fc14ffff640 1 -- 172.21.15.98:0/3876574775 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc1380039f0 msgr2=0x7fc138003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:08.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.281+0000 7fc14ffff640 1 --2- 172.21.15.98:0/3876574775 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc1380039f0 0x7fc138003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:08.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.281+0000 7fc14ffff640 1 -- 172.21.15.98:0/3876574775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1380051d0 msgr2=0x7fc13800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:08.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.281+0000 7fc14ffff640 1 --2- 172.21.15.98:0/3876574775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1380051d0 0x7fc13800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:08.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.281+0000 7fc14ffff640 1 -- 172.21.15.98:0/3876574775 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc138003530 con 0x7fc138004810 2024-09-19T08:43:08.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.281+0000 7fc154f44640 1 --2- 172.21.15.98:0/3876574775 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc1380039f0 0x7fc138003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:43:08.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.285+0000 7fc14ffff640 1 --2- 172.21.15.98:0/3876574775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc138004810 0x7fc138004c90 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fc1440096c0 tx=0x7fc14401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=775b458834f8b5a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:08.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.285+0000 7fc14f7fe640 1 -- 172.21.15.98:0/3876574775 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1440b0070 con 0x7fc138004810 2024-09-19T08:43:08.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.285+0000 7fc14f7fe640 1 -- 172.21.15.98:0/3876574775 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1440ab600 con 0x7fc138004810 2024-09-19T08:43:08.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.285+0000 7fc14f7fe640 1 -- 172.21.15.98:0/3876574775 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14400f040 con 0x7fc138004810 2024-09-19T08:43:08.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.285+0000 7fc1571a8640 1 -- 172.21.15.98:0/3876574775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc138004810 msgr2=0x7fc138004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:08.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.285+0000 7fc1571a8640 1 --2- 172.21.15.98:0/3876574775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc138004810 0x7fc138004c90 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fc1440096c0 tx=0x7fc14401c3f0 comp rx=0 tx=0).stop 2024-09-19T08:43:08.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.285+0000 7fc155745640 1 -- 172.21.15.98:0/3876574775 reap_dead start 2024-09-19T08:43:08.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.285+0000 7fc1571a8640 1 -- 172.21.15.98:0/3876574775 shutdown_connections 2024-09-19T08:43:08.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.285+0000 7fc1571a8640 1 -- 172.21.15.98:0/3876574775 >> 172.21.15.98:0/3876574775 conn(0x7fc1500cedf0 msgr2=0x7fc1500cf200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:08.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.285+0000 7fc1571a8640 1 -- 172.21.15.98:0/3876574775 shutdown_connections 2024-09-19T08:43:08.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.285+0000 7fc1571a8640 1 -- 172.21.15.98:0/3876574775 wait complete. 2024-09-19T08:43:08.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.285+0000 7fc1571a8640 1 Processor -- start 2024-09-19T08:43:08.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.285+0000 7fc1571a8640 1 -- start start 2024-09-19T08:43:08.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.289+0000 7fc1571a8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc138004810 0x7fc1500b5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:08.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.289+0000 7fc1571a8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1501f54c0 0x7fc1501f5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:08.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.289+0000 7fc1571a8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1501f86f0 0x7fc1501f8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:08.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.289+0000 7fc1571a8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc1500d1cb0 con 0x7fc1501f54c0 2024-09-19T08:43:08.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.289+0000 7fc1571a8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc150073560 con 0x7fc1501f86f0 2024-09-19T08:43:08.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.289+0000 7fc1571a8640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc150162ef0 con 0x7fc138004810 2024-09-19T08:43:08.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.289+0000 7fc14ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1501f54c0 0x7fc1501f5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:08.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.289+0000 7fc14ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1501f54c0 0x7fc1501f5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46030/0 (socket says 172.21.15.98:46030) 2024-09-19T08:43:08.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.289+0000 7fc14ffff640 1 -- 172.21.15.98:0/3934556966 learned_addr learned my addr 172.21.15.98:0/3934556966 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:08.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.289+0000 7fc154f44640 1 --2- 172.21.15.98:0/3934556966 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc138004810 0x7fc1500b5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:08.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.289+0000 7fc155745640 1 --2- 172.21.15.98:0/3934556966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1501f86f0 0x7fc1501f8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:08.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.289+0000 7fc14ffff640 1 -- 172.21.15.98:0/3934556966 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc138004810 msgr2=0x7fc1500b5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:08.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.289+0000 7fc14ffff640 1 --2- 172.21.15.98:0/3934556966 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc138004810 0x7fc1500b5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:08.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.289+0000 7fc14ffff640 1 -- 172.21.15.98:0/3934556966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1501f86f0 msgr2=0x7fc1501f8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:08.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.289+0000 7fc14ffff640 1 --2- 172.21.15.98:0/3934556966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1501f86f0 0x7fc1501f8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:08.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.289+0000 7fc14ffff640 1 -- 172.21.15.98:0/3934556966 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1501f9450 con 0x7fc1501f54c0 2024-09-19T08:43:08.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.289+0000 7fc14ffff640 1 --2- 172.21.15.98:0/3934556966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1501f54c0 0x7fc1501f5940 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc14401c220 tx=0x7fc138005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:08.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.289+0000 7fc14d7fa640 1 -- 172.21.15.98:0/3934556966 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1440b0040 con 0x7fc1501f54c0 2024-09-19T08:43:08.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.289+0000 7fc14d7fa640 1 -- 172.21.15.98:0/3934556966 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1440bd070 con 0x7fc1501f54c0 2024-09-19T08:43:08.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.289+0000 7fc14d7fa640 1 -- 172.21.15.98:0/3934556966 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1440b6690 con 0x7fc1501f54c0 2024-09-19T08:43:08.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.289+0000 7fc1571a8640 1 -- 172.21.15.98:0/3934556966 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1500aa540 con 0x7fc1501f54c0 2024-09-19T08:43:08.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.289+0000 7fc1571a8640 1 -- 172.21.15.98:0/3934556966 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1500aab00 con 0x7fc1501f54c0 2024-09-19T08:43:08.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.289+0000 7fc1571a8640 1 -- 172.21.15.98:0/3934556966 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc150073ab0 con 0x7fc1501f54c0 2024-09-19T08:43:08.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.289+0000 7fc14d7fa640 1 -- 172.21.15.98:0/3934556966 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fc1440b6830 con 0x7fc1501f54c0 2024-09-19T08:43:08.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.289+0000 7fc14d7fa640 1 --2- 172.21.15.98:0/3934556966 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc13c067a10 0x7fc13c069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:08.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.289+0000 7fc14d7fa640 1 -- 172.21.15.98:0/3934556966 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6032+0+0 (secure 0 0 0) 0x7fc1441208e0 con 0x7fc1501f54c0 2024-09-19T08:43:08.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.289+0000 7fc154f44640 1 --2- 172.21.15.98:0/3934556966 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc13c067a10 0x7fc13c069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:08.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.293+0000 7fc154f44640 1 --2- 172.21.15.98:0/3934556966 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc13c067a10 0x7fc13c069ed0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc148000f90 tx=0x7fc148009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:08.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.293+0000 7fc14d7fa640 1 -- 172.21.15.98:0/3934556966 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc144016100 con 0x7fc1501f54c0 2024-09-19T08:43:08.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:08.461+0000 7fc1571a8640 1 -- 172.21.15.98:0/3934556966 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc1500713d0 con 0x7fc1501f54c0 2024-09-19T08:43:09.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.133+0000 7fc14d7fa640 1 -- 172.21.15.98:0/3934556966 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v74) ==== 169+0+0 (secure 0 0 0) 0x7fc1440f1180 con 0x7fc1501f54c0 2024-09-19T08:43:09.136 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' removed 2024-09-19T08:43:09.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.141+0000 7fc1571a8640 1 -- 172.21.15.98:0/3934556966 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc13c067a10 msgr2=0x7fc13c069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:09.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.141+0000 7fc1571a8640 1 --2- 172.21.15.98:0/3934556966 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc13c067a10 0x7fc13c069ed0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc148000f90 tx=0x7fc148009040 comp rx=0 tx=0).stop 2024-09-19T08:43:09.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.141+0000 7fc1571a8640 1 -- 172.21.15.98:0/3934556966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1501f54c0 msgr2=0x7fc1501f5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:09.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.141+0000 7fc1571a8640 1 --2- 172.21.15.98:0/3934556966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1501f54c0 0x7fc1501f5940 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc14401c220 tx=0x7fc138005840 comp rx=0 tx=0).stop 2024-09-19T08:43:09.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.141+0000 7fc1571a8640 1 -- 172.21.15.98:0/3934556966 shutdown_connections 2024-09-19T08:43:09.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.141+0000 7fc1571a8640 1 --2- 172.21.15.98:0/3934556966 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc13c067a10 0x7fc13c069ed0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:09.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.141+0000 7fc1571a8640 1 --2- 172.21.15.98:0/3934556966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1501f86f0 0x7fc1501f8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:09.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.141+0000 7fc1571a8640 1 --2- 172.21.15.98:0/3934556966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc1501f54c0 0x7fc1501f5940 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:09.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.141+0000 7fc1571a8640 1 --2- 172.21.15.98:0/3934556966 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc138004810 0x7fc1500b5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:09.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.141+0000 7fc1571a8640 1 -- 172.21.15.98:0/3934556966 >> 172.21.15.98:0/3934556966 conn(0x7fc1500cedf0 msgr2=0x7fc150070580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:09.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.141+0000 7fc1571a8640 1 -- 172.21.15.98:0/3934556966 shutdown_connections 2024-09-19T08:43:09.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.141+0000 7fc1571a8640 1 -- 172.21.15.98:0/3934556966 wait complete. 2024-09-19T08:43:09.162 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-19T08:43:09.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.281+0000 7fef974df640 1 Processor -- start 2024-09-19T08:43:09.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.281+0000 7fef974df640 1 -- start start 2024-09-19T08:43:09.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.285+0000 7fef974df640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fef90159f60 0x7fef900b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:09.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.285+0000 7fef974df640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef9006f2e0 0x7fef9006f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:09.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.285+0000 7fef974df640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef9015d760 con 0x7fef9015c9f0 2024-09-19T08:43:09.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.285+0000 7fef974df640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fef9015a340 con 0x7fef90159f60 2024-09-19T08:43:09.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.285+0000 7fef974df640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fef9006fc60 con 0x7fef9006f2e0 2024-09-19T08:43:09.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.285+0000 7fef9527b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fef90159f60 0x7fef900b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:09.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.285+0000 7fef95a7c640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fef9015c9f0 0x7fef9015ce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41892/0 (socket says 172.21.15.98:41892) 2024-09-19T08:43:09.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.285+0000 7fef95a7c640 1 -- 172.21.15.98:0/784740990 learned_addr learned my addr 172.21.15.98:0/784740990 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:09.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.285+0000 7fef9527b640 1 -- 172.21.15.98:0/784740990 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef9006f2e0 msgr2=0x7fef9006f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:09.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.285+0000 7fef9527b640 1 --2- 172.21.15.98:0/784740990 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef9006f2e0 0x7fef9006f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:09.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.285+0000 7fef9527b640 1 -- 172.21.15.98:0/784740990 >> v1:172.21.15.98:6789/0 conn(0x7fef9015c9f0 legacy=0x7fef9015ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:09.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.285+0000 7fef9527b640 1 -- 172.21.15.98:0/784740990 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef900b5410 con 0x7fef90159f60 2024-09-19T08:43:09.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.285+0000 7fef87fff640 1 -- 172.21.15.98:0/784740990 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3188158154 0 0) 0x7fef9015d760 con 0x7fef9015c9f0 2024-09-19T08:43:09.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.285+0000 7fef9527b640 1 --2- 172.21.15.98:0/784740990 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef90159f60 0x7fef900b61d0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0097c0 tx=0x7fef8c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab15d551a0b3c71 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:09.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.285+0000 7fef87fff640 1 -- 172.21.15.98:0/784740990 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c0b2070 con 0x7fef90159f60 2024-09-19T08:43:09.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.285+0000 7fef87fff640 1 -- 172.21.15.98:0/784740990 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef8c0ad640 con 0x7fef90159f60 2024-09-19T08:43:09.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.285+0000 7fef87fff640 1 -- 172.21.15.98:0/784740990 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c0ada10 con 0x7fef90159f60 2024-09-19T08:43:09.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.285+0000 7fef974df640 1 -- 172.21.15.98:0/784740990 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef90159f60 msgr2=0x7fef900b61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:09.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.285+0000 7fef974df640 1 --2- 172.21.15.98:0/784740990 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef90159f60 0x7fef900b61d0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0097c0 tx=0x7fef8c01c4a0 comp rx=0 tx=0).stop 2024-09-19T08:43:09.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.285+0000 7fef974df640 1 -- 172.21.15.98:0/784740990 shutdown_connections 2024-09-19T08:43:09.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.285+0000 7fef974df640 1 --2- 172.21.15.98:0/784740990 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef9006f2e0 0x7fef9006f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:09.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.285+0000 7fef974df640 1 --2- 172.21.15.98:0/784740990 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef90159f60 0x7fef900b61d0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:09.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.289+0000 7fef974df640 1 -- 172.21.15.98:0/784740990 >> 172.21.15.98:0/784740990 conn(0x7fef9006dea0 msgr2=0x7fef9006e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:09.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.289+0000 7fef974df640 1 -- 172.21.15.98:0/784740990 shutdown_connections 2024-09-19T08:43:09.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.289+0000 7fef974df640 1 -- 172.21.15.98:0/784740990 wait complete. 2024-09-19T08:43:09.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.289+0000 7fef974df640 1 Processor -- start 2024-09-19T08:43:09.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.289+0000 7fef974df640 1 -- start start 2024-09-19T08:43:09.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.289+0000 7fef974df640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef9006f2e0 0x7fef900cc480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:09.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.289+0000 7fef974df640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef90159f60 0x7fef900ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:09.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.289+0000 7fef974df640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef9015c9f0 0x7fef9015a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:09.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.289+0000 7fef974df640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fef90071190 con 0x7fef9015c9f0 2024-09-19T08:43:09.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.289+0000 7fef974df640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fef90071010 con 0x7fef9006f2e0 2024-09-19T08:43:09.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.289+0000 7fef974df640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fef90071310 con 0x7fef90159f60 2024-09-19T08:43:09.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.289+0000 7fef95a7c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef9015c9f0 0x7fef9015a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:09.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.289+0000 7fef95a7c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef9015c9f0 0x7fef9015a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46034/0 (socket says 172.21.15.98:46034) 2024-09-19T08:43:09.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.289+0000 7fef95a7c640 1 -- 172.21.15.98:0/2313674625 learned_addr learned my addr 172.21.15.98:0/2313674625 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:09.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.289+0000 7fef95a7c640 1 -- 172.21.15.98:0/2313674625 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef90159f60 msgr2=0x7fef900ccd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:09.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.289+0000 7fef95a7c640 1 --2- 172.21.15.98:0/2313674625 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef90159f60 0x7fef900ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:09.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.289+0000 7fef95a7c640 1 -- 172.21.15.98:0/2313674625 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef9006f2e0 msgr2=0x7fef900cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:09.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.289+0000 7fef95a7c640 1 --2- 172.21.15.98:0/2313674625 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef9006f2e0 0x7fef900cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:09.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.289+0000 7fef95a7c640 1 -- 172.21.15.98:0/2313674625 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef900cd740 con 0x7fef9015c9f0 2024-09-19T08:43:09.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.289+0000 7fef95a7c640 1 --2- 172.21.15.98:0/2313674625 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef9015c9f0 0x7fef9015a4c0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fef8800c700 tx=0x7fef8800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:09.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.289+0000 7fef85ffb640 1 -- 172.21.15.98:0/2313674625 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88018070 con 0x7fef9015c9f0 2024-09-19T08:43:09.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.289+0000 7fef85ffb640 1 -- 172.21.15.98:0/2313674625 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef8800ce10 con 0x7fef9015c9f0 2024-09-19T08:43:09.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.289+0000 7fef974df640 1 -- 172.21.15.98:0/2313674625 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef900cb9a0 con 0x7fef9015c9f0 2024-09-19T08:43:09.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.289+0000 7fef85ffb640 1 -- 172.21.15.98:0/2313674625 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8800a860 con 0x7fef9015c9f0 2024-09-19T08:43:09.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.289+0000 7fef974df640 1 -- 172.21.15.98:0/2313674625 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef900cc9c0 con 0x7fef9015c9f0 2024-09-19T08:43:09.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.289+0000 7fef974df640 1 -- 172.21.15.98:0/2313674625 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef9006f720 con 0x7fef9015c9f0 2024-09-19T08:43:09.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.293+0000 7fef85ffb640 1 -- 172.21.15.98:0/2313674625 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fef8801b070 con 0x7fef9015c9f0 2024-09-19T08:43:09.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.293+0000 7fef85ffb640 1 --2- 172.21.15.98:0/2313674625 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fef740679e0 0x7fef74069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:09.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.293+0000 7fef85ffb640 1 -- 172.21.15.98:0/2313674625 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5502+0+0 (secure 0 0 0) 0x7fef88085250 con 0x7fef9015c9f0 2024-09-19T08:43:09.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.293+0000 7fef9527b640 1 --2- 172.21.15.98:0/2313674625 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fef740679e0 0x7fef74069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:09.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.293+0000 7fef9527b640 1 --2- 172.21.15.98:0/2313674625 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fef740679e0 0x7fef74069ea0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0af000 tx=0x7fef8c004740 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:09.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.297+0000 7fef85ffb640 1 -- 172.21.15.98:0/2313674625 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fef88055de0 con 0x7fef9015c9f0 2024-09-19T08:43:09.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.469+0000 7fef974df640 1 -- 172.21.15.98:0/2313674625 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fef900b67d0 con 0x7fef9015c9f0 2024-09-19T08:43:09.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.473+0000 7fef85ffb640 1 -- 172.21.15.98:0/2313674625 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7fef880556e0 con 0x7fef9015c9f0 2024-09-19T08:43:09.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.477+0000 7fef974df640 1 -- 172.21.15.98:0/2313674625 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fef740679e0 msgr2=0x7fef74069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:09.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.477+0000 7fef974df640 1 --2- 172.21.15.98:0/2313674625 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fef740679e0 0x7fef74069ea0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fef8c0af000 tx=0x7fef8c004740 comp rx=0 tx=0).stop 2024-09-19T08:43:09.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.477+0000 7fef974df640 1 -- 172.21.15.98:0/2313674625 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef9015c9f0 msgr2=0x7fef9015a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:09.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.477+0000 7fef974df640 1 --2- 172.21.15.98:0/2313674625 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef9015c9f0 0x7fef9015a4c0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fef8800c700 tx=0x7fef8800cbc0 comp rx=0 tx=0).stop 2024-09-19T08:43:09.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.477+0000 7fef974df640 1 -- 172.21.15.98:0/2313674625 shutdown_connections 2024-09-19T08:43:09.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.477+0000 7fef974df640 1 --2- 172.21.15.98:0/2313674625 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fef740679e0 0x7fef74069ea0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:09.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.477+0000 7fef974df640 1 --2- 172.21.15.98:0/2313674625 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fef9015c9f0 0x7fef9015a4c0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:09.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.477+0000 7fef974df640 1 --2- 172.21.15.98:0/2313674625 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fef90159f60 0x7fef900ccd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:09.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.477+0000 7fef974df640 1 --2- 172.21.15.98:0/2313674625 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef9006f2e0 0x7fef900cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:09.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.477+0000 7fef974df640 1 -- 172.21.15.98:0/2313674625 >> 172.21.15.98:0/2313674625 conn(0x7fef9006dea0 msgr2=0x7fef9015ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:09.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.477+0000 7fef974df640 1 -- 172.21.15.98:0/2313674625 shutdown_connections 2024-09-19T08:43:09.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.477+0000 7fef974df640 1 -- 172.21.15.98:0/2313674625 wait complete. 2024-09-19T08:43:09.494 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction' 2024-09-19T08:43:09.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.613+0000 7fc366014640 1 Processor -- start 2024-09-19T08:43:09.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.617+0000 7fc366014640 1 -- start start 2024-09-19T08:43:09.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.617+0000 7fc366014640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc36006c3a0 0x7fc3600cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:09.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.617+0000 7fc366014640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc36006d310 con 0x7fc36015c550 2024-09-19T08:43:09.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.617+0000 7fc366014640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc36006d5f0 con 0x7fc3600cf900 2024-09-19T08:43:09.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.617+0000 7fc366014640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fc36006d8d0 con 0x7fc36006c3a0 2024-09-19T08:43:09.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.617+0000 7fc35f7fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fc36015c550 0x7fc36006bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41908/0 (socket says 172.21.15.98:41908) 2024-09-19T08:43:09.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.617+0000 7fc35f7fe640 1 -- 172.21.15.98:0/3227944038 learned_addr learned my addr 172.21.15.98:0/3227944038 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:09.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.617+0000 7fc35effd640 1 --2- 172.21.15.98:0/3227944038 >> v2:172.21.15.139:3300/0 conn(0x7fc36006c3a0 0x7fc3600cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:09.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.617+0000 7fc35e7fc640 1 -- 172.21.15.98:0/3227944038 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3600991149 0 0) 0x7fc36006d310 con 0x7fc36015c550 2024-09-19T08:43:09.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.617+0000 7fc35e7fc640 1 -- 172.21.15.98:0/3227944038 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc348003530 con 0x7fc36015c550 2024-09-19T08:43:09.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.617+0000 7fc35e7fc640 1 -- 172.21.15.98:0/3227944038 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3598492668 0 0) 0x7fc348003530 con 0x7fc36015c550 2024-09-19T08:43:09.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.617+0000 7fc35e7fc640 1 -- 172.21.15.98:0/3227944038 >> v2:172.21.15.139:3300/0 conn(0x7fc36006c3a0 msgr2=0x7fc3600cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:09.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.617+0000 7fc35e7fc640 1 --2- 172.21.15.98:0/3227944038 >> v2:172.21.15.139:3300/0 conn(0x7fc36006c3a0 0x7fc3600cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:09.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.617+0000 7fc35e7fc640 1 -- 172.21.15.98:0/3227944038 >> v1:172.21.15.139:6789/0 conn(0x7fc3600cf900 legacy=0x7fc3600723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:09.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.617+0000 7fc35e7fc640 1 -- 172.21.15.98:0/3227944038 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc36006dbb0 con 0x7fc36015c550 2024-09-19T08:43:09.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.617+0000 7fc35e7fc640 1 -- 172.21.15.98:0/3227944038 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fc354001680 con 0x7fc36015c550 2024-09-19T08:43:09.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.617+0000 7fc35e7fc640 1 -- 172.21.15.98:0/3227944038 >> v1:172.21.15.98:6789/0 conn(0x7fc36015c550 legacy=0x7fc36006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:09.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.617+0000 7fc35e7fc640 1 --2- 172.21.15.98:0/3227944038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3480039c0 0x7fc348003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:09.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.617+0000 7fc35e7fc640 1 --2- 172.21.15.98:0/3227944038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3480043c0 0x7fc348004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:09.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.617+0000 7fc35e7fc640 1 --2- 172.21.15.98:0/3227944038 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc348005090 0x7fc34800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:09.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.617+0000 7fc35e7fc640 1 -- 172.21.15.98:0/3227944038 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc36006d8d0 con 0x7fc3480043c0 2024-09-19T08:43:09.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.617+0000 7fc35e7fc640 1 -- 172.21.15.98:0/3227944038 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc34800bff0 con 0x7fc3480039c0 2024-09-19T08:43:09.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.617+0000 7fc35e7fc640 1 -- 172.21.15.98:0/3227944038 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc34800c1f0 con 0x7fc348005090 2024-09-19T08:43:09.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.617+0000 7fc35effd640 1 --2- 172.21.15.98:0/3227944038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3480043c0 0x7fc348004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:09.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.617+0000 7fc35f7fe640 1 --2- 172.21.15.98:0/3227944038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3480039c0 0x7fc348003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:09.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.617+0000 7fc35ffff640 1 --2- 172.21.15.98:0/3227944038 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc348005090 0x7fc34800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:09.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.617+0000 7fc35effd640 1 -- 172.21.15.98:0/3227944038 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc348005090 msgr2=0x7fc34800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:09.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.617+0000 7fc35effd640 1 --2- 172.21.15.98:0/3227944038 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc348005090 0x7fc34800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:09.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.617+0000 7fc35effd640 1 -- 172.21.15.98:0/3227944038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3480039c0 msgr2=0x7fc348003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:09.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.617+0000 7fc35effd640 1 --2- 172.21.15.98:0/3227944038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3480039c0 0x7fc348003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:09.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.617+0000 7fc35effd640 1 -- 172.21.15.98:0/3227944038 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc348003530 con 0x7fc3480043c0 2024-09-19T08:43:09.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.617+0000 7fc35ffff640 1 --2- 172.21.15.98:0/3227944038 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc348005090 0x7fc34800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:09.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.617+0000 7fc35ffff640 1 -- 172.21.15.98:0/3227944038 reap_dead start 2024-09-19T08:43:09.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.621+0000 7fc35effd640 1 --2- 172.21.15.98:0/3227944038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3480043c0 0x7fc348004890 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fc350009ac0 tx=0x7fc35001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b23093d55db3c02f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:09.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.621+0000 7fc35e7fc640 1 -- 172.21.15.98:0/3227944038 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3500b2070 con 0x7fc3480043c0 2024-09-19T08:43:09.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.621+0000 7fc35e7fc640 1 -- 172.21.15.98:0/3227944038 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3500a5640 con 0x7fc3480043c0 2024-09-19T08:43:09.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.621+0000 7fc35e7fc640 1 -- 172.21.15.98:0/3227944038 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc350009690 con 0x7fc3480043c0 2024-09-19T08:43:09.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.621+0000 7fc366014640 1 -- 172.21.15.98:0/3227944038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3480043c0 msgr2=0x7fc348004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:09.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.621+0000 7fc366014640 1 --2- 172.21.15.98:0/3227944038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3480043c0 0x7fc348004890 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fc350009ac0 tx=0x7fc35001ca50 comp rx=0 tx=0).stop 2024-09-19T08:43:09.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.621+0000 7fc366014640 1 -- 172.21.15.98:0/3227944038 shutdown_connections 2024-09-19T08:43:09.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.621+0000 7fc366014640 1 --2- 172.21.15.98:0/3227944038 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc3480043c0 0x7fc348004890 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:09.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.621+0000 7fc366014640 1 -- 172.21.15.98:0/3227944038 >> 172.21.15.98:0/3227944038 conn(0x7fc3600b2010 msgr2=0x7fc3600b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:09.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.621+0000 7fc366014640 1 -- 172.21.15.98:0/3227944038 shutdown_connections 2024-09-19T08:43:09.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.621+0000 7fc366014640 1 -- 172.21.15.98:0/3227944038 wait complete. 2024-09-19T08:43:09.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.621+0000 7fc366014640 1 Processor -- start 2024-09-19T08:43:09.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.621+0000 7fc366014640 1 -- start start 2024-09-19T08:43:09.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.621+0000 7fc366014640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc3480043c0 0x7fc3600b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:09.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.621+0000 7fc366014640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc36015dfe0 0x7fc36015f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:09.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.621+0000 7fc366014640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc360151990 0x7fc3601e5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:09.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.621+0000 7fc366014640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc36015cab0 con 0x7fc360151990 2024-09-19T08:43:09.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.621+0000 7fc366014640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc36015c930 con 0x7fc36015dfe0 2024-09-19T08:43:09.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.621+0000 7fc366014640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc3600cf3c0 con 0x7fc3480043c0 2024-09-19T08:43:09.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.621+0000 7fc35ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc360151990 0x7fc3601e5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:09.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.621+0000 7fc35ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc360151990 0x7fc3601e5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46048/0 (socket says 172.21.15.98:46048) 2024-09-19T08:43:09.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.621+0000 7fc35ffff640 1 -- 172.21.15.98:0/2849484564 learned_addr learned my addr 172.21.15.98:0/2849484564 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:09.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.625+0000 7fc35effd640 1 --2- 172.21.15.98:0/2849484564 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc36015dfe0 0x7fc36015f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:09.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.625+0000 7fc35ffff640 1 -- 172.21.15.98:0/2849484564 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc3480043c0 msgr2=0x7fc3600b0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:09.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.625+0000 7fc35ffff640 1 --2- 172.21.15.98:0/2849484564 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc3480043c0 0x7fc3600b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:09.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.625+0000 7fc35ffff640 1 -- 172.21.15.98:0/2849484564 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc36015dfe0 msgr2=0x7fc36015f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:09.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.625+0000 7fc35ffff640 1 --2- 172.21.15.98:0/2849484564 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc36015dfe0 0x7fc36015f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:09.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.625+0000 7fc35ffff640 1 -- 172.21.15.98:0/2849484564 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc36015e470 con 0x7fc360151990 2024-09-19T08:43:09.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.625+0000 7fc35ffff640 1 --2- 172.21.15.98:0/2849484564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc360151990 0x7fc3601e5910 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fc360073b20 tx=0x7fc35800e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:09.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.625+0000 7fc347fff640 1 -- 172.21.15.98:0/2849484564 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35801a070 con 0x7fc360151990 2024-09-19T08:43:09.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.625+0000 7fc347fff640 1 -- 172.21.15.98:0/2849484564 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc358011040 con 0x7fc360151990 2024-09-19T08:43:09.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.625+0000 7fc366014640 1 -- 172.21.15.98:0/2849484564 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc36015e750 con 0x7fc360151990 2024-09-19T08:43:09.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.625+0000 7fc347fff640 1 -- 172.21.15.98:0/2849484564 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc358015840 con 0x7fc360151990 2024-09-19T08:43:09.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.625+0000 7fc366014640 1 -- 172.21.15.98:0/2849484564 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc36015ed10 con 0x7fc360151990 2024-09-19T08:43:09.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.625+0000 7fc366014640 1 -- 172.21.15.98:0/2849484564 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3600cf6c0 con 0x7fc360151990 2024-09-19T08:43:09.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.625+0000 7fc347fff640 1 -- 172.21.15.98:0/2849484564 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fc35801d070 con 0x7fc360151990 2024-09-19T08:43:09.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.625+0000 7fc347fff640 1 --2- 172.21.15.98:0/2849484564 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc34c0679e0 0x7fc34c069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:09.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.625+0000 7fc347fff640 1 -- 172.21.15.98:0/2849484564 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5502+0+0 (secure 0 0 0) 0x7fc358088a90 con 0x7fc360151990 2024-09-19T08:43:09.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.625+0000 7fc35f7fe640 1 --2- 172.21.15.98:0/2849484564 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc34c0679e0 0x7fc34c069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:09.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.625+0000 7fc35f7fe640 1 --2- 172.21.15.98:0/2849484564 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc34c0679e0 0x7fc34c069ea0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc354001cd0 tx=0x7fc354001c20 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:09.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.629+0000 7fc347fff640 1 -- 172.21.15.98:0/2849484564 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc358059600 con 0x7fc360151990 2024-09-19T08:43:09.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:09.849+0000 7fc366014640 1 -- 172.21.15.98:0/2849484564 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]} v 0) -- 0x7fc3600b1200 con 0x7fc360151990 2024-09-19T08:43:10.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.137+0000 7fc347fff640 1 -- 172.21.15.98:0/2849484564 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]}]=0 v249) ==== 131+0+0 (secure 0 0 0) 0x7fc358058f00 con 0x7fc360151990 2024-09-19T08:43:10.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.141+0000 7fc366014640 1 -- 172.21.15.98:0/2849484564 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc34c0679e0 msgr2=0x7fc34c069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:10.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.141+0000 7fc366014640 1 --2- 172.21.15.98:0/2849484564 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc34c0679e0 0x7fc34c069ea0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc354001cd0 tx=0x7fc354001c20 comp rx=0 tx=0).stop 2024-09-19T08:43:10.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.141+0000 7fc366014640 1 -- 172.21.15.98:0/2849484564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc360151990 msgr2=0x7fc3601e5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:10.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.141+0000 7fc366014640 1 --2- 172.21.15.98:0/2849484564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc360151990 0x7fc3601e5910 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fc360073b20 tx=0x7fc35800e4d0 comp rx=0 tx=0).stop 2024-09-19T08:43:10.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.141+0000 7fc366014640 1 -- 172.21.15.98:0/2849484564 shutdown_connections 2024-09-19T08:43:10.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.141+0000 7fc366014640 1 --2- 172.21.15.98:0/2849484564 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc34c0679e0 0x7fc34c069ea0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:10.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.141+0000 7fc366014640 1 --2- 172.21.15.98:0/2849484564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc360151990 0x7fc3601e5910 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:10.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.141+0000 7fc366014640 1 --2- 172.21.15.98:0/2849484564 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc36015dfe0 0x7fc36015f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:10.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.141+0000 7fc366014640 1 --2- 172.21.15.98:0/2849484564 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc3480043c0 0x7fc3600b0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:10.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.141+0000 7fc366014640 1 -- 172.21.15.98:0/2849484564 >> 172.21.15.98:0/2849484564 conn(0x7fc3600b2010 msgr2=0x7fc360157490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:10.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.141+0000 7fc366014640 1 -- 172.21.15.98:0/2849484564 shutdown_connections 2024-09-19T08:43:10.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.141+0000 7fc366014640 1 -- 172.21.15.98:0/2849484564 wait complete. 2024-09-19T08:43:10.167 INFO:tasks.cephfs_test_runner:test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc) ... ok 2024-09-19T08:43:10.167 INFO:tasks.cephfs_test_runner:Starting test: test_names_can_only_be_goodchars (tasks.cephfs.test_volumes.TestMisc) Test the creating vols, subvols subvolgroups fails when their names uses 2024-09-19T08:43:10.167 INFO:tasks.cephfs_test_runner:test_names_can_only_be_goodchars (tasks.cephfs.test_volumes.TestMisc) 2024-09-19T08:43:10.168 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:43:10.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.293+0000 7ff3d5e66640 1 Processor -- start 2024-09-19T08:43:10.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.293+0000 7ff3d5e66640 1 -- start start 2024-09-19T08:43:10.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.293+0000 7ff3d5e66640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3d00b7f80 0x7ff3d015f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:10.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.293+0000 7ff3d5e66640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3d0068440 con 0x7ff3d00b9970 2024-09-19T08:43:10.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.293+0000 7ff3d5e66640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3d0162a90 con 0x7ff3d00d4820 2024-09-19T08:43:10.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.293+0000 7ff3d5e66640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff3d0163050 con 0x7ff3d00b7f80 2024-09-19T08:43:10.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.293+0000 7ff3cffff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7ff3d00b9970 0x7ff3d00b9e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41918/0 (socket says 172.21.15.98:41918) 2024-09-19T08:43:10.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.293+0000 7ff3cffff640 1 -- 172.21.15.98:0/7287202 learned_addr learned my addr 172.21.15.98:0/7287202 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:10.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.293+0000 7ff3cf7fe640 1 --2- 172.21.15.98:0/7287202 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3d00b7f80 0x7ff3d015f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:10.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.293+0000 7ff3ce7fc640 1 -- 172.21.15.98:0/7287202 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3760069073 0 0) 0x7ff3d0068440 con 0x7ff3d00b9970 2024-09-19T08:43:10.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.293+0000 7ff3ce7fc640 1 -- 172.21.15.98:0/7287202 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff3b8003530 con 0x7ff3d00b9970 2024-09-19T08:43:10.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.293+0000 7ff3ce7fc640 1 -- 172.21.15.98:0/7287202 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3999160857 0 0) 0x7ff3b8003530 con 0x7ff3d00b9970 2024-09-19T08:43:10.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.293+0000 7ff3ce7fc640 1 -- 172.21.15.98:0/7287202 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3d00b7f80 msgr2=0x7ff3d015f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:10.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.293+0000 7ff3ce7fc640 1 --2- 172.21.15.98:0/7287202 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3d00b7f80 0x7ff3d015f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:10.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.293+0000 7ff3ce7fc640 1 -- 172.21.15.98:0/7287202 >> v1:172.21.15.139:6789/0 conn(0x7ff3d00d4820 legacy=0x7ff3d01621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:10.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.293+0000 7ff3ce7fc640 1 -- 172.21.15.98:0/7287202 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d0162d70 con 0x7ff3d00b9970 2024-09-19T08:43:10.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.293+0000 7ff3ce7fc640 1 -- 172.21.15.98:0/7287202 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7ff3c8001a00 con 0x7ff3d00b9970 2024-09-19T08:43:10.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.293+0000 7ff3ce7fc640 1 -- 172.21.15.98:0/7287202 >> v1:172.21.15.98:6789/0 conn(0x7ff3d00b9970 legacy=0x7ff3d00b9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:10.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.293+0000 7ff3ce7fc640 1 --2- 172.21.15.98:0/7287202 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3d00b7f80 0x7ff3b80039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:10.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.293+0000 7ff3ce7fc640 1 --2- 172.21.15.98:0/7287202 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3b8003f70 0x7ff3b8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:10.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.293+0000 7ff3ce7fc640 1 --2- 172.21.15.98:0/7287202 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3b8004d50 0x7ff3b80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:10.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.293+0000 7ff3ce7fc640 1 -- 172.21.15.98:0/7287202 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d0163050 con 0x7ff3b8003f70 2024-09-19T08:43:10.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.293+0000 7ff3ce7fc640 1 -- 172.21.15.98:0/7287202 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b800bab0 con 0x7ff3b8004d50 2024-09-19T08:43:10.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.293+0000 7ff3ce7fc640 1 -- 172.21.15.98:0/7287202 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff3b800bcb0 con 0x7ff3d00b7f80 2024-09-19T08:43:10.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.293+0000 7ff3ceffd640 1 --2- 172.21.15.98:0/7287202 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3b8003f70 0x7ff3b8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:10.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.293+0000 7ff3cffff640 1 --2- 172.21.15.98:0/7287202 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3b8004d50 0x7ff3b80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:10.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.293+0000 7ff3cf7fe640 1 --2- 172.21.15.98:0/7287202 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3d00b7f80 0x7ff3b80039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:10.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.293+0000 7ff3ceffd640 1 -- 172.21.15.98:0/7287202 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3d00b7f80 msgr2=0x7ff3b80039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:10.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.293+0000 7ff3ceffd640 1 --2- 172.21.15.98:0/7287202 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3d00b7f80 0x7ff3b80039f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:10.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.293+0000 7ff3ceffd640 1 -- 172.21.15.98:0/7287202 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3b8004d50 msgr2=0x7ff3b80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:10.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.293+0000 7ff3ceffd640 1 --2- 172.21.15.98:0/7287202 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3b8004d50 0x7ff3b80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:10.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.293+0000 7ff3ceffd640 1 -- 172.21.15.98:0/7287202 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3b8003530 con 0x7ff3b8003f70 2024-09-19T08:43:10.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.293+0000 7ff3cffff640 1 --2- 172.21.15.98:0/7287202 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3b8004d50 0x7ff3b80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:10.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.297+0000 7ff3ceffd640 1 --2- 172.21.15.98:0/7287202 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3b8003f70 0x7ff3b8004810 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0001750 tx=0x7ff3c001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9176043270456ed9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:10.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.297+0000 7ff3ce7fc640 1 -- 172.21.15.98:0/7287202 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c00013a0 con 0x7ff3b8003f70 2024-09-19T08:43:10.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.297+0000 7ff3ce7fc640 1 -- 172.21.15.98:0/7287202 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3c00052f0 con 0x7ff3b8003f70 2024-09-19T08:43:10.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.297+0000 7ff3ce7fc640 1 -- 172.21.15.98:0/7287202 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c80020d0 con 0x7ff3b8003f70 2024-09-19T08:43:10.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.297+0000 7ff3d5e66640 1 -- 172.21.15.98:0/7287202 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3b8003f70 msgr2=0x7ff3b8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:10.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.297+0000 7ff3d5e66640 1 --2- 172.21.15.98:0/7287202 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3b8003f70 0x7ff3b8004810 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0001750 tx=0x7ff3c001c3e0 comp rx=0 tx=0).stop 2024-09-19T08:43:10.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.297+0000 7ff3cffff640 1 -- 172.21.15.98:0/7287202 reap_dead start 2024-09-19T08:43:10.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.297+0000 7ff3d5e66640 1 -- 172.21.15.98:0/7287202 shutdown_connections 2024-09-19T08:43:10.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.297+0000 7ff3d5e66640 1 -- 172.21.15.98:0/7287202 >> 172.21.15.98:0/7287202 conn(0x7ff3d0074e40 msgr2=0x7ff3d0075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:10.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.297+0000 7ff3d5e66640 1 -- 172.21.15.98:0/7287202 shutdown_connections 2024-09-19T08:43:10.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.297+0000 7ff3d5e66640 1 -- 172.21.15.98:0/7287202 wait complete. 2024-09-19T08:43:10.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.297+0000 7ff3d5e66640 1 Processor -- start 2024-09-19T08:43:10.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.297+0000 7ff3d5e66640 1 -- start start 2024-09-19T08:43:10.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.297+0000 7ff3d5e66640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3b8003f70 0x7ff3d00cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:10.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.297+0000 7ff3d5e66640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3d0154460 0x7ff3d01548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:10.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.297+0000 7ff3d5e66640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3d0143830 0x7ff3d0143cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:10.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.297+0000 7ff3d5e66640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d00b84e0 con 0x7ff3d0143830 2024-09-19T08:43:10.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.297+0000 7ff3d5e66640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d00b8360 con 0x7ff3b8003f70 2024-09-19T08:43:10.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.297+0000 7ff3d5e66640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff3d015f560 con 0x7ff3d0154460 2024-09-19T08:43:10.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.301+0000 7ff3cffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3d0143830 0x7ff3d0143cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:10.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.301+0000 7ff3cffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3d0143830 0x7ff3d0143cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46056/0 (socket says 172.21.15.98:46056) 2024-09-19T08:43:10.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.301+0000 7ff3cffff640 1 -- 172.21.15.98:0/174973450 learned_addr learned my addr 172.21.15.98:0/174973450 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:10.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.301+0000 7ff3cf7fe640 1 --2- 172.21.15.98:0/174973450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3b8003f70 0x7ff3d00cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:10.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.301+0000 7ff3ceffd640 1 --2- 172.21.15.98:0/174973450 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3d0154460 0x7ff3d01548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:10.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.301+0000 7ff3cffff640 1 -- 172.21.15.98:0/174973450 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3d0154460 msgr2=0x7ff3d01548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:10.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.301+0000 7ff3cffff640 1 --2- 172.21.15.98:0/174973450 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3d0154460 0x7ff3d01548e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:10.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.301+0000 7ff3cffff640 1 -- 172.21.15.98:0/174973450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3b8003f70 msgr2=0x7ff3d00cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:10.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.301+0000 7ff3cffff640 1 --2- 172.21.15.98:0/174973450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3b8003f70 0x7ff3d00cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:10.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.301+0000 7ff3cffff640 1 -- 172.21.15.98:0/174973450 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d01445a0 con 0x7ff3d0143830 2024-09-19T08:43:10.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.301+0000 7ff3cffff640 1 --2- 172.21.15.98:0/174973450 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3d0143830 0x7ff3d0143cb0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7ff3d00b6670 tx=0x7ff3c8010c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:10.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.301+0000 7ff3b7fff640 1 -- 172.21.15.98:0/174973450 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c801a070 con 0x7ff3d0143830 2024-09-19T08:43:10.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.301+0000 7ff3b7fff640 1 -- 172.21.15.98:0/174973450 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3c8011040 con 0x7ff3d0143830 2024-09-19T08:43:10.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.301+0000 7ff3d5e66640 1 -- 172.21.15.98:0/174973450 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3d01520c0 con 0x7ff3d0143830 2024-09-19T08:43:10.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.301+0000 7ff3b7fff640 1 -- 172.21.15.98:0/174973450 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c800ed00 con 0x7ff3d0143830 2024-09-19T08:43:10.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.301+0000 7ff3d5e66640 1 -- 172.21.15.98:0/174973450 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3d0152680 con 0x7ff3d0143830 2024-09-19T08:43:10.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.301+0000 7ff3d5e66640 1 -- 172.21.15.98:0/174973450 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3d015f900 con 0x7ff3d0143830 2024-09-19T08:43:10.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.301+0000 7ff3b7fff640 1 -- 172.21.15.98:0/174973450 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7ff3c801d070 con 0x7ff3d0143830 2024-09-19T08:43:10.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.301+0000 7ff3b7fff640 1 --2- 172.21.15.98:0/174973450 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff3bc067a10 0x7ff3bc069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:10.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.301+0000 7ff3b7fff640 1 -- 172.21.15.98:0/174973450 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5502+0+0 (secure 0 0 0) 0x7ff3c8088320 con 0x7ff3d0143830 2024-09-19T08:43:10.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.301+0000 7ff3cf7fe640 1 --2- 172.21.15.98:0/174973450 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff3bc067a10 0x7ff3bc069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:10.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.301+0000 7ff3cf7fe640 1 --2- 172.21.15.98:0/174973450 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff3bc067a10 0x7ff3bc069ed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4009a30 tx=0x7ff3c4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:10.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.305+0000 7ff3b7fff640 1 -- 172.21.15.98:0/174973450 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff3c8058e20 con 0x7ff3d0143830 2024-09-19T08:43:10.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.469+0000 7ff3d5e66640 1 -- 172.21.15.98:0/174973450 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff3d0161b20 con 0x7ff3d0143830 2024-09-19T08:43:10.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.469+0000 7ff3b7fff640 1 -- 172.21.15.98:0/174973450 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+13845 (secure 0 0 0) 0x7ff3c8058720 con 0x7ff3d0143830 2024-09-19T08:43:10.471 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:10.471 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":74,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:43:09.128762+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.98:0/1649589780":"2024-09-20T08:43:02.843944+0000","172.21.15.98:6833/2040310606":"2024-09-20T08:43:02.843944+0000","172.21.15.98:6832/2040310606":"2024-09-20T08:43:02.843944+0000","172.21.15.98:0/1028111778":"2024-09-20T08:43:02.843944+0000","172.21.15.98:0/2085575004":"2024-09-20T08:43:02.843944+0000","172.21.15.98:0/1465974372":"2024-09-20T08:43:02.843944+0000","172.21.15.98:6837/1767881083":"2024-09-20T08:43:06.112847+0000","172.21.15.98:0/3519546751":"2024-09-20T08:43:02.843944+0000","172.21.15.98:0/1973723249":"2024-09-20T08:43:02.843944+0000","172.21.15.98:0/2131240070":"2024-09-20T08:43:02.843944+0000","172.21.15.98:6835/1767881083":"2024-09-20T08:43:06.112847+0000","172.21.15.98:0/1109131925":"2024-09-20T08:43:02.843944+0000","172.21.15.98:0/1688754871":"2024-09-20T08:43:02.843944+0000","172.21.15.98:0/1268825931":"2024-09-20T08:43:02.843944+0000","172.21.15.98:0/3957109968":"2024-09-20T08:43:02.843944+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:43:10.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.473+0000 7ff3d5e66640 1 -- 172.21.15.98:0/174973450 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff3bc067a10 msgr2=0x7ff3bc069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:10.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.473+0000 7ff3d5e66640 1 --2- 172.21.15.98:0/174973450 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff3bc067a10 0x7ff3bc069ed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4009a30 tx=0x7ff3c4008040 comp rx=0 tx=0).stop 2024-09-19T08:43:10.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.473+0000 7ff3d5e66640 1 -- 172.21.15.98:0/174973450 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3d0143830 msgr2=0x7ff3d0143cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:10.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.473+0000 7ff3d5e66640 1 --2- 172.21.15.98:0/174973450 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3d0143830 0x7ff3d0143cb0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7ff3d00b6670 tx=0x7ff3c8010c30 comp rx=0 tx=0).stop 2024-09-19T08:43:10.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.473+0000 7ff3d5e66640 1 -- 172.21.15.98:0/174973450 shutdown_connections 2024-09-19T08:43:10.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.473+0000 7ff3d5e66640 1 --2- 172.21.15.98:0/174973450 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff3bc067a10 0x7ff3bc069ed0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:10.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.473+0000 7ff3d5e66640 1 --2- 172.21.15.98:0/174973450 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3d0143830 0x7ff3d0143cb0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:10.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.473+0000 7ff3d5e66640 1 --2- 172.21.15.98:0/174973450 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff3d0154460 0x7ff3d01548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:10.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.473+0000 7ff3d5e66640 1 --2- 172.21.15.98:0/174973450 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3b8003f70 0x7ff3d00cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:10.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.473+0000 7ff3d5e66640 1 -- 172.21.15.98:0/174973450 >> 172.21.15.98:0/174973450 conn(0x7ff3d0074e40 msgr2=0x7ff3d00d36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:10.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.473+0000 7ff3d5e66640 1 -- 172.21.15.98:0/174973450 shutdown_connections 2024-09-19T08:43:10.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.473+0000 7ff3d5e66640 1 -- 172.21.15.98:0/174973450 wait complete. 2024-09-19T08:43:10.491 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T08:38:08.105684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T08:43:10.491 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T08:43:10.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.613+0000 7fd716b4a640 1 Processor -- start 2024-09-19T08:43:10.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.613+0000 7fd716b4a640 1 -- start start 2024-09-19T08:43:10.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.613+0000 7fd716b4a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd71015cbb0 0x7fd71015cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:10.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.613+0000 7fd716b4a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd71006c3a0 0x7fd71016c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:10.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.613+0000 7fd716b4a640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7100ce3c0 con 0x7fd710163930 2024-09-19T08:43:10.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.613+0000 7fd716b4a640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fd7100cd090 con 0x7fd71006c3a0 2024-09-19T08:43:10.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.613+0000 7fd716b4a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fd7100cd370 con 0x7fd71015cbb0 2024-09-19T08:43:10.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.613+0000 7fd7150e7640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd71006c3a0 0x7fd71016c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:10.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.613+0000 7fd70ffff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fd710163930 0x7fd71006bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41926/0 (socket says 172.21.15.98:41926) 2024-09-19T08:43:10.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.613+0000 7fd70ffff640 1 -- 172.21.15.98:0/3482749207 learned_addr learned my addr 172.21.15.98:0/3482749207 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:10.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.613+0000 7fd7148e6640 1 --2- 172.21.15.98:0/3482749207 >> v2:172.21.15.139:3300/0 conn(0x7fd71015cbb0 0x7fd71015cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:10.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.613+0000 7fd7150e7640 1 -- 172.21.15.98:0/3482749207 >> v2:172.21.15.139:3300/0 conn(0x7fd71015cbb0 msgr2=0x7fd71015cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:10.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.613+0000 7fd7150e7640 1 --2- 172.21.15.98:0/3482749207 >> v2:172.21.15.139:3300/0 conn(0x7fd71015cbb0 0x7fd71015cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:10.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.613+0000 7fd7150e7640 1 -- 172.21.15.98:0/3482749207 >> v1:172.21.15.98:6789/0 conn(0x7fd710163930 legacy=0x7fd71006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:10.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.613+0000 7fd7150e7640 1 -- 172.21.15.98:0/3482749207 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7100cd650 con 0x7fd71006c3a0 2024-09-19T08:43:10.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.613+0000 7fd70f7fe640 1 -- 172.21.15.98:0/3482749207 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 352171988 0 0) 0x7fd7100ce3c0 con 0x7fd710163930 2024-09-19T08:43:10.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.617+0000 7fd7150e7640 1 --2- 172.21.15.98:0/3482749207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd71006c3a0 0x7fd71016c600 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fd7100684a0 tx=0x7fd70801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79c8f10b877054fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:10.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.617+0000 7fd70f7fe640 1 -- 172.21.15.98:0/3482749207 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7080b0070 con 0x7fd71006c3a0 2024-09-19T08:43:10.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.617+0000 7fd70f7fe640 1 -- 172.21.15.98:0/3482749207 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7080ab600 con 0x7fd71006c3a0 2024-09-19T08:43:10.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.617+0000 7fd70f7fe640 1 -- 172.21.15.98:0/3482749207 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7080ab9d0 con 0x7fd71006c3a0 2024-09-19T08:43:10.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.617+0000 7fd716b4a640 1 -- 172.21.15.98:0/3482749207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd71006c3a0 msgr2=0x7fd71016c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:10.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.617+0000 7fd716b4a640 1 --2- 172.21.15.98:0/3482749207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd71006c3a0 0x7fd71016c600 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fd7100684a0 tx=0x7fd70801c480 comp rx=0 tx=0).stop 2024-09-19T08:43:10.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.617+0000 7fd716b4a640 1 -- 172.21.15.98:0/3482749207 shutdown_connections 2024-09-19T08:43:10.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.617+0000 7fd716b4a640 1 --2- 172.21.15.98:0/3482749207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd71006c3a0 0x7fd71016c600 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:10.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.617+0000 7fd716b4a640 1 --2- 172.21.15.98:0/3482749207 >> v2:172.21.15.139:3300/0 conn(0x7fd71015cbb0 0x7fd71015cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:10.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.617+0000 7fd716b4a640 1 -- 172.21.15.98:0/3482749207 >> 172.21.15.98:0/3482749207 conn(0x7fd7100b2010 msgr2=0x7fd7100b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:10.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.617+0000 7fd716b4a640 1 -- 172.21.15.98:0/3482749207 shutdown_connections 2024-09-19T08:43:10.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.617+0000 7fd716b4a640 1 -- 172.21.15.98:0/3482749207 wait complete. 2024-09-19T08:43:10.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.617+0000 7fd716b4a640 1 Processor -- start 2024-09-19T08:43:10.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.617+0000 7fd716b4a640 1 -- start start 2024-09-19T08:43:10.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.617+0000 7fd716b4a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd71006c3a0 0x7fd710072760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:10.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.617+0000 7fd716b4a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd71015cbb0 0x7fd710072ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:10.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.617+0000 7fd716b4a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd710163930 0x7fd710150b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:10.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.617+0000 7fd716b4a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd7100cccf0 con 0x7fd71006c3a0 2024-09-19T08:43:10.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.617+0000 7fd716b4a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd7100afca0 con 0x7fd710163930 2024-09-19T08:43:10.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.617+0000 7fd716b4a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd7100cce70 con 0x7fd71015cbb0 2024-09-19T08:43:10.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.617+0000 7fd7148e6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd71006c3a0 0x7fd710072760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:10.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.617+0000 7fd7148e6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd71006c3a0 0x7fd710072760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46074/0 (socket says 172.21.15.98:46074) 2024-09-19T08:43:10.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.617+0000 7fd7148e6640 1 -- 172.21.15.98:0/1334241320 learned_addr learned my addr 172.21.15.98:0/1334241320 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:10.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.621+0000 7fd7150e7640 1 --2- 172.21.15.98:0/1334241320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd710163930 0x7fd710150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:10.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.621+0000 7fd7148e6640 1 -- 172.21.15.98:0/1334241320 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd71015cbb0 msgr2=0x7fd710072ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:10.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.621+0000 7fd7148e6640 1 --2- 172.21.15.98:0/1334241320 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd71015cbb0 0x7fd710072ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:10.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.621+0000 7fd7148e6640 1 -- 172.21.15.98:0/1334241320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd710163930 msgr2=0x7fd710150b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:10.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.621+0000 7fd7148e6640 1 --2- 172.21.15.98:0/1334241320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd710163930 0x7fd710150b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:10.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.621+0000 7fd7148e6640 1 -- 172.21.15.98:0/1334241320 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd71015aeb0 con 0x7fd71006c3a0 2024-09-19T08:43:10.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.621+0000 7fd7148e6640 1 --2- 172.21.15.98:0/1334241320 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd71006c3a0 0x7fd710072760 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fd70000d800 tx=0x7fd70000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:10.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.621+0000 7fd70d7fa640 1 -- 172.21.15.98:0/1334241320 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70000b660 con 0x7fd71006c3a0 2024-09-19T08:43:10.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.621+0000 7fd70d7fa640 1 -- 172.21.15.98:0/1334241320 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd700019070 con 0x7fd71006c3a0 2024-09-19T08:43:10.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.621+0000 7fd716b4a640 1 -- 172.21.15.98:0/1334241320 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd71015b190 con 0x7fd71006c3a0 2024-09-19T08:43:10.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.621+0000 7fd70d7fa640 1 -- 172.21.15.98:0/1334241320 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70000ba00 con 0x7fd71006c3a0 2024-09-19T08:43:10.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.621+0000 7fd716b4a640 1 -- 172.21.15.98:0/1334241320 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd71015b750 con 0x7fd71006c3a0 2024-09-19T08:43:10.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.621+0000 7fd716b4a640 1 -- 172.21.15.98:0/1334241320 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd71015cf90 con 0x7fd71006c3a0 2024-09-19T08:43:10.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.621+0000 7fd70d7fa640 1 -- 172.21.15.98:0/1334241320 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fd70000bba0 con 0x7fd71006c3a0 2024-09-19T08:43:10.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.621+0000 7fd70d7fa640 1 --2- 172.21.15.98:0/1334241320 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd6f00679e0 0x7fd6f0069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:10.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.621+0000 7fd70d7fa640 1 -- 172.21.15.98:0/1334241320 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5502+0+0 (secure 0 0 0) 0x7fd700086140 con 0x7fd71006c3a0 2024-09-19T08:43:10.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.621+0000 7fd70ffff640 1 --2- 172.21.15.98:0/1334241320 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd6f00679e0 0x7fd6f0069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:10.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.621+0000 7fd70ffff640 1 --2- 172.21.15.98:0/1334241320 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd6f00679e0 0x7fd6f0069ea0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd7101fe4d0 tx=0x7fd6f8007cf0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:10.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.625+0000 7fd70d7fa640 1 -- 172.21.15.98:0/1334241320 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd700056c40 con 0x7fd71006c3a0 2024-09-19T08:43:10.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.785+0000 7fd716b4a640 1 -- 172.21.15.98:0/1334241320 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd7100b2600 con 0x7fd71006c3a0 2024-09-19T08:43:10.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.789+0000 7fd70d7fa640 1 -- 172.21.15.98:0/1334241320 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) ==== 93+0+10 (secure 0 0 0) 0x7fd700056540 con 0x7fd71006c3a0 2024-09-19T08:43:10.789 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2024-09-19T08:43:10.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.789+0000 7fd716b4a640 1 -- 172.21.15.98:0/1334241320 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd6f00679e0 msgr2=0x7fd6f0069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:10.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.789+0000 7fd716b4a640 1 --2- 172.21.15.98:0/1334241320 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd6f00679e0 0x7fd6f0069ea0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd7101fe4d0 tx=0x7fd6f8007cf0 comp rx=0 tx=0).stop 2024-09-19T08:43:10.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.789+0000 7fd716b4a640 1 -- 172.21.15.98:0/1334241320 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd71006c3a0 msgr2=0x7fd710072760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:10.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.789+0000 7fd716b4a640 1 --2- 172.21.15.98:0/1334241320 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd71006c3a0 0x7fd710072760 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fd70000d800 tx=0x7fd70000dcc0 comp rx=0 tx=0).stop 2024-09-19T08:43:10.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.789+0000 7fd716b4a640 1 -- 172.21.15.98:0/1334241320 shutdown_connections 2024-09-19T08:43:10.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.789+0000 7fd716b4a640 1 --2- 172.21.15.98:0/1334241320 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd6f00679e0 0x7fd6f0069ea0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:10.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.789+0000 7fd716b4a640 1 --2- 172.21.15.98:0/1334241320 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd710163930 0x7fd710150b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:10.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.789+0000 7fd716b4a640 1 --2- 172.21.15.98:0/1334241320 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd71015cbb0 0x7fd710072ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:10.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.789+0000 7fd716b4a640 1 --2- 172.21.15.98:0/1334241320 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd71006c3a0 0x7fd710072760 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:10.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.789+0000 7fd716b4a640 1 -- 172.21.15.98:0/1334241320 >> 172.21.15.98:0/1334241320 conn(0x7fd7100b2010 msgr2=0x7fd7101641b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:10.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.789+0000 7fd716b4a640 1 -- 172.21.15.98:0/1334241320 shutdown_connections 2024-09-19T08:43:10.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.789+0000 7fd716b4a640 1 -- 172.21.15.98:0/1334241320 wait complete. 2024-09-19T08:43:10.808 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars' 2024-09-19T08:43:10.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.933+0000 7f19df1fa640 1 Processor -- start 2024-09-19T08:43:10.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.933+0000 7f19df1fa640 1 -- start start 2024-09-19T08:43:10.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.933+0000 7f19df1fa640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f19d806efa0 0x7f19d80ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:10.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.933+0000 7f19df1fa640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19d80af4a0 con 0x7f19d806f380 2024-09-19T08:43:10.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.933+0000 7f19df1fa640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19d80af780 con 0x7f19d80aefd0 2024-09-19T08:43:10.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.933+0000 7f19df1fa640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f19d815ed00 con 0x7f19d806efa0 2024-09-19T08:43:10.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.933+0000 7f19dcf96640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f19d806efa0 0x7f19d80ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:10.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.933+0000 7f19d7fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f19d806f380 0x7f19d80ae8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41940/0 (socket says 172.21.15.98:41940) 2024-09-19T08:43:10.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.933+0000 7f19d7fff640 1 -- 172.21.15.98:0/2939330744 learned_addr learned my addr 172.21.15.98:0/2939330744 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:10.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.933+0000 7f19dcf96640 1 -- 172.21.15.98:0/2939330744 >> v1:172.21.15.139:6789/0 conn(0x7f19d80aefd0 legacy=0x7f19d815b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:10.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.933+0000 7f19dcf96640 1 -- 172.21.15.98:0/2939330744 >> v1:172.21.15.98:6789/0 conn(0x7f19d806f380 legacy=0x7f19d80ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:10.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.933+0000 7f19dcf96640 1 -- 172.21.15.98:0/2939330744 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19d80afa60 con 0x7f19d806efa0 2024-09-19T08:43:10.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.937+0000 7f19dcf96640 1 --2- 172.21.15.98:0/2939330744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19d806efa0 0x7f19d80ce550 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f19c80097f0 tx=0x7f19c801c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a8f511843f10f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:10.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.937+0000 7f19d77fe640 1 -- 172.21.15.98:0/2939330744 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c80b0070 con 0x7f19d806efa0 2024-09-19T08:43:10.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.937+0000 7f19d77fe640 1 -- 172.21.15.98:0/2939330744 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19c80ab600 con 0x7f19d806efa0 2024-09-19T08:43:10.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.937+0000 7f19d77fe640 1 -- 172.21.15.98:0/2939330744 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c80ab9d0 con 0x7f19d806efa0 2024-09-19T08:43:10.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.937+0000 7f19df1fa640 1 -- 172.21.15.98:0/2939330744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19d806efa0 msgr2=0x7f19d80ce550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:10.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.937+0000 7f19df1fa640 1 --2- 172.21.15.98:0/2939330744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19d806efa0 0x7f19d80ce550 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f19c80097f0 tx=0x7f19c801c340 comp rx=0 tx=0).stop 2024-09-19T08:43:10.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.937+0000 7f19df1fa640 1 -- 172.21.15.98:0/2939330744 shutdown_connections 2024-09-19T08:43:10.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.937+0000 7f19df1fa640 1 --2- 172.21.15.98:0/2939330744 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19d806efa0 0x7f19d80ce550 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:10.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.937+0000 7f19df1fa640 1 -- 172.21.15.98:0/2939330744 >> 172.21.15.98:0/2939330744 conn(0x7f19d80d0980 msgr2=0x7f19d80d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:10.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.937+0000 7f19df1fa640 1 -- 172.21.15.98:0/2939330744 shutdown_connections 2024-09-19T08:43:10.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.937+0000 7f19df1fa640 1 -- 172.21.15.98:0/2939330744 wait complete. 2024-09-19T08:43:10.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.937+0000 7f19df1fa640 1 Processor -- start 2024-09-19T08:43:10.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.937+0000 7f19df1fa640 1 -- start start 2024-09-19T08:43:10.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.937+0000 7f19df1fa640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19d806efa0 0x7f19d80c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:10.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.937+0000 7f19df1fa640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19d80aefd0 0x7f19d8158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:10.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.937+0000 7f19df1fa640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19d81592d0 0x7f19d80c4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:10.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.937+0000 7f19df1fa640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f19d816c6b0 con 0x7f19d806efa0 2024-09-19T08:43:10.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.937+0000 7f19df1fa640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f19d816c530 con 0x7f19d81592d0 2024-09-19T08:43:10.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.937+0000 7f19df1fa640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f19d816c830 con 0x7f19d80aefd0 2024-09-19T08:43:10.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.937+0000 7f19dcf96640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19d806efa0 0x7f19d80c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:10.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.937+0000 7f19dcf96640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19d806efa0 0x7f19d80c6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46090/0 (socket says 172.21.15.98:46090) 2024-09-19T08:43:10.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.937+0000 7f19dcf96640 1 -- 172.21.15.98:0/1124866749 learned_addr learned my addr 172.21.15.98:0/1124866749 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:10.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.937+0000 7f19d7fff640 1 --2- 172.21.15.98:0/1124866749 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19d80aefd0 0x7f19d8158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:10.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.937+0000 7f19dcf96640 1 -- 172.21.15.98:0/1124866749 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19d80aefd0 msgr2=0x7f19d8158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:10.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.937+0000 7f19dcf96640 1 --2- 172.21.15.98:0/1124866749 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19d80aefd0 0x7f19d8158d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:10.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.937+0000 7f19dd797640 1 --2- 172.21.15.98:0/1124866749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19d81592d0 0x7f19d80c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:10.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.937+0000 7f19dcf96640 1 -- 172.21.15.98:0/1124866749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19d81592d0 msgr2=0x7f19d80c4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:10.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.937+0000 7f19dcf96640 1 --2- 172.21.15.98:0/1124866749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19d81592d0 0x7f19d80c4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:10.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.937+0000 7f19dcf96640 1 -- 172.21.15.98:0/1124866749 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19d8159750 con 0x7f19d806efa0 2024-09-19T08:43:10.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.937+0000 7f19dcf96640 1 --2- 172.21.15.98:0/1124866749 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19d806efa0 0x7f19d80c6f40 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f19c80ae040 tx=0x7f19c801ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:10.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.937+0000 7f19d57fa640 1 -- 172.21.15.98:0/1124866749 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c80b0050 con 0x7f19d806efa0 2024-09-19T08:43:10.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.937+0000 7f19d57fa640 1 -- 172.21.15.98:0/1124866749 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19c80bc070 con 0x7f19d806efa0 2024-09-19T08:43:10.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.937+0000 7f19d57fa640 1 -- 172.21.15.98:0/1124866749 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c80b5a50 con 0x7f19d806efa0 2024-09-19T08:43:10.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.937+0000 7f19df1fa640 1 -- 172.21.15.98:0/1124866749 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19d8159a30 con 0x7f19d806efa0 2024-09-19T08:43:10.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.941+0000 7f19df1fa640 1 -- 172.21.15.98:0/1124866749 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19d8159ff0 con 0x7f19d806efa0 2024-09-19T08:43:10.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.941+0000 7f19df1fa640 1 -- 172.21.15.98:0/1124866749 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19d80ce550 con 0x7f19d806efa0 2024-09-19T08:43:10.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.941+0000 7f19d57fa640 1 -- 172.21.15.98:0/1124866749 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f19c80ab4a0 con 0x7f19d806efa0 2024-09-19T08:43:10.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.941+0000 7f19d57fa640 1 --2- 172.21.15.98:0/1124866749 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f19c40679e0 0x7f19c4069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:10.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.941+0000 7f19d57fa640 1 -- 172.21.15.98:0/1124866749 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5502+0+0 (secure 0 0 0) 0x7f19c81205b0 con 0x7f19d806efa0 2024-09-19T08:43:10.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.941+0000 7f19d7fff640 1 --2- 172.21.15.98:0/1124866749 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f19c40679e0 0x7f19c4069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:10.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.941+0000 7f19d7fff640 1 --2- 172.21.15.98:0/1124866749 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f19c40679e0 0x7f19c4069ea0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f19cc000f70 tx=0x7f19cc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:10.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:10.945+0000 7f19d57fa640 1 -- 172.21.15.98:0/1124866749 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f19c8016050 con 0x7f19d806efa0 2024-09-19T08:43:11.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:11.169+0000 7f19df1fa640 1 -- 172.21.15.98:0/1124866749 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]} v 0) -- 0x7f19d806fee0 con 0x7f19d806efa0 2024-09-19T08:43:12.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.149+0000 7f19d57fa640 1 -- 172.21.15.98:0/1124866749 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]}]=0 v251) ==== 149+0+0 (secure 0 0 0) 0x7f19c80f1030 con 0x7f19d806efa0 2024-09-19T08:43:12.152 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.149+0000 7f19df1fa640 1 -- 172.21.15.98:0/1124866749 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f19c40679e0 msgr2=0x7f19c4069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:12.152 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.149+0000 7f19df1fa640 1 --2- 172.21.15.98:0/1124866749 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f19c40679e0 0x7f19c4069ea0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f19cc000f70 tx=0x7f19cc009040 comp rx=0 tx=0).stop 2024-09-19T08:43:12.152 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.149+0000 7f19df1fa640 1 -- 172.21.15.98:0/1124866749 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19d806efa0 msgr2=0x7f19d80c6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:12.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.149+0000 7f19df1fa640 1 --2- 172.21.15.98:0/1124866749 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19d806efa0 0x7f19d80c6f40 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f19c80ae040 tx=0x7f19c801ab80 comp rx=0 tx=0).stop 2024-09-19T08:43:12.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.149+0000 7f19df1fa640 1 -- 172.21.15.98:0/1124866749 shutdown_connections 2024-09-19T08:43:12.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.149+0000 7f19df1fa640 1 --2- 172.21.15.98:0/1124866749 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f19c40679e0 0x7f19c4069ea0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:12.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.149+0000 7f19df1fa640 1 --2- 172.21.15.98:0/1124866749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19d81592d0 0x7f19d80c4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:12.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.149+0000 7f19df1fa640 1 --2- 172.21.15.98:0/1124866749 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19d80aefd0 0x7f19d8158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:12.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.149+0000 7f19df1fa640 1 --2- 172.21.15.98:0/1124866749 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19d806efa0 0x7f19d80c6f40 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:12.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.149+0000 7f19df1fa640 1 -- 172.21.15.98:0/1124866749 >> 172.21.15.98:0/1124866749 conn(0x7f19d80d0980 msgr2=0x7f19d815cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:12.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.149+0000 7f19df1fa640 1 -- 172.21.15.98:0/1124866749 shutdown_connections 2024-09-19T08:43:12.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.149+0000 7f19df1fa640 1 -- 172.21.15.98:0/1124866749 wait complete. 2024-09-19T08:43:12.171 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T08:43:12.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.301+0000 7f1a9dbfd640 1 Processor -- start 2024-09-19T08:43:12.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.301+0000 7f1a9dbfd640 1 -- start start 2024-09-19T08:43:12.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.301+0000 7f1a9dbfd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1a98072f90 0x7f1a980ce400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:12.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.301+0000 7f1a9dbfd640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a9815c820 con 0x7f1a9815c440 2024-09-19T08:43:12.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.301+0000 7f1a9dbfd640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a98070ec0 con 0x7f1a980ce940 2024-09-19T08:43:12.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.301+0000 7f1a9dbfd640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f1a980711a0 con 0x7f1a98072f90 2024-09-19T08:43:12.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.301+0000 7f1a977fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f1a9815c440 0x7f1a980706e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41944/0 (socket says 172.21.15.98:41944) 2024-09-19T08:43:12.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.301+0000 7f1a977fe640 1 -- 172.21.15.98:0/3971439390 learned_addr learned my addr 172.21.15.98:0/3971439390 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:12.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.301+0000 7f1a96ffd640 1 --2- 172.21.15.98:0/3971439390 >> v2:172.21.15.139:3300/0 conn(0x7f1a98072f90 0x7f1a980ce400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:12.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.301+0000 7f1a967fc640 1 -- 172.21.15.98:0/3971439390 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3590063494 0 0) 0x7f1a9815c820 con 0x7f1a9815c440 2024-09-19T08:43:12.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.301+0000 7f1a967fc640 1 -- 172.21.15.98:0/3971439390 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a80003530 con 0x7f1a9815c440 2024-09-19T08:43:12.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.301+0000 7f1a967fc640 1 -- 172.21.15.98:0/3971439390 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 86863312 0 0) 0x7f1a80003530 con 0x7f1a9815c440 2024-09-19T08:43:12.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.301+0000 7f1a967fc640 1 -- 172.21.15.98:0/3971439390 >> v2:172.21.15.139:3300/0 conn(0x7f1a98072f90 msgr2=0x7f1a980ce400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:12.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.301+0000 7f1a967fc640 1 --2- 172.21.15.98:0/3971439390 >> v2:172.21.15.139:3300/0 conn(0x7f1a98072f90 0x7f1a980ce400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:12.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.301+0000 7f1a967fc640 1 -- 172.21.15.98:0/3971439390 >> v1:172.21.15.139:6789/0 conn(0x7f1a980ce940 legacy=0x7f1a980b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:12.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.301+0000 7f1a967fc640 1 -- 172.21.15.98:0/3971439390 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a98071480 con 0x7f1a9815c440 2024-09-19T08:43:12.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.301+0000 7f1a967fc640 1 -- 172.21.15.98:0/3971439390 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f1a88001680 con 0x7f1a9815c440 2024-09-19T08:43:12.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.301+0000 7f1a967fc640 1 -- 172.21.15.98:0/3971439390 >> v1:172.21.15.98:6789/0 conn(0x7f1a9815c440 legacy=0x7f1a980706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:12.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.301+0000 7f1a967fc640 1 --2- 172.21.15.98:0/3971439390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a800039c0 0x7f1a80003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:12.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.301+0000 7f1a967fc640 1 --2- 172.21.15.98:0/3971439390 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1a80004810 0x7f1a80004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:12.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.301+0000 7f1a967fc640 1 --2- 172.21.15.98:0/3971439390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a800051d0 0x7f1a8000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:12.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.301+0000 7f1a967fc640 1 -- 172.21.15.98:0/3971439390 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1a980711a0 con 0x7f1a800051d0 2024-09-19T08:43:12.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.301+0000 7f1a967fc640 1 -- 172.21.15.98:0/3971439390 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8000c070 con 0x7f1a800039c0 2024-09-19T08:43:12.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.301+0000 7f1a967fc640 1 -- 172.21.15.98:0/3971439390 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1a8000c270 con 0x7f1a80004810 2024-09-19T08:43:12.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.301+0000 7f1a97fff640 1 --2- 172.21.15.98:0/3971439390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a800051d0 0x7f1a8000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:12.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.301+0000 7f1a96ffd640 1 --2- 172.21.15.98:0/3971439390 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1a80004810 0x7f1a80004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:12.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.301+0000 7f1a977fe640 1 --2- 172.21.15.98:0/3971439390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a800039c0 0x7f1a80003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:12.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.301+0000 7f1a97fff640 1 -- 172.21.15.98:0/3971439390 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1a80004810 msgr2=0x7f1a80004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:12.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.301+0000 7f1a97fff640 1 --2- 172.21.15.98:0/3971439390 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1a80004810 0x7f1a80004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:12.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.301+0000 7f1a97fff640 1 -- 172.21.15.98:0/3971439390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a800039c0 msgr2=0x7f1a80003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:12.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.301+0000 7f1a97fff640 1 --2- 172.21.15.98:0/3971439390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a800039c0 0x7f1a80003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:12.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.301+0000 7f1a97fff640 1 -- 172.21.15.98:0/3971439390 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a80003530 con 0x7f1a800051d0 2024-09-19T08:43:12.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.301+0000 7f1a96ffd640 1 --2- 172.21.15.98:0/3971439390 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1a80004810 0x7f1a80004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:12.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.301+0000 7f1a977fe640 1 --2- 172.21.15.98:0/3971439390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a800039c0 0x7f1a80003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:12.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.305+0000 7f1a97fff640 1 -- 172.21.15.98:0/3971439390 reap_dead start 2024-09-19T08:43:12.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.305+0000 7f1a97fff640 1 --2- 172.21.15.98:0/3971439390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a800051d0 0x7f1a8000b9d0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f1a90001e70 tx=0x7f1a9001c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c7d92083e2bbb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:12.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.305+0000 7f1a967fc640 1 -- 172.21.15.98:0/3971439390 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a90001a90 con 0x7f1a800051d0 2024-09-19T08:43:12.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.305+0000 7f1a967fc640 1 -- 172.21.15.98:0/3971439390 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a90003bc0 con 0x7f1a800051d0 2024-09-19T08:43:12.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.305+0000 7f1a967fc640 1 -- 172.21.15.98:0/3971439390 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a90005020 con 0x7f1a800051d0 2024-09-19T08:43:12.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.305+0000 7f1a9dbfd640 1 -- 172.21.15.98:0/3971439390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a800051d0 msgr2=0x7f1a8000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:12.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.305+0000 7f1a9dbfd640 1 --2- 172.21.15.98:0/3971439390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a800051d0 0x7f1a8000b9d0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f1a90001e70 tx=0x7f1a9001c780 comp rx=0 tx=0).stop 2024-09-19T08:43:12.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.305+0000 7f1a9dbfd640 1 -- 172.21.15.98:0/3971439390 shutdown_connections 2024-09-19T08:43:12.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.305+0000 7f1a9dbfd640 1 --2- 172.21.15.98:0/3971439390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a800051d0 0x7f1a8000b9d0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:12.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.305+0000 7f1a9dbfd640 1 -- 172.21.15.98:0/3971439390 >> 172.21.15.98:0/3971439390 conn(0x7f1a980d23d0 msgr2=0x7f1a980d27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:12.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.305+0000 7f1a9dbfd640 1 -- 172.21.15.98:0/3971439390 shutdown_connections 2024-09-19T08:43:12.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.305+0000 7f1a9dbfd640 1 -- 172.21.15.98:0/3971439390 wait complete. 2024-09-19T08:43:12.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.305+0000 7f1a9dbfd640 1 Processor -- start 2024-09-19T08:43:12.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.305+0000 7f1a9dbfd640 1 -- start start 2024-09-19T08:43:12.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.309+0000 7f1a9dbfd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1a800051d0 0x7f1a9814d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:12.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.309+0000 7f1a9dbfd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a9815eca0 0x7f1a9815f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:12.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.309+0000 7f1a9dbfd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a980b65c0 0x7f1a980b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:12.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.309+0000 7f1a9dbfd640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1a980ce400 con 0x7f1a980b65c0 2024-09-19T08:43:12.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.309+0000 7f1a9dbfd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1a980cdee0 con 0x7f1a9815eca0 2024-09-19T08:43:12.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.309+0000 7f1a9dbfd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1a980ce580 con 0x7f1a800051d0 2024-09-19T08:43:12.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.309+0000 7f1a97fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a980b65c0 0x7f1a980b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:12.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.309+0000 7f1a97fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a980b65c0 0x7f1a980b6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46112/0 (socket says 172.21.15.98:46112) 2024-09-19T08:43:12.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.309+0000 7f1a97fff640 1 -- 172.21.15.98:0/3895865297 learned_addr learned my addr 172.21.15.98:0/3895865297 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:12.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.309+0000 7f1a977fe640 1 --2- 172.21.15.98:0/3895865297 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1a800051d0 0x7f1a9814d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:12.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.309+0000 7f1a96ffd640 1 --2- 172.21.15.98:0/3895865297 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a9815eca0 0x7f1a9815f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:12.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.309+0000 7f1a97fff640 1 -- 172.21.15.98:0/3895865297 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1a800051d0 msgr2=0x7f1a9814d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:12.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.309+0000 7f1a97fff640 1 --2- 172.21.15.98:0/3895865297 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1a800051d0 0x7f1a9814d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:12.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.309+0000 7f1a97fff640 1 -- 172.21.15.98:0/3895865297 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a9815eca0 msgr2=0x7f1a9815f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:12.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.309+0000 7f1a97fff640 1 --2- 172.21.15.98:0/3895865297 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a9815eca0 0x7f1a9815f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:12.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.309+0000 7f1a97fff640 1 -- 172.21.15.98:0/3895865297 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a980b6f80 con 0x7f1a980b65c0 2024-09-19T08:43:12.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.309+0000 7f1a97fff640 1 --2- 172.21.15.98:0/3895865297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a980b65c0 0x7f1a980b6a40 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f1a9001cc60 tx=0x7f1a80005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:12.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.309+0000 7f1a7ffff640 1 -- 172.21.15.98:0/3895865297 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a90001a90 con 0x7f1a980b65c0 2024-09-19T08:43:12.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.309+0000 7f1a7ffff640 1 -- 172.21.15.98:0/3895865297 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a90003980 con 0x7f1a980b65c0 2024-09-19T08:43:12.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.309+0000 7f1a9dbfd640 1 -- 172.21.15.98:0/3895865297 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a9815ca50 con 0x7f1a980b65c0 2024-09-19T08:43:12.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.309+0000 7f1a7ffff640 1 -- 172.21.15.98:0/3895865297 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a90007dc0 con 0x7f1a980b65c0 2024-09-19T08:43:12.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.309+0000 7f1a9dbfd640 1 -- 172.21.15.98:0/3895865297 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a980b74f0 con 0x7f1a980b65c0 2024-09-19T08:43:12.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.309+0000 7f1a9dbfd640 1 -- 172.21.15.98:0/3895865297 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a9814dd60 con 0x7f1a980b65c0 2024-09-19T08:43:12.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.309+0000 7f1a7ffff640 1 -- 172.21.15.98:0/3895865297 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f1a900a6480 con 0x7f1a980b65c0 2024-09-19T08:43:12.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.309+0000 7f1a7ffff640 1 --2- 172.21.15.98:0/3895865297 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1a840679e0 0x7f1a84069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:12.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.309+0000 7f1a7ffff640 1 -- 172.21.15.98:0/3895865297 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5502+0+0 (secure 0 0 0) 0x7f1a9011f670 con 0x7f1a980b65c0 2024-09-19T08:43:12.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.309+0000 7f1a977fe640 1 --2- 172.21.15.98:0/3895865297 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1a840679e0 0x7f1a84069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:12.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.313+0000 7f1a977fe640 1 --2- 172.21.15.98:0/3895865297 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1a840679e0 0x7f1a84069ea0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1a880017f0 tx=0x7f1a88009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:12.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.313+0000 7f1a7ffff640 1 -- 172.21.15.98:0/3895865297 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f1a900b8060 con 0x7f1a980b65c0 2024-09-19T08:43:12.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.481+0000 7f1a9dbfd640 1 -- 172.21.15.98:0/3895865297 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f1a9815a520 con 0x7f1a980b65c0 2024-09-19T08:43:12.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.485+0000 7f1a7ffff640 1 -- 172.21.15.98:0/3895865297 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f1a90016490 con 0x7f1a980b65c0 2024-09-19T08:43:12.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.485+0000 7f1a9dbfd640 1 -- 172.21.15.98:0/3895865297 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1a840679e0 msgr2=0x7f1a84069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:12.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.485+0000 7f1a9dbfd640 1 --2- 172.21.15.98:0/3895865297 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1a840679e0 0x7f1a84069ea0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1a880017f0 tx=0x7f1a88009040 comp rx=0 tx=0).stop 2024-09-19T08:43:12.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.485+0000 7f1a9dbfd640 1 -- 172.21.15.98:0/3895865297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a980b65c0 msgr2=0x7f1a980b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:12.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.485+0000 7f1a9dbfd640 1 --2- 172.21.15.98:0/3895865297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a980b65c0 0x7f1a980b6a40 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f1a9001cc60 tx=0x7f1a80005840 comp rx=0 tx=0).stop 2024-09-19T08:43:12.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.485+0000 7f1a9dbfd640 1 -- 172.21.15.98:0/3895865297 shutdown_connections 2024-09-19T08:43:12.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.485+0000 7f1a9dbfd640 1 --2- 172.21.15.98:0/3895865297 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1a840679e0 0x7f1a84069ea0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:12.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.485+0000 7f1a9dbfd640 1 --2- 172.21.15.98:0/3895865297 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1a980b65c0 0x7f1a980b6a40 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:12.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.485+0000 7f1a9dbfd640 1 --2- 172.21.15.98:0/3895865297 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1a9815eca0 0x7f1a9815f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:12.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.485+0000 7f1a9dbfd640 1 --2- 172.21.15.98:0/3895865297 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1a800051d0 0x7f1a9814d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:12.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.485+0000 7f1a9dbfd640 1 -- 172.21.15.98:0/3895865297 >> 172.21.15.98:0/3895865297 conn(0x7f1a980d23d0 msgr2=0x7f1a981594b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:12.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.485+0000 7f1a9dbfd640 1 -- 172.21.15.98:0/3895865297 shutdown_connections 2024-09-19T08:43:12.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.485+0000 7f1a9dbfd640 1 -- 172.21.15.98:0/3895865297 wait complete. 2024-09-19T08:43:12.504 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T08:43:12.505 DEBUG:teuthology.orchestra.run.smithi098:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:43:12.569 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:43:12.639 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T08:43:12.639 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:43:12.639 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:43:12.645 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15846 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T08:43:12.645 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:43:12.645 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:43:12.651 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T08:43:12.651 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:43:12.651 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:43:12.693 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15846 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T08:43:12.693 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:43:12.693 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:43:12.699 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T08:43:12.699 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-19T08:43:12.699 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:43:12.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.865+0000 7fcf005e7640 1 Processor -- start 2024-09-19T08:43:12.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.865+0000 7fcf005e7640 1 -- start start 2024-09-19T08:43:12.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.865+0000 7fcf005e7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcef815c550 0x7fcef806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:12.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.865+0000 7fcf005e7640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcef806c1d0 0x7fcef80cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:12.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.865+0000 7fcf005e7640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcef806d310 con 0x7fcef80cf900 2024-09-19T08:43:12.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.865+0000 7fcf005e7640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fcef806c610 con 0x7fcef806c1d0 2024-09-19T08:43:12.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.865+0000 7fcf005e7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcef806d5f0 con 0x7fcef815c550 2024-09-19T08:43:12.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.865+0000 7fcefdb82640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcef806c1d0 0x7fcef80cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:12.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.865+0000 7fcefeb84640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fcef80cf900 0x7fcef80ce1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41952/0 (socket says 172.21.15.98:41952) 2024-09-19T08:43:12.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.865+0000 7fcefeb84640 1 -- 172.21.15.98:0/2696702607 learned_addr learned my addr 172.21.15.98:0/2696702607 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:12.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.865+0000 7fcefdb82640 1 -- 172.21.15.98:0/2696702607 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcef815c550 msgr2=0x7fcef806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:12.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.865+0000 7fcefe383640 1 --2- 172.21.15.98:0/2696702607 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcef815c550 0x7fcef806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:12.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.865+0000 7fcefdb82640 1 --2- 172.21.15.98:0/2696702607 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcef815c550 0x7fcef806bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:12.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.865+0000 7fcefdb82640 1 -- 172.21.15.98:0/2696702607 >> v1:172.21.15.98:6789/0 conn(0x7fcef80cf900 legacy=0x7fcef80ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:12.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.865+0000 7fcefd381640 1 -- 172.21.15.98:0/2696702607 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1546575998 0 0) 0x7fcef806d310 con 0x7fcef80cf900 2024-09-19T08:43:12.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.865+0000 7fcefdb82640 1 -- 172.21.15.98:0/2696702607 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcef806d8d0 con 0x7fcef806c1d0 2024-09-19T08:43:12.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.865+0000 7fcefdb82640 1 --2- 172.21.15.98:0/2696702607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcef806c1d0 0x7fcef80cf3c0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fceec0097c0 tx=0x7fceec01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36278f45e5f2b6f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:12.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.869+0000 7fcefd381640 1 -- 172.21.15.98:0/2696702607 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceec0b0070 con 0x7fcef806c1d0 2024-09-19T08:43:12.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.869+0000 7fcefd381640 1 -- 172.21.15.98:0/2696702607 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fceec0ab600 con 0x7fcef806c1d0 2024-09-19T08:43:12.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.869+0000 7fcefd381640 1 -- 172.21.15.98:0/2696702607 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceec0ab9d0 con 0x7fcef806c1d0 2024-09-19T08:43:12.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.869+0000 7fcf005e7640 1 -- 172.21.15.98:0/2696702607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcef806c1d0 msgr2=0x7fcef80cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:12.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.869+0000 7fcf005e7640 1 --2- 172.21.15.98:0/2696702607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcef806c1d0 0x7fcef80cf3c0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fceec0097c0 tx=0x7fceec01c4a0 comp rx=0 tx=0).stop 2024-09-19T08:43:12.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.869+0000 7fcf005e7640 1 -- 172.21.15.98:0/2696702607 shutdown_connections 2024-09-19T08:43:12.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.869+0000 7fcf005e7640 1 --2- 172.21.15.98:0/2696702607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcef806c1d0 0x7fcef80cf3c0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:12.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.869+0000 7fcf005e7640 1 --2- 172.21.15.98:0/2696702607 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcef815c550 0x7fcef806bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:12.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.869+0000 7fcf005e7640 1 -- 172.21.15.98:0/2696702607 >> 172.21.15.98:0/2696702607 conn(0x7fcef80b2010 msgr2=0x7fcef80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:12.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.869+0000 7fcf005e7640 1 -- 172.21.15.98:0/2696702607 shutdown_connections 2024-09-19T08:43:12.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.869+0000 7fcf005e7640 1 -- 172.21.15.98:0/2696702607 wait complete. 2024-09-19T08:43:12.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.869+0000 7fcf005e7640 1 Processor -- start 2024-09-19T08:43:12.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.869+0000 7fcf005e7640 1 -- start start 2024-09-19T08:43:12.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.869+0000 7fcf005e7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcef806c1d0 0x7fcef81480c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:12.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.869+0000 7fcf005e7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcef80cf900 0x7fcef81533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:12.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.869+0000 7fcf005e7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcef815c550 0x7fcef8153930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:12.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.869+0000 7fcf005e7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcef8169230 con 0x7fcef815c550 2024-09-19T08:43:12.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.869+0000 7fcf005e7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcef815cb40 con 0x7fcef80cf900 2024-09-19T08:43:12.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.869+0000 7fcf005e7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcef81693b0 con 0x7fcef806c1d0 2024-09-19T08:43:12.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.869+0000 7fcefeb84640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcef815c550 0x7fcef8153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:12.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.869+0000 7fcefeb84640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcef815c550 0x7fcef8153930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46134/0 (socket says 172.21.15.98:46134) 2024-09-19T08:43:12.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.869+0000 7fcefeb84640 1 -- 172.21.15.98:0/4063771861 learned_addr learned my addr 172.21.15.98:0/4063771861 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:12.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.869+0000 7fcefe383640 1 --2- 172.21.15.98:0/4063771861 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcef806c1d0 0x7fcef81480c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:12.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.869+0000 7fcefdb82640 1 --2- 172.21.15.98:0/4063771861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcef80cf900 0x7fcef81533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:12.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.869+0000 7fcefeb84640 1 -- 172.21.15.98:0/4063771861 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcef806c1d0 msgr2=0x7fcef81480c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:12.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.869+0000 7fcefeb84640 1 --2- 172.21.15.98:0/4063771861 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcef806c1d0 0x7fcef81480c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:12.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.869+0000 7fcefeb84640 1 -- 172.21.15.98:0/4063771861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcef80cf900 msgr2=0x7fcef81533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:12.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.869+0000 7fcefeb84640 1 --2- 172.21.15.98:0/4063771861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcef80cf900 0x7fcef81533f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:12.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.869+0000 7fcefeb84640 1 -- 172.21.15.98:0/4063771861 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcef8149480 con 0x7fcef815c550 2024-09-19T08:43:12.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.869+0000 7fcefe383640 1 --2- 172.21.15.98:0/4063771861 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcef806c1d0 0x7fcef81480c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:12.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.869+0000 7fcefeb84640 1 --2- 172.21.15.98:0/4063771861 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcef815c550 0x7fcef8153930 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fcee800c7d0 tx=0x7fcee800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:12.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.869+0000 7fcef6ffd640 1 -- 172.21.15.98:0/4063771861 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee8018070 con 0x7fcef815c550 2024-09-19T08:43:12.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.869+0000 7fcef6ffd640 1 -- 172.21.15.98:0/4063771861 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcee800a640 con 0x7fcef815c550 2024-09-19T08:43:12.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.869+0000 7fcf005e7640 1 -- 172.21.15.98:0/4063771861 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcef8149760 con 0x7fcef815c550 2024-09-19T08:43:12.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.869+0000 7fcef6ffd640 1 -- 172.21.15.98:0/4063771861 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee800aa70 con 0x7fcef815c550 2024-09-19T08:43:12.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.869+0000 7fcf005e7640 1 -- 172.21.15.98:0/4063771861 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcef8148600 con 0x7fcef815c550 2024-09-19T08:43:12.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.873+0000 7fcf005e7640 1 -- 172.21.15.98:0/4063771861 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcef80cf3c0 con 0x7fcef815c550 2024-09-19T08:43:12.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.873+0000 7fcef6ffd640 1 -- 172.21.15.98:0/4063771861 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fcee801b070 con 0x7fcef815c550 2024-09-19T08:43:12.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.873+0000 7fcef6ffd640 1 --2- 172.21.15.98:0/4063771861 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcee40679e0 0x7fcee4069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:12.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.873+0000 7fcef6ffd640 1 -- 172.21.15.98:0/4063771861 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5502+0+0 (secure 0 0 0) 0x7fcee8085410 con 0x7fcef815c550 2024-09-19T08:43:12.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.873+0000 7fcefe383640 1 --2- 172.21.15.98:0/4063771861 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcee40679e0 0x7fcee4069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:12.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.873+0000 7fcefe383640 1 --2- 172.21.15.98:0/4063771861 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcee40679e0 0x7fcee4069ea0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcef0006fd0 tx=0x7fcef0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:12.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:12.877+0000 7fcef6ffd640 1 -- 172.21.15.98:0/4063771861 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fcee8055e80 con 0x7fcef815c550 2024-09-19T08:43:13.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.073+0000 7fcf005e7640 1 -- 172.21.15.98:0/4063771861 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcef80b29f0 con 0x7fcef815c550 2024-09-19T08:43:13.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.073+0000 7fcef6ffd640 1 -- 172.21.15.98:0/4063771861 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+3914 (secure 0 0 0) 0x7fcee8055780 con 0x7fcef815c550 2024-09-19T08:43:13.077 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:13.077 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":41,"btime":"2024-09-19T08:43:07:114997+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5548,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3092385040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3092385040},{"type":"v1","addr":"172.21.15.139:6833","nonce":3092385040}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35},{"gid":5551,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3517671290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":3517671290},{"type":"v1","addr":"172.21.15.139:6835","nonce":3517671290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35},{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6836/965821274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":965821274},{"type":"v1","addr":"172.21.15.98:6836","nonce":965821274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35},{"gid":6540,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1335342246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1335342246},{"type":"v1","addr":"172.21.15.98:6833","nonce":1335342246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[]} 2024-09-19T08:43:13.078 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 41 2024-09-19T08:43:13.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.077+0000 7fcf005e7640 1 -- 172.21.15.98:0/4063771861 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcee40679e0 msgr2=0x7fcee4069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:13.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.077+0000 7fcf005e7640 1 --2- 172.21.15.98:0/4063771861 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcee40679e0 0x7fcee4069ea0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcef0006fd0 tx=0x7fcef0008040 comp rx=0 tx=0).stop 2024-09-19T08:43:13.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.077+0000 7fcf005e7640 1 -- 172.21.15.98:0/4063771861 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcef815c550 msgr2=0x7fcef8153930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:13.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.077+0000 7fcf005e7640 1 --2- 172.21.15.98:0/4063771861 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcef815c550 0x7fcef8153930 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fcee800c7d0 tx=0x7fcee800cc90 comp rx=0 tx=0).stop 2024-09-19T08:43:13.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.077+0000 7fcf005e7640 1 -- 172.21.15.98:0/4063771861 shutdown_connections 2024-09-19T08:43:13.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.077+0000 7fcf005e7640 1 --2- 172.21.15.98:0/4063771861 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcee40679e0 0x7fcee4069ea0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:13.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.077+0000 7fcf005e7640 1 --2- 172.21.15.98:0/4063771861 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcef815c550 0x7fcef8153930 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:13.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.077+0000 7fcf005e7640 1 --2- 172.21.15.98:0/4063771861 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcef80cf900 0x7fcef81533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:13.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.077+0000 7fcf005e7640 1 --2- 172.21.15.98:0/4063771861 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcef806c1d0 0x7fcef81480c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:13.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.077+0000 7fcf005e7640 1 -- 172.21.15.98:0/4063771861 >> 172.21.15.98:0/4063771861 conn(0x7fcef80b2010 msgr2=0x7fcef80b0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:13.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.077+0000 7fcf005e7640 1 -- 172.21.15.98:0/4063771861 shutdown_connections 2024-09-19T08:43:13.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.077+0000 7fcf005e7640 1 -- 172.21.15.98:0/4063771861 wait complete. 2024-09-19T08:43:13.092 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-19T08:43:13.092 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-19T08:43:13.092 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-19T08:43:13.093 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:43:13.093 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-19T08:43:13.093 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-19T08:43:13.093 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-19T08:43:13.093 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:43:13.093 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-19T08:43:13.093 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-19T08:43:13.094 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-19T08:43:13.094 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:43:13.094 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-19T08:43:13.094 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-19T08:43:13.094 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-19T08:43:13.094 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:43:13.103 INFO:tasks.ceph.mds.a:Stopped 2024-09-19T08:43:13.103 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-19T08:43:13.104 INFO:tasks.ceph.mds.c:Stopped 2024-09-19T08:43:13.104 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-19T08:43:13.112 INFO:tasks.ceph.mds.d:Stopped 2024-09-19T08:43:13.112 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-19T08:43:13.113 INFO:tasks.ceph.mds.b:Stopped 2024-09-19T08:43:13.113 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-19T08:43:13.145 INFO:tasks.ceph.mds.a:Started 2024-09-19T08:43:13.146 DEBUG:teuthology.parallel:result is None 2024-09-19T08:43:13.146 INFO:tasks.ceph.mds.c:Started 2024-09-19T08:43:13.146 DEBUG:teuthology.parallel:result is None 2024-09-19T08:43:13.154 INFO:tasks.ceph.mds.d:Started 2024-09-19T08:43:13.154 DEBUG:teuthology.parallel:result is None 2024-09-19T08:43:13.154 INFO:tasks.ceph.mds.b:Started 2024-09-19T08:43:13.155 DEBUG:teuthology.parallel:result is None 2024-09-19T08:43:13.155 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-19T08:43:13.206 INFO:tasks.ceph.mds.c.smithi098.stdout:starting mds.c at 2024-09-19T08:43:13.210 INFO:tasks.ceph.mds.a.smithi098.stdout:starting mds.a at 2024-09-19T08:43:13.223 INFO:tasks.ceph.mds.d.smithi139.stdout:starting mds.d at 2024-09-19T08:43:13.225 INFO:tasks.ceph.mds.b.smithi139.stdout:starting mds.b at 2024-09-19T08:43:13.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.269+0000 7fc82008a640 1 Processor -- start 2024-09-19T08:43:13.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.269+0000 7fc82008a640 1 -- start start 2024-09-19T08:43:13.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.269+0000 7fc82008a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc81815dee0 0x7fc8180afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:13.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.269+0000 7fc82008a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc818071c30 0x7fc8180cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:13.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.269+0000 7fc82008a640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8180cffe0 con 0x7fc8180cef60 2024-09-19T08:43:13.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.269+0000 7fc82008a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fc8180cf460 con 0x7fc818071c30 2024-09-19T08:43:13.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.269+0000 7fc82008a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc8180cf740 con 0x7fc81815dee0 2024-09-19T08:43:13.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.269+0000 7fc81d625640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fc8180cef60 0x7fc818071520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41984/0 (socket says 172.21.15.98:41984) 2024-09-19T08:43:13.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.269+0000 7fc81d625640 1 -- 172.21.15.98:0/1872490063 learned_addr learned my addr 172.21.15.98:0/1872490063 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:13.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.269+0000 7fc81e627640 1 --2- 172.21.15.98:0/1872490063 >> v2:172.21.15.139:3300/0 conn(0x7fc818071c30 0x7fc8180cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:13.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.269+0000 7fc81de26640 1 --2- 172.21.15.98:0/1872490063 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc81815dee0 0x7fc8180afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:13.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.269+0000 7fc81ce24640 1 -- 172.21.15.98:0/1872490063 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3790330644 0 0) 0x7fc8180cffe0 con 0x7fc8180cef60 2024-09-19T08:43:13.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.269+0000 7fc81ce24640 1 -- 172.21.15.98:0/1872490063 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc804003530 con 0x7fc8180cef60 2024-09-19T08:43:13.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.269+0000 7fc81e627640 1 -- 172.21.15.98:0/1872490063 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc81815dee0 msgr2=0x7fc8180afaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:13.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.269+0000 7fc81e627640 1 --2- 172.21.15.98:0/1872490063 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc81815dee0 0x7fc8180afaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:13.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.269+0000 7fc81e627640 1 -- 172.21.15.98:0/1872490063 >> v1:172.21.15.98:6789/0 conn(0x7fc8180cef60 legacy=0x7fc818071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:13.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.269+0000 7fc81e627640 1 -- 172.21.15.98:0/1872490063 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8180cfa20 con 0x7fc818071c30 2024-09-19T08:43:13.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.269+0000 7fc81de26640 1 --2- 172.21.15.98:0/1872490063 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc81815dee0 0x7fc8180afaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:43:13.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.269+0000 7fc81e627640 1 --2- 172.21.15.98:0/1872490063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc818071c30 0x7fc8180cb3b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc810009dc0 tx=0x7fc81001c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64963fd8761bed81 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:13.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.273+0000 7fc81ce24640 1 -- 172.21.15.98:0/1872490063 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8100b2070 con 0x7fc818071c30 2024-09-19T08:43:13.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.273+0000 7fc81ce24640 1 -- 172.21.15.98:0/1872490063 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8100a5790 con 0x7fc818071c30 2024-09-19T08:43:13.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.273+0000 7fc81ce24640 1 -- 172.21.15.98:0/1872490063 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8100a5b60 con 0x7fc818071c30 2024-09-19T08:43:13.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.273+0000 7fc82008a640 1 -- 172.21.15.98:0/1872490063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc818071c30 msgr2=0x7fc8180cb3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:13.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.273+0000 7fc82008a640 1 --2- 172.21.15.98:0/1872490063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc818071c30 0x7fc8180cb3b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc810009dc0 tx=0x7fc81001c950 comp rx=0 tx=0).stop 2024-09-19T08:43:13.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.273+0000 7fc82008a640 1 -- 172.21.15.98:0/1872490063 shutdown_connections 2024-09-19T08:43:13.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.273+0000 7fc82008a640 1 --2- 172.21.15.98:0/1872490063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc818071c30 0x7fc8180cb3b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:13.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.273+0000 7fc82008a640 1 --2- 172.21.15.98:0/1872490063 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc81815dee0 0x7fc8180afaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:13.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.273+0000 7fc82008a640 1 -- 172.21.15.98:0/1872490063 >> 172.21.15.98:0/1872490063 conn(0x7fc8180d0980 msgr2=0x7fc8180d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:13.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.273+0000 7fc82008a640 1 -- 172.21.15.98:0/1872490063 shutdown_connections 2024-09-19T08:43:13.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.273+0000 7fc82008a640 1 -- 172.21.15.98:0/1872490063 wait complete. 2024-09-19T08:43:13.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.273+0000 7fc82008a640 1 Processor -- start 2024-09-19T08:43:13.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.273+0000 7fc82008a640 1 -- start start 2024-09-19T08:43:13.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.273+0000 7fc82008a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc818071c30 0x7fc8180c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:13.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.273+0000 7fc82008a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8180cef60 0x7fc8180ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:13.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.273+0000 7fc82008a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc81815dee0 0x7fc8180aee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:13.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.273+0000 7fc82008a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc8180cbc80 con 0x7fc818071c30 2024-09-19T08:43:13.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.273+0000 7fc82008a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc8180cbb00 con 0x7fc8180cef60 2024-09-19T08:43:13.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.273+0000 7fc82008a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc8180cbe00 con 0x7fc81815dee0 2024-09-19T08:43:13.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.273+0000 7fc81de26640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc818071c30 0x7fc8180c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:13.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.273+0000 7fc81de26640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc818071c30 0x7fc8180c66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46162/0 (socket says 172.21.15.98:46162) 2024-09-19T08:43:13.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.273+0000 7fc81de26640 1 -- 172.21.15.98:0/1891319408 learned_addr learned my addr 172.21.15.98:0/1891319408 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:13.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.273+0000 7fc81d625640 1 --2- 172.21.15.98:0/1891319408 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8180cef60 0x7fc8180ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:13.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.273+0000 7fc81e627640 1 --2- 172.21.15.98:0/1891319408 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc81815dee0 0x7fc8180aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:13.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.273+0000 7fc81de26640 1 -- 172.21.15.98:0/1891319408 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc81815dee0 msgr2=0x7fc8180aee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:13.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.273+0000 7fc81de26640 1 --2- 172.21.15.98:0/1891319408 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc81815dee0 0x7fc8180aee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:13.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.273+0000 7fc81de26640 1 -- 172.21.15.98:0/1891319408 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8180cef60 msgr2=0x7fc8180ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:13.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.273+0000 7fc81de26640 1 --2- 172.21.15.98:0/1891319408 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8180cef60 0x7fc8180ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:13.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.273+0000 7fc81de26640 1 -- 172.21.15.98:0/1891319408 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8180b1160 con 0x7fc818071c30 2024-09-19T08:43:13.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.273+0000 7fc81e627640 1 --2- 172.21.15.98:0/1891319408 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc81815dee0 0x7fc8180aee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:13.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.273+0000 7fc81de26640 1 --2- 172.21.15.98:0/1891319408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc818071c30 0x7fc8180c66b0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fc81400e980 tx=0x7fc81400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:13.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.273+0000 7fc80e7fc640 1 -- 172.21.15.98:0/1891319408 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc814017070 con 0x7fc818071c30 2024-09-19T08:43:13.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.273+0000 7fc80e7fc640 1 -- 172.21.15.98:0/1891319408 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc81400ca00 con 0x7fc818071c30 2024-09-19T08:43:13.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.273+0000 7fc82008a640 1 -- 172.21.15.98:0/1891319408 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc818167970 con 0x7fc818071c30 2024-09-19T08:43:13.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.273+0000 7fc80e7fc640 1 -- 172.21.15.98:0/1891319408 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81400cdd0 con 0x7fc818071c30 2024-09-19T08:43:13.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.273+0000 7fc82008a640 1 -- 172.21.15.98:0/1891319408 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8180b1440 con 0x7fc818071c30 2024-09-19T08:43:13.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.277+0000 7fc82008a640 1 -- 172.21.15.98:0/1891319408 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8180cb3b0 con 0x7fc818071c30 2024-09-19T08:43:13.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.277+0000 7fc80e7fc640 1 -- 172.21.15.98:0/1891319408 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fc81401a070 con 0x7fc818071c30 2024-09-19T08:43:13.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.277+0000 7fc80e7fc640 1 --2- 172.21.15.98:0/1891319408 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc7fc0679e0 0x7fc7fc069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:13.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.277+0000 7fc80e7fc640 1 -- 172.21.15.98:0/1891319408 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5502+0+0 (secure 0 0 0) 0x7fc814085fa0 con 0x7fc818071c30 2024-09-19T08:43:13.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.277+0000 7fc81d625640 1 --2- 172.21.15.98:0/1891319408 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc7fc0679e0 0x7fc7fc069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:13.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.277+0000 7fc81d625640 1 --2- 172.21.15.98:0/1891319408 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc7fc0679e0 0x7fc7fc069ea0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc818140c60 tx=0x7fc808009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:13.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.281+0000 7fc80e7fc640 1 -- 172.21.15.98:0/1891319408 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc814056b30 con 0x7fc818071c30 2024-09-19T08:43:13.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:13.453+0000 7fc82008a640 1 -- 172.21.15.98:0/1891319408 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fc8180d05e0 con 0x7fc818071c30 2024-09-19T08:43:14.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.517+0000 7fc80e7fc640 1 -- 172.21.15.98:0/1891319408 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v79) ==== 97+0+0 (secure 0 0 0) 0x7fc814056430 con 0x7fc818071c30 2024-09-19T08:43:14.519 INFO:teuthology.orchestra.run.smithi098.stderr: removed all blocklist entries 2024-09-19T08:43:14.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.521+0000 7fc82008a640 1 -- 172.21.15.98:0/1891319408 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc7fc0679e0 msgr2=0x7fc7fc069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:14.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.521+0000 7fc82008a640 1 --2- 172.21.15.98:0/1891319408 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc7fc0679e0 0x7fc7fc069ea0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc818140c60 tx=0x7fc808009040 comp rx=0 tx=0).stop 2024-09-19T08:43:14.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.521+0000 7fc82008a640 1 -- 172.21.15.98:0/1891319408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc818071c30 msgr2=0x7fc8180c66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:14.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.521+0000 7fc82008a640 1 --2- 172.21.15.98:0/1891319408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc818071c30 0x7fc8180c66b0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fc81400e980 tx=0x7fc81400ee40 comp rx=0 tx=0).stop 2024-09-19T08:43:14.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.521+0000 7fc82008a640 1 -- 172.21.15.98:0/1891319408 shutdown_connections 2024-09-19T08:43:14.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.521+0000 7fc82008a640 1 --2- 172.21.15.98:0/1891319408 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc7fc0679e0 0x7fc7fc069ea0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:14.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.521+0000 7fc82008a640 1 --2- 172.21.15.98:0/1891319408 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc81815dee0 0x7fc8180aee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:14.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.521+0000 7fc82008a640 1 --2- 172.21.15.98:0/1891319408 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8180cef60 0x7fc8180ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:14.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.521+0000 7fc82008a640 1 --2- 172.21.15.98:0/1891319408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc818071c30 0x7fc8180c66b0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:14.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.521+0000 7fc82008a640 1 -- 172.21.15.98:0/1891319408 >> 172.21.15.98:0/1891319408 conn(0x7fc8180d0980 msgr2=0x7fc8180b6120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:14.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.521+0000 7fc82008a640 1 -- 172.21.15.98:0/1891319408 shutdown_connections 2024-09-19T08:43:14.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.521+0000 7fc82008a640 1 -- 172.21.15.98:0/1891319408 wait complete. 2024-09-19T08:43:14.548 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-19T08:43:14.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.673+0000 7f1745a3d640 1 Processor -- start 2024-09-19T08:43:14.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.673+0000 7f1745a3d640 1 -- start start 2024-09-19T08:43:14.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.673+0000 7f1745a3d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f174006bc90 0x7f174006c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:14.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.673+0000 7f1745a3d640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17400ccc60 con 0x7f17400ce840 2024-09-19T08:43:14.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.673+0000 7f1745a3d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17400ccf40 con 0x7f1740158fd0 2024-09-19T08:43:14.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.673+0000 7f1745a3d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f17401593b0 con 0x7f174006bc90 2024-09-19T08:43:14.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.673+0000 7f173f7fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f17400ce840 0x7f17400cec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41994/0 (socket says 172.21.15.98:41994) 2024-09-19T08:43:14.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.673+0000 7f173f7fe640 1 -- 172.21.15.98:0/3217974233 learned_addr learned my addr 172.21.15.98:0/3217974233 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:14.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.673+0000 7f173e7fc640 1 --2- 172.21.15.98:0/3217974233 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f174006bc90 0x7f174006c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:14.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.673+0000 7f173dffb640 1 -- 172.21.15.98:0/3217974233 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3337123264 0 0) 0x7f17400ccc60 con 0x7f17400ce840 2024-09-19T08:43:14.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.673+0000 7f173dffb640 1 -- 172.21.15.98:0/3217974233 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f172c005740 con 0x7f17400ce840 2024-09-19T08:43:14.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.673+0000 7f173dffb640 1 -- 172.21.15.98:0/3217974233 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1507864664 0 0) 0x7f172c005740 con 0x7f17400ce840 2024-09-19T08:43:14.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.673+0000 7f173dffb640 1 -- 172.21.15.98:0/3217974233 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f174006bc90 msgr2=0x7f174006c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:14.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.673+0000 7f173dffb640 1 --2- 172.21.15.98:0/3217974233 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f174006bc90 0x7f174006c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:14.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.673+0000 7f173dffb640 1 -- 172.21.15.98:0/3217974233 >> v1:172.21.15.139:6789/0 conn(0x7f1740158fd0 legacy=0x7f17400b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:14.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.673+0000 7f173dffb640 1 -- 172.21.15.98:0/3217974233 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17400cd220 con 0x7f17400ce840 2024-09-19T08:43:14.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.673+0000 7f173dffb640 1 -- 172.21.15.98:0/3217974233 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f1730001680 con 0x7f17400ce840 2024-09-19T08:43:14.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.673+0000 7f173dffb640 1 -- 172.21.15.98:0/3217974233 >> v1:172.21.15.98:6789/0 conn(0x7f17400ce840 legacy=0x7f17400cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:14.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.673+0000 7f173dffb640 1 --2- 172.21.15.98:0/3217974233 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f172c005c00 0x7f172c006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:14.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.673+0000 7f173dffb640 1 --2- 172.21.15.98:0/3217974233 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f174006bc90 0x7f172c006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:14.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.673+0000 7f173dffb640 1 --2- 172.21.15.98:0/3217974233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f172c006f60 0x7f172c0073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:14.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.673+0000 7f173dffb640 1 -- 172.21.15.98:0/3217974233 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f17401593b0 con 0x7f172c006f60 2024-09-19T08:43:14.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.673+0000 7f173dffb640 1 -- 172.21.15.98:0/3217974233 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f172c00dd40 con 0x7f172c005c00 2024-09-19T08:43:14.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.673+0000 7f173dffb640 1 -- 172.21.15.98:0/3217974233 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f172c00df40 con 0x7f174006bc90 2024-09-19T08:43:14.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.673+0000 7f173f7fe640 1 --2- 172.21.15.98:0/3217974233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f172c006f60 0x7f172c0073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:14.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.673+0000 7f173effd640 1 --2- 172.21.15.98:0/3217974233 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f172c005c00 0x7f172c006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:14.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.673+0000 7f173e7fc640 1 --2- 172.21.15.98:0/3217974233 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f174006bc90 0x7f172c006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:14.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.673+0000 7f173f7fe640 1 -- 172.21.15.98:0/3217974233 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f174006bc90 msgr2=0x7f172c006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:14.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.673+0000 7f173f7fe640 1 --2- 172.21.15.98:0/3217974233 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f174006bc90 0x7f172c006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:14.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.673+0000 7f173f7fe640 1 -- 172.21.15.98:0/3217974233 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f172c005c00 msgr2=0x7f172c006020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:14.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.673+0000 7f173f7fe640 1 --2- 172.21.15.98:0/3217974233 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f172c005c00 0x7f172c006020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:14.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.673+0000 7f173f7fe640 1 -- 172.21.15.98:0/3217974233 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f172c005740 con 0x7f172c006f60 2024-09-19T08:43:14.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.673+0000 7f173e7fc640 1 --2- 172.21.15.98:0/3217974233 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f174006bc90 0x7f172c006a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:43:14.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.677+0000 7f173f7fe640 1 --2- 172.21.15.98:0/3217974233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f172c006f60 0x7f172c0073e0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f173000c240 tx=0x7f173001f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c5b0543c62af9d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:14.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.677+0000 7f173dffb640 1 -- 172.21.15.98:0/3217974233 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1730007d20 con 0x7f172c006f60 2024-09-19T08:43:14.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.677+0000 7f173dffb640 1 -- 172.21.15.98:0/3217974233 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1730005ae0 con 0x7f172c006f60 2024-09-19T08:43:14.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.677+0000 7f173dffb640 1 -- 172.21.15.98:0/3217974233 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1730007070 con 0x7f172c006f60 2024-09-19T08:43:14.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.677+0000 7f1745a3d640 1 -- 172.21.15.98:0/3217974233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f172c006f60 msgr2=0x7f172c0073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:14.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.677+0000 7f1745a3d640 1 --2- 172.21.15.98:0/3217974233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f172c006f60 0x7f172c0073e0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f173000c240 tx=0x7f173001f3f0 comp rx=0 tx=0).stop 2024-09-19T08:43:14.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.677+0000 7f173f7fe640 1 -- 172.21.15.98:0/3217974233 reap_dead start 2024-09-19T08:43:14.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.677+0000 7f1745a3d640 1 -- 172.21.15.98:0/3217974233 shutdown_connections 2024-09-19T08:43:14.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.677+0000 7f1745a3d640 1 -- 172.21.15.98:0/3217974233 >> 172.21.15.98:0/3217974233 conn(0x7f17400b2010 msgr2=0x7f17400b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:14.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.677+0000 7f1745a3d640 1 -- 172.21.15.98:0/3217974233 shutdown_connections 2024-09-19T08:43:14.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.677+0000 7f1745a3d640 1 -- 172.21.15.98:0/3217974233 wait complete. 2024-09-19T08:43:14.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.677+0000 7f1745a3d640 1 Processor -- start 2024-09-19T08:43:14.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.677+0000 7f1745a3d640 1 -- start start 2024-09-19T08:43:14.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.677+0000 7f1745a3d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f172c006f60 0x7f1740071c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:14.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.677+0000 7f1745a3d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f174015f280 0x7f1740073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:14.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.677+0000 7f1745a3d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1740148820 0x7f1740148c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:14.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.677+0000 7f1745a3d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f17400b4530 con 0x7f1740148820 2024-09-19T08:43:14.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.677+0000 7f1745a3d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f17400cd7e0 con 0x7f174015f280 2024-09-19T08:43:14.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.677+0000 7f1745a3d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f174015c7a0 con 0x7f172c006f60 2024-09-19T08:43:14.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.681+0000 7f173f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1740148820 0x7f1740148c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:14.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.681+0000 7f173f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1740148820 0x7f1740148c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46172/0 (socket says 172.21.15.98:46172) 2024-09-19T08:43:14.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.681+0000 7f173f7fe640 1 -- 172.21.15.98:0/1828113704 learned_addr learned my addr 172.21.15.98:0/1828113704 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:14.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.681+0000 7f173effd640 1 --2- 172.21.15.98:0/1828113704 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f172c006f60 0x7f1740071c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:14.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.681+0000 7f173e7fc640 1 --2- 172.21.15.98:0/1828113704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f174015f280 0x7f1740073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:14.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.681+0000 7f173f7fe640 1 -- 172.21.15.98:0/1828113704 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f172c006f60 msgr2=0x7f1740071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:14.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.681+0000 7f173f7fe640 1 --2- 172.21.15.98:0/1828113704 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f172c006f60 0x7f1740071c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:14.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.681+0000 7f173f7fe640 1 -- 172.21.15.98:0/1828113704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f174015f280 msgr2=0x7f1740073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:14.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.681+0000 7f173f7fe640 1 --2- 172.21.15.98:0/1828113704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f174015f280 0x7f1740073210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:14.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.681+0000 7f173f7fe640 1 -- 172.21.15.98:0/1828113704 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1740072720 con 0x7f1740148820 2024-09-19T08:43:14.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.681+0000 7f173f7fe640 1 --2- 172.21.15.98:0/1828113704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1740148820 0x7f1740148c80 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f173001f220 tx=0x7f172c007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:14.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.681+0000 7f1737fff640 1 -- 172.21.15.98:0/1828113704 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1730007d20 con 0x7f1740148820 2024-09-19T08:43:14.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.681+0000 7f1737fff640 1 -- 172.21.15.98:0/1828113704 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1730000e50 con 0x7f1740148820 2024-09-19T08:43:14.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.681+0000 7f1745a3d640 1 -- 172.21.15.98:0/1828113704 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1740072170 con 0x7f1740148820 2024-09-19T08:43:14.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.681+0000 7f1737fff640 1 -- 172.21.15.98:0/1828113704 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17300049f0 con 0x7f1740148820 2024-09-19T08:43:14.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.681+0000 7f1745a3d640 1 -- 172.21.15.98:0/1828113704 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f174015cb30 con 0x7f1740148820 2024-09-19T08:43:14.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.681+0000 7f1745a3d640 1 -- 172.21.15.98:0/1828113704 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f174006c120 con 0x7f1740148820 2024-09-19T08:43:14.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.681+0000 7f1737fff640 1 -- 172.21.15.98:0/1828113704 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f1730004b90 con 0x7f1740148820 2024-09-19T08:43:14.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.681+0000 7f1737fff640 1 --2- 172.21.15.98:0/1828113704 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f17200679e0 0x7f1720069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:14.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.681+0000 7f1737fff640 1 -- 172.21.15.98:0/1828113704 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 4857+0+0 (secure 0 0 0) 0x7f1730122be0 con 0x7f1740148820 2024-09-19T08:43:14.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.681+0000 7f173effd640 1 --2- 172.21.15.98:0/1828113704 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f17200679e0 0x7f1720069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:14.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.681+0000 7f173effd640 1 --2- 172.21.15.98:0/1828113704 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f17200679e0 0x7f1720069ea0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1738005210 tx=0x7f1738009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:14.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.685+0000 7f1737fff640 1 -- 172.21.15.98:0/1828113704 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f17300c1060 con 0x7f1740148820 2024-09-19T08:43:14.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.905+0000 7f1745a3d640 1 -- 172.21.15.98:0/1828113704 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f1740159ae0 con 0x7f1740148820 2024-09-19T08:43:14.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.905+0000 7f1737fff640 1 -- 172.21.15.98:0/1828113704 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) ==== 80+0+5304 (secure 0 0 0) 0x7f1730019100 con 0x7f1740148820 2024-09-19T08:43:14.907 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:14.907 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:43:14.907 INFO:teuthology.orchestra.run.smithi098.stdout: "auth_dump": [ 2024-09-19T08:43:14.907 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:14.908 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.a", 2024-09-19T08:43:14.908 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmuujGBRAAa2k/kW68NJI6nUKx+W+G2Q==", 2024-09-19T08:43:14.908 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:14.908 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:43:14.908 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:43:14.908 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:43:14.908 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:43:14.908 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:14.908 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:14.908 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:14.908 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.b", 2024-09-19T08:43:14.908 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmFoojFBAAaOZPpa41fdjBPEn3CGsJnA==", 2024-09-19T08:43:14.909 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:14.909 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:43:14.909 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:43:14.909 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:43:14.909 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:43:14.909 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:14.909 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:14.909 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:14.909 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.c", 2024-09-19T08:43:14.909 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmcQcDDhAAF0QJMlljMH+DiHfWW+/Sog==", 2024-09-19T08:43:14.909 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:14.909 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:43:14.910 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:43:14.910 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:43:14.910 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:43:14.910 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:14.910 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:14.910 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:14.910 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.d", 2024-09-19T08:43:14.910 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmwWmqHBAAf15VcrdeHf8AzQjGsN9LoA==", 2024-09-19T08:43:14.910 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:14.910 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:43:14.910 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:43:14.910 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:43:14.910 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:43:14.910 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:14.911 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:14.911 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:14.911 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.0", 2024-09-19T08:43:14.911 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDC4utm9PjNKxAAa6SG9AE4smPiu4azRAFKvg==", 2024-09-19T08:43:14.911 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:14.911 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:43:14.911 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:43:14.911 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:43:14.911 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:14.911 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:14.911 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:14.911 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.1", 2024-09-19T08:43:14.911 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDG4utmHy6UCxAAN6hpG4DJEp81iG/KoloTvQ==", 2024-09-19T08:43:14.911 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:14.912 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:43:14.912 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:43:14.912 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:43:14.912 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:14.912 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:14.912 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:14.912 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.2", 2024-09-19T08:43:14.912 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDJ4utmUVP4GhAAJNAL30sEAseoDG2fE6YbZQ==", 2024-09-19T08:43:14.912 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:14.912 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:43:14.912 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:43:14.912 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:43:14.913 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:14.913 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:14.913 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:14.913 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.3", 2024-09-19T08:43:14.913 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDM4utmTKgoLRAAiCrPRLML+wvjOt6RhKay3w==", 2024-09-19T08:43:14.914 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:14.914 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:43:14.914 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:43:14.914 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:43:14.914 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:14.914 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:14.915 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:14.915 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.4", 2024-09-19T08:43:14.915 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDS4utmAf6bMhAAI3uENuFoXLicUeuspgFjYA==", 2024-09-19T08:43:14.915 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:14.915 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:43:14.915 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:43:14.916 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:43:14.916 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:14.916 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:14.916 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:14.916 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.5", 2024-09-19T08:43:14.916 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDW4utmRjxwEhAAF1A2GtrZXVx9BlmjHjhUDQ==", 2024-09-19T08:43:14.917 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:14.917 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:43:14.917 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:43:14.917 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:43:14.917 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:14.918 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:14.918 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:14.918 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.6", 2024-09-19T08:43:14.918 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDZ4utmSk/XIRAAROSfzGL7smR88ZkDSx5jAg==", 2024-09-19T08:43:14.918 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:14.918 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:43:14.919 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:43:14.919 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:43:14.919 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:14.919 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:14.919 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:14.919 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.7", 2024-09-19T08:43:14.920 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDc4utmXfpbMxAAqEM4I/ACFvfu/sMA8bMcrQ==", 2024-09-19T08:43:14.920 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:14.920 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:43:14.920 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:43:14.920 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:43:14.920 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:14.921 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:14.921 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:14.921 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.0", 2024-09-19T08:43:14.921 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utm0vNpIhAA4S1E8dFZ0+ibmSKhmZ+bIA==", 2024-09-19T08:43:14.921 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:14.921 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:43:14.922 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow r", 2024-09-19T08:43:14.922 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-19T08:43:14.922 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:14.922 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:14.922 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:14.922 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.1", 2024-09-19T08:43:14.922 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmx7xyKhAAC84lykPEb01MiypXKQVREQ==", 2024-09-19T08:43:14.923 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:14.923 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:43:14.923 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-19T08:43:14.923 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-19T08:43:14.923 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-19T08:43:14.923 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:14.923 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:14.924 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:14.924 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.2", 2024-09-19T08:43:14.924 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmteOQJxAAtTF2Py21ZE2WRQjeQSrkKQ==", 2024-09-19T08:43:14.924 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:14.924 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:43:14.924 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-19T08:43:14.924 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-19T08:43:14.924 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-19T08:43:14.924 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:14.924 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:14.925 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:14.925 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.3", 2024-09-19T08:43:14.925 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmKN92LxAAMqraZOWRT6IIfx5AuDWS2g==", 2024-09-19T08:43:14.925 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:14.925 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:43:14.925 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-19T08:43:14.925 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-19T08:43:14.925 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-19T08:43:14.925 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:14.925 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:14.925 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:14.925 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.admin", 2024-09-19T08:43:14.925 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC+4utmLH97KhAA1DTYW5bTrGUG+7cb8+d7gA==", 2024-09-19T08:43:14.926 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:14.926 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-09-19T08:43:14.926 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:43:14.926 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:43:14.926 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:43:14.926 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:14.926 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:14.926 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:14.926 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mgr.x", 2024-09-19T08:43:14.926 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmbKuyAhAA0/XjHUIPOIV/YvNEhB8rEQ==", 2024-09-19T08:43:14.926 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:14.926 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-09-19T08:43:14.926 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile mgr", 2024-09-19T08:43:14.926 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:43:14.927 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:14.927 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:14.927 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:14.927 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mgr.y", 2024-09-19T08:43:14.927 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC+4utmU+ZrOxAAsvcwINH6qyxfN7N+rDrkqg==", 2024-09-19T08:43:14.927 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:14.927 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-09-19T08:43:14.927 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile mgr", 2024-09-19T08:43:14.927 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:43:14.927 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:14.927 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:14.927 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-19T08:43:14.927 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:43:14.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.909+0000 7f1745a3d640 1 -- 172.21.15.98:0/1828113704 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f17200679e0 msgr2=0x7f1720069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:14.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.909+0000 7f1745a3d640 1 --2- 172.21.15.98:0/1828113704 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f17200679e0 0x7f1720069ea0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1738005210 tx=0x7f1738009040 comp rx=0 tx=0).stop 2024-09-19T08:43:14.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.909+0000 7f1745a3d640 1 -- 172.21.15.98:0/1828113704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1740148820 msgr2=0x7f1740148c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:14.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.909+0000 7f1745a3d640 1 --2- 172.21.15.98:0/1828113704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1740148820 0x7f1740148c80 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f173001f220 tx=0x7f172c007a50 comp rx=0 tx=0).stop 2024-09-19T08:43:14.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.909+0000 7f1745a3d640 1 -- 172.21.15.98:0/1828113704 shutdown_connections 2024-09-19T08:43:14.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.909+0000 7f1745a3d640 1 --2- 172.21.15.98:0/1828113704 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f17200679e0 0x7f1720069ea0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:14.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.909+0000 7f1745a3d640 1 --2- 172.21.15.98:0/1828113704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1740148820 0x7f1740148c80 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:14.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.909+0000 7f1745a3d640 1 --2- 172.21.15.98:0/1828113704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f174015f280 0x7f1740073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:14.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.909+0000 7f1745a3d640 1 --2- 172.21.15.98:0/1828113704 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f172c006f60 0x7f1740071c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:14.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.909+0000 7f1745a3d640 1 -- 172.21.15.98:0/1828113704 >> 172.21.15.98:0/1828113704 conn(0x7f17400b2010 msgr2=0x7f17400b04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:14.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.909+0000 7f1745a3d640 1 -- 172.21.15.98:0/1828113704 shutdown_connections 2024-09-19T08:43:14.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:14.909+0000 7f1745a3d640 1 -- 172.21.15.98:0/1828113704 wait complete. 2024-09-19T08:43:14.931 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-19T08:43:15.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.041+0000 7f7ebeb9f640 1 Processor -- start 2024-09-19T08:43:15.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.041+0000 7f7ebeb9f640 1 -- start start 2024-09-19T08:43:15.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.041+0000 7f7ebeb9f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7eb8158e40 0x7f7eb806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:15.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.041+0000 7f7ebeb9f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7eb806c1d0 0x7f7eb80ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:15.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.041+0000 7f7ebeb9f640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7eb80ce6e0 0x7f7eb806d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:15.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.041+0000 7f7ebeb9f640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f7eb806c610 con 0x7f7eb80ce6e0 2024-09-19T08:43:15.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.041+0000 7f7ebeb9f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f7eb80ceb90 con 0x7f7eb806c1d0 2024-09-19T08:43:15.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.041+0000 7f7ebeb9f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7eb815fcf0 con 0x7f7eb8158e40 2024-09-19T08:43:15.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.041+0000 7f7ebd13c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7eb80ce6e0 0x7f7eb806d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:15.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.041+0000 7f7ebd13c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7eb80ce6e0 0x7f7eb806d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46186/0 (socket says 172.21.15.98:46186) 2024-09-19T08:43:15.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.041+0000 7f7ebd13c640 1 -- 172.21.15.98:0/3576246432 learned_addr learned my addr 172.21.15.98:0/3576246432 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:15.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.041+0000 7f7ebc93b640 1 --2- 172.21.15.98:0/3576246432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7eb8158e40 0x7f7eb806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:15.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.041+0000 7f7eb7fff640 1 --2- 172.21.15.98:0/3576246432 >> v2:172.21.15.139:3300/0 conn(0x7f7eb806c1d0 0x7f7eb80ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:15.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.041+0000 7f7ebd13c640 1 -- 172.21.15.98:0/3576246432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7eb8158e40 msgr2=0x7f7eb806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:15.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.041+0000 7f7ebd13c640 1 --2- 172.21.15.98:0/3576246432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7eb8158e40 0x7f7eb806bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:15.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.041+0000 7f7ebd13c640 1 -- 172.21.15.98:0/3576246432 >> v2:172.21.15.139:3300/0 conn(0x7f7eb806c1d0 msgr2=0x7f7eb80ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:15.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.041+0000 7f7ebd13c640 1 --2- 172.21.15.98:0/3576246432 >> v2:172.21.15.139:3300/0 conn(0x7f7eb806c1d0 0x7f7eb80ce1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:15.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.041+0000 7f7ebd13c640 1 -- 172.21.15.98:0/3576246432 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb815ef30 con 0x7f7eb80ce6e0 2024-09-19T08:43:15.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.041+0000 7f7eb7fff640 1 --2- 172.21.15.98:0/3576246432 >> v2:172.21.15.139:3300/0 conn(0x7f7eb806c1d0 0x7f7eb80ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:15.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.041+0000 7f7ebd13c640 1 --2- 172.21.15.98:0/3576246432 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7eb80ce6e0 0x7f7eb806d170 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f7eb806f1d0 tx=0x7f7eb001f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b594e2fdad7813 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:15.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.041+0000 7f7eb77fe640 1 -- 172.21.15.98:0/3576246432 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb00b4070 con 0x7f7eb80ce6e0 2024-09-19T08:43:15.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.045+0000 7f7eb77fe640 1 -- 172.21.15.98:0/3576246432 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eb00af400 con 0x7f7eb80ce6e0 2024-09-19T08:43:15.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.045+0000 7f7eb77fe640 1 -- 172.21.15.98:0/3576246432 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb00af7d0 con 0x7f7eb80ce6e0 2024-09-19T08:43:15.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.045+0000 7f7ebeb9f640 1 -- 172.21.15.98:0/3576246432 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7eb80ce6e0 msgr2=0x7f7eb806d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:15.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.045+0000 7f7ebeb9f640 1 --2- 172.21.15.98:0/3576246432 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7eb80ce6e0 0x7f7eb806d170 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f7eb806f1d0 tx=0x7f7eb001f1e0 comp rx=0 tx=0).stop 2024-09-19T08:43:15.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.045+0000 7f7ebeb9f640 1 -- 172.21.15.98:0/3576246432 shutdown_connections 2024-09-19T08:43:15.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.045+0000 7f7ebeb9f640 1 --2- 172.21.15.98:0/3576246432 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7eb80ce6e0 0x7f7eb806d170 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:15.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.045+0000 7f7ebeb9f640 1 --2- 172.21.15.98:0/3576246432 >> v2:172.21.15.139:3300/0 conn(0x7f7eb806c1d0 0x7f7eb80ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:15.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.045+0000 7f7ebeb9f640 1 --2- 172.21.15.98:0/3576246432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7eb8158e40 0x7f7eb806bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:15.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.045+0000 7f7ebeb9f640 1 -- 172.21.15.98:0/3576246432 >> 172.21.15.98:0/3576246432 conn(0x7f7eb80b2010 msgr2=0x7f7eb80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:15.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.045+0000 7f7ebeb9f640 1 -- 172.21.15.98:0/3576246432 shutdown_connections 2024-09-19T08:43:15.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.045+0000 7f7ebeb9f640 1 -- 172.21.15.98:0/3576246432 wait complete. 2024-09-19T08:43:15.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.045+0000 7f7ebeb9f640 1 Processor -- start 2024-09-19T08:43:15.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.045+0000 7f7ebeb9f640 1 -- start start 2024-09-19T08:43:15.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.045+0000 7f7ebeb9f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7eb806c1d0 0x7f7eb80b06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:15.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.045+0000 7f7ebeb9f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7eb80ce6e0 0x7f7eb80b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:15.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.045+0000 7f7ebeb9f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7eb8158e40 0x7f7eb8148080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:15.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.045+0000 7f7ebeb9f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb815f7d0 con 0x7f7eb806c1d0 2024-09-19T08:43:15.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.045+0000 7f7ebeb9f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb815f4f0 con 0x7f7eb8158e40 2024-09-19T08:43:15.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.045+0000 7f7ebeb9f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7eb8159220 con 0x7f7eb80ce6e0 2024-09-19T08:43:15.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.045+0000 7f7ebc93b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7eb806c1d0 0x7f7eb80b06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:15.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.045+0000 7f7ebc93b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7eb806c1d0 0x7f7eb80b06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46202/0 (socket says 172.21.15.98:46202) 2024-09-19T08:43:15.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.045+0000 7f7ebc93b640 1 -- 172.21.15.98:0/453034731 learned_addr learned my addr 172.21.15.98:0/453034731 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:15.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.045+0000 7f7ebd13c640 1 --2- 172.21.15.98:0/453034731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7eb8158e40 0x7f7eb8148080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:15.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.045+0000 7f7ebc93b640 1 -- 172.21.15.98:0/453034731 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7eb80ce6e0 msgr2=0x7f7eb80b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:15.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.045+0000 7f7eb7fff640 1 --2- 172.21.15.98:0/453034731 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7eb80ce6e0 0x7f7eb80b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:15.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.045+0000 7f7ebc93b640 1 --2- 172.21.15.98:0/453034731 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7eb80ce6e0 0x7f7eb80b0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:15.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.045+0000 7f7ebc93b640 1 -- 172.21.15.98:0/453034731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7eb8158e40 msgr2=0x7f7eb8148080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:15.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.045+0000 7f7ebc93b640 1 --2- 172.21.15.98:0/453034731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7eb8158e40 0x7f7eb8148080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:15.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.045+0000 7f7ebc93b640 1 -- 172.21.15.98:0/453034731 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb80b14c0 con 0x7f7eb806c1d0 2024-09-19T08:43:15.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.045+0000 7f7ebd13c640 1 --2- 172.21.15.98:0/453034731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7eb8158e40 0x7f7eb8148080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:15.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.045+0000 7f7ebc93b640 1 --2- 172.21.15.98:0/453034731 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7eb806c1d0 0x7f7eb80b06b0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f7ea800d5f0 tx=0x7f7ea800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:15.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.045+0000 7f7eb57fa640 1 -- 172.21.15.98:0/453034731 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea80125f0 con 0x7f7eb806c1d0 2024-09-19T08:43:15.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.045+0000 7f7eb57fa640 1 -- 172.21.15.98:0/453034731 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ea8019070 con 0x7f7eb806c1d0 2024-09-19T08:43:15.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.045+0000 7f7ebeb9f640 1 -- 172.21.15.98:0/453034731 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eb80b17a0 con 0x7f7eb806c1d0 2024-09-19T08:43:15.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.045+0000 7f7eb57fa640 1 -- 172.21.15.98:0/453034731 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea80129b0 con 0x7f7eb806c1d0 2024-09-19T08:43:15.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.045+0000 7f7ebeb9f640 1 -- 172.21.15.98:0/453034731 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eb8145ca0 con 0x7f7eb806c1d0 2024-09-19T08:43:15.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.049+0000 7f7ebeb9f640 1 -- 172.21.15.98:0/453034731 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eb80b1cc0 con 0x7f7eb806c1d0 2024-09-19T08:43:15.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.049+0000 7f7eb57fa640 1 -- 172.21.15.98:0/453034731 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f7ea8012b50 con 0x7f7eb806c1d0 2024-09-19T08:43:15.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.049+0000 7f7eb57fa640 1 --2- 172.21.15.98:0/453034731 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7e9c0679e0 0x7f7e9c069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:15.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.049+0000 7f7eb57fa640 1 -- 172.21.15.98:0/453034731 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 4857+0+0 (secure 0 0 0) 0x7f7ea8085e60 con 0x7f7eb806c1d0 2024-09-19T08:43:15.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.049+0000 7f7eb7fff640 1 --2- 172.21.15.98:0/453034731 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7e9c0679e0 0x7f7e9c069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:15.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.049+0000 7f7eb7fff640 1 --2- 172.21.15.98:0/453034731 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7e9c0679e0 0x7f7e9c069ea0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0006fd0 tx=0x7f7ea0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:15.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.053+0000 7f7eb57fa640 1 -- 172.21.15.98:0/453034731 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f7ea8056c70 con 0x7f7eb806c1d0 2024-09-19T08:43:15.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.205+0000 7f7ebeb9f640 1 -- 172.21.15.98:0/453034731 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7eb80cee20 con 0x7f7eb806c1d0 2024-09-19T08:43:15.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.205+0000 7f7eb57fa640 1 -- 172.21.15.98:0/453034731 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) ==== 84+0+65 (secure 0 0 0) 0x7f7ea8056570 con 0x7f7eb806c1d0 2024-09-19T08:43:15.208 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:15.208 INFO:teuthology.orchestra.run.smithi098.stdout:[ 2024-09-19T08:43:15.209 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:15.209 INFO:teuthology.orchestra.run.smithi098.stdout: "poolnum": 1, 2024-09-19T08:43:15.209 INFO:teuthology.orchestra.run.smithi098.stdout: "poolname": ".mgr" 2024-09-19T08:43:15.209 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:15.209 INFO:teuthology.orchestra.run.smithi098.stdout:] 2024-09-19T08:43:15.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.209+0000 7f7ebeb9f640 1 -- 172.21.15.98:0/453034731 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7e9c0679e0 msgr2=0x7f7e9c069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:15.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.209+0000 7f7ebeb9f640 1 --2- 172.21.15.98:0/453034731 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7e9c0679e0 0x7f7e9c069ea0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0006fd0 tx=0x7f7ea0008040 comp rx=0 tx=0).stop 2024-09-19T08:43:15.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.209+0000 7f7ebeb9f640 1 -- 172.21.15.98:0/453034731 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7eb806c1d0 msgr2=0x7f7eb80b06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:15.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.209+0000 7f7ebeb9f640 1 --2- 172.21.15.98:0/453034731 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7eb806c1d0 0x7f7eb80b06b0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f7ea800d5f0 tx=0x7f7ea800dab0 comp rx=0 tx=0).stop 2024-09-19T08:43:15.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.209+0000 7f7ebeb9f640 1 -- 172.21.15.98:0/453034731 shutdown_connections 2024-09-19T08:43:15.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.209+0000 7f7ebeb9f640 1 --2- 172.21.15.98:0/453034731 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7e9c0679e0 0x7f7e9c069ea0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:15.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.209+0000 7f7ebeb9f640 1 --2- 172.21.15.98:0/453034731 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7eb8158e40 0x7f7eb8148080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:15.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.209+0000 7f7ebeb9f640 1 --2- 172.21.15.98:0/453034731 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7eb80ce6e0 0x7f7eb80b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:15.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.209+0000 7f7ebeb9f640 1 --2- 172.21.15.98:0/453034731 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7eb806c1d0 0x7f7eb80b06b0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:15.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.209+0000 7f7ebeb9f640 1 -- 172.21.15.98:0/453034731 >> 172.21.15.98:0/453034731 conn(0x7f7eb80b2010 msgr2=0x7f7eb8067a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:15.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.209+0000 7f7ebeb9f640 1 -- 172.21.15.98:0/453034731 shutdown_connections 2024-09-19T08:43:15.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.209+0000 7f7ebeb9f640 1 -- 172.21.15.98:0/453034731 wait complete. 2024-09-19T08:43:15.226 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T08:43:15.226 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T08:43:15.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.349+0000 7f96d3359640 1 Processor -- start 2024-09-19T08:43:15.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.349+0000 7f96d3359640 1 -- start start 2024-09-19T08:43:15.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.349+0000 7f96d3359640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f96cc15c5e0 0x7f96cc06f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:15.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.349+0000 7f96d3359640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96cc0d1490 0x7f96cc06f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:15.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.349+0000 7f96d3359640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96cc0b5d40 con 0x7f96cc073490 2024-09-19T08:43:15.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.349+0000 7f96d3359640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f96cc15c9c0 con 0x7f96cc15c5e0 2024-09-19T08:43:15.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.349+0000 7f96d3359640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f96cc0b6350 con 0x7f96cc0d1490 2024-09-19T08:43:15.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.349+0000 7f96d08f4640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f96cc073490 0x7f96cc070790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41996/0 (socket says 172.21.15.98:41996) 2024-09-19T08:43:15.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.349+0000 7f96d08f4640 1 -- 172.21.15.98:0/4140185246 learned_addr learned my addr 172.21.15.98:0/4140185246 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:15.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.349+0000 7f96d10f5640 1 --2- 172.21.15.98:0/4140185246 >> v2:172.21.15.139:3300/0 conn(0x7f96cc15c5e0 0x7f96cc06f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:15.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.349+0000 7f96d18f6640 1 --2- 172.21.15.98:0/4140185246 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96cc0d1490 0x7f96cc06f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:15.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.349+0000 7f96c3fff640 1 -- 172.21.15.98:0/4140185246 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1455303344 0 0) 0x7f96cc0b5d40 con 0x7f96cc073490 2024-09-19T08:43:15.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.349+0000 7f96c3fff640 1 -- 172.21.15.98:0/4140185246 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96b8003530 con 0x7f96cc073490 2024-09-19T08:43:15.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.349+0000 7f96c3fff640 1 -- 172.21.15.98:0/4140185246 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3156821179 0 0) 0x7f96b8003530 con 0x7f96cc073490 2024-09-19T08:43:15.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.349+0000 7f96c3fff640 1 -- 172.21.15.98:0/4140185246 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96cc0d1490 msgr2=0x7f96cc06f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:15.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.349+0000 7f96c3fff640 1 --2- 172.21.15.98:0/4140185246 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96cc0d1490 0x7f96cc06f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:15.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.349+0000 7f96c3fff640 1 -- 172.21.15.98:0/4140185246 >> v2:172.21.15.139:3300/0 conn(0x7f96cc15c5e0 msgr2=0x7f96cc06f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:15.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.349+0000 7f96c3fff640 1 --2- 172.21.15.98:0/4140185246 >> v2:172.21.15.139:3300/0 conn(0x7f96cc15c5e0 0x7f96cc06f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:15.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.349+0000 7f96c3fff640 1 -- 172.21.15.98:0/4140185246 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96cc0739c0 con 0x7f96cc073490 2024-09-19T08:43:15.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.349+0000 7f96c3fff640 1 -- 172.21.15.98:0/4140185246 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f96bc001680 con 0x7f96cc073490 2024-09-19T08:43:15.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.349+0000 7f96c3fff640 1 -- 172.21.15.98:0/4140185246 >> v1:172.21.15.98:6789/0 conn(0x7f96cc073490 legacy=0x7f96cc070790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:15.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.349+0000 7f96c3fff640 1 --2- 172.21.15.98:0/4140185246 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96b80039c0 0x7f96b8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:15.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.349+0000 7f96c3fff640 1 --2- 172.21.15.98:0/4140185246 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96cc0d1490 0x7f96b8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:15.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.349+0000 7f96c3fff640 1 --2- 172.21.15.98:0/4140185246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96b8004d50 0x7f96b80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:15.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.349+0000 7f96c3fff640 1 -- 172.21.15.98:0/4140185246 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc15c9c0 con 0x7f96b80039c0 2024-09-19T08:43:15.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.349+0000 7f96c3fff640 1 -- 172.21.15.98:0/4140185246 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc0b6350 con 0x7f96b8004d50 2024-09-19T08:43:15.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.349+0000 7f96c3fff640 1 -- 172.21.15.98:0/4140185246 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f96b800b9d0 con 0x7f96cc0d1490 2024-09-19T08:43:15.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.349+0000 7f96d10f5640 1 --2- 172.21.15.98:0/4140185246 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96b80039c0 0x7f96b8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:15.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.349+0000 7f96d08f4640 1 --2- 172.21.15.98:0/4140185246 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96cc0d1490 0x7f96b8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:15.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.349+0000 7f96d10f5640 1 -- 172.21.15.98:0/4140185246 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96cc0d1490 msgr2=0x7f96b8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:15.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.349+0000 7f96d10f5640 1 --2- 172.21.15.98:0/4140185246 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96cc0d1490 0x7f96b8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:15.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.349+0000 7f96d10f5640 1 -- 172.21.15.98:0/4140185246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96b8004d50 msgr2=0x7f96b80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:15.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.349+0000 7f96d10f5640 1 --2- 172.21.15.98:0/4140185246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96b8004d50 0x7f96b80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:15.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.349+0000 7f96d10f5640 1 -- 172.21.15.98:0/4140185246 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b8003530 con 0x7f96b80039c0 2024-09-19T08:43:15.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.353+0000 7f96d10f5640 1 --2- 172.21.15.98:0/4140185246 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96b80039c0 0x7f96b8003de0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f96c4009fc0 tx=0x7f96c401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a3a380bffc844 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:15.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.353+0000 7f96c3fff640 1 -- 172.21.15.98:0/4140185246 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c40b2070 con 0x7f96b80039c0 2024-09-19T08:43:15.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.353+0000 7f96c3fff640 1 -- 172.21.15.98:0/4140185246 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96c40a5ba0 con 0x7f96b80039c0 2024-09-19T08:43:15.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.353+0000 7f96c3fff640 1 -- 172.21.15.98:0/4140185246 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c4009c10 con 0x7f96b80039c0 2024-09-19T08:43:15.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.353+0000 7f96d3359640 1 -- 172.21.15.98:0/4140185246 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96b80039c0 msgr2=0x7f96b8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:15.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.353+0000 7f96d3359640 1 --2- 172.21.15.98:0/4140185246 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96b80039c0 0x7f96b8003de0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f96c4009fc0 tx=0x7f96c401ccc0 comp rx=0 tx=0).stop 2024-09-19T08:43:15.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.353+0000 7f96d18f6640 1 -- 172.21.15.98:0/4140185246 reap_dead start 2024-09-19T08:43:15.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.353+0000 7f96d3359640 1 -- 172.21.15.98:0/4140185246 shutdown_connections 2024-09-19T08:43:15.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.353+0000 7f96d3359640 1 -- 172.21.15.98:0/4140185246 >> 172.21.15.98:0/4140185246 conn(0x7f96cc0cedf0 msgr2=0x7f96cc0cf200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:15.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.353+0000 7f96d3359640 1 -- 172.21.15.98:0/4140185246 shutdown_connections 2024-09-19T08:43:15.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.353+0000 7f96d3359640 1 -- 172.21.15.98:0/4140185246 wait complete. 2024-09-19T08:43:15.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.353+0000 7f96d3359640 1 Processor -- start 2024-09-19T08:43:15.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.353+0000 7f96d3359640 1 -- start start 2024-09-19T08:43:15.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.357+0000 7f96d3359640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96b80039c0 0x7f96cc155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:15.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.357+0000 7f96d3359640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96cc1643b0 0x7f96cc1611a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:15.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.357+0000 7f96d3359640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96cc15a2a0 0x7f96cc15a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:15.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.357+0000 7f96d3359640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc1670b0 con 0x7f96cc15a2a0 2024-09-19T08:43:15.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.357+0000 7f96d3359640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc06fcd0 con 0x7f96cc1643b0 2024-09-19T08:43:15.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.357+0000 7f96d3359640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f96cc167230 con 0x7f96b80039c0 2024-09-19T08:43:15.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.357+0000 7f96d18f6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96cc15a2a0 0x7f96cc15a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:15.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.357+0000 7f96d18f6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96cc15a2a0 0x7f96cc15a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46224/0 (socket says 172.21.15.98:46224) 2024-09-19T08:43:15.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.357+0000 7f96d18f6640 1 -- 172.21.15.98:0/991526810 learned_addr learned my addr 172.21.15.98:0/991526810 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:15.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.357+0000 7f96d08f4640 1 --2- 172.21.15.98:0/991526810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96cc1643b0 0x7f96cc1611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:15.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.357+0000 7f96d10f5640 1 --2- 172.21.15.98:0/991526810 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96b80039c0 0x7f96cc155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:15.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.357+0000 7f96d18f6640 1 -- 172.21.15.98:0/991526810 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96b80039c0 msgr2=0x7f96cc155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:15.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.357+0000 7f96d18f6640 1 --2- 172.21.15.98:0/991526810 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96b80039c0 0x7f96cc155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:15.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.357+0000 7f96d18f6640 1 -- 172.21.15.98:0/991526810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96cc1643b0 msgr2=0x7f96cc1611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:15.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.357+0000 7f96d18f6640 1 --2- 172.21.15.98:0/991526810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96cc1643b0 0x7f96cc1611a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:15.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.357+0000 7f96d18f6640 1 -- 172.21.15.98:0/991526810 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96cc161c40 con 0x7f96cc15a2a0 2024-09-19T08:43:15.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.357+0000 7f96d08f4640 1 --2- 172.21.15.98:0/991526810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96cc1643b0 0x7f96cc1611a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:15.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.357+0000 7f96d18f6640 1 --2- 172.21.15.98:0/991526810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96cc15a2a0 0x7f96cc15a720 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f96cc0726f0 tx=0x7f96c800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:15.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.357+0000 7f96c1ffb640 1 -- 172.21.15.98:0/991526810 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c8017070 con 0x7f96cc15a2a0 2024-09-19T08:43:15.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.357+0000 7f96c1ffb640 1 -- 172.21.15.98:0/991526810 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96c800c620 con 0x7f96cc15a2a0 2024-09-19T08:43:15.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.357+0000 7f96d3359640 1 -- 172.21.15.98:0/991526810 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96cc161f20 con 0x7f96cc15a2a0 2024-09-19T08:43:15.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.357+0000 7f96c1ffb640 1 -- 172.21.15.98:0/991526810 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c800c9f0 con 0x7f96cc15a2a0 2024-09-19T08:43:15.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.357+0000 7f96d3359640 1 -- 172.21.15.98:0/991526810 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96cc1624e0 con 0x7f96cc15a2a0 2024-09-19T08:43:15.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.357+0000 7f96d3359640 1 -- 172.21.15.98:0/991526810 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96cc06f380 con 0x7f96cc15a2a0 2024-09-19T08:43:15.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.357+0000 7f96c1ffb640 1 -- 172.21.15.98:0/991526810 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f96c801a070 con 0x7f96cc15a2a0 2024-09-19T08:43:15.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.357+0000 7f96c1ffb640 1 --2- 172.21.15.98:0/991526810 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f96b0067a10 0x7f96b0069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:15.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.357+0000 7f96c1ffb640 1 -- 172.21.15.98:0/991526810 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 4857+0+0 (secure 0 0 0) 0x7f96c80858c0 con 0x7f96cc15a2a0 2024-09-19T08:43:15.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.357+0000 7f96d10f5640 1 --2- 172.21.15.98:0/991526810 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f96b0067a10 0x7f96b0069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:15.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.361+0000 7f96d10f5640 1 --2- 172.21.15.98:0/991526810 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f96b0067a10 0x7f96b0069ed0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f96c40af000 tx=0x7f96c40a7040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:15.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.361+0000 7f96c1ffb640 1 -- 172.21.15.98:0/991526810 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f96c80566d0 con 0x7f96cc15a2a0 2024-09-19T08:43:15.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:15.525+0000 7f96d3359640 1 -- 172.21.15.98:0/991526810 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f96cc0d30c0 con 0x7f96cc15a2a0 2024-09-19T08:43:16.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.521+0000 7f96c1ffb640 1 -- 172.21.15.98:0/991526810 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) ==== 138+0+0 (secure 0 0 0) 0x7f96c8055fd0 con 0x7f96cc15a2a0 2024-09-19T08:43:16.524 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_metadata' created 2024-09-19T08:43:16.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.525+0000 7f96d3359640 1 -- 172.21.15.98:0/991526810 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f96b0067a10 msgr2=0x7f96b0069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:16.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.525+0000 7f96d3359640 1 --2- 172.21.15.98:0/991526810 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f96b0067a10 0x7f96b0069ed0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f96c40af000 tx=0x7f96c40a7040 comp rx=0 tx=0).stop 2024-09-19T08:43:16.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.525+0000 7f96d3359640 1 -- 172.21.15.98:0/991526810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96cc15a2a0 msgr2=0x7f96cc15a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:16.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.525+0000 7f96d3359640 1 --2- 172.21.15.98:0/991526810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96cc15a2a0 0x7f96cc15a720 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f96cc0726f0 tx=0x7f96c800ec00 comp rx=0 tx=0).stop 2024-09-19T08:43:16.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.525+0000 7f96d3359640 1 -- 172.21.15.98:0/991526810 shutdown_connections 2024-09-19T08:43:16.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.525+0000 7f96d3359640 1 --2- 172.21.15.98:0/991526810 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f96b0067a10 0x7f96b0069ed0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:16.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.525+0000 7f96d3359640 1 --2- 172.21.15.98:0/991526810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96cc15a2a0 0x7f96cc15a720 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:16.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.525+0000 7f96d3359640 1 --2- 172.21.15.98:0/991526810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96cc1643b0 0x7f96cc1611a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:16.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.525+0000 7f96d3359640 1 --2- 172.21.15.98:0/991526810 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96b80039c0 0x7f96cc155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:16.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.525+0000 7f96d3359640 1 -- 172.21.15.98:0/991526810 >> 172.21.15.98:0/991526810 conn(0x7f96cc0cedf0 msgr2=0x7f96cc0d22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:16.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.525+0000 7f96d3359640 1 -- 172.21.15.98:0/991526810 shutdown_connections 2024-09-19T08:43:16.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.525+0000 7f96d3359640 1 -- 172.21.15.98:0/991526810 wait complete. 2024-09-19T08:43:16.553 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T08:43:16.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.685+0000 7f42ca443640 1 Processor -- start 2024-09-19T08:43:16.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.685+0000 7f42ca443640 1 -- start start 2024-09-19T08:43:16.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.685+0000 7f42ca443640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f42c4145b70 0x7f42c406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:16.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.685+0000 7f42ca443640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42c4073530 con 0x7f42c4071ac0 2024-09-19T08:43:16.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.685+0000 7f42ca443640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42c4073810 con 0x7f42c406c1d0 2024-09-19T08:43:16.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.685+0000 7f42ca443640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f42c4071f10 con 0x7f42c4145b70 2024-09-19T08:43:16.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.685+0000 7f42c89e0640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f42c4071ac0 0x7f42c40b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:41998/0 (socket says 172.21.15.98:41998) 2024-09-19T08:43:16.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.685+0000 7f42c89e0640 1 -- 172.21.15.98:0/1320820689 learned_addr learned my addr 172.21.15.98:0/1320820689 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:16.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.685+0000 7f42c3fff640 1 --2- 172.21.15.98:0/1320820689 >> v2:172.21.15.139:3300/0 conn(0x7f42c4145b70 0x7f42c406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:16.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.685+0000 7f42c2ffd640 1 -- 172.21.15.98:0/1320820689 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3442622453 0 0) 0x7f42c4073530 con 0x7f42c4071ac0 2024-09-19T08:43:16.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.685+0000 7f42c2ffd640 1 -- 172.21.15.98:0/1320820689 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42b0003530 con 0x7f42c4071ac0 2024-09-19T08:43:16.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.685+0000 7f42c2ffd640 1 -- 172.21.15.98:0/1320820689 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2295699996 0 0) 0x7f42b0003530 con 0x7f42c4071ac0 2024-09-19T08:43:16.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.685+0000 7f42c2ffd640 1 -- 172.21.15.98:0/1320820689 >> v2:172.21.15.139:3300/0 conn(0x7f42c4145b70 msgr2=0x7f42c406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:16.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.685+0000 7f42c2ffd640 1 --2- 172.21.15.98:0/1320820689 >> v2:172.21.15.139:3300/0 conn(0x7f42c4145b70 0x7f42c406bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:16.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.685+0000 7f42c2ffd640 1 -- 172.21.15.98:0/1320820689 >> v1:172.21.15.139:6789/0 conn(0x7f42c406c1d0 legacy=0x7f42c40713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:16.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.685+0000 7f42c2ffd640 1 -- 172.21.15.98:0/1320820689 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c4073af0 con 0x7f42c4071ac0 2024-09-19T08:43:16.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.685+0000 7f42c2ffd640 1 -- 172.21.15.98:0/1320820689 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f42bc001680 con 0x7f42c4071ac0 2024-09-19T08:43:16.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.685+0000 7f42c2ffd640 1 -- 172.21.15.98:0/1320820689 >> v1:172.21.15.98:6789/0 conn(0x7f42c4071ac0 legacy=0x7f42c40b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:16.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.685+0000 7f42c2ffd640 1 --2- 172.21.15.98:0/1320820689 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42b00039c0 0x7f42b0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:16.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.685+0000 7f42c2ffd640 1 --2- 172.21.15.98:0/1320820689 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42b0004810 0x7f42b0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:16.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.685+0000 7f42c2ffd640 1 --2- 172.21.15.98:0/1320820689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42b00051d0 0x7f42b000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:16.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.685+0000 7f42c2ffd640 1 -- 172.21.15.98:0/1320820689 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f42b000c140 con 0x7f42b00051d0 2024-09-19T08:43:16.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.685+0000 7f42c2ffd640 1 -- 172.21.15.98:0/1320820689 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f42b000c2f0 con 0x7f42b00039c0 2024-09-19T08:43:16.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.685+0000 7f42c2ffd640 1 -- 172.21.15.98:0/1320820689 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f42b000c4f0 con 0x7f42b0004810 2024-09-19T08:43:16.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.689+0000 7f42c89e0640 1 --2- 172.21.15.98:0/1320820689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42b00051d0 0x7f42b000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:16.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.689+0000 7f42c37fe640 1 --2- 172.21.15.98:0/1320820689 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42b0004810 0x7f42b0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:16.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.689+0000 7f42c3fff640 1 --2- 172.21.15.98:0/1320820689 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42b00039c0 0x7f42b0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:16.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.689+0000 7f42c89e0640 1 -- 172.21.15.98:0/1320820689 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42b0004810 msgr2=0x7f42b0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:16.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.689+0000 7f42c89e0640 1 --2- 172.21.15.98:0/1320820689 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42b0004810 0x7f42b0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:16.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.689+0000 7f42c89e0640 1 -- 172.21.15.98:0/1320820689 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42b00039c0 msgr2=0x7f42b0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:16.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.689+0000 7f42c89e0640 1 --2- 172.21.15.98:0/1320820689 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42b00039c0 0x7f42b0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:16.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.689+0000 7f42c89e0640 1 -- 172.21.15.98:0/1320820689 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42b0003530 con 0x7f42b00051d0 2024-09-19T08:43:16.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.689+0000 7f42c89e0640 1 -- 172.21.15.98:0/1320820689 reap_dead start 2024-09-19T08:43:16.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.689+0000 7f42c89e0640 1 --2- 172.21.15.98:0/1320820689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42b00051d0 0x7f42b000b9d0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f42bc00c240 tx=0x7f42bc01f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42764261ba51a8e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:16.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.689+0000 7f42c2ffd640 1 -- 172.21.15.98:0/1320820689 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42bc007d20 con 0x7f42b00051d0 2024-09-19T08:43:16.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.689+0000 7f42c2ffd640 1 -- 172.21.15.98:0/1320820689 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42bc005ae0 con 0x7f42b00051d0 2024-09-19T08:43:16.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.689+0000 7f42c2ffd640 1 -- 172.21.15.98:0/1320820689 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42bc007070 con 0x7f42b00051d0 2024-09-19T08:43:16.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.689+0000 7f42ca443640 1 -- 172.21.15.98:0/1320820689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42b00051d0 msgr2=0x7f42b000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:16.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.689+0000 7f42ca443640 1 --2- 172.21.15.98:0/1320820689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42b00051d0 0x7f42b000b9d0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f42bc00c240 tx=0x7f42bc01f3f0 comp rx=0 tx=0).stop 2024-09-19T08:43:16.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.689+0000 7f42ca443640 1 -- 172.21.15.98:0/1320820689 shutdown_connections 2024-09-19T08:43:16.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.689+0000 7f42ca443640 1 --2- 172.21.15.98:0/1320820689 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42b00051d0 0x7f42b000b9d0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:16.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.689+0000 7f42ca443640 1 -- 172.21.15.98:0/1320820689 >> 172.21.15.98:0/1320820689 conn(0x7f42c40b2010 msgr2=0x7f42c40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:16.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.689+0000 7f42ca443640 1 -- 172.21.15.98:0/1320820689 shutdown_connections 2024-09-19T08:43:16.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.689+0000 7f42ca443640 1 -- 172.21.15.98:0/1320820689 wait complete. 2024-09-19T08:43:16.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.693+0000 7f42ca443640 1 Processor -- start 2024-09-19T08:43:16.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.693+0000 7f42ca443640 1 -- start start 2024-09-19T08:43:16.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.693+0000 7f42ca443640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42b00051d0 0x7f42c415bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:16.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.693+0000 7f42ca443640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42c40aa4a0 0x7f42c40aa920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:16.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.693+0000 7f42ca443640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42c415ebb0 0x7f42c415f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:16.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.693+0000 7f42ca443640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f42c4073dd0 con 0x7f42b00051d0 2024-09-19T08:43:16.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.693+0000 7f42ca443640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f42c406c620 con 0x7f42c40aa4a0 2024-09-19T08:43:16.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.693+0000 7f42ca443640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f42c4145f50 con 0x7f42c415ebb0 2024-09-19T08:43:16.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.693+0000 7f42c3fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42b00051d0 0x7f42c415bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:16.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.693+0000 7f42c3fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42b00051d0 0x7f42c415bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46244/0 (socket says 172.21.15.98:46244) 2024-09-19T08:43:16.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.693+0000 7f42c3fff640 1 -- 172.21.15.98:0/2162290151 learned_addr learned my addr 172.21.15.98:0/2162290151 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:16.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.693+0000 7f42c89e0640 1 --2- 172.21.15.98:0/2162290151 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42c415ebb0 0x7f42c415f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:16.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.693+0000 7f42c37fe640 1 --2- 172.21.15.98:0/2162290151 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42c40aa4a0 0x7f42c40aa920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:16.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.693+0000 7f42c3fff640 1 -- 172.21.15.98:0/2162290151 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42c415ebb0 msgr2=0x7f42c415f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:16.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.693+0000 7f42c3fff640 1 --2- 172.21.15.98:0/2162290151 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42c415ebb0 0x7f42c415f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:16.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.693+0000 7f42c3fff640 1 -- 172.21.15.98:0/2162290151 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42c40aa4a0 msgr2=0x7f42c40aa920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:16.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.693+0000 7f42c3fff640 1 --2- 172.21.15.98:0/2162290151 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42c40aa4a0 0x7f42c40aa920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:16.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.693+0000 7f42c3fff640 1 -- 172.21.15.98:0/2162290151 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c415db10 con 0x7f42b00051d0 2024-09-19T08:43:16.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.693+0000 7f42c37fe640 1 --2- 172.21.15.98:0/2162290151 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42c40aa4a0 0x7f42c40aa920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:16.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.693+0000 7f42c89e0640 1 --2- 172.21.15.98:0/2162290151 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42c415ebb0 0x7f42c415f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:16.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.693+0000 7f42c3fff640 1 --2- 172.21.15.98:0/2162290151 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42b00051d0 0x7f42c415bfd0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f42b4007f00 tx=0x7f42b40115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:16.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.693+0000 7f42c0ff9640 1 -- 172.21.15.98:0/2162290151 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b401a020 con 0x7f42b00051d0 2024-09-19T08:43:16.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.693+0000 7f42c0ff9640 1 -- 172.21.15.98:0/2162290151 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42b4016470 con 0x7f42b00051d0 2024-09-19T08:43:16.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.693+0000 7f42ca443640 1 -- 172.21.15.98:0/2162290151 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42c415ddf0 con 0x7f42b00051d0 2024-09-19T08:43:16.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.693+0000 7f42c0ff9640 1 -- 172.21.15.98:0/2162290151 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b4016860 con 0x7f42b00051d0 2024-09-19T08:43:16.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.693+0000 7f42ca443640 1 -- 172.21.15.98:0/2162290151 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42c415e3b0 con 0x7f42b00051d0 2024-09-19T08:43:16.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.693+0000 7f42ca443640 1 -- 172.21.15.98:0/2162290151 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42c406bc90 con 0x7f42b00051d0 2024-09-19T08:43:16.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.697+0000 7f42c0ff9640 1 -- 172.21.15.98:0/2162290151 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f42b401d020 con 0x7f42b00051d0 2024-09-19T08:43:16.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.697+0000 7f42c0ff9640 1 --2- 172.21.15.98:0/2162290151 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f42a40679e0 0x7f42a4069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:16.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.697+0000 7f42c0ff9640 1 -- 172.21.15.98:0/2162290151 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5227+0+0 (secure 0 0 0) 0x7f42b4086ad0 con 0x7f42b00051d0 2024-09-19T08:43:16.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.697+0000 7f42c37fe640 1 --2- 172.21.15.98:0/2162290151 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f42a40679e0 0x7f42a4069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:16.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.697+0000 7f42c37fe640 1 --2- 172.21.15.98:0/2162290151 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f42a40679e0 0x7f42a4069ea0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f42ac000c10 tx=0x7f42ac009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:16.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.701+0000 7f42c0ff9640 1 -- 172.21.15.98:0/2162290151 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f42b4024030 con 0x7f42b00051d0 2024-09-19T08:43:16.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:16.865+0000 7f42ca443640 1 -- 172.21.15.98:0/2162290151 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f42c406de50 con 0x7f42b00051d0 2024-09-19T08:43:17.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.533+0000 7f42c0ff9640 1 -- 172.21.15.98:0/2162290151 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v81) ==== 170+0+0 (secure 0 0 0) 0x7f42b4057660 con 0x7f42b00051d0 2024-09-19T08:43:17.536 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' created 2024-09-19T08:43:17.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.537+0000 7f42ca443640 1 -- 172.21.15.98:0/2162290151 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f42a40679e0 msgr2=0x7f42a4069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:17.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.537+0000 7f42ca443640 1 --2- 172.21.15.98:0/2162290151 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f42a40679e0 0x7f42a4069ea0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f42ac000c10 tx=0x7f42ac009040 comp rx=0 tx=0).stop 2024-09-19T08:43:17.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.537+0000 7f42ca443640 1 -- 172.21.15.98:0/2162290151 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42b00051d0 msgr2=0x7f42c415bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:17.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.537+0000 7f42ca443640 1 --2- 172.21.15.98:0/2162290151 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42b00051d0 0x7f42c415bfd0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f42b4007f00 tx=0x7f42b40115a0 comp rx=0 tx=0).stop 2024-09-19T08:43:17.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.537+0000 7f42ca443640 1 -- 172.21.15.98:0/2162290151 shutdown_connections 2024-09-19T08:43:17.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.537+0000 7f42ca443640 1 --2- 172.21.15.98:0/2162290151 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f42a40679e0 0x7f42a4069ea0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:17.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.537+0000 7f42ca443640 1 --2- 172.21.15.98:0/2162290151 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42c415ebb0 0x7f42c415f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:17.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.537+0000 7f42ca443640 1 --2- 172.21.15.98:0/2162290151 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42c40aa4a0 0x7f42c40aa920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:17.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.537+0000 7f42ca443640 1 --2- 172.21.15.98:0/2162290151 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42b00051d0 0x7f42c415bfd0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:17.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.537+0000 7f42ca443640 1 -- 172.21.15.98:0/2162290151 >> 172.21.15.98:0/2162290151 conn(0x7f42c40b2010 msgr2=0x7f42c40b4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:17.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.537+0000 7f42ca443640 1 -- 172.21.15.98:0/2162290151 shutdown_connections 2024-09-19T08:43:17.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.537+0000 7f42ca443640 1 -- 172.21.15.98:0/2162290151 wait complete. 2024-09-19T08:43:17.580 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T08:43:17.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.701+0000 7fe624709640 1 Processor -- start 2024-09-19T08:43:17.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.701+0000 7fe624709640 1 -- start start 2024-09-19T08:43:17.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.705+0000 7fe624709640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe61c0d27c0 0x7fe61c0d3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:17.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.705+0000 7fe624709640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe61c0d38a0 0x7fe61c0b8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:17.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.705+0000 7fe624709640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe61c15eae0 con 0x7fe61c1415a0 2024-09-19T08:43:17.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.705+0000 7fe624709640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fe61c15edc0 con 0x7fe61c0d27c0 2024-09-19T08:43:17.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.705+0000 7fe624709640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe61c15f0a0 con 0x7fe61c0d38a0 2024-09-19T08:43:17.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.705+0000 7fe621ca4640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe61c0d27c0 0x7fe61c0d3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:17.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.705+0000 7fe621ca4640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe61c0d27c0 0x7fe61c0d3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41276/0 (socket says 172.21.15.98:41276) 2024-09-19T08:43:17.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.705+0000 7fe621ca4640 1 -- 172.21.15.98:0/3443544108 learned_addr learned my addr 172.21.15.98:0/3443544108 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:17.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.705+0000 7fe622ca6640 1 --2- 172.21.15.98:0/3443544108 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe61c0d38a0 0x7fe61c0b8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:17.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.705+0000 7fe621ca4640 1 -- 172.21.15.98:0/3443544108 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe61c0d38a0 msgr2=0x7fe61c0b8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:17.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.705+0000 7fe621ca4640 1 --2- 172.21.15.98:0/3443544108 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe61c0d38a0 0x7fe61c0b8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:17.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.705+0000 7fe621ca4640 1 -- 172.21.15.98:0/3443544108 >> v1:172.21.15.139:6789/0 conn(0x7fe61c1415a0 legacy=0x7fe61c0d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:17.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.705+0000 7fe621ca4640 1 -- 172.21.15.98:0/3443544108 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe61c15f380 con 0x7fe61c0d27c0 2024-09-19T08:43:17.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.705+0000 7fe622ca6640 1 --2- 172.21.15.98:0/3443544108 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe61c0d38a0 0x7fe61c0b8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:17.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.705+0000 7fe621ca4640 1 --2- 172.21.15.98:0/3443544108 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe61c0d27c0 0x7fe61c0d3360 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fe618009870 tx=0x7fe61801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e1a7413dd0d82f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:17.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.705+0000 7fe6214a3640 1 -- 172.21.15.98:0/3443544108 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6180a7070 con 0x7fe61c0d27c0 2024-09-19T08:43:17.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.705+0000 7fe6214a3640 1 -- 172.21.15.98:0/3443544108 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6180b0070 con 0x7fe61c0d27c0 2024-09-19T08:43:17.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.705+0000 7fe6214a3640 1 -- 172.21.15.98:0/3443544108 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6180ab890 con 0x7fe61c0d27c0 2024-09-19T08:43:17.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.709+0000 7fe624709640 1 -- 172.21.15.98:0/3443544108 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe61c0d27c0 msgr2=0x7fe61c0d3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:17.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.709+0000 7fe624709640 1 --2- 172.21.15.98:0/3443544108 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe61c0d27c0 0x7fe61c0d3360 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fe618009870 tx=0x7fe61801c490 comp rx=0 tx=0).stop 2024-09-19T08:43:17.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.709+0000 7fe624709640 1 -- 172.21.15.98:0/3443544108 shutdown_connections 2024-09-19T08:43:17.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.709+0000 7fe624709640 1 --2- 172.21.15.98:0/3443544108 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe61c0d38a0 0x7fe61c0b8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:17.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.709+0000 7fe624709640 1 --2- 172.21.15.98:0/3443544108 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe61c0d27c0 0x7fe61c0d3360 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:17.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.709+0000 7fe624709640 1 -- 172.21.15.98:0/3443544108 >> 172.21.15.98:0/3443544108 conn(0x7fe61c074c70 msgr2=0x7fe61c075080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:17.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.709+0000 7fe624709640 1 -- 172.21.15.98:0/3443544108 shutdown_connections 2024-09-19T08:43:17.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.709+0000 7fe624709640 1 -- 172.21.15.98:0/3443544108 wait complete. 2024-09-19T08:43:17.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.709+0000 7fe624709640 1 Processor -- start 2024-09-19T08:43:17.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.709+0000 7fe624709640 1 -- start start 2024-09-19T08:43:17.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.709+0000 7fe624709640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe61c0d27c0 0x7fe61c142900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:17.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.709+0000 7fe624709640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe61c0d38a0 0x7fe61c144400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:17.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.709+0000 7fe624709640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe61c1415a0 0x7fe61c1582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:17.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.709+0000 7fe624709640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c0b9660 con 0x7fe61c0d27c0 2024-09-19T08:43:17.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.709+0000 7fe624709640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c0b94e0 con 0x7fe61c1415a0 2024-09-19T08:43:17.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.709+0000 7fe624709640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe61c0b97e0 con 0x7fe61c0d38a0 2024-09-19T08:43:17.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.709+0000 7fe6224a5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe61c0d27c0 0x7fe61c142900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:17.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.709+0000 7fe6224a5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe61c0d27c0 0x7fe61c142900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41286/0 (socket says 172.21.15.98:41286) 2024-09-19T08:43:17.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.709+0000 7fe6224a5640 1 -- 172.21.15.98:0/2042970630 learned_addr learned my addr 172.21.15.98:0/2042970630 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:17.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.709+0000 7fe621ca4640 1 --2- 172.21.15.98:0/2042970630 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe61c0d38a0 0x7fe61c144400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:17.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.709+0000 7fe622ca6640 1 --2- 172.21.15.98:0/2042970630 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe61c1415a0 0x7fe61c1582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:17.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.709+0000 7fe6224a5640 1 -- 172.21.15.98:0/2042970630 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe61c0d38a0 msgr2=0x7fe61c144400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:17.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.709+0000 7fe6224a5640 1 --2- 172.21.15.98:0/2042970630 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe61c0d38a0 0x7fe61c144400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:17.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.709+0000 7fe6224a5640 1 -- 172.21.15.98:0/2042970630 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe61c1415a0 msgr2=0x7fe61c1582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:17.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.709+0000 7fe6224a5640 1 --2- 172.21.15.98:0/2042970630 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe61c1415a0 0x7fe61c1582b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:17.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.709+0000 7fe6224a5640 1 -- 172.21.15.98:0/2042970630 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe61c144940 con 0x7fe61c0d27c0 2024-09-19T08:43:17.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.709+0000 7fe6224a5640 1 --2- 172.21.15.98:0/2042970630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe61c0d27c0 0x7fe61c142900 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00b410 tx=0x7fe60c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:17.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.709+0000 7fe612ffd640 1 -- 172.21.15.98:0/2042970630 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c015030 con 0x7fe61c0d27c0 2024-09-19T08:43:17.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.709+0000 7fe612ffd640 1 -- 172.21.15.98:0/2042970630 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe60c004020 con 0x7fe61c0d27c0 2024-09-19T08:43:17.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.709+0000 7fe624709640 1 -- 172.21.15.98:0/2042970630 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe61c143360 con 0x7fe61c0d27c0 2024-09-19T08:43:17.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.709+0000 7fe612ffd640 1 -- 172.21.15.98:0/2042970630 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c004450 con 0x7fe61c0d27c0 2024-09-19T08:43:17.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.709+0000 7fe624709640 1 -- 172.21.15.98:0/2042970630 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe61c143920 con 0x7fe61c0d27c0 2024-09-19T08:43:17.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.713+0000 7fe624709640 1 -- 172.21.15.98:0/2042970630 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe61c0d3360 con 0x7fe61c0d27c0 2024-09-19T08:43:17.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.713+0000 7fe612ffd640 1 -- 172.21.15.98:0/2042970630 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fe60c018070 con 0x7fe61c0d27c0 2024-09-19T08:43:17.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.713+0000 7fe612ffd640 1 --2- 172.21.15.98:0/2042970630 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe6000679e0 0x7fe600069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:17.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.713+0000 7fe612ffd640 1 -- 172.21.15.98:0/2042970630 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5609+0+0 (secure 0 0 0) 0x7fe60c085950 con 0x7fe61c0d27c0 2024-09-19T08:43:17.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.713+0000 7fe621ca4640 1 --2- 172.21.15.98:0/2042970630 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe6000679e0 0x7fe600069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:17.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.713+0000 7fe621ca4640 1 --2- 172.21.15.98:0/2042970630 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe6000679e0 0x7fe600069ea0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe6180af000 tx=0x7fe6180a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:17.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.717+0000 7fe612ffd640 1 -- 172.21.15.98:0/2042970630 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe60c0563e0 con 0x7fe61c0d27c0 2024-09-19T08:43:17.906 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:43:17.909 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:43:17.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:17.913+0000 7fe624709640 1 -- 172.21.15.98:0/2042970630 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe61c0cb200 con 0x7fe61c0d27c0 2024-09-19T08:43:18.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.553+0000 7fe612ffd640 1 -- 172.21.15.98:0/2042970630 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v43) ==== 174+0+0 (secure 0 0 0) 0x7fe60c055ce0 con 0x7fe61c0d27c0 2024-09-19T08:43:18.554 INFO:teuthology.orchestra.run.smithi098.stderr:new fs with metadata pool 12 and data pool 13 2024-09-19T08:43:18.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.569+0000 7fe624709640 1 -- 172.21.15.98:0/2042970630 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe6000679e0 msgr2=0x7fe600069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:18.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.569+0000 7fe624709640 1 --2- 172.21.15.98:0/2042970630 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe6000679e0 0x7fe600069ea0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe6180af000 tx=0x7fe6180a7000 comp rx=0 tx=0).stop 2024-09-19T08:43:18.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.569+0000 7fe624709640 1 -- 172.21.15.98:0/2042970630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe61c0d27c0 msgr2=0x7fe61c142900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:18.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.569+0000 7fe624709640 1 --2- 172.21.15.98:0/2042970630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe61c0d27c0 0x7fe61c142900 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00b410 tx=0x7fe60c00b8d0 comp rx=0 tx=0).stop 2024-09-19T08:43:18.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.569+0000 7fe624709640 1 -- 172.21.15.98:0/2042970630 shutdown_connections 2024-09-19T08:43:18.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.569+0000 7fe624709640 1 --2- 172.21.15.98:0/2042970630 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe6000679e0 0x7fe600069ea0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:18.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.569+0000 7fe624709640 1 --2- 172.21.15.98:0/2042970630 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe61c1415a0 0x7fe61c1582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:18.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.569+0000 7fe624709640 1 --2- 172.21.15.98:0/2042970630 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe61c0d38a0 0x7fe61c144400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:18.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.569+0000 7fe624709640 1 --2- 172.21.15.98:0/2042970630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe61c0d27c0 0x7fe61c142900 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:18.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.569+0000 7fe624709640 1 -- 172.21.15.98:0/2042970630 >> 172.21.15.98:0/2042970630 conn(0x7fe61c074c70 msgr2=0x7fe61c140800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:18.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.569+0000 7fe624709640 1 -- 172.21.15.98:0/2042970630 shutdown_connections 2024-09-19T08:43:18.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.569+0000 7fe624709640 1 -- 172.21.15.98:0/2042970630 wait complete. 2024-09-19T08:43:18.601 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:43:18.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.713+0000 7f7aeecdb640 1 Processor -- start 2024-09-19T08:43:18.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.713+0000 7f7aeecdb640 1 -- start start 2024-09-19T08:43:18.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.717+0000 7f7aeecdb640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7ae815c550 0x7f7ae806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:18.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.717+0000 7f7aeecdb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ae806c1d0 0x7f7ae80cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:18.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.717+0000 7f7aeecdb640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f7ae80cf900 0x7f7ae80ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:18.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.717+0000 7f7aeecdb640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f7ae806c610 con 0x7f7ae815c550 2024-09-19T08:43:18.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.717+0000 7f7aeecdb640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f7ae806d310 con 0x7f7ae80cf900 2024-09-19T08:43:18.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.717+0000 7f7aeecdb640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7ae806d5f0 con 0x7f7ae806c1d0 2024-09-19T08:43:18.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.717+0000 7f7aeca77640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7ae815c550 0x7f7ae806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:18.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.717+0000 7f7aeca77640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7ae815c550 0x7f7ae806bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41302/0 (socket says 172.21.15.98:41302) 2024-09-19T08:43:18.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.717+0000 7f7aeca77640 1 -- 172.21.15.98:0/1408371506 learned_addr learned my addr 172.21.15.98:0/1408371506 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:18.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.717+0000 7f7ae7fff640 1 --2- 172.21.15.98:0/1408371506 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ae806c1d0 0x7f7ae80cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:18.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.717+0000 7f7aed278640 1 --2- 172.21.15.98:0/1408371506 >> v2:172.21.15.139:3300/0 conn(0x7f7ae80cf900 0x7f7ae80ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:18.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.717+0000 7f7aeca77640 1 -- 172.21.15.98:0/1408371506 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ae806c1d0 msgr2=0x7f7ae80cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:18.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.717+0000 7f7aeca77640 1 --2- 172.21.15.98:0/1408371506 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ae806c1d0 0x7f7ae80cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:18.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.717+0000 7f7aeca77640 1 -- 172.21.15.98:0/1408371506 >> v2:172.21.15.139:3300/0 conn(0x7f7ae80cf900 msgr2=0x7f7ae80ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:18.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.717+0000 7f7aeca77640 1 --2- 172.21.15.98:0/1408371506 >> v2:172.21.15.139:3300/0 conn(0x7f7ae80cf900 0x7f7ae80ce1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:18.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.717+0000 7f7aeca77640 1 -- 172.21.15.98:0/1408371506 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae806d8d0 con 0x7f7ae815c550 2024-09-19T08:43:18.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.717+0000 7f7aed278640 1 --2- 172.21.15.98:0/1408371506 >> v2:172.21.15.139:3300/0 conn(0x7f7ae80cf900 0x7f7ae80ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:43:18.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.717+0000 7f7aeca77640 1 --2- 172.21.15.98:0/1408371506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ae815c550 0x7f7ae806bc90 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8074090 tx=0x7f7adc01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82742832262785cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:18.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.721+0000 7f7ae77fe640 1 -- 172.21.15.98:0/1408371506 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc0b0070 con 0x7f7ae815c550 2024-09-19T08:43:18.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.721+0000 7f7ae77fe640 1 -- 172.21.15.98:0/1408371506 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7adc0ab600 con 0x7f7ae815c550 2024-09-19T08:43:18.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.721+0000 7f7ae77fe640 1 -- 172.21.15.98:0/1408371506 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc0ab9d0 con 0x7f7ae815c550 2024-09-19T08:43:18.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.721+0000 7f7aeecdb640 1 -- 172.21.15.98:0/1408371506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ae815c550 msgr2=0x7f7ae806bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:18.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.721+0000 7f7aeecdb640 1 --2- 172.21.15.98:0/1408371506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ae815c550 0x7f7ae806bc90 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8074090 tx=0x7f7adc01c260 comp rx=0 tx=0).stop 2024-09-19T08:43:18.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.721+0000 7f7aeecdb640 1 -- 172.21.15.98:0/1408371506 shutdown_connections 2024-09-19T08:43:18.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.721+0000 7f7aeecdb640 1 --2- 172.21.15.98:0/1408371506 >> v2:172.21.15.139:3300/0 conn(0x7f7ae80cf900 0x7f7ae80ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:18.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.721+0000 7f7aeecdb640 1 --2- 172.21.15.98:0/1408371506 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ae806c1d0 0x7f7ae80cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:18.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.721+0000 7f7aeecdb640 1 --2- 172.21.15.98:0/1408371506 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ae815c550 0x7f7ae806bc90 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:18.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.721+0000 7f7aeecdb640 1 -- 172.21.15.98:0/1408371506 >> 172.21.15.98:0/1408371506 conn(0x7f7ae80b2010 msgr2=0x7f7ae80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:18.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.721+0000 7f7aeecdb640 1 -- 172.21.15.98:0/1408371506 shutdown_connections 2024-09-19T08:43:18.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.721+0000 7f7aeecdb640 1 -- 172.21.15.98:0/1408371506 wait complete. 2024-09-19T08:43:18.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.721+0000 7f7aeecdb640 1 Processor -- start 2024-09-19T08:43:18.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.721+0000 7f7aeecdb640 1 -- start start 2024-09-19T08:43:18.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.721+0000 7f7aeecdb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ae806c1d0 0x7f7ae814cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:18.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.721+0000 7f7aeecdb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ae80cf900 0x7f7ae814e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:18.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.721+0000 7f7aeecdb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ae815c550 0x7f7ae81525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:18.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.721+0000 7f7aeecdb640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae80723a0 con 0x7f7ae806c1d0 2024-09-19T08:43:18.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.721+0000 7f7aeecdb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae815cb40 con 0x7f7ae815c550 2024-09-19T08:43:18.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.721+0000 7f7aeecdb640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7ae8072520 con 0x7f7ae80cf900 2024-09-19T08:43:18.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.721+0000 7f7aeca77640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ae806c1d0 0x7f7ae814cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:18.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.721+0000 7f7aeca77640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ae806c1d0 0x7f7ae814cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41306/0 (socket says 172.21.15.98:41306) 2024-09-19T08:43:18.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.721+0000 7f7aeca77640 1 -- 172.21.15.98:0/3829838589 learned_addr learned my addr 172.21.15.98:0/3829838589 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:18.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.721+0000 7f7aeca77640 1 -- 172.21.15.98:0/3829838589 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ae80cf900 msgr2=0x7f7ae814e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:18.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.721+0000 7f7aeca77640 1 --2- 172.21.15.98:0/3829838589 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ae80cf900 0x7f7ae814e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:18.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.721+0000 7f7aeca77640 1 -- 172.21.15.98:0/3829838589 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ae815c550 msgr2=0x7f7ae81525a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:18.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.721+0000 7f7aeca77640 1 --2- 172.21.15.98:0/3829838589 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ae815c550 0x7f7ae81525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:18.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.721+0000 7f7aeca77640 1 -- 172.21.15.98:0/3829838589 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae815d850 con 0x7f7ae806c1d0 2024-09-19T08:43:18.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.721+0000 7f7aeca77640 1 --2- 172.21.15.98:0/3829838589 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ae806c1d0 0x7f7ae814cac0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f7adc0ae040 tx=0x7f7adc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:18.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.721+0000 7f7ae57fa640 1 -- 172.21.15.98:0/3829838589 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc0b0040 con 0x7f7ae806c1d0 2024-09-19T08:43:18.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.721+0000 7f7ae57fa640 1 -- 172.21.15.98:0/3829838589 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7adc0b9070 con 0x7f7ae806c1d0 2024-09-19T08:43:18.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.721+0000 7f7aeecdb640 1 -- 172.21.15.98:0/3829838589 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ae815db10 con 0x7f7ae806c1d0 2024-09-19T08:43:18.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.721+0000 7f7ae57fa640 1 -- 172.21.15.98:0/3829838589 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc0b46a0 con 0x7f7ae806c1d0 2024-09-19T08:43:18.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.721+0000 7f7aeecdb640 1 -- 172.21.15.98:0/3829838589 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ae815e0d0 con 0x7f7ae806c1d0 2024-09-19T08:43:18.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.725+0000 7f7aeecdb640 1 -- 172.21.15.98:0/3829838589 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ae80ce1a0 con 0x7f7ae806c1d0 2024-09-19T08:43:18.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.725+0000 7f7ae57fa640 1 -- 172.21.15.98:0/3829838589 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f7adc0b4840 con 0x7f7ae806c1d0 2024-09-19T08:43:18.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.725+0000 7f7ae57fa640 1 --2- 172.21.15.98:0/3829838589 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7ad4067a10 0x7f7ad4069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:18.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.725+0000 7f7ae57fa640 1 -- 172.21.15.98:0/3829838589 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f7adc11fc60 con 0x7f7ae806c1d0 2024-09-19T08:43:18.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.725+0000 7f7ae7fff640 1 --2- 172.21.15.98:0/3829838589 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7ad4067a10 0x7f7ad4069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:18.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.725+0000 7f7ae7fff640 1 --2- 172.21.15.98:0/3829838589 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7ad4067a10 0x7f7ad4069ed0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8155460 tx=0x7f7ad8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:18.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.729+0000 7f7ae57fa640 1 -- 172.21.15.98:0/3829838589 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f7adc016110 con 0x7f7ae806c1d0 2024-09-19T08:43:18.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.889+0000 7f7aeecdb640 1 -- 172.21.15.98:0/3829838589 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ae814d000 con 0x7f7ae806c1d0 2024-09-19T08:43:18.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.893+0000 7f7ae57fa640 1 -- 172.21.15.98:0/3829838589 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+16733 (secure 0 0 0) 0x7f7adc0f04f0 con 0x7f7ae806c1d0 2024-09-19T08:43:18.895 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:18.895 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":82,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:43:17.919961+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:43:15.568647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-19T08:43:16.905534+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:43:18.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.897+0000 7f7aeecdb640 1 -- 172.21.15.98:0/3829838589 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7ad4067a10 msgr2=0x7f7ad4069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:18.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.897+0000 7f7aeecdb640 1 --2- 172.21.15.98:0/3829838589 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7ad4067a10 0x7f7ad4069ed0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8155460 tx=0x7f7ad8008040 comp rx=0 tx=0).stop 2024-09-19T08:43:18.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.897+0000 7f7aeecdb640 1 -- 172.21.15.98:0/3829838589 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ae806c1d0 msgr2=0x7f7ae814cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:18.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.897+0000 7f7aeecdb640 1 --2- 172.21.15.98:0/3829838589 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ae806c1d0 0x7f7ae814cac0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f7adc0ae040 tx=0x7f7adc019000 comp rx=0 tx=0).stop 2024-09-19T08:43:18.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.897+0000 7f7aeecdb640 1 -- 172.21.15.98:0/3829838589 shutdown_connections 2024-09-19T08:43:18.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.897+0000 7f7aeecdb640 1 --2- 172.21.15.98:0/3829838589 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7ad4067a10 0x7f7ad4069ed0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:18.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.897+0000 7f7aeecdb640 1 --2- 172.21.15.98:0/3829838589 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ae815c550 0x7f7ae81525a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:18.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.897+0000 7f7aeecdb640 1 --2- 172.21.15.98:0/3829838589 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7ae80cf900 0x7f7ae814e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:18.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.897+0000 7f7aeecdb640 1 --2- 172.21.15.98:0/3829838589 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7ae806c1d0 0x7f7ae814cac0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:18.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.897+0000 7f7aeecdb640 1 -- 172.21.15.98:0/3829838589 >> 172.21.15.98:0/3829838589 conn(0x7f7ae80b2010 msgr2=0x7f7ae80afb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:18.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.897+0000 7f7aeecdb640 1 -- 172.21.15.98:0/3829838589 shutdown_connections 2024-09-19T08:43:18.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:18.897+0000 7f7aeecdb640 1 -- 172.21.15.98:0/3829838589 wait complete. 2024-09-19T08:43:18.912 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:43:19.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.033+0000 7fb859f42640 1 Processor -- start 2024-09-19T08:43:19.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.033+0000 7fb859f42640 1 -- start start 2024-09-19T08:43:19.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.033+0000 7fb859f42640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb85406efa0 0x7fb8540ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:19.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.033+0000 7fb859f42640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb8540aefd0 0x7fb8540cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:19.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.033+0000 7fb859f42640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8540af4a0 con 0x7fb85406f380 2024-09-19T08:43:19.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.033+0000 7fb859f42640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fb85415ed00 con 0x7fb85406efa0 2024-09-19T08:43:19.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.033+0000 7fb859f42640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fb8540af780 con 0x7fb8540aefd0 2024-09-19T08:43:19.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.033+0000 7fb8537fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb85406efa0 0x7fb8540ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:19.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.033+0000 7fb852ffd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fb85406f380 0x7fb8540ae8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40030/0 (socket says 172.21.15.98:40030) 2024-09-19T08:43:19.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.033+0000 7fb852ffd640 1 -- 172.21.15.98:0/1035087444 learned_addr learned my addr 172.21.15.98:0/1035087444 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:19.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.033+0000 7fb8537fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb85406efa0 0x7fb8540ce550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41308/0 (socket says 172.21.15.98:41308) 2024-09-19T08:43:19.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.033+0000 7fb853fff640 1 --2- 172.21.15.98:0/1035087444 >> v2:172.21.15.139:3300/0 conn(0x7fb8540aefd0 0x7fb8540cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:19.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.033+0000 7fb8537fe640 1 -- 172.21.15.98:0/1035087444 >> v2:172.21.15.139:3300/0 conn(0x7fb8540aefd0 msgr2=0x7fb8540cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:19.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.033+0000 7fb8537fe640 1 --2- 172.21.15.98:0/1035087444 >> v2:172.21.15.139:3300/0 conn(0x7fb8540aefd0 0x7fb8540cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:19.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.033+0000 7fb8537fe640 1 -- 172.21.15.98:0/1035087444 >> v1:172.21.15.98:6789/0 conn(0x7fb85406f380 legacy=0x7fb8540ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:19.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.033+0000 7fb8537fe640 1 -- 172.21.15.98:0/1035087444 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8540afa60 con 0x7fb85406efa0 2024-09-19T08:43:19.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.033+0000 7fb8527fc640 1 -- 172.21.15.98:0/1035087444 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2077157977 0 0) 0x7fb8540af4a0 con 0x7fb85406f380 2024-09-19T08:43:19.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.037+0000 7fb8537fe640 1 --2- 172.21.15.98:0/1035087444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb85406efa0 0x7fb8540ce550 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fb8440097c0 tx=0x7fb84401c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdd1f22acd9ed154 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:19.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.037+0000 7fb8527fc640 1 -- 172.21.15.98:0/1035087444 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8440b0070 con 0x7fb85406efa0 2024-09-19T08:43:19.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.037+0000 7fb8527fc640 1 -- 172.21.15.98:0/1035087444 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8440ab600 con 0x7fb85406efa0 2024-09-19T08:43:19.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.037+0000 7fb8527fc640 1 -- 172.21.15.98:0/1035087444 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8440ab9d0 con 0x7fb85406efa0 2024-09-19T08:43:19.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.037+0000 7fb859f42640 1 -- 172.21.15.98:0/1035087444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb85406efa0 msgr2=0x7fb8540ce550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:19.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.037+0000 7fb859f42640 1 --2- 172.21.15.98:0/1035087444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb85406efa0 0x7fb8540ce550 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fb8440097c0 tx=0x7fb84401c340 comp rx=0 tx=0).stop 2024-09-19T08:43:19.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.037+0000 7fb859f42640 1 -- 172.21.15.98:0/1035087444 shutdown_connections 2024-09-19T08:43:19.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.037+0000 7fb859f42640 1 --2- 172.21.15.98:0/1035087444 >> v2:172.21.15.139:3300/0 conn(0x7fb8540aefd0 0x7fb8540cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:19.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.037+0000 7fb859f42640 1 --2- 172.21.15.98:0/1035087444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb85406efa0 0x7fb8540ce550 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:19.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.037+0000 7fb859f42640 1 -- 172.21.15.98:0/1035087444 >> 172.21.15.98:0/1035087444 conn(0x7fb8540d0980 msgr2=0x7fb8540d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:19.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.037+0000 7fb859f42640 1 -- 172.21.15.98:0/1035087444 shutdown_connections 2024-09-19T08:43:19.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.037+0000 7fb859f42640 1 -- 172.21.15.98:0/1035087444 wait complete. 2024-09-19T08:43:19.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.037+0000 7fb859f42640 1 Processor -- start 2024-09-19T08:43:19.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.041+0000 7fb859f42640 1 -- start start 2024-09-19T08:43:19.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.041+0000 7fb859f42640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb85406efa0 0x7fb854151db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:19.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.041+0000 7fb859f42640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8540aefd0 0x7fb854155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:19.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.041+0000 7fb859f42640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb854073470 0x7fb85414ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:19.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.041+0000 7fb859f42640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb85415c180 con 0x7fb8540aefd0 2024-09-19T08:43:19.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.041+0000 7fb859f42640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb85415c000 con 0x7fb854073470 2024-09-19T08:43:19.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.041+0000 7fb859f42640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb85415c300 con 0x7fb85406efa0 2024-09-19T08:43:19.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.041+0000 7fb852ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8540aefd0 0x7fb854155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:19.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.041+0000 7fb852ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8540aefd0 0x7fb854155580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41318/0 (socket says 172.21.15.98:41318) 2024-09-19T08:43:19.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.041+0000 7fb852ffd640 1 -- 172.21.15.98:0/241469011 learned_addr learned my addr 172.21.15.98:0/241469011 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:19.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.041+0000 7fb853fff640 1 --2- 172.21.15.98:0/241469011 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb854073470 0x7fb85414ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:19.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.041+0000 7fb8537fe640 1 --2- 172.21.15.98:0/241469011 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb85406efa0 0x7fb854151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:19.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.041+0000 7fb852ffd640 1 -- 172.21.15.98:0/241469011 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb85406efa0 msgr2=0x7fb854151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:19.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.041+0000 7fb852ffd640 1 --2- 172.21.15.98:0/241469011 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb85406efa0 0x7fb854151db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:19.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.041+0000 7fb852ffd640 1 -- 172.21.15.98:0/241469011 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb854073470 msgr2=0x7fb85414ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:19.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.041+0000 7fb852ffd640 1 --2- 172.21.15.98:0/241469011 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb854073470 0x7fb85414ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:19.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.041+0000 7fb852ffd640 1 -- 172.21.15.98:0/241469011 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb854072380 con 0x7fb8540aefd0 2024-09-19T08:43:19.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.041+0000 7fb853fff640 1 --2- 172.21.15.98:0/241469011 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb854073470 0x7fb85414ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:19.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.041+0000 7fb8537fe640 1 --2- 172.21.15.98:0/241469011 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb85406efa0 0x7fb854151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:19.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.041+0000 7fb852ffd640 1 --2- 172.21.15.98:0/241469011 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8540aefd0 0x7fb854155580 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fb84800c4b0 tx=0x7fb84800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:19.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.041+0000 7fb83bfff640 1 -- 172.21.15.98:0/241469011 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb848018070 con 0x7fb8540aefd0 2024-09-19T08:43:19.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.041+0000 7fb83bfff640 1 -- 172.21.15.98:0/241469011 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb848007500 con 0x7fb8540aefd0 2024-09-19T08:43:19.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.041+0000 7fb859f42640 1 -- 172.21.15.98:0/241469011 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb854154210 con 0x7fb8540aefd0 2024-09-19T08:43:19.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.041+0000 7fb83bfff640 1 -- 172.21.15.98:0/241469011 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb848004a30 con 0x7fb8540aefd0 2024-09-19T08:43:19.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.041+0000 7fb859f42640 1 -- 172.21.15.98:0/241469011 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8541547d0 con 0x7fb8540aefd0 2024-09-19T08:43:19.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.041+0000 7fb859f42640 1 -- 172.21.15.98:0/241469011 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb854154b10 con 0x7fb8540aefd0 2024-09-19T08:43:19.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.041+0000 7fb83bfff640 1 -- 172.21.15.98:0/241469011 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fb84801b070 con 0x7fb8540aefd0 2024-09-19T08:43:19.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.045+0000 7fb83bfff640 1 --2- 172.21.15.98:0/241469011 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb8400679e0 0x7fb840069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:19.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.045+0000 7fb83bfff640 1 -- 172.21.15.98:0/241469011 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7fb848085450 con 0x7fb8540aefd0 2024-09-19T08:43:19.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.045+0000 7fb8537fe640 1 --2- 172.21.15.98:0/241469011 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb8400679e0 0x7fb840069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:19.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.045+0000 7fb8537fe640 1 --2- 172.21.15.98:0/241469011 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb8400679e0 0x7fb840069ea0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb8440ad000 tx=0x7fb8440a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:19.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.045+0000 7fb83bfff640 1 -- 172.21.15.98:0/241469011 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb854154b10 con 0x7fb8540aefd0 2024-09-19T08:43:19.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.209+0000 7fb859f42640 1 -- 172.21.15.98:0/241469011 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb85406fee0 con 0x7fb8540aefd0 2024-09-19T08:43:19.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.213+0000 7fb83bfff640 1 -- 172.21.15.98:0/241469011 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+16733 (secure 0 0 0) 0x7fb848056020 con 0x7fb8540aefd0 2024-09-19T08:43:19.213 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:19.213 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":82,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:43:17.919961+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:43:15.568647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-19T08:43:16.905534+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:43:19.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.213+0000 7fb859f42640 1 -- 172.21.15.98:0/241469011 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb8400679e0 msgr2=0x7fb840069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:19.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.213+0000 7fb859f42640 1 --2- 172.21.15.98:0/241469011 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb8400679e0 0x7fb840069ea0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb8440ad000 tx=0x7fb8440a6000 comp rx=0 tx=0).stop 2024-09-19T08:43:19.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.213+0000 7fb859f42640 1 -- 172.21.15.98:0/241469011 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8540aefd0 msgr2=0x7fb854155580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:19.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.213+0000 7fb859f42640 1 --2- 172.21.15.98:0/241469011 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8540aefd0 0x7fb854155580 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fb84800c4b0 tx=0x7fb84800c970 comp rx=0 tx=0).stop 2024-09-19T08:43:19.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.213+0000 7fb859f42640 1 -- 172.21.15.98:0/241469011 shutdown_connections 2024-09-19T08:43:19.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.213+0000 7fb859f42640 1 --2- 172.21.15.98:0/241469011 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb8400679e0 0x7fb840069ea0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:19.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.213+0000 7fb859f42640 1 --2- 172.21.15.98:0/241469011 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb854073470 0x7fb85414ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:19.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.213+0000 7fb859f42640 1 --2- 172.21.15.98:0/241469011 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8540aefd0 0x7fb854155580 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:19.216 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.213+0000 7fb859f42640 1 --2- 172.21.15.98:0/241469011 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb85406efa0 0x7fb854151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:19.216 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.213+0000 7fb859f42640 1 -- 172.21.15.98:0/241469011 >> 172.21.15.98:0/241469011 conn(0x7fb8540d0980 msgr2=0x7fb85415c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:19.216 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.213+0000 7fb859f42640 1 -- 172.21.15.98:0/241469011 shutdown_connections 2024-09-19T08:43:19.216 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.213+0000 7fb859f42640 1 -- 172.21.15.98:0/241469011 wait complete. 2024-09-19T08:43:19.231 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-19T08:43:19.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.357+0000 7f8ecf019640 1 Processor -- start 2024-09-19T08:43:19.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.357+0000 7f8ecf019640 1 -- start start 2024-09-19T08:43:19.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.357+0000 7f8ecf019640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ec8145b70 0x7f8ec806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:19.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.357+0000 7f8ecf019640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8ec8071ac0 0x7f8ec80b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:19.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.357+0000 7f8ecf019640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ec806c5f0 con 0x7f8ec806c1d0 2024-09-19T08:43:19.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.357+0000 7f8ecf019640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f8ec8071f20 con 0x7f8ec8071ac0 2024-09-19T08:43:19.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.357+0000 7f8ecf019640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8ec8073530 con 0x7f8ec8145b70 2024-09-19T08:43:19.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.357+0000 7f8ecd5b6640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8ec8071ac0 0x7f8ec80b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:19.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.357+0000 7f8ebffff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f8ec806c1d0 0x7f8ec80713b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40046/0 (socket says 172.21.15.98:40046) 2024-09-19T08:43:19.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.357+0000 7f8ecd5b6640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8ec8071ac0 0x7f8ec80b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41328/0 (socket says 172.21.15.98:41328) 2024-09-19T08:43:19.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.357+0000 7f8ebffff640 1 -- 172.21.15.98:0/262138616 learned_addr learned my addr 172.21.15.98:0/262138616 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:19.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.357+0000 7f8eccdb5640 1 --2- 172.21.15.98:0/262138616 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ec8145b70 0x7f8ec806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:19.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.357+0000 7f8ecd5b6640 1 -- 172.21.15.98:0/262138616 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ec8145b70 msgr2=0x7f8ec806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:19.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.357+0000 7f8ecd5b6640 1 --2- 172.21.15.98:0/262138616 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ec8145b70 0x7f8ec806bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:19.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.357+0000 7f8ecd5b6640 1 -- 172.21.15.98:0/262138616 >> v1:172.21.15.98:6789/0 conn(0x7f8ec806c1d0 legacy=0x7f8ec80713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:19.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.357+0000 7f8ecd5b6640 1 -- 172.21.15.98:0/262138616 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec8073810 con 0x7f8ec8071ac0 2024-09-19T08:43:19.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.361+0000 7f8ecd5b6640 1 --2- 172.21.15.98:0/262138616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ec8071ac0 0x7f8ec80b3410 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0009d90 tx=0x7f8ec0020cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a90ed8917966ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:19.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.361+0000 7f8ebf7fe640 1 -- 172.21.15.98:0/262138616 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec00b6070 con 0x7f8ec8071ac0 2024-09-19T08:43:19.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.361+0000 7f8ebf7fe640 1 -- 172.21.15.98:0/262138616 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ec00a99c0 con 0x7f8ec8071ac0 2024-09-19T08:43:19.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.361+0000 7f8ebf7fe640 1 -- 172.21.15.98:0/262138616 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec00a9d90 con 0x7f8ec8071ac0 2024-09-19T08:43:19.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.361+0000 7f8ecf019640 1 -- 172.21.15.98:0/262138616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ec8071ac0 msgr2=0x7f8ec80b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:19.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.361+0000 7f8ecf019640 1 --2- 172.21.15.98:0/262138616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ec8071ac0 0x7f8ec80b3410 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0009d90 tx=0x7f8ec0020cf0 comp rx=0 tx=0).stop 2024-09-19T08:43:19.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.361+0000 7f8ecf019640 1 -- 172.21.15.98:0/262138616 shutdown_connections 2024-09-19T08:43:19.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.361+0000 7f8ecf019640 1 --2- 172.21.15.98:0/262138616 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ec8071ac0 0x7f8ec80b3410 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:19.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.361+0000 7f8ecf019640 1 --2- 172.21.15.98:0/262138616 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ec8145b70 0x7f8ec806bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:19.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.361+0000 7f8ecf019640 1 -- 172.21.15.98:0/262138616 >> 172.21.15.98:0/262138616 conn(0x7f8ec80b2010 msgr2=0x7f8ec80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:19.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.361+0000 7f8ecf019640 1 -- 172.21.15.98:0/262138616 shutdown_connections 2024-09-19T08:43:19.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.361+0000 7f8ecf019640 1 -- 172.21.15.98:0/262138616 wait complete. 2024-09-19T08:43:19.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.361+0000 7f8ecf019640 1 Processor -- start 2024-09-19T08:43:19.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.361+0000 7f8ecf019640 1 -- start start 2024-09-19T08:43:19.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.361+0000 7f8ecf019640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ec806c1d0 0x7f8ec813bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:19.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.361+0000 7f8ecf019640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ec8071ac0 0x7f8ec80b0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:19.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.361+0000 7f8ecf019640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ec8145b70 0x7f8ec80b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:19.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.361+0000 7f8ecf019640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec80b42e0 con 0x7f8ec8071ac0 2024-09-19T08:43:19.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.361+0000 7f8ecf019640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec80b4160 con 0x7f8ec806c1d0 2024-09-19T08:43:19.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.361+0000 7f8ecf019640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8ec80b4460 con 0x7f8ec8145b70 2024-09-19T08:43:19.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.361+0000 7f8ebffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ec8071ac0 0x7f8ec80b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:19.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.361+0000 7f8ebffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ec8071ac0 0x7f8ec80b0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41342/0 (socket says 172.21.15.98:41342) 2024-09-19T08:43:19.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.361+0000 7f8ebffff640 1 -- 172.21.15.98:0/1292768402 learned_addr learned my addr 172.21.15.98:0/1292768402 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:19.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.361+0000 7f8ebffff640 1 -- 172.21.15.98:0/1292768402 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ec8145b70 msgr2=0x7f8ec80b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:19.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.361+0000 7f8ebffff640 1 --2- 172.21.15.98:0/1292768402 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ec8145b70 0x7f8ec80b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:19.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.361+0000 7f8ebffff640 1 -- 172.21.15.98:0/1292768402 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ec806c1d0 msgr2=0x7f8ec813bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:19.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.361+0000 7f8ebffff640 1 --2- 172.21.15.98:0/1292768402 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ec806c1d0 0x7f8ec813bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:19.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.361+0000 7f8ebffff640 1 -- 172.21.15.98:0/1292768402 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec813c850 con 0x7f8ec8071ac0 2024-09-19T08:43:19.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.361+0000 7f8ebffff640 1 --2- 172.21.15.98:0/1292768402 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ec8071ac0 0x7f8ec80b0900 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f8eb800c430 tx=0x7f8eb800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:19.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.365+0000 7f8ebd7fa640 1 -- 172.21.15.98:0/1292768402 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb8017070 con 0x7f8ec8071ac0 2024-09-19T08:43:19.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.365+0000 7f8ebd7fa640 1 -- 172.21.15.98:0/1292768402 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eb8005cf0 con 0x7f8ec8071ac0 2024-09-19T08:43:19.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.365+0000 7f8ecf019640 1 -- 172.21.15.98:0/1292768402 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ec813cb30 con 0x7f8ec8071ac0 2024-09-19T08:43:19.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.365+0000 7f8ebd7fa640 1 -- 172.21.15.98:0/1292768402 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb80072f0 con 0x7f8ec8071ac0 2024-09-19T08:43:19.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.365+0000 7f8ecf019640 1 -- 172.21.15.98:0/1292768402 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ec813d0f0 con 0x7f8ec8071ac0 2024-09-19T08:43:19.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.365+0000 7f8ecf019640 1 -- 172.21.15.98:0/1292768402 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ec80b3410 con 0x7f8ec8071ac0 2024-09-19T08:43:19.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.365+0000 7f8ebd7fa640 1 -- 172.21.15.98:0/1292768402 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f8eb801a070 con 0x7f8ec8071ac0 2024-09-19T08:43:19.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.365+0000 7f8ebd7fa640 1 --2- 172.21.15.98:0/1292768402 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8eb40679e0 0x7f8eb4069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:19.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.365+0000 7f8ebd7fa640 1 -- 172.21.15.98:0/1292768402 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f8eb8085150 con 0x7f8ec8071ac0 2024-09-19T08:43:19.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.365+0000 7f8eccdb5640 1 --2- 172.21.15.98:0/1292768402 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8eb40679e0 0x7f8eb4069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:19.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.365+0000 7f8eccdb5640 1 --2- 172.21.15.98:0/1292768402 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8eb40679e0 0x7f8eb4069ea0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400b850 tx=0x7f8ec4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:19.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.369+0000 7f8ebd7fa640 1 -- 172.21.15.98:0/1292768402 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f8eb8055c10 con 0x7f8ec8071ac0 2024-09-19T08:43:19.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:19.569+0000 7f8ecf019640 1 -- 172.21.15.98:0/1292768402 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f8ec80b2af0 con 0x7f8ec8071ac0 2024-09-19T08:43:20.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.565+0000 7f8ebd7fa640 1 -- 172.21.15.98:0/1292768402 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v46) ==== 118+0+0 (secure 0 0 0) 0x7f8eb8055510 con 0x7f8ec8071ac0 2024-09-19T08:43:20.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.569+0000 7f8ecf019640 1 -- 172.21.15.98:0/1292768402 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8eb40679e0 msgr2=0x7f8eb4069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:20.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.569+0000 7f8ecf019640 1 --2- 172.21.15.98:0/1292768402 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8eb40679e0 0x7f8eb4069ea0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400b850 tx=0x7f8ec4008040 comp rx=0 tx=0).stop 2024-09-19T08:43:20.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.569+0000 7f8ecf019640 1 -- 172.21.15.98:0/1292768402 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ec8071ac0 msgr2=0x7f8ec80b0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:20.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.569+0000 7f8ecf019640 1 --2- 172.21.15.98:0/1292768402 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ec8071ac0 0x7f8ec80b0900 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f8eb800c430 tx=0x7f8eb800c8f0 comp rx=0 tx=0).stop 2024-09-19T08:43:20.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.569+0000 7f8ecf019640 1 -- 172.21.15.98:0/1292768402 shutdown_connections 2024-09-19T08:43:20.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.569+0000 7f8ecf019640 1 --2- 172.21.15.98:0/1292768402 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8eb40679e0 0x7f8eb4069ea0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:20.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.569+0000 7f8ecf019640 1 --2- 172.21.15.98:0/1292768402 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ec8145b70 0x7f8ec80b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:20.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.569+0000 7f8ecf019640 1 --2- 172.21.15.98:0/1292768402 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ec8071ac0 0x7f8ec80b0900 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:20.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.569+0000 7f8ecf019640 1 --2- 172.21.15.98:0/1292768402 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8ec806c1d0 0x7f8ec813bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:20.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.569+0000 7f8ecf019640 1 -- 172.21.15.98:0/1292768402 >> 172.21.15.98:0/1292768402 conn(0x7f8ec80b2010 msgr2=0x7f8ec8072130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:20.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.569+0000 7f8ecf019640 1 -- 172.21.15.98:0/1292768402 shutdown_connections 2024-09-19T08:43:20.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.569+0000 7f8ecf019640 1 -- 172.21.15.98:0/1292768402 wait complete. 2024-09-19T08:43:20.590 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-19T08:43:20.590 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-19T08:43:20.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.717+0000 7fbd48a7e640 1 Processor -- start 2024-09-19T08:43:20.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.717+0000 7fbd48a7e640 1 -- start start 2024-09-19T08:43:20.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.717+0000 7fbd48a7e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbd44147cb0 0x7fbd4406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:20.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.717+0000 7fbd48a7e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd4406c1d0 0x7fbd44162200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:20.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.717+0000 7fbd48a7e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd44162f90 con 0x7fbd44162740 2024-09-19T08:43:20.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.717+0000 7fbd48a7e640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fbd4406c640 con 0x7fbd44147cb0 2024-09-19T08:43:20.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.717+0000 7fbd48a7e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbd440cee10 con 0x7fbd4406c1d0 2024-09-19T08:43:20.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.717+0000 7fbd4259c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbd44147cb0 0x7fbd4406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:20.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.717+0000 7fbd4259c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbd44147cb0 0x7fbd4406bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41344/0 (socket says 172.21.15.98:41344) 2024-09-19T08:43:20.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.717+0000 7fbd4259c640 1 -- 172.21.15.98:0/1812197322 learned_addr learned my addr 172.21.15.98:0/1812197322 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:20.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.717+0000 7fbd41d9b640 1 --2- 172.21.15.98:0/1812197322 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd4406c1d0 0x7fbd44162200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:20.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.717+0000 7fbd4259c640 1 -- 172.21.15.98:0/1812197322 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd4406c1d0 msgr2=0x7fbd44162200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:20.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.717+0000 7fbd4259c640 1 --2- 172.21.15.98:0/1812197322 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd4406c1d0 0x7fbd44162200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:20.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.717+0000 7fbd4259c640 1 -- 172.21.15.98:0/1812197322 >> v1:172.21.15.139:6789/0 conn(0x7fbd44162740 legacy=0x7fbd440ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:20.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.717+0000 7fbd4259c640 1 -- 172.21.15.98:0/1812197322 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd440cf040 con 0x7fbd44147cb0 2024-09-19T08:43:20.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.717+0000 7fbd41d9b640 1 --2- 172.21.15.98:0/1812197322 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd4406c1d0 0x7fbd44162200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:20.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.721+0000 7fbd4259c640 1 --2- 172.21.15.98:0/1812197322 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd44147cb0 0x7fbd4406bc90 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fbd38009870 tx=0x7fbd3801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed79c043f2aceaf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:20.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.721+0000 7fbd4159a640 1 -- 172.21.15.98:0/1812197322 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd380b0070 con 0x7fbd44147cb0 2024-09-19T08:43:20.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.721+0000 7fbd4159a640 1 -- 172.21.15.98:0/1812197322 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd380ab400 con 0x7fbd44147cb0 2024-09-19T08:43:20.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.721+0000 7fbd4159a640 1 -- 172.21.15.98:0/1812197322 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd380ab7d0 con 0x7fbd44147cb0 2024-09-19T08:43:20.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.721+0000 7fbd48a7e640 1 -- 172.21.15.98:0/1812197322 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd44147cb0 msgr2=0x7fbd4406bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:20.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.721+0000 7fbd48a7e640 1 --2- 172.21.15.98:0/1812197322 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd44147cb0 0x7fbd4406bc90 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fbd38009870 tx=0x7fbd3801c490 comp rx=0 tx=0).stop 2024-09-19T08:43:20.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.721+0000 7fbd48a7e640 1 -- 172.21.15.98:0/1812197322 shutdown_connections 2024-09-19T08:43:20.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.721+0000 7fbd48a7e640 1 --2- 172.21.15.98:0/1812197322 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd4406c1d0 0x7fbd44162200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:20.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.721+0000 7fbd48a7e640 1 --2- 172.21.15.98:0/1812197322 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd44147cb0 0x7fbd4406bc90 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:20.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.721+0000 7fbd48a7e640 1 -- 172.21.15.98:0/1812197322 >> 172.21.15.98:0/1812197322 conn(0x7fbd440b2010 msgr2=0x7fbd440b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:20.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.721+0000 7fbd48a7e640 1 -- 172.21.15.98:0/1812197322 shutdown_connections 2024-09-19T08:43:20.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.721+0000 7fbd48a7e640 1 -- 172.21.15.98:0/1812197322 wait complete. 2024-09-19T08:43:20.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.725+0000 7fbd48a7e640 1 Processor -- start 2024-09-19T08:43:20.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.725+0000 7fbd48a7e640 1 -- start start 2024-09-19T08:43:20.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.725+0000 7fbd48a7e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd4406c1d0 0x7fbd4415de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:20.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.725+0000 7fbd48a7e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd44162740 0x7fbd4415efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:20.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.725+0000 7fbd48a7e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd44160960 0x7fbd441353c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:20.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.725+0000 7fbd48a7e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbd44162d10 con 0x7fbd44162740 2024-09-19T08:43:20.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.725+0000 7fbd48a7e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbd44162b90 con 0x7fbd44160960 2024-09-19T08:43:20.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.725+0000 7fbd48a7e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbd440cfdc0 con 0x7fbd4406c1d0 2024-09-19T08:43:20.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.725+0000 7fbd41d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd44162740 0x7fbd4415efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:20.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.725+0000 7fbd41d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd44162740 0x7fbd4415efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41360/0 (socket says 172.21.15.98:41360) 2024-09-19T08:43:20.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.725+0000 7fbd41d9b640 1 -- 172.21.15.98:0/78826710 learned_addr learned my addr 172.21.15.98:0/78826710 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:20.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.725+0000 7fbd42d9d640 1 --2- 172.21.15.98:0/78826710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd44160960 0x7fbd441353c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:20.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.725+0000 7fbd4259c640 1 --2- 172.21.15.98:0/78826710 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd4406c1d0 0x7fbd4415de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:20.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.725+0000 7fbd41d9b640 1 -- 172.21.15.98:0/78826710 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd4406c1d0 msgr2=0x7fbd4415de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:20.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.725+0000 7fbd41d9b640 1 --2- 172.21.15.98:0/78826710 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd4406c1d0 0x7fbd4415de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:20.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.725+0000 7fbd41d9b640 1 -- 172.21.15.98:0/78826710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd44160960 msgr2=0x7fbd441353c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:20.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.725+0000 7fbd41d9b640 1 --2- 172.21.15.98:0/78826710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd44160960 0x7fbd441353c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:20.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.725+0000 7fbd41d9b640 1 -- 172.21.15.98:0/78826710 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd4415f8c0 con 0x7fbd44162740 2024-09-19T08:43:20.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.725+0000 7fbd41d9b640 1 --2- 172.21.15.98:0/78826710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd44162740 0x7fbd4415efc0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fbd3400c7c0 tx=0x7fbd3400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:20.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.725+0000 7fbd32ffd640 1 -- 172.21.15.98:0/78826710 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd34007a30 con 0x7fbd44162740 2024-09-19T08:43:20.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.725+0000 7fbd32ffd640 1 -- 172.21.15.98:0/78826710 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd34007bd0 con 0x7fbd44162740 2024-09-19T08:43:20.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.725+0000 7fbd32ffd640 1 -- 172.21.15.98:0/78826710 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd340113f0 con 0x7fbd44162740 2024-09-19T08:43:20.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.725+0000 7fbd48a7e640 1 -- 172.21.15.98:0/78826710 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd4415fba0 con 0x7fbd44162740 2024-09-19T08:43:20.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.725+0000 7fbd48a7e640 1 -- 172.21.15.98:0/78826710 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd44160160 con 0x7fbd44162740 2024-09-19T08:43:20.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.725+0000 7fbd48a7e640 1 -- 172.21.15.98:0/78826710 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd4406bc90 con 0x7fbd44162740 2024-09-19T08:43:20.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.729+0000 7fbd32ffd640 1 -- 172.21.15.98:0/78826710 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fbd34011590 con 0x7fbd44162740 2024-09-19T08:43:20.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.729+0000 7fbd32ffd640 1 --2- 172.21.15.98:0/78826710 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbd24067a10 0x7fbd24069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:20.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.729+0000 7fbd32ffd640 1 -- 172.21.15.98:0/78826710 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fbd34085fb0 con 0x7fbd44162740 2024-09-19T08:43:20.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.729+0000 7fbd4259c640 1 --2- 172.21.15.98:0/78826710 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbd24067a10 0x7fbd24069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:20.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.729+0000 7fbd4259c640 1 --2- 172.21.15.98:0/78826710 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbd24067a10 0x7fbd24069ed0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbd380ad000 tx=0x7fbd38008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:20.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.733+0000 7fbd32ffd640 1 -- 172.21.15.98:0/78826710 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fbd34015020 con 0x7fbd44162740 2024-09-19T08:43:20.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:20.929+0000 7fbd48a7e640 1 -- 172.21.15.98:0/78826710 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fbd4406cad0 con 0x7fbd44162740 2024-09-19T08:43:21.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.565+0000 7fbd32ffd640 1 -- 172.21.15.98:0/78826710 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v47) ==== 122+0+0 (secure 0 0 0) 0x7fbd34056310 con 0x7fbd44162740 2024-09-19T08:43:21.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.569+0000 7fbd48a7e640 1 -- 172.21.15.98:0/78826710 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbd24067a10 msgr2=0x7fbd24069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:21.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.569+0000 7fbd48a7e640 1 --2- 172.21.15.98:0/78826710 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbd24067a10 0x7fbd24069ed0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbd380ad000 tx=0x7fbd38008040 comp rx=0 tx=0).stop 2024-09-19T08:43:21.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.569+0000 7fbd48a7e640 1 -- 172.21.15.98:0/78826710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd44162740 msgr2=0x7fbd4415efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:21.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.569+0000 7fbd48a7e640 1 --2- 172.21.15.98:0/78826710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd44162740 0x7fbd4415efc0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fbd3400c7c0 tx=0x7fbd3400cc80 comp rx=0 tx=0).stop 2024-09-19T08:43:21.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.569+0000 7fbd48a7e640 1 -- 172.21.15.98:0/78826710 shutdown_connections 2024-09-19T08:43:21.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.569+0000 7fbd48a7e640 1 --2- 172.21.15.98:0/78826710 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbd24067a10 0x7fbd24069ed0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:21.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.569+0000 7fbd48a7e640 1 --2- 172.21.15.98:0/78826710 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbd44160960 0x7fbd441353c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:21.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.569+0000 7fbd48a7e640 1 --2- 172.21.15.98:0/78826710 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbd44162740 0x7fbd4415efc0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:21.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.569+0000 7fbd48a7e640 1 --2- 172.21.15.98:0/78826710 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbd4406c1d0 0x7fbd4415de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:21.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.569+0000 7fbd48a7e640 1 -- 172.21.15.98:0/78826710 >> 172.21.15.98:0/78826710 conn(0x7fbd440b2010 msgr2=0x7fbd44144d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:21.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.569+0000 7fbd48a7e640 1 -- 172.21.15.98:0/78826710 shutdown_connections 2024-09-19T08:43:21.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.569+0000 7fbd48a7e640 1 -- 172.21.15.98:0/78826710 wait complete. 2024-09-19T08:43:21.591 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-19T08:43:21.591 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:43:21.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.717+0000 7faad6b79640 1 Processor -- start 2024-09-19T08:43:21.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.717+0000 7faad6b79640 1 -- start start 2024-09-19T08:43:21.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.717+0000 7faad6b79640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faad015c550 0x7faad006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:21.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.717+0000 7faad6b79640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faad006d310 con 0x7faad006c1d0 2024-09-19T08:43:21.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.717+0000 7faad6b79640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faad006d5f0 con 0x7faad00cfad0 2024-09-19T08:43:21.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.717+0000 7faad6b79640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7faad006c610 con 0x7faad015c550 2024-09-19T08:43:21.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.717+0000 7faacffff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7faad006c1d0 0x7faad00cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40052/0 (socket says 172.21.15.98:40052) 2024-09-19T08:43:21.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.717+0000 7faacffff640 1 -- 172.21.15.98:0/767249789 learned_addr learned my addr 172.21.15.98:0/767249789 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:21.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.717+0000 7faacf7fe640 1 -- 172.21.15.98:0/767249789 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4072519577 0 0) 0x7faad006d310 con 0x7faad006c1d0 2024-09-19T08:43:21.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.717+0000 7faad4915640 1 --2- 172.21.15.98:0/767249789 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faad015c550 0x7faad006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:21.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.717+0000 7faacf7fe640 1 -- 172.21.15.98:0/767249789 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faab8003530 con 0x7faad006c1d0 2024-09-19T08:43:21.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.717+0000 7faacf7fe640 1 -- 172.21.15.98:0/767249789 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4222474258 0 0) 0x7faab8003530 con 0x7faad006c1d0 2024-09-19T08:43:21.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.717+0000 7faacf7fe640 1 -- 172.21.15.98:0/767249789 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faad015c550 msgr2=0x7faad006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:21.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.717+0000 7faacf7fe640 1 --2- 172.21.15.98:0/767249789 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faad015c550 0x7faad006bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:21.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.717+0000 7faacf7fe640 1 -- 172.21.15.98:0/767249789 >> v1:172.21.15.139:6789/0 conn(0x7faad00cfad0 legacy=0x7faad00723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:21.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.717+0000 7faacf7fe640 1 -- 172.21.15.98:0/767249789 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad006d8d0 con 0x7faad006c1d0 2024-09-19T08:43:21.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.717+0000 7faacf7fe640 1 -- 172.21.15.98:0/767249789 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7faac4001680 con 0x7faad006c1d0 2024-09-19T08:43:21.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.717+0000 7faacf7fe640 1 -- 172.21.15.98:0/767249789 >> v1:172.21.15.98:6789/0 conn(0x7faad006c1d0 legacy=0x7faad00cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:21.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.717+0000 7faacf7fe640 1 --2- 172.21.15.98:0/767249789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faab80039f0 0x7faab8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:21.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.717+0000 7faacf7fe640 1 --2- 172.21.15.98:0/767249789 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faab8004810 0x7faab8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:21.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.717+0000 7faacf7fe640 1 --2- 172.21.15.98:0/767249789 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faad015c550 0x7faab8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:21.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.717+0000 7faacf7fe640 1 -- 172.21.15.98:0/767249789 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faad006c610 con 0x7faab8004810 2024-09-19T08:43:21.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.717+0000 7faacf7fe640 1 -- 172.21.15.98:0/767249789 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faab800bb00 con 0x7faab80039f0 2024-09-19T08:43:21.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.717+0000 7faacf7fe640 1 -- 172.21.15.98:0/767249789 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7faab800bd00 con 0x7faad015c550 2024-09-19T08:43:21.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.717+0000 7faacffff640 1 --2- 172.21.15.98:0/767249789 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faab8004810 0x7faab8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:21.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.717+0000 7faad4915640 1 --2- 172.21.15.98:0/767249789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faab80039f0 0x7faab8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:21.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.717+0000 7faad5116640 1 --2- 172.21.15.98:0/767249789 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faad015c550 0x7faab8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:21.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.717+0000 7faacffff640 1 -- 172.21.15.98:0/767249789 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faad015c550 msgr2=0x7faab8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:21.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.717+0000 7faacffff640 1 --2- 172.21.15.98:0/767249789 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faad015c550 0x7faab8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:21.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.717+0000 7faacffff640 1 -- 172.21.15.98:0/767249789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faab80039f0 msgr2=0x7faab8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:21.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.717+0000 7faacffff640 1 --2- 172.21.15.98:0/767249789 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faab80039f0 0x7faab8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:21.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.717+0000 7faacffff640 1 -- 172.21.15.98:0/767249789 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab8003530 con 0x7faab8004810 2024-09-19T08:43:21.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.717+0000 7faad5116640 1 --2- 172.21.15.98:0/767249789 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faad015c550 0x7faab8005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:21.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.721+0000 7faacffff640 1 --2- 172.21.15.98:0/767249789 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faab8004810 0x7faab8004c90 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7faac400c190 tx=0x7faac401f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd022ec5e0943a03 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:21.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.721+0000 7faacf7fe640 1 -- 172.21.15.98:0/767249789 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac4007d00 con 0x7faab8004810 2024-09-19T08:43:21.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.721+0000 7faacf7fe640 1 -- 172.21.15.98:0/767249789 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faac4005b00 con 0x7faab8004810 2024-09-19T08:43:21.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.721+0000 7faacf7fe640 1 -- 172.21.15.98:0/767249789 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac4007070 con 0x7faab8004810 2024-09-19T08:43:21.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.721+0000 7faad6b79640 1 -- 172.21.15.98:0/767249789 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faab8004810 msgr2=0x7faab8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:21.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.721+0000 7faad6b79640 1 --2- 172.21.15.98:0/767249789 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faab8004810 0x7faab8004c90 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7faac400c190 tx=0x7faac401f390 comp rx=0 tx=0).stop 2024-09-19T08:43:21.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.721+0000 7faad5116640 1 -- 172.21.15.98:0/767249789 reap_dead start 2024-09-19T08:43:21.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.721+0000 7faad6b79640 1 -- 172.21.15.98:0/767249789 shutdown_connections 2024-09-19T08:43:21.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.721+0000 7faad6b79640 1 -- 172.21.15.98:0/767249789 >> 172.21.15.98:0/767249789 conn(0x7faad00b2010 msgr2=0x7faad00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:21.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.721+0000 7faad6b79640 1 -- 172.21.15.98:0/767249789 shutdown_connections 2024-09-19T08:43:21.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.721+0000 7faad6b79640 1 -- 172.21.15.98:0/767249789 wait complete. 2024-09-19T08:43:21.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.721+0000 7faad6b79640 1 Processor -- start 2024-09-19T08:43:21.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.721+0000 7faad6b79640 1 -- start start 2024-09-19T08:43:21.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.725+0000 7faad6b79640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faab8004810 0x7faad0141d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:21.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.725+0000 7faad6b79640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faad00b4420 0x7faad00b4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:21.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.725+0000 7faad6b79640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faad01e4b00 0x7faad01e4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:21.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.725+0000 7faad6b79640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faad015cab0 con 0x7faab8004810 2024-09-19T08:43:21.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.725+0000 7faad6b79640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faad015c930 con 0x7faad01e4b00 2024-09-19T08:43:21.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.725+0000 7faad6b79640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7faad006bc90 con 0x7faad00b4420 2024-09-19T08:43:21.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.725+0000 7faad4915640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faab8004810 0x7faad0141d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:21.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.725+0000 7faad4915640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faab8004810 0x7faad0141d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41388/0 (socket says 172.21.15.98:41388) 2024-09-19T08:43:21.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.725+0000 7faad4915640 1 -- 172.21.15.98:0/2718308292 learned_addr learned my addr 172.21.15.98:0/2718308292 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:21.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.725+0000 7faacffff640 1 --2- 172.21.15.98:0/2718308292 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faad00b4420 0x7faad00b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:21.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.725+0000 7faad4915640 1 -- 172.21.15.98:0/2718308292 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faad00b4420 msgr2=0x7faad00b4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:21.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.725+0000 7faad5116640 1 --2- 172.21.15.98:0/2718308292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faad01e4b00 0x7faad01e4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:21.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.725+0000 7faad4915640 1 --2- 172.21.15.98:0/2718308292 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faad00b4420 0x7faad00b4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:21.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.725+0000 7faad4915640 1 -- 172.21.15.98:0/2718308292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faad01e4b00 msgr2=0x7faad01e4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:21.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.725+0000 7faad4915640 1 --2- 172.21.15.98:0/2718308292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faad01e4b00 0x7faad01e4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:21.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.725+0000 7faad4915640 1 -- 172.21.15.98:0/2718308292 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad015d320 con 0x7faab8004810 2024-09-19T08:43:21.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.725+0000 7faad5116640 1 --2- 172.21.15.98:0/2718308292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faad01e4b00 0x7faad01e4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:21.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.725+0000 7faad4915640 1 --2- 172.21.15.98:0/2718308292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faab8004810 0x7faad0141d60 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7faac000f680 tx=0x7faac000fb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:21.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.725+0000 7faacd7fa640 1 -- 172.21.15.98:0/2718308292 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac0019020 con 0x7faab8004810 2024-09-19T08:43:21.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.725+0000 7faacd7fa640 1 -- 172.21.15.98:0/2718308292 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faac000d960 con 0x7faab8004810 2024-09-19T08:43:21.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.725+0000 7faacd7fa640 1 -- 172.21.15.98:0/2718308292 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac000dd50 con 0x7faab8004810 2024-09-19T08:43:21.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.725+0000 7faad6b79640 1 -- 172.21.15.98:0/2718308292 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faad01e54c0 con 0x7faab8004810 2024-09-19T08:43:21.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.725+0000 7faad6b79640 1 -- 172.21.15.98:0/2718308292 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faad015d8a0 con 0x7faab8004810 2024-09-19T08:43:21.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.725+0000 7faad6b79640 1 -- 172.21.15.98:0/2718308292 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faad006bf80 con 0x7faab8004810 2024-09-19T08:43:21.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.725+0000 7faacd7fa640 1 -- 172.21.15.98:0/2718308292 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7faac001c020 con 0x7faab8004810 2024-09-19T08:43:21.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.725+0000 7faacd7fa640 1 --2- 172.21.15.98:0/2718308292 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7faabc0679e0 0x7faabc069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:21.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.725+0000 7faacd7fa640 1 -- 172.21.15.98:0/2718308292 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7faac0086300 con 0x7faab8004810 2024-09-19T08:43:21.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.725+0000 7faacffff640 1 --2- 172.21.15.98:0/2718308292 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7faabc0679e0 0x7faabc069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:21.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.729+0000 7faacffff640 1 --2- 172.21.15.98:0/2718308292 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7faabc0679e0 0x7faabc069ea0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faac40ae000 tx=0x7faad006d4e0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:21.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.729+0000 7faacd7fa640 1 -- 172.21.15.98:0/2718308292 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7faac0023030 con 0x7faab8004810 2024-09-19T08:43:21.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.925+0000 7faad6b79640 1 -- 172.21.15.98:0/2718308292 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faad0157fc0 con 0x7faab8004810 2024-09-19T08:43:21.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.929+0000 7faacd7fa640 1 -- 172.21.15.98:0/2718308292 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5228 (secure 0 0 0) 0x7faac0056d30 con 0x7faab8004810 2024-09-19T08:43:21.928 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:21.928 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":47,"btime":"2024-09-19T08:43:21:568061+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3182634136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3182634136},{"type":"v1","addr":"172.21.15.139:6833","nonce":3182634136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5830,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1554653172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1554653172},{"type":"v1","addr":"172.21.15.139:6835","nonce":1554653172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6624,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1092838020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1092838020},{"type":"v1","addr":"172.21.15.98:6833","nonce":1092838020}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:43:17.920717+0000","modified":"2024-09-19T08:43:20.933845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6627},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6627":{"gid":6627,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/837817273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":837817273},{"type":"v1","addr":"172.21.15.98:6835","nonce":837817273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6627,"qdb_cluster":[6627]},"id":6}]} 2024-09-19T08:43:21.929 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 47 2024-09-19T08:43:21.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.929+0000 7faad6b79640 1 -- 172.21.15.98:0/2718308292 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7faabc0679e0 msgr2=0x7faabc069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:21.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.929+0000 7faad6b79640 1 --2- 172.21.15.98:0/2718308292 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7faabc0679e0 0x7faabc069ea0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faac40ae000 tx=0x7faad006d4e0 comp rx=0 tx=0).stop 2024-09-19T08:43:21.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.929+0000 7faad6b79640 1 -- 172.21.15.98:0/2718308292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faab8004810 msgr2=0x7faad0141d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:21.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.929+0000 7faad6b79640 1 --2- 172.21.15.98:0/2718308292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faab8004810 0x7faad0141d60 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7faac000f680 tx=0x7faac000fb40 comp rx=0 tx=0).stop 2024-09-19T08:43:21.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.929+0000 7faad6b79640 1 -- 172.21.15.98:0/2718308292 shutdown_connections 2024-09-19T08:43:21.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.929+0000 7faad6b79640 1 --2- 172.21.15.98:0/2718308292 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7faabc0679e0 0x7faabc069ea0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:21.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.929+0000 7faad6b79640 1 --2- 172.21.15.98:0/2718308292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faad01e4b00 0x7faad01e4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:21.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.929+0000 7faad6b79640 1 --2- 172.21.15.98:0/2718308292 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faad00b4420 0x7faad00b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:21.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.929+0000 7faad6b79640 1 --2- 172.21.15.98:0/2718308292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faab8004810 0x7faad0141d60 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:21.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.929+0000 7faad6b79640 1 -- 172.21.15.98:0/2718308292 >> 172.21.15.98:0/2718308292 conn(0x7faad00b2010 msgr2=0x7faad0072d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:21.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.929+0000 7faad6b79640 1 -- 172.21.15.98:0/2718308292 shutdown_connections 2024-09-19T08:43:21.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:21.929+0000 7faad6b79640 1 -- 172.21.15.98:0/2718308292 wait complete. 2024-09-19T08:43:21.948 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:43:22.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.069+0000 7fa2e3e8a640 1 Processor -- start 2024-09-19T08:43:22.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.069+0000 7fa2e3e8a640 1 -- start start 2024-09-19T08:43:22.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.069+0000 7fa2e3e8a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa2dc158fd0 0x7fa2dc0b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:22.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.069+0000 7fa2e3e8a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2dc0b4360 0x7fa2dc06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:22.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.069+0000 7fa2e3e8a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2dc0ccc60 con 0x7fa2dc06c1d0 2024-09-19T08:43:22.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.069+0000 7fa2e3e8a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fa2dc1593b0 con 0x7fa2dc158fd0 2024-09-19T08:43:22.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.069+0000 7fa2e3e8a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa2dc06c600 con 0x7fa2dc0b4360 2024-09-19T08:43:22.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.069+0000 7fa2e1c26640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa2dc158fd0 0x7fa2dc0b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:22.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.069+0000 7fa2e1425640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2dc0b4360 0x7fa2dc06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:22.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.069+0000 7fa2e1c26640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa2dc158fd0 0x7fa2dc0b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:48894/0 (socket says 172.21.15.98:48894) 2024-09-19T08:43:22.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.069+0000 7fa2e1c26640 1 -- 172.21.15.98:0/1896498984 learned_addr learned my addr 172.21.15.98:0/1896498984 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:22.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.069+0000 7fa2e2427640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fa2dc06c1d0 0x7fa2dc15efc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:37116/0 (socket says 172.21.15.98:37116) 2024-09-19T08:43:22.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.069+0000 7fa2e1c26640 1 -- 172.21.15.98:0/1896498984 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2dc0b4360 msgr2=0x7fa2dc06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:22.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.069+0000 7fa2e1c26640 1 --2- 172.21.15.98:0/1896498984 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2dc0b4360 0x7fa2dc06bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:22.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.069+0000 7fa2e1c26640 1 -- 172.21.15.98:0/1896498984 >> v1:172.21.15.139:6789/0 conn(0x7fa2dc06c1d0 legacy=0x7fa2dc15efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:22.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.069+0000 7fa2e0c24640 1 -- 172.21.15.98:0/1896498984 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4180104740 0 0) 0x7fa2dc0ccc60 con 0x7fa2dc06c1d0 2024-09-19T08:43:22.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.069+0000 7fa2e1c26640 1 -- 172.21.15.98:0/1896498984 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2dc0ccf40 con 0x7fa2dc158fd0 2024-09-19T08:43:22.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.073+0000 7fa2e1c26640 1 --2- 172.21.15.98:0/1896498984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2dc158fd0 0x7fa2dc0b3e20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009870 tx=0x7fa2d801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d04d05aaa02d48f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:22.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.073+0000 7fa2e0c24640 1 -- 172.21.15.98:0/1896498984 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d80aa600 con 0x7fa2dc158fd0 2024-09-19T08:43:22.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.073+0000 7fa2e0c24640 1 -- 172.21.15.98:0/1896498984 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2d80af070 con 0x7fa2dc158fd0 2024-09-19T08:43:22.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.073+0000 7fa2e0c24640 1 -- 172.21.15.98:0/1896498984 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d80aa9c0 con 0x7fa2dc158fd0 2024-09-19T08:43:22.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.073+0000 7fa2e3e8a640 1 -- 172.21.15.98:0/1896498984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2dc158fd0 msgr2=0x7fa2dc0b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:22.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.073+0000 7fa2e3e8a640 1 --2- 172.21.15.98:0/1896498984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2dc158fd0 0x7fa2dc0b3e20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009870 tx=0x7fa2d801c4a0 comp rx=0 tx=0).stop 2024-09-19T08:43:22.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.073+0000 7fa2e3e8a640 1 -- 172.21.15.98:0/1896498984 shutdown_connections 2024-09-19T08:43:22.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.073+0000 7fa2e3e8a640 1 --2- 172.21.15.98:0/1896498984 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2dc0b4360 0x7fa2dc06bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:22.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.073+0000 7fa2e3e8a640 1 --2- 172.21.15.98:0/1896498984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2dc158fd0 0x7fa2dc0b3e20 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:22.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.073+0000 7fa2e3e8a640 1 -- 172.21.15.98:0/1896498984 >> 172.21.15.98:0/1896498984 conn(0x7fa2dc0b2010 msgr2=0x7fa2dc0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:22.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.073+0000 7fa2e3e8a640 1 -- 172.21.15.98:0/1896498984 shutdown_connections 2024-09-19T08:43:22.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.073+0000 7fa2e3e8a640 1 -- 172.21.15.98:0/1896498984 wait complete. 2024-09-19T08:43:22.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.073+0000 7fa2e3e8a640 1 Processor -- start 2024-09-19T08:43:22.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.077+0000 7fa2e3e8a640 1 -- start start 2024-09-19T08:43:22.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.077+0000 7fa2e3e8a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2dc06c1d0 0x7fa2dc150720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:22.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.077+0000 7fa2e3e8a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2dc0b4360 0x7fa2dc151180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:22.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.077+0000 7fa2e3e8a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2dc158fd0 0x7fa2dc152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:22.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.077+0000 7fa2e3e8a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc15c920 con 0x7fa2dc0b4360 2024-09-19T08:43:22.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.077+0000 7fa2e3e8a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa2dc15c7a0 con 0x7fa2dc158fd0 2024-09-19T08:43:22.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.077+0000 7fa2e3e8a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa2dc06d170 con 0x7fa2dc06c1d0 2024-09-19T08:43:22.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.077+0000 7fa2e1425640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2dc0b4360 0x7fa2dc151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:22.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.077+0000 7fa2e1425640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2dc0b4360 0x7fa2dc151180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41396/0 (socket says 172.21.15.98:41396) 2024-09-19T08:43:22.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.077+0000 7fa2e1425640 1 -- 172.21.15.98:0/674827912 learned_addr learned my addr 172.21.15.98:0/674827912 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:22.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.077+0000 7fa2e1c26640 1 --2- 172.21.15.98:0/674827912 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2dc06c1d0 0x7fa2dc150720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:22.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.077+0000 7fa2e2427640 1 --2- 172.21.15.98:0/674827912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2dc158fd0 0x7fa2dc152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:22.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.077+0000 7fa2e1425640 1 -- 172.21.15.98:0/674827912 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2dc06c1d0 msgr2=0x7fa2dc150720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:22.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.077+0000 7fa2e1425640 1 --2- 172.21.15.98:0/674827912 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2dc06c1d0 0x7fa2dc150720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:22.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.077+0000 7fa2e1425640 1 -- 172.21.15.98:0/674827912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2dc158fd0 msgr2=0x7fa2dc152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:22.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.077+0000 7fa2e1425640 1 --2- 172.21.15.98:0/674827912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2dc158fd0 0x7fa2dc152cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:22.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.077+0000 7fa2e1425640 1 -- 172.21.15.98:0/674827912 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2dc150c60 con 0x7fa2dc0b4360 2024-09-19T08:43:22.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.077+0000 7fa2e2427640 1 --2- 172.21.15.98:0/674827912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2dc158fd0 0x7fa2dc152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:22.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.077+0000 7fa2e1425640 1 --2- 172.21.15.98:0/674827912 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2dc0b4360 0x7fa2dc151180 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00e960 tx=0x7fa2cc00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:22.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.077+0000 7fa2d27fc640 1 -- 172.21.15.98:0/674827912 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc017070 con 0x7fa2dc0b4360 2024-09-19T08:43:22.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.077+0000 7fa2d27fc640 1 -- 172.21.15.98:0/674827912 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2cc00ca00 con 0x7fa2dc0b4360 2024-09-19T08:43:22.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.077+0000 7fa2d27fc640 1 -- 172.21.15.98:0/674827912 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc00cdb0 con 0x7fa2dc0b4360 2024-09-19T08:43:22.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.077+0000 7fa2e3e8a640 1 -- 172.21.15.98:0/674827912 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2dc153a90 con 0x7fa2dc0b4360 2024-09-19T08:43:22.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.077+0000 7fa2e3e8a640 1 -- 172.21.15.98:0/674827912 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2dc154050 con 0x7fa2dc0b4360 2024-09-19T08:43:22.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.077+0000 7fa2e3e8a640 1 -- 172.21.15.98:0/674827912 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2dc06bc90 con 0x7fa2dc0b4360 2024-09-19T08:43:22.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.077+0000 7fa2d27fc640 1 -- 172.21.15.98:0/674827912 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fa2cc01a070 con 0x7fa2dc0b4360 2024-09-19T08:43:22.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.081+0000 7fa2d27fc640 1 --2- 172.21.15.98:0/674827912 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa2c00679e0 0x7fa2c0069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:22.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.081+0000 7fa2d27fc640 1 -- 172.21.15.98:0/674827912 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fa2cc086220 con 0x7fa2dc0b4360 2024-09-19T08:43:22.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.081+0000 7fa2e1c26640 1 --2- 172.21.15.98:0/674827912 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa2c00679e0 0x7fa2c0069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:22.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.081+0000 7fa2e1c26640 1 --2- 172.21.15.98:0/674827912 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa2c00679e0 0x7fa2c0069ea0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa2d80ae000 tx=0x7fa2d80a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:22.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.085+0000 7fa2d27fc640 1 -- 172.21.15.98:0/674827912 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa2cc056c50 con 0x7fa2dc0b4360 2024-09-19T08:43:22.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.245+0000 7fa2e3e8a640 1 -- 172.21.15.98:0/674827912 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa2dc0b07a0 con 0x7fa2dc0b4360 2024-09-19T08:43:22.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.249+0000 7fa2d27fc640 1 -- 172.21.15.98:0/674827912 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+16720 (secure 0 0 0) 0x7fa2cc056550 con 0x7fa2dc0b4360 2024-09-19T08:43:22.249 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:22.250 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":83,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:43:19.555678+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:43:15.568647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-19T08:43:16.905534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:43:22.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.249+0000 7fa2e3e8a640 1 -- 172.21.15.98:0/674827912 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa2c00679e0 msgr2=0x7fa2c0069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:22.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.249+0000 7fa2e3e8a640 1 --2- 172.21.15.98:0/674827912 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa2c00679e0 0x7fa2c0069ea0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa2d80ae000 tx=0x7fa2d80a7000 comp rx=0 tx=0).stop 2024-09-19T08:43:22.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.249+0000 7fa2e3e8a640 1 -- 172.21.15.98:0/674827912 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2dc0b4360 msgr2=0x7fa2dc151180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:22.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.249+0000 7fa2e3e8a640 1 --2- 172.21.15.98:0/674827912 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2dc0b4360 0x7fa2dc151180 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00e960 tx=0x7fa2cc00ee20 comp rx=0 tx=0).stop 2024-09-19T08:43:22.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.249+0000 7fa2e3e8a640 1 -- 172.21.15.98:0/674827912 shutdown_connections 2024-09-19T08:43:22.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.249+0000 7fa2e3e8a640 1 --2- 172.21.15.98:0/674827912 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa2c00679e0 0x7fa2c0069ea0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:22.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.249+0000 7fa2e3e8a640 1 --2- 172.21.15.98:0/674827912 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2dc158fd0 0x7fa2dc152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:22.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.249+0000 7fa2e3e8a640 1 --2- 172.21.15.98:0/674827912 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2dc0b4360 0x7fa2dc151180 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:22.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.249+0000 7fa2e3e8a640 1 --2- 172.21.15.98:0/674827912 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2dc06c1d0 0x7fa2dc150720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:22.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.249+0000 7fa2e3e8a640 1 -- 172.21.15.98:0/674827912 >> 172.21.15.98:0/674827912 conn(0x7fa2dc0b2010 msgr2=0x7fa2dc15eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:22.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.249+0000 7fa2e3e8a640 1 -- 172.21.15.98:0/674827912 shutdown_connections 2024-09-19T08:43:22.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.249+0000 7fa2e3e8a640 1 -- 172.21.15.98:0/674827912 wait complete. 2024-09-19T08:43:22.268 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T08:43:22.268 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:43:22.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.385+0000 7f9b2c6f0640 1 Processor -- start 2024-09-19T08:43:22.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.385+0000 7f9b2c6f0640 1 -- start start 2024-09-19T08:43:22.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.385+0000 7f9b2c6f0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9b2415c550 0x7f9b2406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:22.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.385+0000 7f9b2c6f0640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9b240cfad0 0x7f9b240723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:22.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.385+0000 7f9b2c6f0640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b2406d310 con 0x7f9b2406c1d0 2024-09-19T08:43:22.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.385+0000 7f9b2c6f0640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f9b2406c610 con 0x7f9b240cfad0 2024-09-19T08:43:22.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.385+0000 7f9b2c6f0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9b2406d5f0 con 0x7f9b2415c550 2024-09-19T08:43:22.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.385+0000 7f9b2ac8d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9b240cfad0 0x7f9b240723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:22.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.385+0000 7f9b2ac8d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9b240cfad0 0x7f9b240723a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41406/0 (socket says 172.21.15.98:41406) 2024-09-19T08:43:22.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.385+0000 7f9b2ac8d640 1 -- 172.21.15.98:0/1525153151 learned_addr learned my addr 172.21.15.98:0/1525153151 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:22.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.385+0000 7f9b29c8b640 1 --1- 172.21.15.98:0/1525153151 >> v1:172.21.15.98:6789/0 conn(0x7f9b2406c1d0 0x7f9b240cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40068/0 (socket says 172.21.15.98:40068) 2024-09-19T08:43:22.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.385+0000 7f9b2a48c640 1 --2- 172.21.15.98:0/1525153151 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9b2415c550 0x7f9b2406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:22.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.385+0000 7f9b2ac8d640 1 -- 172.21.15.98:0/1525153151 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9b2415c550 msgr2=0x7f9b2406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:22.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.385+0000 7f9b2ac8d640 1 --2- 172.21.15.98:0/1525153151 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9b2415c550 0x7f9b2406bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:22.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.385+0000 7f9b2ac8d640 1 -- 172.21.15.98:0/1525153151 >> v1:172.21.15.98:6789/0 conn(0x7f9b2406c1d0 legacy=0x7f9b240cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:22.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.385+0000 7f9b2ac8d640 1 -- 172.21.15.98:0/1525153151 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b2406d8d0 con 0x7f9b240cfad0 2024-09-19T08:43:22.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.385+0000 7f9b2a48c640 1 --2- 172.21.15.98:0/1525153151 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9b2415c550 0x7f9b2406bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:22.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.385+0000 7f9b2ac8d640 1 --2- 172.21.15.98:0/1525153151 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b240cfad0 0x7f9b240723a0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c009870 tx=0x7f9b1c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac505f7bf01acbc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:22.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.389+0000 7f9b2948a640 1 -- 172.21.15.98:0/1525153151 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c0a7070 con 0x7f9b240cfad0 2024-09-19T08:43:22.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.389+0000 7f9b2948a640 1 -- 172.21.15.98:0/1525153151 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b1c0b0070 con 0x7f9b240cfad0 2024-09-19T08:43:22.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.389+0000 7f9b2948a640 1 -- 172.21.15.98:0/1525153151 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c0ab890 con 0x7f9b240cfad0 2024-09-19T08:43:22.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.389+0000 7f9b2c6f0640 1 -- 172.21.15.98:0/1525153151 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b240cfad0 msgr2=0x7f9b240723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:22.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.389+0000 7f9b2c6f0640 1 --2- 172.21.15.98:0/1525153151 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b240cfad0 0x7f9b240723a0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c009870 tx=0x7f9b1c01c560 comp rx=0 tx=0).stop 2024-09-19T08:43:22.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.389+0000 7f9b2c6f0640 1 -- 172.21.15.98:0/1525153151 shutdown_connections 2024-09-19T08:43:22.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.389+0000 7f9b2c6f0640 1 --2- 172.21.15.98:0/1525153151 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b240cfad0 0x7f9b240723a0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:22.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.389+0000 7f9b2c6f0640 1 --2- 172.21.15.98:0/1525153151 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9b2415c550 0x7f9b2406bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:22.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.389+0000 7f9b2c6f0640 1 -- 172.21.15.98:0/1525153151 >> 172.21.15.98:0/1525153151 conn(0x7f9b240b2010 msgr2=0x7f9b240b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:22.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.389+0000 7f9b2c6f0640 1 -- 172.21.15.98:0/1525153151 shutdown_connections 2024-09-19T08:43:22.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.389+0000 7f9b2c6f0640 1 -- 172.21.15.98:0/1525153151 wait complete. 2024-09-19T08:43:22.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.389+0000 7f9b2c6f0640 1 Processor -- start 2024-09-19T08:43:22.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.389+0000 7f9b2c6f0640 1 -- start start 2024-09-19T08:43:22.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.389+0000 7f9b2c6f0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9b2406c1d0 0x7f9b24149770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:22.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.389+0000 7f9b2c6f0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b240cfad0 0x7f9b24149200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:22.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.389+0000 7f9b2c6f0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b2415c550 0x7f9b240b4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:22.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.389+0000 7f9b2c6f0640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9b24073480 con 0x7f9b240cfad0 2024-09-19T08:43:22.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.389+0000 7f9b2c6f0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9b24073300 con 0x7f9b2415c550 2024-09-19T08:43:22.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.389+0000 7f9b2c6f0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9b24073600 con 0x7f9b2406c1d0 2024-09-19T08:43:22.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.389+0000 7f9b29c8b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b240cfad0 0x7f9b24149200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:22.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.389+0000 7f9b29c8b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b240cfad0 0x7f9b24149200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41412/0 (socket says 172.21.15.98:41412) 2024-09-19T08:43:22.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.389+0000 7f9b29c8b640 1 -- 172.21.15.98:0/505208360 learned_addr learned my addr 172.21.15.98:0/505208360 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:22.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.389+0000 7f9b2ac8d640 1 --2- 172.21.15.98:0/505208360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b2415c550 0x7f9b240b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:22.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.389+0000 7f9b2a48c640 1 --2- 172.21.15.98:0/505208360 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9b2406c1d0 0x7f9b24149770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:22.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.389+0000 7f9b29c8b640 1 -- 172.21.15.98:0/505208360 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9b2406c1d0 msgr2=0x7f9b24149770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:22.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.389+0000 7f9b29c8b640 1 --2- 172.21.15.98:0/505208360 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9b2406c1d0 0x7f9b24149770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:22.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.389+0000 7f9b29c8b640 1 -- 172.21.15.98:0/505208360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b2415c550 msgr2=0x7f9b240b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:22.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.389+0000 7f9b29c8b640 1 --2- 172.21.15.98:0/505208360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b2415c550 0x7f9b240b4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:22.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.389+0000 7f9b29c8b640 1 -- 172.21.15.98:0/505208360 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b24152980 con 0x7f9b240cfad0 2024-09-19T08:43:22.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.389+0000 7f9b29c8b640 1 --2- 172.21.15.98:0/505208360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b240cfad0 0x7f9b24149200 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000c760 tx=0x7f9b2000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:22.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.389+0000 7f9b1affd640 1 -- 172.21.15.98:0/505208360 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b20017070 con 0x7f9b240cfad0 2024-09-19T08:43:22.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.389+0000 7f9b1affd640 1 -- 172.21.15.98:0/505208360 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b2000a5e0 con 0x7f9b240cfad0 2024-09-19T08:43:22.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.389+0000 7f9b2c6f0640 1 -- 172.21.15.98:0/505208360 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b241526a0 con 0x7f9b240cfad0 2024-09-19T08:43:22.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.389+0000 7f9b1affd640 1 -- 172.21.15.98:0/505208360 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2000a990 con 0x7f9b240cfad0 2024-09-19T08:43:22.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.389+0000 7f9b2c6f0640 1 -- 172.21.15.98:0/505208360 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b240b3820 con 0x7f9b240cfad0 2024-09-19T08:43:22.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.393+0000 7f9b2c6f0640 1 -- 172.21.15.98:0/505208360 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b240723a0 con 0x7f9b240cfad0 2024-09-19T08:43:22.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.393+0000 7f9b1affd640 1 -- 172.21.15.98:0/505208360 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f9b2001a070 con 0x7f9b240cfad0 2024-09-19T08:43:22.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.393+0000 7f9b1affd640 1 --2- 172.21.15.98:0/505208360 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9b080679e0 0x7f9b08069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:22.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.393+0000 7f9b1affd640 1 -- 172.21.15.98:0/505208360 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f9b200851d0 con 0x7f9b240cfad0 2024-09-19T08:43:22.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.393+0000 7f9b2a48c640 1 --2- 172.21.15.98:0/505208360 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9b080679e0 0x7f9b08069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:22.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.393+0000 7f9b2a48c640 1 --2- 172.21.15.98:0/505208360 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9b080679e0 0x7f9b08069ea0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9b14006fd0 tx=0x7f9b14008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:22.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.397+0000 7f9b1affd640 1 -- 172.21.15.98:0/505208360 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9b20055640 con 0x7f9b240cfad0 2024-09-19T08:43:22.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.553+0000 7f9b2c6f0640 1 -- 172.21.15.98:0/505208360 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9b240742b0 con 0x7f9b080679e0 2024-09-19T08:43:22.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.573+0000 7f9b1affd640 1 -- 172.21.15.98:0/505208360 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252004 (secure 0 0 0) 0x7f9b240742b0 con 0x7f9b080679e0 2024-09-19T08:43:22.575 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:22.576 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:43:22.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.573+0000 7f9b2c6f0640 1 -- 172.21.15.98:0/505208360 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9b080679e0 msgr2=0x7f9b08069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:22.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.573+0000 7f9b2c6f0640 1 --2- 172.21.15.98:0/505208360 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9b080679e0 0x7f9b08069ea0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9b14006fd0 tx=0x7f9b14008040 comp rx=0 tx=0).stop 2024-09-19T08:43:22.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.573+0000 7f9b2c6f0640 1 -- 172.21.15.98:0/505208360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b240cfad0 msgr2=0x7f9b24149200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:22.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.573+0000 7f9b2c6f0640 1 --2- 172.21.15.98:0/505208360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b240cfad0 0x7f9b24149200 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000c760 tx=0x7f9b2000cc20 comp rx=0 tx=0).stop 2024-09-19T08:43:22.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.573+0000 7f9b2c6f0640 1 -- 172.21.15.98:0/505208360 shutdown_connections 2024-09-19T08:43:22.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.573+0000 7f9b2c6f0640 1 --2- 172.21.15.98:0/505208360 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9b080679e0 0x7f9b08069ea0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:22.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.573+0000 7f9b2c6f0640 1 --2- 172.21.15.98:0/505208360 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b2415c550 0x7f9b240b4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:22.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.573+0000 7f9b2c6f0640 1 --2- 172.21.15.98:0/505208360 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b240cfad0 0x7f9b24149200 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:22.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.573+0000 7f9b2c6f0640 1 --2- 172.21.15.98:0/505208360 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9b2406c1d0 0x7f9b24149770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:22.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.573+0000 7f9b2c6f0640 1 -- 172.21.15.98:0/505208360 >> 172.21.15.98:0/505208360 conn(0x7f9b240b2010 msgr2=0x7f9b24159c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:22.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.573+0000 7f9b2c6f0640 1 -- 172.21.15.98:0/505208360 shutdown_connections 2024-09-19T08:43:22.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.573+0000 7f9b2c6f0640 1 -- 172.21.15.98:0/505208360 wait complete. 2024-09-19T08:43:22.592 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-19T08:43:20.877889+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258504,"kb_used_data":8968,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716216,"statfs":{"total":773094113280,"available":772829405184,"internally_reserved":0,"allocated":9183232,"data_stored":4593377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.349133"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.597262+0000","last_change":"2024-09-19T08:43:18.566277+0000","last_active":"2024-09-19T08:43:18.597262+0000","last_peered":"2024-09-19T08:43:18.597262+0000","last_clean":"2024-09-19T08:43:18.597262+0000","last_became_active":"2024-09-19T08:43:18.565557+0000","last_became_peered":"2024-09-19T08:43:18.565557+0000","last_unstale":"2024-09-19T08:43:18.597262+0000","last_undegraded":"2024-09-19T08:43:18.597262+0000","last_fullsized":"2024-09-19T08:43:18.597262+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:42.157284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.578966+0000","last_change":"2024-09-19T08:43:18.575092+0000","last_active":"2024-09-19T08:43:18.578966+0000","last_peered":"2024-09-19T08:43:18.578966+0000","last_clean":"2024-09-19T08:43:18.578966+0000","last_became_active":"2024-09-19T08:43:18.574871+0000","last_became_peered":"2024-09-19T08:43:18.574871+0000","last_unstale":"2024-09-19T08:43:18.578966+0000","last_undegraded":"2024-09-19T08:43:18.578966+0000","last_fullsized":"2024-09-19T08:43:18.578966+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:08:31.920640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.595540+0000","last_change":"2024-09-19T08:43:18.589657+0000","last_active":"2024-09-19T08:43:18.595540+0000","last_peered":"2024-09-19T08:43:18.595540+0000","last_clean":"2024-09-19T08:43:18.595540+0000","last_became_active":"2024-09-19T08:43:18.589051+0000","last_became_peered":"2024-09-19T08:43:18.589051+0000","last_unstale":"2024-09-19T08:43:18.595540+0000","last_undegraded":"2024-09-19T08:43:18.595540+0000","last_fullsized":"2024-09-19T08:43:18.595540+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:01:01.771719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.578153+0000","last_change":"2024-09-19T08:43:18.575315+0000","last_active":"2024-09-19T08:43:18.578153+0000","last_peered":"2024-09-19T08:43:18.578153+0000","last_clean":"2024-09-19T08:43:18.578153+0000","last_became_active":"2024-09-19T08:43:18.575034+0000","last_became_peered":"2024-09-19T08:43:18.575034+0000","last_unstale":"2024-09-19T08:43:18.578153+0000","last_undegraded":"2024-09-19T08:43:18.578153+0000","last_fullsized":"2024-09-19T08:43:18.578153+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:04:08.002113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.590927+0000","last_change":"2024-09-19T08:43:18.584177+0000","last_active":"2024-09-19T08:43:18.590927+0000","last_peered":"2024-09-19T08:43:18.590927+0000","last_clean":"2024-09-19T08:43:18.590927+0000","last_became_active":"2024-09-19T08:43:18.583392+0000","last_became_peered":"2024-09-19T08:43:18.583392+0000","last_unstale":"2024-09-19T08:43:18.590927+0000","last_undegraded":"2024-09-19T08:43:18.590927+0000","last_fullsized":"2024-09-19T08:43:18.590927+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:52.860766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.592012+0000","last_change":"2024-09-19T08:43:18.582470+0000","last_active":"2024-09-19T08:43:18.592012+0000","last_peered":"2024-09-19T08:43:18.592012+0000","last_clean":"2024-09-19T08:43:18.592012+0000","last_became_active":"2024-09-19T08:43:18.581752+0000","last_became_peered":"2024-09-19T08:43:18.581752+0000","last_unstale":"2024-09-19T08:43:18.592012+0000","last_undegraded":"2024-09-19T08:43:18.592012+0000","last_fullsized":"2024-09-19T08:43:18.592012+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:20:43.429735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.572694+0000","last_change":"2024-09-19T08:43:18.564949+0000","last_active":"2024-09-19T08:43:18.572694+0000","last_peered":"2024-09-19T08:43:18.572694+0000","last_clean":"2024-09-19T08:43:18.572694+0000","last_became_active":"2024-09-19T08:43:18.564564+0000","last_became_peered":"2024-09-19T08:43:18.564564+0000","last_unstale":"2024-09-19T08:43:18.572694+0000","last_undegraded":"2024-09-19T08:43:18.572694+0000","last_fullsized":"2024-09-19T08:43:18.572694+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:24.584485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.592908+0000","last_change":"2024-09-19T08:43:18.584505+0000","last_active":"2024-09-19T08:43:18.592908+0000","last_peered":"2024-09-19T08:43:18.592908+0000","last_clean":"2024-09-19T08:43:18.592908+0000","last_became_active":"2024-09-19T08:43:18.583862+0000","last_became_peered":"2024-09-19T08:43:18.583862+0000","last_unstale":"2024-09-19T08:43:18.592908+0000","last_undegraded":"2024-09-19T08:43:18.592908+0000","last_fullsized":"2024-09-19T08:43:18.592908+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:05:52.194614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.598417+0000","last_change":"2024-09-19T08:43:18.591014+0000","last_active":"2024-09-19T08:43:18.598417+0000","last_peered":"2024-09-19T08:43:18.598417+0000","last_clean":"2024-09-19T08:43:18.598417+0000","last_became_active":"2024-09-19T08:43:18.590437+0000","last_became_peered":"2024-09-19T08:43:18.590437+0000","last_unstale":"2024-09-19T08:43:18.598417+0000","last_undegraded":"2024-09-19T08:43:18.598417+0000","last_fullsized":"2024-09-19T08:43:18.598417+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:25:34.623653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.596803+0000","last_change":"2024-09-19T08:43:18.583067+0000","last_active":"2024-09-19T08:43:18.596803+0000","last_peered":"2024-09-19T08:43:18.596803+0000","last_clean":"2024-09-19T08:43:18.596803+0000","last_became_active":"2024-09-19T08:43:18.582125+0000","last_became_peered":"2024-09-19T08:43:18.582125+0000","last_unstale":"2024-09-19T08:43:18.596803+0000","last_undegraded":"2024-09-19T08:43:18.596803+0000","last_fullsized":"2024-09-19T08:43:18.596803+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:23.399707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.581657+0000","last_change":"2024-09-19T08:43:18.578534+0000","last_active":"2024-09-19T08:43:18.581657+0000","last_peered":"2024-09-19T08:43:18.581657+0000","last_clean":"2024-09-19T08:43:18.581657+0000","last_became_active":"2024-09-19T08:43:18.578188+0000","last_became_peered":"2024-09-19T08:43:18.578188+0000","last_unstale":"2024-09-19T08:43:18.581657+0000","last_undegraded":"2024-09-19T08:43:18.581657+0000","last_fullsized":"2024-09-19T08:43:18.581657+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:03:51.686336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.603204+0000","last_change":"2024-09-19T08:43:18.590275+0000","last_active":"2024-09-19T08:43:18.603204+0000","last_peered":"2024-09-19T08:43:18.603204+0000","last_clean":"2024-09-19T08:43:18.603204+0000","last_became_active":"2024-09-19T08:43:18.589517+0000","last_became_peered":"2024-09-19T08:43:18.589517+0000","last_unstale":"2024-09-19T08:43:18.603204+0000","last_undegraded":"2024-09-19T08:43:18.603204+0000","last_fullsized":"2024-09-19T08:43:18.603204+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:47.552603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.586621+0000","last_change":"2024-09-19T08:43:18.574510+0000","last_active":"2024-09-19T08:43:18.586621+0000","last_peered":"2024-09-19T08:43:18.586621+0000","last_clean":"2024-09-19T08:43:18.586621+0000","last_became_active":"2024-09-19T08:43:18.567597+0000","last_became_peered":"2024-09-19T08:43:18.567597+0000","last_unstale":"2024-09-19T08:43:18.586621+0000","last_undegraded":"2024-09-19T08:43:18.586621+0000","last_fullsized":"2024-09-19T08:43:18.586621+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:11:33.266222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.584820+0000","last_change":"2024-09-19T08:43:18.581350+0000","last_active":"2024-09-19T08:43:18.584820+0000","last_peered":"2024-09-19T08:43:18.584820+0000","last_clean":"2024-09-19T08:43:18.584820+0000","last_became_active":"2024-09-19T08:43:18.580682+0000","last_became_peered":"2024-09-19T08:43:18.580682+0000","last_unstale":"2024-09-19T08:43:18.584820+0000","last_undegraded":"2024-09-19T08:43:18.584820+0000","last_fullsized":"2024-09-19T08:43:18.584820+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:25:08.218756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.579637+0000","last_change":"2024-09-19T08:43:18.578294+0000","last_active":"2024-09-19T08:43:18.579637+0000","last_peered":"2024-09-19T08:43:18.579637+0000","last_clean":"2024-09-19T08:43:18.579637+0000","last_became_active":"2024-09-19T08:43:18.577899+0000","last_became_peered":"2024-09-19T08:43:18.577899+0000","last_unstale":"2024-09-19T08:43:18.579637+0000","last_undegraded":"2024-09-19T08:43:18.579637+0000","last_fullsized":"2024-09-19T08:43:18.579637+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:34:01.829055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.585583+0000","last_change":"2024-09-19T08:43:18.580753+0000","last_active":"2024-09-19T08:43:18.585583+0000","last_peered":"2024-09-19T08:43:18.585583+0000","last_clean":"2024-09-19T08:43:18.585583+0000","last_became_active":"2024-09-19T08:43:18.580000+0000","last_became_peered":"2024-09-19T08:43:18.580000+0000","last_unstale":"2024-09-19T08:43:18.585583+0000","last_undegraded":"2024-09-19T08:43:18.585583+0000","last_fullsized":"2024-09-19T08:43:18.585583+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:03:43.662669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.564895+0000","last_change":"2024-09-19T08:43:18.559389+0000","last_active":"2024-09-19T08:43:18.564895+0000","last_peered":"2024-09-19T08:43:18.564895+0000","last_clean":"2024-09-19T08:43:18.564895+0000","last_became_active":"2024-09-19T08:43:18.558616+0000","last_became_peered":"2024-09-19T08:43:18.558616+0000","last_unstale":"2024-09-19T08:43:18.564895+0000","last_undegraded":"2024-09-19T08:43:18.564895+0000","last_fullsized":"2024-09-19T08:43:18.564895+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:53:12.860581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.585834+0000","last_change":"2024-09-19T08:43:18.584103+0000","last_active":"2024-09-19T08:43:18.585834+0000","last_peered":"2024-09-19T08:43:18.585834+0000","last_clean":"2024-09-19T08:43:18.585834+0000","last_became_active":"2024-09-19T08:43:18.583723+0000","last_became_peered":"2024-09-19T08:43:18.583723+0000","last_unstale":"2024-09-19T08:43:18.585834+0000","last_undegraded":"2024-09-19T08:43:18.585834+0000","last_fullsized":"2024-09-19T08:43:18.585834+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:15:02.305566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.562376+0000","last_change":"2024-09-19T08:43:18.558969+0000","last_active":"2024-09-19T08:43:18.562376+0000","last_peered":"2024-09-19T08:43:18.562376+0000","last_clean":"2024-09-19T08:43:18.562376+0000","last_became_active":"2024-09-19T08:43:18.558683+0000","last_became_peered":"2024-09-19T08:43:18.558683+0000","last_unstale":"2024-09-19T08:43:18.562376+0000","last_undegraded":"2024-09-19T08:43:18.562376+0000","last_fullsized":"2024-09-19T08:43:18.562376+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:55.171132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.572707+0000","last_change":"2024-09-19T08:43:18.565861+0000","last_active":"2024-09-19T08:43:18.572707+0000","last_peered":"2024-09-19T08:43:18.572707+0000","last_clean":"2024-09-19T08:43:18.572707+0000","last_became_active":"2024-09-19T08:43:18.565593+0000","last_became_peered":"2024-09-19T08:43:18.565593+0000","last_unstale":"2024-09-19T08:43:18.572707+0000","last_undegraded":"2024-09-19T08:43:18.572707+0000","last_fullsized":"2024-09-19T08:43:18.572707+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:08:04.199261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.572874+0000","last_change":"2024-09-19T08:43:18.565230+0000","last_active":"2024-09-19T08:43:18.572874+0000","last_peered":"2024-09-19T08:43:18.572874+0000","last_clean":"2024-09-19T08:43:18.572874+0000","last_became_active":"2024-09-19T08:43:18.564390+0000","last_became_peered":"2024-09-19T08:43:18.564390+0000","last_unstale":"2024-09-19T08:43:18.572874+0000","last_undegraded":"2024-09-19T08:43:18.572874+0000","last_fullsized":"2024-09-19T08:43:18.572874+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:00:30.923234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.580025+0000","last_change":"2024-09-19T08:43:18.577256+0000","last_active":"2024-09-19T08:43:18.580025+0000","last_peered":"2024-09-19T08:43:18.580025+0000","last_clean":"2024-09-19T08:43:18.580025+0000","last_became_active":"2024-09-19T08:43:18.574458+0000","last_became_peered":"2024-09-19T08:43:18.574458+0000","last_unstale":"2024-09-19T08:43:18.580025+0000","last_undegraded":"2024-09-19T08:43:18.580025+0000","last_fullsized":"2024-09-19T08:43:18.580025+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:15.256400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.590602+0000","last_change":"2024-09-19T08:43:18.579795+0000","last_active":"2024-09-19T08:43:18.590602+0000","last_peered":"2024-09-19T08:43:18.590602+0000","last_clean":"2024-09-19T08:43:18.590602+0000","last_became_active":"2024-09-19T08:43:18.567541+0000","last_became_peered":"2024-09-19T08:43:18.567541+0000","last_unstale":"2024-09-19T08:43:18.590602+0000","last_undegraded":"2024-09-19T08:43:18.590602+0000","last_fullsized":"2024-09-19T08:43:18.590602+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:31:30.847824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.572871+0000","last_change":"2024-09-19T08:43:18.565807+0000","last_active":"2024-09-19T08:43:18.572871+0000","last_peered":"2024-09-19T08:43:18.572871+0000","last_clean":"2024-09-19T08:43:18.572871+0000","last_became_active":"2024-09-19T08:43:18.565531+0000","last_became_peered":"2024-09-19T08:43:18.565531+0000","last_unstale":"2024-09-19T08:43:18.572871+0000","last_undegraded":"2024-09-19T08:43:18.572871+0000","last_fullsized":"2024-09-19T08:43:18.572871+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:16:02.413465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.580311+0000","last_change":"2024-09-19T08:43:18.576737+0000","last_active":"2024-09-19T08:43:18.580311+0000","last_peered":"2024-09-19T08:43:18.580311+0000","last_clean":"2024-09-19T08:43:18.580311+0000","last_became_active":"2024-09-19T08:43:18.575164+0000","last_became_peered":"2024-09-19T08:43:18.575164+0000","last_unstale":"2024-09-19T08:43:18.580311+0000","last_undegraded":"2024-09-19T08:43:18.580311+0000","last_fullsized":"2024-09-19T08:43:18.580311+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:30.004688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.587586+0000","last_change":"2024-09-19T08:43:18.576295+0000","last_active":"2024-09-19T08:43:18.587586+0000","last_peered":"2024-09-19T08:43:18.587586+0000","last_clean":"2024-09-19T08:43:18.587586+0000","last_became_active":"2024-09-19T08:43:18.575555+0000","last_became_peered":"2024-09-19T08:43:18.575555+0000","last_unstale":"2024-09-19T08:43:18.587586+0000","last_undegraded":"2024-09-19T08:43:18.587586+0000","last_fullsized":"2024-09-19T08:43:18.587586+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:49.294782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.574043+0000","last_change":"2024-09-19T08:43:18.563945+0000","last_active":"2024-09-19T08:43:18.574043+0000","last_peered":"2024-09-19T08:43:18.574043+0000","last_clean":"2024-09-19T08:43:18.574043+0000","last_became_active":"2024-09-19T08:43:18.562624+0000","last_became_peered":"2024-09-19T08:43:18.562624+0000","last_unstale":"2024-09-19T08:43:18.574043+0000","last_undegraded":"2024-09-19T08:43:18.574043+0000","last_fullsized":"2024-09-19T08:43:18.574043+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:54:02.984184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.596752+0000","last_change":"2024-09-19T08:43:18.569435+0000","last_active":"2024-09-19T08:43:18.596752+0000","last_peered":"2024-09-19T08:43:18.596752+0000","last_clean":"2024-09-19T08:43:18.596752+0000","last_became_active":"2024-09-19T08:43:18.565378+0000","last_became_peered":"2024-09-19T08:43:18.565378+0000","last_unstale":"2024-09-19T08:43:18.596752+0000","last_undegraded":"2024-09-19T08:43:18.596752+0000","last_fullsized":"2024-09-19T08:43:18.596752+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:04.453546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.582133+0000","last_change":"2024-09-19T08:43:18.580938+0000","last_active":"2024-09-19T08:43:18.582133+0000","last_peered":"2024-09-19T08:43:18.582133+0000","last_clean":"2024-09-19T08:43:18.582133+0000","last_became_active":"2024-09-19T08:43:18.580682+0000","last_became_peered":"2024-09-19T08:43:18.580682+0000","last_unstale":"2024-09-19T08:43:18.582133+0000","last_undegraded":"2024-09-19T08:43:18.582133+0000","last_fullsized":"2024-09-19T08:43:18.582133+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:17:03.137472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.585854+0000","last_change":"2024-09-19T08:43:18.583833+0000","last_active":"2024-09-19T08:43:18.585854+0000","last_peered":"2024-09-19T08:43:18.585854+0000","last_clean":"2024-09-19T08:43:18.585854+0000","last_became_active":"2024-09-19T08:43:18.583059+0000","last_became_peered":"2024-09-19T08:43:18.583059+0000","last_unstale":"2024-09-19T08:43:18.585854+0000","last_undegraded":"2024-09-19T08:43:18.585854+0000","last_fullsized":"2024-09-19T08:43:18.585854+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:21:05.081819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.560841+0000","last_change":"2024-09-19T08:43:18.556307+0000","last_active":"2024-09-19T08:43:18.560841+0000","last_peered":"2024-09-19T08:43:18.560841+0000","last_clean":"2024-09-19T08:43:18.560841+0000","last_became_active":"2024-09-19T08:43:18.555508+0000","last_became_peered":"2024-09-19T08:43:18.555508+0000","last_unstale":"2024-09-19T08:43:18.560841+0000","last_undegraded":"2024-09-19T08:43:18.560841+0000","last_fullsized":"2024-09-19T08:43:18.560841+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:05:52.194614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.600320+0000","last_change":"2024-09-19T08:43:18.589798+0000","last_active":"2024-09-19T08:43:18.600320+0000","last_peered":"2024-09-19T08:43:18.600320+0000","last_clean":"2024-09-19T08:43:18.600320+0000","last_became_active":"2024-09-19T08:43:18.589190+0000","last_became_peered":"2024-09-19T08:43:18.589190+0000","last_unstale":"2024-09-19T08:43:18.600320+0000","last_undegraded":"2024-09-19T08:43:18.600320+0000","last_fullsized":"2024-09-19T08:43:18.600320+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:02.428202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.585815+0000","last_change":"2024-09-19T08:43:18.582575+0000","last_active":"2024-09-19T08:43:18.585815+0000","last_peered":"2024-09-19T08:43:18.585815+0000","last_clean":"2024-09-19T08:43:18.585815+0000","last_became_active":"2024-09-19T08:43:18.581884+0000","last_became_peered":"2024-09-19T08:43:18.581884+0000","last_unstale":"2024-09-19T08:43:18.585815+0000","last_undegraded":"2024-09-19T08:43:18.585815+0000","last_fullsized":"2024-09-19T08:43:18.585815+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:49.294782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"82'4","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.603715+0000","last_change":"2024-09-19T08:43:17.584571+0000","last_active":"2024-09-19T08:43:18.603715+0000","last_peered":"2024-09-19T08:43:18.603715+0000","last_clean":"2024-09-19T08:43:18.603715+0000","last_became_active":"2024-09-19T08:43:17.583331+0000","last_became_peered":"2024-09-19T08:43:17.583331+0000","last_unstale":"2024-09-19T08:43:18.603715+0000","last_undegraded":"2024-09-19T08:43:18.603715+0000","last_fullsized":"2024-09-19T08:43:18.603715+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:34.811252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.597312+0000","last_change":"2024-09-19T08:43:18.570261+0000","last_active":"2024-09-19T08:43:18.597312+0000","last_peered":"2024-09-19T08:43:18.597312+0000","last_clean":"2024-09-19T08:43:18.597312+0000","last_became_active":"2024-09-19T08:43:18.569517+0000","last_became_peered":"2024-09-19T08:43:18.569517+0000","last_unstale":"2024-09-19T08:43:18.597312+0000","last_undegraded":"2024-09-19T08:43:18.597312+0000","last_fullsized":"2024-09-19T08:43:18.597312+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:28:02.369905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.578085+0000","last_change":"2024-09-19T08:43:17.569558+0000","last_active":"2024-09-19T08:43:18.578085+0000","last_peered":"2024-09-19T08:43:18.578085+0000","last_clean":"2024-09-19T08:43:18.578085+0000","last_became_active":"2024-09-19T08:43:17.568823+0000","last_became_peered":"2024-09-19T08:43:17.568823+0000","last_unstale":"2024-09-19T08:43:18.578085+0000","last_undegraded":"2024-09-19T08:43:18.578085+0000","last_fullsized":"2024-09-19T08:43:18.578085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:20:42.425798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.596876+0000","last_change":"2024-09-19T08:43:18.588777+0000","last_active":"2024-09-19T08:43:18.596876+0000","last_peered":"2024-09-19T08:43:18.596876+0000","last_clean":"2024-09-19T08:43:18.596876+0000","last_became_active":"2024-09-19T08:43:18.587991+0000","last_became_peered":"2024-09-19T08:43:18.587991+0000","last_unstale":"2024-09-19T08:43:18.596876+0000","last_undegraded":"2024-09-19T08:43:18.596876+0000","last_fullsized":"2024-09-19T08:43:18.596876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:20.628578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.573469+0000","last_change":"2024-09-19T08:43:17.560190+0000","last_active":"2024-09-19T08:43:18.573469+0000","last_peered":"2024-09-19T08:43:18.573469+0000","last_clean":"2024-09-19T08:43:18.573469+0000","last_became_active":"2024-09-19T08:43:17.559448+0000","last_became_peered":"2024-09-19T08:43:17.559448+0000","last_unstale":"2024-09-19T08:43:18.573469+0000","last_undegraded":"2024-09-19T08:43:18.573469+0000","last_fullsized":"2024-09-19T08:43:18.573469+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:49.585477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.587533+0000","last_change":"2024-09-19T08:43:18.576784+0000","last_active":"2024-09-19T08:43:18.587533+0000","last_peered":"2024-09-19T08:43:18.587533+0000","last_clean":"2024-09-19T08:43:18.587533+0000","last_became_active":"2024-09-19T08:43:18.575989+0000","last_became_peered":"2024-09-19T08:43:18.575989+0000","last_unstale":"2024-09-19T08:43:18.587533+0000","last_undegraded":"2024-09-19T08:43:18.587533+0000","last_fullsized":"2024-09-19T08:43:18.587533+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:52.860766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.573978+0000","last_change":"2024-09-19T08:43:17.559752+0000","last_active":"2024-09-19T08:43:18.573978+0000","last_peered":"2024-09-19T08:43:18.573978+0000","last_clean":"2024-09-19T08:43:18.573978+0000","last_became_active":"2024-09-19T08:43:17.559327+0000","last_became_peered":"2024-09-19T08:43:17.559327+0000","last_unstale":"2024-09-19T08:43:18.573978+0000","last_undegraded":"2024-09-19T08:43:18.573978+0000","last_fullsized":"2024-09-19T08:43:18.573978+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:48.290845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.578882+0000","last_change":"2024-09-19T08:43:18.574634+0000","last_active":"2024-09-19T08:43:18.578882+0000","last_peered":"2024-09-19T08:43:18.578882+0000","last_clean":"2024-09-19T08:43:18.578882+0000","last_became_active":"2024-09-19T08:43:18.574329+0000","last_became_peered":"2024-09-19T08:43:18.574329+0000","last_unstale":"2024-09-19T08:43:18.578882+0000","last_undegraded":"2024-09-19T08:43:18.578882+0000","last_fullsized":"2024-09-19T08:43:18.578882+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:35:37.122315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.582703+0000","last_change":"2024-09-19T08:43:17.570306+0000","last_active":"2024-09-19T08:43:18.582703+0000","last_peered":"2024-09-19T08:43:18.582703+0000","last_clean":"2024-09-19T08:43:18.582703+0000","last_became_active":"2024-09-19T08:43:17.569270+0000","last_became_peered":"2024-09-19T08:43:17.569270+0000","last_unstale":"2024-09-19T08:43:18.582703+0000","last_undegraded":"2024-09-19T08:43:18.582703+0000","last_fullsized":"2024-09-19T08:43:18.582703+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:29.737615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.573985+0000","last_change":"2024-09-19T08:43:18.562021+0000","last_active":"2024-09-19T08:43:18.573985+0000","last_peered":"2024-09-19T08:43:18.573985+0000","last_clean":"2024-09-19T08:43:18.573985+0000","last_became_active":"2024-09-19T08:43:18.560801+0000","last_became_peered":"2024-09-19T08:43:18.560801+0000","last_unstale":"2024-09-19T08:43:18.573985+0000","last_undegraded":"2024-09-19T08:43:18.573985+0000","last_fullsized":"2024-09-19T08:43:18.573985+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:25:34.623653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.578040+0000","last_change":"2024-09-19T08:43:17.570577+0000","last_active":"2024-09-19T08:43:18.578040+0000","last_peered":"2024-09-19T08:43:18.578040+0000","last_clean":"2024-09-19T08:43:18.578040+0000","last_became_active":"2024-09-19T08:43:17.569523+0000","last_became_peered":"2024-09-19T08:43:17.569523+0000","last_unstale":"2024-09-19T08:43:18.578040+0000","last_undegraded":"2024-09-19T08:43:18.578040+0000","last_fullsized":"2024-09-19T08:43:18.578040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:08.991355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.588709+0000","last_change":"2024-09-19T08:43:18.574271+0000","last_active":"2024-09-19T08:43:18.588709+0000","last_peered":"2024-09-19T08:43:18.588709+0000","last_clean":"2024-09-19T08:43:18.588709+0000","last_became_active":"2024-09-19T08:43:18.568518+0000","last_became_peered":"2024-09-19T08:43:18.568518+0000","last_unstale":"2024-09-19T08:43:18.588709+0000","last_undegraded":"2024-09-19T08:43:18.588709+0000","last_fullsized":"2024-09-19T08:43:18.588709+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:12:23.576576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.538991+0000","last_change":"2024-09-19T08:43:17.576694+0000","last_active":"2024-09-19T08:43:18.538991+0000","last_peered":"2024-09-19T08:43:18.538991+0000","last_clean":"2024-09-19T08:43:18.538991+0000","last_became_active":"2024-09-19T08:43:17.576289+0000","last_became_peered":"2024-09-19T08:43:17.576289+0000","last_unstale":"2024-09-19T08:43:18.538991+0000","last_undegraded":"2024-09-19T08:43:18.538991+0000","last_fullsized":"2024-09-19T08:43:18.538991+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:03:42.658731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.573592+0000","last_change":"2024-09-19T08:43:18.569637+0000","last_active":"2024-09-19T08:43:18.573592+0000","last_peered":"2024-09-19T08:43:18.573592+0000","last_clean":"2024-09-19T08:43:18.573592+0000","last_became_active":"2024-09-19T08:43:18.568839+0000","last_became_peered":"2024-09-19T08:43:18.568839+0000","last_unstale":"2024-09-19T08:43:18.573592+0000","last_undegraded":"2024-09-19T08:43:18.573592+0000","last_fullsized":"2024-09-19T08:43:18.573592+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:33:13.094118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.560541+0000","last_change":"2024-09-19T08:43:17.557557+0000","last_active":"2024-09-19T08:43:18.560541+0000","last_peered":"2024-09-19T08:43:18.560541+0000","last_clean":"2024-09-19T08:43:18.560541+0000","last_became_active":"2024-09-19T08:43:17.557173+0000","last_became_peered":"2024-09-19T08:43:17.557173+0000","last_unstale":"2024-09-19T08:43:18.560541+0000","last_undegraded":"2024-09-19T08:43:18.560541+0000","last_fullsized":"2024-09-19T08:43:18.560541+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:30:05.476756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.578527+0000","last_change":"2024-09-19T08:43:18.576101+0000","last_active":"2024-09-19T08:43:18.578527+0000","last_peered":"2024-09-19T08:43:18.578527+0000","last_clean":"2024-09-19T08:43:18.578527+0000","last_became_active":"2024-09-19T08:43:18.575785+0000","last_became_peered":"2024-09-19T08:43:18.575785+0000","last_unstale":"2024-09-19T08:43:18.578527+0000","last_undegraded":"2024-09-19T08:43:18.578527+0000","last_fullsized":"2024-09-19T08:43:18.578527+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:19.547274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.556098+0000","last_change":"2024-09-19T08:43:17.575479+0000","last_active":"2024-09-19T08:43:18.556098+0000","last_peered":"2024-09-19T08:43:18.556098+0000","last_clean":"2024-09-19T08:43:18.556098+0000","last_became_active":"2024-09-19T08:43:17.574714+0000","last_became_peered":"2024-09-19T08:43:17.574714+0000","last_unstale":"2024-09-19T08:43:18.556098+0000","last_undegraded":"2024-09-19T08:43:18.556098+0000","last_fullsized":"2024-09-19T08:43:18.556098+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:50:41.082627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.598920+0000","last_change":"2024-09-19T08:43:18.593882+0000","last_active":"2024-09-19T08:43:18.598920+0000","last_peered":"2024-09-19T08:43:18.598920+0000","last_clean":"2024-09-19T08:43:18.598920+0000","last_became_active":"2024-09-19T08:43:18.590089+0000","last_became_peered":"2024-09-19T08:43:18.590089+0000","last_unstale":"2024-09-19T08:43:18.598920+0000","last_undegraded":"2024-09-19T08:43:18.598920+0000","last_fullsized":"2024-09-19T08:43:18.598920+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:54:02.984184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.558474+0000","last_change":"2024-09-19T08:43:17.583764+0000","last_active":"2024-09-19T08:43:18.558474+0000","last_peered":"2024-09-19T08:43:18.558474+0000","last_clean":"2024-09-19T08:43:18.558474+0000","last_became_active":"2024-09-19T08:43:17.583246+0000","last_became_peered":"2024-09-19T08:43:17.583246+0000","last_unstale":"2024-09-19T08:43:18.558474+0000","last_undegraded":"2024-09-19T08:43:18.558474+0000","last_fullsized":"2024-09-19T08:43:18.558474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:10:16.114687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.572696+0000","last_change":"2024-09-19T08:43:18.571186+0000","last_active":"2024-09-19T08:43:18.572696+0000","last_peered":"2024-09-19T08:43:18.572696+0000","last_clean":"2024-09-19T08:43:18.572696+0000","last_became_active":"2024-09-19T08:43:18.570849+0000","last_became_peered":"2024-09-19T08:43:18.570849+0000","last_unstale":"2024-09-19T08:43:18.572696+0000","last_undegraded":"2024-09-19T08:43:18.572696+0000","last_fullsized":"2024-09-19T08:43:18.572696+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:01:22.218265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"82'3","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.596758+0000","last_change":"2024-09-19T08:43:17.577021+0000","last_active":"2024-09-19T08:43:18.596758+0000","last_peered":"2024-09-19T08:43:18.596758+0000","last_clean":"2024-09-19T08:43:18.596758+0000","last_became_active":"2024-09-19T08:43:17.575945+0000","last_became_peered":"2024-09-19T08:43:17.575945+0000","last_unstale":"2024-09-19T08:43:18.596758+0000","last_undegraded":"2024-09-19T08:43:18.596758+0000","last_fullsized":"2024-09-19T08:43:18.596758+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:44:01.415820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.577612+0000","last_change":"2024-09-19T08:43:18.565469+0000","last_active":"2024-09-19T08:43:18.577612+0000","last_peered":"2024-09-19T08:43:18.577612+0000","last_clean":"2024-09-19T08:43:18.577612+0000","last_became_active":"2024-09-19T08:43:18.564762+0000","last_became_peered":"2024-09-19T08:43:18.564762+0000","last_unstale":"2024-09-19T08:43:18.577612+0000","last_undegraded":"2024-09-19T08:43:18.577612+0000","last_fullsized":"2024-09-19T08:43:18.577612+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:12:23.576576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"82'6","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.600735+0000","last_change":"2024-09-19T08:43:17.572935+0000","last_active":"2024-09-19T08:43:18.600735+0000","last_peered":"2024-09-19T08:43:18.600735+0000","last_clean":"2024-09-19T08:43:18.600735+0000","last_became_active":"2024-09-19T08:43:17.572074+0000","last_became_peered":"2024-09-19T08:43:17.572074+0000","last_unstale":"2024-09-19T08:43:18.600735+0000","last_undegraded":"2024-09-19T08:43:18.600735+0000","last_fullsized":"2024-09-19T08:43:18.600735+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:32.915411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.562052+0000","last_change":"2024-09-19T08:43:18.556991+0000","last_active":"2024-09-19T08:43:18.562052+0000","last_peered":"2024-09-19T08:43:18.562052+0000","last_clean":"2024-09-19T08:43:18.562052+0000","last_became_active":"2024-09-19T08:43:18.555019+0000","last_became_peered":"2024-09-19T08:43:18.555019+0000","last_unstale":"2024-09-19T08:43:18.562052+0000","last_undegraded":"2024-09-19T08:43:18.562052+0000","last_fullsized":"2024-09-19T08:43:18.562052+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:21:35.930304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.583012+0000","last_change":"2024-09-19T08:43:17.565687+0000","last_active":"2024-09-19T08:43:18.583012+0000","last_peered":"2024-09-19T08:43:18.583012+0000","last_clean":"2024-09-19T08:43:18.583012+0000","last_became_active":"2024-09-19T08:43:17.565193+0000","last_became_peered":"2024-09-19T08:43:17.565193+0000","last_unstale":"2024-09-19T08:43:18.583012+0000","last_undegraded":"2024-09-19T08:43:18.583012+0000","last_fullsized":"2024-09-19T08:43:18.583012+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:42.242000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.586711+0000","last_change":"2024-09-19T08:43:18.574914+0000","last_active":"2024-09-19T08:43:18.586711+0000","last_peered":"2024-09-19T08:43:18.586711+0000","last_clean":"2024-09-19T08:43:18.586711+0000","last_became_active":"2024-09-19T08:43:18.568554+0000","last_became_peered":"2024-09-19T08:43:18.568554+0000","last_unstale":"2024-09-19T08:43:18.586711+0000","last_undegraded":"2024-09-19T08:43:18.586711+0000","last_fullsized":"2024-09-19T08:43:18.586711+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:03:43.662669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.548772+0000","last_change":"2024-09-19T08:43:17.565587+0000","last_active":"2024-09-19T08:43:18.548772+0000","last_peered":"2024-09-19T08:43:18.548772+0000","last_clean":"2024-09-19T08:43:18.548772+0000","last_became_active":"2024-09-19T08:43:17.565187+0000","last_became_peered":"2024-09-19T08:43:17.565187+0000","last_unstale":"2024-09-19T08:43:18.548772+0000","last_undegraded":"2024-09-19T08:43:18.548772+0000","last_fullsized":"2024-09-19T08:43:18.548772+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:01.234243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.581752+0000","last_change":"2024-09-19T08:43:18.579348+0000","last_active":"2024-09-19T08:43:18.581752+0000","last_peered":"2024-09-19T08:43:18.581752+0000","last_clean":"2024-09-19T08:43:18.581752+0000","last_became_active":"2024-09-19T08:43:18.578396+0000","last_became_peered":"2024-09-19T08:43:18.578396+0000","last_unstale":"2024-09-19T08:43:18.581752+0000","last_undegraded":"2024-09-19T08:43:18.581752+0000","last_fullsized":"2024-09-19T08:43:18.581752+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:23:22.109467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.557401+0000","last_change":"2024-09-19T08:43:17.581563+0000","last_active":"2024-09-19T08:43:18.557401+0000","last_peered":"2024-09-19T08:43:18.557401+0000","last_clean":"2024-09-19T08:43:18.557401+0000","last_became_active":"2024-09-19T08:43:17.580824+0000","last_became_peered":"2024-09-19T08:43:17.580824+0000","last_unstale":"2024-09-19T08:43:18.557401+0000","last_undegraded":"2024-09-19T08:43:18.557401+0000","last_fullsized":"2024-09-19T08:43:18.557401+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:08.991355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.579907+0000","last_change":"2024-09-19T08:43:18.577673+0000","last_active":"2024-09-19T08:43:18.579907+0000","last_peered":"2024-09-19T08:43:18.579907+0000","last_clean":"2024-09-19T08:43:18.579907+0000","last_became_active":"2024-09-19T08:43:18.576991+0000","last_became_peered":"2024-09-19T08:43:18.576991+0000","last_unstale":"2024-09-19T08:43:18.579907+0000","last_undegraded":"2024-09-19T08:43:18.579907+0000","last_fullsized":"2024-09-19T08:43:18.579907+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:41:56.080757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"82'8","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.595610+0000","last_change":"2024-09-19T08:43:17.574681+0000","last_active":"2024-09-19T08:43:18.595610+0000","last_peered":"2024-09-19T08:43:18.595610+0000","last_clean":"2024-09-19T08:43:18.595610+0000","last_became_active":"2024-09-19T08:43:17.573802+0000","last_became_peered":"2024-09-19T08:43:17.573802+0000","last_unstale":"2024-09-19T08:43:18.595610+0000","last_undegraded":"2024-09-19T08:43:18.595610+0000","last_fullsized":"2024-09-19T08:43:18.595610+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:11:32.262284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.598975+0000","last_change":"2024-09-19T08:43:18.594954+0000","last_active":"2024-09-19T08:43:18.598975+0000","last_peered":"2024-09-19T08:43:18.598975+0000","last_clean":"2024-09-19T08:43:18.598975+0000","last_became_active":"2024-09-19T08:43:18.594502+0000","last_became_peered":"2024-09-19T08:43:18.594502+0000","last_unstale":"2024-09-19T08:43:18.598975+0000","last_undegraded":"2024-09-19T08:43:18.598975+0000","last_fullsized":"2024-09-19T08:43:18.598975+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:50.589415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.599177+0000","last_change":"2024-09-19T08:43:17.577257+0000","last_active":"2024-09-19T08:43:18.599177+0000","last_peered":"2024-09-19T08:43:18.599177+0000","last_clean":"2024-09-19T08:43:18.599177+0000","last_became_active":"2024-09-19T08:43:17.576335+0000","last_became_peered":"2024-09-19T08:43:17.576335+0000","last_unstale":"2024-09-19T08:43:18.599177+0000","last_undegraded":"2024-09-19T08:43:18.599177+0000","last_fullsized":"2024-09-19T08:43:18.599177+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:07.844032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.597351+0000","last_change":"2024-09-19T08:43:18.570748+0000","last_active":"2024-09-19T08:43:18.597351+0000","last_peered":"2024-09-19T08:43:18.597351+0000","last_clean":"2024-09-19T08:43:18.597351+0000","last_became_active":"2024-09-19T08:43:18.569397+0000","last_became_peered":"2024-09-19T08:43:18.569397+0000","last_unstale":"2024-09-19T08:43:18.597351+0000","last_undegraded":"2024-09-19T08:43:18.597351+0000","last_fullsized":"2024-09-19T08:43:18.597351+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:32:13.718833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.584433+0000","last_change":"2024-09-19T08:43:17.577458+0000","last_active":"2024-09-19T08:43:18.584433+0000","last_peered":"2024-09-19T08:43:18.584433+0000","last_clean":"2024-09-19T08:43:18.584433+0000","last_became_active":"2024-09-19T08:43:17.577051+0000","last_became_peered":"2024-09-19T08:43:17.577051+0000","last_unstale":"2024-09-19T08:43:18.584433+0000","last_undegraded":"2024-09-19T08:43:18.584433+0000","last_fullsized":"2024-09-19T08:43:18.584433+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:57.051641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.584340+0000","last_change":"2024-09-19T08:43:18.574818+0000","last_active":"2024-09-19T08:43:18.584340+0000","last_peered":"2024-09-19T08:43:18.584340+0000","last_clean":"2024-09-19T08:43:18.584340+0000","last_became_active":"2024-09-19T08:43:18.573749+0000","last_became_peered":"2024-09-19T08:43:18.573749+0000","last_unstale":"2024-09-19T08:43:18.584340+0000","last_undegraded":"2024-09-19T08:43:18.584340+0000","last_fullsized":"2024-09-19T08:43:18.584340+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:11:33.266222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.539804+0000","last_change":"2024-09-19T08:43:17.576666+0000","last_active":"2024-09-19T08:43:18.539804+0000","last_peered":"2024-09-19T08:43:18.539804+0000","last_clean":"2024-09-19T08:43:18.539804+0000","last_became_active":"2024-09-19T08:43:17.576270+0000","last_became_peered":"2024-09-19T08:43:17.576270+0000","last_unstale":"2024-09-19T08:43:18.539804+0000","last_undegraded":"2024-09-19T08:43:18.539804+0000","last_fullsized":"2024-09-19T08:43:18.539804+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:01:00.767781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.584799+0000","last_change":"2024-09-19T08:43:18.581214+0000","last_active":"2024-09-19T08:43:18.584799+0000","last_peered":"2024-09-19T08:43:18.584799+0000","last_clean":"2024-09-19T08:43:18.584799+0000","last_became_active":"2024-09-19T08:43:18.580344+0000","last_became_peered":"2024-09-19T08:43:18.580344+0000","last_unstale":"2024-09-19T08:43:18.584799+0000","last_undegraded":"2024-09-19T08:43:18.584799+0000","last_fullsized":"2024-09-19T08:43:18.584799+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:01.951721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.573306+0000","last_change":"2024-09-19T08:43:17.569866+0000","last_active":"2024-09-19T08:43:18.573306+0000","last_peered":"2024-09-19T08:43:18.573306+0000","last_clean":"2024-09-19T08:43:18.573306+0000","last_became_active":"2024-09-19T08:43:17.569401+0000","last_became_peered":"2024-09-19T08:43:17.569401+0000","last_unstale":"2024-09-19T08:43:18.573306+0000","last_undegraded":"2024-09-19T08:43:18.573306+0000","last_fullsized":"2024-09-19T08:43:18.573306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:27.223097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.592705+0000","last_change":"2024-09-19T08:43:18.585441+0000","last_active":"2024-09-19T08:43:18.592705+0000","last_peered":"2024-09-19T08:43:18.592705+0000","last_clean":"2024-09-19T08:43:18.592705+0000","last_became_active":"2024-09-19T08:43:18.573687+0000","last_became_peered":"2024-09-19T08:43:18.573687+0000","last_unstale":"2024-09-19T08:43:18.592705+0000","last_undegraded":"2024-09-19T08:43:18.592705+0000","last_fullsized":"2024-09-19T08:43:18.592705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:53:12.860581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.553342+0000","last_change":"2024-09-19T08:43:17.562799+0000","last_active":"2024-09-19T08:43:18.553342+0000","last_peered":"2024-09-19T08:43:18.553342+0000","last_clean":"2024-09-19T08:43:18.553342+0000","last_became_active":"2024-09-19T08:43:17.562194+0000","last_became_peered":"2024-09-19T08:43:17.562194+0000","last_unstale":"2024-09-19T08:43:18.553342+0000","last_undegraded":"2024-09-19T08:43:18.553342+0000","last_fullsized":"2024-09-19T08:43:18.553342+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:23.376881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.594555+0000","last_change":"2024-09-19T08:43:18.593857+0000","last_active":"2024-09-19T08:43:18.594555+0000","last_peered":"2024-09-19T08:43:18.594555+0000","last_clean":"2024-09-19T08:43:18.594555+0000","last_became_active":"2024-09-19T08:43:18.593498+0000","last_became_peered":"2024-09-19T08:43:18.593498+0000","last_unstale":"2024-09-19T08:43:18.594555+0000","last_undegraded":"2024-09-19T08:43:18.594555+0000","last_fullsized":"2024-09-19T08:43:18.594555+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:58.575602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.551881+0000","last_change":"2024-09-19T08:43:17.583981+0000","last_active":"2024-09-19T08:43:18.551881+0000","last_peered":"2024-09-19T08:43:18.551881+0000","last_clean":"2024-09-19T08:43:18.551881+0000","last_became_active":"2024-09-19T08:43:17.583305+0000","last_became_peered":"2024-09-19T08:43:17.583305+0000","last_unstale":"2024-09-19T08:43:18.551881+0000","last_undegraded":"2024-09-19T08:43:18.551881+0000","last_fullsized":"2024-09-19T08:43:18.551881+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:38.205461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.598830+0000","last_change":"2024-09-19T08:43:18.592231+0000","last_active":"2024-09-19T08:43:18.598830+0000","last_peered":"2024-09-19T08:43:18.598830+0000","last_clean":"2024-09-19T08:43:18.598830+0000","last_became_active":"2024-09-19T08:43:18.591670+0000","last_became_peered":"2024-09-19T08:43:18.591670+0000","last_unstale":"2024-09-19T08:43:18.598830+0000","last_undegraded":"2024-09-19T08:43:18.598830+0000","last_fullsized":"2024-09-19T08:43:18.598830+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:58.000860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.547695+0000","last_change":"2024-09-19T08:43:17.570048+0000","last_active":"2024-09-19T08:43:18.547695+0000","last_peered":"2024-09-19T08:43:18.547695+0000","last_clean":"2024-09-19T08:43:18.547695+0000","last_became_active":"2024-09-19T08:43:17.569629+0000","last_became_peered":"2024-09-19T08:43:17.569629+0000","last_unstale":"2024-09-19T08:43:18.547695+0000","last_undegraded":"2024-09-19T08:43:18.547695+0000","last_fullsized":"2024-09-19T08:43:18.547695+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:25:47.531367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.596009+0000","last_change":"2024-09-19T08:43:18.592694+0000","last_active":"2024-09-19T08:43:18.596009+0000","last_peered":"2024-09-19T08:43:18.596009+0000","last_clean":"2024-09-19T08:43:18.596009+0000","last_became_active":"2024-09-19T08:43:18.592050+0000","last_became_peered":"2024-09-19T08:43:18.592050+0000","last_unstale":"2024-09-19T08:43:18.596009+0000","last_undegraded":"2024-09-19T08:43:18.596009+0000","last_fullsized":"2024-09-19T08:43:18.596009+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:58.055578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.595853+0000","last_change":"2024-09-19T08:43:17.568298+0000","last_active":"2024-09-19T08:43:18.595853+0000","last_peered":"2024-09-19T08:43:18.595853+0000","last_clean":"2024-09-19T08:43:18.595853+0000","last_became_active":"2024-09-19T08:43:17.567944+0000","last_became_peered":"2024-09-19T08:43:17.567944+0000","last_unstale":"2024-09-19T08:43:18.595853+0000","last_undegraded":"2024-09-19T08:43:18.595853+0000","last_fullsized":"2024-09-19T08:43:18.595853+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:52:57.202297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.603389+0000","last_change":"2024-09-19T08:43:18.600358+0000","last_active":"2024-09-19T08:43:18.603389+0000","last_peered":"2024-09-19T08:43:18.603389+0000","last_clean":"2024-09-19T08:43:18.603389+0000","last_became_active":"2024-09-19T08:43:18.599737+0000","last_became_peered":"2024-09-19T08:43:18.599737+0000","last_unstale":"2024-09-19T08:43:18.603389+0000","last_undegraded":"2024-09-19T08:43:18.603389+0000","last_fullsized":"2024-09-19T08:43:18.603389+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:27:36.608565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.561851+0000","last_change":"2024-09-19T08:43:17.570348+0000","last_active":"2024-09-19T08:43:18.561851+0000","last_peered":"2024-09-19T08:43:18.561851+0000","last_clean":"2024-09-19T08:43:18.561851+0000","last_became_active":"2024-09-19T08:43:17.561639+0000","last_became_peered":"2024-09-19T08:43:17.561639+0000","last_unstale":"2024-09-19T08:43:18.561851+0000","last_undegraded":"2024-09-19T08:43:18.561851+0000","last_fullsized":"2024-09-19T08:43:18.561851+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:27:51.789175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.596829+0000","last_change":"2024-09-19T08:43:18.588538+0000","last_active":"2024-09-19T08:43:18.596829+0000","last_peered":"2024-09-19T08:43:18.596829+0000","last_clean":"2024-09-19T08:43:18.596829+0000","last_became_active":"2024-09-19T08:43:18.587873+0000","last_became_peered":"2024-09-19T08:43:18.587873+0000","last_unstale":"2024-09-19T08:43:18.596829+0000","last_undegraded":"2024-09-19T08:43:18.596829+0000","last_fullsized":"2024-09-19T08:43:18.596829+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:35:52.860618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.552119+0000","last_change":"2024-09-19T08:43:17.583882+0000","last_active":"2024-09-19T08:43:18.552119+0000","last_peered":"2024-09-19T08:43:18.552119+0000","last_clean":"2024-09-19T08:43:18.552119+0000","last_became_active":"2024-09-19T08:43:17.583109+0000","last_became_peered":"2024-09-19T08:43:17.583109+0000","last_unstale":"2024-09-19T08:43:18.552119+0000","last_undegraded":"2024-09-19T08:43:18.552119+0000","last_fullsized":"2024-09-19T08:43:18.552119+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:45.131969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.577712+0000","last_change":"2024-09-19T08:43:18.563486+0000","last_active":"2024-09-19T08:43:18.577712+0000","last_peered":"2024-09-19T08:43:18.577712+0000","last_clean":"2024-09-19T08:43:18.577712+0000","last_became_active":"2024-09-19T08:43:18.561539+0000","last_became_peered":"2024-09-19T08:43:18.561539+0000","last_unstale":"2024-09-19T08:43:18.577712+0000","last_undegraded":"2024-09-19T08:43:18.577712+0000","last_fullsized":"2024-09-19T08:43:18.577712+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:58.000860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.561860+0000","last_change":"2024-09-19T08:43:17.581881+0000","last_active":"2024-09-19T08:43:18.561860+0000","last_peered":"2024-09-19T08:43:18.561860+0000","last_clean":"2024-09-19T08:43:18.561860+0000","last_became_active":"2024-09-19T08:43:17.581267+0000","last_became_peered":"2024-09-19T08:43:17.581267+0000","last_unstale":"2024-09-19T08:43:18.561860+0000","last_undegraded":"2024-09-19T08:43:18.561860+0000","last_fullsized":"2024-09-19T08:43:18.561860+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:55.318457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.591953+0000","last_change":"2024-09-19T08:43:18.582258+0000","last_active":"2024-09-19T08:43:18.591953+0000","last_peered":"2024-09-19T08:43:18.591953+0000","last_clean":"2024-09-19T08:43:18.591953+0000","last_became_active":"2024-09-19T08:43:18.581647+0000","last_became_peered":"2024-09-19T08:43:18.581647+0000","last_unstale":"2024-09-19T08:43:18.591953+0000","last_undegraded":"2024-09-19T08:43:18.591953+0000","last_fullsized":"2024-09-19T08:43:18.591953+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:30.741553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.539406+0000","last_change":"2024-09-19T08:43:17.564416+0000","last_active":"2024-09-19T08:43:18.539406+0000","last_peered":"2024-09-19T08:43:18.539406+0000","last_clean":"2024-09-19T08:43:18.539406+0000","last_became_active":"2024-09-19T08:43:17.564033+0000","last_became_peered":"2024-09-19T08:43:17.564033+0000","last_unstale":"2024-09-19T08:43:18.539406+0000","last_undegraded":"2024-09-19T08:43:18.539406+0000","last_fullsized":"2024-09-19T08:43:18.539406+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:51.856829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.561822+0000","last_change":"2024-09-19T08:43:18.555397+0000","last_active":"2024-09-19T08:43:18.561822+0000","last_peered":"2024-09-19T08:43:18.561822+0000","last_clean":"2024-09-19T08:43:18.561822+0000","last_became_active":"2024-09-19T08:43:18.554764+0000","last_became_peered":"2024-09-19T08:43:18.554764+0000","last_unstale":"2024-09-19T08:43:18.561822+0000","last_undegraded":"2024-09-19T08:43:18.561822+0000","last_fullsized":"2024-09-19T08:43:18.561822+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:27:36.608565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.546717+0000","last_change":"2024-09-19T08:43:17.562806+0000","last_active":"2024-09-19T08:43:18.546717+0000","last_peered":"2024-09-19T08:43:18.546717+0000","last_clean":"2024-09-19T08:43:18.546717+0000","last_became_active":"2024-09-19T08:43:17.562059+0000","last_became_peered":"2024-09-19T08:43:17.562059+0000","last_unstale":"2024-09-19T08:43:18.546717+0000","last_undegraded":"2024-09-19T08:43:18.546717+0000","last_fullsized":"2024-09-19T08:43:18.546717+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:32.915411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.595501+0000","last_change":"2024-09-19T08:43:18.589394+0000","last_active":"2024-09-19T08:43:18.595501+0000","last_peered":"2024-09-19T08:43:18.595501+0000","last_clean":"2024-09-19T08:43:18.595501+0000","last_became_active":"2024-09-19T08:43:18.588256+0000","last_became_peered":"2024-09-19T08:43:18.588256+0000","last_unstale":"2024-09-19T08:43:18.595501+0000","last_undegraded":"2024-09-19T08:43:18.595501+0000","last_fullsized":"2024-09-19T08:43:18.595501+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:58.055578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.555692+0000","last_change":"2024-09-19T08:43:17.581704+0000","last_active":"2024-09-19T08:43:18.555692+0000","last_peered":"2024-09-19T08:43:18.555692+0000","last_clean":"2024-09-19T08:43:18.555692+0000","last_became_active":"2024-09-19T08:43:17.578243+0000","last_became_peered":"2024-09-19T08:43:17.578243+0000","last_unstale":"2024-09-19T08:43:18.555692+0000","last_undegraded":"2024-09-19T08:43:18.555692+0000","last_fullsized":"2024-09-19T08:43:18.555692+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:44:59.748280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.596364+0000","last_change":"2024-09-19T08:43:18.581782+0000","last_active":"2024-09-19T08:43:18.596364+0000","last_peered":"2024-09-19T08:43:18.596364+0000","last_clean":"2024-09-19T08:43:18.596364+0000","last_became_active":"2024-09-19T08:43:18.580951+0000","last_became_peered":"2024-09-19T08:43:18.580951+0000","last_unstale":"2024-09-19T08:43:18.596364+0000","last_undegraded":"2024-09-19T08:43:18.596364+0000","last_fullsized":"2024-09-19T08:43:18.596364+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:47:28.527734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.544968+0000","last_change":"2024-09-19T08:43:17.586343+0000","last_active":"2024-09-19T08:43:18.544968+0000","last_peered":"2024-09-19T08:43:18.544968+0000","last_clean":"2024-09-19T08:43:18.544968+0000","last_became_active":"2024-09-19T08:43:17.585820+0000","last_became_peered":"2024-09-19T08:43:17.585820+0000","last_unstale":"2024-09-19T08:43:18.544968+0000","last_undegraded":"2024-09-19T08:43:18.544968+0000","last_fullsized":"2024-09-19T08:43:18.544968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:19.291567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.593755+0000","last_change":"2024-09-19T08:43:18.593119+0000","last_active":"2024-09-19T08:43:18.593755+0000","last_peered":"2024-09-19T08:43:18.593755+0000","last_clean":"2024-09-19T08:43:18.593755+0000","last_became_active":"2024-09-19T08:43:18.592841+0000","last_became_peered":"2024-09-19T08:43:18.592841+0000","last_unstale":"2024-09-19T08:43:18.593755+0000","last_undegraded":"2024-09-19T08:43:18.593755+0000","last_fullsized":"2024-09-19T08:43:18.593755+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:48:08.780957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.587408+0000","last_change":"2024-09-19T08:43:17.576964+0000","last_active":"2024-09-19T08:43:18.587408+0000","last_peered":"2024-09-19T08:43:18.587408+0000","last_clean":"2024-09-19T08:43:18.587408+0000","last_became_active":"2024-09-19T08:43:17.576207+0000","last_became_peered":"2024-09-19T08:43:17.576207+0000","last_unstale":"2024-09-19T08:43:18.587408+0000","last_undegraded":"2024-09-19T08:43:18.587408+0000","last_fullsized":"2024-09-19T08:43:18.587408+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:54:01.980247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"71'128","reported_seq":318,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.556994+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:43:18.556994+0000","last_peered":"2024-09-19T08:43:18.556994+0000","last_clean":"2024-09-19T08:43:18.556994+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:43:18.556994+0000","last_undegraded":"2024-09-19T08:43:18.556994+0000","last_fullsized":"2024-09-19T08:43:18.556994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509505,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32756,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339084,"statfs":{"total":96636764160,"available":96603222016,"internally_reserved":0,"allocated":1597440,"data_stored":1019362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574914,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1011712,"data_stored":429618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509507,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":987136,"data_stored":422149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509507,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":987136,"data_stored":422149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542209,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":987136,"data_stored":422149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509504,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1003520,"data_stored":429506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509506,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1011712,"data_stored":429118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509504,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32748,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1597440,"data_stored":1019326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:43:22.596 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:43:22.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.749+0000 7f2159199640 1 Processor -- start 2024-09-19T08:43:22.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.749+0000 7f2159199640 1 -- start start 2024-09-19T08:43:22.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.749+0000 7f2159199640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f215406c3a0 0x7f21540cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:22.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.749+0000 7f2159199640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21540cf900 0x7f21540723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:22.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.749+0000 7f2159199640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f215406d310 con 0x7f215415c550 2024-09-19T08:43:22.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.749+0000 7f2159199640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f215406d5f0 con 0x7f215406c3a0 2024-09-19T08:43:22.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.749+0000 7f2159199640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f215406d8d0 con 0x7f21540cf900 2024-09-19T08:43:22.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.749+0000 7f215259c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f215406c3a0 0x7f21540cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:22.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.749+0000 7f2152d9d640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f215415c550 0x7f215406bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40072/0 (socket says 172.21.15.98:40072) 2024-09-19T08:43:22.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.749+0000 7f2152d9d640 1 -- 172.21.15.98:0/234217794 learned_addr learned my addr 172.21.15.98:0/234217794 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:22.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.749+0000 7f215359e640 1 --2- 172.21.15.98:0/234217794 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21540cf900 0x7f21540723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:22.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.749+0000 7f215259c640 1 -- 172.21.15.98:0/234217794 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21540cf900 msgr2=0x7f21540723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:22.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.749+0000 7f215259c640 1 --2- 172.21.15.98:0/234217794 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21540cf900 0x7f21540723a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:22.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.749+0000 7f215259c640 1 -- 172.21.15.98:0/234217794 >> v1:172.21.15.98:6789/0 conn(0x7f215415c550 legacy=0x7f215406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:22.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.749+0000 7f2151d9b640 1 -- 172.21.15.98:0/234217794 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3219181525 0 0) 0x7f215406d310 con 0x7f215415c550 2024-09-19T08:43:22.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.749+0000 7f215259c640 1 -- 172.21.15.98:0/234217794 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f215406dbb0 con 0x7f215406c3a0 2024-09-19T08:43:22.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.753+0000 7f215259c640 1 --2- 172.21.15.98:0/234217794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f215406c3a0 0x7f21540cf3c0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f21480097c0 tx=0x7f214801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f0a06cbfa61b2a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:22.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.753+0000 7f2151d9b640 1 -- 172.21.15.98:0/234217794 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21480b1070 con 0x7f215406c3a0 2024-09-19T08:43:22.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.753+0000 7f2151d9b640 1 -- 172.21.15.98:0/234217794 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21480ac410 con 0x7f215406c3a0 2024-09-19T08:43:22.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.753+0000 7f2151d9b640 1 -- 172.21.15.98:0/234217794 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21480ac7e0 con 0x7f215406c3a0 2024-09-19T08:43:22.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.753+0000 7f2159199640 1 -- 172.21.15.98:0/234217794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f215406c3a0 msgr2=0x7f21540cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:22.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.753+0000 7f2159199640 1 --2- 172.21.15.98:0/234217794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f215406c3a0 0x7f21540cf3c0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f21480097c0 tx=0x7f214801c4a0 comp rx=0 tx=0).stop 2024-09-19T08:43:22.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.753+0000 7f2159199640 1 -- 172.21.15.98:0/234217794 shutdown_connections 2024-09-19T08:43:22.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.753+0000 7f2159199640 1 --2- 172.21.15.98:0/234217794 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21540cf900 0x7f21540723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:22.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.753+0000 7f2159199640 1 --2- 172.21.15.98:0/234217794 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f215406c3a0 0x7f21540cf3c0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:22.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.753+0000 7f2159199640 1 -- 172.21.15.98:0/234217794 >> 172.21.15.98:0/234217794 conn(0x7f21540b2010 msgr2=0x7f21540b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:22.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.753+0000 7f2159199640 1 -- 172.21.15.98:0/234217794 shutdown_connections 2024-09-19T08:43:22.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.753+0000 7f2159199640 1 -- 172.21.15.98:0/234217794 wait complete. 2024-09-19T08:43:22.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.753+0000 7f2159199640 1 Processor -- start 2024-09-19T08:43:22.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.753+0000 7f2159199640 1 -- start start 2024-09-19T08:43:22.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.753+0000 7f2159199640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f215406c3a0 0x7f21541506e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:22.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.753+0000 7f2159199640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21540cf900 0x7f215415ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:22.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.753+0000 7f2159199640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f215415c550 0x7f215415b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:22.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.753+0000 7f2159199640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2154073500 con 0x7f215415c550 2024-09-19T08:43:22.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.753+0000 7f2159199640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2154073380 con 0x7f21540cf900 2024-09-19T08:43:22.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.753+0000 7f2159199640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2154073680 con 0x7f215406c3a0 2024-09-19T08:43:22.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.753+0000 7f215359e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f215415c550 0x7f215415b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:22.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.753+0000 7f215359e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f215415c550 0x7f215415b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41434/0 (socket says 172.21.15.98:41434) 2024-09-19T08:43:22.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.753+0000 7f215359e640 1 -- 172.21.15.98:0/259212894 learned_addr learned my addr 172.21.15.98:0/259212894 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:22.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.753+0000 7f2152d9d640 1 --2- 172.21.15.98:0/259212894 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f215406c3a0 0x7f21541506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:22.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.753+0000 7f215259c640 1 --2- 172.21.15.98:0/259212894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21540cf900 0x7f215415ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:22.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.757+0000 7f215359e640 1 -- 172.21.15.98:0/259212894 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f215406c3a0 msgr2=0x7f21541506e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:22.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.757+0000 7f215359e640 1 --2- 172.21.15.98:0/259212894 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f215406c3a0 0x7f21541506e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:22.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.757+0000 7f215359e640 1 -- 172.21.15.98:0/259212894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21540cf900 msgr2=0x7f215415ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:22.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.757+0000 7f215359e640 1 --2- 172.21.15.98:0/259212894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21540cf900 0x7f215415ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:22.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.757+0000 7f215359e640 1 -- 172.21.15.98:0/259212894 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21541f7c00 con 0x7f215415c550 2024-09-19T08:43:22.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.757+0000 7f215359e640 1 --2- 172.21.15.98:0/259212894 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f215415c550 0x7f215415b3d0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f214c00cc30 tx=0x7f214c00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:22.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.757+0000 7f213b7fe640 1 -- 172.21.15.98:0/259212894 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c00ce40 con 0x7f215415c550 2024-09-19T08:43:22.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.757+0000 7f213b7fe640 1 -- 172.21.15.98:0/259212894 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f214c0039a0 con 0x7f215415c550 2024-09-19T08:43:22.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.757+0000 7f213b7fe640 1 -- 172.21.15.98:0/259212894 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c003d50 con 0x7f215415c550 2024-09-19T08:43:22.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.757+0000 7f2159199640 1 -- 172.21.15.98:0/259212894 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21541f7ee0 con 0x7f215415c550 2024-09-19T08:43:22.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.757+0000 7f2159199640 1 -- 172.21.15.98:0/259212894 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21541f84a0 con 0x7f215415c550 2024-09-19T08:43:22.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.757+0000 7f2159199640 1 -- 172.21.15.98:0/259212894 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21540cf3c0 con 0x7f215415c550 2024-09-19T08:43:22.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.757+0000 7f213b7fe640 1 -- 172.21.15.98:0/259212894 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f214c003f70 con 0x7f215415c550 2024-09-19T08:43:22.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.757+0000 7f213b7fe640 1 --2- 172.21.15.98:0/259212894 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f21400679e0 0x7f2140069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:22.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.757+0000 7f213b7fe640 1 -- 172.21.15.98:0/259212894 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f214c086440 con 0x7f215415c550 2024-09-19T08:43:22.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.757+0000 7f2152d9d640 1 --2- 172.21.15.98:0/259212894 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f21400679e0 0x7f2140069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:22.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.761+0000 7f2152d9d640 1 --2- 172.21.15.98:0/259212894 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f21400679e0 0x7f2140069ea0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2144000f70 tx=0x7f2144007b80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:22.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.761+0000 7f213b7fe640 1 -- 172.21.15.98:0/259212894 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f214c014030 con 0x7f215415c550 2024-09-19T08:43:22.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.921+0000 7f2159199640 1 -- 172.21.15.98:0/259212894 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f21540b34a0 con 0x7f21400679e0 2024-09-19T08:43:22.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.941+0000 7f213b7fe640 1 -- 172.21.15.98:0/259212894 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252004 (secure 0 0 0) 0x7f21440071f0 con 0x7f21400679e0 2024-09-19T08:43:22.942 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:22.944 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:43:22.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.941+0000 7f2159199640 1 -- 172.21.15.98:0/259212894 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f21400679e0 msgr2=0x7f2140069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:22.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.941+0000 7f2159199640 1 --2- 172.21.15.98:0/259212894 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f21400679e0 0x7f2140069ea0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2144000f70 tx=0x7f2144007b80 comp rx=0 tx=0).stop 2024-09-19T08:43:22.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.941+0000 7f2159199640 1 -- 172.21.15.98:0/259212894 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f215415c550 msgr2=0x7f215415b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:22.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.941+0000 7f2159199640 1 --2- 172.21.15.98:0/259212894 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f215415c550 0x7f215415b3d0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f214c00cc30 tx=0x7f214c00a5c0 comp rx=0 tx=0).stop 2024-09-19T08:43:22.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.941+0000 7f2159199640 1 -- 172.21.15.98:0/259212894 shutdown_connections 2024-09-19T08:43:22.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.941+0000 7f2159199640 1 --2- 172.21.15.98:0/259212894 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f21400679e0 0x7f2140069ea0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:22.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.941+0000 7f2159199640 1 --2- 172.21.15.98:0/259212894 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f215415c550 0x7f215415b3d0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:22.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.941+0000 7f2159199640 1 --2- 172.21.15.98:0/259212894 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21540cf900 0x7f215415ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:22.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.941+0000 7f2159199640 1 --2- 172.21.15.98:0/259212894 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f215406c3a0 0x7f21541506e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:22.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.941+0000 7f2159199640 1 -- 172.21.15.98:0/259212894 >> 172.21.15.98:0/259212894 conn(0x7f21540b2010 msgr2=0x7f215406bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:22.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.941+0000 7f2159199640 1 -- 172.21.15.98:0/259212894 shutdown_connections 2024-09-19T08:43:22.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:22.941+0000 7f2159199640 1 -- 172.21.15.98:0/259212894 wait complete. 2024-09-19T08:43:22.960 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-19T08:43:22.878502+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258504,"kb_used_data":8968,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716216,"statfs":{"total":773094113280,"available":772829405184,"internally_reserved":0,"allocated":9183232,"data_stored":4593377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001989"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.597262+0000","last_change":"2024-09-19T08:43:18.566277+0000","last_active":"2024-09-19T08:43:18.597262+0000","last_peered":"2024-09-19T08:43:18.597262+0000","last_clean":"2024-09-19T08:43:18.597262+0000","last_became_active":"2024-09-19T08:43:18.565557+0000","last_became_peered":"2024-09-19T08:43:18.565557+0000","last_unstale":"2024-09-19T08:43:18.597262+0000","last_undegraded":"2024-09-19T08:43:18.597262+0000","last_fullsized":"2024-09-19T08:43:18.597262+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:02:42.157284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.578966+0000","last_change":"2024-09-19T08:43:18.575092+0000","last_active":"2024-09-19T08:43:18.578966+0000","last_peered":"2024-09-19T08:43:18.578966+0000","last_clean":"2024-09-19T08:43:18.578966+0000","last_became_active":"2024-09-19T08:43:18.574871+0000","last_became_peered":"2024-09-19T08:43:18.574871+0000","last_unstale":"2024-09-19T08:43:18.578966+0000","last_undegraded":"2024-09-19T08:43:18.578966+0000","last_fullsized":"2024-09-19T08:43:18.578966+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:08:31.920640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.595540+0000","last_change":"2024-09-19T08:43:18.589657+0000","last_active":"2024-09-19T08:43:18.595540+0000","last_peered":"2024-09-19T08:43:18.595540+0000","last_clean":"2024-09-19T08:43:18.595540+0000","last_became_active":"2024-09-19T08:43:18.589051+0000","last_became_peered":"2024-09-19T08:43:18.589051+0000","last_unstale":"2024-09-19T08:43:18.595540+0000","last_undegraded":"2024-09-19T08:43:18.595540+0000","last_fullsized":"2024-09-19T08:43:18.595540+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:01:01.771719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.578153+0000","last_change":"2024-09-19T08:43:18.575315+0000","last_active":"2024-09-19T08:43:18.578153+0000","last_peered":"2024-09-19T08:43:18.578153+0000","last_clean":"2024-09-19T08:43:18.578153+0000","last_became_active":"2024-09-19T08:43:18.575034+0000","last_became_peered":"2024-09-19T08:43:18.575034+0000","last_unstale":"2024-09-19T08:43:18.578153+0000","last_undegraded":"2024-09-19T08:43:18.578153+0000","last_fullsized":"2024-09-19T08:43:18.578153+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:04:08.002113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.590927+0000","last_change":"2024-09-19T08:43:18.584177+0000","last_active":"2024-09-19T08:43:18.590927+0000","last_peered":"2024-09-19T08:43:18.590927+0000","last_clean":"2024-09-19T08:43:18.590927+0000","last_became_active":"2024-09-19T08:43:18.583392+0000","last_became_peered":"2024-09-19T08:43:18.583392+0000","last_unstale":"2024-09-19T08:43:18.590927+0000","last_undegraded":"2024-09-19T08:43:18.590927+0000","last_fullsized":"2024-09-19T08:43:18.590927+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:52.860766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.592012+0000","last_change":"2024-09-19T08:43:18.582470+0000","last_active":"2024-09-19T08:43:18.592012+0000","last_peered":"2024-09-19T08:43:18.592012+0000","last_clean":"2024-09-19T08:43:18.592012+0000","last_became_active":"2024-09-19T08:43:18.581752+0000","last_became_peered":"2024-09-19T08:43:18.581752+0000","last_unstale":"2024-09-19T08:43:18.592012+0000","last_undegraded":"2024-09-19T08:43:18.592012+0000","last_fullsized":"2024-09-19T08:43:18.592012+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:20:43.429735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.572694+0000","last_change":"2024-09-19T08:43:18.564949+0000","last_active":"2024-09-19T08:43:18.572694+0000","last_peered":"2024-09-19T08:43:18.572694+0000","last_clean":"2024-09-19T08:43:18.572694+0000","last_became_active":"2024-09-19T08:43:18.564564+0000","last_became_peered":"2024-09-19T08:43:18.564564+0000","last_unstale":"2024-09-19T08:43:18.572694+0000","last_undegraded":"2024-09-19T08:43:18.572694+0000","last_fullsized":"2024-09-19T08:43:18.572694+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:24.584485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.592908+0000","last_change":"2024-09-19T08:43:18.584505+0000","last_active":"2024-09-19T08:43:18.592908+0000","last_peered":"2024-09-19T08:43:18.592908+0000","last_clean":"2024-09-19T08:43:18.592908+0000","last_became_active":"2024-09-19T08:43:18.583862+0000","last_became_peered":"2024-09-19T08:43:18.583862+0000","last_unstale":"2024-09-19T08:43:18.592908+0000","last_undegraded":"2024-09-19T08:43:18.592908+0000","last_fullsized":"2024-09-19T08:43:18.592908+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:05:52.194614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.598417+0000","last_change":"2024-09-19T08:43:18.591014+0000","last_active":"2024-09-19T08:43:18.598417+0000","last_peered":"2024-09-19T08:43:18.598417+0000","last_clean":"2024-09-19T08:43:18.598417+0000","last_became_active":"2024-09-19T08:43:18.590437+0000","last_became_peered":"2024-09-19T08:43:18.590437+0000","last_unstale":"2024-09-19T08:43:18.598417+0000","last_undegraded":"2024-09-19T08:43:18.598417+0000","last_fullsized":"2024-09-19T08:43:18.598417+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:25:34.623653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.596803+0000","last_change":"2024-09-19T08:43:18.583067+0000","last_active":"2024-09-19T08:43:18.596803+0000","last_peered":"2024-09-19T08:43:18.596803+0000","last_clean":"2024-09-19T08:43:18.596803+0000","last_became_active":"2024-09-19T08:43:18.582125+0000","last_became_peered":"2024-09-19T08:43:18.582125+0000","last_unstale":"2024-09-19T08:43:18.596803+0000","last_undegraded":"2024-09-19T08:43:18.596803+0000","last_fullsized":"2024-09-19T08:43:18.596803+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:23.399707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.581657+0000","last_change":"2024-09-19T08:43:18.578534+0000","last_active":"2024-09-19T08:43:18.581657+0000","last_peered":"2024-09-19T08:43:18.581657+0000","last_clean":"2024-09-19T08:43:18.581657+0000","last_became_active":"2024-09-19T08:43:18.578188+0000","last_became_peered":"2024-09-19T08:43:18.578188+0000","last_unstale":"2024-09-19T08:43:18.581657+0000","last_undegraded":"2024-09-19T08:43:18.581657+0000","last_fullsized":"2024-09-19T08:43:18.581657+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:03:51.686336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.603204+0000","last_change":"2024-09-19T08:43:18.590275+0000","last_active":"2024-09-19T08:43:18.603204+0000","last_peered":"2024-09-19T08:43:18.603204+0000","last_clean":"2024-09-19T08:43:18.603204+0000","last_became_active":"2024-09-19T08:43:18.589517+0000","last_became_peered":"2024-09-19T08:43:18.589517+0000","last_unstale":"2024-09-19T08:43:18.603204+0000","last_undegraded":"2024-09-19T08:43:18.603204+0000","last_fullsized":"2024-09-19T08:43:18.603204+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:47.552603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.586621+0000","last_change":"2024-09-19T08:43:18.574510+0000","last_active":"2024-09-19T08:43:18.586621+0000","last_peered":"2024-09-19T08:43:18.586621+0000","last_clean":"2024-09-19T08:43:18.586621+0000","last_became_active":"2024-09-19T08:43:18.567597+0000","last_became_peered":"2024-09-19T08:43:18.567597+0000","last_unstale":"2024-09-19T08:43:18.586621+0000","last_undegraded":"2024-09-19T08:43:18.586621+0000","last_fullsized":"2024-09-19T08:43:18.586621+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:11:33.266222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.584820+0000","last_change":"2024-09-19T08:43:18.581350+0000","last_active":"2024-09-19T08:43:18.584820+0000","last_peered":"2024-09-19T08:43:18.584820+0000","last_clean":"2024-09-19T08:43:18.584820+0000","last_became_active":"2024-09-19T08:43:18.580682+0000","last_became_peered":"2024-09-19T08:43:18.580682+0000","last_unstale":"2024-09-19T08:43:18.584820+0000","last_undegraded":"2024-09-19T08:43:18.584820+0000","last_fullsized":"2024-09-19T08:43:18.584820+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:25:08.218756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.579637+0000","last_change":"2024-09-19T08:43:18.578294+0000","last_active":"2024-09-19T08:43:18.579637+0000","last_peered":"2024-09-19T08:43:18.579637+0000","last_clean":"2024-09-19T08:43:18.579637+0000","last_became_active":"2024-09-19T08:43:18.577899+0000","last_became_peered":"2024-09-19T08:43:18.577899+0000","last_unstale":"2024-09-19T08:43:18.579637+0000","last_undegraded":"2024-09-19T08:43:18.579637+0000","last_fullsized":"2024-09-19T08:43:18.579637+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:34:01.829055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.585583+0000","last_change":"2024-09-19T08:43:18.580753+0000","last_active":"2024-09-19T08:43:18.585583+0000","last_peered":"2024-09-19T08:43:18.585583+0000","last_clean":"2024-09-19T08:43:18.585583+0000","last_became_active":"2024-09-19T08:43:18.580000+0000","last_became_peered":"2024-09-19T08:43:18.580000+0000","last_unstale":"2024-09-19T08:43:18.585583+0000","last_undegraded":"2024-09-19T08:43:18.585583+0000","last_fullsized":"2024-09-19T08:43:18.585583+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:03:43.662669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.564895+0000","last_change":"2024-09-19T08:43:18.559389+0000","last_active":"2024-09-19T08:43:18.564895+0000","last_peered":"2024-09-19T08:43:18.564895+0000","last_clean":"2024-09-19T08:43:18.564895+0000","last_became_active":"2024-09-19T08:43:18.558616+0000","last_became_peered":"2024-09-19T08:43:18.558616+0000","last_unstale":"2024-09-19T08:43:18.564895+0000","last_undegraded":"2024-09-19T08:43:18.564895+0000","last_fullsized":"2024-09-19T08:43:18.564895+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:53:12.860581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.585834+0000","last_change":"2024-09-19T08:43:18.584103+0000","last_active":"2024-09-19T08:43:18.585834+0000","last_peered":"2024-09-19T08:43:18.585834+0000","last_clean":"2024-09-19T08:43:18.585834+0000","last_became_active":"2024-09-19T08:43:18.583723+0000","last_became_peered":"2024-09-19T08:43:18.583723+0000","last_unstale":"2024-09-19T08:43:18.585834+0000","last_undegraded":"2024-09-19T08:43:18.585834+0000","last_fullsized":"2024-09-19T08:43:18.585834+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:15:02.305566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.562376+0000","last_change":"2024-09-19T08:43:18.558969+0000","last_active":"2024-09-19T08:43:18.562376+0000","last_peered":"2024-09-19T08:43:18.562376+0000","last_clean":"2024-09-19T08:43:18.562376+0000","last_became_active":"2024-09-19T08:43:18.558683+0000","last_became_peered":"2024-09-19T08:43:18.558683+0000","last_unstale":"2024-09-19T08:43:18.562376+0000","last_undegraded":"2024-09-19T08:43:18.562376+0000","last_fullsized":"2024-09-19T08:43:18.562376+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:55.171132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.572707+0000","last_change":"2024-09-19T08:43:18.565861+0000","last_active":"2024-09-19T08:43:18.572707+0000","last_peered":"2024-09-19T08:43:18.572707+0000","last_clean":"2024-09-19T08:43:18.572707+0000","last_became_active":"2024-09-19T08:43:18.565593+0000","last_became_peered":"2024-09-19T08:43:18.565593+0000","last_unstale":"2024-09-19T08:43:18.572707+0000","last_undegraded":"2024-09-19T08:43:18.572707+0000","last_fullsized":"2024-09-19T08:43:18.572707+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:08:04.199261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.572874+0000","last_change":"2024-09-19T08:43:18.565230+0000","last_active":"2024-09-19T08:43:18.572874+0000","last_peered":"2024-09-19T08:43:18.572874+0000","last_clean":"2024-09-19T08:43:18.572874+0000","last_became_active":"2024-09-19T08:43:18.564390+0000","last_became_peered":"2024-09-19T08:43:18.564390+0000","last_unstale":"2024-09-19T08:43:18.572874+0000","last_undegraded":"2024-09-19T08:43:18.572874+0000","last_fullsized":"2024-09-19T08:43:18.572874+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:00:30.923234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.580025+0000","last_change":"2024-09-19T08:43:18.577256+0000","last_active":"2024-09-19T08:43:18.580025+0000","last_peered":"2024-09-19T08:43:18.580025+0000","last_clean":"2024-09-19T08:43:18.580025+0000","last_became_active":"2024-09-19T08:43:18.574458+0000","last_became_peered":"2024-09-19T08:43:18.574458+0000","last_unstale":"2024-09-19T08:43:18.580025+0000","last_undegraded":"2024-09-19T08:43:18.580025+0000","last_fullsized":"2024-09-19T08:43:18.580025+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:58:15.256400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.590602+0000","last_change":"2024-09-19T08:43:18.579795+0000","last_active":"2024-09-19T08:43:18.590602+0000","last_peered":"2024-09-19T08:43:18.590602+0000","last_clean":"2024-09-19T08:43:18.590602+0000","last_became_active":"2024-09-19T08:43:18.567541+0000","last_became_peered":"2024-09-19T08:43:18.567541+0000","last_unstale":"2024-09-19T08:43:18.590602+0000","last_undegraded":"2024-09-19T08:43:18.590602+0000","last_fullsized":"2024-09-19T08:43:18.590602+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:31:30.847824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.572871+0000","last_change":"2024-09-19T08:43:18.565807+0000","last_active":"2024-09-19T08:43:18.572871+0000","last_peered":"2024-09-19T08:43:18.572871+0000","last_clean":"2024-09-19T08:43:18.572871+0000","last_became_active":"2024-09-19T08:43:18.565531+0000","last_became_peered":"2024-09-19T08:43:18.565531+0000","last_unstale":"2024-09-19T08:43:18.572871+0000","last_undegraded":"2024-09-19T08:43:18.572871+0000","last_fullsized":"2024-09-19T08:43:18.572871+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:16:02.413465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.580311+0000","last_change":"2024-09-19T08:43:18.576737+0000","last_active":"2024-09-19T08:43:18.580311+0000","last_peered":"2024-09-19T08:43:18.580311+0000","last_clean":"2024-09-19T08:43:18.580311+0000","last_became_active":"2024-09-19T08:43:18.575164+0000","last_became_peered":"2024-09-19T08:43:18.575164+0000","last_unstale":"2024-09-19T08:43:18.580311+0000","last_undegraded":"2024-09-19T08:43:18.580311+0000","last_fullsized":"2024-09-19T08:43:18.580311+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:30.004688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.587586+0000","last_change":"2024-09-19T08:43:18.576295+0000","last_active":"2024-09-19T08:43:18.587586+0000","last_peered":"2024-09-19T08:43:18.587586+0000","last_clean":"2024-09-19T08:43:18.587586+0000","last_became_active":"2024-09-19T08:43:18.575555+0000","last_became_peered":"2024-09-19T08:43:18.575555+0000","last_unstale":"2024-09-19T08:43:18.587586+0000","last_undegraded":"2024-09-19T08:43:18.587586+0000","last_fullsized":"2024-09-19T08:43:18.587586+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:49.294782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.574043+0000","last_change":"2024-09-19T08:43:18.563945+0000","last_active":"2024-09-19T08:43:18.574043+0000","last_peered":"2024-09-19T08:43:18.574043+0000","last_clean":"2024-09-19T08:43:18.574043+0000","last_became_active":"2024-09-19T08:43:18.562624+0000","last_became_peered":"2024-09-19T08:43:18.562624+0000","last_unstale":"2024-09-19T08:43:18.574043+0000","last_undegraded":"2024-09-19T08:43:18.574043+0000","last_fullsized":"2024-09-19T08:43:18.574043+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:54:02.984184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.596752+0000","last_change":"2024-09-19T08:43:18.569435+0000","last_active":"2024-09-19T08:43:18.596752+0000","last_peered":"2024-09-19T08:43:18.596752+0000","last_clean":"2024-09-19T08:43:18.596752+0000","last_became_active":"2024-09-19T08:43:18.565378+0000","last_became_peered":"2024-09-19T08:43:18.565378+0000","last_unstale":"2024-09-19T08:43:18.596752+0000","last_undegraded":"2024-09-19T08:43:18.596752+0000","last_fullsized":"2024-09-19T08:43:18.596752+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:35:04.453546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.582133+0000","last_change":"2024-09-19T08:43:18.580938+0000","last_active":"2024-09-19T08:43:18.582133+0000","last_peered":"2024-09-19T08:43:18.582133+0000","last_clean":"2024-09-19T08:43:18.582133+0000","last_became_active":"2024-09-19T08:43:18.580682+0000","last_became_peered":"2024-09-19T08:43:18.580682+0000","last_unstale":"2024-09-19T08:43:18.582133+0000","last_undegraded":"2024-09-19T08:43:18.582133+0000","last_fullsized":"2024-09-19T08:43:18.582133+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:17:03.137472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.585854+0000","last_change":"2024-09-19T08:43:18.583833+0000","last_active":"2024-09-19T08:43:18.585854+0000","last_peered":"2024-09-19T08:43:18.585854+0000","last_clean":"2024-09-19T08:43:18.585854+0000","last_became_active":"2024-09-19T08:43:18.583059+0000","last_became_peered":"2024-09-19T08:43:18.583059+0000","last_unstale":"2024-09-19T08:43:18.585854+0000","last_undegraded":"2024-09-19T08:43:18.585854+0000","last_fullsized":"2024-09-19T08:43:18.585854+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:21:05.081819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.560841+0000","last_change":"2024-09-19T08:43:18.556307+0000","last_active":"2024-09-19T08:43:18.560841+0000","last_peered":"2024-09-19T08:43:18.560841+0000","last_clean":"2024-09-19T08:43:18.560841+0000","last_became_active":"2024-09-19T08:43:18.555508+0000","last_became_peered":"2024-09-19T08:43:18.555508+0000","last_unstale":"2024-09-19T08:43:18.560841+0000","last_undegraded":"2024-09-19T08:43:18.560841+0000","last_fullsized":"2024-09-19T08:43:18.560841+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:05:52.194614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.600320+0000","last_change":"2024-09-19T08:43:18.589798+0000","last_active":"2024-09-19T08:43:18.600320+0000","last_peered":"2024-09-19T08:43:18.600320+0000","last_clean":"2024-09-19T08:43:18.600320+0000","last_became_active":"2024-09-19T08:43:18.589190+0000","last_became_peered":"2024-09-19T08:43:18.589190+0000","last_unstale":"2024-09-19T08:43:18.600320+0000","last_undegraded":"2024-09-19T08:43:18.600320+0000","last_fullsized":"2024-09-19T08:43:18.600320+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:02.428202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.585815+0000","last_change":"2024-09-19T08:43:18.582575+0000","last_active":"2024-09-19T08:43:18.585815+0000","last_peered":"2024-09-19T08:43:18.585815+0000","last_clean":"2024-09-19T08:43:18.585815+0000","last_became_active":"2024-09-19T08:43:18.581884+0000","last_became_peered":"2024-09-19T08:43:18.581884+0000","last_unstale":"2024-09-19T08:43:18.585815+0000","last_undegraded":"2024-09-19T08:43:18.585815+0000","last_fullsized":"2024-09-19T08:43:18.585815+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:49.294782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"82'4","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.603715+0000","last_change":"2024-09-19T08:43:17.584571+0000","last_active":"2024-09-19T08:43:18.603715+0000","last_peered":"2024-09-19T08:43:18.603715+0000","last_clean":"2024-09-19T08:43:18.603715+0000","last_became_active":"2024-09-19T08:43:17.583331+0000","last_became_peered":"2024-09-19T08:43:17.583331+0000","last_unstale":"2024-09-19T08:43:18.603715+0000","last_undegraded":"2024-09-19T08:43:18.603715+0000","last_fullsized":"2024-09-19T08:43:18.603715+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:34.811252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.597312+0000","last_change":"2024-09-19T08:43:18.570261+0000","last_active":"2024-09-19T08:43:18.597312+0000","last_peered":"2024-09-19T08:43:18.597312+0000","last_clean":"2024-09-19T08:43:18.597312+0000","last_became_active":"2024-09-19T08:43:18.569517+0000","last_became_peered":"2024-09-19T08:43:18.569517+0000","last_unstale":"2024-09-19T08:43:18.597312+0000","last_undegraded":"2024-09-19T08:43:18.597312+0000","last_fullsized":"2024-09-19T08:43:18.597312+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:28:02.369905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.578085+0000","last_change":"2024-09-19T08:43:17.569558+0000","last_active":"2024-09-19T08:43:18.578085+0000","last_peered":"2024-09-19T08:43:18.578085+0000","last_clean":"2024-09-19T08:43:18.578085+0000","last_became_active":"2024-09-19T08:43:17.568823+0000","last_became_peered":"2024-09-19T08:43:17.568823+0000","last_unstale":"2024-09-19T08:43:18.578085+0000","last_undegraded":"2024-09-19T08:43:18.578085+0000","last_fullsized":"2024-09-19T08:43:18.578085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:20:42.425798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.596876+0000","last_change":"2024-09-19T08:43:18.588777+0000","last_active":"2024-09-19T08:43:18.596876+0000","last_peered":"2024-09-19T08:43:18.596876+0000","last_clean":"2024-09-19T08:43:18.596876+0000","last_became_active":"2024-09-19T08:43:18.587991+0000","last_became_peered":"2024-09-19T08:43:18.587991+0000","last_unstale":"2024-09-19T08:43:18.596876+0000","last_undegraded":"2024-09-19T08:43:18.596876+0000","last_fullsized":"2024-09-19T08:43:18.596876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:20.628578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.573469+0000","last_change":"2024-09-19T08:43:17.560190+0000","last_active":"2024-09-19T08:43:18.573469+0000","last_peered":"2024-09-19T08:43:18.573469+0000","last_clean":"2024-09-19T08:43:18.573469+0000","last_became_active":"2024-09-19T08:43:17.559448+0000","last_became_peered":"2024-09-19T08:43:17.559448+0000","last_unstale":"2024-09-19T08:43:18.573469+0000","last_undegraded":"2024-09-19T08:43:18.573469+0000","last_fullsized":"2024-09-19T08:43:18.573469+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:49.585477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.587533+0000","last_change":"2024-09-19T08:43:18.576784+0000","last_active":"2024-09-19T08:43:18.587533+0000","last_peered":"2024-09-19T08:43:18.587533+0000","last_clean":"2024-09-19T08:43:18.587533+0000","last_became_active":"2024-09-19T08:43:18.575989+0000","last_became_peered":"2024-09-19T08:43:18.575989+0000","last_unstale":"2024-09-19T08:43:18.587533+0000","last_undegraded":"2024-09-19T08:43:18.587533+0000","last_fullsized":"2024-09-19T08:43:18.587533+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:52.860766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.573978+0000","last_change":"2024-09-19T08:43:17.559752+0000","last_active":"2024-09-19T08:43:18.573978+0000","last_peered":"2024-09-19T08:43:18.573978+0000","last_clean":"2024-09-19T08:43:18.573978+0000","last_became_active":"2024-09-19T08:43:17.559327+0000","last_became_peered":"2024-09-19T08:43:17.559327+0000","last_unstale":"2024-09-19T08:43:18.573978+0000","last_undegraded":"2024-09-19T08:43:18.573978+0000","last_fullsized":"2024-09-19T08:43:18.573978+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:40:48.290845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.578882+0000","last_change":"2024-09-19T08:43:18.574634+0000","last_active":"2024-09-19T08:43:18.578882+0000","last_peered":"2024-09-19T08:43:18.578882+0000","last_clean":"2024-09-19T08:43:18.578882+0000","last_became_active":"2024-09-19T08:43:18.574329+0000","last_became_peered":"2024-09-19T08:43:18.574329+0000","last_unstale":"2024-09-19T08:43:18.578882+0000","last_undegraded":"2024-09-19T08:43:18.578882+0000","last_fullsized":"2024-09-19T08:43:18.578882+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:35:37.122315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.582703+0000","last_change":"2024-09-19T08:43:17.570306+0000","last_active":"2024-09-19T08:43:18.582703+0000","last_peered":"2024-09-19T08:43:18.582703+0000","last_clean":"2024-09-19T08:43:18.582703+0000","last_became_active":"2024-09-19T08:43:17.569270+0000","last_became_peered":"2024-09-19T08:43:17.569270+0000","last_unstale":"2024-09-19T08:43:18.582703+0000","last_undegraded":"2024-09-19T08:43:18.582703+0000","last_fullsized":"2024-09-19T08:43:18.582703+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:29.737615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.573985+0000","last_change":"2024-09-19T08:43:18.562021+0000","last_active":"2024-09-19T08:43:18.573985+0000","last_peered":"2024-09-19T08:43:18.573985+0000","last_clean":"2024-09-19T08:43:18.573985+0000","last_became_active":"2024-09-19T08:43:18.560801+0000","last_became_peered":"2024-09-19T08:43:18.560801+0000","last_unstale":"2024-09-19T08:43:18.573985+0000","last_undegraded":"2024-09-19T08:43:18.573985+0000","last_fullsized":"2024-09-19T08:43:18.573985+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:25:34.623653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.578040+0000","last_change":"2024-09-19T08:43:17.570577+0000","last_active":"2024-09-19T08:43:18.578040+0000","last_peered":"2024-09-19T08:43:18.578040+0000","last_clean":"2024-09-19T08:43:18.578040+0000","last_became_active":"2024-09-19T08:43:17.569523+0000","last_became_peered":"2024-09-19T08:43:17.569523+0000","last_unstale":"2024-09-19T08:43:18.578040+0000","last_undegraded":"2024-09-19T08:43:18.578040+0000","last_fullsized":"2024-09-19T08:43:18.578040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:08.991355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.588709+0000","last_change":"2024-09-19T08:43:18.574271+0000","last_active":"2024-09-19T08:43:18.588709+0000","last_peered":"2024-09-19T08:43:18.588709+0000","last_clean":"2024-09-19T08:43:18.588709+0000","last_became_active":"2024-09-19T08:43:18.568518+0000","last_became_peered":"2024-09-19T08:43:18.568518+0000","last_unstale":"2024-09-19T08:43:18.588709+0000","last_undegraded":"2024-09-19T08:43:18.588709+0000","last_fullsized":"2024-09-19T08:43:18.588709+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:12:23.576576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.538991+0000","last_change":"2024-09-19T08:43:17.576694+0000","last_active":"2024-09-19T08:43:18.538991+0000","last_peered":"2024-09-19T08:43:18.538991+0000","last_clean":"2024-09-19T08:43:18.538991+0000","last_became_active":"2024-09-19T08:43:17.576289+0000","last_became_peered":"2024-09-19T08:43:17.576289+0000","last_unstale":"2024-09-19T08:43:18.538991+0000","last_undegraded":"2024-09-19T08:43:18.538991+0000","last_fullsized":"2024-09-19T08:43:18.538991+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:03:42.658731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.573592+0000","last_change":"2024-09-19T08:43:18.569637+0000","last_active":"2024-09-19T08:43:18.573592+0000","last_peered":"2024-09-19T08:43:18.573592+0000","last_clean":"2024-09-19T08:43:18.573592+0000","last_became_active":"2024-09-19T08:43:18.568839+0000","last_became_peered":"2024-09-19T08:43:18.568839+0000","last_unstale":"2024-09-19T08:43:18.573592+0000","last_undegraded":"2024-09-19T08:43:18.573592+0000","last_fullsized":"2024-09-19T08:43:18.573592+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:33:13.094118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.560541+0000","last_change":"2024-09-19T08:43:17.557557+0000","last_active":"2024-09-19T08:43:18.560541+0000","last_peered":"2024-09-19T08:43:18.560541+0000","last_clean":"2024-09-19T08:43:18.560541+0000","last_became_active":"2024-09-19T08:43:17.557173+0000","last_became_peered":"2024-09-19T08:43:17.557173+0000","last_unstale":"2024-09-19T08:43:18.560541+0000","last_undegraded":"2024-09-19T08:43:18.560541+0000","last_fullsized":"2024-09-19T08:43:18.560541+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:30:05.476756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.578527+0000","last_change":"2024-09-19T08:43:18.576101+0000","last_active":"2024-09-19T08:43:18.578527+0000","last_peered":"2024-09-19T08:43:18.578527+0000","last_clean":"2024-09-19T08:43:18.578527+0000","last_became_active":"2024-09-19T08:43:18.575785+0000","last_became_peered":"2024-09-19T08:43:18.575785+0000","last_unstale":"2024-09-19T08:43:18.578527+0000","last_undegraded":"2024-09-19T08:43:18.578527+0000","last_fullsized":"2024-09-19T08:43:18.578527+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:19.547274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.556098+0000","last_change":"2024-09-19T08:43:17.575479+0000","last_active":"2024-09-19T08:43:18.556098+0000","last_peered":"2024-09-19T08:43:18.556098+0000","last_clean":"2024-09-19T08:43:18.556098+0000","last_became_active":"2024-09-19T08:43:17.574714+0000","last_became_peered":"2024-09-19T08:43:17.574714+0000","last_unstale":"2024-09-19T08:43:18.556098+0000","last_undegraded":"2024-09-19T08:43:18.556098+0000","last_fullsized":"2024-09-19T08:43:18.556098+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:50:41.082627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.598920+0000","last_change":"2024-09-19T08:43:18.593882+0000","last_active":"2024-09-19T08:43:18.598920+0000","last_peered":"2024-09-19T08:43:18.598920+0000","last_clean":"2024-09-19T08:43:18.598920+0000","last_became_active":"2024-09-19T08:43:18.590089+0000","last_became_peered":"2024-09-19T08:43:18.590089+0000","last_unstale":"2024-09-19T08:43:18.598920+0000","last_undegraded":"2024-09-19T08:43:18.598920+0000","last_fullsized":"2024-09-19T08:43:18.598920+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:54:02.984184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.558474+0000","last_change":"2024-09-19T08:43:17.583764+0000","last_active":"2024-09-19T08:43:18.558474+0000","last_peered":"2024-09-19T08:43:18.558474+0000","last_clean":"2024-09-19T08:43:18.558474+0000","last_became_active":"2024-09-19T08:43:17.583246+0000","last_became_peered":"2024-09-19T08:43:17.583246+0000","last_unstale":"2024-09-19T08:43:18.558474+0000","last_undegraded":"2024-09-19T08:43:18.558474+0000","last_fullsized":"2024-09-19T08:43:18.558474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:10:16.114687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.572696+0000","last_change":"2024-09-19T08:43:18.571186+0000","last_active":"2024-09-19T08:43:18.572696+0000","last_peered":"2024-09-19T08:43:18.572696+0000","last_clean":"2024-09-19T08:43:18.572696+0000","last_became_active":"2024-09-19T08:43:18.570849+0000","last_became_peered":"2024-09-19T08:43:18.570849+0000","last_unstale":"2024-09-19T08:43:18.572696+0000","last_undegraded":"2024-09-19T08:43:18.572696+0000","last_fullsized":"2024-09-19T08:43:18.572696+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:01:22.218265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"82'3","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.596758+0000","last_change":"2024-09-19T08:43:17.577021+0000","last_active":"2024-09-19T08:43:18.596758+0000","last_peered":"2024-09-19T08:43:18.596758+0000","last_clean":"2024-09-19T08:43:18.596758+0000","last_became_active":"2024-09-19T08:43:17.575945+0000","last_became_peered":"2024-09-19T08:43:17.575945+0000","last_unstale":"2024-09-19T08:43:18.596758+0000","last_undegraded":"2024-09-19T08:43:18.596758+0000","last_fullsized":"2024-09-19T08:43:18.596758+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:44:01.415820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.577612+0000","last_change":"2024-09-19T08:43:18.565469+0000","last_active":"2024-09-19T08:43:18.577612+0000","last_peered":"2024-09-19T08:43:18.577612+0000","last_clean":"2024-09-19T08:43:18.577612+0000","last_became_active":"2024-09-19T08:43:18.564762+0000","last_became_peered":"2024-09-19T08:43:18.564762+0000","last_unstale":"2024-09-19T08:43:18.577612+0000","last_undegraded":"2024-09-19T08:43:18.577612+0000","last_fullsized":"2024-09-19T08:43:18.577612+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:12:23.576576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"82'6","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.600735+0000","last_change":"2024-09-19T08:43:17.572935+0000","last_active":"2024-09-19T08:43:18.600735+0000","last_peered":"2024-09-19T08:43:18.600735+0000","last_clean":"2024-09-19T08:43:18.600735+0000","last_became_active":"2024-09-19T08:43:17.572074+0000","last_became_peered":"2024-09-19T08:43:17.572074+0000","last_unstale":"2024-09-19T08:43:18.600735+0000","last_undegraded":"2024-09-19T08:43:18.600735+0000","last_fullsized":"2024-09-19T08:43:18.600735+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:32.915411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.562052+0000","last_change":"2024-09-19T08:43:18.556991+0000","last_active":"2024-09-19T08:43:18.562052+0000","last_peered":"2024-09-19T08:43:18.562052+0000","last_clean":"2024-09-19T08:43:18.562052+0000","last_became_active":"2024-09-19T08:43:18.555019+0000","last_became_peered":"2024-09-19T08:43:18.555019+0000","last_unstale":"2024-09-19T08:43:18.562052+0000","last_undegraded":"2024-09-19T08:43:18.562052+0000","last_fullsized":"2024-09-19T08:43:18.562052+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:21:35.930304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.583012+0000","last_change":"2024-09-19T08:43:17.565687+0000","last_active":"2024-09-19T08:43:18.583012+0000","last_peered":"2024-09-19T08:43:18.583012+0000","last_clean":"2024-09-19T08:43:18.583012+0000","last_became_active":"2024-09-19T08:43:17.565193+0000","last_became_peered":"2024-09-19T08:43:17.565193+0000","last_unstale":"2024-09-19T08:43:18.583012+0000","last_undegraded":"2024-09-19T08:43:18.583012+0000","last_fullsized":"2024-09-19T08:43:18.583012+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:42.242000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.586711+0000","last_change":"2024-09-19T08:43:18.574914+0000","last_active":"2024-09-19T08:43:18.586711+0000","last_peered":"2024-09-19T08:43:18.586711+0000","last_clean":"2024-09-19T08:43:18.586711+0000","last_became_active":"2024-09-19T08:43:18.568554+0000","last_became_peered":"2024-09-19T08:43:18.568554+0000","last_unstale":"2024-09-19T08:43:18.586711+0000","last_undegraded":"2024-09-19T08:43:18.586711+0000","last_fullsized":"2024-09-19T08:43:18.586711+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:03:43.662669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.548772+0000","last_change":"2024-09-19T08:43:17.565587+0000","last_active":"2024-09-19T08:43:18.548772+0000","last_peered":"2024-09-19T08:43:18.548772+0000","last_clean":"2024-09-19T08:43:18.548772+0000","last_became_active":"2024-09-19T08:43:17.565187+0000","last_became_peered":"2024-09-19T08:43:17.565187+0000","last_unstale":"2024-09-19T08:43:18.548772+0000","last_undegraded":"2024-09-19T08:43:18.548772+0000","last_fullsized":"2024-09-19T08:43:18.548772+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:01.234243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.581752+0000","last_change":"2024-09-19T08:43:18.579348+0000","last_active":"2024-09-19T08:43:18.581752+0000","last_peered":"2024-09-19T08:43:18.581752+0000","last_clean":"2024-09-19T08:43:18.581752+0000","last_became_active":"2024-09-19T08:43:18.578396+0000","last_became_peered":"2024-09-19T08:43:18.578396+0000","last_unstale":"2024-09-19T08:43:18.581752+0000","last_undegraded":"2024-09-19T08:43:18.581752+0000","last_fullsized":"2024-09-19T08:43:18.581752+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:23:22.109467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.557401+0000","last_change":"2024-09-19T08:43:17.581563+0000","last_active":"2024-09-19T08:43:18.557401+0000","last_peered":"2024-09-19T08:43:18.557401+0000","last_clean":"2024-09-19T08:43:18.557401+0000","last_became_active":"2024-09-19T08:43:17.580824+0000","last_became_peered":"2024-09-19T08:43:17.580824+0000","last_unstale":"2024-09-19T08:43:18.557401+0000","last_undegraded":"2024-09-19T08:43:18.557401+0000","last_fullsized":"2024-09-19T08:43:18.557401+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:18:08.991355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.579907+0000","last_change":"2024-09-19T08:43:18.577673+0000","last_active":"2024-09-19T08:43:18.579907+0000","last_peered":"2024-09-19T08:43:18.579907+0000","last_clean":"2024-09-19T08:43:18.579907+0000","last_became_active":"2024-09-19T08:43:18.576991+0000","last_became_peered":"2024-09-19T08:43:18.576991+0000","last_unstale":"2024-09-19T08:43:18.579907+0000","last_undegraded":"2024-09-19T08:43:18.579907+0000","last_fullsized":"2024-09-19T08:43:18.579907+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:41:56.080757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"82'8","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.595610+0000","last_change":"2024-09-19T08:43:17.574681+0000","last_active":"2024-09-19T08:43:18.595610+0000","last_peered":"2024-09-19T08:43:18.595610+0000","last_clean":"2024-09-19T08:43:18.595610+0000","last_became_active":"2024-09-19T08:43:17.573802+0000","last_became_peered":"2024-09-19T08:43:17.573802+0000","last_unstale":"2024-09-19T08:43:18.595610+0000","last_undegraded":"2024-09-19T08:43:18.595610+0000","last_fullsized":"2024-09-19T08:43:18.595610+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:11:32.262284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.598975+0000","last_change":"2024-09-19T08:43:18.594954+0000","last_active":"2024-09-19T08:43:18.598975+0000","last_peered":"2024-09-19T08:43:18.598975+0000","last_clean":"2024-09-19T08:43:18.598975+0000","last_became_active":"2024-09-19T08:43:18.594502+0000","last_became_peered":"2024-09-19T08:43:18.594502+0000","last_unstale":"2024-09-19T08:43:18.598975+0000","last_undegraded":"2024-09-19T08:43:18.598975+0000","last_fullsized":"2024-09-19T08:43:18.598975+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:50.589415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.599177+0000","last_change":"2024-09-19T08:43:17.577257+0000","last_active":"2024-09-19T08:43:18.599177+0000","last_peered":"2024-09-19T08:43:18.599177+0000","last_clean":"2024-09-19T08:43:18.599177+0000","last_became_active":"2024-09-19T08:43:17.576335+0000","last_became_peered":"2024-09-19T08:43:17.576335+0000","last_unstale":"2024-09-19T08:43:18.599177+0000","last_undegraded":"2024-09-19T08:43:18.599177+0000","last_fullsized":"2024-09-19T08:43:18.599177+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:07.844032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.597351+0000","last_change":"2024-09-19T08:43:18.570748+0000","last_active":"2024-09-19T08:43:18.597351+0000","last_peered":"2024-09-19T08:43:18.597351+0000","last_clean":"2024-09-19T08:43:18.597351+0000","last_became_active":"2024-09-19T08:43:18.569397+0000","last_became_peered":"2024-09-19T08:43:18.569397+0000","last_unstale":"2024-09-19T08:43:18.597351+0000","last_undegraded":"2024-09-19T08:43:18.597351+0000","last_fullsized":"2024-09-19T08:43:18.597351+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:32:13.718833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.584433+0000","last_change":"2024-09-19T08:43:17.577458+0000","last_active":"2024-09-19T08:43:18.584433+0000","last_peered":"2024-09-19T08:43:18.584433+0000","last_clean":"2024-09-19T08:43:18.584433+0000","last_became_active":"2024-09-19T08:43:17.577051+0000","last_became_peered":"2024-09-19T08:43:17.577051+0000","last_unstale":"2024-09-19T08:43:18.584433+0000","last_undegraded":"2024-09-19T08:43:18.584433+0000","last_fullsized":"2024-09-19T08:43:18.584433+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:57.051641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.584340+0000","last_change":"2024-09-19T08:43:18.574818+0000","last_active":"2024-09-19T08:43:18.584340+0000","last_peered":"2024-09-19T08:43:18.584340+0000","last_clean":"2024-09-19T08:43:18.584340+0000","last_became_active":"2024-09-19T08:43:18.573749+0000","last_became_peered":"2024-09-19T08:43:18.573749+0000","last_unstale":"2024-09-19T08:43:18.584340+0000","last_undegraded":"2024-09-19T08:43:18.584340+0000","last_fullsized":"2024-09-19T08:43:18.584340+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:11:33.266222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.539804+0000","last_change":"2024-09-19T08:43:17.576666+0000","last_active":"2024-09-19T08:43:18.539804+0000","last_peered":"2024-09-19T08:43:18.539804+0000","last_clean":"2024-09-19T08:43:18.539804+0000","last_became_active":"2024-09-19T08:43:17.576270+0000","last_became_peered":"2024-09-19T08:43:17.576270+0000","last_unstale":"2024-09-19T08:43:18.539804+0000","last_undegraded":"2024-09-19T08:43:18.539804+0000","last_fullsized":"2024-09-19T08:43:18.539804+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:01:00.767781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.584799+0000","last_change":"2024-09-19T08:43:18.581214+0000","last_active":"2024-09-19T08:43:18.584799+0000","last_peered":"2024-09-19T08:43:18.584799+0000","last_clean":"2024-09-19T08:43:18.584799+0000","last_became_active":"2024-09-19T08:43:18.580344+0000","last_became_peered":"2024-09-19T08:43:18.580344+0000","last_unstale":"2024-09-19T08:43:18.584799+0000","last_undegraded":"2024-09-19T08:43:18.584799+0000","last_fullsized":"2024-09-19T08:43:18.584799+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:59:01.951721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.573306+0000","last_change":"2024-09-19T08:43:17.569866+0000","last_active":"2024-09-19T08:43:18.573306+0000","last_peered":"2024-09-19T08:43:18.573306+0000","last_clean":"2024-09-19T08:43:18.573306+0000","last_became_active":"2024-09-19T08:43:17.569401+0000","last_became_peered":"2024-09-19T08:43:17.569401+0000","last_unstale":"2024-09-19T08:43:18.573306+0000","last_undegraded":"2024-09-19T08:43:18.573306+0000","last_fullsized":"2024-09-19T08:43:18.573306+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:27.223097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.592705+0000","last_change":"2024-09-19T08:43:18.585441+0000","last_active":"2024-09-19T08:43:18.592705+0000","last_peered":"2024-09-19T08:43:18.592705+0000","last_clean":"2024-09-19T08:43:18.592705+0000","last_became_active":"2024-09-19T08:43:18.573687+0000","last_became_peered":"2024-09-19T08:43:18.573687+0000","last_unstale":"2024-09-19T08:43:18.592705+0000","last_undegraded":"2024-09-19T08:43:18.592705+0000","last_fullsized":"2024-09-19T08:43:18.592705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:53:12.860581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.553342+0000","last_change":"2024-09-19T08:43:17.562799+0000","last_active":"2024-09-19T08:43:18.553342+0000","last_peered":"2024-09-19T08:43:18.553342+0000","last_clean":"2024-09-19T08:43:18.553342+0000","last_became_active":"2024-09-19T08:43:17.562194+0000","last_became_peered":"2024-09-19T08:43:17.562194+0000","last_unstale":"2024-09-19T08:43:18.553342+0000","last_undegraded":"2024-09-19T08:43:18.553342+0000","last_fullsized":"2024-09-19T08:43:18.553342+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:23.376881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.594555+0000","last_change":"2024-09-19T08:43:18.593857+0000","last_active":"2024-09-19T08:43:18.594555+0000","last_peered":"2024-09-19T08:43:18.594555+0000","last_clean":"2024-09-19T08:43:18.594555+0000","last_became_active":"2024-09-19T08:43:18.593498+0000","last_became_peered":"2024-09-19T08:43:18.593498+0000","last_unstale":"2024-09-19T08:43:18.594555+0000","last_undegraded":"2024-09-19T08:43:18.594555+0000","last_fullsized":"2024-09-19T08:43:18.594555+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:09:58.575602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.551881+0000","last_change":"2024-09-19T08:43:17.583981+0000","last_active":"2024-09-19T08:43:18.551881+0000","last_peered":"2024-09-19T08:43:18.551881+0000","last_clean":"2024-09-19T08:43:18.551881+0000","last_became_active":"2024-09-19T08:43:17.583305+0000","last_became_peered":"2024-09-19T08:43:17.583305+0000","last_unstale":"2024-09-19T08:43:18.551881+0000","last_undegraded":"2024-09-19T08:43:18.551881+0000","last_fullsized":"2024-09-19T08:43:18.551881+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:38.205461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.598830+0000","last_change":"2024-09-19T08:43:18.592231+0000","last_active":"2024-09-19T08:43:18.598830+0000","last_peered":"2024-09-19T08:43:18.598830+0000","last_clean":"2024-09-19T08:43:18.598830+0000","last_became_active":"2024-09-19T08:43:18.591670+0000","last_became_peered":"2024-09-19T08:43:18.591670+0000","last_unstale":"2024-09-19T08:43:18.598830+0000","last_undegraded":"2024-09-19T08:43:18.598830+0000","last_fullsized":"2024-09-19T08:43:18.598830+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:58.000860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.547695+0000","last_change":"2024-09-19T08:43:17.570048+0000","last_active":"2024-09-19T08:43:18.547695+0000","last_peered":"2024-09-19T08:43:18.547695+0000","last_clean":"2024-09-19T08:43:18.547695+0000","last_became_active":"2024-09-19T08:43:17.569629+0000","last_became_peered":"2024-09-19T08:43:17.569629+0000","last_unstale":"2024-09-19T08:43:18.547695+0000","last_undegraded":"2024-09-19T08:43:18.547695+0000","last_fullsized":"2024-09-19T08:43:18.547695+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:25:47.531367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.596009+0000","last_change":"2024-09-19T08:43:18.592694+0000","last_active":"2024-09-19T08:43:18.596009+0000","last_peered":"2024-09-19T08:43:18.596009+0000","last_clean":"2024-09-19T08:43:18.596009+0000","last_became_active":"2024-09-19T08:43:18.592050+0000","last_became_peered":"2024-09-19T08:43:18.592050+0000","last_unstale":"2024-09-19T08:43:18.596009+0000","last_undegraded":"2024-09-19T08:43:18.596009+0000","last_fullsized":"2024-09-19T08:43:18.596009+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:58.055578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.595853+0000","last_change":"2024-09-19T08:43:17.568298+0000","last_active":"2024-09-19T08:43:18.595853+0000","last_peered":"2024-09-19T08:43:18.595853+0000","last_clean":"2024-09-19T08:43:18.595853+0000","last_became_active":"2024-09-19T08:43:17.567944+0000","last_became_peered":"2024-09-19T08:43:17.567944+0000","last_unstale":"2024-09-19T08:43:18.595853+0000","last_undegraded":"2024-09-19T08:43:18.595853+0000","last_fullsized":"2024-09-19T08:43:18.595853+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:52:57.202297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.603389+0000","last_change":"2024-09-19T08:43:18.600358+0000","last_active":"2024-09-19T08:43:18.603389+0000","last_peered":"2024-09-19T08:43:18.603389+0000","last_clean":"2024-09-19T08:43:18.603389+0000","last_became_active":"2024-09-19T08:43:18.599737+0000","last_became_peered":"2024-09-19T08:43:18.599737+0000","last_unstale":"2024-09-19T08:43:18.603389+0000","last_undegraded":"2024-09-19T08:43:18.603389+0000","last_fullsized":"2024-09-19T08:43:18.603389+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:27:36.608565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.561851+0000","last_change":"2024-09-19T08:43:17.570348+0000","last_active":"2024-09-19T08:43:18.561851+0000","last_peered":"2024-09-19T08:43:18.561851+0000","last_clean":"2024-09-19T08:43:18.561851+0000","last_became_active":"2024-09-19T08:43:17.561639+0000","last_became_peered":"2024-09-19T08:43:17.561639+0000","last_unstale":"2024-09-19T08:43:18.561851+0000","last_undegraded":"2024-09-19T08:43:18.561851+0000","last_fullsized":"2024-09-19T08:43:18.561851+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:27:51.789175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.596829+0000","last_change":"2024-09-19T08:43:18.588538+0000","last_active":"2024-09-19T08:43:18.596829+0000","last_peered":"2024-09-19T08:43:18.596829+0000","last_clean":"2024-09-19T08:43:18.596829+0000","last_became_active":"2024-09-19T08:43:18.587873+0000","last_became_peered":"2024-09-19T08:43:18.587873+0000","last_unstale":"2024-09-19T08:43:18.596829+0000","last_undegraded":"2024-09-19T08:43:18.596829+0000","last_fullsized":"2024-09-19T08:43:18.596829+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:35:52.860618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.552119+0000","last_change":"2024-09-19T08:43:17.583882+0000","last_active":"2024-09-19T08:43:18.552119+0000","last_peered":"2024-09-19T08:43:18.552119+0000","last_clean":"2024-09-19T08:43:18.552119+0000","last_became_active":"2024-09-19T08:43:17.583109+0000","last_became_peered":"2024-09-19T08:43:17.583109+0000","last_unstale":"2024-09-19T08:43:18.552119+0000","last_undegraded":"2024-09-19T08:43:18.552119+0000","last_fullsized":"2024-09-19T08:43:18.552119+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:45.131969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.577712+0000","last_change":"2024-09-19T08:43:18.563486+0000","last_active":"2024-09-19T08:43:18.577712+0000","last_peered":"2024-09-19T08:43:18.577712+0000","last_clean":"2024-09-19T08:43:18.577712+0000","last_became_active":"2024-09-19T08:43:18.561539+0000","last_became_peered":"2024-09-19T08:43:18.561539+0000","last_unstale":"2024-09-19T08:43:18.577712+0000","last_undegraded":"2024-09-19T08:43:18.577712+0000","last_fullsized":"2024-09-19T08:43:18.577712+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:26:58.000860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.561860+0000","last_change":"2024-09-19T08:43:17.581881+0000","last_active":"2024-09-19T08:43:18.561860+0000","last_peered":"2024-09-19T08:43:18.561860+0000","last_clean":"2024-09-19T08:43:18.561860+0000","last_became_active":"2024-09-19T08:43:17.581267+0000","last_became_peered":"2024-09-19T08:43:17.581267+0000","last_unstale":"2024-09-19T08:43:18.561860+0000","last_undegraded":"2024-09-19T08:43:18.561860+0000","last_fullsized":"2024-09-19T08:43:18.561860+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:55.318457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.591953+0000","last_change":"2024-09-19T08:43:18.582258+0000","last_active":"2024-09-19T08:43:18.591953+0000","last_peered":"2024-09-19T08:43:18.591953+0000","last_clean":"2024-09-19T08:43:18.591953+0000","last_became_active":"2024-09-19T08:43:18.581647+0000","last_became_peered":"2024-09-19T08:43:18.581647+0000","last_unstale":"2024-09-19T08:43:18.591953+0000","last_undegraded":"2024-09-19T08:43:18.591953+0000","last_fullsized":"2024-09-19T08:43:18.591953+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:30.741553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.539406+0000","last_change":"2024-09-19T08:43:17.564416+0000","last_active":"2024-09-19T08:43:18.539406+0000","last_peered":"2024-09-19T08:43:18.539406+0000","last_clean":"2024-09-19T08:43:18.539406+0000","last_became_active":"2024-09-19T08:43:17.564033+0000","last_became_peered":"2024-09-19T08:43:17.564033+0000","last_unstale":"2024-09-19T08:43:18.539406+0000","last_undegraded":"2024-09-19T08:43:18.539406+0000","last_fullsized":"2024-09-19T08:43:18.539406+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:51.856829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.561822+0000","last_change":"2024-09-19T08:43:18.555397+0000","last_active":"2024-09-19T08:43:18.561822+0000","last_peered":"2024-09-19T08:43:18.561822+0000","last_clean":"2024-09-19T08:43:18.561822+0000","last_became_active":"2024-09-19T08:43:18.554764+0000","last_became_peered":"2024-09-19T08:43:18.554764+0000","last_unstale":"2024-09-19T08:43:18.561822+0000","last_undegraded":"2024-09-19T08:43:18.561822+0000","last_fullsized":"2024-09-19T08:43:18.561822+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:27:36.608565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.546717+0000","last_change":"2024-09-19T08:43:17.562806+0000","last_active":"2024-09-19T08:43:18.546717+0000","last_peered":"2024-09-19T08:43:18.546717+0000","last_clean":"2024-09-19T08:43:18.546717+0000","last_became_active":"2024-09-19T08:43:17.562059+0000","last_became_peered":"2024-09-19T08:43:17.562059+0000","last_unstale":"2024-09-19T08:43:18.546717+0000","last_undegraded":"2024-09-19T08:43:18.546717+0000","last_fullsized":"2024-09-19T08:43:18.546717+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:32.915411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.595501+0000","last_change":"2024-09-19T08:43:18.589394+0000","last_active":"2024-09-19T08:43:18.595501+0000","last_peered":"2024-09-19T08:43:18.595501+0000","last_clean":"2024-09-19T08:43:18.595501+0000","last_became_active":"2024-09-19T08:43:18.588256+0000","last_became_peered":"2024-09-19T08:43:18.588256+0000","last_unstale":"2024-09-19T08:43:18.595501+0000","last_undegraded":"2024-09-19T08:43:18.595501+0000","last_fullsized":"2024-09-19T08:43:18.595501+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:58.055578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.555692+0000","last_change":"2024-09-19T08:43:17.581704+0000","last_active":"2024-09-19T08:43:18.555692+0000","last_peered":"2024-09-19T08:43:18.555692+0000","last_clean":"2024-09-19T08:43:18.555692+0000","last_became_active":"2024-09-19T08:43:17.578243+0000","last_became_peered":"2024-09-19T08:43:17.578243+0000","last_unstale":"2024-09-19T08:43:18.555692+0000","last_undegraded":"2024-09-19T08:43:18.555692+0000","last_fullsized":"2024-09-19T08:43:18.555692+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:44:59.748280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.596364+0000","last_change":"2024-09-19T08:43:18.581782+0000","last_active":"2024-09-19T08:43:18.596364+0000","last_peered":"2024-09-19T08:43:18.596364+0000","last_clean":"2024-09-19T08:43:18.596364+0000","last_became_active":"2024-09-19T08:43:18.580951+0000","last_became_peered":"2024-09-19T08:43:18.580951+0000","last_unstale":"2024-09-19T08:43:18.596364+0000","last_undegraded":"2024-09-19T08:43:18.596364+0000","last_fullsized":"2024-09-19T08:43:18.596364+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:47:28.527734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.544968+0000","last_change":"2024-09-19T08:43:17.586343+0000","last_active":"2024-09-19T08:43:18.544968+0000","last_peered":"2024-09-19T08:43:18.544968+0000","last_clean":"2024-09-19T08:43:18.544968+0000","last_became_active":"2024-09-19T08:43:17.585820+0000","last_became_peered":"2024-09-19T08:43:17.585820+0000","last_unstale":"2024-09-19T08:43:18.544968+0000","last_undegraded":"2024-09-19T08:43:18.544968+0000","last_fullsized":"2024-09-19T08:43:18.544968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:19.291567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.593755+0000","last_change":"2024-09-19T08:43:18.593119+0000","last_active":"2024-09-19T08:43:18.593755+0000","last_peered":"2024-09-19T08:43:18.593755+0000","last_clean":"2024-09-19T08:43:18.593755+0000","last_became_active":"2024-09-19T08:43:18.592841+0000","last_became_peered":"2024-09-19T08:43:18.592841+0000","last_unstale":"2024-09-19T08:43:18.593755+0000","last_undegraded":"2024-09-19T08:43:18.593755+0000","last_fullsized":"2024-09-19T08:43:18.593755+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:17.527751+0000","last_clean_scrub_stamp":"2024-09-19T08:43:17.527751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:48:08.780957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.587408+0000","last_change":"2024-09-19T08:43:17.576964+0000","last_active":"2024-09-19T08:43:18.587408+0000","last_peered":"2024-09-19T08:43:18.587408+0000","last_clean":"2024-09-19T08:43:18.587408+0000","last_became_active":"2024-09-19T08:43:17.576207+0000","last_became_peered":"2024-09-19T08:43:17.576207+0000","last_unstale":"2024-09-19T08:43:18.587408+0000","last_undegraded":"2024-09-19T08:43:18.587408+0000","last_fullsized":"2024-09-19T08:43:18.587408+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:16.523813+0000","last_clean_scrub_stamp":"2024-09-19T08:43:16.523813+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:54:01.980247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"71'128","reported_seq":318,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-19T08:43:18.556994+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:43:18.556994+0000","last_peered":"2024-09-19T08:43:18.556994+0000","last_clean":"2024-09-19T08:43:18.556994+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:43:18.556994+0000","last_undegraded":"2024-09-19T08:43:18.556994+0000","last_fullsized":"2024-09-19T08:43:18.556994+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509505,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32756,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339084,"statfs":{"total":96636764160,"available":96603222016,"internally_reserved":0,"allocated":1597440,"data_stored":1019362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574914,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1011712,"data_stored":429618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509507,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":987136,"data_stored":422149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509507,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":987136,"data_stored":422149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542209,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":987136,"data_stored":422149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509504,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1003520,"data_stored":429506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509506,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1011712,"data_stored":429118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509504,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32748,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1597440,"data_stored":1019326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:43:22.964 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T08:43:22.964 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-19T08:43:23.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.129+0000 7f4c440f8640 1 Processor -- start 2024-09-19T08:43:23.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.129+0000 7f4c440f8640 1 -- start start 2024-09-19T08:43:23.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.129+0000 7f4c440f8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c3c145b70 0x7f4c3c06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:23.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.129+0000 7f4c440f8640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c3c06c5f0 con 0x7f4c3c06c1d0 2024-09-19T08:43:23.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.129+0000 7f4c440f8640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c3c073530 con 0x7f4c3c071ac0 2024-09-19T08:43:23.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.129+0000 7f4c440f8640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4c3c071f20 con 0x7f4c3c145b70 2024-09-19T08:43:23.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.129+0000 7f4c41693640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f4c3c06c1d0 0x7f4c3c0713b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40074/0 (socket says 172.21.15.98:40074) 2024-09-19T08:43:23.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.129+0000 7f4c41693640 1 -- 172.21.15.98:0/3573549258 learned_addr learned my addr 172.21.15.98:0/3573549258 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:23.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.129+0000 7f4c41e94640 1 --2- 172.21.15.98:0/3573549258 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c3c145b70 0x7f4c3c06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:23.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.129+0000 7f4c40e92640 1 -- 172.21.15.98:0/3573549258 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 614919257 0 0) 0x7f4c3c06c5f0 con 0x7f4c3c06c1d0 2024-09-19T08:43:23.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.129+0000 7f4c40e92640 1 -- 172.21.15.98:0/3573549258 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c28003530 con 0x7f4c3c06c1d0 2024-09-19T08:43:23.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.129+0000 7f4c40e92640 1 -- 172.21.15.98:0/3573549258 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1792686426 0 0) 0x7f4c28003530 con 0x7f4c3c06c1d0 2024-09-19T08:43:23.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.129+0000 7f4c40e92640 1 -- 172.21.15.98:0/3573549258 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c3c145b70 msgr2=0x7f4c3c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:23.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.129+0000 7f4c40e92640 1 --2- 172.21.15.98:0/3573549258 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c3c145b70 0x7f4c3c06bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:23.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.129+0000 7f4c40e92640 1 -- 172.21.15.98:0/3573549258 >> v1:172.21.15.139:6789/0 conn(0x7f4c3c071ac0 legacy=0x7f4c3c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:23.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.129+0000 7f4c40e92640 1 -- 172.21.15.98:0/3573549258 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c3c073810 con 0x7f4c3c06c1d0 2024-09-19T08:43:23.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.129+0000 7f4c40e92640 1 -- 172.21.15.98:0/3573549258 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f4c2c001680 con 0x7f4c3c06c1d0 2024-09-19T08:43:23.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.129+0000 7f4c40e92640 1 -- 172.21.15.98:0/3573549258 >> v1:172.21.15.98:6789/0 conn(0x7f4c3c06c1d0 legacy=0x7f4c3c0713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:23.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.129+0000 7f4c40e92640 1 --2- 172.21.15.98:0/3573549258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c280039f0 0x7f4c28003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:23.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.129+0000 7f4c40e92640 1 --2- 172.21.15.98:0/3573549258 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c3c145b70 0x7f4c28004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:23.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.129+0000 7f4c40e92640 1 --2- 172.21.15.98:0/3573549258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c28004d50 0x7f4c280051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:23.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.129+0000 7f4c40e92640 1 -- 172.21.15.98:0/3573549258 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c071f20 con 0x7f4c28004d50 2024-09-19T08:43:23.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.129+0000 7f4c40e92640 1 -- 172.21.15.98:0/3573549258 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2800bb00 con 0x7f4c280039f0 2024-09-19T08:43:23.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.129+0000 7f4c40e92640 1 -- 172.21.15.98:0/3573549258 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4c2800bd00 con 0x7f4c3c145b70 2024-09-19T08:43:23.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.129+0000 7f4c42695640 1 --2- 172.21.15.98:0/3573549258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c28004d50 0x7f4c280051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:23.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.133+0000 7f4c41693640 1 --2- 172.21.15.98:0/3573549258 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c3c145b70 0x7f4c28004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:23.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.133+0000 7f4c41e94640 1 --2- 172.21.15.98:0/3573549258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c280039f0 0x7f4c28003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:23.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.133+0000 7f4c42695640 1 -- 172.21.15.98:0/3573549258 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c3c145b70 msgr2=0x7f4c28004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:23.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.133+0000 7f4c42695640 1 --2- 172.21.15.98:0/3573549258 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c3c145b70 0x7f4c28004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:23.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.133+0000 7f4c42695640 1 -- 172.21.15.98:0/3573549258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c280039f0 msgr2=0x7f4c28003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:23.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.133+0000 7f4c42695640 1 --2- 172.21.15.98:0/3573549258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c280039f0 0x7f4c28003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:23.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.133+0000 7f4c42695640 1 -- 172.21.15.98:0/3573549258 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c28003530 con 0x7f4c28004d50 2024-09-19T08:43:23.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.133+0000 7f4c42695640 1 --2- 172.21.15.98:0/3573549258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c28004d50 0x7f4c280051d0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f4c34001e70 tx=0x7f4c3401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=164d807f9b28bbe2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:23.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.133+0000 7f4c40e92640 1 -- 172.21.15.98:0/3573549258 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c34001a90 con 0x7f4c28004d50 2024-09-19T08:43:23.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.133+0000 7f4c40e92640 1 -- 172.21.15.98:0/3573549258 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c34003b80 con 0x7f4c28004d50 2024-09-19T08:43:23.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.133+0000 7f4c40e92640 1 -- 172.21.15.98:0/3573549258 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c34005080 con 0x7f4c28004d50 2024-09-19T08:43:23.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.133+0000 7f4c440f8640 1 -- 172.21.15.98:0/3573549258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c28004d50 msgr2=0x7f4c280051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:23.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.133+0000 7f4c440f8640 1 --2- 172.21.15.98:0/3573549258 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c28004d50 0x7f4c280051d0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f4c34001e70 tx=0x7f4c3401c780 comp rx=0 tx=0).stop 2024-09-19T08:43:23.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.133+0000 7f4c42695640 1 -- 172.21.15.98:0/3573549258 reap_dead start 2024-09-19T08:43:23.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.133+0000 7f4c440f8640 1 -- 172.21.15.98:0/3573549258 shutdown_connections 2024-09-19T08:43:23.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.133+0000 7f4c440f8640 1 -- 172.21.15.98:0/3573549258 >> 172.21.15.98:0/3573549258 conn(0x7f4c3c0b2010 msgr2=0x7f4c3c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:23.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.133+0000 7f4c440f8640 1 -- 172.21.15.98:0/3573549258 shutdown_connections 2024-09-19T08:43:23.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.133+0000 7f4c440f8640 1 -- 172.21.15.98:0/3573549258 wait complete. 2024-09-19T08:43:23.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.137+0000 7f4c440f8640 1 Processor -- start 2024-09-19T08:43:23.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.137+0000 7f4c440f8640 1 -- start start 2024-09-19T08:43:23.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.137+0000 7f4c440f8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c28004d50 0x7f4c3c132d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:23.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.137+0000 7f4c440f8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c3c15bb50 0x7f4c3c1434a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:23.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.137+0000 7f4c440f8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c3c15bfd0 0x7f4c3c1439e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:23.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.137+0000 7f4c440f8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c145f50 con 0x7f4c3c15bb50 2024-09-19T08:43:23.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.137+0000 7f4c440f8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c073dd0 con 0x7f4c28004d50 2024-09-19T08:43:23.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.137+0000 7f4c440f8640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4c3c1460d0 con 0x7f4c3c15bfd0 2024-09-19T08:43:23.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.137+0000 7f4c41693640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c3c15bb50 0x7f4c3c1434a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:23.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.137+0000 7f4c41693640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c3c15bb50 0x7f4c3c1434a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41460/0 (socket says 172.21.15.98:41460) 2024-09-19T08:43:23.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.137+0000 7f4c41693640 1 -- 172.21.15.98:0/866827598 learned_addr learned my addr 172.21.15.98:0/866827598 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:23.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.137+0000 7f4c41e94640 1 --2- 172.21.15.98:0/866827598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c28004d50 0x7f4c3c132d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:23.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.137+0000 7f4c42695640 1 --2- 172.21.15.98:0/866827598 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c3c15bfd0 0x7f4c3c1439e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:23.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.137+0000 7f4c41693640 1 -- 172.21.15.98:0/866827598 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c3c15bfd0 msgr2=0x7f4c3c1439e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:23.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.137+0000 7f4c41693640 1 --2- 172.21.15.98:0/866827598 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c3c15bfd0 0x7f4c3c1439e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:23.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.137+0000 7f4c41693640 1 -- 172.21.15.98:0/866827598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c28004d50 msgr2=0x7f4c3c132d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:23.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.137+0000 7f4c41693640 1 --2- 172.21.15.98:0/866827598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c28004d50 0x7f4c3c132d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:23.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.137+0000 7f4c41693640 1 -- 172.21.15.98:0/866827598 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c3c12f9b0 con 0x7f4c3c15bb50 2024-09-19T08:43:23.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.137+0000 7f4c41693640 1 --2- 172.21.15.98:0/866827598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c3c15bb50 0x7f4c3c1434a0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0118c0 tx=0x7f4c2c011cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:23.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.137+0000 7f4c327fc640 1 -- 172.21.15.98:0/866827598 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c010040 con 0x7f4c3c15bb50 2024-09-19T08:43:23.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.137+0000 7f4c327fc640 1 -- 172.21.15.98:0/866827598 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c2c01a070 con 0x7f4c3c15bb50 2024-09-19T08:43:23.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.137+0000 7f4c440f8640 1 -- 172.21.15.98:0/866827598 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c3c12fc90 con 0x7f4c3c15bb50 2024-09-19T08:43:23.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.137+0000 7f4c327fc640 1 -- 172.21.15.98:0/866827598 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c00fdc0 con 0x7f4c3c15bb50 2024-09-19T08:43:23.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.137+0000 7f4c440f8640 1 -- 172.21.15.98:0/866827598 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c3c130250 con 0x7f4c3c15bb50 2024-09-19T08:43:23.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.137+0000 7f4c440f8640 1 -- 172.21.15.98:0/866827598 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c3c06bc90 con 0x7f4c3c15bb50 2024-09-19T08:43:23.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.137+0000 7f4c327fc640 1 -- 172.21.15.98:0/866827598 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f4c2c01f460 con 0x7f4c3c15bb50 2024-09-19T08:43:23.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.137+0000 7f4c327fc640 1 --2- 172.21.15.98:0/866827598 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4c200679e0 0x7f4c20069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:23.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.141+0000 7f4c327fc640 1 -- 172.21.15.98:0/866827598 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f4c2c088f20 con 0x7f4c3c15bb50 2024-09-19T08:43:23.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.141+0000 7f4c41e94640 1 --2- 172.21.15.98:0/866827598 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4c200679e0 0x7f4c20069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:23.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.141+0000 7f4c41e94640 1 --2- 172.21.15.98:0/866827598 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4c200679e0 0x7f4c20069ea0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4c38009a30 tx=0x7f4c38008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:23.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.145+0000 7f4c327fc640 1 -- 172.21.15.98:0/866827598 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f4c2c0599e0 con 0x7f4c3c15bb50 2024-09-19T08:43:23.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.349+0000 7f4c440f8640 1 -- 172.21.15.98:0/866827598 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f4c3c06d430 con 0x7f4c3c15bb50 2024-09-19T08:43:23.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.353+0000 7f4c327fc640 1 -- 172.21.15.98:0/866827598 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) ==== 190+0+147 (secure 0 0 0) 0x7f4c2c0592e0 con 0x7f4c3c15bb50 2024-09-19T08:43:23.354 INFO:teuthology.orchestra.run.smithi098.stdout:[client.0] 2024-09-19T08:43:23.354 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQC/4utm0vNpIhAA4S1E8dFZ0+ibmSKhmZ+bIA== 2024-09-19T08:43:23.354 INFO:teuthology.orchestra.run.smithi098.stdout: caps mds = "allow" 2024-09-19T08:43:23.355 INFO:teuthology.orchestra.run.smithi098.stdout: caps mon = "allow r" 2024-09-19T08:43:23.355 INFO:teuthology.orchestra.run.smithi098.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-19T08:43:23.355 INFO:teuthology.orchestra.run.smithi098.stderr:updated caps for client.0 2024-09-19T08:43:23.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.357+0000 7f4c440f8640 1 -- 172.21.15.98:0/866827598 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4c200679e0 msgr2=0x7f4c20069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:23.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.357+0000 7f4c440f8640 1 --2- 172.21.15.98:0/866827598 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4c200679e0 0x7f4c20069ea0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4c38009a30 tx=0x7f4c38008040 comp rx=0 tx=0).stop 2024-09-19T08:43:23.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.357+0000 7f4c440f8640 1 -- 172.21.15.98:0/866827598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c3c15bb50 msgr2=0x7f4c3c1434a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:23.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.357+0000 7f4c440f8640 1 --2- 172.21.15.98:0/866827598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c3c15bb50 0x7f4c3c1434a0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0118c0 tx=0x7f4c2c011cc0 comp rx=0 tx=0).stop 2024-09-19T08:43:23.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.357+0000 7f4c440f8640 1 -- 172.21.15.98:0/866827598 shutdown_connections 2024-09-19T08:43:23.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.357+0000 7f4c440f8640 1 --2- 172.21.15.98:0/866827598 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4c200679e0 0x7f4c20069ea0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:23.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.357+0000 7f4c440f8640 1 --2- 172.21.15.98:0/866827598 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4c3c15bfd0 0x7f4c3c1439e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:23.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.357+0000 7f4c440f8640 1 --2- 172.21.15.98:0/866827598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4c3c15bb50 0x7f4c3c1434a0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:23.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.357+0000 7f4c440f8640 1 --2- 172.21.15.98:0/866827598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c28004d50 0x7f4c3c132d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:23.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.357+0000 7f4c440f8640 1 -- 172.21.15.98:0/866827598 >> 172.21.15.98:0/866827598 conn(0x7f4c3c0b2010 msgr2=0x7f4c3c0b0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:23.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.357+0000 7f4c440f8640 1 -- 172.21.15.98:0/866827598 shutdown_connections 2024-09-19T08:43:23.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.357+0000 7f4c440f8640 1 -- 172.21.15.98:0/866827598 wait complete. 2024-09-19T08:43:23.372 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:43:23.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.489+0000 7fc72aa21640 1 Processor -- start 2024-09-19T08:43:23.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.489+0000 7fc72aa21640 1 -- start start 2024-09-19T08:43:23.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.489+0000 7fc72aa21640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7240733b0 0x7fc7240cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:23.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.489+0000 7fc72aa21640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc72406d310 con 0x7fc72415c550 2024-09-19T08:43:23.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.489+0000 7fc72aa21640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc72406d5f0 con 0x7fc72406c3a0 2024-09-19T08:43:23.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.489+0000 7fc72aa21640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc72406d8d0 con 0x7fc7240733b0 2024-09-19T08:43:23.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.489+0000 7fc723fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fc72415c550 0x7fc72406bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40084/0 (socket says 172.21.15.98:40084) 2024-09-19T08:43:23.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.489+0000 7fc723fff640 1 -- 172.21.15.98:0/3911287859 learned_addr learned my addr 172.21.15.98:0/3911287859 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:23.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.489+0000 7fc722ffd640 1 -- 172.21.15.98:0/3911287859 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3816458427 0 0) 0x7fc72406d310 con 0x7fc72415c550 2024-09-19T08:43:23.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.489+0000 7fc722ffd640 1 -- 172.21.15.98:0/3911287859 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc70c003530 con 0x7fc72415c550 2024-09-19T08:43:23.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.489+0000 7fc728fbe640 1 --2- 172.21.15.98:0/3911287859 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7240733b0 0x7fc7240cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:23.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.489+0000 7fc722ffd640 1 -- 172.21.15.98:0/3911287859 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1213437529 0 0) 0x7fc70c003530 con 0x7fc72415c550 2024-09-19T08:43:23.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.489+0000 7fc722ffd640 1 -- 172.21.15.98:0/3911287859 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7240733b0 msgr2=0x7fc7240cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:23.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.489+0000 7fc722ffd640 1 --2- 172.21.15.98:0/3911287859 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7240733b0 0x7fc7240cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:23.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.489+0000 7fc722ffd640 1 -- 172.21.15.98:0/3911287859 >> v1:172.21.15.139:6789/0 conn(0x7fc72406c3a0 legacy=0x7fc7240cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:23.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.489+0000 7fc722ffd640 1 -- 172.21.15.98:0/3911287859 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc72406dbb0 con 0x7fc72415c550 2024-09-19T08:43:23.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.489+0000 7fc722ffd640 1 -- 172.21.15.98:0/3911287859 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fc714001680 con 0x7fc72415c550 2024-09-19T08:43:23.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.489+0000 7fc722ffd640 1 -- 172.21.15.98:0/3911287859 >> v1:172.21.15.98:6789/0 conn(0x7fc72415c550 legacy=0x7fc72406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:23.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.489+0000 7fc722ffd640 1 --2- 172.21.15.98:0/3911287859 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7240733b0 0x7fc70c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:23.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.489+0000 7fc722ffd640 1 --2- 172.21.15.98:0/3911287859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc70c003f70 0x7fc70c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:23.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.489+0000 7fc722ffd640 1 --2- 172.21.15.98:0/3911287859 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc70c004d50 0x7fc70c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:23.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.489+0000 7fc722ffd640 1 -- 172.21.15.98:0/3911287859 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc72406d8d0 con 0x7fc70c004d50 2024-09-19T08:43:23.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.489+0000 7fc722ffd640 1 -- 172.21.15.98:0/3911287859 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c00bab0 con 0x7fc70c003f70 2024-09-19T08:43:23.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.489+0000 7fc722ffd640 1 -- 172.21.15.98:0/3911287859 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc70c00bcb0 con 0x7fc7240733b0 2024-09-19T08:43:23.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.489+0000 7fc728fbe640 1 --2- 172.21.15.98:0/3911287859 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc70c004d50 0x7fc70c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:23.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.489+0000 7fc723fff640 1 --2- 172.21.15.98:0/3911287859 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7240733b0 0x7fc70c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:23.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.489+0000 7fc7237fe640 1 --2- 172.21.15.98:0/3911287859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc70c003f70 0x7fc70c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:23.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.489+0000 7fc728fbe640 1 -- 172.21.15.98:0/3911287859 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7240733b0 msgr2=0x7fc70c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:23.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.489+0000 7fc728fbe640 1 --2- 172.21.15.98:0/3911287859 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7240733b0 0x7fc70c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:23.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.489+0000 7fc728fbe640 1 -- 172.21.15.98:0/3911287859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc70c003f70 msgr2=0x7fc70c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:23.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.489+0000 7fc728fbe640 1 --2- 172.21.15.98:0/3911287859 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc70c003f70 0x7fc70c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:23.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.489+0000 7fc728fbe640 1 -- 172.21.15.98:0/3911287859 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70c003530 con 0x7fc70c004d50 2024-09-19T08:43:23.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.489+0000 7fc723fff640 1 --2- 172.21.15.98:0/3911287859 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc7240733b0 0x7fc70c0039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:43:23.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.493+0000 7fc728fbe640 1 --2- 172.21.15.98:0/3911287859 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc70c004d50 0x7fc70c0051d0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fc71c009fc0 tx=0x7fc71c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05463e50786c770 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:23.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.493+0000 7fc722ffd640 1 -- 172.21.15.98:0/3911287859 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c0b2070 con 0x7fc70c004d50 2024-09-19T08:43:23.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.493+0000 7fc722ffd640 1 -- 172.21.15.98:0/3911287859 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc71c0a5c90 con 0x7fc70c004d50 2024-09-19T08:43:23.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.493+0000 7fc722ffd640 1 -- 172.21.15.98:0/3911287859 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c009c10 con 0x7fc70c004d50 2024-09-19T08:43:23.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.493+0000 7fc72aa21640 1 -- 172.21.15.98:0/3911287859 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc70c004d50 msgr2=0x7fc70c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:23.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.493+0000 7fc72aa21640 1 --2- 172.21.15.98:0/3911287859 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc70c004d50 0x7fc70c0051d0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fc71c009fc0 tx=0x7fc71c01cc80 comp rx=0 tx=0).stop 2024-09-19T08:43:23.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.493+0000 7fc728fbe640 1 -- 172.21.15.98:0/3911287859 reap_dead start 2024-09-19T08:43:23.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.493+0000 7fc72aa21640 1 -- 172.21.15.98:0/3911287859 shutdown_connections 2024-09-19T08:43:23.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.493+0000 7fc72aa21640 1 -- 172.21.15.98:0/3911287859 >> 172.21.15.98:0/3911287859 conn(0x7fc7240b2010 msgr2=0x7fc7240b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:23.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.493+0000 7fc72aa21640 1 -- 172.21.15.98:0/3911287859 shutdown_connections 2024-09-19T08:43:23.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.493+0000 7fc72aa21640 1 -- 172.21.15.98:0/3911287859 wait complete. 2024-09-19T08:43:23.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.493+0000 7fc72aa21640 1 Processor -- start 2024-09-19T08:43:23.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.493+0000 7fc72aa21640 1 -- start start 2024-09-19T08:43:23.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.497+0000 7fc72aa21640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc70c004d50 0x7fc724152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:23.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.497+0000 7fc72aa21640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc72414f260 0x7fc724152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:23.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.497+0000 7fc72aa21640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc724155470 0x7fc7241558f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:23.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.497+0000 7fc72aa21640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc7240cfc50 con 0x7fc72414f260 2024-09-19T08:43:23.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.497+0000 7fc72aa21640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc7240cfad0 con 0x7fc70c004d50 2024-09-19T08:43:23.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.497+0000 7fc72aa21640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc7240cfdd0 con 0x7fc724155470 2024-09-19T08:43:23.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.497+0000 7fc7237fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc72414f260 0x7fc724152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:23.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.497+0000 7fc7237fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc72414f260 0x7fc724152880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41476/0 (socket says 172.21.15.98:41476) 2024-09-19T08:43:23.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.497+0000 7fc7237fe640 1 -- 172.21.15.98:0/1209327208 learned_addr learned my addr 172.21.15.98:0/1209327208 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:23.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.497+0000 7fc723fff640 1 --2- 172.21.15.98:0/1209327208 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc70c004d50 0x7fc724152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:23.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.497+0000 7fc728fbe640 1 --2- 172.21.15.98:0/1209327208 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc724155470 0x7fc7241558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:23.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.497+0000 7fc7237fe640 1 -- 172.21.15.98:0/1209327208 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc724155470 msgr2=0x7fc7241558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:23.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.497+0000 7fc7237fe640 1 --2- 172.21.15.98:0/1209327208 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc724155470 0x7fc7241558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:23.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.497+0000 7fc7237fe640 1 -- 172.21.15.98:0/1209327208 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc70c004d50 msgr2=0x7fc724152340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:23.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.497+0000 7fc7237fe640 1 --2- 172.21.15.98:0/1209327208 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc70c004d50 0x7fc724152340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:23.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.497+0000 7fc7237fe640 1 -- 172.21.15.98:0/1209327208 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc72415cc20 con 0x7fc72414f260 2024-09-19T08:43:23.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.497+0000 7fc728fbe640 1 --2- 172.21.15.98:0/1209327208 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc724155470 0x7fc7241558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:23.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.497+0000 7fc7237fe640 1 --2- 172.21.15.98:0/1209327208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc72414f260 0x7fc724152880 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fc71800e510 tx=0x7fc71800e9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:23.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.497+0000 7fc720ff9640 1 -- 172.21.15.98:0/1209327208 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7180052e0 con 0x7fc72414f260 2024-09-19T08:43:23.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.497+0000 7fc720ff9640 1 -- 172.21.15.98:0/1209327208 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc718018070 con 0x7fc72414f260 2024-09-19T08:43:23.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.497+0000 7fc72aa21640 1 -- 172.21.15.98:0/1209327208 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc72415cf00 con 0x7fc72414f260 2024-09-19T08:43:23.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.497+0000 7fc720ff9640 1 -- 172.21.15.98:0/1209327208 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc718003820 con 0x7fc72414f260 2024-09-19T08:43:23.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.497+0000 7fc72aa21640 1 -- 172.21.15.98:0/1209327208 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc72415d4c0 con 0x7fc72414f260 2024-09-19T08:43:23.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.497+0000 7fc72aa21640 1 -- 172.21.15.98:0/1209327208 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7240cf3c0 con 0x7fc72414f260 2024-09-19T08:43:23.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.497+0000 7fc720ff9640 1 -- 172.21.15.98:0/1209327208 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fc7180039c0 con 0x7fc72414f260 2024-09-19T08:43:23.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.497+0000 7fc720ff9640 1 --2- 172.21.15.98:0/1209327208 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc710067a10 0x7fc710069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:23.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.497+0000 7fc720ff9640 1 -- 172.21.15.98:0/1209327208 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fc718085670 con 0x7fc72414f260 2024-09-19T08:43:23.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.497+0000 7fc723fff640 1 --2- 172.21.15.98:0/1209327208 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc710067a10 0x7fc710069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:23.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.497+0000 7fc723fff640 1 --2- 172.21.15.98:0/1209327208 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc710067a10 0x7fc710069ed0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc714001ec0 tx=0x7fc714009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:23.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.501+0000 7fc720ff9640 1 -- 172.21.15.98:0/1209327208 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc718005480 con 0x7fc72414f260 2024-09-19T08:43:23.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.701+0000 7fc72aa21640 1 -- 172.21.15.98:0/1209327208 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc72415a670 con 0x7fc72414f260 2024-09-19T08:43:23.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.701+0000 7fc720ff9640 1 -- 172.21.15.98:0/1209327208 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5228 (secure 0 0 0) 0x7fc718055a50 con 0x7fc72414f260 2024-09-19T08:43:23.703 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:23.703 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":47,"btime":"2024-09-19T08:43:21:568061+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3182634136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3182634136},{"type":"v1","addr":"172.21.15.139:6833","nonce":3182634136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5830,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1554653172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1554653172},{"type":"v1","addr":"172.21.15.139:6835","nonce":1554653172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6624,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1092838020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1092838020},{"type":"v1","addr":"172.21.15.98:6833","nonce":1092838020}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:43:17.920717+0000","modified":"2024-09-19T08:43:20.933845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6627},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6627":{"gid":6627,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/837817273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":837817273},{"type":"v1","addr":"172.21.15.98:6835","nonce":837817273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6627,"qdb_cluster":[6627]},"id":6}]} 2024-09-19T08:43:23.703 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 47 2024-09-19T08:43:23.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.705+0000 7fc72aa21640 1 -- 172.21.15.98:0/1209327208 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc710067a10 msgr2=0x7fc710069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:23.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.705+0000 7fc72aa21640 1 --2- 172.21.15.98:0/1209327208 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc710067a10 0x7fc710069ed0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc714001ec0 tx=0x7fc714009040 comp rx=0 tx=0).stop 2024-09-19T08:43:23.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.705+0000 7fc72aa21640 1 -- 172.21.15.98:0/1209327208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc72414f260 msgr2=0x7fc724152880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:23.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.705+0000 7fc72aa21640 1 --2- 172.21.15.98:0/1209327208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc72414f260 0x7fc724152880 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fc71800e510 tx=0x7fc71800e9d0 comp rx=0 tx=0).stop 2024-09-19T08:43:23.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.705+0000 7fc72aa21640 1 -- 172.21.15.98:0/1209327208 shutdown_connections 2024-09-19T08:43:23.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.705+0000 7fc72aa21640 1 --2- 172.21.15.98:0/1209327208 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc710067a10 0x7fc710069ed0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:23.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.705+0000 7fc72aa21640 1 --2- 172.21.15.98:0/1209327208 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc724155470 0x7fc7241558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:23.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.705+0000 7fc72aa21640 1 --2- 172.21.15.98:0/1209327208 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc72414f260 0x7fc724152880 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:23.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.705+0000 7fc72aa21640 1 --2- 172.21.15.98:0/1209327208 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc70c004d50 0x7fc724152340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:23.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.705+0000 7fc72aa21640 1 -- 172.21.15.98:0/1209327208 >> 172.21.15.98:0/1209327208 conn(0x7fc7240b2010 msgr2=0x7fc7240ce1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:23.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.705+0000 7fc72aa21640 1 -- 172.21.15.98:0/1209327208 shutdown_connections 2024-09-19T08:43:23.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:23.705+0000 7fc72aa21640 1 -- 172.21.15.98:0/1209327208 wait complete. 2024-09-19T08:43:24.129 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T08:43:17.920717+0000', 'modified': '2024-09-19T08:43:20.933845+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6627}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6627': {'gid': 6627, 'name': 'a', 'rank': 0, 'incarnation': 43, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.98:6835/837817273', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6834', 'nonce': 837817273}, {'type': 'v1', 'addr': '172.21.15.98:6835', 'nonce': 837817273}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6627, 'qdb_cluster': [6627]} 2024-09-19T08:43:24.129 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T08:43:24.129 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-19T08:43:24.169 INFO:tasks.ceph.mds.a.smithi098.stderr:2024-09-19T08:43:24.169+0000 7f0e63cb8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T08:43:24.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.261+0000 7fbf8bbba640 1 Processor -- start 2024-09-19T08:43:24.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.261+0000 7fbf8bbba640 1 -- start start 2024-09-19T08:43:24.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.261+0000 7fbf8bbba640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbf840b6910 0x7fbf840d1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:24.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.261+0000 7fbf8bbba640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf840b8060 con 0x7fbf840d1d70 2024-09-19T08:43:24.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.261+0000 7fbf8bbba640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf840b8340 con 0x7fbf84147530 2024-09-19T08:43:24.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.261+0000 7fbf8bbba640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fbf840d2230 con 0x7fbf840b6910 2024-09-19T08:43:24.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.261+0000 7fbf89155640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fbf840d1d70 0x7fbf840b6200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40094/0 (socket says 172.21.15.98:40094) 2024-09-19T08:43:24.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.261+0000 7fbf89155640 1 -- 172.21.15.98:0/3710274061 learned_addr learned my addr 172.21.15.98:0/3710274061 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:24.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.265+0000 7fbf8a157640 1 --2- 172.21.15.98:0/3710274061 >> v2:172.21.15.139:3300/0 conn(0x7fbf840b6910 0x7fbf840d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:24.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.265+0000 7fbf88954640 1 -- 172.21.15.98:0/3710274061 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2541325845 0 0) 0x7fbf840b8060 con 0x7fbf840d1d70 2024-09-19T08:43:24.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.265+0000 7fbf88954640 1 -- 172.21.15.98:0/3710274061 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf70003530 con 0x7fbf840d1d70 2024-09-19T08:43:24.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.265+0000 7fbf88954640 1 -- 172.21.15.98:0/3710274061 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2473882115 0 0) 0x7fbf70003530 con 0x7fbf840d1d70 2024-09-19T08:43:24.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.265+0000 7fbf88954640 1 -- 172.21.15.98:0/3710274061 >> v2:172.21.15.139:3300/0 conn(0x7fbf840b6910 msgr2=0x7fbf840d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:24.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.265+0000 7fbf88954640 1 --2- 172.21.15.98:0/3710274061 >> v2:172.21.15.139:3300/0 conn(0x7fbf840b6910 0x7fbf840d1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.265+0000 7fbf88954640 1 -- 172.21.15.98:0/3710274061 >> v1:172.21.15.139:6789/0 conn(0x7fbf84147530 legacy=0x7fbf8415ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:24.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.265+0000 7fbf88954640 1 -- 172.21.15.98:0/3710274061 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf840b8620 con 0x7fbf840d1d70 2024-09-19T08:43:24.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.265+0000 7fbf88954640 1 -- 172.21.15.98:0/3710274061 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fbf74001680 con 0x7fbf840d1d70 2024-09-19T08:43:24.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.265+0000 7fbf88954640 1 -- 172.21.15.98:0/3710274061 >> v1:172.21.15.98:6789/0 conn(0x7fbf840d1d70 legacy=0x7fbf840b6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:24.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.265+0000 7fbf88954640 1 --2- 172.21.15.98:0/3710274061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf700039c0 0x7fbf70003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:24.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.265+0000 7fbf88954640 1 --2- 172.21.15.98:0/3710274061 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbf70004810 0x7fbf70004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:24.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.265+0000 7fbf88954640 1 --2- 172.21.15.98:0/3710274061 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf700051d0 0x7fbf7000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:24.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.265+0000 7fbf88954640 1 -- 172.21.15.98:0/3710274061 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbf840d2230 con 0x7fbf700039c0 2024-09-19T08:43:24.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.265+0000 7fbf88954640 1 -- 172.21.15.98:0/3710274061 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7000c0a0 con 0x7fbf700051d0 2024-09-19T08:43:24.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.265+0000 7fbf88954640 1 -- 172.21.15.98:0/3710274061 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbf7000c2a0 con 0x7fbf70004810 2024-09-19T08:43:24.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.265+0000 7fbf89956640 1 --2- 172.21.15.98:0/3710274061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf700039c0 0x7fbf70003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:24.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.265+0000 7fbf8a157640 1 --2- 172.21.15.98:0/3710274061 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf700051d0 0x7fbf7000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:24.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.265+0000 7fbf89155640 1 --2- 172.21.15.98:0/3710274061 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbf70004810 0x7fbf70004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:24.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.265+0000 7fbf89956640 1 -- 172.21.15.98:0/3710274061 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbf70004810 msgr2=0x7fbf70004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:24.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.265+0000 7fbf89956640 1 --2- 172.21.15.98:0/3710274061 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbf70004810 0x7fbf70004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.265+0000 7fbf89956640 1 -- 172.21.15.98:0/3710274061 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf700051d0 msgr2=0x7fbf7000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:24.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.265+0000 7fbf89956640 1 --2- 172.21.15.98:0/3710274061 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf700051d0 0x7fbf7000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.265+0000 7fbf89956640 1 -- 172.21.15.98:0/3710274061 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf70003530 con 0x7fbf700039c0 2024-09-19T08:43:24.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.265+0000 7fbf8a157640 1 --2- 172.21.15.98:0/3710274061 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf700051d0 0x7fbf7000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:24.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.265+0000 7fbf8a157640 1 -- 172.21.15.98:0/3710274061 reap_dead start 2024-09-19T08:43:24.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.265+0000 7fbf89956640 1 --2- 172.21.15.98:0/3710274061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf700039c0 0x7fbf70003e00 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fbf80001780 tx=0x7fbf8001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2540df7f771f82de server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:24.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.265+0000 7fbf88954640 1 -- 172.21.15.98:0/3710274061 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf800013a0 con 0x7fbf700039c0 2024-09-19T08:43:24.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.265+0000 7fbf88954640 1 -- 172.21.15.98:0/3710274061 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf800052f0 con 0x7fbf700039c0 2024-09-19T08:43:24.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.265+0000 7fbf88954640 1 -- 172.21.15.98:0/3710274061 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf80003840 con 0x7fbf700039c0 2024-09-19T08:43:24.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.269+0000 7fbf8bbba640 1 -- 172.21.15.98:0/3710274061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf700039c0 msgr2=0x7fbf70003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:24.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.269+0000 7fbf8bbba640 1 --2- 172.21.15.98:0/3710274061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf700039c0 0x7fbf70003e00 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fbf80001780 tx=0x7fbf8001c3d0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.269+0000 7fbf8bbba640 1 -- 172.21.15.98:0/3710274061 shutdown_connections 2024-09-19T08:43:24.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.269+0000 7fbf8bbba640 1 --2- 172.21.15.98:0/3710274061 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf700039c0 0x7fbf70003e00 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.269+0000 7fbf8bbba640 1 -- 172.21.15.98:0/3710274061 >> 172.21.15.98:0/3710274061 conn(0x7fbf84075070 msgr2=0x7fbf840722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:24.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.269+0000 7fbf8bbba640 1 -- 172.21.15.98:0/3710274061 shutdown_connections 2024-09-19T08:43:24.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.269+0000 7fbf8bbba640 1 -- 172.21.15.98:0/3710274061 wait complete. 2024-09-19T08:43:24.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.269+0000 7fbf8bbba640 1 Processor -- start 2024-09-19T08:43:24.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.269+0000 7fbf8bbba640 1 -- start start 2024-09-19T08:43:24.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.269+0000 7fbf8bbba640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf700039c0 0x7fbf841ea940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:24.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.269+0000 7fbf8bbba640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbf840cbfb0 0x7fbf840cab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:24.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.269+0000 7fbf8bbba640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf840cb090 0x7fbf840cb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:24.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.269+0000 7fbf8bbba640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbf840b6610 con 0x7fbf840cb090 2024-09-19T08:43:24.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.269+0000 7fbf8bbba640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbf840d1920 con 0x7fbf700039c0 2024-09-19T08:43:24.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.269+0000 7fbf8bbba640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbf840b6790 con 0x7fbf840cbfb0 2024-09-19T08:43:24.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.269+0000 7fbf8a157640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf840cb090 0x7fbf840cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:24.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.269+0000 7fbf8a157640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf840cb090 0x7fbf840cb510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41484/0 (socket says 172.21.15.98:41484) 2024-09-19T08:43:24.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.269+0000 7fbf8a157640 1 -- 172.21.15.98:0/3683589140 learned_addr learned my addr 172.21.15.98:0/3683589140 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:24.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.269+0000 7fbf89956640 1 --2- 172.21.15.98:0/3683589140 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf700039c0 0x7fbf841ea940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:24.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.269+0000 7fbf89155640 1 --2- 172.21.15.98:0/3683589140 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbf840cbfb0 0x7fbf840cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:24.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.269+0000 7fbf8a157640 1 -- 172.21.15.98:0/3683589140 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbf840cbfb0 msgr2=0x7fbf840cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:24.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.269+0000 7fbf8a157640 1 --2- 172.21.15.98:0/3683589140 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbf840cbfb0 0x7fbf840cab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.269+0000 7fbf8a157640 1 -- 172.21.15.98:0/3683589140 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf700039c0 msgr2=0x7fbf841ea940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:24.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.269+0000 7fbf8a157640 1 --2- 172.21.15.98:0/3683589140 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf700039c0 0x7fbf841ea940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.269+0000 7fbf8a157640 1 -- 172.21.15.98:0/3683589140 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf841eb8f0 con 0x7fbf840cb090 2024-09-19T08:43:24.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.269+0000 7fbf89956640 1 --2- 172.21.15.98:0/3683589140 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf700039c0 0x7fbf841ea940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:43:24.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.269+0000 7fbf8a157640 1 --2- 172.21.15.98:0/3683589140 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf840cb090 0x7fbf840cb510 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c009f50 tx=0x7fbf7c00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:24.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.269+0000 7fbf7a7fc640 1 -- 172.21.15.98:0/3683589140 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c016420 con 0x7fbf840cb090 2024-09-19T08:43:24.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.269+0000 7fbf7a7fc640 1 -- 172.21.15.98:0/3683589140 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf7c01b070 con 0x7fbf840cb090 2024-09-19T08:43:24.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.269+0000 7fbf8bbba640 1 -- 172.21.15.98:0/3683589140 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf840cdbb0 con 0x7fbf840cb090 2024-09-19T08:43:24.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.269+0000 7fbf7a7fc640 1 -- 172.21.15.98:0/3683589140 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c0167e0 con 0x7fbf840cb090 2024-09-19T08:43:24.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.269+0000 7fbf8bbba640 1 -- 172.21.15.98:0/3683589140 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf841eb330 con 0x7fbf840cb090 2024-09-19T08:43:24.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.273+0000 7fbf7a7fc640 1 -- 172.21.15.98:0/3683589140 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fbf7c016980 con 0x7fbf840cb090 2024-09-19T08:43:24.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.273+0000 7fbf7a7fc640 1 --2- 172.21.15.98:0/3683589140 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbf680679e0 0x7fbf68069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:24.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.273+0000 7fbf7a7fc640 1 -- 172.21.15.98:0/3683589140 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fbf7c086c60 con 0x7fbf840cb090 2024-09-19T08:43:24.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.273+0000 7fbf89956640 1 --2- 172.21.15.98:0/3683589140 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbf680679e0 0x7fbf68069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:24.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.273+0000 7fbf8ec76000 1 Processor -- start 2024-09-19T08:43:24.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.273+0000 7fbf8ec76000 1 -- start start 2024-09-19T08:43:24.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.273+0000 7fbf89956640 1 --2- 172.21.15.98:0/3683589140 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbf680679e0 0x7fbf68069ea0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbf800ac000 tx=0x7fbf800011a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:24.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.273+0000 7fbf8ec76000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562356655b20 0x562356655f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:24.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.273+0000 7fbf8ec76000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562356656440 0x56235660bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:24.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.273+0000 7fbf8ec76000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56235660c290 0x56235660c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:24.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.273+0000 7fbf8ec76000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5623564f3b60 con 0x562356655b20 2024-09-19T08:43:24.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.273+0000 7fbf8ec76000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56235645ac70 con 0x56235660c290 2024-09-19T08:43:24.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.273+0000 7fbf8ec76000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x562356400ed0 con 0x562356656440 2024-09-19T08:43:24.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.273+0000 7fbf89956640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562356655b20 0x562356655f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:24.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.273+0000 7fbf89956640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562356655b20 0x562356655f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41492/0 (socket says 172.21.15.98:41492) 2024-09-19T08:43:24.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.273+0000 7fbf89956640 1 -- 172.21.15.98:0/199923974 learned_addr learned my addr 172.21.15.98:0/199923974 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:24.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.273+0000 7fbf8a157640 1 --2- 172.21.15.98:0/199923974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56235660c290 0x56235660c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:24.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.273+0000 7fbf89956640 1 -- 172.21.15.98:0/199923974 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562356656440 msgr2=0x56235660bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:24.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.273+0000 7fbf89155640 1 --2- 172.21.15.98:0/199923974 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562356656440 0x56235660bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:24.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.273+0000 7fbf89956640 1 --2- 172.21.15.98:0/199923974 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562356656440 0x56235660bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.273+0000 7fbf89956640 1 -- 172.21.15.98:0/199923974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56235660c290 msgr2=0x56235660c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:24.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.273+0000 7fbf89956640 1 --2- 172.21.15.98:0/199923974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56235660c290 0x56235660c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.273+0000 7fbf89956640 1 -- 172.21.15.98:0/199923974 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562356412e50 con 0x562356655b20 2024-09-19T08:43:24.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.273+0000 7fbf8a157640 1 --2- 172.21.15.98:0/199923974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56235660c290 0x56235660c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:24.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.273+0000 7fbf89956640 1 --2- 172.21.15.98:0/199923974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562356655b20 0x562356655f00 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fbf8001c410 tx=0x7fbf80001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2d38fde93d2409 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:24.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf677fe640 1 -- 172.21.15.98:0/199923974 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf800bb040 con 0x562356655b20 2024-09-19T08:43:24.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf677fe640 1 -- 172.21.15.98:0/199923974 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf80007650 con 0x562356655b20 2024-09-19T08:43:24.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf677fe640 1 -- 172.21.15.98:0/199923974 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf80006a00 con 0x562356655b20 2024-09-19T08:43:24.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf8ec76000 1 -- 172.21.15.98:0/199923974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562356655b20 msgr2=0x562356655f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:24.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf8ec76000 1 --2- 172.21.15.98:0/199923974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562356655b20 0x562356655f00 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fbf8001c410 tx=0x7fbf80001130 comp rx=0 tx=0).stop 2024-09-19T08:43:24.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf8ec76000 1 -- 172.21.15.98:0/199923974 shutdown_connections 2024-09-19T08:43:24.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf8ec76000 1 --2- 172.21.15.98:0/199923974 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56235660c290 0x56235660c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf8ec76000 1 --2- 172.21.15.98:0/199923974 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562356656440 0x56235660bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf8ec76000 1 --2- 172.21.15.98:0/199923974 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562356655b20 0x562356655f00 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf8ec76000 1 -- 172.21.15.98:0/199923974 >> 172.21.15.98:0/199923974 conn(0x562356612600 msgr2=0x56235660a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:24.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf8ec76000 1 -- 172.21.15.98:0/199923974 shutdown_connections 2024-09-19T08:43:24.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf8ec76000 1 -- 172.21.15.98:0/199923974 wait complete. 2024-09-19T08:43:24.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf8ec76000 1 -- start start 2024-09-19T08:43:24.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf8ec76000 1 Processor -- start 2024-09-19T08:43:24.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf8ec76000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562356655b20 0x562356610df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:24.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf8ec76000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562356656440 0x562356616cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:24.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf8ec76000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562356611330 0x562356617230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:24.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf8ec76000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x5623564bcb00 con 0x562356656440 2024-09-19T08:43:24.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf8ec76000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562356435360 con 0x562356655b20 2024-09-19T08:43:24.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf8ec76000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x5623564322d0 con 0x562356611330 2024-09-19T08:43:24.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf8ec76000 10 client.0 _ll_register_callbacks cb 0x5623565f7820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:43:24.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf89155640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562356656440 0x562356616cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:24.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf89155640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562356656440 0x562356616cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41506/0 (socket says 172.21.15.98:41506) 2024-09-19T08:43:24.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf89155640 1 -- 172.21.15.98:0/547929408 learned_addr learned my addr 172.21.15.98:0/547929408 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:24.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf89956640 1 --2- 172.21.15.98:0/547929408 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562356655b20 0x562356610df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:24.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf89155640 1 -- 172.21.15.98:0/547929408 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562356611330 msgr2=0x562356617230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:24.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf89155640 1 --2- 172.21.15.98:0/547929408 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562356611330 0x562356617230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf89155640 1 -- 172.21.15.98:0/547929408 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562356655b20 msgr2=0x562356610df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:24.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf89155640 1 --2- 172.21.15.98:0/547929408 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562356655b20 0x562356610df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf89155640 1 -- 172.21.15.98:0/547929408 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562356401120 con 0x562356656440 2024-09-19T08:43:24.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf89155640 1 --2- 172.21.15.98:0/547929408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562356656440 0x562356616cf0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fbf74011270 tx=0x7fbf74011730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95087bb16bc1fa39 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:24.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-19T08:43:24.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf64ff9640 1 -- 172.21.15.98:0/547929408 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf74010040 con 0x562356656440 2024-09-19T08:43:24.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf64ff9640 1 -- 172.21.15.98:0/547929408 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fbf74014780 con 0x562356656440 2024-09-19T08:43:24.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf8ec76000 1 -- 172.21.15.98:0/547929408 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf80007650 con 0x562356656440 2024-09-19T08:43:24.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf64ff9640 1 -- 172.21.15.98:0/547929408 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fbf74022050 con 0x562356656440 2024-09-19T08:43:24.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf8ec76000 10 client.6753 fetch_fsmap learned FSMap version 47 2024-09-19T08:43:24.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf8ec76000 1 -- 172.21.15.98:0/547929408 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) -- 0x5623563fa450 con 0x562356656440 2024-09-19T08:43:24.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf64ff9640 1 -- 172.21.15.98:0/547929408 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf80007650 con 0x562356656440 2024-09-19T08:43:24.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf64ff9640 1 -- 172.21.15.98:0/547929408 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 47) ==== 3502+0+0 (secure 0 0 0) 0x7fbf74025020 con 0x562356656440 2024-09-19T08:43:24.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf8ec76000 10 client.6753 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T08:43:24.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf8ec76000 10 client.6753 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T08:43:24.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf8ec76000 20 client.6753 populate_metadata read hostname 'smithi098' 2024-09-19T08:43:24.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf8ec76000 1 --2- 172.21.15.98:0/547929408 >> [v2:172.21.15.98:6834/837817273,v1:172.21.15.98:6835/837817273] conn(0x562356664280 0x562356666670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:24.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf8ec76000 4 client.6753 mds_command: new command op to 6627 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:43:24.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf8ec76000 1 -- 172.21.15.98:0/547929408 --> [v2:172.21.15.98:6834/837817273,v1:172.21.15.98:6835/837817273] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623563f4c00 con 0x562356664280 2024-09-19T08:43:24.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf89956640 1 --2- 172.21.15.98:0/547929408 >> [v2:172.21.15.98:6834/837817273,v1:172.21.15.98:6835/837817273] conn(0x562356664280 0x562356666670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:24.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf89956640 1 --2- 172.21.15.98:0/547929408 >> [v2:172.21.15.98:6834/837817273,v1:172.21.15.98:6835/837817273] conn(0x562356664280 0x562356666670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d44af32eb81ec2d6 server_cookie=652dd34eb261f038 in_seq=0 out_seq=0 2024-09-19T08:43:24.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.277+0000 7fbf64ff9640 10 client.6753 ms_handle_connect on v2:172.21.15.98:6834/837817273 2024-09-19T08:43:24.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.281+0000 7fbf64ff9640 1 -- 172.21.15.98:0/547929408 <== mds.0 v2:172.21.15.98:6834/837817273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5623563f4c00 con 0x562356664280 2024-09-19T08:43:24.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.281+0000 7fbf64ff9640 10 client.6753 handle_command_reply: tid=0 2024-09-19T08:43:24.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.281+0000 7fbf8ec76000 1 client.6753 shutdown 2024-09-19T08:43:24.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.281+0000 7fbf8ec76000 1 -- 172.21.15.98:0/547929408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562356656440 msgr2=0x562356616cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:24.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.281+0000 7fbf8ec76000 1 --2- 172.21.15.98:0/547929408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562356656440 0x562356616cf0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fbf74011270 tx=0x7fbf74011730 comp rx=0 tx=0).stop 2024-09-19T08:43:24.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.281+0000 7fbf8ec76000 1 -- 172.21.15.98:0/547929408 shutdown_connections 2024-09-19T08:43:24.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.281+0000 7fbf8ec76000 1 --2- 172.21.15.98:0/547929408 >> [v2:172.21.15.98:6834/837817273,v1:172.21.15.98:6835/837817273] conn(0x562356664280 0x562356666670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.281+0000 7fbf8ec76000 1 --2- 172.21.15.98:0/547929408 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562356611330 0x562356617230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.281+0000 7fbf8ec76000 1 --2- 172.21.15.98:0/547929408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562356656440 0x562356616cf0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.281+0000 7fbf64ff9640 0 client.6753 ms_handle_reset on v2:172.21.15.98:6834/837817273 2024-09-19T08:43:24.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.281+0000 7fbf8ec76000 1 --2- 172.21.15.98:0/547929408 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562356655b20 0x562356610df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.281+0000 7fbf8ec76000 1 -- 172.21.15.98:0/547929408 >> 172.21.15.98:0/547929408 conn(0x56235660c290 msgr2=0x56235660ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:24.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.281+0000 7fbf8ec76000 1 -- 172.21.15.98:0/547929408 shutdown_connections 2024-09-19T08:43:24.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.281+0000 7fbf8ec76000 1 -- 172.21.15.98:0/547929408 wait complete. 2024-09-19T08:43:24.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.281+0000 7fbf8ec76000 20 client.6753 trim_cache size 0 max 16384 2024-09-19T08:43:24.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.297+0000 7fbf8ec76000 1 Processor -- start 2024-09-19T08:43:24.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.297+0000 7fbf8ec76000 1 -- start start 2024-09-19T08:43:24.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.297+0000 7fbf8ec76000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562356611330 0x562356654c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:24.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.297+0000 7fbf8ec76000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562356655b20 0x5623566551d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:24.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.297+0000 7fbf8ec76000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562356656440 0x562356666670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:24.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.297+0000 7fbf8ec76000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbf842035e0 con 0x562356655b20 2024-09-19T08:43:24.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.297+0000 7fbf8ec76000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbf84202970 con 0x562356656440 2024-09-19T08:43:24.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.297+0000 7fbf8ec76000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbf84202fc0 con 0x562356611330 2024-09-19T08:43:24.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.297+0000 7fbf89155640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562356655b20 0x5623566551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:24.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.297+0000 7fbf89155640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562356655b20 0x5623566551d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41508/0 (socket says 172.21.15.98:41508) 2024-09-19T08:43:24.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.297+0000 7fbf89155640 1 -- 172.21.15.98:0/2922216672 learned_addr learned my addr 172.21.15.98:0/2922216672 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:24.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.297+0000 7fbf89155640 1 -- 172.21.15.98:0/2922216672 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562356611330 msgr2=0x562356654c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:24.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.297+0000 7fbf89155640 1 --2- 172.21.15.98:0/2922216672 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562356611330 0x562356654c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.297+0000 7fbf89155640 1 -- 172.21.15.98:0/2922216672 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562356656440 msgr2=0x562356666670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:24.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.297+0000 7fbf89155640 1 --2- 172.21.15.98:0/2922216672 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562356656440 0x562356666670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.297+0000 7fbf89155640 1 -- 172.21.15.98:0/2922216672 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623563e1fd0 con 0x562356655b20 2024-09-19T08:43:24.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.297+0000 7fbf89155640 1 --2- 172.21.15.98:0/2922216672 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562356655b20 0x5623566551d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fbf74005c20 tx=0x7fbf7400a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad89d955b42d3f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:24.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf66ffd640 1 -- 172.21.15.98:0/2922216672 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf74014780 con 0x562356655b20 2024-09-19T08:43:24.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf66ffd640 1 -- 172.21.15.98:0/2922216672 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf7401a070 con 0x562356655b20 2024-09-19T08:43:24.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf66ffd640 1 -- 172.21.15.98:0/2922216672 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf74026630 con 0x562356655b20 2024-09-19T08:43:24.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf8ec76000 1 -- 172.21.15.98:0/2922216672 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562356655b20 msgr2=0x5623566551d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:24.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf8ec76000 1 --2- 172.21.15.98:0/2922216672 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562356655b20 0x5623566551d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fbf74005c20 tx=0x7fbf7400a000 comp rx=0 tx=0).stop 2024-09-19T08:43:24.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf8ec76000 1 -- 172.21.15.98:0/2922216672 shutdown_connections 2024-09-19T08:43:24.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf8ec76000 1 --2- 172.21.15.98:0/2922216672 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562356656440 0x562356666670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf8ec76000 1 --2- 172.21.15.98:0/2922216672 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562356655b20 0x5623566551d0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf8ec76000 1 --2- 172.21.15.98:0/2922216672 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562356611330 0x562356654c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf8ec76000 1 -- 172.21.15.98:0/2922216672 >> 172.21.15.98:0/2922216672 conn(0x56235660c290 msgr2=0x5623566105f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:24.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf8ec76000 1 -- 172.21.15.98:0/2922216672 shutdown_connections 2024-09-19T08:43:24.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf8ec76000 1 -- 172.21.15.98:0/2922216672 wait complete. 2024-09-19T08:43:24.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf8ec76000 1 -- start start 2024-09-19T08:43:24.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf8ec76000 1 Processor -- start 2024-09-19T08:43:24.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf8ec76000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562356611330 0x562356669590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:24.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf8ec76000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562356655b20 0x56235668d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:24.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf8ec76000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562356656440 0x56235668d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:24.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf8ec76000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x562356459f80 con 0x562356656440 2024-09-19T08:43:24.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf8ec76000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562356412ca0 con 0x562356611330 2024-09-19T08:43:24.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf8ec76000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x56235645f2a0 con 0x562356655b20 2024-09-19T08:43:24.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf8ec76000 10 client.0 _ll_register_callbacks cb 0x7fbf74001cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:43:24.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf8a157640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562356656440 0x56235668d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:24.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf8a157640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562356656440 0x56235668d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41514/0 (socket says 172.21.15.98:41514) 2024-09-19T08:43:24.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf8a157640 1 -- 172.21.15.98:0/2293547302 learned_addr learned my addr 172.21.15.98:0/2293547302 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:24.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf89956640 1 --2- 172.21.15.98:0/2293547302 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562356611330 0x562356669590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:24.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf89155640 1 --2- 172.21.15.98:0/2293547302 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562356655b20 0x56235668d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:24.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf8a157640 1 -- 172.21.15.98:0/2293547302 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562356655b20 msgr2=0x56235668d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:24.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf8a157640 1 --2- 172.21.15.98:0/2293547302 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562356655b20 0x56235668d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf8a157640 1 -- 172.21.15.98:0/2293547302 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562356611330 msgr2=0x562356669590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:24.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf8a157640 1 --2- 172.21.15.98:0/2293547302 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562356611330 0x562356669590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf8a157640 1 -- 172.21.15.98:0/2293547302 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623564dd750 con 0x562356656440 2024-09-19T08:43:24.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf89956640 1 --2- 172.21.15.98:0/2293547302 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562356611330 0x562356669590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:24.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf8a157640 1 --2- 172.21.15.98:0/2293547302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562356656440 0x56235668d6b0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c056e10 tx=0x7fbf7c0576c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5949d8a29d38b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:24.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-19T08:43:24.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf64ff9640 1 -- 172.21.15.98:0/2293547302 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c05fa60 con 0x562356656440 2024-09-19T08:43:24.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf64ff9640 1 -- 172.21.15.98:0/2293547302 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fbf7c085180 con 0x562356656440 2024-09-19T08:43:24.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf8ec76000 1 -- 172.21.15.98:0/2293547302 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf7401a070 con 0x562356656440 2024-09-19T08:43:24.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf64ff9640 1 -- 172.21.15.98:0/2293547302 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fbf7c075050 con 0x562356656440 2024-09-19T08:43:24.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf8ec76000 10 client.6759 fetch_fsmap learned FSMap version 47 2024-09-19T08:43:24.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf8ec76000 1 -- 172.21.15.98:0/2293547302 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) -- 0x5623563e8a30 con 0x562356656440 2024-09-19T08:43:24.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf64ff9640 1 -- 172.21.15.98:0/2293547302 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf7401a070 con 0x562356656440 2024-09-19T08:43:24.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf64ff9640 1 -- 172.21.15.98:0/2293547302 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 47) ==== 3502+0+0 (secure 0 0 0) 0x7fbf7c0534f0 con 0x562356656440 2024-09-19T08:43:24.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf8ec76000 10 client.6759 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T08:43:24.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf8ec76000 10 client.6759 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T08:43:24.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf8ec76000 20 client.6759 populate_metadata read hostname 'smithi098' 2024-09-19T08:43:24.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf8ec76000 1 --2- 172.21.15.98:0/2293547302 >> [v2:172.21.15.98:6834/837817273,v1:172.21.15.98:6835/837817273] conn(0x56235668dbf0 0x5623566d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:24.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf8ec76000 4 client.6759 mds_command: new command op to 6627 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T08:43:24.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf8ec76000 1 -- 172.21.15.98:0/2293547302 --> [v2:172.21.15.98:6834/837817273,v1:172.21.15.98:6835/837817273] -- command(tid 0: {"prefix": "status"}) -- 0x5623564683a0 con 0x56235668dbf0 2024-09-19T08:43:24.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf89956640 1 --2- 172.21.15.98:0/2293547302 >> [v2:172.21.15.98:6834/837817273,v1:172.21.15.98:6835/837817273] conn(0x56235668dbf0 0x5623566d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:24.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf64ff9640 10 client.6759 ms_handle_connect on v2:172.21.15.98:6834/837817273 2024-09-19T08:43:24.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.301+0000 7fbf89956640 1 --2- 172.21.15.98:0/2293547302 >> [v2:172.21.15.98:6834/837817273,v1:172.21.15.98:6835/837817273] conn(0x56235668dbf0 0x5623566d1560 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4d5e2aab8d3cd1d server_cookie=892d1a473b7e45c6 in_seq=0 out_seq=0 2024-09-19T08:43:24.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.305+0000 7fbf64ff9640 1 -- 172.21.15.98:0/2293547302 <== mds.0 v2:172.21.15.98:6834/837817273 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5623564683a0 con 0x56235668dbf0 2024-09-19T08:43:24.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.305+0000 7fbf64ff9640 10 client.6759 handle_command_reply: tid=0 2024-09-19T08:43:24.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.305+0000 7fbf8ec76000 1 client.6759 shutdown 2024-09-19T08:43:24.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.305+0000 7fbf8ec76000 1 -- 172.21.15.98:0/2293547302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562356656440 msgr2=0x56235668d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:24.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.305+0000 7fbf8ec76000 1 --2- 172.21.15.98:0/2293547302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562356656440 0x56235668d6b0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c056e10 tx=0x7fbf7c0576c0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.305+0000 7fbf8ec76000 1 -- 172.21.15.98:0/2293547302 shutdown_connections 2024-09-19T08:43:24.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.305+0000 7fbf8ec76000 1 --2- 172.21.15.98:0/2293547302 >> [v2:172.21.15.98:6834/837817273,v1:172.21.15.98:6835/837817273] conn(0x56235668dbf0 0x5623566d1560 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.305+0000 7fbf8ec76000 1 --2- 172.21.15.98:0/2293547302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x562356656440 0x56235668d6b0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.305+0000 7fbf8ec76000 1 --2- 172.21.15.98:0/2293547302 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x562356655b20 0x56235668d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.305+0000 7fbf8ec76000 1 --2- 172.21.15.98:0/2293547302 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562356611330 0x562356669590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.305+0000 7fbf64ff9640 0 client.6759 ms_handle_reset on v2:172.21.15.98:6834/837817273 2024-09-19T08:43:24.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.305+0000 7fbf8ec76000 1 -- 172.21.15.98:0/2293547302 >> 172.21.15.98:0/2293547302 conn(0x56235660c290 msgr2=0x562356653c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:24.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.305+0000 7fbf8ec76000 1 -- 172.21.15.98:0/2293547302 shutdown_connections 2024-09-19T08:43:24.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.305+0000 7fbf8ec76000 1 -- 172.21.15.98:0/2293547302 wait complete. 2024-09-19T08:43:24.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.305+0000 7fbf8ec76000 20 client.6759 trim_cache size 0 max 16384 2024-09-19T08:43:24.311 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:43:24.311 INFO:teuthology.orchestra.run.smithi098.stdout: "cluster_fsid": "552bf0ee-1f4f-4644-ba7c-d0931e55a451", 2024-09-19T08:43:24.311 INFO:teuthology.orchestra.run.smithi098.stdout: "whoami": 0, 2024-09-19T08:43:24.311 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 6627, 2024-09-19T08:43:24.311 INFO:teuthology.orchestra.run.smithi098.stdout: "want_state": "up:active", 2024-09-19T08:43:24.311 INFO:teuthology.orchestra.run.smithi098.stdout: "state": "up:active", 2024-09-19T08:43:24.311 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "cephfs", 2024-09-19T08:43:24.311 INFO:teuthology.orchestra.run.smithi098.stdout: "rank_uptime": 5.7504887130000002, 2024-09-19T08:43:24.311 INFO:teuthology.orchestra.run.smithi098.stdout: "mdsmap_epoch": 47, 2024-09-19T08:43:24.311 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch": 83, 2024-09-19T08:43:24.311 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch_barrier": 83, 2024-09-19T08:43:24.311 INFO:teuthology.orchestra.run.smithi098.stdout: "uptime": 11.092783702 2024-09-19T08:43:24.311 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:43:24.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.305+0000 7fbf8bbba640 1 -- 172.21.15.98:0/3683589140 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbf680679e0 msgr2=0x7fbf68069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:24.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.305+0000 7fbf8bbba640 1 --2- 172.21.15.98:0/3683589140 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbf680679e0 0x7fbf68069ea0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbf800ac000 tx=0x7fbf800011a0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.305+0000 7fbf8bbba640 1 -- 172.21.15.98:0/3683589140 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf840cb090 msgr2=0x7fbf840cb510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:24.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.305+0000 7fbf8bbba640 1 --2- 172.21.15.98:0/3683589140 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf840cb090 0x7fbf840cb510 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c009f50 tx=0x7fbf7c00f520 comp rx=0 tx=0).stop 2024-09-19T08:43:24.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.305+0000 7fbf8bbba640 1 -- 172.21.15.98:0/3683589140 shutdown_connections 2024-09-19T08:43:24.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.305+0000 7fbf8bbba640 1 --2- 172.21.15.98:0/3683589140 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbf680679e0 0x7fbf68069ea0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.305+0000 7fbf8bbba640 1 --2- 172.21.15.98:0/3683589140 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf840cb090 0x7fbf840cb510 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.305+0000 7fbf8bbba640 1 --2- 172.21.15.98:0/3683589140 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbf840cbfb0 0x7fbf840cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.305+0000 7fbf8bbba640 1 --2- 172.21.15.98:0/3683589140 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf700039c0 0x7fbf841ea940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.305+0000 7fbf8bbba640 1 -- 172.21.15.98:0/3683589140 >> 172.21.15.98:0/3683589140 conn(0x7fbf84075070 msgr2=0x7fbf84161530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:24.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.305+0000 7fbf8bbba640 1 -- 172.21.15.98:0/3683589140 shutdown_connections 2024-09-19T08:43:24.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.305+0000 7fbf8bbba640 1 -- 172.21.15.98:0/3683589140 wait complete. 2024-09-19T08:43:24.320 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T08:43:24.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.477+0000 7f44f4423640 1 Processor -- start 2024-09-19T08:43:24.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.477+0000 7f44f4423640 1 -- start start 2024-09-19T08:43:24.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.481+0000 7f44f4423640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44ec06bc90 0x7f44ec06c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:24.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.481+0000 7f44f4423640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f44ec0ce840 0x7f44ec0cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:24.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.481+0000 7f44f4423640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44ec0ccc60 con 0x7f44ec158fd0 2024-09-19T08:43:24.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.481+0000 7f44f4423640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f44ec1593b0 con 0x7f44ec0ce840 2024-09-19T08:43:24.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.481+0000 7f44f4423640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f44ec0ccf40 con 0x7f44ec06bc90 2024-09-19T08:43:24.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.481+0000 7f44f21bf640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f44ec158fd0 0x7f44ec0b3e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40102/0 (socket says 172.21.15.98:40102) 2024-09-19T08:43:24.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.481+0000 7f44f21bf640 1 -- 172.21.15.98:0/3804506035 learned_addr learned my addr 172.21.15.98:0/3804506035 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:24.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.481+0000 7f44f29c0640 1 --2- 172.21.15.98:0/3804506035 >> v2:172.21.15.98:3300/0 conn(0x7f44ec0ce840 0x7f44ec0cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:24.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.481+0000 7f44f19be640 1 --2- 172.21.15.98:0/3804506035 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44ec06bc90 0x7f44ec06c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:24.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.481+0000 7f44f29c0640 1 -- 172.21.15.98:0/3804506035 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44ec06bc90 msgr2=0x7f44ec06c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:24.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.481+0000 7f44f29c0640 1 --2- 172.21.15.98:0/3804506035 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44ec06bc90 0x7f44ec06c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.481+0000 7f44f29c0640 1 -- 172.21.15.98:0/3804506035 >> v1:172.21.15.98:6789/0 conn(0x7f44ec158fd0 legacy=0x7f44ec0b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:24.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.481+0000 7f44f11bd640 1 -- 172.21.15.98:0/3804506035 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 568871747 0 0) 0x7f44ec0ccc60 con 0x7f44ec158fd0 2024-09-19T08:43:24.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.481+0000 7f44f29c0640 1 -- 172.21.15.98:0/3804506035 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ec0cd220 con 0x7f44ec0ce840 2024-09-19T08:43:24.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.481+0000 7f44f29c0640 1 --2- 172.21.15.98:0/3804506035 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44ec0ce840 0x7f44ec0cec90 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f44e40097c0 tx=0x7f44e401c150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f788b84303d35c69 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:24.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.481+0000 7f44f11bd640 1 -- 172.21.15.98:0/3804506035 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e40b0070 con 0x7f44ec0ce840 2024-09-19T08:43:24.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.481+0000 7f44f11bd640 1 -- 172.21.15.98:0/3804506035 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44e40ab600 con 0x7f44ec0ce840 2024-09-19T08:43:24.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.481+0000 7f44f11bd640 1 -- 172.21.15.98:0/3804506035 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e40ab9d0 con 0x7f44ec0ce840 2024-09-19T08:43:24.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.485+0000 7f44f4423640 1 -- 172.21.15.98:0/3804506035 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44ec0ce840 msgr2=0x7f44ec0cec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:24.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.485+0000 7f44f4423640 1 --2- 172.21.15.98:0/3804506035 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44ec0ce840 0x7f44ec0cec90 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f44e40097c0 tx=0x7f44e401c150 comp rx=0 tx=0).stop 2024-09-19T08:43:24.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.485+0000 7f44f4423640 1 -- 172.21.15.98:0/3804506035 shutdown_connections 2024-09-19T08:43:24.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.485+0000 7f44f4423640 1 --2- 172.21.15.98:0/3804506035 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44ec0ce840 0x7f44ec0cec90 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.485+0000 7f44f4423640 1 --2- 172.21.15.98:0/3804506035 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44ec06bc90 0x7f44ec06c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.485+0000 7f44f4423640 1 -- 172.21.15.98:0/3804506035 >> 172.21.15.98:0/3804506035 conn(0x7f44ec0b2010 msgr2=0x7f44ec0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:24.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.485+0000 7f44f4423640 1 -- 172.21.15.98:0/3804506035 shutdown_connections 2024-09-19T08:43:24.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.485+0000 7f44f4423640 1 -- 172.21.15.98:0/3804506035 wait complete. 2024-09-19T08:43:24.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.485+0000 7f44f4423640 1 Processor -- start 2024-09-19T08:43:24.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.485+0000 7f44f4423640 1 -- start start 2024-09-19T08:43:24.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.485+0000 7f44f4423640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44ec06bc90 0x7f44ec1f8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:24.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.485+0000 7f44f4423640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44ec0ce840 0x7f44ec1f6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:24.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.485+0000 7f44f4423640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44ec158fd0 0x7f44ec157480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:24.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.485+0000 7f44f4423640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec15c920 con 0x7f44ec06bc90 2024-09-19T08:43:24.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.485+0000 7f44f4423640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec15c7a0 con 0x7f44ec0ce840 2024-09-19T08:43:24.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.485+0000 7f44f4423640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f44ec168110 con 0x7f44ec158fd0 2024-09-19T08:43:24.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.485+0000 7f44f21bf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44ec06bc90 0x7f44ec1f8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:24.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.485+0000 7f44f21bf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44ec06bc90 0x7f44ec1f8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41528/0 (socket says 172.21.15.98:41528) 2024-09-19T08:43:24.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.485+0000 7f44f21bf640 1 -- 172.21.15.98:0/2977644717 learned_addr learned my addr 172.21.15.98:0/2977644717 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:24.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.485+0000 7f44f21bf640 1 -- 172.21.15.98:0/2977644717 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44ec158fd0 msgr2=0x7f44ec157480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:24.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.485+0000 7f44f21bf640 1 --2- 172.21.15.98:0/2977644717 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44ec158fd0 0x7f44ec157480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.485+0000 7f44f21bf640 1 -- 172.21.15.98:0/2977644717 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44ec0ce840 msgr2=0x7f44ec1f6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:24.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.485+0000 7f44f21bf640 1 --2- 172.21.15.98:0/2977644717 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44ec0ce840 0x7f44ec1f6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.485+0000 7f44f21bf640 1 -- 172.21.15.98:0/2977644717 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ec15cbe0 con 0x7f44ec06bc90 2024-09-19T08:43:24.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.485+0000 7f44f21bf640 1 --2- 172.21.15.98:0/2977644717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44ec06bc90 0x7f44ec1f8020 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f44e800c7c0 tx=0x7f44e800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:24.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.485+0000 7f44e2ffd640 1 -- 172.21.15.98:0/2977644717 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e8018070 con 0x7f44ec06bc90 2024-09-19T08:43:24.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.485+0000 7f44e2ffd640 1 -- 172.21.15.98:0/2977644717 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44e800a640 con 0x7f44ec06bc90 2024-09-19T08:43:24.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.485+0000 7f44e2ffd640 1 -- 172.21.15.98:0/2977644717 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e800aa30 con 0x7f44ec06bc90 2024-09-19T08:43:24.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.485+0000 7f44f4423640 1 -- 172.21.15.98:0/2977644717 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44ec15cec0 con 0x7f44ec06bc90 2024-09-19T08:43:24.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.485+0000 7f44f4423640 1 -- 172.21.15.98:0/2977644717 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44ec15d480 con 0x7f44ec06bc90 2024-09-19T08:43:24.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.489+0000 7f44f4423640 1 -- 172.21.15.98:0/2977644717 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44ec06c120 con 0x7f44ec06bc90 2024-09-19T08:43:24.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.489+0000 7f44e2ffd640 1 -- 172.21.15.98:0/2977644717 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f44e801b070 con 0x7f44ec06bc90 2024-09-19T08:43:24.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.489+0000 7f44e2ffd640 1 --2- 172.21.15.98:0/2977644717 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f44d00679e0 0x7f44d0069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:24.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.489+0000 7f44e2ffd640 1 -- 172.21.15.98:0/2977644717 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f44e8085680 con 0x7f44ec06bc90 2024-09-19T08:43:24.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.489+0000 7f44f19be640 1 --2- 172.21.15.98:0/2977644717 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f44d00679e0 0x7f44d0069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:24.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.489+0000 7f44f19be640 1 --2- 172.21.15.98:0/2977644717 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f44d00679e0 0x7f44d0069ea0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f44ec074390 tx=0x7f44dc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:24.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.493+0000 7f44e2ffd640 1 -- 172.21.15.98:0/2977644717 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f44e8056220 con 0x7f44ec06bc90 2024-09-19T08:43:24.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.677+0000 7f44f4423640 1 -- 172.21.15.98:0/2977644717 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f44ec15d650 con 0x7f44ec06bc90 2024-09-19T08:43:24.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.677+0000 7f44e2ffd640 1 -- 172.21.15.98:0/2977644717 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v47) ==== 53+0+73 (secure 0 0 0) 0x7f44e8055b20 con 0x7f44ec06bc90 2024-09-19T08:43:24.679 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T08:43:24.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.681+0000 7f44f4423640 1 -- 172.21.15.98:0/2977644717 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f44d00679e0 msgr2=0x7f44d0069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:24.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.681+0000 7f44f4423640 1 --2- 172.21.15.98:0/2977644717 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f44d00679e0 0x7f44d0069ea0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f44ec074390 tx=0x7f44dc008040 comp rx=0 tx=0).stop 2024-09-19T08:43:24.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.681+0000 7f44f4423640 1 -- 172.21.15.98:0/2977644717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44ec06bc90 msgr2=0x7f44ec1f8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:24.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.681+0000 7f44f4423640 1 --2- 172.21.15.98:0/2977644717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44ec06bc90 0x7f44ec1f8020 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f44e800c7c0 tx=0x7f44e800cc80 comp rx=0 tx=0).stop 2024-09-19T08:43:24.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.681+0000 7f44f4423640 1 -- 172.21.15.98:0/2977644717 shutdown_connections 2024-09-19T08:43:24.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.681+0000 7f44f4423640 1 --2- 172.21.15.98:0/2977644717 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f44d00679e0 0x7f44d0069ea0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.681+0000 7f44f4423640 1 --2- 172.21.15.98:0/2977644717 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f44ec158fd0 0x7f44ec157480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.681+0000 7f44f4423640 1 --2- 172.21.15.98:0/2977644717 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44ec0ce840 0x7f44ec1f6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.681+0000 7f44f4423640 1 --2- 172.21.15.98:0/2977644717 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f44ec06bc90 0x7f44ec1f8020 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:24.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.681+0000 7f44f4423640 1 -- 172.21.15.98:0/2977644717 >> 172.21.15.98:0/2977644717 conn(0x7f44ec0b2010 msgr2=0x7f44ec0d0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:24.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.681+0000 7f44f4423640 1 -- 172.21.15.98:0/2977644717 shutdown_connections 2024-09-19T08:43:24.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24.681+0000 7f44f4423640 1 -- 172.21.15.98:0/2977644717 wait complete. 2024-09-19T08:43:24.696 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T08:43:24.696 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T08:43:24.696 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi098.front.sepia.ceph.com 2024-09-19T08:43:24.697 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T08:43:24.697 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T08:43:24.697 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T08:43:24.697 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T08:43:24.697 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T08:43:24.697 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:43:24.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:43:24.697 DEBUG:teuthology.orchestra.run.smithi098:> ip addr 2024-09-19T08:43:24.703 INFO:teuthology.orchestra.run.smithi098.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T08:43:24.703 INFO:teuthology.orchestra.run.smithi098.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T08:43:24.703 INFO:teuthology.orchestra.run.smithi098.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T08:43:24.703 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:43:24.703 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 ::1/128 scope host 2024-09-19T08:43:24.703 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:43:24.703 INFO:teuthology.orchestra.run.smithi098.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:43:24.703 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b8 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:43:24.703 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f0 2024-09-19T08:43:24.704 INFO:teuthology.orchestra.run.smithi098.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:43:24.704 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b9 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:43:24.704 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f1 2024-09-19T08:43:24.704 INFO:teuthology.orchestra.run.smithi098.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:43:24.704 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6a brd ff:ff:ff:ff:ff:ff 2024-09-19T08:43:24.704 INFO:teuthology.orchestra.run.smithi098.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T08:43:24.704 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6b brd ff:ff:ff:ff:ff:ff 2024-09-19T08:43:24.704 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.21.15.98/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T08:43:24.704 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft 42204sec preferred_lft 42204sec 2024-09-19T08:43:24.704 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::ec4:7aff:fe8f:d26b/64 scope link 2024-09-19T08:43:24.704 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:43:24.704 INFO:teuthology.orchestra.run.smithi098.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T08:43:24.705 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 02:42:98:dd:32:2d brd ff:ff:ff:ff:ff:ff 2024-09-19T08:43:24.705 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T08:43:24.705 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:43:24.705 INFO:teuthology.orchestra.run.smithi098.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T08:43:24.705 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-19T08:43:24.705 INFO:teuthology.orchestra.run.smithi098.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T08:43:24.705 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:43:24.705 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-19T08:43:24.705 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:43:24.705 INFO:teuthology.orchestra.run.smithi098.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:43:24.705 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:43:24.705 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-19T08:43:24.706 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:43:24.706 INFO:teuthology.orchestra.run.smithi098.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:43:24.706 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:43:24.706 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-19T08:43:24.706 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:43:24.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:43:24.706 DEBUG:teuthology.orchestra.run.smithi098:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T08:43:24.752 INFO:teuthology.orchestra.run.smithi098.stdout:1 2024-09-19T08:43:24.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:43:24.754 DEBUG:teuthology.orchestra.run.smithi098:> route 2024-09-19T08:43:24.804 INFO:teuthology.orchestra.run.smithi098.stdout:Kernel IP routing table 2024-09-19T08:43:24.804 INFO:teuthology.orchestra.run.smithi098.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T08:43:24.804 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T08:43:24.804 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T08:43:24.804 INFO:teuthology.orchestra.run.smithi098.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T08:43:24.804 INFO:teuthology.orchestra.run.smithi098.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T08:43:24.804 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T08:43:24.804 INFO:teuthology.orchestra.run.smithi098.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:43:24.804 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:43:24.805 INFO:teuthology.orchestra.run.smithi098.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:43:24.805 INFO:teuthology.orchestra.run.smithi098.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T08:43:24.805 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:43:24.805 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-09-19T08:43:24.805 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T08:43:24.805 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T08:43:24.805 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T08:43:24.805 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-09-19T08:43:24.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:43:24.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:43:24.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:43:24.925 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2024-09-19T08:43:24.972 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-19T08:43:24.972 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T08:43:24.973 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:43:24.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:43:24.973 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set brx.0 up 2024-09-19T08:43:25.026 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-19T08:43:25.026 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:43:25.026 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T08:43:25.073 INFO:teuthology.orchestra.run.smithi098.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:43:25.073 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:43:25.073 DEBUG:teuthology.orchestra.run.smithi098:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T08:43:25.120 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:43:25.178 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:43:25.227 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:43:25.227 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:43:25.228 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:43:25.228 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:43:25.228 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:43:25.228 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:43:25.228 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:43:25.228 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:43:25.228 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:43:25.228 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:43:25.228 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:43:25.228 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:43:25.228 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:43:25.228 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:43:25.228 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:43:25.229 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:43:25.229 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:43:25.229 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:43:25.229 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:43:25.229 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:43:25.229 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:43:25.229 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:43:25.229 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:43:25.229 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:43:25.229 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:43:25.229 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:43:25.229 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:43:25.229 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:43:25.229 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:43:25.230 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:43:25.230 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:43:25.230 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:43:25.230 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:43:25.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:43:25.231 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:43:25.277 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T08:43:25.278 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-19T08:43:25.322 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:43:25.338 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:43:25.385 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:43:25.385 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:43:25.385 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:43:25.385 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:43:25.385 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:43:25.385 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:43:25.385 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:43:25.386 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:43:25.386 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:43:25.386 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:43:25.386 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:43:25.386 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:43:25.386 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:43:25.386 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:43:25.386 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:43:25.386 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:43:25.386 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:43:25.386 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:43:25.386 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:43:25.386 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:43:25.386 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:43:25.387 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:43:25.387 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:43:25.387 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:43:25.387 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:43:25.387 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:43:25.387 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:43:25.387 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:43:25.387 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:43:25.387 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:43:25.387 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:43:25.387 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:43:25.387 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:43:25.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:43:25.388 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:43:25.406 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:2024-09-19T08:43:25.405+0000 7f4a47374580 -1 init, newargv = 0x5596752ba3e0 newargc=15 2024-09-19T08:43:25.406 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:ceph-fuse[44479]: starting ceph client 2024-09-19T08:43:25.412 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:ceph-fuse[44479]: starting fuse 2024-09-19T08:43:26.511 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:43:26.528 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:43:26.586 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:43:26.586 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:43:26.586 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:43:26.586 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:43:26.587 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:43:26.587 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:43:26.587 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:43:26.587 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:43:26.587 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:43:26.587 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:43:26.588 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:43:26.588 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:43:26.588 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:43:26.588 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:43:26.588 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:43:26.588 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:43:26.589 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:43:26.589 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:43:26.589 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:43:26.589 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:43:26.589 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:43:26.590 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:43:26.590 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:43:26.590 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:43:26.590 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:43:26.591 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:43:26.591 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:43:26.591 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:43:26.591 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:43:26.591 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:43:26.591 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:43:26.592 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:43:26.592 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:43:26.592 INFO:teuthology.orchestra.run.smithi098.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-19T08:43:26.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:43:26.593 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:43:26.633 INFO:teuthology.orchestra.run.smithi098.stdout:45 2024-09-19T08:43:26.633 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-19T08:43:26.633 DEBUG:teuthology.orchestra.run.smithi098:> sudo stdin-killer -- python3 -c ' 2024-09-19T08:43:26.633 DEBUG:teuthology.orchestra.run.smithi098:> import glob 2024-09-19T08:43:26.633 DEBUG:teuthology.orchestra.run.smithi098:> import re 2024-09-19T08:43:26.633 DEBUG:teuthology.orchestra.run.smithi098:> import os 2024-09-19T08:43:26.634 DEBUG:teuthology.orchestra.run.smithi098:> import subprocess 2024-09-19T08:43:26.634 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:43:26.634 DEBUG:teuthology.orchestra.run.smithi098:> def _find_admin_socket(client_name): 2024-09-19T08:43:26.634 DEBUG:teuthology.orchestra.run.smithi098:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T08:43:26.634 DEBUG:teuthology.orchestra.run.smithi098:> files = glob.glob(asok_path) 2024-09-19T08:43:26.634 DEBUG:teuthology.orchestra.run.smithi098:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T08:43:26.634 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:43:26.634 DEBUG:teuthology.orchestra.run.smithi098:> # Given a non-glob path, it better be there 2024-09-19T08:43:26.634 DEBUG:teuthology.orchestra.run.smithi098:> if "*" not in asok_path: 2024-09-19T08:43:26.634 DEBUG:teuthology.orchestra.run.smithi098:> assert(len(files) == 1) 2024-09-19T08:43:26.634 DEBUG:teuthology.orchestra.run.smithi098:> return files[0] 2024-09-19T08:43:26.634 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:43:26.635 DEBUG:teuthology.orchestra.run.smithi098:> for f in files: 2024-09-19T08:43:26.635 DEBUG:teuthology.orchestra.run.smithi098:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T08:43:26.635 DEBUG:teuthology.orchestra.run.smithi098:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T08:43:26.635 DEBUG:teuthology.orchestra.run.smithi098:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T08:43:26.635 DEBUG:teuthology.orchestra.run.smithi098:> contents = proc_f.read() 2024-09-19T08:43:26.635 DEBUG:teuthology.orchestra.run.smithi098:> if mountpoint in contents: 2024-09-19T08:43:26.635 DEBUG:teuthology.orchestra.run.smithi098:> return f 2024-09-19T08:43:26.635 DEBUG:teuthology.orchestra.run.smithi098:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T08:43:26.635 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:43:26.635 DEBUG:teuthology.orchestra.run.smithi098:> print(_find_admin_socket("client.0")) 2024-09-19T08:43:26.635 DEBUG:teuthology.orchestra.run.smithi098:> ' 2024-09-19T08:43:26.758 INFO:teuthology.orchestra.run.smithi098.stdout:/var/run/ceph/ceph-client.0.44479.asok 2024-09-19T08:43:26.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:43:26.772 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.44479.asok 2024-09-19T08:43:26.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:43:26.772 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.44479.asok status 2024-09-19T08:43:26.926 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:43:26.926 INFO:teuthology.orchestra.run.smithi098.stdout: "metadata": { 2024-09-19T08:43:26.926 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T08:43:26.926 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T08:43:26.927 INFO:teuthology.orchestra.run.smithi098.stdout: "entity_id": "0", 2024-09-19T08:43:26.927 INFO:teuthology.orchestra.run.smithi098.stdout: "hostname": "smithi098", 2024-09-19T08:43:26.927 INFO:teuthology.orchestra.run.smithi098.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T08:43:26.927 INFO:teuthology.orchestra.run.smithi098.stdout: "pid": "44479", 2024-09-19T08:43:26.927 INFO:teuthology.orchestra.run.smithi098.stdout: "root": "/" 2024-09-19T08:43:26.927 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:26.927 INFO:teuthology.orchestra.run.smithi098.stdout: "dentry_count": 0, 2024-09-19T08:43:26.927 INFO:teuthology.orchestra.run.smithi098.stdout: "dentry_pinned_count": 0, 2024-09-19T08:43:26.927 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 6774, 2024-09-19T08:43:26.927 INFO:teuthology.orchestra.run.smithi098.stdout: "inst": { 2024-09-19T08:43:26.927 INFO:teuthology.orchestra.run.smithi098.stdout: "name": { 2024-09-19T08:43:26.927 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "client", 2024-09-19T08:43:26.927 INFO:teuthology.orchestra.run.smithi098.stdout: "num": 6774 2024-09-19T08:43:26.927 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:26.928 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": { 2024-09-19T08:43:26.928 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v1", 2024-09-19T08:43:26.928 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:43:26.928 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 42608429 2024-09-19T08:43:26.928 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:26.928 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:26.929 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": { 2024-09-19T08:43:26.929 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v1", 2024-09-19T08:43:26.929 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:43:26.929 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 42608429 2024-09-19T08:43:26.929 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:26.929 INFO:teuthology.orchestra.run.smithi098.stdout: "inst_str": "client.6774 192.168.0.1:0/42608429", 2024-09-19T08:43:26.929 INFO:teuthology.orchestra.run.smithi098.stdout: "addr_str": "192.168.0.1:0/42608429", 2024-09-19T08:43:26.929 INFO:teuthology.orchestra.run.smithi098.stdout: "inode_count": 1, 2024-09-19T08:43:26.929 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_epoch": 47, 2024-09-19T08:43:26.930 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_epoch": 83, 2024-09-19T08:43:26.930 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_epoch_barrier": 0, 2024-09-19T08:43:26.930 INFO:teuthology.orchestra.run.smithi098.stdout: "blocklisted": false, 2024-09-19T08:43:26.930 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "cephfs" 2024-09-19T08:43:26.930 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:43:26.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:43:26.941 DEBUG:teuthology.orchestra.run.smithi098:> stat --file-system '--printf=%T 2024-09-19T08:43:26.941 DEBUG:teuthology.orchestra.run.smithi098:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T08:43:26.946 INFO:teuthology.orchestra.run.smithi098.stdout:fuseblk 2024-09-19T08:43:26.947 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T08:43:26.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:43:26.947 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T08:43:27.000 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T08:43:27.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.169+0000 7fe01e85d640 1 Processor -- start 2024-09-19T08:43:27.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.169+0000 7fe01e85d640 1 -- start start 2024-09-19T08:43:27.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.169+0000 7fe01e85d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0181628a0 0x7fe018166f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:27.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.169+0000 7fe01e85d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe0180ced60 0x7fe0180cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:27.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.169+0000 7fe01e85d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe01806d590 con 0x7fe01806c7c0 2024-09-19T08:43:27.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.169+0000 7fe01e85d640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fe01806d870 con 0x7fe0180ced60 2024-09-19T08:43:27.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.169+0000 7fe01e85d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe018166490 con 0x7fe0181628a0 2024-09-19T08:43:27.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.169+0000 7fe01cdfa640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe0180ced60 0x7fe0180cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:27.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.169+0000 7fe01cdfa640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe0180ced60 0x7fe0180cf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41534/0 (socket says 172.21.15.98:41534) 2024-09-19T08:43:27.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.169+0000 7fe01cdfa640 1 -- 172.21.15.98:0/1992168300 learned_addr learned my addr 172.21.15.98:0/1992168300 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:27.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.169+0000 7fe017fff640 1 --2- 172.21.15.98:0/1992168300 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0181628a0 0x7fe018166f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:27.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.169+0000 7fe01cdfa640 1 -- 172.21.15.98:0/1992168300 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0181628a0 msgr2=0x7fe018166f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:27.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.169+0000 7fe01cdfa640 1 --2- 172.21.15.98:0/1992168300 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0181628a0 0x7fe018166f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:27.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.169+0000 7fe01cdfa640 1 -- 172.21.15.98:0/1992168300 >> v1:172.21.15.139:6789/0 conn(0x7fe01806c7c0 legacy=0x7fe01806cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:27.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.169+0000 7fe01cdfa640 1 -- 172.21.15.98:0/1992168300 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe018166770 con 0x7fe0180ced60 2024-09-19T08:43:27.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.169+0000 7fe017fff640 1 --2- 172.21.15.98:0/1992168300 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0181628a0 0x7fe018166f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:27.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.173+0000 7fe01cdfa640 1 --2- 172.21.15.98:0/1992168300 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0180ced60 0x7fe0180cf210 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fe010009e50 tx=0x7fe01001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5e77a2488a21ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:27.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.173+0000 7fe016ffd640 1 -- 172.21.15.98:0/1992168300 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0100a5670 con 0x7fe0180ced60 2024-09-19T08:43:27.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.173+0000 7fe016ffd640 1 -- 172.21.15.98:0/1992168300 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0100a5810 con 0x7fe0180ced60 2024-09-19T08:43:27.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.173+0000 7fe016ffd640 1 -- 172.21.15.98:0/1992168300 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0100a5be0 con 0x7fe0180ced60 2024-09-19T08:43:27.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.173+0000 7fe01e85d640 1 -- 172.21.15.98:0/1992168300 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0180ced60 msgr2=0x7fe0180cf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:27.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.173+0000 7fe01e85d640 1 --2- 172.21.15.98:0/1992168300 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0180ced60 0x7fe0180cf210 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fe010009e50 tx=0x7fe01001c9f0 comp rx=0 tx=0).stop 2024-09-19T08:43:27.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.173+0000 7fe01e85d640 1 -- 172.21.15.98:0/1992168300 shutdown_connections 2024-09-19T08:43:27.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.173+0000 7fe01e85d640 1 --2- 172.21.15.98:0/1992168300 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe0180ced60 0x7fe0180cf210 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:27.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.173+0000 7fe01e85d640 1 --2- 172.21.15.98:0/1992168300 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0181628a0 0x7fe018166f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:27.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.173+0000 7fe01e85d640 1 -- 172.21.15.98:0/1992168300 >> 172.21.15.98:0/1992168300 conn(0x7fe0180b40a0 msgr2=0x7fe0180b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:27.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.173+0000 7fe01e85d640 1 -- 172.21.15.98:0/1992168300 shutdown_connections 2024-09-19T08:43:27.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.173+0000 7fe01e85d640 1 -- 172.21.15.98:0/1992168300 wait complete. 2024-09-19T08:43:27.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.173+0000 7fe01e85d640 1 Processor -- start 2024-09-19T08:43:27.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.173+0000 7fe01e85d640 1 -- start start 2024-09-19T08:43:27.177 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.177+0000 7fe01e85d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe01806c7c0 0x7fe018157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:27.177 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.177+0000 7fe01e85d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0180ced60 0x7fe018160f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:27.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.177+0000 7fe01e85d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0181628a0 0x7fe018158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:27.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.177+0000 7fe01e85d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe0180d1ab0 con 0x7fe01806c7c0 2024-09-19T08:43:27.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.177+0000 7fe01e85d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe0180d1930 con 0x7fe0181628a0 2024-09-19T08:43:27.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.177+0000 7fe01e85d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe0180d1c30 con 0x7fe0180ced60 2024-09-19T08:43:27.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.177+0000 7fe017fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe01806c7c0 0x7fe018157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:27.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.177+0000 7fe017fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe01806c7c0 0x7fe018157710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41550/0 (socket says 172.21.15.98:41550) 2024-09-19T08:43:27.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.177+0000 7fe017fff640 1 -- 172.21.15.98:0/4029909980 learned_addr learned my addr 172.21.15.98:0/4029909980 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:27.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.177+0000 7fe01cdfa640 1 --2- 172.21.15.98:0/4029909980 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0181628a0 0x7fe018158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:27.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.177+0000 7fe0177fe640 1 --2- 172.21.15.98:0/4029909980 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0180ced60 0x7fe018160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:27.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.177+0000 7fe017fff640 1 -- 172.21.15.98:0/4029909980 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0180ced60 msgr2=0x7fe018160f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:27.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.177+0000 7fe017fff640 1 --2- 172.21.15.98:0/4029909980 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0180ced60 0x7fe018160f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:27.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.177+0000 7fe017fff640 1 -- 172.21.15.98:0/4029909980 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0181628a0 msgr2=0x7fe018158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:27.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.177+0000 7fe017fff640 1 --2- 172.21.15.98:0/4029909980 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0181628a0 0x7fe018158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:27.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.177+0000 7fe017fff640 1 -- 172.21.15.98:0/4029909980 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe018163860 con 0x7fe01806c7c0 2024-09-19T08:43:27.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.177+0000 7fe017fff640 1 --2- 172.21.15.98:0/4029909980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe01806c7c0 0x7fe018157710 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fe00800c7c0 tx=0x7fe00800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:27.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.177+0000 7fe014ff9640 1 -- 172.21.15.98:0/4029909980 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe008007a30 con 0x7fe01806c7c0 2024-09-19T08:43:27.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.177+0000 7fe014ff9640 1 -- 172.21.15.98:0/4029909980 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe008007bd0 con 0x7fe01806c7c0 2024-09-19T08:43:27.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.177+0000 7fe01e85d640 1 -- 172.21.15.98:0/4029909980 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe018163af0 con 0x7fe01806c7c0 2024-09-19T08:43:27.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.177+0000 7fe014ff9640 1 -- 172.21.15.98:0/4029909980 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0080113f0 con 0x7fe01806c7c0 2024-09-19T08:43:27.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.177+0000 7fe01e85d640 1 -- 172.21.15.98:0/4029909980 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0181640b0 con 0x7fe01806c7c0 2024-09-19T08:43:27.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.177+0000 7fe01e85d640 1 -- 172.21.15.98:0/4029909980 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0180cf210 con 0x7fe01806c7c0 2024-09-19T08:43:27.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.177+0000 7fe014ff9640 1 -- 172.21.15.98:0/4029909980 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fe008011590 con 0x7fe01806c7c0 2024-09-19T08:43:27.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.177+0000 7fe014ff9640 1 --2- 172.21.15.98:0/4029909980 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe004067a10 0x7fe004069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:27.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.177+0000 7fe014ff9640 1 -- 172.21.15.98:0/4029909980 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fe008086110 con 0x7fe01806c7c0 2024-09-19T08:43:27.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.177+0000 7fe0177fe640 1 --2- 172.21.15.98:0/4029909980 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe004067a10 0x7fe004069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:27.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.181+0000 7fe0177fe640 1 --2- 172.21.15.98:0/4029909980 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe004067a10 0x7fe004069ed0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe018166ee0 tx=0x7fe00c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:27.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.181+0000 7fe014ff9640 1 -- 172.21.15.98:0/4029909980 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe008015020 con 0x7fe01806c7c0 2024-09-19T08:43:27.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.377+0000 7fe01e85d640 1 -- 172.21.15.98:0/4029909980 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe0180688c0 con 0x7fe01806c7c0 2024-09-19T08:43:27.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.377+0000 7fe014ff9640 1 -- 172.21.15.98:0/4029909980 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v47) ==== 53+0+73 (secure 0 0 0) 0x7fe008056a30 con 0x7fe01806c7c0 2024-09-19T08:43:27.380 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T08:43:27.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.381+0000 7fe01e85d640 1 -- 172.21.15.98:0/4029909980 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe004067a10 msgr2=0x7fe004069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:27.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.381+0000 7fe01e85d640 1 --2- 172.21.15.98:0/4029909980 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe004067a10 0x7fe004069ed0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe018166ee0 tx=0x7fe00c008040 comp rx=0 tx=0).stop 2024-09-19T08:43:27.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.381+0000 7fe01e85d640 1 -- 172.21.15.98:0/4029909980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe01806c7c0 msgr2=0x7fe018157710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:27.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.381+0000 7fe01e85d640 1 --2- 172.21.15.98:0/4029909980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe01806c7c0 0x7fe018157710 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fe00800c7c0 tx=0x7fe00800cc80 comp rx=0 tx=0).stop 2024-09-19T08:43:27.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.381+0000 7fe01e85d640 1 -- 172.21.15.98:0/4029909980 shutdown_connections 2024-09-19T08:43:27.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.381+0000 7fe01e85d640 1 --2- 172.21.15.98:0/4029909980 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe004067a10 0x7fe004069ed0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:27.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.381+0000 7fe01e85d640 1 --2- 172.21.15.98:0/4029909980 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe0181628a0 0x7fe018158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:27.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.381+0000 7fe01e85d640 1 --2- 172.21.15.98:0/4029909980 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe0180ced60 0x7fe018160f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:27.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.381+0000 7fe01e85d640 1 --2- 172.21.15.98:0/4029909980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe01806c7c0 0x7fe018157710 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:27.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.381+0000 7fe01e85d640 1 -- 172.21.15.98:0/4029909980 >> 172.21.15.98:0/4029909980 conn(0x7fe0180b40a0 msgr2=0x7fe01806cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:27.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.381+0000 7fe01e85d640 1 -- 172.21.15.98:0/4029909980 shutdown_connections 2024-09-19T08:43:27.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:27.381+0000 7fe01e85d640 1 -- 172.21.15.98:0/4029909980 wait complete. 2024-09-19T08:43:27.401 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T08:43:27.401 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T08:43:27.401 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2024-09-19T08:43:27.401 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-19T08:43:27.402 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-19T08:43:27.402 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T08:43:27.402 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T08:43:27.402 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T08:43:27.402 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T08:43:27.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:43:27.402 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2024-09-19T08:43:27.410 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T08:43:27.410 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T08:43:27.410 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T08:43:27.410 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:43:27.410 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-09-19T08:43:27.410 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:43:27.410 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:43:27.410 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:43:27.410 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2024-09-19T08:43:27.410 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:43:27.411 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:43:27.411 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2024-09-19T08:43:27.411 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:43:27.411 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:43:27.411 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T08:43:27.411 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:43:27.411 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T08:43:27.411 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 42212sec preferred_lft 42212sec 2024-09-19T08:43:27.411 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::ec4:7aff:fe88:7f59/64 scope link 2024-09-19T08:43:27.411 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:43:27.411 INFO:teuthology.orchestra.run.smithi139.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T08:43:27.411 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 02:42:74:ed:e4:60 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:43:27.411 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T08:43:27.412 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:43:27.412 INFO:teuthology.orchestra.run.smithi139.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T08:43:27.412 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-19T08:43:27.412 INFO:teuthology.orchestra.run.smithi139.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T08:43:27.412 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:43:27.412 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-19T08:43:27.412 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:43:27.412 INFO:teuthology.orchestra.run.smithi139.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:43:27.412 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:43:27.412 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-19T08:43:27.412 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:43:27.412 INFO:teuthology.orchestra.run.smithi139.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:43:27.412 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:43:27.412 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-19T08:43:27.413 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:43:27.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:43:27.413 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T08:43:27.464 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-09-19T08:43:27.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:43:27.466 DEBUG:teuthology.orchestra.run.smithi139:> route 2024-09-19T08:43:27.518 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2024-09-19T08:43:27.518 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T08:43:27.518 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T08:43:27.518 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T08:43:27.518 INFO:teuthology.orchestra.run.smithi139.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T08:43:27.518 INFO:teuthology.orchestra.run.smithi139.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T08:43:27.518 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T08:43:27.518 INFO:teuthology.orchestra.run.smithi139.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:43:27.518 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:43:27.518 INFO:teuthology.orchestra.run.smithi139.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:43:27.518 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T08:43:27.519 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:43:27.519 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-09-19T08:43:27.519 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T08:43:27.519 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T08:43:27.519 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T08:43:27.519 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-09-19T08:43:27.606 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:43:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:43:27.634 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:43:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:43:27.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:43:27.642 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-09-19T08:43:27.648 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-19T08:43:27.648 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-19T08:43:27.648 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T08:43:27.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:43:27.648 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.0 up 2024-09-19T08:43:27.704 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-19T08:43:27.705 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:43:27.705 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-19T08:43:27.754 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-19T08:43:27.754 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:43:27.754 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-19T08:43:27.801 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:43:27.858 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:43:27.907 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:43:27.907 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:43:27.907 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:43:27.907 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:43:27.908 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:43:27.908 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:43:27.908 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:43:27.908 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:43:27.908 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:43:27.908 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:43:27.908 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:43:27.908 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:43:27.908 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:43:27.908 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:43:27.908 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:43:27.908 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:43:27.908 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:43:27.909 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:43:27.909 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:43:27.909 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:43:27.909 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:43:27.909 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:43:27.909 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:43:27.909 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:43:27.909 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:43:27.909 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:43:27.909 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:43:27.909 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:43:27.909 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:43:27.909 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:43:27.910 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:43:27.910 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:43:27.910 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:43:27.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:43:27.911 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:43:27.962 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T08:43:27.962 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-19T08:43:28.006 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:43:28.020 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:43:28.069 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:43:28.070 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:43:28.070 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:43:28.070 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:43:28.070 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:43:28.070 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:43:28.070 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:43:28.070 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:43:28.070 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:43:28.070 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:43:28.070 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:43:28.070 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:43:28.071 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:43:28.071 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:43:28.071 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:43:28.071 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:43:28.071 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:43:28.071 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:43:28.071 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:43:28.071 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:43:28.071 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:43:28.071 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:43:28.071 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:43:28.071 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:43:28.071 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:43:28.072 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:43:28.072 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:43:28.072 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:43:28.072 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:43:28.072 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:43:28.072 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:43:28.072 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:43:28.072 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:43:28.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:43:28.073 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:43:28.088 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:2024-09-19T08:43:28.088+0000 7f8db5fb3580 -1 init, newargv = 0x564c042641f0 newargc=15 2024-09-19T08:43:28.088 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[36007]: starting ceph client 2024-09-19T08:43:28.096 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[36007]: starting fuse 2024-09-19T08:43:29.194 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:43:29.211 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:43:29.262 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:43:29.262 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:43:29.262 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:43:29.262 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:43:29.262 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:43:29.262 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:43:29.262 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:43:29.263 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:43:29.263 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:43:29.263 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:43:29.263 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:43:29.263 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:43:29.263 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:43:29.263 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:43:29.263 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:43:29.263 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:43:29.263 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:43:29.263 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:43:29.263 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:43:29.263 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:43:29.264 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:43:29.264 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:43:29.264 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:43:29.264 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:43:29.264 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:43:29.264 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:43:29.264 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:43:29.264 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:43:29.264 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:43:29.264 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:43:29.264 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:43:29.264 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:43:29.264 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:43:29.265 INFO:teuthology.orchestra.run.smithi139.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-19T08:43:29.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:43:29.265 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:43:29.313 INFO:teuthology.orchestra.run.smithi139.stdout:45 2024-09-19T08:43:29.314 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-19T08:43:29.314 DEBUG:teuthology.orchestra.run.smithi139:> sudo stdin-killer -- python3 -c ' 2024-09-19T08:43:29.314 DEBUG:teuthology.orchestra.run.smithi139:> import glob 2024-09-19T08:43:29.314 DEBUG:teuthology.orchestra.run.smithi139:> import re 2024-09-19T08:43:29.314 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-09-19T08:43:29.314 DEBUG:teuthology.orchestra.run.smithi139:> import subprocess 2024-09-19T08:43:29.314 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:43:29.314 DEBUG:teuthology.orchestra.run.smithi139:> def _find_admin_socket(client_name): 2024-09-19T08:43:29.314 DEBUG:teuthology.orchestra.run.smithi139:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-19T08:43:29.314 DEBUG:teuthology.orchestra.run.smithi139:> files = glob.glob(asok_path) 2024-09-19T08:43:29.315 DEBUG:teuthology.orchestra.run.smithi139:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-19T08:43:29.315 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:43:29.315 DEBUG:teuthology.orchestra.run.smithi139:> # Given a non-glob path, it better be there 2024-09-19T08:43:29.315 DEBUG:teuthology.orchestra.run.smithi139:> if "*" not in asok_path: 2024-09-19T08:43:29.315 DEBUG:teuthology.orchestra.run.smithi139:> assert(len(files) == 1) 2024-09-19T08:43:29.315 DEBUG:teuthology.orchestra.run.smithi139:> return files[0] 2024-09-19T08:43:29.315 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:43:29.315 DEBUG:teuthology.orchestra.run.smithi139:> for f in files: 2024-09-19T08:43:29.315 DEBUG:teuthology.orchestra.run.smithi139:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T08:43:29.315 DEBUG:teuthology.orchestra.run.smithi139:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T08:43:29.315 DEBUG:teuthology.orchestra.run.smithi139:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T08:43:29.315 DEBUG:teuthology.orchestra.run.smithi139:> contents = proc_f.read() 2024-09-19T08:43:29.315 DEBUG:teuthology.orchestra.run.smithi139:> if mountpoint in contents: 2024-09-19T08:43:29.315 DEBUG:teuthology.orchestra.run.smithi139:> return f 2024-09-19T08:43:29.315 DEBUG:teuthology.orchestra.run.smithi139:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T08:43:29.316 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:43:29.316 DEBUG:teuthology.orchestra.run.smithi139:> print(_find_admin_socket("client.1")) 2024-09-19T08:43:29.316 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-09-19T08:43:29.437 INFO:teuthology.orchestra.run.smithi139.stdout:/var/run/ceph/ceph-client.1.36007.asok 2024-09-19T08:43:29.441 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:43:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:43:29.449 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.36007.asok 2024-09-19T08:43:29.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:43:29.450 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.36007.asok status 2024-09-19T08:43:29.604 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-19T08:43:29.604 INFO:teuthology.orchestra.run.smithi139.stdout: "metadata": { 2024-09-19T08:43:29.604 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T08:43:29.604 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T08:43:29.604 INFO:teuthology.orchestra.run.smithi139.stdout: "entity_id": "1", 2024-09-19T08:43:29.604 INFO:teuthology.orchestra.run.smithi139.stdout: "hostname": "smithi139", 2024-09-19T08:43:29.604 INFO:teuthology.orchestra.run.smithi139.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-19T08:43:29.604 INFO:teuthology.orchestra.run.smithi139.stdout: "pid": "36007", 2024-09-19T08:43:29.604 INFO:teuthology.orchestra.run.smithi139.stdout: "root": "/" 2024-09-19T08:43:29.605 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:43:29.605 INFO:teuthology.orchestra.run.smithi139.stdout: "dentry_count": 0, 2024-09-19T08:43:29.605 INFO:teuthology.orchestra.run.smithi139.stdout: "dentry_pinned_count": 0, 2024-09-19T08:43:29.605 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 5908, 2024-09-19T08:43:29.605 INFO:teuthology.orchestra.run.smithi139.stdout: "inst": { 2024-09-19T08:43:29.605 INFO:teuthology.orchestra.run.smithi139.stdout: "name": { 2024-09-19T08:43:29.605 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "client", 2024-09-19T08:43:29.605 INFO:teuthology.orchestra.run.smithi139.stdout: "num": 5908 2024-09-19T08:43:29.605 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:43:29.605 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": { 2024-09-19T08:43:29.605 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-19T08:43:29.605 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:43:29.605 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4174028744 2024-09-19T08:43:29.606 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-19T08:43:29.606 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:43:29.606 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": { 2024-09-19T08:43:29.606 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-19T08:43:29.606 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:43:29.606 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 4174028744 2024-09-19T08:43:29.606 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:43:29.606 INFO:teuthology.orchestra.run.smithi139.stdout: "inst_str": "client.5908 192.168.0.1:0/4174028744", 2024-09-19T08:43:29.606 INFO:teuthology.orchestra.run.smithi139.stdout: "addr_str": "192.168.0.1:0/4174028744", 2024-09-19T08:43:29.606 INFO:teuthology.orchestra.run.smithi139.stdout: "inode_count": 1, 2024-09-19T08:43:29.606 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_epoch": 47, 2024-09-19T08:43:29.606 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_epoch": 83, 2024-09-19T08:43:29.606 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_epoch_barrier": 0, 2024-09-19T08:43:29.606 INFO:teuthology.orchestra.run.smithi139.stdout: "blocklisted": false, 2024-09-19T08:43:29.607 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs" 2024-09-19T08:43:29.607 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-19T08:43:29.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:43:29.618 DEBUG:teuthology.orchestra.run.smithi139:> stat --file-system '--printf=%T 2024-09-19T08:43:29.618 DEBUG:teuthology.orchestra.run.smithi139:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-19T08:43:29.624 INFO:teuthology.orchestra.run.smithi139.stdout:fuseblk 2024-09-19T08:43:29.624 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-19T08:43:29.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:43:29.624 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-19T08:43:29.682 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-19T08:43:29.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.817+0000 7efe3e2ba640 1 Processor -- start 2024-09-19T08:43:29.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.817+0000 7efe3e2ba640 1 -- start start 2024-09-19T08:43:29.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.817+0000 7efe3e2ba640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe3815c290 0x7efe380cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:29.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.817+0000 7efe3e2ba640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7efe380cf700 0x7efe3815ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:29.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.817+0000 7efe3e2ba640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe381608c0 con 0x7efe3806d4c0 2024-09-19T08:43:29.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.817+0000 7efe3e2ba640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7efe38160e80 con 0x7efe380cf700 2024-09-19T08:43:29.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.817+0000 7efe3e2ba640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7efe380cd510 con 0x7efe3815c290 2024-09-19T08:43:29.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.817+0000 7efe377fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7efe380cf700 0x7efe3815ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:29.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.817+0000 7efe377fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7efe380cf700 0x7efe3815ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47894/0 (socket says 172.21.15.98:47894) 2024-09-19T08:43:29.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.817+0000 7efe377fe640 1 -- 172.21.15.98:0/2947391554 learned_addr learned my addr 172.21.15.98:0/2947391554 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:29.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.817+0000 7efe37fff640 1 --2- 172.21.15.98:0/2947391554 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe3815c290 0x7efe380cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:29.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.817+0000 7efe377fe640 1 -- 172.21.15.98:0/2947391554 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe3815c290 msgr2=0x7efe380cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:29.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.817+0000 7efe377fe640 1 --2- 172.21.15.98:0/2947391554 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe3815c290 0x7efe380cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:29.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.817+0000 7efe377fe640 1 -- 172.21.15.98:0/2947391554 >> v1:172.21.15.139:6789/0 conn(0x7efe3806d4c0 legacy=0x7efe380b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:29.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.817+0000 7efe377fe640 1 -- 172.21.15.98:0/2947391554 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe38160ba0 con 0x7efe380cf700 2024-09-19T08:43:29.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.817+0000 7efe37fff640 1 --2- 172.21.15.98:0/2947391554 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe3815c290 0x7efe380cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:29.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.821+0000 7efe377fe640 1 --2- 172.21.15.98:0/2947391554 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe380cf700 0x7efe3815ff50 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7efe380cd690 tx=0x7efe2c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0e464dc01dbe25 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:29.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.821+0000 7efe36ffd640 1 -- 172.21.15.98:0/2947391554 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2c0b0070 con 0x7efe380cf700 2024-09-19T08:43:29.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.821+0000 7efe36ffd640 1 -- 172.21.15.98:0/2947391554 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe2c0ab600 con 0x7efe380cf700 2024-09-19T08:43:29.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.821+0000 7efe36ffd640 1 -- 172.21.15.98:0/2947391554 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2c0ab9d0 con 0x7efe380cf700 2024-09-19T08:43:29.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.821+0000 7efe3e2ba640 1 -- 172.21.15.98:0/2947391554 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe380cf700 msgr2=0x7efe3815ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:29.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.821+0000 7efe3e2ba640 1 --2- 172.21.15.98:0/2947391554 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe380cf700 0x7efe3815ff50 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7efe380cd690 tx=0x7efe2c01c500 comp rx=0 tx=0).stop 2024-09-19T08:43:29.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.821+0000 7efe3e2ba640 1 -- 172.21.15.98:0/2947391554 shutdown_connections 2024-09-19T08:43:29.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.821+0000 7efe3e2ba640 1 --2- 172.21.15.98:0/2947391554 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe380cf700 0x7efe3815ff50 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:29.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.821+0000 7efe3e2ba640 1 --2- 172.21.15.98:0/2947391554 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe3815c290 0x7efe380cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:29.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.821+0000 7efe3e2ba640 1 -- 172.21.15.98:0/2947391554 >> 172.21.15.98:0/2947391554 conn(0x7efe3806c090 msgr2=0x7efe3806c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:29.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.821+0000 7efe3e2ba640 1 -- 172.21.15.98:0/2947391554 shutdown_connections 2024-09-19T08:43:29.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.821+0000 7efe3e2ba640 1 -- 172.21.15.98:0/2947391554 wait complete. 2024-09-19T08:43:29.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.825+0000 7efe3e2ba640 1 Processor -- start 2024-09-19T08:43:29.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.825+0000 7efe3e2ba640 1 -- start start 2024-09-19T08:43:29.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.825+0000 7efe3e2ba640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe3806d4c0 0x7efe380b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:29.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.825+0000 7efe3e2ba640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe380cf700 0x7efe38158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:29.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.825+0000 7efe3e2ba640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe3815c290 0x7efe381623c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:29.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.825+0000 7efe3e2ba640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efe3815c7f0 con 0x7efe3806d4c0 2024-09-19T08:43:29.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.825+0000 7efe3e2ba640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efe3815c670 con 0x7efe380cf700 2024-09-19T08:43:29.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.825+0000 7efe3e2ba640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7efe38160490 con 0x7efe3815c290 2024-09-19T08:43:29.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.825+0000 7efe37fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe3806d4c0 0x7efe380b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:29.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.825+0000 7efe37fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe3806d4c0 0x7efe380b1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47898/0 (socket says 172.21.15.98:47898) 2024-09-19T08:43:29.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.825+0000 7efe37fff640 1 -- 172.21.15.98:0/3137494750 learned_addr learned my addr 172.21.15.98:0/3137494750 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:29.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.825+0000 7efe377fe640 1 --2- 172.21.15.98:0/3137494750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe380cf700 0x7efe38158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:29.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.825+0000 7efe3c857640 1 --2- 172.21.15.98:0/3137494750 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe3815c290 0x7efe381623c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:29.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.825+0000 7efe37fff640 1 -- 172.21.15.98:0/3137494750 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe3815c290 msgr2=0x7efe381623c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:29.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.825+0000 7efe37fff640 1 --2- 172.21.15.98:0/3137494750 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe3815c290 0x7efe381623c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:29.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.825+0000 7efe37fff640 1 -- 172.21.15.98:0/3137494750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe380cf700 msgr2=0x7efe38158c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:29.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.825+0000 7efe37fff640 1 --2- 172.21.15.98:0/3137494750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe380cf700 0x7efe38158c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:29.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.825+0000 7efe37fff640 1 -- 172.21.15.98:0/3137494750 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe380ae750 con 0x7efe3806d4c0 2024-09-19T08:43:29.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.825+0000 7efe377fe640 1 --2- 172.21.15.98:0/3137494750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe380cf700 0x7efe38158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:29.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.825+0000 7efe3c857640 1 --2- 172.21.15.98:0/3137494750 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe3815c290 0x7efe381623c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:29.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.825+0000 7efe37fff640 1 --2- 172.21.15.98:0/3137494750 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe3806d4c0 0x7efe380b1b20 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7efe2800c7f0 tx=0x7efe2800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:29.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.825+0000 7efe34ff9640 1 -- 172.21.15.98:0/3137494750 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe28015070 con 0x7efe3806d4c0 2024-09-19T08:43:29.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.825+0000 7efe34ff9640 1 -- 172.21.15.98:0/3137494750 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe280077c0 con 0x7efe3806d4c0 2024-09-19T08:43:29.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.825+0000 7efe3e2ba640 1 -- 172.21.15.98:0/3137494750 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe380aea30 con 0x7efe3806d4c0 2024-09-19T08:43:29.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.825+0000 7efe34ff9640 1 -- 172.21.15.98:0/3137494750 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe28007bb0 con 0x7efe3806d4c0 2024-09-19T08:43:29.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.825+0000 7efe3e2ba640 1 -- 172.21.15.98:0/3137494750 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe380af2d0 con 0x7efe3806d4c0 2024-09-19T08:43:29.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.825+0000 7efe3e2ba640 1 -- 172.21.15.98:0/3137494750 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe380aee20 con 0x7efe3806d4c0 2024-09-19T08:43:29.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.829+0000 7efe34ff9640 1 -- 172.21.15.98:0/3137494750 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7efe28018070 con 0x7efe3806d4c0 2024-09-19T08:43:29.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.829+0000 7efe34ff9640 1 --2- 172.21.15.98:0/3137494750 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7efe240679e0 0x7efe24069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:29.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.829+0000 7efe34ff9640 1 -- 172.21.15.98:0/3137494750 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7efe28085fb0 con 0x7efe3806d4c0 2024-09-19T08:43:29.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.829+0000 7efe377fe640 1 --2- 172.21.15.98:0/3137494750 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7efe240679e0 0x7efe24069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:29.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.829+0000 7efe377fe640 1 --2- 172.21.15.98:0/3137494750 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7efe240679e0 0x7efe24069ea0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efe2c0ad000 tx=0x7efe2c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:29.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:29.833+0000 7efe34ff9640 1 -- 172.21.15.98:0/3137494750 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7efe28056960 con 0x7efe3806d4c0 2024-09-19T08:43:30.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:30.017+0000 7efe3e2ba640 1 -- 172.21.15.98:0/3137494750 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7efe3815ff50 con 0x7efe3806d4c0 2024-09-19T08:43:30.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:30.949+0000 7efe34ff9640 1 -- 172.21.15.98:0/3137494750 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v48) ==== 136+0+0 (secure 0 0 0) 0x7efe28056260 con 0x7efe3806d4c0 2024-09-19T08:43:30.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:30.953+0000 7efe3e2ba640 1 -- 172.21.15.98:0/3137494750 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7efe240679e0 msgr2=0x7efe24069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:30.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:30.953+0000 7efe3e2ba640 1 --2- 172.21.15.98:0/3137494750 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7efe240679e0 0x7efe24069ea0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efe2c0ad000 tx=0x7efe2c008040 comp rx=0 tx=0).stop 2024-09-19T08:43:30.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:30.953+0000 7efe3e2ba640 1 -- 172.21.15.98:0/3137494750 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe3806d4c0 msgr2=0x7efe380b1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:30.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:30.953+0000 7efe3e2ba640 1 --2- 172.21.15.98:0/3137494750 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe3806d4c0 0x7efe380b1b20 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7efe2800c7f0 tx=0x7efe2800ccb0 comp rx=0 tx=0).stop 2024-09-19T08:43:30.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:30.953+0000 7efe3e2ba640 1 -- 172.21.15.98:0/3137494750 shutdown_connections 2024-09-19T08:43:30.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:30.953+0000 7efe3e2ba640 1 --2- 172.21.15.98:0/3137494750 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7efe240679e0 0x7efe24069ea0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:30.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:30.953+0000 7efe3e2ba640 1 --2- 172.21.15.98:0/3137494750 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe3815c290 0x7efe381623c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:30.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:30.953+0000 7efe3e2ba640 1 --2- 172.21.15.98:0/3137494750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe380cf700 0x7efe38158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:30.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:30.953+0000 7efe3e2ba640 1 --2- 172.21.15.98:0/3137494750 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe3806d4c0 0x7efe380b1b20 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:30.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:30.953+0000 7efe3e2ba640 1 -- 172.21.15.98:0/3137494750 >> 172.21.15.98:0/3137494750 conn(0x7efe3806c090 msgr2=0x7efe3815f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:30.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:30.953+0000 7efe3e2ba640 1 -- 172.21.15.98:0/3137494750 shutdown_connections 2024-09-19T08:43:30.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:30.953+0000 7efe3e2ba640 1 -- 172.21.15.98:0/3137494750 wait complete. 2024-09-19T08:43:30.972 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-19T08:43:31.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.093+0000 7f2361177640 1 Processor -- start 2024-09-19T08:43:31.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.093+0000 7f2361177640 1 -- start start 2024-09-19T08:43:31.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.093+0000 7f2361177640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f235c0b7f80 0x7f235c15f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:31.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.093+0000 7f2361177640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f235c068440 con 0x7f235c0b9970 2024-09-19T08:43:31.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.093+0000 7f2361177640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f235c162a90 con 0x7f235c0d4820 2024-09-19T08:43:31.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.093+0000 7f2361177640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f235c163050 con 0x7f235c0b7f80 2024-09-19T08:43:31.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.093+0000 7f235b59e640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f235c0b9970 0x7f235c0b9e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:45992/0 (socket says 172.21.15.98:45992) 2024-09-19T08:43:31.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.093+0000 7f235b59e640 1 -- 172.21.15.98:0/2038056222 learned_addr learned my addr 172.21.15.98:0/2038056222 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:31.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.097+0000 7f235ad9d640 1 --2- 172.21.15.98:0/2038056222 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f235c0b7f80 0x7f235c15f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:31.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.097+0000 7f235a59c640 1 -- 172.21.15.98:0/2038056222 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2647644379 0 0) 0x7f235c068440 con 0x7f235c0b9970 2024-09-19T08:43:31.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.097+0000 7f235a59c640 1 -- 172.21.15.98:0/2038056222 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2348003530 con 0x7f235c0b9970 2024-09-19T08:43:31.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.097+0000 7f235a59c640 1 -- 172.21.15.98:0/2038056222 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4037089930 0 0) 0x7f2348003530 con 0x7f235c0b9970 2024-09-19T08:43:31.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.097+0000 7f235a59c640 1 -- 172.21.15.98:0/2038056222 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f235c0b7f80 msgr2=0x7f235c15f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:31.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.097+0000 7f235a59c640 1 --2- 172.21.15.98:0/2038056222 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f235c0b7f80 0x7f235c15f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:31.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.097+0000 7f235a59c640 1 -- 172.21.15.98:0/2038056222 >> v1:172.21.15.139:6789/0 conn(0x7f235c0d4820 legacy=0x7f235c1621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:31.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.097+0000 7f235a59c640 1 -- 172.21.15.98:0/2038056222 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f235c162d70 con 0x7f235c0b9970 2024-09-19T08:43:31.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.097+0000 7f235a59c640 1 -- 172.21.15.98:0/2038056222 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f2354001a00 con 0x7f235c0b9970 2024-09-19T08:43:31.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.097+0000 7f235ad9d640 1 --2- 172.21.15.98:0/2038056222 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f235c0b7f80 0x7f235c15f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:43:31.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.097+0000 7f235a59c640 1 -- 172.21.15.98:0/2038056222 >> v1:172.21.15.98:6789/0 conn(0x7f235c0b9970 legacy=0x7f235c0b9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:31.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.097+0000 7f235a59c640 1 --2- 172.21.15.98:0/2038056222 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f235c0b7f80 0x7f23480039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:31.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.097+0000 7f235a59c640 1 --2- 172.21.15.98:0/2038056222 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2348003fe0 0x7f2348004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:31.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.097+0000 7f235a59c640 1 --2- 172.21.15.98:0/2038056222 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2348004d50 0x7f23480051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:31.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.097+0000 7f235a59c640 1 -- 172.21.15.98:0/2038056222 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f235c163050 con 0x7f2348003fe0 2024-09-19T08:43:31.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.097+0000 7f235a59c640 1 -- 172.21.15.98:0/2038056222 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f234800bb30 con 0x7f2348004d50 2024-09-19T08:43:31.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.097+0000 7f235a59c640 1 -- 172.21.15.98:0/2038056222 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f234800bd30 con 0x7f235c0b7f80 2024-09-19T08:43:31.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.097+0000 7f235259c640 1 --2- 172.21.15.98:0/2038056222 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2348003fe0 0x7f2348004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:31.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.097+0000 7f235ad9d640 1 --2- 172.21.15.98:0/2038056222 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f235c0b7f80 0x7f23480039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:31.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.097+0000 7f235b59e640 1 --2- 172.21.15.98:0/2038056222 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2348004d50 0x7f23480051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:31.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.097+0000 7f235259c640 1 -- 172.21.15.98:0/2038056222 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f235c0b7f80 msgr2=0x7f23480039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:31.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.097+0000 7f235259c640 1 --2- 172.21.15.98:0/2038056222 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f235c0b7f80 0x7f23480039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:31.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.097+0000 7f235259c640 1 -- 172.21.15.98:0/2038056222 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2348004d50 msgr2=0x7f23480051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:31.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.097+0000 7f235259c640 1 --2- 172.21.15.98:0/2038056222 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2348004d50 0x7f23480051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:31.100 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.097+0000 7f235259c640 1 -- 172.21.15.98:0/2038056222 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2348003530 con 0x7f2348003fe0 2024-09-19T08:43:31.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.097+0000 7f235259c640 1 --2- 172.21.15.98:0/2038056222 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2348003fe0 0x7f2348004810 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f2344001750 tx=0x7f234401c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557203360a9aedda server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:31.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.097+0000 7f235a59c640 1 -- 172.21.15.98:0/2038056222 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23440013a0 con 0x7f2348003fe0 2024-09-19T08:43:31.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.097+0000 7f235a59c640 1 -- 172.21.15.98:0/2038056222 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23440052f0 con 0x7f2348003fe0 2024-09-19T08:43:31.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.097+0000 7f235a59c640 1 -- 172.21.15.98:0/2038056222 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2344003840 con 0x7f2348003fe0 2024-09-19T08:43:31.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.097+0000 7f2361177640 1 -- 172.21.15.98:0/2038056222 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2348003fe0 msgr2=0x7f2348004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:31.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.097+0000 7f2361177640 1 --2- 172.21.15.98:0/2038056222 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2348003fe0 0x7f2348004810 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f2344001750 tx=0x7f234401c320 comp rx=0 tx=0).stop 2024-09-19T08:43:31.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.097+0000 7f235b59e640 1 -- 172.21.15.98:0/2038056222 reap_dead start 2024-09-19T08:43:31.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.097+0000 7f2361177640 1 -- 172.21.15.98:0/2038056222 shutdown_connections 2024-09-19T08:43:31.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.097+0000 7f2361177640 1 -- 172.21.15.98:0/2038056222 >> 172.21.15.98:0/2038056222 conn(0x7f235c074e40 msgr2=0x7f235c075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:31.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.101+0000 7f2361177640 1 -- 172.21.15.98:0/2038056222 shutdown_connections 2024-09-19T08:43:31.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.101+0000 7f2361177640 1 -- 172.21.15.98:0/2038056222 wait complete. 2024-09-19T08:43:31.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.101+0000 7f2361177640 1 Processor -- start 2024-09-19T08:43:31.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.101+0000 7f2361177640 1 -- start start 2024-09-19T08:43:31.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.101+0000 7f2361177640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2348003fe0 0x7f235c136e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:31.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.101+0000 7f2361177640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f235c140c60 0x7f235c0cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:31.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.101+0000 7f2361177640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f235c144220 0x7f235c1446a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:31.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.101+0000 7f2361177640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f235c0b84e0 con 0x7f235c140c60 2024-09-19T08:43:31.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.101+0000 7f2361177640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f235c0b8360 con 0x7f2348003fe0 2024-09-19T08:43:31.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.101+0000 7f2361177640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f235c15f560 con 0x7f235c144220 2024-09-19T08:43:31.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.101+0000 7f235259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f235c140c60 0x7f235c0cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:31.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.101+0000 7f235259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f235c140c60 0x7f235c0cd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47918/0 (socket says 172.21.15.98:47918) 2024-09-19T08:43:31.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.101+0000 7f235259c640 1 -- 172.21.15.98:0/1497714495 learned_addr learned my addr 172.21.15.98:0/1497714495 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:31.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.101+0000 7f235ad9d640 1 --2- 172.21.15.98:0/1497714495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2348003fe0 0x7f235c136e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:31.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.101+0000 7f235b59e640 1 --2- 172.21.15.98:0/1497714495 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f235c144220 0x7f235c1446a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:31.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.101+0000 7f235259c640 1 -- 172.21.15.98:0/1497714495 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f235c144220 msgr2=0x7f235c1446a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:31.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.101+0000 7f235259c640 1 --2- 172.21.15.98:0/1497714495 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f235c144220 0x7f235c1446a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:31.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.101+0000 7f235259c640 1 -- 172.21.15.98:0/1497714495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2348003fe0 msgr2=0x7f235c136e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:31.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.101+0000 7f235259c640 1 --2- 172.21.15.98:0/1497714495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2348003fe0 0x7f235c136e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:31.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.101+0000 7f235259c640 1 -- 172.21.15.98:0/1497714495 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f235c0cda30 con 0x7f235c140c60 2024-09-19T08:43:31.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.101+0000 7f235259c640 1 --2- 172.21.15.98:0/1497714495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f235c140c60 0x7f235c0cd2a0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f2344005040 tx=0x7f2348005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:31.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.101+0000 7f2353fff640 1 -- 172.21.15.98:0/1497714495 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23440054a0 con 0x7f235c140c60 2024-09-19T08:43:31.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.101+0000 7f2353fff640 1 -- 172.21.15.98:0/1497714495 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23440bb070 con 0x7f235c140c60 2024-09-19T08:43:31.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.101+0000 7f2353fff640 1 -- 172.21.15.98:0/1497714495 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23440069e0 con 0x7f235c140c60 2024-09-19T08:43:31.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.101+0000 7f2361177640 1 -- 172.21.15.98:0/1497714495 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f235c0cdd10 con 0x7f235c140c60 2024-09-19T08:43:31.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.101+0000 7f2361177640 1 -- 172.21.15.98:0/1497714495 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f235c0ce2d0 con 0x7f235c140c60 2024-09-19T08:43:31.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.101+0000 7f2361177640 1 -- 172.21.15.98:0/1497714495 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f235c06cca0 con 0x7f235c140c60 2024-09-19T08:43:31.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.105+0000 7f2353fff640 1 -- 172.21.15.98:0/1497714495 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f23440b4040 con 0x7f235c140c60 2024-09-19T08:43:31.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.105+0000 7f2353fff640 1 --2- 172.21.15.98:0/1497714495 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f233c067a10 0x7f233c069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:31.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.105+0000 7f2353fff640 1 -- 172.21.15.98:0/1497714495 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f2344120460 con 0x7f235c140c60 2024-09-19T08:43:31.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.105+0000 7f235ad9d640 1 --2- 172.21.15.98:0/1497714495 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f233c067a10 0x7f233c069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:31.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.105+0000 7f235ad9d640 1 --2- 172.21.15.98:0/1497714495 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f233c067a10 0x7f233c069ed0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f234c00e880 tx=0x7f234c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:31.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.109+0000 7f2353fff640 1 -- 172.21.15.98:0/1497714495 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2344016030 con 0x7f235c140c60 2024-09-19T08:43:31.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.293+0000 7f2361177640 1 -- 172.21.15.98:0/1497714495 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f235c0cd7e0 con 0x7f233c067a10 2024-09-19T08:43:31.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.293+0000 7f2353fff640 1 -- 172.21.15.98:0/1497714495 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f235c0cd7e0 con 0x7f233c067a10 2024-09-19T08:43:31.296 INFO:teuthology.orchestra.run.smithi098.stdout:[ 2024-09-19T08:43:31.296 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:31.296 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "cephfs" 2024-09-19T08:43:31.296 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:31.296 INFO:teuthology.orchestra.run.smithi098.stdout:] 2024-09-19T08:43:31.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.297+0000 7f2361177640 1 -- 172.21.15.98:0/1497714495 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f233c067a10 msgr2=0x7f233c069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:31.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.297+0000 7f2361177640 1 --2- 172.21.15.98:0/1497714495 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f233c067a10 0x7f233c069ed0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f234c00e880 tx=0x7f234c008040 comp rx=0 tx=0).stop 2024-09-19T08:43:31.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.297+0000 7f2361177640 1 -- 172.21.15.98:0/1497714495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f235c140c60 msgr2=0x7f235c0cd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:31.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.297+0000 7f2361177640 1 --2- 172.21.15.98:0/1497714495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f235c140c60 0x7f235c0cd2a0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f2344005040 tx=0x7f2348005840 comp rx=0 tx=0).stop 2024-09-19T08:43:31.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.297+0000 7f2361177640 1 -- 172.21.15.98:0/1497714495 shutdown_connections 2024-09-19T08:43:31.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.297+0000 7f2361177640 1 --2- 172.21.15.98:0/1497714495 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f233c067a10 0x7f233c069ed0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:31.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.297+0000 7f2361177640 1 --2- 172.21.15.98:0/1497714495 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f235c144220 0x7f235c1446a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:31.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.297+0000 7f2361177640 1 --2- 172.21.15.98:0/1497714495 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f235c140c60 0x7f235c0cd2a0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:31.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.297+0000 7f2361177640 1 --2- 172.21.15.98:0/1497714495 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2348003fe0 0x7f235c136e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:31.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.297+0000 7f2361177640 1 -- 172.21.15.98:0/1497714495 >> 172.21.15.98:0/1497714495 conn(0x7f235c074e40 msgr2=0x7f235c161820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:31.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.297+0000 7f2361177640 1 -- 172.21.15.98:0/1497714495 shutdown_connections 2024-09-19T08:43:31.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.297+0000 7f2361177640 1 -- 172.21.15.98:0/1497714495 wait complete. 2024-09-19T08:43:31.315 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T08:43:31.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.437+0000 7f4eab425640 1 Processor -- start 2024-09-19T08:43:31.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.437+0000 7f4eab425640 1 -- start start 2024-09-19T08:43:31.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.437+0000 7f4eab425640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4ea41415a0 0x7f4ea40d20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:31.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.437+0000 7f4eab425640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4ea40d25f0 0x7f4ea40d3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:31.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.437+0000 7f4eab425640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ea415eae0 con 0x7f4ea40d38a0 2024-09-19T08:43:31.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.437+0000 7f4eab425640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f4ea40d2a30 con 0x7f4ea40d25f0 2024-09-19T08:43:31.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.437+0000 7f4eab425640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4ea415edc0 con 0x7f4ea41415a0 2024-09-19T08:43:31.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.437+0000 7f4ea89c0640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4ea40d25f0 0x7f4ea40d3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:31.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.437+0000 7f4ea99c2640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f4ea40d38a0 0x7f4ea4161fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:46000/0 (socket says 172.21.15.98:46000) 2024-09-19T08:43:31.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.437+0000 7f4ea99c2640 1 -- 172.21.15.98:0/3795485725 learned_addr learned my addr 172.21.15.98:0/3795485725 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:31.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.437+0000 7f4ea91c1640 1 --2- 172.21.15.98:0/3795485725 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4ea41415a0 0x7f4ea40d20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:31.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.437+0000 7f4ea89c0640 1 -- 172.21.15.98:0/3795485725 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4ea41415a0 msgr2=0x7f4ea40d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:31.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.437+0000 7f4ea89c0640 1 --2- 172.21.15.98:0/3795485725 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4ea41415a0 0x7f4ea40d20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:31.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.437+0000 7f4ea89c0640 1 -- 172.21.15.98:0/3795485725 >> v1:172.21.15.98:6789/0 conn(0x7f4ea40d38a0 legacy=0x7f4ea4161fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:31.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.437+0000 7f4ea89c0640 1 -- 172.21.15.98:0/3795485725 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea415f0a0 con 0x7f4ea40d25f0 2024-09-19T08:43:31.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.437+0000 7f4e9bfff640 1 -- 172.21.15.98:0/3795485725 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1831667622 0 0) 0x7f4ea415eae0 con 0x7f4ea40d38a0 2024-09-19T08:43:31.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.441+0000 7f4ea89c0640 1 --2- 172.21.15.98:0/3795485725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ea40d25f0 0x7f4ea40d3360 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f4e940097c0 tx=0x7f4e9401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18509e0e482f7649 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:31.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.441+0000 7f4e9bfff640 1 -- 172.21.15.98:0/3795485725 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e940b0070 con 0x7f4ea40d25f0 2024-09-19T08:43:31.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.441+0000 7f4e9bfff640 1 -- 172.21.15.98:0/3795485725 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e940ab600 con 0x7f4ea40d25f0 2024-09-19T08:43:31.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.441+0000 7f4e9bfff640 1 -- 172.21.15.98:0/3795485725 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e940ab9d0 con 0x7f4ea40d25f0 2024-09-19T08:43:31.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.441+0000 7f4eab425640 1 -- 172.21.15.98:0/3795485725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ea40d25f0 msgr2=0x7f4ea40d3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:31.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.441+0000 7f4eab425640 1 --2- 172.21.15.98:0/3795485725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ea40d25f0 0x7f4ea40d3360 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f4e940097c0 tx=0x7f4e9401c470 comp rx=0 tx=0).stop 2024-09-19T08:43:31.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.441+0000 7f4eab425640 1 -- 172.21.15.98:0/3795485725 shutdown_connections 2024-09-19T08:43:31.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.441+0000 7f4eab425640 1 --2- 172.21.15.98:0/3795485725 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ea40d25f0 0x7f4ea40d3360 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:31.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.441+0000 7f4eab425640 1 --2- 172.21.15.98:0/3795485725 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4ea41415a0 0x7f4ea40d20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:31.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.441+0000 7f4eab425640 1 -- 172.21.15.98:0/3795485725 >> 172.21.15.98:0/3795485725 conn(0x7f4ea4074c70 msgr2=0x7f4ea4075080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:31.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.441+0000 7f4eab425640 1 -- 172.21.15.98:0/3795485725 shutdown_connections 2024-09-19T08:43:31.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.441+0000 7f4eab425640 1 -- 172.21.15.98:0/3795485725 wait complete. 2024-09-19T08:43:31.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.441+0000 7f4eab425640 1 Processor -- start 2024-09-19T08:43:31.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.441+0000 7f4eab425640 1 -- start start 2024-09-19T08:43:31.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.441+0000 7f4eab425640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ea40d25f0 0x7f4ea40d12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:31.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.441+0000 7f4eab425640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4ea40d38a0 0x7f4ea40ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:31.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.441+0000 7f4eab425640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ea41415a0 0x7f4ea40cee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:31.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.441+0000 7f4eab425640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea40d42a0 con 0x7f4ea40d25f0 2024-09-19T08:43:31.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.441+0000 7f4eab425640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea40d4120 con 0x7f4ea41415a0 2024-09-19T08:43:31.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.441+0000 7f4eab425640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4ea40d3d10 con 0x7f4ea40d38a0 2024-09-19T08:43:31.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.441+0000 7f4ea91c1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ea40d25f0 0x7f4ea40d12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:31.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.441+0000 7f4ea91c1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ea40d25f0 0x7f4ea40d12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47948/0 (socket says 172.21.15.98:47948) 2024-09-19T08:43:31.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.441+0000 7f4ea91c1640 1 -- 172.21.15.98:0/38443930 learned_addr learned my addr 172.21.15.98:0/38443930 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:31.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.441+0000 7f4ea99c2640 1 --2- 172.21.15.98:0/38443930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ea41415a0 0x7f4ea40cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:31.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.441+0000 7f4ea89c0640 1 --2- 172.21.15.98:0/38443930 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4ea40d38a0 0x7f4ea40ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:31.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.441+0000 7f4ea91c1640 1 -- 172.21.15.98:0/38443930 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4ea40d38a0 msgr2=0x7f4ea40ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:31.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.441+0000 7f4ea91c1640 1 --2- 172.21.15.98:0/38443930 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4ea40d38a0 0x7f4ea40ca170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:31.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.441+0000 7f4ea91c1640 1 -- 172.21.15.98:0/38443930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ea41415a0 msgr2=0x7f4ea40cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:31.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.441+0000 7f4ea91c1640 1 --2- 172.21.15.98:0/38443930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ea41415a0 0x7f4ea40cee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:31.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.445+0000 7f4ea91c1640 1 -- 172.21.15.98:0/38443930 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea40ca6b0 con 0x7f4ea40d25f0 2024-09-19T08:43:31.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.445+0000 7f4ea99c2640 1 --2- 172.21.15.98:0/38443930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ea41415a0 0x7f4ea40cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:31.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.445+0000 7f4ea91c1640 1 --2- 172.21.15.98:0/38443930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ea40d25f0 0x7f4ea40d12d0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000d5f0 tx=0x7f4ea000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:31.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.445+0000 7f4e99ffb640 1 -- 172.21.15.98:0/38443930 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea00125f0 con 0x7f4ea40d25f0 2024-09-19T08:43:31.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.445+0000 7f4e99ffb640 1 -- 172.21.15.98:0/38443930 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ea0019070 con 0x7f4ea40d25f0 2024-09-19T08:43:31.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.445+0000 7f4eab425640 1 -- 172.21.15.98:0/38443930 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ea40d18d0 con 0x7f4ea40d25f0 2024-09-19T08:43:31.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.445+0000 7f4e99ffb640 1 -- 172.21.15.98:0/38443930 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea00129b0 con 0x7f4ea40d25f0 2024-09-19T08:43:31.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.445+0000 7f4eab425640 1 -- 172.21.15.98:0/38443930 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ea40cfc90 con 0x7f4ea40d25f0 2024-09-19T08:43:31.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.445+0000 7f4eab425640 1 -- 172.21.15.98:0/38443930 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ea40d3360 con 0x7f4ea40d25f0 2024-09-19T08:43:31.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.445+0000 7f4e99ffb640 1 -- 172.21.15.98:0/38443930 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f4ea0012b50 con 0x7f4ea40d25f0 2024-09-19T08:43:31.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.445+0000 7f4e99ffb640 1 --2- 172.21.15.98:0/38443930 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4e880679e0 0x7f4e88069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:31.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.445+0000 7f4e99ffb640 1 -- 172.21.15.98:0/38443930 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f4ea0086190 con 0x7f4ea40d25f0 2024-09-19T08:43:31.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.445+0000 7f4ea89c0640 1 --2- 172.21.15.98:0/38443930 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4e880679e0 0x7f4e88069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:31.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.445+0000 7f4ea89c0640 1 --2- 172.21.15.98:0/38443930 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4e880679e0 0x7f4e88069ea0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4e940ad000 tx=0x7f4e940a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:31.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.449+0000 7f4e99ffb640 1 -- 172.21.15.98:0/38443930 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f4ea0056c50 con 0x7f4ea40d25f0 2024-09-19T08:43:31.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.613+0000 7f4eab425640 1 -- 172.21.15.98:0/38443930 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f4ea4168c70 con 0x7f4ea40d25f0 2024-09-19T08:43:31.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.613+0000 7f4e99ffb640 1 -- 172.21.15.98:0/38443930 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f4ea0056550 con 0x7f4ea40d25f0 2024-09-19T08:43:31.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.613+0000 7f4eab425640 1 -- 172.21.15.98:0/38443930 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4e880679e0 msgr2=0x7f4e88069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:31.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.613+0000 7f4eab425640 1 --2- 172.21.15.98:0/38443930 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4e880679e0 0x7f4e88069ea0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4e940ad000 tx=0x7f4e940a6000 comp rx=0 tx=0).stop 2024-09-19T08:43:31.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.613+0000 7f4eab425640 1 -- 172.21.15.98:0/38443930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ea40d25f0 msgr2=0x7f4ea40d12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:31.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.613+0000 7f4eab425640 1 --2- 172.21.15.98:0/38443930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ea40d25f0 0x7f4ea40d12d0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000d5f0 tx=0x7f4ea000dab0 comp rx=0 tx=0).stop 2024-09-19T08:43:31.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.613+0000 7f4eab425640 1 -- 172.21.15.98:0/38443930 shutdown_connections 2024-09-19T08:43:31.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.613+0000 7f4eab425640 1 --2- 172.21.15.98:0/38443930 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4e880679e0 0x7f4e88069ea0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:31.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.613+0000 7f4eab425640 1 --2- 172.21.15.98:0/38443930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4ea41415a0 0x7f4ea40cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:31.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.613+0000 7f4eab425640 1 --2- 172.21.15.98:0/38443930 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4ea40d38a0 0x7f4ea40ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:31.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.613+0000 7f4eab425640 1 --2- 172.21.15.98:0/38443930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4ea40d25f0 0x7f4ea40d12d0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:31.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.613+0000 7f4eab425640 1 -- 172.21.15.98:0/38443930 >> 172.21.15.98:0/38443930 conn(0x7f4ea4074c70 msgr2=0x7f4ea4160920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:31.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.613+0000 7f4eab425640 1 -- 172.21.15.98:0/38443930 shutdown_connections 2024-09-19T08:43:31.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.613+0000 7f4eab425640 1 -- 172.21.15.98:0/38443930 wait complete. 2024-09-19T08:43:31.632 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create 'abcd@#' 2024-09-19T08:43:31.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.749+0000 7fb3be63c640 1 Processor -- start 2024-09-19T08:43:31.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.749+0000 7fb3be63c640 1 -- start start 2024-09-19T08:43:31.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.749+0000 7fb3be63c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb3b80729f0 0x7fb3b806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:31.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.749+0000 7fb3be63c640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3b8072ea0 con 0x7fb3b806c1d0 2024-09-19T08:43:31.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.749+0000 7fb3be63c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3b8073180 con 0x7fb3b8144d10 2024-09-19T08:43:31.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.749+0000 7fb3be63c640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fb3b806c620 con 0x7fb3b80729f0 2024-09-19T08:43:31.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.749+0000 7fb3af7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb3b80729f0 0x7fb3b806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:31.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.749+0000 7fb3bcbd9640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fb3b806c1d0 0x7fb3b80cd250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:46012/0 (socket says 172.21.15.98:46012) 2024-09-19T08:43:31.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.749+0000 7fb3bcbd9640 1 -- 172.21.15.98:0/2048363966 learned_addr learned my addr 172.21.15.98:0/2048363966 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:31.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.749+0000 7fb3af7fe640 1 -- 172.21.15.98:0/2048363966 >> v1:172.21.15.139:6789/0 conn(0x7fb3b8144d10 legacy=0x7fb3b80722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:31.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.749+0000 7fb3af7fe640 1 -- 172.21.15.98:0/2048363966 >> v1:172.21.15.98:6789/0 conn(0x7fb3b806c1d0 legacy=0x7fb3b80cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:31.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.749+0000 7fb3b77fe640 1 -- 172.21.15.98:0/2048363966 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 340071398 0 0) 0x7fb3b8072ea0 con 0x7fb3b806c1d0 2024-09-19T08:43:31.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.749+0000 7fb3af7fe640 1 -- 172.21.15.98:0/2048363966 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b8073460 con 0x7fb3b80729f0 2024-09-19T08:43:31.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.753+0000 7fb3af7fe640 1 --2- 172.21.15.98:0/2048363966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3b80729f0 0x7fb3b806bc90 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fb3a00097c0 tx=0x7fb3a001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f3699cef6a376e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:31.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.753+0000 7fb3b77fe640 1 -- 172.21.15.98:0/2048363966 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a00b0070 con 0x7fb3b80729f0 2024-09-19T08:43:31.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.753+0000 7fb3b77fe640 1 -- 172.21.15.98:0/2048363966 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3a00ab600 con 0x7fb3b80729f0 2024-09-19T08:43:31.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.753+0000 7fb3b77fe640 1 -- 172.21.15.98:0/2048363966 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a00ab9d0 con 0x7fb3b80729f0 2024-09-19T08:43:31.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.753+0000 7fb3be63c640 1 -- 172.21.15.98:0/2048363966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3b80729f0 msgr2=0x7fb3b806bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:31.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.753+0000 7fb3be63c640 1 --2- 172.21.15.98:0/2048363966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3b80729f0 0x7fb3b806bc90 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fb3a00097c0 tx=0x7fb3a001c4a0 comp rx=0 tx=0).stop 2024-09-19T08:43:31.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.753+0000 7fb3be63c640 1 -- 172.21.15.98:0/2048363966 shutdown_connections 2024-09-19T08:43:31.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.753+0000 7fb3be63c640 1 --2- 172.21.15.98:0/2048363966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3b80729f0 0x7fb3b806bc90 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:31.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.753+0000 7fb3be63c640 1 -- 172.21.15.98:0/2048363966 >> 172.21.15.98:0/2048363966 conn(0x7fb3b80b2010 msgr2=0x7fb3b80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:31.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.753+0000 7fb3be63c640 1 -- 172.21.15.98:0/2048363966 shutdown_connections 2024-09-19T08:43:31.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.753+0000 7fb3be63c640 1 -- 172.21.15.98:0/2048363966 wait complete. 2024-09-19T08:43:31.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.757+0000 7fb3be63c640 1 Processor -- start 2024-09-19T08:43:31.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.757+0000 7fb3be63c640 1 -- start start 2024-09-19T08:43:31.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.757+0000 7fb3be63c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3b806c1d0 0x7fb3b815acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:31.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.757+0000 7fb3be63c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3b80729f0 0x7fb3b80ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:31.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.757+0000 7fb3be63c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3b8144d10 0x7fb3b80aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:31.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.757+0000 7fb3be63c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b80ce500 con 0x7fb3b80729f0 2024-09-19T08:43:31.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.757+0000 7fb3be63c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b80ce380 con 0x7fb3b806c1d0 2024-09-19T08:43:31.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.757+0000 7fb3be63c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb3b8168f70 con 0x7fb3b8144d10 2024-09-19T08:43:31.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.757+0000 7fb3af7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3b80729f0 0x7fb3b80ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:31.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.757+0000 7fb3af7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3b80729f0 0x7fb3b80ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47970/0 (socket says 172.21.15.98:47970) 2024-09-19T08:43:31.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.757+0000 7fb3af7fe640 1 -- 172.21.15.98:0/1679660369 learned_addr learned my addr 172.21.15.98:0/1679660369 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:31.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.757+0000 7fb3bcbd9640 1 --2- 172.21.15.98:0/1679660369 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3b8144d10 0x7fb3b80aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:31.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.757+0000 7fb3b7fff640 1 --2- 172.21.15.98:0/1679660369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3b806c1d0 0x7fb3b815acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:31.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.757+0000 7fb3af7fe640 1 -- 172.21.15.98:0/1679660369 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3b8144d10 msgr2=0x7fb3b80aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:31.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.757+0000 7fb3af7fe640 1 --2- 172.21.15.98:0/1679660369 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3b8144d10 0x7fb3b80aedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:31.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.757+0000 7fb3af7fe640 1 -- 172.21.15.98:0/1679660369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3b806c1d0 msgr2=0x7fb3b815acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:31.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.757+0000 7fb3af7fe640 1 --2- 172.21.15.98:0/1679660369 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3b806c1d0 0x7fb3b815acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:31.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.757+0000 7fb3af7fe640 1 -- 172.21.15.98:0/1679660369 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b8142fa0 con 0x7fb3b80729f0 2024-09-19T08:43:31.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.757+0000 7fb3af7fe640 1 --2- 172.21.15.98:0/1679660369 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb3b80729f0 0x7fb3b80ae8b0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fb3a00097c0 tx=0x7fb3a001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:31.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.757+0000 7fb3b57fa640 1 -- 172.21.15.98:0/1679660369 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a00b0050 con 0x7fb3b80729f0 2024-09-19T08:43:31.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.757+0000 7fb3b57fa640 1 -- 172.21.15.98:0/1679660369 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3a00b1040 con 0x7fb3b80729f0 2024-09-19T08:43:31.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.757+0000 7fb3be63c640 1 -- 172.21.15.98:0/1679660369 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3b8143280 con 0x7fb3b80729f0 2024-09-19T08:43:31.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.757+0000 7fb3b57fa640 1 -- 172.21.15.98:0/1679660369 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a0004260 con 0x7fb3b80729f0 2024-09-19T08:43:31.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.757+0000 7fb3be63c640 1 -- 172.21.15.98:0/1679660369 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3b8143840 con 0x7fb3b80729f0 2024-09-19T08:43:31.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.757+0000 7fb3be63c640 1 -- 172.21.15.98:0/1679660369 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3b806bc90 con 0x7fb3b80729f0 2024-09-19T08:43:31.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.757+0000 7fb3b57fa640 1 -- 172.21.15.98:0/1679660369 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fb3a00be060 con 0x7fb3b80729f0 2024-09-19T08:43:31.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.757+0000 7fb3b57fa640 1 --2- 172.21.15.98:0/1679660369 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb3980679e0 0x7fb398069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:31.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.757+0000 7fb3b57fa640 1 -- 172.21.15.98:0/1679660369 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fb3a011fa30 con 0x7fb3b80729f0 2024-09-19T08:43:31.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.761+0000 7fb3b7fff640 1 --2- 172.21.15.98:0/1679660369 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb3980679e0 0x7fb398069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:31.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.761+0000 7fb3b7fff640 1 --2- 172.21.15.98:0/1679660369 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb3980679e0 0x7fb398069ea0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8000f70 tx=0x7fb3a8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:31.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:31.765+0000 7fb3b57fa640 1 -- 172.21.15.98:0/1679660369 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb3a0016090 con 0x7fb3b80729f0 2024-09-19T08:43:31.935 INFO:teuthology.orchestra.run.smithi098.stderr:Invalid command: invalid chars #@ in abcd@# 2024-09-19T08:43:31.935 INFO:teuthology.orchestra.run.smithi098.stderr:fs volume create [] : Create a CephFS volume 2024-09-19T08:43:31.935 INFO:teuthology.orchestra.run.smithi098.stderr:Error EINVAL: invalid command 2024-09-19T08:43:31.940 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-19T08:43:31.941 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create testvol 2024-09-19T08:43:32.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.061+0000 7fbc9b96e640 1 Processor -- start 2024-09-19T08:43:32.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.061+0000 7fbc9b96e640 1 -- start start 2024-09-19T08:43:32.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.065+0000 7fbc9b96e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbc94144d10 0x7fbc940722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:32.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.065+0000 7fbc9b96e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbc94072820 0x7fbc9406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:32.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.065+0000 7fbc9b96e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc9406c1d0 0x7fbc940b4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:32.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.065+0000 7fbc9b96e640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fbc94072c60 con 0x7fbc94072820 2024-09-19T08:43:32.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.065+0000 7fbc9b96e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fbc94073ab0 con 0x7fbc94144d10 2024-09-19T08:43:32.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.065+0000 7fbc9b96e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbc941450f0 con 0x7fbc9406c1d0 2024-09-19T08:43:32.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.065+0000 7fbc98f09640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbc94072820 0x7fbc9406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:32.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.065+0000 7fbc98f09640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbc94072820 0x7fbc9406bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47978/0 (socket says 172.21.15.98:47978) 2024-09-19T08:43:32.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.065+0000 7fbc98f09640 1 -- 172.21.15.98:0/880188314 learned_addr learned my addr 172.21.15.98:0/880188314 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:32.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.065+0000 7fbc9970a640 1 --2- 172.21.15.98:0/880188314 >> v2:172.21.15.139:3300/0 conn(0x7fbc94144d10 0x7fbc940722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:32.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.065+0000 7fbc99f0b640 1 --2- 172.21.15.98:0/880188314 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc9406c1d0 0x7fbc940b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:32.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.065+0000 7fbc98f09640 1 -- 172.21.15.98:0/880188314 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc9406c1d0 msgr2=0x7fbc940b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:32.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.065+0000 7fbc98f09640 1 --2- 172.21.15.98:0/880188314 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc9406c1d0 0x7fbc940b4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:32.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.065+0000 7fbc98f09640 1 -- 172.21.15.98:0/880188314 >> v2:172.21.15.139:3300/0 conn(0x7fbc94144d10 msgr2=0x7fbc940722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:32.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.065+0000 7fbc98f09640 1 --2- 172.21.15.98:0/880188314 >> v2:172.21.15.139:3300/0 conn(0x7fbc94144d10 0x7fbc940722e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:32.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.065+0000 7fbc98f09640 1 -- 172.21.15.98:0/880188314 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc94072ea0 con 0x7fbc94072820 2024-09-19T08:43:32.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.065+0000 7fbc9970a640 1 --2- 172.21.15.98:0/880188314 >> v2:172.21.15.139:3300/0 conn(0x7fbc94144d10 0x7fbc940722e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:32.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.065+0000 7fbc98f09640 1 --2- 172.21.15.98:0/880188314 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc94072820 0x7fbc9406bc90 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fbc940684a0 tx=0x7fbc9001c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29f861f43c2ae9d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:32.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.065+0000 7fbc8bfff640 1 -- 172.21.15.98:0/880188314 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc900b0070 con 0x7fbc94072820 2024-09-19T08:43:32.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.065+0000 7fbc8bfff640 1 -- 172.21.15.98:0/880188314 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc900ab600 con 0x7fbc94072820 2024-09-19T08:43:32.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.065+0000 7fbc8bfff640 1 -- 172.21.15.98:0/880188314 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc900ab9d0 con 0x7fbc94072820 2024-09-19T08:43:32.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.069+0000 7fbc9b96e640 1 -- 172.21.15.98:0/880188314 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc94072820 msgr2=0x7fbc9406bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:32.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.069+0000 7fbc9b96e640 1 --2- 172.21.15.98:0/880188314 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc94072820 0x7fbc9406bc90 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fbc940684a0 tx=0x7fbc9001c260 comp rx=0 tx=0).stop 2024-09-19T08:43:32.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.069+0000 7fbc9b96e640 1 -- 172.21.15.98:0/880188314 shutdown_connections 2024-09-19T08:43:32.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.069+0000 7fbc9b96e640 1 --2- 172.21.15.98:0/880188314 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc94072820 0x7fbc9406bc90 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:32.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.069+0000 7fbc9b96e640 1 --2- 172.21.15.98:0/880188314 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc9406c1d0 0x7fbc940b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:32.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.069+0000 7fbc9b96e640 1 --2- 172.21.15.98:0/880188314 >> v2:172.21.15.139:3300/0 conn(0x7fbc94144d10 0x7fbc940722e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:32.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.069+0000 7fbc9b96e640 1 -- 172.21.15.98:0/880188314 >> 172.21.15.98:0/880188314 conn(0x7fbc940b2010 msgr2=0x7fbc940b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:32.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.069+0000 7fbc9b96e640 1 -- 172.21.15.98:0/880188314 shutdown_connections 2024-09-19T08:43:32.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.069+0000 7fbc9b96e640 1 -- 172.21.15.98:0/880188314 wait complete. 2024-09-19T08:43:32.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.069+0000 7fbc9b96e640 1 Processor -- start 2024-09-19T08:43:32.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.069+0000 7fbc9b96e640 1 -- start start 2024-09-19T08:43:32.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.069+0000 7fbc9b96e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc9406c1d0 0x7fbc9413cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:32.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.069+0000 7fbc9b96e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc94072820 0x7fbc9415dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:32.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.069+0000 7fbc9b96e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc94144d10 0x7fbc940cb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:32.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.069+0000 7fbc9b96e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbc940afa30 con 0x7fbc94144d10 2024-09-19T08:43:32.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.069+0000 7fbc9b96e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbc940af8b0 con 0x7fbc94072820 2024-09-19T08:43:32.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.069+0000 7fbc9b96e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbc940b3410 con 0x7fbc9406c1d0 2024-09-19T08:43:32.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.069+0000 7fbc99f0b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc94144d10 0x7fbc940cb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:32.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.069+0000 7fbc99f0b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc94144d10 0x7fbc940cb920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47992/0 (socket says 172.21.15.98:47992) 2024-09-19T08:43:32.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.069+0000 7fbc99f0b640 1 -- 172.21.15.98:0/29632198 learned_addr learned my addr 172.21.15.98:0/29632198 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:32.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.069+0000 7fbc9970a640 1 --2- 172.21.15.98:0/29632198 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc9406c1d0 0x7fbc9413cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:32.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.069+0000 7fbc98f09640 1 --2- 172.21.15.98:0/29632198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc94072820 0x7fbc9415dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:32.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.069+0000 7fbc99f0b640 1 -- 172.21.15.98:0/29632198 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc9406c1d0 msgr2=0x7fbc9413cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:32.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.069+0000 7fbc99f0b640 1 --2- 172.21.15.98:0/29632198 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc9406c1d0 0x7fbc9413cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:32.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.069+0000 7fbc99f0b640 1 -- 172.21.15.98:0/29632198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc94072820 msgr2=0x7fbc9415dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:32.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.069+0000 7fbc99f0b640 1 --2- 172.21.15.98:0/29632198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc94072820 0x7fbc9415dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:32.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.069+0000 7fbc99f0b640 1 -- 172.21.15.98:0/29632198 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc9415e1c0 con 0x7fbc94144d10 2024-09-19T08:43:32.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.069+0000 7fbc9970a640 1 --2- 172.21.15.98:0/29632198 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc9406c1d0 0x7fbc9413cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:32.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.069+0000 7fbc98f09640 1 --2- 172.21.15.98:0/29632198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc94072820 0x7fbc9415dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:32.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.069+0000 7fbc99f0b640 1 --2- 172.21.15.98:0/29632198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc94144d10 0x7fbc940cb920 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c00cd70 tx=0x7fbc8c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:32.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.069+0000 7fbc89ffb640 1 -- 172.21.15.98:0/29632198 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c003ba0 con 0x7fbc94144d10 2024-09-19T08:43:32.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.069+0000 7fbc89ffb640 1 -- 172.21.15.98:0/29632198 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc8c003d40 con 0x7fbc94144d10 2024-09-19T08:43:32.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.069+0000 7fbc9b96e640 1 -- 172.21.15.98:0/29632198 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc9415cbe0 con 0x7fbc94144d10 2024-09-19T08:43:32.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.069+0000 7fbc89ffb640 1 -- 172.21.15.98:0/29632198 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c004240 con 0x7fbc94144d10 2024-09-19T08:43:32.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.069+0000 7fbc9b96e640 1 -- 172.21.15.98:0/29632198 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc9415d1a0 con 0x7fbc94144d10 2024-09-19T08:43:32.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.073+0000 7fbc9b96e640 1 -- 172.21.15.98:0/29632198 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc9406bc90 con 0x7fbc94144d10 2024-09-19T08:43:32.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.073+0000 7fbc89ffb640 1 -- 172.21.15.98:0/29632198 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fbc8c0043e0 con 0x7fbc94144d10 2024-09-19T08:43:32.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.073+0000 7fbc89ffb640 1 --2- 172.21.15.98:0/29632198 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbc780679e0 0x7fbc78069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:32.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.073+0000 7fbc89ffb640 1 -- 172.21.15.98:0/29632198 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fbc8c0869f0 con 0x7fbc94144d10 2024-09-19T08:43:32.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.073+0000 7fbc9970a640 1 --2- 172.21.15.98:0/29632198 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbc780679e0 0x7fbc78069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:32.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.073+0000 7fbc9970a640 1 --2- 172.21.15.98:0/29632198 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbc780679e0 0x7fbc78069ea0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbc8400b840 tx=0x7fbc84008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:32.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.077+0000 7fbc89ffb640 1 -- 172.21.15.98:0/29632198 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fbc8c0573a0 con 0x7fbc94144d10 2024-09-19T08:43:32.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:32.253+0000 7fbc9b96e640 1 -- 172.21.15.98:0/29632198 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "testvol", "target": ["mon-mgr", ""]}) -- 0x7fbc941433f0 con 0x7fbc780679e0 2024-09-19T08:43:35.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.001+0000 7fbc89ffb640 1 -- 172.21.15.98:0/29632198 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) ==== 60+0+0 (secure 0 0 0) 0x7fbc941433f0 con 0x7fbc780679e0 2024-09-19T08:43:35.001 INFO:teuthology.orchestra.run.smithi098.stderr:Volume created successfully (no MDS daemons created) 2024-09-19T08:43:35.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.005+0000 7fbc9b96e640 1 -- 172.21.15.98:0/29632198 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbc780679e0 msgr2=0x7fbc78069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:35.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.005+0000 7fbc9b96e640 1 --2- 172.21.15.98:0/29632198 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbc780679e0 0x7fbc78069ea0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbc8400b840 tx=0x7fbc84008040 comp rx=0 tx=0).stop 2024-09-19T08:43:35.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.005+0000 7fbc9b96e640 1 -- 172.21.15.98:0/29632198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc94144d10 msgr2=0x7fbc940cb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:35.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.005+0000 7fbc9b96e640 1 --2- 172.21.15.98:0/29632198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc94144d10 0x7fbc940cb920 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c00cd70 tx=0x7fbc8c00a660 comp rx=0 tx=0).stop 2024-09-19T08:43:35.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.005+0000 7fbc9b96e640 1 -- 172.21.15.98:0/29632198 shutdown_connections 2024-09-19T08:43:35.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.005+0000 7fbc9b96e640 1 --2- 172.21.15.98:0/29632198 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbc780679e0 0x7fbc78069ea0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:35.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.005+0000 7fbc9b96e640 1 --2- 172.21.15.98:0/29632198 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc94144d10 0x7fbc940cb920 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:35.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.005+0000 7fbc9b96e640 1 --2- 172.21.15.98:0/29632198 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc94072820 0x7fbc9415dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:35.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.005+0000 7fbc9b96e640 1 --2- 172.21.15.98:0/29632198 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc9406c1d0 0x7fbc9413cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:35.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.005+0000 7fbc9b96e640 1 -- 172.21.15.98:0/29632198 >> 172.21.15.98:0/29632198 conn(0x7fbc940b2010 msgr2=0x7fbc940cd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:35.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.005+0000 7fbc9b96e640 1 -- 172.21.15.98:0/29632198 shutdown_connections 2024-09-19T08:43:35.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.005+0000 7fbc9b96e640 1 -- 172.21.15.98:0/29632198 wait complete. 2024-09-19T08:43:35.029 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create testvol 'abcd@#' 2024-09-19T08:43:35.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.145+0000 7f2a144eb640 1 Processor -- start 2024-09-19T08:43:35.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.145+0000 7f2a144eb640 1 -- start start 2024-09-19T08:43:35.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.149+0000 7f2a144eb640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2a0c06cca0 0x7f2a0c06d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:35.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.149+0000 7f2a144eb640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2a0c06fb20 0x7f2a0c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:35.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.149+0000 7f2a144eb640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a0c148350 con 0x7f2a0c06f6c0 2024-09-19T08:43:35.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.149+0000 7f2a144eb640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f2a0c06d5c0 con 0x7f2a0c06fb20 2024-09-19T08:43:35.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.149+0000 7f2a144eb640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f2a0c06e620 con 0x7f2a0c06cca0 2024-09-19T08:43:35.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.149+0000 7f2a12a88640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2a0c06fb20 0x7f2a0c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:35.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.149+0000 7f2a11a86640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f2a0c06f6c0 0x7f2a0c15fc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:46014/0 (socket says 172.21.15.98:46014) 2024-09-19T08:43:35.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.149+0000 7f2a11a86640 1 -- 172.21.15.98:0/568372118 learned_addr learned my addr 172.21.15.98:0/568372118 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:35.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.149+0000 7f2a12287640 1 --2- 172.21.15.98:0/568372118 >> v2:172.21.15.139:3300/0 conn(0x7f2a0c06cca0 0x7f2a0c06d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:35.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.149+0000 7f2a12a88640 1 -- 172.21.15.98:0/568372118 >> v2:172.21.15.139:3300/0 conn(0x7f2a0c06cca0 msgr2=0x7f2a0c06d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:35.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.149+0000 7f2a12a88640 1 --2- 172.21.15.98:0/568372118 >> v2:172.21.15.139:3300/0 conn(0x7f2a0c06cca0 0x7f2a0c06d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:35.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.149+0000 7f2a12a88640 1 -- 172.21.15.98:0/568372118 >> v1:172.21.15.98:6789/0 conn(0x7f2a0c06f6c0 legacy=0x7f2a0c15fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:35.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.149+0000 7f2a11285640 1 -- 172.21.15.98:0/568372118 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 473368334 0 0) 0x7f29fc001120 con 0x7f2a0c06f6c0 2024-09-19T08:43:35.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.149+0000 7f2a12a88640 1 -- 172.21.15.98:0/568372118 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a0c06e900 con 0x7f2a0c06fb20 2024-09-19T08:43:35.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.149+0000 7f2a12a88640 1 --2- 172.21.15.98:0/568372118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a0c06fb20 0x7f2a0c0b4cf0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c0691c0 tx=0x7f2a0401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b5a31090f2a254f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:35.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.149+0000 7f2a11285640 1 -- 172.21.15.98:0/568372118 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a040b2070 con 0x7f2a0c06fb20 2024-09-19T08:43:35.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.149+0000 7f2a11285640 1 -- 172.21.15.98:0/568372118 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a040a58e0 con 0x7f2a0c06fb20 2024-09-19T08:43:35.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.149+0000 7f2a11285640 1 -- 172.21.15.98:0/568372118 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a040a5cb0 con 0x7f2a0c06fb20 2024-09-19T08:43:35.152 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.149+0000 7f2a144eb640 1 -- 172.21.15.98:0/568372118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a0c06fb20 msgr2=0x7f2a0c0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:35.152 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.149+0000 7f2a144eb640 1 --2- 172.21.15.98:0/568372118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a0c06fb20 0x7f2a0c0b4cf0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c0691c0 tx=0x7f2a0401c930 comp rx=0 tx=0).stop 2024-09-19T08:43:35.152 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.149+0000 7f2a144eb640 1 -- 172.21.15.98:0/568372118 shutdown_connections 2024-09-19T08:43:35.152 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.149+0000 7f2a144eb640 1 --2- 172.21.15.98:0/568372118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a0c06fb20 0x7f2a0c0b4cf0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:35.152 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.149+0000 7f2a144eb640 1 --2- 172.21.15.98:0/568372118 >> v2:172.21.15.139:3300/0 conn(0x7f2a0c06cca0 0x7f2a0c06d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:35.152 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.149+0000 7f2a144eb640 1 -- 172.21.15.98:0/568372118 >> 172.21.15.98:0/568372118 conn(0x7f2a0c0720a0 msgr2=0x7f2a0c0724b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:35.152 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.149+0000 7f2a144eb640 1 -- 172.21.15.98:0/568372118 shutdown_connections 2024-09-19T08:43:35.152 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.149+0000 7f2a144eb640 1 -- 172.21.15.98:0/568372118 wait complete. 2024-09-19T08:43:35.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.153+0000 7f2a144eb640 1 Processor -- start 2024-09-19T08:43:35.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.153+0000 7f2a144eb640 1 -- start start 2024-09-19T08:43:35.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.153+0000 7f2a144eb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a0c06cca0 0x7f2a0c15a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:35.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.153+0000 7f2a144eb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a0c06f6c0 0x7f2a0c165b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:35.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.153+0000 7f2a144eb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a0c06fb20 0x7f2a0c163c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:35.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.153+0000 7f2a144eb640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c0b57c0 con 0x7f2a0c06cca0 2024-09-19T08:43:35.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.153+0000 7f2a144eb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c0b5640 con 0x7f2a0c06fb20 2024-09-19T08:43:35.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.153+0000 7f2a144eb640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2a0c160350 con 0x7f2a0c06f6c0 2024-09-19T08:43:35.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.153+0000 7f2a12287640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a0c06cca0 0x7f2a0c15a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:35.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.153+0000 7f2a12287640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a0c06cca0 0x7f2a0c15a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48008/0 (socket says 172.21.15.98:48008) 2024-09-19T08:43:35.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.153+0000 7f2a12287640 1 -- 172.21.15.98:0/2225662589 learned_addr learned my addr 172.21.15.98:0/2225662589 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:35.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.153+0000 7f2a11a86640 1 --2- 172.21.15.98:0/2225662589 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a0c06f6c0 0x7f2a0c165b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:35.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.153+0000 7f2a12a88640 1 --2- 172.21.15.98:0/2225662589 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a0c06fb20 0x7f2a0c163c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:35.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.153+0000 7f2a12287640 1 -- 172.21.15.98:0/2225662589 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a0c06f6c0 msgr2=0x7f2a0c165b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:35.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.153+0000 7f2a12287640 1 --2- 172.21.15.98:0/2225662589 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a0c06f6c0 0x7f2a0c165b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:35.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.153+0000 7f2a12287640 1 -- 172.21.15.98:0/2225662589 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a0c06fb20 msgr2=0x7f2a0c163c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:35.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.153+0000 7f2a12287640 1 --2- 172.21.15.98:0/2225662589 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a0c06fb20 0x7f2a0c163c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:35.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.153+0000 7f2a12287640 1 -- 172.21.15.98:0/2225662589 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a0c15a6d0 con 0x7f2a0c06cca0 2024-09-19T08:43:35.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.153+0000 7f2a11a86640 1 --2- 172.21.15.98:0/2225662589 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a0c06f6c0 0x7f2a0c165b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:35.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.153+0000 7f2a12287640 1 --2- 172.21.15.98:0/2225662589 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a0c06cca0 0x7f2a0c15a190 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f2a0800d830 tx=0x7f2a0800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:35.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.153+0000 7f2a02ffd640 1 -- 172.21.15.98:0/2225662589 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0800b6c0 con 0x7f2a0c06cca0 2024-09-19T08:43:35.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.153+0000 7f2a02ffd640 1 -- 172.21.15.98:0/2225662589 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a08019070 con 0x7f2a0c06cca0 2024-09-19T08:43:35.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.153+0000 7f2a144eb640 1 -- 172.21.15.98:0/2225662589 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a0c15a950 con 0x7f2a0c06cca0 2024-09-19T08:43:35.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.153+0000 7f2a02ffd640 1 -- 172.21.15.98:0/2225662589 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0800ba60 con 0x7f2a0c06cca0 2024-09-19T08:43:35.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.153+0000 7f2a144eb640 1 -- 172.21.15.98:0/2225662589 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a0c15af10 con 0x7f2a0c06cca0 2024-09-19T08:43:35.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.153+0000 7f2a144eb640 1 -- 172.21.15.98:0/2225662589 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a0c06d080 con 0x7f2a0c06cca0 2024-09-19T08:43:35.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.157+0000 7f2a02ffd640 1 -- 172.21.15.98:0/2225662589 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f2a0801e460 con 0x7f2a0c06cca0 2024-09-19T08:43:35.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.157+0000 7f2a02ffd640 1 --2- 172.21.15.98:0/2225662589 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f29f00679e0 0x7f29f0069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:35.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.157+0000 7f2a02ffd640 1 -- 172.21.15.98:0/2225662589 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6543+0+0 (secure 0 0 0) 0x7f2a08086840 con 0x7f2a0c06cca0 2024-09-19T08:43:35.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.157+0000 7f2a11a86640 1 --2- 172.21.15.98:0/2225662589 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f29f00679e0 0x7f29f0069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:35.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.157+0000 7f2a11a86640 1 --2- 172.21.15.98:0/2225662589 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f29f00679e0 0x7f29f0069ea0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f29fc007fd0 tx=0x7f29fc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:35.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.161+0000 7f2a02ffd640 1 -- 172.21.15.98:0/2225662589 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2a0c15a6d0 con 0x7f2a0c06cca0 2024-09-19T08:43:35.348 INFO:teuthology.orchestra.run.smithi098.stderr:Invalid command: invalid chars #@ in abcd@# 2024-09-19T08:43:35.348 INFO:teuthology.orchestra.run.smithi098.stderr:fs subvolumegroup create [] [] [] [] [] : Create a CephFS subvolume group in a volume, and optionally, with a specific data pool layout, and a specific numeric mode 2024-09-19T08:43:35.348 INFO:teuthology.orchestra.run.smithi098.stderr:Error EINVAL: invalid command 2024-09-19T08:43:35.351 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-19T08:43:35.351 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create testvol 'abcd@#' 2024-09-19T08:43:35.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.469+0000 7f395a1d8640 1 Processor -- start 2024-09-19T08:43:35.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.469+0000 7f395a1d8640 1 -- start start 2024-09-19T08:43:35.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.469+0000 7f395a1d8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f395406c3a0 0x7f39540713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:35.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.469+0000 7f395a1d8640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f39540718f0 0x7f39540b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:35.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.469+0000 7f395a1d8640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3954073530 con 0x7f3954145b70 2024-09-19T08:43:35.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.469+0000 7f395a1d8640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f3954073810 con 0x7f39540718f0 2024-09-19T08:43:35.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.469+0000 7f395a1d8640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3954073af0 con 0x7f395406c3a0 2024-09-19T08:43:35.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.469+0000 7f39537fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f3954145b70 0x7f395406bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:46024/0 (socket says 172.21.15.98:46024) 2024-09-19T08:43:35.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.469+0000 7f39537fe640 1 -- 172.21.15.98:0/1639589723 learned_addr learned my addr 172.21.15.98:0/1639589723 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:35.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.469+0000 7f3953fff640 1 --2- 172.21.15.98:0/1639589723 >> v2:172.21.15.98:3300/0 conn(0x7f39540718f0 0x7f39540b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:35.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.469+0000 7f3952ffd640 1 --2- 172.21.15.98:0/1639589723 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f395406c3a0 0x7f39540713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:35.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.469+0000 7f39527fc640 1 -- 172.21.15.98:0/1639589723 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1970461442 0 0) 0x7f3954073530 con 0x7f3954145b70 2024-09-19T08:43:35.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.469+0000 7f39527fc640 1 -- 172.21.15.98:0/1639589723 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f393c003530 con 0x7f3954145b70 2024-09-19T08:43:35.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.469+0000 7f3953fff640 1 -- 172.21.15.98:0/1639589723 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f395406c3a0 msgr2=0x7f39540713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:35.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.469+0000 7f3953fff640 1 --2- 172.21.15.98:0/1639589723 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f395406c3a0 0x7f39540713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:35.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.469+0000 7f3953fff640 1 -- 172.21.15.98:0/1639589723 >> v1:172.21.15.98:6789/0 conn(0x7f3954145b70 legacy=0x7f395406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:35.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.469+0000 7f3953fff640 1 -- 172.21.15.98:0/1639589723 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3954073dd0 con 0x7f39540718f0 2024-09-19T08:43:35.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.473+0000 7f3953fff640 1 --2- 172.21.15.98:0/1639589723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39540718f0 0x7f39540b3410 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f394c009d90 tx=0x7f394c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e486bf6f0547c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:35.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.473+0000 7f39527fc640 1 -- 172.21.15.98:0/1639589723 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c0b2070 con 0x7f39540718f0 2024-09-19T08:43:35.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.473+0000 7f39527fc640 1 -- 172.21.15.98:0/1639589723 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f394c0a57a0 con 0x7f39540718f0 2024-09-19T08:43:35.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.473+0000 7f39527fc640 1 -- 172.21.15.98:0/1639589723 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c0a5b70 con 0x7f39540718f0 2024-09-19T08:43:35.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.473+0000 7f395a1d8640 1 -- 172.21.15.98:0/1639589723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39540718f0 msgr2=0x7f39540b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:35.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.473+0000 7f395a1d8640 1 --2- 172.21.15.98:0/1639589723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39540718f0 0x7f39540b3410 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f394c009d90 tx=0x7f394c01c8e0 comp rx=0 tx=0).stop 2024-09-19T08:43:35.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.473+0000 7f395a1d8640 1 -- 172.21.15.98:0/1639589723 shutdown_connections 2024-09-19T08:43:35.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.473+0000 7f395a1d8640 1 --2- 172.21.15.98:0/1639589723 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f39540718f0 0x7f39540b3410 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:35.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.473+0000 7f395a1d8640 1 --2- 172.21.15.98:0/1639589723 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f395406c3a0 0x7f39540713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:35.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.473+0000 7f395a1d8640 1 -- 172.21.15.98:0/1639589723 >> 172.21.15.98:0/1639589723 conn(0x7f39540b2010 msgr2=0x7f39540b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:35.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.473+0000 7f395a1d8640 1 -- 172.21.15.98:0/1639589723 shutdown_connections 2024-09-19T08:43:35.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.473+0000 7f395a1d8640 1 -- 172.21.15.98:0/1639589723 wait complete. 2024-09-19T08:43:35.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.473+0000 7f395a1d8640 1 Processor -- start 2024-09-19T08:43:35.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.473+0000 7f395a1d8640 1 -- start start 2024-09-19T08:43:35.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.473+0000 7f395a1d8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f395406c3a0 0x7f395413ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:35.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.473+0000 7f395a1d8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39540718f0 0x7f395415e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:35.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.473+0000 7f395a1d8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3954145b70 0x7f395415ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:35.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.473+0000 7f395a1d8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f39540b44f0 con 0x7f395406c3a0 2024-09-19T08:43:35.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.473+0000 7f395a1d8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f39540b4370 con 0x7f39540718f0 2024-09-19T08:43:35.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.473+0000 7f395a1d8640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f39540b4670 con 0x7f3954145b70 2024-09-19T08:43:35.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.477+0000 7f39537fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f395406c3a0 0x7f395413ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:35.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.477+0000 7f39537fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f395406c3a0 0x7f395413ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48016/0 (socket says 172.21.15.98:48016) 2024-09-19T08:43:35.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.477+0000 7f39537fe640 1 -- 172.21.15.98:0/867303455 learned_addr learned my addr 172.21.15.98:0/867303455 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:35.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.477+0000 7f3953fff640 1 --2- 172.21.15.98:0/867303455 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3954145b70 0x7f395415ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:35.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.477+0000 7f3952ffd640 1 --2- 172.21.15.98:0/867303455 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39540718f0 0x7f395415e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:35.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.477+0000 7f39537fe640 1 -- 172.21.15.98:0/867303455 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3954145b70 msgr2=0x7f395415ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:35.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.477+0000 7f39537fe640 1 --2- 172.21.15.98:0/867303455 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3954145b70 0x7f395415ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:35.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.477+0000 7f39537fe640 1 -- 172.21.15.98:0/867303455 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39540718f0 msgr2=0x7f395415e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:35.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.477+0000 7f39537fe640 1 --2- 172.21.15.98:0/867303455 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f39540718f0 0x7f395415e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:35.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.477+0000 7f39537fe640 1 -- 172.21.15.98:0/867303455 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f395415c430 con 0x7f395406c3a0 2024-09-19T08:43:35.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.477+0000 7f3953fff640 1 --2- 172.21.15.98:0/867303455 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3954145b70 0x7f395415ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:35.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.477+0000 7f39537fe640 1 --2- 172.21.15.98:0/867303455 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f395406c3a0 0x7f395413ece0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f394400c3c0 tx=0x7f394400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:35.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.477+0000 7f393bfff640 1 -- 172.21.15.98:0/867303455 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3944017070 con 0x7f395406c3a0 2024-09-19T08:43:35.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.477+0000 7f393bfff640 1 -- 172.21.15.98:0/867303455 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3944005cf0 con 0x7f395406c3a0 2024-09-19T08:43:35.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.477+0000 7f395a1d8640 1 -- 172.21.15.98:0/867303455 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f395413f220 con 0x7f395406c3a0 2024-09-19T08:43:35.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.477+0000 7f393bfff640 1 -- 172.21.15.98:0/867303455 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39440072f0 con 0x7f395406c3a0 2024-09-19T08:43:35.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.477+0000 7f395a1d8640 1 -- 172.21.15.98:0/867303455 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f395413e200 con 0x7f395406c3a0 2024-09-19T08:43:35.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.477+0000 7f395a1d8640 1 -- 172.21.15.98:0/867303455 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39540713b0 con 0x7f395406c3a0 2024-09-19T08:43:35.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.477+0000 7f393bfff640 1 -- 172.21.15.98:0/867303455 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f394401a070 con 0x7f395406c3a0 2024-09-19T08:43:35.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.477+0000 7f393bfff640 1 --2- 172.21.15.98:0/867303455 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f39400679e0 0x7f3940069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:35.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.477+0000 7f393bfff640 1 -- 172.21.15.98:0/867303455 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6543+0+0 (secure 0 0 0) 0x7f39440855f0 con 0x7f395406c3a0 2024-09-19T08:43:35.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.477+0000 7f3952ffd640 1 --2- 172.21.15.98:0/867303455 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f39400679e0 0x7f3940069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:35.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.481+0000 7f3952ffd640 1 --2- 172.21.15.98:0/867303455 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f39400679e0 0x7f3940069ea0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3954142ce0 tx=0x7f3948008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:35.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.481+0000 7f393bfff640 1 -- 172.21.15.98:0/867303455 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3944055c40 con 0x7f395406c3a0 2024-09-19T08:43:35.660 INFO:teuthology.orchestra.run.smithi098.stderr:Invalid command: invalid chars #@ in abcd@# 2024-09-19T08:43:35.661 INFO:teuthology.orchestra.run.smithi098.stderr:fs subvolume create [] [] [] [] [] [] [--namespace-isolated] : Create a CephFS subvolume in a volume, and optionally, with a specific size (in bytes), a specific data pool layout, a specific mode, in a specific subvolume group and in separate RADOS namespace 2024-09-19T08:43:35.661 INFO:teuthology.orchestra.run.smithi098.stderr:Error EINVAL: invalid command 2024-09-19T08:43:35.665 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-19T08:43:35.665 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm testvol --yes-i-really-mean-it 2024-09-19T08:43:35.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.785+0000 7f3c92a22640 1 Processor -- start 2024-09-19T08:43:35.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.785+0000 7f3c92a22640 1 -- start start 2024-09-19T08:43:35.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.785+0000 7f3c92a22640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3c8c1415a0 0x7f3c8c0d20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:35.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.785+0000 7f3c92a22640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c8c0d3a70 0x7f3c8c0b8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:35.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.785+0000 7f3c92a22640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c8c15eae0 con 0x7f3c8c0d25f0 2024-09-19T08:43:35.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.785+0000 7f3c92a22640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f3c8c0d3f00 con 0x7f3c8c1415a0 2024-09-19T08:43:35.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.785+0000 7f3c92a22640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3c8c0d2a30 con 0x7f3c8c0d3a70 2024-09-19T08:43:35.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.785+0000 7f3c8bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3c8c1415a0 0x7f3c8c0d20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:35.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.785+0000 7f3c90fbf640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c8c0d3a70 0x7f3c8c0b8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:35.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.785+0000 7f3c8b7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f3c8c0d25f0 0x7f3c8c0d3360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:52078/0 (socket says 172.21.15.98:52078) 2024-09-19T08:43:35.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.785+0000 7f3c8b7fe640 1 -- 172.21.15.98:0/254378456 learned_addr learned my addr 172.21.15.98:0/254378456 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:35.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.785+0000 7f3c8affd640 1 -- 172.21.15.98:0/254378456 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4039729260 0 0) 0x7f3c8c15eae0 con 0x7f3c8c0d25f0 2024-09-19T08:43:35.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.785+0000 7f3c8affd640 1 -- 172.21.15.98:0/254378456 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c74003530 con 0x7f3c8c0d25f0 2024-09-19T08:43:35.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.785+0000 7f3c90fbf640 1 -- 172.21.15.98:0/254378456 >> v2:172.21.15.139:3300/0 conn(0x7f3c8c1415a0 msgr2=0x7f3c8c0d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:35.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.785+0000 7f3c90fbf640 1 --2- 172.21.15.98:0/254378456 >> v2:172.21.15.139:3300/0 conn(0x7f3c8c1415a0 0x7f3c8c0d20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:35.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.785+0000 7f3c90fbf640 1 -- 172.21.15.98:0/254378456 >> v1:172.21.15.139:6789/0 conn(0x7f3c8c0d25f0 legacy=0x7f3c8c0d3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:35.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.785+0000 7f3c90fbf640 1 -- 172.21.15.98:0/254378456 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c8c15edc0 con 0x7f3c8c0d3a70 2024-09-19T08:43:35.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.789+0000 7f3c90fbf640 1 --2- 172.21.15.98:0/254378456 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c8c0d3a70 0x7f3c8c0b8790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3c84009d90 tx=0x7f3c8401c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1fdb21d257d06bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:35.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.789+0000 7f3c8affd640 1 -- 172.21.15.98:0/254378456 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c840b2070 con 0x7f3c8c0d3a70 2024-09-19T08:43:35.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.789+0000 7f3c8affd640 1 -- 172.21.15.98:0/254378456 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c840a5a00 con 0x7f3c8c0d3a70 2024-09-19T08:43:35.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.789+0000 7f3c8affd640 1 -- 172.21.15.98:0/254378456 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c840a5db0 con 0x7f3c8c0d3a70 2024-09-19T08:43:35.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.789+0000 7f3c92a22640 1 -- 172.21.15.98:0/254378456 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c8c0d3a70 msgr2=0x7f3c8c0b8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:35.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.789+0000 7f3c92a22640 1 --2- 172.21.15.98:0/254378456 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c8c0d3a70 0x7f3c8c0b8790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3c84009d90 tx=0x7f3c8401c940 comp rx=0 tx=0).stop 2024-09-19T08:43:35.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.789+0000 7f3c92a22640 1 -- 172.21.15.98:0/254378456 shutdown_connections 2024-09-19T08:43:35.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.789+0000 7f3c92a22640 1 --2- 172.21.15.98:0/254378456 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c8c0d3a70 0x7f3c8c0b8790 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:35.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.789+0000 7f3c92a22640 1 --2- 172.21.15.98:0/254378456 >> v2:172.21.15.139:3300/0 conn(0x7f3c8c1415a0 0x7f3c8c0d20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:35.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.789+0000 7f3c92a22640 1 -- 172.21.15.98:0/254378456 >> 172.21.15.98:0/254378456 conn(0x7f3c8c074c70 msgr2=0x7f3c8c075080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:35.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.789+0000 7f3c92a22640 1 -- 172.21.15.98:0/254378456 shutdown_connections 2024-09-19T08:43:35.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.789+0000 7f3c92a22640 1 -- 172.21.15.98:0/254378456 wait complete. 2024-09-19T08:43:35.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.789+0000 7f3c92a22640 1 Processor -- start 2024-09-19T08:43:35.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.789+0000 7f3c92a22640 1 -- start start 2024-09-19T08:43:35.793 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.793+0000 7f3c92a22640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c8c0d25f0 0x7f3c8c13e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:35.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.793+0000 7f3c92a22640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c8c0d3a70 0x7f3c8c13eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:35.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.793+0000 7f3c92a22640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c8c1415a0 0x7f3c8c13f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:35.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.793+0000 7f3c92a22640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8c0d42a0 con 0x7f3c8c0d25f0 2024-09-19T08:43:35.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.793+0000 7f3c92a22640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8c0d4120 con 0x7f3c8c1415a0 2024-09-19T08:43:35.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.793+0000 7f3c92a22640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3c8c0b94e0 con 0x7f3c8c0d3a70 2024-09-19T08:43:35.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.793+0000 7f3c8bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c8c0d25f0 0x7f3c8c13e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:35.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.793+0000 7f3c8bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c8c0d25f0 0x7f3c8c13e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:48020/0 (socket says 172.21.15.98:48020) 2024-09-19T08:43:35.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.793+0000 7f3c8bfff640 1 -- 172.21.15.98:0/965830266 learned_addr learned my addr 172.21.15.98:0/965830266 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:35.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.793+0000 7f3c90fbf640 1 --2- 172.21.15.98:0/965830266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c8c1415a0 0x7f3c8c13f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:35.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.793+0000 7f3c8b7fe640 1 --2- 172.21.15.98:0/965830266 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c8c0d3a70 0x7f3c8c13eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:35.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.793+0000 7f3c8bfff640 1 -- 172.21.15.98:0/965830266 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c8c0d3a70 msgr2=0x7f3c8c13eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:35.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.793+0000 7f3c8bfff640 1 --2- 172.21.15.98:0/965830266 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c8c0d3a70 0x7f3c8c13eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:35.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.793+0000 7f3c8bfff640 1 -- 172.21.15.98:0/965830266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c8c1415a0 msgr2=0x7f3c8c13f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:35.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.793+0000 7f3c8bfff640 1 --2- 172.21.15.98:0/965830266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c8c1415a0 0x7f3c8c13f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:35.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.793+0000 7f3c8bfff640 1 -- 172.21.15.98:0/965830266 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c8c13e9b0 con 0x7f3c8c0d25f0 2024-09-19T08:43:35.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.793+0000 7f3c8bfff640 1 --2- 172.21.15.98:0/965830266 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c8c0d25f0 0x7f3c8c13e470 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c00e8a0 tx=0x7f3c7c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:35.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.793+0000 7f3c88ff9640 1 -- 172.21.15.98:0/965830266 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c017070 con 0x7f3c8c0d25f0 2024-09-19T08:43:35.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.793+0000 7f3c88ff9640 1 -- 172.21.15.98:0/965830266 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c7c00c940 con 0x7f3c8c0d25f0 2024-09-19T08:43:35.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.793+0000 7f3c88ff9640 1 -- 172.21.15.98:0/965830266 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c00ccf0 con 0x7f3c8c0d25f0 2024-09-19T08:43:35.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.793+0000 7f3c92a22640 1 -- 172.21.15.98:0/965830266 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c8c0cb6e0 con 0x7f3c8c0d25f0 2024-09-19T08:43:35.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.793+0000 7f3c92a22640 1 -- 172.21.15.98:0/965830266 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c8c155600 con 0x7f3c8c0d25f0 2024-09-19T08:43:35.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.793+0000 7f3c92a22640 1 -- 172.21.15.98:0/965830266 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c8c15eae0 con 0x7f3c8c0d25f0 2024-09-19T08:43:35.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.793+0000 7f3c88ff9640 1 -- 172.21.15.98:0/965830266 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f3c7c01a070 con 0x7f3c8c0d25f0 2024-09-19T08:43:35.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.793+0000 7f3c88ff9640 1 --2- 172.21.15.98:0/965830266 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3c780679e0 0x7f3c78069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:35.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.793+0000 7f3c88ff9640 1 -- 172.21.15.98:0/965830266 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6543+0+0 (secure 0 0 0) 0x7f3c7c085fc0 con 0x7f3c8c0d25f0 2024-09-19T08:43:35.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.797+0000 7f3c8b7fe640 1 --2- 172.21.15.98:0/965830266 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3c780679e0 0x7f3c78069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:35.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.797+0000 7f3c8b7fe640 1 --2- 172.21.15.98:0/965830266 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3c780679e0 0x7f3c78069ea0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3c80007fd0 tx=0x7f3c80009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:35.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.801+0000 7f3c88ff9640 1 -- 172.21.15.98:0/965830266 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3c7c056740 con 0x7f3c8c0d25f0 2024-09-19T08:43:35.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:35.984+0000 7f3c92a22640 1 -- 172.21.15.98:0/965830266 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "testvol", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) -- 0x7f3c8c140db0 con 0x7f3c780679e0 2024-09-19T08:43:36.017 INFO:tasks.ceph.mon.a.smithi098.stderr:2024-09-19T08:43:36.016+0000 7f5e49017640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T08:43:36.058 INFO:tasks.ceph.mds.c.smithi098.stdout:starting mds.c at 2024-09-19T08:43:39.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.052+0000 7f3c88ff9640 1 -- 172.21.15.98:0/965830266 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+78 (secure 0 0 0) 0x7f3c8c140db0 con 0x7f3c780679e0 2024-09-19T08:43:39.055 INFO:teuthology.orchestra.run.smithi098.stdout:metadata pool: cephfs.testvol.meta data pool: ['cephfs.testvol.data'] removed 2024-09-19T08:43:39.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.056+0000 7f3c92a22640 1 -- 172.21.15.98:0/965830266 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3c780679e0 msgr2=0x7f3c78069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:39.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.056+0000 7f3c92a22640 1 --2- 172.21.15.98:0/965830266 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3c780679e0 0x7f3c78069ea0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3c80007fd0 tx=0x7f3c80009040 comp rx=0 tx=0).stop 2024-09-19T08:43:39.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.056+0000 7f3c92a22640 1 -- 172.21.15.98:0/965830266 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c8c0d25f0 msgr2=0x7f3c8c13e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:39.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.056+0000 7f3c92a22640 1 --2- 172.21.15.98:0/965830266 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c8c0d25f0 0x7f3c8c13e470 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c00e8a0 tx=0x7f3c7c00eca0 comp rx=0 tx=0).stop 2024-09-19T08:43:39.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.056+0000 7f3c92a22640 1 -- 172.21.15.98:0/965830266 shutdown_connections 2024-09-19T08:43:39.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.056+0000 7f3c92a22640 1 --2- 172.21.15.98:0/965830266 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3c780679e0 0x7f3c78069ea0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:39.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.056+0000 7f3c92a22640 1 --2- 172.21.15.98:0/965830266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c8c1415a0 0x7f3c8c13f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:39.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.056+0000 7f3c92a22640 1 --2- 172.21.15.98:0/965830266 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c8c0d3a70 0x7f3c8c13eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:39.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.056+0000 7f3c92a22640 1 --2- 172.21.15.98:0/965830266 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c8c0d25f0 0x7f3c8c13e470 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:39.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.056+0000 7f3c92a22640 1 -- 172.21.15.98:0/965830266 >> 172.21.15.98:0/965830266 conn(0x7f3c8c074c70 msgr2=0x7f3c8c159f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:39.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.056+0000 7f3c92a22640 1 -- 172.21.15.98:0/965830266 shutdown_connections 2024-09-19T08:43:39.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.056+0000 7f3c92a22640 1 -- 172.21.15.98:0/965830266 wait complete. 2024-09-19T08:43:39.081 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T08:43:39.082 DEBUG:teuthology.orchestra.run.smithi098:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:43:39.153 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:43:39.238 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T08:43:39.238 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:43:39.238 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:43:39.239 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:43:39.245 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T08:43:39.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.420+0000 7feb40d4b640 1 Processor -- start 2024-09-19T08:43:39.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.420+0000 7feb40d4b640 1 -- start start 2024-09-19T08:43:39.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.420+0000 7feb40d4b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7feb3c15c550 0x7feb3c06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:39.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.420+0000 7feb40d4b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb3c0cfad0 0x7feb3c0723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:39.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.420+0000 7feb40d4b640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb3c06d310 con 0x7feb3c06c1d0 2024-09-19T08:43:39.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.420+0000 7feb40d4b640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7feb3c06c640 con 0x7feb3c15c550 2024-09-19T08:43:39.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.420+0000 7feb40d4b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7feb3c06d5f0 con 0x7feb3c0cfad0 2024-09-19T08:43:39.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.420+0000 7feb39d9b640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7feb3c06c1d0 0x7feb3c0cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:59296/0 (socket says 172.21.15.98:59296) 2024-09-19T08:43:39.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.420+0000 7feb39d9b640 1 -- 172.21.15.98:0/1847631817 learned_addr learned my addr 172.21.15.98:0/1847631817 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:39.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.420+0000 7feb3a59c640 1 --2- 172.21.15.98:0/1847631817 >> v2:172.21.15.98:3300/0 conn(0x7feb3c15c550 0x7feb3c06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:39.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.420+0000 7feb3959a640 1 -- 172.21.15.98:0/1847631817 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3514903325 0 0) 0x7feb3c06d310 con 0x7feb3c06c1d0 2024-09-19T08:43:39.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.420+0000 7feb3959a640 1 -- 172.21.15.98:0/1847631817 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb28003530 con 0x7feb3c06c1d0 2024-09-19T08:43:39.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.420+0000 7feb3ad9d640 1 --2- 172.21.15.98:0/1847631817 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb3c0cfad0 0x7feb3c0723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:39.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.420+0000 7feb3a59c640 1 -- 172.21.15.98:0/1847631817 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb3c0cfad0 msgr2=0x7feb3c0723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:39.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.420+0000 7feb3a59c640 1 --2- 172.21.15.98:0/1847631817 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb3c0cfad0 0x7feb3c0723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:39.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.420+0000 7feb3a59c640 1 -- 172.21.15.98:0/1847631817 >> v1:172.21.15.98:6789/0 conn(0x7feb3c06c1d0 legacy=0x7feb3c0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:39.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.420+0000 7feb3959a640 1 -- 172.21.15.98:0/1847631817 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2222722769 0 0) 0x7feb28003530 con 0x7feb3c06c1d0 2024-09-19T08:43:39.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.420+0000 7feb3a59c640 1 -- 172.21.15.98:0/1847631817 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb3c06d8d0 con 0x7feb3c15c550 2024-09-19T08:43:39.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.424+0000 7feb3a59c640 1 --2- 172.21.15.98:0/1847631817 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb3c15c550 0x7feb3c06bc90 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0097c0 tx=0x7feb2c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6f47cc3debdb44 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:39.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.424+0000 7feb3959a640 1 -- 172.21.15.98:0/1847631817 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c0b0070 con 0x7feb3c15c550 2024-09-19T08:43:39.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.424+0000 7feb3959a640 1 -- 172.21.15.98:0/1847631817 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb2c0ab400 con 0x7feb3c15c550 2024-09-19T08:43:39.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.424+0000 7feb3959a640 1 -- 172.21.15.98:0/1847631817 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb24001680 con 0x7feb3c15c550 2024-09-19T08:43:39.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.424+0000 7feb40d4b640 1 -- 172.21.15.98:0/1847631817 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb3c15c550 msgr2=0x7feb3c06bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:39.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.424+0000 7feb40d4b640 1 --2- 172.21.15.98:0/1847631817 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb3c15c550 0x7feb3c06bc90 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0097c0 tx=0x7feb2c01c3c0 comp rx=0 tx=0).stop 2024-09-19T08:43:39.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.424+0000 7feb40d4b640 1 -- 172.21.15.98:0/1847631817 shutdown_connections 2024-09-19T08:43:39.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.424+0000 7feb40d4b640 1 --2- 172.21.15.98:0/1847631817 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb3c0cfad0 0x7feb3c0723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:39.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.424+0000 7feb40d4b640 1 --2- 172.21.15.98:0/1847631817 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb3c15c550 0x7feb3c06bc90 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:39.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.424+0000 7feb40d4b640 1 -- 172.21.15.98:0/1847631817 >> 172.21.15.98:0/1847631817 conn(0x7feb3c0b2010 msgr2=0x7feb3c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:39.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.424+0000 7feb40d4b640 1 -- 172.21.15.98:0/1847631817 shutdown_connections 2024-09-19T08:43:39.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.424+0000 7feb40d4b640 1 -- 172.21.15.98:0/1847631817 wait complete. 2024-09-19T08:43:39.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.424+0000 7feb40d4b640 1 Processor -- start 2024-09-19T08:43:39.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.424+0000 7feb40d4b640 1 -- start start 2024-09-19T08:43:39.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.424+0000 7feb40d4b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb3c06c1d0 0x7feb3c151f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:39.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.424+0000 7feb40d4b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb3c0cfad0 0x7feb3c1582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:39.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.428+0000 7feb40d4b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb3c15c550 0x7feb3c0ab080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:39.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.428+0000 7feb40d4b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c073380 con 0x7feb3c15c550 2024-09-19T08:43:39.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.428+0000 7feb40d4b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c15cb40 con 0x7feb3c0cfad0 2024-09-19T08:43:39.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.428+0000 7feb40d4b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7feb3c073500 con 0x7feb3c06c1d0 2024-09-19T08:43:39.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.428+0000 7feb3ad9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb3c15c550 0x7feb3c0ab080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:39.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.428+0000 7feb3ad9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb3c15c550 0x7feb3c0ab080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39890/0 (socket says 172.21.15.98:39890) 2024-09-19T08:43:39.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.428+0000 7feb3ad9d640 1 -- 172.21.15.98:0/1725728941 learned_addr learned my addr 172.21.15.98:0/1725728941 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:39.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.428+0000 7feb3ad9d640 1 -- 172.21.15.98:0/1725728941 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb3c06c1d0 msgr2=0x7feb3c151f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:39.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.428+0000 7feb3ad9d640 1 --2- 172.21.15.98:0/1725728941 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb3c06c1d0 0x7feb3c151f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:39.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.428+0000 7feb3ad9d640 1 -- 172.21.15.98:0/1725728941 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb3c0cfad0 msgr2=0x7feb3c1582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:39.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.428+0000 7feb3ad9d640 1 --2- 172.21.15.98:0/1725728941 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb3c0cfad0 0x7feb3c1582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:39.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.428+0000 7feb3ad9d640 1 -- 172.21.15.98:0/1725728941 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb3c158a60 con 0x7feb3c15c550 2024-09-19T08:43:39.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.428+0000 7feb3ad9d640 1 --2- 172.21.15.98:0/1725728941 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb3c15c550 0x7feb3c0ab080 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7feb34007e70 tx=0x7feb3400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:39.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.428+0000 7feb32ffd640 1 -- 172.21.15.98:0/1725728941 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb34014620 con 0x7feb3c15c550 2024-09-19T08:43:39.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.428+0000 7feb32ffd640 1 -- 172.21.15.98:0/1725728941 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb34019070 con 0x7feb3c15c550 2024-09-19T08:43:39.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.428+0000 7feb40d4b640 1 -- 172.21.15.98:0/1725728941 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb3c158d40 con 0x7feb3c15c550 2024-09-19T08:43:39.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.428+0000 7feb32ffd640 1 -- 172.21.15.98:0/1725728941 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb34014a20 con 0x7feb3c15c550 2024-09-19T08:43:39.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.428+0000 7feb40d4b640 1 -- 172.21.15.98:0/1725728941 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb3c159300 con 0x7feb3c15c550 2024-09-19T08:43:39.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.428+0000 7feb40d4b640 1 -- 172.21.15.98:0/1725728941 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb3c0723a0 con 0x7feb3c15c550 2024-09-19T08:43:39.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.428+0000 7feb32ffd640 1 -- 172.21.15.98:0/1725728941 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7feb34014bc0 con 0x7feb3c15c550 2024-09-19T08:43:39.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.428+0000 7feb32ffd640 1 --2- 172.21.15.98:0/1725728941 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7feb1c0679e0 0x7feb1c069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:39.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.428+0000 7feb32ffd640 1 -- 172.21.15.98:0/1725728941 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7feb34086790 con 0x7feb3c15c550 2024-09-19T08:43:39.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.428+0000 7feb3a59c640 1 --2- 172.21.15.98:0/1725728941 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7feb1c0679e0 0x7feb1c069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:39.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.432+0000 7feb3a59c640 1 --2- 172.21.15.98:0/1725728941 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7feb1c0679e0 0x7feb1c069ea0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0ad000 tx=0x7feb2c004740 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:39.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.432+0000 7feb32ffd640 1 -- 172.21.15.98:0/1725728941 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7feb34057150 con 0x7feb3c15c550 2024-09-19T08:43:39.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.608+0000 7feb40d4b640 1 -- 172.21.15.98:0/1725728941 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7feb3c0ace60 con 0x7feb3c15c550 2024-09-19T08:43:39.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.608+0000 7feb32ffd640 1 -- 172.21.15.98:0/1725728941 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v91) ==== 80+0+122 (secure 0 0 0) 0x7feb3c0ace60 con 0x7feb3c15c550 2024-09-19T08:43:39.610 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.15.98:6833/1092838020 2024-09-20T08:43:36.020574+0000 2024-09-19T08:43:39.610 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.15.98:6832/1092838020 2024-09-20T08:43:36.020574+0000 2024-09-19T08:43:39.610 INFO:teuthology.orchestra.run.smithi098.stderr:listed 2 entries 2024-09-19T08:43:39.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.608+0000 7feb40d4b640 1 -- 172.21.15.98:0/1725728941 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7feb1c0679e0 msgr2=0x7feb1c069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:39.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.608+0000 7feb40d4b640 1 --2- 172.21.15.98:0/1725728941 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7feb1c0679e0 0x7feb1c069ea0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0ad000 tx=0x7feb2c004740 comp rx=0 tx=0).stop 2024-09-19T08:43:39.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.608+0000 7feb40d4b640 1 -- 172.21.15.98:0/1725728941 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb3c15c550 msgr2=0x7feb3c0ab080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:39.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.608+0000 7feb40d4b640 1 --2- 172.21.15.98:0/1725728941 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb3c15c550 0x7feb3c0ab080 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7feb34007e70 tx=0x7feb3400e4b0 comp rx=0 tx=0).stop 2024-09-19T08:43:39.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.608+0000 7feb40d4b640 1 -- 172.21.15.98:0/1725728941 shutdown_connections 2024-09-19T08:43:39.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.608+0000 7feb40d4b640 1 --2- 172.21.15.98:0/1725728941 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7feb1c0679e0 0x7feb1c069ea0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:39.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.608+0000 7feb40d4b640 1 --2- 172.21.15.98:0/1725728941 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb3c15c550 0x7feb3c0ab080 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:39.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.608+0000 7feb40d4b640 1 --2- 172.21.15.98:0/1725728941 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb3c0cfad0 0x7feb3c1582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:39.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.608+0000 7feb40d4b640 1 --2- 172.21.15.98:0/1725728941 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb3c06c1d0 0x7feb3c151f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:39.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.608+0000 7feb40d4b640 1 -- 172.21.15.98:0/1725728941 >> 172.21.15.98:0/1725728941 conn(0x7feb3c0b2010 msgr2=0x7feb3c1561e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:39.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.608+0000 7feb40d4b640 1 -- 172.21.15.98:0/1725728941 shutdown_connections 2024-09-19T08:43:39.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.612+0000 7feb40d4b640 1 -- 172.21.15.98:0/1725728941 wait complete. 2024-09-19T08:43:39.628 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi098.front.sepia.ceph.com... 2024-09-19T08:43:39.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:43:39.629 DEBUG:teuthology.orchestra.run.smithi098:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T08:43:39.640 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:ceph-fuse[44479]: fuse finished with error 0 and tester_r 0 2024-09-19T08:43:39.641 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:43:39.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:43:39.641 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T08:43:39.807 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:43:39.807 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:43:39.807 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:43:39.814 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T08:43:39.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.976+0000 7f32ea9b5640 1 Processor -- start 2024-09-19T08:43:39.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.976+0000 7f32ea9b5640 1 -- start start 2024-09-19T08:43:39.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.976+0000 7f32ea9b5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f32e406c3a0 0x7f32e40cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:39.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.976+0000 7f32ea9b5640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f32e40cf900 0x7f32e40723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:39.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.976+0000 7f32ea9b5640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32e406d310 con 0x7f32e415c550 2024-09-19T08:43:39.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.976+0000 7f32ea9b5640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f32e406d5f0 con 0x7f32e40cf900 2024-09-19T08:43:39.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.976+0000 7f32ea9b5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f32e406d8d0 con 0x7f32e406c3a0 2024-09-19T08:43:39.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.976+0000 7f32e8f52640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f32e40cf900 0x7f32e40723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:39.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.976+0000 7f32e3fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f32e415c550 0x7f32e406bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:59312/0 (socket says 172.21.15.98:59312) 2024-09-19T08:43:39.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.976+0000 7f32e3fff640 1 -- 172.21.15.98:0/999995417 learned_addr learned my addr 172.21.15.98:0/999995417 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:39.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.976+0000 7f32dbfff640 1 --2- 172.21.15.98:0/999995417 >> v2:172.21.15.139:3300/0 conn(0x7f32e406c3a0 0x7f32e40cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:39.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.976+0000 7f32e8f52640 1 -- 172.21.15.98:0/999995417 >> v2:172.21.15.139:3300/0 conn(0x7f32e406c3a0 msgr2=0x7f32e40cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:39.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.976+0000 7f32e8f52640 1 --2- 172.21.15.98:0/999995417 >> v2:172.21.15.139:3300/0 conn(0x7f32e406c3a0 0x7f32e40cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:39.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.976+0000 7f32e8f52640 1 -- 172.21.15.98:0/999995417 >> v1:172.21.15.98:6789/0 conn(0x7f32e415c550 legacy=0x7f32e406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:39.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.976+0000 7f32e8f52640 1 -- 172.21.15.98:0/999995417 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e406dbb0 con 0x7f32e40cf900 2024-09-19T08:43:39.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.976+0000 7f32e37fe640 1 -- 172.21.15.98:0/999995417 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3158467356 0 0) 0x7f32e406d310 con 0x7f32e415c550 2024-09-19T08:43:39.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.976+0000 7f32dbfff640 1 --2- 172.21.15.98:0/999995417 >> v2:172.21.15.139:3300/0 conn(0x7f32e406c3a0 0x7f32e40cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:39.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.980+0000 7f32e8f52640 1 --2- 172.21.15.98:0/999995417 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32e40cf900 0x7f32e40723a0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f32e40ccc60 tx=0x7f32dc01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17eec4e419f3d73 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:39.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.980+0000 7f32e37fe640 1 -- 172.21.15.98:0/999995417 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc0b2070 con 0x7f32e40cf900 2024-09-19T08:43:39.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.980+0000 7f32e37fe640 1 -- 172.21.15.98:0/999995417 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32dc0a5900 con 0x7f32e40cf900 2024-09-19T08:43:39.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.980+0000 7f32e37fe640 1 -- 172.21.15.98:0/999995417 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc0a5cd0 con 0x7f32e40cf900 2024-09-19T08:43:39.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.980+0000 7f32ea9b5640 1 -- 172.21.15.98:0/999995417 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32e40cf900 msgr2=0x7f32e40723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:39.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.980+0000 7f32ea9b5640 1 --2- 172.21.15.98:0/999995417 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32e40cf900 0x7f32e40723a0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f32e40ccc60 tx=0x7f32dc01c930 comp rx=0 tx=0).stop 2024-09-19T08:43:39.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.980+0000 7f32ea9b5640 1 -- 172.21.15.98:0/999995417 shutdown_connections 2024-09-19T08:43:39.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.980+0000 7f32ea9b5640 1 --2- 172.21.15.98:0/999995417 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32e40cf900 0x7f32e40723a0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:39.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.980+0000 7f32ea9b5640 1 --2- 172.21.15.98:0/999995417 >> v2:172.21.15.139:3300/0 conn(0x7f32e406c3a0 0x7f32e40cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:39.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.980+0000 7f32ea9b5640 1 -- 172.21.15.98:0/999995417 >> 172.21.15.98:0/999995417 conn(0x7f32e40b2010 msgr2=0x7f32e40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:39.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.980+0000 7f32ea9b5640 1 -- 172.21.15.98:0/999995417 shutdown_connections 2024-09-19T08:43:39.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.980+0000 7f32ea9b5640 1 -- 172.21.15.98:0/999995417 wait complete. 2024-09-19T08:43:39.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.980+0000 7f32ea9b5640 1 Processor -- start 2024-09-19T08:43:39.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.980+0000 7f32ea9b5640 1 -- start start 2024-09-19T08:43:39.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.984+0000 7f32ea9b5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32e406c3a0 0x7f32e41f6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:39.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.984+0000 7f32ea9b5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32e40cf900 0x7f32e415f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:39.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.984+0000 7f32ea9b5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32e415c550 0x7f32e4152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:39.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.984+0000 7f32ea9b5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f32e4073480 con 0x7f32e415c550 2024-09-19T08:43:39.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.984+0000 7f32ea9b5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f32e4073300 con 0x7f32e40cf900 2024-09-19T08:43:39.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.984+0000 7f32ea9b5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f32e4073600 con 0x7f32e406c3a0 2024-09-19T08:43:39.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.984+0000 7f32e8f52640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32e415c550 0x7f32e4152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:39.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.984+0000 7f32e8f52640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32e415c550 0x7f32e4152160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39908/0 (socket says 172.21.15.98:39908) 2024-09-19T08:43:39.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.984+0000 7f32e8f52640 1 -- 172.21.15.98:0/4223250694 learned_addr learned my addr 172.21.15.98:0/4223250694 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:39.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.984+0000 7f32dbfff640 1 --2- 172.21.15.98:0/4223250694 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32e40cf900 0x7f32e415f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:39.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.984+0000 7f32e3fff640 1 --2- 172.21.15.98:0/4223250694 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32e406c3a0 0x7f32e41f6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:39.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.984+0000 7f32e8f52640 1 -- 172.21.15.98:0/4223250694 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32e406c3a0 msgr2=0x7f32e41f6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:39.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.984+0000 7f32e8f52640 1 --2- 172.21.15.98:0/4223250694 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32e406c3a0 0x7f32e41f6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:39.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.984+0000 7f32e8f52640 1 -- 172.21.15.98:0/4223250694 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32e40cf900 msgr2=0x7f32e415f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:39.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.984+0000 7f32e8f52640 1 --2- 172.21.15.98:0/4223250694 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32e40cf900 0x7f32e415f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:39.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.984+0000 7f32e8f52640 1 -- 172.21.15.98:0/4223250694 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e415fdf0 con 0x7f32e415c550 2024-09-19T08:43:39.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.984+0000 7f32e3fff640 1 --2- 172.21.15.98:0/4223250694 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32e406c3a0 0x7f32e41f6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:39.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.984+0000 7f32dbfff640 1 --2- 172.21.15.98:0/4223250694 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32e40cf900 0x7f32e415f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:39.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.984+0000 7f32e8f52640 1 --2- 172.21.15.98:0/4223250694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32e415c550 0x7f32e4152160 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f32dc019540 tx=0x7f32dc0196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:39.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.984+0000 7f32e17fa640 1 -- 172.21.15.98:0/4223250694 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc0b2050 con 0x7f32e415c550 2024-09-19T08:43:39.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.984+0000 7f32e17fa640 1 -- 172.21.15.98:0/4223250694 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32dc0b3040 con 0x7f32e415c550 2024-09-19T08:43:39.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.984+0000 7f32ea9b5640 1 -- 172.21.15.98:0/4223250694 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32e415fb10 con 0x7f32e415c550 2024-09-19T08:43:39.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.984+0000 7f32e17fa640 1 -- 172.21.15.98:0/4223250694 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc0ad910 con 0x7f32e415c550 2024-09-19T08:43:39.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.984+0000 7f32ea9b5640 1 -- 172.21.15.98:0/4223250694 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32e4157960 con 0x7f32e415c550 2024-09-19T08:43:39.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.984+0000 7f32ea9b5640 1 -- 172.21.15.98:0/4223250694 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32e40cf3c0 con 0x7f32e415c550 2024-09-19T08:43:39.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.984+0000 7f32e17fa640 1 -- 172.21.15.98:0/4223250694 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f32dc0c0020 con 0x7f32e415c550 2024-09-19T08:43:39.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.984+0000 7f32e17fa640 1 --2- 172.21.15.98:0/4223250694 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f32c40679e0 0x7f32c4069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:39.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.984+0000 7f32e17fa640 1 -- 172.21.15.98:0/4223250694 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7f32dc1208e0 con 0x7f32e415c550 2024-09-19T08:43:39.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.984+0000 7f32e3fff640 1 --2- 172.21.15.98:0/4223250694 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f32c40679e0 0x7f32c4069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:39.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.988+0000 7f32e3fff640 1 --2- 172.21.15.98:0/4223250694 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f32c40679e0 0x7f32c4069ea0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f32d4000f70 tx=0x7f32d4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:39.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:39.988+0000 7f32e17fa640 1 -- 172.21.15.98:0/4223250694 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f32dc016520 con 0x7f32e415c550 2024-09-19T08:43:40.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.168+0000 7f32ea9b5640 1 -- 172.21.15.98:0/4223250694 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f32e40008e0 con 0x7f32e415c550 2024-09-19T08:43:40.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.168+0000 7f32e17fa640 1 -- 172.21.15.98:0/4223250694 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v91) ==== 80+0+122 (secure 0 0 0) 0x7f32dc0f13d0 con 0x7f32e415c550 2024-09-19T08:43:40.170 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.15.98:6833/1092838020 2024-09-20T08:43:36.020574+0000 2024-09-19T08:43:40.170 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.15.98:6832/1092838020 2024-09-20T08:43:36.020574+0000 2024-09-19T08:43:40.170 INFO:teuthology.orchestra.run.smithi098.stderr:listed 2 entries 2024-09-19T08:43:40.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.168+0000 7f32ea9b5640 1 -- 172.21.15.98:0/4223250694 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f32c40679e0 msgr2=0x7f32c4069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:40.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.168+0000 7f32ea9b5640 1 --2- 172.21.15.98:0/4223250694 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f32c40679e0 0x7f32c4069ea0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f32d4000f70 tx=0x7f32d4009040 comp rx=0 tx=0).stop 2024-09-19T08:43:40.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.168+0000 7f32ea9b5640 1 -- 172.21.15.98:0/4223250694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32e415c550 msgr2=0x7f32e4152160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:40.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.168+0000 7f32ea9b5640 1 --2- 172.21.15.98:0/4223250694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32e415c550 0x7f32e4152160 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f32dc019540 tx=0x7f32dc0196d0 comp rx=0 tx=0).stop 2024-09-19T08:43:40.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.168+0000 7f32ea9b5640 1 -- 172.21.15.98:0/4223250694 shutdown_connections 2024-09-19T08:43:40.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.168+0000 7f32ea9b5640 1 --2- 172.21.15.98:0/4223250694 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f32c40679e0 0x7f32c4069ea0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:40.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.168+0000 7f32ea9b5640 1 --2- 172.21.15.98:0/4223250694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32e415c550 0x7f32e4152160 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:40.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.168+0000 7f32ea9b5640 1 --2- 172.21.15.98:0/4223250694 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32e40cf900 0x7f32e415f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:40.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.168+0000 7f32ea9b5640 1 --2- 172.21.15.98:0/4223250694 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32e406c3a0 0x7f32e41f6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:40.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.168+0000 7f32ea9b5640 1 -- 172.21.15.98:0/4223250694 >> 172.21.15.98:0/4223250694 conn(0x7f32e40b2010 msgr2=0x7f32e406bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:40.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.168+0000 7f32ea9b5640 1 -- 172.21.15.98:0/4223250694 shutdown_connections 2024-09-19T08:43:40.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.168+0000 7f32ea9b5640 1 -- 172.21.15.98:0/4223250694 wait complete. 2024-09-19T08:43:40.188 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi139.front.sepia.ceph.com... 2024-09-19T08:43:40.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:43:40.188 DEBUG:teuthology.orchestra.run.smithi139:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-19T08:43:40.201 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[36007]: fuse finished with error 0 and tester_r 0 2024-09-19T08:43:40.203 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:43:40.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:43:40.203 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-19T08:43:40.296 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:43:40.297 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:43:40.297 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:43:40.304 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15846 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T08:43:40.304 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:43:40.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:43:40.305 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-19T08:43:40.353 INFO:teuthology.orchestra.run.smithi098.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-19T08:43:40.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:43:40.354 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:43:40.354 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:43:40.354 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:43:40.361 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T08:43:40.361 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:43:40.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:43:40.361 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-19T08:43:40.409 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-19T08:43:40.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:43:40.410 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:43:40.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.540+0000 7fc862929640 1 Processor -- start 2024-09-19T08:43:40.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.540+0000 7fc862929640 1 -- start start 2024-09-19T08:43:40.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.540+0000 7fc862929640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc85c06d310 0x7fc85c075680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:40.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.540+0000 7fc862929640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc85c0b3c70 con 0x7fc85c156be0 2024-09-19T08:43:40.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.540+0000 7fc862929640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc85c0b0ad0 con 0x7fc85c0b68b0 2024-09-19T08:43:40.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.540+0000 7fc862929640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fc85c075bc0 con 0x7fc85c06d310 2024-09-19T08:43:40.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.540+0000 7fc85bfff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fc85c156be0 0x7fc85c15d2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:59314/0 (socket says 172.21.15.98:59314) 2024-09-19T08:43:40.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.540+0000 7fc85bfff640 1 -- 172.21.15.98:0/2106688543 learned_addr learned my addr 172.21.15.98:0/2106688543 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:40.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.540+0000 7fc85b7fe640 1 --2- 172.21.15.98:0/2106688543 >> v2:172.21.15.139:3300/0 conn(0x7fc85c06d310 0x7fc85c075680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:40.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.540+0000 7fc85affd640 1 -- 172.21.15.98:0/2106688543 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3792538953 0 0) 0x7fc84c001120 con 0x7fc85c156be0 2024-09-19T08:43:40.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.540+0000 7fc85affd640 1 -- 172.21.15.98:0/2106688543 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc844003530 con 0x7fc85c156be0 2024-09-19T08:43:40.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.540+0000 7fc85affd640 1 -- 172.21.15.98:0/2106688543 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2098163240 0 0) 0x7fc844003530 con 0x7fc85c156be0 2024-09-19T08:43:40.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.540+0000 7fc85affd640 1 -- 172.21.15.98:0/2106688543 >> v2:172.21.15.139:3300/0 conn(0x7fc85c06d310 msgr2=0x7fc85c075680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:40.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.540+0000 7fc85affd640 1 --2- 172.21.15.98:0/2106688543 >> v2:172.21.15.139:3300/0 conn(0x7fc85c06d310 0x7fc85c075680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:40.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.540+0000 7fc85affd640 1 -- 172.21.15.98:0/2106688543 >> v1:172.21.15.139:6789/0 conn(0x7fc85c0b68b0 legacy=0x7fc85c073320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:40.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.540+0000 7fc85affd640 1 -- 172.21.15.98:0/2106688543 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc85c0b0510 con 0x7fc85c156be0 2024-09-19T08:43:40.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.540+0000 7fc85affd640 1 -- 172.21.15.98:0/2106688543 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fc84c001850 con 0x7fc85c156be0 2024-09-19T08:43:40.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.540+0000 7fc85affd640 1 -- 172.21.15.98:0/2106688543 >> v1:172.21.15.98:6789/0 conn(0x7fc85c156be0 legacy=0x7fc85c15d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:40.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.540+0000 7fc85affd640 1 --2- 172.21.15.98:0/2106688543 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8440039c0 0x7fc844003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:40.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.540+0000 7fc85affd640 1 --2- 172.21.15.98:0/2106688543 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc844004810 0x7fc844004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:40.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.540+0000 7fc85affd640 1 --2- 172.21.15.98:0/2106688543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8440051d0 0x7fc84400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:40.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.540+0000 7fc85affd640 1 -- 172.21.15.98:0/2106688543 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc84400c140 con 0x7fc844004810 2024-09-19T08:43:40.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.540+0000 7fc85affd640 1 -- 172.21.15.98:0/2106688543 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc84400c2f0 con 0x7fc8440051d0 2024-09-19T08:43:40.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.540+0000 7fc85affd640 1 -- 172.21.15.98:0/2106688543 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc84400c4f0 con 0x7fc8440039c0 2024-09-19T08:43:40.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.540+0000 7fc85b7fe640 1 --2- 172.21.15.98:0/2106688543 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc844004810 0x7fc844004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:40.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.540+0000 7fc85bfff640 1 --2- 172.21.15.98:0/2106688543 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8440039c0 0x7fc844003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:40.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.540+0000 7fc860ec6640 1 --2- 172.21.15.98:0/2106688543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8440051d0 0x7fc84400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:40.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.540+0000 7fc85b7fe640 1 -- 172.21.15.98:0/2106688543 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8440039c0 msgr2=0x7fc844003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:40.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.540+0000 7fc85b7fe640 1 --2- 172.21.15.98:0/2106688543 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8440039c0 0x7fc844003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:40.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.540+0000 7fc85b7fe640 1 -- 172.21.15.98:0/2106688543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8440051d0 msgr2=0x7fc84400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:40.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.540+0000 7fc85b7fe640 1 --2- 172.21.15.98:0/2106688543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc8440051d0 0x7fc84400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:40.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.540+0000 7fc85b7fe640 1 -- 172.21.15.98:0/2106688543 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc844003530 con 0x7fc844004810 2024-09-19T08:43:40.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.540+0000 7fc860ec6640 1 -- 172.21.15.98:0/2106688543 reap_dead start 2024-09-19T08:43:40.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.544+0000 7fc85b7fe640 1 --2- 172.21.15.98:0/2106688543 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc844004810 0x7fc844004c90 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fc850009ae0 tx=0x7fc85001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bbd5a28a9dfcb6e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:40.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.544+0000 7fc85affd640 1 -- 172.21.15.98:0/2106688543 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8500096b0 con 0x7fc844004810 2024-09-19T08:43:40.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.544+0000 7fc85affd640 1 -- 172.21.15.98:0/2106688543 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8500a6450 con 0x7fc844004810 2024-09-19T08:43:40.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.544+0000 7fc85affd640 1 -- 172.21.15.98:0/2106688543 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8500a6840 con 0x7fc844004810 2024-09-19T08:43:40.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.544+0000 7fc862929640 1 -- 172.21.15.98:0/2106688543 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc844004810 msgr2=0x7fc844004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:40.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.544+0000 7fc862929640 1 --2- 172.21.15.98:0/2106688543 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc844004810 0x7fc844004c90 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fc850009ae0 tx=0x7fc85001ca70 comp rx=0 tx=0).stop 2024-09-19T08:43:40.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.544+0000 7fc862929640 1 -- 172.21.15.98:0/2106688543 shutdown_connections 2024-09-19T08:43:40.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.544+0000 7fc862929640 1 --2- 172.21.15.98:0/2106688543 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc844004810 0x7fc844004c90 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:40.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.544+0000 7fc862929640 1 -- 172.21.15.98:0/2106688543 >> 172.21.15.98:0/2106688543 conn(0x7fc85c0b2ed0 msgr2=0x7fc85c0b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:40.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.544+0000 7fc862929640 1 -- 172.21.15.98:0/2106688543 shutdown_connections 2024-09-19T08:43:40.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.544+0000 7fc862929640 1 -- 172.21.15.98:0/2106688543 wait complete. 2024-09-19T08:43:40.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.548+0000 7fc862929640 1 Processor -- start 2024-09-19T08:43:40.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.548+0000 7fc862929640 1 -- start start 2024-09-19T08:43:40.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.548+0000 7fc862929640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc844004810 0x7fc85c148260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:40.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.548+0000 7fc862929640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc85c14f9e0 0x7fc85c14fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:40.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.548+0000 7fc862929640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc85c15f2f0 0x7fc85c15f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:40.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.548+0000 7fc862929640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc85c0b70d0 con 0x7fc85c15f2f0 2024-09-19T08:43:40.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.548+0000 7fc862929640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc85c073320 con 0x7fc844004810 2024-09-19T08:43:40.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.548+0000 7fc862929640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc85c14ee30 con 0x7fc85c14f9e0 2024-09-19T08:43:40.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.548+0000 7fc860ec6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc85c15f2f0 0x7fc85c15f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:40.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.548+0000 7fc860ec6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc85c15f2f0 0x7fc85c15f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39928/0 (socket says 172.21.15.98:39928) 2024-09-19T08:43:40.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.548+0000 7fc860ec6640 1 -- 172.21.15.98:0/1689832271 learned_addr learned my addr 172.21.15.98:0/1689832271 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:40.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.548+0000 7fc860ec6640 1 -- 172.21.15.98:0/1689832271 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc85c14f9e0 msgr2=0x7fc85c14fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:40.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.548+0000 7fc860ec6640 1 --2- 172.21.15.98:0/1689832271 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc85c14f9e0 0x7fc85c14fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:40.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.548+0000 7fc860ec6640 1 -- 172.21.15.98:0/1689832271 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc844004810 msgr2=0x7fc85c148260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:40.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.548+0000 7fc860ec6640 1 --2- 172.21.15.98:0/1689832271 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc844004810 0x7fc85c148260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:40.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.548+0000 7fc860ec6640 1 -- 172.21.15.98:0/1689832271 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc85c14e260 con 0x7fc85c15f2f0 2024-09-19T08:43:40.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.548+0000 7fc860ec6640 1 --2- 172.21.15.98:0/1689832271 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc85c15f2f0 0x7fc85c15f770 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fc85c0d2320 tx=0x7fc85400e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:40.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.548+0000 7fc858ff9640 1 -- 172.21.15.98:0/1689832271 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc854019070 con 0x7fc85c15f2f0 2024-09-19T08:43:40.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.548+0000 7fc858ff9640 1 -- 172.21.15.98:0/1689832271 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc854006e30 con 0x7fc85c15f2f0 2024-09-19T08:43:40.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.548+0000 7fc862929640 1 -- 172.21.15.98:0/1689832271 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc85c14e500 con 0x7fc85c15f2f0 2024-09-19T08:43:40.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.548+0000 7fc858ff9640 1 -- 172.21.15.98:0/1689832271 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc854003ba0 con 0x7fc85c15f2f0 2024-09-19T08:43:40.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.548+0000 7fc862929640 1 -- 172.21.15.98:0/1689832271 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc85c14eac0 con 0x7fc85c15f2f0 2024-09-19T08:43:40.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.548+0000 7fc862929640 1 -- 172.21.15.98:0/1689832271 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc85c0736b0 con 0x7fc85c15f2f0 2024-09-19T08:43:40.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.552+0000 7fc858ff9640 1 -- 172.21.15.98:0/1689832271 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fc85401c070 con 0x7fc85c15f2f0 2024-09-19T08:43:40.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.552+0000 7fc858ff9640 1 --2- 172.21.15.98:0/1689832271 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc8480679e0 0x7fc848069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:40.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.552+0000 7fc858ff9640 1 -- 172.21.15.98:0/1689832271 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7fc8540852f0 con 0x7fc85c15f2f0 2024-09-19T08:43:40.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.552+0000 7fc85bfff640 1 --2- 172.21.15.98:0/1689832271 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc8480679e0 0x7fc848069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:40.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.552+0000 7fc85bfff640 1 --2- 172.21.15.98:0/1689832271 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc8480679e0 0x7fc848069ea0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc84c0013e0 tx=0x7fc84c000ff0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:40.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.556+0000 7fc858ff9640 1 -- 172.21.15.98:0/1689832271 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc854055d50 con 0x7fc85c15f2f0 2024-09-19T08:43:40.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.748+0000 7fc862929640 1 -- 172.21.15.98:0/1689832271 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc85c06fb90 con 0x7fc85c15f2f0 2024-09-19T08:43:40.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.752+0000 7fc858ff9640 1 -- 172.21.15.98:0/1689832271 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5225 (secure 0 0 0) 0x7fc854055650 con 0x7fc85c15f2f0 2024-09-19T08:43:40.752 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:40.752 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":52,"btime":"2024-09-19T08:43:37:022736+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3182634136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3182634136},{"type":"v1","addr":"172.21.15.139:6833","nonce":3182634136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5830,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1554653172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1554653172},{"type":"v1","addr":"172.21.15.139:6835","nonce":1554653172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6849,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/872072758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":872072758},{"type":"v1","addr":"172.21.15.98:6833","nonce":872072758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:43:17.920717+0000","modified":"2024-09-19T08:43:20.933845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6627},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6627":{"gid":6627,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/837817273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":837817273},{"type":"v1","addr":"172.21.15.98:6835","nonce":837817273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6627,"qdb_cluster":[6627]},"id":6}]} 2024-09-19T08:43:40.752 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 52 2024-09-19T08:43:40.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.752+0000 7fc862929640 1 -- 172.21.15.98:0/1689832271 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc8480679e0 msgr2=0x7fc848069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:40.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.752+0000 7fc862929640 1 --2- 172.21.15.98:0/1689832271 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc8480679e0 0x7fc848069ea0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc84c0013e0 tx=0x7fc84c000ff0 comp rx=0 tx=0).stop 2024-09-19T08:43:40.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.752+0000 7fc862929640 1 -- 172.21.15.98:0/1689832271 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc85c15f2f0 msgr2=0x7fc85c15f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:40.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.752+0000 7fc862929640 1 --2- 172.21.15.98:0/1689832271 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc85c15f2f0 0x7fc85c15f770 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fc85c0d2320 tx=0x7fc85400e960 comp rx=0 tx=0).stop 2024-09-19T08:43:40.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.752+0000 7fc862929640 1 -- 172.21.15.98:0/1689832271 shutdown_connections 2024-09-19T08:43:40.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.752+0000 7fc862929640 1 --2- 172.21.15.98:0/1689832271 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc8480679e0 0x7fc848069ea0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:40.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.752+0000 7fc862929640 1 --2- 172.21.15.98:0/1689832271 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc85c15f2f0 0x7fc85c15f770 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:40.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.752+0000 7fc862929640 1 --2- 172.21.15.98:0/1689832271 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc85c14f9e0 0x7fc85c14fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:40.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.752+0000 7fc862929640 1 --2- 172.21.15.98:0/1689832271 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc844004810 0x7fc85c148260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:40.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.752+0000 7fc862929640 1 -- 172.21.15.98:0/1689832271 >> 172.21.15.98:0/1689832271 conn(0x7fc85c0b2ed0 msgr2=0x7fc85c14c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:40.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.752+0000 7fc862929640 1 -- 172.21.15.98:0/1689832271 shutdown_connections 2024-09-19T08:43:40.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.752+0000 7fc862929640 1 -- 172.21.15.98:0/1689832271 wait complete. 2024-09-19T08:43:40.770 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:43:40.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e92c7640 1 Processor -- start 2024-09-19T08:43:40.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e92c7640 1 -- start start 2024-09-19T08:43:40.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e92c7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f75e406efa0 0x7f75e40ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:40.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e92c7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75e40aefd0 0x7f75e40cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:40.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e92c7640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75e40af4a0 con 0x7f75e406f380 2024-09-19T08:43:40.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e92c7640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f75e415ed00 con 0x7f75e406efa0 2024-09-19T08:43:40.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e92c7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f75e40af780 con 0x7f75e40aefd0 2024-09-19T08:43:40.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e27fc640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f75e406f380 0x7f75e40ae8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:59316/0 (socket says 172.21.15.98:59316) 2024-09-19T08:43:40.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e27fc640 1 -- 172.21.15.98:0/1334464685 learned_addr learned my addr 172.21.15.98:0/1334464685 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:40.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e37fe640 1 --2- 172.21.15.98:0/1334464685 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75e40aefd0 0x7f75e40cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:40.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e2ffd640 1 --2- 172.21.15.98:0/1334464685 >> v2:172.21.15.139:3300/0 conn(0x7f75e406efa0 0x7f75e40ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:40.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e1ffb640 1 -- 172.21.15.98:0/1334464685 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3396591972 0 0) 0x7f75e40af4a0 con 0x7f75e406f380 2024-09-19T08:43:40.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e1ffb640 1 -- 172.21.15.98:0/1334464685 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75cc003530 con 0x7f75e406f380 2024-09-19T08:43:40.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e1ffb640 1 -- 172.21.15.98:0/1334464685 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 817048307 0 0) 0x7f75cc003530 con 0x7f75e406f380 2024-09-19T08:43:40.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e1ffb640 1 -- 172.21.15.98:0/1334464685 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75e40aefd0 msgr2=0x7f75e40cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:40.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e1ffb640 1 --2- 172.21.15.98:0/1334464685 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75e40aefd0 0x7f75e40cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:40.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e1ffb640 1 -- 172.21.15.98:0/1334464685 >> v2:172.21.15.139:3300/0 conn(0x7f75e406efa0 msgr2=0x7f75e40ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:40.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e1ffb640 1 --2- 172.21.15.98:0/1334464685 >> v2:172.21.15.139:3300/0 conn(0x7f75e406efa0 0x7f75e40ce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:40.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e1ffb640 1 -- 172.21.15.98:0/1334464685 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75e40afa60 con 0x7f75e406f380 2024-09-19T08:43:40.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e1ffb640 1 -- 172.21.15.98:0/1334464685 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f75d8001680 con 0x7f75e406f380 2024-09-19T08:43:40.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e1ffb640 1 -- 172.21.15.98:0/1334464685 >> v1:172.21.15.98:6789/0 conn(0x7f75e406f380 legacy=0x7f75e40ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:40.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e1ffb640 1 --2- 172.21.15.98:0/1334464685 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75e40aefd0 0x7f75cc0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:40.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e1ffb640 1 --2- 172.21.15.98:0/1334464685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75cc003f40 0x7f75cc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:40.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e1ffb640 1 --2- 172.21.15.98:0/1334464685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75cc004d50 0x7f75cc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:40.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e1ffb640 1 -- 172.21.15.98:0/1334464685 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f75e415ed00 con 0x7f75cc004d50 2024-09-19T08:43:40.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e1ffb640 1 -- 172.21.15.98:0/1334464685 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f75e40af780 con 0x7f75cc003f40 2024-09-19T08:43:40.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e1ffb640 1 -- 172.21.15.98:0/1334464685 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f75cc00ba20 con 0x7f75e40aefd0 2024-09-19T08:43:40.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e37fe640 1 --2- 172.21.15.98:0/1334464685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75cc004d50 0x7f75cc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:40.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e2ffd640 1 --2- 172.21.15.98:0/1334464685 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75e40aefd0 0x7f75cc0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:40.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e27fc640 1 --2- 172.21.15.98:0/1334464685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75cc003f40 0x7f75cc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:40.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e37fe640 1 -- 172.21.15.98:0/1334464685 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75e40aefd0 msgr2=0x7f75cc0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:40.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e37fe640 1 --2- 172.21.15.98:0/1334464685 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75e40aefd0 0x7f75cc0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:40.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e37fe640 1 -- 172.21.15.98:0/1334464685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75cc003f40 msgr2=0x7f75cc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:40.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e37fe640 1 --2- 172.21.15.98:0/1334464685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75cc003f40 0x7f75cc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:40.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e37fe640 1 -- 172.21.15.98:0/1334464685 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75cc003530 con 0x7f75cc004d50 2024-09-19T08:43:40.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e2ffd640 1 --2- 172.21.15.98:0/1334464685 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75e40aefd0 0x7f75cc0039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:40.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.876+0000 7f75e27fc640 1 --2- 172.21.15.98:0/1334464685 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75cc003f40 0x7f75cc004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:40.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.880+0000 7f75e37fe640 1 --2- 172.21.15.98:0/1334464685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75cc004d50 0x7f75cc0051d0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f75dc009fa0 tx=0x7f75dc01cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af051d9035a737b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:40.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.880+0000 7f75e1ffb640 1 -- 172.21.15.98:0/1334464685 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc0b2070 con 0x7f75cc004d50 2024-09-19T08:43:40.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.880+0000 7f75e1ffb640 1 -- 172.21.15.98:0/1334464685 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75dc0a5cd0 con 0x7f75cc004d50 2024-09-19T08:43:40.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.880+0000 7f75e1ffb640 1 -- 172.21.15.98:0/1334464685 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc009bf0 con 0x7f75cc004d50 2024-09-19T08:43:40.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.880+0000 7f75e92c7640 1 -- 172.21.15.98:0/1334464685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75cc004d50 msgr2=0x7f75cc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:40.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.880+0000 7f75e92c7640 1 --2- 172.21.15.98:0/1334464685 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75cc004d50 0x7f75cc0051d0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f75dc009fa0 tx=0x7f75dc01cc00 comp rx=0 tx=0).stop 2024-09-19T08:43:40.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.880+0000 7f75e37fe640 1 -- 172.21.15.98:0/1334464685 reap_dead start 2024-09-19T08:43:40.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.880+0000 7f75e92c7640 1 -- 172.21.15.98:0/1334464685 shutdown_connections 2024-09-19T08:43:40.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.880+0000 7f75e92c7640 1 -- 172.21.15.98:0/1334464685 >> 172.21.15.98:0/1334464685 conn(0x7f75e40d0980 msgr2=0x7f75e40d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:40.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.880+0000 7f75e92c7640 1 -- 172.21.15.98:0/1334464685 shutdown_connections 2024-09-19T08:43:40.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.880+0000 7f75e92c7640 1 -- 172.21.15.98:0/1334464685 wait complete. 2024-09-19T08:43:40.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.880+0000 7f75e92c7640 1 Processor -- start 2024-09-19T08:43:40.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.880+0000 7f75e92c7640 1 -- start start 2024-09-19T08:43:40.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.884+0000 7f75e92c7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75e4156080 0x7f75e40c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:40.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.884+0000 7f75e92c7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75e40ca4f0 0x7f75e40cc970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:40.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.884+0000 7f75e92c7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75e40cceb0 0x7f75e41f81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:40.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.884+0000 7f75e92c7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f75e40cec10 con 0x7f75e40cceb0 2024-09-19T08:43:40.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.884+0000 7f75e92c7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f75e40cea90 con 0x7f75e40ca4f0 2024-09-19T08:43:40.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.884+0000 7f75e92c7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f75e40ced90 con 0x7f75e4156080 2024-09-19T08:43:40.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.884+0000 7f75e37fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75e40cceb0 0x7f75e41f81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:40.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.884+0000 7f75e37fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75e40cceb0 0x7f75e41f81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39944/0 (socket says 172.21.15.98:39944) 2024-09-19T08:43:40.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.884+0000 7f75e37fe640 1 -- 172.21.15.98:0/2052264189 learned_addr learned my addr 172.21.15.98:0/2052264189 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:40.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.884+0000 7f75e27fc640 1 --2- 172.21.15.98:0/2052264189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75e40ca4f0 0x7f75e40cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:40.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.884+0000 7f75e2ffd640 1 --2- 172.21.15.98:0/2052264189 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75e4156080 0x7f75e40c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:40.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.884+0000 7f75e37fe640 1 -- 172.21.15.98:0/2052264189 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75e4156080 msgr2=0x7f75e40c9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:40.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.884+0000 7f75e37fe640 1 --2- 172.21.15.98:0/2052264189 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75e4156080 0x7f75e40c9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:40.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.884+0000 7f75e37fe640 1 -- 172.21.15.98:0/2052264189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75e40ca4f0 msgr2=0x7f75e40cc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:40.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.884+0000 7f75e37fe640 1 --2- 172.21.15.98:0/2052264189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75e40ca4f0 0x7f75e40cc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:40.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.884+0000 7f75e37fe640 1 -- 172.21.15.98:0/2052264189 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75e40cb120 con 0x7f75e40cceb0 2024-09-19T08:43:40.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.884+0000 7f75e37fe640 1 --2- 172.21.15.98:0/2052264189 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75e40cceb0 0x7f75e41f81d0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f75dc0b0040 tx=0x7f75cc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:40.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.884+0000 7f75cb7fe640 1 -- 172.21.15.98:0/2052264189 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc0b2050 con 0x7f75e40cceb0 2024-09-19T08:43:40.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.884+0000 7f75cb7fe640 1 -- 172.21.15.98:0/2052264189 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75dc0be070 con 0x7f75e40cceb0 2024-09-19T08:43:40.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.884+0000 7f75cb7fe640 1 -- 172.21.15.98:0/2052264189 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc009bf0 con 0x7f75e40cceb0 2024-09-19T08:43:40.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.884+0000 7f75e92c7640 1 -- 172.21.15.98:0/2052264189 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75e40cb400 con 0x7f75e40cceb0 2024-09-19T08:43:40.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.884+0000 7f75e92c7640 1 -- 172.21.15.98:0/2052264189 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75e40cb9c0 con 0x7f75e40cceb0 2024-09-19T08:43:40.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.884+0000 7f75e92c7640 1 -- 172.21.15.98:0/2052264189 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75e40cbbc0 con 0x7f75e40cceb0 2024-09-19T08:43:40.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.884+0000 7f75cb7fe640 1 -- 172.21.15.98:0/2052264189 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f75dc0add00 con 0x7f75e40cceb0 2024-09-19T08:43:40.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.884+0000 7f75cb7fe640 1 --2- 172.21.15.98:0/2052264189 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f75d00679e0 0x7f75d0069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:40.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.884+0000 7f75cb7fe640 1 -- 172.21.15.98:0/2052264189 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7f75dc120de0 con 0x7f75e40cceb0 2024-09-19T08:43:40.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.884+0000 7f75e2ffd640 1 --2- 172.21.15.98:0/2052264189 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f75d00679e0 0x7f75d0069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:40.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.884+0000 7f75e2ffd640 1 --2- 172.21.15.98:0/2052264189 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f75d00679e0 0x7f75d0069ea0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f75e40b24b0 tx=0x7f75d4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:40.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:40.888+0000 7f75cb7fe640 1 -- 172.21.15.98:0/2052264189 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f75e40cbbc0 con 0x7f75e40cceb0 2024-09-19T08:43:41.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.084+0000 7f75e92c7640 1 -- 172.21.15.98:0/2052264189 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75e40af400 con 0x7f75e40cceb0 2024-09-19T08:43:41.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.088+0000 7f75cb7fe640 1 -- 172.21.15.98:0/2052264189 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5225 (secure 0 0 0) 0x7f75dc016950 con 0x7f75e40cceb0 2024-09-19T08:43:41.089 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:41.089 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":52,"btime":"2024-09-19T08:43:37:022736+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3182634136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3182634136},{"type":"v1","addr":"172.21.15.139:6833","nonce":3182634136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5830,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1554653172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1554653172},{"type":"v1","addr":"172.21.15.139:6835","nonce":1554653172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6849,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/872072758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":872072758},{"type":"v1","addr":"172.21.15.98:6833","nonce":872072758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:43:17.920717+0000","modified":"2024-09-19T08:43:20.933845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6627},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6627":{"gid":6627,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/837817273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":837817273},{"type":"v1","addr":"172.21.15.98:6835","nonce":837817273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6627,"qdb_cluster":[6627]},"id":6}]} 2024-09-19T08:43:41.089 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 52 2024-09-19T08:43:41.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.088+0000 7f75e92c7640 1 -- 172.21.15.98:0/2052264189 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f75d00679e0 msgr2=0x7f75d0069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:41.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.088+0000 7f75e92c7640 1 --2- 172.21.15.98:0/2052264189 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f75d00679e0 0x7f75d0069ea0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f75e40b24b0 tx=0x7f75d4008040 comp rx=0 tx=0).stop 2024-09-19T08:43:41.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.088+0000 7f75e92c7640 1 -- 172.21.15.98:0/2052264189 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75e40cceb0 msgr2=0x7f75e41f81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:41.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.088+0000 7f75e92c7640 1 --2- 172.21.15.98:0/2052264189 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75e40cceb0 0x7f75e41f81d0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f75dc0b0040 tx=0x7f75cc005840 comp rx=0 tx=0).stop 2024-09-19T08:43:41.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.088+0000 7f75e92c7640 1 -- 172.21.15.98:0/2052264189 shutdown_connections 2024-09-19T08:43:41.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.088+0000 7f75e92c7640 1 --2- 172.21.15.98:0/2052264189 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f75d00679e0 0x7f75d0069ea0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.088+0000 7f75e92c7640 1 --2- 172.21.15.98:0/2052264189 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75e40cceb0 0x7f75e41f81d0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.088+0000 7f75e92c7640 1 --2- 172.21.15.98:0/2052264189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75e40ca4f0 0x7f75e40cc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.088+0000 7f75e92c7640 1 --2- 172.21.15.98:0/2052264189 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75e4156080 0x7f75e40c9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.088+0000 7f75e92c7640 1 -- 172.21.15.98:0/2052264189 >> 172.21.15.98:0/2052264189 conn(0x7f75e40d0980 msgr2=0x7f75e40ae8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:41.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.088+0000 7f75e92c7640 1 -- 172.21.15.98:0/2052264189 shutdown_connections 2024-09-19T08:43:41.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.088+0000 7f75e92c7640 1 -- 172.21.15.98:0/2052264189 wait complete. 2024-09-19T08:43:41.107 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:43:41.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.216+0000 7f91c0a7b640 1 Processor -- start 2024-09-19T08:43:41.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.216+0000 7f91c0a7b640 1 -- start start 2024-09-19T08:43:41.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.220+0000 7f91c0a7b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f91bc160740 0x7f91bc163170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:41.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.220+0000 7f91c0a7b640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91bc163bf0 con 0x7f91bc1636b0 2024-09-19T08:43:41.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.220+0000 7f91c0a7b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91bc163ed0 con 0x7f91bc06c3a0 2024-09-19T08:43:41.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.220+0000 7f91c0a7b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f91bc0ce320 con 0x7f91bc160740 2024-09-19T08:43:41.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.220+0000 7f91b9d9b640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f91bc1636b0 0x7f91bc06bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:59328/0 (socket says 172.21.15.98:59328) 2024-09-19T08:43:41.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.220+0000 7f91b9d9b640 1 -- 172.21.15.98:0/4033212167 learned_addr learned my addr 172.21.15.98:0/4033212167 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:41.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.220+0000 7f91ba59c640 1 --2- 172.21.15.98:0/4033212167 >> v2:172.21.15.139:3300/0 conn(0x7f91bc160740 0x7f91bc163170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:41.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.220+0000 7f91b959a640 1 -- 172.21.15.98:0/4033212167 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4043244984 0 0) 0x7f91bc163bf0 con 0x7f91bc1636b0 2024-09-19T08:43:41.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.220+0000 7f91b959a640 1 -- 172.21.15.98:0/4033212167 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91a4003530 con 0x7f91bc1636b0 2024-09-19T08:43:41.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.220+0000 7f91b959a640 1 -- 172.21.15.98:0/4033212167 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3696981793 0 0) 0x7f91a4003530 con 0x7f91bc1636b0 2024-09-19T08:43:41.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.220+0000 7f91b959a640 1 -- 172.21.15.98:0/4033212167 >> v2:172.21.15.139:3300/0 conn(0x7f91bc160740 msgr2=0x7f91bc163170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:41.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.220+0000 7f91b959a640 1 --2- 172.21.15.98:0/4033212167 >> v2:172.21.15.139:3300/0 conn(0x7f91bc160740 0x7f91bc163170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.220+0000 7f91b959a640 1 -- 172.21.15.98:0/4033212167 >> v1:172.21.15.139:6789/0 conn(0x7f91bc06c3a0 legacy=0x7f91bc0ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:41.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.220+0000 7f91b959a640 1 -- 172.21.15.98:0/4033212167 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91bc0ce600 con 0x7f91bc1636b0 2024-09-19T08:43:41.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.220+0000 7f91b959a640 1 -- 172.21.15.98:0/4033212167 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f91b0001680 con 0x7f91bc1636b0 2024-09-19T08:43:41.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.220+0000 7f91b959a640 1 -- 172.21.15.98:0/4033212167 >> v1:172.21.15.98:6789/0 conn(0x7f91bc1636b0 legacy=0x7f91bc06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:41.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.220+0000 7f91b959a640 1 --2- 172.21.15.98:0/4033212167 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91a40039c0 0x7f91a4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:41.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.220+0000 7f91b959a640 1 --2- 172.21.15.98:0/4033212167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91a4004810 0x7f91a4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:41.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.220+0000 7f91b959a640 1 --2- 172.21.15.98:0/4033212167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91a40051d0 0x7f91a400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:41.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.220+0000 7f91b959a640 1 -- 172.21.15.98:0/4033212167 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f91bc0ce320 con 0x7f91a40051d0 2024-09-19T08:43:41.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.220+0000 7f91b959a640 1 -- 172.21.15.98:0/4033212167 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f91a400c070 con 0x7f91a4004810 2024-09-19T08:43:41.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.220+0000 7f91b959a640 1 -- 172.21.15.98:0/4033212167 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f91a400c270 con 0x7f91a40039c0 2024-09-19T08:43:41.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.220+0000 7f91bad9d640 1 --2- 172.21.15.98:0/4033212167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91a40051d0 0x7f91a400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:41.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.220+0000 7f91b9d9b640 1 --2- 172.21.15.98:0/4033212167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91a4004810 0x7f91a4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:41.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.220+0000 7f91ba59c640 1 --2- 172.21.15.98:0/4033212167 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91a40039c0 0x7f91a4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:41.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.220+0000 7f91bad9d640 1 -- 172.21.15.98:0/4033212167 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91a40039c0 msgr2=0x7f91a4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:41.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.220+0000 7f91bad9d640 1 --2- 172.21.15.98:0/4033212167 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91a40039c0 0x7f91a4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.220+0000 7f91bad9d640 1 -- 172.21.15.98:0/4033212167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91a4004810 msgr2=0x7f91a4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:41.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.220+0000 7f91bad9d640 1 --2- 172.21.15.98:0/4033212167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91a4004810 0x7f91a4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.220+0000 7f91bad9d640 1 -- 172.21.15.98:0/4033212167 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a4003530 con 0x7f91a40051d0 2024-09-19T08:43:41.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.220+0000 7f91b9d9b640 1 --2- 172.21.15.98:0/4033212167 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91a4004810 0x7f91a4004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:43:41.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.224+0000 7f91bad9d640 1 -- 172.21.15.98:0/4033212167 reap_dead start 2024-09-19T08:43:41.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.224+0000 7f91bad9d640 1 --2- 172.21.15.98:0/4033212167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91a40051d0 0x7f91a400b9d0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f91b4001e70 tx=0x7f91b401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc6ddbecef4f4c1a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:41.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.224+0000 7f91b959a640 1 -- 172.21.15.98:0/4033212167 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b4001a90 con 0x7f91a40051d0 2024-09-19T08:43:41.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.224+0000 7f91b959a640 1 -- 172.21.15.98:0/4033212167 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91b4003bc0 con 0x7f91a40051d0 2024-09-19T08:43:41.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.224+0000 7f91b959a640 1 -- 172.21.15.98:0/4033212167 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b4005020 con 0x7f91a40051d0 2024-09-19T08:43:41.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.224+0000 7f91c0a7b640 1 -- 172.21.15.98:0/4033212167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91a40051d0 msgr2=0x7f91a400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:41.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.224+0000 7f91c0a7b640 1 --2- 172.21.15.98:0/4033212167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91a40051d0 0x7f91a400b9d0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f91b4001e70 tx=0x7f91b401c780 comp rx=0 tx=0).stop 2024-09-19T08:43:41.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.224+0000 7f91c0a7b640 1 -- 172.21.15.98:0/4033212167 shutdown_connections 2024-09-19T08:43:41.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.224+0000 7f91c0a7b640 1 --2- 172.21.15.98:0/4033212167 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91a40051d0 0x7f91a400b9d0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.224+0000 7f91c0a7b640 1 -- 172.21.15.98:0/4033212167 >> 172.21.15.98:0/4033212167 conn(0x7f91bc0b2010 msgr2=0x7f91bc0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:41.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.224+0000 7f91c0a7b640 1 -- 172.21.15.98:0/4033212167 shutdown_connections 2024-09-19T08:43:41.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.224+0000 7f91c0a7b640 1 -- 172.21.15.98:0/4033212167 wait complete. 2024-09-19T08:43:41.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.224+0000 7f91c0a7b640 1 Processor -- start 2024-09-19T08:43:41.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.224+0000 7f91c0a7b640 1 -- start start 2024-09-19T08:43:41.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.228+0000 7f91c0a7b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91a40051d0 0x7f91bc1588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:41.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.228+0000 7f91c0a7b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91bc071930 0x7f91bc072e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:41.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.228+0000 7f91c0a7b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91bc1f93a0 0x7f91bc158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:41.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.228+0000 7f91c0a7b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f91bc0cce40 con 0x7f91bc1f93a0 2024-09-19T08:43:41.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.228+0000 7f91c0a7b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f91bc0cccc0 con 0x7f91a40051d0 2024-09-19T08:43:41.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.228+0000 7f91c0a7b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f91bc163170 con 0x7f91bc071930 2024-09-19T08:43:41.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.228+0000 7f91bad9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91bc1f93a0 0x7f91bc158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:41.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.228+0000 7f91bad9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91bc1f93a0 0x7f91bc158e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39960/0 (socket says 172.21.15.98:39960) 2024-09-19T08:43:41.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.228+0000 7f91bad9d640 1 -- 172.21.15.98:0/1321356607 learned_addr learned my addr 172.21.15.98:0/1321356607 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:41.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.228+0000 7f91ba59c640 1 --2- 172.21.15.98:0/1321356607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91a40051d0 0x7f91bc1588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:41.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.228+0000 7f91b9d9b640 1 --2- 172.21.15.98:0/1321356607 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91bc071930 0x7f91bc072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:41.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.228+0000 7f91bad9d640 1 -- 172.21.15.98:0/1321356607 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91bc071930 msgr2=0x7f91bc072e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:41.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.228+0000 7f91bad9d640 1 --2- 172.21.15.98:0/1321356607 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91bc071930 0x7f91bc072e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.228+0000 7f91bad9d640 1 -- 172.21.15.98:0/1321356607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91a40051d0 msgr2=0x7f91bc1588e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:41.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.228+0000 7f91bad9d640 1 --2- 172.21.15.98:0/1321356607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91a40051d0 0x7f91bc1588e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.228+0000 7f91bad9d640 1 -- 172.21.15.98:0/1321356607 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91bc159d40 con 0x7f91bc1f93a0 2024-09-19T08:43:41.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.228+0000 7f91ba59c640 1 --2- 172.21.15.98:0/1321356607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91a40051d0 0x7f91bc1588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:41.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.228+0000 7f91bad9d640 1 --2- 172.21.15.98:0/1321356607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91bc1f93a0 0x7f91bc158e80 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f91b401cc60 tx=0x7f91a4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:41.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.228+0000 7f91aaffd640 1 -- 172.21.15.98:0/1321356607 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b40012e0 con 0x7f91bc1f93a0 2024-09-19T08:43:41.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.228+0000 7f91aaffd640 1 -- 172.21.15.98:0/1321356607 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91b4003980 con 0x7f91bc1f93a0 2024-09-19T08:43:41.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.228+0000 7f91c0a7b640 1 -- 172.21.15.98:0/1321356607 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91bc071df0 con 0x7f91bc1f93a0 2024-09-19T08:43:41.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.228+0000 7f91aaffd640 1 -- 172.21.15.98:0/1321356607 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b4006d60 con 0x7f91bc1f93a0 2024-09-19T08:43:41.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.228+0000 7f91c0a7b640 1 -- 172.21.15.98:0/1321356607 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91bc0723b0 con 0x7f91bc1f93a0 2024-09-19T08:43:41.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.228+0000 7f91c0a7b640 1 -- 172.21.15.98:0/1321356607 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91bc1634a0 con 0x7f91bc1f93a0 2024-09-19T08:43:41.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.228+0000 7f91aaffd640 1 -- 172.21.15.98:0/1321356607 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f91b40a6480 con 0x7f91bc1f93a0 2024-09-19T08:43:41.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.228+0000 7f91aaffd640 1 --2- 172.21.15.98:0/1321356607 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f919c0679e0 0x7f919c069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:41.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.228+0000 7f91aaffd640 1 -- 172.21.15.98:0/1321356607 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7f91b411f9a0 con 0x7f91bc1f93a0 2024-09-19T08:43:41.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.228+0000 7f91ba59c640 1 --2- 172.21.15.98:0/1321356607 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f919c0679e0 0x7f919c069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:41.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.228+0000 7f91ba59c640 1 --2- 172.21.15.98:0/1321356607 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f919c0679e0 0x7f919c069ea0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f91ac0099a0 tx=0x7f91ac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:41.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.232+0000 7f91aaffd640 1 -- 172.21.15.98:0/1321356607 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f91b40b8060 con 0x7f91bc1f93a0 2024-09-19T08:43:41.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.396+0000 7f91c0a7b640 1 -- 172.21.15.98:0/1321356607 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f91bc069da0 con 0x7f91bc1f93a0 2024-09-19T08:43:41.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.400+0000 7f91aaffd640 1 -- 172.21.15.98:0/1321356607 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16849 (secure 0 0 0) 0x7f91b4016490 con 0x7f91bc1f93a0 2024-09-19T08:43:41.400 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:41.401 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":91,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:43:39.037396+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:43:15.568647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-19T08:43:16.905534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.98:6833/1092838020":"2024-09-20T08:43:36.020574+0000","172.21.15.98:6832/1092838020":"2024-09-20T08:43:36.020574+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:43:41.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.400+0000 7f91c0a7b640 1 -- 172.21.15.98:0/1321356607 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f919c0679e0 msgr2=0x7f919c069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:41.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.400+0000 7f91c0a7b640 1 --2- 172.21.15.98:0/1321356607 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f919c0679e0 0x7f919c069ea0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f91ac0099a0 tx=0x7f91ac008040 comp rx=0 tx=0).stop 2024-09-19T08:43:41.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.400+0000 7f91c0a7b640 1 -- 172.21.15.98:0/1321356607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91bc1f93a0 msgr2=0x7f91bc158e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:41.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.400+0000 7f91c0a7b640 1 --2- 172.21.15.98:0/1321356607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91bc1f93a0 0x7f91bc158e80 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f91b401cc60 tx=0x7f91a4005840 comp rx=0 tx=0).stop 2024-09-19T08:43:41.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.400+0000 7f91c0a7b640 1 -- 172.21.15.98:0/1321356607 shutdown_connections 2024-09-19T08:43:41.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.400+0000 7f91c0a7b640 1 --2- 172.21.15.98:0/1321356607 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f919c0679e0 0x7f919c069ea0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.400+0000 7f91c0a7b640 1 --2- 172.21.15.98:0/1321356607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f91bc1f93a0 0x7f91bc158e80 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.400+0000 7f91c0a7b640 1 --2- 172.21.15.98:0/1321356607 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f91bc071930 0x7f91bc072e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.400+0000 7f91c0a7b640 1 --2- 172.21.15.98:0/1321356607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f91a40051d0 0x7f91bc1588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.400+0000 7f91c0a7b640 1 -- 172.21.15.98:0/1321356607 >> 172.21.15.98:0/1321356607 conn(0x7f91bc0b2010 msgr2=0x7f91bc0cc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:41.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.400+0000 7f91c0a7b640 1 -- 172.21.15.98:0/1321356607 shutdown_connections 2024-09-19T08:43:41.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.400+0000 7f91c0a7b640 1 -- 172.21.15.98:0/1321356607 wait complete. 2024-09-19T08:43:41.420 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T08:43:41.420 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:43:41.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.536+0000 7f47cf504640 1 Processor -- start 2024-09-19T08:43:41.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.536+0000 7f47cf504640 1 -- start start 2024-09-19T08:43:41.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.540+0000 7f47cf504640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f47c8158fd0 0x7f47c80b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:41.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.540+0000 7f47cf504640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47c806c3a0 0x7f47c80ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:41.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.540+0000 7f47cf504640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47c80ccc60 con 0x7f47c80b4360 2024-09-19T08:43:41.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.540+0000 7f47cf504640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f47c81593b0 con 0x7f47c8158fd0 2024-09-19T08:43:41.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.540+0000 7f47cf504640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f47c80ccf40 con 0x7f47c806c3a0 2024-09-19T08:43:41.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.540+0000 7f47cd2a0640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f47c8158fd0 0x7f47c80b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:41.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.540+0000 7f47cca9f640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f47c80b4360 0x7f47c806bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:59332/0 (socket says 172.21.15.98:59332) 2024-09-19T08:43:41.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.540+0000 7f47cca9f640 1 -- 172.21.15.98:0/1551851384 learned_addr learned my addr 172.21.15.98:0/1551851384 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:41.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.540+0000 7f47cd2a0640 1 -- 172.21.15.98:0/1551851384 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47c806c3a0 msgr2=0x7f47c80ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:41.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.540+0000 7f47cd2a0640 1 --2- 172.21.15.98:0/1551851384 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47c806c3a0 0x7f47c80ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.540+0000 7f47cd2a0640 1 -- 172.21.15.98:0/1551851384 >> v1:172.21.15.98:6789/0 conn(0x7f47c80b4360 legacy=0x7f47c806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:41.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.540+0000 7f47cd2a0640 1 -- 172.21.15.98:0/1551851384 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c80cd220 con 0x7f47c8158fd0 2024-09-19T08:43:41.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.540+0000 7f47c7fff640 1 -- 172.21.15.98:0/1551851384 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1817896215 0 0) 0x7f47c80ccc60 con 0x7f47c80b4360 2024-09-19T08:43:41.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.540+0000 7f47cd2a0640 1 --2- 172.21.15.98:0/1551851384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47c8158fd0 0x7f47c80b3e20 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f47c00097c0 tx=0x7f47c001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34ca9a86efcb741 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:41.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.540+0000 7f47c7fff640 1 -- 172.21.15.98:0/1551851384 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c00b1070 con 0x7f47c8158fd0 2024-09-19T08:43:41.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.540+0000 7f47c7fff640 1 -- 172.21.15.98:0/1551851384 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47c00a8050 con 0x7f47c8158fd0 2024-09-19T08:43:41.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.540+0000 7f47c7fff640 1 -- 172.21.15.98:0/1551851384 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c00ac860 con 0x7f47c8158fd0 2024-09-19T08:43:41.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.544+0000 7f47cf504640 1 -- 172.21.15.98:0/1551851384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47c8158fd0 msgr2=0x7f47c80b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:41.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.544+0000 7f47cf504640 1 --2- 172.21.15.98:0/1551851384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47c8158fd0 0x7f47c80b3e20 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f47c00097c0 tx=0x7f47c001c3e0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.544+0000 7f47cf504640 1 -- 172.21.15.98:0/1551851384 shutdown_connections 2024-09-19T08:43:41.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.544+0000 7f47cf504640 1 --2- 172.21.15.98:0/1551851384 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47c806c3a0 0x7f47c80ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.544+0000 7f47cf504640 1 --2- 172.21.15.98:0/1551851384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47c8158fd0 0x7f47c80b3e20 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.544+0000 7f47cf504640 1 -- 172.21.15.98:0/1551851384 >> 172.21.15.98:0/1551851384 conn(0x7f47c80b2010 msgr2=0x7f47c80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:41.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.544+0000 7f47cf504640 1 -- 172.21.15.98:0/1551851384 shutdown_connections 2024-09-19T08:43:41.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.544+0000 7f47cf504640 1 -- 172.21.15.98:0/1551851384 wait complete. 2024-09-19T08:43:41.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.544+0000 7f47cf504640 1 Processor -- start 2024-09-19T08:43:41.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.544+0000 7f47cf504640 1 -- start start 2024-09-19T08:43:41.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.544+0000 7f47cf504640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47c806c3a0 0x7f47c815edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:41.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.544+0000 7f47cf504640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47c80b4360 0x7f47c81fad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:41.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.544+0000 7f47cf504640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47c8158fd0 0x7f47c81541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:41.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.544+0000 7f47cf504640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f47c80cf710 con 0x7f47c8158fd0 2024-09-19T08:43:41.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.544+0000 7f47cf504640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f47c80cf590 con 0x7f47c806c3a0 2024-09-19T08:43:41.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.544+0000 7f47cf504640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f47c80cf890 con 0x7f47c80b4360 2024-09-19T08:43:41.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.544+0000 7f47cdaa1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47c8158fd0 0x7f47c81541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:41.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.544+0000 7f47cdaa1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47c8158fd0 0x7f47c81541d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39984/0 (socket says 172.21.15.98:39984) 2024-09-19T08:43:41.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.544+0000 7f47cdaa1640 1 -- 172.21.15.98:0/1014678077 learned_addr learned my addr 172.21.15.98:0/1014678077 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:41.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.544+0000 7f47cca9f640 1 --2- 172.21.15.98:0/1014678077 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47c80b4360 0x7f47c81fad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:41.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.544+0000 7f47cd2a0640 1 --2- 172.21.15.98:0/1014678077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47c806c3a0 0x7f47c815edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:41.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.544+0000 7f47cdaa1640 1 -- 172.21.15.98:0/1014678077 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47c80b4360 msgr2=0x7f47c81fad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:41.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.544+0000 7f47cdaa1640 1 --2- 172.21.15.98:0/1014678077 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47c80b4360 0x7f47c81fad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.544+0000 7f47cdaa1640 1 -- 172.21.15.98:0/1014678077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47c806c3a0 msgr2=0x7f47c815edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:41.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.544+0000 7f47cdaa1640 1 --2- 172.21.15.98:0/1014678077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47c806c3a0 0x7f47c815edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.544+0000 7f47cdaa1640 1 -- 172.21.15.98:0/1014678077 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c81fb270 con 0x7f47c8158fd0 2024-09-19T08:43:41.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.544+0000 7f47cd2a0640 1 --2- 172.21.15.98:0/1014678077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47c806c3a0 0x7f47c815edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:43:41.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.544+0000 7f47cdaa1640 1 --2- 172.21.15.98:0/1014678077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47c8158fd0 0x7f47c81541d0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f47b800b970 tx=0x7f47b800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:41.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.544+0000 7f47c5ffb640 1 -- 172.21.15.98:0/1014678077 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b8019070 con 0x7f47c8158fd0 2024-09-19T08:43:41.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.544+0000 7f47c5ffb640 1 -- 172.21.15.98:0/1014678077 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47b800cb60 con 0x7f47c8158fd0 2024-09-19T08:43:41.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.544+0000 7f47cf504640 1 -- 172.21.15.98:0/1014678077 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c815d250 con 0x7f47c8158fd0 2024-09-19T08:43:41.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.544+0000 7f47c5ffb640 1 -- 172.21.15.98:0/1014678077 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b8014440 con 0x7f47c8158fd0 2024-09-19T08:43:41.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.544+0000 7f47cf504640 1 -- 172.21.15.98:0/1014678077 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c815f960 con 0x7f47c8158fd0 2024-09-19T08:43:41.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.548+0000 7f47cf504640 1 -- 172.21.15.98:0/1014678077 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47c80ce840 con 0x7f47c8158fd0 2024-09-19T08:43:41.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.548+0000 7f47c5ffb640 1 -- 172.21.15.98:0/1014678077 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f47b801c070 con 0x7f47c8158fd0 2024-09-19T08:43:41.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.548+0000 7f47c5ffb640 1 --2- 172.21.15.98:0/1014678077 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f47b40679e0 0x7f47b4069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:41.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.548+0000 7f47c5ffb640 1 -- 172.21.15.98:0/1014678077 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7f47b8085b30 con 0x7f47c8158fd0 2024-09-19T08:43:41.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.548+0000 7f47cd2a0640 1 --2- 172.21.15.98:0/1014678077 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f47b40679e0 0x7f47b4069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:41.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.548+0000 7f47cd2a0640 1 --2- 172.21.15.98:0/1014678077 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f47b40679e0 0x7f47b4069ea0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f47c00ae000 tx=0x7f47c00a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:41.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.552+0000 7f47c5ffb640 1 -- 172.21.15.98:0/1014678077 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f47c81fb270 con 0x7f47c8158fd0 2024-09-19T08:43:41.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.752+0000 7f47cf504640 1 -- 172.21.15.98:0/1014678077 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47c815d420 con 0x7f47c8158fd0 2024-09-19T08:43:41.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.752+0000 7f47c5ffb640 1 -- 172.21.15.98:0/1014678077 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5225 (secure 0 0 0) 0x7f47b8056570 con 0x7f47c8158fd0 2024-09-19T08:43:41.754 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:41.754 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":52,"btime":"2024-09-19T08:43:37:022736+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3182634136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3182634136},{"type":"v1","addr":"172.21.15.139:6833","nonce":3182634136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5830,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1554653172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1554653172},{"type":"v1","addr":"172.21.15.139:6835","nonce":1554653172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6849,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/872072758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":872072758},{"type":"v1","addr":"172.21.15.98:6833","nonce":872072758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:43:17.920717+0000","modified":"2024-09-19T08:43:20.933845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6627},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6627":{"gid":6627,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/837817273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":837817273},{"type":"v1","addr":"172.21.15.98:6835","nonce":837817273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6627,"qdb_cluster":[6627]},"id":6}]} 2024-09-19T08:43:41.754 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 52 2024-09-19T08:43:41.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.752+0000 7f47cf504640 1 -- 172.21.15.98:0/1014678077 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f47b40679e0 msgr2=0x7f47b4069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:41.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.752+0000 7f47cf504640 1 --2- 172.21.15.98:0/1014678077 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f47b40679e0 0x7f47b4069ea0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f47c00ae000 tx=0x7f47c00a7000 comp rx=0 tx=0).stop 2024-09-19T08:43:41.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.752+0000 7f47cf504640 1 -- 172.21.15.98:0/1014678077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47c8158fd0 msgr2=0x7f47c81541d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:41.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.752+0000 7f47cf504640 1 --2- 172.21.15.98:0/1014678077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47c8158fd0 0x7f47c81541d0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f47b800b970 tx=0x7f47b800be30 comp rx=0 tx=0).stop 2024-09-19T08:43:41.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.752+0000 7f47cf504640 1 -- 172.21.15.98:0/1014678077 shutdown_connections 2024-09-19T08:43:41.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.752+0000 7f47cf504640 1 --2- 172.21.15.98:0/1014678077 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f47b40679e0 0x7f47b4069ea0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.752+0000 7f47cf504640 1 --2- 172.21.15.98:0/1014678077 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47c8158fd0 0x7f47c81541d0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.752+0000 7f47cf504640 1 --2- 172.21.15.98:0/1014678077 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f47c80b4360 0x7f47c81fad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.752+0000 7f47cf504640 1 --2- 172.21.15.98:0/1014678077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47c806c3a0 0x7f47c815edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.752+0000 7f47cf504640 1 -- 172.21.15.98:0/1014678077 >> 172.21.15.98:0/1014678077 conn(0x7f47c80b2010 msgr2=0x7f47c8159d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:41.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.752+0000 7f47cf504640 1 -- 172.21.15.98:0/1014678077 shutdown_connections 2024-09-19T08:43:41.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.752+0000 7f47cf504640 1 -- 172.21.15.98:0/1014678077 wait complete. 2024-09-19T08:43:41.773 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:43:41.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.892+0000 7f9ad9983640 1 Processor -- start 2024-09-19T08:43:41.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.892+0000 7f9ad9983640 1 -- start start 2024-09-19T08:43:41.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.892+0000 7f9ad9983640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9ad406c3a0 0x7f9ad40ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:41.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.892+0000 7f9ad9983640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9ad40ce6e0 0x7f9ad40b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:41.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.892+0000 7f9ad9983640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ad415fcf0 con 0x7f9ad4158e40 2024-09-19T08:43:41.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.892+0000 7f9ad9983640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f9ad415ef30 con 0x7f9ad40ce6e0 2024-09-19T08:43:41.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.892+0000 7f9ad9983640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f9ad415f210 con 0x7f9ad406c3a0 2024-09-19T08:43:41.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.892+0000 7f9ad37fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9ad40ce6e0 0x7f9ad40b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:41.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.892+0000 7f9ad2ffd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f9ad4158e40 0x7f9ad406bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:59342/0 (socket says 172.21.15.98:59342) 2024-09-19T08:43:41.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.892+0000 7f9ad37fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9ad40ce6e0 0x7f9ad40b3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39988/0 (socket says 172.21.15.98:39988) 2024-09-19T08:43:41.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.892+0000 7f9ad2ffd640 1 -- 172.21.15.98:0/3760689666 learned_addr learned my addr 172.21.15.98:0/3760689666 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:41.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.892+0000 7f9ad27fc640 1 --2- 172.21.15.98:0/3760689666 >> v2:172.21.15.139:3300/0 conn(0x7f9ad406c3a0 0x7f9ad40ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:41.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.892+0000 7f9ad37fe640 1 -- 172.21.15.98:0/3760689666 >> v2:172.21.15.139:3300/0 conn(0x7f9ad406c3a0 msgr2=0x7f9ad40ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:41.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.892+0000 7f9ad37fe640 1 --2- 172.21.15.98:0/3760689666 >> v2:172.21.15.139:3300/0 conn(0x7f9ad406c3a0 0x7f9ad40ce1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.892+0000 7f9ad37fe640 1 -- 172.21.15.98:0/3760689666 >> v1:172.21.15.98:6789/0 conn(0x7f9ad4158e40 legacy=0x7f9ad406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:41.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.892+0000 7f9ad37fe640 1 -- 172.21.15.98:0/3760689666 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad415f4f0 con 0x7f9ad40ce6e0 2024-09-19T08:43:41.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.892+0000 7f9ad1ffb640 1 -- 172.21.15.98:0/3760689666 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 936820387 0 0) 0x7f9ad415fcf0 con 0x7f9ad4158e40 2024-09-19T08:43:41.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.896+0000 7f9ad37fe640 1 --2- 172.21.15.98:0/3760689666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ad40ce6e0 0x7f9ad40b3560 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4070d20 tx=0x7f9ac401f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a4ee44df9a4525f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:41.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.896+0000 7f9ad1ffb640 1 -- 172.21.15.98:0/3760689666 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac40b4070 con 0x7f9ad40ce6e0 2024-09-19T08:43:41.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.896+0000 7f9ad1ffb640 1 -- 172.21.15.98:0/3760689666 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ac40af600 con 0x7f9ad40ce6e0 2024-09-19T08:43:41.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.896+0000 7f9ad1ffb640 1 -- 172.21.15.98:0/3760689666 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac40af9d0 con 0x7f9ad40ce6e0 2024-09-19T08:43:41.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.896+0000 7f9ad9983640 1 -- 172.21.15.98:0/3760689666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ad40ce6e0 msgr2=0x7f9ad40b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:41.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.896+0000 7f9ad9983640 1 --2- 172.21.15.98:0/3760689666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ad40ce6e0 0x7f9ad40b3560 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4070d20 tx=0x7f9ac401f110 comp rx=0 tx=0).stop 2024-09-19T08:43:41.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.896+0000 7f9ad9983640 1 -- 172.21.15.98:0/3760689666 shutdown_connections 2024-09-19T08:43:41.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.896+0000 7f9ad9983640 1 --2- 172.21.15.98:0/3760689666 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ad40ce6e0 0x7f9ad40b3560 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.896+0000 7f9ad9983640 1 --2- 172.21.15.98:0/3760689666 >> v2:172.21.15.139:3300/0 conn(0x7f9ad406c3a0 0x7f9ad40ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.896+0000 7f9ad9983640 1 -- 172.21.15.98:0/3760689666 >> 172.21.15.98:0/3760689666 conn(0x7f9ad40b2010 msgr2=0x7f9ad40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:41.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.896+0000 7f9ad9983640 1 -- 172.21.15.98:0/3760689666 shutdown_connections 2024-09-19T08:43:41.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.896+0000 7f9ad9983640 1 -- 172.21.15.98:0/3760689666 wait complete. 2024-09-19T08:43:41.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.896+0000 7f9ad9983640 1 Processor -- start 2024-09-19T08:43:41.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.896+0000 7f9ad9983640 1 -- start start 2024-09-19T08:43:41.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.896+0000 7f9ad9983640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ad406c3a0 0x7f9ad4155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:41.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.896+0000 7f9ad9983640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ad40ce6e0 0x7f9ad41549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:41.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.896+0000 7f9ad9983640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ad4158e40 0x7f9ad4148db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:41.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.896+0000 7f9ad9983640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad40cee20 con 0x7f9ad40ce6e0 2024-09-19T08:43:41.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.896+0000 7f9ad9983640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad4159220 con 0x7f9ad4158e40 2024-09-19T08:43:41.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.896+0000 7f9ad9983640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9ad40ceb10 con 0x7f9ad406c3a0 2024-09-19T08:43:41.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.896+0000 7f9ad27fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ad40ce6e0 0x7f9ad41549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:41.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.896+0000 7f9ad27fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ad40ce6e0 0x7f9ad41549e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40002/0 (socket says 172.21.15.98:40002) 2024-09-19T08:43:41.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.896+0000 7f9ad27fc640 1 -- 172.21.15.98:0/996914835 learned_addr learned my addr 172.21.15.98:0/996914835 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:41.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.900+0000 7f9ad27fc640 1 -- 172.21.15.98:0/996914835 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ad406c3a0 msgr2=0x7f9ad4155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:41.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.900+0000 7f9ad27fc640 1 --2- 172.21.15.98:0/996914835 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ad406c3a0 0x7f9ad4155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.900+0000 7f9ad37fe640 1 --2- 172.21.15.98:0/996914835 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ad4158e40 0x7f9ad4148db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:41.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.900+0000 7f9ad27fc640 1 -- 172.21.15.98:0/996914835 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ad4158e40 msgr2=0x7f9ad4148db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:41.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.900+0000 7f9ad27fc640 1 --2- 172.21.15.98:0/996914835 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ad4158e40 0x7f9ad4148db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:41.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.900+0000 7f9ad27fc640 1 -- 172.21.15.98:0/996914835 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad4155910 con 0x7f9ad40ce6e0 2024-09-19T08:43:41.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.900+0000 7f9ad27fc640 1 --2- 172.21.15.98:0/996914835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ad40ce6e0 0x7f9ad41549e0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00d830 tx=0x7f9abc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:41.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.900+0000 7f9acbfff640 1 -- 172.21.15.98:0/996914835 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc00b660 con 0x7f9ad40ce6e0 2024-09-19T08:43:41.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.900+0000 7f9acbfff640 1 -- 172.21.15.98:0/996914835 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9abc019070 con 0x7f9ad40ce6e0 2024-09-19T08:43:41.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.900+0000 7f9ad9983640 1 -- 172.21.15.98:0/996914835 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ad4155bf0 con 0x7f9ad40ce6e0 2024-09-19T08:43:41.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.900+0000 7f9acbfff640 1 -- 172.21.15.98:0/996914835 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc00ba00 con 0x7f9ad40ce6e0 2024-09-19T08:43:41.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.900+0000 7f9ad9983640 1 -- 172.21.15.98:0/996914835 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ad41f7bd0 con 0x7f9ad40ce6e0 2024-09-19T08:43:41.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.900+0000 7f9ad9983640 1 -- 172.21.15.98:0/996914835 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ad41f76e0 con 0x7f9ad40ce6e0 2024-09-19T08:43:41.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.900+0000 7f9acbfff640 1 -- 172.21.15.98:0/996914835 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f9abc00bba0 con 0x7f9ad40ce6e0 2024-09-19T08:43:41.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.900+0000 7f9acbfff640 1 --2- 172.21.15.98:0/996914835 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9ab40679e0 0x7f9ab4069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:41.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.900+0000 7f9acbfff640 1 -- 172.21.15.98:0/996914835 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7f9abc086240 con 0x7f9ad40ce6e0 2024-09-19T08:43:41.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.900+0000 7f9ad2ffd640 1 --2- 172.21.15.98:0/996914835 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9ab40679e0 0x7f9ab4069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:41.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.900+0000 7f9ad2ffd640 1 --2- 172.21.15.98:0/996914835 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9ab40679e0 0x7f9ab4069ea0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9acc0015a0 tx=0x7f9acc007970 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:41.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:41.904+0000 7f9acbfff640 1 -- 172.21.15.98:0/996914835 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9abc056ca0 con 0x7f9ad40ce6e0 2024-09-19T08:43:42.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.092+0000 7f9ad9983640 1 -- 172.21.15.98:0/996914835 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ad40cddd0 con 0x7f9ad40ce6e0 2024-09-19T08:43:42.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.092+0000 7f9acbfff640 1 -- 172.21.15.98:0/996914835 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5225 (secure 0 0 0) 0x7f9abc0565a0 con 0x7f9ad40ce6e0 2024-09-19T08:43:42.096 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:42.096 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":52,"btime":"2024-09-19T08:43:37:022736+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3182634136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3182634136},{"type":"v1","addr":"172.21.15.139:6833","nonce":3182634136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5830,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1554653172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1554653172},{"type":"v1","addr":"172.21.15.139:6835","nonce":1554653172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6849,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/872072758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":872072758},{"type":"v1","addr":"172.21.15.98:6833","nonce":872072758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:43:17.920717+0000","modified":"2024-09-19T08:43:20.933845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6627},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6627":{"gid":6627,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/837817273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":837817273},{"type":"v1","addr":"172.21.15.98:6835","nonce":837817273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6627,"qdb_cluster":[6627]},"id":6}]} 2024-09-19T08:43:42.096 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 52 2024-09-19T08:43:42.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.096+0000 7f9ad9983640 1 -- 172.21.15.98:0/996914835 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9ab40679e0 msgr2=0x7f9ab4069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:42.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.096+0000 7f9ad9983640 1 --2- 172.21.15.98:0/996914835 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9ab40679e0 0x7f9ab4069ea0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9acc0015a0 tx=0x7f9acc007970 comp rx=0 tx=0).stop 2024-09-19T08:43:42.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.096+0000 7f9ad9983640 1 -- 172.21.15.98:0/996914835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ad40ce6e0 msgr2=0x7f9ad41549e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:42.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.096+0000 7f9ad9983640 1 --2- 172.21.15.98:0/996914835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ad40ce6e0 0x7f9ad41549e0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00d830 tx=0x7f9abc00dcf0 comp rx=0 tx=0).stop 2024-09-19T08:43:42.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.096+0000 7f9ad9983640 1 -- 172.21.15.98:0/996914835 shutdown_connections 2024-09-19T08:43:42.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.096+0000 7f9ad9983640 1 --2- 172.21.15.98:0/996914835 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9ab40679e0 0x7f9ab4069ea0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:42.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.096+0000 7f9ad9983640 1 --2- 172.21.15.98:0/996914835 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ad4158e40 0x7f9ad4148db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:42.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.096+0000 7f9ad9983640 1 --2- 172.21.15.98:0/996914835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ad40ce6e0 0x7f9ad41549e0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:42.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.096+0000 7f9ad9983640 1 --2- 172.21.15.98:0/996914835 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ad406c3a0 0x7f9ad4155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:42.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.096+0000 7f9ad9983640 1 -- 172.21.15.98:0/996914835 >> 172.21.15.98:0/996914835 conn(0x7f9ad40b2010 msgr2=0x7f9ad40b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:42.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.096+0000 7f9ad9983640 1 -- 172.21.15.98:0/996914835 shutdown_connections 2024-09-19T08:43:42.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.096+0000 7f9ad9983640 1 -- 172.21.15.98:0/996914835 wait complete. 2024-09-19T08:43:42.113 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:43:42.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.224+0000 7fadabc3d640 1 Processor -- start 2024-09-19T08:43:42.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.224+0000 7fadabc3d640 1 -- start start 2024-09-19T08:43:42.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.224+0000 7fadabc3d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fada415c550 0x7fada406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:42.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.224+0000 7fadabc3d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fada40cfad0 0x7fada40723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:42.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.224+0000 7fadabc3d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fada406d310 con 0x7fada406c1d0 2024-09-19T08:43:42.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.224+0000 7fadabc3d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fada406c640 con 0x7fada415c550 2024-09-19T08:43:42.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.224+0000 7fadabc3d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fada406d5f0 con 0x7fada40cfad0 2024-09-19T08:43:42.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.224+0000 7fada99d9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fada415c550 0x7fada406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:42.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.224+0000 7fadaa1da640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fada40cfad0 0x7fada40723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:42.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.224+0000 7fada99d9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fada415c550 0x7fada406bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:33102/0 (socket says 172.21.15.98:33102) 2024-09-19T08:43:42.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.224+0000 7fada91d8640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fada406c1d0 0x7fada40cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:44116/0 (socket says 172.21.15.98:44116) 2024-09-19T08:43:42.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.224+0000 7fada99d9640 1 -- 172.21.15.98:0/3076403482 learned_addr learned my addr 172.21.15.98:0/3076403482 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:42.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.224+0000 7fada89d7640 1 -- 172.21.15.98:0/3076403482 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2402770751 0 0) 0x7fada406d310 con 0x7fada406c1d0 2024-09-19T08:43:42.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.224+0000 7fada89d7640 1 -- 172.21.15.98:0/3076403482 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad90003530 con 0x7fada406c1d0 2024-09-19T08:43:42.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.224+0000 7fada99d9640 1 -- 172.21.15.98:0/3076403482 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fada40cfad0 msgr2=0x7fada40723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:42.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.224+0000 7fada99d9640 1 --2- 172.21.15.98:0/3076403482 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fada40cfad0 0x7fada40723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:42.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.224+0000 7fada99d9640 1 -- 172.21.15.98:0/3076403482 >> v1:172.21.15.139:6789/0 conn(0x7fada406c1d0 legacy=0x7fada40cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:42.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.224+0000 7fada99d9640 1 -- 172.21.15.98:0/3076403482 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada406d8d0 con 0x7fada415c550 2024-09-19T08:43:42.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.224+0000 7fadaa1da640 1 --2- 172.21.15.98:0/3076403482 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fada40cfad0 0x7fada40723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:43:42.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.228+0000 7fada99d9640 1 --2- 172.21.15.98:0/3076403482 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fada415c550 0x7fada406bc90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fada0009870 tx=0x7fada001c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=482d541a8f252fcb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:42.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.228+0000 7fada89d7640 1 -- 172.21.15.98:0/3076403482 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada00b2070 con 0x7fada415c550 2024-09-19T08:43:42.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.228+0000 7fada89d7640 1 -- 172.21.15.98:0/3076403482 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fada00a9070 con 0x7fada415c550 2024-09-19T08:43:42.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.228+0000 7fada89d7640 1 -- 172.21.15.98:0/3076403482 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada00ad860 con 0x7fada415c550 2024-09-19T08:43:42.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.228+0000 7fadabc3d640 1 -- 172.21.15.98:0/3076403482 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fada415c550 msgr2=0x7fada406bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:42.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.228+0000 7fadabc3d640 1 --2- 172.21.15.98:0/3076403482 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fada415c550 0x7fada406bc90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fada0009870 tx=0x7fada001c560 comp rx=0 tx=0).stop 2024-09-19T08:43:42.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.228+0000 7fadabc3d640 1 -- 172.21.15.98:0/3076403482 shutdown_connections 2024-09-19T08:43:42.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.228+0000 7fadabc3d640 1 --2- 172.21.15.98:0/3076403482 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fada40cfad0 0x7fada40723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:42.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.228+0000 7fadabc3d640 1 --2- 172.21.15.98:0/3076403482 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fada415c550 0x7fada406bc90 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:42.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.228+0000 7fadabc3d640 1 -- 172.21.15.98:0/3076403482 >> 172.21.15.98:0/3076403482 conn(0x7fada40b2010 msgr2=0x7fada40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:42.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.228+0000 7fadabc3d640 1 -- 172.21.15.98:0/3076403482 shutdown_connections 2024-09-19T08:43:42.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.228+0000 7fadabc3d640 1 -- 172.21.15.98:0/3076403482 wait complete. 2024-09-19T08:43:42.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.228+0000 7fadabc3d640 1 Processor -- start 2024-09-19T08:43:42.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.228+0000 7fadabc3d640 1 -- start start 2024-09-19T08:43:42.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.232+0000 7fadabc3d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fada406c1d0 0x7fada414f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:42.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.232+0000 7fadabc3d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fada40cfad0 0x7fada40b0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:42.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.232+0000 7fadabc3d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fada415c550 0x7fada40b0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:42.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.232+0000 7fadabc3d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fada4073380 con 0x7fada406c1d0 2024-09-19T08:43:42.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.232+0000 7fadabc3d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fada415cb40 con 0x7fada415c550 2024-09-19T08:43:42.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.232+0000 7fadabc3d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fada4073500 con 0x7fada40cfad0 2024-09-19T08:43:42.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.232+0000 7fada99d9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fada406c1d0 0x7fada414f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:42.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.232+0000 7fada99d9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fada406c1d0 0x7fada414f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40010/0 (socket says 172.21.15.98:40010) 2024-09-19T08:43:42.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.232+0000 7fada99d9640 1 -- 172.21.15.98:0/1995645492 learned_addr learned my addr 172.21.15.98:0/1995645492 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:42.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.232+0000 7fada91d8640 1 --2- 172.21.15.98:0/1995645492 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fada40cfad0 0x7fada40b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:42.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.232+0000 7fada99d9640 1 -- 172.21.15.98:0/1995645492 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fada40cfad0 msgr2=0x7fada40b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:42.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.232+0000 7fada99d9640 1 --2- 172.21.15.98:0/1995645492 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fada40cfad0 0x7fada40b0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:42.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.232+0000 7fada99d9640 1 -- 172.21.15.98:0/1995645492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fada415c550 msgr2=0x7fada40b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:42.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.232+0000 7fadaa1da640 1 --2- 172.21.15.98:0/1995645492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fada415c550 0x7fada40b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:42.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.232+0000 7fada99d9640 1 --2- 172.21.15.98:0/1995645492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fada415c550 0x7fada40b0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:42.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.232+0000 7fada99d9640 1 -- 172.21.15.98:0/1995645492 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada40b3ec0 con 0x7fada406c1d0 2024-09-19T08:43:42.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.232+0000 7fada99d9640 1 --2- 172.21.15.98:0/1995645492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fada406c1d0 0x7fada414f450 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fada00b0040 tx=0x7fada0019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:42.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.232+0000 7fad9a7fc640 1 -- 172.21.15.98:0/1995645492 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada00bc070 con 0x7fada406c1d0 2024-09-19T08:43:42.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.232+0000 7fad9a7fc640 1 -- 172.21.15.98:0/1995645492 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fada0003820 con 0x7fada406c1d0 2024-09-19T08:43:42.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.232+0000 7fadabc3d640 1 -- 172.21.15.98:0/1995645492 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada40b4130 con 0x7fada406c1d0 2024-09-19T08:43:42.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.232+0000 7fad9a7fc640 1 -- 172.21.15.98:0/1995645492 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada00b2040 con 0x7fada406c1d0 2024-09-19T08:43:42.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.232+0000 7fadabc3d640 1 -- 172.21.15.98:0/1995645492 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fada414f990 con 0x7fada406c1d0 2024-09-19T08:43:42.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.232+0000 7fadabc3d640 1 -- 172.21.15.98:0/1995645492 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fada41504e0 con 0x7fada406c1d0 2024-09-19T08:43:42.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.232+0000 7fad9a7fc640 1 -- 172.21.15.98:0/1995645492 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fada00ab070 con 0x7fada406c1d0 2024-09-19T08:43:42.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.232+0000 7fad9a7fc640 1 --2- 172.21.15.98:0/1995645492 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fad880679e0 0x7fad88069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:42.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.232+0000 7fad9a7fc640 1 -- 172.21.15.98:0/1995645492 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7fada011ec30 con 0x7fada406c1d0 2024-09-19T08:43:42.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.232+0000 7fada91d8640 1 --2- 172.21.15.98:0/1995645492 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fad880679e0 0x7fad88069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:42.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.236+0000 7fada91d8640 1 --2- 172.21.15.98:0/1995645492 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fad880679e0 0x7fad88069ea0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fad94007fd0 tx=0x7fad94009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:42.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.236+0000 7fad9a7fc640 1 -- 172.21.15.98:0/1995645492 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fada0016150 con 0x7fada406c1d0 2024-09-19T08:43:42.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.396+0000 7fadabc3d640 1 -- 172.21.15.98:0/1995645492 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fada40723a0 con 0x7fada406c1d0 2024-09-19T08:43:42.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.400+0000 7fad9a7fc640 1 -- 172.21.15.98:0/1995645492 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16849 (secure 0 0 0) 0x7fada00eee20 con 0x7fada406c1d0 2024-09-19T08:43:42.402 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:42.402 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":91,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:43:39.037396+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:43:15.568647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-19T08:43:16.905534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.98:6833/1092838020":"2024-09-20T08:43:36.020574+0000","172.21.15.98:6832/1092838020":"2024-09-20T08:43:36.020574+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:43:42.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.400+0000 7fadabc3d640 1 -- 172.21.15.98:0/1995645492 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fad880679e0 msgr2=0x7fad88069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:42.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.400+0000 7fadabc3d640 1 --2- 172.21.15.98:0/1995645492 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fad880679e0 0x7fad88069ea0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fad94007fd0 tx=0x7fad94009040 comp rx=0 tx=0).stop 2024-09-19T08:43:42.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.404+0000 7fadabc3d640 1 -- 172.21.15.98:0/1995645492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fada406c1d0 msgr2=0x7fada414f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:42.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.404+0000 7fadabc3d640 1 --2- 172.21.15.98:0/1995645492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fada406c1d0 0x7fada414f450 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fada00b0040 tx=0x7fada0019300 comp rx=0 tx=0).stop 2024-09-19T08:43:42.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.404+0000 7fadabc3d640 1 -- 172.21.15.98:0/1995645492 shutdown_connections 2024-09-19T08:43:42.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.404+0000 7fadabc3d640 1 --2- 172.21.15.98:0/1995645492 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fad880679e0 0x7fad88069ea0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:42.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.404+0000 7fadabc3d640 1 --2- 172.21.15.98:0/1995645492 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fada415c550 0x7fada40b0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:42.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.404+0000 7fadabc3d640 1 --2- 172.21.15.98:0/1995645492 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fada40cfad0 0x7fada40b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:42.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.404+0000 7fadabc3d640 1 --2- 172.21.15.98:0/1995645492 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fada406c1d0 0x7fada414f450 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:42.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.404+0000 7fadabc3d640 1 -- 172.21.15.98:0/1995645492 >> 172.21.15.98:0/1995645492 conn(0x7fada40b2010 msgr2=0x7fada4071120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:42.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.404+0000 7fadabc3d640 1 -- 172.21.15.98:0/1995645492 shutdown_connections 2024-09-19T08:43:42.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.404+0000 7fadabc3d640 1 -- 172.21.15.98:0/1995645492 wait complete. 2024-09-19T08:43:42.420 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-19T08:43:42.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.540+0000 7f665b00f640 1 Processor -- start 2024-09-19T08:43:42.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.540+0000 7f665b00f640 1 -- start start 2024-09-19T08:43:42.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.540+0000 7f665b00f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f665415c550 0x7f665406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:42.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.540+0000 7f665b00f640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f66540cfad0 0x7f66540723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:42.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.540+0000 7f665b00f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f665406d310 con 0x7f665406c1d0 2024-09-19T08:43:42.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.540+0000 7f665b00f640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f665406c610 con 0x7f66540cfad0 2024-09-19T08:43:42.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.540+0000 7f665b00f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f665406d5f0 con 0x7f665415c550 2024-09-19T08:43:42.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.540+0000 7f66595ac640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f66540cfad0 0x7f66540723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:42.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.540+0000 7f66595ac640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f66540cfad0 0x7f66540723a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40018/0 (socket says 172.21.15.98:40018) 2024-09-19T08:43:42.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.540+0000 7f66595ac640 1 -- 172.21.15.98:0/2718888259 learned_addr learned my addr 172.21.15.98:0/2718888259 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:42.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.540+0000 7f66595ac640 1 -- 172.21.15.98:0/2718888259 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f665415c550 msgr2=0x7f665406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:42.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.540+0000 7f66595ac640 1 --2- 172.21.15.98:0/2718888259 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f665415c550 0x7f665406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:42.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.540+0000 7f66595ac640 1 -- 172.21.15.98:0/2718888259 >> v1:172.21.15.139:6789/0 conn(0x7f665406c1d0 legacy=0x7f66540cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:42.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.540+0000 7f66595ac640 1 -- 172.21.15.98:0/2718888259 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f665406d8d0 con 0x7f66540cfad0 2024-09-19T08:43:42.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.544+0000 7f66595ac640 1 --2- 172.21.15.98:0/2718888259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66540cfad0 0x7f66540723a0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f664c009e50 tx=0x7f664c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5abde218e59789e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:42.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.544+0000 7f664b7fe640 1 -- 172.21.15.98:0/2718888259 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664c0a5780 con 0x7f66540cfad0 2024-09-19T08:43:42.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.544+0000 7f664b7fe640 1 -- 172.21.15.98:0/2718888259 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f664c0a5920 con 0x7f66540cfad0 2024-09-19T08:43:42.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.544+0000 7f664b7fe640 1 -- 172.21.15.98:0/2718888259 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664c0a5cf0 con 0x7f66540cfad0 2024-09-19T08:43:42.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.544+0000 7f665b00f640 1 -- 172.21.15.98:0/2718888259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66540cfad0 msgr2=0x7f66540723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:42.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.544+0000 7f665b00f640 1 --2- 172.21.15.98:0/2718888259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66540cfad0 0x7f66540723a0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f664c009e50 tx=0x7f664c01c9f0 comp rx=0 tx=0).stop 2024-09-19T08:43:42.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.544+0000 7f665b00f640 1 -- 172.21.15.98:0/2718888259 shutdown_connections 2024-09-19T08:43:42.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.544+0000 7f665b00f640 1 --2- 172.21.15.98:0/2718888259 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66540cfad0 0x7f66540723a0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:42.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.544+0000 7f665b00f640 1 --2- 172.21.15.98:0/2718888259 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f665415c550 0x7f665406bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:42.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.544+0000 7f665b00f640 1 -- 172.21.15.98:0/2718888259 >> 172.21.15.98:0/2718888259 conn(0x7f66540b2010 msgr2=0x7f66540b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:42.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.544+0000 7f665b00f640 1 -- 172.21.15.98:0/2718888259 shutdown_connections 2024-09-19T08:43:42.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.544+0000 7f665b00f640 1 -- 172.21.15.98:0/2718888259 wait complete. 2024-09-19T08:43:42.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.544+0000 7f665b00f640 1 Processor -- start 2024-09-19T08:43:42.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.544+0000 7f665b00f640 1 -- start start 2024-09-19T08:43:42.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.544+0000 7f665b00f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f665406c1d0 0x7f665415dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:42.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.544+0000 7f665b00f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66540cfad0 0x7f665415e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:42.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.544+0000 7f665b00f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f665415c550 0x7f6654158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:42.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.544+0000 7f665b00f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6654073480 con 0x7f66540cfad0 2024-09-19T08:43:42.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.544+0000 7f665b00f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6654073300 con 0x7f665415c550 2024-09-19T08:43:42.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.544+0000 7f665b00f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6654073600 con 0x7f665406c1d0 2024-09-19T08:43:42.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.544+0000 7f664bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66540cfad0 0x7f665415e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:42.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.544+0000 7f664bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66540cfad0 0x7f665415e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40020/0 (socket says 172.21.15.98:40020) 2024-09-19T08:43:42.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.544+0000 7f664bfff640 1 -- 172.21.15.98:0/806408980 learned_addr learned my addr 172.21.15.98:0/806408980 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:42.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.544+0000 7f664bfff640 1 -- 172.21.15.98:0/806408980 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f665406c1d0 msgr2=0x7f665415dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:42.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.544+0000 7f664bfff640 1 --2- 172.21.15.98:0/806408980 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f665406c1d0 0x7f665415dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:42.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.544+0000 7f664bfff640 1 -- 172.21.15.98:0/806408980 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f665415c550 msgr2=0x7f6654158b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:42.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.544+0000 7f664bfff640 1 --2- 172.21.15.98:0/806408980 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f665415c550 0x7f6654158b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:42.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.544+0000 7f664bfff640 1 -- 172.21.15.98:0/806408980 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6654157a80 con 0x7f66540cfad0 2024-09-19T08:43:42.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.544+0000 7f664bfff640 1 --2- 172.21.15.98:0/806408980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66540cfad0 0x7f665415e150 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f664400c440 tx=0x7f664400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:42.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.544+0000 7f66497fa640 1 -- 172.21.15.98:0/806408980 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6644016020 con 0x7f66540cfad0 2024-09-19T08:43:42.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.544+0000 7f66497fa640 1 -- 172.21.15.98:0/806408980 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6644012830 con 0x7f66540cfad0 2024-09-19T08:43:42.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.544+0000 7f665b00f640 1 -- 172.21.15.98:0/806408980 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6654157d60 con 0x7f66540cfad0 2024-09-19T08:43:42.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.544+0000 7f66497fa640 1 -- 172.21.15.98:0/806408980 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6644012c20 con 0x7f66540cfad0 2024-09-19T08:43:42.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.544+0000 7f665b00f640 1 -- 172.21.15.98:0/806408980 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6654158320 con 0x7f66540cfad0 2024-09-19T08:43:42.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.548+0000 7f665b00f640 1 -- 172.21.15.98:0/806408980 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66540723a0 con 0x7f66540cfad0 2024-09-19T08:43:42.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.548+0000 7f66497fa640 1 -- 172.21.15.98:0/806408980 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f6644019020 con 0x7f66540cfad0 2024-09-19T08:43:42.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.548+0000 7f66497fa640 1 --2- 172.21.15.98:0/806408980 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f66400679e0 0x7f6640069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:42.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.548+0000 7f66497fa640 1 -- 172.21.15.98:0/806408980 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5795+0+0 (secure 0 0 0) 0x7f6644085e80 con 0x7f66540cfad0 2024-09-19T08:43:42.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.548+0000 7f6658dab640 1 --2- 172.21.15.98:0/806408980 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f66400679e0 0x7f6640069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:42.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.552+0000 7f6658dab640 1 --2- 172.21.15.98:0/806408980 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f66400679e0 0x7f6640069ea0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f66541f8700 tx=0x7f6650006c00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:42.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.552+0000 7f66497fa640 1 -- 172.21.15.98:0/806408980 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6644020020 con 0x7f66540cfad0 2024-09-19T08:43:42.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.740+0000 7f665b00f640 1 -- 172.21.15.98:0/806408980 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f66540cd4e0 con 0x7f66540cfad0 2024-09-19T08:43:42.742 INFO:tasks.ceph.mon.a.smithi098.stderr:2024-09-19T08:43:42.740+0000 7f5e49017640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T08:43:42.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.744+0000 7f66497fa640 1 -- 172.21.15.98:0/806408980 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v53) ==== 161+0+0 (secure 0 0 0) 0x7f66440182c0 con 0x7f66540cfad0 2024-09-19T08:43:42.745 INFO:teuthology.orchestra.run.smithi098.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T08:43:42.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.744+0000 7f665b00f640 1 -- 172.21.15.98:0/806408980 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f66400679e0 msgr2=0x7f6640069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:42.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.744+0000 7f665b00f640 1 --2- 172.21.15.98:0/806408980 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f66400679e0 0x7f6640069ea0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f66541f8700 tx=0x7f6650006c00 comp rx=0 tx=0).stop 2024-09-19T08:43:42.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.744+0000 7f665b00f640 1 -- 172.21.15.98:0/806408980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66540cfad0 msgr2=0x7f665415e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:42.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.744+0000 7f665b00f640 1 --2- 172.21.15.98:0/806408980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66540cfad0 0x7f665415e150 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f664400c440 tx=0x7f664400c900 comp rx=0 tx=0).stop 2024-09-19T08:43:42.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.744+0000 7f665b00f640 1 -- 172.21.15.98:0/806408980 shutdown_connections 2024-09-19T08:43:42.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.744+0000 7f665b00f640 1 --2- 172.21.15.98:0/806408980 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f66400679e0 0x7f6640069ea0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:42.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.744+0000 7f665b00f640 1 --2- 172.21.15.98:0/806408980 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f665415c550 0x7f6654158b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:42.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.744+0000 7f665b00f640 1 --2- 172.21.15.98:0/806408980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66540cfad0 0x7f665415e150 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:42.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.744+0000 7f665b00f640 1 --2- 172.21.15.98:0/806408980 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f665406c1d0 0x7f665415dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:42.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.744+0000 7f665b00f640 1 -- 172.21.15.98:0/806408980 >> 172.21.15.98:0/806408980 conn(0x7f66540b2010 msgr2=0x7f66541599c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:42.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.744+0000 7f665b00f640 1 -- 172.21.15.98:0/806408980 shutdown_connections 2024-09-19T08:43:42.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.744+0000 7f665b00f640 1 -- 172.21.15.98:0/806408980 wait complete. 2024-09-19T08:43:42.750 INFO:tasks.ceph.mds.a.smithi098.stderr: -1402> 2024-09-19T08:43:24.169+0000 7f0e63cb8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T08:43:42.766 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T08:43:42.785 INFO:tasks.ceph.mds.a.smithi098.stdout:starting mds.a at 2024-09-19T08:43:42.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.884+0000 7f313479d640 1 Processor -- start 2024-09-19T08:43:42.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.884+0000 7f313479d640 1 -- start start 2024-09-19T08:43:42.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.884+0000 7f313479d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f312c0d27c0 0x7f312c0d3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:42.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.884+0000 7f313479d640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f312c15eae0 con 0x7f312c0d38a0 2024-09-19T08:43:42.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.884+0000 7f313479d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f312c15edc0 con 0x7f312c1415a0 2024-09-19T08:43:42.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.884+0000 7f313479d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f312c15f0a0 con 0x7f312c0d27c0 2024-09-19T08:43:42.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.884+0000 7f3132d3a640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f312c0d38a0 0x7f312c0b8790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:59360/0 (socket says 172.21.15.98:59360) 2024-09-19T08:43:42.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.884+0000 7f3132d3a640 1 -- 172.21.15.98:0/581955816 learned_addr learned my addr 172.21.15.98:0/581955816 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:42.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.884+0000 7f3131537640 1 -- 172.21.15.98:0/581955816 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3680739673 0 0) 0x7f312c15eae0 con 0x7f312c0d38a0 2024-09-19T08:43:42.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.884+0000 7f3131d38640 1 --2- 172.21.15.98:0/581955816 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f312c0d27c0 0x7f312c0d3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:42.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.884+0000 7f3131537640 1 -- 172.21.15.98:0/581955816 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3118003530 con 0x7f312c0d38a0 2024-09-19T08:43:42.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.884+0000 7f3131537640 1 -- 172.21.15.98:0/581955816 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 600727785 0 0) 0x7f3118003530 con 0x7f312c0d38a0 2024-09-19T08:43:42.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.884+0000 7f3131537640 1 -- 172.21.15.98:0/581955816 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f312c0d27c0 msgr2=0x7f312c0d3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:42.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.884+0000 7f3131537640 1 --2- 172.21.15.98:0/581955816 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f312c0d27c0 0x7f312c0d3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:42.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.884+0000 7f3131537640 1 -- 172.21.15.98:0/581955816 >> v1:172.21.15.139:6789/0 conn(0x7f312c1415a0 legacy=0x7f312c0d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:42.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.884+0000 7f3131537640 1 -- 172.21.15.98:0/581955816 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f312c15f380 con 0x7f312c0d38a0 2024-09-19T08:43:42.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.884+0000 7f3131537640 1 -- 172.21.15.98:0/581955816 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f3124001a00 con 0x7f312c0d38a0 2024-09-19T08:43:42.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.888+0000 7f3131537640 1 -- 172.21.15.98:0/581955816 >> v1:172.21.15.98:6789/0 conn(0x7f312c0d38a0 legacy=0x7f312c0b8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:42.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.888+0000 7f3131537640 1 --2- 172.21.15.98:0/581955816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31180039f0 0x7f3118003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:42.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.888+0000 7f3131537640 1 --2- 172.21.15.98:0/581955816 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f312c0d27c0 0x7f3118004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:42.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.888+0000 7f3131537640 1 --2- 172.21.15.98:0/581955816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3118004d50 0x7f31180051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:42.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.888+0000 7f3131537640 1 -- 172.21.15.98:0/581955816 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f312c15f0a0 con 0x7f31180039f0 2024-09-19T08:43:42.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.888+0000 7f3131537640 1 -- 172.21.15.98:0/581955816 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f311800ba80 con 0x7f3118004d50 2024-09-19T08:43:42.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.888+0000 7f3131537640 1 -- 172.21.15.98:0/581955816 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f311800bc80 con 0x7f312c0d27c0 2024-09-19T08:43:42.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.888+0000 7f3132539640 1 --2- 172.21.15.98:0/581955816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31180039f0 0x7f3118003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:42.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.888+0000 7f3132539640 1 -- 172.21.15.98:0/581955816 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f312c0d27c0 msgr2=0x7f3118004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:42.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.888+0000 7f3132539640 1 --2- 172.21.15.98:0/581955816 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f312c0d27c0 0x7f3118004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:42.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.888+0000 7f3132539640 1 -- 172.21.15.98:0/581955816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3118004d50 msgr2=0x7f31180051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:42.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.888+0000 7f3132d3a640 1 --2- 172.21.15.98:0/581955816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3118004d50 0x7f31180051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:42.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.888+0000 7f3132539640 1 --2- 172.21.15.98:0/581955816 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3118004d50 0x7f31180051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:42.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.888+0000 7f3132539640 1 -- 172.21.15.98:0/581955816 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3118003530 con 0x7f31180039f0 2024-09-19T08:43:42.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.888+0000 7f3132539640 1 --2- 172.21.15.98:0/581955816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31180039f0 0x7f3118003e10 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f3120001750 tx=0x7f312001c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818d06989d284f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:42.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.888+0000 7f3131537640 1 -- 172.21.15.98:0/581955816 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31200013a0 con 0x7f31180039f0 2024-09-19T08:43:42.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.888+0000 7f3131537640 1 -- 172.21.15.98:0/581955816 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31200050d0 con 0x7f31180039f0 2024-09-19T08:43:42.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.888+0000 7f3131537640 1 -- 172.21.15.98:0/581955816 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31200054a0 con 0x7f31180039f0 2024-09-19T08:43:42.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.888+0000 7f313479d640 1 -- 172.21.15.98:0/581955816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31180039f0 msgr2=0x7f3118003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:42.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.888+0000 7f313479d640 1 --2- 172.21.15.98:0/581955816 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f31180039f0 0x7f3118003e10 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f3120001750 tx=0x7f312001c0c0 comp rx=0 tx=0).stop 2024-09-19T08:43:42.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.888+0000 7f3132d3a640 1 -- 172.21.15.98:0/581955816 reap_dead start 2024-09-19T08:43:42.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.888+0000 7f313479d640 1 -- 172.21.15.98:0/581955816 shutdown_connections 2024-09-19T08:43:42.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.888+0000 7f313479d640 1 -- 172.21.15.98:0/581955816 >> 172.21.15.98:0/581955816 conn(0x7f312c074c70 msgr2=0x7f312c075080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:42.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.888+0000 7f313479d640 1 -- 172.21.15.98:0/581955816 shutdown_connections 2024-09-19T08:43:42.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.888+0000 7f313479d640 1 -- 172.21.15.98:0/581955816 wait complete. 2024-09-19T08:43:42.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.892+0000 7f313479d640 1 Processor -- start 2024-09-19T08:43:42.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.892+0000 7f313479d640 1 -- start start 2024-09-19T08:43:42.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.892+0000 7f313479d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f31180039f0 0x7f312c13feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:42.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.892+0000 7f313479d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f312c0cc960 0x7f312c0ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:42.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.892+0000 7f313479d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f312c14f2e0 0x7f312c14f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:42.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.892+0000 7f313479d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f312c16a110 con 0x7f312c14f2e0 2024-09-19T08:43:42.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.892+0000 7f313479d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f312c0d3f00 con 0x7f312c0cc960 2024-09-19T08:43:42.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.892+0000 7f313479d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f312c16a290 con 0x7f31180039f0 2024-09-19T08:43:42.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.892+0000 7f3132d3a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f312c14f2e0 0x7f312c14f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:42.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.892+0000 7f3132d3a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f312c14f2e0 0x7f312c14f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40052/0 (socket says 172.21.15.98:40052) 2024-09-19T08:43:42.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.892+0000 7f3132d3a640 1 -- 172.21.15.98:0/3404056876 learned_addr learned my addr 172.21.15.98:0/3404056876 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:42.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.892+0000 7f3132d3a640 1 -- 172.21.15.98:0/3404056876 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f31180039f0 msgr2=0x7f312c13feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:42.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.892+0000 7f3132d3a640 1 --2- 172.21.15.98:0/3404056876 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f31180039f0 0x7f312c13feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:42.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.892+0000 7f3132d3a640 1 -- 172.21.15.98:0/3404056876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f312c0cc960 msgr2=0x7f312c0ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:42.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.892+0000 7f3132d3a640 1 --2- 172.21.15.98:0/3404056876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f312c0cc960 0x7f312c0ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:42.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.892+0000 7f3132d3a640 1 -- 172.21.15.98:0/3404056876 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f312c1403f0 con 0x7f312c14f2e0 2024-09-19T08:43:42.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.892+0000 7f3132d3a640 1 --2- 172.21.15.98:0/3404056876 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f312c14f2e0 0x7f312c14f760 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f3124000f90 tx=0x7f312400eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:42.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.892+0000 7f311effd640 1 -- 172.21.15.98:0/3404056876 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3124019070 con 0x7f312c14f2e0 2024-09-19T08:43:42.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.892+0000 7f313479d640 1 -- 172.21.15.98:0/3404056876 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f312c14ffa0 con 0x7f312c14f2e0 2024-09-19T08:43:42.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.892+0000 7f311effd640 1 -- 172.21.15.98:0/3404056876 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3124007030 con 0x7f312c14f2e0 2024-09-19T08:43:42.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.892+0000 7f311effd640 1 -- 172.21.15.98:0/3404056876 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312400c630 con 0x7f312c14f2e0 2024-09-19T08:43:42.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.892+0000 7f313479d640 1 -- 172.21.15.98:0/3404056876 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f312c151e50 con 0x7f312c14f2e0 2024-09-19T08:43:42.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.892+0000 7f313479d640 1 -- 172.21.15.98:0/3404056876 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f312c0b8790 con 0x7f312c14f2e0 2024-09-19T08:43:42.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.896+0000 7f311effd640 1 -- 172.21.15.98:0/3404056876 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f312401c070 con 0x7f312c14f2e0 2024-09-19T08:43:42.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.896+0000 7f311effd640 1 --2- 172.21.15.98:0/3404056876 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3110067a10 0x7f3110069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:42.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.896+0000 7f311effd640 1 -- 172.21.15.98:0/3404056876 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5881+0+0 (secure 0 0 0) 0x7f3124086970 con 0x7f312c14f2e0 2024-09-19T08:43:42.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.896+0000 7f3132539640 1 --2- 172.21.15.98:0/3404056876 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3110067a10 0x7f3110069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:42.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.896+0000 7f3132539640 1 --2- 172.21.15.98:0/3404056876 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3110067a10 0x7f3110069ed0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f31180057f0 tx=0x7f3120001130 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:42.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:42.900+0000 7f311effd640 1 -- 172.21.15.98:0/3404056876 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3124023020 con 0x7f312c14f2e0 2024-09-19T08:43:43.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.084+0000 7f313479d640 1 -- 172.21.15.98:0/3404056876 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f312c169210 con 0x7f312c14f2e0 2024-09-19T08:43:43.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.752+0000 7f311effd640 1 -- 172.21.15.98:0/3404056876 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v54) ==== 104+0+0 (secure 0 0 0) 0x7f31240572d0 con 0x7f312c14f2e0 2024-09-19T08:43:43.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.756+0000 7f313479d640 1 -- 172.21.15.98:0/3404056876 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3110067a10 msgr2=0x7f3110069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:43.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.756+0000 7f313479d640 1 --2- 172.21.15.98:0/3404056876 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3110067a10 0x7f3110069ed0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f31180057f0 tx=0x7f3120001130 comp rx=0 tx=0).stop 2024-09-19T08:43:43.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.756+0000 7f313479d640 1 -- 172.21.15.98:0/3404056876 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f312c14f2e0 msgr2=0x7f312c14f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:43.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.756+0000 7f313479d640 1 --2- 172.21.15.98:0/3404056876 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f312c14f2e0 0x7f312c14f760 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f3124000f90 tx=0x7f312400eba0 comp rx=0 tx=0).stop 2024-09-19T08:43:43.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.756+0000 7f313479d640 1 -- 172.21.15.98:0/3404056876 shutdown_connections 2024-09-19T08:43:43.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.756+0000 7f313479d640 1 --2- 172.21.15.98:0/3404056876 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3110067a10 0x7f3110069ed0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:43.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.756+0000 7f313479d640 1 --2- 172.21.15.98:0/3404056876 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f312c14f2e0 0x7f312c14f760 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:43.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.756+0000 7f313479d640 1 --2- 172.21.15.98:0/3404056876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f312c0cc960 0x7f312c0ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:43.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.756+0000 7f313479d640 1 --2- 172.21.15.98:0/3404056876 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f31180039f0 0x7f312c13feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:43.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.756+0000 7f313479d640 1 -- 172.21.15.98:0/3404056876 >> 172.21.15.98:0/3404056876 conn(0x7f312c074c70 msgr2=0x7f312c1676a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:43.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.756+0000 7f313479d640 1 -- 172.21.15.98:0/3404056876 shutdown_connections 2024-09-19T08:43:43.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.756+0000 7f313479d640 1 -- 172.21.15.98:0/3404056876 wait complete. 2024-09-19T08:43:43.778 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T08:43:43.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.944+0000 7f8b51a78640 1 Processor -- start 2024-09-19T08:43:43.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.944+0000 7f8b51a78640 1 -- start start 2024-09-19T08:43:43.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.944+0000 7f8b51a78640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8b4c15f6e0 0x7f8b4c06d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:43.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.944+0000 7f8b51a78640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b4c073d90 0x7f8b4c06da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:43.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.944+0000 7f8b51a78640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b4c15baf0 con 0x7f8b4c15fac0 2024-09-19T08:43:43.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.944+0000 7f8b51a78640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f8b4c15c8b0 con 0x7f8b4c15f6e0 2024-09-19T08:43:43.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.944+0000 7f8b51a78640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8b4c0742d0 con 0x7f8b4c073d90 2024-09-19T08:43:43.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.944+0000 7f8b4b7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8b4c15f6e0 0x7f8b4c06d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:43.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.944+0000 7f8b4b7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8b4c15f6e0 0x7f8b4c06d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40066/0 (socket says 172.21.15.98:40066) 2024-09-19T08:43:43.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.944+0000 7f8b4b7fe640 1 -- 172.21.15.98:0/2266420777 learned_addr learned my addr 172.21.15.98:0/2266420777 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:43.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.944+0000 7f8b4bfff640 1 --2- 172.21.15.98:0/2266420777 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b4c073d90 0x7f8b4c06da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:43.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.944+0000 7f8b4b7fe640 1 -- 172.21.15.98:0/2266420777 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b4c073d90 msgr2=0x7f8b4c06da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:43.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.944+0000 7f8b4b7fe640 1 --2- 172.21.15.98:0/2266420777 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b4c073d90 0x7f8b4c06da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:43.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.944+0000 7f8b4b7fe640 1 -- 172.21.15.98:0/2266420777 >> v1:172.21.15.139:6789/0 conn(0x7f8b4c15fac0 legacy=0x7f8b4c073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:43.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.944+0000 7f8b4b7fe640 1 -- 172.21.15.98:0/2266420777 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b4c15bdd0 con 0x7f8b4c15f6e0 2024-09-19T08:43:43.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.944+0000 7f8b4bfff640 1 --2- 172.21.15.98:0/2266420777 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b4c073d90 0x7f8b4c06da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:43.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.948+0000 7f8b4b7fe640 1 --2- 172.21.15.98:0/2266420777 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b4c15f6e0 0x7f8b4c06d4c0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c06e080 tx=0x7f8b3c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc088b7639d9c86a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:43.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.948+0000 7f8b4a7fc640 1 -- 172.21.15.98:0/2266420777 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c0b0070 con 0x7f8b4c15f6e0 2024-09-19T08:43:43.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.948+0000 7f8b4a7fc640 1 -- 172.21.15.98:0/2266420777 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b3c0ab600 con 0x7f8b4c15f6e0 2024-09-19T08:43:43.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.948+0000 7f8b4a7fc640 1 -- 172.21.15.98:0/2266420777 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c0ab9d0 con 0x7f8b4c15f6e0 2024-09-19T08:43:43.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.948+0000 7f8b51a78640 1 -- 172.21.15.98:0/2266420777 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b4c15f6e0 msgr2=0x7f8b4c06d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:43.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.948+0000 7f8b51a78640 1 --2- 172.21.15.98:0/2266420777 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b4c15f6e0 0x7f8b4c06d4c0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c06e080 tx=0x7f8b3c01c450 comp rx=0 tx=0).stop 2024-09-19T08:43:43.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.948+0000 7f8b51a78640 1 -- 172.21.15.98:0/2266420777 shutdown_connections 2024-09-19T08:43:43.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.948+0000 7f8b51a78640 1 --2- 172.21.15.98:0/2266420777 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b4c073d90 0x7f8b4c06da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:43.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.948+0000 7f8b51a78640 1 --2- 172.21.15.98:0/2266420777 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b4c15f6e0 0x7f8b4c06d4c0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:43.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.948+0000 7f8b51a78640 1 -- 172.21.15.98:0/2266420777 >> 172.21.15.98:0/2266420777 conn(0x7f8b4c06c090 msgr2=0x7f8b4c06c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:43.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.948+0000 7f8b51a78640 1 -- 172.21.15.98:0/2266420777 shutdown_connections 2024-09-19T08:43:43.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.948+0000 7f8b51a78640 1 -- 172.21.15.98:0/2266420777 wait complete. 2024-09-19T08:43:43.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.948+0000 7f8b51a78640 1 Processor -- start 2024-09-19T08:43:43.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.948+0000 7f8b51a78640 1 -- start start 2024-09-19T08:43:43.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.948+0000 7f8b51a78640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b4c073d90 0x7f8b4c1f6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:43.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.948+0000 7f8b51a78640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b4c15f6e0 0x7f8b4c1606a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:43.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.948+0000 7f8b51a78640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b4c15fac0 0x7f8b4c0aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:43.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.948+0000 7f8b51a78640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c0cefc0 con 0x7f8b4c15f6e0 2024-09-19T08:43:43.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.948+0000 7f8b51a78640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c0cee40 con 0x7f8b4c073d90 2024-09-19T08:43:43.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.948+0000 7f8b51a78640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8b4c0cf140 con 0x7f8b4c15fac0 2024-09-19T08:43:43.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.948+0000 7f8b4affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b4c15f6e0 0x7f8b4c1606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:43.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.948+0000 7f8b4affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b4c15f6e0 0x7f8b4c1606a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40082/0 (socket says 172.21.15.98:40082) 2024-09-19T08:43:43.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.948+0000 7f8b4affd640 1 -- 172.21.15.98:0/2411503905 learned_addr learned my addr 172.21.15.98:0/2411503905 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:43.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.952+0000 7f8b4bfff640 1 --2- 172.21.15.98:0/2411503905 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b4c15fac0 0x7f8b4c0aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:43.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.952+0000 7f8b4affd640 1 -- 172.21.15.98:0/2411503905 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b4c15fac0 msgr2=0x7f8b4c0aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:43.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.952+0000 7f8b4b7fe640 1 --2- 172.21.15.98:0/2411503905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b4c073d90 0x7f8b4c1f6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:43.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.952+0000 7f8b4affd640 1 --2- 172.21.15.98:0/2411503905 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b4c15fac0 0x7f8b4c0aeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:43.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.952+0000 7f8b4affd640 1 -- 172.21.15.98:0/2411503905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b4c073d90 msgr2=0x7f8b4c1f6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:43.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.952+0000 7f8b4affd640 1 --2- 172.21.15.98:0/2411503905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b4c073d90 0x7f8b4c1f6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:43.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.952+0000 7f8b4affd640 1 -- 172.21.15.98:0/2411503905 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b4c160f30 con 0x7f8b4c15f6e0 2024-09-19T08:43:43.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.952+0000 7f8b4bfff640 1 --2- 172.21.15.98:0/2411503905 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b4c15fac0 0x7f8b4c0aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:43.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.952+0000 7f8b4affd640 1 --2- 172.21.15.98:0/2411503905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b4c15f6e0 0x7f8b4c1606a0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f8b4000c450 tx=0x7f8b4000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:43.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.952+0000 7f8b33fff640 1 -- 172.21.15.98:0/2411503905 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b40012830 con 0x7f8b4c15f6e0 2024-09-19T08:43:43.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.952+0000 7f8b33fff640 1 -- 172.21.15.98:0/2411503905 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b40017070 con 0x7f8b4c15f6e0 2024-09-19T08:43:43.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.952+0000 7f8b51a78640 1 -- 172.21.15.98:0/2411503905 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b4c0b0790 con 0x7f8b4c15f6e0 2024-09-19T08:43:43.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.952+0000 7f8b33fff640 1 -- 172.21.15.98:0/2411503905 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b40012c30 con 0x7f8b4c15f6e0 2024-09-19T08:43:43.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.952+0000 7f8b51a78640 1 -- 172.21.15.98:0/2411503905 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b4c160be0 con 0x7f8b4c15f6e0 2024-09-19T08:43:43.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.952+0000 7f8b51a78640 1 -- 172.21.15.98:0/2411503905 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b4c0cf5a0 con 0x7f8b4c15f6e0 2024-09-19T08:43:43.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.952+0000 7f8b33fff640 1 -- 172.21.15.98:0/2411503905 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f8b4001c460 con 0x7f8b4c15f6e0 2024-09-19T08:43:43.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.952+0000 7f8b33fff640 1 --2- 172.21.15.98:0/2411503905 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8b380679e0 0x7f8b38069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:43.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.952+0000 7f8b33fff640 1 -- 172.21.15.98:0/2411503905 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5881+0+0 (secure 0 0 0) 0x7f8b400866f0 con 0x7f8b4c15f6e0 2024-09-19T08:43:43.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.952+0000 7f8b4b7fe640 1 --2- 172.21.15.98:0/2411503905 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8b380679e0 0x7f8b38069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:43.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.952+0000 7f8b4b7fe640 1 --2- 172.21.15.98:0/2411503905 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8b380679e0 0x7f8b38069ea0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c0ad000 tx=0x7f8b3c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:43.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:43.956+0000 7f8b33fff640 1 -- 172.21.15.98:0/2411503905 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f8b40057150 con 0x7f8b4c15f6e0 2024-09-19T08:43:44.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:44.120+0000 7f8b51a78640 1 -- 172.21.15.98:0/2411503905 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8b4c06d6b0 con 0x7f8b4c15f6e0 2024-09-19T08:43:45.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.068+0000 7f8b33fff640 1 -- 172.21.15.98:0/2411503905 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v93) ==== 181+0+0 (secure 0 0 0) 0x7f8b40056a50 con 0x7f8b4c15f6e0 2024-09-19T08:43:45.070 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_metadata' removed 2024-09-19T08:43:45.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.072+0000 7f8b51a78640 1 -- 172.21.15.98:0/2411503905 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8b380679e0 msgr2=0x7f8b38069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:45.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.072+0000 7f8b51a78640 1 --2- 172.21.15.98:0/2411503905 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8b380679e0 0x7f8b38069ea0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c0ad000 tx=0x7f8b3c008040 comp rx=0 tx=0).stop 2024-09-19T08:43:45.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.072+0000 7f8b51a78640 1 -- 172.21.15.98:0/2411503905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b4c15f6e0 msgr2=0x7f8b4c1606a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:45.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.072+0000 7f8b51a78640 1 --2- 172.21.15.98:0/2411503905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b4c15f6e0 0x7f8b4c1606a0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f8b4000c450 tx=0x7f8b4000c910 comp rx=0 tx=0).stop 2024-09-19T08:43:45.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.076+0000 7f8b51a78640 1 -- 172.21.15.98:0/2411503905 shutdown_connections 2024-09-19T08:43:45.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.076+0000 7f8b51a78640 1 --2- 172.21.15.98:0/2411503905 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8b380679e0 0x7f8b38069ea0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:45.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.076+0000 7f8b51a78640 1 --2- 172.21.15.98:0/2411503905 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b4c15fac0 0x7f8b4c0aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:45.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.076+0000 7f8b51a78640 1 --2- 172.21.15.98:0/2411503905 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b4c15f6e0 0x7f8b4c1606a0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:45.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.076+0000 7f8b51a78640 1 --2- 172.21.15.98:0/2411503905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b4c073d90 0x7f8b4c1f6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:45.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.076+0000 7f8b51a78640 1 -- 172.21.15.98:0/2411503905 >> 172.21.15.98:0/2411503905 conn(0x7f8b4c06c090 msgr2=0x7f8b4c15b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:45.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.076+0000 7f8b51a78640 1 -- 172.21.15.98:0/2411503905 shutdown_connections 2024-09-19T08:43:45.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.076+0000 7f8b51a78640 1 -- 172.21.15.98:0/2411503905 wait complete. 2024-09-19T08:43:45.100 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T08:43:45.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.228+0000 7f76038d1640 1 Processor -- start 2024-09-19T08:43:45.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.228+0000 7f76038d1640 1 -- start start 2024-09-19T08:43:45.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.228+0000 7f76038d1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75fc06dbd0 0x7f75fc073680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:45.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.228+0000 7f76038d1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f75fc15fac0 0x7f75fc073bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:45.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.228+0000 7f76038d1640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75fc15baf0 con 0x7f75fc15f6e0 2024-09-19T08:43:45.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.228+0000 7f76038d1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f75fc15c8b0 con 0x7f75fc15fac0 2024-09-19T08:43:45.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.228+0000 7f76038d1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f75fc0742d0 con 0x7f75fc06dbd0 2024-09-19T08:43:45.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.228+0000 7f7601e6e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f75fc15fac0 0x7f75fc073bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:45.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.228+0000 7f7600e6c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75fc06dbd0 0x7f75fc073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:45.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.228+0000 7f7601e6e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f75fc15fac0 0x7f75fc073bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:33190/0 (socket says 172.21.15.98:33190) 2024-09-19T08:43:45.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.228+0000 7f760166d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f75fc15f6e0 0x7f75fc06d4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:44150/0 (socket says 172.21.15.98:44150) 2024-09-19T08:43:45.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.228+0000 7f7601e6e640 1 -- 172.21.15.98:0/2725680287 learned_addr learned my addr 172.21.15.98:0/2725680287 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:45.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.228+0000 7f7600e6c640 1 -- 172.21.15.98:0/2725680287 >> v2:172.21.15.139:3300/0 conn(0x7f75fc15fac0 msgr2=0x7f75fc073bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:45.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.228+0000 7f7600e6c640 1 --2- 172.21.15.98:0/2725680287 >> v2:172.21.15.139:3300/0 conn(0x7f75fc15fac0 0x7f75fc073bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:45.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.228+0000 7f7600e6c640 1 -- 172.21.15.98:0/2725680287 >> v1:172.21.15.139:6789/0 conn(0x7f75fc15f6e0 legacy=0x7f75fc06d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:45.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.228+0000 7f7600e6c640 1 -- 172.21.15.98:0/2725680287 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75fc15bdd0 con 0x7f75fc06dbd0 2024-09-19T08:43:45.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.232+0000 7f7600e6c640 1 --2- 172.21.15.98:0/2725680287 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75fc06dbd0 0x7f75fc073680 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f75ec0097c0 tx=0x7f75ec01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b97ad3b1f7d812e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:45.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.232+0000 7f75f3fff640 1 -- 172.21.15.98:0/2725680287 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec0b0070 con 0x7f75fc06dbd0 2024-09-19T08:43:45.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.232+0000 7f75f3fff640 1 -- 172.21.15.98:0/2725680287 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75ec0ab600 con 0x7f75fc06dbd0 2024-09-19T08:43:45.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.232+0000 7f75f3fff640 1 -- 172.21.15.98:0/2725680287 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec0ab9d0 con 0x7f75fc06dbd0 2024-09-19T08:43:45.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.232+0000 7f76038d1640 1 -- 172.21.15.98:0/2725680287 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75fc06dbd0 msgr2=0x7f75fc073680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:45.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.232+0000 7f76038d1640 1 --2- 172.21.15.98:0/2725680287 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75fc06dbd0 0x7f75fc073680 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f75ec0097c0 tx=0x7f75ec01c4a0 comp rx=0 tx=0).stop 2024-09-19T08:43:45.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.232+0000 7f76038d1640 1 -- 172.21.15.98:0/2725680287 shutdown_connections 2024-09-19T08:43:45.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.232+0000 7f76038d1640 1 --2- 172.21.15.98:0/2725680287 >> v2:172.21.15.139:3300/0 conn(0x7f75fc15fac0 0x7f75fc073bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:45.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.232+0000 7f76038d1640 1 --2- 172.21.15.98:0/2725680287 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75fc06dbd0 0x7f75fc073680 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:45.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.232+0000 7f76038d1640 1 -- 172.21.15.98:0/2725680287 >> 172.21.15.98:0/2725680287 conn(0x7f75fc06c090 msgr2=0x7f75fc06c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:45.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.232+0000 7f76038d1640 1 -- 172.21.15.98:0/2725680287 shutdown_connections 2024-09-19T08:43:45.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.232+0000 7f76038d1640 1 -- 172.21.15.98:0/2725680287 wait complete. 2024-09-19T08:43:45.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.232+0000 7f76038d1640 1 Processor -- start 2024-09-19T08:43:45.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.232+0000 7f76038d1640 1 -- start start 2024-09-19T08:43:45.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.236+0000 7f76038d1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75fc06dbd0 0x7f75fc152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:45.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.236+0000 7f76038d1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75fc15f6e0 0x7f75fc0b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:45.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.236+0000 7f76038d1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75fc15fac0 0x7f75fc1485a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:45.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.236+0000 7f76038d1640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc0b4340 con 0x7f75fc15f6e0 2024-09-19T08:43:45.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.236+0000 7f76038d1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc0b41c0 con 0x7f75fc06dbd0 2024-09-19T08:43:45.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.236+0000 7f76038d1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f75fc0b44c0 con 0x7f75fc15fac0 2024-09-19T08:43:45.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.236+0000 7f7600e6c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75fc15f6e0 0x7f75fc0b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:45.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.236+0000 7f7600e6c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75fc15f6e0 0x7f75fc0b1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40094/0 (socket says 172.21.15.98:40094) 2024-09-19T08:43:45.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.236+0000 7f7600e6c640 1 -- 172.21.15.98:0/4183459196 learned_addr learned my addr 172.21.15.98:0/4183459196 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:45.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.236+0000 7f760166d640 1 --2- 172.21.15.98:0/4183459196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75fc06dbd0 0x7f75fc152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:45.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.236+0000 7f7601e6e640 1 --2- 172.21.15.98:0/4183459196 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75fc15fac0 0x7f75fc1485a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:45.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.236+0000 7f7600e6c640 1 -- 172.21.15.98:0/4183459196 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75fc15fac0 msgr2=0x7f75fc1485a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:45.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.236+0000 7f7600e6c640 1 --2- 172.21.15.98:0/4183459196 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75fc15fac0 0x7f75fc1485a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:45.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.236+0000 7f7600e6c640 1 -- 172.21.15.98:0/4183459196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75fc06dbd0 msgr2=0x7f75fc152a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:45.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.236+0000 7f7600e6c640 1 --2- 172.21.15.98:0/4183459196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75fc06dbd0 0x7f75fc152a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:45.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.236+0000 7f7600e6c640 1 -- 172.21.15.98:0/4183459196 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75fc152f40 con 0x7f75fc15f6e0 2024-09-19T08:43:45.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.236+0000 7f7601e6e640 1 --2- 172.21.15.98:0/4183459196 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75fc15fac0 0x7f75fc1485a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:45.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.236+0000 7f7600e6c640 1 --2- 172.21.15.98:0/4183459196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75fc15f6e0 0x7f75fc0b1d70 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f75ec01a0d0 tx=0x7f75ec019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:45.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.236+0000 7f75f1ffb640 1 -- 172.21.15.98:0/4183459196 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec0b0040 con 0x7f75fc15f6e0 2024-09-19T08:43:45.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.236+0000 7f75f1ffb640 1 -- 172.21.15.98:0/4183459196 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75ec0b43f0 con 0x7f75fc15f6e0 2024-09-19T08:43:45.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.236+0000 7f76038d1640 1 -- 172.21.15.98:0/4183459196 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75fc149530 con 0x7f75fc15f6e0 2024-09-19T08:43:45.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.236+0000 7f75f1ffb640 1 -- 172.21.15.98:0/4183459196 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec0b47c0 con 0x7f75fc15f6e0 2024-09-19T08:43:45.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.236+0000 7f76038d1640 1 -- 172.21.15.98:0/4183459196 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75fc149250 con 0x7f75fc15f6e0 2024-09-19T08:43:45.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.236+0000 7f76038d1640 1 -- 172.21.15.98:0/4183459196 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75fc06d4c0 con 0x7f75fc15f6e0 2024-09-19T08:43:45.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.236+0000 7f75f1ffb640 1 -- 172.21.15.98:0/4183459196 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f75ec0b4960 con 0x7f75fc15f6e0 2024-09-19T08:43:45.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.236+0000 7f75f1ffb640 1 --2- 172.21.15.98:0/4183459196 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f75e00679e0 0x7f75e0069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:45.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.236+0000 7f75f1ffb640 1 -- 172.21.15.98:0/4183459196 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5443+0+0 (secure 0 0 0) 0x7f75ec11f900 con 0x7f75fc15f6e0 2024-09-19T08:43:45.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.236+0000 7f760166d640 1 --2- 172.21.15.98:0/4183459196 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f75e00679e0 0x7f75e0069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:45.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.240+0000 7f760166d640 1 --2- 172.21.15.98:0/4183459196 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f75e00679e0 0x7f75e0069ea0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f75f8000f70 tx=0x7f75f8007c90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:45.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.240+0000 7f75f1ffb640 1 -- 172.21.15.98:0/4183459196 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f75ec016090 con 0x7f75fc15f6e0 2024-09-19T08:43:45.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:45.404+0000 7f76038d1640 1 -- 172.21.15.98:0/4183459196 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f75fc1502d0 con 0x7f75fc15f6e0 2024-09-19T08:43:46.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.072+0000 7f75f1ffb640 1 -- 172.21.15.98:0/4183459196 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v94) ==== 169+0+0 (secure 0 0 0) 0x7f75ec0f0310 con 0x7f75fc15f6e0 2024-09-19T08:43:46.073 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' removed 2024-09-19T08:43:46.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.076+0000 7f76038d1640 1 -- 172.21.15.98:0/4183459196 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f75e00679e0 msgr2=0x7f75e0069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:46.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.076+0000 7f76038d1640 1 --2- 172.21.15.98:0/4183459196 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f75e00679e0 0x7f75e0069ea0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f75f8000f70 tx=0x7f75f8007c90 comp rx=0 tx=0).stop 2024-09-19T08:43:46.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.076+0000 7f76038d1640 1 -- 172.21.15.98:0/4183459196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75fc15f6e0 msgr2=0x7f75fc0b1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:46.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.076+0000 7f76038d1640 1 --2- 172.21.15.98:0/4183459196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75fc15f6e0 0x7f75fc0b1d70 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f75ec01a0d0 tx=0x7f75ec019240 comp rx=0 tx=0).stop 2024-09-19T08:43:46.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.076+0000 7f76038d1640 1 -- 172.21.15.98:0/4183459196 shutdown_connections 2024-09-19T08:43:46.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.076+0000 7f76038d1640 1 --2- 172.21.15.98:0/4183459196 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f75e00679e0 0x7f75e0069ea0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:46.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.076+0000 7f76038d1640 1 --2- 172.21.15.98:0/4183459196 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75fc15fac0 0x7f75fc1485a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:46.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.076+0000 7f76038d1640 1 --2- 172.21.15.98:0/4183459196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75fc15f6e0 0x7f75fc0b1d70 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:46.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.076+0000 7f76038d1640 1 --2- 172.21.15.98:0/4183459196 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75fc06dbd0 0x7f75fc152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:46.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.076+0000 7f76038d1640 1 -- 172.21.15.98:0/4183459196 >> 172.21.15.98:0/4183459196 conn(0x7f75fc06c090 msgr2=0x7f75fc14f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:46.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.076+0000 7f76038d1640 1 -- 172.21.15.98:0/4183459196 shutdown_connections 2024-09-19T08:43:46.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.076+0000 7f76038d1640 1 -- 172.21.15.98:0/4183459196 wait complete. 2024-09-19T08:43:46.100 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-19T08:43:46.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.224+0000 7fb9561cc640 1 Processor -- start 2024-09-19T08:43:46.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.224+0000 7fb9561cc640 1 -- start start 2024-09-19T08:43:46.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.224+0000 7fb9561cc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9501415a0 0x7fb9500d20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:46.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.224+0000 7fb9561cc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb9500d25f0 0x7fb9500d3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:46.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.224+0000 7fb9561cc640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb95015eae0 con 0x7fb9500d38a0 2024-09-19T08:43:46.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.224+0000 7fb9561cc640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fb9500d2a30 con 0x7fb9500d25f0 2024-09-19T08:43:46.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.224+0000 7fb9561cc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb95015edc0 con 0x7fb9501415a0 2024-09-19T08:43:46.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.228+0000 7fb946ffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb9500d25f0 0x7fb9500d3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:46.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.228+0000 7fb946ffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb9500d25f0 0x7fb9500d3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40106/0 (socket says 172.21.15.98:40106) 2024-09-19T08:43:46.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.228+0000 7fb946ffd640 1 -- 172.21.15.98:0/3699436431 learned_addr learned my addr 172.21.15.98:0/3699436431 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:46.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.228+0000 7fb946ffd640 1 -- 172.21.15.98:0/3699436431 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9501415a0 msgr2=0x7fb9500d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:46.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.228+0000 7fb94f7fe640 1 --2- 172.21.15.98:0/3699436431 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9501415a0 0x7fb9500d20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:46.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.228+0000 7fb946ffd640 1 --2- 172.21.15.98:0/3699436431 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9501415a0 0x7fb9500d20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:46.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.228+0000 7fb946ffd640 1 -- 172.21.15.98:0/3699436431 >> v1:172.21.15.139:6789/0 conn(0x7fb9500d38a0 legacy=0x7fb950161fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:46.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.228+0000 7fb946ffd640 1 -- 172.21.15.98:0/3699436431 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb95015f0a0 con 0x7fb9500d25f0 2024-09-19T08:43:46.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.228+0000 7fb94f7fe640 1 --2- 172.21.15.98:0/3699436431 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9501415a0 0x7fb9500d20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:46.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.228+0000 7fb946ffd640 1 --2- 172.21.15.98:0/3699436431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9500d25f0 0x7fb9500d3360 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fb938009870 tx=0x7fb93801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d22c15b74fe78f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:46.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.228+0000 7fb94effd640 1 -- 172.21.15.98:0/3699436431 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9380b0070 con 0x7fb9500d25f0 2024-09-19T08:43:46.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.228+0000 7fb94effd640 1 -- 172.21.15.98:0/3699436431 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9380ab600 con 0x7fb9500d25f0 2024-09-19T08:43:46.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.228+0000 7fb94effd640 1 -- 172.21.15.98:0/3699436431 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9380ab9d0 con 0x7fb9500d25f0 2024-09-19T08:43:46.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.228+0000 7fb9561cc640 1 -- 172.21.15.98:0/3699436431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9500d25f0 msgr2=0x7fb9500d3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:46.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.228+0000 7fb9561cc640 1 --2- 172.21.15.98:0/3699436431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9500d25f0 0x7fb9500d3360 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fb938009870 tx=0x7fb93801c4f0 comp rx=0 tx=0).stop 2024-09-19T08:43:46.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.228+0000 7fb9561cc640 1 -- 172.21.15.98:0/3699436431 shutdown_connections 2024-09-19T08:43:46.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.228+0000 7fb9561cc640 1 --2- 172.21.15.98:0/3699436431 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9500d25f0 0x7fb9500d3360 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:46.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.228+0000 7fb9561cc640 1 --2- 172.21.15.98:0/3699436431 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9501415a0 0x7fb9500d20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:46.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.228+0000 7fb9561cc640 1 -- 172.21.15.98:0/3699436431 >> 172.21.15.98:0/3699436431 conn(0x7fb950074c70 msgr2=0x7fb950075080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:46.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.228+0000 7fb9561cc640 1 -- 172.21.15.98:0/3699436431 shutdown_connections 2024-09-19T08:43:46.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.228+0000 7fb9561cc640 1 -- 172.21.15.98:0/3699436431 wait complete. 2024-09-19T08:43:46.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.232+0000 7fb9561cc640 1 Processor -- start 2024-09-19T08:43:46.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.232+0000 7fb9561cc640 1 -- start start 2024-09-19T08:43:46.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.232+0000 7fb9561cc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9500d25f0 0x7fb95013aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:46.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.232+0000 7fb9561cc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9500d38a0 0x7fb9500cce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:46.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.232+0000 7fb9561cc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9501415a0 0x7fb95013fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:46.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.232+0000 7fb9561cc640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb9500d3ed0 con 0x7fb9500d25f0 2024-09-19T08:43:46.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.232+0000 7fb9561cc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb9500d3d50 con 0x7fb9501415a0 2024-09-19T08:43:46.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.232+0000 7fb9561cc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb9500d4120 con 0x7fb9500d38a0 2024-09-19T08:43:46.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.232+0000 7fb94f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9500d25f0 0x7fb95013aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:46.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.232+0000 7fb94f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9500d25f0 0x7fb95013aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40110/0 (socket says 172.21.15.98:40110) 2024-09-19T08:43:46.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.232+0000 7fb94f7fe640 1 -- 172.21.15.98:0/159215017 learned_addr learned my addr 172.21.15.98:0/159215017 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:46.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.232+0000 7fb946ffd640 1 --2- 172.21.15.98:0/159215017 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9500d38a0 0x7fb9500cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:46.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.232+0000 7fb94ffff640 1 --2- 172.21.15.98:0/159215017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9501415a0 0x7fb95013fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:46.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.232+0000 7fb94f7fe640 1 -- 172.21.15.98:0/159215017 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9500d38a0 msgr2=0x7fb9500cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:46.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.232+0000 7fb94f7fe640 1 --2- 172.21.15.98:0/159215017 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9500d38a0 0x7fb9500cce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:46.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.232+0000 7fb94f7fe640 1 -- 172.21.15.98:0/159215017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9501415a0 msgr2=0x7fb95013fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:46.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.232+0000 7fb94f7fe640 1 --2- 172.21.15.98:0/159215017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9501415a0 0x7fb95013fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:46.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.232+0000 7fb94f7fe640 1 -- 172.21.15.98:0/159215017 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9500cde10 con 0x7fb9500d25f0 2024-09-19T08:43:46.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.232+0000 7fb946ffd640 1 --2- 172.21.15.98:0/159215017 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9500d38a0 0x7fb9500cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:46.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.232+0000 7fb94f7fe640 1 --2- 172.21.15.98:0/159215017 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9500d25f0 0x7fb95013aef0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fb94000c7f0 tx=0x7fb94000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:46.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.232+0000 7fb94cff9640 1 -- 172.21.15.98:0/159215017 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb940015070 con 0x7fb9500d25f0 2024-09-19T08:43:46.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.232+0000 7fb94cff9640 1 -- 172.21.15.98:0/159215017 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb940007820 con 0x7fb9500d25f0 2024-09-19T08:43:46.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.232+0000 7fb9561cc640 1 -- 172.21.15.98:0/159215017 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9500ce0f0 con 0x7fb9500d25f0 2024-09-19T08:43:46.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.232+0000 7fb94cff9640 1 -- 172.21.15.98:0/159215017 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb940007c10 con 0x7fb9500d25f0 2024-09-19T08:43:46.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.232+0000 7fb9561cc640 1 -- 172.21.15.98:0/159215017 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb95014fe40 con 0x7fb9500d25f0 2024-09-19T08:43:46.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.232+0000 7fb9561cc640 1 -- 172.21.15.98:0/159215017 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9500d3360 con 0x7fb9500d25f0 2024-09-19T08:43:46.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.236+0000 7fb94cff9640 1 -- 172.21.15.98:0/159215017 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fb940018070 con 0x7fb9500d25f0 2024-09-19T08:43:46.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.236+0000 7fb94cff9640 1 --2- 172.21.15.98:0/159215017 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb9300679e0 0x7fb930069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:46.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.236+0000 7fb94cff9640 1 -- 172.21.15.98:0/159215017 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7fb940085b40 con 0x7fb9500d25f0 2024-09-19T08:43:46.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.236+0000 7fb946ffd640 1 --2- 172.21.15.98:0/159215017 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb9300679e0 0x7fb930069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:46.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.236+0000 7fb946ffd640 1 --2- 172.21.15.98:0/159215017 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb9300679e0 0x7fb930069ea0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb9380ad000 tx=0x7fb938008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:46.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.240+0000 7fb94cff9640 1 -- 172.21.15.98:0/159215017 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb940089030 con 0x7fb9500d25f0 2024-09-19T08:43:46.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.404+0000 7fb9561cc640 1 -- 172.21.15.98:0/159215017 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fb9501605c0 con 0x7fb9500d25f0 2024-09-19T08:43:46.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.408+0000 7fb94cff9640 1 -- 172.21.15.98:0/159215017 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7fb940056790 con 0x7fb9500d25f0 2024-09-19T08:43:46.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.412+0000 7fb9561cc640 1 -- 172.21.15.98:0/159215017 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb9300679e0 msgr2=0x7fb930069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:46.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.412+0000 7fb9561cc640 1 --2- 172.21.15.98:0/159215017 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb9300679e0 0x7fb930069ea0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb9380ad000 tx=0x7fb938008040 comp rx=0 tx=0).stop 2024-09-19T08:43:46.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.412+0000 7fb9561cc640 1 -- 172.21.15.98:0/159215017 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9500d25f0 msgr2=0x7fb95013aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:46.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.412+0000 7fb9561cc640 1 --2- 172.21.15.98:0/159215017 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9500d25f0 0x7fb95013aef0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fb94000c7f0 tx=0x7fb94000ccb0 comp rx=0 tx=0).stop 2024-09-19T08:43:46.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.412+0000 7fb9561cc640 1 -- 172.21.15.98:0/159215017 shutdown_connections 2024-09-19T08:43:46.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.412+0000 7fb9561cc640 1 --2- 172.21.15.98:0/159215017 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb9300679e0 0x7fb930069ea0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:46.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.412+0000 7fb9561cc640 1 --2- 172.21.15.98:0/159215017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9501415a0 0x7fb95013fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:46.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.412+0000 7fb9561cc640 1 --2- 172.21.15.98:0/159215017 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9500d38a0 0x7fb9500cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:46.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.412+0000 7fb9561cc640 1 --2- 172.21.15.98:0/159215017 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9500d25f0 0x7fb95013aef0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:46.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.412+0000 7fb9561cc640 1 -- 172.21.15.98:0/159215017 >> 172.21.15.98:0/159215017 conn(0x7fb950074c70 msgr2=0x7fb9500b9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:46.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.412+0000 7fb9561cc640 1 -- 172.21.15.98:0/159215017 shutdown_connections 2024-09-19T08:43:46.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.412+0000 7fb9561cc640 1 -- 172.21.15.98:0/159215017 wait complete. 2024-09-19T08:43:46.429 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars' 2024-09-19T08:43:46.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.544+0000 7f18568d1640 1 Processor -- start 2024-09-19T08:43:46.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.544+0000 7f18568d1640 1 -- start start 2024-09-19T08:43:46.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.548+0000 7f18568d1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f18500b7f80 0x7f185015f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:46.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.548+0000 7f18568d1640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f18500d4820 0x7f18501621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:46.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.548+0000 7f18568d1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18500b9970 0x7f18500b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:46.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.548+0000 7f18568d1640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f1850163050 con 0x7f18500d4820 2024-09-19T08:43:46.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.548+0000 7f18568d1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f1850162a90 con 0x7f18500b7f80 2024-09-19T08:43:46.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.548+0000 7f18568d1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1850162d70 con 0x7f18500b9970 2024-09-19T08:43:46.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.548+0000 7f184f7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f18500d4820 0x7f18501621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:46.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.548+0000 7f184f7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f18500d4820 0x7f18501621b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40116/0 (socket says 172.21.15.98:40116) 2024-09-19T08:43:46.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.548+0000 7f184f7fe640 1 -- 172.21.15.98:0/191433280 learned_addr learned my addr 172.21.15.98:0/191433280 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:46.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.548+0000 7f1854e6e640 1 --2- 172.21.15.98:0/191433280 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18500b9970 0x7f18500b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:46.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.548+0000 7f184f7fe640 1 -- 172.21.15.98:0/191433280 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18500b9970 msgr2=0x7f18500b9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:46.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.548+0000 7f184ffff640 1 --2- 172.21.15.98:0/191433280 >> v2:172.21.15.139:3300/0 conn(0x7f18500b7f80 0x7f185015f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:46.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.548+0000 7f184f7fe640 1 --2- 172.21.15.98:0/191433280 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18500b9970 0x7f18500b9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:46.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.548+0000 7f184f7fe640 1 -- 172.21.15.98:0/191433280 >> v2:172.21.15.139:3300/0 conn(0x7f18500b7f80 msgr2=0x7f185015f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:46.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.548+0000 7f184f7fe640 1 --2- 172.21.15.98:0/191433280 >> v2:172.21.15.139:3300/0 conn(0x7f18500b7f80 0x7f185015f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:46.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.548+0000 7f184f7fe640 1 -- 172.21.15.98:0/191433280 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1850068440 con 0x7f18500d4820 2024-09-19T08:43:46.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.548+0000 7f1854e6e640 1 --2- 172.21.15.98:0/191433280 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18500b9970 0x7f18500b9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:46.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.548+0000 7f184f7fe640 1 --2- 172.21.15.98:0/191433280 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18500d4820 0x7f18501621b0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f18500d0dd0 tx=0x7f184001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f321f28caa5492 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:46.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.548+0000 7f184effd640 1 -- 172.21.15.98:0/191433280 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18400a5740 con 0x7f18500d4820 2024-09-19T08:43:46.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.548+0000 7f184effd640 1 -- 172.21.15.98:0/191433280 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18400a58e0 con 0x7f18500d4820 2024-09-19T08:43:46.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.548+0000 7f184effd640 1 -- 172.21.15.98:0/191433280 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18400a5cb0 con 0x7f18500d4820 2024-09-19T08:43:46.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.552+0000 7f18568d1640 1 -- 172.21.15.98:0/191433280 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18500d4820 msgr2=0x7f18501621b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:46.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.552+0000 7f18568d1640 1 --2- 172.21.15.98:0/191433280 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18500d4820 0x7f18501621b0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f18500d0dd0 tx=0x7f184001c9f0 comp rx=0 tx=0).stop 2024-09-19T08:43:46.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.552+0000 7f18568d1640 1 -- 172.21.15.98:0/191433280 shutdown_connections 2024-09-19T08:43:46.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.552+0000 7f18568d1640 1 --2- 172.21.15.98:0/191433280 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18500d4820 0x7f18501621b0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:46.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.552+0000 7f18568d1640 1 --2- 172.21.15.98:0/191433280 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18500b9970 0x7f18500b9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:46.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.552+0000 7f18568d1640 1 --2- 172.21.15.98:0/191433280 >> v2:172.21.15.139:3300/0 conn(0x7f18500b7f80 0x7f185015f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:46.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.552+0000 7f18568d1640 1 -- 172.21.15.98:0/191433280 >> 172.21.15.98:0/191433280 conn(0x7f1850074e40 msgr2=0x7f1850075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:46.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.552+0000 7f18568d1640 1 -- 172.21.15.98:0/191433280 shutdown_connections 2024-09-19T08:43:46.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.552+0000 7f18568d1640 1 -- 172.21.15.98:0/191433280 wait complete. 2024-09-19T08:43:46.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.552+0000 7f18568d1640 1 Processor -- start 2024-09-19T08:43:46.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.552+0000 7f18568d1640 1 -- start start 2024-09-19T08:43:46.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.552+0000 7f18568d1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18500b7f80 0x7f185013c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:46.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.552+0000 7f18568d1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18500b9970 0x7f1850141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:46.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.552+0000 7f18568d1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18500d4820 0x7f1850142130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:46.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.552+0000 7f18568d1640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1850162870 con 0x7f18500b9970 2024-09-19T08:43:46.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.552+0000 7f18568d1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f18501626f0 con 0x7f18500b7f80 2024-09-19T08:43:46.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.552+0000 7f18568d1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f18500b8360 con 0x7f18500d4820 2024-09-19T08:43:46.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.552+0000 7f184f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18500b9970 0x7f1850141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:46.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.552+0000 7f184f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18500b9970 0x7f1850141bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40124/0 (socket says 172.21.15.98:40124) 2024-09-19T08:43:46.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.552+0000 7f184f7fe640 1 -- 172.21.15.98:0/3082123083 learned_addr learned my addr 172.21.15.98:0/3082123083 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:46.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.552+0000 7f1854e6e640 1 --2- 172.21.15.98:0/3082123083 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18500d4820 0x7f1850142130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:46.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.552+0000 7f184ffff640 1 --2- 172.21.15.98:0/3082123083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18500b7f80 0x7f185013c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:46.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.552+0000 7f184f7fe640 1 -- 172.21.15.98:0/3082123083 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18500d4820 msgr2=0x7f1850142130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:46.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.552+0000 7f184f7fe640 1 --2- 172.21.15.98:0/3082123083 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18500d4820 0x7f1850142130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:46.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.552+0000 7f184f7fe640 1 -- 172.21.15.98:0/3082123083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18500b7f80 msgr2=0x7f185013c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:46.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.552+0000 7f184f7fe640 1 --2- 172.21.15.98:0/3082123083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18500b7f80 0x7f185013c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:46.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.552+0000 7f184f7fe640 1 -- 172.21.15.98:0/3082123083 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1850161310 con 0x7f18500b9970 2024-09-19T08:43:46.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.552+0000 7f1854e6e640 1 --2- 172.21.15.98:0/3082123083 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18500d4820 0x7f1850142130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:43:46.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.552+0000 7f184ffff640 1 --2- 172.21.15.98:0/3082123083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18500b7f80 0x7f185013c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:46.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.552+0000 7f184f7fe640 1 --2- 172.21.15.98:0/3082123083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18500b9970 0x7f1850141bf0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f1840009040 tx=0x7f1840019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:46.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.552+0000 7f184cff9640 1 -- 172.21.15.98:0/3082123083 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18400bc070 con 0x7f18500b9970 2024-09-19T08:43:46.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.552+0000 7f184cff9640 1 -- 172.21.15.98:0/3082123083 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18400b8640 con 0x7f18500b9970 2024-09-19T08:43:46.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.552+0000 7f184cff9640 1 -- 172.21.15.98:0/3082123083 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18400b8a30 con 0x7f18500b9970 2024-09-19T08:43:46.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.556+0000 7f18568d1640 1 -- 172.21.15.98:0/3082123083 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f185013bec0 con 0x7f18500b9970 2024-09-19T08:43:46.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.556+0000 7f18568d1640 1 -- 172.21.15.98:0/3082123083 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1850160b10 con 0x7f18500b9970 2024-09-19T08:43:46.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.556+0000 7f18568d1640 1 -- 172.21.15.98:0/3082123083 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18500b9de0 con 0x7f18500b9970 2024-09-19T08:43:46.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.556+0000 7f184cff9640 1 -- 172.21.15.98:0/3082123083 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f18400b8bd0 con 0x7f18500b9970 2024-09-19T08:43:46.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.556+0000 7f184cff9640 1 --2- 172.21.15.98:0/3082123083 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f183c0679e0 0x7f183c069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:46.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.556+0000 7f184cff9640 1 -- 172.21.15.98:0/3082123083 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7f1840120600 con 0x7f18500b9970 2024-09-19T08:43:46.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.556+0000 7f184ffff640 1 --2- 172.21.15.98:0/3082123083 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f183c0679e0 0x7f183c069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:46.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.556+0000 7f184ffff640 1 --2- 172.21.15.98:0/3082123083 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f183c0679e0 0x7f183c069ea0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f184400b800 tx=0x7f1844008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:46.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.560+0000 7f184cff9640 1 -- 172.21.15.98:0/3082123083 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f18400165e0 con 0x7f18500b9970 2024-09-19T08:43:46.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:46.788+0000 7f18568d1640 1 -- 172.21.15.98:0/3082123083 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]} v 0) -- 0x7f1850152920 con 0x7f18500b9970 2024-09-19T08:43:47.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.084+0000 7f184cff9640 1 -- 172.21.15.98:0/3082123083 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]}]=0 v283) ==== 146+0+0 (secure 0 0 0) 0x7f18400f1480 con 0x7f18500b9970 2024-09-19T08:43:47.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.088+0000 7f18568d1640 1 -- 172.21.15.98:0/3082123083 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f183c0679e0 msgr2=0x7f183c069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:47.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.088+0000 7f18568d1640 1 --2- 172.21.15.98:0/3082123083 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f183c0679e0 0x7f183c069ea0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f184400b800 tx=0x7f1844008040 comp rx=0 tx=0).stop 2024-09-19T08:43:47.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.088+0000 7f18568d1640 1 -- 172.21.15.98:0/3082123083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18500b9970 msgr2=0x7f1850141bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:47.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.088+0000 7f18568d1640 1 --2- 172.21.15.98:0/3082123083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18500b9970 0x7f1850141bf0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f1840009040 tx=0x7f1840019790 comp rx=0 tx=0).stop 2024-09-19T08:43:47.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.088+0000 7f18568d1640 1 -- 172.21.15.98:0/3082123083 shutdown_connections 2024-09-19T08:43:47.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.088+0000 7f18568d1640 1 --2- 172.21.15.98:0/3082123083 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f183c0679e0 0x7f183c069ea0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.088+0000 7f18568d1640 1 --2- 172.21.15.98:0/3082123083 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18500d4820 0x7f1850142130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.088+0000 7f18568d1640 1 --2- 172.21.15.98:0/3082123083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18500b9970 0x7f1850141bf0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.088+0000 7f18568d1640 1 --2- 172.21.15.98:0/3082123083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18500b7f80 0x7f185013c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.088+0000 7f18568d1640 1 -- 172.21.15.98:0/3082123083 >> 172.21.15.98:0/3082123083 conn(0x7f1850074e40 msgr2=0x7f18500752b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:47.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.088+0000 7f18568d1640 1 -- 172.21.15.98:0/3082123083 shutdown_connections 2024-09-19T08:43:47.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.088+0000 7f18568d1640 1 -- 172.21.15.98:0/3082123083 wait complete. 2024-09-19T08:43:47.111 INFO:tasks.cephfs_test_runner:Test the creating vols, subvols subvolgroups fails when their names uses ... ok 2024-09-19T08:43:47.112 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2024-09-19T08:43:47.112 INFO:tasks.cephfs_test_runner:test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) 2024-09-19T08:43:47.112 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:43:47.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.236+0000 7fd7192cb640 1 Processor -- start 2024-09-19T08:43:47.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.236+0000 7fd7192cb640 1 -- start start 2024-09-19T08:43:47.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.236+0000 7fd7192cb640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd7140b7f80 0x7fd71415f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:47.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.236+0000 7fd7192cb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7140b9970 0x7fd7140b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:47.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.236+0000 7fd7192cb640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd714068440 con 0x7fd7140d4820 2024-09-19T08:43:47.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.236+0000 7fd7192cb640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fd714163050 con 0x7fd7140b7f80 2024-09-19T08:43:47.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.236+0000 7fd7192cb640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd714162a90 con 0x7fd7140b9970 2024-09-19T08:43:47.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.236+0000 7fd7137fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7140b9970 0x7fd7140b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:47.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.236+0000 7fd712ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd7140b7f80 0x7fd71415f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:47.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.236+0000 7fd7137fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7140b9970 0x7fd7140b9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:38860/0 (socket says 172.21.15.98:38860) 2024-09-19T08:43:47.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.236+0000 7fd70bfff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fd7140d4820 0x7fd7141621b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:44166/0 (socket says 172.21.15.98:44166) 2024-09-19T08:43:47.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.236+0000 7fd7137fe640 1 -- 172.21.15.98:0/1721816194 learned_addr learned my addr 172.21.15.98:0/1721816194 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:47.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.236+0000 7fd7137fe640 1 -- 172.21.15.98:0/1721816194 >> v2:172.21.15.139:3300/0 conn(0x7fd7140b7f80 msgr2=0x7fd71415f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:47.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.236+0000 7fd7137fe640 1 --2- 172.21.15.98:0/1721816194 >> v2:172.21.15.139:3300/0 conn(0x7fd7140b7f80 0x7fd71415f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.236+0000 7fd7127fc640 1 -- 172.21.15.98:0/1721816194 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 867402912 0 0) 0x7fd714068440 con 0x7fd7140d4820 2024-09-19T08:43:47.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.236+0000 7fd7137fe640 1 -- 172.21.15.98:0/1721816194 >> v1:172.21.15.139:6789/0 conn(0x7fd7140d4820 legacy=0x7fd7141621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:47.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.236+0000 7fd7137fe640 1 -- 172.21.15.98:0/1721816194 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd714162d70 con 0x7fd7140b9970 2024-09-19T08:43:47.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.240+0000 7fd7137fe640 1 --2- 172.21.15.98:0/1721816194 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7140b9970 0x7fd7140b9de0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd70c009e50 tx=0x7fd70c01ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2fed74285d3f51b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:47.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.240+0000 7fd7127fc640 1 -- 172.21.15.98:0/1721816194 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70c0a5700 con 0x7fd7140b9970 2024-09-19T08:43:47.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.240+0000 7fd7127fc640 1 -- 172.21.15.98:0/1721816194 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd70c0a58a0 con 0x7fd7140b9970 2024-09-19T08:43:47.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.240+0000 7fd7127fc640 1 -- 172.21.15.98:0/1721816194 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70c0a5c70 con 0x7fd7140b9970 2024-09-19T08:43:47.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.240+0000 7fd7192cb640 1 -- 172.21.15.98:0/1721816194 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7140b9970 msgr2=0x7fd7140b9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:47.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.240+0000 7fd7192cb640 1 --2- 172.21.15.98:0/1721816194 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7140b9970 0x7fd7140b9de0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd70c009e50 tx=0x7fd70c01ca30 comp rx=0 tx=0).stop 2024-09-19T08:43:47.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.240+0000 7fd7192cb640 1 -- 172.21.15.98:0/1721816194 shutdown_connections 2024-09-19T08:43:47.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.240+0000 7fd7192cb640 1 --2- 172.21.15.98:0/1721816194 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7140b9970 0x7fd7140b9de0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.240+0000 7fd7192cb640 1 --2- 172.21.15.98:0/1721816194 >> v2:172.21.15.139:3300/0 conn(0x7fd7140b7f80 0x7fd71415f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.240+0000 7fd7192cb640 1 -- 172.21.15.98:0/1721816194 >> 172.21.15.98:0/1721816194 conn(0x7fd714074e40 msgr2=0x7fd714075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:47.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.240+0000 7fd7192cb640 1 -- 172.21.15.98:0/1721816194 shutdown_connections 2024-09-19T08:43:47.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.240+0000 7fd7192cb640 1 -- 172.21.15.98:0/1721816194 wait complete. 2024-09-19T08:43:47.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.240+0000 7fd7192cb640 1 Processor -- start 2024-09-19T08:43:47.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.240+0000 7fd7192cb640 1 -- start start 2024-09-19T08:43:47.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.244+0000 7fd7192cb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7140b7f80 0x7fd714151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:47.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.244+0000 7fd7192cb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7140b9970 0x7fd714142b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:47.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.244+0000 7fd7192cb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7140d4820 0x7fd71413d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:47.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.244+0000 7fd7192cb640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd7140b84e0 con 0x7fd7140d4820 2024-09-19T08:43:47.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.244+0000 7fd7192cb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd7140b8360 con 0x7fd7140b7f80 2024-09-19T08:43:47.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.244+0000 7fd7192cb640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd714144590 con 0x7fd7140b9970 2024-09-19T08:43:47.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.244+0000 7fd7137fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7140d4820 0x7fd71413d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:47.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.244+0000 7fd7137fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7140d4820 0x7fd71413d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40140/0 (socket says 172.21.15.98:40140) 2024-09-19T08:43:47.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.244+0000 7fd7137fe640 1 -- 172.21.15.98:0/3257271403 learned_addr learned my addr 172.21.15.98:0/3257271403 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:47.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.244+0000 7fd712ffd640 1 --2- 172.21.15.98:0/3257271403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7140b7f80 0x7fd714151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:47.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.244+0000 7fd70bfff640 1 --2- 172.21.15.98:0/3257271403 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7140b9970 0x7fd714142b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:47.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.244+0000 7fd7137fe640 1 -- 172.21.15.98:0/3257271403 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7140b9970 msgr2=0x7fd714142b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:47.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.244+0000 7fd7137fe640 1 --2- 172.21.15.98:0/3257271403 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7140b9970 0x7fd714142b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.244+0000 7fd7137fe640 1 -- 172.21.15.98:0/3257271403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7140b7f80 msgr2=0x7fd714151f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:47.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.244+0000 7fd7137fe640 1 --2- 172.21.15.98:0/3257271403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7140b7f80 0x7fd714151f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.244+0000 7fd7137fe640 1 -- 172.21.15.98:0/3257271403 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd714150ef0 con 0x7fd7140d4820 2024-09-19T08:43:47.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.244+0000 7fd7137fe640 1 --2- 172.21.15.98:0/3257271403 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7140d4820 0x7fd71413d2c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fd70c0af040 tx=0x7fd70c0197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:47.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.244+0000 7fd70b7fe640 1 -- 172.21.15.98:0/3257271403 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70c0bd020 con 0x7fd7140d4820 2024-09-19T08:43:47.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.244+0000 7fd70b7fe640 1 -- 172.21.15.98:0/3257271403 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd70c0b8670 con 0x7fd7140d4820 2024-09-19T08:43:47.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.244+0000 7fd7192cb640 1 -- 172.21.15.98:0/3257271403 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7141511d0 con 0x7fd7140d4820 2024-09-19T08:43:47.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.244+0000 7fd70b7fe640 1 -- 172.21.15.98:0/3257271403 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70c009510 con 0x7fd7140d4820 2024-09-19T08:43:47.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.244+0000 7fd7192cb640 1 -- 172.21.15.98:0/3257271403 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd714151790 con 0x7fd7140d4820 2024-09-19T08:43:47.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.244+0000 7fd7192cb640 1 -- 172.21.15.98:0/3257271403 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd71415f560 con 0x7fd7140d4820 2024-09-19T08:43:47.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.244+0000 7fd70b7fe640 1 -- 172.21.15.98:0/3257271403 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fd70c0c0070 con 0x7fd7140d4820 2024-09-19T08:43:47.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.244+0000 7fd70b7fe640 1 --2- 172.21.15.98:0/3257271403 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd6f40679e0 0x7fd6f4069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:47.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.244+0000 7fd70b7fe640 1 -- 172.21.15.98:0/3257271403 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7fd70c11ffc0 con 0x7fd7140d4820 2024-09-19T08:43:47.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.244+0000 7fd712ffd640 1 --2- 172.21.15.98:0/3257271403 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd6f40679e0 0x7fd6f4069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:47.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.248+0000 7fd712ffd640 1 --2- 172.21.15.98:0/3257271403 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd6f40679e0 0x7fd6f4069ea0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd714155950 tx=0x7fd704008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:47.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.248+0000 7fd70b7fe640 1 -- 172.21.15.98:0/3257271403 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd70c016620 con 0x7fd7140d4820 2024-09-19T08:43:47.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.420+0000 7fd7192cb640 1 -- 172.21.15.98:0/3257271403 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd714143060 con 0x7fd7140d4820 2024-09-19T08:43:47.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.424+0000 7fd70b7fe640 1 -- 172.21.15.98:0/3257271403 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+13161 (secure 0 0 0) 0x7fd70c0f0c90 con 0x7fd7140d4820 2024-09-19T08:43:47.429 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:47.429 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":94,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:43:46.072554+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.98:6835/837817273":"2024-09-20T08:43:42.744959+0000","172.21.15.98:6834/837817273":"2024-09-20T08:43:42.744959+0000","172.21.15.98:6833/1092838020":"2024-09-20T08:43:36.020574+0000","172.21.15.98:6832/1092838020":"2024-09-20T08:43:36.020574+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:43:47.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.428+0000 7fd7192cb640 1 -- 172.21.15.98:0/3257271403 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd6f40679e0 msgr2=0x7fd6f4069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:47.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.428+0000 7fd7192cb640 1 --2- 172.21.15.98:0/3257271403 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd6f40679e0 0x7fd6f4069ea0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd714155950 tx=0x7fd704008040 comp rx=0 tx=0).stop 2024-09-19T08:43:47.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.428+0000 7fd7192cb640 1 -- 172.21.15.98:0/3257271403 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7140d4820 msgr2=0x7fd71413d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:47.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.428+0000 7fd7192cb640 1 --2- 172.21.15.98:0/3257271403 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7140d4820 0x7fd71413d2c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fd70c0af040 tx=0x7fd70c0197d0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.428+0000 7fd7192cb640 1 -- 172.21.15.98:0/3257271403 shutdown_connections 2024-09-19T08:43:47.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.428+0000 7fd7192cb640 1 --2- 172.21.15.98:0/3257271403 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd6f40679e0 0x7fd6f4069ea0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.428+0000 7fd7192cb640 1 --2- 172.21.15.98:0/3257271403 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7140d4820 0x7fd71413d2c0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.428+0000 7fd7192cb640 1 --2- 172.21.15.98:0/3257271403 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7140b9970 0x7fd714142b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.428+0000 7fd7192cb640 1 --2- 172.21.15.98:0/3257271403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7140b7f80 0x7fd714151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.428+0000 7fd7192cb640 1 -- 172.21.15.98:0/3257271403 >> 172.21.15.98:0/3257271403 conn(0x7fd714074e40 msgr2=0x7fd714075220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:47.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.428+0000 7fd7192cb640 1 -- 172.21.15.98:0/3257271403 shutdown_connections 2024-09-19T08:43:47.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.428+0000 7fd7192cb640 1 -- 172.21.15.98:0/3257271403 wait complete. 2024-09-19T08:43:47.445 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T08:38:08.105684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T08:43:47.446 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T08:43:47.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.564+0000 7f2e67d99640 1 Processor -- start 2024-09-19T08:43:47.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.564+0000 7f2e67d99640 1 -- start start 2024-09-19T08:43:47.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.568+0000 7f2e67d99640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e6006c3a0 0x7f2e600713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:47.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.568+0000 7f2e67d99640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e60073530 con 0x7f2e60145b70 2024-09-19T08:43:47.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.568+0000 7f2e67d99640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e60073810 con 0x7f2e600718f0 2024-09-19T08:43:47.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.568+0000 7f2e67d99640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2e60073af0 con 0x7f2e6006c3a0 2024-09-19T08:43:47.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.568+0000 7f2e65b35640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f2e60145b70 0x7f2e6006bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:49324/0 (socket says 172.21.15.98:49324) 2024-09-19T08:43:47.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.568+0000 7f2e65b35640 1 -- 172.21.15.98:0/2262275276 learned_addr learned my addr 172.21.15.98:0/2262275276 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:47.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.568+0000 7f2e65334640 1 --2- 172.21.15.98:0/2262275276 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e6006c3a0 0x7f2e600713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:47.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.568+0000 7f2e64b33640 1 -- 172.21.15.98:0/2262275276 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 403317609 0 0) 0x7f2e60073530 con 0x7f2e60145b70 2024-09-19T08:43:47.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.568+0000 7f2e64b33640 1 -- 172.21.15.98:0/2262275276 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e4c003530 con 0x7f2e60145b70 2024-09-19T08:43:47.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.568+0000 7f2e64b33640 1 -- 172.21.15.98:0/2262275276 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2691650007 0 0) 0x7f2e4c003530 con 0x7f2e60145b70 2024-09-19T08:43:47.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.568+0000 7f2e64b33640 1 -- 172.21.15.98:0/2262275276 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e6006c3a0 msgr2=0x7f2e600713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:47.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.568+0000 7f2e64b33640 1 --2- 172.21.15.98:0/2262275276 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e6006c3a0 0x7f2e600713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.568+0000 7f2e64b33640 1 -- 172.21.15.98:0/2262275276 >> v1:172.21.15.139:6789/0 conn(0x7f2e600718f0 legacy=0x7f2e600b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:47.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.568+0000 7f2e64b33640 1 -- 172.21.15.98:0/2262275276 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e60073dd0 con 0x7f2e60145b70 2024-09-19T08:43:47.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.568+0000 7f2e64b33640 1 -- 172.21.15.98:0/2262275276 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f2e5c001680 con 0x7f2e60145b70 2024-09-19T08:43:47.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.568+0000 7f2e64b33640 1 -- 172.21.15.98:0/2262275276 >> v1:172.21.15.98:6789/0 conn(0x7f2e60145b70 legacy=0x7f2e6006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:47.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.568+0000 7f2e64b33640 1 --2- 172.21.15.98:0/2262275276 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e4c0039f0 0x7f2e4c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:47.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.568+0000 7f2e64b33640 1 --2- 172.21.15.98:0/2262275276 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e6006c3a0 0x7f2e4c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:47.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.568+0000 7f2e64b33640 1 --2- 172.21.15.98:0/2262275276 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e4c004d50 0x7f2e4c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:47.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.568+0000 7f2e64b33640 1 -- 172.21.15.98:0/2262275276 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2e60073af0 con 0x7f2e4c0039f0 2024-09-19T08:43:47.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.568+0000 7f2e64b33640 1 -- 172.21.15.98:0/2262275276 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c00bb30 con 0x7f2e4c004d50 2024-09-19T08:43:47.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.568+0000 7f2e64b33640 1 -- 172.21.15.98:0/2262275276 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2e4c00bd30 con 0x7f2e6006c3a0 2024-09-19T08:43:47.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.568+0000 7f2e65b35640 1 --2- 172.21.15.98:0/2262275276 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e4c0039f0 0x7f2e4c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:47.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.568+0000 7f2e65334640 1 --2- 172.21.15.98:0/2262275276 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e6006c3a0 0x7f2e4c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:47.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.568+0000 7f2e65b35640 1 -- 172.21.15.98:0/2262275276 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e6006c3a0 msgr2=0x7f2e4c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:47.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.568+0000 7f2e65b35640 1 --2- 172.21.15.98:0/2262275276 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e6006c3a0 0x7f2e4c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.568+0000 7f2e65b35640 1 -- 172.21.15.98:0/2262275276 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e4c004d50 msgr2=0x7f2e4c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:47.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.568+0000 7f2e65b35640 1 --2- 172.21.15.98:0/2262275276 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e4c004d50 0x7f2e4c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.568+0000 7f2e65b35640 1 -- 172.21.15.98:0/2262275276 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4c003530 con 0x7f2e4c0039f0 2024-09-19T08:43:47.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.568+0000 7f2e65334640 1 --2- 172.21.15.98:0/2262275276 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e6006c3a0 0x7f2e4c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:47.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.572+0000 7f2e65b35640 1 --2- 172.21.15.98:0/2262275276 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e4c0039f0 0x7f2e4c003e10 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c00c210 tx=0x7f2e5c01cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15d8706d12189e91 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:47.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.572+0000 7f2e64b33640 1 -- 172.21.15.98:0/2262275276 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c007d20 con 0x7f2e4c0039f0 2024-09-19T08:43:47.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.572+0000 7f2e64b33640 1 -- 172.21.15.98:0/2262275276 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e5c0a9020 con 0x7f2e4c0039f0 2024-09-19T08:43:47.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.572+0000 7f2e64b33640 1 -- 172.21.15.98:0/2262275276 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c005b00 con 0x7f2e4c0039f0 2024-09-19T08:43:47.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.572+0000 7f2e67d99640 1 -- 172.21.15.98:0/2262275276 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e4c0039f0 msgr2=0x7f2e4c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:47.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.572+0000 7f2e67d99640 1 --2- 172.21.15.98:0/2262275276 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e4c0039f0 0x7f2e4c003e10 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c00c210 tx=0x7f2e5c01cba0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.572+0000 7f2e66336640 1 -- 172.21.15.98:0/2262275276 reap_dead start 2024-09-19T08:43:47.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.572+0000 7f2e67d99640 1 -- 172.21.15.98:0/2262275276 shutdown_connections 2024-09-19T08:43:47.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.572+0000 7f2e67d99640 1 -- 172.21.15.98:0/2262275276 >> 172.21.15.98:0/2262275276 conn(0x7f2e600b2010 msgr2=0x7f2e600b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:47.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.572+0000 7f2e67d99640 1 -- 172.21.15.98:0/2262275276 shutdown_connections 2024-09-19T08:43:47.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.572+0000 7f2e67d99640 1 -- 172.21.15.98:0/2262275276 wait complete. 2024-09-19T08:43:47.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.572+0000 7f2e67d99640 1 Processor -- start 2024-09-19T08:43:47.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.572+0000 7f2e67d99640 1 -- start start 2024-09-19T08:43:47.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.572+0000 7f2e67d99640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e4c0039f0 0x7f2e601fb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:47.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.572+0000 7f2e67d99640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e600ce580 0x7f2e6015a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:47.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.572+0000 7f2e67d99640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e6013b6d0 0x7f2e6013bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:47.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.572+0000 7f2e67d99640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2e600b46c0 con 0x7f2e4c0039f0 2024-09-19T08:43:47.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.572+0000 7f2e67d99640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2e600b4540 con 0x7f2e6013b6d0 2024-09-19T08:43:47.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.572+0000 7f2e67d99640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2e60071ee0 con 0x7f2e600ce580 2024-09-19T08:43:47.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.572+0000 7f2e65b35640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e4c0039f0 0x7f2e601fb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:47.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.572+0000 7f2e65b35640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e4c0039f0 0x7f2e601fb770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:32782/0 (socket says 172.21.15.98:32782) 2024-09-19T08:43:47.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.572+0000 7f2e65b35640 1 -- 172.21.15.98:0/2136398722 learned_addr learned my addr 172.21.15.98:0/2136398722 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:47.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.572+0000 7f2e65334640 1 --2- 172.21.15.98:0/2136398722 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e600ce580 0x7f2e6015a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:47.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.572+0000 7f2e66336640 1 --2- 172.21.15.98:0/2136398722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e6013b6d0 0x7f2e6013bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:47.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.572+0000 7f2e65b35640 1 -- 172.21.15.98:0/2136398722 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e600ce580 msgr2=0x7f2e6015a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:47.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.572+0000 7f2e65b35640 1 --2- 172.21.15.98:0/2136398722 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e600ce580 0x7f2e6015a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.572+0000 7f2e65b35640 1 -- 172.21.15.98:0/2136398722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e6013b6d0 msgr2=0x7f2e6013bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:47.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.572+0000 7f2e65b35640 1 --2- 172.21.15.98:0/2136398722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e6013b6d0 0x7f2e6013bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.572+0000 7f2e65b35640 1 -- 172.21.15.98:0/2136398722 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e600ced40 con 0x7f2e4c0039f0 2024-09-19T08:43:47.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.572+0000 7f2e66336640 1 --2- 172.21.15.98:0/2136398722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e6013b6d0 0x7f2e6013bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:47.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.572+0000 7f2e65b35640 1 --2- 172.21.15.98:0/2136398722 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e4c0039f0 0x7f2e601fb770 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c0ab040 tx=0x7f2e4c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:47.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.576+0000 7f2e567fc640 1 -- 172.21.15.98:0/2136398722 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c004b30 con 0x7f2e4c0039f0 2024-09-19T08:43:47.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.576+0000 7f2e567fc640 1 -- 172.21.15.98:0/2136398722 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e5c004cd0 con 0x7f2e4c0039f0 2024-09-19T08:43:47.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.576+0000 7f2e67d99640 1 -- 172.21.15.98:0/2136398722 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e600b0d50 con 0x7f2e4c0039f0 2024-09-19T08:43:47.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.576+0000 7f2e567fc640 1 -- 172.21.15.98:0/2136398722 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c0072d0 con 0x7f2e4c0039f0 2024-09-19T08:43:47.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.576+0000 7f2e67d99640 1 -- 172.21.15.98:0/2136398722 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e600b1310 con 0x7f2e4c0039f0 2024-09-19T08:43:47.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.576+0000 7f2e67d99640 1 -- 172.21.15.98:0/2136398722 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e600b3410 con 0x7f2e4c0039f0 2024-09-19T08:43:47.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.576+0000 7f2e567fc640 1 -- 172.21.15.98:0/2136398722 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f2e5c007470 con 0x7f2e4c0039f0 2024-09-19T08:43:47.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.576+0000 7f2e567fc640 1 --2- 172.21.15.98:0/2136398722 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2e440679e0 0x7f2e44069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:47.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.576+0000 7f2e567fc640 1 -- 172.21.15.98:0/2136398722 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7f2e5c11f3c0 con 0x7f2e4c0039f0 2024-09-19T08:43:47.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.576+0000 7f2e65334640 1 --2- 172.21.15.98:0/2136398722 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2e440679e0 0x7f2e44069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:47.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.576+0000 7f2e65334640 1 --2- 172.21.15.98:0/2136398722 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2e440679e0 0x7f2e44069ea0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2e601451a0 tx=0x7f2e50008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:47.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.580+0000 7f2e567fc640 1 -- 172.21.15.98:0/2136398722 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2e5c0be060 con 0x7f2e4c0039f0 2024-09-19T08:43:47.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.744+0000 7f2e67d99640 1 -- 172.21.15.98:0/2136398722 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2e60144ef0 con 0x7f2e4c0039f0 2024-09-19T08:43:47.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.744+0000 7f2e567fc640 1 -- 172.21.15.98:0/2136398722 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f2e5c0168b0 con 0x7f2e4c0039f0 2024-09-19T08:43:47.748 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2024-09-19T08:43:47.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.748+0000 7f2e67d99640 1 -- 172.21.15.98:0/2136398722 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2e440679e0 msgr2=0x7f2e44069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:47.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.748+0000 7f2e67d99640 1 --2- 172.21.15.98:0/2136398722 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2e440679e0 0x7f2e44069ea0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2e601451a0 tx=0x7f2e50008040 comp rx=0 tx=0).stop 2024-09-19T08:43:47.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.748+0000 7f2e67d99640 1 -- 172.21.15.98:0/2136398722 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e4c0039f0 msgr2=0x7f2e601fb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:47.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.748+0000 7f2e67d99640 1 --2- 172.21.15.98:0/2136398722 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e4c0039f0 0x7f2e601fb770 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c0ab040 tx=0x7f2e4c005840 comp rx=0 tx=0).stop 2024-09-19T08:43:47.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.748+0000 7f2e67d99640 1 -- 172.21.15.98:0/2136398722 shutdown_connections 2024-09-19T08:43:47.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.748+0000 7f2e67d99640 1 --2- 172.21.15.98:0/2136398722 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2e440679e0 0x7f2e44069ea0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.748+0000 7f2e67d99640 1 --2- 172.21.15.98:0/2136398722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e6013b6d0 0x7f2e6013bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.748+0000 7f2e67d99640 1 --2- 172.21.15.98:0/2136398722 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2e600ce580 0x7f2e6015a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.748+0000 7f2e67d99640 1 --2- 172.21.15.98:0/2136398722 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2e4c0039f0 0x7f2e601fb770 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.748+0000 7f2e67d99640 1 -- 172.21.15.98:0/2136398722 >> 172.21.15.98:0/2136398722 conn(0x7f2e600b2010 msgr2=0x7f2e6006bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:47.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.748+0000 7f2e67d99640 1 -- 172.21.15.98:0/2136398722 shutdown_connections 2024-09-19T08:43:47.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.748+0000 7f2e67d99640 1 -- 172.21.15.98:0/2136398722 wait complete. 2024-09-19T08:43:47.767 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity' 2024-09-19T08:43:47.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.884+0000 7f2a41bfe640 1 Processor -- start 2024-09-19T08:43:47.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.884+0000 7f2a41bfe640 1 -- start start 2024-09-19T08:43:47.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.884+0000 7f2a41bfe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2a3c0b7f80 0x7f2a3c15f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:47.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.884+0000 7f2a41bfe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2a3c0d4820 0x7f2a3c1621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:47.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.884+0000 7f2a41bfe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a3c0b9970 0x7f2a3c0b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:47.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.884+0000 7f2a41bfe640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f2a3c163050 con 0x7f2a3c0b7f80 2024-09-19T08:43:47.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.884+0000 7f2a41bfe640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f2a3c162a90 con 0x7f2a3c0d4820 2024-09-19T08:43:47.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.884+0000 7f2a41bfe640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2a3c162d70 con 0x7f2a3c0b9970 2024-09-19T08:43:47.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.884+0000 7f2a3b7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2a3c0b7f80 0x7f2a3c15f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:47.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.884+0000 7f2a3b7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2a3c0b7f80 0x7f2a3c15f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:32796/0 (socket says 172.21.15.98:32796) 2024-09-19T08:43:47.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.884+0000 7f2a3b7fe640 1 -- 172.21.15.98:0/3893070404 learned_addr learned my addr 172.21.15.98:0/3893070404 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:47.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.884+0000 7f2a3bfff640 1 --2- 172.21.15.98:0/3893070404 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a3c0b9970 0x7f2a3c0b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:47.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.884+0000 7f2a3affd640 1 --2- 172.21.15.98:0/3893070404 >> v2:172.21.15.139:3300/0 conn(0x7f2a3c0d4820 0x7f2a3c1621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:47.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.884+0000 7f2a3b7fe640 1 -- 172.21.15.98:0/3893070404 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a3c0b9970 msgr2=0x7f2a3c0b9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:47.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.884+0000 7f2a3b7fe640 1 --2- 172.21.15.98:0/3893070404 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a3c0b9970 0x7f2a3c0b9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.884+0000 7f2a3b7fe640 1 -- 172.21.15.98:0/3893070404 >> v2:172.21.15.139:3300/0 conn(0x7f2a3c0d4820 msgr2=0x7f2a3c1621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:47.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.884+0000 7f2a3b7fe640 1 --2- 172.21.15.98:0/3893070404 >> v2:172.21.15.139:3300/0 conn(0x7f2a3c0d4820 0x7f2a3c1621b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.884+0000 7f2a3b7fe640 1 -- 172.21.15.98:0/3893070404 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a3c068440 con 0x7f2a3c0b7f80 2024-09-19T08:43:47.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.888+0000 7f2a3b7fe640 1 --2- 172.21.15.98:0/3893070404 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a3c0b7f80 0x7f2a3c15f560 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c0ceb00 tx=0x7f2a2c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dcc3300db220a92 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:47.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.888+0000 7f2a3a7fc640 1 -- 172.21.15.98:0/3893070404 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c0b1070 con 0x7f2a3c0b7f80 2024-09-19T08:43:47.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.888+0000 7f2a3a7fc640 1 -- 172.21.15.98:0/3893070404 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a2c0ac410 con 0x7f2a3c0b7f80 2024-09-19T08:43:47.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.888+0000 7f2a3a7fc640 1 -- 172.21.15.98:0/3893070404 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c0ac7e0 con 0x7f2a3c0b7f80 2024-09-19T08:43:47.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.888+0000 7f2a41bfe640 1 -- 172.21.15.98:0/3893070404 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a3c0b7f80 msgr2=0x7f2a3c15f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:47.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.888+0000 7f2a41bfe640 1 --2- 172.21.15.98:0/3893070404 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a3c0b7f80 0x7f2a3c15f560 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c0ceb00 tx=0x7f2a2c01c520 comp rx=0 tx=0).stop 2024-09-19T08:43:47.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.888+0000 7f2a41bfe640 1 -- 172.21.15.98:0/3893070404 shutdown_connections 2024-09-19T08:43:47.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.888+0000 7f2a41bfe640 1 --2- 172.21.15.98:0/3893070404 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a3c0b9970 0x7f2a3c0b9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.888+0000 7f2a41bfe640 1 --2- 172.21.15.98:0/3893070404 >> v2:172.21.15.139:3300/0 conn(0x7f2a3c0d4820 0x7f2a3c1621b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.888+0000 7f2a41bfe640 1 --2- 172.21.15.98:0/3893070404 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a3c0b7f80 0x7f2a3c15f560 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.888+0000 7f2a41bfe640 1 -- 172.21.15.98:0/3893070404 >> 172.21.15.98:0/3893070404 conn(0x7f2a3c074e40 msgr2=0x7f2a3c075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:47.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.888+0000 7f2a41bfe640 1 -- 172.21.15.98:0/3893070404 shutdown_connections 2024-09-19T08:43:47.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.888+0000 7f2a41bfe640 1 -- 172.21.15.98:0/3893070404 wait complete. 2024-09-19T08:43:47.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.888+0000 7f2a41bfe640 1 Processor -- start 2024-09-19T08:43:47.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.888+0000 7f2a41bfe640 1 -- start start 2024-09-19T08:43:47.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.888+0000 7f2a41bfe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a3c0b7f80 0x7f2a3c152990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:47.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.888+0000 7f2a41bfe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a3c0b9970 0x7f2a3c13a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:47.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.888+0000 7f2a41bfe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a3c0d4820 0x7f2a3c1421a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:47.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.888+0000 7f2a41bfe640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3c162870 con 0x7f2a3c0b9970 2024-09-19T08:43:47.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.888+0000 7f2a41bfe640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3c1626f0 con 0x7f2a3c0b7f80 2024-09-19T08:43:47.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.888+0000 7f2a41bfe640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2a3c0b8360 con 0x7f2a3c0d4820 2024-09-19T08:43:47.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.888+0000 7f2a3affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a3c0b9970 0x7f2a3c13a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:47.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.888+0000 7f2a3affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a3c0b9970 0x7f2a3c13a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:32800/0 (socket says 172.21.15.98:32800) 2024-09-19T08:43:47.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.888+0000 7f2a3affd640 1 -- 172.21.15.98:0/1354438623 learned_addr learned my addr 172.21.15.98:0/1354438623 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:47.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.888+0000 7f2a3affd640 1 -- 172.21.15.98:0/1354438623 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a3c0d4820 msgr2=0x7f2a3c1421a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T08:43:47.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.888+0000 7f2a3affd640 1 --2- 172.21.15.98:0/1354438623 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a3c0d4820 0x7f2a3c1421a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.888+0000 7f2a3affd640 1 -- 172.21.15.98:0/1354438623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a3c0b7f80 msgr2=0x7f2a3c152990 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T08:43:47.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.888+0000 7f2a3affd640 1 --2- 172.21.15.98:0/1354438623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a3c0b7f80 0x7f2a3c152990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:47.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.888+0000 7f2a3affd640 1 -- 172.21.15.98:0/1354438623 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a3c155cf0 con 0x7f2a3c0b9970 2024-09-19T08:43:47.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.888+0000 7f2a3affd640 1 --2- 172.21.15.98:0/1354438623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a3c0b9970 0x7f2a3c13a580 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000d830 tx=0x7f2a3000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:47.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.888+0000 7f2a23fff640 1 -- 172.21.15.98:0/1354438623 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3000b6c0 con 0x7f2a3c0b9970 2024-09-19T08:43:47.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.888+0000 7f2a23fff640 1 -- 172.21.15.98:0/1354438623 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a30019070 con 0x7f2a3c0b9970 2024-09-19T08:43:47.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.888+0000 7f2a23fff640 1 -- 172.21.15.98:0/1354438623 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3000ba60 con 0x7f2a3c0b9970 2024-09-19T08:43:47.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.892+0000 7f2a41bfe640 1 -- 172.21.15.98:0/1354438623 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a3c1393c0 con 0x7f2a3c0b9970 2024-09-19T08:43:47.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.892+0000 7f2a41bfe640 1 -- 172.21.15.98:0/1354438623 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a3c1396a0 con 0x7f2a3c0b9970 2024-09-19T08:43:47.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.892+0000 7f2a41bfe640 1 -- 172.21.15.98:0/1354438623 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a3c0b9de0 con 0x7f2a3c0b9970 2024-09-19T08:43:47.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.892+0000 7f2a23fff640 1 -- 172.21.15.98:0/1354438623 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f2a3001e460 con 0x7f2a3c0b9970 2024-09-19T08:43:47.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.892+0000 7f2a23fff640 1 --2- 172.21.15.98:0/1354438623 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2a280679e0 0x7f2a28069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:47.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.892+0000 7f2a23fff640 1 -- 172.21.15.98:0/1354438623 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7f2a30086230 con 0x7f2a3c0b9970 2024-09-19T08:43:47.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.892+0000 7f2a3b7fe640 1 --2- 172.21.15.98:0/1354438623 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2a280679e0 0x7f2a28069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:47.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.892+0000 7f2a3b7fe640 1 --2- 172.21.15.98:0/1354438623 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2a280679e0 0x7f2a28069ea0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c0ae000 tx=0x7f2a2c0a6040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:47.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:47.896+0000 7f2a23fff640 1 -- 172.21.15.98:0/1354438623 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2a30056f00 con 0x7f2a3c0b9970 2024-09-19T08:43:48.032 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:43:48.035 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:43:48.124 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:48.124+0000 7f2a41bfe640 1 -- 172.21.15.98:0/1354438623 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]} v 0) -- 0x7f2a3c1577b0 con 0x7f2a3c0b9970 2024-09-19T08:43:49.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.092+0000 7f2a23fff640 1 -- 172.21.15.98:0/1354438623 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]}]=0 v285) ==== 152+0+0 (secure 0 0 0) 0x7f2a30084e20 con 0x7f2a3c0b9970 2024-09-19T08:43:49.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.100+0000 7f2a41bfe640 1 -- 172.21.15.98:0/1354438623 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2a280679e0 msgr2=0x7f2a28069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:49.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.100+0000 7f2a41bfe640 1 --2- 172.21.15.98:0/1354438623 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2a280679e0 0x7f2a28069ea0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c0ae000 tx=0x7f2a2c0a6040 comp rx=0 tx=0).stop 2024-09-19T08:43:49.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.100+0000 7f2a41bfe640 1 -- 172.21.15.98:0/1354438623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a3c0b9970 msgr2=0x7f2a3c13a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:49.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.100+0000 7f2a41bfe640 1 --2- 172.21.15.98:0/1354438623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a3c0b9970 0x7f2a3c13a580 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000d830 tx=0x7f2a3000dcf0 comp rx=0 tx=0).stop 2024-09-19T08:43:49.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.100+0000 7f2a41bfe640 1 -- 172.21.15.98:0/1354438623 shutdown_connections 2024-09-19T08:43:49.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.100+0000 7f2a41bfe640 1 --2- 172.21.15.98:0/1354438623 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2a280679e0 0x7f2a28069ea0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:49.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.100+0000 7f2a41bfe640 1 --2- 172.21.15.98:0/1354438623 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a3c0d4820 0x7f2a3c1421a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:49.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.100+0000 7f2a41bfe640 1 --2- 172.21.15.98:0/1354438623 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a3c0b9970 0x7f2a3c13a580 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:49.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.100+0000 7f2a41bfe640 1 --2- 172.21.15.98:0/1354438623 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a3c0b7f80 0x7f2a3c152990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:49.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.100+0000 7f2a41bfe640 1 -- 172.21.15.98:0/1354438623 >> 172.21.15.98:0/1354438623 conn(0x7f2a3c074e40 msgr2=0x7f2a3c13b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:49.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.100+0000 7f2a41bfe640 1 -- 172.21.15.98:0/1354438623 shutdown_connections 2024-09-19T08:43:49.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.100+0000 7f2a41bfe640 1 -- 172.21.15.98:0/1354438623 wait complete. 2024-09-19T08:43:49.125 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T08:43:49.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.248+0000 7fdf78337640 1 Processor -- start 2024-09-19T08:43:49.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.248+0000 7fdf78337640 1 -- start start 2024-09-19T08:43:49.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.248+0000 7fdf78337640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fdf701415a0 0x7fdf700d20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:49.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.248+0000 7fdf78337640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdf700d25f0 0x7fdf700d3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:49.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.248+0000 7fdf78337640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf700d38a0 0x7fdf70161fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:49.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.248+0000 7fdf78337640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fdf700d2a30 con 0x7fdf701415a0 2024-09-19T08:43:49.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.248+0000 7fdf78337640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fdf7015eae0 con 0x7fdf700d25f0 2024-09-19T08:43:49.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.248+0000 7fdf78337640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdf7015edc0 con 0x7fdf700d38a0 2024-09-19T08:43:49.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.248+0000 7fdf760d3640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fdf701415a0 0x7fdf700d20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:49.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.248+0000 7fdf760d3640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fdf701415a0 0x7fdf700d20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:32804/0 (socket says 172.21.15.98:32804) 2024-09-19T08:43:49.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.248+0000 7fdf760d3640 1 -- 172.21.15.98:0/1728571352 learned_addr learned my addr 172.21.15.98:0/1728571352 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:49.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.248+0000 7fdf758d2640 1 --2- 172.21.15.98:0/1728571352 >> v2:172.21.15.139:3300/0 conn(0x7fdf700d25f0 0x7fdf700d3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:49.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.248+0000 7fdf768d4640 1 --2- 172.21.15.98:0/1728571352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf700d38a0 0x7fdf70161fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:49.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.248+0000 7fdf760d3640 1 -- 172.21.15.98:0/1728571352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf700d38a0 msgr2=0x7fdf70161fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:49.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.248+0000 7fdf760d3640 1 --2- 172.21.15.98:0/1728571352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf700d38a0 0x7fdf70161fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:49.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.248+0000 7fdf760d3640 1 -- 172.21.15.98:0/1728571352 >> v2:172.21.15.139:3300/0 conn(0x7fdf700d25f0 msgr2=0x7fdf700d3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:49.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.248+0000 7fdf760d3640 1 --2- 172.21.15.98:0/1728571352 >> v2:172.21.15.139:3300/0 conn(0x7fdf700d25f0 0x7fdf700d3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:49.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.248+0000 7fdf760d3640 1 -- 172.21.15.98:0/1728571352 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf7015f0a0 con 0x7fdf701415a0 2024-09-19T08:43:49.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.248+0000 7fdf758d2640 1 --2- 172.21.15.98:0/1728571352 >> v2:172.21.15.139:3300/0 conn(0x7fdf700d25f0 0x7fdf700d3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:49.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.252+0000 7fdf760d3640 1 --2- 172.21.15.98:0/1728571352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf701415a0 0x7fdf700d20b0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fdf7015c5c0 tx=0x7fdf6001c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39cf72daddbe02e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:49.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.252+0000 7fdf750d1640 1 -- 172.21.15.98:0/1728571352 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf600b0070 con 0x7fdf701415a0 2024-09-19T08:43:49.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.252+0000 7fdf750d1640 1 -- 172.21.15.98:0/1728571352 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf600ab600 con 0x7fdf701415a0 2024-09-19T08:43:49.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.252+0000 7fdf750d1640 1 -- 172.21.15.98:0/1728571352 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf600ab9d0 con 0x7fdf701415a0 2024-09-19T08:43:49.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.252+0000 7fdf78337640 1 -- 172.21.15.98:0/1728571352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf701415a0 msgr2=0x7fdf700d20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:49.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.252+0000 7fdf78337640 1 --2- 172.21.15.98:0/1728571352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf701415a0 0x7fdf700d20b0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fdf7015c5c0 tx=0x7fdf6001c230 comp rx=0 tx=0).stop 2024-09-19T08:43:49.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.252+0000 7fdf78337640 1 -- 172.21.15.98:0/1728571352 shutdown_connections 2024-09-19T08:43:49.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.252+0000 7fdf78337640 1 --2- 172.21.15.98:0/1728571352 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf700d38a0 0x7fdf70161fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:49.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.252+0000 7fdf78337640 1 --2- 172.21.15.98:0/1728571352 >> v2:172.21.15.139:3300/0 conn(0x7fdf700d25f0 0x7fdf700d3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:49.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.252+0000 7fdf78337640 1 --2- 172.21.15.98:0/1728571352 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf701415a0 0x7fdf700d20b0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:49.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.252+0000 7fdf78337640 1 -- 172.21.15.98:0/1728571352 >> 172.21.15.98:0/1728571352 conn(0x7fdf70074c70 msgr2=0x7fdf70075080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:49.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.252+0000 7fdf78337640 1 -- 172.21.15.98:0/1728571352 shutdown_connections 2024-09-19T08:43:49.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.252+0000 7fdf78337640 1 -- 172.21.15.98:0/1728571352 wait complete. 2024-09-19T08:43:49.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.252+0000 7fdf78337640 1 Processor -- start 2024-09-19T08:43:49.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.252+0000 7fdf78337640 1 -- start start 2024-09-19T08:43:49.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.256+0000 7fdf78337640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf700d25f0 0x7fdf700cd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:49.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.256+0000 7fdf78337640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf700d38a0 0x7fdf700d1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:49.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.256+0000 7fdf78337640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf701415a0 0x7fdf701577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:49.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.256+0000 7fdf78337640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdf700d3e30 con 0x7fdf700d25f0 2024-09-19T08:43:49.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.256+0000 7fdf78337640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdf700d3cb0 con 0x7fdf701415a0 2024-09-19T08:43:49.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.256+0000 7fdf78337640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdf700b8790 con 0x7fdf700d38a0 2024-09-19T08:43:49.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.256+0000 7fdf760d3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf700d25f0 0x7fdf700cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:49.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.256+0000 7fdf760d3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf700d25f0 0x7fdf700cd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:32818/0 (socket says 172.21.15.98:32818) 2024-09-19T08:43:49.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.256+0000 7fdf760d3640 1 -- 172.21.15.98:0/2310320771 learned_addr learned my addr 172.21.15.98:0/2310320771 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:49.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.256+0000 7fdf768d4640 1 --2- 172.21.15.98:0/2310320771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf701415a0 0x7fdf701577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:49.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.256+0000 7fdf758d2640 1 --2- 172.21.15.98:0/2310320771 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf700d38a0 0x7fdf700d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:49.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.256+0000 7fdf760d3640 1 -- 172.21.15.98:0/2310320771 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf700d38a0 msgr2=0x7fdf700d1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:49.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.256+0000 7fdf760d3640 1 --2- 172.21.15.98:0/2310320771 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf700d38a0 0x7fdf700d1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:49.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.256+0000 7fdf760d3640 1 -- 172.21.15.98:0/2310320771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf701415a0 msgr2=0x7fdf701577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:49.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.256+0000 7fdf760d3640 1 --2- 172.21.15.98:0/2310320771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf701415a0 0x7fdf701577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:49.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.256+0000 7fdf760d3640 1 -- 172.21.15.98:0/2310320771 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf700ce330 con 0x7fdf700d25f0 2024-09-19T08:43:49.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.256+0000 7fdf758d2640 1 --2- 172.21.15.98:0/2310320771 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf700d38a0 0x7fdf700d1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:49.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.256+0000 7fdf768d4640 1 --2- 172.21.15.98:0/2310320771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf701415a0 0x7fdf701577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:49.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.256+0000 7fdf760d3640 1 --2- 172.21.15.98:0/2310320771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf700d25f0 0x7fdf700cd880 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fdf6001cad0 tx=0x7fdf6001aa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:49.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.256+0000 7fdf6effd640 1 -- 172.21.15.98:0/2310320771 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf600ba020 con 0x7fdf700d25f0 2024-09-19T08:43:49.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.256+0000 7fdf6effd640 1 -- 172.21.15.98:0/2310320771 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf600b6640 con 0x7fdf700d25f0 2024-09-19T08:43:49.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.256+0000 7fdf6effd640 1 -- 172.21.15.98:0/2310320771 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf600b0040 con 0x7fdf700d25f0 2024-09-19T08:43:49.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.256+0000 7fdf78337640 1 -- 172.21.15.98:0/2310320771 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf700ce050 con 0x7fdf700d25f0 2024-09-19T08:43:49.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.256+0000 7fdf78337640 1 -- 172.21.15.98:0/2310320771 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf700d1100 con 0x7fdf700d25f0 2024-09-19T08:43:49.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.256+0000 7fdf78337640 1 -- 172.21.15.98:0/2310320771 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf70161fc0 con 0x7fdf700d25f0 2024-09-19T08:43:49.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.256+0000 7fdf6effd640 1 -- 172.21.15.98:0/2310320771 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fdf600bd020 con 0x7fdf700d25f0 2024-09-19T08:43:49.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.256+0000 7fdf6effd640 1 --2- 172.21.15.98:0/2310320771 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fdf5c067a10 0x7fdf5c069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:49.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.256+0000 7fdf6effd640 1 -- 172.21.15.98:0/2310320771 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7fdf6011ff70 con 0x7fdf700d25f0 2024-09-19T08:43:49.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.260+0000 7fdf758d2640 1 --2- 172.21.15.98:0/2310320771 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fdf5c067a10 0x7fdf5c069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:49.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.260+0000 7fdf758d2640 1 --2- 172.21.15.98:0/2310320771 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fdf5c067a10 0x7fdf5c069ed0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdf64006fd0 tx=0x7fdf64008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:49.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.264+0000 7fdf6effd640 1 -- 172.21.15.98:0/2310320771 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fdf600c4030 con 0x7fdf700d25f0 2024-09-19T08:43:49.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.424+0000 7fdf78337640 1 -- 172.21.15.98:0/2310320771 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fdf700cb200 con 0x7fdf700d25f0 2024-09-19T08:43:49.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.428+0000 7fdf6effd640 1 -- 172.21.15.98:0/2310320771 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7fdf60016110 con 0x7fdf700d25f0 2024-09-19T08:43:49.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.432+0000 7fdf78337640 1 -- 172.21.15.98:0/2310320771 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fdf5c067a10 msgr2=0x7fdf5c069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:49.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.432+0000 7fdf78337640 1 --2- 172.21.15.98:0/2310320771 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fdf5c067a10 0x7fdf5c069ed0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdf64006fd0 tx=0x7fdf64008040 comp rx=0 tx=0).stop 2024-09-19T08:43:49.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.432+0000 7fdf78337640 1 -- 172.21.15.98:0/2310320771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf700d25f0 msgr2=0x7fdf700cd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:49.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.432+0000 7fdf78337640 1 --2- 172.21.15.98:0/2310320771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf700d25f0 0x7fdf700cd880 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fdf6001cad0 tx=0x7fdf6001aa70 comp rx=0 tx=0).stop 2024-09-19T08:43:49.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.432+0000 7fdf78337640 1 -- 172.21.15.98:0/2310320771 shutdown_connections 2024-09-19T08:43:49.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.432+0000 7fdf78337640 1 --2- 172.21.15.98:0/2310320771 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fdf5c067a10 0x7fdf5c069ed0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:49.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.432+0000 7fdf78337640 1 --2- 172.21.15.98:0/2310320771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf701415a0 0x7fdf701577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:49.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.432+0000 7fdf78337640 1 --2- 172.21.15.98:0/2310320771 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf700d38a0 0x7fdf700d1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:49.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.432+0000 7fdf78337640 1 --2- 172.21.15.98:0/2310320771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf700d25f0 0x7fdf700cd880 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:49.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.432+0000 7fdf78337640 1 -- 172.21.15.98:0/2310320771 >> 172.21.15.98:0/2310320771 conn(0x7fdf70074c70 msgr2=0x7fdf7015d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:49.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.432+0000 7fdf78337640 1 -- 172.21.15.98:0/2310320771 shutdown_connections 2024-09-19T08:43:49.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.432+0000 7fdf78337640 1 -- 172.21.15.98:0/2310320771 wait complete. 2024-09-19T08:43:49.450 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T08:43:49.450 DEBUG:teuthology.orchestra.run.smithi098:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:43:49.513 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:43:49.602 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T08:43:49.602 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:43:49.602 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:43:49.608 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15846 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T08:43:49.609 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:43:49.609 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:43:49.616 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T08:43:49.616 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:43:49.616 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:43:49.657 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15846 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T08:43:49.657 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:43:49.657 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:43:49.665 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T08:43:49.665 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-19T08:43:49.665 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:43:49.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.824+0000 7fbe6b98b640 1 Processor -- start 2024-09-19T08:43:49.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.828+0000 7fbe6b98b640 1 -- start start 2024-09-19T08:43:49.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.828+0000 7fbe6b98b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbe6415c550 0x7fbe6406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:49.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.828+0000 7fbe6b98b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe6406c1d0 0x7fbe640cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:49.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.828+0000 7fbe6b98b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe6406d310 con 0x7fbe640cf900 2024-09-19T08:43:49.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.828+0000 7fbe6b98b640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fbe6406c610 con 0x7fbe6415c550 2024-09-19T08:43:49.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.828+0000 7fbe6b98b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbe6406d5f0 con 0x7fbe6406c1d0 2024-09-19T08:43:49.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.828+0000 7fbe69727640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbe6415c550 0x7fbe6406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:49.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.828+0000 7fbe69727640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbe6415c550 0x7fbe6406bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:32822/0 (socket says 172.21.15.98:32822) 2024-09-19T08:43:49.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.828+0000 7fbe69727640 1 -- 172.21.15.98:0/1777341943 learned_addr learned my addr 172.21.15.98:0/1777341943 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:49.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.828+0000 7fbe68f26640 1 --2- 172.21.15.98:0/1777341943 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe6406c1d0 0x7fbe640cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:49.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.828+0000 7fbe69727640 1 -- 172.21.15.98:0/1777341943 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe6406c1d0 msgr2=0x7fbe640cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:49.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.828+0000 7fbe69727640 1 --2- 172.21.15.98:0/1777341943 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe6406c1d0 0x7fbe640cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:49.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.828+0000 7fbe69727640 1 -- 172.21.15.98:0/1777341943 >> v1:172.21.15.139:6789/0 conn(0x7fbe640cf900 legacy=0x7fbe640ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:49.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.828+0000 7fbe69727640 1 -- 172.21.15.98:0/1777341943 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe6406d8d0 con 0x7fbe6415c550 2024-09-19T08:43:49.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.828+0000 7fbe69727640 1 --2- 172.21.15.98:0/1777341943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe6415c550 0x7fbe6406bc90 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fbe60009870 tx=0x7fbe6001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6e9ed78745f780 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:49.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.828+0000 7fbe5bfff640 1 -- 172.21.15.98:0/1777341943 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe600b0070 con 0x7fbe6415c550 2024-09-19T08:43:49.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.828+0000 7fbe5bfff640 1 -- 172.21.15.98:0/1777341943 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe600ab600 con 0x7fbe6415c550 2024-09-19T08:43:49.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.832+0000 7fbe5bfff640 1 -- 172.21.15.98:0/1777341943 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe600ab9d0 con 0x7fbe6415c550 2024-09-19T08:43:49.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.832+0000 7fbe6b98b640 1 -- 172.21.15.98:0/1777341943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe6415c550 msgr2=0x7fbe6406bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:49.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.832+0000 7fbe6b98b640 1 --2- 172.21.15.98:0/1777341943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe6415c550 0x7fbe6406bc90 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fbe60009870 tx=0x7fbe6001c520 comp rx=0 tx=0).stop 2024-09-19T08:43:49.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.832+0000 7fbe6b98b640 1 -- 172.21.15.98:0/1777341943 shutdown_connections 2024-09-19T08:43:49.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.832+0000 7fbe6b98b640 1 --2- 172.21.15.98:0/1777341943 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe6406c1d0 0x7fbe640cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:49.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.832+0000 7fbe6b98b640 1 --2- 172.21.15.98:0/1777341943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe6415c550 0x7fbe6406bc90 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:49.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.832+0000 7fbe6b98b640 1 -- 172.21.15.98:0/1777341943 >> 172.21.15.98:0/1777341943 conn(0x7fbe640b2010 msgr2=0x7fbe640b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:49.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.832+0000 7fbe6b98b640 1 -- 172.21.15.98:0/1777341943 shutdown_connections 2024-09-19T08:43:49.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.832+0000 7fbe6b98b640 1 -- 172.21.15.98:0/1777341943 wait complete. 2024-09-19T08:43:49.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.832+0000 7fbe6b98b640 1 Processor -- start 2024-09-19T08:43:49.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.832+0000 7fbe6b98b640 1 -- start start 2024-09-19T08:43:49.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.832+0000 7fbe6b98b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe6406c1d0 0x7fbe641fac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:49.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.832+0000 7fbe6b98b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe640cf900 0x7fbe64144b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:49.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.832+0000 7fbe6b98b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe6415c550 0x7fbe640d0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:49.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.832+0000 7fbe6b98b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbe64169230 con 0x7fbe6406c1d0 2024-09-19T08:43:49.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.832+0000 7fbe6b98b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6415cb40 con 0x7fbe6415c550 2024-09-19T08:43:49.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.832+0000 7fbe6b98b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbe641693b0 con 0x7fbe640cf900 2024-09-19T08:43:49.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.832+0000 7fbe69727640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe6406c1d0 0x7fbe641fac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:49.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.832+0000 7fbe69727640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe6406c1d0 0x7fbe641fac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:32834/0 (socket says 172.21.15.98:32834) 2024-09-19T08:43:49.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.832+0000 7fbe69727640 1 -- 172.21.15.98:0/2104642944 learned_addr learned my addr 172.21.15.98:0/2104642944 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:49.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.832+0000 7fbe69727640 1 -- 172.21.15.98:0/2104642944 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe640cf900 msgr2=0x7fbe64144b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:49.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.832+0000 7fbe69727640 1 --2- 172.21.15.98:0/2104642944 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe640cf900 0x7fbe64144b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:49.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.832+0000 7fbe69727640 1 -- 172.21.15.98:0/2104642944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe6415c550 msgr2=0x7fbe640d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:49.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.832+0000 7fbe69727640 1 --2- 172.21.15.98:0/2104642944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe6415c550 0x7fbe640d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:49.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.832+0000 7fbe69727640 1 -- 172.21.15.98:0/2104642944 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe641fbb60 con 0x7fbe6406c1d0 2024-09-19T08:43:49.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.832+0000 7fbe69727640 1 --2- 172.21.15.98:0/2104642944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe6406c1d0 0x7fbe641fac30 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fbe600ae040 tx=0x7fbe6001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:49.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.832+0000 7fbe59ffb640 1 -- 172.21.15.98:0/2104642944 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe600ba020 con 0x7fbe6406c1d0 2024-09-19T08:43:49.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.832+0000 7fbe59ffb640 1 -- 172.21.15.98:0/2104642944 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe600b6640 con 0x7fbe6406c1d0 2024-09-19T08:43:49.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.832+0000 7fbe6b98b640 1 -- 172.21.15.98:0/2104642944 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe641fb2c0 con 0x7fbe6406c1d0 2024-09-19T08:43:49.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.832+0000 7fbe59ffb640 1 -- 172.21.15.98:0/2104642944 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe600b0040 con 0x7fbe6406c1d0 2024-09-19T08:43:49.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.832+0000 7fbe6b98b640 1 -- 172.21.15.98:0/2104642944 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe641fb880 con 0x7fbe6406c1d0 2024-09-19T08:43:49.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.836+0000 7fbe6b98b640 1 -- 172.21.15.98:0/2104642944 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe640cf3c0 con 0x7fbe6406c1d0 2024-09-19T08:43:49.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.836+0000 7fbe59ffb640 1 -- 172.21.15.98:0/2104642944 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fbe600bd020 con 0x7fbe6406c1d0 2024-09-19T08:43:49.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.836+0000 7fbe59ffb640 1 --2- 172.21.15.98:0/2104642944 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbe480679e0 0x7fbe48069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:49.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.836+0000 7fbe59ffb640 1 -- 172.21.15.98:0/2104642944 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7fbe6011ffe0 con 0x7fbe6406c1d0 2024-09-19T08:43:49.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.836+0000 7fbe68f26640 1 --2- 172.21.15.98:0/2104642944 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbe480679e0 0x7fbe48069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:49.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.836+0000 7fbe68f26640 1 --2- 172.21.15.98:0/2104642944 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbe480679e0 0x7fbe48069ea0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbe640b11b0 tx=0x7fbe54008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:49.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:49.840+0000 7fbe59ffb640 1 -- 172.21.15.98:0/2104642944 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fbe600c4030 con 0x7fbe6406c1d0 2024-09-19T08:43:50.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.036+0000 7fbe6b98b640 1 -- 172.21.15.98:0/2104642944 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe640b0a90 con 0x7fbe6406c1d0 2024-09-19T08:43:50.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.040+0000 7fbe59ffb640 1 -- 172.21.15.98:0/2104642944 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+3914 (secure 0 0 0) 0x7fbe60016110 con 0x7fbe6406c1d0 2024-09-19T08:43:50.041 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:50.041 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":54,"btime":"2024-09-19T08:43:43:748755+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3182634136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3182634136},{"type":"v1","addr":"172.21.15.139:6833","nonce":3182634136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5830,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1554653172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1554653172},{"type":"v1","addr":"172.21.15.139:6835","nonce":1554653172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6849,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/872072758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":872072758},{"type":"v1","addr":"172.21.15.98:6833","nonce":872072758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":6918,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/2397696057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":2397696057},{"type":"v1","addr":"172.21.15.98:6835","nonce":2397696057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":54}],"filesystems":[]} 2024-09-19T08:43:50.041 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 54 2024-09-19T08:43:50.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.040+0000 7fbe6b98b640 1 -- 172.21.15.98:0/2104642944 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbe480679e0 msgr2=0x7fbe48069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:50.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.040+0000 7fbe6b98b640 1 --2- 172.21.15.98:0/2104642944 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbe480679e0 0x7fbe48069ea0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbe640b11b0 tx=0x7fbe54008040 comp rx=0 tx=0).stop 2024-09-19T08:43:50.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.040+0000 7fbe6b98b640 1 -- 172.21.15.98:0/2104642944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe6406c1d0 msgr2=0x7fbe641fac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:50.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.040+0000 7fbe6b98b640 1 --2- 172.21.15.98:0/2104642944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe6406c1d0 0x7fbe641fac30 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fbe600ae040 tx=0x7fbe6001ad60 comp rx=0 tx=0).stop 2024-09-19T08:43:50.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.040+0000 7fbe6b98b640 1 -- 172.21.15.98:0/2104642944 shutdown_connections 2024-09-19T08:43:50.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.040+0000 7fbe6b98b640 1 --2- 172.21.15.98:0/2104642944 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbe480679e0 0x7fbe48069ea0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:50.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.040+0000 7fbe6b98b640 1 --2- 172.21.15.98:0/2104642944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbe6415c550 0x7fbe640d0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:50.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.040+0000 7fbe6b98b640 1 --2- 172.21.15.98:0/2104642944 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbe640cf900 0x7fbe64144b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:50.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.040+0000 7fbe6b98b640 1 --2- 172.21.15.98:0/2104642944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbe6406c1d0 0x7fbe641fac30 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:50.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.040+0000 7fbe6b98b640 1 -- 172.21.15.98:0/2104642944 >> 172.21.15.98:0/2104642944 conn(0x7fbe640b2010 msgr2=0x7fbe640b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:50.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.040+0000 7fbe6b98b640 1 -- 172.21.15.98:0/2104642944 shutdown_connections 2024-09-19T08:43:50.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.040+0000 7fbe6b98b640 1 -- 172.21.15.98:0/2104642944 wait complete. 2024-09-19T08:43:50.059 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-19T08:43:50.060 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-19T08:43:50.060 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-19T08:43:50.060 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:43:50.060 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-19T08:43:50.060 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-19T08:43:50.060 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-19T08:43:50.060 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:43:50.061 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-19T08:43:50.061 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-19T08:43:50.061 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-19T08:43:50.061 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:43:50.061 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-19T08:43:50.061 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-19T08:43:50.061 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-19T08:43:50.061 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:43:50.069 INFO:tasks.ceph.mds.a:Stopped 2024-09-19T08:43:50.070 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-19T08:43:50.071 INFO:tasks.ceph.mds.c:Stopped 2024-09-19T08:43:50.071 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-19T08:43:50.073 INFO:tasks.ceph.mds.b:Stopped 2024-09-19T08:43:50.073 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-19T08:43:50.077 INFO:tasks.ceph.mds.d:Stopped 2024-09-19T08:43:50.077 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-19T08:43:50.117 INFO:tasks.ceph.mds.a:Started 2024-09-19T08:43:50.118 DEBUG:teuthology.parallel:result is None 2024-09-19T08:43:50.118 INFO:tasks.ceph.mds.c:Started 2024-09-19T08:43:50.119 DEBUG:teuthology.parallel:result is None 2024-09-19T08:43:50.121 INFO:tasks.ceph.mds.b:Started 2024-09-19T08:43:50.121 DEBUG:teuthology.parallel:result is None 2024-09-19T08:43:50.122 INFO:tasks.ceph.mds.d:Started 2024-09-19T08:43:50.123 DEBUG:teuthology.parallel:result is None 2024-09-19T08:43:50.123 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-19T08:43:50.177 INFO:tasks.ceph.mds.a.smithi098.stdout:starting mds.a at 2024-09-19T08:43:50.184 INFO:tasks.ceph.mds.c.smithi098.stdout:starting mds.c at 2024-09-19T08:43:50.186 INFO:tasks.ceph.mds.b.smithi139.stdout:starting mds.b at 2024-09-19T08:43:50.188 INFO:tasks.ceph.mds.d.smithi139.stdout:starting mds.d at 2024-09-19T08:43:50.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.236+0000 7fc24c3db640 1 Processor -- start 2024-09-19T08:43:50.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.236+0000 7fc24c3db640 1 -- start start 2024-09-19T08:43:50.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.240+0000 7fc24c3db640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc244147530 0x7fc2440d1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:50.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.240+0000 7fc24c3db640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc2440d1d70 0x7fc24415ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:50.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.240+0000 7fc24c3db640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc2440b6200 0x7fc2440b6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:50.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.240+0000 7fc24c3db640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fc2440b6bb0 con 0x7fc2440d1d70 2024-09-19T08:43:50.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.240+0000 7fc24c3db640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fc2440d2230 con 0x7fc2440b6200 2024-09-19T08:43:50.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.240+0000 7fc24c3db640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc2440b8060 con 0x7fc244147530 2024-09-19T08:43:50.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.240+0000 7fc249976640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc2440d1d70 0x7fc24415ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:50.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.240+0000 7fc249976640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc2440d1d70 0x7fc24415ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:32868/0 (socket says 172.21.15.98:32868) 2024-09-19T08:43:50.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.240+0000 7fc249976640 1 -- 172.21.15.98:0/2973476008 learned_addr learned my addr 172.21.15.98:0/2973476008 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:50.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.240+0000 7fc24a177640 1 --2- 172.21.15.98:0/2973476008 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc244147530 0x7fc2440d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:50.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.240+0000 7fc24a978640 1 --2- 172.21.15.98:0/2973476008 >> v2:172.21.15.139:3300/0 conn(0x7fc2440b6200 0x7fc2440b6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:50.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.240+0000 7fc249976640 1 -- 172.21.15.98:0/2973476008 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc244147530 msgr2=0x7fc2440d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:50.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.240+0000 7fc249976640 1 --2- 172.21.15.98:0/2973476008 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc244147530 0x7fc2440d1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:50.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.240+0000 7fc249976640 1 -- 172.21.15.98:0/2973476008 >> v2:172.21.15.139:3300/0 conn(0x7fc2440b6200 msgr2=0x7fc2440b6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:50.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.240+0000 7fc249976640 1 --2- 172.21.15.98:0/2973476008 >> v2:172.21.15.139:3300/0 conn(0x7fc2440b6200 0x7fc2440b6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:50.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.240+0000 7fc249976640 1 -- 172.21.15.98:0/2973476008 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2440b8340 con 0x7fc2440d1d70 2024-09-19T08:43:50.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.240+0000 7fc249976640 1 --2- 172.21.15.98:0/2973476008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2440d1d70 0x7fc24415ec70 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fc2440d0560 tx=0x7fc23401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59bfa9398500b9c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:50.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.240+0000 7fc249175640 1 -- 172.21.15.98:0/2973476008 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2340aa610 con 0x7fc2440d1d70 2024-09-19T08:43:50.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.240+0000 7fc249175640 1 -- 172.21.15.98:0/2973476008 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2340af070 con 0x7fc2440d1d70 2024-09-19T08:43:50.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.240+0000 7fc249175640 1 -- 172.21.15.98:0/2973476008 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2340aa9d0 con 0x7fc2440d1d70 2024-09-19T08:43:50.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.240+0000 7fc24c3db640 1 -- 172.21.15.98:0/2973476008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2440d1d70 msgr2=0x7fc24415ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:50.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.240+0000 7fc24c3db640 1 --2- 172.21.15.98:0/2973476008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2440d1d70 0x7fc24415ec70 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fc2440d0560 tx=0x7fc23401c4f0 comp rx=0 tx=0).stop 2024-09-19T08:43:50.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.240+0000 7fc24c3db640 1 -- 172.21.15.98:0/2973476008 shutdown_connections 2024-09-19T08:43:50.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.240+0000 7fc24c3db640 1 --2- 172.21.15.98:0/2973476008 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2440d1d70 0x7fc24415ec70 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:50.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.240+0000 7fc24c3db640 1 --2- 172.21.15.98:0/2973476008 >> v2:172.21.15.139:3300/0 conn(0x7fc2440b6200 0x7fc2440b6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:50.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.240+0000 7fc24c3db640 1 --2- 172.21.15.98:0/2973476008 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc244147530 0x7fc2440d1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:50.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.240+0000 7fc24c3db640 1 -- 172.21.15.98:0/2973476008 >> 172.21.15.98:0/2973476008 conn(0x7fc244075070 msgr2=0x7fc2440722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:50.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.240+0000 7fc24c3db640 1 -- 172.21.15.98:0/2973476008 shutdown_connections 2024-09-19T08:43:50.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.240+0000 7fc24c3db640 1 -- 172.21.15.98:0/2973476008 wait complete. 2024-09-19T08:43:50.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.244+0000 7fc24c3db640 1 Processor -- start 2024-09-19T08:43:50.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.244+0000 7fc24c3db640 1 -- start start 2024-09-19T08:43:50.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.244+0000 7fc24c3db640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2440b6200 0x7fc2441424e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:50.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.244+0000 7fc24c3db640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2440d1d70 0x7fc24415c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:50.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.244+0000 7fc24c3db640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc244147530 0x7fc2440d4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:50.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.244+0000 7fc24c3db640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc244147910 con 0x7fc2440b6200 2024-09-19T08:43:50.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.244+0000 7fc24c3db640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc2441483a0 con 0x7fc2440d1d70 2024-09-19T08:43:50.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.244+0000 7fc24c3db640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc244147a90 con 0x7fc244147530 2024-09-19T08:43:50.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.244+0000 7fc24a177640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2440b6200 0x7fc2441424e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:50.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.244+0000 7fc24a177640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2440b6200 0x7fc2441424e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:32884/0 (socket says 172.21.15.98:32884) 2024-09-19T08:43:50.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.244+0000 7fc24a177640 1 -- 172.21.15.98:0/648575775 learned_addr learned my addr 172.21.15.98:0/648575775 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:50.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.244+0000 7fc249976640 1 --2- 172.21.15.98:0/648575775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2440d1d70 0x7fc24415c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:50.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.244+0000 7fc24a978640 1 --2- 172.21.15.98:0/648575775 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc244147530 0x7fc2440d4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:50.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.244+0000 7fc24a177640 1 -- 172.21.15.98:0/648575775 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc244147530 msgr2=0x7fc2440d4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:50.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.244+0000 7fc24a177640 1 --2- 172.21.15.98:0/648575775 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc244147530 0x7fc2440d4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:50.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.244+0000 7fc24a177640 1 -- 172.21.15.98:0/648575775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2440d1d70 msgr2=0x7fc24415c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:50.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.244+0000 7fc24a177640 1 --2- 172.21.15.98:0/648575775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2440d1d70 0x7fc24415c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:50.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.244+0000 7fc24a177640 1 -- 172.21.15.98:0/648575775 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc244145830 con 0x7fc2440b6200 2024-09-19T08:43:50.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.244+0000 7fc249976640 1 --2- 172.21.15.98:0/648575775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2440d1d70 0x7fc24415c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:43:50.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.244+0000 7fc24a177640 1 --2- 172.21.15.98:0/648575775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2440b6200 0x7fc2441424e0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fc24000d830 tx=0x7fc24000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:50.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.244+0000 7fc23affd640 1 -- 172.21.15.98:0/648575775 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24000b6c0 con 0x7fc2440b6200 2024-09-19T08:43:50.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.244+0000 7fc23affd640 1 -- 172.21.15.98:0/648575775 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc240019070 con 0x7fc2440b6200 2024-09-19T08:43:50.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.244+0000 7fc24c3db640 1 -- 172.21.15.98:0/648575775 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc244144f90 con 0x7fc2440b6200 2024-09-19T08:43:50.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.244+0000 7fc23affd640 1 -- 172.21.15.98:0/648575775 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24000ba60 con 0x7fc2440b6200 2024-09-19T08:43:50.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.244+0000 7fc24c3db640 1 -- 172.21.15.98:0/648575775 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc244145550 con 0x7fc2440b6200 2024-09-19T08:43:50.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.244+0000 7fc24c3db640 1 -- 172.21.15.98:0/648575775 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc24415ec70 con 0x7fc2440b6200 2024-09-19T08:43:50.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.244+0000 7fc23affd640 1 -- 172.21.15.98:0/648575775 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fc24001e460 con 0x7fc2440b6200 2024-09-19T08:43:50.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.248+0000 7fc23affd640 1 --2- 172.21.15.98:0/648575775 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc2280679e0 0x7fc228069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:50.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.248+0000 7fc23affd640 1 -- 172.21.15.98:0/648575775 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5029+0+0 (secure 0 0 0) 0x7fc240086c70 con 0x7fc2440b6200 2024-09-19T08:43:50.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.248+0000 7fc249976640 1 --2- 172.21.15.98:0/648575775 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc2280679e0 0x7fc228069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:50.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.248+0000 7fc249976640 1 --2- 172.21.15.98:0/648575775 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc2280679e0 0x7fc228069ea0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc2340ae000 tx=0x7fc2340a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:50.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.252+0000 7fc23affd640 1 -- 172.21.15.98:0/648575775 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc2400579b0 con 0x7fc2440b6200 2024-09-19T08:43:50.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:50.424+0000 7fc24c3db640 1 -- 172.21.15.98:0/648575775 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fc2440d05e0 con 0x7fc2440b6200 2024-09-19T08:43:51.123 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.120+0000 7fc23affd640 1 -- 172.21.15.98:0/648575775 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v96) ==== 97+0+0 (secure 0 0 0) 0x7fc2400572b0 con 0x7fc2440b6200 2024-09-19T08:43:51.123 INFO:teuthology.orchestra.run.smithi098.stderr: removed all blocklist entries 2024-09-19T08:43:51.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.124+0000 7fc24c3db640 1 -- 172.21.15.98:0/648575775 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc2280679e0 msgr2=0x7fc228069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:51.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.124+0000 7fc24c3db640 1 --2- 172.21.15.98:0/648575775 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc2280679e0 0x7fc228069ea0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc2340ae000 tx=0x7fc2340a7000 comp rx=0 tx=0).stop 2024-09-19T08:43:51.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.124+0000 7fc24c3db640 1 -- 172.21.15.98:0/648575775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2440b6200 msgr2=0x7fc2441424e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:51.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.124+0000 7fc24c3db640 1 --2- 172.21.15.98:0/648575775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2440b6200 0x7fc2441424e0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fc24000d830 tx=0x7fc24000dcf0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.124+0000 7fc24c3db640 1 -- 172.21.15.98:0/648575775 shutdown_connections 2024-09-19T08:43:51.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.124+0000 7fc24c3db640 1 --2- 172.21.15.98:0/648575775 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc2280679e0 0x7fc228069ea0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.124+0000 7fc24c3db640 1 --2- 172.21.15.98:0/648575775 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc244147530 0x7fc2440d4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.124+0000 7fc24c3db640 1 --2- 172.21.15.98:0/648575775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2440d1d70 0x7fc24415c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.124+0000 7fc24c3db640 1 --2- 172.21.15.98:0/648575775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2440b6200 0x7fc2441424e0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.124+0000 7fc24c3db640 1 -- 172.21.15.98:0/648575775 >> 172.21.15.98:0/648575775 conn(0x7fc244075070 msgr2=0x7fc2441466b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:51.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.124+0000 7fc24c3db640 1 -- 172.21.15.98:0/648575775 shutdown_connections 2024-09-19T08:43:51.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.124+0000 7fc24c3db640 1 -- 172.21.15.98:0/648575775 wait complete. 2024-09-19T08:43:51.151 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-19T08:43:51.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.272+0000 7f24aa888640 1 Processor -- start 2024-09-19T08:43:51.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.272+0000 7f24aa888640 1 -- start start 2024-09-19T08:43:51.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.272+0000 7f24aa888640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f24a415c550 0x7f24a406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:51.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.272+0000 7f24aa888640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f24a40cfad0 0x7f24a40723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:51.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.272+0000 7f24aa888640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24a406d310 con 0x7f24a406c1d0 2024-09-19T08:43:51.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.272+0000 7f24aa888640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f24a406c640 con 0x7f24a40cfad0 2024-09-19T08:43:51.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.272+0000 7f24aa888640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f24a406d5f0 con 0x7f24a415c550 2024-09-19T08:43:51.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.272+0000 7f24a8e25640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f24a40cfad0 0x7f24a40723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:51.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.272+0000 7f24a8e25640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f24a40cfad0 0x7f24a40723a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:32900/0 (socket says 172.21.15.98:32900) 2024-09-19T08:43:51.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.272+0000 7f24a8e25640 1 -- 172.21.15.98:0/4180015773 learned_addr learned my addr 172.21.15.98:0/4180015773 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:51.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.272+0000 7f24a8e25640 1 -- 172.21.15.98:0/4180015773 >> v2:172.21.15.139:3300/0 conn(0x7f24a415c550 msgr2=0x7f24a406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:51.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.272+0000 7f24a8e25640 1 --2- 172.21.15.98:0/4180015773 >> v2:172.21.15.139:3300/0 conn(0x7f24a415c550 0x7f24a406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.272+0000 7f24a8e25640 1 -- 172.21.15.98:0/4180015773 >> v1:172.21.15.139:6789/0 conn(0x7f24a406c1d0 legacy=0x7f24a40cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:51.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.272+0000 7f24a8e25640 1 -- 172.21.15.98:0/4180015773 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a406d8d0 con 0x7f24a40cfad0 2024-09-19T08:43:51.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.276+0000 7f24a8e25640 1 --2- 172.21.15.98:0/4180015773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24a40cfad0 0x7f24a40723a0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f24a40b56b0 tx=0x7f249c01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e30e05f9cfd948 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:51.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.276+0000 7f24a2ffd640 1 -- 172.21.15.98:0/4180015773 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c0a5670 con 0x7f24a40cfad0 2024-09-19T08:43:51.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.276+0000 7f24a2ffd640 1 -- 172.21.15.98:0/4180015773 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f249c0a5810 con 0x7f24a40cfad0 2024-09-19T08:43:51.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.276+0000 7f24a2ffd640 1 -- 172.21.15.98:0/4180015773 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c0a5be0 con 0x7f24a40cfad0 2024-09-19T08:43:51.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.276+0000 7f24aa888640 1 -- 172.21.15.98:0/4180015773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24a40cfad0 msgr2=0x7f24a40723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:51.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.276+0000 7f24aa888640 1 --2- 172.21.15.98:0/4180015773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24a40cfad0 0x7f24a40723a0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f24a40b56b0 tx=0x7f249c01c9b0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.276+0000 7f24aa888640 1 -- 172.21.15.98:0/4180015773 shutdown_connections 2024-09-19T08:43:51.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.276+0000 7f24aa888640 1 --2- 172.21.15.98:0/4180015773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24a40cfad0 0x7f24a40723a0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.276+0000 7f24aa888640 1 --2- 172.21.15.98:0/4180015773 >> v2:172.21.15.139:3300/0 conn(0x7f24a415c550 0x7f24a406bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.276+0000 7f24aa888640 1 -- 172.21.15.98:0/4180015773 >> 172.21.15.98:0/4180015773 conn(0x7f24a40b2010 msgr2=0x7f24a40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:51.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.276+0000 7f24aa888640 1 -- 172.21.15.98:0/4180015773 shutdown_connections 2024-09-19T08:43:51.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.276+0000 7f24aa888640 1 -- 172.21.15.98:0/4180015773 wait complete. 2024-09-19T08:43:51.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.276+0000 7f24aa888640 1 Processor -- start 2024-09-19T08:43:51.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.276+0000 7f24aa888640 1 -- start start 2024-09-19T08:43:51.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.276+0000 7f24aa888640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24a406c1d0 0x7f24a4159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:51.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.276+0000 7f24aa888640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24a40cfad0 0x7f24a41575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:51.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.276+0000 7f24aa888640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24a415c550 0x7f24a41567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:51.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.276+0000 7f24aa888640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f24a40734c0 con 0x7f24a415c550 2024-09-19T08:43:51.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.276+0000 7f24aa888640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f24a4073340 con 0x7f24a406c1d0 2024-09-19T08:43:51.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.276+0000 7f24aa888640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f24a4073640 con 0x7f24a40cfad0 2024-09-19T08:43:51.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.276+0000 7f24a8e25640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24a415c550 0x7f24a41567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:51.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.276+0000 7f24a8e25640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24a415c550 0x7f24a41567f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:32904/0 (socket says 172.21.15.98:32904) 2024-09-19T08:43:51.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.276+0000 7f24a8e25640 1 -- 172.21.15.98:0/4065494282 learned_addr learned my addr 172.21.15.98:0/4065494282 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:51.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.276+0000 7f24a8e25640 1 -- 172.21.15.98:0/4065494282 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24a40cfad0 msgr2=0x7f24a41575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:51.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.276+0000 7f24a8e25640 1 --2- 172.21.15.98:0/4065494282 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24a40cfad0 0x7f24a41575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.276+0000 7f24a8e25640 1 -- 172.21.15.98:0/4065494282 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24a406c1d0 msgr2=0x7f24a4159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:51.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.276+0000 7f24a8e25640 1 --2- 172.21.15.98:0/4065494282 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24a406c1d0 0x7f24a4159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.276+0000 7f24a8e25640 1 -- 172.21.15.98:0/4065494282 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a41585f0 con 0x7f24a415c550 2024-09-19T08:43:51.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.276+0000 7f24a8e25640 1 --2- 172.21.15.98:0/4065494282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24a415c550 0x7f24a41567f0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f249c01a5e0 tx=0x7f249c019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:51.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.276+0000 7f24a0ff9640 1 -- 172.21.15.98:0/4065494282 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c0bc020 con 0x7f24a415c550 2024-09-19T08:43:51.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.276+0000 7f24a0ff9640 1 -- 172.21.15.98:0/4065494282 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f249c0b8670 con 0x7f24a415c550 2024-09-19T08:43:51.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.276+0000 7f24aa888640 1 -- 172.21.15.98:0/4065494282 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24a4157d50 con 0x7f24a415c550 2024-09-19T08:43:51.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.276+0000 7f24a0ff9640 1 -- 172.21.15.98:0/4065494282 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c0b8a20 con 0x7f24a415c550 2024-09-19T08:43:51.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.276+0000 7f24aa888640 1 -- 172.21.15.98:0/4065494282 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24a4158310 con 0x7f24a415c550 2024-09-19T08:43:51.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.280+0000 7f24aa888640 1 -- 172.21.15.98:0/4065494282 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24a40723a0 con 0x7f24a415c550 2024-09-19T08:43:51.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.280+0000 7f24a0ff9640 1 -- 172.21.15.98:0/4065494282 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f249c0bf020 con 0x7f24a415c550 2024-09-19T08:43:51.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.280+0000 7f24a0ff9640 1 --2- 172.21.15.98:0/4065494282 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f24900679e0 0x7f2490069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:51.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.280+0000 7f24a0ff9640 1 -- 172.21.15.98:0/4065494282 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 4857+0+0 (secure 0 0 0) 0x7f249c120e90 con 0x7f24a415c550 2024-09-19T08:43:51.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.280+0000 7f24a3fff640 1 --2- 172.21.15.98:0/4065494282 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f24900679e0 0x7f2490069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:51.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.284+0000 7f24a0ff9640 1 -- 172.21.15.98:0/4065494282 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f249c0c6030 con 0x7f24a415c550 2024-09-19T08:43:51.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.284+0000 7f24a3fff640 1 --2- 172.21.15.98:0/4065494282 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f24900679e0 0x7f2490069ea0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2494009730 tx=0x7f2494006ca0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:51.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.500+0000 7f24aa888640 1 -- 172.21.15.98:0/4065494282 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f24a4150cd0 con 0x7f24a415c550 2024-09-19T08:43:51.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.500+0000 7f24a0ff9640 1 -- 172.21.15.98:0/4065494282 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) ==== 80+0+5304 (secure 0 0 0) 0x7f249c0165a0 con 0x7f24a415c550 2024-09-19T08:43:51.503 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:51.503 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:43:51.503 INFO:teuthology.orchestra.run.smithi098.stdout: "auth_dump": [ 2024-09-19T08:43:51.503 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:51.503 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.a", 2024-09-19T08:43:51.503 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmuujGBRAAa2k/kW68NJI6nUKx+W+G2Q==", 2024-09-19T08:43:51.504 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:51.504 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:43:51.504 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:43:51.504 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:43:51.504 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:43:51.504 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:51.504 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:51.504 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:51.504 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.b", 2024-09-19T08:43:51.504 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmFoojFBAAaOZPpa41fdjBPEn3CGsJnA==", 2024-09-19T08:43:51.504 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:51.504 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:43:51.504 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:43:51.505 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:43:51.505 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:43:51.505 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:51.505 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:51.505 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:51.505 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.c", 2024-09-19T08:43:51.505 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmcQcDDhAAF0QJMlljMH+DiHfWW+/Sog==", 2024-09-19T08:43:51.505 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:51.505 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:43:51.505 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:43:51.505 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:43:51.505 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:43:51.505 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:51.505 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:51.506 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:51.506 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.d", 2024-09-19T08:43:51.506 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmwWmqHBAAf15VcrdeHf8AzQjGsN9LoA==", 2024-09-19T08:43:51.506 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:51.506 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:43:51.506 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:43:51.506 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:43:51.506 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:43:51.506 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:51.506 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:51.506 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:51.506 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.0", 2024-09-19T08:43:51.506 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDC4utm9PjNKxAAa6SG9AE4smPiu4azRAFKvg==", 2024-09-19T08:43:51.506 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:51.507 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:43:51.507 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:43:51.507 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:43:51.507 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:51.507 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:51.507 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:51.507 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.1", 2024-09-19T08:43:51.507 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDG4utmHy6UCxAAN6hpG4DJEp81iG/KoloTvQ==", 2024-09-19T08:43:51.507 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:51.508 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:43:51.508 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:43:51.508 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:43:51.508 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:51.508 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:51.508 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:51.508 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.2", 2024-09-19T08:43:51.508 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDJ4utmUVP4GhAAJNAL30sEAseoDG2fE6YbZQ==", 2024-09-19T08:43:51.508 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:51.508 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:43:51.508 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:43:51.508 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:43:51.508 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:51.509 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:51.509 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:51.509 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.3", 2024-09-19T08:43:51.509 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDM4utmTKgoLRAAiCrPRLML+wvjOt6RhKay3w==", 2024-09-19T08:43:51.509 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:51.509 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:43:51.509 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:43:51.509 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:43:51.509 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:51.509 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:51.509 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:51.509 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.4", 2024-09-19T08:43:51.509 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDS4utmAf6bMhAAI3uENuFoXLicUeuspgFjYA==", 2024-09-19T08:43:51.510 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:51.510 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:43:51.510 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:43:51.510 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:43:51.510 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:51.510 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:51.510 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:51.510 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.5", 2024-09-19T08:43:51.510 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDW4utmRjxwEhAAF1A2GtrZXVx9BlmjHjhUDQ==", 2024-09-19T08:43:51.510 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:51.510 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:43:51.510 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:43:51.510 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:43:51.510 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:51.511 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:51.511 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:51.511 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.6", 2024-09-19T08:43:51.511 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDZ4utmSk/XIRAAROSfzGL7smR88ZkDSx5jAg==", 2024-09-19T08:43:51.511 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:51.511 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:43:51.511 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:43:51.511 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:43:51.511 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:51.511 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:51.511 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:51.511 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.7", 2024-09-19T08:43:51.511 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDc4utmXfpbMxAAqEM4I/ACFvfu/sMA8bMcrQ==", 2024-09-19T08:43:51.512 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:51.512 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:43:51.512 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:43:51.512 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:43:51.512 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:51.512 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:51.512 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:51.512 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.0", 2024-09-19T08:43:51.512 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utm0vNpIhAA4S1E8dFZ0+ibmSKhmZ+bIA==", 2024-09-19T08:43:51.512 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:51.512 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:43:51.512 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow r", 2024-09-19T08:43:51.512 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-19T08:43:51.512 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:51.513 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:51.513 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:51.513 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.1", 2024-09-19T08:43:51.513 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmx7xyKhAAC84lykPEb01MiypXKQVREQ==", 2024-09-19T08:43:51.513 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:51.513 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:43:51.513 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-19T08:43:51.513 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-19T08:43:51.513 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-19T08:43:51.513 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:51.514 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:51.514 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:51.514 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.2", 2024-09-19T08:43:51.514 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmteOQJxAAtTF2Py21ZE2WRQjeQSrkKQ==", 2024-09-19T08:43:51.516 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:51.516 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:43:51.516 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-19T08:43:51.516 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-19T08:43:51.516 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-19T08:43:51.516 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:51.516 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:51.516 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:51.517 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.3", 2024-09-19T08:43:51.517 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmKN92LxAAMqraZOWRT6IIfx5AuDWS2g==", 2024-09-19T08:43:51.517 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:51.517 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:43:51.517 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-19T08:43:51.517 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-19T08:43:51.517 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-19T08:43:51.518 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:51.518 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:51.518 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:51.518 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.admin", 2024-09-19T08:43:51.518 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC+4utmLH97KhAA1DTYW5bTrGUG+7cb8+d7gA==", 2024-09-19T08:43:51.518 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:51.518 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-09-19T08:43:51.518 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:43:51.518 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:43:51.518 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:43:51.518 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:51.518 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:51.518 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:51.519 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mgr.x", 2024-09-19T08:43:51.519 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmbKuyAhAA0/XjHUIPOIV/YvNEhB8rEQ==", 2024-09-19T08:43:51.519 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:51.519 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-09-19T08:43:51.519 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile mgr", 2024-09-19T08:43:51.519 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:43:51.519 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:51.519 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:43:51.519 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:51.519 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mgr.y", 2024-09-19T08:43:51.519 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC+4utmU+ZrOxAAsvcwINH6qyxfN7N+rDrkqg==", 2024-09-19T08:43:51.519 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:43:51.519 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-09-19T08:43:51.519 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile mgr", 2024-09-19T08:43:51.520 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:43:51.520 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:51.520 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:51.520 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-19T08:43:51.520 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:43:51.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.504+0000 7f24aa888640 1 -- 172.21.15.98:0/4065494282 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f24900679e0 msgr2=0x7f2490069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:51.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.504+0000 7f24aa888640 1 --2- 172.21.15.98:0/4065494282 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f24900679e0 0x7f2490069ea0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2494009730 tx=0x7f2494006ca0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.504+0000 7f24aa888640 1 -- 172.21.15.98:0/4065494282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24a415c550 msgr2=0x7f24a41567f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:51.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.504+0000 7f24aa888640 1 --2- 172.21.15.98:0/4065494282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24a415c550 0x7f24a41567f0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f249c01a5e0 tx=0x7f249c019750 comp rx=0 tx=0).stop 2024-09-19T08:43:51.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.504+0000 7f24aa888640 1 -- 172.21.15.98:0/4065494282 shutdown_connections 2024-09-19T08:43:51.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.504+0000 7f24aa888640 1 --2- 172.21.15.98:0/4065494282 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f24900679e0 0x7f2490069ea0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.504+0000 7f24aa888640 1 --2- 172.21.15.98:0/4065494282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f24a415c550 0x7f24a41567f0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.504+0000 7f24aa888640 1 --2- 172.21.15.98:0/4065494282 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24a40cfad0 0x7f24a41575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.504+0000 7f24aa888640 1 --2- 172.21.15.98:0/4065494282 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24a406c1d0 0x7f24a4159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.504+0000 7f24aa888640 1 -- 172.21.15.98:0/4065494282 >> 172.21.15.98:0/4065494282 conn(0x7f24a40b2010 msgr2=0x7f24a40cf3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:51.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.504+0000 7f24aa888640 1 -- 172.21.15.98:0/4065494282 shutdown_connections 2024-09-19T08:43:51.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.504+0000 7f24aa888640 1 -- 172.21.15.98:0/4065494282 wait complete. 2024-09-19T08:43:51.523 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-19T08:43:51.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.632+0000 7fa0b6f60640 1 Processor -- start 2024-09-19T08:43:51.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.632+0000 7fa0b6f60640 1 -- start start 2024-09-19T08:43:51.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.636+0000 7fa0b6f60640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa0b006bc90 0x7fa0b006c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:51.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.636+0000 7fa0b6f60640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0b00ce840 0x7fa0b00cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:51.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.636+0000 7fa0b6f60640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0b00ccc60 con 0x7fa0b0158fd0 2024-09-19T08:43:51.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.636+0000 7fa0b6f60640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fa0b01593b0 con 0x7fa0b006bc90 2024-09-19T08:43:51.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.636+0000 7fa0b6f60640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa0b00ccf40 con 0x7fa0b00ce840 2024-09-19T08:43:51.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.636+0000 7fa0a7fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa0b006bc90 0x7fa0b006c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:51.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.636+0000 7fa0a7fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa0b006bc90 0x7fa0b006c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:32920/0 (socket says 172.21.15.98:32920) 2024-09-19T08:43:51.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.636+0000 7fa0a7fff640 1 -- 172.21.15.98:0/3412628638 learned_addr learned my addr 172.21.15.98:0/3412628638 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:51.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.636+0000 7fa0a7fff640 1 -- 172.21.15.98:0/3412628638 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0b00ce840 msgr2=0x7fa0b00cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:51.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.636+0000 7fa0b54fd640 1 --2- 172.21.15.98:0/3412628638 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0b00ce840 0x7fa0b00cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:51.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.636+0000 7fa0a7fff640 1 --2- 172.21.15.98:0/3412628638 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0b00ce840 0x7fa0b00cec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.636+0000 7fa0a7fff640 1 -- 172.21.15.98:0/3412628638 >> v1:172.21.15.139:6789/0 conn(0x7fa0b0158fd0 legacy=0x7fa0b00b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:51.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.636+0000 7fa0a7fff640 1 -- 172.21.15.98:0/3412628638 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b00cd220 con 0x7fa0b006bc90 2024-09-19T08:43:51.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.636+0000 7fa0a7fff640 1 --2- 172.21.15.98:0/3412628638 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0b006bc90 0x7fa0b006c120 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000bb10 tx=0x7fa0a001eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1234b940be0a241e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:51.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.636+0000 7fa0a77fe640 1 -- 172.21.15.98:0/3412628638 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a00b4070 con 0x7fa0b006bc90 2024-09-19T08:43:51.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.636+0000 7fa0a77fe640 1 -- 172.21.15.98:0/3412628638 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0a00a7ae0 con 0x7fa0b006bc90 2024-09-19T08:43:51.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.636+0000 7fa0a77fe640 1 -- 172.21.15.98:0/3412628638 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a00af440 con 0x7fa0b006bc90 2024-09-19T08:43:51.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.636+0000 7fa0b6f60640 1 -- 172.21.15.98:0/3412628638 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0b006bc90 msgr2=0x7fa0b006c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:51.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.636+0000 7fa0b6f60640 1 --2- 172.21.15.98:0/3412628638 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0b006bc90 0x7fa0b006c120 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000bb10 tx=0x7fa0a001eb60 comp rx=0 tx=0).stop 2024-09-19T08:43:51.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.636+0000 7fa0b6f60640 1 -- 172.21.15.98:0/3412628638 shutdown_connections 2024-09-19T08:43:51.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.636+0000 7fa0b6f60640 1 --2- 172.21.15.98:0/3412628638 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0b00ce840 0x7fa0b00cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.636+0000 7fa0b6f60640 1 --2- 172.21.15.98:0/3412628638 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0b006bc90 0x7fa0b006c120 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.636+0000 7fa0b6f60640 1 -- 172.21.15.98:0/3412628638 >> 172.21.15.98:0/3412628638 conn(0x7fa0b00b2010 msgr2=0x7fa0b00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:51.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.640+0000 7fa0b6f60640 1 -- 172.21.15.98:0/3412628638 shutdown_connections 2024-09-19T08:43:51.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.640+0000 7fa0b6f60640 1 -- 172.21.15.98:0/3412628638 wait complete. 2024-09-19T08:43:51.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.640+0000 7fa0b6f60640 1 Processor -- start 2024-09-19T08:43:51.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.640+0000 7fa0b6f60640 1 -- start start 2024-09-19T08:43:51.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.640+0000 7fa0b6f60640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa0b006bc90 0x7fa0b0155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:51.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.640+0000 7fa0b6f60640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0b00ce840 0x7fa0b0164280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:51.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.640+0000 7fa0b6f60640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0b0158fd0 0x7fa0b0165450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:51.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.640+0000 7fa0b6f60640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b015c920 con 0x7fa0b0158fd0 2024-09-19T08:43:51.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.640+0000 7fa0b6f60640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b015c7a0 con 0x7fa0b006bc90 2024-09-19T08:43:51.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.640+0000 7fa0b6f60640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa0b006d170 con 0x7fa0b00ce840 2024-09-19T08:43:51.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.640+0000 7fa0b54fd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0b0158fd0 0x7fa0b0165450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:51.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.640+0000 7fa0b54fd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0b0158fd0 0x7fa0b0165450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:32936/0 (socket says 172.21.15.98:32936) 2024-09-19T08:43:51.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.640+0000 7fa0b54fd640 1 -- 172.21.15.98:0/78977510 learned_addr learned my addr 172.21.15.98:0/78977510 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:51.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.640+0000 7fa0b54fd640 1 -- 172.21.15.98:0/78977510 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0b00ce840 msgr2=0x7fa0b0164280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:51.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.640+0000 7fa0b54fd640 1 --2- 172.21.15.98:0/78977510 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0b00ce840 0x7fa0b0164280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.640+0000 7fa0b54fd640 1 -- 172.21.15.98:0/78977510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa0b006bc90 msgr2=0x7fa0b0155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:51.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.640+0000 7fa0b54fd640 1 --2- 172.21.15.98:0/78977510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa0b006bc90 0x7fa0b0155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.640+0000 7fa0b54fd640 1 -- 172.21.15.98:0/78977510 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b01634c0 con 0x7fa0b0158fd0 2024-09-19T08:43:51.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.640+0000 7fa0b54fd640 1 --2- 172.21.15.98:0/78977510 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0b0158fd0 0x7fa0b0165450 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800cda0 tx=0x7fa0a800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:51.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.640+0000 7fa0a57fa640 1 -- 172.21.15.98:0/78977510 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8003ba0 con 0x7fa0b0158fd0 2024-09-19T08:43:51.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.640+0000 7fa0a57fa640 1 -- 172.21.15.98:0/78977510 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0a8003d40 con 0x7fa0b0158fd0 2024-09-19T08:43:51.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.640+0000 7fa0b6f60640 1 -- 172.21.15.98:0/78977510 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b01637a0 con 0x7fa0b0158fd0 2024-09-19T08:43:51.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.640+0000 7fa0a57fa640 1 -- 172.21.15.98:0/78977510 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8004240 con 0x7fa0b0158fd0 2024-09-19T08:43:51.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.640+0000 7fa0b6f60640 1 -- 172.21.15.98:0/78977510 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b0163d60 con 0x7fa0b0158fd0 2024-09-19T08:43:51.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.644+0000 7fa0b6f60640 1 -- 172.21.15.98:0/78977510 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0b006c120 con 0x7fa0b0158fd0 2024-09-19T08:43:51.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.644+0000 7fa0a57fa640 1 -- 172.21.15.98:0/78977510 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fa0a80043e0 con 0x7fa0b0158fd0 2024-09-19T08:43:51.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.644+0000 7fa0a57fa640 1 --2- 172.21.15.98:0/78977510 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa09c0679e0 0x7fa09c069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:51.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.644+0000 7fa0a57fa640 1 -- 172.21.15.98:0/78977510 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 4857+0+0 (secure 0 0 0) 0x7fa0a8086730 con 0x7fa0b0158fd0 2024-09-19T08:43:51.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.644+0000 7fa0b4cfc640 1 --2- 172.21.15.98:0/78977510 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa09c0679e0 0x7fa09c069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:51.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.644+0000 7fa0b4cfc640 1 --2- 172.21.15.98:0/78977510 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa09c0679e0 0x7fa09c069ea0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac009670 tx=0x7fa0ac006bd0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:51.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.648+0000 7fa0a57fa640 1 -- 172.21.15.98:0/78977510 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa0a80573a0 con 0x7fa0b0158fd0 2024-09-19T08:43:51.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.800+0000 7fa0b6f60640 1 -- 172.21.15.98:0/78977510 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa0b015fc50 con 0x7fa0b0158fd0 2024-09-19T08:43:51.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.800+0000 7fa0a57fa640 1 -- 172.21.15.98:0/78977510 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v98) ==== 84+0+65 (secure 0 0 0) 0x7fa0a8056ca0 con 0x7fa0b0158fd0 2024-09-19T08:43:51.801 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:51.801 INFO:teuthology.orchestra.run.smithi098.stdout:[ 2024-09-19T08:43:51.801 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:43:51.801 INFO:teuthology.orchestra.run.smithi098.stdout: "poolnum": 1, 2024-09-19T08:43:51.801 INFO:teuthology.orchestra.run.smithi098.stdout: "poolname": ".mgr" 2024-09-19T08:43:51.802 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:43:51.802 INFO:teuthology.orchestra.run.smithi098.stdout:] 2024-09-19T08:43:51.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.800+0000 7fa0b6f60640 1 -- 172.21.15.98:0/78977510 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa09c0679e0 msgr2=0x7fa09c069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:51.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.800+0000 7fa0b6f60640 1 --2- 172.21.15.98:0/78977510 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa09c0679e0 0x7fa09c069ea0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac009670 tx=0x7fa0ac006bd0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.800+0000 7fa0b6f60640 1 -- 172.21.15.98:0/78977510 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0b0158fd0 msgr2=0x7fa0b0165450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:51.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.800+0000 7fa0b6f60640 1 --2- 172.21.15.98:0/78977510 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0b0158fd0 0x7fa0b0165450 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800cda0 tx=0x7fa0a800a660 comp rx=0 tx=0).stop 2024-09-19T08:43:51.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.800+0000 7fa0b6f60640 1 -- 172.21.15.98:0/78977510 shutdown_connections 2024-09-19T08:43:51.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.800+0000 7fa0b6f60640 1 --2- 172.21.15.98:0/78977510 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa09c0679e0 0x7fa09c069ea0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.800+0000 7fa0b6f60640 1 --2- 172.21.15.98:0/78977510 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0b0158fd0 0x7fa0b0165450 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.800+0000 7fa0b6f60640 1 --2- 172.21.15.98:0/78977510 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0b00ce840 0x7fa0b0164280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.800+0000 7fa0b6f60640 1 --2- 172.21.15.98:0/78977510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa0b006bc90 0x7fa0b0155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.800+0000 7fa0b6f60640 1 -- 172.21.15.98:0/78977510 >> 172.21.15.98:0/78977510 conn(0x7fa0b00b2010 msgr2=0x7fa0b015b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:51.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.800+0000 7fa0b6f60640 1 -- 172.21.15.98:0/78977510 shutdown_connections 2024-09-19T08:43:51.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.800+0000 7fa0b6f60640 1 -- 172.21.15.98:0/78977510 wait complete. 2024-09-19T08:43:51.818 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T08:43:51.819 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T08:43:51.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.936+0000 7f0eb4581640 1 Processor -- start 2024-09-19T08:43:51.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.936+0000 7f0eb4581640 1 -- start start 2024-09-19T08:43:51.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.940+0000 7f0eb4581640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0eac15f6e0 0x7f0eac06d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:51.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.940+0000 7f0eb4581640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0eac15baf0 con 0x7f0eac073d90 2024-09-19T08:43:51.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.940+0000 7f0eb4581640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0eac15bdd0 con 0x7f0eac15fac0 2024-09-19T08:43:51.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.940+0000 7f0eb4581640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0eac15c8b0 con 0x7f0eac15f6e0 2024-09-19T08:43:51.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.940+0000 7f0eb2b1e640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f0eac073d90 0x7f0eac0ce420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:49356/0 (socket says 172.21.15.98:49356) 2024-09-19T08:43:51.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.940+0000 7f0eb2b1e640 1 -- 172.21.15.98:0/901682281 learned_addr learned my addr 172.21.15.98:0/901682281 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:51.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.940+0000 7f0eb131b640 1 -- 172.21.15.98:0/901682281 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2627657876 0 0) 0x7f0eac15baf0 con 0x7f0eac073d90 2024-09-19T08:43:51.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.940+0000 7f0eb131b640 1 -- 172.21.15.98:0/901682281 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e98003530 con 0x7f0eac073d90 2024-09-19T08:43:51.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.940+0000 7f0eb231d640 1 --2- 172.21.15.98:0/901682281 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0eac15f6e0 0x7f0eac06d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:51.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.940+0000 7f0eb131b640 1 -- 172.21.15.98:0/901682281 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3031015473 0 0) 0x7f0e98003530 con 0x7f0eac073d90 2024-09-19T08:43:51.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.940+0000 7f0eb131b640 1 -- 172.21.15.98:0/901682281 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0eac15f6e0 msgr2=0x7f0eac06d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:51.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.940+0000 7f0eb131b640 1 --2- 172.21.15.98:0/901682281 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0eac15f6e0 0x7f0eac06d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.940+0000 7f0eb131b640 1 -- 172.21.15.98:0/901682281 >> v1:172.21.15.139:6789/0 conn(0x7f0eac15fac0 legacy=0x7f0eac073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:51.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.940+0000 7f0eb131b640 1 -- 172.21.15.98:0/901682281 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eac15c0b0 con 0x7f0eac073d90 2024-09-19T08:43:51.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.940+0000 7f0eb131b640 1 -- 172.21.15.98:0/901682281 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f0ea4001a00 con 0x7f0eac073d90 2024-09-19T08:43:51.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.940+0000 7f0eb131b640 1 -- 172.21.15.98:0/901682281 >> v1:172.21.15.98:6789/0 conn(0x7f0eac073d90 legacy=0x7f0eac0ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:51.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.940+0000 7f0eb131b640 1 --2- 172.21.15.98:0/901682281 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e980039f0 0x7f0e98003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:51.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.940+0000 7f0eb131b640 1 --2- 172.21.15.98:0/901682281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e98004810 0x7f0e98004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:51.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.940+0000 7f0eb131b640 1 --2- 172.21.15.98:0/901682281 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0eac15f6e0 0x7f0e9800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:51.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.940+0000 7f0eb131b640 1 -- 172.21.15.98:0/901682281 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac15c8b0 con 0x7f0e98004810 2024-09-19T08:43:51.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.940+0000 7f0eb131b640 1 -- 172.21.15.98:0/901682281 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0e98005530 con 0x7f0e980039f0 2024-09-19T08:43:51.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.940+0000 7f0eb131b640 1 -- 172.21.15.98:0/901682281 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0e9800bf10 con 0x7f0eac15f6e0 2024-09-19T08:43:51.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.940+0000 7f0eb1b1c640 1 --2- 172.21.15.98:0/901682281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e98004810 0x7f0e98004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:51.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.940+0000 7f0eb231d640 1 --2- 172.21.15.98:0/901682281 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e980039f0 0x7f0e98003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:51.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.940+0000 7f0eb2b1e640 1 --2- 172.21.15.98:0/901682281 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0eac15f6e0 0x7f0e9800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:51.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.940+0000 7f0eb1b1c640 1 -- 172.21.15.98:0/901682281 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0eac15f6e0 msgr2=0x7f0e9800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:51.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.940+0000 7f0eb1b1c640 1 --2- 172.21.15.98:0/901682281 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0eac15f6e0 0x7f0e9800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.940+0000 7f0eb1b1c640 1 -- 172.21.15.98:0/901682281 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e980039f0 msgr2=0x7f0e98003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:51.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.940+0000 7f0eb1b1c640 1 --2- 172.21.15.98:0/901682281 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e980039f0 0x7f0e98003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.940+0000 7f0eb1b1c640 1 -- 172.21.15.98:0/901682281 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e98003530 con 0x7f0e98004810 2024-09-19T08:43:51.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.944+0000 7f0eb1b1c640 1 --2- 172.21.15.98:0/901682281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e98004810 0x7f0e98004c90 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c001750 tx=0x7f0e9c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f0082f73946124 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:51.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.944+0000 7f0eb131b640 1 -- 172.21.15.98:0/901682281 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c0013a0 con 0x7f0e98004810 2024-09-19T08:43:51.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.944+0000 7f0eb131b640 1 -- 172.21.15.98:0/901682281 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e9c0052f0 con 0x7f0e98004810 2024-09-19T08:43:51.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.944+0000 7f0eb131b640 1 -- 172.21.15.98:0/901682281 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c003840 con 0x7f0e98004810 2024-09-19T08:43:51.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.944+0000 7f0eb4581640 1 -- 172.21.15.98:0/901682281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e98004810 msgr2=0x7f0e98004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:51.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.944+0000 7f0eb4581640 1 --2- 172.21.15.98:0/901682281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e98004810 0x7f0e98004c90 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c001750 tx=0x7f0e9c01c3e0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.944+0000 7f0eb2b1e640 1 -- 172.21.15.98:0/901682281 reap_dead start 2024-09-19T08:43:51.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.944+0000 7f0eb4581640 1 -- 172.21.15.98:0/901682281 shutdown_connections 2024-09-19T08:43:51.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.944+0000 7f0eb4581640 1 -- 172.21.15.98:0/901682281 >> 172.21.15.98:0/901682281 conn(0x7f0eac06c090 msgr2=0x7f0eac06c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:51.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.944+0000 7f0eb4581640 1 -- 172.21.15.98:0/901682281 shutdown_connections 2024-09-19T08:43:51.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.944+0000 7f0eb4581640 1 -- 172.21.15.98:0/901682281 wait complete. 2024-09-19T08:43:51.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.944+0000 7f0eb4581640 1 Processor -- start 2024-09-19T08:43:51.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.944+0000 7f0eb4581640 1 -- start start 2024-09-19T08:43:51.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.944+0000 7f0eb4581640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e98004810 0x7f0eac151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:51.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.944+0000 7f0eb4581640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0eac152190 0x7f0eac155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:51.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.944+0000 7f0eb4581640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0eac154140 0x7f0eac15d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:51.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.944+0000 7f0eb4581640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac167060 con 0x7f0eac152190 2024-09-19T08:43:51.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.944+0000 7f0eb4581640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac0cf610 con 0x7f0e98004810 2024-09-19T08:43:51.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.944+0000 7f0eb4581640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0eac1671e0 con 0x7f0eac154140 2024-09-19T08:43:51.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.944+0000 7f0eb1b1c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0eac152190 0x7f0eac155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:51.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.944+0000 7f0eb1b1c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0eac152190 0x7f0eac155750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:32950/0 (socket says 172.21.15.98:32950) 2024-09-19T08:43:51.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.944+0000 7f0eb1b1c640 1 -- 172.21.15.98:0/1973377607 learned_addr learned my addr 172.21.15.98:0/1973377607 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:51.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.944+0000 7f0eb231d640 1 --2- 172.21.15.98:0/1973377607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e98004810 0x7f0eac151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:51.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.944+0000 7f0eb2b1e640 1 --2- 172.21.15.98:0/1973377607 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0eac154140 0x7f0eac15d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:51.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.944+0000 7f0eb1b1c640 1 -- 172.21.15.98:0/1973377607 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0eac154140 msgr2=0x7f0eac15d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:51.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.944+0000 7f0eb1b1c640 1 --2- 172.21.15.98:0/1973377607 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0eac154140 0x7f0eac15d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.944+0000 7f0eb1b1c640 1 -- 172.21.15.98:0/1973377607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e98004810 msgr2=0x7f0eac151510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:51.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.944+0000 7f0eb1b1c640 1 --2- 172.21.15.98:0/1973377607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e98004810 0x7f0eac151510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:51.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.944+0000 7f0eb1b1c640 1 -- 172.21.15.98:0/1973377607 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eac154640 con 0x7f0eac152190 2024-09-19T08:43:51.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.944+0000 7f0eb1b1c640 1 --2- 172.21.15.98:0/1973377607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0eac152190 0x7f0eac155750 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c005040 tx=0x7f0e98005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:51.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.944+0000 7f0ea2ffd640 1 -- 172.21.15.98:0/1973377607 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c007a40 con 0x7f0eac152190 2024-09-19T08:43:51.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.944+0000 7f0ea2ffd640 1 -- 172.21.15.98:0/1973377607 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e9c0bb070 con 0x7f0eac152190 2024-09-19T08:43:51.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.944+0000 7f0eb4581640 1 -- 172.21.15.98:0/1973377607 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eac154c70 con 0x7f0eac152190 2024-09-19T08:43:51.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.944+0000 7f0ea2ffd640 1 -- 172.21.15.98:0/1973377607 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c0013a0 con 0x7f0eac152190 2024-09-19T08:43:51.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.944+0000 7f0eb4581640 1 -- 172.21.15.98:0/1973377607 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eac155230 con 0x7f0eac152190 2024-09-19T08:43:51.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.948+0000 7f0eb4581640 1 -- 172.21.15.98:0/1973377607 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0eac0ce420 con 0x7f0eac152190 2024-09-19T08:43:51.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.948+0000 7f0ea2ffd640 1 -- 172.21.15.98:0/1973377607 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f0e9c0b4040 con 0x7f0eac152190 2024-09-19T08:43:51.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.948+0000 7f0ea2ffd640 1 --2- 172.21.15.98:0/1973377607 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0e900679e0 0x7f0e90069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:51.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.948+0000 7f0ea2ffd640 1 -- 172.21.15.98:0/1973377607 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 4857+0+0 (secure 0 0 0) 0x7f0e9c11ff50 con 0x7f0eac152190 2024-09-19T08:43:51.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.948+0000 7f0eb231d640 1 --2- 172.21.15.98:0/1973377607 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0e900679e0 0x7f0e90069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:51.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.948+0000 7f0eb231d640 1 --2- 172.21.15.98:0/1973377607 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0e900679e0 0x7f0e90069ea0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800b9b0 tx=0x7f0ea8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:51.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:51.952+0000 7f0ea2ffd640 1 -- 172.21.15.98:0/1973377607 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f0e9c0160f0 con 0x7f0eac152190 2024-09-19T08:43:52.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:52.116+0000 7f0eb4581640 1 -- 172.21.15.98:0/1973377607 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f0eac075880 con 0x7f0eac152190 2024-09-19T08:43:53.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.124+0000 7f0ea2ffd640 1 -- 172.21.15.98:0/1973377607 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v99) ==== 138+0+0 (secure 0 0 0) 0x7f0e9c0f0ce0 con 0x7f0eac152190 2024-09-19T08:43:53.127 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_metadata' created 2024-09-19T08:43:53.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.128+0000 7f0eb4581640 1 -- 172.21.15.98:0/1973377607 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0e900679e0 msgr2=0x7f0e90069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:53.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.128+0000 7f0eb4581640 1 --2- 172.21.15.98:0/1973377607 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0e900679e0 0x7f0e90069ea0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800b9b0 tx=0x7f0ea8008040 comp rx=0 tx=0).stop 2024-09-19T08:43:53.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.128+0000 7f0eb4581640 1 -- 172.21.15.98:0/1973377607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0eac152190 msgr2=0x7f0eac155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:53.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.128+0000 7f0eb4581640 1 --2- 172.21.15.98:0/1973377607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0eac152190 0x7f0eac155750 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c005040 tx=0x7f0e98005840 comp rx=0 tx=0).stop 2024-09-19T08:43:53.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.128+0000 7f0eb4581640 1 -- 172.21.15.98:0/1973377607 shutdown_connections 2024-09-19T08:43:53.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.128+0000 7f0eb4581640 1 --2- 172.21.15.98:0/1973377607 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0e900679e0 0x7f0e90069ea0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:53.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.128+0000 7f0eb4581640 1 --2- 172.21.15.98:0/1973377607 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0eac154140 0x7f0eac15d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:53.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.128+0000 7f0eb4581640 1 --2- 172.21.15.98:0/1973377607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0eac152190 0x7f0eac155750 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:53.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.128+0000 7f0eb4581640 1 --2- 172.21.15.98:0/1973377607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e98004810 0x7f0eac151510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:53.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.128+0000 7f0eb4581640 1 -- 172.21.15.98:0/1973377607 >> 172.21.15.98:0/1973377607 conn(0x7f0eac06c090 msgr2=0x7f0eac15ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:53.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.128+0000 7f0eb4581640 1 -- 172.21.15.98:0/1973377607 shutdown_connections 2024-09-19T08:43:53.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.128+0000 7f0eb4581640 1 -- 172.21.15.98:0/1973377607 wait complete. 2024-09-19T08:43:53.153 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T08:43:53.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.276+0000 7f0799b7f640 1 Processor -- start 2024-09-19T08:43:53.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.276+0000 7f0799b7f640 1 -- start start 2024-09-19T08:43:53.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.276+0000 7f0799b7f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f07940b4420 0x7f0794071ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:53.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.276+0000 7f0799b7f640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0794073530 con 0x7f0794145b70 2024-09-19T08:43:53.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.276+0000 7f0799b7f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0794073810 con 0x7f079406c3a0 2024-09-19T08:43:53.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.276+0000 7f0799b7f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f0794073af0 con 0x7f07940b4420 2024-09-19T08:43:53.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.276+0000 7f07937fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f0794145b70 0x7f079406bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:49372/0 (socket says 172.21.15.98:49372) 2024-09-19T08:43:53.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.276+0000 7f07937fe640 1 -- 172.21.15.98:0/3646657828 learned_addr learned my addr 172.21.15.98:0/3646657828 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:53.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.276+0000 7f0793fff640 1 --2- 172.21.15.98:0/3646657828 >> v2:172.21.15.139:3300/0 conn(0x7f07940b4420 0x7f0794071ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:53.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.276+0000 7f07927fc640 1 -- 172.21.15.98:0/3646657828 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2520274022 0 0) 0x7f0794073530 con 0x7f0794145b70 2024-09-19T08:43:53.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.276+0000 7f07927fc640 1 -- 172.21.15.98:0/3646657828 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0780003530 con 0x7f0794145b70 2024-09-19T08:43:53.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.280+0000 7f07927fc640 1 -- 172.21.15.98:0/3646657828 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2995656001 0 0) 0x7f0780003530 con 0x7f0794145b70 2024-09-19T08:43:53.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.280+0000 7f07927fc640 1 -- 172.21.15.98:0/3646657828 >> v2:172.21.15.139:3300/0 conn(0x7f07940b4420 msgr2=0x7f0794071ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:53.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.280+0000 7f07927fc640 1 --2- 172.21.15.98:0/3646657828 >> v2:172.21.15.139:3300/0 conn(0x7f07940b4420 0x7f0794071ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:53.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.280+0000 7f07927fc640 1 -- 172.21.15.98:0/3646657828 >> v1:172.21.15.139:6789/0 conn(0x7f079406c3a0 legacy=0x7f07940713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:53.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.280+0000 7f07927fc640 1 -- 172.21.15.98:0/3646657828 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0794073dd0 con 0x7f0794145b70 2024-09-19T08:43:53.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.280+0000 7f07927fc640 1 -- 172.21.15.98:0/3646657828 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f0784001680 con 0x7f0794145b70 2024-09-19T08:43:53.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.280+0000 7f07927fc640 1 -- 172.21.15.98:0/3646657828 >> v1:172.21.15.98:6789/0 conn(0x7f0794145b70 legacy=0x7f079406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:53.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.280+0000 7f07927fc640 1 --2- 172.21.15.98:0/3646657828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07800039c0 0x7f0780003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:53.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.280+0000 7f07927fc640 1 --2- 172.21.15.98:0/3646657828 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0780004810 0x7f0780004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:53.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.280+0000 7f07927fc640 1 --2- 172.21.15.98:0/3646657828 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07800051d0 0x7f078000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:53.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.280+0000 7f07927fc640 1 -- 172.21.15.98:0/3646657828 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0794073af0 con 0x7f07800039c0 2024-09-19T08:43:53.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.280+0000 7f07927fc640 1 -- 172.21.15.98:0/3646657828 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f078000c170 con 0x7f0780004810 2024-09-19T08:43:53.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.280+0000 7f07927fc640 1 -- 172.21.15.98:0/3646657828 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f078000c370 con 0x7f07800051d0 2024-09-19T08:43:53.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.280+0000 7f07937fe640 1 --2- 172.21.15.98:0/3646657828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07800039c0 0x7f0780003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:53.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.280+0000 7f0792ffd640 1 --2- 172.21.15.98:0/3646657828 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0780004810 0x7f0780004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:53.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.280+0000 7f0793fff640 1 --2- 172.21.15.98:0/3646657828 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07800051d0 0x7f078000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:53.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.280+0000 7f07937fe640 1 -- 172.21.15.98:0/3646657828 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07800051d0 msgr2=0x7f078000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:53.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.280+0000 7f07937fe640 1 --2- 172.21.15.98:0/3646657828 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07800051d0 0x7f078000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:53.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.280+0000 7f07937fe640 1 -- 172.21.15.98:0/3646657828 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0780004810 msgr2=0x7f0780004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:53.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.280+0000 7f07937fe640 1 --2- 172.21.15.98:0/3646657828 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0780004810 0x7f0780004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:53.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.280+0000 7f07937fe640 1 -- 172.21.15.98:0/3646657828 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0780003530 con 0x7f07800039c0 2024-09-19T08:43:53.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.280+0000 7f0793fff640 1 --2- 172.21.15.98:0/3646657828 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07800051d0 0x7f078000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:53.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.280+0000 7f0793fff640 1 -- 172.21.15.98:0/3646657828 reap_dead start 2024-09-19T08:43:53.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.280+0000 7f07937fe640 1 --2- 172.21.15.98:0/3646657828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07800039c0 0x7f0780003de0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f078400c240 tx=0x7f078401cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c7c249c629f5bab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:53.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.280+0000 7f07927fc640 1 -- 172.21.15.98:0/3646657828 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0784007d20 con 0x7f07800039c0 2024-09-19T08:43:53.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.280+0000 7f07927fc640 1 -- 172.21.15.98:0/3646657828 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07840a9070 con 0x7f07800039c0 2024-09-19T08:43:53.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.280+0000 7f07927fc640 1 -- 172.21.15.98:0/3646657828 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0784005da0 con 0x7f07800039c0 2024-09-19T08:43:53.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.280+0000 7f0799b7f640 1 -- 172.21.15.98:0/3646657828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07800039c0 msgr2=0x7f0780003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:53.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.280+0000 7f0799b7f640 1 --2- 172.21.15.98:0/3646657828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07800039c0 0x7f0780003de0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f078400c240 tx=0x7f078401cbc0 comp rx=0 tx=0).stop 2024-09-19T08:43:53.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.280+0000 7f0799b7f640 1 -- 172.21.15.98:0/3646657828 shutdown_connections 2024-09-19T08:43:53.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.280+0000 7f0799b7f640 1 --2- 172.21.15.98:0/3646657828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07800039c0 0x7f0780003de0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:53.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.280+0000 7f0799b7f640 1 -- 172.21.15.98:0/3646657828 >> 172.21.15.98:0/3646657828 conn(0x7f07940b2010 msgr2=0x7f07940b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:53.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.280+0000 7f0799b7f640 1 -- 172.21.15.98:0/3646657828 shutdown_connections 2024-09-19T08:43:53.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.280+0000 7f0799b7f640 1 -- 172.21.15.98:0/3646657828 wait complete. 2024-09-19T08:43:53.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.284+0000 7f0799b7f640 1 Processor -- start 2024-09-19T08:43:53.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.284+0000 7f0799b7f640 1 -- start start 2024-09-19T08:43:53.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.284+0000 7f0799b7f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07800039c0 0x7f07941f8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:53.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.284+0000 7f0799b7f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f079413f030 0x7f079413f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:53.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.284+0000 7f0799b7f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f079413f4b0 0x7f079413ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:53.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.284+0000 7f0799b7f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f07940b3410 con 0x7f079413f030 2024-09-19T08:43:53.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.284+0000 7f0799b7f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0794071dd0 con 0x7f079413f4b0 2024-09-19T08:43:53.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.284+0000 7f0799b7f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f07940b3590 con 0x7f07800039c0 2024-09-19T08:43:53.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.284+0000 7f0792ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f079413f030 0x7f079413f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:53.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.284+0000 7f0792ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f079413f030 0x7f079413f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:32980/0 (socket says 172.21.15.98:32980) 2024-09-19T08:43:53.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.284+0000 7f0792ffd640 1 -- 172.21.15.98:0/1657305068 learned_addr learned my addr 172.21.15.98:0/1657305068 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:53.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.284+0000 7f07937fe640 1 --2- 172.21.15.98:0/1657305068 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07800039c0 0x7f07941f8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:53.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.284+0000 7f0793fff640 1 --2- 172.21.15.98:0/1657305068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f079413f4b0 0x7f079413ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:53.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.284+0000 7f0792ffd640 1 -- 172.21.15.98:0/1657305068 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07800039c0 msgr2=0x7f07941f8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:53.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.284+0000 7f0792ffd640 1 --2- 172.21.15.98:0/1657305068 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07800039c0 0x7f07941f8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:53.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.284+0000 7f0792ffd640 1 -- 172.21.15.98:0/1657305068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f079413f4b0 msgr2=0x7f079413ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:53.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.284+0000 7f0792ffd640 1 --2- 172.21.15.98:0/1657305068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f079413f4b0 0x7f079413ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:53.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.284+0000 7f0792ffd640 1 -- 172.21.15.98:0/1657305068 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f079415f4c0 con 0x7f079413f030 2024-09-19T08:43:53.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.284+0000 7f07937fe640 1 --2- 172.21.15.98:0/1657305068 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07800039c0 0x7f07941f8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:43:53.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.284+0000 7f0793fff640 1 --2- 172.21.15.98:0/1657305068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f079413f4b0 0x7f079413ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:43:53.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.284+0000 7f0792ffd640 1 --2- 172.21.15.98:0/1657305068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f079413f030 0x7f079413f9d0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f077c00e4b0 tx=0x7f077c00e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:53.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.284+0000 7f078bfff640 1 -- 172.21.15.98:0/1657305068 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077c0052e0 con 0x7f079413f030 2024-09-19T08:43:53.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.284+0000 7f078bfff640 1 -- 172.21.15.98:0/1657305068 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f077c018070 con 0x7f079413f030 2024-09-19T08:43:53.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.284+0000 7f078bfff640 1 -- 172.21.15.98:0/1657305068 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077c005480 con 0x7f079413f030 2024-09-19T08:43:53.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.284+0000 7f0799b7f640 1 -- 172.21.15.98:0/1657305068 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f079413e7e0 con 0x7f079413f030 2024-09-19T08:43:53.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.284+0000 7f0799b7f640 1 -- 172.21.15.98:0/1657305068 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f079415fa20 con 0x7f079413f030 2024-09-19T08:43:53.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.284+0000 7f0799b7f640 1 -- 172.21.15.98:0/1657305068 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07940b3920 con 0x7f079413f030 2024-09-19T08:43:53.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.288+0000 7f078bfff640 1 -- 172.21.15.98:0/1657305068 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f077c003840 con 0x7f079413f030 2024-09-19T08:43:53.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.288+0000 7f078bfff640 1 --2- 172.21.15.98:0/1657305068 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0774067a10 0x7f0774069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:53.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.288+0000 7f078bfff640 1 -- 172.21.15.98:0/1657305068 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5227+0+0 (secure 0 0 0) 0x7f077c085820 con 0x7f079413f030 2024-09-19T08:43:53.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.288+0000 7f07937fe640 1 --2- 172.21.15.98:0/1657305068 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0774067a10 0x7f0774069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:53.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.288+0000 7f07937fe640 1 --2- 172.21.15.98:0/1657305068 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0774067a10 0x7f0774069ed0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f07840ab000 tx=0x7f07840b0040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:53.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.292+0000 7f078bfff640 1 -- 172.21.15.98:0/1657305068 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f079415f4c0 con 0x7f079413f030 2024-09-19T08:43:53.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:53.452+0000 7f0799b7f640 1 -- 172.21.15.98:0/1657305068 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f07940b5080 con 0x7f079413f030 2024-09-19T08:43:54.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.128+0000 7f078bfff640 1 -- 172.21.15.98:0/1657305068 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v100) ==== 170+0+0 (secure 0 0 0) 0x7f077c0564c0 con 0x7f079413f030 2024-09-19T08:43:54.131 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' created 2024-09-19T08:43:54.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.132+0000 7f0799b7f640 1 -- 172.21.15.98:0/1657305068 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0774067a10 msgr2=0x7f0774069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:54.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.132+0000 7f0799b7f640 1 --2- 172.21.15.98:0/1657305068 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0774067a10 0x7f0774069ed0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f07840ab000 tx=0x7f07840b0040 comp rx=0 tx=0).stop 2024-09-19T08:43:54.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.132+0000 7f0799b7f640 1 -- 172.21.15.98:0/1657305068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f079413f030 msgr2=0x7f079413f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:54.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.132+0000 7f0799b7f640 1 --2- 172.21.15.98:0/1657305068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f079413f030 0x7f079413f9d0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f077c00e4b0 tx=0x7f077c00e970 comp rx=0 tx=0).stop 2024-09-19T08:43:54.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.132+0000 7f0799b7f640 1 -- 172.21.15.98:0/1657305068 shutdown_connections 2024-09-19T08:43:54.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.132+0000 7f0799b7f640 1 --2- 172.21.15.98:0/1657305068 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0774067a10 0x7f0774069ed0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:54.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.132+0000 7f0799b7f640 1 --2- 172.21.15.98:0/1657305068 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f079413f4b0 0x7f079413ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:54.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.132+0000 7f0799b7f640 1 --2- 172.21.15.98:0/1657305068 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f079413f030 0x7f079413f9d0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:54.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.132+0000 7f0799b7f640 1 --2- 172.21.15.98:0/1657305068 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07800039c0 0x7f07941f8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:54.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.132+0000 7f0799b7f640 1 -- 172.21.15.98:0/1657305068 >> 172.21.15.98:0/1657305068 conn(0x7f07940b2010 msgr2=0x7f0794142ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:54.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.132+0000 7f0799b7f640 1 -- 172.21.15.98:0/1657305068 shutdown_connections 2024-09-19T08:43:54.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.132+0000 7f0799b7f640 1 -- 172.21.15.98:0/1657305068 wait complete. 2024-09-19T08:43:54.175 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T08:43:54.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.288+0000 7f0543536640 1 Processor -- start 2024-09-19T08:43:54.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.288+0000 7f0543536640 1 -- start start 2024-09-19T08:43:54.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.288+0000 7f0543536640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f053c144d10 0x7f053c0722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:54.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.288+0000 7f0543536640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f053c072820 0x7f053c06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:54.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.288+0000 7f0543536640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f053c072ea0 con 0x7f053c06c1d0 2024-09-19T08:43:54.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.288+0000 7f0543536640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f053c06c640 con 0x7f053c072820 2024-09-19T08:43:54.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.288+0000 7f0543536640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f053c072c60 con 0x7f053c144d10 2024-09-19T08:43:54.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.288+0000 7f0540ad1640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f053c072820 0x7f053c06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:54.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.288+0000 7f0541ad3640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f053c06c1d0 0x7f053c0b4520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:49386/0 (socket says 172.21.15.98:49386) 2024-09-19T08:43:54.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.288+0000 7f0541ad3640 1 -- 172.21.15.98:0/1534023107 learned_addr learned my addr 172.21.15.98:0/1534023107 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:54.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.288+0000 7f0540ad1640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f053c072820 0x7f053c06bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:32996/0 (socket says 172.21.15.98:32996) 2024-09-19T08:43:54.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.288+0000 7f05412d2640 1 --2- 172.21.15.98:0/1534023107 >> v2:172.21.15.139:3300/0 conn(0x7f053c144d10 0x7f053c0722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:54.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.288+0000 7f0540ad1640 1 -- 172.21.15.98:0/1534023107 >> v2:172.21.15.139:3300/0 conn(0x7f053c144d10 msgr2=0x7f053c0722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:54.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.288+0000 7f0540ad1640 1 --2- 172.21.15.98:0/1534023107 >> v2:172.21.15.139:3300/0 conn(0x7f053c144d10 0x7f053c0722e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:54.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.288+0000 7f0540ad1640 1 -- 172.21.15.98:0/1534023107 >> v1:172.21.15.98:6789/0 conn(0x7f053c06c1d0 legacy=0x7f053c0b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:54.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.288+0000 7f0533fff640 1 -- 172.21.15.98:0/1534023107 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1428039215 0 0) 0x7f053c072ea0 con 0x7f053c06c1d0 2024-09-19T08:43:54.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.288+0000 7f0540ad1640 1 -- 172.21.15.98:0/1534023107 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f053c073180 con 0x7f053c072820 2024-09-19T08:43:54.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.292+0000 7f0540ad1640 1 --2- 172.21.15.98:0/1534023107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f053c072820 0x7f053c06bc90 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f053c0684a0 tx=0x7f053801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74af80d589aa3e26 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:54.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.292+0000 7f0533fff640 1 -- 172.21.15.98:0/1534023107 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05380b0070 con 0x7f053c072820 2024-09-19T08:43:54.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.292+0000 7f0533fff640 1 -- 172.21.15.98:0/1534023107 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05380ab600 con 0x7f053c072820 2024-09-19T08:43:54.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.292+0000 7f0533fff640 1 -- 172.21.15.98:0/1534023107 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05380ab9d0 con 0x7f053c072820 2024-09-19T08:43:54.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.292+0000 7f0543536640 1 -- 172.21.15.98:0/1534023107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f053c072820 msgr2=0x7f053c06bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:54.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.292+0000 7f0543536640 1 --2- 172.21.15.98:0/1534023107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f053c072820 0x7f053c06bc90 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f053c0684a0 tx=0x7f053801c480 comp rx=0 tx=0).stop 2024-09-19T08:43:54.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.292+0000 7f0543536640 1 -- 172.21.15.98:0/1534023107 shutdown_connections 2024-09-19T08:43:54.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.292+0000 7f0543536640 1 --2- 172.21.15.98:0/1534023107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f053c072820 0x7f053c06bc90 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:54.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.292+0000 7f0543536640 1 --2- 172.21.15.98:0/1534023107 >> v2:172.21.15.139:3300/0 conn(0x7f053c144d10 0x7f053c0722e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:54.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.292+0000 7f0543536640 1 -- 172.21.15.98:0/1534023107 >> 172.21.15.98:0/1534023107 conn(0x7f053c0b2010 msgr2=0x7f053c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:54.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.292+0000 7f0543536640 1 -- 172.21.15.98:0/1534023107 shutdown_connections 2024-09-19T08:43:54.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.292+0000 7f0543536640 1 -- 172.21.15.98:0/1534023107 wait complete. 2024-09-19T08:43:54.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.292+0000 7f0543536640 1 Processor -- start 2024-09-19T08:43:54.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.292+0000 7f0543536640 1 -- start start 2024-09-19T08:43:54.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.292+0000 7f0543536640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f053c06c1d0 0x7f053c140780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:54.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.292+0000 7f0543536640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f053c072820 0x7f053c158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:54.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.292+0000 7f0543536640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f053c144d10 0x7f053c1382e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:54.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.292+0000 7f0543536640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f053c1680e0 con 0x7f053c06c1d0 2024-09-19T08:43:54.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.292+0000 7f0543536640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f053c167f60 con 0x7f053c072820 2024-09-19T08:43:54.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.292+0000 7f0543536640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f053c168260 con 0x7f053c144d10 2024-09-19T08:43:54.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.292+0000 7f05412d2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f053c06c1d0 0x7f053c140780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:54.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.292+0000 7f05412d2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f053c06c1d0 0x7f053c140780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33004/0 (socket says 172.21.15.98:33004) 2024-09-19T08:43:54.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.292+0000 7f05412d2640 1 -- 172.21.15.98:0/2569446462 learned_addr learned my addr 172.21.15.98:0/2569446462 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:54.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.292+0000 7f05412d2640 1 -- 172.21.15.98:0/2569446462 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f053c144d10 msgr2=0x7f053c1382e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T08:43:54.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.292+0000 7f05412d2640 1 --2- 172.21.15.98:0/2569446462 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f053c144d10 0x7f053c1382e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:54.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.292+0000 7f0540ad1640 1 --2- 172.21.15.98:0/2569446462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f053c072820 0x7f053c158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:54.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.292+0000 7f05412d2640 1 -- 172.21.15.98:0/2569446462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f053c072820 msgr2=0x7f053c158500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:54.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.292+0000 7f05412d2640 1 --2- 172.21.15.98:0/2569446462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f053c072820 0x7f053c158500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:54.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.292+0000 7f05412d2640 1 -- 172.21.15.98:0/2569446462 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f053c13b650 con 0x7f053c06c1d0 2024-09-19T08:43:54.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.292+0000 7f05412d2640 1 --2- 172.21.15.98:0/2569446462 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f053c06c1d0 0x7f053c140780 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f052c00c7f0 tx=0x7f052c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:54.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.292+0000 7f0531ffb640 1 -- 172.21.15.98:0/2569446462 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c015070 con 0x7f053c06c1d0 2024-09-19T08:43:54.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.292+0000 7f0531ffb640 1 -- 172.21.15.98:0/2569446462 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f052c0077c0 con 0x7f053c06c1d0 2024-09-19T08:43:54.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.292+0000 7f0543536640 1 -- 172.21.15.98:0/2569446462 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f053c13a5b0 con 0x7f053c06c1d0 2024-09-19T08:43:54.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.292+0000 7f0531ffb640 1 -- 172.21.15.98:0/2569446462 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c007bb0 con 0x7f053c06c1d0 2024-09-19T08:43:54.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.292+0000 7f0543536640 1 -- 172.21.15.98:0/2569446462 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f053c13ab70 con 0x7f053c06c1d0 2024-09-19T08:43:54.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.296+0000 7f0543536640 1 -- 172.21.15.98:0/2569446462 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f053c06bc90 con 0x7f053c06c1d0 2024-09-19T08:43:54.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.296+0000 7f0531ffb640 1 -- 172.21.15.98:0/2569446462 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f052c018070 con 0x7f053c06c1d0 2024-09-19T08:43:54.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.296+0000 7f0531ffb640 1 --2- 172.21.15.98:0/2569446462 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f05200679e0 0x7f0520069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:54.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.296+0000 7f0531ffb640 1 -- 172.21.15.98:0/2569446462 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5609+0+0 (secure 0 0 0) 0x7f052c085d30 con 0x7f053c06c1d0 2024-09-19T08:43:54.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.296+0000 7f0540ad1640 1 --2- 172.21.15.98:0/2569446462 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f05200679e0 0x7f0520069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:54.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.296+0000 7f0540ad1640 1 --2- 172.21.15.98:0/2569446462 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f05200679e0 0x7f0520069ea0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f05380ad000 tx=0x7f0538008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:54.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.300+0000 7f0531ffb640 1 -- 172.21.15.98:0/2569446462 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f052c0566b0 con 0x7f053c06c1d0 2024-09-19T08:43:54.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:54.484+0000 7f0543536640 1 -- 172.21.15.98:0/2569446462 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f053c1415b0 con 0x7f053c06c1d0 2024-09-19T08:43:55.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.144+0000 7f0531ffb640 1 -- 172.21.15.98:0/2569446462 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v57) ==== 174+0+0 (secure 0 0 0) 0x7f052c055fb0 con 0x7f053c06c1d0 2024-09-19T08:43:55.147 INFO:teuthology.orchestra.run.smithi098.stderr:new fs with metadata pool 16 and data pool 17 2024-09-19T08:43:55.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.148+0000 7f0543536640 1 -- 172.21.15.98:0/2569446462 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f05200679e0 msgr2=0x7f0520069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:55.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.148+0000 7f0543536640 1 --2- 172.21.15.98:0/2569446462 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f05200679e0 0x7f0520069ea0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f05380ad000 tx=0x7f0538008040 comp rx=0 tx=0).stop 2024-09-19T08:43:55.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.148+0000 7f0543536640 1 -- 172.21.15.98:0/2569446462 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f053c06c1d0 msgr2=0x7f053c140780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:55.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.148+0000 7f0543536640 1 --2- 172.21.15.98:0/2569446462 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f053c06c1d0 0x7f053c140780 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f052c00c7f0 tx=0x7f052c00ccb0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.148+0000 7f0543536640 1 -- 172.21.15.98:0/2569446462 shutdown_connections 2024-09-19T08:43:55.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.148+0000 7f0543536640 1 --2- 172.21.15.98:0/2569446462 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f05200679e0 0x7f0520069ea0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.148+0000 7f0543536640 1 --2- 172.21.15.98:0/2569446462 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f053c144d10 0x7f053c1382e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.148+0000 7f0543536640 1 --2- 172.21.15.98:0/2569446462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f053c072820 0x7f053c158500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.148+0000 7f0543536640 1 --2- 172.21.15.98:0/2569446462 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f053c06c1d0 0x7f053c140780 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.152 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.148+0000 7f0543536640 1 -- 172.21.15.98:0/2569446462 >> 172.21.15.98:0/2569446462 conn(0x7f053c0b2010 msgr2=0x7f053c0ce700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:55.152 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.148+0000 7f0543536640 1 -- 172.21.15.98:0/2569446462 shutdown_connections 2024-09-19T08:43:55.152 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.148+0000 7f0543536640 1 -- 172.21.15.98:0/2569446462 wait complete. 2024-09-19T08:43:55.187 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:43:55.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.304+0000 7f4930f1e640 1 Processor -- start 2024-09-19T08:43:55.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.304+0000 7f4930f1e640 1 -- start start 2024-09-19T08:43:55.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.308+0000 7f4930f1e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f492c06c3a0 0x7f492c0cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:55.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.308+0000 7f4930f1e640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f492c06d310 con 0x7f492c0cf900 2024-09-19T08:43:55.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.308+0000 7f4930f1e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f492c06d5f0 con 0x7f492c15c550 2024-09-19T08:43:55.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.308+0000 7f4930f1e640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f492c06d8d0 con 0x7f492c06c3a0 2024-09-19T08:43:55.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.308+0000 7f4929d9b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f492c06c3a0 0x7f492c0cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:55.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.308+0000 7f492ad9d640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f492c0cf900 0x7f492c0723a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:49388/0 (socket says 172.21.15.98:49388) 2024-09-19T08:43:55.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.308+0000 7f492ad9d640 1 -- 172.21.15.98:0/61795652 learned_addr learned my addr 172.21.15.98:0/61795652 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:55.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.308+0000 7f4929d9b640 1 -- 172.21.15.98:0/61795652 >> v1:172.21.15.139:6789/0 conn(0x7f492c15c550 legacy=0x7f492c06bc90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:43:55.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.308+0000 7f4929d9b640 1 -- 172.21.15.98:0/61795652 >> v1:172.21.15.98:6789/0 conn(0x7f492c0cf900 legacy=0x7f492c0723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:55.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.308+0000 7f4929d9b640 1 -- 172.21.15.98:0/61795652 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f492c06dbb0 con 0x7f492c06c3a0 2024-09-19T08:43:55.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.308+0000 7f4929d9b640 1 --2- 172.21.15.98:0/61795652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f492c06c3a0 0x7f492c0cf3c0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f49140097c0 tx=0x7f491401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d8baf8245324c3f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:55.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.308+0000 7f492959a640 1 -- 172.21.15.98:0/61795652 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49140b0070 con 0x7f492c06c3a0 2024-09-19T08:43:55.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.308+0000 7f492959a640 1 -- 172.21.15.98:0/61795652 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49140ab600 con 0x7f492c06c3a0 2024-09-19T08:43:55.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.308+0000 7f492959a640 1 -- 172.21.15.98:0/61795652 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49140ab9d0 con 0x7f492c06c3a0 2024-09-19T08:43:55.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.312+0000 7f4930f1e640 1 -- 172.21.15.98:0/61795652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f492c06c3a0 msgr2=0x7f492c0cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:55.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.312+0000 7f4930f1e640 1 --2- 172.21.15.98:0/61795652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f492c06c3a0 0x7f492c0cf3c0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f49140097c0 tx=0x7f491401c460 comp rx=0 tx=0).stop 2024-09-19T08:43:55.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.312+0000 7f4930f1e640 1 -- 172.21.15.98:0/61795652 shutdown_connections 2024-09-19T08:43:55.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.312+0000 7f4930f1e640 1 --2- 172.21.15.98:0/61795652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f492c06c3a0 0x7f492c0cf3c0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.312+0000 7f4930f1e640 1 -- 172.21.15.98:0/61795652 >> 172.21.15.98:0/61795652 conn(0x7f492c0b2010 msgr2=0x7f492c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:55.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.312+0000 7f4930f1e640 1 -- 172.21.15.98:0/61795652 shutdown_connections 2024-09-19T08:43:55.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.312+0000 7f4930f1e640 1 -- 172.21.15.98:0/61795652 wait complete. 2024-09-19T08:43:55.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.312+0000 7f4930f1e640 1 Processor -- start 2024-09-19T08:43:55.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.312+0000 7f4930f1e640 1 -- start start 2024-09-19T08:43:55.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.312+0000 7f4930f1e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f492c06c3a0 0x7f492c14bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:55.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.312+0000 7f4930f1e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f492c0cf900 0x7f492c158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:55.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.312+0000 7f4930f1e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f492c15c550 0x7f492c147ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:55.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.312+0000 7f4930f1e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f492c0736d0 con 0x7f492c06c3a0 2024-09-19T08:43:55.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.312+0000 7f4930f1e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f492c073550 con 0x7f492c15c550 2024-09-19T08:43:55.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.312+0000 7f4930f1e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f492c16a240 con 0x7f492c0cf900 2024-09-19T08:43:55.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.312+0000 7f492a59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f492c06c3a0 0x7f492c14bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:55.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.312+0000 7f492a59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f492c06c3a0 0x7f492c14bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33032/0 (socket says 172.21.15.98:33032) 2024-09-19T08:43:55.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.312+0000 7f492a59c640 1 -- 172.21.15.98:0/2208690174 learned_addr learned my addr 172.21.15.98:0/2208690174 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:55.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.312+0000 7f492a59c640 1 -- 172.21.15.98:0/2208690174 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f492c0cf900 msgr2=0x7f492c158870 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T08:43:55.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.312+0000 7f492a59c640 1 --2- 172.21.15.98:0/2208690174 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f492c0cf900 0x7f492c158870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.312+0000 7f492a59c640 1 -- 172.21.15.98:0/2208690174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f492c15c550 msgr2=0x7f492c147ad0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T08:43:55.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.312+0000 7f492a59c640 1 --2- 172.21.15.98:0/2208690174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f492c15c550 0x7f492c147ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.312+0000 7f492a59c640 1 -- 172.21.15.98:0/2208690174 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f492c0cc960 con 0x7f492c06c3a0 2024-09-19T08:43:55.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.312+0000 7f492a59c640 1 --2- 172.21.15.98:0/2208690174 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f492c06c3a0 0x7f492c14bfc0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f491c00b4d0 tx=0x7f491c00b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:55.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.312+0000 7f4922ffd640 1 -- 172.21.15.98:0/2208690174 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491c014070 con 0x7f492c06c3a0 2024-09-19T08:43:55.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.312+0000 7f4922ffd640 1 -- 172.21.15.98:0/2208690174 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f491c010890 con 0x7f492c06c3a0 2024-09-19T08:43:55.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.312+0000 7f4922ffd640 1 -- 172.21.15.98:0/2208690174 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491c010c80 con 0x7f492c06c3a0 2024-09-19T08:43:55.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.312+0000 7f4930f1e640 1 -- 172.21.15.98:0/2208690174 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f492c156770 con 0x7f492c06c3a0 2024-09-19T08:43:55.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.312+0000 7f4930f1e640 1 -- 172.21.15.98:0/2208690174 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f492c156d30 con 0x7f492c06c3a0 2024-09-19T08:43:55.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.316+0000 7f4930f1e640 1 -- 172.21.15.98:0/2208690174 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f492c0cf3c0 con 0x7f492c06c3a0 2024-09-19T08:43:55.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.316+0000 7f4922ffd640 1 -- 172.21.15.98:0/2208690174 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f491c017070 con 0x7f492c06c3a0 2024-09-19T08:43:55.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.316+0000 7f4922ffd640 1 --2- 172.21.15.98:0/2208690174 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f490c0679e0 0x7f490c069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:55.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.316+0000 7f4922ffd640 1 -- 172.21.15.98:0/2208690174 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f491c085b40 con 0x7f492c06c3a0 2024-09-19T08:43:55.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.316+0000 7f4929d9b640 1 --2- 172.21.15.98:0/2208690174 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f490c0679e0 0x7f490c069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:55.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.316+0000 7f4929d9b640 1 --2- 172.21.15.98:0/2208690174 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f490c0679e0 0x7f490c069ea0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f49140ad000 tx=0x7f4914008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:55.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.320+0000 7f4922ffd640 1 -- 172.21.15.98:0/2208690174 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f491c056600 con 0x7f492c06c3a0 2024-09-19T08:43:55.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.484+0000 7f4930f1e640 1 -- 172.21.15.98:0/2208690174 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f492c0cdc90 con 0x7f492c06c3a0 2024-09-19T08:43:55.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.488+0000 7f4922ffd640 1 -- 172.21.15.98:0/2208690174 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+16700 (secure 0 0 0) 0x7f491c010e20 con 0x7f492c06c3a0 2024-09-19T08:43:55.490 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:55.490 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":101,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:43:54.489024+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:43:52.161395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-19T08:43:53.497026+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:43:55.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.488+0000 7f4930f1e640 1 -- 172.21.15.98:0/2208690174 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f490c0679e0 msgr2=0x7f490c069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:55.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.488+0000 7f4930f1e640 1 --2- 172.21.15.98:0/2208690174 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f490c0679e0 0x7f490c069ea0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f49140ad000 tx=0x7f4914008040 comp rx=0 tx=0).stop 2024-09-19T08:43:55.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.488+0000 7f4930f1e640 1 -- 172.21.15.98:0/2208690174 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f492c06c3a0 msgr2=0x7f492c14bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:55.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.488+0000 7f4930f1e640 1 --2- 172.21.15.98:0/2208690174 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f492c06c3a0 0x7f492c14bfc0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f491c00b4d0 tx=0x7f491c00b990 comp rx=0 tx=0).stop 2024-09-19T08:43:55.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.488+0000 7f4930f1e640 1 -- 172.21.15.98:0/2208690174 shutdown_connections 2024-09-19T08:43:55.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.488+0000 7f4930f1e640 1 --2- 172.21.15.98:0/2208690174 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f490c0679e0 0x7f490c069ea0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.488+0000 7f4930f1e640 1 --2- 172.21.15.98:0/2208690174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f492c15c550 0x7f492c147ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.488+0000 7f4930f1e640 1 --2- 172.21.15.98:0/2208690174 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f492c0cf900 0x7f492c158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.488+0000 7f4930f1e640 1 --2- 172.21.15.98:0/2208690174 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f492c06c3a0 0x7f492c14bfc0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.492+0000 7f4930f1e640 1 -- 172.21.15.98:0/2208690174 >> 172.21.15.98:0/2208690174 conn(0x7f492c0b2010 msgr2=0x7f492c067a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:55.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.492+0000 7f4930f1e640 1 -- 172.21.15.98:0/2208690174 shutdown_connections 2024-09-19T08:43:55.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.492+0000 7f4930f1e640 1 -- 172.21.15.98:0/2208690174 wait complete. 2024-09-19T08:43:55.509 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:43:55.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.624+0000 7f232d324640 1 Processor -- start 2024-09-19T08:43:55.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.624+0000 7f232d324640 1 -- start start 2024-09-19T08:43:55.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.624+0000 7f232d324640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f232806c120 0x7f23280cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:55.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.624+0000 7f232d324640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23280ccc60 con 0x7f2328158fd0 2024-09-19T08:43:55.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.624+0000 7f232d324640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23280ccf40 con 0x7f232806bc90 2024-09-19T08:43:55.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.624+0000 7f232d324640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f23281593b0 con 0x7f232806c120 2024-09-19T08:43:55.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.624+0000 7f2326ffd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f2328158fd0 0x7f23280b3e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:49390/0 (socket says 172.21.15.98:49390) 2024-09-19T08:43:55.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.624+0000 7f2326ffd640 1 -- 172.21.15.98:0/2657601216 learned_addr learned my addr 172.21.15.98:0/2657601216 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:55.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.624+0000 7f23277fe640 1 --2- 172.21.15.98:0/2657601216 >> v2:172.21.15.139:3300/0 conn(0x7f232806c120 0x7f23280cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:55.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.624+0000 7f2325ffb640 1 -- 172.21.15.98:0/2657601216 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2960533505 0 0) 0x7f23280ccc60 con 0x7f2328158fd0 2024-09-19T08:43:55.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.624+0000 7f2325ffb640 1 -- 172.21.15.98:0/2657601216 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2310003530 con 0x7f2328158fd0 2024-09-19T08:43:55.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.624+0000 7f2325ffb640 1 -- 172.21.15.98:0/2657601216 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 995666972 0 0) 0x7f2310003530 con 0x7f2328158fd0 2024-09-19T08:43:55.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.624+0000 7f2325ffb640 1 -- 172.21.15.98:0/2657601216 >> v2:172.21.15.139:3300/0 conn(0x7f232806c120 msgr2=0x7f23280cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:55.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.624+0000 7f2325ffb640 1 --2- 172.21.15.98:0/2657601216 >> v2:172.21.15.139:3300/0 conn(0x7f232806c120 0x7f23280cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.624+0000 7f2325ffb640 1 -- 172.21.15.98:0/2657601216 >> v1:172.21.15.139:6789/0 conn(0x7f232806bc90 legacy=0x7f23280ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:55.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.624+0000 7f2325ffb640 1 -- 172.21.15.98:0/2657601216 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23280cd220 con 0x7f2328158fd0 2024-09-19T08:43:55.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.624+0000 7f2325ffb640 1 -- 172.21.15.98:0/2657601216 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f231c001680 con 0x7f2328158fd0 2024-09-19T08:43:55.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.624+0000 7f2325ffb640 1 -- 172.21.15.98:0/2657601216 >> v1:172.21.15.98:6789/0 conn(0x7f2328158fd0 legacy=0x7f23280b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:55.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.624+0000 7f2325ffb640 1 --2- 172.21.15.98:0/2657601216 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23100039c0 0x7f2310003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:55.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.624+0000 7f2325ffb640 1 --2- 172.21.15.98:0/2657601216 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2310004810 0x7f2310004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:55.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.624+0000 7f2325ffb640 1 --2- 172.21.15.98:0/2657601216 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23100051d0 0x7f231000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:55.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.624+0000 7f2325ffb640 1 -- 172.21.15.98:0/2657601216 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f23281593b0 con 0x7f2310004810 2024-09-19T08:43:55.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.624+0000 7f2325ffb640 1 -- 172.21.15.98:0/2657601216 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f231000c170 con 0x7f23100051d0 2024-09-19T08:43:55.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.624+0000 7f2325ffb640 1 -- 172.21.15.98:0/2657601216 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f231000c370 con 0x7f23100039c0 2024-09-19T08:43:55.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.624+0000 7f23267fc640 1 --2- 172.21.15.98:0/2657601216 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2310004810 0x7f2310004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:55.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.624+0000 7f23277fe640 1 --2- 172.21.15.98:0/2657601216 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23100051d0 0x7f231000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:55.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.624+0000 7f23267fc640 1 -- 172.21.15.98:0/2657601216 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23100039c0 msgr2=0x7f2310003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:55.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.624+0000 7f23267fc640 1 --2- 172.21.15.98:0/2657601216 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23100039c0 0x7f2310003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.624+0000 7f23267fc640 1 -- 172.21.15.98:0/2657601216 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23100051d0 msgr2=0x7f231000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:55.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.624+0000 7f23267fc640 1 --2- 172.21.15.98:0/2657601216 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23100051d0 0x7f231000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.624+0000 7f23277fe640 1 -- 172.21.15.98:0/2657601216 reap_dead start 2024-09-19T08:43:55.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.624+0000 7f23267fc640 1 -- 172.21.15.98:0/2657601216 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2310003530 con 0x7f2310004810 2024-09-19T08:43:55.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.628+0000 7f23267fc640 1 --2- 172.21.15.98:0/2657601216 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2310004810 0x7f2310004c90 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f2318001780 tx=0x7f231801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20042c32e3d3423c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:55.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.628+0000 7f2325ffb640 1 -- 172.21.15.98:0/2657601216 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23180013a0 con 0x7f2310004810 2024-09-19T08:43:55.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.628+0000 7f2325ffb640 1 -- 172.21.15.98:0/2657601216 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23180052f0 con 0x7f2310004810 2024-09-19T08:43:55.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.628+0000 7f2325ffb640 1 -- 172.21.15.98:0/2657601216 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2318003840 con 0x7f2310004810 2024-09-19T08:43:55.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.628+0000 7f232d324640 1 -- 172.21.15.98:0/2657601216 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2310004810 msgr2=0x7f2310004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:55.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.628+0000 7f232d324640 1 --2- 172.21.15.98:0/2657601216 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2310004810 0x7f2310004c90 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f2318001780 tx=0x7f231801c3d0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.628+0000 7f232d324640 1 -- 172.21.15.98:0/2657601216 shutdown_connections 2024-09-19T08:43:55.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.628+0000 7f232d324640 1 --2- 172.21.15.98:0/2657601216 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2310004810 0x7f2310004c90 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.628+0000 7f232d324640 1 -- 172.21.15.98:0/2657601216 >> 172.21.15.98:0/2657601216 conn(0x7f23280b2010 msgr2=0x7f23280b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:55.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.628+0000 7f232d324640 1 -- 172.21.15.98:0/2657601216 shutdown_connections 2024-09-19T08:43:55.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.628+0000 7f232d324640 1 -- 172.21.15.98:0/2657601216 wait complete. 2024-09-19T08:43:55.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.628+0000 7f232d324640 1 Processor -- start 2024-09-19T08:43:55.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.628+0000 7f232d324640 1 -- start start 2024-09-19T08:43:55.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.628+0000 7f232d324640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2310004810 0x7f23280d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:55.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.628+0000 7f232d324640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f232815ebb0 0x7f2328155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:55.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.628+0000 7f232d324640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f232815d9c0 0x7f23280ce730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:55.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.628+0000 7f232d324640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f232815c920 con 0x7f232815d9c0 2024-09-19T08:43:55.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.628+0000 7f232d324640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f232815c7a0 con 0x7f2310004810 2024-09-19T08:43:55.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.628+0000 7f232d324640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f232815efc0 con 0x7f232815ebb0 2024-09-19T08:43:55.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.628+0000 7f23277fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f232815d9c0 0x7f23280ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:55.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.628+0000 7f23277fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f232815d9c0 0x7f23280ce730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33050/0 (socket says 172.21.15.98:33050) 2024-09-19T08:43:55.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.628+0000 7f23277fe640 1 -- 172.21.15.98:0/3897202067 learned_addr learned my addr 172.21.15.98:0/3897202067 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:55.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.632+0000 7f2326ffd640 1 --2- 172.21.15.98:0/3897202067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2310004810 0x7f23280d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:55.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.632+0000 7f23267fc640 1 --2- 172.21.15.98:0/3897202067 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f232815ebb0 0x7f2328155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:55.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.632+0000 7f23277fe640 1 -- 172.21.15.98:0/3897202067 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f232815ebb0 msgr2=0x7f2328155640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:55.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.632+0000 7f23277fe640 1 --2- 172.21.15.98:0/3897202067 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f232815ebb0 0x7f2328155640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.632+0000 7f23277fe640 1 -- 172.21.15.98:0/3897202067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2310004810 msgr2=0x7f23280d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:55.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.632+0000 7f23277fe640 1 --2- 172.21.15.98:0/3897202067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2310004810 0x7f23280d05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.632+0000 7f23277fe640 1 -- 172.21.15.98:0/3897202067 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232815dec0 con 0x7f232815d9c0 2024-09-19T08:43:55.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.632+0000 7f23267fc640 1 --2- 172.21.15.98:0/3897202067 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f232815ebb0 0x7f2328155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:55.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.632+0000 7f23277fe640 1 --2- 172.21.15.98:0/3897202067 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f232815d9c0 0x7f23280ce730 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f232815c770 tx=0x7f23200115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:55.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.632+0000 7f230f7fe640 1 -- 172.21.15.98:0/3897202067 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2320011db0 con 0x7f232815d9c0 2024-09-19T08:43:55.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.632+0000 7f230f7fe640 1 -- 172.21.15.98:0/3897202067 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f232001b070 con 0x7f232815d9c0 2024-09-19T08:43:55.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.632+0000 7f230f7fe640 1 -- 172.21.15.98:0/3897202067 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23200039a0 con 0x7f232815d9c0 2024-09-19T08:43:55.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.632+0000 7f232d324640 1 -- 172.21.15.98:0/3897202067 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23280714b0 con 0x7f232815d9c0 2024-09-19T08:43:55.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.632+0000 7f232d324640 1 -- 172.21.15.98:0/3897202067 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2328155b80 con 0x7f232815d9c0 2024-09-19T08:43:55.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.632+0000 7f232d324640 1 -- 172.21.15.98:0/3897202067 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23280d0210 con 0x7f232815d9c0 2024-09-19T08:43:55.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.632+0000 7f230f7fe640 1 -- 172.21.15.98:0/3897202067 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f2320003b40 con 0x7f232815d9c0 2024-09-19T08:43:55.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.632+0000 7f230f7fe640 1 --2- 172.21.15.98:0/3897202067 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f23140679e0 0x7f2314069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:55.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.632+0000 7f230f7fe640 1 -- 172.21.15.98:0/3897202067 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f23200866f0 con 0x7f232815d9c0 2024-09-19T08:43:55.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.632+0000 7f2326ffd640 1 --2- 172.21.15.98:0/3897202067 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f23140679e0 0x7f2314069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:55.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.632+0000 7f2326ffd640 1 --2- 172.21.15.98:0/3897202067 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f23140679e0 0x7f2314069ea0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f231c001870 tx=0x7f231c001660 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:55.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.636+0000 7f230f7fe640 1 -- 172.21.15.98:0/3897202067 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f23200570a0 con 0x7f232815d9c0 2024-09-19T08:43:55.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.796+0000 7f232d324640 1 -- 172.21.15.98:0/3897202067 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f23280712a0 con 0x7f232815d9c0 2024-09-19T08:43:55.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.796+0000 7f230f7fe640 1 -- 172.21.15.98:0/3897202067 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+16700 (secure 0 0 0) 0x7f23200569a0 con 0x7f232815d9c0 2024-09-19T08:43:55.799 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:55.799 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":101,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:43:54.489024+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:43:52.161395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-19T08:43:53.497026+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:43:55.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.800+0000 7f232d324640 1 -- 172.21.15.98:0/3897202067 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f23140679e0 msgr2=0x7f2314069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:55.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.800+0000 7f232d324640 1 --2- 172.21.15.98:0/3897202067 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f23140679e0 0x7f2314069ea0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f231c001870 tx=0x7f231c001660 comp rx=0 tx=0).stop 2024-09-19T08:43:55.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.800+0000 7f232d324640 1 -- 172.21.15.98:0/3897202067 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f232815d9c0 msgr2=0x7f23280ce730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:55.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.800+0000 7f232d324640 1 --2- 172.21.15.98:0/3897202067 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f232815d9c0 0x7f23280ce730 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f232815c770 tx=0x7f23200115a0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.800+0000 7f232d324640 1 -- 172.21.15.98:0/3897202067 shutdown_connections 2024-09-19T08:43:55.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.800+0000 7f232d324640 1 --2- 172.21.15.98:0/3897202067 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f23140679e0 0x7f2314069ea0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.800+0000 7f232d324640 1 --2- 172.21.15.98:0/3897202067 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f232815d9c0 0x7f23280ce730 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.800+0000 7f232d324640 1 --2- 172.21.15.98:0/3897202067 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f232815ebb0 0x7f2328155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.800+0000 7f232d324640 1 --2- 172.21.15.98:0/3897202067 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2310004810 0x7f23280d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.800+0000 7f232d324640 1 -- 172.21.15.98:0/3897202067 >> 172.21.15.98:0/3897202067 conn(0x7f23280b2010 msgr2=0x7f2328156e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:55.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.800+0000 7f232d324640 1 -- 172.21.15.98:0/3897202067 shutdown_connections 2024-09-19T08:43:55.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.800+0000 7f232d324640 1 -- 172.21.15.98:0/3897202067 wait complete. 2024-09-19T08:43:55.817 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-19T08:43:55.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.928+0000 7f2c33a48640 1 Processor -- start 2024-09-19T08:43:55.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.928+0000 7f2c33a48640 1 -- start start 2024-09-19T08:43:55.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.928+0000 7f2c33a48640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2c2c0b4420 0x7f2c2c071ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:55.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.928+0000 7f2c33a48640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c2c073530 con 0x7f2c2c06c3a0 2024-09-19T08:43:55.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.928+0000 7f2c33a48640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c2c073810 con 0x7f2c2c145b70 2024-09-19T08:43:55.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.928+0000 7f2c33a48640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f2c2c073af0 con 0x7f2c2c0b4420 2024-09-19T08:43:55.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.928+0000 7f2c30fe3640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f2c2c06c3a0 0x7f2c2c0713b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:49406/0 (socket says 172.21.15.98:49406) 2024-09-19T08:43:55.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.928+0000 7f2c30fe3640 1 -- 172.21.15.98:0/2406625820 learned_addr learned my addr 172.21.15.98:0/2406625820 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:55.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.928+0000 7f2c31fe5640 1 --2- 172.21.15.98:0/2406625820 >> v2:172.21.15.139:3300/0 conn(0x7f2c2c0b4420 0x7f2c2c071ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:55.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.928+0000 7f2c23fff640 1 -- 172.21.15.98:0/2406625820 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 149637320 0 0) 0x7f2c2c073530 con 0x7f2c2c06c3a0 2024-09-19T08:43:55.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.928+0000 7f2c23fff640 1 -- 172.21.15.98:0/2406625820 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c18003530 con 0x7f2c2c06c3a0 2024-09-19T08:43:55.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.932+0000 7f2c23fff640 1 -- 172.21.15.98:0/2406625820 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3358201695 0 0) 0x7f2c18003530 con 0x7f2c2c06c3a0 2024-09-19T08:43:55.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.932+0000 7f2c23fff640 1 -- 172.21.15.98:0/2406625820 >> v2:172.21.15.139:3300/0 conn(0x7f2c2c0b4420 msgr2=0x7f2c2c071ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:55.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.932+0000 7f2c23fff640 1 --2- 172.21.15.98:0/2406625820 >> v2:172.21.15.139:3300/0 conn(0x7f2c2c0b4420 0x7f2c2c071ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.932+0000 7f2c23fff640 1 -- 172.21.15.98:0/2406625820 >> v1:172.21.15.139:6789/0 conn(0x7f2c2c145b70 legacy=0x7f2c2c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:55.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.932+0000 7f2c23fff640 1 -- 172.21.15.98:0/2406625820 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c2c073dd0 con 0x7f2c2c06c3a0 2024-09-19T08:43:55.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.932+0000 7f2c23fff640 1 -- 172.21.15.98:0/2406625820 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f2c1c001680 con 0x7f2c2c06c3a0 2024-09-19T08:43:55.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.932+0000 7f2c23fff640 1 -- 172.21.15.98:0/2406625820 >> v1:172.21.15.98:6789/0 conn(0x7f2c2c06c3a0 legacy=0x7f2c2c0713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:55.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.932+0000 7f2c23fff640 1 --2- 172.21.15.98:0/2406625820 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c180039c0 0x7f2c18003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:55.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.932+0000 7f2c23fff640 1 --2- 172.21.15.98:0/2406625820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c18004810 0x7f2c18004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:55.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.932+0000 7f2c23fff640 1 --2- 172.21.15.98:0/2406625820 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c180051d0 0x7f2c1800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:55.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.932+0000 7f2c23fff640 1 -- 172.21.15.98:0/2406625820 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c073af0 con 0x7f2c18004810 2024-09-19T08:43:55.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.932+0000 7f2c23fff640 1 -- 172.21.15.98:0/2406625820 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1800c170 con 0x7f2c180039c0 2024-09-19T08:43:55.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.932+0000 7f2c23fff640 1 -- 172.21.15.98:0/2406625820 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2c1800c370 con 0x7f2c180051d0 2024-09-19T08:43:55.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.932+0000 7f2c30fe3640 1 --2- 172.21.15.98:0/2406625820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c18004810 0x7f2c18004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:55.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.932+0000 7f2c317e4640 1 --2- 172.21.15.98:0/2406625820 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c180039c0 0x7f2c18003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:55.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.932+0000 7f2c30fe3640 1 -- 172.21.15.98:0/2406625820 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c180051d0 msgr2=0x7f2c1800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:55.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.932+0000 7f2c30fe3640 1 --2- 172.21.15.98:0/2406625820 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c180051d0 0x7f2c1800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.932+0000 7f2c30fe3640 1 -- 172.21.15.98:0/2406625820 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c180039c0 msgr2=0x7f2c18003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:55.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.932+0000 7f2c30fe3640 1 --2- 172.21.15.98:0/2406625820 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c180039c0 0x7f2c18003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.932+0000 7f2c30fe3640 1 -- 172.21.15.98:0/2406625820 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c18003530 con 0x7f2c18004810 2024-09-19T08:43:55.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.932+0000 7f2c31fe5640 1 -- 172.21.15.98:0/2406625820 reap_dead start 2024-09-19T08:43:55.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.932+0000 7f2c30fe3640 1 --2- 172.21.15.98:0/2406625820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c18004810 0x7f2c18004c90 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c00c240 tx=0x7f2c1c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325a590ac9af1521 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:55.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.932+0000 7f2c23fff640 1 -- 172.21.15.98:0/2406625820 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c007d20 con 0x7f2c18004810 2024-09-19T08:43:55.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.932+0000 7f2c23fff640 1 -- 172.21.15.98:0/2406625820 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c1c0a9070 con 0x7f2c18004810 2024-09-19T08:43:55.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.932+0000 7f2c23fff640 1 -- 172.21.15.98:0/2406625820 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c005da0 con 0x7f2c18004810 2024-09-19T08:43:55.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.932+0000 7f2c33a48640 1 -- 172.21.15.98:0/2406625820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c18004810 msgr2=0x7f2c18004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:55.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.932+0000 7f2c33a48640 1 --2- 172.21.15.98:0/2406625820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c18004810 0x7f2c18004c90 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c00c240 tx=0x7f2c1c01cc40 comp rx=0 tx=0).stop 2024-09-19T08:43:55.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.932+0000 7f2c33a48640 1 -- 172.21.15.98:0/2406625820 shutdown_connections 2024-09-19T08:43:55.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.932+0000 7f2c33a48640 1 --2- 172.21.15.98:0/2406625820 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c18004810 0x7f2c18004c90 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.932+0000 7f2c33a48640 1 -- 172.21.15.98:0/2406625820 >> 172.21.15.98:0/2406625820 conn(0x7f2c2c0b2010 msgr2=0x7f2c2c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:55.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.932+0000 7f2c33a48640 1 -- 172.21.15.98:0/2406625820 shutdown_connections 2024-09-19T08:43:55.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.932+0000 7f2c33a48640 1 -- 172.21.15.98:0/2406625820 wait complete. 2024-09-19T08:43:55.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.936+0000 7f2c33a48640 1 Processor -- start 2024-09-19T08:43:55.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.936+0000 7f2c33a48640 1 -- start start 2024-09-19T08:43:55.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.936+0000 7f2c33a48640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c18004810 0x7f2c2c0ce870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:55.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.936+0000 7f2c33a48640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c2c15c860 0x7f2c2c15de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:55.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.936+0000 7f2c33a48640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c2c139f90 0x7f2c2c15e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:55.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.936+0000 7f2c33a48640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c0b3410 con 0x7f2c2c139f90 2024-09-19T08:43:55.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.936+0000 7f2c33a48640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c071dd0 con 0x7f2c2c15c860 2024-09-19T08:43:55.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.936+0000 7f2c33a48640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2c2c0b3590 con 0x7f2c18004810 2024-09-19T08:43:55.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.936+0000 7f2c31fe5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c2c139f90 0x7f2c2c15e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:55.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.936+0000 7f2c31fe5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c2c139f90 0x7f2c2c15e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33074/0 (socket says 172.21.15.98:33074) 2024-09-19T08:43:55.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.936+0000 7f2c31fe5640 1 -- 172.21.15.98:0/1792609368 learned_addr learned my addr 172.21.15.98:0/1792609368 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:55.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.936+0000 7f2c317e4640 1 --2- 172.21.15.98:0/1792609368 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c18004810 0x7f2c2c0ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:55.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.936+0000 7f2c30fe3640 1 --2- 172.21.15.98:0/1792609368 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c2c15c860 0x7f2c2c15de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:55.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.936+0000 7f2c31fe5640 1 -- 172.21.15.98:0/1792609368 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c18004810 msgr2=0x7f2c2c0ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:55.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.936+0000 7f2c31fe5640 1 --2- 172.21.15.98:0/1792609368 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c18004810 0x7f2c2c0ce870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.936+0000 7f2c31fe5640 1 -- 172.21.15.98:0/1792609368 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c2c15c860 msgr2=0x7f2c2c15de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:55.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.936+0000 7f2c31fe5640 1 --2- 172.21.15.98:0/1792609368 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c2c15c860 0x7f2c2c15de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:55.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.936+0000 7f2c31fe5640 1 -- 172.21.15.98:0/1792609368 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c2c15cdd0 con 0x7f2c2c139f90 2024-09-19T08:43:55.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.936+0000 7f2c31fe5640 1 --2- 172.21.15.98:0/1792609368 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c2c139f90 0x7f2c2c15e3b0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0b2ce0 tx=0x7f2c2400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:55.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.936+0000 7f2c21ffb640 1 -- 172.21.15.98:0/1792609368 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2400bc80 con 0x7f2c2c139f90 2024-09-19T08:43:55.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.936+0000 7f2c21ffb640 1 -- 172.21.15.98:0/1792609368 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c2401a070 con 0x7f2c2c139f90 2024-09-19T08:43:55.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.936+0000 7f2c33a48640 1 -- 172.21.15.98:0/1792609368 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c2c15d0b0 con 0x7f2c2c139f90 2024-09-19T08:43:55.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.936+0000 7f2c21ffb640 1 -- 172.21.15.98:0/1792609368 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c24014640 con 0x7f2c2c139f90 2024-09-19T08:43:55.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.936+0000 7f2c33a48640 1 -- 172.21.15.98:0/1792609368 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c2c15d670 con 0x7f2c2c139f90 2024-09-19T08:43:55.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.936+0000 7f2c33a48640 1 -- 172.21.15.98:0/1792609368 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c2c0b3920 con 0x7f2c2c139f90 2024-09-19T08:43:55.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.936+0000 7f2c21ffb640 1 -- 172.21.15.98:0/1792609368 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f2c24013070 con 0x7f2c2c139f90 2024-09-19T08:43:55.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.940+0000 7f2c21ffb640 1 --2- 172.21.15.98:0/1792609368 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2c100679e0 0x7f2c10069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:55.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.940+0000 7f2c21ffb640 1 -- 172.21.15.98:0/1792609368 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f2c240862b0 con 0x7f2c2c139f90 2024-09-19T08:43:55.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.940+0000 7f2c317e4640 1 --2- 172.21.15.98:0/1792609368 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2c100679e0 0x7f2c10069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:55.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.940+0000 7f2c317e4640 1 --2- 172.21.15.98:0/1792609368 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2c100679e0 0x7f2c10069ea0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2c28000c10 tx=0x7f2c28009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:55.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:55.944+0000 7f2c21ffb640 1 -- 172.21.15.98:0/1792609368 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2c2400be20 con 0x7f2c2c139f90 2024-09-19T08:43:56.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.124+0000 7f2c33a48640 1 -- 172.21.15.98:0/1792609368 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f2c2c1426c0 con 0x7f2c2c139f90 2024-09-19T08:43:56.128 INFO:tasks.ceph.mds.c.smithi098.stderr:2024-09-19T08:43:56.128+0000 7f026afd2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T08:43:56.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.212+0000 7f2c21ffb640 1 -- 172.21.15.98:0/1792609368 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v59) ==== 118+0+0 (secure 0 0 0) 0x7f2c24056b40 con 0x7f2c2c139f90 2024-09-19T08:43:56.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.216+0000 7f2c33a48640 1 -- 172.21.15.98:0/1792609368 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2c100679e0 msgr2=0x7f2c10069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:56.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.216+0000 7f2c33a48640 1 --2- 172.21.15.98:0/1792609368 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2c100679e0 0x7f2c10069ea0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2c28000c10 tx=0x7f2c28009040 comp rx=0 tx=0).stop 2024-09-19T08:43:56.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.216+0000 7f2c33a48640 1 -- 172.21.15.98:0/1792609368 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c2c139f90 msgr2=0x7f2c2c15e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:56.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.216+0000 7f2c33a48640 1 --2- 172.21.15.98:0/1792609368 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c2c139f90 0x7f2c2c15e3b0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0b2ce0 tx=0x7f2c2400b480 comp rx=0 tx=0).stop 2024-09-19T08:43:56.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.216+0000 7f2c33a48640 1 -- 172.21.15.98:0/1792609368 shutdown_connections 2024-09-19T08:43:56.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.216+0000 7f2c33a48640 1 --2- 172.21.15.98:0/1792609368 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2c100679e0 0x7f2c10069ea0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:56.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.216+0000 7f2c33a48640 1 --2- 172.21.15.98:0/1792609368 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c2c139f90 0x7f2c2c15e3b0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:56.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.216+0000 7f2c33a48640 1 --2- 172.21.15.98:0/1792609368 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c2c15c860 0x7f2c2c15de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:56.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.216+0000 7f2c33a48640 1 --2- 172.21.15.98:0/1792609368 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c18004810 0x7f2c2c0ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:56.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.216+0000 7f2c33a48640 1 -- 172.21.15.98:0/1792609368 >> 172.21.15.98:0/1792609368 conn(0x7f2c2c0b2010 msgr2=0x7f2c2c0b4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:56.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.216+0000 7f2c33a48640 1 -- 172.21.15.98:0/1792609368 shutdown_connections 2024-09-19T08:43:56.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.216+0000 7f2c33a48640 1 -- 172.21.15.98:0/1792609368 wait complete. 2024-09-19T08:43:56.236 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-19T08:43:56.236 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-19T08:43:56.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.356+0000 7fd18bdf9640 1 Processor -- start 2024-09-19T08:43:56.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.356+0000 7fd18bdf9640 1 -- start start 2024-09-19T08:43:56.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.360+0000 7fd18bdf9640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd1840729f0 0x7fd18406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:56.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.360+0000 7fd18bdf9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd18406c1d0 0x7fd1840cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:56.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.360+0000 7fd18bdf9640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd184072ea0 con 0x7fd184144d10 2024-09-19T08:43:56.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.360+0000 7fd18bdf9640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fd184073ab0 con 0x7fd1840729f0 2024-09-19T08:43:56.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.360+0000 7fd18bdf9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd1841450f0 con 0x7fd18406c1d0 2024-09-19T08:43:56.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.360+0000 7fd189b95640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fd184144d10 0x7fd1840722e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:49412/0 (socket says 172.21.15.98:49412) 2024-09-19T08:43:56.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.360+0000 7fd189394640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd1840729f0 0x7fd18406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:56.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.360+0000 7fd189b95640 1 -- 172.21.15.98:0/3531532215 learned_addr learned my addr 172.21.15.98:0/3531532215 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:56.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.360+0000 7fd189394640 1 -- 172.21.15.98:0/3531532215 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd18406c1d0 msgr2=0x7fd1840cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:56.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.360+0000 7fd189394640 1 --2- 172.21.15.98:0/3531532215 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd18406c1d0 0x7fd1840cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:56.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.360+0000 7fd189394640 1 -- 172.21.15.98:0/3531532215 >> v1:172.21.15.98:6789/0 conn(0x7fd184144d10 legacy=0x7fd1840722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:56.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.360+0000 7fd189394640 1 -- 172.21.15.98:0/3531532215 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd184073180 con 0x7fd1840729f0 2024-09-19T08:43:56.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.360+0000 7fd188b93640 1 -- 172.21.15.98:0/3531532215 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2467926642 0 0) 0x7fd184072ea0 con 0x7fd184144d10 2024-09-19T08:43:56.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.360+0000 7fd189394640 1 --2- 172.21.15.98:0/3531532215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1840729f0 0x7fd18406bc90 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fd1740097c0 tx=0x7fd17401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af7600cc1a97c9b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:56.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.360+0000 7fd188b93640 1 -- 172.21.15.98:0/3531532215 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1740b2070 con 0x7fd1840729f0 2024-09-19T08:43:56.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.360+0000 7fd188b93640 1 -- 172.21.15.98:0/3531532215 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1740ad640 con 0x7fd1840729f0 2024-09-19T08:43:56.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.360+0000 7fd188b93640 1 -- 172.21.15.98:0/3531532215 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1740ada10 con 0x7fd1840729f0 2024-09-19T08:43:56.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.364+0000 7fd18bdf9640 1 -- 172.21.15.98:0/3531532215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1840729f0 msgr2=0x7fd18406bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:56.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.364+0000 7fd18bdf9640 1 --2- 172.21.15.98:0/3531532215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1840729f0 0x7fd18406bc90 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fd1740097c0 tx=0x7fd17401c4a0 comp rx=0 tx=0).stop 2024-09-19T08:43:56.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.364+0000 7fd18bdf9640 1 -- 172.21.15.98:0/3531532215 shutdown_connections 2024-09-19T08:43:56.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.364+0000 7fd18bdf9640 1 --2- 172.21.15.98:0/3531532215 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd18406c1d0 0x7fd1840cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:56.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.364+0000 7fd18bdf9640 1 --2- 172.21.15.98:0/3531532215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1840729f0 0x7fd18406bc90 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:56.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.364+0000 7fd18bdf9640 1 -- 172.21.15.98:0/3531532215 >> 172.21.15.98:0/3531532215 conn(0x7fd1840b2010 msgr2=0x7fd1840b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:56.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.364+0000 7fd18bdf9640 1 -- 172.21.15.98:0/3531532215 shutdown_connections 2024-09-19T08:43:56.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.364+0000 7fd18bdf9640 1 -- 172.21.15.98:0/3531532215 wait complete. 2024-09-19T08:43:56.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.364+0000 7fd18bdf9640 1 Processor -- start 2024-09-19T08:43:56.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.364+0000 7fd18bdf9640 1 -- start start 2024-09-19T08:43:56.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.364+0000 7fd18bdf9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd18406c1d0 0x7fd184143ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:56.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.364+0000 7fd18bdf9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1840729f0 0x7fd18415a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:56.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.364+0000 7fd18bdf9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd184144d10 0x7fd184159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:56.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.364+0000 7fd18bdf9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd1840ce120 con 0x7fd184144d10 2024-09-19T08:43:56.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.364+0000 7fd18bdf9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd1840cdfa0 con 0x7fd18406c1d0 2024-09-19T08:43:56.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.364+0000 7fd18bdf9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd1840ce2a0 con 0x7fd1840729f0 2024-09-19T08:43:56.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.364+0000 7fd18a396640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd184144d10 0x7fd184159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:56.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.364+0000 7fd18a396640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd184144d10 0x7fd184159fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33092/0 (socket says 172.21.15.98:33092) 2024-09-19T08:43:56.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.364+0000 7fd18a396640 1 -- 172.21.15.98:0/4011450290 learned_addr learned my addr 172.21.15.98:0/4011450290 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:56.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.364+0000 7fd189b95640 1 --2- 172.21.15.98:0/4011450290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd18406c1d0 0x7fd184143ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:56.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.364+0000 7fd189394640 1 --2- 172.21.15.98:0/4011450290 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1840729f0 0x7fd18415a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:56.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.364+0000 7fd18a396640 1 -- 172.21.15.98:0/4011450290 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1840729f0 msgr2=0x7fd18415a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:56.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.364+0000 7fd18a396640 1 --2- 172.21.15.98:0/4011450290 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1840729f0 0x7fd18415a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:56.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.364+0000 7fd18a396640 1 -- 172.21.15.98:0/4011450290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd18406c1d0 msgr2=0x7fd184143ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:56.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.364+0000 7fd18a396640 1 --2- 172.21.15.98:0/4011450290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd18406c1d0 0x7fd184143ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:56.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.364+0000 7fd18a396640 1 -- 172.21.15.98:0/4011450290 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd184142a10 con 0x7fd184144d10 2024-09-19T08:43:56.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.364+0000 7fd18a396640 1 --2- 172.21.15.98:0/4011450290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd184144d10 0x7fd184159fd0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00b970 tx=0x7fd17c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:56.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.364+0000 7fd17a7fc640 1 -- 172.21.15.98:0/4011450290 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c019070 con 0x7fd184144d10 2024-09-19T08:43:56.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.364+0000 7fd17a7fc640 1 -- 172.21.15.98:0/4011450290 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd17c00cb60 con 0x7fd184144d10 2024-09-19T08:43:56.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.364+0000 7fd18bdf9640 1 -- 172.21.15.98:0/4011450290 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd184142cf0 con 0x7fd184144d10 2024-09-19T08:43:56.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.364+0000 7fd17a7fc640 1 -- 172.21.15.98:0/4011450290 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c014440 con 0x7fd184144d10 2024-09-19T08:43:56.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.364+0000 7fd18bdf9640 1 -- 172.21.15.98:0/4011450290 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1841432b0 con 0x7fd184144d10 2024-09-19T08:43:56.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.368+0000 7fd18bdf9640 1 -- 172.21.15.98:0/4011450290 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1840cd250 con 0x7fd184144d10 2024-09-19T08:43:56.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.368+0000 7fd17a7fc640 1 -- 172.21.15.98:0/4011450290 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fd17c01c070 con 0x7fd184144d10 2024-09-19T08:43:56.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.368+0000 7fd17a7fc640 1 --2- 172.21.15.98:0/4011450290 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd168067a10 0x7fd168069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:56.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.368+0000 7fd17a7fc640 1 -- 172.21.15.98:0/4011450290 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fd17c085ad0 con 0x7fd184144d10 2024-09-19T08:43:56.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.368+0000 7fd189b95640 1 --2- 172.21.15.98:0/4011450290 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd168067a10 0x7fd168069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:56.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.368+0000 7fd189b95640 1 --2- 172.21.15.98:0/4011450290 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd168067a10 0x7fd168069ed0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd180000f70 tx=0x7fd180009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:56.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.372+0000 7fd17a7fc640 1 -- 172.21.15.98:0/4011450290 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd17c056590 con 0x7fd184144d10 2024-09-19T08:43:56.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:56.572+0000 7fd18bdf9640 1 -- 172.21.15.98:0/4011450290 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd184143ff0 con 0x7fd184144d10 2024-09-19T08:43:57.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.216+0000 7fd17a7fc640 1 -- 172.21.15.98:0/4011450290 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v60) ==== 122+0+0 (secure 0 0 0) 0x7fd17c055e90 con 0x7fd184144d10 2024-09-19T08:43:57.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.220+0000 7fd18bdf9640 1 -- 172.21.15.98:0/4011450290 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd168067a10 msgr2=0x7fd168069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:57.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.220+0000 7fd18bdf9640 1 --2- 172.21.15.98:0/4011450290 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd168067a10 0x7fd168069ed0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd180000f70 tx=0x7fd180009040 comp rx=0 tx=0).stop 2024-09-19T08:43:57.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.220+0000 7fd18bdf9640 1 -- 172.21.15.98:0/4011450290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd184144d10 msgr2=0x7fd184159fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:57.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.220+0000 7fd18bdf9640 1 --2- 172.21.15.98:0/4011450290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd184144d10 0x7fd184159fd0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00b970 tx=0x7fd17c00be30 comp rx=0 tx=0).stop 2024-09-19T08:43:57.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.220+0000 7fd18bdf9640 1 -- 172.21.15.98:0/4011450290 shutdown_connections 2024-09-19T08:43:57.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.220+0000 7fd18bdf9640 1 --2- 172.21.15.98:0/4011450290 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd168067a10 0x7fd168069ed0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:57.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.220+0000 7fd18bdf9640 1 --2- 172.21.15.98:0/4011450290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd184144d10 0x7fd184159fd0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:57.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.220+0000 7fd18bdf9640 1 --2- 172.21.15.98:0/4011450290 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1840729f0 0x7fd18415a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:57.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.220+0000 7fd18bdf9640 1 --2- 172.21.15.98:0/4011450290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd18406c1d0 0x7fd184143ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:57.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.220+0000 7fd18bdf9640 1 -- 172.21.15.98:0/4011450290 >> 172.21.15.98:0/4011450290 conn(0x7fd1840b2010 msgr2=0x7fd1840b3920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:57.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.220+0000 7fd18bdf9640 1 -- 172.21.15.98:0/4011450290 shutdown_connections 2024-09-19T08:43:57.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.220+0000 7fd18bdf9640 1 -- 172.21.15.98:0/4011450290 wait complete. 2024-09-19T08:43:57.245 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-19T08:43:57.245 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:43:57.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.368+0000 7f5bee89c640 1 Processor -- start 2024-09-19T08:43:57.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.368+0000 7f5bee89c640 1 -- start start 2024-09-19T08:43:57.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.372+0000 7f5bee89c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5be80d4820 0x7f5be81621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:57.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.372+0000 7f5bee89c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5be80b9970 0x7f5be80b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:57.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.372+0000 7f5bee89c640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5be8068440 con 0x7f5be80b7f80 2024-09-19T08:43:57.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.372+0000 7f5bee89c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f5be8163050 con 0x7f5be80d4820 2024-09-19T08:43:57.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.372+0000 7f5bee89c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5be8162a90 con 0x7f5be80b9970 2024-09-19T08:43:57.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.372+0000 7f5be7fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f5be80b7f80 0x7f5be815f560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:49420/0 (socket says 172.21.15.98:49420) 2024-09-19T08:43:57.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.372+0000 7f5be7fff640 1 -- 172.21.15.98:0/202835862 learned_addr learned my addr 172.21.15.98:0/202835862 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:57.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.372+0000 7f5bece39640 1 --2- 172.21.15.98:0/202835862 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5be80b9970 0x7f5be80b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:57.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.372+0000 7f5be6ffd640 1 -- 172.21.15.98:0/202835862 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 782296032 0 0) 0x7f5be8068440 con 0x7f5be80b7f80 2024-09-19T08:43:57.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.372+0000 7f5be6ffd640 1 -- 172.21.15.98:0/202835862 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5bd0003530 con 0x7f5be80b7f80 2024-09-19T08:43:57.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.372+0000 7f5be77fe640 1 --2- 172.21.15.98:0/202835862 >> v2:172.21.15.139:3300/0 conn(0x7f5be80d4820 0x7f5be81621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:57.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.372+0000 7f5be6ffd640 1 -- 172.21.15.98:0/202835862 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3352555173 0 0) 0x7f5bd0003530 con 0x7f5be80b7f80 2024-09-19T08:43:57.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.372+0000 7f5be6ffd640 1 -- 172.21.15.98:0/202835862 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5be80b9970 msgr2=0x7f5be80b9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:57.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.372+0000 7f5be6ffd640 1 --2- 172.21.15.98:0/202835862 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5be80b9970 0x7f5be80b9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:57.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.372+0000 7f5be6ffd640 1 -- 172.21.15.98:0/202835862 >> v2:172.21.15.139:3300/0 conn(0x7f5be80d4820 msgr2=0x7f5be81621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:57.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.372+0000 7f5be6ffd640 1 --2- 172.21.15.98:0/202835862 >> v2:172.21.15.139:3300/0 conn(0x7f5be80d4820 0x7f5be81621b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:57.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.372+0000 7f5be6ffd640 1 -- 172.21.15.98:0/202835862 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be8162d70 con 0x7f5be80b7f80 2024-09-19T08:43:57.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.372+0000 7f5be6ffd640 1 -- 172.21.15.98:0/202835862 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f5bdc0016a0 con 0x7f5be80b7f80 2024-09-19T08:43:57.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.372+0000 7f5be6ffd640 1 -- 172.21.15.98:0/202835862 >> v1:172.21.15.98:6789/0 conn(0x7f5be80b7f80 legacy=0x7f5be815f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:57.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.372+0000 7f5be6ffd640 1 --2- 172.21.15.98:0/202835862 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5be80b9970 0x7f5bd00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:57.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.372+0000 7f5be6ffd640 1 --2- 172.21.15.98:0/202835862 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5bd0003f40 0x7f5bd0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:57.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.372+0000 7f5be6ffd640 1 --2- 172.21.15.98:0/202835862 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5bd0004d50 0x7f5bd00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:57.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.372+0000 7f5be6ffd640 1 -- 172.21.15.98:0/202835862 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5be8163050 con 0x7f5bd0003f40 2024-09-19T08:43:57.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.372+0000 7f5be6ffd640 1 -- 172.21.15.98:0/202835862 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5be8162a90 con 0x7f5bd0004d50 2024-09-19T08:43:57.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.372+0000 7f5be6ffd640 1 -- 172.21.15.98:0/202835862 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5bd000ba20 con 0x7f5be80b9970 2024-09-19T08:43:57.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.372+0000 7f5be77fe640 1 --2- 172.21.15.98:0/202835862 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5bd0003f40 0x7f5bd0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:57.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.372+0000 7f5bece39640 1 --2- 172.21.15.98:0/202835862 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5bd0004d50 0x7f5bd00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:57.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.372+0000 7f5be77fe640 1 -- 172.21.15.98:0/202835862 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5be80b9970 msgr2=0x7f5bd00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:57.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.372+0000 7f5be77fe640 1 --2- 172.21.15.98:0/202835862 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5be80b9970 0x7f5bd00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:57.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.372+0000 7f5be77fe640 1 -- 172.21.15.98:0/202835862 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5bd0004d50 msgr2=0x7f5bd00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:57.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.372+0000 7f5be77fe640 1 --2- 172.21.15.98:0/202835862 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5bd0004d50 0x7f5bd00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:57.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.372+0000 7f5be77fe640 1 -- 172.21.15.98:0/202835862 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd0003530 con 0x7f5bd0003f40 2024-09-19T08:43:57.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.372+0000 7f5bece39640 1 --2- 172.21.15.98:0/202835862 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5bd0004d50 0x7f5bd00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:57.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.376+0000 7f5be77fe640 1 --2- 172.21.15.98:0/202835862 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5bd0003f40 0x7f5bd0004810 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f5bd800ba30 tx=0x7f5bd801c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee26b20f2450ebc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:57.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.376+0000 7f5be6ffd640 1 -- 172.21.15.98:0/202835862 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd800b600 con 0x7f5bd0003f40 2024-09-19T08:43:57.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.376+0000 7f5be6ffd640 1 -- 172.21.15.98:0/202835862 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bd80aa650 con 0x7f5bd0003f40 2024-09-19T08:43:57.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.376+0000 7f5be6ffd640 1 -- 172.21.15.98:0/202835862 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd80aaa40 con 0x7f5bd0003f40 2024-09-19T08:43:57.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.376+0000 7f5bee89c640 1 -- 172.21.15.98:0/202835862 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5bd0003f40 msgr2=0x7f5bd0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:57.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.376+0000 7f5bee89c640 1 --2- 172.21.15.98:0/202835862 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5bd0003f40 0x7f5bd0004810 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f5bd800ba30 tx=0x7f5bd801c6e0 comp rx=0 tx=0).stop 2024-09-19T08:43:57.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.376+0000 7f5bece39640 1 -- 172.21.15.98:0/202835862 reap_dead start 2024-09-19T08:43:57.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.376+0000 7f5bee89c640 1 -- 172.21.15.98:0/202835862 shutdown_connections 2024-09-19T08:43:57.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.376+0000 7f5bee89c640 1 -- 172.21.15.98:0/202835862 >> 172.21.15.98:0/202835862 conn(0x7f5be8074e40 msgr2=0x7f5be8075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:57.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.376+0000 7f5bee89c640 1 -- 172.21.15.98:0/202835862 shutdown_connections 2024-09-19T08:43:57.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.376+0000 7f5bee89c640 1 -- 172.21.15.98:0/202835862 wait complete. 2024-09-19T08:43:57.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.376+0000 7f5bee89c640 1 Processor -- start 2024-09-19T08:43:57.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.376+0000 7f5bee89c640 1 -- start start 2024-09-19T08:43:57.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.376+0000 7f5bee89c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5be80d04a0 0x7f5be81f7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:57.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.376+0000 7f5bee89c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5be80ca110 0x7f5be81f8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:57.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.376+0000 7f5bee89c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5be81f88d0 0x7f5be80caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:57.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.376+0000 7f5bee89c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5be8162870 con 0x7f5be80ca110 2024-09-19T08:43:57.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.376+0000 7f5bee89c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5be81626f0 con 0x7f5be81f88d0 2024-09-19T08:43:57.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.376+0000 7f5bee89c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5be80b8360 con 0x7f5be80d04a0 2024-09-19T08:43:57.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.376+0000 7f5be77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5be80ca110 0x7f5be81f8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:57.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.376+0000 7f5be77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5be80ca110 0x7f5be81f8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:33114/0 (socket says 172.21.15.98:33114) 2024-09-19T08:43:57.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.376+0000 7f5be77fe640 1 -- 172.21.15.98:0/2846672893 learned_addr learned my addr 172.21.15.98:0/2846672893 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:57.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.376+0000 7f5be7fff640 1 --2- 172.21.15.98:0/2846672893 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5be80d04a0 0x7f5be81f7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:57.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.376+0000 7f5bece39640 1 --2- 172.21.15.98:0/2846672893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5be81f88d0 0x7f5be80caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:57.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.376+0000 7f5be77fe640 1 -- 172.21.15.98:0/2846672893 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5be80d04a0 msgr2=0x7f5be81f7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:57.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.376+0000 7f5be77fe640 1 --2- 172.21.15.98:0/2846672893 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5be80d04a0 0x7f5be81f7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:57.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.376+0000 7f5be77fe640 1 -- 172.21.15.98:0/2846672893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5be81f88d0 msgr2=0x7f5be80caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:57.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.376+0000 7f5be77fe640 1 --2- 172.21.15.98:0/2846672893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5be81f88d0 0x7f5be80caaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:57.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.376+0000 7f5be77fe640 1 -- 172.21.15.98:0/2846672893 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be80ca680 con 0x7f5be80ca110 2024-09-19T08:43:57.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.376+0000 7f5be77fe640 1 --2- 172.21.15.98:0/2846672893 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5be80ca110 0x7f5be81f8390 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f5bd801cbc0 tx=0x7f5bd0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:57.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.376+0000 7f5be4ff9640 1 -- 172.21.15.98:0/2846672893 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd80aae60 con 0x7f5be80ca110 2024-09-19T08:43:57.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.380+0000 7f5be4ff9640 1 -- 172.21.15.98:0/2846672893 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bd80a8630 con 0x7f5be80ca110 2024-09-19T08:43:57.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.380+0000 7f5bee89c640 1 -- 172.21.15.98:0/2846672893 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be80cccf0 con 0x7f5be80ca110 2024-09-19T08:43:57.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.380+0000 7f5be4ff9640 1 -- 172.21.15.98:0/2846672893 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd800b600 con 0x7f5be80ca110 2024-09-19T08:43:57.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.380+0000 7f5bee89c640 1 -- 172.21.15.98:0/2846672893 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be80cd2b0 con 0x7f5be80ca110 2024-09-19T08:43:57.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.380+0000 7f5bee89c640 1 -- 172.21.15.98:0/2846672893 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5be81621b0 con 0x7f5be80ca110 2024-09-19T08:43:57.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.380+0000 7f5be4ff9640 1 -- 172.21.15.98:0/2846672893 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f5bd80a8a00 con 0x7f5be80ca110 2024-09-19T08:43:57.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.380+0000 7f5be4ff9640 1 --2- 172.21.15.98:0/2846672893 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5bd40679e0 0x7f5bd4069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:57.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.380+0000 7f5be4ff9640 1 -- 172.21.15.98:0/2846672893 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f5bd81205a0 con 0x7f5be80ca110 2024-09-19T08:43:57.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.380+0000 7f5be7fff640 1 --2- 172.21.15.98:0/2846672893 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5bd40679e0 0x7f5bd4069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:57.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.380+0000 7f5be7fff640 1 --2- 172.21.15.98:0/2846672893 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5bd40679e0 0x7f5bd4069ea0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc001850 tx=0x7f5bdc001700 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:57.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.384+0000 7f5be4ff9640 1 -- 172.21.15.98:0/2846672893 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f5bd80c1060 con 0x7f5be80ca110 2024-09-19T08:43:57.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.580+0000 7f5bee89c640 1 -- 172.21.15.98:0/2846672893 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5be80d26a0 con 0x7f5be80ca110 2024-09-19T08:43:57.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.580+0000 7f5be4ff9640 1 -- 172.21.15.98:0/2846672893 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+5225 (secure 0 0 0) 0x7f5bd80ae020 con 0x7f5be80ca110 2024-09-19T08:43:57.584 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:57.584 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":60,"btime":"2024-09-19T08:43:57:218267+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6004,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3739962222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3739962222},{"type":"v1","addr":"172.21.15.139:6833","nonce":3739962222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6007,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/421473653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":421473653},{"type":"v1","addr":"172.21.15.139:6835","nonce":421473653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3126941916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3126941916},{"type":"v1","addr":"172.21.15.98:6833","nonce":3126941916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:43:54.489407+0000","modified":"2024-09-19T08:43:56.577758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6996},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6996":{"gid":6996,"name":"c","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/273091170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":273091170},{"type":"v1","addr":"172.21.15.98:6835","nonce":273091170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6996,"qdb_cluster":[6996]},"id":8}]} 2024-09-19T08:43:57.584 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 60 2024-09-19T08:43:57.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.584+0000 7f5bee89c640 1 -- 172.21.15.98:0/2846672893 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5bd40679e0 msgr2=0x7f5bd4069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:57.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.584+0000 7f5bee89c640 1 --2- 172.21.15.98:0/2846672893 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5bd40679e0 0x7f5bd4069ea0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc001850 tx=0x7f5bdc001700 comp rx=0 tx=0).stop 2024-09-19T08:43:57.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.584+0000 7f5bee89c640 1 -- 172.21.15.98:0/2846672893 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5be80ca110 msgr2=0x7f5be81f8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:57.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.584+0000 7f5bee89c640 1 --2- 172.21.15.98:0/2846672893 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5be80ca110 0x7f5be81f8390 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f5bd801cbc0 tx=0x7f5bd0005840 comp rx=0 tx=0).stop 2024-09-19T08:43:57.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.584+0000 7f5bee89c640 1 -- 172.21.15.98:0/2846672893 shutdown_connections 2024-09-19T08:43:57.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.584+0000 7f5bee89c640 1 --2- 172.21.15.98:0/2846672893 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5bd40679e0 0x7f5bd4069ea0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:57.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.584+0000 7f5bee89c640 1 --2- 172.21.15.98:0/2846672893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5be81f88d0 0x7f5be80caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:57.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.584+0000 7f5bee89c640 1 --2- 172.21.15.98:0/2846672893 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5be80ca110 0x7f5be81f8390 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:57.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.584+0000 7f5bee89c640 1 --2- 172.21.15.98:0/2846672893 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5be80d04a0 0x7f5be81f7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:57.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.584+0000 7f5bee89c640 1 -- 172.21.15.98:0/2846672893 >> 172.21.15.98:0/2846672893 conn(0x7f5be8074e40 msgr2=0x7f5be80b9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:57.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.584+0000 7f5bee89c640 1 -- 172.21.15.98:0/2846672893 shutdown_connections 2024-09-19T08:43:57.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.584+0000 7f5bee89c640 1 -- 172.21.15.98:0/2846672893 wait complete. 2024-09-19T08:43:57.603 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:43:57.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.712+0000 7f869bd75640 1 Processor -- start 2024-09-19T08:43:57.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.712+0000 7f869bd75640 1 -- start start 2024-09-19T08:43:57.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.712+0000 7f869bd75640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f869415c550 0x7f869406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:57.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.712+0000 7f869bd75640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f869406c1d0 0x7f86940cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:57.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.712+0000 7f869bd75640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f869406d310 con 0x7f86940cf900 2024-09-19T08:43:57.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.712+0000 7f869bd75640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f869406c640 con 0x7f869415c550 2024-09-19T08:43:57.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.712+0000 7f869bd75640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f869406d5f0 con 0x7f869406c1d0 2024-09-19T08:43:57.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.712+0000 7f8699b11640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f869415c550 0x7f869406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:57.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.712+0000 7f869a312640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f86940cf900 0x7f86940ce1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:33164/0 (socket says 172.21.15.98:33164) 2024-09-19T08:43:57.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.712+0000 7f869a312640 1 -- 172.21.15.98:0/1386255773 learned_addr learned my addr 172.21.15.98:0/1386255773 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:57.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.712+0000 7f8699b11640 1 -- 172.21.15.98:0/1386255773 >> v2:172.21.15.139:3300/0 conn(0x7f869406c1d0 msgr2=0x7f86940cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:57.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.712+0000 7f8699b11640 1 --2- 172.21.15.98:0/1386255773 >> v2:172.21.15.139:3300/0 conn(0x7f869406c1d0 0x7f86940cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:57.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.712+0000 7f8699b11640 1 -- 172.21.15.98:0/1386255773 >> v1:172.21.15.98:6789/0 conn(0x7f86940cf900 legacy=0x7f86940ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:57.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.712+0000 7f8698b0f640 1 -- 172.21.15.98:0/1386255773 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 343973017 0 0) 0x7f869406d310 con 0x7f86940cf900 2024-09-19T08:43:57.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.712+0000 7f8699b11640 1 -- 172.21.15.98:0/1386255773 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f869406d8d0 con 0x7f869415c550 2024-09-19T08:43:57.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f8699b11640 1 --2- 172.21.15.98:0/1386255773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f869415c550 0x7f869406bc90 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f86940ccc60 tx=0x7f869001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322c51892e89635d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:57.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f8698b0f640 1 -- 172.21.15.98:0/1386255773 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86900b2070 con 0x7f869415c550 2024-09-19T08:43:57.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f8698b0f640 1 -- 172.21.15.98:0/1386255773 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86900a5810 con 0x7f869415c550 2024-09-19T08:43:57.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f8698b0f640 1 -- 172.21.15.98:0/1386255773 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86900a5be0 con 0x7f869415c550 2024-09-19T08:43:57.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f869bd75640 1 -- 172.21.15.98:0/1386255773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f869415c550 msgr2=0x7f869406bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:57.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f869bd75640 1 --2- 172.21.15.98:0/1386255773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f869415c550 0x7f869406bc90 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f86940ccc60 tx=0x7f869001c950 comp rx=0 tx=0).stop 2024-09-19T08:43:57.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f869bd75640 1 -- 172.21.15.98:0/1386255773 shutdown_connections 2024-09-19T08:43:57.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f869bd75640 1 --2- 172.21.15.98:0/1386255773 >> v2:172.21.15.139:3300/0 conn(0x7f869406c1d0 0x7f86940cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:57.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f869bd75640 1 --2- 172.21.15.98:0/1386255773 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f869415c550 0x7f869406bc90 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:57.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f869bd75640 1 -- 172.21.15.98:0/1386255773 >> 172.21.15.98:0/1386255773 conn(0x7f86940b2010 msgr2=0x7f86940b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:57.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f869bd75640 1 -- 172.21.15.98:0/1386255773 shutdown_connections 2024-09-19T08:43:57.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f869bd75640 1 -- 172.21.15.98:0/1386255773 wait complete. 2024-09-19T08:43:57.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f869bd75640 1 Processor -- start 2024-09-19T08:43:57.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f869bd75640 1 -- start start 2024-09-19T08:43:57.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f869bd75640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f869406c1d0 0x7f869414d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:57.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f869bd75640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86940cf900 0x7f869413dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:57.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f869bd75640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f869415c550 0x7f869414ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:57.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f869bd75640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f86941693b0 con 0x7f869415c550 2024-09-19T08:43:57.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f869bd75640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8694169230 con 0x7f86940cf900 2024-09-19T08:43:57.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f869bd75640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8694169530 con 0x7f869406c1d0 2024-09-19T08:43:57.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f869a312640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f869415c550 0x7f869414ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:57.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f869a312640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f869415c550 0x7f869414ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:50694/0 (socket says 172.21.15.98:50694) 2024-09-19T08:43:57.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f869a312640 1 -- 172.21.15.98:0/3202282951 learned_addr learned my addr 172.21.15.98:0/3202282951 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:57.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f8699310640 1 --2- 172.21.15.98:0/3202282951 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86940cf900 0x7f869413dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:57.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f8699b11640 1 --2- 172.21.15.98:0/3202282951 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f869406c1d0 0x7f869414d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:57.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f869a312640 1 -- 172.21.15.98:0/3202282951 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f869406c1d0 msgr2=0x7f869414d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:57.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f869a312640 1 --2- 172.21.15.98:0/3202282951 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f869406c1d0 0x7f869414d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:57.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f869a312640 1 -- 172.21.15.98:0/3202282951 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86940cf900 msgr2=0x7f869413dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:57.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f869a312640 1 --2- 172.21.15.98:0/3202282951 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86940cf900 0x7f869413dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:57.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f869a312640 1 -- 172.21.15.98:0/3202282951 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f869414ddd0 con 0x7f869415c550 2024-09-19T08:43:57.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f8699310640 1 --2- 172.21.15.98:0/3202282951 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86940cf900 0x7f869413dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:43:57.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f869a312640 1 --2- 172.21.15.98:0/3202282951 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f869415c550 0x7f869414ee70 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f868c00c4b0 tx=0x7f868c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:57.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f868a7fc640 1 -- 172.21.15.98:0/3202282951 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868c018070 con 0x7f869415c550 2024-09-19T08:43:57.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f868a7fc640 1 -- 172.21.15.98:0/3202282951 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f868c0074e0 con 0x7f869415c550 2024-09-19T08:43:57.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f869bd75640 1 -- 172.21.15.98:0/3202282951 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f869414e0b0 con 0x7f869415c550 2024-09-19T08:43:57.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f868a7fc640 1 -- 172.21.15.98:0/3202282951 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868c004a30 con 0x7f869415c550 2024-09-19T08:43:57.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.716+0000 7f869bd75640 1 -- 172.21.15.98:0/3202282951 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f869414e670 con 0x7f869415c550 2024-09-19T08:43:57.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.720+0000 7f869bd75640 1 -- 172.21.15.98:0/3202282951 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86940cf3c0 con 0x7f869415c550 2024-09-19T08:43:57.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.720+0000 7f868a7fc640 1 -- 172.21.15.98:0/3202282951 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f868c01b070 con 0x7f869415c550 2024-09-19T08:43:57.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.720+0000 7f868a7fc640 1 --2- 172.21.15.98:0/3202282951 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f86780679e0 0x7f8678069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:57.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.720+0000 7f868a7fc640 1 -- 172.21.15.98:0/3202282951 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f868c085090 con 0x7f869415c550 2024-09-19T08:43:57.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.720+0000 7f8699b11640 1 --2- 172.21.15.98:0/3202282951 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f86780679e0 0x7f8678069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:57.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.720+0000 7f8699b11640 1 --2- 172.21.15.98:0/3202282951 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f86780679e0 0x7f8678069ea0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f86900af000 tx=0x7f86900a7070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:57.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.724+0000 7f868a7fc640 1 -- 172.21.15.98:0/3202282951 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f868c055ac0 con 0x7f869415c550 2024-09-19T08:43:57.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.872+0000 7f869bd75640 1 -- 172.21.15.98:0/3202282951 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8694153520 con 0x7f869415c550 2024-09-19T08:43:57.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.876+0000 7f868a7fc640 1 -- 172.21.15.98:0/3202282951 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+16687 (secure 0 0 0) 0x7f868c0553c0 con 0x7f869415c550 2024-09-19T08:43:57.877 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:57.877 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":102,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:43:56.149959+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:43:52.161395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-19T08:43:53.497026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:43:57.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.876+0000 7f869bd75640 1 -- 172.21.15.98:0/3202282951 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f86780679e0 msgr2=0x7f8678069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:57.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.876+0000 7f869bd75640 1 --2- 172.21.15.98:0/3202282951 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f86780679e0 0x7f8678069ea0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f86900af000 tx=0x7f86900a7070 comp rx=0 tx=0).stop 2024-09-19T08:43:57.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.876+0000 7f869bd75640 1 -- 172.21.15.98:0/3202282951 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f869415c550 msgr2=0x7f869414ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:57.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.876+0000 7f869bd75640 1 --2- 172.21.15.98:0/3202282951 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f869415c550 0x7f869414ee70 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f868c00c4b0 tx=0x7f868c00c970 comp rx=0 tx=0).stop 2024-09-19T08:43:57.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.876+0000 7f869bd75640 1 -- 172.21.15.98:0/3202282951 shutdown_connections 2024-09-19T08:43:57.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.876+0000 7f869bd75640 1 --2- 172.21.15.98:0/3202282951 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f86780679e0 0x7f8678069ea0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:57.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.876+0000 7f869bd75640 1 --2- 172.21.15.98:0/3202282951 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f869415c550 0x7f869414ee70 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:57.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.876+0000 7f869bd75640 1 --2- 172.21.15.98:0/3202282951 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86940cf900 0x7f869413dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:57.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.876+0000 7f869bd75640 1 --2- 172.21.15.98:0/3202282951 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f869406c1d0 0x7f869414d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:57.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.876+0000 7f869bd75640 1 -- 172.21.15.98:0/3202282951 >> 172.21.15.98:0/3202282951 conn(0x7f86940b2010 msgr2=0x7f8694153a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:57.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.876+0000 7f869bd75640 1 -- 172.21.15.98:0/3202282951 shutdown_connections 2024-09-19T08:43:57.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:57.876+0000 7f869bd75640 1 -- 172.21.15.98:0/3202282951 wait complete. 2024-09-19T08:43:57.895 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T08:43:57.895 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:43:58.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.000+0000 7f64c4992640 1 Processor -- start 2024-09-19T08:43:58.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.000+0000 7f64c4992640 1 -- start start 2024-09-19T08:43:58.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.004+0000 7f64c4992640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64c0140d30 0x7f64c006d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:58.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.004+0000 7f64c4992640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64c015ea10 con 0x7f64c00b3a20 2024-09-19T08:43:58.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.004+0000 7f64c4992640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64c015ecf0 con 0x7f64c0141110 2024-09-19T08:43:58.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.004+0000 7f64c4992640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f64c015e7c0 con 0x7f64c0140d30 2024-09-19T08:43:58.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.004+0000 7f64bed9d640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f64c00b3a20 0x7f64c015b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:33170/0 (socket says 172.21.15.98:33170) 2024-09-19T08:43:58.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.004+0000 7f64bed9d640 1 -- 172.21.15.98:0/2972040251 learned_addr learned my addr 172.21.15.98:0/2972040251 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:43:58.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.004+0000 7f64bdd9b640 1 -- 172.21.15.98:0/2972040251 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1956372992 0 0) 0x7f64c015ea10 con 0x7f64c00b3a20 2024-09-19T08:43:58.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.004+0000 7f64bdd9b640 1 -- 172.21.15.98:0/2972040251 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64ac003530 con 0x7f64c00b3a20 2024-09-19T08:43:58.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.004+0000 7f64bdd9b640 1 -- 172.21.15.98:0/2972040251 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2142024225 0 0) 0x7f64ac003530 con 0x7f64c00b3a20 2024-09-19T08:43:58.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.004+0000 7f64bdd9b640 1 -- 172.21.15.98:0/2972040251 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64c0140d30 msgr2=0x7f64c006d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:58.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.004+0000 7f64bdd9b640 1 --2- 172.21.15.98:0/2972040251 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64c0140d30 0x7f64c006d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.004+0000 7f64bdd9b640 1 -- 172.21.15.98:0/2972040251 >> v1:172.21.15.139:6789/0 conn(0x7f64c0141110 legacy=0x7f64c00b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:58.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.004+0000 7f64bdd9b640 1 -- 172.21.15.98:0/2972040251 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64c015efd0 con 0x7f64c00b3a20 2024-09-19T08:43:58.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.004+0000 7f64bdd9b640 1 -- 172.21.15.98:0/2972040251 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f64b8001a00 con 0x7f64c00b3a20 2024-09-19T08:43:58.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.004+0000 7f64bdd9b640 1 -- 172.21.15.98:0/2972040251 >> v1:172.21.15.98:6789/0 conn(0x7f64c00b3a20 legacy=0x7f64c015b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:58.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.004+0000 7f64bdd9b640 1 --2- 172.21.15.98:0/2972040251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64ac0039f0 0x7f64ac003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:58.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.004+0000 7f64bdd9b640 1 --2- 172.21.15.98:0/2972040251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64ac004810 0x7f64ac004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:58.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.004+0000 7f64bdd9b640 1 --2- 172.21.15.98:0/2972040251 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64c0140d30 0x7f64ac00cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:58.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.004+0000 7f64bdd9b640 1 -- 172.21.15.98:0/2972040251 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f64c015e7c0 con 0x7f64ac004810 2024-09-19T08:43:58.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.004+0000 7f64bdd9b640 1 -- 172.21.15.98:0/2972040251 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f64ac005530 con 0x7f64ac0039f0 2024-09-19T08:43:58.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.004+0000 7f64bdd9b640 1 -- 172.21.15.98:0/2972040251 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f64ac00d020 con 0x7f64c0140d30 2024-09-19T08:43:58.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.004+0000 7f64b7fff640 1 --2- 172.21.15.98:0/2972040251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64ac004810 0x7f64ac004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:58.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.004+0000 7f64be59c640 1 --2- 172.21.15.98:0/2972040251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64ac0039f0 0x7f64ac003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:58.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.004+0000 7f64b7fff640 1 -- 172.21.15.98:0/2972040251 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64c0140d30 msgr2=0x7f64ac00cae0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T08:43:58.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.004+0000 7f64b7fff640 1 --2- 172.21.15.98:0/2972040251 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64c0140d30 0x7f64ac00cae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.004+0000 7f64b7fff640 1 -- 172.21.15.98:0/2972040251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64ac0039f0 msgr2=0x7f64ac003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:58.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.004+0000 7f64b7fff640 1 --2- 172.21.15.98:0/2972040251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64ac0039f0 0x7f64ac003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.004+0000 7f64b7fff640 1 -- 172.21.15.98:0/2972040251 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64ac003530 con 0x7f64ac004810 2024-09-19T08:43:58.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.008+0000 7f64b7fff640 1 --2- 172.21.15.98:0/2972040251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64ac004810 0x7f64ac004c90 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f64c00d1340 tx=0x7f64a801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=559dd57083d9a270 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:58.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.008+0000 7f64bdd9b640 1 -- 172.21.15.98:0/2972040251 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a80012b0 con 0x7f64ac004810 2024-09-19T08:43:58.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.008+0000 7f64bdd9b640 1 -- 172.21.15.98:0/2972040251 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64a8005270 con 0x7f64ac004810 2024-09-19T08:43:58.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.008+0000 7f64bdd9b640 1 -- 172.21.15.98:0/2972040251 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a8003620 con 0x7f64ac004810 2024-09-19T08:43:58.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.008+0000 7f64c4992640 1 -- 172.21.15.98:0/2972040251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64ac004810 msgr2=0x7f64ac004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:58.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.008+0000 7f64c4992640 1 --2- 172.21.15.98:0/2972040251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64ac004810 0x7f64ac004c90 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f64c00d1340 tx=0x7f64a801c260 comp rx=0 tx=0).stop 2024-09-19T08:43:58.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.008+0000 7f64bed9d640 1 -- 172.21.15.98:0/2972040251 reap_dead start 2024-09-19T08:43:58.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.008+0000 7f64c4992640 1 -- 172.21.15.98:0/2972040251 shutdown_connections 2024-09-19T08:43:58.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.008+0000 7f64c4992640 1 -- 172.21.15.98:0/2972040251 >> 172.21.15.98:0/2972040251 conn(0x7f64c00b2010 msgr2=0x7f64c00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:58.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.008+0000 7f64c4992640 1 -- 172.21.15.98:0/2972040251 shutdown_connections 2024-09-19T08:43:58.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.008+0000 7f64c4992640 1 -- 172.21.15.98:0/2972040251 wait complete. 2024-09-19T08:43:58.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.008+0000 7f64c4992640 1 Processor -- start 2024-09-19T08:43:58.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.008+0000 7f64c4992640 1 -- start start 2024-09-19T08:43:58.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.008+0000 7f64c4992640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64ac004810 0x7f64c01f5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:58.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.008+0000 7f64c4992640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64c00cbac0 0x7f64c0072c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:58.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.008+0000 7f64c4992640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64c00cbf40 0x7f64c0157d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:58.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.008+0000 7f64c4992640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f64c006bc90 con 0x7f64ac004810 2024-09-19T08:43:58.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.008+0000 7f64c4992640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f64c00ce1a0 con 0x7f64c00cbf40 2024-09-19T08:43:58.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.008+0000 7f64c4992640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f64c006be10 con 0x7f64c00cbac0 2024-09-19T08:43:58.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.008+0000 7f64be59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64ac004810 0x7f64c01f5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:58.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.008+0000 7f64be59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64ac004810 0x7f64c01f5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:50720/0 (socket says 172.21.15.98:50720) 2024-09-19T08:43:58.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.008+0000 7f64be59c640 1 -- 172.21.15.98:0/738256072 learned_addr learned my addr 172.21.15.98:0/738256072 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:58.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.008+0000 7f64be59c640 1 -- 172.21.15.98:0/738256072 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64c00cbac0 msgr2=0x7f64c0072c50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T08:43:58.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.008+0000 7f64be59c640 1 --2- 172.21.15.98:0/738256072 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64c00cbac0 0x7f64c0072c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.008+0000 7f64be59c640 1 -- 172.21.15.98:0/738256072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64c00cbf40 msgr2=0x7f64c0157d00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T08:43:58.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.008+0000 7f64be59c640 1 --2- 172.21.15.98:0/738256072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64c00cbf40 0x7f64c0157d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.008+0000 7f64be59c640 1 -- 172.21.15.98:0/738256072 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64c00aa620 con 0x7f64ac004810 2024-09-19T08:43:58.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.008+0000 7f64be59c640 1 --2- 172.21.15.98:0/738256072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64ac004810 0x7f64c01f5430 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f64b000d5a0 tx=0x7f64b000da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:58.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.008+0000 7f64b6ffd640 1 -- 172.21.15.98:0/738256072 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b0018020 con 0x7f64ac004810 2024-09-19T08:43:58.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.008+0000 7f64b6ffd640 1 -- 172.21.15.98:0/738256072 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64b0014620 con 0x7f64ac004810 2024-09-19T08:43:58.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.008+0000 7f64b6ffd640 1 -- 172.21.15.98:0/738256072 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b00149d0 con 0x7f64ac004810 2024-09-19T08:43:58.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.008+0000 7f64c4992640 1 -- 172.21.15.98:0/738256072 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64c00aa900 con 0x7f64ac004810 2024-09-19T08:43:58.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.008+0000 7f64c4992640 1 -- 172.21.15.98:0/738256072 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64c00aaec0 con 0x7f64ac004810 2024-09-19T08:43:58.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.012+0000 7f64c4992640 1 -- 172.21.15.98:0/738256072 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64c006c1f0 con 0x7f64ac004810 2024-09-19T08:43:58.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.012+0000 7f64b6ffd640 1 -- 172.21.15.98:0/738256072 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f64b001b020 con 0x7f64ac004810 2024-09-19T08:43:58.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.012+0000 7f64b6ffd640 1 --2- 172.21.15.98:0/738256072 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f64a00679e0 0x7f64a0069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:58.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.012+0000 7f64b6ffd640 1 -- 172.21.15.98:0/738256072 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f64b0085ec0 con 0x7f64ac004810 2024-09-19T08:43:58.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.012+0000 7f64b7fff640 1 --2- 172.21.15.98:0/738256072 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f64a00679e0 0x7f64a0069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:58.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.012+0000 7f64b7fff640 1 --2- 172.21.15.98:0/738256072 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f64a00679e0 0x7f64a0069ea0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f64c006e040 tx=0x7f64a8001050 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:58.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.016+0000 7f64b6ffd640 1 -- 172.21.15.98:0/738256072 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f64b0022030 con 0x7f64ac004810 2024-09-19T08:43:58.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.176+0000 7f64c4992640 1 -- 172.21.15.98:0/738256072 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f64c0159a50 con 0x7f64a00679e0 2024-09-19T08:43:58.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.192+0000 7f64b6ffd640 1 -- 172.21.15.98:0/738256072 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240183 (secure 0 0 0) 0x7f64c0159a50 con 0x7f64a00679e0 2024-09-19T08:43:58.196 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:58.198 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:43:58.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.196+0000 7f64c4992640 1 -- 172.21.15.98:0/738256072 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f64a00679e0 msgr2=0x7f64a0069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:58.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.196+0000 7f64c4992640 1 --2- 172.21.15.98:0/738256072 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f64a00679e0 0x7f64a0069ea0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f64c006e040 tx=0x7f64a8001050 comp rx=0 tx=0).stop 2024-09-19T08:43:58.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.196+0000 7f64c4992640 1 -- 172.21.15.98:0/738256072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64ac004810 msgr2=0x7f64c01f5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:58.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.196+0000 7f64c4992640 1 --2- 172.21.15.98:0/738256072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64ac004810 0x7f64c01f5430 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f64b000d5a0 tx=0x7f64b000da60 comp rx=0 tx=0).stop 2024-09-19T08:43:58.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.196+0000 7f64c4992640 1 -- 172.21.15.98:0/738256072 shutdown_connections 2024-09-19T08:43:58.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.196+0000 7f64c4992640 1 --2- 172.21.15.98:0/738256072 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f64a00679e0 0x7f64a0069ea0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.196+0000 7f64c4992640 1 --2- 172.21.15.98:0/738256072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f64c00cbf40 0x7f64c0157d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.196+0000 7f64c4992640 1 --2- 172.21.15.98:0/738256072 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f64c00cbac0 0x7f64c0072c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.196+0000 7f64c4992640 1 --2- 172.21.15.98:0/738256072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f64ac004810 0x7f64c01f5430 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.196+0000 7f64c4992640 1 -- 172.21.15.98:0/738256072 >> 172.21.15.98:0/738256072 conn(0x7f64c00b2010 msgr2=0x7f64c00b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:58.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.196+0000 7f64c4992640 1 -- 172.21.15.98:0/738256072 shutdown_connections 2024-09-19T08:43:58.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.196+0000 7f64c4992640 1 -- 172.21.15.98:0/738256072 wait complete. 2024-09-19T08:43:58.213 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2024-09-19T08:43:56.890011+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260232,"kb_used_data":10716,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714488,"statfs":{"total":773094113280,"available":772827635712,"internally_reserved":0,"allocated":10973184,"data_stored":5457071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001163"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.147140+0000","last_change":"2024-09-19T08:43:53.147140+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.147140+0000","last_undegraded":"2024-09-19T08:43:53.147140+0000","last_fullsized":"2024-09-19T08:43:53.147140+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.143133+0000","last_change":"2024-09-19T08:43:53.143133+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.143133+0000","last_undegraded":"2024-09-19T08:43:53.143133+0000","last_fullsized":"2024-09-19T08:43:53.143133+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.142190+0000","last_change":"2024-09-19T08:43:53.142190+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.142190+0000","last_undegraded":"2024-09-19T08:43:53.142190+0000","last_fullsized":"2024-09-19T08:43:53.142190+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.141020+0000","last_change":"2024-09-19T08:43:53.141020+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.141020+0000","last_undegraded":"2024-09-19T08:43:53.141020+0000","last_fullsized":"2024-09-19T08:43:53.141020+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.149299+0000","last_change":"2024-09-19T08:43:53.149299+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.149299+0000","last_undegraded":"2024-09-19T08:43:53.149299+0000","last_fullsized":"2024-09-19T08:43:53.149299+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.138068+0000","last_change":"2024-09-19T08:43:53.138068+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.138068+0000","last_undegraded":"2024-09-19T08:43:53.138068+0000","last_fullsized":"2024-09-19T08:43:53.138068+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.136114+0000","last_change":"2024-09-19T08:43:53.136114+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.136114+0000","last_undegraded":"2024-09-19T08:43:53.136114+0000","last_fullsized":"2024-09-19T08:43:53.136114+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.142751+0000","last_change":"2024-09-19T08:43:53.142751+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.142751+0000","last_undegraded":"2024-09-19T08:43:53.142751+0000","last_fullsized":"2024-09-19T08:43:53.142751+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.140651+0000","last_change":"2024-09-19T08:43:53.140651+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.140651+0000","last_undegraded":"2024-09-19T08:43:53.140651+0000","last_fullsized":"2024-09-19T08:43:53.140651+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.138101+0000","last_change":"2024-09-19T08:43:53.138101+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.138101+0000","last_undegraded":"2024-09-19T08:43:53.138101+0000","last_fullsized":"2024-09-19T08:43:53.138101+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.141331+0000","last_change":"2024-09-19T08:43:53.141331+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.141331+0000","last_undegraded":"2024-09-19T08:43:53.141331+0000","last_fullsized":"2024-09-19T08:43:53.141331+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.142992+0000","last_change":"2024-09-19T08:43:53.142992+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.142992+0000","last_undegraded":"2024-09-19T08:43:53.142992+0000","last_fullsized":"2024-09-19T08:43:53.142992+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.138625+0000","last_change":"2024-09-19T08:43:53.138625+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.138625+0000","last_undegraded":"2024-09-19T08:43:53.138625+0000","last_fullsized":"2024-09-19T08:43:53.138625+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.142322+0000","last_change":"2024-09-19T08:43:53.142322+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.142322+0000","last_undegraded":"2024-09-19T08:43:53.142322+0000","last_fullsized":"2024-09-19T08:43:53.142322+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.141433+0000","last_change":"2024-09-19T08:43:53.141433+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.141433+0000","last_undegraded":"2024-09-19T08:43:53.141433+0000","last_fullsized":"2024-09-19T08:43:53.141433+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.141855+0000","last_change":"2024-09-19T08:43:53.141855+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.141855+0000","last_undegraded":"2024-09-19T08:43:53.141855+0000","last_fullsized":"2024-09-19T08:43:53.141855+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.140657+0000","last_change":"2024-09-19T08:43:53.140657+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.140657+0000","last_undegraded":"2024-09-19T08:43:53.140657+0000","last_fullsized":"2024-09-19T08:43:53.140657+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.142608+0000","last_change":"2024-09-19T08:43:53.142608+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.142608+0000","last_undegraded":"2024-09-19T08:43:53.142608+0000","last_fullsized":"2024-09-19T08:43:53.142608+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.147149+0000","last_change":"2024-09-19T08:43:53.147149+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.147149+0000","last_undegraded":"2024-09-19T08:43:53.147149+0000","last_fullsized":"2024-09-19T08:43:53.147149+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.138202+0000","last_change":"2024-09-19T08:43:53.138202+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.138202+0000","last_undegraded":"2024-09-19T08:43:53.138202+0000","last_fullsized":"2024-09-19T08:43:53.138202+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.140693+0000","last_change":"2024-09-19T08:43:53.140693+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.140693+0000","last_undegraded":"2024-09-19T08:43:53.140693+0000","last_fullsized":"2024-09-19T08:43:53.140693+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.138099+0000","last_change":"2024-09-19T08:43:53.138099+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.138099+0000","last_undegraded":"2024-09-19T08:43:53.138099+0000","last_fullsized":"2024-09-19T08:43:53.138099+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.142390+0000","last_change":"2024-09-19T08:43:53.142390+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.142390+0000","last_undegraded":"2024-09-19T08:43:53.142390+0000","last_fullsized":"2024-09-19T08:43:53.142390+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.138074+0000","last_change":"2024-09-19T08:43:53.138074+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.138074+0000","last_undegraded":"2024-09-19T08:43:53.138074+0000","last_fullsized":"2024-09-19T08:43:53.138074+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.142577+0000","last_change":"2024-09-19T08:43:53.142577+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.142577+0000","last_undegraded":"2024-09-19T08:43:53.142577+0000","last_fullsized":"2024-09-19T08:43:53.142577+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.140622+0000","last_change":"2024-09-19T08:43:53.140622+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.140622+0000","last_undegraded":"2024-09-19T08:43:53.140622+0000","last_fullsized":"2024-09-19T08:43:53.140622+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.138475+0000","last_change":"2024-09-19T08:43:53.138475+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.138475+0000","last_undegraded":"2024-09-19T08:43:53.138475+0000","last_fullsized":"2024-09-19T08:43:53.138475+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.147110+0000","last_change":"2024-09-19T08:43:53.147110+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.147110+0000","last_undegraded":"2024-09-19T08:43:53.147110+0000","last_fullsized":"2024-09-19T08:43:53.147110+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.135914+0000","last_change":"2024-09-19T08:43:53.135914+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.135914+0000","last_undegraded":"2024-09-19T08:43:53.135914+0000","last_fullsized":"2024-09-19T08:43:53.135914+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.140942+0000","last_change":"2024-09-19T08:43:53.140942+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.140942+0000","last_undegraded":"2024-09-19T08:43:53.140942+0000","last_fullsized":"2024-09-19T08:43:53.140942+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.140689+0000","last_change":"2024-09-19T08:43:53.140689+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.140689+0000","last_undegraded":"2024-09-19T08:43:53.140689+0000","last_fullsized":"2024-09-19T08:43:53.140689+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.147121+0000","last_change":"2024-09-19T08:43:53.147121+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.147121+0000","last_undegraded":"2024-09-19T08:43:53.147121+0000","last_fullsized":"2024-09-19T08:43:53.147121+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"71'128","reported_seq":349,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T08:43:53.142818+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:43:53.142818+0000","last_peered":"2024-09-19T08:43:53.142818+0000","last_clean":"2024-09-19T08:43:53.142818+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:43:53.142818+0000","last_undegraded":"2024-09-19T08:43:53.142818+0000","last_fullsized":"2024-09-19T08:43:53.142818+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509512,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32964,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338876,"statfs":{"total":96636764160,"available":96603009024,"internally_reserved":0,"allocated":1818624,"data_stored":1125637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1224704,"data_stored":535269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509514,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1224704,"data_stored":535269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509514,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1224704,"data_stored":535269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542216,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1224704,"data_stored":535269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509511,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1224704,"data_stored":535269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509513,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1212416,"data_stored":529452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509511,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32968,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338872,"statfs":{"total":96636764160,"available":96603004928,"internally_reserved":0,"allocated":1818624,"data_stored":1125637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:43:58.218 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:43:58.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.372+0000 7fafb18e0640 1 Processor -- start 2024-09-19T08:43:58.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.372+0000 7fafb18e0640 1 -- start start 2024-09-19T08:43:58.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.376+0000 7fafb18e0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafac158e40 0x7fafac06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:58.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.376+0000 7fafb18e0640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fafac0ce8b0 0x7fafac0b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:58.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.376+0000 7fafb18e0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafac15fcf0 con 0x7fafac06c1d0 2024-09-19T08:43:58.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.376+0000 7fafb18e0640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fafac06c610 con 0x7fafac0ce8b0 2024-09-19T08:43:58.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.376+0000 7fafb18e0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fafac15ef30 con 0x7fafac158e40 2024-09-19T08:43:58.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.376+0000 7fafab7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fafac0ce8b0 0x7fafac0b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:58.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.376+0000 7fafab7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fafac0ce8b0 0x7fafac0b3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:50726/0 (socket says 172.21.15.98:50726) 2024-09-19T08:43:58.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.376+0000 7fafab7fe640 1 -- 172.21.15.98:0/1072217917 learned_addr learned my addr 172.21.15.98:0/1072217917 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:58.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.376+0000 7fafab7fe640 1 -- 172.21.15.98:0/1072217917 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafac158e40 msgr2=0x7fafac06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:58.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.376+0000 7fafaaffd640 1 --2- 172.21.15.98:0/1072217917 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafac158e40 0x7fafac06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:58.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.376+0000 7fafab7fe640 1 --2- 172.21.15.98:0/1072217917 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafac158e40 0x7fafac06bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.376+0000 7fafab7fe640 1 -- 172.21.15.98:0/1072217917 >> v1:172.21.15.139:6789/0 conn(0x7fafac06c1d0 legacy=0x7fafac0ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:58.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.376+0000 7fafab7fe640 1 -- 172.21.15.98:0/1072217917 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafac15f210 con 0x7fafac0ce8b0 2024-09-19T08:43:58.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.380+0000 7fafab7fe640 1 --2- 172.21.15.98:0/1072217917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafac0ce8b0 0x7fafac0b3560 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7faf9c009870 tx=0x7faf9c01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d31ab3bfb1441a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:58.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.380+0000 7fafa9ffb640 1 -- 172.21.15.98:0/1072217917 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c0b5070 con 0x7fafac0ce8b0 2024-09-19T08:43:58.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.380+0000 7fafa9ffb640 1 -- 172.21.15.98:0/1072217917 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf9c0a89d0 con 0x7fafac0ce8b0 2024-09-19T08:43:58.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.380+0000 7fafa9ffb640 1 -- 172.21.15.98:0/1072217917 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c0a8da0 con 0x7fafac0ce8b0 2024-09-19T08:43:58.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.380+0000 7fafb18e0640 1 -- 172.21.15.98:0/1072217917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafac0ce8b0 msgr2=0x7fafac0b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:58.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.380+0000 7fafb18e0640 1 --2- 172.21.15.98:0/1072217917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafac0ce8b0 0x7fafac0b3560 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7faf9c009870 tx=0x7faf9c01ed10 comp rx=0 tx=0).stop 2024-09-19T08:43:58.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.380+0000 7fafb18e0640 1 -- 172.21.15.98:0/1072217917 shutdown_connections 2024-09-19T08:43:58.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.380+0000 7fafb18e0640 1 --2- 172.21.15.98:0/1072217917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafac0ce8b0 0x7fafac0b3560 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.380+0000 7fafb18e0640 1 --2- 172.21.15.98:0/1072217917 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafac158e40 0x7fafac06bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.380+0000 7fafb18e0640 1 -- 172.21.15.98:0/1072217917 >> 172.21.15.98:0/1072217917 conn(0x7fafac0b2010 msgr2=0x7fafac0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:58.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.380+0000 7fafb18e0640 1 -- 172.21.15.98:0/1072217917 shutdown_connections 2024-09-19T08:43:58.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.380+0000 7fafb18e0640 1 -- 172.21.15.98:0/1072217917 wait complete. 2024-09-19T08:43:58.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.380+0000 7fafb18e0640 1 Processor -- start 2024-09-19T08:43:58.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.380+0000 7fafb18e0640 1 -- start start 2024-09-19T08:43:58.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.380+0000 7fafb18e0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafac06c1d0 0x7fafac1517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:58.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.380+0000 7fafb18e0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafac0ce8b0 0x7fafac153ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:58.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.380+0000 7fafb18e0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fafac158e40 0x7fafac145840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:58.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.380+0000 7fafb18e0640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fafac159220 con 0x7fafac0ce8b0 2024-09-19T08:43:58.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.380+0000 7fafb18e0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fafac15f7d0 con 0x7fafac158e40 2024-09-19T08:43:58.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.380+0000 7fafb18e0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fafac0cee20 con 0x7fafac06c1d0 2024-09-19T08:43:58.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.380+0000 7fafaa7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafac0ce8b0 0x7fafac153ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:58.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.380+0000 7fafaa7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafac0ce8b0 0x7fafac153ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:50734/0 (socket says 172.21.15.98:50734) 2024-09-19T08:43:58.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.380+0000 7fafaa7fc640 1 -- 172.21.15.98:0/2323618486 learned_addr learned my addr 172.21.15.98:0/2323618486 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:58.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.384+0000 7fafaaffd640 1 --2- 172.21.15.98:0/2323618486 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafac06c1d0 0x7fafac1517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:58.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.384+0000 7fafab7fe640 1 --2- 172.21.15.98:0/2323618486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fafac158e40 0x7fafac145840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:58.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.384+0000 7fafaa7fc640 1 -- 172.21.15.98:0/2323618486 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafac06c1d0 msgr2=0x7fafac1517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:58.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.384+0000 7fafaa7fc640 1 --2- 172.21.15.98:0/2323618486 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafac06c1d0 0x7fafac1517a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.384+0000 7fafaa7fc640 1 -- 172.21.15.98:0/2323618486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fafac158e40 msgr2=0x7fafac145840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:58.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.384+0000 7fafaa7fc640 1 --2- 172.21.15.98:0/2323618486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fafac158e40 0x7fafac145840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.384+0000 7fafaa7fc640 1 -- 172.21.15.98:0/2323618486 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafac1525d0 con 0x7fafac0ce8b0 2024-09-19T08:43:58.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.384+0000 7fafaa7fc640 1 --2- 172.21.15.98:0/2323618486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafac0ce8b0 0x7fafac153ec0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fafa000c7a0 tx=0x7fafa000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:58.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.384+0000 7faf93fff640 1 -- 172.21.15.98:0/2323618486 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa0016020 con 0x7fafac0ce8b0 2024-09-19T08:43:58.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.384+0000 7faf93fff640 1 -- 172.21.15.98:0/2323618486 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafa000aa50 con 0x7fafac0ce8b0 2024-09-19T08:43:58.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.384+0000 7fafb18e0640 1 -- 172.21.15.98:0/2323618486 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafac0b03b0 con 0x7fafac0ce8b0 2024-09-19T08:43:58.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.384+0000 7faf93fff640 1 -- 172.21.15.98:0/2323618486 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa0012420 con 0x7fafac0ce8b0 2024-09-19T08:43:58.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.384+0000 7fafb18e0640 1 -- 172.21.15.98:0/2323618486 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafac0b0970 con 0x7fafac0ce8b0 2024-09-19T08:43:58.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.384+0000 7fafb18e0640 1 -- 172.21.15.98:0/2323618486 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafac06bc90 con 0x7fafac0ce8b0 2024-09-19T08:43:58.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.384+0000 7faf93fff640 1 -- 172.21.15.98:0/2323618486 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fafa0019020 con 0x7fafac0ce8b0 2024-09-19T08:43:58.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.384+0000 7faf93fff640 1 --2- 172.21.15.98:0/2323618486 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7faf980679e0 0x7faf98069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:58.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.384+0000 7faf93fff640 1 -- 172.21.15.98:0/2323618486 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fafa0085e30 con 0x7fafac0ce8b0 2024-09-19T08:43:58.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.384+0000 7fafaaffd640 1 --2- 172.21.15.98:0/2323618486 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7faf980679e0 0x7faf98069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:58.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.384+0000 7fafaaffd640 1 --2- 172.21.15.98:0/2323618486 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7faf980679e0 0x7faf98069ea0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fafa400b7c0 tx=0x7fafa4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:58.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.388+0000 7faf93fff640 1 -- 172.21.15.98:0/2323618486 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fafa0020030 con 0x7fafac0ce8b0 2024-09-19T08:43:58.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.548+0000 7fafb18e0640 1 -- 172.21.15.98:0/2323618486 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fafac15cde0 con 0x7faf980679e0 2024-09-19T08:43:58.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.568+0000 7faf93fff640 1 -- 172.21.15.98:0/2323618486 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240183 (secure 0 0 0) 0x7fafac15cde0 con 0x7faf980679e0 2024-09-19T08:43:58.571 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:58.573 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:43:58.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.572+0000 7fafb18e0640 1 -- 172.21.15.98:0/2323618486 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7faf980679e0 msgr2=0x7faf98069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:58.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.572+0000 7fafb18e0640 1 --2- 172.21.15.98:0/2323618486 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7faf980679e0 0x7faf98069ea0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fafa400b7c0 tx=0x7fafa4008040 comp rx=0 tx=0).stop 2024-09-19T08:43:58.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.572+0000 7fafb18e0640 1 -- 172.21.15.98:0/2323618486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafac0ce8b0 msgr2=0x7fafac153ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:58.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.572+0000 7fafb18e0640 1 --2- 172.21.15.98:0/2323618486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafac0ce8b0 0x7fafac153ec0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fafa000c7a0 tx=0x7fafa000cc60 comp rx=0 tx=0).stop 2024-09-19T08:43:58.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.572+0000 7fafb18e0640 1 -- 172.21.15.98:0/2323618486 shutdown_connections 2024-09-19T08:43:58.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.572+0000 7fafb18e0640 1 --2- 172.21.15.98:0/2323618486 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7faf980679e0 0x7faf98069ea0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.572+0000 7fafb18e0640 1 --2- 172.21.15.98:0/2323618486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fafac158e40 0x7fafac145840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.572+0000 7fafb18e0640 1 --2- 172.21.15.98:0/2323618486 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fafac0ce8b0 0x7fafac153ec0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.572+0000 7fafb18e0640 1 --2- 172.21.15.98:0/2323618486 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fafac06c1d0 0x7fafac1517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.572+0000 7fafb18e0640 1 -- 172.21.15.98:0/2323618486 >> 172.21.15.98:0/2323618486 conn(0x7fafac0b2010 msgr2=0x7fafac072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:58.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.572+0000 7fafb18e0640 1 -- 172.21.15.98:0/2323618486 shutdown_connections 2024-09-19T08:43:58.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.572+0000 7fafb18e0640 1 -- 172.21.15.98:0/2323618486 wait complete. 2024-09-19T08:43:58.590 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2024-09-19T08:43:56.890011+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260232,"kb_used_data":10716,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714488,"statfs":{"total":773094113280,"available":772827635712,"internally_reserved":0,"allocated":10973184,"data_stored":5457071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001163"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.147140+0000","last_change":"2024-09-19T08:43:53.147140+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.147140+0000","last_undegraded":"2024-09-19T08:43:53.147140+0000","last_fullsized":"2024-09-19T08:43:53.147140+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.143133+0000","last_change":"2024-09-19T08:43:53.143133+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.143133+0000","last_undegraded":"2024-09-19T08:43:53.143133+0000","last_fullsized":"2024-09-19T08:43:53.143133+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.142190+0000","last_change":"2024-09-19T08:43:53.142190+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.142190+0000","last_undegraded":"2024-09-19T08:43:53.142190+0000","last_fullsized":"2024-09-19T08:43:53.142190+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.141020+0000","last_change":"2024-09-19T08:43:53.141020+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.141020+0000","last_undegraded":"2024-09-19T08:43:53.141020+0000","last_fullsized":"2024-09-19T08:43:53.141020+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.149299+0000","last_change":"2024-09-19T08:43:53.149299+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.149299+0000","last_undegraded":"2024-09-19T08:43:53.149299+0000","last_fullsized":"2024-09-19T08:43:53.149299+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.138068+0000","last_change":"2024-09-19T08:43:53.138068+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.138068+0000","last_undegraded":"2024-09-19T08:43:53.138068+0000","last_fullsized":"2024-09-19T08:43:53.138068+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.136114+0000","last_change":"2024-09-19T08:43:53.136114+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.136114+0000","last_undegraded":"2024-09-19T08:43:53.136114+0000","last_fullsized":"2024-09-19T08:43:53.136114+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.142751+0000","last_change":"2024-09-19T08:43:53.142751+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.142751+0000","last_undegraded":"2024-09-19T08:43:53.142751+0000","last_fullsized":"2024-09-19T08:43:53.142751+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.140651+0000","last_change":"2024-09-19T08:43:53.140651+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.140651+0000","last_undegraded":"2024-09-19T08:43:53.140651+0000","last_fullsized":"2024-09-19T08:43:53.140651+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.138101+0000","last_change":"2024-09-19T08:43:53.138101+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.138101+0000","last_undegraded":"2024-09-19T08:43:53.138101+0000","last_fullsized":"2024-09-19T08:43:53.138101+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.141331+0000","last_change":"2024-09-19T08:43:53.141331+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.141331+0000","last_undegraded":"2024-09-19T08:43:53.141331+0000","last_fullsized":"2024-09-19T08:43:53.141331+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.142992+0000","last_change":"2024-09-19T08:43:53.142992+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.142992+0000","last_undegraded":"2024-09-19T08:43:53.142992+0000","last_fullsized":"2024-09-19T08:43:53.142992+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.138625+0000","last_change":"2024-09-19T08:43:53.138625+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.138625+0000","last_undegraded":"2024-09-19T08:43:53.138625+0000","last_fullsized":"2024-09-19T08:43:53.138625+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.142322+0000","last_change":"2024-09-19T08:43:53.142322+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.142322+0000","last_undegraded":"2024-09-19T08:43:53.142322+0000","last_fullsized":"2024-09-19T08:43:53.142322+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.141433+0000","last_change":"2024-09-19T08:43:53.141433+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.141433+0000","last_undegraded":"2024-09-19T08:43:53.141433+0000","last_fullsized":"2024-09-19T08:43:53.141433+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.141855+0000","last_change":"2024-09-19T08:43:53.141855+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.141855+0000","last_undegraded":"2024-09-19T08:43:53.141855+0000","last_fullsized":"2024-09-19T08:43:53.141855+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.140657+0000","last_change":"2024-09-19T08:43:53.140657+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.140657+0000","last_undegraded":"2024-09-19T08:43:53.140657+0000","last_fullsized":"2024-09-19T08:43:53.140657+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.142608+0000","last_change":"2024-09-19T08:43:53.142608+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.142608+0000","last_undegraded":"2024-09-19T08:43:53.142608+0000","last_fullsized":"2024-09-19T08:43:53.142608+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.147149+0000","last_change":"2024-09-19T08:43:53.147149+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.147149+0000","last_undegraded":"2024-09-19T08:43:53.147149+0000","last_fullsized":"2024-09-19T08:43:53.147149+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.138202+0000","last_change":"2024-09-19T08:43:53.138202+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.138202+0000","last_undegraded":"2024-09-19T08:43:53.138202+0000","last_fullsized":"2024-09-19T08:43:53.138202+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.140693+0000","last_change":"2024-09-19T08:43:53.140693+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.140693+0000","last_undegraded":"2024-09-19T08:43:53.140693+0000","last_fullsized":"2024-09-19T08:43:53.140693+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.138099+0000","last_change":"2024-09-19T08:43:53.138099+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.138099+0000","last_undegraded":"2024-09-19T08:43:53.138099+0000","last_fullsized":"2024-09-19T08:43:53.138099+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.142390+0000","last_change":"2024-09-19T08:43:53.142390+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.142390+0000","last_undegraded":"2024-09-19T08:43:53.142390+0000","last_fullsized":"2024-09-19T08:43:53.142390+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.138074+0000","last_change":"2024-09-19T08:43:53.138074+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.138074+0000","last_undegraded":"2024-09-19T08:43:53.138074+0000","last_fullsized":"2024-09-19T08:43:53.138074+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.142577+0000","last_change":"2024-09-19T08:43:53.142577+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.142577+0000","last_undegraded":"2024-09-19T08:43:53.142577+0000","last_fullsized":"2024-09-19T08:43:53.142577+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.140622+0000","last_change":"2024-09-19T08:43:53.140622+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.140622+0000","last_undegraded":"2024-09-19T08:43:53.140622+0000","last_fullsized":"2024-09-19T08:43:53.140622+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.138475+0000","last_change":"2024-09-19T08:43:53.138475+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.138475+0000","last_undegraded":"2024-09-19T08:43:53.138475+0000","last_fullsized":"2024-09-19T08:43:53.138475+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.147110+0000","last_change":"2024-09-19T08:43:53.147110+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.147110+0000","last_undegraded":"2024-09-19T08:43:53.147110+0000","last_fullsized":"2024-09-19T08:43:53.147110+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.135914+0000","last_change":"2024-09-19T08:43:53.135914+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.135914+0000","last_undegraded":"2024-09-19T08:43:53.135914+0000","last_fullsized":"2024-09-19T08:43:53.135914+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.140942+0000","last_change":"2024-09-19T08:43:53.140942+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.140942+0000","last_undegraded":"2024-09-19T08:43:53.140942+0000","last_fullsized":"2024-09-19T08:43:53.140942+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.140689+0000","last_change":"2024-09-19T08:43:53.140689+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.140689+0000","last_undegraded":"2024-09-19T08:43:53.140689+0000","last_fullsized":"2024-09-19T08:43:53.140689+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:43:54.129671+0000","last_change":"2024-09-19T08:43:54.129671+0000","last_active":"2024-09-19T08:43:54.129671+0000","last_peered":"2024-09-19T08:43:54.129671+0000","last_clean":"2024-09-19T08:43:54.129671+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:54.129671+0000","last_undegraded":"2024-09-19T08:43:54.129671+0000","last_fullsized":"2024-09-19T08:43:54.129671+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":99,"state":"creating+peering","last_fresh":"2024-09-19T08:43:53.147121+0000","last_change":"2024-09-19T08:43:53.147121+0000","last_active":"2024-09-19T08:43:53.120145+0000","last_peered":"2024-09-19T08:43:53.120145+0000","last_clean":"2024-09-19T08:43:53.120145+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:43:53.147121+0000","last_undegraded":"2024-09-19T08:43:53.147121+0000","last_fullsized":"2024-09-19T08:43:53.147121+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"71'128","reported_seq":349,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-19T08:43:53.142818+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:43:53.142818+0000","last_peered":"2024-09-19T08:43:53.142818+0000","last_clean":"2024-09-19T08:43:53.142818+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:43:53.142818+0000","last_undegraded":"2024-09-19T08:43:53.142818+0000","last_fullsized":"2024-09-19T08:43:53.142818+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509512,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32964,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338876,"statfs":{"total":96636764160,"available":96603009024,"internally_reserved":0,"allocated":1818624,"data_stored":1125637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574921,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1224704,"data_stored":535269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509514,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1224704,"data_stored":535269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509514,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1224704,"data_stored":535269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542216,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1224704,"data_stored":535269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509511,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1224704,"data_stored":535269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509513,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32372,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339468,"statfs":{"total":96636764160,"available":96603615232,"internally_reserved":0,"allocated":1212416,"data_stored":529452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509511,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32968,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338872,"statfs":{"total":96636764160,"available":96603004928,"internally_reserved":0,"allocated":1818624,"data_stored":1125637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:43:58.594 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:43:58.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.744+0000 7f71cbe29640 1 Processor -- start 2024-09-19T08:43:58.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.744+0000 7f71cbe29640 1 -- start start 2024-09-19T08:43:58.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.748+0000 7f71cbe29640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71c4158fd0 0x7f71c40b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:58.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.748+0000 7f71cbe29640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f71c40b4360 0x7f71c406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:58.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.748+0000 7f71cbe29640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f71c406c1d0 0x7f71c415efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:58.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.748+0000 7f71cbe29640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f71c406c620 con 0x7f71c40b4360 2024-09-19T08:43:58.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.748+0000 7f71cbe29640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f71c41593b0 con 0x7f71c406c1d0 2024-09-19T08:43:58.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.748+0000 7f71cbe29640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f71c40ccc60 con 0x7f71c4158fd0 2024-09-19T08:43:58.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.748+0000 7f71c93c4640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f71c40b4360 0x7f71c406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:58.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.748+0000 7f71c93c4640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f71c40b4360 0x7f71c406bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:50742/0 (socket says 172.21.15.98:50742) 2024-09-19T08:43:58.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.748+0000 7f71c93c4640 1 -- 172.21.15.98:0/2304990247 learned_addr learned my addr 172.21.15.98:0/2304990247 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:58.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.748+0000 7f71c93c4640 1 -- 172.21.15.98:0/2304990247 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71c4158fd0 msgr2=0x7f71c40b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:58.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.748+0000 7f71c93c4640 1 --2- 172.21.15.98:0/2304990247 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71c4158fd0 0x7f71c40b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.748+0000 7f71c93c4640 1 -- 172.21.15.98:0/2304990247 >> v2:172.21.15.139:3300/0 conn(0x7f71c406c1d0 msgr2=0x7f71c415efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:58.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.748+0000 7f71c93c4640 1 --2- 172.21.15.98:0/2304990247 >> v2:172.21.15.139:3300/0 conn(0x7f71c406c1d0 0x7f71c415efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.748+0000 7f71c93c4640 1 -- 172.21.15.98:0/2304990247 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c40ccf40 con 0x7f71c40b4360 2024-09-19T08:43:58.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.748+0000 7f71c93c4640 1 --2- 172.21.15.98:0/2304990247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71c40b4360 0x7f71c406bc90 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f71c40ce4d0 tx=0x7f71c001c1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55bf8d8f4ed0b889 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:58.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.748+0000 7f71c8bc3640 1 -- 172.21.15.98:0/2304990247 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c00b0070 con 0x7f71c40b4360 2024-09-19T08:43:58.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.748+0000 7f71c8bc3640 1 -- 172.21.15.98:0/2304990247 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71c00ab600 con 0x7f71c40b4360 2024-09-19T08:43:58.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.748+0000 7f71c8bc3640 1 -- 172.21.15.98:0/2304990247 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c00ab9d0 con 0x7f71c40b4360 2024-09-19T08:43:58.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.752+0000 7f71cbe29640 1 -- 172.21.15.98:0/2304990247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71c40b4360 msgr2=0x7f71c406bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:58.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.752+0000 7f71cbe29640 1 --2- 172.21.15.98:0/2304990247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71c40b4360 0x7f71c406bc90 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f71c40ce4d0 tx=0x7f71c001c1a0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.752+0000 7f71cbe29640 1 -- 172.21.15.98:0/2304990247 shutdown_connections 2024-09-19T08:43:58.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.752+0000 7f71cbe29640 1 --2- 172.21.15.98:0/2304990247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71c40b4360 0x7f71c406bc90 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.752+0000 7f71cbe29640 1 --2- 172.21.15.98:0/2304990247 >> v2:172.21.15.139:3300/0 conn(0x7f71c406c1d0 0x7f71c415efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.752+0000 7f71cbe29640 1 --2- 172.21.15.98:0/2304990247 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71c4158fd0 0x7f71c40b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.752+0000 7f71cbe29640 1 -- 172.21.15.98:0/2304990247 >> 172.21.15.98:0/2304990247 conn(0x7f71c40b2010 msgr2=0x7f71c40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:58.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.752+0000 7f71cbe29640 1 -- 172.21.15.98:0/2304990247 shutdown_connections 2024-09-19T08:43:58.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.752+0000 7f71cbe29640 1 -- 172.21.15.98:0/2304990247 wait complete. 2024-09-19T08:43:58.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.752+0000 7f71cbe29640 1 Processor -- start 2024-09-19T08:43:58.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.752+0000 7f71cbe29640 1 -- start start 2024-09-19T08:43:58.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.752+0000 7f71cbe29640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71c406c1d0 0x7f71c41fa420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:58.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.752+0000 7f71cbe29640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71c40b4360 0x7f71c40ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:58.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.752+0000 7f71cbe29640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71c4158fd0 0x7f71c40aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:58.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.752+0000 7f71cbe29640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f71c415c920 con 0x7f71c406c1d0 2024-09-19T08:43:58.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.752+0000 7f71cbe29640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f71c415c7a0 con 0x7f71c40b4360 2024-09-19T08:43:58.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.752+0000 7f71cbe29640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f71c40ce840 con 0x7f71c4158fd0 2024-09-19T08:43:58.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.752+0000 7f71c9bc5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71c406c1d0 0x7f71c41fa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:58.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.752+0000 7f71c9bc5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71c406c1d0 0x7f71c41fa420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:50756/0 (socket says 172.21.15.98:50756) 2024-09-19T08:43:58.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.752+0000 7f71c9bc5640 1 -- 172.21.15.98:0/2501120543 learned_addr learned my addr 172.21.15.98:0/2501120543 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:58.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.752+0000 7f71c9bc5640 1 -- 172.21.15.98:0/2501120543 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71c4158fd0 msgr2=0x7f71c40aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:58.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.752+0000 7f71c9bc5640 1 --2- 172.21.15.98:0/2501120543 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71c4158fd0 0x7f71c40aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.752+0000 7f71c9bc5640 1 -- 172.21.15.98:0/2501120543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71c40b4360 msgr2=0x7f71c40ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:58.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.752+0000 7f71c9bc5640 1 --2- 172.21.15.98:0/2501120543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71c40b4360 0x7f71c40ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.752+0000 7f71c9bc5640 1 -- 172.21.15.98:0/2501120543 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c4071e40 con 0x7f71c406c1d0 2024-09-19T08:43:58.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.752+0000 7f71c9bc5640 1 --2- 172.21.15.98:0/2501120543 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71c406c1d0 0x7f71c41fa420 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f71b800b410 tx=0x7f71b800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:58.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.752+0000 7f71b67fc640 1 -- 172.21.15.98:0/2501120543 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b8015020 con 0x7f71c406c1d0 2024-09-19T08:43:58.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.752+0000 7f71b67fc640 1 -- 172.21.15.98:0/2501120543 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71b8011830 con 0x7f71c406c1d0 2024-09-19T08:43:58.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.752+0000 7f71cbe29640 1 -- 172.21.15.98:0/2501120543 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c4072120 con 0x7f71c406c1d0 2024-09-19T08:43:58.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.752+0000 7f71b67fc640 1 -- 172.21.15.98:0/2501120543 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b8011c20 con 0x7f71c406c1d0 2024-09-19T08:43:58.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.752+0000 7f71cbe29640 1 -- 172.21.15.98:0/2501120543 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c40726e0 con 0x7f71c406c1d0 2024-09-19T08:43:58.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.752+0000 7f71cbe29640 1 -- 172.21.15.98:0/2501120543 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71c406bc90 con 0x7f71c406c1d0 2024-09-19T08:43:58.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.756+0000 7f71b67fc640 1 -- 172.21.15.98:0/2501120543 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f71b8018020 con 0x7f71c406c1d0 2024-09-19T08:43:58.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.756+0000 7f71b67fc640 1 --2- 172.21.15.98:0/2501120543 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f71a80679e0 0x7f71a8069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:58.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.756+0000 7f71b67fc640 1 -- 172.21.15.98:0/2501120543 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f71b8085d80 con 0x7f71c406c1d0 2024-09-19T08:43:58.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.756+0000 7f71c93c4640 1 --2- 172.21.15.98:0/2501120543 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f71a80679e0 0x7f71a8069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:58.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.756+0000 7f71c93c4640 1 --2- 172.21.15.98:0/2501120543 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f71a80679e0 0x7f71a8069ea0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f71c00ad000 tx=0x7f71c0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:58.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.760+0000 7f71b67fc640 1 -- 172.21.15.98:0/2501120543 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f71b801f020 con 0x7f71c406c1d0 2024-09-19T08:43:58.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.916+0000 7f71cbe29640 1 -- 172.21.15.98:0/2501120543 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f71c41fa960 con 0x7f71a80679e0 2024-09-19T08:43:58.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.932+0000 7f71b67fc640 1 -- 172.21.15.98:0/2501120543 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252426 (secure 0 0 0) 0x7f71c41fa960 con 0x7f71a80679e0 2024-09-19T08:43:58.938 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:58.940 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:43:58.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.936+0000 7f71cbe29640 1 -- 172.21.15.98:0/2501120543 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f71a80679e0 msgr2=0x7f71a8069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:58.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.936+0000 7f71cbe29640 1 --2- 172.21.15.98:0/2501120543 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f71a80679e0 0x7f71a8069ea0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f71c00ad000 tx=0x7f71c0008040 comp rx=0 tx=0).stop 2024-09-19T08:43:58.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.936+0000 7f71cbe29640 1 -- 172.21.15.98:0/2501120543 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71c406c1d0 msgr2=0x7f71c41fa420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:58.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.936+0000 7f71cbe29640 1 --2- 172.21.15.98:0/2501120543 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71c406c1d0 0x7f71c41fa420 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f71b800b410 tx=0x7f71b800b8d0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.936+0000 7f71cbe29640 1 -- 172.21.15.98:0/2501120543 shutdown_connections 2024-09-19T08:43:58.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.936+0000 7f71cbe29640 1 --2- 172.21.15.98:0/2501120543 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f71a80679e0 0x7f71a8069ea0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.936+0000 7f71cbe29640 1 --2- 172.21.15.98:0/2501120543 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71c4158fd0 0x7f71c40aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.936+0000 7f71cbe29640 1 --2- 172.21.15.98:0/2501120543 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71c40b4360 0x7f71c40ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.936+0000 7f71cbe29640 1 --2- 172.21.15.98:0/2501120543 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71c406c1d0 0x7f71c41fa420 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:58.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.936+0000 7f71cbe29640 1 -- 172.21.15.98:0/2501120543 >> 172.21.15.98:0/2501120543 conn(0x7f71c40b2010 msgr2=0x7f71c4155650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:58.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.936+0000 7f71cbe29640 1 -- 172.21.15.98:0/2501120543 shutdown_connections 2024-09-19T08:43:58.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:58.936+0000 7f71cbe29640 1 -- 172.21.15.98:0/2501120543 wait complete. 2024-09-19T08:43:58.955 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2024-09-19T08:43:58.890652+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260600,"kb_used_data":11080,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714120,"statfs":{"total":773094113280,"available":772827258880,"internally_reserved":0,"allocated":11345920,"data_stored":5622508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.758732"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.156262+0000","last_change":"2024-09-19T08:43:55.175947+0000","last_active":"2024-09-19T08:43:56.156262+0000","last_peered":"2024-09-19T08:43:56.156262+0000","last_clean":"2024-09-19T08:43:56.156262+0000","last_became_active":"2024-09-19T08:43:55.175606+0000","last_became_peered":"2024-09-19T08:43:55.175606+0000","last_unstale":"2024-09-19T08:43:56.156262+0000","last_undegraded":"2024-09-19T08:43:56.156262+0000","last_fullsized":"2024-09-19T08:43:56.156262+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:46:51.742785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.161571+0000","last_change":"2024-09-19T08:43:55.206900+0000","last_active":"2024-09-19T08:43:56.161571+0000","last_peered":"2024-09-19T08:43:56.161571+0000","last_clean":"2024-09-19T08:43:56.161571+0000","last_became_active":"2024-09-19T08:43:55.200022+0000","last_became_peered":"2024-09-19T08:43:55.200022+0000","last_unstale":"2024-09-19T08:43:56.161571+0000","last_undegraded":"2024-09-19T08:43:56.161571+0000","last_fullsized":"2024-09-19T08:43:56.161571+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:39:48.558843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736619+0000","last_change":"2024-09-19T08:43:55.191789+0000","last_active":"2024-09-19T08:43:56.736619+0000","last_peered":"2024-09-19T08:43:56.736619+0000","last_clean":"2024-09-19T08:43:56.736619+0000","last_became_active":"2024-09-19T08:43:55.191065+0000","last_became_peered":"2024-09-19T08:43:55.191065+0000","last_unstale":"2024-09-19T08:43:56.736619+0000","last_undegraded":"2024-09-19T08:43:56.736619+0000","last_fullsized":"2024-09-19T08:43:56.736619+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:15.340286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.156243+0000","last_change":"2024-09-19T08:43:55.199879+0000","last_active":"2024-09-19T08:43:56.156243+0000","last_peered":"2024-09-19T08:43:56.156243+0000","last_clean":"2024-09-19T08:43:56.156243+0000","last_became_active":"2024-09-19T08:43:55.199679+0000","last_became_peered":"2024-09-19T08:43:55.199679+0000","last_unstale":"2024-09-19T08:43:56.156243+0000","last_undegraded":"2024-09-19T08:43:56.156243+0000","last_fullsized":"2024-09-19T08:43:56.156243+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:54:02.803710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736288+0000","last_change":"2024-09-19T08:43:55.168455+0000","last_active":"2024-09-19T08:43:56.736288+0000","last_peered":"2024-09-19T08:43:56.736288+0000","last_clean":"2024-09-19T08:43:56.736288+0000","last_became_active":"2024-09-19T08:43:55.168218+0000","last_became_peered":"2024-09-19T08:43:55.168218+0000","last_unstale":"2024-09-19T08:43:56.736288+0000","last_undegraded":"2024-09-19T08:43:56.736288+0000","last_fullsized":"2024-09-19T08:43:56.736288+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:05:28.010611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.575891+0000","last_change":"2024-09-19T08:43:55.169699+0000","last_active":"2024-09-19T08:43:56.575891+0000","last_peered":"2024-09-19T08:43:56.575891+0000","last_clean":"2024-09-19T08:43:56.575891+0000","last_became_active":"2024-09-19T08:43:55.169436+0000","last_became_peered":"2024-09-19T08:43:55.169436+0000","last_unstale":"2024-09-19T08:43:56.575891+0000","last_undegraded":"2024-09-19T08:43:56.575891+0000","last_fullsized":"2024-09-19T08:43:56.575891+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:29:31.356189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.169833+0000","last_change":"2024-09-19T08:43:55.194043+0000","last_active":"2024-09-19T08:43:56.169833+0000","last_peered":"2024-09-19T08:43:56.169833+0000","last_clean":"2024-09-19T08:43:56.169833+0000","last_became_active":"2024-09-19T08:43:55.193266+0000","last_became_peered":"2024-09-19T08:43:55.193266+0000","last_unstale":"2024-09-19T08:43:56.169833+0000","last_undegraded":"2024-09-19T08:43:56.169833+0000","last_fullsized":"2024-09-19T08:43:56.169833+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:28.089099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736336+0000","last_change":"2024-09-19T08:43:55.177179+0000","last_active":"2024-09-19T08:43:56.736336+0000","last_peered":"2024-09-19T08:43:56.736336+0000","last_clean":"2024-09-19T08:43:56.736336+0000","last_became_active":"2024-09-19T08:43:55.174397+0000","last_became_peered":"2024-09-19T08:43:55.174397+0000","last_unstale":"2024-09-19T08:43:56.736336+0000","last_undegraded":"2024-09-19T08:43:56.736336+0000","last_fullsized":"2024-09-19T08:43:56.736336+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:08:22.088644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.156620+0000","last_change":"2024-09-19T08:43:55.187917+0000","last_active":"2024-09-19T08:43:56.156620+0000","last_peered":"2024-09-19T08:43:56.156620+0000","last_clean":"2024-09-19T08:43:56.156620+0000","last_became_active":"2024-09-19T08:43:55.187097+0000","last_became_peered":"2024-09-19T08:43:55.187097+0000","last_unstale":"2024-09-19T08:43:56.156620+0000","last_undegraded":"2024-09-19T08:43:56.156620+0000","last_fullsized":"2024-09-19T08:43:56.156620+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:34:28.929131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.155582+0000","last_change":"2024-09-19T08:43:55.176966+0000","last_active":"2024-09-19T08:43:56.155582+0000","last_peered":"2024-09-19T08:43:56.155582+0000","last_clean":"2024-09-19T08:43:56.155582+0000","last_became_active":"2024-09-19T08:43:55.175980+0000","last_became_peered":"2024-09-19T08:43:55.175980+0000","last_unstale":"2024-09-19T08:43:56.155582+0000","last_undegraded":"2024-09-19T08:43:56.155582+0000","last_fullsized":"2024-09-19T08:43:56.155582+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:46.206086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.160446+0000","last_change":"2024-09-19T08:43:55.179425+0000","last_active":"2024-09-19T08:43:56.160446+0000","last_peered":"2024-09-19T08:43:56.160446+0000","last_clean":"2024-09-19T08:43:56.160446+0000","last_became_active":"2024-09-19T08:43:55.178734+0000","last_became_peered":"2024-09-19T08:43:55.178734+0000","last_unstale":"2024-09-19T08:43:56.160446+0000","last_undegraded":"2024-09-19T08:43:56.160446+0000","last_fullsized":"2024-09-19T08:43:56.160446+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:51.785619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.156543+0000","last_change":"2024-09-19T08:43:55.180274+0000","last_active":"2024-09-19T08:43:56.156543+0000","last_peered":"2024-09-19T08:43:56.156543+0000","last_clean":"2024-09-19T08:43:56.156543+0000","last_became_active":"2024-09-19T08:43:55.179930+0000","last_became_peered":"2024-09-19T08:43:55.179930+0000","last_unstale":"2024-09-19T08:43:56.156543+0000","last_undegraded":"2024-09-19T08:43:56.156543+0000","last_fullsized":"2024-09-19T08:43:56.156543+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:23.428495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.737001+0000","last_change":"2024-09-19T08:43:55.170744+0000","last_active":"2024-09-19T08:43:56.737001+0000","last_peered":"2024-09-19T08:43:56.737001+0000","last_clean":"2024-09-19T08:43:56.737001+0000","last_became_active":"2024-09-19T08:43:55.169601+0000","last_became_peered":"2024-09-19T08:43:55.169601+0000","last_unstale":"2024-09-19T08:43:56.737001+0000","last_undegraded":"2024-09-19T08:43:56.737001+0000","last_fullsized":"2024-09-19T08:43:56.737001+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:39:02.010421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736207+0000","last_change":"2024-09-19T08:43:55.173507+0000","last_active":"2024-09-19T08:43:56.736207+0000","last_peered":"2024-09-19T08:43:56.736207+0000","last_clean":"2024-09-19T08:43:56.736207+0000","last_became_active":"2024-09-19T08:43:55.172785+0000","last_became_peered":"2024-09-19T08:43:55.172785+0000","last_unstale":"2024-09-19T08:43:56.736207+0000","last_undegraded":"2024-09-19T08:43:56.736207+0000","last_fullsized":"2024-09-19T08:43:56.736207+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:02:46.286404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.575763+0000","last_change":"2024-09-19T08:43:55.170172+0000","last_active":"2024-09-19T08:43:56.575763+0000","last_peered":"2024-09-19T08:43:56.575763+0000","last_clean":"2024-09-19T08:43:56.575763+0000","last_became_active":"2024-09-19T08:43:55.169053+0000","last_became_peered":"2024-09-19T08:43:55.169053+0000","last_unstale":"2024-09-19T08:43:56.575763+0000","last_undegraded":"2024-09-19T08:43:56.575763+0000","last_fullsized":"2024-09-19T08:43:56.575763+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:23.435267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.155922+0000","last_change":"2024-09-19T08:43:55.181396+0000","last_active":"2024-09-19T08:43:56.155922+0000","last_peered":"2024-09-19T08:43:56.155922+0000","last_clean":"2024-09-19T08:43:56.155922+0000","last_became_active":"2024-09-19T08:43:55.180917+0000","last_became_peered":"2024-09-19T08:43:55.180917+0000","last_unstale":"2024-09-19T08:43:56.155922+0000","last_undegraded":"2024-09-19T08:43:56.155922+0000","last_fullsized":"2024-09-19T08:43:56.155922+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:40:18.247113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.161176+0000","last_change":"2024-09-19T08:43:55.186312+0000","last_active":"2024-09-19T08:43:56.161176+0000","last_peered":"2024-09-19T08:43:56.161176+0000","last_clean":"2024-09-19T08:43:56.161176+0000","last_became_active":"2024-09-19T08:43:55.185577+0000","last_became_peered":"2024-09-19T08:43:55.185577+0000","last_unstale":"2024-09-19T08:43:56.161176+0000","last_undegraded":"2024-09-19T08:43:56.161176+0000","last_fullsized":"2024-09-19T08:43:56.161176+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:41.369862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.165469+0000","last_change":"2024-09-19T08:43:55.209049+0000","last_active":"2024-09-19T08:43:56.165469+0000","last_peered":"2024-09-19T08:43:56.165469+0000","last_clean":"2024-09-19T08:43:56.165469+0000","last_became_active":"2024-09-19T08:43:55.208404+0000","last_became_peered":"2024-09-19T08:43:55.208404+0000","last_unstale":"2024-09-19T08:43:56.165469+0000","last_undegraded":"2024-09-19T08:43:56.165469+0000","last_fullsized":"2024-09-19T08:43:56.165469+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:55.541839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736693+0000","last_change":"2024-09-19T08:43:55.179067+0000","last_active":"2024-09-19T08:43:56.736693+0000","last_peered":"2024-09-19T08:43:56.736693+0000","last_clean":"2024-09-19T08:43:56.736693+0000","last_became_active":"2024-09-19T08:43:55.178109+0000","last_became_peered":"2024-09-19T08:43:55.178109+0000","last_unstale":"2024-09-19T08:43:56.736693+0000","last_undegraded":"2024-09-19T08:43:56.736693+0000","last_fullsized":"2024-09-19T08:43:56.736693+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:51.785619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.161144+0000","last_change":"2024-09-19T08:43:55.201728+0000","last_active":"2024-09-19T08:43:56.161144+0000","last_peered":"2024-09-19T08:43:56.161144+0000","last_clean":"2024-09-19T08:43:56.161144+0000","last_became_active":"2024-09-19T08:43:55.201157+0000","last_became_peered":"2024-09-19T08:43:55.201157+0000","last_unstale":"2024-09-19T08:43:56.161144+0000","last_undegraded":"2024-09-19T08:43:56.161144+0000","last_fullsized":"2024-09-19T08:43:56.161144+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:51:42.503330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.160768+0000","last_change":"2024-09-19T08:43:55.188891+0000","last_active":"2024-09-19T08:43:56.160768+0000","last_peered":"2024-09-19T08:43:56.160768+0000","last_clean":"2024-09-19T08:43:56.160768+0000","last_became_active":"2024-09-19T08:43:55.188094+0000","last_became_peered":"2024-09-19T08:43:55.188094+0000","last_unstale":"2024-09-19T08:43:56.160768+0000","last_undegraded":"2024-09-19T08:43:56.160768+0000","last_fullsized":"2024-09-19T08:43:56.160768+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:31:11.824242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.735712+0000","last_change":"2024-09-19T08:43:55.171322+0000","last_active":"2024-09-19T08:43:56.735712+0000","last_peered":"2024-09-19T08:43:56.735712+0000","last_clean":"2024-09-19T08:43:56.735712+0000","last_became_active":"2024-09-19T08:43:55.171040+0000","last_became_peered":"2024-09-19T08:43:55.171040+0000","last_unstale":"2024-09-19T08:43:56.735712+0000","last_undegraded":"2024-09-19T08:43:56.735712+0000","last_fullsized":"2024-09-19T08:43:56.735712+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:44.393216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.161536+0000","last_change":"2024-09-19T08:43:55.208256+0000","last_active":"2024-09-19T08:43:56.161536+0000","last_peered":"2024-09-19T08:43:56.161536+0000","last_clean":"2024-09-19T08:43:56.161536+0000","last_became_active":"2024-09-19T08:43:55.207547+0000","last_became_peered":"2024-09-19T08:43:55.207547+0000","last_unstale":"2024-09-19T08:43:56.161536+0000","last_undegraded":"2024-09-19T08:43:56.161536+0000","last_fullsized":"2024-09-19T08:43:56.161536+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:37:19.729827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.735102+0000","last_change":"2024-09-19T08:43:55.172018+0000","last_active":"2024-09-19T08:43:56.735102+0000","last_peered":"2024-09-19T08:43:56.735102+0000","last_clean":"2024-09-19T08:43:56.735102+0000","last_became_active":"2024-09-19T08:43:55.171714+0000","last_became_peered":"2024-09-19T08:43:55.171714+0000","last_unstale":"2024-09-19T08:43:56.735102+0000","last_undegraded":"2024-09-19T08:43:56.735102+0000","last_fullsized":"2024-09-19T08:43:56.735102+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:46:43.070713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.161452+0000","last_change":"2024-09-19T08:43:55.176249+0000","last_active":"2024-09-19T08:43:56.161452+0000","last_peered":"2024-09-19T08:43:56.161452+0000","last_clean":"2024-09-19T08:43:56.161452+0000","last_became_active":"2024-09-19T08:43:55.175331+0000","last_became_peered":"2024-09-19T08:43:55.175331+0000","last_unstale":"2024-09-19T08:43:56.161452+0000","last_undegraded":"2024-09-19T08:43:56.161452+0000","last_fullsized":"2024-09-19T08:43:56.161452+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:31:31.316017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.155735+0000","last_change":"2024-09-19T08:43:55.198912+0000","last_active":"2024-09-19T08:43:56.155735+0000","last_peered":"2024-09-19T08:43:56.155735+0000","last_clean":"2024-09-19T08:43:56.155735+0000","last_became_active":"2024-09-19T08:43:55.198693+0000","last_became_peered":"2024-09-19T08:43:55.198693+0000","last_unstale":"2024-09-19T08:43:56.155735+0000","last_undegraded":"2024-09-19T08:43:56.155735+0000","last_fullsized":"2024-09-19T08:43:56.155735+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:43.862034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.576308+0000","last_change":"2024-09-19T08:43:55.175621+0000","last_active":"2024-09-19T08:43:56.576308+0000","last_peered":"2024-09-19T08:43:56.576308+0000","last_clean":"2024-09-19T08:43:56.576308+0000","last_became_active":"2024-09-19T08:43:55.170608+0000","last_became_peered":"2024-09-19T08:43:55.170608+0000","last_unstale":"2024-09-19T08:43:56.576308+0000","last_undegraded":"2024-09-19T08:43:56.576308+0000","last_fullsized":"2024-09-19T08:43:56.576308+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:45:37.312790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.737173+0000","last_change":"2024-09-19T08:43:55.186309+0000","last_active":"2024-09-19T08:43:56.737173+0000","last_peered":"2024-09-19T08:43:56.737173+0000","last_clean":"2024-09-19T08:43:56.737173+0000","last_became_active":"2024-09-19T08:43:55.185612+0000","last_became_peered":"2024-09-19T08:43:55.185612+0000","last_unstale":"2024-09-19T08:43:56.737173+0000","last_undegraded":"2024-09-19T08:43:56.737173+0000","last_fullsized":"2024-09-19T08:43:56.737173+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:55:18.505199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.160111+0000","last_change":"2024-09-19T08:43:55.206052+0000","last_active":"2024-09-19T08:43:56.160111+0000","last_peered":"2024-09-19T08:43:56.160111+0000","last_clean":"2024-09-19T08:43:56.160111+0000","last_became_active":"2024-09-19T08:43:55.205325+0000","last_became_peered":"2024-09-19T08:43:55.205325+0000","last_unstale":"2024-09-19T08:43:56.160111+0000","last_undegraded":"2024-09-19T08:43:56.160111+0000","last_fullsized":"2024-09-19T08:43:56.160111+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:54:49.099389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.170072+0000","last_change":"2024-09-19T08:43:55.193647+0000","last_active":"2024-09-19T08:43:56.170072+0000","last_peered":"2024-09-19T08:43:56.170072+0000","last_clean":"2024-09-19T08:43:56.170072+0000","last_became_active":"2024-09-19T08:43:55.192947+0000","last_became_peered":"2024-09-19T08:43:55.192947+0000","last_unstale":"2024-09-19T08:43:56.170072+0000","last_undegraded":"2024-09-19T08:43:56.170072+0000","last_fullsized":"2024-09-19T08:43:56.170072+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:56.087733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.155719+0000","last_change":"2024-09-19T08:43:55.194691+0000","last_active":"2024-09-19T08:43:56.155719+0000","last_peered":"2024-09-19T08:43:56.155719+0000","last_clean":"2024-09-19T08:43:56.155719+0000","last_became_active":"2024-09-19T08:43:55.194441+0000","last_became_peered":"2024-09-19T08:43:55.194441+0000","last_unstale":"2024-09-19T08:43:56.155719+0000","last_undegraded":"2024-09-19T08:43:56.155719+0000","last_fullsized":"2024-09-19T08:43:56.155719+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:06:45.109119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.575520+0000","last_change":"2024-09-19T08:43:55.175081+0000","last_active":"2024-09-19T08:43:56.575520+0000","last_peered":"2024-09-19T08:43:56.575520+0000","last_clean":"2024-09-19T08:43:56.575520+0000","last_became_active":"2024-09-19T08:43:55.170833+0000","last_became_peered":"2024-09-19T08:43:55.170833+0000","last_unstale":"2024-09-19T08:43:56.575520+0000","last_undegraded":"2024-09-19T08:43:56.575520+0000","last_fullsized":"2024-09-19T08:43:56.575520+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:23.675291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.576479+0000","last_change":"2024-09-19T08:43:55.176053+0000","last_active":"2024-09-19T08:43:56.576479+0000","last_peered":"2024-09-19T08:43:56.576479+0000","last_clean":"2024-09-19T08:43:56.576479+0000","last_became_active":"2024-09-19T08:43:55.175394+0000","last_became_peered":"2024-09-19T08:43:55.175394+0000","last_unstale":"2024-09-19T08:43:56.576479+0000","last_undegraded":"2024-09-19T08:43:56.576479+0000","last_fullsized":"2024-09-19T08:43:56.576479+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:46:30.215239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"101'4","reported_seq":18,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.170239+0000","last_change":"2024-09-19T08:43:54.173670+0000","last_active":"2024-09-19T08:43:56.170239+0000","last_peered":"2024-09-19T08:43:56.170239+0000","last_clean":"2024-09-19T08:43:56.170239+0000","last_became_active":"2024-09-19T08:43:54.172904+0000","last_became_peered":"2024-09-19T08:43:54.172904+0000","last_unstale":"2024-09-19T08:43:56.170239+0000","last_undegraded":"2024-09-19T08:43:56.170239+0000","last_fullsized":"2024-09-19T08:43:56.170239+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:22.425741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.165640+0000","last_change":"2024-09-19T08:43:55.187361+0000","last_active":"2024-09-19T08:43:56.165640+0000","last_peered":"2024-09-19T08:43:56.165640+0000","last_clean":"2024-09-19T08:43:56.165640+0000","last_became_active":"2024-09-19T08:43:55.186144+0000","last_became_peered":"2024-09-19T08:43:55.186144+0000","last_unstale":"2024-09-19T08:43:56.165640+0000","last_undegraded":"2024-09-19T08:43:56.165640+0000","last_fullsized":"2024-09-19T08:43:56.165640+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:30:19.243180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"101'1","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736331+0000","last_change":"2024-09-19T08:43:54.175936+0000","last_active":"2024-09-19T08:43:56.736331+0000","last_peered":"2024-09-19T08:43:56.736331+0000","last_clean":"2024-09-19T08:43:56.736331+0000","last_became_active":"2024-09-19T08:43:54.175215+0000","last_became_peered":"2024-09-19T08:43:54.175215+0000","last_unstale":"2024-09-19T08:43:56.736331+0000","last_undegraded":"2024-09-19T08:43:56.736331+0000","last_fullsized":"2024-09-19T08:43:56.736331+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:27:33.779456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.576311+0000","last_change":"2024-09-19T08:43:55.182209+0000","last_active":"2024-09-19T08:43:56.576311+0000","last_peered":"2024-09-19T08:43:56.576311+0000","last_clean":"2024-09-19T08:43:56.576311+0000","last_became_active":"2024-09-19T08:43:55.181787+0000","last_became_peered":"2024-09-19T08:43:55.181787+0000","last_unstale":"2024-09-19T08:43:56.576311+0000","last_undegraded":"2024-09-19T08:43:56.576311+0000","last_fullsized":"2024-09-19T08:43:56.576311+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:54:28.125782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"101'2","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736070+0000","last_change":"2024-09-19T08:43:54.148059+0000","last_active":"2024-09-19T08:43:56.736070+0000","last_peered":"2024-09-19T08:43:56.736070+0000","last_clean":"2024-09-19T08:43:56.736070+0000","last_became_active":"2024-09-19T08:43:54.147311+0000","last_became_peered":"2024-09-19T08:43:54.147311+0000","last_unstale":"2024-09-19T08:43:56.736070+0000","last_undegraded":"2024-09-19T08:43:56.736070+0000","last_fullsized":"2024-09-19T08:43:56.736070+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:59:59.076766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.737066+0000","last_change":"2024-09-19T08:43:55.181960+0000","last_active":"2024-09-19T08:43:56.737066+0000","last_peered":"2024-09-19T08:43:56.737066+0000","last_clean":"2024-09-19T08:43:56.737066+0000","last_became_active":"2024-09-19T08:43:55.181038+0000","last_became_peered":"2024-09-19T08:43:55.181038+0000","last_unstale":"2024-09-19T08:43:56.737066+0000","last_undegraded":"2024-09-19T08:43:56.737066+0000","last_fullsized":"2024-09-19T08:43:56.737066+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:41.369862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"101'2","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.735543+0000","last_change":"2024-09-19T08:43:54.148088+0000","last_active":"2024-09-19T08:43:56.735543+0000","last_peered":"2024-09-19T08:43:56.735543+0000","last_clean":"2024-09-19T08:43:56.735543+0000","last_became_active":"2024-09-19T08:43:54.147729+0000","last_became_peered":"2024-09-19T08:43:54.147729+0000","last_unstale":"2024-09-19T08:43:56.735543+0000","last_undegraded":"2024-09-19T08:43:56.735543+0000","last_fullsized":"2024-09-19T08:43:56.735543+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:01:57.810659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.155662+0000","last_change":"2024-09-19T08:43:55.189242+0000","last_active":"2024-09-19T08:43:56.155662+0000","last_peered":"2024-09-19T08:43:56.155662+0000","last_clean":"2024-09-19T08:43:56.155662+0000","last_became_active":"2024-09-19T08:43:55.188976+0000","last_became_peered":"2024-09-19T08:43:55.188976+0000","last_unstale":"2024-09-19T08:43:56.155662+0000","last_undegraded":"2024-09-19T08:43:56.155662+0000","last_fullsized":"2024-09-19T08:43:56.155662+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:05.505122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"101'1","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.169629+0000","last_change":"2024-09-19T08:43:54.174001+0000","last_active":"2024-09-19T08:43:56.169629+0000","last_peered":"2024-09-19T08:43:56.169629+0000","last_clean":"2024-09-19T08:43:56.169629+0000","last_became_active":"2024-09-19T08:43:54.171901+0000","last_became_peered":"2024-09-19T08:43:54.171901+0000","last_unstale":"2024-09-19T08:43:56.169629+0000","last_undegraded":"2024-09-19T08:43:56.169629+0000","last_fullsized":"2024-09-19T08:43:56.169629+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:22.665765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.160211+0000","last_change":"2024-09-19T08:43:55.205723+0000","last_active":"2024-09-19T08:43:56.160211+0000","last_peered":"2024-09-19T08:43:56.160211+0000","last_clean":"2024-09-19T08:43:56.160211+0000","last_became_active":"2024-09-19T08:43:55.205248+0000","last_became_peered":"2024-09-19T08:43:55.205248+0000","last_unstale":"2024-09-19T08:43:56.160211+0000","last_undegraded":"2024-09-19T08:43:56.160211+0000","last_fullsized":"2024-09-19T08:43:56.160211+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:15.340286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"101'1","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.735857+0000","last_change":"2024-09-19T08:43:54.169422+0000","last_active":"2024-09-19T08:43:56.735857+0000","last_peered":"2024-09-19T08:43:56.735857+0000","last_clean":"2024-09-19T08:43:56.735857+0000","last_became_active":"2024-09-19T08:43:54.169152+0000","last_became_peered":"2024-09-19T08:43:54.169152+0000","last_unstale":"2024-09-19T08:43:56.735857+0000","last_undegraded":"2024-09-19T08:43:56.735857+0000","last_fullsized":"2024-09-19T08:43:56.735857+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:54:49.167389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.155373+0000","last_change":"2024-09-19T08:43:55.178286+0000","last_active":"2024-09-19T08:43:56.155373+0000","last_peered":"2024-09-19T08:43:56.155373+0000","last_clean":"2024-09-19T08:43:56.155373+0000","last_became_active":"2024-09-19T08:43:55.177900+0000","last_became_peered":"2024-09-19T08:43:55.177900+0000","last_unstale":"2024-09-19T08:43:56.155373+0000","last_undegraded":"2024-09-19T08:43:56.155373+0000","last_fullsized":"2024-09-19T08:43:56.155373+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:08:10.535405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.159848+0000","last_change":"2024-09-19T08:43:54.170080+0000","last_active":"2024-09-19T08:43:56.159848+0000","last_peered":"2024-09-19T08:43:56.159848+0000","last_clean":"2024-09-19T08:43:56.159848+0000","last_became_active":"2024-09-19T08:43:54.168481+0000","last_became_peered":"2024-09-19T08:43:54.168481+0000","last_unstale":"2024-09-19T08:43:56.159848+0000","last_undegraded":"2024-09-19T08:43:56.159848+0000","last_fullsized":"2024-09-19T08:43:56.159848+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:55.078207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.169711+0000","last_change":"2024-09-19T08:43:55.195850+0000","last_active":"2024-09-19T08:43:56.169711+0000","last_peered":"2024-09-19T08:43:56.169711+0000","last_clean":"2024-09-19T08:43:56.169711+0000","last_became_active":"2024-09-19T08:43:55.194786+0000","last_became_peered":"2024-09-19T08:43:55.194786+0000","last_unstale":"2024-09-19T08:43:56.169711+0000","last_undegraded":"2024-09-19T08:43:56.169711+0000","last_fullsized":"2024-09-19T08:43:56.169711+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:31:59.684059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.155893+0000","last_change":"2024-09-19T08:43:54.158627+0000","last_active":"2024-09-19T08:43:56.155893+0000","last_peered":"2024-09-19T08:43:56.155893+0000","last_clean":"2024-09-19T08:43:56.155893+0000","last_became_active":"2024-09-19T08:43:54.158333+0000","last_became_peered":"2024-09-19T08:43:54.158333+0000","last_unstale":"2024-09-19T08:43:56.155893+0000","last_undegraded":"2024-09-19T08:43:56.155893+0000","last_fullsized":"2024-09-19T08:43:56.155893+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:31:30.306491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.170558+0000","last_change":"2024-09-19T08:43:55.195678+0000","last_active":"2024-09-19T08:43:56.170558+0000","last_peered":"2024-09-19T08:43:56.170558+0000","last_clean":"2024-09-19T08:43:56.170558+0000","last_became_active":"2024-09-19T08:43:55.194952+0000","last_became_peered":"2024-09-19T08:43:55.194952+0000","last_unstale":"2024-09-19T08:43:56.170558+0000","last_undegraded":"2024-09-19T08:43:56.170558+0000","last_fullsized":"2024-09-19T08:43:56.170558+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:37.648111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736244+0000","last_change":"2024-09-19T08:43:54.148128+0000","last_active":"2024-09-19T08:43:56.736244+0000","last_peered":"2024-09-19T08:43:56.736244+0000","last_clean":"2024-09-19T08:43:56.736244+0000","last_became_active":"2024-09-19T08:43:54.147519+0000","last_became_peered":"2024-09-19T08:43:54.147519+0000","last_unstale":"2024-09-19T08:43:56.736244+0000","last_undegraded":"2024-09-19T08:43:56.736244+0000","last_fullsized":"2024-09-19T08:43:56.736244+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:04:22.016732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.165566+0000","last_change":"2024-09-19T08:43:55.190134+0000","last_active":"2024-09-19T08:43:56.165566+0000","last_peered":"2024-09-19T08:43:56.165566+0000","last_clean":"2024-09-19T08:43:56.165566+0000","last_became_active":"2024-09-19T08:43:55.189377+0000","last_became_peered":"2024-09-19T08:43:55.189377+0000","last_unstale":"2024-09-19T08:43:56.165566+0000","last_undegraded":"2024-09-19T08:43:56.165566+0000","last_fullsized":"2024-09-19T08:43:56.165566+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:15:39.583209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736887+0000","last_change":"2024-09-19T08:43:54.169583+0000","last_active":"2024-09-19T08:43:56.736887+0000","last_peered":"2024-09-19T08:43:56.736887+0000","last_clean":"2024-09-19T08:43:56.736887+0000","last_became_active":"2024-09-19T08:43:54.169238+0000","last_became_peered":"2024-09-19T08:43:54.169238+0000","last_unstale":"2024-09-19T08:43:56.736887+0000","last_undegraded":"2024-09-19T08:43:56.736887+0000","last_fullsized":"2024-09-19T08:43:56.736887+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:42.579423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.737558+0000","last_change":"2024-09-19T08:43:55.200840+0000","last_active":"2024-09-19T08:43:56.737558+0000","last_peered":"2024-09-19T08:43:56.737558+0000","last_clean":"2024-09-19T08:43:56.737558+0000","last_became_active":"2024-09-19T08:43:55.200586+0000","last_became_peered":"2024-09-19T08:43:55.200586+0000","last_unstale":"2024-09-19T08:43:56.737558+0000","last_undegraded":"2024-09-19T08:43:56.737558+0000","last_fullsized":"2024-09-19T08:43:56.737558+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:22.019255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"101'3","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.155459+0000","last_change":"2024-09-19T08:43:54.156828+0000","last_active":"2024-09-19T08:43:56.155459+0000","last_peered":"2024-09-19T08:43:56.155459+0000","last_clean":"2024-09-19T08:43:56.155459+0000","last_became_active":"2024-09-19T08:43:54.156521+0000","last_became_peered":"2024-09-19T08:43:54.156521+0000","last_unstale":"2024-09-19T08:43:56.155459+0000","last_undegraded":"2024-09-19T08:43:56.155459+0000","last_fullsized":"2024-09-19T08:43:56.155459+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:51:41.493803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.170290+0000","last_change":"2024-09-19T08:43:55.205270+0000","last_active":"2024-09-19T08:43:56.170290+0000","last_peered":"2024-09-19T08:43:56.170290+0000","last_clean":"2024-09-19T08:43:56.170290+0000","last_became_active":"2024-09-19T08:43:55.204556+0000","last_became_peered":"2024-09-19T08:43:55.204556+0000","last_unstale":"2024-09-19T08:43:56.170290+0000","last_undegraded":"2024-09-19T08:43:56.170290+0000","last_fullsized":"2024-09-19T08:43:56.170290+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:53:33.564978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"101'6","reported_seq":20,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.156666+0000","last_change":"2024-09-19T08:43:54.159068+0000","last_active":"2024-09-19T08:43:56.156666+0000","last_peered":"2024-09-19T08:43:56.156666+0000","last_clean":"2024-09-19T08:43:56.156666+0000","last_became_active":"2024-09-19T08:43:54.158698+0000","last_became_peered":"2024-09-19T08:43:54.158698+0000","last_unstale":"2024-09-19T08:43:56.156666+0000","last_undegraded":"2024-09-19T08:43:56.156666+0000","last_fullsized":"2024-09-19T08:43:56.156666+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:39:01.903420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736353+0000","last_change":"2024-09-19T08:43:55.194646+0000","last_active":"2024-09-19T08:43:56.736353+0000","last_peered":"2024-09-19T08:43:56.736353+0000","last_clean":"2024-09-19T08:43:56.736353+0000","last_became_active":"2024-09-19T08:43:55.194235+0000","last_became_peered":"2024-09-19T08:43:55.194235+0000","last_unstale":"2024-09-19T08:43:56.736353+0000","last_undegraded":"2024-09-19T08:43:56.736353+0000","last_fullsized":"2024-09-19T08:43:56.736353+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:06:29.306359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"101'2","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.575865+0000","last_change":"2024-09-19T08:43:54.165780+0000","last_active":"2024-09-19T08:43:56.575865+0000","last_peered":"2024-09-19T08:43:56.575865+0000","last_clean":"2024-09-19T08:43:56.575865+0000","last_became_active":"2024-09-19T08:43:54.165527+0000","last_became_peered":"2024-09-19T08:43:54.165527+0000","last_unstale":"2024-09-19T08:43:56.575865+0000","last_undegraded":"2024-09-19T08:43:56.575865+0000","last_fullsized":"2024-09-19T08:43:56.575865+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:18.059525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736953+0000","last_change":"2024-09-19T08:43:55.167204+0000","last_active":"2024-09-19T08:43:56.736953+0000","last_peered":"2024-09-19T08:43:56.736953+0000","last_clean":"2024-09-19T08:43:56.736953+0000","last_became_active":"2024-09-19T08:43:55.166818+0000","last_became_peered":"2024-09-19T08:43:55.166818+0000","last_unstale":"2024-09-19T08:43:56.736953+0000","last_undegraded":"2024-09-19T08:43:56.736953+0000","last_fullsized":"2024-09-19T08:43:56.736953+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:08:12.006209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.164934+0000","last_change":"2024-09-19T08:43:54.167563+0000","last_active":"2024-09-19T08:43:56.164934+0000","last_peered":"2024-09-19T08:43:56.164934+0000","last_clean":"2024-09-19T08:43:56.164934+0000","last_became_active":"2024-09-19T08:43:54.166815+0000","last_became_peered":"2024-09-19T08:43:54.166815+0000","last_unstale":"2024-09-19T08:43:56.164934+0000","last_undegraded":"2024-09-19T08:43:56.164934+0000","last_fullsized":"2024-09-19T08:43:56.164934+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:03.023614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.735248+0000","last_change":"2024-09-19T08:43:55.200724+0000","last_active":"2024-09-19T08:43:56.735248+0000","last_peered":"2024-09-19T08:43:56.735248+0000","last_clean":"2024-09-19T08:43:56.735248+0000","last_became_active":"2024-09-19T08:43:55.200505+0000","last_became_peered":"2024-09-19T08:43:55.200505+0000","last_unstale":"2024-09-19T08:43:56.735248+0000","last_undegraded":"2024-09-19T08:43:56.735248+0000","last_fullsized":"2024-09-19T08:43:56.735248+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:51:14.085642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"101'8","reported_seq":22,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.735407+0000","last_change":"2024-09-19T08:43:54.153224+0000","last_active":"2024-09-19T08:43:56.735407+0000","last_peered":"2024-09-19T08:43:56.735407+0000","last_clean":"2024-09-19T08:43:56.735407+0000","last_became_active":"2024-09-19T08:43:54.152861+0000","last_became_peered":"2024-09-19T08:43:54.152861+0000","last_unstale":"2024-09-19T08:43:56.735407+0000","last_undegraded":"2024-09-19T08:43:56.735407+0000","last_fullsized":"2024-09-19T08:43:56.735407+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:28.306074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736714+0000","last_change":"2024-09-19T08:43:55.179921+0000","last_active":"2024-09-19T08:43:56.736714+0000","last_peered":"2024-09-19T08:43:56.736714+0000","last_clean":"2024-09-19T08:43:56.736714+0000","last_became_active":"2024-09-19T08:43:55.179672+0000","last_became_peered":"2024-09-19T08:43:55.179672+0000","last_unstale":"2024-09-19T08:43:56.736714+0000","last_undegraded":"2024-09-19T08:43:56.736714+0000","last_fullsized":"2024-09-19T08:43:56.736714+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:03.688703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"101'1","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.157106+0000","last_change":"2024-09-19T08:43:54.175081+0000","last_active":"2024-09-19T08:43:56.157106+0000","last_peered":"2024-09-19T08:43:56.157106+0000","last_clean":"2024-09-19T08:43:56.157106+0000","last_became_active":"2024-09-19T08:43:54.174828+0000","last_became_peered":"2024-09-19T08:43:54.174828+0000","last_unstale":"2024-09-19T08:43:56.157106+0000","last_undegraded":"2024-09-19T08:43:56.157106+0000","last_fullsized":"2024-09-19T08:43:56.157106+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:42:04.302601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.576192+0000","last_change":"2024-09-19T08:43:55.169745+0000","last_active":"2024-09-19T08:43:56.576192+0000","last_peered":"2024-09-19T08:43:56.576192+0000","last_clean":"2024-09-19T08:43:56.576192+0000","last_became_active":"2024-09-19T08:43:55.169407+0000","last_became_peered":"2024-09-19T08:43:55.169407+0000","last_unstale":"2024-09-19T08:43:56.576192+0000","last_undegraded":"2024-09-19T08:43:56.576192+0000","last_fullsized":"2024-09-19T08:43:56.576192+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:26:01.318672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"101'2","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736434+0000","last_change":"2024-09-19T08:43:54.167839+0000","last_active":"2024-09-19T08:43:56.736434+0000","last_peered":"2024-09-19T08:43:56.736434+0000","last_clean":"2024-09-19T08:43:56.736434+0000","last_became_active":"2024-09-19T08:43:54.167548+0000","last_became_peered":"2024-09-19T08:43:54.167548+0000","last_unstale":"2024-09-19T08:43:56.736434+0000","last_undegraded":"2024-09-19T08:43:56.736434+0000","last_fullsized":"2024-09-19T08:43:56.736434+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:37.105176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.737017+0000","last_change":"2024-09-19T08:43:55.186681+0000","last_active":"2024-09-19T08:43:56.737017+0000","last_peered":"2024-09-19T08:43:56.737017+0000","last_clean":"2024-09-19T08:43:56.737017+0000","last_became_active":"2024-09-19T08:43:55.185978+0000","last_became_peered":"2024-09-19T08:43:55.185978+0000","last_unstale":"2024-09-19T08:43:56.737017+0000","last_undegraded":"2024-09-19T08:43:56.737017+0000","last_fullsized":"2024-09-19T08:43:56.737017+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:51:42.503330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"101'1","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.165383+0000","last_change":"2024-09-19T08:43:54.166019+0000","last_active":"2024-09-19T08:43:56.165383+0000","last_peered":"2024-09-19T08:43:56.165383+0000","last_clean":"2024-09-19T08:43:56.165383+0000","last_became_active":"2024-09-19T08:43:54.165330+0000","last_became_peered":"2024-09-19T08:43:54.165330+0000","last_unstale":"2024-09-19T08:43:56.165383+0000","last_undegraded":"2024-09-19T08:43:56.165383+0000","last_fullsized":"2024-09-19T08:43:56.165383+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:02:58.278139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736946+0000","last_change":"2024-09-19T08:43:55.204016+0000","last_active":"2024-09-19T08:43:56.736946+0000","last_peered":"2024-09-19T08:43:56.736946+0000","last_clean":"2024-09-19T08:43:56.736946+0000","last_became_active":"2024-09-19T08:43:55.203041+0000","last_became_peered":"2024-09-19T08:43:55.203041+0000","last_unstale":"2024-09-19T08:43:56.736946+0000","last_undegraded":"2024-09-19T08:43:56.736946+0000","last_fullsized":"2024-09-19T08:43:56.736946+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:31:11.824242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.169927+0000","last_change":"2024-09-19T08:43:54.171525+0000","last_active":"2024-09-19T08:43:56.169927+0000","last_peered":"2024-09-19T08:43:56.169927+0000","last_clean":"2024-09-19T08:43:56.169927+0000","last_became_active":"2024-09-19T08:43:54.170720+0000","last_became_peered":"2024-09-19T08:43:54.170720+0000","last_unstale":"2024-09-19T08:43:56.169927+0000","last_undegraded":"2024-09-19T08:43:56.169927+0000","last_fullsized":"2024-09-19T08:43:56.169927+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:26:00.309146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736210+0000","last_change":"2024-09-19T08:43:55.191951+0000","last_active":"2024-09-19T08:43:56.736210+0000","last_peered":"2024-09-19T08:43:56.736210+0000","last_clean":"2024-09-19T08:43:56.736210+0000","last_became_active":"2024-09-19T08:43:55.191187+0000","last_became_peered":"2024-09-19T08:43:55.191187+0000","last_unstale":"2024-09-19T08:43:56.736210+0000","last_undegraded":"2024-09-19T08:43:56.736210+0000","last_fullsized":"2024-09-19T08:43:56.736210+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:09:54.189345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"101'2","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.575972+0000","last_change":"2024-09-19T08:43:54.156834+0000","last_active":"2024-09-19T08:43:56.575972+0000","last_peered":"2024-09-19T08:43:56.575972+0000","last_clean":"2024-09-19T08:43:56.575972+0000","last_became_active":"2024-09-19T08:43:54.156436+0000","last_became_peered":"2024-09-19T08:43:54.156436+0000","last_unstale":"2024-09-19T08:43:56.575972+0000","last_undegraded":"2024-09-19T08:43:56.575972+0000","last_fullsized":"2024-09-19T08:43:56.575972+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:06:49.884305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.155968+0000","last_change":"2024-09-19T08:43:55.180818+0000","last_active":"2024-09-19T08:43:56.155968+0000","last_peered":"2024-09-19T08:43:56.155968+0000","last_clean":"2024-09-19T08:43:56.155968+0000","last_became_active":"2024-09-19T08:43:55.180523+0000","last_became_peered":"2024-09-19T08:43:55.180523+0000","last_unstale":"2024-09-19T08:43:56.155968+0000","last_undegraded":"2024-09-19T08:43:56.155968+0000","last_fullsized":"2024-09-19T08:43:56.155968+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:19:35.054836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.735879+0000","last_change":"2024-09-19T08:43:54.148186+0000","last_active":"2024-09-19T08:43:56.735879+0000","last_peered":"2024-09-19T08:43:56.735879+0000","last_clean":"2024-09-19T08:43:56.735879+0000","last_became_active":"2024-09-19T08:43:54.147929+0000","last_became_peered":"2024-09-19T08:43:54.147929+0000","last_unstale":"2024-09-19T08:43:56.735879+0000","last_undegraded":"2024-09-19T08:43:56.735879+0000","last_fullsized":"2024-09-19T08:43:56.735879+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:22:11.522698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736419+0000","last_change":"2024-09-19T08:43:55.185737+0000","last_active":"2024-09-19T08:43:56.736419+0000","last_peered":"2024-09-19T08:43:56.736419+0000","last_clean":"2024-09-19T08:43:56.736419+0000","last_became_active":"2024-09-19T08:43:55.184994+0000","last_became_peered":"2024-09-19T08:43:55.184994+0000","last_unstale":"2024-09-19T08:43:56.736419+0000","last_undegraded":"2024-09-19T08:43:56.736419+0000","last_fullsized":"2024-09-19T08:43:56.736419+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:54:49.099389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.575901+0000","last_change":"2024-09-19T08:43:54.156400+0000","last_active":"2024-09-19T08:43:56.575901+0000","last_peered":"2024-09-19T08:43:56.575901+0000","last_clean":"2024-09-19T08:43:56.575901+0000","last_became_active":"2024-09-19T08:43:54.156000+0000","last_became_peered":"2024-09-19T08:43:54.156000+0000","last_unstale":"2024-09-19T08:43:56.575901+0000","last_undegraded":"2024-09-19T08:43:56.575901+0000","last_fullsized":"2024-09-19T08:43:56.575901+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:11.225788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736338+0000","last_change":"2024-09-19T08:43:55.181342+0000","last_active":"2024-09-19T08:43:56.736338+0000","last_peered":"2024-09-19T08:43:56.736338+0000","last_clean":"2024-09-19T08:43:56.736338+0000","last_became_active":"2024-09-19T08:43:55.181018+0000","last_became_peered":"2024-09-19T08:43:55.181018+0000","last_unstale":"2024-09-19T08:43:56.736338+0000","last_undegraded":"2024-09-19T08:43:56.736338+0000","last_fullsized":"2024-09-19T08:43:56.736338+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:16.137533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.155966+0000","last_change":"2024-09-19T08:43:54.164739+0000","last_active":"2024-09-19T08:43:56.155966+0000","last_peered":"2024-09-19T08:43:56.155966+0000","last_clean":"2024-09-19T08:43:56.155966+0000","last_became_active":"2024-09-19T08:43:54.164107+0000","last_became_peered":"2024-09-19T08:43:54.164107+0000","last_unstale":"2024-09-19T08:43:56.155966+0000","last_undegraded":"2024-09-19T08:43:56.155966+0000","last_fullsized":"2024-09-19T08:43:56.155966+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:37:18.720301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.165560+0000","last_change":"2024-09-19T08:43:55.190836+0000","last_active":"2024-09-19T08:43:56.165560+0000","last_peered":"2024-09-19T08:43:56.165560+0000","last_clean":"2024-09-19T08:43:56.165560+0000","last_became_active":"2024-09-19T08:43:55.190000+0000","last_became_peered":"2024-09-19T08:43:55.190000+0000","last_unstale":"2024-09-19T08:43:56.165560+0000","last_undegraded":"2024-09-19T08:43:56.165560+0000","last_fullsized":"2024-09-19T08:43:56.165560+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:38:24.253914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"101'2","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.737656+0000","last_change":"2024-09-19T08:43:54.156950+0000","last_active":"2024-09-19T08:43:56.737656+0000","last_peered":"2024-09-19T08:43:56.737656+0000","last_clean":"2024-09-19T08:43:56.737656+0000","last_became_active":"2024-09-19T08:43:54.156680+0000","last_became_peered":"2024-09-19T08:43:54.156680+0000","last_unstale":"2024-09-19T08:43:56.737656+0000","last_undegraded":"2024-09-19T08:43:56.737656+0000","last_fullsized":"2024-09-19T08:43:56.737656+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:17:05.043117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.156975+0000","last_change":"2024-09-19T08:43:55.187342+0000","last_active":"2024-09-19T08:43:56.156975+0000","last_peered":"2024-09-19T08:43:56.156975+0000","last_clean":"2024-09-19T08:43:56.156975+0000","last_became_active":"2024-09-19T08:43:55.186731+0000","last_became_peered":"2024-09-19T08:43:55.186731+0000","last_unstale":"2024-09-19T08:43:56.156975+0000","last_undegraded":"2024-09-19T08:43:56.156975+0000","last_fullsized":"2024-09-19T08:43:56.156975+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:49.028651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"102'1","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.165662+0000","last_change":"2024-09-19T08:43:54.175992+0000","last_active":"2024-09-19T08:43:56.165662+0000","last_peered":"2024-09-19T08:43:56.165662+0000","last_clean":"2024-09-19T08:43:56.165662+0000","last_became_active":"2024-09-19T08:43:54.175245+0000","last_became_peered":"2024-09-19T08:43:54.175245+0000","last_unstale":"2024-09-19T08:43:56.165662+0000","last_undegraded":"2024-09-19T08:43:56.165662+0000","last_fullsized":"2024-09-19T08:43:56.165662+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:14.806379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.156584+0000","last_change":"2024-09-19T08:43:55.183466+0000","last_active":"2024-09-19T08:43:56.156584+0000","last_peered":"2024-09-19T08:43:56.156584+0000","last_clean":"2024-09-19T08:43:56.156584+0000","last_became_active":"2024-09-19T08:43:55.183213+0000","last_became_peered":"2024-09-19T08:43:55.183213+0000","last_unstale":"2024-09-19T08:43:56.156584+0000","last_undegraded":"2024-09-19T08:43:56.156584+0000","last_fullsized":"2024-09-19T08:43:56.156584+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:07:50.043881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.155580+0000","last_change":"2024-09-19T08:43:54.166062+0000","last_active":"2024-09-19T08:43:56.155580+0000","last_peered":"2024-09-19T08:43:56.155580+0000","last_clean":"2024-09-19T08:43:56.155580+0000","last_became_active":"2024-09-19T08:43:54.165346+0000","last_became_peered":"2024-09-19T08:43:54.165346+0000","last_unstale":"2024-09-19T08:43:56.155580+0000","last_undegraded":"2024-09-19T08:43:56.155580+0000","last_fullsized":"2024-09-19T08:43:56.155580+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:40.360335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.165461+0000","last_change":"2024-09-19T08:43:55.190220+0000","last_active":"2024-09-19T08:43:56.165461+0000","last_peered":"2024-09-19T08:43:56.165461+0000","last_clean":"2024-09-19T08:43:56.165461+0000","last_became_active":"2024-09-19T08:43:55.188987+0000","last_became_peered":"2024-09-19T08:43:55.188987+0000","last_unstale":"2024-09-19T08:43:56.165461+0000","last_undegraded":"2024-09-19T08:43:56.165461+0000","last_fullsized":"2024-09-19T08:43:56.165461+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:06:59.363181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.737357+0000","last_change":"2024-09-19T08:43:54.157928+0000","last_active":"2024-09-19T08:43:56.737357+0000","last_peered":"2024-09-19T08:43:56.737357+0000","last_clean":"2024-09-19T08:43:56.737357+0000","last_became_active":"2024-09-19T08:43:54.157571+0000","last_became_peered":"2024-09-19T08:43:54.157571+0000","last_unstale":"2024-09-19T08:43:56.737357+0000","last_undegraded":"2024-09-19T08:43:56.737357+0000","last_fullsized":"2024-09-19T08:43:56.737357+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:31.692173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.161089+0000","last_change":"2024-09-19T08:43:55.205294+0000","last_active":"2024-09-19T08:43:56.161089+0000","last_peered":"2024-09-19T08:43:56.161089+0000","last_clean":"2024-09-19T08:43:56.161089+0000","last_became_active":"2024-09-19T08:43:55.204683+0000","last_became_peered":"2024-09-19T08:43:55.204683+0000","last_unstale":"2024-09-19T08:43:56.161089+0000","last_undegraded":"2024-09-19T08:43:56.161089+0000","last_fullsized":"2024-09-19T08:43:56.161089+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:39:02.010421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.169697+0000","last_change":"2024-09-19T08:43:54.174126+0000","last_active":"2024-09-19T08:43:56.169697+0000","last_peered":"2024-09-19T08:43:56.169697+0000","last_clean":"2024-09-19T08:43:56.169697+0000","last_became_active":"2024-09-19T08:43:54.173403+0000","last_became_peered":"2024-09-19T08:43:54.173403+0000","last_unstale":"2024-09-19T08:43:56.169697+0000","last_undegraded":"2024-09-19T08:43:56.169697+0000","last_fullsized":"2024-09-19T08:43:56.169697+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:29:30.346662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.160539+0000","last_change":"2024-09-19T08:43:55.204555+0000","last_active":"2024-09-19T08:43:56.160539+0000","last_peered":"2024-09-19T08:43:56.160539+0000","last_clean":"2024-09-19T08:43:56.160539+0000","last_became_active":"2024-09-19T08:43:55.201572+0000","last_became_peered":"2024-09-19T08:43:55.201572+0000","last_unstale":"2024-09-19T08:43:56.160539+0000","last_undegraded":"2024-09-19T08:43:56.160539+0000","last_fullsized":"2024-09-19T08:43:56.160539+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:23.679385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.160261+0000","last_change":"2024-09-19T08:43:54.161229+0000","last_active":"2024-09-19T08:43:56.160261+0000","last_peered":"2024-09-19T08:43:56.160261+0000","last_clean":"2024-09-19T08:43:56.160261+0000","last_became_active":"2024-09-19T08:43:54.146726+0000","last_became_peered":"2024-09-19T08:43:54.146726+0000","last_unstale":"2024-09-19T08:43:56.160261+0000","last_undegraded":"2024-09-19T08:43:56.160261+0000","last_fullsized":"2024-09-19T08:43:56.160261+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:27.079573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736911+0000","last_change":"2024-09-19T08:43:55.180738+0000","last_active":"2024-09-19T08:43:56.736911+0000","last_peered":"2024-09-19T08:43:56.736911+0000","last_clean":"2024-09-19T08:43:56.736911+0000","last_became_active":"2024-09-19T08:43:55.179169+0000","last_became_peered":"2024-09-19T08:43:55.179169+0000","last_unstale":"2024-09-19T08:43:56.736911+0000","last_undegraded":"2024-09-19T08:43:56.736911+0000","last_fullsized":"2024-09-19T08:43:56.736911+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:23.679385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.735440+0000","last_change":"2024-09-19T08:43:54.148684+0000","last_active":"2024-09-19T08:43:56.735440+0000","last_peered":"2024-09-19T08:43:56.735440+0000","last_clean":"2024-09-19T08:43:56.735440+0000","last_became_active":"2024-09-19T08:43:54.148479+0000","last_became_peered":"2024-09-19T08:43:54.148479+0000","last_unstale":"2024-09-19T08:43:56.735440+0000","last_undegraded":"2024-09-19T08:43:56.735440+0000","last_fullsized":"2024-09-19T08:43:56.735440+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:30.763525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.155474+0000","last_change":"2024-09-19T08:43:55.181103+0000","last_active":"2024-09-19T08:43:56.155474+0000","last_peered":"2024-09-19T08:43:56.155474+0000","last_clean":"2024-09-19T08:43:56.155474+0000","last_became_active":"2024-09-19T08:43:55.180719+0000","last_became_peered":"2024-09-19T08:43:55.180719+0000","last_unstale":"2024-09-19T08:43:56.155474+0000","last_undegraded":"2024-09-19T08:43:56.155474+0000","last_fullsized":"2024-09-19T08:43:56.155474+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:23.898525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.156061+0000","last_change":"2024-09-19T08:43:54.163982+0000","last_active":"2024-09-19T08:43:56.156061+0000","last_peered":"2024-09-19T08:43:56.156061+0000","last_clean":"2024-09-19T08:43:56.156061+0000","last_became_active":"2024-09-19T08:43:54.163705+0000","last_became_peered":"2024-09-19T08:43:54.163705+0000","last_unstale":"2024-09-19T08:43:56.156061+0000","last_undegraded":"2024-09-19T08:43:56.156061+0000","last_fullsized":"2024-09-19T08:43:56.156061+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:39:47.549316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.735072+0000","last_change":"2024-09-19T08:43:55.192869+0000","last_active":"2024-09-19T08:43:56.735072+0000","last_peered":"2024-09-19T08:43:56.735072+0000","last_clean":"2024-09-19T08:43:56.735072+0000","last_became_active":"2024-09-19T08:43:55.192580+0000","last_became_peered":"2024-09-19T08:43:55.192580+0000","last_unstale":"2024-09-19T08:43:56.735072+0000","last_undegraded":"2024-09-19T08:43:56.735072+0000","last_fullsized":"2024-09-19T08:43:56.735072+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:09.148051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.169923+0000","last_change":"2024-09-19T08:43:54.163349+0000","last_active":"2024-09-19T08:43:56.169923+0000","last_peered":"2024-09-19T08:43:56.169923+0000","last_clean":"2024-09-19T08:43:56.169923+0000","last_became_active":"2024-09-19T08:43:54.162600+0000","last_became_peered":"2024-09-19T08:43:54.162600+0000","last_unstale":"2024-09-19T08:43:56.169923+0000","last_undegraded":"2024-09-19T08:43:56.169923+0000","last_fullsized":"2024-09-19T08:43:56.169923+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:54:27.116255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"71'128","reported_seq":355,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736808+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:43:56.736808+0000","last_peered":"2024-09-19T08:43:56.736808+0000","last_clean":"2024-09-19T08:43:56.736808+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:43:56.736808+0000","last_undegraded":"2024-09-19T08:43:56.736808+0000","last_fullsized":"2024-09-19T08:43:56.736808+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509513,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33012,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338828,"statfs":{"total":96636764160,"available":96602959872,"internally_reserved":0,"allocated":1863680,"data_stored":1145603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574922,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1265664,"data_stored":554689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509515,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1269760,"data_stored":555223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509515,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32420,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1261568,"data_stored":554619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542217,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1277952,"data_stored":556301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509512,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32420,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1261568,"data_stored":554619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509514,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1269760,"data_stored":554743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509512,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33024,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338816,"statfs":{"total":96636764160,"available":96602947584,"internally_reserved":0,"allocated":1875968,"data_stored":1146711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:43:58.959 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T08:43:59.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.100+0000 7f9711658640 1 Processor -- start 2024-09-19T08:43:59.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.100+0000 7f9711658640 1 -- start start 2024-09-19T08:43:59.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.100+0000 7f9711658640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f970c071950 0x7f970c0af580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:59.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.100+0000 7f9711658640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f970c143420 0x7f970c143890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:59.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.100+0000 7f9711658640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f970c06f4e0 con 0x7f970c071d30 2024-09-19T08:43:59.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.100+0000 7f9711658640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f970c0b0320 con 0x7f970c071950 2024-09-19T08:43:59.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.100+0000 7f9711658640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f970c06d710 con 0x7f970c143420 2024-09-19T08:43:59.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.100+0000 7f970b7fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f970c143420 0x7f970c143890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:59.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.100+0000 7f970b7fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f970c143420 0x7f970c143890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:55410/0 (socket says 172.21.15.98:55410) 2024-09-19T08:43:59.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.100+0000 7f970b7fe640 1 -- 172.21.15.98:0/4028137949 learned_addr learned my addr 172.21.15.98:0/4028137949 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:59.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.100+0000 7f970affd640 1 --2- 172.21.15.98:0/4028137949 >> v2:172.21.15.139:3300/0 conn(0x7f970c071950 0x7f970c0af580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:59.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.100+0000 7f970affd640 1 -- 172.21.15.98:0/4028137949 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f970c143420 msgr2=0x7f970c143890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:59.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.100+0000 7f970affd640 1 --2- 172.21.15.98:0/4028137949 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f970c143420 0x7f970c143890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:59.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.100+0000 7f970affd640 1 -- 172.21.15.98:0/4028137949 >> v1:172.21.15.139:6789/0 conn(0x7f970c071d30 legacy=0x7f970c0afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:59.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.100+0000 7f970affd640 1 -- 172.21.15.98:0/4028137949 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f970c06d9f0 con 0x7f970c071950 2024-09-19T08:43:59.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.104+0000 7f970affd640 1 --2- 172.21.15.98:0/4028137949 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f970c071950 0x7f970c0af580 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f97040097c0 tx=0x7f970401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd10e478cbcec51a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:59.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.104+0000 7f9709ffb640 1 -- 172.21.15.98:0/4028137949 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97040a7070 con 0x7f970c071950 2024-09-19T08:43:59.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.104+0000 7f9709ffb640 1 -- 172.21.15.98:0/4028137949 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97040b0070 con 0x7f970c071950 2024-09-19T08:43:59.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.104+0000 7f9709ffb640 1 -- 172.21.15.98:0/4028137949 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97040ab890 con 0x7f970c071950 2024-09-19T08:43:59.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.104+0000 7f9711658640 1 -- 172.21.15.98:0/4028137949 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f970c071950 msgr2=0x7f970c0af580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:59.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.104+0000 7f9711658640 1 --2- 172.21.15.98:0/4028137949 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f970c071950 0x7f970c0af580 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f97040097c0 tx=0x7f970401c470 comp rx=0 tx=0).stop 2024-09-19T08:43:59.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.104+0000 7f9711658640 1 -- 172.21.15.98:0/4028137949 shutdown_connections 2024-09-19T08:43:59.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.104+0000 7f9711658640 1 --2- 172.21.15.98:0/4028137949 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f970c143420 0x7f970c143890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:59.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.104+0000 7f9711658640 1 --2- 172.21.15.98:0/4028137949 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f970c071950 0x7f970c0af580 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:59.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.104+0000 7f9711658640 1 -- 172.21.15.98:0/4028137949 >> 172.21.15.98:0/4028137949 conn(0x7f970c0b26a0 msgr2=0x7f970c0b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:59.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.104+0000 7f9711658640 1 -- 172.21.15.98:0/4028137949 shutdown_connections 2024-09-19T08:43:59.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.104+0000 7f9711658640 1 -- 172.21.15.98:0/4028137949 wait complete. 2024-09-19T08:43:59.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.108+0000 7f9711658640 1 Processor -- start 2024-09-19T08:43:59.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.108+0000 7f9711658640 1 -- start start 2024-09-19T08:43:59.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.108+0000 7f9711658640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f970c071950 0x7f970c13da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:59.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.108+0000 7f9711658640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f970c071d30 0x7f970c0b4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:59.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.108+0000 7f9711658640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f970c143420 0x7f970c13e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:59.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.108+0000 7f9711658640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f970c144760 con 0x7f970c071d30 2024-09-19T08:43:59.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.108+0000 7f9711658640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f970c1445e0 con 0x7f970c071950 2024-09-19T08:43:59.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.108+0000 7f9711658640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f970c1448e0 con 0x7f970c143420 2024-09-19T08:43:59.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.108+0000 7f970a7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f970c071d30 0x7f970c0b4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:59.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.108+0000 7f970a7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f970c071d30 0x7f970c0b4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:50760/0 (socket says 172.21.15.98:50760) 2024-09-19T08:43:59.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.108+0000 7f970a7fc640 1 -- 172.21.15.98:0/3673271139 learned_addr learned my addr 172.21.15.98:0/3673271139 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:59.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.108+0000 7f970affd640 1 --2- 172.21.15.98:0/3673271139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f970c071950 0x7f970c13da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:59.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.108+0000 7f970b7fe640 1 --2- 172.21.15.98:0/3673271139 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f970c143420 0x7f970c13e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:59.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.108+0000 7f970a7fc640 1 -- 172.21.15.98:0/3673271139 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f970c143420 msgr2=0x7f970c13e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:59.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.108+0000 7f970a7fc640 1 --2- 172.21.15.98:0/3673271139 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f970c143420 0x7f970c13e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:59.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.108+0000 7f970a7fc640 1 -- 172.21.15.98:0/3673271139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f970c071950 msgr2=0x7f970c13da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:59.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.108+0000 7f970a7fc640 1 --2- 172.21.15.98:0/3673271139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f970c071950 0x7f970c13da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:59.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.108+0000 7f970a7fc640 1 -- 172.21.15.98:0/3673271139 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f970c13c990 con 0x7f970c071d30 2024-09-19T08:43:59.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.108+0000 7f970a7fc640 1 --2- 172.21.15.98:0/3673271139 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f970c071d30 0x7f970c0b4c50 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f9700007fd0 tx=0x7f970000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:59.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.108+0000 7f96f37fe640 1 -- 172.21.15.98:0/3673271139 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9700017070 con 0x7f970c071d30 2024-09-19T08:43:59.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.108+0000 7f96f37fe640 1 -- 172.21.15.98:0/3673271139 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97000052d0 con 0x7f970c071d30 2024-09-19T08:43:59.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.108+0000 7f9711658640 1 -- 172.21.15.98:0/3673271139 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f970c13cc70 con 0x7f970c071d30 2024-09-19T08:43:59.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.108+0000 7f96f37fe640 1 -- 172.21.15.98:0/3673271139 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9700003840 con 0x7f970c071d30 2024-09-19T08:43:59.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.108+0000 7f9711658640 1 -- 172.21.15.98:0/3673271139 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f970c13d230 con 0x7f970c071d30 2024-09-19T08:43:59.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.108+0000 7f9711658640 1 -- 172.21.15.98:0/3673271139 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f970c143890 con 0x7f970c071d30 2024-09-19T08:43:59.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.108+0000 7f96f37fe640 1 -- 172.21.15.98:0/3673271139 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f970001a070 con 0x7f970c071d30 2024-09-19T08:43:59.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.108+0000 7f96f37fe640 1 --2- 172.21.15.98:0/3673271139 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f96f80679e0 0x7f96f8069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:59.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.108+0000 7f96f37fe640 1 -- 172.21.15.98:0/3673271139 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f9700085260 con 0x7f970c071d30 2024-09-19T08:43:59.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.112+0000 7f970affd640 1 --2- 172.21.15.98:0/3673271139 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f96f80679e0 0x7f96f8069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:59.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.112+0000 7f970affd640 1 --2- 172.21.15.98:0/3673271139 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f96f80679e0 0x7f96f8069ea0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f97040af000 tx=0x7f9704008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:59.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.112+0000 7f96f37fe640 1 -- 172.21.15.98:0/3673271139 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9700005480 con 0x7f970c071d30 2024-09-19T08:43:59.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.320+0000 7f9711658640 1 -- 172.21.15.98:0/3673271139 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f970c15e810 con 0x7f970c071d30 2024-09-19T08:43:59.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.320+0000 7f96f37fe640 1 -- 172.21.15.98:0/3673271139 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+3252 (secure 0 0 0) 0x7f9700055bf0 con 0x7f970c071d30 2024-09-19T08:43:59.324 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:59.324 INFO:teuthology.orchestra.run.smithi098.stdout:{"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":363,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":102,"num_osds":8,"num_up_osds":8,"osd_up_since":1726735088,"num_in_osds":8,"osd_in_since":1726735077,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":266854400,"bytes_avail":772827258880,"bytes_total":773094113280,"write_bytes_sec":2489,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":60,"btime":"2024-09-19T08:43:57:218267+0000","id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":6996}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-09-19T08:43:42.885701+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"y":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mon":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-19T08:43:59.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.324+0000 7f9711658640 1 -- 172.21.15.98:0/3673271139 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f96f80679e0 msgr2=0x7f96f8069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:59.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.324+0000 7f9711658640 1 --2- 172.21.15.98:0/3673271139 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f96f80679e0 0x7f96f8069ea0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f97040af000 tx=0x7f9704008040 comp rx=0 tx=0).stop 2024-09-19T08:43:59.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.324+0000 7f9711658640 1 -- 172.21.15.98:0/3673271139 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f970c071d30 msgr2=0x7f970c0b4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:59.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.324+0000 7f9711658640 1 --2- 172.21.15.98:0/3673271139 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f970c071d30 0x7f970c0b4c50 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f9700007fd0 tx=0x7f970000c890 comp rx=0 tx=0).stop 2024-09-19T08:43:59.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.324+0000 7f9711658640 1 -- 172.21.15.98:0/3673271139 shutdown_connections 2024-09-19T08:43:59.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.324+0000 7f9711658640 1 --2- 172.21.15.98:0/3673271139 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f96f80679e0 0x7f96f8069ea0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:59.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.324+0000 7f9711658640 1 --2- 172.21.15.98:0/3673271139 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f970c143420 0x7f970c13e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:59.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.324+0000 7f9711658640 1 --2- 172.21.15.98:0/3673271139 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f970c071d30 0x7f970c0b4c50 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:59.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.324+0000 7f9711658640 1 --2- 172.21.15.98:0/3673271139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f970c071950 0x7f970c13da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:59.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.324+0000 7f9711658640 1 -- 172.21.15.98:0/3673271139 >> 172.21.15.98:0/3673271139 conn(0x7f970c0b26a0 msgr2=0x7f970c142420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:59.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.324+0000 7f9711658640 1 -- 172.21.15.98:0/3673271139 shutdown_connections 2024-09-19T08:43:59.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.324+0000 7f9711658640 1 -- 172.21.15.98:0/3673271139 wait complete. 2024-09-19T08:43:59.341 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-19T08:43:59.341 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:43:59.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.444+0000 7f368eb9a640 1 Processor -- start 2024-09-19T08:43:59.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.444+0000 7f368eb9a640 1 -- start start 2024-09-19T08:43:59.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.448+0000 7f368eb9a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f368815c550 0x7f368806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:59.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.448+0000 7f368eb9a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f368806c1d0 0x7f36880cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:59.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.448+0000 7f368eb9a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36880cf900 0x7f36880ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:59.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.448+0000 7f368eb9a640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f368806c640 con 0x7f368806c1d0 2024-09-19T08:43:59.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.448+0000 7f368eb9a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f368806d310 con 0x7f368815c550 2024-09-19T08:43:59.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.448+0000 7f368eb9a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f368806d5f0 con 0x7f36880cf900 2024-09-19T08:43:59.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.448+0000 7f3687fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f368806c1d0 0x7f36880cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:59.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.448+0000 7f3687fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f368806c1d0 0x7f36880cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:50766/0 (socket says 172.21.15.98:50766) 2024-09-19T08:43:59.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.448+0000 7f3687fff640 1 -- 172.21.15.98:0/1006817295 learned_addr learned my addr 172.21.15.98:0/1006817295 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:59.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.448+0000 7f3687fff640 1 -- 172.21.15.98:0/1006817295 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36880cf900 msgr2=0x7f36880ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:59.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.448+0000 7f3687fff640 1 --2- 172.21.15.98:0/1006817295 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36880cf900 0x7f36880ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:59.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.448+0000 7f3687fff640 1 -- 172.21.15.98:0/1006817295 >> v2:172.21.15.139:3300/0 conn(0x7f368815c550 msgr2=0x7f368806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:43:59.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.448+0000 7f368c936640 1 --2- 172.21.15.98:0/1006817295 >> v2:172.21.15.139:3300/0 conn(0x7f368815c550 0x7f368806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:59.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.448+0000 7f3687fff640 1 --2- 172.21.15.98:0/1006817295 >> v2:172.21.15.139:3300/0 conn(0x7f368815c550 0x7f368806bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:59.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.448+0000 7f3687fff640 1 -- 172.21.15.98:0/1006817295 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f368806d8d0 con 0x7f368806c1d0 2024-09-19T08:43:59.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.448+0000 7f3687fff640 1 --2- 172.21.15.98:0/1006817295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f368806c1d0 0x7f36880cf3c0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f36880b56b0 tx=0x7f367c01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad2eb4e6f65039c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:59.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.448+0000 7f36877fe640 1 -- 172.21.15.98:0/1006817295 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c0b0070 con 0x7f368806c1d0 2024-09-19T08:43:59.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.448+0000 7f36877fe640 1 -- 172.21.15.98:0/1006817295 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f367c0ab600 con 0x7f368806c1d0 2024-09-19T08:43:59.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.448+0000 7f36877fe640 1 -- 172.21.15.98:0/1006817295 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c0ab9d0 con 0x7f368806c1d0 2024-09-19T08:43:59.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.448+0000 7f368eb9a640 1 -- 172.21.15.98:0/1006817295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f368806c1d0 msgr2=0x7f36880cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:59.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.448+0000 7f368eb9a640 1 --2- 172.21.15.98:0/1006817295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f368806c1d0 0x7f36880cf3c0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f36880b56b0 tx=0x7f367c01c260 comp rx=0 tx=0).stop 2024-09-19T08:43:59.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.448+0000 7f368eb9a640 1 -- 172.21.15.98:0/1006817295 shutdown_connections 2024-09-19T08:43:59.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.448+0000 7f368eb9a640 1 --2- 172.21.15.98:0/1006817295 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f368806c1d0 0x7f36880cf3c0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:59.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.448+0000 7f368eb9a640 1 --2- 172.21.15.98:0/1006817295 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36880cf900 0x7f36880ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:59.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.448+0000 7f368eb9a640 1 --2- 172.21.15.98:0/1006817295 >> v2:172.21.15.139:3300/0 conn(0x7f368815c550 0x7f368806bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:59.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.448+0000 7f368eb9a640 1 -- 172.21.15.98:0/1006817295 >> 172.21.15.98:0/1006817295 conn(0x7f36880b2010 msgr2=0x7f36880b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:59.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.448+0000 7f368eb9a640 1 -- 172.21.15.98:0/1006817295 shutdown_connections 2024-09-19T08:43:59.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.448+0000 7f368eb9a640 1 -- 172.21.15.98:0/1006817295 wait complete. 2024-09-19T08:43:59.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.452+0000 7f368eb9a640 1 Processor -- start 2024-09-19T08:43:59.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.452+0000 7f368eb9a640 1 -- start start 2024-09-19T08:43:59.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.452+0000 7f368eb9a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f368806c1d0 0x7f368814af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:59.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.452+0000 7f368eb9a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36880cf900 0x7f36881536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:59.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.452+0000 7f368eb9a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f368815c550 0x7f3688155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:59.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.452+0000 7f368eb9a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3688072520 con 0x7f36880cf900 2024-09-19T08:43:59.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.452+0000 7f368eb9a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f36880723a0 con 0x7f368815c550 2024-09-19T08:43:59.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.452+0000 7f368eb9a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f36880726a0 con 0x7f368806c1d0 2024-09-19T08:43:59.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.452+0000 7f3687fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36880cf900 0x7f36881536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:59.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.452+0000 7f3687fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36880cf900 0x7f36881536e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:50770/0 (socket says 172.21.15.98:50770) 2024-09-19T08:43:59.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.452+0000 7f3687fff640 1 -- 172.21.15.98:0/2247072965 learned_addr learned my addr 172.21.15.98:0/2247072965 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:43:59.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.452+0000 7f368d137640 1 --2- 172.21.15.98:0/2247072965 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f368815c550 0x7f3688155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:59.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.452+0000 7f368c936640 1 --2- 172.21.15.98:0/2247072965 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f368806c1d0 0x7f368814af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:59.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.452+0000 7f3687fff640 1 -- 172.21.15.98:0/2247072965 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f368806c1d0 msgr2=0x7f368814af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:59.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.452+0000 7f3687fff640 1 --2- 172.21.15.98:0/2247072965 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f368806c1d0 0x7f368814af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:59.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.452+0000 7f3687fff640 1 -- 172.21.15.98:0/2247072965 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f368815c550 msgr2=0x7f3688155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:59.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.452+0000 7f3687fff640 1 --2- 172.21.15.98:0/2247072965 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f368815c550 0x7f3688155710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:59.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.452+0000 7f3687fff640 1 -- 172.21.15.98:0/2247072965 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f368814de50 con 0x7f36880cf900 2024-09-19T08:43:59.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.452+0000 7f3687fff640 1 --2- 172.21.15.98:0/2247072965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36880cf900 0x7f36881536e0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f367c0ae040 tx=0x7f367c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:59.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.452+0000 7f36857fa640 1 -- 172.21.15.98:0/2247072965 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c0b0040 con 0x7f36880cf900 2024-09-19T08:43:59.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.452+0000 7f36857fa640 1 -- 172.21.15.98:0/2247072965 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f367c0b9070 con 0x7f36880cf900 2024-09-19T08:43:59.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.452+0000 7f368eb9a640 1 -- 172.21.15.98:0/2247072965 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f368814e130 con 0x7f36880cf900 2024-09-19T08:43:59.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.452+0000 7f36857fa640 1 -- 172.21.15.98:0/2247072965 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c0b4830 con 0x7f36880cf900 2024-09-19T08:43:59.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.452+0000 7f368eb9a640 1 -- 172.21.15.98:0/2247072965 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f368814e6f0 con 0x7f36880cf900 2024-09-19T08:43:59.457 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.452+0000 7f368eb9a640 1 -- 172.21.15.98:0/2247072965 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36880ce1a0 con 0x7f36880cf900 2024-09-19T08:43:59.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.456+0000 7f36857fa640 1 -- 172.21.15.98:0/2247072965 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f367c0b49d0 con 0x7f36880cf900 2024-09-19T08:43:59.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.456+0000 7f36857fa640 1 --2- 172.21.15.98:0/2247072965 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f36740679e0 0x7f3674069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:43:59.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.456+0000 7f36857fa640 1 -- 172.21.15.98:0/2247072965 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f367c11fe10 con 0x7f36880cf900 2024-09-19T08:43:59.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.456+0000 7f368c936640 1 --2- 172.21.15.98:0/2247072965 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f36740679e0 0x7f3674069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:43:59.459 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.456+0000 7f368c936640 1 --2- 172.21.15.98:0/2247072965 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f36740679e0 0x7f3674069ea0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f367800b7c0 tx=0x7f3678008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:43:59.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.460+0000 7f36857fa640 1 -- 172.21.15.98:0/2247072965 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f367c016110 con 0x7f36880cf900 2024-09-19T08:43:59.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.616+0000 7f368eb9a640 1 -- 172.21.15.98:0/2247072965 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f368814e930 con 0x7f36740679e0 2024-09-19T08:43:59.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.632+0000 7f36857fa640 1 -- 172.21.15.98:0/2247072965 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252426 (secure 0 0 0) 0x7f367800bda0 con 0x7f36740679e0 2024-09-19T08:43:59.639 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:43:59.640 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:43:59.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.636+0000 7f368eb9a640 1 -- 172.21.15.98:0/2247072965 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f36740679e0 msgr2=0x7f3674069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:59.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.636+0000 7f368eb9a640 1 --2- 172.21.15.98:0/2247072965 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f36740679e0 0x7f3674069ea0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f367800b7c0 tx=0x7f3678008040 comp rx=0 tx=0).stop 2024-09-19T08:43:59.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.636+0000 7f368eb9a640 1 -- 172.21.15.98:0/2247072965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36880cf900 msgr2=0x7f36881536e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:43:59.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.636+0000 7f368eb9a640 1 --2- 172.21.15.98:0/2247072965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36880cf900 0x7f36881536e0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f367c0ae040 tx=0x7f367c019000 comp rx=0 tx=0).stop 2024-09-19T08:43:59.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.640+0000 7f368eb9a640 1 -- 172.21.15.98:0/2247072965 shutdown_connections 2024-09-19T08:43:59.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.640+0000 7f368eb9a640 1 --2- 172.21.15.98:0/2247072965 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f36740679e0 0x7f3674069ea0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:59.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.640+0000 7f368eb9a640 1 --2- 172.21.15.98:0/2247072965 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f368815c550 0x7f3688155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:59.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.640+0000 7f368eb9a640 1 --2- 172.21.15.98:0/2247072965 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36880cf900 0x7f36881536e0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:59.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.640+0000 7f368eb9a640 1 --2- 172.21.15.98:0/2247072965 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f368806c1d0 0x7f368814af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:43:59.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.640+0000 7f368eb9a640 1 -- 172.21.15.98:0/2247072965 >> 172.21.15.98:0/2247072965 conn(0x7f36880b2010 msgr2=0x7f36881543d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:43:59.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.640+0000 7f368eb9a640 1 -- 172.21.15.98:0/2247072965 shutdown_connections 2024-09-19T08:43:59.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:43:59.640+0000 7f368eb9a640 1 -- 172.21.15.98:0/2247072965 wait complete. 2024-09-19T08:43:59.656 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2024-09-19T08:43:58.890652+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260600,"kb_used_data":11080,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714120,"statfs":{"total":773094113280,"available":772827258880,"internally_reserved":0,"allocated":11345920,"data_stored":5622508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.758732"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.156262+0000","last_change":"2024-09-19T08:43:55.175947+0000","last_active":"2024-09-19T08:43:56.156262+0000","last_peered":"2024-09-19T08:43:56.156262+0000","last_clean":"2024-09-19T08:43:56.156262+0000","last_became_active":"2024-09-19T08:43:55.175606+0000","last_became_peered":"2024-09-19T08:43:55.175606+0000","last_unstale":"2024-09-19T08:43:56.156262+0000","last_undegraded":"2024-09-19T08:43:56.156262+0000","last_fullsized":"2024-09-19T08:43:56.156262+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:46:51.742785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.161571+0000","last_change":"2024-09-19T08:43:55.206900+0000","last_active":"2024-09-19T08:43:56.161571+0000","last_peered":"2024-09-19T08:43:56.161571+0000","last_clean":"2024-09-19T08:43:56.161571+0000","last_became_active":"2024-09-19T08:43:55.200022+0000","last_became_peered":"2024-09-19T08:43:55.200022+0000","last_unstale":"2024-09-19T08:43:56.161571+0000","last_undegraded":"2024-09-19T08:43:56.161571+0000","last_fullsized":"2024-09-19T08:43:56.161571+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:39:48.558843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736619+0000","last_change":"2024-09-19T08:43:55.191789+0000","last_active":"2024-09-19T08:43:56.736619+0000","last_peered":"2024-09-19T08:43:56.736619+0000","last_clean":"2024-09-19T08:43:56.736619+0000","last_became_active":"2024-09-19T08:43:55.191065+0000","last_became_peered":"2024-09-19T08:43:55.191065+0000","last_unstale":"2024-09-19T08:43:56.736619+0000","last_undegraded":"2024-09-19T08:43:56.736619+0000","last_fullsized":"2024-09-19T08:43:56.736619+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:15.340286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.156243+0000","last_change":"2024-09-19T08:43:55.199879+0000","last_active":"2024-09-19T08:43:56.156243+0000","last_peered":"2024-09-19T08:43:56.156243+0000","last_clean":"2024-09-19T08:43:56.156243+0000","last_became_active":"2024-09-19T08:43:55.199679+0000","last_became_peered":"2024-09-19T08:43:55.199679+0000","last_unstale":"2024-09-19T08:43:56.156243+0000","last_undegraded":"2024-09-19T08:43:56.156243+0000","last_fullsized":"2024-09-19T08:43:56.156243+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:54:02.803710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736288+0000","last_change":"2024-09-19T08:43:55.168455+0000","last_active":"2024-09-19T08:43:56.736288+0000","last_peered":"2024-09-19T08:43:56.736288+0000","last_clean":"2024-09-19T08:43:56.736288+0000","last_became_active":"2024-09-19T08:43:55.168218+0000","last_became_peered":"2024-09-19T08:43:55.168218+0000","last_unstale":"2024-09-19T08:43:56.736288+0000","last_undegraded":"2024-09-19T08:43:56.736288+0000","last_fullsized":"2024-09-19T08:43:56.736288+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:05:28.010611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.575891+0000","last_change":"2024-09-19T08:43:55.169699+0000","last_active":"2024-09-19T08:43:56.575891+0000","last_peered":"2024-09-19T08:43:56.575891+0000","last_clean":"2024-09-19T08:43:56.575891+0000","last_became_active":"2024-09-19T08:43:55.169436+0000","last_became_peered":"2024-09-19T08:43:55.169436+0000","last_unstale":"2024-09-19T08:43:56.575891+0000","last_undegraded":"2024-09-19T08:43:56.575891+0000","last_fullsized":"2024-09-19T08:43:56.575891+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:29:31.356189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.169833+0000","last_change":"2024-09-19T08:43:55.194043+0000","last_active":"2024-09-19T08:43:56.169833+0000","last_peered":"2024-09-19T08:43:56.169833+0000","last_clean":"2024-09-19T08:43:56.169833+0000","last_became_active":"2024-09-19T08:43:55.193266+0000","last_became_peered":"2024-09-19T08:43:55.193266+0000","last_unstale":"2024-09-19T08:43:56.169833+0000","last_undegraded":"2024-09-19T08:43:56.169833+0000","last_fullsized":"2024-09-19T08:43:56.169833+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:28.089099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736336+0000","last_change":"2024-09-19T08:43:55.177179+0000","last_active":"2024-09-19T08:43:56.736336+0000","last_peered":"2024-09-19T08:43:56.736336+0000","last_clean":"2024-09-19T08:43:56.736336+0000","last_became_active":"2024-09-19T08:43:55.174397+0000","last_became_peered":"2024-09-19T08:43:55.174397+0000","last_unstale":"2024-09-19T08:43:56.736336+0000","last_undegraded":"2024-09-19T08:43:56.736336+0000","last_fullsized":"2024-09-19T08:43:56.736336+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:08:22.088644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.156620+0000","last_change":"2024-09-19T08:43:55.187917+0000","last_active":"2024-09-19T08:43:56.156620+0000","last_peered":"2024-09-19T08:43:56.156620+0000","last_clean":"2024-09-19T08:43:56.156620+0000","last_became_active":"2024-09-19T08:43:55.187097+0000","last_became_peered":"2024-09-19T08:43:55.187097+0000","last_unstale":"2024-09-19T08:43:56.156620+0000","last_undegraded":"2024-09-19T08:43:56.156620+0000","last_fullsized":"2024-09-19T08:43:56.156620+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:34:28.929131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.155582+0000","last_change":"2024-09-19T08:43:55.176966+0000","last_active":"2024-09-19T08:43:56.155582+0000","last_peered":"2024-09-19T08:43:56.155582+0000","last_clean":"2024-09-19T08:43:56.155582+0000","last_became_active":"2024-09-19T08:43:55.175980+0000","last_became_peered":"2024-09-19T08:43:55.175980+0000","last_unstale":"2024-09-19T08:43:56.155582+0000","last_undegraded":"2024-09-19T08:43:56.155582+0000","last_fullsized":"2024-09-19T08:43:56.155582+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:46.206086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.160446+0000","last_change":"2024-09-19T08:43:55.179425+0000","last_active":"2024-09-19T08:43:56.160446+0000","last_peered":"2024-09-19T08:43:56.160446+0000","last_clean":"2024-09-19T08:43:56.160446+0000","last_became_active":"2024-09-19T08:43:55.178734+0000","last_became_peered":"2024-09-19T08:43:55.178734+0000","last_unstale":"2024-09-19T08:43:56.160446+0000","last_undegraded":"2024-09-19T08:43:56.160446+0000","last_fullsized":"2024-09-19T08:43:56.160446+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:51.785619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.156543+0000","last_change":"2024-09-19T08:43:55.180274+0000","last_active":"2024-09-19T08:43:56.156543+0000","last_peered":"2024-09-19T08:43:56.156543+0000","last_clean":"2024-09-19T08:43:56.156543+0000","last_became_active":"2024-09-19T08:43:55.179930+0000","last_became_peered":"2024-09-19T08:43:55.179930+0000","last_unstale":"2024-09-19T08:43:56.156543+0000","last_undegraded":"2024-09-19T08:43:56.156543+0000","last_fullsized":"2024-09-19T08:43:56.156543+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:23.428495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.737001+0000","last_change":"2024-09-19T08:43:55.170744+0000","last_active":"2024-09-19T08:43:56.737001+0000","last_peered":"2024-09-19T08:43:56.737001+0000","last_clean":"2024-09-19T08:43:56.737001+0000","last_became_active":"2024-09-19T08:43:55.169601+0000","last_became_peered":"2024-09-19T08:43:55.169601+0000","last_unstale":"2024-09-19T08:43:56.737001+0000","last_undegraded":"2024-09-19T08:43:56.737001+0000","last_fullsized":"2024-09-19T08:43:56.737001+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:39:02.010421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736207+0000","last_change":"2024-09-19T08:43:55.173507+0000","last_active":"2024-09-19T08:43:56.736207+0000","last_peered":"2024-09-19T08:43:56.736207+0000","last_clean":"2024-09-19T08:43:56.736207+0000","last_became_active":"2024-09-19T08:43:55.172785+0000","last_became_peered":"2024-09-19T08:43:55.172785+0000","last_unstale":"2024-09-19T08:43:56.736207+0000","last_undegraded":"2024-09-19T08:43:56.736207+0000","last_fullsized":"2024-09-19T08:43:56.736207+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:02:46.286404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.575763+0000","last_change":"2024-09-19T08:43:55.170172+0000","last_active":"2024-09-19T08:43:56.575763+0000","last_peered":"2024-09-19T08:43:56.575763+0000","last_clean":"2024-09-19T08:43:56.575763+0000","last_became_active":"2024-09-19T08:43:55.169053+0000","last_became_peered":"2024-09-19T08:43:55.169053+0000","last_unstale":"2024-09-19T08:43:56.575763+0000","last_undegraded":"2024-09-19T08:43:56.575763+0000","last_fullsized":"2024-09-19T08:43:56.575763+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:23.435267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.155922+0000","last_change":"2024-09-19T08:43:55.181396+0000","last_active":"2024-09-19T08:43:56.155922+0000","last_peered":"2024-09-19T08:43:56.155922+0000","last_clean":"2024-09-19T08:43:56.155922+0000","last_became_active":"2024-09-19T08:43:55.180917+0000","last_became_peered":"2024-09-19T08:43:55.180917+0000","last_unstale":"2024-09-19T08:43:56.155922+0000","last_undegraded":"2024-09-19T08:43:56.155922+0000","last_fullsized":"2024-09-19T08:43:56.155922+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:40:18.247113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.161176+0000","last_change":"2024-09-19T08:43:55.186312+0000","last_active":"2024-09-19T08:43:56.161176+0000","last_peered":"2024-09-19T08:43:56.161176+0000","last_clean":"2024-09-19T08:43:56.161176+0000","last_became_active":"2024-09-19T08:43:55.185577+0000","last_became_peered":"2024-09-19T08:43:55.185577+0000","last_unstale":"2024-09-19T08:43:56.161176+0000","last_undegraded":"2024-09-19T08:43:56.161176+0000","last_fullsized":"2024-09-19T08:43:56.161176+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:41.369862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.165469+0000","last_change":"2024-09-19T08:43:55.209049+0000","last_active":"2024-09-19T08:43:56.165469+0000","last_peered":"2024-09-19T08:43:56.165469+0000","last_clean":"2024-09-19T08:43:56.165469+0000","last_became_active":"2024-09-19T08:43:55.208404+0000","last_became_peered":"2024-09-19T08:43:55.208404+0000","last_unstale":"2024-09-19T08:43:56.165469+0000","last_undegraded":"2024-09-19T08:43:56.165469+0000","last_fullsized":"2024-09-19T08:43:56.165469+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:55.541839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736693+0000","last_change":"2024-09-19T08:43:55.179067+0000","last_active":"2024-09-19T08:43:56.736693+0000","last_peered":"2024-09-19T08:43:56.736693+0000","last_clean":"2024-09-19T08:43:56.736693+0000","last_became_active":"2024-09-19T08:43:55.178109+0000","last_became_peered":"2024-09-19T08:43:55.178109+0000","last_unstale":"2024-09-19T08:43:56.736693+0000","last_undegraded":"2024-09-19T08:43:56.736693+0000","last_fullsized":"2024-09-19T08:43:56.736693+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:51.785619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.161144+0000","last_change":"2024-09-19T08:43:55.201728+0000","last_active":"2024-09-19T08:43:56.161144+0000","last_peered":"2024-09-19T08:43:56.161144+0000","last_clean":"2024-09-19T08:43:56.161144+0000","last_became_active":"2024-09-19T08:43:55.201157+0000","last_became_peered":"2024-09-19T08:43:55.201157+0000","last_unstale":"2024-09-19T08:43:56.161144+0000","last_undegraded":"2024-09-19T08:43:56.161144+0000","last_fullsized":"2024-09-19T08:43:56.161144+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:51:42.503330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.160768+0000","last_change":"2024-09-19T08:43:55.188891+0000","last_active":"2024-09-19T08:43:56.160768+0000","last_peered":"2024-09-19T08:43:56.160768+0000","last_clean":"2024-09-19T08:43:56.160768+0000","last_became_active":"2024-09-19T08:43:55.188094+0000","last_became_peered":"2024-09-19T08:43:55.188094+0000","last_unstale":"2024-09-19T08:43:56.160768+0000","last_undegraded":"2024-09-19T08:43:56.160768+0000","last_fullsized":"2024-09-19T08:43:56.160768+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:31:11.824242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.735712+0000","last_change":"2024-09-19T08:43:55.171322+0000","last_active":"2024-09-19T08:43:56.735712+0000","last_peered":"2024-09-19T08:43:56.735712+0000","last_clean":"2024-09-19T08:43:56.735712+0000","last_became_active":"2024-09-19T08:43:55.171040+0000","last_became_peered":"2024-09-19T08:43:55.171040+0000","last_unstale":"2024-09-19T08:43:56.735712+0000","last_undegraded":"2024-09-19T08:43:56.735712+0000","last_fullsized":"2024-09-19T08:43:56.735712+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:44.393216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.161536+0000","last_change":"2024-09-19T08:43:55.208256+0000","last_active":"2024-09-19T08:43:56.161536+0000","last_peered":"2024-09-19T08:43:56.161536+0000","last_clean":"2024-09-19T08:43:56.161536+0000","last_became_active":"2024-09-19T08:43:55.207547+0000","last_became_peered":"2024-09-19T08:43:55.207547+0000","last_unstale":"2024-09-19T08:43:56.161536+0000","last_undegraded":"2024-09-19T08:43:56.161536+0000","last_fullsized":"2024-09-19T08:43:56.161536+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:37:19.729827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.735102+0000","last_change":"2024-09-19T08:43:55.172018+0000","last_active":"2024-09-19T08:43:56.735102+0000","last_peered":"2024-09-19T08:43:56.735102+0000","last_clean":"2024-09-19T08:43:56.735102+0000","last_became_active":"2024-09-19T08:43:55.171714+0000","last_became_peered":"2024-09-19T08:43:55.171714+0000","last_unstale":"2024-09-19T08:43:56.735102+0000","last_undegraded":"2024-09-19T08:43:56.735102+0000","last_fullsized":"2024-09-19T08:43:56.735102+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:46:43.070713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.161452+0000","last_change":"2024-09-19T08:43:55.176249+0000","last_active":"2024-09-19T08:43:56.161452+0000","last_peered":"2024-09-19T08:43:56.161452+0000","last_clean":"2024-09-19T08:43:56.161452+0000","last_became_active":"2024-09-19T08:43:55.175331+0000","last_became_peered":"2024-09-19T08:43:55.175331+0000","last_unstale":"2024-09-19T08:43:56.161452+0000","last_undegraded":"2024-09-19T08:43:56.161452+0000","last_fullsized":"2024-09-19T08:43:56.161452+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:31:31.316017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.155735+0000","last_change":"2024-09-19T08:43:55.198912+0000","last_active":"2024-09-19T08:43:56.155735+0000","last_peered":"2024-09-19T08:43:56.155735+0000","last_clean":"2024-09-19T08:43:56.155735+0000","last_became_active":"2024-09-19T08:43:55.198693+0000","last_became_peered":"2024-09-19T08:43:55.198693+0000","last_unstale":"2024-09-19T08:43:56.155735+0000","last_undegraded":"2024-09-19T08:43:56.155735+0000","last_fullsized":"2024-09-19T08:43:56.155735+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:43.862034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.576308+0000","last_change":"2024-09-19T08:43:55.175621+0000","last_active":"2024-09-19T08:43:56.576308+0000","last_peered":"2024-09-19T08:43:56.576308+0000","last_clean":"2024-09-19T08:43:56.576308+0000","last_became_active":"2024-09-19T08:43:55.170608+0000","last_became_peered":"2024-09-19T08:43:55.170608+0000","last_unstale":"2024-09-19T08:43:56.576308+0000","last_undegraded":"2024-09-19T08:43:56.576308+0000","last_fullsized":"2024-09-19T08:43:56.576308+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:45:37.312790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.737173+0000","last_change":"2024-09-19T08:43:55.186309+0000","last_active":"2024-09-19T08:43:56.737173+0000","last_peered":"2024-09-19T08:43:56.737173+0000","last_clean":"2024-09-19T08:43:56.737173+0000","last_became_active":"2024-09-19T08:43:55.185612+0000","last_became_peered":"2024-09-19T08:43:55.185612+0000","last_unstale":"2024-09-19T08:43:56.737173+0000","last_undegraded":"2024-09-19T08:43:56.737173+0000","last_fullsized":"2024-09-19T08:43:56.737173+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:55:18.505199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.160111+0000","last_change":"2024-09-19T08:43:55.206052+0000","last_active":"2024-09-19T08:43:56.160111+0000","last_peered":"2024-09-19T08:43:56.160111+0000","last_clean":"2024-09-19T08:43:56.160111+0000","last_became_active":"2024-09-19T08:43:55.205325+0000","last_became_peered":"2024-09-19T08:43:55.205325+0000","last_unstale":"2024-09-19T08:43:56.160111+0000","last_undegraded":"2024-09-19T08:43:56.160111+0000","last_fullsized":"2024-09-19T08:43:56.160111+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:54:49.099389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.170072+0000","last_change":"2024-09-19T08:43:55.193647+0000","last_active":"2024-09-19T08:43:56.170072+0000","last_peered":"2024-09-19T08:43:56.170072+0000","last_clean":"2024-09-19T08:43:56.170072+0000","last_became_active":"2024-09-19T08:43:55.192947+0000","last_became_peered":"2024-09-19T08:43:55.192947+0000","last_unstale":"2024-09-19T08:43:56.170072+0000","last_undegraded":"2024-09-19T08:43:56.170072+0000","last_fullsized":"2024-09-19T08:43:56.170072+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:56.087733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.155719+0000","last_change":"2024-09-19T08:43:55.194691+0000","last_active":"2024-09-19T08:43:56.155719+0000","last_peered":"2024-09-19T08:43:56.155719+0000","last_clean":"2024-09-19T08:43:56.155719+0000","last_became_active":"2024-09-19T08:43:55.194441+0000","last_became_peered":"2024-09-19T08:43:55.194441+0000","last_unstale":"2024-09-19T08:43:56.155719+0000","last_undegraded":"2024-09-19T08:43:56.155719+0000","last_fullsized":"2024-09-19T08:43:56.155719+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:06:45.109119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.575520+0000","last_change":"2024-09-19T08:43:55.175081+0000","last_active":"2024-09-19T08:43:56.575520+0000","last_peered":"2024-09-19T08:43:56.575520+0000","last_clean":"2024-09-19T08:43:56.575520+0000","last_became_active":"2024-09-19T08:43:55.170833+0000","last_became_peered":"2024-09-19T08:43:55.170833+0000","last_unstale":"2024-09-19T08:43:56.575520+0000","last_undegraded":"2024-09-19T08:43:56.575520+0000","last_fullsized":"2024-09-19T08:43:56.575520+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:23.675291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.576479+0000","last_change":"2024-09-19T08:43:55.176053+0000","last_active":"2024-09-19T08:43:56.576479+0000","last_peered":"2024-09-19T08:43:56.576479+0000","last_clean":"2024-09-19T08:43:56.576479+0000","last_became_active":"2024-09-19T08:43:55.175394+0000","last_became_peered":"2024-09-19T08:43:55.175394+0000","last_unstale":"2024-09-19T08:43:56.576479+0000","last_undegraded":"2024-09-19T08:43:56.576479+0000","last_fullsized":"2024-09-19T08:43:56.576479+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:46:30.215239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"101'4","reported_seq":18,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.170239+0000","last_change":"2024-09-19T08:43:54.173670+0000","last_active":"2024-09-19T08:43:56.170239+0000","last_peered":"2024-09-19T08:43:56.170239+0000","last_clean":"2024-09-19T08:43:56.170239+0000","last_became_active":"2024-09-19T08:43:54.172904+0000","last_became_peered":"2024-09-19T08:43:54.172904+0000","last_unstale":"2024-09-19T08:43:56.170239+0000","last_undegraded":"2024-09-19T08:43:56.170239+0000","last_fullsized":"2024-09-19T08:43:56.170239+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:22.425741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.165640+0000","last_change":"2024-09-19T08:43:55.187361+0000","last_active":"2024-09-19T08:43:56.165640+0000","last_peered":"2024-09-19T08:43:56.165640+0000","last_clean":"2024-09-19T08:43:56.165640+0000","last_became_active":"2024-09-19T08:43:55.186144+0000","last_became_peered":"2024-09-19T08:43:55.186144+0000","last_unstale":"2024-09-19T08:43:56.165640+0000","last_undegraded":"2024-09-19T08:43:56.165640+0000","last_fullsized":"2024-09-19T08:43:56.165640+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:30:19.243180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"101'1","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736331+0000","last_change":"2024-09-19T08:43:54.175936+0000","last_active":"2024-09-19T08:43:56.736331+0000","last_peered":"2024-09-19T08:43:56.736331+0000","last_clean":"2024-09-19T08:43:56.736331+0000","last_became_active":"2024-09-19T08:43:54.175215+0000","last_became_peered":"2024-09-19T08:43:54.175215+0000","last_unstale":"2024-09-19T08:43:56.736331+0000","last_undegraded":"2024-09-19T08:43:56.736331+0000","last_fullsized":"2024-09-19T08:43:56.736331+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:27:33.779456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.576311+0000","last_change":"2024-09-19T08:43:55.182209+0000","last_active":"2024-09-19T08:43:56.576311+0000","last_peered":"2024-09-19T08:43:56.576311+0000","last_clean":"2024-09-19T08:43:56.576311+0000","last_became_active":"2024-09-19T08:43:55.181787+0000","last_became_peered":"2024-09-19T08:43:55.181787+0000","last_unstale":"2024-09-19T08:43:56.576311+0000","last_undegraded":"2024-09-19T08:43:56.576311+0000","last_fullsized":"2024-09-19T08:43:56.576311+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:54:28.125782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"101'2","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736070+0000","last_change":"2024-09-19T08:43:54.148059+0000","last_active":"2024-09-19T08:43:56.736070+0000","last_peered":"2024-09-19T08:43:56.736070+0000","last_clean":"2024-09-19T08:43:56.736070+0000","last_became_active":"2024-09-19T08:43:54.147311+0000","last_became_peered":"2024-09-19T08:43:54.147311+0000","last_unstale":"2024-09-19T08:43:56.736070+0000","last_undegraded":"2024-09-19T08:43:56.736070+0000","last_fullsized":"2024-09-19T08:43:56.736070+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:59:59.076766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.737066+0000","last_change":"2024-09-19T08:43:55.181960+0000","last_active":"2024-09-19T08:43:56.737066+0000","last_peered":"2024-09-19T08:43:56.737066+0000","last_clean":"2024-09-19T08:43:56.737066+0000","last_became_active":"2024-09-19T08:43:55.181038+0000","last_became_peered":"2024-09-19T08:43:55.181038+0000","last_unstale":"2024-09-19T08:43:56.737066+0000","last_undegraded":"2024-09-19T08:43:56.737066+0000","last_fullsized":"2024-09-19T08:43:56.737066+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:41.369862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"101'2","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.735543+0000","last_change":"2024-09-19T08:43:54.148088+0000","last_active":"2024-09-19T08:43:56.735543+0000","last_peered":"2024-09-19T08:43:56.735543+0000","last_clean":"2024-09-19T08:43:56.735543+0000","last_became_active":"2024-09-19T08:43:54.147729+0000","last_became_peered":"2024-09-19T08:43:54.147729+0000","last_unstale":"2024-09-19T08:43:56.735543+0000","last_undegraded":"2024-09-19T08:43:56.735543+0000","last_fullsized":"2024-09-19T08:43:56.735543+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:01:57.810659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.155662+0000","last_change":"2024-09-19T08:43:55.189242+0000","last_active":"2024-09-19T08:43:56.155662+0000","last_peered":"2024-09-19T08:43:56.155662+0000","last_clean":"2024-09-19T08:43:56.155662+0000","last_became_active":"2024-09-19T08:43:55.188976+0000","last_became_peered":"2024-09-19T08:43:55.188976+0000","last_unstale":"2024-09-19T08:43:56.155662+0000","last_undegraded":"2024-09-19T08:43:56.155662+0000","last_fullsized":"2024-09-19T08:43:56.155662+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:05.505122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"101'1","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.169629+0000","last_change":"2024-09-19T08:43:54.174001+0000","last_active":"2024-09-19T08:43:56.169629+0000","last_peered":"2024-09-19T08:43:56.169629+0000","last_clean":"2024-09-19T08:43:56.169629+0000","last_became_active":"2024-09-19T08:43:54.171901+0000","last_became_peered":"2024-09-19T08:43:54.171901+0000","last_unstale":"2024-09-19T08:43:56.169629+0000","last_undegraded":"2024-09-19T08:43:56.169629+0000","last_fullsized":"2024-09-19T08:43:56.169629+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:22.665765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.160211+0000","last_change":"2024-09-19T08:43:55.205723+0000","last_active":"2024-09-19T08:43:56.160211+0000","last_peered":"2024-09-19T08:43:56.160211+0000","last_clean":"2024-09-19T08:43:56.160211+0000","last_became_active":"2024-09-19T08:43:55.205248+0000","last_became_peered":"2024-09-19T08:43:55.205248+0000","last_unstale":"2024-09-19T08:43:56.160211+0000","last_undegraded":"2024-09-19T08:43:56.160211+0000","last_fullsized":"2024-09-19T08:43:56.160211+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:15.340286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"101'1","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.735857+0000","last_change":"2024-09-19T08:43:54.169422+0000","last_active":"2024-09-19T08:43:56.735857+0000","last_peered":"2024-09-19T08:43:56.735857+0000","last_clean":"2024-09-19T08:43:56.735857+0000","last_became_active":"2024-09-19T08:43:54.169152+0000","last_became_peered":"2024-09-19T08:43:54.169152+0000","last_unstale":"2024-09-19T08:43:56.735857+0000","last_undegraded":"2024-09-19T08:43:56.735857+0000","last_fullsized":"2024-09-19T08:43:56.735857+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:54:49.167389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.155373+0000","last_change":"2024-09-19T08:43:55.178286+0000","last_active":"2024-09-19T08:43:56.155373+0000","last_peered":"2024-09-19T08:43:56.155373+0000","last_clean":"2024-09-19T08:43:56.155373+0000","last_became_active":"2024-09-19T08:43:55.177900+0000","last_became_peered":"2024-09-19T08:43:55.177900+0000","last_unstale":"2024-09-19T08:43:56.155373+0000","last_undegraded":"2024-09-19T08:43:56.155373+0000","last_fullsized":"2024-09-19T08:43:56.155373+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:08:10.535405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.159848+0000","last_change":"2024-09-19T08:43:54.170080+0000","last_active":"2024-09-19T08:43:56.159848+0000","last_peered":"2024-09-19T08:43:56.159848+0000","last_clean":"2024-09-19T08:43:56.159848+0000","last_became_active":"2024-09-19T08:43:54.168481+0000","last_became_peered":"2024-09-19T08:43:54.168481+0000","last_unstale":"2024-09-19T08:43:56.159848+0000","last_undegraded":"2024-09-19T08:43:56.159848+0000","last_fullsized":"2024-09-19T08:43:56.159848+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:55.078207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.169711+0000","last_change":"2024-09-19T08:43:55.195850+0000","last_active":"2024-09-19T08:43:56.169711+0000","last_peered":"2024-09-19T08:43:56.169711+0000","last_clean":"2024-09-19T08:43:56.169711+0000","last_became_active":"2024-09-19T08:43:55.194786+0000","last_became_peered":"2024-09-19T08:43:55.194786+0000","last_unstale":"2024-09-19T08:43:56.169711+0000","last_undegraded":"2024-09-19T08:43:56.169711+0000","last_fullsized":"2024-09-19T08:43:56.169711+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:31:59.684059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.155893+0000","last_change":"2024-09-19T08:43:54.158627+0000","last_active":"2024-09-19T08:43:56.155893+0000","last_peered":"2024-09-19T08:43:56.155893+0000","last_clean":"2024-09-19T08:43:56.155893+0000","last_became_active":"2024-09-19T08:43:54.158333+0000","last_became_peered":"2024-09-19T08:43:54.158333+0000","last_unstale":"2024-09-19T08:43:56.155893+0000","last_undegraded":"2024-09-19T08:43:56.155893+0000","last_fullsized":"2024-09-19T08:43:56.155893+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:31:30.306491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.170558+0000","last_change":"2024-09-19T08:43:55.195678+0000","last_active":"2024-09-19T08:43:56.170558+0000","last_peered":"2024-09-19T08:43:56.170558+0000","last_clean":"2024-09-19T08:43:56.170558+0000","last_became_active":"2024-09-19T08:43:55.194952+0000","last_became_peered":"2024-09-19T08:43:55.194952+0000","last_unstale":"2024-09-19T08:43:56.170558+0000","last_undegraded":"2024-09-19T08:43:56.170558+0000","last_fullsized":"2024-09-19T08:43:56.170558+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:37.648111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736244+0000","last_change":"2024-09-19T08:43:54.148128+0000","last_active":"2024-09-19T08:43:56.736244+0000","last_peered":"2024-09-19T08:43:56.736244+0000","last_clean":"2024-09-19T08:43:56.736244+0000","last_became_active":"2024-09-19T08:43:54.147519+0000","last_became_peered":"2024-09-19T08:43:54.147519+0000","last_unstale":"2024-09-19T08:43:56.736244+0000","last_undegraded":"2024-09-19T08:43:56.736244+0000","last_fullsized":"2024-09-19T08:43:56.736244+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:04:22.016732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.165566+0000","last_change":"2024-09-19T08:43:55.190134+0000","last_active":"2024-09-19T08:43:56.165566+0000","last_peered":"2024-09-19T08:43:56.165566+0000","last_clean":"2024-09-19T08:43:56.165566+0000","last_became_active":"2024-09-19T08:43:55.189377+0000","last_became_peered":"2024-09-19T08:43:55.189377+0000","last_unstale":"2024-09-19T08:43:56.165566+0000","last_undegraded":"2024-09-19T08:43:56.165566+0000","last_fullsized":"2024-09-19T08:43:56.165566+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:15:39.583209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736887+0000","last_change":"2024-09-19T08:43:54.169583+0000","last_active":"2024-09-19T08:43:56.736887+0000","last_peered":"2024-09-19T08:43:56.736887+0000","last_clean":"2024-09-19T08:43:56.736887+0000","last_became_active":"2024-09-19T08:43:54.169238+0000","last_became_peered":"2024-09-19T08:43:54.169238+0000","last_unstale":"2024-09-19T08:43:56.736887+0000","last_undegraded":"2024-09-19T08:43:56.736887+0000","last_fullsized":"2024-09-19T08:43:56.736887+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:42.579423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.737558+0000","last_change":"2024-09-19T08:43:55.200840+0000","last_active":"2024-09-19T08:43:56.737558+0000","last_peered":"2024-09-19T08:43:56.737558+0000","last_clean":"2024-09-19T08:43:56.737558+0000","last_became_active":"2024-09-19T08:43:55.200586+0000","last_became_peered":"2024-09-19T08:43:55.200586+0000","last_unstale":"2024-09-19T08:43:56.737558+0000","last_undegraded":"2024-09-19T08:43:56.737558+0000","last_fullsized":"2024-09-19T08:43:56.737558+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:22.019255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"101'3","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.155459+0000","last_change":"2024-09-19T08:43:54.156828+0000","last_active":"2024-09-19T08:43:56.155459+0000","last_peered":"2024-09-19T08:43:56.155459+0000","last_clean":"2024-09-19T08:43:56.155459+0000","last_became_active":"2024-09-19T08:43:54.156521+0000","last_became_peered":"2024-09-19T08:43:54.156521+0000","last_unstale":"2024-09-19T08:43:56.155459+0000","last_undegraded":"2024-09-19T08:43:56.155459+0000","last_fullsized":"2024-09-19T08:43:56.155459+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:51:41.493803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.170290+0000","last_change":"2024-09-19T08:43:55.205270+0000","last_active":"2024-09-19T08:43:56.170290+0000","last_peered":"2024-09-19T08:43:56.170290+0000","last_clean":"2024-09-19T08:43:56.170290+0000","last_became_active":"2024-09-19T08:43:55.204556+0000","last_became_peered":"2024-09-19T08:43:55.204556+0000","last_unstale":"2024-09-19T08:43:56.170290+0000","last_undegraded":"2024-09-19T08:43:56.170290+0000","last_fullsized":"2024-09-19T08:43:56.170290+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:53:33.564978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"101'6","reported_seq":20,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.156666+0000","last_change":"2024-09-19T08:43:54.159068+0000","last_active":"2024-09-19T08:43:56.156666+0000","last_peered":"2024-09-19T08:43:56.156666+0000","last_clean":"2024-09-19T08:43:56.156666+0000","last_became_active":"2024-09-19T08:43:54.158698+0000","last_became_peered":"2024-09-19T08:43:54.158698+0000","last_unstale":"2024-09-19T08:43:56.156666+0000","last_undegraded":"2024-09-19T08:43:56.156666+0000","last_fullsized":"2024-09-19T08:43:56.156666+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:39:01.903420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736353+0000","last_change":"2024-09-19T08:43:55.194646+0000","last_active":"2024-09-19T08:43:56.736353+0000","last_peered":"2024-09-19T08:43:56.736353+0000","last_clean":"2024-09-19T08:43:56.736353+0000","last_became_active":"2024-09-19T08:43:55.194235+0000","last_became_peered":"2024-09-19T08:43:55.194235+0000","last_unstale":"2024-09-19T08:43:56.736353+0000","last_undegraded":"2024-09-19T08:43:56.736353+0000","last_fullsized":"2024-09-19T08:43:56.736353+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:06:29.306359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"101'2","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.575865+0000","last_change":"2024-09-19T08:43:54.165780+0000","last_active":"2024-09-19T08:43:56.575865+0000","last_peered":"2024-09-19T08:43:56.575865+0000","last_clean":"2024-09-19T08:43:56.575865+0000","last_became_active":"2024-09-19T08:43:54.165527+0000","last_became_peered":"2024-09-19T08:43:54.165527+0000","last_unstale":"2024-09-19T08:43:56.575865+0000","last_undegraded":"2024-09-19T08:43:56.575865+0000","last_fullsized":"2024-09-19T08:43:56.575865+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:18.059525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736953+0000","last_change":"2024-09-19T08:43:55.167204+0000","last_active":"2024-09-19T08:43:56.736953+0000","last_peered":"2024-09-19T08:43:56.736953+0000","last_clean":"2024-09-19T08:43:56.736953+0000","last_became_active":"2024-09-19T08:43:55.166818+0000","last_became_peered":"2024-09-19T08:43:55.166818+0000","last_unstale":"2024-09-19T08:43:56.736953+0000","last_undegraded":"2024-09-19T08:43:56.736953+0000","last_fullsized":"2024-09-19T08:43:56.736953+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:08:12.006209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.164934+0000","last_change":"2024-09-19T08:43:54.167563+0000","last_active":"2024-09-19T08:43:56.164934+0000","last_peered":"2024-09-19T08:43:56.164934+0000","last_clean":"2024-09-19T08:43:56.164934+0000","last_became_active":"2024-09-19T08:43:54.166815+0000","last_became_peered":"2024-09-19T08:43:54.166815+0000","last_unstale":"2024-09-19T08:43:56.164934+0000","last_undegraded":"2024-09-19T08:43:56.164934+0000","last_fullsized":"2024-09-19T08:43:56.164934+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:03.023614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.735248+0000","last_change":"2024-09-19T08:43:55.200724+0000","last_active":"2024-09-19T08:43:56.735248+0000","last_peered":"2024-09-19T08:43:56.735248+0000","last_clean":"2024-09-19T08:43:56.735248+0000","last_became_active":"2024-09-19T08:43:55.200505+0000","last_became_peered":"2024-09-19T08:43:55.200505+0000","last_unstale":"2024-09-19T08:43:56.735248+0000","last_undegraded":"2024-09-19T08:43:56.735248+0000","last_fullsized":"2024-09-19T08:43:56.735248+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:51:14.085642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"101'8","reported_seq":22,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.735407+0000","last_change":"2024-09-19T08:43:54.153224+0000","last_active":"2024-09-19T08:43:56.735407+0000","last_peered":"2024-09-19T08:43:56.735407+0000","last_clean":"2024-09-19T08:43:56.735407+0000","last_became_active":"2024-09-19T08:43:54.152861+0000","last_became_peered":"2024-09-19T08:43:54.152861+0000","last_unstale":"2024-09-19T08:43:56.735407+0000","last_undegraded":"2024-09-19T08:43:56.735407+0000","last_fullsized":"2024-09-19T08:43:56.735407+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:28.306074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736714+0000","last_change":"2024-09-19T08:43:55.179921+0000","last_active":"2024-09-19T08:43:56.736714+0000","last_peered":"2024-09-19T08:43:56.736714+0000","last_clean":"2024-09-19T08:43:56.736714+0000","last_became_active":"2024-09-19T08:43:55.179672+0000","last_became_peered":"2024-09-19T08:43:55.179672+0000","last_unstale":"2024-09-19T08:43:56.736714+0000","last_undegraded":"2024-09-19T08:43:56.736714+0000","last_fullsized":"2024-09-19T08:43:56.736714+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:03.688703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"101'1","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.157106+0000","last_change":"2024-09-19T08:43:54.175081+0000","last_active":"2024-09-19T08:43:56.157106+0000","last_peered":"2024-09-19T08:43:56.157106+0000","last_clean":"2024-09-19T08:43:56.157106+0000","last_became_active":"2024-09-19T08:43:54.174828+0000","last_became_peered":"2024-09-19T08:43:54.174828+0000","last_unstale":"2024-09-19T08:43:56.157106+0000","last_undegraded":"2024-09-19T08:43:56.157106+0000","last_fullsized":"2024-09-19T08:43:56.157106+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:42:04.302601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.576192+0000","last_change":"2024-09-19T08:43:55.169745+0000","last_active":"2024-09-19T08:43:56.576192+0000","last_peered":"2024-09-19T08:43:56.576192+0000","last_clean":"2024-09-19T08:43:56.576192+0000","last_became_active":"2024-09-19T08:43:55.169407+0000","last_became_peered":"2024-09-19T08:43:55.169407+0000","last_unstale":"2024-09-19T08:43:56.576192+0000","last_undegraded":"2024-09-19T08:43:56.576192+0000","last_fullsized":"2024-09-19T08:43:56.576192+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:26:01.318672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"101'2","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736434+0000","last_change":"2024-09-19T08:43:54.167839+0000","last_active":"2024-09-19T08:43:56.736434+0000","last_peered":"2024-09-19T08:43:56.736434+0000","last_clean":"2024-09-19T08:43:56.736434+0000","last_became_active":"2024-09-19T08:43:54.167548+0000","last_became_peered":"2024-09-19T08:43:54.167548+0000","last_unstale":"2024-09-19T08:43:56.736434+0000","last_undegraded":"2024-09-19T08:43:56.736434+0000","last_fullsized":"2024-09-19T08:43:56.736434+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:37.105176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.737017+0000","last_change":"2024-09-19T08:43:55.186681+0000","last_active":"2024-09-19T08:43:56.737017+0000","last_peered":"2024-09-19T08:43:56.737017+0000","last_clean":"2024-09-19T08:43:56.737017+0000","last_became_active":"2024-09-19T08:43:55.185978+0000","last_became_peered":"2024-09-19T08:43:55.185978+0000","last_unstale":"2024-09-19T08:43:56.737017+0000","last_undegraded":"2024-09-19T08:43:56.737017+0000","last_fullsized":"2024-09-19T08:43:56.737017+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:51:42.503330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"101'1","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.165383+0000","last_change":"2024-09-19T08:43:54.166019+0000","last_active":"2024-09-19T08:43:56.165383+0000","last_peered":"2024-09-19T08:43:56.165383+0000","last_clean":"2024-09-19T08:43:56.165383+0000","last_became_active":"2024-09-19T08:43:54.165330+0000","last_became_peered":"2024-09-19T08:43:54.165330+0000","last_unstale":"2024-09-19T08:43:56.165383+0000","last_undegraded":"2024-09-19T08:43:56.165383+0000","last_fullsized":"2024-09-19T08:43:56.165383+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:02:58.278139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736946+0000","last_change":"2024-09-19T08:43:55.204016+0000","last_active":"2024-09-19T08:43:56.736946+0000","last_peered":"2024-09-19T08:43:56.736946+0000","last_clean":"2024-09-19T08:43:56.736946+0000","last_became_active":"2024-09-19T08:43:55.203041+0000","last_became_peered":"2024-09-19T08:43:55.203041+0000","last_unstale":"2024-09-19T08:43:56.736946+0000","last_undegraded":"2024-09-19T08:43:56.736946+0000","last_fullsized":"2024-09-19T08:43:56.736946+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:31:11.824242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.169927+0000","last_change":"2024-09-19T08:43:54.171525+0000","last_active":"2024-09-19T08:43:56.169927+0000","last_peered":"2024-09-19T08:43:56.169927+0000","last_clean":"2024-09-19T08:43:56.169927+0000","last_became_active":"2024-09-19T08:43:54.170720+0000","last_became_peered":"2024-09-19T08:43:54.170720+0000","last_unstale":"2024-09-19T08:43:56.169927+0000","last_undegraded":"2024-09-19T08:43:56.169927+0000","last_fullsized":"2024-09-19T08:43:56.169927+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:26:00.309146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736210+0000","last_change":"2024-09-19T08:43:55.191951+0000","last_active":"2024-09-19T08:43:56.736210+0000","last_peered":"2024-09-19T08:43:56.736210+0000","last_clean":"2024-09-19T08:43:56.736210+0000","last_became_active":"2024-09-19T08:43:55.191187+0000","last_became_peered":"2024-09-19T08:43:55.191187+0000","last_unstale":"2024-09-19T08:43:56.736210+0000","last_undegraded":"2024-09-19T08:43:56.736210+0000","last_fullsized":"2024-09-19T08:43:56.736210+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:09:54.189345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"101'2","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.575972+0000","last_change":"2024-09-19T08:43:54.156834+0000","last_active":"2024-09-19T08:43:56.575972+0000","last_peered":"2024-09-19T08:43:56.575972+0000","last_clean":"2024-09-19T08:43:56.575972+0000","last_became_active":"2024-09-19T08:43:54.156436+0000","last_became_peered":"2024-09-19T08:43:54.156436+0000","last_unstale":"2024-09-19T08:43:56.575972+0000","last_undegraded":"2024-09-19T08:43:56.575972+0000","last_fullsized":"2024-09-19T08:43:56.575972+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:06:49.884305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.155968+0000","last_change":"2024-09-19T08:43:55.180818+0000","last_active":"2024-09-19T08:43:56.155968+0000","last_peered":"2024-09-19T08:43:56.155968+0000","last_clean":"2024-09-19T08:43:56.155968+0000","last_became_active":"2024-09-19T08:43:55.180523+0000","last_became_peered":"2024-09-19T08:43:55.180523+0000","last_unstale":"2024-09-19T08:43:56.155968+0000","last_undegraded":"2024-09-19T08:43:56.155968+0000","last_fullsized":"2024-09-19T08:43:56.155968+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:19:35.054836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.735879+0000","last_change":"2024-09-19T08:43:54.148186+0000","last_active":"2024-09-19T08:43:56.735879+0000","last_peered":"2024-09-19T08:43:56.735879+0000","last_clean":"2024-09-19T08:43:56.735879+0000","last_became_active":"2024-09-19T08:43:54.147929+0000","last_became_peered":"2024-09-19T08:43:54.147929+0000","last_unstale":"2024-09-19T08:43:56.735879+0000","last_undegraded":"2024-09-19T08:43:56.735879+0000","last_fullsized":"2024-09-19T08:43:56.735879+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:22:11.522698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736419+0000","last_change":"2024-09-19T08:43:55.185737+0000","last_active":"2024-09-19T08:43:56.736419+0000","last_peered":"2024-09-19T08:43:56.736419+0000","last_clean":"2024-09-19T08:43:56.736419+0000","last_became_active":"2024-09-19T08:43:55.184994+0000","last_became_peered":"2024-09-19T08:43:55.184994+0000","last_unstale":"2024-09-19T08:43:56.736419+0000","last_undegraded":"2024-09-19T08:43:56.736419+0000","last_fullsized":"2024-09-19T08:43:56.736419+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:54:49.099389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.575901+0000","last_change":"2024-09-19T08:43:54.156400+0000","last_active":"2024-09-19T08:43:56.575901+0000","last_peered":"2024-09-19T08:43:56.575901+0000","last_clean":"2024-09-19T08:43:56.575901+0000","last_became_active":"2024-09-19T08:43:54.156000+0000","last_became_peered":"2024-09-19T08:43:54.156000+0000","last_unstale":"2024-09-19T08:43:56.575901+0000","last_undegraded":"2024-09-19T08:43:56.575901+0000","last_fullsized":"2024-09-19T08:43:56.575901+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:11.225788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736338+0000","last_change":"2024-09-19T08:43:55.181342+0000","last_active":"2024-09-19T08:43:56.736338+0000","last_peered":"2024-09-19T08:43:56.736338+0000","last_clean":"2024-09-19T08:43:56.736338+0000","last_became_active":"2024-09-19T08:43:55.181018+0000","last_became_peered":"2024-09-19T08:43:55.181018+0000","last_unstale":"2024-09-19T08:43:56.736338+0000","last_undegraded":"2024-09-19T08:43:56.736338+0000","last_fullsized":"2024-09-19T08:43:56.736338+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:16.137533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.155966+0000","last_change":"2024-09-19T08:43:54.164739+0000","last_active":"2024-09-19T08:43:56.155966+0000","last_peered":"2024-09-19T08:43:56.155966+0000","last_clean":"2024-09-19T08:43:56.155966+0000","last_became_active":"2024-09-19T08:43:54.164107+0000","last_became_peered":"2024-09-19T08:43:54.164107+0000","last_unstale":"2024-09-19T08:43:56.155966+0000","last_undegraded":"2024-09-19T08:43:56.155966+0000","last_fullsized":"2024-09-19T08:43:56.155966+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:37:18.720301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.165560+0000","last_change":"2024-09-19T08:43:55.190836+0000","last_active":"2024-09-19T08:43:56.165560+0000","last_peered":"2024-09-19T08:43:56.165560+0000","last_clean":"2024-09-19T08:43:56.165560+0000","last_became_active":"2024-09-19T08:43:55.190000+0000","last_became_peered":"2024-09-19T08:43:55.190000+0000","last_unstale":"2024-09-19T08:43:56.165560+0000","last_undegraded":"2024-09-19T08:43:56.165560+0000","last_fullsized":"2024-09-19T08:43:56.165560+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:38:24.253914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"101'2","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.737656+0000","last_change":"2024-09-19T08:43:54.156950+0000","last_active":"2024-09-19T08:43:56.737656+0000","last_peered":"2024-09-19T08:43:56.737656+0000","last_clean":"2024-09-19T08:43:56.737656+0000","last_became_active":"2024-09-19T08:43:54.156680+0000","last_became_peered":"2024-09-19T08:43:54.156680+0000","last_unstale":"2024-09-19T08:43:56.737656+0000","last_undegraded":"2024-09-19T08:43:56.737656+0000","last_fullsized":"2024-09-19T08:43:56.737656+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:17:05.043117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.156975+0000","last_change":"2024-09-19T08:43:55.187342+0000","last_active":"2024-09-19T08:43:56.156975+0000","last_peered":"2024-09-19T08:43:56.156975+0000","last_clean":"2024-09-19T08:43:56.156975+0000","last_became_active":"2024-09-19T08:43:55.186731+0000","last_became_peered":"2024-09-19T08:43:55.186731+0000","last_unstale":"2024-09-19T08:43:56.156975+0000","last_undegraded":"2024-09-19T08:43:56.156975+0000","last_fullsized":"2024-09-19T08:43:56.156975+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:49.028651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"102'1","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.165662+0000","last_change":"2024-09-19T08:43:54.175992+0000","last_active":"2024-09-19T08:43:56.165662+0000","last_peered":"2024-09-19T08:43:56.165662+0000","last_clean":"2024-09-19T08:43:56.165662+0000","last_became_active":"2024-09-19T08:43:54.175245+0000","last_became_peered":"2024-09-19T08:43:54.175245+0000","last_unstale":"2024-09-19T08:43:56.165662+0000","last_undegraded":"2024-09-19T08:43:56.165662+0000","last_fullsized":"2024-09-19T08:43:56.165662+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:14.806379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.156584+0000","last_change":"2024-09-19T08:43:55.183466+0000","last_active":"2024-09-19T08:43:56.156584+0000","last_peered":"2024-09-19T08:43:56.156584+0000","last_clean":"2024-09-19T08:43:56.156584+0000","last_became_active":"2024-09-19T08:43:55.183213+0000","last_became_peered":"2024-09-19T08:43:55.183213+0000","last_unstale":"2024-09-19T08:43:56.156584+0000","last_undegraded":"2024-09-19T08:43:56.156584+0000","last_fullsized":"2024-09-19T08:43:56.156584+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:07:50.043881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.155580+0000","last_change":"2024-09-19T08:43:54.166062+0000","last_active":"2024-09-19T08:43:56.155580+0000","last_peered":"2024-09-19T08:43:56.155580+0000","last_clean":"2024-09-19T08:43:56.155580+0000","last_became_active":"2024-09-19T08:43:54.165346+0000","last_became_peered":"2024-09-19T08:43:54.165346+0000","last_unstale":"2024-09-19T08:43:56.155580+0000","last_undegraded":"2024-09-19T08:43:56.155580+0000","last_fullsized":"2024-09-19T08:43:56.155580+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:40.360335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.165461+0000","last_change":"2024-09-19T08:43:55.190220+0000","last_active":"2024-09-19T08:43:56.165461+0000","last_peered":"2024-09-19T08:43:56.165461+0000","last_clean":"2024-09-19T08:43:56.165461+0000","last_became_active":"2024-09-19T08:43:55.188987+0000","last_became_peered":"2024-09-19T08:43:55.188987+0000","last_unstale":"2024-09-19T08:43:56.165461+0000","last_undegraded":"2024-09-19T08:43:56.165461+0000","last_fullsized":"2024-09-19T08:43:56.165461+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:06:59.363181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.737357+0000","last_change":"2024-09-19T08:43:54.157928+0000","last_active":"2024-09-19T08:43:56.737357+0000","last_peered":"2024-09-19T08:43:56.737357+0000","last_clean":"2024-09-19T08:43:56.737357+0000","last_became_active":"2024-09-19T08:43:54.157571+0000","last_became_peered":"2024-09-19T08:43:54.157571+0000","last_unstale":"2024-09-19T08:43:56.737357+0000","last_undegraded":"2024-09-19T08:43:56.737357+0000","last_fullsized":"2024-09-19T08:43:56.737357+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:31.692173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.161089+0000","last_change":"2024-09-19T08:43:55.205294+0000","last_active":"2024-09-19T08:43:56.161089+0000","last_peered":"2024-09-19T08:43:56.161089+0000","last_clean":"2024-09-19T08:43:56.161089+0000","last_became_active":"2024-09-19T08:43:55.204683+0000","last_became_peered":"2024-09-19T08:43:55.204683+0000","last_unstale":"2024-09-19T08:43:56.161089+0000","last_undegraded":"2024-09-19T08:43:56.161089+0000","last_fullsized":"2024-09-19T08:43:56.161089+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:39:02.010421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.169697+0000","last_change":"2024-09-19T08:43:54.174126+0000","last_active":"2024-09-19T08:43:56.169697+0000","last_peered":"2024-09-19T08:43:56.169697+0000","last_clean":"2024-09-19T08:43:56.169697+0000","last_became_active":"2024-09-19T08:43:54.173403+0000","last_became_peered":"2024-09-19T08:43:54.173403+0000","last_unstale":"2024-09-19T08:43:56.169697+0000","last_undegraded":"2024-09-19T08:43:56.169697+0000","last_fullsized":"2024-09-19T08:43:56.169697+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:29:30.346662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.160539+0000","last_change":"2024-09-19T08:43:55.204555+0000","last_active":"2024-09-19T08:43:56.160539+0000","last_peered":"2024-09-19T08:43:56.160539+0000","last_clean":"2024-09-19T08:43:56.160539+0000","last_became_active":"2024-09-19T08:43:55.201572+0000","last_became_peered":"2024-09-19T08:43:55.201572+0000","last_unstale":"2024-09-19T08:43:56.160539+0000","last_undegraded":"2024-09-19T08:43:56.160539+0000","last_fullsized":"2024-09-19T08:43:56.160539+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:23.679385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.160261+0000","last_change":"2024-09-19T08:43:54.161229+0000","last_active":"2024-09-19T08:43:56.160261+0000","last_peered":"2024-09-19T08:43:56.160261+0000","last_clean":"2024-09-19T08:43:56.160261+0000","last_became_active":"2024-09-19T08:43:54.146726+0000","last_became_peered":"2024-09-19T08:43:54.146726+0000","last_unstale":"2024-09-19T08:43:56.160261+0000","last_undegraded":"2024-09-19T08:43:56.160261+0000","last_fullsized":"2024-09-19T08:43:56.160261+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:27.079573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736911+0000","last_change":"2024-09-19T08:43:55.180738+0000","last_active":"2024-09-19T08:43:56.736911+0000","last_peered":"2024-09-19T08:43:56.736911+0000","last_clean":"2024-09-19T08:43:56.736911+0000","last_became_active":"2024-09-19T08:43:55.179169+0000","last_became_peered":"2024-09-19T08:43:55.179169+0000","last_unstale":"2024-09-19T08:43:56.736911+0000","last_undegraded":"2024-09-19T08:43:56.736911+0000","last_fullsized":"2024-09-19T08:43:56.736911+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:23.679385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.735440+0000","last_change":"2024-09-19T08:43:54.148684+0000","last_active":"2024-09-19T08:43:56.735440+0000","last_peered":"2024-09-19T08:43:56.735440+0000","last_clean":"2024-09-19T08:43:56.735440+0000","last_became_active":"2024-09-19T08:43:54.148479+0000","last_became_peered":"2024-09-19T08:43:54.148479+0000","last_unstale":"2024-09-19T08:43:56.735440+0000","last_undegraded":"2024-09-19T08:43:56.735440+0000","last_fullsized":"2024-09-19T08:43:56.735440+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:30.763525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.155474+0000","last_change":"2024-09-19T08:43:55.181103+0000","last_active":"2024-09-19T08:43:56.155474+0000","last_peered":"2024-09-19T08:43:56.155474+0000","last_clean":"2024-09-19T08:43:56.155474+0000","last_became_active":"2024-09-19T08:43:55.180719+0000","last_became_peered":"2024-09-19T08:43:55.180719+0000","last_unstale":"2024-09-19T08:43:56.155474+0000","last_undegraded":"2024-09-19T08:43:56.155474+0000","last_fullsized":"2024-09-19T08:43:56.155474+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:23.898525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.156061+0000","last_change":"2024-09-19T08:43:54.163982+0000","last_active":"2024-09-19T08:43:56.156061+0000","last_peered":"2024-09-19T08:43:56.156061+0000","last_clean":"2024-09-19T08:43:56.156061+0000","last_became_active":"2024-09-19T08:43:54.163705+0000","last_became_peered":"2024-09-19T08:43:54.163705+0000","last_unstale":"2024-09-19T08:43:56.156061+0000","last_undegraded":"2024-09-19T08:43:56.156061+0000","last_fullsized":"2024-09-19T08:43:56.156061+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:39:47.549316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.735072+0000","last_change":"2024-09-19T08:43:55.192869+0000","last_active":"2024-09-19T08:43:56.735072+0000","last_peered":"2024-09-19T08:43:56.735072+0000","last_clean":"2024-09-19T08:43:56.735072+0000","last_became_active":"2024-09-19T08:43:55.192580+0000","last_became_peered":"2024-09-19T08:43:55.192580+0000","last_unstale":"2024-09-19T08:43:56.735072+0000","last_undegraded":"2024-09-19T08:43:56.735072+0000","last_fullsized":"2024-09-19T08:43:56.735072+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:09.148051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.169923+0000","last_change":"2024-09-19T08:43:54.163349+0000","last_active":"2024-09-19T08:43:56.169923+0000","last_peered":"2024-09-19T08:43:56.169923+0000","last_clean":"2024-09-19T08:43:56.169923+0000","last_became_active":"2024-09-19T08:43:54.162600+0000","last_became_peered":"2024-09-19T08:43:54.162600+0000","last_unstale":"2024-09-19T08:43:56.169923+0000","last_undegraded":"2024-09-19T08:43:56.169923+0000","last_fullsized":"2024-09-19T08:43:56.169923+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:54:27.116255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"71'128","reported_seq":355,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736808+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:43:56.736808+0000","last_peered":"2024-09-19T08:43:56.736808+0000","last_clean":"2024-09-19T08:43:56.736808+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:43:56.736808+0000","last_undegraded":"2024-09-19T08:43:56.736808+0000","last_fullsized":"2024-09-19T08:43:56.736808+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509513,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33012,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338828,"statfs":{"total":96636764160,"available":96602959872,"internally_reserved":0,"allocated":1863680,"data_stored":1145603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574922,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1265664,"data_stored":554689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509515,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1269760,"data_stored":555223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509515,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32420,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1261568,"data_stored":554619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542217,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1277952,"data_stored":556301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509512,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32420,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1261568,"data_stored":554619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509514,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1269760,"data_stored":554743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509512,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33024,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338816,"statfs":{"total":96636764160,"available":96602947584,"internally_reserved":0,"allocated":1875968,"data_stored":1146711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:44:02.661 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:44:02.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.792+0000 7fb92055f640 1 Processor -- start 2024-09-19T08:44:02.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.792+0000 7fb92055f640 1 -- start start 2024-09-19T08:44:02.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.792+0000 7fb92055f640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb918074c70 0x7fb91806d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:02.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.796+0000 7fb92055f640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb918160350 con 0x7fb9180b6f60 2024-09-19T08:44:02.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.796+0000 7fb92055f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb918160630 con 0x7fb9180b0280 2024-09-19T08:44:02.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.796+0000 7fb92055f640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fb918160910 con 0x7fb918074c70 2024-09-19T08:44:02.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.796+0000 7fb91eafc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb918074c70 0x7fb91806d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:02.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.796+0000 7fb91dafa640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fb9180b6f60 0x7fb9180ae340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:33182/0 (socket says 172.21.15.98:33182) 2024-09-19T08:44:02.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.796+0000 7fb91dafa640 1 -- 172.21.15.98:0/45680194 learned_addr learned my addr 172.21.15.98:0/45680194 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:02.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.796+0000 7fb91d2f9640 1 -- 172.21.15.98:0/45680194 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2543125173 0 0) 0x7fb918160350 con 0x7fb9180b6f60 2024-09-19T08:44:02.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.796+0000 7fb91d2f9640 1 -- 172.21.15.98:0/45680194 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb904003530 con 0x7fb9180b6f60 2024-09-19T08:44:02.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.796+0000 7fb91eafc640 1 -- 172.21.15.98:0/45680194 >> v1:172.21.15.139:6789/0 conn(0x7fb9180b0280 legacy=0x7fb9180b6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:02.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.796+0000 7fb91eafc640 1 -- 172.21.15.98:0/45680194 >> v1:172.21.15.98:6789/0 conn(0x7fb9180b6f60 legacy=0x7fb9180ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:02.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.796+0000 7fb91eafc640 1 -- 172.21.15.98:0/45680194 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb918160bf0 con 0x7fb918074c70 2024-09-19T08:44:02.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.796+0000 7fb91eafc640 1 --2- 172.21.15.98:0/45680194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb918074c70 0x7fb91806d1d0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fb910009d90 tx=0x7fb91001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3684ae54a7909ecb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:02.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.796+0000 7fb91d2f9640 1 -- 172.21.15.98:0/45680194 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9100b2070 con 0x7fb918074c70 2024-09-19T08:44:02.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.796+0000 7fb91d2f9640 1 -- 172.21.15.98:0/45680194 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9100a5810 con 0x7fb918074c70 2024-09-19T08:44:02.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.796+0000 7fb91d2f9640 1 -- 172.21.15.98:0/45680194 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9100a5be0 con 0x7fb918074c70 2024-09-19T08:44:02.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.796+0000 7fb92055f640 1 -- 172.21.15.98:0/45680194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb918074c70 msgr2=0x7fb91806d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:02.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.796+0000 7fb92055f640 1 --2- 172.21.15.98:0/45680194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb918074c70 0x7fb91806d1d0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fb910009d90 tx=0x7fb91001c8a0 comp rx=0 tx=0).stop 2024-09-19T08:44:02.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.796+0000 7fb92055f640 1 -- 172.21.15.98:0/45680194 shutdown_connections 2024-09-19T08:44:02.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.796+0000 7fb92055f640 1 --2- 172.21.15.98:0/45680194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb918074c70 0x7fb91806d1d0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:02.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.796+0000 7fb92055f640 1 -- 172.21.15.98:0/45680194 >> 172.21.15.98:0/45680194 conn(0x7fb918073170 msgr2=0x7fb918073580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:02.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.796+0000 7fb92055f640 1 -- 172.21.15.98:0/45680194 shutdown_connections 2024-09-19T08:44:02.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.800+0000 7fb92055f640 1 -- 172.21.15.98:0/45680194 wait complete. 2024-09-19T08:44:02.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.800+0000 7fb92055f640 1 Processor -- start 2024-09-19T08:44:02.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.800+0000 7fb92055f640 1 -- start start 2024-09-19T08:44:02.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.800+0000 7fb92055f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb918074c70 0x7fb918156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:02.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.800+0000 7fb92055f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9180b0280 0x7fb91809a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:02.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.800+0000 7fb92055f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9180b6f60 0x7fb91815f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:02.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.800+0000 7fb92055f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb9180af5f0 con 0x7fb9180b0280 2024-09-19T08:44:02.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.800+0000 7fb92055f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb9180af470 con 0x7fb9180b6f60 2024-09-19T08:44:02.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.800+0000 7fb92055f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb91816a390 con 0x7fb918074c70 2024-09-19T08:44:02.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.800+0000 7fb91dafa640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9180b0280 0x7fb91809a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:02.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.800+0000 7fb91dafa640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9180b0280 0x7fb91809a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:50782/0 (socket says 172.21.15.98:50782) 2024-09-19T08:44:02.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.800+0000 7fb91dafa640 1 -- 172.21.15.98:0/368657858 learned_addr learned my addr 172.21.15.98:0/368657858 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:02.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.800+0000 7fb91eafc640 1 --2- 172.21.15.98:0/368657858 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9180b6f60 0x7fb91815f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:02.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.800+0000 7fb91e2fb640 1 --2- 172.21.15.98:0/368657858 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb918074c70 0x7fb918156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:02.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.800+0000 7fb91dafa640 1 -- 172.21.15.98:0/368657858 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb918074c70 msgr2=0x7fb918156bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:02.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.800+0000 7fb91dafa640 1 --2- 172.21.15.98:0/368657858 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb918074c70 0x7fb918156bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:02.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.800+0000 7fb91dafa640 1 -- 172.21.15.98:0/368657858 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9180b6f60 msgr2=0x7fb91815f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:02.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.800+0000 7fb91dafa640 1 --2- 172.21.15.98:0/368657858 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9180b6f60 0x7fb91815f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:02.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.800+0000 7fb91dafa640 1 -- 172.21.15.98:0/368657858 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb91815fba0 con 0x7fb9180b0280 2024-09-19T08:44:02.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.800+0000 7fb91eafc640 1 --2- 172.21.15.98:0/368657858 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9180b6f60 0x7fb91815f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:02.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.800+0000 7fb91dafa640 1 --2- 172.21.15.98:0/368657858 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9180b0280 0x7fb91809a730 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fb90800c450 tx=0x7fb90800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:02.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.800+0000 7fb90effd640 1 -- 172.21.15.98:0/368657858 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb908017070 con 0x7fb9180b0280 2024-09-19T08:44:02.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.800+0000 7fb90effd640 1 -- 172.21.15.98:0/368657858 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb908005cf0 con 0x7fb9180b0280 2024-09-19T08:44:02.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.800+0000 7fb92055f640 1 -- 172.21.15.98:0/368657858 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb918157400 con 0x7fb9180b0280 2024-09-19T08:44:02.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.800+0000 7fb90effd640 1 -- 172.21.15.98:0/368657858 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb908007290 con 0x7fb9180b0280 2024-09-19T08:44:02.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.800+0000 7fb92055f640 1 -- 172.21.15.98:0/368657858 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9181579c0 con 0x7fb9180b0280 2024-09-19T08:44:02.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.800+0000 7fb92055f640 1 -- 172.21.15.98:0/368657858 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb91806d1d0 con 0x7fb9180b0280 2024-09-19T08:44:02.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.804+0000 7fb90effd640 1 -- 172.21.15.98:0/368657858 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fb90801a070 con 0x7fb9180b0280 2024-09-19T08:44:02.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.804+0000 7fb90effd640 1 --2- 172.21.15.98:0/368657858 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb8fc0679e0 0x7fb8fc069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:02.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.804+0000 7fb90effd640 1 -- 172.21.15.98:0/368657858 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fb908085190 con 0x7fb9180b0280 2024-09-19T08:44:02.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.804+0000 7fb91e2fb640 1 --2- 172.21.15.98:0/368657858 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb8fc0679e0 0x7fb8fc069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:02.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.804+0000 7fb91e2fb640 1 --2- 172.21.15.98:0/368657858 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb8fc0679e0 0x7fb8fc069ea0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb91400a750 tx=0x7fb914008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:02.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.808+0000 7fb90effd640 1 -- 172.21.15.98:0/368657858 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb908055c50 con 0x7fb9180b0280 2024-09-19T08:44:02.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.968+0000 7fb92055f640 1 -- 172.21.15.98:0/368657858 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb918157c00 con 0x7fb8fc0679e0 2024-09-19T08:44:02.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.984+0000 7fb90effd640 1 -- 172.21.15.98:0/368657858 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252426 (secure 0 0 0) 0x7fb91400a380 con 0x7fb8fc0679e0 2024-09-19T08:44:02.990 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:44:02.992 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:44:02.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.988+0000 7fb92055f640 1 -- 172.21.15.98:0/368657858 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb8fc0679e0 msgr2=0x7fb8fc069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:02.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.988+0000 7fb92055f640 1 --2- 172.21.15.98:0/368657858 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb8fc0679e0 0x7fb8fc069ea0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb91400a750 tx=0x7fb914008040 comp rx=0 tx=0).stop 2024-09-19T08:44:02.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.988+0000 7fb92055f640 1 -- 172.21.15.98:0/368657858 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9180b0280 msgr2=0x7fb91809a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:02.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.988+0000 7fb92055f640 1 --2- 172.21.15.98:0/368657858 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9180b0280 0x7fb91809a730 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fb90800c450 tx=0x7fb90800c910 comp rx=0 tx=0).stop 2024-09-19T08:44:02.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.988+0000 7fb92055f640 1 -- 172.21.15.98:0/368657858 shutdown_connections 2024-09-19T08:44:02.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.988+0000 7fb92055f640 1 --2- 172.21.15.98:0/368657858 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb8fc0679e0 0x7fb8fc069ea0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:02.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.988+0000 7fb92055f640 1 --2- 172.21.15.98:0/368657858 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb9180b6f60 0x7fb91815f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:02.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.988+0000 7fb92055f640 1 --2- 172.21.15.98:0/368657858 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb9180b0280 0x7fb91809a730 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:02.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.988+0000 7fb92055f640 1 --2- 172.21.15.98:0/368657858 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb918074c70 0x7fb918156bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:02.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.988+0000 7fb92055f640 1 -- 172.21.15.98:0/368657858 >> 172.21.15.98:0/368657858 conn(0x7fb918073170 msgr2=0x7fb9180b06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:02.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.988+0000 7fb92055f640 1 -- 172.21.15.98:0/368657858 shutdown_connections 2024-09-19T08:44:02.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:02.988+0000 7fb92055f640 1 -- 172.21.15.98:0/368657858 wait complete. 2024-09-19T08:44:03.008 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2024-09-19T08:44:02.892099+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260600,"kb_used_data":11080,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714120,"statfs":{"total":773094113280,"available":772827258880,"internally_reserved":0,"allocated":11345920,"data_stored":5622508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002730"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.156262+0000","last_change":"2024-09-19T08:43:55.175947+0000","last_active":"2024-09-19T08:43:56.156262+0000","last_peered":"2024-09-19T08:43:56.156262+0000","last_clean":"2024-09-19T08:43:56.156262+0000","last_became_active":"2024-09-19T08:43:55.175606+0000","last_became_peered":"2024-09-19T08:43:55.175606+0000","last_unstale":"2024-09-19T08:43:56.156262+0000","last_undegraded":"2024-09-19T08:43:56.156262+0000","last_fullsized":"2024-09-19T08:43:56.156262+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:46:51.742785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.161571+0000","last_change":"2024-09-19T08:43:55.206900+0000","last_active":"2024-09-19T08:43:56.161571+0000","last_peered":"2024-09-19T08:43:56.161571+0000","last_clean":"2024-09-19T08:43:56.161571+0000","last_became_active":"2024-09-19T08:43:55.200022+0000","last_became_peered":"2024-09-19T08:43:55.200022+0000","last_unstale":"2024-09-19T08:43:56.161571+0000","last_undegraded":"2024-09-19T08:43:56.161571+0000","last_fullsized":"2024-09-19T08:43:56.161571+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:39:48.558843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736619+0000","last_change":"2024-09-19T08:43:55.191789+0000","last_active":"2024-09-19T08:43:56.736619+0000","last_peered":"2024-09-19T08:43:56.736619+0000","last_clean":"2024-09-19T08:43:56.736619+0000","last_became_active":"2024-09-19T08:43:55.191065+0000","last_became_peered":"2024-09-19T08:43:55.191065+0000","last_unstale":"2024-09-19T08:43:56.736619+0000","last_undegraded":"2024-09-19T08:43:56.736619+0000","last_fullsized":"2024-09-19T08:43:56.736619+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:15.340286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.156243+0000","last_change":"2024-09-19T08:43:55.199879+0000","last_active":"2024-09-19T08:43:56.156243+0000","last_peered":"2024-09-19T08:43:56.156243+0000","last_clean":"2024-09-19T08:43:56.156243+0000","last_became_active":"2024-09-19T08:43:55.199679+0000","last_became_peered":"2024-09-19T08:43:55.199679+0000","last_unstale":"2024-09-19T08:43:56.156243+0000","last_undegraded":"2024-09-19T08:43:56.156243+0000","last_fullsized":"2024-09-19T08:43:56.156243+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:54:02.803710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736288+0000","last_change":"2024-09-19T08:43:55.168455+0000","last_active":"2024-09-19T08:43:56.736288+0000","last_peered":"2024-09-19T08:43:56.736288+0000","last_clean":"2024-09-19T08:43:56.736288+0000","last_became_active":"2024-09-19T08:43:55.168218+0000","last_became_peered":"2024-09-19T08:43:55.168218+0000","last_unstale":"2024-09-19T08:43:56.736288+0000","last_undegraded":"2024-09-19T08:43:56.736288+0000","last_fullsized":"2024-09-19T08:43:56.736288+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:05:28.010611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.575891+0000","last_change":"2024-09-19T08:43:55.169699+0000","last_active":"2024-09-19T08:43:56.575891+0000","last_peered":"2024-09-19T08:43:56.575891+0000","last_clean":"2024-09-19T08:43:56.575891+0000","last_became_active":"2024-09-19T08:43:55.169436+0000","last_became_peered":"2024-09-19T08:43:55.169436+0000","last_unstale":"2024-09-19T08:43:56.575891+0000","last_undegraded":"2024-09-19T08:43:56.575891+0000","last_fullsized":"2024-09-19T08:43:56.575891+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:29:31.356189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.169833+0000","last_change":"2024-09-19T08:43:55.194043+0000","last_active":"2024-09-19T08:43:56.169833+0000","last_peered":"2024-09-19T08:43:56.169833+0000","last_clean":"2024-09-19T08:43:56.169833+0000","last_became_active":"2024-09-19T08:43:55.193266+0000","last_became_peered":"2024-09-19T08:43:55.193266+0000","last_unstale":"2024-09-19T08:43:56.169833+0000","last_undegraded":"2024-09-19T08:43:56.169833+0000","last_fullsized":"2024-09-19T08:43:56.169833+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:28.089099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736336+0000","last_change":"2024-09-19T08:43:55.177179+0000","last_active":"2024-09-19T08:43:56.736336+0000","last_peered":"2024-09-19T08:43:56.736336+0000","last_clean":"2024-09-19T08:43:56.736336+0000","last_became_active":"2024-09-19T08:43:55.174397+0000","last_became_peered":"2024-09-19T08:43:55.174397+0000","last_unstale":"2024-09-19T08:43:56.736336+0000","last_undegraded":"2024-09-19T08:43:56.736336+0000","last_fullsized":"2024-09-19T08:43:56.736336+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:08:22.088644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.156620+0000","last_change":"2024-09-19T08:43:55.187917+0000","last_active":"2024-09-19T08:43:56.156620+0000","last_peered":"2024-09-19T08:43:56.156620+0000","last_clean":"2024-09-19T08:43:56.156620+0000","last_became_active":"2024-09-19T08:43:55.187097+0000","last_became_peered":"2024-09-19T08:43:55.187097+0000","last_unstale":"2024-09-19T08:43:56.156620+0000","last_undegraded":"2024-09-19T08:43:56.156620+0000","last_fullsized":"2024-09-19T08:43:56.156620+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:34:28.929131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.155582+0000","last_change":"2024-09-19T08:43:55.176966+0000","last_active":"2024-09-19T08:43:56.155582+0000","last_peered":"2024-09-19T08:43:56.155582+0000","last_clean":"2024-09-19T08:43:56.155582+0000","last_became_active":"2024-09-19T08:43:55.175980+0000","last_became_peered":"2024-09-19T08:43:55.175980+0000","last_unstale":"2024-09-19T08:43:56.155582+0000","last_undegraded":"2024-09-19T08:43:56.155582+0000","last_fullsized":"2024-09-19T08:43:56.155582+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:04:46.206086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.160446+0000","last_change":"2024-09-19T08:43:55.179425+0000","last_active":"2024-09-19T08:43:56.160446+0000","last_peered":"2024-09-19T08:43:56.160446+0000","last_clean":"2024-09-19T08:43:56.160446+0000","last_became_active":"2024-09-19T08:43:55.178734+0000","last_became_peered":"2024-09-19T08:43:55.178734+0000","last_unstale":"2024-09-19T08:43:56.160446+0000","last_undegraded":"2024-09-19T08:43:56.160446+0000","last_fullsized":"2024-09-19T08:43:56.160446+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:51.785619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.156543+0000","last_change":"2024-09-19T08:43:55.180274+0000","last_active":"2024-09-19T08:43:56.156543+0000","last_peered":"2024-09-19T08:43:56.156543+0000","last_clean":"2024-09-19T08:43:56.156543+0000","last_became_active":"2024-09-19T08:43:55.179930+0000","last_became_peered":"2024-09-19T08:43:55.179930+0000","last_unstale":"2024-09-19T08:43:56.156543+0000","last_undegraded":"2024-09-19T08:43:56.156543+0000","last_fullsized":"2024-09-19T08:43:56.156543+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:23.428495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.737001+0000","last_change":"2024-09-19T08:43:55.170744+0000","last_active":"2024-09-19T08:43:56.737001+0000","last_peered":"2024-09-19T08:43:56.737001+0000","last_clean":"2024-09-19T08:43:56.737001+0000","last_became_active":"2024-09-19T08:43:55.169601+0000","last_became_peered":"2024-09-19T08:43:55.169601+0000","last_unstale":"2024-09-19T08:43:56.737001+0000","last_undegraded":"2024-09-19T08:43:56.737001+0000","last_fullsized":"2024-09-19T08:43:56.737001+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:39:02.010421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736207+0000","last_change":"2024-09-19T08:43:55.173507+0000","last_active":"2024-09-19T08:43:56.736207+0000","last_peered":"2024-09-19T08:43:56.736207+0000","last_clean":"2024-09-19T08:43:56.736207+0000","last_became_active":"2024-09-19T08:43:55.172785+0000","last_became_peered":"2024-09-19T08:43:55.172785+0000","last_unstale":"2024-09-19T08:43:56.736207+0000","last_undegraded":"2024-09-19T08:43:56.736207+0000","last_fullsized":"2024-09-19T08:43:56.736207+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:02:46.286404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.575763+0000","last_change":"2024-09-19T08:43:55.170172+0000","last_active":"2024-09-19T08:43:56.575763+0000","last_peered":"2024-09-19T08:43:56.575763+0000","last_clean":"2024-09-19T08:43:56.575763+0000","last_became_active":"2024-09-19T08:43:55.169053+0000","last_became_peered":"2024-09-19T08:43:55.169053+0000","last_unstale":"2024-09-19T08:43:56.575763+0000","last_undegraded":"2024-09-19T08:43:56.575763+0000","last_fullsized":"2024-09-19T08:43:56.575763+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:23.435267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.155922+0000","last_change":"2024-09-19T08:43:55.181396+0000","last_active":"2024-09-19T08:43:56.155922+0000","last_peered":"2024-09-19T08:43:56.155922+0000","last_clean":"2024-09-19T08:43:56.155922+0000","last_became_active":"2024-09-19T08:43:55.180917+0000","last_became_peered":"2024-09-19T08:43:55.180917+0000","last_unstale":"2024-09-19T08:43:56.155922+0000","last_undegraded":"2024-09-19T08:43:56.155922+0000","last_fullsized":"2024-09-19T08:43:56.155922+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:40:18.247113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.161176+0000","last_change":"2024-09-19T08:43:55.186312+0000","last_active":"2024-09-19T08:43:56.161176+0000","last_peered":"2024-09-19T08:43:56.161176+0000","last_clean":"2024-09-19T08:43:56.161176+0000","last_became_active":"2024-09-19T08:43:55.185577+0000","last_became_peered":"2024-09-19T08:43:55.185577+0000","last_unstale":"2024-09-19T08:43:56.161176+0000","last_undegraded":"2024-09-19T08:43:56.161176+0000","last_fullsized":"2024-09-19T08:43:56.161176+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:41.369862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.165469+0000","last_change":"2024-09-19T08:43:55.209049+0000","last_active":"2024-09-19T08:43:56.165469+0000","last_peered":"2024-09-19T08:43:56.165469+0000","last_clean":"2024-09-19T08:43:56.165469+0000","last_became_active":"2024-09-19T08:43:55.208404+0000","last_became_peered":"2024-09-19T08:43:55.208404+0000","last_unstale":"2024-09-19T08:43:56.165469+0000","last_undegraded":"2024-09-19T08:43:56.165469+0000","last_fullsized":"2024-09-19T08:43:56.165469+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:55.541839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736693+0000","last_change":"2024-09-19T08:43:55.179067+0000","last_active":"2024-09-19T08:43:56.736693+0000","last_peered":"2024-09-19T08:43:56.736693+0000","last_clean":"2024-09-19T08:43:56.736693+0000","last_became_active":"2024-09-19T08:43:55.178109+0000","last_became_peered":"2024-09-19T08:43:55.178109+0000","last_unstale":"2024-09-19T08:43:56.736693+0000","last_undegraded":"2024-09-19T08:43:56.736693+0000","last_fullsized":"2024-09-19T08:43:56.736693+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:30:51.785619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.161144+0000","last_change":"2024-09-19T08:43:55.201728+0000","last_active":"2024-09-19T08:43:56.161144+0000","last_peered":"2024-09-19T08:43:56.161144+0000","last_clean":"2024-09-19T08:43:56.161144+0000","last_became_active":"2024-09-19T08:43:55.201157+0000","last_became_peered":"2024-09-19T08:43:55.201157+0000","last_unstale":"2024-09-19T08:43:56.161144+0000","last_undegraded":"2024-09-19T08:43:56.161144+0000","last_fullsized":"2024-09-19T08:43:56.161144+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:51:42.503330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.160768+0000","last_change":"2024-09-19T08:43:55.188891+0000","last_active":"2024-09-19T08:43:56.160768+0000","last_peered":"2024-09-19T08:43:56.160768+0000","last_clean":"2024-09-19T08:43:56.160768+0000","last_became_active":"2024-09-19T08:43:55.188094+0000","last_became_peered":"2024-09-19T08:43:55.188094+0000","last_unstale":"2024-09-19T08:43:56.160768+0000","last_undegraded":"2024-09-19T08:43:56.160768+0000","last_fullsized":"2024-09-19T08:43:56.160768+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:31:11.824242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.735712+0000","last_change":"2024-09-19T08:43:55.171322+0000","last_active":"2024-09-19T08:43:56.735712+0000","last_peered":"2024-09-19T08:43:56.735712+0000","last_clean":"2024-09-19T08:43:56.735712+0000","last_became_active":"2024-09-19T08:43:55.171040+0000","last_became_peered":"2024-09-19T08:43:55.171040+0000","last_unstale":"2024-09-19T08:43:56.735712+0000","last_undegraded":"2024-09-19T08:43:56.735712+0000","last_fullsized":"2024-09-19T08:43:56.735712+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:44.393216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.161536+0000","last_change":"2024-09-19T08:43:55.208256+0000","last_active":"2024-09-19T08:43:56.161536+0000","last_peered":"2024-09-19T08:43:56.161536+0000","last_clean":"2024-09-19T08:43:56.161536+0000","last_became_active":"2024-09-19T08:43:55.207547+0000","last_became_peered":"2024-09-19T08:43:55.207547+0000","last_unstale":"2024-09-19T08:43:56.161536+0000","last_undegraded":"2024-09-19T08:43:56.161536+0000","last_fullsized":"2024-09-19T08:43:56.161536+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:37:19.729827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.735102+0000","last_change":"2024-09-19T08:43:55.172018+0000","last_active":"2024-09-19T08:43:56.735102+0000","last_peered":"2024-09-19T08:43:56.735102+0000","last_clean":"2024-09-19T08:43:56.735102+0000","last_became_active":"2024-09-19T08:43:55.171714+0000","last_became_peered":"2024-09-19T08:43:55.171714+0000","last_unstale":"2024-09-19T08:43:56.735102+0000","last_undegraded":"2024-09-19T08:43:56.735102+0000","last_fullsized":"2024-09-19T08:43:56.735102+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:46:43.070713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.161452+0000","last_change":"2024-09-19T08:43:55.176249+0000","last_active":"2024-09-19T08:43:56.161452+0000","last_peered":"2024-09-19T08:43:56.161452+0000","last_clean":"2024-09-19T08:43:56.161452+0000","last_became_active":"2024-09-19T08:43:55.175331+0000","last_became_peered":"2024-09-19T08:43:55.175331+0000","last_unstale":"2024-09-19T08:43:56.161452+0000","last_undegraded":"2024-09-19T08:43:56.161452+0000","last_fullsized":"2024-09-19T08:43:56.161452+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:31:31.316017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.155735+0000","last_change":"2024-09-19T08:43:55.198912+0000","last_active":"2024-09-19T08:43:56.155735+0000","last_peered":"2024-09-19T08:43:56.155735+0000","last_clean":"2024-09-19T08:43:56.155735+0000","last_became_active":"2024-09-19T08:43:55.198693+0000","last_became_peered":"2024-09-19T08:43:55.198693+0000","last_unstale":"2024-09-19T08:43:56.155735+0000","last_undegraded":"2024-09-19T08:43:56.155735+0000","last_fullsized":"2024-09-19T08:43:56.155735+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:51:43.862034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.576308+0000","last_change":"2024-09-19T08:43:55.175621+0000","last_active":"2024-09-19T08:43:56.576308+0000","last_peered":"2024-09-19T08:43:56.576308+0000","last_clean":"2024-09-19T08:43:56.576308+0000","last_became_active":"2024-09-19T08:43:55.170608+0000","last_became_peered":"2024-09-19T08:43:55.170608+0000","last_unstale":"2024-09-19T08:43:56.576308+0000","last_undegraded":"2024-09-19T08:43:56.576308+0000","last_fullsized":"2024-09-19T08:43:56.576308+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:45:37.312790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.737173+0000","last_change":"2024-09-19T08:43:55.186309+0000","last_active":"2024-09-19T08:43:56.737173+0000","last_peered":"2024-09-19T08:43:56.737173+0000","last_clean":"2024-09-19T08:43:56.737173+0000","last_became_active":"2024-09-19T08:43:55.185612+0000","last_became_peered":"2024-09-19T08:43:55.185612+0000","last_unstale":"2024-09-19T08:43:56.737173+0000","last_undegraded":"2024-09-19T08:43:56.737173+0000","last_fullsized":"2024-09-19T08:43:56.737173+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:55:18.505199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.160111+0000","last_change":"2024-09-19T08:43:55.206052+0000","last_active":"2024-09-19T08:43:56.160111+0000","last_peered":"2024-09-19T08:43:56.160111+0000","last_clean":"2024-09-19T08:43:56.160111+0000","last_became_active":"2024-09-19T08:43:55.205325+0000","last_became_peered":"2024-09-19T08:43:55.205325+0000","last_unstale":"2024-09-19T08:43:56.160111+0000","last_undegraded":"2024-09-19T08:43:56.160111+0000","last_fullsized":"2024-09-19T08:43:56.160111+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:54:49.099389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.170072+0000","last_change":"2024-09-19T08:43:55.193647+0000","last_active":"2024-09-19T08:43:56.170072+0000","last_peered":"2024-09-19T08:43:56.170072+0000","last_clean":"2024-09-19T08:43:56.170072+0000","last_became_active":"2024-09-19T08:43:55.192947+0000","last_became_peered":"2024-09-19T08:43:55.192947+0000","last_unstale":"2024-09-19T08:43:56.170072+0000","last_undegraded":"2024-09-19T08:43:56.170072+0000","last_fullsized":"2024-09-19T08:43:56.170072+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:56.087733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.155719+0000","last_change":"2024-09-19T08:43:55.194691+0000","last_active":"2024-09-19T08:43:56.155719+0000","last_peered":"2024-09-19T08:43:56.155719+0000","last_clean":"2024-09-19T08:43:56.155719+0000","last_became_active":"2024-09-19T08:43:55.194441+0000","last_became_peered":"2024-09-19T08:43:55.194441+0000","last_unstale":"2024-09-19T08:43:56.155719+0000","last_undegraded":"2024-09-19T08:43:56.155719+0000","last_fullsized":"2024-09-19T08:43:56.155719+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:06:45.109119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.575520+0000","last_change":"2024-09-19T08:43:55.175081+0000","last_active":"2024-09-19T08:43:56.575520+0000","last_peered":"2024-09-19T08:43:56.575520+0000","last_clean":"2024-09-19T08:43:56.575520+0000","last_became_active":"2024-09-19T08:43:55.170833+0000","last_became_peered":"2024-09-19T08:43:55.170833+0000","last_unstale":"2024-09-19T08:43:56.575520+0000","last_undegraded":"2024-09-19T08:43:56.575520+0000","last_fullsized":"2024-09-19T08:43:56.575520+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:23.675291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.576479+0000","last_change":"2024-09-19T08:43:55.176053+0000","last_active":"2024-09-19T08:43:56.576479+0000","last_peered":"2024-09-19T08:43:56.576479+0000","last_clean":"2024-09-19T08:43:56.576479+0000","last_became_active":"2024-09-19T08:43:55.175394+0000","last_became_peered":"2024-09-19T08:43:55.175394+0000","last_unstale":"2024-09-19T08:43:56.576479+0000","last_undegraded":"2024-09-19T08:43:56.576479+0000","last_fullsized":"2024-09-19T08:43:56.576479+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:46:30.215239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"101'4","reported_seq":18,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.170239+0000","last_change":"2024-09-19T08:43:54.173670+0000","last_active":"2024-09-19T08:43:56.170239+0000","last_peered":"2024-09-19T08:43:56.170239+0000","last_clean":"2024-09-19T08:43:56.170239+0000","last_became_active":"2024-09-19T08:43:54.172904+0000","last_became_peered":"2024-09-19T08:43:54.172904+0000","last_unstale":"2024-09-19T08:43:56.170239+0000","last_undegraded":"2024-09-19T08:43:56.170239+0000","last_fullsized":"2024-09-19T08:43:56.170239+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:22.425741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.165640+0000","last_change":"2024-09-19T08:43:55.187361+0000","last_active":"2024-09-19T08:43:56.165640+0000","last_peered":"2024-09-19T08:43:56.165640+0000","last_clean":"2024-09-19T08:43:56.165640+0000","last_became_active":"2024-09-19T08:43:55.186144+0000","last_became_peered":"2024-09-19T08:43:55.186144+0000","last_unstale":"2024-09-19T08:43:56.165640+0000","last_undegraded":"2024-09-19T08:43:56.165640+0000","last_fullsized":"2024-09-19T08:43:56.165640+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:30:19.243180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"101'1","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736331+0000","last_change":"2024-09-19T08:43:54.175936+0000","last_active":"2024-09-19T08:43:56.736331+0000","last_peered":"2024-09-19T08:43:56.736331+0000","last_clean":"2024-09-19T08:43:56.736331+0000","last_became_active":"2024-09-19T08:43:54.175215+0000","last_became_peered":"2024-09-19T08:43:54.175215+0000","last_unstale":"2024-09-19T08:43:56.736331+0000","last_undegraded":"2024-09-19T08:43:56.736331+0000","last_fullsized":"2024-09-19T08:43:56.736331+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:27:33.779456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.576311+0000","last_change":"2024-09-19T08:43:55.182209+0000","last_active":"2024-09-19T08:43:56.576311+0000","last_peered":"2024-09-19T08:43:56.576311+0000","last_clean":"2024-09-19T08:43:56.576311+0000","last_became_active":"2024-09-19T08:43:55.181787+0000","last_became_peered":"2024-09-19T08:43:55.181787+0000","last_unstale":"2024-09-19T08:43:56.576311+0000","last_undegraded":"2024-09-19T08:43:56.576311+0000","last_fullsized":"2024-09-19T08:43:56.576311+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:54:28.125782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"101'2","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736070+0000","last_change":"2024-09-19T08:43:54.148059+0000","last_active":"2024-09-19T08:43:56.736070+0000","last_peered":"2024-09-19T08:43:56.736070+0000","last_clean":"2024-09-19T08:43:56.736070+0000","last_became_active":"2024-09-19T08:43:54.147311+0000","last_became_peered":"2024-09-19T08:43:54.147311+0000","last_unstale":"2024-09-19T08:43:56.736070+0000","last_undegraded":"2024-09-19T08:43:56.736070+0000","last_fullsized":"2024-09-19T08:43:56.736070+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:59:59.076766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.737066+0000","last_change":"2024-09-19T08:43:55.181960+0000","last_active":"2024-09-19T08:43:56.737066+0000","last_peered":"2024-09-19T08:43:56.737066+0000","last_clean":"2024-09-19T08:43:56.737066+0000","last_became_active":"2024-09-19T08:43:55.181038+0000","last_became_peered":"2024-09-19T08:43:55.181038+0000","last_unstale":"2024-09-19T08:43:56.737066+0000","last_undegraded":"2024-09-19T08:43:56.737066+0000","last_fullsized":"2024-09-19T08:43:56.737066+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:41.369862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"101'2","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.735543+0000","last_change":"2024-09-19T08:43:54.148088+0000","last_active":"2024-09-19T08:43:56.735543+0000","last_peered":"2024-09-19T08:43:56.735543+0000","last_clean":"2024-09-19T08:43:56.735543+0000","last_became_active":"2024-09-19T08:43:54.147729+0000","last_became_peered":"2024-09-19T08:43:54.147729+0000","last_unstale":"2024-09-19T08:43:56.735543+0000","last_undegraded":"2024-09-19T08:43:56.735543+0000","last_fullsized":"2024-09-19T08:43:56.735543+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:01:57.810659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.155662+0000","last_change":"2024-09-19T08:43:55.189242+0000","last_active":"2024-09-19T08:43:56.155662+0000","last_peered":"2024-09-19T08:43:56.155662+0000","last_clean":"2024-09-19T08:43:56.155662+0000","last_became_active":"2024-09-19T08:43:55.188976+0000","last_became_peered":"2024-09-19T08:43:55.188976+0000","last_unstale":"2024-09-19T08:43:56.155662+0000","last_undegraded":"2024-09-19T08:43:56.155662+0000","last_fullsized":"2024-09-19T08:43:56.155662+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:05.505122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"101'1","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.169629+0000","last_change":"2024-09-19T08:43:54.174001+0000","last_active":"2024-09-19T08:43:56.169629+0000","last_peered":"2024-09-19T08:43:56.169629+0000","last_clean":"2024-09-19T08:43:56.169629+0000","last_became_active":"2024-09-19T08:43:54.171901+0000","last_became_peered":"2024-09-19T08:43:54.171901+0000","last_unstale":"2024-09-19T08:43:56.169629+0000","last_undegraded":"2024-09-19T08:43:56.169629+0000","last_fullsized":"2024-09-19T08:43:56.169629+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:22.665765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.160211+0000","last_change":"2024-09-19T08:43:55.205723+0000","last_active":"2024-09-19T08:43:56.160211+0000","last_peered":"2024-09-19T08:43:56.160211+0000","last_clean":"2024-09-19T08:43:56.160211+0000","last_became_active":"2024-09-19T08:43:55.205248+0000","last_became_peered":"2024-09-19T08:43:55.205248+0000","last_unstale":"2024-09-19T08:43:56.160211+0000","last_undegraded":"2024-09-19T08:43:56.160211+0000","last_fullsized":"2024-09-19T08:43:56.160211+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:40:15.340286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"101'1","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.735857+0000","last_change":"2024-09-19T08:43:54.169422+0000","last_active":"2024-09-19T08:43:56.735857+0000","last_peered":"2024-09-19T08:43:56.735857+0000","last_clean":"2024-09-19T08:43:56.735857+0000","last_became_active":"2024-09-19T08:43:54.169152+0000","last_became_peered":"2024-09-19T08:43:54.169152+0000","last_unstale":"2024-09-19T08:43:56.735857+0000","last_undegraded":"2024-09-19T08:43:56.735857+0000","last_fullsized":"2024-09-19T08:43:56.735857+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:54:49.167389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.155373+0000","last_change":"2024-09-19T08:43:55.178286+0000","last_active":"2024-09-19T08:43:56.155373+0000","last_peered":"2024-09-19T08:43:56.155373+0000","last_clean":"2024-09-19T08:43:56.155373+0000","last_became_active":"2024-09-19T08:43:55.177900+0000","last_became_peered":"2024-09-19T08:43:55.177900+0000","last_unstale":"2024-09-19T08:43:56.155373+0000","last_undegraded":"2024-09-19T08:43:56.155373+0000","last_fullsized":"2024-09-19T08:43:56.155373+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:08:10.535405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.159848+0000","last_change":"2024-09-19T08:43:54.170080+0000","last_active":"2024-09-19T08:43:56.159848+0000","last_peered":"2024-09-19T08:43:56.159848+0000","last_clean":"2024-09-19T08:43:56.159848+0000","last_became_active":"2024-09-19T08:43:54.168481+0000","last_became_peered":"2024-09-19T08:43:54.168481+0000","last_unstale":"2024-09-19T08:43:56.159848+0000","last_undegraded":"2024-09-19T08:43:56.159848+0000","last_fullsized":"2024-09-19T08:43:56.159848+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:55.078207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.169711+0000","last_change":"2024-09-19T08:43:55.195850+0000","last_active":"2024-09-19T08:43:56.169711+0000","last_peered":"2024-09-19T08:43:56.169711+0000","last_clean":"2024-09-19T08:43:56.169711+0000","last_became_active":"2024-09-19T08:43:55.194786+0000","last_became_peered":"2024-09-19T08:43:55.194786+0000","last_unstale":"2024-09-19T08:43:56.169711+0000","last_undegraded":"2024-09-19T08:43:56.169711+0000","last_fullsized":"2024-09-19T08:43:56.169711+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:31:59.684059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.155893+0000","last_change":"2024-09-19T08:43:54.158627+0000","last_active":"2024-09-19T08:43:56.155893+0000","last_peered":"2024-09-19T08:43:56.155893+0000","last_clean":"2024-09-19T08:43:56.155893+0000","last_became_active":"2024-09-19T08:43:54.158333+0000","last_became_peered":"2024-09-19T08:43:54.158333+0000","last_unstale":"2024-09-19T08:43:56.155893+0000","last_undegraded":"2024-09-19T08:43:56.155893+0000","last_fullsized":"2024-09-19T08:43:56.155893+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:31:30.306491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.170558+0000","last_change":"2024-09-19T08:43:55.195678+0000","last_active":"2024-09-19T08:43:56.170558+0000","last_peered":"2024-09-19T08:43:56.170558+0000","last_clean":"2024-09-19T08:43:56.170558+0000","last_became_active":"2024-09-19T08:43:55.194952+0000","last_became_peered":"2024-09-19T08:43:55.194952+0000","last_unstale":"2024-09-19T08:43:56.170558+0000","last_undegraded":"2024-09-19T08:43:56.170558+0000","last_fullsized":"2024-09-19T08:43:56.170558+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:37.648111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736244+0000","last_change":"2024-09-19T08:43:54.148128+0000","last_active":"2024-09-19T08:43:56.736244+0000","last_peered":"2024-09-19T08:43:56.736244+0000","last_clean":"2024-09-19T08:43:56.736244+0000","last_became_active":"2024-09-19T08:43:54.147519+0000","last_became_peered":"2024-09-19T08:43:54.147519+0000","last_unstale":"2024-09-19T08:43:56.736244+0000","last_undegraded":"2024-09-19T08:43:56.736244+0000","last_fullsized":"2024-09-19T08:43:56.736244+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:04:22.016732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.165566+0000","last_change":"2024-09-19T08:43:55.190134+0000","last_active":"2024-09-19T08:43:56.165566+0000","last_peered":"2024-09-19T08:43:56.165566+0000","last_clean":"2024-09-19T08:43:56.165566+0000","last_became_active":"2024-09-19T08:43:55.189377+0000","last_became_peered":"2024-09-19T08:43:55.189377+0000","last_unstale":"2024-09-19T08:43:56.165566+0000","last_undegraded":"2024-09-19T08:43:56.165566+0000","last_fullsized":"2024-09-19T08:43:56.165566+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:15:39.583209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736887+0000","last_change":"2024-09-19T08:43:54.169583+0000","last_active":"2024-09-19T08:43:56.736887+0000","last_peered":"2024-09-19T08:43:56.736887+0000","last_clean":"2024-09-19T08:43:56.736887+0000","last_became_active":"2024-09-19T08:43:54.169238+0000","last_became_peered":"2024-09-19T08:43:54.169238+0000","last_unstale":"2024-09-19T08:43:56.736887+0000","last_undegraded":"2024-09-19T08:43:56.736887+0000","last_fullsized":"2024-09-19T08:43:56.736887+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:42.579423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.737558+0000","last_change":"2024-09-19T08:43:55.200840+0000","last_active":"2024-09-19T08:43:56.737558+0000","last_peered":"2024-09-19T08:43:56.737558+0000","last_clean":"2024-09-19T08:43:56.737558+0000","last_became_active":"2024-09-19T08:43:55.200586+0000","last_became_peered":"2024-09-19T08:43:55.200586+0000","last_unstale":"2024-09-19T08:43:56.737558+0000","last_undegraded":"2024-09-19T08:43:56.737558+0000","last_fullsized":"2024-09-19T08:43:56.737558+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:22.019255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"101'3","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.155459+0000","last_change":"2024-09-19T08:43:54.156828+0000","last_active":"2024-09-19T08:43:56.155459+0000","last_peered":"2024-09-19T08:43:56.155459+0000","last_clean":"2024-09-19T08:43:56.155459+0000","last_became_active":"2024-09-19T08:43:54.156521+0000","last_became_peered":"2024-09-19T08:43:54.156521+0000","last_unstale":"2024-09-19T08:43:56.155459+0000","last_undegraded":"2024-09-19T08:43:56.155459+0000","last_fullsized":"2024-09-19T08:43:56.155459+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:51:41.493803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.170290+0000","last_change":"2024-09-19T08:43:55.205270+0000","last_active":"2024-09-19T08:43:56.170290+0000","last_peered":"2024-09-19T08:43:56.170290+0000","last_clean":"2024-09-19T08:43:56.170290+0000","last_became_active":"2024-09-19T08:43:55.204556+0000","last_became_peered":"2024-09-19T08:43:55.204556+0000","last_unstale":"2024-09-19T08:43:56.170290+0000","last_undegraded":"2024-09-19T08:43:56.170290+0000","last_fullsized":"2024-09-19T08:43:56.170290+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:53:33.564978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"101'6","reported_seq":20,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.156666+0000","last_change":"2024-09-19T08:43:54.159068+0000","last_active":"2024-09-19T08:43:56.156666+0000","last_peered":"2024-09-19T08:43:56.156666+0000","last_clean":"2024-09-19T08:43:56.156666+0000","last_became_active":"2024-09-19T08:43:54.158698+0000","last_became_peered":"2024-09-19T08:43:54.158698+0000","last_unstale":"2024-09-19T08:43:56.156666+0000","last_undegraded":"2024-09-19T08:43:56.156666+0000","last_fullsized":"2024-09-19T08:43:56.156666+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:39:01.903420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736353+0000","last_change":"2024-09-19T08:43:55.194646+0000","last_active":"2024-09-19T08:43:56.736353+0000","last_peered":"2024-09-19T08:43:56.736353+0000","last_clean":"2024-09-19T08:43:56.736353+0000","last_became_active":"2024-09-19T08:43:55.194235+0000","last_became_peered":"2024-09-19T08:43:55.194235+0000","last_unstale":"2024-09-19T08:43:56.736353+0000","last_undegraded":"2024-09-19T08:43:56.736353+0000","last_fullsized":"2024-09-19T08:43:56.736353+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:06:29.306359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"101'2","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.575865+0000","last_change":"2024-09-19T08:43:54.165780+0000","last_active":"2024-09-19T08:43:56.575865+0000","last_peered":"2024-09-19T08:43:56.575865+0000","last_clean":"2024-09-19T08:43:56.575865+0000","last_became_active":"2024-09-19T08:43:54.165527+0000","last_became_peered":"2024-09-19T08:43:54.165527+0000","last_unstale":"2024-09-19T08:43:56.575865+0000","last_undegraded":"2024-09-19T08:43:56.575865+0000","last_fullsized":"2024-09-19T08:43:56.575865+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:18.059525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736953+0000","last_change":"2024-09-19T08:43:55.167204+0000","last_active":"2024-09-19T08:43:56.736953+0000","last_peered":"2024-09-19T08:43:56.736953+0000","last_clean":"2024-09-19T08:43:56.736953+0000","last_became_active":"2024-09-19T08:43:55.166818+0000","last_became_peered":"2024-09-19T08:43:55.166818+0000","last_unstale":"2024-09-19T08:43:56.736953+0000","last_undegraded":"2024-09-19T08:43:56.736953+0000","last_fullsized":"2024-09-19T08:43:56.736953+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:08:12.006209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.164934+0000","last_change":"2024-09-19T08:43:54.167563+0000","last_active":"2024-09-19T08:43:56.164934+0000","last_peered":"2024-09-19T08:43:56.164934+0000","last_clean":"2024-09-19T08:43:56.164934+0000","last_became_active":"2024-09-19T08:43:54.166815+0000","last_became_peered":"2024-09-19T08:43:54.166815+0000","last_unstale":"2024-09-19T08:43:56.164934+0000","last_undegraded":"2024-09-19T08:43:56.164934+0000","last_fullsized":"2024-09-19T08:43:56.164934+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:03.023614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.735248+0000","last_change":"2024-09-19T08:43:55.200724+0000","last_active":"2024-09-19T08:43:56.735248+0000","last_peered":"2024-09-19T08:43:56.735248+0000","last_clean":"2024-09-19T08:43:56.735248+0000","last_became_active":"2024-09-19T08:43:55.200505+0000","last_became_peered":"2024-09-19T08:43:55.200505+0000","last_unstale":"2024-09-19T08:43:56.735248+0000","last_undegraded":"2024-09-19T08:43:56.735248+0000","last_fullsized":"2024-09-19T08:43:56.735248+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:51:14.085642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"101'8","reported_seq":22,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.735407+0000","last_change":"2024-09-19T08:43:54.153224+0000","last_active":"2024-09-19T08:43:56.735407+0000","last_peered":"2024-09-19T08:43:56.735407+0000","last_clean":"2024-09-19T08:43:56.735407+0000","last_became_active":"2024-09-19T08:43:54.152861+0000","last_became_peered":"2024-09-19T08:43:54.152861+0000","last_unstale":"2024-09-19T08:43:56.735407+0000","last_undegraded":"2024-09-19T08:43:56.735407+0000","last_fullsized":"2024-09-19T08:43:56.735407+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:08:28.306074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736714+0000","last_change":"2024-09-19T08:43:55.179921+0000","last_active":"2024-09-19T08:43:56.736714+0000","last_peered":"2024-09-19T08:43:56.736714+0000","last_clean":"2024-09-19T08:43:56.736714+0000","last_became_active":"2024-09-19T08:43:55.179672+0000","last_became_peered":"2024-09-19T08:43:55.179672+0000","last_unstale":"2024-09-19T08:43:56.736714+0000","last_undegraded":"2024-09-19T08:43:56.736714+0000","last_fullsized":"2024-09-19T08:43:56.736714+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:03.688703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"101'1","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.157106+0000","last_change":"2024-09-19T08:43:54.175081+0000","last_active":"2024-09-19T08:43:56.157106+0000","last_peered":"2024-09-19T08:43:56.157106+0000","last_clean":"2024-09-19T08:43:56.157106+0000","last_became_active":"2024-09-19T08:43:54.174828+0000","last_became_peered":"2024-09-19T08:43:54.174828+0000","last_unstale":"2024-09-19T08:43:56.157106+0000","last_undegraded":"2024-09-19T08:43:56.157106+0000","last_fullsized":"2024-09-19T08:43:56.157106+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:42:04.302601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.576192+0000","last_change":"2024-09-19T08:43:55.169745+0000","last_active":"2024-09-19T08:43:56.576192+0000","last_peered":"2024-09-19T08:43:56.576192+0000","last_clean":"2024-09-19T08:43:56.576192+0000","last_became_active":"2024-09-19T08:43:55.169407+0000","last_became_peered":"2024-09-19T08:43:55.169407+0000","last_unstale":"2024-09-19T08:43:56.576192+0000","last_undegraded":"2024-09-19T08:43:56.576192+0000","last_fullsized":"2024-09-19T08:43:56.576192+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:26:01.318672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"101'2","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736434+0000","last_change":"2024-09-19T08:43:54.167839+0000","last_active":"2024-09-19T08:43:56.736434+0000","last_peered":"2024-09-19T08:43:56.736434+0000","last_clean":"2024-09-19T08:43:56.736434+0000","last_became_active":"2024-09-19T08:43:54.167548+0000","last_became_peered":"2024-09-19T08:43:54.167548+0000","last_unstale":"2024-09-19T08:43:56.736434+0000","last_undegraded":"2024-09-19T08:43:56.736434+0000","last_fullsized":"2024-09-19T08:43:56.736434+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:37.105176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.737017+0000","last_change":"2024-09-19T08:43:55.186681+0000","last_active":"2024-09-19T08:43:56.737017+0000","last_peered":"2024-09-19T08:43:56.737017+0000","last_clean":"2024-09-19T08:43:56.737017+0000","last_became_active":"2024-09-19T08:43:55.185978+0000","last_became_peered":"2024-09-19T08:43:55.185978+0000","last_unstale":"2024-09-19T08:43:56.737017+0000","last_undegraded":"2024-09-19T08:43:56.737017+0000","last_fullsized":"2024-09-19T08:43:56.737017+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:51:42.503330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"101'1","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.165383+0000","last_change":"2024-09-19T08:43:54.166019+0000","last_active":"2024-09-19T08:43:56.165383+0000","last_peered":"2024-09-19T08:43:56.165383+0000","last_clean":"2024-09-19T08:43:56.165383+0000","last_became_active":"2024-09-19T08:43:54.165330+0000","last_became_peered":"2024-09-19T08:43:54.165330+0000","last_unstale":"2024-09-19T08:43:56.165383+0000","last_undegraded":"2024-09-19T08:43:56.165383+0000","last_fullsized":"2024-09-19T08:43:56.165383+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:02:58.278139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736946+0000","last_change":"2024-09-19T08:43:55.204016+0000","last_active":"2024-09-19T08:43:56.736946+0000","last_peered":"2024-09-19T08:43:56.736946+0000","last_clean":"2024-09-19T08:43:56.736946+0000","last_became_active":"2024-09-19T08:43:55.203041+0000","last_became_peered":"2024-09-19T08:43:55.203041+0000","last_unstale":"2024-09-19T08:43:56.736946+0000","last_undegraded":"2024-09-19T08:43:56.736946+0000","last_fullsized":"2024-09-19T08:43:56.736946+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:31:11.824242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.169927+0000","last_change":"2024-09-19T08:43:54.171525+0000","last_active":"2024-09-19T08:43:56.169927+0000","last_peered":"2024-09-19T08:43:56.169927+0000","last_clean":"2024-09-19T08:43:56.169927+0000","last_became_active":"2024-09-19T08:43:54.170720+0000","last_became_peered":"2024-09-19T08:43:54.170720+0000","last_unstale":"2024-09-19T08:43:56.169927+0000","last_undegraded":"2024-09-19T08:43:56.169927+0000","last_fullsized":"2024-09-19T08:43:56.169927+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:26:00.309146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736210+0000","last_change":"2024-09-19T08:43:55.191951+0000","last_active":"2024-09-19T08:43:56.736210+0000","last_peered":"2024-09-19T08:43:56.736210+0000","last_clean":"2024-09-19T08:43:56.736210+0000","last_became_active":"2024-09-19T08:43:55.191187+0000","last_became_peered":"2024-09-19T08:43:55.191187+0000","last_unstale":"2024-09-19T08:43:56.736210+0000","last_undegraded":"2024-09-19T08:43:56.736210+0000","last_fullsized":"2024-09-19T08:43:56.736210+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:09:54.189345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"101'2","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.575972+0000","last_change":"2024-09-19T08:43:54.156834+0000","last_active":"2024-09-19T08:43:56.575972+0000","last_peered":"2024-09-19T08:43:56.575972+0000","last_clean":"2024-09-19T08:43:56.575972+0000","last_became_active":"2024-09-19T08:43:54.156436+0000","last_became_peered":"2024-09-19T08:43:54.156436+0000","last_unstale":"2024-09-19T08:43:56.575972+0000","last_undegraded":"2024-09-19T08:43:56.575972+0000","last_fullsized":"2024-09-19T08:43:56.575972+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:06:49.884305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.155968+0000","last_change":"2024-09-19T08:43:55.180818+0000","last_active":"2024-09-19T08:43:56.155968+0000","last_peered":"2024-09-19T08:43:56.155968+0000","last_clean":"2024-09-19T08:43:56.155968+0000","last_became_active":"2024-09-19T08:43:55.180523+0000","last_became_peered":"2024-09-19T08:43:55.180523+0000","last_unstale":"2024-09-19T08:43:56.155968+0000","last_undegraded":"2024-09-19T08:43:56.155968+0000","last_fullsized":"2024-09-19T08:43:56.155968+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:19:35.054836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.735879+0000","last_change":"2024-09-19T08:43:54.148186+0000","last_active":"2024-09-19T08:43:56.735879+0000","last_peered":"2024-09-19T08:43:56.735879+0000","last_clean":"2024-09-19T08:43:56.735879+0000","last_became_active":"2024-09-19T08:43:54.147929+0000","last_became_peered":"2024-09-19T08:43:54.147929+0000","last_unstale":"2024-09-19T08:43:56.735879+0000","last_undegraded":"2024-09-19T08:43:56.735879+0000","last_fullsized":"2024-09-19T08:43:56.735879+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:22:11.522698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736419+0000","last_change":"2024-09-19T08:43:55.185737+0000","last_active":"2024-09-19T08:43:56.736419+0000","last_peered":"2024-09-19T08:43:56.736419+0000","last_clean":"2024-09-19T08:43:56.736419+0000","last_became_active":"2024-09-19T08:43:55.184994+0000","last_became_peered":"2024-09-19T08:43:55.184994+0000","last_unstale":"2024-09-19T08:43:56.736419+0000","last_undegraded":"2024-09-19T08:43:56.736419+0000","last_fullsized":"2024-09-19T08:43:56.736419+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:54:49.099389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.575901+0000","last_change":"2024-09-19T08:43:54.156400+0000","last_active":"2024-09-19T08:43:56.575901+0000","last_peered":"2024-09-19T08:43:56.575901+0000","last_clean":"2024-09-19T08:43:56.575901+0000","last_became_active":"2024-09-19T08:43:54.156000+0000","last_became_peered":"2024-09-19T08:43:54.156000+0000","last_unstale":"2024-09-19T08:43:56.575901+0000","last_undegraded":"2024-09-19T08:43:56.575901+0000","last_fullsized":"2024-09-19T08:43:56.575901+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:11.225788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736338+0000","last_change":"2024-09-19T08:43:55.181342+0000","last_active":"2024-09-19T08:43:56.736338+0000","last_peered":"2024-09-19T08:43:56.736338+0000","last_clean":"2024-09-19T08:43:56.736338+0000","last_became_active":"2024-09-19T08:43:55.181018+0000","last_became_peered":"2024-09-19T08:43:55.181018+0000","last_unstale":"2024-09-19T08:43:56.736338+0000","last_undegraded":"2024-09-19T08:43:56.736338+0000","last_fullsized":"2024-09-19T08:43:56.736338+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:43:16.137533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.155966+0000","last_change":"2024-09-19T08:43:54.164739+0000","last_active":"2024-09-19T08:43:56.155966+0000","last_peered":"2024-09-19T08:43:56.155966+0000","last_clean":"2024-09-19T08:43:56.155966+0000","last_became_active":"2024-09-19T08:43:54.164107+0000","last_became_peered":"2024-09-19T08:43:54.164107+0000","last_unstale":"2024-09-19T08:43:56.155966+0000","last_undegraded":"2024-09-19T08:43:56.155966+0000","last_fullsized":"2024-09-19T08:43:56.155966+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:37:18.720301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.165560+0000","last_change":"2024-09-19T08:43:55.190836+0000","last_active":"2024-09-19T08:43:56.165560+0000","last_peered":"2024-09-19T08:43:56.165560+0000","last_clean":"2024-09-19T08:43:56.165560+0000","last_became_active":"2024-09-19T08:43:55.190000+0000","last_became_peered":"2024-09-19T08:43:55.190000+0000","last_unstale":"2024-09-19T08:43:56.165560+0000","last_undegraded":"2024-09-19T08:43:56.165560+0000","last_fullsized":"2024-09-19T08:43:56.165560+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:38:24.253914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"101'2","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.737656+0000","last_change":"2024-09-19T08:43:54.156950+0000","last_active":"2024-09-19T08:43:56.737656+0000","last_peered":"2024-09-19T08:43:56.737656+0000","last_clean":"2024-09-19T08:43:56.737656+0000","last_became_active":"2024-09-19T08:43:54.156680+0000","last_became_peered":"2024-09-19T08:43:54.156680+0000","last_unstale":"2024-09-19T08:43:56.737656+0000","last_undegraded":"2024-09-19T08:43:56.737656+0000","last_fullsized":"2024-09-19T08:43:56.737656+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:17:05.043117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.156975+0000","last_change":"2024-09-19T08:43:55.187342+0000","last_active":"2024-09-19T08:43:56.156975+0000","last_peered":"2024-09-19T08:43:56.156975+0000","last_clean":"2024-09-19T08:43:56.156975+0000","last_became_active":"2024-09-19T08:43:55.186731+0000","last_became_peered":"2024-09-19T08:43:55.186731+0000","last_unstale":"2024-09-19T08:43:56.156975+0000","last_undegraded":"2024-09-19T08:43:56.156975+0000","last_fullsized":"2024-09-19T08:43:56.156975+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:49.028651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"102'1","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.165662+0000","last_change":"2024-09-19T08:43:54.175992+0000","last_active":"2024-09-19T08:43:56.165662+0000","last_peered":"2024-09-19T08:43:56.165662+0000","last_clean":"2024-09-19T08:43:56.165662+0000","last_became_active":"2024-09-19T08:43:54.175245+0000","last_became_peered":"2024-09-19T08:43:54.175245+0000","last_unstale":"2024-09-19T08:43:56.165662+0000","last_undegraded":"2024-09-19T08:43:56.165662+0000","last_fullsized":"2024-09-19T08:43:56.165662+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:14.806379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.156584+0000","last_change":"2024-09-19T08:43:55.183466+0000","last_active":"2024-09-19T08:43:56.156584+0000","last_peered":"2024-09-19T08:43:56.156584+0000","last_clean":"2024-09-19T08:43:56.156584+0000","last_became_active":"2024-09-19T08:43:55.183213+0000","last_became_peered":"2024-09-19T08:43:55.183213+0000","last_unstale":"2024-09-19T08:43:56.156584+0000","last_undegraded":"2024-09-19T08:43:56.156584+0000","last_fullsized":"2024-09-19T08:43:56.156584+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:07:50.043881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.155580+0000","last_change":"2024-09-19T08:43:54.166062+0000","last_active":"2024-09-19T08:43:56.155580+0000","last_peered":"2024-09-19T08:43:56.155580+0000","last_clean":"2024-09-19T08:43:56.155580+0000","last_became_active":"2024-09-19T08:43:54.165346+0000","last_became_peered":"2024-09-19T08:43:54.165346+0000","last_unstale":"2024-09-19T08:43:56.155580+0000","last_undegraded":"2024-09-19T08:43:56.155580+0000","last_fullsized":"2024-09-19T08:43:56.155580+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:40.360335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.165461+0000","last_change":"2024-09-19T08:43:55.190220+0000","last_active":"2024-09-19T08:43:56.165461+0000","last_peered":"2024-09-19T08:43:56.165461+0000","last_clean":"2024-09-19T08:43:56.165461+0000","last_became_active":"2024-09-19T08:43:55.188987+0000","last_became_peered":"2024-09-19T08:43:55.188987+0000","last_unstale":"2024-09-19T08:43:56.165461+0000","last_undegraded":"2024-09-19T08:43:56.165461+0000","last_fullsized":"2024-09-19T08:43:56.165461+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:06:59.363181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.737357+0000","last_change":"2024-09-19T08:43:54.157928+0000","last_active":"2024-09-19T08:43:56.737357+0000","last_peered":"2024-09-19T08:43:56.737357+0000","last_clean":"2024-09-19T08:43:56.737357+0000","last_became_active":"2024-09-19T08:43:54.157571+0000","last_became_peered":"2024-09-19T08:43:54.157571+0000","last_unstale":"2024-09-19T08:43:56.737357+0000","last_undegraded":"2024-09-19T08:43:56.737357+0000","last_fullsized":"2024-09-19T08:43:56.737357+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:31.692173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.161089+0000","last_change":"2024-09-19T08:43:55.205294+0000","last_active":"2024-09-19T08:43:56.161089+0000","last_peered":"2024-09-19T08:43:56.161089+0000","last_clean":"2024-09-19T08:43:56.161089+0000","last_became_active":"2024-09-19T08:43:55.204683+0000","last_became_peered":"2024-09-19T08:43:55.204683+0000","last_unstale":"2024-09-19T08:43:56.161089+0000","last_undegraded":"2024-09-19T08:43:56.161089+0000","last_fullsized":"2024-09-19T08:43:56.161089+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:39:02.010421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.169697+0000","last_change":"2024-09-19T08:43:54.174126+0000","last_active":"2024-09-19T08:43:56.169697+0000","last_peered":"2024-09-19T08:43:56.169697+0000","last_clean":"2024-09-19T08:43:56.169697+0000","last_became_active":"2024-09-19T08:43:54.173403+0000","last_became_peered":"2024-09-19T08:43:54.173403+0000","last_unstale":"2024-09-19T08:43:56.169697+0000","last_undegraded":"2024-09-19T08:43:56.169697+0000","last_fullsized":"2024-09-19T08:43:56.169697+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:29:30.346662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.160539+0000","last_change":"2024-09-19T08:43:55.204555+0000","last_active":"2024-09-19T08:43:56.160539+0000","last_peered":"2024-09-19T08:43:56.160539+0000","last_clean":"2024-09-19T08:43:56.160539+0000","last_became_active":"2024-09-19T08:43:55.201572+0000","last_became_peered":"2024-09-19T08:43:55.201572+0000","last_unstale":"2024-09-19T08:43:56.160539+0000","last_undegraded":"2024-09-19T08:43:56.160539+0000","last_fullsized":"2024-09-19T08:43:56.160539+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:23.679385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.160261+0000","last_change":"2024-09-19T08:43:54.161229+0000","last_active":"2024-09-19T08:43:56.160261+0000","last_peered":"2024-09-19T08:43:56.160261+0000","last_clean":"2024-09-19T08:43:56.160261+0000","last_became_active":"2024-09-19T08:43:54.146726+0000","last_became_peered":"2024-09-19T08:43:54.146726+0000","last_unstale":"2024-09-19T08:43:56.160261+0000","last_undegraded":"2024-09-19T08:43:56.160261+0000","last_fullsized":"2024-09-19T08:43:56.160261+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:27.079573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736911+0000","last_change":"2024-09-19T08:43:55.180738+0000","last_active":"2024-09-19T08:43:56.736911+0000","last_peered":"2024-09-19T08:43:56.736911+0000","last_clean":"2024-09-19T08:43:56.736911+0000","last_became_active":"2024-09-19T08:43:55.179169+0000","last_became_peered":"2024-09-19T08:43:55.179169+0000","last_unstale":"2024-09-19T08:43:56.736911+0000","last_undegraded":"2024-09-19T08:43:56.736911+0000","last_fullsized":"2024-09-19T08:43:56.736911+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:23.679385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.735440+0000","last_change":"2024-09-19T08:43:54.148684+0000","last_active":"2024-09-19T08:43:56.735440+0000","last_peered":"2024-09-19T08:43:56.735440+0000","last_clean":"2024-09-19T08:43:56.735440+0000","last_became_active":"2024-09-19T08:43:54.148479+0000","last_became_peered":"2024-09-19T08:43:54.148479+0000","last_unstale":"2024-09-19T08:43:56.735440+0000","last_undegraded":"2024-09-19T08:43:56.735440+0000","last_fullsized":"2024-09-19T08:43:56.735440+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:30.763525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.155474+0000","last_change":"2024-09-19T08:43:55.181103+0000","last_active":"2024-09-19T08:43:56.155474+0000","last_peered":"2024-09-19T08:43:56.155474+0000","last_clean":"2024-09-19T08:43:56.155474+0000","last_became_active":"2024-09-19T08:43:55.180719+0000","last_became_peered":"2024-09-19T08:43:55.180719+0000","last_unstale":"2024-09-19T08:43:56.155474+0000","last_undegraded":"2024-09-19T08:43:56.155474+0000","last_fullsized":"2024-09-19T08:43:56.155474+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:23.898525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.156061+0000","last_change":"2024-09-19T08:43:54.163982+0000","last_active":"2024-09-19T08:43:56.156061+0000","last_peered":"2024-09-19T08:43:56.156061+0000","last_clean":"2024-09-19T08:43:56.156061+0000","last_became_active":"2024-09-19T08:43:54.163705+0000","last_became_peered":"2024-09-19T08:43:54.163705+0000","last_unstale":"2024-09-19T08:43:56.156061+0000","last_undegraded":"2024-09-19T08:43:56.156061+0000","last_fullsized":"2024-09-19T08:43:56.156061+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:39:47.549316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.735072+0000","last_change":"2024-09-19T08:43:55.192869+0000","last_active":"2024-09-19T08:43:56.735072+0000","last_peered":"2024-09-19T08:43:56.735072+0000","last_clean":"2024-09-19T08:43:56.735072+0000","last_became_active":"2024-09-19T08:43:55.192580+0000","last_became_peered":"2024-09-19T08:43:55.192580+0000","last_unstale":"2024-09-19T08:43:56.735072+0000","last_undegraded":"2024-09-19T08:43:56.735072+0000","last_fullsized":"2024-09-19T08:43:56.735072+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:54.129671+0000","last_clean_scrub_stamp":"2024-09-19T08:43:54.129671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:09.148051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.169923+0000","last_change":"2024-09-19T08:43:54.163349+0000","last_active":"2024-09-19T08:43:56.169923+0000","last_peered":"2024-09-19T08:43:56.169923+0000","last_clean":"2024-09-19T08:43:56.169923+0000","last_became_active":"2024-09-19T08:43:54.162600+0000","last_became_peered":"2024-09-19T08:43:54.162600+0000","last_unstale":"2024-09-19T08:43:56.169923+0000","last_undegraded":"2024-09-19T08:43:56.169923+0000","last_fullsized":"2024-09-19T08:43:56.169923+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:43:53.120145+0000","last_clean_scrub_stamp":"2024-09-19T08:43:53.120145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:54:27.116255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"71'128","reported_seq":355,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-19T08:43:56.736808+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:43:56.736808+0000","last_peered":"2024-09-19T08:43:56.736808+0000","last_clean":"2024-09-19T08:43:56.736808+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:43:56.736808+0000","last_undegraded":"2024-09-19T08:43:56.736808+0000","last_fullsized":"2024-09-19T08:43:56.736808+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509513,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33012,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338828,"statfs":{"total":96636764160,"available":96602959872,"internally_reserved":0,"allocated":1863680,"data_stored":1145603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574922,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1265664,"data_stored":554689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509515,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1269760,"data_stored":555223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509515,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32420,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1261568,"data_stored":554619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542217,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1277952,"data_stored":556301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509512,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32420,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1261568,"data_stored":554619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509514,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1269760,"data_stored":554743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509512,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33024,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338816,"statfs":{"total":96636764160,"available":96602947584,"internally_reserved":0,"allocated":1875968,"data_stored":1146711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:44:03.012 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T08:44:03.013 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-19T08:44:03.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.184+0000 7f24502d5640 1 Processor -- start 2024-09-19T08:44:03.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.184+0000 7f24502d5640 1 -- start start 2024-09-19T08:44:03.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.184+0000 7f24502d5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f244806f9f0 0x7f24480b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:03.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.184+0000 7f24502d5640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f244815d760 con 0x7f244815c9f0 2024-09-19T08:44:03.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.184+0000 7f24502d5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24480b5410 con 0x7f2448159f60 2024-09-19T08:44:03.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.184+0000 7f24502d5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f244815a340 con 0x7f244806f9f0 2024-09-19T08:44:03.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.188+0000 7f244e872640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f244815c9f0 0x7f244815ce80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:33190/0 (socket says 172.21.15.98:33190) 2024-09-19T08:44:03.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.188+0000 7f244e872640 1 -- 172.21.15.98:0/896765417 learned_addr learned my addr 172.21.15.98:0/896765417 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:03.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.188+0000 7f244d870640 1 --2- 172.21.15.98:0/896765417 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f244806f9f0 0x7f24480b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:03.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.188+0000 7f244d06f640 1 -- 172.21.15.98:0/896765417 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4158523047 0 0) 0x7f244815d760 con 0x7f244815c9f0 2024-09-19T08:44:03.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.188+0000 7f244d06f640 1 -- 172.21.15.98:0/896765417 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2434003530 con 0x7f244815c9f0 2024-09-19T08:44:03.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.188+0000 7f244d06f640 1 -- 172.21.15.98:0/896765417 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1832497994 0 0) 0x7f2434003530 con 0x7f244815c9f0 2024-09-19T08:44:03.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.188+0000 7f244d06f640 1 -- 172.21.15.98:0/896765417 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f244806f9f0 msgr2=0x7f24480b61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:03.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.188+0000 7f244d06f640 1 --2- 172.21.15.98:0/896765417 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f244806f9f0 0x7f24480b61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:03.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.188+0000 7f244d06f640 1 -- 172.21.15.98:0/896765417 >> v1:172.21.15.139:6789/0 conn(0x7f2448159f60 legacy=0x7f244806f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:03.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.188+0000 7f244d06f640 1 -- 172.21.15.98:0/896765417 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24480b56f0 con 0x7f244815c9f0 2024-09-19T08:44:03.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.188+0000 7f244d06f640 1 -- 172.21.15.98:0/896765417 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f2440001680 con 0x7f244815c9f0 2024-09-19T08:44:03.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.188+0000 7f244d06f640 1 -- 172.21.15.98:0/896765417 >> v1:172.21.15.98:6789/0 conn(0x7f244815c9f0 legacy=0x7f244815ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:03.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.188+0000 7f244d06f640 1 --2- 172.21.15.98:0/896765417 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f244806f9f0 0x7f24340039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:03.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.188+0000 7f244d06f640 1 --2- 172.21.15.98:0/896765417 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2434003f70 0x7f2434004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:03.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.188+0000 7f244d06f640 1 --2- 172.21.15.98:0/896765417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2434004d50 0x7f24340051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:03.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.188+0000 7f244d06f640 1 -- 172.21.15.98:0/896765417 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f244815a340 con 0x7f2434003f70 2024-09-19T08:44:03.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.188+0000 7f244d06f640 1 -- 172.21.15.98:0/896765417 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f243400bab0 con 0x7f2434004d50 2024-09-19T08:44:03.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.188+0000 7f244d06f640 1 -- 172.21.15.98:0/896765417 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f243400bcb0 con 0x7f244806f9f0 2024-09-19T08:44:03.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.188+0000 7f244d870640 1 --2- 172.21.15.98:0/896765417 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2434003f70 0x7f2434004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:03.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.188+0000 7f244e071640 1 --2- 172.21.15.98:0/896765417 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f244806f9f0 0x7f24340039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:03.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.188+0000 7f244e872640 1 --2- 172.21.15.98:0/896765417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2434004d50 0x7f24340051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:03.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.188+0000 7f244d870640 1 -- 172.21.15.98:0/896765417 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f244806f9f0 msgr2=0x7f24340039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:03.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.188+0000 7f244d870640 1 --2- 172.21.15.98:0/896765417 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f244806f9f0 0x7f24340039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:03.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.188+0000 7f244d870640 1 -- 172.21.15.98:0/896765417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2434004d50 msgr2=0x7f24340051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:03.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.188+0000 7f244d870640 1 --2- 172.21.15.98:0/896765417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2434004d50 0x7f24340051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:03.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.188+0000 7f244d870640 1 -- 172.21.15.98:0/896765417 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2434003530 con 0x7f2434003f70 2024-09-19T08:44:03.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.188+0000 7f244e872640 1 --2- 172.21.15.98:0/896765417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2434004d50 0x7f24340051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:03.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.188+0000 7f244d870640 1 --2- 172.21.15.98:0/896765417 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2434003f70 0x7f2434004810 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f243c009ae0 tx=0x7f243c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0c1e2a8ef138ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:03.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.192+0000 7f244d06f640 1 -- 172.21.15.98:0/896765417 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243c0b2070 con 0x7f2434003f70 2024-09-19T08:44:03.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.192+0000 7f244d06f640 1 -- 172.21.15.98:0/896765417 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f243c0a5660 con 0x7f2434003f70 2024-09-19T08:44:03.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.192+0000 7f244d06f640 1 -- 172.21.15.98:0/896765417 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243c0096b0 con 0x7f2434003f70 2024-09-19T08:44:03.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.192+0000 7f24502d5640 1 -- 172.21.15.98:0/896765417 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2434003f70 msgr2=0x7f2434004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:03.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.192+0000 7f24502d5640 1 --2- 172.21.15.98:0/896765417 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2434003f70 0x7f2434004810 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f243c009ae0 tx=0x7f243c01ca70 comp rx=0 tx=0).stop 2024-09-19T08:44:03.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.192+0000 7f244e872640 1 -- 172.21.15.98:0/896765417 reap_dead start 2024-09-19T08:44:03.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.192+0000 7f24502d5640 1 -- 172.21.15.98:0/896765417 shutdown_connections 2024-09-19T08:44:03.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.192+0000 7f24502d5640 1 -- 172.21.15.98:0/896765417 >> 172.21.15.98:0/896765417 conn(0x7f244806dea0 msgr2=0x7f244806e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:03.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.192+0000 7f24502d5640 1 -- 172.21.15.98:0/896765417 shutdown_connections 2024-09-19T08:44:03.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.192+0000 7f24502d5640 1 -- 172.21.15.98:0/896765417 wait complete. 2024-09-19T08:44:03.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.192+0000 7f24502d5640 1 Processor -- start 2024-09-19T08:44:03.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.192+0000 7f24502d5640 1 -- start start 2024-09-19T08:44:03.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.192+0000 7f24502d5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2434003f70 0x7f24481556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:03.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.192+0000 7f24502d5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24481ec2d0 0x7f2448156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:03.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.192+0000 7f24502d5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24481f57d0 0x7f24481f5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:03.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.192+0000 7f24502d5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f24480b61d0 con 0x7f2434003f70 2024-09-19T08:44:03.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.192+0000 7f24502d5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f24480b5cb0 con 0x7f24481ec2d0 2024-09-19T08:44:03.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.192+0000 7f24502d5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f24480b6350 con 0x7f24481f57d0 2024-09-19T08:44:03.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.192+0000 7f244e071640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2434003f70 0x7f24481556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:03.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.192+0000 7f244e071640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2434003f70 0x7f24481556a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:50806/0 (socket says 172.21.15.98:50806) 2024-09-19T08:44:03.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.192+0000 7f244e071640 1 -- 172.21.15.98:0/2515818691 learned_addr learned my addr 172.21.15.98:0/2515818691 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:03.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.192+0000 7f244d870640 1 --2- 172.21.15.98:0/2515818691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24481ec2d0 0x7f2448156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:03.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.192+0000 7f244e872640 1 --2- 172.21.15.98:0/2515818691 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24481f57d0 0x7f24481f5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:03.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.192+0000 7f244e071640 1 -- 172.21.15.98:0/2515818691 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24481f57d0 msgr2=0x7f24481f5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:03.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.192+0000 7f244e071640 1 --2- 172.21.15.98:0/2515818691 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24481f57d0 0x7f24481f5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:03.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.192+0000 7f244e071640 1 -- 172.21.15.98:0/2515818691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24481ec2d0 msgr2=0x7f2448156f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:03.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.192+0000 7f244e071640 1 --2- 172.21.15.98:0/2515818691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24481ec2d0 0x7f2448156f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:03.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.192+0000 7f244e071640 1 -- 172.21.15.98:0/2515818691 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2448155c90 con 0x7f2434003f70 2024-09-19T08:44:03.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.192+0000 7f244e071640 1 --2- 172.21.15.98:0/2515818691 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2434003f70 0x7f24481556a0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f244400e900 tx=0x7f244400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:03.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.192+0000 7f243affd640 1 -- 172.21.15.98:0/2515818691 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2444003b90 con 0x7f2434003f70 2024-09-19T08:44:03.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.192+0000 7f243affd640 1 -- 172.21.15.98:0/2515818691 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2444018070 con 0x7f2434003f70 2024-09-19T08:44:03.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.192+0000 7f24502d5640 1 -- 172.21.15.98:0/2515818691 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2448155f70 con 0x7f2434003f70 2024-09-19T08:44:03.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.192+0000 7f243affd640 1 -- 172.21.15.98:0/2515818691 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24440050d0 con 0x7f2434003f70 2024-09-19T08:44:03.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.192+0000 7f24502d5640 1 -- 172.21.15.98:0/2515818691 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2448156530 con 0x7f2434003f70 2024-09-19T08:44:03.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.196+0000 7f24502d5640 1 -- 172.21.15.98:0/2515818691 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24481ec6b0 con 0x7f2434003f70 2024-09-19T08:44:03.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.196+0000 7f243affd640 1 -- 172.21.15.98:0/2515818691 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f2444005270 con 0x7f2434003f70 2024-09-19T08:44:03.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.196+0000 7f243affd640 1 --2- 172.21.15.98:0/2515818691 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f242c0679e0 0x7f242c069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:03.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.196+0000 7f243affd640 1 -- 172.21.15.98:0/2515818691 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f2444085dd0 con 0x7f2434003f70 2024-09-19T08:44:03.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.196+0000 7f244d870640 1 --2- 172.21.15.98:0/2515818691 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f242c0679e0 0x7f242c069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:03.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.196+0000 7f244d870640 1 --2- 172.21.15.98:0/2515818691 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f242c0679e0 0x7f242c069ea0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f243c0af000 tx=0x7f243c0046d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:03.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.200+0000 7f243affd640 1 -- 172.21.15.98:0/2515818691 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2444056890 con 0x7f2434003f70 2024-09-19T08:44:03.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.416+0000 7f24502d5640 1 -- 172.21.15.98:0/2515818691 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f244806f2e0 con 0x7f2434003f70 2024-09-19T08:44:03.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.420+0000 7f243affd640 1 -- 172.21.15.98:0/2515818691 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) ==== 190+0+147 (secure 0 0 0) 0x7f24440870b0 con 0x7f2434003f70 2024-09-19T08:44:03.424 INFO:teuthology.orchestra.run.smithi098.stdout:[client.0] 2024-09-19T08:44:03.424 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQC/4utm0vNpIhAA4S1E8dFZ0+ibmSKhmZ+bIA== 2024-09-19T08:44:03.424 INFO:teuthology.orchestra.run.smithi098.stdout: caps mds = "allow" 2024-09-19T08:44:03.424 INFO:teuthology.orchestra.run.smithi098.stdout: caps mon = "allow r" 2024-09-19T08:44:03.424 INFO:teuthology.orchestra.run.smithi098.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-19T08:44:03.424 INFO:teuthology.orchestra.run.smithi098.stderr:updated caps for client.0 2024-09-19T08:44:03.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.424+0000 7f24502d5640 1 -- 172.21.15.98:0/2515818691 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f242c0679e0 msgr2=0x7f242c069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:03.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.424+0000 7f24502d5640 1 --2- 172.21.15.98:0/2515818691 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f242c0679e0 0x7f242c069ea0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f243c0af000 tx=0x7f243c0046d0 comp rx=0 tx=0).stop 2024-09-19T08:44:03.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.424+0000 7f24502d5640 1 -- 172.21.15.98:0/2515818691 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2434003f70 msgr2=0x7f24481556a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:03.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.424+0000 7f24502d5640 1 --2- 172.21.15.98:0/2515818691 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2434003f70 0x7f24481556a0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f244400e900 tx=0x7f244400edc0 comp rx=0 tx=0).stop 2024-09-19T08:44:03.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.424+0000 7f24502d5640 1 -- 172.21.15.98:0/2515818691 shutdown_connections 2024-09-19T08:44:03.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.424+0000 7f24502d5640 1 --2- 172.21.15.98:0/2515818691 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f242c0679e0 0x7f242c069ea0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:03.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.424+0000 7f24502d5640 1 --2- 172.21.15.98:0/2515818691 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f24481f57d0 0x7f24481f5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:03.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.424+0000 7f24502d5640 1 --2- 172.21.15.98:0/2515818691 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f24481ec2d0 0x7f2448156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:03.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.424+0000 7f24502d5640 1 --2- 172.21.15.98:0/2515818691 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2434003f70 0x7f24481556a0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:03.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.424+0000 7f24502d5640 1 -- 172.21.15.98:0/2515818691 >> 172.21.15.98:0/2515818691 conn(0x7f244806dea0 msgr2=0x7f2448073410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:03.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.424+0000 7f24502d5640 1 -- 172.21.15.98:0/2515818691 shutdown_connections 2024-09-19T08:44:03.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.424+0000 7f24502d5640 1 -- 172.21.15.98:0/2515818691 wait complete. 2024-09-19T08:44:03.441 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:44:03.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.556+0000 7fcf57364640 1 Processor -- start 2024-09-19T08:44:03.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.556+0000 7fcf57364640 1 -- start start 2024-09-19T08:44:03.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.556+0000 7fcf57364640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcf50158fd0 0x7fcf500b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:03.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.556+0000 7fcf57364640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcf5006c3a0 0x7fcf500ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:03.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.556+0000 7fcf57364640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf500ccc60 con 0x7fcf500b4360 2024-09-19T08:44:03.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.556+0000 7fcf57364640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fcf501593b0 con 0x7fcf5006c3a0 2024-09-19T08:44:03.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.556+0000 7fcf57364640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fcf500ccf40 con 0x7fcf50158fd0 2024-09-19T08:44:03.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.560+0000 7fcf55901640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcf5006c3a0 0x7fcf500ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:03.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.560+0000 7fcf55901640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcf5006c3a0 0x7fcf500ce840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:50816/0 (socket says 172.21.15.98:50816) 2024-09-19T08:44:03.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.560+0000 7fcf548ff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fcf500b4360 0x7fcf5006bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:33196/0 (socket says 172.21.15.98:33196) 2024-09-19T08:44:03.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.560+0000 7fcf55901640 1 -- 172.21.15.98:0/824840747 learned_addr learned my addr 172.21.15.98:0/824840747 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:03.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.560+0000 7fcf55100640 1 --2- 172.21.15.98:0/824840747 >> v2:172.21.15.139:3300/0 conn(0x7fcf50158fd0 0x7fcf500b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:03.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.560+0000 7fcf55901640 1 -- 172.21.15.98:0/824840747 >> v2:172.21.15.139:3300/0 conn(0x7fcf50158fd0 msgr2=0x7fcf500b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:03.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.560+0000 7fcf55901640 1 --2- 172.21.15.98:0/824840747 >> v2:172.21.15.139:3300/0 conn(0x7fcf50158fd0 0x7fcf500b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:03.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.560+0000 7fcf55901640 1 -- 172.21.15.98:0/824840747 >> v1:172.21.15.98:6789/0 conn(0x7fcf500b4360 legacy=0x7fcf5006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:03.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.560+0000 7fcf47fff640 1 -- 172.21.15.98:0/824840747 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1744047939 0 0) 0x7fcf500ccc60 con 0x7fcf500b4360 2024-09-19T08:44:03.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.560+0000 7fcf55901640 1 -- 172.21.15.98:0/824840747 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf500cd220 con 0x7fcf5006c3a0 2024-09-19T08:44:03.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.560+0000 7fcf55100640 1 --2- 172.21.15.98:0/824840747 >> v2:172.21.15.139:3300/0 conn(0x7fcf50158fd0 0x7fcf500b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:03.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.560+0000 7fcf55901640 1 --2- 172.21.15.98:0/824840747 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf5006c3a0 0x7fcf500ce840 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fcf5006e310 tx=0x7fcf4801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=308057ee3218e9d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:03.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.560+0000 7fcf47fff640 1 -- 172.21.15.98:0/824840747 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf480a57f0 con 0x7fcf5006c3a0 2024-09-19T08:44:03.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.560+0000 7fcf47fff640 1 -- 172.21.15.98:0/824840747 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf480a5990 con 0x7fcf5006c3a0 2024-09-19T08:44:03.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.560+0000 7fcf47fff640 1 -- 172.21.15.98:0/824840747 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf480a5d60 con 0x7fcf5006c3a0 2024-09-19T08:44:03.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.560+0000 7fcf57364640 1 -- 172.21.15.98:0/824840747 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf5006c3a0 msgr2=0x7fcf500ce840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:03.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.560+0000 7fcf57364640 1 --2- 172.21.15.98:0/824840747 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf5006c3a0 0x7fcf500ce840 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fcf5006e310 tx=0x7fcf4801c930 comp rx=0 tx=0).stop 2024-09-19T08:44:03.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.560+0000 7fcf57364640 1 -- 172.21.15.98:0/824840747 shutdown_connections 2024-09-19T08:44:03.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.560+0000 7fcf57364640 1 --2- 172.21.15.98:0/824840747 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf5006c3a0 0x7fcf500ce840 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:03.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.560+0000 7fcf57364640 1 --2- 172.21.15.98:0/824840747 >> v2:172.21.15.139:3300/0 conn(0x7fcf50158fd0 0x7fcf500b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:03.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.560+0000 7fcf57364640 1 -- 172.21.15.98:0/824840747 >> 172.21.15.98:0/824840747 conn(0x7fcf500b2010 msgr2=0x7fcf500b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:03.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.560+0000 7fcf57364640 1 -- 172.21.15.98:0/824840747 shutdown_connections 2024-09-19T08:44:03.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.560+0000 7fcf57364640 1 -- 172.21.15.98:0/824840747 wait complete. 2024-09-19T08:44:03.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.564+0000 7fcf57364640 1 Processor -- start 2024-09-19T08:44:03.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.564+0000 7fcf57364640 1 -- start start 2024-09-19T08:44:03.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.564+0000 7fcf57364640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf5006c3a0 0x7fcf50154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:03.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.564+0000 7fcf57364640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcf500b4360 0x7fcf50151630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:03.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.564+0000 7fcf57364640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf50158fd0 0x7fcf50151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:03.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.564+0000 7fcf57364640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcf500cf710 con 0x7fcf5006c3a0 2024-09-19T08:44:03.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.564+0000 7fcf57364640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcf500cf590 con 0x7fcf50158fd0 2024-09-19T08:44:03.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.564+0000 7fcf57364640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcf500cf890 con 0x7fcf500b4360 2024-09-19T08:44:03.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.564+0000 7fcf55100640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf5006c3a0 0x7fcf50154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:03.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.564+0000 7fcf55100640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf5006c3a0 0x7fcf50154f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:50826/0 (socket says 172.21.15.98:50826) 2024-09-19T08:44:03.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.564+0000 7fcf55100640 1 -- 172.21.15.98:0/761947275 learned_addr learned my addr 172.21.15.98:0/761947275 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:03.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.564+0000 7fcf55100640 1 -- 172.21.15.98:0/761947275 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcf500b4360 msgr2=0x7fcf50151630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:03.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.564+0000 7fcf55100640 1 --2- 172.21.15.98:0/761947275 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcf500b4360 0x7fcf50151630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:03.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.564+0000 7fcf55901640 1 --2- 172.21.15.98:0/761947275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf50158fd0 0x7fcf50151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:03.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.564+0000 7fcf55100640 1 -- 172.21.15.98:0/761947275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf50158fd0 msgr2=0x7fcf50151b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:03.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.564+0000 7fcf55100640 1 --2- 172.21.15.98:0/761947275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf50158fd0 0x7fcf50151b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:03.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.564+0000 7fcf55100640 1 -- 172.21.15.98:0/761947275 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf50155480 con 0x7fcf5006c3a0 2024-09-19T08:44:03.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.564+0000 7fcf55100640 1 --2- 172.21.15.98:0/761947275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf5006c3a0 0x7fcf50154f40 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00c7f0 tx=0x7fcf4c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:03.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.564+0000 7fcf45ffb640 1 -- 172.21.15.98:0/761947275 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c015070 con 0x7fcf5006c3a0 2024-09-19T08:44:03.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.564+0000 7fcf45ffb640 1 -- 172.21.15.98:0/761947275 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf4c007820 con 0x7fcf5006c3a0 2024-09-19T08:44:03.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.564+0000 7fcf57364640 1 -- 172.21.15.98:0/761947275 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf50071d60 con 0x7fcf5006c3a0 2024-09-19T08:44:03.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.564+0000 7fcf45ffb640 1 -- 172.21.15.98:0/761947275 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c007c10 con 0x7fcf5006c3a0 2024-09-19T08:44:03.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.564+0000 7fcf57364640 1 -- 172.21.15.98:0/761947275 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf50072320 con 0x7fcf5006c3a0 2024-09-19T08:44:03.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.564+0000 7fcf57364640 1 -- 172.21.15.98:0/761947275 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf501556b0 con 0x7fcf5006c3a0 2024-09-19T08:44:03.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.564+0000 7fcf45ffb640 1 -- 172.21.15.98:0/761947275 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fcf4c018070 con 0x7fcf5006c3a0 2024-09-19T08:44:03.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.564+0000 7fcf45ffb640 1 --2- 172.21.15.98:0/761947275 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcf34067a10 0x7fcf34069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:03.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.568+0000 7fcf45ffb640 1 -- 172.21.15.98:0/761947275 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5767+0+0 (secure 0 0 0) 0x7fcf4c085fe0 con 0x7fcf5006c3a0 2024-09-19T08:44:03.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.568+0000 7fcf548ff640 1 --2- 172.21.15.98:0/761947275 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcf34067a10 0x7fcf34069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:03.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.568+0000 7fcf548ff640 1 --2- 172.21.15.98:0/761947275 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcf34067a10 0x7fcf34069ed0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcf400011f0 tx=0x7fcf40007930 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:03.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.572+0000 7fcf45ffb640 1 -- 172.21.15.98:0/761947275 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fcf50155480 con 0x7fcf5006c3a0 2024-09-19T08:44:03.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.752+0000 7fcf57364640 1 -- 172.21.15.98:0/761947275 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf500b35f0 con 0x7fcf5006c3a0 2024-09-19T08:44:03.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.752+0000 7fcf45ffb640 1 -- 172.21.15.98:0/761947275 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+5225 (secure 0 0 0) 0x7fcf4c056930 con 0x7fcf5006c3a0 2024-09-19T08:44:03.755 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:44:03.755 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":60,"btime":"2024-09-19T08:43:57:218267+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6004,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3739962222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3739962222},{"type":"v1","addr":"172.21.15.139:6833","nonce":3739962222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6007,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/421473653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":421473653},{"type":"v1","addr":"172.21.15.139:6835","nonce":421473653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3126941916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3126941916},{"type":"v1","addr":"172.21.15.98:6833","nonce":3126941916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:43:54.489407+0000","modified":"2024-09-19T08:43:56.577758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6996},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6996":{"gid":6996,"name":"c","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/273091170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":273091170},{"type":"v1","addr":"172.21.15.98:6835","nonce":273091170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6996,"qdb_cluster":[6996]},"id":8}]} 2024-09-19T08:44:03.756 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 60 2024-09-19T08:44:03.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.756+0000 7fcf57364640 1 -- 172.21.15.98:0/761947275 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcf34067a10 msgr2=0x7fcf34069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:03.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.756+0000 7fcf57364640 1 --2- 172.21.15.98:0/761947275 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcf34067a10 0x7fcf34069ed0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcf400011f0 tx=0x7fcf40007930 comp rx=0 tx=0).stop 2024-09-19T08:44:03.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.756+0000 7fcf57364640 1 -- 172.21.15.98:0/761947275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf5006c3a0 msgr2=0x7fcf50154f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:03.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.756+0000 7fcf57364640 1 --2- 172.21.15.98:0/761947275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf5006c3a0 0x7fcf50154f40 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00c7f0 tx=0x7fcf4c00ccb0 comp rx=0 tx=0).stop 2024-09-19T08:44:03.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.756+0000 7fcf57364640 1 -- 172.21.15.98:0/761947275 shutdown_connections 2024-09-19T08:44:03.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.756+0000 7fcf57364640 1 --2- 172.21.15.98:0/761947275 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcf34067a10 0x7fcf34069ed0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:03.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.756+0000 7fcf57364640 1 --2- 172.21.15.98:0/761947275 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf50158fd0 0x7fcf50151b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:03.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.756+0000 7fcf57364640 1 --2- 172.21.15.98:0/761947275 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcf500b4360 0x7fcf50151630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:03.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.756+0000 7fcf57364640 1 --2- 172.21.15.98:0/761947275 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf5006c3a0 0x7fcf50154f40 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:03.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.756+0000 7fcf57364640 1 -- 172.21.15.98:0/761947275 >> 172.21.15.98:0/761947275 conn(0x7fcf500b2010 msgr2=0x7fcf500cfc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:03.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.756+0000 7fcf57364640 1 -- 172.21.15.98:0/761947275 shutdown_connections 2024-09-19T08:44:03.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:03.756+0000 7fcf57364640 1 -- 172.21.15.98:0/761947275 wait complete. 2024-09-19T08:44:04.176 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 60, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T08:43:54.489407+0000', 'modified': '2024-09-19T08:43:56.577758+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6996}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6996': {'gid': 6996, 'name': 'c', 'rank': 0, 'incarnation': 57, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.98:6835/273091170', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6834', 'nonce': 273091170}, {'type': 'v1', 'addr': '172.21.15.98:6835', 'nonce': 273091170}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6996, 'qdb_cluster': [6996]} 2024-09-19T08:44:04.177 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T08:44:04.177 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-19T08:44:04.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.312+0000 7f3bdc409640 1 Processor -- start 2024-09-19T08:44:04.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.312+0000 7f3bdc409640 1 -- start start 2024-09-19T08:44:04.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.312+0000 7f3bdc409640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3bd415c550 0x7f3bd406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:04.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.312+0000 7f3bdc409640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3bd406c1d0 0x7f3bd40cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:04.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.312+0000 7f3bdc409640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3bd40cf900 0x7f3bd40ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:04.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.312+0000 7f3bdc409640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f3bd406c610 con 0x7f3bd415c550 2024-09-19T08:44:04.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.312+0000 7f3bdc409640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f3bd406d310 con 0x7f3bd40cf900 2024-09-19T08:44:04.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.312+0000 7f3bdc409640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3bd406d5f0 con 0x7f3bd406c1d0 2024-09-19T08:44:04.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.312+0000 7f3bda1a5640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3bd415c550 0x7f3bd406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:04.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.312+0000 7f3bda1a5640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3bd415c550 0x7f3bd406bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:50834/0 (socket says 172.21.15.98:50834) 2024-09-19T08:44:04.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.312+0000 7f3bda1a5640 1 -- 172.21.15.98:0/1306945568 learned_addr learned my addr 172.21.15.98:0/1306945568 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:04.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.312+0000 7f3bda9a6640 1 --2- 172.21.15.98:0/1306945568 >> v2:172.21.15.139:3300/0 conn(0x7f3bd40cf900 0x7f3bd40ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:04.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.312+0000 7f3bd99a4640 1 --2- 172.21.15.98:0/1306945568 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3bd406c1d0 0x7f3bd40cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:04.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.312+0000 7f3bda1a5640 1 -- 172.21.15.98:0/1306945568 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3bd406c1d0 msgr2=0x7f3bd40cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:04.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.312+0000 7f3bda1a5640 1 --2- 172.21.15.98:0/1306945568 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3bd406c1d0 0x7f3bd40cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.312+0000 7f3bda1a5640 1 -- 172.21.15.98:0/1306945568 >> v2:172.21.15.139:3300/0 conn(0x7f3bd40cf900 msgr2=0x7f3bd40ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:04.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.312+0000 7f3bda1a5640 1 --2- 172.21.15.98:0/1306945568 >> v2:172.21.15.139:3300/0 conn(0x7f3bd40cf900 0x7f3bd40ce1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.312+0000 7f3bda1a5640 1 -- 172.21.15.98:0/1306945568 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bd406d8d0 con 0x7f3bd415c550 2024-09-19T08:44:04.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.312+0000 7f3bda9a6640 1 --2- 172.21.15.98:0/1306945568 >> v2:172.21.15.139:3300/0 conn(0x7f3bd40cf900 0x7f3bd40ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:04.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.312+0000 7f3bd99a4640 1 --2- 172.21.15.98:0/1306945568 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3bd406c1d0 0x7f3bd40cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:04.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.316+0000 7f3bda1a5640 1 --2- 172.21.15.98:0/1306945568 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bd415c550 0x7f3bd406bc90 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4074090 tx=0x7f3bd001c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e645a15d31b6fa38 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:04.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.316+0000 7f3bd91a3640 1 -- 172.21.15.98:0/1306945568 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd00b0070 con 0x7f3bd415c550 2024-09-19T08:44:04.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.316+0000 7f3bd91a3640 1 -- 172.21.15.98:0/1306945568 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bd00ab600 con 0x7f3bd415c550 2024-09-19T08:44:04.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.316+0000 7f3bd91a3640 1 -- 172.21.15.98:0/1306945568 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd00ab9d0 con 0x7f3bd415c550 2024-09-19T08:44:04.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.316+0000 7f3bdc409640 1 -- 172.21.15.98:0/1306945568 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bd415c550 msgr2=0x7f3bd406bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:04.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.316+0000 7f3bdc409640 1 --2- 172.21.15.98:0/1306945568 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bd415c550 0x7f3bd406bc90 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4074090 tx=0x7f3bd001c260 comp rx=0 tx=0).stop 2024-09-19T08:44:04.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.316+0000 7f3bdc409640 1 -- 172.21.15.98:0/1306945568 shutdown_connections 2024-09-19T08:44:04.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.316+0000 7f3bdc409640 1 --2- 172.21.15.98:0/1306945568 >> v2:172.21.15.139:3300/0 conn(0x7f3bd40cf900 0x7f3bd40ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.316+0000 7f3bdc409640 1 --2- 172.21.15.98:0/1306945568 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3bd406c1d0 0x7f3bd40cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.316+0000 7f3bdc409640 1 --2- 172.21.15.98:0/1306945568 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bd415c550 0x7f3bd406bc90 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.316+0000 7f3bdc409640 1 -- 172.21.15.98:0/1306945568 >> 172.21.15.98:0/1306945568 conn(0x7f3bd40b2010 msgr2=0x7f3bd40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:04.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.316+0000 7f3bdc409640 1 -- 172.21.15.98:0/1306945568 shutdown_connections 2024-09-19T08:44:04.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.316+0000 7f3bdc409640 1 -- 172.21.15.98:0/1306945568 wait complete. 2024-09-19T08:44:04.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.316+0000 7f3bdc409640 1 Processor -- start 2024-09-19T08:44:04.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.316+0000 7f3bdc409640 1 -- start start 2024-09-19T08:44:04.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.316+0000 7f3bdc409640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bd406c1d0 0x7f3bd414f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:04.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.316+0000 7f3bdc409640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bd40cf900 0x7f3bd414da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:04.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.316+0000 7f3bdc409640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3bd415c550 0x7f3bd4156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:04.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.316+0000 7f3bdc409640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd40723a0 con 0x7f3bd406c1d0 2024-09-19T08:44:04.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.316+0000 7f3bdc409640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd415cb40 con 0x7f3bd40cf900 2024-09-19T08:44:04.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.316+0000 7f3bdc409640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3bd4072520 con 0x7f3bd415c550 2024-09-19T08:44:04.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.316+0000 7f3bda1a5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bd406c1d0 0x7f3bd414f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:04.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.316+0000 7f3bda1a5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bd406c1d0 0x7f3bd414f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:50840/0 (socket says 172.21.15.98:50840) 2024-09-19T08:44:04.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.316+0000 7f3bda1a5640 1 -- 172.21.15.98:0/4115723755 learned_addr learned my addr 172.21.15.98:0/4115723755 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:04.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.320+0000 7f3bd99a4640 1 --2- 172.21.15.98:0/4115723755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bd40cf900 0x7f3bd414da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:04.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.320+0000 7f3bda9a6640 1 --2- 172.21.15.98:0/4115723755 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3bd415c550 0x7f3bd4156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:04.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.320+0000 7f3bda1a5640 1 -- 172.21.15.98:0/4115723755 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3bd415c550 msgr2=0x7f3bd4156cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:04.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.320+0000 7f3bda1a5640 1 --2- 172.21.15.98:0/4115723755 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3bd415c550 0x7f3bd4156cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.320+0000 7f3bda1a5640 1 -- 172.21.15.98:0/4115723755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bd40cf900 msgr2=0x7f3bd414da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:04.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.320+0000 7f3bda1a5640 1 --2- 172.21.15.98:0/4115723755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bd40cf900 0x7f3bd414da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.320+0000 7f3bda1a5640 1 -- 172.21.15.98:0/4115723755 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bd414df60 con 0x7f3bd406c1d0 2024-09-19T08:44:04.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.320+0000 7f3bd99a4640 1 --2- 172.21.15.98:0/4115723755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bd40cf900 0x7f3bd414da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:04.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.320+0000 7f3bda1a5640 1 --2- 172.21.15.98:0/4115723755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bd406c1d0 0x7f3bd414f930 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f3bd00ae040 tx=0x7f3bd0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:04.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.320+0000 7f3bcaffd640 1 -- 172.21.15.98:0/4115723755 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd00b0040 con 0x7f3bd406c1d0 2024-09-19T08:44:04.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.320+0000 7f3bcaffd640 1 -- 172.21.15.98:0/4115723755 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bd00b9070 con 0x7f3bd406c1d0 2024-09-19T08:44:04.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.320+0000 7f3bdc409640 1 -- 172.21.15.98:0/4115723755 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bd4157b90 con 0x7f3bd406c1d0 2024-09-19T08:44:04.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.320+0000 7f3bcaffd640 1 -- 172.21.15.98:0/4115723755 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd00b48a0 con 0x7f3bd406c1d0 2024-09-19T08:44:04.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.320+0000 7f3bdc409640 1 -- 172.21.15.98:0/4115723755 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bd4158150 con 0x7f3bd406c1d0 2024-09-19T08:44:04.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.320+0000 7f3bcaffd640 1 -- 172.21.15.98:0/4115723755 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f3bd00ab4c0 con 0x7f3bd406c1d0 2024-09-19T08:44:04.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.320+0000 7f3bcaffd640 1 --2- 172.21.15.98:0/4115723755 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3bb80679e0 0x7f3bb8069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:04.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.320+0000 7f3bcaffd640 1 -- 172.21.15.98:0/4115723755 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f3bd011fc90 con 0x7f3bd406c1d0 2024-09-19T08:44:04.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.320+0000 7f3bd99a4640 1 --2- 172.21.15.98:0/4115723755 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3bb80679e0 0x7f3bb8069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:04.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.320+0000 7f3bdf4c5000 1 Processor -- start 2024-09-19T08:44:04.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.320+0000 7f3bdf4c5000 1 -- start start 2024-09-19T08:44:04.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.320+0000 7f3bd99a4640 1 --2- 172.21.15.98:0/4115723755 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3bb80679e0 0x7f3bb8069ea0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4006fd0 tx=0x7f3bc4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:04.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bdf4c5000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561de257ab20 0x561de257af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:04.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bdf4c5000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561de257b440 0x561de2530d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:04.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bdf4c5000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561de2531290 0x561de2531670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:04.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bdf4c5000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561de2418b60 con 0x561de2531290 2024-09-19T08:44:04.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bdf4c5000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561de237fc70 con 0x561de257b440 2024-09-19T08:44:04.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bdf4c5000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x561de2325ed0 con 0x561de257ab20 2024-09-19T08:44:04.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bda9a6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561de2531290 0x561de2531670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:04.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bda9a6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561de2531290 0x561de2531670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:50842/0 (socket says 172.21.15.98:50842) 2024-09-19T08:44:04.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bda9a6640 1 -- 172.21.15.98:0/1973050813 learned_addr learned my addr 172.21.15.98:0/1973050813 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:04.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bda9a6640 1 -- 172.21.15.98:0/1973050813 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561de257ab20 msgr2=0x561de257af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:04.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bda9a6640 1 --2- 172.21.15.98:0/1973050813 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561de257ab20 0x561de257af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bda9a6640 1 -- 172.21.15.98:0/1973050813 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561de257b440 msgr2=0x561de2530d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:04.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bd99a4640 1 --2- 172.21.15.98:0/1973050813 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561de257b440 0x561de2530d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:04.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bda9a6640 1 --2- 172.21.15.98:0/1973050813 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561de257b440 0x561de2530d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bda9a6640 1 -- 172.21.15.98:0/1973050813 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561de2337e50 con 0x561de2531290 2024-09-19T08:44:04.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bda9a6640 1 --2- 172.21.15.98:0/1973050813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561de2531290 0x561de2531670 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc008fd0 tx=0x7f3bcc00d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da061314218dd43 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:04.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bb7fff640 1 -- 172.21.15.98:0/1973050813 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc01a070 con 0x561de2531290 2024-09-19T08:44:04.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bb7fff640 1 -- 172.21.15.98:0/1973050813 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bcc00b6e0 con 0x561de2531290 2024-09-19T08:44:04.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bb7fff640 1 -- 172.21.15.98:0/1973050813 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc00bab0 con 0x561de2531290 2024-09-19T08:44:04.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bdf4c5000 1 -- 172.21.15.98:0/1973050813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561de2531290 msgr2=0x561de2531670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:04.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bdf4c5000 1 --2- 172.21.15.98:0/1973050813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561de2531290 0x561de2531670 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc008fd0 tx=0x7f3bcc00d8c0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bdf4c5000 1 -- 172.21.15.98:0/1973050813 shutdown_connections 2024-09-19T08:44:04.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bdf4c5000 1 --2- 172.21.15.98:0/1973050813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561de2531290 0x561de2531670 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bdf4c5000 1 --2- 172.21.15.98:0/1973050813 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561de257b440 0x561de2530d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bdf4c5000 1 --2- 172.21.15.98:0/1973050813 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561de257ab20 0x561de257af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bdf4c5000 1 -- 172.21.15.98:0/1973050813 >> 172.21.15.98:0/1973050813 conn(0x561de2537600 msgr2=0x561de252f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:04.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bdf4c5000 1 -- 172.21.15.98:0/1973050813 shutdown_connections 2024-09-19T08:44:04.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bdf4c5000 1 -- 172.21.15.98:0/1973050813 wait complete. 2024-09-19T08:44:04.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bdf4c5000 1 -- start start 2024-09-19T08:44:04.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bdf4c5000 1 Processor -- start 2024-09-19T08:44:04.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bdf4c5000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561de257ab20 0x561de2535e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:04.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bdf4c5000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561de257b440 0x561de253bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:04.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bdf4c5000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561de2536390 0x561de253c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:04.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bdf4c5000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561de23e1b00 con 0x561de257ab20 2024-09-19T08:44:04.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bdf4c5000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561de235a360 con 0x561de2536390 2024-09-19T08:44:04.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bdf4c5000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x561de23572d0 con 0x561de257b440 2024-09-19T08:44:04.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bdf4c5000 10 client.0 _ll_register_callbacks cb 0x561de251c820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:44:04.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bda1a5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561de257ab20 0x561de2535e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:04.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bda1a5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561de257ab20 0x561de2535e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:50848/0 (socket says 172.21.15.98:50848) 2024-09-19T08:44:04.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bda1a5640 1 -- 172.21.15.98:0/981621596 learned_addr learned my addr 172.21.15.98:0/981621596 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:04.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bda9a6640 1 --2- 172.21.15.98:0/981621596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561de2536390 0x561de253c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:04.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bd99a4640 1 --2- 172.21.15.98:0/981621596 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561de257b440 0x561de253bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:04.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bda1a5640 1 -- 172.21.15.98:0/981621596 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561de257b440 msgr2=0x561de253bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:04.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bda1a5640 1 --2- 172.21.15.98:0/981621596 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561de257b440 0x561de253bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bda1a5640 1 -- 172.21.15.98:0/981621596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561de2536390 msgr2=0x561de253c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:04.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bda1a5640 1 --2- 172.21.15.98:0/981621596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561de2536390 0x561de253c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bda1a5640 1 -- 172.21.15.98:0/981621596 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561de2326120 con 0x561de257ab20 2024-09-19T08:44:04.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bda9a6640 1 --2- 172.21.15.98:0/981621596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561de2536390 0x561de253c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:04.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bda1a5640 1 --2- 172.21.15.98:0/981621596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561de257ab20 0x561de2535e50 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0097c00 tx=0x7f3bd00f06c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2709cbb14c3b2de server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:04.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.324+0000 7f3bb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-19T08:44:04.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.328+0000 7f3bb57fa640 1 -- 172.21.15.98:0/981621596 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd00f8ae0 con 0x561de257ab20 2024-09-19T08:44:04.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.328+0000 7f3bb57fa640 1 -- 172.21.15.98:0/981621596 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f3bd00f0160 con 0x561de257ab20 2024-09-19T08:44:04.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.328+0000 7f3bdf4c5000 1 -- 172.21.15.98:0/981621596 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bcc00b6e0 con 0x561de257ab20 2024-09-19T08:44:04.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.328+0000 7f3bb57fa640 1 -- 172.21.15.98:0/981621596 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f3bd00f03c0 con 0x561de257ab20 2024-09-19T08:44:04.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.328+0000 7f3bdf4c5000 10 client.7146 fetch_fsmap learned FSMap version 60 2024-09-19T08:44:04.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.328+0000 7f3bdf4c5000 1 -- 172.21.15.98:0/981621596 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) -- 0x561de231f450 con 0x561de257ab20 2024-09-19T08:44:04.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.328+0000 7f3bb57fa640 1 -- 172.21.15.98:0/981621596 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bcc00b6e0 con 0x561de257ab20 2024-09-19T08:44:04.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.328+0000 7f3bb57fa640 1 -- 172.21.15.98:0/981621596 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 60) ==== 3502+0+0 (secure 0 0 0) 0x7f3bd00f8650 con 0x561de257ab20 2024-09-19T08:44:04.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.328+0000 7f3bdf4c5000 10 client.7146 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T08:44:04.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.328+0000 7f3bdf4c5000 10 client.7146 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T08:44:04.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.328+0000 7f3bdf4c5000 20 client.7146 populate_metadata read hostname 'smithi098' 2024-09-19T08:44:04.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.328+0000 7f3bdf4c5000 1 --2- 172.21.15.98:0/981621596 >> [v2:172.21.15.98:6834/273091170,v1:172.21.15.98:6835/273091170] conn(0x561de2589280 0x561de258b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:04.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.328+0000 7f3bdf4c5000 4 client.7146 mds_command: new command op to 6996 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:44:04.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.328+0000 7f3bdf4c5000 1 -- 172.21.15.98:0/981621596 --> [v2:172.21.15.98:6834/273091170,v1:172.21.15.98:6835/273091170] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561de2319c00 con 0x561de2589280 2024-09-19T08:44:04.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.328+0000 7f3bd99a4640 1 --2- 172.21.15.98:0/981621596 >> [v2:172.21.15.98:6834/273091170,v1:172.21.15.98:6835/273091170] conn(0x561de2589280 0x561de258b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:04.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.328+0000 7f3bb57fa640 10 client.7146 ms_handle_connect on v2:172.21.15.98:6834/273091170 2024-09-19T08:44:04.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.328+0000 7f3bd99a4640 1 --2- 172.21.15.98:0/981621596 >> [v2:172.21.15.98:6834/273091170,v1:172.21.15.98:6835/273091170] conn(0x561de2589280 0x561de258b670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb228d0026278553 server_cookie=f548896b054d47aa in_seq=0 out_seq=0 2024-09-19T08:44:04.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.328+0000 7f3bb57fa640 1 -- 172.21.15.98:0/981621596 <== mds.0 v2:172.21.15.98:6834/273091170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561de2319c00 con 0x561de2589280 2024-09-19T08:44:04.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.328+0000 7f3bb57fa640 10 client.7146 handle_command_reply: tid=0 2024-09-19T08:44:04.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.328+0000 7f3bdf4c5000 1 client.7146 shutdown 2024-09-19T08:44:04.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.328+0000 7f3bdf4c5000 1 -- 172.21.15.98:0/981621596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561de257ab20 msgr2=0x561de2535e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:04.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.328+0000 7f3bdf4c5000 1 --2- 172.21.15.98:0/981621596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561de257ab20 0x561de2535e50 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0097c00 tx=0x7f3bd00f06c0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.328+0000 7f3bdf4c5000 1 -- 172.21.15.98:0/981621596 shutdown_connections 2024-09-19T08:44:04.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.328+0000 7f3bdf4c5000 1 --2- 172.21.15.98:0/981621596 >> [v2:172.21.15.98:6834/273091170,v1:172.21.15.98:6835/273091170] conn(0x561de2589280 0x561de258b670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.328+0000 7f3bdf4c5000 1 --2- 172.21.15.98:0/981621596 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561de2536390 0x561de253c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.328+0000 7f3bdf4c5000 1 --2- 172.21.15.98:0/981621596 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561de257b440 0x561de253bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.328+0000 7f3bdf4c5000 1 --2- 172.21.15.98:0/981621596 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561de257ab20 0x561de2535e50 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.328+0000 7f3bdf4c5000 1 -- 172.21.15.98:0/981621596 >> 172.21.15.98:0/981621596 conn(0x561de2531290 msgr2=0x561de252fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:04.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.328+0000 7f3bb57fa640 0 client.7146 ms_handle_reset on v2:172.21.15.98:6834/273091170 2024-09-19T08:44:04.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.328+0000 7f3bdf4c5000 1 -- 172.21.15.98:0/981621596 shutdown_connections 2024-09-19T08:44:04.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.328+0000 7f3bdf4c5000 1 -- 172.21.15.98:0/981621596 wait complete. 2024-09-19T08:44:04.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.328+0000 7f3bdf4c5000 20 client.7146 trim_cache size 0 max 16384 2024-09-19T08:44:04.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.344+0000 7f3bdf4c5000 1 Processor -- start 2024-09-19T08:44:04.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.344+0000 7f3bdf4c5000 1 -- start start 2024-09-19T08:44:04.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bdf4c5000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561de2531290 0x561de2579cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:04.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bdf4c5000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561de2536390 0x561de257a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:04.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bdf4c5000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561de257b440 0x561de258b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:04.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bdf4c5000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd40af550 con 0x561de2536390 2024-09-19T08:44:04.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bdf4c5000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd40ae8b0 con 0x561de257b440 2024-09-19T08:44:04.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bdf4c5000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3bd40aef00 con 0x561de2531290 2024-09-19T08:44:04.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bd99a4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561de2536390 0x561de257a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:04.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bd99a4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561de2536390 0x561de257a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:50850/0 (socket says 172.21.15.98:50850) 2024-09-19T08:44:04.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bd99a4640 1 -- 172.21.15.98:0/2481790586 learned_addr learned my addr 172.21.15.98:0/2481790586 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:04.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bda1a5640 1 --2- 172.21.15.98:0/2481790586 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561de2531290 0x561de2579cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:04.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bd99a4640 1 -- 172.21.15.98:0/2481790586 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561de2531290 msgr2=0x561de2579cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:04.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bd99a4640 1 --2- 172.21.15.98:0/2481790586 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561de2531290 0x561de2579cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bd99a4640 1 -- 172.21.15.98:0/2481790586 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561de257b440 msgr2=0x561de258b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:04.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bd99a4640 1 --2- 172.21.15.98:0/2481790586 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561de257b440 0x561de258b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bd99a4640 1 -- 172.21.15.98:0/2481790586 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561de2306fd0 con 0x561de2536390 2024-09-19T08:44:04.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bd99a4640 1 --2- 172.21.15.98:0/2481790586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561de2536390 0x561de257a230 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f3bd40cdce0 tx=0x7f3bc4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab201a90e4eb55b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:04.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bb77fe640 1 -- 172.21.15.98:0/2481790586 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4012700 con 0x561de2536390 2024-09-19T08:44:04.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bb77fe640 1 -- 172.21.15.98:0/2481790586 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bc4025070 con 0x561de2536390 2024-09-19T08:44:04.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bb77fe640 1 -- 172.21.15.98:0/2481790586 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4012aa0 con 0x561de2536390 2024-09-19T08:44:04.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bdf4c5000 1 -- 172.21.15.98:0/2481790586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561de2536390 msgr2=0x561de257a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:04.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bdf4c5000 1 --2- 172.21.15.98:0/2481790586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561de2536390 0x561de257a230 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f3bd40cdce0 tx=0x7f3bc4013000 comp rx=0 tx=0).stop 2024-09-19T08:44:04.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bdf4c5000 1 -- 172.21.15.98:0/2481790586 shutdown_connections 2024-09-19T08:44:04.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bdf4c5000 1 --2- 172.21.15.98:0/2481790586 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561de257b440 0x561de258b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bdf4c5000 1 --2- 172.21.15.98:0/2481790586 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561de2536390 0x561de257a230 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bdf4c5000 1 --2- 172.21.15.98:0/2481790586 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561de2531290 0x561de2579cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bdf4c5000 1 -- 172.21.15.98:0/2481790586 >> 172.21.15.98:0/2481790586 conn(0x561de2537070 msgr2=0x561de2535650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:04.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bdf4c5000 1 -- 172.21.15.98:0/2481790586 shutdown_connections 2024-09-19T08:44:04.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bdf4c5000 1 -- 172.21.15.98:0/2481790586 wait complete. 2024-09-19T08:44:04.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bdf4c5000 1 -- start start 2024-09-19T08:44:04.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bdf4c5000 1 Processor -- start 2024-09-19T08:44:04.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bdf4c5000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561de2531290 0x561de258e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:04.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bdf4c5000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561de2536390 0x561de25a9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:04.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bdf4c5000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561de257b440 0x561de25a9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:04.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bdf4c5000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x561de237ef80 con 0x561de2531290 2024-09-19T08:44:04.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bdf4c5000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561de2337ca0 con 0x561de257b440 2024-09-19T08:44:04.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bdf4c5000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x561de23842a0 con 0x561de2536390 2024-09-19T08:44:04.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bdf4c5000 10 client.0 _ll_register_callbacks cb 0x7f3bd0006d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:44:04.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bda1a5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561de2531290 0x561de258e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:04.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bda1a5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561de2531290 0x561de258e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:50862/0 (socket says 172.21.15.98:50862) 2024-09-19T08:44:04.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bda1a5640 1 -- 172.21.15.98:0/2337137282 learned_addr learned my addr 172.21.15.98:0/2337137282 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:04.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bd99a4640 1 --2- 172.21.15.98:0/2337137282 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561de2536390 0x561de25a9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:04.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bda1a5640 1 -- 172.21.15.98:0/2337137282 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561de2536390 msgr2=0x561de25a9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:04.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bda9a6640 1 --2- 172.21.15.98:0/2337137282 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561de257b440 0x561de25a9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:04.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bda1a5640 1 --2- 172.21.15.98:0/2337137282 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561de2536390 0x561de25a9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bda1a5640 1 -- 172.21.15.98:0/2337137282 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561de257b440 msgr2=0x561de25a9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:04.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bda1a5640 1 --2- 172.21.15.98:0/2337137282 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561de257b440 0x561de25a9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bda1a5640 1 -- 172.21.15.98:0/2337137282 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561de2402750 con 0x561de2531290 2024-09-19T08:44:04.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bd99a4640 1 --2- 172.21.15.98:0/2337137282 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561de2536390 0x561de25a9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:04.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bda1a5640 1 --2- 172.21.15.98:0/2337137282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561de2531290 0x561de258e1b0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f3bd001a910 tx=0x7f3bd00bf000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705f8ff6624c7c44 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:04.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-19T08:44:04.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bb57fa640 1 -- 172.21.15.98:0/2337137282 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd00f8ae0 con 0x561de2531290 2024-09-19T08:44:04.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bb57fa640 1 -- 172.21.15.98:0/2337137282 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f3bd00ef350 con 0x561de2531290 2024-09-19T08:44:04.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bdf4c5000 1 -- 172.21.15.98:0/2337137282 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bc4025070 con 0x561de2531290 2024-09-19T08:44:04.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bb57fa640 1 -- 172.21.15.98:0/2337137282 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7f3bd00ef550 con 0x561de2531290 2024-09-19T08:44:04.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bdf4c5000 10 client.7152 fetch_fsmap learned FSMap version 60 2024-09-19T08:44:04.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bdf4c5000 1 -- 172.21.15.98:0/2337137282 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) -- 0x561de230da30 con 0x561de2531290 2024-09-19T08:44:04.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bb57fa640 1 -- 172.21.15.98:0/2337137282 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bc4025070 con 0x561de2531290 2024-09-19T08:44:04.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bb57fa640 1 -- 172.21.15.98:0/2337137282 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 60) ==== 3502+0+0 (secure 0 0 0) 0x7f3bd00ebc50 con 0x561de2531290 2024-09-19T08:44:04.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bdf4c5000 10 client.7152 fetch_fsmap finished waiting for FSMap version 60 2024-09-19T08:44:04.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bdf4c5000 10 client.7152 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T08:44:04.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bdf4c5000 20 client.7152 populate_metadata read hostname 'smithi098' 2024-09-19T08:44:04.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bdf4c5000 1 --2- 172.21.15.98:0/2337137282 >> [v2:172.21.15.98:6834/273091170,v1:172.21.15.98:6835/273091170] conn(0x561de25aa3e0 0x561de25ee2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:04.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bdf4c5000 4 client.7152 mds_command: new command op to 6996 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T08:44:04.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.348+0000 7f3bdf4c5000 1 -- 172.21.15.98:0/2337137282 --> [v2:172.21.15.98:6834/273091170,v1:172.21.15.98:6835/273091170] -- command(tid 0: {"prefix": "status"}) -- 0x561de2537450 con 0x561de25aa3e0 2024-09-19T08:44:04.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.352+0000 7f3bd99a4640 1 --2- 172.21.15.98:0/2337137282 >> [v2:172.21.15.98:6834/273091170,v1:172.21.15.98:6835/273091170] conn(0x561de25aa3e0 0x561de25ee2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:04.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.352+0000 7f3bd99a4640 1 --2- 172.21.15.98:0/2337137282 >> [v2:172.21.15.98:6834/273091170,v1:172.21.15.98:6835/273091170] conn(0x561de25aa3e0 0x561de25ee2b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6e15c152f46677f server_cookie=8f95f65bbe6c67c6 in_seq=0 out_seq=0 2024-09-19T08:44:04.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.352+0000 7f3bb57fa640 10 client.7152 ms_handle_connect on v2:172.21.15.98:6834/273091170 2024-09-19T08:44:04.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.352+0000 7f3bb57fa640 1 -- 172.21.15.98:0/2337137282 <== mds.0 v2:172.21.15.98:6834/273091170 1 ==== command_reply(tid 0: 0 ) ==== 8+0+320 (crc 0 0 0) 0x561de2537450 con 0x561de25aa3e0 2024-09-19T08:44:04.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.352+0000 7f3bb57fa640 10 client.7152 handle_command_reply: tid=0 2024-09-19T08:44:04.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.352+0000 7f3bdf4c5000 1 client.7152 shutdown 2024-09-19T08:44:04.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.352+0000 7f3bdf4c5000 1 -- 172.21.15.98:0/2337137282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561de2531290 msgr2=0x561de258e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:04.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.352+0000 7f3bdf4c5000 1 --2- 172.21.15.98:0/2337137282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561de2531290 0x561de258e1b0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f3bd001a910 tx=0x7f3bd00bf000 comp rx=0 tx=0).stop 2024-09-19T08:44:04.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.352+0000 7f3bdf4c5000 1 -- 172.21.15.98:0/2337137282 shutdown_connections 2024-09-19T08:44:04.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.352+0000 7f3bdf4c5000 1 --2- 172.21.15.98:0/2337137282 >> [v2:172.21.15.98:6834/273091170,v1:172.21.15.98:6835/273091170] conn(0x561de25aa3e0 0x561de25ee2b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.352+0000 7f3bdf4c5000 1 --2- 172.21.15.98:0/2337137282 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561de257b440 0x561de25a9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.352+0000 7f3bdf4c5000 1 --2- 172.21.15.98:0/2337137282 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x561de2536390 0x561de25a9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.352+0000 7f3bdf4c5000 1 --2- 172.21.15.98:0/2337137282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x561de2531290 0x561de258e1b0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.352+0000 7f3bdf4c5000 1 -- 172.21.15.98:0/2337137282 >> 172.21.15.98:0/2337137282 conn(0x561de2537070 msgr2=0x561de2578ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:04.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.352+0000 7f3bb57fa640 0 client.7152 ms_handle_reset on v2:172.21.15.98:6834/273091170 2024-09-19T08:44:04.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.352+0000 7f3bdf4c5000 1 -- 172.21.15.98:0/2337137282 shutdown_connections 2024-09-19T08:44:04.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.352+0000 7f3bdf4c5000 1 -- 172.21.15.98:0/2337137282 wait complete. 2024-09-19T08:44:04.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.352+0000 7f3bdf4c5000 20 client.7152 trim_cache size 0 max 16384 2024-09-19T08:44:04.361 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:44:04.361 INFO:teuthology.orchestra.run.smithi098.stdout: "cluster_fsid": "552bf0ee-1f4f-4644-ba7c-d0931e55a451", 2024-09-19T08:44:04.361 INFO:teuthology.orchestra.run.smithi098.stdout: "whoami": 0, 2024-09-19T08:44:04.361 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 6996, 2024-09-19T08:44:04.361 INFO:teuthology.orchestra.run.smithi098.stdout: "want_state": "up:active", 2024-09-19T08:44:04.361 INFO:teuthology.orchestra.run.smithi098.stdout: "state": "up:active", 2024-09-19T08:44:04.362 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "cephfs", 2024-09-19T08:44:04.362 INFO:teuthology.orchestra.run.smithi098.stdout: "rank_uptime": 9.204892396, 2024-09-19T08:44:04.362 INFO:teuthology.orchestra.run.smithi098.stdout: "mdsmap_epoch": 60, 2024-09-19T08:44:04.362 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch": 104, 2024-09-19T08:44:04.362 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch_barrier": 102, 2024-09-19T08:44:04.362 INFO:teuthology.orchestra.run.smithi098.stdout: "uptime": 14.166962325 2024-09-19T08:44:04.362 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:44:04.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.352+0000 7f3bdc409640 1 -- 172.21.15.98:0/4115723755 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3bb80679e0 msgr2=0x7f3bb8069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:04.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.352+0000 7f3bdc409640 1 --2- 172.21.15.98:0/4115723755 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3bb80679e0 0x7f3bb8069ea0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4006fd0 tx=0x7f3bc4008040 comp rx=0 tx=0).stop 2024-09-19T08:44:04.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.352+0000 7f3bdc409640 1 -- 172.21.15.98:0/4115723755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bd406c1d0 msgr2=0x7f3bd414f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:04.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.352+0000 7f3bdc409640 1 --2- 172.21.15.98:0/4115723755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bd406c1d0 0x7f3bd414f930 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f3bd00ae040 tx=0x7f3bd0019000 comp rx=0 tx=0).stop 2024-09-19T08:44:04.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.352+0000 7f3bdc409640 1 -- 172.21.15.98:0/4115723755 shutdown_connections 2024-09-19T08:44:04.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.352+0000 7f3bdc409640 1 --2- 172.21.15.98:0/4115723755 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3bb80679e0 0x7f3bb8069ea0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.352+0000 7f3bdc409640 1 --2- 172.21.15.98:0/4115723755 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3bd415c550 0x7f3bd4156cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.352+0000 7f3bdc409640 1 --2- 172.21.15.98:0/4115723755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3bd40cf900 0x7f3bd414da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.352+0000 7f3bdc409640 1 --2- 172.21.15.98:0/4115723755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3bd406c1d0 0x7f3bd414f930 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.352+0000 7f3bdc409640 1 -- 172.21.15.98:0/4115723755 >> 172.21.15.98:0/4115723755 conn(0x7f3bd40b2010 msgr2=0x7f3bd40afd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:04.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.352+0000 7f3bdc409640 1 -- 172.21.15.98:0/4115723755 shutdown_connections 2024-09-19T08:44:04.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.352+0000 7f3bdc409640 1 -- 172.21.15.98:0/4115723755 wait complete. 2024-09-19T08:44:04.368 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T08:44:04.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.476+0000 7f02fb072640 1 Processor -- start 2024-09-19T08:44:04.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.476+0000 7f02fb072640 1 -- start start 2024-09-19T08:44:04.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.476+0000 7f02fb072640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f02f415c550 0x7f02f406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:04.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.476+0000 7f02fb072640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f02f406c1d0 0x7f02f40cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:04.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.476+0000 7f02fb072640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02f406d310 con 0x7f02f40cf900 2024-09-19T08:44:04.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.476+0000 7f02fb072640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f02f406c640 con 0x7f02f415c550 2024-09-19T08:44:04.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.476+0000 7f02fb072640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f02f406d5f0 con 0x7f02f406c1d0 2024-09-19T08:44:04.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.476+0000 7f02f8e0e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f02f415c550 0x7f02f406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:04.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.476+0000 7f02f8e0e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f02f415c550 0x7f02f406bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:50874/0 (socket says 172.21.15.98:50874) 2024-09-19T08:44:04.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.476+0000 7f02f8e0e640 1 -- 172.21.15.98:0/2349401517 learned_addr learned my addr 172.21.15.98:0/2349401517 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:04.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.476+0000 7f02f3fff640 1 --2- 172.21.15.98:0/2349401517 >> v2:172.21.15.139:3300/0 conn(0x7f02f406c1d0 0x7f02f40cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:04.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.476+0000 7f02f8e0e640 1 -- 172.21.15.98:0/2349401517 >> v2:172.21.15.139:3300/0 conn(0x7f02f406c1d0 msgr2=0x7f02f40cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:04.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.476+0000 7f02f8e0e640 1 --2- 172.21.15.98:0/2349401517 >> v2:172.21.15.139:3300/0 conn(0x7f02f406c1d0 0x7f02f40cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.476+0000 7f02f8e0e640 1 -- 172.21.15.98:0/2349401517 >> v1:172.21.15.139:6789/0 conn(0x7f02f40cf900 legacy=0x7f02f40ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:04.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.476+0000 7f02f8e0e640 1 -- 172.21.15.98:0/2349401517 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f406d8d0 con 0x7f02f415c550 2024-09-19T08:44:04.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.480+0000 7f02f8e0e640 1 --2- 172.21.15.98:0/2349401517 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02f415c550 0x7f02f406bc90 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f02f40ccc60 tx=0x7f02e401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b8ef3b09e0768ae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:04.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.480+0000 7f02f37fe640 1 -- 172.21.15.98:0/2349401517 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e40b0070 con 0x7f02f415c550 2024-09-19T08:44:04.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.480+0000 7f02f37fe640 1 -- 172.21.15.98:0/2349401517 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02e40ab600 con 0x7f02f415c550 2024-09-19T08:44:04.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.480+0000 7f02f37fe640 1 -- 172.21.15.98:0/2349401517 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e40ab9d0 con 0x7f02f415c550 2024-09-19T08:44:04.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.480+0000 7f02fb072640 1 -- 172.21.15.98:0/2349401517 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02f415c550 msgr2=0x7f02f406bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:04.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.480+0000 7f02fb072640 1 --2- 172.21.15.98:0/2349401517 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02f415c550 0x7f02f406bc90 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f02f40ccc60 tx=0x7f02e401c4e0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.480+0000 7f02fb072640 1 -- 172.21.15.98:0/2349401517 shutdown_connections 2024-09-19T08:44:04.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.480+0000 7f02fb072640 1 --2- 172.21.15.98:0/2349401517 >> v2:172.21.15.139:3300/0 conn(0x7f02f406c1d0 0x7f02f40cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.480+0000 7f02fb072640 1 --2- 172.21.15.98:0/2349401517 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02f415c550 0x7f02f406bc90 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.480+0000 7f02fb072640 1 -- 172.21.15.98:0/2349401517 >> 172.21.15.98:0/2349401517 conn(0x7f02f40b2010 msgr2=0x7f02f40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:04.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.480+0000 7f02fb072640 1 -- 172.21.15.98:0/2349401517 shutdown_connections 2024-09-19T08:44:04.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.480+0000 7f02fb072640 1 -- 172.21.15.98:0/2349401517 wait complete. 2024-09-19T08:44:04.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.480+0000 7f02fb072640 1 Processor -- start 2024-09-19T08:44:04.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.480+0000 7f02fb072640 1 -- start start 2024-09-19T08:44:04.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.484+0000 7f02fb072640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f02f406c1d0 0x7f02f40aef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:04.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.484+0000 7f02fb072640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02f40cf900 0x7f02f4152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:04.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.484+0000 7f02fb072640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02f415c550 0x7f02f41594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:04.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.484+0000 7f02fb072640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f02f41693b0 con 0x7f02f415c550 2024-09-19T08:44:04.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.484+0000 7f02fb072640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f02f4169230 con 0x7f02f40cf900 2024-09-19T08:44:04.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.484+0000 7f02fb072640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f02f4169530 con 0x7f02f406c1d0 2024-09-19T08:44:04.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.484+0000 7f02f960f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02f415c550 0x7f02f41594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:04.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.484+0000 7f02f960f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02f415c550 0x7f02f41594c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:50876/0 (socket says 172.21.15.98:50876) 2024-09-19T08:44:04.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.484+0000 7f02f960f640 1 -- 172.21.15.98:0/117421971 learned_addr learned my addr 172.21.15.98:0/117421971 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:04.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.484+0000 7f02f960f640 1 -- 172.21.15.98:0/117421971 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f02f406c1d0 msgr2=0x7f02f40aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:04.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.484+0000 7f02f960f640 1 --2- 172.21.15.98:0/117421971 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f02f406c1d0 0x7f02f40aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.484+0000 7f02f960f640 1 -- 172.21.15.98:0/117421971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02f40cf900 msgr2=0x7f02f4152730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:04.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.484+0000 7f02f3fff640 1 --2- 172.21.15.98:0/117421971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02f40cf900 0x7f02f4152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:04.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.484+0000 7f02f960f640 1 --2- 172.21.15.98:0/117421971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02f40cf900 0x7f02f4152730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.484+0000 7f02f960f640 1 -- 172.21.15.98:0/117421971 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f40ae7c0 con 0x7f02f415c550 2024-09-19T08:44:04.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.484+0000 7f02f960f640 1 --2- 172.21.15.98:0/117421971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02f415c550 0x7f02f41594c0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f02ec00b970 tx=0x7f02ec00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:04.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.484+0000 7f02f17fa640 1 -- 172.21.15.98:0/117421971 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ec019070 con 0x7f02f415c550 2024-09-19T08:44:04.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.484+0000 7f02f17fa640 1 -- 172.21.15.98:0/117421971 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02ec00cae0 con 0x7f02f415c550 2024-09-19T08:44:04.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.484+0000 7f02f17fa640 1 -- 172.21.15.98:0/117421971 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ec014440 con 0x7f02f415c550 2024-09-19T08:44:04.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.484+0000 7f02fb072640 1 -- 172.21.15.98:0/117421971 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02f40cbfc0 con 0x7f02f415c550 2024-09-19T08:44:04.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.484+0000 7f02fb072640 1 -- 172.21.15.98:0/117421971 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02f40cc580 con 0x7f02f415c550 2024-09-19T08:44:04.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.484+0000 7f02fb072640 1 -- 172.21.15.98:0/117421971 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02f40cf3c0 con 0x7f02f415c550 2024-09-19T08:44:04.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.484+0000 7f02f17fa640 1 -- 172.21.15.98:0/117421971 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f02ec01c070 con 0x7f02f415c550 2024-09-19T08:44:04.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.484+0000 7f02f17fa640 1 --2- 172.21.15.98:0/117421971 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f02d40679e0 0x7f02d4069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:04.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.484+0000 7f02f17fa640 1 -- 172.21.15.98:0/117421971 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f02ec085970 con 0x7f02f415c550 2024-09-19T08:44:04.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.484+0000 7f02f8e0e640 1 --2- 172.21.15.98:0/117421971 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f02d40679e0 0x7f02d4069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:04.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.488+0000 7f02f8e0e640 1 --2- 172.21.15.98:0/117421971 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f02d40679e0 0x7f02d4069ea0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f02e40ad000 tx=0x7f02e4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:04.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.488+0000 7f02f17fa640 1 -- 172.21.15.98:0/117421971 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f02ec0563f0 con 0x7f02f415c550 2024-09-19T08:44:04.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.684+0000 7f02fb072640 1 -- 172.21.15.98:0/117421971 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f02f40aa750 con 0x7f02f415c550 2024-09-19T08:44:04.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.684+0000 7f02f17fa640 1 -- 172.21.15.98:0/117421971 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v60) ==== 53+0+73 (secure 0 0 0) 0x7f02ec055cf0 con 0x7f02f415c550 2024-09-19T08:44:04.686 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T08:44:04.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.684+0000 7f02fb072640 1 -- 172.21.15.98:0/117421971 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f02d40679e0 msgr2=0x7f02d4069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:04.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.684+0000 7f02fb072640 1 --2- 172.21.15.98:0/117421971 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f02d40679e0 0x7f02d4069ea0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f02e40ad000 tx=0x7f02e4008040 comp rx=0 tx=0).stop 2024-09-19T08:44:04.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.688+0000 7f02fb072640 1 -- 172.21.15.98:0/117421971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02f415c550 msgr2=0x7f02f41594c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:04.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.688+0000 7f02fb072640 1 --2- 172.21.15.98:0/117421971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02f415c550 0x7f02f41594c0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f02ec00b970 tx=0x7f02ec00be30 comp rx=0 tx=0).stop 2024-09-19T08:44:04.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.688+0000 7f02fb072640 1 -- 172.21.15.98:0/117421971 shutdown_connections 2024-09-19T08:44:04.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.688+0000 7f02fb072640 1 --2- 172.21.15.98:0/117421971 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f02d40679e0 0x7f02d4069ea0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.688+0000 7f02fb072640 1 --2- 172.21.15.98:0/117421971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f02f415c550 0x7f02f41594c0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.688+0000 7f02fb072640 1 --2- 172.21.15.98:0/117421971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f02f40cf900 0x7f02f4152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.688+0000 7f02fb072640 1 --2- 172.21.15.98:0/117421971 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f02f406c1d0 0x7f02f40aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:04.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.688+0000 7f02fb072640 1 -- 172.21.15.98:0/117421971 >> 172.21.15.98:0/117421971 conn(0x7f02f40b2010 msgr2=0x7f02f40b0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:04.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.688+0000 7f02fb072640 1 -- 172.21.15.98:0/117421971 shutdown_connections 2024-09-19T08:44:04.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04.688+0000 7f02fb072640 1 -- 172.21.15.98:0/117421971 wait complete. 2024-09-19T08:44:04.705 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T08:44:04.705 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T08:44:04.705 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi098.front.sepia.ceph.com 2024-09-19T08:44:04.705 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T08:44:04.705 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T08:44:04.705 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T08:44:04.705 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T08:44:04.705 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T08:44:04.705 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:44:04.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:04.706 DEBUG:teuthology.orchestra.run.smithi098:> ip addr 2024-09-19T08:44:04.711 INFO:teuthology.orchestra.run.smithi098.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T08:44:04.711 INFO:teuthology.orchestra.run.smithi098.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T08:44:04.712 INFO:teuthology.orchestra.run.smithi098.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T08:44:04.712 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:44:04.712 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 ::1/128 scope host 2024-09-19T08:44:04.712 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:44:04.712 INFO:teuthology.orchestra.run.smithi098.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:44:04.712 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b8 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:44:04.712 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f0 2024-09-19T08:44:04.712 INFO:teuthology.orchestra.run.smithi098.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:44:04.712 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b9 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:44:04.712 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f1 2024-09-19T08:44:04.712 INFO:teuthology.orchestra.run.smithi098.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:44:04.712 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6a brd ff:ff:ff:ff:ff:ff 2024-09-19T08:44:04.712 INFO:teuthology.orchestra.run.smithi098.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T08:44:04.713 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6b brd ff:ff:ff:ff:ff:ff 2024-09-19T08:44:04.713 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.21.15.98/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T08:44:04.713 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft 42164sec preferred_lft 42164sec 2024-09-19T08:44:04.713 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::ec4:7aff:fe8f:d26b/64 scope link 2024-09-19T08:44:04.713 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:44:04.713 INFO:teuthology.orchestra.run.smithi098.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T08:44:04.713 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 02:42:98:dd:32:2d brd ff:ff:ff:ff:ff:ff 2024-09-19T08:44:04.713 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T08:44:04.713 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:44:04.713 INFO:teuthology.orchestra.run.smithi098.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T08:44:04.713 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-19T08:44:04.713 INFO:teuthology.orchestra.run.smithi098.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T08:44:04.713 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:44:04.713 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-19T08:44:04.713 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:44:04.714 INFO:teuthology.orchestra.run.smithi098.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:44:04.714 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:44:04.714 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-19T08:44:04.714 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:44:04.714 INFO:teuthology.orchestra.run.smithi098.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:44:04.714 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:44:04.714 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-19T08:44:04.714 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:44:04.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:04.714 DEBUG:teuthology.orchestra.run.smithi098:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T08:44:04.764 INFO:teuthology.orchestra.run.smithi098.stdout:1 2024-09-19T08:44:04.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:04.766 DEBUG:teuthology.orchestra.run.smithi098:> route 2024-09-19T08:44:04.818 INFO:teuthology.orchestra.run.smithi098.stdout:Kernel IP routing table 2024-09-19T08:44:04.818 INFO:teuthology.orchestra.run.smithi098.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T08:44:04.818 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T08:44:04.819 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T08:44:04.819 INFO:teuthology.orchestra.run.smithi098.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T08:44:04.819 INFO:teuthology.orchestra.run.smithi098.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T08:44:04.819 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T08:44:04.819 INFO:teuthology.orchestra.run.smithi098.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:44:04.819 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:44:04.819 INFO:teuthology.orchestra.run.smithi098.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:44:04.820 INFO:teuthology.orchestra.run.smithi098.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T08:44:04.820 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:44:04.820 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-09-19T08:44:04.820 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T08:44:04.820 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T08:44:04.820 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T08:44:04.820 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-09-19T08:44:04.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:44:04.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:44:04.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:04.939 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2024-09-19T08:44:04.985 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-19T08:44:04.985 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T08:44:04.985 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:44:04.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:04.986 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set brx.0 up 2024-09-19T08:44:05.038 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-19T08:44:05.039 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:44:05.039 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T08:44:05.086 INFO:teuthology.orchestra.run.smithi098.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:44:05.086 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:44:05.086 DEBUG:teuthology.orchestra.run.smithi098:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T08:44:05.132 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:44:05.190 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:44:05.237 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:44:05.238 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:44:05.238 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:44:05.238 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:44:05.238 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:44:05.238 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:44:05.238 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:44:05.238 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:44:05.238 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:44:05.238 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:44:05.238 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:44:05.238 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:44:05.238 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:44:05.239 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:44:05.239 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:44:05.239 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:44:05.239 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:44:05.239 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:44:05.239 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:44:05.239 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:44:05.239 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:44:05.239 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:44:05.239 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:44:05.239 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:44:05.239 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:44:05.239 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:44:05.239 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:44:05.240 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:44:05.240 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:44:05.240 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:44:05.240 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:44:05.240 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:44:05.240 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:44:05.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:05.241 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:44:05.290 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T08:44:05.290 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-19T08:44:05.334 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:44:05.349 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:44:05.401 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:44:05.401 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:44:05.401 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:44:05.401 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:44:05.401 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:44:05.402 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:44:05.402 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:44:05.402 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:44:05.402 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:44:05.402 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:44:05.402 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:44:05.402 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:44:05.402 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:44:05.402 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:44:05.402 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:44:05.402 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:44:05.402 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:44:05.402 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:44:05.402 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:44:05.403 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:44:05.403 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:44:05.403 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:44:05.403 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:44:05.403 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:44:05.403 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:44:05.403 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:44:05.403 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:44:05.403 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:44:05.403 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:44:05.403 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:44:05.403 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:44:05.403 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:44:05.404 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:44:05.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:05.404 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:44:05.413 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:2024-09-19T08:44:05.412+0000 7f392d2ad580 -1 init, newargv = 0x55e5dd713d00 newargc=15 2024-09-19T08:44:05.413 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:ceph-fuse[46027]: starting ceph client 2024-09-19T08:44:05.418 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:ceph-fuse[46027]: starting fuse 2024-09-19T08:44:06.518 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:44:06.534 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:44:06.582 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:44:06.582 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:44:06.582 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:44:06.582 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:44:06.582 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:44:06.582 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:44:06.582 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:44:06.582 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:44:06.582 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:44:06.582 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:44:06.583 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:44:06.583 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:44:06.583 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:44:06.583 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:44:06.583 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:44:06.583 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:44:06.583 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:44:06.583 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:44:06.583 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:44:06.583 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:44:06.583 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:44:06.583 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:44:06.583 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:44:06.583 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:44:06.584 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:44:06.584 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:44:06.584 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:44:06.584 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:44:06.584 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:44:06.584 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:44:06.584 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:44:06.584 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:44:06.584 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:44:06.584 INFO:teuthology.orchestra.run.smithi098.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-19T08:44:06.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:06.585 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:44:06.633 INFO:teuthology.orchestra.run.smithi098.stdout:45 2024-09-19T08:44:06.633 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-19T08:44:06.633 DEBUG:teuthology.orchestra.run.smithi098:> sudo stdin-killer -- python3 -c ' 2024-09-19T08:44:06.634 DEBUG:teuthology.orchestra.run.smithi098:> import glob 2024-09-19T08:44:06.634 DEBUG:teuthology.orchestra.run.smithi098:> import re 2024-09-19T08:44:06.634 DEBUG:teuthology.orchestra.run.smithi098:> import os 2024-09-19T08:44:06.634 DEBUG:teuthology.orchestra.run.smithi098:> import subprocess 2024-09-19T08:44:06.634 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:44:06.634 DEBUG:teuthology.orchestra.run.smithi098:> def _find_admin_socket(client_name): 2024-09-19T08:44:06.634 DEBUG:teuthology.orchestra.run.smithi098:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T08:44:06.634 DEBUG:teuthology.orchestra.run.smithi098:> files = glob.glob(asok_path) 2024-09-19T08:44:06.634 DEBUG:teuthology.orchestra.run.smithi098:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T08:44:06.634 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:44:06.634 DEBUG:teuthology.orchestra.run.smithi098:> # Given a non-glob path, it better be there 2024-09-19T08:44:06.634 DEBUG:teuthology.orchestra.run.smithi098:> if "*" not in asok_path: 2024-09-19T08:44:06.634 DEBUG:teuthology.orchestra.run.smithi098:> assert(len(files) == 1) 2024-09-19T08:44:06.634 DEBUG:teuthology.orchestra.run.smithi098:> return files[0] 2024-09-19T08:44:06.635 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:44:06.635 DEBUG:teuthology.orchestra.run.smithi098:> for f in files: 2024-09-19T08:44:06.635 DEBUG:teuthology.orchestra.run.smithi098:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T08:44:06.635 DEBUG:teuthology.orchestra.run.smithi098:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T08:44:06.635 DEBUG:teuthology.orchestra.run.smithi098:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T08:44:06.635 DEBUG:teuthology.orchestra.run.smithi098:> contents = proc_f.read() 2024-09-19T08:44:06.635 DEBUG:teuthology.orchestra.run.smithi098:> if mountpoint in contents: 2024-09-19T08:44:06.635 DEBUG:teuthology.orchestra.run.smithi098:> return f 2024-09-19T08:44:06.635 DEBUG:teuthology.orchestra.run.smithi098:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T08:44:06.635 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:44:06.635 DEBUG:teuthology.orchestra.run.smithi098:> print(_find_admin_socket("client.0")) 2024-09-19T08:44:06.635 DEBUG:teuthology.orchestra.run.smithi098:> ' 2024-09-19T08:44:06.755 INFO:teuthology.orchestra.run.smithi098.stdout:/var/run/ceph/ceph-client.0.46027.asok 2024-09-19T08:44:06.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:44:06.767 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.46027.asok 2024-09-19T08:44:06.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:06.767 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.46027.asok status 2024-09-19T08:44:06.918 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:44:06.918 INFO:teuthology.orchestra.run.smithi098.stdout: "metadata": { 2024-09-19T08:44:06.918 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T08:44:06.918 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T08:44:06.919 INFO:teuthology.orchestra.run.smithi098.stdout: "entity_id": "0", 2024-09-19T08:44:06.919 INFO:teuthology.orchestra.run.smithi098.stdout: "hostname": "smithi098", 2024-09-19T08:44:06.919 INFO:teuthology.orchestra.run.smithi098.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T08:44:06.919 INFO:teuthology.orchestra.run.smithi098.stdout: "pid": "46027", 2024-09-19T08:44:06.919 INFO:teuthology.orchestra.run.smithi098.stdout: "root": "/" 2024-09-19T08:44:06.919 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:44:06.919 INFO:teuthology.orchestra.run.smithi098.stdout: "dentry_count": 0, 2024-09-19T08:44:06.919 INFO:teuthology.orchestra.run.smithi098.stdout: "dentry_pinned_count": 0, 2024-09-19T08:44:06.919 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 7164, 2024-09-19T08:44:06.919 INFO:teuthology.orchestra.run.smithi098.stdout: "inst": { 2024-09-19T08:44:06.919 INFO:teuthology.orchestra.run.smithi098.stdout: "name": { 2024-09-19T08:44:06.919 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "client", 2024-09-19T08:44:06.919 INFO:teuthology.orchestra.run.smithi098.stdout: "num": 7164 2024-09-19T08:44:06.920 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:44:06.920 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": { 2024-09-19T08:44:06.920 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v1", 2024-09-19T08:44:06.920 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:44:06.920 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 2999871010 2024-09-19T08:44:06.920 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:44:06.920 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:44:06.920 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": { 2024-09-19T08:44:06.920 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v1", 2024-09-19T08:44:06.920 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:44:06.920 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 2999871010 2024-09-19T08:44:06.920 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:44:06.920 INFO:teuthology.orchestra.run.smithi098.stdout: "inst_str": "client.7164 192.168.0.1:0/2999871010", 2024-09-19T08:44:06.920 INFO:teuthology.orchestra.run.smithi098.stdout: "addr_str": "192.168.0.1:0/2999871010", 2024-09-19T08:44:06.921 INFO:teuthology.orchestra.run.smithi098.stdout: "inode_count": 1, 2024-09-19T08:44:06.921 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_epoch": 60, 2024-09-19T08:44:06.921 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_epoch": 104, 2024-09-19T08:44:06.921 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_epoch_barrier": 0, 2024-09-19T08:44:06.921 INFO:teuthology.orchestra.run.smithi098.stdout: "blocklisted": false, 2024-09-19T08:44:06.921 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "cephfs" 2024-09-19T08:44:06.921 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:44:06.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:06.932 DEBUG:teuthology.orchestra.run.smithi098:> stat --file-system '--printf=%T 2024-09-19T08:44:06.932 DEBUG:teuthology.orchestra.run.smithi098:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T08:44:06.938 INFO:teuthology.orchestra.run.smithi098.stdout:fuseblk 2024-09-19T08:44:06.939 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T08:44:06.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:06.939 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T08:44:06.993 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T08:44:07.162 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.160+0000 7f88dfa05640 1 Processor -- start 2024-09-19T08:44:07.162 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.160+0000 7f88dfa05640 1 -- start start 2024-09-19T08:44:07.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.160+0000 7f88dfa05640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f88d80733b0 0x7f88d80cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:07.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.160+0000 7f88dfa05640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88d806d310 con 0x7f88d806c3a0 2024-09-19T08:44:07.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.160+0000 7f88dfa05640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88d806d5f0 con 0x7f88d815c550 2024-09-19T08:44:07.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.160+0000 7f88dfa05640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f88d806d8d0 con 0x7f88d80733b0 2024-09-19T08:44:07.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.160+0000 7f88ddfa2640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f88d80733b0 0x7f88d80cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:07.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.160+0000 7f88ddfa2640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f88d80733b0 0x7f88d80cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:50882/0 (socket says 172.21.15.98:50882) 2024-09-19T08:44:07.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.160+0000 7f88ddfa2640 1 -- 172.21.15.98:0/3232004153 learned_addr learned my addr 172.21.15.98:0/3232004153 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:07.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.160+0000 7f88ddfa2640 1 -- 172.21.15.98:0/3232004153 >> v1:172.21.15.139:6789/0 conn(0x7f88d815c550 legacy=0x7f88d806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:07.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.160+0000 7f88ddfa2640 1 -- 172.21.15.98:0/3232004153 >> v1:172.21.15.98:6789/0 conn(0x7f88d806c3a0 legacy=0x7f88d80cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:07.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.160+0000 7f88ddfa2640 1 -- 172.21.15.98:0/3232004153 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d806dbb0 con 0x7f88d80733b0 2024-09-19T08:44:07.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.164+0000 7f88ddfa2640 1 --2- 172.21.15.98:0/3232004153 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88d80733b0 0x7f88d80cfad0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f88d0009e50 tx=0x7f88d001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9870685995e65e47 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:07.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.164+0000 7f88cffff640 1 -- 172.21.15.98:0/3232004153 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d00a56c0 con 0x7f88d80733b0 2024-09-19T08:44:07.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.164+0000 7f88cffff640 1 -- 172.21.15.98:0/3232004153 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88d00a5860 con 0x7f88d80733b0 2024-09-19T08:44:07.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.164+0000 7f88cffff640 1 -- 172.21.15.98:0/3232004153 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d00a5c30 con 0x7f88d80733b0 2024-09-19T08:44:07.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.164+0000 7f88dfa05640 1 -- 172.21.15.98:0/3232004153 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88d80733b0 msgr2=0x7f88d80cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:07.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.164+0000 7f88dfa05640 1 --2- 172.21.15.98:0/3232004153 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88d80733b0 0x7f88d80cfad0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f88d0009e50 tx=0x7f88d001c9f0 comp rx=0 tx=0).stop 2024-09-19T08:44:07.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.164+0000 7f88dfa05640 1 -- 172.21.15.98:0/3232004153 shutdown_connections 2024-09-19T08:44:07.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.164+0000 7f88dfa05640 1 --2- 172.21.15.98:0/3232004153 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88d80733b0 0x7f88d80cfad0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:07.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.164+0000 7f88dfa05640 1 -- 172.21.15.98:0/3232004153 >> 172.21.15.98:0/3232004153 conn(0x7f88d80b2010 msgr2=0x7f88d80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:07.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.164+0000 7f88dfa05640 1 -- 172.21.15.98:0/3232004153 shutdown_connections 2024-09-19T08:44:07.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.164+0000 7f88dfa05640 1 -- 172.21.15.98:0/3232004153 wait complete. 2024-09-19T08:44:07.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.168+0000 7f88dfa05640 1 Processor -- start 2024-09-19T08:44:07.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.168+0000 7f88dfa05640 1 -- start start 2024-09-19T08:44:07.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.168+0000 7f88dfa05640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88d806c3a0 0x7f88d81476b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:07.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.168+0000 7f88dfa05640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f88d80733b0 0x7f88d8154d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:07.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.168+0000 7f88dfa05640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88d815c550 0x7f88d8156900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:07.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.168+0000 7f88dfa05640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f88d8169dd0 con 0x7f88d806c3a0 2024-09-19T08:44:07.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.168+0000 7f88dfa05640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f88d8169c50 con 0x7f88d815c550 2024-09-19T08:44:07.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.168+0000 7f88dfa05640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f88d8169f50 con 0x7f88d80733b0 2024-09-19T08:44:07.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.168+0000 7f88dd7a1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88d806c3a0 0x7f88d81476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:07.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.168+0000 7f88dd7a1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88d806c3a0 0x7f88d81476b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:50892/0 (socket says 172.21.15.98:50892) 2024-09-19T08:44:07.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.168+0000 7f88dd7a1640 1 -- 172.21.15.98:0/3880926545 learned_addr learned my addr 172.21.15.98:0/3880926545 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:07.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.168+0000 7f88ddfa2640 1 --2- 172.21.15.98:0/3880926545 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88d815c550 0x7f88d8156900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:07.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.168+0000 7f88dd7a1640 1 -- 172.21.15.98:0/3880926545 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f88d80733b0 msgr2=0x7f88d8154d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:07.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.168+0000 7f88dd7a1640 1 --2- 172.21.15.98:0/3880926545 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f88d80733b0 0x7f88d8154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:07.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.168+0000 7f88dd7a1640 1 -- 172.21.15.98:0/3880926545 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88d815c550 msgr2=0x7f88d8156900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:07.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.168+0000 7f88dd7a1640 1 --2- 172.21.15.98:0/3880926545 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88d815c550 0x7f88d8156900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:07.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.168+0000 7f88dd7a1640 1 -- 172.21.15.98:0/3880926545 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d814da30 con 0x7f88d806c3a0 2024-09-19T08:44:07.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.168+0000 7f88dd7a1640 1 --2- 172.21.15.98:0/3880926545 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88d806c3a0 0x7f88d81476b0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f88d400b3e0 tx=0x7f88d400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:07.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.168+0000 7f88cdffb640 1 -- 172.21.15.98:0/3880926545 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d4015020 con 0x7f88d806c3a0 2024-09-19T08:44:07.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.168+0000 7f88cdffb640 1 -- 172.21.15.98:0/3880926545 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88d4011850 con 0x7f88d806c3a0 2024-09-19T08:44:07.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.168+0000 7f88dfa05640 1 -- 172.21.15.98:0/3880926545 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88d814dd10 con 0x7f88d806c3a0 2024-09-19T08:44:07.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.168+0000 7f88cdffb640 1 -- 172.21.15.98:0/3880926545 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d4011c40 con 0x7f88d806c3a0 2024-09-19T08:44:07.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.168+0000 7f88dfa05640 1 -- 172.21.15.98:0/3880926545 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88d814e5b0 con 0x7f88d806c3a0 2024-09-19T08:44:07.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.168+0000 7f88dfa05640 1 -- 172.21.15.98:0/3880926545 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88d814dee0 con 0x7f88d806c3a0 2024-09-19T08:44:07.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.168+0000 7f88cdffb640 1 -- 172.21.15.98:0/3880926545 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f88d4018020 con 0x7f88d806c3a0 2024-09-19T08:44:07.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.168+0000 7f88cdffb640 1 --2- 172.21.15.98:0/3880926545 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f88bc0679e0 0x7f88bc069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:07.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.172+0000 7f88cdffb640 1 -- 172.21.15.98:0/3880926545 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f88d40862a0 con 0x7f88d806c3a0 2024-09-19T08:44:07.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.172+0000 7f88dcfa0640 1 --2- 172.21.15.98:0/3880926545 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f88bc0679e0 0x7f88bc069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:07.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.172+0000 7f88dcfa0640 1 --2- 172.21.15.98:0/3880926545 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f88bc0679e0 0x7f88bc069ea0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f88c8000f70 tx=0x7f88c80079e0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:07.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.172+0000 7f88cdffb640 1 -- 172.21.15.98:0/3880926545 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f88d401f020 con 0x7f88d806c3a0 2024-09-19T08:44:07.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.376+0000 7f88dfa05640 1 -- 172.21.15.98:0/3880926545 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f88d806bc90 con 0x7f88d806c3a0 2024-09-19T08:44:07.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.376+0000 7f88cdffb640 1 -- 172.21.15.98:0/3880926545 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v60) ==== 53+0+73 (secure 0 0 0) 0x7f88d400f2e0 con 0x7f88d806c3a0 2024-09-19T08:44:07.377 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T08:44:07.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.376+0000 7f88dfa05640 1 -- 172.21.15.98:0/3880926545 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f88bc0679e0 msgr2=0x7f88bc069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:07.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.376+0000 7f88dfa05640 1 --2- 172.21.15.98:0/3880926545 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f88bc0679e0 0x7f88bc069ea0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f88c8000f70 tx=0x7f88c80079e0 comp rx=0 tx=0).stop 2024-09-19T08:44:07.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.376+0000 7f88dfa05640 1 -- 172.21.15.98:0/3880926545 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88d806c3a0 msgr2=0x7f88d81476b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:07.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.376+0000 7f88dfa05640 1 --2- 172.21.15.98:0/3880926545 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88d806c3a0 0x7f88d81476b0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f88d400b3e0 tx=0x7f88d400b8a0 comp rx=0 tx=0).stop 2024-09-19T08:44:07.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.376+0000 7f88dfa05640 1 -- 172.21.15.98:0/3880926545 shutdown_connections 2024-09-19T08:44:07.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.376+0000 7f88dfa05640 1 --2- 172.21.15.98:0/3880926545 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f88bc0679e0 0x7f88bc069ea0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:07.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.376+0000 7f88dfa05640 1 --2- 172.21.15.98:0/3880926545 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88d815c550 0x7f88d8156900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:07.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.376+0000 7f88dfa05640 1 --2- 172.21.15.98:0/3880926545 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f88d80733b0 0x7f88d8154d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:07.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.376+0000 7f88dfa05640 1 --2- 172.21.15.98:0/3880926545 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f88d806c3a0 0x7f88d81476b0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:07.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.376+0000 7f88dfa05640 1 -- 172.21.15.98:0/3880926545 >> 172.21.15.98:0/3880926545 conn(0x7f88d80b2010 msgr2=0x7f88d815a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:07.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.376+0000 7f88dfa05640 1 -- 172.21.15.98:0/3880926545 shutdown_connections 2024-09-19T08:44:07.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:07.376+0000 7f88dfa05640 1 -- 172.21.15.98:0/3880926545 wait complete. 2024-09-19T08:44:07.396 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T08:44:07.396 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T08:44:07.397 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2024-09-19T08:44:07.397 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-19T08:44:07.397 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-19T08:44:07.397 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T08:44:07.397 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T08:44:07.397 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T08:44:07.397 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T08:44:07.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:07.398 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2024-09-19T08:44:07.405 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T08:44:07.405 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T08:44:07.405 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T08:44:07.405 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:44:07.405 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-09-19T08:44:07.405 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:44:07.405 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:44:07.405 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:44:07.405 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2024-09-19T08:44:07.405 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:44:07.405 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:44:07.406 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2024-09-19T08:44:07.406 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:44:07.406 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:44:07.406 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T08:44:07.406 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:44:07.406 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T08:44:07.406 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 42172sec preferred_lft 42172sec 2024-09-19T08:44:07.406 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::ec4:7aff:fe88:7f59/64 scope link 2024-09-19T08:44:07.406 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:44:07.406 INFO:teuthology.orchestra.run.smithi139.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T08:44:07.406 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 02:42:74:ed:e4:60 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:44:07.406 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T08:44:07.406 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:44:07.407 INFO:teuthology.orchestra.run.smithi139.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T08:44:07.407 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-19T08:44:07.407 INFO:teuthology.orchestra.run.smithi139.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T08:44:07.407 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:44:07.407 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-19T08:44:07.407 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:44:07.407 INFO:teuthology.orchestra.run.smithi139.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:44:07.407 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:44:07.407 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-19T08:44:07.407 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:44:07.407 INFO:teuthology.orchestra.run.smithi139.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:44:07.408 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:44:07.408 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-19T08:44:07.408 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:44:07.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:07.408 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T08:44:07.460 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-09-19T08:44:07.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:07.461 DEBUG:teuthology.orchestra.run.smithi139:> route 2024-09-19T08:44:07.514 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2024-09-19T08:44:07.514 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T08:44:07.515 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T08:44:07.515 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T08:44:07.515 INFO:teuthology.orchestra.run.smithi139.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T08:44:07.515 INFO:teuthology.orchestra.run.smithi139.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T08:44:07.515 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T08:44:07.515 INFO:teuthology.orchestra.run.smithi139.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:44:07.515 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:44:07.515 INFO:teuthology.orchestra.run.smithi139.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:44:07.515 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T08:44:07.515 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:44:07.516 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-09-19T08:44:07.516 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T08:44:07.516 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T08:44:07.516 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T08:44:07.516 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-09-19T08:44:07.602 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:44:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:44:07.632 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:44:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:44:07.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:07.642 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-09-19T08:44:07.647 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-19T08:44:07.647 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-19T08:44:07.647 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T08:44:07.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:07.648 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.0 up 2024-09-19T08:44:07.699 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-19T08:44:07.699 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:44:07.699 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-19T08:44:07.745 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-19T08:44:07.745 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:44:07.745 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-19T08:44:07.792 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:44:07.849 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:44:07.897 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:44:07.897 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:44:07.897 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:44:07.897 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:44:07.897 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:44:07.897 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:44:07.898 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:44:07.898 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:44:07.898 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:44:07.898 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:44:07.898 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:44:07.898 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:44:07.898 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:44:07.898 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:44:07.898 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:44:07.898 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:44:07.898 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:44:07.898 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:44:07.898 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:44:07.898 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:44:07.899 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:44:07.899 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:44:07.899 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:44:07.899 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:44:07.899 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:44:07.899 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:44:07.899 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:44:07.899 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:44:07.899 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:44:07.899 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:44:07.899 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:44:07.899 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:44:07.899 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:44:07.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:07.901 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:44:07.949 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T08:44:07.949 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-19T08:44:07.997 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:44:08.012 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:44:08.061 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:44:08.061 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:44:08.061 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:44:08.061 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:44:08.061 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:44:08.061 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:44:08.061 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:44:08.061 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:44:08.061 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:44:08.062 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:44:08.062 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:44:08.062 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:44:08.062 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:44:08.062 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:44:08.062 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:44:08.062 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:44:08.062 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:44:08.062 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:44:08.062 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:44:08.062 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:44:08.062 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:44:08.062 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:44:08.063 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:44:08.063 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:44:08.063 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:44:08.063 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:44:08.063 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:44:08.063 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:44:08.063 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:44:08.063 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:44:08.063 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:44:08.063 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:44:08.063 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:44:08.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:08.064 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:44:08.076 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:2024-09-19T08:44:08.079+0000 7f5811dc9580 -1 init, newargv = 0x55cea36dd2d0 newargc=15 2024-09-19T08:44:08.076 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[36162]: starting ceph client 2024-09-19T08:44:08.084 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[36162]: starting fuse 2024-09-19T08:44:09.182 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:44:09.198 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:44:09.245 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:44:09.245 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:44:09.246 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:44:09.246 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:44:09.246 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:44:09.246 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:44:09.246 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:44:09.246 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:44:09.246 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:44:09.246 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:44:09.246 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:44:09.246 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:44:09.246 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:44:09.246 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:44:09.247 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:44:09.247 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:44:09.247 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:44:09.247 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:44:09.247 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:44:09.247 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:44:09.247 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:44:09.247 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:44:09.247 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:44:09.247 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:44:09.247 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:44:09.247 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:44:09.247 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:44:09.248 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:44:09.248 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:44:09.248 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:44:09.248 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:44:09.248 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:44:09.248 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:44:09.248 INFO:teuthology.orchestra.run.smithi139.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-19T08:44:09.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:09.249 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:44:09.297 INFO:teuthology.orchestra.run.smithi139.stdout:45 2024-09-19T08:44:09.298 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-19T08:44:09.298 DEBUG:teuthology.orchestra.run.smithi139:> sudo stdin-killer -- python3 -c ' 2024-09-19T08:44:09.298 DEBUG:teuthology.orchestra.run.smithi139:> import glob 2024-09-19T08:44:09.298 DEBUG:teuthology.orchestra.run.smithi139:> import re 2024-09-19T08:44:09.298 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-09-19T08:44:09.298 DEBUG:teuthology.orchestra.run.smithi139:> import subprocess 2024-09-19T08:44:09.298 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:44:09.298 DEBUG:teuthology.orchestra.run.smithi139:> def _find_admin_socket(client_name): 2024-09-19T08:44:09.298 DEBUG:teuthology.orchestra.run.smithi139:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-19T08:44:09.298 DEBUG:teuthology.orchestra.run.smithi139:> files = glob.glob(asok_path) 2024-09-19T08:44:09.299 DEBUG:teuthology.orchestra.run.smithi139:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-19T08:44:09.299 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:44:09.299 DEBUG:teuthology.orchestra.run.smithi139:> # Given a non-glob path, it better be there 2024-09-19T08:44:09.299 DEBUG:teuthology.orchestra.run.smithi139:> if "*" not in asok_path: 2024-09-19T08:44:09.299 DEBUG:teuthology.orchestra.run.smithi139:> assert(len(files) == 1) 2024-09-19T08:44:09.299 DEBUG:teuthology.orchestra.run.smithi139:> return files[0] 2024-09-19T08:44:09.299 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:44:09.299 DEBUG:teuthology.orchestra.run.smithi139:> for f in files: 2024-09-19T08:44:09.299 DEBUG:teuthology.orchestra.run.smithi139:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T08:44:09.299 DEBUG:teuthology.orchestra.run.smithi139:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T08:44:09.299 DEBUG:teuthology.orchestra.run.smithi139:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T08:44:09.299 DEBUG:teuthology.orchestra.run.smithi139:> contents = proc_f.read() 2024-09-19T08:44:09.299 DEBUG:teuthology.orchestra.run.smithi139:> if mountpoint in contents: 2024-09-19T08:44:09.299 DEBUG:teuthology.orchestra.run.smithi139:> return f 2024-09-19T08:44:09.299 DEBUG:teuthology.orchestra.run.smithi139:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T08:44:09.300 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:44:09.300 DEBUG:teuthology.orchestra.run.smithi139:> print(_find_admin_socket("client.1")) 2024-09-19T08:44:09.300 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-09-19T08:44:09.421 INFO:teuthology.orchestra.run.smithi139.stdout:/var/run/ceph/ceph-client.1.36162.asok 2024-09-19T08:44:09.425 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:44:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:44:09.435 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.36162.asok 2024-09-19T08:44:09.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:09.435 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.36162.asok status 2024-09-19T08:44:09.590 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-19T08:44:09.590 INFO:teuthology.orchestra.run.smithi139.stdout: "metadata": { 2024-09-19T08:44:09.591 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T08:44:09.591 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T08:44:09.591 INFO:teuthology.orchestra.run.smithi139.stdout: "entity_id": "1", 2024-09-19T08:44:09.591 INFO:teuthology.orchestra.run.smithi139.stdout: "hostname": "smithi139", 2024-09-19T08:44:09.591 INFO:teuthology.orchestra.run.smithi139.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-19T08:44:09.591 INFO:teuthology.orchestra.run.smithi139.stdout: "pid": "36162", 2024-09-19T08:44:09.591 INFO:teuthology.orchestra.run.smithi139.stdout: "root": "/" 2024-09-19T08:44:09.591 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:44:09.591 INFO:teuthology.orchestra.run.smithi139.stdout: "dentry_count": 0, 2024-09-19T08:44:09.591 INFO:teuthology.orchestra.run.smithi139.stdout: "dentry_pinned_count": 0, 2024-09-19T08:44:09.591 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 5468, 2024-09-19T08:44:09.591 INFO:teuthology.orchestra.run.smithi139.stdout: "inst": { 2024-09-19T08:44:09.591 INFO:teuthology.orchestra.run.smithi139.stdout: "name": { 2024-09-19T08:44:09.592 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "client", 2024-09-19T08:44:09.592 INFO:teuthology.orchestra.run.smithi139.stdout: "num": 5468 2024-09-19T08:44:09.592 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:44:09.592 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": { 2024-09-19T08:44:09.592 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-19T08:44:09.592 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:44:09.592 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3631364868 2024-09-19T08:44:09.592 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-19T08:44:09.592 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:44:09.592 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": { 2024-09-19T08:44:09.592 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-19T08:44:09.592 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:44:09.592 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3631364868 2024-09-19T08:44:09.593 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:44:09.593 INFO:teuthology.orchestra.run.smithi139.stdout: "inst_str": "client.5468 192.168.0.1:0/3631364868", 2024-09-19T08:44:09.593 INFO:teuthology.orchestra.run.smithi139.stdout: "addr_str": "192.168.0.1:0/3631364868", 2024-09-19T08:44:09.593 INFO:teuthology.orchestra.run.smithi139.stdout: "inode_count": 1, 2024-09-19T08:44:09.593 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_epoch": 60, 2024-09-19T08:44:09.593 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_epoch": 104, 2024-09-19T08:44:09.593 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_epoch_barrier": 0, 2024-09-19T08:44:09.593 INFO:teuthology.orchestra.run.smithi139.stdout: "blocklisted": false, 2024-09-19T08:44:09.593 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs" 2024-09-19T08:44:09.593 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-19T08:44:09.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:09.604 DEBUG:teuthology.orchestra.run.smithi139:> stat --file-system '--printf=%T 2024-09-19T08:44:09.604 DEBUG:teuthology.orchestra.run.smithi139:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-19T08:44:09.611 INFO:teuthology.orchestra.run.smithi139.stdout:fuseblk 2024-09-19T08:44:09.611 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-19T08:44:09.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:09.611 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-19T08:44:09.664 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-19T08:44:09.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.796+0000 7f5ddda75640 1 Processor -- start 2024-09-19T08:44:09.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.796+0000 7f5ddda75640 1 -- start start 2024-09-19T08:44:09.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.800+0000 7f5ddda75640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5dd80b7770 0x7f5dd80b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:09.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.800+0000 7f5ddda75640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dd80cfd20 con 0x7f5dd815e720 2024-09-19T08:44:09.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.800+0000 7f5ddda75640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dd80d29c0 con 0x7f5dd80d3a60 2024-09-19T08:44:09.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.800+0000 7f5ddda75640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f5dd80d2ca0 con 0x7f5dd80b7770 2024-09-19T08:44:09.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.800+0000 7f5dd77fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f5dd815e720 0x7f5dd80cf610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42452/0 (socket says 172.21.15.98:42452) 2024-09-19T08:44:09.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.800+0000 7f5dd77fe640 1 -- 172.21.15.98:0/3848235998 learned_addr learned my addr 172.21.15.98:0/3848235998 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:09.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.800+0000 7f5dcffff640 1 --2- 172.21.15.98:0/3848235998 >> v2:172.21.15.139:3300/0 conn(0x7f5dd80b7770 0x7f5dd80b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:09.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.800+0000 7f5dd6ffd640 1 -- 172.21.15.98:0/3848235998 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3431297546 0 0) 0x7f5dd80cfd20 con 0x7f5dd815e720 2024-09-19T08:44:09.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.800+0000 7f5dd6ffd640 1 -- 172.21.15.98:0/3848235998 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5dc4003530 con 0x7f5dd815e720 2024-09-19T08:44:09.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.800+0000 7f5dd6ffd640 1 -- 172.21.15.98:0/3848235998 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 814173598 0 0) 0x7f5dc4003530 con 0x7f5dd815e720 2024-09-19T08:44:09.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.800+0000 7f5dd6ffd640 1 -- 172.21.15.98:0/3848235998 >> v2:172.21.15.139:3300/0 conn(0x7f5dd80b7770 msgr2=0x7f5dd80b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:09.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.800+0000 7f5dd6ffd640 1 --2- 172.21.15.98:0/3848235998 >> v2:172.21.15.139:3300/0 conn(0x7f5dd80b7770 0x7f5dd80b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:09.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.800+0000 7f5dd6ffd640 1 -- 172.21.15.98:0/3848235998 >> v1:172.21.15.139:6789/0 conn(0x7f5dd80d3a60 legacy=0x7f5dd81623b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:09.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.800+0000 7f5dd6ffd640 1 -- 172.21.15.98:0/3848235998 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dd80d2f80 con 0x7f5dd815e720 2024-09-19T08:44:09.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.800+0000 7f5dd6ffd640 1 -- 172.21.15.98:0/3848235998 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f5dc8001680 con 0x7f5dd815e720 2024-09-19T08:44:09.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.800+0000 7f5dd6ffd640 1 -- 172.21.15.98:0/3848235998 >> v1:172.21.15.98:6789/0 conn(0x7f5dd815e720 legacy=0x7f5dd80cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:09.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.800+0000 7f5dd6ffd640 1 --2- 172.21.15.98:0/3848235998 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5dc40039c0 0x7f5dc4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:09.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.800+0000 7f5dd6ffd640 1 --2- 172.21.15.98:0/3848235998 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5dc4004810 0x7f5dc4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:09.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.800+0000 7f5dd6ffd640 1 --2- 172.21.15.98:0/3848235998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5dc40051d0 0x7f5dc400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:09.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.800+0000 7f5dd6ffd640 1 -- 172.21.15.98:0/3848235998 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd80d2ca0 con 0x7f5dc40039c0 2024-09-19T08:44:09.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.800+0000 7f5dd6ffd640 1 -- 172.21.15.98:0/3848235998 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc400c120 con 0x7f5dc40051d0 2024-09-19T08:44:09.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.800+0000 7f5dd6ffd640 1 -- 172.21.15.98:0/3848235998 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5dc400c320 con 0x7f5dc4004810 2024-09-19T08:44:09.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.800+0000 7f5dd77fe640 1 --2- 172.21.15.98:0/3848235998 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5dc40039c0 0x7f5dc4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:09.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.800+0000 7f5dcffff640 1 --2- 172.21.15.98:0/3848235998 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5dc4004810 0x7f5dc4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:09.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.800+0000 7f5dd7fff640 1 --2- 172.21.15.98:0/3848235998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5dc40051d0 0x7f5dc400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:09.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.800+0000 7f5dd77fe640 1 -- 172.21.15.98:0/3848235998 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5dc4004810 msgr2=0x7f5dc4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:09.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.800+0000 7f5dd77fe640 1 --2- 172.21.15.98:0/3848235998 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5dc4004810 0x7f5dc4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:09.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.800+0000 7f5dd77fe640 1 -- 172.21.15.98:0/3848235998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5dc40051d0 msgr2=0x7f5dc400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:09.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.800+0000 7f5dd77fe640 1 --2- 172.21.15.98:0/3848235998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5dc40051d0 0x7f5dc400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:09.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.800+0000 7f5dd77fe640 1 -- 172.21.15.98:0/3848235998 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc4003530 con 0x7f5dc40039c0 2024-09-19T08:44:09.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.800+0000 7f5dd7fff640 1 --2- 172.21.15.98:0/3848235998 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5dc40051d0 0x7f5dc400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:09.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.800+0000 7f5dcffff640 1 --2- 172.21.15.98:0/3848235998 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5dc4004810 0x7f5dc4004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:09.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.800+0000 7f5dd7fff640 1 -- 172.21.15.98:0/3848235998 reap_dead start 2024-09-19T08:44:09.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.804+0000 7f5dd77fe640 1 --2- 172.21.15.98:0/3848235998 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5dc40039c0 0x7f5dc4003de0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f5dc800c210 tx=0x7f5dc801cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2846c7cd5cb454de server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:09.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.804+0000 7f5dd6ffd640 1 -- 172.21.15.98:0/3848235998 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc8007d40 con 0x7f5dc40039c0 2024-09-19T08:44:09.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.804+0000 7f5dd6ffd640 1 -- 172.21.15.98:0/3848235998 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dc80a9070 con 0x7f5dc40039c0 2024-09-19T08:44:09.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.804+0000 7f5dd6ffd640 1 -- 172.21.15.98:0/3848235998 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc80a5df0 con 0x7f5dc40039c0 2024-09-19T08:44:09.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.804+0000 7f5ddda75640 1 -- 172.21.15.98:0/3848235998 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5dc40039c0 msgr2=0x7f5dc4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:09.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.804+0000 7f5ddda75640 1 --2- 172.21.15.98:0/3848235998 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5dc40039c0 0x7f5dc4003de0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f5dc800c210 tx=0x7f5dc801cc70 comp rx=0 tx=0).stop 2024-09-19T08:44:09.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.804+0000 7f5ddda75640 1 -- 172.21.15.98:0/3848235998 shutdown_connections 2024-09-19T08:44:09.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.804+0000 7f5ddda75640 1 --2- 172.21.15.98:0/3848235998 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5dc40039c0 0x7f5dc4003de0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:09.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.804+0000 7f5ddda75640 1 -- 172.21.15.98:0/3848235998 >> 172.21.15.98:0/3848235998 conn(0x7f5dd80d4640 msgr2=0x7f5dd80d4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:09.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.804+0000 7f5ddda75640 1 -- 172.21.15.98:0/3848235998 shutdown_connections 2024-09-19T08:44:09.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.804+0000 7f5ddda75640 1 -- 172.21.15.98:0/3848235998 wait complete. 2024-09-19T08:44:09.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.804+0000 7f5ddda75640 1 Processor -- start 2024-09-19T08:44:09.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.804+0000 7f5ddda75640 1 -- start start 2024-09-19T08:44:09.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.804+0000 7f5ddda75640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5dc40039c0 0x7f5dd80c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:09.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.804+0000 7f5ddda75640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5dd80cb710 0x7f5dd81f7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:09.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.804+0000 7f5ddda75640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5dd80cd2f0 0x7f5dd80cd770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:09.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.804+0000 7f5ddda75640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd8162bc0 con 0x7f5dc40039c0 2024-09-19T08:44:09.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.804+0000 7f5ddda75640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd80b7f60 con 0x7f5dd80cd2f0 2024-09-19T08:44:09.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.804+0000 7f5ddda75640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5dd8162d40 con 0x7f5dd80cb710 2024-09-19T08:44:09.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.804+0000 7f5dd77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5dc40039c0 0x7f5dd80c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:09.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.804+0000 7f5dd77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5dc40039c0 0x7f5dd80c86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39748/0 (socket says 172.21.15.98:39748) 2024-09-19T08:44:09.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.808+0000 7f5dd77fe640 1 -- 172.21.15.98:0/3378020549 learned_addr learned my addr 172.21.15.98:0/3378020549 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:09.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.808+0000 7f5dcffff640 1 --2- 172.21.15.98:0/3378020549 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5dd80cb710 0x7f5dd81f7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:09.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.808+0000 7f5dd7fff640 1 --2- 172.21.15.98:0/3378020549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5dd80cd2f0 0x7f5dd80cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:09.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.808+0000 7f5dd77fe640 1 -- 172.21.15.98:0/3378020549 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5dd80cb710 msgr2=0x7f5dd81f7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:09.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.808+0000 7f5dd77fe640 1 --2- 172.21.15.98:0/3378020549 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5dd80cb710 0x7f5dd81f7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:09.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.808+0000 7f5dd77fe640 1 -- 172.21.15.98:0/3378020549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5dd80cd2f0 msgr2=0x7f5dd80cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:09.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.808+0000 7f5dd77fe640 1 --2- 172.21.15.98:0/3378020549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5dd80cd2f0 0x7f5dd80cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:09.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.808+0000 7f5dd77fe640 1 -- 172.21.15.98:0/3378020549 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dd8160390 con 0x7f5dc40039c0 2024-09-19T08:44:09.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.808+0000 7f5dd7fff640 1 --2- 172.21.15.98:0/3378020549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5dd80cd2f0 0x7f5dd80cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:09.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.808+0000 7f5dd77fe640 1 --2- 172.21.15.98:0/3378020549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5dc40039c0 0x7f5dd80c86b0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f5dc801caa0 tx=0x7f5dc4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:09.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.808+0000 7f5dd4ff9640 1 -- 172.21.15.98:0/3378020549 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc80a5950 con 0x7f5dc40039c0 2024-09-19T08:44:09.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.808+0000 7f5dd4ff9640 1 -- 172.21.15.98:0/3378020549 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dc80a5af0 con 0x7f5dc40039c0 2024-09-19T08:44:09.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.808+0000 7f5ddda75640 1 -- 172.21.15.98:0/3378020549 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dd8160670 con 0x7f5dc40039c0 2024-09-19T08:44:09.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.808+0000 7f5dd4ff9640 1 -- 172.21.15.98:0/3378020549 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc80047f0 con 0x7f5dc40039c0 2024-09-19T08:44:09.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.808+0000 7f5ddda75640 1 -- 172.21.15.98:0/3378020549 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dd80cbb70 con 0x7f5dc40039c0 2024-09-19T08:44:09.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.808+0000 7f5ddda75640 1 -- 172.21.15.98:0/3378020549 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dd81630a0 con 0x7f5dc40039c0 2024-09-19T08:44:09.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.808+0000 7f5dd4ff9640 1 -- 172.21.15.98:0/3378020549 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f5dc8004990 con 0x7f5dc40039c0 2024-09-19T08:44:09.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.808+0000 7f5dd4ff9640 1 --2- 172.21.15.98:0/3378020549 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5db8067a10 0x7f5db8069ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:09.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.808+0000 7f5dd4ff9640 1 -- 172.21.15.98:0/3378020549 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f5dc811f530 con 0x7f5dc40039c0 2024-09-19T08:44:09.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.808+0000 7f5dcffff640 1 --2- 172.21.15.98:0/3378020549 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5db8067a10 0x7f5db8069ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:09.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.812+0000 7f5dcffff640 1 --2- 172.21.15.98:0/3378020549 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5db8067a10 0x7f5db8069ed0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8161be0 tx=0x7f5dc0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:09.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:09.812+0000 7f5dd4ff9640 1 -- 172.21.15.98:0/3378020549 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f5dc80169b0 con 0x7f5dc40039c0 2024-09-19T08:44:10.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.004+0000 7f5ddda75640 1 -- 172.21.15.98:0/3378020549 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f5dd80cf1e0 con 0x7f5dc40039c0 2024-09-19T08:44:10.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.288+0000 7f5dd4ff9640 1 -- 172.21.15.98:0/3378020549 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v61) ==== 136+0+0 (secure 0 0 0) 0x7f5dc80b7320 con 0x7f5dc40039c0 2024-09-19T08:44:10.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.288+0000 7f5ddda75640 1 -- 172.21.15.98:0/3378020549 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5db8067a10 msgr2=0x7f5db8069ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:10.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.288+0000 7f5ddda75640 1 --2- 172.21.15.98:0/3378020549 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5db8067a10 0x7f5db8069ed0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8161be0 tx=0x7f5dc0008040 comp rx=0 tx=0).stop 2024-09-19T08:44:10.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.288+0000 7f5ddda75640 1 -- 172.21.15.98:0/3378020549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5dc40039c0 msgr2=0x7f5dd80c86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:10.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.288+0000 7f5ddda75640 1 --2- 172.21.15.98:0/3378020549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5dc40039c0 0x7f5dd80c86b0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f5dc801caa0 tx=0x7f5dc4005840 comp rx=0 tx=0).stop 2024-09-19T08:44:10.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.288+0000 7f5ddda75640 1 -- 172.21.15.98:0/3378020549 shutdown_connections 2024-09-19T08:44:10.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.288+0000 7f5ddda75640 1 --2- 172.21.15.98:0/3378020549 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5db8067a10 0x7f5db8069ed0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:10.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.288+0000 7f5ddda75640 1 --2- 172.21.15.98:0/3378020549 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5dd80cd2f0 0x7f5dd80cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:10.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.288+0000 7f5ddda75640 1 --2- 172.21.15.98:0/3378020549 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5dd80cb710 0x7f5dd81f7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:10.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.288+0000 7f5ddda75640 1 --2- 172.21.15.98:0/3378020549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5dc40039c0 0x7f5dd80c86b0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:10.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.288+0000 7f5ddda75640 1 -- 172.21.15.98:0/3378020549 >> 172.21.15.98:0/3378020549 conn(0x7f5dd80d4640 msgr2=0x7f5dd815d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:10.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.288+0000 7f5ddda75640 1 -- 172.21.15.98:0/3378020549 shutdown_connections 2024-09-19T08:44:10.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.288+0000 7f5ddda75640 1 -- 172.21.15.98:0/3378020549 wait complete. 2024-09-19T08:44:10.311 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-19T08:44:10.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.432+0000 7fb556c8d640 1 Processor -- start 2024-09-19T08:44:10.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.432+0000 7fb556c8d640 1 -- start start 2024-09-19T08:44:10.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.432+0000 7fb556c8d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb55006d880 0x7fb5500b3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:10.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.432+0000 7fb556c8d640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb55015ea10 con 0x7fb550140d30 2024-09-19T08:44:10.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.432+0000 7fb556c8d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb55015ecf0 con 0x7fb550141110 2024-09-19T08:44:10.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.432+0000 7fb556c8d640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fb55015e7c0 con 0x7fb55006d880 2024-09-19T08:44:10.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.432+0000 7fb54ffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb55006d880 0x7fb5500b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:10.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.432+0000 7fb554a29640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fb550140d30 0x7fb55006d170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42468/0 (socket says 172.21.15.98:42468) 2024-09-19T08:44:10.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.432+0000 7fb554a29640 1 -- 172.21.15.98:0/1419886886 learned_addr learned my addr 172.21.15.98:0/1419886886 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:10.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.432+0000 7fb54ffff640 1 -- 172.21.15.98:0/1419886886 >> v1:172.21.15.139:6789/0 conn(0x7fb550141110 legacy=0x7fb55015b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:10.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.432+0000 7fb54ffff640 1 -- 172.21.15.98:0/1419886886 >> v1:172.21.15.98:6789/0 conn(0x7fb550140d30 legacy=0x7fb55006d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:10.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.432+0000 7fb54ffff640 1 -- 172.21.15.98:0/1419886886 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb55015efd0 con 0x7fb55006d880 2024-09-19T08:44:10.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.432+0000 7fb54f7fe640 1 -- 172.21.15.98:0/1419886886 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3335292717 0 0) 0x7fb55015ea10 con 0x7fb550140d30 2024-09-19T08:44:10.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.436+0000 7fb54ffff640 1 --2- 172.21.15.98:0/1419886886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb55006d880 0x7fb5500b3310 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fb55015e940 tx=0x7fb54401e8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a8acd1933180ed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:10.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.436+0000 7fb54f7fe640 1 -- 172.21.15.98:0/1419886886 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5440ad070 con 0x7fb55006d880 2024-09-19T08:44:10.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.436+0000 7fb54f7fe640 1 -- 172.21.15.98:0/1419886886 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5440a7960 con 0x7fb55006d880 2024-09-19T08:44:10.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.436+0000 7fb54f7fe640 1 -- 172.21.15.98:0/1419886886 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5440a7d30 con 0x7fb55006d880 2024-09-19T08:44:10.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.436+0000 7fb556c8d640 1 -- 172.21.15.98:0/1419886886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb55006d880 msgr2=0x7fb5500b3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:10.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.436+0000 7fb556c8d640 1 --2- 172.21.15.98:0/1419886886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb55006d880 0x7fb5500b3310 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fb55015e940 tx=0x7fb54401e8f0 comp rx=0 tx=0).stop 2024-09-19T08:44:10.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.436+0000 7fb556c8d640 1 -- 172.21.15.98:0/1419886886 shutdown_connections 2024-09-19T08:44:10.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.436+0000 7fb556c8d640 1 --2- 172.21.15.98:0/1419886886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb55006d880 0x7fb5500b3310 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:10.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.436+0000 7fb556c8d640 1 -- 172.21.15.98:0/1419886886 >> 172.21.15.98:0/1419886886 conn(0x7fb5500b2010 msgr2=0x7fb5500b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:10.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.436+0000 7fb556c8d640 1 -- 172.21.15.98:0/1419886886 shutdown_connections 2024-09-19T08:44:10.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.436+0000 7fb556c8d640 1 -- 172.21.15.98:0/1419886886 wait complete. 2024-09-19T08:44:10.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.436+0000 7fb556c8d640 1 Processor -- start 2024-09-19T08:44:10.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.436+0000 7fb556c8d640 1 -- start start 2024-09-19T08:44:10.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.436+0000 7fb556c8d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb55006d880 0x7fb5500ab120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:10.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.436+0000 7fb556c8d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb550140d30 0x7fb550073a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:10.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.436+0000 7fb556c8d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb550141110 0x7fb5500ce460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:10.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.436+0000 7fb556c8d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb55015ba80 con 0x7fb550141110 2024-09-19T08:44:10.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.436+0000 7fb556c8d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb55015b900 con 0x7fb550140d30 2024-09-19T08:44:10.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.436+0000 7fb556c8d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb55015bc00 con 0x7fb55006d880 2024-09-19T08:44:10.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.436+0000 7fb55522a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb550141110 0x7fb5500ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:10.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.436+0000 7fb55522a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb550141110 0x7fb5500ce460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39754/0 (socket says 172.21.15.98:39754) 2024-09-19T08:44:10.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.436+0000 7fb55522a640 1 -- 172.21.15.98:0/201189028 learned_addr learned my addr 172.21.15.98:0/201189028 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:10.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.440+0000 7fb54ffff640 1 --2- 172.21.15.98:0/201189028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb550140d30 0x7fb550073a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:10.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.440+0000 7fb554a29640 1 --2- 172.21.15.98:0/201189028 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb55006d880 0x7fb5500ab120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:10.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.440+0000 7fb55522a640 1 -- 172.21.15.98:0/201189028 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb55006d880 msgr2=0x7fb5500ab120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:10.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.440+0000 7fb55522a640 1 --2- 172.21.15.98:0/201189028 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb55006d880 0x7fb5500ab120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:10.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.440+0000 7fb55522a640 1 -- 172.21.15.98:0/201189028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb550140d30 msgr2=0x7fb550073a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:10.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.440+0000 7fb55522a640 1 --2- 172.21.15.98:0/201189028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb550140d30 0x7fb550073a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:10.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.440+0000 7fb55522a640 1 -- 172.21.15.98:0/201189028 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb550165360 con 0x7fb550141110 2024-09-19T08:44:10.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.440+0000 7fb554a29640 1 --2- 172.21.15.98:0/201189028 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb55006d880 0x7fb5500ab120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:10.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.440+0000 7fb54ffff640 1 --2- 172.21.15.98:0/201189028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb550140d30 0x7fb550073a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:10.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.440+0000 7fb55522a640 1 --2- 172.21.15.98:0/201189028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb550141110 0x7fb5500ce460 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fb54800b9a0 tx=0x7fb54800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:10.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.440+0000 7fb54d7fa640 1 -- 172.21.15.98:0/201189028 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb548019070 con 0x7fb550141110 2024-09-19T08:44:10.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.440+0000 7fb54d7fa640 1 -- 172.21.15.98:0/201189028 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb54800cba0 con 0x7fb550141110 2024-09-19T08:44:10.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.440+0000 7fb556c8d640 1 -- 172.21.15.98:0/201189028 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5500aa640 con 0x7fb550141110 2024-09-19T08:44:10.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.440+0000 7fb54d7fa640 1 -- 172.21.15.98:0/201189028 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb548014440 con 0x7fb550141110 2024-09-19T08:44:10.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.440+0000 7fb556c8d640 1 -- 172.21.15.98:0/201189028 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5500aac00 con 0x7fb550141110 2024-09-19T08:44:10.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.440+0000 7fb556c8d640 1 -- 172.21.15.98:0/201189028 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb55015bfd0 con 0x7fb550141110 2024-09-19T08:44:10.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.440+0000 7fb54d7fa640 1 -- 172.21.15.98:0/201189028 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7fb54801c070 con 0x7fb550141110 2024-09-19T08:44:10.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.440+0000 7fb54d7fa640 1 --2- 172.21.15.98:0/201189028 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb53c0679e0 0x7fb53c069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:10.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.440+0000 7fb54d7fa640 1 -- 172.21.15.98:0/201189028 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7fb548085ab0 con 0x7fb550141110 2024-09-19T08:44:10.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.440+0000 7fb554a29640 1 --2- 172.21.15.98:0/201189028 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb53c0679e0 0x7fb53c069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:10.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.440+0000 7fb554a29640 1 --2- 172.21.15.98:0/201189028 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb53c0679e0 0x7fb53c069ea0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb540000f70 tx=0x7fb540009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:10.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.444+0000 7fb54d7fa640 1 -- 172.21.15.98:0/201189028 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb5480564a0 con 0x7fb550141110 2024-09-19T08:44:10.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.612+0000 7fb556c8d640 1 -- 172.21.15.98:0/201189028 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fb5500b5460 con 0x7fb53c0679e0 2024-09-19T08:44:10.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.616+0000 7fb54d7fa640 1 -- 172.21.15.98:0/201189028 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fb5500b5460 con 0x7fb53c0679e0 2024-09-19T08:44:10.619 INFO:teuthology.orchestra.run.smithi098.stdout:[ 2024-09-19T08:44:10.619 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:44:10.619 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "cephfs" 2024-09-19T08:44:10.619 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:44:10.619 INFO:teuthology.orchestra.run.smithi098.stdout:] 2024-09-19T08:44:10.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.620+0000 7fb556c8d640 1 -- 172.21.15.98:0/201189028 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb53c0679e0 msgr2=0x7fb53c069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:10.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.620+0000 7fb556c8d640 1 --2- 172.21.15.98:0/201189028 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb53c0679e0 0x7fb53c069ea0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb540000f70 tx=0x7fb540009040 comp rx=0 tx=0).stop 2024-09-19T08:44:10.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.620+0000 7fb556c8d640 1 -- 172.21.15.98:0/201189028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb550141110 msgr2=0x7fb5500ce460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:10.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.620+0000 7fb556c8d640 1 --2- 172.21.15.98:0/201189028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb550141110 0x7fb5500ce460 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fb54800b9a0 tx=0x7fb54800be60 comp rx=0 tx=0).stop 2024-09-19T08:44:10.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.620+0000 7fb556c8d640 1 -- 172.21.15.98:0/201189028 shutdown_connections 2024-09-19T08:44:10.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.620+0000 7fb556c8d640 1 --2- 172.21.15.98:0/201189028 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb53c0679e0 0x7fb53c069ea0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:10.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.620+0000 7fb556c8d640 1 --2- 172.21.15.98:0/201189028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb550141110 0x7fb5500ce460 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:10.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.620+0000 7fb556c8d640 1 --2- 172.21.15.98:0/201189028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb550140d30 0x7fb550073a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:10.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.620+0000 7fb556c8d640 1 --2- 172.21.15.98:0/201189028 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb55006d880 0x7fb5500ab120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:10.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.620+0000 7fb556c8d640 1 -- 172.21.15.98:0/201189028 >> 172.21.15.98:0/201189028 conn(0x7fb5500b2010 msgr2=0x7fb550159800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:10.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.620+0000 7fb556c8d640 1 -- 172.21.15.98:0/201189028 shutdown_connections 2024-09-19T08:44:10.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.620+0000 7fb556c8d640 1 -- 172.21.15.98:0/201189028 wait complete. 2024-09-19T08:44:10.637 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T08:44:10.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.744+0000 7f406a495640 1 Processor -- start 2024-09-19T08:44:10.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.744+0000 7f406a495640 1 -- start start 2024-09-19T08:44:10.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.744+0000 7f406a495640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4064145b70 0x7f406406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:10.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.744+0000 7f406a495640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4064071ac0 0x7f40640b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:10.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.744+0000 7f406a495640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4064073530 con 0x7f406406c1d0 2024-09-19T08:44:10.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.744+0000 7f406a495640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f406406c620 con 0x7f4064145b70 2024-09-19T08:44:10.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.744+0000 7f406a495640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4064073810 con 0x7f4064071ac0 2024-09-19T08:44:10.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.744+0000 7f405b7fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f406406c1d0 0x7f40640713b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42478/0 (socket says 172.21.15.98:42478) 2024-09-19T08:44:10.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.744+0000 7f405b7fe640 1 -- 172.21.15.98:0/246258959 learned_addr learned my addr 172.21.15.98:0/246258959 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:10.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.744+0000 7f4063fff640 1 --2- 172.21.15.98:0/246258959 >> v2:172.21.15.139:3300/0 conn(0x7f4064145b70 0x7f406406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:10.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.744+0000 7f4068a32640 1 --2- 172.21.15.98:0/246258959 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4064071ac0 0x7f40640b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:10.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.744+0000 7f40637fe640 1 -- 172.21.15.98:0/246258959 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4106822971 0 0) 0x7f4064073530 con 0x7f406406c1d0 2024-09-19T08:44:10.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.744+0000 7f40637fe640 1 -- 172.21.15.98:0/246258959 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4050003530 con 0x7f406406c1d0 2024-09-19T08:44:10.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.744+0000 7f40637fe640 1 -- 172.21.15.98:0/246258959 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3148281844 0 0) 0x7f4050003530 con 0x7f406406c1d0 2024-09-19T08:44:10.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.744+0000 7f40637fe640 1 -- 172.21.15.98:0/246258959 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4064071ac0 msgr2=0x7f40640b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:10.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.744+0000 7f40637fe640 1 --2- 172.21.15.98:0/246258959 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4064071ac0 0x7f40640b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:10.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.744+0000 7f40637fe640 1 -- 172.21.15.98:0/246258959 >> v2:172.21.15.139:3300/0 conn(0x7f4064145b70 msgr2=0x7f406406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:10.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.744+0000 7f40637fe640 1 --2- 172.21.15.98:0/246258959 >> v2:172.21.15.139:3300/0 conn(0x7f4064145b70 0x7f406406bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:10.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.744+0000 7f40637fe640 1 -- 172.21.15.98:0/246258959 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4064073af0 con 0x7f406406c1d0 2024-09-19T08:44:10.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.744+0000 7f40637fe640 1 -- 172.21.15.98:0/246258959 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f404c001680 con 0x7f406406c1d0 2024-09-19T08:44:10.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.744+0000 7f40637fe640 1 -- 172.21.15.98:0/246258959 >> v1:172.21.15.98:6789/0 conn(0x7f406406c1d0 legacy=0x7f40640713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:10.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.744+0000 7f40637fe640 1 --2- 172.21.15.98:0/246258959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40500039c0 0x7f4050003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:10.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.744+0000 7f40637fe640 1 --2- 172.21.15.98:0/246258959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4050004810 0x7f4050004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:10.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.744+0000 7f40637fe640 1 --2- 172.21.15.98:0/246258959 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4064071ac0 0x7f40500051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:10.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.744+0000 7f40637fe640 1 -- 172.21.15.98:0/246258959 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f406406c620 con 0x7f4050004810 2024-09-19T08:44:10.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.744+0000 7f40637fe640 1 -- 172.21.15.98:0/246258959 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4064073810 con 0x7f40500039c0 2024-09-19T08:44:10.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.744+0000 7f40637fe640 1 -- 172.21.15.98:0/246258959 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f405000b9d0 con 0x7f4064071ac0 2024-09-19T08:44:10.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.744+0000 7f405b7fe640 1 --2- 172.21.15.98:0/246258959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4050004810 0x7f4050004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:10.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.744+0000 7f4063fff640 1 --2- 172.21.15.98:0/246258959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40500039c0 0x7f4050003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:10.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.744+0000 7f4068a32640 1 --2- 172.21.15.98:0/246258959 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4064071ac0 0x7f40500051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:10.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.744+0000 7f405b7fe640 1 -- 172.21.15.98:0/246258959 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4064071ac0 msgr2=0x7f40500051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:10.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.744+0000 7f405b7fe640 1 --2- 172.21.15.98:0/246258959 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4064071ac0 0x7f40500051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:10.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.744+0000 7f405b7fe640 1 -- 172.21.15.98:0/246258959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40500039c0 msgr2=0x7f4050003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:10.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.744+0000 7f405b7fe640 1 --2- 172.21.15.98:0/246258959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40500039c0 0x7f4050003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:10.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.744+0000 7f405b7fe640 1 -- 172.21.15.98:0/246258959 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4050003530 con 0x7f4050004810 2024-09-19T08:44:10.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.744+0000 7f4063fff640 1 --2- 172.21.15.98:0/246258959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40500039c0 0x7f4050003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:10.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.748+0000 7f405b7fe640 1 --2- 172.21.15.98:0/246258959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4050004810 0x7f4050004c90 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f404c00c240 tx=0x7f404c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749fb967f20d8fef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:10.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.748+0000 7f40637fe640 1 -- 172.21.15.98:0/246258959 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c007d60 con 0x7f4050004810 2024-09-19T08:44:10.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.748+0000 7f40637fe640 1 -- 172.21.15.98:0/246258959 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f404c0a9020 con 0x7f4050004810 2024-09-19T08:44:10.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.748+0000 7f40637fe640 1 -- 172.21.15.98:0/246258959 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c005b20 con 0x7f4050004810 2024-09-19T08:44:10.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.748+0000 7f406a495640 1 -- 172.21.15.98:0/246258959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4050004810 msgr2=0x7f4050004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:10.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.748+0000 7f406a495640 1 --2- 172.21.15.98:0/246258959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4050004810 0x7f4050004c90 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f404c00c240 tx=0x7f404c01cc80 comp rx=0 tx=0).stop 2024-09-19T08:44:10.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.748+0000 7f4068a32640 1 -- 172.21.15.98:0/246258959 reap_dead start 2024-09-19T08:44:10.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.748+0000 7f406a495640 1 -- 172.21.15.98:0/246258959 shutdown_connections 2024-09-19T08:44:10.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.748+0000 7f406a495640 1 -- 172.21.15.98:0/246258959 >> 172.21.15.98:0/246258959 conn(0x7f40640b2010 msgr2=0x7f40640b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:10.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.748+0000 7f406a495640 1 -- 172.21.15.98:0/246258959 shutdown_connections 2024-09-19T08:44:10.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.748+0000 7f406a495640 1 -- 172.21.15.98:0/246258959 wait complete. 2024-09-19T08:44:10.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.748+0000 7f406a495640 1 Processor -- start 2024-09-19T08:44:10.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.748+0000 7f406a495640 1 -- start start 2024-09-19T08:44:10.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.752+0000 7f406a495640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4050004810 0x7f40641f74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:10.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.752+0000 7f406a495640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40641f8070 0x7f4064135d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:10.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.752+0000 7f406a495640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40640ae8b0 0x7f40640aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:10.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.752+0000 7f406a495640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f40640b3fb0 con 0x7f40641f8070 2024-09-19T08:44:10.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.752+0000 7f406a495640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f40640b3e30 con 0x7f40640ae8b0 2024-09-19T08:44:10.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.752+0000 7f406a495640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f40640b4130 con 0x7f4050004810 2024-09-19T08:44:10.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.752+0000 7f405b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40641f8070 0x7f4064135d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:10.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.752+0000 7f405b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40641f8070 0x7f4064135d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39768/0 (socket says 172.21.15.98:39768) 2024-09-19T08:44:10.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.752+0000 7f405b7fe640 1 -- 172.21.15.98:0/3444464703 learned_addr learned my addr 172.21.15.98:0/3444464703 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:10.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.752+0000 7f4063fff640 1 --2- 172.21.15.98:0/3444464703 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4050004810 0x7f40641f74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:10.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.752+0000 7f4068a32640 1 --2- 172.21.15.98:0/3444464703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40640ae8b0 0x7f40640aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:10.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.752+0000 7f405b7fe640 1 -- 172.21.15.98:0/3444464703 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4050004810 msgr2=0x7f40641f74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:10.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.752+0000 7f405b7fe640 1 --2- 172.21.15.98:0/3444464703 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4050004810 0x7f40641f74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:10.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.752+0000 7f405b7fe640 1 -- 172.21.15.98:0/3444464703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40640ae8b0 msgr2=0x7f40640aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:10.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.752+0000 7f405b7fe640 1 --2- 172.21.15.98:0/3444464703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40640ae8b0 0x7f40640aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:10.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.752+0000 7f405b7fe640 1 -- 172.21.15.98:0/3444464703 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4064136640 con 0x7f40641f8070 2024-09-19T08:44:10.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.752+0000 7f4068a32640 1 --2- 172.21.15.98:0/3444464703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40640ae8b0 0x7f40640aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:10.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.752+0000 7f405b7fe640 1 --2- 172.21.15.98:0/3444464703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40641f8070 0x7f4064135d60 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f404c00c240 tx=0x7f4050005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:10.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.752+0000 7f40617fa640 1 -- 172.21.15.98:0/3444464703 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c005d00 con 0x7f40641f8070 2024-09-19T08:44:10.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.752+0000 7f40617fa640 1 -- 172.21.15.98:0/3444464703 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f404c0047f0 con 0x7f40641f8070 2024-09-19T08:44:10.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.752+0000 7f406a495640 1 -- 172.21.15.98:0/3444464703 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4064136920 con 0x7f40641f8070 2024-09-19T08:44:10.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.752+0000 7f40617fa640 1 -- 172.21.15.98:0/3444464703 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404c004bc0 con 0x7f40641f8070 2024-09-19T08:44:10.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.752+0000 7f406a495640 1 -- 172.21.15.98:0/3444464703 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4064136ee0 con 0x7f40641f8070 2024-09-19T08:44:10.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.752+0000 7f406a495640 1 -- 172.21.15.98:0/3444464703 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40640b4510 con 0x7f40641f8070 2024-09-19T08:44:10.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.752+0000 7f40617fa640 1 -- 172.21.15.98:0/3444464703 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f404c0070d0 con 0x7f40641f8070 2024-09-19T08:44:10.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.752+0000 7f40617fa640 1 --2- 172.21.15.98:0/3444464703 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f40440679e0 0x7f4044069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:10.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.752+0000 7f40617fa640 1 -- 172.21.15.98:0/3444464703 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f404c11f220 con 0x7f40641f8070 2024-09-19T08:44:10.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.752+0000 7f4063fff640 1 --2- 172.21.15.98:0/3444464703 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f40440679e0 0x7f4044069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:10.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.756+0000 7f4063fff640 1 --2- 172.21.15.98:0/3444464703 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f40440679e0 0x7f4044069ea0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4054009a60 tx=0x7f4054008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:10.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.756+0000 7f40617fa640 1 -- 172.21.15.98:0/3444464703 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f404c016990 con 0x7f40641f8070 2024-09-19T08:44:10.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.920+0000 7f406a495640 1 -- 172.21.15.98:0/3444464703 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f406413d290 con 0x7f40641f8070 2024-09-19T08:44:10.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.920+0000 7f40617fa640 1 -- 172.21.15.98:0/3444464703 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7f404c0efc10 con 0x7f40641f8070 2024-09-19T08:44:10.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.920+0000 7f406a495640 1 -- 172.21.15.98:0/3444464703 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f40440679e0 msgr2=0x7f4044069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:10.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.920+0000 7f406a495640 1 --2- 172.21.15.98:0/3444464703 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f40440679e0 0x7f4044069ea0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4054009a60 tx=0x7f4054008040 comp rx=0 tx=0).stop 2024-09-19T08:44:10.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.920+0000 7f406a495640 1 -- 172.21.15.98:0/3444464703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40641f8070 msgr2=0x7f4064135d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:10.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.920+0000 7f406a495640 1 --2- 172.21.15.98:0/3444464703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40641f8070 0x7f4064135d60 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f404c00c240 tx=0x7f4050005840 comp rx=0 tx=0).stop 2024-09-19T08:44:10.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.920+0000 7f406a495640 1 -- 172.21.15.98:0/3444464703 shutdown_connections 2024-09-19T08:44:10.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.920+0000 7f406a495640 1 --2- 172.21.15.98:0/3444464703 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f40440679e0 0x7f4044069ea0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:10.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.920+0000 7f406a495640 1 --2- 172.21.15.98:0/3444464703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40640ae8b0 0x7f40640aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:10.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.920+0000 7f406a495640 1 --2- 172.21.15.98:0/3444464703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40641f8070 0x7f4064135d60 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:10.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.920+0000 7f406a495640 1 --2- 172.21.15.98:0/3444464703 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4050004810 0x7f40641f74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:10.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.920+0000 7f406a495640 1 -- 172.21.15.98:0/3444464703 >> 172.21.15.98:0/3444464703 conn(0x7f40640b2010 msgr2=0x7f40640720e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:10.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.920+0000 7f406a495640 1 -- 172.21.15.98:0/3444464703 shutdown_connections 2024-09-19T08:44:10.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:10.920+0000 7f406a495640 1 -- 172.21.15.98:0/3444464703 wait complete. 2024-09-19T08:44:10.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:10.939 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a) 2024-09-19T08:44:10.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:10.965 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/fake) 2024-09-19T08:44:11.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:11.030 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-19T08:44:11.082 INFO:teuthology.orchestra.run.smithi098.stdout:cephfs_data 2024-09-19T08:44:11.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:11.082 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a) 2024-09-19T08:44:11.139 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:44:11.139 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol41500/.meta 2024-09-19T08:44:11.197 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol41500 2024-09-19T08:44:11.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.368+0000 7f0c47724640 1 Processor -- start 2024-09-19T08:44:11.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.368+0000 7f0c47724640 1 -- start start 2024-09-19T08:44:11.872 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:44:11.575+0000 7ff73a69c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:44:11.873 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:44:11.575+0000 7ff73a69c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:44:11.873 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:44:11.575+0000 7ff73a69c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:44:11.873 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:44:11.575+0000 7ff73a69c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:44:11.873 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:44:11.575+0000 7ff73a69c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:44:11.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.372+0000 7f0c47724640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0c4006c3a0 0x7f0c400713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:11.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.372+0000 7f0c47724640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0c400718f0 0x7f0c400b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:11.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.372+0000 7f0c47724640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c40073530 con 0x7f0c40145b70 2024-09-19T08:44:11.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.372+0000 7f0c47724640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f0c40073810 con 0x7f0c4006c3a0 2024-09-19T08:44:11.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.372+0000 7f0c47724640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f0c40073af0 con 0x7f0c400718f0 2024-09-19T08:44:11.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.372+0000 7f0c44cbf640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0c4006c3a0 0x7f0c400713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:11.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.372+0000 7f0c44cbf640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0c4006c3a0 0x7f0c400713b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39778/0 (socket says 172.21.15.98:39778) 2024-09-19T08:44:11.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.372+0000 7f0c44cbf640 1 -- 172.21.15.98:0/700537042 learned_addr learned my addr 172.21.15.98:0/700537042 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:11.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.372+0000 7f0c45cc1640 1 --2- 172.21.15.98:0/700537042 >> v2:172.21.15.139:3300/0 conn(0x7f0c400718f0 0x7f0c400b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:11.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.372+0000 7f0c44cbf640 1 -- 172.21.15.98:0/700537042 >> v2:172.21.15.139:3300/0 conn(0x7f0c400718f0 msgr2=0x7f0c400b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:11.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.372+0000 7f0c44cbf640 1 --2- 172.21.15.98:0/700537042 >> v2:172.21.15.139:3300/0 conn(0x7f0c400718f0 0x7f0c400b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:11.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.372+0000 7f0c44cbf640 1 -- 172.21.15.98:0/700537042 >> v1:172.21.15.139:6789/0 conn(0x7f0c40145b70 legacy=0x7f0c4006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:11.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.372+0000 7f0c44cbf640 1 -- 172.21.15.98:0/700537042 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c40073dd0 con 0x7f0c4006c3a0 2024-09-19T08:44:11.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.376+0000 7f0c44cbf640 1 --2- 172.21.15.98:0/700537042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c4006c3a0 0x7f0c400713b0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f0c40072e60 tx=0x7f0c3001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15667f15a2a91451 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:11.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.376+0000 7f0c37fff640 1 -- 172.21.15.98:0/700537042 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c300b0070 con 0x7f0c4006c3a0 2024-09-19T08:44:11.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.376+0000 7f0c37fff640 1 -- 172.21.15.98:0/700537042 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c300ab600 con 0x7f0c4006c3a0 2024-09-19T08:44:11.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.376+0000 7f0c37fff640 1 -- 172.21.15.98:0/700537042 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c300ab9d0 con 0x7f0c4006c3a0 2024-09-19T08:44:11.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.376+0000 7f0c47724640 1 -- 172.21.15.98:0/700537042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c4006c3a0 msgr2=0x7f0c400713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:11.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.376+0000 7f0c47724640 1 --2- 172.21.15.98:0/700537042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c4006c3a0 0x7f0c400713b0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f0c40072e60 tx=0x7f0c3001c450 comp rx=0 tx=0).stop 2024-09-19T08:44:11.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.376+0000 7f0c47724640 1 -- 172.21.15.98:0/700537042 shutdown_connections 2024-09-19T08:44:11.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.376+0000 7f0c47724640 1 --2- 172.21.15.98:0/700537042 >> v2:172.21.15.139:3300/0 conn(0x7f0c400718f0 0x7f0c400b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:11.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.376+0000 7f0c47724640 1 --2- 172.21.15.98:0/700537042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c4006c3a0 0x7f0c400713b0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:11.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.376+0000 7f0c47724640 1 -- 172.21.15.98:0/700537042 >> 172.21.15.98:0/700537042 conn(0x7f0c400b2010 msgr2=0x7f0c400b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:11.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.376+0000 7f0c47724640 1 -- 172.21.15.98:0/700537042 shutdown_connections 2024-09-19T08:44:11.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.376+0000 7f0c47724640 1 -- 172.21.15.98:0/700537042 wait complete. 2024-09-19T08:44:11.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.380+0000 7f0c47724640 1 Processor -- start 2024-09-19T08:44:11.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.380+0000 7f0c47724640 1 -- start start 2024-09-19T08:44:11.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.380+0000 7f0c47724640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0c4006c3a0 0x7f0c4015c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:11.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.380+0000 7f0c47724640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c400718f0 0x7f0c4015b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:11.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.380+0000 7f0c47724640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c40145b70 0x7f0c4013c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:11.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.380+0000 7f0c47724640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0c400b44f0 con 0x7f0c40145b70 2024-09-19T08:44:11.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.380+0000 7f0c47724640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0c400b4370 con 0x7f0c400718f0 2024-09-19T08:44:11.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.380+0000 7f0c47724640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0c400b4670 con 0x7f0c4006c3a0 2024-09-19T08:44:11.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.380+0000 7f0c45cc1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c40145b70 0x7f0c4013c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:11.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.380+0000 7f0c45cc1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c40145b70 0x7f0c4013c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39780/0 (socket says 172.21.15.98:39780) 2024-09-19T08:44:11.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.380+0000 7f0c45cc1640 1 -- 172.21.15.98:0/1613814672 learned_addr learned my addr 172.21.15.98:0/1613814672 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:11.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.380+0000 7f0c44cbf640 1 --2- 172.21.15.98:0/1613814672 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c400718f0 0x7f0c4015b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:11.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.380+0000 7f0c45cc1640 1 -- 172.21.15.98:0/1613814672 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0c4006c3a0 msgr2=0x7f0c4015c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:11.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.380+0000 7f0c45cc1640 1 --2- 172.21.15.98:0/1613814672 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0c4006c3a0 0x7f0c4015c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:11.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.380+0000 7f0c45cc1640 1 -- 172.21.15.98:0/1613814672 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c400718f0 msgr2=0x7f0c4015b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:11.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.380+0000 7f0c45cc1640 1 --2- 172.21.15.98:0/1613814672 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c400718f0 0x7f0c4015b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:11.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.380+0000 7f0c45cc1640 1 -- 172.21.15.98:0/1613814672 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c4015ade0 con 0x7f0c40145b70 2024-09-19T08:44:11.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.380+0000 7f0c45cc1640 1 --2- 172.21.15.98:0/1613814672 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c40145b70 0x7f0c4013c8d0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f0c3800cd70 tx=0x7f0c3800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:11.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.380+0000 7f0c35ffb640 1 -- 172.21.15.98:0/1613814672 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c38003ba0 con 0x7f0c40145b70 2024-09-19T08:44:11.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.380+0000 7f0c35ffb640 1 -- 172.21.15.98:0/1613814672 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c38003d40 con 0x7f0c40145b70 2024-09-19T08:44:11.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.380+0000 7f0c47724640 1 -- 172.21.15.98:0/1613814672 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c4013af80 con 0x7f0c40145b70 2024-09-19T08:44:11.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.380+0000 7f0c35ffb640 1 -- 172.21.15.98:0/1613814672 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c38004240 con 0x7f0c40145b70 2024-09-19T08:44:11.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.380+0000 7f0c47724640 1 -- 172.21.15.98:0/1613814672 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c4015b330 con 0x7f0c40145b70 2024-09-19T08:44:11.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.380+0000 7f0c47724640 1 -- 172.21.15.98:0/1613814672 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c4015b530 con 0x7f0c40145b70 2024-09-19T08:44:11.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.380+0000 7f0c35ffb640 1 -- 172.21.15.98:0/1613814672 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7f0c380043e0 con 0x7f0c40145b70 2024-09-19T08:44:11.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.384+0000 7f0c35ffb640 1 --2- 172.21.15.98:0/1613814672 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0c240679e0 0x7f0c24069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:11.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.384+0000 7f0c35ffb640 1 -- 172.21.15.98:0/1613814672 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f0c38086a70 con 0x7f0c40145b70 2024-09-19T08:44:11.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.384+0000 7f0c454c0640 1 --2- 172.21.15.98:0/1613814672 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0c240679e0 0x7f0c24069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:11.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.384+0000 7f0c454c0640 1 --2- 172.21.15.98:0/1613814672 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0c240679e0 0x7f0c24069ea0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0c4013a390 tx=0x7f0c3c006bd0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:11.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.388+0000 7f0c35ffb640 1 -- 172.21.15.98:0/1613814672 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f0c380572b0 con 0x7f0c40145b70 2024-09-19T08:44:11.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.564+0000 7f0c47724640 1 -- 172.21.15.98:0/1613814672 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol41500", "target": ["mon-mgr", ""]}) -- 0x7f0c400b1aa0 con 0x7f0c240679e0 2024-09-19T08:44:11.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.784+0000 7f0c35ffb640 1 -- 172.21.15.98:0/1613814672 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f0c400b1aa0 con 0x7f0c240679e0 2024-09-19T08:44:11.885 INFO:teuthology.orchestra.run.smithi098.stdout:/volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a 2024-09-19T08:44:11.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.788+0000 7f0c47724640 1 -- 172.21.15.98:0/1613814672 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0c240679e0 msgr2=0x7f0c24069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:11.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.788+0000 7f0c47724640 1 --2- 172.21.15.98:0/1613814672 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0c240679e0 0x7f0c24069ea0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0c4013a390 tx=0x7f0c3c006bd0 comp rx=0 tx=0).stop 2024-09-19T08:44:11.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.788+0000 7f0c47724640 1 -- 172.21.15.98:0/1613814672 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c40145b70 msgr2=0x7f0c4013c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:11.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.788+0000 7f0c47724640 1 --2- 172.21.15.98:0/1613814672 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c40145b70 0x7f0c4013c8d0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f0c3800cd70 tx=0x7f0c3800a660 comp rx=0 tx=0).stop 2024-09-19T08:44:11.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.788+0000 7f0c47724640 1 -- 172.21.15.98:0/1613814672 shutdown_connections 2024-09-19T08:44:11.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.788+0000 7f0c47724640 1 --2- 172.21.15.98:0/1613814672 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0c240679e0 0x7f0c24069ea0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:11.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.788+0000 7f0c47724640 1 --2- 172.21.15.98:0/1613814672 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c40145b70 0x7f0c4013c8d0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:11.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.788+0000 7f0c47724640 1 --2- 172.21.15.98:0/1613814672 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c400718f0 0x7f0c4015b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:11.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.788+0000 7f0c47724640 1 --2- 172.21.15.98:0/1613814672 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0c4006c3a0 0x7f0c4015c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:11.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.788+0000 7f0c47724640 1 -- 172.21.15.98:0/1613814672 >> 172.21.15.98:0/1613814672 conn(0x7f0c400b2010 msgr2=0x7f0c4006d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:11.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.788+0000 7f0c47724640 1 -- 172.21.15.98:0/1613814672 shutdown_connections 2024-09-19T08:44:11.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:11.788+0000 7f0c47724640 1 -- 172.21.15.98:0/1613814672 wait complete. 2024-09-19T08:44:11.889 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs 2024-09-19T08:44:12.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.016+0000 7efe9b099640 1 Processor -- start 2024-09-19T08:44:12.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.016+0000 7efe9b099640 1 -- start start 2024-09-19T08:44:12.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.016+0000 7efe9b099640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe94145b70 0x7efe9406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:12.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.016+0000 7efe9b099640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7efe94071ac0 0x7efe940b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:12.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.016+0000 7efe9b099640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe9406c5f0 con 0x7efe9406c1d0 2024-09-19T08:44:12.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.016+0000 7efe9b099640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7efe94071f20 con 0x7efe94071ac0 2024-09-19T08:44:12.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.016+0000 7efe9b099640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7efe94073530 con 0x7efe94145b70 2024-09-19T08:44:12.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.020+0000 7efe93fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7efe9406c1d0 0x7efe940713b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42492/0 (socket says 172.21.15.98:42492) 2024-09-19T08:44:12.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.020+0000 7efe93fff640 1 -- 172.21.15.98:0/2296857554 learned_addr learned my addr 172.21.15.98:0/2296857554 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:12.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.020+0000 7efe99636640 1 --2- 172.21.15.98:0/2296857554 >> v2:172.21.15.98:3300/0 conn(0x7efe94071ac0 0x7efe940b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:12.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.020+0000 7efe98e35640 1 --2- 172.21.15.98:0/2296857554 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe94145b70 0x7efe9406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:12.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.020+0000 7efe937fe640 1 -- 172.21.15.98:0/2296857554 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2649551355 0 0) 0x7efe9406c5f0 con 0x7efe9406c1d0 2024-09-19T08:44:12.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.020+0000 7efe937fe640 1 -- 172.21.15.98:0/2296857554 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe80003530 con 0x7efe9406c1d0 2024-09-19T08:44:12.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.020+0000 7efe99636640 1 -- 172.21.15.98:0/2296857554 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe94145b70 msgr2=0x7efe9406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:12.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.020+0000 7efe99636640 1 --2- 172.21.15.98:0/2296857554 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe94145b70 0x7efe9406bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:12.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.020+0000 7efe99636640 1 -- 172.21.15.98:0/2296857554 >> v1:172.21.15.98:6789/0 conn(0x7efe9406c1d0 legacy=0x7efe940713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:12.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.020+0000 7efe99636640 1 -- 172.21.15.98:0/2296857554 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe94073810 con 0x7efe94071ac0 2024-09-19T08:44:12.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.020+0000 7efe99636640 1 --2- 172.21.15.98:0/2296857554 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe94071ac0 0x7efe940b3410 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7efe8c009d90 tx=0x7efe8c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab928085902389c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:12.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.020+0000 7efe937fe640 1 -- 172.21.15.98:0/2296857554 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8c0b2070 con 0x7efe94071ac0 2024-09-19T08:44:12.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.020+0000 7efe937fe640 1 -- 172.21.15.98:0/2296857554 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe8c0a57a0 con 0x7efe94071ac0 2024-09-19T08:44:12.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.020+0000 7efe937fe640 1 -- 172.21.15.98:0/2296857554 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8c0a5b50 con 0x7efe94071ac0 2024-09-19T08:44:12.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.020+0000 7efe9b099640 1 -- 172.21.15.98:0/2296857554 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe94071ac0 msgr2=0x7efe940b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:12.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.020+0000 7efe9b099640 1 --2- 172.21.15.98:0/2296857554 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe94071ac0 0x7efe940b3410 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7efe8c009d90 tx=0x7efe8c01c8e0 comp rx=0 tx=0).stop 2024-09-19T08:44:12.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.020+0000 7efe9b099640 1 -- 172.21.15.98:0/2296857554 shutdown_connections 2024-09-19T08:44:12.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.020+0000 7efe9b099640 1 --2- 172.21.15.98:0/2296857554 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe94071ac0 0x7efe940b3410 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:12.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.020+0000 7efe9b099640 1 --2- 172.21.15.98:0/2296857554 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe94145b70 0x7efe9406bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:12.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.020+0000 7efe9b099640 1 -- 172.21.15.98:0/2296857554 >> 172.21.15.98:0/2296857554 conn(0x7efe940b2010 msgr2=0x7efe940b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:12.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.020+0000 7efe9b099640 1 -- 172.21.15.98:0/2296857554 shutdown_connections 2024-09-19T08:44:12.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.020+0000 7efe9b099640 1 -- 172.21.15.98:0/2296857554 wait complete. 2024-09-19T08:44:12.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.024+0000 7efe9b099640 1 Processor -- start 2024-09-19T08:44:12.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.024+0000 7efe9b099640 1 -- start start 2024-09-19T08:44:12.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.024+0000 7efe9b099640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe9406c1d0 0x7efe9415d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:12.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.024+0000 7efe9b099640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe94071ac0 0x7efe9415ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:12.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.024+0000 7efe9b099640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe94145b70 0x7efe94165420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:12.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.024+0000 7efe9b099640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efe940b42e0 con 0x7efe9406c1d0 2024-09-19T08:44:12.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.024+0000 7efe9b099640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efe940b4160 con 0x7efe94145b70 2024-09-19T08:44:12.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.024+0000 7efe9b099640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7efe940b4460 con 0x7efe94071ac0 2024-09-19T08:44:12.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.024+0000 7efe98e35640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe9406c1d0 0x7efe9415d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:12.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.024+0000 7efe98e35640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe9406c1d0 0x7efe9415d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39806/0 (socket says 172.21.15.98:39806) 2024-09-19T08:44:12.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.024+0000 7efe98e35640 1 -- 172.21.15.98:0/3323856771 learned_addr learned my addr 172.21.15.98:0/3323856771 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:12.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.024+0000 7efe93fff640 1 --2- 172.21.15.98:0/3323856771 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe94071ac0 0x7efe9415ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:12.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.024+0000 7efe98e35640 1 -- 172.21.15.98:0/3323856771 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe94071ac0 msgr2=0x7efe9415ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:12.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.024+0000 7efe98e35640 1 --2- 172.21.15.98:0/3323856771 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe94071ac0 0x7efe9415ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:12.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.024+0000 7efe98e35640 1 -- 172.21.15.98:0/3323856771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe94145b70 msgr2=0x7efe94165420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:12.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.024+0000 7efe98e35640 1 --2- 172.21.15.98:0/3323856771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe94145b70 0x7efe94165420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:12.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.024+0000 7efe98e35640 1 -- 172.21.15.98:0/3323856771 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe940ce6a0 con 0x7efe9406c1d0 2024-09-19T08:44:12.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.024+0000 7efe98e35640 1 --2- 172.21.15.98:0/3323856771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe9406c1d0 0x7efe9415d9b0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00d620 tx=0x7efe7c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:12.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.024+0000 7efe917fa640 1 -- 172.21.15.98:0/3323856771 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c00b620 con 0x7efe9406c1d0 2024-09-19T08:44:12.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.024+0000 7efe917fa640 1 -- 172.21.15.98:0/3323856771 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe7c019070 con 0x7efe9406c1d0 2024-09-19T08:44:12.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.024+0000 7efe9b099640 1 -- 172.21.15.98:0/3323856771 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe940ce980 con 0x7efe9406c1d0 2024-09-19T08:44:12.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.024+0000 7efe917fa640 1 -- 172.21.15.98:0/3323856771 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c00ba20 con 0x7efe9406c1d0 2024-09-19T08:44:12.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.024+0000 7efe9b099640 1 -- 172.21.15.98:0/3323856771 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe9413c670 con 0x7efe9406c1d0 2024-09-19T08:44:12.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.024+0000 7efe9b099640 1 -- 172.21.15.98:0/3323856771 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe940b3410 con 0x7efe9406c1d0 2024-09-19T08:44:12.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.024+0000 7efe917fa640 1 -- 172.21.15.98:0/3323856771 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 16) ==== 87547+0+0 (secure 0 0 0) 0x7efe7c00bbc0 con 0x7efe9406c1d0 2024-09-19T08:44:12.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.028+0000 7efe917fa640 1 --2- 172.21.15.98:0/3323856771 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7efe740679e0 0x7efe74069ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:12.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.028+0000 7efe917fa640 1 -- 172.21.15.98:0/3323856771 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7efe7c086360 con 0x7efe9406c1d0 2024-09-19T08:44:12.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.028+0000 7efe93fff640 1 --2- 172.21.15.98:0/3323856771 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7efe740679e0 0x7efe74069ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:12.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.028+0000 7efe93fff640 1 --2- 172.21.15.98:0/3323856771 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7efe740679e0 0x7efe74069ea0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efe84007fd0 tx=0x7efe84009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:12.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.032+0000 7efe917fa640 1 -- 172.21.15.98:0/3323856771 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7efe7c056ca0 con 0x7efe9406c1d0 2024-09-19T08:44:12.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.216+0000 7efe9b099640 1 -- 172.21.15.98:0/3323856771 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7efe94160500 con 0x7efe740679e0 2024-09-19T08:44:12.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.224+0000 7efe917fa640 1 -- 172.21.15.98:0/3323856771 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7efe94160500 con 0x7efe740679e0 2024-09-19T08:44:12.227 INFO:teuthology.orchestra.run.smithi098.stdout:[ 2024-09-19T08:44:12.227 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:44:12.227 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "subvol41500" 2024-09-19T08:44:12.227 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:44:12.227 INFO:teuthology.orchestra.run.smithi098.stdout:] 2024-09-19T08:44:12.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.228+0000 7efe9b099640 1 -- 172.21.15.98:0/3323856771 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7efe740679e0 msgr2=0x7efe74069ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:12.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.228+0000 7efe9b099640 1 --2- 172.21.15.98:0/3323856771 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7efe740679e0 0x7efe74069ea0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efe84007fd0 tx=0x7efe84009040 comp rx=0 tx=0).stop 2024-09-19T08:44:12.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.228+0000 7efe9b099640 1 -- 172.21.15.98:0/3323856771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe9406c1d0 msgr2=0x7efe9415d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:12.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.228+0000 7efe9b099640 1 --2- 172.21.15.98:0/3323856771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe9406c1d0 0x7efe9415d9b0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00d620 tx=0x7efe7c00dae0 comp rx=0 tx=0).stop 2024-09-19T08:44:12.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.228+0000 7efe9b099640 1 -- 172.21.15.98:0/3323856771 shutdown_connections 2024-09-19T08:44:12.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.228+0000 7efe9b099640 1 --2- 172.21.15.98:0/3323856771 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7efe740679e0 0x7efe74069ea0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:12.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.228+0000 7efe9b099640 1 --2- 172.21.15.98:0/3323856771 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efe94145b70 0x7efe94165420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:12.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.228+0000 7efe9b099640 1 --2- 172.21.15.98:0/3323856771 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efe94071ac0 0x7efe9415ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:12.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.228+0000 7efe9b099640 1 --2- 172.21.15.98:0/3323856771 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efe9406c1d0 0x7efe9415d9b0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:12.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.228+0000 7efe9b099640 1 -- 172.21.15.98:0/3323856771 >> 172.21.15.98:0/3323856771 conn(0x7efe940b2010 msgr2=0x7efe940b17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:12.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.228+0000 7efe9b099640 1 -- 172.21.15.98:0/3323856771 shutdown_connections 2024-09-19T08:44:12.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.228+0000 7efe9b099640 1 -- 172.21.15.98:0/3323856771 wait complete. 2024-09-19T08:44:12.244 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol41500 2024-09-19T08:44:12.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.372+0000 7fa7f6017640 1 Processor -- start 2024-09-19T08:44:12.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.372+0000 7fa7f6017640 1 -- start start 2024-09-19T08:44:12.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.372+0000 7fa7f6017640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7f0145b70 0x7fa7f006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:12.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.372+0000 7fa7f6017640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa7f006c1d0 0x7fa7f00713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:12.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.372+0000 7fa7f6017640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7f006c5f0 con 0x7fa7f00718f0 2024-09-19T08:44:12.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.372+0000 7fa7f6017640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fa7f0073530 con 0x7fa7f006c1d0 2024-09-19T08:44:12.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.372+0000 7fa7f6017640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa7f0073810 con 0x7fa7f0145b70 2024-09-19T08:44:12.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.372+0000 7fa7effff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fa7f00718f0 0x7fa7f00cef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42504/0 (socket says 172.21.15.98:42504) 2024-09-19T08:44:12.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.372+0000 7fa7effff640 1 -- 172.21.15.98:0/3900311168 learned_addr learned my addr 172.21.15.98:0/3900311168 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:12.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.372+0000 7fa7ef7fe640 1 --2- 172.21.15.98:0/3900311168 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7f0145b70 0x7fa7f006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:12.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.372+0000 7fa7ee7fc640 1 -- 172.21.15.98:0/3900311168 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1614063164 0 0) 0x7fa7f006c5f0 con 0x7fa7f00718f0 2024-09-19T08:44:12.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.372+0000 7fa7eeffd640 1 --2- 172.21.15.98:0/3900311168 >> v2:172.21.15.139:3300/0 conn(0x7fa7f006c1d0 0x7fa7f00713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:12.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.372+0000 7fa7ee7fc640 1 -- 172.21.15.98:0/3900311168 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa7dc003530 con 0x7fa7f00718f0 2024-09-19T08:44:12.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.372+0000 7fa7ee7fc640 1 -- 172.21.15.98:0/3900311168 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3868608088 0 0) 0x7fa7dc003530 con 0x7fa7f00718f0 2024-09-19T08:44:12.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.372+0000 7fa7ee7fc640 1 -- 172.21.15.98:0/3900311168 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7f0145b70 msgr2=0x7fa7f006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:12.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.372+0000 7fa7ee7fc640 1 --2- 172.21.15.98:0/3900311168 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7f0145b70 0x7fa7f006bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:12.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.372+0000 7fa7ee7fc640 1 -- 172.21.15.98:0/3900311168 >> v2:172.21.15.139:3300/0 conn(0x7fa7f006c1d0 msgr2=0x7fa7f00713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:12.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.372+0000 7fa7ee7fc640 1 --2- 172.21.15.98:0/3900311168 >> v2:172.21.15.139:3300/0 conn(0x7fa7f006c1d0 0x7fa7f00713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:12.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.372+0000 7fa7ee7fc640 1 -- 172.21.15.98:0/3900311168 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f0073af0 con 0x7fa7f00718f0 2024-09-19T08:44:12.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.372+0000 7fa7ee7fc640 1 -- 172.21.15.98:0/3900311168 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fa7e0001a00 con 0x7fa7f00718f0 2024-09-19T08:44:12.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.372+0000 7fa7ee7fc640 1 -- 172.21.15.98:0/3900311168 >> v1:172.21.15.98:6789/0 conn(0x7fa7f00718f0 legacy=0x7fa7f00cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:12.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.372+0000 7fa7ee7fc640 1 --2- 172.21.15.98:0/3900311168 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7dc0039c0 0x7fa7dc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:12.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.372+0000 7fa7ee7fc640 1 --2- 172.21.15.98:0/3900311168 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7f0145b70 0x7fa7dc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:12.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.372+0000 7fa7ee7fc640 1 --2- 172.21.15.98:0/3900311168 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7dc004d50 0x7fa7dc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:12.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.372+0000 7fa7ee7fc640 1 -- 172.21.15.98:0/3900311168 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f0073530 con 0x7fa7dc004d50 2024-09-19T08:44:12.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.372+0000 7fa7ee7fc640 1 -- 172.21.15.98:0/3900311168 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f0073810 con 0x7fa7dc0039c0 2024-09-19T08:44:12.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.372+0000 7fa7ee7fc640 1 -- 172.21.15.98:0/3900311168 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa7dc00b9d0 con 0x7fa7f0145b70 2024-09-19T08:44:12.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.372+0000 7fa7effff640 1 --2- 172.21.15.98:0/3900311168 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7dc004d50 0x7fa7dc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:12.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.372+0000 7fa7ef7fe640 1 --2- 172.21.15.98:0/3900311168 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7dc0039c0 0x7fa7dc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:12.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.372+0000 7fa7effff640 1 -- 172.21.15.98:0/3900311168 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7f0145b70 msgr2=0x7fa7dc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:12.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.372+0000 7fa7effff640 1 --2- 172.21.15.98:0/3900311168 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7f0145b70 0x7fa7dc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:12.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.372+0000 7fa7effff640 1 -- 172.21.15.98:0/3900311168 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7dc0039c0 msgr2=0x7fa7dc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:12.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.372+0000 7fa7effff640 1 --2- 172.21.15.98:0/3900311168 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7dc0039c0 0x7fa7dc003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:12.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.372+0000 7fa7effff640 1 -- 172.21.15.98:0/3900311168 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7dc003530 con 0x7fa7dc004d50 2024-09-19T08:44:12.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.376+0000 7fa7effff640 1 --2- 172.21.15.98:0/3900311168 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7dc004d50 0x7fa7dc0051d0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fa7e000a870 tx=0x7fa7e001fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d885555ad46813 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:12.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.376+0000 7fa7ee7fc640 1 -- 172.21.15.98:0/3900311168 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e000a4c0 con 0x7fa7dc004d50 2024-09-19T08:44:12.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.376+0000 7fa7ee7fc640 1 -- 172.21.15.98:0/3900311168 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7e00ad020 con 0x7fa7dc004d50 2024-09-19T08:44:12.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.376+0000 7fa7ee7fc640 1 -- 172.21.15.98:0/3900311168 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e00a9760 con 0x7fa7dc004d50 2024-09-19T08:44:12.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.376+0000 7fa7f6017640 1 -- 172.21.15.98:0/3900311168 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7dc004d50 msgr2=0x7fa7dc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:12.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.376+0000 7fa7f6017640 1 --2- 172.21.15.98:0/3900311168 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7dc004d50 0x7fa7dc0051d0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fa7e000a870 tx=0x7fa7e001fb00 comp rx=0 tx=0).stop 2024-09-19T08:44:12.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.376+0000 7fa7effff640 1 -- 172.21.15.98:0/3900311168 reap_dead start 2024-09-19T08:44:12.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.376+0000 7fa7f6017640 1 -- 172.21.15.98:0/3900311168 shutdown_connections 2024-09-19T08:44:12.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.376+0000 7fa7f6017640 1 -- 172.21.15.98:0/3900311168 >> 172.21.15.98:0/3900311168 conn(0x7fa7f00b2010 msgr2=0x7fa7f00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:12.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.376+0000 7fa7f6017640 1 -- 172.21.15.98:0/3900311168 shutdown_connections 2024-09-19T08:44:12.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.376+0000 7fa7f6017640 1 -- 172.21.15.98:0/3900311168 wait complete. 2024-09-19T08:44:12.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.376+0000 7fa7f6017640 1 Processor -- start 2024-09-19T08:44:12.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.376+0000 7fa7f6017640 1 -- start start 2024-09-19T08:44:12.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.376+0000 7fa7f6017640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7f013bbd0 0x7fa7f01f2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:12.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.376+0000 7fa7f6017640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7f01579c0 0x7fa7f01446f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:12.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.376+0000 7fa7f6017640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7f0157e40 0x7fa7f0144c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:12.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.376+0000 7fa7f6017640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f0071ed0 con 0x7fa7f013bbd0 2024-09-19T08:44:12.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.376+0000 7fa7f6017640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f0071d50 con 0x7fa7f01579c0 2024-09-19T08:44:12.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.376+0000 7fa7f6017640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa7f0145f50 con 0x7fa7f0157e40 2024-09-19T08:44:12.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.376+0000 7fa7ef7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7f013bbd0 0x7fa7f01f2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:12.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.376+0000 7fa7ef7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7f013bbd0 0x7fa7f01f2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39822/0 (socket says 172.21.15.98:39822) 2024-09-19T08:44:12.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.376+0000 7fa7ef7fe640 1 -- 172.21.15.98:0/630974438 learned_addr learned my addr 172.21.15.98:0/630974438 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:12.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.380+0000 7fa7eeffd640 1 --2- 172.21.15.98:0/630974438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7f01579c0 0x7fa7f01446f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:12.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.380+0000 7fa7effff640 1 --2- 172.21.15.98:0/630974438 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7f0157e40 0x7fa7f0144c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:12.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.380+0000 7fa7ef7fe640 1 -- 172.21.15.98:0/630974438 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7f0157e40 msgr2=0x7fa7f0144c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:12.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.380+0000 7fa7ef7fe640 1 --2- 172.21.15.98:0/630974438 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7f0157e40 0x7fa7f0144c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:12.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.380+0000 7fa7ef7fe640 1 -- 172.21.15.98:0/630974438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7f01579c0 msgr2=0x7fa7f01446f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:12.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.380+0000 7fa7ef7fe640 1 --2- 172.21.15.98:0/630974438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7f01579c0 0x7fa7f01446f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:12.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.380+0000 7fa7ef7fe640 1 -- 172.21.15.98:0/630974438 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f0158330 con 0x7fa7f013bbd0 2024-09-19T08:44:12.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.380+0000 7fa7ef7fe640 1 --2- 172.21.15.98:0/630974438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7f013bbd0 0x7fa7f01f2960 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8010890 tx=0x7fa7e8010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:12.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.380+0000 7fa7e7fff640 1 -- 172.21.15.98:0/630974438 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e800f040 con 0x7fa7f013bbd0 2024-09-19T08:44:12.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.380+0000 7fa7e7fff640 1 -- 172.21.15.98:0/630974438 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7e800eb60 con 0x7fa7f013bbd0 2024-09-19T08:44:12.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.380+0000 7fa7f6017640 1 -- 172.21.15.98:0/630974438 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7f0158610 con 0x7fa7f013bbd0 2024-09-19T08:44:12.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.380+0000 7fa7e7fff640 1 -- 172.21.15.98:0/630974438 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e801c400 con 0x7fa7f013bbd0 2024-09-19T08:44:12.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.380+0000 7fa7f6017640 1 -- 172.21.15.98:0/630974438 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7f01f2ea0 con 0x7fa7f013bbd0 2024-09-19T08:44:12.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.380+0000 7fa7f6017640 1 -- 172.21.15.98:0/630974438 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7f00713b0 con 0x7fa7f013bbd0 2024-09-19T08:44:12.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.380+0000 7fa7e7fff640 1 -- 172.21.15.98:0/630974438 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 17) ==== 87598+0+0 (secure 0 0 0) 0x7fa7e801c5a0 con 0x7fa7f013bbd0 2024-09-19T08:44:12.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.380+0000 7fa7e7fff640 1 --2- 172.21.15.98:0/630974438 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa7d0067a70 0x7fa7d0069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:12.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.380+0000 7fa7e7fff640 1 -- 172.21.15.98:0/630974438 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7fa7e8086600 con 0x7fa7f013bbd0 2024-09-19T08:44:12.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.380+0000 7fa7eeffd640 1 --2- 172.21.15.98:0/630974438 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa7d0067a70 0x7fa7d0069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:12.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.380+0000 7fa7eeffd640 1 --2- 172.21.15.98:0/630974438 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa7d0067a70 0x7fa7d0069f30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa7d80047c0 tx=0x7fa7d8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:12.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.384+0000 7fa7e7fff640 1 -- 172.21.15.98:0/630974438 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa7e8018030 con 0x7fa7f013bbd0 2024-09-19T08:44:12.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.564+0000 7fa7f6017640 1 -- 172.21.15.98:0/630974438 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol41500", "target": ["mon-mgr", ""]}) -- 0x7fa7f00cfc80 con 0x7fa7d0067a70 2024-09-19T08:44:12.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.592+0000 7fa7e7fff640 1 -- 172.21.15.98:0/630974438 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+686 (secure 0 0 0) 0x7fa7f00cfc80 con 0x7fa7d0067a70 2024-09-19T08:44:12.595 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:44:12.595 INFO:teuthology.orchestra.run.smithi098.stdout: "atime": "2024-09-19 08:44:10", 2024-09-19T08:44:12.596 INFO:teuthology.orchestra.run.smithi098.stdout: "bytes_pcent": "undefined", 2024-09-19T08:44:12.596 INFO:teuthology.orchestra.run.smithi098.stdout: "bytes_quota": "infinite", 2024-09-19T08:44:12.596 INFO:teuthology.orchestra.run.smithi098.stdout: "bytes_used": 0, 2024-09-19T08:44:12.596 INFO:teuthology.orchestra.run.smithi098.stdout: "created_at": "2024-09-19 08:44:10", 2024-09-19T08:44:12.596 INFO:teuthology.orchestra.run.smithi098.stdout: "ctime": "2024-09-19 08:44:11", 2024-09-19T08:44:12.596 INFO:teuthology.orchestra.run.smithi098.stdout: "data_pool": "cephfs_data", 2024-09-19T08:44:12.596 INFO:teuthology.orchestra.run.smithi098.stdout: "features": [ 2024-09-19T08:44:12.596 INFO:teuthology.orchestra.run.smithi098.stdout: "snapshot-clone", 2024-09-19T08:44:12.596 INFO:teuthology.orchestra.run.smithi098.stdout: "snapshot-autoprotect" 2024-09-19T08:44:12.596 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:44:12.596 INFO:teuthology.orchestra.run.smithi098.stdout: "flavor": 1, 2024-09-19T08:44:12.596 INFO:teuthology.orchestra.run.smithi098.stdout: "gid": 0, 2024-09-19T08:44:12.596 INFO:teuthology.orchestra.run.smithi098.stdout: "mode": 16877, 2024-09-19T08:44:12.597 INFO:teuthology.orchestra.run.smithi098.stdout: "mon_addrs": [ 2024-09-19T08:44:12.597 INFO:teuthology.orchestra.run.smithi098.stdout: "172.21.15.98:6789", 2024-09-19T08:44:12.597 INFO:teuthology.orchestra.run.smithi098.stdout: "172.21.15.139:6789", 2024-09-19T08:44:12.597 INFO:teuthology.orchestra.run.smithi098.stdout: "172.21.15.139:6790" 2024-09-19T08:44:12.597 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:44:12.597 INFO:teuthology.orchestra.run.smithi098.stdout: "mtime": "2024-09-19 08:44:10", 2024-09-19T08:44:12.597 INFO:teuthology.orchestra.run.smithi098.stdout: "path": "/volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a", 2024-09-19T08:44:12.597 INFO:teuthology.orchestra.run.smithi098.stdout: "pool_namespace": "", 2024-09-19T08:44:12.597 INFO:teuthology.orchestra.run.smithi098.stdout: "state": "complete", 2024-09-19T08:44:12.597 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "subvolume", 2024-09-19T08:44:12.597 INFO:teuthology.orchestra.run.smithi098.stdout: "uid": 0 2024-09-19T08:44:12.597 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:44:12.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.596+0000 7fa7f6017640 1 -- 172.21.15.98:0/630974438 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa7d0067a70 msgr2=0x7fa7d0069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:12.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.596+0000 7fa7f6017640 1 --2- 172.21.15.98:0/630974438 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa7d0067a70 0x7fa7d0069f30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa7d80047c0 tx=0x7fa7d8008040 comp rx=0 tx=0).stop 2024-09-19T08:44:12.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.596+0000 7fa7f6017640 1 -- 172.21.15.98:0/630974438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7f013bbd0 msgr2=0x7fa7f01f2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:12.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.596+0000 7fa7f6017640 1 --2- 172.21.15.98:0/630974438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7f013bbd0 0x7fa7f01f2960 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8010890 tx=0x7fa7e8010d50 comp rx=0 tx=0).stop 2024-09-19T08:44:12.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.596+0000 7fa7f6017640 1 -- 172.21.15.98:0/630974438 shutdown_connections 2024-09-19T08:44:12.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.596+0000 7fa7f6017640 1 --2- 172.21.15.98:0/630974438 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa7d0067a70 0x7fa7d0069f30 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:12.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.596+0000 7fa7f6017640 1 --2- 172.21.15.98:0/630974438 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa7f0157e40 0x7fa7f0144c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:12.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.596+0000 7fa7f6017640 1 --2- 172.21.15.98:0/630974438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa7f01579c0 0x7fa7f01446f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:12.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.596+0000 7fa7f6017640 1 --2- 172.21.15.98:0/630974438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa7f013bbd0 0x7fa7f01f2960 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:12.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.596+0000 7fa7f6017640 1 -- 172.21.15.98:0/630974438 >> 172.21.15.98:0/630974438 conn(0x7fa7f00b2010 msgr2=0x7fa7f015af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:12.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.596+0000 7fa7f6017640 1 -- 172.21.15.98:0/630974438 shutdown_connections 2024-09-19T08:44:12.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.596+0000 7fa7f6017640 1 -- 172.21.15.98:0/630974438 wait complete. 2024-09-19T08:44:12.613 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume resize cephfs subvol41500 10485760 2024-09-19T08:44:12.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.736+0000 7f291f9dd640 1 Processor -- start 2024-09-19T08:44:12.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.736+0000 7f291f9dd640 1 -- start start 2024-09-19T08:44:12.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.736+0000 7f291f9dd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f291806dbd0 0x7f2918073680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:12.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.736+0000 7f291f9dd640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f291815baf0 con 0x7f291815fac0 2024-09-19T08:44:12.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.736+0000 7f291f9dd640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f291815bdd0 con 0x7f291815f6e0 2024-09-19T08:44:12.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.736+0000 7f291f9dd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f291815c8b0 con 0x7f291806dbd0 2024-09-19T08:44:12.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.736+0000 7f291df7a640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f291815fac0 0x7f29180ce420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42520/0 (socket says 172.21.15.98:42520) 2024-09-19T08:44:12.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.736+0000 7f291df7a640 1 -- 172.21.15.98:0/1849021267 learned_addr learned my addr 172.21.15.98:0/1849021267 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:12.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.736+0000 7f291cf78640 1 --2- 172.21.15.98:0/1849021267 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f291806dbd0 0x7f2918073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:12.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.736+0000 7f290ffff640 1 -- 172.21.15.98:0/1849021267 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1280478060 0 0) 0x7f291815baf0 con 0x7f291815fac0 2024-09-19T08:44:12.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.736+0000 7f290ffff640 1 -- 172.21.15.98:0/1849021267 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2904003530 con 0x7f291815fac0 2024-09-19T08:44:12.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.736+0000 7f290ffff640 1 -- 172.21.15.98:0/1849021267 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 134494461 0 0) 0x7f2904003530 con 0x7f291815fac0 2024-09-19T08:44:12.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.740+0000 7f290ffff640 1 -- 172.21.15.98:0/1849021267 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f291806dbd0 msgr2=0x7f2918073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:12.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.740+0000 7f290ffff640 1 --2- 172.21.15.98:0/1849021267 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f291806dbd0 0x7f2918073680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:12.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.740+0000 7f290ffff640 1 -- 172.21.15.98:0/1849021267 >> v1:172.21.15.139:6789/0 conn(0x7f291815f6e0 legacy=0x7f291806d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:12.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.740+0000 7f290ffff640 1 -- 172.21.15.98:0/1849021267 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291815c0b0 con 0x7f291815fac0 2024-09-19T08:44:12.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.740+0000 7f290ffff640 1 -- 172.21.15.98:0/1849021267 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f2910001a00 con 0x7f291815fac0 2024-09-19T08:44:12.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.740+0000 7f290ffff640 1 -- 172.21.15.98:0/1849021267 >> v1:172.21.15.98:6789/0 conn(0x7f291815fac0 legacy=0x7f29180ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:12.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.740+0000 7f290ffff640 1 --2- 172.21.15.98:0/1849021267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29040039f0 0x7f2904003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:12.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.740+0000 7f290ffff640 1 --2- 172.21.15.98:0/1849021267 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2904004810 0x7f2904004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:12.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.740+0000 7f290ffff640 1 --2- 172.21.15.98:0/1849021267 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f291806dbd0 0x7f29040051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:12.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.740+0000 7f290ffff640 1 -- 172.21.15.98:0/1849021267 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f291815c8b0 con 0x7f2904004810 2024-09-19T08:44:12.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.740+0000 7f290ffff640 1 -- 172.21.15.98:0/1849021267 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f290400ba80 con 0x7f29040039f0 2024-09-19T08:44:12.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.740+0000 7f290ffff640 1 -- 172.21.15.98:0/1849021267 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f290400bc80 con 0x7f291806dbd0 2024-09-19T08:44:12.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.740+0000 7f291cf78640 1 --2- 172.21.15.98:0/1849021267 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2904004810 0x7f2904004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:12.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.740+0000 7f291d779640 1 --2- 172.21.15.98:0/1849021267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29040039f0 0x7f2904003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:12.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.740+0000 7f291df7a640 1 --2- 172.21.15.98:0/1849021267 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f291806dbd0 0x7f29040051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:12.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.740+0000 7f291cf78640 1 -- 172.21.15.98:0/1849021267 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f291806dbd0 msgr2=0x7f29040051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:12.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.740+0000 7f291cf78640 1 --2- 172.21.15.98:0/1849021267 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f291806dbd0 0x7f29040051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:12.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.740+0000 7f291cf78640 1 -- 172.21.15.98:0/1849021267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29040039f0 msgr2=0x7f2904003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:12.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.740+0000 7f291cf78640 1 --2- 172.21.15.98:0/1849021267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29040039f0 0x7f2904003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:12.744 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.740+0000 7f291cf78640 1 -- 172.21.15.98:0/1849021267 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2904003530 con 0x7f2904004810 2024-09-19T08:44:12.744 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.740+0000 7f291df7a640 1 --2- 172.21.15.98:0/1849021267 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f291806dbd0 0x7f29040051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:12.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.740+0000 7f291cf78640 1 --2- 172.21.15.98:0/1849021267 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2904004810 0x7f2904004c90 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f2908009ab0 tx=0x7f290801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa4818ed3d9c9004 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:12.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.740+0000 7f290ffff640 1 -- 172.21.15.98:0/1849021267 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29080b2070 con 0x7f2904004810 2024-09-19T08:44:12.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.740+0000 7f290ffff640 1 -- 172.21.15.98:0/1849021267 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29080a5620 con 0x7f2904004810 2024-09-19T08:44:12.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.740+0000 7f290ffff640 1 -- 172.21.15.98:0/1849021267 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29080096b0 con 0x7f2904004810 2024-09-19T08:44:12.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.740+0000 7f291f9dd640 1 -- 172.21.15.98:0/1849021267 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2904004810 msgr2=0x7f2904004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:12.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.740+0000 7f291f9dd640 1 --2- 172.21.15.98:0/1849021267 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2904004810 0x7f2904004c90 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f2908009ab0 tx=0x7f290801ca30 comp rx=0 tx=0).stop 2024-09-19T08:44:12.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.740+0000 7f291df7a640 1 -- 172.21.15.98:0/1849021267 reap_dead start 2024-09-19T08:44:12.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.740+0000 7f291f9dd640 1 -- 172.21.15.98:0/1849021267 shutdown_connections 2024-09-19T08:44:12.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.740+0000 7f291f9dd640 1 -- 172.21.15.98:0/1849021267 >> 172.21.15.98:0/1849021267 conn(0x7f291806c090 msgr2=0x7f291806c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:12.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.740+0000 7f291f9dd640 1 -- 172.21.15.98:0/1849021267 shutdown_connections 2024-09-19T08:44:12.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.740+0000 7f291f9dd640 1 -- 172.21.15.98:0/1849021267 wait complete. 2024-09-19T08:44:12.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.744+0000 7f291f9dd640 1 Processor -- start 2024-09-19T08:44:12.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.744+0000 7f291f9dd640 1 -- start start 2024-09-19T08:44:12.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.744+0000 7f291f9dd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2904004810 0x7f29180ac9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:12.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.744+0000 7f291f9dd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29180ad8c0 0x7f291814f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:12.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.744+0000 7f291f9dd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29180add40 0x7f291814fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:12.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.744+0000 7f291f9dd640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f291815b890 con 0x7f2904004810 2024-09-19T08:44:12.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.744+0000 7f291f9dd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f29180742d0 con 0x7f29180ad8c0 2024-09-19T08:44:12.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.744+0000 7f291f9dd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2918073680 con 0x7f29180add40 2024-09-19T08:44:12.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.744+0000 7f291d779640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2904004810 0x7f29180ac9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:12.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.744+0000 7f291d779640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2904004810 0x7f29180ac9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39828/0 (socket says 172.21.15.98:39828) 2024-09-19T08:44:12.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.744+0000 7f291d779640 1 -- 172.21.15.98:0/1949413303 learned_addr learned my addr 172.21.15.98:0/1949413303 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:12.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.744+0000 7f291cf78640 1 --2- 172.21.15.98:0/1949413303 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29180ad8c0 0x7f291814f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:12.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.744+0000 7f291df7a640 1 --2- 172.21.15.98:0/1949413303 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29180add40 0x7f291814fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:12.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.744+0000 7f291d779640 1 -- 172.21.15.98:0/1949413303 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29180add40 msgr2=0x7f291814fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:12.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.744+0000 7f291d779640 1 --2- 172.21.15.98:0/1949413303 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29180add40 0x7f291814fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:12.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.744+0000 7f291d779640 1 -- 172.21.15.98:0/1949413303 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29180ad8c0 msgr2=0x7f291814f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:12.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.744+0000 7f291d779640 1 --2- 172.21.15.98:0/1949413303 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29180ad8c0 0x7f291814f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:12.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.744+0000 7f291d779640 1 -- 172.21.15.98:0/1949413303 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29180ab920 con 0x7f2904004810 2024-09-19T08:44:12.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.744+0000 7f291d779640 1 --2- 172.21.15.98:0/1949413303 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2904004810 0x7f29180ac9c0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f291400e550 tx=0x7f291400ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:12.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.744+0000 7f290dffb640 1 -- 172.21.15.98:0/1949413303 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29140052e0 con 0x7f2904004810 2024-09-19T08:44:12.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.744+0000 7f290dffb640 1 -- 172.21.15.98:0/1949413303 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2914018070 con 0x7f2904004810 2024-09-19T08:44:12.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.744+0000 7f290dffb640 1 -- 172.21.15.98:0/1949413303 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914003820 con 0x7f2904004810 2024-09-19T08:44:12.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.744+0000 7f291f9dd640 1 -- 172.21.15.98:0/1949413303 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29180abc00 con 0x7f2904004810 2024-09-19T08:44:12.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.744+0000 7f291f9dd640 1 -- 172.21.15.98:0/1949413303 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29180ac1c0 con 0x7f2904004810 2024-09-19T08:44:12.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.744+0000 7f291f9dd640 1 -- 172.21.15.98:0/1949413303 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f291806d4c0 con 0x7f2904004810 2024-09-19T08:44:12.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.744+0000 7f290dffb640 1 -- 172.21.15.98:0/1949413303 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 17) ==== 87598+0+0 (secure 0 0 0) 0x7f29140039c0 con 0x7f2904004810 2024-09-19T08:44:12.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.748+0000 7f290dffb640 1 --2- 172.21.15.98:0/1949413303 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f28fc067a70 0x7f28fc069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:12.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.748+0000 7f290dffb640 1 -- 172.21.15.98:0/1949413303 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f2914085c70 con 0x7f2904004810 2024-09-19T08:44:12.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.748+0000 7f291cf78640 1 --2- 172.21.15.98:0/1949413303 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f28fc067a70 0x7f28fc069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:12.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.748+0000 7f291cf78640 1 --2- 172.21.15.98:0/1949413303 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f28fc067a70 0x7f28fc069f30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f29080af000 tx=0x7f29080046d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:12.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.752+0000 7f290dffb640 1 -- 172.21.15.98:0/1949413303 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2914056580 con 0x7f2904004810 2024-09-19T08:44:12.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.948+0000 7f291f9dd640 1 -- 172.21.15.98:0/1949413303 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume resize", "vol_name": "cephfs", "sub_name": "subvol41500", "new_size": "10485760", "target": ["mon-mgr", ""]}) -- 0x7f291815e790 con 0x7f28fc067a70 2024-09-19T08:44:12.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.980+0000 7f290dffb640 1 -- 172.21.15.98:0/1949413303 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+128 (secure 0 0 0) 0x7f291815e790 con 0x7f28fc067a70 2024-09-19T08:44:12.984 INFO:teuthology.orchestra.run.smithi098.stdout:[ 2024-09-19T08:44:12.985 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:44:12.985 INFO:teuthology.orchestra.run.smithi098.stdout: "bytes_used": 0 2024-09-19T08:44:12.985 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:44:12.985 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:44:12.985 INFO:teuthology.orchestra.run.smithi098.stdout: "bytes_quota": 10485760 2024-09-19T08:44:12.985 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:44:12.985 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:44:12.986 INFO:teuthology.orchestra.run.smithi098.stdout: "bytes_pcent": "0.00" 2024-09-19T08:44:12.986 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:44:12.986 INFO:teuthology.orchestra.run.smithi098.stdout:] 2024-09-19T08:44:12.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.984+0000 7f291f9dd640 1 -- 172.21.15.98:0/1949413303 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f28fc067a70 msgr2=0x7f28fc069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:12.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.984+0000 7f291f9dd640 1 --2- 172.21.15.98:0/1949413303 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f28fc067a70 0x7f28fc069f30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f29080af000 tx=0x7f29080046d0 comp rx=0 tx=0).stop 2024-09-19T08:44:12.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.984+0000 7f291f9dd640 1 -- 172.21.15.98:0/1949413303 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2904004810 msgr2=0x7f29180ac9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:12.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.984+0000 7f291f9dd640 1 --2- 172.21.15.98:0/1949413303 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2904004810 0x7f29180ac9c0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f291400e550 tx=0x7f291400ea10 comp rx=0 tx=0).stop 2024-09-19T08:44:12.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.984+0000 7f291f9dd640 1 -- 172.21.15.98:0/1949413303 shutdown_connections 2024-09-19T08:44:12.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.984+0000 7f291f9dd640 1 --2- 172.21.15.98:0/1949413303 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f28fc067a70 0x7f28fc069f30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:12.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.984+0000 7f291f9dd640 1 --2- 172.21.15.98:0/1949413303 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f29180add40 0x7f291814fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:12.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.984+0000 7f291f9dd640 1 --2- 172.21.15.98:0/1949413303 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f29180ad8c0 0x7f291814f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:12.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.984+0000 7f291f9dd640 1 --2- 172.21.15.98:0/1949413303 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2904004810 0x7f29180ac9c0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:12.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.984+0000 7f291f9dd640 1 -- 172.21.15.98:0/1949413303 >> 172.21.15.98:0/1949413303 conn(0x7f291806c090 msgr2=0x7f291815e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:12.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.984+0000 7f291f9dd640 1 -- 172.21.15.98:0/1949413303 shutdown_connections 2024-09-19T08:44:12.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:12.984+0000 7f291f9dd640 1 -- 172.21.15.98:0/1949413303 wait complete. 2024-09-19T08:44:13.004 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol41500 2024-09-19T08:44:13.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.128+0000 7fe5e1a20640 1 Processor -- start 2024-09-19T08:44:13.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.128+0000 7fe5e1a20640 1 -- start start 2024-09-19T08:44:13.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.128+0000 7fe5e1a20640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe5dc0b5640 0x7fe5dc161840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:13.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.128+0000 7fe5e1a20640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5dc0b7580 0x7fe5dc161d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:13.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.128+0000 7fe5e1a20640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5dc073270 con 0x7fe5dc0b71a0 2024-09-19T08:44:13.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.128+0000 7fe5e1a20640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fe5dc073550 con 0x7fe5dc0b5640 2024-09-19T08:44:13.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.128+0000 7fe5e1a20640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe5dc073830 con 0x7fe5dc0b7580 2024-09-19T08:44:13.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.128+0000 7fe5da7fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe5dc0b5640 0x7fe5dc161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:13.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.128+0000 7fe5da7fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe5dc0b5640 0x7fe5dc161840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39830/0 (socket says 172.21.15.98:39830) 2024-09-19T08:44:13.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.128+0000 7fe5da7fc640 1 -- 172.21.15.98:0/2024178182 learned_addr learned my addr 172.21.15.98:0/2024178182 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:13.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.128+0000 7fe5daffd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fe5dc0b71a0 0x7fe5dc06d660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42524/0 (socket says 172.21.15.98:42524) 2024-09-19T08:44:13.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.128+0000 7fe5db7fe640 1 --2- 172.21.15.98:0/2024178182 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5dc0b7580 0x7fe5dc161d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:13.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.128+0000 7fe5da7fc640 1 -- 172.21.15.98:0/2024178182 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5dc0b7580 msgr2=0x7fe5dc161d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:13.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.128+0000 7fe5da7fc640 1 --2- 172.21.15.98:0/2024178182 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5dc0b7580 0x7fe5dc161d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:13.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.128+0000 7fe5da7fc640 1 -- 172.21.15.98:0/2024178182 >> v1:172.21.15.98:6789/0 conn(0x7fe5dc0b71a0 legacy=0x7fe5dc06d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:13.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.128+0000 7fe5da7fc640 1 -- 172.21.15.98:0/2024178182 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5dc073b10 con 0x7fe5dc0b5640 2024-09-19T08:44:13.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.132+0000 7fe5da7fc640 1 --2- 172.21.15.98:0/2024178182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe5dc0b5640 0x7fe5dc161840 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0009870 tx=0x7fe5d001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615d9118064ce50e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:13.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.132+0000 7fe5d9ffb640 1 -- 172.21.15.98:0/2024178182 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d00b0070 con 0x7fe5dc0b5640 2024-09-19T08:44:13.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.132+0000 7fe5d9ffb640 1 -- 172.21.15.98:0/2024178182 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5d00ab600 con 0x7fe5dc0b5640 2024-09-19T08:44:13.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.132+0000 7fe5d9ffb640 1 -- 172.21.15.98:0/2024178182 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d00ab9d0 con 0x7fe5dc0b5640 2024-09-19T08:44:13.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.132+0000 7fe5e1a20640 1 -- 172.21.15.98:0/2024178182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe5dc0b5640 msgr2=0x7fe5dc161840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:13.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.132+0000 7fe5e1a20640 1 --2- 172.21.15.98:0/2024178182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe5dc0b5640 0x7fe5dc161840 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0009870 tx=0x7fe5d001c3f0 comp rx=0 tx=0).stop 2024-09-19T08:44:13.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.132+0000 7fe5e1a20640 1 -- 172.21.15.98:0/2024178182 shutdown_connections 2024-09-19T08:44:13.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.132+0000 7fe5e1a20640 1 --2- 172.21.15.98:0/2024178182 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5dc0b7580 0x7fe5dc161d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:13.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.132+0000 7fe5e1a20640 1 --2- 172.21.15.98:0/2024178182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe5dc0b5640 0x7fe5dc161840 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:13.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.132+0000 7fe5e1a20640 1 -- 172.21.15.98:0/2024178182 >> 172.21.15.98:0/2024178182 conn(0x7fe5dc0b4260 msgr2=0x7fe5dc0b4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:13.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.132+0000 7fe5e1a20640 1 -- 172.21.15.98:0/2024178182 shutdown_connections 2024-09-19T08:44:13.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.132+0000 7fe5e1a20640 1 -- 172.21.15.98:0/2024178182 wait complete. 2024-09-19T08:44:13.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.132+0000 7fe5e1a20640 1 Processor -- start 2024-09-19T08:44:13.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.132+0000 7fe5e1a20640 1 -- start start 2024-09-19T08:44:13.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.132+0000 7fe5e1a20640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe5dc0b5640 0x7fe5dc152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:13.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.132+0000 7fe5e1a20640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5dc0b71a0 0x7fe5dc15c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:13.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.132+0000 7fe5e1a20640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5dc0b7580 0x7fe5dc157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:13.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.132+0000 7fe5e1a20640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc06ca70 con 0x7fe5dc0b5640 2024-09-19T08:44:13.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.132+0000 7fe5e1a20640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc06c8f0 con 0x7fe5dc0b7580 2024-09-19T08:44:13.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.132+0000 7fe5e1a20640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe5dc06cbf0 con 0x7fe5dc0b71a0 2024-09-19T08:44:13.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.132+0000 7fe5daffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe5dc0b5640 0x7fe5dc152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:13.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.132+0000 7fe5daffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe5dc0b5640 0x7fe5dc152f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39838/0 (socket says 172.21.15.98:39838) 2024-09-19T08:44:13.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.132+0000 7fe5daffd640 1 -- 172.21.15.98:0/265108239 learned_addr learned my addr 172.21.15.98:0/265108239 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:13.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.136+0000 7fe5db7fe640 1 --2- 172.21.15.98:0/265108239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5dc0b7580 0x7fe5dc157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:13.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.136+0000 7fe5daffd640 1 -- 172.21.15.98:0/265108239 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5dc0b71a0 msgr2=0x7fe5dc15c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:13.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.136+0000 7fe5daffd640 1 --2- 172.21.15.98:0/265108239 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5dc0b71a0 0x7fe5dc15c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:13.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.136+0000 7fe5daffd640 1 -- 172.21.15.98:0/265108239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5dc0b7580 msgr2=0x7fe5dc157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:13.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.136+0000 7fe5daffd640 1 --2- 172.21.15.98:0/265108239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5dc0b7580 0x7fe5dc157f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:13.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.136+0000 7fe5daffd640 1 -- 172.21.15.98:0/265108239 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5dc153a10 con 0x7fe5dc0b5640 2024-09-19T08:44:13.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.136+0000 7fe5db7fe640 1 --2- 172.21.15.98:0/265108239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5dc0b7580 0x7fe5dc157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:13.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.136+0000 7fe5daffd640 1 --2- 172.21.15.98:0/265108239 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe5dc0b5640 0x7fe5dc152f70 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00c780 tx=0x7fe5cc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:13.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.136+0000 7fe5c3fff640 1 -- 172.21.15.98:0/265108239 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc018070 con 0x7fe5dc0b5640 2024-09-19T08:44:13.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.136+0000 7fe5c3fff640 1 -- 172.21.15.98:0/265108239 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5cc0072d0 con 0x7fe5dc0b5640 2024-09-19T08:44:13.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.136+0000 7fe5e1a20640 1 -- 172.21.15.98:0/265108239 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5dc153cf0 con 0x7fe5dc0b5640 2024-09-19T08:44:13.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.136+0000 7fe5c3fff640 1 -- 172.21.15.98:0/265108239 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc0047f0 con 0x7fe5dc0b5640 2024-09-19T08:44:13.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.136+0000 7fe5e1a20640 1 -- 172.21.15.98:0/265108239 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5dc1542b0 con 0x7fe5dc0b5640 2024-09-19T08:44:13.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.136+0000 7fe5e1a20640 1 -- 172.21.15.98:0/265108239 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5dc06d050 con 0x7fe5dc0b5640 2024-09-19T08:44:13.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.136+0000 7fe5c3fff640 1 -- 172.21.15.98:0/265108239 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 17) ==== 87598+0+0 (secure 0 0 0) 0x7fe5cc01b070 con 0x7fe5dc0b5640 2024-09-19T08:44:13.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.136+0000 7fe5c3fff640 1 --2- 172.21.15.98:0/265108239 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe5c8067a70 0x7fe5c8069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:13.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.136+0000 7fe5c3fff640 1 -- 172.21.15.98:0/265108239 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7fe5cc0854a0 con 0x7fe5dc0b5640 2024-09-19T08:44:13.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.136+0000 7fe5da7fc640 1 --2- 172.21.15.98:0/265108239 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe5c8067a70 0x7fe5c8069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:13.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.136+0000 7fe5da7fc640 1 --2- 172.21.15.98:0/265108239 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe5c8067a70 0x7fe5c8069f30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe5d00ad000 tx=0x7fe5d0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:13.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.140+0000 7fe5c3fff640 1 -- 172.21.15.98:0/265108239 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe5cc055db0 con 0x7fe5dc0b5640 2024-09-19T08:44:13.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.332+0000 7fe5e1a20640 1 -- 172.21.15.98:0/265108239 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol41500", "target": ["mon-mgr", ""]}) -- 0x7fe5dc06d900 con 0x7fe5c8067a70 2024-09-19T08:44:13.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.360+0000 7fe5c3fff640 1 -- 172.21.15.98:0/265108239 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+679 (secure 0 0 0) 0x7fe5dc06d900 con 0x7fe5c8067a70 2024-09-19T08:44:13.361 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:44:13.361 INFO:teuthology.orchestra.run.smithi098.stdout: "atime": "2024-09-19 08:44:10", 2024-09-19T08:44:13.361 INFO:teuthology.orchestra.run.smithi098.stdout: "bytes_pcent": "0.00", 2024-09-19T08:44:13.361 INFO:teuthology.orchestra.run.smithi098.stdout: "bytes_quota": 10485760, 2024-09-19T08:44:13.362 INFO:teuthology.orchestra.run.smithi098.stdout: "bytes_used": 0, 2024-09-19T08:44:13.362 INFO:teuthology.orchestra.run.smithi098.stdout: "created_at": "2024-09-19 08:44:10", 2024-09-19T08:44:13.362 INFO:teuthology.orchestra.run.smithi098.stdout: "ctime": "2024-09-19 08:44:12", 2024-09-19T08:44:13.362 INFO:teuthology.orchestra.run.smithi098.stdout: "data_pool": "cephfs_data", 2024-09-19T08:44:13.362 INFO:teuthology.orchestra.run.smithi098.stdout: "features": [ 2024-09-19T08:44:13.362 INFO:teuthology.orchestra.run.smithi098.stdout: "snapshot-clone", 2024-09-19T08:44:13.362 INFO:teuthology.orchestra.run.smithi098.stdout: "snapshot-autoprotect" 2024-09-19T08:44:13.362 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:44:13.362 INFO:teuthology.orchestra.run.smithi098.stdout: "flavor": 1, 2024-09-19T08:44:13.362 INFO:teuthology.orchestra.run.smithi098.stdout: "gid": 0, 2024-09-19T08:44:13.362 INFO:teuthology.orchestra.run.smithi098.stdout: "mode": 16877, 2024-09-19T08:44:13.362 INFO:teuthology.orchestra.run.smithi098.stdout: "mon_addrs": [ 2024-09-19T08:44:13.362 INFO:teuthology.orchestra.run.smithi098.stdout: "172.21.15.98:6789", 2024-09-19T08:44:13.363 INFO:teuthology.orchestra.run.smithi098.stdout: "172.21.15.139:6789", 2024-09-19T08:44:13.363 INFO:teuthology.orchestra.run.smithi098.stdout: "172.21.15.139:6790" 2024-09-19T08:44:13.363 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:44:13.363 INFO:teuthology.orchestra.run.smithi098.stdout: "mtime": "2024-09-19 08:44:10", 2024-09-19T08:44:13.363 INFO:teuthology.orchestra.run.smithi098.stdout: "path": "/volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a", 2024-09-19T08:44:13.363 INFO:teuthology.orchestra.run.smithi098.stdout: "pool_namespace": "", 2024-09-19T08:44:13.363 INFO:teuthology.orchestra.run.smithi098.stdout: "state": "complete", 2024-09-19T08:44:13.363 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "subvolume", 2024-09-19T08:44:13.363 INFO:teuthology.orchestra.run.smithi098.stdout: "uid": 0 2024-09-19T08:44:13.363 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:44:13.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.360+0000 7fe5e1a20640 1 -- 172.21.15.98:0/265108239 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe5c8067a70 msgr2=0x7fe5c8069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:13.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.360+0000 7fe5e1a20640 1 --2- 172.21.15.98:0/265108239 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe5c8067a70 0x7fe5c8069f30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe5d00ad000 tx=0x7fe5d0008040 comp rx=0 tx=0).stop 2024-09-19T08:44:13.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.360+0000 7fe5e1a20640 1 -- 172.21.15.98:0/265108239 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe5dc0b5640 msgr2=0x7fe5dc152f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:13.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.360+0000 7fe5e1a20640 1 --2- 172.21.15.98:0/265108239 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe5dc0b5640 0x7fe5dc152f70 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00c780 tx=0x7fe5cc00cc40 comp rx=0 tx=0).stop 2024-09-19T08:44:13.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.360+0000 7fe5e1a20640 1 -- 172.21.15.98:0/265108239 shutdown_connections 2024-09-19T08:44:13.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.360+0000 7fe5e1a20640 1 --2- 172.21.15.98:0/265108239 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe5c8067a70 0x7fe5c8069f30 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:13.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.360+0000 7fe5e1a20640 1 --2- 172.21.15.98:0/265108239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5dc0b7580 0x7fe5dc157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:13.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.360+0000 7fe5e1a20640 1 --2- 172.21.15.98:0/265108239 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5dc0b71a0 0x7fe5dc15c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:13.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.360+0000 7fe5e1a20640 1 --2- 172.21.15.98:0/265108239 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe5dc0b5640 0x7fe5dc152f70 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:13.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.360+0000 7fe5e1a20640 1 -- 172.21.15.98:0/265108239 >> 172.21.15.98:0/265108239 conn(0x7fe5dc0b4260 msgr2=0x7fe5dc0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:13.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.360+0000 7fe5e1a20640 1 -- 172.21.15.98:0/265108239 shutdown_connections 2024-09-19T08:44:13.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.360+0000 7fe5e1a20640 1 -- 172.21.15.98:0/265108239 wait complete. 2024-09-19T08:44:13.381 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol41500 --mode 777 --uid 1000 --gid 1000 2024-09-19T08:44:13.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.512+0000 7ff675277640 1 Processor -- start 2024-09-19T08:44:13.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.512+0000 7ff675277640 1 -- start start 2024-09-19T08:44:13.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.512+0000 7ff675277640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff670161920 0x7ff6700cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:13.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.512+0000 7ff675277640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6700b7e40 0x7ff6700d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:13.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.512+0000 7ff675277640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6700d20a0 con 0x7ff6700d14b0 2024-09-19T08:44:13.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.512+0000 7ff675277640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff6700d2380 con 0x7ff670161920 2024-09-19T08:44:13.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.512+0000 7ff675277640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff6700d2660 con 0x7ff6700b7e40 2024-09-19T08:44:13.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.512+0000 7ff66f7fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7ff6700d14b0 0x7ff670162010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42536/0 (socket says 172.21.15.98:42536) 2024-09-19T08:44:13.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.512+0000 7ff66f7fe640 1 -- 172.21.15.98:0/3377227290 learned_addr learned my addr 172.21.15.98:0/3377227290 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:13.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.516+0000 7ff66effd640 1 --2- 172.21.15.98:0/3377227290 >> v2:172.21.15.139:3300/0 conn(0x7ff670161920 0x7ff6700cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:13.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.516+0000 7ff667fff640 1 --2- 172.21.15.98:0/3377227290 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6700b7e40 0x7ff6700d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:13.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.516+0000 7ff66e7fc640 1 -- 172.21.15.98:0/3377227290 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2142796207 0 0) 0x7ff6700d20a0 con 0x7ff6700d14b0 2024-09-19T08:44:13.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.516+0000 7ff66e7fc640 1 -- 172.21.15.98:0/3377227290 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff65c003530 con 0x7ff6700d14b0 2024-09-19T08:44:13.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.516+0000 7ff66e7fc640 1 -- 172.21.15.98:0/3377227290 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3765792416 0 0) 0x7ff65c003530 con 0x7ff6700d14b0 2024-09-19T08:44:13.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.516+0000 7ff66e7fc640 1 -- 172.21.15.98:0/3377227290 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6700b7e40 msgr2=0x7ff6700d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:13.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.516+0000 7ff66e7fc640 1 --2- 172.21.15.98:0/3377227290 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6700b7e40 0x7ff6700d0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:13.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.516+0000 7ff66e7fc640 1 -- 172.21.15.98:0/3377227290 >> v2:172.21.15.139:3300/0 conn(0x7ff670161920 msgr2=0x7ff6700cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:13.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.516+0000 7ff66e7fc640 1 --2- 172.21.15.98:0/3377227290 >> v2:172.21.15.139:3300/0 conn(0x7ff670161920 0x7ff6700cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:13.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.516+0000 7ff66e7fc640 1 -- 172.21.15.98:0/3377227290 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6700d2940 con 0x7ff6700d14b0 2024-09-19T08:44:13.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.516+0000 7ff66e7fc640 1 -- 172.21.15.98:0/3377227290 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7ff668001a00 con 0x7ff6700d14b0 2024-09-19T08:44:13.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.516+0000 7ff66e7fc640 1 -- 172.21.15.98:0/3377227290 >> v1:172.21.15.98:6789/0 conn(0x7ff6700d14b0 legacy=0x7ff670162010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:13.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.516+0000 7ff66e7fc640 1 --2- 172.21.15.98:0/3377227290 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6700b7e40 0x7ff65c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:13.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.516+0000 7ff66e7fc640 1 --2- 172.21.15.98:0/3377227290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff65c003f40 0x7ff65c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:13.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.516+0000 7ff66e7fc640 1 --2- 172.21.15.98:0/3377227290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff65c004d50 0x7ff65c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:13.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.516+0000 7ff66e7fc640 1 -- 172.21.15.98:0/3377227290 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff6700d2380 con 0x7ff65c003f40 2024-09-19T08:44:13.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.516+0000 7ff66e7fc640 1 -- 172.21.15.98:0/3377227290 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff6700d2660 con 0x7ff65c004d50 2024-09-19T08:44:13.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.516+0000 7ff66e7fc640 1 -- 172.21.15.98:0/3377227290 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff65c00ba00 con 0x7ff6700b7e40 2024-09-19T08:44:13.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.516+0000 7ff667fff640 1 --2- 172.21.15.98:0/3377227290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff65c003f40 0x7ff65c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:13.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.516+0000 7ff66effd640 1 --2- 172.21.15.98:0/3377227290 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6700b7e40 0x7ff65c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:13.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.516+0000 7ff66f7fe640 1 --2- 172.21.15.98:0/3377227290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff65c004d50 0x7ff65c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:13.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.516+0000 7ff667fff640 1 -- 172.21.15.98:0/3377227290 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6700b7e40 msgr2=0x7ff65c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:13.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.516+0000 7ff667fff640 1 --2- 172.21.15.98:0/3377227290 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6700b7e40 0x7ff65c0039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:13.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.516+0000 7ff667fff640 1 -- 172.21.15.98:0/3377227290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff65c004d50 msgr2=0x7ff65c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:13.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.516+0000 7ff667fff640 1 --2- 172.21.15.98:0/3377227290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff65c004d50 0x7ff65c0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:13.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.516+0000 7ff667fff640 1 -- 172.21.15.98:0/3377227290 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff65c003530 con 0x7ff65c003f40 2024-09-19T08:44:13.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.516+0000 7ff667fff640 1 --2- 172.21.15.98:0/3377227290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff65c003f40 0x7ff65c004810 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7ff658009aa0 tx=0x7ff65801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9602261d76228271 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:13.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.516+0000 7ff66e7fc640 1 -- 172.21.15.98:0/3377227290 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff658009670 con 0x7ff65c003f40 2024-09-19T08:44:13.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.516+0000 7ff66e7fc640 1 -- 172.21.15.98:0/3377227290 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6580059a0 con 0x7ff65c003f40 2024-09-19T08:44:13.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.516+0000 7ff66e7fc640 1 -- 172.21.15.98:0/3377227290 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff658005d70 con 0x7ff65c003f40 2024-09-19T08:44:13.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.516+0000 7ff675277640 1 -- 172.21.15.98:0/3377227290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff65c003f40 msgr2=0x7ff65c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:13.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.516+0000 7ff675277640 1 --2- 172.21.15.98:0/3377227290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff65c003f40 0x7ff65c004810 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7ff658009aa0 tx=0x7ff65801ca30 comp rx=0 tx=0).stop 2024-09-19T08:44:13.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.516+0000 7ff66f7fe640 1 -- 172.21.15.98:0/3377227290 reap_dead start 2024-09-19T08:44:13.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.516+0000 7ff675277640 1 -- 172.21.15.98:0/3377227290 shutdown_connections 2024-09-19T08:44:13.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.516+0000 7ff675277640 1 -- 172.21.15.98:0/3377227290 >> 172.21.15.98:0/3377227290 conn(0x7ff670074e40 msgr2=0x7ff67006bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:13.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.516+0000 7ff675277640 1 -- 172.21.15.98:0/3377227290 shutdown_connections 2024-09-19T08:44:13.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.520+0000 7ff675277640 1 -- 172.21.15.98:0/3377227290 wait complete. 2024-09-19T08:44:13.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.520+0000 7ff675277640 1 Processor -- start 2024-09-19T08:44:13.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.520+0000 7ff675277640 1 -- start start 2024-09-19T08:44:13.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.520+0000 7ff675277640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff65c003f40 0x7ff670157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:13.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.520+0000 7ff675277640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff670069b80 0x7ff67015a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:13.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.520+0000 7ff675277640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff67015aa90 0x7ff670158740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:13.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.520+0000 7ff675277640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff670161d00 con 0x7ff65c003f40 2024-09-19T08:44:13.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.520+0000 7ff675277640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff6700d2f00 con 0x7ff67015aa90 2024-09-19T08:44:13.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.520+0000 7ff675277640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff6700b8260 con 0x7ff670069b80 2024-09-19T08:44:13.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.520+0000 7ff66effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff65c003f40 0x7ff670157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:13.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.520+0000 7ff66effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff65c003f40 0x7ff670157f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39856/0 (socket says 172.21.15.98:39856) 2024-09-19T08:44:13.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.520+0000 7ff66effd640 1 -- 172.21.15.98:0/3601150141 learned_addr learned my addr 172.21.15.98:0/3601150141 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:13.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.520+0000 7ff667fff640 1 --2- 172.21.15.98:0/3601150141 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff670069b80 0x7ff67015a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:13.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.520+0000 7ff66f7fe640 1 --2- 172.21.15.98:0/3601150141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff67015aa90 0x7ff670158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:13.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.520+0000 7ff66effd640 1 -- 172.21.15.98:0/3601150141 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff670069b80 msgr2=0x7ff67015a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:13.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.520+0000 7ff66effd640 1 --2- 172.21.15.98:0/3601150141 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff670069b80 0x7ff67015a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:13.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.520+0000 7ff66effd640 1 -- 172.21.15.98:0/3601150141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff67015aa90 msgr2=0x7ff670158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:13.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.520+0000 7ff66effd640 1 --2- 172.21.15.98:0/3601150141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff67015aa90 0x7ff670158740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:13.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.520+0000 7ff66effd640 1 -- 172.21.15.98:0/3601150141 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff67015fe20 con 0x7ff65c003f40 2024-09-19T08:44:13.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.520+0000 7ff66f7fe640 1 --2- 172.21.15.98:0/3601150141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff67015aa90 0x7ff670158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:13.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.520+0000 7ff66effd640 1 --2- 172.21.15.98:0/3601150141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff65c003f40 0x7ff670157f10 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7ff660010900 tx=0x7ff660010dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:13.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.520+0000 7ff6677fe640 1 -- 172.21.15.98:0/3601150141 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66000f040 con 0x7ff65c003f40 2024-09-19T08:44:13.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.520+0000 7ff6677fe640 1 -- 172.21.15.98:0/3601150141 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff66000ec20 con 0x7ff65c003f40 2024-09-19T08:44:13.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.520+0000 7ff6677fe640 1 -- 172.21.15.98:0/3601150141 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66001c400 con 0x7ff65c003f40 2024-09-19T08:44:13.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.520+0000 7ff675277640 1 -- 172.21.15.98:0/3601150141 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff670160100 con 0x7ff65c003f40 2024-09-19T08:44:13.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.520+0000 7ff675277640 1 -- 172.21.15.98:0/3601150141 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6701606c0 con 0x7ff65c003f40 2024-09-19T08:44:13.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.524+0000 7ff675277640 1 -- 172.21.15.98:0/3601150141 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff67015aea0 con 0x7ff65c003f40 2024-09-19T08:44:13.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.524+0000 7ff6677fe640 1 -- 172.21.15.98:0/3601150141 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 17) ==== 87598+0+0 (secure 0 0 0) 0x7ff66001c5a0 con 0x7ff65c003f40 2024-09-19T08:44:13.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.524+0000 7ff6677fe640 1 --2- 172.21.15.98:0/3601150141 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff650067a70 0x7ff650069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:13.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.524+0000 7ff6677fe640 1 -- 172.21.15.98:0/3601150141 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7ff660086690 con 0x7ff65c003f40 2024-09-19T08:44:13.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.524+0000 7ff667fff640 1 --2- 172.21.15.98:0/3601150141 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff650067a70 0x7ff650069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:13.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.524+0000 7ff667fff640 1 --2- 172.21.15.98:0/3601150141 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff650067a70 0x7ff650069f30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff6580ae000 tx=0x7ff6580046d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:13.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.528+0000 7ff6677fe640 1 -- 172.21.15.98:0/3601150141 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff67015aea0 con 0x7ff65c003f40 2024-09-19T08:44:13.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.708+0000 7ff675277640 1 -- 172.21.15.98:0/3601150141 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol41500", "mode": "777", "uid": 1000, "gid": 1000, "target": ["mon-mgr", ""]}) -- 0x7ff6700b94b0 con 0x7ff650067a70 2024-09-19T08:44:13.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.748+0000 7ff6677fe640 1 -- 172.21.15.98:0/3601150141 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff6700b94b0 con 0x7ff650067a70 2024-09-19T08:44:13.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.748+0000 7ff675277640 1 -- 172.21.15.98:0/3601150141 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff650067a70 msgr2=0x7ff650069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:13.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.748+0000 7ff675277640 1 --2- 172.21.15.98:0/3601150141 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff650067a70 0x7ff650069f30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff6580ae000 tx=0x7ff6580046d0 comp rx=0 tx=0).stop 2024-09-19T08:44:13.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.748+0000 7ff675277640 1 -- 172.21.15.98:0/3601150141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff65c003f40 msgr2=0x7ff670157f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:13.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.748+0000 7ff675277640 1 --2- 172.21.15.98:0/3601150141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff65c003f40 0x7ff670157f10 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7ff660010900 tx=0x7ff660010dc0 comp rx=0 tx=0).stop 2024-09-19T08:44:13.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.748+0000 7ff675277640 1 -- 172.21.15.98:0/3601150141 shutdown_connections 2024-09-19T08:44:13.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.748+0000 7ff675277640 1 --2- 172.21.15.98:0/3601150141 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff650067a70 0x7ff650069f30 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:13.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.748+0000 7ff675277640 1 --2- 172.21.15.98:0/3601150141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff67015aa90 0x7ff670158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:13.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.748+0000 7ff675277640 1 --2- 172.21.15.98:0/3601150141 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff670069b80 0x7ff67015a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:13.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.748+0000 7ff675277640 1 --2- 172.21.15.98:0/3601150141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff65c003f40 0x7ff670157f10 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:13.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.748+0000 7ff675277640 1 -- 172.21.15.98:0/3601150141 >> 172.21.15.98:0/3601150141 conn(0x7ff670074e40 msgr2=0x7ff6700d3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:13.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.748+0000 7ff675277640 1 -- 172.21.15.98:0/3601150141 shutdown_connections 2024-09-19T08:44:13.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.748+0000 7ff675277640 1 -- 172.21.15.98:0/3601150141 wait complete. 2024-09-19T08:44:13.767 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol41500 2024-09-19T08:44:13.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.892+0000 7f224eed8640 1 Processor -- start 2024-09-19T08:44:13.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.892+0000 7f224eed8640 1 -- start start 2024-09-19T08:44:13.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.892+0000 7f224eed8640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2248144d10 0x7f22480722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:13.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.892+0000 7f224eed8640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f224806c3a0 0x7f22480cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:13.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.892+0000 7f224eed8640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2248072ea0 con 0x7f2248072820 2024-09-19T08:44:13.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.892+0000 7f224eed8640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f2248072c60 con 0x7f2248144d10 2024-09-19T08:44:13.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.892+0000 7f224eed8640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f2248073ab0 con 0x7f224806c3a0 2024-09-19T08:44:13.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.892+0000 7f224cc74640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2248144d10 0x7f22480722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:13.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.892+0000 7f223ffff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f2248072820 0x7f224806bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42542/0 (socket says 172.21.15.98:42542) 2024-09-19T08:44:13.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.892+0000 7f224cc74640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2248144d10 0x7f22480722e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39858/0 (socket says 172.21.15.98:39858) 2024-09-19T08:44:13.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.892+0000 7f223ffff640 1 -- 172.21.15.98:0/499281645 learned_addr learned my addr 172.21.15.98:0/499281645 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:13.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.892+0000 7f224d475640 1 --2- 172.21.15.98:0/499281645 >> v2:172.21.15.139:3300/0 conn(0x7f224806c3a0 0x7f22480cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:13.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.892+0000 7f224cc74640 1 -- 172.21.15.98:0/499281645 >> v2:172.21.15.139:3300/0 conn(0x7f224806c3a0 msgr2=0x7f22480cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:13.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.892+0000 7f224cc74640 1 --2- 172.21.15.98:0/499281645 >> v2:172.21.15.139:3300/0 conn(0x7f224806c3a0 0x7f22480cd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:13.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.892+0000 7f224cc74640 1 -- 172.21.15.98:0/499281645 >> v1:172.21.15.98:6789/0 conn(0x7f2248072820 legacy=0x7f224806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:13.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.892+0000 7f224cc74640 1 -- 172.21.15.98:0/499281645 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2248073180 con 0x7f2248144d10 2024-09-19T08:44:13.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.892+0000 7f223f7fe640 1 -- 172.21.15.98:0/499281645 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2157168243 0 0) 0x7f2248072ea0 con 0x7f2248072820 2024-09-19T08:44:13.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.896+0000 7f224cc74640 1 --2- 172.21.15.98:0/499281645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2248144d10 0x7f22480722e0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f224806f4b0 tx=0x7f224401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=247982b1b9014485 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:13.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.896+0000 7f223f7fe640 1 -- 172.21.15.98:0/499281645 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22440b0070 con 0x7f2248144d10 2024-09-19T08:44:13.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.896+0000 7f223f7fe640 1 -- 172.21.15.98:0/499281645 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22440ab600 con 0x7f2248144d10 2024-09-19T08:44:13.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.896+0000 7f223f7fe640 1 -- 172.21.15.98:0/499281645 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22440ab9d0 con 0x7f2248144d10 2024-09-19T08:44:13.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.896+0000 7f224eed8640 1 -- 172.21.15.98:0/499281645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2248144d10 msgr2=0x7f22480722e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:13.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.896+0000 7f224eed8640 1 --2- 172.21.15.98:0/499281645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2248144d10 0x7f22480722e0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f224806f4b0 tx=0x7f224401c480 comp rx=0 tx=0).stop 2024-09-19T08:44:13.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.896+0000 7f224eed8640 1 -- 172.21.15.98:0/499281645 shutdown_connections 2024-09-19T08:44:13.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.896+0000 7f224eed8640 1 --2- 172.21.15.98:0/499281645 >> v2:172.21.15.139:3300/0 conn(0x7f224806c3a0 0x7f22480cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:13.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.896+0000 7f224eed8640 1 --2- 172.21.15.98:0/499281645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2248144d10 0x7f22480722e0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:13.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.896+0000 7f224eed8640 1 -- 172.21.15.98:0/499281645 >> 172.21.15.98:0/499281645 conn(0x7f22480b2010 msgr2=0x7f22480b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:13.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.896+0000 7f224eed8640 1 -- 172.21.15.98:0/499281645 shutdown_connections 2024-09-19T08:44:13.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.896+0000 7f224eed8640 1 -- 172.21.15.98:0/499281645 wait complete. 2024-09-19T08:44:13.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.896+0000 7f224eed8640 1 Processor -- start 2024-09-19T08:44:13.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.896+0000 7f224eed8640 1 -- start start 2024-09-19T08:44:13.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.896+0000 7f224eed8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f224806c3a0 0x7f22480ce690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:13.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.896+0000 7f224eed8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2248072820 0x7f22480cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:13.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.896+0000 7f224eed8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2248144d10 0x7f224815b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:13.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.896+0000 7f224eed8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f22480ce120 con 0x7f2248072820 2024-09-19T08:44:13.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.896+0000 7f224eed8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f22480cdfa0 con 0x7f224806c3a0 2024-09-19T08:44:13.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.896+0000 7f224eed8640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f22480ce2a0 con 0x7f2248144d10 2024-09-19T08:44:13.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.896+0000 7f223ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2248072820 0x7f22480cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:13.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.896+0000 7f223ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2248072820 0x7f22480cebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39860/0 (socket says 172.21.15.98:39860) 2024-09-19T08:44:13.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.896+0000 7f223ffff640 1 -- 172.21.15.98:0/3656432936 learned_addr learned my addr 172.21.15.98:0/3656432936 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:13.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.896+0000 7f224cc74640 1 --2- 172.21.15.98:0/3656432936 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f224806c3a0 0x7f22480ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:13.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.896+0000 7f223ffff640 1 -- 172.21.15.98:0/3656432936 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2248144d10 msgr2=0x7f224815b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:13.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.896+0000 7f223ffff640 1 --2- 172.21.15.98:0/3656432936 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2248144d10 0x7f224815b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:13.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.896+0000 7f223ffff640 1 -- 172.21.15.98:0/3656432936 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f224806c3a0 msgr2=0x7f22480ce690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:13.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.896+0000 7f223ffff640 1 --2- 172.21.15.98:0/3656432936 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f224806c3a0 0x7f22480ce690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:13.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.896+0000 7f223ffff640 1 -- 172.21.15.98:0/3656432936 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22481408c0 con 0x7f2248072820 2024-09-19T08:44:13.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.896+0000 7f223ffff640 1 --2- 172.21.15.98:0/3656432936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2248072820 0x7f22480cebd0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f22380011f0 tx=0x7f223800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:13.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.896+0000 7f223d7fa640 1 -- 172.21.15.98:0/3656432936 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2238018070 con 0x7f2248072820 2024-09-19T08:44:13.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.900+0000 7f223d7fa640 1 -- 172.21.15.98:0/3656432936 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22380074e0 con 0x7f2248072820 2024-09-19T08:44:13.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.900+0000 7f224eed8640 1 -- 172.21.15.98:0/3656432936 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2248140ba0 con 0x7f2248072820 2024-09-19T08:44:13.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.900+0000 7f223d7fa640 1 -- 172.21.15.98:0/3656432936 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2238004a10 con 0x7f2248072820 2024-09-19T08:44:13.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.900+0000 7f224eed8640 1 -- 172.21.15.98:0/3656432936 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2248141160 con 0x7f2248072820 2024-09-19T08:44:13.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.900+0000 7f224eed8640 1 -- 172.21.15.98:0/3656432936 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22480cd230 con 0x7f2248072820 2024-09-19T08:44:13.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.900+0000 7f223d7fa640 1 -- 172.21.15.98:0/3656432936 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 17) ==== 87598+0+0 (secure 0 0 0) 0x7f223801b070 con 0x7f2248072820 2024-09-19T08:44:13.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.900+0000 7f223d7fa640 1 --2- 172.21.15.98:0/3656432936 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2234067a70 0x7f2234069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:13.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.900+0000 7f223d7fa640 1 -- 172.21.15.98:0/3656432936 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f2238085380 con 0x7f2248072820 2024-09-19T08:44:13.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.900+0000 7f224cc74640 1 --2- 172.21.15.98:0/3656432936 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2234067a70 0x7f2234069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:13.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.900+0000 7f224cc74640 1 --2- 172.21.15.98:0/3656432936 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2234067a70 0x7f2234069f30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f22440ad000 tx=0x7f2244008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:13.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:13.904+0000 7f223d7fa640 1 -- 172.21.15.98:0/3656432936 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2238055c00 con 0x7f2248072820 2024-09-19T08:44:14.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:14.092+0000 7f224eed8640 1 -- 172.21.15.98:0/3656432936 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol41500", "target": ["mon-mgr", ""]}) -- 0x7f224813ecf0 con 0x7f2234067a70 2024-09-19T08:44:14.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:14.128+0000 7f223d7fa640 1 -- 172.21.15.98:0/3656432936 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f224813ecf0 con 0x7f2234067a70 2024-09-19T08:44:14.129 INFO:teuthology.orchestra.run.smithi098.stdout:/volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a 2024-09-19T08:44:14.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:14.128+0000 7f224eed8640 1 -- 172.21.15.98:0/3656432936 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2234067a70 msgr2=0x7f2234069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:14.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:14.128+0000 7f224eed8640 1 --2- 172.21.15.98:0/3656432936 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2234067a70 0x7f2234069f30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f22440ad000 tx=0x7f2244008040 comp rx=0 tx=0).stop 2024-09-19T08:44:14.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:14.128+0000 7f224eed8640 1 -- 172.21.15.98:0/3656432936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2248072820 msgr2=0x7f22480cebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:14.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:14.128+0000 7f224eed8640 1 --2- 172.21.15.98:0/3656432936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2248072820 0x7f22480cebd0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f22380011f0 tx=0x7f223800c9b0 comp rx=0 tx=0).stop 2024-09-19T08:44:14.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:14.128+0000 7f224eed8640 1 -- 172.21.15.98:0/3656432936 shutdown_connections 2024-09-19T08:44:14.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:14.128+0000 7f224eed8640 1 --2- 172.21.15.98:0/3656432936 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2234067a70 0x7f2234069f30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:14.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:14.128+0000 7f224eed8640 1 --2- 172.21.15.98:0/3656432936 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2248144d10 0x7f224815b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:14.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:14.128+0000 7f224eed8640 1 --2- 172.21.15.98:0/3656432936 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2248072820 0x7f22480cebd0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:14.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:14.128+0000 7f224eed8640 1 --2- 172.21.15.98:0/3656432936 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f224806c3a0 0x7f22480ce690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:14.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:14.128+0000 7f224eed8640 1 -- 172.21.15.98:0/3656432936 >> 172.21.15.98:0/3656432936 conn(0x7f22480b2010 msgr2=0x7f224806d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:14.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:14.128+0000 7f224eed8640 1 -- 172.21.15.98:0/3656432936 shutdown_connections 2024-09-19T08:44:14.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:14.128+0000 7f224eed8640 1 -- 172.21.15.98:0/3656432936 wait complete. 2024-09-19T08:44:14.150 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvol41500 with 8 files each 1MB size under directory volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a 2024-09-19T08:44:14.150 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:44:14.150 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:44:14.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:14.156 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0) 2024-09-19T08:44:14.233 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-09-19T08:44:14.233 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-09-19T08:44:14.233 INFO:teuthology.orchestra.run.smithi098.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166271 s, 63.1 MB/s 2024-09-19T08:44:14.233 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:44:14.234 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:44:14.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:14.282 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0) 2024-09-19T08:44:14.363 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-09-19T08:44:14.363 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-09-19T08:44:14.363 INFO:teuthology.orchestra.run.smithi098.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191396 s, 54.8 MB/s 2024-09-19T08:44:14.364 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:44:14.364 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:44:14.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:14.414 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0) 2024-09-19T08:44:14.492 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-09-19T08:44:14.493 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-09-19T08:44:14.493 INFO:teuthology.orchestra.run.smithi098.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191079 s, 54.9 MB/s 2024-09-19T08:44:14.494 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:44:14.494 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:44:14.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:14.542 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0) 2024-09-19T08:44:14.621 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-09-19T08:44:14.621 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-09-19T08:44:14.621 INFO:teuthology.orchestra.run.smithi098.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183308 s, 57.2 MB/s 2024-09-19T08:44:14.622 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:44:14.622 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:44:14.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:14.673 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0) 2024-09-19T08:44:14.748 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-09-19T08:44:14.748 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-09-19T08:44:14.748 INFO:teuthology.orchestra.run.smithi098.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157606 s, 66.5 MB/s 2024-09-19T08:44:14.749 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:44:14.749 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:44:14.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:14.798 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0) 2024-09-19T08:44:14.868 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-09-19T08:44:14.868 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-09-19T08:44:14.868 INFO:teuthology.orchestra.run.smithi098.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0130109 s, 80.6 MB/s 2024-09-19T08:44:14.869 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:44:14.869 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:44:14.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:14.918 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0) 2024-09-19T08:44:14.995 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-09-19T08:44:14.995 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-09-19T08:44:14.995 INFO:teuthology.orchestra.run.smithi098.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179393 s, 58.5 MB/s 2024-09-19T08:44:14.996 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:44:14.996 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:44:15.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:15.047 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0) 2024-09-19T08:44:15.123 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records in 2024-09-19T08:44:15.124 INFO:teuthology.orchestra.run.smithi098.stderr:1+0 records out 2024-09-19T08:44:15.124 INFO:teuthology.orchestra.run.smithi098.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201979 s, 51.9 MB/s 2024-09-19T08:44:15.125 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvol41500 subvol_snap63800 2024-09-19T08:44:15.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.300+0000 7fb21264f640 1 Processor -- start 2024-09-19T08:44:15.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.300+0000 7fb21264f640 1 -- start start 2024-09-19T08:44:15.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.300+0000 7fb21264f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb20c15a930 0x7fb20c0b3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:15.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.300+0000 7fb21264f640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb20c0b3b60 0x7fb20c0b5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:15.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.304+0000 7fb21264f640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb20c0b5110 con 0x7fb20c0b6200 2024-09-19T08:44:15.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.304+0000 7fb21264f640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fb20c0b4070 con 0x7fb20c0b3b60 2024-09-19T08:44:15.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.304+0000 7fb21264f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb20c0b4350 con 0x7fb20c15a930 2024-09-19T08:44:15.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.304+0000 7fb210bec640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fb20c0b6200 0x7fb20c071030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42554/0 (socket says 172.21.15.98:42554) 2024-09-19T08:44:15.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.304+0000 7fb210bec640 1 -- 172.21.15.98:0/1785481531 learned_addr learned my addr 172.21.15.98:0/1785481531 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:15.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.304+0000 7fb20b7fe640 1 --2- 172.21.15.98:0/1785481531 >> v2:172.21.15.98:3300/0 conn(0x7fb20c0b3b60 0x7fb20c0b5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:15.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.304+0000 7fb20bfff640 1 --2- 172.21.15.98:0/1785481531 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb20c15a930 0x7fb20c0b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:15.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.304+0000 7fb20affd640 1 -- 172.21.15.98:0/1785481531 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3229519994 0 0) 0x7fb20c0b5110 con 0x7fb20c0b6200 2024-09-19T08:44:15.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.304+0000 7fb20affd640 1 -- 172.21.15.98:0/1785481531 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb1f4003530 con 0x7fb20c0b6200 2024-09-19T08:44:15.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.304+0000 7fb20b7fe640 1 -- 172.21.15.98:0/1785481531 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb20c15a930 msgr2=0x7fb20c0b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:15.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.304+0000 7fb20b7fe640 1 --2- 172.21.15.98:0/1785481531 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb20c15a930 0x7fb20c0b3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:15.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.304+0000 7fb20b7fe640 1 -- 172.21.15.98:0/1785481531 >> v1:172.21.15.98:6789/0 conn(0x7fb20c0b6200 legacy=0x7fb20c071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:15.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.304+0000 7fb20b7fe640 1 -- 172.21.15.98:0/1785481531 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb20c0b4630 con 0x7fb20c0b3b60 2024-09-19T08:44:15.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.304+0000 7fb20b7fe640 1 --2- 172.21.15.98:0/1785481531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb20c0b3b60 0x7fb20c0b5cc0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc0097c0 tx=0x7fb1fc01d640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9b37fefa85de91d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:15.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.304+0000 7fb20affd640 1 -- 172.21.15.98:0/1785481531 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc0a7040 con 0x7fb20c0b3b60 2024-09-19T08:44:15.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.304+0000 7fb20affd640 1 -- 172.21.15.98:0/1785481531 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1fc0b0070 con 0x7fb20c0b3b60 2024-09-19T08:44:15.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.304+0000 7fb20affd640 1 -- 172.21.15.98:0/1785481531 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc0ab840 con 0x7fb20c0b3b60 2024-09-19T08:44:15.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.304+0000 7fb21264f640 1 -- 172.21.15.98:0/1785481531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb20c0b3b60 msgr2=0x7fb20c0b5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:15.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.304+0000 7fb21264f640 1 --2- 172.21.15.98:0/1785481531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb20c0b3b60 0x7fb20c0b5cc0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc0097c0 tx=0x7fb1fc01d640 comp rx=0 tx=0).stop 2024-09-19T08:44:15.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.308+0000 7fb21264f640 1 -- 172.21.15.98:0/1785481531 shutdown_connections 2024-09-19T08:44:15.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.308+0000 7fb21264f640 1 --2- 172.21.15.98:0/1785481531 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb20c0b3b60 0x7fb20c0b5cc0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:15.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.308+0000 7fb21264f640 1 --2- 172.21.15.98:0/1785481531 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb20c15a930 0x7fb20c0b3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:15.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.308+0000 7fb21264f640 1 -- 172.21.15.98:0/1785481531 >> 172.21.15.98:0/1785481531 conn(0x7fb20c06c090 msgr2=0x7fb20c06c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:15.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.308+0000 7fb21264f640 1 -- 172.21.15.98:0/1785481531 shutdown_connections 2024-09-19T08:44:15.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.308+0000 7fb21264f640 1 -- 172.21.15.98:0/1785481531 wait complete. 2024-09-19T08:44:15.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.308+0000 7fb21264f640 1 Processor -- start 2024-09-19T08:44:15.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.308+0000 7fb21264f640 1 -- start start 2024-09-19T08:44:15.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.308+0000 7fb21264f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb20c0b3b60 0x7fb20c072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:15.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.308+0000 7fb21264f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb20c0b6200 0x7fb20c154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:15.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.308+0000 7fb21264f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb20c15a930 0x7fb20c155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:15.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.308+0000 7fb21264f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c0ced10 con 0x7fb20c0b6200 2024-09-19T08:44:15.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.308+0000 7fb21264f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c0ceb90 con 0x7fb20c15a930 2024-09-19T08:44:15.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.308+0000 7fb21264f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb20c071740 con 0x7fb20c0b3b60 2024-09-19T08:44:15.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.308+0000 7fb20b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb20c0b6200 0x7fb20c154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:15.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.308+0000 7fb20b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb20c0b6200 0x7fb20c154ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39876/0 (socket says 172.21.15.98:39876) 2024-09-19T08:44:15.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.308+0000 7fb20b7fe640 1 -- 172.21.15.98:0/2133150417 learned_addr learned my addr 172.21.15.98:0/2133150417 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:15.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.308+0000 7fb20bfff640 1 --2- 172.21.15.98:0/2133150417 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb20c0b3b60 0x7fb20c072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:15.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.308+0000 7fb20b7fe640 1 -- 172.21.15.98:0/2133150417 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb20c0b3b60 msgr2=0x7fb20c072be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:15.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.308+0000 7fb20b7fe640 1 --2- 172.21.15.98:0/2133150417 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb20c0b3b60 0x7fb20c072be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:15.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.308+0000 7fb20b7fe640 1 -- 172.21.15.98:0/2133150417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb20c15a930 msgr2=0x7fb20c155000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:15.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.308+0000 7fb20b7fe640 1 --2- 172.21.15.98:0/2133150417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb20c15a930 0x7fb20c155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:15.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.308+0000 7fb20b7fe640 1 -- 172.21.15.98:0/2133150417 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb20c155540 con 0x7fb20c0b6200 2024-09-19T08:44:15.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.308+0000 7fb20b7fe640 1 --2- 172.21.15.98:0/2133150417 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb20c0b6200 0x7fb20c154ac0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc0ad040 tx=0x7fb1fc01a3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:15.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.308+0000 7fb208ff9640 1 -- 172.21.15.98:0/2133150417 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc003920 con 0x7fb20c0b6200 2024-09-19T08:44:15.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.308+0000 7fb208ff9640 1 -- 172.21.15.98:0/2133150417 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1fc0a7030 con 0x7fb20c0b6200 2024-09-19T08:44:15.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.308+0000 7fb21264f640 1 -- 172.21.15.98:0/2133150417 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb20c155810 con 0x7fb20c0b6200 2024-09-19T08:44:15.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.308+0000 7fb208ff9640 1 -- 172.21.15.98:0/2133150417 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc003cc0 con 0x7fb20c0b6200 2024-09-19T08:44:15.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.308+0000 7fb21264f640 1 -- 172.21.15.98:0/2133150417 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb20c150ef0 con 0x7fb20c0b6200 2024-09-19T08:44:15.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.308+0000 7fb21264f640 1 -- 172.21.15.98:0/2133150417 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb20c071b10 con 0x7fb20c0b6200 2024-09-19T08:44:15.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.312+0000 7fb208ff9640 1 -- 172.21.15.98:0/2133150417 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 17) ==== 87598+0+0 (secure 0 0 0) 0x7fb1fc0ab440 con 0x7fb20c0b6200 2024-09-19T08:44:15.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.312+0000 7fb208ff9640 1 --2- 172.21.15.98:0/2133150417 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb1f8067a70 0x7fb1f8069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:15.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.312+0000 7fb208ff9640 1 -- 172.21.15.98:0/2133150417 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7fb1fc1201c0 con 0x7fb20c0b6200 2024-09-19T08:44:15.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.312+0000 7fb20bfff640 1 --2- 172.21.15.98:0/2133150417 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb1f8067a70 0x7fb1f8069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:15.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.312+0000 7fb20bfff640 1 --2- 172.21.15.98:0/2133150417 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb1f8067a70 0x7fb1f8069f30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb2000096f0 tx=0x7fb200008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:15.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.316+0000 7fb208ff9640 1 -- 172.21.15.98:0/2133150417 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb1fc0172c0 con 0x7fb20c0b6200 2024-09-19T08:44:15.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.508+0000 7fb21264f640 1 -- 172.21.15.98:0/2133150417 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvol41500", "snap_name": "subvol_snap63800", "target": ["mon-mgr", ""]}) -- 0x7fb20c0afe70 con 0x7fb1f8067a70 2024-09-19T08:44:15.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.548+0000 7fb208ff9640 1 -- 172.21.15.98:0/2133150417 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb20c0afe70 con 0x7fb1f8067a70 2024-09-19T08:44:15.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.552+0000 7fb21264f640 1 -- 172.21.15.98:0/2133150417 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb1f8067a70 msgr2=0x7fb1f8069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:15.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.552+0000 7fb21264f640 1 --2- 172.21.15.98:0/2133150417 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb1f8067a70 0x7fb1f8069f30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb2000096f0 tx=0x7fb200008040 comp rx=0 tx=0).stop 2024-09-19T08:44:15.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.552+0000 7fb21264f640 1 -- 172.21.15.98:0/2133150417 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb20c0b6200 msgr2=0x7fb20c154ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:15.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.552+0000 7fb21264f640 1 --2- 172.21.15.98:0/2133150417 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb20c0b6200 0x7fb20c154ac0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc0ad040 tx=0x7fb1fc01a3e0 comp rx=0 tx=0).stop 2024-09-19T08:44:15.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.552+0000 7fb21264f640 1 -- 172.21.15.98:0/2133150417 shutdown_connections 2024-09-19T08:44:15.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.552+0000 7fb21264f640 1 --2- 172.21.15.98:0/2133150417 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb1f8067a70 0x7fb1f8069f30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:15.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.552+0000 7fb21264f640 1 --2- 172.21.15.98:0/2133150417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb20c15a930 0x7fb20c155000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:15.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.552+0000 7fb21264f640 1 --2- 172.21.15.98:0/2133150417 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb20c0b6200 0x7fb20c154ac0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:15.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.552+0000 7fb21264f640 1 --2- 172.21.15.98:0/2133150417 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb20c0b3b60 0x7fb20c072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:15.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.552+0000 7fb21264f640 1 -- 172.21.15.98:0/2133150417 >> 172.21.15.98:0/2133150417 conn(0x7fb20c06c090 msgr2=0x7fb20c15af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:15.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.552+0000 7fb21264f640 1 -- 172.21.15.98:0/2133150417 shutdown_connections 2024-09-19T08:44:15.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.552+0000 7fb21264f640 1 -- 172.21.15.98:0/2133150417 wait complete. 2024-09-19T08:44:15.571 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs subvol41500 subvol_snap63800 subvol_clone38220 2024-09-19T08:44:15.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.700+0000 7f3c98569640 1 Processor -- start 2024-09-19T08:44:15.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.700+0000 7f3c98569640 1 -- start start 2024-09-19T08:44:15.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.700+0000 7f3c98569640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3c9006c3a0 0x7f3c900713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:15.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.700+0000 7f3c98569640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c900718f0 0x7f3c900b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:15.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.700+0000 7f3c98569640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c90073530 con 0x7f3c90145b70 2024-09-19T08:44:15.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.700+0000 7f3c98569640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f3c90073810 con 0x7f3c9006c3a0 2024-09-19T08:44:15.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.700+0000 7f3c98569640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3c90073af0 con 0x7f3c900718f0 2024-09-19T08:44:15.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.700+0000 7f3c95b04640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3c9006c3a0 0x7f3c900713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:15.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.700+0000 7f3c96305640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f3c90145b70 0x7f3c9006bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42556/0 (socket says 172.21.15.98:42556) 2024-09-19T08:44:15.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.700+0000 7f3c96305640 1 -- 172.21.15.98:0/840364895 learned_addr learned my addr 172.21.15.98:0/840364895 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:15.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.700+0000 7f3c96b06640 1 --2- 172.21.15.98:0/840364895 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c900718f0 0x7f3c900b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:15.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.700+0000 7f3c95b04640 1 -- 172.21.15.98:0/840364895 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c900718f0 msgr2=0x7f3c900b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:15.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.700+0000 7f3c95b04640 1 --2- 172.21.15.98:0/840364895 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c900718f0 0x7f3c900b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:15.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.700+0000 7f3c95b04640 1 -- 172.21.15.98:0/840364895 >> v1:172.21.15.98:6789/0 conn(0x7f3c90145b70 legacy=0x7f3c9006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:15.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.700+0000 7f3c95303640 1 -- 172.21.15.98:0/840364895 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 646100462 0 0) 0x7f3c90073530 con 0x7f3c90145b70 2024-09-19T08:44:15.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.700+0000 7f3c95b04640 1 -- 172.21.15.98:0/840364895 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c90073dd0 con 0x7f3c9006c3a0 2024-09-19T08:44:15.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.704+0000 7f3c95b04640 1 --2- 172.21.15.98:0/840364895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c9006c3a0 0x7f3c900713b0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f3c800097c0 tx=0x7f3c8001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85ce9785657f73a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:15.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.704+0000 7f3c95303640 1 -- 172.21.15.98:0/840364895 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c800b0070 con 0x7f3c9006c3a0 2024-09-19T08:44:15.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.704+0000 7f3c95303640 1 -- 172.21.15.98:0/840364895 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c800ab600 con 0x7f3c9006c3a0 2024-09-19T08:44:15.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.704+0000 7f3c95303640 1 -- 172.21.15.98:0/840364895 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c800ab9d0 con 0x7f3c9006c3a0 2024-09-19T08:44:15.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.704+0000 7f3c98569640 1 -- 172.21.15.98:0/840364895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c9006c3a0 msgr2=0x7f3c900713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:15.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.704+0000 7f3c98569640 1 --2- 172.21.15.98:0/840364895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c9006c3a0 0x7f3c900713b0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f3c800097c0 tx=0x7f3c8001c410 comp rx=0 tx=0).stop 2024-09-19T08:44:15.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.704+0000 7f3c98569640 1 -- 172.21.15.98:0/840364895 shutdown_connections 2024-09-19T08:44:15.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.704+0000 7f3c98569640 1 --2- 172.21.15.98:0/840364895 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c900718f0 0x7f3c900b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:15.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.704+0000 7f3c98569640 1 --2- 172.21.15.98:0/840364895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c9006c3a0 0x7f3c900713b0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:15.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.704+0000 7f3c98569640 1 -- 172.21.15.98:0/840364895 >> 172.21.15.98:0/840364895 conn(0x7f3c900b2010 msgr2=0x7f3c900b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:15.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.704+0000 7f3c98569640 1 -- 172.21.15.98:0/840364895 shutdown_connections 2024-09-19T08:44:15.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.704+0000 7f3c98569640 1 -- 172.21.15.98:0/840364895 wait complete. 2024-09-19T08:44:15.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.704+0000 7f3c98569640 1 Processor -- start 2024-09-19T08:44:15.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.704+0000 7f3c98569640 1 -- start start 2024-09-19T08:44:15.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.704+0000 7f3c98569640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c9006c3a0 0x7f3c900cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:15.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.704+0000 7f3c98569640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c900718f0 0x7f3c9015f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:15.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.704+0000 7f3c98569640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c90145b70 0x7f3c9013ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:15.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.704+0000 7f3c98569640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3c900b44f0 con 0x7f3c90145b70 2024-09-19T08:44:15.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.704+0000 7f3c98569640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3c900b4370 con 0x7f3c9006c3a0 2024-09-19T08:44:15.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.704+0000 7f3c98569640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3c900b4670 con 0x7f3c900718f0 2024-09-19T08:44:15.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.704+0000 7f3c96b06640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c90145b70 0x7f3c9013ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:15.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.704+0000 7f3c96b06640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c90145b70 0x7f3c9013ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39894/0 (socket says 172.21.15.98:39894) 2024-09-19T08:44:15.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.704+0000 7f3c96b06640 1 -- 172.21.15.98:0/3156465227 learned_addr learned my addr 172.21.15.98:0/3156465227 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:15.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.708+0000 7f3c96305640 1 --2- 172.21.15.98:0/3156465227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c9006c3a0 0x7f3c900cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:15.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.708+0000 7f3c95b04640 1 --2- 172.21.15.98:0/3156465227 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c900718f0 0x7f3c9015f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:15.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.708+0000 7f3c96b06640 1 -- 172.21.15.98:0/3156465227 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c900718f0 msgr2=0x7f3c9015f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:15.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.708+0000 7f3c96b06640 1 --2- 172.21.15.98:0/3156465227 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c900718f0 0x7f3c9015f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:15.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.708+0000 7f3c96b06640 1 -- 172.21.15.98:0/3156465227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c9006c3a0 msgr2=0x7f3c900cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:15.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.708+0000 7f3c96b06640 1 --2- 172.21.15.98:0/3156465227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c9006c3a0 0x7f3c900cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:15.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.708+0000 7f3c96b06640 1 -- 172.21.15.98:0/3156465227 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c9015b810 con 0x7f3c90145b70 2024-09-19T08:44:15.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.708+0000 7f3c95b04640 1 --2- 172.21.15.98:0/3156465227 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c900718f0 0x7f3c9015f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:15.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.708+0000 7f3c96b06640 1 --2- 172.21.15.98:0/3156465227 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c90145b70 0x7f3c9013ef20 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f3c88007e70 tx=0x7f3c8800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:15.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.708+0000 7f3c86ffd640 1 -- 172.21.15.98:0/3156465227 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c88014420 con 0x7f3c90145b70 2024-09-19T08:44:15.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.708+0000 7f3c86ffd640 1 -- 172.21.15.98:0/3156465227 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c88019070 con 0x7f3c90145b70 2024-09-19T08:44:15.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.708+0000 7f3c98569640 1 -- 172.21.15.98:0/3156465227 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c9015e520 con 0x7f3c90145b70 2024-09-19T08:44:15.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.708+0000 7f3c86ffd640 1 -- 172.21.15.98:0/3156465227 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c88014820 con 0x7f3c90145b70 2024-09-19T08:44:15.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.708+0000 7f3c98569640 1 -- 172.21.15.98:0/3156465227 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c9015b440 con 0x7f3c90145b70 2024-09-19T08:44:15.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.708+0000 7f3c98569640 1 -- 172.21.15.98:0/3156465227 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c90142a60 con 0x7f3c90145b70 2024-09-19T08:44:15.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.708+0000 7f3c86ffd640 1 -- 172.21.15.98:0/3156465227 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 17) ==== 87598+0+0 (secure 0 0 0) 0x7f3c880149c0 con 0x7f3c90145b70 2024-09-19T08:44:15.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.708+0000 7f3c86ffd640 1 --2- 172.21.15.98:0/3156465227 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3c74067a70 0x7f3c74069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:15.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.708+0000 7f3c86ffd640 1 -- 172.21.15.98:0/3156465227 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f3c880868e0 con 0x7f3c90145b70 2024-09-19T08:44:15.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.708+0000 7f3c96305640 1 --2- 172.21.15.98:0/3156465227 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3c74067a70 0x7f3c74069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:15.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.708+0000 7f3c96305640 1 --2- 172.21.15.98:0/3156465227 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3c74067a70 0x7f3c74069f30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c000f70 tx=0x7f3c8c007c10 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:15.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.712+0000 7f3c86ffd640 1 -- 172.21.15.98:0/3156465227 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3c88057160 con 0x7f3c90145b70 2024-09-19T08:44:15.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:15.908+0000 7f3c98569640 1 -- 172.21.15.98:0/3156465227 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvol41500", "snap_name": "subvol_snap63800", "target_sub_name": "subvol_clone38220", "target": ["mon-mgr", ""]}) -- 0x7f3c9015baf0 con 0x7f3c74067a70 2024-09-19T08:44:16.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.080+0000 7f3c86ffd640 1 -- 172.21.15.98:0/3156465227 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3c9015baf0 con 0x7f3c74067a70 2024-09-19T08:44:16.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.084+0000 7f3c98569640 1 -- 172.21.15.98:0/3156465227 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3c74067a70 msgr2=0x7f3c74069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:16.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.084+0000 7f3c98569640 1 --2- 172.21.15.98:0/3156465227 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3c74067a70 0x7f3c74069f30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c000f70 tx=0x7f3c8c007c10 comp rx=0 tx=0).stop 2024-09-19T08:44:16.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.084+0000 7f3c98569640 1 -- 172.21.15.98:0/3156465227 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c90145b70 msgr2=0x7f3c9013ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:16.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.084+0000 7f3c98569640 1 --2- 172.21.15.98:0/3156465227 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c90145b70 0x7f3c9013ef20 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f3c88007e70 tx=0x7f3c8800e4b0 comp rx=0 tx=0).stop 2024-09-19T08:44:16.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.084+0000 7f3c98569640 1 -- 172.21.15.98:0/3156465227 shutdown_connections 2024-09-19T08:44:16.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.084+0000 7f3c98569640 1 --2- 172.21.15.98:0/3156465227 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3c74067a70 0x7f3c74069f30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:16.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.084+0000 7f3c98569640 1 --2- 172.21.15.98:0/3156465227 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3c90145b70 0x7f3c9013ef20 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:16.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.084+0000 7f3c98569640 1 --2- 172.21.15.98:0/3156465227 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3c900718f0 0x7f3c9015f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:16.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.084+0000 7f3c98569640 1 --2- 172.21.15.98:0/3156465227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3c9006c3a0 0x7f3c900cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:16.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.084+0000 7f3c98569640 1 -- 172.21.15.98:0/3156465227 >> 172.21.15.98:0/3156465227 conn(0x7f3c900b2010 msgr2=0x7f3c901421b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:16.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.084+0000 7f3c98569640 1 -- 172.21.15.98:0/3156465227 shutdown_connections 2024-09-19T08:44:16.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.084+0000 7f3c98569640 1 -- 172.21.15.98:0/3156465227 wait complete. 2024-09-19T08:44:16.091 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:44:16.091+0000 7ff7406a8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:44:16.091 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:44:16.091+0000 7ff7406a8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:44:16.091 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:44:16.091+0000 7ff7406a8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:44:16.091 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:44:16.091+0000 7ff7406a8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:44:16.091 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:44:16.091+0000 7ff7406a8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:44:16.110 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone38220 2024-09-19T08:44:16.111 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:44:16.111+0000 7ff73fea7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:44:16.111 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:44:16.111+0000 7ff73fea7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:44:16.111 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:44:16.111+0000 7ff73fea7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:44:16.111 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:44:16.111+0000 7ff73fea7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:44:16.111 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:44:16.111+0000 7ff73fea7640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:44:16.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.252+0000 7f6c2ee33640 1 Processor -- start 2024-09-19T08:44:16.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.252+0000 7f6c2ee33640 1 -- start start 2024-09-19T08:44:16.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.252+0000 7f6c2ee33640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6c28158ae0 0x7f6c2815eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:16.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.252+0000 7f6c2ee33640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6c280b1a00 0x7f6c28073290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:16.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.252+0000 7f6c2ee33640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c28074060 con 0x7f6c280737d0 2024-09-19T08:44:16.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.252+0000 7f6c2ee33640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f6c280b2cd0 con 0x7f6c28158ae0 2024-09-19T08:44:16.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.252+0000 7f6c2ee33640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6c280b1e60 con 0x7f6c280b1a00 2024-09-19T08:44:16.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.252+0000 7f6c2cbcf640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6c28158ae0 0x7f6c2815eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:16.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.252+0000 7f6c2cbcf640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6c28158ae0 0x7f6c2815eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:47944/0 (socket says 172.21.15.98:47944) 2024-09-19T08:44:16.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.252+0000 7f6c2d3d0640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f6c280737d0 0x7f6c28074c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:55528/0 (socket says 172.21.15.98:55528) 2024-09-19T08:44:16.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.252+0000 7f6c2cbcf640 1 -- 172.21.15.98:0/1577828409 learned_addr learned my addr 172.21.15.98:0/1577828409 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:16.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.252+0000 7f6c27fff640 1 --2- 172.21.15.98:0/1577828409 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6c280b1a00 0x7f6c28073290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:16.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.252+0000 7f6c2cbcf640 1 -- 172.21.15.98:0/1577828409 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6c280b1a00 msgr2=0x7f6c28073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:16.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.252+0000 7f6c2cbcf640 1 --2- 172.21.15.98:0/1577828409 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6c280b1a00 0x7f6c28073290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:16.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.252+0000 7f6c2cbcf640 1 -- 172.21.15.98:0/1577828409 >> v1:172.21.15.139:6789/0 conn(0x7f6c280737d0 legacy=0x7f6c28074c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:16.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.252+0000 7f6c2cbcf640 1 -- 172.21.15.98:0/1577828409 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c280b20c0 con 0x7f6c28158ae0 2024-09-19T08:44:16.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.256+0000 7f6c2cbcf640 1 --2- 172.21.15.98:0/1577828409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c28158ae0 0x7f6c2815eb30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c0098a0 tx=0x7f6c1c01c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a1489f88c8a67bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:16.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.256+0000 7f6c277fe640 1 -- 172.21.15.98:0/1577828409 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c0b0070 con 0x7f6c28158ae0 2024-09-19T08:44:16.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.256+0000 7f6c277fe640 1 -- 172.21.15.98:0/1577828409 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c1c004230 con 0x7f6c28158ae0 2024-09-19T08:44:16.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.256+0000 7f6c277fe640 1 -- 172.21.15.98:0/1577828409 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c003620 con 0x7f6c28158ae0 2024-09-19T08:44:16.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.256+0000 7f6c2ee33640 1 -- 172.21.15.98:0/1577828409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c28158ae0 msgr2=0x7f6c2815eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:16.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.256+0000 7f6c2ee33640 1 --2- 172.21.15.98:0/1577828409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c28158ae0 0x7f6c2815eb30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c0098a0 tx=0x7f6c1c01c540 comp rx=0 tx=0).stop 2024-09-19T08:44:16.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.256+0000 7f6c2ee33640 1 -- 172.21.15.98:0/1577828409 shutdown_connections 2024-09-19T08:44:16.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.256+0000 7f6c2ee33640 1 --2- 172.21.15.98:0/1577828409 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6c280b1a00 0x7f6c28073290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:16.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.256+0000 7f6c2ee33640 1 --2- 172.21.15.98:0/1577828409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c28158ae0 0x7f6c2815eb30 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:16.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.256+0000 7f6c2ee33640 1 -- 172.21.15.98:0/1577828409 >> 172.21.15.98:0/1577828409 conn(0x7f6c280b4d60 msgr2=0x7f6c280b5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:16.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.256+0000 7f6c2ee33640 1 -- 172.21.15.98:0/1577828409 shutdown_connections 2024-09-19T08:44:16.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.256+0000 7f6c2ee33640 1 -- 172.21.15.98:0/1577828409 wait complete. 2024-09-19T08:44:16.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.260+0000 7f6c2ee33640 1 Processor -- start 2024-09-19T08:44:16.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.260+0000 7f6c2ee33640 1 -- start start 2024-09-19T08:44:16.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.260+0000 7f6c2ee33640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c280737d0 0x7f6c28154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:16.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.260+0000 7f6c2ee33640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6c280b1a00 0x7f6c28157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:16.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.260+0000 7f6c2ee33640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c28158ae0 0x7f6c2814ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:16.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.260+0000 7f6c2ee33640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6c28158ec0 con 0x7f6c280737d0 2024-09-19T08:44:16.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.260+0000 7f6c2ee33640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2806d190 con 0x7f6c28158ae0 2024-09-19T08:44:16.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.260+0000 7f6c2ee33640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6c28159040 con 0x7f6c280b1a00 2024-09-19T08:44:16.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.260+0000 7f6c2cbcf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c280737d0 0x7f6c28154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:16.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.260+0000 7f6c2cbcf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c280737d0 0x7f6c28154010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39902/0 (socket says 172.21.15.98:39902) 2024-09-19T08:44:16.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.260+0000 7f6c2cbcf640 1 -- 172.21.15.98:0/1755116347 learned_addr learned my addr 172.21.15.98:0/1755116347 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:16.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.260+0000 7f6c27fff640 1 --2- 172.21.15.98:0/1755116347 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6c280b1a00 0x7f6c28157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:16.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.260+0000 7f6c2cbcf640 1 -- 172.21.15.98:0/1755116347 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6c280b1a00 msgr2=0x7f6c28157780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:16.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.260+0000 7f6c2cbcf640 1 --2- 172.21.15.98:0/1755116347 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6c280b1a00 0x7f6c28157780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:16.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.260+0000 7f6c2d3d0640 1 --2- 172.21.15.98:0/1755116347 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c28158ae0 0x7f6c2814ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:16.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.260+0000 7f6c2cbcf640 1 -- 172.21.15.98:0/1755116347 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c28158ae0 msgr2=0x7f6c2814ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:16.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.260+0000 7f6c2cbcf640 1 --2- 172.21.15.98:0/1755116347 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c28158ae0 0x7f6c2814ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:16.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.260+0000 7f6c2cbcf640 1 -- 172.21.15.98:0/1755116347 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c28157cc0 con 0x7f6c280737d0 2024-09-19T08:44:16.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.260+0000 7f6c2d3d0640 1 --2- 172.21.15.98:0/1755116347 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c28158ae0 0x7f6c2814ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:16.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.260+0000 7f6c2cbcf640 1 --2- 172.21.15.98:0/1755116347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c280737d0 0x7f6c28154010 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c0ae040 tx=0x7f6c1c0192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:16.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.260+0000 7f6c257fa640 1 -- 172.21.15.98:0/1755116347 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c0b9020 con 0x7f6c280737d0 2024-09-19T08:44:16.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.260+0000 7f6c257fa640 1 -- 172.21.15.98:0/1755116347 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c1c0b1040 con 0x7f6c280737d0 2024-09-19T08:44:16.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.260+0000 7f6c257fa640 1 -- 172.21.15.98:0/1755116347 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c0b0050 con 0x7f6c280737d0 2024-09-19T08:44:16.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.260+0000 7f6c2ee33640 1 -- 172.21.15.98:0/1755116347 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c28156fc0 con 0x7f6c280737d0 2024-09-19T08:44:16.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.260+0000 7f6c2ee33640 1 -- 172.21.15.98:0/1755116347 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c28154b90 con 0x7f6c280737d0 2024-09-19T08:44:16.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.264+0000 7f6c257fa640 1 -- 172.21.15.98:0/1755116347 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 17) ==== 87598+0+0 (secure 0 0 0) 0x7f6c1c0bc020 con 0x7f6c280737d0 2024-09-19T08:44:16.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.264+0000 7f6c2ee33640 1 -- 172.21.15.98:0/1755116347 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c2815fe10 con 0x7f6c280737d0 2024-09-19T08:44:16.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.264+0000 7f6c257fa640 1 --2- 172.21.15.98:0/1755116347 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6c0c067a70 0x7f6c0c069f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:16.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.264+0000 7f6c257fa640 1 -- 172.21.15.98:0/1755116347 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f6c1c11fa50 con 0x7f6c280737d0 2024-09-19T08:44:16.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.264+0000 7f6c27fff640 1 --2- 172.21.15.98:0/1755116347 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6c0c067a70 0x7f6c0c069f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:16.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.264+0000 7f6c27fff640 1 --2- 172.21.15.98:0/1755116347 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6c0c067a70 0x7f6c0c069f30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6c2814f5e0 tx=0x7f6c14008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:16.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.268+0000 7f6c257fa640 1 -- 172.21.15.98:0/1755116347 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6c1c0c3070 con 0x7f6c280737d0 2024-09-19T08:44:16.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:16.480+0000 7f6c2ee33640 1 -- 172.21.15.98:0/1755116347 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone38220", "target": ["mon-mgr", ""]}) -- 0x7f6c281545f0 con 0x7f6c0c067a70 2024-09-19T08:44:18.082 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:44:18.086 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:44:18.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:18.356+0000 7f6c257fa640 1 -- 172.21.15.98:0/1755116347 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mgrmap(e 18) ==== 87700+0+0 (secure 0 0 0) 0x7f6c1c0ec140 con 0x7f6c280737d0 2024-09-19T08:44:20.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.208+0000 7f6c257fa640 1 -- 172.21.15.98:0/1755116347 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7f6c281545f0 con 0x7f6c0c067a70 2024-09-19T08:44:20.212 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:44:20.212 INFO:teuthology.orchestra.run.smithi098.stdout: "status": { 2024-09-19T08:44:20.212 INFO:teuthology.orchestra.run.smithi098.stdout: "state": "complete" 2024-09-19T08:44:20.212 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:44:20.212 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:44:20.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.212+0000 7f6c2ee33640 1 -- 172.21.15.98:0/1755116347 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6c0c067a70 msgr2=0x7f6c0c069f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:20.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.212+0000 7f6c2ee33640 1 --2- 172.21.15.98:0/1755116347 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6c0c067a70 0x7f6c0c069f30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6c2814f5e0 tx=0x7f6c14008040 comp rx=0 tx=0).stop 2024-09-19T08:44:20.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.212+0000 7f6c2ee33640 1 -- 172.21.15.98:0/1755116347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c280737d0 msgr2=0x7f6c28154010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:20.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.212+0000 7f6c2ee33640 1 --2- 172.21.15.98:0/1755116347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c280737d0 0x7f6c28154010 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c0ae040 tx=0x7f6c1c0192e0 comp rx=0 tx=0).stop 2024-09-19T08:44:20.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.212+0000 7f6c2ee33640 1 -- 172.21.15.98:0/1755116347 shutdown_connections 2024-09-19T08:44:20.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.212+0000 7f6c2ee33640 1 --2- 172.21.15.98:0/1755116347 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6c0c067a70 0x7f6c0c069f30 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:20.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.212+0000 7f6c2ee33640 1 --2- 172.21.15.98:0/1755116347 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c28158ae0 0x7f6c2814ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:20.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.212+0000 7f6c2ee33640 1 --2- 172.21.15.98:0/1755116347 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6c280b1a00 0x7f6c28157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:20.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.212+0000 7f6c2ee33640 1 --2- 172.21.15.98:0/1755116347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6c280737d0 0x7f6c28154010 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:20.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.212+0000 7f6c2ee33640 1 -- 172.21.15.98:0/1755116347 >> 172.21.15.98:0/1755116347 conn(0x7f6c280b4d60 msgr2=0x7f6c280afc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:20.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.212+0000 7f6c2ee33640 1 -- 172.21.15.98:0/1755116347 shutdown_connections 2024-09-19T08:44:20.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.212+0000 7f6c2ee33640 1 -- 172.21.15.98:0/1755116347 wait complete. 2024-09-19T08:44:20.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:20.234 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol_clone38220/.meta) 2024-09-19T08:44:20.249 INFO:teuthology.orchestra.run.smithi098.stdout:[GLOBAL] 2024-09-19T08:44:20.249 INFO:teuthology.orchestra.run.smithi098.stdout:version = 2 2024-09-19T08:44:20.249 INFO:teuthology.orchestra.run.smithi098.stdout:type = clone 2024-09-19T08:44:20.249 INFO:teuthology.orchestra.run.smithi098.stdout:path = /volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148 2024-09-19T08:44:20.249 INFO:teuthology.orchestra.run.smithi098.stdout:state = complete 2024-09-19T08:44:20.249 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:44:20.250 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol41500 2024-09-19T08:44:20.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.416+0000 7f97eaf60640 1 Processor -- start 2024-09-19T08:44:20.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.416+0000 7f97eaf60640 1 -- start start 2024-09-19T08:44:20.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.420+0000 7f97eaf60640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f97e406da30 0x7f97e406f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:20.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.420+0000 7f97eaf60640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97e415ea20 con 0x7f97e406f820 2024-09-19T08:44:20.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.420+0000 7f97eaf60640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97e4070980 con 0x7f97e40b7190 2024-09-19T08:44:20.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.420+0000 7f97eaf60640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f97e4070c40 con 0x7f97e406da30 2024-09-19T08:44:20.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.420+0000 7f97e8cfc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f97e406da30 0x7f97e406f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:20.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.420+0000 7f97e3fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f97e406f820 0x7f97e40b6a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:34530/0 (socket says 172.21.15.98:34530) 2024-09-19T08:44:20.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.420+0000 7f97e3fff640 1 -- 172.21.15.98:0/2917736865 learned_addr learned my addr 172.21.15.98:0/2917736865 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:20.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.420+0000 7f97e8cfc640 1 -- 172.21.15.98:0/2917736865 >> v1:172.21.15.139:6789/0 conn(0x7f97e40b7190 legacy=0x7f97e406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:20.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.420+0000 7f97e8cfc640 1 -- 172.21.15.98:0/2917736865 >> v1:172.21.15.98:6789/0 conn(0x7f97e406f820 legacy=0x7f97e40b6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:20.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.420+0000 7f97e8cfc640 1 -- 172.21.15.98:0/2917736865 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97e4070f20 con 0x7f97e406da30 2024-09-19T08:44:20.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.420+0000 7f97e37fe640 1 -- 172.21.15.98:0/2917736865 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2965857432 0 0) 0x7f97e415ea20 con 0x7f97e406f820 2024-09-19T08:44:20.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.420+0000 7f97e8cfc640 1 --2- 172.21.15.98:0/2917736865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97e406da30 0x7f97e406f2e0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f97dc009d90 tx=0x7f97dc01c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c809d074d8cd4d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:20.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.420+0000 7f97e37fe640 1 -- 172.21.15.98:0/2917736865 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc0b2070 con 0x7f97e406da30 2024-09-19T08:44:20.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.420+0000 7f97e37fe640 1 -- 172.21.15.98:0/2917736865 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97dc0a57e0 con 0x7f97e406da30 2024-09-19T08:44:20.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.420+0000 7f97e37fe640 1 -- 172.21.15.98:0/2917736865 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97dc0a5bb0 con 0x7f97e406da30 2024-09-19T08:44:20.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.424+0000 7f97eaf60640 1 -- 172.21.15.98:0/2917736865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97e406da30 msgr2=0x7f97e406f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:20.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.424+0000 7f97eaf60640 1 --2- 172.21.15.98:0/2917736865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97e406da30 0x7f97e406f2e0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f97dc009d90 tx=0x7f97dc01c810 comp rx=0 tx=0).stop 2024-09-19T08:44:20.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.424+0000 7f97eaf60640 1 -- 172.21.15.98:0/2917736865 shutdown_connections 2024-09-19T08:44:20.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.424+0000 7f97eaf60640 1 --2- 172.21.15.98:0/2917736865 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97e406da30 0x7f97e406f2e0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:20.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.424+0000 7f97eaf60640 1 -- 172.21.15.98:0/2917736865 >> 172.21.15.98:0/2917736865 conn(0x7f97e406dea0 msgr2=0x7f97e406e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:20.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.424+0000 7f97eaf60640 1 -- 172.21.15.98:0/2917736865 shutdown_connections 2024-09-19T08:44:20.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.424+0000 7f97eaf60640 1 -- 172.21.15.98:0/2917736865 wait complete. 2024-09-19T08:44:20.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.424+0000 7f97eaf60640 1 Processor -- start 2024-09-19T08:44:20.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.424+0000 7f97eaf60640 1 -- start start 2024-09-19T08:44:20.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.424+0000 7f97eaf60640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97e406da30 0x7f97e4156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:20.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.424+0000 7f97eaf60640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97e406f820 0x7f97e4157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:20.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.424+0000 7f97eaf60640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f97e40b7190 0x7f97e415a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:20.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.424+0000 7f97eaf60640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f97e416c480 con 0x7f97e406f820 2024-09-19T08:44:20.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.424+0000 7f97eaf60640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f97e406fc70 con 0x7f97e406da30 2024-09-19T08:44:20.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.424+0000 7f97eaf60640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f97e416c600 con 0x7f97e40b7190 2024-09-19T08:44:20.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.424+0000 7f97e3fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97e406f820 0x7f97e4157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:20.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.424+0000 7f97e3fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97e406f820 0x7f97e4157e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42154/0 (socket says 172.21.15.98:42154) 2024-09-19T08:44:20.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.424+0000 7f97e3fff640 1 -- 172.21.15.98:0/3108354863 learned_addr learned my addr 172.21.15.98:0/3108354863 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:20.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.424+0000 7f97e8cfc640 1 --2- 172.21.15.98:0/3108354863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97e406da30 0x7f97e4156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:20.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.424+0000 7f97e94fd640 1 --2- 172.21.15.98:0/3108354863 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f97e40b7190 0x7f97e415a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:20.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.424+0000 7f97e3fff640 1 -- 172.21.15.98:0/3108354863 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f97e40b7190 msgr2=0x7f97e415a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:20.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.424+0000 7f97e3fff640 1 --2- 172.21.15.98:0/3108354863 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f97e40b7190 0x7f97e415a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:20.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.424+0000 7f97e3fff640 1 -- 172.21.15.98:0/3108354863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97e406da30 msgr2=0x7f97e4156580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:20.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.424+0000 7f97e3fff640 1 --2- 172.21.15.98:0/3108354863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97e406da30 0x7f97e4156580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:20.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.424+0000 7f97e3fff640 1 -- 172.21.15.98:0/3108354863 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97e41573a0 con 0x7f97e406f820 2024-09-19T08:44:20.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.424+0000 7f97e3fff640 1 --2- 172.21.15.98:0/3108354863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97e406f820 0x7f97e4157e40 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00c4b0 tx=0x7f97cc00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:20.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.424+0000 7f97e17fa640 1 -- 172.21.15.98:0/3108354863 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc018070 con 0x7f97e406f820 2024-09-19T08:44:20.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.424+0000 7f97e17fa640 1 -- 172.21.15.98:0/3108354863 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97cc007500 con 0x7f97e406f820 2024-09-19T08:44:20.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.424+0000 7f97eaf60640 1 -- 172.21.15.98:0/3108354863 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97e415e170 con 0x7f97e406f820 2024-09-19T08:44:20.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.424+0000 7f97e17fa640 1 -- 172.21.15.98:0/3108354863 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc004a30 con 0x7f97e406f820 2024-09-19T08:44:20.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.424+0000 7f97eaf60640 1 -- 172.21.15.98:0/3108354863 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97e4157920 con 0x7f97e406f820 2024-09-19T08:44:20.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.428+0000 7f97eaf60640 1 -- 172.21.15.98:0/3108354863 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97e406f2e0 con 0x7f97e406f820 2024-09-19T08:44:20.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.428+0000 7f97e17fa640 1 -- 172.21.15.98:0/3108354863 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 87700+0+0 (secure 0 0 0) 0x7f97cc01b070 con 0x7f97e406f820 2024-09-19T08:44:20.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.428+0000 7f97e17fa640 1 --2- 172.21.15.98:0/3108354863 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f97c8067b90 0x7f97c806a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:20.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.428+0000 7f97e17fa640 1 -- 172.21.15.98:0/3108354863 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f97cc085370 con 0x7f97e406f820 2024-09-19T08:44:20.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.428+0000 7f97e8cfc640 1 --2- 172.21.15.98:0/3108354863 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f97c8067b90 0x7f97c806a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:20.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.428+0000 7f97e8cfc640 1 --2- 172.21.15.98:0/3108354863 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f97c8067b90 0x7f97c806a050 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f97dc0af000 tx=0x7f97dc0a7070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:20.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.432+0000 7f97e17fa640 1 -- 172.21.15.98:0/3108354863 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f97cc055b20 con 0x7f97e406f820 2024-09-19T08:44:20.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.620+0000 7f97eaf60640 1 -- 172.21.15.98:0/3108354863 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol41500", "target": ["mon-mgr", ""]}) -- 0x7f97e40b2750 con 0x7f97c8067b90 2024-09-19T08:44:20.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.652+0000 7f97e17fa640 1 -- 172.21.15.98:0/3108354863 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f97e40b2750 con 0x7f97c8067b90 2024-09-19T08:44:20.653 INFO:teuthology.orchestra.run.smithi098.stdout:/volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a 2024-09-19T08:44:20.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.652+0000 7f97eaf60640 1 -- 172.21.15.98:0/3108354863 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f97c8067b90 msgr2=0x7f97c806a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:20.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.652+0000 7f97eaf60640 1 --2- 172.21.15.98:0/3108354863 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f97c8067b90 0x7f97c806a050 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f97dc0af000 tx=0x7f97dc0a7070 comp rx=0 tx=0).stop 2024-09-19T08:44:20.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.652+0000 7f97eaf60640 1 -- 172.21.15.98:0/3108354863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97e406f820 msgr2=0x7f97e4157e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:20.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.652+0000 7f97eaf60640 1 --2- 172.21.15.98:0/3108354863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97e406f820 0x7f97e4157e40 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00c4b0 tx=0x7f97cc00c970 comp rx=0 tx=0).stop 2024-09-19T08:44:20.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.652+0000 7f97eaf60640 1 -- 172.21.15.98:0/3108354863 shutdown_connections 2024-09-19T08:44:20.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.652+0000 7f97eaf60640 1 --2- 172.21.15.98:0/3108354863 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f97c8067b90 0x7f97c806a050 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:20.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.652+0000 7f97eaf60640 1 --2- 172.21.15.98:0/3108354863 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f97e40b7190 0x7f97e415a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:20.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.652+0000 7f97eaf60640 1 --2- 172.21.15.98:0/3108354863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97e406f820 0x7f97e4157e40 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:20.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.652+0000 7f97eaf60640 1 --2- 172.21.15.98:0/3108354863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f97e406da30 0x7f97e4156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:20.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.652+0000 7f97eaf60640 1 -- 172.21.15.98:0/3108354863 >> 172.21.15.98:0/3108354863 conn(0x7f97e406dea0 msgr2=0x7f97e415e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:20.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.652+0000 7f97eaf60640 1 -- 172.21.15.98:0/3108354863 shutdown_connections 2024-09-19T08:44:20.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.652+0000 7f97eaf60640 1 -- 172.21.15.98:0/3108354863 wait complete. 2024-09-19T08:44:20.673 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol_clone38220 2024-09-19T08:44:20.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.800+0000 7f4945123640 1 Processor -- start 2024-09-19T08:44:20.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.800+0000 7f4945123640 1 -- start start 2024-09-19T08:44:20.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.800+0000 7f4945123640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4940145b70 0x7f494006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:20.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.800+0000 7f4945123640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4940071ac0 0x7f49400b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:20.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.800+0000 7f4945123640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4940073530 con 0x7f494006c1d0 2024-09-19T08:44:20.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.800+0000 7f4945123640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f494006c620 con 0x7f4940145b70 2024-09-19T08:44:20.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.800+0000 7f4945123640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4940073810 con 0x7f4940071ac0 2024-09-19T08:44:20.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.800+0000 7f493ed9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4940145b70 0x7f494006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:20.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.800+0000 7f493ed9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4940145b70 0x7f494006bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42166/0 (socket says 172.21.15.98:42166) 2024-09-19T08:44:20.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.800+0000 7f493ed9d640 1 -- 172.21.15.98:0/443775683 learned_addr learned my addr 172.21.15.98:0/443775683 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:20.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.800+0000 7f493f59e640 1 --2- 172.21.15.98:0/443775683 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4940071ac0 0x7f49400b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:20.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.800+0000 7f493ed9d640 1 -- 172.21.15.98:0/443775683 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4940071ac0 msgr2=0x7f49400b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:20.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.800+0000 7f493ed9d640 1 --2- 172.21.15.98:0/443775683 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4940071ac0 0x7f49400b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:20.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.800+0000 7f493ed9d640 1 -- 172.21.15.98:0/443775683 >> v1:172.21.15.139:6789/0 conn(0x7f494006c1d0 legacy=0x7f49400713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:20.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.800+0000 7f493ed9d640 1 -- 172.21.15.98:0/443775683 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4940073af0 con 0x7f4940145b70 2024-09-19T08:44:20.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.800+0000 7f493f59e640 1 --2- 172.21.15.98:0/443775683 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4940071ac0 0x7f49400b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:20.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.800+0000 7f493ed9d640 1 --2- 172.21.15.98:0/443775683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4940145b70 0x7f494006bc90 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f4938009e50 tx=0x7f493801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a262a6496bb119e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:20.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.804+0000 7f493dd9b640 1 -- 172.21.15.98:0/443775683 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49380a55e0 con 0x7f4940145b70 2024-09-19T08:44:20.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.804+0000 7f493dd9b640 1 -- 172.21.15.98:0/443775683 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49380a5780 con 0x7f4940145b70 2024-09-19T08:44:20.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.804+0000 7f493dd9b640 1 -- 172.21.15.98:0/443775683 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49380a5b50 con 0x7f4940145b70 2024-09-19T08:44:20.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.804+0000 7f4945123640 1 -- 172.21.15.98:0/443775683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4940145b70 msgr2=0x7f494006bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:20.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.804+0000 7f4945123640 1 --2- 172.21.15.98:0/443775683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4940145b70 0x7f494006bc90 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f4938009e50 tx=0x7f493801c960 comp rx=0 tx=0).stop 2024-09-19T08:44:20.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.804+0000 7f4945123640 1 -- 172.21.15.98:0/443775683 shutdown_connections 2024-09-19T08:44:20.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.804+0000 7f4945123640 1 --2- 172.21.15.98:0/443775683 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4940071ac0 0x7f49400b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:20.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.804+0000 7f4945123640 1 --2- 172.21.15.98:0/443775683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4940145b70 0x7f494006bc90 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:20.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.804+0000 7f4945123640 1 -- 172.21.15.98:0/443775683 >> 172.21.15.98:0/443775683 conn(0x7f49400b2010 msgr2=0x7f49400b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:20.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.804+0000 7f4945123640 1 -- 172.21.15.98:0/443775683 shutdown_connections 2024-09-19T08:44:20.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.804+0000 7f4945123640 1 -- 172.21.15.98:0/443775683 wait complete. 2024-09-19T08:44:20.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.804+0000 7f4945123640 1 Processor -- start 2024-09-19T08:44:20.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.804+0000 7f4945123640 1 -- start start 2024-09-19T08:44:20.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.804+0000 7f4945123640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f494006c1d0 0x7f49401315c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:20.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.804+0000 7f4945123640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4940071ac0 0x7f4940139780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:20.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.804+0000 7f4945123640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4940145b70 0x7f4940144670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:20.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.804+0000 7f4945123640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f49400b4530 con 0x7f4940145b70 2024-09-19T08:44:20.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.804+0000 7f4945123640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f49400b43b0 con 0x7f494006c1d0 2024-09-19T08:44:20.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.804+0000 7f4945123640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f49400b46b0 con 0x7f4940071ac0 2024-09-19T08:44:20.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.804+0000 7f493f59e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4940145b70 0x7f4940144670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:20.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.804+0000 7f493f59e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4940145b70 0x7f4940144670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42180/0 (socket says 172.21.15.98:42180) 2024-09-19T08:44:20.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.804+0000 7f493f59e640 1 -- 172.21.15.98:0/3429690460 learned_addr learned my addr 172.21.15.98:0/3429690460 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:20.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.804+0000 7f493e59c640 1 --2- 172.21.15.98:0/3429690460 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4940071ac0 0x7f4940139780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:20.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.804+0000 7f493ed9d640 1 --2- 172.21.15.98:0/3429690460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f494006c1d0 0x7f49401315c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:20.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.804+0000 7f493f59e640 1 -- 172.21.15.98:0/3429690460 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4940071ac0 msgr2=0x7f4940139780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:20.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.804+0000 7f493f59e640 1 --2- 172.21.15.98:0/3429690460 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4940071ac0 0x7f4940139780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:20.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.804+0000 7f493f59e640 1 -- 172.21.15.98:0/3429690460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f494006c1d0 msgr2=0x7f49401315c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:20.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.804+0000 7f493f59e640 1 --2- 172.21.15.98:0/3429690460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f494006c1d0 0x7f49401315c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:20.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.804+0000 7f493f59e640 1 -- 172.21.15.98:0/3429690460 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4940139cc0 con 0x7f4940145b70 2024-09-19T08:44:20.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.804+0000 7f493e59c640 1 --2- 172.21.15.98:0/3429690460 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4940071ac0 0x7f4940139780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:20.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.804+0000 7f493ed9d640 1 --2- 172.21.15.98:0/3429690460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f494006c1d0 0x7f49401315c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:20.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.804+0000 7f493f59e640 1 --2- 172.21.15.98:0/3429690460 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4940145b70 0x7f4940144670 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f493000c7f0 tx=0x7f493000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:20.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.804+0000 7f49277fe640 1 -- 172.21.15.98:0/3429690460 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4930015070 con 0x7f4940145b70 2024-09-19T08:44:20.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.804+0000 7f49277fe640 1 -- 172.21.15.98:0/3429690460 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4930007860 con 0x7f4940145b70 2024-09-19T08:44:20.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.804+0000 7f49277fe640 1 -- 172.21.15.98:0/3429690460 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4930007c50 con 0x7f4940145b70 2024-09-19T08:44:20.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.808+0000 7f4945123640 1 -- 172.21.15.98:0/3429690460 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4940143e80 con 0x7f4940145b70 2024-09-19T08:44:20.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.808+0000 7f4945123640 1 -- 172.21.15.98:0/3429690460 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4940134e20 con 0x7f4940145b70 2024-09-19T08:44:20.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.808+0000 7f4945123640 1 -- 172.21.15.98:0/3429690460 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49400b3410 con 0x7f4940145b70 2024-09-19T08:44:20.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.808+0000 7f49277fe640 1 -- 172.21.15.98:0/3429690460 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 87700+0+0 (secure 0 0 0) 0x7f4930018070 con 0x7f4940145b70 2024-09-19T08:44:20.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.808+0000 7f49277fe640 1 --2- 172.21.15.98:0/3429690460 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f492c067b90 0x7f492c06a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:20.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.808+0000 7f49277fe640 1 -- 172.21.15.98:0/3429690460 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f4930086340 con 0x7f4940145b70 2024-09-19T08:44:20.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.808+0000 7f493ed9d640 1 --2- 172.21.15.98:0/3429690460 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f492c067b90 0x7f492c06a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:20.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.808+0000 7f493ed9d640 1 --2- 172.21.15.98:0/3429690460 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f492c067b90 0x7f492c06a050 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f49380ae000 tx=0x7f49380a7070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:20.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:20.812+0000 7f49277fe640 1 -- 172.21.15.98:0/3429690460 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f4930056a70 con 0x7f4940145b70 2024-09-19T08:44:21.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.000+0000 7f4945123640 1 -- 172.21.15.98:0/3429690460 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol_clone38220", "target": ["mon-mgr", ""]}) -- 0x7f49400affa0 con 0x7f492c067b90 2024-09-19T08:44:21.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.012+0000 7f49277fe640 1 -- 172.21.15.98:0/3429690460 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+73 (secure 0 0 0) 0x7f49400affa0 con 0x7f492c067b90 2024-09-19T08:44:21.014 INFO:teuthology.orchestra.run.smithi098.stdout:/volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148 2024-09-19T08:44:21.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.016+0000 7f4945123640 1 -- 172.21.15.98:0/3429690460 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f492c067b90 msgr2=0x7f492c06a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:21.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.016+0000 7f4945123640 1 --2- 172.21.15.98:0/3429690460 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f492c067b90 0x7f492c06a050 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f49380ae000 tx=0x7f49380a7070 comp rx=0 tx=0).stop 2024-09-19T08:44:21.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.016+0000 7f4945123640 1 -- 172.21.15.98:0/3429690460 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4940145b70 msgr2=0x7f4940144670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:21.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.016+0000 7f4945123640 1 --2- 172.21.15.98:0/3429690460 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4940145b70 0x7f4940144670 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f493000c7f0 tx=0x7f493000ccb0 comp rx=0 tx=0).stop 2024-09-19T08:44:21.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.016+0000 7f4945123640 1 -- 172.21.15.98:0/3429690460 shutdown_connections 2024-09-19T08:44:21.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.016+0000 7f4945123640 1 --2- 172.21.15.98:0/3429690460 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f492c067b90 0x7f492c06a050 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:21.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.016+0000 7f4945123640 1 --2- 172.21.15.98:0/3429690460 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4940145b70 0x7f4940144670 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:21.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.016+0000 7f4945123640 1 --2- 172.21.15.98:0/3429690460 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4940071ac0 0x7f4940139780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:21.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.016+0000 7f4945123640 1 --2- 172.21.15.98:0/3429690460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f494006c1d0 0x7f49401315c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:21.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.016+0000 7f4945123640 1 -- 172.21.15.98:0/3429690460 >> 172.21.15.98:0/3429690460 conn(0x7f49400b2010 msgr2=0x7f49400b0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:21.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.016+0000 7f4945123640 1 -- 172.21.15.98:0/3429690460 shutdown_connections 2024-09-19T08:44:21.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.016+0000 7f4945123640 1 -- 172.21.15.98:0/3429690460 wait complete. 2024-09-19T08:44:21.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:21.033 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.rentries volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800) 2024-09-19T08:44:21.047 INFO:teuthology.orchestra.run.smithi098.stdout:9 2024-09-19T08:44:21.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:21.048 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.rentries volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148) 2024-09-19T08:44:21.099 INFO:teuthology.orchestra.run.smithi098.stdout:9 2024-09-19T08:44:21.099 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol_clone38220 2024-09-19T08:44:21.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.268+0000 7f329de67640 1 Processor -- start 2024-09-19T08:44:21.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.268+0000 7f329de67640 1 -- start start 2024-09-19T08:44:21.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.268+0000 7f329de67640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f329806c3a0 0x7f32980713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:21.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.268+0000 7f329de67640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32980718f0 0x7f32980b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:21.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.268+0000 7f329de67640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3298073530 con 0x7f3298145b70 2024-09-19T08:44:21.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.268+0000 7f329de67640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f3298073810 con 0x7f329806c3a0 2024-09-19T08:44:21.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.268+0000 7f329de67640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3298073af0 con 0x7f32980718f0 2024-09-19T08:44:21.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.268+0000 7f32977fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f3298145b70 0x7f329806bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:34532/0 (socket says 172.21.15.98:34532) 2024-09-19T08:44:21.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.268+0000 7f32977fe640 1 -- 172.21.15.98:0/1136926808 learned_addr learned my addr 172.21.15.98:0/1136926808 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:21.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.268+0000 7f3296ffd640 1 -- 172.21.15.98:0/1136926808 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2166843859 0 0) 0x7f3298073530 con 0x7f3298145b70 2024-09-19T08:44:21.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.268+0000 7f3296ffd640 1 -- 172.21.15.98:0/1136926808 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3284003530 con 0x7f3298145b70 2024-09-19T08:44:21.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.268+0000 7f328ffff640 1 --2- 172.21.15.98:0/1136926808 >> v2:172.21.15.139:3300/0 conn(0x7f329806c3a0 0x7f32980713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:21.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.268+0000 7f3297fff640 1 --2- 172.21.15.98:0/1136926808 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32980718f0 0x7f32980b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:21.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.268+0000 7f3296ffd640 1 -- 172.21.15.98:0/1136926808 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1061016183 0 0) 0x7f3284003530 con 0x7f3298145b70 2024-09-19T08:44:21.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.268+0000 7f3296ffd640 1 -- 172.21.15.98:0/1136926808 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32980718f0 msgr2=0x7f32980b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:21.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.268+0000 7f3296ffd640 1 --2- 172.21.15.98:0/1136926808 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32980718f0 0x7f32980b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:21.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.268+0000 7f3296ffd640 1 -- 172.21.15.98:0/1136926808 >> v2:172.21.15.139:3300/0 conn(0x7f329806c3a0 msgr2=0x7f32980713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:21.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.268+0000 7f3296ffd640 1 --2- 172.21.15.98:0/1136926808 >> v2:172.21.15.139:3300/0 conn(0x7f329806c3a0 0x7f32980713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:21.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.268+0000 7f3296ffd640 1 -- 172.21.15.98:0/1136926808 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3298073dd0 con 0x7f3298145b70 2024-09-19T08:44:21.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.268+0000 7f3296ffd640 1 -- 172.21.15.98:0/1136926808 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f3288001680 con 0x7f3298145b70 2024-09-19T08:44:21.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.268+0000 7f3296ffd640 1 -- 172.21.15.98:0/1136926808 >> v1:172.21.15.98:6789/0 conn(0x7f3298145b70 legacy=0x7f329806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:21.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.268+0000 7f3296ffd640 1 --2- 172.21.15.98:0/1136926808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32840039c0 0x7f3284003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:21.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.268+0000 7f3296ffd640 1 --2- 172.21.15.98:0/1136926808 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32980718f0 0x7f3284004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:21.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.268+0000 7f3296ffd640 1 --2- 172.21.15.98:0/1136926808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3284004d50 0x7f32840051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:21.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.268+0000 7f3296ffd640 1 -- 172.21.15.98:0/1136926808 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3298073810 con 0x7f32840039c0 2024-09-19T08:44:21.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.268+0000 7f3296ffd640 1 -- 172.21.15.98:0/1136926808 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3298073af0 con 0x7f3284004d50 2024-09-19T08:44:21.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.268+0000 7f3296ffd640 1 -- 172.21.15.98:0/1136926808 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f328400bb30 con 0x7f32980718f0 2024-09-19T08:44:21.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.268+0000 7f32977fe640 1 --2- 172.21.15.98:0/1136926808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32840039c0 0x7f3284003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:21.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.268+0000 7f3297fff640 1 --2- 172.21.15.98:0/1136926808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3284004d50 0x7f32840051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:21.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.268+0000 7f328ffff640 1 --2- 172.21.15.98:0/1136926808 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32980718f0 0x7f3284004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:21.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.268+0000 7f32977fe640 1 -- 172.21.15.98:0/1136926808 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32980718f0 msgr2=0x7f3284004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:21.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.268+0000 7f32977fe640 1 --2- 172.21.15.98:0/1136926808 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32980718f0 0x7f3284004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:21.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.268+0000 7f32977fe640 1 -- 172.21.15.98:0/1136926808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3284004d50 msgr2=0x7f32840051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:21.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.268+0000 7f32977fe640 1 --2- 172.21.15.98:0/1136926808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3284004d50 0x7f32840051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:21.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.268+0000 7f32977fe640 1 -- 172.21.15.98:0/1136926808 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3284003530 con 0x7f32840039c0 2024-09-19T08:44:21.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.272+0000 7f32977fe640 1 --2- 172.21.15.98:0/1136926808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32840039c0 0x7f3284003de0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f328800c210 tx=0x7f328801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc722fdb07824881 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:21.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.272+0000 7f3296ffd640 1 -- 172.21.15.98:0/1136926808 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3288007d60 con 0x7f32840039c0 2024-09-19T08:44:21.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.272+0000 7f3296ffd640 1 -- 172.21.15.98:0/1136926808 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32880a9020 con 0x7f32840039c0 2024-09-19T08:44:21.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.272+0000 7f3296ffd640 1 -- 172.21.15.98:0/1136926808 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3288005b20 con 0x7f32840039c0 2024-09-19T08:44:21.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.272+0000 7f329de67640 1 -- 172.21.15.98:0/1136926808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32840039c0 msgr2=0x7f3284003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:21.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.272+0000 7f329de67640 1 --2- 172.21.15.98:0/1136926808 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32840039c0 0x7f3284003de0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f328800c210 tx=0x7f328801cca0 comp rx=0 tx=0).stop 2024-09-19T08:44:21.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.272+0000 7f3297fff640 1 -- 172.21.15.98:0/1136926808 reap_dead start 2024-09-19T08:44:21.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.272+0000 7f329de67640 1 -- 172.21.15.98:0/1136926808 shutdown_connections 2024-09-19T08:44:21.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.272+0000 7f329de67640 1 -- 172.21.15.98:0/1136926808 >> 172.21.15.98:0/1136926808 conn(0x7f32980b2010 msgr2=0x7f32980b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:21.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.272+0000 7f329de67640 1 -- 172.21.15.98:0/1136926808 shutdown_connections 2024-09-19T08:44:21.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.272+0000 7f329de67640 1 -- 172.21.15.98:0/1136926808 wait complete. 2024-09-19T08:44:21.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.272+0000 7f329de67640 1 Processor -- start 2024-09-19T08:44:21.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.272+0000 7f329de67640 1 -- start start 2024-09-19T08:44:21.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.272+0000 7f329de67640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32840039c0 0x7f32980cea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:21.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.272+0000 7f329de67640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f329815c070 0x7f32980cd740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:21.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.272+0000 7f329de67640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32980ae8b0 0x7f32980aed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:21.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.272+0000 7f329de67640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f32980b3fb0 con 0x7f32980ae8b0 2024-09-19T08:44:21.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.272+0000 7f329de67640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f32980b3e30 con 0x7f329815c070 2024-09-19T08:44:21.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.272+0000 7f329de67640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f32980b4130 con 0x7f32840039c0 2024-09-19T08:44:21.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.272+0000 7f3297fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32980ae8b0 0x7f32980aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:21.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.272+0000 7f3297fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32980ae8b0 0x7f32980aed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42196/0 (socket says 172.21.15.98:42196) 2024-09-19T08:44:21.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.272+0000 7f3297fff640 1 -- 172.21.15.98:0/2532429573 learned_addr learned my addr 172.21.15.98:0/2532429573 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:21.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.272+0000 7f32977fe640 1 --2- 172.21.15.98:0/2532429573 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32840039c0 0x7f32980cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:21.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.276+0000 7f328ffff640 1 --2- 172.21.15.98:0/2532429573 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f329815c070 0x7f32980cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:21.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.276+0000 7f3297fff640 1 -- 172.21.15.98:0/2532429573 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32840039c0 msgr2=0x7f32980cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:21.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.276+0000 7f3297fff640 1 --2- 172.21.15.98:0/2532429573 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32840039c0 0x7f32980cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:21.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.276+0000 7f3297fff640 1 -- 172.21.15.98:0/2532429573 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f329815c070 msgr2=0x7f32980cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:21.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.276+0000 7f3297fff640 1 --2- 172.21.15.98:0/2532429573 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f329815c070 0x7f32980cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:21.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.276+0000 7f3297fff640 1 -- 172.21.15.98:0/2532429573 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32980cdc80 con 0x7f32980ae8b0 2024-09-19T08:44:21.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.276+0000 7f328ffff640 1 --2- 172.21.15.98:0/2532429573 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f329815c070 0x7f32980cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:21.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.276+0000 7f3297fff640 1 --2- 172.21.15.98:0/2532429573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32980ae8b0 0x7f32980aed30 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f32980ccd30 tx=0x7f3290009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:21.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.276+0000 7f3294ff9640 1 -- 172.21.15.98:0/2532429573 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3290007c90 con 0x7f32980ae8b0 2024-09-19T08:44:21.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.276+0000 7f3294ff9640 1 -- 172.21.15.98:0/2532429573 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3290019070 con 0x7f32980ae8b0 2024-09-19T08:44:21.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.276+0000 7f329de67640 1 -- 172.21.15.98:0/2532429573 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32980c7130 con 0x7f32980ae8b0 2024-09-19T08:44:21.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.276+0000 7f3294ff9640 1 -- 172.21.15.98:0/2532429573 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3290014650 con 0x7f32980ae8b0 2024-09-19T08:44:21.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.276+0000 7f329de67640 1 -- 172.21.15.98:0/2532429573 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32980c76f0 con 0x7f32980ae8b0 2024-09-19T08:44:21.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.276+0000 7f329de67640 1 -- 172.21.15.98:0/2532429573 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f329815c450 con 0x7f32980ae8b0 2024-09-19T08:44:21.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.276+0000 7f3294ff9640 1 -- 172.21.15.98:0/2532429573 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 87700+0+0 (secure 0 0 0) 0x7f32900147f0 con 0x7f32980ae8b0 2024-09-19T08:44:21.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.276+0000 7f3294ff9640 1 --2- 172.21.15.98:0/2532429573 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3278067bc0 0x7f327806a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:21.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.276+0000 7f3294ff9640 1 -- 172.21.15.98:0/2532429573 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f3290087440 con 0x7f32980ae8b0 2024-09-19T08:44:21.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.276+0000 7f32977fe640 1 --2- 172.21.15.98:0/2532429573 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3278067bc0 0x7f327806a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:21.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.280+0000 7f32977fe640 1 --2- 172.21.15.98:0/2532429573 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3278067bc0 0x7f327806a080 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f32880aa000 tx=0x7f3288000ff0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:21.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.280+0000 7f3294ff9640 1 -- 172.21.15.98:0/2532429573 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f329815c450 con 0x7f32980ae8b0 2024-09-19T08:44:21.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.468+0000 7f329de67640 1 -- 172.21.15.98:0/2532429573 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol_clone38220", "target": ["mon-mgr", ""]}) -- 0x7f32980b4540 con 0x7f3278067bc0 2024-09-19T08:44:21.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.488+0000 7f3294ff9640 1 -- 172.21.15.98:0/2532429573 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+724 (secure 0 0 0) 0x7f32880a92f0 con 0x7f3278067bc0 2024-09-19T08:44:21.490 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:44:21.490 INFO:teuthology.orchestra.run.smithi098.stdout: "atime": "2024-09-19 08:44:10", 2024-09-19T08:44:21.490 INFO:teuthology.orchestra.run.smithi098.stdout: "bytes_pcent": "80.00", 2024-09-19T08:44:21.491 INFO:teuthology.orchestra.run.smithi098.stdout: "bytes_quota": 10485760, 2024-09-19T08:44:21.491 INFO:teuthology.orchestra.run.smithi098.stdout: "bytes_used": 8388608, 2024-09-19T08:44:21.491 INFO:teuthology.orchestra.run.smithi098.stdout: "created_at": "2024-09-19 08:44:15", 2024-09-19T08:44:21.491 INFO:teuthology.orchestra.run.smithi098.stdout: "ctime": "2024-09-19 08:44:16", 2024-09-19T08:44:21.491 INFO:teuthology.orchestra.run.smithi098.stdout: "data_pool": "cephfs_data", 2024-09-19T08:44:21.491 INFO:teuthology.orchestra.run.smithi098.stdout: "features": [ 2024-09-19T08:44:21.491 INFO:teuthology.orchestra.run.smithi098.stdout: "snapshot-clone", 2024-09-19T08:44:21.491 INFO:teuthology.orchestra.run.smithi098.stdout: "snapshot-autoprotect", 2024-09-19T08:44:21.491 INFO:teuthology.orchestra.run.smithi098.stdout: "snapshot-retention" 2024-09-19T08:44:21.491 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:44:21.491 INFO:teuthology.orchestra.run.smithi098.stdout: "flavor": 2, 2024-09-19T08:44:21.491 INFO:teuthology.orchestra.run.smithi098.stdout: "gid": 1000, 2024-09-19T08:44:21.492 INFO:teuthology.orchestra.run.smithi098.stdout: "mode": 16895, 2024-09-19T08:44:21.492 INFO:teuthology.orchestra.run.smithi098.stdout: "mon_addrs": [ 2024-09-19T08:44:21.492 INFO:teuthology.orchestra.run.smithi098.stdout: "172.21.15.98:6789", 2024-09-19T08:44:21.492 INFO:teuthology.orchestra.run.smithi098.stdout: "172.21.15.139:6789", 2024-09-19T08:44:21.492 INFO:teuthology.orchestra.run.smithi098.stdout: "172.21.15.139:6790" 2024-09-19T08:44:21.492 INFO:teuthology.orchestra.run.smithi098.stdout: ], 2024-09-19T08:44:21.492 INFO:teuthology.orchestra.run.smithi098.stdout: "mtime": "2024-09-19 08:44:15", 2024-09-19T08:44:21.492 INFO:teuthology.orchestra.run.smithi098.stdout: "path": "/volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148", 2024-09-19T08:44:21.492 INFO:teuthology.orchestra.run.smithi098.stdout: "pool_namespace": "", 2024-09-19T08:44:21.492 INFO:teuthology.orchestra.run.smithi098.stdout: "state": "complete", 2024-09-19T08:44:21.492 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "clone", 2024-09-19T08:44:21.492 INFO:teuthology.orchestra.run.smithi098.stdout: "uid": 1000 2024-09-19T08:44:21.492 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:44:21.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.492+0000 7f329de67640 1 -- 172.21.15.98:0/2532429573 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3278067bc0 msgr2=0x7f327806a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:21.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.492+0000 7f329de67640 1 --2- 172.21.15.98:0/2532429573 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3278067bc0 0x7f327806a080 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f32880aa000 tx=0x7f3288000ff0 comp rx=0 tx=0).stop 2024-09-19T08:44:21.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.492+0000 7f329de67640 1 -- 172.21.15.98:0/2532429573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32980ae8b0 msgr2=0x7f32980aed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:21.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.492+0000 7f329de67640 1 --2- 172.21.15.98:0/2532429573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32980ae8b0 0x7f32980aed30 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f32980ccd30 tx=0x7f3290009d40 comp rx=0 tx=0).stop 2024-09-19T08:44:21.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.492+0000 7f329de67640 1 -- 172.21.15.98:0/2532429573 shutdown_connections 2024-09-19T08:44:21.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.492+0000 7f329de67640 1 --2- 172.21.15.98:0/2532429573 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3278067bc0 0x7f327806a080 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:21.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.492+0000 7f329de67640 1 --2- 172.21.15.98:0/2532429573 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32980ae8b0 0x7f32980aed30 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:21.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.492+0000 7f329de67640 1 --2- 172.21.15.98:0/2532429573 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f329815c070 0x7f32980cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:21.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.492+0000 7f329de67640 1 --2- 172.21.15.98:0/2532429573 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32840039c0 0x7f32980cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:21.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.492+0000 7f329de67640 1 -- 172.21.15.98:0/2532429573 >> 172.21.15.98:0/2532429573 conn(0x7f32980b2010 msgr2=0x7f3298140730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:21.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.492+0000 7f329de67640 1 -- 172.21.15.98:0/2532429573 shutdown_connections 2024-09-19T08:44:21.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:21.492+0000 7f329de67640 1 -- 172.21.15.98:0/2532429573 wait complete. 2024-09-19T08:44:21.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:21.509 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.quota.max_bytes volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800) 2024-09-19T08:44:21.521 INFO:teuthology.orchestra.run.smithi098.stdout:10485760 2024-09-19T08:44:21.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:21.521 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800) 2024-09-19T08:44:21.574 INFO:teuthology.orchestra.run.smithi098.stdout:cephfs_data 2024-09-19T08:44:21.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:21.575 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool_namespace volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800) 2024-09-19T08:44:21.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:21.631 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec find volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800) 2024-09-19T08:44:21.692 INFO:teuthology.orchestra.run.smithi098.stdout:volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800 2024-09-19T08:44:21.692 INFO:teuthology.orchestra.run.smithi098.stdout:volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.2 2024-09-19T08:44:21.692 INFO:teuthology.orchestra.run.smithi098.stdout:volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.5 2024-09-19T08:44:21.692 INFO:teuthology.orchestra.run.smithi098.stdout:volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.4 2024-09-19T08:44:21.692 INFO:teuthology.orchestra.run.smithi098.stdout:volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.7 2024-09-19T08:44:21.692 INFO:teuthology.orchestra.run.smithi098.stdout:volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.6 2024-09-19T08:44:21.692 INFO:teuthology.orchestra.run.smithi098.stdout:volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.0 2024-09-19T08:44:21.692 INFO:teuthology.orchestra.run.smithi098.stdout:volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.3 2024-09-19T08:44:21.693 INFO:teuthology.orchestra.run.smithi098.stdout:volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.1 2024-09-19T08:44:21.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:21.693 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800) 2024-09-19T08:44:21.743 INFO:teuthology.orchestra.run.smithi098.stdout:41ff 2024-09-19T08:44:21.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:21.745 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148/) 2024-09-19T08:44:21.794 INFO:teuthology.orchestra.run.smithi098.stdout:41ff 2024-09-19T08:44:21.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:21.795 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800) 2024-09-19T08:44:21.845 INFO:teuthology.orchestra.run.smithi098.stdout:1000 2024-09-19T08:44:21.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:21.846 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148/) 2024-09-19T08:44:21.894 INFO:teuthology.orchestra.run.smithi098.stdout:1000 2024-09-19T08:44:21.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:21.895 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800) 2024-09-19T08:44:21.948 INFO:teuthology.orchestra.run.smithi098.stdout:1000 2024-09-19T08:44:21.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:21.949 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148/) 2024-09-19T08:44:21.999 INFO:teuthology.orchestra.run.smithi098.stdout:1000 2024-09-19T08:44:21.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:22.000 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800) 2024-09-19T08:44:22.049 INFO:teuthology.orchestra.run.smithi098.stdout:1726735455 2024-09-19T08:44:22.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:22.050 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148/) 2024-09-19T08:44:22.098 INFO:teuthology.orchestra.run.smithi098.stdout:1726735455 2024-09-19T08:44:22.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:22.099 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.2) 2024-09-19T08:44:22.149 INFO:teuthology.orchestra.run.smithi098.stdout:81b4 2024-09-19T08:44:22.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:22.150 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148/subvolume_file.2) 2024-09-19T08:44:22.203 INFO:teuthology.orchestra.run.smithi098.stdout:81b4 2024-09-19T08:44:22.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:22.204 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.2) 2024-09-19T08:44:22.257 INFO:teuthology.orchestra.run.smithi098.stdout:1000 2024-09-19T08:44:22.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:22.258 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148/subvolume_file.2) 2024-09-19T08:44:22.306 INFO:teuthology.orchestra.run.smithi098.stdout:1000 2024-09-19T08:44:22.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:22.307 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.2) 2024-09-19T08:44:22.357 INFO:teuthology.orchestra.run.smithi098.stdout:1271 2024-09-19T08:44:22.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:22.358 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148/subvolume_file.2) 2024-09-19T08:44:22.407 INFO:teuthology.orchestra.run.smithi098.stdout:1271 2024-09-19T08:44:22.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:22.408 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.2) 2024-09-19T08:44:22.465 INFO:teuthology.orchestra.run.smithi098.stdout:1726735454 2024-09-19T08:44:22.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:22.466 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148/subvolume_file.2) 2024-09-19T08:44:22.515 INFO:teuthology.orchestra.run.smithi098.stdout:1726735454 2024-09-19T08:44:22.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:22.516 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.5) 2024-09-19T08:44:22.568 INFO:teuthology.orchestra.run.smithi098.stdout:81b4 2024-09-19T08:44:22.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:22.569 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148/subvolume_file.5) 2024-09-19T08:44:22.625 INFO:teuthology.orchestra.run.smithi098.stdout:81b4 2024-09-19T08:44:22.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:22.626 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.5) 2024-09-19T08:44:22.681 INFO:teuthology.orchestra.run.smithi098.stdout:1000 2024-09-19T08:44:22.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:22.682 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148/subvolume_file.5) 2024-09-19T08:44:22.735 INFO:teuthology.orchestra.run.smithi098.stdout:1000 2024-09-19T08:44:22.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:22.736 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.5) 2024-09-19T08:44:22.788 INFO:teuthology.orchestra.run.smithi098.stdout:1271 2024-09-19T08:44:22.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:22.789 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148/subvolume_file.5) 2024-09-19T08:44:22.838 INFO:teuthology.orchestra.run.smithi098.stdout:1271 2024-09-19T08:44:22.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:22.839 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.5) 2024-09-19T08:44:22.887 INFO:teuthology.orchestra.run.smithi098.stdout:1726735454 2024-09-19T08:44:22.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:22.889 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148/subvolume_file.5) 2024-09-19T08:44:22.938 INFO:teuthology.orchestra.run.smithi098.stdout:1726735454 2024-09-19T08:44:22.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:22.939 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.4) 2024-09-19T08:44:22.988 INFO:teuthology.orchestra.run.smithi098.stdout:81b4 2024-09-19T08:44:22.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:22.989 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148/subvolume_file.4) 2024-09-19T08:44:23.041 INFO:teuthology.orchestra.run.smithi098.stdout:81b4 2024-09-19T08:44:23.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:23.041 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.4) 2024-09-19T08:44:23.094 INFO:teuthology.orchestra.run.smithi098.stdout:1000 2024-09-19T08:44:23.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:23.095 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148/subvolume_file.4) 2024-09-19T08:44:23.142 INFO:teuthology.orchestra.run.smithi098.stdout:1000 2024-09-19T08:44:23.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:23.143 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.4) 2024-09-19T08:44:23.200 INFO:teuthology.orchestra.run.smithi098.stdout:1271 2024-09-19T08:44:23.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:23.201 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148/subvolume_file.4) 2024-09-19T08:44:23.250 INFO:teuthology.orchestra.run.smithi098.stdout:1271 2024-09-19T08:44:23.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:23.252 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.4) 2024-09-19T08:44:23.305 INFO:teuthology.orchestra.run.smithi098.stdout:1726735454 2024-09-19T08:44:23.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:23.306 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148/subvolume_file.4) 2024-09-19T08:44:23.354 INFO:teuthology.orchestra.run.smithi098.stdout:1726735454 2024-09-19T08:44:23.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:23.355 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.7) 2024-09-19T08:44:23.410 INFO:teuthology.orchestra.run.smithi098.stdout:81b4 2024-09-19T08:44:23.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:23.410 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148/subvolume_file.7) 2024-09-19T08:44:23.460 INFO:teuthology.orchestra.run.smithi098.stdout:81b4 2024-09-19T08:44:23.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:23.461 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.7) 2024-09-19T08:44:23.517 INFO:teuthology.orchestra.run.smithi098.stdout:1000 2024-09-19T08:44:23.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:23.518 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148/subvolume_file.7) 2024-09-19T08:44:23.566 INFO:teuthology.orchestra.run.smithi098.stdout:1000 2024-09-19T08:44:23.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:23.567 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.7) 2024-09-19T08:44:23.617 INFO:teuthology.orchestra.run.smithi098.stdout:1271 2024-09-19T08:44:23.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:23.617 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148/subvolume_file.7) 2024-09-19T08:44:23.670 INFO:teuthology.orchestra.run.smithi098.stdout:1271 2024-09-19T08:44:23.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:23.671 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.7) 2024-09-19T08:44:23.724 INFO:teuthology.orchestra.run.smithi098.stdout:1726735455 2024-09-19T08:44:23.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:23.725 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148/subvolume_file.7) 2024-09-19T08:44:23.778 INFO:teuthology.orchestra.run.smithi098.stdout:1726735455 2024-09-19T08:44:23.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:23.779 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.6) 2024-09-19T08:44:23.828 INFO:teuthology.orchestra.run.smithi098.stdout:81b4 2024-09-19T08:44:23.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:23.830 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148/subvolume_file.6) 2024-09-19T08:44:23.880 INFO:teuthology.orchestra.run.smithi098.stdout:81b4 2024-09-19T08:44:23.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:23.881 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.6) 2024-09-19T08:44:23.931 INFO:teuthology.orchestra.run.smithi098.stdout:1000 2024-09-19T08:44:23.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:23.932 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148/subvolume_file.6) 2024-09-19T08:44:23.981 INFO:teuthology.orchestra.run.smithi098.stdout:1000 2024-09-19T08:44:23.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:23.982 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.6) 2024-09-19T08:44:24.032 INFO:teuthology.orchestra.run.smithi098.stdout:1271 2024-09-19T08:44:24.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:24.033 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148/subvolume_file.6) 2024-09-19T08:44:24.083 INFO:teuthology.orchestra.run.smithi098.stdout:1271 2024-09-19T08:44:24.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:24.084 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.6) 2024-09-19T08:44:24.133 INFO:teuthology.orchestra.run.smithi098.stdout:1726735454 2024-09-19T08:44:24.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:24.134 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148/subvolume_file.6) 2024-09-19T08:44:24.182 INFO:teuthology.orchestra.run.smithi098.stdout:1726735454 2024-09-19T08:44:24.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:24.183 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.0) 2024-09-19T08:44:24.232 INFO:teuthology.orchestra.run.smithi098.stdout:81b4 2024-09-19T08:44:24.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:24.234 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148/subvolume_file.0) 2024-09-19T08:44:24.283 INFO:teuthology.orchestra.run.smithi098.stdout:81b4 2024-09-19T08:44:24.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:24.285 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.0) 2024-09-19T08:44:24.336 INFO:teuthology.orchestra.run.smithi098.stdout:1000 2024-09-19T08:44:24.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:24.337 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148/subvolume_file.0) 2024-09-19T08:44:24.387 INFO:teuthology.orchestra.run.smithi098.stdout:1000 2024-09-19T08:44:24.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:24.388 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.0) 2024-09-19T08:44:24.441 INFO:teuthology.orchestra.run.smithi098.stdout:1271 2024-09-19T08:44:24.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:24.442 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148/subvolume_file.0) 2024-09-19T08:44:24.490 INFO:teuthology.orchestra.run.smithi098.stdout:1271 2024-09-19T08:44:24.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:24.492 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.0) 2024-09-19T08:44:24.544 INFO:teuthology.orchestra.run.smithi098.stdout:1726735454 2024-09-19T08:44:24.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:24.545 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148/subvolume_file.0) 2024-09-19T08:44:24.594 INFO:teuthology.orchestra.run.smithi098.stdout:1726735454 2024-09-19T08:44:24.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:24.596 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.3) 2024-09-19T08:44:24.643 INFO:teuthology.orchestra.run.smithi098.stdout:81b4 2024-09-19T08:44:24.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:24.645 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148/subvolume_file.3) 2024-09-19T08:44:24.695 INFO:teuthology.orchestra.run.smithi098.stdout:81b4 2024-09-19T08:44:24.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:24.696 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.3) 2024-09-19T08:44:24.747 INFO:teuthology.orchestra.run.smithi098.stdout:1000 2024-09-19T08:44:24.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:24.749 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148/subvolume_file.3) 2024-09-19T08:44:24.799 INFO:teuthology.orchestra.run.smithi098.stdout:1000 2024-09-19T08:44:24.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:24.800 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.3) 2024-09-19T08:44:24.852 INFO:teuthology.orchestra.run.smithi098.stdout:1271 2024-09-19T08:44:24.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:24.853 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148/subvolume_file.3) 2024-09-19T08:44:24.902 INFO:teuthology.orchestra.run.smithi098.stdout:1271 2024-09-19T08:44:24.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:24.903 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.3) 2024-09-19T08:44:24.953 INFO:teuthology.orchestra.run.smithi098.stdout:1726735454 2024-09-19T08:44:24.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:24.954 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148/subvolume_file.3) 2024-09-19T08:44:25.002 INFO:teuthology.orchestra.run.smithi098.stdout:1726735454 2024-09-19T08:44:25.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:25.003 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.1) 2024-09-19T08:44:25.053 INFO:teuthology.orchestra.run.smithi098.stdout:81b4 2024-09-19T08:44:25.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:25.054 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148/subvolume_file.1) 2024-09-19T08:44:25.103 INFO:teuthology.orchestra.run.smithi098.stdout:81b4 2024-09-19T08:44:25.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:25.105 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.1) 2024-09-19T08:44:25.157 INFO:teuthology.orchestra.run.smithi098.stdout:1000 2024-09-19T08:44:25.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:25.158 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148/subvolume_file.1) 2024-09-19T08:44:25.206 INFO:teuthology.orchestra.run.smithi098.stdout:1000 2024-09-19T08:44:25.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:25.207 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.1) 2024-09-19T08:44:25.261 INFO:teuthology.orchestra.run.smithi098.stdout:1271 2024-09-19T08:44:25.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:25.262 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148/subvolume_file.1) 2024-09-19T08:44:25.310 INFO:teuthology.orchestra.run.smithi098.stdout:1271 2024-09-19T08:44:25.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:25.311 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a/.snap/subvol_snap63800/subvolume_file.1) 2024-09-19T08:44:25.362 INFO:teuthology.orchestra.run.smithi098.stdout:1726735454 2024-09-19T08:44:25.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:25.363 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone38220/bf2ebd1f-6168-40a5-a092-b02562add148/subvolume_file.1) 2024-09-19T08:44:25.410 INFO:teuthology.orchestra.run.smithi098.stdout:1726735454 2024-09-19T08:44:25.412 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs subvol41500 fake subvol_clone83561 2024-09-19T08:44:25.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.596+0000 7fc91540b640 1 Processor -- start 2024-09-19T08:44:25.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.596+0000 7fc91540b640 1 -- start start 2024-09-19T08:44:25.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.596+0000 7fc91540b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc910145b70 0x7fc91006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:25.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.596+0000 7fc91540b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc91006c1d0 0x7fc9100713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:25.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.596+0000 7fc91540b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc9100718f0 0x7fc9100cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:25.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.596+0000 7fc91540b640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fc91006c5f0 con 0x7fc9100718f0 2024-09-19T08:44:25.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.596+0000 7fc91540b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fc910073530 con 0x7fc91006c1d0 2024-09-19T08:44:25.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.596+0000 7fc91540b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc910073810 con 0x7fc910145b70 2024-09-19T08:44:25.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.596+0000 7fc90f7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc9100718f0 0x7fc9100cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:25.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.596+0000 7fc90f7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc9100718f0 0x7fc9100cef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42210/0 (socket says 172.21.15.98:42210) 2024-09-19T08:44:25.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.596+0000 7fc90f7fe640 1 -- 172.21.15.98:0/1675112168 learned_addr learned my addr 172.21.15.98:0/1675112168 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:25.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.596+0000 7fc90effd640 1 --2- 172.21.15.98:0/1675112168 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc910145b70 0x7fc91006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:25.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.596+0000 7fc90e7fc640 1 --2- 172.21.15.98:0/1675112168 >> v2:172.21.15.139:3300/0 conn(0x7fc91006c1d0 0x7fc9100713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:25.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.596+0000 7fc90f7fe640 1 -- 172.21.15.98:0/1675112168 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc910145b70 msgr2=0x7fc91006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:25.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.596+0000 7fc90f7fe640 1 --2- 172.21.15.98:0/1675112168 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc910145b70 0x7fc91006bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:25.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.596+0000 7fc90f7fe640 1 -- 172.21.15.98:0/1675112168 >> v2:172.21.15.139:3300/0 conn(0x7fc91006c1d0 msgr2=0x7fc9100713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:25.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.596+0000 7fc90f7fe640 1 --2- 172.21.15.98:0/1675112168 >> v2:172.21.15.139:3300/0 conn(0x7fc91006c1d0 0x7fc9100713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:25.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.596+0000 7fc90f7fe640 1 -- 172.21.15.98:0/1675112168 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc910073af0 con 0x7fc9100718f0 2024-09-19T08:44:25.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.596+0000 7fc90e7fc640 1 --2- 172.21.15.98:0/1675112168 >> v2:172.21.15.139:3300/0 conn(0x7fc91006c1d0 0x7fc9100713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:25.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.600+0000 7fc90f7fe640 1 --2- 172.21.15.98:0/1675112168 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9100718f0 0x7fc9100cef70 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fc91006f1a0 tx=0x7fc90801c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370629480eec19b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:25.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.600+0000 7fc90dffb640 1 -- 172.21.15.98:0/1675112168 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9080a9430 con 0x7fc9100718f0 2024-09-19T08:44:25.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.600+0000 7fc90dffb640 1 -- 172.21.15.98:0/1675112168 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9080ae070 con 0x7fc9100718f0 2024-09-19T08:44:25.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.600+0000 7fc90dffb640 1 -- 172.21.15.98:0/1675112168 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9080a97f0 con 0x7fc9100718f0 2024-09-19T08:44:25.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.600+0000 7fc91540b640 1 -- 172.21.15.98:0/1675112168 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9100718f0 msgr2=0x7fc9100cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:25.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.600+0000 7fc91540b640 1 --2- 172.21.15.98:0/1675112168 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9100718f0 0x7fc9100cef70 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fc91006f1a0 tx=0x7fc90801c6a0 comp rx=0 tx=0).stop 2024-09-19T08:44:25.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.600+0000 7fc91540b640 1 -- 172.21.15.98:0/1675112168 shutdown_connections 2024-09-19T08:44:25.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.600+0000 7fc91540b640 1 --2- 172.21.15.98:0/1675112168 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9100718f0 0x7fc9100cef70 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:25.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.600+0000 7fc91540b640 1 --2- 172.21.15.98:0/1675112168 >> v2:172.21.15.139:3300/0 conn(0x7fc91006c1d0 0x7fc9100713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:25.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.600+0000 7fc91540b640 1 --2- 172.21.15.98:0/1675112168 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc910145b70 0x7fc91006bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:25.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.600+0000 7fc91540b640 1 -- 172.21.15.98:0/1675112168 >> 172.21.15.98:0/1675112168 conn(0x7fc9100b2010 msgr2=0x7fc9100b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:25.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.600+0000 7fc91540b640 1 -- 172.21.15.98:0/1675112168 shutdown_connections 2024-09-19T08:44:25.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.600+0000 7fc91540b640 1 -- 172.21.15.98:0/1675112168 wait complete. 2024-09-19T08:44:25.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.600+0000 7fc91540b640 1 Processor -- start 2024-09-19T08:44:25.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.600+0000 7fc91540b640 1 -- start start 2024-09-19T08:44:25.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.600+0000 7fc91540b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc91006c1d0 0x7fc91015bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:25.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.600+0000 7fc91540b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9100718f0 0x7fc9100acc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:25.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.600+0000 7fc91540b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc910145b70 0x7fc9100aad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:25.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.600+0000 7fc91540b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc9100b3590 con 0x7fc9100718f0 2024-09-19T08:44:25.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.600+0000 7fc91540b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc9100b3410 con 0x7fc910145b70 2024-09-19T08:44:25.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.600+0000 7fc91540b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc9100b3710 con 0x7fc91006c1d0 2024-09-19T08:44:25.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.600+0000 7fc90e7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9100718f0 0x7fc9100acc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:25.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.604+0000 7fc90e7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9100718f0 0x7fc9100acc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42212/0 (socket says 172.21.15.98:42212) 2024-09-19T08:44:25.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.604+0000 7fc90e7fc640 1 -- 172.21.15.98:0/234850818 learned_addr learned my addr 172.21.15.98:0/234850818 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:25.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.604+0000 7fc90effd640 1 --2- 172.21.15.98:0/234850818 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc91006c1d0 0x7fc91015bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:25.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.604+0000 7fc90f7fe640 1 --2- 172.21.15.98:0/234850818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc910145b70 0x7fc9100aad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:25.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.604+0000 7fc90e7fc640 1 -- 172.21.15.98:0/234850818 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc91006c1d0 msgr2=0x7fc91015bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:25.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.604+0000 7fc90e7fc640 1 --2- 172.21.15.98:0/234850818 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc91006c1d0 0x7fc91015bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:25.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.604+0000 7fc90e7fc640 1 -- 172.21.15.98:0/234850818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc910145b70 msgr2=0x7fc9100aad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:25.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.604+0000 7fc90e7fc640 1 --2- 172.21.15.98:0/234850818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc910145b70 0x7fc9100aad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:25.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.604+0000 7fc90e7fc640 1 -- 172.21.15.98:0/234850818 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc91015ece0 con 0x7fc9100718f0 2024-09-19T08:44:25.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.604+0000 7fc90effd640 1 --2- 172.21.15.98:0/234850818 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc91006c1d0 0x7fc91015bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:25.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.604+0000 7fc90f7fe640 1 --2- 172.21.15.98:0/234850818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc910145b70 0x7fc9100aad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:25.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.604+0000 7fc90e7fc640 1 --2- 172.21.15.98:0/234850818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9100718f0 0x7fc9100acc90 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fc90400c820 tx=0x7fc90400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:25.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.604+0000 7fc8f77fe640 1 -- 172.21.15.98:0/234850818 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc904004230 con 0x7fc9100718f0 2024-09-19T08:44:25.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.604+0000 7fc8f77fe640 1 -- 172.21.15.98:0/234850818 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9040043d0 con 0x7fc9100718f0 2024-09-19T08:44:25.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.604+0000 7fc91540b640 1 -- 172.21.15.98:0/234850818 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc91015efc0 con 0x7fc9100718f0 2024-09-19T08:44:25.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.604+0000 7fc8f77fe640 1 -- 172.21.15.98:0/234850818 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc904007a50 con 0x7fc9100718f0 2024-09-19T08:44:25.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.604+0000 7fc91540b640 1 -- 172.21.15.98:0/234850818 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc91015f580 con 0x7fc9100718f0 2024-09-19T08:44:25.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.604+0000 7fc91540b640 1 -- 172.21.15.98:0/234850818 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9100cef70 con 0x7fc9100718f0 2024-09-19T08:44:25.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.604+0000 7fc8f77fe640 1 -- 172.21.15.98:0/234850818 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 87700+0+0 (secure 0 0 0) 0x7fc904007bf0 con 0x7fc9100718f0 2024-09-19T08:44:25.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.604+0000 7fc8f77fe640 1 --2- 172.21.15.98:0/234850818 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc8fc067b90 0x7fc8fc06a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:25.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.604+0000 7fc8f77fe640 1 -- 172.21.15.98:0/234850818 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7fc904086400 con 0x7fc9100718f0 2024-09-19T08:44:25.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.604+0000 7fc90effd640 1 --2- 172.21.15.98:0/234850818 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc8fc067b90 0x7fc8fc06a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:25.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.604+0000 7fc90effd640 1 --2- 172.21.15.98:0/234850818 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc8fc067b90 0x7fc8fc06a050 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc9000097a0 tx=0x7fc900008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:25.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.608+0000 7fc8f77fe640 1 -- 172.21.15.98:0/234850818 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc904015020 con 0x7fc9100718f0 2024-09-19T08:44:25.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.808+0000 7fc91540b640 1 -- 172.21.15.98:0/234850818 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvol41500", "snap_name": "fake", "target_sub_name": "subvol_clone83561", "target": ["mon-mgr", ""]}) -- 0x7fc9100757a0 con 0x7fc8fc067b90 2024-09-19T08:44:25.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.964+0000 7fc8f77fe640 1 -- 172.21.15.98:0/234850818 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc9100757a0 con 0x7fc8fc067b90 2024-09-19T08:44:25.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.968+0000 7fc91540b640 1 -- 172.21.15.98:0/234850818 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc8fc067b90 msgr2=0x7fc8fc06a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:25.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.968+0000 7fc91540b640 1 --2- 172.21.15.98:0/234850818 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc8fc067b90 0x7fc8fc06a050 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc9000097a0 tx=0x7fc900008040 comp rx=0 tx=0).stop 2024-09-19T08:44:25.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.968+0000 7fc91540b640 1 -- 172.21.15.98:0/234850818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9100718f0 msgr2=0x7fc9100acc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:25.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.968+0000 7fc91540b640 1 --2- 172.21.15.98:0/234850818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9100718f0 0x7fc9100acc90 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fc90400c820 tx=0x7fc90400cce0 comp rx=0 tx=0).stop 2024-09-19T08:44:25.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.968+0000 7fc91540b640 1 -- 172.21.15.98:0/234850818 shutdown_connections 2024-09-19T08:44:25.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.968+0000 7fc91540b640 1 --2- 172.21.15.98:0/234850818 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc8fc067b90 0x7fc8fc06a050 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:25.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.968+0000 7fc91540b640 1 --2- 172.21.15.98:0/234850818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc910145b70 0x7fc9100aad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:25.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.968+0000 7fc91540b640 1 --2- 172.21.15.98:0/234850818 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc9100718f0 0x7fc9100acc90 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:25.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.968+0000 7fc91540b640 1 --2- 172.21.15.98:0/234850818 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc91006c1d0 0x7fc91015bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:25.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.968+0000 7fc91540b640 1 -- 172.21.15.98:0/234850818 >> 172.21.15.98:0/234850818 conn(0x7fc9100b2010 msgr2=0x7fc9100b0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:25.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.968+0000 7fc91540b640 1 -- 172.21.15.98:0/234850818 shutdown_connections 2024-09-19T08:44:25.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:25.968+0000 7fc91540b640 1 -- 172.21.15.98:0/234850818 wait complete. 2024-09-19T08:44:25.988 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone83561 2024-09-19T08:44:26.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.108+0000 7f2be0962640 1 Processor -- start 2024-09-19T08:44:26.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.108+0000 7f2be0962640 1 -- start start 2024-09-19T08:44:26.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.108+0000 7f2be0962640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2bdc06c3a0 0x7f2bdc165f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:26.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.108+0000 7f2be0962640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bdc072ea0 con 0x7f2bdc144d10 2024-09-19T08:44:26.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.108+0000 7f2be0962640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bdc073180 con 0x7f2bdc0729f0 2024-09-19T08:44:26.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.108+0000 7f2be0962640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f2bdc073ab0 con 0x7f2bdc06c3a0 2024-09-19T08:44:26.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.108+0000 7f2bda59c640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f2bdc144d10 0x7f2bdc0722e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:34544/0 (socket says 172.21.15.98:34544) 2024-09-19T08:44:26.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.108+0000 7f2bda59c640 1 -- 172.21.15.98:0/2678653062 learned_addr learned my addr 172.21.15.98:0/2678653062 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:26.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.108+0000 7f2bdad9d640 1 --2- 172.21.15.98:0/2678653062 >> v2:172.21.15.98:3300/0 conn(0x7f2bdc06c3a0 0x7f2bdc165f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:26.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.108+0000 7f2bd959a640 1 -- 172.21.15.98:0/2678653062 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1900511031 0 0) 0x7f2bdc072ea0 con 0x7f2bdc144d10 2024-09-19T08:44:26.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.108+0000 7f2bd959a640 1 -- 172.21.15.98:0/2678653062 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2bc4003530 con 0x7f2bdc144d10 2024-09-19T08:44:26.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.108+0000 7f2bdad9d640 1 -- 172.21.15.98:0/2678653062 >> v1:172.21.15.139:6789/0 conn(0x7f2bdc0729f0 legacy=0x7f2bdc06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:26.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.108+0000 7f2bdad9d640 1 -- 172.21.15.98:0/2678653062 >> v1:172.21.15.98:6789/0 conn(0x7f2bdc144d10 legacy=0x7f2bdc0722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:26.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.108+0000 7f2bdad9d640 1 -- 172.21.15.98:0/2678653062 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bdc073460 con 0x7f2bdc06c3a0 2024-09-19T08:44:26.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.112+0000 7f2bdad9d640 1 --2- 172.21.15.98:0/2678653062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bdc06c3a0 0x7f2bdc165f50 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009d90 tx=0x7f2bd401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73002d0d551cac9c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:26.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.112+0000 7f2bd959a640 1 -- 172.21.15.98:0/2678653062 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd40b2070 con 0x7f2bdc06c3a0 2024-09-19T08:44:26.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.112+0000 7f2bd959a640 1 -- 172.21.15.98:0/2678653062 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bd40a5850 con 0x7f2bdc06c3a0 2024-09-19T08:44:26.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.112+0000 7f2bd959a640 1 -- 172.21.15.98:0/2678653062 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd40a5c20 con 0x7f2bdc06c3a0 2024-09-19T08:44:26.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.112+0000 7f2be0962640 1 -- 172.21.15.98:0/2678653062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bdc06c3a0 msgr2=0x7f2bdc165f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:26.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.112+0000 7f2be0962640 1 --2- 172.21.15.98:0/2678653062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bdc06c3a0 0x7f2bdc165f50 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009d90 tx=0x7f2bd401c970 comp rx=0 tx=0).stop 2024-09-19T08:44:26.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.112+0000 7f2be0962640 1 -- 172.21.15.98:0/2678653062 shutdown_connections 2024-09-19T08:44:26.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.112+0000 7f2be0962640 1 --2- 172.21.15.98:0/2678653062 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bdc06c3a0 0x7f2bdc165f50 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:26.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.112+0000 7f2be0962640 1 -- 172.21.15.98:0/2678653062 >> 172.21.15.98:0/2678653062 conn(0x7f2bdc0b2010 msgr2=0x7f2bdc0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:26.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.112+0000 7f2be0962640 1 -- 172.21.15.98:0/2678653062 shutdown_connections 2024-09-19T08:44:26.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.112+0000 7f2be0962640 1 -- 172.21.15.98:0/2678653062 wait complete. 2024-09-19T08:44:26.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.112+0000 7f2be0962640 1 Processor -- start 2024-09-19T08:44:26.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.112+0000 7f2be0962640 1 -- start start 2024-09-19T08:44:26.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.116+0000 7f2be0962640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2bdc06c3a0 0x7f2bdc144090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:26.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.116+0000 7f2be0962640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bdc0729f0 0x7f2bdc142780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:26.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.116+0000 7f2be0962640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2bdc144d10 0x7f2bdc0b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:26.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.116+0000 7f2be0962640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc169040 con 0x7f2bdc0729f0 2024-09-19T08:44:26.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.116+0000 7f2be0962640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc168ec0 con 0x7f2bdc06c3a0 2024-09-19T08:44:26.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.116+0000 7f2be0962640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2bdc1691c0 con 0x7f2bdc144d10 2024-09-19T08:44:26.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.116+0000 7f2bd9d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bdc0729f0 0x7f2bdc142780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:26.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.116+0000 7f2bd9d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bdc0729f0 0x7f2bdc142780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42228/0 (socket says 172.21.15.98:42228) 2024-09-19T08:44:26.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.116+0000 7f2bd9d9b640 1 -- 172.21.15.98:0/3638709931 learned_addr learned my addr 172.21.15.98:0/3638709931 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:26.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.116+0000 7f2bda59c640 1 --2- 172.21.15.98:0/3638709931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2bdc06c3a0 0x7f2bdc144090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:26.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.116+0000 7f2bdad9d640 1 --2- 172.21.15.98:0/3638709931 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2bdc144d10 0x7f2bdc0b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:26.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.116+0000 7f2bd9d9b640 1 -- 172.21.15.98:0/3638709931 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2bdc144d10 msgr2=0x7f2bdc0b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:26.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.116+0000 7f2bd9d9b640 1 --2- 172.21.15.98:0/3638709931 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2bdc144d10 0x7f2bdc0b4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:26.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.116+0000 7f2bd9d9b640 1 -- 172.21.15.98:0/3638709931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2bdc06c3a0 msgr2=0x7f2bdc144090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:26.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.116+0000 7f2bd9d9b640 1 --2- 172.21.15.98:0/3638709931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2bdc06c3a0 0x7f2bdc144090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:26.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.116+0000 7f2bd9d9b640 1 -- 172.21.15.98:0/3638709931 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bdc142ff0 con 0x7f2bdc0729f0 2024-09-19T08:44:26.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.116+0000 7f2bda59c640 1 --2- 172.21.15.98:0/3638709931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2bdc06c3a0 0x7f2bdc144090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:26.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.116+0000 7f2bd9d9b640 1 --2- 172.21.15.98:0/3638709931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bdc0729f0 0x7f2bdc142780 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000c4d0 tx=0x7f2bd000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:26.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.116+0000 7f2bcaffd640 1 -- 172.21.15.98:0/3638709931 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd00052d0 con 0x7f2bdc0729f0 2024-09-19T08:44:26.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.116+0000 7f2bcaffd640 1 -- 172.21.15.98:0/3638709931 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bd0005470 con 0x7f2bdc0729f0 2024-09-19T08:44:26.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.116+0000 7f2be0962640 1 -- 172.21.15.98:0/3638709931 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bdc1432d0 con 0x7f2bdc0729f0 2024-09-19T08:44:26.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.116+0000 7f2bcaffd640 1 -- 172.21.15.98:0/3638709931 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0012830 con 0x7f2bdc0729f0 2024-09-19T08:44:26.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.116+0000 7f2be0962640 1 -- 172.21.15.98:0/3638709931 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bdc143890 con 0x7f2bdc0729f0 2024-09-19T08:44:26.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.116+0000 7f2be0962640 1 -- 172.21.15.98:0/3638709931 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bdc072ea0 con 0x7f2bdc0729f0 2024-09-19T08:44:26.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.116+0000 7f2bcaffd640 1 -- 172.21.15.98:0/3638709931 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 87700+0+0 (secure 0 0 0) 0x7f2bd00129d0 con 0x7f2bdc0729f0 2024-09-19T08:44:26.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.116+0000 7f2bcaffd640 1 --2- 172.21.15.98:0/3638709931 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2bbc067b90 0x7f2bbc06a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:26.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.116+0000 7f2bcaffd640 1 -- 172.21.15.98:0/3638709931 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f2bd0085d20 con 0x7f2bdc0729f0 2024-09-19T08:44:26.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.116+0000 7f2bda59c640 1 --2- 172.21.15.98:0/3638709931 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2bbc067b90 0x7f2bbc06a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:26.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.120+0000 7f2bda59c640 1 --2- 172.21.15.98:0/3638709931 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2bbc067b90 0x7f2bbc06a050 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc0011f0 tx=0x7f2bcc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:26.123 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.120+0000 7f2bcaffd640 1 -- 172.21.15.98:0/3638709931 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2bd00564d0 con 0x7f2bdc0729f0 2024-09-19T08:44:26.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:26.324+0000 7f2be0962640 1 -- 172.21.15.98:0/3638709931 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone83561", "target": ["mon-mgr", ""]}) -- 0x7f2bdc0cfc50 con 0x7f2bbc067b90 2024-09-19T08:44:30.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:30.224+0000 7f2bcaffd640 1 -- 172.21.15.98:0/3638709931 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+153 (secure 0 0 0) 0x7f2bdc0cfc50 con 0x7f2bbc067b90 2024-09-19T08:44:30.227 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:44:30.227 INFO:teuthology.orchestra.run.smithi098.stdout: "status": { 2024-09-19T08:44:30.227 INFO:teuthology.orchestra.run.smithi098.stdout: "state": "pending", 2024-09-19T08:44:30.227 INFO:teuthology.orchestra.run.smithi098.stdout: "source": { 2024-09-19T08:44:30.227 INFO:teuthology.orchestra.run.smithi098.stdout: "volume": "cephfs", 2024-09-19T08:44:30.227 INFO:teuthology.orchestra.run.smithi098.stdout: "subvolume": "subvol41500", 2024-09-19T08:44:30.227 INFO:teuthology.orchestra.run.smithi098.stdout: "snapshot": "fake" 2024-09-19T08:44:30.227 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:44:30.228 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:44:30.228 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:44:30.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:30.228+0000 7f2be0962640 1 -- 172.21.15.98:0/3638709931 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2bbc067b90 msgr2=0x7f2bbc06a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:30.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:30.228+0000 7f2be0962640 1 --2- 172.21.15.98:0/3638709931 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2bbc067b90 0x7f2bbc06a050 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc0011f0 tx=0x7f2bcc009040 comp rx=0 tx=0).stop 2024-09-19T08:44:30.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:30.228+0000 7f2be0962640 1 -- 172.21.15.98:0/3638709931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bdc0729f0 msgr2=0x7f2bdc142780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:30.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:30.228+0000 7f2be0962640 1 --2- 172.21.15.98:0/3638709931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bdc0729f0 0x7f2bdc142780 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000c4d0 tx=0x7f2bd000c990 comp rx=0 tx=0).stop 2024-09-19T08:44:30.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:30.228+0000 7f2be0962640 1 -- 172.21.15.98:0/3638709931 shutdown_connections 2024-09-19T08:44:30.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:30.228+0000 7f2be0962640 1 --2- 172.21.15.98:0/3638709931 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2bbc067b90 0x7f2bbc06a050 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:30.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:30.228+0000 7f2be0962640 1 --2- 172.21.15.98:0/3638709931 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2bdc144d10 0x7f2bdc0b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:30.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:30.228+0000 7f2be0962640 1 --2- 172.21.15.98:0/3638709931 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2bdc0729f0 0x7f2bdc142780 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:30.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:30.228+0000 7f2be0962640 1 --2- 172.21.15.98:0/3638709931 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2bdc06c3a0 0x7f2bdc144090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:30.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:30.228+0000 7f2be0962640 1 -- 172.21.15.98:0/3638709931 >> 172.21.15.98:0/3638709931 conn(0x7f2bdc0b2010 msgr2=0x7f2bdc0b0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:30.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:30.228+0000 7f2be0962640 1 -- 172.21.15.98:0/3638709931 shutdown_connections 2024-09-19T08:44:30.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:30.228+0000 7f2be0962640 1 -- 172.21.15.98:0/3638709931 wait complete. 2024-09-19T08:44:31.252 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone83561 2024-09-19T08:44:31.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.388+0000 7f8b148a7640 1 Processor -- start 2024-09-19T08:44:31.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.388+0000 7f8b148a7640 1 -- start start 2024-09-19T08:44:31.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.388+0000 7f8b148a7640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8b1006c3a0 0x7f8b100713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:31.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.388+0000 7f8b148a7640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b10073530 con 0x7f8b10145b70 2024-09-19T08:44:31.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.388+0000 7f8b148a7640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b10073810 con 0x7f8b100718f0 2024-09-19T08:44:31.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.388+0000 7f8b148a7640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f8b10073af0 con 0x7f8b1006c3a0 2024-09-19T08:44:31.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.388+0000 7f8b0dd9b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8b1006c3a0 0x7f8b100713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:31.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.388+0000 7f8b0e59c640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f8b10145b70 0x7f8b1006bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:45550/0 (socket says 172.21.15.98:45550) 2024-09-19T08:44:31.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.388+0000 7f8b0e59c640 1 -- 172.21.15.98:0/4119574091 learned_addr learned my addr 172.21.15.98:0/4119574091 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:31.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.388+0000 7f8b0dd9b640 1 -- 172.21.15.98:0/4119574091 >> v1:172.21.15.139:6789/0 conn(0x7f8b100718f0 legacy=0x7f8b100b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:31.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.388+0000 7f8b0dd9b640 1 -- 172.21.15.98:0/4119574091 >> v1:172.21.15.98:6789/0 conn(0x7f8b10145b70 legacy=0x7f8b1006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:31.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.388+0000 7f8b0dd9b640 1 -- 172.21.15.98:0/4119574091 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b10073dd0 con 0x7f8b1006c3a0 2024-09-19T08:44:31.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.388+0000 7f8b0d59a640 1 -- 172.21.15.98:0/4119574091 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2402232674 0 0) 0x7f8b10073530 con 0x7f8b10145b70 2024-09-19T08:44:31.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.392+0000 7f8b0dd9b640 1 --2- 172.21.15.98:0/4119574091 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b1006c3a0 0x7f8b100713b0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f8b040097c0 tx=0x7f8b0401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba9d83788c0aa69 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:31.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.392+0000 7f8b0d59a640 1 -- 172.21.15.98:0/4119574091 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b040b0070 con 0x7f8b1006c3a0 2024-09-19T08:44:31.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.392+0000 7f8b0d59a640 1 -- 172.21.15.98:0/4119574091 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b040ab600 con 0x7f8b1006c3a0 2024-09-19T08:44:31.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.392+0000 7f8b0d59a640 1 -- 172.21.15.98:0/4119574091 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b040a4e60 con 0x7f8b1006c3a0 2024-09-19T08:44:31.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.392+0000 7f8b148a7640 1 -- 172.21.15.98:0/4119574091 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b1006c3a0 msgr2=0x7f8b100713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:31.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.392+0000 7f8b148a7640 1 --2- 172.21.15.98:0/4119574091 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b1006c3a0 0x7f8b100713b0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f8b040097c0 tx=0x7f8b0401c3d0 comp rx=0 tx=0).stop 2024-09-19T08:44:31.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.392+0000 7f8b148a7640 1 -- 172.21.15.98:0/4119574091 shutdown_connections 2024-09-19T08:44:31.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.392+0000 7f8b148a7640 1 --2- 172.21.15.98:0/4119574091 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b1006c3a0 0x7f8b100713b0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:31.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.392+0000 7f8b148a7640 1 -- 172.21.15.98:0/4119574091 >> 172.21.15.98:0/4119574091 conn(0x7f8b100b2010 msgr2=0x7f8b100b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:31.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.392+0000 7f8b148a7640 1 -- 172.21.15.98:0/4119574091 shutdown_connections 2024-09-19T08:44:31.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.392+0000 7f8b148a7640 1 -- 172.21.15.98:0/4119574091 wait complete. 2024-09-19T08:44:31.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.392+0000 7f8b148a7640 1 Processor -- start 2024-09-19T08:44:31.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.392+0000 7f8b148a7640 1 -- start start 2024-09-19T08:44:31.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.392+0000 7f8b148a7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b1006c3a0 0x7f8b100ce230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:31.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.392+0000 7f8b148a7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b100718f0 0x7f8b1013e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:31.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.392+0000 7f8b148a7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b10145b70 0x7f8b1015be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:31.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.392+0000 7f8b148a7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8b100b4700 con 0x7f8b100718f0 2024-09-19T08:44:31.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.392+0000 7f8b148a7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8b100b4580 con 0x7f8b10145b70 2024-09-19T08:44:31.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.392+0000 7f8b148a7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8b1016a080 con 0x7f8b1006c3a0 2024-09-19T08:44:31.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.392+0000 7f8b0dd9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b100718f0 0x7f8b1013e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:31.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.392+0000 7f8b0dd9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b100718f0 0x7f8b1013e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41284/0 (socket says 172.21.15.98:41284) 2024-09-19T08:44:31.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.392+0000 7f8b0dd9b640 1 -- 172.21.15.98:0/4909799 learned_addr learned my addr 172.21.15.98:0/4909799 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:31.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.392+0000 7f8b0e59c640 1 --2- 172.21.15.98:0/4909799 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b1006c3a0 0x7f8b100ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:31.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.392+0000 7f8b0ed9d640 1 --2- 172.21.15.98:0/4909799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b10145b70 0x7f8b1015be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:31.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.392+0000 7f8b0dd9b640 1 -- 172.21.15.98:0/4909799 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b1006c3a0 msgr2=0x7f8b100ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:31.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.392+0000 7f8b0dd9b640 1 --2- 172.21.15.98:0/4909799 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b1006c3a0 0x7f8b100ce230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:31.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.392+0000 7f8b0dd9b640 1 -- 172.21.15.98:0/4909799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b10145b70 msgr2=0x7f8b1015be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:31.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.392+0000 7f8b0dd9b640 1 --2- 172.21.15.98:0/4909799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b10145b70 0x7f8b1015be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:31.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.392+0000 7f8b0dd9b640 1 -- 172.21.15.98:0/4909799 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b1013ea40 con 0x7f8b100718f0 2024-09-19T08:44:31.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.392+0000 7f8b0ed9d640 1 --2- 172.21.15.98:0/4909799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b10145b70 0x7f8b1015be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:31.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.392+0000 7f8b0dd9b640 1 --2- 172.21.15.98:0/4909799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b100718f0 0x7f8b1013e2d0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f8b04018fe0 tx=0x7f8b0401ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:31.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.392+0000 7f8afeffd640 1 -- 172.21.15.98:0/4909799 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b040b0050 con 0x7f8b100718f0 2024-09-19T08:44:31.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.392+0000 7f8afeffd640 1 -- 172.21.15.98:0/4909799 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b040bc070 con 0x7f8b100718f0 2024-09-19T08:44:31.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.392+0000 7f8b148a7640 1 -- 172.21.15.98:0/4909799 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b1013ed20 con 0x7f8b100718f0 2024-09-19T08:44:31.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.392+0000 7f8afeffd640 1 -- 172.21.15.98:0/4909799 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b040b5850 con 0x7f8b100718f0 2024-09-19T08:44:31.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.396+0000 7f8b148a7640 1 -- 172.21.15.98:0/4909799 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b1013f2e0 con 0x7f8b100718f0 2024-09-19T08:44:31.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.396+0000 7f8b148a7640 1 -- 172.21.15.98:0/4909799 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b100713b0 con 0x7f8b100718f0 2024-09-19T08:44:31.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.396+0000 7f8afeffd640 1 -- 172.21.15.98:0/4909799 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 87700+0+0 (secure 0 0 0) 0x7f8b040b59f0 con 0x7f8b100718f0 2024-09-19T08:44:31.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.396+0000 7f8afeffd640 1 --2- 172.21.15.98:0/4909799 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8af0067b90 0x7f8af006a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:31.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.396+0000 7f8afeffd640 1 -- 172.21.15.98:0/4909799 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f8b041209f0 con 0x7f8b100718f0 2024-09-19T08:44:31.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.396+0000 7f8b0e59c640 1 --2- 172.21.15.98:0/4909799 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8af0067b90 0x7f8af006a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:31.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.396+0000 7f8b0e59c640 1 --2- 172.21.15.98:0/4909799 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8af0067b90 0x7f8af006a050 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8b00000f70 tx=0x7f8b00007c10 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:31.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.400+0000 7f8afeffd640 1 -- 172.21.15.98:0/4909799 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f8b04016050 con 0x7f8b100718f0 2024-09-19T08:44:31.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:31.604+0000 7f8b148a7640 1 -- 172.21.15.98:0/4909799 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone83561", "target": ["mon-mgr", ""]}) -- 0x7f8b10143640 con 0x7f8af0067b90 2024-09-19T08:44:35.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.204+0000 7f8afeffd640 1 -- 172.21.15.98:0/4909799 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7f8b10143640 con 0x7f8af0067b90 2024-09-19T08:44:35.207 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:44:35.207 INFO:teuthology.orchestra.run.smithi098.stdout: "status": { 2024-09-19T08:44:35.207 INFO:teuthology.orchestra.run.smithi098.stdout: "state": "complete" 2024-09-19T08:44:35.207 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:44:35.207 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:44:35.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.208+0000 7f8b148a7640 1 -- 172.21.15.98:0/4909799 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8af0067b90 msgr2=0x7f8af006a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:35.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.208+0000 7f8b148a7640 1 --2- 172.21.15.98:0/4909799 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8af0067b90 0x7f8af006a050 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8b00000f70 tx=0x7f8b00007c10 comp rx=0 tx=0).stop 2024-09-19T08:44:35.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.208+0000 7f8b148a7640 1 -- 172.21.15.98:0/4909799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b100718f0 msgr2=0x7f8b1013e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:35.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.208+0000 7f8b148a7640 1 --2- 172.21.15.98:0/4909799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b100718f0 0x7f8b1013e2d0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f8b04018fe0 tx=0x7f8b0401ac10 comp rx=0 tx=0).stop 2024-09-19T08:44:35.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.208+0000 7f8b148a7640 1 -- 172.21.15.98:0/4909799 shutdown_connections 2024-09-19T08:44:35.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.208+0000 7f8b148a7640 1 --2- 172.21.15.98:0/4909799 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8af0067b90 0x7f8af006a050 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:35.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.208+0000 7f8b148a7640 1 --2- 172.21.15.98:0/4909799 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8b10145b70 0x7f8b1015be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:35.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.208+0000 7f8b148a7640 1 --2- 172.21.15.98:0/4909799 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8b100718f0 0x7f8b1013e2d0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:35.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.208+0000 7f8b148a7640 1 --2- 172.21.15.98:0/4909799 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8b1006c3a0 0x7f8b100ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:35.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.208+0000 7f8b148a7640 1 -- 172.21.15.98:0/4909799 >> 172.21.15.98:0/4909799 conn(0x7f8b100b2010 msgr2=0x7f8b100aff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:35.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.208+0000 7f8b148a7640 1 -- 172.21.15.98:0/4909799 shutdown_connections 2024-09-19T08:44:35.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.208+0000 7f8b148a7640 1 -- 172.21.15.98:0/4909799 wait complete. 2024-09-19T08:44:35.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:35.230 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol_clone83561/.meta) 2024-09-19T08:44:35.245 INFO:teuthology.orchestra.run.smithi098.stdout:[GLOBAL] 2024-09-19T08:44:35.245 INFO:teuthology.orchestra.run.smithi098.stdout:version = 2 2024-09-19T08:44:35.245 INFO:teuthology.orchestra.run.smithi098.stdout:type = clone 2024-09-19T08:44:35.245 INFO:teuthology.orchestra.run.smithi098.stdout:path = /volumes/_nogroup/subvol_clone83561/24d35547-c5ce-4abe-b7c8-04986c9a52bd 2024-09-19T08:44:35.245 INFO:teuthology.orchestra.run.smithi098.stdout:state = complete 2024-09-19T08:44:35.245 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:44:35.246 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info cephfs subvol41500 subvol_snap63800 2024-09-19T08:44:35.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.424+0000 7fcff9fbe640 1 Processor -- start 2024-09-19T08:44:35.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.428+0000 7fcff9fbe640 1 -- start start 2024-09-19T08:44:35.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.428+0000 7fcff9fbe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcff415a990 0x7fcff406e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:35.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.428+0000 7fcff9fbe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcff406e8e0 0x7fcff406ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:35.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.428+0000 7fcff9fbe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcff40cf850 0x7fcff40cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:35.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.428+0000 7fcff9fbe640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fcff406f240 con 0x7fcff40cf850 2024-09-19T08:44:35.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.428+0000 7fcff9fbe640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fcff40d05f0 con 0x7fcff415a990 2024-09-19T08:44:35.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.428+0000 7fcff9fbe640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcff406d780 con 0x7fcff406e8e0 2024-09-19T08:44:35.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.428+0000 7fcff3fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcff40cf850 0x7fcff40cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:35.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.428+0000 7fcff3fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcff40cf850 0x7fcff40cfce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41300/0 (socket says 172.21.15.98:41300) 2024-09-19T08:44:35.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.428+0000 7fcff3fff640 1 -- 172.21.15.98:0/3892897601 learned_addr learned my addr 172.21.15.98:0/3892897601 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:35.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.428+0000 7fcff37fe640 1 --2- 172.21.15.98:0/3892897601 >> v2:172.21.15.139:3300/0 conn(0x7fcff415a990 0x7fcff406e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:35.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.428+0000 7fcfeaffd640 1 --2- 172.21.15.98:0/3892897601 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcff406e8e0 0x7fcff406ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:35.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.428+0000 7fcff3fff640 1 -- 172.21.15.98:0/3892897601 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcff406e8e0 msgr2=0x7fcff406ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:35.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.428+0000 7fcff3fff640 1 --2- 172.21.15.98:0/3892897601 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcff406e8e0 0x7fcff406ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:35.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.428+0000 7fcff3fff640 1 -- 172.21.15.98:0/3892897601 >> v2:172.21.15.139:3300/0 conn(0x7fcff415a990 msgr2=0x7fcff406e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:35.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.428+0000 7fcff3fff640 1 --2- 172.21.15.98:0/3892897601 >> v2:172.21.15.139:3300/0 conn(0x7fcff415a990 0x7fcff406e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:35.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.428+0000 7fcff3fff640 1 -- 172.21.15.98:0/3892897601 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff406da60 con 0x7fcff40cf850 2024-09-19T08:44:35.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.432+0000 7fcff3fff640 1 --2- 172.21.15.98:0/3892897601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcff40cf850 0x7fcff40cfce0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fcff40d19e0 tx=0x7fcfec01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=257f3d8673444624 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:35.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.432+0000 7fcff2ffd640 1 -- 172.21.15.98:0/3892897601 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec0a5670 con 0x7fcff40cf850 2024-09-19T08:44:35.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.432+0000 7fcff2ffd640 1 -- 172.21.15.98:0/3892897601 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfec0a5810 con 0x7fcff40cf850 2024-09-19T08:44:35.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.432+0000 7fcff2ffd640 1 -- 172.21.15.98:0/3892897601 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec0a5be0 con 0x7fcff40cf850 2024-09-19T08:44:35.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.432+0000 7fcff9fbe640 1 -- 172.21.15.98:0/3892897601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcff40cf850 msgr2=0x7fcff40cfce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:35.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.432+0000 7fcff9fbe640 1 --2- 172.21.15.98:0/3892897601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcff40cf850 0x7fcff40cfce0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fcff40d19e0 tx=0x7fcfec01c960 comp rx=0 tx=0).stop 2024-09-19T08:44:35.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.432+0000 7fcff9fbe640 1 -- 172.21.15.98:0/3892897601 shutdown_connections 2024-09-19T08:44:35.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.432+0000 7fcff9fbe640 1 --2- 172.21.15.98:0/3892897601 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcff40cf850 0x7fcff40cfce0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:35.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.432+0000 7fcff9fbe640 1 --2- 172.21.15.98:0/3892897601 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcff406e8e0 0x7fcff406ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:35.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.432+0000 7fcff9fbe640 1 --2- 172.21.15.98:0/3892897601 >> v2:172.21.15.139:3300/0 conn(0x7fcff415a990 0x7fcff406e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:35.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.432+0000 7fcff9fbe640 1 -- 172.21.15.98:0/3892897601 >> 172.21.15.98:0/3892897601 conn(0x7fcff40b25c0 msgr2=0x7fcff40b29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:35.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.432+0000 7fcff9fbe640 1 -- 172.21.15.98:0/3892897601 shutdown_connections 2024-09-19T08:44:35.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.432+0000 7fcff9fbe640 1 -- 172.21.15.98:0/3892897601 wait complete. 2024-09-19T08:44:35.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.432+0000 7fcff9fbe640 1 Processor -- start 2024-09-19T08:44:35.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.432+0000 7fcff9fbe640 1 -- start start 2024-09-19T08:44:35.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.432+0000 7fcff9fbe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcff406e8e0 0x7fcff414cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:35.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.432+0000 7fcff9fbe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcff40cf850 0x7fcff40b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:35.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.432+0000 7fcff9fbe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcff415a990 0x7fcff4157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:35.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.432+0000 7fcff9fbe640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcff40719a0 con 0x7fcff415a990 2024-09-19T08:44:35.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.432+0000 7fcff9fbe640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcff4071820 con 0x7fcff406e8e0 2024-09-19T08:44:35.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.432+0000 7fcff9fbe640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcff4071b20 con 0x7fcff40cf850 2024-09-19T08:44:35.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.436+0000 7fcff3fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcff415a990 0x7fcff4157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:35.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.436+0000 7fcff3fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcff415a990 0x7fcff4157b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41306/0 (socket says 172.21.15.98:41306) 2024-09-19T08:44:35.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.436+0000 7fcff3fff640 1 -- 172.21.15.98:0/3884928783 learned_addr learned my addr 172.21.15.98:0/3884928783 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:35.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.436+0000 7fcff37fe640 1 --2- 172.21.15.98:0/3884928783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcff406e8e0 0x7fcff414cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:35.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.436+0000 7fcfeaffd640 1 --2- 172.21.15.98:0/3884928783 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcff40cf850 0x7fcff40b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:35.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.436+0000 7fcff3fff640 1 -- 172.21.15.98:0/3884928783 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcff40cf850 msgr2=0x7fcff40b42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:35.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.436+0000 7fcff3fff640 1 --2- 172.21.15.98:0/3884928783 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcff40cf850 0x7fcff40b42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:35.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.436+0000 7fcff3fff640 1 -- 172.21.15.98:0/3884928783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcff406e8e0 msgr2=0x7fcff414cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:35.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.436+0000 7fcff3fff640 1 --2- 172.21.15.98:0/3884928783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcff406e8e0 0x7fcff414cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:35.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.436+0000 7fcff3fff640 1 -- 172.21.15.98:0/3884928783 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff40b4c00 con 0x7fcff415a990 2024-09-19T08:44:35.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.436+0000 7fcff3fff640 1 --2- 172.21.15.98:0/3884928783 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcff415a990 0x7fcff4157b50 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fcfec0af040 tx=0x7fcfec019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:35.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.436+0000 7fcff0ff9640 1 -- 172.21.15.98:0/3884928783 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec0bc020 con 0x7fcff415a990 2024-09-19T08:44:35.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.436+0000 7fcff0ff9640 1 -- 172.21.15.98:0/3884928783 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfec0b8670 con 0x7fcff415a990 2024-09-19T08:44:35.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.436+0000 7fcff0ff9640 1 -- 172.21.15.98:0/3884928783 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec009510 con 0x7fcff415a990 2024-09-19T08:44:35.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.436+0000 7fcff9fbe640 1 -- 172.21.15.98:0/3884928783 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcff40b4ee0 con 0x7fcff415a990 2024-09-19T08:44:35.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.436+0000 7fcff9fbe640 1 -- 172.21.15.98:0/3884928783 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcff4155ac0 con 0x7fcff415a990 2024-09-19T08:44:35.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.436+0000 7fcff9fbe640 1 -- 172.21.15.98:0/3884928783 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcff4166560 con 0x7fcff415a990 2024-09-19T08:44:35.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.436+0000 7fcff0ff9640 1 -- 172.21.15.98:0/3884928783 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 87700+0+0 (secure 0 0 0) 0x7fcfec0bf020 con 0x7fcff415a990 2024-09-19T08:44:35.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.436+0000 7fcff0ff9640 1 --2- 172.21.15.98:0/3884928783 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcfd4067bc0 0x7fcfd406a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:35.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.436+0000 7fcff0ff9640 1 -- 172.21.15.98:0/3884928783 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7fcfec120de0 con 0x7fcff415a990 2024-09-19T08:44:35.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.436+0000 7fcff37fe640 1 --2- 172.21.15.98:0/3884928783 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcfd4067bc0 0x7fcfd406a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:35.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.440+0000 7fcff37fe640 1 --2- 172.21.15.98:0/3884928783 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcfd4067bc0 0x7fcfd406a080 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcff40c7a50 tx=0x7fcfe4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:35.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.440+0000 7fcff0ff9640 1 -- 172.21.15.98:0/3884928783 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fcfec0c6030 con 0x7fcff415a990 2024-09-19T08:44:35.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.636+0000 7fcff9fbe640 1 -- 172.21.15.98:0/3884928783 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "cephfs", "sub_name": "subvol41500", "snap_name": "subvol_snap63800", "target": ["mon-mgr", ""]}) -- 0x7fcff414d2f0 con 0x7fcfd4067bc0 2024-09-19T08:44:35.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.672+0000 7fcff0ff9640 1 -- 172.21.15.98:0/3884928783 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+115 (secure 0 0 0) 0x7fcff414d2f0 con 0x7fcfd4067bc0 2024-09-19T08:44:35.675 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:44:35.675 INFO:teuthology.orchestra.run.smithi098.stdout: "created_at": "2024-09-19 08:44:15.545353", 2024-09-19T08:44:35.676 INFO:teuthology.orchestra.run.smithi098.stdout: "data_pool": "cephfs_data", 2024-09-19T08:44:35.676 INFO:teuthology.orchestra.run.smithi098.stdout: "has_pending_clones": "no" 2024-09-19T08:44:35.676 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:44:35.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.676+0000 7fcff9fbe640 1 -- 172.21.15.98:0/3884928783 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcfd4067bc0 msgr2=0x7fcfd406a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:35.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.676+0000 7fcff9fbe640 1 --2- 172.21.15.98:0/3884928783 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcfd4067bc0 0x7fcfd406a080 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcff40c7a50 tx=0x7fcfe4008040 comp rx=0 tx=0).stop 2024-09-19T08:44:35.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.676+0000 7fcff9fbe640 1 -- 172.21.15.98:0/3884928783 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcff415a990 msgr2=0x7fcff4157b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:35.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.676+0000 7fcff9fbe640 1 --2- 172.21.15.98:0/3884928783 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcff415a990 0x7fcff4157b50 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fcfec0af040 tx=0x7fcfec019700 comp rx=0 tx=0).stop 2024-09-19T08:44:35.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.676+0000 7fcff9fbe640 1 -- 172.21.15.98:0/3884928783 shutdown_connections 2024-09-19T08:44:35.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.676+0000 7fcff9fbe640 1 --2- 172.21.15.98:0/3884928783 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcfd4067bc0 0x7fcfd406a080 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:35.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.676+0000 7fcff9fbe640 1 --2- 172.21.15.98:0/3884928783 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcff415a990 0x7fcff4157b50 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:35.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.676+0000 7fcff9fbe640 1 --2- 172.21.15.98:0/3884928783 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcff40cf850 0x7fcff40b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:35.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.676+0000 7fcff9fbe640 1 --2- 172.21.15.98:0/3884928783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcff406e8e0 0x7fcff414cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:35.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.676+0000 7fcff9fbe640 1 -- 172.21.15.98:0/3884928783 >> 172.21.15.98:0/3884928783 conn(0x7fcff40b25c0 msgr2=0x7fcff40ccbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:35.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.676+0000 7fcff9fbe640 1 -- 172.21.15.98:0/3884928783 shutdown_connections 2024-09-19T08:44:35.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.676+0000 7fcff9fbe640 1 -- 172.21.15.98:0/3884928783 wait complete. 2024-09-19T08:44:35.695 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs subvol41500 2024-09-19T08:44:35.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.816+0000 7f3d8899d640 1 Processor -- start 2024-09-19T08:44:35.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.816+0000 7f3d8899d640 1 -- start start 2024-09-19T08:44:35.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.816+0000 7f3d8899d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d8415c290 0x7f3d840cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:35.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.816+0000 7f3d8899d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3d8406d4c0 0x7f3d8406d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:35.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.816+0000 7f3d8899d640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d841608c0 con 0x7f3d840cf700 2024-09-19T08:44:35.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.816+0000 7f3d8899d640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f3d8406de70 con 0x7f3d8406d4c0 2024-09-19T08:44:35.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.816+0000 7f3d8899d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3d84160e80 con 0x7f3d8415c290 2024-09-19T08:44:35.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.820+0000 7f3d82d9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3d8406d4c0 0x7f3d8406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:35.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.820+0000 7f3d7bfff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f3d840cf700 0x7f3d8415ff50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:45564/0 (socket says 172.21.15.98:45564) 2024-09-19T08:44:35.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.820+0000 7f3d7bfff640 1 -- 172.21.15.98:0/3411524072 learned_addr learned my addr 172.21.15.98:0/3411524072 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:35.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.820+0000 7f3d8259c640 1 --2- 172.21.15.98:0/3411524072 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d8415c290 0x7f3d840cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:35.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.820+0000 7f3d82d9d640 1 -- 172.21.15.98:0/3411524072 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d8415c290 msgr2=0x7f3d840cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:35.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.820+0000 7f3d82d9d640 1 --2- 172.21.15.98:0/3411524072 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d8415c290 0x7f3d840cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:35.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.820+0000 7f3d82d9d640 1 -- 172.21.15.98:0/3411524072 >> v1:172.21.15.98:6789/0 conn(0x7f3d840cf700 legacy=0x7f3d8415ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:35.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.820+0000 7f3d82d9d640 1 -- 172.21.15.98:0/3411524072 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d84160ba0 con 0x7f3d8406d4c0 2024-09-19T08:44:35.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.820+0000 7f3d81d9b640 1 -- 172.21.15.98:0/3411524072 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4080931346 0 0) 0x7f3d841608c0 con 0x7f3d840cf700 2024-09-19T08:44:35.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.820+0000 7f3d8259c640 1 --2- 172.21.15.98:0/3411524072 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d8415c290 0x7f3d840cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:35.822 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.820+0000 7f3d82d9d640 1 --2- 172.21.15.98:0/3411524072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d8406d4c0 0x7f3d8406d930 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c009d90 tx=0x7f3d7c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d82e321c24bcb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:35.822 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.820+0000 7f3d81d9b640 1 -- 172.21.15.98:0/3411524072 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c0b2070 con 0x7f3d8406d4c0 2024-09-19T08:44:35.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.820+0000 7f3d81d9b640 1 -- 172.21.15.98:0/3411524072 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d7c0a5830 con 0x7f3d8406d4c0 2024-09-19T08:44:35.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.820+0000 7f3d81d9b640 1 -- 172.21.15.98:0/3411524072 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c0a5c00 con 0x7f3d8406d4c0 2024-09-19T08:44:35.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.820+0000 7f3d8899d640 1 -- 172.21.15.98:0/3411524072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d8406d4c0 msgr2=0x7f3d8406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:35.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.820+0000 7f3d8899d640 1 --2- 172.21.15.98:0/3411524072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d8406d4c0 0x7f3d8406d930 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c009d90 tx=0x7f3d7c01c970 comp rx=0 tx=0).stop 2024-09-19T08:44:35.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.820+0000 7f3d8899d640 1 -- 172.21.15.98:0/3411524072 shutdown_connections 2024-09-19T08:44:35.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.820+0000 7f3d8899d640 1 --2- 172.21.15.98:0/3411524072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d8406d4c0 0x7f3d8406d930 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:35.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.820+0000 7f3d8899d640 1 --2- 172.21.15.98:0/3411524072 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d8415c290 0x7f3d840cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:35.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.820+0000 7f3d8899d640 1 -- 172.21.15.98:0/3411524072 >> 172.21.15.98:0/3411524072 conn(0x7f3d8406c090 msgr2=0x7f3d8406c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:35.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.820+0000 7f3d8899d640 1 -- 172.21.15.98:0/3411524072 shutdown_connections 2024-09-19T08:44:35.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.820+0000 7f3d8899d640 1 -- 172.21.15.98:0/3411524072 wait complete. 2024-09-19T08:44:35.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.824+0000 7f3d8899d640 1 Processor -- start 2024-09-19T08:44:35.825 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.824+0000 7f3d8899d640 1 -- start start 2024-09-19T08:44:35.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.824+0000 7f3d8899d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d8406d4c0 0x7f3d84155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:35.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.824+0000 7f3d8899d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d840cf700 0x7f3d84154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:35.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.824+0000 7f3d8899d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d8415c290 0x7f3d840b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:35.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.824+0000 7f3d8899d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3d840b5400 con 0x7f3d8415c290 2024-09-19T08:44:35.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.824+0000 7f3d8899d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3d84160660 con 0x7f3d8406d4c0 2024-09-19T08:44:35.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.824+0000 7f3d8899d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3d840b5580 con 0x7f3d840cf700 2024-09-19T08:44:35.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.824+0000 7f3d82d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d8415c290 0x7f3d840b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:35.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.824+0000 7f3d82d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d8415c290 0x7f3d840b24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41322/0 (socket says 172.21.15.98:41322) 2024-09-19T08:44:35.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.824+0000 7f3d82d9d640 1 -- 172.21.15.98:0/3889839002 learned_addr learned my addr 172.21.15.98:0/3889839002 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:35.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.824+0000 7f3d7bfff640 1 --2- 172.21.15.98:0/3889839002 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d840cf700 0x7f3d84154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:35.827 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.824+0000 7f3d8259c640 1 --2- 172.21.15.98:0/3889839002 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d8406d4c0 0x7f3d84155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:35.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.824+0000 7f3d82d9d640 1 -- 172.21.15.98:0/3889839002 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d840cf700 msgr2=0x7f3d84154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:35.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.824+0000 7f3d82d9d640 1 --2- 172.21.15.98:0/3889839002 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d840cf700 0x7f3d84154930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:35.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.824+0000 7f3d82d9d640 1 -- 172.21.15.98:0/3889839002 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d8406d4c0 msgr2=0x7f3d84155870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:35.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.824+0000 7f3d82d9d640 1 --2- 172.21.15.98:0/3889839002 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d8406d4c0 0x7f3d84155870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:35.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.824+0000 7f3d82d9d640 1 -- 172.21.15.98:0/3889839002 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d84150c70 con 0x7f3d8415c290 2024-09-19T08:44:35.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.824+0000 7f3d7bfff640 1 --2- 172.21.15.98:0/3889839002 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d840cf700 0x7f3d84154930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:35.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.824+0000 7f3d8259c640 1 --2- 172.21.15.98:0/3889839002 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d8406d4c0 0x7f3d84155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:35.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.824+0000 7f3d82d9d640 1 --2- 172.21.15.98:0/3889839002 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d8415c290 0x7f3d840b24d0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c0b0040 tx=0x7f3d7c019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:35.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.824+0000 7f3d7affd640 1 -- 172.21.15.98:0/3889839002 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c0b2050 con 0x7f3d8415c290 2024-09-19T08:44:35.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.824+0000 7f3d7affd640 1 -- 172.21.15.98:0/3889839002 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d7c0be070 con 0x7f3d8415c290 2024-09-19T08:44:35.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.824+0000 7f3d8899d640 1 -- 172.21.15.98:0/3889839002 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d84158e10 con 0x7f3d8415c290 2024-09-19T08:44:35.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.824+0000 7f3d7affd640 1 -- 172.21.15.98:0/3889839002 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c0ada50 con 0x7f3d8415c290 2024-09-19T08:44:35.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.824+0000 7f3d8899d640 1 -- 172.21.15.98:0/3889839002 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d841593d0 con 0x7f3d8415c290 2024-09-19T08:44:35.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.824+0000 7f3d8899d640 1 -- 172.21.15.98:0/3889839002 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d8406d930 con 0x7f3d8415c290 2024-09-19T08:44:35.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.824+0000 7f3d7affd640 1 -- 172.21.15.98:0/3889839002 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 87700+0+0 (secure 0 0 0) 0x7f3d7c0adbf0 con 0x7f3d8415c290 2024-09-19T08:44:35.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.828+0000 7f3d7affd640 1 --2- 172.21.15.98:0/3889839002 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3d64067b90 0x7f3d6406a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:35.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.828+0000 7f3d7affd640 1 -- 172.21.15.98:0/3889839002 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f3d7c1215a0 con 0x7f3d8415c290 2024-09-19T08:44:35.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.828+0000 7f3d8259c640 1 --2- 172.21.15.98:0/3889839002 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3d64067b90 0x7f3d6406a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:35.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.828+0000 7f3d8259c640 1 --2- 172.21.15.98:0/3889839002 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3d64067b90 0x7f3d6406a050 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3d74008890 tx=0x7f3d74008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:35.832 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:35.832+0000 7f3d7affd640 1 -- 172.21.15.98:0/3889839002 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3d7c016560 con 0x7f3d8415c290 2024-09-19T08:44:36.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.024+0000 7f3d8899d640 1 -- 172.21.15.98:0/3889839002 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "subvol41500", "target": ["mon-mgr", ""]}) -- 0x7f3d84154690 con 0x7f3d64067b90 2024-09-19T08:44:36.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.052+0000 7f3d7affd640 1 -- 172.21.15.98:0/3889839002 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+87 (secure 0 0 0) 0x7f3d84154690 con 0x7f3d64067b90 2024-09-19T08:44:36.056 INFO:teuthology.orchestra.run.smithi098.stdout:[ 2024-09-19T08:44:36.056 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:44:36.056 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "fake" 2024-09-19T08:44:36.056 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:44:36.056 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:44:36.056 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "subvol_snap63800" 2024-09-19T08:44:36.056 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:44:36.056 INFO:teuthology.orchestra.run.smithi098.stdout:] 2024-09-19T08:44:36.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.056+0000 7f3d8899d640 1 -- 172.21.15.98:0/3889839002 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3d64067b90 msgr2=0x7f3d6406a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:36.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.056+0000 7f3d8899d640 1 --2- 172.21.15.98:0/3889839002 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3d64067b90 0x7f3d6406a050 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3d74008890 tx=0x7f3d74008040 comp rx=0 tx=0).stop 2024-09-19T08:44:36.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.056+0000 7f3d8899d640 1 -- 172.21.15.98:0/3889839002 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d8415c290 msgr2=0x7f3d840b24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:36.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.056+0000 7f3d8899d640 1 --2- 172.21.15.98:0/3889839002 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d8415c290 0x7f3d840b24d0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c0b0040 tx=0x7f3d7c019710 comp rx=0 tx=0).stop 2024-09-19T08:44:36.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.056+0000 7f3d8899d640 1 -- 172.21.15.98:0/3889839002 shutdown_connections 2024-09-19T08:44:36.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.056+0000 7f3d8899d640 1 --2- 172.21.15.98:0/3889839002 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3d64067b90 0x7f3d6406a050 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:36.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.056+0000 7f3d8899d640 1 --2- 172.21.15.98:0/3889839002 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3d8415c290 0x7f3d840b24d0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:36.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.056+0000 7f3d8899d640 1 --2- 172.21.15.98:0/3889839002 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3d840cf700 0x7f3d84154930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:36.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.056+0000 7f3d8899d640 1 --2- 172.21.15.98:0/3889839002 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3d8406d4c0 0x7f3d84155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:36.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.056+0000 7f3d8899d640 1 -- 172.21.15.98:0/3889839002 >> 172.21.15.98:0/3889839002 conn(0x7f3d8406c090 msgr2=0x7f3d8415f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:36.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.056+0000 7f3d8899d640 1 -- 172.21.15.98:0/3889839002 shutdown_connections 2024-09-19T08:44:36.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.056+0000 7f3d8899d640 1 -- 172.21.15.98:0/3889839002 wait complete. 2024-09-19T08:44:36.076 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol41500 subvol_snap63800 2024-09-19T08:44:36.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.200+0000 7f4a59fba640 1 Processor -- start 2024-09-19T08:44:36.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.200+0000 7f4a59fba640 1 -- start start 2024-09-19T08:44:36.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.200+0000 7f4a59fba640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4a5415f6e0 0x7f4a5406d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:36.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.200+0000 7f4a59fba640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a5415baf0 con 0x7f4a54073d90 2024-09-19T08:44:36.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.200+0000 7f4a59fba640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a5415bdd0 con 0x7f4a5415fac0 2024-09-19T08:44:36.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.200+0000 7f4a59fba640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f4a5415c8b0 con 0x7f4a5415f6e0 2024-09-19T08:44:36.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.200+0000 7f4a53fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f4a54073d90 0x7f4a540ce420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:45580/0 (socket says 172.21.15.98:45580) 2024-09-19T08:44:36.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.200+0000 7f4a53fff640 1 -- 172.21.15.98:0/3030021188 learned_addr learned my addr 172.21.15.98:0/3030021188 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:36.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.200+0000 7f4a537fe640 1 --2- 172.21.15.98:0/3030021188 >> v2:172.21.15.139:3300/0 conn(0x7f4a5415f6e0 0x7f4a5406d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:36.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.200+0000 7f4a52ffd640 1 -- 172.21.15.98:0/3030021188 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1022509232 0 0) 0x7f4a5415baf0 con 0x7f4a54073d90 2024-09-19T08:44:36.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.200+0000 7f4a52ffd640 1 -- 172.21.15.98:0/3030021188 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a40003530 con 0x7f4a54073d90 2024-09-19T08:44:36.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.200+0000 7f4a52ffd640 1 -- 172.21.15.98:0/3030021188 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2058134049 0 0) 0x7f4a40003530 con 0x7f4a54073d90 2024-09-19T08:44:36.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.200+0000 7f4a52ffd640 1 -- 172.21.15.98:0/3030021188 >> v2:172.21.15.139:3300/0 conn(0x7f4a5415f6e0 msgr2=0x7f4a5406d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:36.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.200+0000 7f4a52ffd640 1 --2- 172.21.15.98:0/3030021188 >> v2:172.21.15.139:3300/0 conn(0x7f4a5415f6e0 0x7f4a5406d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:36.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.200+0000 7f4a52ffd640 1 -- 172.21.15.98:0/3030021188 >> v1:172.21.15.139:6789/0 conn(0x7f4a5415fac0 legacy=0x7f4a54073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:36.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.200+0000 7f4a52ffd640 1 -- 172.21.15.98:0/3030021188 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a5415c0b0 con 0x7f4a54073d90 2024-09-19T08:44:36.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.200+0000 7f4a52ffd640 1 -- 172.21.15.98:0/3030021188 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f4a4c001a00 con 0x7f4a54073d90 2024-09-19T08:44:36.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.200+0000 7f4a52ffd640 1 -- 172.21.15.98:0/3030021188 >> v1:172.21.15.98:6789/0 conn(0x7f4a54073d90 legacy=0x7f4a540ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:36.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.200+0000 7f4a52ffd640 1 --2- 172.21.15.98:0/3030021188 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a400039c0 0x7f4a40003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:36.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.200+0000 7f4a52ffd640 1 --2- 172.21.15.98:0/3030021188 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a40004810 0x7f4a40004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:36.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.200+0000 7f4a52ffd640 1 --2- 172.21.15.98:0/3030021188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a400051d0 0x7f4a4000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:36.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.200+0000 7f4a52ffd640 1 -- 172.21.15.98:0/3030021188 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5415c8b0 con 0x7f4a40004810 2024-09-19T08:44:36.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.200+0000 7f4a52ffd640 1 -- 172.21.15.98:0/3030021188 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4000c170 con 0x7f4a400051d0 2024-09-19T08:44:36.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.200+0000 7f4a52ffd640 1 -- 172.21.15.98:0/3030021188 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4a4000c370 con 0x7f4a400039c0 2024-09-19T08:44:36.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.200+0000 7f4a4bfff640 1 --2- 172.21.15.98:0/3030021188 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a40004810 0x7f4a40004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:36.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.204+0000 7f4a537fe640 1 --2- 172.21.15.98:0/3030021188 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a400039c0 0x7f4a40003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:36.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.204+0000 7f4a4bfff640 1 -- 172.21.15.98:0/3030021188 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a400039c0 msgr2=0x7f4a40003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:36.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.204+0000 7f4a4bfff640 1 --2- 172.21.15.98:0/3030021188 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a400039c0 0x7f4a40003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:36.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.204+0000 7f4a4bfff640 1 -- 172.21.15.98:0/3030021188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a400051d0 msgr2=0x7f4a4000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:36.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.204+0000 7f4a4bfff640 1 --2- 172.21.15.98:0/3030021188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a400051d0 0x7f4a4000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:36.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.204+0000 7f4a4bfff640 1 -- 172.21.15.98:0/3030021188 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a40003530 con 0x7f4a40004810 2024-09-19T08:44:36.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.204+0000 7f4a53fff640 1 -- 172.21.15.98:0/3030021188 reap_dead start 2024-09-19T08:44:36.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.204+0000 7f4a4bfff640 1 --2- 172.21.15.98:0/3030021188 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a40004810 0x7f4a40004c90 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c001780 tx=0x7f4a3c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc5d44296d17ad0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:36.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.204+0000 7f4a52ffd640 1 -- 172.21.15.98:0/3030021188 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c0013a0 con 0x7f4a40004810 2024-09-19T08:44:36.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.204+0000 7f4a52ffd640 1 -- 172.21.15.98:0/3030021188 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a3c0052f0 con 0x7f4a40004810 2024-09-19T08:44:36.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.204+0000 7f4a52ffd640 1 -- 172.21.15.98:0/3030021188 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c003840 con 0x7f4a40004810 2024-09-19T08:44:36.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.204+0000 7f4a59fba640 1 -- 172.21.15.98:0/3030021188 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a40004810 msgr2=0x7f4a40004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:36.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.204+0000 7f4a59fba640 1 --2- 172.21.15.98:0/3030021188 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a40004810 0x7f4a40004c90 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c001780 tx=0x7f4a3c01c420 comp rx=0 tx=0).stop 2024-09-19T08:44:36.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.204+0000 7f4a59fba640 1 -- 172.21.15.98:0/3030021188 shutdown_connections 2024-09-19T08:44:36.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.204+0000 7f4a59fba640 1 --2- 172.21.15.98:0/3030021188 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a40004810 0x7f4a40004c90 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:36.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.204+0000 7f4a59fba640 1 -- 172.21.15.98:0/3030021188 >> 172.21.15.98:0/3030021188 conn(0x7f4a5406c090 msgr2=0x7f4a5406c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:36.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.204+0000 7f4a59fba640 1 -- 172.21.15.98:0/3030021188 shutdown_connections 2024-09-19T08:44:36.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.204+0000 7f4a59fba640 1 -- 172.21.15.98:0/3030021188 wait complete. 2024-09-19T08:44:36.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.208+0000 7f4a59fba640 1 Processor -- start 2024-09-19T08:44:36.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.208+0000 7f4a59fba640 1 -- start start 2024-09-19T08:44:36.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.208+0000 7f4a59fba640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a40004810 0x7f4a541f73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:36.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.208+0000 7f4a59fba640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a540b1010 0x7f4a54147670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:36.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.208+0000 7f4a59fba640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a54147bb0 0x7f4a540ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:36.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.208+0000 7f4a59fba640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4a540b4c00 con 0x7f4a54147bb0 2024-09-19T08:44:36.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.208+0000 7f4a59fba640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4a540b4a80 con 0x7f4a540b1010 2024-09-19T08:44:36.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.208+0000 7f4a59fba640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4a540b4d80 con 0x7f4a40004810 2024-09-19T08:44:36.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.208+0000 7f4a53fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a54147bb0 0x7f4a540ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:36.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.208+0000 7f4a53fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a54147bb0 0x7f4a540ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41344/0 (socket says 172.21.15.98:41344) 2024-09-19T08:44:36.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.208+0000 7f4a53fff640 1 -- 172.21.15.98:0/3613007063 learned_addr learned my addr 172.21.15.98:0/3613007063 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:36.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.208+0000 7f4a537fe640 1 --2- 172.21.15.98:0/3613007063 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a40004810 0x7f4a541f73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:36.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.208+0000 7f4a4bfff640 1 --2- 172.21.15.98:0/3613007063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a540b1010 0x7f4a54147670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:36.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.208+0000 7f4a53fff640 1 -- 172.21.15.98:0/3613007063 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a40004810 msgr2=0x7f4a541f73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:36.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.208+0000 7f4a53fff640 1 --2- 172.21.15.98:0/3613007063 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a40004810 0x7f4a541f73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:36.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.208+0000 7f4a53fff640 1 -- 172.21.15.98:0/3613007063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a540b1010 msgr2=0x7f4a54147670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:36.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.208+0000 7f4a53fff640 1 --2- 172.21.15.98:0/3613007063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a540b1010 0x7f4a54147670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:36.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.208+0000 7f4a53fff640 1 -- 172.21.15.98:0/3613007063 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a54148030 con 0x7f4a54147bb0 2024-09-19T08:44:36.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.208+0000 7f4a53fff640 1 --2- 172.21.15.98:0/3613007063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a54147bb0 0x7f4a540ae8b0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c001380 tx=0x7f4a4c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:36.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.208+0000 7f4a50ff9640 1 -- 172.21.15.98:0/3613007063 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c00a610 con 0x7f4a54147bb0 2024-09-19T08:44:36.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.208+0000 7f4a50ff9640 1 -- 172.21.15.98:0/3613007063 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a4c00a7b0 con 0x7f4a54147bb0 2024-09-19T08:44:36.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.208+0000 7f4a59fba640 1 -- 172.21.15.98:0/3613007063 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a540b0530 con 0x7f4a54147bb0 2024-09-19T08:44:36.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.208+0000 7f4a50ff9640 1 -- 172.21.15.98:0/3613007063 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c00aba0 con 0x7f4a54147bb0 2024-09-19T08:44:36.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.208+0000 7f4a59fba640 1 -- 172.21.15.98:0/3613007063 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a541f7910 con 0x7f4a54147bb0 2024-09-19T08:44:36.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.208+0000 7f4a59fba640 1 -- 172.21.15.98:0/3613007063 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a540b5110 con 0x7f4a54147bb0 2024-09-19T08:44:36.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.208+0000 7f4a50ff9640 1 -- 172.21.15.98:0/3613007063 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 87700+0+0 (secure 0 0 0) 0x7f4a4c011070 con 0x7f4a54147bb0 2024-09-19T08:44:36.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.208+0000 7f4a50ff9640 1 --2- 172.21.15.98:0/3613007063 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4a34067b90 0x7f4a3406a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:36.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.212+0000 7f4a50ff9640 1 -- 172.21.15.98:0/3613007063 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f4a4c085a10 con 0x7f4a54147bb0 2024-09-19T08:44:36.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.212+0000 7f4a537fe640 1 --2- 172.21.15.98:0/3613007063 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4a34067b90 0x7f4a3406a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:36.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.212+0000 7f4a537fe640 1 --2- 172.21.15.98:0/3613007063 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4a34067b90 0x7f4a3406a050 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4a44009ab0 tx=0x7f4a44008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:36.216 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.216+0000 7f4a50ff9640 1 -- 172.21.15.98:0/3613007063 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f4a4c01a060 con 0x7f4a54147bb0 2024-09-19T08:44:36.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.404+0000 7f4a59fba640 1 -- 172.21.15.98:0/3613007063 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol41500", "snap_name": "subvol_snap63800", "target": ["mon-mgr", ""]}) -- 0x7f4a5415e2e0 con 0x7f4a34067b90 2024-09-19T08:44:36.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.464+0000 7f4a50ff9640 1 -- 172.21.15.98:0/3613007063 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4a5415e2e0 con 0x7f4a34067b90 2024-09-19T08:44:36.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.468+0000 7f4a59fba640 1 -- 172.21.15.98:0/3613007063 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4a34067b90 msgr2=0x7f4a3406a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:36.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.468+0000 7f4a59fba640 1 --2- 172.21.15.98:0/3613007063 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4a34067b90 0x7f4a3406a050 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4a44009ab0 tx=0x7f4a44008040 comp rx=0 tx=0).stop 2024-09-19T08:44:36.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.468+0000 7f4a59fba640 1 -- 172.21.15.98:0/3613007063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a54147bb0 msgr2=0x7f4a540ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:36.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.468+0000 7f4a59fba640 1 --2- 172.21.15.98:0/3613007063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a54147bb0 0x7f4a540ae8b0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c001380 tx=0x7f4a4c00cc00 comp rx=0 tx=0).stop 2024-09-19T08:44:36.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.468+0000 7f4a59fba640 1 -- 172.21.15.98:0/3613007063 shutdown_connections 2024-09-19T08:44:36.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.468+0000 7f4a59fba640 1 --2- 172.21.15.98:0/3613007063 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4a34067b90 0x7f4a3406a050 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:36.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.468+0000 7f4a59fba640 1 --2- 172.21.15.98:0/3613007063 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a54147bb0 0x7f4a540ae8b0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:36.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.468+0000 7f4a59fba640 1 --2- 172.21.15.98:0/3613007063 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a540b1010 0x7f4a54147670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:36.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.468+0000 7f4a59fba640 1 --2- 172.21.15.98:0/3613007063 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a40004810 0x7f4a541f73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:36.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.468+0000 7f4a59fba640 1 -- 172.21.15.98:0/3613007063 >> 172.21.15.98:0/3613007063 conn(0x7f4a5406c090 msgr2=0x7f4a540b24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:36.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.468+0000 7f4a59fba640 1 -- 172.21.15.98:0/3613007063 shutdown_connections 2024-09-19T08:44:36.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.468+0000 7f4a59fba640 1 -- 172.21.15.98:0/3613007063 wait complete. 2024-09-19T08:44:36.484 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol41500 fake 2024-09-19T08:44:36.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.604+0000 7f14eea9d640 1 Processor -- start 2024-09-19T08:44:36.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.604+0000 7f14eea9d640 1 -- start start 2024-09-19T08:44:36.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.608+0000 7f14eea9d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14e815c290 0x7f14e80cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:36.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.608+0000 7f14eea9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f14e80cf700 0x7f14e815ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:36.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.608+0000 7f14eea9d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14e81608c0 con 0x7f14e806d4c0 2024-09-19T08:44:36.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.608+0000 7f14eea9d640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f14e8160e80 con 0x7f14e80cf700 2024-09-19T08:44:36.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.608+0000 7f14eea9d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f14e80cd510 con 0x7f14e815c290 2024-09-19T08:44:36.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.608+0000 7f14dffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f14e80cf700 0x7f14e815ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:36.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.608+0000 7f14dffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f14e80cf700 0x7f14e815ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41358/0 (socket says 172.21.15.98:41358) 2024-09-19T08:44:36.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.608+0000 7f14dffff640 1 -- 172.21.15.98:0/2216386107 learned_addr learned my addr 172.21.15.98:0/2216386107 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:36.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.608+0000 7f14dffff640 1 -- 172.21.15.98:0/2216386107 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14e815c290 msgr2=0x7f14e80cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:36.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.608+0000 7f14dffff640 1 --2- 172.21.15.98:0/2216386107 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14e815c290 0x7f14e80cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:36.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.608+0000 7f14dffff640 1 -- 172.21.15.98:0/2216386107 >> v1:172.21.15.139:6789/0 conn(0x7f14e806d4c0 legacy=0x7f14e80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:36.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.608+0000 7f14dffff640 1 -- 172.21.15.98:0/2216386107 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e8160ba0 con 0x7f14e80cf700 2024-09-19T08:44:36.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.608+0000 7f14dffff640 1 --2- 172.21.15.98:0/2216386107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14e80cf700 0x7f14e815ff50 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f14e80cd690 tx=0x7f14d801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f042ef0333ab6873 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:36.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.608+0000 7f14df7fe640 1 -- 172.21.15.98:0/2216386107 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d80b0070 con 0x7f14e80cf700 2024-09-19T08:44:36.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.608+0000 7f14df7fe640 1 -- 172.21.15.98:0/2216386107 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14d80ab600 con 0x7f14e80cf700 2024-09-19T08:44:36.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.608+0000 7f14df7fe640 1 -- 172.21.15.98:0/2216386107 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d80ab9d0 con 0x7f14e80cf700 2024-09-19T08:44:36.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.608+0000 7f14eea9d640 1 -- 172.21.15.98:0/2216386107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14e80cf700 msgr2=0x7f14e815ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:36.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.608+0000 7f14eea9d640 1 --2- 172.21.15.98:0/2216386107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14e80cf700 0x7f14e815ff50 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f14e80cd690 tx=0x7f14d801c500 comp rx=0 tx=0).stop 2024-09-19T08:44:36.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.608+0000 7f14eea9d640 1 -- 172.21.15.98:0/2216386107 shutdown_connections 2024-09-19T08:44:36.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.608+0000 7f14eea9d640 1 --2- 172.21.15.98:0/2216386107 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14e80cf700 0x7f14e815ff50 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:36.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.608+0000 7f14eea9d640 1 --2- 172.21.15.98:0/2216386107 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14e815c290 0x7f14e80cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:36.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.608+0000 7f14eea9d640 1 -- 172.21.15.98:0/2216386107 >> 172.21.15.98:0/2216386107 conn(0x7f14e806c090 msgr2=0x7f14e806c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:36.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.608+0000 7f14eea9d640 1 -- 172.21.15.98:0/2216386107 shutdown_connections 2024-09-19T08:44:36.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.608+0000 7f14eea9d640 1 -- 172.21.15.98:0/2216386107 wait complete. 2024-09-19T08:44:36.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.612+0000 7f14eea9d640 1 Processor -- start 2024-09-19T08:44:36.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.612+0000 7f14eea9d640 1 -- start start 2024-09-19T08:44:36.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.612+0000 7f14eea9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14e806d4c0 0x7f14e8147ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:36.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.612+0000 7f14eea9d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14e80cf700 0x7f14e8163510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:36.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.612+0000 7f14eea9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14e815c290 0x7f14e8159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:36.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.612+0000 7f14eea9d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f14e815c7f0 con 0x7f14e815c290 2024-09-19T08:44:36.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.612+0000 7f14eea9d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f14e815c670 con 0x7f14e806d4c0 2024-09-19T08:44:36.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.612+0000 7f14eea9d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f14e8160490 con 0x7f14e80cf700 2024-09-19T08:44:36.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.612+0000 7f14ed03a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14e815c290 0x7f14e8159ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:36.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.612+0000 7f14ed03a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14e815c290 0x7f14e8159ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41362/0 (socket says 172.21.15.98:41362) 2024-09-19T08:44:36.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.612+0000 7f14ed03a640 1 -- 172.21.15.98:0/2098348687 learned_addr learned my addr 172.21.15.98:0/2098348687 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:36.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.612+0000 7f14ec839640 1 --2- 172.21.15.98:0/2098348687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14e806d4c0 0x7f14e8147ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:36.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.612+0000 7f14dffff640 1 --2- 172.21.15.98:0/2098348687 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14e80cf700 0x7f14e8163510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:36.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.612+0000 7f14ed03a640 1 -- 172.21.15.98:0/2098348687 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14e80cf700 msgr2=0x7f14e8163510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:36.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.612+0000 7f14ed03a640 1 --2- 172.21.15.98:0/2098348687 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14e80cf700 0x7f14e8163510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:36.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.612+0000 7f14ed03a640 1 -- 172.21.15.98:0/2098348687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14e806d4c0 msgr2=0x7f14e8147ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:36.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.612+0000 7f14ed03a640 1 --2- 172.21.15.98:0/2098348687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14e806d4c0 0x7f14e8147ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:36.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.612+0000 7f14ed03a640 1 -- 172.21.15.98:0/2098348687 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e81483e0 con 0x7f14e815c290 2024-09-19T08:44:36.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.612+0000 7f14ec839640 1 --2- 172.21.15.98:0/2098348687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14e806d4c0 0x7f14e8147ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:36.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.612+0000 7f14ed03a640 1 --2- 172.21.15.98:0/2098348687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14e815c290 0x7f14e8159ae0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f14e000c7b0 tx=0x7f14e000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:36.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.612+0000 7f14dd7fa640 1 -- 172.21.15.98:0/2098348687 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e0004b70 con 0x7f14e815c290 2024-09-19T08:44:36.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.612+0000 7f14dd7fa640 1 -- 172.21.15.98:0/2098348687 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e0004d10 con 0x7f14e815c290 2024-09-19T08:44:36.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.612+0000 7f14eea9d640 1 -- 172.21.15.98:0/2098348687 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14e8163ea0 con 0x7f14e815c290 2024-09-19T08:44:36.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.612+0000 7f14dd7fa640 1 -- 172.21.15.98:0/2098348687 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e000aa90 con 0x7f14e815c290 2024-09-19T08:44:36.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.612+0000 7f14eea9d640 1 -- 172.21.15.98:0/2098348687 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14e815e3f0 con 0x7f14e815c290 2024-09-19T08:44:36.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.612+0000 7f14eea9d640 1 -- 172.21.15.98:0/2098348687 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14e815ff50 con 0x7f14e815c290 2024-09-19T08:44:36.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.616+0000 7f14dd7fa640 1 -- 172.21.15.98:0/2098348687 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 87700+0+0 (secure 0 0 0) 0x7f14e000ac30 con 0x7f14e815c290 2024-09-19T08:44:36.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.616+0000 7f14dd7fa640 1 --2- 172.21.15.98:0/2098348687 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f14d4067b90 0x7f14d406a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:36.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.616+0000 7f14dd7fa640 1 -- 172.21.15.98:0/2098348687 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f14e00864b0 con 0x7f14e815c290 2024-09-19T08:44:36.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.616+0000 7f14ec839640 1 --2- 172.21.15.98:0/2098348687 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f14d4067b90 0x7f14d406a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:36.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.616+0000 7f14ec839640 1 --2- 172.21.15.98:0/2098348687 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f14d4067b90 0x7f14d406a050 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f14e400a870 tx=0x7f14e4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:36.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.620+0000 7f14dd7fa640 1 -- 172.21.15.98:0/2098348687 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f14e0020070 con 0x7f14e815c290 2024-09-19T08:44:36.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.796+0000 7f14eea9d640 1 -- 172.21.15.98:0/2098348687 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol41500", "snap_name": "fake", "target": ["mon-mgr", ""]}) -- 0x7f14e80b66f0 con 0x7f14d4067b90 2024-09-19T08:44:36.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.848+0000 7f14dd7fa640 1 -- 172.21.15.98:0/2098348687 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f14e80b66f0 con 0x7f14d4067b90 2024-09-19T08:44:36.853 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.852+0000 7f14eea9d640 1 -- 172.21.15.98:0/2098348687 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f14d4067b90 msgr2=0x7f14d406a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:36.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.852+0000 7f14eea9d640 1 --2- 172.21.15.98:0/2098348687 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f14d4067b90 0x7f14d406a050 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f14e400a870 tx=0x7f14e4008040 comp rx=0 tx=0).stop 2024-09-19T08:44:36.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.852+0000 7f14eea9d640 1 -- 172.21.15.98:0/2098348687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14e815c290 msgr2=0x7f14e8159ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:36.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.852+0000 7f14eea9d640 1 --2- 172.21.15.98:0/2098348687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14e815c290 0x7f14e8159ae0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f14e000c7b0 tx=0x7f14e000cc70 comp rx=0 tx=0).stop 2024-09-19T08:44:36.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.852+0000 7f14eea9d640 1 -- 172.21.15.98:0/2098348687 shutdown_connections 2024-09-19T08:44:36.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.852+0000 7f14eea9d640 1 --2- 172.21.15.98:0/2098348687 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f14d4067b90 0x7f14d406a050 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:36.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.852+0000 7f14eea9d640 1 --2- 172.21.15.98:0/2098348687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14e815c290 0x7f14e8159ae0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:36.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.852+0000 7f14eea9d640 1 --2- 172.21.15.98:0/2098348687 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14e80cf700 0x7f14e8163510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:36.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.852+0000 7f14eea9d640 1 --2- 172.21.15.98:0/2098348687 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14e806d4c0 0x7f14e8147ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:36.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.852+0000 7f14eea9d640 1 -- 172.21.15.98:0/2098348687 >> 172.21.15.98:0/2098348687 conn(0x7f14e806c090 msgr2=0x7f14e815d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:36.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.852+0000 7f14eea9d640 1 -- 172.21.15.98:0/2098348687 shutdown_connections 2024-09-19T08:44:36.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:36.852+0000 7f14eea9d640 1 -- 172.21.15.98:0/2098348687 wait complete. 2024-09-19T08:44:36.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:36.869 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol41500/.meta) 2024-09-19T08:44:40.198 INFO:teuthology.orchestra.run.smithi098.stdout:[GLOBAL] 2024-09-19T08:44:40.199 INFO:teuthology.orchestra.run.smithi098.stdout:version = 1 2024-09-19T08:44:40.199 INFO:teuthology.orchestra.run.smithi098.stdout:type = subvolume 2024-09-19T08:44:40.199 INFO:teuthology.orchestra.run.smithi098.stdout:path = /volumes/_nogroup/subvol41500/e2a09114-b6be-47c8-8599-a8190ac98e2a 2024-09-19T08:44:40.199 INFO:teuthology.orchestra.run.smithi098.stdout:state = complete 2024-09-19T08:44:40.199 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:44:40.201 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol41500 2024-09-19T08:44:40.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.332+0000 7f1176bda640 1 Processor -- start 2024-09-19T08:44:40.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.332+0000 7f1176bda640 1 -- start start 2024-09-19T08:44:40.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.332+0000 7f1176bda640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1170158db0 0x7f117015f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:40.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.332+0000 7f1176bda640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11700ce710 con 0x7f11700b0620 2024-09-19T08:44:40.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.332+0000 7f1176bda640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1170159190 con 0x7f1170075380 2024-09-19T08:44:40.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.332+0000 7f1176bda640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1170075ad0 con 0x7f1170158db0 2024-09-19T08:44:40.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.332+0000 7f116ffff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f11700b0620 0x7f1170074c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40634/0 (socket says 172.21.15.98:40634) 2024-09-19T08:44:40.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.332+0000 7f116ffff640 1 -- 172.21.15.98:0/3557818138 learned_addr learned my addr 172.21.15.98:0/3557818138 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:40.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.332+0000 7f1174976640 1 --2- 172.21.15.98:0/3557818138 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1170158db0 0x7f117015f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:40.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.332+0000 7f116f7fe640 1 -- 172.21.15.98:0/3557818138 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 682684927 0 0) 0x7f11700ce710 con 0x7f11700b0620 2024-09-19T08:44:40.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.332+0000 7f116f7fe640 1 -- 172.21.15.98:0/3557818138 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1158003530 con 0x7f11700b0620 2024-09-19T08:44:40.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.332+0000 7f116f7fe640 1 -- 172.21.15.98:0/3557818138 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 543660806 0 0) 0x7f1158003530 con 0x7f11700b0620 2024-09-19T08:44:40.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.332+0000 7f116f7fe640 1 -- 172.21.15.98:0/3557818138 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1170158db0 msgr2=0x7f117015f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:40.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.332+0000 7f116f7fe640 1 --2- 172.21.15.98:0/3557818138 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1170158db0 0x7f117015f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:40.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.332+0000 7f116f7fe640 1 -- 172.21.15.98:0/3557818138 >> v1:172.21.15.139:6789/0 conn(0x7f1170075380 legacy=0x7f117006c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:40.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.332+0000 7f116f7fe640 1 -- 172.21.15.98:0/3557818138 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11700ce9b0 con 0x7f11700b0620 2024-09-19T08:44:40.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.332+0000 7f116f7fe640 1 -- 172.21.15.98:0/3557818138 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f11640016a0 con 0x7f11700b0620 2024-09-19T08:44:40.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.332+0000 7f116f7fe640 1 -- 172.21.15.98:0/3557818138 >> v1:172.21.15.98:6789/0 conn(0x7f11700b0620 legacy=0x7f1170074c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:40.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.332+0000 7f116f7fe640 1 --2- 172.21.15.98:0/3557818138 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11580039f0 0x7f1158003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:40.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.332+0000 7f116f7fe640 1 --2- 172.21.15.98:0/3557818138 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1158004810 0x7f1158004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:40.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.332+0000 7f116f7fe640 1 --2- 172.21.15.98:0/3557818138 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1170158db0 0x7f11580051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:40.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.332+0000 7f116f7fe640 1 -- 172.21.15.98:0/3557818138 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1170075ad0 con 0x7f1158004810 2024-09-19T08:44:40.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.332+0000 7f116f7fe640 1 -- 172.21.15.98:0/3557818138 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f115800ba80 con 0x7f11580039f0 2024-09-19T08:44:40.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.332+0000 7f116f7fe640 1 -- 172.21.15.98:0/3557818138 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f115800bc80 con 0x7f1170158db0 2024-09-19T08:44:40.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.332+0000 7f116ffff640 1 --2- 172.21.15.98:0/3557818138 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1158004810 0x7f1158004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:40.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.332+0000 7f1174976640 1 --2- 172.21.15.98:0/3557818138 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11580039f0 0x7f1158003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:40.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.332+0000 7f116ffff640 1 -- 172.21.15.98:0/3557818138 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1170158db0 msgr2=0x7f11580051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:40.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.332+0000 7f1175177640 1 --2- 172.21.15.98:0/3557818138 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1170158db0 0x7f11580051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:40.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.332+0000 7f116ffff640 1 --2- 172.21.15.98:0/3557818138 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1170158db0 0x7f11580051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:40.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.332+0000 7f116ffff640 1 -- 172.21.15.98:0/3557818138 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11580039f0 msgr2=0x7f1158003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:40.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.332+0000 7f116ffff640 1 --2- 172.21.15.98:0/3557818138 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11580039f0 0x7f1158003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:40.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.332+0000 7f116ffff640 1 -- 172.21.15.98:0/3557818138 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1158003530 con 0x7f1158004810 2024-09-19T08:44:40.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.332+0000 7f1174976640 1 --2- 172.21.15.98:0/3557818138 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11580039f0 0x7f1158003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:40.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.336+0000 7f116ffff640 1 --2- 172.21.15.98:0/3557818138 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1158004810 0x7f1158004c90 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f11700ce8e0 tx=0x7f116401f370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a92e358b8cd542c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:40.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.336+0000 7f116f7fe640 1 -- 172.21.15.98:0/3557818138 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164007d20 con 0x7f1158004810 2024-09-19T08:44:40.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.336+0000 7f116f7fe640 1 -- 172.21.15.98:0/3557818138 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1164005ae0 con 0x7f1158004810 2024-09-19T08:44:40.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.336+0000 7f116f7fe640 1 -- 172.21.15.98:0/3557818138 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164007070 con 0x7f1158004810 2024-09-19T08:44:40.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.336+0000 7f1176bda640 1 -- 172.21.15.98:0/3557818138 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1158004810 msgr2=0x7f1158004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:40.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.336+0000 7f1176bda640 1 --2- 172.21.15.98:0/3557818138 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1158004810 0x7f1158004c90 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f11700ce8e0 tx=0x7f116401f370 comp rx=0 tx=0).stop 2024-09-19T08:44:40.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.336+0000 7f1175177640 1 -- 172.21.15.98:0/3557818138 reap_dead start 2024-09-19T08:44:40.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.336+0000 7f1176bda640 1 -- 172.21.15.98:0/3557818138 shutdown_connections 2024-09-19T08:44:40.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.336+0000 7f1176bda640 1 -- 172.21.15.98:0/3557818138 >> 172.21.15.98:0/3557818138 conn(0x7f11700b5f50 msgr2=0x7f11700b6650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:40.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.336+0000 7f1176bda640 1 -- 172.21.15.98:0/3557818138 shutdown_connections 2024-09-19T08:44:40.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.336+0000 7f1176bda640 1 -- 172.21.15.98:0/3557818138 wait complete. 2024-09-19T08:44:40.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.340+0000 7f1176bda640 1 Processor -- start 2024-09-19T08:44:40.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.340+0000 7f1176bda640 1 -- start start 2024-09-19T08:44:40.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.340+0000 7f1176bda640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1158004810 0x7f11700cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:40.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.340+0000 7f1176bda640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11700d0d70 0x7f11700b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:40.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.340+0000 7f1176bda640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11700b3370 0x7f11701f31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:40.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.340+0000 7f1176bda640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f117015f770 con 0x7f11700d0d70 2024-09-19T08:44:40.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.340+0000 7f1176bda640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f117006bfa0 con 0x7f11700b3370 2024-09-19T08:44:40.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.340+0000 7f1176bda640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f117015f8f0 con 0x7f1158004810 2024-09-19T08:44:40.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.340+0000 7f116ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11700d0d70 0x7f11700b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:40.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.340+0000 7f116ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11700d0d70 0x7f11700b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40508/0 (socket says 172.21.15.98:40508) 2024-09-19T08:44:40.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.340+0000 7f116ffff640 1 -- 172.21.15.98:0/2449727709 learned_addr learned my addr 172.21.15.98:0/2449727709 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:40.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.340+0000 7f1175177640 1 --2- 172.21.15.98:0/2449727709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11700b3370 0x7f11701f31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:40.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.340+0000 7f1174976640 1 --2- 172.21.15.98:0/2449727709 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1158004810 0x7f11700cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:40.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.340+0000 7f116ffff640 1 -- 172.21.15.98:0/2449727709 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1158004810 msgr2=0x7f11700cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:40.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.340+0000 7f116ffff640 1 --2- 172.21.15.98:0/2449727709 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1158004810 0x7f11700cfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:40.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.340+0000 7f116ffff640 1 -- 172.21.15.98:0/2449727709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11700b3370 msgr2=0x7f11701f31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:40.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.340+0000 7f116ffff640 1 --2- 172.21.15.98:0/2449727709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11700b3370 0x7f11701f31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:40.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.340+0000 7f116ffff640 1 -- 172.21.15.98:0/2449727709 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11700d1330 con 0x7f11700d0d70 2024-09-19T08:44:40.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.340+0000 7f1174976640 1 --2- 172.21.15.98:0/2449727709 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1158004810 0x7f11700cfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:40.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.340+0000 7f1175177640 1 --2- 172.21.15.98:0/2449727709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11700b3370 0x7f11701f31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:40.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.340+0000 7f116ffff640 1 --2- 172.21.15.98:0/2449727709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11700d0d70 0x7f11700b2e30 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f11640019a0 tx=0x7f1158005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:40.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.340+0000 7f116d7fa640 1 -- 172.21.15.98:0/2449727709 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164005c90 con 0x7f11700d0d70 2024-09-19T08:44:40.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.340+0000 7f116d7fa640 1 -- 172.21.15.98:0/2449727709 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1164005e30 con 0x7f11700d0d70 2024-09-19T08:44:40.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.340+0000 7f1176bda640 1 -- 172.21.15.98:0/2449727709 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11700d1610 con 0x7f11700d0d70 2024-09-19T08:44:40.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.340+0000 7f116d7fa640 1 -- 172.21.15.98:0/2449727709 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11640049f0 con 0x7f11700d0d70 2024-09-19T08:44:40.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.340+0000 7f1176bda640 1 -- 172.21.15.98:0/2449727709 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11700d1bd0 con 0x7f11700d0d70 2024-09-19T08:44:40.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.340+0000 7f1176bda640 1 -- 172.21.15.98:0/2449727709 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1170074c70 con 0x7f11700d0d70 2024-09-19T08:44:40.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.340+0000 7f116d7fa640 1 -- 172.21.15.98:0/2449727709 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 87700+0+0 (secure 0 0 0) 0x7f1164004b90 con 0x7f11700d0d70 2024-09-19T08:44:40.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.340+0000 7f116d7fa640 1 --2- 172.21.15.98:0/2449727709 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f115c067bc0 0x7f115c06a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:40.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.344+0000 7f116d7fa640 1 -- 172.21.15.98:0/2449727709 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5767+0+0 (secure 0 0 0) 0x7f11641232c0 con 0x7f11700d0d70 2024-09-19T08:44:40.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.344+0000 7f1174976640 1 --2- 172.21.15.98:0/2449727709 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f115c067bc0 0x7f115c06a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:40.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.344+0000 7f1174976640 1 --2- 172.21.15.98:0/2449727709 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f115c067bc0 0x7f115c06a080 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1168009ac0 tx=0x7f1168008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:40.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.348+0000 7f116d7fa640 1 -- 172.21.15.98:0/2449727709 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f1164019080 con 0x7f11700d0d70 2024-09-19T08:44:40.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.540+0000 7f1176bda640 1 -- 172.21.15.98:0/2449727709 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol41500", "target": ["mon-mgr", ""]}) -- 0x7f11700008e0 con 0x7f115c067bc0 2024-09-19T08:44:40.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.608+0000 7f116d7fa640 1 -- 172.21.15.98:0/2449727709 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f11700008e0 con 0x7f115c067bc0 2024-09-19T08:44:40.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.612+0000 7f1176bda640 1 -- 172.21.15.98:0/2449727709 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f115c067bc0 msgr2=0x7f115c06a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:40.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.612+0000 7f1176bda640 1 --2- 172.21.15.98:0/2449727709 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f115c067bc0 0x7f115c06a080 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1168009ac0 tx=0x7f1168008040 comp rx=0 tx=0).stop 2024-09-19T08:44:40.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.612+0000 7f1176bda640 1 -- 172.21.15.98:0/2449727709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11700d0d70 msgr2=0x7f11700b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:40.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.612+0000 7f1176bda640 1 --2- 172.21.15.98:0/2449727709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11700d0d70 0x7f11700b2e30 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f11640019a0 tx=0x7f1158005840 comp rx=0 tx=0).stop 2024-09-19T08:44:40.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.612+0000 7f1176bda640 1 -- 172.21.15.98:0/2449727709 shutdown_connections 2024-09-19T08:44:40.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.612+0000 7f1176bda640 1 --2- 172.21.15.98:0/2449727709 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f115c067bc0 0x7f115c06a080 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:40.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.612+0000 7f1176bda640 1 --2- 172.21.15.98:0/2449727709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11700b3370 0x7f11701f31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:40.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.612+0000 7f1176bda640 1 --2- 172.21.15.98:0/2449727709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11700d0d70 0x7f11700b2e30 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:40.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.612+0000 7f1176bda640 1 --2- 172.21.15.98:0/2449727709 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1158004810 0x7f11700cfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:40.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.612+0000 7f1176bda640 1 -- 172.21.15.98:0/2449727709 >> 172.21.15.98:0/2449727709 conn(0x7f11700b5f50 msgr2=0x7f11700cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:40.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.612+0000 7f1176bda640 1 -- 172.21.15.98:0/2449727709 shutdown_connections 2024-09-19T08:44:40.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.612+0000 7f1176bda640 1 -- 172.21.15.98:0/2449727709 wait complete. 2024-09-19T08:44:40.617 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:44:40.619+0000 7ff73cea1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:44:40.617 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:44:40.619+0000 7ff73cea1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:44:40.618 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:44:40.619+0000 7ff73cea1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:44:40.618 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:44:40.619+0000 7ff73cea1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:44:40.618 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:44:40.619+0000 7ff73cea1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:44:40.635 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol_clone38220 2024-09-19T08:44:40.636 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:44:40.635+0000 7ff73be9f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:44:40.636 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:44:40.635+0000 7ff73be9f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:44:40.636 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:44:40.635+0000 7ff73be9f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:44:40.636 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:44:40.635+0000 7ff73be9f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:44:40.636 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:44:40.635+0000 7ff73be9f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:44:40.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.756+0000 7f4041e7c640 1 Processor -- start 2024-09-19T08:44:40.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.756+0000 7f4041e7c640 1 -- start start 2024-09-19T08:44:40.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.756+0000 7f4041e7c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f403c15a090 0x7f403c15c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:40.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.756+0000 7f4041e7c640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f403c06dbf0 con 0x7f403c06bc90 2024-09-19T08:44:40.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.756+0000 7f4041e7c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f403c15d370 con 0x7f403c15cdf0 2024-09-19T08:44:40.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.756+0000 7f4041e7c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f403c15d650 con 0x7f403c15a090 2024-09-19T08:44:40.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.756+0000 7f403bfff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f403c06bc90 0x7f403c06e3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40648/0 (socket says 172.21.15.98:40648) 2024-09-19T08:44:40.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.756+0000 7f403bfff640 1 -- 172.21.15.98:0/1032850662 learned_addr learned my addr 172.21.15.98:0/1032850662 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:40.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.756+0000 7f403b7fe640 1 --2- 172.21.15.98:0/1032850662 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f403c15a090 0x7f403c15c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:40.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.756+0000 7f403a7fc640 1 -- 172.21.15.98:0/1032850662 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1328861315 0 0) 0x7f403c06dbf0 con 0x7f403c06bc90 2024-09-19T08:44:40.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.756+0000 7f403a7fc640 1 -- 172.21.15.98:0/1032850662 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4028003530 con 0x7f403c06bc90 2024-09-19T08:44:40.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.756+0000 7f403a7fc640 1 -- 172.21.15.98:0/1032850662 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 482825563 0 0) 0x7f4028003530 con 0x7f403c06bc90 2024-09-19T08:44:40.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.756+0000 7f403a7fc640 1 -- 172.21.15.98:0/1032850662 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f403c15a090 msgr2=0x7f403c15c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:40.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.756+0000 7f403a7fc640 1 --2- 172.21.15.98:0/1032850662 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f403c15a090 0x7f403c15c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:40.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.756+0000 7f403a7fc640 1 -- 172.21.15.98:0/1032850662 >> v1:172.21.15.139:6789/0 conn(0x7f403c15cdf0 legacy=0x7f403c0aee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:40.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.756+0000 7f403a7fc640 1 -- 172.21.15.98:0/1032850662 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403c06deb0 con 0x7f403c06bc90 2024-09-19T08:44:40.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.756+0000 7f403a7fc640 1 -- 172.21.15.98:0/1032850662 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f402c001a00 con 0x7f403c06bc90 2024-09-19T08:44:40.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.756+0000 7f403a7fc640 1 -- 172.21.15.98:0/1032850662 >> v1:172.21.15.98:6789/0 conn(0x7f403c06bc90 legacy=0x7f403c06e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:40.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.756+0000 7f403a7fc640 1 --2- 172.21.15.98:0/1032850662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40280039f0 0x7f4028003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:40.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.756+0000 7f403a7fc640 1 --2- 172.21.15.98:0/1032850662 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f403c15a090 0x7f4028004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:40.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.756+0000 7f403a7fc640 1 --2- 172.21.15.98:0/1032850662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4028004d50 0x7f40280051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:40.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.756+0000 7f403a7fc640 1 -- 172.21.15.98:0/1032850662 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f403c15d650 con 0x7f4028004d50 2024-09-19T08:44:40.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.756+0000 7f403a7fc640 1 -- 172.21.15.98:0/1032850662 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f402800ba80 con 0x7f40280039f0 2024-09-19T08:44:40.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.756+0000 7f403a7fc640 1 -- 172.21.15.98:0/1032850662 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f402800bc80 con 0x7f403c15a090 2024-09-19T08:44:40.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.756+0000 7f403bfff640 1 --2- 172.21.15.98:0/1032850662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4028004d50 0x7f40280051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:40.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.756+0000 7f403b7fe640 1 --2- 172.21.15.98:0/1032850662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40280039f0 0x7f4028003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:40.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.756+0000 7f403bfff640 1 -- 172.21.15.98:0/1032850662 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f403c15a090 msgr2=0x7f4028004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:40.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.756+0000 7f403bfff640 1 --2- 172.21.15.98:0/1032850662 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f403c15a090 0x7f4028004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:40.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.756+0000 7f403bfff640 1 -- 172.21.15.98:0/1032850662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40280039f0 msgr2=0x7f4028003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:40.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.756+0000 7f403bfff640 1 --2- 172.21.15.98:0/1032850662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40280039f0 0x7f4028003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:40.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.756+0000 7f403bfff640 1 -- 172.21.15.98:0/1032850662 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4028003530 con 0x7f4028004d50 2024-09-19T08:44:40.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.760+0000 7f403bfff640 1 --2- 172.21.15.98:0/1032850662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4028004d50 0x7f40280051d0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f402c0013e0 tx=0x7f402c01d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7385fad5741780f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:40.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.760+0000 7f403a7fc640 1 -- 172.21.15.98:0/1032850662 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c00a4a0 con 0x7f4028004d50 2024-09-19T08:44:40.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.760+0000 7f403a7fc640 1 -- 172.21.15.98:0/1032850662 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f402c0ad650 con 0x7f4028004d50 2024-09-19T08:44:40.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.760+0000 7f403a7fc640 1 -- 172.21.15.98:0/1032850662 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c0ada40 con 0x7f4028004d50 2024-09-19T08:44:40.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.760+0000 7f4041e7c640 1 -- 172.21.15.98:0/1032850662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4028004d50 msgr2=0x7f40280051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:40.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.760+0000 7f4041e7c640 1 --2- 172.21.15.98:0/1032850662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4028004d50 0x7f40280051d0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f402c0013e0 tx=0x7f402c01d3c0 comp rx=0 tx=0).stop 2024-09-19T08:44:40.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.760+0000 7f403bfff640 1 -- 172.21.15.98:0/1032850662 reap_dead start 2024-09-19T08:44:40.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.760+0000 7f4041e7c640 1 -- 172.21.15.98:0/1032850662 shutdown_connections 2024-09-19T08:44:40.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.760+0000 7f4041e7c640 1 -- 172.21.15.98:0/1032850662 >> 172.21.15.98:0/1032850662 conn(0x7f403c0b2a90 msgr2=0x7f403c0b2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:40.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.760+0000 7f4041e7c640 1 -- 172.21.15.98:0/1032850662 shutdown_connections 2024-09-19T08:44:40.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.760+0000 7f4041e7c640 1 -- 172.21.15.98:0/1032850662 wait complete. 2024-09-19T08:44:40.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.760+0000 7f4041e7c640 1 Processor -- start 2024-09-19T08:44:40.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.760+0000 7f4041e7c640 1 -- start start 2024-09-19T08:44:40.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.764+0000 7f4041e7c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f403c155a00 0x7f403c09ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:40.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.764+0000 7f4041e7c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f403c155de0 0x7f403c0ca110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:40.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.764+0000 7f4041e7c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f403c09e3e0 0x7f403c0d0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:40.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.764+0000 7f4041e7c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f403c0af6d0 con 0x7f403c155a00 2024-09-19T08:44:40.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.764+0000 7f4041e7c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f403c0af550 con 0x7f403c155de0 2024-09-19T08:44:40.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.764+0000 7f4041e7c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f403c06eae0 con 0x7f403c09e3e0 2024-09-19T08:44:40.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.764+0000 7f403b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f403c155a00 0x7f403c09ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:40.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.764+0000 7f403b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f403c155a00 0x7f403c09ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40534/0 (socket says 172.21.15.98:40534) 2024-09-19T08:44:40.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.764+0000 7f403b7fe640 1 -- 172.21.15.98:0/2694594703 learned_addr learned my addr 172.21.15.98:0/2694594703 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:40.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.764+0000 7f403affd640 1 --2- 172.21.15.98:0/2694594703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f403c155de0 0x7f403c0ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:40.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.764+0000 7f403b7fe640 1 -- 172.21.15.98:0/2694594703 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f403c09e3e0 msgr2=0x7f403c0d0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:40.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.764+0000 7f403bfff640 1 --2- 172.21.15.98:0/2694594703 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f403c09e3e0 0x7f403c0d0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:40.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.764+0000 7f403b7fe640 1 --2- 172.21.15.98:0/2694594703 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f403c09e3e0 0x7f403c0d0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:40.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.764+0000 7f403b7fe640 1 -- 172.21.15.98:0/2694594703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f403c155de0 msgr2=0x7f403c0ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:40.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.764+0000 7f403b7fe640 1 --2- 172.21.15.98:0/2694594703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f403c155de0 0x7f403c0ca110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:40.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.764+0000 7f403b7fe640 1 -- 172.21.15.98:0/2694594703 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403c0ca650 con 0x7f403c155a00 2024-09-19T08:44:40.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.764+0000 7f403affd640 1 --2- 172.21.15.98:0/2694594703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f403c155de0 0x7f403c0ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:40.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.764+0000 7f403b7fe640 1 --2- 172.21.15.98:0/2694594703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f403c155a00 0x7f403c09ba40 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f4034011690 tx=0x7f4034011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:40.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.764+0000 7f4033fff640 1 -- 172.21.15.98:0/2694594703 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403400f960 con 0x7f403c155a00 2024-09-19T08:44:40.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.764+0000 7f4033fff640 1 -- 172.21.15.98:0/2694594703 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f403400fb00 con 0x7f403c155a00 2024-09-19T08:44:40.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.764+0000 7f4041e7c640 1 -- 172.21.15.98:0/2694594703 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f403c0b4220 con 0x7f403c155a00 2024-09-19T08:44:40.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.764+0000 7f4033fff640 1 -- 172.21.15.98:0/2694594703 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4034015460 con 0x7f403c155a00 2024-09-19T08:44:40.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.764+0000 7f4041e7c640 1 -- 172.21.15.98:0/2694594703 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f403c0b47e0 con 0x7f403c155a00 2024-09-19T08:44:40.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.764+0000 7f4041e7c640 1 -- 172.21.15.98:0/2694594703 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f403c15c8b0 con 0x7f403c155a00 2024-09-19T08:44:40.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.764+0000 7f4033fff640 1 -- 172.21.15.98:0/2694594703 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 87700+0+0 (secure 0 0 0) 0x7f4034015600 con 0x7f403c155a00 2024-09-19T08:44:40.769 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.764+0000 7f4033fff640 1 --2- 172.21.15.98:0/2694594703 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f401c067bc0 0x7f401c06a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:40.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.764+0000 7f4033fff640 1 -- 172.21.15.98:0/2694594703 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5823+0+0 (secure 0 0 0) 0x7f4034086bc0 con 0x7f403c155a00 2024-09-19T08:44:40.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.764+0000 7f403affd640 1 --2- 172.21.15.98:0/2694594703 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f401c067bc0 0x7f401c06a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:40.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.768+0000 7f403affd640 1 --2- 172.21.15.98:0/2694594703 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f401c067bc0 0x7f401c06a080 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f403c0b0be0 tx=0x7f4024009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:40.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.768+0000 7f4033fff640 1 -- 172.21.15.98:0/2694594703 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f4034019030 con 0x7f403c155a00 2024-09-19T08:44:40.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.956+0000 7f4041e7c640 1 -- 172.21.15.98:0/2694594703 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol_clone38220", "target": ["mon-mgr", ""]}) -- 0x7f403c09bf80 con 0x7f401c067bc0 2024-09-19T08:44:40.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:40.996+0000 7f4033fff640 1 -- 172.21.15.98:0/2694594703 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f403c09bf80 con 0x7f401c067bc0 2024-09-19T08:44:41.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.000+0000 7f4041e7c640 1 -- 172.21.15.98:0/2694594703 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f401c067bc0 msgr2=0x7f401c06a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:41.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.000+0000 7f4041e7c640 1 --2- 172.21.15.98:0/2694594703 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f401c067bc0 0x7f401c06a080 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f403c0b0be0 tx=0x7f4024009040 comp rx=0 tx=0).stop 2024-09-19T08:44:41.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.000+0000 7f4041e7c640 1 -- 172.21.15.98:0/2694594703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f403c155a00 msgr2=0x7f403c09ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:41.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.000+0000 7f4041e7c640 1 --2- 172.21.15.98:0/2694594703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f403c155a00 0x7f403c09ba40 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f4034011690 tx=0x7f4034011b50 comp rx=0 tx=0).stop 2024-09-19T08:44:41.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.000+0000 7f4041e7c640 1 -- 172.21.15.98:0/2694594703 shutdown_connections 2024-09-19T08:44:41.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.000+0000 7f4041e7c640 1 --2- 172.21.15.98:0/2694594703 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f401c067bc0 0x7f401c06a080 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:41.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.000+0000 7f4041e7c640 1 --2- 172.21.15.98:0/2694594703 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f403c09e3e0 0x7f403c0d0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:41.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.000+0000 7f4041e7c640 1 --2- 172.21.15.98:0/2694594703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f403c155de0 0x7f403c0ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:41.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.000+0000 7f4041e7c640 1 --2- 172.21.15.98:0/2694594703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f403c155a00 0x7f403c09ba40 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:41.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.000+0000 7f4041e7c640 1 -- 172.21.15.98:0/2694594703 >> 172.21.15.98:0/2694594703 conn(0x7f403c0b2a90 msgr2=0x7f403c06c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:41.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.000+0000 7f4041e7c640 1 -- 172.21.15.98:0/2694594703 shutdown_connections 2024-09-19T08:44:41.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.000+0000 7f4041e7c640 1 -- 172.21.15.98:0/2694594703 wait complete. 2024-09-19T08:44:41.020 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol_clone83561 2024-09-19T08:44:41.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.164+0000 7fb7233aa640 1 Processor -- start 2024-09-19T08:44:41.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.164+0000 7fb7233aa640 1 -- start start 2024-09-19T08:44:41.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.164+0000 7fb7233aa640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb71c06c3a0 0x7fb71c0713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:41.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.164+0000 7fb7233aa640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb71c0718f0 0x7fb71c0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:41.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.164+0000 7fb7233aa640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb71c073530 con 0x7fb71c145b70 2024-09-19T08:44:41.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.164+0000 7fb7233aa640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fb71c073810 con 0x7fb71c06c3a0 2024-09-19T08:44:41.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.164+0000 7fb7233aa640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb71c073af0 con 0x7fb71c0718f0 2024-09-19T08:44:41.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.164+0000 7fb721146640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fb71c145b70 0x7fb71c06bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40658/0 (socket says 172.21.15.98:40658) 2024-09-19T08:44:41.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.164+0000 7fb721146640 1 -- 172.21.15.98:0/641594144 learned_addr learned my addr 172.21.15.98:0/641594144 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:41.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.164+0000 7fb720945640 1 --2- 172.21.15.98:0/641594144 >> v2:172.21.15.98:3300/0 conn(0x7fb71c06c3a0 0x7fb71c0713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:41.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.164+0000 7fb721947640 1 --2- 172.21.15.98:0/641594144 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb71c0718f0 0x7fb71c0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:41.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.164+0000 7fb713fff640 1 -- 172.21.15.98:0/641594144 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3705646668 0 0) 0x7fb71c073530 con 0x7fb71c145b70 2024-09-19T08:44:41.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.164+0000 7fb713fff640 1 -- 172.21.15.98:0/641594144 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb708003530 con 0x7fb71c145b70 2024-09-19T08:44:41.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.164+0000 7fb720945640 1 -- 172.21.15.98:0/641594144 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb71c0718f0 msgr2=0x7fb71c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:41.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.164+0000 7fb720945640 1 --2- 172.21.15.98:0/641594144 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb71c0718f0 0x7fb71c0b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:41.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.164+0000 7fb720945640 1 -- 172.21.15.98:0/641594144 >> v1:172.21.15.98:6789/0 conn(0x7fb71c145b70 legacy=0x7fb71c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:41.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.164+0000 7fb720945640 1 -- 172.21.15.98:0/641594144 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb71c073dd0 con 0x7fb71c06c3a0 2024-09-19T08:44:41.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.168+0000 7fb720945640 1 --2- 172.21.15.98:0/641594144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb71c06c3a0 0x7fb71c0713b0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fb70c0097c0 tx=0x7fb70c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c119a007ee720dd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:41.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.168+0000 7fb713fff640 1 -- 172.21.15.98:0/641594144 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70c0b0070 con 0x7fb71c06c3a0 2024-09-19T08:44:41.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.168+0000 7fb713fff640 1 -- 172.21.15.98:0/641594144 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb70c0ab600 con 0x7fb71c06c3a0 2024-09-19T08:44:41.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.168+0000 7fb713fff640 1 -- 172.21.15.98:0/641594144 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70c0ab9d0 con 0x7fb71c06c3a0 2024-09-19T08:44:41.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.168+0000 7fb7233aa640 1 -- 172.21.15.98:0/641594144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb71c06c3a0 msgr2=0x7fb71c0713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:41.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.168+0000 7fb7233aa640 1 --2- 172.21.15.98:0/641594144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb71c06c3a0 0x7fb71c0713b0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fb70c0097c0 tx=0x7fb70c01c410 comp rx=0 tx=0).stop 2024-09-19T08:44:41.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.168+0000 7fb7233aa640 1 -- 172.21.15.98:0/641594144 shutdown_connections 2024-09-19T08:44:41.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.168+0000 7fb7233aa640 1 --2- 172.21.15.98:0/641594144 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb71c0718f0 0x7fb71c0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:41.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.168+0000 7fb7233aa640 1 --2- 172.21.15.98:0/641594144 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb71c06c3a0 0x7fb71c0713b0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:41.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.168+0000 7fb7233aa640 1 -- 172.21.15.98:0/641594144 >> 172.21.15.98:0/641594144 conn(0x7fb71c0b2010 msgr2=0x7fb71c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:41.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.168+0000 7fb7233aa640 1 -- 172.21.15.98:0/641594144 shutdown_connections 2024-09-19T08:44:41.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.168+0000 7fb7233aa640 1 -- 172.21.15.98:0/641594144 wait complete. 2024-09-19T08:44:41.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.168+0000 7fb7233aa640 1 Processor -- start 2024-09-19T08:44:41.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.168+0000 7fb7233aa640 1 -- start start 2024-09-19T08:44:41.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.168+0000 7fb7233aa640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb71c06c3a0 0x7fb71c15b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:41.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.168+0000 7fb7233aa640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb71c0718f0 0x7fb71c0cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:41.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.168+0000 7fb7233aa640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb71c145b70 0x7fb71c165420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:41.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.168+0000 7fb7233aa640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c0b44f0 con 0x7fb71c145b70 2024-09-19T08:44:41.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.168+0000 7fb7233aa640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c0b4370 con 0x7fb71c0718f0 2024-09-19T08:44:41.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.168+0000 7fb7233aa640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb71c0b4670 con 0x7fb71c06c3a0 2024-09-19T08:44:41.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.168+0000 7fb721947640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb71c145b70 0x7fb71c165420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:41.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.168+0000 7fb721947640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb71c145b70 0x7fb71c165420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40552/0 (socket says 172.21.15.98:40552) 2024-09-19T08:44:41.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.168+0000 7fb721947640 1 -- 172.21.15.98:0/156116389 learned_addr learned my addr 172.21.15.98:0/156116389 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:41.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.168+0000 7fb721146640 1 --2- 172.21.15.98:0/156116389 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb71c06c3a0 0x7fb71c15b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:41.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.168+0000 7fb720945640 1 --2- 172.21.15.98:0/156116389 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb71c0718f0 0x7fb71c0cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:41.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.172+0000 7fb721947640 1 -- 172.21.15.98:0/156116389 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb71c06c3a0 msgr2=0x7fb71c15b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:41.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.172+0000 7fb721947640 1 --2- 172.21.15.98:0/156116389 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb71c06c3a0 0x7fb71c15b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:41.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.172+0000 7fb721947640 1 -- 172.21.15.98:0/156116389 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb71c0718f0 msgr2=0x7fb71c0cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:41.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.172+0000 7fb721947640 1 --2- 172.21.15.98:0/156116389 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb71c0718f0 0x7fb71c0cd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:41.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.172+0000 7fb721947640 1 -- 172.21.15.98:0/156116389 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb71c15bab0 con 0x7fb71c145b70 2024-09-19T08:44:41.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.172+0000 7fb721146640 1 --2- 172.21.15.98:0/156116389 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb71c06c3a0 0x7fb71c15b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:44:41.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.172+0000 7fb721947640 1 --2- 172.21.15.98:0/156116389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb71c145b70 0x7fb71c165420 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fb714007e40 tx=0x7fb71400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:41.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.172+0000 7fb711ffb640 1 -- 172.21.15.98:0/156116389 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb714014620 con 0x7fb71c145b70 2024-09-19T08:44:41.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.172+0000 7fb711ffb640 1 -- 172.21.15.98:0/156116389 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb714019070 con 0x7fb71c145b70 2024-09-19T08:44:41.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.172+0000 7fb7233aa640 1 -- 172.21.15.98:0/156116389 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb71c15c210 con 0x7fb71c145b70 2024-09-19T08:44:41.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.172+0000 7fb711ffb640 1 -- 172.21.15.98:0/156116389 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb714014a20 con 0x7fb71c145b70 2024-09-19T08:44:41.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.172+0000 7fb7233aa640 1 -- 172.21.15.98:0/156116389 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb71c0cce00 con 0x7fb71c145b70 2024-09-19T08:44:41.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.172+0000 7fb7233aa640 1 -- 172.21.15.98:0/156116389 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb71c0713b0 con 0x7fb71c145b70 2024-09-19T08:44:41.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.172+0000 7fb711ffb640 1 -- 172.21.15.98:0/156116389 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 87700+0+0 (secure 0 0 0) 0x7fb714014bc0 con 0x7fb71c145b70 2024-09-19T08:44:41.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.172+0000 7fb711ffb640 1 --2- 172.21.15.98:0/156116389 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb700067b90 0x7fb70006a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:41.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.172+0000 7fb711ffb640 1 -- 172.21.15.98:0/156116389 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5823+0+0 (secure 0 0 0) 0x7fb7140869f0 con 0x7fb71c145b70 2024-09-19T08:44:41.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.172+0000 7fb721146640 1 --2- 172.21.15.98:0/156116389 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb700067b90 0x7fb70006a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:41.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.172+0000 7fb721146640 1 --2- 172.21.15.98:0/156116389 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb700067b90 0x7fb70006a050 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb718005890 tx=0x7fb718009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:41.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.176+0000 7fb711ffb640 1 -- 172.21.15.98:0/156116389 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb714057170 con 0x7fb71c145b70 2024-09-19T08:44:41.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.364+0000 7fb7233aa640 1 -- 172.21.15.98:0/156116389 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol_clone83561", "target": ["mon-mgr", ""]}) -- 0x7fb71c0b1da0 con 0x7fb700067b90 2024-09-19T08:44:41.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.400+0000 7fb711ffb640 1 -- 172.21.15.98:0/156116389 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb71c0b1da0 con 0x7fb700067b90 2024-09-19T08:44:41.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.400+0000 7fb7233aa640 1 -- 172.21.15.98:0/156116389 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb700067b90 msgr2=0x7fb70006a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:41.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.400+0000 7fb7233aa640 1 --2- 172.21.15.98:0/156116389 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb700067b90 0x7fb70006a050 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb718005890 tx=0x7fb718009040 comp rx=0 tx=0).stop 2024-09-19T08:44:41.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.400+0000 7fb7233aa640 1 -- 172.21.15.98:0/156116389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb71c145b70 msgr2=0x7fb71c165420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:41.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.400+0000 7fb7233aa640 1 --2- 172.21.15.98:0/156116389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb71c145b70 0x7fb71c165420 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fb714007e40 tx=0x7fb71400e4b0 comp rx=0 tx=0).stop 2024-09-19T08:44:41.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.400+0000 7fb7233aa640 1 -- 172.21.15.98:0/156116389 shutdown_connections 2024-09-19T08:44:41.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.400+0000 7fb7233aa640 1 --2- 172.21.15.98:0/156116389 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb700067b90 0x7fb70006a050 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:41.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.400+0000 7fb7233aa640 1 --2- 172.21.15.98:0/156116389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb71c145b70 0x7fb71c165420 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:41.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.400+0000 7fb7233aa640 1 --2- 172.21.15.98:0/156116389 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb71c0718f0 0x7fb71c0cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:41.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.400+0000 7fb7233aa640 1 --2- 172.21.15.98:0/156116389 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb71c06c3a0 0x7fb71c15b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:41.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.400+0000 7fb7233aa640 1 -- 172.21.15.98:0/156116389 >> 172.21.15.98:0/156116389 conn(0x7fb71c0b2010 msgr2=0x7fb71c0cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:41.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.404+0000 7fb7233aa640 1 -- 172.21.15.98:0/156116389 shutdown_connections 2024-09-19T08:44:41.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.404+0000 7fb7233aa640 1 -- 172.21.15.98:0/156116389 wait complete. 2024-09-19T08:44:41.422 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-19T08:44:41.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T08:44:41.475 INFO:teuthology.orchestra.run.smithi098.stdout:2 2024-09-19T08:44:41.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:44:41.482 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T08:44:41.482 DEBUG:teuthology.orchestra.run.smithi098:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:44:41.606 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:44:41.685 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T08:44:41.686 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:44:41.686 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:44:41.686 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:44:41.693 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T08:44:41.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.864+0000 7f7a7e784640 1 Processor -- start 2024-09-19T08:44:41.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.864+0000 7f7a7e784640 1 -- start start 2024-09-19T08:44:41.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.864+0000 7f7a7e784640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7a78158e40 0x7f7a7806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:41.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.864+0000 7f7a7e784640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a7815fcf0 con 0x7f7a780ce8b0 2024-09-19T08:44:41.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.864+0000 7f7a7e784640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a7815ef30 con 0x7f7a7806c1d0 2024-09-19T08:44:41.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.864+0000 7f7a7e784640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f7a7806c640 con 0x7f7a78158e40 2024-09-19T08:44:41.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.864+0000 7f7a77fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7a78158e40 0x7f7a7806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:41.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.864+0000 7f7a7cd21640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f7a780ce8b0 0x7f7a780b3560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40662/0 (socket says 172.21.15.98:40662) 2024-09-19T08:44:41.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.864+0000 7f7a7cd21640 1 -- 172.21.15.98:0/1820785584 learned_addr learned my addr 172.21.15.98:0/1820785584 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:41.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.864+0000 7f7a76ffd640 1 -- 172.21.15.98:0/1820785584 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 752426988 0 0) 0x7f7a7815fcf0 con 0x7f7a780ce8b0 2024-09-19T08:44:41.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.864+0000 7f7a76ffd640 1 -- 172.21.15.98:0/1820785584 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a60003530 con 0x7f7a780ce8b0 2024-09-19T08:44:41.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.864+0000 7f7a77fff640 1 -- 172.21.15.98:0/1820785584 >> v1:172.21.15.139:6789/0 conn(0x7f7a7806c1d0 legacy=0x7f7a780ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:41.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.864+0000 7f7a77fff640 1 -- 172.21.15.98:0/1820785584 >> v1:172.21.15.98:6789/0 conn(0x7f7a780ce8b0 legacy=0x7f7a780b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:41.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.864+0000 7f7a77fff640 1 -- 172.21.15.98:0/1820785584 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a7815f210 con 0x7f7a78158e40 2024-09-19T08:44:41.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.868+0000 7f7a77fff640 1 --2- 172.21.15.98:0/1820785584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a78158e40 0x7f7a7806bc90 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f7a680097c0 tx=0x7f7a6801ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff8c6942b88dde1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:41.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.868+0000 7f7a76ffd640 1 -- 172.21.15.98:0/1820785584 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a680b5070 con 0x7f7a78158e40 2024-09-19T08:44:41.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.868+0000 7f7a76ffd640 1 -- 172.21.15.98:0/1820785584 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a680a8990 con 0x7f7a78158e40 2024-09-19T08:44:41.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.868+0000 7f7a76ffd640 1 -- 172.21.15.98:0/1820785584 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a680a8d60 con 0x7f7a78158e40 2024-09-19T08:44:41.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.868+0000 7f7a7e784640 1 -- 172.21.15.98:0/1820785584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a78158e40 msgr2=0x7f7a7806bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:41.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.868+0000 7f7a7e784640 1 --2- 172.21.15.98:0/1820785584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a78158e40 0x7f7a7806bc90 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f7a680097c0 tx=0x7f7a6801ec90 comp rx=0 tx=0).stop 2024-09-19T08:44:41.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.868+0000 7f7a7e784640 1 -- 172.21.15.98:0/1820785584 shutdown_connections 2024-09-19T08:44:41.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.868+0000 7f7a7e784640 1 --2- 172.21.15.98:0/1820785584 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a78158e40 0x7f7a7806bc90 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:41.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.868+0000 7f7a7e784640 1 -- 172.21.15.98:0/1820785584 >> 172.21.15.98:0/1820785584 conn(0x7f7a780b2010 msgr2=0x7f7a780b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:41.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.868+0000 7f7a7e784640 1 -- 172.21.15.98:0/1820785584 shutdown_connections 2024-09-19T08:44:41.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.868+0000 7f7a7e784640 1 -- 172.21.15.98:0/1820785584 wait complete. 2024-09-19T08:44:41.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.868+0000 7f7a7e784640 1 Processor -- start 2024-09-19T08:44:41.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.868+0000 7f7a7e784640 1 -- start start 2024-09-19T08:44:41.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.872+0000 7f7a7e784640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a7806c1d0 0x7f7a780735c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:41.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.872+0000 7f7a7e784640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a780ce8b0 0x7f7a781f4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:41.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.872+0000 7f7a7e784640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7a78158e40 0x7f7a7815b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:41.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.872+0000 7f7a7e784640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7a78159220 con 0x7f7a780ce8b0 2024-09-19T08:44:41.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.872+0000 7f7a7e784640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7815f7d0 con 0x7f7a7806c1d0 2024-09-19T08:44:41.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.872+0000 7f7a7e784640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7a780cee20 con 0x7f7a78158e40 2024-09-19T08:44:41.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.872+0000 7f7a777fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a780ce8b0 0x7f7a781f4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:41.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.872+0000 7f7a777fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a780ce8b0 0x7f7a781f4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40566/0 (socket says 172.21.15.98:40566) 2024-09-19T08:44:41.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.872+0000 7f7a777fe640 1 -- 172.21.15.98:0/575347863 learned_addr learned my addr 172.21.15.98:0/575347863 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:41.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.872+0000 7f7a77fff640 1 --2- 172.21.15.98:0/575347863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a7806c1d0 0x7f7a780735c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:41.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.872+0000 7f7a777fe640 1 -- 172.21.15.98:0/575347863 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7a78158e40 msgr2=0x7f7a7815b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:41.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.872+0000 7f7a777fe640 1 --2- 172.21.15.98:0/575347863 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7a78158e40 0x7f7a7815b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:41.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.872+0000 7f7a777fe640 1 -- 172.21.15.98:0/575347863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a7806c1d0 msgr2=0x7f7a780735c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:41.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.872+0000 7f7a777fe640 1 --2- 172.21.15.98:0/575347863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a7806c1d0 0x7f7a780735c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:41.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.872+0000 7f7a777fe640 1 -- 172.21.15.98:0/575347863 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a7815a530 con 0x7f7a780ce8b0 2024-09-19T08:44:41.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.872+0000 7f7a77fff640 1 --2- 172.21.15.98:0/575347863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a7806c1d0 0x7f7a780735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:41.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.872+0000 7f7a777fe640 1 --2- 172.21.15.98:0/575347863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a780ce8b0 0x7f7a781f4920 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c00c530 tx=0x7f7a6c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:41.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.872+0000 7f7a74ff9640 1 -- 172.21.15.98:0/575347863 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c016020 con 0x7f7a780ce8b0 2024-09-19T08:44:41.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.872+0000 7f7a74ff9640 1 -- 172.21.15.98:0/575347863 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a6c005220 con 0x7f7a780ce8b0 2024-09-19T08:44:41.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.872+0000 7f7a7e784640 1 -- 172.21.15.98:0/575347863 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a781594d0 con 0x7f7a780ce8b0 2024-09-19T08:44:41.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.872+0000 7f7a74ff9640 1 -- 172.21.15.98:0/575347863 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c007030 con 0x7f7a780ce8b0 2024-09-19T08:44:41.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.872+0000 7f7a7e784640 1 -- 172.21.15.98:0/575347863 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a78159a50 con 0x7f7a780ce8b0 2024-09-19T08:44:41.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.872+0000 7f7a7e784640 1 -- 172.21.15.98:0/575347863 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a7806bc90 con 0x7f7a780ce8b0 2024-09-19T08:44:41.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.872+0000 7f7a74ff9640 1 -- 172.21.15.98:0/575347863 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 87700+0+0 (secure 0 0 0) 0x7f7a6c0071d0 con 0x7f7a780ce8b0 2024-09-19T08:44:41.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.872+0000 7f7a74ff9640 1 --2- 172.21.15.98:0/575347863 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7a64067b90 0x7f7a6406a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:41.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.872+0000 7f7a74ff9640 1 -- 172.21.15.98:0/575347863 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5839+0+0 (secure 0 0 0) 0x7f7a6c085ac0 con 0x7f7a780ce8b0 2024-09-19T08:44:41.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.872+0000 7f7a77fff640 1 --2- 172.21.15.98:0/575347863 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7a64067b90 0x7f7a6406a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:41.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.876+0000 7f7a77fff640 1 --2- 172.21.15.98:0/575347863 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7a64067b90 0x7f7a6406a050 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7a680b2000 tx=0x7f7a680aa070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:41.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:41.876+0000 7f7a74ff9640 1 -- 172.21.15.98:0/575347863 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f7a6c020020 con 0x7f7a780ce8b0 2024-09-19T08:44:42.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.044+0000 7f7a7e784640 1 -- 172.21.15.98:0/575347863 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7a780b3930 con 0x7f7a780ce8b0 2024-09-19T08:44:42.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.048+0000 7f7a74ff9640 1 -- 172.21.15.98:0/575347863 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v106) ==== 80+0+0 (secure 0 0 0) 0x7f7a6c0074c0 con 0x7f7a780ce8b0 2024-09-19T08:44:42.048 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-19T08:44:42.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.048+0000 7f7a7e784640 1 -- 172.21.15.98:0/575347863 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7a64067b90 msgr2=0x7f7a6406a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:42.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.048+0000 7f7a7e784640 1 --2- 172.21.15.98:0/575347863 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7a64067b90 0x7f7a6406a050 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7a680b2000 tx=0x7f7a680aa070 comp rx=0 tx=0).stop 2024-09-19T08:44:42.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.048+0000 7f7a7e784640 1 -- 172.21.15.98:0/575347863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a780ce8b0 msgr2=0x7f7a781f4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:42.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.048+0000 7f7a7e784640 1 --2- 172.21.15.98:0/575347863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a780ce8b0 0x7f7a781f4920 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c00c530 tx=0x7f7a6c00c9f0 comp rx=0 tx=0).stop 2024-09-19T08:44:42.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.048+0000 7f7a7e784640 1 -- 172.21.15.98:0/575347863 shutdown_connections 2024-09-19T08:44:42.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.048+0000 7f7a7e784640 1 --2- 172.21.15.98:0/575347863 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7a64067b90 0x7f7a6406a050 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:42.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.048+0000 7f7a7e784640 1 --2- 172.21.15.98:0/575347863 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7a78158e40 0x7f7a7815b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:42.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.048+0000 7f7a7e784640 1 --2- 172.21.15.98:0/575347863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7a780ce8b0 0x7f7a781f4920 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:42.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.048+0000 7f7a7e784640 1 --2- 172.21.15.98:0/575347863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7a7806c1d0 0x7f7a780735c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:42.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.048+0000 7f7a7e784640 1 -- 172.21.15.98:0/575347863 >> 172.21.15.98:0/575347863 conn(0x7f7a780b2010 msgr2=0x7f7a780cc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:42.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.048+0000 7f7a7e784640 1 -- 172.21.15.98:0/575347863 shutdown_connections 2024-09-19T08:44:42.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.048+0000 7f7a7e784640 1 -- 172.21.15.98:0/575347863 wait complete. 2024-09-19T08:44:42.066 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi098.front.sepia.ceph.com... 2024-09-19T08:44:42.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:42.066 DEBUG:teuthology.orchestra.run.smithi098:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T08:44:42.077 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:ceph-fuse[46027]: fuse finished with error 0 and tester_r 0 2024-09-19T08:44:42.079 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:44:42.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:42.079 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T08:44:42.248 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:44:42.248 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:44:42.248 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:44:42.255 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T08:44:42.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.416+0000 7f437066f640 1 Processor -- start 2024-09-19T08:44:42.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.416+0000 7f437066f640 1 -- start start 2024-09-19T08:44:42.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.416+0000 7f437066f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f436815e650 0x7f436806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:42.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.416+0000 7f437066f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43680d2e00 0x7f43680707b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:42.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.416+0000 7f437066f640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f43680b73d0 0x7f4368070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:42.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.416+0000 7f437066f640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f4368162fc0 con 0x7f43680b73d0 2024-09-19T08:44:42.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.416+0000 7f437066f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f436806c1d0 con 0x7f436815e650 2024-09-19T08:44:42.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.416+0000 7f437066f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4368162440 con 0x7f43680d2e00 2024-09-19T08:44:42.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.416+0000 7f436ec0c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f43680b73d0 0x7f4368070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:42.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.416+0000 7f436ec0c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f43680b73d0 0x7f4368070cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40568/0 (socket says 172.21.15.98:40568) 2024-09-19T08:44:42.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.416+0000 7f436ec0c640 1 -- 172.21.15.98:0/85561323 learned_addr learned my addr 172.21.15.98:0/85561323 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:42.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.416+0000 7f436dc0a640 1 --2- 172.21.15.98:0/85561323 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43680d2e00 0x7f43680707b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:42.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.416+0000 7f436e40b640 1 --2- 172.21.15.98:0/85561323 >> v2:172.21.15.139:3300/0 conn(0x7f436815e650 0x7f436806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:42.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.416+0000 7f436ec0c640 1 -- 172.21.15.98:0/85561323 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43680d2e00 msgr2=0x7f43680707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:42.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.416+0000 7f436ec0c640 1 --2- 172.21.15.98:0/85561323 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43680d2e00 0x7f43680707b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:42.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.416+0000 7f436ec0c640 1 -- 172.21.15.98:0/85561323 >> v2:172.21.15.139:3300/0 conn(0x7f436815e650 msgr2=0x7f436806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:42.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.416+0000 7f436ec0c640 1 --2- 172.21.15.98:0/85561323 >> v2:172.21.15.139:3300/0 conn(0x7f436815e650 0x7f436806bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:42.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.416+0000 7f436ec0c640 1 -- 172.21.15.98:0/85561323 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4368162720 con 0x7f43680b73d0 2024-09-19T08:44:42.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.420+0000 7f436ec0c640 1 --2- 172.21.15.98:0/85561323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43680b73d0 0x7f4368070cf0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f4368072c80 tx=0x7f436001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978097ea18eb0e96 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:42.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.420+0000 7f436d409640 1 -- 172.21.15.98:0/85561323 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43600a5840 con 0x7f43680b73d0 2024-09-19T08:44:42.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.420+0000 7f436d409640 1 -- 172.21.15.98:0/85561323 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43600a59e0 con 0x7f43680b73d0 2024-09-19T08:44:42.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.420+0000 7f436d409640 1 -- 172.21.15.98:0/85561323 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43600a5db0 con 0x7f43680b73d0 2024-09-19T08:44:42.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.420+0000 7f437066f640 1 -- 172.21.15.98:0/85561323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43680b73d0 msgr2=0x7f4368070cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:42.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.420+0000 7f437066f640 1 --2- 172.21.15.98:0/85561323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43680b73d0 0x7f4368070cf0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f4368072c80 tx=0x7f436001c9c0 comp rx=0 tx=0).stop 2024-09-19T08:44:42.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.420+0000 7f437066f640 1 -- 172.21.15.98:0/85561323 shutdown_connections 2024-09-19T08:44:42.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.420+0000 7f437066f640 1 --2- 172.21.15.98:0/85561323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43680b73d0 0x7f4368070cf0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:42.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.420+0000 7f437066f640 1 --2- 172.21.15.98:0/85561323 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43680d2e00 0x7f43680707b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:42.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.420+0000 7f437066f640 1 --2- 172.21.15.98:0/85561323 >> v2:172.21.15.139:3300/0 conn(0x7f436815e650 0x7f436806bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:42.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.420+0000 7f437066f640 1 -- 172.21.15.98:0/85561323 >> 172.21.15.98:0/85561323 conn(0x7f43680d2310 msgr2=0x7f43680d26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:42.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.420+0000 7f437066f640 1 -- 172.21.15.98:0/85561323 shutdown_connections 2024-09-19T08:44:42.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.420+0000 7f437066f640 1 -- 172.21.15.98:0/85561323 wait complete. 2024-09-19T08:44:42.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.420+0000 7f437066f640 1 Processor -- start 2024-09-19T08:44:42.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.420+0000 7f437066f640 1 -- start start 2024-09-19T08:44:42.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.424+0000 7f437066f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43680b73d0 0x7f4368149ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:42.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.424+0000 7f437066f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43680d2e00 0x7f4368157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:42.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.424+0000 7f437066f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f436815e650 0x7f4368150f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:42.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.424+0000 7f437066f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4368073ca0 con 0x7f43680d2e00 2024-09-19T08:44:42.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.424+0000 7f437066f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4368073b20 con 0x7f436815e650 2024-09-19T08:44:42.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.424+0000 7f437066f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4368073e20 con 0x7f43680b73d0 2024-09-19T08:44:42.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.424+0000 7f436dc0a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43680d2e00 0x7f4368157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:42.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.424+0000 7f436dc0a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43680d2e00 0x7f4368157600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40584/0 (socket says 172.21.15.98:40584) 2024-09-19T08:44:42.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.424+0000 7f436dc0a640 1 -- 172.21.15.98:0/595374777 learned_addr learned my addr 172.21.15.98:0/595374777 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:42.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.424+0000 7f436e40b640 1 --2- 172.21.15.98:0/595374777 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43680b73d0 0x7f4368149ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:42.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.424+0000 7f436ec0c640 1 --2- 172.21.15.98:0/595374777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f436815e650 0x7f4368150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:42.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.424+0000 7f436dc0a640 1 -- 172.21.15.98:0/595374777 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43680b73d0 msgr2=0x7f4368149ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:42.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.424+0000 7f436dc0a640 1 --2- 172.21.15.98:0/595374777 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43680b73d0 0x7f4368149ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:42.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.424+0000 7f436dc0a640 1 -- 172.21.15.98:0/595374777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f436815e650 msgr2=0x7f4368150f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:42.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.424+0000 7f436dc0a640 1 --2- 172.21.15.98:0/595374777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f436815e650 0x7f4368150f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:42.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.424+0000 7f436dc0a640 1 -- 172.21.15.98:0/595374777 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f436814d890 con 0x7f43680d2e00 2024-09-19T08:44:42.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.424+0000 7f436e40b640 1 --2- 172.21.15.98:0/595374777 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43680b73d0 0x7f4368149ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:42.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.424+0000 7f436dc0a640 1 --2- 172.21.15.98:0/595374777 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43680d2e00 0x7f4368157600 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f435800c7f0 tx=0x7f435800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:42.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.424+0000 7f435effd640 1 -- 172.21.15.98:0/595374777 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4358015070 con 0x7f43680d2e00 2024-09-19T08:44:42.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.424+0000 7f435effd640 1 -- 172.21.15.98:0/595374777 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4358004220 con 0x7f43680d2e00 2024-09-19T08:44:42.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.424+0000 7f437066f640 1 -- 172.21.15.98:0/595374777 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f436814db30 con 0x7f43680d2e00 2024-09-19T08:44:42.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.424+0000 7f435effd640 1 -- 172.21.15.98:0/595374777 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4358003620 con 0x7f43680d2e00 2024-09-19T08:44:42.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.424+0000 7f437066f640 1 -- 172.21.15.98:0/595374777 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4368157fc0 con 0x7f43680d2e00 2024-09-19T08:44:42.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.424+0000 7f437066f640 1 -- 172.21.15.98:0/595374777 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f436806bc90 con 0x7f43680d2e00 2024-09-19T08:44:42.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.424+0000 7f435effd640 1 -- 172.21.15.98:0/595374777 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 18) ==== 87700+0+0 (secure 0 0 0) 0x7f43580039d0 con 0x7f43680d2e00 2024-09-19T08:44:42.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.424+0000 7f435effd640 1 --2- 172.21.15.98:0/595374777 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f434c067bc0 0x7f434c06a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:42.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.424+0000 7f435effd640 1 -- 172.21.15.98:0/595374777 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5839+0+0 (secure 0 0 0) 0x7f4358086230 con 0x7f43680d2e00 2024-09-19T08:44:42.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.424+0000 7f436e40b640 1 --2- 172.21.15.98:0/595374777 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f434c067bc0 0x7f434c06a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:42.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.428+0000 7f436e40b640 1 --2- 172.21.15.98:0/595374777 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f434c067bc0 0x7f434c06a080 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4364009840 tx=0x7f4364008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:42.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.428+0000 7f435effd640 1 -- 172.21.15.98:0/595374777 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f4358056890 con 0x7f43680d2e00 2024-09-19T08:44:42.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.472+0000 7f435effd640 1 -- 172.21.15.98:0/595374777 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f4358052800 con 0x7f43680d2e00 2024-09-19T08:44:42.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.596+0000 7f437066f640 1 -- 172.21.15.98:0/595374777 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f436815ebd0 con 0x7f43680d2e00 2024-09-19T08:44:42.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.596+0000 7f435effd640 1 -- 172.21.15.98:0/595374777 <== mon.0 v2:172.21.15.98:3300/0 8 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v106) ==== 80+0+0 (secure 0 0 0) 0x7f4358072a40 con 0x7f43680d2e00 2024-09-19T08:44:42.597 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-19T08:44:42.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.596+0000 7f437066f640 1 -- 172.21.15.98:0/595374777 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f434c067bc0 msgr2=0x7f434c06a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:42.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.596+0000 7f437066f640 1 --2- 172.21.15.98:0/595374777 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f434c067bc0 0x7f434c06a080 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4364009840 tx=0x7f4364008040 comp rx=0 tx=0).stop 2024-09-19T08:44:42.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.596+0000 7f437066f640 1 -- 172.21.15.98:0/595374777 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43680d2e00 msgr2=0x7f4368157600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:42.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.596+0000 7f437066f640 1 --2- 172.21.15.98:0/595374777 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43680d2e00 0x7f4368157600 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f435800c7f0 tx=0x7f435800ccb0 comp rx=0 tx=0).stop 2024-09-19T08:44:42.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.596+0000 7f437066f640 1 -- 172.21.15.98:0/595374777 shutdown_connections 2024-09-19T08:44:42.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.596+0000 7f437066f640 1 --2- 172.21.15.98:0/595374777 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f434c067bc0 0x7f434c06a080 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:42.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.596+0000 7f437066f640 1 --2- 172.21.15.98:0/595374777 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f436815e650 0x7f4368150f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:42.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.596+0000 7f437066f640 1 --2- 172.21.15.98:0/595374777 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f43680d2e00 0x7f4368157600 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:42.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.596+0000 7f437066f640 1 --2- 172.21.15.98:0/595374777 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f43680b73d0 0x7f4368149ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:42.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.596+0000 7f437066f640 1 -- 172.21.15.98:0/595374777 >> 172.21.15.98:0/595374777 conn(0x7f43680d2310 msgr2=0x7f436815d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:42.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.596+0000 7f437066f640 1 -- 172.21.15.98:0/595374777 shutdown_connections 2024-09-19T08:44:42.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.596+0000 7f437066f640 1 -- 172.21.15.98:0/595374777 wait complete. 2024-09-19T08:44:42.614 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi139.front.sepia.ceph.com... 2024-09-19T08:44:42.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:42.615 DEBUG:teuthology.orchestra.run.smithi139:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-19T08:44:42.628 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[36162]: fuse finished with error 0 and tester_r 0 2024-09-19T08:44:42.630 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:44:42.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:42.630 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-19T08:44:42.717 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:44:42.718 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:44:42.718 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:44:42.723 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15846 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T08:44:42.724 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:44:42.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:42.724 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-19T08:44:42.772 INFO:teuthology.orchestra.run.smithi098.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-19T08:44:42.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:44:42.773 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:44:42.773 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:44:42.773 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:44:42.779 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T08:44:42.779 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:44:42.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:44:42.779 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-19T08:44:42.828 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-19T08:44:42.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:44:42.829 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:44:42.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.952+0000 7fbefb4e4640 1 Processor -- start 2024-09-19T08:44:42.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.952+0000 7fbefb4e4640 1 -- start start 2024-09-19T08:44:42.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.952+0000 7fbefb4e4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbef4158fd0 0x7fbef40b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:42.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.952+0000 7fbefb4e4640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbef406c3a0 0x7fbef40ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:42.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.952+0000 7fbefb4e4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbef40ccc60 con 0x7fbef40b4360 2024-09-19T08:44:42.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.952+0000 7fbefb4e4640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fbef41593b0 con 0x7fbef406c3a0 2024-09-19T08:44:42.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.952+0000 7fbefb4e4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fbef40ccf40 con 0x7fbef4158fd0 2024-09-19T08:44:42.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.952+0000 7fbef9a81640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbef406c3a0 0x7fbef40ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:42.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.952+0000 7fbef9a81640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbef406c3a0 0x7fbef40ce840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40600/0 (socket says 172.21.15.98:40600) 2024-09-19T08:44:42.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.952+0000 7fbef9a81640 1 -- 172.21.15.98:0/2935089830 learned_addr learned my addr 172.21.15.98:0/2935089830 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:42.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.952+0000 7fbef9280640 1 --2- 172.21.15.98:0/2935089830 >> v2:172.21.15.139:3300/0 conn(0x7fbef4158fd0 0x7fbef40b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:42.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.952+0000 7fbef9a81640 1 -- 172.21.15.98:0/2935089830 >> v2:172.21.15.139:3300/0 conn(0x7fbef4158fd0 msgr2=0x7fbef40b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:42.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.952+0000 7fbef9a81640 1 --2- 172.21.15.98:0/2935089830 >> v2:172.21.15.139:3300/0 conn(0x7fbef4158fd0 0x7fbef40b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:42.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.952+0000 7fbef9a81640 1 -- 172.21.15.98:0/2935089830 >> v1:172.21.15.139:6789/0 conn(0x7fbef40b4360 legacy=0x7fbef406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:42.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.952+0000 7fbef9a81640 1 -- 172.21.15.98:0/2935089830 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef40cd220 con 0x7fbef406c3a0 2024-09-19T08:44:42.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.952+0000 7fbef9a81640 1 --2- 172.21.15.98:0/2935089830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbef406c3a0 0x7fbef40ce840 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fbef406e310 tx=0x7fbeec01eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14b9ee1b22cb8db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:42.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.956+0000 7fbeebfff640 1 -- 172.21.15.98:0/2935089830 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec00b2d0 con 0x7fbef406c3a0 2024-09-19T08:44:42.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.956+0000 7fbeebfff640 1 -- 172.21.15.98:0/2935089830 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbeec00b470 con 0x7fbef406c3a0 2024-09-19T08:44:42.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.956+0000 7fbeebfff640 1 -- 172.21.15.98:0/2935089830 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec0a7be0 con 0x7fbef406c3a0 2024-09-19T08:44:42.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.956+0000 7fbefb4e4640 1 -- 172.21.15.98:0/2935089830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbef406c3a0 msgr2=0x7fbef40ce840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:42.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.956+0000 7fbefb4e4640 1 --2- 172.21.15.98:0/2935089830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbef406c3a0 0x7fbef40ce840 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fbef406e310 tx=0x7fbeec01eaf0 comp rx=0 tx=0).stop 2024-09-19T08:44:42.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.956+0000 7fbefb4e4640 1 -- 172.21.15.98:0/2935089830 shutdown_connections 2024-09-19T08:44:42.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.956+0000 7fbefb4e4640 1 --2- 172.21.15.98:0/2935089830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbef406c3a0 0x7fbef40ce840 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:42.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.956+0000 7fbefb4e4640 1 --2- 172.21.15.98:0/2935089830 >> v2:172.21.15.139:3300/0 conn(0x7fbef4158fd0 0x7fbef40b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:42.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.956+0000 7fbefb4e4640 1 -- 172.21.15.98:0/2935089830 >> 172.21.15.98:0/2935089830 conn(0x7fbef40b2010 msgr2=0x7fbef40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:42.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.956+0000 7fbefb4e4640 1 -- 172.21.15.98:0/2935089830 shutdown_connections 2024-09-19T08:44:42.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.956+0000 7fbefb4e4640 1 -- 172.21.15.98:0/2935089830 wait complete. 2024-09-19T08:44:42.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.956+0000 7fbefb4e4640 1 Processor -- start 2024-09-19T08:44:42.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.956+0000 7fbefb4e4640 1 -- start start 2024-09-19T08:44:42.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.956+0000 7fbefb4e4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbef406c3a0 0x7fbef40cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:42.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.956+0000 7fbefb4e4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbef40b4360 0x7fbef40b0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:42.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.956+0000 7fbefb4e4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbef4158fd0 0x7fbef4151ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:42.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.956+0000 7fbefb4e4640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbef40cf710 con 0x7fbef40b4360 2024-09-19T08:44:42.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.956+0000 7fbefb4e4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbef40cf590 con 0x7fbef406c3a0 2024-09-19T08:44:42.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.956+0000 7fbefb4e4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbef40cf890 con 0x7fbef4158fd0 2024-09-19T08:44:42.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.956+0000 7fbef8a7f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbef40b4360 0x7fbef40b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:42.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.956+0000 7fbef8a7f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbef40b4360 0x7fbef40b0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40612/0 (socket says 172.21.15.98:40612) 2024-09-19T08:44:42.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.956+0000 7fbef8a7f640 1 -- 172.21.15.98:0/3390511932 learned_addr learned my addr 172.21.15.98:0/3390511932 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:42.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.956+0000 7fbef9a81640 1 --2- 172.21.15.98:0/3390511932 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbef4158fd0 0x7fbef4151ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:42.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.956+0000 7fbef9280640 1 --2- 172.21.15.98:0/3390511932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbef406c3a0 0x7fbef40cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:42.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.956+0000 7fbef8a7f640 1 -- 172.21.15.98:0/3390511932 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbef4158fd0 msgr2=0x7fbef4151ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:42.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.956+0000 7fbef8a7f640 1 --2- 172.21.15.98:0/3390511932 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbef4158fd0 0x7fbef4151ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:42.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.956+0000 7fbef8a7f640 1 -- 172.21.15.98:0/3390511932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbef406c3a0 msgr2=0x7fbef40cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:42.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.956+0000 7fbef8a7f640 1 --2- 172.21.15.98:0/3390511932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbef406c3a0 0x7fbef40cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:42.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.956+0000 7fbef8a7f640 1 -- 172.21.15.98:0/3390511932 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef4157470 con 0x7fbef40b4360 2024-09-19T08:44:42.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.956+0000 7fbef8a7f640 1 --2- 172.21.15.98:0/3390511932 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbef40b4360 0x7fbef40b0630 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fbee400c450 tx=0x7fbee400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:42.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.956+0000 7fbee9ffb640 1 -- 172.21.15.98:0/3390511932 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee4012830 con 0x7fbef40b4360 2024-09-19T08:44:42.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.956+0000 7fbee9ffb640 1 -- 172.21.15.98:0/3390511932 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbee4017070 con 0x7fbef40b4360 2024-09-19T08:44:42.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.956+0000 7fbefb4e4640 1 -- 172.21.15.98:0/3390511932 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbef4157750 con 0x7fbef40b4360 2024-09-19T08:44:42.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.956+0000 7fbee9ffb640 1 -- 172.21.15.98:0/3390511932 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee4012c30 con 0x7fbef40b4360 2024-09-19T08:44:42.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.956+0000 7fbefb4e4640 1 -- 172.21.15.98:0/3390511932 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbef4157a30 con 0x7fbef40b4360 2024-09-19T08:44:42.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.960+0000 7fbefb4e4640 1 -- 172.21.15.98:0/3390511932 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbef415bd60 con 0x7fbef40b4360 2024-09-19T08:44:42.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.960+0000 7fbee9ffb640 1 -- 172.21.15.98:0/3390511932 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7fbee401c460 con 0x7fbef40b4360 2024-09-19T08:44:42.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.960+0000 7fbee9ffb640 1 --2- 172.21.15.98:0/3390511932 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbed8067ce0 0x7fbed806a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:42.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.960+0000 7fbee9ffb640 1 -- 172.21.15.98:0/3390511932 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5839+0+0 (secure 0 0 0) 0x7fbee4086250 con 0x7fbef40b4360 2024-09-19T08:44:42.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.960+0000 7fbef9280640 1 --2- 172.21.15.98:0/3390511932 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbed8067ce0 0x7fbed806a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:42.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.960+0000 7fbef9280640 1 --2- 172.21.15.98:0/3390511932 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbed8067ce0 0x7fbed806a1a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbef000b930 tx=0x7fbef0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:42.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:42.964+0000 7fbee9ffb640 1 -- 172.21.15.98:0/3390511932 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fbee4056730 con 0x7fbef40b4360 2024-09-19T08:44:43.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.144+0000 7fbefb4e4640 1 -- 172.21.15.98:0/3390511932 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbef40cfcf0 con 0x7fbef40b4360 2024-09-19T08:44:43.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.144+0000 7fbee9ffb640 1 -- 172.21.15.98:0/3390511932 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5225 (secure 0 0 0) 0x7fbee4056030 con 0x7fbef40b4360 2024-09-19T08:44:43.147 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:44:43.147 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":61,"btime":"2024-09-19T08:44:10:283464+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6004,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3739962222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3739962222},{"type":"v1","addr":"172.21.15.139:6833","nonce":3739962222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6007,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/421473653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":421473653},{"type":"v1","addr":"172.21.15.139:6835","nonce":421473653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3126941916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3126941916},{"type":"v1","addr":"172.21.15.98:6833","nonce":3126941916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:43:54.489407+0000","modified":"2024-09-19T08:43:56.577758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6996},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6996":{"gid":6996,"name":"c","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/273091170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":273091170},{"type":"v1","addr":"172.21.15.98:6835","nonce":273091170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6996,"qdb_cluster":[6996]},"id":8}]} 2024-09-19T08:44:43.147 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 61 2024-09-19T08:44:43.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.148+0000 7fbefb4e4640 1 -- 172.21.15.98:0/3390511932 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbed8067ce0 msgr2=0x7fbed806a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:43.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.148+0000 7fbefb4e4640 1 --2- 172.21.15.98:0/3390511932 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbed8067ce0 0x7fbed806a1a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbef000b930 tx=0x7fbef0008040 comp rx=0 tx=0).stop 2024-09-19T08:44:43.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.148+0000 7fbefb4e4640 1 -- 172.21.15.98:0/3390511932 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbef40b4360 msgr2=0x7fbef40b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:43.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.148+0000 7fbefb4e4640 1 --2- 172.21.15.98:0/3390511932 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbef40b4360 0x7fbef40b0630 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fbee400c450 tx=0x7fbee400c910 comp rx=0 tx=0).stop 2024-09-19T08:44:43.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.148+0000 7fbefb4e4640 1 -- 172.21.15.98:0/3390511932 shutdown_connections 2024-09-19T08:44:43.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.148+0000 7fbefb4e4640 1 --2- 172.21.15.98:0/3390511932 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbed8067ce0 0x7fbed806a1a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:43.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.148+0000 7fbefb4e4640 1 --2- 172.21.15.98:0/3390511932 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbef4158fd0 0x7fbef4151ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:43.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.148+0000 7fbefb4e4640 1 --2- 172.21.15.98:0/3390511932 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbef40b4360 0x7fbef40b0630 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:43.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.148+0000 7fbefb4e4640 1 --2- 172.21.15.98:0/3390511932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbef406c3a0 0x7fbef40cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:43.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.148+0000 7fbefb4e4640 1 -- 172.21.15.98:0/3390511932 >> 172.21.15.98:0/3390511932 conn(0x7fbef40b2010 msgr2=0x7fbef406bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:43.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.148+0000 7fbefb4e4640 1 -- 172.21.15.98:0/3390511932 shutdown_connections 2024-09-19T08:44:43.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.148+0000 7fbefb4e4640 1 -- 172.21.15.98:0/3390511932 wait complete. 2024-09-19T08:44:43.165 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:44:43.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.280+0000 7f6f96473640 1 Processor -- start 2024-09-19T08:44:43.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.280+0000 7f6f96473640 1 -- start start 2024-09-19T08:44:43.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.280+0000 7f6f96473640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f90158fd0 0x7f6f900b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:43.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.280+0000 7f6f96473640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6f900b4360 0x7f6f9006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:43.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.280+0000 7f6f96473640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f900ccc60 con 0x7f6f9006c1d0 2024-09-19T08:44:43.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.280+0000 7f6f96473640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f6f9006c620 con 0x7f6f900b4360 2024-09-19T08:44:43.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.280+0000 7f6f96473640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6f901593b0 con 0x7f6f90158fd0 2024-09-19T08:44:43.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.280+0000 7f6f8ffff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f90158fd0 0x7f6f900b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:43.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.280+0000 7f6f87fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6f900b4360 0x7f6f9006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:43.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.280+0000 7f6f8ffff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f90158fd0 0x7f6f900b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:36644/0 (socket says 172.21.15.98:36644) 2024-09-19T08:44:43.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.280+0000 7f6f8ffff640 1 -- 172.21.15.98:0/79986787 learned_addr learned my addr 172.21.15.98:0/79986787 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:43.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.280+0000 7f6f8ffff640 1 -- 172.21.15.98:0/79986787 >> v2:172.21.15.139:3300/0 conn(0x7f6f900b4360 msgr2=0x7f6f9006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:43.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.280+0000 7f6f8ffff640 1 --2- 172.21.15.98:0/79986787 >> v2:172.21.15.139:3300/0 conn(0x7f6f900b4360 0x7f6f9006bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:43.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.280+0000 7f6f8ffff640 1 -- 172.21.15.98:0/79986787 >> v1:172.21.15.139:6789/0 conn(0x7f6f9006c1d0 legacy=0x7f6f9015efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:43.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.280+0000 7f6f8ffff640 1 -- 172.21.15.98:0/79986787 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f900ccf40 con 0x7f6f90158fd0 2024-09-19T08:44:43.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.280+0000 7f6f87fff640 1 --2- 172.21.15.98:0/79986787 >> v2:172.21.15.139:3300/0 conn(0x7f6f900b4360 0x7f6f9006bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:44:43.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.284+0000 7f6f8ffff640 1 --2- 172.21.15.98:0/79986787 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f90158fd0 0x7f6f900b3e20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6f80009870 tx=0x7f6f8001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8e95b247d1b64a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:43.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.284+0000 7f6f8f7fe640 1 -- 172.21.15.98:0/79986787 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f800aa600 con 0x7f6f90158fd0 2024-09-19T08:44:43.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.284+0000 7f6f8f7fe640 1 -- 172.21.15.98:0/79986787 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f800af070 con 0x7f6f90158fd0 2024-09-19T08:44:43.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.284+0000 7f6f8f7fe640 1 -- 172.21.15.98:0/79986787 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f800aa9c0 con 0x7f6f90158fd0 2024-09-19T08:44:43.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.284+0000 7f6f96473640 1 -- 172.21.15.98:0/79986787 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f90158fd0 msgr2=0x7f6f900b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:43.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.284+0000 7f6f96473640 1 --2- 172.21.15.98:0/79986787 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f90158fd0 0x7f6f900b3e20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6f80009870 tx=0x7f6f8001c4a0 comp rx=0 tx=0).stop 2024-09-19T08:44:43.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.284+0000 7f6f96473640 1 -- 172.21.15.98:0/79986787 shutdown_connections 2024-09-19T08:44:43.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.284+0000 7f6f96473640 1 --2- 172.21.15.98:0/79986787 >> v2:172.21.15.139:3300/0 conn(0x7f6f900b4360 0x7f6f9006bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:43.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.284+0000 7f6f96473640 1 --2- 172.21.15.98:0/79986787 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f90158fd0 0x7f6f900b3e20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:43.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.284+0000 7f6f96473640 1 -- 172.21.15.98:0/79986787 >> 172.21.15.98:0/79986787 conn(0x7f6f900b2010 msgr2=0x7f6f900b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:43.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.284+0000 7f6f96473640 1 -- 172.21.15.98:0/79986787 shutdown_connections 2024-09-19T08:44:43.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.284+0000 7f6f96473640 1 -- 172.21.15.98:0/79986787 wait complete. 2024-09-19T08:44:43.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.284+0000 7f6f96473640 1 Processor -- start 2024-09-19T08:44:43.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.284+0000 7f6f96473640 1 -- start start 2024-09-19T08:44:43.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.288+0000 7f6f96473640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f9006c1d0 0x7f6f90157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:43.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.288+0000 7f6f96473640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f900b4360 0x7f6f901643e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:43.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.288+0000 7f6f96473640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f90158fd0 0x7f6f90164920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:43.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.288+0000 7f6f96473640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9015c920 con 0x7f6f900b4360 2024-09-19T08:44:43.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.288+0000 7f6f96473640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9015c7a0 con 0x7f6f9006c1d0 2024-09-19T08:44:43.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.288+0000 7f6f96473640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6f90168110 con 0x7f6f90158fd0 2024-09-19T08:44:43.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.288+0000 7f6f87fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f900b4360 0x7f6f901643e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:43.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.288+0000 7f6f87fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f900b4360 0x7f6f901643e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40614/0 (socket says 172.21.15.98:40614) 2024-09-19T08:44:43.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.288+0000 7f6f87fff640 1 -- 172.21.15.98:0/396111709 learned_addr learned my addr 172.21.15.98:0/396111709 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:43.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.288+0000 7f6f8ffff640 1 --2- 172.21.15.98:0/396111709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f9006c1d0 0x7f6f90157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:43.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.288+0000 7f6f87fff640 1 -- 172.21.15.98:0/396111709 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f90158fd0 msgr2=0x7f6f90164920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:43.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.288+0000 7f6f94a10640 1 --2- 172.21.15.98:0/396111709 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f90158fd0 0x7f6f90164920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:43.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.288+0000 7f6f87fff640 1 --2- 172.21.15.98:0/396111709 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f90158fd0 0x7f6f90164920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:43.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.288+0000 7f6f87fff640 1 -- 172.21.15.98:0/396111709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f9006c1d0 msgr2=0x7f6f90157f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:43.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.288+0000 7f6f87fff640 1 --2- 172.21.15.98:0/396111709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f9006c1d0 0x7f6f90157f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:43.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.288+0000 7f6f87fff640 1 -- 172.21.15.98:0/396111709 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f90159730 con 0x7f6f900b4360 2024-09-19T08:44:43.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.288+0000 7f6f87fff640 1 --2- 172.21.15.98:0/396111709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f900b4360 0x7f6f901643e0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f6f7800ea00 tx=0x7f6f7800eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:43.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.288+0000 7f6f8d7fa640 1 -- 172.21.15.98:0/396111709 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f78017070 con 0x7f6f900b4360 2024-09-19T08:44:43.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.288+0000 7f6f96473640 1 -- 172.21.15.98:0/396111709 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f90161950 con 0x7f6f900b4360 2024-09-19T08:44:43.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.288+0000 7f6f8d7fa640 1 -- 172.21.15.98:0/396111709 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f7800ca60 con 0x7f6f900b4360 2024-09-19T08:44:43.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.288+0000 7f6f8d7fa640 1 -- 172.21.15.98:0/396111709 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7800ce50 con 0x7f6f900b4360 2024-09-19T08:44:43.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.288+0000 7f6f96473640 1 -- 172.21.15.98:0/396111709 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f90161f10 con 0x7f6f900b4360 2024-09-19T08:44:43.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.288+0000 7f6f96473640 1 -- 172.21.15.98:0/396111709 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f9006bc90 con 0x7f6f900b4360 2024-09-19T08:44:43.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.288+0000 7f6f8d7fa640 1 -- 172.21.15.98:0/396111709 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f6f7801a070 con 0x7f6f900b4360 2024-09-19T08:44:43.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.288+0000 7f6f8d7fa640 1 --2- 172.21.15.98:0/396111709 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6f70067ce0 0x7f6f7006a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:43.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.288+0000 7f6f8d7fa640 1 -- 172.21.15.98:0/396111709 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5839+0+0 (secure 0 0 0) 0x7f6f78086390 con 0x7f6f900b4360 2024-09-19T08:44:43.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.288+0000 7f6f8ffff640 1 --2- 172.21.15.98:0/396111709 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6f70067ce0 0x7f6f7006a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:43.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.292+0000 7f6f8ffff640 1 --2- 172.21.15.98:0/396111709 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6f70067ce0 0x7f6f7006a1a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6f800ae000 tx=0x7f6f80008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:43.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.292+0000 7f6f8d7fa640 1 -- 172.21.15.98:0/396111709 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6f78056880 con 0x7f6f900b4360 2024-09-19T08:44:43.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.488+0000 7f6f96473640 1 -- 172.21.15.98:0/396111709 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f900757a0 con 0x7f6f900b4360 2024-09-19T08:44:43.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.488+0000 7f6f8d7fa640 1 -- 172.21.15.98:0/396111709 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5225 (secure 0 0 0) 0x7f6f78056180 con 0x7f6f900b4360 2024-09-19T08:44:43.491 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:44:43.491 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":61,"btime":"2024-09-19T08:44:10:283464+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6004,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3739962222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3739962222},{"type":"v1","addr":"172.21.15.139:6833","nonce":3739962222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6007,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/421473653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":421473653},{"type":"v1","addr":"172.21.15.139:6835","nonce":421473653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3126941916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3126941916},{"type":"v1","addr":"172.21.15.98:6833","nonce":3126941916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:43:54.489407+0000","modified":"2024-09-19T08:43:56.577758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6996},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6996":{"gid":6996,"name":"c","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/273091170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":273091170},{"type":"v1","addr":"172.21.15.98:6835","nonce":273091170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6996,"qdb_cluster":[6996]},"id":8}]} 2024-09-19T08:44:43.492 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 61 2024-09-19T08:44:43.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.492+0000 7f6f96473640 1 -- 172.21.15.98:0/396111709 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6f70067ce0 msgr2=0x7f6f7006a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:43.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.492+0000 7f6f96473640 1 --2- 172.21.15.98:0/396111709 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6f70067ce0 0x7f6f7006a1a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6f800ae000 tx=0x7f6f80008040 comp rx=0 tx=0).stop 2024-09-19T08:44:43.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.492+0000 7f6f96473640 1 -- 172.21.15.98:0/396111709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f900b4360 msgr2=0x7f6f901643e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:43.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.492+0000 7f6f96473640 1 --2- 172.21.15.98:0/396111709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f900b4360 0x7f6f901643e0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f6f7800ea00 tx=0x7f6f7800eec0 comp rx=0 tx=0).stop 2024-09-19T08:44:43.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.492+0000 7f6f96473640 1 -- 172.21.15.98:0/396111709 shutdown_connections 2024-09-19T08:44:43.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.492+0000 7f6f96473640 1 --2- 172.21.15.98:0/396111709 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6f70067ce0 0x7f6f7006a1a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:43.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.492+0000 7f6f96473640 1 --2- 172.21.15.98:0/396111709 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f90158fd0 0x7f6f90164920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:43.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.492+0000 7f6f96473640 1 --2- 172.21.15.98:0/396111709 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f900b4360 0x7f6f901643e0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:43.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.492+0000 7f6f96473640 1 --2- 172.21.15.98:0/396111709 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f9006c1d0 0x7f6f90157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:43.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.492+0000 7f6f96473640 1 -- 172.21.15.98:0/396111709 >> 172.21.15.98:0/396111709 conn(0x7f6f900b2010 msgr2=0x7f6f900b1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:43.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.492+0000 7f6f96473640 1 -- 172.21.15.98:0/396111709 shutdown_connections 2024-09-19T08:44:43.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.492+0000 7f6f96473640 1 -- 172.21.15.98:0/396111709 wait complete. 2024-09-19T08:44:43.509 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:44:43.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.616+0000 7f5335046640 1 Processor -- start 2024-09-19T08:44:43.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.616+0000 7f5335046640 1 -- start start 2024-09-19T08:44:43.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.616+0000 7f5335046640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f533015c550 0x7f533006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:43.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.616+0000 7f5335046640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f533006c1d0 0x7f53300cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:43.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.616+0000 7f5335046640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f533006d310 con 0x7f53300cf900 2024-09-19T08:44:43.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.616+0000 7f5335046640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f533006c610 con 0x7f533015c550 2024-09-19T08:44:43.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.616+0000 7f5335046640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f533006d5f0 con 0x7f533006c1d0 2024-09-19T08:44:43.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.616+0000 7f5327fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f533006c1d0 0x7f53300cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:43.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.616+0000 7f532ed9d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f533015c550 0x7f533006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:43.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.616+0000 7f5327fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f533006c1d0 0x7f53300cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:36674/0 (socket says 172.21.15.98:36674) 2024-09-19T08:44:43.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.616+0000 7f532f59e640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f53300cf900 0x7f53300ce1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:44872/0 (socket says 172.21.15.98:44872) 2024-09-19T08:44:43.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.616+0000 7f5327fff640 1 -- 172.21.15.98:0/212282628 learned_addr learned my addr 172.21.15.98:0/212282628 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:43.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.616+0000 7f5327fff640 1 -- 172.21.15.98:0/212282628 >> v2:172.21.15.139:3300/0 conn(0x7f533015c550 msgr2=0x7f533006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:43.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.616+0000 7f5327fff640 1 --2- 172.21.15.98:0/212282628 >> v2:172.21.15.139:3300/0 conn(0x7f533015c550 0x7f533006bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:43.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.616+0000 7f5327fff640 1 -- 172.21.15.98:0/212282628 >> v1:172.21.15.139:6789/0 conn(0x7f53300cf900 legacy=0x7f53300ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:43.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.616+0000 7f5327fff640 1 -- 172.21.15.98:0/212282628 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f533006d8d0 con 0x7f533006c1d0 2024-09-19T08:44:43.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.616+0000 7f532ed9d640 1 --2- 172.21.15.98:0/212282628 >> v2:172.21.15.139:3300/0 conn(0x7f533015c550 0x7f533006bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:44:43.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.620+0000 7f5327fff640 1 --2- 172.21.15.98:0/212282628 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f533006c1d0 0x7f53300cf3c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5318009870 tx=0x7f531801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9521ef21f92fea58 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:43.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.620+0000 7f532e59c640 1 -- 172.21.15.98:0/212282628 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53180b2070 con 0x7f533006c1d0 2024-09-19T08:44:43.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.620+0000 7f532e59c640 1 -- 172.21.15.98:0/212282628 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53180a9070 con 0x7f533006c1d0 2024-09-19T08:44:43.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.620+0000 7f532e59c640 1 -- 172.21.15.98:0/212282628 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53180ad880 con 0x7f533006c1d0 2024-09-19T08:44:43.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.620+0000 7f5335046640 1 -- 172.21.15.98:0/212282628 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f533006c1d0 msgr2=0x7f53300cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:43.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.620+0000 7f5335046640 1 --2- 172.21.15.98:0/212282628 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f533006c1d0 0x7f53300cf3c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5318009870 tx=0x7f531801c560 comp rx=0 tx=0).stop 2024-09-19T08:44:43.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.620+0000 7f5335046640 1 -- 172.21.15.98:0/212282628 shutdown_connections 2024-09-19T08:44:43.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.620+0000 7f5335046640 1 --2- 172.21.15.98:0/212282628 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f533006c1d0 0x7f53300cf3c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:43.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.620+0000 7f5335046640 1 --2- 172.21.15.98:0/212282628 >> v2:172.21.15.139:3300/0 conn(0x7f533015c550 0x7f533006bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:43.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.620+0000 7f5335046640 1 -- 172.21.15.98:0/212282628 >> 172.21.15.98:0/212282628 conn(0x7f53300b2010 msgr2=0x7f53300b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:43.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.620+0000 7f5335046640 1 -- 172.21.15.98:0/212282628 shutdown_connections 2024-09-19T08:44:43.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.620+0000 7f5335046640 1 -- 172.21.15.98:0/212282628 wait complete. 2024-09-19T08:44:43.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.624+0000 7f5335046640 1 Processor -- start 2024-09-19T08:44:43.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.624+0000 7f5335046640 1 -- start start 2024-09-19T08:44:43.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.624+0000 7f5335046640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f533006c1d0 0x7f53301486c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:43.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.624+0000 7f5335046640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53300cf900 0x7f5330151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:43.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.624+0000 7f5335046640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f533015c550 0x7f5330149a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:43.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.624+0000 7f5335046640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5330169230 con 0x7f533015c550 2024-09-19T08:44:43.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.624+0000 7f5335046640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f533015cb40 con 0x7f533006c1d0 2024-09-19T08:44:43.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.624+0000 7f5335046640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f53301693b0 con 0x7f53300cf900 2024-09-19T08:44:43.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.624+0000 7f532f59e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f533015c550 0x7f5330149a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:43.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.624+0000 7f532f59e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f533015c550 0x7f5330149a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40626/0 (socket says 172.21.15.98:40626) 2024-09-19T08:44:43.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.624+0000 7f532f59e640 1 -- 172.21.15.98:0/2179991705 learned_addr learned my addr 172.21.15.98:0/2179991705 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:43.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.624+0000 7f5327fff640 1 --2- 172.21.15.98:0/2179991705 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53300cf900 0x7f5330151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:43.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.624+0000 7f532ed9d640 1 --2- 172.21.15.98:0/2179991705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f533006c1d0 0x7f53301486c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:43.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.624+0000 7f532f59e640 1 -- 172.21.15.98:0/2179991705 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53300cf900 msgr2=0x7f5330151b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:43.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.624+0000 7f532f59e640 1 --2- 172.21.15.98:0/2179991705 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53300cf900 0x7f5330151b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:43.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.624+0000 7f532f59e640 1 -- 172.21.15.98:0/2179991705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f533006c1d0 msgr2=0x7f53301486c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:43.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.624+0000 7f532f59e640 1 --2- 172.21.15.98:0/2179991705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f533006c1d0 0x7f53301486c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:43.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.624+0000 7f532f59e640 1 -- 172.21.15.98:0/2179991705 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5330149fc0 con 0x7f533015c550 2024-09-19T08:44:43.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.624+0000 7f532f59e640 1 --2- 172.21.15.98:0/2179991705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f533015c550 0x7f5330149a80 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f532800c7d0 tx=0x7f532800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:43.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.624+0000 7f53277fe640 1 -- 172.21.15.98:0/2179991705 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5328017070 con 0x7f533015c550 2024-09-19T08:44:43.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.624+0000 7f53277fe640 1 -- 172.21.15.98:0/2179991705 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f532800a640 con 0x7f533015c550 2024-09-19T08:44:43.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.624+0000 7f5335046640 1 -- 172.21.15.98:0/2179991705 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5330152290 con 0x7f533015c550 2024-09-19T08:44:43.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.624+0000 7f53277fe640 1 -- 172.21.15.98:0/2179991705 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532800a9f0 con 0x7f533015c550 2024-09-19T08:44:43.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.624+0000 7f5335046640 1 -- 172.21.15.98:0/2179991705 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5330152850 con 0x7f533015c550 2024-09-19T08:44:43.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.624+0000 7f5335046640 1 -- 172.21.15.98:0/2179991705 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53300cf3c0 con 0x7f533015c550 2024-09-19T08:44:43.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.624+0000 7f53277fe640 1 -- 172.21.15.98:0/2179991705 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f532801a070 con 0x7f533015c550 2024-09-19T08:44:43.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.624+0000 7f53277fe640 1 --2- 172.21.15.98:0/2179991705 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5310067cb0 0x7f531006a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:43.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.624+0000 7f53277fe640 1 -- 172.21.15.98:0/2179991705 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5839+0+0 (secure 0 0 0) 0x7f5328085a20 con 0x7f533015c550 2024-09-19T08:44:43.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.628+0000 7f532ed9d640 1 --2- 172.21.15.98:0/2179991705 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5310067cb0 0x7f531006a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:43.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.628+0000 7f532ed9d640 1 --2- 172.21.15.98:0/2179991705 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5310067cb0 0x7f531006a170 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5320009a50 tx=0x7f5320008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:43.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.628+0000 7f53277fe640 1 -- 172.21.15.98:0/2179991705 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f5330149fc0 con 0x7f533015c550 2024-09-19T08:44:43.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.792+0000 7f5335046640 1 -- 172.21.15.98:0/2179991705 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5330073e70 con 0x7f533015c550 2024-09-19T08:44:43.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.792+0000 7f53277fe640 1 -- 172.21.15.98:0/2179991705 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+16915 (secure 0 0 0) 0x7f5328055820 con 0x7f533015c550 2024-09-19T08:44:43.796 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:44:43.796 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":106,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:44:41.439895+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:43:52.161395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-19T08:43:53.497026+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.3c","mappings":[{"from":0,"to":5}]},{"pgid":"17.3f","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:44:43.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.796+0000 7f5335046640 1 -- 172.21.15.98:0/2179991705 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5310067cb0 msgr2=0x7f531006a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:43.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.796+0000 7f5335046640 1 --2- 172.21.15.98:0/2179991705 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5310067cb0 0x7f531006a170 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5320009a50 tx=0x7f5320008040 comp rx=0 tx=0).stop 2024-09-19T08:44:43.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.796+0000 7f5335046640 1 -- 172.21.15.98:0/2179991705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f533015c550 msgr2=0x7f5330149a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:43.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.796+0000 7f5335046640 1 --2- 172.21.15.98:0/2179991705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f533015c550 0x7f5330149a80 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f532800c7d0 tx=0x7f532800cc90 comp rx=0 tx=0).stop 2024-09-19T08:44:43.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.796+0000 7f5335046640 1 -- 172.21.15.98:0/2179991705 shutdown_connections 2024-09-19T08:44:43.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.796+0000 7f5335046640 1 --2- 172.21.15.98:0/2179991705 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5310067cb0 0x7f531006a170 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:43.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.796+0000 7f5335046640 1 --2- 172.21.15.98:0/2179991705 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f533015c550 0x7f5330149a80 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:43.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.796+0000 7f5335046640 1 --2- 172.21.15.98:0/2179991705 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53300cf900 0x7f5330151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:43.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.796+0000 7f5335046640 1 --2- 172.21.15.98:0/2179991705 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f533006c1d0 0x7f53301486c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:43.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.796+0000 7f5335046640 1 -- 172.21.15.98:0/2179991705 >> 172.21.15.98:0/2179991705 conn(0x7f53300b2010 msgr2=0x7f53300d0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:43.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.796+0000 7f5335046640 1 -- 172.21.15.98:0/2179991705 shutdown_connections 2024-09-19T08:44:43.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.796+0000 7f5335046640 1 -- 172.21.15.98:0/2179991705 wait complete. 2024-09-19T08:44:43.814 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T08:44:43.814 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:44:43.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.932+0000 7f8e3ed5f640 1 Processor -- start 2024-09-19T08:44:43.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.932+0000 7f8e3ed5f640 1 -- start start 2024-09-19T08:44:43.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.936+0000 7f8e3ed5f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8e380b7520 0x7f8e380b7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:43.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.936+0000 7f8e3ed5f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8e380b6200 0x7f8e380b6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:43.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.936+0000 7f8e3ed5f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e380b99a0 con 0x7f8e38147770 2024-09-19T08:44:43.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.936+0000 7f8e3ed5f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f8e380b85a0 con 0x7f8e380b6200 2024-09-19T08:44:43.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.936+0000 7f8e3ed5f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8e380b7fe0 con 0x7f8e380b7520 2024-09-19T08:44:43.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.936+0000 7f8e37fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8e380b7520 0x7f8e380b7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:43.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.936+0000 7f8e3d2fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8e380b6200 0x7f8e380b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:43.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.936+0000 7f8e37fff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8e380b7520 0x7f8e380b7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:36688/0 (socket says 172.21.15.98:36688) 2024-09-19T08:44:43.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.936+0000 7f8e37fff640 1 -- 172.21.15.98:0/374897403 learned_addr learned my addr 172.21.15.98:0/374897403 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:43.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.936+0000 7f8e377fe640 1 -- 172.21.15.98:0/374897403 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 946809336 0 0) 0x7f8e380b99a0 con 0x7f8e38147770 2024-09-19T08:44:43.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.936+0000 7f8e377fe640 1 -- 172.21.15.98:0/374897403 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e24003530 con 0x7f8e38147770 2024-09-19T08:44:43.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.936+0000 7f8e37fff640 1 -- 172.21.15.98:0/374897403 >> v2:172.21.15.139:3300/0 conn(0x7f8e380b6200 msgr2=0x7f8e380b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:43.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.936+0000 7f8e37fff640 1 --2- 172.21.15.98:0/374897403 >> v2:172.21.15.139:3300/0 conn(0x7f8e380b6200 0x7f8e380b6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:43.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.936+0000 7f8e37fff640 1 -- 172.21.15.98:0/374897403 >> v1:172.21.15.139:6789/0 conn(0x7f8e38147770 legacy=0x7f8e380b9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:43.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.936+0000 7f8e37fff640 1 -- 172.21.15.98:0/374897403 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e380b82c0 con 0x7f8e380b7520 2024-09-19T08:44:43.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.940+0000 7f8e37fff640 1 --2- 172.21.15.98:0/374897403 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8e380b7520 0x7f8e380b7960 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8e380b8720 tx=0x7f8e2001c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51fe811befa400cc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:43.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.940+0000 7f8e377fe640 1 -- 172.21.15.98:0/374897403 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e200b1070 con 0x7f8e380b7520 2024-09-19T08:44:43.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.940+0000 7f8e377fe640 1 -- 172.21.15.98:0/374897403 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e200ac600 con 0x7f8e380b7520 2024-09-19T08:44:43.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.940+0000 7f8e377fe640 1 -- 172.21.15.98:0/374897403 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e200ac9b0 con 0x7f8e380b7520 2024-09-19T08:44:43.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.940+0000 7f8e3ed5f640 1 -- 172.21.15.98:0/374897403 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8e380b7520 msgr2=0x7f8e380b7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:43.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.940+0000 7f8e3ed5f640 1 --2- 172.21.15.98:0/374897403 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8e380b7520 0x7f8e380b7960 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8e380b8720 tx=0x7f8e2001c4b0 comp rx=0 tx=0).stop 2024-09-19T08:44:43.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.940+0000 7f8e3ed5f640 1 -- 172.21.15.98:0/374897403 shutdown_connections 2024-09-19T08:44:43.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.940+0000 7f8e3ed5f640 1 --2- 172.21.15.98:0/374897403 >> v2:172.21.15.139:3300/0 conn(0x7f8e380b6200 0x7f8e380b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:43.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.940+0000 7f8e3ed5f640 1 --2- 172.21.15.98:0/374897403 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8e380b7520 0x7f8e380b7960 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:43.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.940+0000 7f8e3ed5f640 1 -- 172.21.15.98:0/374897403 >> 172.21.15.98:0/374897403 conn(0x7f8e38074e40 msgr2=0x7f8e380722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:43.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.940+0000 7f8e3ed5f640 1 -- 172.21.15.98:0/374897403 shutdown_connections 2024-09-19T08:44:43.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.940+0000 7f8e3ed5f640 1 -- 172.21.15.98:0/374897403 wait complete. 2024-09-19T08:44:43.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.940+0000 7f8e3ed5f640 1 Processor -- start 2024-09-19T08:44:43.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.940+0000 7f8e3ed5f640 1 -- start start 2024-09-19T08:44:43.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.944+0000 7f8e3ed5f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e380b6200 0x7f8e38155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:43.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.944+0000 7f8e3ed5f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8e380b7520 0x7f8e3813f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:43.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.944+0000 7f8e3ed5f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e38147770 0x7f8e38157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:43.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.944+0000 7f8e3ed5f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8e38147cd0 con 0x7f8e380b6200 2024-09-19T08:44:43.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.944+0000 7f8e3ed5f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8e38147b50 con 0x7f8e38147770 2024-09-19T08:44:43.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.944+0000 7f8e3ed5f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8e3816b1f0 con 0x7f8e380b7520 2024-09-19T08:44:43.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.944+0000 7f8e3cafb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e380b6200 0x7f8e38155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:43.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.944+0000 7f8e3cafb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e380b6200 0x7f8e38155aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40632/0 (socket says 172.21.15.98:40632) 2024-09-19T08:44:43.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.944+0000 7f8e3cafb640 1 -- 172.21.15.98:0/3542953707 learned_addr learned my addr 172.21.15.98:0/3542953707 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:43.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.944+0000 7f8e3d2fc640 1 --2- 172.21.15.98:0/3542953707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e38147770 0x7f8e38157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:43.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.944+0000 7f8e37fff640 1 --2- 172.21.15.98:0/3542953707 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8e380b7520 0x7f8e3813f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:43.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.944+0000 7f8e3cafb640 1 -- 172.21.15.98:0/3542953707 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8e380b7520 msgr2=0x7f8e3813f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:43.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.944+0000 7f8e3cafb640 1 --2- 172.21.15.98:0/3542953707 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8e380b7520 0x7f8e3813f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:43.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.944+0000 7f8e3cafb640 1 -- 172.21.15.98:0/3542953707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e38147770 msgr2=0x7f8e38157780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:43.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.944+0000 7f8e3cafb640 1 --2- 172.21.15.98:0/3542953707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e38147770 0x7f8e38157780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:43.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.944+0000 7f8e3cafb640 1 -- 172.21.15.98:0/3542953707 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e38156080 con 0x7f8e380b6200 2024-09-19T08:44:43.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.944+0000 7f8e3cafb640 1 --2- 172.21.15.98:0/3542953707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e380b6200 0x7f8e38155aa0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000c4c0 tx=0x7f8e3000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:43.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.944+0000 7f8e357fa640 1 -- 172.21.15.98:0/3542953707 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e300052d0 con 0x7f8e380b6200 2024-09-19T08:44:43.946 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.944+0000 7f8e357fa640 1 -- 172.21.15.98:0/3542953707 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e30005470 con 0x7f8e380b6200 2024-09-19T08:44:43.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.944+0000 7f8e3ed5f640 1 -- 172.21.15.98:0/3542953707 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e381591c0 con 0x7f8e380b6200 2024-09-19T08:44:43.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.944+0000 7f8e357fa640 1 -- 172.21.15.98:0/3542953707 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e30012870 con 0x7f8e380b6200 2024-09-19T08:44:43.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.944+0000 7f8e3ed5f640 1 -- 172.21.15.98:0/3542953707 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e381411e0 con 0x7f8e380b6200 2024-09-19T08:44:43.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.944+0000 7f8e3ed5f640 1 -- 172.21.15.98:0/3542953707 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e380b6690 con 0x7f8e380b6200 2024-09-19T08:44:43.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.944+0000 7f8e357fa640 1 -- 172.21.15.98:0/3542953707 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f8e30012a10 con 0x7f8e380b6200 2024-09-19T08:44:43.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.944+0000 7f8e357fa640 1 --2- 172.21.15.98:0/3542953707 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8e18067cb0 0x7f8e1806a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:43.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.944+0000 7f8e357fa640 1 -- 172.21.15.98:0/3542953707 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5839+0+0 (secure 0 0 0) 0x7f8e30086110 con 0x7f8e380b6200 2024-09-19T08:44:43.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.944+0000 7f8e37fff640 1 --2- 172.21.15.98:0/3542953707 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8e18067cb0 0x7f8e1806a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:43.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.948+0000 7f8e37fff640 1 --2- 172.21.15.98:0/3542953707 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8e18067cb0 0x7f8e1806a170 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8e200ae000 tx=0x7f8e20008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:43.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:43.948+0000 7f8e357fa640 1 -- 172.21.15.98:0/3542953707 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f8e300565f0 con 0x7f8e380b6200 2024-09-19T08:44:44.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.140+0000 7f8e3ed5f640 1 -- 172.21.15.98:0/3542953707 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e38158300 con 0x7f8e380b6200 2024-09-19T08:44:44.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.140+0000 7f8e357fa640 1 -- 172.21.15.98:0/3542953707 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5225 (secure 0 0 0) 0x7f8e30055ef0 con 0x7f8e380b6200 2024-09-19T08:44:44.144 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:44:44.144 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":61,"btime":"2024-09-19T08:44:10:283464+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6004,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3739962222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3739962222},{"type":"v1","addr":"172.21.15.139:6833","nonce":3739962222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6007,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/421473653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":421473653},{"type":"v1","addr":"172.21.15.139:6835","nonce":421473653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3126941916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3126941916},{"type":"v1","addr":"172.21.15.98:6833","nonce":3126941916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:43:54.489407+0000","modified":"2024-09-19T08:43:56.577758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6996},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6996":{"gid":6996,"name":"c","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/273091170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":273091170},{"type":"v1","addr":"172.21.15.98:6835","nonce":273091170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6996,"qdb_cluster":[6996]},"id":8}]} 2024-09-19T08:44:44.145 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 61 2024-09-19T08:44:44.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.144+0000 7f8e3ed5f640 1 -- 172.21.15.98:0/3542953707 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8e18067cb0 msgr2=0x7f8e1806a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:44.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.144+0000 7f8e3ed5f640 1 --2- 172.21.15.98:0/3542953707 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8e18067cb0 0x7f8e1806a170 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8e200ae000 tx=0x7f8e20008040 comp rx=0 tx=0).stop 2024-09-19T08:44:44.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.144+0000 7f8e3ed5f640 1 -- 172.21.15.98:0/3542953707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e380b6200 msgr2=0x7f8e38155aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:44.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.144+0000 7f8e3ed5f640 1 --2- 172.21.15.98:0/3542953707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e380b6200 0x7f8e38155aa0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000c4c0 tx=0x7f8e3000c980 comp rx=0 tx=0).stop 2024-09-19T08:44:44.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.144+0000 7f8e3ed5f640 1 -- 172.21.15.98:0/3542953707 shutdown_connections 2024-09-19T08:44:44.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.144+0000 7f8e3ed5f640 1 --2- 172.21.15.98:0/3542953707 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8e18067cb0 0x7f8e1806a170 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.144+0000 7f8e3ed5f640 1 --2- 172.21.15.98:0/3542953707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8e38147770 0x7f8e38157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.144+0000 7f8e3ed5f640 1 --2- 172.21.15.98:0/3542953707 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8e380b7520 0x7f8e3813f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.144+0000 7f8e3ed5f640 1 --2- 172.21.15.98:0/3542953707 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8e380b6200 0x7f8e38155aa0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.144+0000 7f8e3ed5f640 1 -- 172.21.15.98:0/3542953707 >> 172.21.15.98:0/3542953707 conn(0x7f8e38074e40 msgr2=0x7f8e38072420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:44.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.144+0000 7f8e3ed5f640 1 -- 172.21.15.98:0/3542953707 shutdown_connections 2024-09-19T08:44:44.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.144+0000 7f8e3ed5f640 1 -- 172.21.15.98:0/3542953707 wait complete. 2024-09-19T08:44:44.161 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:44:44.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.264+0000 7f1111c68640 1 Processor -- start 2024-09-19T08:44:44.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.264+0000 7f1111c68640 1 -- start start 2024-09-19T08:44:44.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.268+0000 7f1111c68640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f110c06c3a0 0x7f110c0cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:44.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.268+0000 7f1111c68640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f110c06d310 con 0x7f110c0cf900 2024-09-19T08:44:44.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.268+0000 7f1111c68640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f110c06d5f0 con 0x7f110c15c550 2024-09-19T08:44:44.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.268+0000 7f1111c68640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f110c06d8d0 con 0x7f110c06c3a0 2024-09-19T08:44:44.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.268+0000 7f1103fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f110c06c3a0 0x7f110c0cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:44.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.268+0000 7f110bfff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f110c0cf900 0x7f110c0723a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40664/0 (socket says 172.21.15.98:40664) 2024-09-19T08:44:44.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.268+0000 7f110bfff640 1 -- 172.21.15.98:0/4239416078 learned_addr learned my addr 172.21.15.98:0/4239416078 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:44.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.268+0000 7f1103fff640 1 -- 172.21.15.98:0/4239416078 >> v1:172.21.15.139:6789/0 conn(0x7f110c15c550 legacy=0x7f110c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:44.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.268+0000 7f1103fff640 1 -- 172.21.15.98:0/4239416078 >> v1:172.21.15.98:6789/0 conn(0x7f110c0cf900 legacy=0x7f110c0723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:44.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.268+0000 7f1103fff640 1 -- 172.21.15.98:0/4239416078 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f110c06dbb0 con 0x7f110c06c3a0 2024-09-19T08:44:44.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.268+0000 7f1103fff640 1 --2- 172.21.15.98:0/4239416078 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f110c06c3a0 0x7f110c0cf3c0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f10f40097c0 tx=0x7f10f401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7a9b35b6df9cae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:44.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.268+0000 7f110affd640 1 -- 172.21.15.98:0/4239416078 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f40b0070 con 0x7f110c06c3a0 2024-09-19T08:44:44.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.268+0000 7f110affd640 1 -- 172.21.15.98:0/4239416078 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10f40ab600 con 0x7f110c06c3a0 2024-09-19T08:44:44.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.268+0000 7f110affd640 1 -- 172.21.15.98:0/4239416078 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f40ab9d0 con 0x7f110c06c3a0 2024-09-19T08:44:44.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.268+0000 7f1111c68640 1 -- 172.21.15.98:0/4239416078 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f110c06c3a0 msgr2=0x7f110c0cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:44.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.268+0000 7f1111c68640 1 --2- 172.21.15.98:0/4239416078 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f110c06c3a0 0x7f110c0cf3c0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f10f40097c0 tx=0x7f10f401c460 comp rx=0 tx=0).stop 2024-09-19T08:44:44.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.268+0000 7f1111c68640 1 -- 172.21.15.98:0/4239416078 shutdown_connections 2024-09-19T08:44:44.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.268+0000 7f1111c68640 1 --2- 172.21.15.98:0/4239416078 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f110c06c3a0 0x7f110c0cf3c0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.268+0000 7f1111c68640 1 -- 172.21.15.98:0/4239416078 >> 172.21.15.98:0/4239416078 conn(0x7f110c0b2010 msgr2=0x7f110c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:44.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.268+0000 7f1111c68640 1 -- 172.21.15.98:0/4239416078 shutdown_connections 2024-09-19T08:44:44.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.272+0000 7f1111c68640 1 -- 172.21.15.98:0/4239416078 wait complete. 2024-09-19T08:44:44.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.272+0000 7f1111c68640 1 Processor -- start 2024-09-19T08:44:44.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.272+0000 7f1111c68640 1 -- start start 2024-09-19T08:44:44.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.272+0000 7f1111c68640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f110c06c3a0 0x7f110c14bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:44.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.272+0000 7f1111c68640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f110c0cf900 0x7f110c158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:44.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.272+0000 7f1111c68640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f110c15c550 0x7f110c147ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:44.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.272+0000 7f1111c68640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f110c0736d0 con 0x7f110c15c550 2024-09-19T08:44:44.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.272+0000 7f1111c68640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f110c073550 con 0x7f110c06c3a0 2024-09-19T08:44:44.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.272+0000 7f1111c68640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f110c16a240 con 0x7f110c0cf900 2024-09-19T08:44:44.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.272+0000 7f110bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f110c15c550 0x7f110c147ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:44.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.272+0000 7f110bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f110c15c550 0x7f110c147ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40644/0 (socket says 172.21.15.98:40644) 2024-09-19T08:44:44.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.272+0000 7f110bfff640 1 -- 172.21.15.98:0/2581091277 learned_addr learned my addr 172.21.15.98:0/2581091277 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:44.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.272+0000 7f110b7fe640 1 --2- 172.21.15.98:0/2581091277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f110c06c3a0 0x7f110c14bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:44.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.272+0000 7f110bfff640 1 -- 172.21.15.98:0/2581091277 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f110c0cf900 msgr2=0x7f110c158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:44.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.272+0000 7f110bfff640 1 --2- 172.21.15.98:0/2581091277 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f110c0cf900 0x7f110c158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.272+0000 7f110bfff640 1 -- 172.21.15.98:0/2581091277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f110c06c3a0 msgr2=0x7f110c14bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:44.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.272+0000 7f110bfff640 1 --2- 172.21.15.98:0/2581091277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f110c06c3a0 0x7f110c14bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.272+0000 7f110bfff640 1 -- 172.21.15.98:0/2581091277 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f110c0cc960 con 0x7f110c15c550 2024-09-19T08:44:44.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.272+0000 7f110bfff640 1 --2- 172.21.15.98:0/2581091277 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f110c15c550 0x7f110c147ad0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f110400cbf0 tx=0x7f110400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:44.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.272+0000 7f1108ff9640 1 -- 172.21.15.98:0/2581091277 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1104017070 con 0x7f110c15c550 2024-09-19T08:44:44.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.272+0000 7f1108ff9640 1 -- 172.21.15.98:0/2581091277 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f110400ad40 con 0x7f110c15c550 2024-09-19T08:44:44.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.272+0000 7f1111c68640 1 -- 172.21.15.98:0/2581091277 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f110c156770 con 0x7f110c15c550 2024-09-19T08:44:44.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.272+0000 7f1108ff9640 1 -- 172.21.15.98:0/2581091277 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1104012600 con 0x7f110c15c550 2024-09-19T08:44:44.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.272+0000 7f1111c68640 1 -- 172.21.15.98:0/2581091277 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f110c156d30 con 0x7f110c15c550 2024-09-19T08:44:44.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.272+0000 7f1111c68640 1 -- 172.21.15.98:0/2581091277 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f110c0cf3c0 con 0x7f110c15c550 2024-09-19T08:44:44.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.276+0000 7f1108ff9640 1 -- 172.21.15.98:0/2581091277 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f110401a070 con 0x7f110c15c550 2024-09-19T08:44:44.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.276+0000 7f1108ff9640 1 --2- 172.21.15.98:0/2581091277 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f10ec067cb0 0x7f10ec06a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:44.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.276+0000 7f1108ff9640 1 -- 172.21.15.98:0/2581091277 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5839+0+0 (secure 0 0 0) 0x7f1104086440 con 0x7f110c15c550 2024-09-19T08:44:44.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.276+0000 7f110b7fe640 1 --2- 172.21.15.98:0/2581091277 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f10ec067cb0 0x7f10ec06a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:44.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.276+0000 7f110b7fe640 1 --2- 172.21.15.98:0/2581091277 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f10ec067cb0 0x7f10ec06a170 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f10fc007fd0 tx=0x7f10fc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:44.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.280+0000 7f1108ff9640 1 -- 172.21.15.98:0/2581091277 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f1104056460 con 0x7f110c15c550 2024-09-19T08:44:44.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.468+0000 7f1111c68640 1 -- 172.21.15.98:0/2581091277 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f110c0cdc90 con 0x7f110c15c550 2024-09-19T08:44:44.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.468+0000 7f1108ff9640 1 -- 172.21.15.98:0/2581091277 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5225 (secure 0 0 0) 0x7f110405a760 con 0x7f110c15c550 2024-09-19T08:44:44.471 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:44:44.471 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":61,"btime":"2024-09-19T08:44:10:283464+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6004,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3739962222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3739962222},{"type":"v1","addr":"172.21.15.139:6833","nonce":3739962222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6007,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/421473653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":421473653},{"type":"v1","addr":"172.21.15.139:6835","nonce":421473653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3126941916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3126941916},{"type":"v1","addr":"172.21.15.98:6833","nonce":3126941916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:43:54.489407+0000","modified":"2024-09-19T08:43:56.577758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6996},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6996":{"gid":6996,"name":"c","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/273091170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":273091170},{"type":"v1","addr":"172.21.15.98:6835","nonce":273091170}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6996,"qdb_cluster":[6996]},"id":8}]} 2024-09-19T08:44:44.472 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 61 2024-09-19T08:44:44.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.472+0000 7f1111c68640 1 -- 172.21.15.98:0/2581091277 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f10ec067cb0 msgr2=0x7f10ec06a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:44.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.472+0000 7f1111c68640 1 --2- 172.21.15.98:0/2581091277 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f10ec067cb0 0x7f10ec06a170 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f10fc007fd0 tx=0x7f10fc009040 comp rx=0 tx=0).stop 2024-09-19T08:44:44.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.472+0000 7f1111c68640 1 -- 172.21.15.98:0/2581091277 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f110c15c550 msgr2=0x7f110c147ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:44.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.472+0000 7f1111c68640 1 --2- 172.21.15.98:0/2581091277 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f110c15c550 0x7f110c147ad0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f110400cbf0 tx=0x7f110400a430 comp rx=0 tx=0).stop 2024-09-19T08:44:44.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.472+0000 7f1111c68640 1 -- 172.21.15.98:0/2581091277 shutdown_connections 2024-09-19T08:44:44.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.472+0000 7f1111c68640 1 --2- 172.21.15.98:0/2581091277 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f10ec067cb0 0x7f10ec06a170 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.472+0000 7f1111c68640 1 --2- 172.21.15.98:0/2581091277 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f110c15c550 0x7f110c147ad0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.472+0000 7f1111c68640 1 --2- 172.21.15.98:0/2581091277 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f110c0cf900 0x7f110c158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.472+0000 7f1111c68640 1 --2- 172.21.15.98:0/2581091277 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f110c06c3a0 0x7f110c14bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.472+0000 7f1111c68640 1 -- 172.21.15.98:0/2581091277 >> 172.21.15.98:0/2581091277 conn(0x7f110c0b2010 msgr2=0x7f110c067a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:44.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.472+0000 7f1111c68640 1 -- 172.21.15.98:0/2581091277 shutdown_connections 2024-09-19T08:44:44.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.472+0000 7f1111c68640 1 -- 172.21.15.98:0/2581091277 wait complete. 2024-09-19T08:44:44.489 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:44:44.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.608+0000 7ffb03ce0640 1 Processor -- start 2024-09-19T08:44:44.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.608+0000 7ffb03ce0640 1 -- start start 2024-09-19T08:44:44.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.608+0000 7ffb03ce0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffafc158e40 0x7ffafc06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:44.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.608+0000 7ffb03ce0640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ffafc0ce8b0 0x7ffafc0b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:44.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.608+0000 7ffb03ce0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffafc15fcf0 con 0x7ffafc06c1d0 2024-09-19T08:44:44.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.608+0000 7ffb03ce0640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ffafc06c610 con 0x7ffafc0ce8b0 2024-09-19T08:44:44.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.608+0000 7ffb03ce0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ffafc15ef30 con 0x7ffafc158e40 2024-09-19T08:44:44.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.608+0000 7ffb0227d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ffafc0ce8b0 0x7ffafc0b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:44.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.608+0000 7ffb01a7c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffafc158e40 0x7ffafc06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:44.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.608+0000 7ffb01a7c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffafc158e40 0x7ffafc06bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:36716/0 (socket says 172.21.15.98:36716) 2024-09-19T08:44:44.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.608+0000 7ffb0227d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ffafc0ce8b0 0x7ffafc0b3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:46148/0 (socket says 172.21.15.98:46148) 2024-09-19T08:44:44.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.608+0000 7ffb01a7c640 1 -- 172.21.15.98:0/434409201 learned_addr learned my addr 172.21.15.98:0/434409201 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:44.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.608+0000 7ffb0127b640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7ffafc06c1d0 0x7ffafc0ce1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:44902/0 (socket says 172.21.15.98:44902) 2024-09-19T08:44:44.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.608+0000 7ffb01a7c640 1 -- 172.21.15.98:0/434409201 >> v2:172.21.15.139:3300/0 conn(0x7ffafc0ce8b0 msgr2=0x7ffafc0b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:44.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.608+0000 7ffb01a7c640 1 --2- 172.21.15.98:0/434409201 >> v2:172.21.15.139:3300/0 conn(0x7ffafc0ce8b0 0x7ffafc0b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.608+0000 7ffb01a7c640 1 -- 172.21.15.98:0/434409201 >> v1:172.21.15.139:6789/0 conn(0x7ffafc06c1d0 legacy=0x7ffafc0ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:44.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.608+0000 7ffb01a7c640 1 -- 172.21.15.98:0/434409201 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffafc15f210 con 0x7ffafc158e40 2024-09-19T08:44:44.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.612+0000 7ffb01a7c640 1 --2- 172.21.15.98:0/434409201 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffafc158e40 0x7ffafc06bc90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8009870 tx=0x7ffaf801ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31f96d4e1509391b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:44.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.612+0000 7ffb00a7a640 1 -- 172.21.15.98:0/434409201 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf80b5070 con 0x7ffafc158e40 2024-09-19T08:44:44.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.612+0000 7ffb00a7a640 1 -- 172.21.15.98:0/434409201 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaf80a8a50 con 0x7ffafc158e40 2024-09-19T08:44:44.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.612+0000 7ffb00a7a640 1 -- 172.21.15.98:0/434409201 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf80a8e20 con 0x7ffafc158e40 2024-09-19T08:44:44.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.612+0000 7ffb03ce0640 1 -- 172.21.15.98:0/434409201 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffafc158e40 msgr2=0x7ffafc06bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:44.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.612+0000 7ffb03ce0640 1 --2- 172.21.15.98:0/434409201 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffafc158e40 0x7ffafc06bc90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8009870 tx=0x7ffaf801ed50 comp rx=0 tx=0).stop 2024-09-19T08:44:44.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.612+0000 7ffb03ce0640 1 -- 172.21.15.98:0/434409201 shutdown_connections 2024-09-19T08:44:44.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.612+0000 7ffb03ce0640 1 --2- 172.21.15.98:0/434409201 >> v2:172.21.15.139:3300/0 conn(0x7ffafc0ce8b0 0x7ffafc0b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.612+0000 7ffb03ce0640 1 --2- 172.21.15.98:0/434409201 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffafc158e40 0x7ffafc06bc90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.612+0000 7ffb03ce0640 1 -- 172.21.15.98:0/434409201 >> 172.21.15.98:0/434409201 conn(0x7ffafc0b2010 msgr2=0x7ffafc0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:44.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.612+0000 7ffb03ce0640 1 -- 172.21.15.98:0/434409201 shutdown_connections 2024-09-19T08:44:44.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.612+0000 7ffb03ce0640 1 -- 172.21.15.98:0/434409201 wait complete. 2024-09-19T08:44:44.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.612+0000 7ffb03ce0640 1 Processor -- start 2024-09-19T08:44:44.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.612+0000 7ffb03ce0640 1 -- start start 2024-09-19T08:44:44.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.612+0000 7ffb03ce0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffafc06c1d0 0x7ffafc15c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:44.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.612+0000 7ffb03ce0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffafc0ce8b0 0x7ffafc149d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:44.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.616+0000 7ffb03ce0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffafc158e40 0x7ffafc0b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:44.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.616+0000 7ffb03ce0640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ffafc159220 con 0x7ffafc0ce8b0 2024-09-19T08:44:44.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.616+0000 7ffb03ce0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffafc15f7d0 con 0x7ffafc158e40 2024-09-19T08:44:44.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.616+0000 7ffb03ce0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ffafc0cee20 con 0x7ffafc06c1d0 2024-09-19T08:44:44.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.616+0000 7ffb0127b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffafc0ce8b0 0x7ffafc149d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:44.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.616+0000 7ffb0127b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffafc0ce8b0 0x7ffafc149d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40656/0 (socket says 172.21.15.98:40656) 2024-09-19T08:44:44.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.616+0000 7ffb0127b640 1 -- 172.21.15.98:0/3142246955 learned_addr learned my addr 172.21.15.98:0/3142246955 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:44.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.616+0000 7ffb01a7c640 1 --2- 172.21.15.98:0/3142246955 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffafc06c1d0 0x7ffafc15c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:44.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.616+0000 7ffb0227d640 1 --2- 172.21.15.98:0/3142246955 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffafc158e40 0x7ffafc0b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:44.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.616+0000 7ffb0127b640 1 -- 172.21.15.98:0/3142246955 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffafc06c1d0 msgr2=0x7ffafc15c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:44.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.616+0000 7ffb0127b640 1 --2- 172.21.15.98:0/3142246955 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffafc06c1d0 0x7ffafc15c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.616+0000 7ffb0127b640 1 -- 172.21.15.98:0/3142246955 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffafc158e40 msgr2=0x7ffafc0b0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:44.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.616+0000 7ffb0127b640 1 --2- 172.21.15.98:0/3142246955 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffafc158e40 0x7ffafc0b0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.616+0000 7ffb0127b640 1 -- 172.21.15.98:0/3142246955 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffafc15b150 con 0x7ffafc0ce8b0 2024-09-19T08:44:44.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.616+0000 7ffb0127b640 1 --2- 172.21.15.98:0/3142246955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffafc0ce8b0 0x7ffafc149d50 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7ffaf000c870 tx=0x7ffaf000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:44.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.616+0000 7ffaee7fc640 1 -- 172.21.15.98:0/3142246955 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0017070 con 0x7ffafc0ce8b0 2024-09-19T08:44:44.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.616+0000 7ffaee7fc640 1 -- 172.21.15.98:0/3142246955 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaf000a640 con 0x7ffafc0ce8b0 2024-09-19T08:44:44.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.616+0000 7ffaee7fc640 1 -- 172.21.15.98:0/3142246955 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf000aa30 con 0x7ffafc0ce8b0 2024-09-19T08:44:44.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.616+0000 7ffb03ce0640 1 -- 172.21.15.98:0/3142246955 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffafc15b430 con 0x7ffafc0ce8b0 2024-09-19T08:44:44.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.616+0000 7ffb03ce0640 1 -- 172.21.15.98:0/3142246955 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffafc15b9f0 con 0x7ffafc0ce8b0 2024-09-19T08:44:44.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.616+0000 7ffb03ce0640 1 -- 172.21.15.98:0/3142246955 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffafc06bc90 con 0x7ffafc0ce8b0 2024-09-19T08:44:44.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.616+0000 7ffaee7fc640 1 -- 172.21.15.98:0/3142246955 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7ffaf001a070 con 0x7ffafc0ce8b0 2024-09-19T08:44:44.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.616+0000 7ffaee7fc640 1 --2- 172.21.15.98:0/3142246955 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ffae0067cb0 0x7ffae006a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:44.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.616+0000 7ffaee7fc640 1 -- 172.21.15.98:0/3142246955 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5839+0+0 (secure 0 0 0) 0x7ffaf0085a00 con 0x7ffafc0ce8b0 2024-09-19T08:44:44.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.616+0000 7ffb01a7c640 1 --2- 172.21.15.98:0/3142246955 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ffae0067cb0 0x7ffae006a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:44.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.620+0000 7ffb01a7c640 1 --2- 172.21.15.98:0/3142246955 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ffae0067cb0 0x7ffae006a170 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ffaf80b2000 tx=0x7ffaf80aa070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:44.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.620+0000 7ffaee7fc640 1 -- 172.21.15.98:0/3142246955 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ffaf0055890 con 0x7ffafc0ce8b0 2024-09-19T08:44:44.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.784+0000 7ffb03ce0640 1 -- 172.21.15.98:0/3142246955 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffafc0b0a80 con 0x7ffafc0ce8b0 2024-09-19T08:44:44.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.784+0000 7ffaee7fc640 1 -- 172.21.15.98:0/3142246955 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+16915 (secure 0 0 0) 0x7ffaf0059b90 con 0x7ffafc0ce8b0 2024-09-19T08:44:44.787 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:44:44.787 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":106,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:44:41.439895+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:43:52.161395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-19T08:43:53.497026+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.3c","mappings":[{"from":0,"to":5}]},{"pgid":"17.3f","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:44:44.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.788+0000 7ffb03ce0640 1 -- 172.21.15.98:0/3142246955 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ffae0067cb0 msgr2=0x7ffae006a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:44.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.788+0000 7ffb03ce0640 1 --2- 172.21.15.98:0/3142246955 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ffae0067cb0 0x7ffae006a170 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ffaf80b2000 tx=0x7ffaf80aa070 comp rx=0 tx=0).stop 2024-09-19T08:44:44.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.788+0000 7ffb03ce0640 1 -- 172.21.15.98:0/3142246955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffafc0ce8b0 msgr2=0x7ffafc149d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:44.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.788+0000 7ffb03ce0640 1 --2- 172.21.15.98:0/3142246955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffafc0ce8b0 0x7ffafc149d50 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7ffaf000c870 tx=0x7ffaf000cd30 comp rx=0 tx=0).stop 2024-09-19T08:44:44.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.788+0000 7ffb03ce0640 1 -- 172.21.15.98:0/3142246955 shutdown_connections 2024-09-19T08:44:44.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.788+0000 7ffb03ce0640 1 --2- 172.21.15.98:0/3142246955 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ffae0067cb0 0x7ffae006a170 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.788+0000 7ffb03ce0640 1 --2- 172.21.15.98:0/3142246955 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffafc158e40 0x7ffafc0b0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.788+0000 7ffb03ce0640 1 --2- 172.21.15.98:0/3142246955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffafc0ce8b0 0x7ffafc149d50 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.788+0000 7ffb03ce0640 1 --2- 172.21.15.98:0/3142246955 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffafc06c1d0 0x7ffafc15c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.788+0000 7ffb03ce0640 1 -- 172.21.15.98:0/3142246955 >> 172.21.15.98:0/3142246955 conn(0x7ffafc0b2010 msgr2=0x7ffafc0ae8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:44.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.788+0000 7ffb03ce0640 1 -- 172.21.15.98:0/3142246955 shutdown_connections 2024-09-19T08:44:44.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.788+0000 7ffb03ce0640 1 -- 172.21.15.98:0/3142246955 wait complete. 2024-09-19T08:44:44.806 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-19T08:44:44.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.920+0000 7ff79aea6640 1 Processor -- start 2024-09-19T08:44:44.921 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.920+0000 7ff79aea6640 1 -- start start 2024-09-19T08:44:44.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.920+0000 7ff79aea6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7940733b0 0x7ff7940cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:44.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.920+0000 7ff79aea6640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff79406d310 con 0x7ff79415c550 2024-09-19T08:44:44.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.920+0000 7ff79aea6640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff79406d5f0 con 0x7ff79406c3a0 2024-09-19T08:44:44.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.920+0000 7ff79aea6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff79406d8d0 con 0x7ff7940733b0 2024-09-19T08:44:44.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.920+0000 7ff798c42640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7ff79415c550 0x7ff79406bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40678/0 (socket says 172.21.15.98:40678) 2024-09-19T08:44:44.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.920+0000 7ff798c42640 1 -- 172.21.15.98:0/617583001 learned_addr learned my addr 172.21.15.98:0/617583001 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:44.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.920+0000 7ff799443640 1 --2- 172.21.15.98:0/617583001 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7940733b0 0x7ff7940cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:44.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.920+0000 7ff7937fe640 1 -- 172.21.15.98:0/617583001 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3827464380 0 0) 0x7ff79406d310 con 0x7ff79415c550 2024-09-19T08:44:44.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.920+0000 7ff7937fe640 1 -- 172.21.15.98:0/617583001 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff77c003530 con 0x7ff79415c550 2024-09-19T08:44:44.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.920+0000 7ff7937fe640 1 -- 172.21.15.98:0/617583001 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3215762394 0 0) 0x7ff77c003530 con 0x7ff79415c550 2024-09-19T08:44:44.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.920+0000 7ff7937fe640 1 -- 172.21.15.98:0/617583001 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7940733b0 msgr2=0x7ff7940cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:44.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.920+0000 7ff7937fe640 1 --2- 172.21.15.98:0/617583001 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7940733b0 0x7ff7940cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.920+0000 7ff7937fe640 1 -- 172.21.15.98:0/617583001 >> v1:172.21.15.139:6789/0 conn(0x7ff79406c3a0 legacy=0x7ff7940cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:44.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.920+0000 7ff7937fe640 1 -- 172.21.15.98:0/617583001 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff79406dbb0 con 0x7ff79415c550 2024-09-19T08:44:44.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.920+0000 7ff7937fe640 1 -- 172.21.15.98:0/617583001 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7ff784001680 con 0x7ff79415c550 2024-09-19T08:44:44.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.920+0000 7ff7937fe640 1 -- 172.21.15.98:0/617583001 >> v1:172.21.15.98:6789/0 conn(0x7ff79415c550 legacy=0x7ff79406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:44.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.920+0000 7ff7937fe640 1 --2- 172.21.15.98:0/617583001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff77c0039f0 0x7ff77c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:44.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.920+0000 7ff7937fe640 1 --2- 172.21.15.98:0/617583001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff77c004810 0x7ff77c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:44.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.920+0000 7ff7937fe640 1 --2- 172.21.15.98:0/617583001 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff77c0051d0 0x7ff77c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:44.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.920+0000 7ff7937fe640 1 -- 172.21.15.98:0/617583001 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff79406d8d0 con 0x7ff77c0039f0 2024-09-19T08:44:44.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.920+0000 7ff7937fe640 1 -- 172.21.15.98:0/617583001 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff77c00bff0 con 0x7ff77c004810 2024-09-19T08:44:44.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.920+0000 7ff7937fe640 1 -- 172.21.15.98:0/617583001 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff77c00c1f0 con 0x7ff77c0051d0 2024-09-19T08:44:44.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.920+0000 7ff799443640 1 --2- 172.21.15.98:0/617583001 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7940733b0 0x7ff7940cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:44:44.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.920+0000 7ff798c42640 1 --2- 172.21.15.98:0/617583001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff77c0039f0 0x7ff77c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:44.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.920+0000 7ff793fff640 1 --2- 172.21.15.98:0/617583001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff77c004810 0x7ff77c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:44.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.920+0000 7ff798c42640 1 -- 172.21.15.98:0/617583001 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff77c0051d0 msgr2=0x7ff77c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:44.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.920+0000 7ff798c42640 1 --2- 172.21.15.98:0/617583001 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff77c0051d0 0x7ff77c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.920+0000 7ff798c42640 1 -- 172.21.15.98:0/617583001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff77c004810 msgr2=0x7ff77c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:44.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.920+0000 7ff798c42640 1 --2- 172.21.15.98:0/617583001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff77c004810 0x7ff77c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.920+0000 7ff798c42640 1 -- 172.21.15.98:0/617583001 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff77c003530 con 0x7ff77c0039f0 2024-09-19T08:44:44.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.920+0000 7ff793fff640 1 --2- 172.21.15.98:0/617583001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff77c004810 0x7ff77c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:44.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.924+0000 7ff798c42640 1 --2- 172.21.15.98:0/617583001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff77c0039f0 0x7ff77c003e10 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7ff78400c1c0 tx=0x7ff78401cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb8471e19aa9006 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:44.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.924+0000 7ff7937fe640 1 -- 172.21.15.98:0/617583001 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff784007d00 con 0x7ff77c0039f0 2024-09-19T08:44:44.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.924+0000 7ff7937fe640 1 -- 172.21.15.98:0/617583001 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7840a9020 con 0x7ff77c0039f0 2024-09-19T08:44:44.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.924+0000 7ff7937fe640 1 -- 172.21.15.98:0/617583001 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff784005ae0 con 0x7ff77c0039f0 2024-09-19T08:44:44.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.924+0000 7ff79aea6640 1 -- 172.21.15.98:0/617583001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff77c0039f0 msgr2=0x7ff77c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:44.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.924+0000 7ff79aea6640 1 --2- 172.21.15.98:0/617583001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff77c0039f0 0x7ff77c003e10 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7ff78400c1c0 tx=0x7ff78401cbe0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.924+0000 7ff799443640 1 -- 172.21.15.98:0/617583001 reap_dead start 2024-09-19T08:44:44.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.924+0000 7ff79aea6640 1 -- 172.21.15.98:0/617583001 shutdown_connections 2024-09-19T08:44:44.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.924+0000 7ff79aea6640 1 -- 172.21.15.98:0/617583001 >> 172.21.15.98:0/617583001 conn(0x7ff7940b2010 msgr2=0x7ff7940b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:44.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.924+0000 7ff79aea6640 1 -- 172.21.15.98:0/617583001 shutdown_connections 2024-09-19T08:44:44.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.924+0000 7ff79aea6640 1 -- 172.21.15.98:0/617583001 wait complete. 2024-09-19T08:44:44.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.924+0000 7ff79aea6640 1 Processor -- start 2024-09-19T08:44:44.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.924+0000 7ff79aea6640 1 -- start start 2024-09-19T08:44:44.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.924+0000 7ff79aea6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff77c0039f0 0x7ff7940b0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:44.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.928+0000 7ff79aea6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff794156130 0x7ff794150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:44.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.928+0000 7ff79aea6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff794164580 0x7ff794164a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:44.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.928+0000 7ff79aea6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff794167c40 con 0x7ff794156130 2024-09-19T08:44:44.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.928+0000 7ff79aea6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff79415cb40 con 0x7ff794164580 2024-09-19T08:44:44.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.928+0000 7ff79aea6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff794167dc0 con 0x7ff77c0039f0 2024-09-19T08:44:44.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.928+0000 7ff793fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff794156130 0x7ff794150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:44.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.928+0000 7ff793fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff794156130 0x7ff794150b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40680/0 (socket says 172.21.15.98:40680) 2024-09-19T08:44:44.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.928+0000 7ff793fff640 1 -- 172.21.15.98:0/3596506652 learned_addr learned my addr 172.21.15.98:0/3596506652 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:44.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.928+0000 7ff798c42640 1 --2- 172.21.15.98:0/3596506652 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff77c0039f0 0x7ff7940b0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:44.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.928+0000 7ff799443640 1 --2- 172.21.15.98:0/3596506652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff794164580 0x7ff794164a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:44.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.928+0000 7ff793fff640 1 -- 172.21.15.98:0/3596506652 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff77c0039f0 msgr2=0x7ff7940b0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:44.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.928+0000 7ff793fff640 1 --2- 172.21.15.98:0/3596506652 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff77c0039f0 0x7ff7940b0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.928+0000 7ff793fff640 1 -- 172.21.15.98:0/3596506652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff794164580 msgr2=0x7ff794164a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:44.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.928+0000 7ff793fff640 1 --2- 172.21.15.98:0/3596506652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff794164580 0x7ff794164a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:44.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.928+0000 7ff793fff640 1 -- 172.21.15.98:0/3596506652 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff79415ede0 con 0x7ff794156130 2024-09-19T08:44:44.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.928+0000 7ff793fff640 1 --2- 172.21.15.98:0/3596506652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff794156130 0x7ff794150b00 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7ff78800d540 tx=0x7ff78800da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:44.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.928+0000 7ff7917fa640 1 -- 172.21.15.98:0/3596506652 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788005480 con 0x7ff794156130 2024-09-19T08:44:44.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.928+0000 7ff7917fa640 1 -- 172.21.15.98:0/3596506652 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff788017070 con 0x7ff794156130 2024-09-19T08:44:44.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.928+0000 7ff79aea6640 1 -- 172.21.15.98:0/3596506652 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff794158b80 con 0x7ff794156130 2024-09-19T08:44:44.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.928+0000 7ff7917fa640 1 -- 172.21.15.98:0/3596506652 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788003840 con 0x7ff794156130 2024-09-19T08:44:44.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.928+0000 7ff79aea6640 1 -- 172.21.15.98:0/3596506652 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff79415f330 con 0x7ff794156130 2024-09-19T08:44:44.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.928+0000 7ff79aea6640 1 -- 172.21.15.98:0/3596506652 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff794164f40 con 0x7ff794156130 2024-09-19T08:44:44.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.928+0000 7ff7917fa640 1 -- 172.21.15.98:0/3596506652 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7ff7880039e0 con 0x7ff794156130 2024-09-19T08:44:44.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.928+0000 7ff7917fa640 1 --2- 172.21.15.98:0/3596506652 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff780067ce0 0x7ff78006a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:44.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.928+0000 7ff7917fa640 1 -- 172.21.15.98:0/3596506652 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5839+0+0 (secure 0 0 0) 0x7ff788086300 con 0x7ff794156130 2024-09-19T08:44:44.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.928+0000 7ff798c42640 1 --2- 172.21.15.98:0/3596506652 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff780067ce0 0x7ff78006a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:44.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.928+0000 7ff798c42640 1 --2- 172.21.15.98:0/3596506652 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff780067ce0 0x7ff78006a1a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff7840aa000 tx=0x7ff794073860 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:44.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:44.932+0000 7ff7917fa640 1 -- 172.21.15.98:0/3596506652 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff7880566d0 con 0x7ff794156130 2024-09-19T08:44:45.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.116+0000 7ff79aea6640 1 -- 172.21.15.98:0/3596506652 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff7940cf3c0 con 0x7ff794156130 2024-09-19T08:44:45.118 INFO:tasks.ceph.mon.a.smithi098.stderr:2024-09-19T08:44:45.116+0000 7f5e49017640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T08:44:45.120 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.120+0000 7ff7917fa640 1 -- 172.21.15.98:0/3596506652 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v62) ==== 161+0+0 (secure 0 0 0) 0x7ff788055fd0 con 0x7ff794156130 2024-09-19T08:44:45.120 INFO:teuthology.orchestra.run.smithi098.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T08:44:45.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.124+0000 7ff79aea6640 1 -- 172.21.15.98:0/3596506652 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff780067ce0 msgr2=0x7ff78006a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:45.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.124+0000 7ff79aea6640 1 --2- 172.21.15.98:0/3596506652 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff780067ce0 0x7ff78006a1a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff7840aa000 tx=0x7ff794073860 comp rx=0 tx=0).stop 2024-09-19T08:44:45.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.124+0000 7ff79aea6640 1 -- 172.21.15.98:0/3596506652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff794156130 msgr2=0x7ff794150b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:45.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.124+0000 7ff79aea6640 1 --2- 172.21.15.98:0/3596506652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff794156130 0x7ff794150b00 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7ff78800d540 tx=0x7ff78800da00 comp rx=0 tx=0).stop 2024-09-19T08:44:45.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.124+0000 7ff79aea6640 1 -- 172.21.15.98:0/3596506652 shutdown_connections 2024-09-19T08:44:45.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.124+0000 7ff79aea6640 1 --2- 172.21.15.98:0/3596506652 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff780067ce0 0x7ff78006a1a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:45.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.124+0000 7ff79aea6640 1 --2- 172.21.15.98:0/3596506652 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff794164580 0x7ff794164a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:45.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.124+0000 7ff79aea6640 1 --2- 172.21.15.98:0/3596506652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff794156130 0x7ff794150b00 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:45.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.124+0000 7ff79aea6640 1 --2- 172.21.15.98:0/3596506652 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff77c0039f0 0x7ff7940b0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:45.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.124+0000 7ff79aea6640 1 -- 172.21.15.98:0/3596506652 >> 172.21.15.98:0/3596506652 conn(0x7ff7940b2010 msgr2=0x7ff7940cd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:45.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.124+0000 7ff79aea6640 1 -- 172.21.15.98:0/3596506652 shutdown_connections 2024-09-19T08:44:45.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.124+0000 7ff79aea6640 1 -- 172.21.15.98:0/3596506652 wait complete. 2024-09-19T08:44:45.142 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T08:44:45.174 INFO:tasks.ceph.mds.c.smithi098.stdout:starting mds.c at 2024-09-19T08:44:45.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.252+0000 7f23c2a70640 1 Processor -- start 2024-09-19T08:44:45.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.252+0000 7f23c2a70640 1 -- start start 2024-09-19T08:44:45.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.252+0000 7f23c2a70640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f23bc06caf0 0x7f23bc0b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:45.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.252+0000 7f23c2a70640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f23bc15c730 0x7f23bc06ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:45.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.252+0000 7f23c2a70640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23bc06d410 con 0x7f23bc0b3770 2024-09-19T08:44:45.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.252+0000 7f23c2a70640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f23bc0cc310 con 0x7f23bc15c730 2024-09-19T08:44:45.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.252+0000 7f23c2a70640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f23bc0b08c0 con 0x7f23bc06caf0 2024-09-19T08:44:45.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.252+0000 7f23bbfff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f23bc15c730 0x7f23bc06ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:45.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.252+0000 7f23c100d640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f23bc0b3770 0x7f23bc0cbc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40704/0 (socket says 172.21.15.98:40704) 2024-09-19T08:44:45.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.252+0000 7f23c100d640 1 -- 172.21.15.98:0/2838449018 learned_addr learned my addr 172.21.15.98:0/2838449018 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:45.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.252+0000 7f23c080c640 1 --2- 172.21.15.98:0/2838449018 >> v2:172.21.15.139:3300/0 conn(0x7f23bc06caf0 0x7f23bc0b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:45.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.252+0000 7f23bbfff640 1 -- 172.21.15.98:0/2838449018 >> v2:172.21.15.139:3300/0 conn(0x7f23bc06caf0 msgr2=0x7f23bc0b02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:45.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.252+0000 7f23bbfff640 1 --2- 172.21.15.98:0/2838449018 >> v2:172.21.15.139:3300/0 conn(0x7f23bc06caf0 0x7f23bc0b02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:45.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.252+0000 7f23bbfff640 1 -- 172.21.15.98:0/2838449018 >> v1:172.21.15.98:6789/0 conn(0x7f23bc0b3770 legacy=0x7f23bc0cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:45.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.252+0000 7f23bbfff640 1 -- 172.21.15.98:0/2838449018 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23bc0b0ba0 con 0x7f23bc15c730 2024-09-19T08:44:45.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.252+0000 7f23bb7fe640 1 -- 172.21.15.98:0/2838449018 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4141054976 0 0) 0x7f23bc06d410 con 0x7f23bc0b3770 2024-09-19T08:44:45.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.256+0000 7f23bbfff640 1 --2- 172.21.15.98:0/2838449018 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23bc15c730 0x7f23bc06ced0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f23bc075c80 tx=0x7f23b001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f34130e0acf03e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:45.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.256+0000 7f23bb7fe640 1 -- 172.21.15.98:0/2838449018 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b00ab650 con 0x7f23bc15c730 2024-09-19T08:44:45.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.256+0000 7f23bb7fe640 1 -- 172.21.15.98:0/2838449018 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23b00b0070 con 0x7f23bc15c730 2024-09-19T08:44:45.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.256+0000 7f23bb7fe640 1 -- 172.21.15.98:0/2838449018 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b00aba10 con 0x7f23bc15c730 2024-09-19T08:44:45.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.256+0000 7f23c2a70640 1 -- 172.21.15.98:0/2838449018 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23bc15c730 msgr2=0x7f23bc06ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:45.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.256+0000 7f23c2a70640 1 --2- 172.21.15.98:0/2838449018 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23bc15c730 0x7f23bc06ced0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f23bc075c80 tx=0x7f23b001c450 comp rx=0 tx=0).stop 2024-09-19T08:44:45.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.256+0000 7f23c2a70640 1 -- 172.21.15.98:0/2838449018 shutdown_connections 2024-09-19T08:44:45.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.256+0000 7f23c2a70640 1 --2- 172.21.15.98:0/2838449018 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23bc15c730 0x7f23bc06ced0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:45.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.256+0000 7f23c2a70640 1 --2- 172.21.15.98:0/2838449018 >> v2:172.21.15.139:3300/0 conn(0x7f23bc06caf0 0x7f23bc0b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:45.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.256+0000 7f23c2a70640 1 -- 172.21.15.98:0/2838449018 >> 172.21.15.98:0/2838449018 conn(0x7f23bc0b5f50 msgr2=0x7f23bc0b6650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:45.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.256+0000 7f23c2a70640 1 -- 172.21.15.98:0/2838449018 shutdown_connections 2024-09-19T08:44:45.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.256+0000 7f23c2a70640 1 -- 172.21.15.98:0/2838449018 wait complete. 2024-09-19T08:44:45.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.256+0000 7f23c2a70640 1 Processor -- start 2024-09-19T08:44:45.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.256+0000 7f23c2a70640 1 -- start start 2024-09-19T08:44:45.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.256+0000 7f23c2a70640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23bc06caf0 0x7f23bc149120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:45.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.256+0000 7f23c2a70640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23bc0b3770 0x7f23bc149660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:45.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.256+0000 7f23c2a70640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23bc15c730 0x7f23bc150d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:45.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.256+0000 7f23c2a70640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f23bc0b3cd0 con 0x7f23bc15c730 2024-09-19T08:44:45.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.256+0000 7f23c2a70640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f23bc0b3b50 con 0x7f23bc0b3770 2024-09-19T08:44:45.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.256+0000 7f23c2a70640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f23bc169120 con 0x7f23bc06caf0 2024-09-19T08:44:45.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.256+0000 7f23c100d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23bc15c730 0x7f23bc150d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:45.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.256+0000 7f23c100d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23bc15c730 0x7f23bc150d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40716/0 (socket says 172.21.15.98:40716) 2024-09-19T08:44:45.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.256+0000 7f23c100d640 1 -- 172.21.15.98:0/2136112675 learned_addr learned my addr 172.21.15.98:0/2136112675 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:45.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.256+0000 7f23bbfff640 1 --2- 172.21.15.98:0/2136112675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23bc0b3770 0x7f23bc149660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:45.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.256+0000 7f23c080c640 1 --2- 172.21.15.98:0/2136112675 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23bc06caf0 0x7f23bc149120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:45.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.256+0000 7f23c100d640 1 -- 172.21.15.98:0/2136112675 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23bc06caf0 msgr2=0x7f23bc149120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:45.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.256+0000 7f23c100d640 1 --2- 172.21.15.98:0/2136112675 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23bc06caf0 0x7f23bc149120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:45.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.256+0000 7f23c100d640 1 -- 172.21.15.98:0/2136112675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23bc0b3770 msgr2=0x7f23bc149660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:45.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.256+0000 7f23c100d640 1 --2- 172.21.15.98:0/2136112675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23bc0b3770 0x7f23bc149660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:45.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.260+0000 7f23c100d640 1 -- 172.21.15.98:0/2136112675 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23bc14a3b0 con 0x7f23bc15c730 2024-09-19T08:44:45.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.260+0000 7f23c100d640 1 --2- 172.21.15.98:0/2136112675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23bc15c730 0x7f23bc150d80 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f23b400c780 tx=0x7f23b400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:45.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.260+0000 7f23b97fa640 1 -- 172.21.15.98:0/2136112675 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b4018070 con 0x7f23bc15c730 2024-09-19T08:44:45.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.260+0000 7f23b97fa640 1 -- 172.21.15.98:0/2136112675 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23b400ce30 con 0x7f23bc15c730 2024-09-19T08:44:45.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.260+0000 7f23c2a70640 1 -- 172.21.15.98:0/2136112675 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23bc152c90 con 0x7f23bc15c730 2024-09-19T08:44:45.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.260+0000 7f23b97fa640 1 -- 172.21.15.98:0/2136112675 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b400a860 con 0x7f23bc15c730 2024-09-19T08:44:45.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.260+0000 7f23c2a70640 1 -- 172.21.15.98:0/2136112675 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23bc1528e0 con 0x7f23bc15c730 2024-09-19T08:44:45.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.260+0000 7f23c2a70640 1 -- 172.21.15.98:0/2136112675 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23bc0b02e0 con 0x7f23bc15c730 2024-09-19T08:44:45.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.260+0000 7f23b97fa640 1 -- 172.21.15.98:0/2136112675 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f23b401b070 con 0x7f23bc15c730 2024-09-19T08:44:45.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.260+0000 7f23b97fa640 1 --2- 172.21.15.98:0/2136112675 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f23a8067ce0 0x7f23a806a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:45.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.260+0000 7f23b97fa640 1 -- 172.21.15.98:0/2136112675 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5881+0+0 (secure 0 0 0) 0x7f23b4085a30 con 0x7f23bc15c730 2024-09-19T08:44:45.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.260+0000 7f23c080c640 1 --2- 172.21.15.98:0/2136112675 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f23a8067ce0 0x7f23a806a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:45.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.260+0000 7f23c080c640 1 --2- 172.21.15.98:0/2136112675 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f23a8067ce0 0x7f23a806a1a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f23ac00b930 tx=0x7f23ac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:45.266 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.264+0000 7f23b97fa640 1 -- 172.21.15.98:0/2136112675 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f23bc152c90 con 0x7f23bc15c730 2024-09-19T08:44:45.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:45.448+0000 7f23c2a70640 1 -- 172.21.15.98:0/2136112675 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f23bc15b230 con 0x7f23bc15c730 2024-09-19T08:44:46.123 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.120+0000 7f23b97fa640 1 -- 172.21.15.98:0/2136112675 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v63) ==== 104+0+0 (secure 0 0 0) 0x7f23b4055f70 con 0x7f23bc15c730 2024-09-19T08:44:46.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.124+0000 7f23c2a70640 1 -- 172.21.15.98:0/2136112675 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f23a8067ce0 msgr2=0x7f23a806a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:46.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.124+0000 7f23c2a70640 1 --2- 172.21.15.98:0/2136112675 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f23a8067ce0 0x7f23a806a1a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f23ac00b930 tx=0x7f23ac008040 comp rx=0 tx=0).stop 2024-09-19T08:44:46.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.124+0000 7f23c2a70640 1 -- 172.21.15.98:0/2136112675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23bc15c730 msgr2=0x7f23bc150d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:46.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.124+0000 7f23c2a70640 1 --2- 172.21.15.98:0/2136112675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23bc15c730 0x7f23bc150d80 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f23b400c780 tx=0x7f23b400cc40 comp rx=0 tx=0).stop 2024-09-19T08:44:46.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.124+0000 7f23c2a70640 1 -- 172.21.15.98:0/2136112675 shutdown_connections 2024-09-19T08:44:46.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.124+0000 7f23c2a70640 1 --2- 172.21.15.98:0/2136112675 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f23a8067ce0 0x7f23a806a1a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:46.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.124+0000 7f23c2a70640 1 --2- 172.21.15.98:0/2136112675 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f23bc15c730 0x7f23bc150d80 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:46.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.124+0000 7f23c2a70640 1 --2- 172.21.15.98:0/2136112675 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f23bc0b3770 0x7f23bc149660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:46.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.124+0000 7f23c2a70640 1 --2- 172.21.15.98:0/2136112675 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f23bc06caf0 0x7f23bc149120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:46.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.124+0000 7f23c2a70640 1 -- 172.21.15.98:0/2136112675 >> 172.21.15.98:0/2136112675 conn(0x7f23bc0b5f50 msgr2=0x7f23bc06ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:46.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.124+0000 7f23c2a70640 1 -- 172.21.15.98:0/2136112675 shutdown_connections 2024-09-19T08:44:46.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.124+0000 7f23c2a70640 1 -- 172.21.15.98:0/2136112675 wait complete. 2024-09-19T08:44:46.148 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T08:44:46.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.320+0000 7fd3aa269640 1 Processor -- start 2024-09-19T08:44:46.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.320+0000 7fd3aa269640 1 -- start start 2024-09-19T08:44:46.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.320+0000 7fd3aa269640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3a415e880 0x7fd3a4074c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:46.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.320+0000 7fd3aa269640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd3a40d0410 0x7fd3a40751b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:46.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.320+0000 7fd3aa269640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3a4074260 con 0x7fd3a406bc90 2024-09-19T08:44:46.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.320+0000 7fd3aa269640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fd3a40cef70 con 0x7fd3a40d0410 2024-09-19T08:44:46.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.320+0000 7fd3aa269640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd3a40b6b70 con 0x7fd3a415e880 2024-09-19T08:44:46.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.320+0000 7fd3a37fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fd3a406bc90 0x7fd3a4162430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40712/0 (socket says 172.21.15.98:40712) 2024-09-19T08:44:46.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.320+0000 7fd3a37fe640 1 -- 172.21.15.98:0/2640926863 learned_addr learned my addr 172.21.15.98:0/2640926863 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:46.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.320+0000 7fd3a8806640 1 --2- 172.21.15.98:0/2640926863 >> v2:172.21.15.139:3300/0 conn(0x7fd3a40d0410 0x7fd3a40751b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:46.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.320+0000 7fd3a3fff640 1 --2- 172.21.15.98:0/2640926863 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3a415e880 0x7fd3a4074c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:46.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.320+0000 7fd3a2ffd640 1 -- 172.21.15.98:0/2640926863 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3544373312 0 0) 0x7fd3a4074260 con 0x7fd3a406bc90 2024-09-19T08:44:46.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.320+0000 7fd3a2ffd640 1 -- 172.21.15.98:0/2640926863 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd38c003530 con 0x7fd3a406bc90 2024-09-19T08:44:46.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.320+0000 7fd3a2ffd640 1 -- 172.21.15.98:0/2640926863 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3884195609 0 0) 0x7fd38c003530 con 0x7fd3a406bc90 2024-09-19T08:44:46.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.320+0000 7fd3a2ffd640 1 -- 172.21.15.98:0/2640926863 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3a415e880 msgr2=0x7fd3a4074c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:46.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.320+0000 7fd3a2ffd640 1 --2- 172.21.15.98:0/2640926863 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3a415e880 0x7fd3a4074c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:46.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.320+0000 7fd3a2ffd640 1 -- 172.21.15.98:0/2640926863 >> v2:172.21.15.139:3300/0 conn(0x7fd3a40d0410 msgr2=0x7fd3a40751b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:46.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.320+0000 7fd3a2ffd640 1 --2- 172.21.15.98:0/2640926863 >> v2:172.21.15.139:3300/0 conn(0x7fd3a40d0410 0x7fd3a40751b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:46.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.320+0000 7fd3a2ffd640 1 -- 172.21.15.98:0/2640926863 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a40b6e10 con 0x7fd3a406bc90 2024-09-19T08:44:46.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.320+0000 7fd3a2ffd640 1 -- 172.21.15.98:0/2640926863 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fd398001680 con 0x7fd3a406bc90 2024-09-19T08:44:46.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.320+0000 7fd3a2ffd640 1 -- 172.21.15.98:0/2640926863 >> v1:172.21.15.98:6789/0 conn(0x7fd3a406bc90 legacy=0x7fd3a4162430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:46.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.320+0000 7fd3a2ffd640 1 --2- 172.21.15.98:0/2640926863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd38c0039c0 0x7fd38c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:46.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.320+0000 7fd3a2ffd640 1 --2- 172.21.15.98:0/2640926863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd38c004810 0x7fd38c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:46.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.320+0000 7fd3a2ffd640 1 --2- 172.21.15.98:0/2640926863 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3a415e880 0x7fd38c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:46.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.320+0000 7fd3a2ffd640 1 -- 172.21.15.98:0/2640926863 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a40cef70 con 0x7fd38c0039c0 2024-09-19T08:44:46.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.324+0000 7fd3a2ffd640 1 -- 172.21.15.98:0/2640926863 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a40b6b70 con 0x7fd38c004810 2024-09-19T08:44:46.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.324+0000 7fd3a2ffd640 1 -- 172.21.15.98:0/2640926863 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd38c00ba50 con 0x7fd3a415e880 2024-09-19T08:44:46.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.324+0000 7fd3a3fff640 1 --2- 172.21.15.98:0/2640926863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd38c0039c0 0x7fd38c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:46.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.324+0000 7fd3a37fe640 1 --2- 172.21.15.98:0/2640926863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd38c004810 0x7fd38c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:46.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.324+0000 7fd3a8806640 1 --2- 172.21.15.98:0/2640926863 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3a415e880 0x7fd38c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:46.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.324+0000 7fd3a3fff640 1 -- 172.21.15.98:0/2640926863 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3a415e880 msgr2=0x7fd38c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:46.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.324+0000 7fd3a3fff640 1 --2- 172.21.15.98:0/2640926863 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3a415e880 0x7fd38c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:46.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.324+0000 7fd3a3fff640 1 -- 172.21.15.98:0/2640926863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd38c004810 msgr2=0x7fd38c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:46.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.324+0000 7fd3a3fff640 1 --2- 172.21.15.98:0/2640926863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd38c004810 0x7fd38c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:46.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.324+0000 7fd3a3fff640 1 -- 172.21.15.98:0/2640926863 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd38c003530 con 0x7fd38c0039c0 2024-09-19T08:44:46.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.324+0000 7fd3a8806640 1 --2- 172.21.15.98:0/2640926863 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3a415e880 0x7fd38c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:46.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.324+0000 7fd3a3fff640 1 --2- 172.21.15.98:0/2640926863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd38c0039c0 0x7fd38c003de0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fd394009ac0 tx=0x7fd39401ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73b0cf58eb61831 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:46.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.324+0000 7fd3a2ffd640 1 -- 172.21.15.98:0/2640926863 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd394009690 con 0x7fd38c0039c0 2024-09-19T08:44:46.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.324+0000 7fd3a2ffd640 1 -- 172.21.15.98:0/2640926863 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3940a6510 con 0x7fd38c0039c0 2024-09-19T08:44:46.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.324+0000 7fd3a2ffd640 1 -- 172.21.15.98:0/2640926863 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3940a68e0 con 0x7fd38c0039c0 2024-09-19T08:44:46.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.324+0000 7fd3aa269640 1 -- 172.21.15.98:0/2640926863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd38c0039c0 msgr2=0x7fd38c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:46.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.324+0000 7fd3aa269640 1 --2- 172.21.15.98:0/2640926863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd38c0039c0 0x7fd38c003de0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fd394009ac0 tx=0x7fd39401ca20 comp rx=0 tx=0).stop 2024-09-19T08:44:46.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.324+0000 7fd3a8806640 1 -- 172.21.15.98:0/2640926863 reap_dead start 2024-09-19T08:44:46.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.324+0000 7fd3aa269640 1 -- 172.21.15.98:0/2640926863 shutdown_connections 2024-09-19T08:44:46.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.324+0000 7fd3aa269640 1 -- 172.21.15.98:0/2640926863 >> 172.21.15.98:0/2640926863 conn(0x7fd3a40b2e60 msgr2=0x7fd3a40b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:46.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.324+0000 7fd3aa269640 1 -- 172.21.15.98:0/2640926863 shutdown_connections 2024-09-19T08:44:46.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.324+0000 7fd3aa269640 1 -- 172.21.15.98:0/2640926863 wait complete. 2024-09-19T08:44:46.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.328+0000 7fd3aa269640 1 Processor -- start 2024-09-19T08:44:46.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.328+0000 7fd3aa269640 1 -- start start 2024-09-19T08:44:46.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.328+0000 7fd3aa269640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd38c0039c0 0x7fd3a40cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:46.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.328+0000 7fd3aa269640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3a40cbf10 0x7fd3a41f6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:46.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.328+0000 7fd3aa269640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3a41f7440 0x7fd3a41f78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:46.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.328+0000 7fd3aa269640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a41698e0 con 0x7fd3a40cbf10 2024-09-19T08:44:46.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.328+0000 7fd3aa269640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a4169760 con 0x7fd38c0039c0 2024-09-19T08:44:46.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.328+0000 7fd3aa269640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd3a4169a60 con 0x7fd3a41f7440 2024-09-19T08:44:46.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.328+0000 7fd3a37fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3a40cbf10 0x7fd3a41f6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:46.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.328+0000 7fd3a37fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3a40cbf10 0x7fd3a41f6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:40732/0 (socket says 172.21.15.98:40732) 2024-09-19T08:44:46.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.328+0000 7fd3a37fe640 1 -- 172.21.15.98:0/4078017398 learned_addr learned my addr 172.21.15.98:0/4078017398 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:46.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.328+0000 7fd3a8806640 1 --2- 172.21.15.98:0/4078017398 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3a41f7440 0x7fd3a41f78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:46.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.328+0000 7fd3a37fe640 1 -- 172.21.15.98:0/4078017398 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3a41f7440 msgr2=0x7fd3a41f78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:46.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.328+0000 7fd3a37fe640 1 --2- 172.21.15.98:0/4078017398 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3a41f7440 0x7fd3a41f78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:46.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.328+0000 7fd3a37fe640 1 -- 172.21.15.98:0/4078017398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd38c0039c0 msgr2=0x7fd3a40cb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:46.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.328+0000 7fd3a37fe640 1 --2- 172.21.15.98:0/4078017398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd38c0039c0 0x7fd3a40cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:46.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.328+0000 7fd3a37fe640 1 -- 172.21.15.98:0/4078017398 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a40cc720 con 0x7fd3a40cbf10 2024-09-19T08:44:46.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.328+0000 7fd3a37fe640 1 --2- 172.21.15.98:0/4078017398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3a40cbf10 0x7fd3a41f6f00 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fd39800e760 tx=0x7fd39800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:46.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.328+0000 7fd3a0ff9640 1 -- 172.21.15.98:0/4078017398 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39800c6c0 con 0x7fd3a40cbf10 2024-09-19T08:44:46.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.328+0000 7fd3a0ff9640 1 -- 172.21.15.98:0/4078017398 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd398018070 con 0x7fd3a40cbf10 2024-09-19T08:44:46.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.328+0000 7fd3aa269640 1 -- 172.21.15.98:0/4078017398 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3a40cca00 con 0x7fd3a40cbf10 2024-09-19T08:44:46.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.328+0000 7fd3a0ff9640 1 -- 172.21.15.98:0/4078017398 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39800cac0 con 0x7fd3a40cbf10 2024-09-19T08:44:46.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.328+0000 7fd3aa269640 1 -- 172.21.15.98:0/4078017398 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3a40ccfc0 con 0x7fd3a40cbf10 2024-09-19T08:44:46.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.328+0000 7fd3aa269640 1 -- 172.21.15.98:0/4078017398 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3a4075080 con 0x7fd3a40cbf10 2024-09-19T08:44:46.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.328+0000 7fd3a0ff9640 1 -- 172.21.15.98:0/4078017398 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7fd39800cc60 con 0x7fd3a40cbf10 2024-09-19T08:44:46.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.328+0000 7fd3a0ff9640 1 --2- 172.21.15.98:0/4078017398 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd390067ce0 0x7fd39006a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:46.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.328+0000 7fd3a0ff9640 1 -- 172.21.15.98:0/4078017398 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5881+0+0 (secure 0 0 0) 0x7fd398086250 con 0x7fd3a40cbf10 2024-09-19T08:44:46.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.332+0000 7fd3a3fff640 1 --2- 172.21.15.98:0/4078017398 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd390067ce0 0x7fd39006a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:46.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.332+0000 7fd3a3fff640 1 --2- 172.21.15.98:0/4078017398 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd390067ce0 0x7fd39006a1a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd3940af000 tx=0x7fd394004740 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:46.336 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.336+0000 7fd3a0ff9640 1 -- 172.21.15.98:0/4078017398 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd398056700 con 0x7fd3a40cbf10 2024-09-19T08:44:46.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:46.496+0000 7fd3aa269640 1 -- 172.21.15.98:0/4078017398 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd3a40758a0 con 0x7fd3a40cbf10 2024-09-19T08:44:47.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.496+0000 7fd3a0ff9640 1 -- 172.21.15.98:0/4078017398 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v108) ==== 181+0+0 (secure 0 0 0) 0x7fd398056000 con 0x7fd3a40cbf10 2024-09-19T08:44:47.498 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_metadata' removed 2024-09-19T08:44:47.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.500+0000 7fd3aa269640 1 -- 172.21.15.98:0/4078017398 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd390067ce0 msgr2=0x7fd39006a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:47.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.500+0000 7fd3aa269640 1 --2- 172.21.15.98:0/4078017398 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd390067ce0 0x7fd39006a1a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd3940af000 tx=0x7fd394004740 comp rx=0 tx=0).stop 2024-09-19T08:44:47.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.500+0000 7fd3aa269640 1 -- 172.21.15.98:0/4078017398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3a40cbf10 msgr2=0x7fd3a41f6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:47.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.500+0000 7fd3aa269640 1 --2- 172.21.15.98:0/4078017398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3a40cbf10 0x7fd3a41f6f00 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fd39800e760 tx=0x7fd39800ec20 comp rx=0 tx=0).stop 2024-09-19T08:44:47.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.500+0000 7fd3aa269640 1 -- 172.21.15.98:0/4078017398 shutdown_connections 2024-09-19T08:44:47.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.500+0000 7fd3aa269640 1 --2- 172.21.15.98:0/4078017398 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd390067ce0 0x7fd39006a1a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:47.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.500+0000 7fd3aa269640 1 --2- 172.21.15.98:0/4078017398 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd3a41f7440 0x7fd3a41f78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:47.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.500+0000 7fd3aa269640 1 --2- 172.21.15.98:0/4078017398 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd3a40cbf10 0x7fd3a41f6f00 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:47.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.500+0000 7fd3aa269640 1 --2- 172.21.15.98:0/4078017398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd38c0039c0 0x7fd3a40cb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:47.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.500+0000 7fd3aa269640 1 -- 172.21.15.98:0/4078017398 >> 172.21.15.98:0/4078017398 conn(0x7fd3a40b2e60 msgr2=0x7fd3a40c8970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:47.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.500+0000 7fd3aa269640 1 -- 172.21.15.98:0/4078017398 shutdown_connections 2024-09-19T08:44:47.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.500+0000 7fd3aa269640 1 -- 172.21.15.98:0/4078017398 wait complete. 2024-09-19T08:44:47.526 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T08:44:47.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.660+0000 7f19b024b640 1 Processor -- start 2024-09-19T08:44:47.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.660+0000 7f19b024b640 1 -- start start 2024-09-19T08:44:47.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.660+0000 7f19b024b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f19a806dbd0 0x7f19a8073680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:47.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.660+0000 7f19b024b640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19a815baf0 con 0x7f19a815fac0 2024-09-19T08:44:47.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.660+0000 7f19b024b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19a815bdd0 con 0x7f19a815f6e0 2024-09-19T08:44:47.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.660+0000 7f19b024b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f19a815c8b0 con 0x7f19a806dbd0 2024-09-19T08:44:47.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.660+0000 7f19ae7e8640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f19a815fac0 0x7f19a80ce420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56474/0 (socket says 172.21.15.98:56474) 2024-09-19T08:44:47.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.660+0000 7f19ae7e8640 1 -- 172.21.15.98:0/2590800883 learned_addr learned my addr 172.21.15.98:0/2590800883 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:47.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.660+0000 7f19ad7e6640 1 --2- 172.21.15.98:0/2590800883 >> v2:172.21.15.139:3300/0 conn(0x7f19a806dbd0 0x7f19a8073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:47.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.660+0000 7f19acfe5640 1 -- 172.21.15.98:0/2590800883 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2073277400 0 0) 0x7f19a815baf0 con 0x7f19a815fac0 2024-09-19T08:44:47.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.660+0000 7f19acfe5640 1 -- 172.21.15.98:0/2590800883 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1994003530 con 0x7f19a815fac0 2024-09-19T08:44:47.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.660+0000 7f19acfe5640 1 -- 172.21.15.98:0/2590800883 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1519900361 0 0) 0x7f1994003530 con 0x7f19a815fac0 2024-09-19T08:44:47.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.660+0000 7f19acfe5640 1 -- 172.21.15.98:0/2590800883 >> v2:172.21.15.139:3300/0 conn(0x7f19a806dbd0 msgr2=0x7f19a8073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:47.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.660+0000 7f19acfe5640 1 --2- 172.21.15.98:0/2590800883 >> v2:172.21.15.139:3300/0 conn(0x7f19a806dbd0 0x7f19a8073680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:47.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.660+0000 7f19acfe5640 1 -- 172.21.15.98:0/2590800883 >> v1:172.21.15.139:6789/0 conn(0x7f19a815f6e0 legacy=0x7f19a806d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:47.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.660+0000 7f19acfe5640 1 -- 172.21.15.98:0/2590800883 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a815c0b0 con 0x7f19a815fac0 2024-09-19T08:44:47.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.660+0000 7f19acfe5640 1 -- 172.21.15.98:0/2590800883 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f19a0001a00 con 0x7f19a815fac0 2024-09-19T08:44:47.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.660+0000 7f19acfe5640 1 -- 172.21.15.98:0/2590800883 >> v1:172.21.15.98:6789/0 conn(0x7f19a815fac0 legacy=0x7f19a80ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:47.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.660+0000 7f19acfe5640 1 --2- 172.21.15.98:0/2590800883 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19940039c0 0x7f1994003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:47.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.660+0000 7f19acfe5640 1 --2- 172.21.15.98:0/2590800883 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1994004810 0x7f1994004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:47.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.660+0000 7f19acfe5640 1 --2- 172.21.15.98:0/2590800883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19940051d0 0x7f199400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:47.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.660+0000 7f19acfe5640 1 -- 172.21.15.98:0/2590800883 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f19a815c8b0 con 0x7f19940051d0 2024-09-19T08:44:47.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.660+0000 7f19acfe5640 1 -- 172.21.15.98:0/2590800883 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f199400c070 con 0x7f1994004810 2024-09-19T08:44:47.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.660+0000 7f19acfe5640 1 -- 172.21.15.98:0/2590800883 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f199400c270 con 0x7f19940039c0 2024-09-19T08:44:47.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.660+0000 7f19ae7e8640 1 --2- 172.21.15.98:0/2590800883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19940051d0 0x7f199400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:47.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.660+0000 7f19ad7e6640 1 --2- 172.21.15.98:0/2590800883 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1994004810 0x7f1994004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:47.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.660+0000 7f19adfe7640 1 --2- 172.21.15.98:0/2590800883 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19940039c0 0x7f1994003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:47.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.660+0000 7f19ae7e8640 1 -- 172.21.15.98:0/2590800883 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19940039c0 msgr2=0x7f1994003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:47.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.660+0000 7f19ae7e8640 1 --2- 172.21.15.98:0/2590800883 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19940039c0 0x7f1994003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:47.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.660+0000 7f19ae7e8640 1 -- 172.21.15.98:0/2590800883 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1994004810 msgr2=0x7f1994004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:47.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.660+0000 7f19ae7e8640 1 --2- 172.21.15.98:0/2590800883 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1994004810 0x7f1994004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:47.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.660+0000 7f19ae7e8640 1 -- 172.21.15.98:0/2590800883 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1994003530 con 0x7f19940051d0 2024-09-19T08:44:47.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.660+0000 7f19ad7e6640 1 --2- 172.21.15.98:0/2590800883 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1994004810 0x7f1994004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:44:47.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.664+0000 7f19ae7e8640 1 -- 172.21.15.98:0/2590800883 reap_dead start 2024-09-19T08:44:47.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.664+0000 7f19ae7e8640 1 --2- 172.21.15.98:0/2590800883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19940051d0 0x7f199400b9d0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f19a000a840 tx=0x7f19a001d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7b19abb1b87c861 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:47.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.664+0000 7f19acfe5640 1 -- 172.21.15.98:0/2590800883 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a000a460 con 0x7f19940051d0 2024-09-19T08:44:47.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.664+0000 7f19acfe5640 1 -- 172.21.15.98:0/2590800883 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19a00ac640 con 0x7f19940051d0 2024-09-19T08:44:47.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.664+0000 7f19acfe5640 1 -- 172.21.15.98:0/2590800883 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a00aca30 con 0x7f19940051d0 2024-09-19T08:44:47.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.664+0000 7f19b024b640 1 -- 172.21.15.98:0/2590800883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19940051d0 msgr2=0x7f199400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:47.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.664+0000 7f19b024b640 1 --2- 172.21.15.98:0/2590800883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19940051d0 0x7f199400b9d0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f19a000a840 tx=0x7f19a001d360 comp rx=0 tx=0).stop 2024-09-19T08:44:47.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.664+0000 7f19b024b640 1 -- 172.21.15.98:0/2590800883 shutdown_connections 2024-09-19T08:44:47.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.664+0000 7f19b024b640 1 --2- 172.21.15.98:0/2590800883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19940051d0 0x7f199400b9d0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:47.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.664+0000 7f19b024b640 1 -- 172.21.15.98:0/2590800883 >> 172.21.15.98:0/2590800883 conn(0x7f19a806c090 msgr2=0x7f19a806c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:47.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.664+0000 7f19b024b640 1 -- 172.21.15.98:0/2590800883 shutdown_connections 2024-09-19T08:44:47.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.664+0000 7f19b024b640 1 -- 172.21.15.98:0/2590800883 wait complete. 2024-09-19T08:44:47.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.668+0000 7f19b024b640 1 Processor -- start 2024-09-19T08:44:47.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.668+0000 7f19b024b640 1 -- start start 2024-09-19T08:44:47.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.668+0000 7f19b024b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19940051d0 0x7f19a81f14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:47.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.668+0000 7f19b024b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19a81f2d10 0x7f19a81f25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:47.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.668+0000 7f19b024b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19a8143c20 0x7f19a80b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:47.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.668+0000 7f19b024b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f19a806d4c0 con 0x7f19a81f2d10 2024-09-19T08:44:47.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.668+0000 7f19b024b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f19a815b890 con 0x7f19a8143c20 2024-09-19T08:44:47.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.668+0000 7f19b024b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f19a806d640 con 0x7f19940051d0 2024-09-19T08:44:47.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.668+0000 7f19ad7e6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19a81f2d10 0x7f19a81f25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:47.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.668+0000 7f19ad7e6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19a81f2d10 0x7f19a81f25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34520/0 (socket says 172.21.15.98:34520) 2024-09-19T08:44:47.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.668+0000 7f19ad7e6640 1 -- 172.21.15.98:0/1844217633 learned_addr learned my addr 172.21.15.98:0/1844217633 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:47.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.668+0000 7f19adfe7640 1 --2- 172.21.15.98:0/1844217633 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19940051d0 0x7f19a81f14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:47.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.668+0000 7f19ae7e8640 1 --2- 172.21.15.98:0/1844217633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19a8143c20 0x7f19a80b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:47.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.668+0000 7f19ad7e6640 1 -- 172.21.15.98:0/1844217633 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19940051d0 msgr2=0x7f19a81f14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:47.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.668+0000 7f19ad7e6640 1 --2- 172.21.15.98:0/1844217633 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19940051d0 0x7f19a81f14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:47.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.668+0000 7f19ad7e6640 1 -- 172.21.15.98:0/1844217633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19a8143c20 msgr2=0x7f19a80b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:47.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.668+0000 7f19ad7e6640 1 --2- 172.21.15.98:0/1844217633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19a8143c20 0x7f19a80b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:47.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.668+0000 7f19ad7e6640 1 -- 172.21.15.98:0/1844217633 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a81f2b10 con 0x7f19a81f2d10 2024-09-19T08:44:47.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.668+0000 7f19ad7e6640 1 --2- 172.21.15.98:0/1844217633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19a81f2d10 0x7f19a81f25d0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f1998011540 tx=0x7f1998011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:47.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.668+0000 7f199e7fc640 1 -- 172.21.15.98:0/1844217633 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1998004220 con 0x7f19a81f2d10 2024-09-19T08:44:47.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.668+0000 7f199e7fc640 1 -- 172.21.15.98:0/1844217633 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19980043c0 con 0x7f19a81f2d10 2024-09-19T08:44:47.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.668+0000 7f19b024b640 1 -- 172.21.15.98:0/1844217633 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19a81f32b0 con 0x7f19a81f2d10 2024-09-19T08:44:47.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.668+0000 7f199e7fc640 1 -- 172.21.15.98:0/1844217633 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1998015870 con 0x7f19a81f2d10 2024-09-19T08:44:47.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.668+0000 7f19b024b640 1 -- 172.21.15.98:0/1844217633 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19a81f3870 con 0x7f19a81f2d10 2024-09-19T08:44:47.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.668+0000 7f19b024b640 1 -- 172.21.15.98:0/1844217633 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19a8073e40 con 0x7f19a81f2d10 2024-09-19T08:44:47.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.668+0000 7f199e7fc640 1 -- 172.21.15.98:0/1844217633 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f1998015a10 con 0x7f19a81f2d10 2024-09-19T08:44:47.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.672+0000 7f199e7fc640 1 --2- 172.21.15.98:0/1844217633 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f198c067cb0 0x7f198c06a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:47.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.672+0000 7f199e7fc640 1 -- 172.21.15.98:0/1844217633 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5443+0+0 (secure 0 0 0) 0x7f1998086890 con 0x7f19a81f2d10 2024-09-19T08:44:47.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.672+0000 7f19adfe7640 1 --2- 172.21.15.98:0/1844217633 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f198c067cb0 0x7f198c06a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:47.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.672+0000 7f19adfe7640 1 --2- 172.21.15.98:0/1844217633 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f198c067cb0 0x7f198c06a170 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f19a4007fd0 tx=0x7f19a4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:47.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.676+0000 7f199e7fc640 1 -- 172.21.15.98:0/1844217633 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f19a81f2b10 con 0x7f19a81f2d10 2024-09-19T08:44:47.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:47.836+0000 7f19b024b640 1 -- 172.21.15.98:0/1844217633 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f19a815f280 con 0x7f19a81f2d10 2024-09-19T08:44:48.187 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:44:48.190 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:44:48.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.500+0000 7f199e7fc640 1 -- 172.21.15.98:0/1844217633 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v109) ==== 169+0+0 (secure 0 0 0) 0x7f1998019030 con 0x7f19a81f2d10 2024-09-19T08:44:48.501 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' removed 2024-09-19T08:44:48.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.500+0000 7f19b024b640 1 -- 172.21.15.98:0/1844217633 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f198c067cb0 msgr2=0x7f198c06a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:48.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.500+0000 7f19b024b640 1 --2- 172.21.15.98:0/1844217633 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f198c067cb0 0x7f198c06a170 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f19a4007fd0 tx=0x7f19a4009040 comp rx=0 tx=0).stop 2024-09-19T08:44:48.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.500+0000 7f19b024b640 1 -- 172.21.15.98:0/1844217633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19a81f2d10 msgr2=0x7f19a81f25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:48.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.500+0000 7f19b024b640 1 --2- 172.21.15.98:0/1844217633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19a81f2d10 0x7f19a81f25d0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f1998011540 tx=0x7f1998011a00 comp rx=0 tx=0).stop 2024-09-19T08:44:48.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.500+0000 7f19b024b640 1 -- 172.21.15.98:0/1844217633 shutdown_connections 2024-09-19T08:44:48.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.500+0000 7f19b024b640 1 --2- 172.21.15.98:0/1844217633 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f198c067cb0 0x7f198c06a170 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:48.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.500+0000 7f19b024b640 1 --2- 172.21.15.98:0/1844217633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19a8143c20 0x7f19a80b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:48.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.500+0000 7f19b024b640 1 --2- 172.21.15.98:0/1844217633 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19a81f2d10 0x7f19a81f25d0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:48.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.500+0000 7f19b024b640 1 --2- 172.21.15.98:0/1844217633 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19940051d0 0x7f19a81f14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:48.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.500+0000 7f19b024b640 1 -- 172.21.15.98:0/1844217633 >> 172.21.15.98:0/1844217633 conn(0x7f19a806c090 msgr2=0x7f19a8150910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:48.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.500+0000 7f19b024b640 1 -- 172.21.15.98:0/1844217633 shutdown_connections 2024-09-19T08:44:48.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.500+0000 7f19b024b640 1 -- 172.21.15.98:0/1844217633 wait complete. 2024-09-19T08:44:48.531 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-19T08:44:48.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.656+0000 7fd0dfcb0640 1 Processor -- start 2024-09-19T08:44:48.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.656+0000 7fd0dfcb0640 1 -- start start 2024-09-19T08:44:48.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.656+0000 7fd0dfcb0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0d806c3a0 0x7fd0d80713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:48.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.660+0000 7fd0dfcb0640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0d8073530 con 0x7fd0d8145b70 2024-09-19T08:44:48.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.660+0000 7fd0dfcb0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0d8073810 con 0x7fd0d80718f0 2024-09-19T08:44:48.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.660+0000 7fd0dfcb0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd0d8073af0 con 0x7fd0d806c3a0 2024-09-19T08:44:48.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.660+0000 7fd0dda4c640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fd0d8145b70 0x7fd0d806bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56480/0 (socket says 172.21.15.98:56480) 2024-09-19T08:44:48.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.660+0000 7fd0dda4c640 1 -- 172.21.15.98:0/2001004321 learned_addr learned my addr 172.21.15.98:0/2001004321 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:48.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.660+0000 7fd0dd24b640 1 --2- 172.21.15.98:0/2001004321 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0d806c3a0 0x7fd0d80713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:48.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.660+0000 7fd0dca4a640 1 -- 172.21.15.98:0/2001004321 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3272921486 0 0) 0x7fd0d8073530 con 0x7fd0d8145b70 2024-09-19T08:44:48.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.660+0000 7fd0dca4a640 1 -- 172.21.15.98:0/2001004321 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd0c4003530 con 0x7fd0d8145b70 2024-09-19T08:44:48.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.660+0000 7fd0dca4a640 1 -- 172.21.15.98:0/2001004321 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1933528905 0 0) 0x7fd0c4003530 con 0x7fd0d8145b70 2024-09-19T08:44:48.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.660+0000 7fd0dca4a640 1 -- 172.21.15.98:0/2001004321 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0d806c3a0 msgr2=0x7fd0d80713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:48.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.660+0000 7fd0dca4a640 1 --2- 172.21.15.98:0/2001004321 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0d806c3a0 0x7fd0d80713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:48.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.660+0000 7fd0dca4a640 1 -- 172.21.15.98:0/2001004321 >> v1:172.21.15.139:6789/0 conn(0x7fd0d80718f0 legacy=0x7fd0d80b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:48.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.660+0000 7fd0dca4a640 1 -- 172.21.15.98:0/2001004321 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d8073dd0 con 0x7fd0d8145b70 2024-09-19T08:44:48.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.660+0000 7fd0dca4a640 1 -- 172.21.15.98:0/2001004321 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fd0d4001a00 con 0x7fd0d8145b70 2024-09-19T08:44:48.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.660+0000 7fd0dca4a640 1 -- 172.21.15.98:0/2001004321 >> v1:172.21.15.98:6789/0 conn(0x7fd0d8145b70 legacy=0x7fd0d806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:48.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.660+0000 7fd0dca4a640 1 --2- 172.21.15.98:0/2001004321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0c40039f0 0x7fd0c4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:48.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.660+0000 7fd0dca4a640 1 --2- 172.21.15.98:0/2001004321 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0d806c3a0 0x7fd0c4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:48.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.660+0000 7fd0dca4a640 1 --2- 172.21.15.98:0/2001004321 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0c4004d50 0x7fd0c40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:48.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.660+0000 7fd0dca4a640 1 -- 172.21.15.98:0/2001004321 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d8073af0 con 0x7fd0c40039f0 2024-09-19T08:44:48.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.660+0000 7fd0dca4a640 1 -- 172.21.15.98:0/2001004321 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c400bb30 con 0x7fd0c4004d50 2024-09-19T08:44:48.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.660+0000 7fd0dca4a640 1 -- 172.21.15.98:0/2001004321 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd0c400bd30 con 0x7fd0d806c3a0 2024-09-19T08:44:48.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.660+0000 7fd0dda4c640 1 --2- 172.21.15.98:0/2001004321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0c40039f0 0x7fd0c4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:48.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.660+0000 7fd0dd24b640 1 --2- 172.21.15.98:0/2001004321 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0d806c3a0 0x7fd0c4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:48.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.660+0000 7fd0de24d640 1 --2- 172.21.15.98:0/2001004321 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0c4004d50 0x7fd0c40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:48.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.660+0000 7fd0dda4c640 1 -- 172.21.15.98:0/2001004321 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0d806c3a0 msgr2=0x7fd0c4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:48.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.660+0000 7fd0dda4c640 1 --2- 172.21.15.98:0/2001004321 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0d806c3a0 0x7fd0c4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:48.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.660+0000 7fd0dda4c640 1 -- 172.21.15.98:0/2001004321 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0c4004d50 msgr2=0x7fd0c40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:48.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.660+0000 7fd0dda4c640 1 --2- 172.21.15.98:0/2001004321 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0c4004d50 0x7fd0c40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:48.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.660+0000 7fd0dda4c640 1 -- 172.21.15.98:0/2001004321 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0c4003530 con 0x7fd0c40039f0 2024-09-19T08:44:48.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.660+0000 7fd0dda4c640 1 --2- 172.21.15.98:0/2001004321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0c40039f0 0x7fd0c4003e10 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fd0d400a830 tx=0x7fd0d401d2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a97cd005dc89eec7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:48.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.660+0000 7fd0dca4a640 1 -- 172.21.15.98:0/2001004321 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d400a480 con 0x7fd0c40039f0 2024-09-19T08:44:48.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.660+0000 7fd0dca4a640 1 -- 172.21.15.98:0/2001004321 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0d40ad650 con 0x7fd0c40039f0 2024-09-19T08:44:48.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.660+0000 7fd0dca4a640 1 -- 172.21.15.98:0/2001004321 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d40ada40 con 0x7fd0c40039f0 2024-09-19T08:44:48.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.664+0000 7fd0dfcb0640 1 -- 172.21.15.98:0/2001004321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0c40039f0 msgr2=0x7fd0c4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:48.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.664+0000 7fd0dfcb0640 1 --2- 172.21.15.98:0/2001004321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0c40039f0 0x7fd0c4003e10 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fd0d400a830 tx=0x7fd0d401d2a0 comp rx=0 tx=0).stop 2024-09-19T08:44:48.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.664+0000 7fd0de24d640 1 -- 172.21.15.98:0/2001004321 reap_dead start 2024-09-19T08:44:48.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.664+0000 7fd0dfcb0640 1 -- 172.21.15.98:0/2001004321 shutdown_connections 2024-09-19T08:44:48.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.664+0000 7fd0dfcb0640 1 -- 172.21.15.98:0/2001004321 >> 172.21.15.98:0/2001004321 conn(0x7fd0d80b2010 msgr2=0x7fd0d80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:48.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.664+0000 7fd0dfcb0640 1 -- 172.21.15.98:0/2001004321 shutdown_connections 2024-09-19T08:44:48.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.664+0000 7fd0dfcb0640 1 -- 172.21.15.98:0/2001004321 wait complete. 2024-09-19T08:44:48.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.664+0000 7fd0dfcb0640 1 Processor -- start 2024-09-19T08:44:48.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.664+0000 7fd0dfcb0640 1 -- start start 2024-09-19T08:44:48.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.664+0000 7fd0dfcb0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0c40039f0 0x7fd0d815b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:48.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.664+0000 7fd0dfcb0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0d813d4d0 0x7fd0d813ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:48.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.664+0000 7fd0dfcb0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0d813d950 0x7fd0d80ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:48.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.664+0000 7fd0dfcb0640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d80b46c0 con 0x7fd0c40039f0 2024-09-19T08:44:48.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.664+0000 7fd0dfcb0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d80b4540 con 0x7fd0d813d4d0 2024-09-19T08:44:48.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.664+0000 7fd0dfcb0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd0d8071ee0 con 0x7fd0d813d950 2024-09-19T08:44:48.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.664+0000 7fd0dda4c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0c40039f0 0x7fd0d815b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:48.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.664+0000 7fd0dda4c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0c40039f0 0x7fd0d815b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34540/0 (socket says 172.21.15.98:34540) 2024-09-19T08:44:48.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.664+0000 7fd0dda4c640 1 -- 172.21.15.98:0/3048595291 learned_addr learned my addr 172.21.15.98:0/3048595291 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:48.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.664+0000 7fd0de24d640 1 --2- 172.21.15.98:0/3048595291 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0d813d950 0x7fd0d80ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:48.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.664+0000 7fd0dd24b640 1 --2- 172.21.15.98:0/3048595291 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0d813d4d0 0x7fd0d813ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:48.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.664+0000 7fd0dda4c640 1 -- 172.21.15.98:0/3048595291 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0d813d950 msgr2=0x7fd0d80ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:48.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.664+0000 7fd0dda4c640 1 --2- 172.21.15.98:0/3048595291 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0d813d950 0x7fd0d80ccf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:48.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.664+0000 7fd0dda4c640 1 -- 172.21.15.98:0/3048595291 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0d813d4d0 msgr2=0x7fd0d813ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:48.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.664+0000 7fd0dda4c640 1 --2- 172.21.15.98:0/3048595291 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0d813d4d0 0x7fd0d813ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:48.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.664+0000 7fd0dda4c640 1 -- 172.21.15.98:0/3048595291 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d813c710 con 0x7fd0c40039f0 2024-09-19T08:44:48.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.664+0000 7fd0de24d640 1 --2- 172.21.15.98:0/3048595291 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0d813d950 0x7fd0d80ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:48.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.664+0000 7fd0dda4c640 1 --2- 172.21.15.98:0/3048595291 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0c40039f0 0x7fd0d815b620 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fd0d400a830 tx=0x7fd0c4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:48.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.664+0000 7fd0ce7fc640 1 -- 172.21.15.98:0/3048595291 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d40bb070 con 0x7fd0c40039f0 2024-09-19T08:44:48.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.664+0000 7fd0ce7fc640 1 -- 172.21.15.98:0/3048595291 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0d40017c0 con 0x7fd0c40039f0 2024-09-19T08:44:48.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.664+0000 7fd0dfcb0640 1 -- 172.21.15.98:0/3048595291 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0d813c9f0 con 0x7fd0c40039f0 2024-09-19T08:44:48.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.664+0000 7fd0ce7fc640 1 -- 172.21.15.98:0/3048595291 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4004b70 con 0x7fd0c40039f0 2024-09-19T08:44:48.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.664+0000 7fd0dfcb0640 1 -- 172.21.15.98:0/3048595291 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0d815c110 con 0x7fd0c40039f0 2024-09-19T08:44:48.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.668+0000 7fd0dfcb0640 1 -- 172.21.15.98:0/3048595291 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0d80b3410 con 0x7fd0c40039f0 2024-09-19T08:44:48.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.668+0000 7fd0ce7fc640 1 -- 172.21.15.98:0/3048595291 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7fd0d40be070 con 0x7fd0c40039f0 2024-09-19T08:44:48.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.668+0000 7fd0ce7fc640 1 --2- 172.21.15.98:0/3048595291 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd0bc067ce0 0x7fd0bc06a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:48.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.668+0000 7fd0ce7fc640 1 -- 172.21.15.98:0/3048595291 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4971+0+0 (secure 0 0 0) 0x7fd0d4120330 con 0x7fd0c40039f0 2024-09-19T08:44:48.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.668+0000 7fd0dd24b640 1 --2- 172.21.15.98:0/3048595291 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd0bc067ce0 0x7fd0bc06a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:48.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.668+0000 7fd0dd24b640 1 --2- 172.21.15.98:0/3048595291 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd0bc067ce0 0x7fd0bc06a1a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd0d813c5e0 tx=0x7fd0c8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:48.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.672+0000 7fd0ce7fc640 1 -- 172.21.15.98:0/3048595291 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd0d4016fb0 con 0x7fd0c40039f0 2024-09-19T08:44:48.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.832+0000 7fd0dfcb0640 1 -- 172.21.15.98:0/3048595291 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fd0d8142870 con 0x7fd0c40039f0 2024-09-19T08:44:48.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.836+0000 7fd0ce7fc640 1 -- 172.21.15.98:0/3048595291 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7fd0d40f0a70 con 0x7fd0c40039f0 2024-09-19T08:44:48.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.840+0000 7fd0dfcb0640 1 -- 172.21.15.98:0/3048595291 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd0bc067ce0 msgr2=0x7fd0bc06a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:48.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.840+0000 7fd0dfcb0640 1 --2- 172.21.15.98:0/3048595291 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd0bc067ce0 0x7fd0bc06a1a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd0d813c5e0 tx=0x7fd0c8008040 comp rx=0 tx=0).stop 2024-09-19T08:44:48.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.840+0000 7fd0dfcb0640 1 -- 172.21.15.98:0/3048595291 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0c40039f0 msgr2=0x7fd0d815b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:48.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.840+0000 7fd0dfcb0640 1 --2- 172.21.15.98:0/3048595291 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0c40039f0 0x7fd0d815b620 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fd0d400a830 tx=0x7fd0c4005840 comp rx=0 tx=0).stop 2024-09-19T08:44:48.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.840+0000 7fd0dfcb0640 1 -- 172.21.15.98:0/3048595291 shutdown_connections 2024-09-19T08:44:48.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.840+0000 7fd0dfcb0640 1 --2- 172.21.15.98:0/3048595291 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd0bc067ce0 0x7fd0bc06a1a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:48.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.840+0000 7fd0dfcb0640 1 --2- 172.21.15.98:0/3048595291 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0d813d950 0x7fd0d80ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:48.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.840+0000 7fd0dfcb0640 1 --2- 172.21.15.98:0/3048595291 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0d813d4d0 0x7fd0d813ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:48.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.840+0000 7fd0dfcb0640 1 --2- 172.21.15.98:0/3048595291 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0c40039f0 0x7fd0d815b620 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:48.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.840+0000 7fd0dfcb0640 1 -- 172.21.15.98:0/3048595291 >> 172.21.15.98:0/3048595291 conn(0x7fd0d80b2010 msgr2=0x7fd0d806bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:48.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.840+0000 7fd0dfcb0640 1 -- 172.21.15.98:0/3048595291 shutdown_connections 2024-09-19T08:44:48.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.840+0000 7fd0dfcb0640 1 -- 172.21.15.98:0/3048595291 wait complete. 2024-09-19T08:44:48.863 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity' 2024-09-19T08:44:48.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.968+0000 7f1dd377e640 1 Processor -- start 2024-09-19T08:44:48.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.968+0000 7f1dd377e640 1 -- start start 2024-09-19T08:44:48.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.972+0000 7f1dd377e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1dcc158e40 0x7f1dcc06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:48.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.972+0000 7f1dd377e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f1dcc06c1d0 0x7f1dcc0ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:48.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.972+0000 7f1dd377e640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dcc15fcf0 con 0x7f1dcc0ce6e0 2024-09-19T08:44:48.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.972+0000 7f1dd377e640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f1dcc06c640 con 0x7f1dcc06c1d0 2024-09-19T08:44:48.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.972+0000 7f1dd377e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f1dcc15ef30 con 0x7f1dcc158e40 2024-09-19T08:44:48.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.972+0000 7f1dd0d19640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f1dcc06c1d0 0x7f1dcc0ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:48.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.972+0000 7f1dd1d1b640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f1dcc0ce6e0 0x7f1dcc06d170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56482/0 (socket says 172.21.15.98:56482) 2024-09-19T08:44:48.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.972+0000 7f1dd1d1b640 1 -- 172.21.15.98:0/3045997232 learned_addr learned my addr 172.21.15.98:0/3045997232 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:48.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.972+0000 7f1dd0d19640 1 -- 172.21.15.98:0/3045997232 >> v2:172.21.15.139:3300/0 conn(0x7f1dcc158e40 msgr2=0x7f1dcc06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:48.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.972+0000 7f1dd0d19640 1 --2- 172.21.15.98:0/3045997232 >> v2:172.21.15.139:3300/0 conn(0x7f1dcc158e40 0x7f1dcc06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:48.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.972+0000 7f1dd0d19640 1 -- 172.21.15.98:0/3045997232 >> v1:172.21.15.98:6789/0 conn(0x7f1dcc0ce6e0 legacy=0x7f1dcc06d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:48.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.972+0000 7f1dd0d19640 1 -- 172.21.15.98:0/3045997232 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dcc15f210 con 0x7f1dcc06c1d0 2024-09-19T08:44:48.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.972+0000 7f1dd0d19640 1 --2- 172.21.15.98:0/3045997232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dcc06c1d0 0x7f1dcc0ce1a0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc06e1e0 tx=0x7f1dbc01ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ebaa470441a2fba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:48.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.972+0000 7f1dc3fff640 1 -- 172.21.15.98:0/3045997232 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc0b5070 con 0x7f1dcc06c1d0 2024-09-19T08:44:48.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.972+0000 7f1dc3fff640 1 -- 172.21.15.98:0/3045997232 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dbc0a8970 con 0x7f1dcc06c1d0 2024-09-19T08:44:48.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.972+0000 7f1dc3fff640 1 -- 172.21.15.98:0/3045997232 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc0a8d40 con 0x7f1dcc06c1d0 2024-09-19T08:44:48.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.976+0000 7f1dd377e640 1 -- 172.21.15.98:0/3045997232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dcc06c1d0 msgr2=0x7f1dcc0ce1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:48.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.976+0000 7f1dd377e640 1 --2- 172.21.15.98:0/3045997232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dcc06c1d0 0x7f1dcc0ce1a0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc06e1e0 tx=0x7f1dbc01ec70 comp rx=0 tx=0).stop 2024-09-19T08:44:48.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.976+0000 7f1dd377e640 1 -- 172.21.15.98:0/3045997232 shutdown_connections 2024-09-19T08:44:48.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.976+0000 7f1dd377e640 1 --2- 172.21.15.98:0/3045997232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dcc06c1d0 0x7f1dcc0ce1a0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:48.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.976+0000 7f1dd377e640 1 --2- 172.21.15.98:0/3045997232 >> v2:172.21.15.139:3300/0 conn(0x7f1dcc158e40 0x7f1dcc06bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:48.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.976+0000 7f1dd377e640 1 -- 172.21.15.98:0/3045997232 >> 172.21.15.98:0/3045997232 conn(0x7f1dcc0b2010 msgr2=0x7f1dcc0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:48.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.976+0000 7f1dd377e640 1 -- 172.21.15.98:0/3045997232 shutdown_connections 2024-09-19T08:44:48.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.976+0000 7f1dd377e640 1 -- 172.21.15.98:0/3045997232 wait complete. 2024-09-19T08:44:48.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.976+0000 7f1dd377e640 1 Processor -- start 2024-09-19T08:44:48.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.976+0000 7f1dd377e640 1 -- start start 2024-09-19T08:44:48.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.976+0000 7f1dd377e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1dcc06c1d0 0x7f1dcc1425b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:48.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.976+0000 7f1dd377e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1dcc0ce6e0 0x7f1dcc1547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:48.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.976+0000 7f1dd377e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dcc158e40 0x7f1dcc154d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:48.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.976+0000 7f1dd377e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1dcc159220 con 0x7f1dcc158e40 2024-09-19T08:44:48.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.976+0000 7f1dd377e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1dcc15f7d0 con 0x7f1dcc06c1d0 2024-09-19T08:44:48.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.976+0000 7f1dd377e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1dcc0cee20 con 0x7f1dcc0ce6e0 2024-09-19T08:44:48.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.976+0000 7f1dd1d1b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dcc158e40 0x7f1dcc154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:48.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.976+0000 7f1dd1d1b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dcc158e40 0x7f1dcc154d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34568/0 (socket says 172.21.15.98:34568) 2024-09-19T08:44:48.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.976+0000 7f1dd1d1b640 1 -- 172.21.15.98:0/3438770128 learned_addr learned my addr 172.21.15.98:0/3438770128 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:48.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.976+0000 7f1dd151a640 1 --2- 172.21.15.98:0/3438770128 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1dcc06c1d0 0x7f1dcc1425b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:48.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.976+0000 7f1dd1d1b640 1 -- 172.21.15.98:0/3438770128 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1dcc0ce6e0 msgr2=0x7f1dcc1547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:48.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.976+0000 7f1dd1d1b640 1 --2- 172.21.15.98:0/3438770128 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1dcc0ce6e0 0x7f1dcc1547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:48.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.976+0000 7f1dd1d1b640 1 -- 172.21.15.98:0/3438770128 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1dcc06c1d0 msgr2=0x7f1dcc1425b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:48.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.976+0000 7f1dd1d1b640 1 --2- 172.21.15.98:0/3438770128 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1dcc06c1d0 0x7f1dcc1425b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:48.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.976+0000 7f1dd1d1b640 1 -- 172.21.15.98:0/3438770128 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dcc071fc0 con 0x7f1dcc158e40 2024-09-19T08:44:48.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.976+0000 7f1dd1d1b640 1 --2- 172.21.15.98:0/3438770128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dcc158e40 0x7f1dcc154d10 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f1dc400cbc0 tx=0x7f1dc400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:48.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.976+0000 7f1dc1ffb640 1 -- 172.21.15.98:0/3438770128 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4017070 con 0x7f1dcc158e40 2024-09-19T08:44:48.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.976+0000 7f1dc1ffb640 1 -- 172.21.15.98:0/3438770128 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dc400ad40 con 0x7f1dcc158e40 2024-09-19T08:44:48.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.976+0000 7f1dc1ffb640 1 -- 172.21.15.98:0/3438770128 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4012600 con 0x7f1dcc158e40 2024-09-19T08:44:48.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.980+0000 7f1dd377e640 1 -- 172.21.15.98:0/3438770128 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dcc155320 con 0x7f1dcc158e40 2024-09-19T08:44:48.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.980+0000 7f1dd377e640 1 -- 172.21.15.98:0/3438770128 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dcc1558e0 con 0x7f1dcc158e40 2024-09-19T08:44:48.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.980+0000 7f1dd377e640 1 -- 172.21.15.98:0/3438770128 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dcc072190 con 0x7f1dcc158e40 2024-09-19T08:44:48.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.980+0000 7f1dc1ffb640 1 -- 172.21.15.98:0/3438770128 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f1dc401a070 con 0x7f1dcc158e40 2024-09-19T08:44:48.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.980+0000 7f1dc1ffb640 1 --2- 172.21.15.98:0/3438770128 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1db0067cb0 0x7f1db006a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:48.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.980+0000 7f1dc1ffb640 1 -- 172.21.15.98:0/3438770128 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4971+0+0 (secure 0 0 0) 0x7f1dc4085d50 con 0x7f1dcc158e40 2024-09-19T08:44:48.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.980+0000 7f1dd151a640 1 --2- 172.21.15.98:0/3438770128 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1db0067cb0 0x7f1db006a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:48.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.980+0000 7f1dd151a640 1 --2- 172.21.15.98:0/3438770128 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1db0067cb0 0x7f1db006a170 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc073c10 tx=0x7f1dc8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:48.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:48.984+0000 7f1dc1ffb640 1 -- 172.21.15.98:0/3438770128 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f1dc4055f40 con 0x7f1dcc158e40 2024-09-19T08:44:49.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.188+0000 7f1dd377e640 1 -- 172.21.15.98:0/3438770128 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]} v 0) -- 0x7f1dcc0ceb10 con 0x7f1dcc158e40 2024-09-19T08:44:49.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.512+0000 7f1dc1ffb640 1 -- 172.21.15.98:0/3438770128 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]}]=0 v336) ==== 149+0+0 (secure 0 0 0) 0x7f1dc405a240 con 0x7f1dcc158e40 2024-09-19T08:44:49.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.512+0000 7f1dd377e640 1 -- 172.21.15.98:0/3438770128 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1db0067cb0 msgr2=0x7f1db006a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:49.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.512+0000 7f1dd377e640 1 --2- 172.21.15.98:0/3438770128 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1db0067cb0 0x7f1db006a170 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc073c10 tx=0x7f1dc8008040 comp rx=0 tx=0).stop 2024-09-19T08:44:49.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.512+0000 7f1dd377e640 1 -- 172.21.15.98:0/3438770128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dcc158e40 msgr2=0x7f1dcc154d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:49.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.512+0000 7f1dd377e640 1 --2- 172.21.15.98:0/3438770128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dcc158e40 0x7f1dcc154d10 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f1dc400cbc0 tx=0x7f1dc400a430 comp rx=0 tx=0).stop 2024-09-19T08:44:49.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.512+0000 7f1dd377e640 1 -- 172.21.15.98:0/3438770128 shutdown_connections 2024-09-19T08:44:49.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.512+0000 7f1dd377e640 1 --2- 172.21.15.98:0/3438770128 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1db0067cb0 0x7f1db006a170 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.512+0000 7f1dd377e640 1 --2- 172.21.15.98:0/3438770128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dcc158e40 0x7f1dcc154d10 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.512+0000 7f1dd377e640 1 --2- 172.21.15.98:0/3438770128 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1dcc0ce6e0 0x7f1dcc1547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.512+0000 7f1dd377e640 1 --2- 172.21.15.98:0/3438770128 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1dcc06c1d0 0x7f1dcc1425b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.512+0000 7f1dd377e640 1 -- 172.21.15.98:0/3438770128 >> 172.21.15.98:0/3438770128 conn(0x7f1dcc0b2010 msgr2=0x7f1dcc15ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:49.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.512+0000 7f1dd377e640 1 -- 172.21.15.98:0/3438770128 shutdown_connections 2024-09-19T08:44:49.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.512+0000 7f1dd377e640 1 -- 172.21.15.98:0/3438770128 wait complete. 2024-09-19T08:44:49.536 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2024-09-19T08:44:49.537 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_no_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2024-09-19T08:44:49.537 INFO:tasks.cephfs_test_runner:test_subvolume_no_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) 2024-09-19T08:44:49.537 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:44:49.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.664+0000 7f33c2195640 1 Processor -- start 2024-09-19T08:44:49.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.664+0000 7f33c2195640 1 -- start start 2024-09-19T08:44:49.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.664+0000 7f33c2195640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f33bc158fd0 0x7f33bc0b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:49.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.664+0000 7f33c2195640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33bc0b4360 0x7f33bc06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:49.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.664+0000 7f33c2195640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33bc0ccc60 con 0x7f33bc06c1d0 2024-09-19T08:44:49.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.664+0000 7f33c2195640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f33bc1593b0 con 0x7f33bc158fd0 2024-09-19T08:44:49.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.664+0000 7f33c2195640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f33bc06c600 con 0x7f33bc0b4360 2024-09-19T08:44:49.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.664+0000 7f33bb7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f33bc158fd0 0x7f33bc0b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:49.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.664+0000 7f33bbfff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f33bc06c1d0 0x7f33bc15efc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:36272/0 (socket says 172.21.15.98:36272) 2024-09-19T08:44:49.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.664+0000 7f33bbfff640 1 -- 172.21.15.98:0/3410105956 learned_addr learned my addr 172.21.15.98:0/3410105956 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:49.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.664+0000 7f33baffd640 1 --2- 172.21.15.98:0/3410105956 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33bc0b4360 0x7f33bc06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:49.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.664+0000 7f33bb7fe640 1 -- 172.21.15.98:0/3410105956 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33bc0b4360 msgr2=0x7f33bc06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:49.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.664+0000 7f33bb7fe640 1 --2- 172.21.15.98:0/3410105956 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33bc0b4360 0x7f33bc06bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.664+0000 7f33bb7fe640 1 -- 172.21.15.98:0/3410105956 >> v1:172.21.15.139:6789/0 conn(0x7f33bc06c1d0 legacy=0x7f33bc15efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:49.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.664+0000 7f33bb7fe640 1 -- 172.21.15.98:0/3410105956 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33bc0ccf40 con 0x7f33bc158fd0 2024-09-19T08:44:49.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.668+0000 7f33bb7fe640 1 --2- 172.21.15.98:0/3410105956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33bc158fd0 0x7f33bc0b3e20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f33b40097c0 tx=0x7f33b401c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b86ec8cd00c9bca1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:49.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.668+0000 7f33ba7fc640 1 -- 172.21.15.98:0/3410105956 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b40b2070 con 0x7f33bc158fd0 2024-09-19T08:44:49.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.668+0000 7f33ba7fc640 1 -- 172.21.15.98:0/3410105956 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33b40a9070 con 0x7f33bc158fd0 2024-09-19T08:44:49.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.668+0000 7f33ba7fc640 1 -- 172.21.15.98:0/3410105956 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b40ad880 con 0x7f33bc158fd0 2024-09-19T08:44:49.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.668+0000 7f33c2195640 1 -- 172.21.15.98:0/3410105956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33bc158fd0 msgr2=0x7f33bc0b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:49.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.668+0000 7f33c2195640 1 --2- 172.21.15.98:0/3410105956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33bc158fd0 0x7f33bc0b3e20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f33b40097c0 tx=0x7f33b401c3a0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.668+0000 7f33c2195640 1 -- 172.21.15.98:0/3410105956 shutdown_connections 2024-09-19T08:44:49.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.668+0000 7f33c2195640 1 --2- 172.21.15.98:0/3410105956 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33bc0b4360 0x7f33bc06bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.668+0000 7f33c2195640 1 --2- 172.21.15.98:0/3410105956 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33bc158fd0 0x7f33bc0b3e20 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.668+0000 7f33c2195640 1 -- 172.21.15.98:0/3410105956 >> 172.21.15.98:0/3410105956 conn(0x7f33bc0b2010 msgr2=0x7f33bc0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:49.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.668+0000 7f33c2195640 1 -- 172.21.15.98:0/3410105956 shutdown_connections 2024-09-19T08:44:49.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.668+0000 7f33c2195640 1 -- 172.21.15.98:0/3410105956 wait complete. 2024-09-19T08:44:49.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.672+0000 7f33c2195640 1 Processor -- start 2024-09-19T08:44:49.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.672+0000 7f33c2195640 1 -- start start 2024-09-19T08:44:49.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.672+0000 7f33c2195640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33bc06c1d0 0x7f33bc1540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:49.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.672+0000 7f33c2195640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33bc0b4360 0x7f33bc1579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:49.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.672+0000 7f33c2195640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33bc158fd0 0x7f33bc154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:49.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.672+0000 7f33c2195640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc15c920 con 0x7f33bc06c1d0 2024-09-19T08:44:49.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.672+0000 7f33c2195640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc15c7a0 con 0x7f33bc0b4360 2024-09-19T08:44:49.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.672+0000 7f33c2195640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f33bc166100 con 0x7f33bc158fd0 2024-09-19T08:44:49.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.672+0000 7f33bb7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33bc06c1d0 0x7f33bc1540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:49.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.672+0000 7f33bb7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33bc06c1d0 0x7f33bc1540a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34582/0 (socket says 172.21.15.98:34582) 2024-09-19T08:44:49.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.672+0000 7f33bb7fe640 1 -- 172.21.15.98:0/3718440145 learned_addr learned my addr 172.21.15.98:0/3718440145 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:49.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.672+0000 7f33baffd640 1 --2- 172.21.15.98:0/3718440145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33bc0b4360 0x7f33bc1579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:49.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.672+0000 7f33bbfff640 1 --2- 172.21.15.98:0/3718440145 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33bc158fd0 0x7f33bc154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:49.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.672+0000 7f33bb7fe640 1 -- 172.21.15.98:0/3718440145 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33bc158fd0 msgr2=0x7f33bc154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:49.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.672+0000 7f33bb7fe640 1 --2- 172.21.15.98:0/3718440145 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33bc158fd0 0x7f33bc154af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.672+0000 7f33bb7fe640 1 -- 172.21.15.98:0/3718440145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33bc0b4360 msgr2=0x7f33bc1579d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:49.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.672+0000 7f33bb7fe640 1 --2- 172.21.15.98:0/3718440145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33bc0b4360 0x7f33bc1579d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.672+0000 7f33bb7fe640 1 -- 172.21.15.98:0/3718440145 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33bc1545e0 con 0x7f33bc06c1d0 2024-09-19T08:44:49.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.672+0000 7f33bbfff640 1 --2- 172.21.15.98:0/3718440145 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33bc158fd0 0x7f33bc154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:49.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.672+0000 7f33bb7fe640 1 --2- 172.21.15.98:0/3718440145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33bc06c1d0 0x7f33bc1540a0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f33b4019fd0 tx=0x7f33b4019140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:49.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.672+0000 7f33b3fff640 1 -- 172.21.15.98:0/3718440145 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b40bb070 con 0x7f33bc06c1d0 2024-09-19T08:44:49.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.672+0000 7f33b3fff640 1 -- 172.21.15.98:0/3718440145 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33b40b7480 con 0x7f33bc06c1d0 2024-09-19T08:44:49.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.672+0000 7f33b3fff640 1 -- 172.21.15.98:0/3718440145 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b40b2040 con 0x7f33bc06c1d0 2024-09-19T08:44:49.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.672+0000 7f33c2195640 1 -- 172.21.15.98:0/3718440145 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33bc155660 con 0x7f33bc06c1d0 2024-09-19T08:44:49.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.672+0000 7f33c2195640 1 -- 172.21.15.98:0/3718440145 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33bc1571d0 con 0x7f33bc06c1d0 2024-09-19T08:44:49.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.672+0000 7f33c2195640 1 -- 172.21.15.98:0/3718440145 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33bc06bc90 con 0x7f33bc06c1d0 2024-09-19T08:44:49.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.672+0000 7f33b3fff640 1 -- 172.21.15.98:0/3718440145 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f33b40be070 con 0x7f33bc06c1d0 2024-09-19T08:44:49.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.672+0000 7f33b3fff640 1 --2- 172.21.15.98:0/3718440145 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f339c067cb0 0x7f339c06a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:49.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.676+0000 7f33b3fff640 1 -- 172.21.15.98:0/3718440145 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4971+0+0 (secure 0 0 0) 0x7f33b4120420 con 0x7f33bc06c1d0 2024-09-19T08:44:49.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.676+0000 7f33baffd640 1 --2- 172.21.15.98:0/3718440145 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f339c067cb0 0x7f339c06a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:49.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.676+0000 7f33baffd640 1 --2- 172.21.15.98:0/3718440145 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f339c067cb0 0x7f339c06a170 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f33a4003fd0 tx=0x7f33a40039a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:49.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.680+0000 7f33b3fff640 1 -- 172.21.15.98:0/3718440145 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f33b40c5030 con 0x7f33bc06c1d0 2024-09-19T08:44:49.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.836+0000 7f33c2195640 1 -- 172.21.15.98:0/3718440145 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f33bc15b9d0 con 0x7f33bc06c1d0 2024-09-19T08:44:49.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.836+0000 7f33b3fff640 1 -- 172.21.15.98:0/3718440145 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+13084 (secure 0 0 0) 0x7f33b4016050 con 0x7f33bc06c1d0 2024-09-19T08:44:49.839 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:44:49.839 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":109,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:44:48.500839+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.98:6835/273091170":"2024-09-20T08:44:45.121221+0000","172.21.15.98:6834/273091170":"2024-09-20T08:44:45.121221+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:44:49.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.840+0000 7f33c2195640 1 -- 172.21.15.98:0/3718440145 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f339c067cb0 msgr2=0x7f339c06a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:49.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.840+0000 7f33c2195640 1 --2- 172.21.15.98:0/3718440145 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f339c067cb0 0x7f339c06a170 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f33a4003fd0 tx=0x7f33a40039a0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.840+0000 7f33c2195640 1 -- 172.21.15.98:0/3718440145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33bc06c1d0 msgr2=0x7f33bc1540a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:49.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.840+0000 7f33c2195640 1 --2- 172.21.15.98:0/3718440145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33bc06c1d0 0x7f33bc1540a0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f33b4019fd0 tx=0x7f33b4019140 comp rx=0 tx=0).stop 2024-09-19T08:44:49.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.840+0000 7f33c2195640 1 -- 172.21.15.98:0/3718440145 shutdown_connections 2024-09-19T08:44:49.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.840+0000 7f33c2195640 1 --2- 172.21.15.98:0/3718440145 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f339c067cb0 0x7f339c06a170 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.840+0000 7f33c2195640 1 --2- 172.21.15.98:0/3718440145 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33bc158fd0 0x7f33bc154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.840+0000 7f33c2195640 1 --2- 172.21.15.98:0/3718440145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33bc0b4360 0x7f33bc1579d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.840+0000 7f33c2195640 1 --2- 172.21.15.98:0/3718440145 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33bc06c1d0 0x7f33bc1540a0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.840+0000 7f33c2195640 1 -- 172.21.15.98:0/3718440145 >> 172.21.15.98:0/3718440145 conn(0x7f33bc0b2010 msgr2=0x7f33bc0b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:49.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.840+0000 7f33c2195640 1 -- 172.21.15.98:0/3718440145 shutdown_connections 2024-09-19T08:44:49.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.840+0000 7f33c2195640 1 -- 172.21.15.98:0/3718440145 wait complete. 2024-09-19T08:44:49.858 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T08:38:08.105684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T08:44:49.858 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T08:44:49.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.968+0000 7fb16ab75640 1 Processor -- start 2024-09-19T08:44:49.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.968+0000 7fb16ab75640 1 -- start start 2024-09-19T08:44:49.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.968+0000 7fb16ab75640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb16415e640 0x7fb1640b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:49.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.968+0000 7fb16ab75640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb1640d2e50 0x7fb16406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:49.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.968+0000 7fb16ab75640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb16406c1d0 0x7fb1640b5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:49.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.968+0000 7fb16ab75640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fb16415ea20 con 0x7fb1640d2e50 2024-09-19T08:44:49.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.968+0000 7fb16ab75640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fb16406c620 con 0x7fb16415e640 2024-09-19T08:44:49.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.968+0000 7fb16ab75640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb1640cfea0 con 0x7fb16406c1d0 2024-09-19T08:44:49.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.968+0000 7fb163fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb1640d2e50 0x7fb16406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:49.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.968+0000 7fb163fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb1640d2e50 0x7fb16406bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34596/0 (socket says 172.21.15.98:34596) 2024-09-19T08:44:49.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.968+0000 7fb163fff640 1 -- 172.21.15.98:0/874497342 learned_addr learned my addr 172.21.15.98:0/874497342 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:49.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.968+0000 7fb168911640 1 --2- 172.21.15.98:0/874497342 >> v2:172.21.15.139:3300/0 conn(0x7fb16415e640 0x7fb1640b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:49.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.968+0000 7fb169112640 1 --2- 172.21.15.98:0/874497342 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb16406c1d0 0x7fb1640b5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:49.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.968+0000 7fb163fff640 1 -- 172.21.15.98:0/874497342 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb16406c1d0 msgr2=0x7fb1640b5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:49.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.968+0000 7fb163fff640 1 --2- 172.21.15.98:0/874497342 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb16406c1d0 0x7fb1640b5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.968+0000 7fb163fff640 1 -- 172.21.15.98:0/874497342 >> v2:172.21.15.139:3300/0 conn(0x7fb16415e640 msgr2=0x7fb1640b35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:49.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.968+0000 7fb163fff640 1 --2- 172.21.15.98:0/874497342 >> v2:172.21.15.139:3300/0 conn(0x7fb16415e640 0x7fb1640b35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.968+0000 7fb163fff640 1 -- 172.21.15.98:0/874497342 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1640d0140 con 0x7fb1640d2e50 2024-09-19T08:44:49.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.968+0000 7fb169112640 1 --2- 172.21.15.98:0/874497342 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb16406c1d0 0x7fb1640b5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:49.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.972+0000 7fb163fff640 1 --2- 172.21.15.98:0/874497342 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1640d2e50 0x7fb16406bc90 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fb164069a50 tx=0x7fb15801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6eea6c7a07b4de server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:49.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.972+0000 7fb1637fe640 1 -- 172.21.15.98:0/874497342 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1580b5070 con 0x7fb1640d2e50 2024-09-19T08:44:49.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.972+0000 7fb1637fe640 1 -- 172.21.15.98:0/874497342 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1580a89d0 con 0x7fb1640d2e50 2024-09-19T08:44:49.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.972+0000 7fb1637fe640 1 -- 172.21.15.98:0/874497342 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1580a8da0 con 0x7fb1640d2e50 2024-09-19T08:44:49.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.972+0000 7fb16ab75640 1 -- 172.21.15.98:0/874497342 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1640d2e50 msgr2=0x7fb16406bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:49.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.972+0000 7fb16ab75640 1 --2- 172.21.15.98:0/874497342 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1640d2e50 0x7fb16406bc90 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fb164069a50 tx=0x7fb15801ed10 comp rx=0 tx=0).stop 2024-09-19T08:44:49.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.972+0000 7fb16ab75640 1 -- 172.21.15.98:0/874497342 shutdown_connections 2024-09-19T08:44:49.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.972+0000 7fb16ab75640 1 --2- 172.21.15.98:0/874497342 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1640d2e50 0x7fb16406bc90 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.972+0000 7fb16ab75640 1 --2- 172.21.15.98:0/874497342 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb16406c1d0 0x7fb1640b5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.972+0000 7fb16ab75640 1 --2- 172.21.15.98:0/874497342 >> v2:172.21.15.139:3300/0 conn(0x7fb16415e640 0x7fb1640b35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.972+0000 7fb16ab75640 1 -- 172.21.15.98:0/874497342 >> 172.21.15.98:0/874497342 conn(0x7fb1640d0fe0 msgr2=0x7fb1640d13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:49.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.972+0000 7fb16ab75640 1 -- 172.21.15.98:0/874497342 shutdown_connections 2024-09-19T08:44:49.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.972+0000 7fb16ab75640 1 -- 172.21.15.98:0/874497342 wait complete. 2024-09-19T08:44:49.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.976+0000 7fb16ab75640 1 Processor -- start 2024-09-19T08:44:49.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.976+0000 7fb16ab75640 1 -- start start 2024-09-19T08:44:49.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.976+0000 7fb16ab75640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb16406c1d0 0x7fb1640cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:49.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.976+0000 7fb16ab75640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1640d2e50 0x7fb1641f7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:49.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.976+0000 7fb16ab75640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb16415e640 0x7fb164159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:49.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.976+0000 7fb16ab75640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb1640d09e0 con 0x7fb16406c1d0 2024-09-19T08:44:49.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.976+0000 7fb16ab75640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb1640d0700 con 0x7fb1640d2e50 2024-09-19T08:44:49.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.976+0000 7fb16ab75640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb1640b3b00 con 0x7fb16415e640 2024-09-19T08:44:49.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.976+0000 7fb168911640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb16406c1d0 0x7fb1640cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:49.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.976+0000 7fb168911640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb16406c1d0 0x7fb1640cdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34602/0 (socket says 172.21.15.98:34602) 2024-09-19T08:44:49.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.976+0000 7fb168911640 1 -- 172.21.15.98:0/654031051 learned_addr learned my addr 172.21.15.98:0/654031051 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:49.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.976+0000 7fb163fff640 1 --2- 172.21.15.98:0/654031051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1640d2e50 0x7fb1641f7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:49.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.976+0000 7fb168911640 1 -- 172.21.15.98:0/654031051 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb16415e640 msgr2=0x7fb164159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:49.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.976+0000 7fb168911640 1 --2- 172.21.15.98:0/654031051 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb16415e640 0x7fb164159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.976+0000 7fb168911640 1 -- 172.21.15.98:0/654031051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1640d2e50 msgr2=0x7fb1641f7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:49.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.976+0000 7fb168911640 1 --2- 172.21.15.98:0/654031051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1640d2e50 0x7fb1641f7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:49.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.976+0000 7fb168911640 1 -- 172.21.15.98:0/654031051 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1640ce440 con 0x7fb16406c1d0 2024-09-19T08:44:49.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.976+0000 7fb163fff640 1 --2- 172.21.15.98:0/654031051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1640d2e50 0x7fb1641f7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:49.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.976+0000 7fb168911640 1 --2- 172.21.15.98:0/654031051 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb16406c1d0 0x7fb1640cdc10 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fb15400d800 tx=0x7fb15400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:49.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.976+0000 7fb1617fa640 1 -- 172.21.15.98:0/654031051 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15400b6c0 con 0x7fb16406c1d0 2024-09-19T08:44:49.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.976+0000 7fb1617fa640 1 -- 172.21.15.98:0/654031051 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb154019070 con 0x7fb16406c1d0 2024-09-19T08:44:49.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.976+0000 7fb16ab75640 1 -- 172.21.15.98:0/654031051 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1640ce720 con 0x7fb16406c1d0 2024-09-19T08:44:49.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.976+0000 7fb1617fa640 1 -- 172.21.15.98:0/654031051 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15400ba60 con 0x7fb16406c1d0 2024-09-19T08:44:49.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.976+0000 7fb16ab75640 1 -- 172.21.15.98:0/654031051 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1641f75e0 con 0x7fb16406c1d0 2024-09-19T08:44:49.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.976+0000 7fb16ab75640 1 -- 172.21.15.98:0/654031051 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb16406bc90 con 0x7fb16406c1d0 2024-09-19T08:44:49.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.976+0000 7fb1617fa640 1 -- 172.21.15.98:0/654031051 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7fb15400bc00 con 0x7fb16406c1d0 2024-09-19T08:44:49.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.976+0000 7fb1617fa640 1 --2- 172.21.15.98:0/654031051 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb150067ce0 0x7fb15006a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:49.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.976+0000 7fb1617fa640 1 -- 172.21.15.98:0/654031051 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4971+0+0 (secure 0 0 0) 0x7fb154086500 con 0x7fb16406c1d0 2024-09-19T08:44:49.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.976+0000 7fb163fff640 1 --2- 172.21.15.98:0/654031051 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb150067ce0 0x7fb15006a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:49.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.980+0000 7fb163fff640 1 --2- 172.21.15.98:0/654031051 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb150067ce0 0x7fb15006a1a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb1580b2000 tx=0x7fb158008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:49.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:49.980+0000 7fb1617fa640 1 -- 172.21.15.98:0/654031051 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb154056dd0 con 0x7fb16406c1d0 2024-09-19T08:44:50.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.136+0000 7fb16ab75640 1 -- 172.21.15.98:0/654031051 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb1640b92c0 con 0x7fb16406c1d0 2024-09-19T08:44:50.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.136+0000 7fb1617fa640 1 -- 172.21.15.98:0/654031051 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) ==== 93+0+10 (secure 0 0 0) 0x7fb1540566d0 con 0x7fb16406c1d0 2024-09-19T08:44:50.138 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2024-09-19T08:44:50.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.136+0000 7fb16ab75640 1 -- 172.21.15.98:0/654031051 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb150067ce0 msgr2=0x7fb15006a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:50.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.136+0000 7fb16ab75640 1 --2- 172.21.15.98:0/654031051 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb150067ce0 0x7fb15006a1a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb1580b2000 tx=0x7fb158008040 comp rx=0 tx=0).stop 2024-09-19T08:44:50.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.136+0000 7fb16ab75640 1 -- 172.21.15.98:0/654031051 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb16406c1d0 msgr2=0x7fb1640cdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:50.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.136+0000 7fb16ab75640 1 --2- 172.21.15.98:0/654031051 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb16406c1d0 0x7fb1640cdc10 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fb15400d800 tx=0x7fb15400dcc0 comp rx=0 tx=0).stop 2024-09-19T08:44:50.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.136+0000 7fb16ab75640 1 -- 172.21.15.98:0/654031051 shutdown_connections 2024-09-19T08:44:50.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.136+0000 7fb16ab75640 1 --2- 172.21.15.98:0/654031051 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb150067ce0 0x7fb15006a1a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:50.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.136+0000 7fb16ab75640 1 --2- 172.21.15.98:0/654031051 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb16415e640 0x7fb164159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:50.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.136+0000 7fb16ab75640 1 --2- 172.21.15.98:0/654031051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1640d2e50 0x7fb1641f7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:50.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.136+0000 7fb16ab75640 1 --2- 172.21.15.98:0/654031051 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb16406c1d0 0x7fb1640cdc10 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:50.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.136+0000 7fb16ab75640 1 -- 172.21.15.98:0/654031051 >> 172.21.15.98:0/654031051 conn(0x7fb1640d0fe0 msgr2=0x7fb1640b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:50.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.136+0000 7fb16ab75640 1 -- 172.21.15.98:0/654031051 shutdown_connections 2024-09-19T08:44:50.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.136+0000 7fb16ab75640 1 -- 172.21.15.98:0/654031051 wait complete. 2024-09-19T08:44:50.155 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2' 2024-09-19T08:44:50.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.264+0000 7f3ef53c5640 1 Processor -- start 2024-09-19T08:44:50.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.264+0000 7f3ef53c5640 1 -- start start 2024-09-19T08:44:50.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.264+0000 7f3ef53c5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3ef006f380 0x7f3ef00ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:50.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.264+0000 7f3ef53c5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3ef00cec60 0x7f3ef00aee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:50.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.264+0000 7f3ef53c5640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ef00af4a0 con 0x7f3ef006efa0 2024-09-19T08:44:50.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.264+0000 7f3ef53c5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f3ef015ed00 con 0x7f3ef00cec60 2024-09-19T08:44:50.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.264+0000 7f3ef53c5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3ef00af780 con 0x7f3ef006f380 2024-09-19T08:44:50.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.264+0000 7f3eeeffd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f3ef006efa0 0x7f3ef00ce550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56486/0 (socket says 172.21.15.98:56486) 2024-09-19T08:44:50.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.264+0000 7f3eeeffd640 1 -- 172.21.15.98:0/74337448 learned_addr learned my addr 172.21.15.98:0/74337448 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:50.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.264+0000 7f3eedffb640 1 -- 172.21.15.98:0/74337448 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3441735973 0 0) 0x7f3ef00af4a0 con 0x7f3ef006efa0 2024-09-19T08:44:50.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.264+0000 7f3eee7fc640 1 --2- 172.21.15.98:0/74337448 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3ef006f380 0x7f3ef00ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:50.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.264+0000 7f3eef7fe640 1 --2- 172.21.15.98:0/74337448 >> v2:172.21.15.139:3300/0 conn(0x7f3ef00cec60 0x7f3ef00aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:50.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.264+0000 7f3eedffb640 1 -- 172.21.15.98:0/74337448 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3edc003530 con 0x7f3ef006efa0 2024-09-19T08:44:50.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.264+0000 7f3eedffb640 1 -- 172.21.15.98:0/74337448 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 910127183 0 0) 0x7f3edc003530 con 0x7f3ef006efa0 2024-09-19T08:44:50.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.264+0000 7f3eedffb640 1 -- 172.21.15.98:0/74337448 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3ef006f380 msgr2=0x7f3ef00ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:50.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.264+0000 7f3eedffb640 1 --2- 172.21.15.98:0/74337448 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3ef006f380 0x7f3ef00ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:50.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.264+0000 7f3eedffb640 1 -- 172.21.15.98:0/74337448 >> v2:172.21.15.139:3300/0 conn(0x7f3ef00cec60 msgr2=0x7f3ef00aee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:50.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.264+0000 7f3eedffb640 1 --2- 172.21.15.98:0/74337448 >> v2:172.21.15.139:3300/0 conn(0x7f3ef00cec60 0x7f3ef00aee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:50.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.264+0000 7f3eedffb640 1 -- 172.21.15.98:0/74337448 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef00afa60 con 0x7f3ef006efa0 2024-09-19T08:44:50.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.264+0000 7f3eedffb640 1 -- 172.21.15.98:0/74337448 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f3ee0001a00 con 0x7f3ef006efa0 2024-09-19T08:44:50.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.264+0000 7f3eef7fe640 1 --2- 172.21.15.98:0/74337448 >> v2:172.21.15.139:3300/0 conn(0x7f3ef00cec60 0x7f3ef00aee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:44:50.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.264+0000 7f3eedffb640 1 -- 172.21.15.98:0/74337448 >> v1:172.21.15.98:6789/0 conn(0x7f3ef006efa0 legacy=0x7f3ef00ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:50.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.264+0000 7f3eedffb640 1 --2- 172.21.15.98:0/74337448 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3edc0039c0 0x7f3edc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:50.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.264+0000 7f3eedffb640 1 --2- 172.21.15.98:0/74337448 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3edc004810 0x7f3edc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:50.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.264+0000 7f3eedffb640 1 --2- 172.21.15.98:0/74337448 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3edc0051d0 0x7f3edc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:50.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.264+0000 7f3eedffb640 1 -- 172.21.15.98:0/74337448 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef015ed00 con 0x7f3edc004810 2024-09-19T08:44:50.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.264+0000 7f3eedffb640 1 -- 172.21.15.98:0/74337448 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef00af780 con 0x7f3edc0039c0 2024-09-19T08:44:50.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.264+0000 7f3eedffb640 1 -- 172.21.15.98:0/74337448 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3edc00bf10 con 0x7f3edc0051d0 2024-09-19T08:44:50.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.264+0000 7f3eee7fc640 1 --2- 172.21.15.98:0/74337448 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3edc004810 0x7f3edc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:50.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.264+0000 7f3eeeffd640 1 --2- 172.21.15.98:0/74337448 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3edc0039c0 0x7f3edc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:50.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.264+0000 7f3eee7fc640 1 -- 172.21.15.98:0/74337448 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3edc0051d0 msgr2=0x7f3edc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:50.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.264+0000 7f3eee7fc640 1 --2- 172.21.15.98:0/74337448 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3edc0051d0 0x7f3edc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:50.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.264+0000 7f3eee7fc640 1 -- 172.21.15.98:0/74337448 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3edc0039c0 msgr2=0x7f3edc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:50.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.264+0000 7f3eee7fc640 1 --2- 172.21.15.98:0/74337448 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3edc0039c0 0x7f3edc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:50.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.264+0000 7f3eee7fc640 1 -- 172.21.15.98:0/74337448 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3edc003530 con 0x7f3edc004810 2024-09-19T08:44:50.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.264+0000 7f3eeeffd640 1 --2- 172.21.15.98:0/74337448 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3edc0039c0 0x7f3edc003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:50.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.268+0000 7f3eee7fc640 1 --2- 172.21.15.98:0/74337448 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3edc004810 0x7f3edc004c90 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8009af0 tx=0x7f3ed801ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61149c289bdf8908 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:50.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.268+0000 7f3eedffb640 1 -- 172.21.15.98:0/74337448 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed8009690 con 0x7f3edc004810 2024-09-19T08:44:50.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.268+0000 7f3eedffb640 1 -- 172.21.15.98:0/74337448 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ed80a6450 con 0x7f3edc004810 2024-09-19T08:44:50.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.268+0000 7f3eedffb640 1 -- 172.21.15.98:0/74337448 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed80a6820 con 0x7f3edc004810 2024-09-19T08:44:50.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.268+0000 7f3ef53c5640 1 -- 172.21.15.98:0/74337448 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3edc004810 msgr2=0x7f3edc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:50.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.268+0000 7f3ef53c5640 1 --2- 172.21.15.98:0/74337448 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3edc004810 0x7f3edc004c90 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8009af0 tx=0x7f3ed801ca10 comp rx=0 tx=0).stop 2024-09-19T08:44:50.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.268+0000 7f3eef7fe640 1 -- 172.21.15.98:0/74337448 reap_dead start 2024-09-19T08:44:50.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.268+0000 7f3ef53c5640 1 -- 172.21.15.98:0/74337448 shutdown_connections 2024-09-19T08:44:50.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.268+0000 7f3ef53c5640 1 -- 172.21.15.98:0/74337448 >> 172.21.15.98:0/74337448 conn(0x7f3ef00d0980 msgr2=0x7f3ef00d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:50.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.268+0000 7f3ef53c5640 1 -- 172.21.15.98:0/74337448 shutdown_connections 2024-09-19T08:44:50.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.268+0000 7f3ef53c5640 1 -- 172.21.15.98:0/74337448 wait complete. 2024-09-19T08:44:50.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.268+0000 7f3ef53c5640 1 Processor -- start 2024-09-19T08:44:50.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.268+0000 7f3ef53c5640 1 -- start start 2024-09-19T08:44:50.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.268+0000 7f3ef53c5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3edc004810 0x7f3ef00b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:50.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.268+0000 7f3ef53c5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3ef00b1770 0x7f3ef01658e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:50.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.268+0000 7f3ef53c5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ef00b1bd0 0x7f3ef00c4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:50.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.268+0000 7f3ef53c5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef015c180 con 0x7f3ef00b1bd0 2024-09-19T08:44:50.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.268+0000 7f3ef53c5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef015c000 con 0x7f3edc004810 2024-09-19T08:44:50.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.268+0000 7f3ef53c5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3ef015c300 con 0x7f3ef00b1770 2024-09-19T08:44:50.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.268+0000 7f3eef7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ef00b1bd0 0x7f3ef00c4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:50.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.268+0000 7f3eef7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ef00b1bd0 0x7f3ef00c4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34620/0 (socket says 172.21.15.98:34620) 2024-09-19T08:44:50.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.268+0000 7f3eef7fe640 1 -- 172.21.15.98:0/1747116678 learned_addr learned my addr 172.21.15.98:0/1747116678 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:50.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.268+0000 7f3eeeffd640 1 --2- 172.21.15.98:0/1747116678 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3edc004810 0x7f3ef00b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:50.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.268+0000 7f3eee7fc640 1 --2- 172.21.15.98:0/1747116678 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3ef00b1770 0x7f3ef01658e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:50.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.268+0000 7f3eef7fe640 1 -- 172.21.15.98:0/1747116678 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3ef00b1770 msgr2=0x7f3ef01658e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:50.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.268+0000 7f3eef7fe640 1 --2- 172.21.15.98:0/1747116678 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3ef00b1770 0x7f3ef01658e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:50.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.268+0000 7f3eef7fe640 1 -- 172.21.15.98:0/1747116678 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3edc004810 msgr2=0x7f3ef00b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:50.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.268+0000 7f3eef7fe640 1 --2- 172.21.15.98:0/1747116678 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3edc004810 0x7f3ef00b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:50.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.268+0000 7f3eef7fe640 1 -- 172.21.15.98:0/1747116678 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef00b3d90 con 0x7f3ef00b1bd0 2024-09-19T08:44:50.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.268+0000 7f3eee7fc640 1 --2- 172.21.15.98:0/1747116678 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3ef00b1770 0x7f3ef01658e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:50.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.272+0000 7f3eef7fe640 1 --2- 172.21.15.98:0/1747116678 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ef00b1bd0 0x7f3ef00c4140 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f3ef015e9c0 tx=0x7f3ee800e830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:50.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.272+0000 7f3ee77fe640 1 -- 172.21.15.98:0/1747116678 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee800f040 con 0x7f3ef00b1bd0 2024-09-19T08:44:50.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.272+0000 7f3ee77fe640 1 -- 172.21.15.98:0/1747116678 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ee8004020 con 0x7f3ef00b1bd0 2024-09-19T08:44:50.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.272+0000 7f3ef53c5640 1 -- 172.21.15.98:0/1747116678 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ef0164840 con 0x7f3ef00b1bd0 2024-09-19T08:44:50.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.272+0000 7f3ee77fe640 1 -- 172.21.15.98:0/1747116678 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee80043f0 con 0x7f3ef00b1bd0 2024-09-19T08:44:50.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.272+0000 7f3ef53c5640 1 -- 172.21.15.98:0/1747116678 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ef0164e00 con 0x7f3ef00b1bd0 2024-09-19T08:44:50.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.272+0000 7f3ef53c5640 1 -- 172.21.15.98:0/1747116678 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ef006f380 con 0x7f3ef00b1bd0 2024-09-19T08:44:50.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.272+0000 7f3ee77fe640 1 -- 172.21.15.98:0/1747116678 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f3ee801b020 con 0x7f3ef00b1bd0 2024-09-19T08:44:50.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.272+0000 7f3ee77fe640 1 --2- 172.21.15.98:0/1747116678 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3ed0067ce0 0x7f3ed006a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:50.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.272+0000 7f3ee77fe640 1 -- 172.21.15.98:0/1747116678 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4971+0+0 (secure 0 0 0) 0x7f3ee80852d0 con 0x7f3ef00b1bd0 2024-09-19T08:44:50.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.272+0000 7f3eeeffd640 1 --2- 172.21.15.98:0/1747116678 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3ed0067ce0 0x7f3ed006a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:50.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.272+0000 7f3eeeffd640 1 --2- 172.21.15.98:0/1747116678 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3ed0067ce0 0x7f3ed006a1a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0001bf0 tx=0x7f3ee0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:50.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.276+0000 7f3ee77fe640 1 -- 172.21.15.98:0/1747116678 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3ee8055a70 con 0x7f3ef00b1bd0 2024-09-19T08:44:50.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.480+0000 7f3ef53c5640 1 -- 172.21.15.98:0/1747116678 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]} v 0) -- 0x7f3ef006fee0 con 0x7f3ef00b1bd0 2024-09-19T08:44:50.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.512+0000 7f3ee77fe640 1 -- 172.21.15.98:0/1747116678 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]}]=0 v337) ==== 151+0+0 (secure 0 0 0) 0x7f3ee8055370 con 0x7f3ef00b1bd0 2024-09-19T08:44:50.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.516+0000 7f3ef53c5640 1 -- 172.21.15.98:0/1747116678 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3ed0067ce0 msgr2=0x7f3ed006a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:50.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.516+0000 7f3ef53c5640 1 --2- 172.21.15.98:0/1747116678 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3ed0067ce0 0x7f3ed006a1a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3ee0001bf0 tx=0x7f3ee0009040 comp rx=0 tx=0).stop 2024-09-19T08:44:50.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.516+0000 7f3ef53c5640 1 -- 172.21.15.98:0/1747116678 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ef00b1bd0 msgr2=0x7f3ef00c4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:50.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.516+0000 7f3ef53c5640 1 --2- 172.21.15.98:0/1747116678 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ef00b1bd0 0x7f3ef00c4140 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f3ef015e9c0 tx=0x7f3ee800e830 comp rx=0 tx=0).stop 2024-09-19T08:44:50.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.516+0000 7f3ef53c5640 1 -- 172.21.15.98:0/1747116678 shutdown_connections 2024-09-19T08:44:50.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.516+0000 7f3ef53c5640 1 --2- 172.21.15.98:0/1747116678 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3ed0067ce0 0x7f3ed006a1a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:50.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.516+0000 7f3ef53c5640 1 --2- 172.21.15.98:0/1747116678 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3ef00b1bd0 0x7f3ef00c4140 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:50.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.516+0000 7f3ef53c5640 1 --2- 172.21.15.98:0/1747116678 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3ef00b1770 0x7f3ef01658e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:50.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.516+0000 7f3ef53c5640 1 --2- 172.21.15.98:0/1747116678 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3edc004810 0x7f3ef00b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:50.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.516+0000 7f3ef53c5640 1 -- 172.21.15.98:0/1747116678 >> 172.21.15.98:0/1747116678 conn(0x7f3ef00d0980 msgr2=0x7f3ef015d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:50.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.516+0000 7f3ef53c5640 1 -- 172.21.15.98:0/1747116678 shutdown_connections 2024-09-19T08:44:50.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.516+0000 7f3ef53c5640 1 -- 172.21.15.98:0/1747116678 wait complete. 2024-09-19T08:44:50.533 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T08:44:50.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.644+0000 7f4d2291d640 1 Processor -- start 2024-09-19T08:44:50.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.644+0000 7f4d2291d640 1 -- start start 2024-09-19T08:44:50.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.644+0000 7f4d2291d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4d1c145b70 0x7f4d1c06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:50.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.644+0000 7f4d2291d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d1c071ac0 0x7f4d1c0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:50.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.644+0000 7f4d2291d640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d1c073530 con 0x7f4d1c06c1d0 2024-09-19T08:44:50.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.644+0000 7f4d2291d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f4d1c06c620 con 0x7f4d1c145b70 2024-09-19T08:44:50.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.644+0000 7f4d2291d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4d1c073810 con 0x7f4d1c071ac0 2024-09-19T08:44:50.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.644+0000 7f4d13fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f4d1c06c1d0 0x7f4d1c0713b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56498/0 (socket says 172.21.15.98:56498) 2024-09-19T08:44:50.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.644+0000 7f4d13fff640 1 -- 172.21.15.98:0/2506514323 learned_addr learned my addr 172.21.15.98:0/2506514323 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:50.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.644+0000 7f4d1bfff640 1 --2- 172.21.15.98:0/2506514323 >> v2:172.21.15.139:3300/0 conn(0x7f4d1c145b70 0x7f4d1c06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:50.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.644+0000 7f4d20eba640 1 --2- 172.21.15.98:0/2506514323 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d1c071ac0 0x7f4d1c0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:50.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.644+0000 7f4d1b7fe640 1 -- 172.21.15.98:0/2506514323 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1302488480 0 0) 0x7f4d1c073530 con 0x7f4d1c06c1d0 2024-09-19T08:44:50.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.644+0000 7f4d1b7fe640 1 -- 172.21.15.98:0/2506514323 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d08003530 con 0x7f4d1c06c1d0 2024-09-19T08:44:50.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.644+0000 7f4d1b7fe640 1 -- 172.21.15.98:0/2506514323 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4184145778 0 0) 0x7f4d08003530 con 0x7f4d1c06c1d0 2024-09-19T08:44:50.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.644+0000 7f4d1b7fe640 1 -- 172.21.15.98:0/2506514323 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d1c071ac0 msgr2=0x7f4d1c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:50.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.644+0000 7f4d1b7fe640 1 --2- 172.21.15.98:0/2506514323 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d1c071ac0 0x7f4d1c0b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:50.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.644+0000 7f4d1b7fe640 1 -- 172.21.15.98:0/2506514323 >> v2:172.21.15.139:3300/0 conn(0x7f4d1c145b70 msgr2=0x7f4d1c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:50.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.644+0000 7f4d1b7fe640 1 --2- 172.21.15.98:0/2506514323 >> v2:172.21.15.139:3300/0 conn(0x7f4d1c145b70 0x7f4d1c06bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:50.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.644+0000 7f4d1b7fe640 1 -- 172.21.15.98:0/2506514323 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d1c073af0 con 0x7f4d1c06c1d0 2024-09-19T08:44:50.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.644+0000 7f4d1b7fe640 1 -- 172.21.15.98:0/2506514323 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f4d0c001680 con 0x7f4d1c06c1d0 2024-09-19T08:44:50.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.644+0000 7f4d1b7fe640 1 -- 172.21.15.98:0/2506514323 >> v1:172.21.15.98:6789/0 conn(0x7f4d1c06c1d0 legacy=0x7f4d1c0713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:50.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.644+0000 7f4d1b7fe640 1 --2- 172.21.15.98:0/2506514323 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d1c071ac0 0x7f4d080039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:50.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.644+0000 7f4d1b7fe640 1 --2- 172.21.15.98:0/2506514323 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d08003f40 0x7f4d08004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:50.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.644+0000 7f4d1b7fe640 1 --2- 172.21.15.98:0/2506514323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d08004d50 0x7f4d080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:50.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.644+0000 7f4d1b7fe640 1 -- 172.21.15.98:0/2506514323 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c06c620 con 0x7f4d08004d50 2024-09-19T08:44:50.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.644+0000 7f4d1b7fe640 1 -- 172.21.15.98:0/2506514323 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c073810 con 0x7f4d08003f40 2024-09-19T08:44:50.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.644+0000 7f4d1b7fe640 1 -- 172.21.15.98:0/2506514323 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4d0800bad0 con 0x7f4d1c071ac0 2024-09-19T08:44:50.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.644+0000 7f4d20eba640 1 --2- 172.21.15.98:0/2506514323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d08004d50 0x7f4d080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:50.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.644+0000 7f4d1bfff640 1 --2- 172.21.15.98:0/2506514323 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d1c071ac0 0x7f4d080039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:50.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.644+0000 7f4d13fff640 1 --2- 172.21.15.98:0/2506514323 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d08003f40 0x7f4d08004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:50.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.644+0000 7f4d20eba640 1 -- 172.21.15.98:0/2506514323 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d1c071ac0 msgr2=0x7f4d080039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:50.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.644+0000 7f4d20eba640 1 --2- 172.21.15.98:0/2506514323 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d1c071ac0 0x7f4d080039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:50.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.644+0000 7f4d20eba640 1 -- 172.21.15.98:0/2506514323 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d08003f40 msgr2=0x7f4d08004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:50.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.644+0000 7f4d20eba640 1 --2- 172.21.15.98:0/2506514323 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d08003f40 0x7f4d08004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:50.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.644+0000 7f4d20eba640 1 -- 172.21.15.98:0/2506514323 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d08003530 con 0x7f4d08004d50 2024-09-19T08:44:50.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.644+0000 7f4d13fff640 1 --2- 172.21.15.98:0/2506514323 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d08003f40 0x7f4d08004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:50.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.648+0000 7f4d20eba640 1 --2- 172.21.15.98:0/2506514323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d08004d50 0x7f4d080051d0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f4d1400e170 tx=0x7f4d1401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac793928e34d3ef5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:50.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.648+0000 7f4d1b7fe640 1 -- 172.21.15.98:0/2506514323 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d140b2070 con 0x7f4d08004d50 2024-09-19T08:44:50.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.648+0000 7f4d1b7fe640 1 -- 172.21.15.98:0/2506514323 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d140a5c70 con 0x7f4d08004d50 2024-09-19T08:44:50.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.648+0000 7f4d1b7fe640 1 -- 172.21.15.98:0/2506514323 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d14009c10 con 0x7f4d08004d50 2024-09-19T08:44:50.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.648+0000 7f4d2291d640 1 -- 172.21.15.98:0/2506514323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d08004d50 msgr2=0x7f4d080051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:50.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.648+0000 7f4d2291d640 1 --2- 172.21.15.98:0/2506514323 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d08004d50 0x7f4d080051d0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f4d1400e170 tx=0x7f4d1401cc60 comp rx=0 tx=0).stop 2024-09-19T08:44:50.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.648+0000 7f4d20eba640 1 -- 172.21.15.98:0/2506514323 reap_dead start 2024-09-19T08:44:50.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.648+0000 7f4d2291d640 1 -- 172.21.15.98:0/2506514323 shutdown_connections 2024-09-19T08:44:50.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.648+0000 7f4d2291d640 1 -- 172.21.15.98:0/2506514323 >> 172.21.15.98:0/2506514323 conn(0x7f4d1c0b2010 msgr2=0x7f4d1c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:50.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.648+0000 7f4d2291d640 1 -- 172.21.15.98:0/2506514323 shutdown_connections 2024-09-19T08:44:50.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.648+0000 7f4d2291d640 1 -- 172.21.15.98:0/2506514323 wait complete. 2024-09-19T08:44:50.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.648+0000 7f4d2291d640 1 Processor -- start 2024-09-19T08:44:50.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.648+0000 7f4d2291d640 1 -- start start 2024-09-19T08:44:50.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.648+0000 7f4d2291d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d08004d50 0x7f4d1c15bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:50.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.648+0000 7f4d2291d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d1c0ac5f0 0x7f4d1c0aca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:50.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.648+0000 7f4d2291d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d1c1446f0 0x7f4d1c144b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:50.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.648+0000 7f4d2291d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c1460d0 con 0x7f4d1c1446f0 2024-09-19T08:44:50.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.648+0000 7f4d2291d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c145f50 con 0x7f4d08004d50 2024-09-19T08:44:50.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.648+0000 7f4d2291d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4d1c0b3410 con 0x7f4d1c0ac5f0 2024-09-19T08:44:50.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.648+0000 7f4d20eba640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d1c1446f0 0x7f4d1c144b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:50.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.648+0000 7f4d20eba640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d1c1446f0 0x7f4d1c144b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34630/0 (socket says 172.21.15.98:34630) 2024-09-19T08:44:50.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.648+0000 7f4d20eba640 1 -- 172.21.15.98:0/130625778 learned_addr learned my addr 172.21.15.98:0/130625778 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:50.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.652+0000 7f4d20eba640 1 -- 172.21.15.98:0/130625778 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d1c0ac5f0 msgr2=0x7f4d1c0aca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:50.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.652+0000 7f4d1bfff640 1 --2- 172.21.15.98:0/130625778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d08004d50 0x7f4d1c15bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:50.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.652+0000 7f4d13fff640 1 --2- 172.21.15.98:0/130625778 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d1c0ac5f0 0x7f4d1c0aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:50.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.652+0000 7f4d20eba640 1 --2- 172.21.15.98:0/130625778 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d1c0ac5f0 0x7f4d1c0aca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:50.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.652+0000 7f4d20eba640 1 -- 172.21.15.98:0/130625778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d08004d50 msgr2=0x7f4d1c15bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:50.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.652+0000 7f4d20eba640 1 --2- 172.21.15.98:0/130625778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d08004d50 0x7f4d1c15bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:50.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.652+0000 7f4d20eba640 1 -- 172.21.15.98:0/130625778 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d1c0c5580 con 0x7f4d1c1446f0 2024-09-19T08:44:50.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.652+0000 7f4d1bfff640 1 --2- 172.21.15.98:0/130625778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d08004d50 0x7f4d1c15bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:50.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.652+0000 7f4d20eba640 1 --2- 172.21.15.98:0/130625778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d1c1446f0 0x7f4d1c144b70 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009f90 tx=0x7f4d14009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:50.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.652+0000 7f4d197fa640 1 -- 172.21.15.98:0/130625778 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d140b2040 con 0x7f4d1c1446f0 2024-09-19T08:44:50.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.652+0000 7f4d197fa640 1 -- 172.21.15.98:0/130625778 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d140a5a70 con 0x7f4d1c1446f0 2024-09-19T08:44:50.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.652+0000 7f4d2291d640 1 -- 172.21.15.98:0/130625778 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d1c0ad160 con 0x7f4d1c1446f0 2024-09-19T08:44:50.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.652+0000 7f4d197fa640 1 -- 172.21.15.98:0/130625778 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d14009c10 con 0x7f4d1c1446f0 2024-09-19T08:44:50.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.652+0000 7f4d2291d640 1 -- 172.21.15.98:0/130625778 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d1c0ad720 con 0x7f4d1c1446f0 2024-09-19T08:44:50.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.652+0000 7f4d2291d640 1 -- 172.21.15.98:0/130625778 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d1c1450b0 con 0x7f4d1c1446f0 2024-09-19T08:44:50.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.652+0000 7f4d197fa640 1 -- 172.21.15.98:0/130625778 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f4d140be020 con 0x7f4d1c1446f0 2024-09-19T08:44:50.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.652+0000 7f4d197fa640 1 --2- 172.21.15.98:0/130625778 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4cfc067cb0 0x7f4cfc06a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:50.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.652+0000 7f4d197fa640 1 -- 172.21.15.98:0/130625778 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4971+0+0 (secure 0 0 0) 0x7f4d1411ff10 con 0x7f4d1c1446f0 2024-09-19T08:44:50.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.652+0000 7f4d1bfff640 1 --2- 172.21.15.98:0/130625778 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4cfc067cb0 0x7f4cfc06a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:50.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.652+0000 7f4d1bfff640 1 --2- 172.21.15.98:0/130625778 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4cfc067cb0 0x7f4cfc06a170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4d04009940 tx=0x7f4d04008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:50.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.656+0000 7f4d197fa640 1 -- 172.21.15.98:0/130625778 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f4d140c5020 con 0x7f4d1c1446f0 2024-09-19T08:44:50.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.808+0000 7f4d2291d640 1 -- 172.21.15.98:0/130625778 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f4d1c073320 con 0x7f4d1c1446f0 2024-09-19T08:44:50.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.812+0000 7f4d197fa640 1 -- 172.21.15.98:0/130625778 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7f4d140a5e20 con 0x7f4d1c1446f0 2024-09-19T08:44:50.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.816+0000 7f4d2291d640 1 -- 172.21.15.98:0/130625778 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4cfc067cb0 msgr2=0x7f4cfc06a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:50.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.816+0000 7f4d2291d640 1 --2- 172.21.15.98:0/130625778 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4cfc067cb0 0x7f4cfc06a170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4d04009940 tx=0x7f4d04008040 comp rx=0 tx=0).stop 2024-09-19T08:44:50.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.816+0000 7f4d2291d640 1 -- 172.21.15.98:0/130625778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d1c1446f0 msgr2=0x7f4d1c144b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:50.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.816+0000 7f4d2291d640 1 --2- 172.21.15.98:0/130625778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d1c1446f0 0x7f4d1c144b70 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009f90 tx=0x7f4d14009750 comp rx=0 tx=0).stop 2024-09-19T08:44:50.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.816+0000 7f4d2291d640 1 -- 172.21.15.98:0/130625778 shutdown_connections 2024-09-19T08:44:50.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.816+0000 7f4d2291d640 1 --2- 172.21.15.98:0/130625778 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4cfc067cb0 0x7f4cfc06a170 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:50.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.816+0000 7f4d2291d640 1 --2- 172.21.15.98:0/130625778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4d1c1446f0 0x7f4d1c144b70 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:50.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.816+0000 7f4d2291d640 1 --2- 172.21.15.98:0/130625778 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4d1c0ac5f0 0x7f4d1c0aca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:50.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.816+0000 7f4d2291d640 1 --2- 172.21.15.98:0/130625778 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4d08004d50 0x7f4d1c15bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:50.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.816+0000 7f4d2291d640 1 -- 172.21.15.98:0/130625778 >> 172.21.15.98:0/130625778 conn(0x7f4d1c0b2010 msgr2=0x7f4d1c1407b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:50.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.816+0000 7f4d2291d640 1 -- 172.21.15.98:0/130625778 shutdown_connections 2024-09-19T08:44:50.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:50.816+0000 7f4d2291d640 1 -- 172.21.15.98:0/130625778 wait complete. 2024-09-19T08:44:50.833 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T08:44:50.834 DEBUG:teuthology.orchestra.run.smithi098:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:44:50.909 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:44:50.985 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T08:44:50.985 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:44:50.985 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:44:50.991 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15846 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T08:44:50.991 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:44:50.992 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:44:50.997 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T08:44:50.998 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:44:50.998 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:44:51.040 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15846 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T08:44:51.041 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:44:51.041 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:44:51.046 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T08:44:51.046 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-19T08:44:51.047 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:44:51.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.204+0000 7f633f63d640 1 Processor -- start 2024-09-19T08:44:51.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.204+0000 7f633f63d640 1 -- start start 2024-09-19T08:44:51.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.208+0000 7f633f63d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f633815c550 0x7f633806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:51.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.208+0000 7f633f63d640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f633806d310 con 0x7f63380cfad0 2024-09-19T08:44:51.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.208+0000 7f633f63d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f633806d5f0 con 0x7f633806c1d0 2024-09-19T08:44:51.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.208+0000 7f633f63d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f633806c610 con 0x7f633815c550 2024-09-19T08:44:51.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.208+0000 7f633dbda640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f63380cfad0 0x7f63380723a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56514/0 (socket says 172.21.15.98:56514) 2024-09-19T08:44:51.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.208+0000 7f633dbda640 1 -- 172.21.15.98:0/207501563 learned_addr learned my addr 172.21.15.98:0/207501563 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:51.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.208+0000 7f632ffff640 1 -- 172.21.15.98:0/207501563 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2392598093 0 0) 0x7f633806d310 con 0x7f63380cfad0 2024-09-19T08:44:51.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.208+0000 7f633d3d9640 1 --2- 172.21.15.98:0/207501563 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f633815c550 0x7f633806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:51.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.208+0000 7f632ffff640 1 -- 172.21.15.98:0/207501563 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6324003530 con 0x7f63380cfad0 2024-09-19T08:44:51.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.208+0000 7f632ffff640 1 -- 172.21.15.98:0/207501563 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 191663682 0 0) 0x7f6324003530 con 0x7f63380cfad0 2024-09-19T08:44:51.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.208+0000 7f632ffff640 1 -- 172.21.15.98:0/207501563 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f633815c550 msgr2=0x7f633806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:51.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.208+0000 7f632ffff640 1 --2- 172.21.15.98:0/207501563 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f633815c550 0x7f633806bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:51.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.208+0000 7f632ffff640 1 -- 172.21.15.98:0/207501563 >> v1:172.21.15.139:6789/0 conn(0x7f633806c1d0 legacy=0x7f63380cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:51.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.208+0000 7f632ffff640 1 -- 172.21.15.98:0/207501563 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f633806d8d0 con 0x7f63380cfad0 2024-09-19T08:44:51.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.208+0000 7f632ffff640 1 -- 172.21.15.98:0/207501563 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f6330001a00 con 0x7f63380cfad0 2024-09-19T08:44:51.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.208+0000 7f632ffff640 1 -- 172.21.15.98:0/207501563 >> v1:172.21.15.98:6789/0 conn(0x7f63380cfad0 legacy=0x7f63380723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:51.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.208+0000 7f632ffff640 1 --2- 172.21.15.98:0/207501563 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63240039f0 0x7f6324003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:51.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.208+0000 7f632ffff640 1 --2- 172.21.15.98:0/207501563 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6324004810 0x7f6324004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:51.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.208+0000 7f632ffff640 1 --2- 172.21.15.98:0/207501563 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f633815c550 0x7f6324005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:51.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.208+0000 7f632ffff640 1 -- 172.21.15.98:0/207501563 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f633806c610 con 0x7f63240039f0 2024-09-19T08:44:51.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.208+0000 7f632ffff640 1 -- 172.21.15.98:0/207501563 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f632400bb00 con 0x7f6324004810 2024-09-19T08:44:51.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.208+0000 7f632ffff640 1 -- 172.21.15.98:0/207501563 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f632400bd00 con 0x7f633815c550 2024-09-19T08:44:51.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.208+0000 7f633d3d9640 1 --2- 172.21.15.98:0/207501563 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63240039f0 0x7f6324003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:51.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.208+0000 7f633dbda640 1 --2- 172.21.15.98:0/207501563 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f633815c550 0x7f6324005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:51.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.208+0000 7f633cbd8640 1 --2- 172.21.15.98:0/207501563 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6324004810 0x7f6324004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:51.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.208+0000 7f633d3d9640 1 -- 172.21.15.98:0/207501563 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f633815c550 msgr2=0x7f6324005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:51.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.208+0000 7f633d3d9640 1 --2- 172.21.15.98:0/207501563 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f633815c550 0x7f6324005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:51.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.208+0000 7f633d3d9640 1 -- 172.21.15.98:0/207501563 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6324004810 msgr2=0x7f6324004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:51.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.208+0000 7f633d3d9640 1 --2- 172.21.15.98:0/207501563 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6324004810 0x7f6324004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:51.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.208+0000 7f633d3d9640 1 -- 172.21.15.98:0/207501563 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6324003530 con 0x7f63240039f0 2024-09-19T08:44:51.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.208+0000 7f633cbd8640 1 --2- 172.21.15.98:0/207501563 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6324004810 0x7f6324004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:51.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.212+0000 7f633d3d9640 1 --2- 172.21.15.98:0/207501563 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63240039f0 0x7f6324003e10 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f633400ba30 tx=0x7f633401c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5511ff3eccd26312 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:51.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.212+0000 7f632ffff640 1 -- 172.21.15.98:0/207501563 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63340b3070 con 0x7f63240039f0 2024-09-19T08:44:51.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.212+0000 7f632ffff640 1 -- 172.21.15.98:0/207501563 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63340ae630 con 0x7f63240039f0 2024-09-19T08:44:51.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.212+0000 7f632ffff640 1 -- 172.21.15.98:0/207501563 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63340aa050 con 0x7f63240039f0 2024-09-19T08:44:51.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.212+0000 7f633f63d640 1 -- 172.21.15.98:0/207501563 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63240039f0 msgr2=0x7f6324003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:51.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.212+0000 7f633f63d640 1 --2- 172.21.15.98:0/207501563 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63240039f0 0x7f6324003e10 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f633400ba30 tx=0x7f633401c6a0 comp rx=0 tx=0).stop 2024-09-19T08:44:51.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.212+0000 7f633dbda640 1 -- 172.21.15.98:0/207501563 reap_dead start 2024-09-19T08:44:51.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.212+0000 7f633f63d640 1 -- 172.21.15.98:0/207501563 shutdown_connections 2024-09-19T08:44:51.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.212+0000 7f633f63d640 1 -- 172.21.15.98:0/207501563 >> 172.21.15.98:0/207501563 conn(0x7f63380b2010 msgr2=0x7f63380b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:51.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.212+0000 7f633f63d640 1 -- 172.21.15.98:0/207501563 shutdown_connections 2024-09-19T08:44:51.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.212+0000 7f633f63d640 1 -- 172.21.15.98:0/207501563 wait complete. 2024-09-19T08:44:51.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.212+0000 7f633f63d640 1 Processor -- start 2024-09-19T08:44:51.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.212+0000 7f633f63d640 1 -- start start 2024-09-19T08:44:51.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.212+0000 7f633f63d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63240039f0 0x7f633814ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:51.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.212+0000 7f633f63d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6338152d30 0x7f63380ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:51.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.212+0000 7f633f63d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63381531b0 0x7f63380aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:51.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.212+0000 7f633f63d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6338073510 con 0x7f63240039f0 2024-09-19T08:44:51.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.212+0000 7f633f63d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f633815cb40 con 0x7f63381531b0 2024-09-19T08:44:51.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.212+0000 7f633f63d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6338073690 con 0x7f6338152d30 2024-09-19T08:44:51.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.212+0000 7f633d3d9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63240039f0 0x7f633814ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:51.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.212+0000 7f633d3d9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63240039f0 0x7f633814ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34652/0 (socket says 172.21.15.98:34652) 2024-09-19T08:44:51.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.212+0000 7f633d3d9640 1 -- 172.21.15.98:0/323292529 learned_addr learned my addr 172.21.15.98:0/323292529 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:51.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.212+0000 7f633cbd8640 1 --2- 172.21.15.98:0/323292529 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6338152d30 0x7f63380ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:51.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.212+0000 7f633dbda640 1 --2- 172.21.15.98:0/323292529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63381531b0 0x7f63380aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:51.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.212+0000 7f633d3d9640 1 -- 172.21.15.98:0/323292529 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6338152d30 msgr2=0x7f63380ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:51.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.212+0000 7f633d3d9640 1 --2- 172.21.15.98:0/323292529 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6338152d30 0x7f63380ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:51.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.212+0000 7f633d3d9640 1 -- 172.21.15.98:0/323292529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63381531b0 msgr2=0x7f63380aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:51.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.212+0000 7f633d3d9640 1 --2- 172.21.15.98:0/323292529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63381531b0 0x7f63380aedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:51.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.212+0000 7f633d3d9640 1 -- 172.21.15.98:0/323292529 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63380b4570 con 0x7f63240039f0 2024-09-19T08:44:51.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.212+0000 7f633d3d9640 1 --2- 172.21.15.98:0/323292529 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63240039f0 0x7f633814ef30 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f633401cb80 tx=0x7f6324005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:51.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.216+0000 7f632dffb640 1 -- 172.21.15.98:0/323292529 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63340bf070 con 0x7f63240039f0 2024-09-19T08:44:51.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.216+0000 7f632dffb640 1 -- 172.21.15.98:0/323292529 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63340b9630 con 0x7f63240039f0 2024-09-19T08:44:51.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.216+0000 7f633f63d640 1 -- 172.21.15.98:0/323292529 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63380b4850 con 0x7f63240039f0 2024-09-19T08:44:51.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.216+0000 7f632dffb640 1 -- 172.21.15.98:0/323292529 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63340b3040 con 0x7f63240039f0 2024-09-19T08:44:51.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.216+0000 7f633f63d640 1 -- 172.21.15.98:0/323292529 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63380b5330 con 0x7f63240039f0 2024-09-19T08:44:51.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.216+0000 7f633f63d640 1 -- 172.21.15.98:0/323292529 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63380cf3c0 con 0x7f63240039f0 2024-09-19T08:44:51.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.216+0000 7f632dffb640 1 -- 172.21.15.98:0/323292529 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f63340c2070 con 0x7f63240039f0 2024-09-19T08:44:51.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.216+0000 7f632dffb640 1 --2- 172.21.15.98:0/323292529 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f631c067ce0 0x7f631c06a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:51.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.216+0000 7f632dffb640 1 -- 172.21.15.98:0/323292529 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4971+0+0 (secure 0 0 0) 0x7f633411fee0 con 0x7f63240039f0 2024-09-19T08:44:51.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.216+0000 7f633cbd8640 1 --2- 172.21.15.98:0/323292529 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f631c067ce0 0x7f631c06a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:51.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.216+0000 7f633cbd8640 1 --2- 172.21.15.98:0/323292529 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f631c067ce0 0x7f631c06a1a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f63380af640 tx=0x7f6328009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:51.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.220+0000 7f632dffb640 1 -- 172.21.15.98:0/323292529 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f63340163b0 con 0x7f63240039f0 2024-09-19T08:44:51.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.416+0000 7f633f63d640 1 -- 172.21.15.98:0/323292529 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f633815a4e0 con 0x7f63240039f0 2024-09-19T08:44:51.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.420+0000 7f632dffb640 1 -- 172.21.15.98:0/323292529 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+3911 (secure 0 0 0) 0x7f63340effc0 con 0x7f63240039f0 2024-09-19T08:44:51.420 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:44:51.421 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":63,"btime":"2024-09-19T08:44:46:123633+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6004,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3739962222","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3739962222},{"type":"v1","addr":"172.21.15.139:6833","nonce":3739962222}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6007,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/421473653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":421473653},{"type":"v1","addr":"172.21.15.139:6835","nonce":421473653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6981,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3126941916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3126941916},{"type":"v1","addr":"172.21.15.98:6833","nonce":3126941916}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":7449,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/316655233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":316655233},{"type":"v1","addr":"172.21.15.98:6835","nonce":316655233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[]} 2024-09-19T08:44:51.421 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 63 2024-09-19T08:44:51.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.420+0000 7f633f63d640 1 -- 172.21.15.98:0/323292529 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f631c067ce0 msgr2=0x7f631c06a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:51.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.420+0000 7f633f63d640 1 --2- 172.21.15.98:0/323292529 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f631c067ce0 0x7f631c06a1a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f63380af640 tx=0x7f6328009040 comp rx=0 tx=0).stop 2024-09-19T08:44:51.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.420+0000 7f633f63d640 1 -- 172.21.15.98:0/323292529 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63240039f0 msgr2=0x7f633814ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:51.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.420+0000 7f633f63d640 1 --2- 172.21.15.98:0/323292529 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63240039f0 0x7f633814ef30 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f633401cb80 tx=0x7f6324005840 comp rx=0 tx=0).stop 2024-09-19T08:44:51.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.420+0000 7f633f63d640 1 -- 172.21.15.98:0/323292529 shutdown_connections 2024-09-19T08:44:51.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.420+0000 7f633f63d640 1 --2- 172.21.15.98:0/323292529 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f631c067ce0 0x7f631c06a1a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:51.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.420+0000 7f633f63d640 1 --2- 172.21.15.98:0/323292529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63381531b0 0x7f63380aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:51.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.420+0000 7f633f63d640 1 --2- 172.21.15.98:0/323292529 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6338152d30 0x7f63380ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:51.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.420+0000 7f633f63d640 1 --2- 172.21.15.98:0/323292529 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63240039f0 0x7f633814ef30 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:51.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.420+0000 7f633f63d640 1 -- 172.21.15.98:0/323292529 >> 172.21.15.98:0/323292529 conn(0x7f63380b2010 msgr2=0x7f63381598f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:51.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.420+0000 7f633f63d640 1 -- 172.21.15.98:0/323292529 shutdown_connections 2024-09-19T08:44:51.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.420+0000 7f633f63d640 1 -- 172.21.15.98:0/323292529 wait complete. 2024-09-19T08:44:51.439 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-19T08:44:51.439 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-19T08:44:51.439 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-19T08:44:51.439 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:44:51.440 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-19T08:44:51.440 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-19T08:44:51.440 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-19T08:44:51.440 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:44:51.440 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-19T08:44:51.440 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-19T08:44:51.440 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-19T08:44:51.440 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:44:51.441 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-19T08:44:51.441 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-19T08:44:51.441 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-19T08:44:51.441 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:44:51.448 INFO:tasks.ceph.mds.a:Stopped 2024-09-19T08:44:51.448 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-19T08:44:51.451 INFO:tasks.ceph.mds.c:Stopped 2024-09-19T08:44:51.452 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-19T08:44:51.453 INFO:tasks.ceph.mds.d:Stopped 2024-09-19T08:44:51.453 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-19T08:44:51.454 INFO:tasks.ceph.mds.b:Stopped 2024-09-19T08:44:51.455 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-19T08:44:51.493 INFO:tasks.ceph.mds.a:Started 2024-09-19T08:44:51.493 DEBUG:teuthology.parallel:result is None 2024-09-19T08:44:51.494 INFO:tasks.ceph.mds.c:Started 2024-09-19T08:44:51.494 DEBUG:teuthology.parallel:result is None 2024-09-19T08:44:51.500 INFO:tasks.ceph.mds.d:Started 2024-09-19T08:44:51.501 DEBUG:teuthology.parallel:result is None 2024-09-19T08:44:51.502 INFO:tasks.ceph.mds.b:Started 2024-09-19T08:44:51.502 DEBUG:teuthology.parallel:result is None 2024-09-19T08:44:51.502 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-19T08:44:51.558 INFO:tasks.ceph.mds.a.smithi098.stdout:starting mds.a at 2024-09-19T08:44:51.559 INFO:tasks.ceph.mds.c.smithi098.stdout:starting mds.c at 2024-09-19T08:44:51.570 INFO:tasks.ceph.mds.d.smithi139.stdout:starting mds.d at 2024-09-19T08:44:51.581 INFO:tasks.ceph.mds.b.smithi139.stdout:starting mds.b at 2024-09-19T08:44:51.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.612+0000 7f27f1737640 1 Processor -- start 2024-09-19T08:44:51.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.612+0000 7f27f1737640 1 -- start start 2024-09-19T08:44:51.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.612+0000 7f27f1737640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f27ec15c220 0x7f27ec0733f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:51.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.612+0000 7f27f1737640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27ec0740b0 con 0x7f27ec073930 2024-09-19T08:44:51.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.612+0000 7f27f1737640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27ec073dd0 con 0x7f27ec0d1700 2024-09-19T08:44:51.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.612+0000 7f27f1737640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f27ec0b6460 con 0x7f27ec15c220 2024-09-19T08:44:51.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.612+0000 7f27ea7fc640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f27ec073930 0x7f27ec0d0ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56520/0 (socket says 172.21.15.98:56520) 2024-09-19T08:44:51.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.612+0000 7f27ea7fc640 1 -- 172.21.15.98:0/104854365 learned_addr learned my addr 172.21.15.98:0/104854365 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:51.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.612+0000 7f27eaffd640 1 --2- 172.21.15.98:0/104854365 >> v2:172.21.15.139:3300/0 conn(0x7f27ec15c220 0x7f27ec0733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:51.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.612+0000 7f27e9ffb640 1 -- 172.21.15.98:0/104854365 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1162782900 0 0) 0x7f27ec0740b0 con 0x7f27ec073930 2024-09-19T08:44:51.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.612+0000 7f27e9ffb640 1 -- 172.21.15.98:0/104854365 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f27d4003530 con 0x7f27ec073930 2024-09-19T08:44:51.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.612+0000 7f27eaffd640 1 -- 172.21.15.98:0/104854365 >> v1:172.21.15.139:6789/0 conn(0x7f27ec0d1700 legacy=0x7f27ec0b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:51.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.612+0000 7f27eaffd640 1 -- 172.21.15.98:0/104854365 >> v1:172.21.15.98:6789/0 conn(0x7f27ec073930 legacy=0x7f27ec0d0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:51.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.612+0000 7f27e9ffb640 1 -- 172.21.15.98:0/104854365 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2083086939 0 0) 0x7f27d4003530 con 0x7f27ec073930 2024-09-19T08:44:51.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.612+0000 7f27eaffd640 1 -- 172.21.15.98:0/104854365 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27ec0b6740 con 0x7f27ec15c220 2024-09-19T08:44:51.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.616+0000 7f27eaffd640 1 --2- 172.21.15.98:0/104854365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27ec15c220 0x7f27ec0733f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f27dc009d90 tx=0x7f27dc01c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=734b358e50272533 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:51.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.616+0000 7f27e9ffb640 1 -- 172.21.15.98:0/104854365 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc0b2070 con 0x7f27ec15c220 2024-09-19T08:44:51.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.616+0000 7f27e9ffb640 1 -- 172.21.15.98:0/104854365 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27dc0a56c0 con 0x7f27ec15c220 2024-09-19T08:44:51.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.616+0000 7f27e9ffb640 1 -- 172.21.15.98:0/104854365 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e0001680 con 0x7f27ec15c220 2024-09-19T08:44:51.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.616+0000 7f27f1737640 1 -- 172.21.15.98:0/104854365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27ec15c220 msgr2=0x7f27ec0733f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:51.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.616+0000 7f27f1737640 1 --2- 172.21.15.98:0/104854365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27ec15c220 0x7f27ec0733f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f27dc009d90 tx=0x7f27dc01c860 comp rx=0 tx=0).stop 2024-09-19T08:44:51.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.616+0000 7f27f1737640 1 -- 172.21.15.98:0/104854365 shutdown_connections 2024-09-19T08:44:51.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.616+0000 7f27f1737640 1 --2- 172.21.15.98:0/104854365 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27ec15c220 0x7f27ec0733f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:51.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.616+0000 7f27f1737640 1 -- 172.21.15.98:0/104854365 >> 172.21.15.98:0/104854365 conn(0x7f27ec0b40a0 msgr2=0x7f27ec0b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:51.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.616+0000 7f27f1737640 1 -- 172.21.15.98:0/104854365 shutdown_connections 2024-09-19T08:44:51.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.616+0000 7f27f1737640 1 -- 172.21.15.98:0/104854365 wait complete. 2024-09-19T08:44:51.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.616+0000 7f27f1737640 1 Processor -- start 2024-09-19T08:44:51.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.616+0000 7f27f1737640 1 -- start start 2024-09-19T08:44:51.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.616+0000 7f27f1737640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27ec073930 0x7f27ec14b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:51.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.616+0000 7f27f1737640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27ec0d1700 0x7f27ec156330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:51.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.616+0000 7f27f1737640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f27ec15c220 0x7f27ec0d2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:51.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.616+0000 7f27f1737640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec169eb0 con 0x7f27ec0d1700 2024-09-19T08:44:51.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.616+0000 7f27f1737640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec15d060 con 0x7f27ec073930 2024-09-19T08:44:51.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.616+0000 7f27f1737640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f27ec16a030 con 0x7f27ec15c220 2024-09-19T08:44:51.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.616+0000 7f27ea7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27ec0d1700 0x7f27ec156330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:51.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.616+0000 7f27ea7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27ec0d1700 0x7f27ec156330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34692/0 (socket says 172.21.15.98:34692) 2024-09-19T08:44:51.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.616+0000 7f27ea7fc640 1 -- 172.21.15.98:0/3732691603 learned_addr learned my addr 172.21.15.98:0/3732691603 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:51.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.616+0000 7f27eaffd640 1 --2- 172.21.15.98:0/3732691603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27ec073930 0x7f27ec14b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:51.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.616+0000 7f27eb7fe640 1 --2- 172.21.15.98:0/3732691603 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f27ec15c220 0x7f27ec0d2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:51.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.620+0000 7f27ea7fc640 1 -- 172.21.15.98:0/3732691603 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f27ec15c220 msgr2=0x7f27ec0d2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:51.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.620+0000 7f27ea7fc640 1 --2- 172.21.15.98:0/3732691603 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f27ec15c220 0x7f27ec0d2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:51.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.620+0000 7f27ea7fc640 1 -- 172.21.15.98:0/3732691603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27ec073930 msgr2=0x7f27ec14b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:51.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.620+0000 7f27ea7fc640 1 --2- 172.21.15.98:0/3732691603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27ec073930 0x7f27ec14b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:51.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.620+0000 7f27ea7fc640 1 -- 172.21.15.98:0/3732691603 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27ec14bc90 con 0x7f27ec0d1700 2024-09-19T08:44:51.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.620+0000 7f27ea7fc640 1 --2- 172.21.15.98:0/3732691603 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27ec0d1700 0x7f27ec156330 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f27e000c6f0 tx=0x7f27e000caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:51.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.620+0000 7f27d37fe640 1 -- 172.21.15.98:0/3732691603 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e0005cf0 con 0x7f27ec0d1700 2024-09-19T08:44:51.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.620+0000 7f27d37fe640 1 -- 172.21.15.98:0/3732691603 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27e0007070 con 0x7f27ec0d1700 2024-09-19T08:44:51.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.620+0000 7f27d37fe640 1 -- 172.21.15.98:0/3732691603 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e0007460 con 0x7f27ec0d1700 2024-09-19T08:44:51.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.620+0000 7f27f1737640 1 -- 172.21.15.98:0/3732691603 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27ec14bf70 con 0x7f27ec0d1700 2024-09-19T08:44:51.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.620+0000 7f27f1737640 1 -- 172.21.15.98:0/3732691603 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27ec15d4b0 con 0x7f27ec0d1700 2024-09-19T08:44:51.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.620+0000 7f27f1737640 1 -- 172.21.15.98:0/3732691603 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27ec0733f0 con 0x7f27ec0d1700 2024-09-19T08:44:51.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.620+0000 7f27d37fe640 1 -- 172.21.15.98:0/3732691603 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f27e0007600 con 0x7f27ec0d1700 2024-09-19T08:44:51.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.620+0000 7f27d37fe640 1 --2- 172.21.15.98:0/3732691603 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f27d8067cb0 0x7f27d806a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:51.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.620+0000 7f27d37fe640 1 -- 172.21.15.98:0/3732691603 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 4971+0+0 (secure 0 0 0) 0x7f27e0085e70 con 0x7f27ec0d1700 2024-09-19T08:44:51.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.620+0000 7f27eaffd640 1 --2- 172.21.15.98:0/3732691603 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f27d8067cb0 0x7f27d806a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:51.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.620+0000 7f27eaffd640 1 --2- 172.21.15.98:0/3732691603 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f27d8067cb0 0x7f27d806a170 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f27dc0af000 tx=0x7f27dc0a7070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:51.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.624+0000 7f27d37fe640 1 -- 172.21.15.98:0/3732691603 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f27e0056060 con 0x7f27ec0d1700 2024-09-19T08:44:51.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:51.784+0000 7f27f1737640 1 -- 172.21.15.98:0/3732691603 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f27ec15d260 con 0x7f27ec0d1700 2024-09-19T08:44:52.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.844+0000 7f27d37fe640 1 -- 172.21.15.98:0/3732691603 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v114) ==== 97+0+0 (secure 0 0 0) 0x7f27e005a360 con 0x7f27ec0d1700 2024-09-19T08:44:52.847 INFO:teuthology.orchestra.run.smithi098.stderr: removed all blocklist entries 2024-09-19T08:44:52.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.848+0000 7f27f1737640 1 -- 172.21.15.98:0/3732691603 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f27d8067cb0 msgr2=0x7f27d806a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:52.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.848+0000 7f27f1737640 1 --2- 172.21.15.98:0/3732691603 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f27d8067cb0 0x7f27d806a170 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f27dc0af000 tx=0x7f27dc0a7070 comp rx=0 tx=0).stop 2024-09-19T08:44:52.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.848+0000 7f27f1737640 1 -- 172.21.15.98:0/3732691603 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27ec0d1700 msgr2=0x7f27ec156330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:52.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.848+0000 7f27f1737640 1 --2- 172.21.15.98:0/3732691603 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27ec0d1700 0x7f27ec156330 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f27e000c6f0 tx=0x7f27e000caf0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.848+0000 7f27f1737640 1 -- 172.21.15.98:0/3732691603 shutdown_connections 2024-09-19T08:44:52.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.848+0000 7f27f1737640 1 --2- 172.21.15.98:0/3732691603 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f27d8067cb0 0x7f27d806a170 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.848+0000 7f27f1737640 1 --2- 172.21.15.98:0/3732691603 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f27ec15c220 0x7f27ec0d2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.848+0000 7f27f1737640 1 --2- 172.21.15.98:0/3732691603 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f27ec0d1700 0x7f27ec156330 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.848+0000 7f27f1737640 1 --2- 172.21.15.98:0/3732691603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27ec073930 0x7f27ec14b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.848+0000 7f27f1737640 1 -- 172.21.15.98:0/3732691603 >> 172.21.15.98:0/3732691603 conn(0x7f27ec0b40a0 msgr2=0x7f27ec0b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:52.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.848+0000 7f27f1737640 1 -- 172.21.15.98:0/3732691603 shutdown_connections 2024-09-19T08:44:52.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.848+0000 7f27f1737640 1 -- 172.21.15.98:0/3732691603 wait complete. 2024-09-19T08:44:52.872 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-19T08:44:52.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.992+0000 7f6baf666640 1 Processor -- start 2024-09-19T08:44:52.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.992+0000 7f6baf666640 1 -- start start 2024-09-19T08:44:52.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.992+0000 7f6baf666640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6ba8160740 0x7f6ba8163170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:52.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.992+0000 7f6baf666640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ba8163bf0 con 0x7f6ba806c3a0 2024-09-19T08:44:52.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.992+0000 7f6baf666640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ba8163ed0 con 0x7f6ba81636b0 2024-09-19T08:44:52.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.992+0000 7f6baf666640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f6ba80ce320 con 0x7f6ba8160740 2024-09-19T08:44:52.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.992+0000 7f6badc03640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f6ba806c3a0 0x7f6ba80ceb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56534/0 (socket says 172.21.15.98:56534) 2024-09-19T08:44:52.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.992+0000 7f6badc03640 1 -- 172.21.15.98:0/610436760 learned_addr learned my addr 172.21.15.98:0/610436760 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:52.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.992+0000 7f6bad402640 1 --2- 172.21.15.98:0/610436760 >> v2:172.21.15.139:3300/0 conn(0x7f6ba8160740 0x7f6ba8163170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:52.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.992+0000 7f6b9ffff640 1 -- 172.21.15.98:0/610436760 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2394634376 0 0) 0x7f6ba8163bf0 con 0x7f6ba806c3a0 2024-09-19T08:44:52.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.992+0000 7f6b9ffff640 1 -- 172.21.15.98:0/610436760 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b94003530 con 0x7f6ba806c3a0 2024-09-19T08:44:52.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.992+0000 7f6b9ffff640 1 -- 172.21.15.98:0/610436760 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2238196171 0 0) 0x7f6b94003530 con 0x7f6ba806c3a0 2024-09-19T08:44:52.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.992+0000 7f6b9ffff640 1 -- 172.21.15.98:0/610436760 >> v2:172.21.15.139:3300/0 conn(0x7f6ba8160740 msgr2=0x7f6ba8163170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:52.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.992+0000 7f6b9ffff640 1 --2- 172.21.15.98:0/610436760 >> v2:172.21.15.139:3300/0 conn(0x7f6ba8160740 0x7f6ba8163170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.992+0000 7f6b9ffff640 1 -- 172.21.15.98:0/610436760 >> v1:172.21.15.139:6789/0 conn(0x7f6ba81636b0 legacy=0x7f6ba806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:52.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.992+0000 7f6b9ffff640 1 -- 172.21.15.98:0/610436760 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba80ce600 con 0x7f6ba806c3a0 2024-09-19T08:44:52.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.992+0000 7f6b9ffff640 1 -- 172.21.15.98:0/610436760 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f6ba0001a00 con 0x7f6ba806c3a0 2024-09-19T08:44:52.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.992+0000 7f6bad402640 1 --2- 172.21.15.98:0/610436760 >> v2:172.21.15.139:3300/0 conn(0x7f6ba8160740 0x7f6ba8163170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:44:52.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.992+0000 7f6b9ffff640 1 -- 172.21.15.98:0/610436760 >> v1:172.21.15.98:6789/0 conn(0x7f6ba806c3a0 legacy=0x7f6ba80ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:52.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.992+0000 7f6b9ffff640 1 --2- 172.21.15.98:0/610436760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b940039c0 0x7f6b94003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:52.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.992+0000 7f6b9ffff640 1 --2- 172.21.15.98:0/610436760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b94004810 0x7f6b94004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:52.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.992+0000 7f6b9ffff640 1 --2- 172.21.15.98:0/610436760 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b940051d0 0x7f6b9400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:52.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.992+0000 7f6b9ffff640 1 -- 172.21.15.98:0/610436760 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba80ce320 con 0x7f6b94004810 2024-09-19T08:44:52.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.992+0000 7f6b9ffff640 1 -- 172.21.15.98:0/610436760 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9400c120 con 0x7f6b940039c0 2024-09-19T08:44:52.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.992+0000 7f6b9ffff640 1 -- 172.21.15.98:0/610436760 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6b9400c320 con 0x7f6b940051d0 2024-09-19T08:44:52.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.992+0000 7f6bacc01640 1 --2- 172.21.15.98:0/610436760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b94004810 0x7f6b94004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:52.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.992+0000 7f6bad402640 1 --2- 172.21.15.98:0/610436760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b940039c0 0x7f6b94003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:52.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.992+0000 7f6badc03640 1 --2- 172.21.15.98:0/610436760 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b940051d0 0x7f6b9400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:52.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.992+0000 7f6bacc01640 1 -- 172.21.15.98:0/610436760 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b940051d0 msgr2=0x7f6b9400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:52.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.992+0000 7f6bacc01640 1 --2- 172.21.15.98:0/610436760 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6b940051d0 0x7f6b9400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.992+0000 7f6bacc01640 1 -- 172.21.15.98:0/610436760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b940039c0 msgr2=0x7f6b94003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:52.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.992+0000 7f6bacc01640 1 --2- 172.21.15.98:0/610436760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b940039c0 0x7f6b94003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:52.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.992+0000 7f6bacc01640 1 -- 172.21.15.98:0/610436760 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b94003530 con 0x7f6b94004810 2024-09-19T08:44:52.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.992+0000 7f6badc03640 1 -- 172.21.15.98:0/610436760 reap_dead start 2024-09-19T08:44:52.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.996+0000 7f6bacc01640 1 --2- 172.21.15.98:0/610436760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b94004810 0x7f6b94004c90 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f6b98001780 tx=0x7f6b9801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c168fa24a061b59 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:52.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.996+0000 7f6b9ffff640 1 -- 172.21.15.98:0/610436760 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b980013a0 con 0x7f6b94004810 2024-09-19T08:44:52.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.996+0000 7f6b9ffff640 1 -- 172.21.15.98:0/610436760 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b980052f0 con 0x7f6b94004810 2024-09-19T08:44:52.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.996+0000 7f6b9ffff640 1 -- 172.21.15.98:0/610436760 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98003840 con 0x7f6b94004810 2024-09-19T08:44:52.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.996+0000 7f6baf666640 1 -- 172.21.15.98:0/610436760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b94004810 msgr2=0x7f6b94004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:53.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.996+0000 7f6baf666640 1 --2- 172.21.15.98:0/610436760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b94004810 0x7f6b94004c90 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f6b98001780 tx=0x7f6b9801c420 comp rx=0 tx=0).stop 2024-09-19T08:44:53.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.996+0000 7f6baf666640 1 -- 172.21.15.98:0/610436760 shutdown_connections 2024-09-19T08:44:53.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.996+0000 7f6baf666640 1 --2- 172.21.15.98:0/610436760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6b94004810 0x7f6b94004c90 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:53.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.996+0000 7f6baf666640 1 -- 172.21.15.98:0/610436760 >> 172.21.15.98:0/610436760 conn(0x7f6ba80b2010 msgr2=0x7f6ba80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:53.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.996+0000 7f6baf666640 1 -- 172.21.15.98:0/610436760 shutdown_connections 2024-09-19T08:44:53.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.996+0000 7f6baf666640 1 -- 172.21.15.98:0/610436760 wait complete. 2024-09-19T08:44:53.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.996+0000 7f6baf666640 1 Processor -- start 2024-09-19T08:44:53.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:52.996+0000 7f6baf666640 1 -- start start 2024-09-19T08:44:53.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.000+0000 7f6baf666640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b94004810 0x7f6ba80714c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:53.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.000+0000 7f6baf666640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ba8158350 0x7f6ba80cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:53.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.000+0000 7f6baf666640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ba80d0130 0x7f6ba81492d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:53.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.000+0000 7f6baf666640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba80cce40 con 0x7f6ba8158350 2024-09-19T08:44:53.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.000+0000 7f6baf666640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba80cccc0 con 0x7f6b94004810 2024-09-19T08:44:53.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.000+0000 7f6baf666640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6ba8163170 con 0x7f6ba80d0130 2024-09-19T08:44:53.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.000+0000 7f6bacc01640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ba8158350 0x7f6ba80cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:53.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.000+0000 7f6bacc01640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ba8158350 0x7f6ba80cfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34700/0 (socket says 172.21.15.98:34700) 2024-09-19T08:44:53.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.000+0000 7f6bacc01640 1 -- 172.21.15.98:0/4169265554 learned_addr learned my addr 172.21.15.98:0/4169265554 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:53.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.000+0000 7f6bad402640 1 --2- 172.21.15.98:0/4169265554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b94004810 0x7f6ba80714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:53.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.000+0000 7f6bacc01640 1 -- 172.21.15.98:0/4169265554 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ba80d0130 msgr2=0x7f6ba81492d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:53.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.000+0000 7f6bacc01640 1 --2- 172.21.15.98:0/4169265554 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ba80d0130 0x7f6ba81492d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:53.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.000+0000 7f6bacc01640 1 -- 172.21.15.98:0/4169265554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b94004810 msgr2=0x7f6ba80714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:53.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.000+0000 7f6bacc01640 1 --2- 172.21.15.98:0/4169265554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b94004810 0x7f6ba80714c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:53.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.000+0000 7f6bacc01640 1 -- 172.21.15.98:0/4169265554 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba80ceb50 con 0x7f6ba8158350 2024-09-19T08:44:53.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.000+0000 7f6bacc01640 1 --2- 172.21.15.98:0/4169265554 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ba8158350 0x7f6ba80cfbf0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f6b9801c250 tx=0x7f6b94005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:53.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.000+0000 7f6b9dffb640 1 -- 172.21.15.98:0/4169265554 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b980b8070 con 0x7f6ba8158350 2024-09-19T08:44:53.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.000+0000 7f6b9dffb640 1 -- 172.21.15.98:0/4169265554 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b98000ee0 con 0x7f6ba8158350 2024-09-19T08:44:53.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.000+0000 7f6baf666640 1 -- 172.21.15.98:0/4169265554 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba80cee30 con 0x7f6ba8158350 2024-09-19T08:44:53.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.000+0000 7f6b9dffb640 1 -- 172.21.15.98:0/4169265554 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98007a40 con 0x7f6ba8158350 2024-09-19T08:44:53.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.000+0000 7f6baf666640 1 -- 172.21.15.98:0/4169265554 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba80cf3f0 con 0x7f6ba8158350 2024-09-19T08:44:53.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.000+0000 7f6baf666640 1 -- 172.21.15.98:0/4169265554 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ba81634a0 con 0x7f6ba8158350 2024-09-19T08:44:53.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.000+0000 7f6b9dffb640 1 -- 172.21.15.98:0/4169265554 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f6b98007480 con 0x7f6ba8158350 2024-09-19T08:44:53.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.000+0000 7f6b9dffb640 1 --2- 172.21.15.98:0/4169265554 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6b8c067ce0 0x7f6b8c06a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:53.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.000+0000 7f6b9dffb640 1 -- 172.21.15.98:0/4169265554 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4885+0+0 (secure 0 0 0) 0x7f6b98120360 con 0x7f6ba8158350 2024-09-19T08:44:53.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.000+0000 7f6bad402640 1 --2- 172.21.15.98:0/4169265554 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6b8c067ce0 0x7f6b8c06a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:53.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.000+0000 7f6bad402640 1 --2- 172.21.15.98:0/4169265554 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6b8c067ce0 0x7f6b8c06a1a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6ba806be70 tx=0x7f6ba4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:53.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.004+0000 7f6b9dffb640 1 -- 172.21.15.98:0/4169265554 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6b98016130 con 0x7f6ba8158350 2024-09-19T08:44:53.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.208+0000 7f6baf666640 1 -- 172.21.15.98:0/4169265554 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f6ba80742b0 con 0x7f6ba8158350 2024-09-19T08:44:53.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.208+0000 7f6b9dffb640 1 -- 172.21.15.98:0/4169265554 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) ==== 80+0+5304 (secure 0 0 0) 0x7f6b980f0b70 con 0x7f6ba8158350 2024-09-19T08:44:53.211 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:44:53.212 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:44:53.212 INFO:teuthology.orchestra.run.smithi098.stdout: "auth_dump": [ 2024-09-19T08:44:53.212 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:44:53.212 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.a", 2024-09-19T08:44:53.212 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmuujGBRAAa2k/kW68NJI6nUKx+W+G2Q==", 2024-09-19T08:44:53.212 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:44:53.212 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:44:53.212 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:44:53.212 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:44:53.212 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:44:53.212 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:44:53.212 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:44:53.213 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:44:53.213 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.b", 2024-09-19T08:44:53.213 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmFoojFBAAaOZPpa41fdjBPEn3CGsJnA==", 2024-09-19T08:44:53.213 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:44:53.213 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:44:53.213 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:44:53.213 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:44:53.213 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:44:53.213 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:44:53.213 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:44:53.213 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:44:53.213 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.c", 2024-09-19T08:44:53.213 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmcQcDDhAAF0QJMlljMH+DiHfWW+/Sog==", 2024-09-19T08:44:53.213 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:44:53.214 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:44:53.214 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:44:53.214 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:44:53.214 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:44:53.214 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:44:53.214 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:44:53.214 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:44:53.214 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.d", 2024-09-19T08:44:53.214 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmwWmqHBAAf15VcrdeHf8AzQjGsN9LoA==", 2024-09-19T08:44:53.214 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:44:53.214 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:44:53.214 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:44:53.214 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:44:53.214 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:44:53.215 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:44:53.215 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:44:53.215 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:44:53.215 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.0", 2024-09-19T08:44:53.215 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDC4utm9PjNKxAAa6SG9AE4smPiu4azRAFKvg==", 2024-09-19T08:44:53.215 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:44:53.215 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:44:53.215 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:44:53.215 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:44:53.215 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:44:53.215 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:44:53.215 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:44:53.215 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.1", 2024-09-19T08:44:53.215 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDG4utmHy6UCxAAN6hpG4DJEp81iG/KoloTvQ==", 2024-09-19T08:44:53.216 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:44:53.216 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:44:53.216 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:44:53.216 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:44:53.216 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:44:53.216 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:44:53.216 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:44:53.216 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.2", 2024-09-19T08:44:53.216 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDJ4utmUVP4GhAAJNAL30sEAseoDG2fE6YbZQ==", 2024-09-19T08:44:53.216 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:44:53.216 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:44:53.216 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:44:53.216 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:44:53.216 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:44:53.217 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:44:53.217 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:44:53.217 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.3", 2024-09-19T08:44:53.217 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDM4utmTKgoLRAAiCrPRLML+wvjOt6RhKay3w==", 2024-09-19T08:44:53.217 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:44:53.217 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:44:53.217 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:44:53.217 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:44:53.217 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:44:53.217 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:44:53.217 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:44:53.217 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.4", 2024-09-19T08:44:53.217 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDS4utmAf6bMhAAI3uENuFoXLicUeuspgFjYA==", 2024-09-19T08:44:53.218 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:44:53.218 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:44:53.218 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:44:53.218 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:44:53.218 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:44:53.218 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:44:53.218 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:44:53.218 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.5", 2024-09-19T08:44:53.218 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDW4utmRjxwEhAAF1A2GtrZXVx9BlmjHjhUDQ==", 2024-09-19T08:44:53.218 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:44:53.218 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:44:53.219 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:44:53.219 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:44:53.219 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:44:53.219 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:44:53.219 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:44:53.219 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.6", 2024-09-19T08:44:53.219 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDZ4utmSk/XIRAAROSfzGL7smR88ZkDSx5jAg==", 2024-09-19T08:44:53.219 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:44:53.219 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:44:53.219 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:44:53.219 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:44:53.219 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:44:53.219 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:44:53.220 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:44:53.220 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.7", 2024-09-19T08:44:53.220 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDc4utmXfpbMxAAqEM4I/ACFvfu/sMA8bMcrQ==", 2024-09-19T08:44:53.220 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:44:53.220 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:44:53.220 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:44:53.220 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:44:53.220 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:44:53.220 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:44:53.220 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:44:53.220 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.0", 2024-09-19T08:44:53.220 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utm0vNpIhAA4S1E8dFZ0+ibmSKhmZ+bIA==", 2024-09-19T08:44:53.220 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:44:53.221 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:44:53.221 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow r", 2024-09-19T08:44:53.221 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-19T08:44:53.221 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:44:53.221 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:44:53.221 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:44:53.221 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.1", 2024-09-19T08:44:53.221 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmx7xyKhAAC84lykPEb01MiypXKQVREQ==", 2024-09-19T08:44:53.221 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:44:53.221 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:44:53.221 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-19T08:44:53.221 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-19T08:44:53.221 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-19T08:44:53.221 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:44:53.222 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:44:53.222 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:44:53.222 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.2", 2024-09-19T08:44:53.222 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmteOQJxAAtTF2Py21ZE2WRQjeQSrkKQ==", 2024-09-19T08:44:53.222 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:44:53.222 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:44:53.222 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-19T08:44:53.222 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-19T08:44:53.222 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-19T08:44:53.223 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:44:53.223 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:44:53.223 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:44:53.223 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.3", 2024-09-19T08:44:53.223 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmKN92LxAAMqraZOWRT6IIfx5AuDWS2g==", 2024-09-19T08:44:53.223 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:44:53.223 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:44:53.223 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-19T08:44:53.223 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-19T08:44:53.223 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-19T08:44:53.223 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:44:53.224 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:44:53.224 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:44:53.224 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.admin", 2024-09-19T08:44:53.224 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC+4utmLH97KhAA1DTYW5bTrGUG+7cb8+d7gA==", 2024-09-19T08:44:53.224 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:44:53.224 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-09-19T08:44:53.224 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:44:53.224 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:44:53.224 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:44:53.224 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:44:53.225 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:44:53.225 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:44:53.225 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mgr.x", 2024-09-19T08:44:53.225 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmbKuyAhAA0/XjHUIPOIV/YvNEhB8rEQ==", 2024-09-19T08:44:53.225 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:44:53.225 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-09-19T08:44:53.225 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile mgr", 2024-09-19T08:44:53.225 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:44:53.225 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:44:53.225 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:44:53.225 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:44:53.225 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mgr.y", 2024-09-19T08:44:53.225 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC+4utmU+ZrOxAAsvcwINH6qyxfN7N+rDrkqg==", 2024-09-19T08:44:53.226 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:44:53.226 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-09-19T08:44:53.226 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile mgr", 2024-09-19T08:44:53.226 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:44:53.226 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:44:53.226 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:44:53.226 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-19T08:44:53.226 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:44:53.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.212+0000 7f6baf666640 1 -- 172.21.15.98:0/4169265554 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6b8c067ce0 msgr2=0x7f6b8c06a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:53.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.212+0000 7f6baf666640 1 --2- 172.21.15.98:0/4169265554 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6b8c067ce0 0x7f6b8c06a1a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6ba806be70 tx=0x7f6ba4008040 comp rx=0 tx=0).stop 2024-09-19T08:44:53.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.212+0000 7f6baf666640 1 -- 172.21.15.98:0/4169265554 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ba8158350 msgr2=0x7f6ba80cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:53.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.212+0000 7f6baf666640 1 --2- 172.21.15.98:0/4169265554 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ba8158350 0x7f6ba80cfbf0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f6b9801c250 tx=0x7f6b94005840 comp rx=0 tx=0).stop 2024-09-19T08:44:53.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.212+0000 7f6baf666640 1 -- 172.21.15.98:0/4169265554 shutdown_connections 2024-09-19T08:44:53.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.212+0000 7f6baf666640 1 --2- 172.21.15.98:0/4169265554 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6b8c067ce0 0x7f6b8c06a1a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:53.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.212+0000 7f6baf666640 1 --2- 172.21.15.98:0/4169265554 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ba80d0130 0x7f6ba81492d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:53.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.212+0000 7f6baf666640 1 --2- 172.21.15.98:0/4169265554 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ba8158350 0x7f6ba80cfbf0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:53.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.212+0000 7f6baf666640 1 --2- 172.21.15.98:0/4169265554 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6b94004810 0x7f6ba80714c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:53.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.212+0000 7f6baf666640 1 -- 172.21.15.98:0/4169265554 >> 172.21.15.98:0/4169265554 conn(0x7f6ba80b2010 msgr2=0x7f6ba8160d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:53.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.212+0000 7f6baf666640 1 -- 172.21.15.98:0/4169265554 shutdown_connections 2024-09-19T08:44:53.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.212+0000 7f6baf666640 1 -- 172.21.15.98:0/4169265554 wait complete. 2024-09-19T08:44:53.229 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-19T08:44:53.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.336+0000 7f45e74ac640 1 Processor -- start 2024-09-19T08:44:53.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.336+0000 7f45e74ac640 1 -- start start 2024-09-19T08:44:53.339 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.336+0000 7f45e74ac640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f45e015c220 0x7f45e00733f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:53.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.336+0000 7f45e74ac640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45e0073930 0x7f45e00d0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:53.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.336+0000 7f45e74ac640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f45e00d1530 0x7f45e00b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:53.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.336+0000 7f45e74ac640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f45e00740b0 con 0x7f45e015c220 2024-09-19T08:44:53.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.336+0000 7f45e74ac640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f45e0073dd0 con 0x7f45e00d1530 2024-09-19T08:44:53.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.336+0000 7f45e74ac640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f45e00b6460 con 0x7f45e0073930 2024-09-19T08:44:53.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.336+0000 7f45e5248640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f45e015c220 0x7f45e00733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:53.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.336+0000 7f45e5248640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f45e015c220 0x7f45e00733f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34712/0 (socket says 172.21.15.98:34712) 2024-09-19T08:44:53.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.336+0000 7f45e5248640 1 -- 172.21.15.98:0/2384605227 learned_addr learned my addr 172.21.15.98:0/2384605227 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:53.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.336+0000 7f45e4a47640 1 --2- 172.21.15.98:0/2384605227 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45e0073930 0x7f45e00d0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:53.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.336+0000 7f45e5a49640 1 --2- 172.21.15.98:0/2384605227 >> v2:172.21.15.139:3300/0 conn(0x7f45e00d1530 0x7f45e00b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:53.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.336+0000 7f45e5248640 1 -- 172.21.15.98:0/2384605227 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45e0073930 msgr2=0x7f45e00d0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:53.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.336+0000 7f45e5248640 1 --2- 172.21.15.98:0/2384605227 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45e0073930 0x7f45e00d0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:53.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.336+0000 7f45e5248640 1 -- 172.21.15.98:0/2384605227 >> v2:172.21.15.139:3300/0 conn(0x7f45e00d1530 msgr2=0x7f45e00b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:53.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.336+0000 7f45e5248640 1 --2- 172.21.15.98:0/2384605227 >> v2:172.21.15.139:3300/0 conn(0x7f45e00d1530 0x7f45e00b6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:53.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.336+0000 7f45e5248640 1 -- 172.21.15.98:0/2384605227 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e00b6740 con 0x7f45e015c220 2024-09-19T08:44:53.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.340+0000 7f45e5248640 1 --2- 172.21.15.98:0/2384605227 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45e015c220 0x7f45e00733f0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f45e00d2340 tx=0x7f45dc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244819c16205dd4d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:53.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.340+0000 7f45d7fff640 1 -- 172.21.15.98:0/2384605227 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45dc0b1070 con 0x7f45e015c220 2024-09-19T08:44:53.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.340+0000 7f45d7fff640 1 -- 172.21.15.98:0/2384605227 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45dc0ac630 con 0x7f45e015c220 2024-09-19T08:44:53.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.340+0000 7f45d7fff640 1 -- 172.21.15.98:0/2384605227 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45dc0a8050 con 0x7f45e015c220 2024-09-19T08:44:53.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.340+0000 7f45e74ac640 1 -- 172.21.15.98:0/2384605227 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45e015c220 msgr2=0x7f45e00733f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:53.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.340+0000 7f45e74ac640 1 --2- 172.21.15.98:0/2384605227 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45e015c220 0x7f45e00733f0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f45e00d2340 tx=0x7f45dc01c520 comp rx=0 tx=0).stop 2024-09-19T08:44:53.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.340+0000 7f45e74ac640 1 -- 172.21.15.98:0/2384605227 shutdown_connections 2024-09-19T08:44:53.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.340+0000 7f45e74ac640 1 --2- 172.21.15.98:0/2384605227 >> v2:172.21.15.139:3300/0 conn(0x7f45e00d1530 0x7f45e00b6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:53.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.340+0000 7f45e74ac640 1 --2- 172.21.15.98:0/2384605227 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45e0073930 0x7f45e00d0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:53.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.340+0000 7f45e74ac640 1 --2- 172.21.15.98:0/2384605227 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45e015c220 0x7f45e00733f0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:53.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.340+0000 7f45e74ac640 1 -- 172.21.15.98:0/2384605227 >> 172.21.15.98:0/2384605227 conn(0x7f45e00b40a0 msgr2=0x7f45e00b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:53.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.340+0000 7f45e74ac640 1 -- 172.21.15.98:0/2384605227 shutdown_connections 2024-09-19T08:44:53.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.340+0000 7f45e74ac640 1 -- 172.21.15.98:0/2384605227 wait complete. 2024-09-19T08:44:53.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.340+0000 7f45e74ac640 1 Processor -- start 2024-09-19T08:44:53.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.344+0000 7f45e74ac640 1 -- start start 2024-09-19T08:44:53.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.344+0000 7f45e74ac640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45e0073930 0x7f45e0148e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:53.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.344+0000 7f45e74ac640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45e00d1530 0x7f45e0155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:53.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.344+0000 7f45e74ac640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45e015c220 0x7f45e015acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:53.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.344+0000 7f45e74ac640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f45e015ce10 con 0x7f45e015c220 2024-09-19T08:44:53.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.344+0000 7f45e74ac640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f45e00d0e40 con 0x7f45e00d1530 2024-09-19T08:44:53.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.344+0000 7f45e74ac640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f45e015cf90 con 0x7f45e0073930 2024-09-19T08:44:53.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.344+0000 7f45e5a49640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45e015c220 0x7f45e015acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:53.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.344+0000 7f45e5a49640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45e015c220 0x7f45e015acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34714/0 (socket says 172.21.15.98:34714) 2024-09-19T08:44:53.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.344+0000 7f45e5a49640 1 -- 172.21.15.98:0/1215600497 learned_addr learned my addr 172.21.15.98:0/1215600497 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:53.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.344+0000 7f45e4a47640 1 --2- 172.21.15.98:0/1215600497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45e00d1530 0x7f45e0155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:53.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.344+0000 7f45e5248640 1 --2- 172.21.15.98:0/1215600497 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45e0073930 0x7f45e0148e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:53.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.344+0000 7f45e5a49640 1 -- 172.21.15.98:0/1215600497 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45e0073930 msgr2=0x7f45e0148e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:53.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.344+0000 7f45e5a49640 1 --2- 172.21.15.98:0/1215600497 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45e0073930 0x7f45e0148e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:53.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.344+0000 7f45e5a49640 1 -- 172.21.15.98:0/1215600497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45e00d1530 msgr2=0x7f45e0155560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:53.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.344+0000 7f45e5a49640 1 --2- 172.21.15.98:0/1215600497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45e00d1530 0x7f45e0155560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:53.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.344+0000 7f45e5a49640 1 -- 172.21.15.98:0/1215600497 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e015d230 con 0x7f45e015c220 2024-09-19T08:44:53.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.344+0000 7f45e4a47640 1 --2- 172.21.15.98:0/1215600497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45e00d1530 0x7f45e0155560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:53.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.344+0000 7f45e5a49640 1 --2- 172.21.15.98:0/1215600497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45e015c220 0x7f45e015acd0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f45d8007e40 tx=0x7f45d800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:53.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.344+0000 7f45d5ffb640 1 -- 172.21.15.98:0/1215600497 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d8014620 con 0x7f45e015c220 2024-09-19T08:44:53.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.344+0000 7f45d5ffb640 1 -- 172.21.15.98:0/1215600497 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45d8019070 con 0x7f45e015c220 2024-09-19T08:44:53.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.344+0000 7f45e74ac640 1 -- 172.21.15.98:0/1215600497 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45e015d510 con 0x7f45e015c220 2024-09-19T08:44:53.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.344+0000 7f45d5ffb640 1 -- 172.21.15.98:0/1215600497 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d8014a20 con 0x7f45e015c220 2024-09-19T08:44:53.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.344+0000 7f45e74ac640 1 -- 172.21.15.98:0/1215600497 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45e015dad0 con 0x7f45e015c220 2024-09-19T08:44:53.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.344+0000 7f45e74ac640 1 -- 172.21.15.98:0/1215600497 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45e00b6f40 con 0x7f45e015c220 2024-09-19T08:44:53.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.344+0000 7f45d5ffb640 1 -- 172.21.15.98:0/1215600497 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f45d8014bc0 con 0x7f45e015c220 2024-09-19T08:44:53.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.344+0000 7f45d5ffb640 1 --2- 172.21.15.98:0/1215600497 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f45c4067cb0 0x7f45c406a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:53.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.344+0000 7f45d5ffb640 1 -- 172.21.15.98:0/1215600497 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4885+0+0 (secure 0 0 0) 0x7f45d8086990 con 0x7f45e015c220 2024-09-19T08:44:53.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.344+0000 7f45e5248640 1 --2- 172.21.15.98:0/1215600497 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f45c4067cb0 0x7f45c406a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:53.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.348+0000 7f45e5248640 1 --2- 172.21.15.98:0/1215600497 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f45c4067cb0 0x7f45c406a170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f45dc0ae000 tx=0x7f45dc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:53.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.348+0000 7f45d5ffb640 1 -- 172.21.15.98:0/1215600497 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f45d80571e0 con 0x7f45e015c220 2024-09-19T08:44:53.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.512+0000 7f45e74ac640 1 -- 172.21.15.98:0/1215600497 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f45e00707e0 con 0x7f45e015c220 2024-09-19T08:44:53.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.512+0000 7f45d5ffb640 1 -- 172.21.15.98:0/1215600497 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v114) ==== 84+0+65 (secure 0 0 0) 0x7f45d8056ae0 con 0x7f45e015c220 2024-09-19T08:44:53.514 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:44:53.514 INFO:teuthology.orchestra.run.smithi098.stdout:[ 2024-09-19T08:44:53.514 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:44:53.514 INFO:teuthology.orchestra.run.smithi098.stdout: "poolnum": 1, 2024-09-19T08:44:53.514 INFO:teuthology.orchestra.run.smithi098.stdout: "poolname": ".mgr" 2024-09-19T08:44:53.514 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:44:53.514 INFO:teuthology.orchestra.run.smithi098.stdout:] 2024-09-19T08:44:53.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.516+0000 7f45e74ac640 1 -- 172.21.15.98:0/1215600497 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f45c4067cb0 msgr2=0x7f45c406a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:53.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.516+0000 7f45e74ac640 1 --2- 172.21.15.98:0/1215600497 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f45c4067cb0 0x7f45c406a170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f45dc0ae000 tx=0x7f45dc008040 comp rx=0 tx=0).stop 2024-09-19T08:44:53.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.516+0000 7f45e74ac640 1 -- 172.21.15.98:0/1215600497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45e015c220 msgr2=0x7f45e015acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:53.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.516+0000 7f45e74ac640 1 --2- 172.21.15.98:0/1215600497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45e015c220 0x7f45e015acd0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f45d8007e40 tx=0x7f45d800e4b0 comp rx=0 tx=0).stop 2024-09-19T08:44:53.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.516+0000 7f45e74ac640 1 -- 172.21.15.98:0/1215600497 shutdown_connections 2024-09-19T08:44:53.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.516+0000 7f45e74ac640 1 --2- 172.21.15.98:0/1215600497 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f45c4067cb0 0x7f45c406a170 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:53.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.516+0000 7f45e74ac640 1 --2- 172.21.15.98:0/1215600497 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45e015c220 0x7f45e015acd0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:53.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.516+0000 7f45e74ac640 1 --2- 172.21.15.98:0/1215600497 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45e00d1530 0x7f45e0155560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:53.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.516+0000 7f45e74ac640 1 --2- 172.21.15.98:0/1215600497 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45e0073930 0x7f45e0148e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:53.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.516+0000 7f45e74ac640 1 -- 172.21.15.98:0/1215600497 >> 172.21.15.98:0/1215600497 conn(0x7f45e00b40a0 msgr2=0x7f45e015a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:53.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.516+0000 7f45e74ac640 1 -- 172.21.15.98:0/1215600497 shutdown_connections 2024-09-19T08:44:53.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.516+0000 7f45e74ac640 1 -- 172.21.15.98:0/1215600497 wait complete. 2024-09-19T08:44:53.532 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T08:44:53.532 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T08:44:53.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.640+0000 7f46e4c34640 1 Processor -- start 2024-09-19T08:44:53.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.640+0000 7f46e4c34640 1 -- start start 2024-09-19T08:44:53.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.640+0000 7f46e4c34640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f46e015c290 0x7f46e00cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:53.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.640+0000 7f46e4c34640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46e01608c0 con 0x7f46e006d4c0 2024-09-19T08:44:53.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.640+0000 7f46e4c34640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46e0160ba0 con 0x7f46e00cf700 2024-09-19T08:44:53.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.640+0000 7f46e4c34640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f46e0160e80 con 0x7f46e015c290 2024-09-19T08:44:53.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.640+0000 7f46ded9d640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f46e006d4c0 0x7f46e006d8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56540/0 (socket says 172.21.15.98:56540) 2024-09-19T08:44:53.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.640+0000 7f46ded9d640 1 -- 172.21.15.98:0/150085292 learned_addr learned my addr 172.21.15.98:0/150085292 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:53.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.640+0000 7f46de59c640 1 --2- 172.21.15.98:0/150085292 >> v2:172.21.15.139:3300/0 conn(0x7f46e015c290 0x7f46e00cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:53.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.640+0000 7f46dd59a640 1 -- 172.21.15.98:0/150085292 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 195326169 0 0) 0x7f46e01608c0 con 0x7f46e006d4c0 2024-09-19T08:44:53.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.640+0000 7f46dd59a640 1 -- 172.21.15.98:0/150085292 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46c8003530 con 0x7f46e006d4c0 2024-09-19T08:44:53.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.640+0000 7f46dd59a640 1 -- 172.21.15.98:0/150085292 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4011436447 0 0) 0x7f46c8003530 con 0x7f46e006d4c0 2024-09-19T08:44:53.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.640+0000 7f46dd59a640 1 -- 172.21.15.98:0/150085292 >> v2:172.21.15.139:3300/0 conn(0x7f46e015c290 msgr2=0x7f46e00cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:53.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.640+0000 7f46dd59a640 1 --2- 172.21.15.98:0/150085292 >> v2:172.21.15.139:3300/0 conn(0x7f46e015c290 0x7f46e00cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:53.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.640+0000 7f46dd59a640 1 -- 172.21.15.98:0/150085292 >> v1:172.21.15.139:6789/0 conn(0x7f46e00cf700 legacy=0x7f46e015ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:53.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.640+0000 7f46dd59a640 1 -- 172.21.15.98:0/150085292 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e00cd1a0 con 0x7f46e006d4c0 2024-09-19T08:44:53.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.640+0000 7f46dd59a640 1 -- 172.21.15.98:0/150085292 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f46d8001a00 con 0x7f46e006d4c0 2024-09-19T08:44:53.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.640+0000 7f46dd59a640 1 -- 172.21.15.98:0/150085292 >> v1:172.21.15.98:6789/0 conn(0x7f46e006d4c0 legacy=0x7f46e006d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:53.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.640+0000 7f46dd59a640 1 --2- 172.21.15.98:0/150085292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f46c8003a10 0x7f46c8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:53.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.640+0000 7f46dd59a640 1 --2- 172.21.15.98:0/150085292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46c8004810 0x7f46c8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:53.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.640+0000 7f46dd59a640 1 --2- 172.21.15.98:0/150085292 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f46c8005220 0x7f46c800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:53.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.640+0000 7f46dd59a640 1 -- 172.21.15.98:0/150085292 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f46c800c140 con 0x7f46c8003a10 2024-09-19T08:44:53.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.640+0000 7f46dd59a640 1 -- 172.21.15.98:0/150085292 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f46c800c2f0 con 0x7f46c8004810 2024-09-19T08:44:53.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.640+0000 7f46dd59a640 1 -- 172.21.15.98:0/150085292 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f46c800c4f0 con 0x7f46c8005220 2024-09-19T08:44:53.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.640+0000 7f46de59c640 1 --2- 172.21.15.98:0/150085292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f46c8003a10 0x7f46c8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:53.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.640+0000 7f46ded9d640 1 --2- 172.21.15.98:0/150085292 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f46c8005220 0x7f46c800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:53.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.640+0000 7f46de59c640 1 -- 172.21.15.98:0/150085292 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f46c8005220 msgr2=0x7f46c800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:53.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.640+0000 7f46de59c640 1 --2- 172.21.15.98:0/150085292 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f46c8005220 0x7f46c800b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:53.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.640+0000 7f46ddd9b640 1 --2- 172.21.15.98:0/150085292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46c8004810 0x7f46c8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:53.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.640+0000 7f46de59c640 1 -- 172.21.15.98:0/150085292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46c8004810 msgr2=0x7f46c8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:53.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.644+0000 7f46de59c640 1 --2- 172.21.15.98:0/150085292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46c8004810 0x7f46c8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:53.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.644+0000 7f46ded9d640 1 -- 172.21.15.98:0/150085292 reap_dead start 2024-09-19T08:44:53.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.644+0000 7f46de59c640 1 -- 172.21.15.98:0/150085292 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c8003530 con 0x7f46c8003a10 2024-09-19T08:44:53.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.644+0000 7f46ddd9b640 1 --2- 172.21.15.98:0/150085292 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46c8004810 0x7f46c8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:53.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.644+0000 7f46de59c640 1 --2- 172.21.15.98:0/150085292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f46c8003a10 0x7f46c8003e30 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f46d0009ab0 tx=0x7f46d001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c1b954c1d77f85 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:53.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.644+0000 7f46dd59a640 1 -- 172.21.15.98:0/150085292 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d00096b0 con 0x7f46c8003a10 2024-09-19T08:44:53.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.644+0000 7f46dd59a640 1 -- 172.21.15.98:0/150085292 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46d00a6430 con 0x7f46c8003a10 2024-09-19T08:44:53.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.644+0000 7f46dd59a640 1 -- 172.21.15.98:0/150085292 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d00a6820 con 0x7f46c8003a10 2024-09-19T08:44:53.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.644+0000 7f46e4c34640 1 -- 172.21.15.98:0/150085292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f46c8003a10 msgr2=0x7f46c8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:53.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.644+0000 7f46e4c34640 1 --2- 172.21.15.98:0/150085292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f46c8003a10 0x7f46c8003e30 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f46d0009ab0 tx=0x7f46d001ca70 comp rx=0 tx=0).stop 2024-09-19T08:44:53.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.644+0000 7f46e4c34640 1 -- 172.21.15.98:0/150085292 shutdown_connections 2024-09-19T08:44:53.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.644+0000 7f46e4c34640 1 --2- 172.21.15.98:0/150085292 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f46c8003a10 0x7f46c8003e30 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:53.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.644+0000 7f46e4c34640 1 -- 172.21.15.98:0/150085292 >> 172.21.15.98:0/150085292 conn(0x7f46e006c090 msgr2=0x7f46e006c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:53.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.644+0000 7f46e4c34640 1 -- 172.21.15.98:0/150085292 shutdown_connections 2024-09-19T08:44:53.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.644+0000 7f46e4c34640 1 -- 172.21.15.98:0/150085292 wait complete. 2024-09-19T08:44:53.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.648+0000 7f46e4c34640 1 Processor -- start 2024-09-19T08:44:53.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.648+0000 7f46e4c34640 1 -- start start 2024-09-19T08:44:53.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.648+0000 7f46e4c34640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46c8003a10 0x7f46e00aa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:53.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.648+0000 7f46e4c34640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f46e0155420 0x7f46e00ac430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:53.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.648+0000 7f46e4c34640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f46e00ac970 0x7f46e00ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:53.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.648+0000 7f46e4c34640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f46e0160320 con 0x7f46e00ac970 2024-09-19T08:44:53.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.648+0000 7f46e4c34640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f46e01601a0 con 0x7f46c8003a10 2024-09-19T08:44:53.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.648+0000 7f46e4c34640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f46e01604a0 con 0x7f46e0155420 2024-09-19T08:44:53.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.648+0000 7f46ded9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f46e00ac970 0x7f46e00ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:53.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.648+0000 7f46ded9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f46e00ac970 0x7f46e00ad0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34734/0 (socket says 172.21.15.98:34734) 2024-09-19T08:44:53.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.648+0000 7f46ded9d640 1 -- 172.21.15.98:0/1423331390 learned_addr learned my addr 172.21.15.98:0/1423331390 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:53.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.648+0000 7f46ddd9b640 1 --2- 172.21.15.98:0/1423331390 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f46e0155420 0x7f46e00ac430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:53.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.648+0000 7f46de59c640 1 --2- 172.21.15.98:0/1423331390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46c8003a10 0x7f46e00aa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:53.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.648+0000 7f46ded9d640 1 -- 172.21.15.98:0/1423331390 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f46e0155420 msgr2=0x7f46e00ac430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:53.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.648+0000 7f46ded9d640 1 --2- 172.21.15.98:0/1423331390 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f46e0155420 0x7f46e00ac430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:53.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.648+0000 7f46ded9d640 1 -- 172.21.15.98:0/1423331390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46c8003a10 msgr2=0x7f46e00aa7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:53.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.648+0000 7f46ded9d640 1 --2- 172.21.15.98:0/1423331390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46c8003a10 0x7f46e00aa7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:53.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.648+0000 7f46ded9d640 1 -- 172.21.15.98:0/1423331390 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e00aacf0 con 0x7f46e00ac970 2024-09-19T08:44:53.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.648+0000 7f46ded9d640 1 --2- 172.21.15.98:0/1423331390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f46e00ac970 0x7f46e00ad0e0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f46d8000f90 tx=0x7f46d800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:53.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.648+0000 7f46ceffd640 1 -- 172.21.15.98:0/1423331390 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d8004d70 con 0x7f46e00ac970 2024-09-19T08:44:53.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.648+0000 7f46ceffd640 1 -- 172.21.15.98:0/1423331390 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46d800bb80 con 0x7f46e00ac970 2024-09-19T08:44:53.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.648+0000 7f46e4c34640 1 -- 172.21.15.98:0/1423331390 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46e00ab390 con 0x7f46e00ac970 2024-09-19T08:44:53.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.648+0000 7f46ceffd640 1 -- 172.21.15.98:0/1423331390 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d8007070 con 0x7f46e00ac970 2024-09-19T08:44:53.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.648+0000 7f46e4c34640 1 -- 172.21.15.98:0/1423331390 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46e00ab950 con 0x7f46e00ac970 2024-09-19T08:44:53.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.648+0000 7f46e4c34640 1 -- 172.21.15.98:0/1423331390 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46e015e9d0 con 0x7f46e00ac970 2024-09-19T08:44:53.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.648+0000 7f46ceffd640 1 -- 172.21.15.98:0/1423331390 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f46d801b020 con 0x7f46e00ac970 2024-09-19T08:44:53.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.648+0000 7f46ceffd640 1 --2- 172.21.15.98:0/1423331390 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f46c0067cb0 0x7f46c006a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:53.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.652+0000 7f46ceffd640 1 -- 172.21.15.98:0/1423331390 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 4885+0+0 (secure 0 0 0) 0x7f46d8086280 con 0x7f46e00ac970 2024-09-19T08:44:53.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.652+0000 7f46de59c640 1 --2- 172.21.15.98:0/1423331390 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f46c0067cb0 0x7f46c006a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:53.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.652+0000 7f46de59c640 1 --2- 172.21.15.98:0/1423331390 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f46c0067cb0 0x7f46c006a170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f46d00ad000 tx=0x7f46d00046d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:53.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.656+0000 7f46ceffd640 1 -- 172.21.15.98:0/1423331390 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f46e00aacf0 con 0x7f46e00ac970 2024-09-19T08:44:53.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:53.816+0000 7f46e4c34640 1 -- 172.21.15.98:0/1423331390 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f46e015ecb0 con 0x7f46e00ac970 2024-09-19T08:44:54.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.528+0000 7f46ceffd640 1 -- 172.21.15.98:0/1423331390 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v115) ==== 138+0+0 (secure 0 0 0) 0x7f46d8056b00 con 0x7f46e00ac970 2024-09-19T08:44:54.530 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_metadata' created 2024-09-19T08:44:54.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.532+0000 7f46e4c34640 1 -- 172.21.15.98:0/1423331390 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f46c0067cb0 msgr2=0x7f46c006a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:54.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.532+0000 7f46e4c34640 1 --2- 172.21.15.98:0/1423331390 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f46c0067cb0 0x7f46c006a170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f46d00ad000 tx=0x7f46d00046d0 comp rx=0 tx=0).stop 2024-09-19T08:44:54.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.532+0000 7f46e4c34640 1 -- 172.21.15.98:0/1423331390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f46e00ac970 msgr2=0x7f46e00ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:54.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.532+0000 7f46e4c34640 1 --2- 172.21.15.98:0/1423331390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f46e00ac970 0x7f46e00ad0e0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f46d8000f90 tx=0x7f46d800b4a0 comp rx=0 tx=0).stop 2024-09-19T08:44:54.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.532+0000 7f46e4c34640 1 -- 172.21.15.98:0/1423331390 shutdown_connections 2024-09-19T08:44:54.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.532+0000 7f46e4c34640 1 --2- 172.21.15.98:0/1423331390 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f46c0067cb0 0x7f46c006a170 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:54.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.532+0000 7f46e4c34640 1 --2- 172.21.15.98:0/1423331390 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f46e00ac970 0x7f46e00ad0e0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:54.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.532+0000 7f46e4c34640 1 --2- 172.21.15.98:0/1423331390 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f46e0155420 0x7f46e00ac430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:54.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.532+0000 7f46e4c34640 1 --2- 172.21.15.98:0/1423331390 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f46c8003a10 0x7f46e00aa7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:54.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.532+0000 7f46e4c34640 1 -- 172.21.15.98:0/1423331390 >> 172.21.15.98:0/1423331390 conn(0x7f46e006c090 msgr2=0x7f46e0073240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:54.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.532+0000 7f46e4c34640 1 -- 172.21.15.98:0/1423331390 shutdown_connections 2024-09-19T08:44:54.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.532+0000 7f46e4c34640 1 -- 172.21.15.98:0/1423331390 wait complete. 2024-09-19T08:44:54.561 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T08:44:54.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.688+0000 7f35659a3640 1 Processor -- start 2024-09-19T08:44:54.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.688+0000 7f35659a3640 1 -- start start 2024-09-19T08:44:54.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.688+0000 7f35659a3640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3560145b70 0x7f356006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:54.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.688+0000 7f35659a3640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f356006c5f0 con 0x7f3560071ac0 2024-09-19T08:44:54.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.688+0000 7f35659a3640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3560073530 con 0x7f356006c1d0 2024-09-19T08:44:54.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.688+0000 7f35659a3640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3560071f20 con 0x7f3560145b70 2024-09-19T08:44:54.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.688+0000 7f355f7fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f3560071ac0 0x7f35600b3410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56542/0 (socket says 172.21.15.98:56542) 2024-09-19T08:44:54.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.688+0000 7f355f7fe640 1 -- 172.21.15.98:0/2711684959 learned_addr learned my addr 172.21.15.98:0/2711684959 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:54.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.688+0000 7f355effd640 1 --2- 172.21.15.98:0/2711684959 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3560145b70 0x7f356006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:54.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.688+0000 7f355dffb640 1 -- 172.21.15.98:0/2711684959 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2292940416 0 0) 0x7f356006c5f0 con 0x7f3560071ac0 2024-09-19T08:44:54.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.688+0000 7f355dffb640 1 -- 172.21.15.98:0/2711684959 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3548003530 con 0x7f3560071ac0 2024-09-19T08:44:54.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.688+0000 7f355dffb640 1 -- 172.21.15.98:0/2711684959 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1202149544 0 0) 0x7f3548003530 con 0x7f3560071ac0 2024-09-19T08:44:54.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.688+0000 7f355dffb640 1 -- 172.21.15.98:0/2711684959 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3560145b70 msgr2=0x7f356006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:54.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.688+0000 7f355dffb640 1 --2- 172.21.15.98:0/2711684959 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3560145b70 0x7f356006bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:54.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.688+0000 7f355dffb640 1 -- 172.21.15.98:0/2711684959 >> v1:172.21.15.139:6789/0 conn(0x7f356006c1d0 legacy=0x7f35600713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:54.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.688+0000 7f355dffb640 1 -- 172.21.15.98:0/2711684959 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3560073810 con 0x7f3560071ac0 2024-09-19T08:44:54.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.692+0000 7f355dffb640 1 -- 172.21.15.98:0/2711684959 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f3558001a00 con 0x7f3560071ac0 2024-09-19T08:44:54.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.692+0000 7f355dffb640 1 -- 172.21.15.98:0/2711684959 >> v1:172.21.15.98:6789/0 conn(0x7f3560071ac0 legacy=0x7f35600b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:54.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.692+0000 7f355dffb640 1 --2- 172.21.15.98:0/2711684959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35480039f0 0x7f3548003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:54.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.692+0000 7f355dffb640 1 --2- 172.21.15.98:0/2711684959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3548004810 0x7f3548004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:54.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.692+0000 7f355dffb640 1 --2- 172.21.15.98:0/2711684959 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3560145b70 0x7f3548005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:54.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.692+0000 7f355dffb640 1 -- 172.21.15.98:0/2711684959 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3560071f20 con 0x7f35480039f0 2024-09-19T08:44:54.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.692+0000 7f355dffb640 1 -- 172.21.15.98:0/2711684959 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f354800bb00 con 0x7f3548004810 2024-09-19T08:44:54.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.692+0000 7f355dffb640 1 -- 172.21.15.98:0/2711684959 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f354800bd00 con 0x7f3560145b70 2024-09-19T08:44:54.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.692+0000 7f355effd640 1 --2- 172.21.15.98:0/2711684959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35480039f0 0x7f3548003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:54.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.692+0000 7f355e7fc640 1 --2- 172.21.15.98:0/2711684959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3548004810 0x7f3548004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:54.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.692+0000 7f355effd640 1 -- 172.21.15.98:0/2711684959 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3560145b70 msgr2=0x7f3548005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:54.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.692+0000 7f355effd640 1 --2- 172.21.15.98:0/2711684959 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3560145b70 0x7f3548005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:54.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.692+0000 7f355effd640 1 -- 172.21.15.98:0/2711684959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3548004810 msgr2=0x7f3548004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:54.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.692+0000 7f355effd640 1 --2- 172.21.15.98:0/2711684959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3548004810 0x7f3548004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:54.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.692+0000 7f355effd640 1 -- 172.21.15.98:0/2711684959 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3548003530 con 0x7f35480039f0 2024-09-19T08:44:54.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.692+0000 7f355e7fc640 1 --2- 172.21.15.98:0/2711684959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3548004810 0x7f3548004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:54.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.692+0000 7f355effd640 1 --2- 172.21.15.98:0/2711684959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35480039f0 0x7f3548003e10 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f3550009ae0 tx=0x7f355001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f8afaab9c90cc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:54.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.692+0000 7f355dffb640 1 -- 172.21.15.98:0/2711684959 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35500096b0 con 0x7f35480039f0 2024-09-19T08:44:54.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.692+0000 7f355dffb640 1 -- 172.21.15.98:0/2711684959 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35500a6490 con 0x7f35480039f0 2024-09-19T08:44:54.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.692+0000 7f355dffb640 1 -- 172.21.15.98:0/2711684959 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35500a6880 con 0x7f35480039f0 2024-09-19T08:44:54.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.692+0000 7f35659a3640 1 -- 172.21.15.98:0/2711684959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35480039f0 msgr2=0x7f3548003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:54.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.692+0000 7f35659a3640 1 --2- 172.21.15.98:0/2711684959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35480039f0 0x7f3548003e10 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f3550009ae0 tx=0x7f355001cab0 comp rx=0 tx=0).stop 2024-09-19T08:44:54.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.692+0000 7f355f7fe640 1 -- 172.21.15.98:0/2711684959 reap_dead start 2024-09-19T08:44:54.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.692+0000 7f35659a3640 1 -- 172.21.15.98:0/2711684959 shutdown_connections 2024-09-19T08:44:54.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.692+0000 7f35659a3640 1 -- 172.21.15.98:0/2711684959 >> 172.21.15.98:0/2711684959 conn(0x7f35600b2010 msgr2=0x7f35600b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:54.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.692+0000 7f35659a3640 1 -- 172.21.15.98:0/2711684959 shutdown_connections 2024-09-19T08:44:54.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.692+0000 7f35659a3640 1 -- 172.21.15.98:0/2711684959 wait complete. 2024-09-19T08:44:54.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.696+0000 7f35659a3640 1 Processor -- start 2024-09-19T08:44:54.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.696+0000 7f35659a3640 1 -- start start 2024-09-19T08:44:54.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.696+0000 7f35659a3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35480039f0 0x7f35601fb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:54.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.696+0000 7f35659a3640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3560139660 0x7f356015a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:54.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.696+0000 7f35659a3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35601417f0 0x7f3560141c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:54.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.696+0000 7f35659a3640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f35600b44b0 con 0x7f35480039f0 2024-09-19T08:44:54.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.696+0000 7f35659a3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f35600b4330 con 0x7f35601417f0 2024-09-19T08:44:54.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.696+0000 7f35659a3640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f35600b4630 con 0x7f3560139660 2024-09-19T08:44:54.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.696+0000 7f355effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35480039f0 0x7f35601fb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:54.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.696+0000 7f355effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35480039f0 0x7f35601fb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34746/0 (socket says 172.21.15.98:34746) 2024-09-19T08:44:54.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.696+0000 7f355effd640 1 -- 172.21.15.98:0/3346092967 learned_addr learned my addr 172.21.15.98:0/3346092967 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:54.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.696+0000 7f355f7fe640 1 --2- 172.21.15.98:0/3346092967 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35601417f0 0x7f3560141c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:54.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.696+0000 7f355e7fc640 1 --2- 172.21.15.98:0/3346092967 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3560139660 0x7f356015a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:54.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.696+0000 7f355effd640 1 -- 172.21.15.98:0/3346092967 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3560139660 msgr2=0x7f356015a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:54.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.696+0000 7f355effd640 1 --2- 172.21.15.98:0/3346092967 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3560139660 0x7f356015a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:54.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.696+0000 7f355effd640 1 -- 172.21.15.98:0/3346092967 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35601417f0 msgr2=0x7f3560141c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:54.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.696+0000 7f355effd640 1 --2- 172.21.15.98:0/3346092967 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35601417f0 0x7f3560141c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:54.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.696+0000 7f355effd640 1 -- 172.21.15.98:0/3346092967 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f356013f0c0 con 0x7f35480039f0 2024-09-19T08:44:54.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.696+0000 7f355effd640 1 --2- 172.21.15.98:0/3346092967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35480039f0 0x7f35601fb9e0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f355001cf90 tx=0x7f3550019850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:54.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.696+0000 7f3547fff640 1 -- 172.21.15.98:0/3346092967 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35500a6bc0 con 0x7f35480039f0 2024-09-19T08:44:54.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.696+0000 7f3547fff640 1 -- 172.21.15.98:0/3346092967 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35500a6d60 con 0x7f35480039f0 2024-09-19T08:44:54.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.696+0000 7f35659a3640 1 -- 172.21.15.98:0/3346092967 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f356013f3a0 con 0x7f35480039f0 2024-09-19T08:44:54.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.696+0000 7f3547fff640 1 -- 172.21.15.98:0/3346092967 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35500a76b0 con 0x7f35480039f0 2024-09-19T08:44:54.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.696+0000 7f35659a3640 1 -- 172.21.15.98:0/3346092967 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3560139ae0 con 0x7f35480039f0 2024-09-19T08:44:54.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.696+0000 7f35659a3640 1 -- 172.21.15.98:0/3346092967 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35600713b0 con 0x7f35480039f0 2024-09-19T08:44:54.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.696+0000 7f3547fff640 1 -- 172.21.15.98:0/3346092967 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f35500c1060 con 0x7f35480039f0 2024-09-19T08:44:54.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.700+0000 7f3547fff640 1 --2- 172.21.15.98:0/3346092967 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f354c067ce0 0x7f354c06a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:54.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.700+0000 7f3547fff640 1 -- 172.21.15.98:0/3346092967 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5255+0+0 (secure 0 0 0) 0x7f35501201d0 con 0x7f35480039f0 2024-09-19T08:44:54.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.700+0000 7f355e7fc640 1 --2- 172.21.15.98:0/3346092967 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f354c067ce0 0x7f354c06a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:54.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.700+0000 7f355e7fc640 1 --2- 172.21.15.98:0/3346092967 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f354c067ce0 0x7f354c06a1a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3554000c10 tx=0x7f3554009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:54.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.704+0000 7f3547fff640 1 -- 172.21.15.98:0/3346092967 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f35500b5070 con 0x7f35480039f0 2024-09-19T08:44:54.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:54.864+0000 7f35659a3640 1 -- 172.21.15.98:0/3346092967 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f35601451a0 con 0x7f35480039f0 2024-09-19T08:44:55.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.532+0000 7f3547fff640 1 -- 172.21.15.98:0/3346092967 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v116) ==== 170+0+0 (secure 0 0 0) 0x7f35500ac020 con 0x7f35480039f0 2024-09-19T08:44:55.536 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' created 2024-09-19T08:44:55.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.536+0000 7f35659a3640 1 -- 172.21.15.98:0/3346092967 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f354c067ce0 msgr2=0x7f354c06a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:55.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.536+0000 7f35659a3640 1 --2- 172.21.15.98:0/3346092967 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f354c067ce0 0x7f354c06a1a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3554000c10 tx=0x7f3554009040 comp rx=0 tx=0).stop 2024-09-19T08:44:55.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.536+0000 7f35659a3640 1 -- 172.21.15.98:0/3346092967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35480039f0 msgr2=0x7f35601fb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:55.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.536+0000 7f35659a3640 1 --2- 172.21.15.98:0/3346092967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35480039f0 0x7f35601fb9e0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f355001cf90 tx=0x7f3550019850 comp rx=0 tx=0).stop 2024-09-19T08:44:55.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.536+0000 7f35659a3640 1 -- 172.21.15.98:0/3346092967 shutdown_connections 2024-09-19T08:44:55.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.536+0000 7f35659a3640 1 --2- 172.21.15.98:0/3346092967 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f354c067ce0 0x7f354c06a1a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:55.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.536+0000 7f35659a3640 1 --2- 172.21.15.98:0/3346092967 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35601417f0 0x7f3560141c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:55.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.536+0000 7f35659a3640 1 --2- 172.21.15.98:0/3346092967 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3560139660 0x7f356015a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:55.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.536+0000 7f35659a3640 1 --2- 172.21.15.98:0/3346092967 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35480039f0 0x7f35601fb9e0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:55.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.536+0000 7f35659a3640 1 -- 172.21.15.98:0/3346092967 >> 172.21.15.98:0/3346092967 conn(0x7f35600b2010 msgr2=0x7f35600b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:55.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.540+0000 7f35659a3640 1 -- 172.21.15.98:0/3346092967 shutdown_connections 2024-09-19T08:44:55.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.540+0000 7f35659a3640 1 -- 172.21.15.98:0/3346092967 wait complete. 2024-09-19T08:44:55.576 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T08:44:55.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.696+0000 7f2c93efc640 1 Processor -- start 2024-09-19T08:44:55.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.696+0000 7f2c93efc640 1 -- start start 2024-09-19T08:44:55.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.700+0000 7f2c93efc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2c8c0729f0 0x7f2c8c06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:55.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.700+0000 7f2c93efc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2c8c06c1d0 0x7f2c8c0cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:55.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.700+0000 7f2c93efc640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c8c072ea0 con 0x7f2c8c144d10 2024-09-19T08:44:55.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.700+0000 7f2c93efc640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f2c8c06c620 con 0x7f2c8c0729f0 2024-09-19T08:44:55.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.700+0000 7f2c93efc640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f2c8c073ab0 con 0x7f2c8c06c1d0 2024-09-19T08:44:55.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.700+0000 7f2c91497640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2c8c0729f0 0x7f2c8c06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:55.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.700+0000 7f2c91497640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2c8c0729f0 0x7f2c8c06bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34756/0 (socket says 172.21.15.98:34756) 2024-09-19T08:44:55.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.700+0000 7f2c91497640 1 -- 172.21.15.98:0/2254331416 learned_addr learned my addr 172.21.15.98:0/2254331416 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:55.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.700+0000 7f2c91c98640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f2c8c144d10 0x7f2c8c0722e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56548/0 (socket says 172.21.15.98:56548) 2024-09-19T08:44:55.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.700+0000 7f2c91497640 1 -- 172.21.15.98:0/2254331416 >> v2:172.21.15.139:3300/0 conn(0x7f2c8c06c1d0 msgr2=0x7f2c8c0cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:55.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.700+0000 7f2c92499640 1 --2- 172.21.15.98:0/2254331416 >> v2:172.21.15.139:3300/0 conn(0x7f2c8c06c1d0 0x7f2c8c0cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:55.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.700+0000 7f2c91497640 1 --2- 172.21.15.98:0/2254331416 >> v2:172.21.15.139:3300/0 conn(0x7f2c8c06c1d0 0x7f2c8c0cd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:55.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.700+0000 7f2c91497640 1 -- 172.21.15.98:0/2254331416 >> v1:172.21.15.98:6789/0 conn(0x7f2c8c144d10 legacy=0x7f2c8c0722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:55.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.700+0000 7f2c90c96640 1 -- 172.21.15.98:0/2254331416 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1400166311 0 0) 0x7f2c8c072ea0 con 0x7f2c8c144d10 2024-09-19T08:44:55.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.700+0000 7f2c91497640 1 -- 172.21.15.98:0/2254331416 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c8c073180 con 0x7f2c8c0729f0 2024-09-19T08:44:55.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.700+0000 7f2c92499640 1 --2- 172.21.15.98:0/2254331416 >> v2:172.21.15.139:3300/0 conn(0x7f2c8c06c1d0 0x7f2c8c0cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:55.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.700+0000 7f2c91497640 1 --2- 172.21.15.98:0/2254331416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c8c0729f0 0x7f2c8c06bc90 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c06f0b0 tx=0x7f2c8001ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6b10d3b492de54 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:55.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.700+0000 7f2c90c96640 1 -- 172.21.15.98:0/2254331416 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c800b4070 con 0x7f2c8c0729f0 2024-09-19T08:44:55.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.700+0000 7f2c90c96640 1 -- 172.21.15.98:0/2254331416 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c800a7bb0 con 0x7f2c8c0729f0 2024-09-19T08:44:55.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.700+0000 7f2c90c96640 1 -- 172.21.15.98:0/2254331416 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c800af440 con 0x7f2c8c0729f0 2024-09-19T08:44:55.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.700+0000 7f2c93efc640 1 -- 172.21.15.98:0/2254331416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c8c0729f0 msgr2=0x7f2c8c06bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:55.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.700+0000 7f2c93efc640 1 --2- 172.21.15.98:0/2254331416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c8c0729f0 0x7f2c8c06bc90 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c06f0b0 tx=0x7f2c8001ebf0 comp rx=0 tx=0).stop 2024-09-19T08:44:55.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.700+0000 7f2c93efc640 1 -- 172.21.15.98:0/2254331416 shutdown_connections 2024-09-19T08:44:55.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.700+0000 7f2c93efc640 1 --2- 172.21.15.98:0/2254331416 >> v2:172.21.15.139:3300/0 conn(0x7f2c8c06c1d0 0x7f2c8c0cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:55.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.700+0000 7f2c93efc640 1 --2- 172.21.15.98:0/2254331416 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c8c0729f0 0x7f2c8c06bc90 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:55.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.704+0000 7f2c93efc640 1 -- 172.21.15.98:0/2254331416 >> 172.21.15.98:0/2254331416 conn(0x7f2c8c0b2010 msgr2=0x7f2c8c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:55.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.704+0000 7f2c93efc640 1 -- 172.21.15.98:0/2254331416 shutdown_connections 2024-09-19T08:44:55.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.704+0000 7f2c93efc640 1 -- 172.21.15.98:0/2254331416 wait complete. 2024-09-19T08:44:55.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.704+0000 7f2c93efc640 1 Processor -- start 2024-09-19T08:44:55.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.704+0000 7f2c93efc640 1 -- start start 2024-09-19T08:44:55.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.704+0000 7f2c93efc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c8c06c1d0 0x7f2c8c141da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:55.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.704+0000 7f2c93efc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c8c0729f0 0x7f2c8c0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:55.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.704+0000 7f2c93efc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c8c144d10 0x7f2c8c0aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:55.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.704+0000 7f2c93efc640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8c0ce120 con 0x7f2c8c0729f0 2024-09-19T08:44:55.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.704+0000 7f2c93efc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8c0cdfa0 con 0x7f2c8c144d10 2024-09-19T08:44:55.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.704+0000 7f2c93efc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2c8c0ce2a0 con 0x7f2c8c06c1d0 2024-09-19T08:44:55.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.704+0000 7f2c91497640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c8c0729f0 0x7f2c8c0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:55.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.704+0000 7f2c91497640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c8c0729f0 0x7f2c8c0ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34760/0 (socket says 172.21.15.98:34760) 2024-09-19T08:44:55.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.704+0000 7f2c91497640 1 -- 172.21.15.98:0/3504324449 learned_addr learned my addr 172.21.15.98:0/3504324449 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:55.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.704+0000 7f2c91c98640 1 --2- 172.21.15.98:0/3504324449 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c8c06c1d0 0x7f2c8c141da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:55.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.704+0000 7f2c92499640 1 --2- 172.21.15.98:0/3504324449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c8c144d10 0x7f2c8c0aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:55.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.704+0000 7f2c91497640 1 -- 172.21.15.98:0/3504324449 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c8c06c1d0 msgr2=0x7f2c8c141da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:55.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.704+0000 7f2c91497640 1 --2- 172.21.15.98:0/3504324449 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c8c06c1d0 0x7f2c8c141da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:55.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.704+0000 7f2c91497640 1 -- 172.21.15.98:0/3504324449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c8c144d10 msgr2=0x7f2c8c0aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:55.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.704+0000 7f2c91497640 1 --2- 172.21.15.98:0/3504324449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c8c144d10 0x7f2c8c0aedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:55.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.704+0000 7f2c91497640 1 -- 172.21.15.98:0/3504324449 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c8c1424d0 con 0x7f2c8c0729f0 2024-09-19T08:44:55.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.704+0000 7f2c91497640 1 --2- 172.21.15.98:0/3504324449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c8c0729f0 0x7f2c8c0ae8b0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f2c8001b800 tx=0x7f2c8001d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:55.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.704+0000 7f2c7e7fc640 1 -- 172.21.15.98:0/3504324449 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c800b4070 con 0x7f2c8c0729f0 2024-09-19T08:44:55.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.704+0000 7f2c7e7fc640 1 -- 172.21.15.98:0/3504324449 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c80004020 con 0x7f2c8c0729f0 2024-09-19T08:44:55.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.704+0000 7f2c93efc640 1 -- 172.21.15.98:0/3504324449 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c8c1427b0 con 0x7f2c8c0729f0 2024-09-19T08:44:55.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.704+0000 7f2c7e7fc640 1 -- 172.21.15.98:0/3504324449 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c800043d0 con 0x7f2c8c0729f0 2024-09-19T08:44:55.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.704+0000 7f2c93efc640 1 -- 172.21.15.98:0/3504324449 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c8c142d70 con 0x7f2c8c0729f0 2024-09-19T08:44:55.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.704+0000 7f2c93efc640 1 -- 172.21.15.98:0/3504324449 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c8c0cd230 con 0x7f2c8c0729f0 2024-09-19T08:44:55.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.708+0000 7f2c7e7fc640 1 -- 172.21.15.98:0/3504324449 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f2c800c1060 con 0x7f2c8c0729f0 2024-09-19T08:44:55.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.708+0000 7f2c7e7fc640 1 --2- 172.21.15.98:0/3504324449 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2c70067cb0 0x7f2c7006a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:55.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.708+0000 7f2c7e7fc640 1 -- 172.21.15.98:0/3504324449 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5637+0+0 (secure 0 0 0) 0x7f2c80120b80 con 0x7f2c8c0729f0 2024-09-19T08:44:55.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.708+0000 7f2c91c98640 1 --2- 172.21.15.98:0/3504324449 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2c70067cb0 0x7f2c7006a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:55.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.708+0000 7f2c91c98640 1 --2- 172.21.15.98:0/3504324449 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2c70067cb0 0x7f2c7006a170 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2c88000f70 tx=0x7f2c88007c10 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:55.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.712+0000 7f2c7e7fc640 1 -- 172.21.15.98:0/3504324449 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2c800187e0 con 0x7f2c8c0729f0 2024-09-19T08:44:55.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:55.904+0000 7f2c93efc640 1 -- 172.21.15.98:0/3504324449 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f2c8c0cf530 con 0x7f2c8c0729f0 2024-09-19T08:44:56.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.620+0000 7f2c7e7fc640 1 -- 172.21.15.98:0/3504324449 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v65) ==== 174+0+0 (secure 0 0 0) 0x7f2c800ae3a0 con 0x7f2c8c0729f0 2024-09-19T08:44:56.627 INFO:teuthology.orchestra.run.smithi098.stderr:new fs with metadata pool 18 and data pool 19 2024-09-19T08:44:56.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.632+0000 7f2c93efc640 1 -- 172.21.15.98:0/3504324449 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2c70067cb0 msgr2=0x7f2c7006a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:56.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.632+0000 7f2c93efc640 1 --2- 172.21.15.98:0/3504324449 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2c70067cb0 0x7f2c7006a170 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2c88000f70 tx=0x7f2c88007c10 comp rx=0 tx=0).stop 2024-09-19T08:44:56.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.632+0000 7f2c93efc640 1 -- 172.21.15.98:0/3504324449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c8c0729f0 msgr2=0x7f2c8c0ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:56.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.632+0000 7f2c93efc640 1 --2- 172.21.15.98:0/3504324449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c8c0729f0 0x7f2c8c0ae8b0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f2c8001b800 tx=0x7f2c8001d430 comp rx=0 tx=0).stop 2024-09-19T08:44:56.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.632+0000 7f2c93efc640 1 -- 172.21.15.98:0/3504324449 shutdown_connections 2024-09-19T08:44:56.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.632+0000 7f2c93efc640 1 --2- 172.21.15.98:0/3504324449 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2c70067cb0 0x7f2c7006a170 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:56.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.632+0000 7f2c93efc640 1 --2- 172.21.15.98:0/3504324449 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c8c144d10 0x7f2c8c0aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:56.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.632+0000 7f2c93efc640 1 --2- 172.21.15.98:0/3504324449 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2c8c0729f0 0x7f2c8c0ae8b0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:56.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.632+0000 7f2c93efc640 1 --2- 172.21.15.98:0/3504324449 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2c8c06c1d0 0x7f2c8c141da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:56.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.632+0000 7f2c93efc640 1 -- 172.21.15.98:0/3504324449 >> 172.21.15.98:0/3504324449 conn(0x7f2c8c0b2010 msgr2=0x7f2c8c15de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:56.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.632+0000 7f2c93efc640 1 -- 172.21.15.98:0/3504324449 shutdown_connections 2024-09-19T08:44:56.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.632+0000 7f2c93efc640 1 -- 172.21.15.98:0/3504324449 wait complete. 2024-09-19T08:44:56.658 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:44:56.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.768+0000 7f93551df640 1 Processor -- start 2024-09-19T08:44:56.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.768+0000 7f93551df640 1 -- start start 2024-09-19T08:44:56.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.768+0000 7f93551df640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f93500d4820 0x7f93501621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:56.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.768+0000 7f93551df640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9350068440 con 0x7f93500b7f80 2024-09-19T08:44:56.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.768+0000 7f93551df640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9350162a90 con 0x7f93500b9970 2024-09-19T08:44:56.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.768+0000 7f93551df640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f9350163050 con 0x7f93500d4820 2024-09-19T08:44:56.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.768+0000 7f934ed9d640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f93500b7f80 0x7f935015f560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56564/0 (socket says 172.21.15.98:56564) 2024-09-19T08:44:56.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.768+0000 7f934ed9d640 1 -- 172.21.15.98:0/3488972971 learned_addr learned my addr 172.21.15.98:0/3488972971 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:56.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.768+0000 7f934dd9b640 1 -- 172.21.15.98:0/3488972971 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1112231970 0 0) 0x7f9350068440 con 0x7f93500b7f80 2024-09-19T08:44:56.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.768+0000 7f934dd9b640 1 -- 172.21.15.98:0/3488972971 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f933c003530 con 0x7f93500b7f80 2024-09-19T08:44:56.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.768+0000 7f934e59c640 1 --2- 172.21.15.98:0/3488972971 >> v2:172.21.15.139:3300/0 conn(0x7f93500d4820 0x7f93501621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:56.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.768+0000 7f934dd9b640 1 -- 172.21.15.98:0/3488972971 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 638013796 0 0) 0x7f933c003530 con 0x7f93500b7f80 2024-09-19T08:44:56.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.768+0000 7f934dd9b640 1 -- 172.21.15.98:0/3488972971 >> v2:172.21.15.139:3300/0 conn(0x7f93500d4820 msgr2=0x7f93501621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:56.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.768+0000 7f934dd9b640 1 --2- 172.21.15.98:0/3488972971 >> v2:172.21.15.139:3300/0 conn(0x7f93500d4820 0x7f93501621b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:56.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.768+0000 7f934dd9b640 1 -- 172.21.15.98:0/3488972971 >> v1:172.21.15.139:6789/0 conn(0x7f93500b9970 legacy=0x7f93500b9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:56.773 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.768+0000 7f934dd9b640 1 -- 172.21.15.98:0/3488972971 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9350162d70 con 0x7f93500b7f80 2024-09-19T08:44:56.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.768+0000 7f934dd9b640 1 -- 172.21.15.98:0/3488972971 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f9340001a00 con 0x7f93500b7f80 2024-09-19T08:44:56.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.768+0000 7f934dd9b640 1 -- 172.21.15.98:0/3488972971 >> v1:172.21.15.98:6789/0 conn(0x7f93500b7f80 legacy=0x7f935015f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:56.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.768+0000 7f934dd9b640 1 --2- 172.21.15.98:0/3488972971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f933c0039c0 0x7f933c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:56.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.768+0000 7f934dd9b640 1 --2- 172.21.15.98:0/3488972971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f933c004810 0x7f933c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:56.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.768+0000 7f934dd9b640 1 --2- 172.21.15.98:0/3488972971 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f933c0052f0 0x7f933c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:56.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.768+0000 7f934dd9b640 1 -- 172.21.15.98:0/3488972971 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9350163050 con 0x7f933c004810 2024-09-19T08:44:56.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.768+0000 7f934dd9b640 1 -- 172.21.15.98:0/3488972971 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f933c00c1a0 con 0x7f933c0039c0 2024-09-19T08:44:56.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.768+0000 7f934dd9b640 1 -- 172.21.15.98:0/3488972971 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f933c00c3a0 con 0x7f933c0052f0 2024-09-19T08:44:56.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.768+0000 7f934e59c640 1 --2- 172.21.15.98:0/3488972971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f933c004810 0x7f933c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:56.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.768+0000 7f934ed9d640 1 --2- 172.21.15.98:0/3488972971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f933c0039c0 0x7f933c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:56.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.772+0000 7f934e59c640 1 -- 172.21.15.98:0/3488972971 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f933c0052f0 msgr2=0x7f933c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:56.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.772+0000 7f934f59e640 1 --2- 172.21.15.98:0/3488972971 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f933c0052f0 0x7f933c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:56.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.772+0000 7f934e59c640 1 --2- 172.21.15.98:0/3488972971 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f933c0052f0 0x7f933c00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:56.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.772+0000 7f934e59c640 1 -- 172.21.15.98:0/3488972971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f933c0039c0 msgr2=0x7f933c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:56.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.772+0000 7f934e59c640 1 --2- 172.21.15.98:0/3488972971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f933c0039c0 0x7f933c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:56.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.772+0000 7f934e59c640 1 -- 172.21.15.98:0/3488972971 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f933c003530 con 0x7f933c004810 2024-09-19T08:44:56.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.772+0000 7f934ed9d640 1 --2- 172.21.15.98:0/3488972971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f933c0039c0 0x7f933c003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:44:56.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.772+0000 7f934f59e640 1 -- 172.21.15.98:0/3488972971 reap_dead start 2024-09-19T08:44:56.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.772+0000 7f934e59c640 1 --2- 172.21.15.98:0/3488972971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f933c004810 0x7f933c004c90 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f933800ba00 tx=0x7f933801c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebbdbee6bd8ce613 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:56.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.772+0000 7f934dd9b640 1 -- 172.21.15.98:0/3488972971 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93380b3070 con 0x7f933c004810 2024-09-19T08:44:56.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.772+0000 7f934dd9b640 1 -- 172.21.15.98:0/3488972971 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93380a9070 con 0x7f933c004810 2024-09-19T08:44:56.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.772+0000 7f934dd9b640 1 -- 172.21.15.98:0/3488972971 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933800b600 con 0x7f933c004810 2024-09-19T08:44:56.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.772+0000 7f93551df640 1 -- 172.21.15.98:0/3488972971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f933c004810 msgr2=0x7f933c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:56.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.772+0000 7f93551df640 1 --2- 172.21.15.98:0/3488972971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f933c004810 0x7f933c004c90 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f933800ba00 tx=0x7f933801c660 comp rx=0 tx=0).stop 2024-09-19T08:44:56.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.772+0000 7f93551df640 1 -- 172.21.15.98:0/3488972971 shutdown_connections 2024-09-19T08:44:56.776 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.772+0000 7f93551df640 1 --2- 172.21.15.98:0/3488972971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f933c004810 0x7f933c004c90 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:56.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.772+0000 7f93551df640 1 -- 172.21.15.98:0/3488972971 >> 172.21.15.98:0/3488972971 conn(0x7f9350074e40 msgr2=0x7f9350075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:56.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.772+0000 7f93551df640 1 -- 172.21.15.98:0/3488972971 shutdown_connections 2024-09-19T08:44:56.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.772+0000 7f93551df640 1 -- 172.21.15.98:0/3488972971 wait complete. 2024-09-19T08:44:56.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.776+0000 7f93551df640 1 Processor -- start 2024-09-19T08:44:56.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.776+0000 7f93551df640 1 -- start start 2024-09-19T08:44:56.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.776+0000 7f93551df640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f933c004810 0x7f9350140ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:56.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.776+0000 7f93551df640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9350141bd0 0x7f9350142050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:56.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.776+0000 7f93551df640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93501533f0 0x7f9350153870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:56.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.776+0000 7f93551df640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f935015f950 con 0x7f9350141bd0 2024-09-19T08:44:56.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.776+0000 7f93551df640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f93500b8570 con 0x7f933c004810 2024-09-19T08:44:56.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.776+0000 7f93551df640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f935015fad0 con 0x7f93501533f0 2024-09-19T08:44:56.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.776+0000 7f934e59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9350141bd0 0x7f9350142050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:56.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.776+0000 7f934e59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9350141bd0 0x7f9350142050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34768/0 (socket says 172.21.15.98:34768) 2024-09-19T08:44:56.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.776+0000 7f934e59c640 1 -- 172.21.15.98:0/3372749303 learned_addr learned my addr 172.21.15.98:0/3372749303 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:56.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.776+0000 7f934ed9d640 1 --2- 172.21.15.98:0/3372749303 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f933c004810 0x7f9350140ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:56.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.776+0000 7f934e59c640 1 -- 172.21.15.98:0/3372749303 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93501533f0 msgr2=0x7f9350153870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:56.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.776+0000 7f934e59c640 1 --2- 172.21.15.98:0/3372749303 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93501533f0 0x7f9350153870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:56.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.776+0000 7f934e59c640 1 -- 172.21.15.98:0/3372749303 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f933c004810 msgr2=0x7f9350140ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:56.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.776+0000 7f934e59c640 1 --2- 172.21.15.98:0/3372749303 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f933c004810 0x7f9350140ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:56.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.776+0000 7f934e59c640 1 -- 172.21.15.98:0/3372749303 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9350142690 con 0x7f9350141bd0 2024-09-19T08:44:56.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.776+0000 7f934e59c640 1 --2- 172.21.15.98:0/3372749303 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9350141bd0 0x7f9350142050 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f933801c6a0 tx=0x7f933c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:56.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.776+0000 7f93477fe640 1 -- 172.21.15.98:0/3372749303 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93380b3040 con 0x7f9350141bd0 2024-09-19T08:44:56.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.776+0000 7f93477fe640 1 -- 172.21.15.98:0/3372749303 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93380aecd0 con 0x7f9350141bd0 2024-09-19T08:44:56.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.776+0000 7f93551df640 1 -- 172.21.15.98:0/3372749303 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9350142970 con 0x7f9350141bd0 2024-09-19T08:44:56.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.776+0000 7f93477fe640 1 -- 172.21.15.98:0/3372749303 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9338004240 con 0x7f9350141bd0 2024-09-19T08:44:56.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.776+0000 7f93551df640 1 -- 172.21.15.98:0/3372749303 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9350142f30 con 0x7f9350141bd0 2024-09-19T08:44:56.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.776+0000 7f93551df640 1 -- 172.21.15.98:0/3372749303 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9350143e10 con 0x7f9350141bd0 2024-09-19T08:44:56.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.776+0000 7f93477fe640 1 -- 172.21.15.98:0/3372749303 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f9338003620 con 0x7f9350141bd0 2024-09-19T08:44:56.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.776+0000 7f93477fe640 1 --2- 172.21.15.98:0/3372749303 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9330067ce0 0x7f933006a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:56.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.776+0000 7f93477fe640 1 -- 172.21.15.98:0/3372749303 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5737+0+0 (secure 0 0 0) 0x7f933811fbc0 con 0x7f9350141bd0 2024-09-19T08:44:56.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.780+0000 7f934ed9d640 1 --2- 172.21.15.98:0/3372749303 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9330067ce0 0x7f933006a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:56.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.780+0000 7f934ed9d640 1 --2- 172.21.15.98:0/3372749303 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9330067ce0 0x7f933006a1a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9340000f90 tx=0x7f9340009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:56.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.784+0000 7f93477fe640 1 -- 172.21.15.98:0/3372749303 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f93380ab070 con 0x7f9350141bd0 2024-09-19T08:44:56.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.932+0000 7f93551df640 1 -- 172.21.15.98:0/3372749303 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f93500d0700 con 0x7f9350141bd0 2024-09-19T08:44:56.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.936+0000 7f93477fe640 1 -- 172.21.15.98:0/3372749303 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) ==== 74+0+16796 (secure 0 0 0) 0x7f9338016370 con 0x7f9350141bd0 2024-09-19T08:44:56.938 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:44:56.938 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":117,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:44:55.909662+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:44:53.861869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-19T08:44:54.909433+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:44:56.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.940+0000 7f93551df640 1 -- 172.21.15.98:0/3372749303 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9330067ce0 msgr2=0x7f933006a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:56.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.940+0000 7f93551df640 1 --2- 172.21.15.98:0/3372749303 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9330067ce0 0x7f933006a1a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9340000f90 tx=0x7f9340009040 comp rx=0 tx=0).stop 2024-09-19T08:44:56.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.940+0000 7f93551df640 1 -- 172.21.15.98:0/3372749303 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9350141bd0 msgr2=0x7f9350142050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:56.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.940+0000 7f93551df640 1 --2- 172.21.15.98:0/3372749303 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9350141bd0 0x7f9350142050 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f933801c6a0 tx=0x7f933c005840 comp rx=0 tx=0).stop 2024-09-19T08:44:56.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.940+0000 7f93551df640 1 -- 172.21.15.98:0/3372749303 shutdown_connections 2024-09-19T08:44:56.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.940+0000 7f93551df640 1 --2- 172.21.15.98:0/3372749303 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9330067ce0 0x7f933006a1a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:56.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.940+0000 7f93551df640 1 --2- 172.21.15.98:0/3372749303 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f93501533f0 0x7f9350153870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:56.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.940+0000 7f93551df640 1 --2- 172.21.15.98:0/3372749303 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9350141bd0 0x7f9350142050 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:56.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.940+0000 7f93551df640 1 --2- 172.21.15.98:0/3372749303 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f933c004810 0x7f9350140ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:56.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.940+0000 7f93551df640 1 -- 172.21.15.98:0/3372749303 >> 172.21.15.98:0/3372749303 conn(0x7f9350074e40 msgr2=0x7f93500d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:56.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.940+0000 7f93551df640 1 -- 172.21.15.98:0/3372749303 shutdown_connections 2024-09-19T08:44:56.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:56.940+0000 7f93551df640 1 -- 172.21.15.98:0/3372749303 wait complete. 2024-09-19T08:44:56.956 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:44:57.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.072+0000 7fa6b3754640 1 Processor -- start 2024-09-19T08:44:57.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.072+0000 7fa6b3754640 1 -- start start 2024-09-19T08:44:57.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.076+0000 7fa6b3754640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa6ac158fd0 0x7fa6ac0b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:57.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.076+0000 7fa6b3754640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa6ac0b4360 0x7fa6ac06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:57.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.076+0000 7fa6b3754640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6ac0ccc60 con 0x7fa6ac06c1d0 2024-09-19T08:44:57.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.076+0000 7fa6b3754640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fa6ac06c640 con 0x7fa6ac0b4360 2024-09-19T08:44:57.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.076+0000 7fa6b3754640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fa6ac1593b0 con 0x7fa6ac158fd0 2024-09-19T08:44:57.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.076+0000 7fa6b0cef640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa6ac0b4360 0x7fa6ac06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:57.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.076+0000 7fa6b1cf1640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fa6ac06c1d0 0x7fa6ac15efc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56580/0 (socket says 172.21.15.98:56580) 2024-09-19T08:44:57.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.076+0000 7fa6b1cf1640 1 -- 172.21.15.98:0/17234219 learned_addr learned my addr 172.21.15.98:0/17234219 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:57.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.076+0000 7fa6b14f0640 1 --2- 172.21.15.98:0/17234219 >> v2:172.21.15.139:3300/0 conn(0x7fa6ac158fd0 0x7fa6ac0b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:57.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.076+0000 7fa6b0cef640 1 -- 172.21.15.98:0/17234219 >> v2:172.21.15.139:3300/0 conn(0x7fa6ac158fd0 msgr2=0x7fa6ac0b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:57.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.076+0000 7fa6b0cef640 1 --2- 172.21.15.98:0/17234219 >> v2:172.21.15.139:3300/0 conn(0x7fa6ac158fd0 0x7fa6ac0b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:57.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.076+0000 7fa6b0cef640 1 -- 172.21.15.98:0/17234219 >> v1:172.21.15.98:6789/0 conn(0x7fa6ac06c1d0 legacy=0x7fa6ac15efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:57.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.076+0000 7fa6a3fff640 1 -- 172.21.15.98:0/17234219 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4005190989 0 0) 0x7fa6ac0ccc60 con 0x7fa6ac06c1d0 2024-09-19T08:44:57.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.076+0000 7fa6b0cef640 1 -- 172.21.15.98:0/17234219 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6ac0ccf40 con 0x7fa6ac0b4360 2024-09-19T08:44:57.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.076+0000 7fa6b0cef640 1 --2- 172.21.15.98:0/17234219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6ac0b4360 0x7fa6ac06bc90 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac06e310 tx=0x7fa69c01e5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13f6cd5b10a7fcf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:57.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.076+0000 7fa6a3fff640 1 -- 172.21.15.98:0/17234219 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69c0b2600 con 0x7fa6ac0b4360 2024-09-19T08:44:57.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.076+0000 7fa6a3fff640 1 -- 172.21.15.98:0/17234219 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa69c0b27a0 con 0x7fa6ac0b4360 2024-09-19T08:44:57.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.076+0000 7fa6a3fff640 1 -- 172.21.15.98:0/17234219 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69c0b2b70 con 0x7fa6ac0b4360 2024-09-19T08:44:57.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.080+0000 7fa6b3754640 1 -- 172.21.15.98:0/17234219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6ac0b4360 msgr2=0x7fa6ac06bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:57.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.080+0000 7fa6b3754640 1 --2- 172.21.15.98:0/17234219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6ac0b4360 0x7fa6ac06bc90 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac06e310 tx=0x7fa69c01e5d0 comp rx=0 tx=0).stop 2024-09-19T08:44:57.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.080+0000 7fa6b3754640 1 -- 172.21.15.98:0/17234219 shutdown_connections 2024-09-19T08:44:57.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.080+0000 7fa6b3754640 1 --2- 172.21.15.98:0/17234219 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6ac0b4360 0x7fa6ac06bc90 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:57.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.080+0000 7fa6b3754640 1 --2- 172.21.15.98:0/17234219 >> v2:172.21.15.139:3300/0 conn(0x7fa6ac158fd0 0x7fa6ac0b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:57.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.080+0000 7fa6b3754640 1 -- 172.21.15.98:0/17234219 >> 172.21.15.98:0/17234219 conn(0x7fa6ac0b2010 msgr2=0x7fa6ac0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:57.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.080+0000 7fa6b3754640 1 -- 172.21.15.98:0/17234219 shutdown_connections 2024-09-19T08:44:57.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.080+0000 7fa6b3754640 1 -- 172.21.15.98:0/17234219 wait complete. 2024-09-19T08:44:57.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.080+0000 7fa6b3754640 1 Processor -- start 2024-09-19T08:44:57.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.080+0000 7fa6b3754640 1 -- start start 2024-09-19T08:44:57.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.080+0000 7fa6b3754640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6ac06c1d0 0x7fa6ac150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:57.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.080+0000 7fa6b3754640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6ac0b4360 0x7fa6ac1f6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:57.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.080+0000 7fa6b3754640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6ac158fd0 0x7fa6ac155b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:57.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.080+0000 7fa6b3754640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac15c920 con 0x7fa6ac158fd0 2024-09-19T08:44:57.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.080+0000 7fa6b3754640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac15c7a0 con 0x7fa6ac06c1d0 2024-09-19T08:44:57.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.080+0000 7fa6b3754640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa6ac06d170 con 0x7fa6ac0b4360 2024-09-19T08:44:57.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.080+0000 7fa6b1cf1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6ac158fd0 0x7fa6ac155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:57.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.080+0000 7fa6b1cf1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6ac158fd0 0x7fa6ac155b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34790/0 (socket says 172.21.15.98:34790) 2024-09-19T08:44:57.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.080+0000 7fa6b1cf1640 1 -- 172.21.15.98:0/94965039 learned_addr learned my addr 172.21.15.98:0/94965039 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:57.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.080+0000 7fa6b14f0640 1 --2- 172.21.15.98:0/94965039 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6ac06c1d0 0x7fa6ac150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:57.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.080+0000 7fa6b0cef640 1 --2- 172.21.15.98:0/94965039 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6ac0b4360 0x7fa6ac1f6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:57.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.080+0000 7fa6b1cf1640 1 -- 172.21.15.98:0/94965039 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6ac0b4360 msgr2=0x7fa6ac1f6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:57.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.080+0000 7fa6b1cf1640 1 --2- 172.21.15.98:0/94965039 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6ac0b4360 0x7fa6ac1f6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:57.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.080+0000 7fa6b1cf1640 1 -- 172.21.15.98:0/94965039 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6ac06c1d0 msgr2=0x7fa6ac150390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:57.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.080+0000 7fa6b1cf1640 1 --2- 172.21.15.98:0/94965039 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6ac06c1d0 0x7fa6ac150390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:57.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.080+0000 7fa6b1cf1640 1 -- 172.21.15.98:0/94965039 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6ac1f74f0 con 0x7fa6ac158fd0 2024-09-19T08:44:57.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.080+0000 7fa6b1cf1640 1 --2- 172.21.15.98:0/94965039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6ac158fd0 0x7fa6ac155b80 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fa6a400c7a0 tx=0x7fa6a400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:57.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.080+0000 7fa6a1ffb640 1 -- 172.21.15.98:0/94965039 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4018070 con 0x7fa6ac158fd0 2024-09-19T08:44:57.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.080+0000 7fa6a1ffb640 1 -- 172.21.15.98:0/94965039 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6a400a640 con 0x7fa6ac158fd0 2024-09-19T08:44:57.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.080+0000 7fa6b3754640 1 -- 172.21.15.98:0/94965039 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6ac154840 con 0x7fa6ac158fd0 2024-09-19T08:44:57.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.080+0000 7fa6a1ffb640 1 -- 172.21.15.98:0/94965039 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a400aa70 con 0x7fa6ac158fd0 2024-09-19T08:44:57.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.080+0000 7fa6b3754640 1 -- 172.21.15.98:0/94965039 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6ac154dc0 con 0x7fa6ac158fd0 2024-09-19T08:44:57.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.084+0000 7fa6b3754640 1 -- 172.21.15.98:0/94965039 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6ac06bc90 con 0x7fa6ac158fd0 2024-09-19T08:44:57.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.084+0000 7fa6a1ffb640 1 -- 172.21.15.98:0/94965039 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7fa6a401b070 con 0x7fa6ac158fd0 2024-09-19T08:44:57.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.084+0000 7fa6a1ffb640 1 --2- 172.21.15.98:0/94965039 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa690067cb0 0x7fa69006a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:57.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.084+0000 7fa6a1ffb640 1 -- 172.21.15.98:0/94965039 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5737+0+0 (secure 0 0 0) 0x7fa6a4085a20 con 0x7fa6ac158fd0 2024-09-19T08:44:57.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.084+0000 7fa6b14f0640 1 --2- 172.21.15.98:0/94965039 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa690067cb0 0x7fa69006a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:57.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.084+0000 7fa6b14f0640 1 --2- 172.21.15.98:0/94965039 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa690067cb0 0x7fa69006a170 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa6a800b930 tx=0x7fa6a8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:57.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.088+0000 7fa6a1ffb640 1 -- 172.21.15.98:0/94965039 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa6ac1f74f0 con 0x7fa6ac158fd0 2024-09-19T08:44:57.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.236+0000 7fa6b3754640 1 -- 172.21.15.98:0/94965039 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6ac15b4d0 con 0x7fa6ac158fd0 2024-09-19T08:44:57.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.240+0000 7fa6a1ffb640 1 -- 172.21.15.98:0/94965039 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) ==== 74+0+16796 (secure 0 0 0) 0x7fa6a4055f60 con 0x7fa6ac158fd0 2024-09-19T08:44:57.242 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:44:57.243 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":117,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:44:55.909662+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:44:53.861869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-19T08:44:54.909433+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:44:57.244 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.244+0000 7fa6b3754640 1 -- 172.21.15.98:0/94965039 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa690067cb0 msgr2=0x7fa69006a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:57.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.244+0000 7fa6b3754640 1 --2- 172.21.15.98:0/94965039 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa690067cb0 0x7fa69006a170 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa6a800b930 tx=0x7fa6a8008040 comp rx=0 tx=0).stop 2024-09-19T08:44:57.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.244+0000 7fa6b3754640 1 -- 172.21.15.98:0/94965039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6ac158fd0 msgr2=0x7fa6ac155b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:57.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.244+0000 7fa6b3754640 1 --2- 172.21.15.98:0/94965039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6ac158fd0 0x7fa6ac155b80 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fa6a400c7a0 tx=0x7fa6a400cc60 comp rx=0 tx=0).stop 2024-09-19T08:44:57.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.244+0000 7fa6b3754640 1 -- 172.21.15.98:0/94965039 shutdown_connections 2024-09-19T08:44:57.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.244+0000 7fa6b3754640 1 --2- 172.21.15.98:0/94965039 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa690067cb0 0x7fa69006a170 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:57.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.244+0000 7fa6b3754640 1 --2- 172.21.15.98:0/94965039 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6ac158fd0 0x7fa6ac155b80 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:57.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.244+0000 7fa6b3754640 1 --2- 172.21.15.98:0/94965039 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6ac0b4360 0x7fa6ac1f6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:57.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.244+0000 7fa6b3754640 1 --2- 172.21.15.98:0/94965039 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6ac06c1d0 0x7fa6ac150390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:57.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.244+0000 7fa6b3754640 1 -- 172.21.15.98:0/94965039 >> 172.21.15.98:0/94965039 conn(0x7fa6ac0b2010 msgr2=0x7fa6ac0b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:57.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.244+0000 7fa6b3754640 1 -- 172.21.15.98:0/94965039 shutdown_connections 2024-09-19T08:44:57.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.244+0000 7fa6b3754640 1 -- 172.21.15.98:0/94965039 wait complete. 2024-09-19T08:44:57.260 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-19T08:44:57.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.372+0000 7f35f84ee640 1 Processor -- start 2024-09-19T08:44:57.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.372+0000 7f35f84ee640 1 -- start start 2024-09-19T08:44:57.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.372+0000 7f35f84ee640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f35f0144d10 0x7f35f00722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:57.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.372+0000 7f35f84ee640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f35f006c3a0 0x7f35f00cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:57.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.372+0000 7f35f84ee640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35f0072ea0 con 0x7f35f0072820 2024-09-19T08:44:57.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.372+0000 7f35f84ee640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f35f0072c60 con 0x7f35f0144d10 2024-09-19T08:44:57.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.372+0000 7f35f84ee640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f35f0073ab0 con 0x7f35f006c3a0 2024-09-19T08:44:57.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.372+0000 7f35f628a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f35f0144d10 0x7f35f00722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:57.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.372+0000 7f35f5a89640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f35f0072820 0x7f35f006bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56584/0 (socket says 172.21.15.98:56584) 2024-09-19T08:44:57.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.372+0000 7f35f5a89640 1 -- 172.21.15.98:0/46962022 learned_addr learned my addr 172.21.15.98:0/46962022 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:57.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.372+0000 7f35f6a8b640 1 --2- 172.21.15.98:0/46962022 >> v2:172.21.15.139:3300/0 conn(0x7f35f006c3a0 0x7f35f00cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:57.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.372+0000 7f35f5288640 1 -- 172.21.15.98:0/46962022 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 892438967 0 0) 0x7f35f0072ea0 con 0x7f35f0072820 2024-09-19T08:44:57.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.372+0000 7f35f5288640 1 -- 172.21.15.98:0/46962022 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35dc003530 con 0x7f35f0072820 2024-09-19T08:44:57.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.372+0000 7f35f628a640 1 -- 172.21.15.98:0/46962022 >> v2:172.21.15.139:3300/0 conn(0x7f35f006c3a0 msgr2=0x7f35f00cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:57.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.372+0000 7f35f628a640 1 --2- 172.21.15.98:0/46962022 >> v2:172.21.15.139:3300/0 conn(0x7f35f006c3a0 0x7f35f00cd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:57.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.372+0000 7f35f628a640 1 -- 172.21.15.98:0/46962022 >> v1:172.21.15.98:6789/0 conn(0x7f35f0072820 legacy=0x7f35f006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:57.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.372+0000 7f35f628a640 1 -- 172.21.15.98:0/46962022 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f0073180 con 0x7f35f0144d10 2024-09-19T08:44:57.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.372+0000 7f35f6a8b640 1 --2- 172.21.15.98:0/46962022 >> v2:172.21.15.139:3300/0 conn(0x7f35f006c3a0 0x7f35f00cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:44:57.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.372+0000 7f35f628a640 1 --2- 172.21.15.98:0/46962022 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35f0144d10 0x7f35f00722e0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f35f006f4b0 tx=0x7f35e401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef17bafc418ec83a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:57.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.372+0000 7f35f5288640 1 -- 172.21.15.98:0/46962022 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e40b0070 con 0x7f35f0144d10 2024-09-19T08:44:57.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.376+0000 7f35f5288640 1 -- 172.21.15.98:0/46962022 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35e40ab600 con 0x7f35f0144d10 2024-09-19T08:44:57.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.376+0000 7f35f5288640 1 -- 172.21.15.98:0/46962022 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e40ab9b0 con 0x7f35f0144d10 2024-09-19T08:44:57.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.376+0000 7f35f84ee640 1 -- 172.21.15.98:0/46962022 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35f0144d10 msgr2=0x7f35f00722e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:57.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.376+0000 7f35f84ee640 1 --2- 172.21.15.98:0/46962022 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35f0144d10 0x7f35f00722e0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f35f006f4b0 tx=0x7f35e401c480 comp rx=0 tx=0).stop 2024-09-19T08:44:57.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.376+0000 7f35f84ee640 1 -- 172.21.15.98:0/46962022 shutdown_connections 2024-09-19T08:44:57.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.376+0000 7f35f84ee640 1 --2- 172.21.15.98:0/46962022 >> v2:172.21.15.139:3300/0 conn(0x7f35f006c3a0 0x7f35f00cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:57.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.376+0000 7f35f84ee640 1 --2- 172.21.15.98:0/46962022 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35f0144d10 0x7f35f00722e0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:57.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.376+0000 7f35f84ee640 1 -- 172.21.15.98:0/46962022 >> 172.21.15.98:0/46962022 conn(0x7f35f00b2010 msgr2=0x7f35f00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:57.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.376+0000 7f35f84ee640 1 -- 172.21.15.98:0/46962022 shutdown_connections 2024-09-19T08:44:57.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.376+0000 7f35f84ee640 1 -- 172.21.15.98:0/46962022 wait complete. 2024-09-19T08:44:57.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.376+0000 7f35f84ee640 1 Processor -- start 2024-09-19T08:44:57.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.376+0000 7f35f84ee640 1 -- start start 2024-09-19T08:44:57.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.376+0000 7f35f84ee640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f35f006c3a0 0x7f35f015a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:57.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.376+0000 7f35f84ee640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35f0072820 0x7f35f00acb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:57.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.376+0000 7f35f84ee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35f0144d10 0x7f35f00aa9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:57.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.376+0000 7f35f84ee640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f35f00ce120 con 0x7f35f0072820 2024-09-19T08:44:57.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.376+0000 7f35f84ee640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f35f00cdfa0 con 0x7f35f0144d10 2024-09-19T08:44:57.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.376+0000 7f35f84ee640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f35f00ce2a0 con 0x7f35f006c3a0 2024-09-19T08:44:57.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.376+0000 7f35f5a89640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35f0072820 0x7f35f00acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:57.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.376+0000 7f35f5a89640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35f0072820 0x7f35f00acb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34818/0 (socket says 172.21.15.98:34818) 2024-09-19T08:44:57.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.376+0000 7f35f5a89640 1 -- 172.21.15.98:0/2690860578 learned_addr learned my addr 172.21.15.98:0/2690860578 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:57.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.376+0000 7f35f5a89640 1 -- 172.21.15.98:0/2690860578 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f35f006c3a0 msgr2=0x7f35f015a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:57.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.376+0000 7f35f5a89640 1 --2- 172.21.15.98:0/2690860578 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f35f006c3a0 0x7f35f015a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:57.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.376+0000 7f35f5a89640 1 -- 172.21.15.98:0/2690860578 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35f0144d10 msgr2=0x7f35f00aa9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:57.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.376+0000 7f35f5a89640 1 --2- 172.21.15.98:0/2690860578 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35f0144d10 0x7f35f00aa9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:57.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.376+0000 7f35f5a89640 1 -- 172.21.15.98:0/2690860578 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f00718c0 con 0x7f35f0072820 2024-09-19T08:44:57.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.376+0000 7f35f5a89640 1 --2- 172.21.15.98:0/2690860578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35f0072820 0x7f35f00acb50 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00c450 tx=0x7f35ec00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:57.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.376+0000 7f35e2ffd640 1 -- 172.21.15.98:0/2690860578 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec017070 con 0x7f35f0072820 2024-09-19T08:44:57.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.376+0000 7f35e2ffd640 1 -- 172.21.15.98:0/2690860578 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35ec005cf0 con 0x7f35f0072820 2024-09-19T08:44:57.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.376+0000 7f35f84ee640 1 -- 172.21.15.98:0/2690860578 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35f0141fe0 con 0x7f35f0072820 2024-09-19T08:44:57.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.376+0000 7f35e2ffd640 1 -- 172.21.15.98:0/2690860578 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ec007290 con 0x7f35f0072820 2024-09-19T08:44:57.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.376+0000 7f35f84ee640 1 -- 172.21.15.98:0/2690860578 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35f01425a0 con 0x7f35f0072820 2024-09-19T08:44:57.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.380+0000 7f35f84ee640 1 -- 172.21.15.98:0/2690860578 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35f00cd250 con 0x7f35f0072820 2024-09-19T08:44:57.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.380+0000 7f35e2ffd640 1 -- 172.21.15.98:0/2690860578 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f35ec01a070 con 0x7f35f0072820 2024-09-19T08:44:57.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.380+0000 7f35e2ffd640 1 --2- 172.21.15.98:0/2690860578 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f35d4067cb0 0x7f35d406a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:57.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.380+0000 7f35e2ffd640 1 -- 172.21.15.98:0/2690860578 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5737+0+0 (secure 0 0 0) 0x7f35ec085610 con 0x7f35f0072820 2024-09-19T08:44:57.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.380+0000 7f35f628a640 1 --2- 172.21.15.98:0/2690860578 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f35d4067cb0 0x7f35d406a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:57.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.380+0000 7f35f628a640 1 --2- 172.21.15.98:0/2690860578 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f35d4067cb0 0x7f35d406a170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f35e40ad000 tx=0x7f35e4004740 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:57.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.384+0000 7f35e2ffd640 1 -- 172.21.15.98:0/2690860578 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f35ec055ce0 con 0x7f35f0072820 2024-09-19T08:44:57.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.568+0000 7f35f84ee640 1 -- 172.21.15.98:0/2690860578 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f35f006bd30 con 0x7f35f0072820 2024-09-19T08:44:57.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.616+0000 7f35e2ffd640 1 -- 172.21.15.98:0/2690860578 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v66) ==== 118+0+0 (secure 0 0 0) 0x7f35ec0555e0 con 0x7f35f0072820 2024-09-19T08:44:57.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.616+0000 7f35f84ee640 1 -- 172.21.15.98:0/2690860578 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f35d4067cb0 msgr2=0x7f35d406a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:57.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.616+0000 7f35f84ee640 1 --2- 172.21.15.98:0/2690860578 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f35d4067cb0 0x7f35d406a170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f35e40ad000 tx=0x7f35e4004740 comp rx=0 tx=0).stop 2024-09-19T08:44:57.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.616+0000 7f35f84ee640 1 -- 172.21.15.98:0/2690860578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35f0072820 msgr2=0x7f35f00acb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:57.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.616+0000 7f35f84ee640 1 --2- 172.21.15.98:0/2690860578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35f0072820 0x7f35f00acb50 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f35ec00c450 tx=0x7f35ec00c910 comp rx=0 tx=0).stop 2024-09-19T08:44:57.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.616+0000 7f35f84ee640 1 -- 172.21.15.98:0/2690860578 shutdown_connections 2024-09-19T08:44:57.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.616+0000 7f35f84ee640 1 --2- 172.21.15.98:0/2690860578 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f35d4067cb0 0x7f35d406a170 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:57.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.616+0000 7f35f84ee640 1 --2- 172.21.15.98:0/2690860578 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f35f0144d10 0x7f35f00aa9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:57.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.616+0000 7f35f84ee640 1 --2- 172.21.15.98:0/2690860578 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f35f0072820 0x7f35f00acb50 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:57.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.616+0000 7f35f84ee640 1 --2- 172.21.15.98:0/2690860578 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f35f006c3a0 0x7f35f015a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:57.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.616+0000 7f35f84ee640 1 -- 172.21.15.98:0/2690860578 >> 172.21.15.98:0/2690860578 conn(0x7f35f00b2010 msgr2=0x7f35f013f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:57.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.616+0000 7f35f84ee640 1 -- 172.21.15.98:0/2690860578 shutdown_connections 2024-09-19T08:44:57.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.616+0000 7f35f84ee640 1 -- 172.21.15.98:0/2690860578 wait complete. 2024-09-19T08:44:57.637 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-19T08:44:57.637 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-19T08:44:57.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.748+0000 7fe7f00ad640 1 Processor -- start 2024-09-19T08:44:57.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.748+0000 7fe7f00ad640 1 -- start start 2024-09-19T08:44:57.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.748+0000 7fe7f00ad640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe7e815b4f0 0x7fe7e80b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:57.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.748+0000 7fe7f00ad640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7e80af4a0 con 0x7fe7e806cb90 2024-09-19T08:44:57.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.748+0000 7fe7f00ad640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7e80af780 con 0x7fe7e80b0a70 2024-09-19T08:44:57.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.748+0000 7fe7f00ad640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe7e80afa60 con 0x7fe7e815b4f0 2024-09-19T08:44:57.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.748+0000 7fe7ee64a640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fe7e806cb90 0x7fe7e80b1bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52560/0 (socket says 172.21.15.98:52560) 2024-09-19T08:44:57.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.748+0000 7fe7ee64a640 1 -- 172.21.15.98:0/446321319 learned_addr learned my addr 172.21.15.98:0/446321319 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:57.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.748+0000 7fe7ece47640 1 -- 172.21.15.98:0/446321319 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2417172078 0 0) 0x7fe7e80af4a0 con 0x7fe7e806cb90 2024-09-19T08:44:57.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.748+0000 7fe7ece47640 1 -- 172.21.15.98:0/446321319 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7d4005740 con 0x7fe7e806cb90 2024-09-19T08:44:57.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.748+0000 7fe7ede49640 1 --2- 172.21.15.98:0/446321319 >> v2:172.21.15.139:3300/0 conn(0x7fe7e815b4f0 0x7fe7e80b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:57.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.748+0000 7fe7ece47640 1 -- 172.21.15.98:0/446321319 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1955356082 0 0) 0x7fe7d4005740 con 0x7fe7e806cb90 2024-09-19T08:44:57.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.748+0000 7fe7ece47640 1 -- 172.21.15.98:0/446321319 >> v2:172.21.15.139:3300/0 conn(0x7fe7e815b4f0 msgr2=0x7fe7e80b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:57.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.748+0000 7fe7ece47640 1 --2- 172.21.15.98:0/446321319 >> v2:172.21.15.139:3300/0 conn(0x7fe7e815b4f0 0x7fe7e80b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:57.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.748+0000 7fe7ece47640 1 -- 172.21.15.98:0/446321319 >> v1:172.21.15.139:6789/0 conn(0x7fe7e80b0a70 legacy=0x7fe7e806c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:57.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.748+0000 7fe7ece47640 1 -- 172.21.15.98:0/446321319 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e80afd40 con 0x7fe7e806cb90 2024-09-19T08:44:57.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.748+0000 7fe7ece47640 1 -- 172.21.15.98:0/446321319 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fe7e0001a00 con 0x7fe7e806cb90 2024-09-19T08:44:57.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.748+0000 7fe7ede49640 1 --2- 172.21.15.98:0/446321319 >> v2:172.21.15.139:3300/0 conn(0x7fe7e815b4f0 0x7fe7e80b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:44:57.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.748+0000 7fe7ece47640 1 -- 172.21.15.98:0/446321319 >> v1:172.21.15.98:6789/0 conn(0x7fe7e806cb90 legacy=0x7fe7e80b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:57.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.748+0000 7fe7ece47640 1 --2- 172.21.15.98:0/446321319 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7d4005bd0 0x7fe7d4005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:57.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.748+0000 7fe7ece47640 1 --2- 172.21.15.98:0/446321319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7d4006a20 0x7fe7d4006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:57.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.748+0000 7fe7ece47640 1 --2- 172.21.15.98:0/446321319 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7d40073e0 0x7fe7d400dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:57.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.748+0000 7fe7ece47640 1 -- 172.21.15.98:0/446321319 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e80afa60 con 0x7fe7d4006a20 2024-09-19T08:44:57.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.748+0000 7fe7ece47640 1 -- 172.21.15.98:0/446321319 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d400e310 con 0x7fe7d4005bd0 2024-09-19T08:44:57.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.748+0000 7fe7ece47640 1 -- 172.21.15.98:0/446321319 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe7d400e510 con 0x7fe7d40073e0 2024-09-19T08:44:57.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.748+0000 7fe7ed648640 1 --2- 172.21.15.98:0/446321319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7d4006a20 0x7fe7d4006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:57.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.748+0000 7fe7ede49640 1 --2- 172.21.15.98:0/446321319 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7d4005bd0 0x7fe7d4005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:57.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.748+0000 7fe7ed648640 1 -- 172.21.15.98:0/446321319 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7d40073e0 msgr2=0x7fe7d400dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:57.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.748+0000 7fe7ee64a640 1 --2- 172.21.15.98:0/446321319 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7d40073e0 0x7fe7d400dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:57.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.748+0000 7fe7ed648640 1 --2- 172.21.15.98:0/446321319 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7d40073e0 0x7fe7d400dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:57.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.748+0000 7fe7ed648640 1 -- 172.21.15.98:0/446321319 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7d4005bd0 msgr2=0x7fe7d4005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:57.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.748+0000 7fe7ed648640 1 --2- 172.21.15.98:0/446321319 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7d4005bd0 0x7fe7d4005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:57.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.748+0000 7fe7ed648640 1 -- 172.21.15.98:0/446321319 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d4005740 con 0x7fe7d4006a20 2024-09-19T08:44:57.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.748+0000 7fe7ee64a640 1 -- 172.21.15.98:0/446321319 reap_dead start 2024-09-19T08:44:57.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.748+0000 7fe7ede49640 1 --2- 172.21.15.98:0/446321319 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7d4005bd0 0x7fe7d4005ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:57.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.752+0000 7fe7ed648640 1 --2- 172.21.15.98:0/446321319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7d4006a20 0x7fe7d4006ea0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8001780 tx=0x7fe7d801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1678fd50e89a051e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:57.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.752+0000 7fe7ece47640 1 -- 172.21.15.98:0/446321319 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d80013a0 con 0x7fe7d4006a20 2024-09-19T08:44:57.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.752+0000 7fe7ece47640 1 -- 172.21.15.98:0/446321319 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7d80052f0 con 0x7fe7d4006a20 2024-09-19T08:44:57.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.752+0000 7fe7ece47640 1 -- 172.21.15.98:0/446321319 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8003840 con 0x7fe7d4006a20 2024-09-19T08:44:57.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.752+0000 7fe7f00ad640 1 -- 172.21.15.98:0/446321319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7d4006a20 msgr2=0x7fe7d4006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:57.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.752+0000 7fe7f00ad640 1 --2- 172.21.15.98:0/446321319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7d4006a20 0x7fe7d4006ea0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8001780 tx=0x7fe7d801c400 comp rx=0 tx=0).stop 2024-09-19T08:44:57.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.752+0000 7fe7f00ad640 1 -- 172.21.15.98:0/446321319 shutdown_connections 2024-09-19T08:44:57.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.752+0000 7fe7f00ad640 1 --2- 172.21.15.98:0/446321319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7d4006a20 0x7fe7d4006ea0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:57.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.752+0000 7fe7f00ad640 1 -- 172.21.15.98:0/446321319 >> 172.21.15.98:0/446321319 conn(0x7fe7e80b5f50 msgr2=0x7fe7e80b6650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:57.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.752+0000 7fe7f00ad640 1 -- 172.21.15.98:0/446321319 shutdown_connections 2024-09-19T08:44:57.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.752+0000 7fe7f00ad640 1 -- 172.21.15.98:0/446321319 wait complete. 2024-09-19T08:44:57.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.752+0000 7fe7f00ad640 1 Processor -- start 2024-09-19T08:44:57.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.752+0000 7fe7f00ad640 1 -- start start 2024-09-19T08:44:57.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.752+0000 7fe7f00ad640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7d4006a20 0x7fe7e80cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:57.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.752+0000 7fe7f00ad640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7e80d1280 0x7fe7e8152820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:57.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.752+0000 7fe7f00ad640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7e815de00 0x7fe7e80b2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:57.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.752+0000 7fe7f00ad640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e80b0530 con 0x7fe7e815de00 2024-09-19T08:44:57.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.752+0000 7fe7f00ad640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e815b8d0 con 0x7fe7d4006a20 2024-09-19T08:44:57.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.752+0000 7fe7f00ad640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe7e80b06b0 con 0x7fe7e80d1280 2024-09-19T08:44:57.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.752+0000 7fe7ee64a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7e815de00 0x7fe7e80b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:57.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.752+0000 7fe7ee64a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7e815de00 0x7fe7e80b2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47776/0 (socket says 172.21.15.98:47776) 2024-09-19T08:44:57.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.752+0000 7fe7ee64a640 1 -- 172.21.15.98:0/3771561005 learned_addr learned my addr 172.21.15.98:0/3771561005 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:57.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.752+0000 7fe7ede49640 1 --2- 172.21.15.98:0/3771561005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7d4006a20 0x7fe7e80cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:57.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.752+0000 7fe7ed648640 1 --2- 172.21.15.98:0/3771561005 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7e80d1280 0x7fe7e8152820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:57.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.756+0000 7fe7ee64a640 1 -- 172.21.15.98:0/3771561005 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7e80d1280 msgr2=0x7fe7e8152820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:57.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.756+0000 7fe7ee64a640 1 --2- 172.21.15.98:0/3771561005 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7e80d1280 0x7fe7e8152820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:57.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.756+0000 7fe7ee64a640 1 -- 172.21.15.98:0/3771561005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7d4006a20 msgr2=0x7fe7e80cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:57.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.756+0000 7fe7ee64a640 1 --2- 172.21.15.98:0/3771561005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7d4006a20 0x7fe7e80cfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:57.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.756+0000 7fe7ee64a640 1 -- 172.21.15.98:0/3771561005 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e80d01e0 con 0x7fe7e815de00 2024-09-19T08:44:57.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.756+0000 7fe7ee64a640 1 --2- 172.21.15.98:0/3771561005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7e815de00 0x7fe7e80b2440 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fe7e806d270 tx=0x7fe7e000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:57.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.756+0000 7fe7de7fc640 1 -- 172.21.15.98:0/3771561005 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0004d70 con 0x7fe7e815de00 2024-09-19T08:44:57.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.756+0000 7fe7de7fc640 1 -- 172.21.15.98:0/3771561005 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7e000bb20 con 0x7fe7e815de00 2024-09-19T08:44:57.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.756+0000 7fe7f00ad640 1 -- 172.21.15.98:0/3771561005 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7e80d04c0 con 0x7fe7e815de00 2024-09-19T08:44:57.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.756+0000 7fe7de7fc640 1 -- 172.21.15.98:0/3771561005 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0007110 con 0x7fe7e815de00 2024-09-19T08:44:57.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.756+0000 7fe7f00ad640 1 -- 172.21.15.98:0/3771561005 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7e80d0a80 con 0x7fe7e815de00 2024-09-19T08:44:57.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.756+0000 7fe7f00ad640 1 -- 172.21.15.98:0/3771561005 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7e806c480 con 0x7fe7e815de00 2024-09-19T08:44:57.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.756+0000 7fe7de7fc640 1 -- 172.21.15.98:0/3771561005 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7fe7e001b020 con 0x7fe7e815de00 2024-09-19T08:44:57.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.756+0000 7fe7de7fc640 1 --2- 172.21.15.98:0/3771561005 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe7cc067ce0 0x7fe7cc06a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:57.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.756+0000 7fe7de7fc640 1 -- 172.21.15.98:0/3771561005 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5737+0+0 (secure 0 0 0) 0x7fe7e0086540 con 0x7fe7e815de00 2024-09-19T08:44:57.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.756+0000 7fe7ede49640 1 --2- 172.21.15.98:0/3771561005 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe7cc067ce0 0x7fe7cc06a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:57.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.756+0000 7fe7ede49640 1 --2- 172.21.15.98:0/3771561005 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe7cc067ce0 0x7fe7cc06a1a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8051960 tx=0x7fe7e4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:57.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.760+0000 7fe7de7fc640 1 -- 172.21.15.98:0/3771561005 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe7e0056970 con 0x7fe7e815de00 2024-09-19T08:44:57.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:57.940+0000 7fe7f00ad640 1 -- 172.21.15.98:0/3771561005 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe7e80b6ee0 con 0x7fe7e815de00 2024-09-19T08:44:58.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.624+0000 7fe7de7fc640 1 -- 172.21.15.98:0/3771561005 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v67) ==== 122+0+0 (secure 0 0 0) 0x7fe7e0056270 con 0x7fe7e815de00 2024-09-19T08:44:58.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.628+0000 7fe7f00ad640 1 -- 172.21.15.98:0/3771561005 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe7cc067ce0 msgr2=0x7fe7cc06a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:58.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.628+0000 7fe7f00ad640 1 --2- 172.21.15.98:0/3771561005 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe7cc067ce0 0x7fe7cc06a1a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8051960 tx=0x7fe7e4008040 comp rx=0 tx=0).stop 2024-09-19T08:44:58.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.628+0000 7fe7f00ad640 1 -- 172.21.15.98:0/3771561005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7e815de00 msgr2=0x7fe7e80b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:58.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.628+0000 7fe7f00ad640 1 --2- 172.21.15.98:0/3771561005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7e815de00 0x7fe7e80b2440 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fe7e806d270 tx=0x7fe7e000b4a0 comp rx=0 tx=0).stop 2024-09-19T08:44:58.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.628+0000 7fe7f00ad640 1 -- 172.21.15.98:0/3771561005 shutdown_connections 2024-09-19T08:44:58.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.628+0000 7fe7f00ad640 1 --2- 172.21.15.98:0/3771561005 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe7cc067ce0 0x7fe7cc06a1a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:58.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.628+0000 7fe7f00ad640 1 --2- 172.21.15.98:0/3771561005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe7e815de00 0x7fe7e80b2440 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:58.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.628+0000 7fe7f00ad640 1 --2- 172.21.15.98:0/3771561005 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe7e80d1280 0x7fe7e8152820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:58.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.628+0000 7fe7f00ad640 1 --2- 172.21.15.98:0/3771561005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7d4006a20 0x7fe7e80cfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:58.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.628+0000 7fe7f00ad640 1 -- 172.21.15.98:0/3771561005 >> 172.21.15.98:0/3771561005 conn(0x7fe7e80b5f50 msgr2=0x7fe7e80b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:58.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.628+0000 7fe7f00ad640 1 -- 172.21.15.98:0/3771561005 shutdown_connections 2024-09-19T08:44:58.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.628+0000 7fe7f00ad640 1 -- 172.21.15.98:0/3771561005 wait complete. 2024-09-19T08:44:58.647 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-19T08:44:58.647 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:44:58.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.772+0000 7f2ead750640 1 Processor -- start 2024-09-19T08:44:58.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.772+0000 7f2ead750640 1 -- start start 2024-09-19T08:44:58.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.776+0000 7f2ead750640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2ea8158fd0 0x7f2ea80b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:58.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.776+0000 7f2ead750640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2ea80b4360 0x7f2ea806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:58.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.776+0000 7f2ead750640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ea806c1d0 0x7f2ea815efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:58.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.776+0000 7f2ead750640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f2ea806c640 con 0x7f2ea8158fd0 2024-09-19T08:44:58.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.776+0000 7f2ead750640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f2ea81593b0 con 0x7f2ea80b4360 2024-09-19T08:44:58.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.776+0000 7f2ead750640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2ea80ccc60 con 0x7f2ea806c1d0 2024-09-19T08:44:58.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.776+0000 7f2ea6ffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2ea8158fd0 0x7f2ea80b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:58.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.776+0000 7f2ea6ffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2ea8158fd0 0x7f2ea80b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47782/0 (socket says 172.21.15.98:47782) 2024-09-19T08:44:58.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.776+0000 7f2ea6ffd640 1 -- 172.21.15.98:0/2359481982 learned_addr learned my addr 172.21.15.98:0/2359481982 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:58.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.776+0000 7f2ea67fc640 1 --2- 172.21.15.98:0/2359481982 >> v2:172.21.15.139:3300/0 conn(0x7f2ea80b4360 0x7f2ea806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:58.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.776+0000 7f2ea77fe640 1 --2- 172.21.15.98:0/2359481982 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ea806c1d0 0x7f2ea815efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:58.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.776+0000 7f2ea6ffd640 1 -- 172.21.15.98:0/2359481982 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ea806c1d0 msgr2=0x7f2ea815efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:58.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.776+0000 7f2ea6ffd640 1 --2- 172.21.15.98:0/2359481982 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ea806c1d0 0x7f2ea815efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:58.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.776+0000 7f2ea6ffd640 1 -- 172.21.15.98:0/2359481982 >> v2:172.21.15.139:3300/0 conn(0x7f2ea80b4360 msgr2=0x7f2ea806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:58.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.776+0000 7f2ea6ffd640 1 --2- 172.21.15.98:0/2359481982 >> v2:172.21.15.139:3300/0 conn(0x7f2ea80b4360 0x7f2ea806bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:58.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.776+0000 7f2ea6ffd640 1 -- 172.21.15.98:0/2359481982 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea80ccf40 con 0x7f2ea8158fd0 2024-09-19T08:44:58.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.776+0000 7f2ea6ffd640 1 --2- 172.21.15.98:0/2359481982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ea8158fd0 0x7f2ea80b3e20 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f2ea80cdef0 tx=0x7f2e9801c1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681c8942327e261d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:58.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2ea5ffb640 1 -- 172.21.15.98:0/2359481982 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e980b0070 con 0x7f2ea8158fd0 2024-09-19T08:44:58.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2ea5ffb640 1 -- 172.21.15.98:0/2359481982 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e980ab600 con 0x7f2ea8158fd0 2024-09-19T08:44:58.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2ea5ffb640 1 -- 172.21.15.98:0/2359481982 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e980ab9d0 con 0x7f2ea8158fd0 2024-09-19T08:44:58.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2ead750640 1 -- 172.21.15.98:0/2359481982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ea8158fd0 msgr2=0x7f2ea80b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:58.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2ead750640 1 --2- 172.21.15.98:0/2359481982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ea8158fd0 0x7f2ea80b3e20 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f2ea80cdef0 tx=0x7f2e9801c1a0 comp rx=0 tx=0).stop 2024-09-19T08:44:58.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2ead750640 1 -- 172.21.15.98:0/2359481982 shutdown_connections 2024-09-19T08:44:58.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2ead750640 1 --2- 172.21.15.98:0/2359481982 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ea806c1d0 0x7f2ea815efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:58.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2ead750640 1 --2- 172.21.15.98:0/2359481982 >> v2:172.21.15.139:3300/0 conn(0x7f2ea80b4360 0x7f2ea806bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:58.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2ead750640 1 --2- 172.21.15.98:0/2359481982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ea8158fd0 0x7f2ea80b3e20 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:58.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2ead750640 1 -- 172.21.15.98:0/2359481982 >> 172.21.15.98:0/2359481982 conn(0x7f2ea80b2010 msgr2=0x7f2ea80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:58.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2ead750640 1 -- 172.21.15.98:0/2359481982 shutdown_connections 2024-09-19T08:44:58.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2ead750640 1 -- 172.21.15.98:0/2359481982 wait complete. 2024-09-19T08:44:58.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2ead750640 1 Processor -- start 2024-09-19T08:44:58.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2ead750640 1 -- start start 2024-09-19T08:44:58.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2ead750640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ea806c1d0 0x7f2ea8160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:58.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2ead750640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ea80b4360 0x7f2ea80c5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:58.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2ead750640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ea8148050 0x7f2ea80c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:58.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2ead750640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea815c920 con 0x7f2ea806c1d0 2024-09-19T08:44:58.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2ead750640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea815c7a0 con 0x7f2ea80b4360 2024-09-19T08:44:58.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2ead750640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2ea80ce840 con 0x7f2ea8148050 2024-09-19T08:44:58.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2ea6ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ea806c1d0 0x7f2ea8160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:58.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2ea6ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ea806c1d0 0x7f2ea8160ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47796/0 (socket says 172.21.15.98:47796) 2024-09-19T08:44:58.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2ea6ffd640 1 -- 172.21.15.98:0/1955986200 learned_addr learned my addr 172.21.15.98:0/1955986200 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:58.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2ea67fc640 1 --2- 172.21.15.98:0/1955986200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ea80b4360 0x7f2ea80c5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:58.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2ea77fe640 1 --2- 172.21.15.98:0/1955986200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ea8148050 0x7f2ea80c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:58.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2ea6ffd640 1 -- 172.21.15.98:0/1955986200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ea8148050 msgr2=0x7f2ea80c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:58.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2ea6ffd640 1 --2- 172.21.15.98:0/1955986200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ea8148050 0x7f2ea80c57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:58.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2ea6ffd640 1 -- 172.21.15.98:0/1955986200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ea80b4360 msgr2=0x7f2ea80c5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:58.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2ea6ffd640 1 --2- 172.21.15.98:0/1955986200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ea80b4360 0x7f2ea80c5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:58.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2ea6ffd640 1 -- 172.21.15.98:0/1955986200 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea8164fa0 con 0x7f2ea806c1d0 2024-09-19T08:44:58.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2ea67fc640 1 --2- 172.21.15.98:0/1955986200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ea80b4360 0x7f2ea80c5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:58.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2ea6ffd640 1 --2- 172.21.15.98:0/1955986200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ea806c1d0 0x7f2ea8160ae0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f2e98000c10 tx=0x7f2e9801a9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:58.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2e8f7fe640 1 -- 172.21.15.98:0/1955986200 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e980ba020 con 0x7f2ea806c1d0 2024-09-19T08:44:58.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2e8f7fe640 1 -- 172.21.15.98:0/1955986200 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e980b6640 con 0x7f2ea806c1d0 2024-09-19T08:44:58.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2ead750640 1 -- 172.21.15.98:0/1955986200 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ea81611c0 con 0x7f2ea806c1d0 2024-09-19T08:44:58.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2e8f7fe640 1 -- 172.21.15.98:0/1955986200 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e980b0040 con 0x7f2ea806c1d0 2024-09-19T08:44:58.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.780+0000 7f2ead750640 1 -- 172.21.15.98:0/1955986200 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ea8161780 con 0x7f2ea806c1d0 2024-09-19T08:44:58.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.784+0000 7f2ead750640 1 -- 172.21.15.98:0/1955986200 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ea815efc0 con 0x7f2ea806c1d0 2024-09-19T08:44:58.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.784+0000 7f2e8f7fe640 1 -- 172.21.15.98:0/1955986200 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f2e980bd020 con 0x7f2ea806c1d0 2024-09-19T08:44:58.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.784+0000 7f2e8f7fe640 1 --2- 172.21.15.98:0/1955986200 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2e94067cb0 0x7f2e9406a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:58.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.784+0000 7f2e8f7fe640 1 -- 172.21.15.98:0/1955986200 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5737+0+0 (secure 0 0 0) 0x7f2e98120630 con 0x7f2ea806c1d0 2024-09-19T08:44:58.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.784+0000 7f2ea67fc640 1 --2- 172.21.15.98:0/1955986200 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2e94067cb0 0x7f2e9406a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:58.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.784+0000 7f2ea67fc640 1 --2- 172.21.15.98:0/1955986200 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2e94067cb0 0x7f2e9406a170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2ea81f9d90 tx=0x7f2e9c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:58.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.788+0000 7f2e8f7fe640 1 -- 172.21.15.98:0/1955986200 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2e980c4030 con 0x7f2ea806c1d0 2024-09-19T08:44:58.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.972+0000 7f2ead750640 1 -- 172.21.15.98:0/1955986200 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ea80cc690 con 0x7f2ea806c1d0 2024-09-19T08:44:58.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.972+0000 7f2e8f7fe640 1 -- 172.21.15.98:0/1955986200 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5228 (secure 0 0 0) 0x7f2e98016110 con 0x7f2ea806c1d0 2024-09-19T08:44:58.974 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:44:58.974 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":67,"btime":"2024-09-19T08:44:58:625080+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5645,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/1334032829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":1334032829},{"type":"v1","addr":"172.21.15.139:6833","nonce":1334032829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":5651,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2541341035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2541341035},{"type":"v1","addr":"172.21.15.139:6835","nonce":2541341035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":7521,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3632520818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3632520818},{"type":"v1","addr":"172.21.15.98:6833","nonce":3632520818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:44:55.910161+0000","modified":"2024-09-19T08:44:57.947782+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7524},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7524":{"gid":7524,"name":"c","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/502938874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":502938874},{"type":"v1","addr":"172.21.15.98:6835","nonce":502938874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7524,"qdb_cluster":[7524]},"id":9}]} 2024-09-19T08:44:58.974 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 67 2024-09-19T08:44:58.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.972+0000 7f2ead750640 1 -- 172.21.15.98:0/1955986200 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2e94067cb0 msgr2=0x7f2e9406a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:58.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.972+0000 7f2ead750640 1 --2- 172.21.15.98:0/1955986200 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2e94067cb0 0x7f2e9406a170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2ea81f9d90 tx=0x7f2e9c008040 comp rx=0 tx=0).stop 2024-09-19T08:44:58.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.972+0000 7f2ead750640 1 -- 172.21.15.98:0/1955986200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ea806c1d0 msgr2=0x7f2ea8160ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:58.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.972+0000 7f2ead750640 1 --2- 172.21.15.98:0/1955986200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ea806c1d0 0x7f2ea8160ae0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f2e98000c10 tx=0x7f2e9801a9e0 comp rx=0 tx=0).stop 2024-09-19T08:44:58.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.972+0000 7f2ead750640 1 -- 172.21.15.98:0/1955986200 shutdown_connections 2024-09-19T08:44:58.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.972+0000 7f2ead750640 1 --2- 172.21.15.98:0/1955986200 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2e94067cb0 0x7f2e9406a170 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:58.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.972+0000 7f2ead750640 1 --2- 172.21.15.98:0/1955986200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ea8148050 0x7f2ea80c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:58.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.972+0000 7f2ead750640 1 --2- 172.21.15.98:0/1955986200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ea80b4360 0x7f2ea80c5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:58.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.972+0000 7f2ead750640 1 --2- 172.21.15.98:0/1955986200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ea806c1d0 0x7f2ea8160ae0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:58.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.972+0000 7f2ead750640 1 -- 172.21.15.98:0/1955986200 >> 172.21.15.98:0/1955986200 conn(0x7f2ea80b2010 msgr2=0x7f2ea815cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:58.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.972+0000 7f2ead750640 1 -- 172.21.15.98:0/1955986200 shutdown_connections 2024-09-19T08:44:58.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:58.972+0000 7f2ead750640 1 -- 172.21.15.98:0/1955986200 wait complete. 2024-09-19T08:44:58.993 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:44:59.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.104+0000 7efc276ca640 1 Processor -- start 2024-09-19T08:44:59.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.104+0000 7efc276ca640 1 -- start start 2024-09-19T08:44:59.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.108+0000 7efc276ca640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7efc2015c550 0x7efc2006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:59.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.108+0000 7efc276ca640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc200cfad0 0x7efc200723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:59.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.108+0000 7efc276ca640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc2006d310 con 0x7efc2006c1d0 2024-09-19T08:44:59.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.108+0000 7efc276ca640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7efc2006c640 con 0x7efc2015c550 2024-09-19T08:44:59.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.108+0000 7efc276ca640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7efc2006d5f0 con 0x7efc200cfad0 2024-09-19T08:44:59.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.108+0000 7efc25c67640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc200cfad0 0x7efc200723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:59.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.108+0000 7efc25466640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7efc2015c550 0x7efc2006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:59.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.108+0000 7efc25c67640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc200cfad0 0x7efc200723a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:49964/0 (socket says 172.21.15.98:49964) 2024-09-19T08:44:59.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.108+0000 7efc25c67640 1 -- 172.21.15.98:0/855335974 learned_addr learned my addr 172.21.15.98:0/855335974 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:59.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.108+0000 7efc25c67640 1 -- 172.21.15.98:0/855335974 >> v2:172.21.15.139:3300/0 conn(0x7efc2015c550 msgr2=0x7efc2006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:59.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.108+0000 7efc25c67640 1 --2- 172.21.15.98:0/855335974 >> v2:172.21.15.139:3300/0 conn(0x7efc2015c550 0x7efc2006bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:59.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.108+0000 7efc25c67640 1 -- 172.21.15.98:0/855335974 >> v1:172.21.15.139:6789/0 conn(0x7efc2006c1d0 legacy=0x7efc200cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:59.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.108+0000 7efc25c67640 1 -- 172.21.15.98:0/855335974 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc2006d8d0 con 0x7efc200cfad0 2024-09-19T08:44:59.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.108+0000 7efc25466640 1 --2- 172.21.15.98:0/855335974 >> v2:172.21.15.139:3300/0 conn(0x7efc2015c550 0x7efc2006bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:44:59.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.108+0000 7efc25c67640 1 --2- 172.21.15.98:0/855335974 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc200cfad0 0x7efc200723a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efc18009e50 tx=0x7efc1801ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ede12c6a7cd51c6d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:59.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.112+0000 7efc17fff640 1 -- 172.21.15.98:0/855335974 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc180a5810 con 0x7efc200cfad0 2024-09-19T08:44:59.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.112+0000 7efc17fff640 1 -- 172.21.15.98:0/855335974 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc180a59b0 con 0x7efc200cfad0 2024-09-19T08:44:59.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.112+0000 7efc17fff640 1 -- 172.21.15.98:0/855335974 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc180a5d80 con 0x7efc200cfad0 2024-09-19T08:44:59.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.112+0000 7efc276ca640 1 -- 172.21.15.98:0/855335974 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc200cfad0 msgr2=0x7efc200723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:59.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.112+0000 7efc276ca640 1 --2- 172.21.15.98:0/855335974 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc200cfad0 0x7efc200723a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efc18009e50 tx=0x7efc1801ca30 comp rx=0 tx=0).stop 2024-09-19T08:44:59.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.112+0000 7efc276ca640 1 -- 172.21.15.98:0/855335974 shutdown_connections 2024-09-19T08:44:59.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.112+0000 7efc276ca640 1 --2- 172.21.15.98:0/855335974 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc200cfad0 0x7efc200723a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:59.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.112+0000 7efc276ca640 1 --2- 172.21.15.98:0/855335974 >> v2:172.21.15.139:3300/0 conn(0x7efc2015c550 0x7efc2006bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:59.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.112+0000 7efc276ca640 1 -- 172.21.15.98:0/855335974 >> 172.21.15.98:0/855335974 conn(0x7efc200b2010 msgr2=0x7efc200b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:59.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.112+0000 7efc276ca640 1 -- 172.21.15.98:0/855335974 shutdown_connections 2024-09-19T08:44:59.113 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.112+0000 7efc276ca640 1 -- 172.21.15.98:0/855335974 wait complete. 2024-09-19T08:44:59.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.112+0000 7efc276ca640 1 Processor -- start 2024-09-19T08:44:59.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.112+0000 7efc276ca640 1 -- start start 2024-09-19T08:44:59.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.112+0000 7efc276ca640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc2006c1d0 0x7efc20147450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:59.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.112+0000 7efc276ca640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc200cfad0 0x7efc201531c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:59.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.112+0000 7efc276ca640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc2015c550 0x7efc2014ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:59.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.112+0000 7efc276ca640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efc20073500 con 0x7efc2006c1d0 2024-09-19T08:44:59.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.112+0000 7efc276ca640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efc20073380 con 0x7efc2015c550 2024-09-19T08:44:59.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.112+0000 7efc276ca640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7efc20073680 con 0x7efc200cfad0 2024-09-19T08:44:59.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.112+0000 7efc25466640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc2006c1d0 0x7efc20147450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:59.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.112+0000 7efc25466640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc2006c1d0 0x7efc20147450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47810/0 (socket says 172.21.15.98:47810) 2024-09-19T08:44:59.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.112+0000 7efc25466640 1 -- 172.21.15.98:0/1424252704 learned_addr learned my addr 172.21.15.98:0/1424252704 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:59.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.112+0000 7efc24c65640 1 --2- 172.21.15.98:0/1424252704 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc200cfad0 0x7efc201531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:59.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.112+0000 7efc25c67640 1 --2- 172.21.15.98:0/1424252704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc2015c550 0x7efc2014ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:59.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.112+0000 7efc25466640 1 -- 172.21.15.98:0/1424252704 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc200cfad0 msgr2=0x7efc201531c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:59.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.112+0000 7efc25466640 1 --2- 172.21.15.98:0/1424252704 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc200cfad0 0x7efc201531c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:59.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.112+0000 7efc25466640 1 -- 172.21.15.98:0/1424252704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc2015c550 msgr2=0x7efc2014ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:59.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.112+0000 7efc25466640 1 --2- 172.21.15.98:0/1424252704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc2015c550 0x7efc2014ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:59.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.112+0000 7efc25466640 1 -- 172.21.15.98:0/1424252704 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc20154020 con 0x7efc2006c1d0 2024-09-19T08:44:59.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.112+0000 7efc25466640 1 --2- 172.21.15.98:0/1424252704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc2006c1d0 0x7efc20147450 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00e980 tx=0x7efc1c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:59.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.112+0000 7efc15ffb640 1 -- 172.21.15.98:0/1424252704 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c017070 con 0x7efc2006c1d0 2024-09-19T08:44:59.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.112+0000 7efc15ffb640 1 -- 172.21.15.98:0/1424252704 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc1c00ca00 con 0x7efc2006c1d0 2024-09-19T08:44:59.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.112+0000 7efc276ca640 1 -- 172.21.15.98:0/1424252704 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc20156420 con 0x7efc2006c1d0 2024-09-19T08:44:59.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.112+0000 7efc15ffb640 1 -- 172.21.15.98:0/1424252704 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c00cdd0 con 0x7efc2006c1d0 2024-09-19T08:44:59.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.112+0000 7efc276ca640 1 -- 172.21.15.98:0/1424252704 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc201569e0 con 0x7efc2006c1d0 2024-09-19T08:44:59.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.116+0000 7efc276ca640 1 -- 172.21.15.98:0/1424252704 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc200723a0 con 0x7efc2006c1d0 2024-09-19T08:44:59.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.116+0000 7efc15ffb640 1 -- 172.21.15.98:0/1424252704 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7efc1c01a070 con 0x7efc2006c1d0 2024-09-19T08:44:59.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.116+0000 7efc15ffb640 1 --2- 172.21.15.98:0/1424252704 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7efc04067cb0 0x7efc0406a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:59.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.116+0000 7efc15ffb640 1 -- 172.21.15.98:0/1424252704 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5737+0+0 (secure 0 0 0) 0x7efc1c0862b0 con 0x7efc2006c1d0 2024-09-19T08:44:59.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.116+0000 7efc24c65640 1 --2- 172.21.15.98:0/1424252704 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7efc04067cb0 0x7efc0406a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:59.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.116+0000 7efc24c65640 1 --2- 172.21.15.98:0/1424252704 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7efc04067cb0 0x7efc0406a170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efc20071f60 tx=0x7efc10009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:59.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.120+0000 7efc15ffb640 1 -- 172.21.15.98:0/1424252704 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7efc1c0567f0 con 0x7efc2006c1d0 2024-09-19T08:44:59.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.268+0000 7efc276ca640 1 -- 172.21.15.98:0/1424252704 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc201518a0 con 0x7efc2006c1d0 2024-09-19T08:44:59.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.272+0000 7efc15ffb640 1 -- 172.21.15.98:0/1424252704 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+16783 (secure 0 0 0) 0x7efc1c0560f0 con 0x7efc2006c1d0 2024-09-19T08:44:59.274 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:44:59.275 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":118,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:44:57.586783+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:44:53.861869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-19T08:44:54.909433+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:44:59.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.276+0000 7efc276ca640 1 -- 172.21.15.98:0/1424252704 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7efc04067cb0 msgr2=0x7efc0406a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:59.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.276+0000 7efc276ca640 1 --2- 172.21.15.98:0/1424252704 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7efc04067cb0 0x7efc0406a170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efc20071f60 tx=0x7efc10009040 comp rx=0 tx=0).stop 2024-09-19T08:44:59.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.276+0000 7efc276ca640 1 -- 172.21.15.98:0/1424252704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc2006c1d0 msgr2=0x7efc20147450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:59.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.276+0000 7efc276ca640 1 --2- 172.21.15.98:0/1424252704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc2006c1d0 0x7efc20147450 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00e980 tx=0x7efc1c00ee40 comp rx=0 tx=0).stop 2024-09-19T08:44:59.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.276+0000 7efc276ca640 1 -- 172.21.15.98:0/1424252704 shutdown_connections 2024-09-19T08:44:59.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.276+0000 7efc276ca640 1 --2- 172.21.15.98:0/1424252704 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7efc04067cb0 0x7efc0406a170 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:59.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.276+0000 7efc276ca640 1 --2- 172.21.15.98:0/1424252704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc2015c550 0x7efc2014ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:59.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.276+0000 7efc276ca640 1 --2- 172.21.15.98:0/1424252704 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc200cfad0 0x7efc201531c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:59.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.276+0000 7efc276ca640 1 --2- 172.21.15.98:0/1424252704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc2006c1d0 0x7efc20147450 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:59.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.276+0000 7efc276ca640 1 -- 172.21.15.98:0/1424252704 >> 172.21.15.98:0/1424252704 conn(0x7efc200b2010 msgr2=0x7efc200cf3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:59.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.276+0000 7efc276ca640 1 -- 172.21.15.98:0/1424252704 shutdown_connections 2024-09-19T08:44:59.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.276+0000 7efc276ca640 1 -- 172.21.15.98:0/1424252704 wait complete. 2024-09-19T08:44:59.292 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T08:44:59.292 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:44:59.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.396+0000 7fa6fa7ac640 1 Processor -- start 2024-09-19T08:44:59.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.396+0000 7fa6fa7ac640 1 -- start start 2024-09-19T08:44:59.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.400+0000 7fa6fa7ac640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa6f415c550 0x7fa6f406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:59.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.400+0000 7fa6fa7ac640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6f406c1d0 0x7fa6f40cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:59.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.400+0000 7fa6fa7ac640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6f406d310 con 0x7fa6f40cf900 2024-09-19T08:44:59.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.400+0000 7fa6fa7ac640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fa6f406c610 con 0x7fa6f415c550 2024-09-19T08:44:59.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.400+0000 7fa6fa7ac640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa6f406d5f0 con 0x7fa6f406c1d0 2024-09-19T08:44:59.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.400+0000 7fa6f37fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6f406c1d0 0x7fa6f40cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:59.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.400+0000 7fa6f3fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa6f415c550 0x7fa6f406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:59.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.400+0000 7fa6f3fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa6f415c550 0x7fa6f406bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:36860/0 (socket says 172.21.15.98:36860) 2024-09-19T08:44:59.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.400+0000 7fa6f3fff640 1 -- 172.21.15.98:0/729845814 learned_addr learned my addr 172.21.15.98:0/729845814 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:59.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.400+0000 7fa6f8d49640 1 --1- 172.21.15.98:0/729845814 >> v1:172.21.15.139:6789/0 conn(0x7fa6f40cf900 0x7fa6f40ce1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:41886/0 (socket says 172.21.15.98:41886) 2024-09-19T08:44:59.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.400+0000 7fa6f3fff640 1 -- 172.21.15.98:0/729845814 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6f406c1d0 msgr2=0x7fa6f40cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:59.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.400+0000 7fa6f3fff640 1 --2- 172.21.15.98:0/729845814 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6f406c1d0 0x7fa6f40cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:59.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.400+0000 7fa6f3fff640 1 -- 172.21.15.98:0/729845814 >> v1:172.21.15.139:6789/0 conn(0x7fa6f40cf900 legacy=0x7fa6f40ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:59.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.400+0000 7fa6f3fff640 1 -- 172.21.15.98:0/729845814 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f406d8d0 con 0x7fa6f415c550 2024-09-19T08:44:59.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.404+0000 7fa6f3fff640 1 --2- 172.21.15.98:0/729845814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6f415c550 0x7fa6f406bc90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4009870 tx=0x7fa6e401c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=163f0af530d523f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:59.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.404+0000 7fa6f2ffd640 1 -- 172.21.15.98:0/729845814 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e40b2070 con 0x7fa6f415c550 2024-09-19T08:44:59.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.404+0000 7fa6f2ffd640 1 -- 172.21.15.98:0/729845814 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6e40a9070 con 0x7fa6f415c550 2024-09-19T08:44:59.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.404+0000 7fa6f2ffd640 1 -- 172.21.15.98:0/729845814 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e40ad880 con 0x7fa6f415c550 2024-09-19T08:44:59.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.404+0000 7fa6fa7ac640 1 -- 172.21.15.98:0/729845814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6f415c550 msgr2=0x7fa6f406bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:59.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.404+0000 7fa6fa7ac640 1 --2- 172.21.15.98:0/729845814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6f415c550 0x7fa6f406bc90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4009870 tx=0x7fa6e401c560 comp rx=0 tx=0).stop 2024-09-19T08:44:59.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.404+0000 7fa6fa7ac640 1 -- 172.21.15.98:0/729845814 shutdown_connections 2024-09-19T08:44:59.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.404+0000 7fa6fa7ac640 1 --2- 172.21.15.98:0/729845814 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6f406c1d0 0x7fa6f40cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:59.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.404+0000 7fa6fa7ac640 1 --2- 172.21.15.98:0/729845814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6f415c550 0x7fa6f406bc90 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:59.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.404+0000 7fa6fa7ac640 1 -- 172.21.15.98:0/729845814 >> 172.21.15.98:0/729845814 conn(0x7fa6f40b2010 msgr2=0x7fa6f40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:59.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.404+0000 7fa6fa7ac640 1 -- 172.21.15.98:0/729845814 shutdown_connections 2024-09-19T08:44:59.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.404+0000 7fa6fa7ac640 1 -- 172.21.15.98:0/729845814 wait complete. 2024-09-19T08:44:59.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.404+0000 7fa6fa7ac640 1 Processor -- start 2024-09-19T08:44:59.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.404+0000 7fa6fa7ac640 1 -- start start 2024-09-19T08:44:59.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.404+0000 7fa6fa7ac640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6f406c1d0 0x7fa6f41598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:59.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.404+0000 7fa6fa7ac640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6f40cf900 0x7fa6f415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:59.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.404+0000 7fa6fa7ac640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6f415c550 0x7fa6f415db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:59.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.404+0000 7fa6fa7ac640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f4169230 con 0x7fa6f406c1d0 2024-09-19T08:44:59.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.404+0000 7fa6fa7ac640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f415cb40 con 0x7fa6f415c550 2024-09-19T08:44:59.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.404+0000 7fa6fa7ac640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa6f41693b0 con 0x7fa6f40cf900 2024-09-19T08:44:59.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.404+0000 7fa6f3fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6f406c1d0 0x7fa6f41598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:59.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.404+0000 7fa6f3fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6f406c1d0 0x7fa6f41598b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47822/0 (socket says 172.21.15.98:47822) 2024-09-19T08:44:59.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.404+0000 7fa6f3fff640 1 -- 172.21.15.98:0/2375302595 learned_addr learned my addr 172.21.15.98:0/2375302595 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:59.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.404+0000 7fa6f37fe640 1 --2- 172.21.15.98:0/2375302595 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6f40cf900 0x7fa6f415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:59.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.404+0000 7fa6f8d49640 1 --2- 172.21.15.98:0/2375302595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6f415c550 0x7fa6f415db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:59.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.404+0000 7fa6f3fff640 1 -- 172.21.15.98:0/2375302595 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6f40cf900 msgr2=0x7fa6f415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:59.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.404+0000 7fa6f3fff640 1 --2- 172.21.15.98:0/2375302595 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6f40cf900 0x7fa6f415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:59.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.408+0000 7fa6f3fff640 1 -- 172.21.15.98:0/2375302595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6f415c550 msgr2=0x7fa6f415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:59.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.408+0000 7fa6f3fff640 1 --2- 172.21.15.98:0/2375302595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6f415c550 0x7fa6f415db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:59.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.408+0000 7fa6f3fff640 1 -- 172.21.15.98:0/2375302595 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f415a100 con 0x7fa6f406c1d0 2024-09-19T08:44:59.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.408+0000 7fa6f37fe640 1 --2- 172.21.15.98:0/2375302595 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6f40cf900 0x7fa6f415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:59.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.408+0000 7fa6f8d49640 1 --2- 172.21.15.98:0/2375302595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6f415c550 0x7fa6f415db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:44:59.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.408+0000 7fa6f3fff640 1 --2- 172.21.15.98:0/2375302595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6f406c1d0 0x7fa6f41598b0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fa6e401a190 tx=0x7fa6e401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:59.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.408+0000 7fa6f0ff9640 1 -- 172.21.15.98:0/2375302595 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e40b2040 con 0x7fa6f406c1d0 2024-09-19T08:44:59.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.408+0000 7fa6f0ff9640 1 -- 172.21.15.98:0/2375302595 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6e40be070 con 0x7fa6f406c1d0 2024-09-19T08:44:59.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.408+0000 7fa6fa7ac640 1 -- 172.21.15.98:0/2375302595 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f415a3e0 con 0x7fa6f406c1d0 2024-09-19T08:44:59.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.408+0000 7fa6f0ff9640 1 -- 172.21.15.98:0/2375302595 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e40b8420 con 0x7fa6f406c1d0 2024-09-19T08:44:59.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.408+0000 7fa6fa7ac640 1 -- 172.21.15.98:0/2375302595 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f415d0b0 con 0x7fa6f406c1d0 2024-09-19T08:44:59.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.408+0000 7fa6fa7ac640 1 -- 172.21.15.98:0/2375302595 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6f40cf3c0 con 0x7fa6f406c1d0 2024-09-19T08:44:59.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.408+0000 7fa6f0ff9640 1 -- 172.21.15.98:0/2375302595 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7fa6e40ab070 con 0x7fa6f406c1d0 2024-09-19T08:44:59.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.408+0000 7fa6f0ff9640 1 --2- 172.21.15.98:0/2375302595 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa6e0067cb0 0x7fa6e006a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:59.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.408+0000 7fa6f0ff9640 1 -- 172.21.15.98:0/2375302595 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5737+0+0 (secure 0 0 0) 0x7fa6e41204c0 con 0x7fa6f406c1d0 2024-09-19T08:44:59.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.408+0000 7fa6f37fe640 1 --2- 172.21.15.98:0/2375302595 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa6e0067cb0 0x7fa6e006a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:59.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.408+0000 7fa6f37fe640 1 --2- 172.21.15.98:0/2375302595 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa6e0067cb0 0x7fa6e006a170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa6f414b1a0 tx=0x7fa6e8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:59.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.412+0000 7fa6f0ff9640 1 -- 172.21.15.98:0/2375302595 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa6e4016150 con 0x7fa6f406c1d0 2024-09-19T08:44:59.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.572+0000 7fa6fa7ac640 1 -- 172.21.15.98:0/2375302595 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa6f4153800 con 0x7fa6e0067cb0 2024-09-19T08:44:59.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.592+0000 7fa6f0ff9640 1 -- 172.21.15.98:0/2375302595 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252484 (secure 0 0 0) 0x7fa6f4153800 con 0x7fa6e0067cb0 2024-09-19T08:44:59.595 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:44:59.597 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:44:59.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.596+0000 7fa6fa7ac640 1 -- 172.21.15.98:0/2375302595 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa6e0067cb0 msgr2=0x7fa6e006a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:59.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.596+0000 7fa6fa7ac640 1 --2- 172.21.15.98:0/2375302595 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa6e0067cb0 0x7fa6e006a170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa6f414b1a0 tx=0x7fa6e8008040 comp rx=0 tx=0).stop 2024-09-19T08:44:59.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.596+0000 7fa6fa7ac640 1 -- 172.21.15.98:0/2375302595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6f406c1d0 msgr2=0x7fa6f41598b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:59.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.596+0000 7fa6fa7ac640 1 --2- 172.21.15.98:0/2375302595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6f406c1d0 0x7fa6f41598b0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fa6e401a190 tx=0x7fa6e401ada0 comp rx=0 tx=0).stop 2024-09-19T08:44:59.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.596+0000 7fa6fa7ac640 1 -- 172.21.15.98:0/2375302595 shutdown_connections 2024-09-19T08:44:59.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.596+0000 7fa6fa7ac640 1 --2- 172.21.15.98:0/2375302595 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa6e0067cb0 0x7fa6e006a170 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:59.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.596+0000 7fa6fa7ac640 1 --2- 172.21.15.98:0/2375302595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6f415c550 0x7fa6f415db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:59.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.596+0000 7fa6fa7ac640 1 --2- 172.21.15.98:0/2375302595 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6f40cf900 0x7fa6f415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:59.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.596+0000 7fa6fa7ac640 1 --2- 172.21.15.98:0/2375302595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6f406c1d0 0x7fa6f41598b0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:59.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.596+0000 7fa6fa7ac640 1 -- 172.21.15.98:0/2375302595 >> 172.21.15.98:0/2375302595 conn(0x7fa6f40b2010 msgr2=0x7fa6f414c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:59.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.596+0000 7fa6fa7ac640 1 -- 172.21.15.98:0/2375302595 shutdown_connections 2024-09-19T08:44:59.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.596+0000 7fa6fa7ac640 1 -- 172.21.15.98:0/2375302595 wait complete. 2024-09-19T08:44:59.614 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2024-09-19T08:44:58.911525+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":113,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262144,"kb_used_data":12620,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712576,"statfs":{"total":773094113280,"available":772825677824,"internally_reserved":0,"allocated":12922880,"data_stored":6342268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.375121"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.746428+0000","last_change":"2024-09-19T08:44:56.639227+0000","last_active":"2024-09-19T08:44:58.746428+0000","last_peered":"2024-09-19T08:44:58.746428+0000","last_clean":"2024-09-19T08:44:58.746428+0000","last_became_active":"2024-09-19T08:44:56.638909+0000","last_became_peered":"2024-09-19T08:44:56.638909+0000","last_unstale":"2024-09-19T08:44:58.746428+0000","last_undegraded":"2024-09-19T08:44:58.746428+0000","last_fullsized":"2024-09-19T08:44:58.746428+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:49.962545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.591373+0000","last_change":"2024-09-19T08:44:56.604280+0000","last_active":"2024-09-19T08:44:57.591373+0000","last_peered":"2024-09-19T08:44:57.591373+0000","last_clean":"2024-09-19T08:44:57.591373+0000","last_became_active":"2024-09-19T08:44:56.603523+0000","last_became_peered":"2024-09-19T08:44:56.603523+0000","last_unstale":"2024-09-19T08:44:57.591373+0000","last_undegraded":"2024-09-19T08:44:57.591373+0000","last_fullsized":"2024-09-19T08:44:57.591373+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:03:16.391474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.749090+0000","last_change":"2024-09-19T08:44:56.619936+0000","last_active":"2024-09-19T08:44:58.749090+0000","last_peered":"2024-09-19T08:44:58.749090+0000","last_clean":"2024-09-19T08:44:58.749090+0000","last_became_active":"2024-09-19T08:44:56.619196+0000","last_became_peered":"2024-09-19T08:44:56.619196+0000","last_unstale":"2024-09-19T08:44:58.749090+0000","last_undegraded":"2024-09-19T08:44:58.749090+0000","last_fullsized":"2024-09-19T08:44:58.749090+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:27.887227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.745823+0000","last_change":"2024-09-19T08:44:56.623732+0000","last_active":"2024-09-19T08:44:58.745823+0000","last_peered":"2024-09-19T08:44:58.745823+0000","last_clean":"2024-09-19T08:44:58.745823+0000","last_became_active":"2024-09-19T08:44:56.622354+0000","last_became_peered":"2024-09-19T08:44:56.622354+0000","last_unstale":"2024-09-19T08:44:58.745823+0000","last_undegraded":"2024-09-19T08:44:58.745823+0000","last_fullsized":"2024-09-19T08:44:58.745823+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:32:32.719720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.633670+0000","last_change":"2024-09-19T08:44:56.634714+0000","last_active":"2024-09-19T08:44:57.633670+0000","last_peered":"2024-09-19T08:44:57.633670+0000","last_clean":"2024-09-19T08:44:57.633670+0000","last_became_active":"2024-09-19T08:44:56.633493+0000","last_became_peered":"2024-09-19T08:44:56.633493+0000","last_unstale":"2024-09-19T08:44:57.633670+0000","last_undegraded":"2024-09-19T08:44:57.633670+0000","last_fullsized":"2024-09-19T08:44:57.633670+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:53.603787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.597941+0000","last_change":"2024-09-19T08:44:56.624900+0000","last_active":"2024-09-19T08:44:57.597941+0000","last_peered":"2024-09-19T08:44:57.597941+0000","last_clean":"2024-09-19T08:44:57.597941+0000","last_became_active":"2024-09-19T08:44:56.623944+0000","last_became_peered":"2024-09-19T08:44:56.623944+0000","last_unstale":"2024-09-19T08:44:57.597941+0000","last_undegraded":"2024-09-19T08:44:57.597941+0000","last_fullsized":"2024-09-19T08:44:57.597941+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:16.566499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.597283+0000","last_change":"2024-09-19T08:44:56.617780+0000","last_active":"2024-09-19T08:44:57.597283+0000","last_peered":"2024-09-19T08:44:57.597283+0000","last_clean":"2024-09-19T08:44:57.597283+0000","last_became_active":"2024-09-19T08:44:56.617050+0000","last_became_peered":"2024-09-19T08:44:56.617050+0000","last_unstale":"2024-09-19T08:44:57.597283+0000","last_undegraded":"2024-09-19T08:44:57.597283+0000","last_fullsized":"2024-09-19T08:44:57.597283+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:29:42.509418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.749705+0000","last_change":"2024-09-19T08:44:56.653073+0000","last_active":"2024-09-19T08:44:58.749705+0000","last_peered":"2024-09-19T08:44:58.749705+0000","last_clean":"2024-09-19T08:44:58.749705+0000","last_became_active":"2024-09-19T08:44:56.651513+0000","last_became_peered":"2024-09-19T08:44:56.651513+0000","last_unstale":"2024-09-19T08:44:58.749705+0000","last_undegraded":"2024-09-19T08:44:58.749705+0000","last_fullsized":"2024-09-19T08:44:58.749705+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:35:10.729538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.749635+0000","last_change":"2024-09-19T08:44:56.628070+0000","last_active":"2024-09-19T08:44:58.749635+0000","last_peered":"2024-09-19T08:44:58.749635+0000","last_clean":"2024-09-19T08:44:58.749635+0000","last_became_active":"2024-09-19T08:44:56.627394+0000","last_became_peered":"2024-09-19T08:44:56.627394+0000","last_unstale":"2024-09-19T08:44:58.749635+0000","last_undegraded":"2024-09-19T08:44:58.749635+0000","last_fullsized":"2024-09-19T08:44:58.749635+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:40.416631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.591184+0000","last_change":"2024-09-19T08:44:56.604361+0000","last_active":"2024-09-19T08:44:57.591184+0000","last_peered":"2024-09-19T08:44:57.591184+0000","last_clean":"2024-09-19T08:44:57.591184+0000","last_became_active":"2024-09-19T08:44:56.603636+0000","last_became_peered":"2024-09-19T08:44:56.603636+0000","last_unstale":"2024-09-19T08:44:57.591184+0000","last_undegraded":"2024-09-19T08:44:57.591184+0000","last_fullsized":"2024-09-19T08:44:57.591184+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:59.215648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.425331+0000","last_change":"2024-09-19T08:44:56.621177+0000","last_active":"2024-09-19T08:44:58.425331+0000","last_peered":"2024-09-19T08:44:58.425331+0000","last_clean":"2024-09-19T08:44:58.425331+0000","last_became_active":"2024-09-19T08:44:56.620516+0000","last_became_peered":"2024-09-19T08:44:56.620516+0000","last_unstale":"2024-09-19T08:44:58.425331+0000","last_undegraded":"2024-09-19T08:44:58.425331+0000","last_fullsized":"2024-09-19T08:44:58.425331+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:09:46.759640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.748691+0000","last_change":"2024-09-19T08:44:56.627982+0000","last_active":"2024-09-19T08:44:58.748691+0000","last_peered":"2024-09-19T08:44:58.748691+0000","last_clean":"2024-09-19T08:44:58.748691+0000","last_became_active":"2024-09-19T08:44:56.627281+0000","last_became_peered":"2024-09-19T08:44:56.627281+0000","last_unstale":"2024-09-19T08:44:58.748691+0000","last_undegraded":"2024-09-19T08:44:58.748691+0000","last_fullsized":"2024-09-19T08:44:58.748691+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:24:53.239123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.745594+0000","last_change":"2024-09-19T08:44:56.640239+0000","last_active":"2024-09-19T08:44:58.745594+0000","last_peered":"2024-09-19T08:44:58.745594+0000","last_clean":"2024-09-19T08:44:58.745594+0000","last_became_active":"2024-09-19T08:44:56.639949+0000","last_became_peered":"2024-09-19T08:44:56.639949+0000","last_unstale":"2024-09-19T08:44:58.745594+0000","last_undegraded":"2024-09-19T08:44:58.745594+0000","last_fullsized":"2024-09-19T08:44:58.745594+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:03.414124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.591096+0000","last_change":"2024-09-19T08:44:56.628181+0000","last_active":"2024-09-19T08:44:57.591096+0000","last_peered":"2024-09-19T08:44:57.591096+0000","last_clean":"2024-09-19T08:44:57.591096+0000","last_became_active":"2024-09-19T08:44:56.627824+0000","last_became_peered":"2024-09-19T08:44:56.627824+0000","last_unstale":"2024-09-19T08:44:57.591096+0000","last_undegraded":"2024-09-19T08:44:57.591096+0000","last_fullsized":"2024-09-19T08:44:57.591096+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:33:41.515270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.744671+0000","last_change":"2024-09-19T08:44:56.627419+0000","last_active":"2024-09-19T08:44:58.744671+0000","last_peered":"2024-09-19T08:44:58.744671+0000","last_clean":"2024-09-19T08:44:58.744671+0000","last_became_active":"2024-09-19T08:44:56.624251+0000","last_became_peered":"2024-09-19T08:44:56.624251+0000","last_unstale":"2024-09-19T08:44:58.744671+0000","last_undegraded":"2024-09-19T08:44:58.744671+0000","last_fullsized":"2024-09-19T08:44:58.744671+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:55:50.503091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.745335+0000","last_change":"2024-09-19T08:44:56.631755+0000","last_active":"2024-09-19T08:44:58.745335+0000","last_peered":"2024-09-19T08:44:58.745335+0000","last_clean":"2024-09-19T08:44:58.745335+0000","last_became_active":"2024-09-19T08:44:56.631432+0000","last_became_peered":"2024-09-19T08:44:56.631432+0000","last_unstale":"2024-09-19T08:44:58.745335+0000","last_undegraded":"2024-09-19T08:44:58.745335+0000","last_fullsized":"2024-09-19T08:44:58.745335+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:02:15.630802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.598359+0000","last_change":"2024-09-19T08:44:56.613209+0000","last_active":"2024-09-19T08:44:57.598359+0000","last_peered":"2024-09-19T08:44:57.598359+0000","last_clean":"2024-09-19T08:44:57.598359+0000","last_became_active":"2024-09-19T08:44:56.612439+0000","last_became_peered":"2024-09-19T08:44:56.612439+0000","last_unstale":"2024-09-19T08:44:57.598359+0000","last_undegraded":"2024-09-19T08:44:57.598359+0000","last_fullsized":"2024-09-19T08:44:57.598359+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:53.603787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.745655+0000","last_change":"2024-09-19T08:44:56.641215+0000","last_active":"2024-09-19T08:44:58.745655+0000","last_peered":"2024-09-19T08:44:58.745655+0000","last_clean":"2024-09-19T08:44:58.745655+0000","last_became_active":"2024-09-19T08:44:56.640784+0000","last_became_peered":"2024-09-19T08:44:56.640784+0000","last_unstale":"2024-09-19T08:44:58.745655+0000","last_undegraded":"2024-09-19T08:44:58.745655+0000","last_fullsized":"2024-09-19T08:44:58.745655+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:08:18.033098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.425378+0000","last_change":"2024-09-19T08:44:56.640519+0000","last_active":"2024-09-19T08:44:58.425378+0000","last_peered":"2024-09-19T08:44:58.425378+0000","last_clean":"2024-09-19T08:44:58.425378+0000","last_became_active":"2024-09-19T08:44:56.640216+0000","last_became_peered":"2024-09-19T08:44:56.640216+0000","last_unstale":"2024-09-19T08:44:58.425378+0000","last_undegraded":"2024-09-19T08:44:58.425378+0000","last_fullsized":"2024-09-19T08:44:58.425378+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:31:13.838216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.746180+0000","last_change":"2024-09-19T08:44:56.622948+0000","last_active":"2024-09-19T08:44:58.746180+0000","last_peered":"2024-09-19T08:44:58.746180+0000","last_clean":"2024-09-19T08:44:58.746180+0000","last_became_active":"2024-09-19T08:44:56.621105+0000","last_became_peered":"2024-09-19T08:44:56.621105+0000","last_unstale":"2024-09-19T08:44:58.746180+0000","last_undegraded":"2024-09-19T08:44:58.746180+0000","last_fullsized":"2024-09-19T08:44:58.746180+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:43.766467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.634105+0000","last_change":"2024-09-19T08:44:56.634172+0000","last_active":"2024-09-19T08:44:57.634105+0000","last_peered":"2024-09-19T08:44:57.634105+0000","last_clean":"2024-09-19T08:44:57.634105+0000","last_became_active":"2024-09-19T08:44:56.632589+0000","last_became_peered":"2024-09-19T08:44:56.632589+0000","last_unstale":"2024-09-19T08:44:57.634105+0000","last_undegraded":"2024-09-19T08:44:57.634105+0000","last_fullsized":"2024-09-19T08:44:57.634105+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:38:14.461908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.424890+0000","last_change":"2024-09-19T08:44:56.620891+0000","last_active":"2024-09-19T08:44:58.424890+0000","last_peered":"2024-09-19T08:44:58.424890+0000","last_clean":"2024-09-19T08:44:58.424890+0000","last_became_active":"2024-09-19T08:44:56.620215+0000","last_became_peered":"2024-09-19T08:44:56.620215+0000","last_unstale":"2024-09-19T08:44:58.424890+0000","last_undegraded":"2024-09-19T08:44:58.424890+0000","last_fullsized":"2024-09-19T08:44:58.424890+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:39.099740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.749000+0000","last_change":"2024-09-19T08:44:56.629006+0000","last_active":"2024-09-19T08:44:58.749000+0000","last_peered":"2024-09-19T08:44:58.749000+0000","last_clean":"2024-09-19T08:44:58.749000+0000","last_became_active":"2024-09-19T08:44:56.627353+0000","last_became_peered":"2024-09-19T08:44:56.627353+0000","last_unstale":"2024-09-19T08:44:58.749000+0000","last_undegraded":"2024-09-19T08:44:58.749000+0000","last_fullsized":"2024-09-19T08:44:58.749000+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:24:16.145118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.424637+0000","last_change":"2024-09-19T08:44:56.630278+0000","last_active":"2024-09-19T08:44:58.424637+0000","last_peered":"2024-09-19T08:44:58.424637+0000","last_clean":"2024-09-19T08:44:58.424637+0000","last_became_active":"2024-09-19T08:44:56.629876+0000","last_became_peered":"2024-09-19T08:44:56.629876+0000","last_unstale":"2024-09-19T08:44:58.424637+0000","last_undegraded":"2024-09-19T08:44:58.424637+0000","last_fullsized":"2024-09-19T08:44:58.424637+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:50:06.823214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.591016+0000","last_change":"2024-09-19T08:44:56.618618+0000","last_active":"2024-09-19T08:44:57.591016+0000","last_peered":"2024-09-19T08:44:57.591016+0000","last_clean":"2024-09-19T08:44:57.591016+0000","last_became_active":"2024-09-19T08:44:56.617802+0000","last_became_peered":"2024-09-19T08:44:56.617802+0000","last_unstale":"2024-09-19T08:44:57.591016+0000","last_undegraded":"2024-09-19T08:44:57.591016+0000","last_fullsized":"2024-09-19T08:44:57.591016+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:04:45.876735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.745798+0000","last_change":"2024-09-19T08:44:56.637309+0000","last_active":"2024-09-19T08:44:58.745798+0000","last_peered":"2024-09-19T08:44:58.745798+0000","last_clean":"2024-09-19T08:44:58.745798+0000","last_became_active":"2024-09-19T08:44:56.637076+0000","last_became_peered":"2024-09-19T08:44:56.637076+0000","last_unstale":"2024-09-19T08:44:58.745798+0000","last_undegraded":"2024-09-19T08:44:58.745798+0000","last_fullsized":"2024-09-19T08:44:58.745798+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:27:13.514549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.744883+0000","last_change":"2024-09-19T08:44:56.639195+0000","last_active":"2024-09-19T08:44:58.744883+0000","last_peered":"2024-09-19T08:44:58.744883+0000","last_clean":"2024-09-19T08:44:58.744883+0000","last_became_active":"2024-09-19T08:44:56.638886+0000","last_became_peered":"2024-09-19T08:44:56.638886+0000","last_unstale":"2024-09-19T08:44:58.744883+0000","last_undegraded":"2024-09-19T08:44:58.744883+0000","last_fullsized":"2024-09-19T08:44:58.744883+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:16.743988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.748269+0000","last_change":"2024-09-19T08:44:56.634911+0000","last_active":"2024-09-19T08:44:58.748269+0000","last_peered":"2024-09-19T08:44:58.748269+0000","last_clean":"2024-09-19T08:44:58.748269+0000","last_became_active":"2024-09-19T08:44:56.634224+0000","last_became_peered":"2024-09-19T08:44:56.634224+0000","last_unstale":"2024-09-19T08:44:58.748269+0000","last_undegraded":"2024-09-19T08:44:58.748269+0000","last_fullsized":"2024-09-19T08:44:58.748269+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:30.332833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.745485+0000","last_change":"2024-09-19T08:44:56.628614+0000","last_active":"2024-09-19T08:44:58.745485+0000","last_peered":"2024-09-19T08:44:58.745485+0000","last_clean":"2024-09-19T08:44:58.745485+0000","last_became_active":"2024-09-19T08:44:56.628336+0000","last_became_peered":"2024-09-19T08:44:56.628336+0000","last_unstale":"2024-09-19T08:44:58.745485+0000","last_undegraded":"2024-09-19T08:44:58.745485+0000","last_fullsized":"2024-09-19T08:44:58.745485+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:32:13.227944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.744908+0000","last_change":"2024-09-19T08:44:56.630967+0000","last_active":"2024-09-19T08:44:58.744908+0000","last_peered":"2024-09-19T08:44:58.744908+0000","last_clean":"2024-09-19T08:44:58.744908+0000","last_became_active":"2024-09-19T08:44:56.630172+0000","last_became_peered":"2024-09-19T08:44:56.630172+0000","last_unstale":"2024-09-19T08:44:58.744908+0000","last_undegraded":"2024-09-19T08:44:58.744908+0000","last_fullsized":"2024-09-19T08:44:58.744908+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:53.189321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.633623+0000","last_change":"2024-09-19T08:44:56.634769+0000","last_active":"2024-09-19T08:44:57.633623+0000","last_peered":"2024-09-19T08:44:57.633623+0000","last_clean":"2024-09-19T08:44:57.633623+0000","last_became_active":"2024-09-19T08:44:56.634013+0000","last_became_peered":"2024-09-19T08:44:56.634013+0000","last_unstale":"2024-09-19T08:44:57.633623+0000","last_undegraded":"2024-09-19T08:44:57.633623+0000","last_fullsized":"2024-09-19T08:44:57.633623+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:16.566499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.597299+0000","last_change":"2024-09-19T08:44:56.627395+0000","last_active":"2024-09-19T08:44:57.597299+0000","last_peered":"2024-09-19T08:44:57.597299+0000","last_clean":"2024-09-19T08:44:57.597299+0000","last_became_active":"2024-09-19T08:44:56.626662+0000","last_became_peered":"2024-09-19T08:44:56.626662+0000","last_unstale":"2024-09-19T08:44:57.597299+0000","last_undegraded":"2024-09-19T08:44:57.597299+0000","last_fullsized":"2024-09-19T08:44:57.597299+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:36:07.346236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.745956+0000","last_change":"2024-09-19T08:44:56.633036+0000","last_active":"2024-09-19T08:44:58.745956+0000","last_peered":"2024-09-19T08:44:58.745956+0000","last_clean":"2024-09-19T08:44:58.745956+0000","last_became_active":"2024-09-19T08:44:56.632697+0000","last_became_peered":"2024-09-19T08:44:56.632697+0000","last_unstale":"2024-09-19T08:44:58.745956+0000","last_undegraded":"2024-09-19T08:44:58.745956+0000","last_fullsized":"2024-09-19T08:44:58.745956+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:03.011362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"117'4","reported_seq":18,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.634379+0000","last_change":"2024-09-19T08:44:55.582796+0000","last_active":"2024-09-19T08:44:57.634379+0000","last_peered":"2024-09-19T08:44:57.634379+0000","last_clean":"2024-09-19T08:44:57.634379+0000","last_became_active":"2024-09-19T08:44:55.582055+0000","last_became_peered":"2024-09-19T08:44:55.582055+0000","last_unstale":"2024-09-19T08:44:57.634379+0000","last_undegraded":"2024-09-19T08:44:57.634379+0000","last_fullsized":"2024-09-19T08:44:57.634379+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:01.341975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.749790+0000","last_change":"2024-09-19T08:44:56.626375+0000","last_active":"2024-09-19T08:44:58.749790+0000","last_peered":"2024-09-19T08:44:58.749790+0000","last_clean":"2024-09-19T08:44:58.749790+0000","last_became_active":"2024-09-19T08:44:56.625571+0000","last_became_peered":"2024-09-19T08:44:56.625571+0000","last_unstale":"2024-09-19T08:44:58.749790+0000","last_undegraded":"2024-09-19T08:44:58.749790+0000","last_fullsized":"2024-09-19T08:44:58.749790+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:49.992899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.748317+0000","last_change":"2024-09-19T08:44:55.566152+0000","last_active":"2024-09-19T08:44:58.748317+0000","last_peered":"2024-09-19T08:44:58.748317+0000","last_clean":"2024-09-19T08:44:58.748317+0000","last_became_active":"2024-09-19T08:44:55.565226+0000","last_became_peered":"2024-09-19T08:44:55.565226+0000","last_unstale":"2024-09-19T08:44:58.748317+0000","last_undegraded":"2024-09-19T08:44:58.748317+0000","last_fullsized":"2024-09-19T08:44:58.748317+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:07:45.509629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.633976+0000","last_change":"2024-09-19T08:44:56.642587+0000","last_active":"2024-09-19T08:44:57.633976+0000","last_peered":"2024-09-19T08:44:57.633976+0000","last_clean":"2024-09-19T08:44:57.633976+0000","last_became_active":"2024-09-19T08:44:56.640966+0000","last_became_peered":"2024-09-19T08:44:56.640966+0000","last_unstale":"2024-09-19T08:44:57.633976+0000","last_undegraded":"2024-09-19T08:44:57.633976+0000","last_fullsized":"2024-09-19T08:44:57.633976+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:22:10.389495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.425607+0000","last_change":"2024-09-19T08:44:55.555645+0000","last_active":"2024-09-19T08:44:58.425607+0000","last_peered":"2024-09-19T08:44:58.425607+0000","last_clean":"2024-09-19T08:44:58.425607+0000","last_became_active":"2024-09-19T08:44:55.555244+0000","last_became_peered":"2024-09-19T08:44:55.555244+0000","last_unstale":"2024-09-19T08:44:58.425607+0000","last_undegraded":"2024-09-19T08:44:58.425607+0000","last_fullsized":"2024-09-19T08:44:58.425607+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:15:10.647852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.634323+0000","last_change":"2024-09-19T08:44:56.620256+0000","last_active":"2024-09-19T08:44:57.634323+0000","last_peered":"2024-09-19T08:44:57.634323+0000","last_clean":"2024-09-19T08:44:57.634323+0000","last_became_active":"2024-09-19T08:44:56.618552+0000","last_became_peered":"2024-09-19T08:44:56.618552+0000","last_unstale":"2024-09-19T08:44:57.634323+0000","last_undegraded":"2024-09-19T08:44:57.634323+0000","last_fullsized":"2024-09-19T08:44:57.634323+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:46.494060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.591722+0000","last_change":"2024-09-19T08:44:55.556416+0000","last_active":"2024-09-19T08:44:57.591722+0000","last_peered":"2024-09-19T08:44:57.591722+0000","last_clean":"2024-09-19T08:44:57.591722+0000","last_became_active":"2024-09-19T08:44:55.556039+0000","last_became_peered":"2024-09-19T08:44:55.556039+0000","last_unstale":"2024-09-19T08:44:57.591722+0000","last_undegraded":"2024-09-19T08:44:57.591722+0000","last_fullsized":"2024-09-19T08:44:57.591722+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:24.239361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.745744+0000","last_change":"2024-09-19T08:44:56.640990+0000","last_active":"2024-09-19T08:44:58.745744+0000","last_peered":"2024-09-19T08:44:58.745744+0000","last_clean":"2024-09-19T08:44:58.745744+0000","last_became_active":"2024-09-19T08:44:56.640744+0000","last_became_peered":"2024-09-19T08:44:56.640744+0000","last_unstale":"2024-09-19T08:44:58.745744+0000","last_undegraded":"2024-09-19T08:44:58.745744+0000","last_fullsized":"2024-09-19T08:44:58.745744+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:18.197696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.591646+0000","last_change":"2024-09-19T08:44:55.557528+0000","last_active":"2024-09-19T08:44:57.591646+0000","last_peered":"2024-09-19T08:44:57.591646+0000","last_clean":"2024-09-19T08:44:57.591646+0000","last_became_active":"2024-09-19T08:44:55.557209+0000","last_became_peered":"2024-09-19T08:44:55.557209+0000","last_unstale":"2024-09-19T08:44:57.591646+0000","last_undegraded":"2024-09-19T08:44:57.591646+0000","last_fullsized":"2024-09-19T08:44:57.591646+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:47:47.959474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.597468+0000","last_change":"2024-09-19T08:44:56.611611+0000","last_active":"2024-09-19T08:44:57.597468+0000","last_peered":"2024-09-19T08:44:57.597468+0000","last_clean":"2024-09-19T08:44:57.597468+0000","last_became_active":"2024-09-19T08:44:56.610910+0000","last_became_peered":"2024-09-19T08:44:56.610910+0000","last_unstale":"2024-09-19T08:44:57.597468+0000","last_undegraded":"2024-09-19T08:44:57.597468+0000","last_fullsized":"2024-09-19T08:44:57.597468+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:46:41.767059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.746029+0000","last_change":"2024-09-19T08:44:55.555018+0000","last_active":"2024-09-19T08:44:58.746029+0000","last_peered":"2024-09-19T08:44:58.746029+0000","last_clean":"2024-09-19T08:44:58.746029+0000","last_became_active":"2024-09-19T08:44:55.554750+0000","last_became_peered":"2024-09-19T08:44:55.554750+0000","last_unstale":"2024-09-19T08:44:58.746029+0000","last_undegraded":"2024-09-19T08:44:58.746029+0000","last_fullsized":"2024-09-19T08:44:58.746029+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:34.048242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.749904+0000","last_change":"2024-09-19T08:44:56.639019+0000","last_active":"2024-09-19T08:44:58.749904+0000","last_peered":"2024-09-19T08:44:58.749904+0000","last_clean":"2024-09-19T08:44:58.749904+0000","last_became_active":"2024-09-19T08:44:56.638039+0000","last_became_peered":"2024-09-19T08:44:56.638039+0000","last_unstale":"2024-09-19T08:44:58.749904+0000","last_undegraded":"2024-09-19T08:44:58.749904+0000","last_fullsized":"2024-09-19T08:44:58.749904+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:08:06.100292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.597565+0000","last_change":"2024-09-19T08:44:55.568111+0000","last_active":"2024-09-19T08:44:57.597565+0000","last_peered":"2024-09-19T08:44:57.597565+0000","last_clean":"2024-09-19T08:44:57.597565+0000","last_became_active":"2024-09-19T08:44:55.567310+0000","last_became_peered":"2024-09-19T08:44:55.567310+0000","last_unstale":"2024-09-19T08:44:57.597565+0000","last_undegraded":"2024-09-19T08:44:57.597565+0000","last_fullsized":"2024-09-19T08:44:57.597565+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:37.224553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.590980+0000","last_change":"2024-09-19T08:44:56.609418+0000","last_active":"2024-09-19T08:44:57.590980+0000","last_peered":"2024-09-19T08:44:57.590980+0000","last_clean":"2024-09-19T08:44:57.590980+0000","last_became_active":"2024-09-19T08:44:56.608605+0000","last_became_peered":"2024-09-19T08:44:56.608605+0000","last_unstale":"2024-09-19T08:44:57.590980+0000","last_undegraded":"2024-09-19T08:44:57.590980+0000","last_fullsized":"2024-09-19T08:44:57.590980+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:50.432353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.590863+0000","last_change":"2024-09-19T08:44:55.581814+0000","last_active":"2024-09-19T08:44:57.590863+0000","last_peered":"2024-09-19T08:44:57.590863+0000","last_clean":"2024-09-19T08:44:57.590863+0000","last_became_active":"2024-09-19T08:44:55.581468+0000","last_became_peered":"2024-09-19T08:44:55.581468+0000","last_unstale":"2024-09-19T08:44:57.590863+0000","last_undegraded":"2024-09-19T08:44:57.590863+0000","last_fullsized":"2024-09-19T08:44:57.590863+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:50:18.779182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.745740+0000","last_change":"2024-09-19T08:44:56.615106+0000","last_active":"2024-09-19T08:44:58.745740+0000","last_peered":"2024-09-19T08:44:58.745740+0000","last_clean":"2024-09-19T08:44:58.745740+0000","last_became_active":"2024-09-19T08:44:56.614365+0000","last_became_peered":"2024-09-19T08:44:56.614365+0000","last_unstale":"2024-09-19T08:44:58.745740+0000","last_undegraded":"2024-09-19T08:44:58.745740+0000","last_fullsized":"2024-09-19T08:44:58.745740+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:58:42.773564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.748749+0000","last_change":"2024-09-19T08:44:55.574633+0000","last_active":"2024-09-19T08:44:58.748749+0000","last_peered":"2024-09-19T08:44:58.748749+0000","last_clean":"2024-09-19T08:44:58.748749+0000","last_became_active":"2024-09-19T08:44:55.573955+0000","last_became_peered":"2024-09-19T08:44:55.573955+0000","last_unstale":"2024-09-19T08:44:58.748749+0000","last_undegraded":"2024-09-19T08:44:58.748749+0000","last_fullsized":"2024-09-19T08:44:58.748749+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:11:24.299035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.748705+0000","last_change":"2024-09-19T08:44:56.631805+0000","last_active":"2024-09-19T08:44:58.748705+0000","last_peered":"2024-09-19T08:44:58.748705+0000","last_clean":"2024-09-19T08:44:58.748705+0000","last_became_active":"2024-09-19T08:44:56.630780+0000","last_became_peered":"2024-09-19T08:44:56.630780+0000","last_unstale":"2024-09-19T08:44:58.748705+0000","last_undegraded":"2024-09-19T08:44:58.748705+0000","last_fullsized":"2024-09-19T08:44:58.748705+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:04:45.876735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.746090+0000","last_change":"2024-09-19T08:44:55.574426+0000","last_active":"2024-09-19T08:44:58.746090+0000","last_peered":"2024-09-19T08:44:58.746090+0000","last_clean":"2024-09-19T08:44:58.746090+0000","last_became_active":"2024-09-19T08:44:55.574094+0000","last_became_peered":"2024-09-19T08:44:55.574094+0000","last_unstale":"2024-09-19T08:44:58.746090+0000","last_undegraded":"2024-09-19T08:44:58.746090+0000","last_fullsized":"2024-09-19T08:44:58.746090+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:08:05.097099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.634185+0000","last_change":"2024-09-19T08:44:56.631032+0000","last_active":"2024-09-19T08:44:57.634185+0000","last_peered":"2024-09-19T08:44:57.634185+0000","last_clean":"2024-09-19T08:44:57.634185+0000","last_became_active":"2024-09-19T08:44:56.630305+0000","last_became_peered":"2024-09-19T08:44:56.630305+0000","last_unstale":"2024-09-19T08:44:57.634185+0000","last_undegraded":"2024-09-19T08:44:57.634185+0000","last_fullsized":"2024-09-19T08:44:57.634185+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:48:08.894727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"117'3","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.748356+0000","last_change":"2024-09-19T08:44:55.572275+0000","last_active":"2024-09-19T08:44:58.748356+0000","last_peered":"2024-09-19T08:44:58.748356+0000","last_clean":"2024-09-19T08:44:58.748356+0000","last_became_active":"2024-09-19T08:44:55.571553+0000","last_became_peered":"2024-09-19T08:44:55.571553+0000","last_unstale":"2024-09-19T08:44:58.748356+0000","last_undegraded":"2024-09-19T08:44:58.748356+0000","last_fullsized":"2024-09-19T08:44:58.748356+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:41:18.647623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.634327+0000","last_change":"2024-09-19T08:44:56.620881+0000","last_active":"2024-09-19T08:44:57.634327+0000","last_peered":"2024-09-19T08:44:57.634327+0000","last_clean":"2024-09-19T08:44:57.634327+0000","last_became_active":"2024-09-19T08:44:56.619561+0000","last_became_peered":"2024-09-19T08:44:56.619561+0000","last_unstale":"2024-09-19T08:44:57.634327+0000","last_undegraded":"2024-09-19T08:44:57.634327+0000","last_fullsized":"2024-09-19T08:44:57.634327+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:05:50.023175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"117'6","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.748718+0000","last_change":"2024-09-19T08:44:55.584512+0000","last_active":"2024-09-19T08:44:58.748718+0000","last_peered":"2024-09-19T08:44:58.748718+0000","last_clean":"2024-09-19T08:44:58.748718+0000","last_became_active":"2024-09-19T08:44:55.582785+0000","last_became_peered":"2024-09-19T08:44:55.582785+0000","last_unstale":"2024-09-19T08:44:58.748718+0000","last_undegraded":"2024-09-19T08:44:58.748718+0000","last_fullsized":"2024-09-19T08:44:58.748718+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:57:29.207492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.745668+0000","last_change":"2024-09-19T08:44:56.630623+0000","last_active":"2024-09-19T08:44:58.745668+0000","last_peered":"2024-09-19T08:44:58.745668+0000","last_clean":"2024-09-19T08:44:58.745668+0000","last_became_active":"2024-09-19T08:44:56.630267+0000","last_became_peered":"2024-09-19T08:44:56.630267+0000","last_unstale":"2024-09-19T08:44:58.745668+0000","last_undegraded":"2024-09-19T08:44:58.745668+0000","last_fullsized":"2024-09-19T08:44:58.745668+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:43.907032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.745283+0000","last_change":"2024-09-19T08:44:55.556466+0000","last_active":"2024-09-19T08:44:58.745283+0000","last_peered":"2024-09-19T08:44:58.745283+0000","last_clean":"2024-09-19T08:44:58.745283+0000","last_became_active":"2024-09-19T08:44:55.555465+0000","last_became_peered":"2024-09-19T08:44:55.555465+0000","last_unstale":"2024-09-19T08:44:58.745283+0000","last_undegraded":"2024-09-19T08:44:58.745283+0000","last_fullsized":"2024-09-19T08:44:58.745283+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:56.488243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.591478+0000","last_change":"2024-09-19T08:44:56.635800+0000","last_active":"2024-09-19T08:44:57.591478+0000","last_peered":"2024-09-19T08:44:57.591478+0000","last_clean":"2024-09-19T08:44:57.591478+0000","last_became_active":"2024-09-19T08:44:56.635436+0000","last_became_peered":"2024-09-19T08:44:56.635436+0000","last_unstale":"2024-09-19T08:44:57.591478+0000","last_undegraded":"2024-09-19T08:44:57.591478+0000","last_fullsized":"2024-09-19T08:44:57.591478+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:07:01.879746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.591154+0000","last_change":"2024-09-19T08:44:55.569706+0000","last_active":"2024-09-19T08:44:57.591154+0000","last_peered":"2024-09-19T08:44:57.591154+0000","last_clean":"2024-09-19T08:44:57.591154+0000","last_became_active":"2024-09-19T08:44:55.569327+0000","last_became_peered":"2024-09-19T08:44:55.569327+0000","last_unstale":"2024-09-19T08:44:57.591154+0000","last_undegraded":"2024-09-19T08:44:57.591154+0000","last_fullsized":"2024-09-19T08:44:57.591154+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:54.250727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.748795+0000","last_change":"2024-09-19T08:44:56.632580+0000","last_active":"2024-09-19T08:44:58.748795+0000","last_peered":"2024-09-19T08:44:58.748795+0000","last_clean":"2024-09-19T08:44:58.748795+0000","last_became_active":"2024-09-19T08:44:56.628321+0000","last_became_peered":"2024-09-19T08:44:56.628321+0000","last_unstale":"2024-09-19T08:44:58.748795+0000","last_undegraded":"2024-09-19T08:44:58.748795+0000","last_fullsized":"2024-09-19T08:44:58.748795+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:21.031965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"117'8","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.750190+0000","last_change":"2024-09-19T08:44:55.581910+0000","last_active":"2024-09-19T08:44:58.750190+0000","last_peered":"2024-09-19T08:44:58.750190+0000","last_clean":"2024-09-19T08:44:58.750190+0000","last_became_active":"2024-09-19T08:44:55.580092+0000","last_became_peered":"2024-09-19T08:44:55.580092+0000","last_unstale":"2024-09-19T08:44:58.750190+0000","last_undegraded":"2024-09-19T08:44:58.750190+0000","last_fullsized":"2024-09-19T08:44:58.750190+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:20.098972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.749071+0000","last_change":"2024-09-19T08:44:56.626219+0000","last_active":"2024-09-19T08:44:58.749071+0000","last_peered":"2024-09-19T08:44:58.749071+0000","last_clean":"2024-09-19T08:44:58.749071+0000","last_became_active":"2024-09-19T08:44:56.625431+0000","last_became_peered":"2024-09-19T08:44:56.625431+0000","last_unstale":"2024-09-19T08:44:58.749071+0000","last_undegraded":"2024-09-19T08:44:58.749071+0000","last_fullsized":"2024-09-19T08:44:58.749071+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:33:51.641152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.748999+0000","last_change":"2024-09-19T08:44:55.574210+0000","last_active":"2024-09-19T08:44:58.748999+0000","last_peered":"2024-09-19T08:44:58.748999+0000","last_clean":"2024-09-19T08:44:58.748999+0000","last_became_active":"2024-09-19T08:44:55.573515+0000","last_became_peered":"2024-09-19T08:44:55.573515+0000","last_unstale":"2024-09-19T08:44:58.748999+0000","last_undegraded":"2024-09-19T08:44:58.748999+0000","last_fullsized":"2024-09-19T08:44:58.748999+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:46.606596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.748332+0000","last_change":"2024-09-19T08:44:56.622872+0000","last_active":"2024-09-19T08:44:58.748332+0000","last_peered":"2024-09-19T08:44:58.748332+0000","last_clean":"2024-09-19T08:44:58.748332+0000","last_became_active":"2024-09-19T08:44:56.622117+0000","last_became_peered":"2024-09-19T08:44:56.622117+0000","last_unstale":"2024-09-19T08:44:58.748332+0000","last_undegraded":"2024-09-19T08:44:58.748332+0000","last_fullsized":"2024-09-19T08:44:58.748332+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:50.432353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.425347+0000","last_change":"2024-09-19T08:44:55.579531+0000","last_active":"2024-09-19T08:44:58.425347+0000","last_peered":"2024-09-19T08:44:58.425347+0000","last_clean":"2024-09-19T08:44:58.425347+0000","last_became_active":"2024-09-19T08:44:55.579121+0000","last_became_peered":"2024-09-19T08:44:55.579121+0000","last_unstale":"2024-09-19T08:44:58.425347+0000","last_undegraded":"2024-09-19T08:44:58.425347+0000","last_fullsized":"2024-09-19T08:44:58.425347+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:58.553231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.746250+0000","last_change":"2024-09-19T08:44:56.635836+0000","last_active":"2024-09-19T08:44:58.746250+0000","last_peered":"2024-09-19T08:44:58.746250+0000","last_clean":"2024-09-19T08:44:58.746250+0000","last_became_active":"2024-09-19T08:44:56.635524+0000","last_became_peered":"2024-09-19T08:44:56.635524+0000","last_unstale":"2024-09-19T08:44:58.746250+0000","last_undegraded":"2024-09-19T08:44:58.746250+0000","last_fullsized":"2024-09-19T08:44:58.746250+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:10:14.650297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.598044+0000","last_change":"2024-09-19T08:44:55.567683+0000","last_active":"2024-09-19T08:44:57.598044+0000","last_peered":"2024-09-19T08:44:57.598044+0000","last_clean":"2024-09-19T08:44:57.598044+0000","last_became_active":"2024-09-19T08:44:55.566980+0000","last_became_peered":"2024-09-19T08:44:55.566980+0000","last_unstale":"2024-09-19T08:44:57.598044+0000","last_undegraded":"2024-09-19T08:44:57.598044+0000","last_fullsized":"2024-09-19T08:44:57.598044+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:54:32.799876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.591770+0000","last_change":"2024-09-19T08:44:56.624949+0000","last_active":"2024-09-19T08:44:57.591770+0000","last_peered":"2024-09-19T08:44:57.591770+0000","last_clean":"2024-09-19T08:44:57.591770+0000","last_became_active":"2024-09-19T08:44:56.624574+0000","last_became_peered":"2024-09-19T08:44:56.624574+0000","last_unstale":"2024-09-19T08:44:57.591770+0000","last_undegraded":"2024-09-19T08:44:57.591770+0000","last_fullsized":"2024-09-19T08:44:57.591770+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:35.167074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.748046+0000","last_change":"2024-09-19T08:44:55.569681+0000","last_active":"2024-09-19T08:44:58.748046+0000","last_peered":"2024-09-19T08:44:58.748046+0000","last_clean":"2024-09-19T08:44:58.748046+0000","last_became_active":"2024-09-19T08:44:55.569017+0000","last_became_peered":"2024-09-19T08:44:55.569017+0000","last_unstale":"2024-09-19T08:44:58.748046+0000","last_undegraded":"2024-09-19T08:44:58.748046+0000","last_fullsized":"2024-09-19T08:44:58.748046+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:10.935915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.745074+0000","last_change":"2024-09-19T08:44:56.639336+0000","last_active":"2024-09-19T08:44:58.745074+0000","last_peered":"2024-09-19T08:44:58.745074+0000","last_clean":"2024-09-19T08:44:58.745074+0000","last_became_active":"2024-09-19T08:44:56.639074+0000","last_became_peered":"2024-09-19T08:44:56.639074+0000","last_unstale":"2024-09-19T08:44:58.745074+0000","last_undegraded":"2024-09-19T08:44:58.745074+0000","last_fullsized":"2024-09-19T08:44:58.745074+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:45:25.083087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.747999+0000","last_change":"2024-09-19T08:44:55.573790+0000","last_active":"2024-09-19T08:44:58.747999+0000","last_peered":"2024-09-19T08:44:58.747999+0000","last_clean":"2024-09-19T08:44:58.747999+0000","last_became_active":"2024-09-19T08:44:55.572994+0000","last_became_peered":"2024-09-19T08:44:55.572994+0000","last_unstale":"2024-09-19T08:44:58.747999+0000","last_undegraded":"2024-09-19T08:44:58.747999+0000","last_fullsized":"2024-09-19T08:44:58.747999+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:05.905632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.597796+0000","last_change":"2024-09-19T08:44:56.623873+0000","last_active":"2024-09-19T08:44:57.597796+0000","last_peered":"2024-09-19T08:44:57.597796+0000","last_clean":"2024-09-19T08:44:57.597796+0000","last_became_active":"2024-09-19T08:44:56.622707+0000","last_became_peered":"2024-09-19T08:44:56.622707+0000","last_unstale":"2024-09-19T08:44:57.597796+0000","last_undegraded":"2024-09-19T08:44:57.597796+0000","last_fullsized":"2024-09-19T08:44:57.597796+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:11.808221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.746286+0000","last_change":"2024-09-19T08:44:55.576857+0000","last_active":"2024-09-19T08:44:58.746286+0000","last_peered":"2024-09-19T08:44:58.746286+0000","last_clean":"2024-09-19T08:44:58.746286+0000","last_became_active":"2024-09-19T08:44:55.576542+0000","last_became_peered":"2024-09-19T08:44:55.576542+0000","last_unstale":"2024-09-19T08:44:58.746286+0000","last_undegraded":"2024-09-19T08:44:58.746286+0000","last_fullsized":"2024-09-19T08:44:58.746286+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:39:56.548896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.590752+0000","last_change":"2024-09-19T08:44:56.611484+0000","last_active":"2024-09-19T08:44:57.590752+0000","last_peered":"2024-09-19T08:44:57.590752+0000","last_clean":"2024-09-19T08:44:57.590752+0000","last_became_active":"2024-09-19T08:44:56.610714+0000","last_became_peered":"2024-09-19T08:44:56.610714+0000","last_unstale":"2024-09-19T08:44:57.590752+0000","last_undegraded":"2024-09-19T08:44:57.590752+0000","last_fullsized":"2024-09-19T08:44:57.590752+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:08:51.447583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.745235+0000","last_change":"2024-09-19T08:44:55.560466+0000","last_active":"2024-09-19T08:44:58.745235+0000","last_peered":"2024-09-19T08:44:58.745235+0000","last_clean":"2024-09-19T08:44:58.745235+0000","last_became_active":"2024-09-19T08:44:55.560056+0000","last_became_peered":"2024-09-19T08:44:55.560056+0000","last_unstale":"2024-09-19T08:44:58.745235+0000","last_undegraded":"2024-09-19T08:44:58.745235+0000","last_fullsized":"2024-09-19T08:44:58.745235+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:35:09.726345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.633615+0000","last_change":"2024-09-19T08:44:56.639971+0000","last_active":"2024-09-19T08:44:57.633615+0000","last_peered":"2024-09-19T08:44:57.633615+0000","last_clean":"2024-09-19T08:44:57.633615+0000","last_became_active":"2024-09-19T08:44:56.639294+0000","last_became_peered":"2024-09-19T08:44:56.639294+0000","last_unstale":"2024-09-19T08:44:57.633615+0000","last_undegraded":"2024-09-19T08:44:57.633615+0000","last_fullsized":"2024-09-19T08:44:57.633615+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:03.260426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.424722+0000","last_change":"2024-09-19T08:44:55.581400+0000","last_active":"2024-09-19T08:44:58.424722+0000","last_peered":"2024-09-19T08:44:58.424722+0000","last_clean":"2024-09-19T08:44:58.424722+0000","last_became_active":"2024-09-19T08:44:55.581120+0000","last_became_peered":"2024-09-19T08:44:55.581120+0000","last_unstale":"2024-09-19T08:44:58.424722+0000","last_undegraded":"2024-09-19T08:44:58.424722+0000","last_fullsized":"2024-09-19T08:44:58.424722+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:09.932658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.598294+0000","last_change":"2024-09-19T08:44:56.620256+0000","last_active":"2024-09-19T08:44:57.598294+0000","last_peered":"2024-09-19T08:44:57.598294+0000","last_clean":"2024-09-19T08:44:57.598294+0000","last_became_active":"2024-09-19T08:44:56.619537+0000","last_became_peered":"2024-09-19T08:44:56.619537+0000","last_unstale":"2024-09-19T08:44:57.598294+0000","last_undegraded":"2024-09-19T08:44:57.598294+0000","last_fullsized":"2024-09-19T08:44:57.598294+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:50:06.707221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.746500+0000","last_change":"2024-09-19T08:44:55.570742+0000","last_active":"2024-09-19T08:44:58.746500+0000","last_peered":"2024-09-19T08:44:58.746500+0000","last_clean":"2024-09-19T08:44:58.746500+0000","last_became_active":"2024-09-19T08:44:55.570351+0000","last_became_peered":"2024-09-19T08:44:55.570351+0000","last_unstale":"2024-09-19T08:44:58.746500+0000","last_undegraded":"2024-09-19T08:44:58.746500+0000","last_fullsized":"2024-09-19T08:44:58.746500+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:28.489609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.746424+0000","last_change":"2024-09-19T08:44:56.628250+0000","last_active":"2024-09-19T08:44:58.746424+0000","last_peered":"2024-09-19T08:44:58.746424+0000","last_clean":"2024-09-19T08:44:58.746424+0000","last_became_active":"2024-09-19T08:44:56.627785+0000","last_became_peered":"2024-09-19T08:44:56.627785+0000","last_unstale":"2024-09-19T08:44:58.746424+0000","last_undegraded":"2024-09-19T08:44:58.746424+0000","last_fullsized":"2024-09-19T08:44:58.746424+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:38:21.133529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"118'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.636612+0000","last_change":"2024-09-19T08:44:55.582963+0000","last_active":"2024-09-19T08:44:57.636612+0000","last_peered":"2024-09-19T08:44:57.636612+0000","last_clean":"2024-09-19T08:44:57.636612+0000","last_became_active":"2024-09-19T08:44:55.582275+0000","last_became_peered":"2024-09-19T08:44:55.582275+0000","last_unstale":"2024-09-19T08:44:57.636612+0000","last_undegraded":"2024-09-19T08:44:57.636612+0000","last_fullsized":"2024-09-19T08:44:57.636612+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:36.920703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.591855+0000","last_change":"2024-09-19T08:44:56.618479+0000","last_active":"2024-09-19T08:44:57.591855+0000","last_peered":"2024-09-19T08:44:57.591855+0000","last_clean":"2024-09-19T08:44:57.591855+0000","last_became_active":"2024-09-19T08:44:56.617686+0000","last_became_peered":"2024-09-19T08:44:56.617686+0000","last_unstale":"2024-09-19T08:44:57.591855+0000","last_undegraded":"2024-09-19T08:44:57.591855+0000","last_fullsized":"2024-09-19T08:44:57.591855+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:27.565546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.591486+0000","last_change":"2024-09-19T08:44:55.579123+0000","last_active":"2024-09-19T08:44:57.591486+0000","last_peered":"2024-09-19T08:44:57.591486+0000","last_clean":"2024-09-19T08:44:57.591486+0000","last_became_active":"2024-09-19T08:44:55.578750+0000","last_became_peered":"2024-09-19T08:44:55.578750+0000","last_unstale":"2024-09-19T08:44:57.591486+0000","last_undegraded":"2024-09-19T08:44:57.591486+0000","last_fullsized":"2024-09-19T08:44:57.591486+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:54:23.386609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.597823+0000","last_change":"2024-09-19T08:44:56.625453+0000","last_active":"2024-09-19T08:44:57.597823+0000","last_peered":"2024-09-19T08:44:57.597823+0000","last_clean":"2024-09-19T08:44:57.597823+0000","last_became_active":"2024-09-19T08:44:56.624073+0000","last_became_peered":"2024-09-19T08:44:56.624073+0000","last_unstale":"2024-09-19T08:44:57.597823+0000","last_undegraded":"2024-09-19T08:44:57.597823+0000","last_fullsized":"2024-09-19T08:44:57.597823+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:03.260426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.424930+0000","last_change":"2024-09-19T08:44:55.557235+0000","last_active":"2024-09-19T08:44:58.424930+0000","last_peered":"2024-09-19T08:44:58.424930+0000","last_clean":"2024-09-19T08:44:58.424930+0000","last_became_active":"2024-09-19T08:44:55.556850+0000","last_became_peered":"2024-09-19T08:44:55.556850+0000","last_unstale":"2024-09-19T08:44:58.424930+0000","last_undegraded":"2024-09-19T08:44:58.424930+0000","last_fullsized":"2024-09-19T08:44:58.424930+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:30:23.671808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.633655+0000","last_change":"2024-09-19T08:44:56.641555+0000","last_active":"2024-09-19T08:44:57.633655+0000","last_peered":"2024-09-19T08:44:57.633655+0000","last_clean":"2024-09-19T08:44:57.633655+0000","last_became_active":"2024-09-19T08:44:56.640892+0000","last_became_peered":"2024-09-19T08:44:56.640892+0000","last_unstale":"2024-09-19T08:44:57.633655+0000","last_undegraded":"2024-09-19T08:44:57.633655+0000","last_fullsized":"2024-09-19T08:44:57.633655+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:50:06.707221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.591132+0000","last_change":"2024-09-19T08:44:55.572599+0000","last_active":"2024-09-19T08:44:57.591132+0000","last_peered":"2024-09-19T08:44:57.591132+0000","last_clean":"2024-09-19T08:44:57.591132+0000","last_became_active":"2024-09-19T08:44:55.572180+0000","last_became_peered":"2024-09-19T08:44:55.572180+0000","last_unstale":"2024-09-19T08:44:57.591132+0000","last_undegraded":"2024-09-19T08:44:57.591132+0000","last_fullsized":"2024-09-19T08:44:57.591132+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:21:29.592176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.748127+0000","last_change":"2024-09-19T08:44:56.625157+0000","last_active":"2024-09-19T08:44:58.748127+0000","last_peered":"2024-09-19T08:44:58.748127+0000","last_clean":"2024-09-19T08:44:58.748127+0000","last_became_active":"2024-09-19T08:44:56.624387+0000","last_became_peered":"2024-09-19T08:44:56.624387+0000","last_unstale":"2024-09-19T08:44:58.748127+0000","last_undegraded":"2024-09-19T08:44:58.748127+0000","last_fullsized":"2024-09-19T08:44:58.748127+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:08:51.447583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.633927+0000","last_change":"2024-09-19T08:44:55.584136+0000","last_active":"2024-09-19T08:44:57.633927+0000","last_peered":"2024-09-19T08:44:57.633927+0000","last_clean":"2024-09-19T08:44:57.633927+0000","last_became_active":"2024-09-19T08:44:55.583541+0000","last_became_peered":"2024-09-19T08:44:55.583541+0000","last_unstale":"2024-09-19T08:44:57.633927+0000","last_undegraded":"2024-09-19T08:44:57.633927+0000","last_fullsized":"2024-09-19T08:44:57.633927+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:54:32.799876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.745488+0000","last_change":"2024-09-19T08:44:56.651957+0000","last_active":"2024-09-19T08:44:58.745488+0000","last_peered":"2024-09-19T08:44:58.745488+0000","last_clean":"2024-09-19T08:44:58.745488+0000","last_became_active":"2024-09-19T08:44:56.651695+0000","last_became_peered":"2024-09-19T08:44:56.651695+0000","last_unstale":"2024-09-19T08:44:58.745488+0000","last_undegraded":"2024-09-19T08:44:58.745488+0000","last_fullsized":"2024-09-19T08:44:58.745488+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:14:36.251824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.745352+0000","last_change":"2024-09-19T08:44:55.579436+0000","last_active":"2024-09-19T08:44:58.745352+0000","last_peered":"2024-09-19T08:44:58.745352+0000","last_clean":"2024-09-19T08:44:58.745352+0000","last_became_active":"2024-09-19T08:44:55.579060+0000","last_became_peered":"2024-09-19T08:44:55.579060+0000","last_unstale":"2024-09-19T08:44:58.745352+0000","last_undegraded":"2024-09-19T08:44:58.745352+0000","last_fullsized":"2024-09-19T08:44:58.745352+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:48.989707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.590958+0000","last_change":"2024-09-19T08:44:56.624931+0000","last_active":"2024-09-19T08:44:57.590958+0000","last_peered":"2024-09-19T08:44:57.590958+0000","last_clean":"2024-09-19T08:44:57.590958+0000","last_became_active":"2024-09-19T08:44:56.622751+0000","last_became_peered":"2024-09-19T08:44:56.622751+0000","last_unstale":"2024-09-19T08:44:57.590958+0000","last_undegraded":"2024-09-19T08:44:57.590958+0000","last_fullsized":"2024-09-19T08:44:57.590958+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:30.332833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.425010+0000","last_change":"2024-09-19T08:44:55.560067+0000","last_active":"2024-09-19T08:44:58.425010+0000","last_peered":"2024-09-19T08:44:58.425010+0000","last_clean":"2024-09-19T08:44:58.425010+0000","last_became_active":"2024-09-19T08:44:55.559788+0000","last_became_peered":"2024-09-19T08:44:55.559788+0000","last_unstale":"2024-09-19T08:44:58.425010+0000","last_undegraded":"2024-09-19T08:44:58.425010+0000","last_fullsized":"2024-09-19T08:44:58.425010+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:28:28.146159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.633857+0000","last_change":"2024-09-19T08:44:56.631380+0000","last_active":"2024-09-19T08:44:57.633857+0000","last_peered":"2024-09-19T08:44:57.633857+0000","last_clean":"2024-09-19T08:44:57.633857+0000","last_became_active":"2024-09-19T08:44:56.630563+0000","last_became_peered":"2024-09-19T08:44:56.630563+0000","last_unstale":"2024-09-19T08:44:57.633857+0000","last_undegraded":"2024-09-19T08:44:57.633857+0000","last_fullsized":"2024-09-19T08:44:57.633857+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:02.548375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.633867+0000","last_change":"2024-09-19T08:44:55.582714+0000","last_active":"2024-09-19T08:44:57.633867+0000","last_peered":"2024-09-19T08:44:57.633867+0000","last_clean":"2024-09-19T08:44:57.633867+0000","last_became_active":"2024-09-19T08:44:55.582005+0000","last_became_peered":"2024-09-19T08:44:55.582005+0000","last_unstale":"2024-09-19T08:44:57.633867+0000","last_undegraded":"2024-09-19T08:44:57.633867+0000","last_fullsized":"2024-09-19T08:44:57.633867+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:37.224553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"71'128","reported_seq":384,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.632934+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:44:57.632934+0000","last_peered":"2024-09-19T08:44:57.632934+0000","last_clean":"2024-09-19T08:44:57.632934+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:44:57.632934+0000","last_undegraded":"2024-09-19T08:44:57.632934+0000","last_fullsized":"2024-09-19T08:44:57.632934+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509525,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33232,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338608,"statfs":{"total":96636764160,"available":96602734592,"internally_reserved":0,"allocated":2084864,"data_stored":1241162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574934,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1466368,"data_stored":643828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509527,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32616,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339224,"statfs":{"total":96636764160,"available":96603365376,"internally_reserved":0,"allocated":1462272,"data_stored":642328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509527,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32628,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1470464,"data_stored":648614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542229,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32616,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339224,"statfs":{"total":96636764160,"available":96603365376,"internally_reserved":0,"allocated":1462272,"data_stored":642376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509524,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32616,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339224,"statfs":{"total":96636764160,"available":96603365376,"internally_reserved":0,"allocated":1462272,"data_stored":642832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509526,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1466368,"data_stored":648544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509524,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33188,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338652,"statfs":{"total":96636764160,"available":96602779648,"internally_reserved":0,"allocated":2048000,"data_stored":1232584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:44:59.618 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:44:59.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.780+0000 7f6e1734e640 1 Processor -- start 2024-09-19T08:44:59.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.780+0000 7f6e1734e640 1 -- start start 2024-09-19T08:44:59.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.780+0000 7f6e1734e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6e100b6f60 0x7f6e1006d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:59.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.780+0000 7f6e1734e640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e10160350 con 0x7f6e10074c70 2024-09-19T08:44:59.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.780+0000 7f6e1734e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e10160630 con 0x7f6e100b0280 2024-09-19T08:44:59.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.780+0000 7f6e1734e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6e10160910 con 0x7f6e100b6f60 2024-09-19T08:44:59.781 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.780+0000 7f6e158eb640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f6e10074c70 0x7f6e100ae340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52576/0 (socket says 172.21.15.98:52576) 2024-09-19T08:44:59.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.780+0000 7f6e158eb640 1 -- 172.21.15.98:0/3344007850 learned_addr learned my addr 172.21.15.98:0/3344007850 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:44:59.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.780+0000 7f6e148e9640 1 --2- 172.21.15.98:0/3344007850 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6e100b6f60 0x7f6e1006d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:59.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.780+0000 7f6e07fff640 1 -- 172.21.15.98:0/3344007850 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2976880889 0 0) 0x7f6e10160350 con 0x7f6e10074c70 2024-09-19T08:44:59.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.780+0000 7f6e07fff640 1 -- 172.21.15.98:0/3344007850 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6dfc003530 con 0x7f6e10074c70 2024-09-19T08:44:59.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.780+0000 7f6e07fff640 1 -- 172.21.15.98:0/3344007850 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3806993502 0 0) 0x7f6e10160630 con 0x7f6e100b0280 2024-09-19T08:44:59.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.780+0000 7f6e07fff640 1 -- 172.21.15.98:0/3344007850 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e10160350 con 0x7f6e100b0280 2024-09-19T08:44:59.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.780+0000 7f6e07fff640 1 -- 172.21.15.98:0/3344007850 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3146955723 0 0) 0x7f6dfc003530 con 0x7f6e10074c70 2024-09-19T08:44:59.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.780+0000 7f6e07fff640 1 -- 172.21.15.98:0/3344007850 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6e100b6f60 msgr2=0x7f6e1006d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:59.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.780+0000 7f6e07fff640 1 --2- 172.21.15.98:0/3344007850 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6e100b6f60 0x7f6e1006d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:59.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.780+0000 7f6e07fff640 1 -- 172.21.15.98:0/3344007850 >> v1:172.21.15.139:6789/0 conn(0x7f6e100b0280 legacy=0x7f6e100b6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:59.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.780+0000 7f6e07fff640 1 -- 172.21.15.98:0/3344007850 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e10160bf0 con 0x7f6e10074c70 2024-09-19T08:44:59.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.780+0000 7f6e07fff640 1 -- 172.21.15.98:0/3344007850 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f6e08001a00 con 0x7f6e10074c70 2024-09-19T08:44:59.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.780+0000 7f6e07fff640 1 -- 172.21.15.98:0/3344007850 >> v1:172.21.15.98:6789/0 conn(0x7f6e10074c70 legacy=0x7f6e100ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:59.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.780+0000 7f6e07fff640 1 --2- 172.21.15.98:0/3344007850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dfc003b80 0x7f6dfc003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:59.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.780+0000 7f6e07fff640 1 --2- 172.21.15.98:0/3344007850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6dfc004810 0x7f6dfc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:59.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.780+0000 7f6e07fff640 1 --2- 172.21.15.98:0/3344007850 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6e100b6f60 0x7f6dfc0051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:59.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.780+0000 7f6e07fff640 1 -- 172.21.15.98:0/3344007850 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6e10160910 con 0x7f6dfc004810 2024-09-19T08:44:59.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.780+0000 7f6e07fff640 1 -- 172.21.15.98:0/3344007850 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc00bb10 con 0x7f6dfc003b80 2024-09-19T08:44:59.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.780+0000 7f6e07fff640 1 -- 172.21.15.98:0/3344007850 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6dfc00bd10 con 0x7f6e100b6f60 2024-09-19T08:44:59.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.780+0000 7f6e148e9640 1 --2- 172.21.15.98:0/3344007850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6dfc004810 0x7f6dfc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:59.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.780+0000 7f6e150ea640 1 --2- 172.21.15.98:0/3344007850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dfc003b80 0x7f6dfc003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:59.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.780+0000 7f6e148e9640 1 -- 172.21.15.98:0/3344007850 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6e100b6f60 msgr2=0x7f6dfc0051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:59.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.780+0000 7f6e148e9640 1 --2- 172.21.15.98:0/3344007850 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6e100b6f60 0x7f6dfc0051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:59.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.780+0000 7f6e148e9640 1 -- 172.21.15.98:0/3344007850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dfc003b80 msgr2=0x7f6dfc003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:44:59.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.780+0000 7f6e148e9640 1 --2- 172.21.15.98:0/3344007850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dfc003b80 0x7f6dfc003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:59.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.780+0000 7f6e148e9640 1 -- 172.21.15.98:0/3344007850 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dfc003530 con 0x7f6dfc004810 2024-09-19T08:44:59.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.784+0000 7f6e148e9640 1 --2- 172.21.15.98:0/3344007850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6dfc004810 0x7f6dfc004c90 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f6e00009ab0 tx=0x7f6e0001ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=524f492edd5f8f9d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:59.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.784+0000 7f6e07fff640 1 -- 172.21.15.98:0/3344007850 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e000096b0 con 0x7f6dfc004810 2024-09-19T08:44:59.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.784+0000 7f6e07fff640 1 -- 172.21.15.98:0/3344007850 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e000a6470 con 0x7f6dfc004810 2024-09-19T08:44:59.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.784+0000 7f6e07fff640 1 -- 172.21.15.98:0/3344007850 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e000a6860 con 0x7f6dfc004810 2024-09-19T08:44:59.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.784+0000 7f6e1734e640 1 -- 172.21.15.98:0/3344007850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6dfc004810 msgr2=0x7f6dfc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:59.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.784+0000 7f6e1734e640 1 --2- 172.21.15.98:0/3344007850 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6dfc004810 0x7f6dfc004c90 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f6e00009ab0 tx=0x7f6e0001ca90 comp rx=0 tx=0).stop 2024-09-19T08:44:59.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.784+0000 7f6e158eb640 1 -- 172.21.15.98:0/3344007850 reap_dead start 2024-09-19T08:44:59.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.784+0000 7f6e1734e640 1 -- 172.21.15.98:0/3344007850 shutdown_connections 2024-09-19T08:44:59.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.784+0000 7f6e1734e640 1 -- 172.21.15.98:0/3344007850 >> 172.21.15.98:0/3344007850 conn(0x7f6e10073170 msgr2=0x7f6e10073580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:59.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.784+0000 7f6e1734e640 1 -- 172.21.15.98:0/3344007850 shutdown_connections 2024-09-19T08:44:59.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.784+0000 7f6e1734e640 1 -- 172.21.15.98:0/3344007850 wait complete. 2024-09-19T08:44:59.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.784+0000 7f6e1734e640 1 Processor -- start 2024-09-19T08:44:59.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.784+0000 7f6e1734e640 1 -- start start 2024-09-19T08:44:59.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.784+0000 7f6e1734e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6dfc004810 0x7f6e101664e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:59.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.784+0000 7f6e1734e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e1015b4d0 0x7f6e1009a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:59.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.784+0000 7f6e1734e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e10162350 0x7f6e101627d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:59.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.784+0000 7f6e1734e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6e100af5f0 con 0x7f6e10162350 2024-09-19T08:44:59.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.784+0000 7f6e1734e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6e100af470 con 0x7f6e1015b4d0 2024-09-19T08:44:59.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.784+0000 7f6e1734e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6e10168380 con 0x7f6dfc004810 2024-09-19T08:44:59.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.784+0000 7f6e158eb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e10162350 0x7f6e101627d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:59.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.784+0000 7f6e158eb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e10162350 0x7f6e101627d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47838/0 (socket says 172.21.15.98:47838) 2024-09-19T08:44:59.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.784+0000 7f6e158eb640 1 -- 172.21.15.98:0/3458118260 learned_addr learned my addr 172.21.15.98:0/3458118260 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:44:59.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.784+0000 7f6e148e9640 1 --2- 172.21.15.98:0/3458118260 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e1015b4d0 0x7f6e1009a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:59.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.784+0000 7f6e150ea640 1 --2- 172.21.15.98:0/3458118260 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6dfc004810 0x7f6e101664e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:59.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.788+0000 7f6e158eb640 1 -- 172.21.15.98:0/3458118260 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6dfc004810 msgr2=0x7f6e101664e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:59.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.788+0000 7f6e158eb640 1 --2- 172.21.15.98:0/3458118260 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6dfc004810 0x7f6e101664e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:59.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.788+0000 7f6e158eb640 1 -- 172.21.15.98:0/3458118260 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e1015b4d0 msgr2=0x7f6e1009a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:59.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.788+0000 7f6e158eb640 1 --2- 172.21.15.98:0/3458118260 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e1015b4d0 0x7f6e1009a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:59.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.788+0000 7f6e158eb640 1 -- 172.21.15.98:0/3458118260 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e101618e0 con 0x7f6e10162350 2024-09-19T08:44:59.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.788+0000 7f6e158eb640 1 --2- 172.21.15.98:0/3458118260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e10162350 0x7f6e101627d0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc004790 tx=0x7f6e0800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:59.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.788+0000 7f6e05ffb640 1 -- 172.21.15.98:0/3458118260 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0800a640 con 0x7f6e10162350 2024-09-19T08:44:59.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.788+0000 7f6e05ffb640 1 -- 172.21.15.98:0/3458118260 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e0800a7e0 con 0x7f6e10162350 2024-09-19T08:44:59.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.788+0000 7f6e1734e640 1 -- 172.21.15.98:0/3458118260 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e1015a710 con 0x7f6e10162350 2024-09-19T08:44:59.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.788+0000 7f6e05ffb640 1 -- 172.21.15.98:0/3458118260 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0800abd0 con 0x7f6e10162350 2024-09-19T08:44:59.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.788+0000 7f6e1734e640 1 -- 172.21.15.98:0/3458118260 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e1015acd0 con 0x7f6e10162350 2024-09-19T08:44:59.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.788+0000 7f6e1734e640 1 -- 172.21.15.98:0/3458118260 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e10162d10 con 0x7f6e10162350 2024-09-19T08:44:59.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.788+0000 7f6e05ffb640 1 -- 172.21.15.98:0/3458118260 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f6e08007070 con 0x7f6e10162350 2024-09-19T08:44:59.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.788+0000 7f6e05ffb640 1 --2- 172.21.15.98:0/3458118260 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6df4067cb0 0x7f6df406a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:44:59.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.788+0000 7f6e05ffb640 1 -- 172.21.15.98:0/3458118260 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5737+0+0 (secure 0 0 0) 0x7f6e08085b30 con 0x7f6e10162350 2024-09-19T08:44:59.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.788+0000 7f6e150ea640 1 --2- 172.21.15.98:0/3458118260 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6df4067cb0 0x7f6df406a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:44:59.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.788+0000 7f6e150ea640 1 --2- 172.21.15.98:0/3458118260 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6df4067cb0 0x7f6df406a170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c007fd0 tx=0x7f6e0c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:44:59.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.792+0000 7f6e05ffb640 1 -- 172.21.15.98:0/3458118260 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6e08055a20 con 0x7f6e10162350 2024-09-19T08:44:59.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.952+0000 7f6e1734e640 1 -- 172.21.15.98:0/3458118260 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6e100ae340 con 0x7f6df4067cb0 2024-09-19T08:44:59.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.972+0000 7f6e05ffb640 1 -- 172.21.15.98:0/3458118260 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252484 (secure 0 0 0) 0x7f6e100ae340 con 0x7f6df4067cb0 2024-09-19T08:44:59.973 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:44:59.974 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:44:59.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.972+0000 7f6e1734e640 1 -- 172.21.15.98:0/3458118260 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6df4067cb0 msgr2=0x7f6df406a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:59.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.972+0000 7f6e1734e640 1 --2- 172.21.15.98:0/3458118260 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6df4067cb0 0x7f6df406a170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c007fd0 tx=0x7f6e0c009040 comp rx=0 tx=0).stop 2024-09-19T08:44:59.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.972+0000 7f6e1734e640 1 -- 172.21.15.98:0/3458118260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e10162350 msgr2=0x7f6e101627d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:44:59.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.972+0000 7f6e1734e640 1 --2- 172.21.15.98:0/3458118260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e10162350 0x7f6e101627d0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc004790 tx=0x7f6e0800cc00 comp rx=0 tx=0).stop 2024-09-19T08:44:59.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.972+0000 7f6e1734e640 1 -- 172.21.15.98:0/3458118260 shutdown_connections 2024-09-19T08:44:59.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.972+0000 7f6e1734e640 1 --2- 172.21.15.98:0/3458118260 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6df4067cb0 0x7f6df406a170 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:59.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.972+0000 7f6e1734e640 1 --2- 172.21.15.98:0/3458118260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e10162350 0x7f6e101627d0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:59.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.972+0000 7f6e1734e640 1 --2- 172.21.15.98:0/3458118260 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e1015b4d0 0x7f6e1009a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:59.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.972+0000 7f6e1734e640 1 --2- 172.21.15.98:0/3458118260 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6dfc004810 0x7f6e101664e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:44:59.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.972+0000 7f6e1734e640 1 -- 172.21.15.98:0/3458118260 >> 172.21.15.98:0/3458118260 conn(0x7f6e10073170 msgr2=0x7f6e101611d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:44:59.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.972+0000 7f6e1734e640 1 -- 172.21.15.98:0/3458118260 shutdown_connections 2024-09-19T08:44:59.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:44:59.972+0000 7f6e1734e640 1 -- 172.21.15.98:0/3458118260 wait complete. 2024-09-19T08:44:59.992 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2024-09-19T08:44:58.911525+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":113,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262144,"kb_used_data":12620,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712576,"statfs":{"total":773094113280,"available":772825677824,"internally_reserved":0,"allocated":12922880,"data_stored":6342268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.375121"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.746428+0000","last_change":"2024-09-19T08:44:56.639227+0000","last_active":"2024-09-19T08:44:58.746428+0000","last_peered":"2024-09-19T08:44:58.746428+0000","last_clean":"2024-09-19T08:44:58.746428+0000","last_became_active":"2024-09-19T08:44:56.638909+0000","last_became_peered":"2024-09-19T08:44:56.638909+0000","last_unstale":"2024-09-19T08:44:58.746428+0000","last_undegraded":"2024-09-19T08:44:58.746428+0000","last_fullsized":"2024-09-19T08:44:58.746428+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:40:49.962545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.591373+0000","last_change":"2024-09-19T08:44:56.604280+0000","last_active":"2024-09-19T08:44:57.591373+0000","last_peered":"2024-09-19T08:44:57.591373+0000","last_clean":"2024-09-19T08:44:57.591373+0000","last_became_active":"2024-09-19T08:44:56.603523+0000","last_became_peered":"2024-09-19T08:44:56.603523+0000","last_unstale":"2024-09-19T08:44:57.591373+0000","last_undegraded":"2024-09-19T08:44:57.591373+0000","last_fullsized":"2024-09-19T08:44:57.591373+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:03:16.391474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.749090+0000","last_change":"2024-09-19T08:44:56.619936+0000","last_active":"2024-09-19T08:44:58.749090+0000","last_peered":"2024-09-19T08:44:58.749090+0000","last_clean":"2024-09-19T08:44:58.749090+0000","last_became_active":"2024-09-19T08:44:56.619196+0000","last_became_peered":"2024-09-19T08:44:56.619196+0000","last_unstale":"2024-09-19T08:44:58.749090+0000","last_undegraded":"2024-09-19T08:44:58.749090+0000","last_fullsized":"2024-09-19T08:44:58.749090+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:07:27.887227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.745823+0000","last_change":"2024-09-19T08:44:56.623732+0000","last_active":"2024-09-19T08:44:58.745823+0000","last_peered":"2024-09-19T08:44:58.745823+0000","last_clean":"2024-09-19T08:44:58.745823+0000","last_became_active":"2024-09-19T08:44:56.622354+0000","last_became_peered":"2024-09-19T08:44:56.622354+0000","last_unstale":"2024-09-19T08:44:58.745823+0000","last_undegraded":"2024-09-19T08:44:58.745823+0000","last_fullsized":"2024-09-19T08:44:58.745823+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:32:32.719720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.633670+0000","last_change":"2024-09-19T08:44:56.634714+0000","last_active":"2024-09-19T08:44:57.633670+0000","last_peered":"2024-09-19T08:44:57.633670+0000","last_clean":"2024-09-19T08:44:57.633670+0000","last_became_active":"2024-09-19T08:44:56.633493+0000","last_became_peered":"2024-09-19T08:44:56.633493+0000","last_unstale":"2024-09-19T08:44:57.633670+0000","last_undegraded":"2024-09-19T08:44:57.633670+0000","last_fullsized":"2024-09-19T08:44:57.633670+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:53.603787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.597941+0000","last_change":"2024-09-19T08:44:56.624900+0000","last_active":"2024-09-19T08:44:57.597941+0000","last_peered":"2024-09-19T08:44:57.597941+0000","last_clean":"2024-09-19T08:44:57.597941+0000","last_became_active":"2024-09-19T08:44:56.623944+0000","last_became_peered":"2024-09-19T08:44:56.623944+0000","last_unstale":"2024-09-19T08:44:57.597941+0000","last_undegraded":"2024-09-19T08:44:57.597941+0000","last_fullsized":"2024-09-19T08:44:57.597941+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:16.566499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.597283+0000","last_change":"2024-09-19T08:44:56.617780+0000","last_active":"2024-09-19T08:44:57.597283+0000","last_peered":"2024-09-19T08:44:57.597283+0000","last_clean":"2024-09-19T08:44:57.597283+0000","last_became_active":"2024-09-19T08:44:56.617050+0000","last_became_peered":"2024-09-19T08:44:56.617050+0000","last_unstale":"2024-09-19T08:44:57.597283+0000","last_undegraded":"2024-09-19T08:44:57.597283+0000","last_fullsized":"2024-09-19T08:44:57.597283+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:29:42.509418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.749705+0000","last_change":"2024-09-19T08:44:56.653073+0000","last_active":"2024-09-19T08:44:58.749705+0000","last_peered":"2024-09-19T08:44:58.749705+0000","last_clean":"2024-09-19T08:44:58.749705+0000","last_became_active":"2024-09-19T08:44:56.651513+0000","last_became_peered":"2024-09-19T08:44:56.651513+0000","last_unstale":"2024-09-19T08:44:58.749705+0000","last_undegraded":"2024-09-19T08:44:58.749705+0000","last_fullsized":"2024-09-19T08:44:58.749705+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:35:10.729538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.749635+0000","last_change":"2024-09-19T08:44:56.628070+0000","last_active":"2024-09-19T08:44:58.749635+0000","last_peered":"2024-09-19T08:44:58.749635+0000","last_clean":"2024-09-19T08:44:58.749635+0000","last_became_active":"2024-09-19T08:44:56.627394+0000","last_became_peered":"2024-09-19T08:44:56.627394+0000","last_unstale":"2024-09-19T08:44:58.749635+0000","last_undegraded":"2024-09-19T08:44:58.749635+0000","last_fullsized":"2024-09-19T08:44:58.749635+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:53:40.416631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.591184+0000","last_change":"2024-09-19T08:44:56.604361+0000","last_active":"2024-09-19T08:44:57.591184+0000","last_peered":"2024-09-19T08:44:57.591184+0000","last_clean":"2024-09-19T08:44:57.591184+0000","last_became_active":"2024-09-19T08:44:56.603636+0000","last_became_peered":"2024-09-19T08:44:56.603636+0000","last_unstale":"2024-09-19T08:44:57.591184+0000","last_undegraded":"2024-09-19T08:44:57.591184+0000","last_fullsized":"2024-09-19T08:44:57.591184+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:59.215648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.425331+0000","last_change":"2024-09-19T08:44:56.621177+0000","last_active":"2024-09-19T08:44:58.425331+0000","last_peered":"2024-09-19T08:44:58.425331+0000","last_clean":"2024-09-19T08:44:58.425331+0000","last_became_active":"2024-09-19T08:44:56.620516+0000","last_became_peered":"2024-09-19T08:44:56.620516+0000","last_unstale":"2024-09-19T08:44:58.425331+0000","last_undegraded":"2024-09-19T08:44:58.425331+0000","last_fullsized":"2024-09-19T08:44:58.425331+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:09:46.759640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.748691+0000","last_change":"2024-09-19T08:44:56.627982+0000","last_active":"2024-09-19T08:44:58.748691+0000","last_peered":"2024-09-19T08:44:58.748691+0000","last_clean":"2024-09-19T08:44:58.748691+0000","last_became_active":"2024-09-19T08:44:56.627281+0000","last_became_peered":"2024-09-19T08:44:56.627281+0000","last_unstale":"2024-09-19T08:44:58.748691+0000","last_undegraded":"2024-09-19T08:44:58.748691+0000","last_fullsized":"2024-09-19T08:44:58.748691+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:24:53.239123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.745594+0000","last_change":"2024-09-19T08:44:56.640239+0000","last_active":"2024-09-19T08:44:58.745594+0000","last_peered":"2024-09-19T08:44:58.745594+0000","last_clean":"2024-09-19T08:44:58.745594+0000","last_became_active":"2024-09-19T08:44:56.639949+0000","last_became_peered":"2024-09-19T08:44:56.639949+0000","last_unstale":"2024-09-19T08:44:58.745594+0000","last_undegraded":"2024-09-19T08:44:58.745594+0000","last_fullsized":"2024-09-19T08:44:58.745594+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:40:03.414124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.591096+0000","last_change":"2024-09-19T08:44:56.628181+0000","last_active":"2024-09-19T08:44:57.591096+0000","last_peered":"2024-09-19T08:44:57.591096+0000","last_clean":"2024-09-19T08:44:57.591096+0000","last_became_active":"2024-09-19T08:44:56.627824+0000","last_became_peered":"2024-09-19T08:44:56.627824+0000","last_unstale":"2024-09-19T08:44:57.591096+0000","last_undegraded":"2024-09-19T08:44:57.591096+0000","last_fullsized":"2024-09-19T08:44:57.591096+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:33:41.515270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.744671+0000","last_change":"2024-09-19T08:44:56.627419+0000","last_active":"2024-09-19T08:44:58.744671+0000","last_peered":"2024-09-19T08:44:58.744671+0000","last_clean":"2024-09-19T08:44:58.744671+0000","last_became_active":"2024-09-19T08:44:56.624251+0000","last_became_peered":"2024-09-19T08:44:56.624251+0000","last_unstale":"2024-09-19T08:44:58.744671+0000","last_undegraded":"2024-09-19T08:44:58.744671+0000","last_fullsized":"2024-09-19T08:44:58.744671+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:55:50.503091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.745335+0000","last_change":"2024-09-19T08:44:56.631755+0000","last_active":"2024-09-19T08:44:58.745335+0000","last_peered":"2024-09-19T08:44:58.745335+0000","last_clean":"2024-09-19T08:44:58.745335+0000","last_became_active":"2024-09-19T08:44:56.631432+0000","last_became_peered":"2024-09-19T08:44:56.631432+0000","last_unstale":"2024-09-19T08:44:58.745335+0000","last_undegraded":"2024-09-19T08:44:58.745335+0000","last_fullsized":"2024-09-19T08:44:58.745335+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:02:15.630802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.598359+0000","last_change":"2024-09-19T08:44:56.613209+0000","last_active":"2024-09-19T08:44:57.598359+0000","last_peered":"2024-09-19T08:44:57.598359+0000","last_clean":"2024-09-19T08:44:57.598359+0000","last_became_active":"2024-09-19T08:44:56.612439+0000","last_became_peered":"2024-09-19T08:44:56.612439+0000","last_unstale":"2024-09-19T08:44:57.598359+0000","last_undegraded":"2024-09-19T08:44:57.598359+0000","last_fullsized":"2024-09-19T08:44:57.598359+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:19:53.603787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.745655+0000","last_change":"2024-09-19T08:44:56.641215+0000","last_active":"2024-09-19T08:44:58.745655+0000","last_peered":"2024-09-19T08:44:58.745655+0000","last_clean":"2024-09-19T08:44:58.745655+0000","last_became_active":"2024-09-19T08:44:56.640784+0000","last_became_peered":"2024-09-19T08:44:56.640784+0000","last_unstale":"2024-09-19T08:44:58.745655+0000","last_undegraded":"2024-09-19T08:44:58.745655+0000","last_fullsized":"2024-09-19T08:44:58.745655+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:08:18.033098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.425378+0000","last_change":"2024-09-19T08:44:56.640519+0000","last_active":"2024-09-19T08:44:58.425378+0000","last_peered":"2024-09-19T08:44:58.425378+0000","last_clean":"2024-09-19T08:44:58.425378+0000","last_became_active":"2024-09-19T08:44:56.640216+0000","last_became_peered":"2024-09-19T08:44:56.640216+0000","last_unstale":"2024-09-19T08:44:58.425378+0000","last_undegraded":"2024-09-19T08:44:58.425378+0000","last_fullsized":"2024-09-19T08:44:58.425378+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:31:13.838216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.746180+0000","last_change":"2024-09-19T08:44:56.622948+0000","last_active":"2024-09-19T08:44:58.746180+0000","last_peered":"2024-09-19T08:44:58.746180+0000","last_clean":"2024-09-19T08:44:58.746180+0000","last_became_active":"2024-09-19T08:44:56.621105+0000","last_became_peered":"2024-09-19T08:44:56.621105+0000","last_unstale":"2024-09-19T08:44:58.746180+0000","last_undegraded":"2024-09-19T08:44:58.746180+0000","last_fullsized":"2024-09-19T08:44:58.746180+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:43.766467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.634105+0000","last_change":"2024-09-19T08:44:56.634172+0000","last_active":"2024-09-19T08:44:57.634105+0000","last_peered":"2024-09-19T08:44:57.634105+0000","last_clean":"2024-09-19T08:44:57.634105+0000","last_became_active":"2024-09-19T08:44:56.632589+0000","last_became_peered":"2024-09-19T08:44:56.632589+0000","last_unstale":"2024-09-19T08:44:57.634105+0000","last_undegraded":"2024-09-19T08:44:57.634105+0000","last_fullsized":"2024-09-19T08:44:57.634105+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:38:14.461908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.424890+0000","last_change":"2024-09-19T08:44:56.620891+0000","last_active":"2024-09-19T08:44:58.424890+0000","last_peered":"2024-09-19T08:44:58.424890+0000","last_clean":"2024-09-19T08:44:58.424890+0000","last_became_active":"2024-09-19T08:44:56.620215+0000","last_became_peered":"2024-09-19T08:44:56.620215+0000","last_unstale":"2024-09-19T08:44:58.424890+0000","last_undegraded":"2024-09-19T08:44:58.424890+0000","last_fullsized":"2024-09-19T08:44:58.424890+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:39.099740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.749000+0000","last_change":"2024-09-19T08:44:56.629006+0000","last_active":"2024-09-19T08:44:58.749000+0000","last_peered":"2024-09-19T08:44:58.749000+0000","last_clean":"2024-09-19T08:44:58.749000+0000","last_became_active":"2024-09-19T08:44:56.627353+0000","last_became_peered":"2024-09-19T08:44:56.627353+0000","last_unstale":"2024-09-19T08:44:58.749000+0000","last_undegraded":"2024-09-19T08:44:58.749000+0000","last_fullsized":"2024-09-19T08:44:58.749000+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:24:16.145118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.424637+0000","last_change":"2024-09-19T08:44:56.630278+0000","last_active":"2024-09-19T08:44:58.424637+0000","last_peered":"2024-09-19T08:44:58.424637+0000","last_clean":"2024-09-19T08:44:58.424637+0000","last_became_active":"2024-09-19T08:44:56.629876+0000","last_became_peered":"2024-09-19T08:44:56.629876+0000","last_unstale":"2024-09-19T08:44:58.424637+0000","last_undegraded":"2024-09-19T08:44:58.424637+0000","last_fullsized":"2024-09-19T08:44:58.424637+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:50:06.823214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.591016+0000","last_change":"2024-09-19T08:44:56.618618+0000","last_active":"2024-09-19T08:44:57.591016+0000","last_peered":"2024-09-19T08:44:57.591016+0000","last_clean":"2024-09-19T08:44:57.591016+0000","last_became_active":"2024-09-19T08:44:56.617802+0000","last_became_peered":"2024-09-19T08:44:56.617802+0000","last_unstale":"2024-09-19T08:44:57.591016+0000","last_undegraded":"2024-09-19T08:44:57.591016+0000","last_fullsized":"2024-09-19T08:44:57.591016+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:04:45.876735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.745798+0000","last_change":"2024-09-19T08:44:56.637309+0000","last_active":"2024-09-19T08:44:58.745798+0000","last_peered":"2024-09-19T08:44:58.745798+0000","last_clean":"2024-09-19T08:44:58.745798+0000","last_became_active":"2024-09-19T08:44:56.637076+0000","last_became_peered":"2024-09-19T08:44:56.637076+0000","last_unstale":"2024-09-19T08:44:58.745798+0000","last_undegraded":"2024-09-19T08:44:58.745798+0000","last_fullsized":"2024-09-19T08:44:58.745798+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:27:13.514549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.744883+0000","last_change":"2024-09-19T08:44:56.639195+0000","last_active":"2024-09-19T08:44:58.744883+0000","last_peered":"2024-09-19T08:44:58.744883+0000","last_clean":"2024-09-19T08:44:58.744883+0000","last_became_active":"2024-09-19T08:44:56.638886+0000","last_became_peered":"2024-09-19T08:44:56.638886+0000","last_unstale":"2024-09-19T08:44:58.744883+0000","last_undegraded":"2024-09-19T08:44:58.744883+0000","last_fullsized":"2024-09-19T08:44:58.744883+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:16.743988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.748269+0000","last_change":"2024-09-19T08:44:56.634911+0000","last_active":"2024-09-19T08:44:58.748269+0000","last_peered":"2024-09-19T08:44:58.748269+0000","last_clean":"2024-09-19T08:44:58.748269+0000","last_became_active":"2024-09-19T08:44:56.634224+0000","last_became_peered":"2024-09-19T08:44:56.634224+0000","last_unstale":"2024-09-19T08:44:58.748269+0000","last_undegraded":"2024-09-19T08:44:58.748269+0000","last_fullsized":"2024-09-19T08:44:58.748269+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:30.332833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.745485+0000","last_change":"2024-09-19T08:44:56.628614+0000","last_active":"2024-09-19T08:44:58.745485+0000","last_peered":"2024-09-19T08:44:58.745485+0000","last_clean":"2024-09-19T08:44:58.745485+0000","last_became_active":"2024-09-19T08:44:56.628336+0000","last_became_peered":"2024-09-19T08:44:56.628336+0000","last_unstale":"2024-09-19T08:44:58.745485+0000","last_undegraded":"2024-09-19T08:44:58.745485+0000","last_fullsized":"2024-09-19T08:44:58.745485+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:32:13.227944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.744908+0000","last_change":"2024-09-19T08:44:56.630967+0000","last_active":"2024-09-19T08:44:58.744908+0000","last_peered":"2024-09-19T08:44:58.744908+0000","last_clean":"2024-09-19T08:44:58.744908+0000","last_became_active":"2024-09-19T08:44:56.630172+0000","last_became_peered":"2024-09-19T08:44:56.630172+0000","last_unstale":"2024-09-19T08:44:58.744908+0000","last_undegraded":"2024-09-19T08:44:58.744908+0000","last_fullsized":"2024-09-19T08:44:58.744908+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:53.189321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.633623+0000","last_change":"2024-09-19T08:44:56.634769+0000","last_active":"2024-09-19T08:44:57.633623+0000","last_peered":"2024-09-19T08:44:57.633623+0000","last_clean":"2024-09-19T08:44:57.633623+0000","last_became_active":"2024-09-19T08:44:56.634013+0000","last_became_peered":"2024-09-19T08:44:56.634013+0000","last_unstale":"2024-09-19T08:44:57.633623+0000","last_undegraded":"2024-09-19T08:44:57.633623+0000","last_fullsized":"2024-09-19T08:44:57.633623+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:16.566499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.597299+0000","last_change":"2024-09-19T08:44:56.627395+0000","last_active":"2024-09-19T08:44:57.597299+0000","last_peered":"2024-09-19T08:44:57.597299+0000","last_clean":"2024-09-19T08:44:57.597299+0000","last_became_active":"2024-09-19T08:44:56.626662+0000","last_became_peered":"2024-09-19T08:44:56.626662+0000","last_unstale":"2024-09-19T08:44:57.597299+0000","last_undegraded":"2024-09-19T08:44:57.597299+0000","last_fullsized":"2024-09-19T08:44:57.597299+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:36:07.346236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.745956+0000","last_change":"2024-09-19T08:44:56.633036+0000","last_active":"2024-09-19T08:44:58.745956+0000","last_peered":"2024-09-19T08:44:58.745956+0000","last_clean":"2024-09-19T08:44:58.745956+0000","last_became_active":"2024-09-19T08:44:56.632697+0000","last_became_peered":"2024-09-19T08:44:56.632697+0000","last_unstale":"2024-09-19T08:44:58.745956+0000","last_undegraded":"2024-09-19T08:44:58.745956+0000","last_fullsized":"2024-09-19T08:44:58.745956+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:03.011362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"117'4","reported_seq":18,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.634379+0000","last_change":"2024-09-19T08:44:55.582796+0000","last_active":"2024-09-19T08:44:57.634379+0000","last_peered":"2024-09-19T08:44:57.634379+0000","last_clean":"2024-09-19T08:44:57.634379+0000","last_became_active":"2024-09-19T08:44:55.582055+0000","last_became_peered":"2024-09-19T08:44:55.582055+0000","last_unstale":"2024-09-19T08:44:57.634379+0000","last_undegraded":"2024-09-19T08:44:57.634379+0000","last_fullsized":"2024-09-19T08:44:57.634379+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:01.341975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.749790+0000","last_change":"2024-09-19T08:44:56.626375+0000","last_active":"2024-09-19T08:44:58.749790+0000","last_peered":"2024-09-19T08:44:58.749790+0000","last_clean":"2024-09-19T08:44:58.749790+0000","last_became_active":"2024-09-19T08:44:56.625571+0000","last_became_peered":"2024-09-19T08:44:56.625571+0000","last_unstale":"2024-09-19T08:44:58.749790+0000","last_undegraded":"2024-09-19T08:44:58.749790+0000","last_fullsized":"2024-09-19T08:44:58.749790+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:49.992899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.748317+0000","last_change":"2024-09-19T08:44:55.566152+0000","last_active":"2024-09-19T08:44:58.748317+0000","last_peered":"2024-09-19T08:44:58.748317+0000","last_clean":"2024-09-19T08:44:58.748317+0000","last_became_active":"2024-09-19T08:44:55.565226+0000","last_became_peered":"2024-09-19T08:44:55.565226+0000","last_unstale":"2024-09-19T08:44:58.748317+0000","last_undegraded":"2024-09-19T08:44:58.748317+0000","last_fullsized":"2024-09-19T08:44:58.748317+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:07:45.509629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.633976+0000","last_change":"2024-09-19T08:44:56.642587+0000","last_active":"2024-09-19T08:44:57.633976+0000","last_peered":"2024-09-19T08:44:57.633976+0000","last_clean":"2024-09-19T08:44:57.633976+0000","last_became_active":"2024-09-19T08:44:56.640966+0000","last_became_peered":"2024-09-19T08:44:56.640966+0000","last_unstale":"2024-09-19T08:44:57.633976+0000","last_undegraded":"2024-09-19T08:44:57.633976+0000","last_fullsized":"2024-09-19T08:44:57.633976+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:22:10.389495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.425607+0000","last_change":"2024-09-19T08:44:55.555645+0000","last_active":"2024-09-19T08:44:58.425607+0000","last_peered":"2024-09-19T08:44:58.425607+0000","last_clean":"2024-09-19T08:44:58.425607+0000","last_became_active":"2024-09-19T08:44:55.555244+0000","last_became_peered":"2024-09-19T08:44:55.555244+0000","last_unstale":"2024-09-19T08:44:58.425607+0000","last_undegraded":"2024-09-19T08:44:58.425607+0000","last_fullsized":"2024-09-19T08:44:58.425607+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:15:10.647852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.634323+0000","last_change":"2024-09-19T08:44:56.620256+0000","last_active":"2024-09-19T08:44:57.634323+0000","last_peered":"2024-09-19T08:44:57.634323+0000","last_clean":"2024-09-19T08:44:57.634323+0000","last_became_active":"2024-09-19T08:44:56.618552+0000","last_became_peered":"2024-09-19T08:44:56.618552+0000","last_unstale":"2024-09-19T08:44:57.634323+0000","last_undegraded":"2024-09-19T08:44:57.634323+0000","last_fullsized":"2024-09-19T08:44:57.634323+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:46.494060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.591722+0000","last_change":"2024-09-19T08:44:55.556416+0000","last_active":"2024-09-19T08:44:57.591722+0000","last_peered":"2024-09-19T08:44:57.591722+0000","last_clean":"2024-09-19T08:44:57.591722+0000","last_became_active":"2024-09-19T08:44:55.556039+0000","last_became_peered":"2024-09-19T08:44:55.556039+0000","last_unstale":"2024-09-19T08:44:57.591722+0000","last_undegraded":"2024-09-19T08:44:57.591722+0000","last_fullsized":"2024-09-19T08:44:57.591722+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:24.239361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.745744+0000","last_change":"2024-09-19T08:44:56.640990+0000","last_active":"2024-09-19T08:44:58.745744+0000","last_peered":"2024-09-19T08:44:58.745744+0000","last_clean":"2024-09-19T08:44:58.745744+0000","last_became_active":"2024-09-19T08:44:56.640744+0000","last_became_peered":"2024-09-19T08:44:56.640744+0000","last_unstale":"2024-09-19T08:44:58.745744+0000","last_undegraded":"2024-09-19T08:44:58.745744+0000","last_fullsized":"2024-09-19T08:44:58.745744+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:45:18.197696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.591646+0000","last_change":"2024-09-19T08:44:55.557528+0000","last_active":"2024-09-19T08:44:57.591646+0000","last_peered":"2024-09-19T08:44:57.591646+0000","last_clean":"2024-09-19T08:44:57.591646+0000","last_became_active":"2024-09-19T08:44:55.557209+0000","last_became_peered":"2024-09-19T08:44:55.557209+0000","last_unstale":"2024-09-19T08:44:57.591646+0000","last_undegraded":"2024-09-19T08:44:57.591646+0000","last_fullsized":"2024-09-19T08:44:57.591646+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:47:47.959474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.597468+0000","last_change":"2024-09-19T08:44:56.611611+0000","last_active":"2024-09-19T08:44:57.597468+0000","last_peered":"2024-09-19T08:44:57.597468+0000","last_clean":"2024-09-19T08:44:57.597468+0000","last_became_active":"2024-09-19T08:44:56.610910+0000","last_became_peered":"2024-09-19T08:44:56.610910+0000","last_unstale":"2024-09-19T08:44:57.597468+0000","last_undegraded":"2024-09-19T08:44:57.597468+0000","last_fullsized":"2024-09-19T08:44:57.597468+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:46:41.767059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.746029+0000","last_change":"2024-09-19T08:44:55.555018+0000","last_active":"2024-09-19T08:44:58.746029+0000","last_peered":"2024-09-19T08:44:58.746029+0000","last_clean":"2024-09-19T08:44:58.746029+0000","last_became_active":"2024-09-19T08:44:55.554750+0000","last_became_peered":"2024-09-19T08:44:55.554750+0000","last_unstale":"2024-09-19T08:44:58.746029+0000","last_undegraded":"2024-09-19T08:44:58.746029+0000","last_fullsized":"2024-09-19T08:44:58.746029+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:34.048242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.749904+0000","last_change":"2024-09-19T08:44:56.639019+0000","last_active":"2024-09-19T08:44:58.749904+0000","last_peered":"2024-09-19T08:44:58.749904+0000","last_clean":"2024-09-19T08:44:58.749904+0000","last_became_active":"2024-09-19T08:44:56.638039+0000","last_became_peered":"2024-09-19T08:44:56.638039+0000","last_unstale":"2024-09-19T08:44:58.749904+0000","last_undegraded":"2024-09-19T08:44:58.749904+0000","last_fullsized":"2024-09-19T08:44:58.749904+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:08:06.100292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.597565+0000","last_change":"2024-09-19T08:44:55.568111+0000","last_active":"2024-09-19T08:44:57.597565+0000","last_peered":"2024-09-19T08:44:57.597565+0000","last_clean":"2024-09-19T08:44:57.597565+0000","last_became_active":"2024-09-19T08:44:55.567310+0000","last_became_peered":"2024-09-19T08:44:55.567310+0000","last_unstale":"2024-09-19T08:44:57.597565+0000","last_undegraded":"2024-09-19T08:44:57.597565+0000","last_fullsized":"2024-09-19T08:44:57.597565+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:37.224553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.590980+0000","last_change":"2024-09-19T08:44:56.609418+0000","last_active":"2024-09-19T08:44:57.590980+0000","last_peered":"2024-09-19T08:44:57.590980+0000","last_clean":"2024-09-19T08:44:57.590980+0000","last_became_active":"2024-09-19T08:44:56.608605+0000","last_became_peered":"2024-09-19T08:44:56.608605+0000","last_unstale":"2024-09-19T08:44:57.590980+0000","last_undegraded":"2024-09-19T08:44:57.590980+0000","last_fullsized":"2024-09-19T08:44:57.590980+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:50.432353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.590863+0000","last_change":"2024-09-19T08:44:55.581814+0000","last_active":"2024-09-19T08:44:57.590863+0000","last_peered":"2024-09-19T08:44:57.590863+0000","last_clean":"2024-09-19T08:44:57.590863+0000","last_became_active":"2024-09-19T08:44:55.581468+0000","last_became_peered":"2024-09-19T08:44:55.581468+0000","last_unstale":"2024-09-19T08:44:57.590863+0000","last_undegraded":"2024-09-19T08:44:57.590863+0000","last_fullsized":"2024-09-19T08:44:57.590863+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:50:18.779182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.745740+0000","last_change":"2024-09-19T08:44:56.615106+0000","last_active":"2024-09-19T08:44:58.745740+0000","last_peered":"2024-09-19T08:44:58.745740+0000","last_clean":"2024-09-19T08:44:58.745740+0000","last_became_active":"2024-09-19T08:44:56.614365+0000","last_became_peered":"2024-09-19T08:44:56.614365+0000","last_unstale":"2024-09-19T08:44:58.745740+0000","last_undegraded":"2024-09-19T08:44:58.745740+0000","last_fullsized":"2024-09-19T08:44:58.745740+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:58:42.773564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.748749+0000","last_change":"2024-09-19T08:44:55.574633+0000","last_active":"2024-09-19T08:44:58.748749+0000","last_peered":"2024-09-19T08:44:58.748749+0000","last_clean":"2024-09-19T08:44:58.748749+0000","last_became_active":"2024-09-19T08:44:55.573955+0000","last_became_peered":"2024-09-19T08:44:55.573955+0000","last_unstale":"2024-09-19T08:44:58.748749+0000","last_undegraded":"2024-09-19T08:44:58.748749+0000","last_fullsized":"2024-09-19T08:44:58.748749+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:11:24.299035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.748705+0000","last_change":"2024-09-19T08:44:56.631805+0000","last_active":"2024-09-19T08:44:58.748705+0000","last_peered":"2024-09-19T08:44:58.748705+0000","last_clean":"2024-09-19T08:44:58.748705+0000","last_became_active":"2024-09-19T08:44:56.630780+0000","last_became_peered":"2024-09-19T08:44:56.630780+0000","last_unstale":"2024-09-19T08:44:58.748705+0000","last_undegraded":"2024-09-19T08:44:58.748705+0000","last_fullsized":"2024-09-19T08:44:58.748705+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:04:45.876735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.746090+0000","last_change":"2024-09-19T08:44:55.574426+0000","last_active":"2024-09-19T08:44:58.746090+0000","last_peered":"2024-09-19T08:44:58.746090+0000","last_clean":"2024-09-19T08:44:58.746090+0000","last_became_active":"2024-09-19T08:44:55.574094+0000","last_became_peered":"2024-09-19T08:44:55.574094+0000","last_unstale":"2024-09-19T08:44:58.746090+0000","last_undegraded":"2024-09-19T08:44:58.746090+0000","last_fullsized":"2024-09-19T08:44:58.746090+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:08:05.097099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.634185+0000","last_change":"2024-09-19T08:44:56.631032+0000","last_active":"2024-09-19T08:44:57.634185+0000","last_peered":"2024-09-19T08:44:57.634185+0000","last_clean":"2024-09-19T08:44:57.634185+0000","last_became_active":"2024-09-19T08:44:56.630305+0000","last_became_peered":"2024-09-19T08:44:56.630305+0000","last_unstale":"2024-09-19T08:44:57.634185+0000","last_undegraded":"2024-09-19T08:44:57.634185+0000","last_fullsized":"2024-09-19T08:44:57.634185+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:48:08.894727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"117'3","reported_seq":17,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.748356+0000","last_change":"2024-09-19T08:44:55.572275+0000","last_active":"2024-09-19T08:44:58.748356+0000","last_peered":"2024-09-19T08:44:58.748356+0000","last_clean":"2024-09-19T08:44:58.748356+0000","last_became_active":"2024-09-19T08:44:55.571553+0000","last_became_peered":"2024-09-19T08:44:55.571553+0000","last_unstale":"2024-09-19T08:44:58.748356+0000","last_undegraded":"2024-09-19T08:44:58.748356+0000","last_fullsized":"2024-09-19T08:44:58.748356+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:41:18.647623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.634327+0000","last_change":"2024-09-19T08:44:56.620881+0000","last_active":"2024-09-19T08:44:57.634327+0000","last_peered":"2024-09-19T08:44:57.634327+0000","last_clean":"2024-09-19T08:44:57.634327+0000","last_became_active":"2024-09-19T08:44:56.619561+0000","last_became_peered":"2024-09-19T08:44:56.619561+0000","last_unstale":"2024-09-19T08:44:57.634327+0000","last_undegraded":"2024-09-19T08:44:57.634327+0000","last_fullsized":"2024-09-19T08:44:57.634327+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:05:50.023175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"117'6","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.748718+0000","last_change":"2024-09-19T08:44:55.584512+0000","last_active":"2024-09-19T08:44:58.748718+0000","last_peered":"2024-09-19T08:44:58.748718+0000","last_clean":"2024-09-19T08:44:58.748718+0000","last_became_active":"2024-09-19T08:44:55.582785+0000","last_became_peered":"2024-09-19T08:44:55.582785+0000","last_unstale":"2024-09-19T08:44:58.748718+0000","last_undegraded":"2024-09-19T08:44:58.748718+0000","last_fullsized":"2024-09-19T08:44:58.748718+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:57:29.207492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.745668+0000","last_change":"2024-09-19T08:44:56.630623+0000","last_active":"2024-09-19T08:44:58.745668+0000","last_peered":"2024-09-19T08:44:58.745668+0000","last_clean":"2024-09-19T08:44:58.745668+0000","last_became_active":"2024-09-19T08:44:56.630267+0000","last_became_peered":"2024-09-19T08:44:56.630267+0000","last_unstale":"2024-09-19T08:44:58.745668+0000","last_undegraded":"2024-09-19T08:44:58.745668+0000","last_fullsized":"2024-09-19T08:44:58.745668+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:43.907032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.745283+0000","last_change":"2024-09-19T08:44:55.556466+0000","last_active":"2024-09-19T08:44:58.745283+0000","last_peered":"2024-09-19T08:44:58.745283+0000","last_clean":"2024-09-19T08:44:58.745283+0000","last_became_active":"2024-09-19T08:44:55.555465+0000","last_became_peered":"2024-09-19T08:44:55.555465+0000","last_unstale":"2024-09-19T08:44:58.745283+0000","last_undegraded":"2024-09-19T08:44:58.745283+0000","last_fullsized":"2024-09-19T08:44:58.745283+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:56.488243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.591478+0000","last_change":"2024-09-19T08:44:56.635800+0000","last_active":"2024-09-19T08:44:57.591478+0000","last_peered":"2024-09-19T08:44:57.591478+0000","last_clean":"2024-09-19T08:44:57.591478+0000","last_became_active":"2024-09-19T08:44:56.635436+0000","last_became_peered":"2024-09-19T08:44:56.635436+0000","last_unstale":"2024-09-19T08:44:57.591478+0000","last_undegraded":"2024-09-19T08:44:57.591478+0000","last_fullsized":"2024-09-19T08:44:57.591478+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:07:01.879746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.591154+0000","last_change":"2024-09-19T08:44:55.569706+0000","last_active":"2024-09-19T08:44:57.591154+0000","last_peered":"2024-09-19T08:44:57.591154+0000","last_clean":"2024-09-19T08:44:57.591154+0000","last_became_active":"2024-09-19T08:44:55.569327+0000","last_became_peered":"2024-09-19T08:44:55.569327+0000","last_unstale":"2024-09-19T08:44:57.591154+0000","last_undegraded":"2024-09-19T08:44:57.591154+0000","last_fullsized":"2024-09-19T08:44:57.591154+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:18:54.250727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.748795+0000","last_change":"2024-09-19T08:44:56.632580+0000","last_active":"2024-09-19T08:44:58.748795+0000","last_peered":"2024-09-19T08:44:58.748795+0000","last_clean":"2024-09-19T08:44:58.748795+0000","last_became_active":"2024-09-19T08:44:56.628321+0000","last_became_peered":"2024-09-19T08:44:56.628321+0000","last_unstale":"2024-09-19T08:44:58.748795+0000","last_undegraded":"2024-09-19T08:44:58.748795+0000","last_fullsized":"2024-09-19T08:44:58.748795+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:21.031965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"117'8","reported_seq":22,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.750190+0000","last_change":"2024-09-19T08:44:55.581910+0000","last_active":"2024-09-19T08:44:58.750190+0000","last_peered":"2024-09-19T08:44:58.750190+0000","last_clean":"2024-09-19T08:44:58.750190+0000","last_became_active":"2024-09-19T08:44:55.580092+0000","last_became_peered":"2024-09-19T08:44:55.580092+0000","last_unstale":"2024-09-19T08:44:58.750190+0000","last_undegraded":"2024-09-19T08:44:58.750190+0000","last_fullsized":"2024-09-19T08:44:58.750190+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:20.098972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.749071+0000","last_change":"2024-09-19T08:44:56.626219+0000","last_active":"2024-09-19T08:44:58.749071+0000","last_peered":"2024-09-19T08:44:58.749071+0000","last_clean":"2024-09-19T08:44:58.749071+0000","last_became_active":"2024-09-19T08:44:56.625431+0000","last_became_peered":"2024-09-19T08:44:56.625431+0000","last_unstale":"2024-09-19T08:44:58.749071+0000","last_undegraded":"2024-09-19T08:44:58.749071+0000","last_fullsized":"2024-09-19T08:44:58.749071+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:33:51.641152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.748999+0000","last_change":"2024-09-19T08:44:55.574210+0000","last_active":"2024-09-19T08:44:58.748999+0000","last_peered":"2024-09-19T08:44:58.748999+0000","last_clean":"2024-09-19T08:44:58.748999+0000","last_became_active":"2024-09-19T08:44:55.573515+0000","last_became_peered":"2024-09-19T08:44:55.573515+0000","last_unstale":"2024-09-19T08:44:58.748999+0000","last_undegraded":"2024-09-19T08:44:58.748999+0000","last_fullsized":"2024-09-19T08:44:58.748999+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:46.606596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.748332+0000","last_change":"2024-09-19T08:44:56.622872+0000","last_active":"2024-09-19T08:44:58.748332+0000","last_peered":"2024-09-19T08:44:58.748332+0000","last_clean":"2024-09-19T08:44:58.748332+0000","last_became_active":"2024-09-19T08:44:56.622117+0000","last_became_peered":"2024-09-19T08:44:56.622117+0000","last_unstale":"2024-09-19T08:44:58.748332+0000","last_undegraded":"2024-09-19T08:44:58.748332+0000","last_fullsized":"2024-09-19T08:44:58.748332+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:21:50.432353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.425347+0000","last_change":"2024-09-19T08:44:55.579531+0000","last_active":"2024-09-19T08:44:58.425347+0000","last_peered":"2024-09-19T08:44:58.425347+0000","last_clean":"2024-09-19T08:44:58.425347+0000","last_became_active":"2024-09-19T08:44:55.579121+0000","last_became_peered":"2024-09-19T08:44:55.579121+0000","last_unstale":"2024-09-19T08:44:58.425347+0000","last_undegraded":"2024-09-19T08:44:58.425347+0000","last_fullsized":"2024-09-19T08:44:58.425347+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:58.553231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.746250+0000","last_change":"2024-09-19T08:44:56.635836+0000","last_active":"2024-09-19T08:44:58.746250+0000","last_peered":"2024-09-19T08:44:58.746250+0000","last_clean":"2024-09-19T08:44:58.746250+0000","last_became_active":"2024-09-19T08:44:56.635524+0000","last_became_peered":"2024-09-19T08:44:56.635524+0000","last_unstale":"2024-09-19T08:44:58.746250+0000","last_undegraded":"2024-09-19T08:44:58.746250+0000","last_fullsized":"2024-09-19T08:44:58.746250+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:10:14.650297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"117'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.598044+0000","last_change":"2024-09-19T08:44:55.567683+0000","last_active":"2024-09-19T08:44:57.598044+0000","last_peered":"2024-09-19T08:44:57.598044+0000","last_clean":"2024-09-19T08:44:57.598044+0000","last_became_active":"2024-09-19T08:44:55.566980+0000","last_became_peered":"2024-09-19T08:44:55.566980+0000","last_unstale":"2024-09-19T08:44:57.598044+0000","last_undegraded":"2024-09-19T08:44:57.598044+0000","last_fullsized":"2024-09-19T08:44:57.598044+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:54:32.799876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.591770+0000","last_change":"2024-09-19T08:44:56.624949+0000","last_active":"2024-09-19T08:44:57.591770+0000","last_peered":"2024-09-19T08:44:57.591770+0000","last_clean":"2024-09-19T08:44:57.591770+0000","last_became_active":"2024-09-19T08:44:56.624574+0000","last_became_peered":"2024-09-19T08:44:56.624574+0000","last_unstale":"2024-09-19T08:44:57.591770+0000","last_undegraded":"2024-09-19T08:44:57.591770+0000","last_fullsized":"2024-09-19T08:44:57.591770+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:35.167074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.748046+0000","last_change":"2024-09-19T08:44:55.569681+0000","last_active":"2024-09-19T08:44:58.748046+0000","last_peered":"2024-09-19T08:44:58.748046+0000","last_clean":"2024-09-19T08:44:58.748046+0000","last_became_active":"2024-09-19T08:44:55.569017+0000","last_became_peered":"2024-09-19T08:44:55.569017+0000","last_unstale":"2024-09-19T08:44:58.748046+0000","last_undegraded":"2024-09-19T08:44:58.748046+0000","last_fullsized":"2024-09-19T08:44:58.748046+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:10.935915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.745074+0000","last_change":"2024-09-19T08:44:56.639336+0000","last_active":"2024-09-19T08:44:58.745074+0000","last_peered":"2024-09-19T08:44:58.745074+0000","last_clean":"2024-09-19T08:44:58.745074+0000","last_became_active":"2024-09-19T08:44:56.639074+0000","last_became_peered":"2024-09-19T08:44:56.639074+0000","last_unstale":"2024-09-19T08:44:58.745074+0000","last_undegraded":"2024-09-19T08:44:58.745074+0000","last_fullsized":"2024-09-19T08:44:58.745074+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:45:25.083087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.747999+0000","last_change":"2024-09-19T08:44:55.573790+0000","last_active":"2024-09-19T08:44:58.747999+0000","last_peered":"2024-09-19T08:44:58.747999+0000","last_clean":"2024-09-19T08:44:58.747999+0000","last_became_active":"2024-09-19T08:44:55.572994+0000","last_became_peered":"2024-09-19T08:44:55.572994+0000","last_unstale":"2024-09-19T08:44:58.747999+0000","last_undegraded":"2024-09-19T08:44:58.747999+0000","last_fullsized":"2024-09-19T08:44:58.747999+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:05.905632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.597796+0000","last_change":"2024-09-19T08:44:56.623873+0000","last_active":"2024-09-19T08:44:57.597796+0000","last_peered":"2024-09-19T08:44:57.597796+0000","last_clean":"2024-09-19T08:44:57.597796+0000","last_became_active":"2024-09-19T08:44:56.622707+0000","last_became_peered":"2024-09-19T08:44:56.622707+0000","last_unstale":"2024-09-19T08:44:57.597796+0000","last_undegraded":"2024-09-19T08:44:57.597796+0000","last_fullsized":"2024-09-19T08:44:57.597796+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:11.808221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.746286+0000","last_change":"2024-09-19T08:44:55.576857+0000","last_active":"2024-09-19T08:44:58.746286+0000","last_peered":"2024-09-19T08:44:58.746286+0000","last_clean":"2024-09-19T08:44:58.746286+0000","last_became_active":"2024-09-19T08:44:55.576542+0000","last_became_peered":"2024-09-19T08:44:55.576542+0000","last_unstale":"2024-09-19T08:44:58.746286+0000","last_undegraded":"2024-09-19T08:44:58.746286+0000","last_fullsized":"2024-09-19T08:44:58.746286+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:39:56.548896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.590752+0000","last_change":"2024-09-19T08:44:56.611484+0000","last_active":"2024-09-19T08:44:57.590752+0000","last_peered":"2024-09-19T08:44:57.590752+0000","last_clean":"2024-09-19T08:44:57.590752+0000","last_became_active":"2024-09-19T08:44:56.610714+0000","last_became_peered":"2024-09-19T08:44:56.610714+0000","last_unstale":"2024-09-19T08:44:57.590752+0000","last_undegraded":"2024-09-19T08:44:57.590752+0000","last_fullsized":"2024-09-19T08:44:57.590752+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:08:51.447583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.745235+0000","last_change":"2024-09-19T08:44:55.560466+0000","last_active":"2024-09-19T08:44:58.745235+0000","last_peered":"2024-09-19T08:44:58.745235+0000","last_clean":"2024-09-19T08:44:58.745235+0000","last_became_active":"2024-09-19T08:44:55.560056+0000","last_became_peered":"2024-09-19T08:44:55.560056+0000","last_unstale":"2024-09-19T08:44:58.745235+0000","last_undegraded":"2024-09-19T08:44:58.745235+0000","last_fullsized":"2024-09-19T08:44:58.745235+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:35:09.726345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.633615+0000","last_change":"2024-09-19T08:44:56.639971+0000","last_active":"2024-09-19T08:44:57.633615+0000","last_peered":"2024-09-19T08:44:57.633615+0000","last_clean":"2024-09-19T08:44:57.633615+0000","last_became_active":"2024-09-19T08:44:56.639294+0000","last_became_peered":"2024-09-19T08:44:56.639294+0000","last_unstale":"2024-09-19T08:44:57.633615+0000","last_undegraded":"2024-09-19T08:44:57.633615+0000","last_fullsized":"2024-09-19T08:44:57.633615+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:03.260426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.424722+0000","last_change":"2024-09-19T08:44:55.581400+0000","last_active":"2024-09-19T08:44:58.424722+0000","last_peered":"2024-09-19T08:44:58.424722+0000","last_clean":"2024-09-19T08:44:58.424722+0000","last_became_active":"2024-09-19T08:44:55.581120+0000","last_became_peered":"2024-09-19T08:44:55.581120+0000","last_unstale":"2024-09-19T08:44:58.424722+0000","last_undegraded":"2024-09-19T08:44:58.424722+0000","last_fullsized":"2024-09-19T08:44:58.424722+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:09.932658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.598294+0000","last_change":"2024-09-19T08:44:56.620256+0000","last_active":"2024-09-19T08:44:57.598294+0000","last_peered":"2024-09-19T08:44:57.598294+0000","last_clean":"2024-09-19T08:44:57.598294+0000","last_became_active":"2024-09-19T08:44:56.619537+0000","last_became_peered":"2024-09-19T08:44:56.619537+0000","last_unstale":"2024-09-19T08:44:57.598294+0000","last_undegraded":"2024-09-19T08:44:57.598294+0000","last_fullsized":"2024-09-19T08:44:57.598294+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:50:06.707221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.746500+0000","last_change":"2024-09-19T08:44:55.570742+0000","last_active":"2024-09-19T08:44:58.746500+0000","last_peered":"2024-09-19T08:44:58.746500+0000","last_clean":"2024-09-19T08:44:58.746500+0000","last_became_active":"2024-09-19T08:44:55.570351+0000","last_became_peered":"2024-09-19T08:44:55.570351+0000","last_unstale":"2024-09-19T08:44:58.746500+0000","last_undegraded":"2024-09-19T08:44:58.746500+0000","last_fullsized":"2024-09-19T08:44:58.746500+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:28.489609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.746424+0000","last_change":"2024-09-19T08:44:56.628250+0000","last_active":"2024-09-19T08:44:58.746424+0000","last_peered":"2024-09-19T08:44:58.746424+0000","last_clean":"2024-09-19T08:44:58.746424+0000","last_became_active":"2024-09-19T08:44:56.627785+0000","last_became_peered":"2024-09-19T08:44:56.627785+0000","last_unstale":"2024-09-19T08:44:58.746424+0000","last_undegraded":"2024-09-19T08:44:58.746424+0000","last_fullsized":"2024-09-19T08:44:58.746424+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:38:21.133529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"118'1","reported_seq":15,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.636612+0000","last_change":"2024-09-19T08:44:55.582963+0000","last_active":"2024-09-19T08:44:57.636612+0000","last_peered":"2024-09-19T08:44:57.636612+0000","last_clean":"2024-09-19T08:44:57.636612+0000","last_became_active":"2024-09-19T08:44:55.582275+0000","last_became_peered":"2024-09-19T08:44:55.582275+0000","last_unstale":"2024-09-19T08:44:57.636612+0000","last_undegraded":"2024-09-19T08:44:57.636612+0000","last_fullsized":"2024-09-19T08:44:57.636612+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:36:36.920703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.591855+0000","last_change":"2024-09-19T08:44:56.618479+0000","last_active":"2024-09-19T08:44:57.591855+0000","last_peered":"2024-09-19T08:44:57.591855+0000","last_clean":"2024-09-19T08:44:57.591855+0000","last_became_active":"2024-09-19T08:44:56.617686+0000","last_became_peered":"2024-09-19T08:44:56.617686+0000","last_unstale":"2024-09-19T08:44:57.591855+0000","last_undegraded":"2024-09-19T08:44:57.591855+0000","last_fullsized":"2024-09-19T08:44:57.591855+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:27.565546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.591486+0000","last_change":"2024-09-19T08:44:55.579123+0000","last_active":"2024-09-19T08:44:57.591486+0000","last_peered":"2024-09-19T08:44:57.591486+0000","last_clean":"2024-09-19T08:44:57.591486+0000","last_became_active":"2024-09-19T08:44:55.578750+0000","last_became_peered":"2024-09-19T08:44:55.578750+0000","last_unstale":"2024-09-19T08:44:57.591486+0000","last_undegraded":"2024-09-19T08:44:57.591486+0000","last_fullsized":"2024-09-19T08:44:57.591486+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:54:23.386609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.597823+0000","last_change":"2024-09-19T08:44:56.625453+0000","last_active":"2024-09-19T08:44:57.597823+0000","last_peered":"2024-09-19T08:44:57.597823+0000","last_clean":"2024-09-19T08:44:57.597823+0000","last_became_active":"2024-09-19T08:44:56.624073+0000","last_became_peered":"2024-09-19T08:44:56.624073+0000","last_unstale":"2024-09-19T08:44:57.597823+0000","last_undegraded":"2024-09-19T08:44:57.597823+0000","last_fullsized":"2024-09-19T08:44:57.597823+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:03.260426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.424930+0000","last_change":"2024-09-19T08:44:55.557235+0000","last_active":"2024-09-19T08:44:58.424930+0000","last_peered":"2024-09-19T08:44:58.424930+0000","last_clean":"2024-09-19T08:44:58.424930+0000","last_became_active":"2024-09-19T08:44:55.556850+0000","last_became_peered":"2024-09-19T08:44:55.556850+0000","last_unstale":"2024-09-19T08:44:58.424930+0000","last_undegraded":"2024-09-19T08:44:58.424930+0000","last_fullsized":"2024-09-19T08:44:58.424930+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:30:23.671808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.633655+0000","last_change":"2024-09-19T08:44:56.641555+0000","last_active":"2024-09-19T08:44:57.633655+0000","last_peered":"2024-09-19T08:44:57.633655+0000","last_clean":"2024-09-19T08:44:57.633655+0000","last_became_active":"2024-09-19T08:44:56.640892+0000","last_became_peered":"2024-09-19T08:44:56.640892+0000","last_unstale":"2024-09-19T08:44:57.633655+0000","last_undegraded":"2024-09-19T08:44:57.633655+0000","last_fullsized":"2024-09-19T08:44:57.633655+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:50:06.707221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.591132+0000","last_change":"2024-09-19T08:44:55.572599+0000","last_active":"2024-09-19T08:44:57.591132+0000","last_peered":"2024-09-19T08:44:57.591132+0000","last_clean":"2024-09-19T08:44:57.591132+0000","last_became_active":"2024-09-19T08:44:55.572180+0000","last_became_peered":"2024-09-19T08:44:55.572180+0000","last_unstale":"2024-09-19T08:44:57.591132+0000","last_undegraded":"2024-09-19T08:44:57.591132+0000","last_fullsized":"2024-09-19T08:44:57.591132+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:21:29.592176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.748127+0000","last_change":"2024-09-19T08:44:56.625157+0000","last_active":"2024-09-19T08:44:58.748127+0000","last_peered":"2024-09-19T08:44:58.748127+0000","last_clean":"2024-09-19T08:44:58.748127+0000","last_became_active":"2024-09-19T08:44:56.624387+0000","last_became_peered":"2024-09-19T08:44:56.624387+0000","last_unstale":"2024-09-19T08:44:58.748127+0000","last_undegraded":"2024-09-19T08:44:58.748127+0000","last_fullsized":"2024-09-19T08:44:58.748127+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:08:51.447583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.633927+0000","last_change":"2024-09-19T08:44:55.584136+0000","last_active":"2024-09-19T08:44:57.633927+0000","last_peered":"2024-09-19T08:44:57.633927+0000","last_clean":"2024-09-19T08:44:57.633927+0000","last_became_active":"2024-09-19T08:44:55.583541+0000","last_became_peered":"2024-09-19T08:44:55.583541+0000","last_unstale":"2024-09-19T08:44:57.633927+0000","last_undegraded":"2024-09-19T08:44:57.633927+0000","last_fullsized":"2024-09-19T08:44:57.633927+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:54:32.799876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.745488+0000","last_change":"2024-09-19T08:44:56.651957+0000","last_active":"2024-09-19T08:44:58.745488+0000","last_peered":"2024-09-19T08:44:58.745488+0000","last_clean":"2024-09-19T08:44:58.745488+0000","last_became_active":"2024-09-19T08:44:56.651695+0000","last_became_peered":"2024-09-19T08:44:56.651695+0000","last_unstale":"2024-09-19T08:44:58.745488+0000","last_undegraded":"2024-09-19T08:44:58.745488+0000","last_fullsized":"2024-09-19T08:44:58.745488+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:14:36.251824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.745352+0000","last_change":"2024-09-19T08:44:55.579436+0000","last_active":"2024-09-19T08:44:58.745352+0000","last_peered":"2024-09-19T08:44:58.745352+0000","last_clean":"2024-09-19T08:44:58.745352+0000","last_became_active":"2024-09-19T08:44:55.579060+0000","last_became_peered":"2024-09-19T08:44:55.579060+0000","last_unstale":"2024-09-19T08:44:58.745352+0000","last_undegraded":"2024-09-19T08:44:58.745352+0000","last_fullsized":"2024-09-19T08:44:58.745352+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:48.989707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.590958+0000","last_change":"2024-09-19T08:44:56.624931+0000","last_active":"2024-09-19T08:44:57.590958+0000","last_peered":"2024-09-19T08:44:57.590958+0000","last_clean":"2024-09-19T08:44:57.590958+0000","last_became_active":"2024-09-19T08:44:56.622751+0000","last_became_peered":"2024-09-19T08:44:56.622751+0000","last_unstale":"2024-09-19T08:44:57.590958+0000","last_undegraded":"2024-09-19T08:44:57.590958+0000","last_fullsized":"2024-09-19T08:44:57.590958+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:30.332833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:58.425010+0000","last_change":"2024-09-19T08:44:55.560067+0000","last_active":"2024-09-19T08:44:58.425010+0000","last_peered":"2024-09-19T08:44:58.425010+0000","last_clean":"2024-09-19T08:44:58.425010+0000","last_became_active":"2024-09-19T08:44:55.559788+0000","last_became_peered":"2024-09-19T08:44:55.559788+0000","last_unstale":"2024-09-19T08:44:58.425010+0000","last_undegraded":"2024-09-19T08:44:58.425010+0000","last_fullsized":"2024-09-19T08:44:58.425010+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:28:28.146159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.633857+0000","last_change":"2024-09-19T08:44:56.631380+0000","last_active":"2024-09-19T08:44:57.633857+0000","last_peered":"2024-09-19T08:44:57.633857+0000","last_clean":"2024-09-19T08:44:57.633857+0000","last_became_active":"2024-09-19T08:44:56.630563+0000","last_became_peered":"2024-09-19T08:44:56.630563+0000","last_unstale":"2024-09-19T08:44:57.633857+0000","last_undegraded":"2024-09-19T08:44:57.633857+0000","last_fullsized":"2024-09-19T08:44:57.633857+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:55.533373+0000","last_clean_scrub_stamp":"2024-09-19T08:44:55.533373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:26:02.548375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"117'2","reported_seq":16,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.633867+0000","last_change":"2024-09-19T08:44:55.582714+0000","last_active":"2024-09-19T08:44:57.633867+0000","last_peered":"2024-09-19T08:44:57.633867+0000","last_clean":"2024-09-19T08:44:57.633867+0000","last_became_active":"2024-09-19T08:44:55.582005+0000","last_became_peered":"2024-09-19T08:44:55.582005+0000","last_unstale":"2024-09-19T08:44:57.633867+0000","last_undegraded":"2024-09-19T08:44:57.633867+0000","last_fullsized":"2024-09-19T08:44:57.633867+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:44:54.530181+0000","last_clean_scrub_stamp":"2024-09-19T08:44:54.530181+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:18:37.224553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"71'128","reported_seq":384,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-19T08:44:57.632934+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:44:57.632934+0000","last_peered":"2024-09-19T08:44:57.632934+0000","last_clean":"2024-09-19T08:44:57.632934+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:44:57.632934+0000","last_undegraded":"2024-09-19T08:44:57.632934+0000","last_fullsized":"2024-09-19T08:44:57.632934+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509525,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33232,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338608,"statfs":{"total":96636764160,"available":96602734592,"internally_reserved":0,"allocated":2084864,"data_stored":1241162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574934,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1466368,"data_stored":643828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509527,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32616,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339224,"statfs":{"total":96636764160,"available":96603365376,"internally_reserved":0,"allocated":1462272,"data_stored":642328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509527,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32628,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339212,"statfs":{"total":96636764160,"available":96603353088,"internally_reserved":0,"allocated":1470464,"data_stored":648614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542229,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32616,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339224,"statfs":{"total":96636764160,"available":96603365376,"internally_reserved":0,"allocated":1462272,"data_stored":642376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509524,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32616,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339224,"statfs":{"total":96636764160,"available":96603365376,"internally_reserved":0,"allocated":1462272,"data_stored":642832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509526,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1466368,"data_stored":648544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509524,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33188,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338652,"statfs":{"total":96636764160,"available":96602779648,"internally_reserved":0,"allocated":2048000,"data_stored":1232584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:44:59.995 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T08:44:59.996 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-19T08:45:00.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.155+0000 7f367bc60640 1 Processor -- start 2024-09-19T08:45:00.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.155+0000 7f367bc60640 1 -- start start 2024-09-19T08:45:00.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.155+0000 7f367bc60640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f36740729f0 0x7f367406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:00.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.155+0000 7f367bc60640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f367406c1d0 0x7f36740cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:00.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.155+0000 7f367bc60640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3674072ea0 con 0x7f3674144d10 2024-09-19T08:45:00.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.155+0000 7f367bc60640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f367406c620 con 0x7f367406c1d0 2024-09-19T08:45:00.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.155+0000 7f367bc60640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f3674073ab0 con 0x7f36740729f0 2024-09-19T08:45:00.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.155+0000 7f36799fc640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f3674144d10 0x7f36740722e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52590/0 (socket says 172.21.15.98:52590) 2024-09-19T08:45:00.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.155+0000 7f36799fc640 1 -- 172.21.15.98:0/1152613439 learned_addr learned my addr 172.21.15.98:0/1152613439 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:00.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.155+0000 7f367a1fd640 1 --2- 172.21.15.98:0/1152613439 >> v2:172.21.15.98:3300/0 conn(0x7f367406c1d0 0x7f36740cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:00.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.155+0000 7f367a1fd640 1 -- 172.21.15.98:0/1152613439 >> v2:172.21.15.139:3300/0 conn(0x7f36740729f0 msgr2=0x7f367406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:00.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.155+0000 7f367a1fd640 1 --2- 172.21.15.98:0/1152613439 >> v2:172.21.15.139:3300/0 conn(0x7f36740729f0 0x7f367406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:00.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.155+0000 7f367a1fd640 1 -- 172.21.15.98:0/1152613439 >> v1:172.21.15.98:6789/0 conn(0x7f3674144d10 legacy=0x7f36740722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:00.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.155+0000 7f367a1fd640 1 -- 172.21.15.98:0/1152613439 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3674073180 con 0x7f367406c1d0 2024-09-19T08:45:00.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.155+0000 7f36789fa640 1 -- 172.21.15.98:0/1152613439 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 490596502 0 0) 0x7f3674072ea0 con 0x7f3674144d10 2024-09-19T08:45:00.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.159+0000 7f367a1fd640 1 --2- 172.21.15.98:0/1152613439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f367406c1d0 0x7f36740cd250 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f36740aff50 tx=0x7f366c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee64b7ef73408066 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:00.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.159+0000 7f36789fa640 1 -- 172.21.15.98:0/1152613439 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366c0b2070 con 0x7f367406c1d0 2024-09-19T08:45:00.162 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.159+0000 7f36789fa640 1 -- 172.21.15.98:0/1152613439 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f366c0a58e0 con 0x7f367406c1d0 2024-09-19T08:45:00.162 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.159+0000 7f36789fa640 1 -- 172.21.15.98:0/1152613439 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366c0a5cb0 con 0x7f367406c1d0 2024-09-19T08:45:00.162 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.159+0000 7f367bc60640 1 -- 172.21.15.98:0/1152613439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f367406c1d0 msgr2=0x7f36740cd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:00.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.159+0000 7f367bc60640 1 --2- 172.21.15.98:0/1152613439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f367406c1d0 0x7f36740cd250 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f36740aff50 tx=0x7f366c01c930 comp rx=0 tx=0).stop 2024-09-19T08:45:00.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.159+0000 7f367bc60640 1 -- 172.21.15.98:0/1152613439 shutdown_connections 2024-09-19T08:45:00.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.159+0000 7f367bc60640 1 --2- 172.21.15.98:0/1152613439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f367406c1d0 0x7f36740cd250 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:00.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.159+0000 7f367bc60640 1 --2- 172.21.15.98:0/1152613439 >> v2:172.21.15.139:3300/0 conn(0x7f36740729f0 0x7f367406bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:00.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.159+0000 7f367bc60640 1 -- 172.21.15.98:0/1152613439 >> 172.21.15.98:0/1152613439 conn(0x7f36740b2010 msgr2=0x7f36740b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:00.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.159+0000 7f367bc60640 1 -- 172.21.15.98:0/1152613439 shutdown_connections 2024-09-19T08:45:00.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.159+0000 7f367bc60640 1 -- 172.21.15.98:0/1152613439 wait complete. 2024-09-19T08:45:00.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.159+0000 7f367bc60640 1 Processor -- start 2024-09-19T08:45:00.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.159+0000 7f367bc60640 1 -- start start 2024-09-19T08:45:00.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.163+0000 7f367bc60640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f367406c1d0 0x7f36741f4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:00.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.163+0000 7f367bc60640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36740729f0 0x7f36740c5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:00.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.163+0000 7f367bc60640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3674144d10 0x7f36740c6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:00.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.163+0000 7f367bc60640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f36740ce120 con 0x7f367406c1d0 2024-09-19T08:45:00.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.163+0000 7f367bc60640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f36740cdfa0 con 0x7f3674144d10 2024-09-19T08:45:00.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.163+0000 7f367bc60640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f36740ce2a0 con 0x7f36740729f0 2024-09-19T08:45:00.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.163+0000 7f36799fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f367406c1d0 0x7f36741f4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:00.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.163+0000 7f36799fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f367406c1d0 0x7f36741f4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47850/0 (socket says 172.21.15.98:47850) 2024-09-19T08:45:00.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.163+0000 7f36799fc640 1 -- 172.21.15.98:0/924941906 learned_addr learned my addr 172.21.15.98:0/924941906 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:00.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.163+0000 7f367a1fd640 1 --2- 172.21.15.98:0/924941906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3674144d10 0x7f36740c6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:00.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.163+0000 7f36791fb640 1 --2- 172.21.15.98:0/924941906 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36740729f0 0x7f36740c5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:00.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.163+0000 7f36799fc640 1 -- 172.21.15.98:0/924941906 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36740729f0 msgr2=0x7f36740c5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:00.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.163+0000 7f36799fc640 1 --2- 172.21.15.98:0/924941906 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36740729f0 0x7f36740c5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:00.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.163+0000 7f36799fc640 1 -- 172.21.15.98:0/924941906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3674144d10 msgr2=0x7f36740c6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:00.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.163+0000 7f36799fc640 1 --2- 172.21.15.98:0/924941906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3674144d10 0x7f36740c6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:00.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.163+0000 7f36799fc640 1 -- 172.21.15.98:0/924941906 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36741f81a0 con 0x7f367406c1d0 2024-09-19T08:45:00.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.163+0000 7f367a1fd640 1 --2- 172.21.15.98:0/924941906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3674144d10 0x7f36740c6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:00.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.163+0000 7f36799fc640 1 --2- 172.21.15.98:0/924941906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f367406c1d0 0x7f36741f4790 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f366800c4b0 tx=0x7f366800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:00.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.163+0000 7f36667fc640 1 -- 172.21.15.98:0/924941906 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3668018070 con 0x7f367406c1d0 2024-09-19T08:45:00.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.163+0000 7f36667fc640 1 -- 172.21.15.98:0/924941906 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3668007500 con 0x7f367406c1d0 2024-09-19T08:45:00.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.163+0000 7f367bc60640 1 -- 172.21.15.98:0/924941906 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36741f8480 con 0x7f367406c1d0 2024-09-19T08:45:00.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.163+0000 7f36667fc640 1 -- 172.21.15.98:0/924941906 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3668004a30 con 0x7f367406c1d0 2024-09-19T08:45:00.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.163+0000 7f367bc60640 1 -- 172.21.15.98:0/924941906 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36741f8a40 con 0x7f367406c1d0 2024-09-19T08:45:00.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.163+0000 7f367bc60640 1 -- 172.21.15.98:0/924941906 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36740cd230 con 0x7f367406c1d0 2024-09-19T08:45:00.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.163+0000 7f36667fc640 1 -- 172.21.15.98:0/924941906 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f366801b070 con 0x7f367406c1d0 2024-09-19T08:45:00.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.163+0000 7f36667fc640 1 --2- 172.21.15.98:0/924941906 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3658067cb0 0x7f365806a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:00.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.163+0000 7f36667fc640 1 -- 172.21.15.98:0/924941906 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5737+0+0 (secure 0 0 0) 0x7f3668085670 con 0x7f367406c1d0 2024-09-19T08:45:00.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.163+0000 7f36791fb640 1 --2- 172.21.15.98:0/924941906 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3658067cb0 0x7f365806a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:00.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.167+0000 7f36791fb640 1 --2- 172.21.15.98:0/924941906 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3658067cb0 0x7f365806a170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f367413e5f0 tx=0x7f3670008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:00.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.167+0000 7f36667fc640 1 -- 172.21.15.98:0/924941906 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3668055c00 con 0x7f367406c1d0 2024-09-19T08:45:00.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.375+0000 7f367bc60640 1 -- 172.21.15.98:0/924941906 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f36740b2920 con 0x7f367406c1d0 2024-09-19T08:45:00.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.379+0000 7f36667fc640 1 -- 172.21.15.98:0/924941906 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) ==== 190+0+147 (secure 0 0 0) 0x7f3668055500 con 0x7f367406c1d0 2024-09-19T08:45:00.380 INFO:teuthology.orchestra.run.smithi098.stdout:[client.0] 2024-09-19T08:45:00.380 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQC/4utm0vNpIhAA4S1E8dFZ0+ibmSKhmZ+bIA== 2024-09-19T08:45:00.380 INFO:teuthology.orchestra.run.smithi098.stdout: caps mds = "allow" 2024-09-19T08:45:00.381 INFO:teuthology.orchestra.run.smithi098.stdout: caps mon = "allow r" 2024-09-19T08:45:00.381 INFO:teuthology.orchestra.run.smithi098.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-19T08:45:00.381 INFO:teuthology.orchestra.run.smithi098.stderr:updated caps for client.0 2024-09-19T08:45:00.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.379+0000 7f367bc60640 1 -- 172.21.15.98:0/924941906 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3658067cb0 msgr2=0x7f365806a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:00.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.379+0000 7f367bc60640 1 --2- 172.21.15.98:0/924941906 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3658067cb0 0x7f365806a170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f367413e5f0 tx=0x7f3670008040 comp rx=0 tx=0).stop 2024-09-19T08:45:00.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.379+0000 7f367bc60640 1 -- 172.21.15.98:0/924941906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f367406c1d0 msgr2=0x7f36741f4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:00.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.379+0000 7f367bc60640 1 --2- 172.21.15.98:0/924941906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f367406c1d0 0x7f36741f4790 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f366800c4b0 tx=0x7f366800c970 comp rx=0 tx=0).stop 2024-09-19T08:45:00.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.379+0000 7f367bc60640 1 -- 172.21.15.98:0/924941906 shutdown_connections 2024-09-19T08:45:00.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.379+0000 7f367bc60640 1 --2- 172.21.15.98:0/924941906 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3658067cb0 0x7f365806a170 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:00.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.379+0000 7f367bc60640 1 --2- 172.21.15.98:0/924941906 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3674144d10 0x7f36740c6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:00.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.379+0000 7f367bc60640 1 --2- 172.21.15.98:0/924941906 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36740729f0 0x7f36740c5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:00.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.379+0000 7f367bc60640 1 --2- 172.21.15.98:0/924941906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f367406c1d0 0x7f36741f4790 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:00.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.379+0000 7f367bc60640 1 -- 172.21.15.98:0/924941906 >> 172.21.15.98:0/924941906 conn(0x7f36740b2010 msgr2=0x7f36740b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:00.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.383+0000 7f367bc60640 1 -- 172.21.15.98:0/924941906 shutdown_connections 2024-09-19T08:45:00.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.383+0000 7f367bc60640 1 -- 172.21.15.98:0/924941906 wait complete. 2024-09-19T08:45:00.399 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:45:00.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.519+0000 7f98104e1640 1 Processor -- start 2024-09-19T08:45:00.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.519+0000 7f98104e1640 1 -- start start 2024-09-19T08:45:00.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.519+0000 7f98104e1640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f98080b7f80 0x7f980815f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:00.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.519+0000 7f98104e1640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9808068440 con 0x7f98080b9970 2024-09-19T08:45:00.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.519+0000 7f98104e1640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9808162a90 con 0x7f98080d4820 2024-09-19T08:45:00.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.519+0000 7f98104e1640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f9808163050 con 0x7f98080b7f80 2024-09-19T08:45:00.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.519+0000 7f980e27d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f98080b7f80 0x7f980815f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:00.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.519+0000 7f980ea7e640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f98080b9970 0x7f98080b9de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52600/0 (socket says 172.21.15.98:52600) 2024-09-19T08:45:00.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.519+0000 7f980e27d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f98080b7f80 0x7f980815f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47860/0 (socket says 172.21.15.98:47860) 2024-09-19T08:45:00.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.519+0000 7f980ea7e640 1 -- 172.21.15.98:0/4160735912 learned_addr learned my addr 172.21.15.98:0/4160735912 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:00.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.519+0000 7f980e27d640 1 -- 172.21.15.98:0/4160735912 >> v1:172.21.15.139:6789/0 conn(0x7f98080d4820 legacy=0x7f98081621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:00.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.519+0000 7f980e27d640 1 -- 172.21.15.98:0/4160735912 >> v1:172.21.15.98:6789/0 conn(0x7f98080b9970 legacy=0x7f98080b9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:00.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.519+0000 7f980d27b640 1 -- 172.21.15.98:0/4160735912 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4247361195 0 0) 0x7f9808068440 con 0x7f98080b9970 2024-09-19T08:45:00.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.519+0000 7f980e27d640 1 -- 172.21.15.98:0/4160735912 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9808162d70 con 0x7f98080b7f80 2024-09-19T08:45:00.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.523+0000 7f980e27d640 1 --2- 172.21.15.98:0/4160735912 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98080b7f80 0x7f980815f560 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f98040097c0 tx=0x7f980401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d64e63c40fb98ef8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:00.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.523+0000 7f980d27b640 1 -- 172.21.15.98:0/4160735912 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98040b0070 con 0x7f98080b7f80 2024-09-19T08:45:00.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.523+0000 7f980d27b640 1 -- 172.21.15.98:0/4160735912 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98040ab600 con 0x7f98080b7f80 2024-09-19T08:45:00.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.523+0000 7f980d27b640 1 -- 172.21.15.98:0/4160735912 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98040ab9d0 con 0x7f98080b7f80 2024-09-19T08:45:00.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.523+0000 7f98104e1640 1 -- 172.21.15.98:0/4160735912 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98080b7f80 msgr2=0x7f980815f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:00.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.523+0000 7f98104e1640 1 --2- 172.21.15.98:0/4160735912 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98080b7f80 0x7f980815f560 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f98040097c0 tx=0x7f980401c410 comp rx=0 tx=0).stop 2024-09-19T08:45:00.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.523+0000 7f98104e1640 1 -- 172.21.15.98:0/4160735912 shutdown_connections 2024-09-19T08:45:00.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.523+0000 7f98104e1640 1 --2- 172.21.15.98:0/4160735912 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98080b7f80 0x7f980815f560 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:00.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.523+0000 7f98104e1640 1 -- 172.21.15.98:0/4160735912 >> 172.21.15.98:0/4160735912 conn(0x7f9808074e40 msgr2=0x7f9808075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:00.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.523+0000 7f98104e1640 1 -- 172.21.15.98:0/4160735912 shutdown_connections 2024-09-19T08:45:00.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.523+0000 7f98104e1640 1 -- 172.21.15.98:0/4160735912 wait complete. 2024-09-19T08:45:00.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.523+0000 7f98104e1640 1 Processor -- start 2024-09-19T08:45:00.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.523+0000 7f98104e1640 1 -- start start 2024-09-19T08:45:00.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.523+0000 7f98104e1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98080b7f80 0x7f9808156700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:00.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.523+0000 7f98104e1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98080b9970 0x7f980813e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:00.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.523+0000 7f98104e1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98080d4820 0x7f98081538a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:00.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.523+0000 7f98104e1640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9808144590 con 0x7f98080d4820 2024-09-19T08:45:00.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.523+0000 7f98104e1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f98080b8570 con 0x7f98080b9970 2024-09-19T08:45:00.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.523+0000 7f98104e1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9808144710 con 0x7f98080b7f80 2024-09-19T08:45:00.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.523+0000 7f980ea7e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98080d4820 0x7f98081538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:00.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.523+0000 7f980ea7e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98080d4820 0x7f98081538a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47864/0 (socket says 172.21.15.98:47864) 2024-09-19T08:45:00.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.523+0000 7f980ea7e640 1 -- 172.21.15.98:0/714648474 learned_addr learned my addr 172.21.15.98:0/714648474 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:00.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.527+0000 7f980da7c640 1 --2- 172.21.15.98:0/714648474 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98080b9970 0x7f980813e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:00.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.527+0000 7f980e27d640 1 --2- 172.21.15.98:0/714648474 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98080b7f80 0x7f9808156700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:00.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.527+0000 7f980ea7e640 1 -- 172.21.15.98:0/714648474 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98080b7f80 msgr2=0x7f9808156700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:00.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.527+0000 7f980ea7e640 1 --2- 172.21.15.98:0/714648474 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98080b7f80 0x7f9808156700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:00.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.527+0000 7f980ea7e640 1 -- 172.21.15.98:0/714648474 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98080b9970 msgr2=0x7f980813e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:00.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.527+0000 7f980ea7e640 1 --2- 172.21.15.98:0/714648474 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98080b9970 0x7f980813e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:00.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.527+0000 7f980ea7e640 1 -- 172.21.15.98:0/714648474 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9808156ce0 con 0x7f98080d4820 2024-09-19T08:45:00.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.527+0000 7f980ea7e640 1 --2- 172.21.15.98:0/714648474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98080d4820 0x7f98081538a0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f980000c7d0 tx=0x7f980000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:00.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.527+0000 7f97feffd640 1 -- 172.21.15.98:0/714648474 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9800018070 con 0x7f98080d4820 2024-09-19T08:45:00.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.527+0000 7f97feffd640 1 -- 172.21.15.98:0/714648474 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f980000a640 con 0x7f98080d4820 2024-09-19T08:45:00.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.527+0000 7f98104e1640 1 -- 172.21.15.98:0/714648474 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9808156fc0 con 0x7f98080d4820 2024-09-19T08:45:00.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.527+0000 7f97feffd640 1 -- 172.21.15.98:0/714648474 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980000aa70 con 0x7f98080d4820 2024-09-19T08:45:00.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.527+0000 7f98104e1640 1 -- 172.21.15.98:0/714648474 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9808157580 con 0x7f98080d4820 2024-09-19T08:45:00.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.527+0000 7f98104e1640 1 -- 172.21.15.98:0/714648474 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f980815f540 con 0x7f98080d4820 2024-09-19T08:45:00.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.527+0000 7f97feffd640 1 -- 172.21.15.98:0/714648474 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f980001b070 con 0x7f98080d4820 2024-09-19T08:45:00.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.527+0000 7f97feffd640 1 --2- 172.21.15.98:0/714648474 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f97ec067cb0 0x7f97ec06a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:00.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.527+0000 7f97feffd640 1 -- 172.21.15.98:0/714648474 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5737+0+0 (secure 0 0 0) 0x7f9800085c80 con 0x7f98080d4820 2024-09-19T08:45:00.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.527+0000 7f980e27d640 1 --2- 172.21.15.98:0/714648474 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f97ec067cb0 0x7f97ec06a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:00.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.531+0000 7f980e27d640 1 --2- 172.21.15.98:0/714648474 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f97ec067cb0 0x7f97ec06a170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f98040ad000 tx=0x7f9804008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:00.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.531+0000 7f97feffd640 1 -- 172.21.15.98:0/714648474 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9800056350 con 0x7f98080d4820 2024-09-19T08:45:00.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.723+0000 7f98104e1640 1 -- 172.21.15.98:0/714648474 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9808161980 con 0x7f98080d4820 2024-09-19T08:45:00.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.723+0000 7f97feffd640 1 -- 172.21.15.98:0/714648474 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5228 (secure 0 0 0) 0x7f9800055c50 con 0x7f98080d4820 2024-09-19T08:45:00.726 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:45:00.727 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":67,"btime":"2024-09-19T08:44:58:625080+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5645,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/1334032829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":1334032829},{"type":"v1","addr":"172.21.15.139:6833","nonce":1334032829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":5651,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2541341035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2541341035},{"type":"v1","addr":"172.21.15.139:6835","nonce":2541341035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":7521,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3632520818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3632520818},{"type":"v1","addr":"172.21.15.98:6833","nonce":3632520818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:44:55.910161+0000","modified":"2024-09-19T08:44:57.947782+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7524},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7524":{"gid":7524,"name":"c","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/502938874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":502938874},{"type":"v1","addr":"172.21.15.98:6835","nonce":502938874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7524,"qdb_cluster":[7524]},"id":9}]} 2024-09-19T08:45:00.727 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 67 2024-09-19T08:45:00.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.727+0000 7f98104e1640 1 -- 172.21.15.98:0/714648474 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f97ec067cb0 msgr2=0x7f97ec06a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:00.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.727+0000 7f98104e1640 1 --2- 172.21.15.98:0/714648474 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f97ec067cb0 0x7f97ec06a170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f98040ad000 tx=0x7f9804008040 comp rx=0 tx=0).stop 2024-09-19T08:45:00.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.727+0000 7f98104e1640 1 -- 172.21.15.98:0/714648474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98080d4820 msgr2=0x7f98081538a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:00.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.727+0000 7f98104e1640 1 --2- 172.21.15.98:0/714648474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98080d4820 0x7f98081538a0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f980000c7d0 tx=0x7f980000cc90 comp rx=0 tx=0).stop 2024-09-19T08:45:00.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.727+0000 7f98104e1640 1 -- 172.21.15.98:0/714648474 shutdown_connections 2024-09-19T08:45:00.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.727+0000 7f98104e1640 1 --2- 172.21.15.98:0/714648474 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f97ec067cb0 0x7f97ec06a170 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:00.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.727+0000 7f98104e1640 1 --2- 172.21.15.98:0/714648474 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98080d4820 0x7f98081538a0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:00.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.727+0000 7f98104e1640 1 --2- 172.21.15.98:0/714648474 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98080b9970 0x7f980813e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:00.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.727+0000 7f98104e1640 1 --2- 172.21.15.98:0/714648474 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98080b7f80 0x7f9808156700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:00.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.727+0000 7f98104e1640 1 -- 172.21.15.98:0/714648474 >> 172.21.15.98:0/714648474 conn(0x7f9808074e40 msgr2=0x7f98080d28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:00.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.727+0000 7f98104e1640 1 -- 172.21.15.98:0/714648474 shutdown_connections 2024-09-19T08:45:00.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:00.727+0000 7f98104e1640 1 -- 172.21.15.98:0/714648474 wait complete. 2024-09-19T08:45:01.147 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T08:44:55.910161+0000', 'modified': '2024-09-19T08:44:57.947782+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7524}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7524': {'gid': 7524, 'name': 'c', 'rank': 0, 'incarnation': 65, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.98:6835/502938874', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6834', 'nonce': 502938874}, {'type': 'v1', 'addr': '172.21.15.98:6835', 'nonce': 502938874}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7524, 'qdb_cluster': [7524]} 2024-09-19T08:45:01.147 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T08:45:01.147 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-19T08:45:01.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.275+0000 7f2f5efb3640 1 Processor -- start 2024-09-19T08:45:01.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.275+0000 7f2f5efb3640 1 -- start start 2024-09-19T08:45:01.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.275+0000 7f2f5efb3640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2f58158fd0 0x7f2f580b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:01.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.275+0000 7f2f5efb3640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2f5806c3a0 0x7f2f580ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:01.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.275+0000 7f2f5efb3640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f580ccc60 con 0x7f2f580b4360 2024-09-19T08:45:01.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.275+0000 7f2f5efb3640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f2f581593b0 con 0x7f2f58158fd0 2024-09-19T08:45:01.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.275+0000 7f2f5efb3640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f2f580ccf40 con 0x7f2f5806c3a0 2024-09-19T08:45:01.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.275+0000 7f2f5cd4f640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2f58158fd0 0x7f2f580b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:01.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.275+0000 7f2f57fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f2f580b4360 0x7f2f5806bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52604/0 (socket says 172.21.15.98:52604) 2024-09-19T08:45:01.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.275+0000 7f2f57fff640 1 -- 172.21.15.98:0/792035594 learned_addr learned my addr 172.21.15.98:0/792035594 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:01.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.275+0000 7f2f5cd4f640 1 -- 172.21.15.98:0/792035594 >> v2:172.21.15.139:3300/0 conn(0x7f2f5806c3a0 msgr2=0x7f2f580ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:01.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.275+0000 7f2f5d550640 1 --2- 172.21.15.98:0/792035594 >> v2:172.21.15.139:3300/0 conn(0x7f2f5806c3a0 0x7f2f580ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:01.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.275+0000 7f2f5cd4f640 1 --2- 172.21.15.98:0/792035594 >> v2:172.21.15.139:3300/0 conn(0x7f2f5806c3a0 0x7f2f580ce840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.275+0000 7f2f5cd4f640 1 -- 172.21.15.98:0/792035594 >> v1:172.21.15.98:6789/0 conn(0x7f2f580b4360 legacy=0x7f2f5806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:01.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.275+0000 7f2f5cd4f640 1 -- 172.21.15.98:0/792035594 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f580cd220 con 0x7f2f58158fd0 2024-09-19T08:45:01.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.279+0000 7f2f5cd4f640 1 --2- 172.21.15.98:0/792035594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f58158fd0 0x7f2f580b3e20 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f2f580afe30 tx=0x7f2f4801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c456bd5223f70996 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:01.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.279+0000 7f2f577fe640 1 -- 172.21.15.98:0/792035594 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f480b1070 con 0x7f2f58158fd0 2024-09-19T08:45:01.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.279+0000 7f2f577fe640 1 -- 172.21.15.98:0/792035594 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f480ac620 con 0x7f2f58158fd0 2024-09-19T08:45:01.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.279+0000 7f2f577fe640 1 -- 172.21.15.98:0/792035594 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f480ac9f0 con 0x7f2f58158fd0 2024-09-19T08:45:01.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.279+0000 7f2f5efb3640 1 -- 172.21.15.98:0/792035594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f58158fd0 msgr2=0x7f2f580b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:01.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.279+0000 7f2f5efb3640 1 --2- 172.21.15.98:0/792035594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f58158fd0 0x7f2f580b3e20 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f2f580afe30 tx=0x7f2f4801c3c0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.279+0000 7f2f5efb3640 1 -- 172.21.15.98:0/792035594 shutdown_connections 2024-09-19T08:45:01.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.279+0000 7f2f5efb3640 1 --2- 172.21.15.98:0/792035594 >> v2:172.21.15.139:3300/0 conn(0x7f2f5806c3a0 0x7f2f580ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.279+0000 7f2f5efb3640 1 --2- 172.21.15.98:0/792035594 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f58158fd0 0x7f2f580b3e20 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.279+0000 7f2f5efb3640 1 -- 172.21.15.98:0/792035594 >> 172.21.15.98:0/792035594 conn(0x7f2f580b2010 msgr2=0x7f2f580b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:01.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.279+0000 7f2f5efb3640 1 -- 172.21.15.98:0/792035594 shutdown_connections 2024-09-19T08:45:01.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.279+0000 7f2f5efb3640 1 -- 172.21.15.98:0/792035594 wait complete. 2024-09-19T08:45:01.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.279+0000 7f2f5efb3640 1 Processor -- start 2024-09-19T08:45:01.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.283+0000 7f2f5efb3640 1 -- start start 2024-09-19T08:45:01.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.283+0000 7f2f5efb3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f5806c3a0 0x7f2f581521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:01.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.283+0000 7f2f5efb3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f580b4360 0x7f2f58165890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:01.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.283+0000 7f2f5efb3640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f58158fd0 0x7f2f58150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:01.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.283+0000 7f2f5efb3640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2f580cf710 con 0x7f2f5806c3a0 2024-09-19T08:45:01.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.283+0000 7f2f5efb3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2f580cf590 con 0x7f2f580b4360 2024-09-19T08:45:01.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.283+0000 7f2f5efb3640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2f580cf890 con 0x7f2f58158fd0 2024-09-19T08:45:01.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.283+0000 7f2f5cd4f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f5806c3a0 0x7f2f581521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:01.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.283+0000 7f2f5cd4f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f5806c3a0 0x7f2f581521b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47870/0 (socket says 172.21.15.98:47870) 2024-09-19T08:45:01.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.283+0000 7f2f5cd4f640 1 -- 172.21.15.98:0/1370634239 learned_addr learned my addr 172.21.15.98:0/1370634239 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:01.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.283+0000 7f2f5cd4f640 1 -- 172.21.15.98:0/1370634239 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f58158fd0 msgr2=0x7f2f58150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:01.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.283+0000 7f2f5cd4f640 1 --2- 172.21.15.98:0/1370634239 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f58158fd0 0x7f2f58150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.283+0000 7f2f5cd4f640 1 -- 172.21.15.98:0/1370634239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f580b4360 msgr2=0x7f2f58165890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:01.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.283+0000 7f2f5cd4f640 1 --2- 172.21.15.98:0/1370634239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f580b4360 0x7f2f58165890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.283+0000 7f2f5cd4f640 1 -- 172.21.15.98:0/1370634239 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f58153110 con 0x7f2f5806c3a0 2024-09-19T08:45:01.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.283+0000 7f2f5cd4f640 1 --2- 172.21.15.98:0/1370634239 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f5806c3a0 0x7f2f581521b0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f2f480af040 tx=0x7f2f4801ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:01.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.283+0000 7f2f557fa640 1 -- 172.21.15.98:0/1370634239 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f480b1050 con 0x7f2f5806c3a0 2024-09-19T08:45:01.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.283+0000 7f2f557fa640 1 -- 172.21.15.98:0/1370634239 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f480bd070 con 0x7f2f5806c3a0 2024-09-19T08:45:01.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.283+0000 7f2f5efb3640 1 -- 172.21.15.98:0/1370634239 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f58152870 con 0x7f2f5806c3a0 2024-09-19T08:45:01.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.283+0000 7f2f557fa640 1 -- 172.21.15.98:0/1370634239 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f480b88b0 con 0x7f2f5806c3a0 2024-09-19T08:45:01.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.283+0000 7f2f5efb3640 1 -- 172.21.15.98:0/1370634239 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f58152e30 con 0x7f2f5806c3a0 2024-09-19T08:45:01.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.283+0000 7f2f557fa640 1 -- 172.21.15.98:0/1370634239 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f2f480ac4e0 con 0x7f2f5806c3a0 2024-09-19T08:45:01.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.283+0000 7f2f557fa640 1 --2- 172.21.15.98:0/1370634239 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2f44067ce0 0x7f2f4406a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:01.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f557fa640 1 -- 172.21.15.98:0/1370634239 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5737+0+0 (secure 0 0 0) 0x7f2f48120da0 con 0x7f2f5806c3a0 2024-09-19T08:45:01.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f57fff640 1 --2- 172.21.15.98:0/1370634239 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2f44067ce0 0x7f2f4406a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:01.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f6206f000 1 Processor -- start 2024-09-19T08:45:01.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f6206f000 1 -- start start 2024-09-19T08:45:01.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f57fff640 1 --2- 172.21.15.98:0/1370634239 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2f44067ce0 0x7f2f4406a1a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c000f70 tx=0x7f2f4c007a90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:01.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f6206f000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x558dbc758d70 0x558dbc759150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:01.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f6206f000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dbc759690 0x558dbc70f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:01.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f6206f000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558dbc70f6a0 0x558dbc70fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:01.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f6206f000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558dbc5f2b60 con 0x558dbc759690 2024-09-19T08:45:01.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f6206f000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558dbc559c70 con 0x558dbc70f6a0 2024-09-19T08:45:01.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f6206f000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x558dbc4ffed0 con 0x558dbc758d70 2024-09-19T08:45:01.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f57fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dbc759690 0x558dbc70f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:01.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f57fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dbc759690 0x558dbc70f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47886/0 (socket says 172.21.15.98:47886) 2024-09-19T08:45:01.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f57fff640 1 -- 172.21.15.98:0/1216049361 learned_addr learned my addr 172.21.15.98:0/1216049361 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:01.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f5cd4f640 1 --2- 172.21.15.98:0/1216049361 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x558dbc758d70 0x558dbc759150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:01.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f5d550640 1 --2- 172.21.15.98:0/1216049361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558dbc70f6a0 0x558dbc70fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:01.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f57fff640 1 -- 172.21.15.98:0/1216049361 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x558dbc758d70 msgr2=0x558dbc759150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:01.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f57fff640 1 --2- 172.21.15.98:0/1216049361 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x558dbc758d70 0x558dbc759150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f57fff640 1 -- 172.21.15.98:0/1216049361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558dbc70f6a0 msgr2=0x558dbc70fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:01.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f57fff640 1 --2- 172.21.15.98:0/1216049361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558dbc70f6a0 0x558dbc70fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f57fff640 1 -- 172.21.15.98:0/1216049361 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dbc511e50 con 0x558dbc759690 2024-09-19T08:45:01.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f5d550640 1 --2- 172.21.15.98:0/1216049361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558dbc70f6a0 0x558dbc70fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:01.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f57fff640 1 --2- 172.21.15.98:0/1216049361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dbc759690 0x558dbc70f160 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c005cc0 tx=0x7f2f4c00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c50afcdd9495cb22 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:01.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f3e7fc640 1 -- 172.21.15.98:0/1216049361 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c016110 con 0x558dbc759690 2024-09-19T08:45:01.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f3e7fc640 1 -- 172.21.15.98:0/1216049361 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f4c007560 con 0x558dbc759690 2024-09-19T08:45:01.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f3e7fc640 1 -- 172.21.15.98:0/1216049361 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c0162b0 con 0x558dbc759690 2024-09-19T08:45:01.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f6206f000 1 -- 172.21.15.98:0/1216049361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dbc759690 msgr2=0x558dbc70f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:01.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f6206f000 1 --2- 172.21.15.98:0/1216049361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dbc759690 0x558dbc70f160 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c005cc0 tx=0x7f2f4c00f000 comp rx=0 tx=0).stop 2024-09-19T08:45:01.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f6206f000 1 -- 172.21.15.98:0/1216049361 shutdown_connections 2024-09-19T08:45:01.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f6206f000 1 --2- 172.21.15.98:0/1216049361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558dbc70f6a0 0x558dbc70fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f6206f000 1 --2- 172.21.15.98:0/1216049361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dbc759690 0x558dbc70f160 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f6206f000 1 --2- 172.21.15.98:0/1216049361 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x558dbc758d70 0x558dbc759150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f6206f000 1 -- 172.21.15.98:0/1216049361 >> 172.21.15.98:0/1216049361 conn(0x558dbc711600 msgr2=0x558dbc709630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:01.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f6206f000 1 -- 172.21.15.98:0/1216049361 shutdown_connections 2024-09-19T08:45:01.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f6206f000 1 -- 172.21.15.98:0/1216049361 wait complete. 2024-09-19T08:45:01.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f6206f000 1 -- start start 2024-09-19T08:45:01.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.287+0000 7f2f6206f000 1 Processor -- start 2024-09-19T08:45:01.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f6206f000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558dbc758d70 0x558dbc745670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:01.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f6206f000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x558dbc759690 0x558dbc745bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:01.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f6206f000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dbc7460f0 0x558dbc7464d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:01.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f6206f000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558dbc5bbb00 con 0x558dbc7460f0 2024-09-19T08:45:01.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f6206f000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558dbc534360 con 0x558dbc758d70 2024-09-19T08:45:01.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f6206f000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x558dbc5312d0 con 0x558dbc759690 2024-09-19T08:45:01.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f6206f000 10 client.0 _ll_register_callbacks cb 0x558dbc6f6820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:45:01.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f5d550640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dbc7460f0 0x558dbc7464d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:01.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f5d550640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dbc7460f0 0x558dbc7464d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47898/0 (socket says 172.21.15.98:47898) 2024-09-19T08:45:01.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f5d550640 1 -- 172.21.15.98:0/860282141 learned_addr learned my addr 172.21.15.98:0/860282141 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:01.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f5cd4f640 1 --2- 172.21.15.98:0/860282141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558dbc758d70 0x558dbc745670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:01.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f57fff640 1 --2- 172.21.15.98:0/860282141 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x558dbc759690 0x558dbc745bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:01.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f5d550640 1 -- 172.21.15.98:0/860282141 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x558dbc759690 msgr2=0x558dbc745bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:01.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f5d550640 1 --2- 172.21.15.98:0/860282141 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x558dbc759690 0x558dbc745bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f5d550640 1 -- 172.21.15.98:0/860282141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558dbc758d70 msgr2=0x558dbc745670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:01.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f5d550640 1 --2- 172.21.15.98:0/860282141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558dbc758d70 0x558dbc745670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f5d550640 1 -- 172.21.15.98:0/860282141 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dbc500120 con 0x558dbc7460f0 2024-09-19T08:45:01.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f57fff640 1 --2- 172.21.15.98:0/860282141 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x558dbc759690 0x558dbc745bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:01.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f5cd4f640 1 --2- 172.21.15.98:0/860282141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558dbc758d70 0x558dbc745670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:01.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f5d550640 1 --2- 172.21.15.98:0/860282141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dbc7460f0 0x558dbc7464d0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f2f5000da20 tx=0x7f2f5000dee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3661d011c9dc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:01.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f377fe640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-19T08:45:01.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f377fe640 1 -- 172.21.15.98:0/860282141 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5001b060 con 0x558dbc7460f0 2024-09-19T08:45:01.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f377fe640 1 -- 172.21.15.98:0/860282141 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 5737+0+0 (secure 0 0 0) 0x7f2f50016440 con 0x558dbc7460f0 2024-09-19T08:45:01.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f6206f000 1 -- 172.21.15.98:0/860282141 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f4c007560 con 0x558dbc7460f0 2024-09-19T08:45:01.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f377fe640 1 -- 172.21.15.98:0/860282141 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f2f50021050 con 0x558dbc7460f0 2024-09-19T08:45:01.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f6206f000 10 client.7653 fetch_fsmap learned FSMap version 67 2024-09-19T08:45:01.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f6206f000 1 -- 172.21.15.98:0/860282141 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x558dbc4f9450 con 0x558dbc7460f0 2024-09-19T08:45:01.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f377fe640 1 -- 172.21.15.98:0/860282141 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f4c007560 con 0x558dbc7460f0 2024-09-19T08:45:01.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f377fe640 1 -- 172.21.15.98:0/860282141 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 67) ==== 3502+0+0 (secure 0 0 0) 0x7f2f50024020 con 0x558dbc7460f0 2024-09-19T08:45:01.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f6206f000 10 client.7653 fetch_fsmap finished waiting for FSMap version 67 2024-09-19T08:45:01.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f6206f000 10 client.7653 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T08:45:01.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f6206f000 20 client.7653 populate_metadata read hostname 'smithi098' 2024-09-19T08:45:01.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f6206f000 1 --2- 172.21.15.98:0/860282141 >> [v2:172.21.15.98:6834/502938874,v1:172.21.15.98:6835/502938874] conn(0x558dbc7674d0 0x558dbc7698c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:01.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f6206f000 4 client.7653 mds_command: new command op to 7524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:45:01.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f6206f000 1 -- 172.21.15.98:0/860282141 --> [v2:172.21.15.98:6834/502938874,v1:172.21.15.98:6835/502938874] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558dbc4f3c00 con 0x558dbc7674d0 2024-09-19T08:45:01.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f5cd4f640 1 --2- 172.21.15.98:0/860282141 >> [v2:172.21.15.98:6834/502938874,v1:172.21.15.98:6835/502938874] conn(0x558dbc7674d0 0x558dbc7698c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:01.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f377fe640 10 client.7653 ms_handle_connect on v2:172.21.15.98:6834/502938874 2024-09-19T08:45:01.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f5cd4f640 1 --2- 172.21.15.98:0/860282141 >> [v2:172.21.15.98:6834/502938874,v1:172.21.15.98:6835/502938874] conn(0x558dbc7674d0 0x558dbc7698c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38d4850a2d839221 server_cookie=7bfec3ffb5f26f6f in_seq=0 out_seq=0 2024-09-19T08:45:01.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f377fe640 1 -- 172.21.15.98:0/860282141 <== mds.0 v2:172.21.15.98:6834/502938874 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558dbc4f3c00 con 0x558dbc7674d0 2024-09-19T08:45:01.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f377fe640 10 client.7653 handle_command_reply: tid=0 2024-09-19T08:45:01.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f6206f000 1 client.7653 shutdown 2024-09-19T08:45:01.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f6206f000 1 -- 172.21.15.98:0/860282141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dbc7460f0 msgr2=0x558dbc7464d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:01.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f6206f000 1 --2- 172.21.15.98:0/860282141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dbc7460f0 0x558dbc7464d0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f2f5000da20 tx=0x7f2f5000dee0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f6206f000 1 -- 172.21.15.98:0/860282141 shutdown_connections 2024-09-19T08:45:01.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f6206f000 1 --2- 172.21.15.98:0/860282141 >> [v2:172.21.15.98:6834/502938874,v1:172.21.15.98:6835/502938874] conn(0x558dbc7674d0 0x558dbc7698c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f6206f000 1 --2- 172.21.15.98:0/860282141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dbc7460f0 0x558dbc7464d0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f377fe640 0 client.7653 ms_handle_reset on v2:172.21.15.98:6834/502938874 2024-09-19T08:45:01.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f6206f000 1 --2- 172.21.15.98:0/860282141 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x558dbc759690 0x558dbc745bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f6206f000 1 --2- 172.21.15.98:0/860282141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558dbc758d70 0x558dbc745670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f6206f000 1 -- 172.21.15.98:0/860282141 >> 172.21.15.98:0/860282141 conn(0x558dbc70f6a0 msgr2=0x558dbc7109b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:01.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f6206f000 1 -- 172.21.15.98:0/860282141 shutdown_connections 2024-09-19T08:45:01.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f6206f000 1 -- 172.21.15.98:0/860282141 wait complete. 2024-09-19T08:45:01.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.291+0000 7f2f6206f000 20 client.7653 trim_cache size 0 max 16384 2024-09-19T08:45:01.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.311+0000 7f2f6206f000 1 Processor -- start 2024-09-19T08:45:01.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.311+0000 7f2f6206f000 1 -- start start 2024-09-19T08:45:01.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.311+0000 7f2f6206f000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x558dbc7460f0 0x558dbc757e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:01.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.311+0000 7f2f6206f000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dbc758d70 0x558dbc758340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:01.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.311+0000 7f2f6206f000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558dbc759690 0x558dbc7698c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:01.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.311+0000 7f2f6206f000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5815efe0 con 0x558dbc758d70 2024-09-19T08:45:01.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.311+0000 7f2f6206f000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2f581f9340 con 0x558dbc759690 2024-09-19T08:45:01.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.311+0000 7f2f6206f000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2f5815e990 con 0x558dbc7460f0 2024-09-19T08:45:01.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.311+0000 7f2f57fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dbc758d70 0x558dbc758340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:01.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.311+0000 7f2f57fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dbc758d70 0x558dbc758340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47902/0 (socket says 172.21.15.98:47902) 2024-09-19T08:45:01.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.311+0000 7f2f57fff640 1 -- 172.21.15.98:0/1479777900 learned_addr learned my addr 172.21.15.98:0/1479777900 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:01.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.311+0000 7f2f57fff640 1 -- 172.21.15.98:0/1479777900 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x558dbc7460f0 msgr2=0x558dbc757e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:01.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.311+0000 7f2f5cd4f640 1 --2- 172.21.15.98:0/1479777900 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x558dbc7460f0 0x558dbc757e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:01.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.311+0000 7f2f5d550640 1 --2- 172.21.15.98:0/1479777900 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558dbc759690 0x558dbc7698c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:01.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.311+0000 7f2f57fff640 1 --2- 172.21.15.98:0/1479777900 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x558dbc7460f0 0x558dbc757e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.311+0000 7f2f57fff640 1 -- 172.21.15.98:0/1479777900 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558dbc759690 msgr2=0x558dbc7698c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:01.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.311+0000 7f2f57fff640 1 --2- 172.21.15.98:0/1479777900 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558dbc759690 0x558dbc7698c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.311+0000 7f2f57fff640 1 -- 172.21.15.98:0/1479777900 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dbc4e0fd0 con 0x558dbc758d70 2024-09-19T08:45:01.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.311+0000 7f2f57fff640 1 --2- 172.21.15.98:0/1479777900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dbc758d70 0x558dbc758340 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x558dbc611ac0 tx=0x7f2f4c007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa62668f1aef8e4c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:01.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.311+0000 7f2f3dffb640 1 -- 172.21.15.98:0/1479777900 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c016090 con 0x558dbc758d70 2024-09-19T08:45:01.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.311+0000 7f2f3dffb640 1 -- 172.21.15.98:0/1479777900 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f4c016230 con 0x558dbc758d70 2024-09-19T08:45:01.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.311+0000 7f2f3dffb640 1 -- 172.21.15.98:0/1479777900 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c016950 con 0x558dbc758d70 2024-09-19T08:45:01.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.311+0000 7f2f6206f000 1 -- 172.21.15.98:0/1479777900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dbc758d70 msgr2=0x558dbc758340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:01.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.311+0000 7f2f6206f000 1 --2- 172.21.15.98:0/1479777900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dbc758d70 0x558dbc758340 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x558dbc611ac0 tx=0x7f2f4c007000 comp rx=0 tx=0).stop 2024-09-19T08:45:01.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.311+0000 7f2f6206f000 1 -- 172.21.15.98:0/1479777900 shutdown_connections 2024-09-19T08:45:01.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.311+0000 7f2f6206f000 1 --2- 172.21.15.98:0/1479777900 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558dbc759690 0x558dbc7698c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.311+0000 7f2f6206f000 1 --2- 172.21.15.98:0/1479777900 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dbc758d70 0x558dbc758340 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.311+0000 7f2f6206f000 1 --2- 172.21.15.98:0/1479777900 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x558dbc7460f0 0x558dbc757e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.311+0000 7f2f6206f000 1 -- 172.21.15.98:0/1479777900 >> 172.21.15.98:0/1479777900 conn(0x558dbc70f6a0 msgr2=0x558dbc709ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:01.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.311+0000 7f2f6206f000 1 -- 172.21.15.98:0/1479777900 shutdown_connections 2024-09-19T08:45:01.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.311+0000 7f2f6206f000 1 -- 172.21.15.98:0/1479777900 wait complete. 2024-09-19T08:45:01.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f6206f000 1 -- start start 2024-09-19T08:45:01.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f6206f000 1 Processor -- start 2024-09-19T08:45:01.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f6206f000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dbc7460f0 0x558dbc7cc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:01.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f6206f000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x558dbc758d70 0x558dbc7ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:01.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f6206f000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558dbc759690 0x558dbc7cd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:01.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f6206f000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x558dbc558f80 con 0x558dbc7460f0 2024-09-19T08:45:01.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f6206f000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558dbc511ca0 con 0x558dbc759690 2024-09-19T08:45:01.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f6206f000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x558dbc55e2a0 con 0x558dbc758d70 2024-09-19T08:45:01.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f6206f000 10 client.0 _ll_register_callbacks cb 0x7f2f500076f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:45:01.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f5cd4f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dbc7460f0 0x558dbc7cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:01.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f5cd4f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dbc7460f0 0x558dbc7cc980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47908/0 (socket says 172.21.15.98:47908) 2024-09-19T08:45:01.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f5cd4f640 1 -- 172.21.15.98:0/2900254928 learned_addr learned my addr 172.21.15.98:0/2900254928 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:01.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f57fff640 1 --2- 172.21.15.98:0/2900254928 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x558dbc758d70 0x558dbc7ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:01.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f5d550640 1 --2- 172.21.15.98:0/2900254928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558dbc759690 0x558dbc7cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:01.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f5cd4f640 1 -- 172.21.15.98:0/2900254928 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x558dbc758d70 msgr2=0x558dbc7ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:01.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f5cd4f640 1 --2- 172.21.15.98:0/2900254928 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x558dbc758d70 0x558dbc7ccec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f5cd4f640 1 -- 172.21.15.98:0/2900254928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558dbc759690 msgr2=0x558dbc7cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:01.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f5cd4f640 1 --2- 172.21.15.98:0/2900254928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558dbc759690 0x558dbc7cd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f5cd4f640 1 -- 172.21.15.98:0/2900254928 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dbc5dc750 con 0x558dbc7460f0 2024-09-19T08:45:01.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f5d550640 1 --2- 172.21.15.98:0/2900254928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558dbc759690 0x558dbc7cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:01.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f5cd4f640 1 --2- 172.21.15.98:0/2900254928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dbc7460f0 0x558dbc7cc980 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f2f480f0790 tx=0x7f2f48000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0a44293bdc10a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:01.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f377fe640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-19T08:45:01.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f377fe640 1 -- 172.21.15.98:0/2900254928 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f480f9670 con 0x558dbc7460f0 2024-09-19T08:45:01.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f377fe640 1 -- 172.21.15.98:0/2900254928 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 5737+0+0 (secure 0 0 0) 0x7f2f480ed100 con 0x558dbc7460f0 2024-09-19T08:45:01.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f6206f000 1 -- 172.21.15.98:0/2900254928 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f4c016230 con 0x558dbc7460f0 2024-09-19T08:45:01.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f377fe640 1 -- 172.21.15.98:0/2900254928 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f2f480f91e0 con 0x558dbc7460f0 2024-09-19T08:45:01.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f6206f000 10 client.7659 fetch_fsmap learned FSMap version 67 2024-09-19T08:45:01.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f6206f000 1 -- 172.21.15.98:0/2900254928 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x558dbc4e7a30 con 0x558dbc7460f0 2024-09-19T08:45:01.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f377fe640 1 -- 172.21.15.98:0/2900254928 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f4c016230 con 0x558dbc7460f0 2024-09-19T08:45:01.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f377fe640 1 -- 172.21.15.98:0/2900254928 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 67) ==== 3502+0+0 (secure 0 0 0) 0x7f2f480f70b0 con 0x558dbc7460f0 2024-09-19T08:45:01.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f6206f000 10 client.7659 fetch_fsmap finished waiting for FSMap version 67 2024-09-19T08:45:01.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f6206f000 10 client.7659 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T08:45:01.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f6206f000 20 client.7659 populate_metadata read hostname 'smithi098' 2024-09-19T08:45:01.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f6206f000 1 --2- 172.21.15.98:0/2900254928 >> [v2:172.21.15.98:6834/502938874,v1:172.21.15.98:6835/502938874] conn(0x558dbc788030 0x558dbc788410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:01.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f6206f000 4 client.7659 mds_command: new command op to 7524 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T08:45:01.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f6206f000 1 -- 172.21.15.98:0/2900254928 --> [v2:172.21.15.98:6834/502938874,v1:172.21.15.98:6835/502938874] -- command(tid 0: {"prefix": "status"}) -- 0x558dbc5673a0 con 0x558dbc788030 2024-09-19T08:45:01.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f57fff640 1 --2- 172.21.15.98:0/2900254928 >> [v2:172.21.15.98:6834/502938874,v1:172.21.15.98:6835/502938874] conn(0x558dbc788030 0x558dbc788410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:01.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f57fff640 1 --2- 172.21.15.98:0/2900254928 >> [v2:172.21.15.98:6834/502938874,v1:172.21.15.98:6835/502938874] conn(0x558dbc788030 0x558dbc788410 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e634a00e79216df6 server_cookie=322054cc88bddf54 in_seq=0 out_seq=0 2024-09-19T08:45:01.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f377fe640 10 client.7659 ms_handle_connect on v2:172.21.15.98:6834/502938874 2024-09-19T08:45:01.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f377fe640 1 -- 172.21.15.98:0/2900254928 <== mds.0 v2:172.21.15.98:6834/502938874 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x558dbc5673a0 con 0x558dbc788030 2024-09-19T08:45:01.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f377fe640 10 client.7659 handle_command_reply: tid=0 2024-09-19T08:45:01.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f6206f000 1 client.7659 shutdown 2024-09-19T08:45:01.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f6206f000 1 -- 172.21.15.98:0/2900254928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dbc7460f0 msgr2=0x558dbc7cc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:01.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f6206f000 1 --2- 172.21.15.98:0/2900254928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dbc7460f0 0x558dbc7cc980 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f2f480f0790 tx=0x7f2f48000bd0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f6206f000 1 -- 172.21.15.98:0/2900254928 shutdown_connections 2024-09-19T08:45:01.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f6206f000 1 --2- 172.21.15.98:0/2900254928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558dbc759690 0x558dbc7cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f6206f000 1 --2- 172.21.15.98:0/2900254928 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x558dbc758d70 0x558dbc7ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f6206f000 1 --2- 172.21.15.98:0/2900254928 >> [v2:172.21.15.98:6834/502938874,v1:172.21.15.98:6835/502938874] conn(0x558dbc788030 0x558dbc788410 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f6206f000 1 --2- 172.21.15.98:0/2900254928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x558dbc7460f0 0x558dbc7cc980 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f377fe640 0 client.7659 ms_handle_reset on v2:172.21.15.98:6834/502938874 2024-09-19T08:45:01.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f6206f000 1 -- 172.21.15.98:0/2900254928 >> 172.21.15.98:0/2900254928 conn(0x558dbc70f6a0 msgr2=0x558dbc756df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:01.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f6206f000 1 -- 172.21.15.98:0/2900254928 shutdown_connections 2024-09-19T08:45:01.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f6206f000 1 -- 172.21.15.98:0/2900254928 wait complete. 2024-09-19T08:45:01.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f6206f000 20 client.7659 trim_cache size 0 max 16384 2024-09-19T08:45:01.326 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:45:01.326 INFO:teuthology.orchestra.run.smithi098.stdout: "cluster_fsid": "552bf0ee-1f4f-4644-ba7c-d0931e55a451", 2024-09-19T08:45:01.326 INFO:teuthology.orchestra.run.smithi098.stdout: "whoami": 0, 2024-09-19T08:45:01.326 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 7524, 2024-09-19T08:45:01.326 INFO:teuthology.orchestra.run.smithi098.stdout: "want_state": "up:active", 2024-09-19T08:45:01.326 INFO:teuthology.orchestra.run.smithi098.stdout: "state": "up:active", 2024-09-19T08:45:01.326 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "cephfs", 2024-09-19T08:45:01.326 INFO:teuthology.orchestra.run.smithi098.stdout: "rank_uptime": 4.7018620200000001, 2024-09-19T08:45:01.327 INFO:teuthology.orchestra.run.smithi098.stdout: "mdsmap_epoch": 67, 2024-09-19T08:45:01.327 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch": 118, 2024-09-19T08:45:01.327 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch_barrier": 118, 2024-09-19T08:45:01.327 INFO:teuthology.orchestra.run.smithi098.stdout: "uptime": 9.7577674289999994 2024-09-19T08:45:01.327 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:45:01.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f5efb3640 1 -- 172.21.15.98:0/1370634239 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2f44067ce0 msgr2=0x7f2f4406a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:01.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f5efb3640 1 --2- 172.21.15.98:0/1370634239 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2f44067ce0 0x7f2f4406a1a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c000f70 tx=0x7f2f4c007a90 comp rx=0 tx=0).stop 2024-09-19T08:45:01.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f5efb3640 1 -- 172.21.15.98:0/1370634239 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f5806c3a0 msgr2=0x7f2f581521b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:01.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.315+0000 7f2f5efb3640 1 --2- 172.21.15.98:0/1370634239 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f5806c3a0 0x7f2f581521b0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f2f480af040 tx=0x7f2f4801ac00 comp rx=0 tx=0).stop 2024-09-19T08:45:01.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.319+0000 7f2f5efb3640 1 -- 172.21.15.98:0/1370634239 shutdown_connections 2024-09-19T08:45:01.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.319+0000 7f2f5efb3640 1 --2- 172.21.15.98:0/1370634239 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2f44067ce0 0x7f2f4406a1a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.319+0000 7f2f5efb3640 1 --2- 172.21.15.98:0/1370634239 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f58158fd0 0x7f2f58150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.319+0000 7f2f5efb3640 1 --2- 172.21.15.98:0/1370634239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f580b4360 0x7f2f58165890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.319+0000 7f2f5efb3640 1 --2- 172.21.15.98:0/1370634239 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f5806c3a0 0x7f2f581521b0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.319+0000 7f2f5efb3640 1 -- 172.21.15.98:0/1370634239 >> 172.21.15.98:0/1370634239 conn(0x7f2f580b2010 msgr2=0x7f2f580b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:01.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.319+0000 7f2f5efb3640 1 -- 172.21.15.98:0/1370634239 shutdown_connections 2024-09-19T08:45:01.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.319+0000 7f2f5efb3640 1 -- 172.21.15.98:0/1370634239 wait complete. 2024-09-19T08:45:01.336 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T08:45:01.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.435+0000 7fdc7ed87640 1 Processor -- start 2024-09-19T08:45:01.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.435+0000 7fdc7ed87640 1 -- start start 2024-09-19T08:45:01.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.439+0000 7fdc7ed87640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fdc780d1d70 0x7fdc780d1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:01.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.439+0000 7fdc7ed87640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc780b8060 con 0x7fdc78147530 2024-09-19T08:45:01.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.439+0000 7fdc7ed87640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc780b8340 con 0x7fdc780b6200 2024-09-19T08:45:01.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.439+0000 7fdc7ed87640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fdc780d2230 con 0x7fdc780d1d70 2024-09-19T08:45:01.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.439+0000 7fdc7cb23640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fdc78147530 0x7fdc7815ec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52610/0 (socket says 172.21.15.98:52610) 2024-09-19T08:45:01.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.439+0000 7fdc7cb23640 1 -- 172.21.15.98:0/2679072624 learned_addr learned my addr 172.21.15.98:0/2679072624 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:01.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.439+0000 7fdc6f7fe640 1 -- 172.21.15.98:0/2679072624 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 198013980 0 0) 0x7fdc780b8060 con 0x7fdc78147530 2024-09-19T08:45:01.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.439+0000 7fdc6f7fe640 1 -- 172.21.15.98:0/2679072624 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc60003530 con 0x7fdc78147530 2024-09-19T08:45:01.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.439+0000 7fdc6ffff640 1 --2- 172.21.15.98:0/2679072624 >> v2:172.21.15.139:3300/0 conn(0x7fdc780d1d70 0x7fdc780d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:01.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.439+0000 7fdc6f7fe640 1 -- 172.21.15.98:0/2679072624 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3498886589 0 0) 0x7fdc60003530 con 0x7fdc78147530 2024-09-19T08:45:01.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.439+0000 7fdc6f7fe640 1 -- 172.21.15.98:0/2679072624 >> v2:172.21.15.139:3300/0 conn(0x7fdc780d1d70 msgr2=0x7fdc780d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:01.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.439+0000 7fdc6f7fe640 1 --2- 172.21.15.98:0/2679072624 >> v2:172.21.15.139:3300/0 conn(0x7fdc780d1d70 0x7fdc780d1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.439+0000 7fdc6f7fe640 1 -- 172.21.15.98:0/2679072624 >> v1:172.21.15.139:6789/0 conn(0x7fdc780b6200 legacy=0x7fdc780b8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:01.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.439+0000 7fdc6f7fe640 1 -- 172.21.15.98:0/2679072624 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc780b8620 con 0x7fdc78147530 2024-09-19T08:45:01.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.439+0000 7fdc6f7fe640 1 -- 172.21.15.98:0/2679072624 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fdc74001680 con 0x7fdc78147530 2024-09-19T08:45:01.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.439+0000 7fdc6ffff640 1 --2- 172.21.15.98:0/2679072624 >> v2:172.21.15.139:3300/0 conn(0x7fdc780d1d70 0x7fdc780d1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:01.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.439+0000 7fdc6f7fe640 1 -- 172.21.15.98:0/2679072624 >> v1:172.21.15.98:6789/0 conn(0x7fdc78147530 legacy=0x7fdc7815ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:01.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.439+0000 7fdc6f7fe640 1 --2- 172.21.15.98:0/2679072624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc600039c0 0x7fdc60003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:01.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.439+0000 7fdc6f7fe640 1 --2- 172.21.15.98:0/2679072624 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc60004810 0x7fdc60004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:01.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.439+0000 7fdc6f7fe640 1 --2- 172.21.15.98:0/2679072624 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc600051d0 0x7fdc6000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:01.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.439+0000 7fdc6f7fe640 1 -- 172.21.15.98:0/2679072624 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdc780d2230 con 0x7fdc600039c0 2024-09-19T08:45:01.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.439+0000 7fdc6f7fe640 1 -- 172.21.15.98:0/2679072624 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6000c120 con 0x7fdc60004810 2024-09-19T08:45:01.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.439+0000 7fdc6f7fe640 1 -- 172.21.15.98:0/2679072624 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdc6000c320 con 0x7fdc600051d0 2024-09-19T08:45:01.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.439+0000 7fdc7cb23640 1 --2- 172.21.15.98:0/2679072624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc600039c0 0x7fdc60003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:01.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.439+0000 7fdc6ffff640 1 --2- 172.21.15.98:0/2679072624 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc60004810 0x7fdc60004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:01.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.439+0000 7fdc7d324640 1 --2- 172.21.15.98:0/2679072624 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc600051d0 0x7fdc6000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:01.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.439+0000 7fdc7cb23640 1 -- 172.21.15.98:0/2679072624 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc600051d0 msgr2=0x7fdc6000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:01.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.439+0000 7fdc7cb23640 1 --2- 172.21.15.98:0/2679072624 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc600051d0 0x7fdc6000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.439+0000 7fdc7cb23640 1 -- 172.21.15.98:0/2679072624 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc60004810 msgr2=0x7fdc60004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:01.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.439+0000 7fdc7cb23640 1 --2- 172.21.15.98:0/2679072624 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc60004810 0x7fdc60004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.439+0000 7fdc7cb23640 1 -- 172.21.15.98:0/2679072624 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc60003530 con 0x7fdc600039c0 2024-09-19T08:45:01.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.439+0000 7fdc7d324640 1 -- 172.21.15.98:0/2679072624 reap_dead start 2024-09-19T08:45:01.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.439+0000 7fdc7cb23640 1 --2- 172.21.15.98:0/2679072624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc600039c0 0x7fdc60003de0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fdc7400c1c0 tx=0x7fdc7401cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ad01536db343a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:01.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.443+0000 7fdc6f7fe640 1 -- 172.21.15.98:0/2679072624 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc74007d00 con 0x7fdc600039c0 2024-09-19T08:45:01.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.443+0000 7fdc6f7fe640 1 -- 172.21.15.98:0/2679072624 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc740a9020 con 0x7fdc600039c0 2024-09-19T08:45:01.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.443+0000 7fdc6f7fe640 1 -- 172.21.15.98:0/2679072624 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc74005ae0 con 0x7fdc600039c0 2024-09-19T08:45:01.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.443+0000 7fdc7ed87640 1 -- 172.21.15.98:0/2679072624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc600039c0 msgr2=0x7fdc60003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:01.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.443+0000 7fdc7ed87640 1 --2- 172.21.15.98:0/2679072624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc600039c0 0x7fdc60003de0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fdc7400c1c0 tx=0x7fdc7401cc30 comp rx=0 tx=0).stop 2024-09-19T08:45:01.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.443+0000 7fdc7ed87640 1 -- 172.21.15.98:0/2679072624 shutdown_connections 2024-09-19T08:45:01.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.443+0000 7fdc7ed87640 1 --2- 172.21.15.98:0/2679072624 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc600039c0 0x7fdc60003de0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.443+0000 7fdc7ed87640 1 -- 172.21.15.98:0/2679072624 >> 172.21.15.98:0/2679072624 conn(0x7fdc78075070 msgr2=0x7fdc780722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:01.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.443+0000 7fdc7ed87640 1 -- 172.21.15.98:0/2679072624 shutdown_connections 2024-09-19T08:45:01.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.443+0000 7fdc7ed87640 1 -- 172.21.15.98:0/2679072624 wait complete. 2024-09-19T08:45:01.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.443+0000 7fdc7ed87640 1 Processor -- start 2024-09-19T08:45:01.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.443+0000 7fdc7ed87640 1 -- start start 2024-09-19T08:45:01.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.443+0000 7fdc7ed87640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc600039c0 0x7fdc7815d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:01.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.443+0000 7fdc7ed87640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc7815dd80 0x7fdc780d2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:01.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.443+0000 7fdc7ed87640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc780d2a50 0x7fdc78145f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:01.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.443+0000 7fdc7ed87640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdc780b69f0 con 0x7fdc780d2a50 2024-09-19T08:45:01.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.443+0000 7fdc7ed87640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdc780d1920 con 0x7fdc7815dd80 2024-09-19T08:45:01.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.443+0000 7fdc7ed87640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdc780b6b70 con 0x7fdc600039c0 2024-09-19T08:45:01.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.443+0000 7fdc7d324640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc780d2a50 0x7fdc78145f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:01.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.443+0000 7fdc7d324640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc780d2a50 0x7fdc78145f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47924/0 (socket says 172.21.15.98:47924) 2024-09-19T08:45:01.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.443+0000 7fdc7d324640 1 -- 172.21.15.98:0/4095967025 learned_addr learned my addr 172.21.15.98:0/4095967025 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:01.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.443+0000 7fdc6ffff640 1 --2- 172.21.15.98:0/4095967025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc7815dd80 0x7fdc780d2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:01.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.443+0000 7fdc7cb23640 1 --2- 172.21.15.98:0/4095967025 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc600039c0 0x7fdc7815d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:01.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.443+0000 7fdc7d324640 1 -- 172.21.15.98:0/4095967025 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc600039c0 msgr2=0x7fdc7815d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:01.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.443+0000 7fdc7d324640 1 --2- 172.21.15.98:0/4095967025 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc600039c0 0x7fdc7815d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.443+0000 7fdc7d324640 1 -- 172.21.15.98:0/4095967025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc7815dd80 msgr2=0x7fdc780d2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:01.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.443+0000 7fdc7d324640 1 --2- 172.21.15.98:0/4095967025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc7815dd80 0x7fdc780d2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.443+0000 7fdc7d324640 1 -- 172.21.15.98:0/4095967025 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc780d34b0 con 0x7fdc780d2a50 2024-09-19T08:45:01.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.443+0000 7fdc7d324640 1 --2- 172.21.15.98:0/4095967025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc780d2a50 0x7fdc78145f80 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fdc780c97a0 tx=0x7fdc7000db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:01.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.443+0000 7fdc6d7fa640 1 -- 172.21.15.98:0/4095967025 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc70018070 con 0x7fdc780d2a50 2024-09-19T08:45:01.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.443+0000 7fdc6d7fa640 1 -- 172.21.15.98:0/4095967025 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc70003b80 con 0x7fdc780d2a50 2024-09-19T08:45:01.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.443+0000 7fdc7ed87640 1 -- 172.21.15.98:0/4095967025 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc780d2ef0 con 0x7fdc780d2a50 2024-09-19T08:45:01.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.443+0000 7fdc6d7fa640 1 -- 172.21.15.98:0/4095967025 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc70005070 con 0x7fdc780d2a50 2024-09-19T08:45:01.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.443+0000 7fdc7ed87640 1 -- 172.21.15.98:0/4095967025 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc7813b9f0 con 0x7fdc780d2a50 2024-09-19T08:45:01.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.447+0000 7fdc7ed87640 1 -- 172.21.15.98:0/4095967025 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc780b96c0 con 0x7fdc780d2a50 2024-09-19T08:45:01.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.447+0000 7fdc6d7fa640 1 -- 172.21.15.98:0/4095967025 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7fdc7001b070 con 0x7fdc780d2a50 2024-09-19T08:45:01.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.447+0000 7fdc6d7fa640 1 --2- 172.21.15.98:0/4095967025 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fdc64067cb0 0x7fdc6406a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:01.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.447+0000 7fdc6d7fa640 1 -- 172.21.15.98:0/4095967025 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5737+0+0 (secure 0 0 0) 0x7fdc70085ab0 con 0x7fdc780d2a50 2024-09-19T08:45:01.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.447+0000 7fdc7cb23640 1 --2- 172.21.15.98:0/4095967025 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fdc64067cb0 0x7fdc6406a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:01.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.447+0000 7fdc7cb23640 1 --2- 172.21.15.98:0/4095967025 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fdc64067cb0 0x7fdc6406a170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdc740aa000 tx=0x7fdc780b8230 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:01.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.451+0000 7fdc6d7fa640 1 -- 172.21.15.98:0/4095967025 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fdc70055fa0 con 0x7fdc780d2a50 2024-09-19T08:45:01.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.639+0000 7fdc7ed87640 1 -- 172.21.15.98:0/4095967025 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fdc78144ba0 con 0x7fdc780d2a50 2024-09-19T08:45:01.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.639+0000 7fdc6d7fa640 1 -- 172.21.15.98:0/4095967025 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) ==== 53+0+73 (secure 0 0 0) 0x7fdc78144ba0 con 0x7fdc780d2a50 2024-09-19T08:45:01.642 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T08:45:01.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.639+0000 7fdc7ed87640 1 -- 172.21.15.98:0/4095967025 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fdc64067cb0 msgr2=0x7fdc6406a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:01.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.639+0000 7fdc7ed87640 1 --2- 172.21.15.98:0/4095967025 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fdc64067cb0 0x7fdc6406a170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdc740aa000 tx=0x7fdc780b8230 comp rx=0 tx=0).stop 2024-09-19T08:45:01.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.639+0000 7fdc7ed87640 1 -- 172.21.15.98:0/4095967025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc780d2a50 msgr2=0x7fdc78145f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:01.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.639+0000 7fdc7ed87640 1 --2- 172.21.15.98:0/4095967025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc780d2a50 0x7fdc78145f80 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fdc780c97a0 tx=0x7fdc7000db80 comp rx=0 tx=0).stop 2024-09-19T08:45:01.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.639+0000 7fdc7ed87640 1 -- 172.21.15.98:0/4095967025 shutdown_connections 2024-09-19T08:45:01.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.639+0000 7fdc7ed87640 1 --2- 172.21.15.98:0/4095967025 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fdc64067cb0 0x7fdc6406a170 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.639+0000 7fdc7ed87640 1 --2- 172.21.15.98:0/4095967025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc780d2a50 0x7fdc78145f80 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.639+0000 7fdc7ed87640 1 --2- 172.21.15.98:0/4095967025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc7815dd80 0x7fdc780d2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.639+0000 7fdc7ed87640 1 --2- 172.21.15.98:0/4095967025 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc600039c0 0x7fdc7815d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:01.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.639+0000 7fdc7ed87640 1 -- 172.21.15.98:0/4095967025 >> 172.21.15.98:0/4095967025 conn(0x7fdc78075070 msgr2=0x7fdc78145340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:01.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.639+0000 7fdc7ed87640 1 -- 172.21.15.98:0/4095967025 shutdown_connections 2024-09-19T08:45:01.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:01.639+0000 7fdc7ed87640 1 -- 172.21.15.98:0/4095967025 wait complete. 2024-09-19T08:45:01.660 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T08:45:01.660 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T08:45:01.660 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi098.front.sepia.ceph.com 2024-09-19T08:45:01.660 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T08:45:01.660 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T08:45:01.660 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T08:45:01.660 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T08:45:01.660 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T08:45:01.660 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:45:01.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:01.661 DEBUG:teuthology.orchestra.run.smithi098:> ip addr 2024-09-19T08:45:01.666 INFO:teuthology.orchestra.run.smithi098.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T08:45:01.666 INFO:teuthology.orchestra.run.smithi098.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T08:45:01.666 INFO:teuthology.orchestra.run.smithi098.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T08:45:01.666 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:45:01.666 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 ::1/128 scope host 2024-09-19T08:45:01.666 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:45:01.667 INFO:teuthology.orchestra.run.smithi098.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:45:01.667 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b8 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:45:01.667 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f0 2024-09-19T08:45:01.667 INFO:teuthology.orchestra.run.smithi098.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:45:01.667 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b9 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:45:01.667 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f1 2024-09-19T08:45:01.667 INFO:teuthology.orchestra.run.smithi098.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:45:01.667 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6a brd ff:ff:ff:ff:ff:ff 2024-09-19T08:45:01.667 INFO:teuthology.orchestra.run.smithi098.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T08:45:01.667 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6b brd ff:ff:ff:ff:ff:ff 2024-09-19T08:45:01.667 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.21.15.98/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T08:45:01.667 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft 42107sec preferred_lft 42107sec 2024-09-19T08:45:01.667 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::ec4:7aff:fe8f:d26b/64 scope link 2024-09-19T08:45:01.667 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:45:01.668 INFO:teuthology.orchestra.run.smithi098.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T08:45:01.668 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 02:42:98:dd:32:2d brd ff:ff:ff:ff:ff:ff 2024-09-19T08:45:01.668 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T08:45:01.668 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:45:01.668 INFO:teuthology.orchestra.run.smithi098.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T08:45:01.668 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-19T08:45:01.668 INFO:teuthology.orchestra.run.smithi098.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T08:45:01.668 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:45:01.668 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-19T08:45:01.668 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:45:01.668 INFO:teuthology.orchestra.run.smithi098.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:45:01.668 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:45:01.668 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-19T08:45:01.669 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:45:01.669 INFO:teuthology.orchestra.run.smithi098.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:45:01.669 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:45:01.669 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-19T08:45:01.669 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:45:01.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:01.669 DEBUG:teuthology.orchestra.run.smithi098:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T08:45:01.718 INFO:teuthology.orchestra.run.smithi098.stdout:1 2024-09-19T08:45:01.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:01.719 DEBUG:teuthology.orchestra.run.smithi098:> route 2024-09-19T08:45:01.968 INFO:teuthology.orchestra.run.smithi098.stdout:Kernel IP routing table 2024-09-19T08:45:01.968 INFO:teuthology.orchestra.run.smithi098.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T08:45:01.969 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T08:45:01.969 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T08:45:01.969 INFO:teuthology.orchestra.run.smithi098.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T08:45:01.969 INFO:teuthology.orchestra.run.smithi098.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T08:45:01.969 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T08:45:01.969 INFO:teuthology.orchestra.run.smithi098.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:45:01.969 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:45:01.969 INFO:teuthology.orchestra.run.smithi098.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:45:01.969 INFO:teuthology.orchestra.run.smithi098.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T08:45:01.969 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:45:01.970 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-09-19T08:45:01.970 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T08:45:01.970 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T08:45:01.970 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T08:45:01.970 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-09-19T08:45:02.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:45:02.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:45:02.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:02.052 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2024-09-19T08:45:02.101 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-19T08:45:02.101 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T08:45:02.101 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:45:02.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:02.101 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set brx.0 up 2024-09-19T08:45:02.157 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-19T08:45:02.157 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:45:02.157 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T08:45:02.204 INFO:teuthology.orchestra.run.smithi098.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:45:02.204 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:45:02.204 DEBUG:teuthology.orchestra.run.smithi098:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T08:45:02.252 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:45:02.310 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:45:02.357 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:45:02.357 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:45:02.357 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:45:02.357 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:45:02.357 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:45:02.357 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:45:02.357 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:45:02.358 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:45:02.358 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:45:02.358 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:45:02.358 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:45:02.358 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:45:02.358 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:45:02.358 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:45:02.358 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:45:02.358 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:45:02.358 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:45:02.358 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:45:02.359 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:45:02.359 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:45:02.359 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:45:02.359 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:45:02.359 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:45:02.359 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:45:02.359 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:45:02.359 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:45:02.359 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:45:02.359 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:45:02.359 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:45:02.359 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:45:02.360 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:45:02.360 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:45:02.360 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:45:02.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:02.361 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:45:02.408 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T08:45:02.409 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-19T08:45:02.458 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:45:02.473 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:45:02.521 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:45:02.521 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:45:02.521 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:45:02.521 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:45:02.521 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:45:02.521 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:45:02.521 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:45:02.521 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:45:02.522 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:45:02.522 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:45:02.522 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:45:02.522 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:45:02.522 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:45:02.522 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:45:02.522 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:45:02.522 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:45:02.522 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:45:02.522 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:45:02.522 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:45:02.522 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:45:02.522 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:45:02.522 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:45:02.522 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:45:02.523 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:45:02.523 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:45:02.523 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:45:02.523 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:45:02.523 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:45:02.523 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:45:02.523 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:45:02.523 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:45:02.523 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:45:02.523 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:45:02.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:02.524 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:45:02.531 INFO:tasks.ceph.mds.c.smithi098.stderr:2024-09-19T08:45:02.527+0000 7f02fc80f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T08:45:02.533 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:2024-09-19T08:45:02.531+0000 7f3ad0c0c580 -1 init, newargv = 0x564a1abc0fc0 newargc=15 2024-09-19T08:45:02.533 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:ceph-fuse[48118]: starting ceph client 2024-09-19T08:45:02.539 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:ceph-fuse[48118]: starting fuse 2024-09-19T08:45:03.637 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:45:03.654 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:45:03.703 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:45:03.704 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:45:03.704 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:45:03.704 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:45:03.704 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:45:03.704 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:45:03.704 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:45:03.704 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:45:03.704 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:45:03.704 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:45:03.704 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:45:03.705 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:45:03.705 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:45:03.705 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:45:03.705 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:45:03.705 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:45:03.705 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:45:03.705 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:45:03.705 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:45:03.705 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:45:03.705 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:45:03.705 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:45:03.705 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:45:03.706 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:45:03.706 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:45:03.706 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:45:03.706 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:45:03.706 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:45:03.706 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:45:03.706 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:45:03.706 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:45:03.706 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:45:03.706 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:45:03.706 INFO:teuthology.orchestra.run.smithi098.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-19T08:45:03.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:03.707 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:45:03.757 INFO:teuthology.orchestra.run.smithi098.stdout:45 2024-09-19T08:45:03.757 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-19T08:45:03.757 DEBUG:teuthology.orchestra.run.smithi098:> sudo stdin-killer -- python3 -c ' 2024-09-19T08:45:03.758 DEBUG:teuthology.orchestra.run.smithi098:> import glob 2024-09-19T08:45:03.758 DEBUG:teuthology.orchestra.run.smithi098:> import re 2024-09-19T08:45:03.758 DEBUG:teuthology.orchestra.run.smithi098:> import os 2024-09-19T08:45:03.758 DEBUG:teuthology.orchestra.run.smithi098:> import subprocess 2024-09-19T08:45:03.758 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:45:03.758 DEBUG:teuthology.orchestra.run.smithi098:> def _find_admin_socket(client_name): 2024-09-19T08:45:03.758 DEBUG:teuthology.orchestra.run.smithi098:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T08:45:03.758 DEBUG:teuthology.orchestra.run.smithi098:> files = glob.glob(asok_path) 2024-09-19T08:45:03.758 DEBUG:teuthology.orchestra.run.smithi098:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T08:45:03.758 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:45:03.758 DEBUG:teuthology.orchestra.run.smithi098:> # Given a non-glob path, it better be there 2024-09-19T08:45:03.758 DEBUG:teuthology.orchestra.run.smithi098:> if "*" not in asok_path: 2024-09-19T08:45:03.758 DEBUG:teuthology.orchestra.run.smithi098:> assert(len(files) == 1) 2024-09-19T08:45:03.758 DEBUG:teuthology.orchestra.run.smithi098:> return files[0] 2024-09-19T08:45:03.758 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:45:03.759 DEBUG:teuthology.orchestra.run.smithi098:> for f in files: 2024-09-19T08:45:03.759 DEBUG:teuthology.orchestra.run.smithi098:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T08:45:03.759 DEBUG:teuthology.orchestra.run.smithi098:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T08:45:03.759 DEBUG:teuthology.orchestra.run.smithi098:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T08:45:03.759 DEBUG:teuthology.orchestra.run.smithi098:> contents = proc_f.read() 2024-09-19T08:45:03.759 DEBUG:teuthology.orchestra.run.smithi098:> if mountpoint in contents: 2024-09-19T08:45:03.759 DEBUG:teuthology.orchestra.run.smithi098:> return f 2024-09-19T08:45:03.759 DEBUG:teuthology.orchestra.run.smithi098:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T08:45:03.759 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:45:03.759 DEBUG:teuthology.orchestra.run.smithi098:> print(_find_admin_socket("client.0")) 2024-09-19T08:45:03.759 DEBUG:teuthology.orchestra.run.smithi098:> ' 2024-09-19T08:45:03.881 INFO:teuthology.orchestra.run.smithi098.stdout:/var/run/ceph/ceph-client.0.48118.asok 2024-09-19T08:45:03.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:45:03.897 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.48118.asok 2024-09-19T08:45:03.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:03.897 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.48118.asok status 2024-09-19T08:45:04.046 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:45:04.046 INFO:teuthology.orchestra.run.smithi098.stdout: "metadata": { 2024-09-19T08:45:04.046 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T08:45:04.046 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T08:45:04.047 INFO:teuthology.orchestra.run.smithi098.stdout: "entity_id": "0", 2024-09-19T08:45:04.047 INFO:teuthology.orchestra.run.smithi098.stdout: "hostname": "smithi098", 2024-09-19T08:45:04.047 INFO:teuthology.orchestra.run.smithi098.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T08:45:04.047 INFO:teuthology.orchestra.run.smithi098.stdout: "pid": "48118", 2024-09-19T08:45:04.047 INFO:teuthology.orchestra.run.smithi098.stdout: "root": "/" 2024-09-19T08:45:04.047 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:45:04.047 INFO:teuthology.orchestra.run.smithi098.stdout: "dentry_count": 0, 2024-09-19T08:45:04.047 INFO:teuthology.orchestra.run.smithi098.stdout: "dentry_pinned_count": 0, 2024-09-19T08:45:04.047 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 7671, 2024-09-19T08:45:04.047 INFO:teuthology.orchestra.run.smithi098.stdout: "inst": { 2024-09-19T08:45:04.047 INFO:teuthology.orchestra.run.smithi098.stdout: "name": { 2024-09-19T08:45:04.047 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "client", 2024-09-19T08:45:04.047 INFO:teuthology.orchestra.run.smithi098.stdout: "num": 7671 2024-09-19T08:45:04.048 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:45:04.048 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": { 2024-09-19T08:45:04.048 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v1", 2024-09-19T08:45:04.048 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:45:04.048 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 3731606227 2024-09-19T08:45:04.048 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:45:04.048 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:45:04.048 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": { 2024-09-19T08:45:04.048 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v1", 2024-09-19T08:45:04.048 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:45:04.048 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 3731606227 2024-09-19T08:45:04.048 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:45:04.048 INFO:teuthology.orchestra.run.smithi098.stdout: "inst_str": "client.7671 192.168.0.1:0/3731606227", 2024-09-19T08:45:04.048 INFO:teuthology.orchestra.run.smithi098.stdout: "addr_str": "192.168.0.1:0/3731606227", 2024-09-19T08:45:04.049 INFO:teuthology.orchestra.run.smithi098.stdout: "inode_count": 1, 2024-09-19T08:45:04.049 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_epoch": 67, 2024-09-19T08:45:04.049 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_epoch": 118, 2024-09-19T08:45:04.049 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_epoch_barrier": 0, 2024-09-19T08:45:04.049 INFO:teuthology.orchestra.run.smithi098.stdout: "blocklisted": false, 2024-09-19T08:45:04.049 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "cephfs" 2024-09-19T08:45:04.049 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:45:04.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:04.060 DEBUG:teuthology.orchestra.run.smithi098:> stat --file-system '--printf=%T 2024-09-19T08:45:04.060 DEBUG:teuthology.orchestra.run.smithi098:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T08:45:04.066 INFO:teuthology.orchestra.run.smithi098.stdout:fuseblk 2024-09-19T08:45:04.066 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T08:45:04.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:04.066 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T08:45:04.121 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T08:45:04.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.287+0000 7f2ad3800640 1 Processor -- start 2024-09-19T08:45:04.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.287+0000 7f2ad3800640 1 -- start start 2024-09-19T08:45:04.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.287+0000 7f2ad3800640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2acc15c550 0x7f2acc06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:04.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.287+0000 7f2ad3800640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2acc06c1d0 0x7f2acc0cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:04.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.287+0000 7f2ad3800640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2acc0cf900 0x7f2acc0ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:04.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.287+0000 7f2ad3800640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f2acc06c640 con 0x7f2acc06c1d0 2024-09-19T08:45:04.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.287+0000 7f2ad3800640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f2acc06d310 con 0x7f2acc15c550 2024-09-19T08:45:04.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.287+0000 7f2ad3800640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2acc06d5f0 con 0x7f2acc0cf900 2024-09-19T08:45:04.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.287+0000 7f2ad0d9b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2acc06c1d0 0x7f2acc0cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:04.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.287+0000 7f2ad0d9b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2acc06c1d0 0x7f2acc0cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47936/0 (socket says 172.21.15.98:47936) 2024-09-19T08:45:04.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.287+0000 7f2ad0d9b640 1 -- 172.21.15.98:0/3165538211 learned_addr learned my addr 172.21.15.98:0/3165538211 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:04.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.287+0000 7f2ad159c640 1 --2- 172.21.15.98:0/3165538211 >> v2:172.21.15.139:3300/0 conn(0x7f2acc15c550 0x7f2acc06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:04.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.287+0000 7f2ad1d9d640 1 --2- 172.21.15.98:0/3165538211 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2acc0cf900 0x7f2acc0ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:04.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.287+0000 7f2ad0d9b640 1 -- 172.21.15.98:0/3165538211 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2acc0cf900 msgr2=0x7f2acc0ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:04.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.287+0000 7f2ad0d9b640 1 --2- 172.21.15.98:0/3165538211 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2acc0cf900 0x7f2acc0ce1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:04.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.287+0000 7f2ad0d9b640 1 -- 172.21.15.98:0/3165538211 >> v2:172.21.15.139:3300/0 conn(0x7f2acc15c550 msgr2=0x7f2acc06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:04.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.287+0000 7f2ad0d9b640 1 --2- 172.21.15.98:0/3165538211 >> v2:172.21.15.139:3300/0 conn(0x7f2acc15c550 0x7f2acc06bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:04.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.287+0000 7f2ad0d9b640 1 -- 172.21.15.98:0/3165538211 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2acc06d8d0 con 0x7f2acc06c1d0 2024-09-19T08:45:04.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.287+0000 7f2ad0d9b640 1 --2- 172.21.15.98:0/3165538211 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2acc06c1d0 0x7f2acc0cf3c0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f2acc0b56b0 tx=0x7f2abc01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=132ee59c0b0d886c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:04.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.291+0000 7f2acbfff640 1 -- 172.21.15.98:0/3165538211 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2abc0b0070 con 0x7f2acc06c1d0 2024-09-19T08:45:04.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.291+0000 7f2acbfff640 1 -- 172.21.15.98:0/3165538211 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2abc0ab600 con 0x7f2acc06c1d0 2024-09-19T08:45:04.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.291+0000 7f2acbfff640 1 -- 172.21.15.98:0/3165538211 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2abc0ab9d0 con 0x7f2acc06c1d0 2024-09-19T08:45:04.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.291+0000 7f2ad3800640 1 -- 172.21.15.98:0/3165538211 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2acc06c1d0 msgr2=0x7f2acc0cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:04.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.291+0000 7f2ad3800640 1 --2- 172.21.15.98:0/3165538211 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2acc06c1d0 0x7f2acc0cf3c0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f2acc0b56b0 tx=0x7f2abc01c260 comp rx=0 tx=0).stop 2024-09-19T08:45:04.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.291+0000 7f2ad3800640 1 -- 172.21.15.98:0/3165538211 shutdown_connections 2024-09-19T08:45:04.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.291+0000 7f2ad3800640 1 --2- 172.21.15.98:0/3165538211 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2acc06c1d0 0x7f2acc0cf3c0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:04.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.291+0000 7f2ad3800640 1 --2- 172.21.15.98:0/3165538211 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2acc0cf900 0x7f2acc0ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:04.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.291+0000 7f2ad3800640 1 --2- 172.21.15.98:0/3165538211 >> v2:172.21.15.139:3300/0 conn(0x7f2acc15c550 0x7f2acc06bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:04.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.291+0000 7f2ad3800640 1 -- 172.21.15.98:0/3165538211 >> 172.21.15.98:0/3165538211 conn(0x7f2acc0b2010 msgr2=0x7f2acc0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:04.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.291+0000 7f2ad3800640 1 -- 172.21.15.98:0/3165538211 shutdown_connections 2024-09-19T08:45:04.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.291+0000 7f2ad3800640 1 -- 172.21.15.98:0/3165538211 wait complete. 2024-09-19T08:45:04.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.291+0000 7f2ad3800640 1 Processor -- start 2024-09-19T08:45:04.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.291+0000 7f2ad3800640 1 -- start start 2024-09-19T08:45:04.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.291+0000 7f2ad3800640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2acc06c1d0 0x7f2acc151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:04.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.291+0000 7f2ad3800640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2acc0cf900 0x7f2acc14fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:04.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.291+0000 7f2ad3800640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2acc15c550 0x7f2acc14eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:04.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.291+0000 7f2ad3800640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2acc072520 con 0x7f2acc06c1d0 2024-09-19T08:45:04.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.291+0000 7f2ad3800640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2acc0723a0 con 0x7f2acc0cf900 2024-09-19T08:45:04.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.291+0000 7f2ad3800640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2acc0726a0 con 0x7f2acc15c550 2024-09-19T08:45:04.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.291+0000 7f2ad159c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2acc06c1d0 0x7f2acc151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:04.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.291+0000 7f2ad159c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2acc06c1d0 0x7f2acc151150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47940/0 (socket says 172.21.15.98:47940) 2024-09-19T08:45:04.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.291+0000 7f2ad159c640 1 -- 172.21.15.98:0/1762918269 learned_addr learned my addr 172.21.15.98:0/1762918269 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:04.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.291+0000 7f2ad0d9b640 1 --2- 172.21.15.98:0/1762918269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2acc0cf900 0x7f2acc14fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:04.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.291+0000 7f2ad1d9d640 1 --2- 172.21.15.98:0/1762918269 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2acc15c550 0x7f2acc14eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:04.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.291+0000 7f2ad159c640 1 -- 172.21.15.98:0/1762918269 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2acc15c550 msgr2=0x7f2acc14eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:04.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.291+0000 7f2ad159c640 1 --2- 172.21.15.98:0/1762918269 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2acc15c550 0x7f2acc14eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:04.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.291+0000 7f2ad159c640 1 -- 172.21.15.98:0/1762918269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2acc0cf900 msgr2=0x7f2acc14fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:04.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.291+0000 7f2ad159c640 1 --2- 172.21.15.98:0/1762918269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2acc0cf900 0x7f2acc14fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:04.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.291+0000 7f2ad159c640 1 -- 172.21.15.98:0/1762918269 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2acc152020 con 0x7f2acc06c1d0 2024-09-19T08:45:04.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.295+0000 7f2ad1d9d640 1 --2- 172.21.15.98:0/1762918269 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2acc15c550 0x7f2acc14eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:04.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.295+0000 7f2ad159c640 1 --2- 172.21.15.98:0/1762918269 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2acc06c1d0 0x7f2acc151150 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0007e40 tx=0x7f2ac000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:04.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.295+0000 7f2ac9ffb640 1 -- 172.21.15.98:0/1762918269 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0014620 con 0x7f2acc06c1d0 2024-09-19T08:45:04.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.295+0000 7f2ac9ffb640 1 -- 172.21.15.98:0/1762918269 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ac0019070 con 0x7f2acc06c1d0 2024-09-19T08:45:04.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.295+0000 7f2ad3800640 1 -- 172.21.15.98:0/1762918269 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2acc152300 con 0x7f2acc06c1d0 2024-09-19T08:45:04.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.295+0000 7f2ac9ffb640 1 -- 172.21.15.98:0/1762918269 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0014a00 con 0x7f2acc06c1d0 2024-09-19T08:45:04.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.295+0000 7f2ad3800640 1 -- 172.21.15.98:0/1762918269 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2acc152ba0 con 0x7f2acc06c1d0 2024-09-19T08:45:04.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.295+0000 7f2ad3800640 1 -- 172.21.15.98:0/1762918269 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2acc0ce1a0 con 0x7f2acc06c1d0 2024-09-19T08:45:04.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.295+0000 7f2ac9ffb640 1 -- 172.21.15.98:0/1762918269 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f2ac0014ba0 con 0x7f2acc06c1d0 2024-09-19T08:45:04.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.295+0000 7f2ac9ffb640 1 --2- 172.21.15.98:0/1762918269 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2ab8067cb0 0x7f2ab806a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:04.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.295+0000 7f2ac9ffb640 1 -- 172.21.15.98:0/1762918269 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5795+0+0 (secure 0 0 0) 0x7f2ac0086cf0 con 0x7f2acc06c1d0 2024-09-19T08:45:04.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.295+0000 7f2ad0d9b640 1 --2- 172.21.15.98:0/1762918269 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2ab8067cb0 0x7f2ab806a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:04.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.295+0000 7f2ad0d9b640 1 --2- 172.21.15.98:0/1762918269 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2ab8067cb0 0x7f2ab806a170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2abc0ad000 tx=0x7f2abc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:04.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.299+0000 7f2ac9ffb640 1 -- 172.21.15.98:0/1762918269 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2ac00571f0 con 0x7f2acc06c1d0 2024-09-19T08:45:04.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.491+0000 7f2ad3800640 1 -- 172.21.15.98:0/1762918269 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2acc154bd0 con 0x7f2acc06c1d0 2024-09-19T08:45:04.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.495+0000 7f2ac9ffb640 1 -- 172.21.15.98:0/1762918269 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) ==== 53+0+73 (secure 0 0 0) 0x7f2ac0056af0 con 0x7f2acc06c1d0 2024-09-19T08:45:04.496 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T08:45:04.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.495+0000 7f2ad3800640 1 -- 172.21.15.98:0/1762918269 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2ab8067cb0 msgr2=0x7f2ab806a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:04.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.495+0000 7f2ad3800640 1 --2- 172.21.15.98:0/1762918269 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2ab8067cb0 0x7f2ab806a170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2abc0ad000 tx=0x7f2abc008040 comp rx=0 tx=0).stop 2024-09-19T08:45:04.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.495+0000 7f2ad3800640 1 -- 172.21.15.98:0/1762918269 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2acc06c1d0 msgr2=0x7f2acc151150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:04.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.495+0000 7f2ad3800640 1 --2- 172.21.15.98:0/1762918269 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2acc06c1d0 0x7f2acc151150 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0007e40 tx=0x7f2ac000e4b0 comp rx=0 tx=0).stop 2024-09-19T08:45:04.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.495+0000 7f2ad3800640 1 -- 172.21.15.98:0/1762918269 shutdown_connections 2024-09-19T08:45:04.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.495+0000 7f2ad3800640 1 --2- 172.21.15.98:0/1762918269 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2ab8067cb0 0x7f2ab806a170 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:04.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.495+0000 7f2ad3800640 1 --2- 172.21.15.98:0/1762918269 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2acc15c550 0x7f2acc14eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:04.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.495+0000 7f2ad3800640 1 --2- 172.21.15.98:0/1762918269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2acc0cf900 0x7f2acc14fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:04.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.495+0000 7f2ad3800640 1 --2- 172.21.15.98:0/1762918269 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2acc06c1d0 0x7f2acc151150 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:04.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.495+0000 7f2ad3800640 1 -- 172.21.15.98:0/1762918269 >> 172.21.15.98:0/1762918269 conn(0x7f2acc0b2010 msgr2=0x7f2acc0afe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:04.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.495+0000 7f2ad3800640 1 -- 172.21.15.98:0/1762918269 shutdown_connections 2024-09-19T08:45:04.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:04.495+0000 7f2ad3800640 1 -- 172.21.15.98:0/1762918269 wait complete. 2024-09-19T08:45:04.513 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T08:45:04.513 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T08:45:04.514 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2024-09-19T08:45:04.514 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-19T08:45:04.514 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-19T08:45:04.514 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T08:45:04.514 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T08:45:04.514 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T08:45:04.514 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T08:45:04.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:04.514 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2024-09-19T08:45:04.521 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T08:45:04.522 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T08:45:04.522 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T08:45:04.522 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:45:04.522 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-09-19T08:45:04.522 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:45:04.522 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:45:04.522 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:45:04.522 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2024-09-19T08:45:04.523 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:45:04.523 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:45:04.523 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2024-09-19T08:45:04.523 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:45:04.523 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:45:04.523 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T08:45:04.523 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:45:04.523 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T08:45:04.523 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 42115sec preferred_lft 42115sec 2024-09-19T08:45:04.524 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::ec4:7aff:fe88:7f59/64 scope link 2024-09-19T08:45:04.524 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:45:04.524 INFO:teuthology.orchestra.run.smithi139.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T08:45:04.524 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 02:42:74:ed:e4:60 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:45:04.524 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T08:45:04.524 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:45:04.524 INFO:teuthology.orchestra.run.smithi139.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T08:45:04.524 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-19T08:45:04.524 INFO:teuthology.orchestra.run.smithi139.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T08:45:04.525 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:45:04.525 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-19T08:45:04.525 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:45:04.525 INFO:teuthology.orchestra.run.smithi139.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:45:04.525 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:45:04.525 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-19T08:45:04.525 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:45:04.525 INFO:teuthology.orchestra.run.smithi139.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:45:04.525 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:45:04.525 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-19T08:45:04.526 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:45:04.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:04.526 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T08:45:04.574 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-09-19T08:45:04.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:04.576 DEBUG:teuthology.orchestra.run.smithi139:> route 2024-09-19T08:45:04.628 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2024-09-19T08:45:04.628 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T08:45:04.628 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T08:45:04.628 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T08:45:04.628 INFO:teuthology.orchestra.run.smithi139.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T08:45:04.628 INFO:teuthology.orchestra.run.smithi139.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T08:45:04.628 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T08:45:04.628 INFO:teuthology.orchestra.run.smithi139.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:45:04.628 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:45:04.629 INFO:teuthology.orchestra.run.smithi139.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:45:04.629 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T08:45:04.629 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:45:04.629 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-09-19T08:45:04.629 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T08:45:04.629 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T08:45:04.629 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T08:45:04.629 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-09-19T08:45:04.715 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:45:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:45:04.745 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:45:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:45:04.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:04.752 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-09-19T08:45:04.758 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-19T08:45:04.758 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-19T08:45:04.758 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T08:45:04.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:04.758 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.0 up 2024-09-19T08:45:04.812 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-19T08:45:04.812 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:45:04.812 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-19T08:45:04.860 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-19T08:45:04.860 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:45:04.860 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-19T08:45:04.908 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:45:04.966 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:45:05.013 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:45:05.013 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:45:05.013 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:45:05.013 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:45:05.013 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:45:05.013 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:45:05.013 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:45:05.013 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:45:05.013 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:45:05.013 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:45:05.013 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:45:05.013 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:45:05.014 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:45:05.014 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:45:05.014 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:45:05.014 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:45:05.014 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:45:05.014 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:45:05.014 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:45:05.014 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:45:05.014 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:45:05.014 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:45:05.014 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:45:05.014 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:45:05.014 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:45:05.014 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:45:05.015 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:45:05.015 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:45:05.015 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:45:05.015 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:45:05.015 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:45:05.015 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:45:05.015 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:45:05.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:05.016 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:45:05.064 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T08:45:05.064 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-19T08:45:05.109 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:45:05.122 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:45:05.168 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:45:05.168 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:45:05.169 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:45:05.169 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:45:05.169 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:45:05.169 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:45:05.169 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:45:05.169 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:45:05.169 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:45:05.169 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:45:05.169 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:45:05.169 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:45:05.169 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:45:05.169 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:45:05.169 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:45:05.170 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:45:05.170 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:45:05.170 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:45:05.170 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:45:05.170 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:45:05.170 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:45:05.170 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:45:05.170 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:45:05.170 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:45:05.170 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:45:05.170 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:45:05.170 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:45:05.170 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:45:05.170 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:45:05.171 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:45:05.171 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:45:05.171 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:45:05.171 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:45:05.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:05.171 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:45:05.188 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:2024-09-19T08:45:05.191+0000 7f71f382f580 -1 init, newargv = 0x559cd4e3f150 newargc=15 2024-09-19T08:45:05.188 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[36386]: starting ceph client 2024-09-19T08:45:05.194 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[36386]: starting fuse 2024-09-19T08:45:06.294 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:45:06.310 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:45:06.357 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:45:06.357 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:45:06.357 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:45:06.357 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:45:06.357 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:45:06.357 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:45:06.357 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:45:06.357 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:45:06.357 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:45:06.357 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:45:06.357 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:45:06.357 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:45:06.358 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:45:06.358 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:45:06.358 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:45:06.358 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:45:06.358 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:45:06.358 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:45:06.358 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:45:06.358 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:45:06.358 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:45:06.358 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:45:06.358 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:45:06.358 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:45:06.358 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:45:06.359 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:45:06.359 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:45:06.359 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:45:06.359 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:45:06.359 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:45:06.359 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:45:06.359 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:45:06.359 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:45:06.359 INFO:teuthology.orchestra.run.smithi139.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-19T08:45:06.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:06.360 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:45:06.407 INFO:teuthology.orchestra.run.smithi139.stdout:45 2024-09-19T08:45:06.408 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-19T08:45:06.408 DEBUG:teuthology.orchestra.run.smithi139:> sudo stdin-killer -- python3 -c ' 2024-09-19T08:45:06.408 DEBUG:teuthology.orchestra.run.smithi139:> import glob 2024-09-19T08:45:06.408 DEBUG:teuthology.orchestra.run.smithi139:> import re 2024-09-19T08:45:06.408 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-09-19T08:45:06.408 DEBUG:teuthology.orchestra.run.smithi139:> import subprocess 2024-09-19T08:45:06.408 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:45:06.408 DEBUG:teuthology.orchestra.run.smithi139:> def _find_admin_socket(client_name): 2024-09-19T08:45:06.409 DEBUG:teuthology.orchestra.run.smithi139:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-19T08:45:06.409 DEBUG:teuthology.orchestra.run.smithi139:> files = glob.glob(asok_path) 2024-09-19T08:45:06.409 DEBUG:teuthology.orchestra.run.smithi139:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-19T08:45:06.409 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:45:06.409 DEBUG:teuthology.orchestra.run.smithi139:> # Given a non-glob path, it better be there 2024-09-19T08:45:06.409 DEBUG:teuthology.orchestra.run.smithi139:> if "*" not in asok_path: 2024-09-19T08:45:06.409 DEBUG:teuthology.orchestra.run.smithi139:> assert(len(files) == 1) 2024-09-19T08:45:06.409 DEBUG:teuthology.orchestra.run.smithi139:> return files[0] 2024-09-19T08:45:06.409 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:45:06.409 DEBUG:teuthology.orchestra.run.smithi139:> for f in files: 2024-09-19T08:45:06.409 DEBUG:teuthology.orchestra.run.smithi139:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T08:45:06.409 DEBUG:teuthology.orchestra.run.smithi139:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T08:45:06.409 DEBUG:teuthology.orchestra.run.smithi139:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T08:45:06.410 DEBUG:teuthology.orchestra.run.smithi139:> contents = proc_f.read() 2024-09-19T08:45:06.410 DEBUG:teuthology.orchestra.run.smithi139:> if mountpoint in contents: 2024-09-19T08:45:06.410 DEBUG:teuthology.orchestra.run.smithi139:> return f 2024-09-19T08:45:06.410 DEBUG:teuthology.orchestra.run.smithi139:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T08:45:06.410 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:45:06.410 DEBUG:teuthology.orchestra.run.smithi139:> print(_find_admin_socket("client.1")) 2024-09-19T08:45:06.410 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-09-19T08:45:06.533 INFO:teuthology.orchestra.run.smithi139.stdout:/var/run/ceph/ceph-client.1.36386.asok 2024-09-19T08:45:06.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:45:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:45:06.544 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.36386.asok 2024-09-19T08:45:06.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:06.545 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.36386.asok status 2024-09-19T08:45:06.704 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-19T08:45:06.704 INFO:teuthology.orchestra.run.smithi139.stdout: "metadata": { 2024-09-19T08:45:06.704 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T08:45:06.704 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T08:45:06.704 INFO:teuthology.orchestra.run.smithi139.stdout: "entity_id": "1", 2024-09-19T08:45:06.704 INFO:teuthology.orchestra.run.smithi139.stdout: "hostname": "smithi139", 2024-09-19T08:45:06.704 INFO:teuthology.orchestra.run.smithi139.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-19T08:45:06.704 INFO:teuthology.orchestra.run.smithi139.stdout: "pid": "36386", 2024-09-19T08:45:06.704 INFO:teuthology.orchestra.run.smithi139.stdout: "root": "/" 2024-09-19T08:45:06.705 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:45:06.705 INFO:teuthology.orchestra.run.smithi139.stdout: "dentry_count": 0, 2024-09-19T08:45:06.705 INFO:teuthology.orchestra.run.smithi139.stdout: "dentry_pinned_count": 0, 2024-09-19T08:45:06.705 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 5702, 2024-09-19T08:45:06.705 INFO:teuthology.orchestra.run.smithi139.stdout: "inst": { 2024-09-19T08:45:06.705 INFO:teuthology.orchestra.run.smithi139.stdout: "name": { 2024-09-19T08:45:06.705 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "client", 2024-09-19T08:45:06.705 INFO:teuthology.orchestra.run.smithi139.stdout: "num": 5702 2024-09-19T08:45:06.705 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:45:06.705 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": { 2024-09-19T08:45:06.705 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-19T08:45:06.705 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:45:06.705 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1569694109 2024-09-19T08:45:06.705 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-19T08:45:06.706 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:45:06.706 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": { 2024-09-19T08:45:06.706 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-19T08:45:06.706 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:45:06.706 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 1569694109 2024-09-19T08:45:06.706 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:45:06.706 INFO:teuthology.orchestra.run.smithi139.stdout: "inst_str": "client.5702 192.168.0.1:0/1569694109", 2024-09-19T08:45:06.706 INFO:teuthology.orchestra.run.smithi139.stdout: "addr_str": "192.168.0.1:0/1569694109", 2024-09-19T08:45:06.706 INFO:teuthology.orchestra.run.smithi139.stdout: "inode_count": 1, 2024-09-19T08:45:06.707 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_epoch": 67, 2024-09-19T08:45:06.707 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_epoch": 120, 2024-09-19T08:45:06.707 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_epoch_barrier": 0, 2024-09-19T08:45:06.707 INFO:teuthology.orchestra.run.smithi139.stdout: "blocklisted": false, 2024-09-19T08:45:06.707 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs" 2024-09-19T08:45:06.707 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-19T08:45:06.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:06.719 DEBUG:teuthology.orchestra.run.smithi139:> stat --file-system '--printf=%T 2024-09-19T08:45:06.719 DEBUG:teuthology.orchestra.run.smithi139:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-19T08:45:06.725 INFO:teuthology.orchestra.run.smithi139.stdout:fuseblk 2024-09-19T08:45:06.725 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-19T08:45:06.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:06.725 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-19T08:45:06.777 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-19T08:45:06.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.915+0000 7efedffbc640 1 Processor -- start 2024-09-19T08:45:06.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.915+0000 7efedffbc640 1 -- start start 2024-09-19T08:45:06.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.919+0000 7efedffbc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7efed806dbd0 0x7efed8073680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:06.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.919+0000 7efedffbc640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efed815baf0 con 0x7efed815fac0 2024-09-19T08:45:06.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.919+0000 7efedffbc640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efed815bdd0 con 0x7efed815f6e0 2024-09-19T08:45:06.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.919+0000 7efedffbc640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7efed815c8b0 con 0x7efed806dbd0 2024-09-19T08:45:06.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.919+0000 7efede559640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7efed815fac0 0x7efed80ce420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:52616/0 (socket says 172.21.15.98:52616) 2024-09-19T08:45:06.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.919+0000 7efede559640 1 -- 172.21.15.98:0/3770169227 learned_addr learned my addr 172.21.15.98:0/3770169227 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:06.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.919+0000 7efedd557640 1 --2- 172.21.15.98:0/3770169227 >> v2:172.21.15.139:3300/0 conn(0x7efed806dbd0 0x7efed8073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:06.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.919+0000 7efedcd56640 1 -- 172.21.15.98:0/3770169227 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1606455517 0 0) 0x7efed815baf0 con 0x7efed815fac0 2024-09-19T08:45:06.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.919+0000 7efedcd56640 1 -- 172.21.15.98:0/3770169227 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efec4003530 con 0x7efed815fac0 2024-09-19T08:45:06.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.919+0000 7efedcd56640 1 -- 172.21.15.98:0/3770169227 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1367967973 0 0) 0x7efec4003530 con 0x7efed815fac0 2024-09-19T08:45:06.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.919+0000 7efedcd56640 1 -- 172.21.15.98:0/3770169227 >> v2:172.21.15.139:3300/0 conn(0x7efed806dbd0 msgr2=0x7efed8073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:06.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.919+0000 7efedcd56640 1 --2- 172.21.15.98:0/3770169227 >> v2:172.21.15.139:3300/0 conn(0x7efed806dbd0 0x7efed8073680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:06.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.919+0000 7efedcd56640 1 -- 172.21.15.98:0/3770169227 >> v1:172.21.15.139:6789/0 conn(0x7efed815f6e0 legacy=0x7efed806d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:06.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.919+0000 7efedcd56640 1 -- 172.21.15.98:0/3770169227 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed815c0b0 con 0x7efed815fac0 2024-09-19T08:45:06.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.919+0000 7efedcd56640 1 -- 172.21.15.98:0/3770169227 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7efed0001a00 con 0x7efed815fac0 2024-09-19T08:45:06.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.919+0000 7efedcd56640 1 -- 172.21.15.98:0/3770169227 >> v1:172.21.15.98:6789/0 conn(0x7efed815fac0 legacy=0x7efed80ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:06.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.919+0000 7efedcd56640 1 --2- 172.21.15.98:0/3770169227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efec40039c0 0x7efec4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:06.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.919+0000 7efedcd56640 1 --2- 172.21.15.98:0/3770169227 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efec4004810 0x7efec4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:06.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.919+0000 7efedcd56640 1 --2- 172.21.15.98:0/3770169227 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efec40051d0 0x7efec400dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:06.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.919+0000 7efedcd56640 1 -- 172.21.15.98:0/3770169227 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efed815c8b0 con 0x7efec4004810 2024-09-19T08:45:06.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.919+0000 7efedcd56640 1 -- 172.21.15.98:0/3770169227 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efec400e280 con 0x7efec40039c0 2024-09-19T08:45:06.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.919+0000 7efedcd56640 1 -- 172.21.15.98:0/3770169227 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7efec400e480 con 0x7efec40051d0 2024-09-19T08:45:06.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.919+0000 7efedd557640 1 --2- 172.21.15.98:0/3770169227 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efec4004810 0x7efec4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:06.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.919+0000 7efeddd58640 1 --2- 172.21.15.98:0/3770169227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efec40039c0 0x7efec4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:06.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.919+0000 7efede559640 1 --2- 172.21.15.98:0/3770169227 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efec40051d0 0x7efec400dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:06.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.919+0000 7efedd557640 1 -- 172.21.15.98:0/3770169227 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efec40051d0 msgr2=0x7efec400dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:06.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.919+0000 7efedd557640 1 --2- 172.21.15.98:0/3770169227 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efec40051d0 0x7efec400dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:06.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.919+0000 7efedd557640 1 -- 172.21.15.98:0/3770169227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efec40039c0 msgr2=0x7efec4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:06.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.919+0000 7efedd557640 1 --2- 172.21.15.98:0/3770169227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efec40039c0 0x7efec4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:06.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.919+0000 7efedd557640 1 -- 172.21.15.98:0/3770169227 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec4003530 con 0x7efec4004810 2024-09-19T08:45:06.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.919+0000 7efede559640 1 -- 172.21.15.98:0/3770169227 reap_dead start 2024-09-19T08:45:06.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.923+0000 7efedd557640 1 --2- 172.21.15.98:0/3770169227 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efec4004810 0x7efec4004c90 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7efec8009ab0 tx=0x7efec801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd0486ddf4807a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:06.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.923+0000 7efedcd56640 1 -- 172.21.15.98:0/3770169227 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec80b2070 con 0x7efec4004810 2024-09-19T08:45:06.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.923+0000 7efedcd56640 1 -- 172.21.15.98:0/3770169227 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efec80a5530 con 0x7efec4004810 2024-09-19T08:45:06.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.923+0000 7efedcd56640 1 -- 172.21.15.98:0/3770169227 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec80096b0 con 0x7efec4004810 2024-09-19T08:45:06.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.923+0000 7efedffbc640 1 -- 172.21.15.98:0/3770169227 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efec4004810 msgr2=0x7efec4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:06.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.923+0000 7efedffbc640 1 --2- 172.21.15.98:0/3770169227 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efec4004810 0x7efec4004c90 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7efec8009ab0 tx=0x7efec801ca50 comp rx=0 tx=0).stop 2024-09-19T08:45:06.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.923+0000 7efedffbc640 1 -- 172.21.15.98:0/3770169227 shutdown_connections 2024-09-19T08:45:06.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.923+0000 7efedffbc640 1 --2- 172.21.15.98:0/3770169227 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efec4004810 0x7efec4004c90 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:06.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.923+0000 7efedffbc640 1 -- 172.21.15.98:0/3770169227 >> 172.21.15.98:0/3770169227 conn(0x7efed806c090 msgr2=0x7efed806c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:06.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.923+0000 7efedffbc640 1 -- 172.21.15.98:0/3770169227 shutdown_connections 2024-09-19T08:45:06.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.923+0000 7efedffbc640 1 -- 172.21.15.98:0/3770169227 wait complete. 2024-09-19T08:45:06.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.923+0000 7efedffbc640 1 Processor -- start 2024-09-19T08:45:06.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.923+0000 7efedffbc640 1 -- start start 2024-09-19T08:45:06.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.923+0000 7efedffbc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efec4004810 0x7efed8155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:06.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.923+0000 7efedffbc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efed8152a00 0x7efed814a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:06.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.923+0000 7efedffbc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efed80b2a10 0x7efed80b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:06.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.923+0000 7efedffbc640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efed815b890 con 0x7efec4004810 2024-09-19T08:45:06.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.923+0000 7efedffbc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efed80742d0 con 0x7efed8152a00 2024-09-19T08:45:06.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.923+0000 7efedffbc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7efed806d4c0 con 0x7efed80b2a10 2024-09-19T08:45:06.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.923+0000 7efeddd58640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efec4004810 0x7efed8155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:06.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.923+0000 7efeddd58640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efec4004810 0x7efed8155550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:47964/0 (socket says 172.21.15.98:47964) 2024-09-19T08:45:06.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.923+0000 7efeddd58640 1 -- 172.21.15.98:0/874990526 learned_addr learned my addr 172.21.15.98:0/874990526 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:06.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.923+0000 7efedd557640 1 --2- 172.21.15.98:0/874990526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efed8152a00 0x7efed814a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:06.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.923+0000 7efede559640 1 --2- 172.21.15.98:0/874990526 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efed80b2a10 0x7efed80b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:06.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.923+0000 7efeddd58640 1 -- 172.21.15.98:0/874990526 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efed80b2a10 msgr2=0x7efed80b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:06.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.923+0000 7efeddd58640 1 --2- 172.21.15.98:0/874990526 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efed80b2a10 0x7efed80b2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:06.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.923+0000 7efeddd58640 1 -- 172.21.15.98:0/874990526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efed8152a00 msgr2=0x7efed814a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:06.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.923+0000 7efeddd58640 1 --2- 172.21.15.98:0/874990526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efed8152a00 0x7efed814a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:06.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.923+0000 7efeddd58640 1 -- 172.21.15.98:0/874990526 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed814a7f0 con 0x7efec4004810 2024-09-19T08:45:06.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.923+0000 7efede559640 1 --2- 172.21.15.98:0/874990526 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efed80b2a10 0x7efed80b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:06.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.923+0000 7efeddd58640 1 --2- 172.21.15.98:0/874990526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efec4004810 0x7efed8155550 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7efed400f620 tx=0x7efed400fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:06.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.927+0000 7efece7fc640 1 -- 172.21.15.98:0/874990526 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed401a070 con 0x7efec4004810 2024-09-19T08:45:06.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.927+0000 7efece7fc640 1 -- 172.21.15.98:0/874990526 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efed4005320 con 0x7efec4004810 2024-09-19T08:45:06.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.927+0000 7efece7fc640 1 -- 172.21.15.98:0/874990526 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4007250 con 0x7efec4004810 2024-09-19T08:45:06.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.927+0000 7efedffbc640 1 -- 172.21.15.98:0/874990526 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efed80b1970 con 0x7efec4004810 2024-09-19T08:45:06.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.927+0000 7efedffbc640 1 -- 172.21.15.98:0/874990526 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efed80b1f30 con 0x7efec4004810 2024-09-19T08:45:06.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.927+0000 7efedffbc640 1 -- 172.21.15.98:0/874990526 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efed806d850 con 0x7efec4004810 2024-09-19T08:45:06.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.927+0000 7efece7fc640 1 -- 172.21.15.98:0/874990526 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7efed401d070 con 0x7efec4004810 2024-09-19T08:45:06.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.927+0000 7efece7fc640 1 --2- 172.21.15.98:0/874990526 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7efebc067cb0 0x7efebc06a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:06.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.927+0000 7efece7fc640 1 -- 172.21.15.98:0/874990526 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5795+0+0 (secure 0 0 0) 0x7efed4085b70 con 0x7efec4004810 2024-09-19T08:45:06.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.927+0000 7efedd557640 1 --2- 172.21.15.98:0/874990526 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7efebc067cb0 0x7efebc06a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:06.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.927+0000 7efedd557640 1 --2- 172.21.15.98:0/874990526 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7efebc067cb0 0x7efebc06a170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efec80af000 tx=0x7efec8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:06.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:06.931+0000 7efece7fc640 1 -- 172.21.15.98:0/874990526 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7efed4056070 con 0x7efec4004810 2024-09-19T08:45:07.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:07.127+0000 7efedffbc640 1 -- 172.21.15.98:0/874990526 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7efed815b200 con 0x7efec4004810 2024-09-19T08:45:07.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:07.967+0000 7efece7fc640 1 -- 172.21.15.98:0/874990526 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v68) ==== 136+0+0 (secure 0 0 0) 0x7efed4055970 con 0x7efec4004810 2024-09-19T08:45:07.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:07.971+0000 7efedffbc640 1 -- 172.21.15.98:0/874990526 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7efebc067cb0 msgr2=0x7efebc06a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:07.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:07.971+0000 7efedffbc640 1 --2- 172.21.15.98:0/874990526 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7efebc067cb0 0x7efebc06a170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efec80af000 tx=0x7efec8008040 comp rx=0 tx=0).stop 2024-09-19T08:45:07.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:07.971+0000 7efedffbc640 1 -- 172.21.15.98:0/874990526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efec4004810 msgr2=0x7efed8155550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:07.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:07.971+0000 7efedffbc640 1 --2- 172.21.15.98:0/874990526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efec4004810 0x7efed8155550 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7efed400f620 tx=0x7efed400fae0 comp rx=0 tx=0).stop 2024-09-19T08:45:07.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:07.971+0000 7efedffbc640 1 -- 172.21.15.98:0/874990526 shutdown_connections 2024-09-19T08:45:07.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:07.971+0000 7efedffbc640 1 --2- 172.21.15.98:0/874990526 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7efebc067cb0 0x7efebc06a170 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:07.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:07.971+0000 7efedffbc640 1 --2- 172.21.15.98:0/874990526 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efed80b2a10 0x7efed80b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:07.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:07.971+0000 7efedffbc640 1 --2- 172.21.15.98:0/874990526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efed8152a00 0x7efed814a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:07.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:07.971+0000 7efedffbc640 1 --2- 172.21.15.98:0/874990526 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efec4004810 0x7efed8155550 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:07.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:07.971+0000 7efedffbc640 1 -- 172.21.15.98:0/874990526 >> 172.21.15.98:0/874990526 conn(0x7efed806c090 msgr2=0x7efed8073680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:07.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:07.971+0000 7efedffbc640 1 -- 172.21.15.98:0/874990526 shutdown_connections 2024-09-19T08:45:07.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:07.971+0000 7efedffbc640 1 -- 172.21.15.98:0/874990526 wait complete. 2024-09-19T08:45:07.999 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-19T08:45:08.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.123+0000 7f840a173640 1 Processor -- start 2024-09-19T08:45:08.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.123+0000 7f840a173640 1 -- start start 2024-09-19T08:45:08.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.123+0000 7f840a173640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f840415e420 0x7f840415f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:08.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.123+0000 7f840a173640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f840415e800 0x7f840406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:08.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.123+0000 7f840a173640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f840406c1d0 0x7f84040b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:08.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.123+0000 7f840a173640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f840406c6e0 con 0x7f840415e800 2024-09-19T08:45:08.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.123+0000 7f840a173640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f840406c9c0 con 0x7f840406c1d0 2024-09-19T08:45:08.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.123+0000 7f840a173640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f840415ed60 con 0x7f840415e420 2024-09-19T08:45:08.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.123+0000 7f8402ffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f840415e800 0x7f840406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:08.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.123+0000 7f8402ffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f840415e800 0x7f840406bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59362/0 (socket says 172.21.15.98:59362) 2024-09-19T08:45:08.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.123+0000 7f8402ffd640 1 -- 172.21.15.98:0/2889682338 learned_addr learned my addr 172.21.15.98:0/2889682338 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:08.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.123+0000 7f8403fff640 1 --2- 172.21.15.98:0/2889682338 >> v2:172.21.15.139:3300/0 conn(0x7f840406c1d0 0x7f84040b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:08.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.123+0000 7f84037fe640 1 --2- 172.21.15.98:0/2889682338 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f840415e420 0x7f840415f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:08.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.123+0000 7f8402ffd640 1 -- 172.21.15.98:0/2889682338 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f840415e420 msgr2=0x7f840415f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:08.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.123+0000 7f8402ffd640 1 --2- 172.21.15.98:0/2889682338 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f840415e420 0x7f840415f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:08.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.123+0000 7f8402ffd640 1 -- 172.21.15.98:0/2889682338 >> v2:172.21.15.139:3300/0 conn(0x7f840406c1d0 msgr2=0x7f84040b78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:08.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.123+0000 7f8402ffd640 1 --2- 172.21.15.98:0/2889682338 >> v2:172.21.15.139:3300/0 conn(0x7f840406c1d0 0x7f84040b78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:08.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.123+0000 7f8402ffd640 1 -- 172.21.15.98:0/2889682338 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f840415f040 con 0x7f840415e800 2024-09-19T08:45:08.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.127+0000 7f8402ffd640 1 --2- 172.21.15.98:0/2889682338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f840415e800 0x7f840406bc90 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f8404070350 tx=0x7f83f801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e34110a7fc705d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:08.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.127+0000 7f84027fc640 1 -- 172.21.15.98:0/2889682338 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f80ab5f0 con 0x7f840415e800 2024-09-19T08:45:08.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.127+0000 7f84027fc640 1 -- 172.21.15.98:0/2889682338 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83f80b0070 con 0x7f840415e800 2024-09-19T08:45:08.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.127+0000 7f84027fc640 1 -- 172.21.15.98:0/2889682338 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f80ab9b0 con 0x7f840415e800 2024-09-19T08:45:08.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.127+0000 7f840a173640 1 -- 172.21.15.98:0/2889682338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f840415e800 msgr2=0x7f840406bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:08.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.127+0000 7f840a173640 1 --2- 172.21.15.98:0/2889682338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f840415e800 0x7f840406bc90 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f8404070350 tx=0x7f83f801c520 comp rx=0 tx=0).stop 2024-09-19T08:45:08.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.127+0000 7f840a173640 1 -- 172.21.15.98:0/2889682338 shutdown_connections 2024-09-19T08:45:08.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.127+0000 7f840a173640 1 --2- 172.21.15.98:0/2889682338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f840415e800 0x7f840406bc90 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:08.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.127+0000 7f840a173640 1 --2- 172.21.15.98:0/2889682338 >> v2:172.21.15.139:3300/0 conn(0x7f840406c1d0 0x7f84040b78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:08.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.127+0000 7f840a173640 1 --2- 172.21.15.98:0/2889682338 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f840415e420 0x7f840415f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:08.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.127+0000 7f840a173640 1 -- 172.21.15.98:0/2889682338 >> 172.21.15.98:0/2889682338 conn(0x7f84040b40a0 msgr2=0x7f84040b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:08.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.127+0000 7f840a173640 1 -- 172.21.15.98:0/2889682338 shutdown_connections 2024-09-19T08:45:08.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.127+0000 7f840a173640 1 -- 172.21.15.98:0/2889682338 wait complete. 2024-09-19T08:45:08.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.127+0000 7f840a173640 1 Processor -- start 2024-09-19T08:45:08.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.127+0000 7f840a173640 1 -- start start 2024-09-19T08:45:08.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.131+0000 7f840a173640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f840406c1d0 0x7f84040d0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:08.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.131+0000 7f840a173640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f840415e420 0x7f84040d1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:08.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.131+0000 7f840a173640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f840415e800 0x7f8404157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:08.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.131+0000 7f840a173640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8404162c50 con 0x7f840415e800 2024-09-19T08:45:08.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.131+0000 7f840a173640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8404162ad0 con 0x7f840415e420 2024-09-19T08:45:08.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.131+0000 7f840a173640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f84040b8200 con 0x7f840406c1d0 2024-09-19T08:45:08.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.131+0000 7f8403fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f840415e800 0x7f8404157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:08.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.131+0000 7f8403fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f840415e800 0x7f8404157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59364/0 (socket says 172.21.15.98:59364) 2024-09-19T08:45:08.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.131+0000 7f8403fff640 1 -- 172.21.15.98:0/3772161657 learned_addr learned my addr 172.21.15.98:0/3772161657 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:08.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.131+0000 7f84037fe640 1 --2- 172.21.15.98:0/3772161657 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f840406c1d0 0x7f84040d0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:08.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.131+0000 7f8402ffd640 1 --2- 172.21.15.98:0/3772161657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f840415e420 0x7f84040d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:08.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.131+0000 7f8403fff640 1 -- 172.21.15.98:0/3772161657 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f840406c1d0 msgr2=0x7f84040d0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:08.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.131+0000 7f8403fff640 1 --2- 172.21.15.98:0/3772161657 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f840406c1d0 0x7f84040d0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:08.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.131+0000 7f8403fff640 1 -- 172.21.15.98:0/3772161657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f840415e420 msgr2=0x7f84040d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:08.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.131+0000 7f8403fff640 1 --2- 172.21.15.98:0/3772161657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f840415e420 0x7f84040d1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:08.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.131+0000 7f8403fff640 1 -- 172.21.15.98:0/3772161657 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84041584a0 con 0x7f840415e800 2024-09-19T08:45:08.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.131+0000 7f8403fff640 1 --2- 172.21.15.98:0/3772161657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f840415e800 0x7f8404157770 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f83f400d600 tx=0x7f83f400dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:08.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.131+0000 7f83ebfff640 1 -- 172.21.15.98:0/3772161657 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f4018020 con 0x7f840415e800 2024-09-19T08:45:08.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.131+0000 7f83ebfff640 1 -- 172.21.15.98:0/3772161657 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83f4014620 con 0x7f840415e800 2024-09-19T08:45:08.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.131+0000 7f840a173640 1 -- 172.21.15.98:0/3772161657 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8404158780 con 0x7f840415e800 2024-09-19T08:45:08.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.131+0000 7f83ebfff640 1 -- 172.21.15.98:0/3772161657 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f40149f0 con 0x7f840415e800 2024-09-19T08:45:08.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.131+0000 7f840a173640 1 -- 172.21.15.98:0/3772161657 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8404164800 con 0x7f840415e800 2024-09-19T08:45:08.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.131+0000 7f840a173640 1 -- 172.21.15.98:0/3772161657 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8404164a00 con 0x7f840415e800 2024-09-19T08:45:08.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.131+0000 7f83ebfff640 1 -- 172.21.15.98:0/3772161657 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f83f401b020 con 0x7f840415e800 2024-09-19T08:45:08.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.131+0000 7f83ebfff640 1 --2- 172.21.15.98:0/3772161657 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f83f0067cb0 0x7f83f006a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:08.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.131+0000 7f83ebfff640 1 -- 172.21.15.98:0/3772161657 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5795+0+0 (secure 0 0 0) 0x7f83f4086980 con 0x7f840415e800 2024-09-19T08:45:08.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.131+0000 7f84037fe640 1 --2- 172.21.15.98:0/3772161657 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f83f0067cb0 0x7f83f006a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:08.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.135+0000 7f84037fe640 1 --2- 172.21.15.98:0/3772161657 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f83f0067cb0 0x7f83f006a170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f83fc009f60 tx=0x7f83fc00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:08.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.139+0000 7f83ebfff640 1 -- 172.21.15.98:0/3772161657 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f8404164a00 con 0x7f840415e800 2024-09-19T08:45:08.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.323+0000 7f840a173640 1 -- 172.21.15.98:0/3772161657 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f840415f560 con 0x7f83f0067cb0 2024-09-19T08:45:08.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.327+0000 7f83ebfff640 1 -- 172.21.15.98:0/3772161657 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f840415f560 con 0x7f83f0067cb0 2024-09-19T08:45:08.328 INFO:teuthology.orchestra.run.smithi098.stdout:[ 2024-09-19T08:45:08.329 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:45:08.329 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "cephfs" 2024-09-19T08:45:08.329 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:45:08.329 INFO:teuthology.orchestra.run.smithi098.stdout:] 2024-09-19T08:45:08.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.327+0000 7f840a173640 1 -- 172.21.15.98:0/3772161657 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f83f0067cb0 msgr2=0x7f83f006a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:08.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.327+0000 7f840a173640 1 --2- 172.21.15.98:0/3772161657 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f83f0067cb0 0x7f83f006a170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f83fc009f60 tx=0x7f83fc00b040 comp rx=0 tx=0).stop 2024-09-19T08:45:08.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.327+0000 7f840a173640 1 -- 172.21.15.98:0/3772161657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f840415e800 msgr2=0x7f8404157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:08.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.327+0000 7f840a173640 1 --2- 172.21.15.98:0/3772161657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f840415e800 0x7f8404157770 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f83f400d600 tx=0x7f83f400dac0 comp rx=0 tx=0).stop 2024-09-19T08:45:08.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.327+0000 7f840a173640 1 -- 172.21.15.98:0/3772161657 shutdown_connections 2024-09-19T08:45:08.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.327+0000 7f840a173640 1 --2- 172.21.15.98:0/3772161657 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f83f0067cb0 0x7f83f006a170 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:08.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.327+0000 7f840a173640 1 --2- 172.21.15.98:0/3772161657 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f840415e800 0x7f8404157770 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:08.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.327+0000 7f840a173640 1 --2- 172.21.15.98:0/3772161657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f840415e420 0x7f84040d1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:08.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.327+0000 7f840a173640 1 --2- 172.21.15.98:0/3772161657 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f840406c1d0 0x7f84040d0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:08.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.327+0000 7f840a173640 1 -- 172.21.15.98:0/3772161657 >> 172.21.15.98:0/3772161657 conn(0x7f84040b40a0 msgr2=0x7f8404070d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:08.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.327+0000 7f840a173640 1 -- 172.21.15.98:0/3772161657 shutdown_connections 2024-09-19T08:45:08.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.327+0000 7f840a173640 1 -- 172.21.15.98:0/3772161657 wait complete. 2024-09-19T08:45:08.347 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T08:45:08.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.471+0000 7f0e6dcba640 1 Processor -- start 2024-09-19T08:45:08.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.471+0000 7f0e6dcba640 1 -- start start 2024-09-19T08:45:08.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.471+0000 7f0e6dcba640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e68145b70 0x7f0e6806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:08.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.471+0000 7f0e6dcba640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0e68071ac0 0x7f0e680b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:08.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.471+0000 7f0e6dcba640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e6806c5f0 con 0x7f0e6806c1d0 2024-09-19T08:45:08.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.471+0000 7f0e6dcba640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f0e68071f20 con 0x7f0e68071ac0 2024-09-19T08:45:08.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.471+0000 7f0e6dcba640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0e68073530 con 0x7f0e68145b70 2024-09-19T08:45:08.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.471+0000 7f0e67fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0e68071ac0 0x7f0e680b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:08.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.471+0000 7f0e67fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0e68071ac0 0x7f0e680b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59380/0 (socket says 172.21.15.98:59380) 2024-09-19T08:45:08.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.471+0000 7f0e67fff640 1 -- 172.21.15.98:0/2119276652 learned_addr learned my addr 172.21.15.98:0/2119276652 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:08.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.471+0000 7f0e677fe640 1 --2- 172.21.15.98:0/2119276652 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e68145b70 0x7f0e6806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:08.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.471+0000 7f0e67fff640 1 -- 172.21.15.98:0/2119276652 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e68145b70 msgr2=0x7f0e6806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:08.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.471+0000 7f0e67fff640 1 --2- 172.21.15.98:0/2119276652 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e68145b70 0x7f0e6806bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:08.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.471+0000 7f0e67fff640 1 -- 172.21.15.98:0/2119276652 >> v1:172.21.15.139:6789/0 conn(0x7f0e6806c1d0 legacy=0x7f0e680713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:08.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.471+0000 7f0e67fff640 1 -- 172.21.15.98:0/2119276652 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e68073810 con 0x7f0e68071ac0 2024-09-19T08:45:08.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.475+0000 7f0e67fff640 1 --2- 172.21.15.98:0/2119276652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e68071ac0 0x7f0e680b3410 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f0e60009e50 tx=0x7f0e6001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c138a6095759f049 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:08.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.475+0000 7f0e667fc640 1 -- 172.21.15.98:0/2119276652 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e600a5660 con 0x7f0e68071ac0 2024-09-19T08:45:08.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.475+0000 7f0e667fc640 1 -- 172.21.15.98:0/2119276652 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e600a5800 con 0x7f0e68071ac0 2024-09-19T08:45:08.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.475+0000 7f0e667fc640 1 -- 172.21.15.98:0/2119276652 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e600a5bd0 con 0x7f0e68071ac0 2024-09-19T08:45:08.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.475+0000 7f0e6dcba640 1 -- 172.21.15.98:0/2119276652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e68071ac0 msgr2=0x7f0e680b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:08.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.475+0000 7f0e6dcba640 1 --2- 172.21.15.98:0/2119276652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e68071ac0 0x7f0e680b3410 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f0e60009e50 tx=0x7f0e6001c9a0 comp rx=0 tx=0).stop 2024-09-19T08:45:08.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.475+0000 7f0e6dcba640 1 -- 172.21.15.98:0/2119276652 shutdown_connections 2024-09-19T08:45:08.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.475+0000 7f0e6dcba640 1 --2- 172.21.15.98:0/2119276652 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e68071ac0 0x7f0e680b3410 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:08.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.475+0000 7f0e6dcba640 1 --2- 172.21.15.98:0/2119276652 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e68145b70 0x7f0e6806bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:08.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.475+0000 7f0e6dcba640 1 -- 172.21.15.98:0/2119276652 >> 172.21.15.98:0/2119276652 conn(0x7f0e680b2010 msgr2=0x7f0e680b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:08.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.475+0000 7f0e6dcba640 1 -- 172.21.15.98:0/2119276652 shutdown_connections 2024-09-19T08:45:08.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.475+0000 7f0e6dcba640 1 -- 172.21.15.98:0/2119276652 wait complete. 2024-09-19T08:45:08.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.475+0000 7f0e6dcba640 1 Processor -- start 2024-09-19T08:45:08.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.475+0000 7f0e6dcba640 1 -- start start 2024-09-19T08:45:08.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.475+0000 7f0e6dcba640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e6806c1d0 0x7f0e681f6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:08.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.475+0000 7f0e6dcba640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e68071ac0 0x7f0e6813c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:08.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.475+0000 7f0e6dcba640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e68145b70 0x7f0e6813d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:08.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.475+0000 7f0e6dcba640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0e680b42e0 con 0x7f0e68145b70 2024-09-19T08:45:08.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.475+0000 7f0e6dcba640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0e680b4160 con 0x7f0e68071ac0 2024-09-19T08:45:08.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.475+0000 7f0e6dcba640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0e680b4460 con 0x7f0e6806c1d0 2024-09-19T08:45:08.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.475+0000 7f0e67fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e68145b70 0x7f0e6813d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:08.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.475+0000 7f0e67fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e68145b70 0x7f0e6813d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59390/0 (socket says 172.21.15.98:59390) 2024-09-19T08:45:08.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.475+0000 7f0e67fff640 1 -- 172.21.15.98:0/2432275755 learned_addr learned my addr 172.21.15.98:0/2432275755 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:08.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.475+0000 7f0e677fe640 1 --2- 172.21.15.98:0/2432275755 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e6806c1d0 0x7f0e681f6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:08.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.475+0000 7f0e66ffd640 1 --2- 172.21.15.98:0/2432275755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e68071ac0 0x7f0e6813c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:08.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.479+0000 7f0e67fff640 1 -- 172.21.15.98:0/2432275755 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e6806c1d0 msgr2=0x7f0e681f6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:08.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.479+0000 7f0e67fff640 1 --2- 172.21.15.98:0/2432275755 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e6806c1d0 0x7f0e681f6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:08.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.479+0000 7f0e67fff640 1 -- 172.21.15.98:0/2432275755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e68071ac0 msgr2=0x7f0e6813c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:08.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.479+0000 7f0e67fff640 1 --2- 172.21.15.98:0/2432275755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e68071ac0 0x7f0e6813c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:08.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.479+0000 7f0e67fff640 1 -- 172.21.15.98:0/2432275755 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e6813f250 con 0x7f0e68145b70 2024-09-19T08:45:08.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.479+0000 7f0e677fe640 1 --2- 172.21.15.98:0/2432275755 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e6806c1d0 0x7f0e681f6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:08.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.479+0000 7f0e66ffd640 1 --2- 172.21.15.98:0/2432275755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e68071ac0 0x7f0e6813c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:08.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.479+0000 7f0e67fff640 1 --2- 172.21.15.98:0/2432275755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e68145b70 0x7f0e6813d530 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f0e600af040 tx=0x7f0e60019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:08.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.479+0000 7f0e4ffff640 1 -- 172.21.15.98:0/2432275755 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e600bc020 con 0x7f0e68145b70 2024-09-19T08:45:08.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.479+0000 7f0e4ffff640 1 -- 172.21.15.98:0/2432275755 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e600b8670 con 0x7f0e68145b70 2024-09-19T08:45:08.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.479+0000 7f0e6dcba640 1 -- 172.21.15.98:0/2432275755 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e6813f530 con 0x7f0e68145b70 2024-09-19T08:45:08.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.479+0000 7f0e4ffff640 1 -- 172.21.15.98:0/2432275755 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e60009510 con 0x7f0e68145b70 2024-09-19T08:45:08.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.479+0000 7f0e6dcba640 1 -- 172.21.15.98:0/2432275755 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e680cd360 con 0x7f0e68145b70 2024-09-19T08:45:08.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.479+0000 7f0e6dcba640 1 -- 172.21.15.98:0/2432275755 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e681f7510 con 0x7f0e68145b70 2024-09-19T08:45:08.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.479+0000 7f0e4ffff640 1 -- 172.21.15.98:0/2432275755 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f0e600bf020 con 0x7f0e68145b70 2024-09-19T08:45:08.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.479+0000 7f0e4ffff640 1 --2- 172.21.15.98:0/2432275755 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0e54067cb0 0x7f0e5406a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:08.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.479+0000 7f0e4ffff640 1 -- 172.21.15.98:0/2432275755 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5795+0+0 (secure 0 0 0) 0x7f0e601213c0 con 0x7f0e68145b70 2024-09-19T08:45:08.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.479+0000 7f0e677fe640 1 --2- 172.21.15.98:0/2432275755 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0e54067cb0 0x7f0e5406a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:08.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.479+0000 7f0e677fe640 1 --2- 172.21.15.98:0/2432275755 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0e54067cb0 0x7f0e5406a170 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0e680ccf30 tx=0x7f0e58008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:08.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.483+0000 7f0e4ffff640 1 -- 172.21.15.98:0/2432275755 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f0e600c6030 con 0x7f0e68145b70 2024-09-19T08:45:08.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.643+0000 7f0e6dcba640 1 -- 172.21.15.98:0/2432275755 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f0e680757a0 con 0x7f0e68145b70 2024-09-19T08:45:08.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.643+0000 7f0e4ffff640 1 -- 172.21.15.98:0/2432275755 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7f0e60016620 con 0x7f0e68145b70 2024-09-19T08:45:08.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.647+0000 7f0e6dcba640 1 -- 172.21.15.98:0/2432275755 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0e54067cb0 msgr2=0x7f0e5406a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:08.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.647+0000 7f0e6dcba640 1 --2- 172.21.15.98:0/2432275755 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0e54067cb0 0x7f0e5406a170 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0e680ccf30 tx=0x7f0e58008040 comp rx=0 tx=0).stop 2024-09-19T08:45:08.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.647+0000 7f0e6dcba640 1 -- 172.21.15.98:0/2432275755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e68145b70 msgr2=0x7f0e6813d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:08.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.647+0000 7f0e6dcba640 1 --2- 172.21.15.98:0/2432275755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e68145b70 0x7f0e6813d530 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f0e600af040 tx=0x7f0e60019740 comp rx=0 tx=0).stop 2024-09-19T08:45:08.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.647+0000 7f0e6dcba640 1 -- 172.21.15.98:0/2432275755 shutdown_connections 2024-09-19T08:45:08.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.647+0000 7f0e6dcba640 1 --2- 172.21.15.98:0/2432275755 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0e54067cb0 0x7f0e5406a170 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:08.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.647+0000 7f0e6dcba640 1 --2- 172.21.15.98:0/2432275755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e68145b70 0x7f0e6813d530 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:08.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.647+0000 7f0e6dcba640 1 --2- 172.21.15.98:0/2432275755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e68071ac0 0x7f0e6813c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:08.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.647+0000 7f0e6dcba640 1 --2- 172.21.15.98:0/2432275755 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e6806c1d0 0x7f0e681f6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:08.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.647+0000 7f0e6dcba640 1 -- 172.21.15.98:0/2432275755 >> 172.21.15.98:0/2432275755 conn(0x7f0e680b2010 msgr2=0x7f0e680b0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:08.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.647+0000 7f0e6dcba640 1 -- 172.21.15.98:0/2432275755 shutdown_connections 2024-09-19T08:45:08.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:08.647+0000 7f0e6dcba640 1 -- 172.21.15.98:0/2432275755 wait complete. 2024-09-19T08:45:08.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:08.666 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol41980/66f5ddee-1364-4607-8f74-c9617e17c680) 2024-09-19T08:45:08.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:08.690 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol41980/66f5ddee-1364-4607-8f74-c9617e17c680/.snap/fake) 2024-09-19T08:45:08.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:08.755 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-19T08:45:08.803 INFO:teuthology.orchestra.run.smithi098.stdout:cephfs_data 2024-09-19T08:45:08.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:08.804 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol41980/66f5ddee-1364-4607-8f74-c9617e17c680) 2024-09-19T08:45:08.863 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:45:08.863 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol41980/.meta 2024-09-19T08:45:08.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:08.925 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/subvol_grp33620/subvol16391/f0511ac8-2f0c-4269-bb20-9ba71129e008) 2024-09-19T08:45:08.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:08.993 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/subvol_grp33620/subvol16391/f0511ac8-2f0c-4269-bb20-9ba71129e008/.snap/fake) 2024-09-19T08:45:09.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:09.065 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-19T08:45:09.115 INFO:teuthology.orchestra.run.smithi098.stdout:cephfs_data 2024-09-19T08:45:09.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:09.115 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/subvol_grp33620/subvol16391/f0511ac8-2f0c-4269-bb20-9ba71129e008) 2024-09-19T08:45:09.177 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:45:09.177 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/subvol_grp33620/subvol16391/.meta 2024-09-19T08:45:09.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:09.233 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol56792/6636d43c-dc2a-49bd-b834-7c8a36fd9b4d) 2024-09-19T08:45:09.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:09.293 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-19T08:45:09.345 INFO:teuthology.orchestra.run.smithi098.stdout:cephfs_data 2024-09-19T08:45:09.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:09.346 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol56792/6636d43c-dc2a-49bd-b834-7c8a36fd9b4d) 2024-09-19T08:45:09.409 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:45:09.409 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol56792/.meta 2024-09-19T08:45:09.468 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol41980 2024-09-19T08:45:09.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.639+0000 7fe6ef5fc640 1 Processor -- start 2024-09-19T08:45:09.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.643+0000 7fe6ef5fc640 1 -- start start 2024-09-19T08:45:09.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.643+0000 7fe6ef5fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe6e8145b70 0x7fe6e806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:09.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.643+0000 7fe6ef5fc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6e806c1d0 0x7fe6e80713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:09.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.643+0000 7fe6ef5fc640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6e806c5f0 con 0x7fe6e80718f0 2024-09-19T08:45:09.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.643+0000 7fe6ef5fc640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe6e8073530 con 0x7fe6e8145b70 2024-09-19T08:45:09.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.643+0000 7fe6ef5fc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe6e8073810 con 0x7fe6e806c1d0 2024-09-19T08:45:09.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.643+0000 7fe6ecb97640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6e806c1d0 0x7fe6e80713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:09.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.643+0000 7fe6ed398640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe6e8145b70 0x7fe6e806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:09.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.643+0000 7fe6ecb97640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6e806c1d0 0x7fe6e80713b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:59182/0 (socket says 172.21.15.98:59182) 2024-09-19T08:45:09.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.643+0000 7fe6ecb97640 1 -- 172.21.15.98:0/4288583464 learned_addr learned my addr 172.21.15.98:0/4288583464 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:09.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.643+0000 7fe6ecb97640 1 -- 172.21.15.98:0/4288583464 >> v2:172.21.15.139:3300/0 conn(0x7fe6e8145b70 msgr2=0x7fe6e806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:09.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.643+0000 7fe6ecb97640 1 --2- 172.21.15.98:0/4288583464 >> v2:172.21.15.139:3300/0 conn(0x7fe6e8145b70 0x7fe6e806bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:09.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.643+0000 7fe6ecb97640 1 -- 172.21.15.98:0/4288583464 >> v1:172.21.15.139:6789/0 conn(0x7fe6e80718f0 legacy=0x7fe6e80cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:09.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.643+0000 7fe6ecb97640 1 -- 172.21.15.98:0/4288583464 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e8073af0 con 0x7fe6e806c1d0 2024-09-19T08:45:09.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.647+0000 7fe6ecb97640 1 --2- 172.21.15.98:0/4288583464 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6e806c1d0 0x7fe6e80713b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4009870 tx=0x7fe6e401c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fa6104b2057ba71 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:09.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.647+0000 7fe6dffff640 1 -- 172.21.15.98:0/4288583464 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e40b0070 con 0x7fe6e806c1d0 2024-09-19T08:45:09.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.647+0000 7fe6dffff640 1 -- 172.21.15.98:0/4288583464 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6e40ab600 con 0x7fe6e806c1d0 2024-09-19T08:45:09.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.647+0000 7fe6dffff640 1 -- 172.21.15.98:0/4288583464 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e40ab9d0 con 0x7fe6e806c1d0 2024-09-19T08:45:09.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.647+0000 7fe6ef5fc640 1 -- 172.21.15.98:0/4288583464 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6e806c1d0 msgr2=0x7fe6e80713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:09.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.647+0000 7fe6ef5fc640 1 --2- 172.21.15.98:0/4288583464 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6e806c1d0 0x7fe6e80713b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4009870 tx=0x7fe6e401c4d0 comp rx=0 tx=0).stop 2024-09-19T08:45:09.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.647+0000 7fe6ef5fc640 1 -- 172.21.15.98:0/4288583464 shutdown_connections 2024-09-19T08:45:09.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.647+0000 7fe6ef5fc640 1 --2- 172.21.15.98:0/4288583464 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6e806c1d0 0x7fe6e80713b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:09.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.647+0000 7fe6ef5fc640 1 --2- 172.21.15.98:0/4288583464 >> v2:172.21.15.139:3300/0 conn(0x7fe6e8145b70 0x7fe6e806bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:09.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.647+0000 7fe6ef5fc640 1 -- 172.21.15.98:0/4288583464 >> 172.21.15.98:0/4288583464 conn(0x7fe6e80b2010 msgr2=0x7fe6e80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:09.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.647+0000 7fe6ef5fc640 1 -- 172.21.15.98:0/4288583464 shutdown_connections 2024-09-19T08:45:09.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.647+0000 7fe6ef5fc640 1 -- 172.21.15.98:0/4288583464 wait complete. 2024-09-19T08:45:09.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.647+0000 7fe6ef5fc640 1 Processor -- start 2024-09-19T08:45:09.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.647+0000 7fe6ef5fc640 1 -- start start 2024-09-19T08:45:09.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.647+0000 7fe6ef5fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6e806c1d0 0x7fe6e813d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:09.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.647+0000 7fe6ef5fc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6e80718f0 0x7fe6e8159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:09.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.647+0000 7fe6ef5fc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6e8145b70 0x7fe6e815fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:09.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.647+0000 7fe6ef5fc640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e81691f0 con 0x7fe6e806c1d0 2024-09-19T08:45:09.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.647+0000 7fe6ef5fc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e8169070 con 0x7fe6e8145b70 2024-09-19T08:45:09.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.647+0000 7fe6ef5fc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe6e8169370 con 0x7fe6e80718f0 2024-09-19T08:45:09.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.651+0000 7fe6ed398640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6e806c1d0 0x7fe6e813d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:09.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.651+0000 7fe6ed398640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6e806c1d0 0x7fe6e813d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59402/0 (socket says 172.21.15.98:59402) 2024-09-19T08:45:09.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.651+0000 7fe6ed398640 1 -- 172.21.15.98:0/3788919513 learned_addr learned my addr 172.21.15.98:0/3788919513 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:09.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.651+0000 7fe6ecb97640 1 --2- 172.21.15.98:0/3788919513 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6e80718f0 0x7fe6e8159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:09.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.651+0000 7fe6edb99640 1 --2- 172.21.15.98:0/3788919513 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6e8145b70 0x7fe6e815fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:09.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.651+0000 7fe6ed398640 1 -- 172.21.15.98:0/3788919513 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6e80718f0 msgr2=0x7fe6e8159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:09.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.651+0000 7fe6ed398640 1 --2- 172.21.15.98:0/3788919513 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6e80718f0 0x7fe6e8159c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:09.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.651+0000 7fe6ed398640 1 -- 172.21.15.98:0/3788919513 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6e8145b70 msgr2=0x7fe6e815fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:09.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.651+0000 7fe6ed398640 1 --2- 172.21.15.98:0/3788919513 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6e8145b70 0x7fe6e815fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:09.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.651+0000 7fe6ed398640 1 -- 172.21.15.98:0/3788919513 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e815a140 con 0x7fe6e806c1d0 2024-09-19T08:45:09.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.651+0000 7fe6edb99640 1 --2- 172.21.15.98:0/3788919513 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6e8145b70 0x7fe6e815fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:09.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.651+0000 7fe6ed398640 1 --2- 172.21.15.98:0/3788919513 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6e806c1d0 0x7fe6e813d8c0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fe6d800e8a0 tx=0x7fe6d800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:09.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.651+0000 7fe6ddffb640 1 -- 172.21.15.98:0/3788919513 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8017070 con 0x7fe6e806c1d0 2024-09-19T08:45:09.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.651+0000 7fe6ddffb640 1 -- 172.21.15.98:0/3788919513 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6d800c940 con 0x7fe6e806c1d0 2024-09-19T08:45:09.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.651+0000 7fe6ef5fc640 1 -- 172.21.15.98:0/3788919513 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6e81601f0 con 0x7fe6e806c1d0 2024-09-19T08:45:09.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.651+0000 7fe6ddffb640 1 -- 172.21.15.98:0/3788919513 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d800ccf0 con 0x7fe6e806c1d0 2024-09-19T08:45:09.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.651+0000 7fe6ef5fc640 1 -- 172.21.15.98:0/3788919513 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6e80cd440 con 0x7fe6e806c1d0 2024-09-19T08:45:09.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.651+0000 7fe6ef5fc640 1 -- 172.21.15.98:0/3788919513 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6e80713b0 con 0x7fe6e806c1d0 2024-09-19T08:45:09.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.651+0000 7fe6ddffb640 1 -- 172.21.15.98:0/3788919513 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7fe6d801a070 con 0x7fe6e806c1d0 2024-09-19T08:45:09.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.651+0000 7fe6ddffb640 1 --2- 172.21.15.98:0/3788919513 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe6cc067ce0 0x7fe6cc06a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:09.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.651+0000 7fe6ddffb640 1 -- 172.21.15.98:0/3788919513 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5795+0+0 (secure 0 0 0) 0x7fe6d80862c0 con 0x7fe6e806c1d0 2024-09-19T08:45:09.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.651+0000 7fe6ecb97640 1 --2- 172.21.15.98:0/3788919513 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe6cc067ce0 0x7fe6cc06a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:09.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.651+0000 7fe6ecb97640 1 --2- 172.21.15.98:0/3788919513 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe6cc067ce0 0x7fe6cc06a1a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe6e40ad000 tx=0x7fe6e4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:09.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.655+0000 7fe6ddffb640 1 -- 172.21.15.98:0/3788919513 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe6e81601f0 con 0x7fe6e806c1d0 2024-09-19T08:45:09.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.847+0000 7fe6ef5fc640 1 -- 172.21.15.98:0/3788919513 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol41980", "target": ["mon-mgr", ""]}) -- 0x7fe6e8142c30 con 0x7fe6cc067ce0 2024-09-19T08:45:09.861 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:09.863+0000 7ff73a69c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:45:09.862 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:09.863+0000 7ff73a69c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:45:09.862 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:09.863+0000 7ff73a69c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:45:09.862 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:09.863+0000 7ff73a69c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:45:09.862 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:09.863+0000 7ff73a69c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:45:09.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.895+0000 7fe6ddffb640 1 -- 172.21.15.98:0/3788919513 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7fe6e8142c30 con 0x7fe6cc067ce0 2024-09-19T08:45:09.897 INFO:teuthology.orchestra.run.smithi098.stdout:/volumes/_nogroup/subvol41980/66f5ddee-1364-4607-8f74-c9617e17c680 2024-09-19T08:45:09.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.899+0000 7fe6ef5fc640 1 -- 172.21.15.98:0/3788919513 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe6cc067ce0 msgr2=0x7fe6cc06a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:09.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.899+0000 7fe6ef5fc640 1 --2- 172.21.15.98:0/3788919513 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe6cc067ce0 0x7fe6cc06a1a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe6e40ad000 tx=0x7fe6e4008040 comp rx=0 tx=0).stop 2024-09-19T08:45:09.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.899+0000 7fe6ef5fc640 1 -- 172.21.15.98:0/3788919513 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6e806c1d0 msgr2=0x7fe6e813d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:09.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.899+0000 7fe6ef5fc640 1 --2- 172.21.15.98:0/3788919513 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6e806c1d0 0x7fe6e813d8c0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fe6d800e8a0 tx=0x7fe6d800eca0 comp rx=0 tx=0).stop 2024-09-19T08:45:09.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.899+0000 7fe6ef5fc640 1 -- 172.21.15.98:0/3788919513 shutdown_connections 2024-09-19T08:45:09.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.899+0000 7fe6ef5fc640 1 --2- 172.21.15.98:0/3788919513 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe6cc067ce0 0x7fe6cc06a1a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:09.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.899+0000 7fe6ef5fc640 1 --2- 172.21.15.98:0/3788919513 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6e8145b70 0x7fe6e815fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:09.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.899+0000 7fe6ef5fc640 1 --2- 172.21.15.98:0/3788919513 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe6e80718f0 0x7fe6e8159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:09.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.899+0000 7fe6ef5fc640 1 --2- 172.21.15.98:0/3788919513 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6e806c1d0 0x7fe6e813d8c0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:09.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.899+0000 7fe6ef5fc640 1 -- 172.21.15.98:0/3788919513 >> 172.21.15.98:0/3788919513 conn(0x7fe6e80b2010 msgr2=0x7fe6e80b0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:09.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.899+0000 7fe6ef5fc640 1 -- 172.21.15.98:0/3788919513 shutdown_connections 2024-09-19T08:45:09.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:09.899+0000 7fe6ef5fc640 1 -- 172.21.15.98:0/3788919513 wait complete. 2024-09-19T08:45:09.918 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol16391 subvol_grp33620 2024-09-19T08:45:10.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.039+0000 7ff1063e1640 1 Processor -- start 2024-09-19T08:45:10.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.039+0000 7ff1063e1640 1 -- start start 2024-09-19T08:45:10.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.039+0000 7ff1063e1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff10015c290 0x7ff1000cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.039+0000 7ff1063e1640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff10006d4c0 0x7ff10006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.039+0000 7ff1063e1640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff10006dde0 con 0x7ff1000cf700 2024-09-19T08:45:10.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.039+0000 7ff1063e1640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7ff100160e80 con 0x7ff10006d4c0 2024-09-19T08:45:10.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.039+0000 7ff1063e1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff1000cd510 con 0x7ff10015c290 2024-09-19T08:45:10.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.039+0000 7ff10497e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff10006d4c0 0x7ff10006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.039+0000 7ff0ff7fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7ff1000cf700 0x7ff10015ff50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56272/0 (socket says 172.21.15.98:56272) 2024-09-19T08:45:10.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.039+0000 7ff0ff7fe640 1 -- 172.21.15.98:0/3562056153 learned_addr learned my addr 172.21.15.98:0/3562056153 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:10.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.039+0000 7ff0fffff640 1 --2- 172.21.15.98:0/3562056153 >> v2:172.21.15.139:3300/0 conn(0x7ff10015c290 0x7ff1000cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.039+0000 7ff10497e640 1 -- 172.21.15.98:0/3562056153 >> v2:172.21.15.139:3300/0 conn(0x7ff10015c290 msgr2=0x7ff1000cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:10.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.039+0000 7ff10497e640 1 --2- 172.21.15.98:0/3562056153 >> v2:172.21.15.139:3300/0 conn(0x7ff10015c290 0x7ff1000cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.039+0000 7ff10497e640 1 -- 172.21.15.98:0/3562056153 >> v1:172.21.15.98:6789/0 conn(0x7ff1000cf700 legacy=0x7ff10015ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:10.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.039+0000 7ff10497e640 1 -- 172.21.15.98:0/3562056153 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1001608c0 con 0x7ff10006d4c0 2024-09-19T08:45:10.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.039+0000 7ff0feffd640 1 -- 172.21.15.98:0/3562056153 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2118584176 0 0) 0x7ff10006dde0 con 0x7ff1000cf700 2024-09-19T08:45:10.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.039+0000 7ff0fffff640 1 --2- 172.21.15.98:0/3562056153 >> v2:172.21.15.139:3300/0 conn(0x7ff10015c290 0x7ff1000cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:10.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.043+0000 7ff10497e640 1 --2- 172.21.15.98:0/3562056153 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff10006d4c0 0x7ff10006d8a0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7ff1000742e0 tx=0x7ff0f801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3513ff5e93a243c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:10.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.047+0000 7ff0feffd640 1 -- 172.21.15.98:0/3562056153 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f80b2070 con 0x7ff10006d4c0 2024-09-19T08:45:10.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.047+0000 7ff0feffd640 1 -- 172.21.15.98:0/3562056153 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0f80a5760 con 0x7ff10006d4c0 2024-09-19T08:45:10.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.047+0000 7ff0feffd640 1 -- 172.21.15.98:0/3562056153 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f80a5b30 con 0x7ff10006d4c0 2024-09-19T08:45:10.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.047+0000 7ff1063e1640 1 -- 172.21.15.98:0/3562056153 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff10006d4c0 msgr2=0x7ff10006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:10.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.047+0000 7ff1063e1640 1 --2- 172.21.15.98:0/3562056153 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff10006d4c0 0x7ff10006d8a0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7ff1000742e0 tx=0x7ff0f801c8a0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.047+0000 7ff1063e1640 1 -- 172.21.15.98:0/3562056153 shutdown_connections 2024-09-19T08:45:10.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.047+0000 7ff1063e1640 1 --2- 172.21.15.98:0/3562056153 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff10006d4c0 0x7ff10006d8a0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.047+0000 7ff1063e1640 1 --2- 172.21.15.98:0/3562056153 >> v2:172.21.15.139:3300/0 conn(0x7ff10015c290 0x7ff1000cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.047+0000 7ff1063e1640 1 -- 172.21.15.98:0/3562056153 >> 172.21.15.98:0/3562056153 conn(0x7ff10006c090 msgr2=0x7ff10006c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:10.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.047+0000 7ff1063e1640 1 -- 172.21.15.98:0/3562056153 shutdown_connections 2024-09-19T08:45:10.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.047+0000 7ff1063e1640 1 -- 172.21.15.98:0/3562056153 wait complete. 2024-09-19T08:45:10.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.047+0000 7ff1063e1640 1 Processor -- start 2024-09-19T08:45:10.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.047+0000 7ff1063e1640 1 -- start start 2024-09-19T08:45:10.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.047+0000 7ff1063e1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff10006d4c0 0x7ff100157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.047+0000 7ff1063e1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1000cf700 0x7ff100159590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.047+0000 7ff1063e1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff10015c290 0x7ff100157e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.047+0000 7ff1063e1640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff10015c7f0 con 0x7ff10006d4c0 2024-09-19T08:45:10.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.047+0000 7ff1063e1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff10015c670 con 0x7ff10015c290 2024-09-19T08:45:10.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.047+0000 7ff1063e1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff1000b5400 con 0x7ff1000cf700 2024-09-19T08:45:10.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.047+0000 7ff0fffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff10006d4c0 0x7ff100157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.047+0000 7ff0fffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff10006d4c0 0x7ff100157000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59424/0 (socket says 172.21.15.98:59424) 2024-09-19T08:45:10.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.047+0000 7ff0fffff640 1 -- 172.21.15.98:0/2749615653 learned_addr learned my addr 172.21.15.98:0/2749615653 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:10.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.047+0000 7ff0ff7fe640 1 --2- 172.21.15.98:0/2749615653 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1000cf700 0x7ff100159590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.047+0000 7ff10497e640 1 --2- 172.21.15.98:0/2749615653 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff10015c290 0x7ff100157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.047+0000 7ff0fffff640 1 -- 172.21.15.98:0/2749615653 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1000cf700 msgr2=0x7ff100159590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:10.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.047+0000 7ff0fffff640 1 --2- 172.21.15.98:0/2749615653 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1000cf700 0x7ff100159590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.047+0000 7ff0fffff640 1 -- 172.21.15.98:0/2749615653 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff10015c290 msgr2=0x7ff100157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:10.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.047+0000 7ff0fffff640 1 --2- 172.21.15.98:0/2749615653 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff10015c290 0x7ff100157e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.047+0000 7ff0fffff640 1 -- 172.21.15.98:0/2749615653 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff100158560 con 0x7ff10006d4c0 2024-09-19T08:45:10.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.047+0000 7ff0fffff640 1 --2- 172.21.15.98:0/2749615653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff10006d4c0 0x7ff100157000 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7ff0f000c7f0 tx=0x7ff0f000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:10.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.047+0000 7ff0fcff9640 1 -- 172.21.15.98:0/2749615653 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0015070 con 0x7ff10006d4c0 2024-09-19T08:45:10.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.047+0000 7ff0fcff9640 1 -- 172.21.15.98:0/2749615653 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0f00077c0 con 0x7ff10006d4c0 2024-09-19T08:45:10.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.047+0000 7ff1063e1640 1 -- 172.21.15.98:0/2749615653 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff100158840 con 0x7ff10006d4c0 2024-09-19T08:45:10.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.047+0000 7ff0fcff9640 1 -- 172.21.15.98:0/2749615653 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0007bb0 con 0x7ff10006d4c0 2024-09-19T08:45:10.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.047+0000 7ff1063e1640 1 -- 172.21.15.98:0/2749615653 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff100158e00 con 0x7ff10006d4c0 2024-09-19T08:45:10.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.051+0000 7ff1063e1640 1 -- 172.21.15.98:0/2749615653 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1000cd9a0 con 0x7ff10006d4c0 2024-09-19T08:45:10.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.051+0000 7ff0fcff9640 1 -- 172.21.15.98:0/2749615653 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7ff0f0018070 con 0x7ff10006d4c0 2024-09-19T08:45:10.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.051+0000 7ff0fcff9640 1 --2- 172.21.15.98:0/2749615653 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff0ec067cb0 0x7ff0ec06a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.051+0000 7ff0fcff9640 1 -- 172.21.15.98:0/2749615653 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5795+0+0 (secure 0 0 0) 0x7ff0f0086670 con 0x7ff10006d4c0 2024-09-19T08:45:10.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.051+0000 7ff0ff7fe640 1 --2- 172.21.15.98:0/2749615653 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff0ec067cb0 0x7ff0ec06a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.051+0000 7ff0ff7fe640 1 --2- 172.21.15.98:0/2749615653 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff0ec067cb0 0x7ff0ec06a170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff10015e960 tx=0x7ff0f4007b60 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:10.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.055+0000 7ff0fcff9640 1 -- 172.21.15.98:0/2749615653 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff0f0056a60 con 0x7ff10006d4c0 2024-09-19T08:45:10.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.231+0000 7ff1063e1640 1 -- 172.21.15.98:0/2749615653 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol16391", "group_name": "subvol_grp33620", "target": ["mon-mgr", ""]}) -- 0x7ff100163ec0 con 0x7ff0ec067cb0 2024-09-19T08:45:10.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.283+0000 7ff0fcff9640 1 -- 172.21.15.98:0/2749615653 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+74 (secure 0 0 0) 0x7ff0f40071f0 con 0x7ff0ec067cb0 2024-09-19T08:45:10.287 INFO:teuthology.orchestra.run.smithi098.stdout:/volumes/subvol_grp33620/subvol16391/f0511ac8-2f0c-4269-bb20-9ba71129e008 2024-09-19T08:45:10.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.287+0000 7ff1063e1640 1 -- 172.21.15.98:0/2749615653 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff0ec067cb0 msgr2=0x7ff0ec06a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:10.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.287+0000 7ff1063e1640 1 --2- 172.21.15.98:0/2749615653 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff0ec067cb0 0x7ff0ec06a170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff10015e960 tx=0x7ff0f4007b60 comp rx=0 tx=0).stop 2024-09-19T08:45:10.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.287+0000 7ff1063e1640 1 -- 172.21.15.98:0/2749615653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff10006d4c0 msgr2=0x7ff100157000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:10.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.287+0000 7ff1063e1640 1 --2- 172.21.15.98:0/2749615653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff10006d4c0 0x7ff100157000 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7ff0f000c7f0 tx=0x7ff0f000ccb0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.287+0000 7ff1063e1640 1 -- 172.21.15.98:0/2749615653 shutdown_connections 2024-09-19T08:45:10.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.287+0000 7ff1063e1640 1 --2- 172.21.15.98:0/2749615653 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff0ec067cb0 0x7ff0ec06a170 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.287+0000 7ff1063e1640 1 --2- 172.21.15.98:0/2749615653 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff10015c290 0x7ff100157e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.287+0000 7ff1063e1640 1 --2- 172.21.15.98:0/2749615653 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1000cf700 0x7ff100159590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.287+0000 7ff1063e1640 1 --2- 172.21.15.98:0/2749615653 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff10006d4c0 0x7ff100157000 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.287+0000 7ff1063e1640 1 -- 172.21.15.98:0/2749615653 >> 172.21.15.98:0/2749615653 conn(0x7ff10006c090 msgr2=0x7ff10015ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:10.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.287+0000 7ff1063e1640 1 -- 172.21.15.98:0/2749615653 shutdown_connections 2024-09-19T08:45:10.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.287+0000 7ff1063e1640 1 -- 172.21.15.98:0/2749615653 wait complete. 2024-09-19T08:45:10.305 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol56792 2024-09-19T08:45:10.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.423+0000 7f5fae8ac640 1 Processor -- start 2024-09-19T08:45:10.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.423+0000 7f5fae8ac640 1 -- start start 2024-09-19T08:45:10.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.423+0000 7f5fae8ac640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5fa8145b70 0x7f5fa806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.423+0000 7f5fae8ac640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f5fa8071ac0 0x7f5fa80b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.423+0000 7f5fae8ac640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fa806c5f0 con 0x7f5fa806c1d0 2024-09-19T08:45:10.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.423+0000 7f5fae8ac640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f5fa8071f20 con 0x7f5fa8071ac0 2024-09-19T08:45:10.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.423+0000 7f5fae8ac640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5fa8073530 con 0x7f5fa8145b70 2024-09-19T08:45:10.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.423+0000 7f5fa77fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f5fa806c1d0 0x7f5fa80713b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56288/0 (socket says 172.21.15.98:56288) 2024-09-19T08:45:10.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.423+0000 7f5fa77fe640 1 -- 172.21.15.98:0/3715362059 learned_addr learned my addr 172.21.15.98:0/3715362059 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:10.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.423+0000 7f5fa7fff640 1 --2- 172.21.15.98:0/3715362059 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5fa8145b70 0x7f5fa806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.423+0000 7f5face49640 1 --2- 172.21.15.98:0/3715362059 >> v2:172.21.15.139:3300/0 conn(0x7f5fa8071ac0 0x7f5fa80b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.423+0000 7f5fa6ffd640 1 -- 172.21.15.98:0/3715362059 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1827274772 0 0) 0x7f5fa806c5f0 con 0x7f5fa806c1d0 2024-09-19T08:45:10.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.423+0000 7f5fa6ffd640 1 -- 172.21.15.98:0/3715362059 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f90003530 con 0x7f5fa806c1d0 2024-09-19T08:45:10.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.423+0000 7f5fa6ffd640 1 -- 172.21.15.98:0/3715362059 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 784099298 0 0) 0x7f5f90003530 con 0x7f5fa806c1d0 2024-09-19T08:45:10.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.423+0000 7f5fa6ffd640 1 -- 172.21.15.98:0/3715362059 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5fa8145b70 msgr2=0x7f5fa806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:10.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.423+0000 7f5fa6ffd640 1 --2- 172.21.15.98:0/3715362059 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5fa8145b70 0x7f5fa806bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.423+0000 7f5fa6ffd640 1 -- 172.21.15.98:0/3715362059 >> v2:172.21.15.139:3300/0 conn(0x7f5fa8071ac0 msgr2=0x7f5fa80b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:10.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.423+0000 7f5fa6ffd640 1 --2- 172.21.15.98:0/3715362059 >> v2:172.21.15.139:3300/0 conn(0x7f5fa8071ac0 0x7f5fa80b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.423+0000 7f5fa6ffd640 1 -- 172.21.15.98:0/3715362059 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fa8073810 con 0x7f5fa806c1d0 2024-09-19T08:45:10.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.423+0000 7f5fa6ffd640 1 -- 172.21.15.98:0/3715362059 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f5f98001680 con 0x7f5fa806c1d0 2024-09-19T08:45:10.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.423+0000 7f5fa6ffd640 1 -- 172.21.15.98:0/3715362059 >> v1:172.21.15.98:6789/0 conn(0x7f5fa806c1d0 legacy=0x7f5fa80713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:10.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.423+0000 7f5fa6ffd640 1 --2- 172.21.15.98:0/3715362059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f900039c0 0x7f5f90003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.423+0000 7f5fa6ffd640 1 --2- 172.21.15.98:0/3715362059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f90004810 0x7f5f90004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.423+0000 7f5fa6ffd640 1 --2- 172.21.15.98:0/3715362059 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5fa8145b70 0x7f5f900051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.423+0000 7f5fa6ffd640 1 -- 172.21.15.98:0/3715362059 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa8071f20 con 0x7f5f900039c0 2024-09-19T08:45:10.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.423+0000 7f5fa6ffd640 1 -- 172.21.15.98:0/3715362059 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa8073530 con 0x7f5f90004810 2024-09-19T08:45:10.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.423+0000 7f5fa6ffd640 1 -- 172.21.15.98:0/3715362059 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5f9000ba80 con 0x7f5fa8145b70 2024-09-19T08:45:10.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.423+0000 7f5fa7fff640 1 --2- 172.21.15.98:0/3715362059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f900039c0 0x7f5f90003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.423+0000 7f5fa77fe640 1 --2- 172.21.15.98:0/3715362059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f90004810 0x7f5f90004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.423+0000 7f5fa7fff640 1 -- 172.21.15.98:0/3715362059 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5fa8145b70 msgr2=0x7f5f900051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:10.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.423+0000 7f5fa7fff640 1 --2- 172.21.15.98:0/3715362059 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5fa8145b70 0x7f5f900051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.423+0000 7f5fa7fff640 1 -- 172.21.15.98:0/3715362059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f90004810 msgr2=0x7f5f90004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:10.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.423+0000 7f5fa7fff640 1 --2- 172.21.15.98:0/3715362059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f90004810 0x7f5f90004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.423+0000 7f5fa7fff640 1 -- 172.21.15.98:0/3715362059 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f90003530 con 0x7f5f900039c0 2024-09-19T08:45:10.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.423+0000 7f5fa77fe640 1 --2- 172.21.15.98:0/3715362059 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f90004810 0x7f5f90004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:10.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.427+0000 7f5fa7fff640 1 --2- 172.21.15.98:0/3715362059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f900039c0 0x7f5f90003de0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c009ab0 tx=0x7f5f9c01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e6645125efcf12 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:10.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.427+0000 7f5fa6ffd640 1 -- 172.21.15.98:0/3715362059 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c0096b0 con 0x7f5f900039c0 2024-09-19T08:45:10.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.427+0000 7f5fa6ffd640 1 -- 172.21.15.98:0/3715362059 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f9c0b0400 con 0x7f5f900039c0 2024-09-19T08:45:10.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.427+0000 7f5fa6ffd640 1 -- 172.21.15.98:0/3715362059 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c0b07f0 con 0x7f5f900039c0 2024-09-19T08:45:10.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.427+0000 7f5fae8ac640 1 -- 172.21.15.98:0/3715362059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f900039c0 msgr2=0x7f5f90003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:10.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.427+0000 7f5fae8ac640 1 --2- 172.21.15.98:0/3715362059 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f900039c0 0x7f5f90003de0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c009ab0 tx=0x7f5f9c01c9b0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.427+0000 7f5face49640 1 -- 172.21.15.98:0/3715362059 reap_dead start 2024-09-19T08:45:10.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.427+0000 7f5fae8ac640 1 -- 172.21.15.98:0/3715362059 shutdown_connections 2024-09-19T08:45:10.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.427+0000 7f5fae8ac640 1 -- 172.21.15.98:0/3715362059 >> 172.21.15.98:0/3715362059 conn(0x7f5fa80b2010 msgr2=0x7f5fa80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:10.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.427+0000 7f5fae8ac640 1 -- 172.21.15.98:0/3715362059 shutdown_connections 2024-09-19T08:45:10.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.427+0000 7f5fae8ac640 1 -- 172.21.15.98:0/3715362059 wait complete. 2024-09-19T08:45:10.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.427+0000 7f5fae8ac640 1 Processor -- start 2024-09-19T08:45:10.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.427+0000 7f5fae8ac640 1 -- start start 2024-09-19T08:45:10.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.427+0000 7f5fae8ac640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5fa815bc90 0x7f5fa815fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.427+0000 7f5fae8ac640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5fa815c070 0x7f5fa81f7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.427+0000 7f5fae8ac640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5fa80ae8b0 0x7f5fa80aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.427+0000 7f5fae8ac640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa80b42e0 con 0x7f5fa815c070 2024-09-19T08:45:10.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.427+0000 7f5fae8ac640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa80b4160 con 0x7f5fa815bc90 2024-09-19T08:45:10.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.427+0000 7f5fae8ac640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5fa80b4460 con 0x7f5fa80ae8b0 2024-09-19T08:45:10.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.427+0000 7f5fa77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5fa815c070 0x7f5fa81f7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.427+0000 7f5fa77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5fa815c070 0x7f5fa81f7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59444/0 (socket says 172.21.15.98:59444) 2024-09-19T08:45:10.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.427+0000 7f5fa77fe640 1 -- 172.21.15.98:0/882195091 learned_addr learned my addr 172.21.15.98:0/882195091 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:10.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.431+0000 7f5fa7fff640 1 --2- 172.21.15.98:0/882195091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5fa815bc90 0x7f5fa815fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.431+0000 7f5face49640 1 --2- 172.21.15.98:0/882195091 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5fa80ae8b0 0x7f5fa80aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.431+0000 7f5fa77fe640 1 -- 172.21.15.98:0/882195091 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5fa80ae8b0 msgr2=0x7f5fa80aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:10.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.431+0000 7f5fa77fe640 1 --2- 172.21.15.98:0/882195091 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5fa80ae8b0 0x7f5fa80aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.431+0000 7f5fa77fe640 1 -- 172.21.15.98:0/882195091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5fa815bc90 msgr2=0x7f5fa815fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:10.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.431+0000 7f5fa77fe640 1 --2- 172.21.15.98:0/882195091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5fa815bc90 0x7f5fa815fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.431+0000 7f5fa77fe640 1 -- 172.21.15.98:0/882195091 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fa81601c0 con 0x7f5fa815c070 2024-09-19T08:45:10.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.431+0000 7f5fa77fe640 1 --2- 172.21.15.98:0/882195091 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5fa815c070 0x7f5fa81f7b90 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f5f9800d930 tx=0x7f5f9800ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:10.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.431+0000 7f5fa4ff9640 1 -- 172.21.15.98:0/882195091 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f98019070 con 0x7f5fa815c070 2024-09-19T08:45:10.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.431+0000 7f5fa4ff9640 1 -- 172.21.15.98:0/882195091 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f98004cc0 con 0x7f5fa815c070 2024-09-19T08:45:10.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.431+0000 7f5fa4ff9640 1 -- 172.21.15.98:0/882195091 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f98007310 con 0x7f5fa815c070 2024-09-19T08:45:10.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.431+0000 7f5fae8ac640 1 -- 172.21.15.98:0/882195091 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fa81fb520 con 0x7f5fa815c070 2024-09-19T08:45:10.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.431+0000 7f5fae8ac640 1 -- 172.21.15.98:0/882195091 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fa81fbae0 con 0x7f5fa815c070 2024-09-19T08:45:10.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.431+0000 7f5fae8ac640 1 -- 172.21.15.98:0/882195091 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fa80b3410 con 0x7f5fa815c070 2024-09-19T08:45:10.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.431+0000 7f5fa4ff9640 1 -- 172.21.15.98:0/882195091 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7f5f980074b0 con 0x7f5fa815c070 2024-09-19T08:45:10.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.431+0000 7f5fa4ff9640 1 --2- 172.21.15.98:0/882195091 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5f94067cb0 0x7f5f9406a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.431+0000 7f5fa4ff9640 1 -- 172.21.15.98:0/882195091 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5795+0+0 (secure 0 0 0) 0x7f5f98085fe0 con 0x7f5fa815c070 2024-09-19T08:45:10.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.435+0000 7f5fa4ff9640 1 -- 172.21.15.98:0/882195091 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f5f98056520 con 0x7f5fa815c070 2024-09-19T08:45:10.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.443+0000 7f5fa7fff640 1 --2- 172.21.15.98:0/882195091 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5f94067cb0 0x7f5f9406a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.443+0000 7f5fa7fff640 1 --2- 172.21.15.98:0/882195091 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5f94067cb0 0x7f5f9406a170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c0af000 tx=0x7f5f9c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:10.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.619+0000 7f5fae8ac640 1 -- 172.21.15.98:0/882195091 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol56792", "target": ["mon-mgr", ""]}) -- 0x7f5fa80b1960 con 0x7f5f94067cb0 2024-09-19T08:45:10.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.655+0000 7f5fa4ff9640 1 -- 172.21.15.98:0/882195091 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+144 (secure 0 0 0) 0x7f5fa80b1960 con 0x7f5f94067cb0 2024-09-19T08:45:10.660 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:45:10.660 INFO:teuthology.orchestra.run.smithi098.stdout: "status": { 2024-09-19T08:45:10.660 INFO:teuthology.orchestra.run.smithi098.stdout: "state": "pending", 2024-09-19T08:45:10.660 INFO:teuthology.orchestra.run.smithi098.stdout: "source": { 2024-09-19T08:45:10.660 INFO:teuthology.orchestra.run.smithi098.stdout: "volume": "fake", 2024-09-19T08:45:10.660 INFO:teuthology.orchestra.run.smithi098.stdout: "subvolume": "fake", 2024-09-19T08:45:10.660 INFO:teuthology.orchestra.run.smithi098.stdout: "snapshot": "fake" 2024-09-19T08:45:10.660 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:45:10.661 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:45:10.661 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:45:10.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.659+0000 7f5fae8ac640 1 -- 172.21.15.98:0/882195091 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5f94067cb0 msgr2=0x7f5f9406a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:10.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.659+0000 7f5fae8ac640 1 --2- 172.21.15.98:0/882195091 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5f94067cb0 0x7f5f9406a170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c0af000 tx=0x7f5f9c008040 comp rx=0 tx=0).stop 2024-09-19T08:45:10.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.659+0000 7f5fae8ac640 1 -- 172.21.15.98:0/882195091 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5fa815c070 msgr2=0x7f5fa81f7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:10.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.659+0000 7f5fae8ac640 1 --2- 172.21.15.98:0/882195091 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5fa815c070 0x7f5fa81f7b90 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f5f9800d930 tx=0x7f5f9800ddf0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.659+0000 7f5fae8ac640 1 -- 172.21.15.98:0/882195091 shutdown_connections 2024-09-19T08:45:10.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.659+0000 7f5fae8ac640 1 --2- 172.21.15.98:0/882195091 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5f94067cb0 0x7f5f9406a170 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.659+0000 7f5fae8ac640 1 --2- 172.21.15.98:0/882195091 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5fa80ae8b0 0x7f5fa80aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.659+0000 7f5fae8ac640 1 --2- 172.21.15.98:0/882195091 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5fa815c070 0x7f5fa81f7b90 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.659+0000 7f5fae8ac640 1 --2- 172.21.15.98:0/882195091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5fa815bc90 0x7f5fa815fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.659+0000 7f5fae8ac640 1 -- 172.21.15.98:0/882195091 >> 172.21.15.98:0/882195091 conn(0x7f5fa80b2010 msgr2=0x7f5fa80cf6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:10.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.659+0000 7f5fae8ac640 1 -- 172.21.15.98:0/882195091 shutdown_connections 2024-09-19T08:45:10.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.659+0000 7f5fae8ac640 1 -- 172.21.15.98:0/882195091 wait complete. 2024-09-19T08:45:10.677 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol41980 fake 2024-09-19T08:45:10.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.799+0000 7fb475128640 1 Processor -- start 2024-09-19T08:45:10.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.799+0000 7fb475128640 1 -- start start 2024-09-19T08:45:10.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.799+0000 7fb475128640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb47015e690 0x7fb4700b6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.799+0000 7fb475128640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4701630b0 con 0x7fb4700d1da0 2024-09-19T08:45:10.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.799+0000 7fb475128640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb470073fc0 con 0x7fb4700ceb20 2024-09-19T08:45:10.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.799+0000 7fb475128640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb470071bc0 con 0x7fb47015e690 2024-09-19T08:45:10.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.803+0000 7fb46f59e640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fb4700d1da0 0x7fb4701629a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56294/0 (socket says 172.21.15.98:56294) 2024-09-19T08:45:10.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.803+0000 7fb46f59e640 1 -- 172.21.15.98:0/3890654769 learned_addr learned my addr 172.21.15.98:0/3890654769 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:10.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.803+0000 7fb46ed9d640 1 --2- 172.21.15.98:0/3890654769 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb47015e690 0x7fb4700b6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.803+0000 7fb46dd9b640 1 -- 172.21.15.98:0/3890654769 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2397120967 0 0) 0x7fb4701630b0 con 0x7fb4700d1da0 2024-09-19T08:45:10.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.803+0000 7fb46dd9b640 1 -- 172.21.15.98:0/3890654769 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb45c003530 con 0x7fb4700d1da0 2024-09-19T08:45:10.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.803+0000 7fb46dd9b640 1 -- 172.21.15.98:0/3890654769 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 241345700 0 0) 0x7fb45c003530 con 0x7fb4700d1da0 2024-09-19T08:45:10.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.803+0000 7fb46dd9b640 1 -- 172.21.15.98:0/3890654769 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb47015e690 msgr2=0x7fb4700b6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:10.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.803+0000 7fb46dd9b640 1 --2- 172.21.15.98:0/3890654769 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb47015e690 0x7fb4700b6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.803+0000 7fb46dd9b640 1 -- 172.21.15.98:0/3890654769 >> v1:172.21.15.139:6789/0 conn(0x7fb4700ceb20 legacy=0x7fb470162290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:10.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.803+0000 7fb46dd9b640 1 -- 172.21.15.98:0/3890654769 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb47006d5d0 con 0x7fb4700d1da0 2024-09-19T08:45:10.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.803+0000 7fb46dd9b640 1 -- 172.21.15.98:0/3890654769 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fb460001a00 con 0x7fb4700d1da0 2024-09-19T08:45:10.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.803+0000 7fb46dd9b640 1 -- 172.21.15.98:0/3890654769 >> v1:172.21.15.98:6789/0 conn(0x7fb4700d1da0 legacy=0x7fb4701629a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:10.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.803+0000 7fb46dd9b640 1 --2- 172.21.15.98:0/3890654769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb45c0039f0 0x7fb45c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.803+0000 7fb46dd9b640 1 --2- 172.21.15.98:0/3890654769 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb47015e690 0x7fb45c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.803+0000 7fb46dd9b640 1 --2- 172.21.15.98:0/3890654769 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb45c004d50 0x7fb45c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.803+0000 7fb46dd9b640 1 -- 172.21.15.98:0/3890654769 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c00bb60 con 0x7fb45c004d50 2024-09-19T08:45:10.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.803+0000 7fb46dd9b640 1 -- 172.21.15.98:0/3890654769 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c00bce0 con 0x7fb45c0039f0 2024-09-19T08:45:10.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.803+0000 7fb46dd9b640 1 -- 172.21.15.98:0/3890654769 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb45c00bee0 con 0x7fb47015e690 2024-09-19T08:45:10.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.803+0000 7fb46f59e640 1 --2- 172.21.15.98:0/3890654769 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb45c004d50 0x7fb45c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.803+0000 7fb46ed9d640 1 --2- 172.21.15.98:0/3890654769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb45c0039f0 0x7fb45c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.803+0000 7fb46f59e640 1 -- 172.21.15.98:0/3890654769 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb47015e690 msgr2=0x7fb45c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:10.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.803+0000 7fb46e59c640 1 --2- 172.21.15.98:0/3890654769 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb47015e690 0x7fb45c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.803+0000 7fb46f59e640 1 --2- 172.21.15.98:0/3890654769 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb47015e690 0x7fb45c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.803+0000 7fb46f59e640 1 -- 172.21.15.98:0/3890654769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb45c0039f0 msgr2=0x7fb45c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:10.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.803+0000 7fb46f59e640 1 --2- 172.21.15.98:0/3890654769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb45c0039f0 0x7fb45c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.803+0000 7fb46f59e640 1 -- 172.21.15.98:0/3890654769 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb45c003530 con 0x7fb45c004d50 2024-09-19T08:45:10.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.803+0000 7fb46e59c640 1 --2- 172.21.15.98:0/3890654769 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb47015e690 0x7fb45c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:10.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.803+0000 7fb46f59e640 1 --2- 172.21.15.98:0/3890654769 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb45c004d50 0x7fb45c0051d0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fb46000a7b0 tx=0x7fb46001fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3a650d9c976569b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:10.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.803+0000 7fb46dd9b640 1 -- 172.21.15.98:0/3890654769 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46000a3d0 con 0x7fb45c004d50 2024-09-19T08:45:10.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.803+0000 7fb46dd9b640 1 -- 172.21.15.98:0/3890654769 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4600aa690 con 0x7fb45c004d50 2024-09-19T08:45:10.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.803+0000 7fb46dd9b640 1 -- 172.21.15.98:0/3890654769 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4600aaac0 con 0x7fb45c004d50 2024-09-19T08:45:10.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.807+0000 7fb475128640 1 -- 172.21.15.98:0/3890654769 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb45c004d50 msgr2=0x7fb45c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:10.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.807+0000 7fb475128640 1 --2- 172.21.15.98:0/3890654769 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb45c004d50 0x7fb45c0051d0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fb46000a7b0 tx=0x7fb46001fa40 comp rx=0 tx=0).stop 2024-09-19T08:45:10.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.807+0000 7fb46f59e640 1 -- 172.21.15.98:0/3890654769 reap_dead start 2024-09-19T08:45:10.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.807+0000 7fb475128640 1 -- 172.21.15.98:0/3890654769 shutdown_connections 2024-09-19T08:45:10.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.807+0000 7fb475128640 1 -- 172.21.15.98:0/3890654769 >> 172.21.15.98:0/3890654769 conn(0x7fb4700b4260 msgr2=0x7fb4700b4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:10.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.807+0000 7fb475128640 1 -- 172.21.15.98:0/3890654769 shutdown_connections 2024-09-19T08:45:10.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.807+0000 7fb475128640 1 -- 172.21.15.98:0/3890654769 wait complete. 2024-09-19T08:45:10.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.807+0000 7fb475128640 1 Processor -- start 2024-09-19T08:45:10.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.807+0000 7fb475128640 1 -- start start 2024-09-19T08:45:10.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.807+0000 7fb475128640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb470161b10 0x7fb47015f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.807+0000 7fb475128640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb47015fcf0 0x7fb4701513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.807+0000 7fb475128640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb470151920 0x7fb4701437e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.807+0000 7fb475128640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb47006db90 con 0x7fb470151920 2024-09-19T08:45:10.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.807+0000 7fb475128640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb4700742a0 con 0x7fb470161b10 2024-09-19T08:45:10.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.807+0000 7fb475128640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb47015ea70 con 0x7fb47015fcf0 2024-09-19T08:45:10.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.807+0000 7fb46f59e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb470151920 0x7fb4701437e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.807+0000 7fb46f59e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb470151920 0x7fb4701437e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59468/0 (socket says 172.21.15.98:59468) 2024-09-19T08:45:10.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.807+0000 7fb46f59e640 1 -- 172.21.15.98:0/4274889017 learned_addr learned my addr 172.21.15.98:0/4274889017 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:10.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.807+0000 7fb46e59c640 1 --2- 172.21.15.98:0/4274889017 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb47015fcf0 0x7fb4701513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.807+0000 7fb46ed9d640 1 --2- 172.21.15.98:0/4274889017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb470161b10 0x7fb47015f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.807+0000 7fb46f59e640 1 -- 172.21.15.98:0/4274889017 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb47015fcf0 msgr2=0x7fb4701513e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:10.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.807+0000 7fb46f59e640 1 --2- 172.21.15.98:0/4274889017 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb47015fcf0 0x7fb4701513e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.807+0000 7fb46f59e640 1 -- 172.21.15.98:0/4274889017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb470161b10 msgr2=0x7fb47015f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:10.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.807+0000 7fb46f59e640 1 --2- 172.21.15.98:0/4274889017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb470161b10 0x7fb47015f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:10.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.807+0000 7fb46f59e640 1 -- 172.21.15.98:0/4274889017 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb470154010 con 0x7fb470151920 2024-09-19T08:45:10.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.807+0000 7fb46e59c640 1 --2- 172.21.15.98:0/4274889017 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb47015fcf0 0x7fb4701513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:10.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.807+0000 7fb46f59e640 1 --2- 172.21.15.98:0/4274889017 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb470151920 0x7fb4701437e0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fb46000a7b0 tx=0x7fb46001c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:10.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.807+0000 7fb4677fe640 1 -- 172.21.15.98:0/4274889017 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46000a3d0 con 0x7fb470151920 2024-09-19T08:45:10.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.807+0000 7fb4677fe640 1 -- 172.21.15.98:0/4274889017 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4600011d0 con 0x7fb470151920 2024-09-19T08:45:10.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.807+0000 7fb475128640 1 -- 172.21.15.98:0/4274889017 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4701542f0 con 0x7fb470151920 2024-09-19T08:45:10.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.807+0000 7fb4677fe640 1 -- 172.21.15.98:0/4274889017 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4600aaa40 con 0x7fb470151920 2024-09-19T08:45:10.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.807+0000 7fb475128640 1 -- 172.21.15.98:0/4274889017 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb470161ef0 con 0x7fb470151920 2024-09-19T08:45:10.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.811+0000 7fb475128640 1 -- 172.21.15.98:0/4274889017 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb47006bc90 con 0x7fb470151920 2024-09-19T08:45:10.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.811+0000 7fb4677fe640 1 -- 172.21.15.98:0/4274889017 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 19) ==== 87802+0+0 (secure 0 0 0) 0x7fb4600ba070 con 0x7fb470151920 2024-09-19T08:45:10.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.811+0000 7fb4677fe640 1 --2- 172.21.15.98:0/4274889017 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb450067ce0 0x7fb45006a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:10.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.811+0000 7fb4677fe640 1 -- 172.21.15.98:0/4274889017 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5795+0+0 (secure 0 0 0) 0x7fb460123ce0 con 0x7fb470151920 2024-09-19T08:45:10.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.811+0000 7fb46ed9d640 1 --2- 172.21.15.98:0/4274889017 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb450067ce0 0x7fb45006a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:10.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.811+0000 7fb46ed9d640 1 --2- 172.21.15.98:0/4274889017 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb450067ce0 0x7fb45006a1a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb468009a30 tx=0x7fb468008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:10.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.815+0000 7fb4677fe640 1 -- 172.21.15.98:0/4274889017 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb4600c2060 con 0x7fb470151920 2024-09-19T08:45:11.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:10.999+0000 7fb4677fe640 1 -- 172.21.15.98:0/4274889017 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mgrmap(e 20) ==== 87853+0+0 (secure 0 0 0) 0x7fb4600f00d0 con 0x7fb470151920 2024-09-19T08:45:11.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.011+0000 7fb475128640 1 -- 172.21.15.98:0/4274889017 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol41980", "snap_name": "fake", "target": ["mon-mgr", ""]}) -- 0x7fb4700b1920 con 0x7fb450067ce0 2024-09-19T08:45:11.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.075+0000 7fb4677fe640 1 -- 172.21.15.98:0/4274889017 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb4700b1920 con 0x7fb450067ce0 2024-09-19T08:45:11.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.075+0000 7fb475128640 1 -- 172.21.15.98:0/4274889017 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb450067ce0 msgr2=0x7fb45006a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:11.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.075+0000 7fb475128640 1 --2- 172.21.15.98:0/4274889017 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb450067ce0 0x7fb45006a1a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb468009a30 tx=0x7fb468008040 comp rx=0 tx=0).stop 2024-09-19T08:45:11.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.075+0000 7fb475128640 1 -- 172.21.15.98:0/4274889017 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb470151920 msgr2=0x7fb4701437e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:11.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.075+0000 7fb475128640 1 --2- 172.21.15.98:0/4274889017 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb470151920 0x7fb4701437e0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fb46000a7b0 tx=0x7fb46001c7e0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.075+0000 7fb475128640 1 -- 172.21.15.98:0/4274889017 shutdown_connections 2024-09-19T08:45:11.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.075+0000 7fb475128640 1 --2- 172.21.15.98:0/4274889017 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb450067ce0 0x7fb45006a1a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.075+0000 7fb475128640 1 --2- 172.21.15.98:0/4274889017 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb470151920 0x7fb4701437e0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.075+0000 7fb475128640 1 --2- 172.21.15.98:0/4274889017 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb47015fcf0 0x7fb4701513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.075+0000 7fb475128640 1 --2- 172.21.15.98:0/4274889017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb470161b10 0x7fb47015f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.075+0000 7fb475128640 1 -- 172.21.15.98:0/4274889017 >> 172.21.15.98:0/4274889017 conn(0x7fb4700b4260 msgr2=0x7fb4700b3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:11.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.079+0000 7fb475128640 1 -- 172.21.15.98:0/4274889017 shutdown_connections 2024-09-19T08:45:11.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.079+0000 7fb475128640 1 -- 172.21.15.98:0/4274889017 wait complete. 2024-09-19T08:45:11.099 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol16391 fake subvol_grp33620 2024-09-19T08:45:11.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.223+0000 7f9eb15db640 1 Processor -- start 2024-09-19T08:45:11.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.223+0000 7f9eb15db640 1 -- start start 2024-09-19T08:45:11.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.223+0000 7f9eb15db640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9eac06c3a0 0x7f9eac0713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:11.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.223+0000 7f9eb15db640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9eac0718f0 0x7f9eac0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:11.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.223+0000 7f9eb15db640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9eac073530 con 0x7f9eac145b70 2024-09-19T08:45:11.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.223+0000 7f9eb15db640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f9eac073810 con 0x7f9eac06c3a0 2024-09-19T08:45:11.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.223+0000 7f9eb15db640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f9eac073af0 con 0x7f9eac0718f0 2024-09-19T08:45:11.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.223+0000 7f9eaa7fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9eac06c3a0 0x7f9eac0713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:11.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.223+0000 7f9eaaffd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f9eac145b70 0x7f9eac06bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56304/0 (socket says 172.21.15.98:56304) 2024-09-19T08:45:11.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.223+0000 7f9eaaffd640 1 -- 172.21.15.98:0/4287763468 learned_addr learned my addr 172.21.15.98:0/4287763468 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:11.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.223+0000 7f9eaa7fc640 1 -- 172.21.15.98:0/4287763468 >> v2:172.21.15.139:3300/0 conn(0x7f9eac0718f0 msgr2=0x7f9eac0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:11.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.223+0000 7f9eaa7fc640 1 --2- 172.21.15.98:0/4287763468 >> v2:172.21.15.139:3300/0 conn(0x7f9eac0718f0 0x7f9eac0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.223+0000 7f9eaa7fc640 1 -- 172.21.15.98:0/4287763468 >> v1:172.21.15.98:6789/0 conn(0x7f9eac145b70 legacy=0x7f9eac06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:11.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.223+0000 7f9ea9ffb640 1 -- 172.21.15.98:0/4287763468 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3492433647 0 0) 0x7f9eac073530 con 0x7f9eac145b70 2024-09-19T08:45:11.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.223+0000 7f9eaa7fc640 1 -- 172.21.15.98:0/4287763468 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eac073dd0 con 0x7f9eac06c3a0 2024-09-19T08:45:11.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.227+0000 7f9eaa7fc640 1 --2- 172.21.15.98:0/4287763468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9eac06c3a0 0x7f9eac0713b0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f9eac072e60 tx=0x7f9e9401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f7d55857d1b448 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:11.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.227+0000 7f9ea9ffb640 1 -- 172.21.15.98:0/4287763468 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e940b0070 con 0x7f9eac06c3a0 2024-09-19T08:45:11.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.227+0000 7f9ea9ffb640 1 -- 172.21.15.98:0/4287763468 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e940ab600 con 0x7f9eac06c3a0 2024-09-19T08:45:11.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.227+0000 7f9ea9ffb640 1 -- 172.21.15.98:0/4287763468 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e940ab9d0 con 0x7f9eac06c3a0 2024-09-19T08:45:11.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.227+0000 7f9eb15db640 1 -- 172.21.15.98:0/4287763468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9eac06c3a0 msgr2=0x7f9eac0713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:11.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.227+0000 7f9eb15db640 1 --2- 172.21.15.98:0/4287763468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9eac06c3a0 0x7f9eac0713b0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f9eac072e60 tx=0x7f9e9401c3f0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.227+0000 7f9eb15db640 1 -- 172.21.15.98:0/4287763468 shutdown_connections 2024-09-19T08:45:11.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.227+0000 7f9eb15db640 1 --2- 172.21.15.98:0/4287763468 >> v2:172.21.15.139:3300/0 conn(0x7f9eac0718f0 0x7f9eac0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.227+0000 7f9eb15db640 1 --2- 172.21.15.98:0/4287763468 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9eac06c3a0 0x7f9eac0713b0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.227+0000 7f9eb15db640 1 -- 172.21.15.98:0/4287763468 >> 172.21.15.98:0/4287763468 conn(0x7f9eac0b2010 msgr2=0x7f9eac0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:11.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.227+0000 7f9eb15db640 1 -- 172.21.15.98:0/4287763468 shutdown_connections 2024-09-19T08:45:11.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.227+0000 7f9eb15db640 1 -- 172.21.15.98:0/4287763468 wait complete. 2024-09-19T08:45:11.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.227+0000 7f9eb15db640 1 Processor -- start 2024-09-19T08:45:11.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.227+0000 7f9eb15db640 1 -- start start 2024-09-19T08:45:11.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.227+0000 7f9eb15db640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9eac06c3a0 0x7f9eac13bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:11.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.227+0000 7f9eb15db640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9eac0718f0 0x7f9eac158ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:11.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.227+0000 7f9eb15db640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9eac145b70 0x7f9eac159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:11.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.227+0000 7f9eb15db640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac0b44f0 con 0x7f9eac06c3a0 2024-09-19T08:45:11.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.227+0000 7f9eb15db640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac0b4370 con 0x7f9eac145b70 2024-09-19T08:45:11.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.227+0000 7f9eb15db640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9eac0b4670 con 0x7f9eac0718f0 2024-09-19T08:45:11.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.227+0000 7f9eaaffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9eac06c3a0 0x7f9eac13bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:11.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.227+0000 7f9eaaffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9eac06c3a0 0x7f9eac13bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59476/0 (socket says 172.21.15.98:59476) 2024-09-19T08:45:11.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.227+0000 7f9eaaffd640 1 -- 172.21.15.98:0/3103104837 learned_addr learned my addr 172.21.15.98:0/3103104837 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:11.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.227+0000 7f9eab7fe640 1 --2- 172.21.15.98:0/3103104837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9eac145b70 0x7f9eac159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:11.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.227+0000 7f9eaaffd640 1 -- 172.21.15.98:0/3103104837 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9eac0718f0 msgr2=0x7f9eac158ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:11.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.227+0000 7f9eaa7fc640 1 --2- 172.21.15.98:0/3103104837 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9eac0718f0 0x7f9eac158ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:11.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.227+0000 7f9eaaffd640 1 --2- 172.21.15.98:0/3103104837 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9eac0718f0 0x7f9eac158ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.227+0000 7f9eaaffd640 1 -- 172.21.15.98:0/3103104837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9eac145b70 msgr2=0x7f9eac159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:11.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.227+0000 7f9eaaffd640 1 --2- 172.21.15.98:0/3103104837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9eac145b70 0x7f9eac159420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.227+0000 7f9eaaffd640 1 -- 172.21.15.98:0/3103104837 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eac141b40 con 0x7f9eac06c3a0 2024-09-19T08:45:11.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.227+0000 7f9eaaffd640 1 --2- 172.21.15.98:0/3103104837 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9eac06c3a0 0x7f9eac13bcc0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f9ea400c4b0 tx=0x7f9ea400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:11.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.231+0000 7f9ea37fe640 1 -- 172.21.15.98:0/3103104837 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4018070 con 0x7f9eac06c3a0 2024-09-19T08:45:11.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.231+0000 7f9ea37fe640 1 -- 172.21.15.98:0/3103104837 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ea4007500 con 0x7f9eac06c3a0 2024-09-19T08:45:11.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.231+0000 7f9eb15db640 1 -- 172.21.15.98:0/3103104837 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eac141e20 con 0x7f9eac06c3a0 2024-09-19T08:45:11.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.231+0000 7f9ea37fe640 1 -- 172.21.15.98:0/3103104837 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4004a30 con 0x7f9eac06c3a0 2024-09-19T08:45:11.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.231+0000 7f9eb15db640 1 -- 172.21.15.98:0/3103104837 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eac13c8c0 con 0x7f9eac06c3a0 2024-09-19T08:45:11.235 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.231+0000 7f9eb15db640 1 -- 172.21.15.98:0/3103104837 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eac0713b0 con 0x7f9eac06c3a0 2024-09-19T08:45:11.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.231+0000 7f9ea37fe640 1 -- 172.21.15.98:0/3103104837 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 87853+0+0 (secure 0 0 0) 0x7f9ea401b070 con 0x7f9eac06c3a0 2024-09-19T08:45:11.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.231+0000 7f9ea37fe640 1 --2- 172.21.15.98:0/3103104837 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9e8c067d40 0x7f9e8c06a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:11.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.231+0000 7f9ea37fe640 1 -- 172.21.15.98:0/3103104837 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5795+0+0 (secure 0 0 0) 0x7f9ea4085680 con 0x7f9eac06c3a0 2024-09-19T08:45:11.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.231+0000 7f9eaa7fc640 1 --2- 172.21.15.98:0/3103104837 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9e8c067d40 0x7f9e8c06a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:11.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.231+0000 7f9eaa7fc640 1 --2- 172.21.15.98:0/3103104837 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9e8c067d40 0x7f9e8c06a200 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9e940ad000 tx=0x7f9e94008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:11.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.235+0000 7f9ea37fe640 1 -- 172.21.15.98:0/3103104837 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9ea4055c30 con 0x7f9eac06c3a0 2024-09-19T08:45:11.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.427+0000 7f9eb15db640 1 -- 172.21.15.98:0/3103104837 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol16391", "snap_name": "fake", "group_name": "subvol_grp33620", "target": ["mon-mgr", ""]}) -- 0x7f9eac143540 con 0x7f9e8c067d40 2024-09-19T08:45:11.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.487+0000 7f9ea37fe640 1 -- 172.21.15.98:0/3103104837 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9eac143540 con 0x7f9e8c067d40 2024-09-19T08:45:11.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.491+0000 7f9eb15db640 1 -- 172.21.15.98:0/3103104837 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9e8c067d40 msgr2=0x7f9e8c06a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:11.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.491+0000 7f9eb15db640 1 --2- 172.21.15.98:0/3103104837 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9e8c067d40 0x7f9e8c06a200 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9e940ad000 tx=0x7f9e94008040 comp rx=0 tx=0).stop 2024-09-19T08:45:11.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.491+0000 7f9eb15db640 1 -- 172.21.15.98:0/3103104837 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9eac06c3a0 msgr2=0x7f9eac13bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:11.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.491+0000 7f9eb15db640 1 --2- 172.21.15.98:0/3103104837 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9eac06c3a0 0x7f9eac13bcc0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f9ea400c4b0 tx=0x7f9ea400c970 comp rx=0 tx=0).stop 2024-09-19T08:45:11.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.491+0000 7f9eb15db640 1 -- 172.21.15.98:0/3103104837 shutdown_connections 2024-09-19T08:45:11.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.491+0000 7f9eb15db640 1 --2- 172.21.15.98:0/3103104837 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9e8c067d40 0x7f9e8c06a200 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.491+0000 7f9eb15db640 1 --2- 172.21.15.98:0/3103104837 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9eac145b70 0x7f9eac159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.491+0000 7f9eb15db640 1 --2- 172.21.15.98:0/3103104837 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9eac0718f0 0x7f9eac158ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.491+0000 7f9eb15db640 1 --2- 172.21.15.98:0/3103104837 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9eac06c3a0 0x7f9eac13bcc0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.491+0000 7f9eb15db640 1 -- 172.21.15.98:0/3103104837 >> 172.21.15.98:0/3103104837 conn(0x7f9eac0b2010 msgr2=0x7f9eac160440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:11.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.491+0000 7f9eb15db640 1 -- 172.21.15.98:0/3103104837 shutdown_connections 2024-09-19T08:45:11.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.491+0000 7f9eb15db640 1 -- 172.21.15.98:0/3103104837 wait complete. 2024-09-19T08:45:11.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:11.511 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol41980/.meta) 2024-09-19T08:45:11.529 INFO:teuthology.orchestra.run.smithi098.stdout:[GLOBAL] 2024-09-19T08:45:11.529 INFO:teuthology.orchestra.run.smithi098.stdout:version = 1 2024-09-19T08:45:11.529 INFO:teuthology.orchestra.run.smithi098.stdout:type = subvolume 2024-09-19T08:45:11.529 INFO:teuthology.orchestra.run.smithi098.stdout:path = /volumes/_nogroup/subvol41980/66f5ddee-1364-4607-8f74-c9617e17c680 2024-09-19T08:45:11.529 INFO:teuthology.orchestra.run.smithi098.stdout:state = complete 2024-09-19T08:45:11.529 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:45:11.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:11.530 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/subvol_grp33620/subvol16391/.meta) 2024-09-19T08:45:11.587 INFO:teuthology.orchestra.run.smithi098.stdout:[GLOBAL] 2024-09-19T08:45:11.588 INFO:teuthology.orchestra.run.smithi098.stdout:version = 1 2024-09-19T08:45:11.588 INFO:teuthology.orchestra.run.smithi098.stdout:type = subvolume 2024-09-19T08:45:11.588 INFO:teuthology.orchestra.run.smithi098.stdout:path = /volumes/subvol_grp33620/subvol16391/f0511ac8-2f0c-4269-bb20-9ba71129e008 2024-09-19T08:45:11.588 INFO:teuthology.orchestra.run.smithi098.stdout:state = complete 2024-09-19T08:45:11.588 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:45:11.589 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol41980 2024-09-19T08:45:11.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.755+0000 7faea2e6d640 1 Processor -- start 2024-09-19T08:45:11.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.755+0000 7faea2e6d640 1 -- start start 2024-09-19T08:45:11.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.755+0000 7faea2e6d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fae9c06c3a0 0x7fae9c165f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:11.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.755+0000 7faea2e6d640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae9c072ea0 con 0x7fae9c144d10 2024-09-19T08:45:11.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.755+0000 7faea2e6d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae9c073180 con 0x7fae9c0729f0 2024-09-19T08:45:11.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.755+0000 7faea2e6d640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fae9c073ab0 con 0x7fae9c06c3a0 2024-09-19T08:45:11.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.755+0000 7faea140a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fae9c06c3a0 0x7fae9c165f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:11.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.755+0000 7faea0c09640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fae9c144d10 0x7fae9c0722e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56318/0 (socket says 172.21.15.98:56318) 2024-09-19T08:45:11.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.755+0000 7faea140a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fae9c06c3a0 0x7fae9c165f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59492/0 (socket says 172.21.15.98:59492) 2024-09-19T08:45:11.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.755+0000 7faea0c09640 1 -- 172.21.15.98:0/1010995841 learned_addr learned my addr 172.21.15.98:0/1010995841 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:11.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.755+0000 7faea140a640 1 -- 172.21.15.98:0/1010995841 >> v1:172.21.15.139:6789/0 conn(0x7fae9c0729f0 legacy=0x7fae9c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:11.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.755+0000 7faea140a640 1 -- 172.21.15.98:0/1010995841 >> v1:172.21.15.98:6789/0 conn(0x7fae9c144d10 legacy=0x7fae9c0722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:11.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.755+0000 7fae9b7fe640 1 -- 172.21.15.98:0/1010995841 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2110406173 0 0) 0x7fae9c072ea0 con 0x7fae9c144d10 2024-09-19T08:45:11.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.755+0000 7faea140a640 1 -- 172.21.15.98:0/1010995841 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae9c073460 con 0x7fae9c06c3a0 2024-09-19T08:45:11.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.759+0000 7faea140a640 1 --2- 172.21.15.98:0/1010995841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae9c06c3a0 0x7fae9c165f50 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fae94009d90 tx=0x7fae9401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=151e4fa21d834224 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:11.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.759+0000 7fae9b7fe640 1 -- 172.21.15.98:0/1010995841 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae940b2070 con 0x7fae9c06c3a0 2024-09-19T08:45:11.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.759+0000 7fae9b7fe640 1 -- 172.21.15.98:0/1010995841 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae940a5810 con 0x7fae9c06c3a0 2024-09-19T08:45:11.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.759+0000 7fae9b7fe640 1 -- 172.21.15.98:0/1010995841 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae940a5be0 con 0x7fae9c06c3a0 2024-09-19T08:45:11.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.759+0000 7faea2e6d640 1 -- 172.21.15.98:0/1010995841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae9c06c3a0 msgr2=0x7fae9c165f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:11.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.759+0000 7faea2e6d640 1 --2- 172.21.15.98:0/1010995841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae9c06c3a0 0x7fae9c165f50 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fae94009d90 tx=0x7fae9401c930 comp rx=0 tx=0).stop 2024-09-19T08:45:11.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.759+0000 7faea2e6d640 1 -- 172.21.15.98:0/1010995841 shutdown_connections 2024-09-19T08:45:11.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.759+0000 7faea2e6d640 1 --2- 172.21.15.98:0/1010995841 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae9c06c3a0 0x7fae9c165f50 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.759+0000 7faea2e6d640 1 -- 172.21.15.98:0/1010995841 >> 172.21.15.98:0/1010995841 conn(0x7fae9c0b2010 msgr2=0x7fae9c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:11.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.759+0000 7faea2e6d640 1 -- 172.21.15.98:0/1010995841 shutdown_connections 2024-09-19T08:45:11.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.759+0000 7faea2e6d640 1 -- 172.21.15.98:0/1010995841 wait complete. 2024-09-19T08:45:11.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.759+0000 7faea2e6d640 1 Processor -- start 2024-09-19T08:45:11.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.759+0000 7faea2e6d640 1 -- start start 2024-09-19T08:45:11.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.759+0000 7faea2e6d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae9c06c3a0 0x7fae9c13e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:11.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.759+0000 7faea2e6d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae9c0729f0 0x7fae9c13a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:11.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.759+0000 7faea2e6d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae9c144d10 0x7fae9c0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:11.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.759+0000 7faea2e6d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fae9c169040 con 0x7fae9c144d10 2024-09-19T08:45:11.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.759+0000 7faea2e6d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fae9c168ec0 con 0x7fae9c0729f0 2024-09-19T08:45:11.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.759+0000 7faea2e6d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fae9c1691c0 con 0x7fae9c06c3a0 2024-09-19T08:45:11.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.759+0000 7faea140a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae9c144d10 0x7fae9c0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:11.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.759+0000 7faea140a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae9c144d10 0x7fae9c0ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59500/0 (socket says 172.21.15.98:59500) 2024-09-19T08:45:11.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.759+0000 7faea140a640 1 -- 172.21.15.98:0/103421225 learned_addr learned my addr 172.21.15.98:0/103421225 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:11.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.759+0000 7faea0c09640 1 --2- 172.21.15.98:0/103421225 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae9c06c3a0 0x7fae9c13e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:11.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.759+0000 7fae9bfff640 1 --2- 172.21.15.98:0/103421225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae9c0729f0 0x7fae9c13a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:11.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.759+0000 7faea140a640 1 -- 172.21.15.98:0/103421225 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae9c06c3a0 msgr2=0x7fae9c13e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:11.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.759+0000 7faea140a640 1 --2- 172.21.15.98:0/103421225 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae9c06c3a0 0x7fae9c13e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.759+0000 7faea140a640 1 -- 172.21.15.98:0/103421225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae9c0729f0 msgr2=0x7fae9c13a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:11.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.759+0000 7faea140a640 1 --2- 172.21.15.98:0/103421225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae9c0729f0 0x7fae9c13a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:11.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.759+0000 7faea140a640 1 -- 172.21.15.98:0/103421225 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae9c13d4b0 con 0x7fae9c144d10 2024-09-19T08:45:11.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.763+0000 7faea140a640 1 --2- 172.21.15.98:0/103421225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae9c144d10 0x7fae9c0ae8b0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fae940b0040 tx=0x7fae940196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:11.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.763+0000 7fae997fa640 1 -- 172.21.15.98:0/103421225 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae940bd020 con 0x7fae9c144d10 2024-09-19T08:45:11.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.763+0000 7fae997fa640 1 -- 172.21.15.98:0/103421225 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae940b8670 con 0x7fae9c144d10 2024-09-19T08:45:11.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.763+0000 7faea2e6d640 1 -- 172.21.15.98:0/103421225 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae9c13d790 con 0x7fae9c144d10 2024-09-19T08:45:11.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.763+0000 7fae997fa640 1 -- 172.21.15.98:0/103421225 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae940b2040 con 0x7fae9c144d10 2024-09-19T08:45:11.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.763+0000 7faea2e6d640 1 -- 172.21.15.98:0/103421225 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae9c13dd50 con 0x7fae9c144d10 2024-09-19T08:45:11.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.763+0000 7faea2e6d640 1 -- 172.21.15.98:0/103421225 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae9c072ea0 con 0x7fae9c144d10 2024-09-19T08:45:11.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.763+0000 7fae997fa640 1 -- 172.21.15.98:0/103421225 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 87853+0+0 (secure 0 0 0) 0x7fae940c0020 con 0x7fae9c144d10 2024-09-19T08:45:11.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.763+0000 7fae997fa640 1 --2- 172.21.15.98:0/103421225 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fae88067d70 0x7fae8806a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:11.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.763+0000 7fae997fa640 1 -- 172.21.15.98:0/103421225 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5795+0+0 (secure 0 0 0) 0x7fae941215e0 con 0x7fae9c144d10 2024-09-19T08:45:11.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.763+0000 7faea0c09640 1 --2- 172.21.15.98:0/103421225 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fae88067d70 0x7fae8806a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:11.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.763+0000 7faea0c09640 1 --2- 172.21.15.98:0/103421225 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fae88067d70 0x7fae8806a230 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fae8c000f70 tx=0x7fae8c007c10 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:11.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.767+0000 7fae997fa640 1 -- 172.21.15.98:0/103421225 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fae940c7070 con 0x7fae9c144d10 2024-09-19T08:45:11.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:11.955+0000 7faea2e6d640 1 -- 172.21.15.98:0/103421225 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol41980", "target": ["mon-mgr", ""]}) -- 0x7fae9c06e490 con 0x7fae88067d70 2024-09-19T08:45:12.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.055+0000 7fae997fa640 1 -- 172.21.15.98:0/103421225 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fae9c06e490 con 0x7fae88067d70 2024-09-19T08:45:12.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.059+0000 7faea2e6d640 1 -- 172.21.15.98:0/103421225 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fae88067d70 msgr2=0x7fae8806a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:12.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.059+0000 7faea2e6d640 1 --2- 172.21.15.98:0/103421225 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fae88067d70 0x7fae8806a230 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fae8c000f70 tx=0x7fae8c007c10 comp rx=0 tx=0).stop 2024-09-19T08:45:12.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.059+0000 7faea2e6d640 1 -- 172.21.15.98:0/103421225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae9c144d10 msgr2=0x7fae9c0ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:12.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.059+0000 7faea2e6d640 1 --2- 172.21.15.98:0/103421225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae9c144d10 0x7fae9c0ae8b0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fae940b0040 tx=0x7fae940196d0 comp rx=0 tx=0).stop 2024-09-19T08:45:12.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.059+0000 7faea2e6d640 1 -- 172.21.15.98:0/103421225 shutdown_connections 2024-09-19T08:45:12.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.059+0000 7faea2e6d640 1 --2- 172.21.15.98:0/103421225 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fae88067d70 0x7fae8806a230 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:12.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.059+0000 7faea2e6d640 1 --2- 172.21.15.98:0/103421225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae9c144d10 0x7fae9c0ae8b0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:12.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.059+0000 7faea2e6d640 1 --2- 172.21.15.98:0/103421225 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae9c0729f0 0x7fae9c13a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:12.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.059+0000 7faea2e6d640 1 --2- 172.21.15.98:0/103421225 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae9c06c3a0 0x7fae9c13e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:12.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.059+0000 7faea2e6d640 1 -- 172.21.15.98:0/103421225 >> 172.21.15.98:0/103421225 conn(0x7fae9c0b2010 msgr2=0x7fae9c0cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:12.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.059+0000 7faea2e6d640 1 -- 172.21.15.98:0/103421225 shutdown_connections 2024-09-19T08:45:12.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.059+0000 7faea2e6d640 1 -- 172.21.15.98:0/103421225 wait complete. 2024-09-19T08:45:12.069 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:12.071+0000 7ff73d6a2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:45:12.069 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:12.071+0000 7ff73d6a2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:45:12.069 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:12.071+0000 7ff73d6a2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:45:12.069 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:12.071+0000 7ff73d6a2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:45:12.070 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:12.071+0000 7ff73d6a2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:45:12.082 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol16391 subvol_grp33620 2024-09-19T08:45:12.085 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:12.087+0000 7ff73c6a0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:45:12.085 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:12.087+0000 7ff73c6a0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:45:12.085 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:12.087+0000 7ff73c6a0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:45:12.085 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:12.087+0000 7ff73c6a0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:45:12.085 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:12.087+0000 7ff73c6a0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:45:12.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.211+0000 7fac556ab640 1 Processor -- start 2024-09-19T08:45:12.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.211+0000 7fac556ab640 1 -- start start 2024-09-19T08:45:12.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.211+0000 7fac556ab640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac5006d4c0 0x7fac5006d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:12.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.211+0000 7fac556ab640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac501608c0 con 0x7fac500cf700 2024-09-19T08:45:12.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.211+0000 7fac556ab640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac50160ba0 con 0x7fac5015c290 2024-09-19T08:45:12.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.211+0000 7fac556ab640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fac50160e80 con 0x7fac5006d4c0 2024-09-19T08:45:12.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.211+0000 7fac4e7fc640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fac500cf700 0x7fac5015ff50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56332/0 (socket says 172.21.15.98:56332) 2024-09-19T08:45:12.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.211+0000 7fac4e7fc640 1 -- 172.21.15.98:0/876705139 learned_addr learned my addr 172.21.15.98:0/876705139 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:12.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.211+0000 7fac4f7fe640 1 --2- 172.21.15.98:0/876705139 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac5006d4c0 0x7fac5006d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:12.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.211+0000 7fac4dffb640 1 -- 172.21.15.98:0/876705139 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 522157905 0 0) 0x7fac501608c0 con 0x7fac500cf700 2024-09-19T08:45:12.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.211+0000 7fac4dffb640 1 -- 172.21.15.98:0/876705139 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac38003530 con 0x7fac500cf700 2024-09-19T08:45:12.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.215+0000 7fac4dffb640 1 -- 172.21.15.98:0/876705139 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 805593095 0 0) 0x7fac38003530 con 0x7fac500cf700 2024-09-19T08:45:12.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.215+0000 7fac4dffb640 1 -- 172.21.15.98:0/876705139 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac5006d4c0 msgr2=0x7fac5006d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:12.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.215+0000 7fac4dffb640 1 --2- 172.21.15.98:0/876705139 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac5006d4c0 0x7fac5006d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:12.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.215+0000 7fac4dffb640 1 -- 172.21.15.98:0/876705139 >> v1:172.21.15.139:6789/0 conn(0x7fac5015c290 legacy=0x7fac500cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:12.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.215+0000 7fac4dffb640 1 -- 172.21.15.98:0/876705139 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac500cd1a0 con 0x7fac500cf700 2024-09-19T08:45:12.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.215+0000 7fac4dffb640 1 -- 172.21.15.98:0/876705139 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fac440016a0 con 0x7fac500cf700 2024-09-19T08:45:12.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.215+0000 7fac4dffb640 1 -- 172.21.15.98:0/876705139 >> v1:172.21.15.98:6789/0 conn(0x7fac500cf700 legacy=0x7fac5015ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:12.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.215+0000 7fac4dffb640 1 --2- 172.21.15.98:0/876705139 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac5006d4c0 0x7fac380039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:12.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.215+0000 7fac4dffb640 1 --2- 172.21.15.98:0/876705139 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac38003f70 0x7fac38004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:12.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.215+0000 7fac4dffb640 1 --2- 172.21.15.98:0/876705139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac38004d50 0x7fac380051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:12.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.215+0000 7fac4dffb640 1 -- 172.21.15.98:0/876705139 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fac3800ba80 con 0x7fac38003f70 2024-09-19T08:45:12.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.215+0000 7fac4dffb640 1 -- 172.21.15.98:0/876705139 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fac3800bc30 con 0x7fac38004d50 2024-09-19T08:45:12.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.215+0000 7fac4dffb640 1 -- 172.21.15.98:0/876705139 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fac3800be30 con 0x7fac5006d4c0 2024-09-19T08:45:12.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.215+0000 7fac4e7fc640 1 --2- 172.21.15.98:0/876705139 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac38003f70 0x7fac38004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:12.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.215+0000 7fac4effd640 1 --2- 172.21.15.98:0/876705139 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac5006d4c0 0x7fac380039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:12.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.215+0000 7fac4f7fe640 1 --2- 172.21.15.98:0/876705139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac38004d50 0x7fac380051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:12.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.215+0000 7fac4e7fc640 1 -- 172.21.15.98:0/876705139 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac5006d4c0 msgr2=0x7fac380039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:12.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.215+0000 7fac4e7fc640 1 --2- 172.21.15.98:0/876705139 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac5006d4c0 0x7fac380039f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:12.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.215+0000 7fac4e7fc640 1 -- 172.21.15.98:0/876705139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac38004d50 msgr2=0x7fac380051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:12.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.215+0000 7fac4e7fc640 1 --2- 172.21.15.98:0/876705139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac38004d50 0x7fac380051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:12.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.215+0000 7fac4e7fc640 1 -- 172.21.15.98:0/876705139 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac38003530 con 0x7fac38003f70 2024-09-19T08:45:12.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.215+0000 7fac4e7fc640 1 --2- 172.21.15.98:0/876705139 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac38003f70 0x7fac38004810 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fac4400c210 tx=0x7fac4401f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4119564148777b29 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:12.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.215+0000 7fac4dffb640 1 -- 172.21.15.98:0/876705139 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac44007d40 con 0x7fac38003f70 2024-09-19T08:45:12.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.215+0000 7fac4dffb640 1 -- 172.21.15.98:0/876705139 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac44005ae0 con 0x7fac38003f70 2024-09-19T08:45:12.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.215+0000 7fac4dffb640 1 -- 172.21.15.98:0/876705139 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac44007070 con 0x7fac38003f70 2024-09-19T08:45:12.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.215+0000 7fac556ab640 1 -- 172.21.15.98:0/876705139 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac38003f70 msgr2=0x7fac38004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:12.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.215+0000 7fac556ab640 1 --2- 172.21.15.98:0/876705139 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac38003f70 0x7fac38004810 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fac4400c210 tx=0x7fac4401f3b0 comp rx=0 tx=0).stop 2024-09-19T08:45:12.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.215+0000 7fac4f7fe640 1 -- 172.21.15.98:0/876705139 reap_dead start 2024-09-19T08:45:12.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.215+0000 7fac556ab640 1 -- 172.21.15.98:0/876705139 shutdown_connections 2024-09-19T08:45:12.221 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.215+0000 7fac556ab640 1 -- 172.21.15.98:0/876705139 >> 172.21.15.98:0/876705139 conn(0x7fac5006c090 msgr2=0x7fac5006c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:12.221 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.215+0000 7fac556ab640 1 -- 172.21.15.98:0/876705139 shutdown_connections 2024-09-19T08:45:12.221 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.215+0000 7fac556ab640 1 -- 172.21.15.98:0/876705139 wait complete. 2024-09-19T08:45:12.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.219+0000 7fac556ab640 1 Processor -- start 2024-09-19T08:45:12.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.219+0000 7fac556ab640 1 -- start start 2024-09-19T08:45:12.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.219+0000 7fac556ab640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac38003f70 0x7fac50153090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:12.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.219+0000 7fac556ab640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac50155970 0x7fac501527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:12.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.219+0000 7fac556ab640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac5015e280 0x7fac5015e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:12.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.219+0000 7fac556ab640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fac50160660 con 0x7fac50155970 2024-09-19T08:45:12.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.219+0000 7fac556ab640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fac500cd510 con 0x7fac38003f70 2024-09-19T08:45:12.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.219+0000 7fac556ab640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fac5006d8f0 con 0x7fac5015e280 2024-09-19T08:45:12.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.219+0000 7fac4e7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac50155970 0x7fac501527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:12.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.219+0000 7fac4e7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac50155970 0x7fac501527d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59514/0 (socket says 172.21.15.98:59514) 2024-09-19T08:45:12.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.219+0000 7fac4e7fc640 1 -- 172.21.15.98:0/3104535248 learned_addr learned my addr 172.21.15.98:0/3104535248 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:12.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.219+0000 7fac4f7fe640 1 --2- 172.21.15.98:0/3104535248 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac5015e280 0x7fac5015e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:12.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.219+0000 7fac4effd640 1 --2- 172.21.15.98:0/3104535248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac38003f70 0x7fac50153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:12.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.219+0000 7fac4e7fc640 1 -- 172.21.15.98:0/3104535248 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac5015e280 msgr2=0x7fac5015e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:12.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.219+0000 7fac4e7fc640 1 --2- 172.21.15.98:0/3104535248 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac5015e280 0x7fac5015e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:12.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.219+0000 7fac4e7fc640 1 -- 172.21.15.98:0/3104535248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac38003f70 msgr2=0x7fac50153090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:12.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.219+0000 7fac4e7fc640 1 --2- 172.21.15.98:0/3104535248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac38003f70 0x7fac50153090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:12.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.219+0000 7fac4e7fc640 1 -- 172.21.15.98:0/3104535248 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac50156150 con 0x7fac50155970 2024-09-19T08:45:12.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.219+0000 7fac4effd640 1 --2- 172.21.15.98:0/3104535248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac38003f70 0x7fac50153090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:12.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.219+0000 7fac4e7fc640 1 --2- 172.21.15.98:0/3104535248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac50155970 0x7fac501527d0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fac4401f1e0 tx=0x7fac38005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:12.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.219+0000 7fac377fe640 1 -- 172.21.15.98:0/3104535248 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac44000ee0 con 0x7fac50155970 2024-09-19T08:45:12.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.219+0000 7fac377fe640 1 -- 172.21.15.98:0/3104535248 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac44005c90 con 0x7fac50155970 2024-09-19T08:45:12.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.219+0000 7fac556ab640 1 -- 172.21.15.98:0/3104535248 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac5015d500 con 0x7fac50155970 2024-09-19T08:45:12.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.219+0000 7fac377fe640 1 -- 172.21.15.98:0/3104535248 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac440049f0 con 0x7fac50155970 2024-09-19T08:45:12.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.219+0000 7fac556ab640 1 -- 172.21.15.98:0/3104535248 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac5015da80 con 0x7fac50155970 2024-09-19T08:45:12.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.219+0000 7fac556ab640 1 -- 172.21.15.98:0/3104535248 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac5006dcd0 con 0x7fac50155970 2024-09-19T08:45:12.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.219+0000 7fac377fe640 1 -- 172.21.15.98:0/3104535248 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 87853+0+0 (secure 0 0 0) 0x7fac440ba030 con 0x7fac50155970 2024-09-19T08:45:12.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.223+0000 7fac377fe640 1 --2- 172.21.15.98:0/3104535248 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fac3c067ee0 0x7fac3c06a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:12.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.223+0000 7fac377fe640 1 -- 172.21.15.98:0/3104535248 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5883+0+0 (secure 0 0 0) 0x7fac44123050 con 0x7fac50155970 2024-09-19T08:45:12.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.223+0000 7fac4effd640 1 --2- 172.21.15.98:0/3104535248 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fac3c067ee0 0x7fac3c06a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:12.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.223+0000 7fac4effd640 1 --2- 172.21.15.98:0/3104535248 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fac3c067ee0 0x7fac3c06a3a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fac40001680 tx=0x7fac40009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:12.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.227+0000 7fac377fe640 1 -- 172.21.15.98:0/3104535248 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fac440190c0 con 0x7fac50155970 2024-09-19T08:45:12.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.415+0000 7fac556ab640 1 -- 172.21.15.98:0/3104535248 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol16391", "group_name": "subvol_grp33620", "target": ["mon-mgr", ""]}) -- 0x7fac50073550 con 0x7fac3c067ee0 2024-09-19T08:45:12.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.503+0000 7fac377fe640 1 -- 172.21.15.98:0/3104535248 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fac50073550 con 0x7fac3c067ee0 2024-09-19T08:45:12.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.503+0000 7fac556ab640 1 -- 172.21.15.98:0/3104535248 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fac3c067ee0 msgr2=0x7fac3c06a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:12.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.503+0000 7fac556ab640 1 --2- 172.21.15.98:0/3104535248 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fac3c067ee0 0x7fac3c06a3a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fac40001680 tx=0x7fac40009040 comp rx=0 tx=0).stop 2024-09-19T08:45:12.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.503+0000 7fac556ab640 1 -- 172.21.15.98:0/3104535248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac50155970 msgr2=0x7fac501527d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:12.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.503+0000 7fac556ab640 1 --2- 172.21.15.98:0/3104535248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac50155970 0x7fac501527d0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fac4401f1e0 tx=0x7fac38005840 comp rx=0 tx=0).stop 2024-09-19T08:45:12.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.503+0000 7fac556ab640 1 -- 172.21.15.98:0/3104535248 shutdown_connections 2024-09-19T08:45:12.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.503+0000 7fac556ab640 1 --2- 172.21.15.98:0/3104535248 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fac3c067ee0 0x7fac3c06a3a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:12.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.503+0000 7fac556ab640 1 --2- 172.21.15.98:0/3104535248 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fac5015e280 0x7fac5015e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:12.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.503+0000 7fac556ab640 1 --2- 172.21.15.98:0/3104535248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fac50155970 0x7fac501527d0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:12.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.507+0000 7fac556ab640 1 --2- 172.21.15.98:0/3104535248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fac38003f70 0x7fac50153090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:12.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.507+0000 7fac556ab640 1 -- 172.21.15.98:0/3104535248 >> 172.21.15.98:0/3104535248 conn(0x7fac5006c090 msgr2=0x7fac500cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:12.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.507+0000 7fac556ab640 1 -- 172.21.15.98:0/3104535248 shutdown_connections 2024-09-19T08:45:12.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.507+0000 7fac556ab640 1 -- 172.21.15.98:0/3104535248 wait complete. 2024-09-19T08:45:12.529 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol56792 2024-09-19T08:45:12.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.671+0000 7f7cf464f640 1 Processor -- start 2024-09-19T08:45:12.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.671+0000 7f7cf464f640 1 -- start start 2024-09-19T08:45:12.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.671+0000 7f7cf464f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cec06c3a0 0x7f7cec162200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:12.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.671+0000 7f7cf464f640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cec162f90 con 0x7f7cec162740 2024-09-19T08:45:12.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.671+0000 7f7cf464f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cec0cf040 con 0x7f7cec147cb0 2024-09-19T08:45:12.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.671+0000 7f7cf464f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7cec0cee10 con 0x7f7cec06c3a0 2024-09-19T08:45:12.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.671+0000 7f7cf2bec640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f7cec162740 0x7f7cec0b3310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56338/0 (socket says 172.21.15.98:56338) 2024-09-19T08:45:12.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.671+0000 7f7cf2bec640 1 -- 172.21.15.98:0/3875682233 learned_addr learned my addr 172.21.15.98:0/3875682233 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:12.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.671+0000 7f7cf1bea640 1 --2- 172.21.15.98:0/3875682233 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cec06c3a0 0x7f7cec162200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:12.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.671+0000 7f7cf13e9640 1 -- 172.21.15.98:0/3875682233 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3718842423 0 0) 0x7f7cec162f90 con 0x7f7cec162740 2024-09-19T08:45:12.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.671+0000 7f7cf13e9640 1 -- 172.21.15.98:0/3875682233 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7cd8003530 con 0x7f7cec162740 2024-09-19T08:45:12.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.671+0000 7f7cf13e9640 1 -- 172.21.15.98:0/3875682233 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1291596819 0 0) 0x7f7cd8003530 con 0x7f7cec162740 2024-09-19T08:45:12.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.671+0000 7f7cf13e9640 1 -- 172.21.15.98:0/3875682233 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cec06c3a0 msgr2=0x7f7cec162200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:12.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.671+0000 7f7cf13e9640 1 --2- 172.21.15.98:0/3875682233 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cec06c3a0 0x7f7cec162200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:12.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.671+0000 7f7cf13e9640 1 -- 172.21.15.98:0/3875682233 >> v1:172.21.15.139:6789/0 conn(0x7f7cec147cb0 legacy=0x7f7cec06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:12.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.671+0000 7f7cf13e9640 1 -- 172.21.15.98:0/3875682233 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cec0cf2e0 con 0x7f7cec162740 2024-09-19T08:45:12.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.671+0000 7f7cf13e9640 1 -- 172.21.15.98:0/3875682233 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f7ce4001a00 con 0x7f7cec162740 2024-09-19T08:45:12.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.671+0000 7f7cf13e9640 1 -- 172.21.15.98:0/3875682233 >> v1:172.21.15.98:6789/0 conn(0x7f7cec162740 legacy=0x7f7cec0b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:12.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.671+0000 7f7cf13e9640 1 --2- 172.21.15.98:0/3875682233 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cd80039f0 0x7f7cd8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:12.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.671+0000 7f7cf13e9640 1 --2- 172.21.15.98:0/3875682233 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cec06c3a0 0x7f7cd8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:12.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.671+0000 7f7cf13e9640 1 --2- 172.21.15.98:0/3875682233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cd8004d50 0x7f7cd80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:12.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.671+0000 7f7cf13e9640 1 -- 172.21.15.98:0/3875682233 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd800bb60 con 0x7f7cd8004d50 2024-09-19T08:45:12.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.671+0000 7f7cf13e9640 1 -- 172.21.15.98:0/3875682233 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd800bce0 con 0x7f7cd80039f0 2024-09-19T08:45:12.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.671+0000 7f7cf13e9640 1 -- 172.21.15.98:0/3875682233 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7cd800bee0 con 0x7f7cec06c3a0 2024-09-19T08:45:12.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.671+0000 7f7cf2bec640 1 --2- 172.21.15.98:0/3875682233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cd8004d50 0x7f7cd80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:12.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.671+0000 7f7cf1bea640 1 --2- 172.21.15.98:0/3875682233 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cec06c3a0 0x7f7cd8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:12.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.671+0000 7f7cf23eb640 1 --2- 172.21.15.98:0/3875682233 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cd80039f0 0x7f7cd8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:12.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.671+0000 7f7cf2bec640 1 -- 172.21.15.98:0/3875682233 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cec06c3a0 msgr2=0x7f7cd8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:12.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.671+0000 7f7cf2bec640 1 --2- 172.21.15.98:0/3875682233 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cec06c3a0 0x7f7cd8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:12.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.671+0000 7f7cf2bec640 1 -- 172.21.15.98:0/3875682233 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cd80039f0 msgr2=0x7f7cd8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:12.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.671+0000 7f7cf2bec640 1 --2- 172.21.15.98:0/3875682233 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cd80039f0 0x7f7cd8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:12.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.671+0000 7f7cf2bec640 1 -- 172.21.15.98:0/3875682233 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd8003530 con 0x7f7cd8004d50 2024-09-19T08:45:12.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.675+0000 7f7cf2bec640 1 --2- 172.21.15.98:0/3875682233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cd8004d50 0x7f7cd80051d0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f7ce400a850 tx=0x7f7ce401fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85af7fa77312822 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:12.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.675+0000 7f7cf13e9640 1 -- 172.21.15.98:0/3875682233 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce400a4a0 con 0x7f7cd8004d50 2024-09-19T08:45:12.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.675+0000 7f7cf13e9640 1 -- 172.21.15.98:0/3875682233 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ce40ad070 con 0x7f7cd8004d50 2024-09-19T08:45:12.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.675+0000 7f7cf13e9640 1 -- 172.21.15.98:0/3875682233 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce40a9980 con 0x7f7cd8004d50 2024-09-19T08:45:12.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.675+0000 7f7cf464f640 1 -- 172.21.15.98:0/3875682233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cd8004d50 msgr2=0x7f7cd80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:12.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.675+0000 7f7cf464f640 1 --2- 172.21.15.98:0/3875682233 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cd8004d50 0x7f7cd80051d0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f7ce400a850 tx=0x7f7ce401fb00 comp rx=0 tx=0).stop 2024-09-19T08:45:12.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.675+0000 7f7cf2bec640 1 -- 172.21.15.98:0/3875682233 reap_dead start 2024-09-19T08:45:12.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.675+0000 7f7cf464f640 1 -- 172.21.15.98:0/3875682233 shutdown_connections 2024-09-19T08:45:12.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.675+0000 7f7cf464f640 1 -- 172.21.15.98:0/3875682233 >> 172.21.15.98:0/3875682233 conn(0x7f7cec0b2010 msgr2=0x7f7cec0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:12.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.675+0000 7f7cf464f640 1 -- 172.21.15.98:0/3875682233 shutdown_connections 2024-09-19T08:45:12.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.675+0000 7f7cf464f640 1 -- 172.21.15.98:0/3875682233 wait complete. 2024-09-19T08:45:12.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.675+0000 7f7cf464f640 1 Processor -- start 2024-09-19T08:45:12.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.675+0000 7f7cf464f640 1 -- start start 2024-09-19T08:45:12.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.675+0000 7f7cf464f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cd8004d50 0x7f7cec1f3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:12.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.675+0000 7f7cf464f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cec0c5c90 0x7f7cec0ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:12.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.675+0000 7f7cf464f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cec160010 0x7f7cec160490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:12.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.675+0000 7f7cf464f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec162b90 con 0x7f7cec160010 2024-09-19T08:45:12.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.675+0000 7f7cf464f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec148090 con 0x7f7cec0c5c90 2024-09-19T08:45:12.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.675+0000 7f7cf464f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7cec162d10 con 0x7f7cd8004d50 2024-09-19T08:45:12.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.679+0000 7f7cf2bec640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cec160010 0x7f7cec160490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:12.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.679+0000 7f7cf2bec640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cec160010 0x7f7cec160490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59530/0 (socket says 172.21.15.98:59530) 2024-09-19T08:45:12.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.679+0000 7f7cf2bec640 1 -- 172.21.15.98:0/696358028 learned_addr learned my addr 172.21.15.98:0/696358028 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:12.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.679+0000 7f7cf23eb640 1 --2- 172.21.15.98:0/696358028 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cd8004d50 0x7f7cec1f3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:12.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.679+0000 7f7cf1bea640 1 --2- 172.21.15.98:0/696358028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cec0c5c90 0x7f7cec0ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:12.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.679+0000 7f7cf2bec640 1 -- 172.21.15.98:0/696358028 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cd8004d50 msgr2=0x7f7cec1f3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:12.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.679+0000 7f7cf2bec640 1 --2- 172.21.15.98:0/696358028 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7cd8004d50 0x7f7cec1f3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:12.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.679+0000 7f7cf2bec640 1 -- 172.21.15.98:0/696358028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cec0c5c90 msgr2=0x7f7cec0ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:12.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.679+0000 7f7cf2bec640 1 --2- 172.21.15.98:0/696358028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cec0c5c90 0x7f7cec0ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:12.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.679+0000 7f7cf2bec640 1 -- 172.21.15.98:0/696358028 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cec1f43d0 con 0x7f7cec160010 2024-09-19T08:45:12.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.679+0000 7f7cf1bea640 1 --2- 172.21.15.98:0/696358028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7cec0c5c90 0x7f7cec0ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:12.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.679+0000 7f7cf2bec640 1 --2- 172.21.15.98:0/696358028 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7cec160010 0x7f7cec160490 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f7ce400a940 tx=0x7f7cd8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:12.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.679+0000 7f7ce2ffd640 1 -- 172.21.15.98:0/696358028 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce40a9720 con 0x7f7cec160010 2024-09-19T08:45:12.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.679+0000 7f7ce2ffd640 1 -- 172.21.15.98:0/696358028 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ce40a98c0 con 0x7f7cec160010 2024-09-19T08:45:12.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.679+0000 7f7ce2ffd640 1 -- 172.21.15.98:0/696358028 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce40a9cb0 con 0x7f7cec160010 2024-09-19T08:45:12.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.679+0000 7f7cf464f640 1 -- 172.21.15.98:0/696358028 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cec1f46b0 con 0x7f7cec160010 2024-09-19T08:45:12.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.679+0000 7f7cf464f640 1 -- 172.21.15.98:0/696358028 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cec1f4c70 con 0x7f7cec160010 2024-09-19T08:45:12.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.679+0000 7f7cf464f640 1 -- 172.21.15.98:0/696358028 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cec0cfdc0 con 0x7f7cec160010 2024-09-19T08:45:12.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.679+0000 7f7ce2ffd640 1 -- 172.21.15.98:0/696358028 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 20) ==== 87853+0+0 (secure 0 0 0) 0x7f7ce40b2470 con 0x7f7cec160010 2024-09-19T08:45:12.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.679+0000 7f7ce2ffd640 1 --2- 172.21.15.98:0/696358028 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7cd0067d70 0x7f7cd006a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:12.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.679+0000 7f7ce2ffd640 1 -- 172.21.15.98:0/696358028 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5883+0+0 (secure 0 0 0) 0x7f7ce4123a40 con 0x7f7cec160010 2024-09-19T08:45:12.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.679+0000 7f7cf23eb640 1 --2- 172.21.15.98:0/696358028 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7cd0067d70 0x7f7cd006a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:12.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.679+0000 7f7cf23eb640 1 --2- 172.21.15.98:0/696358028 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7cd0067d70 0x7f7cd006a230 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7cec1f19e0 tx=0x7f7ce8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:12.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.683+0000 7f7ce2ffd640 1 -- 172.21.15.98:0/696358028 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f7ce4019810 con 0x7f7cec160010 2024-09-19T08:45:12.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.875+0000 7f7cf464f640 1 -- 172.21.15.98:0/696358028 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol56792", "target": ["mon-mgr", ""]}) -- 0x7f7cec0c6070 con 0x7f7cd0067d70 2024-09-19T08:45:12.897 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:12.899+0000 7ff73a69c640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvolume 'subvol56792' is not ready for operation rm (use --force to override) 2024-09-19T08:45:12.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:12.895+0000 7f7ce2ffd640 1 -- 172.21.15.98:0/696358028 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: -11 subvolume 'subvol56792' is not ready for operation rm (use --force to override)) ==== 87+0+0 (secure 0 0 0) 0x7f7cec0c6070 con 0x7f7cd0067d70 2024-09-19T08:45:12.898 INFO:teuthology.orchestra.run.smithi098.stderr:Error EAGAIN: subvolume 'subvol56792' is not ready for operation rm (use --force to override) 2024-09-19T08:45:12.904 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-09-19T08:45:12.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:12.905 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol56792/.meta) 2024-09-19T08:45:12.921 INFO:teuthology.orchestra.run.smithi098.stdout:[GLOBAL] 2024-09-19T08:45:12.921 INFO:teuthology.orchestra.run.smithi098.stdout:version = 1 2024-09-19T08:45:12.921 INFO:teuthology.orchestra.run.smithi098.stdout:type = clone 2024-09-19T08:45:12.921 INFO:teuthology.orchestra.run.smithi098.stdout:path = /volumes/_nogroup/subvol56792/6636d43c-dc2a-49bd-b834-7c8a36fd9b4d 2024-09-19T08:45:12.921 INFO:teuthology.orchestra.run.smithi098.stdout:state = pending 2024-09-19T08:45:12.922 INFO:teuthology.orchestra.run.smithi098.stdout:[source] 2024-09-19T08:45:12.922 INFO:teuthology.orchestra.run.smithi098.stdout:volume = fake 2024-09-19T08:45:12.922 INFO:teuthology.orchestra.run.smithi098.stdout:subvolume = fake 2024-09-19T08:45:12.922 INFO:teuthology.orchestra.run.smithi098.stdout:snapshot = fake 2024-09-19T08:45:12.923 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol56792 --force 2024-09-19T08:45:13.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.095+0000 7fccaac20640 1 Processor -- start 2024-09-19T08:45:13.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.095+0000 7fccaac20640 1 -- start start 2024-09-19T08:45:13.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.095+0000 7fccaac20640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcca40cf700 0x7fcca415ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:13.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.095+0000 7fccaac20640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcca406d4c0 0x7fcca406d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:13.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.095+0000 7fccaac20640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcca41608c0 con 0x7fcca415c290 2024-09-19T08:45:13.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.095+0000 7fccaac20640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fcca4160e80 con 0x7fcca40cf700 2024-09-19T08:45:13.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.095+0000 7fccaac20640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcca40cd510 con 0x7fcca406d4c0 2024-09-19T08:45:13.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.095+0000 7fcca89bc640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fcca415c290 0x7fcca40cd9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56344/0 (socket says 172.21.15.98:56344) 2024-09-19T08:45:13.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.095+0000 7fcca89bc640 1 -- 172.21.15.98:0/733127375 learned_addr learned my addr 172.21.15.98:0/733127375 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:13.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.095+0000 7fcca91bd640 1 --2- 172.21.15.98:0/733127375 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcca406d4c0 0x7fcca406d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:13.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.095+0000 7fcca37fe640 1 -- 172.21.15.98:0/733127375 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 529590371 0 0) 0x7fcca41608c0 con 0x7fcca415c290 2024-09-19T08:45:13.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.095+0000 7fcca3fff640 1 --2- 172.21.15.98:0/733127375 >> v2:172.21.15.139:3300/0 conn(0x7fcca40cf700 0x7fcca415ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:13.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.095+0000 7fcca37fe640 1 -- 172.21.15.98:0/733127375 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc8c003530 con 0x7fcca415c290 2024-09-19T08:45:13.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.095+0000 7fcca37fe640 1 -- 172.21.15.98:0/733127375 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1840922641 0 0) 0x7fcc8c003530 con 0x7fcca415c290 2024-09-19T08:45:13.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.095+0000 7fcca37fe640 1 -- 172.21.15.98:0/733127375 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcca406d4c0 msgr2=0x7fcca406d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:13.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.095+0000 7fcca37fe640 1 --2- 172.21.15.98:0/733127375 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcca406d4c0 0x7fcca406d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.095+0000 7fcca37fe640 1 -- 172.21.15.98:0/733127375 >> v2:172.21.15.139:3300/0 conn(0x7fcca40cf700 msgr2=0x7fcca415ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:13.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.095+0000 7fcca37fe640 1 --2- 172.21.15.98:0/733127375 >> v2:172.21.15.139:3300/0 conn(0x7fcca40cf700 0x7fcca415ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.095+0000 7fcca37fe640 1 -- 172.21.15.98:0/733127375 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca4160ba0 con 0x7fcca415c290 2024-09-19T08:45:13.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.095+0000 7fcca37fe640 1 -- 172.21.15.98:0/733127375 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fcc940016a0 con 0x7fcca415c290 2024-09-19T08:45:13.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.095+0000 7fcca37fe640 1 -- 172.21.15.98:0/733127375 >> v1:172.21.15.98:6789/0 conn(0x7fcca415c290 legacy=0x7fcca40cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:13.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.095+0000 7fcca37fe640 1 --2- 172.21.15.98:0/733127375 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcca406d4c0 0x7fcc8c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:13.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.095+0000 7fcca37fe640 1 --2- 172.21.15.98:0/733127375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc8c003f40 0x7fcc8c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:13.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.095+0000 7fcca37fe640 1 --2- 172.21.15.98:0/733127375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc8c004d50 0x7fcc8c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:13.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.095+0000 7fcca37fe640 1 -- 172.21.15.98:0/733127375 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcca40cd510 con 0x7fcc8c004d50 2024-09-19T08:45:13.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.095+0000 7fcca37fe640 1 -- 172.21.15.98:0/733127375 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8c00b9d0 con 0x7fcc8c003f40 2024-09-19T08:45:13.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.095+0000 7fcca37fe640 1 -- 172.21.15.98:0/733127375 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcc8c00bba0 con 0x7fcca406d4c0 2024-09-19T08:45:13.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.095+0000 7fcca3fff640 1 --2- 172.21.15.98:0/733127375 >> v2:172.21.15.139:3300/0 conn(0x7fcca40cf700 0x7fcca415ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:13.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.095+0000 7fcca91bd640 1 --2- 172.21.15.98:0/733127375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc8c004d50 0x7fcc8c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:13.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.095+0000 7fcca89bc640 1 --2- 172.21.15.98:0/733127375 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcca406d4c0 0x7fcc8c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:13.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.095+0000 7fcca91bd640 1 -- 172.21.15.98:0/733127375 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcca406d4c0 msgr2=0x7fcc8c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:13.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.095+0000 7fcca91bd640 1 --2- 172.21.15.98:0/733127375 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcca406d4c0 0x7fcc8c0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.095+0000 7fcca91bd640 1 -- 172.21.15.98:0/733127375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc8c003f40 msgr2=0x7fcc8c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:13.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.095+0000 7fcca91bd640 1 --2- 172.21.15.98:0/733127375 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc8c003f40 0x7fcc8c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.095+0000 7fcca91bd640 1 -- 172.21.15.98:0/733127375 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc8c003530 con 0x7fcc8c004d50 2024-09-19T08:45:13.104 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.099+0000 7fcca89bc640 1 --2- 172.21.15.98:0/733127375 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcca406d4c0 0x7fcc8c0039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:13.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.099+0000 7fcca91bd640 1 --2- 172.21.15.98:0/733127375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc8c004d50 0x7fcc8c0051d0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c009fc0 tx=0x7fcc9c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04f824ea6ad9ed3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:13.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.099+0000 7fcca37fe640 1 -- 172.21.15.98:0/733127375 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c0b2070 con 0x7fcc8c004d50 2024-09-19T08:45:13.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.099+0000 7fcca37fe640 1 -- 172.21.15.98:0/733127375 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc9c0a5c90 con 0x7fcc8c004d50 2024-09-19T08:45:13.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.099+0000 7fcca37fe640 1 -- 172.21.15.98:0/733127375 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c009c10 con 0x7fcc8c004d50 2024-09-19T08:45:13.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.099+0000 7fccaac20640 1 -- 172.21.15.98:0/733127375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc8c004d50 msgr2=0x7fcc8c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:13.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.099+0000 7fccaac20640 1 --2- 172.21.15.98:0/733127375 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc8c004d50 0x7fcc8c0051d0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c009fc0 tx=0x7fcc9c01cc80 comp rx=0 tx=0).stop 2024-09-19T08:45:13.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.099+0000 7fcca91bd640 1 -- 172.21.15.98:0/733127375 reap_dead start 2024-09-19T08:45:13.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.099+0000 7fccaac20640 1 -- 172.21.15.98:0/733127375 shutdown_connections 2024-09-19T08:45:13.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.099+0000 7fccaac20640 1 -- 172.21.15.98:0/733127375 >> 172.21.15.98:0/733127375 conn(0x7fcca406c090 msgr2=0x7fcca406c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:13.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.099+0000 7fccaac20640 1 -- 172.21.15.98:0/733127375 shutdown_connections 2024-09-19T08:45:13.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.099+0000 7fccaac20640 1 -- 172.21.15.98:0/733127375 wait complete. 2024-09-19T08:45:13.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.103+0000 7fccaac20640 1 Processor -- start 2024-09-19T08:45:13.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.103+0000 7fccaac20640 1 -- start start 2024-09-19T08:45:13.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.103+0000 7fccaac20640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc8c004d50 0x7fcca40ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:13.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.103+0000 7fccaac20640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcca40ae5f0 0x7fcca40aea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:13.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.103+0000 7fccaac20640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcca414eb10 0x7fcca414efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:13.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.103+0000 7fccaac20640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcca415c7f0 con 0x7fcca414eb10 2024-09-19T08:45:13.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.103+0000 7fccaac20640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcca415c670 con 0x7fcc8c004d50 2024-09-19T08:45:13.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.103+0000 7fccaac20640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcca4160490 con 0x7fcca40ae5f0 2024-09-19T08:45:13.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.103+0000 7fcca91bd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcca414eb10 0x7fcca414efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:13.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.103+0000 7fcca91bd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcca414eb10 0x7fcca414efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59554/0 (socket says 172.21.15.98:59554) 2024-09-19T08:45:13.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.103+0000 7fcca91bd640 1 -- 172.21.15.98:0/231148042 learned_addr learned my addr 172.21.15.98:0/231148042 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:13.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.103+0000 7fcca91bd640 1 -- 172.21.15.98:0/231148042 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcca40ae5f0 msgr2=0x7fcca40aea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:13.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.103+0000 7fcca3fff640 1 --2- 172.21.15.98:0/231148042 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcca40ae5f0 0x7fcca40aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:13.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.103+0000 7fcca91bd640 1 --2- 172.21.15.98:0/231148042 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcca40ae5f0 0x7fcca40aea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.103+0000 7fcca91bd640 1 -- 172.21.15.98:0/231148042 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc8c004d50 msgr2=0x7fcca40ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:13.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.103+0000 7fcca89bc640 1 --2- 172.21.15.98:0/231148042 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc8c004d50 0x7fcca40ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:13.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.103+0000 7fcca91bd640 1 --2- 172.21.15.98:0/231148042 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc8c004d50 0x7fcca40ac890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.103+0000 7fcca91bd640 1 -- 172.21.15.98:0/231148042 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca40af1b0 con 0x7fcca414eb10 2024-09-19T08:45:13.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.103+0000 7fcca91bd640 1 --2- 172.21.15.98:0/231148042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcca414eb10 0x7fcca414efc0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c002240 tx=0x7fcc9c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:13.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.103+0000 7fcca17fa640 1 -- 172.21.15.98:0/231148042 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c0bd070 con 0x7fcca414eb10 2024-09-19T08:45:13.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.103+0000 7fcca17fa640 1 -- 172.21.15.98:0/231148042 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc9c0a5a90 con 0x7fcca414eb10 2024-09-19T08:45:13.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.103+0000 7fccaac20640 1 -- 172.21.15.98:0/231148042 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcca40af490 con 0x7fcca414eb10 2024-09-19T08:45:13.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.103+0000 7fcca17fa640 1 -- 172.21.15.98:0/231148042 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c0b2040 con 0x7fcca414eb10 2024-09-19T08:45:13.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.103+0000 7fccaac20640 1 -- 172.21.15.98:0/231148042 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcca40afa50 con 0x7fcca414eb10 2024-09-19T08:45:13.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.103+0000 7fccaac20640 1 -- 172.21.15.98:0/231148042 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcca4160800 con 0x7fcca414eb10 2024-09-19T08:45:13.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.107+0000 7fcca17fa640 1 -- 172.21.15.98:0/231148042 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fcc9c0c0070 con 0x7fcca414eb10 2024-09-19T08:45:13.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.107+0000 7fcca17fa640 1 --2- 172.21.15.98:0/231148042 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcc90067e90 0x7fcc9006a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:13.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.107+0000 7fcca17fa640 1 -- 172.21.15.98:0/231148042 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5867+0+0 (secure 0 0 0) 0x7fcc9c120980 con 0x7fcca414eb10 2024-09-19T08:45:13.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.107+0000 7fcca89bc640 1 --2- 172.21.15.98:0/231148042 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcc90067e90 0x7fcc9006a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:13.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.107+0000 7fcca89bc640 1 --2- 172.21.15.98:0/231148042 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcc90067e90 0x7fcc9006a350 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcc94001810 tx=0x7fcc94009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:13.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.111+0000 7fcca17fa640 1 -- 172.21.15.98:0/231148042 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fcc9c0c7020 con 0x7fcca414eb10 2024-09-19T08:45:13.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.299+0000 7fccaac20640 1 -- 172.21.15.98:0/231148042 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol56792", "force": true, "target": ["mon-mgr", ""]}) -- 0x7fcca415f940 con 0x7fcc90067e90 2024-09-19T08:45:13.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.343+0000 7fcca17fa640 1 -- 172.21.15.98:0/231148042 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcca415f940 con 0x7fcc90067e90 2024-09-19T08:45:13.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.347+0000 7fccaac20640 1 -- 172.21.15.98:0/231148042 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcc90067e90 msgr2=0x7fcc9006a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:13.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.347+0000 7fccaac20640 1 --2- 172.21.15.98:0/231148042 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcc90067e90 0x7fcc9006a350 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcc94001810 tx=0x7fcc94009040 comp rx=0 tx=0).stop 2024-09-19T08:45:13.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.347+0000 7fccaac20640 1 -- 172.21.15.98:0/231148042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcca414eb10 msgr2=0x7fcca414efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:13.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.347+0000 7fccaac20640 1 --2- 172.21.15.98:0/231148042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcca414eb10 0x7fcca414efc0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c002240 tx=0x7fcc9c009750 comp rx=0 tx=0).stop 2024-09-19T08:45:13.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.347+0000 7fccaac20640 1 -- 172.21.15.98:0/231148042 shutdown_connections 2024-09-19T08:45:13.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.347+0000 7fccaac20640 1 --2- 172.21.15.98:0/231148042 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcc90067e90 0x7fcc9006a350 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.347+0000 7fccaac20640 1 --2- 172.21.15.98:0/231148042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcca414eb10 0x7fcca414efc0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.347+0000 7fccaac20640 1 --2- 172.21.15.98:0/231148042 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcca40ae5f0 0x7fcca40aea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.347+0000 7fccaac20640 1 --2- 172.21.15.98:0/231148042 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc8c004d50 0x7fcca40ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.347+0000 7fccaac20640 1 -- 172.21.15.98:0/231148042 >> 172.21.15.98:0/231148042 conn(0x7fcca406c090 msgr2=0x7fcca40b4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:13.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.347+0000 7fccaac20640 1 -- 172.21.15.98:0/231148042 shutdown_connections 2024-09-19T08:45:13.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.347+0000 7fccaac20640 1 -- 172.21.15.98:0/231148042 wait complete. 2024-09-19T08:45:13.369 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs 2024-09-19T08:45:13.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.487+0000 7fed1d251640 1 Processor -- start 2024-09-19T08:45:13.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.487+0000 7fed1d251640 1 -- start start 2024-09-19T08:45:13.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.487+0000 7fed1d251640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fed1815c440 0x7fed180706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:13.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.487+0000 7fed1d251640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fed18072f90 0x7fed180ce400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:13.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.487+0000 7fed1d251640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed18070c20 con 0x7fed180ce940 2024-09-19T08:45:13.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.487+0000 7fed1d251640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fed1815c820 con 0x7fed18072f90 2024-09-19T08:45:13.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.487+0000 7fed1d251640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fed18070ec0 con 0x7fed1815c440 2024-09-19T08:45:13.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.487+0000 7fed1659c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fed18072f90 0x7fed180ce400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:13.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.487+0000 7fed1759e640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fed180ce940 0x7fed180b53a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56350/0 (socket says 172.21.15.98:56350) 2024-09-19T08:45:13.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.487+0000 7fed1759e640 1 -- 172.21.15.98:0/82330365 learned_addr learned my addr 172.21.15.98:0/82330365 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:13.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.487+0000 7fed16d9d640 1 --2- 172.21.15.98:0/82330365 >> v2:172.21.15.139:3300/0 conn(0x7fed1815c440 0x7fed180706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:13.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.491+0000 7fed1659c640 1 -- 172.21.15.98:0/82330365 >> v2:172.21.15.139:3300/0 conn(0x7fed1815c440 msgr2=0x7fed180706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:13.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.491+0000 7fed1659c640 1 --2- 172.21.15.98:0/82330365 >> v2:172.21.15.139:3300/0 conn(0x7fed1815c440 0x7fed180706e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.491+0000 7fed1659c640 1 -- 172.21.15.98:0/82330365 >> v1:172.21.15.98:6789/0 conn(0x7fed180ce940 legacy=0x7fed180b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:13.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.491+0000 7fed1659c640 1 -- 172.21.15.98:0/82330365 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed180711a0 con 0x7fed18072f90 2024-09-19T08:45:13.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.491+0000 7fed15d9b640 1 -- 172.21.15.98:0/82330365 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1480742895 0 0) 0x7fed18070c20 con 0x7fed180ce940 2024-09-19T08:45:13.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.491+0000 7fed1659c640 1 --2- 172.21.15.98:0/82330365 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed18072f90 0x7fed180ce400 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fed18069830 tx=0x7fed0c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1462c0e06d3ef8f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:13.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.491+0000 7fed15d9b640 1 -- 172.21.15.98:0/82330365 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0c0ab650 con 0x7fed18072f90 2024-09-19T08:45:13.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.491+0000 7fed15d9b640 1 -- 172.21.15.98:0/82330365 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed0c0b0070 con 0x7fed18072f90 2024-09-19T08:45:13.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.491+0000 7fed15d9b640 1 -- 172.21.15.98:0/82330365 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0c0aba10 con 0x7fed18072f90 2024-09-19T08:45:13.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.491+0000 7fed1d251640 1 -- 172.21.15.98:0/82330365 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed18072f90 msgr2=0x7fed180ce400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:13.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.491+0000 7fed1d251640 1 --2- 172.21.15.98:0/82330365 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed18072f90 0x7fed180ce400 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fed18069830 tx=0x7fed0c01c450 comp rx=0 tx=0).stop 2024-09-19T08:45:13.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.491+0000 7fed1d251640 1 -- 172.21.15.98:0/82330365 shutdown_connections 2024-09-19T08:45:13.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.491+0000 7fed1d251640 1 --2- 172.21.15.98:0/82330365 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed18072f90 0x7fed180ce400 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.491+0000 7fed1d251640 1 --2- 172.21.15.98:0/82330365 >> v2:172.21.15.139:3300/0 conn(0x7fed1815c440 0x7fed180706e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.491+0000 7fed1d251640 1 -- 172.21.15.98:0/82330365 >> 172.21.15.98:0/82330365 conn(0x7fed180d23d0 msgr2=0x7fed180d27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:13.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.491+0000 7fed1d251640 1 -- 172.21.15.98:0/82330365 shutdown_connections 2024-09-19T08:45:13.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.491+0000 7fed1d251640 1 -- 172.21.15.98:0/82330365 wait complete. 2024-09-19T08:45:13.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.495+0000 7fed1d251640 1 Processor -- start 2024-09-19T08:45:13.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.495+0000 7fed1d251640 1 -- start start 2024-09-19T08:45:13.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.495+0000 7fed1d251640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed18072f90 0x7fed1814e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:13.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.495+0000 7fed1d251640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed180ce940 0x7fed1806c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:13.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.495+0000 7fed1d251640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed1815c440 0x7fed180b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:13.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.495+0000 7fed1d251640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fed180b5ab0 con 0x7fed180ce940 2024-09-19T08:45:13.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.495+0000 7fed1d251640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fed18168fa0 con 0x7fed18072f90 2024-09-19T08:45:13.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.495+0000 7fed1d251640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fed1806d350 con 0x7fed1815c440 2024-09-19T08:45:13.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.495+0000 7fed1659c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed180ce940 0x7fed1806c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:13.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.495+0000 7fed1659c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed180ce940 0x7fed1806c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59576/0 (socket says 172.21.15.98:59576) 2024-09-19T08:45:13.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.495+0000 7fed1659c640 1 -- 172.21.15.98:0/2993130876 learned_addr learned my addr 172.21.15.98:0/2993130876 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:13.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.495+0000 7fed16d9d640 1 --2- 172.21.15.98:0/2993130876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed18072f90 0x7fed1814e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:13.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.495+0000 7fed1759e640 1 --2- 172.21.15.98:0/2993130876 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed1815c440 0x7fed180b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:13.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.495+0000 7fed1659c640 1 -- 172.21.15.98:0/2993130876 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed1815c440 msgr2=0x7fed180b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:13.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.495+0000 7fed1659c640 1 --2- 172.21.15.98:0/2993130876 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed1815c440 0x7fed180b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.495+0000 7fed1659c640 1 -- 172.21.15.98:0/2993130876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed18072f90 msgr2=0x7fed1814e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:13.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.495+0000 7fed1659c640 1 --2- 172.21.15.98:0/2993130876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed18072f90 0x7fed1814e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.495+0000 7fed1659c640 1 -- 172.21.15.98:0/2993130876 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed1806d5a0 con 0x7fed180ce940 2024-09-19T08:45:13.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.495+0000 7fed1759e640 1 --2- 172.21.15.98:0/2993130876 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed1815c440 0x7fed180b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:13.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.495+0000 7fed1659c640 1 --2- 172.21.15.98:0/2993130876 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed180ce940 0x7fed1806c490 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fed0c01a080 tx=0x7fed0c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:13.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.495+0000 7fecff7fe640 1 -- 172.21.15.98:0/2993130876 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0c0bf610 con 0x7fed180ce940 2024-09-19T08:45:13.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.495+0000 7fecff7fe640 1 -- 172.21.15.98:0/2993130876 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed0c0bf7d0 con 0x7fed180ce940 2024-09-19T08:45:13.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.495+0000 7fed1d251640 1 -- 172.21.15.98:0/2993130876 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed1806d880 con 0x7fed180ce940 2024-09-19T08:45:13.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.495+0000 7fecff7fe640 1 -- 172.21.15.98:0/2993130876 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0c0bfbc0 con 0x7fed180ce940 2024-09-19T08:45:13.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.495+0000 7fed1d251640 1 -- 172.21.15.98:0/2993130876 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed1806bc90 con 0x7fed180ce940 2024-09-19T08:45:13.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.495+0000 7fed1d251640 1 -- 172.21.15.98:0/2993130876 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed180ce400 con 0x7fed180ce940 2024-09-19T08:45:13.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.495+0000 7fecff7fe640 1 -- 172.21.15.98:0/2993130876 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fed0c0bd020 con 0x7fed180ce940 2024-09-19T08:45:13.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.499+0000 7fecff7fe640 1 --2- 172.21.15.98:0/2993130876 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fed04067e60 0x7fed0406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:13.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.499+0000 7fecff7fe640 1 -- 172.21.15.98:0/2993130876 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5867+0+0 (secure 0 0 0) 0x7fed0c121250 con 0x7fed180ce940 2024-09-19T08:45:13.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.499+0000 7fed16d9d640 1 --2- 172.21.15.98:0/2993130876 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fed04067e60 0x7fed0406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:13.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.499+0000 7fed16d9d640 1 --2- 172.21.15.98:0/2993130876 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fed04067e60 0x7fed0406a320 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fed180c8870 tx=0x7fed08008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:13.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.503+0000 7fecff7fe640 1 -- 172.21.15.98:0/2993130876 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fed0c016070 con 0x7fed180ce940 2024-09-19T08:45:13.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.699+0000 7fed1d251640 1 -- 172.21.15.98:0/2993130876 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7fed180741c0 con 0x7fed04067e60 2024-09-19T08:45:13.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.707+0000 7fecff7fe640 1 -- 172.21.15.98:0/2993130876 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fed180741c0 con 0x7fed04067e60 2024-09-19T08:45:13.738 INFO:teuthology.orchestra.run.smithi098.stdout:[] 2024-09-19T08:45:13.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.707+0000 7fed1d251640 1 -- 172.21.15.98:0/2993130876 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fed04067e60 msgr2=0x7fed0406a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:13.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.707+0000 7fed1d251640 1 --2- 172.21.15.98:0/2993130876 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fed04067e60 0x7fed0406a320 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fed180c8870 tx=0x7fed08008040 comp rx=0 tx=0).stop 2024-09-19T08:45:13.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.707+0000 7fed1d251640 1 -- 172.21.15.98:0/2993130876 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed180ce940 msgr2=0x7fed1806c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:13.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.707+0000 7fed1d251640 1 --2- 172.21.15.98:0/2993130876 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed180ce940 0x7fed1806c490 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fed0c01a080 tx=0x7fed0c01ac90 comp rx=0 tx=0).stop 2024-09-19T08:45:13.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.707+0000 7fed1d251640 1 -- 172.21.15.98:0/2993130876 shutdown_connections 2024-09-19T08:45:13.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.707+0000 7fed1d251640 1 --2- 172.21.15.98:0/2993130876 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fed04067e60 0x7fed0406a320 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.707+0000 7fed1d251640 1 --2- 172.21.15.98:0/2993130876 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed1815c440 0x7fed180b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.707+0000 7fed1d251640 1 --2- 172.21.15.98:0/2993130876 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed180ce940 0x7fed1806c490 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.707+0000 7fed1d251640 1 --2- 172.21.15.98:0/2993130876 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed18072f90 0x7fed1814e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:13.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.707+0000 7fed1d251640 1 -- 172.21.15.98:0/2993130876 >> 172.21.15.98:0/2993130876 conn(0x7fed180d23d0 msgr2=0x7fed180b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:13.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.707+0000 7fed1d251640 1 -- 172.21.15.98:0/2993130876 shutdown_connections 2024-09-19T08:45:13.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13.707+0000 7fed1d251640 1 -- 172.21.15.98:0/2993130876 wait complete. 2024-09-19T08:45:13.740 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-19T08:45:13.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T08:45:13.790 INFO:teuthology.orchestra.run.smithi098.stdout:2 2024-09-19T08:45:13.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:45:13.797 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T08:45:13.797 DEBUG:teuthology.orchestra.run.smithi098:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:45:13.894 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:45:13.977 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T08:45:13.977 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:45:13.977 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:45:13.977 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:45:13.984 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T08:45:14.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.155+0000 7f9f62650640 1 Processor -- start 2024-09-19T08:45:14.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.155+0000 7f9f62650640 1 -- start start 2024-09-19T08:45:14.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.155+0000 7f9f62650640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9f5c15c220 0x7f9f5c0733f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:14.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.155+0000 7f9f62650640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f5c0d1700 0x7f9f5c0b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:14.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.155+0000 7f9f62650640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f5c0740b0 con 0x7f9f5c073930 2024-09-19T08:45:14.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.155+0000 7f9f62650640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f9f5c073dd0 con 0x7f9f5c15c220 2024-09-19T08:45:14.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.155+0000 7f9f62650640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9f5c0b6460 con 0x7f9f5c0d1700 2024-09-19T08:45:14.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.155+0000 7f9f5bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9f5c15c220 0x7f9f5c0733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:14.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.155+0000 7f9f60bed640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f5c0d1700 0x7f9f5c0b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:14.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.155+0000 7f9f5b7fe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f9f5c073930 0x7f9f5c0d0ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:44922/0 (socket says 172.21.15.98:44922) 2024-09-19T08:45:14.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.155+0000 7f9f5b7fe640 1 -- 172.21.15.98:0/1446947093 learned_addr learned my addr 172.21.15.98:0/1446947093 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:14.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.155+0000 7f9f60bed640 1 -- 172.21.15.98:0/1446947093 >> v2:172.21.15.139:3300/0 conn(0x7f9f5c15c220 msgr2=0x7f9f5c0733f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:14.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.155+0000 7f9f60bed640 1 --2- 172.21.15.98:0/1446947093 >> v2:172.21.15.139:3300/0 conn(0x7f9f5c15c220 0x7f9f5c0733f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.155+0000 7f9f60bed640 1 -- 172.21.15.98:0/1446947093 >> v1:172.21.15.139:6789/0 conn(0x7f9f5c073930 legacy=0x7f9f5c0d0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:14.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.155+0000 7f9f60bed640 1 -- 172.21.15.98:0/1446947093 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f5c0b6740 con 0x7f9f5c0d1700 2024-09-19T08:45:14.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.159+0000 7f9f60bed640 1 --2- 172.21.15.98:0/1446947093 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f5c0d1700 0x7f9f5c0b6f40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0097c0 tx=0x7f9f4c01c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19c6f838448c9d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:14.162 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.159+0000 7f9f5affd640 1 -- 172.21.15.98:0/1446947093 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c0a7070 con 0x7f9f5c0d1700 2024-09-19T08:45:14.162 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.159+0000 7f9f5affd640 1 -- 172.21.15.98:0/1446947093 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f4c0b0070 con 0x7f9f5c0d1700 2024-09-19T08:45:14.162 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.159+0000 7f9f5affd640 1 -- 172.21.15.98:0/1446947093 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c0ab890 con 0x7f9f5c0d1700 2024-09-19T08:45:14.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.159+0000 7f9f62650640 1 -- 172.21.15.98:0/1446947093 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f5c0d1700 msgr2=0x7f9f5c0b6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:14.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.159+0000 7f9f62650640 1 --2- 172.21.15.98:0/1446947093 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f5c0d1700 0x7f9f5c0b6f40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0097c0 tx=0x7f9f4c01c470 comp rx=0 tx=0).stop 2024-09-19T08:45:14.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.159+0000 7f9f62650640 1 -- 172.21.15.98:0/1446947093 shutdown_connections 2024-09-19T08:45:14.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.159+0000 7f9f62650640 1 --2- 172.21.15.98:0/1446947093 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f5c0d1700 0x7f9f5c0b6f40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.159+0000 7f9f62650640 1 --2- 172.21.15.98:0/1446947093 >> v2:172.21.15.139:3300/0 conn(0x7f9f5c15c220 0x7f9f5c0733f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.159+0000 7f9f62650640 1 -- 172.21.15.98:0/1446947093 >> 172.21.15.98:0/1446947093 conn(0x7f9f5c0b40a0 msgr2=0x7f9f5c0b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:14.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.159+0000 7f9f62650640 1 -- 172.21.15.98:0/1446947093 shutdown_connections 2024-09-19T08:45:14.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.159+0000 7f9f62650640 1 -- 172.21.15.98:0/1446947093 wait complete. 2024-09-19T08:45:14.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.163+0000 7f9f62650640 1 Processor -- start 2024-09-19T08:45:14.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.163+0000 7f9f62650640 1 -- start start 2024-09-19T08:45:14.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.163+0000 7f9f62650640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f5c073930 0x7f9f5c151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:14.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.163+0000 7f9f62650640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f5c0d1700 0x7f9f5c150080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:14.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.163+0000 7f9f62650640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f5c15c220 0x7f9f5c14c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:14.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.163+0000 7f9f62650640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c15cf90 con 0x7f9f5c15c220 2024-09-19T08:45:14.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.163+0000 7f9f62650640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c15ce10 con 0x7f9f5c073930 2024-09-19T08:45:14.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.163+0000 7f9f62650640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9f5c168ea0 con 0x7f9f5c0d1700 2024-09-19T08:45:14.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.163+0000 7f9f60bed640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f5c15c220 0x7f9f5c14c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:14.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.163+0000 7f9f60bed640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f5c15c220 0x7f9f5c14c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59578/0 (socket says 172.21.15.98:59578) 2024-09-19T08:45:14.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.163+0000 7f9f60bed640 1 -- 172.21.15.98:0/1920325123 learned_addr learned my addr 172.21.15.98:0/1920325123 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:14.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.163+0000 7f9f5bfff640 1 --2- 172.21.15.98:0/1920325123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f5c073930 0x7f9f5c151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:14.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.163+0000 7f9f5b7fe640 1 --2- 172.21.15.98:0/1920325123 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f5c0d1700 0x7f9f5c150080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:14.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.163+0000 7f9f60bed640 1 -- 172.21.15.98:0/1920325123 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f5c0d1700 msgr2=0x7f9f5c150080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:14.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.163+0000 7f9f60bed640 1 --2- 172.21.15.98:0/1920325123 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f5c0d1700 0x7f9f5c150080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.163+0000 7f9f60bed640 1 -- 172.21.15.98:0/1920325123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f5c073930 msgr2=0x7f9f5c151ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:14.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.163+0000 7f9f60bed640 1 --2- 172.21.15.98:0/1920325123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f5c073930 0x7f9f5c151ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.163+0000 7f9f60bed640 1 -- 172.21.15.98:0/1920325123 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f5c152cf0 con 0x7f9f5c15c220 2024-09-19T08:45:14.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.163+0000 7f9f5bfff640 1 --2- 172.21.15.98:0/1920325123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f5c073930 0x7f9f5c151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:14.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.163+0000 7f9f60bed640 1 --2- 172.21.15.98:0/1920325123 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f5c15c220 0x7f9f5c14c670 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0097c0 tx=0x7f9f4c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:14.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.163+0000 7f9f58ff9640 1 -- 172.21.15.98:0/1920325123 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c006e20 con 0x7f9f5c15c220 2024-09-19T08:45:14.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.163+0000 7f9f58ff9640 1 -- 172.21.15.98:0/1920325123 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f4c0bf610 con 0x7f9f5c15c220 2024-09-19T08:45:14.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.163+0000 7f9f62650640 1 -- 172.21.15.98:0/1920325123 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f5c1522f0 con 0x7f9f5c15c220 2024-09-19T08:45:14.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.163+0000 7f9f58ff9640 1 -- 172.21.15.98:0/1920325123 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c0bfa00 con 0x7f9f5c15c220 2024-09-19T08:45:14.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.163+0000 7f9f62650640 1 -- 172.21.15.98:0/1920325123 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f5c151010 con 0x7f9f5c15c220 2024-09-19T08:45:14.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.163+0000 7f9f62650640 1 -- 172.21.15.98:0/1920325123 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f5c0b6f40 con 0x7f9f5c15c220 2024-09-19T08:45:14.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.167+0000 7f9f58ff9640 1 -- 172.21.15.98:0/1920325123 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f9f4c0bd020 con 0x7f9f5c15c220 2024-09-19T08:45:14.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.167+0000 7f9f58ff9640 1 --2- 172.21.15.98:0/1920325123 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9f3c067e60 0x7f9f3c06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:14.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.167+0000 7f9f58ff9640 1 -- 172.21.15.98:0/1920325123 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5867+0+0 (secure 0 0 0) 0x7f9f4c121470 con 0x7f9f5c15c220 2024-09-19T08:45:14.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.167+0000 7f9f5bfff640 1 --2- 172.21.15.98:0/1920325123 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9f3c067e60 0x7f9f3c06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:14.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.167+0000 7f9f5bfff640 1 --2- 172.21.15.98:0/1920325123 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9f3c067e60 0x7f9f3c06a320 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9f540074c0 tx=0x7f9f5400b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:14.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.171+0000 7f9f58ff9640 1 -- 172.21.15.98:0/1920325123 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9f4c016090 con 0x7f9f5c15c220 2024-09-19T08:45:14.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.343+0000 7f9f62650640 1 -- 172.21.15.98:0/1920325123 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9f5c15f790 con 0x7f9f5c15c220 2024-09-19T08:45:14.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.343+0000 7f9f58ff9640 1 -- 172.21.15.98:0/1920325123 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v122) ==== 80+0+0 (secure 0 0 0) 0x7f9f4c0f1840 con 0x7f9f5c15c220 2024-09-19T08:45:14.345 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-19T08:45:14.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.343+0000 7f9f62650640 1 -- 172.21.15.98:0/1920325123 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9f3c067e60 msgr2=0x7f9f3c06a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:14.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.343+0000 7f9f62650640 1 --2- 172.21.15.98:0/1920325123 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9f3c067e60 0x7f9f3c06a320 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9f540074c0 tx=0x7f9f5400b040 comp rx=0 tx=0).stop 2024-09-19T08:45:14.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.343+0000 7f9f62650640 1 -- 172.21.15.98:0/1920325123 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f5c15c220 msgr2=0x7f9f5c14c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:14.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.343+0000 7f9f62650640 1 --2- 172.21.15.98:0/1920325123 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f5c15c220 0x7f9f5c14c670 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0097c0 tx=0x7f9f4c01acb0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.343+0000 7f9f62650640 1 -- 172.21.15.98:0/1920325123 shutdown_connections 2024-09-19T08:45:14.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.343+0000 7f9f62650640 1 --2- 172.21.15.98:0/1920325123 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9f3c067e60 0x7f9f3c06a320 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.343+0000 7f9f62650640 1 --2- 172.21.15.98:0/1920325123 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f5c15c220 0x7f9f5c14c670 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.343+0000 7f9f62650640 1 --2- 172.21.15.98:0/1920325123 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f5c0d1700 0x7f9f5c150080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.343+0000 7f9f62650640 1 --2- 172.21.15.98:0/1920325123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f5c073930 0x7f9f5c151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.343+0000 7f9f62650640 1 -- 172.21.15.98:0/1920325123 >> 172.21.15.98:0/1920325123 conn(0x7f9f5c0b40a0 msgr2=0x7f9f5c152f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:14.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.343+0000 7f9f62650640 1 -- 172.21.15.98:0/1920325123 shutdown_connections 2024-09-19T08:45:14.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.343+0000 7f9f62650640 1 -- 172.21.15.98:0/1920325123 wait complete. 2024-09-19T08:45:14.364 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi098.front.sepia.ceph.com... 2024-09-19T08:45:14.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:14.364 DEBUG:teuthology.orchestra.run.smithi098:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T08:45:14.374 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:ceph-fuse[48118]: fuse finished with error 0 and tester_r 0 2024-09-19T08:45:14.375 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:45:14.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:14.375 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T08:45:14.532 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:45:14.532 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:45:14.532 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:45:14.539 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T08:45:14.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.703+0000 7f51fd043640 1 Processor -- start 2024-09-19T08:45:14.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.703+0000 7f51fd043640 1 -- start start 2024-09-19T08:45:14.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.703+0000 7f51fd043640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f51f815c220 0x7f51f80733f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:14.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.703+0000 7f51fd043640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f51f80d1700 0x7f51f80b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:14.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.703+0000 7f51fd043640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51f80740b0 con 0x7f51f8073930 2024-09-19T08:45:14.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.703+0000 7f51fd043640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f51f8073dd0 con 0x7f51f80d1700 2024-09-19T08:45:14.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.703+0000 7f51fd043640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f51f80b6460 con 0x7f51f815c220 2024-09-19T08:45:14.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.703+0000 7f51f759e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f51f80d1700 0x7f51f80b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:14.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.703+0000 7f51f759e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f51f80d1700 0x7f51f80b6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59582/0 (socket says 172.21.15.98:59582) 2024-09-19T08:45:14.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.703+0000 7f51f759e640 1 -- 172.21.15.98:0/3414158331 learned_addr learned my addr 172.21.15.98:0/3414158331 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:14.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.703+0000 7f51f6d9d640 1 --2- 172.21.15.98:0/3414158331 >> v2:172.21.15.139:3300/0 conn(0x7f51f815c220 0x7f51f80733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:14.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.703+0000 7f51f759e640 1 -- 172.21.15.98:0/3414158331 >> v2:172.21.15.139:3300/0 conn(0x7f51f815c220 msgr2=0x7f51f80733f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:14.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.703+0000 7f51f759e640 1 --2- 172.21.15.98:0/3414158331 >> v2:172.21.15.139:3300/0 conn(0x7f51f815c220 0x7f51f80733f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.703+0000 7f51f759e640 1 -- 172.21.15.98:0/3414158331 >> v1:172.21.15.139:6789/0 conn(0x7f51f8073930 legacy=0x7f51f80d0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:14.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.703+0000 7f51f759e640 1 -- 172.21.15.98:0/3414158331 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f80b6740 con 0x7f51f80d1700 2024-09-19T08:45:14.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.707+0000 7f51f759e640 1 --2- 172.21.15.98:0/3414158331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51f80d1700 0x7f51f80b6f40 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f51f8070350 tx=0x7f51e801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a96c265950fe7d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:14.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.707+0000 7f51f5d9b640 1 -- 172.21.15.98:0/3414158331 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e80aa610 con 0x7f51f80d1700 2024-09-19T08:45:14.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.707+0000 7f51f5d9b640 1 -- 172.21.15.98:0/3414158331 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51e80af070 con 0x7f51f80d1700 2024-09-19T08:45:14.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.707+0000 7f51f5d9b640 1 -- 172.21.15.98:0/3414158331 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e80aa9d0 con 0x7f51f80d1700 2024-09-19T08:45:14.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.707+0000 7f51fd043640 1 -- 172.21.15.98:0/3414158331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51f80d1700 msgr2=0x7f51f80b6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:14.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.707+0000 7f51fd043640 1 --2- 172.21.15.98:0/3414158331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51f80d1700 0x7f51f80b6f40 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f51f8070350 tx=0x7f51e801c4b0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.707+0000 7f51fd043640 1 -- 172.21.15.98:0/3414158331 shutdown_connections 2024-09-19T08:45:14.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.707+0000 7f51fd043640 1 --2- 172.21.15.98:0/3414158331 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51f80d1700 0x7f51f80b6f40 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.707+0000 7f51fd043640 1 --2- 172.21.15.98:0/3414158331 >> v2:172.21.15.139:3300/0 conn(0x7f51f815c220 0x7f51f80733f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.707+0000 7f51fd043640 1 -- 172.21.15.98:0/3414158331 >> 172.21.15.98:0/3414158331 conn(0x7f51f80b40a0 msgr2=0x7f51f80b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:14.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.707+0000 7f51fd043640 1 -- 172.21.15.98:0/3414158331 shutdown_connections 2024-09-19T08:45:14.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.707+0000 7f51fd043640 1 -- 172.21.15.98:0/3414158331 wait complete. 2024-09-19T08:45:14.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.707+0000 7f51fd043640 1 Processor -- start 2024-09-19T08:45:14.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.707+0000 7f51fd043640 1 -- start start 2024-09-19T08:45:14.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.707+0000 7f51fd043640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51f8073930 0x7f51f815aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:14.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.711+0000 7f51fd043640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51f80d1700 0x7f51f8158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:14.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.711+0000 7f51fd043640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f51f815c220 0x7f51f814ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:14.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.711+0000 7f51fd043640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f51f815cf90 con 0x7f51f80d1700 2024-09-19T08:45:14.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.711+0000 7f51fd043640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f51f815ce10 con 0x7f51f8073930 2024-09-19T08:45:14.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.711+0000 7f51fd043640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f51f8168ea0 con 0x7f51f815c220 2024-09-19T08:45:14.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.711+0000 7f51f659c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51f80d1700 0x7f51f8158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:14.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.711+0000 7f51f659c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51f80d1700 0x7f51f8158620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59598/0 (socket says 172.21.15.98:59598) 2024-09-19T08:45:14.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.711+0000 7f51f659c640 1 -- 172.21.15.98:0/1353831133 learned_addr learned my addr 172.21.15.98:0/1353831133 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:14.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.711+0000 7f51f759e640 1 --2- 172.21.15.98:0/1353831133 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f51f815c220 0x7f51f814ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:14.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.711+0000 7f51f6d9d640 1 --2- 172.21.15.98:0/1353831133 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51f8073930 0x7f51f815aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:14.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.711+0000 7f51f659c640 1 -- 172.21.15.98:0/1353831133 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f51f815c220 msgr2=0x7f51f814ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:14.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.711+0000 7f51f659c640 1 --2- 172.21.15.98:0/1353831133 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f51f815c220 0x7f51f814ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.711+0000 7f51f659c640 1 -- 172.21.15.98:0/1353831133 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51f8073930 msgr2=0x7f51f815aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:14.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.711+0000 7f51f659c640 1 --2- 172.21.15.98:0/1353831133 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51f8073930 0x7f51f815aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.711+0000 7f51f659c640 1 -- 172.21.15.98:0/1353831133 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f8155660 con 0x7f51f80d1700 2024-09-19T08:45:14.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.711+0000 7f51f6d9d640 1 --2- 172.21.15.98:0/1353831133 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51f8073930 0x7f51f815aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:14.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.711+0000 7f51f659c640 1 --2- 172.21.15.98:0/1353831133 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51f80d1700 0x7f51f8158620 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f51e000c480 tx=0x7f51e000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:14.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.711+0000 7f51ef7fe640 1 -- 172.21.15.98:0/1353831133 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e0016020 con 0x7f51f80d1700 2024-09-19T08:45:14.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.711+0000 7f51ef7fe640 1 -- 172.21.15.98:0/1353831133 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51e0012830 con 0x7f51f80d1700 2024-09-19T08:45:14.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.711+0000 7f51fd043640 1 -- 172.21.15.98:0/1353831133 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51f8155940 con 0x7f51f80d1700 2024-09-19T08:45:14.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.711+0000 7f51ef7fe640 1 -- 172.21.15.98:0/1353831133 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e0012c20 con 0x7f51f80d1700 2024-09-19T08:45:14.715 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.711+0000 7f51fd043640 1 -- 172.21.15.98:0/1353831133 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51f815afd0 con 0x7f51f80d1700 2024-09-19T08:45:14.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.711+0000 7f51fd043640 1 -- 172.21.15.98:0/1353831133 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51f80b6f40 con 0x7f51f80d1700 2024-09-19T08:45:14.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.711+0000 7f51ef7fe640 1 -- 172.21.15.98:0/1353831133 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f51e0019020 con 0x7f51f80d1700 2024-09-19T08:45:14.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.711+0000 7f51ef7fe640 1 --2- 172.21.15.98:0/1353831133 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f51d8067e60 0x7f51d806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:14.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.711+0000 7f51ef7fe640 1 -- 172.21.15.98:0/1353831133 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5867+0+0 (secure 0 0 0) 0x7f51e00861c0 con 0x7f51f80d1700 2024-09-19T08:45:14.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.711+0000 7f51f6d9d640 1 --2- 172.21.15.98:0/1353831133 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f51d8067e60 0x7f51d806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:14.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.711+0000 7f51f6d9d640 1 --2- 172.21.15.98:0/1353831133 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f51d8067e60 0x7f51d806a320 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f51f80b2420 tx=0x7f51f000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:14.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.715+0000 7f51ef7fe640 1 -- 172.21.15.98:0/1353831133 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f51e0020070 con 0x7f51f80d1700 2024-09-19T08:45:14.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.891+0000 7f51fd043640 1 -- 172.21.15.98:0/1353831133 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f51f80d0640 con 0x7f51f80d1700 2024-09-19T08:45:14.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.891+0000 7f51ef7fe640 1 -- 172.21.15.98:0/1353831133 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v122) ==== 80+0+0 (secure 0 0 0) 0x7f51e0056480 con 0x7f51f80d1700 2024-09-19T08:45:14.894 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-19T08:45:14.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.895+0000 7f51fd043640 1 -- 172.21.15.98:0/1353831133 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f51d8067e60 msgr2=0x7f51d806a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:14.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.895+0000 7f51fd043640 1 --2- 172.21.15.98:0/1353831133 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f51d8067e60 0x7f51d806a320 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f51f80b2420 tx=0x7f51f000b040 comp rx=0 tx=0).stop 2024-09-19T08:45:14.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.895+0000 7f51fd043640 1 -- 172.21.15.98:0/1353831133 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51f80d1700 msgr2=0x7f51f8158620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:14.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.895+0000 7f51fd043640 1 --2- 172.21.15.98:0/1353831133 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51f80d1700 0x7f51f8158620 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f51e000c480 tx=0x7f51e000c940 comp rx=0 tx=0).stop 2024-09-19T08:45:14.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.895+0000 7f51fd043640 1 -- 172.21.15.98:0/1353831133 shutdown_connections 2024-09-19T08:45:14.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.895+0000 7f51fd043640 1 --2- 172.21.15.98:0/1353831133 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f51d8067e60 0x7f51d806a320 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.895+0000 7f51fd043640 1 --2- 172.21.15.98:0/1353831133 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f51f815c220 0x7f51f814ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.895+0000 7f51fd043640 1 --2- 172.21.15.98:0/1353831133 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f51f80d1700 0x7f51f8158620 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.895+0000 7f51fd043640 1 --2- 172.21.15.98:0/1353831133 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51f8073930 0x7f51f815aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:14.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.895+0000 7f51fd043640 1 -- 172.21.15.98:0/1353831133 >> 172.21.15.98:0/1353831133 conn(0x7f51f80b40a0 msgr2=0x7f51f80703d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:14.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.895+0000 7f51fd043640 1 -- 172.21.15.98:0/1353831133 shutdown_connections 2024-09-19T08:45:14.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:14.895+0000 7f51fd043640 1 -- 172.21.15.98:0/1353831133 wait complete. 2024-09-19T08:45:14.913 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi139.front.sepia.ceph.com... 2024-09-19T08:45:14.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:14.913 DEBUG:teuthology.orchestra.run.smithi139:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-19T08:45:14.926 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[36386]: fuse finished with error 0 and tester_r 0 2024-09-19T08:45:14.928 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:45:14.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:14.928 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-19T08:45:14.990 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:45:14.990 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:45:14.990 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:45:14.996 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15846 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T08:45:14.996 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:45:14.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:14.996 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-19T08:45:15.048 INFO:teuthology.orchestra.run.smithi098.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-19T08:45:15.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:45:15.049 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:45:15.050 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:45:15.050 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:45:15.056 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T08:45:15.056 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:45:15.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:15.056 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-19T08:45:15.103 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-19T08:45:15.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:45:15.104 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:45:15.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.231+0000 7f0293b6f640 1 Processor -- start 2024-09-19T08:45:15.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.231+0000 7f0293b6f640 1 -- start start 2024-09-19T08:45:15.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.231+0000 7f0293b6f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f028c15c550 0x7f028c06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:15.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.235+0000 7f0293b6f640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f028c06d310 con 0x7f028c0cfad0 2024-09-19T08:45:15.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.235+0000 7f0293b6f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f028c06d5f0 con 0x7f028c06c1d0 2024-09-19T08:45:15.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.235+0000 7f0293b6f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f028c06c610 con 0x7f028c15c550 2024-09-19T08:45:15.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.235+0000 7f029210c640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f028c0cfad0 0x7f028c0723a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56366/0 (socket says 172.21.15.98:56366) 2024-09-19T08:45:15.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.235+0000 7f029210c640 1 -- 172.21.15.98:0/801663585 learned_addr learned my addr 172.21.15.98:0/801663585 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:15.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.235+0000 7f029190b640 1 --2- 172.21.15.98:0/801663585 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f028c15c550 0x7f028c06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:15.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.235+0000 7f029190b640 1 -- 172.21.15.98:0/801663585 >> v1:172.21.15.139:6789/0 conn(0x7f028c06c1d0 legacy=0x7f028c0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:15.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.235+0000 7f029190b640 1 -- 172.21.15.98:0/801663585 >> v1:172.21.15.98:6789/0 conn(0x7f028c0cfad0 legacy=0x7f028c0723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:15.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.235+0000 7f029190b640 1 -- 172.21.15.98:0/801663585 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f028c06d8d0 con 0x7f028c15c550 2024-09-19T08:45:15.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.235+0000 7f029190b640 1 --2- 172.21.15.98:0/801663585 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f028c15c550 0x7f028c06bc90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f02880097f0 tx=0x7f028801c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc418189703b9c8a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:15.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.239+0000 7f0290909640 1 -- 172.21.15.98:0/801663585 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02880b0070 con 0x7f028c15c550 2024-09-19T08:45:15.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.239+0000 7f0290909640 1 -- 172.21.15.98:0/801663585 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02880ab600 con 0x7f028c15c550 2024-09-19T08:45:15.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.239+0000 7f0290909640 1 -- 172.21.15.98:0/801663585 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02880ab9d0 con 0x7f028c15c550 2024-09-19T08:45:15.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.239+0000 7f0293b6f640 1 -- 172.21.15.98:0/801663585 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f028c15c550 msgr2=0x7f028c06bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:15.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.239+0000 7f0293b6f640 1 --2- 172.21.15.98:0/801663585 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f028c15c550 0x7f028c06bc90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f02880097f0 tx=0x7f028801c4e0 comp rx=0 tx=0).stop 2024-09-19T08:45:15.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.239+0000 7f0293b6f640 1 -- 172.21.15.98:0/801663585 shutdown_connections 2024-09-19T08:45:15.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.239+0000 7f0293b6f640 1 --2- 172.21.15.98:0/801663585 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f028c15c550 0x7f028c06bc90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:15.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.239+0000 7f0293b6f640 1 -- 172.21.15.98:0/801663585 >> 172.21.15.98:0/801663585 conn(0x7f028c0b2010 msgr2=0x7f028c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:15.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.239+0000 7f0293b6f640 1 -- 172.21.15.98:0/801663585 shutdown_connections 2024-09-19T08:45:15.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.239+0000 7f0293b6f640 1 -- 172.21.15.98:0/801663585 wait complete. 2024-09-19T08:45:15.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.239+0000 7f0293b6f640 1 Processor -- start 2024-09-19T08:45:15.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.239+0000 7f0293b6f640 1 -- start start 2024-09-19T08:45:15.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.243+0000 7f0293b6f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f028c06c1d0 0x7f028c148a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:15.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.243+0000 7f0293b6f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f028c0cfad0 0x7f028c1507b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:15.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.243+0000 7f0293b6f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f028c15c550 0x7f028c150cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:15.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.243+0000 7f0293b6f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f028c073510 con 0x7f028c15c550 2024-09-19T08:45:15.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.243+0000 7f0293b6f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f028c15cb40 con 0x7f028c0cfad0 2024-09-19T08:45:15.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.243+0000 7f0293b6f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f028c073690 con 0x7f028c06c1d0 2024-09-19T08:45:15.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.243+0000 7f029210c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f028c15c550 0x7f028c150cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:15.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.243+0000 7f029210c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f028c15c550 0x7f028c150cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59612/0 (socket says 172.21.15.98:59612) 2024-09-19T08:45:15.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.243+0000 7f029210c640 1 -- 172.21.15.98:0/218765401 learned_addr learned my addr 172.21.15.98:0/218765401 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:15.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.243+0000 7f029190b640 1 --2- 172.21.15.98:0/218765401 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f028c06c1d0 0x7f028c148a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:15.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.243+0000 7f029110a640 1 --2- 172.21.15.98:0/218765401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f028c0cfad0 0x7f028c1507b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:15.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.243+0000 7f029210c640 1 -- 172.21.15.98:0/218765401 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f028c06c1d0 msgr2=0x7f028c148a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:15.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.243+0000 7f029210c640 1 --2- 172.21.15.98:0/218765401 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f028c06c1d0 0x7f028c148a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:15.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.243+0000 7f029210c640 1 -- 172.21.15.98:0/218765401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f028c0cfad0 msgr2=0x7f028c1507b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:15.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.243+0000 7f029210c640 1 --2- 172.21.15.98:0/218765401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f028c0cfad0 0x7f028c1507b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:15.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.243+0000 7f029210c640 1 -- 172.21.15.98:0/218765401 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f028c15ea50 con 0x7f028c15c550 2024-09-19T08:45:15.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.243+0000 7f029210c640 1 --2- 172.21.15.98:0/218765401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f028c15c550 0x7f028c150cf0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f028400ca80 tx=0x7f028400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:15.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.243+0000 7f02827fc640 1 -- 172.21.15.98:0/218765401 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0284017070 con 0x7f028c15c550 2024-09-19T08:45:15.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.243+0000 7f02827fc640 1 -- 172.21.15.98:0/218765401 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f028400ac30 con 0x7f028c15c550 2024-09-19T08:45:15.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.243+0000 7f0293b6f640 1 -- 172.21.15.98:0/218765401 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f028c15d9b0 con 0x7f028c15c550 2024-09-19T08:45:15.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.243+0000 7f02827fc640 1 -- 172.21.15.98:0/218765401 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0284012400 con 0x7f028c15c550 2024-09-19T08:45:15.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.243+0000 7f0293b6f640 1 -- 172.21.15.98:0/218765401 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f028c15df70 con 0x7f028c15c550 2024-09-19T08:45:15.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.243+0000 7f0293b6f640 1 -- 172.21.15.98:0/218765401 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f028c06bc90 con 0x7f028c15c550 2024-09-19T08:45:15.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.243+0000 7f02827fc640 1 -- 172.21.15.98:0/218765401 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f028401a070 con 0x7f028c15c550 2024-09-19T08:45:15.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.243+0000 7f02827fc640 1 --2- 172.21.15.98:0/218765401 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0270067e60 0x7f027006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:15.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.243+0000 7f02827fc640 1 -- 172.21.15.98:0/218765401 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5823+0+0 (secure 0 0 0) 0x7f0284086320 con 0x7f028c15c550 2024-09-19T08:45:15.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.243+0000 7f029190b640 1 --2- 172.21.15.98:0/218765401 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0270067e60 0x7f027006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:15.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.247+0000 7f029190b640 1 --2- 172.21.15.98:0/218765401 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0270067e60 0x7f027006a320 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f02880ad000 tx=0x7f02880a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:15.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.247+0000 7f02827fc640 1 -- 172.21.15.98:0/218765401 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f0284055f30 con 0x7f028c15c550 2024-09-19T08:45:15.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.451+0000 7f0293b6f640 1 -- 172.21.15.98:0/218765401 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f028c0d0ca0 con 0x7f028c15c550 2024-09-19T08:45:15.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.451+0000 7f02827fc640 1 -- 172.21.15.98:0/218765401 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5228 (secure 0 0 0) 0x7f028405a230 con 0x7f028c15c550 2024-09-19T08:45:15.453 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:45:15.453 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":68,"btime":"2024-09-19T08:45:07:966984+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5645,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/1334032829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":1334032829},{"type":"v1","addr":"172.21.15.139:6833","nonce":1334032829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":5651,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2541341035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2541341035},{"type":"v1","addr":"172.21.15.139:6835","nonce":2541341035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":7521,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3632520818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3632520818},{"type":"v1","addr":"172.21.15.98:6833","nonce":3632520818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:44:55.910161+0000","modified":"2024-09-19T08:44:57.947782+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7524},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7524":{"gid":7524,"name":"c","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/502938874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":502938874},{"type":"v1","addr":"172.21.15.98:6835","nonce":502938874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7524,"qdb_cluster":[7524]},"id":9}]} 2024-09-19T08:45:15.453 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 68 2024-09-19T08:45:15.455 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.451+0000 7f0293b6f640 1 -- 172.21.15.98:0/218765401 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0270067e60 msgr2=0x7f027006a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:15.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.451+0000 7f0293b6f640 1 --2- 172.21.15.98:0/218765401 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0270067e60 0x7f027006a320 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f02880ad000 tx=0x7f02880a6000 comp rx=0 tx=0).stop 2024-09-19T08:45:15.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.451+0000 7f0293b6f640 1 -- 172.21.15.98:0/218765401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f028c15c550 msgr2=0x7f028c150cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:15.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.451+0000 7f0293b6f640 1 --2- 172.21.15.98:0/218765401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f028c15c550 0x7f028c150cf0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f028400ca80 tx=0x7f028400cf40 comp rx=0 tx=0).stop 2024-09-19T08:45:15.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.451+0000 7f0293b6f640 1 -- 172.21.15.98:0/218765401 shutdown_connections 2024-09-19T08:45:15.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.451+0000 7f0293b6f640 1 --2- 172.21.15.98:0/218765401 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0270067e60 0x7f027006a320 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:15.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.451+0000 7f0293b6f640 1 --2- 172.21.15.98:0/218765401 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f028c15c550 0x7f028c150cf0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:15.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.451+0000 7f0293b6f640 1 --2- 172.21.15.98:0/218765401 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f028c0cfad0 0x7f028c1507b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:15.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.451+0000 7f0293b6f640 1 --2- 172.21.15.98:0/218765401 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f028c06c1d0 0x7f028c148a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:15.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.451+0000 7f0293b6f640 1 -- 172.21.15.98:0/218765401 >> 172.21.15.98:0/218765401 conn(0x7f028c0b2010 msgr2=0x7f028c0cf3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:15.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.451+0000 7f0293b6f640 1 -- 172.21.15.98:0/218765401 shutdown_connections 2024-09-19T08:45:15.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.451+0000 7f0293b6f640 1 -- 172.21.15.98:0/218765401 wait complete. 2024-09-19T08:45:15.473 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:45:15.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.583+0000 7fa1f8709640 1 Processor -- start 2024-09-19T08:45:15.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.583+0000 7fa1f8709640 1 -- start start 2024-09-19T08:45:15.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.587+0000 7fa1f8709640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa1f006bc90 0x7fa1f006c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:15.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.587+0000 7fa1f8709640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1f00ccc60 con 0x7fa1f00ce840 2024-09-19T08:45:15.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.587+0000 7fa1f8709640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1f00ccf40 con 0x7fa1f0158fd0 2024-09-19T08:45:15.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.587+0000 7fa1f8709640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa1f01593b0 con 0x7fa1f006bc90 2024-09-19T08:45:15.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.587+0000 7fa1f6ca6640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fa1f00ce840 0x7fa1f00cec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56380/0 (socket says 172.21.15.98:56380) 2024-09-19T08:45:15.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.587+0000 7fa1f6ca6640 1 -- 172.21.15.98:0/3680166261 learned_addr learned my addr 172.21.15.98:0/3680166261 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:15.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.587+0000 7fa1f5ca4640 1 --2- 172.21.15.98:0/3680166261 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa1f006bc90 0x7fa1f006c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:15.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.587+0000 7fa1f54a3640 1 -- 172.21.15.98:0/3680166261 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3364967929 0 0) 0x7fa1f00ccc60 con 0x7fa1f00ce840 2024-09-19T08:45:15.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.587+0000 7fa1f54a3640 1 -- 172.21.15.98:0/3680166261 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1dc003530 con 0x7fa1f00ce840 2024-09-19T08:45:15.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.587+0000 7fa1f54a3640 1 -- 172.21.15.98:0/3680166261 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 29625027 0 0) 0x7fa1dc003530 con 0x7fa1f00ce840 2024-09-19T08:45:15.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.587+0000 7fa1f54a3640 1 -- 172.21.15.98:0/3680166261 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa1f006bc90 msgr2=0x7fa1f006c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:15.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.587+0000 7fa1f54a3640 1 --2- 172.21.15.98:0/3680166261 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa1f006bc90 0x7fa1f006c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:15.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.587+0000 7fa1f54a3640 1 -- 172.21.15.98:0/3680166261 >> v1:172.21.15.139:6789/0 conn(0x7fa1f0158fd0 legacy=0x7fa1f00b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:15.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.587+0000 7fa1f54a3640 1 -- 172.21.15.98:0/3680166261 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f00cd220 con 0x7fa1f00ce840 2024-09-19T08:45:15.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.587+0000 7fa1f54a3640 1 -- 172.21.15.98:0/3680166261 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fa1e8001a00 con 0x7fa1f00ce840 2024-09-19T08:45:15.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.587+0000 7fa1f54a3640 1 -- 172.21.15.98:0/3680166261 >> v1:172.21.15.98:6789/0 conn(0x7fa1f00ce840 legacy=0x7fa1f00cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:15.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.587+0000 7fa1f54a3640 1 --2- 172.21.15.98:0/3680166261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1dc0039f0 0x7fa1dc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:15.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.587+0000 7fa1f54a3640 1 --2- 172.21.15.98:0/3680166261 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa1f006bc90 0x7fa1dc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:15.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.587+0000 7fa1f54a3640 1 --2- 172.21.15.98:0/3680166261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1dc004d50 0x7fa1dc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:15.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.587+0000 7fa1f54a3640 1 -- 172.21.15.98:0/3680166261 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f01593b0 con 0x7fa1dc004d50 2024-09-19T08:45:15.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.587+0000 7fa1f54a3640 1 -- 172.21.15.98:0/3680166261 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa1dc00bb30 con 0x7fa1dc0039f0 2024-09-19T08:45:15.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.587+0000 7fa1f54a3640 1 -- 172.21.15.98:0/3680166261 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa1dc00bd30 con 0x7fa1f006bc90 2024-09-19T08:45:15.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.587+0000 7fa1f6ca6640 1 --2- 172.21.15.98:0/3680166261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1dc004d50 0x7fa1dc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:15.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.587+0000 7fa1f64a5640 1 --2- 172.21.15.98:0/3680166261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1dc0039f0 0x7fa1dc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:15.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.587+0000 7fa1f6ca6640 1 -- 172.21.15.98:0/3680166261 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa1f006bc90 msgr2=0x7fa1dc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:15.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.587+0000 7fa1f6ca6640 1 --2- 172.21.15.98:0/3680166261 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa1f006bc90 0x7fa1dc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:15.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.587+0000 7fa1f6ca6640 1 -- 172.21.15.98:0/3680166261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1dc0039f0 msgr2=0x7fa1dc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:15.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.587+0000 7fa1f6ca6640 1 --2- 172.21.15.98:0/3680166261 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1dc0039f0 0x7fa1dc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:15.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.587+0000 7fa1f6ca6640 1 -- 172.21.15.98:0/3680166261 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1dc003530 con 0x7fa1dc004d50 2024-09-19T08:45:15.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.591+0000 7fa1f6ca6640 1 --2- 172.21.15.98:0/3680166261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1dc004d50 0x7fa1dc0051d0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800a860 tx=0x7fa1e801fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a843ab16c41eb1ca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:15.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.591+0000 7fa1f54a3640 1 -- 172.21.15.98:0/3680166261 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e800a480 con 0x7fa1dc004d50 2024-09-19T08:45:15.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.591+0000 7fa1f54a3640 1 -- 172.21.15.98:0/3680166261 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1e80aa710 con 0x7fa1dc004d50 2024-09-19T08:45:15.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.591+0000 7fa1f54a3640 1 -- 172.21.15.98:0/3680166261 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e80aab40 con 0x7fa1dc004d50 2024-09-19T08:45:15.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.591+0000 7fa1f8709640 1 -- 172.21.15.98:0/3680166261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1dc004d50 msgr2=0x7fa1dc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:15.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.591+0000 7fa1f8709640 1 --2- 172.21.15.98:0/3680166261 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1dc004d50 0x7fa1dc0051d0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800a860 tx=0x7fa1e801fae0 comp rx=0 tx=0).stop 2024-09-19T08:45:15.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.591+0000 7fa1f6ca6640 1 -- 172.21.15.98:0/3680166261 reap_dead start 2024-09-19T08:45:15.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.591+0000 7fa1f8709640 1 -- 172.21.15.98:0/3680166261 shutdown_connections 2024-09-19T08:45:15.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.591+0000 7fa1f8709640 1 -- 172.21.15.98:0/3680166261 >> 172.21.15.98:0/3680166261 conn(0x7fa1f00b2010 msgr2=0x7fa1f00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:15.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.591+0000 7fa1f8709640 1 -- 172.21.15.98:0/3680166261 shutdown_connections 2024-09-19T08:45:15.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.591+0000 7fa1f8709640 1 -- 172.21.15.98:0/3680166261 wait complete. 2024-09-19T08:45:15.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.591+0000 7fa1f8709640 1 Processor -- start 2024-09-19T08:45:15.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.591+0000 7fa1f8709640 1 -- start start 2024-09-19T08:45:15.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.591+0000 7fa1f8709640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa1f0157fc0 0x7fa1f00b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:15.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.591+0000 7fa1f8709640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1f00b1490 0x7fa1f01f2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:15.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.591+0000 7fa1f8709640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1f01f2580 0x7fa1f00ae370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:15.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.591+0000 7fa1f8709640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f00b4530 con 0x7fa1f01f2580 2024-09-19T08:45:15.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.591+0000 7fa1f8709640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f00cd7e0 con 0x7fa1f00b1490 2024-09-19T08:45:15.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.591+0000 7fa1f8709640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa1f015c7a0 con 0x7fa1f0157fc0 2024-09-19T08:45:15.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.591+0000 7fa1f6ca6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1f01f2580 0x7fa1f00ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:15.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.591+0000 7fa1f6ca6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1f01f2580 0x7fa1f00ae370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59630/0 (socket says 172.21.15.98:59630) 2024-09-19T08:45:15.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.591+0000 7fa1f6ca6640 1 -- 172.21.15.98:0/86148270 learned_addr learned my addr 172.21.15.98:0/86148270 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:15.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.591+0000 7fa1f5ca4640 1 --2- 172.21.15.98:0/86148270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1f00b1490 0x7fa1f01f2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:15.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.591+0000 7fa1f64a5640 1 --2- 172.21.15.98:0/86148270 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa1f0157fc0 0x7fa1f00b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:15.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.591+0000 7fa1f6ca6640 1 -- 172.21.15.98:0/86148270 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa1f0157fc0 msgr2=0x7fa1f00b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:15.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.591+0000 7fa1f6ca6640 1 --2- 172.21.15.98:0/86148270 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa1f0157fc0 0x7fa1f00b0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:15.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.591+0000 7fa1f6ca6640 1 -- 172.21.15.98:0/86148270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1f00b1490 msgr2=0x7fa1f01f2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:15.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.591+0000 7fa1f6ca6640 1 --2- 172.21.15.98:0/86148270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1f00b1490 0x7fa1f01f2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:15.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.591+0000 7fa1f6ca6640 1 -- 172.21.15.98:0/86148270 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f00aa640 con 0x7fa1f01f2580 2024-09-19T08:45:15.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.591+0000 7fa1f6ca6640 1 --2- 172.21.15.98:0/86148270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1f01f2580 0x7fa1f00ae370 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8001d00 tx=0x7fa1dc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:15.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.591+0000 7fa1e6ffd640 1 -- 172.21.15.98:0/86148270 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e800a480 con 0x7fa1f01f2580 2024-09-19T08:45:15.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.591+0000 7fa1e6ffd640 1 -- 172.21.15.98:0/86148270 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1e80aa510 con 0x7fa1f01f2580 2024-09-19T08:45:15.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.591+0000 7fa1f8709640 1 -- 172.21.15.98:0/86148270 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f00aa920 con 0x7fa1f01f2580 2024-09-19T08:45:15.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.595+0000 7fa1e6ffd640 1 -- 172.21.15.98:0/86148270 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e80aa8c0 con 0x7fa1f01f2580 2024-09-19T08:45:15.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.595+0000 7fa1f8709640 1 -- 172.21.15.98:0/86148270 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f00ab400 con 0x7fa1f01f2580 2024-09-19T08:45:15.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.595+0000 7fa1f8709640 1 -- 172.21.15.98:0/86148270 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1f006c120 con 0x7fa1f01f2580 2024-09-19T08:45:15.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.595+0000 7fa1e6ffd640 1 -- 172.21.15.98:0/86148270 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fa1e80ba030 con 0x7fa1f01f2580 2024-09-19T08:45:15.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.595+0000 7fa1e6ffd640 1 --2- 172.21.15.98:0/86148270 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa1d4067e90 0x7fa1d406a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:15.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.595+0000 7fa1e6ffd640 1 -- 172.21.15.98:0/86148270 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5823+0+0 (secure 0 0 0) 0x7fa1e8123660 con 0x7fa1f01f2580 2024-09-19T08:45:15.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.595+0000 7fa1f64a5640 1 --2- 172.21.15.98:0/86148270 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa1d4067e90 0x7fa1d406a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:15.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.595+0000 7fa1f64a5640 1 --2- 172.21.15.98:0/86148270 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa1d4067e90 0x7fa1d406a350 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec005510 tx=0x7fa1ec009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:15.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.599+0000 7fa1e6ffd640 1 -- 172.21.15.98:0/86148270 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa1e80ae030 con 0x7fa1f01f2580 2024-09-19T08:45:15.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.787+0000 7fa1f8709640 1 -- 172.21.15.98:0/86148270 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1f00cfdd0 con 0x7fa1f01f2580 2024-09-19T08:45:15.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.787+0000 7fa1e6ffd640 1 -- 172.21.15.98:0/86148270 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5228 (secure 0 0 0) 0x7fa1e80197f0 con 0x7fa1f01f2580 2024-09-19T08:45:15.789 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:45:15.790 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":68,"btime":"2024-09-19T08:45:07:966984+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5645,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/1334032829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":1334032829},{"type":"v1","addr":"172.21.15.139:6833","nonce":1334032829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":5651,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2541341035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2541341035},{"type":"v1","addr":"172.21.15.139:6835","nonce":2541341035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":7521,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3632520818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3632520818},{"type":"v1","addr":"172.21.15.98:6833","nonce":3632520818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:44:55.910161+0000","modified":"2024-09-19T08:44:57.947782+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7524},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7524":{"gid":7524,"name":"c","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/502938874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":502938874},{"type":"v1","addr":"172.21.15.98:6835","nonce":502938874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7524,"qdb_cluster":[7524]},"id":9}]} 2024-09-19T08:45:15.790 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 68 2024-09-19T08:45:15.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.787+0000 7fa1f8709640 1 -- 172.21.15.98:0/86148270 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa1d4067e90 msgr2=0x7fa1d406a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:15.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.787+0000 7fa1f8709640 1 --2- 172.21.15.98:0/86148270 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa1d4067e90 0x7fa1d406a350 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec005510 tx=0x7fa1ec009040 comp rx=0 tx=0).stop 2024-09-19T08:45:15.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.787+0000 7fa1f8709640 1 -- 172.21.15.98:0/86148270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1f01f2580 msgr2=0x7fa1f00ae370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:15.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.787+0000 7fa1f8709640 1 --2- 172.21.15.98:0/86148270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1f01f2580 0x7fa1f00ae370 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8001d00 tx=0x7fa1dc005840 comp rx=0 tx=0).stop 2024-09-19T08:45:15.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.787+0000 7fa1f8709640 1 -- 172.21.15.98:0/86148270 shutdown_connections 2024-09-19T08:45:15.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.787+0000 7fa1f8709640 1 --2- 172.21.15.98:0/86148270 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa1d4067e90 0x7fa1d406a350 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:15.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.787+0000 7fa1f8709640 1 --2- 172.21.15.98:0/86148270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa1f01f2580 0x7fa1f00ae370 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:15.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.787+0000 7fa1f8709640 1 --2- 172.21.15.98:0/86148270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa1f00b1490 0x7fa1f01f2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:15.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.787+0000 7fa1f8709640 1 --2- 172.21.15.98:0/86148270 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa1f0157fc0 0x7fa1f00b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:15.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.787+0000 7fa1f8709640 1 -- 172.21.15.98:0/86148270 >> 172.21.15.98:0/86148270 conn(0x7fa1f00b2010 msgr2=0x7fa1f0071600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:15.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.787+0000 7fa1f8709640 1 -- 172.21.15.98:0/86148270 shutdown_connections 2024-09-19T08:45:15.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.787+0000 7fa1f8709640 1 -- 172.21.15.98:0/86148270 wait complete. 2024-09-19T08:45:15.807 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:45:15.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.923+0000 7f7c6695e640 1 Processor -- start 2024-09-19T08:45:15.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.923+0000 7f7c6695e640 1 -- start start 2024-09-19T08:45:15.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.923+0000 7f7c6695e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7c600d4820 0x7f7c601621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:15.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.923+0000 7f7c6695e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c600b9970 0x7f7c600b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:15.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.923+0000 7f7c6695e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c60068440 con 0x7f7c600b7f80 2024-09-19T08:45:15.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.923+0000 7f7c6695e640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f7c60163050 con 0x7f7c600d4820 2024-09-19T08:45:15.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.923+0000 7f7c6695e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7c60162a90 con 0x7f7c600b9970 2024-09-19T08:45:15.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.923+0000 7f7c5f7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7c600d4820 0x7f7c601621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:15.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.923+0000 7f7c5f7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7c600d4820 0x7f7c601621b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59642/0 (socket says 172.21.15.98:59642) 2024-09-19T08:45:15.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.923+0000 7f7c5f7fe640 1 -- 172.21.15.98:0/2591785389 learned_addr learned my addr 172.21.15.98:0/2591785389 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:15.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.923+0000 7f7c64efb640 1 --2- 172.21.15.98:0/2591785389 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c600b9970 0x7f7c600b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:15.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.923+0000 7f7c5f7fe640 1 -- 172.21.15.98:0/2591785389 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c600b9970 msgr2=0x7f7c600b9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:15.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.923+0000 7f7c5f7fe640 1 --2- 172.21.15.98:0/2591785389 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c600b9970 0x7f7c600b9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:15.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.923+0000 7f7c5f7fe640 1 -- 172.21.15.98:0/2591785389 >> v1:172.21.15.139:6789/0 conn(0x7f7c600b7f80 legacy=0x7f7c6015f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:15.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.923+0000 7f7c5f7fe640 1 -- 172.21.15.98:0/2591785389 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c60162d70 con 0x7f7c600d4820 2024-09-19T08:45:15.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.923+0000 7f7c64efb640 1 --2- 172.21.15.98:0/2591785389 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c600b9970 0x7f7c600b9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:15.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.927+0000 7f7c5f7fe640 1 --2- 172.21.15.98:0/2591785389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c600d4820 0x7f7c601621b0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f7c600ceb00 tx=0x7f7c5401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a53d67fc3004021 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:15.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.927+0000 7f7c5effd640 1 -- 172.21.15.98:0/2591785389 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c540b1070 con 0x7f7c600d4820 2024-09-19T08:45:15.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.927+0000 7f7c5effd640 1 -- 172.21.15.98:0/2591785389 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c540ac410 con 0x7f7c600d4820 2024-09-19T08:45:15.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.927+0000 7f7c5effd640 1 -- 172.21.15.98:0/2591785389 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c540ac7e0 con 0x7f7c600d4820 2024-09-19T08:45:15.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.927+0000 7f7c6695e640 1 -- 172.21.15.98:0/2591785389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c600d4820 msgr2=0x7f7c601621b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:15.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.927+0000 7f7c6695e640 1 --2- 172.21.15.98:0/2591785389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c600d4820 0x7f7c601621b0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f7c600ceb00 tx=0x7f7c5401c4e0 comp rx=0 tx=0).stop 2024-09-19T08:45:15.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.927+0000 7f7c6695e640 1 -- 172.21.15.98:0/2591785389 shutdown_connections 2024-09-19T08:45:15.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.927+0000 7f7c6695e640 1 --2- 172.21.15.98:0/2591785389 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c600b9970 0x7f7c600b9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:15.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.927+0000 7f7c6695e640 1 --2- 172.21.15.98:0/2591785389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c600d4820 0x7f7c601621b0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:15.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.927+0000 7f7c6695e640 1 -- 172.21.15.98:0/2591785389 >> 172.21.15.98:0/2591785389 conn(0x7f7c60074e40 msgr2=0x7f7c60075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:15.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.927+0000 7f7c6695e640 1 -- 172.21.15.98:0/2591785389 shutdown_connections 2024-09-19T08:45:15.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.927+0000 7f7c6695e640 1 -- 172.21.15.98:0/2591785389 wait complete. 2024-09-19T08:45:15.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.927+0000 7f7c6695e640 1 Processor -- start 2024-09-19T08:45:15.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.927+0000 7f7c6695e640 1 -- start start 2024-09-19T08:45:15.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.931+0000 7f7c6695e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c600b7f80 0x7f7c60152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:15.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.931+0000 7f7c6695e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c600b9970 0x7f7c60140e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:15.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.931+0000 7f7c6695e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c600d4820 0x7f7c60158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:15.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.931+0000 7f7c6695e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7c600b84e0 con 0x7f7c600b9970 2024-09-19T08:45:15.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.931+0000 7f7c6695e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7c600b8360 con 0x7f7c600b7f80 2024-09-19T08:45:15.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.931+0000 7f7c6695e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7c60144590 con 0x7f7c600d4820 2024-09-19T08:45:15.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.931+0000 7f7c5f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c600b9970 0x7f7c60140e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:15.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.931+0000 7f7c5f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c600b9970 0x7f7c60140e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59644/0 (socket says 172.21.15.98:59644) 2024-09-19T08:45:15.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.931+0000 7f7c5f7fe640 1 -- 172.21.15.98:0/3444705478 learned_addr learned my addr 172.21.15.98:0/3444705478 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:15.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.931+0000 7f7c64efb640 1 --2- 172.21.15.98:0/3444705478 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c600d4820 0x7f7c60158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:15.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.931+0000 7f7c5ffff640 1 --2- 172.21.15.98:0/3444705478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c600b7f80 0x7f7c60152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:15.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.931+0000 7f7c5f7fe640 1 -- 172.21.15.98:0/3444705478 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c600d4820 msgr2=0x7f7c60158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:15.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.931+0000 7f7c5f7fe640 1 --2- 172.21.15.98:0/3444705478 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c600d4820 0x7f7c60158210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:15.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.931+0000 7f7c5f7fe640 1 -- 172.21.15.98:0/3444705478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c600b7f80 msgr2=0x7f7c60152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:15.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.931+0000 7f7c5f7fe640 1 --2- 172.21.15.98:0/3444705478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c600b7f80 0x7f7c60152150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:15.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.931+0000 7f7c5f7fe640 1 -- 172.21.15.98:0/3444705478 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c60151390 con 0x7f7c600b9970 2024-09-19T08:45:15.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.931+0000 7f7c64efb640 1 --2- 172.21.15.98:0/3444705478 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c600d4820 0x7f7c60158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:15.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.931+0000 7f7c5f7fe640 1 --2- 172.21.15.98:0/3444705478 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c600b9970 0x7f7c60140e30 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f7c540af040 tx=0x7f7c54019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:15.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.931+0000 7f7c5cff9640 1 -- 172.21.15.98:0/3444705478 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c540b1040 con 0x7f7c600b9970 2024-09-19T08:45:15.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.931+0000 7f7c5cff9640 1 -- 172.21.15.98:0/3444705478 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c540bc070 con 0x7f7c600b9970 2024-09-19T08:45:15.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.931+0000 7f7c6695e640 1 -- 172.21.15.98:0/3444705478 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c60151670 con 0x7f7c600b9970 2024-09-19T08:45:15.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.931+0000 7f7c5cff9640 1 -- 172.21.15.98:0/3444705478 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c540b7680 con 0x7f7c600b9970 2024-09-19T08:45:15.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.931+0000 7f7c6695e640 1 -- 172.21.15.98:0/3444705478 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c60141370 con 0x7f7c600b9970 2024-09-19T08:45:15.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.931+0000 7f7c6695e640 1 -- 172.21.15.98:0/3444705478 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c600b9de0 con 0x7f7c600b9970 2024-09-19T08:45:15.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.931+0000 7f7c5cff9640 1 -- 172.21.15.98:0/3444705478 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f7c540b7820 con 0x7f7c600b9970 2024-09-19T08:45:15.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.931+0000 7f7c5cff9640 1 --2- 172.21.15.98:0/3444705478 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7c4c067e60 0x7f7c4c06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:15.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.931+0000 7f7c5cff9640 1 -- 172.21.15.98:0/3444705478 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5823+0+0 (secure 0 0 0) 0x7f7c54120b40 con 0x7f7c600b9970 2024-09-19T08:45:15.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.931+0000 7f7c5ffff640 1 --2- 172.21.15.98:0/3444705478 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7c4c067e60 0x7f7c4c06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:15.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.935+0000 7f7c5ffff640 1 --2- 172.21.15.98:0/3444705478 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7c4c067e60 0x7f7c4c06a320 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000a750 tx=0x7f7c50008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:15.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:15.935+0000 7f7c5cff9640 1 -- 172.21.15.98:0/3444705478 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f7c540160d0 con 0x7f7c600b9970 2024-09-19T08:45:16.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.087+0000 7f7c6695e640 1 -- 172.21.15.98:0/3444705478 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7c60153380 con 0x7f7c600b9970 2024-09-19T08:45:16.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.091+0000 7f7c5cff9640 1 -- 172.21.15.98:0/3444705478 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) ==== 74+0+16946 (secure 0 0 0) 0x7f7c540f0d20 con 0x7f7c600b9970 2024-09-19T08:45:16.092 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:45:16.093 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":123,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:45:15.233634+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:44:53.861869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-19T08:44:54.909433+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":122,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.5","mappings":[{"from":2,"to":0}]},{"pgid":"19.9","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:45:16.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.091+0000 7f7c6695e640 1 -- 172.21.15.98:0/3444705478 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7c4c067e60 msgr2=0x7f7c4c06a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:16.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.091+0000 7f7c6695e640 1 --2- 172.21.15.98:0/3444705478 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7c4c067e60 0x7f7c4c06a320 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000a750 tx=0x7f7c50008040 comp rx=0 tx=0).stop 2024-09-19T08:45:16.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.091+0000 7f7c6695e640 1 -- 172.21.15.98:0/3444705478 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c600b9970 msgr2=0x7f7c60140e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:16.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.091+0000 7f7c6695e640 1 --2- 172.21.15.98:0/3444705478 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c600b9970 0x7f7c60140e30 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f7c540af040 tx=0x7f7c54019280 comp rx=0 tx=0).stop 2024-09-19T08:45:16.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.091+0000 7f7c6695e640 1 -- 172.21.15.98:0/3444705478 shutdown_connections 2024-09-19T08:45:16.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.091+0000 7f7c6695e640 1 --2- 172.21.15.98:0/3444705478 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7c4c067e60 0x7f7c4c06a320 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.091+0000 7f7c6695e640 1 --2- 172.21.15.98:0/3444705478 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c600d4820 0x7f7c60158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.091+0000 7f7c6695e640 1 --2- 172.21.15.98:0/3444705478 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c600b9970 0x7f7c60140e30 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.091+0000 7f7c6695e640 1 --2- 172.21.15.98:0/3444705478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c600b7f80 0x7f7c60152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.091+0000 7f7c6695e640 1 -- 172.21.15.98:0/3444705478 >> 172.21.15.98:0/3444705478 conn(0x7f7c60074e40 msgr2=0x7f7c600d0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:16.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.091+0000 7f7c6695e640 1 -- 172.21.15.98:0/3444705478 shutdown_connections 2024-09-19T08:45:16.095 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.091+0000 7f7c6695e640 1 -- 172.21.15.98:0/3444705478 wait complete. 2024-09-19T08:45:16.110 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T08:45:16.110 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:45:16.216 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.215+0000 7f3025309640 1 Processor -- start 2024-09-19T08:45:16.216 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.215+0000 7f3025309640 1 -- start start 2024-09-19T08:45:16.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.215+0000 7f3025309640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f302015f590 0x7f30200cf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.215+0000 7f3025309640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f302015fb50 con 0x7f302006c6b0 2024-09-19T08:45:16.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.215+0000 7f3025309640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f302015fdc0 con 0x7f30200cf8b0 2024-09-19T08:45:16.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.215+0000 7f3025309640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f302015f970 con 0x7f302015f590 2024-09-19T08:45:16.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.215+0000 7f301f7fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f302006c6b0 0x7f3020160e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56394/0 (socket says 172.21.15.98:56394) 2024-09-19T08:45:16.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.215+0000 7f301f7fe640 1 -- 172.21.15.98:0/2945026247 learned_addr learned my addr 172.21.15.98:0/2945026247 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:16.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.215+0000 7f301effd640 1 --2- 172.21.15.98:0/2945026247 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f302015f590 0x7f30200cf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:16.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.215+0000 7f301e7fc640 1 -- 172.21.15.98:0/2945026247 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2342108223 0 0) 0x7f302015fb50 con 0x7f302006c6b0 2024-09-19T08:45:16.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.215+0000 7f301e7fc640 1 -- 172.21.15.98:0/2945026247 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f300c003530 con 0x7f302006c6b0 2024-09-19T08:45:16.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.215+0000 7f301e7fc640 1 -- 172.21.15.98:0/2945026247 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 490954505 0 0) 0x7f300c003530 con 0x7f302006c6b0 2024-09-19T08:45:16.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.215+0000 7f301e7fc640 1 -- 172.21.15.98:0/2945026247 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f302015f590 msgr2=0x7f30200cf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:16.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.215+0000 7f301e7fc640 1 --2- 172.21.15.98:0/2945026247 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f302015f590 0x7f30200cf370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.215+0000 7f301e7fc640 1 -- 172.21.15.98:0/2945026247 >> v1:172.21.15.139:6789/0 conn(0x7f30200cf8b0 legacy=0x7f302006bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:16.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.215+0000 7f301e7fc640 1 -- 172.21.15.98:0/2945026247 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30201600a0 con 0x7f302006c6b0 2024-09-19T08:45:16.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.215+0000 7f301e7fc640 1 -- 172.21.15.98:0/2945026247 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f3018001a00 con 0x7f302006c6b0 2024-09-19T08:45:16.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.215+0000 7f301e7fc640 1 -- 172.21.15.98:0/2945026247 >> v1:172.21.15.98:6789/0 conn(0x7f302006c6b0 legacy=0x7f3020160e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:16.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.215+0000 7f301e7fc640 1 --2- 172.21.15.98:0/2945026247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f300c0039f0 0x7f300c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.215+0000 7f301e7fc640 1 --2- 172.21.15.98:0/2945026247 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f302015f590 0x7f300c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.215+0000 7f301e7fc640 1 --2- 172.21.15.98:0/2945026247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f300c004d50 0x7f300c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.215+0000 7f301e7fc640 1 -- 172.21.15.98:0/2945026247 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f302015f970 con 0x7f300c0039f0 2024-09-19T08:45:16.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.215+0000 7f301e7fc640 1 -- 172.21.15.98:0/2945026247 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f300c00ba80 con 0x7f300c004d50 2024-09-19T08:45:16.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.215+0000 7f301e7fc640 1 -- 172.21.15.98:0/2945026247 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f300c00bc80 con 0x7f302015f590 2024-09-19T08:45:16.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.215+0000 7f301effd640 1 --2- 172.21.15.98:0/2945026247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f300c0039f0 0x7f300c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:16.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.215+0000 7f301effd640 1 -- 172.21.15.98:0/2945026247 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f302015f590 msgr2=0x7f300c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:16.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.215+0000 7f301effd640 1 --2- 172.21.15.98:0/2945026247 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f302015f590 0x7f300c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.215+0000 7f301effd640 1 -- 172.21.15.98:0/2945026247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f300c004d50 msgr2=0x7f300c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:16.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.215+0000 7f301effd640 1 --2- 172.21.15.98:0/2945026247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f300c004d50 0x7f300c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.215+0000 7f301effd640 1 -- 172.21.15.98:0/2945026247 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300c003530 con 0x7f300c0039f0 2024-09-19T08:45:16.221 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.219+0000 7f301effd640 1 --2- 172.21.15.98:0/2945026247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f300c0039f0 0x7f300c003e30 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f3010009a90 tx=0x7f301001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd3a566f8d33b9f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:16.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.219+0000 7f301e7fc640 1 -- 172.21.15.98:0/2945026247 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010009690 con 0x7f300c0039f0 2024-09-19T08:45:16.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.219+0000 7f301e7fc640 1 -- 172.21.15.98:0/2945026247 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30100a6450 con 0x7f300c0039f0 2024-09-19T08:45:16.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.219+0000 7f301e7fc640 1 -- 172.21.15.98:0/2945026247 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30100a6820 con 0x7f300c0039f0 2024-09-19T08:45:16.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.219+0000 7f3025309640 1 -- 172.21.15.98:0/2945026247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f300c0039f0 msgr2=0x7f300c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:16.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.219+0000 7f3025309640 1 --2- 172.21.15.98:0/2945026247 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f300c0039f0 0x7f300c003e30 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f3010009a90 tx=0x7f301001ca70 comp rx=0 tx=0).stop 2024-09-19T08:45:16.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.219+0000 7f301f7fe640 1 -- 172.21.15.98:0/2945026247 reap_dead start 2024-09-19T08:45:16.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.219+0000 7f3025309640 1 -- 172.21.15.98:0/2945026247 shutdown_connections 2024-09-19T08:45:16.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.219+0000 7f3025309640 1 -- 172.21.15.98:0/2945026247 >> 172.21.15.98:0/2945026247 conn(0x7f3020074c70 msgr2=0x7f3020075080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:16.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.219+0000 7f3025309640 1 -- 172.21.15.98:0/2945026247 shutdown_connections 2024-09-19T08:45:16.223 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.219+0000 7f3025309640 1 -- 172.21.15.98:0/2945026247 wait complete. 2024-09-19T08:45:16.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.219+0000 7f3025309640 1 Processor -- start 2024-09-19T08:45:16.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.219+0000 7f3025309640 1 -- start start 2024-09-19T08:45:16.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.219+0000 7f3025309640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f300c0039f0 0x7f30201f7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.219+0000 7f3025309640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3020159070 0x7f30201594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.219+0000 7f3025309640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30200b4ea0 0x7f30200b5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.219+0000 7f3025309640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f302015c770 con 0x7f30200b4ea0 2024-09-19T08:45:16.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.219+0000 7f3025309640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f302015c5f0 con 0x7f300c0039f0 2024-09-19T08:45:16.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.219+0000 7f3025309640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f302016a1a0 con 0x7f3020159070 2024-09-19T08:45:16.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.219+0000 7f301f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30200b4ea0 0x7f30200b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:16.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.219+0000 7f301f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30200b4ea0 0x7f30200b5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59664/0 (socket says 172.21.15.98:59664) 2024-09-19T08:45:16.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.219+0000 7f301f7fe640 1 -- 172.21.15.98:0/1414740757 learned_addr learned my addr 172.21.15.98:0/1414740757 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:16.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.219+0000 7f301f7fe640 1 -- 172.21.15.98:0/1414740757 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3020159070 msgr2=0x7f30201594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:16.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.219+0000 7f3017fff640 1 --2- 172.21.15.98:0/1414740757 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3020159070 0x7f30201594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:16.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.219+0000 7f301effd640 1 --2- 172.21.15.98:0/1414740757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f300c0039f0 0x7f30201f7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:16.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.219+0000 7f301f7fe640 1 --2- 172.21.15.98:0/1414740757 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3020159070 0x7f30201594d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.219+0000 7f301f7fe640 1 -- 172.21.15.98:0/1414740757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f300c0039f0 msgr2=0x7f30201f7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:16.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.219+0000 7f301f7fe640 1 --2- 172.21.15.98:0/1414740757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f300c0039f0 0x7f30201f7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.219+0000 7f301f7fe640 1 -- 172.21.15.98:0/1414740757 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30200b5a80 con 0x7f30200b4ea0 2024-09-19T08:45:16.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.223+0000 7f301f7fe640 1 --2- 172.21.15.98:0/1414740757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30200b4ea0 0x7f30200b5320 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f3018001570 tx=0x7f301800cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:16.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.223+0000 7f30177fe640 1 -- 172.21.15.98:0/1414740757 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301800a5b0 con 0x7f30200b4ea0 2024-09-19T08:45:16.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.223+0000 7f30177fe640 1 -- 172.21.15.98:0/1414740757 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f301800a770 con 0x7f30200b4ea0 2024-09-19T08:45:16.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.223+0000 7f3025309640 1 -- 172.21.15.98:0/1414740757 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30200b5d60 con 0x7f30200b4ea0 2024-09-19T08:45:16.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.223+0000 7f30177fe640 1 -- 172.21.15.98:0/1414740757 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301800ab60 con 0x7f30200b4ea0 2024-09-19T08:45:16.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.223+0000 7f3025309640 1 -- 172.21.15.98:0/1414740757 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30200b7930 con 0x7f30200b4ea0 2024-09-19T08:45:16.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.223+0000 7f3025309640 1 -- 172.21.15.98:0/1414740757 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3020160e60 con 0x7f30200b4ea0 2024-09-19T08:45:16.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.223+0000 7f30177fe640 1 -- 172.21.15.98:0/1414740757 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f3018011070 con 0x7f30200b4ea0 2024-09-19T08:45:16.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.223+0000 7f30177fe640 1 --2- 172.21.15.98:0/1414740757 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3000067e90 0x7f300006a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.223+0000 7f30177fe640 1 -- 172.21.15.98:0/1414740757 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5823+0+0 (secure 0 0 0) 0x7f3018085f10 con 0x7f30200b4ea0 2024-09-19T08:45:16.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.223+0000 7f301effd640 1 --2- 172.21.15.98:0/1414740757 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3000067e90 0x7f300006a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:16.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.223+0000 7f301effd640 1 --2- 172.21.15.98:0/1414740757 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3000067e90 0x7f300006a350 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f30100ad000 tx=0x7f30100046d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:16.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.227+0000 7f30177fe640 1 -- 172.21.15.98:0/1414740757 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f301801a060 con 0x7f30200b4ea0 2024-09-19T08:45:16.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.423+0000 7f3025309640 1 -- 172.21.15.98:0/1414740757 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30200b70a0 con 0x7f30200b4ea0 2024-09-19T08:45:16.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.423+0000 7f30177fe640 1 -- 172.21.15.98:0/1414740757 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5228 (secure 0 0 0) 0x7f301800f260 con 0x7f30200b4ea0 2024-09-19T08:45:16.425 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:45:16.425 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":68,"btime":"2024-09-19T08:45:07:966984+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5645,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/1334032829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":1334032829},{"type":"v1","addr":"172.21.15.139:6833","nonce":1334032829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":5651,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2541341035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2541341035},{"type":"v1","addr":"172.21.15.139:6835","nonce":2541341035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":7521,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3632520818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3632520818},{"type":"v1","addr":"172.21.15.98:6833","nonce":3632520818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:44:55.910161+0000","modified":"2024-09-19T08:44:57.947782+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7524},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7524":{"gid":7524,"name":"c","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/502938874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":502938874},{"type":"v1","addr":"172.21.15.98:6835","nonce":502938874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7524,"qdb_cluster":[7524]},"id":9}]} 2024-09-19T08:45:16.426 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 68 2024-09-19T08:45:16.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.423+0000 7f3025309640 1 -- 172.21.15.98:0/1414740757 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3000067e90 msgr2=0x7f300006a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:16.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.423+0000 7f3025309640 1 --2- 172.21.15.98:0/1414740757 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3000067e90 0x7f300006a350 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f30100ad000 tx=0x7f30100046d0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.423+0000 7f3025309640 1 -- 172.21.15.98:0/1414740757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30200b4ea0 msgr2=0x7f30200b5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:16.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.423+0000 7f3025309640 1 --2- 172.21.15.98:0/1414740757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30200b4ea0 0x7f30200b5320 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f3018001570 tx=0x7f301800cf00 comp rx=0 tx=0).stop 2024-09-19T08:45:16.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.423+0000 7f3025309640 1 -- 172.21.15.98:0/1414740757 shutdown_connections 2024-09-19T08:45:16.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.423+0000 7f3025309640 1 --2- 172.21.15.98:0/1414740757 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3000067e90 0x7f300006a350 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.423+0000 7f3025309640 1 --2- 172.21.15.98:0/1414740757 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f30200b4ea0 0x7f30200b5320 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.423+0000 7f3025309640 1 --2- 172.21.15.98:0/1414740757 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3020159070 0x7f30201594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.423+0000 7f3025309640 1 --2- 172.21.15.98:0/1414740757 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f300c0039f0 0x7f30201f7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.423+0000 7f3025309640 1 -- 172.21.15.98:0/1414740757 >> 172.21.15.98:0/1414740757 conn(0x7f3020074c70 msgr2=0x7f302015c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:16.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.423+0000 7f3025309640 1 -- 172.21.15.98:0/1414740757 shutdown_connections 2024-09-19T08:45:16.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.423+0000 7f3025309640 1 -- 172.21.15.98:0/1414740757 wait complete. 2024-09-19T08:45:16.443 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:45:16.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.551+0000 7f1364ce6640 1 Processor -- start 2024-09-19T08:45:16.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.551+0000 7f1364ce6640 1 -- start start 2024-09-19T08:45:16.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.555+0000 7f1364ce6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f13600733b0 0x7f13600cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.555+0000 7f1364ce6640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f136006d310 con 0x7f136006c3a0 2024-09-19T08:45:16.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.555+0000 7f1364ce6640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f136006d5f0 con 0x7f136015c550 2024-09-19T08:45:16.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.555+0000 7f1364ce6640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f136006d8d0 con 0x7f13600733b0 2024-09-19T08:45:16.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.555+0000 7f135dd9b640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f136006c3a0 0x7f13600cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56402/0 (socket says 172.21.15.98:56402) 2024-09-19T08:45:16.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.555+0000 7f135dd9b640 1 -- 172.21.15.98:0/299894268 learned_addr learned my addr 172.21.15.98:0/299894268 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:16.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.555+0000 7f135d59a640 1 -- 172.21.15.98:0/299894268 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3388313383 0 0) 0x7f136006d310 con 0x7f136006c3a0 2024-09-19T08:45:16.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.555+0000 7f135d59a640 1 -- 172.21.15.98:0/299894268 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1348003530 con 0x7f136006c3a0 2024-09-19T08:45:16.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.555+0000 7f135ed9d640 1 --2- 172.21.15.98:0/299894268 >> v2:172.21.15.139:3300/0 conn(0x7f13600733b0 0x7f13600cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:16.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.555+0000 7f135d59a640 1 -- 172.21.15.98:0/299894268 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4132722678 0 0) 0x7f1348003530 con 0x7f136006c3a0 2024-09-19T08:45:16.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.555+0000 7f135d59a640 1 -- 172.21.15.98:0/299894268 >> v2:172.21.15.139:3300/0 conn(0x7f13600733b0 msgr2=0x7f13600cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:16.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.555+0000 7f135d59a640 1 --2- 172.21.15.98:0/299894268 >> v2:172.21.15.139:3300/0 conn(0x7f13600733b0 0x7f13600cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.555+0000 7f135d59a640 1 -- 172.21.15.98:0/299894268 >> v1:172.21.15.139:6789/0 conn(0x7f136015c550 legacy=0x7f136006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:16.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.555+0000 7f135d59a640 1 -- 172.21.15.98:0/299894268 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f136006dbb0 con 0x7f136006c3a0 2024-09-19T08:45:16.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.555+0000 7f135d59a640 1 -- 172.21.15.98:0/299894268 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f1354001680 con 0x7f136006c3a0 2024-09-19T08:45:16.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.555+0000 7f135d59a640 1 -- 172.21.15.98:0/299894268 >> v1:172.21.15.98:6789/0 conn(0x7f136006c3a0 legacy=0x7f13600cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:16.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.555+0000 7f135d59a640 1 --2- 172.21.15.98:0/299894268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13480039c0 0x7f1348003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.555+0000 7f135d59a640 1 --2- 172.21.15.98:0/299894268 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1348004810 0x7f1348004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.555+0000 7f135d59a640 1 --2- 172.21.15.98:0/299894268 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13480051d0 0x7f134800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.555+0000 7f135d59a640 1 -- 172.21.15.98:0/299894268 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f136006d8d0 con 0x7f13480039c0 2024-09-19T08:45:16.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.555+0000 7f135d59a640 1 -- 172.21.15.98:0/299894268 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f134800c120 con 0x7f13480051d0 2024-09-19T08:45:16.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.555+0000 7f135d59a640 1 -- 172.21.15.98:0/299894268 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f134800c320 con 0x7f1348004810 2024-09-19T08:45:16.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.555+0000 7f135e59c640 1 --2- 172.21.15.98:0/299894268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13480039c0 0x7f1348003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:16.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.555+0000 7f135ed9d640 1 --2- 172.21.15.98:0/299894268 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13480051d0 0x7f134800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:16.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.555+0000 7f135e59c640 1 -- 172.21.15.98:0/299894268 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1348004810 msgr2=0x7f1348004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:16.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.555+0000 7f135e59c640 1 --2- 172.21.15.98:0/299894268 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1348004810 0x7f1348004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.555+0000 7f135e59c640 1 -- 172.21.15.98:0/299894268 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13480051d0 msgr2=0x7f134800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:16.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.555+0000 7f135e59c640 1 --2- 172.21.15.98:0/299894268 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13480051d0 0x7f134800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.555+0000 7f135e59c640 1 -- 172.21.15.98:0/299894268 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1348003530 con 0x7f13480039c0 2024-09-19T08:45:16.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.555+0000 7f135ed9d640 1 --2- 172.21.15.98:0/299894268 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13480051d0 0x7f134800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:16.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.555+0000 7f135ed9d640 1 -- 172.21.15.98:0/299894268 reap_dead start 2024-09-19T08:45:16.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.559+0000 7f135e59c640 1 --2- 172.21.15.98:0/299894268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13480039c0 0x7f1348003de0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f1358001e70 tx=0x7f135801c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51b27851f163896 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:16.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.559+0000 7f135d59a640 1 -- 172.21.15.98:0/299894268 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1358001a90 con 0x7f13480039c0 2024-09-19T08:45:16.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.559+0000 7f135d59a640 1 -- 172.21.15.98:0/299894268 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1358003b80 con 0x7f13480039c0 2024-09-19T08:45:16.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.559+0000 7f135d59a640 1 -- 172.21.15.98:0/299894268 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13580050d0 con 0x7f13480039c0 2024-09-19T08:45:16.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.559+0000 7f1364ce6640 1 -- 172.21.15.98:0/299894268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13480039c0 msgr2=0x7f1348003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:16.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.559+0000 7f1364ce6640 1 --2- 172.21.15.98:0/299894268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13480039c0 0x7f1348003de0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f1358001e70 tx=0x7f135801c760 comp rx=0 tx=0).stop 2024-09-19T08:45:16.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.559+0000 7f1364ce6640 1 -- 172.21.15.98:0/299894268 shutdown_connections 2024-09-19T08:45:16.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.559+0000 7f1364ce6640 1 --2- 172.21.15.98:0/299894268 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13480039c0 0x7f1348003de0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.559+0000 7f1364ce6640 1 -- 172.21.15.98:0/299894268 >> 172.21.15.98:0/299894268 conn(0x7f13600b2010 msgr2=0x7f13600b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:16.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.559+0000 7f1364ce6640 1 -- 172.21.15.98:0/299894268 shutdown_connections 2024-09-19T08:45:16.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.559+0000 7f1364ce6640 1 -- 172.21.15.98:0/299894268 wait complete. 2024-09-19T08:45:16.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.559+0000 7f1364ce6640 1 Processor -- start 2024-09-19T08:45:16.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.559+0000 7f1364ce6640 1 -- start start 2024-09-19T08:45:16.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.559+0000 7f1364ce6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13480039c0 0x7f136014cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.559+0000 7f1364ce6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f136014dd20 0x7f1360158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.559+0000 7f1364ce6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1360150cc0 0x7f1360158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.559+0000 7f1364ce6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1360072ae0 con 0x7f136014dd20 2024-09-19T08:45:16.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.559+0000 7f1364ce6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f136015cb40 con 0x7f1360150cc0 2024-09-19T08:45:16.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.559+0000 7f1364ce6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1360072c60 con 0x7f13480039c0 2024-09-19T08:45:16.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.559+0000 7f135dd9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f136014dd20 0x7f1360158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:16.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.559+0000 7f135dd9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f136014dd20 0x7f1360158630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59674/0 (socket says 172.21.15.98:59674) 2024-09-19T08:45:16.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.559+0000 7f135dd9b640 1 -- 172.21.15.98:0/1042330579 learned_addr learned my addr 172.21.15.98:0/1042330579 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:16.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.559+0000 7f135e59c640 1 --2- 172.21.15.98:0/1042330579 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13480039c0 0x7f136014cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:16.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.559+0000 7f135dd9b640 1 -- 172.21.15.98:0/1042330579 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13480039c0 msgr2=0x7f136014cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:16.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.559+0000 7f135dd9b640 1 --2- 172.21.15.98:0/1042330579 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13480039c0 0x7f136014cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.559+0000 7f135dd9b640 1 -- 172.21.15.98:0/1042330579 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1360150cc0 msgr2=0x7f1360158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:16.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.559+0000 7f135dd9b640 1 --2- 172.21.15.98:0/1042330579 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1360150cc0 0x7f1360158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.559+0000 7f135dd9b640 1 -- 172.21.15.98:0/1042330579 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f136014e5a0 con 0x7f136014dd20 2024-09-19T08:45:16.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.559+0000 7f135dd9b640 1 --2- 172.21.15.98:0/1042330579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f136014dd20 0x7f1360158630 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f135400c730 tx=0x7f135400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:16.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.559+0000 7f134effd640 1 -- 172.21.15.98:0/1042330579 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1354018070 con 0x7f136014dd20 2024-09-19T08:45:16.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.559+0000 7f134effd640 1 -- 172.21.15.98:0/1042330579 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13540072f0 con 0x7f136014dd20 2024-09-19T08:45:16.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.559+0000 7f1364ce6640 1 -- 172.21.15.98:0/1042330579 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f136014e880 con 0x7f136014dd20 2024-09-19T08:45:16.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.559+0000 7f134effd640 1 -- 172.21.15.98:0/1042330579 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13540047f0 con 0x7f136014dd20 2024-09-19T08:45:16.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.559+0000 7f1364ce6640 1 -- 172.21.15.98:0/1042330579 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f136015ccc0 con 0x7f136014dd20 2024-09-19T08:45:16.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.563+0000 7f1364ce6640 1 -- 172.21.15.98:0/1042330579 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1360072ff0 con 0x7f136014dd20 2024-09-19T08:45:16.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.563+0000 7f134effd640 1 -- 172.21.15.98:0/1042330579 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f135401b070 con 0x7f136014dd20 2024-09-19T08:45:16.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.563+0000 7f134effd640 1 --2- 172.21.15.98:0/1042330579 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1340067e90 0x7f134006a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.563+0000 7f134effd640 1 -- 172.21.15.98:0/1042330579 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5823+0+0 (secure 0 0 0) 0x7f1354085df0 con 0x7f136014dd20 2024-09-19T08:45:16.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.563+0000 7f135e59c640 1 --2- 172.21.15.98:0/1042330579 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1340067e90 0x7f134006a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:16.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.563+0000 7f135e59c640 1 --2- 172.21.15.98:0/1042330579 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1340067e90 0x7f134006a350 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f13580ad000 tx=0x7f1358009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:16.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.567+0000 7f134effd640 1 -- 172.21.15.98:0/1042330579 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f13540560e0 con 0x7f136014dd20 2024-09-19T08:45:16.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.747+0000 7f1364ce6640 1 -- 172.21.15.98:0/1042330579 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f136015b0e0 con 0x7f136014dd20 2024-09-19T08:45:16.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.747+0000 7f134effd640 1 -- 172.21.15.98:0/1042330579 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5228 (secure 0 0 0) 0x7f13540559e0 con 0x7f136014dd20 2024-09-19T08:45:16.751 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:45:16.751 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":68,"btime":"2024-09-19T08:45:07:966984+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5645,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/1334032829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":1334032829},{"type":"v1","addr":"172.21.15.139:6833","nonce":1334032829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":5651,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2541341035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2541341035},{"type":"v1","addr":"172.21.15.139:6835","nonce":2541341035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":7521,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3632520818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3632520818},{"type":"v1","addr":"172.21.15.98:6833","nonce":3632520818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:44:55.910161+0000","modified":"2024-09-19T08:44:57.947782+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7524},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7524":{"gid":7524,"name":"c","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/502938874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":502938874},{"type":"v1","addr":"172.21.15.98:6835","nonce":502938874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7524,"qdb_cluster":[7524]},"id":9}]} 2024-09-19T08:45:16.751 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 68 2024-09-19T08:45:16.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.751+0000 7f1364ce6640 1 -- 172.21.15.98:0/1042330579 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1340067e90 msgr2=0x7f134006a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:16.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.751+0000 7f1364ce6640 1 --2- 172.21.15.98:0/1042330579 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1340067e90 0x7f134006a350 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f13580ad000 tx=0x7f1358009040 comp rx=0 tx=0).stop 2024-09-19T08:45:16.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.751+0000 7f1364ce6640 1 -- 172.21.15.98:0/1042330579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f136014dd20 msgr2=0x7f1360158630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:16.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.751+0000 7f1364ce6640 1 --2- 172.21.15.98:0/1042330579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f136014dd20 0x7f1360158630 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f135400c730 tx=0x7f135400cbf0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.751+0000 7f1364ce6640 1 -- 172.21.15.98:0/1042330579 shutdown_connections 2024-09-19T08:45:16.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.751+0000 7f1364ce6640 1 --2- 172.21.15.98:0/1042330579 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1340067e90 0x7f134006a350 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.751+0000 7f1364ce6640 1 --2- 172.21.15.98:0/1042330579 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1360150cc0 0x7f1360158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.751+0000 7f1364ce6640 1 --2- 172.21.15.98:0/1042330579 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f136014dd20 0x7f1360158630 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.751+0000 7f1364ce6640 1 --2- 172.21.15.98:0/1042330579 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13480039c0 0x7f136014cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.751+0000 7f1364ce6640 1 -- 172.21.15.98:0/1042330579 >> 172.21.15.98:0/1042330579 conn(0x7f13600b2010 msgr2=0x7f136015a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:16.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.751+0000 7f1364ce6640 1 -- 172.21.15.98:0/1042330579 shutdown_connections 2024-09-19T08:45:16.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.751+0000 7f1364ce6640 1 -- 172.21.15.98:0/1042330579 wait complete. 2024-09-19T08:45:16.769 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:45:16.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.883+0000 7f1b32463640 1 Processor -- start 2024-09-19T08:45:16.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.883+0000 7f1b32463640 1 -- start start 2024-09-19T08:45:16.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.883+0000 7f1b32463640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f1b2c06bc90 0x7f1b2c06c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.883+0000 7f1b32463640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1b2c0ce840 0x7f1b2c0cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.883+0000 7f1b32463640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b2c0ccc60 con 0x7f1b2c158fd0 2024-09-19T08:45:16.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.883+0000 7f1b32463640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f1b2c1593b0 con 0x7f1b2c06bc90 2024-09-19T08:45:16.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.883+0000 7f1b32463640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f1b2c0ccf40 con 0x7f1b2c0ce840 2024-09-19T08:45:16.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.883+0000 7f1b2b7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f1b2c06bc90 0x7f1b2c06c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:16.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.883+0000 7f1b2b7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f1b2c06bc90 0x7f1b2c06c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59680/0 (socket says 172.21.15.98:59680) 2024-09-19T08:45:16.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.883+0000 7f1b2b7fe640 1 -- 172.21.15.98:0/1525702147 learned_addr learned my addr 172.21.15.98:0/1525702147 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:16.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.883+0000 7f1b30a00640 1 --2- 172.21.15.98:0/1525702147 >> v2:172.21.15.139:3300/0 conn(0x7f1b2c0ce840 0x7f1b2c0cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:16.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.883+0000 7f1b2b7fe640 1 -- 172.21.15.98:0/1525702147 >> v2:172.21.15.139:3300/0 conn(0x7f1b2c0ce840 msgr2=0x7f1b2c0cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:16.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.883+0000 7f1b2b7fe640 1 --2- 172.21.15.98:0/1525702147 >> v2:172.21.15.139:3300/0 conn(0x7f1b2c0ce840 0x7f1b2c0cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.883+0000 7f1b2b7fe640 1 -- 172.21.15.98:0/1525702147 >> v1:172.21.15.139:6789/0 conn(0x7f1b2c158fd0 legacy=0x7f1b2c0b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:16.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.883+0000 7f1b2b7fe640 1 -- 172.21.15.98:0/1525702147 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b2c0cd220 con 0x7f1b2c06bc90 2024-09-19T08:45:16.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.883+0000 7f1b2b7fe640 1 --2- 172.21.15.98:0/1525702147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b2c06bc90 0x7f1b2c06c120 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c0afe30 tx=0x7f1b2001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed421adf68f2e2a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:16.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.887+0000 7f1b2affd640 1 -- 172.21.15.98:0/1525702147 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b200b0070 con 0x7f1b2c06bc90 2024-09-19T08:45:16.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.887+0000 7f1b2affd640 1 -- 172.21.15.98:0/1525702147 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b200ab600 con 0x7f1b2c06bc90 2024-09-19T08:45:16.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.887+0000 7f1b2affd640 1 -- 172.21.15.98:0/1525702147 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b200ab9d0 con 0x7f1b2c06bc90 2024-09-19T08:45:16.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.887+0000 7f1b32463640 1 -- 172.21.15.98:0/1525702147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b2c06bc90 msgr2=0x7f1b2c06c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:16.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.887+0000 7f1b32463640 1 --2- 172.21.15.98:0/1525702147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b2c06bc90 0x7f1b2c06c120 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c0afe30 tx=0x7f1b2001c450 comp rx=0 tx=0).stop 2024-09-19T08:45:16.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.887+0000 7f1b32463640 1 -- 172.21.15.98:0/1525702147 shutdown_connections 2024-09-19T08:45:16.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.887+0000 7f1b32463640 1 --2- 172.21.15.98:0/1525702147 >> v2:172.21.15.139:3300/0 conn(0x7f1b2c0ce840 0x7f1b2c0cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.887+0000 7f1b32463640 1 --2- 172.21.15.98:0/1525702147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b2c06bc90 0x7f1b2c06c120 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.887+0000 7f1b32463640 1 -- 172.21.15.98:0/1525702147 >> 172.21.15.98:0/1525702147 conn(0x7f1b2c0b2010 msgr2=0x7f1b2c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:16.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.887+0000 7f1b32463640 1 -- 172.21.15.98:0/1525702147 shutdown_connections 2024-09-19T08:45:16.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.887+0000 7f1b32463640 1 -- 172.21.15.98:0/1525702147 wait complete. 2024-09-19T08:45:16.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.887+0000 7f1b32463640 1 Processor -- start 2024-09-19T08:45:16.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.887+0000 7f1b32463640 1 -- start start 2024-09-19T08:45:16.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.887+0000 7f1b32463640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b2c06bc90 0x7f1b2c1543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.887+0000 7f1b32463640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b2c0ce840 0x7f1b2c154b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.887+0000 7f1b32463640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b2c158fd0 0x7f1b2c1550c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.887+0000 7f1b32463640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2c15c920 con 0x7f1b2c0ce840 2024-09-19T08:45:16.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.887+0000 7f1b32463640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2c15c7a0 con 0x7f1b2c06bc90 2024-09-19T08:45:16.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.887+0000 7f1b32463640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1b2c168110 con 0x7f1b2c158fd0 2024-09-19T08:45:16.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.887+0000 7f1b2b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b2c0ce840 0x7f1b2c154b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:16.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.887+0000 7f1b2b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b2c0ce840 0x7f1b2c154b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59690/0 (socket says 172.21.15.98:59690) 2024-09-19T08:45:16.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.887+0000 7f1b2b7fe640 1 -- 172.21.15.98:0/3965067216 learned_addr learned my addr 172.21.15.98:0/3965067216 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:16.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.887+0000 7f1b2b7fe640 1 -- 172.21.15.98:0/3965067216 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b2c158fd0 msgr2=0x7f1b2c1550c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:16.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.887+0000 7f1b2b7fe640 1 --2- 172.21.15.98:0/3965067216 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b2c158fd0 0x7f1b2c1550c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.887+0000 7f1b2b7fe640 1 -- 172.21.15.98:0/3965067216 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b2c06bc90 msgr2=0x7f1b2c1543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:16.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.887+0000 7f1b2b7fe640 1 --2- 172.21.15.98:0/3965067216 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b2c06bc90 0x7f1b2c1543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:16.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.887+0000 7f1b2b7fe640 1 -- 172.21.15.98:0/3965067216 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b2c15f780 con 0x7f1b2c0ce840 2024-09-19T08:45:16.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.887+0000 7f1b2b7fe640 1 --2- 172.21.15.98:0/3965067216 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b2c0ce840 0x7f1b2c154b80 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f1b2001a080 tx=0x7f1b200191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:16.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.887+0000 7f1b28ff9640 1 -- 172.21.15.98:0/3965067216 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b200b9070 con 0x7f1b2c0ce840 2024-09-19T08:45:16.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.887+0000 7f1b28ff9640 1 -- 172.21.15.98:0/3965067216 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b200b4420 con 0x7f1b2c0ce840 2024-09-19T08:45:16.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.887+0000 7f1b32463640 1 -- 172.21.15.98:0/3965067216 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b2c15fb60 con 0x7f1b2c0ce840 2024-09-19T08:45:16.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.887+0000 7f1b28ff9640 1 -- 172.21.15.98:0/3965067216 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b200b0040 con 0x7f1b2c0ce840 2024-09-19T08:45:16.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.887+0000 7f1b32463640 1 -- 172.21.15.98:0/3965067216 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b2c160b80 con 0x7f1b2c0ce840 2024-09-19T08:45:16.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.891+0000 7f1b32463640 1 -- 172.21.15.98:0/3965067216 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b2c06c120 con 0x7f1b2c0ce840 2024-09-19T08:45:16.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.891+0000 7f1b28ff9640 1 -- 172.21.15.98:0/3965067216 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f1b200bc070 con 0x7f1b2c0ce840 2024-09-19T08:45:16.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.891+0000 7f1b28ff9640 1 --2- 172.21.15.98:0/3965067216 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1b18067e60 0x7f1b1806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:16.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.891+0000 7f1b28ff9640 1 -- 172.21.15.98:0/3965067216 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5823+0+0 (secure 0 0 0) 0x7f1b2011fc30 con 0x7f1b2c0ce840 2024-09-19T08:45:16.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.891+0000 7f1b2bfff640 1 --2- 172.21.15.98:0/3965067216 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1b18067e60 0x7f1b1806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:16.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.891+0000 7f1b2bfff640 1 --2- 172.21.15.98:0/3965067216 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1b18067e60 0x7f1b1806a320 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c06c990 tx=0x7f1b1c009230 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:16.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:16.895+0000 7f1b28ff9640 1 -- 172.21.15.98:0/3965067216 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f1b200c3080 con 0x7f1b2c0ce840 2024-09-19T08:45:17.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.043+0000 7f1b32463640 1 -- 172.21.15.98:0/3965067216 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b2c158630 con 0x7f1b2c0ce840 2024-09-19T08:45:17.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.047+0000 7f1b28ff9640 1 -- 172.21.15.98:0/3965067216 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) ==== 74+0+16946 (secure 0 0 0) 0x7f1b200160d0 con 0x7f1b2c0ce840 2024-09-19T08:45:17.050 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:45:17.050 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":123,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:45:15.233634+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:44:53.861869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-19T08:44:54.909433+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":122,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.5","mappings":[{"from":2,"to":0}]},{"pgid":"19.9","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:45:17.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.051+0000 7f1b32463640 1 -- 172.21.15.98:0/3965067216 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1b18067e60 msgr2=0x7f1b1806a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:17.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.051+0000 7f1b32463640 1 --2- 172.21.15.98:0/3965067216 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1b18067e60 0x7f1b1806a320 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c06c990 tx=0x7f1b1c009230 comp rx=0 tx=0).stop 2024-09-19T08:45:17.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.051+0000 7f1b32463640 1 -- 172.21.15.98:0/3965067216 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b2c0ce840 msgr2=0x7f1b2c154b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:17.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.051+0000 7f1b32463640 1 --2- 172.21.15.98:0/3965067216 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b2c0ce840 0x7f1b2c154b80 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f1b2001a080 tx=0x7f1b200191f0 comp rx=0 tx=0).stop 2024-09-19T08:45:17.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.051+0000 7f1b32463640 1 -- 172.21.15.98:0/3965067216 shutdown_connections 2024-09-19T08:45:17.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.051+0000 7f1b32463640 1 --2- 172.21.15.98:0/3965067216 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1b18067e60 0x7f1b1806a320 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:17.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.051+0000 7f1b32463640 1 --2- 172.21.15.98:0/3965067216 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1b2c158fd0 0x7f1b2c1550c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:17.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.051+0000 7f1b32463640 1 --2- 172.21.15.98:0/3965067216 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1b2c0ce840 0x7f1b2c154b80 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:17.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.051+0000 7f1b32463640 1 --2- 172.21.15.98:0/3965067216 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1b2c06bc90 0x7f1b2c1543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:17.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.051+0000 7f1b32463640 1 -- 172.21.15.98:0/3965067216 >> 172.21.15.98:0/3965067216 conn(0x7f1b2c0b2010 msgr2=0x7f1b2c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:17.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.051+0000 7f1b32463640 1 -- 172.21.15.98:0/3965067216 shutdown_connections 2024-09-19T08:45:17.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.051+0000 7f1b32463640 1 -- 172.21.15.98:0/3965067216 wait complete. 2024-09-19T08:45:17.068 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-19T08:45:17.177 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.175+0000 7ff1e1944640 1 Processor -- start 2024-09-19T08:45:17.177 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.175+0000 7ff1e1944640 1 -- start start 2024-09-19T08:45:17.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.175+0000 7ff1e1944640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff1dc0b7f80 0x7ff1dc15f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:17.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.175+0000 7ff1e1944640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1dc0d4820 0x7ff1dc1621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:17.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.175+0000 7ff1e1944640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1dc068440 con 0x7ff1dc0b9970 2024-09-19T08:45:17.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.175+0000 7ff1e1944640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7ff1dc163050 con 0x7ff1dc0b7f80 2024-09-19T08:45:17.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.175+0000 7ff1e1944640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff1dc162a90 con 0x7ff1dc0d4820 2024-09-19T08:45:17.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.175+0000 7ff1daffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff1dc0b7f80 0x7ff1dc15f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:17.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.175+0000 7ff1db7fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7ff1dc0b9970 0x7ff1dc0b9e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:56418/0 (socket says 172.21.15.98:56418) 2024-09-19T08:45:17.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.175+0000 7ff1db7fe640 1 -- 172.21.15.98:0/4179422499 learned_addr learned my addr 172.21.15.98:0/4179422499 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:17.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.175+0000 7ff1da7fc640 1 --2- 172.21.15.98:0/4179422499 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1dc0d4820 0x7ff1dc1621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:17.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.175+0000 7ff1daffd640 1 -- 172.21.15.98:0/4179422499 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1dc0d4820 msgr2=0x7ff1dc1621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:17.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.175+0000 7ff1daffd640 1 --2- 172.21.15.98:0/4179422499 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1dc0d4820 0x7ff1dc1621b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:17.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.175+0000 7ff1daffd640 1 -- 172.21.15.98:0/4179422499 >> v1:172.21.15.98:6789/0 conn(0x7ff1dc0b9970 legacy=0x7ff1dc0b9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:17.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.175+0000 7ff1daffd640 1 -- 172.21.15.98:0/4179422499 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1dc162d70 con 0x7ff1dc0b7f80 2024-09-19T08:45:17.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.175+0000 7ff1d9ffb640 1 -- 172.21.15.98:0/4179422499 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4257129193 0 0) 0x7ff1dc068440 con 0x7ff1dc0b9970 2024-09-19T08:45:17.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.179+0000 7ff1daffd640 1 --2- 172.21.15.98:0/4179422499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1dc0b7f80 0x7ff1dc15f560 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00097c0 tx=0x7ff1d001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=135aa09d78635b88 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:17.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.179+0000 7ff1d9ffb640 1 -- 172.21.15.98:0/4179422499 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d00b0070 con 0x7ff1dc0b7f80 2024-09-19T08:45:17.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.179+0000 7ff1d9ffb640 1 -- 172.21.15.98:0/4179422499 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1d00ab600 con 0x7ff1dc0b7f80 2024-09-19T08:45:17.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.179+0000 7ff1d9ffb640 1 -- 172.21.15.98:0/4179422499 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d00ab9d0 con 0x7ff1dc0b7f80 2024-09-19T08:45:17.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.179+0000 7ff1e1944640 1 -- 172.21.15.98:0/4179422499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1dc0b7f80 msgr2=0x7ff1dc15f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:17.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.179+0000 7ff1e1944640 1 --2- 172.21.15.98:0/4179422499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1dc0b7f80 0x7ff1dc15f560 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00097c0 tx=0x7ff1d001c4a0 comp rx=0 tx=0).stop 2024-09-19T08:45:17.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.179+0000 7ff1e1944640 1 -- 172.21.15.98:0/4179422499 shutdown_connections 2024-09-19T08:45:17.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.179+0000 7ff1e1944640 1 --2- 172.21.15.98:0/4179422499 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1dc0d4820 0x7ff1dc1621b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:17.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.179+0000 7ff1e1944640 1 --2- 172.21.15.98:0/4179422499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1dc0b7f80 0x7ff1dc15f560 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:17.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.179+0000 7ff1e1944640 1 -- 172.21.15.98:0/4179422499 >> 172.21.15.98:0/4179422499 conn(0x7ff1dc074e40 msgr2=0x7ff1dc075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:17.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.179+0000 7ff1e1944640 1 -- 172.21.15.98:0/4179422499 shutdown_connections 2024-09-19T08:45:17.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.179+0000 7ff1e1944640 1 -- 172.21.15.98:0/4179422499 wait complete. 2024-09-19T08:45:17.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.179+0000 7ff1e1944640 1 Processor -- start 2024-09-19T08:45:17.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.179+0000 7ff1e1944640 1 -- start start 2024-09-19T08:45:17.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.179+0000 7ff1e1944640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1dc0b7f80 0x7ff1dc15c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:17.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.179+0000 7ff1e1944640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1dc0b9970 0x7ff1dc0d3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:17.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.179+0000 7ff1e1944640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1dc0d4820 0x7ff1dc1fa710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:17.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.179+0000 7ff1e1944640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc0b84e0 con 0x7ff1dc0b9970 2024-09-19T08:45:17.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.179+0000 7ff1e1944640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc0b8360 con 0x7ff1dc0b7f80 2024-09-19T08:45:17.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.179+0000 7ff1e1944640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff1dc144590 con 0x7ff1dc0d4820 2024-09-19T08:45:17.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.179+0000 7ff1da7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1dc0b9970 0x7ff1dc0d3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:17.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.179+0000 7ff1da7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1dc0b9970 0x7ff1dc0d3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59706/0 (socket says 172.21.15.98:59706) 2024-09-19T08:45:17.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.179+0000 7ff1da7fc640 1 -- 172.21.15.98:0/3986547384 learned_addr learned my addr 172.21.15.98:0/3986547384 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:17.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.183+0000 7ff1daffd640 1 --2- 172.21.15.98:0/3986547384 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1dc0b7f80 0x7ff1dc15c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:17.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.183+0000 7ff1db7fe640 1 --2- 172.21.15.98:0/3986547384 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1dc0d4820 0x7ff1dc1fa710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:17.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.183+0000 7ff1da7fc640 1 -- 172.21.15.98:0/3986547384 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1dc0d4820 msgr2=0x7ff1dc1fa710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:17.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.183+0000 7ff1da7fc640 1 --2- 172.21.15.98:0/3986547384 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1dc0d4820 0x7ff1dc1fa710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:17.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.183+0000 7ff1da7fc640 1 -- 172.21.15.98:0/3986547384 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1dc0b7f80 msgr2=0x7ff1dc15c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:17.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.183+0000 7ff1da7fc640 1 --2- 172.21.15.98:0/3986547384 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1dc0b7f80 0x7ff1dc15c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:17.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.183+0000 7ff1da7fc640 1 -- 172.21.15.98:0/3986547384 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1dc1557e0 con 0x7ff1dc0b9970 2024-09-19T08:45:17.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.183+0000 7ff1da7fc640 1 --2- 172.21.15.98:0/3986547384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1dc0b9970 0x7ff1dc0d3590 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00d7d0 tx=0x7ff1cc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:17.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.183+0000 7ff1c3fff640 1 -- 172.21.15.98:0/3986547384 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc004020 con 0x7ff1dc0b9970 2024-09-19T08:45:17.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.183+0000 7ff1c3fff640 1 -- 172.21.15.98:0/3986547384 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1cc0041c0 con 0x7ff1dc0b9970 2024-09-19T08:45:17.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.183+0000 7ff1c3fff640 1 -- 172.21.15.98:0/3986547384 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc0043b0 con 0x7ff1dc0b9970 2024-09-19T08:45:17.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.183+0000 7ff1e1944640 1 -- 172.21.15.98:0/3986547384 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1dc155ac0 con 0x7ff1dc0b9970 2024-09-19T08:45:17.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.183+0000 7ff1e1944640 1 -- 172.21.15.98:0/3986547384 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1dc156080 con 0x7ff1dc0b9970 2024-09-19T08:45:17.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.183+0000 7ff1e1944640 1 -- 172.21.15.98:0/3986547384 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1dc0ce8e0 con 0x7ff1dc0b9970 2024-09-19T08:45:17.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.183+0000 7ff1c3fff640 1 -- 172.21.15.98:0/3986547384 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7ff1cc01b020 con 0x7ff1dc0b9970 2024-09-19T08:45:17.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.183+0000 7ff1c3fff640 1 --2- 172.21.15.98:0/3986547384 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff1c8067e60 0x7ff1c806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:17.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.183+0000 7ff1c3fff640 1 -- 172.21.15.98:0/3986547384 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5823+0+0 (secure 0 0 0) 0x7ff1cc0863e0 con 0x7ff1dc0b9970 2024-09-19T08:45:17.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.183+0000 7ff1daffd640 1 --2- 172.21.15.98:0/3986547384 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff1c8067e60 0x7ff1c806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:17.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.183+0000 7ff1daffd640 1 --2- 172.21.15.98:0/3986547384 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff1c8067e60 0x7ff1c806a320 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00ad000 tx=0x7ff1d00a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:17.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.187+0000 7ff1c3fff640 1 -- 172.21.15.98:0/3986547384 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff1cc056830 con 0x7ff1dc0b9970 2024-09-19T08:45:17.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.383+0000 7ff1e1944640 1 -- 172.21.15.98:0/3986547384 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff1dc15f540 con 0x7ff1dc0b9970 2024-09-19T08:45:17.386 INFO:tasks.ceph.mon.a.smithi098.stderr:2024-09-19T08:45:17.383+0000 7f5e49017640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T08:45:17.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.387+0000 7ff1c3fff640 1 -- 172.21.15.98:0/3986547384 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v69) ==== 161+0+0 (secure 0 0 0) 0x7ff1cc056130 con 0x7ff1dc0b9970 2024-09-19T08:45:17.389 INFO:teuthology.orchestra.run.smithi098.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T08:45:17.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.387+0000 7ff1e1944640 1 -- 172.21.15.98:0/3986547384 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff1c8067e60 msgr2=0x7ff1c806a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:17.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.387+0000 7ff1e1944640 1 --2- 172.21.15.98:0/3986547384 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff1c8067e60 0x7ff1c806a320 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00ad000 tx=0x7ff1d00a6000 comp rx=0 tx=0).stop 2024-09-19T08:45:17.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.387+0000 7ff1e1944640 1 -- 172.21.15.98:0/3986547384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1dc0b9970 msgr2=0x7ff1dc0d3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:17.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.387+0000 7ff1e1944640 1 --2- 172.21.15.98:0/3986547384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1dc0b9970 0x7ff1dc0d3590 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00d7d0 tx=0x7ff1cc00dc90 comp rx=0 tx=0).stop 2024-09-19T08:45:17.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.387+0000 7ff1e1944640 1 -- 172.21.15.98:0/3986547384 shutdown_connections 2024-09-19T08:45:17.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.387+0000 7ff1e1944640 1 --2- 172.21.15.98:0/3986547384 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff1c8067e60 0x7ff1c806a320 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:17.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.387+0000 7ff1e1944640 1 --2- 172.21.15.98:0/3986547384 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1dc0d4820 0x7ff1dc1fa710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:17.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.387+0000 7ff1e1944640 1 --2- 172.21.15.98:0/3986547384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1dc0b9970 0x7ff1dc0d3590 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:17.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.387+0000 7ff1e1944640 1 --2- 172.21.15.98:0/3986547384 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1dc0b7f80 0x7ff1dc15c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:17.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.387+0000 7ff1e1944640 1 -- 172.21.15.98:0/3986547384 >> 172.21.15.98:0/3986547384 conn(0x7ff1dc074e40 msgr2=0x7ff1dc158280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:17.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.387+0000 7ff1e1944640 1 -- 172.21.15.98:0/3986547384 shutdown_connections 2024-09-19T08:45:17.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.387+0000 7ff1e1944640 1 -- 172.21.15.98:0/3986547384 wait complete. 2024-09-19T08:45:17.409 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T08:45:17.441 INFO:tasks.ceph.mds.c.smithi098.stdout:starting mds.c at 2024-09-19T08:45:17.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.515+0000 7fc0df9d6640 1 Processor -- start 2024-09-19T08:45:17.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.515+0000 7fc0df9d6640 1 -- start start 2024-09-19T08:45:17.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.515+0000 7fc0df9d6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc0d8072f90 0x7fc0d80ce400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:17.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.515+0000 7fc0df9d6640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0d815c820 con 0x7fc0d815c440 2024-09-19T08:45:17.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.515+0000 7fc0df9d6640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0d8070ec0 con 0x7fc0d80ce940 2024-09-19T08:45:17.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.515+0000 7fc0df9d6640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fc0d80711a0 con 0x7fc0d8072f90 2024-09-19T08:45:17.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.515+0000 7fc0dd772640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fc0d815c440 0x7fc0d80706e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:35810/0 (socket says 172.21.15.98:35810) 2024-09-19T08:45:17.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.515+0000 7fc0dd772640 1 -- 172.21.15.98:0/1253753095 learned_addr learned my addr 172.21.15.98:0/1253753095 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:17.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.515+0000 7fc0dcf71640 1 --2- 172.21.15.98:0/1253753095 >> v2:172.21.15.139:3300/0 conn(0x7fc0d8072f90 0x7fc0d80ce400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:17.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.515+0000 7fc0cffff640 1 -- 172.21.15.98:0/1253753095 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4268641617 0 0) 0x7fc0d815c820 con 0x7fc0d815c440 2024-09-19T08:45:17.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.515+0000 7fc0cffff640 1 -- 172.21.15.98:0/1253753095 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc0c4003530 con 0x7fc0d815c440 2024-09-19T08:45:17.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.515+0000 7fc0cffff640 1 -- 172.21.15.98:0/1253753095 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1299474620 0 0) 0x7fc0c4003530 con 0x7fc0d815c440 2024-09-19T08:45:17.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.515+0000 7fc0cffff640 1 -- 172.21.15.98:0/1253753095 >> v2:172.21.15.139:3300/0 conn(0x7fc0d8072f90 msgr2=0x7fc0d80ce400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:17.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.515+0000 7fc0cffff640 1 --2- 172.21.15.98:0/1253753095 >> v2:172.21.15.139:3300/0 conn(0x7fc0d8072f90 0x7fc0d80ce400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:17.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.515+0000 7fc0cffff640 1 -- 172.21.15.98:0/1253753095 >> v1:172.21.15.139:6789/0 conn(0x7fc0d80ce940 legacy=0x7fc0d80b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:17.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.515+0000 7fc0cffff640 1 -- 172.21.15.98:0/1253753095 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d8071480 con 0x7fc0d815c440 2024-09-19T08:45:17.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.515+0000 7fc0cffff640 1 -- 172.21.15.98:0/1253753095 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fc0d4001680 con 0x7fc0d815c440 2024-09-19T08:45:17.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.515+0000 7fc0cffff640 1 -- 172.21.15.98:0/1253753095 >> v1:172.21.15.98:6789/0 conn(0x7fc0d815c440 legacy=0x7fc0d80706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:17.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.515+0000 7fc0cffff640 1 --2- 172.21.15.98:0/1253753095 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0c40039c0 0x7fc0c4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:17.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.515+0000 7fc0cffff640 1 --2- 172.21.15.98:0/1253753095 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0c4004810 0x7fc0c4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:17.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.515+0000 7fc0cffff640 1 --2- 172.21.15.98:0/1253753095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0c40051d0 0x7fc0c400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:17.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.515+0000 7fc0cffff640 1 -- 172.21.15.98:0/1253753095 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d80711a0 con 0x7fc0c40051d0 2024-09-19T08:45:17.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.515+0000 7fc0cffff640 1 -- 172.21.15.98:0/1253753095 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c400c070 con 0x7fc0c4004810 2024-09-19T08:45:17.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.515+0000 7fc0cffff640 1 -- 172.21.15.98:0/1253753095 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc0c400c270 con 0x7fc0c40039c0 2024-09-19T08:45:17.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.515+0000 7fc0ddf73640 1 --2- 172.21.15.98:0/1253753095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0c40051d0 0x7fc0c400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:17.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.515+0000 7fc0dcf71640 1 --2- 172.21.15.98:0/1253753095 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0c4004810 0x7fc0c4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:17.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.515+0000 7fc0dd772640 1 --2- 172.21.15.98:0/1253753095 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0c40039c0 0x7fc0c4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:17.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.515+0000 7fc0ddf73640 1 -- 172.21.15.98:0/1253753095 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0c40039c0 msgr2=0x7fc0c4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:17.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.515+0000 7fc0ddf73640 1 --2- 172.21.15.98:0/1253753095 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0c40039c0 0x7fc0c4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:17.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.515+0000 7fc0ddf73640 1 -- 172.21.15.98:0/1253753095 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0c4004810 msgr2=0x7fc0c4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:17.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.515+0000 7fc0ddf73640 1 --2- 172.21.15.98:0/1253753095 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0c4004810 0x7fc0c4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:17.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.515+0000 7fc0ddf73640 1 -- 172.21.15.98:0/1253753095 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c4003530 con 0x7fc0c40051d0 2024-09-19T08:45:17.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.515+0000 7fc0dd772640 1 --2- 172.21.15.98:0/1253753095 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0c40039c0 0x7fc0c4003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:17.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.519+0000 7fc0ddf73640 1 -- 172.21.15.98:0/1253753095 reap_dead start 2024-09-19T08:45:17.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.519+0000 7fc0ddf73640 1 --2- 172.21.15.98:0/1253753095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0c40051d0 0x7fc0c400b9d0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0001e70 tx=0x7fc0d001c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f85dc399c6c2f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:17.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.519+0000 7fc0cffff640 1 -- 172.21.15.98:0/1253753095 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d0001a90 con 0x7fc0c40051d0 2024-09-19T08:45:17.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.519+0000 7fc0cffff640 1 -- 172.21.15.98:0/1253753095 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0d0003bc0 con 0x7fc0c40051d0 2024-09-19T08:45:17.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.519+0000 7fc0cffff640 1 -- 172.21.15.98:0/1253753095 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d0005020 con 0x7fc0c40051d0 2024-09-19T08:45:17.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.519+0000 7fc0df9d6640 1 -- 172.21.15.98:0/1253753095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0c40051d0 msgr2=0x7fc0c400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:17.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.519+0000 7fc0df9d6640 1 --2- 172.21.15.98:0/1253753095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0c40051d0 0x7fc0c400b9d0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0001e70 tx=0x7fc0d001c780 comp rx=0 tx=0).stop 2024-09-19T08:45:17.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.519+0000 7fc0df9d6640 1 -- 172.21.15.98:0/1253753095 shutdown_connections 2024-09-19T08:45:17.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.519+0000 7fc0df9d6640 1 --2- 172.21.15.98:0/1253753095 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0c40051d0 0x7fc0c400b9d0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:17.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.519+0000 7fc0df9d6640 1 -- 172.21.15.98:0/1253753095 >> 172.21.15.98:0/1253753095 conn(0x7fc0d80d23d0 msgr2=0x7fc0d80d27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:17.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.519+0000 7fc0df9d6640 1 -- 172.21.15.98:0/1253753095 shutdown_connections 2024-09-19T08:45:17.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.519+0000 7fc0df9d6640 1 -- 172.21.15.98:0/1253753095 wait complete. 2024-09-19T08:45:17.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.519+0000 7fc0df9d6640 1 Processor -- start 2024-09-19T08:45:17.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.519+0000 7fc0df9d6640 1 -- start start 2024-09-19T08:45:17.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.523+0000 7fc0df9d6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0c40051d0 0x7fc0d8154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:17.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.523+0000 7fc0df9d6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0d8155230 0x7fc0d815efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:17.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.523+0000 7fc0df9d6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0d815f4f0 0x7fc0d814d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:17.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.523+0000 7fc0df9d6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d80ce400 con 0x7fc0d8155230 2024-09-19T08:45:17.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.523+0000 7fc0df9d6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d80cdee0 con 0x7fc0c40051d0 2024-09-19T08:45:17.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.523+0000 7fc0df9d6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc0d80ce580 con 0x7fc0d815f4f0 2024-09-19T08:45:17.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.523+0000 7fc0dcf71640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0d8155230 0x7fc0d815efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:17.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.523+0000 7fc0dcf71640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0d8155230 0x7fc0d815efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38748/0 (socket says 172.21.15.98:38748) 2024-09-19T08:45:17.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.523+0000 7fc0dcf71640 1 -- 172.21.15.98:0/24176051 learned_addr learned my addr 172.21.15.98:0/24176051 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:17.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.523+0000 7fc0ddf73640 1 --2- 172.21.15.98:0/24176051 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0d815f4f0 0x7fc0d814d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:17.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.523+0000 7fc0dd772640 1 --2- 172.21.15.98:0/24176051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0c40051d0 0x7fc0d8154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:17.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.523+0000 7fc0dcf71640 1 -- 172.21.15.98:0/24176051 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0d815f4f0 msgr2=0x7fc0d814d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:17.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.523+0000 7fc0dcf71640 1 --2- 172.21.15.98:0/24176051 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0d815f4f0 0x7fc0d814d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:17.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.523+0000 7fc0dcf71640 1 -- 172.21.15.98:0/24176051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0c40051d0 msgr2=0x7fc0d8154cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:17.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.523+0000 7fc0dcf71640 1 --2- 172.21.15.98:0/24176051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0c40051d0 0x7fc0d8154cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:17.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.523+0000 7fc0dcf71640 1 -- 172.21.15.98:0/24176051 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d815f9a0 con 0x7fc0d8155230 2024-09-19T08:45:17.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.523+0000 7fc0dcf71640 1 --2- 172.21.15.98:0/24176051 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0d8155230 0x7fc0d815efb0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8011560 tx=0x7fc0c8011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:17.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.523+0000 7fc0cdffb640 1 -- 172.21.15.98:0/24176051 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8015930 con 0x7fc0d8155230 2024-09-19T08:45:17.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.523+0000 7fc0cdffb640 1 -- 172.21.15.98:0/24176051 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0c8015ad0 con 0x7fc0d8155230 2024-09-19T08:45:17.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.523+0000 7fc0df9d6640 1 -- 172.21.15.98:0/24176051 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0d8155790 con 0x7fc0d8155230 2024-09-19T08:45:17.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.523+0000 7fc0cdffb640 1 -- 172.21.15.98:0/24176051 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c800f460 con 0x7fc0d8155230 2024-09-19T08:45:17.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.523+0000 7fc0df9d6640 1 -- 172.21.15.98:0/24176051 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0d8155d50 con 0x7fc0d8155230 2024-09-19T08:45:17.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.523+0000 7fc0df9d6640 1 -- 172.21.15.98:0/24176051 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0d815a210 con 0x7fc0d8155230 2024-09-19T08:45:17.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.523+0000 7fc0cdffb640 1 -- 172.21.15.98:0/24176051 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fc0c800f600 con 0x7fc0d8155230 2024-09-19T08:45:17.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.523+0000 7fc0cdffb640 1 --2- 172.21.15.98:0/24176051 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc0bc067e90 0x7fc0bc06a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:17.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.523+0000 7fc0cdffb640 1 -- 172.21.15.98:0/24176051 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5881+0+0 (secure 0 0 0) 0x7fc0c8086ff0 con 0x7fc0d8155230 2024-09-19T08:45:17.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.523+0000 7fc0dd772640 1 --2- 172.21.15.98:0/24176051 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc0bc067e90 0x7fc0bc06a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:17.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.527+0000 7fc0dd772640 1 --2- 172.21.15.98:0/24176051 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc0bc067e90 0x7fc0bc06a350 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc0d40017f0 tx=0x7fc0d4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:17.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.527+0000 7fc0cdffb640 1 -- 172.21.15.98:0/24176051 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc0d815f9a0 con 0x7fc0d8155230 2024-09-19T08:45:17.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:17.723+0000 7fc0df9d6640 1 -- 172.21.15.98:0/24176051 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc0d8159f30 con 0x7fc0d8155230 2024-09-19T08:45:18.249 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:45:18.252 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:45:18.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.387+0000 7fc0cdffb640 1 -- 172.21.15.98:0/24176051 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v70) ==== 104+0+0 (secure 0 0 0) 0x7fc0c8019030 con 0x7fc0d8155230 2024-09-19T08:45:18.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.391+0000 7fc0df9d6640 1 -- 172.21.15.98:0/24176051 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc0bc067e90 msgr2=0x7fc0bc06a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:18.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.391+0000 7fc0df9d6640 1 --2- 172.21.15.98:0/24176051 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc0bc067e90 0x7fc0bc06a350 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc0d40017f0 tx=0x7fc0d4009040 comp rx=0 tx=0).stop 2024-09-19T08:45:18.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.391+0000 7fc0df9d6640 1 -- 172.21.15.98:0/24176051 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0d8155230 msgr2=0x7fc0d815efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:18.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.391+0000 7fc0df9d6640 1 --2- 172.21.15.98:0/24176051 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0d8155230 0x7fc0d815efb0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8011560 tx=0x7fc0c8011a20 comp rx=0 tx=0).stop 2024-09-19T08:45:18.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.391+0000 7fc0df9d6640 1 -- 172.21.15.98:0/24176051 shutdown_connections 2024-09-19T08:45:18.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.391+0000 7fc0df9d6640 1 --2- 172.21.15.98:0/24176051 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc0bc067e90 0x7fc0bc06a350 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:18.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.391+0000 7fc0df9d6640 1 --2- 172.21.15.98:0/24176051 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc0d815f4f0 0x7fc0d814d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:18.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.391+0000 7fc0df9d6640 1 --2- 172.21.15.98:0/24176051 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc0d8155230 0x7fc0d815efb0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:18.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.391+0000 7fc0df9d6640 1 --2- 172.21.15.98:0/24176051 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc0c40051d0 0x7fc0d8154cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:18.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.391+0000 7fc0df9d6640 1 -- 172.21.15.98:0/24176051 >> 172.21.15.98:0/24176051 conn(0x7fc0d80d23d0 msgr2=0x7fc0d806f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:18.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.391+0000 7fc0df9d6640 1 -- 172.21.15.98:0/24176051 shutdown_connections 2024-09-19T08:45:18.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.391+0000 7fc0df9d6640 1 -- 172.21.15.98:0/24176051 wait complete. 2024-09-19T08:45:18.419 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T08:45:18.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.543+0000 7f5239772640 1 Processor -- start 2024-09-19T08:45:18.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.543+0000 7f5239772640 1 -- start start 2024-09-19T08:45:18.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.547+0000 7f5239772640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f52340b7770 0x7f52340b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:18.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.547+0000 7f5239772640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f52340d3a60 0x7f52340d3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:18.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.547+0000 7f5239772640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52340cfd20 con 0x7f523415e720 2024-09-19T08:45:18.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.547+0000 7f5239772640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f52340d29c0 con 0x7f52340b7770 2024-09-19T08:45:18.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.547+0000 7f5239772640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f52340d2ca0 con 0x7f52340d3a60 2024-09-19T08:45:18.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.547+0000 7f52327fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f52340b7770 0x7f52340b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:18.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.547+0000 7f52327fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f52340b7770 0x7f52340b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38752/0 (socket says 172.21.15.98:38752) 2024-09-19T08:45:18.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.547+0000 7f52327fc640 1 -- 172.21.15.98:0/3642127968 learned_addr learned my addr 172.21.15.98:0/3642127968 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:18.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.547+0000 7f52337fe640 1 --2- 172.21.15.98:0/3642127968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f52340d3a60 0x7f52340d3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:18.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.547+0000 7f52327fc640 1 -- 172.21.15.98:0/3642127968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f52340d3a60 msgr2=0x7f52340d3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:18.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.547+0000 7f52327fc640 1 --2- 172.21.15.98:0/3642127968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f52340d3a60 0x7f52340d3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:18.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.547+0000 7f52327fc640 1 -- 172.21.15.98:0/3642127968 >> v1:172.21.15.139:6789/0 conn(0x7f523415e720 legacy=0x7f52340cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:18.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.547+0000 7f52327fc640 1 -- 172.21.15.98:0/3642127968 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52340d2f80 con 0x7f52340b7770 2024-09-19T08:45:18.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.547+0000 7f52327fc640 1 --2- 172.21.15.98:0/3642127968 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52340b7770 0x7f52340b7be0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f5224009870 tx=0x7f522401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f5e0aa210dbc8b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:18.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.547+0000 7f5231ffb640 1 -- 172.21.15.98:0/3642127968 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52240b2070 con 0x7f52340b7770 2024-09-19T08:45:18.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.547+0000 7f5231ffb640 1 -- 172.21.15.98:0/3642127968 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52240a9070 con 0x7f52340b7770 2024-09-19T08:45:18.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.547+0000 7f5231ffb640 1 -- 172.21.15.98:0/3642127968 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52240ad880 con 0x7f52340b7770 2024-09-19T08:45:18.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.551+0000 7f5239772640 1 -- 172.21.15.98:0/3642127968 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52340b7770 msgr2=0x7f52340b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:18.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.551+0000 7f5239772640 1 --2- 172.21.15.98:0/3642127968 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52340b7770 0x7f52340b7be0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f5224009870 tx=0x7f522401c560 comp rx=0 tx=0).stop 2024-09-19T08:45:18.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.551+0000 7f5239772640 1 -- 172.21.15.98:0/3642127968 shutdown_connections 2024-09-19T08:45:18.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.551+0000 7f5239772640 1 --2- 172.21.15.98:0/3642127968 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f52340d3a60 0x7f52340d3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:18.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.551+0000 7f5239772640 1 --2- 172.21.15.98:0/3642127968 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52340b7770 0x7f52340b7be0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:18.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.551+0000 7f5239772640 1 -- 172.21.15.98:0/3642127968 >> 172.21.15.98:0/3642127968 conn(0x7f52340d4640 msgr2=0x7f52340d4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:18.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.551+0000 7f5239772640 1 -- 172.21.15.98:0/3642127968 shutdown_connections 2024-09-19T08:45:18.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.551+0000 7f5239772640 1 -- 172.21.15.98:0/3642127968 wait complete. 2024-09-19T08:45:18.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.551+0000 7f5239772640 1 Processor -- start 2024-09-19T08:45:18.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.551+0000 7f5239772640 1 -- start start 2024-09-19T08:45:18.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.551+0000 7f5239772640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f52340b7770 0x7f5234161a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:18.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.551+0000 7f5239772640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52340d3a60 0x7f5234153440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:18.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.551+0000 7f5239772640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f523415e720 0x7f52340cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:18.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.551+0000 7f5239772640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f523415ec80 con 0x7f52340d3a60 2024-09-19T08:45:18.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.551+0000 7f5239772640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f523415eb00 con 0x7f523415e720 2024-09-19T08:45:18.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.551+0000 7f5239772640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f523416b6f0 con 0x7f52340b7770 2024-09-19T08:45:18.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.551+0000 7f52327fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52340d3a60 0x7f5234153440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:18.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.551+0000 7f52327fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52340d3a60 0x7f5234153440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38764/0 (socket says 172.21.15.98:38764) 2024-09-19T08:45:18.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.551+0000 7f52327fc640 1 -- 172.21.15.98:0/959805830 learned_addr learned my addr 172.21.15.98:0/959805830 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:18.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.551+0000 7f5232ffd640 1 --2- 172.21.15.98:0/959805830 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f52340b7770 0x7f5234161a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:18.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.551+0000 7f52337fe640 1 --2- 172.21.15.98:0/959805830 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f523415e720 0x7f52340cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:18.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.551+0000 7f52327fc640 1 -- 172.21.15.98:0/959805830 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f52340b7770 msgr2=0x7f5234161a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:18.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.551+0000 7f52327fc640 1 --2- 172.21.15.98:0/959805830 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f52340b7770 0x7f5234161a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:18.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.551+0000 7f52327fc640 1 -- 172.21.15.98:0/959805830 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f523415e720 msgr2=0x7f52340cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:18.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.551+0000 7f52327fc640 1 --2- 172.21.15.98:0/959805830 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f523415e720 0x7f52340cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:18.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.551+0000 7f52327fc640 1 -- 172.21.15.98:0/959805830 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52340cbfc0 con 0x7f52340d3a60 2024-09-19T08:45:18.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.551+0000 7f52337fe640 1 --2- 172.21.15.98:0/959805830 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f523415e720 0x7f52340cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:18.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.551+0000 7f52327fc640 1 --2- 172.21.15.98:0/959805830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52340d3a60 0x7f5234153440 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f522401a190 tx=0x7f5224019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:18.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.551+0000 7f521b7fe640 1 -- 172.21.15.98:0/959805830 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52240bb070 con 0x7f52340d3a60 2024-09-19T08:45:18.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.551+0000 7f521b7fe640 1 -- 172.21.15.98:0/959805830 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52240add90 con 0x7f52340d3a60 2024-09-19T08:45:18.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.551+0000 7f5239772640 1 -- 172.21.15.98:0/959805830 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52340cc260 con 0x7f52340d3a60 2024-09-19T08:45:18.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.551+0000 7f521b7fe640 1 -- 172.21.15.98:0/959805830 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52240b2040 con 0x7f52340d3a60 2024-09-19T08:45:18.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.551+0000 7f5239772640 1 -- 172.21.15.98:0/959805830 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52340b82d0 con 0x7f52340d3a60 2024-09-19T08:45:18.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.555+0000 7f5239772640 1 -- 172.21.15.98:0/959805830 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52340b87c0 con 0x7f52340d3a60 2024-09-19T08:45:18.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.555+0000 7f521b7fe640 1 -- 172.21.15.98:0/959805830 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f52240be070 con 0x7f52340d3a60 2024-09-19T08:45:18.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.555+0000 7f521b7fe640 1 --2- 172.21.15.98:0/959805830 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5220067e60 0x7f522006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:18.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.555+0000 7f521b7fe640 1 -- 172.21.15.98:0/959805830 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5881+0+0 (secure 0 0 0) 0x7f522411f6f0 con 0x7f52340d3a60 2024-09-19T08:45:18.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.555+0000 7f5232ffd640 1 --2- 172.21.15.98:0/959805830 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5220067e60 0x7f522006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:18.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.555+0000 7f5232ffd640 1 --2- 172.21.15.98:0/959805830 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5220067e60 0x7f522006a320 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5228000f70 tx=0x7f5228009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:18.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.559+0000 7f521b7fe640 1 -- 172.21.15.98:0/959805830 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f5224016150 con 0x7f52340d3a60 2024-09-19T08:45:18.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:18.723+0000 7f5239772640 1 -- 172.21.15.98:0/959805830 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f52340b7be0 con 0x7f52340d3a60 2024-09-19T08:45:19.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.391+0000 7f521b7fe640 1 -- 172.21.15.98:0/959805830 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v125) ==== 181+0+0 (secure 0 0 0) 0x7f52240ef900 con 0x7f52340d3a60 2024-09-19T08:45:19.394 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_metadata' removed 2024-09-19T08:45:19.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.395+0000 7f5239772640 1 -- 172.21.15.98:0/959805830 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5220067e60 msgr2=0x7f522006a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:19.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.395+0000 7f5239772640 1 --2- 172.21.15.98:0/959805830 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5220067e60 0x7f522006a320 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5228000f70 tx=0x7f5228009040 comp rx=0 tx=0).stop 2024-09-19T08:45:19.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.395+0000 7f5239772640 1 -- 172.21.15.98:0/959805830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52340d3a60 msgr2=0x7f5234153440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:19.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.395+0000 7f5239772640 1 --2- 172.21.15.98:0/959805830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52340d3a60 0x7f5234153440 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f522401a190 tx=0x7f5224019300 comp rx=0 tx=0).stop 2024-09-19T08:45:19.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.395+0000 7f5239772640 1 -- 172.21.15.98:0/959805830 shutdown_connections 2024-09-19T08:45:19.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.395+0000 7f5239772640 1 --2- 172.21.15.98:0/959805830 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5220067e60 0x7f522006a320 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:19.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.395+0000 7f5239772640 1 --2- 172.21.15.98:0/959805830 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f523415e720 0x7f52340cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:19.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.395+0000 7f5239772640 1 --2- 172.21.15.98:0/959805830 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f52340d3a60 0x7f5234153440 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:19.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.395+0000 7f5239772640 1 --2- 172.21.15.98:0/959805830 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f52340b7770 0x7f5234161a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:19.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.395+0000 7f5239772640 1 -- 172.21.15.98:0/959805830 >> 172.21.15.98:0/959805830 conn(0x7f52340d4640 msgr2=0x7f5234158aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:19.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.395+0000 7f5239772640 1 -- 172.21.15.98:0/959805830 shutdown_connections 2024-09-19T08:45:19.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.395+0000 7f5239772640 1 -- 172.21.15.98:0/959805830 wait complete. 2024-09-19T08:45:19.420 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T08:45:19.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.547+0000 7f21d3357640 1 Processor -- start 2024-09-19T08:45:19.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.547+0000 7f21d3357640 1 -- start start 2024-09-19T08:45:19.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.547+0000 7f21d3357640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f21cc15c570 0x7f21cc160030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:19.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.547+0000 7f21d3357640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f21cc0d0480 0x7f21cc160570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:19.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.547+0000 7f21d3357640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21cc0d1740 con 0x7f21cc160ab0 2024-09-19T08:45:19.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.547+0000 7f21d3357640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f21cc0d1a20 con 0x7f21cc0d0480 2024-09-19T08:45:19.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.547+0000 7f21d3357640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f21cc0cf630 con 0x7f21cc15c570 2024-09-19T08:45:19.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.547+0000 7f21d08f2640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f21cc0d0480 0x7f21cc160570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:19.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.547+0000 7f21d08f2640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f21cc0d0480 0x7f21cc160570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38770/0 (socket says 172.21.15.98:38770) 2024-09-19T08:45:19.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.547+0000 7f21d08f2640 1 -- 172.21.15.98:0/879575687 learned_addr learned my addr 172.21.15.98:0/879575687 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:19.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.547+0000 7f21d10f3640 1 --2- 172.21.15.98:0/879575687 >> v2:172.21.15.139:3300/0 conn(0x7f21cc15c570 0x7f21cc160030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:19.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.547+0000 7f21d08f2640 1 -- 172.21.15.98:0/879575687 >> v2:172.21.15.139:3300/0 conn(0x7f21cc15c570 msgr2=0x7f21cc160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:19.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.547+0000 7f21d08f2640 1 --2- 172.21.15.98:0/879575687 >> v2:172.21.15.139:3300/0 conn(0x7f21cc15c570 0x7f21cc160030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:19.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.547+0000 7f21d08f2640 1 -- 172.21.15.98:0/879575687 >> v1:172.21.15.139:6789/0 conn(0x7f21cc160ab0 legacy=0x7f21cc0b4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:19.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.547+0000 7f21d08f2640 1 -- 172.21.15.98:0/879575687 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21cc0cf910 con 0x7f21cc0d0480 2024-09-19T08:45:19.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.551+0000 7f21d08f2640 1 --2- 172.21.15.98:0/879575687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21cc0d0480 0x7f21cc160570 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f21cc050140 tx=0x7f21bc01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d39d2b37988f4e18 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:19.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.551+0000 7f21c3fff640 1 -- 172.21.15.98:0/879575687 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21bc0b0070 con 0x7f21cc0d0480 2024-09-19T08:45:19.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.551+0000 7f21c3fff640 1 -- 172.21.15.98:0/879575687 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21bc0ab600 con 0x7f21cc0d0480 2024-09-19T08:45:19.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.551+0000 7f21c3fff640 1 -- 172.21.15.98:0/879575687 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21bc0ab9d0 con 0x7f21cc0d0480 2024-09-19T08:45:19.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.551+0000 7f21d3357640 1 -- 172.21.15.98:0/879575687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21cc0d0480 msgr2=0x7f21cc160570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:19.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.551+0000 7f21d3357640 1 --2- 172.21.15.98:0/879575687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21cc0d0480 0x7f21cc160570 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f21cc050140 tx=0x7f21bc01c3c0 comp rx=0 tx=0).stop 2024-09-19T08:45:19.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.551+0000 7f21d3357640 1 -- 172.21.15.98:0/879575687 shutdown_connections 2024-09-19T08:45:19.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.551+0000 7f21d3357640 1 --2- 172.21.15.98:0/879575687 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21cc0d0480 0x7f21cc160570 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:19.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.551+0000 7f21d3357640 1 --2- 172.21.15.98:0/879575687 >> v2:172.21.15.139:3300/0 conn(0x7f21cc15c570 0x7f21cc160030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:19.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.551+0000 7f21d3357640 1 -- 172.21.15.98:0/879575687 >> 172.21.15.98:0/879575687 conn(0x7f21cc0d0af0 msgr2=0x7f21cc0d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:19.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.551+0000 7f21d3357640 1 -- 172.21.15.98:0/879575687 shutdown_connections 2024-09-19T08:45:19.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.551+0000 7f21d3357640 1 -- 172.21.15.98:0/879575687 wait complete. 2024-09-19T08:45:19.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.551+0000 7f21d3357640 1 Processor -- start 2024-09-19T08:45:19.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.551+0000 7f21d3357640 1 -- start start 2024-09-19T08:45:19.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.555+0000 7f21d3357640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21cc0d0480 0x7f21cc072440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:19.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.555+0000 7f21d3357640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21cc15c570 0x7f21cc152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:19.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.555+0000 7f21d3357640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21cc160ab0 0x7f21cc154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:19.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.555+0000 7f21d3357640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f21cc169550 con 0x7f21cc15c570 2024-09-19T08:45:19.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.555+0000 7f21d3357640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f21cc1693d0 con 0x7f21cc0d0480 2024-09-19T08:45:19.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.555+0000 7f21d3357640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f21cc1696d0 con 0x7f21cc160ab0 2024-09-19T08:45:19.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.555+0000 7f21d08f2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21cc15c570 0x7f21cc152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:19.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.555+0000 7f21d08f2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21cc15c570 0x7f21cc152040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38780/0 (socket says 172.21.15.98:38780) 2024-09-19T08:45:19.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.555+0000 7f21d08f2640 1 -- 172.21.15.98:0/3004672194 learned_addr learned my addr 172.21.15.98:0/3004672194 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:19.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.555+0000 7f21d18f4640 1 --2- 172.21.15.98:0/3004672194 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21cc160ab0 0x7f21cc154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:19.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.555+0000 7f21d10f3640 1 --2- 172.21.15.98:0/3004672194 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21cc0d0480 0x7f21cc072440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:19.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.555+0000 7f21d08f2640 1 -- 172.21.15.98:0/3004672194 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21cc160ab0 msgr2=0x7f21cc154a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:19.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.555+0000 7f21d08f2640 1 --2- 172.21.15.98:0/3004672194 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21cc160ab0 0x7f21cc154a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:19.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.555+0000 7f21d08f2640 1 -- 172.21.15.98:0/3004672194 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21cc0d0480 msgr2=0x7f21cc072440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:19.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.555+0000 7f21d08f2640 1 --2- 172.21.15.98:0/3004672194 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21cc0d0480 0x7f21cc072440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:19.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.555+0000 7f21d08f2640 1 -- 172.21.15.98:0/3004672194 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21cc152580 con 0x7f21cc15c570 2024-09-19T08:45:19.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.555+0000 7f21d18f4640 1 --2- 172.21.15.98:0/3004672194 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21cc160ab0 0x7f21cc154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:19.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.555+0000 7f21d08f2640 1 --2- 172.21.15.98:0/3004672194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21cc15c570 0x7f21cc152040 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f21bc01aa70 tx=0x7f21bc019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:19.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.555+0000 7f21c1ffb640 1 -- 172.21.15.98:0/3004672194 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21bc0b0040 con 0x7f21cc15c570 2024-09-19T08:45:19.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.555+0000 7f21c1ffb640 1 -- 172.21.15.98:0/3004672194 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21bc0b9070 con 0x7f21cc15c570 2024-09-19T08:45:19.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.555+0000 7f21d3357640 1 -- 172.21.15.98:0/3004672194 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21cc1554c0 con 0x7f21cc15c570 2024-09-19T08:45:19.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.555+0000 7f21c1ffb640 1 -- 172.21.15.98:0/3004672194 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21bc0b4680 con 0x7f21cc15c570 2024-09-19T08:45:19.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.555+0000 7f21d3357640 1 -- 172.21.15.98:0/3004672194 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21cc071c40 con 0x7f21cc15c570 2024-09-19T08:45:19.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.555+0000 7f21d3357640 1 -- 172.21.15.98:0/3004672194 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21cc069230 con 0x7f21cc15c570 2024-09-19T08:45:19.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.555+0000 7f21c1ffb640 1 -- 172.21.15.98:0/3004672194 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f21bc0b4820 con 0x7f21cc15c570 2024-09-19T08:45:19.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.555+0000 7f21c1ffb640 1 --2- 172.21.15.98:0/3004672194 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f21b0067e60 0x7f21b006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:19.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.555+0000 7f21c1ffb640 1 -- 172.21.15.98:0/3004672194 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5443+0+0 (secure 0 0 0) 0x7f21bc1203b0 con 0x7f21cc15c570 2024-09-19T08:45:19.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.555+0000 7f21d10f3640 1 --2- 172.21.15.98:0/3004672194 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f21b0067e60 0x7f21b006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:19.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.559+0000 7f21d10f3640 1 --2- 172.21.15.98:0/3004672194 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f21b0067e60 0x7f21b006a320 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f21cc1664c0 tx=0x7f21c8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:19.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.559+0000 7f21c1ffb640 1 -- 172.21.15.98:0/3004672194 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f21bc0160d0 con 0x7f21cc15c570 2024-09-19T08:45:19.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:19.723+0000 7f21d3357640 1 -- 172.21.15.98:0/3004672194 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f21cc0b4c90 con 0x7f21cc15c570 2024-09-19T08:45:20.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.403+0000 7f21c1ffb640 1 -- 172.21.15.98:0/3004672194 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v126) ==== 169+0+0 (secure 0 0 0) 0x7f21bc0f0670 con 0x7f21cc15c570 2024-09-19T08:45:20.405 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' removed 2024-09-19T08:45:20.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.403+0000 7f21d3357640 1 -- 172.21.15.98:0/3004672194 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f21b0067e60 msgr2=0x7f21b006a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:20.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.403+0000 7f21d3357640 1 --2- 172.21.15.98:0/3004672194 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f21b0067e60 0x7f21b006a320 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f21cc1664c0 tx=0x7f21c8008040 comp rx=0 tx=0).stop 2024-09-19T08:45:20.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.403+0000 7f21d3357640 1 -- 172.21.15.98:0/3004672194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21cc15c570 msgr2=0x7f21cc152040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:20.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.403+0000 7f21d3357640 1 --2- 172.21.15.98:0/3004672194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21cc15c570 0x7f21cc152040 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f21bc01aa70 tx=0x7f21bc019160 comp rx=0 tx=0).stop 2024-09-19T08:45:20.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.407+0000 7f21d3357640 1 -- 172.21.15.98:0/3004672194 shutdown_connections 2024-09-19T08:45:20.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.407+0000 7f21d3357640 1 --2- 172.21.15.98:0/3004672194 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f21b0067e60 0x7f21b006a320 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:20.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.407+0000 7f21d3357640 1 --2- 172.21.15.98:0/3004672194 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f21cc160ab0 0x7f21cc154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:20.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.407+0000 7f21d3357640 1 --2- 172.21.15.98:0/3004672194 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f21cc15c570 0x7f21cc152040 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:20.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.407+0000 7f21d3357640 1 --2- 172.21.15.98:0/3004672194 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f21cc0d0480 0x7f21cc072440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:20.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.407+0000 7f21d3357640 1 -- 172.21.15.98:0/3004672194 >> 172.21.15.98:0/3004672194 conn(0x7f21cc0d0af0 msgr2=0x7f21cc0b15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:20.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.407+0000 7f21d3357640 1 -- 172.21.15.98:0/3004672194 shutdown_connections 2024-09-19T08:45:20.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.407+0000 7f21d3357640 1 -- 172.21.15.98:0/3004672194 wait complete. 2024-09-19T08:45:20.433 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-19T08:45:20.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.551+0000 7f9112cc7640 1 Processor -- start 2024-09-19T08:45:20.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.551+0000 7f9112cc7640 1 -- start start 2024-09-19T08:45:20.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.551+0000 7f9112cc7640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f910c145b70 0x7f910c06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:20.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.551+0000 7f9112cc7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f910c071ac0 0x7f910c0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:20.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.551+0000 7f9112cc7640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f910c073530 con 0x7f910c06c1d0 2024-09-19T08:45:20.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.551+0000 7f9112cc7640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f910c071f10 con 0x7f910c145b70 2024-09-19T08:45:20.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.551+0000 7f9112cc7640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f910c06c620 con 0x7f910c071ac0 2024-09-19T08:45:20.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.555+0000 7f9110a63640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f910c145b70 0x7f910c06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:20.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.555+0000 7f9110a63640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f910c145b70 0x7f910c06bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38782/0 (socket says 172.21.15.98:38782) 2024-09-19T08:45:20.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.555+0000 7f9110a63640 1 -- 172.21.15.98:0/586786006 learned_addr learned my addr 172.21.15.98:0/586786006 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:20.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.555+0000 7f9111264640 1 --2- 172.21.15.98:0/586786006 >> v2:172.21.15.139:3300/0 conn(0x7f910c071ac0 0x7f910c0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:20.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.555+0000 7f9110a63640 1 -- 172.21.15.98:0/586786006 >> v2:172.21.15.139:3300/0 conn(0x7f910c071ac0 msgr2=0x7f910c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:20.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.555+0000 7f9110a63640 1 --2- 172.21.15.98:0/586786006 >> v2:172.21.15.139:3300/0 conn(0x7f910c071ac0 0x7f910c0b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:20.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.555+0000 7f9110a63640 1 -- 172.21.15.98:0/586786006 >> v1:172.21.15.139:6789/0 conn(0x7f910c06c1d0 legacy=0x7f910c0713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:20.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.555+0000 7f9110a63640 1 -- 172.21.15.98:0/586786006 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f910c073810 con 0x7f910c145b70 2024-09-19T08:45:20.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.555+0000 7f9110a63640 1 --2- 172.21.15.98:0/586786006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f910c145b70 0x7f910c06bc90 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f910c074450 tx=0x7f910001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba8c15121b495b64 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:20.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.555+0000 7f910b7fe640 1 -- 172.21.15.98:0/586786006 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91000b0070 con 0x7f910c145b70 2024-09-19T08:45:20.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.555+0000 7f910b7fe640 1 -- 172.21.15.98:0/586786006 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91000ab600 con 0x7f910c145b70 2024-09-19T08:45:20.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.555+0000 7f910b7fe640 1 -- 172.21.15.98:0/586786006 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91000ab9d0 con 0x7f910c145b70 2024-09-19T08:45:20.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.559+0000 7f9112cc7640 1 -- 172.21.15.98:0/586786006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f910c145b70 msgr2=0x7f910c06bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:20.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.559+0000 7f9112cc7640 1 --2- 172.21.15.98:0/586786006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f910c145b70 0x7f910c06bc90 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f910c074450 tx=0x7f910001c490 comp rx=0 tx=0).stop 2024-09-19T08:45:20.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.559+0000 7f9112cc7640 1 -- 172.21.15.98:0/586786006 shutdown_connections 2024-09-19T08:45:20.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.559+0000 7f9112cc7640 1 --2- 172.21.15.98:0/586786006 >> v2:172.21.15.139:3300/0 conn(0x7f910c071ac0 0x7f910c0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:20.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.559+0000 7f9112cc7640 1 --2- 172.21.15.98:0/586786006 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f910c145b70 0x7f910c06bc90 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:20.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.559+0000 7f9112cc7640 1 -- 172.21.15.98:0/586786006 >> 172.21.15.98:0/586786006 conn(0x7f910c0b2010 msgr2=0x7f910c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:20.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.559+0000 7f9112cc7640 1 -- 172.21.15.98:0/586786006 shutdown_connections 2024-09-19T08:45:20.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.559+0000 7f9112cc7640 1 -- 172.21.15.98:0/586786006 wait complete. 2024-09-19T08:45:20.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.559+0000 7f9112cc7640 1 Processor -- start 2024-09-19T08:45:20.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.559+0000 7f9112cc7640 1 -- start start 2024-09-19T08:45:20.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.559+0000 7f9112cc7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f910c06c1d0 0x7f910c159590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:20.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.559+0000 7f9112cc7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f910c071ac0 0x7f910c162450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:20.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.559+0000 7f9112cc7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f910c145b70 0x7f910c158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:20.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.559+0000 7f9112cc7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f910c0b42e0 con 0x7f910c071ac0 2024-09-19T08:45:20.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.559+0000 7f9112cc7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f910c0b4160 con 0x7f910c06c1d0 2024-09-19T08:45:20.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.559+0000 7f9112cc7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f910c0b4460 con 0x7f910c145b70 2024-09-19T08:45:20.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.559+0000 7f910bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f910c071ac0 0x7f910c162450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:20.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.559+0000 7f910bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f910c071ac0 0x7f910c162450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38798/0 (socket says 172.21.15.98:38798) 2024-09-19T08:45:20.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.559+0000 7f910bfff640 1 -- 172.21.15.98:0/3317201220 learned_addr learned my addr 172.21.15.98:0/3317201220 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:20.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.559+0000 7f9111264640 1 --2- 172.21.15.98:0/3317201220 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f910c145b70 0x7f910c158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:20.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.559+0000 7f9110a63640 1 --2- 172.21.15.98:0/3317201220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f910c06c1d0 0x7f910c159590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:20.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.559+0000 7f910bfff640 1 -- 172.21.15.98:0/3317201220 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f910c145b70 msgr2=0x7f910c158d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:20.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.559+0000 7f910bfff640 1 --2- 172.21.15.98:0/3317201220 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f910c145b70 0x7f910c158d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:20.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.559+0000 7f910bfff640 1 -- 172.21.15.98:0/3317201220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f910c06c1d0 msgr2=0x7f910c159590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:20.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.559+0000 7f910bfff640 1 --2- 172.21.15.98:0/3317201220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f910c06c1d0 0x7f910c159590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:20.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.559+0000 7f910bfff640 1 -- 172.21.15.98:0/3317201220 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f910c162b30 con 0x7f910c071ac0 2024-09-19T08:45:20.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.559+0000 7f9110a63640 1 --2- 172.21.15.98:0/3317201220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f910c06c1d0 0x7f910c159590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:20.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.559+0000 7f910bfff640 1 --2- 172.21.15.98:0/3317201220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f910c071ac0 0x7f910c162450 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f90f800c420 tx=0x7f90f800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:20.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.559+0000 7f91097fa640 1 -- 172.21.15.98:0/3317201220 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f80052d0 con 0x7f910c071ac0 2024-09-19T08:45:20.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.559+0000 7f91097fa640 1 -- 172.21.15.98:0/3317201220 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90f8005470 con 0x7f910c071ac0 2024-09-19T08:45:20.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.559+0000 7f9112cc7640 1 -- 172.21.15.98:0/3317201220 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f910c162e10 con 0x7f910c071ac0 2024-09-19T08:45:20.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.559+0000 7f91097fa640 1 -- 172.21.15.98:0/3317201220 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f8007260 con 0x7f910c071ac0 2024-09-19T08:45:20.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.559+0000 7f9112cc7640 1 -- 172.21.15.98:0/3317201220 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f910c1592c0 con 0x7f910c071ac0 2024-09-19T08:45:20.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.563+0000 7f9112cc7640 1 -- 172.21.15.98:0/3317201220 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f910c0b3410 con 0x7f910c071ac0 2024-09-19T08:45:20.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.563+0000 7f91097fa640 1 -- 172.21.15.98:0/3317201220 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f90f8007400 con 0x7f910c071ac0 2024-09-19T08:45:20.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.563+0000 7f91097fa640 1 --2- 172.21.15.98:0/3317201220 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f90f0067e60 0x7f90f006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:20.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.563+0000 7f91097fa640 1 -- 172.21.15.98:0/3317201220 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 4971+0+0 (secure 0 0 0) 0x7f90f8085960 con 0x7f910c071ac0 2024-09-19T08:45:20.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.563+0000 7f9110a63640 1 --2- 172.21.15.98:0/3317201220 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f90f0067e60 0x7f90f006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:20.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.563+0000 7f9110a63640 1 --2- 172.21.15.98:0/3317201220 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f90f0067e60 0x7f90f006a320 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f91000ad000 tx=0x7f9100008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:20.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.567+0000 7f91097fa640 1 -- 172.21.15.98:0/3317201220 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f90f8055f10 con 0x7f910c071ac0 2024-09-19T08:45:20.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.731+0000 7f9112cc7640 1 -- 172.21.15.98:0/3317201220 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f910c0b2520 con 0x7f910c071ac0 2024-09-19T08:45:20.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.735+0000 7f91097fa640 1 -- 172.21.15.98:0/3317201220 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7f90f8055810 con 0x7f910c071ac0 2024-09-19T08:45:20.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.739+0000 7f9112cc7640 1 -- 172.21.15.98:0/3317201220 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f90f0067e60 msgr2=0x7f90f006a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:20.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.739+0000 7f9112cc7640 1 --2- 172.21.15.98:0/3317201220 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f90f0067e60 0x7f90f006a320 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f91000ad000 tx=0x7f9100008040 comp rx=0 tx=0).stop 2024-09-19T08:45:20.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.739+0000 7f9112cc7640 1 -- 172.21.15.98:0/3317201220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f910c071ac0 msgr2=0x7f910c162450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:20.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.739+0000 7f9112cc7640 1 --2- 172.21.15.98:0/3317201220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f910c071ac0 0x7f910c162450 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f90f800c420 tx=0x7f90f800c8e0 comp rx=0 tx=0).stop 2024-09-19T08:45:20.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.739+0000 7f9112cc7640 1 -- 172.21.15.98:0/3317201220 shutdown_connections 2024-09-19T08:45:20.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.739+0000 7f9112cc7640 1 --2- 172.21.15.98:0/3317201220 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f90f0067e60 0x7f90f006a320 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:20.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.739+0000 7f9112cc7640 1 --2- 172.21.15.98:0/3317201220 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f910c145b70 0x7f910c158d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:20.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.739+0000 7f9112cc7640 1 --2- 172.21.15.98:0/3317201220 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f910c071ac0 0x7f910c162450 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:20.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.739+0000 7f9112cc7640 1 --2- 172.21.15.98:0/3317201220 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f910c06c1d0 0x7f910c159590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:20.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.739+0000 7f9112cc7640 1 -- 172.21.15.98:0/3317201220 >> 172.21.15.98:0/3317201220 conn(0x7f910c0b2010 msgr2=0x7f910c0b0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:20.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.739+0000 7f9112cc7640 1 -- 172.21.15.98:0/3317201220 shutdown_connections 2024-09-19T08:45:20.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.739+0000 7f9112cc7640 1 -- 172.21.15.98:0/3317201220 wait complete. 2024-09-19T08:45:20.759 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2' 2024-09-19T08:45:20.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.875+0000 7fea6c30c640 1 Processor -- start 2024-09-19T08:45:20.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.875+0000 7fea6c30c640 1 -- start start 2024-09-19T08:45:20.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.875+0000 7fea6c30c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fea6406c3a0 0x7fea640cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:20.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.875+0000 7fea6c30c640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea6406d310 con 0x7fea6415c550 2024-09-19T08:45:20.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.875+0000 7fea6c30c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea6406d5f0 con 0x7fea640cf900 2024-09-19T08:45:20.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.875+0000 7fea6c30c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fea6406d8d0 con 0x7fea6406c3a0 2024-09-19T08:45:20.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.879+0000 7fea6a0a8640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fea6415c550 0x7fea6406bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:35812/0 (socket says 172.21.15.98:35812) 2024-09-19T08:45:20.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.879+0000 7fea6a0a8640 1 -- 172.21.15.98:0/3263280141 learned_addr learned my addr 172.21.15.98:0/3263280141 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:20.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.879+0000 7fea698a7640 1 --2- 172.21.15.98:0/3263280141 >> v2:172.21.15.139:3300/0 conn(0x7fea6406c3a0 0x7fea640cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:20.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.879+0000 7fea690a6640 1 -- 172.21.15.98:0/3263280141 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2997608943 0 0) 0x7fea6406d310 con 0x7fea6415c550 2024-09-19T08:45:20.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.879+0000 7fea690a6640 1 -- 172.21.15.98:0/3263280141 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea50003530 con 0x7fea6415c550 2024-09-19T08:45:20.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.883+0000 7fea690a6640 1 -- 172.21.15.98:0/3263280141 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2702915193 0 0) 0x7fea50003530 con 0x7fea6415c550 2024-09-19T08:45:20.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.883+0000 7fea690a6640 1 -- 172.21.15.98:0/3263280141 >> v2:172.21.15.139:3300/0 conn(0x7fea6406c3a0 msgr2=0x7fea640cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:20.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.883+0000 7fea690a6640 1 --2- 172.21.15.98:0/3263280141 >> v2:172.21.15.139:3300/0 conn(0x7fea6406c3a0 0x7fea640cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:20.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.883+0000 7fea690a6640 1 -- 172.21.15.98:0/3263280141 >> v1:172.21.15.139:6789/0 conn(0x7fea640cf900 legacy=0x7fea640723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:20.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.883+0000 7fea690a6640 1 -- 172.21.15.98:0/3263280141 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea6406dbb0 con 0x7fea6415c550 2024-09-19T08:45:20.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.883+0000 7fea690a6640 1 -- 172.21.15.98:0/3263280141 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fea60001680 con 0x7fea6415c550 2024-09-19T08:45:20.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.883+0000 7fea690a6640 1 -- 172.21.15.98:0/3263280141 >> v1:172.21.15.98:6789/0 conn(0x7fea6415c550 legacy=0x7fea6406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:20.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.883+0000 7fea690a6640 1 --2- 172.21.15.98:0/3263280141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea500039c0 0x7fea50003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:20.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.883+0000 7fea690a6640 1 --2- 172.21.15.98:0/3263280141 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fea50004810 0x7fea50004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:20.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.883+0000 7fea690a6640 1 --2- 172.21.15.98:0/3263280141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea500051d0 0x7fea500079b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:20.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.883+0000 7fea690a6640 1 -- 172.21.15.98:0/3263280141 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fea6406d8d0 con 0x7fea500051d0 2024-09-19T08:45:20.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.883+0000 7fea690a6640 1 -- 172.21.15.98:0/3263280141 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fea50008100 con 0x7fea500039c0 2024-09-19T08:45:20.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.883+0000 7fea690a6640 1 -- 172.21.15.98:0/3263280141 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fea50008300 con 0x7fea50004810 2024-09-19T08:45:20.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.883+0000 7fea6a8a9640 1 --2- 172.21.15.98:0/3263280141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea500051d0 0x7fea500079b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:20.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.883+0000 7fea698a7640 1 --2- 172.21.15.98:0/3263280141 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fea50004810 0x7fea50004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:20.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.883+0000 7fea6a8a9640 1 -- 172.21.15.98:0/3263280141 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fea50004810 msgr2=0x7fea50004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:20.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.883+0000 7fea6a8a9640 1 --2- 172.21.15.98:0/3263280141 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fea50004810 0x7fea50004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:20.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.883+0000 7fea6a8a9640 1 -- 172.21.15.98:0/3263280141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea500039c0 msgr2=0x7fea50003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:20.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.883+0000 7fea6a0a8640 1 --2- 172.21.15.98:0/3263280141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea500039c0 0x7fea50003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:20.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.883+0000 7fea6a8a9640 1 --2- 172.21.15.98:0/3263280141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea500039c0 0x7fea50003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:20.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.883+0000 7fea6a8a9640 1 -- 172.21.15.98:0/3263280141 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea50003530 con 0x7fea500051d0 2024-09-19T08:45:20.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.883+0000 7fea6a0a8640 1 --2- 172.21.15.98:0/3263280141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea500039c0 0x7fea50003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:20.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.887+0000 7fea6a8a9640 1 -- 172.21.15.98:0/3263280141 reap_dead start 2024-09-19T08:45:20.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.887+0000 7fea6a8a9640 1 --2- 172.21.15.98:0/3263280141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea500051d0 0x7fea500079b0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fea5c001e70 tx=0x7fea5c01c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5208cfa5b8cf52bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:20.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.887+0000 7fea690a6640 1 -- 172.21.15.98:0/3263280141 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c001a90 con 0x7fea500051d0 2024-09-19T08:45:20.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.887+0000 7fea690a6640 1 -- 172.21.15.98:0/3263280141 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea5c003bc0 con 0x7fea500051d0 2024-09-19T08:45:20.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.887+0000 7fea690a6640 1 -- 172.21.15.98:0/3263280141 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c005020 con 0x7fea500051d0 2024-09-19T08:45:20.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.887+0000 7fea6c30c640 1 -- 172.21.15.98:0/3263280141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea500051d0 msgr2=0x7fea500079b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:20.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.887+0000 7fea6c30c640 1 --2- 172.21.15.98:0/3263280141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea500051d0 0x7fea500079b0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fea5c001e70 tx=0x7fea5c01c780 comp rx=0 tx=0).stop 2024-09-19T08:45:20.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.887+0000 7fea6c30c640 1 -- 172.21.15.98:0/3263280141 shutdown_connections 2024-09-19T08:45:20.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.887+0000 7fea6c30c640 1 --2- 172.21.15.98:0/3263280141 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea500051d0 0x7fea500079b0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:20.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.887+0000 7fea6c30c640 1 -- 172.21.15.98:0/3263280141 >> 172.21.15.98:0/3263280141 conn(0x7fea640b2010 msgr2=0x7fea640b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:20.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.887+0000 7fea6c30c640 1 -- 172.21.15.98:0/3263280141 shutdown_connections 2024-09-19T08:45:20.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.887+0000 7fea6c30c640 1 -- 172.21.15.98:0/3263280141 wait complete. 2024-09-19T08:45:20.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.887+0000 7fea6c30c640 1 Processor -- start 2024-09-19T08:45:20.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.887+0000 7fea6c30c640 1 -- start start 2024-09-19T08:45:20.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.891+0000 7fea6c30c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fea500051d0 0x7fea64151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:20.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.891+0000 7fea6c30c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea64153340 0x7fea641515c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:20.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.891+0000 7fea6c30c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea64154f60 0x7fea641553e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:20.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.891+0000 7fea6c30c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fea640cfd70 con 0x7fea64154f60 2024-09-19T08:45:20.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.891+0000 7fea6c30c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fea6415cb40 con 0x7fea64153340 2024-09-19T08:45:20.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.891+0000 7fea6c30c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fea640cfef0 con 0x7fea500051d0 2024-09-19T08:45:20.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.891+0000 7fea6a8a9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea64154f60 0x7fea641553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:20.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.891+0000 7fea6a8a9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea64154f60 0x7fea641553e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38804/0 (socket says 172.21.15.98:38804) 2024-09-19T08:45:20.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.891+0000 7fea6a8a9640 1 -- 172.21.15.98:0/2332473251 learned_addr learned my addr 172.21.15.98:0/2332473251 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:20.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.891+0000 7fea6a0a8640 1 --2- 172.21.15.98:0/2332473251 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fea500051d0 0x7fea64151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:20.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.891+0000 7fea698a7640 1 --2- 172.21.15.98:0/2332473251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea64153340 0x7fea641515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:20.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.891+0000 7fea6a8a9640 1 -- 172.21.15.98:0/2332473251 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fea500051d0 msgr2=0x7fea64151e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:20.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.891+0000 7fea6a8a9640 1 --2- 172.21.15.98:0/2332473251 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fea500051d0 0x7fea64151e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:20.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.891+0000 7fea6a8a9640 1 -- 172.21.15.98:0/2332473251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea64153340 msgr2=0x7fea641515c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:20.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.891+0000 7fea6a8a9640 1 --2- 172.21.15.98:0/2332473251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea64153340 0x7fea641515c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:20.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.891+0000 7fea6a8a9640 1 -- 172.21.15.98:0/2332473251 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea6415da10 con 0x7fea64154f60 2024-09-19T08:45:20.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.891+0000 7fea6a8a9640 1 --2- 172.21.15.98:0/2332473251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea64154f60 0x7fea641553e0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fea5c01cc60 tx=0x7fea50005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:20.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.891+0000 7fea5affd640 1 -- 172.21.15.98:0/2332473251 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c001a90 con 0x7fea64154f60 2024-09-19T08:45:20.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.891+0000 7fea5affd640 1 -- 172.21.15.98:0/2332473251 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea5c005810 con 0x7fea64154f60 2024-09-19T08:45:20.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.891+0000 7fea6c30c640 1 -- 172.21.15.98:0/2332473251 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea6415dcf0 con 0x7fea64154f60 2024-09-19T08:45:20.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.891+0000 7fea5affd640 1 -- 172.21.15.98:0/2332473251 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea5c007dc0 con 0x7fea64154f60 2024-09-19T08:45:20.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.891+0000 7fea6c30c640 1 -- 172.21.15.98:0/2332473251 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea6415e2b0 con 0x7fea64154f60 2024-09-19T08:45:20.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.891+0000 7fea6c30c640 1 -- 172.21.15.98:0/2332473251 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea640cf3c0 con 0x7fea64154f60 2024-09-19T08:45:20.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.891+0000 7fea5affd640 1 -- 172.21.15.98:0/2332473251 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fea5c0a6480 con 0x7fea64154f60 2024-09-19T08:45:20.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.891+0000 7fea5affd640 1 --2- 172.21.15.98:0/2332473251 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fea48067e90 0x7fea4806a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:20.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.891+0000 7fea5affd640 1 -- 172.21.15.98:0/2332473251 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 4971+0+0 (secure 0 0 0) 0x7fea5c11fd30 con 0x7fea64154f60 2024-09-19T08:45:20.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.895+0000 7fea6a0a8640 1 --2- 172.21.15.98:0/2332473251 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fea48067e90 0x7fea4806a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:20.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.895+0000 7fea6a0a8640 1 --2- 172.21.15.98:0/2332473251 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fea48067e90 0x7fea4806a350 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fea60000c10 tx=0x7fea60009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:20.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:20.899+0000 7fea5affd640 1 -- 172.21.15.98:0/2332473251 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fea5c0b8060 con 0x7fea64154f60 2024-09-19T08:45:21.124 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.123+0000 7fea6c30c640 1 -- 172.21.15.98:0/2332473251 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]} v 0) -- 0x7fea640d0ce0 con 0x7fea64154f60 2024-09-19T08:45:21.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.403+0000 7fea5affd640 1 -- 172.21.15.98:0/2332473251 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]}]=0 v366) ==== 148+0+0 (secure 0 0 0) 0x7fea5c016490 con 0x7fea64154f60 2024-09-19T08:45:21.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.407+0000 7fea6c30c640 1 -- 172.21.15.98:0/2332473251 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fea48067e90 msgr2=0x7fea4806a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:21.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.407+0000 7fea6c30c640 1 --2- 172.21.15.98:0/2332473251 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fea48067e90 0x7fea4806a350 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fea60000c10 tx=0x7fea60009040 comp rx=0 tx=0).stop 2024-09-19T08:45:21.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.407+0000 7fea6c30c640 1 -- 172.21.15.98:0/2332473251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea64154f60 msgr2=0x7fea641553e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:21.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.407+0000 7fea6c30c640 1 --2- 172.21.15.98:0/2332473251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea64154f60 0x7fea641553e0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fea5c01cc60 tx=0x7fea50005840 comp rx=0 tx=0).stop 2024-09-19T08:45:21.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.407+0000 7fea6c30c640 1 -- 172.21.15.98:0/2332473251 shutdown_connections 2024-09-19T08:45:21.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.407+0000 7fea6c30c640 1 --2- 172.21.15.98:0/2332473251 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fea48067e90 0x7fea4806a350 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:21.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.407+0000 7fea6c30c640 1 --2- 172.21.15.98:0/2332473251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fea64154f60 0x7fea641553e0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:21.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.407+0000 7fea6c30c640 1 --2- 172.21.15.98:0/2332473251 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea64153340 0x7fea641515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:21.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.407+0000 7fea6c30c640 1 --2- 172.21.15.98:0/2332473251 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fea500051d0 0x7fea64151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:21.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.407+0000 7fea6c30c640 1 -- 172.21.15.98:0/2332473251 >> 172.21.15.98:0/2332473251 conn(0x7fea640b2010 msgr2=0x7fea640d0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:21.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.407+0000 7fea6c30c640 1 -- 172.21.15.98:0/2332473251 shutdown_connections 2024-09-19T08:45:21.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.407+0000 7fea6c30c640 1 -- 172.21.15.98:0/2332473251 wait complete. 2024-09-19T08:45:21.432 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2024-09-19T08:45:21.432 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_ops_on_nonexistent_vol (tasks.cephfs.test_volumes.TestMisc) 2024-09-19T08:45:21.433 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:45:21.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.555+0000 7f17a47d6640 1 Processor -- start 2024-09-19T08:45:21.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.555+0000 7f17a47d6640 1 -- start start 2024-09-19T08:45:21.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.555+0000 7f17a47d6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f179c06c3a0 0x7f179c0cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:21.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.555+0000 7f17a47d6640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f179c0cf900 0x7f179c0723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:21.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.555+0000 7f17a47d6640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f179c06d310 con 0x7f179c15c550 2024-09-19T08:45:21.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.555+0000 7f17a47d6640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f179c06d5f0 con 0x7f179c0cf900 2024-09-19T08:45:21.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.555+0000 7f17a47d6640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f179c06d8d0 con 0x7f179c06c3a0 2024-09-19T08:45:21.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.555+0000 7f17a2d73640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f179c0cf900 0x7f179c0723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:21.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.555+0000 7f17a2d73640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f179c0cf900 0x7f179c0723a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38808/0 (socket says 172.21.15.98:38808) 2024-09-19T08:45:21.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.555+0000 7f17a2d73640 1 -- 172.21.15.98:0/144866196 learned_addr learned my addr 172.21.15.98:0/144866196 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:21.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.555+0000 7f17a1d71640 1 --2- 172.21.15.98:0/144866196 >> v2:172.21.15.139:3300/0 conn(0x7f179c06c3a0 0x7f179c0cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:21.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.555+0000 7f17a2d73640 1 -- 172.21.15.98:0/144866196 >> v2:172.21.15.139:3300/0 conn(0x7f179c06c3a0 msgr2=0x7f179c0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:21.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.555+0000 7f17a2d73640 1 --2- 172.21.15.98:0/144866196 >> v2:172.21.15.139:3300/0 conn(0x7f179c06c3a0 0x7f179c0cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:21.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.555+0000 7f17a2d73640 1 -- 172.21.15.98:0/144866196 >> v1:172.21.15.139:6789/0 conn(0x7f179c15c550 legacy=0x7f179c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:21.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.555+0000 7f17a2d73640 1 -- 172.21.15.98:0/144866196 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f179c06dbb0 con 0x7f179c0cf900 2024-09-19T08:45:21.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f17a2d73640 1 --2- 172.21.15.98:0/144866196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f179c0cf900 0x7f179c0723a0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f179c0ccc60 tx=0x7f179401c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c778ff9293d0bf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:21.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f17a1570640 1 -- 172.21.15.98:0/144866196 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17940a5780 con 0x7f179c0cf900 2024-09-19T08:45:21.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f17a1570640 1 -- 172.21.15.98:0/144866196 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17940a5920 con 0x7f179c0cf900 2024-09-19T08:45:21.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f17a1570640 1 -- 172.21.15.98:0/144866196 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17940a5cf0 con 0x7f179c0cf900 2024-09-19T08:45:21.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f17a47d6640 1 -- 172.21.15.98:0/144866196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f179c0cf900 msgr2=0x7f179c0723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:21.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f17a47d6640 1 --2- 172.21.15.98:0/144866196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f179c0cf900 0x7f179c0723a0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f179c0ccc60 tx=0x7f179401c9b0 comp rx=0 tx=0).stop 2024-09-19T08:45:21.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f17a47d6640 1 -- 172.21.15.98:0/144866196 shutdown_connections 2024-09-19T08:45:21.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f17a47d6640 1 --2- 172.21.15.98:0/144866196 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f179c0cf900 0x7f179c0723a0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:21.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f17a47d6640 1 --2- 172.21.15.98:0/144866196 >> v2:172.21.15.139:3300/0 conn(0x7f179c06c3a0 0x7f179c0cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:21.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f17a47d6640 1 -- 172.21.15.98:0/144866196 >> 172.21.15.98:0/144866196 conn(0x7f179c0b2010 msgr2=0x7f179c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:21.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f17a47d6640 1 -- 172.21.15.98:0/144866196 shutdown_connections 2024-09-19T08:45:21.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f17a47d6640 1 -- 172.21.15.98:0/144866196 wait complete. 2024-09-19T08:45:21.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f17a47d6640 1 Processor -- start 2024-09-19T08:45:21.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f17a47d6640 1 -- start start 2024-09-19T08:45:21.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f17a47d6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f179c06c3a0 0x7f179c14a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:21.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f17a47d6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f179c0cf900 0x7f179c1575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:21.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f17a47d6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f179c15c550 0x7f179c14d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:21.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f17a47d6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f179c073480 con 0x7f179c0cf900 2024-09-19T08:45:21.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f17a47d6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f179c073300 con 0x7f179c15c550 2024-09-19T08:45:21.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f17a47d6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f179c073600 con 0x7f179c06c3a0 2024-09-19T08:45:21.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f17a1d71640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f179c0cf900 0x7f179c1575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:21.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f17a1d71640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f179c0cf900 0x7f179c1575e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38814/0 (socket says 172.21.15.98:38814) 2024-09-19T08:45:21.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f17a1d71640 1 -- 172.21.15.98:0/3389993025 learned_addr learned my addr 172.21.15.98:0/3389993025 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:21.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f17a1d71640 1 -- 172.21.15.98:0/3389993025 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f179c06c3a0 msgr2=0x7f179c14a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:21.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f17a2d73640 1 --2- 172.21.15.98:0/3389993025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f179c15c550 0x7f179c14d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:21.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f17a2572640 1 --2- 172.21.15.98:0/3389993025 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f179c06c3a0 0x7f179c14a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:21.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f17a1d71640 1 --2- 172.21.15.98:0/3389993025 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f179c06c3a0 0x7f179c14a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:21.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f17a1d71640 1 -- 172.21.15.98:0/3389993025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f179c15c550 msgr2=0x7f179c14d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:21.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f17a1d71640 1 --2- 172.21.15.98:0/3389993025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f179c15c550 0x7f179c14d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:21.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f17a1d71640 1 -- 172.21.15.98:0/3389993025 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f179c14aca0 con 0x7f179c0cf900 2024-09-19T08:45:21.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f17a2572640 1 --2- 172.21.15.98:0/3389993025 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f179c06c3a0 0x7f179c14a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:21.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f17a2d73640 1 --2- 172.21.15.98:0/3389993025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f179c15c550 0x7f179c14d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:21.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f17a1d71640 1 --2- 172.21.15.98:0/3389993025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f179c0cf900 0x7f179c1575e0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f179800c7f0 tx=0x7f179800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:21.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f1792ffd640 1 -- 172.21.15.98:0/3389993025 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1798015070 con 0x7f179c0cf900 2024-09-19T08:45:21.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f1792ffd640 1 -- 172.21.15.98:0/3389993025 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1798004240 con 0x7f179c0cf900 2024-09-19T08:45:21.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f17a47d6640 1 -- 172.21.15.98:0/3389993025 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f179c14af80 con 0x7f179c0cf900 2024-09-19T08:45:21.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f1792ffd640 1 -- 172.21.15.98:0/3389993025 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1798003620 con 0x7f179c0cf900 2024-09-19T08:45:21.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.559+0000 7f17a47d6640 1 -- 172.21.15.98:0/3389993025 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f179c14b540 con 0x7f179c0cf900 2024-09-19T08:45:21.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.563+0000 7f17a47d6640 1 -- 172.21.15.98:0/3389993025 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f179c0cf3c0 con 0x7f179c0cf900 2024-09-19T08:45:21.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.563+0000 7f1792ffd640 1 -- 172.21.15.98:0/3389993025 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f17980037c0 con 0x7f179c0cf900 2024-09-19T08:45:21.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.563+0000 7f1792ffd640 1 --2- 172.21.15.98:0/3389993025 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1780067e60 0x7f178006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:21.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.563+0000 7f1792ffd640 1 -- 172.21.15.98:0/3389993025 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 4971+0+0 (secure 0 0 0) 0x7f1798086600 con 0x7f179c0cf900 2024-09-19T08:45:21.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.563+0000 7f17a2572640 1 --2- 172.21.15.98:0/3389993025 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1780067e60 0x7f178006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:21.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.563+0000 7f17a2572640 1 --2- 172.21.15.98:0/3389993025 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1780067e60 0x7f178006a320 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f179c14c700 tx=0x7f178c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:21.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.567+0000 7f1792ffd640 1 -- 172.21.15.98:0/3389993025 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f1798056890 con 0x7f179c0cf900 2024-09-19T08:45:21.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.727+0000 7f17a47d6640 1 -- 172.21.15.98:0/3389993025 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f179c0d0af0 con 0x7f179c0cf900 2024-09-19T08:45:21.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.727+0000 7f1792ffd640 1 -- 172.21.15.98:0/3389993025 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) ==== 74+0+13084 (secure 0 0 0) 0x7f1798056190 con 0x7f179c0cf900 2024-09-19T08:45:21.730 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:45:21.731 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":126,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:45:20.405669+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.98:6835/502938874":"2024-09-20T08:45:17.389567+0000","172.21.15.98:6834/502938874":"2024-09-20T08:45:17.389567+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:45:21.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.731+0000 7f17a47d6640 1 -- 172.21.15.98:0/3389993025 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1780067e60 msgr2=0x7f178006a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:21.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.731+0000 7f17a47d6640 1 --2- 172.21.15.98:0/3389993025 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1780067e60 0x7f178006a320 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f179c14c700 tx=0x7f178c009040 comp rx=0 tx=0).stop 2024-09-19T08:45:21.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.731+0000 7f17a47d6640 1 -- 172.21.15.98:0/3389993025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f179c0cf900 msgr2=0x7f179c1575e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:21.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.731+0000 7f17a47d6640 1 --2- 172.21.15.98:0/3389993025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f179c0cf900 0x7f179c1575e0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f179800c7f0 tx=0x7f179800ccb0 comp rx=0 tx=0).stop 2024-09-19T08:45:21.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.731+0000 7f17a47d6640 1 -- 172.21.15.98:0/3389993025 shutdown_connections 2024-09-19T08:45:21.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.731+0000 7f17a47d6640 1 --2- 172.21.15.98:0/3389993025 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1780067e60 0x7f178006a320 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:21.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.731+0000 7f17a47d6640 1 --2- 172.21.15.98:0/3389993025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f179c15c550 0x7f179c14d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:21.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.731+0000 7f17a47d6640 1 --2- 172.21.15.98:0/3389993025 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f179c0cf900 0x7f179c1575e0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:21.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.731+0000 7f17a47d6640 1 --2- 172.21.15.98:0/3389993025 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f179c06c3a0 0x7f179c14a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:21.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.731+0000 7f17a47d6640 1 -- 172.21.15.98:0/3389993025 >> 172.21.15.98:0/3389993025 conn(0x7f179c0b2010 msgr2=0x7f179c06bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:21.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.731+0000 7f17a47d6640 1 -- 172.21.15.98:0/3389993025 shutdown_connections 2024-09-19T08:45:21.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.731+0000 7f17a47d6640 1 -- 172.21.15.98:0/3389993025 wait complete. 2024-09-19T08:45:21.751 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T08:38:08.105684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T08:45:21.751 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T08:45:21.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.867+0000 7f137d9a9640 1 Processor -- start 2024-09-19T08:45:21.868 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.867+0000 7f137d9a9640 1 -- start start 2024-09-19T08:45:21.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.867+0000 7f137d9a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1378070790 0x7f1378070b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:21.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.867+0000 7f137d9a9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f13780b1b70 0x7f13780d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:21.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.867+0000 7f137d9a9640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13780710b0 con 0x7f137815c5c0 2024-09-19T08:45:21.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.867+0000 7f137d9a9640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f1378073e40 con 0x7f13780b1b70 2024-09-19T08:45:21.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.867+0000 7f137d9a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1378074120 con 0x7f1378070790 2024-09-19T08:45:21.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.867+0000 7f1376ffd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1378070790 0x7f1378070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:21.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.867+0000 7f13777fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f13780b1b70 0x7f13780d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:21.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.867+0000 7f1376ffd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1378070790 0x7f1378070b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:37352/0 (socket says 172.21.15.98:37352) 2024-09-19T08:45:21.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.867+0000 7f1376ffd640 1 -- 172.21.15.98:0/3791694921 learned_addr learned my addr 172.21.15.98:0/3791694921 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:21.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.867+0000 7f13777fe640 1 -- 172.21.15.98:0/3791694921 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1378070790 msgr2=0x7f1378070b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:21.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.867+0000 7f13777fe640 1 --2- 172.21.15.98:0/3791694921 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1378070790 0x7f1378070b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:21.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.867+0000 7f13777fe640 1 -- 172.21.15.98:0/3791694921 >> v1:172.21.15.139:6789/0 conn(0x7f137815c5c0 legacy=0x7f13780b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:21.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.867+0000 7f13777fe640 1 -- 172.21.15.98:0/3791694921 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13780d0420 con 0x7f13780b1b70 2024-09-19T08:45:21.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.867+0000 7f1376ffd640 1 --2- 172.21.15.98:0/3791694921 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1378070790 0x7f1378070b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:45:21.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.871+0000 7f13777fe640 1 --2- 172.21.15.98:0/3791694921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13780b1b70 0x7f13780d0c20 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1370009d90 tx=0x7f137001c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=901475b31e26fe13 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:21.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.871+0000 7f13767fc640 1 -- 172.21.15.98:0/3791694921 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13700b2070 con 0x7f13780b1b70 2024-09-19T08:45:21.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.871+0000 7f13767fc640 1 -- 172.21.15.98:0/3791694921 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13700a59c0 con 0x7f13780b1b70 2024-09-19T08:45:21.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.871+0000 7f13767fc640 1 -- 172.21.15.98:0/3791694921 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13700a5d90 con 0x7f13780b1b70 2024-09-19T08:45:21.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.871+0000 7f137d9a9640 1 -- 172.21.15.98:0/3791694921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13780b1b70 msgr2=0x7f13780d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:21.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.871+0000 7f137d9a9640 1 --2- 172.21.15.98:0/3791694921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13780b1b70 0x7f13780d0c20 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1370009d90 tx=0x7f137001c940 comp rx=0 tx=0).stop 2024-09-19T08:45:21.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.871+0000 7f137d9a9640 1 -- 172.21.15.98:0/3791694921 shutdown_connections 2024-09-19T08:45:21.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.871+0000 7f137d9a9640 1 --2- 172.21.15.98:0/3791694921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13780b1b70 0x7f13780d0c20 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:21.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.871+0000 7f137d9a9640 1 --2- 172.21.15.98:0/3791694921 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1378070790 0x7f1378070b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:21.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.871+0000 7f137d9a9640 1 -- 172.21.15.98:0/3791694921 >> 172.21.15.98:0/3791694921 conn(0x7f13780d2330 msgr2=0x7f13780d2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:21.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.871+0000 7f137d9a9640 1 -- 172.21.15.98:0/3791694921 shutdown_connections 2024-09-19T08:45:21.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.871+0000 7f137d9a9640 1 -- 172.21.15.98:0/3791694921 wait complete. 2024-09-19T08:45:21.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.871+0000 7f137d9a9640 1 Processor -- start 2024-09-19T08:45:21.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.871+0000 7f137d9a9640 1 -- start start 2024-09-19T08:45:21.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.875+0000 7f137d9a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1378070790 0x7f1378155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:21.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.875+0000 7f137d9a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13780b1b70 0x7f137815f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:21.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.875+0000 7f137d9a9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13781f5690 0x7f1378150c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:21.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.875+0000 7f137d9a9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1378169080 con 0x7f13781f5690 2024-09-19T08:45:21.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.875+0000 7f137d9a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1378073be0 con 0x7f13780b1b70 2024-09-19T08:45:21.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.875+0000 7f137d9a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1378169200 con 0x7f1378070790 2024-09-19T08:45:21.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.875+0000 7f13777fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13781f5690 0x7f1378150c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:21.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.875+0000 7f13777fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13781f5690 0x7f1378150c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38816/0 (socket says 172.21.15.98:38816) 2024-09-19T08:45:21.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.875+0000 7f13777fe640 1 -- 172.21.15.98:0/226320606 learned_addr learned my addr 172.21.15.98:0/226320606 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:21.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.875+0000 7f1376ffd640 1 --2- 172.21.15.98:0/226320606 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1378070790 0x7f1378155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:21.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.875+0000 7f136ffff640 1 --2- 172.21.15.98:0/226320606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13780b1b70 0x7f137815f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:21.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.875+0000 7f13777fe640 1 -- 172.21.15.98:0/226320606 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1378070790 msgr2=0x7f1378155900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:21.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.875+0000 7f13777fe640 1 --2- 172.21.15.98:0/226320606 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1378070790 0x7f1378155900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:21.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.875+0000 7f13777fe640 1 -- 172.21.15.98:0/226320606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13780b1b70 msgr2=0x7f137815f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:21.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.875+0000 7f13777fe640 1 --2- 172.21.15.98:0/226320606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13780b1b70 0x7f137815f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:21.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.875+0000 7f13777fe640 1 -- 172.21.15.98:0/226320606 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1378155f00 con 0x7f13781f5690 2024-09-19T08:45:21.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.875+0000 7f13777fe640 1 --2- 172.21.15.98:0/226320606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13781f5690 0x7f1378150c00 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f13700b0040 tx=0x7f13700196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:21.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.875+0000 7f1374ff9640 1 -- 172.21.15.98:0/226320606 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13700b2050 con 0x7f13781f5690 2024-09-19T08:45:21.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.875+0000 7f1374ff9640 1 -- 172.21.15.98:0/226320606 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13700be070 con 0x7f13781f5690 2024-09-19T08:45:21.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.875+0000 7f1374ff9640 1 -- 172.21.15.98:0/226320606 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13700ada90 con 0x7f13781f5690 2024-09-19T08:45:21.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.875+0000 7f137d9a9640 1 -- 172.21.15.98:0/226320606 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1378152ae0 con 0x7f13781f5690 2024-09-19T08:45:21.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.875+0000 7f137d9a9640 1 -- 172.21.15.98:0/226320606 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13781530a0 con 0x7f13781f5690 2024-09-19T08:45:21.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.875+0000 7f137d9a9640 1 -- 172.21.15.98:0/226320606 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13781528f0 con 0x7f13781f5690 2024-09-19T08:45:21.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.875+0000 7f1374ff9640 1 -- 172.21.15.98:0/226320606 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f13700adc30 con 0x7f13781f5690 2024-09-19T08:45:21.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.875+0000 7f1374ff9640 1 --2- 172.21.15.98:0/226320606 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f135c067e60 0x7f135c06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:21.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.879+0000 7f1374ff9640 1 -- 172.21.15.98:0/226320606 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 4971+0+0 (secure 0 0 0) 0x7f1370121520 con 0x7f13781f5690 2024-09-19T08:45:21.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.879+0000 7f1376ffd640 1 --2- 172.21.15.98:0/226320606 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f135c067e60 0x7f135c06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:21.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.879+0000 7f1376ffd640 1 --2- 172.21.15.98:0/226320606 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f135c067e60 0x7f135c06a320 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1368006cb0 tx=0x7f1368009240 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:21.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:21.883+0000 7f1374ff9640 1 -- 172.21.15.98:0/226320606 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f1370016560 con 0x7f13781f5690 2024-09-19T08:45:22.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.035+0000 7f137d9a9640 1 -- 172.21.15.98:0/226320606 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1378070b70 con 0x7f13781f5690 2024-09-19T08:45:22.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.039+0000 7f1374ff9640 1 -- 172.21.15.98:0/226320606 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v126) ==== 93+0+10 (secure 0 0 0) 0x7f13700f1510 con 0x7f13781f5690 2024-09-19T08:45:22.040 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2024-09-19T08:45:22.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.039+0000 7f137d9a9640 1 -- 172.21.15.98:0/226320606 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f135c067e60 msgr2=0x7f135c06a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:22.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.039+0000 7f137d9a9640 1 --2- 172.21.15.98:0/226320606 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f135c067e60 0x7f135c06a320 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1368006cb0 tx=0x7f1368009240 comp rx=0 tx=0).stop 2024-09-19T08:45:22.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.039+0000 7f137d9a9640 1 -- 172.21.15.98:0/226320606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13781f5690 msgr2=0x7f1378150c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:22.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.039+0000 7f137d9a9640 1 --2- 172.21.15.98:0/226320606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13781f5690 0x7f1378150c00 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f13700b0040 tx=0x7f13700196e0 comp rx=0 tx=0).stop 2024-09-19T08:45:22.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.039+0000 7f137d9a9640 1 -- 172.21.15.98:0/226320606 shutdown_connections 2024-09-19T08:45:22.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.039+0000 7f137d9a9640 1 --2- 172.21.15.98:0/226320606 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f135c067e60 0x7f135c06a320 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:22.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.039+0000 7f137d9a9640 1 --2- 172.21.15.98:0/226320606 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13781f5690 0x7f1378150c00 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:22.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.039+0000 7f137d9a9640 1 --2- 172.21.15.98:0/226320606 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13780b1b70 0x7f137815f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:22.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.039+0000 7f137d9a9640 1 --2- 172.21.15.98:0/226320606 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1378070790 0x7f1378155900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:22.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.039+0000 7f137d9a9640 1 -- 172.21.15.98:0/226320606 >> 172.21.15.98:0/226320606 conn(0x7f13780d2330 msgr2=0x7f13780b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:22.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.039+0000 7f137d9a9640 1 -- 172.21.15.98:0/226320606 shutdown_connections 2024-09-19T08:45:22.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.039+0000 7f137d9a9640 1 -- 172.21.15.98:0/226320606 wait complete. 2024-09-19T08:45:22.058 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol' 2024-09-19T08:45:22.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.171+0000 7f9da36d3640 1 Processor -- start 2024-09-19T08:45:22.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.171+0000 7f9da36d3640 1 -- start start 2024-09-19T08:45:22.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.175+0000 7f9da36d3640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9d9c15c550 0x7f9d9c06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:22.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.175+0000 7f9da36d3640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9d9c06c1d0 0x7f9d9c0cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:22.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.175+0000 7f9da36d3640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d9c06d310 con 0x7f9d9c0cf900 2024-09-19T08:45:22.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.175+0000 7f9da36d3640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f9d9c06c610 con 0x7f9d9c06c1d0 2024-09-19T08:45:22.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.175+0000 7f9da36d3640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9d9c06d5f0 con 0x7f9d9c15c550 2024-09-19T08:45:22.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.175+0000 7f9da146f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9d9c15c550 0x7f9d9c06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:22.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.175+0000 7f9da0c6e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9d9c06c1d0 0x7f9d9c0cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:22.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.175+0000 7f9da0c6e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9d9c06c1d0 0x7f9d9c0cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:49530/0 (socket says 172.21.15.98:49530) 2024-09-19T08:45:22.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.175+0000 7f9da0c6e640 1 -- 172.21.15.98:0/125745069 learned_addr learned my addr 172.21.15.98:0/125745069 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:22.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.175+0000 7f9da1c70640 1 --1- 172.21.15.98:0/125745069 >> v1:172.21.15.139:6789/0 conn(0x7f9d9c0cf900 0x7f9d9c0ce1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:54386/0 (socket says 172.21.15.98:54386) 2024-09-19T08:45:22.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.175+0000 7f9da0c6e640 1 -- 172.21.15.98:0/125745069 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9d9c15c550 msgr2=0x7f9d9c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:22.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.175+0000 7f9da0c6e640 1 --2- 172.21.15.98:0/125745069 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9d9c15c550 0x7f9d9c06bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:22.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.175+0000 7f9da0c6e640 1 -- 172.21.15.98:0/125745069 >> v1:172.21.15.139:6789/0 conn(0x7f9d9c0cf900 legacy=0x7f9d9c0ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:22.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.175+0000 7f9da0c6e640 1 -- 172.21.15.98:0/125745069 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d9c06d8d0 con 0x7f9d9c06c1d0 2024-09-19T08:45:22.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.175+0000 7f9da0c6e640 1 --2- 172.21.15.98:0/125745069 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d9c06c1d0 0x7f9d9c0cf3c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c009870 tx=0x7f9d8c01c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eec5bdfc3563e10 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:22.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.175+0000 7f9d93fff640 1 -- 172.21.15.98:0/125745069 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c0b2070 con 0x7f9d9c06c1d0 2024-09-19T08:45:22.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.175+0000 7f9d93fff640 1 -- 172.21.15.98:0/125745069 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d8c0a9070 con 0x7f9d9c06c1d0 2024-09-19T08:45:22.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.175+0000 7f9d93fff640 1 -- 172.21.15.98:0/125745069 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c0ad880 con 0x7f9d9c06c1d0 2024-09-19T08:45:22.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.179+0000 7f9da36d3640 1 -- 172.21.15.98:0/125745069 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d9c06c1d0 msgr2=0x7f9d9c0cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:22.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.179+0000 7f9da36d3640 1 --2- 172.21.15.98:0/125745069 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d9c06c1d0 0x7f9d9c0cf3c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c009870 tx=0x7f9d8c01c560 comp rx=0 tx=0).stop 2024-09-19T08:45:22.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.179+0000 7f9da36d3640 1 -- 172.21.15.98:0/125745069 shutdown_connections 2024-09-19T08:45:22.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.179+0000 7f9da36d3640 1 --2- 172.21.15.98:0/125745069 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d9c06c1d0 0x7f9d9c0cf3c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:22.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.179+0000 7f9da36d3640 1 --2- 172.21.15.98:0/125745069 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9d9c15c550 0x7f9d9c06bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:22.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.179+0000 7f9da36d3640 1 -- 172.21.15.98:0/125745069 >> 172.21.15.98:0/125745069 conn(0x7f9d9c0b2010 msgr2=0x7f9d9c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:22.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.179+0000 7f9da36d3640 1 -- 172.21.15.98:0/125745069 shutdown_connections 2024-09-19T08:45:22.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.179+0000 7f9da36d3640 1 -- 172.21.15.98:0/125745069 wait complete. 2024-09-19T08:45:22.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.179+0000 7f9da36d3640 1 Processor -- start 2024-09-19T08:45:22.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.179+0000 7f9da36d3640 1 -- start start 2024-09-19T08:45:22.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.179+0000 7f9da36d3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d9c06c1d0 0x7f9d9c15ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:22.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.179+0000 7f9da36d3640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9d9c0cf900 0x7f9d9c136990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:22.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.179+0000 7f9da36d3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d9c15c550 0x7f9d9c136ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:22.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.179+0000 7f9da36d3640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c169230 con 0x7f9d9c06c1d0 2024-09-19T08:45:22.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.179+0000 7f9da36d3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c15cb40 con 0x7f9d9c15c550 2024-09-19T08:45:22.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.179+0000 7f9da36d3640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9d9c1693b0 con 0x7f9d9c0cf900 2024-09-19T08:45:22.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.179+0000 7f9da146f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d9c06c1d0 0x7f9d9c15ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:22.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.179+0000 7f9da146f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d9c06c1d0 0x7f9d9c15ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38830/0 (socket says 172.21.15.98:38830) 2024-09-19T08:45:22.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.179+0000 7f9da146f640 1 -- 172.21.15.98:0/3860546774 learned_addr learned my addr 172.21.15.98:0/3860546774 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:22.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.179+0000 7f9da1c70640 1 --2- 172.21.15.98:0/3860546774 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d9c15c550 0x7f9d9c136ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:22.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.179+0000 7f9da0c6e640 1 --2- 172.21.15.98:0/3860546774 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9d9c0cf900 0x7f9d9c136990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:22.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.179+0000 7f9da146f640 1 -- 172.21.15.98:0/3860546774 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9d9c0cf900 msgr2=0x7f9d9c136990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:22.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.179+0000 7f9da146f640 1 --2- 172.21.15.98:0/3860546774 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9d9c0cf900 0x7f9d9c136990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:22.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.179+0000 7f9da146f640 1 -- 172.21.15.98:0/3860546774 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d9c15c550 msgr2=0x7f9d9c136ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:22.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.179+0000 7f9da146f640 1 --2- 172.21.15.98:0/3860546774 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d9c15c550 0x7f9d9c136ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:22.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.179+0000 7f9da146f640 1 -- 172.21.15.98:0/3860546774 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d9c154330 con 0x7f9d9c06c1d0 2024-09-19T08:45:22.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.179+0000 7f9da146f640 1 --2- 172.21.15.98:0/3860546774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d9c06c1d0 0x7f9d9c15ccc0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f9d9800efc0 tx=0x7f9d9800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:22.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.179+0000 7f9d91ffb640 1 -- 172.21.15.98:0/3860546774 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d98018020 con 0x7f9d9c06c1d0 2024-09-19T08:45:22.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.179+0000 7f9d91ffb640 1 -- 172.21.15.98:0/3860546774 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d98014480 con 0x7f9d9c06c1d0 2024-09-19T08:45:22.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.179+0000 7f9d91ffb640 1 -- 172.21.15.98:0/3860546774 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d98014850 con 0x7f9d9c06c1d0 2024-09-19T08:45:22.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.179+0000 7f9da36d3640 1 -- 172.21.15.98:0/3860546774 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d9c154610 con 0x7f9d9c06c1d0 2024-09-19T08:45:22.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.179+0000 7f9da36d3640 1 -- 172.21.15.98:0/3860546774 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d9c154bd0 con 0x7f9d9c06c1d0 2024-09-19T08:45:22.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.183+0000 7f9da36d3640 1 -- 172.21.15.98:0/3860546774 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d9c0cf3c0 con 0x7f9d9c06c1d0 2024-09-19T08:45:22.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.183+0000 7f9d91ffb640 1 -- 172.21.15.98:0/3860546774 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f9d9801b020 con 0x7f9d9c06c1d0 2024-09-19T08:45:22.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.183+0000 7f9d91ffb640 1 --2- 172.21.15.98:0/3860546774 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9d80067e60 0x7f9d8006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:22.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.183+0000 7f9d91ffb640 1 -- 172.21.15.98:0/3860546774 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 4971+0+0 (secure 0 0 0) 0x7f9d98087010 con 0x7f9d9c06c1d0 2024-09-19T08:45:22.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.183+0000 7f9da0c6e640 1 --2- 172.21.15.98:0/3860546774 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9d80067e60 0x7f9d8006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:22.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.183+0000 7f9da0c6e640 1 --2- 172.21.15.98:0/3860546774 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9d80067e60 0x7f9d8006a320 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0af000 tx=0x7f9d8c0a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:22.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.187+0000 7f9d91ffb640 1 -- 172.21.15.98:0/3860546774 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9d98022070 con 0x7f9d9c06c1d0 2024-09-19T08:45:22.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.391+0000 7f9da36d3640 1 -- 172.21.15.98:0/3860546774 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]} v 0) -- 0x7f9d9c0b0290 con 0x7f9d9c06c1d0 2024-09-19T08:45:22.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.407+0000 7f9d91ffb640 1 -- 172.21.15.98:0/3860546774 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]}]=0 v367) ==== 154+0+0 (secure 0 0 0) 0x7f9d98010040 con 0x7f9d9c06c1d0 2024-09-19T08:45:22.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.407+0000 7f9da36d3640 1 -- 172.21.15.98:0/3860546774 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9d80067e60 msgr2=0x7f9d8006a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:22.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.407+0000 7f9da36d3640 1 --2- 172.21.15.98:0/3860546774 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9d80067e60 0x7f9d8006a320 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0af000 tx=0x7f9d8c0a7000 comp rx=0 tx=0).stop 2024-09-19T08:45:22.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.407+0000 7f9da36d3640 1 -- 172.21.15.98:0/3860546774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d9c06c1d0 msgr2=0x7f9d9c15ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:22.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.407+0000 7f9da36d3640 1 --2- 172.21.15.98:0/3860546774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d9c06c1d0 0x7f9d9c15ccc0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f9d9800efc0 tx=0x7f9d9800c490 comp rx=0 tx=0).stop 2024-09-19T08:45:22.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.407+0000 7f9da36d3640 1 -- 172.21.15.98:0/3860546774 shutdown_connections 2024-09-19T08:45:22.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.407+0000 7f9da36d3640 1 --2- 172.21.15.98:0/3860546774 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9d80067e60 0x7f9d8006a320 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:22.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.407+0000 7f9da36d3640 1 --2- 172.21.15.98:0/3860546774 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d9c15c550 0x7f9d9c136ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:22.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.407+0000 7f9da36d3640 1 --2- 172.21.15.98:0/3860546774 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9d9c0cf900 0x7f9d9c136990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:22.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.407+0000 7f9da36d3640 1 --2- 172.21.15.98:0/3860546774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9d9c06c1d0 0x7f9d9c15ccc0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:22.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.407+0000 7f9da36d3640 1 -- 172.21.15.98:0/3860546774 >> 172.21.15.98:0/3860546774 conn(0x7f9d9c0b2010 msgr2=0x7f9d9c15a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:22.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.407+0000 7f9da36d3640 1 -- 172.21.15.98:0/3860546774 shutdown_connections 2024-09-19T08:45:22.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.407+0000 7f9da36d3640 1 -- 172.21.15.98:0/3860546774 wait complete. 2024-09-19T08:45:22.426 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T08:45:22.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.535+0000 7faab627b640 1 Processor -- start 2024-09-19T08:45:22.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.535+0000 7faab627b640 1 -- start start 2024-09-19T08:45:22.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.539+0000 7faab627b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7faab00b4420 0x7faab0071ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:22.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.539+0000 7faab627b640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faab0073530 con 0x7faab006c3a0 2024-09-19T08:45:22.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.539+0000 7faab627b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faab0073810 con 0x7faab0145b70 2024-09-19T08:45:22.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.539+0000 7faab627b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7faab0073af0 con 0x7faab00b4420 2024-09-19T08:45:22.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.539+0000 7faaaf7fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7faab006c3a0 0x7faab00713b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:35814/0 (socket says 172.21.15.98:35814) 2024-09-19T08:45:22.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.539+0000 7faaaf7fe640 1 -- 172.21.15.98:0/3478643325 learned_addr learned my addr 172.21.15.98:0/3478643325 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:22.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.539+0000 7faab4818640 1 --2- 172.21.15.98:0/3478643325 >> v2:172.21.15.139:3300/0 conn(0x7faab00b4420 0x7faab0071ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:22.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.539+0000 7faaaeffd640 1 -- 172.21.15.98:0/3478643325 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 956528793 0 0) 0x7faab0073530 con 0x7faab006c3a0 2024-09-19T08:45:22.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.539+0000 7faaaeffd640 1 -- 172.21.15.98:0/3478643325 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa98003530 con 0x7faab006c3a0 2024-09-19T08:45:22.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.539+0000 7faaaeffd640 1 -- 172.21.15.98:0/3478643325 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1150201971 0 0) 0x7faa98003530 con 0x7faab006c3a0 2024-09-19T08:45:22.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.539+0000 7faaaeffd640 1 -- 172.21.15.98:0/3478643325 >> v2:172.21.15.139:3300/0 conn(0x7faab00b4420 msgr2=0x7faab0071ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:22.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.539+0000 7faaaeffd640 1 --2- 172.21.15.98:0/3478643325 >> v2:172.21.15.139:3300/0 conn(0x7faab00b4420 0x7faab0071ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:22.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.539+0000 7faaaeffd640 1 -- 172.21.15.98:0/3478643325 >> v1:172.21.15.139:6789/0 conn(0x7faab0145b70 legacy=0x7faab006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:22.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.539+0000 7faaaeffd640 1 -- 172.21.15.98:0/3478643325 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab0073dd0 con 0x7faab006c3a0 2024-09-19T08:45:22.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.539+0000 7faaaeffd640 1 -- 172.21.15.98:0/3478643325 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7faaa4001680 con 0x7faab006c3a0 2024-09-19T08:45:22.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.539+0000 7faaaeffd640 1 -- 172.21.15.98:0/3478643325 >> v1:172.21.15.98:6789/0 conn(0x7faab006c3a0 legacy=0x7faab00713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:22.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.539+0000 7faaaeffd640 1 --2- 172.21.15.98:0/3478643325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa980039c0 0x7faa98003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:22.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.539+0000 7faaaeffd640 1 --2- 172.21.15.98:0/3478643325 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa98004810 0x7faa98004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:22.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.539+0000 7faaaeffd640 1 --2- 172.21.15.98:0/3478643325 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa980051d0 0x7faa9800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:22.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.539+0000 7faaaeffd640 1 -- 172.21.15.98:0/3478643325 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faab0073af0 con 0x7faa980039c0 2024-09-19T08:45:22.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.539+0000 7faaaeffd640 1 -- 172.21.15.98:0/3478643325 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faa9800c170 con 0x7faa980051d0 2024-09-19T08:45:22.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.539+0000 7faaaeffd640 1 -- 172.21.15.98:0/3478643325 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7faa9800c370 con 0x7faa98004810 2024-09-19T08:45:22.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.539+0000 7faaaffff640 1 --2- 172.21.15.98:0/3478643325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa980039c0 0x7faa98003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:22.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.539+0000 7faaaffff640 1 -- 172.21.15.98:0/3478643325 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa98004810 msgr2=0x7faa98004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:22.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.539+0000 7faaaffff640 1 --2- 172.21.15.98:0/3478643325 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faa98004810 0x7faa98004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:22.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.539+0000 7faaaffff640 1 -- 172.21.15.98:0/3478643325 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa980051d0 msgr2=0x7faa9800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:22.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.539+0000 7faaaffff640 1 --2- 172.21.15.98:0/3478643325 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faa980051d0 0x7faa9800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:22.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.539+0000 7faaaffff640 1 -- 172.21.15.98:0/3478643325 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa98003530 con 0x7faa980039c0 2024-09-19T08:45:22.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.539+0000 7faab4818640 1 -- 172.21.15.98:0/3478643325 reap_dead start 2024-09-19T08:45:22.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.543+0000 7faaaffff640 1 --2- 172.21.15.98:0/3478643325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa980039c0 0x7faa98003de0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7faaa0001780 tx=0x7faaa001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f44d5faa3655b384 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:22.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.543+0000 7faaaeffd640 1 -- 172.21.15.98:0/3478643325 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa00013a0 con 0x7faa980039c0 2024-09-19T08:45:22.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.543+0000 7faaaeffd640 1 -- 172.21.15.98:0/3478643325 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaa00052f0 con 0x7faa980039c0 2024-09-19T08:45:22.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.543+0000 7faaaeffd640 1 -- 172.21.15.98:0/3478643325 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa0003840 con 0x7faa980039c0 2024-09-19T08:45:22.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.543+0000 7faab627b640 1 -- 172.21.15.98:0/3478643325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa980039c0 msgr2=0x7faa98003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:22.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.543+0000 7faab627b640 1 --2- 172.21.15.98:0/3478643325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa980039c0 0x7faa98003de0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7faaa0001780 tx=0x7faaa001c420 comp rx=0 tx=0).stop 2024-09-19T08:45:22.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.543+0000 7faab627b640 1 -- 172.21.15.98:0/3478643325 shutdown_connections 2024-09-19T08:45:22.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.543+0000 7faab627b640 1 --2- 172.21.15.98:0/3478643325 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa980039c0 0x7faa98003de0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:22.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.543+0000 7faab627b640 1 -- 172.21.15.98:0/3478643325 >> 172.21.15.98:0/3478643325 conn(0x7faab00b2010 msgr2=0x7faab00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:22.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.543+0000 7faab627b640 1 -- 172.21.15.98:0/3478643325 shutdown_connections 2024-09-19T08:45:22.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.543+0000 7faab627b640 1 -- 172.21.15.98:0/3478643325 wait complete. 2024-09-19T08:45:22.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.543+0000 7faab627b640 1 Processor -- start 2024-09-19T08:45:22.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.543+0000 7faab627b640 1 -- start start 2024-09-19T08:45:22.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.543+0000 7faab627b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa980039c0 0x7faab015bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:22.548 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.543+0000 7faab627b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faab015f190 0x7faab00aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:22.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.543+0000 7faab627b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faab00aafb0 0x7faab01421c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:22.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.543+0000 7faab627b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7faab00b3410 con 0x7faa980039c0 2024-09-19T08:45:22.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.543+0000 7faab627b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faab0071dd0 con 0x7faab00aafb0 2024-09-19T08:45:22.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.543+0000 7faab627b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7faab00b3590 con 0x7faab015f190 2024-09-19T08:45:22.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.543+0000 7faaaffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa980039c0 0x7faab015bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:22.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.543+0000 7faaaffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa980039c0 0x7faab015bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38850/0 (socket says 172.21.15.98:38850) 2024-09-19T08:45:22.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.543+0000 7faaaffff640 1 -- 172.21.15.98:0/3886413199 learned_addr learned my addr 172.21.15.98:0/3886413199 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:22.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.543+0000 7faaaf7fe640 1 --2- 172.21.15.98:0/3886413199 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faab015f190 0x7faab00aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:22.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.543+0000 7faaaffff640 1 -- 172.21.15.98:0/3886413199 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faab015f190 msgr2=0x7faab00aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:22.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.543+0000 7faaaffff640 1 --2- 172.21.15.98:0/3886413199 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faab015f190 0x7faab00aaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:22.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.543+0000 7faaaffff640 1 -- 172.21.15.98:0/3886413199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faab00aafb0 msgr2=0x7faab01421c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:22.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.543+0000 7faaaffff640 1 --2- 172.21.15.98:0/3886413199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faab00aafb0 0x7faab01421c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:22.549 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.543+0000 7faaaffff640 1 -- 172.21.15.98:0/3886413199 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab0160210 con 0x7faa980039c0 2024-09-19T08:45:22.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.543+0000 7faaaffff640 1 --2- 172.21.15.98:0/3886413199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa980039c0 0x7faab015bd80 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7faaa001c250 tx=0x7faa98005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:22.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.547+0000 7faaacff9640 1 -- 172.21.15.98:0/3886413199 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa0007a40 con 0x7faa980039c0 2024-09-19T08:45:22.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.547+0000 7faaacff9640 1 -- 172.21.15.98:0/3886413199 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaa0000ee0 con 0x7faa980039c0 2024-09-19T08:45:22.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.547+0000 7faaacff9640 1 -- 172.21.15.98:0/3886413199 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa00054a0 con 0x7faa980039c0 2024-09-19T08:45:22.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.547+0000 7faab627b640 1 -- 172.21.15.98:0/3886413199 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faab015f690 con 0x7faa980039c0 2024-09-19T08:45:22.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.547+0000 7faab627b640 1 -- 172.21.15.98:0/3886413199 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faab015fc50 con 0x7faa980039c0 2024-09-19T08:45:22.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.547+0000 7faab627b640 1 -- 172.21.15.98:0/3886413199 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faab00b3920 con 0x7faa980039c0 2024-09-19T08:45:22.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.547+0000 7faaacff9640 1 -- 172.21.15.98:0/3886413199 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7faaa00bd020 con 0x7faa980039c0 2024-09-19T08:45:22.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.547+0000 7faaacff9640 1 --2- 172.21.15.98:0/3886413199 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7faa9c067e60 0x7faa9c06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:22.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.547+0000 7faaacff9640 1 -- 172.21.15.98:0/3886413199 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 4971+0+0 (secure 0 0 0) 0x7faaa0120070 con 0x7faa980039c0 2024-09-19T08:45:22.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.547+0000 7faaaf7fe640 1 --2- 172.21.15.98:0/3886413199 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7faa9c067e60 0x7faa9c06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:22.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.547+0000 7faaaf7fe640 1 --2- 172.21.15.98:0/3886413199 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7faa9c067e60 0x7faa9c06a320 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7faaa40011f0 tx=0x7faaa4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:22.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.551+0000 7faaacff9640 1 -- 172.21.15.98:0/3886413199 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7faaa0016130 con 0x7faa980039c0 2024-09-19T08:45:22.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.711+0000 7faab627b640 1 -- 172.21.15.98:0/3886413199 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7faab00757a0 con 0x7faa980039c0 2024-09-19T08:45:22.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.719+0000 7faaacff9640 1 -- 172.21.15.98:0/3886413199 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7faaa00f0630 con 0x7faa980039c0 2024-09-19T08:45:22.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.719+0000 7faab627b640 1 -- 172.21.15.98:0/3886413199 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7faa9c067e60 msgr2=0x7faa9c06a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:22.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.719+0000 7faab627b640 1 --2- 172.21.15.98:0/3886413199 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7faa9c067e60 0x7faa9c06a320 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7faaa40011f0 tx=0x7faaa4009040 comp rx=0 tx=0).stop 2024-09-19T08:45:22.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.719+0000 7faab627b640 1 -- 172.21.15.98:0/3886413199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa980039c0 msgr2=0x7faab015bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:22.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.719+0000 7faab627b640 1 --2- 172.21.15.98:0/3886413199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa980039c0 0x7faab015bd80 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7faaa001c250 tx=0x7faa98005840 comp rx=0 tx=0).stop 2024-09-19T08:45:22.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.719+0000 7faab627b640 1 -- 172.21.15.98:0/3886413199 shutdown_connections 2024-09-19T08:45:22.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.719+0000 7faab627b640 1 --2- 172.21.15.98:0/3886413199 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7faa9c067e60 0x7faa9c06a320 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:22.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.719+0000 7faab627b640 1 --2- 172.21.15.98:0/3886413199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faab00aafb0 0x7faab01421c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:22.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.719+0000 7faab627b640 1 --2- 172.21.15.98:0/3886413199 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7faab015f190 0x7faab00aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:22.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.719+0000 7faab627b640 1 --2- 172.21.15.98:0/3886413199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7faa980039c0 0x7faab015bd80 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:22.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.719+0000 7faab627b640 1 -- 172.21.15.98:0/3886413199 >> 172.21.15.98:0/3886413199 conn(0x7faab00b2010 msgr2=0x7faab00b4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:22.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.719+0000 7faab627b640 1 -- 172.21.15.98:0/3886413199 shutdown_connections 2024-09-19T08:45:22.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:22.719+0000 7faab627b640 1 -- 172.21.15.98:0/3886413199 wait complete. 2024-09-19T08:45:22.738 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T08:45:22.738 DEBUG:teuthology.orchestra.run.smithi098:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:45:22.830 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:45:22.897 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T08:45:22.897 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:45:22.897 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:45:22.903 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15846 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T08:45:22.903 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:45:22.903 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:45:22.909 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T08:45:22.909 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:45:22.909 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:45:22.949 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15846 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T08:45:22.949 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:45:22.949 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:45:22.959 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T08:45:22.960 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-19T08:45:22.960 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:45:23.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.115+0000 7ff739bba640 1 Processor -- start 2024-09-19T08:45:23.117 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.115+0000 7ff739bba640 1 -- start start 2024-09-19T08:45:23.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.115+0000 7ff739bba640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff73415c550 0x7ff73406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:23.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.115+0000 7ff739bba640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff73406c1d0 0x7ff7340cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:23.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.115+0000 7ff739bba640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff73406d310 con 0x7ff7340cf900 2024-09-19T08:45:23.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.115+0000 7ff739bba640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7ff73406c610 con 0x7ff73415c550 2024-09-19T08:45:23.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.115+0000 7ff739bba640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff73406d5f0 con 0x7ff73406c1d0 2024-09-19T08:45:23.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.115+0000 7ff7337fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff73415c550 0x7ff73406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:23.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.115+0000 7ff7337fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff73415c550 0x7ff73406bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38858/0 (socket says 172.21.15.98:38858) 2024-09-19T08:45:23.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.115+0000 7ff7337fe640 1 -- 172.21.15.98:0/2540725930 learned_addr learned my addr 172.21.15.98:0/2540725930 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:23.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.115+0000 7ff72bfff640 1 --2- 172.21.15.98:0/2540725930 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff73406c1d0 0x7ff7340cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:23.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.115+0000 7ff7337fe640 1 -- 172.21.15.98:0/2540725930 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff73406c1d0 msgr2=0x7ff7340cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:23.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.115+0000 7ff7337fe640 1 --2- 172.21.15.98:0/2540725930 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff73406c1d0 0x7ff7340cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:23.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.115+0000 7ff7337fe640 1 -- 172.21.15.98:0/2540725930 >> v1:172.21.15.139:6789/0 conn(0x7ff7340cf900 legacy=0x7ff7340ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:23.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.115+0000 7ff7337fe640 1 -- 172.21.15.98:0/2540725930 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff73406d8d0 con 0x7ff73415c550 2024-09-19T08:45:23.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.119+0000 7ff7337fe640 1 --2- 172.21.15.98:0/2540725930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff73415c550 0x7ff73406bc90 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7ff724009870 tx=0x7ff72401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae62f1710a295dbb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:23.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.119+0000 7ff732ffd640 1 -- 172.21.15.98:0/2540725930 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7240b0070 con 0x7ff73415c550 2024-09-19T08:45:23.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.119+0000 7ff732ffd640 1 -- 172.21.15.98:0/2540725930 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7240ab600 con 0x7ff73415c550 2024-09-19T08:45:23.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.119+0000 7ff732ffd640 1 -- 172.21.15.98:0/2540725930 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7240ab9d0 con 0x7ff73415c550 2024-09-19T08:45:23.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.119+0000 7ff739bba640 1 -- 172.21.15.98:0/2540725930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff73415c550 msgr2=0x7ff73406bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:23.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.119+0000 7ff739bba640 1 --2- 172.21.15.98:0/2540725930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff73415c550 0x7ff73406bc90 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7ff724009870 tx=0x7ff72401c520 comp rx=0 tx=0).stop 2024-09-19T08:45:23.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.119+0000 7ff739bba640 1 -- 172.21.15.98:0/2540725930 shutdown_connections 2024-09-19T08:45:23.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.119+0000 7ff739bba640 1 --2- 172.21.15.98:0/2540725930 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff73406c1d0 0x7ff7340cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:23.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.119+0000 7ff739bba640 1 --2- 172.21.15.98:0/2540725930 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff73415c550 0x7ff73406bc90 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:23.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.119+0000 7ff739bba640 1 -- 172.21.15.98:0/2540725930 >> 172.21.15.98:0/2540725930 conn(0x7ff7340b2010 msgr2=0x7ff7340b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:23.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.119+0000 7ff739bba640 1 -- 172.21.15.98:0/2540725930 shutdown_connections 2024-09-19T08:45:23.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.119+0000 7ff739bba640 1 -- 172.21.15.98:0/2540725930 wait complete. 2024-09-19T08:45:23.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.123+0000 7ff739bba640 1 Processor -- start 2024-09-19T08:45:23.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.123+0000 7ff739bba640 1 -- start start 2024-09-19T08:45:23.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.123+0000 7ff739bba640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff73406c1d0 0x7ff734149900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:23.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.123+0000 7ff739bba640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7340cf900 0x7ff7341f6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:23.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.123+0000 7ff739bba640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff73415c550 0x7ff7340d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:23.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.123+0000 7ff739bba640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff734169230 con 0x7ff73406c1d0 2024-09-19T08:45:23.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.123+0000 7ff739bba640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff73415cb40 con 0x7ff73415c550 2024-09-19T08:45:23.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.123+0000 7ff739bba640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff7341693b0 con 0x7ff7340cf900 2024-09-19T08:45:23.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.123+0000 7ff7337fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff73406c1d0 0x7ff734149900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:23.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.123+0000 7ff7337fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff73406c1d0 0x7ff734149900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38862/0 (socket says 172.21.15.98:38862) 2024-09-19T08:45:23.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.123+0000 7ff7337fe640 1 -- 172.21.15.98:0/1183179772 learned_addr learned my addr 172.21.15.98:0/1183179772 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:23.130 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.123+0000 7ff72bfff640 1 --2- 172.21.15.98:0/1183179772 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7340cf900 0x7ff7341f6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:23.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.123+0000 7ff733fff640 1 --2- 172.21.15.98:0/1183179772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff73415c550 0x7ff7340d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:23.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.123+0000 7ff7337fe640 1 -- 172.21.15.98:0/1183179772 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7340cf900 msgr2=0x7ff7341f6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:23.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.123+0000 7ff7337fe640 1 --2- 172.21.15.98:0/1183179772 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7340cf900 0x7ff7341f6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:23.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.123+0000 7ff7337fe640 1 -- 172.21.15.98:0/1183179772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff73415c550 msgr2=0x7ff7340d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:23.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.123+0000 7ff7337fe640 1 --2- 172.21.15.98:0/1183179772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff73415c550 0x7ff7340d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:23.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.123+0000 7ff7337fe640 1 -- 172.21.15.98:0/1183179772 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff73414a210 con 0x7ff73406c1d0 2024-09-19T08:45:23.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.123+0000 7ff72bfff640 1 --2- 172.21.15.98:0/1183179772 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7340cf900 0x7ff7341f6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:23.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.123+0000 7ff7337fe640 1 --2- 172.21.15.98:0/1183179772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff73406c1d0 0x7ff734149900 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7ff7240ae040 tx=0x7ff72401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:23.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.123+0000 7ff730ff9640 1 -- 172.21.15.98:0/1183179772 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7240ba020 con 0x7ff73406c1d0 2024-09-19T08:45:23.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.123+0000 7ff730ff9640 1 -- 172.21.15.98:0/1183179772 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7240b6420 con 0x7ff73406c1d0 2024-09-19T08:45:23.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.123+0000 7ff730ff9640 1 -- 172.21.15.98:0/1183179772 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7240b0040 con 0x7ff73406c1d0 2024-09-19T08:45:23.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.123+0000 7ff739bba640 1 -- 172.21.15.98:0/1183179772 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff734155380 con 0x7ff73406c1d0 2024-09-19T08:45:23.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.123+0000 7ff739bba640 1 -- 172.21.15.98:0/1183179772 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff734155940 con 0x7ff73406c1d0 2024-09-19T08:45:23.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.123+0000 7ff739bba640 1 -- 172.21.15.98:0/1183179772 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7340cf3c0 con 0x7ff73406c1d0 2024-09-19T08:45:23.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.123+0000 7ff730ff9640 1 -- 172.21.15.98:0/1183179772 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7ff7240bd020 con 0x7ff73406c1d0 2024-09-19T08:45:23.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.127+0000 7ff730ff9640 1 --2- 172.21.15.98:0/1183179772 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff714067e60 0x7ff71406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:23.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.127+0000 7ff730ff9640 1 -- 172.21.15.98:0/1183179772 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 4971+0+0 (secure 0 0 0) 0x7ff724120600 con 0x7ff73406c1d0 2024-09-19T08:45:23.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.127+0000 7ff72bfff640 1 --2- 172.21.15.98:0/1183179772 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff714067e60 0x7ff71406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:23.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.127+0000 7ff72bfff640 1 --2- 172.21.15.98:0/1183179772 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff714067e60 0x7ff71406a320 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff734153a40 tx=0x7ff71c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:23.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.131+0000 7ff730ff9640 1 -- 172.21.15.98:0/1183179772 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff7240c4030 con 0x7ff73406c1d0 2024-09-19T08:45:23.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.319+0000 7ff739bba640 1 -- 172.21.15.98:0/1183179772 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff73415a670 con 0x7ff73406c1d0 2024-09-19T08:45:23.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.319+0000 7ff730ff9640 1 -- 172.21.15.98:0/1183179772 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+3917 (secure 0 0 0) 0x7ff724016110 con 0x7ff73406c1d0 2024-09-19T08:45:23.323 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:45:23.324 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":70,"btime":"2024-09-19T08:45:18:392463+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5645,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/1334032829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":1334032829},{"type":"v1","addr":"172.21.15.139:6833","nonce":1334032829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":5651,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/2541341035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":2541341035},{"type":"v1","addr":"172.21.15.139:6835","nonce":2541341035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":7521,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/3632520818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3632520818},{"type":"v1","addr":"172.21.15.98:6833","nonce":3632520818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":7839,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/4139313141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":4139313141},{"type":"v1","addr":"172.21.15.98:6835","nonce":4139313141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[]} 2024-09-19T08:45:23.324 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 70 2024-09-19T08:45:23.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.323+0000 7ff739bba640 1 -- 172.21.15.98:0/1183179772 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff714067e60 msgr2=0x7ff71406a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:23.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.323+0000 7ff739bba640 1 --2- 172.21.15.98:0/1183179772 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff714067e60 0x7ff71406a320 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff734153a40 tx=0x7ff71c008040 comp rx=0 tx=0).stop 2024-09-19T08:45:23.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.323+0000 7ff739bba640 1 -- 172.21.15.98:0/1183179772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff73406c1d0 msgr2=0x7ff734149900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:23.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.323+0000 7ff739bba640 1 --2- 172.21.15.98:0/1183179772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff73406c1d0 0x7ff734149900 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7ff7240ae040 tx=0x7ff72401ad60 comp rx=0 tx=0).stop 2024-09-19T08:45:23.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.323+0000 7ff739bba640 1 -- 172.21.15.98:0/1183179772 shutdown_connections 2024-09-19T08:45:23.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.323+0000 7ff739bba640 1 --2- 172.21.15.98:0/1183179772 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff714067e60 0x7ff71406a320 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:23.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.323+0000 7ff739bba640 1 --2- 172.21.15.98:0/1183179772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff73415c550 0x7ff7340d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:23.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.323+0000 7ff739bba640 1 --2- 172.21.15.98:0/1183179772 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff7340cf900 0x7ff7341f6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:23.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.323+0000 7ff739bba640 1 --2- 172.21.15.98:0/1183179772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff73406c1d0 0x7ff734149900 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:23.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.323+0000 7ff739bba640 1 -- 172.21.15.98:0/1183179772 >> 172.21.15.98:0/1183179772 conn(0x7ff7340b2010 msgr2=0x7ff7340afd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:23.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.323+0000 7ff739bba640 1 -- 172.21.15.98:0/1183179772 shutdown_connections 2024-09-19T08:45:23.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.323+0000 7ff739bba640 1 -- 172.21.15.98:0/1183179772 wait complete. 2024-09-19T08:45:23.341 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-19T08:45:23.341 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-19T08:45:23.341 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-19T08:45:23.341 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:45:23.342 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-19T08:45:23.342 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-19T08:45:23.342 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-19T08:45:23.342 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:45:23.342 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-19T08:45:23.342 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-19T08:45:23.342 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-19T08:45:23.342 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:45:23.343 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-19T08:45:23.343 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-19T08:45:23.343 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-19T08:45:23.343 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:45:23.352 INFO:tasks.ceph.mds.a:Stopped 2024-09-19T08:45:23.352 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-19T08:45:23.353 INFO:tasks.ceph.mds.c:Stopped 2024-09-19T08:45:23.354 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-19T08:45:23.355 INFO:tasks.ceph.mds.d:Stopped 2024-09-19T08:45:23.355 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-19T08:45:23.356 INFO:tasks.ceph.mds.b:Stopped 2024-09-19T08:45:23.356 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-19T08:45:23.397 INFO:tasks.ceph.mds.a:Started 2024-09-19T08:45:23.397 DEBUG:teuthology.parallel:result is None 2024-09-19T08:45:23.398 INFO:tasks.ceph.mds.c:Started 2024-09-19T08:45:23.399 DEBUG:teuthology.parallel:result is None 2024-09-19T08:45:23.400 INFO:tasks.ceph.mds.d:Started 2024-09-19T08:45:23.401 DEBUG:teuthology.parallel:result is None 2024-09-19T08:45:23.401 INFO:tasks.ceph.mds.b:Started 2024-09-19T08:45:23.401 DEBUG:teuthology.parallel:result is None 2024-09-19T08:45:23.401 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-19T08:45:23.462 INFO:tasks.ceph.mds.a.smithi098.stdout:starting mds.a at 2024-09-19T08:45:23.468 INFO:tasks.ceph.mds.c.smithi098.stdout:starting mds.c at 2024-09-19T08:45:23.475 INFO:tasks.ceph.mds.d.smithi139.stdout:starting mds.d at 2024-09-19T08:45:23.475 INFO:tasks.ceph.mds.b.smithi139.stdout:starting mds.b at 2024-09-19T08:45:23.475 INFO:tasks.ceph.mds.b.smithi139.stderr:2024-09-19T08:45:23.479+0000 7f2d73261640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-09-19T08:45:23.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.507+0000 7ff88084d640 1 Processor -- start 2024-09-19T08:45:23.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.507+0000 7ff88084d640 1 -- start start 2024-09-19T08:45:23.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.511+0000 7ff88084d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff87c0b6f60 0x7ff87c06d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:23.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.511+0000 7ff88084d640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff87c160350 con 0x7ff87c0b0280 2024-09-19T08:45:23.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.511+0000 7ff88084d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff87c160630 con 0x7ff87c074c70 2024-09-19T08:45:23.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.511+0000 7ff88084d640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7ff87c160910 con 0x7ff87c0b6f60 2024-09-19T08:45:23.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.511+0000 7ff879d9b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff87c0b6f60 0x7ff87c06d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:23.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.511+0000 7ff879d9b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff87c0b6f60 0x7ff87c06d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38894/0 (socket says 172.21.15.98:38894) 2024-09-19T08:45:23.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.511+0000 7ff879d9b640 1 -- 172.21.15.98:0/4122688251 learned_addr learned my addr 172.21.15.98:0/4122688251 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:23.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.511+0000 7ff879d9b640 1 -- 172.21.15.98:0/4122688251 >> v1:172.21.15.139:6789/0 conn(0x7ff87c074c70 legacy=0x7ff87c0ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:23.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.511+0000 7ff879d9b640 1 -- 172.21.15.98:0/4122688251 >> v1:172.21.15.98:6789/0 conn(0x7ff87c0b0280 legacy=0x7ff87c0b6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:23.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.511+0000 7ff879d9b640 1 -- 172.21.15.98:0/4122688251 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff87c160bf0 con 0x7ff87c0b6f60 2024-09-19T08:45:23.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.511+0000 7ff879d9b640 1 --2- 172.21.15.98:0/4122688251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff87c0b6f60 0x7ff87c06d1d0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7ff86c009870 tx=0x7ff86c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=971363039747d005 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:23.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.511+0000 7ff87959a640 1 -- 172.21.15.98:0/4122688251 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c0b0070 con 0x7ff87c0b6f60 2024-09-19T08:45:23.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.511+0000 7ff87959a640 1 -- 172.21.15.98:0/4122688251 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff86c0ab600 con 0x7ff87c0b6f60 2024-09-19T08:45:23.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.511+0000 7ff87959a640 1 -- 172.21.15.98:0/4122688251 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c0ab9d0 con 0x7ff87c0b6f60 2024-09-19T08:45:23.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.511+0000 7ff88084d640 1 -- 172.21.15.98:0/4122688251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff87c0b6f60 msgr2=0x7ff87c06d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:23.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.511+0000 7ff88084d640 1 --2- 172.21.15.98:0/4122688251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff87c0b6f60 0x7ff87c06d1d0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7ff86c009870 tx=0x7ff86c01c490 comp rx=0 tx=0).stop 2024-09-19T08:45:23.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.511+0000 7ff88084d640 1 -- 172.21.15.98:0/4122688251 shutdown_connections 2024-09-19T08:45:23.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.511+0000 7ff88084d640 1 --2- 172.21.15.98:0/4122688251 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff87c0b6f60 0x7ff87c06d1d0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:23.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.511+0000 7ff88084d640 1 -- 172.21.15.98:0/4122688251 >> 172.21.15.98:0/4122688251 conn(0x7ff87c073170 msgr2=0x7ff87c073580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:23.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.511+0000 7ff88084d640 1 -- 172.21.15.98:0/4122688251 shutdown_connections 2024-09-19T08:45:23.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.511+0000 7ff88084d640 1 -- 172.21.15.98:0/4122688251 wait complete. 2024-09-19T08:45:23.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.515+0000 7ff88084d640 1 Processor -- start 2024-09-19T08:45:23.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.515+0000 7ff88084d640 1 -- start start 2024-09-19T08:45:23.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.515+0000 7ff88084d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff87c074c70 0x7ff87c0b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:23.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.515+0000 7ff88084d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff87c0b0280 0x7ff87c099fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:23.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.515+0000 7ff88084d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff87c0b6f60 0x7ff87c154190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:23.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.515+0000 7ff88084d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff87c0af670 con 0x7ff87c0b6f60 2024-09-19T08:45:23.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.515+0000 7ff88084d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff87c0af4f0 con 0x7ff87c0b0280 2024-09-19T08:45:23.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.515+0000 7ff88084d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff87c16a390 con 0x7ff87c074c70 2024-09-19T08:45:23.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.515+0000 7ff87ad9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff87c0b6f60 0x7ff87c154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:23.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.515+0000 7ff87ad9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff87c0b6f60 0x7ff87c154190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38906/0 (socket says 172.21.15.98:38906) 2024-09-19T08:45:23.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.515+0000 7ff87ad9d640 1 -- 172.21.15.98:0/2494711223 learned_addr learned my addr 172.21.15.98:0/2494711223 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:23.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.515+0000 7ff87a59c640 1 --2- 172.21.15.98:0/2494711223 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff87c074c70 0x7ff87c0b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:23.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.515+0000 7ff879d9b640 1 --2- 172.21.15.98:0/2494711223 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff87c0b0280 0x7ff87c099fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:23.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.515+0000 7ff87ad9d640 1 -- 172.21.15.98:0/2494711223 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff87c074c70 msgr2=0x7ff87c0b0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:23.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.515+0000 7ff87ad9d640 1 --2- 172.21.15.98:0/2494711223 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff87c074c70 0x7ff87c0b0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:23.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.515+0000 7ff87ad9d640 1 -- 172.21.15.98:0/2494711223 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff87c0b0280 msgr2=0x7ff87c099fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:23.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.515+0000 7ff87ad9d640 1 --2- 172.21.15.98:0/2494711223 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff87c0b0280 0x7ff87c099fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:23.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.515+0000 7ff87ad9d640 1 -- 172.21.15.98:0/2494711223 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff87c155650 con 0x7ff87c0b6f60 2024-09-19T08:45:23.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.515+0000 7ff87a59c640 1 --2- 172.21.15.98:0/2494711223 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff87c074c70 0x7ff87c0b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:23.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.515+0000 7ff879d9b640 1 --2- 172.21.15.98:0/2494711223 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff87c0b0280 0x7ff87c099fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:23.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.515+0000 7ff87ad9d640 1 --2- 172.21.15.98:0/2494711223 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff87c0b6f60 0x7ff87c154190 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7ff87400b970 tx=0x7ff87400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:23.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.515+0000 7ff86affd640 1 -- 172.21.15.98:0/2494711223 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff874019070 con 0x7ff87c0b6f60 2024-09-19T08:45:23.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.515+0000 7ff86affd640 1 -- 172.21.15.98:0/2494711223 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff87400cb40 con 0x7ff87c0b6f60 2024-09-19T08:45:23.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.515+0000 7ff88084d640 1 -- 172.21.15.98:0/2494711223 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff87c15cd80 con 0x7ff87c0b6f60 2024-09-19T08:45:23.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.515+0000 7ff86affd640 1 -- 172.21.15.98:0/2494711223 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff874014440 con 0x7ff87c0b6f60 2024-09-19T08:45:23.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.515+0000 7ff88084d640 1 -- 172.21.15.98:0/2494711223 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff87c15f2d0 con 0x7ff87c0b6f60 2024-09-19T08:45:23.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.515+0000 7ff88084d640 1 -- 172.21.15.98:0/2494711223 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff87c06d1d0 con 0x7ff87c0b6f60 2024-09-19T08:45:23.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.515+0000 7ff86affd640 1 -- 172.21.15.98:0/2494711223 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7ff87401c070 con 0x7ff87c0b6f60 2024-09-19T08:45:23.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.519+0000 7ff86affd640 1 --2- 172.21.15.98:0/2494711223 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff85c067e90 0x7ff85c06a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:23.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.519+0000 7ff86affd640 1 -- 172.21.15.98:0/2494711223 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 4971+0+0 (secure 0 0 0) 0x7ff874085fa0 con 0x7ff87c0b6f60 2024-09-19T08:45:23.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.519+0000 7ff87a59c640 1 --2- 172.21.15.98:0/2494711223 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff85c067e90 0x7ff85c06a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:23.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.519+0000 7ff87a59c640 1 --2- 172.21.15.98:0/2494711223 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff85c067e90 0x7ff85c06a350 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff870001710 tx=0x7ff870009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:23.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.523+0000 7ff86affd640 1 -- 172.21.15.98:0/2494711223 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff8740565e0 con 0x7ff87c0b6f60 2024-09-19T08:45:23.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:23.695+0000 7ff88084d640 1 -- 172.21.15.98:0/2494711223 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7ff87c072060 con 0x7ff87c0b6f60 2024-09-19T08:45:24.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.747+0000 7ff86affd640 1 -- 172.21.15.98:0/2494711223 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v131) ==== 97+0+0 (secure 0 0 0) 0x7ff874055ee0 con 0x7ff87c0b6f60 2024-09-19T08:45:24.750 INFO:teuthology.orchestra.run.smithi098.stderr: removed all blocklist entries 2024-09-19T08:45:24.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.751+0000 7ff88084d640 1 -- 172.21.15.98:0/2494711223 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff85c067e90 msgr2=0x7ff85c06a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:24.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.751+0000 7ff88084d640 1 --2- 172.21.15.98:0/2494711223 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff85c067e90 0x7ff85c06a350 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff870001710 tx=0x7ff870009040 comp rx=0 tx=0).stop 2024-09-19T08:45:24.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.751+0000 7ff88084d640 1 -- 172.21.15.98:0/2494711223 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff87c0b6f60 msgr2=0x7ff87c154190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:24.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.751+0000 7ff88084d640 1 --2- 172.21.15.98:0/2494711223 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff87c0b6f60 0x7ff87c154190 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7ff87400b970 tx=0x7ff87400be30 comp rx=0 tx=0).stop 2024-09-19T08:45:24.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.751+0000 7ff88084d640 1 -- 172.21.15.98:0/2494711223 shutdown_connections 2024-09-19T08:45:24.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.751+0000 7ff88084d640 1 --2- 172.21.15.98:0/2494711223 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff85c067e90 0x7ff85c06a350 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:24.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.751+0000 7ff88084d640 1 --2- 172.21.15.98:0/2494711223 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff87c0b6f60 0x7ff87c154190 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:24.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.751+0000 7ff88084d640 1 --2- 172.21.15.98:0/2494711223 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff87c0b0280 0x7ff87c099fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:24.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.751+0000 7ff88084d640 1 --2- 172.21.15.98:0/2494711223 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff87c074c70 0x7ff87c0b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:24.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.751+0000 7ff88084d640 1 -- 172.21.15.98:0/2494711223 >> 172.21.15.98:0/2494711223 conn(0x7ff87c073170 msgr2=0x7ff87c0b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:24.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.751+0000 7ff88084d640 1 -- 172.21.15.98:0/2494711223 shutdown_connections 2024-09-19T08:45:24.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.751+0000 7ff88084d640 1 -- 172.21.15.98:0/2494711223 wait complete. 2024-09-19T08:45:24.776 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-19T08:45:24.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.895+0000 7fee10fe3640 1 Processor -- start 2024-09-19T08:45:24.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.895+0000 7fee10fe3640 1 -- start start 2024-09-19T08:45:24.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.895+0000 7fee10fe3640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee0c0b9970 0x7fee0c0b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:24.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.895+0000 7fee10fe3640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee0c068440 con 0x7fee0c0d4820 2024-09-19T08:45:24.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.895+0000 7fee10fe3640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee0c162a90 con 0x7fee0c0b7f80 2024-09-19T08:45:24.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.895+0000 7fee10fe3640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fee0c163050 con 0x7fee0c0b9970 2024-09-19T08:45:24.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.895+0000 7fee09d9b640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fee0c0d4820 0x7fee0c1621b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:35826/0 (socket says 172.21.15.98:35826) 2024-09-19T08:45:24.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.895+0000 7fee09d9b640 1 -- 172.21.15.98:0/3704762341 learned_addr learned my addr 172.21.15.98:0/3704762341 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:24.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.895+0000 7fee0ad9d640 1 --2- 172.21.15.98:0/3704762341 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee0c0b9970 0x7fee0c0b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:24.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.895+0000 7fee0959a640 1 -- 172.21.15.98:0/3704762341 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 994769545 0 0) 0x7fee0c068440 con 0x7fee0c0d4820 2024-09-19T08:45:24.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.895+0000 7fee0959a640 1 -- 172.21.15.98:0/3704762341 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fedf4003530 con 0x7fee0c0d4820 2024-09-19T08:45:24.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.895+0000 7fee0959a640 1 -- 172.21.15.98:0/3704762341 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1690702802 0 0) 0x7fedf4003530 con 0x7fee0c0d4820 2024-09-19T08:45:24.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.895+0000 7fee0959a640 1 -- 172.21.15.98:0/3704762341 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee0c0b9970 msgr2=0x7fee0c0b9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:24.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.895+0000 7fee0959a640 1 --2- 172.21.15.98:0/3704762341 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee0c0b9970 0x7fee0c0b9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:24.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.895+0000 7fee0959a640 1 -- 172.21.15.98:0/3704762341 >> v1:172.21.15.139:6789/0 conn(0x7fee0c0b7f80 legacy=0x7fee0c15f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:24.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.895+0000 7fee0959a640 1 -- 172.21.15.98:0/3704762341 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee0c162d70 con 0x7fee0c0d4820 2024-09-19T08:45:24.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.895+0000 7fee0959a640 1 -- 172.21.15.98:0/3704762341 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fee000016a0 con 0x7fee0c0d4820 2024-09-19T08:45:24.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.895+0000 7fee0959a640 1 -- 172.21.15.98:0/3704762341 >> v1:172.21.15.98:6789/0 conn(0x7fee0c0d4820 legacy=0x7fee0c1621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:24.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.895+0000 7fee0959a640 1 --2- 172.21.15.98:0/3704762341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedf40039f0 0x7fedf4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:24.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.895+0000 7fee0959a640 1 --2- 172.21.15.98:0/3704762341 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fedf4004810 0x7fedf4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:24.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.895+0000 7fee0959a640 1 --2- 172.21.15.98:0/3704762341 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee0c0b9970 0x7fedf40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:24.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.895+0000 7fee0959a640 1 -- 172.21.15.98:0/3704762341 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c163050 con 0x7fedf4004810 2024-09-19T08:45:24.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.895+0000 7fee0959a640 1 -- 172.21.15.98:0/3704762341 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fedf400bab0 con 0x7fedf40039f0 2024-09-19T08:45:24.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.895+0000 7fee0959a640 1 -- 172.21.15.98:0/3704762341 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fedf400bcb0 con 0x7fee0c0b9970 2024-09-19T08:45:24.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.895+0000 7fee09d9b640 1 --2- 172.21.15.98:0/3704762341 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fedf4004810 0x7fedf4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:24.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.895+0000 7fee0ad9d640 1 --2- 172.21.15.98:0/3704762341 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee0c0b9970 0x7fedf40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:24.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.895+0000 7fee0a59c640 1 --2- 172.21.15.98:0/3704762341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedf40039f0 0x7fedf4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:24.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.895+0000 7fee09d9b640 1 -- 172.21.15.98:0/3704762341 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee0c0b9970 msgr2=0x7fedf40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:24.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.895+0000 7fee09d9b640 1 --2- 172.21.15.98:0/3704762341 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee0c0b9970 0x7fedf40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:24.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.895+0000 7fee09d9b640 1 -- 172.21.15.98:0/3704762341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedf40039f0 msgr2=0x7fedf4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:24.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.895+0000 7fee09d9b640 1 --2- 172.21.15.98:0/3704762341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedf40039f0 0x7fedf4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:24.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.895+0000 7fee09d9b640 1 -- 172.21.15.98:0/3704762341 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedf4003530 con 0x7fedf4004810 2024-09-19T08:45:24.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.899+0000 7fee09d9b640 1 --2- 172.21.15.98:0/3704762341 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fedf4004810 0x7fedf4004c90 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fee0c144e40 tx=0x7fee0001f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4179257deb6bf7b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:24.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.899+0000 7fee0959a640 1 -- 172.21.15.98:0/3704762341 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee00007d40 con 0x7fedf4004810 2024-09-19T08:45:24.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.899+0000 7fee0959a640 1 -- 172.21.15.98:0/3704762341 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee00005ae0 con 0x7fedf4004810 2024-09-19T08:45:24.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.899+0000 7fee0959a640 1 -- 172.21.15.98:0/3704762341 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee00007070 con 0x7fedf4004810 2024-09-19T08:45:24.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.899+0000 7fee10fe3640 1 -- 172.21.15.98:0/3704762341 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fedf4004810 msgr2=0x7fedf4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:24.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.899+0000 7fee10fe3640 1 --2- 172.21.15.98:0/3704762341 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fedf4004810 0x7fedf4004c90 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fee0c144e40 tx=0x7fee0001f390 comp rx=0 tx=0).stop 2024-09-19T08:45:24.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.899+0000 7fee0ad9d640 1 -- 172.21.15.98:0/3704762341 reap_dead start 2024-09-19T08:45:24.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.899+0000 7fee10fe3640 1 -- 172.21.15.98:0/3704762341 shutdown_connections 2024-09-19T08:45:24.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.899+0000 7fee10fe3640 1 -- 172.21.15.98:0/3704762341 >> 172.21.15.98:0/3704762341 conn(0x7fee0c074e40 msgr2=0x7fee0c075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:24.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.899+0000 7fee10fe3640 1 -- 172.21.15.98:0/3704762341 shutdown_connections 2024-09-19T08:45:24.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.899+0000 7fee10fe3640 1 -- 172.21.15.98:0/3704762341 wait complete. 2024-09-19T08:45:24.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.899+0000 7fee10fe3640 1 Processor -- start 2024-09-19T08:45:24.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.899+0000 7fee10fe3640 1 -- start start 2024-09-19T08:45:24.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.899+0000 7fee10fe3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedf4004810 0x7fee0c13ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:24.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.899+0000 7fee10fe3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee0c1f4510 0x7fee0c140940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:24.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.899+0000 7fee10fe3640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee0c1f4990 0x7fee0c15ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:24.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.899+0000 7fee10fe3640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c0b84e0 con 0x7fee0c1f4510 2024-09-19T08:45:24.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.899+0000 7fee10fe3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c0b8360 con 0x7fedf4004810 2024-09-19T08:45:24.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.899+0000 7fee10fe3640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fee0c0b9de0 con 0x7fee0c1f4990 2024-09-19T08:45:24.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.899+0000 7fee09d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee0c1f4510 0x7fee0c140940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:24.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.899+0000 7fee09d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee0c1f4510 0x7fee0c140940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38918/0 (socket says 172.21.15.98:38918) 2024-09-19T08:45:24.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.899+0000 7fee09d9b640 1 -- 172.21.15.98:0/3851938478 learned_addr learned my addr 172.21.15.98:0/3851938478 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:24.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.903+0000 7fee0a59c640 1 --2- 172.21.15.98:0/3851938478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedf4004810 0x7fee0c13ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:24.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.903+0000 7fee0ad9d640 1 --2- 172.21.15.98:0/3851938478 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee0c1f4990 0x7fee0c15ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:24.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.903+0000 7fee09d9b640 1 -- 172.21.15.98:0/3851938478 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee0c1f4990 msgr2=0x7fee0c15ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:24.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.903+0000 7fee09d9b640 1 --2- 172.21.15.98:0/3851938478 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee0c1f4990 0x7fee0c15ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:24.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.903+0000 7fee09d9b640 1 -- 172.21.15.98:0/3851938478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedf4004810 msgr2=0x7fee0c13ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:24.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.903+0000 7fee09d9b640 1 --2- 172.21.15.98:0/3851938478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedf4004810 0x7fee0c13ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:24.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.903+0000 7fee09d9b640 1 -- 172.21.15.98:0/3851938478 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee0c158f10 con 0x7fee0c1f4510 2024-09-19T08:45:24.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.903+0000 7fee09d9b640 1 --2- 172.21.15.98:0/3851938478 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee0c1f4510 0x7fee0c140940 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fee00000f70 tx=0x7fedf4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:24.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.903+0000 7fedfaffd640 1 -- 172.21.15.98:0/3851938478 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee000047f0 con 0x7fee0c1f4510 2024-09-19T08:45:24.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.903+0000 7fedfaffd640 1 -- 172.21.15.98:0/3851938478 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee000b6040 con 0x7fee0c1f4510 2024-09-19T08:45:24.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.903+0000 7fee10fe3640 1 -- 172.21.15.98:0/3851938478 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee0c1591f0 con 0x7fee0c1f4510 2024-09-19T08:45:24.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.903+0000 7fedfaffd640 1 -- 172.21.15.98:0/3851938478 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee00004bb0 con 0x7fee0c1f4510 2024-09-19T08:45:24.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.903+0000 7fee10fe3640 1 -- 172.21.15.98:0/3851938478 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee0c1597b0 con 0x7fee0c1f4510 2024-09-19T08:45:24.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.903+0000 7fee10fe3640 1 -- 172.21.15.98:0/3851938478 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee0c0ba130 con 0x7fee0c1f4510 2024-09-19T08:45:24.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.903+0000 7fedfaffd640 1 -- 172.21.15.98:0/3851938478 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fee00007070 con 0x7fee0c1f4510 2024-09-19T08:45:24.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.903+0000 7fedfaffd640 1 --2- 172.21.15.98:0/3851938478 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fedec067e60 0x7fedec06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:24.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.903+0000 7fedfaffd640 1 -- 172.21.15.98:0/3851938478 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 4885+0+0 (secure 0 0 0) 0x7fee001235c0 con 0x7fee0c1f4510 2024-09-19T08:45:24.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.903+0000 7fee0a59c640 1 --2- 172.21.15.98:0/3851938478 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fedec067e60 0x7fedec06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:24.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.903+0000 7fee0a59c640 1 --2- 172.21.15.98:0/3851938478 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fedec067e60 0x7fedec06a320 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fedfc001660 tx=0x7fedfc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:24.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:24.907+0000 7fedfaffd640 1 -- 172.21.15.98:0/3851938478 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fee000190a0 con 0x7fee0c1f4510 2024-09-19T08:45:25.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.127+0000 7fee10fe3640 1 -- 172.21.15.98:0/3851938478 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fee0c0d27a0 con 0x7fee0c1f4510 2024-09-19T08:45:25.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.131+0000 7fedfaffd640 1 -- 172.21.15.98:0/3851938478 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) ==== 80+0+5304 (secure 0 0 0) 0x7fee000f33a0 con 0x7fee0c1f4510 2024-09-19T08:45:25.133 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:45:25.133 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:45:25.133 INFO:teuthology.orchestra.run.smithi098.stdout: "auth_dump": [ 2024-09-19T08:45:25.133 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:45:25.133 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.a", 2024-09-19T08:45:25.133 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmuujGBRAAa2k/kW68NJI6nUKx+W+G2Q==", 2024-09-19T08:45:25.133 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:45:25.133 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:45:25.133 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:45:25.133 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:45:25.134 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:45:25.134 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:45:25.134 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:45:25.134 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:45:25.134 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.b", 2024-09-19T08:45:25.134 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmFoojFBAAaOZPpa41fdjBPEn3CGsJnA==", 2024-09-19T08:45:25.134 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:45:25.134 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:45:25.134 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:45:25.134 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:45:25.134 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:45:25.134 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:45:25.134 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:45:25.134 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:45:25.135 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.c", 2024-09-19T08:45:25.135 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmcQcDDhAAF0QJMlljMH+DiHfWW+/Sog==", 2024-09-19T08:45:25.135 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:45:25.135 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:45:25.135 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:45:25.135 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:45:25.135 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:45:25.135 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:45:25.135 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:45:25.135 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:45:25.135 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.d", 2024-09-19T08:45:25.135 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmwWmqHBAAf15VcrdeHf8AzQjGsN9LoA==", 2024-09-19T08:45:25.135 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:45:25.136 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:45:25.136 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:45:25.136 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:45:25.136 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:45:25.136 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:45:25.136 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:45:25.136 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:45:25.136 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.0", 2024-09-19T08:45:25.136 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDC4utm9PjNKxAAa6SG9AE4smPiu4azRAFKvg==", 2024-09-19T08:45:25.136 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:45:25.136 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:45:25.136 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:45:25.136 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:45:25.137 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:45:25.137 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:45:25.137 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:45:25.137 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.1", 2024-09-19T08:45:25.137 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDG4utmHy6UCxAAN6hpG4DJEp81iG/KoloTvQ==", 2024-09-19T08:45:25.137 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:45:25.137 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:45:25.137 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:45:25.137 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:45:25.137 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:45:25.137 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:45:25.138 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:45:25.138 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.2", 2024-09-19T08:45:25.138 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDJ4utmUVP4GhAAJNAL30sEAseoDG2fE6YbZQ==", 2024-09-19T08:45:25.138 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:45:25.138 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:45:25.138 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:45:25.138 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:45:25.138 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:45:25.138 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:45:25.138 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:45:25.138 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.3", 2024-09-19T08:45:25.138 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDM4utmTKgoLRAAiCrPRLML+wvjOt6RhKay3w==", 2024-09-19T08:45:25.138 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:45:25.139 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:45:25.139 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:45:25.139 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:45:25.140 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:45:25.140 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:45:25.140 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:45:25.140 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.4", 2024-09-19T08:45:25.140 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDS4utmAf6bMhAAI3uENuFoXLicUeuspgFjYA==", 2024-09-19T08:45:25.140 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:45:25.140 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:45:25.140 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:45:25.140 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:45:25.140 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:45:25.140 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:45:25.140 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:45:25.141 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.5", 2024-09-19T08:45:25.141 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDW4utmRjxwEhAAF1A2GtrZXVx9BlmjHjhUDQ==", 2024-09-19T08:45:25.141 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:45:25.141 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:45:25.141 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:45:25.141 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:45:25.141 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:45:25.141 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:45:25.141 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:45:25.141 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.6", 2024-09-19T08:45:25.141 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDZ4utmSk/XIRAAROSfzGL7smR88ZkDSx5jAg==", 2024-09-19T08:45:25.141 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:45:25.141 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:45:25.142 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:45:25.142 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:45:25.142 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:45:25.142 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:45:25.142 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:45:25.142 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.7", 2024-09-19T08:45:25.142 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDc4utmXfpbMxAAqEM4I/ACFvfu/sMA8bMcrQ==", 2024-09-19T08:45:25.142 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:45:25.142 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:45:25.142 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:45:25.142 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:45:25.142 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:45:25.142 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:45:25.142 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:45:25.143 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.0", 2024-09-19T08:45:25.143 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utm0vNpIhAA4S1E8dFZ0+ibmSKhmZ+bIA==", 2024-09-19T08:45:25.143 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:45:25.143 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:45:25.143 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow r", 2024-09-19T08:45:25.143 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-19T08:45:25.143 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:45:25.143 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:45:25.143 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:45:25.143 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.1", 2024-09-19T08:45:25.143 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmx7xyKhAAC84lykPEb01MiypXKQVREQ==", 2024-09-19T08:45:25.143 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:45:25.143 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:45:25.144 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-19T08:45:25.144 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-19T08:45:25.144 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-19T08:45:25.144 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:45:25.144 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:45:25.144 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:45:25.144 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.2", 2024-09-19T08:45:25.144 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmteOQJxAAtTF2Py21ZE2WRQjeQSrkKQ==", 2024-09-19T08:45:25.144 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:45:25.144 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:45:25.144 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-19T08:45:25.144 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-19T08:45:25.144 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-19T08:45:25.144 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:45:25.145 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:45:25.145 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:45:25.145 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.3", 2024-09-19T08:45:25.145 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmKN92LxAAMqraZOWRT6IIfx5AuDWS2g==", 2024-09-19T08:45:25.145 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:45:25.145 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:45:25.145 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-19T08:45:25.145 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-19T08:45:25.145 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-19T08:45:25.145 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:45:25.145 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:45:25.145 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:45:25.145 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.admin", 2024-09-19T08:45:25.146 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC+4utmLH97KhAA1DTYW5bTrGUG+7cb8+d7gA==", 2024-09-19T08:45:25.146 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:45:25.146 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-09-19T08:45:25.146 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:45:25.146 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:45:25.146 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:45:25.146 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:45:25.146 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:45:25.146 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:45:25.146 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mgr.x", 2024-09-19T08:45:25.146 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmbKuyAhAA0/XjHUIPOIV/YvNEhB8rEQ==", 2024-09-19T08:45:25.146 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:45:25.146 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-09-19T08:45:25.147 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile mgr", 2024-09-19T08:45:25.147 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:45:25.147 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:45:25.147 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:45:25.147 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:45:25.147 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mgr.y", 2024-09-19T08:45:25.147 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC+4utmU+ZrOxAAsvcwINH6qyxfN7N+rDrkqg==", 2024-09-19T08:45:25.147 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:45:25.147 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-09-19T08:45:25.147 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile mgr", 2024-09-19T08:45:25.147 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:45:25.147 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:45:25.147 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:45:25.147 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-19T08:45:25.148 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:45:25.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.131+0000 7fee10fe3640 1 -- 172.21.15.98:0/3851938478 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fedec067e60 msgr2=0x7fedec06a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:25.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.131+0000 7fee10fe3640 1 --2- 172.21.15.98:0/3851938478 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fedec067e60 0x7fedec06a320 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fedfc001660 tx=0x7fedfc009040 comp rx=0 tx=0).stop 2024-09-19T08:45:25.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.131+0000 7fee10fe3640 1 -- 172.21.15.98:0/3851938478 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee0c1f4510 msgr2=0x7fee0c140940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:25.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.131+0000 7fee10fe3640 1 --2- 172.21.15.98:0/3851938478 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee0c1f4510 0x7fee0c140940 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fee00000f70 tx=0x7fedf4005840 comp rx=0 tx=0).stop 2024-09-19T08:45:25.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.131+0000 7fee10fe3640 1 -- 172.21.15.98:0/3851938478 shutdown_connections 2024-09-19T08:45:25.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.131+0000 7fee10fe3640 1 --2- 172.21.15.98:0/3851938478 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fedec067e60 0x7fedec06a320 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:25.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.131+0000 7fee10fe3640 1 --2- 172.21.15.98:0/3851938478 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fee0c1f4990 0x7fee0c15ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:25.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.131+0000 7fee10fe3640 1 --2- 172.21.15.98:0/3851938478 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fee0c1f4510 0x7fee0c140940 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:25.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.131+0000 7fee10fe3640 1 --2- 172.21.15.98:0/3851938478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedf4004810 0x7fee0c13ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:25.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.131+0000 7fee10fe3640 1 -- 172.21.15.98:0/3851938478 >> 172.21.15.98:0/3851938478 conn(0x7fee0c074e40 msgr2=0x7fee0c0d4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:25.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.131+0000 7fee10fe3640 1 -- 172.21.15.98:0/3851938478 shutdown_connections 2024-09-19T08:45:25.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.131+0000 7fee10fe3640 1 -- 172.21.15.98:0/3851938478 wait complete. 2024-09-19T08:45:25.152 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-19T08:45:25.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.267+0000 7f8bb6046640 1 Processor -- start 2024-09-19T08:45:25.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.267+0000 7f8bb6046640 1 -- start start 2024-09-19T08:45:25.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.267+0000 7f8bb6046640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8bb015c550 0x7f8bb006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:25.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.267+0000 7f8bb6046640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8bb00cfad0 0x7f8bb00723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:25.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.267+0000 7f8bb6046640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bb006d310 con 0x7f8bb006c1d0 2024-09-19T08:45:25.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.267+0000 7f8bb6046640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f8bb006c640 con 0x7f8bb015c550 2024-09-19T08:45:25.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.267+0000 7f8bb6046640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f8bb006d5f0 con 0x7f8bb00cfad0 2024-09-19T08:45:25.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.267+0000 7f8baf7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8bb015c550 0x7f8bb006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:25.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.267+0000 7f8baf7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8bb015c550 0x7f8bb006bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38920/0 (socket says 172.21.15.98:38920) 2024-09-19T08:45:25.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.267+0000 7f8baf7fe640 1 -- 172.21.15.98:0/3774581227 learned_addr learned my addr 172.21.15.98:0/3774581227 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:25.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.271+0000 7f8baffff640 1 --2- 172.21.15.98:0/3774581227 >> v2:172.21.15.139:3300/0 conn(0x7f8bb00cfad0 0x7f8bb00723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:25.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.271+0000 7f8baf7fe640 1 -- 172.21.15.98:0/3774581227 >> v2:172.21.15.139:3300/0 conn(0x7f8bb00cfad0 msgr2=0x7f8bb00723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:25.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.271+0000 7f8baf7fe640 1 --2- 172.21.15.98:0/3774581227 >> v2:172.21.15.139:3300/0 conn(0x7f8bb00cfad0 0x7f8bb00723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:25.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.271+0000 7f8baf7fe640 1 -- 172.21.15.98:0/3774581227 >> v1:172.21.15.139:6789/0 conn(0x7f8bb006c1d0 legacy=0x7f8bb00cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:25.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.271+0000 7f8baf7fe640 1 -- 172.21.15.98:0/3774581227 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb006d8d0 con 0x7f8bb015c550 2024-09-19T08:45:25.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.271+0000 7f8baffff640 1 --2- 172.21.15.98:0/3774581227 >> v2:172.21.15.139:3300/0 conn(0x7f8bb00cfad0 0x7f8bb00723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:25.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.271+0000 7f8baf7fe640 1 --2- 172.21.15.98:0/3774581227 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8bb015c550 0x7f8bb006bc90 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f8bb00b33a0 tx=0x7f8ba001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d6e066c8f196b6a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:25.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.271+0000 7f8bae7fc640 1 -- 172.21.15.98:0/3774581227 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba00b0070 con 0x7f8bb015c550 2024-09-19T08:45:25.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.271+0000 7f8bae7fc640 1 -- 172.21.15.98:0/3774581227 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ba00ab600 con 0x7f8bb015c550 2024-09-19T08:45:25.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.271+0000 7f8bae7fc640 1 -- 172.21.15.98:0/3774581227 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba00ab9d0 con 0x7f8bb015c550 2024-09-19T08:45:25.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.271+0000 7f8bb6046640 1 -- 172.21.15.98:0/3774581227 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8bb015c550 msgr2=0x7f8bb006bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:25.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.271+0000 7f8bb6046640 1 --2- 172.21.15.98:0/3774581227 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8bb015c550 0x7f8bb006bc90 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f8bb00b33a0 tx=0x7f8ba001c4e0 comp rx=0 tx=0).stop 2024-09-19T08:45:25.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.271+0000 7f8bb6046640 1 -- 172.21.15.98:0/3774581227 shutdown_connections 2024-09-19T08:45:25.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.271+0000 7f8bb6046640 1 --2- 172.21.15.98:0/3774581227 >> v2:172.21.15.139:3300/0 conn(0x7f8bb00cfad0 0x7f8bb00723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:25.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.271+0000 7f8bb6046640 1 --2- 172.21.15.98:0/3774581227 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8bb015c550 0x7f8bb006bc90 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:25.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.271+0000 7f8bb6046640 1 -- 172.21.15.98:0/3774581227 >> 172.21.15.98:0/3774581227 conn(0x7f8bb00b2010 msgr2=0x7f8bb00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:25.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.275+0000 7f8bb6046640 1 -- 172.21.15.98:0/3774581227 shutdown_connections 2024-09-19T08:45:25.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.275+0000 7f8bb6046640 1 -- 172.21.15.98:0/3774581227 wait complete. 2024-09-19T08:45:25.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.275+0000 7f8bb6046640 1 Processor -- start 2024-09-19T08:45:25.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.275+0000 7f8bb6046640 1 -- start start 2024-09-19T08:45:25.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.275+0000 7f8bb6046640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8bb006c1d0 0x7f8bb0151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:25.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.275+0000 7f8bb6046640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8bb00cfad0 0x7f8bb013f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:25.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.275+0000 7f8bb6046640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bb015c550 0x7f8bb0153960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:25.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.275+0000 7f8bb6046640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb00734c0 con 0x7f8bb006c1d0 2024-09-19T08:45:25.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.275+0000 7f8bb6046640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb0073340 con 0x7f8bb015c550 2024-09-19T08:45:25.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.275+0000 7f8bb6046640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8bb0073640 con 0x7f8bb00cfad0 2024-09-19T08:45:25.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.275+0000 7f8baf7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8bb006c1d0 0x7f8bb0151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:25.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.275+0000 7f8baf7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8bb006c1d0 0x7f8bb0151a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38928/0 (socket says 172.21.15.98:38928) 2024-09-19T08:45:25.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.275+0000 7f8baf7fe640 1 -- 172.21.15.98:0/4105651348 learned_addr learned my addr 172.21.15.98:0/4105651348 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:25.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.275+0000 7f8baffff640 1 --2- 172.21.15.98:0/4105651348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bb015c550 0x7f8bb0153960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:25.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.275+0000 7f8baf7fe640 1 -- 172.21.15.98:0/4105651348 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8bb00cfad0 msgr2=0x7f8bb013f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:25.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.275+0000 7f8baf7fe640 1 --2- 172.21.15.98:0/4105651348 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8bb00cfad0 0x7f8bb013f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:25.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.275+0000 7f8baf7fe640 1 -- 172.21.15.98:0/4105651348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bb015c550 msgr2=0x7f8bb0153960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:25.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.275+0000 7f8baf7fe640 1 --2- 172.21.15.98:0/4105651348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bb015c550 0x7f8bb0153960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:25.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.275+0000 7f8baf7fe640 1 -- 172.21.15.98:0/4105651348 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb013fa60 con 0x7f8bb006c1d0 2024-09-19T08:45:25.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.275+0000 7f8baf7fe640 1 --2- 172.21.15.98:0/4105651348 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8bb006c1d0 0x7f8bb0151a40 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f8ba00ae040 tx=0x7f8ba0019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:25.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.275+0000 7f8b97fff640 1 -- 172.21.15.98:0/4105651348 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba00b0040 con 0x7f8bb006c1d0 2024-09-19T08:45:25.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.275+0000 7f8b97fff640 1 -- 172.21.15.98:0/4105651348 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ba00b9070 con 0x7f8bb006c1d0 2024-09-19T08:45:25.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.275+0000 7f8bb6046640 1 -- 172.21.15.98:0/4105651348 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bb00aaa40 con 0x7f8bb006c1d0 2024-09-19T08:45:25.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.275+0000 7f8b97fff640 1 -- 172.21.15.98:0/4105651348 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba00b4660 con 0x7f8bb006c1d0 2024-09-19T08:45:25.280 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.275+0000 7f8bb6046640 1 -- 172.21.15.98:0/4105651348 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bb013cf00 con 0x7f8bb006c1d0 2024-09-19T08:45:25.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.275+0000 7f8bb6046640 1 -- 172.21.15.98:0/4105651348 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bb00723a0 con 0x7f8bb006c1d0 2024-09-19T08:45:25.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.279+0000 7f8b97fff640 1 -- 172.21.15.98:0/4105651348 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f8ba00b4800 con 0x7f8bb006c1d0 2024-09-19T08:45:25.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.279+0000 7f8b97fff640 1 --2- 172.21.15.98:0/4105651348 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8b9c067e60 0x7f8b9c06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:25.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.279+0000 7f8b97fff640 1 -- 172.21.15.98:0/4105651348 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 4885+0+0 (secure 0 0 0) 0x7f8ba01201a0 con 0x7f8bb006c1d0 2024-09-19T08:45:25.281 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.279+0000 7f8baeffd640 1 --2- 172.21.15.98:0/4105651348 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8b9c067e60 0x7f8b9c06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:25.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.279+0000 7f8baeffd640 1 --2- 172.21.15.98:0/4105651348 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8b9c067e60 0x7f8b9c06a320 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8bb015a7d0 tx=0x7f8ba4006bf0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:25.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.283+0000 7f8b97fff640 1 -- 172.21.15.98:0/4105651348 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f8bb013fa60 con 0x7f8bb006c1d0 2024-09-19T08:45:25.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.447+0000 7f8bb6046640 1 -- 172.21.15.98:0/4105651348 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f8bb00d0fb0 con 0x7f8bb006c1d0 2024-09-19T08:45:25.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.447+0000 7f8b97fff640 1 -- 172.21.15.98:0/4105651348 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v131) ==== 84+0+65 (secure 0 0 0) 0x7f8ba011ee20 con 0x7f8bb006c1d0 2024-09-19T08:45:25.450 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:45:25.450 INFO:teuthology.orchestra.run.smithi098.stdout:[ 2024-09-19T08:45:25.450 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:45:25.450 INFO:teuthology.orchestra.run.smithi098.stdout: "poolnum": 1, 2024-09-19T08:45:25.450 INFO:teuthology.orchestra.run.smithi098.stdout: "poolname": ".mgr" 2024-09-19T08:45:25.450 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:45:25.450 INFO:teuthology.orchestra.run.smithi098.stdout:] 2024-09-19T08:45:25.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.447+0000 7f8bb6046640 1 -- 172.21.15.98:0/4105651348 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8b9c067e60 msgr2=0x7f8b9c06a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:25.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.447+0000 7f8bb6046640 1 --2- 172.21.15.98:0/4105651348 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8b9c067e60 0x7f8b9c06a320 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8bb015a7d0 tx=0x7f8ba4006bf0 comp rx=0 tx=0).stop 2024-09-19T08:45:25.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.447+0000 7f8bb6046640 1 -- 172.21.15.98:0/4105651348 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8bb006c1d0 msgr2=0x7f8bb0151a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:25.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.447+0000 7f8bb6046640 1 --2- 172.21.15.98:0/4105651348 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8bb006c1d0 0x7f8bb0151a40 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f8ba00ae040 tx=0x7f8ba0019280 comp rx=0 tx=0).stop 2024-09-19T08:45:25.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.451+0000 7f8bb6046640 1 -- 172.21.15.98:0/4105651348 shutdown_connections 2024-09-19T08:45:25.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.451+0000 7f8bb6046640 1 --2- 172.21.15.98:0/4105651348 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8b9c067e60 0x7f8b9c06a320 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:25.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.451+0000 7f8bb6046640 1 --2- 172.21.15.98:0/4105651348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8bb015c550 0x7f8bb0153960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:25.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.451+0000 7f8bb6046640 1 --2- 172.21.15.98:0/4105651348 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8bb00cfad0 0x7f8bb013f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:25.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.451+0000 7f8bb6046640 1 --2- 172.21.15.98:0/4105651348 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8bb006c1d0 0x7f8bb0151a40 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:25.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.451+0000 7f8bb6046640 1 -- 172.21.15.98:0/4105651348 >> 172.21.15.98:0/4105651348 conn(0x7f8bb00b2010 msgr2=0x7f8bb00cd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:25.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.451+0000 7f8bb6046640 1 -- 172.21.15.98:0/4105651348 shutdown_connections 2024-09-19T08:45:25.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.451+0000 7f8bb6046640 1 -- 172.21.15.98:0/4105651348 wait complete. 2024-09-19T08:45:25.469 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T08:45:25.469 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T08:45:25.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.591+0000 7fb075b2b640 1 Processor -- start 2024-09-19T08:45:25.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.591+0000 7fb075b2b640 1 -- start start 2024-09-19T08:45:25.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.591+0000 7fb075b2b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb07006d230 0x7fb070075680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:25.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.591+0000 7fb075b2b640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0700c83e0 con 0x7fb0700cb2c0 2024-09-19T08:45:25.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.591+0000 7fb075b2b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0700c86c0 con 0x7fb0700b6b90 2024-09-19T08:45:25.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.591+0000 7fb075b2b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb070075bc0 con 0x7fb07006d230 2024-09-19T08:45:25.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.591+0000 7fb06f7fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fb0700cb2c0 0x7fb0700b6480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:35836/0 (socket says 172.21.15.98:35836) 2024-09-19T08:45:25.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.591+0000 7fb06f7fe640 1 -- 172.21.15.98:0/588372353 learned_addr learned my addr 172.21.15.98:0/588372353 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:25.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.591+0000 7fb06e7fc640 1 -- 172.21.15.98:0/588372353 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2983258015 0 0) 0x7fb0700c83e0 con 0x7fb0700cb2c0 2024-09-19T08:45:25.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.591+0000 7fb06e7fc640 1 -- 172.21.15.98:0/588372353 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb058003530 con 0x7fb0700cb2c0 2024-09-19T08:45:25.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.591+0000 7fb06ffff640 1 --2- 172.21.15.98:0/588372353 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb07006d230 0x7fb070075680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:25.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.591+0000 7fb06e7fc640 1 -- 172.21.15.98:0/588372353 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1946433537 0 0) 0x7fb058003530 con 0x7fb0700cb2c0 2024-09-19T08:45:25.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.591+0000 7fb06e7fc640 1 -- 172.21.15.98:0/588372353 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb07006d230 msgr2=0x7fb070075680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:25.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.591+0000 7fb06e7fc640 1 --2- 172.21.15.98:0/588372353 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb07006d230 0x7fb070075680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:25.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.591+0000 7fb06e7fc640 1 -- 172.21.15.98:0/588372353 >> v1:172.21.15.139:6789/0 conn(0x7fb0700b6b90 legacy=0x7fb0700c9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:25.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.591+0000 7fb06e7fc640 1 -- 172.21.15.98:0/588372353 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0700c89a0 con 0x7fb0700cb2c0 2024-09-19T08:45:25.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.591+0000 7fb06e7fc640 1 -- 172.21.15.98:0/588372353 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fb068001a00 con 0x7fb0700cb2c0 2024-09-19T08:45:25.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.591+0000 7fb06e7fc640 1 -- 172.21.15.98:0/588372353 >> v1:172.21.15.98:6789/0 conn(0x7fb0700cb2c0 legacy=0x7fb0700b6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:25.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.591+0000 7fb06e7fc640 1 --2- 172.21.15.98:0/588372353 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0580039f0 0x7fb058003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:25.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.591+0000 7fb06e7fc640 1 --2- 172.21.15.98:0/588372353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb058004810 0x7fb058004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:25.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.591+0000 7fb06e7fc640 1 --2- 172.21.15.98:0/588372353 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb07006d230 0x7fb058005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:25.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.591+0000 7fb06e7fc640 1 -- 172.21.15.98:0/588372353 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb05800bb00 con 0x7fb058004810 2024-09-19T08:45:25.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.591+0000 7fb06e7fc640 1 -- 172.21.15.98:0/588372353 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb05800bcb0 con 0x7fb0580039f0 2024-09-19T08:45:25.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.591+0000 7fb06e7fc640 1 -- 172.21.15.98:0/588372353 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb05800beb0 con 0x7fb07006d230 2024-09-19T08:45:25.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.591+0000 7fb06effd640 1 --2- 172.21.15.98:0/588372353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb058004810 0x7fb058004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:25.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.591+0000 7fb06ffff640 1 --2- 172.21.15.98:0/588372353 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb07006d230 0x7fb058005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:25.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.591+0000 7fb06f7fe640 1 --2- 172.21.15.98:0/588372353 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0580039f0 0x7fb058003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:25.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.591+0000 7fb06effd640 1 -- 172.21.15.98:0/588372353 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb07006d230 msgr2=0x7fb058005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:25.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.591+0000 7fb06effd640 1 --2- 172.21.15.98:0/588372353 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb07006d230 0x7fb058005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:25.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.591+0000 7fb06effd640 1 -- 172.21.15.98:0/588372353 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0580039f0 msgr2=0x7fb058003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:25.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.591+0000 7fb06effd640 1 --2- 172.21.15.98:0/588372353 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0580039f0 0x7fb058003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:25.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.591+0000 7fb06effd640 1 -- 172.21.15.98:0/588372353 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb058003530 con 0x7fb058004810 2024-09-19T08:45:25.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.591+0000 7fb06ffff640 1 --2- 172.21.15.98:0/588372353 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb07006d230 0x7fb058005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:25.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.595+0000 7fb06effd640 1 --2- 172.21.15.98:0/588372353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb058004810 0x7fb058004c90 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fb064001780 tx=0x7fb06401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43323bf421347c0e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:25.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.595+0000 7fb06e7fc640 1 -- 172.21.15.98:0/588372353 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0640013a0 con 0x7fb058004810 2024-09-19T08:45:25.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.595+0000 7fb06e7fc640 1 -- 172.21.15.98:0/588372353 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0640052f0 con 0x7fb058004810 2024-09-19T08:45:25.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.595+0000 7fb06e7fc640 1 -- 172.21.15.98:0/588372353 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb064003840 con 0x7fb058004810 2024-09-19T08:45:25.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.595+0000 7fb075b2b640 1 -- 172.21.15.98:0/588372353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb058004810 msgr2=0x7fb058004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:25.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.595+0000 7fb075b2b640 1 --2- 172.21.15.98:0/588372353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb058004810 0x7fb058004c90 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fb064001780 tx=0x7fb06401c420 comp rx=0 tx=0).stop 2024-09-19T08:45:25.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.595+0000 7fb06ffff640 1 -- 172.21.15.98:0/588372353 reap_dead start 2024-09-19T08:45:25.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.595+0000 7fb075b2b640 1 -- 172.21.15.98:0/588372353 shutdown_connections 2024-09-19T08:45:25.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.595+0000 7fb075b2b640 1 -- 172.21.15.98:0/588372353 >> 172.21.15.98:0/588372353 conn(0x7fb0700b2e60 msgr2=0x7fb0700b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:25.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.595+0000 7fb075b2b640 1 -- 172.21.15.98:0/588372353 shutdown_connections 2024-09-19T08:45:25.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.595+0000 7fb075b2b640 1 -- 172.21.15.98:0/588372353 wait complete. 2024-09-19T08:45:25.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.595+0000 7fb075b2b640 1 Processor -- start 2024-09-19T08:45:25.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.599+0000 7fb075b2b640 1 -- start start 2024-09-19T08:45:25.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.599+0000 7fb075b2b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb058004810 0x7fb0700d16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:25.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.599+0000 7fb075b2b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0700c7810 0x7fb0700c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:25.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.599+0000 7fb075b2b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0700c6330 0x7fb0700c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:25.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.599+0000 7fb075b2b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb070075680 con 0x7fb0700c6330 2024-09-19T08:45:25.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.599+0000 7fb075b2b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb0700c8f60 con 0x7fb0700c7810 2024-09-19T08:45:25.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.599+0000 7fb075b2b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb070075800 con 0x7fb058004810 2024-09-19T08:45:25.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.599+0000 7fb06ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0700c6330 0x7fb0700c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:25.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.599+0000 7fb06ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0700c6330 0x7fb0700c67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38940/0 (socket says 172.21.15.98:38940) 2024-09-19T08:45:25.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.599+0000 7fb06ffff640 1 -- 172.21.15.98:0/3920409421 learned_addr learned my addr 172.21.15.98:0/3920409421 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:25.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.599+0000 7fb06effd640 1 --2- 172.21.15.98:0/3920409421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0700c7810 0x7fb0700c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:25.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.599+0000 7fb06f7fe640 1 --2- 172.21.15.98:0/3920409421 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb058004810 0x7fb0700d16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:25.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.599+0000 7fb06ffff640 1 -- 172.21.15.98:0/3920409421 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb058004810 msgr2=0x7fb0700d16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:25.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.599+0000 7fb06ffff640 1 --2- 172.21.15.98:0/3920409421 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb058004810 0x7fb0700d16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:25.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.599+0000 7fb06ffff640 1 -- 172.21.15.98:0/3920409421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0700c7810 msgr2=0x7fb0700c5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:25.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.599+0000 7fb06ffff640 1 --2- 172.21.15.98:0/3920409421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0700c7810 0x7fb0700c5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:25.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.599+0000 7fb06ffff640 1 -- 172.21.15.98:0/3920409421 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb07015ca80 con 0x7fb0700c6330 2024-09-19T08:45:25.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.599+0000 7fb06f7fe640 1 --2- 172.21.15.98:0/3920409421 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb058004810 0x7fb0700d16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:25.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.599+0000 7fb06ffff640 1 --2- 172.21.15.98:0/3920409421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0700c6330 0x7fb0700c67b0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fb070071830 tx=0x7fb06000f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:25.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.599+0000 7fb057fff640 1 -- 172.21.15.98:0/3920409421 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb060014660 con 0x7fb0700c6330 2024-09-19T08:45:25.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.599+0000 7fb057fff640 1 -- 172.21.15.98:0/3920409421 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb06001a070 con 0x7fb0700c6330 2024-09-19T08:45:25.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.599+0000 7fb075b2b640 1 -- 172.21.15.98:0/3920409421 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb07015cd60 con 0x7fb0700c6330 2024-09-19T08:45:25.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.599+0000 7fb057fff640 1 -- 172.21.15.98:0/3920409421 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb060014a40 con 0x7fb0700c6330 2024-09-19T08:45:25.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.599+0000 7fb075b2b640 1 -- 172.21.15.98:0/3920409421 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb07015d320 con 0x7fb0700c6330 2024-09-19T08:45:25.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.599+0000 7fb075b2b640 1 -- 172.21.15.98:0/3920409421 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb070068fb0 con 0x7fb0700c6330 2024-09-19T08:45:25.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.599+0000 7fb057fff640 1 -- 172.21.15.98:0/3920409421 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fb060012070 con 0x7fb0700c6330 2024-09-19T08:45:25.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.599+0000 7fb057fff640 1 --2- 172.21.15.98:0/3920409421 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb05c067e60 0x7fb05c06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:25.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.599+0000 7fb057fff640 1 -- 172.21.15.98:0/3920409421 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 4885+0+0 (secure 0 0 0) 0x7fb060085fc0 con 0x7fb0700c6330 2024-09-19T08:45:25.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.599+0000 7fb06f7fe640 1 --2- 172.21.15.98:0/3920409421 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb05c067e60 0x7fb05c06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:25.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.603+0000 7fb06f7fe640 1 --2- 172.21.15.98:0/3920409421 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb05c067e60 0x7fb05c06a320 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb068001590 tx=0x7fb0680014a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:25.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.603+0000 7fb057fff640 1 -- 172.21.15.98:0/3920409421 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb0600565c0 con 0x7fb0700c6330 2024-09-19T08:45:25.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:25.767+0000 7fb075b2b640 1 -- 172.21.15.98:0/3920409421 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb070068220 con 0x7fb0700c6330 2024-09-19T08:45:26.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.751+0000 7fb057fff640 1 -- 172.21.15.98:0/3920409421 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v132) ==== 138+0+0 (secure 0 0 0) 0x7fb060055ec0 con 0x7fb0700c6330 2024-09-19T08:45:26.757 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_metadata' created 2024-09-19T08:45:26.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.755+0000 7fb075b2b640 1 -- 172.21.15.98:0/3920409421 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb05c067e60 msgr2=0x7fb05c06a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:26.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.755+0000 7fb075b2b640 1 --2- 172.21.15.98:0/3920409421 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb05c067e60 0x7fb05c06a320 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb068001590 tx=0x7fb0680014a0 comp rx=0 tx=0).stop 2024-09-19T08:45:26.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.755+0000 7fb075b2b640 1 -- 172.21.15.98:0/3920409421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0700c6330 msgr2=0x7fb0700c67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:26.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.755+0000 7fb075b2b640 1 --2- 172.21.15.98:0/3920409421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0700c6330 0x7fb0700c67b0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fb070071830 tx=0x7fb06000f670 comp rx=0 tx=0).stop 2024-09-19T08:45:26.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.755+0000 7fb075b2b640 1 -- 172.21.15.98:0/3920409421 shutdown_connections 2024-09-19T08:45:26.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.755+0000 7fb075b2b640 1 --2- 172.21.15.98:0/3920409421 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb05c067e60 0x7fb05c06a320 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:26.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.755+0000 7fb075b2b640 1 --2- 172.21.15.98:0/3920409421 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb0700c6330 0x7fb0700c67b0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:26.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.755+0000 7fb075b2b640 1 --2- 172.21.15.98:0/3920409421 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb0700c7810 0x7fb0700c5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:26.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.755+0000 7fb075b2b640 1 --2- 172.21.15.98:0/3920409421 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb058004810 0x7fb0700d16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:26.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.755+0000 7fb075b2b640 1 -- 172.21.15.98:0/3920409421 >> 172.21.15.98:0/3920409421 conn(0x7fb0700b2e60 msgr2=0x7fb07006f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:26.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.755+0000 7fb075b2b640 1 -- 172.21.15.98:0/3920409421 shutdown_connections 2024-09-19T08:45:26.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.755+0000 7fb075b2b640 1 -- 172.21.15.98:0/3920409421 wait complete. 2024-09-19T08:45:26.785 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T08:45:26.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.911+0000 7fd9ecbd5640 1 Processor -- start 2024-09-19T08:45:26.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.911+0000 7fd9ecbd5640 1 -- start start 2024-09-19T08:45:26.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.911+0000 7fd9ecbd5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd9e8145b70 0x7fd9e806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:26.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.911+0000 7fd9ecbd5640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd9e806c1d0 0x7fd9e80713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:26.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.911+0000 7fd9ecbd5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9e80718f0 0x7fd9e80cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:26.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.911+0000 7fd9ecbd5640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fd9e806c620 con 0x7fd9e806c1d0 2024-09-19T08:45:26.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.911+0000 7fd9ecbd5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fd9e8073530 con 0x7fd9e8145b70 2024-09-19T08:45:26.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.911+0000 7fd9ecbd5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd9e8073810 con 0x7fd9e80718f0 2024-09-19T08:45:26.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.911+0000 7fd9e5d9b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd9e806c1d0 0x7fd9e80713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:26.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.911+0000 7fd9e5d9b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd9e806c1d0 0x7fd9e80713b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38948/0 (socket says 172.21.15.98:38948) 2024-09-19T08:45:26.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.911+0000 7fd9e5d9b640 1 -- 172.21.15.98:0/3148331895 learned_addr learned my addr 172.21.15.98:0/3148331895 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:26.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.911+0000 7fd9e6d9d640 1 --2- 172.21.15.98:0/3148331895 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9e80718f0 0x7fd9e80cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:26.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.911+0000 7fd9e659c640 1 --2- 172.21.15.98:0/3148331895 >> v2:172.21.15.139:3300/0 conn(0x7fd9e8145b70 0x7fd9e806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:26.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.911+0000 7fd9e5d9b640 1 -- 172.21.15.98:0/3148331895 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9e80718f0 msgr2=0x7fd9e80cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:26.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.911+0000 7fd9e5d9b640 1 --2- 172.21.15.98:0/3148331895 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9e80718f0 0x7fd9e80cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:26.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.911+0000 7fd9e5d9b640 1 -- 172.21.15.98:0/3148331895 >> v2:172.21.15.139:3300/0 conn(0x7fd9e8145b70 msgr2=0x7fd9e806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:26.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.911+0000 7fd9e5d9b640 1 --2- 172.21.15.98:0/3148331895 >> v2:172.21.15.139:3300/0 conn(0x7fd9e8145b70 0x7fd9e806bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:26.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.911+0000 7fd9e5d9b640 1 -- 172.21.15.98:0/3148331895 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e8073af0 con 0x7fd9e806c1d0 2024-09-19T08:45:26.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.915+0000 7fd9e5d9b640 1 --2- 172.21.15.98:0/3148331895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd9e806c1d0 0x7fd9e80713b0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fd9e80b2d30 tx=0x7fd9dc01c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd42910905ab3e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:26.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.915+0000 7fd9e559a640 1 -- 172.21.15.98:0/3148331895 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc0b0070 con 0x7fd9e806c1d0 2024-09-19T08:45:26.918 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.915+0000 7fd9e559a640 1 -- 172.21.15.98:0/3148331895 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9dc0ab600 con 0x7fd9e806c1d0 2024-09-19T08:45:26.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.915+0000 7fd9e559a640 1 -- 172.21.15.98:0/3148331895 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc0ab9d0 con 0x7fd9e806c1d0 2024-09-19T08:45:26.919 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.915+0000 7fd9ecbd5640 1 -- 172.21.15.98:0/3148331895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd9e806c1d0 msgr2=0x7fd9e80713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:26.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.915+0000 7fd9ecbd5640 1 --2- 172.21.15.98:0/3148331895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd9e806c1d0 0x7fd9e80713b0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fd9e80b2d30 tx=0x7fd9dc01c1d0 comp rx=0 tx=0).stop 2024-09-19T08:45:26.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.915+0000 7fd9ecbd5640 1 -- 172.21.15.98:0/3148331895 shutdown_connections 2024-09-19T08:45:26.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.915+0000 7fd9ecbd5640 1 --2- 172.21.15.98:0/3148331895 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd9e806c1d0 0x7fd9e80713b0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:26.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.915+0000 7fd9ecbd5640 1 --2- 172.21.15.98:0/3148331895 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9e80718f0 0x7fd9e80cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:26.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.915+0000 7fd9ecbd5640 1 --2- 172.21.15.98:0/3148331895 >> v2:172.21.15.139:3300/0 conn(0x7fd9e8145b70 0x7fd9e806bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:26.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.915+0000 7fd9ecbd5640 1 -- 172.21.15.98:0/3148331895 >> 172.21.15.98:0/3148331895 conn(0x7fd9e80b2010 msgr2=0x7fd9e80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:26.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.915+0000 7fd9ecbd5640 1 -- 172.21.15.98:0/3148331895 shutdown_connections 2024-09-19T08:45:26.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.915+0000 7fd9ecbd5640 1 -- 172.21.15.98:0/3148331895 wait complete. 2024-09-19T08:45:26.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.919+0000 7fd9ecbd5640 1 Processor -- start 2024-09-19T08:45:26.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.919+0000 7fd9ecbd5640 1 -- start start 2024-09-19T08:45:26.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.919+0000 7fd9ecbd5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9e806c1d0 0x7fd9e815a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:26.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.919+0000 7fd9ecbd5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd9e80718f0 0x7fd9e80b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:26.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.919+0000 7fd9ecbd5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9e8145b70 0x7fd9e8140550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:26.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.919+0000 7fd9ecbd5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e80b3410 con 0x7fd9e80718f0 2024-09-19T08:45:26.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.919+0000 7fd9ecbd5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e80b3590 con 0x7fd9e806c1d0 2024-09-19T08:45:26.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.919+0000 7fd9ecbd5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd9e80b3710 con 0x7fd9e8145b70 2024-09-19T08:45:26.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.919+0000 7fd9e5d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd9e80718f0 0x7fd9e80b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:26.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.919+0000 7fd9e5d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd9e80718f0 0x7fd9e80b3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:38950/0 (socket says 172.21.15.98:38950) 2024-09-19T08:45:26.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.919+0000 7fd9e5d9b640 1 -- 172.21.15.98:0/2930762389 learned_addr learned my addr 172.21.15.98:0/2930762389 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:26.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.919+0000 7fd9e659c640 1 --2- 172.21.15.98:0/2930762389 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9e806c1d0 0x7fd9e815a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:26.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.919+0000 7fd9e6d9d640 1 --2- 172.21.15.98:0/2930762389 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9e8145b70 0x7fd9e8140550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:26.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.919+0000 7fd9e5d9b640 1 -- 172.21.15.98:0/2930762389 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9e8145b70 msgr2=0x7fd9e8140550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:26.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.919+0000 7fd9e5d9b640 1 --2- 172.21.15.98:0/2930762389 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9e8145b70 0x7fd9e8140550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:26.923 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.919+0000 7fd9e5d9b640 1 -- 172.21.15.98:0/2930762389 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9e806c1d0 msgr2=0x7fd9e815a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:26.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.919+0000 7fd9e5d9b640 1 --2- 172.21.15.98:0/2930762389 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9e806c1d0 0x7fd9e815a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:26.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.919+0000 7fd9e5d9b640 1 -- 172.21.15.98:0/2930762389 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e813fd90 con 0x7fd9e80718f0 2024-09-19T08:45:26.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.919+0000 7fd9e6d9d640 1 --2- 172.21.15.98:0/2930762389 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9e8145b70 0x7fd9e8140550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:26.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.919+0000 7fd9e659c640 1 --2- 172.21.15.98:0/2930762389 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9e806c1d0 0x7fd9e815a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:26.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.919+0000 7fd9e5d9b640 1 --2- 172.21.15.98:0/2930762389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd9e80718f0 0x7fd9e80b3b70 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc0ae040 tx=0x7fd9dc018f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:26.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.919+0000 7fd9d6ffd640 1 -- 172.21.15.98:0/2930762389 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc0b0040 con 0x7fd9e80718f0 2024-09-19T08:45:26.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.919+0000 7fd9d6ffd640 1 -- 172.21.15.98:0/2930762389 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9dc0b9070 con 0x7fd9e80718f0 2024-09-19T08:45:26.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.919+0000 7fd9ecbd5640 1 -- 172.21.15.98:0/2930762389 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9e80b4270 con 0x7fd9e80718f0 2024-09-19T08:45:26.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.919+0000 7fd9d6ffd640 1 -- 172.21.15.98:0/2930762389 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc0b4680 con 0x7fd9e80718f0 2024-09-19T08:45:26.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.919+0000 7fd9ecbd5640 1 -- 172.21.15.98:0/2930762389 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9e8140030 con 0x7fd9e80718f0 2024-09-19T08:45:26.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.919+0000 7fd9ecbd5640 1 -- 172.21.15.98:0/2930762389 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9e80cef70 con 0x7fd9e80718f0 2024-09-19T08:45:26.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.923+0000 7fd9d6ffd640 1 -- 172.21.15.98:0/2930762389 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fd9dc0b4820 con 0x7fd9e80718f0 2024-09-19T08:45:26.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.923+0000 7fd9d6ffd640 1 --2- 172.21.15.98:0/2930762389 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd9c8067e60 0x7fd9c806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:26.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.923+0000 7fd9d6ffd640 1 -- 172.21.15.98:0/2930762389 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5255+0+0 (secure 0 0 0) 0x7fd9dc120d40 con 0x7fd9e80718f0 2024-09-19T08:45:26.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.923+0000 7fd9e659c640 1 --2- 172.21.15.98:0/2930762389 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd9c8067e60 0x7fd9c806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:26.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.923+0000 7fd9e659c640 1 --2- 172.21.15.98:0/2930762389 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd9c8067e60 0x7fd9c806a320 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd9d800b9f0 tx=0x7fd9d8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:26.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:26.927+0000 7fd9d6ffd640 1 -- 172.21.15.98:0/2930762389 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd9dc016110 con 0x7fd9e80718f0 2024-09-19T08:45:27.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.095+0000 7fd9ecbd5640 1 -- 172.21.15.98:0/2930762389 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fd9e80b4790 con 0x7fd9e80718f0 2024-09-19T08:45:27.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.763+0000 7fd9d6ffd640 1 -- 172.21.15.98:0/2930762389 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v133) ==== 170+0+0 (secure 0 0 0) 0x7fd9dc0f11c0 con 0x7fd9e80718f0 2024-09-19T08:45:27.767 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' created 2024-09-19T08:45:27.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.767+0000 7fd9ecbd5640 1 -- 172.21.15.98:0/2930762389 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd9c8067e60 msgr2=0x7fd9c806a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:27.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.767+0000 7fd9ecbd5640 1 --2- 172.21.15.98:0/2930762389 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd9c8067e60 0x7fd9c806a320 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd9d800b9f0 tx=0x7fd9d8008040 comp rx=0 tx=0).stop 2024-09-19T08:45:27.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.767+0000 7fd9ecbd5640 1 -- 172.21.15.98:0/2930762389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd9e80718f0 msgr2=0x7fd9e80b3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:27.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.767+0000 7fd9ecbd5640 1 --2- 172.21.15.98:0/2930762389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd9e80718f0 0x7fd9e80b3b70 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc0ae040 tx=0x7fd9dc018f70 comp rx=0 tx=0).stop 2024-09-19T08:45:27.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.767+0000 7fd9ecbd5640 1 -- 172.21.15.98:0/2930762389 shutdown_connections 2024-09-19T08:45:27.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.767+0000 7fd9ecbd5640 1 --2- 172.21.15.98:0/2930762389 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd9c8067e60 0x7fd9c806a320 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:27.771 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.767+0000 7fd9ecbd5640 1 --2- 172.21.15.98:0/2930762389 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9e8145b70 0x7fd9e8140550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:27.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.767+0000 7fd9ecbd5640 1 --2- 172.21.15.98:0/2930762389 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd9e80718f0 0x7fd9e80b3b70 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:27.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.767+0000 7fd9ecbd5640 1 --2- 172.21.15.98:0/2930762389 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9e806c1d0 0x7fd9e815a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:27.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.767+0000 7fd9ecbd5640 1 -- 172.21.15.98:0/2930762389 >> 172.21.15.98:0/2930762389 conn(0x7fd9e80b2010 msgr2=0x7fd9e80b0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:27.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.767+0000 7fd9ecbd5640 1 -- 172.21.15.98:0/2930762389 shutdown_connections 2024-09-19T08:45:27.772 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.767+0000 7fd9ecbd5640 1 -- 172.21.15.98:0/2930762389 wait complete. 2024-09-19T08:45:27.801 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T08:45:27.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.923+0000 7f01b8a64640 1 Processor -- start 2024-09-19T08:45:27.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.923+0000 7f01b8a64640 1 -- start start 2024-09-19T08:45:27.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.923+0000 7f01b8a64640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f01b4142540 0x7f01b40ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:27.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.923+0000 7f01b8a64640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f01b4072a80 0x7f01b415fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:27.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.923+0000 7f01b8a64640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01b40d0500 con 0x7f01b41455a0 2024-09-19T08:45:27.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.923+0000 7f01b8a64640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f01b40d02b0 con 0x7f01b4072a80 2024-09-19T08:45:27.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.923+0000 7f01b8a64640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f01b4073920 con 0x7f01b4142540 2024-09-19T08:45:27.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.923+0000 7f01b2d9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f01b4072a80 0x7f01b415fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:27.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.923+0000 7f01b2d9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f01b4072a80 0x7f01b415fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35414/0 (socket says 172.21.15.98:35414) 2024-09-19T08:45:27.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.923+0000 7f01b2d9d640 1 -- 172.21.15.98:0/1969380321 learned_addr learned my addr 172.21.15.98:0/1969380321 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:27.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.923+0000 7f01b259c640 1 --2- 172.21.15.98:0/1969380321 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f01b4142540 0x7f01b40ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:27.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.923+0000 7f01b2d9d640 1 -- 172.21.15.98:0/1969380321 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f01b4142540 msgr2=0x7f01b40ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:27.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.923+0000 7f01b2d9d640 1 --2- 172.21.15.98:0/1969380321 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f01b4142540 0x7f01b40ce5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:27.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.923+0000 7f01b2d9d640 1 -- 172.21.15.98:0/1969380321 >> v1:172.21.15.98:6789/0 conn(0x7f01b41455a0 legacy=0x7f01b4072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:27.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.923+0000 7f01b2d9d640 1 -- 172.21.15.98:0/1969380321 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b40732d0 con 0x7f01b4072a80 2024-09-19T08:45:27.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.927+0000 7f01b2d9d640 1 --2- 172.21.15.98:0/1969380321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01b4072a80 0x7f01b415fe50 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f01a4009870 tx=0x7f01a401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5819823a0504b3bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:27.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.927+0000 7f01b159a640 1 -- 172.21.15.98:0/1969380321 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a40a7070 con 0x7f01b4072a80 2024-09-19T08:45:27.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.927+0000 7f01b159a640 1 -- 172.21.15.98:0/1969380321 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a40b0070 con 0x7f01b4072a80 2024-09-19T08:45:27.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.927+0000 7f01b159a640 1 -- 172.21.15.98:0/1969380321 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a40ab890 con 0x7f01b4072a80 2024-09-19T08:45:27.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.927+0000 7f01b8a64640 1 -- 172.21.15.98:0/1969380321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01b4072a80 msgr2=0x7f01b415fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:27.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.927+0000 7f01b8a64640 1 --2- 172.21.15.98:0/1969380321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01b4072a80 0x7f01b415fe50 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f01a4009870 tx=0x7f01a401c560 comp rx=0 tx=0).stop 2024-09-19T08:45:27.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.927+0000 7f01b8a64640 1 -- 172.21.15.98:0/1969380321 shutdown_connections 2024-09-19T08:45:27.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.927+0000 7f01b8a64640 1 --2- 172.21.15.98:0/1969380321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01b4072a80 0x7f01b415fe50 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:27.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.927+0000 7f01b8a64640 1 --2- 172.21.15.98:0/1969380321 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f01b4142540 0x7f01b40ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:27.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.927+0000 7f01b8a64640 1 -- 172.21.15.98:0/1969380321 >> 172.21.15.98:0/1969380321 conn(0x7f01b40d0a10 msgr2=0x7f01b40d0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:27.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.927+0000 7f01b8a64640 1 -- 172.21.15.98:0/1969380321 shutdown_connections 2024-09-19T08:45:27.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.927+0000 7f01b8a64640 1 -- 172.21.15.98:0/1969380321 wait complete. 2024-09-19T08:45:27.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.927+0000 7f01b8a64640 1 Processor -- start 2024-09-19T08:45:27.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.927+0000 7f01b8a64640 1 -- start start 2024-09-19T08:45:27.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.927+0000 7f01b8a64640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01b4072a80 0x7f01b41fa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:27.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.927+0000 7f01b8a64640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01b4142540 0x7f01b41fab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:27.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.927+0000 7f01b8a64640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f01b41455a0 0x7f01b415e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:27.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.927+0000 7f01b8a64640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f01b4169160 con 0x7f01b4072a80 2024-09-19T08:45:27.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.927+0000 7f01b8a64640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f01b4168fe0 con 0x7f01b4142540 2024-09-19T08:45:27.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.927+0000 7f01b8a64640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f01b41692e0 con 0x7f01b41455a0 2024-09-19T08:45:27.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.931+0000 7f01b259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01b4072a80 0x7f01b41fa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:27.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.931+0000 7f01b259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01b4072a80 0x7f01b41fa5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35422/0 (socket says 172.21.15.98:35422) 2024-09-19T08:45:27.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.931+0000 7f01b259c640 1 -- 172.21.15.98:0/848875499 learned_addr learned my addr 172.21.15.98:0/848875499 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:27.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.931+0000 7f01b1d9b640 1 --2- 172.21.15.98:0/848875499 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01b4142540 0x7f01b41fab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:27.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.931+0000 7f01b2d9d640 1 --2- 172.21.15.98:0/848875499 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f01b41455a0 0x7f01b415e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:27.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.931+0000 7f01b259c640 1 -- 172.21.15.98:0/848875499 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f01b41455a0 msgr2=0x7f01b415e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:27.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.931+0000 7f01b259c640 1 --2- 172.21.15.98:0/848875499 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f01b41455a0 0x7f01b415e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:27.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.931+0000 7f01b259c640 1 -- 172.21.15.98:0/848875499 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01b4142540 msgr2=0x7f01b41fab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:27.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.931+0000 7f01b259c640 1 --2- 172.21.15.98:0/848875499 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01b4142540 0x7f01b41fab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:27.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.931+0000 7f01b259c640 1 -- 172.21.15.98:0/848875499 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b415f300 con 0x7f01b4072a80 2024-09-19T08:45:27.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.931+0000 7f01b259c640 1 --2- 172.21.15.98:0/848875499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01b4072a80 0x7f01b41fa5f0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f01ac007ec0 tx=0x7f01ac00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:27.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.931+0000 7f01aaffd640 1 -- 172.21.15.98:0/848875499 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ac014620 con 0x7f01b4072a80 2024-09-19T08:45:27.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.931+0000 7f01aaffd640 1 -- 172.21.15.98:0/848875499 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01ac019070 con 0x7f01b4072a80 2024-09-19T08:45:27.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.931+0000 7f01b8a64640 1 -- 172.21.15.98:0/848875499 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01b415f5e0 con 0x7f01b4072a80 2024-09-19T08:45:27.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.931+0000 7f01aaffd640 1 -- 172.21.15.98:0/848875499 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ac014a20 con 0x7f01b4072a80 2024-09-19T08:45:27.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.931+0000 7f01b8a64640 1 -- 172.21.15.98:0/848875499 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01b415fba0 con 0x7f01b4072a80 2024-09-19T08:45:27.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.931+0000 7f01b8a64640 1 -- 172.21.15.98:0/848875499 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01b415fe50 con 0x7f01b4072a80 2024-09-19T08:45:27.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.931+0000 7f01aaffd640 1 -- 172.21.15.98:0/848875499 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f01ac014bc0 con 0x7f01b4072a80 2024-09-19T08:45:27.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.931+0000 7f01aaffd640 1 --2- 172.21.15.98:0/848875499 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0194067e60 0x7f019406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:27.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.931+0000 7f01aaffd640 1 -- 172.21.15.98:0/848875499 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5637+0+0 (secure 0 0 0) 0x7f01ac087b50 con 0x7f01b4072a80 2024-09-19T08:45:27.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.931+0000 7f01b1d9b640 1 --2- 172.21.15.98:0/848875499 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0194067e60 0x7f019406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:27.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.935+0000 7f01b1d9b640 1 --2- 172.21.15.98:0/848875499 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0194067e60 0x7f019406a320 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f01b4166ad0 tx=0x7f019c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:27.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:27.935+0000 7f01aaffd640 1 -- 172.21.15.98:0/848875499 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f01ac057ef0 con 0x7f01b4072a80 2024-09-19T08:45:28.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.131+0000 7f01b8a64640 1 -- 172.21.15.98:0/848875499 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f01b40b4af0 con 0x7f01b4072a80 2024-09-19T08:45:28.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.767+0000 7f01aaffd640 1 -- 172.21.15.98:0/848875499 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v72) ==== 174+0+0 (secure 0 0 0) 0x7f01ac05e0c0 con 0x7f01b4072a80 2024-09-19T08:45:28.770 INFO:teuthology.orchestra.run.smithi098.stderr:new fs with metadata pool 20 and data pool 21 2024-09-19T08:45:28.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.771+0000 7f01b8a64640 1 -- 172.21.15.98:0/848875499 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0194067e60 msgr2=0x7f019406a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:28.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.771+0000 7f01b8a64640 1 --2- 172.21.15.98:0/848875499 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0194067e60 0x7f019406a320 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f01b4166ad0 tx=0x7f019c009040 comp rx=0 tx=0).stop 2024-09-19T08:45:28.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.771+0000 7f01b8a64640 1 -- 172.21.15.98:0/848875499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01b4072a80 msgr2=0x7f01b41fa5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:28.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.771+0000 7f01b8a64640 1 --2- 172.21.15.98:0/848875499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01b4072a80 0x7f01b41fa5f0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f01ac007ec0 tx=0x7f01ac00e4b0 comp rx=0 tx=0).stop 2024-09-19T08:45:28.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.771+0000 7f01b8a64640 1 -- 172.21.15.98:0/848875499 shutdown_connections 2024-09-19T08:45:28.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.771+0000 7f01b8a64640 1 --2- 172.21.15.98:0/848875499 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0194067e60 0x7f019406a320 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:28.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.771+0000 7f01b8a64640 1 --2- 172.21.15.98:0/848875499 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f01b41455a0 0x7f01b415e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:28.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.771+0000 7f01b8a64640 1 --2- 172.21.15.98:0/848875499 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01b4142540 0x7f01b41fab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:28.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.771+0000 7f01b8a64640 1 --2- 172.21.15.98:0/848875499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01b4072a80 0x7f01b41fa5f0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:28.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.771+0000 7f01b8a64640 1 -- 172.21.15.98:0/848875499 >> 172.21.15.98:0/848875499 conn(0x7f01b40d0a10 msgr2=0x7f01b40d0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:28.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.771+0000 7f01b8a64640 1 -- 172.21.15.98:0/848875499 shutdown_connections 2024-09-19T08:45:28.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.771+0000 7f01b8a64640 1 -- 172.21.15.98:0/848875499 wait complete. 2024-09-19T08:45:28.813 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:45:28.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.927+0000 7f7859732640 1 Processor -- start 2024-09-19T08:45:28.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.927+0000 7f7859732640 1 -- start start 2024-09-19T08:45:28.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.931+0000 7f7859732640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f785406bc90 0x7f785406c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:28.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.931+0000 7f7859732640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78540ce840 0x7f78540cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:28.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.931+0000 7f7859732640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78540ccc60 con 0x7f7854158fd0 2024-09-19T08:45:28.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.931+0000 7f7859732640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f78541593b0 con 0x7f785406bc90 2024-09-19T08:45:28.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.931+0000 7f7859732640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f78540ccf40 con 0x7f78540ce840 2024-09-19T08:45:28.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.931+0000 7f7852ffd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f7854158fd0 0x7f78540b3e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:43962/0 (socket says 172.21.15.98:43962) 2024-09-19T08:45:28.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.931+0000 7f7852ffd640 1 -- 172.21.15.98:0/944973516 learned_addr learned my addr 172.21.15.98:0/944973516 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:28.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.931+0000 7f7851ffb640 1 -- 172.21.15.98:0/944973516 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3482856154 0 0) 0x7f78540ccc60 con 0x7f7854158fd0 2024-09-19T08:45:28.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.931+0000 7f78527fc640 1 --2- 172.21.15.98:0/944973516 >> v2:172.21.15.139:3300/0 conn(0x7f785406bc90 0x7f785406c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:28.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.931+0000 7f78537fe640 1 --2- 172.21.15.98:0/944973516 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78540ce840 0x7f78540cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:28.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.931+0000 7f7851ffb640 1 -- 172.21.15.98:0/944973516 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f783c003530 con 0x7f7854158fd0 2024-09-19T08:45:28.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.931+0000 7f7851ffb640 1 -- 172.21.15.98:0/944973516 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2137005197 0 0) 0x7f783c003530 con 0x7f7854158fd0 2024-09-19T08:45:28.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.931+0000 7f7851ffb640 1 -- 172.21.15.98:0/944973516 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78540ce840 msgr2=0x7f78540cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:28.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.931+0000 7f7851ffb640 1 --2- 172.21.15.98:0/944973516 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78540ce840 0x7f78540cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:28.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.931+0000 7f7851ffb640 1 -- 172.21.15.98:0/944973516 >> v2:172.21.15.139:3300/0 conn(0x7f785406bc90 msgr2=0x7f785406c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:28.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.931+0000 7f7851ffb640 1 --2- 172.21.15.98:0/944973516 >> v2:172.21.15.139:3300/0 conn(0x7f785406bc90 0x7f785406c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:28.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.931+0000 7f7851ffb640 1 -- 172.21.15.98:0/944973516 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78540cd220 con 0x7f7854158fd0 2024-09-19T08:45:28.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.931+0000 7f7851ffb640 1 -- 172.21.15.98:0/944973516 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f7848001680 con 0x7f7854158fd0 2024-09-19T08:45:28.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.931+0000 7f7851ffb640 1 -- 172.21.15.98:0/944973516 >> v1:172.21.15.98:6789/0 conn(0x7f7854158fd0 legacy=0x7f78540b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:28.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.931+0000 7f7851ffb640 1 --2- 172.21.15.98:0/944973516 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f783c0039c0 0x7f783c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:28.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.931+0000 7f7851ffb640 1 --2- 172.21.15.98:0/944973516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f783c004810 0x7f783c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:28.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.931+0000 7f7851ffb640 1 --2- 172.21.15.98:0/944973516 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78540ce840 0x7f783c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:28.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.931+0000 7f7851ffb640 1 -- 172.21.15.98:0/944973516 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f78541593b0 con 0x7f783c0039c0 2024-09-19T08:45:28.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.931+0000 7f7851ffb640 1 -- 172.21.15.98:0/944973516 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f78540ccf40 con 0x7f783c004810 2024-09-19T08:45:28.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.931+0000 7f7851ffb640 1 -- 172.21.15.98:0/944973516 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f783c00bab0 con 0x7f78540ce840 2024-09-19T08:45:28.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.931+0000 7f78527fc640 1 --2- 172.21.15.98:0/944973516 >> v2:172.21.15.139:3300/0 conn(0x7f785406bc90 0x7f785406c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:28.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.931+0000 7f7852ffd640 1 --2- 172.21.15.98:0/944973516 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f783c0039c0 0x7f783c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:28.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.931+0000 7f78537fe640 1 --2- 172.21.15.98:0/944973516 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78540ce840 0x7f783c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:28.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.931+0000 7f7852ffd640 1 -- 172.21.15.98:0/944973516 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78540ce840 msgr2=0x7f783c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:28.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.931+0000 7f7852ffd640 1 --2- 172.21.15.98:0/944973516 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f78540ce840 0x7f783c0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:28.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.931+0000 7f78527fc640 1 --2- 172.21.15.98:0/944973516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f783c004810 0x7f783c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:28.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.931+0000 7f7852ffd640 1 -- 172.21.15.98:0/944973516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f783c004810 msgr2=0x7f783c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:28.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.931+0000 7f7852ffd640 1 --2- 172.21.15.98:0/944973516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f783c004810 0x7f783c004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:28.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.931+0000 7f7852ffd640 1 -- 172.21.15.98:0/944973516 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f783c003530 con 0x7f783c0039c0 2024-09-19T08:45:28.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.931+0000 7f7852ffd640 1 --2- 172.21.15.98:0/944973516 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f783c0039c0 0x7f783c003de0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f784800c210 tx=0x7f784801cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b105f9e13a9b20cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:28.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.935+0000 7f7851ffb640 1 -- 172.21.15.98:0/944973516 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7848007d60 con 0x7f783c0039c0 2024-09-19T08:45:28.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.935+0000 7f7851ffb640 1 -- 172.21.15.98:0/944973516 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78480a9070 con 0x7f783c0039c0 2024-09-19T08:45:28.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.935+0000 7f7851ffb640 1 -- 172.21.15.98:0/944973516 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7848005d40 con 0x7f783c0039c0 2024-09-19T08:45:28.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.935+0000 7f7859732640 1 -- 172.21.15.98:0/944973516 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f783c0039c0 msgr2=0x7f783c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:28.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.935+0000 7f7859732640 1 --2- 172.21.15.98:0/944973516 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f783c0039c0 0x7f783c003de0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f784800c210 tx=0x7f784801cb80 comp rx=0 tx=0).stop 2024-09-19T08:45:28.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.935+0000 7f78537fe640 1 -- 172.21.15.98:0/944973516 reap_dead start 2024-09-19T08:45:28.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.935+0000 7f7859732640 1 -- 172.21.15.98:0/944973516 shutdown_connections 2024-09-19T08:45:28.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.935+0000 7f7859732640 1 -- 172.21.15.98:0/944973516 >> 172.21.15.98:0/944973516 conn(0x7f78540b2010 msgr2=0x7f78540b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:28.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.935+0000 7f7859732640 1 -- 172.21.15.98:0/944973516 shutdown_connections 2024-09-19T08:45:28.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.935+0000 7f7859732640 1 -- 172.21.15.98:0/944973516 wait complete. 2024-09-19T08:45:28.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.935+0000 7f7859732640 1 Processor -- start 2024-09-19T08:45:28.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.935+0000 7f7859732640 1 -- start start 2024-09-19T08:45:28.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.935+0000 7f7859732640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f783c0039c0 0x7f7854071f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:28.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.935+0000 7f7859732640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7854155f10 0x7f7854160730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:28.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.935+0000 7f7859732640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7854160c70 0x7f78540cef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:28.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.935+0000 7f7859732640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f785415c920 con 0x7f783c0039c0 2024-09-19T08:45:28.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.935+0000 7f7859732640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f785415c7a0 con 0x7f7854155f10 2024-09-19T08:45:28.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.935+0000 7f7859732640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7854166910 con 0x7f7854160c70 2024-09-19T08:45:28.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.935+0000 7f7852ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f783c0039c0 0x7f7854071f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:28.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.935+0000 7f7852ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f783c0039c0 0x7f7854071f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35438/0 (socket says 172.21.15.98:35438) 2024-09-19T08:45:28.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.935+0000 7f7852ffd640 1 -- 172.21.15.98:0/477826907 learned_addr learned my addr 172.21.15.98:0/477826907 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:28.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.935+0000 7f78527fc640 1 --2- 172.21.15.98:0/477826907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7854155f10 0x7f7854160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:28.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.935+0000 7f78537fe640 1 --2- 172.21.15.98:0/477826907 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7854160c70 0x7f78540cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:28.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.935+0000 7f7852ffd640 1 -- 172.21.15.98:0/477826907 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7854160c70 msgr2=0x7f78540cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:28.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.935+0000 7f7852ffd640 1 --2- 172.21.15.98:0/477826907 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7854160c70 0x7f78540cef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:28.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.935+0000 7f7852ffd640 1 -- 172.21.15.98:0/477826907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7854155f10 msgr2=0x7f7854160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:28.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.935+0000 7f7852ffd640 1 --2- 172.21.15.98:0/477826907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7854155f10 0x7f7854160730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:28.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.935+0000 7f7852ffd640 1 -- 172.21.15.98:0/477826907 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7854156390 con 0x7f783c0039c0 2024-09-19T08:45:28.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.935+0000 7f7852ffd640 1 --2- 172.21.15.98:0/477826907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f783c0039c0 0x7f7854071f50 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f784801c5f0 tx=0x7f783c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:28.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.935+0000 7f783b7fe640 1 -- 172.21.15.98:0/477826907 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78480047f0 con 0x7f783c0039c0 2024-09-19T08:45:28.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.935+0000 7f783b7fe640 1 -- 172.21.15.98:0/477826907 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7848004990 con 0x7f783c0039c0 2024-09-19T08:45:28.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.935+0000 7f7859732640 1 -- 172.21.15.98:0/477826907 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78541611f0 con 0x7f783c0039c0 2024-09-19T08:45:28.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.935+0000 7f783b7fe640 1 -- 172.21.15.98:0/477826907 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7848004d40 con 0x7f783c0039c0 2024-09-19T08:45:28.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.935+0000 7f7859732640 1 -- 172.21.15.98:0/477826907 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7854071750 con 0x7f783c0039c0 2024-09-19T08:45:28.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.939+0000 7f7859732640 1 -- 172.21.15.98:0/477826907 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7854166c70 con 0x7f783c0039c0 2024-09-19T08:45:28.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.939+0000 7f783b7fe640 1 -- 172.21.15.98:0/477826907 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f78480b7030 con 0x7f783c0039c0 2024-09-19T08:45:28.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.939+0000 7f783b7fe640 1 --2- 172.21.15.98:0/477826907 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7840067e60 0x7f784006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:28.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.939+0000 7f783b7fe640 1 -- 172.21.15.98:0/477826907 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5737+0+0 (secure 0 0 0) 0x7f784811f5d0 con 0x7f783c0039c0 2024-09-19T08:45:28.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.939+0000 7f78527fc640 1 --2- 172.21.15.98:0/477826907 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7840067e60 0x7f784006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:28.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.939+0000 7f78527fc640 1 --2- 172.21.15.98:0/477826907 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7840067e60 0x7f784006a320 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7844003fd0 tx=0x7f784400b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:28.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:28.943+0000 7f783b7fe640 1 -- 172.21.15.98:0/477826907 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f7848016890 con 0x7f783c0039c0 2024-09-19T08:45:29.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.107+0000 7f7859732640 1 -- 172.21.15.98:0/477826907 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f78540742b0 con 0x7f783c0039c0 2024-09-19T08:45:29.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.107+0000 7f783b7fe640 1 -- 172.21.15.98:0/477826907 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) ==== 74+0+16796 (secure 0 0 0) 0x7f78480efb70 con 0x7f783c0039c0 2024-09-19T08:45:29.112 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:45:29.112 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":134,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:45:28.140031+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:45:25.813513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-19T08:45:27.137275+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:45:29.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.111+0000 7f7859732640 1 -- 172.21.15.98:0/477826907 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7840067e60 msgr2=0x7f784006a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:29.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.111+0000 7f7859732640 1 --2- 172.21.15.98:0/477826907 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7840067e60 0x7f784006a320 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7844003fd0 tx=0x7f784400b040 comp rx=0 tx=0).stop 2024-09-19T08:45:29.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.111+0000 7f7859732640 1 -- 172.21.15.98:0/477826907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f783c0039c0 msgr2=0x7f7854071f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:29.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.111+0000 7f7859732640 1 --2- 172.21.15.98:0/477826907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f783c0039c0 0x7f7854071f50 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f784801c5f0 tx=0x7f783c005840 comp rx=0 tx=0).stop 2024-09-19T08:45:29.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.111+0000 7f7859732640 1 -- 172.21.15.98:0/477826907 shutdown_connections 2024-09-19T08:45:29.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.111+0000 7f7859732640 1 --2- 172.21.15.98:0/477826907 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7840067e60 0x7f784006a320 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.111+0000 7f7859732640 1 --2- 172.21.15.98:0/477826907 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7854160c70 0x7f78540cef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.114 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.111+0000 7f7859732640 1 --2- 172.21.15.98:0/477826907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7854155f10 0x7f7854160730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.111+0000 7f7859732640 1 --2- 172.21.15.98:0/477826907 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f783c0039c0 0x7f7854071f50 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.111+0000 7f7859732640 1 -- 172.21.15.98:0/477826907 >> 172.21.15.98:0/477826907 conn(0x7f78540b2010 msgr2=0x7f78540b48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:29.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.111+0000 7f7859732640 1 -- 172.21.15.98:0/477826907 shutdown_connections 2024-09-19T08:45:29.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.111+0000 7f7859732640 1 -- 172.21.15.98:0/477826907 wait complete. 2024-09-19T08:45:29.132 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:45:29.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.247+0000 7f07e88a6640 1 Processor -- start 2024-09-19T08:45:29.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.247+0000 7f07e88a6640 1 -- start start 2024-09-19T08:45:29.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.247+0000 7f07e88a6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f07e40d4820 0x7f07e41621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:29.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.247+0000 7f07e88a6640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07e4068440 con 0x7f07e40b7f80 2024-09-19T08:45:29.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.247+0000 7f07e88a6640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07e4162a90 con 0x7f07e40b9970 2024-09-19T08:45:29.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.247+0000 7f07e88a6640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f07e4163050 con 0x7f07e40d4820 2024-09-19T08:45:29.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.247+0000 7f07e259c640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f07e40b7f80 0x7f07e415f560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:43972/0 (socket says 172.21.15.98:43972) 2024-09-19T08:45:29.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.247+0000 7f07e259c640 1 -- 172.21.15.98:0/2098205361 learned_addr learned my addr 172.21.15.98:0/2098205361 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:29.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.247+0000 7f07e1d9b640 1 --2- 172.21.15.98:0/2098205361 >> v2:172.21.15.139:3300/0 conn(0x7f07e40d4820 0x7f07e41621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:29.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.247+0000 7f07e159a640 1 -- 172.21.15.98:0/2098205361 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3415575157 0 0) 0x7f07e4068440 con 0x7f07e40b7f80 2024-09-19T08:45:29.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.247+0000 7f07e159a640 1 -- 172.21.15.98:0/2098205361 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07cc003530 con 0x7f07e40b7f80 2024-09-19T08:45:29.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.247+0000 7f07e159a640 1 -- 172.21.15.98:0/2098205361 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4102111083 0 0) 0x7f07cc003530 con 0x7f07e40b7f80 2024-09-19T08:45:29.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.247+0000 7f07e159a640 1 -- 172.21.15.98:0/2098205361 >> v2:172.21.15.139:3300/0 conn(0x7f07e40d4820 msgr2=0x7f07e41621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:29.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.247+0000 7f07e159a640 1 --2- 172.21.15.98:0/2098205361 >> v2:172.21.15.139:3300/0 conn(0x7f07e40d4820 0x7f07e41621b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.247+0000 7f07e159a640 1 -- 172.21.15.98:0/2098205361 >> v1:172.21.15.139:6789/0 conn(0x7f07e40b9970 legacy=0x7f07e40b9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:29.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.247+0000 7f07e159a640 1 -- 172.21.15.98:0/2098205361 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e4162d70 con 0x7f07e40b7f80 2024-09-19T08:45:29.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.247+0000 7f07e159a640 1 -- 172.21.15.98:0/2098205361 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f07d4001680 con 0x7f07e40b7f80 2024-09-19T08:45:29.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.247+0000 7f07e159a640 1 -- 172.21.15.98:0/2098205361 >> v1:172.21.15.98:6789/0 conn(0x7f07e40b7f80 legacy=0x7f07e415f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:29.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.247+0000 7f07e159a640 1 --2- 172.21.15.98:0/2098205361 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07cc0039c0 0x7f07cc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:29.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.247+0000 7f07e159a640 1 --2- 172.21.15.98:0/2098205361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07cc004810 0x7f07cc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:29.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.247+0000 7f07e159a640 1 --2- 172.21.15.98:0/2098205361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07cc0051d0 0x7f07cc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:29.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.247+0000 7f07e159a640 1 -- 172.21.15.98:0/2098205361 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f07e4163050 con 0x7f07cc004810 2024-09-19T08:45:29.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.247+0000 7f07e159a640 1 -- 172.21.15.98:0/2098205361 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f07cc00c170 con 0x7f07cc0051d0 2024-09-19T08:45:29.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.247+0000 7f07e159a640 1 -- 172.21.15.98:0/2098205361 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f07cc00c370 con 0x7f07cc0039c0 2024-09-19T08:45:29.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.247+0000 7f07e1d9b640 1 --2- 172.21.15.98:0/2098205361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07cc004810 0x7f07cc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:29.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.247+0000 7f07e259c640 1 --2- 172.21.15.98:0/2098205361 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07cc0039c0 0x7f07cc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:29.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.247+0000 7f07e2d9d640 1 --2- 172.21.15.98:0/2098205361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07cc0051d0 0x7f07cc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:29.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.247+0000 7f07e1d9b640 1 -- 172.21.15.98:0/2098205361 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07cc0039c0 msgr2=0x7f07cc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:29.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.247+0000 7f07e1d9b640 1 --2- 172.21.15.98:0/2098205361 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07cc0039c0 0x7f07cc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.247+0000 7f07e1d9b640 1 -- 172.21.15.98:0/2098205361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07cc0051d0 msgr2=0x7f07cc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:29.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.247+0000 7f07e1d9b640 1 --2- 172.21.15.98:0/2098205361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07cc0051d0 0x7f07cc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.247+0000 7f07e1d9b640 1 -- 172.21.15.98:0/2098205361 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07cc003530 con 0x7f07cc004810 2024-09-19T08:45:29.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.247+0000 7f07e2d9d640 1 -- 172.21.15.98:0/2098205361 reap_dead start 2024-09-19T08:45:29.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.251+0000 7f07e1d9b640 1 --2- 172.21.15.98:0/2098205361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07cc004810 0x7f07cc004c90 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f07d8009ae0 tx=0x7f07d801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf3e6b70d93b89e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:29.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.251+0000 7f07e159a640 1 -- 172.21.15.98:0/2098205361 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d80b2070 con 0x7f07cc004810 2024-09-19T08:45:29.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.251+0000 7f07e159a640 1 -- 172.21.15.98:0/2098205361 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07d80a5550 con 0x7f07cc004810 2024-09-19T08:45:29.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.251+0000 7f07e159a640 1 -- 172.21.15.98:0/2098205361 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d80096b0 con 0x7f07cc004810 2024-09-19T08:45:29.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.251+0000 7f07e88a6640 1 -- 172.21.15.98:0/2098205361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07cc004810 msgr2=0x7f07cc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:29.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.251+0000 7f07e88a6640 1 --2- 172.21.15.98:0/2098205361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07cc004810 0x7f07cc004c90 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f07d8009ae0 tx=0x7f07d801ca70 comp rx=0 tx=0).stop 2024-09-19T08:45:29.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.251+0000 7f07e88a6640 1 -- 172.21.15.98:0/2098205361 shutdown_connections 2024-09-19T08:45:29.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.251+0000 7f07e88a6640 1 --2- 172.21.15.98:0/2098205361 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07cc004810 0x7f07cc004c90 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.251+0000 7f07e88a6640 1 -- 172.21.15.98:0/2098205361 >> 172.21.15.98:0/2098205361 conn(0x7f07e4074e40 msgr2=0x7f07e4075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:29.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.251+0000 7f07e88a6640 1 -- 172.21.15.98:0/2098205361 shutdown_connections 2024-09-19T08:45:29.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.251+0000 7f07e88a6640 1 -- 172.21.15.98:0/2098205361 wait complete. 2024-09-19T08:45:29.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.251+0000 7f07e88a6640 1 Processor -- start 2024-09-19T08:45:29.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.251+0000 7f07e88a6640 1 -- start start 2024-09-19T08:45:29.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.255+0000 7f07e88a6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07cc004810 0x7f07e4151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:29.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.255+0000 7f07e88a6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07e412d4d0 0x7f07e412d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:29.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.255+0000 7f07e88a6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07e41eb920 0x7f07e41ebda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:29.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.255+0000 7f07e88a6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f07e415f950 con 0x7f07e412d4d0 2024-09-19T08:45:29.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.255+0000 7f07e88a6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f07e40b8570 con 0x7f07cc004810 2024-09-19T08:45:29.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.255+0000 7f07e88a6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f07e415fad0 con 0x7f07e41eb920 2024-09-19T08:45:29.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.255+0000 7f07e1d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07e412d4d0 0x7f07e412d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:29.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.255+0000 7f07e1d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07e412d4d0 0x7f07e412d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35460/0 (socket says 172.21.15.98:35460) 2024-09-19T08:45:29.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.255+0000 7f07e1d9b640 1 -- 172.21.15.98:0/3308228088 learned_addr learned my addr 172.21.15.98:0/3308228088 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:29.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.255+0000 7f07e259c640 1 --2- 172.21.15.98:0/3308228088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07cc004810 0x7f07e4151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:29.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.255+0000 7f07e2d9d640 1 --2- 172.21.15.98:0/3308228088 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07e41eb920 0x7f07e41ebda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:29.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.255+0000 7f07e1d9b640 1 -- 172.21.15.98:0/3308228088 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07e41eb920 msgr2=0x7f07e41ebda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:29.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.255+0000 7f07e1d9b640 1 --2- 172.21.15.98:0/3308228088 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07e41eb920 0x7f07e41ebda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.255+0000 7f07e1d9b640 1 -- 172.21.15.98:0/3308228088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07cc004810 msgr2=0x7f07e4151960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:29.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.255+0000 7f07e1d9b640 1 --2- 172.21.15.98:0/3308228088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07cc004810 0x7f07e4151960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.255+0000 7f07e1d9b640 1 -- 172.21.15.98:0/3308228088 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e41eab60 con 0x7f07e412d4d0 2024-09-19T08:45:29.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.255+0000 7f07e259c640 1 --2- 172.21.15.98:0/3308228088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07cc004810 0x7f07e4151960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:29.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.255+0000 7f07e2d9d640 1 --2- 172.21.15.98:0/3308228088 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07e41eb920 0x7f07e41ebda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:29.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.255+0000 7f07e1d9b640 1 --2- 172.21.15.98:0/3308228088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07e412d4d0 0x7f07e412d930 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f07d801c8a0 tx=0x7f07d8019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:29.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.255+0000 7f07d2ffd640 1 -- 172.21.15.98:0/3308228088 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d80b2040 con 0x7f07e412d4d0 2024-09-19T08:45:29.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.255+0000 7f07d2ffd640 1 -- 172.21.15.98:0/3308228088 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07d80a5e20 con 0x7f07e412d4d0 2024-09-19T08:45:29.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.255+0000 7f07d2ffd640 1 -- 172.21.15.98:0/3308228088 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d8004020 con 0x7f07e412d4d0 2024-09-19T08:45:29.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.255+0000 7f07e88a6640 1 -- 172.21.15.98:0/3308228088 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07e41eae40 con 0x7f07e412d4d0 2024-09-19T08:45:29.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.255+0000 7f07e88a6640 1 -- 172.21.15.98:0/3308228088 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07e41eb400 con 0x7f07e412d4d0 2024-09-19T08:45:29.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.255+0000 7f07e88a6640 1 -- 172.21.15.98:0/3308228088 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07e4144920 con 0x7f07e412d4d0 2024-09-19T08:45:29.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.255+0000 7f07d2ffd640 1 -- 172.21.15.98:0/3308228088 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f07d80bf020 con 0x7f07e412d4d0 2024-09-19T08:45:29.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.255+0000 7f07d2ffd640 1 --2- 172.21.15.98:0/3308228088 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f07c4067e90 0x7f07c406a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:29.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.255+0000 7f07d2ffd640 1 -- 172.21.15.98:0/3308228088 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5737+0+0 (secure 0 0 0) 0x7f07d8120270 con 0x7f07e412d4d0 2024-09-19T08:45:29.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.255+0000 7f07e259c640 1 --2- 172.21.15.98:0/3308228088 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f07c4067e90 0x7f07c406a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:29.264 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.259+0000 7f07e259c640 1 --2- 172.21.15.98:0/3308228088 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f07c4067e90 0x7f07c406a350 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f07d4001ec0 tx=0x7f07d4000ee0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:29.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.263+0000 7f07d2ffd640 1 -- 172.21.15.98:0/3308228088 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f07d80c6020 con 0x7f07e412d4d0 2024-09-19T08:45:29.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.419+0000 7f07e88a6640 1 -- 172.21.15.98:0/3308228088 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f07e4143d80 con 0x7f07e412d4d0 2024-09-19T08:45:29.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.423+0000 7f07d2ffd640 1 -- 172.21.15.98:0/3308228088 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) ==== 74+0+16796 (secure 0 0 0) 0x7f07d8016780 con 0x7f07e412d4d0 2024-09-19T08:45:29.425 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:45:29.426 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":134,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:45:28.140031+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:45:25.813513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-19T08:45:27.137275+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:45:29.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.427+0000 7f07e88a6640 1 -- 172.21.15.98:0/3308228088 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f07c4067e90 msgr2=0x7f07c406a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:29.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.427+0000 7f07e88a6640 1 --2- 172.21.15.98:0/3308228088 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f07c4067e90 0x7f07c406a350 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f07d4001ec0 tx=0x7f07d4000ee0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.427+0000 7f07e88a6640 1 -- 172.21.15.98:0/3308228088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07e412d4d0 msgr2=0x7f07e412d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:29.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.427+0000 7f07e88a6640 1 --2- 172.21.15.98:0/3308228088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07e412d4d0 0x7f07e412d930 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f07d801c8a0 tx=0x7f07d8019810 comp rx=0 tx=0).stop 2024-09-19T08:45:29.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.427+0000 7f07e88a6640 1 -- 172.21.15.98:0/3308228088 shutdown_connections 2024-09-19T08:45:29.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.427+0000 7f07e88a6640 1 --2- 172.21.15.98:0/3308228088 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f07c4067e90 0x7f07c406a350 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.427+0000 7f07e88a6640 1 --2- 172.21.15.98:0/3308228088 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07e41eb920 0x7f07e41ebda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.427+0000 7f07e88a6640 1 --2- 172.21.15.98:0/3308228088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f07e412d4d0 0x7f07e412d930 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.427+0000 7f07e88a6640 1 --2- 172.21.15.98:0/3308228088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07cc004810 0x7f07e4151960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.427+0000 7f07e88a6640 1 -- 172.21.15.98:0/3308228088 >> 172.21.15.98:0/3308228088 conn(0x7f07e4074e40 msgr2=0x7f07e41617e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:29.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.427+0000 7f07e88a6640 1 -- 172.21.15.98:0/3308228088 shutdown_connections 2024-09-19T08:45:29.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.427+0000 7f07e88a6640 1 -- 172.21.15.98:0/3308228088 wait complete. 2024-09-19T08:45:29.446 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-19T08:45:29.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.567+0000 7f73ef903640 1 Processor -- start 2024-09-19T08:45:29.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.567+0000 7f73ef903640 1 -- start start 2024-09-19T08:45:29.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.571+0000 7f73ef903640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f73e81415a0 0x7f73e80d20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:29.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.571+0000 7f73ef903640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73e80d25f0 0x7f73e80d3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:29.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.571+0000 7f73ef903640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73e815eae0 con 0x7f73e80d38a0 2024-09-19T08:45:29.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.571+0000 7f73ef903640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f73e80d2a30 con 0x7f73e81415a0 2024-09-19T08:45:29.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.571+0000 7f73ef903640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f73e815edc0 con 0x7f73e80d25f0 2024-09-19T08:45:29.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.571+0000 7f73ed69f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f73e81415a0 0x7f73e80d20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:29.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.571+0000 7f73ece9e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73e80d25f0 0x7f73e80d3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:29.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.571+0000 7f73ed69f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f73e81415a0 0x7f73e80d20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:36948/0 (socket says 172.21.15.98:36948) 2024-09-19T08:45:29.573 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.571+0000 7f73ed69f640 1 -- 172.21.15.98:0/3949645920 learned_addr learned my addr 172.21.15.98:0/3949645920 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:29.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.571+0000 7f73ed69f640 1 -- 172.21.15.98:0/3949645920 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73e80d25f0 msgr2=0x7f73e80d3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:29.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.571+0000 7f73ed69f640 1 --2- 172.21.15.98:0/3949645920 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73e80d25f0 0x7f73e80d3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.571+0000 7f73ed69f640 1 -- 172.21.15.98:0/3949645920 >> v1:172.21.15.139:6789/0 conn(0x7f73e80d38a0 legacy=0x7f73e8161fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:29.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.571+0000 7f73ed69f640 1 -- 172.21.15.98:0/3949645920 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e815f0a0 con 0x7f73e81415a0 2024-09-19T08:45:29.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.571+0000 7f73ed69f640 1 --2- 172.21.15.98:0/3949645920 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73e81415a0 0x7f73e80d20b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f73e4009870 tx=0x7f73e401c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c50c4d4f41212072 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:29.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.571+0000 7f73dffff640 1 -- 172.21.15.98:0/3949645920 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e40ab5f0 con 0x7f73e81415a0 2024-09-19T08:45:29.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.575+0000 7f73dffff640 1 -- 172.21.15.98:0/3949645920 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73e40b0070 con 0x7f73e81415a0 2024-09-19T08:45:29.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.575+0000 7f73dffff640 1 -- 172.21.15.98:0/3949645920 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e40ab9b0 con 0x7f73e81415a0 2024-09-19T08:45:29.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.575+0000 7f73ef903640 1 -- 172.21.15.98:0/3949645920 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73e81415a0 msgr2=0x7f73e80d20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:29.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.575+0000 7f73ef903640 1 --2- 172.21.15.98:0/3949645920 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73e81415a0 0x7f73e80d20b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f73e4009870 tx=0x7f73e401c530 comp rx=0 tx=0).stop 2024-09-19T08:45:29.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.575+0000 7f73ef903640 1 -- 172.21.15.98:0/3949645920 shutdown_connections 2024-09-19T08:45:29.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.575+0000 7f73ef903640 1 --2- 172.21.15.98:0/3949645920 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73e80d25f0 0x7f73e80d3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.575+0000 7f73ef903640 1 --2- 172.21.15.98:0/3949645920 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73e81415a0 0x7f73e80d20b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.577 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.575+0000 7f73ef903640 1 -- 172.21.15.98:0/3949645920 >> 172.21.15.98:0/3949645920 conn(0x7f73e8074c70 msgr2=0x7f73e8075080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:29.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.575+0000 7f73ef903640 1 -- 172.21.15.98:0/3949645920 shutdown_connections 2024-09-19T08:45:29.578 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.575+0000 7f73ef903640 1 -- 172.21.15.98:0/3949645920 wait complete. 2024-09-19T08:45:29.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.575+0000 7f73ef903640 1 Processor -- start 2024-09-19T08:45:29.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.575+0000 7f73ef903640 1 -- start start 2024-09-19T08:45:29.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.575+0000 7f73ef903640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f73e80d25f0 0x7f73e81527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:29.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.575+0000 7f73ef903640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73e80d38a0 0x7f73e80cdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:29.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.575+0000 7f73ef903640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73e81415a0 0x7f73e80cb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:29.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.575+0000 7f73ef903640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f73e80d42a0 con 0x7f73e80d25f0 2024-09-19T08:45:29.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.575+0000 7f73ef903640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f73e80d4120 con 0x7f73e80d38a0 2024-09-19T08:45:29.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.575+0000 7f73ef903640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f73e80d3cb0 con 0x7f73e81415a0 2024-09-19T08:45:29.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.575+0000 7f73ed69f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f73e80d25f0 0x7f73e81527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:29.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.575+0000 7f73ed69f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f73e80d25f0 0x7f73e81527d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35468/0 (socket says 172.21.15.98:35468) 2024-09-19T08:45:29.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.575+0000 7f73ed69f640 1 -- 172.21.15.98:0/1392989267 learned_addr learned my addr 172.21.15.98:0/1392989267 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:29.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.575+0000 7f73edea0640 1 --2- 172.21.15.98:0/1392989267 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73e81415a0 0x7f73e80cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:29.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.575+0000 7f73ece9e640 1 --2- 172.21.15.98:0/1392989267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73e80d38a0 0x7f73e80cdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:29.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.575+0000 7f73ed69f640 1 -- 172.21.15.98:0/1392989267 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73e81415a0 msgr2=0x7f73e80cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:29.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.575+0000 7f73ed69f640 1 --2- 172.21.15.98:0/1392989267 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73e81415a0 0x7f73e80cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.575+0000 7f73ed69f640 1 -- 172.21.15.98:0/1392989267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73e80d38a0 msgr2=0x7f73e80cdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:29.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.575+0000 7f73ed69f640 1 --2- 172.21.15.98:0/1392989267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73e80d38a0 0x7f73e80cdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.575+0000 7f73ed69f640 1 -- 172.21.15.98:0/1392989267 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e8139130 con 0x7f73e80d25f0 2024-09-19T08:45:29.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.575+0000 7f73ed69f640 1 --2- 172.21.15.98:0/1392989267 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f73e80d25f0 0x7f73e81527d0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f73e401cdd0 tx=0x7f73e401ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:29.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.575+0000 7f73ddffb640 1 -- 172.21.15.98:0/1392989267 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e40be3f0 con 0x7f73e80d25f0 2024-09-19T08:45:29.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.575+0000 7f73ddffb640 1 -- 172.21.15.98:0/1392989267 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73e40be590 con 0x7f73e80d25f0 2024-09-19T08:45:29.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.575+0000 7f73ef903640 1 -- 172.21.15.98:0/1392989267 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73e80cbd90 con 0x7f73e80d25f0 2024-09-19T08:45:29.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.575+0000 7f73ddffb640 1 -- 172.21.15.98:0/1392989267 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e40be980 con 0x7f73e80d25f0 2024-09-19T08:45:29.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.575+0000 7f73ef903640 1 -- 172.21.15.98:0/1392989267 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73e81393d0 con 0x7f73e80d25f0 2024-09-19T08:45:29.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.579+0000 7f73ef903640 1 -- 172.21.15.98:0/1392989267 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73e80d3360 con 0x7f73e80d25f0 2024-09-19T08:45:29.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.579+0000 7f73ddffb640 1 -- 172.21.15.98:0/1392989267 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f73e40b43f0 con 0x7f73e80d25f0 2024-09-19T08:45:29.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.579+0000 7f73ddffb640 1 --2- 172.21.15.98:0/1392989267 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f73cc067e60 0x7f73cc06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:29.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.579+0000 7f73ddffb640 1 -- 172.21.15.98:0/1392989267 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5737+0+0 (secure 0 0 0) 0x7f73e4120ae0 con 0x7f73e80d25f0 2024-09-19T08:45:29.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.579+0000 7f73ece9e640 1 --2- 172.21.15.98:0/1392989267 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f73cc067e60 0x7f73cc06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:29.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.579+0000 7f73ece9e640 1 --2- 172.21.15.98:0/1392989267 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f73cc067e60 0x7f73cc06a320 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f73e815e8d0 tx=0x7f73d8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:29.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.583+0000 7f73ddffb640 1 -- 172.21.15.98:0/1392989267 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f73e40c4060 con 0x7f73e80d25f0 2024-09-19T08:45:29.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.779+0000 7f73ef903640 1 -- 172.21.15.98:0/1392989267 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f73e80c9b00 con 0x7f73e80d25f0 2024-09-19T08:45:29.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.791+0000 7f73ddffb640 1 -- 172.21.15.98:0/1392989267 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v73) ==== 118+0+0 (secure 0 0 0) 0x7f73e4016150 con 0x7f73e80d25f0 2024-09-19T08:45:29.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.795+0000 7f73ef903640 1 -- 172.21.15.98:0/1392989267 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f73cc067e60 msgr2=0x7f73cc06a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:29.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.795+0000 7f73ef903640 1 --2- 172.21.15.98:0/1392989267 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f73cc067e60 0x7f73cc06a320 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f73e815e8d0 tx=0x7f73d8008040 comp rx=0 tx=0).stop 2024-09-19T08:45:29.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.795+0000 7f73ef903640 1 -- 172.21.15.98:0/1392989267 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f73e80d25f0 msgr2=0x7f73e81527d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:29.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.795+0000 7f73ef903640 1 --2- 172.21.15.98:0/1392989267 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f73e80d25f0 0x7f73e81527d0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f73e401cdd0 tx=0x7f73e401ad70 comp rx=0 tx=0).stop 2024-09-19T08:45:29.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.795+0000 7f73ef903640 1 -- 172.21.15.98:0/1392989267 shutdown_connections 2024-09-19T08:45:29.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.795+0000 7f73ef903640 1 --2- 172.21.15.98:0/1392989267 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f73cc067e60 0x7f73cc06a320 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.795+0000 7f73ef903640 1 --2- 172.21.15.98:0/1392989267 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f73e81415a0 0x7f73e80cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.795+0000 7f73ef903640 1 --2- 172.21.15.98:0/1392989267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73e80d38a0 0x7f73e80cdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.795+0000 7f73ef903640 1 --2- 172.21.15.98:0/1392989267 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f73e80d25f0 0x7f73e81527d0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.795+0000 7f73ef903640 1 -- 172.21.15.98:0/1392989267 >> 172.21.15.98:0/1392989267 conn(0x7f73e8074c70 msgr2=0x7f73e815fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:29.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.795+0000 7f73ef903640 1 -- 172.21.15.98:0/1392989267 shutdown_connections 2024-09-19T08:45:29.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.795+0000 7f73ef903640 1 -- 172.21.15.98:0/1392989267 wait complete. 2024-09-19T08:45:29.815 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-19T08:45:29.816 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-19T08:45:29.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.931+0000 7f6cec47a640 1 Processor -- start 2024-09-19T08:45:29.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.931+0000 7f6cec47a640 1 -- start start 2024-09-19T08:45:29.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.931+0000 7f6cec47a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6ce4145b70 0x7f6ce406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:29.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.931+0000 7f6cec47a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ce4071ac0 0x7f6ce40b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:29.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.931+0000 7f6cec47a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ce4073530 con 0x7f6ce406c1d0 2024-09-19T08:45:29.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.931+0000 7f6cec47a640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f6ce406c620 con 0x7f6ce4145b70 2024-09-19T08:45:29.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.931+0000 7f6cec47a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6ce4073810 con 0x7f6ce4071ac0 2024-09-19T08:45:29.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.931+0000 7f6cea216640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6ce4145b70 0x7f6ce406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:29.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.931+0000 7f6cea216640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6ce4145b70 0x7f6ce406bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35472/0 (socket says 172.21.15.98:35472) 2024-09-19T08:45:29.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.931+0000 7f6cea216640 1 -- 172.21.15.98:0/2411334087 learned_addr learned my addr 172.21.15.98:0/2411334087 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:29.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.931+0000 7f6ceaa17640 1 --2- 172.21.15.98:0/2411334087 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ce4071ac0 0x7f6ce40b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:29.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.931+0000 7f6cea216640 1 -- 172.21.15.98:0/2411334087 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ce4071ac0 msgr2=0x7f6ce40b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:29.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.931+0000 7f6cea216640 1 --2- 172.21.15.98:0/2411334087 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ce4071ac0 0x7f6ce40b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.931+0000 7f6cea216640 1 -- 172.21.15.98:0/2411334087 >> v1:172.21.15.139:6789/0 conn(0x7f6ce406c1d0 legacy=0x7f6ce40713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:29.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.931+0000 7f6cea216640 1 -- 172.21.15.98:0/2411334087 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ce4073af0 con 0x7f6ce4145b70 2024-09-19T08:45:29.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.931+0000 7f6ceaa17640 1 --2- 172.21.15.98:0/2411334087 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ce4071ac0 0x7f6ce40b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:29.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.935+0000 7f6cea216640 1 --2- 172.21.15.98:0/2411334087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ce4145b70 0x7f6ce406bc90 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc009870 tx=0x7f6cdc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be6aec2741c34a6e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:29.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.935+0000 7f6ce9214640 1 -- 172.21.15.98:0/2411334087 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc0b0070 con 0x7f6ce4145b70 2024-09-19T08:45:29.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.935+0000 7f6ce9214640 1 -- 172.21.15.98:0/2411334087 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cdc0ab600 con 0x7f6ce4145b70 2024-09-19T08:45:29.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.935+0000 7f6ce9214640 1 -- 172.21.15.98:0/2411334087 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc0ab9d0 con 0x7f6ce4145b70 2024-09-19T08:45:29.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.935+0000 7f6cec47a640 1 -- 172.21.15.98:0/2411334087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ce4145b70 msgr2=0x7f6ce406bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:29.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.935+0000 7f6cec47a640 1 --2- 172.21.15.98:0/2411334087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ce4145b70 0x7f6ce406bc90 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc009870 tx=0x7f6cdc01c490 comp rx=0 tx=0).stop 2024-09-19T08:45:29.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.935+0000 7f6cec47a640 1 -- 172.21.15.98:0/2411334087 shutdown_connections 2024-09-19T08:45:29.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.935+0000 7f6cec47a640 1 --2- 172.21.15.98:0/2411334087 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ce4071ac0 0x7f6ce40b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.935+0000 7f6cec47a640 1 --2- 172.21.15.98:0/2411334087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ce4145b70 0x7f6ce406bc90 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.935+0000 7f6cec47a640 1 -- 172.21.15.98:0/2411334087 >> 172.21.15.98:0/2411334087 conn(0x7f6ce40b2010 msgr2=0x7f6ce40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:29.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.935+0000 7f6cec47a640 1 -- 172.21.15.98:0/2411334087 shutdown_connections 2024-09-19T08:45:29.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.935+0000 7f6cec47a640 1 -- 172.21.15.98:0/2411334087 wait complete. 2024-09-19T08:45:29.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.939+0000 7f6cec47a640 1 Processor -- start 2024-09-19T08:45:29.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.939+0000 7f6cec47a640 1 -- start start 2024-09-19T08:45:29.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.939+0000 7f6cec47a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ce406c1d0 0x7f6ce41f5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:29.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.939+0000 7f6cec47a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ce4071ac0 0x7f6ce4165ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:29.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.939+0000 7f6cec47a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ce4145b70 0x7f6ce41434a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:29.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.939+0000 7f6cec47a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce40b4530 con 0x7f6ce4071ac0 2024-09-19T08:45:29.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.939+0000 7f6cec47a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce40b43b0 con 0x7f6ce406c1d0 2024-09-19T08:45:29.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.939+0000 7f6cec47a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6ce40b46b0 con 0x7f6ce4145b70 2024-09-19T08:45:29.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.939+0000 7f6ce9a15640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ce4071ac0 0x7f6ce4165ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:29.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.939+0000 7f6ce9a15640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ce4071ac0 0x7f6ce4165ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35478/0 (socket says 172.21.15.98:35478) 2024-09-19T08:45:29.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.939+0000 7f6ce9a15640 1 -- 172.21.15.98:0/2467516213 learned_addr learned my addr 172.21.15.98:0/2467516213 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:29.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.939+0000 7f6ceaa17640 1 --2- 172.21.15.98:0/2467516213 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ce4145b70 0x7f6ce41434a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:29.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.939+0000 7f6cea216640 1 --2- 172.21.15.98:0/2467516213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ce406c1d0 0x7f6ce41f5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:29.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.939+0000 7f6ce9a15640 1 -- 172.21.15.98:0/2467516213 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ce4145b70 msgr2=0x7f6ce41434a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:29.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.939+0000 7f6ce9a15640 1 --2- 172.21.15.98:0/2467516213 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ce4145b70 0x7f6ce41434a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.939+0000 7f6ce9a15640 1 -- 172.21.15.98:0/2467516213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ce406c1d0 msgr2=0x7f6ce41f5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:29.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.939+0000 7f6ce9a15640 1 --2- 172.21.15.98:0/2467516213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ce406c1d0 0x7f6ce41f5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:29.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.939+0000 7f6ce9a15640 1 -- 172.21.15.98:0/2467516213 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ce41f6280 con 0x7f6ce4071ac0 2024-09-19T08:45:29.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.939+0000 7f6ce9a15640 1 --2- 172.21.15.98:0/2467516213 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ce4071ac0 0x7f6ce4165ad0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f6cd800c450 tx=0x7f6cd800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:29.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.939+0000 7f6ce2ffd640 1 -- 172.21.15.98:0/2467516213 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8011790 con 0x7f6ce4071ac0 2024-09-19T08:45:29.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.939+0000 7f6ce2ffd640 1 -- 172.21.15.98:0/2467516213 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cd8017070 con 0x7f6ce4071ac0 2024-09-19T08:45:29.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.939+0000 7f6cec47a640 1 -- 172.21.15.98:0/2467516213 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ce41f6560 con 0x7f6ce4071ac0 2024-09-19T08:45:29.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.939+0000 7f6ce2ffd640 1 -- 172.21.15.98:0/2467516213 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8011b90 con 0x7f6ce4071ac0 2024-09-19T08:45:29.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.939+0000 7f6cec47a640 1 -- 172.21.15.98:0/2467516213 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ce41f6b20 con 0x7f6ce4071ac0 2024-09-19T08:45:29.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.939+0000 7f6ce2ffd640 1 -- 172.21.15.98:0/2467516213 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f6cd8011d30 con 0x7f6ce4071ac0 2024-09-19T08:45:29.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.939+0000 7f6cec47a640 1 -- 172.21.15.98:0/2467516213 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ce40b3410 con 0x7f6ce4071ac0 2024-09-19T08:45:29.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.939+0000 7f6ce2ffd640 1 --2- 172.21.15.98:0/2467516213 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6cd0067e60 0x7f6cd006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:29.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.943+0000 7f6ce2ffd640 1 -- 172.21.15.98:0/2467516213 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7f6cd8086530 con 0x7f6ce4071ac0 2024-09-19T08:45:29.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.943+0000 7f6cea216640 1 --2- 172.21.15.98:0/2467516213 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6cd0067e60 0x7f6cd006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:29.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.943+0000 7f6cea216640 1 --2- 172.21.15.98:0/2467516213 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6cd0067e60 0x7f6cd006a320 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0ad000 tx=0x7f6cdc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:29.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:29.947+0000 7f6ce2ffd640 1 -- 172.21.15.98:0/2467516213 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6cd8056870 con 0x7f6ce4071ac0 2024-09-19T08:45:30.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.139+0000 7f6cec47a640 1 -- 172.21.15.98:0/2467516213 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f6ce40b2490 con 0x7f6ce4071ac0 2024-09-19T08:45:30.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.807+0000 7f6ce2ffd640 1 -- 172.21.15.98:0/2467516213 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v74) ==== 122+0+0 (secure 0 0 0) 0x7f6cd8056170 con 0x7f6ce4071ac0 2024-09-19T08:45:30.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.811+0000 7f6cec47a640 1 -- 172.21.15.98:0/2467516213 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6cd0067e60 msgr2=0x7f6cd006a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:30.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.811+0000 7f6cec47a640 1 --2- 172.21.15.98:0/2467516213 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6cd0067e60 0x7f6cd006a320 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0ad000 tx=0x7f6cdc008040 comp rx=0 tx=0).stop 2024-09-19T08:45:30.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.811+0000 7f6cec47a640 1 -- 172.21.15.98:0/2467516213 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ce4071ac0 msgr2=0x7f6ce4165ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:30.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.811+0000 7f6cec47a640 1 --2- 172.21.15.98:0/2467516213 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ce4071ac0 0x7f6ce4165ad0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f6cd800c450 tx=0x7f6cd800c910 comp rx=0 tx=0).stop 2024-09-19T08:45:30.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.811+0000 7f6cec47a640 1 -- 172.21.15.98:0/2467516213 shutdown_connections 2024-09-19T08:45:30.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.811+0000 7f6cec47a640 1 --2- 172.21.15.98:0/2467516213 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6cd0067e60 0x7f6cd006a320 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:30.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.811+0000 7f6cec47a640 1 --2- 172.21.15.98:0/2467516213 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6ce4145b70 0x7f6ce41434a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:30.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.811+0000 7f6cec47a640 1 --2- 172.21.15.98:0/2467516213 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6ce4071ac0 0x7f6ce4165ad0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:30.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.811+0000 7f6cec47a640 1 --2- 172.21.15.98:0/2467516213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ce406c1d0 0x7f6ce41f5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:30.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.811+0000 7f6cec47a640 1 -- 172.21.15.98:0/2467516213 >> 172.21.15.98:0/2467516213 conn(0x7f6ce40b2010 msgr2=0x7f6ce4142be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:30.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.811+0000 7f6cec47a640 1 -- 172.21.15.98:0/2467516213 shutdown_connections 2024-09-19T08:45:30.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.811+0000 7f6cec47a640 1 -- 172.21.15.98:0/2467516213 wait complete. 2024-09-19T08:45:30.833 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-19T08:45:30.833 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:45:30.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.955+0000 7fdc5c5b9640 1 Processor -- start 2024-09-19T08:45:30.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.955+0000 7fdc5c5b9640 1 -- start start 2024-09-19T08:45:30.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.955+0000 7fdc5c5b9640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fdc5406efa0 0x7fdc540ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:30.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.955+0000 7fdc5c5b9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc5406f380 0x7fdc540cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:30.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.955+0000 7fdc5c5b9640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc540af4a0 con 0x7fdc540ae8c0 2024-09-19T08:45:30.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.955+0000 7fdc5c5b9640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fdc5415ed00 con 0x7fdc5406efa0 2024-09-19T08:45:30.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.955+0000 7fdc5c5b9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdc540af780 con 0x7fdc5406f380 2024-09-19T08:45:30.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.955+0000 7fdc5a355640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fdc5406efa0 0x7fdc540ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:30.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.955+0000 7fdc5a355640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fdc5406efa0 0x7fdc540ce550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35480/0 (socket says 172.21.15.98:35480) 2024-09-19T08:45:30.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.955+0000 7fdc5a355640 1 -- 172.21.15.98:0/1132166223 learned_addr learned my addr 172.21.15.98:0/1132166223 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:30.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.955+0000 7fdc59b54640 1 --2- 172.21.15.98:0/1132166223 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc5406f380 0x7fdc540cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:30.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.955+0000 7fdc5a355640 1 -- 172.21.15.98:0/1132166223 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc5406f380 msgr2=0x7fdc540cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:30.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.955+0000 7fdc5a355640 1 --2- 172.21.15.98:0/1132166223 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc5406f380 0x7fdc540cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:30.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.955+0000 7fdc5a355640 1 -- 172.21.15.98:0/1132166223 >> v1:172.21.15.139:6789/0 conn(0x7fdc540ae8c0 legacy=0x7fdc5415b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:30.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.955+0000 7fdc5a355640 1 -- 172.21.15.98:0/1132166223 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc540afa60 con 0x7fdc5406efa0 2024-09-19T08:45:30.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.959+0000 7fdc5a355640 1 --2- 172.21.15.98:0/1132166223 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc5406efa0 0x7fdc540ce550 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fdc50009870 tx=0x7fdc5001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c15b4c459e4bac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:30.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.959+0000 7fdc59353640 1 -- 172.21.15.98:0/1132166223 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc500b0070 con 0x7fdc5406efa0 2024-09-19T08:45:30.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.959+0000 7fdc59353640 1 -- 172.21.15.98:0/1132166223 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc500ab600 con 0x7fdc5406efa0 2024-09-19T08:45:30.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.959+0000 7fdc59353640 1 -- 172.21.15.98:0/1132166223 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc500ab9d0 con 0x7fdc5406efa0 2024-09-19T08:45:30.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.959+0000 7fdc5c5b9640 1 -- 172.21.15.98:0/1132166223 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc5406efa0 msgr2=0x7fdc540ce550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:30.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.959+0000 7fdc5c5b9640 1 --2- 172.21.15.98:0/1132166223 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc5406efa0 0x7fdc540ce550 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fdc50009870 tx=0x7fdc5001c4d0 comp rx=0 tx=0).stop 2024-09-19T08:45:30.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.959+0000 7fdc5c5b9640 1 -- 172.21.15.98:0/1132166223 shutdown_connections 2024-09-19T08:45:30.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.959+0000 7fdc5c5b9640 1 --2- 172.21.15.98:0/1132166223 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc5406f380 0x7fdc540cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:30.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.959+0000 7fdc5c5b9640 1 --2- 172.21.15.98:0/1132166223 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc5406efa0 0x7fdc540ce550 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:30.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.959+0000 7fdc5c5b9640 1 -- 172.21.15.98:0/1132166223 >> 172.21.15.98:0/1132166223 conn(0x7fdc540d0980 msgr2=0x7fdc540d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:30.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.959+0000 7fdc5c5b9640 1 -- 172.21.15.98:0/1132166223 shutdown_connections 2024-09-19T08:45:30.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.959+0000 7fdc5c5b9640 1 -- 172.21.15.98:0/1132166223 wait complete. 2024-09-19T08:45:30.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.963+0000 7fdc5c5b9640 1 Processor -- start 2024-09-19T08:45:30.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.963+0000 7fdc5c5b9640 1 -- start start 2024-09-19T08:45:30.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.963+0000 7fdc5c5b9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc5406efa0 0x7fdc5414db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:30.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.963+0000 7fdc5c5b9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc540ae8c0 0x7fdc5414bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:30.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.963+0000 7fdc5c5b9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc541578d0 0x7fdc54160b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:30.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.963+0000 7fdc5c5b9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5406d940 con 0x7fdc5406efa0 2024-09-19T08:45:30.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.963+0000 7fdc5c5b9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5406d7c0 con 0x7fdc541578d0 2024-09-19T08:45:30.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.963+0000 7fdc5c5b9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdc5406dac0 con 0x7fdc540ae8c0 2024-09-19T08:45:30.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.963+0000 7fdc5a355640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc5406efa0 0x7fdc5414db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:30.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.963+0000 7fdc5a355640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc5406efa0 0x7fdc5414db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35496/0 (socket says 172.21.15.98:35496) 2024-09-19T08:45:30.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.963+0000 7fdc5a355640 1 -- 172.21.15.98:0/3047304171 learned_addr learned my addr 172.21.15.98:0/3047304171 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:30.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.963+0000 7fdc59b54640 1 --2- 172.21.15.98:0/3047304171 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc540ae8c0 0x7fdc5414bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:30.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.963+0000 7fdc5ab56640 1 --2- 172.21.15.98:0/3047304171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc541578d0 0x7fdc54160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:30.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.963+0000 7fdc5a355640 1 -- 172.21.15.98:0/3047304171 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc540ae8c0 msgr2=0x7fdc5414bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:30.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.963+0000 7fdc5a355640 1 --2- 172.21.15.98:0/3047304171 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc540ae8c0 0x7fdc5414bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:30.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.963+0000 7fdc5a355640 1 -- 172.21.15.98:0/3047304171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc541578d0 msgr2=0x7fdc54160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:30.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.963+0000 7fdc5a355640 1 --2- 172.21.15.98:0/3047304171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc541578d0 0x7fdc54160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:30.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.963+0000 7fdc5a355640 1 -- 172.21.15.98:0/3047304171 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc540b67f0 con 0x7fdc5406efa0 2024-09-19T08:45:30.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.963+0000 7fdc5a355640 1 --2- 172.21.15.98:0/3047304171 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc5406efa0 0x7fdc5414db50 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fdc500ae040 tx=0x7fdc5001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:30.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.963+0000 7fdc4affd640 1 -- 172.21.15.98:0/3047304171 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc500b0040 con 0x7fdc5406efa0 2024-09-19T08:45:30.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.963+0000 7fdc4affd640 1 -- 172.21.15.98:0/3047304171 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc500bc070 con 0x7fdc5406efa0 2024-09-19T08:45:30.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.963+0000 7fdc5c5b9640 1 -- 172.21.15.98:0/3047304171 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc54153830 con 0x7fdc5406efa0 2024-09-19T08:45:30.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.963+0000 7fdc4affd640 1 -- 172.21.15.98:0/3047304171 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc500b6880 con 0x7fdc5406efa0 2024-09-19T08:45:30.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.963+0000 7fdc5c5b9640 1 -- 172.21.15.98:0/3047304171 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc541555d0 con 0x7fdc5406efa0 2024-09-19T08:45:30.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.963+0000 7fdc5c5b9640 1 -- 172.21.15.98:0/3047304171 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc541553f0 con 0x7fdc5406efa0 2024-09-19T08:45:30.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.963+0000 7fdc4affd640 1 -- 172.21.15.98:0/3047304171 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fdc500ab480 con 0x7fdc5406efa0 2024-09-19T08:45:30.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.967+0000 7fdc4affd640 1 --2- 172.21.15.98:0/3047304171 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fdc38067e60 0x7fdc3806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:30.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.967+0000 7fdc4affd640 1 -- 172.21.15.98:0/3047304171 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7fdc50120b20 con 0x7fdc5406efa0 2024-09-19T08:45:30.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.967+0000 7fdc59b54640 1 --2- 172.21.15.98:0/3047304171 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fdc38067e60 0x7fdc3806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:30.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.967+0000 7fdc59b54640 1 --2- 172.21.15.98:0/3047304171 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fdc38067e60 0x7fdc3806a320 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdc541537c0 tx=0x7fdc44008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:30.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:30.971+0000 7fdc4affd640 1 -- 172.21.15.98:0/3047304171 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fdc541553f0 con 0x7fdc5406efa0 2024-09-19T08:45:31.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.163+0000 7fdc5c5b9640 1 -- 172.21.15.98:0/3047304171 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc5415b5e0 con 0x7fdc5406efa0 2024-09-19T08:45:31.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.163+0000 7fdc4affd640 1 -- 172.21.15.98:0/3047304171 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5230 (secure 0 0 0) 0x7fdc50016150 con 0x7fdc5406efa0 2024-09-19T08:45:31.165 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:45:31.165 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":74,"btime":"2024-09-19T08:45:30:805559+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/1907681163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":1907681163},{"type":"v1","addr":"172.21.15.139:6834","nonce":1907681163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7896,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2481662340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2481662340},{"type":"v1","addr":"172.21.15.98:6833","nonce":2481662340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7902,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/286818221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":286818221},{"type":"v1","addr":"172.21.15.98:6835","nonce":286818221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:45:28.140764+0000","modified":"2024-09-19T08:45:30.144582+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7914},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7914":{"gid":7914,"name":"b","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6835/4255670104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6833","nonce":4255670104},{"type":"v1","addr":"172.21.15.139:6835","nonce":4255670104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7914,"qdb_cluster":[7914]},"id":10}]} 2024-09-19T08:45:31.166 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 74 2024-09-19T08:45:31.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.163+0000 7fdc5c5b9640 1 -- 172.21.15.98:0/3047304171 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fdc38067e60 msgr2=0x7fdc3806a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:31.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.163+0000 7fdc5c5b9640 1 --2- 172.21.15.98:0/3047304171 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fdc38067e60 0x7fdc3806a320 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdc541537c0 tx=0x7fdc44008040 comp rx=0 tx=0).stop 2024-09-19T08:45:31.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.163+0000 7fdc5c5b9640 1 -- 172.21.15.98:0/3047304171 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc5406efa0 msgr2=0x7fdc5414db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:31.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.163+0000 7fdc5c5b9640 1 --2- 172.21.15.98:0/3047304171 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc5406efa0 0x7fdc5414db50 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fdc500ae040 tx=0x7fdc5001ad10 comp rx=0 tx=0).stop 2024-09-19T08:45:31.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.163+0000 7fdc5c5b9640 1 -- 172.21.15.98:0/3047304171 shutdown_connections 2024-09-19T08:45:31.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.163+0000 7fdc5c5b9640 1 --2- 172.21.15.98:0/3047304171 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fdc38067e60 0x7fdc3806a320 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:31.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.163+0000 7fdc5c5b9640 1 --2- 172.21.15.98:0/3047304171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc541578d0 0x7fdc54160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:31.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.163+0000 7fdc5c5b9640 1 --2- 172.21.15.98:0/3047304171 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdc540ae8c0 0x7fdc5414bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:31.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.163+0000 7fdc5c5b9640 1 --2- 172.21.15.98:0/3047304171 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdc5406efa0 0x7fdc5414db50 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:31.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.163+0000 7fdc5c5b9640 1 -- 172.21.15.98:0/3047304171 >> 172.21.15.98:0/3047304171 conn(0x7fdc540d0980 msgr2=0x7fdc5415c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:31.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.163+0000 7fdc5c5b9640 1 -- 172.21.15.98:0/3047304171 shutdown_connections 2024-09-19T08:45:31.168 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.163+0000 7fdc5c5b9640 1 -- 172.21.15.98:0/3047304171 wait complete. 2024-09-19T08:45:31.184 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:45:31.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.299+0000 7ffbd3cfa640 1 Processor -- start 2024-09-19T08:45:31.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.299+0000 7ffbd3cfa640 1 -- start start 2024-09-19T08:45:31.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.303+0000 7ffbd3cfa640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ffbcc06c3a0 0x7ffbcc0cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:31.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.303+0000 7ffbd3cfa640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbcc06d310 con 0x7ffbcc15c550 2024-09-19T08:45:31.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.303+0000 7ffbd3cfa640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbcc06d5f0 con 0x7ffbcc0cf900 2024-09-19T08:45:31.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.303+0000 7ffbd3cfa640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7ffbcc06d8d0 con 0x7ffbcc06c3a0 2024-09-19T08:45:31.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.303+0000 7ffbd1295640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ffbcc06c3a0 0x7ffbcc0cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:31.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.303+0000 7ffbd1295640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ffbcc06c3a0 0x7ffbcc0cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35512/0 (socket says 172.21.15.98:35512) 2024-09-19T08:45:31.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.303+0000 7ffbd1295640 1 -- 172.21.15.98:0/2906033260 learned_addr learned my addr 172.21.15.98:0/2906033260 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:31.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.303+0000 7ffbd1295640 1 -- 172.21.15.98:0/2906033260 >> v1:172.21.15.139:6789/0 conn(0x7ffbcc0cf900 legacy=0x7ffbcc0723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:31.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.303+0000 7ffbd1295640 1 -- 172.21.15.98:0/2906033260 >> v1:172.21.15.98:6789/0 conn(0x7ffbcc15c550 legacy=0x7ffbcc06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:31.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.303+0000 7ffbd1295640 1 -- 172.21.15.98:0/2906033260 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbcc06dbb0 con 0x7ffbcc06c3a0 2024-09-19T08:45:31.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.303+0000 7ffbd1295640 1 --2- 172.21.15.98:0/2906033260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbcc06c3a0 0x7ffbcc0cf3c0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0009870 tx=0x7ffbc001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33c8ba14af7ccd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:31.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.303+0000 7ffbd0a94640 1 -- 172.21.15.98:0/2906033260 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc00ab430 con 0x7ffbcc06c3a0 2024-09-19T08:45:31.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.303+0000 7ffbd0a94640 1 -- 172.21.15.98:0/2906033260 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbc00b0070 con 0x7ffbcc06c3a0 2024-09-19T08:45:31.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.303+0000 7ffbd0a94640 1 -- 172.21.15.98:0/2906033260 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc00ab7f0 con 0x7ffbcc06c3a0 2024-09-19T08:45:31.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.303+0000 7ffbd3cfa640 1 -- 172.21.15.98:0/2906033260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbcc06c3a0 msgr2=0x7ffbcc0cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:31.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.307+0000 7ffbd3cfa640 1 --2- 172.21.15.98:0/2906033260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbcc06c3a0 0x7ffbcc0cf3c0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0009870 tx=0x7ffbc001c520 comp rx=0 tx=0).stop 2024-09-19T08:45:31.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.307+0000 7ffbd3cfa640 1 -- 172.21.15.98:0/2906033260 shutdown_connections 2024-09-19T08:45:31.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.307+0000 7ffbd3cfa640 1 --2- 172.21.15.98:0/2906033260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbcc06c3a0 0x7ffbcc0cf3c0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:31.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.307+0000 7ffbd3cfa640 1 -- 172.21.15.98:0/2906033260 >> 172.21.15.98:0/2906033260 conn(0x7ffbcc0b2010 msgr2=0x7ffbcc0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:31.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.307+0000 7ffbd3cfa640 1 -- 172.21.15.98:0/2906033260 shutdown_connections 2024-09-19T08:45:31.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.307+0000 7ffbd3cfa640 1 -- 172.21.15.98:0/2906033260 wait complete. 2024-09-19T08:45:31.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.307+0000 7ffbd3cfa640 1 Processor -- start 2024-09-19T08:45:31.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.307+0000 7ffbd3cfa640 1 -- start start 2024-09-19T08:45:31.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.307+0000 7ffbd3cfa640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbcc06c3a0 0x7ffbcc149dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:31.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.307+0000 7ffbd3cfa640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbcc0cf900 0x7ffbcc1510a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:31.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.307+0000 7ffbd3cfa640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffbcc15c550 0x7ffbcc0b4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:31.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.307+0000 7ffbd3cfa640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc0736d0 con 0x7ffbcc06c3a0 2024-09-19T08:45:31.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.307+0000 7ffbd3cfa640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffbcc073550 con 0x7ffbcc0cf900 2024-09-19T08:45:31.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.307+0000 7ffbd3cfa640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ffbcc16a240 con 0x7ffbcc15c550 2024-09-19T08:45:31.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.307+0000 7ffbd1a96640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbcc06c3a0 0x7ffbcc149dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:31.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.307+0000 7ffbd1a96640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbcc06c3a0 0x7ffbcc149dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35520/0 (socket says 172.21.15.98:35520) 2024-09-19T08:45:31.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.307+0000 7ffbd1a96640 1 -- 172.21.15.98:0/3001544201 learned_addr learned my addr 172.21.15.98:0/3001544201 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:31.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.307+0000 7ffbd1a96640 1 -- 172.21.15.98:0/3001544201 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffbcc15c550 msgr2=0x7ffbcc0b4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:31.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.307+0000 7ffbd1a96640 1 --2- 172.21.15.98:0/3001544201 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffbcc15c550 0x7ffbcc0b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:31.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.307+0000 7ffbd1a96640 1 -- 172.21.15.98:0/3001544201 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbcc0cf900 msgr2=0x7ffbcc1510a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:31.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.307+0000 7ffbd1a96640 1 --2- 172.21.15.98:0/3001544201 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbcc0cf900 0x7ffbcc1510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:31.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.307+0000 7ffbd1a96640 1 -- 172.21.15.98:0/3001544201 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbcc155240 con 0x7ffbcc06c3a0 2024-09-19T08:45:31.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.307+0000 7ffbd1a96640 1 --2- 172.21.15.98:0/3001544201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbcc06c3a0 0x7ffbcc149dc0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7ffbc400c3a0 tx=0x7ffbc400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:31.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.307+0000 7ffbca7fc640 1 -- 172.21.15.98:0/3001544201 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc40052d0 con 0x7ffbcc06c3a0 2024-09-19T08:45:31.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.307+0000 7ffbca7fc640 1 -- 172.21.15.98:0/3001544201 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbc4005470 con 0x7ffbcc06c3a0 2024-09-19T08:45:31.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.307+0000 7ffbd3cfa640 1 -- 172.21.15.98:0/3001544201 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbcc14a300 con 0x7ffbcc06c3a0 2024-09-19T08:45:31.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.307+0000 7ffbca7fc640 1 -- 172.21.15.98:0/3001544201 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4007260 con 0x7ffbcc06c3a0 2024-09-19T08:45:31.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.307+0000 7ffbd3cfa640 1 -- 172.21.15.98:0/3001544201 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbcc155790 con 0x7ffbcc06c3a0 2024-09-19T08:45:31.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.311+0000 7ffbd3cfa640 1 -- 172.21.15.98:0/3001544201 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbcc0cf3c0 con 0x7ffbcc06c3a0 2024-09-19T08:45:31.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.311+0000 7ffbca7fc640 1 -- 172.21.15.98:0/3001544201 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7ffbc4007400 con 0x7ffbcc06c3a0 2024-09-19T08:45:31.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.311+0000 7ffbca7fc640 1 --2- 172.21.15.98:0/3001544201 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ffbb8067e60 0x7ffbb806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:31.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.311+0000 7ffbca7fc640 1 -- 172.21.15.98:0/3001544201 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7ffbc4085b10 con 0x7ffbcc06c3a0 2024-09-19T08:45:31.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.311+0000 7ffbd1295640 1 --2- 172.21.15.98:0/3001544201 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ffbb8067e60 0x7ffbb806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:31.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.311+0000 7ffbd1295640 1 --2- 172.21.15.98:0/3001544201 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ffbb8067e60 0x7ffbb806a320 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ffbc00af000 tx=0x7ffbc0004740 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:31.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.315+0000 7ffbca7fc640 1 -- 172.21.15.98:0/3001544201 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ffbc4087e20 con 0x7ffbcc06c3a0 2024-09-19T08:45:31.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.479+0000 7ffbd3cfa640 1 -- 172.21.15.98:0/3001544201 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffbcc150250 con 0x7ffbcc06c3a0 2024-09-19T08:45:31.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.479+0000 7ffbca7fc640 1 -- 172.21.15.98:0/3001544201 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) ==== 74+0+16783 (secure 0 0 0) 0x7ffbc4055e50 con 0x7ffbcc06c3a0 2024-09-19T08:45:31.482 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:45:31.482 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":135,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:45:29.773038+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:45:25.813513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-19T08:45:27.137275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:45:31.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.479+0000 7ffbd3cfa640 1 -- 172.21.15.98:0/3001544201 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ffbb8067e60 msgr2=0x7ffbb806a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:31.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.479+0000 7ffbd3cfa640 1 --2- 172.21.15.98:0/3001544201 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ffbb8067e60 0x7ffbb806a320 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ffbc00af000 tx=0x7ffbc0004740 comp rx=0 tx=0).stop 2024-09-19T08:45:31.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.479+0000 7ffbd3cfa640 1 -- 172.21.15.98:0/3001544201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbcc06c3a0 msgr2=0x7ffbcc149dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:31.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.479+0000 7ffbd3cfa640 1 --2- 172.21.15.98:0/3001544201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbcc06c3a0 0x7ffbcc149dc0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7ffbc400c3a0 tx=0x7ffbc400c860 comp rx=0 tx=0).stop 2024-09-19T08:45:31.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.483+0000 7ffbd3cfa640 1 -- 172.21.15.98:0/3001544201 shutdown_connections 2024-09-19T08:45:31.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.483+0000 7ffbd3cfa640 1 --2- 172.21.15.98:0/3001544201 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ffbb8067e60 0x7ffbb806a320 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:31.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.483+0000 7ffbd3cfa640 1 --2- 172.21.15.98:0/3001544201 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffbcc15c550 0x7ffbcc0b4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:31.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.483+0000 7ffbd3cfa640 1 --2- 172.21.15.98:0/3001544201 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffbcc0cf900 0x7ffbcc1510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:31.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.483+0000 7ffbd3cfa640 1 --2- 172.21.15.98:0/3001544201 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffbcc06c3a0 0x7ffbcc149dc0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:31.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.483+0000 7ffbd3cfa640 1 -- 172.21.15.98:0/3001544201 >> 172.21.15.98:0/3001544201 conn(0x7ffbcc0b2010 msgr2=0x7ffbcc067a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:31.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.483+0000 7ffbd3cfa640 1 -- 172.21.15.98:0/3001544201 shutdown_connections 2024-09-19T08:45:31.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.483+0000 7ffbd3cfa640 1 -- 172.21.15.98:0/3001544201 wait complete. 2024-09-19T08:45:31.501 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T08:45:31.502 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:45:31.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.619+0000 7fd6ceec6640 1 Processor -- start 2024-09-19T08:45:31.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.619+0000 7fd6ceec6640 1 -- start start 2024-09-19T08:45:31.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.619+0000 7fd6ceec6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6c815c550 0x7fd6c806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:31.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.619+0000 7fd6ceec6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd6c80cfad0 0x7fd6c80723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:31.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.619+0000 7fd6ceec6640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6c806d310 con 0x7fd6c806c1d0 2024-09-19T08:45:31.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.619+0000 7fd6ceec6640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fd6c806c610 con 0x7fd6c80cfad0 2024-09-19T08:45:31.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.619+0000 7fd6ceec6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd6c806d5f0 con 0x7fd6c815c550 2024-09-19T08:45:31.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.619+0000 7fd6c7fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fd6c806c1d0 0x7fd6c80cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:43986/0 (socket says 172.21.15.98:43986) 2024-09-19T08:45:31.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.619+0000 7fd6c7fff640 1 -- 172.21.15.98:0/880273150 learned_addr learned my addr 172.21.15.98:0/880273150 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:31.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.619+0000 7fd6ccc62640 1 --2- 172.21.15.98:0/880273150 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6c815c550 0x7fd6c806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:31.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.619+0000 7fd6cd463640 1 --2- 172.21.15.98:0/880273150 >> v2:172.21.15.139:3300/0 conn(0x7fd6c80cfad0 0x7fd6c80723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:31.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.619+0000 7fd6c77fe640 1 -- 172.21.15.98:0/880273150 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2507100098 0 0) 0x7fd6c806d310 con 0x7fd6c806c1d0 2024-09-19T08:45:31.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.619+0000 7fd6c77fe640 1 -- 172.21.15.98:0/880273150 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6b8003530 con 0x7fd6c806c1d0 2024-09-19T08:45:31.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.619+0000 7fd6c77fe640 1 -- 172.21.15.98:0/880273150 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 275573520 0 0) 0x7fd6b8003530 con 0x7fd6c806c1d0 2024-09-19T08:45:31.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.619+0000 7fd6c77fe640 1 -- 172.21.15.98:0/880273150 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6c815c550 msgr2=0x7fd6c806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:31.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.619+0000 7fd6c77fe640 1 --2- 172.21.15.98:0/880273150 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6c815c550 0x7fd6c806bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:31.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.619+0000 7fd6c77fe640 1 -- 172.21.15.98:0/880273150 >> v2:172.21.15.139:3300/0 conn(0x7fd6c80cfad0 msgr2=0x7fd6c80723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:31.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.619+0000 7fd6c77fe640 1 --2- 172.21.15.98:0/880273150 >> v2:172.21.15.139:3300/0 conn(0x7fd6c80cfad0 0x7fd6c80723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:31.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.619+0000 7fd6c77fe640 1 -- 172.21.15.98:0/880273150 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c806d8d0 con 0x7fd6c806c1d0 2024-09-19T08:45:31.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.619+0000 7fd6c77fe640 1 -- 172.21.15.98:0/880273150 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fd6b4001680 con 0x7fd6c806c1d0 2024-09-19T08:45:31.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.619+0000 7fd6c77fe640 1 -- 172.21.15.98:0/880273150 >> v1:172.21.15.98:6789/0 conn(0x7fd6c806c1d0 legacy=0x7fd6c80cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:31.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.619+0000 7fd6c77fe640 1 --2- 172.21.15.98:0/880273150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6b80039c0 0x7fd6b8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:31.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.619+0000 7fd6c77fe640 1 --2- 172.21.15.98:0/880273150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6b8004810 0x7fd6b8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:31.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.619+0000 7fd6c77fe640 1 --2- 172.21.15.98:0/880273150 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6c815c550 0x7fd6b80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:31.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.619+0000 7fd6c77fe640 1 -- 172.21.15.98:0/880273150 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c806c610 con 0x7fd6b80039c0 2024-09-19T08:45:31.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.619+0000 7fd6c77fe640 1 -- 172.21.15.98:0/880273150 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c806d5f0 con 0x7fd6b8004810 2024-09-19T08:45:31.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.619+0000 7fd6c77fe640 1 -- 172.21.15.98:0/880273150 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd6b800b9d0 con 0x7fd6c815c550 2024-09-19T08:45:31.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.619+0000 7fd6ccc62640 1 --2- 172.21.15.98:0/880273150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6b80039c0 0x7fd6b8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:31.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.619+0000 7fd6c7fff640 1 --2- 172.21.15.98:0/880273150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6b8004810 0x7fd6b8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:31.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.619+0000 7fd6ccc62640 1 -- 172.21.15.98:0/880273150 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6c815c550 msgr2=0x7fd6b80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:31.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.619+0000 7fd6ccc62640 1 --2- 172.21.15.98:0/880273150 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6c815c550 0x7fd6b80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:31.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.619+0000 7fd6ccc62640 1 -- 172.21.15.98:0/880273150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6b8004810 msgr2=0x7fd6b8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:31.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.619+0000 7fd6ccc62640 1 --2- 172.21.15.98:0/880273150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6b8004810 0x7fd6b8004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:31.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.619+0000 7fd6ccc62640 1 -- 172.21.15.98:0/880273150 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b8003530 con 0x7fd6b80039c0 2024-09-19T08:45:31.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.623+0000 7fd6ccc62640 1 --2- 172.21.15.98:0/880273150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6b80039c0 0x7fd6b8003de0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc009ae0 tx=0x7fd6bc01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ccb558d7f94fc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:31.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.623+0000 7fd6c77fe640 1 -- 172.21.15.98:0/880273150 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc0096b0 con 0x7fd6b80039c0 2024-09-19T08:45:31.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.623+0000 7fd6c77fe640 1 -- 172.21.15.98:0/880273150 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6bc0a8410 con 0x7fd6b80039c0 2024-09-19T08:45:31.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.623+0000 7fd6c77fe640 1 -- 172.21.15.98:0/880273150 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc0a8800 con 0x7fd6b80039c0 2024-09-19T08:45:31.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.623+0000 7fd6ceec6640 1 -- 172.21.15.98:0/880273150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6b80039c0 msgr2=0x7fd6b8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:31.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.623+0000 7fd6ceec6640 1 --2- 172.21.15.98:0/880273150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6b80039c0 0x7fd6b8003de0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc009ae0 tx=0x7fd6bc01cab0 comp rx=0 tx=0).stop 2024-09-19T08:45:31.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.623+0000 7fd6cd463640 1 -- 172.21.15.98:0/880273150 reap_dead start 2024-09-19T08:45:31.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.623+0000 7fd6ceec6640 1 -- 172.21.15.98:0/880273150 shutdown_connections 2024-09-19T08:45:31.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.623+0000 7fd6ceec6640 1 -- 172.21.15.98:0/880273150 >> 172.21.15.98:0/880273150 conn(0x7fd6c80b2010 msgr2=0x7fd6c80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:31.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.623+0000 7fd6ceec6640 1 -- 172.21.15.98:0/880273150 shutdown_connections 2024-09-19T08:45:31.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.623+0000 7fd6ceec6640 1 -- 172.21.15.98:0/880273150 wait complete. 2024-09-19T08:45:31.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.623+0000 7fd6ceec6640 1 Processor -- start 2024-09-19T08:45:31.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.623+0000 7fd6ceec6640 1 -- start start 2024-09-19T08:45:31.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.623+0000 7fd6ceec6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6c8140640 0x7fd6c8151760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:31.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.623+0000 7fd6ceec6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6c8151ca0 0x7fd6c815ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:31.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.623+0000 7fd6ceec6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6c815d200 0x7fd6c8142d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:31.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.623+0000 7fd6ceec6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c8073300 con 0x7fd6c8151ca0 2024-09-19T08:45:31.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.623+0000 7fd6ceec6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c815cb40 con 0x7fd6c8140640 2024-09-19T08:45:31.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.623+0000 7fd6ceec6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd6c8073480 con 0x7fd6c815d200 2024-09-19T08:45:31.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.627+0000 7fd6c7fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6c8151ca0 0x7fd6c815ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:31.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.627+0000 7fd6c7fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6c8151ca0 0x7fd6c815ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35540/0 (socket says 172.21.15.98:35540) 2024-09-19T08:45:31.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.627+0000 7fd6c7fff640 1 -- 172.21.15.98:0/3940346293 learned_addr learned my addr 172.21.15.98:0/3940346293 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:31.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.627+0000 7fd6ccc62640 1 --2- 172.21.15.98:0/3940346293 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6c8140640 0x7fd6c8151760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:31.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.627+0000 7fd6cd463640 1 --2- 172.21.15.98:0/3940346293 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6c815d200 0x7fd6c8142d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:31.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.627+0000 7fd6c7fff640 1 -- 172.21.15.98:0/3940346293 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6c815d200 msgr2=0x7fd6c8142d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:31.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.627+0000 7fd6c7fff640 1 --2- 172.21.15.98:0/3940346293 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6c815d200 0x7fd6c8142d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:31.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.627+0000 7fd6c7fff640 1 -- 172.21.15.98:0/3940346293 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6c8140640 msgr2=0x7fd6c8151760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:31.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.627+0000 7fd6c7fff640 1 --2- 172.21.15.98:0/3940346293 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6c8140640 0x7fd6c8151760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:31.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.627+0000 7fd6c7fff640 1 -- 172.21.15.98:0/3940346293 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c8152190 con 0x7fd6c8151ca0 2024-09-19T08:45:31.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.627+0000 7fd6c7fff640 1 --2- 172.21.15.98:0/3940346293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6c8151ca0 0x7fd6c815ccc0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fd6b400dae0 tx=0x7fd6b4007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:31.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.627+0000 7fd6c57fa640 1 -- 172.21.15.98:0/3940346293 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b4019070 con 0x7fd6c8151ca0 2024-09-19T08:45:31.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.627+0000 7fd6c57fa640 1 -- 172.21.15.98:0/3940346293 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6b4004cc0 con 0x7fd6c8151ca0 2024-09-19T08:45:31.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.627+0000 7fd6ceec6640 1 -- 172.21.15.98:0/3940346293 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6c8152470 con 0x7fd6c8151ca0 2024-09-19T08:45:31.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.627+0000 7fd6c57fa640 1 -- 172.21.15.98:0/3940346293 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b40072f0 con 0x7fd6c8151ca0 2024-09-19T08:45:31.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.627+0000 7fd6ceec6640 1 -- 172.21.15.98:0/3940346293 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6c8152a30 con 0x7fd6c8151ca0 2024-09-19T08:45:31.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.627+0000 7fd6ceec6640 1 -- 172.21.15.98:0/3940346293 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6c815d610 con 0x7fd6c8151ca0 2024-09-19T08:45:31.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.627+0000 7fd6c57fa640 1 -- 172.21.15.98:0/3940346293 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fd6b401c070 con 0x7fd6c8151ca0 2024-09-19T08:45:31.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.627+0000 7fd6c57fa640 1 --2- 172.21.15.98:0/3940346293 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd6ac067e60 0x7fd6ac06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:31.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.627+0000 7fd6c57fa640 1 -- 172.21.15.98:0/3940346293 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7fd6b4086280 con 0x7fd6c8151ca0 2024-09-19T08:45:31.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.627+0000 7fd6ccc62640 1 --2- 172.21.15.98:0/3940346293 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd6ac067e60 0x7fd6ac06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:31.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.627+0000 7fd6ccc62640 1 --2- 172.21.15.98:0/3940346293 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd6ac067e60 0x7fd6ac06a320 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc0af000 tx=0x7fd6bc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:31.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.631+0000 7fd6c57fa640 1 -- 172.21.15.98:0/3940346293 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd6c815d610 con 0x7fd6c8151ca0 2024-09-19T08:45:31.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.779+0000 7fd6ceec6640 1 -- 172.21.15.98:0/3940346293 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd6c8153230 con 0x7fd6ac067e60 2024-09-19T08:45:31.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.799+0000 7fd6c57fa640 1 -- 172.21.15.98:0/3940346293 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252420 (secure 0 0 0) 0x7fd6c8153230 con 0x7fd6ac067e60 2024-09-19T08:45:31.803 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:45:31.805 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:45:31.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.803+0000 7fd6ceec6640 1 -- 172.21.15.98:0/3940346293 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd6ac067e60 msgr2=0x7fd6ac06a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:31.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.803+0000 7fd6ceec6640 1 --2- 172.21.15.98:0/3940346293 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd6ac067e60 0x7fd6ac06a320 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc0af000 tx=0x7fd6bc008040 comp rx=0 tx=0).stop 2024-09-19T08:45:31.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.803+0000 7fd6ceec6640 1 -- 172.21.15.98:0/3940346293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6c8151ca0 msgr2=0x7fd6c815ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:31.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.803+0000 7fd6ceec6640 1 --2- 172.21.15.98:0/3940346293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6c8151ca0 0x7fd6c815ccc0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fd6b400dae0 tx=0x7fd6b4007e40 comp rx=0 tx=0).stop 2024-09-19T08:45:31.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.803+0000 7fd6ceec6640 1 -- 172.21.15.98:0/3940346293 shutdown_connections 2024-09-19T08:45:31.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.803+0000 7fd6ceec6640 1 --2- 172.21.15.98:0/3940346293 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd6ac067e60 0x7fd6ac06a320 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:31.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.803+0000 7fd6ceec6640 1 --2- 172.21.15.98:0/3940346293 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6c815d200 0x7fd6c8142d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:31.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.803+0000 7fd6ceec6640 1 --2- 172.21.15.98:0/3940346293 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6c8151ca0 0x7fd6c815ccc0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:31.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.803+0000 7fd6ceec6640 1 --2- 172.21.15.98:0/3940346293 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6c8140640 0x7fd6c8151760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:31.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.803+0000 7fd6ceec6640 1 -- 172.21.15.98:0/3940346293 >> 172.21.15.98:0/3940346293 conn(0x7fd6c80b2010 msgr2=0x7fd6c815ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:31.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.803+0000 7fd6ceec6640 1 -- 172.21.15.98:0/3940346293 shutdown_connections 2024-09-19T08:45:31.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.803+0000 7fd6ceec6640 1 -- 172.21.15.98:0/3940346293 wait complete. 2024-09-19T08:45:31.821 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-09-19T08:45:30.922248+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":101,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263672,"kb_used_data":14136,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711048,"statfs":{"total":773094113280,"available":772824113152,"internally_reserved":0,"allocated":14475264,"data_stored":7072513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.160038"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.820463+0000","last_change":"2024-09-19T08:45:28.818532+0000","last_active":"2024-09-19T08:45:28.820463+0000","last_peered":"2024-09-19T08:45:28.820463+0000","last_clean":"2024-09-19T08:45:28.820463+0000","last_became_active":"2024-09-19T08:45:28.818086+0000","last_became_peered":"2024-09-19T08:45:28.818086+0000","last_unstale":"2024-09-19T08:45:28.820463+0000","last_undegraded":"2024-09-19T08:45:28.820463+0000","last_fullsized":"2024-09-19T08:45:28.820463+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:01.374609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.790613+0000","last_change":"2024-09-19T08:45:28.789674+0000","last_active":"2024-09-19T08:45:28.790613+0000","last_peered":"2024-09-19T08:45:28.790613+0000","last_clean":"2024-09-19T08:45:28.790613+0000","last_became_active":"2024-09-19T08:45:28.789349+0000","last_became_peered":"2024-09-19T08:45:28.789349+0000","last_unstale":"2024-09-19T08:45:28.790613+0000","last_undegraded":"2024-09-19T08:45:28.790613+0000","last_fullsized":"2024-09-19T08:45:28.790613+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:32:17.965494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.809339+0000","last_change":"2024-09-19T08:45:28.792925+0000","last_active":"2024-09-19T08:45:28.809339+0000","last_peered":"2024-09-19T08:45:28.809339+0000","last_clean":"2024-09-19T08:45:28.809339+0000","last_became_active":"2024-09-19T08:45:28.792658+0000","last_became_peered":"2024-09-19T08:45:28.792658+0000","last_unstale":"2024-09-19T08:45:28.809339+0000","last_undegraded":"2024-09-19T08:45:28.809339+0000","last_fullsized":"2024-09-19T08:45:28.809339+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:14.888029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.801609+0000","last_change":"2024-09-19T08:45:28.794262+0000","last_active":"2024-09-19T08:45:28.801609+0000","last_peered":"2024-09-19T08:45:28.801609+0000","last_clean":"2024-09-19T08:45:28.801609+0000","last_became_active":"2024-09-19T08:45:28.793953+0000","last_became_peered":"2024-09-19T08:45:28.793953+0000","last_unstale":"2024-09-19T08:45:28.801609+0000","last_undegraded":"2024-09-19T08:45:28.801609+0000","last_fullsized":"2024-09-19T08:45:28.801609+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:26.181098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.821104+0000","last_change":"2024-09-19T08:45:28.818516+0000","last_active":"2024-09-19T08:45:28.821104+0000","last_peered":"2024-09-19T08:45:28.821104+0000","last_clean":"2024-09-19T08:45:28.821104+0000","last_became_active":"2024-09-19T08:45:28.818203+0000","last_became_peered":"2024-09-19T08:45:28.818203+0000","last_unstale":"2024-09-19T08:45:28.821104+0000","last_undegraded":"2024-09-19T08:45:28.821104+0000","last_fullsized":"2024-09-19T08:45:28.821104+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:50:38.932478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.820406+0000","last_change":"2024-09-19T08:45:28.818155+0000","last_active":"2024-09-19T08:45:28.820406+0000","last_peered":"2024-09-19T08:45:28.820406+0000","last_clean":"2024-09-19T08:45:28.820406+0000","last_became_active":"2024-09-19T08:45:28.817366+0000","last_became_peered":"2024-09-19T08:45:28.817366+0000","last_unstale":"2024-09-19T08:45:28.820406+0000","last_undegraded":"2024-09-19T08:45:28.820406+0000","last_fullsized":"2024-09-19T08:45:28.820406+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:30:56.900258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.813830+0000","last_change":"2024-09-19T08:45:28.810318+0000","last_active":"2024-09-19T08:45:28.813830+0000","last_peered":"2024-09-19T08:45:28.813830+0000","last_clean":"2024-09-19T08:45:28.813830+0000","last_became_active":"2024-09-19T08:45:28.807727+0000","last_became_peered":"2024-09-19T08:45:28.807727+0000","last_unstale":"2024-09-19T08:45:28.813830+0000","last_undegraded":"2024-09-19T08:45:28.813830+0000","last_fullsized":"2024-09-19T08:45:28.813830+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:18.902222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.817864+0000","last_change":"2024-09-19T08:45:28.816065+0000","last_active":"2024-09-19T08:45:28.817864+0000","last_peered":"2024-09-19T08:45:28.817864+0000","last_clean":"2024-09-19T08:45:28.817864+0000","last_became_active":"2024-09-19T08:45:28.810847+0000","last_became_peered":"2024-09-19T08:45:28.810847+0000","last_unstale":"2024-09-19T08:45:28.817864+0000","last_undegraded":"2024-09-19T08:45:28.817864+0000","last_fullsized":"2024-09-19T08:45:28.817864+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:21:50.849379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.812893+0000","last_change":"2024-09-19T08:45:28.811675+0000","last_active":"2024-09-19T08:45:28.812893+0000","last_peered":"2024-09-19T08:45:28.812893+0000","last_clean":"2024-09-19T08:45:28.812893+0000","last_became_active":"2024-09-19T08:45:28.811341+0000","last_became_peered":"2024-09-19T08:45:28.811341+0000","last_unstale":"2024-09-19T08:45:28.812893+0000","last_undegraded":"2024-09-19T08:45:28.812893+0000","last_fullsized":"2024-09-19T08:45:28.812893+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:30:13.270651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.814971+0000","last_change":"2024-09-19T08:45:28.799996+0000","last_active":"2024-09-19T08:45:28.814971+0000","last_peered":"2024-09-19T08:45:28.814971+0000","last_clean":"2024-09-19T08:45:28.814971+0000","last_became_active":"2024-09-19T08:45:28.799716+0000","last_became_peered":"2024-09-19T08:45:28.799716+0000","last_unstale":"2024-09-19T08:45:28.814971+0000","last_undegraded":"2024-09-19T08:45:28.814971+0000","last_fullsized":"2024-09-19T08:45:28.814971+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:46:06.635440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.817672+0000","last_change":"2024-09-19T08:45:28.810317+0000","last_active":"2024-09-19T08:45:28.817672+0000","last_peered":"2024-09-19T08:45:28.817672+0000","last_clean":"2024-09-19T08:45:28.817672+0000","last_became_active":"2024-09-19T08:45:28.809726+0000","last_became_peered":"2024-09-19T08:45:28.809726+0000","last_unstale":"2024-09-19T08:45:28.817672+0000","last_undegraded":"2024-09-19T08:45:28.817672+0000","last_fullsized":"2024-09-19T08:45:28.817672+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:46.076410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.816595+0000","last_change":"2024-09-19T08:45:28.814040+0000","last_active":"2024-09-19T08:45:28.816595+0000","last_peered":"2024-09-19T08:45:28.816595+0000","last_clean":"2024-09-19T08:45:28.816595+0000","last_became_active":"2024-09-19T08:45:28.809048+0000","last_became_peered":"2024-09-19T08:45:28.809048+0000","last_unstale":"2024-09-19T08:45:28.816595+0000","last_undegraded":"2024-09-19T08:45:28.816595+0000","last_fullsized":"2024-09-19T08:45:28.816595+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:32:04.302543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.819178+0000","last_change":"2024-09-19T08:45:28.817217+0000","last_active":"2024-09-19T08:45:28.819178+0000","last_peered":"2024-09-19T08:45:28.819178+0000","last_clean":"2024-09-19T08:45:28.819178+0000","last_became_active":"2024-09-19T08:45:28.816948+0000","last_became_peered":"2024-09-19T08:45:28.816948+0000","last_unstale":"2024-09-19T08:45:28.819178+0000","last_undegraded":"2024-09-19T08:45:28.819178+0000","last_fullsized":"2024-09-19T08:45:28.819178+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:44.033478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.811462+0000","last_change":"2024-09-19T08:45:28.810385+0000","last_active":"2024-09-19T08:45:28.811462+0000","last_peered":"2024-09-19T08:45:28.811462+0000","last_clean":"2024-09-19T08:45:28.811462+0000","last_became_active":"2024-09-19T08:45:28.807669+0000","last_became_peered":"2024-09-19T08:45:28.807669+0000","last_unstale":"2024-09-19T08:45:28.811462+0000","last_undegraded":"2024-09-19T08:45:28.811462+0000","last_fullsized":"2024-09-19T08:45:28.811462+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:21:45.051375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.815205+0000","last_change":"2024-09-19T08:45:28.799989+0000","last_active":"2024-09-19T08:45:28.815205+0000","last_peered":"2024-09-19T08:45:28.815205+0000","last_clean":"2024-09-19T08:45:28.815205+0000","last_became_active":"2024-09-19T08:45:28.799254+0000","last_became_peered":"2024-09-19T08:45:28.799254+0000","last_unstale":"2024-09-19T08:45:28.815205+0000","last_undegraded":"2024-09-19T08:45:28.815205+0000","last_fullsized":"2024-09-19T08:45:28.815205+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:17.355685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.824354+0000","last_change":"2024-09-19T08:45:28.818924+0000","last_active":"2024-09-19T08:45:28.824354+0000","last_peered":"2024-09-19T08:45:28.824354+0000","last_clean":"2024-09-19T08:45:28.824354+0000","last_became_active":"2024-09-19T08:45:28.817375+0000","last_became_peered":"2024-09-19T08:45:28.817375+0000","last_unstale":"2024-09-19T08:45:28.824354+0000","last_undegraded":"2024-09-19T08:45:28.824354+0000","last_fullsized":"2024-09-19T08:45:28.824354+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:34.570425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.815330+0000","last_change":"2024-09-19T08:45:28.798749+0000","last_active":"2024-09-19T08:45:28.815330+0000","last_peered":"2024-09-19T08:45:28.815330+0000","last_clean":"2024-09-19T08:45:28.815330+0000","last_became_active":"2024-09-19T08:45:28.798192+0000","last_became_peered":"2024-09-19T08:45:28.798192+0000","last_unstale":"2024-09-19T08:45:28.815330+0000","last_undegraded":"2024-09-19T08:45:28.815330+0000","last_fullsized":"2024-09-19T08:45:28.815330+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:47:13.332636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.803992+0000","last_change":"2024-09-19T08:45:28.793006+0000","last_active":"2024-09-19T08:45:28.803992+0000","last_peered":"2024-09-19T08:45:28.803992+0000","last_clean":"2024-09-19T08:45:28.803992+0000","last_became_active":"2024-09-19T08:45:28.792594+0000","last_became_peered":"2024-09-19T08:45:28.792594+0000","last_unstale":"2024-09-19T08:45:28.803992+0000","last_undegraded":"2024-09-19T08:45:28.803992+0000","last_fullsized":"2024-09-19T08:45:28.803992+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:36.941444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.818367+0000","last_change":"2024-09-19T08:45:28.817164+0000","last_active":"2024-09-19T08:45:28.818367+0000","last_peered":"2024-09-19T08:45:28.818367+0000","last_clean":"2024-09-19T08:45:28.818367+0000","last_became_active":"2024-09-19T08:45:28.816853+0000","last_became_peered":"2024-09-19T08:45:28.816853+0000","last_unstale":"2024-09-19T08:45:28.818367+0000","last_undegraded":"2024-09-19T08:45:28.818367+0000","last_fullsized":"2024-09-19T08:45:28.818367+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:43.161108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.815058+0000","last_change":"2024-09-19T08:45:28.793134+0000","last_active":"2024-09-19T08:45:28.815058+0000","last_peered":"2024-09-19T08:45:28.815058+0000","last_clean":"2024-09-19T08:45:28.815058+0000","last_became_active":"2024-09-19T08:45:28.792715+0000","last_became_peered":"2024-09-19T08:45:28.792715+0000","last_unstale":"2024-09-19T08:45:28.815058+0000","last_undegraded":"2024-09-19T08:45:28.815058+0000","last_fullsized":"2024-09-19T08:45:28.815058+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:17.303164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.799632+0000","last_change":"2024-09-19T08:45:28.798462+0000","last_active":"2024-09-19T08:45:28.799632+0000","last_peered":"2024-09-19T08:45:28.799632+0000","last_clean":"2024-09-19T08:45:28.799632+0000","last_became_active":"2024-09-19T08:45:28.798115+0000","last_became_peered":"2024-09-19T08:45:28.798115+0000","last_unstale":"2024-09-19T08:45:28.799632+0000","last_undegraded":"2024-09-19T08:45:28.799632+0000","last_fullsized":"2024-09-19T08:45:28.799632+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:33.019482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.804069+0000","last_change":"2024-09-19T08:45:28.797340+0000","last_active":"2024-09-19T08:45:28.804069+0000","last_peered":"2024-09-19T08:45:28.804069+0000","last_clean":"2024-09-19T08:45:28.804069+0000","last_became_active":"2024-09-19T08:45:28.796312+0000","last_became_peered":"2024-09-19T08:45:28.796312+0000","last_unstale":"2024-09-19T08:45:28.804069+0000","last_undegraded":"2024-09-19T08:45:28.804069+0000","last_fullsized":"2024-09-19T08:45:28.804069+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:34:59.424826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.811737+0000","last_change":"2024-09-19T08:45:28.808317+0000","last_active":"2024-09-19T08:45:28.811737+0000","last_peered":"2024-09-19T08:45:28.811737+0000","last_clean":"2024-09-19T08:45:28.811737+0000","last_became_active":"2024-09-19T08:45:28.808001+0000","last_became_peered":"2024-09-19T08:45:28.808001+0000","last_unstale":"2024-09-19T08:45:28.811737+0000","last_undegraded":"2024-09-19T08:45:28.811737+0000","last_fullsized":"2024-09-19T08:45:28.811737+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:57:28.874099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.799571+0000","last_change":"2024-09-19T08:45:28.788955+0000","last_active":"2024-09-19T08:45:28.799571+0000","last_peered":"2024-09-19T08:45:28.799571+0000","last_clean":"2024-09-19T08:45:28.799571+0000","last_became_active":"2024-09-19T08:45:28.788634+0000","last_became_peered":"2024-09-19T08:45:28.788634+0000","last_unstale":"2024-09-19T08:45:28.799571+0000","last_undegraded":"2024-09-19T08:45:28.799571+0000","last_fullsized":"2024-09-19T08:45:28.799571+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:46.650745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.824485+0000","last_change":"2024-09-19T08:45:28.819371+0000","last_active":"2024-09-19T08:45:28.824485+0000","last_peered":"2024-09-19T08:45:28.824485+0000","last_clean":"2024-09-19T08:45:28.824485+0000","last_became_active":"2024-09-19T08:45:28.819089+0000","last_became_peered":"2024-09-19T08:45:28.819089+0000","last_unstale":"2024-09-19T08:45:28.824485+0000","last_undegraded":"2024-09-19T08:45:28.824485+0000","last_fullsized":"2024-09-19T08:45:28.824485+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:19:22.434059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.810802+0000","last_change":"2024-09-19T08:45:28.806385+0000","last_active":"2024-09-19T08:45:28.810802+0000","last_peered":"2024-09-19T08:45:28.810802+0000","last_clean":"2024-09-19T08:45:28.810802+0000","last_became_active":"2024-09-19T08:45:28.802540+0000","last_became_peered":"2024-09-19T08:45:28.802540+0000","last_unstale":"2024-09-19T08:45:28.810802+0000","last_undegraded":"2024-09-19T08:45:28.810802+0000","last_fullsized":"2024-09-19T08:45:28.810802+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:56.173971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.824256+0000","last_change":"2024-09-19T08:45:28.817893+0000","last_active":"2024-09-19T08:45:28.824256+0000","last_peered":"2024-09-19T08:45:28.824256+0000","last_clean":"2024-09-19T08:45:28.824256+0000","last_became_active":"2024-09-19T08:45:28.817647+0000","last_became_peered":"2024-09-19T08:45:28.817647+0000","last_unstale":"2024-09-19T08:45:28.824256+0000","last_undegraded":"2024-09-19T08:45:28.824256+0000","last_fullsized":"2024-09-19T08:45:28.824256+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:55:06.028325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.814059+0000","last_change":"2024-09-19T08:45:28.812274+0000","last_active":"2024-09-19T08:45:28.814059+0000","last_peered":"2024-09-19T08:45:28.814059+0000","last_clean":"2024-09-19T08:45:28.814059+0000","last_became_active":"2024-09-19T08:45:28.811997+0000","last_became_peered":"2024-09-19T08:45:28.811997+0000","last_unstale":"2024-09-19T08:45:28.814059+0000","last_undegraded":"2024-09-19T08:45:28.814059+0000","last_fullsized":"2024-09-19T08:45:28.814059+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:35:47.868929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.810341+0000","last_change":"2024-09-19T08:45:28.802035+0000","last_active":"2024-09-19T08:45:28.810341+0000","last_peered":"2024-09-19T08:45:28.810341+0000","last_clean":"2024-09-19T08:45:28.810341+0000","last_became_active":"2024-09-19T08:45:28.801460+0000","last_became_peered":"2024-09-19T08:45:28.801460+0000","last_unstale":"2024-09-19T08:45:28.810341+0000","last_undegraded":"2024-09-19T08:45:28.810341+0000","last_fullsized":"2024-09-19T08:45:28.810341+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:03.685888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.823199+0000","last_change":"2024-09-19T08:45:28.821114+0000","last_active":"2024-09-19T08:45:28.823199+0000","last_peered":"2024-09-19T08:45:28.823199+0000","last_clean":"2024-09-19T08:45:28.823199+0000","last_became_active":"2024-09-19T08:45:28.820712+0000","last_became_peered":"2024-09-19T08:45:28.820712+0000","last_unstale":"2024-09-19T08:45:28.823199+0000","last_undegraded":"2024-09-19T08:45:28.823199+0000","last_fullsized":"2024-09-19T08:45:28.823199+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:04.999051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.799164+0000","last_change":"2024-09-19T08:45:28.791517+0000","last_active":"2024-09-19T08:45:28.799164+0000","last_peered":"2024-09-19T08:45:28.799164+0000","last_clean":"2024-09-19T08:45:28.799164+0000","last_became_active":"2024-09-19T08:45:28.791247+0000","last_became_peered":"2024-09-19T08:45:28.791247+0000","last_unstale":"2024-09-19T08:45:28.799164+0000","last_undegraded":"2024-09-19T08:45:28.799164+0000","last_fullsized":"2024-09-19T08:45:28.799164+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:26:17.904918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.824318+0000","last_change":"2024-09-19T08:45:28.818666+0000","last_active":"2024-09-19T08:45:28.824318+0000","last_peered":"2024-09-19T08:45:28.824318+0000","last_clean":"2024-09-19T08:45:28.824318+0000","last_became_active":"2024-09-19T08:45:28.813155+0000","last_became_peered":"2024-09-19T08:45:28.813155+0000","last_unstale":"2024-09-19T08:45:28.824318+0000","last_undegraded":"2024-09-19T08:45:28.824318+0000","last_fullsized":"2024-09-19T08:45:28.824318+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:37:10.149152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.816656+0000","last_change":"2024-09-19T08:45:28.814034+0000","last_active":"2024-09-19T08:45:28.816656+0000","last_peered":"2024-09-19T08:45:28.816656+0000","last_clean":"2024-09-19T08:45:28.816656+0000","last_became_active":"2024-09-19T08:45:28.813724+0000","last_became_peered":"2024-09-19T08:45:28.813724+0000","last_unstale":"2024-09-19T08:45:28.816656+0000","last_undegraded":"2024-09-19T08:45:28.816656+0000","last_fullsized":"2024-09-19T08:45:28.816656+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:07.612401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"134'4","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.826792+0000","last_change":"2024-09-19T08:45:27.820921+0000","last_active":"2024-09-19T08:45:28.826792+0000","last_peered":"2024-09-19T08:45:28.826792+0000","last_clean":"2024-09-19T08:45:28.826792+0000","last_became_active":"2024-09-19T08:45:27.820200+0000","last_became_peered":"2024-09-19T08:45:27.820200+0000","last_unstale":"2024-09-19T08:45:28.826792+0000","last_undegraded":"2024-09-19T08:45:28.826792+0000","last_fullsized":"2024-09-19T08:45:28.826792+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:22.555962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.802488+0000","last_change":"2024-09-19T08:45:28.798810+0000","last_active":"2024-09-19T08:45:28.802488+0000","last_peered":"2024-09-19T08:45:28.802488+0000","last_clean":"2024-09-19T08:45:28.802488+0000","last_became_active":"2024-09-19T08:45:28.795552+0000","last_became_peered":"2024-09-19T08:45:28.795552+0000","last_unstale":"2024-09-19T08:45:28.802488+0000","last_undegraded":"2024-09-19T08:45:28.802488+0000","last_fullsized":"2024-09-19T08:45:28.802488+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:18.681030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"134'1","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.817189+0000","last_change":"2024-09-19T08:45:27.816108+0000","last_active":"2024-09-19T08:45:28.817189+0000","last_peered":"2024-09-19T08:45:28.817189+0000","last_clean":"2024-09-19T08:45:28.817189+0000","last_became_active":"2024-09-19T08:45:27.815701+0000","last_became_peered":"2024-09-19T08:45:27.815701+0000","last_unstale":"2024-09-19T08:45:28.817189+0000","last_undegraded":"2024-09-19T08:45:28.817189+0000","last_fullsized":"2024-09-19T08:45:28.817189+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:45.554977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.817855+0000","last_change":"2024-09-19T08:45:28.810608+0000","last_active":"2024-09-19T08:45:28.817855+0000","last_peered":"2024-09-19T08:45:28.817855+0000","last_clean":"2024-09-19T08:45:28.817855+0000","last_became_active":"2024-09-19T08:45:28.809370+0000","last_became_peered":"2024-09-19T08:45:28.809370+0000","last_unstale":"2024-09-19T08:45:28.817855+0000","last_undegraded":"2024-09-19T08:45:28.817855+0000","last_fullsized":"2024-09-19T08:45:28.817855+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:44.272919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"134'2","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.815417+0000","last_change":"2024-09-19T08:45:27.797683+0000","last_active":"2024-09-19T08:45:28.815417+0000","last_peered":"2024-09-19T08:45:28.815417+0000","last_clean":"2024-09-19T08:45:28.815417+0000","last_became_active":"2024-09-19T08:45:27.796852+0000","last_became_peered":"2024-09-19T08:45:27.796852+0000","last_unstale":"2024-09-19T08:45:28.815417+0000","last_undegraded":"2024-09-19T08:45:28.815417+0000","last_fullsized":"2024-09-19T08:45:28.815417+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:31:45.059438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.815736+0000","last_change":"2024-09-19T08:45:28.807910+0000","last_active":"2024-09-19T08:45:28.815736+0000","last_peered":"2024-09-19T08:45:28.815736+0000","last_clean":"2024-09-19T08:45:28.815736+0000","last_became_active":"2024-09-19T08:45:28.807605+0000","last_became_peered":"2024-09-19T08:45:28.807605+0000","last_unstale":"2024-09-19T08:45:28.815736+0000","last_undegraded":"2024-09-19T08:45:28.815736+0000","last_fullsized":"2024-09-19T08:45:28.815736+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:33:31.910044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"134'2","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.820526+0000","last_change":"2024-09-19T08:45:27.786505+0000","last_active":"2024-09-19T08:45:28.820526+0000","last_peered":"2024-09-19T08:45:28.820526+0000","last_clean":"2024-09-19T08:45:28.820526+0000","last_became_active":"2024-09-19T08:45:27.785674+0000","last_became_peered":"2024-09-19T08:45:27.785674+0000","last_unstale":"2024-09-19T08:45:28.820526+0000","last_undegraded":"2024-09-19T08:45:28.820526+0000","last_fullsized":"2024-09-19T08:45:28.820526+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:10:17.980862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.799645+0000","last_change":"2024-09-19T08:45:28.791665+0000","last_active":"2024-09-19T08:45:28.799645+0000","last_peered":"2024-09-19T08:45:28.799645+0000","last_clean":"2024-09-19T08:45:28.799645+0000","last_became_active":"2024-09-19T08:45:28.791349+0000","last_became_peered":"2024-09-19T08:45:28.791349+0000","last_unstale":"2024-09-19T08:45:28.799645+0000","last_undegraded":"2024-09-19T08:45:28.799645+0000","last_fullsized":"2024-09-19T08:45:28.799645+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:21:09.499251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"134'1","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.824328+0000","last_change":"2024-09-19T08:45:27.809526+0000","last_active":"2024-09-19T08:45:28.824328+0000","last_peered":"2024-09-19T08:45:28.824328+0000","last_clean":"2024-09-19T08:45:28.824328+0000","last_became_active":"2024-09-19T08:45:27.809171+0000","last_became_peered":"2024-09-19T08:45:27.809171+0000","last_unstale":"2024-09-19T08:45:28.824328+0000","last_undegraded":"2024-09-19T08:45:28.824328+0000","last_fullsized":"2024-09-19T08:45:28.824328+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:00:04.373962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.801677+0000","last_change":"2024-09-19T08:45:28.794324+0000","last_active":"2024-09-19T08:45:28.801677+0000","last_peered":"2024-09-19T08:45:28.801677+0000","last_clean":"2024-09-19T08:45:28.801677+0000","last_became_active":"2024-09-19T08:45:28.793947+0000","last_became_peered":"2024-09-19T08:45:28.793947+0000","last_unstale":"2024-09-19T08:45:28.801677+0000","last_undegraded":"2024-09-19T08:45:28.801677+0000","last_fullsized":"2024-09-19T08:45:28.801677+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:37.913412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"134'1","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.818010+0000","last_change":"2024-09-19T08:45:27.815404+0000","last_active":"2024-09-19T08:45:28.818010+0000","last_peered":"2024-09-19T08:45:28.818010+0000","last_clean":"2024-09-19T08:45:28.818010+0000","last_became_active":"2024-09-19T08:45:27.814735+0000","last_became_peered":"2024-09-19T08:45:27.814735+0000","last_unstale":"2024-09-19T08:45:28.818010+0000","last_undegraded":"2024-09-19T08:45:28.818010+0000","last_fullsized":"2024-09-19T08:45:28.818010+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:26:29.893419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.802919+0000","last_change":"2024-09-19T08:45:28.799288+0000","last_active":"2024-09-19T08:45:28.802919+0000","last_peered":"2024-09-19T08:45:28.802919+0000","last_clean":"2024-09-19T08:45:28.802919+0000","last_became_active":"2024-09-19T08:45:28.795468+0000","last_became_peered":"2024-09-19T08:45:28.795468+0000","last_unstale":"2024-09-19T08:45:28.802919+0000","last_undegraded":"2024-09-19T08:45:28.802919+0000","last_fullsized":"2024-09-19T08:45:28.802919+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:09.023436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.794533+0000","last_change":"2024-09-19T08:45:27.797807+0000","last_active":"2024-09-19T08:45:28.794533+0000","last_peered":"2024-09-19T08:45:28.794533+0000","last_clean":"2024-09-19T08:45:28.794533+0000","last_became_active":"2024-09-19T08:45:27.797423+0000","last_became_peered":"2024-09-19T08:45:27.797423+0000","last_unstale":"2024-09-19T08:45:28.794533+0000","last_undegraded":"2024-09-19T08:45:28.794533+0000","last_fullsized":"2024-09-19T08:45:28.794533+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:09.728422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.812066+0000","last_change":"2024-09-19T08:45:28.811558+0000","last_active":"2024-09-19T08:45:28.812066+0000","last_peered":"2024-09-19T08:45:28.812066+0000","last_clean":"2024-09-19T08:45:28.812066+0000","last_became_active":"2024-09-19T08:45:28.811313+0000","last_became_peered":"2024-09-19T08:45:28.811313+0000","last_unstale":"2024-09-19T08:45:28.812066+0000","last_undegraded":"2024-09-19T08:45:28.812066+0000","last_fullsized":"2024-09-19T08:45:28.812066+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:30:14.734675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.785492+0000","last_change":"2024-09-19T08:45:27.811525+0000","last_active":"2024-09-19T08:45:28.785492+0000","last_peered":"2024-09-19T08:45:28.785492+0000","last_clean":"2024-09-19T08:45:28.785492+0000","last_became_active":"2024-09-19T08:45:27.810635+0000","last_became_peered":"2024-09-19T08:45:27.810635+0000","last_unstale":"2024-09-19T08:45:28.785492+0000","last_undegraded":"2024-09-19T08:45:28.785492+0000","last_fullsized":"2024-09-19T08:45:28.785492+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:10.320962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.810560+0000","last_change":"2024-09-19T08:45:28.810166+0000","last_active":"2024-09-19T08:45:28.810560+0000","last_peered":"2024-09-19T08:45:28.810560+0000","last_clean":"2024-09-19T08:45:28.810560+0000","last_became_active":"2024-09-19T08:45:28.809816+0000","last_became_peered":"2024-09-19T08:45:28.809816+0000","last_unstale":"2024-09-19T08:45:28.810560+0000","last_undegraded":"2024-09-19T08:45:28.810560+0000","last_fullsized":"2024-09-19T08:45:28.810560+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:44:16.379183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.780184+0000","last_change":"2024-09-19T08:45:27.793871+0000","last_active":"2024-09-19T08:45:28.780184+0000","last_peered":"2024-09-19T08:45:28.780184+0000","last_clean":"2024-09-19T08:45:28.780184+0000","last_became_active":"2024-09-19T08:45:27.792973+0000","last_became_peered":"2024-09-19T08:45:27.792973+0000","last_unstale":"2024-09-19T08:45:28.780184+0000","last_undegraded":"2024-09-19T08:45:28.780184+0000","last_fullsized":"2024-09-19T08:45:28.780184+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:04:48.478624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.815379+0000","last_change":"2024-09-19T08:45:28.798990+0000","last_active":"2024-09-19T08:45:28.815379+0000","last_peered":"2024-09-19T08:45:28.815379+0000","last_clean":"2024-09-19T08:45:28.815379+0000","last_became_active":"2024-09-19T08:45:28.798553+0000","last_became_peered":"2024-09-19T08:45:28.798553+0000","last_unstale":"2024-09-19T08:45:28.815379+0000","last_undegraded":"2024-09-19T08:45:28.815379+0000","last_fullsized":"2024-09-19T08:45:28.815379+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:15.792372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.781720+0000","last_change":"2024-09-19T08:45:27.823816+0000","last_active":"2024-09-19T08:45:28.781720+0000","last_peered":"2024-09-19T08:45:28.781720+0000","last_clean":"2024-09-19T08:45:28.781720+0000","last_became_active":"2024-09-19T08:45:27.822250+0000","last_became_peered":"2024-09-19T08:45:27.822250+0000","last_unstale":"2024-09-19T08:45:28.781720+0000","last_undegraded":"2024-09-19T08:45:28.781720+0000","last_fullsized":"2024-09-19T08:45:28.781720+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:34:56.926103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.821202+0000","last_change":"2024-09-19T08:45:28.814480+0000","last_active":"2024-09-19T08:45:28.821202+0000","last_peered":"2024-09-19T08:45:28.821202+0000","last_clean":"2024-09-19T08:45:28.821202+0000","last_became_active":"2024-09-19T08:45:28.807443+0000","last_became_peered":"2024-09-19T08:45:28.807443+0000","last_unstale":"2024-09-19T08:45:28.821202+0000","last_undegraded":"2024-09-19T08:45:28.821202+0000","last_fullsized":"2024-09-19T08:45:28.821202+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:19:10.453003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"134'6","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.826951+0000","last_change":"2024-09-19T08:45:27.791751+0000","last_active":"2024-09-19T08:45:28.826951+0000","last_peered":"2024-09-19T08:45:28.826951+0000","last_clean":"2024-09-19T08:45:28.826951+0000","last_became_active":"2024-09-19T08:45:27.791474+0000","last_became_peered":"2024-09-19T08:45:27.791474+0000","last_unstale":"2024-09-19T08:45:28.826951+0000","last_undegraded":"2024-09-19T08:45:28.826951+0000","last_fullsized":"2024-09-19T08:45:28.826951+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:16.351650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.808697+0000","last_change":"2024-09-19T08:45:28.803503+0000","last_active":"2024-09-19T08:45:28.808697+0000","last_peered":"2024-09-19T08:45:28.808697+0000","last_clean":"2024-09-19T08:45:28.808697+0000","last_became_active":"2024-09-19T08:45:28.802949+0000","last_became_peered":"2024-09-19T08:45:28.802949+0000","last_unstale":"2024-09-19T08:45:28.808697+0000","last_undegraded":"2024-09-19T08:45:28.808697+0000","last_fullsized":"2024-09-19T08:45:28.808697+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:24:43.012611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"134'2","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.816231+0000","last_change":"2024-09-19T08:45:27.817233+0000","last_active":"2024-09-19T08:45:28.816231+0000","last_peered":"2024-09-19T08:45:28.816231+0000","last_clean":"2024-09-19T08:45:28.816231+0000","last_became_active":"2024-09-19T08:45:27.809278+0000","last_became_peered":"2024-09-19T08:45:27.809278+0000","last_unstale":"2024-09-19T08:45:28.816231+0000","last_undegraded":"2024-09-19T08:45:28.816231+0000","last_fullsized":"2024-09-19T08:45:28.816231+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:43.220541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.789861+0000","last_change":"2024-09-19T08:45:28.788398+0000","last_active":"2024-09-19T08:45:28.789861+0000","last_peered":"2024-09-19T08:45:28.789861+0000","last_clean":"2024-09-19T08:45:28.789861+0000","last_became_active":"2024-09-19T08:45:28.788104+0000","last_became_peered":"2024-09-19T08:45:28.788104+0000","last_unstale":"2024-09-19T08:45:28.789861+0000","last_undegraded":"2024-09-19T08:45:28.789861+0000","last_fullsized":"2024-09-19T08:45:28.789861+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:51:54.858323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.788696+0000","last_change":"2024-09-19T08:45:27.809112+0000","last_active":"2024-09-19T08:45:28.788696+0000","last_peered":"2024-09-19T08:45:28.788696+0000","last_clean":"2024-09-19T08:45:28.788696+0000","last_became_active":"2024-09-19T08:45:27.807985+0000","last_became_peered":"2024-09-19T08:45:27.807985+0000","last_unstale":"2024-09-19T08:45:28.788696+0000","last_undegraded":"2024-09-19T08:45:28.788696+0000","last_fullsized":"2024-09-19T08:45:28.788696+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:33:29.140035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.821122+0000","last_change":"2024-09-19T08:45:28.817985+0000","last_active":"2024-09-19T08:45:28.821122+0000","last_peered":"2024-09-19T08:45:28.821122+0000","last_clean":"2024-09-19T08:45:28.821122+0000","last_became_active":"2024-09-19T08:45:28.817596+0000","last_became_peered":"2024-09-19T08:45:28.817596+0000","last_unstale":"2024-09-19T08:45:28.821122+0000","last_undegraded":"2024-09-19T08:45:28.821122+0000","last_fullsized":"2024-09-19T08:45:28.821122+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:48.791756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.784703+0000","last_change":"2024-09-19T08:45:27.793154+0000","last_active":"2024-09-19T08:45:28.784703+0000","last_peered":"2024-09-19T08:45:28.784703+0000","last_clean":"2024-09-19T08:45:28.784703+0000","last_became_active":"2024-09-19T08:45:27.792439+0000","last_became_peered":"2024-09-19T08:45:27.792439+0000","last_unstale":"2024-09-19T08:45:28.784703+0000","last_undegraded":"2024-09-19T08:45:28.784703+0000","last_fullsized":"2024-09-19T08:45:28.784703+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:34:02.879548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.823661+0000","last_change":"2024-09-19T08:45:28.817504+0000","last_active":"2024-09-19T08:45:28.823661+0000","last_peered":"2024-09-19T08:45:28.823661+0000","last_clean":"2024-09-19T08:45:28.823661+0000","last_became_active":"2024-09-19T08:45:28.817214+0000","last_became_peered":"2024-09-19T08:45:28.817214+0000","last_unstale":"2024-09-19T08:45:28.823661+0000","last_undegraded":"2024-09-19T08:45:28.823661+0000","last_fullsized":"2024-09-19T08:45:28.823661+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:35.661617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"134'8","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.825331+0000","last_change":"2024-09-19T08:45:27.808101+0000","last_active":"2024-09-19T08:45:28.825331+0000","last_peered":"2024-09-19T08:45:28.825331+0000","last_clean":"2024-09-19T08:45:28.825331+0000","last_became_active":"2024-09-19T08:45:27.807651+0000","last_became_peered":"2024-09-19T08:45:27.807651+0000","last_unstale":"2024-09-19T08:45:28.825331+0000","last_undegraded":"2024-09-19T08:45:28.825331+0000","last_fullsized":"2024-09-19T08:45:28.825331+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:59.019811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.823344+0000","last_change":"2024-09-19T08:45:28.819904+0000","last_active":"2024-09-19T08:45:28.823344+0000","last_peered":"2024-09-19T08:45:28.823344+0000","last_clean":"2024-09-19T08:45:28.823344+0000","last_became_active":"2024-09-19T08:45:28.819610+0000","last_became_peered":"2024-09-19T08:45:28.819610+0000","last_unstale":"2024-09-19T08:45:28.823344+0000","last_undegraded":"2024-09-19T08:45:28.823344+0000","last_fullsized":"2024-09-19T08:45:28.823344+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:45.220093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"134'1","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.808754+0000","last_change":"2024-09-19T08:45:27.810578+0000","last_active":"2024-09-19T08:45:28.808754+0000","last_peered":"2024-09-19T08:45:28.808754+0000","last_clean":"2024-09-19T08:45:28.808754+0000","last_became_active":"2024-09-19T08:45:27.810015+0000","last_became_peered":"2024-09-19T08:45:27.810015+0000","last_unstale":"2024-09-19T08:45:28.808754+0000","last_undegraded":"2024-09-19T08:45:28.808754+0000","last_fullsized":"2024-09-19T08:45:28.808754+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:54:32.805475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.807316+0000","last_change":"2024-09-19T08:45:28.793419+0000","last_active":"2024-09-19T08:45:28.807316+0000","last_peered":"2024-09-19T08:45:28.807316+0000","last_clean":"2024-09-19T08:45:28.807316+0000","last_became_active":"2024-09-19T08:45:28.792776+0000","last_became_peered":"2024-09-19T08:45:28.792776+0000","last_unstale":"2024-09-19T08:45:28.807316+0000","last_undegraded":"2024-09-19T08:45:28.807316+0000","last_fullsized":"2024-09-19T08:45:28.807316+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:40:12.922896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"134'2","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.812745+0000","last_change":"2024-09-19T08:45:27.817247+0000","last_active":"2024-09-19T08:45:28.812745+0000","last_peered":"2024-09-19T08:45:28.812745+0000","last_clean":"2024-09-19T08:45:28.812745+0000","last_became_active":"2024-09-19T08:45:27.816821+0000","last_became_peered":"2024-09-19T08:45:27.816821+0000","last_unstale":"2024-09-19T08:45:28.812745+0000","last_undegraded":"2024-09-19T08:45:28.812745+0000","last_fullsized":"2024-09-19T08:45:28.812745+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:59.825105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.813385+0000","last_change":"2024-09-19T08:45:28.804407+0000","last_active":"2024-09-19T08:45:28.813385+0000","last_peered":"2024-09-19T08:45:28.813385+0000","last_clean":"2024-09-19T08:45:28.813385+0000","last_became_active":"2024-09-19T08:45:28.802045+0000","last_became_peered":"2024-09-19T08:45:28.802045+0000","last_unstale":"2024-09-19T08:45:28.813385+0000","last_undegraded":"2024-09-19T08:45:28.813385+0000","last_fullsized":"2024-09-19T08:45:28.813385+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:54:57.800074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"134'1","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.825052+0000","last_change":"2024-09-19T08:45:27.810213+0000","last_active":"2024-09-19T08:45:28.825052+0000","last_peered":"2024-09-19T08:45:28.825052+0000","last_clean":"2024-09-19T08:45:28.825052+0000","last_became_active":"2024-09-19T08:45:27.809868+0000","last_became_peered":"2024-09-19T08:45:27.809868+0000","last_unstale":"2024-09-19T08:45:28.825052+0000","last_undegraded":"2024-09-19T08:45:28.825052+0000","last_fullsized":"2024-09-19T08:45:28.825052+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:36:39.435937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.818342+0000","last_change":"2024-09-19T08:45:28.816908+0000","last_active":"2024-09-19T08:45:28.818342+0000","last_peered":"2024-09-19T08:45:28.818342+0000","last_clean":"2024-09-19T08:45:28.818342+0000","last_became_active":"2024-09-19T08:45:28.816657+0000","last_became_peered":"2024-09-19T08:45:28.816657+0000","last_unstale":"2024-09-19T08:45:28.818342+0000","last_undegraded":"2024-09-19T08:45:28.818342+0000","last_fullsized":"2024-09-19T08:45:28.818342+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:47:58.235785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.778770+0000","last_change":"2024-09-19T08:45:27.792135+0000","last_active":"2024-09-19T08:45:28.778770+0000","last_peered":"2024-09-19T08:45:28.778770+0000","last_clean":"2024-09-19T08:45:28.778770+0000","last_became_active":"2024-09-19T08:45:27.791716+0000","last_became_peered":"2024-09-19T08:45:27.791716+0000","last_unstale":"2024-09-19T08:45:28.778770+0000","last_undegraded":"2024-09-19T08:45:28.778770+0000","last_fullsized":"2024-09-19T08:45:28.778770+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:23.014741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.823178+0000","last_change":"2024-09-19T08:45:28.820924+0000","last_active":"2024-09-19T08:45:28.823178+0000","last_peered":"2024-09-19T08:45:28.823178+0000","last_clean":"2024-09-19T08:45:28.823178+0000","last_became_active":"2024-09-19T08:45:28.820627+0000","last_became_peered":"2024-09-19T08:45:28.820627+0000","last_unstale":"2024-09-19T08:45:28.823178+0000","last_undegraded":"2024-09-19T08:45:28.823178+0000","last_fullsized":"2024-09-19T08:45:28.823178+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:05:04.705215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"134'2","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.812665+0000","last_change":"2024-09-19T08:45:27.816594+0000","last_active":"2024-09-19T08:45:28.812665+0000","last_peered":"2024-09-19T08:45:28.812665+0000","last_clean":"2024-09-19T08:45:28.812665+0000","last_became_active":"2024-09-19T08:45:27.816199+0000","last_became_peered":"2024-09-19T08:45:27.816199+0000","last_unstale":"2024-09-19T08:45:28.812665+0000","last_undegraded":"2024-09-19T08:45:28.812665+0000","last_fullsized":"2024-09-19T08:45:28.812665+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:33:30.906009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.813319+0000","last_change":"2024-09-19T08:45:28.809326+0000","last_active":"2024-09-19T08:45:28.813319+0000","last_peered":"2024-09-19T08:45:28.813319+0000","last_clean":"2024-09-19T08:45:28.813319+0000","last_became_active":"2024-09-19T08:45:28.806005+0000","last_became_peered":"2024-09-19T08:45:28.806005+0000","last_unstale":"2024-09-19T08:45:28.813319+0000","last_undegraded":"2024-09-19T08:45:28.813319+0000","last_fullsized":"2024-09-19T08:45:28.813319+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:32:04.302543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.779880+0000","last_change":"2024-09-19T08:45:27.813063+0000","last_active":"2024-09-19T08:45:28.779880+0000","last_peered":"2024-09-19T08:45:28.779880+0000","last_clean":"2024-09-19T08:45:28.779880+0000","last_became_active":"2024-09-19T08:45:27.812277+0000","last_became_peered":"2024-09-19T08:45:27.812277+0000","last_unstale":"2024-09-19T08:45:28.779880+0000","last_undegraded":"2024-09-19T08:45:28.779880+0000","last_fullsized":"2024-09-19T08:45:28.779880+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:15:43.220533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.804382+0000","last_change":"2024-09-19T08:45:28.800831+0000","last_active":"2024-09-19T08:45:28.804382+0000","last_peered":"2024-09-19T08:45:28.804382+0000","last_clean":"2024-09-19T08:45:28.804382+0000","last_became_active":"2024-09-19T08:45:28.800302+0000","last_became_peered":"2024-09-19T08:45:28.800302+0000","last_unstale":"2024-09-19T08:45:28.804382+0000","last_undegraded":"2024-09-19T08:45:28.804382+0000","last_fullsized":"2024-09-19T08:45:28.804382+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:47:20.670900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.779466+0000","last_change":"2024-09-19T08:45:27.812887+0000","last_active":"2024-09-19T08:45:28.779466+0000","last_peered":"2024-09-19T08:45:28.779466+0000","last_clean":"2024-09-19T08:45:28.779466+0000","last_became_active":"2024-09-19T08:45:27.812085+0000","last_became_peered":"2024-09-19T08:45:27.812085+0000","last_unstale":"2024-09-19T08:45:28.779466+0000","last_undegraded":"2024-09-19T08:45:28.779466+0000","last_fullsized":"2024-09-19T08:45:28.779466+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:38:54.140642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.813623+0000","last_change":"2024-09-19T08:45:28.809240+0000","last_active":"2024-09-19T08:45:28.813623+0000","last_peered":"2024-09-19T08:45:28.813623+0000","last_clean":"2024-09-19T08:45:28.813623+0000","last_became_active":"2024-09-19T08:45:28.806104+0000","last_became_peered":"2024-09-19T08:45:28.806104+0000","last_unstale":"2024-09-19T08:45:28.813623+0000","last_undegraded":"2024-09-19T08:45:28.813623+0000","last_fullsized":"2024-09-19T08:45:28.813623+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:07.612401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.784876+0000","last_change":"2024-09-19T08:45:27.808244+0000","last_active":"2024-09-19T08:45:28.784876+0000","last_peered":"2024-09-19T08:45:28.784876+0000","last_clean":"2024-09-19T08:45:28.784876+0000","last_became_active":"2024-09-19T08:45:27.807341+0000","last_became_peered":"2024-09-19T08:45:27.807341+0000","last_unstale":"2024-09-19T08:45:28.784876+0000","last_undegraded":"2024-09-19T08:45:28.784876+0000","last_fullsized":"2024-09-19T08:45:28.784876+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:24.008928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.819157+0000","last_change":"2024-09-19T08:45:28.817319+0000","last_active":"2024-09-19T08:45:28.819157+0000","last_peered":"2024-09-19T08:45:28.819157+0000","last_clean":"2024-09-19T08:45:28.819157+0000","last_became_active":"2024-09-19T08:45:28.817041+0000","last_became_peered":"2024-09-19T08:45:28.817041+0000","last_unstale":"2024-09-19T08:45:28.819157+0000","last_undegraded":"2024-09-19T08:45:28.819157+0000","last_fullsized":"2024-09-19T08:45:28.819157+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:35.485683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"134'2","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.824232+0000","last_change":"2024-09-19T08:45:27.823341+0000","last_active":"2024-09-19T08:45:28.824232+0000","last_peered":"2024-09-19T08:45:28.824232+0000","last_clean":"2024-09-19T08:45:28.824232+0000","last_became_active":"2024-09-19T08:45:27.822777+0000","last_became_peered":"2024-09-19T08:45:27.822777+0000","last_unstale":"2024-09-19T08:45:28.824232+0000","last_undegraded":"2024-09-19T08:45:28.824232+0000","last_fullsized":"2024-09-19T08:45:28.824232+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:00:01.538292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.808071+0000","last_change":"2024-09-19T08:45:28.803094+0000","last_active":"2024-09-19T08:45:28.808071+0000","last_peered":"2024-09-19T08:45:28.808071+0000","last_clean":"2024-09-19T08:45:28.808071+0000","last_became_active":"2024-09-19T08:45:28.802778+0000","last_became_peered":"2024-09-19T08:45:28.802778+0000","last_unstale":"2024-09-19T08:45:28.808071+0000","last_undegraded":"2024-09-19T08:45:28.808071+0000","last_fullsized":"2024-09-19T08:45:28.808071+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:38:15.564188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.791448+0000","last_change":"2024-09-19T08:45:27.816218+0000","last_active":"2024-09-19T08:45:28.791448+0000","last_peered":"2024-09-19T08:45:28.791448+0000","last_clean":"2024-09-19T08:45:28.791448+0000","last_became_active":"2024-09-19T08:45:27.815828+0000","last_became_peered":"2024-09-19T08:45:27.815828+0000","last_unstale":"2024-09-19T08:45:28.791448+0000","last_undegraded":"2024-09-19T08:45:28.791448+0000","last_fullsized":"2024-09-19T08:45:28.791448+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:46:50.000531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.799778+0000","last_change":"2024-09-19T08:45:28.794134+0000","last_active":"2024-09-19T08:45:28.799778+0000","last_peered":"2024-09-19T08:45:28.799778+0000","last_clean":"2024-09-19T08:45:28.799778+0000","last_became_active":"2024-09-19T08:45:28.793812+0000","last_became_peered":"2024-09-19T08:45:28.793812+0000","last_unstale":"2024-09-19T08:45:28.799778+0000","last_undegraded":"2024-09-19T08:45:28.799778+0000","last_fullsized":"2024-09-19T08:45:28.799778+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:06:08.019991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.799211+0000","last_change":"2024-09-19T08:45:27.790487+0000","last_active":"2024-09-19T08:45:28.799211+0000","last_peered":"2024-09-19T08:45:28.799211+0000","last_clean":"2024-09-19T08:45:28.799211+0000","last_became_active":"2024-09-19T08:45:27.790041+0000","last_became_peered":"2024-09-19T08:45:27.790041+0000","last_unstale":"2024-09-19T08:45:28.799211+0000","last_undegraded":"2024-09-19T08:45:28.799211+0000","last_fullsized":"2024-09-19T08:45:28.799211+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:38:16.900241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.816060+0000","last_change":"2024-09-19T08:45:28.805958+0000","last_active":"2024-09-19T08:45:28.816060+0000","last_peered":"2024-09-19T08:45:28.816060+0000","last_clean":"2024-09-19T08:45:28.816060+0000","last_became_active":"2024-09-19T08:45:28.802562+0000","last_became_peered":"2024-09-19T08:45:28.802562+0000","last_unstale":"2024-09-19T08:45:28.816060+0000","last_undegraded":"2024-09-19T08:45:28.816060+0000","last_fullsized":"2024-09-19T08:45:28.816060+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:36:37.461145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.793302+0000","last_change":"2024-09-19T08:45:27.791679+0000","last_active":"2024-09-19T08:45:28.793302+0000","last_peered":"2024-09-19T08:45:28.793302+0000","last_clean":"2024-09-19T08:45:28.793302+0000","last_became_active":"2024-09-19T08:45:27.791127+0000","last_became_peered":"2024-09-19T08:45:27.791127+0000","last_unstale":"2024-09-19T08:45:28.793302+0000","last_undegraded":"2024-09-19T08:45:28.793302+0000","last_fullsized":"2024-09-19T08:45:28.793302+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:53.762737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.807508+0000","last_change":"2024-09-19T08:45:28.805693+0000","last_active":"2024-09-19T08:45:28.807508+0000","last_peered":"2024-09-19T08:45:28.807508+0000","last_clean":"2024-09-19T08:45:28.807508+0000","last_became_active":"2024-09-19T08:45:28.804944+0000","last_became_peered":"2024-09-19T08:45:28.804944+0000","last_unstale":"2024-09-19T08:45:28.807508+0000","last_undegraded":"2024-09-19T08:45:28.807508+0000","last_fullsized":"2024-09-19T08:45:28.807508+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:41.779324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.790536+0000","last_change":"2024-09-19T08:45:27.796633+0000","last_active":"2024-09-19T08:45:28.790536+0000","last_peered":"2024-09-19T08:45:28.790536+0000","last_clean":"2024-09-19T08:45:28.790536+0000","last_became_active":"2024-09-19T08:45:27.795201+0000","last_became_peered":"2024-09-19T08:45:27.795201+0000","last_unstale":"2024-09-19T08:45:28.790536+0000","last_undegraded":"2024-09-19T08:45:28.790536+0000","last_fullsized":"2024-09-19T08:45:28.790536+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:20:51.060225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.818016+0000","last_change":"2024-09-19T08:45:28.814539+0000","last_active":"2024-09-19T08:45:28.818016+0000","last_peered":"2024-09-19T08:45:28.818016+0000","last_clean":"2024-09-19T08:45:28.818016+0000","last_became_active":"2024-09-19T08:45:28.814215+0000","last_became_peered":"2024-09-19T08:45:28.814215+0000","last_unstale":"2024-09-19T08:45:28.818016+0000","last_undegraded":"2024-09-19T08:45:28.818016+0000","last_fullsized":"2024-09-19T08:45:28.818016+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:36:39.571493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.780047+0000","last_change":"2024-09-19T08:45:27.823111+0000","last_active":"2024-09-19T08:45:28.780047+0000","last_peered":"2024-09-19T08:45:28.780047+0000","last_clean":"2024-09-19T08:45:28.780047+0000","last_became_active":"2024-09-19T08:45:27.822315+0000","last_became_peered":"2024-09-19T08:45:27.822315+0000","last_unstale":"2024-09-19T08:45:28.780047+0000","last_undegraded":"2024-09-19T08:45:28.780047+0000","last_fullsized":"2024-09-19T08:45:28.780047+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:40.411815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.815777+0000","last_change":"2024-09-19T08:45:28.808737+0000","last_active":"2024-09-19T08:45:28.815777+0000","last_peered":"2024-09-19T08:45:28.815777+0000","last_clean":"2024-09-19T08:45:28.815777+0000","last_became_active":"2024-09-19T08:45:28.808439+0000","last_became_peered":"2024-09-19T08:45:28.808439+0000","last_unstale":"2024-09-19T08:45:28.815777+0000","last_undegraded":"2024-09-19T08:45:28.815777+0000","last_fullsized":"2024-09-19T08:45:28.815777+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:20:52.064260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.787053+0000","last_change":"2024-09-19T08:45:27.818609+0000","last_active":"2024-09-19T08:45:28.787053+0000","last_peered":"2024-09-19T08:45:28.787053+0000","last_clean":"2024-09-19T08:45:28.787053+0000","last_became_active":"2024-09-19T08:45:27.817870+0000","last_became_peered":"2024-09-19T08:45:27.817870+0000","last_unstale":"2024-09-19T08:45:28.787053+0000","last_undegraded":"2024-09-19T08:45:28.787053+0000","last_fullsized":"2024-09-19T08:45:28.787053+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:52.344306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.819076+0000","last_change":"2024-09-19T08:45:28.815490+0000","last_active":"2024-09-19T08:45:28.819076+0000","last_peered":"2024-09-19T08:45:28.819076+0000","last_clean":"2024-09-19T08:45:28.819076+0000","last_became_active":"2024-09-19T08:45:28.815219+0000","last_became_peered":"2024-09-19T08:45:28.815219+0000","last_unstale":"2024-09-19T08:45:28.819076+0000","last_undegraded":"2024-09-19T08:45:28.819076+0000","last_fullsized":"2024-09-19T08:45:28.819076+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:13.992317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.778005+0000","last_change":"2024-09-19T08:45:27.808519+0000","last_active":"2024-09-19T08:45:28.778005+0000","last_peered":"2024-09-19T08:45:28.778005+0000","last_clean":"2024-09-19T08:45:28.778005+0000","last_became_active":"2024-09-19T08:45:27.808121+0000","last_became_peered":"2024-09-19T08:45:27.808121+0000","last_unstale":"2024-09-19T08:45:28.778005+0000","last_undegraded":"2024-09-19T08:45:28.778005+0000","last_fullsized":"2024-09-19T08:45:28.778005+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:42:26.773772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.813800+0000","last_change":"2024-09-19T08:45:28.809942+0000","last_active":"2024-09-19T08:45:28.813800+0000","last_peered":"2024-09-19T08:45:28.813800+0000","last_clean":"2024-09-19T08:45:28.813800+0000","last_became_active":"2024-09-19T08:45:28.806616+0000","last_became_peered":"2024-09-19T08:45:28.806616+0000","last_unstale":"2024-09-19T08:45:28.813800+0000","last_undegraded":"2024-09-19T08:45:28.813800+0000","last_fullsized":"2024-09-19T08:45:28.813800+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:45.220093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"134'3","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.813764+0000","last_change":"2024-09-19T08:45:27.808111+0000","last_active":"2024-09-19T08:45:28.813764+0000","last_peered":"2024-09-19T08:45:28.813764+0000","last_clean":"2024-09-19T08:45:28.813764+0000","last_became_active":"2024-09-19T08:45:27.807232+0000","last_became_peered":"2024-09-19T08:45:27.807232+0000","last_unstale":"2024-09-19T08:45:28.813764+0000","last_undegraded":"2024-09-19T08:45:28.813764+0000","last_fullsized":"2024-09-19T08:45:28.813764+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:40.476763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"71'128","reported_seq":413,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.780363+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:45:28.780363+0000","last_peered":"2024-09-19T08:45:28.780363+0000","last_clean":"2024-09-19T08:45:28.780363+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:45:28.780363+0000","last_undegraded":"2024-09-19T08:45:28.780363+0000","last_fullsized":"2024-09-19T08:45:28.780363+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509531,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33392,"kb_used_data":2200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338448,"statfs":{"total":96636764160,"available":96602570752,"internally_reserved":0,"allocated":2252800,"data_stored":1325917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32808,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339032,"statfs":{"total":96636764160,"available":96603168768,"internally_reserved":0,"allocated":1658880,"data_stored":735549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509533,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1658880,"data_stored":735549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509533,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1658880,"data_stored":735549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542235,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1658880,"data_stored":735549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509530,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1658880,"data_stored":735549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509532,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32832,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339008,"statfs":{"total":96636764160,"available":96603144192,"internally_reserved":0,"allocated":1675264,"data_stored":742934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509530,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33392,"kb_used_data":2200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338448,"statfs":{"total":96636764160,"available":96602570752,"internally_reserved":0,"allocated":2252800,"data_stored":1325917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:45:31.825 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:45:31.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.983+0000 7fc22e01c640 1 Processor -- start 2024-09-19T08:45:31.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.983+0000 7fc22e01c640 1 -- start start 2024-09-19T08:45:31.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.983+0000 7fc22e01c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc228140d30 0x7fc22806d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:31.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.983+0000 7fc22e01c640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc22815ea10 con 0x7fc228141110 2024-09-19T08:45:31.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.983+0000 7fc22e01c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc22815ecf0 con 0x7fc2280b3a20 2024-09-19T08:45:31.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.983+0000 7fc22e01c640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fc22815e7c0 con 0x7fc228140d30 2024-09-19T08:45:31.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.983+0000 7fc226ffd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fc228141110 0x7fc2280b3310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:43994/0 (socket says 172.21.15.98:43994) 2024-09-19T08:45:31.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.983+0000 7fc226ffd640 1 -- 172.21.15.98:0/3953826203 learned_addr learned my addr 172.21.15.98:0/3953826203 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:31.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.983+0000 7fc2277fe640 1 --2- 172.21.15.98:0/3953826203 >> v2:172.21.15.98:3300/0 conn(0x7fc228140d30 0x7fc22806d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:31.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.983+0000 7fc2277fe640 1 -- 172.21.15.98:0/3953826203 >> v1:172.21.15.139:6789/0 conn(0x7fc2280b3a20 legacy=0x7fc22815b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:31.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.983+0000 7fc2277fe640 1 -- 172.21.15.98:0/3953826203 >> v1:172.21.15.98:6789/0 conn(0x7fc228141110 legacy=0x7fc2280b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:31.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.983+0000 7fc2277fe640 1 -- 172.21.15.98:0/3953826203 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22815efd0 con 0x7fc228140d30 2024-09-19T08:45:31.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.987+0000 7fc2277fe640 1 --2- 172.21.15.98:0/3953826203 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc228140d30 0x7fc22806d170 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fc21800a960 tx=0x7fc21801d830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f65c6b10818539c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:31.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.987+0000 7fc2267fc640 1 -- 172.21.15.98:0/3953826203 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2180b3070 con 0x7fc228140d30 2024-09-19T08:45:31.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.987+0000 7fc2267fc640 1 -- 172.21.15.98:0/3953826203 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2180a68a0 con 0x7fc228140d30 2024-09-19T08:45:31.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.987+0000 7fc2267fc640 1 -- 172.21.15.98:0/3953826203 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21800b040 con 0x7fc228140d30 2024-09-19T08:45:31.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.987+0000 7fc22e01c640 1 -- 172.21.15.98:0/3953826203 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc228140d30 msgr2=0x7fc22806d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:31.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.987+0000 7fc22e01c640 1 --2- 172.21.15.98:0/3953826203 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc228140d30 0x7fc22806d170 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fc21800a960 tx=0x7fc21801d830 comp rx=0 tx=0).stop 2024-09-19T08:45:31.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.987+0000 7fc22e01c640 1 -- 172.21.15.98:0/3953826203 shutdown_connections 2024-09-19T08:45:31.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.987+0000 7fc22e01c640 1 --2- 172.21.15.98:0/3953826203 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc228140d30 0x7fc22806d170 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:31.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.987+0000 7fc22e01c640 1 -- 172.21.15.98:0/3953826203 >> 172.21.15.98:0/3953826203 conn(0x7fc2280b2010 msgr2=0x7fc2280b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:31.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.987+0000 7fc22e01c640 1 -- 172.21.15.98:0/3953826203 shutdown_connections 2024-09-19T08:45:31.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.987+0000 7fc22e01c640 1 -- 172.21.15.98:0/3953826203 wait complete. 2024-09-19T08:45:31.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.987+0000 7fc22e01c640 1 Processor -- start 2024-09-19T08:45:31.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.987+0000 7fc22e01c640 1 -- start start 2024-09-19T08:45:31.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.987+0000 7fc22e01c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc2280b3a20 0x7fc228153f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:31.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.987+0000 7fc22e01c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc228140d30 0x7fc22813f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:31.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.987+0000 7fc22e01c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc228141110 0x7fc22815c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:31.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.987+0000 7fc22e01c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc22815bcd0 con 0x7fc228141110 2024-09-19T08:45:31.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.987+0000 7fc22e01c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc22815bb50 con 0x7fc228140d30 2024-09-19T08:45:31.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.987+0000 7fc22e01c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc22815be50 con 0x7fc2280b3a20 2024-09-19T08:45:31.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.987+0000 7fc227fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc228141110 0x7fc22815c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:31.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.987+0000 7fc227fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc228141110 0x7fc22815c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35554/0 (socket says 172.21.15.98:35554) 2024-09-19T08:45:31.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.987+0000 7fc227fff640 1 -- 172.21.15.98:0/2048089489 learned_addr learned my addr 172.21.15.98:0/2048089489 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:31.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.987+0000 7fc226ffd640 1 --2- 172.21.15.98:0/2048089489 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc228140d30 0x7fc22813f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:31.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.987+0000 7fc2277fe640 1 --2- 172.21.15.98:0/2048089489 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc2280b3a20 0x7fc228153f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:31.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.987+0000 7fc227fff640 1 -- 172.21.15.98:0/2048089489 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc2280b3a20 msgr2=0x7fc228153f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:31.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.987+0000 7fc227fff640 1 --2- 172.21.15.98:0/2048089489 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc2280b3a20 0x7fc228153f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:31.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.987+0000 7fc227fff640 1 -- 172.21.15.98:0/2048089489 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc228140d30 msgr2=0x7fc22813f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:31.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.987+0000 7fc227fff640 1 --2- 172.21.15.98:0/2048089489 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc228140d30 0x7fc22813f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:31.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.987+0000 7fc227fff640 1 -- 172.21.15.98:0/2048089489 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc228155130 con 0x7fc228141110 2024-09-19T08:45:31.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.987+0000 7fc227fff640 1 --2- 172.21.15.98:0/2048089489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc228141110 0x7fc22815c200 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fc22000c780 tx=0x7fc22000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:31.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.987+0000 7fc20ffff640 1 -- 172.21.15.98:0/2048089489 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc220004b70 con 0x7fc228141110 2024-09-19T08:45:31.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.987+0000 7fc20ffff640 1 -- 172.21.15.98:0/2048089489 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc220004d10 con 0x7fc228141110 2024-09-19T08:45:31.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.987+0000 7fc22e01c640 1 -- 172.21.15.98:0/2048089489 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc228155410 con 0x7fc228141110 2024-09-19T08:45:31.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.987+0000 7fc20ffff640 1 -- 172.21.15.98:0/2048089489 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22000aa90 con 0x7fc228141110 2024-09-19T08:45:31.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.987+0000 7fc22e01c640 1 -- 172.21.15.98:0/2048089489 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc228155ae0 con 0x7fc228141110 2024-09-19T08:45:31.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.991+0000 7fc22e01c640 1 -- 172.21.15.98:0/2048089489 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2281555e0 con 0x7fc228141110 2024-09-19T08:45:31.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.991+0000 7fc20ffff640 1 -- 172.21.15.98:0/2048089489 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fc22000ac30 con 0x7fc228141110 2024-09-19T08:45:31.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.991+0000 7fc20ffff640 1 --2- 172.21.15.98:0/2048089489 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc214067e60 0x7fc21406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:31.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.991+0000 7fc20ffff640 1 -- 172.21.15.98:0/2048089489 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7fc220086990 con 0x7fc228141110 2024-09-19T08:45:31.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.991+0000 7fc2277fe640 1 --2- 172.21.15.98:0/2048089489 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc214067e60 0x7fc21406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:31.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.991+0000 7fc2277fe640 1 --2- 172.21.15.98:0/2048089489 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc214067e60 0x7fc21406a320 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc2180b0000 tx=0x7fc21800c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:31.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:31.995+0000 7fc20ffff640 1 -- 172.21.15.98:0/2048089489 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc220020070 con 0x7fc228141110 2024-09-19T08:45:32.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.155+0000 7fc22e01c640 1 -- 172.21.15.98:0/2048089489 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc2280d1030 con 0x7fc214067e60 2024-09-19T08:45:32.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.175+0000 7fc20ffff640 1 -- 172.21.15.98:0/2048089489 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252420 (secure 0 0 0) 0x7fc2280d1030 con 0x7fc214067e60 2024-09-19T08:45:32.179 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:45:32.180 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:45:32.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.179+0000 7fc22e01c640 1 -- 172.21.15.98:0/2048089489 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc214067e60 msgr2=0x7fc21406a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:32.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.179+0000 7fc22e01c640 1 --2- 172.21.15.98:0/2048089489 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc214067e60 0x7fc21406a320 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc2180b0000 tx=0x7fc21800c040 comp rx=0 tx=0).stop 2024-09-19T08:45:32.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.179+0000 7fc22e01c640 1 -- 172.21.15.98:0/2048089489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc228141110 msgr2=0x7fc22815c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:32.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.179+0000 7fc22e01c640 1 --2- 172.21.15.98:0/2048089489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc228141110 0x7fc22815c200 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fc22000c780 tx=0x7fc22000cc40 comp rx=0 tx=0).stop 2024-09-19T08:45:32.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.179+0000 7fc22e01c640 1 -- 172.21.15.98:0/2048089489 shutdown_connections 2024-09-19T08:45:32.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.179+0000 7fc22e01c640 1 --2- 172.21.15.98:0/2048089489 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc214067e60 0x7fc21406a320 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:32.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.179+0000 7fc22e01c640 1 --2- 172.21.15.98:0/2048089489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc228141110 0x7fc22815c200 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:32.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.179+0000 7fc22e01c640 1 --2- 172.21.15.98:0/2048089489 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc228140d30 0x7fc22813f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:32.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.179+0000 7fc22e01c640 1 --2- 172.21.15.98:0/2048089489 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc2280b3a20 0x7fc228153f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:32.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.179+0000 7fc22e01c640 1 -- 172.21.15.98:0/2048089489 >> 172.21.15.98:0/2048089489 conn(0x7fc2280b2010 msgr2=0x7fc22815d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:32.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.179+0000 7fc22e01c640 1 -- 172.21.15.98:0/2048089489 shutdown_connections 2024-09-19T08:45:32.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.179+0000 7fc22e01c640 1 -- 172.21.15.98:0/2048089489 wait complete. 2024-09-19T08:45:32.198 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-09-19T08:45:30.922248+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":101,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263672,"kb_used_data":14136,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711048,"statfs":{"total":773094113280,"available":772824113152,"internally_reserved":0,"allocated":14475264,"data_stored":7072513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.160038"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.820463+0000","last_change":"2024-09-19T08:45:28.818532+0000","last_active":"2024-09-19T08:45:28.820463+0000","last_peered":"2024-09-19T08:45:28.820463+0000","last_clean":"2024-09-19T08:45:28.820463+0000","last_became_active":"2024-09-19T08:45:28.818086+0000","last_became_peered":"2024-09-19T08:45:28.818086+0000","last_unstale":"2024-09-19T08:45:28.820463+0000","last_undegraded":"2024-09-19T08:45:28.820463+0000","last_fullsized":"2024-09-19T08:45:28.820463+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:01.374609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.790613+0000","last_change":"2024-09-19T08:45:28.789674+0000","last_active":"2024-09-19T08:45:28.790613+0000","last_peered":"2024-09-19T08:45:28.790613+0000","last_clean":"2024-09-19T08:45:28.790613+0000","last_became_active":"2024-09-19T08:45:28.789349+0000","last_became_peered":"2024-09-19T08:45:28.789349+0000","last_unstale":"2024-09-19T08:45:28.790613+0000","last_undegraded":"2024-09-19T08:45:28.790613+0000","last_fullsized":"2024-09-19T08:45:28.790613+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:32:17.965494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.809339+0000","last_change":"2024-09-19T08:45:28.792925+0000","last_active":"2024-09-19T08:45:28.809339+0000","last_peered":"2024-09-19T08:45:28.809339+0000","last_clean":"2024-09-19T08:45:28.809339+0000","last_became_active":"2024-09-19T08:45:28.792658+0000","last_became_peered":"2024-09-19T08:45:28.792658+0000","last_unstale":"2024-09-19T08:45:28.809339+0000","last_undegraded":"2024-09-19T08:45:28.809339+0000","last_fullsized":"2024-09-19T08:45:28.809339+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:14.888029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.801609+0000","last_change":"2024-09-19T08:45:28.794262+0000","last_active":"2024-09-19T08:45:28.801609+0000","last_peered":"2024-09-19T08:45:28.801609+0000","last_clean":"2024-09-19T08:45:28.801609+0000","last_became_active":"2024-09-19T08:45:28.793953+0000","last_became_peered":"2024-09-19T08:45:28.793953+0000","last_unstale":"2024-09-19T08:45:28.801609+0000","last_undegraded":"2024-09-19T08:45:28.801609+0000","last_fullsized":"2024-09-19T08:45:28.801609+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:26.181098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.821104+0000","last_change":"2024-09-19T08:45:28.818516+0000","last_active":"2024-09-19T08:45:28.821104+0000","last_peered":"2024-09-19T08:45:28.821104+0000","last_clean":"2024-09-19T08:45:28.821104+0000","last_became_active":"2024-09-19T08:45:28.818203+0000","last_became_peered":"2024-09-19T08:45:28.818203+0000","last_unstale":"2024-09-19T08:45:28.821104+0000","last_undegraded":"2024-09-19T08:45:28.821104+0000","last_fullsized":"2024-09-19T08:45:28.821104+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:50:38.932478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.820406+0000","last_change":"2024-09-19T08:45:28.818155+0000","last_active":"2024-09-19T08:45:28.820406+0000","last_peered":"2024-09-19T08:45:28.820406+0000","last_clean":"2024-09-19T08:45:28.820406+0000","last_became_active":"2024-09-19T08:45:28.817366+0000","last_became_peered":"2024-09-19T08:45:28.817366+0000","last_unstale":"2024-09-19T08:45:28.820406+0000","last_undegraded":"2024-09-19T08:45:28.820406+0000","last_fullsized":"2024-09-19T08:45:28.820406+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:30:56.900258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.813830+0000","last_change":"2024-09-19T08:45:28.810318+0000","last_active":"2024-09-19T08:45:28.813830+0000","last_peered":"2024-09-19T08:45:28.813830+0000","last_clean":"2024-09-19T08:45:28.813830+0000","last_became_active":"2024-09-19T08:45:28.807727+0000","last_became_peered":"2024-09-19T08:45:28.807727+0000","last_unstale":"2024-09-19T08:45:28.813830+0000","last_undegraded":"2024-09-19T08:45:28.813830+0000","last_fullsized":"2024-09-19T08:45:28.813830+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:18.902222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.817864+0000","last_change":"2024-09-19T08:45:28.816065+0000","last_active":"2024-09-19T08:45:28.817864+0000","last_peered":"2024-09-19T08:45:28.817864+0000","last_clean":"2024-09-19T08:45:28.817864+0000","last_became_active":"2024-09-19T08:45:28.810847+0000","last_became_peered":"2024-09-19T08:45:28.810847+0000","last_unstale":"2024-09-19T08:45:28.817864+0000","last_undegraded":"2024-09-19T08:45:28.817864+0000","last_fullsized":"2024-09-19T08:45:28.817864+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:21:50.849379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.812893+0000","last_change":"2024-09-19T08:45:28.811675+0000","last_active":"2024-09-19T08:45:28.812893+0000","last_peered":"2024-09-19T08:45:28.812893+0000","last_clean":"2024-09-19T08:45:28.812893+0000","last_became_active":"2024-09-19T08:45:28.811341+0000","last_became_peered":"2024-09-19T08:45:28.811341+0000","last_unstale":"2024-09-19T08:45:28.812893+0000","last_undegraded":"2024-09-19T08:45:28.812893+0000","last_fullsized":"2024-09-19T08:45:28.812893+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:30:13.270651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.814971+0000","last_change":"2024-09-19T08:45:28.799996+0000","last_active":"2024-09-19T08:45:28.814971+0000","last_peered":"2024-09-19T08:45:28.814971+0000","last_clean":"2024-09-19T08:45:28.814971+0000","last_became_active":"2024-09-19T08:45:28.799716+0000","last_became_peered":"2024-09-19T08:45:28.799716+0000","last_unstale":"2024-09-19T08:45:28.814971+0000","last_undegraded":"2024-09-19T08:45:28.814971+0000","last_fullsized":"2024-09-19T08:45:28.814971+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:46:06.635440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.817672+0000","last_change":"2024-09-19T08:45:28.810317+0000","last_active":"2024-09-19T08:45:28.817672+0000","last_peered":"2024-09-19T08:45:28.817672+0000","last_clean":"2024-09-19T08:45:28.817672+0000","last_became_active":"2024-09-19T08:45:28.809726+0000","last_became_peered":"2024-09-19T08:45:28.809726+0000","last_unstale":"2024-09-19T08:45:28.817672+0000","last_undegraded":"2024-09-19T08:45:28.817672+0000","last_fullsized":"2024-09-19T08:45:28.817672+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:46.076410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.816595+0000","last_change":"2024-09-19T08:45:28.814040+0000","last_active":"2024-09-19T08:45:28.816595+0000","last_peered":"2024-09-19T08:45:28.816595+0000","last_clean":"2024-09-19T08:45:28.816595+0000","last_became_active":"2024-09-19T08:45:28.809048+0000","last_became_peered":"2024-09-19T08:45:28.809048+0000","last_unstale":"2024-09-19T08:45:28.816595+0000","last_undegraded":"2024-09-19T08:45:28.816595+0000","last_fullsized":"2024-09-19T08:45:28.816595+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:32:04.302543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.819178+0000","last_change":"2024-09-19T08:45:28.817217+0000","last_active":"2024-09-19T08:45:28.819178+0000","last_peered":"2024-09-19T08:45:28.819178+0000","last_clean":"2024-09-19T08:45:28.819178+0000","last_became_active":"2024-09-19T08:45:28.816948+0000","last_became_peered":"2024-09-19T08:45:28.816948+0000","last_unstale":"2024-09-19T08:45:28.819178+0000","last_undegraded":"2024-09-19T08:45:28.819178+0000","last_fullsized":"2024-09-19T08:45:28.819178+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:44.033478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.811462+0000","last_change":"2024-09-19T08:45:28.810385+0000","last_active":"2024-09-19T08:45:28.811462+0000","last_peered":"2024-09-19T08:45:28.811462+0000","last_clean":"2024-09-19T08:45:28.811462+0000","last_became_active":"2024-09-19T08:45:28.807669+0000","last_became_peered":"2024-09-19T08:45:28.807669+0000","last_unstale":"2024-09-19T08:45:28.811462+0000","last_undegraded":"2024-09-19T08:45:28.811462+0000","last_fullsized":"2024-09-19T08:45:28.811462+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:21:45.051375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.815205+0000","last_change":"2024-09-19T08:45:28.799989+0000","last_active":"2024-09-19T08:45:28.815205+0000","last_peered":"2024-09-19T08:45:28.815205+0000","last_clean":"2024-09-19T08:45:28.815205+0000","last_became_active":"2024-09-19T08:45:28.799254+0000","last_became_peered":"2024-09-19T08:45:28.799254+0000","last_unstale":"2024-09-19T08:45:28.815205+0000","last_undegraded":"2024-09-19T08:45:28.815205+0000","last_fullsized":"2024-09-19T08:45:28.815205+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:17.355685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.824354+0000","last_change":"2024-09-19T08:45:28.818924+0000","last_active":"2024-09-19T08:45:28.824354+0000","last_peered":"2024-09-19T08:45:28.824354+0000","last_clean":"2024-09-19T08:45:28.824354+0000","last_became_active":"2024-09-19T08:45:28.817375+0000","last_became_peered":"2024-09-19T08:45:28.817375+0000","last_unstale":"2024-09-19T08:45:28.824354+0000","last_undegraded":"2024-09-19T08:45:28.824354+0000","last_fullsized":"2024-09-19T08:45:28.824354+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:34.570425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.815330+0000","last_change":"2024-09-19T08:45:28.798749+0000","last_active":"2024-09-19T08:45:28.815330+0000","last_peered":"2024-09-19T08:45:28.815330+0000","last_clean":"2024-09-19T08:45:28.815330+0000","last_became_active":"2024-09-19T08:45:28.798192+0000","last_became_peered":"2024-09-19T08:45:28.798192+0000","last_unstale":"2024-09-19T08:45:28.815330+0000","last_undegraded":"2024-09-19T08:45:28.815330+0000","last_fullsized":"2024-09-19T08:45:28.815330+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:47:13.332636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.803992+0000","last_change":"2024-09-19T08:45:28.793006+0000","last_active":"2024-09-19T08:45:28.803992+0000","last_peered":"2024-09-19T08:45:28.803992+0000","last_clean":"2024-09-19T08:45:28.803992+0000","last_became_active":"2024-09-19T08:45:28.792594+0000","last_became_peered":"2024-09-19T08:45:28.792594+0000","last_unstale":"2024-09-19T08:45:28.803992+0000","last_undegraded":"2024-09-19T08:45:28.803992+0000","last_fullsized":"2024-09-19T08:45:28.803992+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:36.941444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.818367+0000","last_change":"2024-09-19T08:45:28.817164+0000","last_active":"2024-09-19T08:45:28.818367+0000","last_peered":"2024-09-19T08:45:28.818367+0000","last_clean":"2024-09-19T08:45:28.818367+0000","last_became_active":"2024-09-19T08:45:28.816853+0000","last_became_peered":"2024-09-19T08:45:28.816853+0000","last_unstale":"2024-09-19T08:45:28.818367+0000","last_undegraded":"2024-09-19T08:45:28.818367+0000","last_fullsized":"2024-09-19T08:45:28.818367+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:43.161108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.815058+0000","last_change":"2024-09-19T08:45:28.793134+0000","last_active":"2024-09-19T08:45:28.815058+0000","last_peered":"2024-09-19T08:45:28.815058+0000","last_clean":"2024-09-19T08:45:28.815058+0000","last_became_active":"2024-09-19T08:45:28.792715+0000","last_became_peered":"2024-09-19T08:45:28.792715+0000","last_unstale":"2024-09-19T08:45:28.815058+0000","last_undegraded":"2024-09-19T08:45:28.815058+0000","last_fullsized":"2024-09-19T08:45:28.815058+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:17.303164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.799632+0000","last_change":"2024-09-19T08:45:28.798462+0000","last_active":"2024-09-19T08:45:28.799632+0000","last_peered":"2024-09-19T08:45:28.799632+0000","last_clean":"2024-09-19T08:45:28.799632+0000","last_became_active":"2024-09-19T08:45:28.798115+0000","last_became_peered":"2024-09-19T08:45:28.798115+0000","last_unstale":"2024-09-19T08:45:28.799632+0000","last_undegraded":"2024-09-19T08:45:28.799632+0000","last_fullsized":"2024-09-19T08:45:28.799632+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:02:33.019482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.804069+0000","last_change":"2024-09-19T08:45:28.797340+0000","last_active":"2024-09-19T08:45:28.804069+0000","last_peered":"2024-09-19T08:45:28.804069+0000","last_clean":"2024-09-19T08:45:28.804069+0000","last_became_active":"2024-09-19T08:45:28.796312+0000","last_became_peered":"2024-09-19T08:45:28.796312+0000","last_unstale":"2024-09-19T08:45:28.804069+0000","last_undegraded":"2024-09-19T08:45:28.804069+0000","last_fullsized":"2024-09-19T08:45:28.804069+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:34:59.424826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.811737+0000","last_change":"2024-09-19T08:45:28.808317+0000","last_active":"2024-09-19T08:45:28.811737+0000","last_peered":"2024-09-19T08:45:28.811737+0000","last_clean":"2024-09-19T08:45:28.811737+0000","last_became_active":"2024-09-19T08:45:28.808001+0000","last_became_peered":"2024-09-19T08:45:28.808001+0000","last_unstale":"2024-09-19T08:45:28.811737+0000","last_undegraded":"2024-09-19T08:45:28.811737+0000","last_fullsized":"2024-09-19T08:45:28.811737+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:57:28.874099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.799571+0000","last_change":"2024-09-19T08:45:28.788955+0000","last_active":"2024-09-19T08:45:28.799571+0000","last_peered":"2024-09-19T08:45:28.799571+0000","last_clean":"2024-09-19T08:45:28.799571+0000","last_became_active":"2024-09-19T08:45:28.788634+0000","last_became_peered":"2024-09-19T08:45:28.788634+0000","last_unstale":"2024-09-19T08:45:28.799571+0000","last_undegraded":"2024-09-19T08:45:28.799571+0000","last_fullsized":"2024-09-19T08:45:28.799571+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:46.650745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.824485+0000","last_change":"2024-09-19T08:45:28.819371+0000","last_active":"2024-09-19T08:45:28.824485+0000","last_peered":"2024-09-19T08:45:28.824485+0000","last_clean":"2024-09-19T08:45:28.824485+0000","last_became_active":"2024-09-19T08:45:28.819089+0000","last_became_peered":"2024-09-19T08:45:28.819089+0000","last_unstale":"2024-09-19T08:45:28.824485+0000","last_undegraded":"2024-09-19T08:45:28.824485+0000","last_fullsized":"2024-09-19T08:45:28.824485+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:19:22.434059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.810802+0000","last_change":"2024-09-19T08:45:28.806385+0000","last_active":"2024-09-19T08:45:28.810802+0000","last_peered":"2024-09-19T08:45:28.810802+0000","last_clean":"2024-09-19T08:45:28.810802+0000","last_became_active":"2024-09-19T08:45:28.802540+0000","last_became_peered":"2024-09-19T08:45:28.802540+0000","last_unstale":"2024-09-19T08:45:28.810802+0000","last_undegraded":"2024-09-19T08:45:28.810802+0000","last_fullsized":"2024-09-19T08:45:28.810802+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:56.173971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.824256+0000","last_change":"2024-09-19T08:45:28.817893+0000","last_active":"2024-09-19T08:45:28.824256+0000","last_peered":"2024-09-19T08:45:28.824256+0000","last_clean":"2024-09-19T08:45:28.824256+0000","last_became_active":"2024-09-19T08:45:28.817647+0000","last_became_peered":"2024-09-19T08:45:28.817647+0000","last_unstale":"2024-09-19T08:45:28.824256+0000","last_undegraded":"2024-09-19T08:45:28.824256+0000","last_fullsized":"2024-09-19T08:45:28.824256+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:55:06.028325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.814059+0000","last_change":"2024-09-19T08:45:28.812274+0000","last_active":"2024-09-19T08:45:28.814059+0000","last_peered":"2024-09-19T08:45:28.814059+0000","last_clean":"2024-09-19T08:45:28.814059+0000","last_became_active":"2024-09-19T08:45:28.811997+0000","last_became_peered":"2024-09-19T08:45:28.811997+0000","last_unstale":"2024-09-19T08:45:28.814059+0000","last_undegraded":"2024-09-19T08:45:28.814059+0000","last_fullsized":"2024-09-19T08:45:28.814059+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:35:47.868929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.810341+0000","last_change":"2024-09-19T08:45:28.802035+0000","last_active":"2024-09-19T08:45:28.810341+0000","last_peered":"2024-09-19T08:45:28.810341+0000","last_clean":"2024-09-19T08:45:28.810341+0000","last_became_active":"2024-09-19T08:45:28.801460+0000","last_became_peered":"2024-09-19T08:45:28.801460+0000","last_unstale":"2024-09-19T08:45:28.810341+0000","last_undegraded":"2024-09-19T08:45:28.810341+0000","last_fullsized":"2024-09-19T08:45:28.810341+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:03.685888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.823199+0000","last_change":"2024-09-19T08:45:28.821114+0000","last_active":"2024-09-19T08:45:28.823199+0000","last_peered":"2024-09-19T08:45:28.823199+0000","last_clean":"2024-09-19T08:45:28.823199+0000","last_became_active":"2024-09-19T08:45:28.820712+0000","last_became_peered":"2024-09-19T08:45:28.820712+0000","last_unstale":"2024-09-19T08:45:28.823199+0000","last_undegraded":"2024-09-19T08:45:28.823199+0000","last_fullsized":"2024-09-19T08:45:28.823199+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:04.999051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.799164+0000","last_change":"2024-09-19T08:45:28.791517+0000","last_active":"2024-09-19T08:45:28.799164+0000","last_peered":"2024-09-19T08:45:28.799164+0000","last_clean":"2024-09-19T08:45:28.799164+0000","last_became_active":"2024-09-19T08:45:28.791247+0000","last_became_peered":"2024-09-19T08:45:28.791247+0000","last_unstale":"2024-09-19T08:45:28.799164+0000","last_undegraded":"2024-09-19T08:45:28.799164+0000","last_fullsized":"2024-09-19T08:45:28.799164+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:26:17.904918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.824318+0000","last_change":"2024-09-19T08:45:28.818666+0000","last_active":"2024-09-19T08:45:28.824318+0000","last_peered":"2024-09-19T08:45:28.824318+0000","last_clean":"2024-09-19T08:45:28.824318+0000","last_became_active":"2024-09-19T08:45:28.813155+0000","last_became_peered":"2024-09-19T08:45:28.813155+0000","last_unstale":"2024-09-19T08:45:28.824318+0000","last_undegraded":"2024-09-19T08:45:28.824318+0000","last_fullsized":"2024-09-19T08:45:28.824318+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:37:10.149152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.816656+0000","last_change":"2024-09-19T08:45:28.814034+0000","last_active":"2024-09-19T08:45:28.816656+0000","last_peered":"2024-09-19T08:45:28.816656+0000","last_clean":"2024-09-19T08:45:28.816656+0000","last_became_active":"2024-09-19T08:45:28.813724+0000","last_became_peered":"2024-09-19T08:45:28.813724+0000","last_unstale":"2024-09-19T08:45:28.816656+0000","last_undegraded":"2024-09-19T08:45:28.816656+0000","last_fullsized":"2024-09-19T08:45:28.816656+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:07.612401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"134'4","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.826792+0000","last_change":"2024-09-19T08:45:27.820921+0000","last_active":"2024-09-19T08:45:28.826792+0000","last_peered":"2024-09-19T08:45:28.826792+0000","last_clean":"2024-09-19T08:45:28.826792+0000","last_became_active":"2024-09-19T08:45:27.820200+0000","last_became_peered":"2024-09-19T08:45:27.820200+0000","last_unstale":"2024-09-19T08:45:28.826792+0000","last_undegraded":"2024-09-19T08:45:28.826792+0000","last_fullsized":"2024-09-19T08:45:28.826792+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:41:22.555962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.802488+0000","last_change":"2024-09-19T08:45:28.798810+0000","last_active":"2024-09-19T08:45:28.802488+0000","last_peered":"2024-09-19T08:45:28.802488+0000","last_clean":"2024-09-19T08:45:28.802488+0000","last_became_active":"2024-09-19T08:45:28.795552+0000","last_became_peered":"2024-09-19T08:45:28.795552+0000","last_unstale":"2024-09-19T08:45:28.802488+0000","last_undegraded":"2024-09-19T08:45:28.802488+0000","last_fullsized":"2024-09-19T08:45:28.802488+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:18.681030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"134'1","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.817189+0000","last_change":"2024-09-19T08:45:27.816108+0000","last_active":"2024-09-19T08:45:28.817189+0000","last_peered":"2024-09-19T08:45:28.817189+0000","last_clean":"2024-09-19T08:45:28.817189+0000","last_became_active":"2024-09-19T08:45:27.815701+0000","last_became_peered":"2024-09-19T08:45:27.815701+0000","last_unstale":"2024-09-19T08:45:28.817189+0000","last_undegraded":"2024-09-19T08:45:28.817189+0000","last_fullsized":"2024-09-19T08:45:28.817189+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:24:45.554977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.817855+0000","last_change":"2024-09-19T08:45:28.810608+0000","last_active":"2024-09-19T08:45:28.817855+0000","last_peered":"2024-09-19T08:45:28.817855+0000","last_clean":"2024-09-19T08:45:28.817855+0000","last_became_active":"2024-09-19T08:45:28.809370+0000","last_became_peered":"2024-09-19T08:45:28.809370+0000","last_unstale":"2024-09-19T08:45:28.817855+0000","last_undegraded":"2024-09-19T08:45:28.817855+0000","last_fullsized":"2024-09-19T08:45:28.817855+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:06:44.272919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"134'2","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.815417+0000","last_change":"2024-09-19T08:45:27.797683+0000","last_active":"2024-09-19T08:45:28.815417+0000","last_peered":"2024-09-19T08:45:28.815417+0000","last_clean":"2024-09-19T08:45:28.815417+0000","last_became_active":"2024-09-19T08:45:27.796852+0000","last_became_peered":"2024-09-19T08:45:27.796852+0000","last_unstale":"2024-09-19T08:45:28.815417+0000","last_undegraded":"2024-09-19T08:45:28.815417+0000","last_fullsized":"2024-09-19T08:45:28.815417+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:31:45.059438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.815736+0000","last_change":"2024-09-19T08:45:28.807910+0000","last_active":"2024-09-19T08:45:28.815736+0000","last_peered":"2024-09-19T08:45:28.815736+0000","last_clean":"2024-09-19T08:45:28.815736+0000","last_became_active":"2024-09-19T08:45:28.807605+0000","last_became_peered":"2024-09-19T08:45:28.807605+0000","last_unstale":"2024-09-19T08:45:28.815736+0000","last_undegraded":"2024-09-19T08:45:28.815736+0000","last_fullsized":"2024-09-19T08:45:28.815736+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:33:31.910044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"134'2","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.820526+0000","last_change":"2024-09-19T08:45:27.786505+0000","last_active":"2024-09-19T08:45:28.820526+0000","last_peered":"2024-09-19T08:45:28.820526+0000","last_clean":"2024-09-19T08:45:28.820526+0000","last_became_active":"2024-09-19T08:45:27.785674+0000","last_became_peered":"2024-09-19T08:45:27.785674+0000","last_unstale":"2024-09-19T08:45:28.820526+0000","last_undegraded":"2024-09-19T08:45:28.820526+0000","last_fullsized":"2024-09-19T08:45:28.820526+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:10:17.980862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.799645+0000","last_change":"2024-09-19T08:45:28.791665+0000","last_active":"2024-09-19T08:45:28.799645+0000","last_peered":"2024-09-19T08:45:28.799645+0000","last_clean":"2024-09-19T08:45:28.799645+0000","last_became_active":"2024-09-19T08:45:28.791349+0000","last_became_peered":"2024-09-19T08:45:28.791349+0000","last_unstale":"2024-09-19T08:45:28.799645+0000","last_undegraded":"2024-09-19T08:45:28.799645+0000","last_fullsized":"2024-09-19T08:45:28.799645+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:21:09.499251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"134'1","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.824328+0000","last_change":"2024-09-19T08:45:27.809526+0000","last_active":"2024-09-19T08:45:28.824328+0000","last_peered":"2024-09-19T08:45:28.824328+0000","last_clean":"2024-09-19T08:45:28.824328+0000","last_became_active":"2024-09-19T08:45:27.809171+0000","last_became_peered":"2024-09-19T08:45:27.809171+0000","last_unstale":"2024-09-19T08:45:28.824328+0000","last_undegraded":"2024-09-19T08:45:28.824328+0000","last_fullsized":"2024-09-19T08:45:28.824328+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:00:04.373962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.801677+0000","last_change":"2024-09-19T08:45:28.794324+0000","last_active":"2024-09-19T08:45:28.801677+0000","last_peered":"2024-09-19T08:45:28.801677+0000","last_clean":"2024-09-19T08:45:28.801677+0000","last_became_active":"2024-09-19T08:45:28.793947+0000","last_became_peered":"2024-09-19T08:45:28.793947+0000","last_unstale":"2024-09-19T08:45:28.801677+0000","last_undegraded":"2024-09-19T08:45:28.801677+0000","last_fullsized":"2024-09-19T08:45:28.801677+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:37.913412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"134'1","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.818010+0000","last_change":"2024-09-19T08:45:27.815404+0000","last_active":"2024-09-19T08:45:28.818010+0000","last_peered":"2024-09-19T08:45:28.818010+0000","last_clean":"2024-09-19T08:45:28.818010+0000","last_became_active":"2024-09-19T08:45:27.814735+0000","last_became_peered":"2024-09-19T08:45:27.814735+0000","last_unstale":"2024-09-19T08:45:28.818010+0000","last_undegraded":"2024-09-19T08:45:28.818010+0000","last_fullsized":"2024-09-19T08:45:28.818010+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:26:29.893419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.802919+0000","last_change":"2024-09-19T08:45:28.799288+0000","last_active":"2024-09-19T08:45:28.802919+0000","last_peered":"2024-09-19T08:45:28.802919+0000","last_clean":"2024-09-19T08:45:28.802919+0000","last_became_active":"2024-09-19T08:45:28.795468+0000","last_became_peered":"2024-09-19T08:45:28.795468+0000","last_unstale":"2024-09-19T08:45:28.802919+0000","last_undegraded":"2024-09-19T08:45:28.802919+0000","last_fullsized":"2024-09-19T08:45:28.802919+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:09.023436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.794533+0000","last_change":"2024-09-19T08:45:27.797807+0000","last_active":"2024-09-19T08:45:28.794533+0000","last_peered":"2024-09-19T08:45:28.794533+0000","last_clean":"2024-09-19T08:45:28.794533+0000","last_became_active":"2024-09-19T08:45:27.797423+0000","last_became_peered":"2024-09-19T08:45:27.797423+0000","last_unstale":"2024-09-19T08:45:28.794533+0000","last_undegraded":"2024-09-19T08:45:28.794533+0000","last_fullsized":"2024-09-19T08:45:28.794533+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:45:09.728422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.812066+0000","last_change":"2024-09-19T08:45:28.811558+0000","last_active":"2024-09-19T08:45:28.812066+0000","last_peered":"2024-09-19T08:45:28.812066+0000","last_clean":"2024-09-19T08:45:28.812066+0000","last_became_active":"2024-09-19T08:45:28.811313+0000","last_became_peered":"2024-09-19T08:45:28.811313+0000","last_unstale":"2024-09-19T08:45:28.812066+0000","last_undegraded":"2024-09-19T08:45:28.812066+0000","last_fullsized":"2024-09-19T08:45:28.812066+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:30:14.734675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.785492+0000","last_change":"2024-09-19T08:45:27.811525+0000","last_active":"2024-09-19T08:45:28.785492+0000","last_peered":"2024-09-19T08:45:28.785492+0000","last_clean":"2024-09-19T08:45:28.785492+0000","last_became_active":"2024-09-19T08:45:27.810635+0000","last_became_peered":"2024-09-19T08:45:27.810635+0000","last_unstale":"2024-09-19T08:45:28.785492+0000","last_undegraded":"2024-09-19T08:45:28.785492+0000","last_fullsized":"2024-09-19T08:45:28.785492+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:10.320962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.810560+0000","last_change":"2024-09-19T08:45:28.810166+0000","last_active":"2024-09-19T08:45:28.810560+0000","last_peered":"2024-09-19T08:45:28.810560+0000","last_clean":"2024-09-19T08:45:28.810560+0000","last_became_active":"2024-09-19T08:45:28.809816+0000","last_became_peered":"2024-09-19T08:45:28.809816+0000","last_unstale":"2024-09-19T08:45:28.810560+0000","last_undegraded":"2024-09-19T08:45:28.810560+0000","last_fullsized":"2024-09-19T08:45:28.810560+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:44:16.379183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.780184+0000","last_change":"2024-09-19T08:45:27.793871+0000","last_active":"2024-09-19T08:45:28.780184+0000","last_peered":"2024-09-19T08:45:28.780184+0000","last_clean":"2024-09-19T08:45:28.780184+0000","last_became_active":"2024-09-19T08:45:27.792973+0000","last_became_peered":"2024-09-19T08:45:27.792973+0000","last_unstale":"2024-09-19T08:45:28.780184+0000","last_undegraded":"2024-09-19T08:45:28.780184+0000","last_fullsized":"2024-09-19T08:45:28.780184+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:04:48.478624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.815379+0000","last_change":"2024-09-19T08:45:28.798990+0000","last_active":"2024-09-19T08:45:28.815379+0000","last_peered":"2024-09-19T08:45:28.815379+0000","last_clean":"2024-09-19T08:45:28.815379+0000","last_became_active":"2024-09-19T08:45:28.798553+0000","last_became_peered":"2024-09-19T08:45:28.798553+0000","last_unstale":"2024-09-19T08:45:28.815379+0000","last_undegraded":"2024-09-19T08:45:28.815379+0000","last_fullsized":"2024-09-19T08:45:28.815379+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:15.792372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.781720+0000","last_change":"2024-09-19T08:45:27.823816+0000","last_active":"2024-09-19T08:45:28.781720+0000","last_peered":"2024-09-19T08:45:28.781720+0000","last_clean":"2024-09-19T08:45:28.781720+0000","last_became_active":"2024-09-19T08:45:27.822250+0000","last_became_peered":"2024-09-19T08:45:27.822250+0000","last_unstale":"2024-09-19T08:45:28.781720+0000","last_undegraded":"2024-09-19T08:45:28.781720+0000","last_fullsized":"2024-09-19T08:45:28.781720+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:34:56.926103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.821202+0000","last_change":"2024-09-19T08:45:28.814480+0000","last_active":"2024-09-19T08:45:28.821202+0000","last_peered":"2024-09-19T08:45:28.821202+0000","last_clean":"2024-09-19T08:45:28.821202+0000","last_became_active":"2024-09-19T08:45:28.807443+0000","last_became_peered":"2024-09-19T08:45:28.807443+0000","last_unstale":"2024-09-19T08:45:28.821202+0000","last_undegraded":"2024-09-19T08:45:28.821202+0000","last_fullsized":"2024-09-19T08:45:28.821202+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:19:10.453003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"134'6","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.826951+0000","last_change":"2024-09-19T08:45:27.791751+0000","last_active":"2024-09-19T08:45:28.826951+0000","last_peered":"2024-09-19T08:45:28.826951+0000","last_clean":"2024-09-19T08:45:28.826951+0000","last_became_active":"2024-09-19T08:45:27.791474+0000","last_became_peered":"2024-09-19T08:45:27.791474+0000","last_unstale":"2024-09-19T08:45:28.826951+0000","last_undegraded":"2024-09-19T08:45:28.826951+0000","last_fullsized":"2024-09-19T08:45:28.826951+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:16.351650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.808697+0000","last_change":"2024-09-19T08:45:28.803503+0000","last_active":"2024-09-19T08:45:28.808697+0000","last_peered":"2024-09-19T08:45:28.808697+0000","last_clean":"2024-09-19T08:45:28.808697+0000","last_became_active":"2024-09-19T08:45:28.802949+0000","last_became_peered":"2024-09-19T08:45:28.802949+0000","last_unstale":"2024-09-19T08:45:28.808697+0000","last_undegraded":"2024-09-19T08:45:28.808697+0000","last_fullsized":"2024-09-19T08:45:28.808697+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:24:43.012611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"134'2","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.816231+0000","last_change":"2024-09-19T08:45:27.817233+0000","last_active":"2024-09-19T08:45:28.816231+0000","last_peered":"2024-09-19T08:45:28.816231+0000","last_clean":"2024-09-19T08:45:28.816231+0000","last_became_active":"2024-09-19T08:45:27.809278+0000","last_became_peered":"2024-09-19T08:45:27.809278+0000","last_unstale":"2024-09-19T08:45:28.816231+0000","last_undegraded":"2024-09-19T08:45:28.816231+0000","last_fullsized":"2024-09-19T08:45:28.816231+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:18:43.220541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.789861+0000","last_change":"2024-09-19T08:45:28.788398+0000","last_active":"2024-09-19T08:45:28.789861+0000","last_peered":"2024-09-19T08:45:28.789861+0000","last_clean":"2024-09-19T08:45:28.789861+0000","last_became_active":"2024-09-19T08:45:28.788104+0000","last_became_peered":"2024-09-19T08:45:28.788104+0000","last_unstale":"2024-09-19T08:45:28.789861+0000","last_undegraded":"2024-09-19T08:45:28.789861+0000","last_fullsized":"2024-09-19T08:45:28.789861+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:51:54.858323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.788696+0000","last_change":"2024-09-19T08:45:27.809112+0000","last_active":"2024-09-19T08:45:28.788696+0000","last_peered":"2024-09-19T08:45:28.788696+0000","last_clean":"2024-09-19T08:45:28.788696+0000","last_became_active":"2024-09-19T08:45:27.807985+0000","last_became_peered":"2024-09-19T08:45:27.807985+0000","last_unstale":"2024-09-19T08:45:28.788696+0000","last_undegraded":"2024-09-19T08:45:28.788696+0000","last_fullsized":"2024-09-19T08:45:28.788696+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:33:29.140035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.821122+0000","last_change":"2024-09-19T08:45:28.817985+0000","last_active":"2024-09-19T08:45:28.821122+0000","last_peered":"2024-09-19T08:45:28.821122+0000","last_clean":"2024-09-19T08:45:28.821122+0000","last_became_active":"2024-09-19T08:45:28.817596+0000","last_became_peered":"2024-09-19T08:45:28.817596+0000","last_unstale":"2024-09-19T08:45:28.821122+0000","last_undegraded":"2024-09-19T08:45:28.821122+0000","last_fullsized":"2024-09-19T08:45:28.821122+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:48.791756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.784703+0000","last_change":"2024-09-19T08:45:27.793154+0000","last_active":"2024-09-19T08:45:28.784703+0000","last_peered":"2024-09-19T08:45:28.784703+0000","last_clean":"2024-09-19T08:45:28.784703+0000","last_became_active":"2024-09-19T08:45:27.792439+0000","last_became_peered":"2024-09-19T08:45:27.792439+0000","last_unstale":"2024-09-19T08:45:28.784703+0000","last_undegraded":"2024-09-19T08:45:28.784703+0000","last_fullsized":"2024-09-19T08:45:28.784703+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:34:02.879548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.823661+0000","last_change":"2024-09-19T08:45:28.817504+0000","last_active":"2024-09-19T08:45:28.823661+0000","last_peered":"2024-09-19T08:45:28.823661+0000","last_clean":"2024-09-19T08:45:28.823661+0000","last_became_active":"2024-09-19T08:45:28.817214+0000","last_became_peered":"2024-09-19T08:45:28.817214+0000","last_unstale":"2024-09-19T08:45:28.823661+0000","last_undegraded":"2024-09-19T08:45:28.823661+0000","last_fullsized":"2024-09-19T08:45:28.823661+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:55:35.661617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"134'8","reported_seq":20,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.825331+0000","last_change":"2024-09-19T08:45:27.808101+0000","last_active":"2024-09-19T08:45:28.825331+0000","last_peered":"2024-09-19T08:45:28.825331+0000","last_clean":"2024-09-19T08:45:28.825331+0000","last_became_active":"2024-09-19T08:45:27.807651+0000","last_became_peered":"2024-09-19T08:45:27.807651+0000","last_unstale":"2024-09-19T08:45:28.825331+0000","last_undegraded":"2024-09-19T08:45:28.825331+0000","last_fullsized":"2024-09-19T08:45:28.825331+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:28:59.019811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.823344+0000","last_change":"2024-09-19T08:45:28.819904+0000","last_active":"2024-09-19T08:45:28.823344+0000","last_peered":"2024-09-19T08:45:28.823344+0000","last_clean":"2024-09-19T08:45:28.823344+0000","last_became_active":"2024-09-19T08:45:28.819610+0000","last_became_peered":"2024-09-19T08:45:28.819610+0000","last_unstale":"2024-09-19T08:45:28.823344+0000","last_undegraded":"2024-09-19T08:45:28.823344+0000","last_fullsized":"2024-09-19T08:45:28.823344+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:45.220093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"134'1","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.808754+0000","last_change":"2024-09-19T08:45:27.810578+0000","last_active":"2024-09-19T08:45:28.808754+0000","last_peered":"2024-09-19T08:45:28.808754+0000","last_clean":"2024-09-19T08:45:28.808754+0000","last_became_active":"2024-09-19T08:45:27.810015+0000","last_became_peered":"2024-09-19T08:45:27.810015+0000","last_unstale":"2024-09-19T08:45:28.808754+0000","last_undegraded":"2024-09-19T08:45:28.808754+0000","last_fullsized":"2024-09-19T08:45:28.808754+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:54:32.805475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.807316+0000","last_change":"2024-09-19T08:45:28.793419+0000","last_active":"2024-09-19T08:45:28.807316+0000","last_peered":"2024-09-19T08:45:28.807316+0000","last_clean":"2024-09-19T08:45:28.807316+0000","last_became_active":"2024-09-19T08:45:28.792776+0000","last_became_peered":"2024-09-19T08:45:28.792776+0000","last_unstale":"2024-09-19T08:45:28.807316+0000","last_undegraded":"2024-09-19T08:45:28.807316+0000","last_fullsized":"2024-09-19T08:45:28.807316+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:40:12.922896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"134'2","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.812745+0000","last_change":"2024-09-19T08:45:27.817247+0000","last_active":"2024-09-19T08:45:28.812745+0000","last_peered":"2024-09-19T08:45:28.812745+0000","last_clean":"2024-09-19T08:45:28.812745+0000","last_became_active":"2024-09-19T08:45:27.816821+0000","last_became_peered":"2024-09-19T08:45:27.816821+0000","last_unstale":"2024-09-19T08:45:28.812745+0000","last_undegraded":"2024-09-19T08:45:28.812745+0000","last_fullsized":"2024-09-19T08:45:28.812745+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:59.825105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.813385+0000","last_change":"2024-09-19T08:45:28.804407+0000","last_active":"2024-09-19T08:45:28.813385+0000","last_peered":"2024-09-19T08:45:28.813385+0000","last_clean":"2024-09-19T08:45:28.813385+0000","last_became_active":"2024-09-19T08:45:28.802045+0000","last_became_peered":"2024-09-19T08:45:28.802045+0000","last_unstale":"2024-09-19T08:45:28.813385+0000","last_undegraded":"2024-09-19T08:45:28.813385+0000","last_fullsized":"2024-09-19T08:45:28.813385+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:54:57.800074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"134'1","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.825052+0000","last_change":"2024-09-19T08:45:27.810213+0000","last_active":"2024-09-19T08:45:28.825052+0000","last_peered":"2024-09-19T08:45:28.825052+0000","last_clean":"2024-09-19T08:45:28.825052+0000","last_became_active":"2024-09-19T08:45:27.809868+0000","last_became_peered":"2024-09-19T08:45:27.809868+0000","last_unstale":"2024-09-19T08:45:28.825052+0000","last_undegraded":"2024-09-19T08:45:28.825052+0000","last_fullsized":"2024-09-19T08:45:28.825052+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:36:39.435937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.818342+0000","last_change":"2024-09-19T08:45:28.816908+0000","last_active":"2024-09-19T08:45:28.818342+0000","last_peered":"2024-09-19T08:45:28.818342+0000","last_clean":"2024-09-19T08:45:28.818342+0000","last_became_active":"2024-09-19T08:45:28.816657+0000","last_became_peered":"2024-09-19T08:45:28.816657+0000","last_unstale":"2024-09-19T08:45:28.818342+0000","last_undegraded":"2024-09-19T08:45:28.818342+0000","last_fullsized":"2024-09-19T08:45:28.818342+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:47:58.235785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.778770+0000","last_change":"2024-09-19T08:45:27.792135+0000","last_active":"2024-09-19T08:45:28.778770+0000","last_peered":"2024-09-19T08:45:28.778770+0000","last_clean":"2024-09-19T08:45:28.778770+0000","last_became_active":"2024-09-19T08:45:27.791716+0000","last_became_peered":"2024-09-19T08:45:27.791716+0000","last_unstale":"2024-09-19T08:45:28.778770+0000","last_undegraded":"2024-09-19T08:45:28.778770+0000","last_fullsized":"2024-09-19T08:45:28.778770+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:23.014741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.823178+0000","last_change":"2024-09-19T08:45:28.820924+0000","last_active":"2024-09-19T08:45:28.823178+0000","last_peered":"2024-09-19T08:45:28.823178+0000","last_clean":"2024-09-19T08:45:28.823178+0000","last_became_active":"2024-09-19T08:45:28.820627+0000","last_became_peered":"2024-09-19T08:45:28.820627+0000","last_unstale":"2024-09-19T08:45:28.823178+0000","last_undegraded":"2024-09-19T08:45:28.823178+0000","last_fullsized":"2024-09-19T08:45:28.823178+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:05:04.705215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"134'2","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.812665+0000","last_change":"2024-09-19T08:45:27.816594+0000","last_active":"2024-09-19T08:45:28.812665+0000","last_peered":"2024-09-19T08:45:28.812665+0000","last_clean":"2024-09-19T08:45:28.812665+0000","last_became_active":"2024-09-19T08:45:27.816199+0000","last_became_peered":"2024-09-19T08:45:27.816199+0000","last_unstale":"2024-09-19T08:45:28.812665+0000","last_undegraded":"2024-09-19T08:45:28.812665+0000","last_fullsized":"2024-09-19T08:45:28.812665+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:33:30.906009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.813319+0000","last_change":"2024-09-19T08:45:28.809326+0000","last_active":"2024-09-19T08:45:28.813319+0000","last_peered":"2024-09-19T08:45:28.813319+0000","last_clean":"2024-09-19T08:45:28.813319+0000","last_became_active":"2024-09-19T08:45:28.806005+0000","last_became_peered":"2024-09-19T08:45:28.806005+0000","last_unstale":"2024-09-19T08:45:28.813319+0000","last_undegraded":"2024-09-19T08:45:28.813319+0000","last_fullsized":"2024-09-19T08:45:28.813319+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:32:04.302543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.779880+0000","last_change":"2024-09-19T08:45:27.813063+0000","last_active":"2024-09-19T08:45:28.779880+0000","last_peered":"2024-09-19T08:45:28.779880+0000","last_clean":"2024-09-19T08:45:28.779880+0000","last_became_active":"2024-09-19T08:45:27.812277+0000","last_became_peered":"2024-09-19T08:45:27.812277+0000","last_unstale":"2024-09-19T08:45:28.779880+0000","last_undegraded":"2024-09-19T08:45:28.779880+0000","last_fullsized":"2024-09-19T08:45:28.779880+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:15:43.220533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.804382+0000","last_change":"2024-09-19T08:45:28.800831+0000","last_active":"2024-09-19T08:45:28.804382+0000","last_peered":"2024-09-19T08:45:28.804382+0000","last_clean":"2024-09-19T08:45:28.804382+0000","last_became_active":"2024-09-19T08:45:28.800302+0000","last_became_peered":"2024-09-19T08:45:28.800302+0000","last_unstale":"2024-09-19T08:45:28.804382+0000","last_undegraded":"2024-09-19T08:45:28.804382+0000","last_fullsized":"2024-09-19T08:45:28.804382+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:47:20.670900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.779466+0000","last_change":"2024-09-19T08:45:27.812887+0000","last_active":"2024-09-19T08:45:28.779466+0000","last_peered":"2024-09-19T08:45:28.779466+0000","last_clean":"2024-09-19T08:45:28.779466+0000","last_became_active":"2024-09-19T08:45:27.812085+0000","last_became_peered":"2024-09-19T08:45:27.812085+0000","last_unstale":"2024-09-19T08:45:28.779466+0000","last_undegraded":"2024-09-19T08:45:28.779466+0000","last_fullsized":"2024-09-19T08:45:28.779466+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:38:54.140642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.813623+0000","last_change":"2024-09-19T08:45:28.809240+0000","last_active":"2024-09-19T08:45:28.813623+0000","last_peered":"2024-09-19T08:45:28.813623+0000","last_clean":"2024-09-19T08:45:28.813623+0000","last_became_active":"2024-09-19T08:45:28.806104+0000","last_became_peered":"2024-09-19T08:45:28.806104+0000","last_unstale":"2024-09-19T08:45:28.813623+0000","last_undegraded":"2024-09-19T08:45:28.813623+0000","last_fullsized":"2024-09-19T08:45:28.813623+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:07.612401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.784876+0000","last_change":"2024-09-19T08:45:27.808244+0000","last_active":"2024-09-19T08:45:28.784876+0000","last_peered":"2024-09-19T08:45:28.784876+0000","last_clean":"2024-09-19T08:45:28.784876+0000","last_became_active":"2024-09-19T08:45:27.807341+0000","last_became_peered":"2024-09-19T08:45:27.807341+0000","last_unstale":"2024-09-19T08:45:28.784876+0000","last_undegraded":"2024-09-19T08:45:28.784876+0000","last_fullsized":"2024-09-19T08:45:28.784876+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:47:24.008928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.819157+0000","last_change":"2024-09-19T08:45:28.817319+0000","last_active":"2024-09-19T08:45:28.819157+0000","last_peered":"2024-09-19T08:45:28.819157+0000","last_clean":"2024-09-19T08:45:28.819157+0000","last_became_active":"2024-09-19T08:45:28.817041+0000","last_became_peered":"2024-09-19T08:45:28.817041+0000","last_unstale":"2024-09-19T08:45:28.819157+0000","last_undegraded":"2024-09-19T08:45:28.819157+0000","last_fullsized":"2024-09-19T08:45:28.819157+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:00:35.485683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"134'2","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.824232+0000","last_change":"2024-09-19T08:45:27.823341+0000","last_active":"2024-09-19T08:45:28.824232+0000","last_peered":"2024-09-19T08:45:28.824232+0000","last_clean":"2024-09-19T08:45:28.824232+0000","last_became_active":"2024-09-19T08:45:27.822777+0000","last_became_peered":"2024-09-19T08:45:27.822777+0000","last_unstale":"2024-09-19T08:45:28.824232+0000","last_undegraded":"2024-09-19T08:45:28.824232+0000","last_fullsized":"2024-09-19T08:45:28.824232+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:00:01.538292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.808071+0000","last_change":"2024-09-19T08:45:28.803094+0000","last_active":"2024-09-19T08:45:28.808071+0000","last_peered":"2024-09-19T08:45:28.808071+0000","last_clean":"2024-09-19T08:45:28.808071+0000","last_became_active":"2024-09-19T08:45:28.802778+0000","last_became_peered":"2024-09-19T08:45:28.802778+0000","last_unstale":"2024-09-19T08:45:28.808071+0000","last_undegraded":"2024-09-19T08:45:28.808071+0000","last_fullsized":"2024-09-19T08:45:28.808071+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:38:15.564188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.791448+0000","last_change":"2024-09-19T08:45:27.816218+0000","last_active":"2024-09-19T08:45:28.791448+0000","last_peered":"2024-09-19T08:45:28.791448+0000","last_clean":"2024-09-19T08:45:28.791448+0000","last_became_active":"2024-09-19T08:45:27.815828+0000","last_became_peered":"2024-09-19T08:45:27.815828+0000","last_unstale":"2024-09-19T08:45:28.791448+0000","last_undegraded":"2024-09-19T08:45:28.791448+0000","last_fullsized":"2024-09-19T08:45:28.791448+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:46:50.000531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.799778+0000","last_change":"2024-09-19T08:45:28.794134+0000","last_active":"2024-09-19T08:45:28.799778+0000","last_peered":"2024-09-19T08:45:28.799778+0000","last_clean":"2024-09-19T08:45:28.799778+0000","last_became_active":"2024-09-19T08:45:28.793812+0000","last_became_peered":"2024-09-19T08:45:28.793812+0000","last_unstale":"2024-09-19T08:45:28.799778+0000","last_undegraded":"2024-09-19T08:45:28.799778+0000","last_fullsized":"2024-09-19T08:45:28.799778+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:06:08.019991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.799211+0000","last_change":"2024-09-19T08:45:27.790487+0000","last_active":"2024-09-19T08:45:28.799211+0000","last_peered":"2024-09-19T08:45:28.799211+0000","last_clean":"2024-09-19T08:45:28.799211+0000","last_became_active":"2024-09-19T08:45:27.790041+0000","last_became_peered":"2024-09-19T08:45:27.790041+0000","last_unstale":"2024-09-19T08:45:28.799211+0000","last_undegraded":"2024-09-19T08:45:28.799211+0000","last_fullsized":"2024-09-19T08:45:28.799211+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:38:16.900241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.816060+0000","last_change":"2024-09-19T08:45:28.805958+0000","last_active":"2024-09-19T08:45:28.816060+0000","last_peered":"2024-09-19T08:45:28.816060+0000","last_clean":"2024-09-19T08:45:28.816060+0000","last_became_active":"2024-09-19T08:45:28.802562+0000","last_became_peered":"2024-09-19T08:45:28.802562+0000","last_unstale":"2024-09-19T08:45:28.816060+0000","last_undegraded":"2024-09-19T08:45:28.816060+0000","last_fullsized":"2024-09-19T08:45:28.816060+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:36:37.461145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.793302+0000","last_change":"2024-09-19T08:45:27.791679+0000","last_active":"2024-09-19T08:45:28.793302+0000","last_peered":"2024-09-19T08:45:28.793302+0000","last_clean":"2024-09-19T08:45:28.793302+0000","last_became_active":"2024-09-19T08:45:27.791127+0000","last_became_peered":"2024-09-19T08:45:27.791127+0000","last_unstale":"2024-09-19T08:45:28.793302+0000","last_undegraded":"2024-09-19T08:45:28.793302+0000","last_fullsized":"2024-09-19T08:45:28.793302+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:53.762737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.807508+0000","last_change":"2024-09-19T08:45:28.805693+0000","last_active":"2024-09-19T08:45:28.807508+0000","last_peered":"2024-09-19T08:45:28.807508+0000","last_clean":"2024-09-19T08:45:28.807508+0000","last_became_active":"2024-09-19T08:45:28.804944+0000","last_became_peered":"2024-09-19T08:45:28.804944+0000","last_unstale":"2024-09-19T08:45:28.807508+0000","last_undegraded":"2024-09-19T08:45:28.807508+0000","last_fullsized":"2024-09-19T08:45:28.807508+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:41.779324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.790536+0000","last_change":"2024-09-19T08:45:27.796633+0000","last_active":"2024-09-19T08:45:28.790536+0000","last_peered":"2024-09-19T08:45:28.790536+0000","last_clean":"2024-09-19T08:45:28.790536+0000","last_became_active":"2024-09-19T08:45:27.795201+0000","last_became_peered":"2024-09-19T08:45:27.795201+0000","last_unstale":"2024-09-19T08:45:28.790536+0000","last_undegraded":"2024-09-19T08:45:28.790536+0000","last_fullsized":"2024-09-19T08:45:28.790536+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:20:51.060225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.818016+0000","last_change":"2024-09-19T08:45:28.814539+0000","last_active":"2024-09-19T08:45:28.818016+0000","last_peered":"2024-09-19T08:45:28.818016+0000","last_clean":"2024-09-19T08:45:28.818016+0000","last_became_active":"2024-09-19T08:45:28.814215+0000","last_became_peered":"2024-09-19T08:45:28.814215+0000","last_unstale":"2024-09-19T08:45:28.818016+0000","last_undegraded":"2024-09-19T08:45:28.818016+0000","last_fullsized":"2024-09-19T08:45:28.818016+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:36:39.571493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.780047+0000","last_change":"2024-09-19T08:45:27.823111+0000","last_active":"2024-09-19T08:45:28.780047+0000","last_peered":"2024-09-19T08:45:28.780047+0000","last_clean":"2024-09-19T08:45:28.780047+0000","last_became_active":"2024-09-19T08:45:27.822315+0000","last_became_peered":"2024-09-19T08:45:27.822315+0000","last_unstale":"2024-09-19T08:45:28.780047+0000","last_undegraded":"2024-09-19T08:45:28.780047+0000","last_fullsized":"2024-09-19T08:45:28.780047+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:40.411815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.815777+0000","last_change":"2024-09-19T08:45:28.808737+0000","last_active":"2024-09-19T08:45:28.815777+0000","last_peered":"2024-09-19T08:45:28.815777+0000","last_clean":"2024-09-19T08:45:28.815777+0000","last_became_active":"2024-09-19T08:45:28.808439+0000","last_became_peered":"2024-09-19T08:45:28.808439+0000","last_unstale":"2024-09-19T08:45:28.815777+0000","last_undegraded":"2024-09-19T08:45:28.815777+0000","last_fullsized":"2024-09-19T08:45:28.815777+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:20:52.064260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.787053+0000","last_change":"2024-09-19T08:45:27.818609+0000","last_active":"2024-09-19T08:45:28.787053+0000","last_peered":"2024-09-19T08:45:28.787053+0000","last_clean":"2024-09-19T08:45:28.787053+0000","last_became_active":"2024-09-19T08:45:27.817870+0000","last_became_peered":"2024-09-19T08:45:27.817870+0000","last_unstale":"2024-09-19T08:45:28.787053+0000","last_undegraded":"2024-09-19T08:45:28.787053+0000","last_fullsized":"2024-09-19T08:45:28.787053+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:52.344306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.819076+0000","last_change":"2024-09-19T08:45:28.815490+0000","last_active":"2024-09-19T08:45:28.819076+0000","last_peered":"2024-09-19T08:45:28.819076+0000","last_clean":"2024-09-19T08:45:28.819076+0000","last_became_active":"2024-09-19T08:45:28.815219+0000","last_became_peered":"2024-09-19T08:45:28.815219+0000","last_unstale":"2024-09-19T08:45:28.819076+0000","last_undegraded":"2024-09-19T08:45:28.819076+0000","last_fullsized":"2024-09-19T08:45:28.819076+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:13.992317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.778005+0000","last_change":"2024-09-19T08:45:27.808519+0000","last_active":"2024-09-19T08:45:28.778005+0000","last_peered":"2024-09-19T08:45:28.778005+0000","last_clean":"2024-09-19T08:45:28.778005+0000","last_became_active":"2024-09-19T08:45:27.808121+0000","last_became_peered":"2024-09-19T08:45:27.808121+0000","last_unstale":"2024-09-19T08:45:28.778005+0000","last_undegraded":"2024-09-19T08:45:28.778005+0000","last_fullsized":"2024-09-19T08:45:28.778005+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:42:26.773772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.813800+0000","last_change":"2024-09-19T08:45:28.809942+0000","last_active":"2024-09-19T08:45:28.813800+0000","last_peered":"2024-09-19T08:45:28.813800+0000","last_clean":"2024-09-19T08:45:28.813800+0000","last_became_active":"2024-09-19T08:45:28.806616+0000","last_became_peered":"2024-09-19T08:45:28.806616+0000","last_unstale":"2024-09-19T08:45:28.813800+0000","last_undegraded":"2024-09-19T08:45:28.813800+0000","last_fullsized":"2024-09-19T08:45:28.813800+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:27.758631+0000","last_clean_scrub_stamp":"2024-09-19T08:45:27.758631+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:45.220093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"134'3","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.813764+0000","last_change":"2024-09-19T08:45:27.808111+0000","last_active":"2024-09-19T08:45:28.813764+0000","last_peered":"2024-09-19T08:45:28.813764+0000","last_clean":"2024-09-19T08:45:28.813764+0000","last_became_active":"2024-09-19T08:45:27.807232+0000","last_became_peered":"2024-09-19T08:45:27.807232+0000","last_unstale":"2024-09-19T08:45:28.813764+0000","last_undegraded":"2024-09-19T08:45:28.813764+0000","last_fullsized":"2024-09-19T08:45:28.813764+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:45:26.754595+0000","last_clean_scrub_stamp":"2024-09-19T08:45:26.754595+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:40.476763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"71'128","reported_seq":413,"reported_epoch":134,"state":"active+clean","last_fresh":"2024-09-19T08:45:28.780363+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:45:28.780363+0000","last_peered":"2024-09-19T08:45:28.780363+0000","last_clean":"2024-09-19T08:45:28.780363+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:45:28.780363+0000","last_undegraded":"2024-09-19T08:45:28.780363+0000","last_fullsized":"2024-09-19T08:45:28.780363+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509531,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33392,"kb_used_data":2200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338448,"statfs":{"total":96636764160,"available":96602570752,"internally_reserved":0,"allocated":2252800,"data_stored":1325917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32808,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339032,"statfs":{"total":96636764160,"available":96603168768,"internally_reserved":0,"allocated":1658880,"data_stored":735549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509533,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1658880,"data_stored":735549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509533,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1658880,"data_stored":735549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542235,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1658880,"data_stored":735549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509530,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1658880,"data_stored":735549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509532,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32832,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339008,"statfs":{"total":96636764160,"available":96603144192,"internally_reserved":0,"allocated":1675264,"data_stored":742934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509530,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33392,"kb_used_data":2200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338448,"statfs":{"total":96636764160,"available":96602570752,"internally_reserved":0,"allocated":2252800,"data_stored":1325917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:45:32.202 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T08:45:32.202 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-19T08:45:32.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.363+0000 7f79c374c640 1 Processor -- start 2024-09-19T08:45:32.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.363+0000 7f79c374c640 1 -- start start 2024-09-19T08:45:32.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.363+0000 7f79c374c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f79bc145b70 0x7f79bc06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:32.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.363+0000 7f79c374c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f79bc06c1d0 0x7f79bc0713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:32.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.363+0000 7f79c374c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79bc0718f0 0x7f79bc0cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:32.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.363+0000 7f79c374c640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f79bc06c620 con 0x7f79bc06c1d0 2024-09-19T08:45:32.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.363+0000 7f79c374c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f79bc073530 con 0x7f79bc145b70 2024-09-19T08:45:32.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.363+0000 7f79c374c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f79bc073810 con 0x7f79bc0718f0 2024-09-19T08:45:32.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.363+0000 7f79c0ce7640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f79bc06c1d0 0x7f79bc0713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:32.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.363+0000 7f79c0ce7640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f79bc06c1d0 0x7f79bc0713b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35562/0 (socket says 172.21.15.98:35562) 2024-09-19T08:45:32.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.363+0000 7f79c0ce7640 1 -- 172.21.15.98:0/4140183869 learned_addr learned my addr 172.21.15.98:0/4140183869 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:32.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.363+0000 7f79c14e8640 1 --2- 172.21.15.98:0/4140183869 >> v2:172.21.15.139:3300/0 conn(0x7f79bc145b70 0x7f79bc06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:32.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.363+0000 7f79c1ce9640 1 --2- 172.21.15.98:0/4140183869 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79bc0718f0 0x7f79bc0cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:32.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.363+0000 7f79c0ce7640 1 -- 172.21.15.98:0/4140183869 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79bc0718f0 msgr2=0x7f79bc0cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:32.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.363+0000 7f79c0ce7640 1 --2- 172.21.15.98:0/4140183869 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79bc0718f0 0x7f79bc0cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:32.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.363+0000 7f79c0ce7640 1 -- 172.21.15.98:0/4140183869 >> v2:172.21.15.139:3300/0 conn(0x7f79bc145b70 msgr2=0x7f79bc06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:32.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.363+0000 7f79c0ce7640 1 --2- 172.21.15.98:0/4140183869 >> v2:172.21.15.139:3300/0 conn(0x7f79bc145b70 0x7f79bc06bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:32.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.363+0000 7f79c0ce7640 1 -- 172.21.15.98:0/4140183869 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79bc073af0 con 0x7f79bc06c1d0 2024-09-19T08:45:32.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.367+0000 7f79c0ce7640 1 --2- 172.21.15.98:0/4140183869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79bc06c1d0 0x7f79bc0713b0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f79bc0b2d30 tx=0x7f79ac01c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4fdf9f4c87a127 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:32.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.367+0000 7f79b3fff640 1 -- 172.21.15.98:0/4140183869 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac0b0070 con 0x7f79bc06c1d0 2024-09-19T08:45:32.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.367+0000 7f79b3fff640 1 -- 172.21.15.98:0/4140183869 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79ac0ab600 con 0x7f79bc06c1d0 2024-09-19T08:45:32.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.367+0000 7f79b3fff640 1 -- 172.21.15.98:0/4140183869 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac0ab9d0 con 0x7f79bc06c1d0 2024-09-19T08:45:32.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.367+0000 7f79c374c640 1 -- 172.21.15.98:0/4140183869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79bc06c1d0 msgr2=0x7f79bc0713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:32.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.367+0000 7f79c374c640 1 --2- 172.21.15.98:0/4140183869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79bc06c1d0 0x7f79bc0713b0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f79bc0b2d30 tx=0x7f79ac01c1d0 comp rx=0 tx=0).stop 2024-09-19T08:45:32.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.367+0000 7f79c374c640 1 -- 172.21.15.98:0/4140183869 shutdown_connections 2024-09-19T08:45:32.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.367+0000 7f79c374c640 1 --2- 172.21.15.98:0/4140183869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79bc06c1d0 0x7f79bc0713b0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:32.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.367+0000 7f79c374c640 1 --2- 172.21.15.98:0/4140183869 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79bc0718f0 0x7f79bc0cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:32.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.367+0000 7f79c374c640 1 --2- 172.21.15.98:0/4140183869 >> v2:172.21.15.139:3300/0 conn(0x7f79bc145b70 0x7f79bc06bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:32.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.367+0000 7f79c374c640 1 -- 172.21.15.98:0/4140183869 >> 172.21.15.98:0/4140183869 conn(0x7f79bc0b2010 msgr2=0x7f79bc0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:32.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.367+0000 7f79c374c640 1 -- 172.21.15.98:0/4140183869 shutdown_connections 2024-09-19T08:45:32.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.367+0000 7f79c374c640 1 -- 172.21.15.98:0/4140183869 wait complete. 2024-09-19T08:45:32.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.367+0000 7f79c374c640 1 Processor -- start 2024-09-19T08:45:32.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.367+0000 7f79c374c640 1 -- start start 2024-09-19T08:45:32.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.371+0000 7f79c374c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79bc06c1d0 0x7f79bc1434a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:32.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.371+0000 7f79c374c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79bc0718f0 0x7f79bc15f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:32.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.371+0000 7f79c374c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79bc145b70 0x7f79bc165810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:32.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.371+0000 7f79c374c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f79bc0b3410 con 0x7f79bc145b70 2024-09-19T08:45:32.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.371+0000 7f79c374c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f79bc0b3590 con 0x7f79bc0718f0 2024-09-19T08:45:32.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.371+0000 7f79c374c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f79bc0b3710 con 0x7f79bc06c1d0 2024-09-19T08:45:32.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.371+0000 7f79c1ce9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79bc145b70 0x7f79bc165810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:32.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.371+0000 7f79c1ce9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79bc145b70 0x7f79bc165810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35572/0 (socket says 172.21.15.98:35572) 2024-09-19T08:45:32.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.371+0000 7f79c1ce9640 1 -- 172.21.15.98:0/2705835738 learned_addr learned my addr 172.21.15.98:0/2705835738 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:32.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.371+0000 7f79c14e8640 1 --2- 172.21.15.98:0/2705835738 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79bc06c1d0 0x7f79bc1434a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:32.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.371+0000 7f79c0ce7640 1 --2- 172.21.15.98:0/2705835738 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79bc0718f0 0x7f79bc15f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:32.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.371+0000 7f79c1ce9640 1 -- 172.21.15.98:0/2705835738 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79bc06c1d0 msgr2=0x7f79bc1434a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:32.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.371+0000 7f79c1ce9640 1 --2- 172.21.15.98:0/2705835738 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79bc06c1d0 0x7f79bc1434a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:32.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.371+0000 7f79c1ce9640 1 -- 172.21.15.98:0/2705835738 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79bc0718f0 msgr2=0x7f79bc15f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:32.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.371+0000 7f79c1ce9640 1 --2- 172.21.15.98:0/2705835738 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79bc0718f0 0x7f79bc15f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:32.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.371+0000 7f79c1ce9640 1 -- 172.21.15.98:0/2705835738 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79bc143b80 con 0x7f79bc145b70 2024-09-19T08:45:32.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.371+0000 7f79c14e8640 1 --2- 172.21.15.98:0/2705835738 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79bc06c1d0 0x7f79bc1434a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:32.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.371+0000 7f79c1ce9640 1 --2- 172.21.15.98:0/2705835738 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79bc145b70 0x7f79bc165810 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f79b4007e60 tx=0x7f79b400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:32.374 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.371+0000 7f79b1ffb640 1 -- 172.21.15.98:0/2705835738 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4014620 con 0x7f79bc145b70 2024-09-19T08:45:32.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.371+0000 7f79b1ffb640 1 -- 172.21.15.98:0/2705835738 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79b4019070 con 0x7f79bc145b70 2024-09-19T08:45:32.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.371+0000 7f79c374c640 1 -- 172.21.15.98:0/2705835738 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79bc143e60 con 0x7f79bc145b70 2024-09-19T08:45:32.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.371+0000 7f79b1ffb640 1 -- 172.21.15.98:0/2705835738 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4014a20 con 0x7f79bc145b70 2024-09-19T08:45:32.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.371+0000 7f79c374c640 1 -- 172.21.15.98:0/2705835738 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79bc144420 con 0x7f79bc145b70 2024-09-19T08:45:32.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.371+0000 7f79c374c640 1 -- 172.21.15.98:0/2705835738 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79bc0cef70 con 0x7f79bc145b70 2024-09-19T08:45:32.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.371+0000 7f79b1ffb640 1 -- 172.21.15.98:0/2705835738 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f79b4014bc0 con 0x7f79bc145b70 2024-09-19T08:45:32.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.371+0000 7f79b1ffb640 1 --2- 172.21.15.98:0/2705835738 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f79a0067e90 0x7f79a006a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:32.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.371+0000 7f79b1ffb640 1 -- 172.21.15.98:0/2705835738 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7f79b4086f10 con 0x7f79bc145b70 2024-09-19T08:45:32.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.371+0000 7f79c14e8640 1 --2- 172.21.15.98:0/2705835738 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f79a0067e90 0x7f79a006a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:32.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.375+0000 7f79c14e8640 1 --2- 172.21.15.98:0/2705835738 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f79a0067e90 0x7f79a006a350 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f79bc0cc500 tx=0x7f79b8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:32.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.375+0000 7f79b1ffb640 1 -- 172.21.15.98:0/2705835738 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f79b4011260 con 0x7f79bc145b70 2024-09-19T08:45:32.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.583+0000 7f79c374c640 1 -- 172.21.15.98:0/2705835738 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f79bc0b3d10 con 0x7f79bc145b70 2024-09-19T08:45:32.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.587+0000 7f79b1ffb640 1 -- 172.21.15.98:0/2705835738 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) ==== 190+0+147 (secure 0 0 0) 0x7f79b40571c0 con 0x7f79bc145b70 2024-09-19T08:45:32.591 INFO:teuthology.orchestra.run.smithi098.stdout:[client.0] 2024-09-19T08:45:32.591 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQC/4utm0vNpIhAA4S1E8dFZ0+ibmSKhmZ+bIA== 2024-09-19T08:45:32.591 INFO:teuthology.orchestra.run.smithi098.stdout: caps mds = "allow" 2024-09-19T08:45:32.591 INFO:teuthology.orchestra.run.smithi098.stdout: caps mon = "allow r" 2024-09-19T08:45:32.591 INFO:teuthology.orchestra.run.smithi098.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-19T08:45:32.591 INFO:teuthology.orchestra.run.smithi098.stderr:updated caps for client.0 2024-09-19T08:45:32.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.591+0000 7f79c374c640 1 -- 172.21.15.98:0/2705835738 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f79a0067e90 msgr2=0x7f79a006a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:32.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.591+0000 7f79c374c640 1 --2- 172.21.15.98:0/2705835738 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f79a0067e90 0x7f79a006a350 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f79bc0cc500 tx=0x7f79b8008040 comp rx=0 tx=0).stop 2024-09-19T08:45:32.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.591+0000 7f79c374c640 1 -- 172.21.15.98:0/2705835738 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79bc145b70 msgr2=0x7f79bc165810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:32.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.591+0000 7f79c374c640 1 --2- 172.21.15.98:0/2705835738 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79bc145b70 0x7f79bc165810 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f79b4007e60 tx=0x7f79b400e4b0 comp rx=0 tx=0).stop 2024-09-19T08:45:32.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.591+0000 7f79c374c640 1 -- 172.21.15.98:0/2705835738 shutdown_connections 2024-09-19T08:45:32.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.591+0000 7f79c374c640 1 --2- 172.21.15.98:0/2705835738 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f79a0067e90 0x7f79a006a350 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:32.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.591+0000 7f79c374c640 1 --2- 172.21.15.98:0/2705835738 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79bc145b70 0x7f79bc165810 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:32.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.591+0000 7f79c374c640 1 --2- 172.21.15.98:0/2705835738 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79bc0718f0 0x7f79bc15f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:32.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.591+0000 7f79c374c640 1 --2- 172.21.15.98:0/2705835738 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79bc06c1d0 0x7f79bc1434a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:32.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.591+0000 7f79c374c640 1 -- 172.21.15.98:0/2705835738 >> 172.21.15.98:0/2705835738 conn(0x7f79bc0b2010 msgr2=0x7f79bc1665b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:32.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.591+0000 7f79c374c640 1 -- 172.21.15.98:0/2705835738 shutdown_connections 2024-09-19T08:45:32.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.591+0000 7f79c374c640 1 -- 172.21.15.98:0/2705835738 wait complete. 2024-09-19T08:45:32.610 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:45:32.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.723+0000 7fb16a045640 1 Processor -- start 2024-09-19T08:45:32.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.723+0000 7fb16a045640 1 -- start start 2024-09-19T08:45:32.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.723+0000 7fb16a045640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb16415c550 0x7fb16406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:32.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.723+0000 7fb16a045640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb1640cfad0 0x7fb1640723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:32.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.723+0000 7fb16a045640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb16406d310 con 0x7fb16406c1d0 2024-09-19T08:45:32.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.723+0000 7fb16a045640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fb16406c640 con 0x7fb1640cfad0 2024-09-19T08:45:32.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.723+0000 7fb16a045640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fb16406d5f0 con 0x7fb16415c550 2024-09-19T08:45:32.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.723+0000 7fb163fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb1640cfad0 0x7fb1640723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:32.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.723+0000 7fb163fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb1640cfad0 0x7fb1640723a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35588/0 (socket says 172.21.15.98:35588) 2024-09-19T08:45:32.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.723+0000 7fb163fff640 1 -- 172.21.15.98:0/3470308703 learned_addr learned my addr 172.21.15.98:0/3470308703 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:32.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.723+0000 7fb1637fe640 1 --2- 172.21.15.98:0/3470308703 >> v2:172.21.15.139:3300/0 conn(0x7fb16415c550 0x7fb16406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:32.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.723+0000 7fb163fff640 1 -- 172.21.15.98:0/3470308703 >> v2:172.21.15.139:3300/0 conn(0x7fb16415c550 msgr2=0x7fb16406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:32.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.723+0000 7fb163fff640 1 --2- 172.21.15.98:0/3470308703 >> v2:172.21.15.139:3300/0 conn(0x7fb16415c550 0x7fb16406bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:32.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.723+0000 7fb163fff640 1 -- 172.21.15.98:0/3470308703 >> v1:172.21.15.98:6789/0 conn(0x7fb16406c1d0 legacy=0x7fb1640cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:32.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.723+0000 7fb163fff640 1 -- 172.21.15.98:0/3470308703 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb16406d8d0 con 0x7fb1640cfad0 2024-09-19T08:45:32.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.723+0000 7fb1637fe640 1 --2- 172.21.15.98:0/3470308703 >> v2:172.21.15.139:3300/0 conn(0x7fb16415c550 0x7fb16406bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:32.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.727+0000 7fb163fff640 1 --2- 172.21.15.98:0/3470308703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1640cfad0 0x7fb1640723a0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fb1640b56b0 tx=0x7fb15c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af2a0c98269c2ccd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:32.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.727+0000 7fb1627fc640 1 -- 172.21.15.98:0/3470308703 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c0a56c0 con 0x7fb1640cfad0 2024-09-19T08:45:32.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.727+0000 7fb1627fc640 1 -- 172.21.15.98:0/3470308703 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb15c0a5860 con 0x7fb1640cfad0 2024-09-19T08:45:32.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.727+0000 7fb1627fc640 1 -- 172.21.15.98:0/3470308703 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c0a5c30 con 0x7fb1640cfad0 2024-09-19T08:45:32.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.727+0000 7fb16a045640 1 -- 172.21.15.98:0/3470308703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1640cfad0 msgr2=0x7fb1640723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:32.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.727+0000 7fb16a045640 1 --2- 172.21.15.98:0/3470308703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1640cfad0 0x7fb1640723a0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fb1640b56b0 tx=0x7fb15c01c9f0 comp rx=0 tx=0).stop 2024-09-19T08:45:32.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.727+0000 7fb16a045640 1 -- 172.21.15.98:0/3470308703 shutdown_connections 2024-09-19T08:45:32.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.727+0000 7fb16a045640 1 --2- 172.21.15.98:0/3470308703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1640cfad0 0x7fb1640723a0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:32.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.727+0000 7fb16a045640 1 --2- 172.21.15.98:0/3470308703 >> v2:172.21.15.139:3300/0 conn(0x7fb16415c550 0x7fb16406bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:32.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.727+0000 7fb16a045640 1 -- 172.21.15.98:0/3470308703 >> 172.21.15.98:0/3470308703 conn(0x7fb1640b2010 msgr2=0x7fb1640b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:32.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.727+0000 7fb16a045640 1 -- 172.21.15.98:0/3470308703 shutdown_connections 2024-09-19T08:45:32.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.727+0000 7fb16a045640 1 -- 172.21.15.98:0/3470308703 wait complete. 2024-09-19T08:45:32.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.727+0000 7fb16a045640 1 Processor -- start 2024-09-19T08:45:32.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.727+0000 7fb16a045640 1 -- start start 2024-09-19T08:45:32.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.727+0000 7fb16a045640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb16406c1d0 0x7fb16414ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:32.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.727+0000 7fb16a045640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1640cfad0 0x7fb1641502c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:32.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.727+0000 7fb16a045640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb16415c550 0x7fb164155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:32.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.727+0000 7fb16a045640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb1640734c0 con 0x7fb16415c550 2024-09-19T08:45:32.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.727+0000 7fb16a045640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb164073340 con 0x7fb16406c1d0 2024-09-19T08:45:32.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.727+0000 7fb16a045640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb164073640 con 0x7fb1640cfad0 2024-09-19T08:45:32.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.727+0000 7fb163fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb16415c550 0x7fb164155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:32.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.727+0000 7fb163fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb16415c550 0x7fb164155da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35604/0 (socket says 172.21.15.98:35604) 2024-09-19T08:45:32.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.727+0000 7fb163fff640 1 -- 172.21.15.98:0/2914791617 learned_addr learned my addr 172.21.15.98:0/2914791617 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:32.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.727+0000 7fb1637fe640 1 --2- 172.21.15.98:0/2914791617 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb16406c1d0 0x7fb16414ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:32.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.727+0000 7fb162ffd640 1 --2- 172.21.15.98:0/2914791617 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1640cfad0 0x7fb1641502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:32.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.727+0000 7fb163fff640 1 -- 172.21.15.98:0/2914791617 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1640cfad0 msgr2=0x7fb1641502c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:32.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.727+0000 7fb163fff640 1 --2- 172.21.15.98:0/2914791617 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1640cfad0 0x7fb1641502c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:32.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.727+0000 7fb163fff640 1 -- 172.21.15.98:0/2914791617 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb16406c1d0 msgr2=0x7fb16414ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:32.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.727+0000 7fb163fff640 1 --2- 172.21.15.98:0/2914791617 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb16406c1d0 0x7fb16414ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:32.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.727+0000 7fb163fff640 1 -- 172.21.15.98:0/2914791617 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb16414f310 con 0x7fb16415c550 2024-09-19T08:45:32.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.727+0000 7fb162ffd640 1 --2- 172.21.15.98:0/2914791617 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1640cfad0 0x7fb1641502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:32.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.731+0000 7fb163fff640 1 --2- 172.21.15.98:0/2914791617 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb16415c550 0x7fb164155da0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fb15c0af040 tx=0x7fb15c019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:32.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.731+0000 7fb14bfff640 1 -- 172.21.15.98:0/2914791617 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c0b8670 con 0x7fb16415c550 2024-09-19T08:45:32.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.731+0000 7fb14bfff640 1 -- 172.21.15.98:0/2914791617 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb15c0be070 con 0x7fb16415c550 2024-09-19T08:45:32.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.731+0000 7fb16a045640 1 -- 172.21.15.98:0/2914791617 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb16414f5f0 con 0x7fb16415c550 2024-09-19T08:45:32.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.731+0000 7fb14bfff640 1 -- 172.21.15.98:0/2914791617 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c0b8a70 con 0x7fb16415c550 2024-09-19T08:45:32.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.731+0000 7fb16a045640 1 -- 172.21.15.98:0/2914791617 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb16414fbb0 con 0x7fb16415c550 2024-09-19T08:45:32.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.731+0000 7fb16a045640 1 -- 172.21.15.98:0/2914791617 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1640723a0 con 0x7fb16415c550 2024-09-19T08:45:32.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.731+0000 7fb14bfff640 1 -- 172.21.15.98:0/2914791617 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fb15c0b8c10 con 0x7fb16415c550 2024-09-19T08:45:32.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.731+0000 7fb14bfff640 1 --2- 172.21.15.98:0/2914791617 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb150067e60 0x7fb15006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:32.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.731+0000 7fb14bfff640 1 -- 172.21.15.98:0/2914791617 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7fb15c122250 con 0x7fb16415c550 2024-09-19T08:45:32.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.731+0000 7fb1637fe640 1 --2- 172.21.15.98:0/2914791617 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb150067e60 0x7fb15006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:32.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.731+0000 7fb1637fe640 1 --2- 172.21.15.98:0/2914791617 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb150067e60 0x7fb15006a320 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb158006fd0 tx=0x7fb158008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:32.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.735+0000 7fb14bfff640 1 -- 172.21.15.98:0/2914791617 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb15c0165e0 con 0x7fb16415c550 2024-09-19T08:45:32.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.931+0000 7fb16a045640 1 -- 172.21.15.98:0/2914791617 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb164158f90 con 0x7fb16415c550 2024-09-19T08:45:32.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.935+0000 7fb14bfff640 1 -- 172.21.15.98:0/2914791617 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5230 (secure 0 0 0) 0x7fb15c0c23c0 con 0x7fb16415c550 2024-09-19T08:45:32.937 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:45:32.937 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":74,"btime":"2024-09-19T08:45:30:805559+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/1907681163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":1907681163},{"type":"v1","addr":"172.21.15.139:6834","nonce":1907681163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7896,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2481662340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2481662340},{"type":"v1","addr":"172.21.15.98:6833","nonce":2481662340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7902,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/286818221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":286818221},{"type":"v1","addr":"172.21.15.98:6835","nonce":286818221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:45:28.140764+0000","modified":"2024-09-19T08:45:30.144582+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7914},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7914":{"gid":7914,"name":"b","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6835/4255670104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6833","nonce":4255670104},{"type":"v1","addr":"172.21.15.139:6835","nonce":4255670104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7914,"qdb_cluster":[7914]},"id":10}]} 2024-09-19T08:45:32.937 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 74 2024-09-19T08:45:32.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.939+0000 7fb16a045640 1 -- 172.21.15.98:0/2914791617 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb150067e60 msgr2=0x7fb15006a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:32.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.939+0000 7fb16a045640 1 --2- 172.21.15.98:0/2914791617 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb150067e60 0x7fb15006a320 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb158006fd0 tx=0x7fb158008040 comp rx=0 tx=0).stop 2024-09-19T08:45:32.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.939+0000 7fb16a045640 1 -- 172.21.15.98:0/2914791617 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb16415c550 msgr2=0x7fb164155da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:32.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.939+0000 7fb16a045640 1 --2- 172.21.15.98:0/2914791617 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb16415c550 0x7fb164155da0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fb15c0af040 tx=0x7fb15c019790 comp rx=0 tx=0).stop 2024-09-19T08:45:32.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.939+0000 7fb16a045640 1 -- 172.21.15.98:0/2914791617 shutdown_connections 2024-09-19T08:45:32.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.939+0000 7fb16a045640 1 --2- 172.21.15.98:0/2914791617 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb150067e60 0x7fb15006a320 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:32.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.939+0000 7fb16a045640 1 --2- 172.21.15.98:0/2914791617 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb16415c550 0x7fb164155da0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:32.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.939+0000 7fb16a045640 1 --2- 172.21.15.98:0/2914791617 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1640cfad0 0x7fb1641502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:32.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.939+0000 7fb16a045640 1 --2- 172.21.15.98:0/2914791617 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb16406c1d0 0x7fb16414ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:32.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.939+0000 7fb16a045640 1 -- 172.21.15.98:0/2914791617 >> 172.21.15.98:0/2914791617 conn(0x7fb1640b2010 msgr2=0x7fb1640aff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:32.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.939+0000 7fb16a045640 1 -- 172.21.15.98:0/2914791617 shutdown_connections 2024-09-19T08:45:32.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:32.939+0000 7fb16a045640 1 -- 172.21.15.98:0/2914791617 wait complete. 2024-09-19T08:45:33.359 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T08:45:28.140764+0000', 'modified': '2024-09-19T08:45:30.144582+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7914}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7914': {'gid': 7914, 'name': 'b', 'rank': 0, 'incarnation': 72, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.139:6835/4255670104', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6833', 'nonce': 4255670104}, {'type': 'v1', 'addr': '172.21.15.139:6835', 'nonce': 4255670104}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7914, 'qdb_cluster': [7914]} 2024-09-19T08:45:33.360 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T08:45:33.360 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-19T08:45:33.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.487+0000 7f587e80f640 1 Processor -- start 2024-09-19T08:45:33.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.487+0000 7f587e80f640 1 -- start start 2024-09-19T08:45:33.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.487+0000 7f587e80f640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f587806bc90 0x7f587806c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:33.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.487+0000 7f587e80f640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58780ccc60 con 0x7f5878158fd0 2024-09-19T08:45:33.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.487+0000 7f587e80f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58780ccf40 con 0x7f58780ce840 2024-09-19T08:45:33.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.487+0000 7f587e80f640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f58781593b0 con 0x7f587806bc90 2024-09-19T08:45:33.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.491+0000 7f58777fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f587806bc90 0x7f587806c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:33.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.491+0000 7f58777fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f587806bc90 0x7f587806c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35610/0 (socket says 172.21.15.98:35610) 2024-09-19T08:45:33.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.491+0000 7f58777fe640 1 -- 172.21.15.98:0/3349706165 learned_addr learned my addr 172.21.15.98:0/3349706165 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:33.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.491+0000 7f58777fe640 1 -- 172.21.15.98:0/3349706165 >> v1:172.21.15.139:6789/0 conn(0x7f58780ce840 legacy=0x7f58780cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:33.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.491+0000 7f58777fe640 1 -- 172.21.15.98:0/3349706165 >> v1:172.21.15.98:6789/0 conn(0x7f5878158fd0 legacy=0x7f58780b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:33.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.491+0000 7f58777fe640 1 -- 172.21.15.98:0/3349706165 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58780cd220 con 0x7f587806bc90 2024-09-19T08:45:33.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.491+0000 7f58777fe640 1 --2- 172.21.15.98:0/3349706165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f587806bc90 0x7f587806c120 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f586c009870 tx=0x7f586c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725199374cff6e0d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:33.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.491+0000 7f5876ffd640 1 -- 172.21.15.98:0/3349706165 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c0b0070 con 0x7f587806bc90 2024-09-19T08:45:33.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.491+0000 7f5876ffd640 1 -- 172.21.15.98:0/3349706165 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f586c0ab600 con 0x7f587806bc90 2024-09-19T08:45:33.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.491+0000 7f5876ffd640 1 -- 172.21.15.98:0/3349706165 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c0ab9d0 con 0x7f587806bc90 2024-09-19T08:45:33.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.491+0000 7f587e80f640 1 -- 172.21.15.98:0/3349706165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f587806bc90 msgr2=0x7f587806c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:33.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.491+0000 7f587e80f640 1 --2- 172.21.15.98:0/3349706165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f587806bc90 0x7f587806c120 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f586c009870 tx=0x7f586c01c490 comp rx=0 tx=0).stop 2024-09-19T08:45:33.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.491+0000 7f587e80f640 1 -- 172.21.15.98:0/3349706165 shutdown_connections 2024-09-19T08:45:33.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.491+0000 7f587e80f640 1 --2- 172.21.15.98:0/3349706165 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f587806bc90 0x7f587806c120 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.491+0000 7f587e80f640 1 -- 172.21.15.98:0/3349706165 >> 172.21.15.98:0/3349706165 conn(0x7f58780b2010 msgr2=0x7f58780b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:33.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.491+0000 7f587e80f640 1 -- 172.21.15.98:0/3349706165 shutdown_connections 2024-09-19T08:45:33.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.491+0000 7f587e80f640 1 -- 172.21.15.98:0/3349706165 wait complete. 2024-09-19T08:45:33.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.495+0000 7f587e80f640 1 Processor -- start 2024-09-19T08:45:33.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.495+0000 7f587e80f640 1 -- start start 2024-09-19T08:45:33.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.495+0000 7f587e80f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f587806bc90 0x7f587815f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:33.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.495+0000 7f587e80f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58780ce840 0x7f5878151710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:33.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.495+0000 7f587e80f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5878158fd0 0x7f587815d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:33.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.495+0000 7f587e80f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f587815c920 con 0x7f587806bc90 2024-09-19T08:45:33.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.495+0000 7f587e80f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f587815c7a0 con 0x7f58780ce840 2024-09-19T08:45:33.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.495+0000 7f587e80f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5878169120 con 0x7f5878158fd0 2024-09-19T08:45:33.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.495+0000 7f5877fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f587806bc90 0x7f587815f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:33.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.495+0000 7f5877fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f587806bc90 0x7f587815f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35614/0 (socket says 172.21.15.98:35614) 2024-09-19T08:45:33.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.495+0000 7f5877fff640 1 -- 172.21.15.98:0/3858379935 learned_addr learned my addr 172.21.15.98:0/3858379935 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:33.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.495+0000 7f58777fe640 1 --2- 172.21.15.98:0/3858379935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58780ce840 0x7f5878151710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:33.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.495+0000 7f587cdac640 1 --2- 172.21.15.98:0/3858379935 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5878158fd0 0x7f587815d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:33.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.495+0000 7f5877fff640 1 -- 172.21.15.98:0/3858379935 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5878158fd0 msgr2=0x7f587815d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:33.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.495+0000 7f5877fff640 1 --2- 172.21.15.98:0/3858379935 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5878158fd0 0x7f587815d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.495+0000 7f5877fff640 1 -- 172.21.15.98:0/3858379935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58780ce840 msgr2=0x7f5878151710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:33.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.495+0000 7f5877fff640 1 --2- 172.21.15.98:0/3858379935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58780ce840 0x7f5878151710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.495+0000 7f5877fff640 1 -- 172.21.15.98:0/3858379935 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58781520f0 con 0x7f587806bc90 2024-09-19T08:45:33.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.495+0000 7f587cdac640 1 --2- 172.21.15.98:0/3858379935 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5878158fd0 0x7f587815d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:33.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.495+0000 7f5877fff640 1 --2- 172.21.15.98:0/3858379935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f587806bc90 0x7f587815f7e0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f586800c370 tx=0x7f586800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:33.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.495+0000 7f5874ff9640 1 -- 172.21.15.98:0/3858379935 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58680052d0 con 0x7f587806bc90 2024-09-19T08:45:33.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.495+0000 7f5874ff9640 1 -- 172.21.15.98:0/3858379935 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5868005470 con 0x7f587806bc90 2024-09-19T08:45:33.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.495+0000 7f587e80f640 1 -- 172.21.15.98:0/3858379935 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58781523d0 con 0x7f587806bc90 2024-09-19T08:45:33.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.495+0000 7f5874ff9640 1 -- 172.21.15.98:0/3858379935 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5868007260 con 0x7f587806bc90 2024-09-19T08:45:33.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.495+0000 7f587e80f640 1 -- 172.21.15.98:0/3858379935 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5878152990 con 0x7f587806bc90 2024-09-19T08:45:33.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f5874ff9640 1 -- 172.21.15.98:0/3858379935 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f5868007400 con 0x7f587806bc90 2024-09-19T08:45:33.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f5874ff9640 1 --2- 172.21.15.98:0/3858379935 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5864067e60 0x7f586406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:33.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f5874ff9640 1 -- 172.21.15.98:0/3858379935 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7f5868085bb0 con 0x7f587806bc90 2024-09-19T08:45:33.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f58777fe640 1 --2- 172.21.15.98:0/3858379935 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5864067e60 0x7f586406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:33.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f58818cb000 1 Processor -- start 2024-09-19T08:45:33.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f58818cb000 1 -- start start 2024-09-19T08:45:33.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f58777fe640 1 --2- 172.21.15.98:0/3858379935 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5864067e60 0x7f586406a320 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f586c0ad000 tx=0x7f586c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:33.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f58818cb000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55b0f25e5d50 0x55b0f25e6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:33.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f58818cb000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0f25e6670 0x55b0f259bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:33.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f58818cb000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b0f259c4a0 0x55b0f259c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:33.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f58818cb000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b0f2481b60 con 0x55b0f25e6670 2024-09-19T08:45:33.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f58818cb000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b0f23e8c70 con 0x55b0f259c4a0 2024-09-19T08:45:33.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f58818cb000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x55b0f238eed0 con 0x55b0f25e5d50 2024-09-19T08:45:33.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f58777fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0f25e6670 0x55b0f259bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:33.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f58777fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0f25e6670 0x55b0f259bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35620/0 (socket says 172.21.15.98:35620) 2024-09-19T08:45:33.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f58777fe640 1 -- 172.21.15.98:0/653418274 learned_addr learned my addr 172.21.15.98:0/653418274 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:33.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f5877fff640 1 --2- 172.21.15.98:0/653418274 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55b0f25e5d50 0x55b0f25e6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:33.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f58777fe640 1 -- 172.21.15.98:0/653418274 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55b0f25e5d50 msgr2=0x55b0f25e6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:33.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f587cdac640 1 --2- 172.21.15.98:0/653418274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b0f259c4a0 0x55b0f259c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:33.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f58777fe640 1 --2- 172.21.15.98:0/653418274 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55b0f25e5d50 0x55b0f25e6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f58777fe640 1 -- 172.21.15.98:0/653418274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b0f259c4a0 msgr2=0x55b0f259c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:33.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f58777fe640 1 --2- 172.21.15.98:0/653418274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b0f259c4a0 0x55b0f259c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f58777fe640 1 -- 172.21.15.98:0/653418274 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0f23a0e50 con 0x55b0f25e6670 2024-09-19T08:45:33.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f5877fff640 1 --2- 172.21.15.98:0/653418274 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55b0f25e5d50 0x55b0f25e6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:33.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f58777fe640 1 --2- 172.21.15.98:0/653418274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0f25e6670 0x55b0f259bf60 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f586c01c4d0 tx=0x7f586c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=949f1208dc383598 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:33.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f585dffb640 1 -- 172.21.15.98:0/653418274 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c0c6070 con 0x55b0f25e6670 2024-09-19T08:45:33.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f585dffb640 1 -- 172.21.15.98:0/653418274 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f586c0abe40 con 0x55b0f25e6670 2024-09-19T08:45:33.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f585dffb640 1 -- 172.21.15.98:0/653418274 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c0b0040 con 0x55b0f25e6670 2024-09-19T08:45:33.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f58818cb000 1 -- 172.21.15.98:0/653418274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0f25e6670 msgr2=0x55b0f259bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:33.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f58818cb000 1 --2- 172.21.15.98:0/653418274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0f25e6670 0x55b0f259bf60 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f586c01c4d0 tx=0x7f586c004740 comp rx=0 tx=0).stop 2024-09-19T08:45:33.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f58818cb000 1 -- 172.21.15.98:0/653418274 shutdown_connections 2024-09-19T08:45:33.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f58818cb000 1 --2- 172.21.15.98:0/653418274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b0f259c4a0 0x55b0f259c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f58818cb000 1 --2- 172.21.15.98:0/653418274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0f25e6670 0x55b0f259bf60 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f58818cb000 1 --2- 172.21.15.98:0/653418274 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55b0f25e5d50 0x55b0f25e6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f58818cb000 1 -- 172.21.15.98:0/653418274 >> 172.21.15.98:0/653418274 conn(0x55b0f25a0600 msgr2=0x55b0f2598630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:33.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f58818cb000 1 -- 172.21.15.98:0/653418274 shutdown_connections 2024-09-19T08:45:33.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.499+0000 7f58818cb000 1 -- 172.21.15.98:0/653418274 wait complete. 2024-09-19T08:45:33.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f58818cb000 1 -- start start 2024-09-19T08:45:33.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f58818cb000 1 Processor -- start 2024-09-19T08:45:33.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f58818cb000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0f25e5d50 0x55b0f2599290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:33.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f58818cb000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b0f25e6670 0x55b0f25997d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:33.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f58818cb000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55b0f25d2210 0x55b0f25d25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:33.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f58818cb000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b0f244ab00 con 0x55b0f25e5d50 2024-09-19T08:45:33.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f58818cb000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b0f23c3360 con 0x55b0f25e6670 2024-09-19T08:45:33.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f58818cb000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x55b0f23c02d0 con 0x55b0f25d2210 2024-09-19T08:45:33.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f58818cb000 10 client.0 _ll_register_callbacks cb 0x55b0f2585820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:45:33.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f5877fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0f25e5d50 0x55b0f2599290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:33.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f5877fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0f25e5d50 0x55b0f2599290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35632/0 (socket says 172.21.15.98:35632) 2024-09-19T08:45:33.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f5877fff640 1 -- 172.21.15.98:0/4123728183 learned_addr learned my addr 172.21.15.98:0/4123728183 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:33.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f587cdac640 1 --2- 172.21.15.98:0/4123728183 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55b0f25d2210 0x55b0f25d25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:33.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f58777fe640 1 --2- 172.21.15.98:0/4123728183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b0f25e6670 0x55b0f25997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:33.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f5877fff640 1 -- 172.21.15.98:0/4123728183 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55b0f25d2210 msgr2=0x55b0f25d25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:33.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f5877fff640 1 --2- 172.21.15.98:0/4123728183 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55b0f25d2210 0x55b0f25d25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f5877fff640 1 -- 172.21.15.98:0/4123728183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b0f25e6670 msgr2=0x55b0f25997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:33.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f5877fff640 1 --2- 172.21.15.98:0/4123728183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b0f25e6670 0x55b0f25997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f5877fff640 1 -- 172.21.15.98:0/4123728183 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0f238f120 con 0x55b0f25e5d50 2024-09-19T08:45:33.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f5877fff640 1 --2- 172.21.15.98:0/4123728183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0f25e5d50 0x55b0f2599290 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f586800cd30 tx=0x7f5868055f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a0a970012812e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:33.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f5856ffd640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-19T08:45:33.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f5856ffd640 1 -- 172.21.15.98:0/4123728183 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5868054db0 con 0x55b0f25e5d50 2024-09-19T08:45:33.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f5856ffd640 1 -- 172.21.15.98:0/4123728183 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7f58680514b0 con 0x55b0f25e5d50 2024-09-19T08:45:33.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f58818cb000 1 -- 172.21.15.98:0/4123728183 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b0f2382c00 con 0x55b0f25e5d50 2024-09-19T08:45:33.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f5856ffd640 1 -- 172.21.15.98:0/4123728183 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f586805e340 con 0x55b0f25e5d50 2024-09-19T08:45:33.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f58818cb000 10 client.8034 fetch_fsmap learned FSMap version 74 2024-09-19T08:45:33.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f58818cb000 1 -- 172.21.15.98:0/4123728183 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55b0f2388450 con 0x55b0f25e5d50 2024-09-19T08:45:33.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f5856ffd640 1 -- 172.21.15.98:0/4123728183 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55b0f2382c00 con 0x55b0f25e5d50 2024-09-19T08:45:33.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f5856ffd640 1 -- 172.21.15.98:0/4123728183 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 74) ==== 3502+0+0 (secure 0 0 0) 0x7f58680516b0 con 0x55b0f25e5d50 2024-09-19T08:45:33.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f58818cb000 10 client.8034 fetch_fsmap finished waiting for FSMap version 74 2024-09-19T08:45:33.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f58818cb000 10 client.8034 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T08:45:33.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f58818cb000 20 client.8034 populate_metadata read hostname 'smithi098' 2024-09-19T08:45:33.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f58818cb000 1 --2- 172.21.15.98:0/4123728183 >> [v2:172.21.15.139:6833/4255670104,v1:172.21.15.139:6835/4255670104] conn(0x55b0f25f44b0 0x55b0f25f68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:33.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f58818cb000 4 client.8034 mds_command: new command op to 7914 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:45:33.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f58818cb000 1 -- 172.21.15.98:0/4123728183 --> [v2:172.21.15.139:6833/4255670104,v1:172.21.15.139:6835/4255670104] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0f23f63a0 con 0x55b0f25f44b0 2024-09-19T08:45:33.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f58777fe640 1 --2- 172.21.15.98:0/4123728183 >> [v2:172.21.15.139:6833/4255670104,v1:172.21.15.139:6835/4255670104] conn(0x55b0f25f44b0 0x55b0f25f68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:33.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f58777fe640 1 --2- 172.21.15.98:0/4123728183 >> [v2:172.21.15.139:6833/4255670104,v1:172.21.15.139:6835/4255670104] conn(0x55b0f25f44b0 0x55b0f25f68a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a232c43b5d199d9e server_cookie=554bef4183630b86 in_seq=0 out_seq=0 2024-09-19T08:45:33.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.503+0000 7f5856ffd640 10 client.8034 ms_handle_connect on v2:172.21.15.139:6833/4255670104 2024-09-19T08:45:33.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.507+0000 7f5856ffd640 1 -- 172.21.15.98:0/4123728183 <== mds.0 v2:172.21.15.139:6833/4255670104 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b0f23f63a0 con 0x55b0f25f44b0 2024-09-19T08:45:33.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.507+0000 7f5856ffd640 10 client.8034 handle_command_reply: tid=0 2024-09-19T08:45:33.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.507+0000 7f58818cb000 1 client.8034 shutdown 2024-09-19T08:45:33.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.507+0000 7f58818cb000 1 -- 172.21.15.98:0/4123728183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0f25e5d50 msgr2=0x55b0f2599290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:33.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.507+0000 7f58818cb000 1 --2- 172.21.15.98:0/4123728183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0f25e5d50 0x55b0f2599290 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f586800cd30 tx=0x7f5868055f20 comp rx=0 tx=0).stop 2024-09-19T08:45:33.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.507+0000 7f58818cb000 1 -- 172.21.15.98:0/4123728183 shutdown_connections 2024-09-19T08:45:33.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.507+0000 7f58818cb000 1 --2- 172.21.15.98:0/4123728183 >> [v2:172.21.15.139:6833/4255670104,v1:172.21.15.139:6835/4255670104] conn(0x55b0f25f44b0 0x55b0f25f68a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.507+0000 7f58818cb000 1 --2- 172.21.15.98:0/4123728183 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55b0f25d2210 0x55b0f25d25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.507+0000 7f58818cb000 1 --2- 172.21.15.98:0/4123728183 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b0f25e6670 0x55b0f25997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.507+0000 7f58818cb000 1 --2- 172.21.15.98:0/4123728183 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0f25e5d50 0x55b0f2599290 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.507+0000 7f5856ffd640 0 client.8034 ms_handle_reset on v2:172.21.15.139:6833/4255670104 2024-09-19T08:45:33.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.507+0000 7f58818cb000 1 -- 172.21.15.98:0/4123728183 >> 172.21.15.98:0/4123728183 conn(0x55b0f259c4a0 msgr2=0x55b0f259f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:33.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.507+0000 7f58818cb000 1 -- 172.21.15.98:0/4123728183 shutdown_connections 2024-09-19T08:45:33.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.507+0000 7f58818cb000 1 -- 172.21.15.98:0/4123728183 wait complete. 2024-09-19T08:45:33.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.507+0000 7f58818cb000 20 client.8034 trim_cache size 0 max 16384 2024-09-19T08:45:33.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.523+0000 7f58818cb000 1 Processor -- start 2024-09-19T08:45:33.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.523+0000 7f58818cb000 1 -- start start 2024-09-19T08:45:33.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.523+0000 7f58818cb000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b0f259c4a0 0x55b0f26577e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:33.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.523+0000 7f58818cb000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0f25d2210 0x55b0f2657d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:33.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.523+0000 7f58818cb000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55b0f25e5d50 0x55b0f2658260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:33.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.523+0000 7f58818cb000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5878073b20 con 0x55b0f25d2210 2024-09-19T08:45:33.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.523+0000 7f58818cb000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f587806dc90 con 0x55b0f259c4a0 2024-09-19T08:45:33.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.523+0000 7f58818cb000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f58780734d0 con 0x55b0f25e5d50 2024-09-19T08:45:33.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.523+0000 7f58777fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0f25d2210 0x55b0f2657d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:33.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.523+0000 7f58777fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0f25d2210 0x55b0f2657d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35648/0 (socket says 172.21.15.98:35648) 2024-09-19T08:45:33.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.523+0000 7f58777fe640 1 -- 172.21.15.98:0/2528790459 learned_addr learned my addr 172.21.15.98:0/2528790459 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:33.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58777fe640 1 -- 172.21.15.98:0/2528790459 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55b0f25e5d50 msgr2=0x55b0f2658260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:33.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f587cdac640 1 --2- 172.21.15.98:0/2528790459 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55b0f25e5d50 0x55b0f2658260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:33.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f5877fff640 1 --2- 172.21.15.98:0/2528790459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b0f259c4a0 0x55b0f26577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:33.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58777fe640 1 --2- 172.21.15.98:0/2528790459 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55b0f25e5d50 0x55b0f2658260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58777fe640 1 -- 172.21.15.98:0/2528790459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b0f259c4a0 msgr2=0x55b0f26577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:33.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58777fe640 1 --2- 172.21.15.98:0/2528790459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b0f259c4a0 0x55b0f26577e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58777fe640 1 -- 172.21.15.98:0/2528790459 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0f236ffd0 con 0x55b0f25d2210 2024-09-19T08:45:33.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58777fe640 1 --2- 172.21.15.98:0/2528790459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0f25d2210 0x55b0f2657d20 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f586c01c4d0 tx=0x7f586c009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c4c1c26606dfa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:33.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f585d7fa640 1 -- 172.21.15.98:0/2528790459 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c0d0040 con 0x55b0f25d2210 2024-09-19T08:45:33.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f585d7fa640 1 -- 172.21.15.98:0/2528790459 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f586c003620 con 0x55b0f25d2210 2024-09-19T08:45:33.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f585d7fa640 1 -- 172.21.15.98:0/2528790459 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c0c6040 con 0x55b0f25d2210 2024-09-19T08:45:33.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58818cb000 1 -- 172.21.15.98:0/2528790459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0f25d2210 msgr2=0x55b0f2657d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:33.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58818cb000 1 --2- 172.21.15.98:0/2528790459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0f25d2210 0x55b0f2657d20 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f586c01c4d0 tx=0x7f586c009240 comp rx=0 tx=0).stop 2024-09-19T08:45:33.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58818cb000 1 -- 172.21.15.98:0/2528790459 shutdown_connections 2024-09-19T08:45:33.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58818cb000 1 --2- 172.21.15.98:0/2528790459 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55b0f25e5d50 0x55b0f2658260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58818cb000 1 --2- 172.21.15.98:0/2528790459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0f25d2210 0x55b0f2657d20 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58818cb000 1 --2- 172.21.15.98:0/2528790459 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b0f259c4a0 0x55b0f26577e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58818cb000 1 -- 172.21.15.98:0/2528790459 >> 172.21.15.98:0/2528790459 conn(0x55b0f25d19b0 msgr2=0x55b0f259f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:33.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58818cb000 1 -- 172.21.15.98:0/2528790459 shutdown_connections 2024-09-19T08:45:33.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58818cb000 1 -- 172.21.15.98:0/2528790459 wait complete. 2024-09-19T08:45:33.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58818cb000 1 -- start start 2024-09-19T08:45:33.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58818cb000 1 Processor -- start 2024-09-19T08:45:33.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58818cb000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b0f259c4a0 0x55b0f265b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:33.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58818cb000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0f25d2210 0x55b0f265bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:33.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58818cb000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55b0f25e5d50 0x55b0f265c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:33.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58818cb000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55b0f23e7f80 con 0x55b0f25d2210 2024-09-19T08:45:33.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58818cb000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b0f23a0ca0 con 0x55b0f259c4a0 2024-09-19T08:45:33.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58818cb000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x55b0f23ed2a0 con 0x55b0f25e5d50 2024-09-19T08:45:33.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58818cb000 10 client.0 _ll_register_callbacks cb 0x7f586805c0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:45:33.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58777fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0f25d2210 0x55b0f265bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:33.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58777fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0f25d2210 0x55b0f265bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35650/0 (socket says 172.21.15.98:35650) 2024-09-19T08:45:33.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58777fe640 1 -- 172.21.15.98:0/2630157266 learned_addr learned my addr 172.21.15.98:0/2630157266 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:33.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58777fe640 1 -- 172.21.15.98:0/2630157266 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55b0f25e5d50 msgr2=0x55b0f265c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:33.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58777fe640 1 --2- 172.21.15.98:0/2630157266 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55b0f25e5d50 0x55b0f265c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58777fe640 1 -- 172.21.15.98:0/2630157266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b0f259c4a0 msgr2=0x55b0f265b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:33.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58777fe640 1 --2- 172.21.15.98:0/2630157266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b0f259c4a0 0x55b0f265b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58777fe640 1 -- 172.21.15.98:0/2630157266 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0f246b750 con 0x55b0f25d2210 2024-09-19T08:45:33.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58777fe640 1 --2- 172.21.15.98:0/2630157266 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0f25d2210 0x55b0f265bca0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f586c01c2c0 tx=0x7f586c01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a19d87b8d3ff57cc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:33.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f5856ffd640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-19T08:45:33.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f5856ffd640 1 -- 172.21.15.98:0/2630157266 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c0d0070 con 0x55b0f25d2210 2024-09-19T08:45:33.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f5856ffd640 1 -- 172.21.15.98:0/2630157266 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7f586c003b30 con 0x55b0f25d2210 2024-09-19T08:45:33.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58818cb000 1 -- 172.21.15.98:0/2630157266 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f586c003620 con 0x55b0f25d2210 2024-09-19T08:45:33.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f5856ffd640 1 -- 172.21.15.98:0/2630157266 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f586c0da050 con 0x55b0f25d2210 2024-09-19T08:45:33.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58818cb000 10 client.8040 fetch_fsmap learned FSMap version 74 2024-09-19T08:45:33.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58818cb000 1 -- 172.21.15.98:0/2630157266 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55b0f2376a30 con 0x55b0f25d2210 2024-09-19T08:45:33.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f5856ffd640 1 -- 172.21.15.98:0/2630157266 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f586c003620 con 0x55b0f25d2210 2024-09-19T08:45:33.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f5856ffd640 1 -- 172.21.15.98:0/2630157266 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 74) ==== 3502+0+0 (secure 0 0 0) 0x7f586c0dd020 con 0x55b0f25d2210 2024-09-19T08:45:33.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58818cb000 10 client.8040 fetch_fsmap finished waiting for FSMap version 74 2024-09-19T08:45:33.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58818cb000 10 client.8040 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T08:45:33.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58818cb000 20 client.8040 populate_metadata read hostname 'smithi098' 2024-09-19T08:45:33.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58818cb000 1 --2- 172.21.15.98:0/2630157266 >> [v2:172.21.15.139:6833/4255670104,v1:172.21.15.139:6835/4255670104] conn(0x55b0f25e6670 0x55b0f2617020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:33.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58818cb000 4 client.8040 mds_command: new command op to 7914 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T08:45:33.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.527+0000 7f58818cb000 1 -- 172.21.15.98:0/2630157266 --> [v2:172.21.15.139:6833/4255670104,v1:172.21.15.139:6835/4255670104] -- command(tid 0: {"prefix": "status"}) -- 0x55b0f23f06f0 con 0x55b0f25e6670 2024-09-19T08:45:33.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.531+0000 7f5877fff640 1 --2- 172.21.15.98:0/2630157266 >> [v2:172.21.15.139:6833/4255670104,v1:172.21.15.139:6835/4255670104] conn(0x55b0f25e6670 0x55b0f2617020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:33.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.531+0000 7f5856ffd640 10 client.8040 ms_handle_connect on v2:172.21.15.139:6833/4255670104 2024-09-19T08:45:33.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.531+0000 7f5877fff640 1 --2- 172.21.15.98:0/2630157266 >> [v2:172.21.15.139:6833/4255670104,v1:172.21.15.139:6835/4255670104] conn(0x55b0f25e6670 0x55b0f2617020 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bf422b8a25829cf server_cookie=2bd3036825bde450 in_seq=0 out_seq=0 2024-09-19T08:45:33.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.531+0000 7f5856ffd640 1 -- 172.21.15.98:0/2630157266 <== mds.0 v2:172.21.15.139:6833/4255670104 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55b0f23f06f0 con 0x55b0f25e6670 2024-09-19T08:45:33.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.531+0000 7f5856ffd640 10 client.8040 handle_command_reply: tid=0 2024-09-19T08:45:33.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.531+0000 7f58818cb000 1 client.8040 shutdown 2024-09-19T08:45:33.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.531+0000 7f58818cb000 1 -- 172.21.15.98:0/2630157266 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0f25d2210 msgr2=0x55b0f265bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:33.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.531+0000 7f58818cb000 1 --2- 172.21.15.98:0/2630157266 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0f25d2210 0x55b0f265bca0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f586c01c2c0 tx=0x7f586c01acd0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.531+0000 7f58818cb000 1 -- 172.21.15.98:0/2630157266 shutdown_connections 2024-09-19T08:45:33.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.531+0000 7f58818cb000 1 --2- 172.21.15.98:0/2630157266 >> [v2:172.21.15.139:6833/4255670104,v1:172.21.15.139:6835/4255670104] conn(0x55b0f25e6670 0x55b0f2617020 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.531+0000 7f58818cb000 1 --2- 172.21.15.98:0/2630157266 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55b0f25e5d50 0x55b0f265c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.531+0000 7f58818cb000 1 --2- 172.21.15.98:0/2630157266 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55b0f25d2210 0x55b0f265bca0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.531+0000 7f58818cb000 1 --2- 172.21.15.98:0/2630157266 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b0f259c4a0 0x55b0f265b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.531+0000 7f58818cb000 1 -- 172.21.15.98:0/2630157266 >> 172.21.15.98:0/2630157266 conn(0x55b0f25d19b0 msgr2=0x55b0f25f8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:33.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.531+0000 7f5856ffd640 0 client.8040 ms_handle_reset on v2:172.21.15.139:6833/4255670104 2024-09-19T08:45:33.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.531+0000 7f58818cb000 1 -- 172.21.15.98:0/2630157266 shutdown_connections 2024-09-19T08:45:33.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.531+0000 7f58818cb000 1 -- 172.21.15.98:0/2630157266 wait complete. 2024-09-19T08:45:33.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.531+0000 7f58818cb000 20 client.8040 trim_cache size 0 max 16384 2024-09-19T08:45:33.538 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:45:33.538 INFO:teuthology.orchestra.run.smithi098.stdout: "cluster_fsid": "552bf0ee-1f4f-4644-ba7c-d0931e55a451", 2024-09-19T08:45:33.538 INFO:teuthology.orchestra.run.smithi098.stdout: "whoami": 0, 2024-09-19T08:45:33.538 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 7914, 2024-09-19T08:45:33.538 INFO:teuthology.orchestra.run.smithi098.stdout: "want_state": "up:active", 2024-09-19T08:45:33.538 INFO:teuthology.orchestra.run.smithi098.stdout: "state": "up:active", 2024-09-19T08:45:33.538 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "cephfs", 2024-09-19T08:45:33.539 INFO:teuthology.orchestra.run.smithi098.stdout: "rank_uptime": 4.7619829869999997, 2024-09-19T08:45:33.539 INFO:teuthology.orchestra.run.smithi098.stdout: "mdsmap_epoch": 74, 2024-09-19T08:45:33.539 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch": 135, 2024-09-19T08:45:33.539 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch_barrier": 135, 2024-09-19T08:45:33.539 INFO:teuthology.orchestra.run.smithi098.stdout: "uptime": 10.056205282000001 2024-09-19T08:45:33.539 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:45:33.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.531+0000 7f587e80f640 1 -- 172.21.15.98:0/3858379935 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5864067e60 msgr2=0x7f586406a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:33.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.531+0000 7f587e80f640 1 --2- 172.21.15.98:0/3858379935 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5864067e60 0x7f586406a320 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f586c0ad000 tx=0x7f586c008040 comp rx=0 tx=0).stop 2024-09-19T08:45:33.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.531+0000 7f587e80f640 1 -- 172.21.15.98:0/3858379935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f587806bc90 msgr2=0x7f587815f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:33.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.531+0000 7f587e80f640 1 --2- 172.21.15.98:0/3858379935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f587806bc90 0x7f587815f7e0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f586800c370 tx=0x7f586800c830 comp rx=0 tx=0).stop 2024-09-19T08:45:33.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.531+0000 7f587e80f640 1 -- 172.21.15.98:0/3858379935 shutdown_connections 2024-09-19T08:45:33.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.531+0000 7f587e80f640 1 --2- 172.21.15.98:0/3858379935 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5864067e60 0x7f586406a320 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.531+0000 7f587e80f640 1 --2- 172.21.15.98:0/3858379935 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5878158fd0 0x7f587815d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.531+0000 7f587e80f640 1 --2- 172.21.15.98:0/3858379935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58780ce840 0x7f5878151710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.531+0000 7f587e80f640 1 --2- 172.21.15.98:0/3858379935 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f587806bc90 0x7f587815f7e0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.531+0000 7f587e80f640 1 -- 172.21.15.98:0/3858379935 >> 172.21.15.98:0/3858379935 conn(0x7f58780b2010 msgr2=0x7f5878156780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:33.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.531+0000 7f587e80f640 1 -- 172.21.15.98:0/3858379935 shutdown_connections 2024-09-19T08:45:33.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.531+0000 7f587e80f640 1 -- 172.21.15.98:0/3858379935 wait complete. 2024-09-19T08:45:33.549 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T08:45:33.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.651+0000 7f6a6228d640 1 Processor -- start 2024-09-19T08:45:33.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.651+0000 7f6a6228d640 1 -- start start 2024-09-19T08:45:33.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.655+0000 7f6a6228d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6a5c15e420 0x7f6a5c15f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:33.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.655+0000 7f6a6228d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a5c15e800 0x7f6a5c06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:33.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.655+0000 7f6a6228d640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a5c06c6e0 con 0x7f6a5c06c1d0 2024-09-19T08:45:33.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.655+0000 7f6a6228d640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f6a5c06c9c0 con 0x7f6a5c15e420 2024-09-19T08:45:33.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.655+0000 7f6a6228d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6a5c15ed60 con 0x7f6a5c15e800 2024-09-19T08:45:33.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.655+0000 7f6a5bfff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6a5c15e420 0x7f6a5c15f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:33.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.655+0000 7f6a5bfff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6a5c15e420 0x7f6a5c15f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35652/0 (socket says 172.21.15.98:35652) 2024-09-19T08:45:33.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.655+0000 7f6a6082a640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f6a5c06c1d0 0x7f6a5c0b78b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:44014/0 (socket says 172.21.15.98:44014) 2024-09-19T08:45:33.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.655+0000 7f6a5bfff640 1 -- 172.21.15.98:0/3908036299 learned_addr learned my addr 172.21.15.98:0/3908036299 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:33.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.655+0000 7f6a537fe640 1 --2- 172.21.15.98:0/3908036299 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a5c15e800 0x7f6a5c06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:33.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.655+0000 7f6a5bfff640 1 -- 172.21.15.98:0/3908036299 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a5c15e800 msgr2=0x7f6a5c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:33.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.655+0000 7f6a5bfff640 1 --2- 172.21.15.98:0/3908036299 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a5c15e800 0x7f6a5c06bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.655+0000 7f6a5bfff640 1 -- 172.21.15.98:0/3908036299 >> v1:172.21.15.98:6789/0 conn(0x7f6a5c06c1d0 legacy=0x7f6a5c0b78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:33.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.655+0000 7f6a5bfff640 1 -- 172.21.15.98:0/3908036299 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a5c15f040 con 0x7f6a5c15e420 2024-09-19T08:45:33.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.655+0000 7f6a5bfff640 1 --2- 172.21.15.98:0/3908036299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a5c15e420 0x7f6a5c15f560 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009870 tx=0x7f6a4c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=457c045a8c16ab2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:33.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.655+0000 7f6a5b7fe640 1 -- 172.21.15.98:0/3908036299 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c0a7070 con 0x7f6a5c15e420 2024-09-19T08:45:33.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.655+0000 7f6a5b7fe640 1 -- 172.21.15.98:0/3908036299 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a4c0b0070 con 0x7f6a5c15e420 2024-09-19T08:45:33.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.655+0000 7f6a5b7fe640 1 -- 172.21.15.98:0/3908036299 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c0ab890 con 0x7f6a5c15e420 2024-09-19T08:45:33.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.659+0000 7f6a6228d640 1 -- 172.21.15.98:0/3908036299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a5c15e420 msgr2=0x7f6a5c15f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:33.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.659+0000 7f6a6228d640 1 --2- 172.21.15.98:0/3908036299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a5c15e420 0x7f6a5c15f560 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009870 tx=0x7f6a4c01c560 comp rx=0 tx=0).stop 2024-09-19T08:45:33.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.659+0000 7f6a6228d640 1 -- 172.21.15.98:0/3908036299 shutdown_connections 2024-09-19T08:45:33.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.659+0000 7f6a6228d640 1 --2- 172.21.15.98:0/3908036299 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a5c15e800 0x7f6a5c06bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.659+0000 7f6a6228d640 1 --2- 172.21.15.98:0/3908036299 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a5c15e420 0x7f6a5c15f560 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.659+0000 7f6a6228d640 1 -- 172.21.15.98:0/3908036299 >> 172.21.15.98:0/3908036299 conn(0x7f6a5c0b40a0 msgr2=0x7f6a5c0b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:33.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.659+0000 7f6a6228d640 1 -- 172.21.15.98:0/3908036299 shutdown_connections 2024-09-19T08:45:33.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.659+0000 7f6a6228d640 1 -- 172.21.15.98:0/3908036299 wait complete. 2024-09-19T08:45:33.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.659+0000 7f6a6228d640 1 Processor -- start 2024-09-19T08:45:33.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.659+0000 7f6a6228d640 1 -- start start 2024-09-19T08:45:33.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.659+0000 7f6a6228d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a5c06c1d0 0x7f6a5c143700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:33.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.659+0000 7f6a6228d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a5c15e420 0x7f6a5c15cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:33.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.659+0000 7f6a6228d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6a5c15e800 0x7f6a5c15d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:33.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.659+0000 7f6a6228d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5c162880 con 0x7f6a5c15e420 2024-09-19T08:45:33.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.659+0000 7f6a6228d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5c162a00 con 0x7f6a5c15e800 2024-09-19T08:45:33.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.659+0000 7f6a6228d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6a5c162b80 con 0x7f6a5c06c1d0 2024-09-19T08:45:33.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.659+0000 7f6a537fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a5c15e420 0x7f6a5c15cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:33.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.659+0000 7f6a537fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a5c15e420 0x7f6a5c15cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35666/0 (socket says 172.21.15.98:35666) 2024-09-19T08:45:33.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.659+0000 7f6a537fe640 1 -- 172.21.15.98:0/3792660260 learned_addr learned my addr 172.21.15.98:0/3792660260 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:33.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.659+0000 7f6a5bfff640 1 --2- 172.21.15.98:0/3792660260 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a5c06c1d0 0x7f6a5c143700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:33.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.659+0000 7f6a537fe640 1 -- 172.21.15.98:0/3792660260 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a5c06c1d0 msgr2=0x7f6a5c143700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:33.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.659+0000 7f6a537fe640 1 --2- 172.21.15.98:0/3792660260 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a5c06c1d0 0x7f6a5c143700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.659+0000 7f6a537fe640 1 -- 172.21.15.98:0/3792660260 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6a5c15e800 msgr2=0x7f6a5c15d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:33.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.659+0000 7f6a6082a640 1 --2- 172.21.15.98:0/3792660260 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6a5c15e800 0x7f6a5c15d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:33.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.659+0000 7f6a537fe640 1 --2- 172.21.15.98:0/3792660260 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6a5c15e800 0x7f6a5c15d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.659+0000 7f6a537fe640 1 -- 172.21.15.98:0/3792660260 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a5c144b20 con 0x7f6a5c15e420 2024-09-19T08:45:33.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.659+0000 7f6a537fe640 1 --2- 172.21.15.98:0/3792660260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a5c15e420 0x7f6a5c15cff0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f6a4400c7f0 tx=0x7f6a4400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:33.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.659+0000 7f6a597fa640 1 -- 172.21.15.98:0/3792660260 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a44015070 con 0x7f6a5c15e420 2024-09-19T08:45:33.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.659+0000 7f6a597fa640 1 -- 172.21.15.98:0/3792660260 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a44004240 con 0x7f6a5c15e420 2024-09-19T08:45:33.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.659+0000 7f6a597fa640 1 -- 172.21.15.98:0/3792660260 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a44003620 con 0x7f6a5c15e420 2024-09-19T08:45:33.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.659+0000 7f6a6228d640 1 -- 172.21.15.98:0/3792660260 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a5c144e00 con 0x7f6a5c15e420 2024-09-19T08:45:33.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.659+0000 7f6a6228d640 1 -- 172.21.15.98:0/3792660260 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a5c1453c0 con 0x7f6a5c15e420 2024-09-19T08:45:33.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.663+0000 7f6a6228d640 1 -- 172.21.15.98:0/3792660260 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a5c15f560 con 0x7f6a5c15e420 2024-09-19T08:45:33.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.663+0000 7f6a597fa640 1 -- 172.21.15.98:0/3792660260 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f6a440039d0 con 0x7f6a5c15e420 2024-09-19T08:45:33.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.663+0000 7f6a597fa640 1 --2- 172.21.15.98:0/3792660260 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6a3c067e60 0x7f6a3c06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:33.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.663+0000 7f6a597fa640 1 -- 172.21.15.98:0/3792660260 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7f6a44086920 con 0x7f6a5c15e420 2024-09-19T08:45:33.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.663+0000 7f6a5bfff640 1 --2- 172.21.15.98:0/3792660260 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6a3c067e60 0x7f6a3c06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:33.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.663+0000 7f6a5bfff640 1 --2- 172.21.15.98:0/3792660260 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6a3c067e60 0x7f6a3c06a320 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c0af000 tx=0x7f6a4c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:33.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.667+0000 7f6a597fa640 1 -- 172.21.15.98:0/3792660260 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6a44056ac0 con 0x7f6a5c15e420 2024-09-19T08:45:33.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.851+0000 7f6a6228d640 1 -- 172.21.15.98:0/3792660260 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6a5c06dc80 con 0x7f6a5c15e420 2024-09-19T08:45:33.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.855+0000 7f6a597fa640 1 -- 172.21.15.98:0/3792660260 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v74) ==== 53+0+73 (secure 0 0 0) 0x7f6a440563c0 con 0x7f6a5c15e420 2024-09-19T08:45:33.856 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T08:45:33.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.855+0000 7f6a6228d640 1 -- 172.21.15.98:0/3792660260 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6a3c067e60 msgr2=0x7f6a3c06a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:33.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.855+0000 7f6a6228d640 1 --2- 172.21.15.98:0/3792660260 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6a3c067e60 0x7f6a3c06a320 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c0af000 tx=0x7f6a4c008040 comp rx=0 tx=0).stop 2024-09-19T08:45:33.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.855+0000 7f6a6228d640 1 -- 172.21.15.98:0/3792660260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a5c15e420 msgr2=0x7f6a5c15cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:33.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.855+0000 7f6a6228d640 1 --2- 172.21.15.98:0/3792660260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a5c15e420 0x7f6a5c15cff0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f6a4400c7f0 tx=0x7f6a4400ccb0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.855+0000 7f6a6228d640 1 -- 172.21.15.98:0/3792660260 shutdown_connections 2024-09-19T08:45:33.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.855+0000 7f6a6228d640 1 --2- 172.21.15.98:0/3792660260 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6a3c067e60 0x7f6a3c06a320 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.855+0000 7f6a6228d640 1 --2- 172.21.15.98:0/3792660260 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6a5c15e800 0x7f6a5c15d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.855+0000 7f6a6228d640 1 --2- 172.21.15.98:0/3792660260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6a5c15e420 0x7f6a5c15cff0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.855+0000 7f6a6228d640 1 --2- 172.21.15.98:0/3792660260 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6a5c06c1d0 0x7f6a5c143700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:33.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.855+0000 7f6a6228d640 1 -- 172.21.15.98:0/3792660260 >> 172.21.15.98:0/3792660260 conn(0x7f6a5c0b40a0 msgr2=0x7f6a5c0b68d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:33.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.855+0000 7f6a6228d640 1 -- 172.21.15.98:0/3792660260 shutdown_connections 2024-09-19T08:45:33.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:33.855+0000 7f6a6228d640 1 -- 172.21.15.98:0/3792660260 wait complete. 2024-09-19T08:45:33.873 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T08:45:33.873 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T08:45:33.873 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi098.front.sepia.ceph.com 2024-09-19T08:45:33.874 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T08:45:33.874 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T08:45:33.874 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T08:45:33.874 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T08:45:33.874 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T08:45:33.874 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:45:33.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:33.874 DEBUG:teuthology.orchestra.run.smithi098:> ip addr 2024-09-19T08:45:33.880 INFO:teuthology.orchestra.run.smithi098.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T08:45:33.880 INFO:teuthology.orchestra.run.smithi098.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T08:45:33.880 INFO:teuthology.orchestra.run.smithi098.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T08:45:33.880 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:45:33.881 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 ::1/128 scope host 2024-09-19T08:45:33.881 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:45:33.881 INFO:teuthology.orchestra.run.smithi098.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:45:33.881 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b8 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:45:33.881 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f0 2024-09-19T08:45:33.881 INFO:teuthology.orchestra.run.smithi098.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:45:33.881 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b9 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:45:33.881 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f1 2024-09-19T08:45:33.881 INFO:teuthology.orchestra.run.smithi098.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:45:33.881 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6a brd ff:ff:ff:ff:ff:ff 2024-09-19T08:45:33.881 INFO:teuthology.orchestra.run.smithi098.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T08:45:33.881 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6b brd ff:ff:ff:ff:ff:ff 2024-09-19T08:45:33.881 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.21.15.98/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T08:45:33.882 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft 42074sec preferred_lft 42074sec 2024-09-19T08:45:33.882 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::ec4:7aff:fe8f:d26b/64 scope link 2024-09-19T08:45:33.882 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:45:33.882 INFO:teuthology.orchestra.run.smithi098.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T08:45:33.882 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 02:42:98:dd:32:2d brd ff:ff:ff:ff:ff:ff 2024-09-19T08:45:33.882 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T08:45:33.882 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:45:33.882 INFO:teuthology.orchestra.run.smithi098.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T08:45:33.882 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-19T08:45:33.882 INFO:teuthology.orchestra.run.smithi098.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T08:45:33.882 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:45:33.882 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-19T08:45:33.882 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:45:33.883 INFO:teuthology.orchestra.run.smithi098.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:45:33.883 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:45:33.883 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-19T08:45:33.883 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:45:33.883 INFO:teuthology.orchestra.run.smithi098.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:45:33.883 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:45:33.883 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-19T08:45:33.883 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:45:33.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:33.884 DEBUG:teuthology.orchestra.run.smithi098:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T08:45:33.934 INFO:teuthology.orchestra.run.smithi098.stdout:1 2024-09-19T08:45:33.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:33.936 DEBUG:teuthology.orchestra.run.smithi098:> route 2024-09-19T08:45:33.988 INFO:teuthology.orchestra.run.smithi098.stdout:Kernel IP routing table 2024-09-19T08:45:33.988 INFO:teuthology.orchestra.run.smithi098.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T08:45:33.988 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T08:45:33.989 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T08:45:33.989 INFO:teuthology.orchestra.run.smithi098.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T08:45:33.989 INFO:teuthology.orchestra.run.smithi098.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T08:45:33.989 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T08:45:33.989 INFO:teuthology.orchestra.run.smithi098.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:45:33.989 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:45:33.989 INFO:teuthology.orchestra.run.smithi098.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:45:33.989 INFO:teuthology.orchestra.run.smithi098.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T08:45:33.990 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:45:33.990 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-09-19T08:45:33.990 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T08:45:33.990 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T08:45:33.990 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T08:45:33.990 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-09-19T08:45:34.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:45:34.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:45:34.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:34.108 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2024-09-19T08:45:34.113 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-19T08:45:34.113 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T08:45:34.113 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:45:34.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:34.114 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set brx.0 up 2024-09-19T08:45:34.168 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-19T08:45:34.168 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:45:34.169 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T08:45:34.216 INFO:teuthology.orchestra.run.smithi098.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:45:34.217 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:45:34.217 DEBUG:teuthology.orchestra.run.smithi098:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T08:45:34.264 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:45:34.321 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:45:34.370 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:45:34.370 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:45:34.370 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:45:34.370 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:45:34.370 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:45:34.371 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:45:34.371 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:45:34.371 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:45:34.371 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:45:34.371 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:45:34.371 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:45:34.371 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:45:34.371 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:45:34.371 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:45:34.371 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:45:34.371 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:45:34.371 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:45:34.371 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:45:34.372 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:45:34.372 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:45:34.372 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:45:34.372 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:45:34.372 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:45:34.372 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:45:34.372 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:45:34.372 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:45:34.372 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:45:34.373 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:45:34.373 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:45:34.373 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:45:34.373 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:45:34.373 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:45:34.373 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:45:34.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:34.374 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:45:34.421 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T08:45:34.421 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-19T08:45:34.434 INFO:tasks.ceph.mds.b.smithi139.stderr:2024-09-19T08:45:34.434+0000 7f2d6d255640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T08:45:34.466 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:45:34.481 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:45:34.529 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:45:34.529 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:45:34.529 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:45:34.529 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:45:34.529 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:45:34.529 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:45:34.529 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:45:34.529 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:45:34.529 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:45:34.529 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:45:34.530 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:45:34.530 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:45:34.530 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:45:34.530 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:45:34.530 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:45:34.530 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:45:34.530 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:45:34.530 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:45:34.530 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:45:34.530 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:45:34.530 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:45:34.530 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:45:34.530 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:45:34.530 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:45:34.531 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:45:34.531 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:45:34.531 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:45:34.531 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:45:34.531 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:45:34.531 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:45:34.531 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:45:34.531 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:45:34.531 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:45:34.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:34.532 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:45:34.547 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:2024-09-19T08:45:34.543+0000 7f4c90427580 -1 init, newargv = 0x56013ba32fc0 newargc=15 2024-09-19T08:45:34.548 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:ceph-fuse[49695]: starting ceph client 2024-09-19T08:45:34.554 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:ceph-fuse[49695]: starting fuse 2024-09-19T08:45:35.654 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:45:35.670 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:45:35.723 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:45:35.723 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:45:35.723 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:45:35.723 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:45:35.724 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:45:35.724 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:45:35.724 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:45:35.724 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:45:35.724 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:45:35.724 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:45:35.724 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:45:35.724 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:45:35.724 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:45:35.724 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:45:35.724 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:45:35.724 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:45:35.724 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:45:35.725 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:45:35.725 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:45:35.725 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:45:35.725 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:45:35.725 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:45:35.725 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:45:35.725 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:45:35.725 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:45:35.725 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:45:35.725 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:45:35.725 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:45:35.725 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:45:35.725 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:45:35.725 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:45:35.726 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:45:35.726 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:45:35.726 INFO:teuthology.orchestra.run.smithi098.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-19T08:45:35.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:35.727 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:45:35.773 INFO:teuthology.orchestra.run.smithi098.stdout:45 2024-09-19T08:45:35.773 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-19T08:45:35.773 DEBUG:teuthology.orchestra.run.smithi098:> sudo stdin-killer -- python3 -c ' 2024-09-19T08:45:35.773 DEBUG:teuthology.orchestra.run.smithi098:> import glob 2024-09-19T08:45:35.774 DEBUG:teuthology.orchestra.run.smithi098:> import re 2024-09-19T08:45:35.774 DEBUG:teuthology.orchestra.run.smithi098:> import os 2024-09-19T08:45:35.774 DEBUG:teuthology.orchestra.run.smithi098:> import subprocess 2024-09-19T08:45:35.774 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:45:35.774 DEBUG:teuthology.orchestra.run.smithi098:> def _find_admin_socket(client_name): 2024-09-19T08:45:35.774 DEBUG:teuthology.orchestra.run.smithi098:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T08:45:35.774 DEBUG:teuthology.orchestra.run.smithi098:> files = glob.glob(asok_path) 2024-09-19T08:45:35.774 DEBUG:teuthology.orchestra.run.smithi098:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T08:45:35.774 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:45:35.774 DEBUG:teuthology.orchestra.run.smithi098:> # Given a non-glob path, it better be there 2024-09-19T08:45:35.774 DEBUG:teuthology.orchestra.run.smithi098:> if "*" not in asok_path: 2024-09-19T08:45:35.774 DEBUG:teuthology.orchestra.run.smithi098:> assert(len(files) == 1) 2024-09-19T08:45:35.774 DEBUG:teuthology.orchestra.run.smithi098:> return files[0] 2024-09-19T08:45:35.774 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:45:35.774 DEBUG:teuthology.orchestra.run.smithi098:> for f in files: 2024-09-19T08:45:35.775 DEBUG:teuthology.orchestra.run.smithi098:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T08:45:35.775 DEBUG:teuthology.orchestra.run.smithi098:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T08:45:35.775 DEBUG:teuthology.orchestra.run.smithi098:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T08:45:35.775 DEBUG:teuthology.orchestra.run.smithi098:> contents = proc_f.read() 2024-09-19T08:45:35.775 DEBUG:teuthology.orchestra.run.smithi098:> if mountpoint in contents: 2024-09-19T08:45:35.775 DEBUG:teuthology.orchestra.run.smithi098:> return f 2024-09-19T08:45:35.775 DEBUG:teuthology.orchestra.run.smithi098:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T08:45:35.775 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:45:35.775 DEBUG:teuthology.orchestra.run.smithi098:> print(_find_admin_socket("client.0")) 2024-09-19T08:45:35.775 DEBUG:teuthology.orchestra.run.smithi098:> ' 2024-09-19T08:45:35.897 INFO:teuthology.orchestra.run.smithi098.stdout:/var/run/ceph/ceph-client.0.49695.asok 2024-09-19T08:45:35.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:45:35.908 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.49695.asok 2024-09-19T08:45:35.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:35.909 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.49695.asok status 2024-09-19T08:45:36.060 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:45:36.060 INFO:teuthology.orchestra.run.smithi098.stdout: "metadata": { 2024-09-19T08:45:36.060 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T08:45:36.060 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T08:45:36.060 INFO:teuthology.orchestra.run.smithi098.stdout: "entity_id": "0", 2024-09-19T08:45:36.060 INFO:teuthology.orchestra.run.smithi098.stdout: "hostname": "smithi098", 2024-09-19T08:45:36.060 INFO:teuthology.orchestra.run.smithi098.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T08:45:36.060 INFO:teuthology.orchestra.run.smithi098.stdout: "pid": "49695", 2024-09-19T08:45:36.060 INFO:teuthology.orchestra.run.smithi098.stdout: "root": "/" 2024-09-19T08:45:36.060 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:45:36.061 INFO:teuthology.orchestra.run.smithi098.stdout: "dentry_count": 0, 2024-09-19T08:45:36.061 INFO:teuthology.orchestra.run.smithi098.stdout: "dentry_pinned_count": 0, 2024-09-19T08:45:36.061 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 8058, 2024-09-19T08:45:36.061 INFO:teuthology.orchestra.run.smithi098.stdout: "inst": { 2024-09-19T08:45:36.061 INFO:teuthology.orchestra.run.smithi098.stdout: "name": { 2024-09-19T08:45:36.061 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "client", 2024-09-19T08:45:36.061 INFO:teuthology.orchestra.run.smithi098.stdout: "num": 8058 2024-09-19T08:45:36.061 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:45:36.061 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": { 2024-09-19T08:45:36.061 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v1", 2024-09-19T08:45:36.061 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:45:36.061 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 1922868006 2024-09-19T08:45:36.061 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:45:36.062 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:45:36.062 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": { 2024-09-19T08:45:36.062 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v1", 2024-09-19T08:45:36.062 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:45:36.062 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 1922868006 2024-09-19T08:45:36.062 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:45:36.062 INFO:teuthology.orchestra.run.smithi098.stdout: "inst_str": "client.8058 192.168.0.1:0/1922868006", 2024-09-19T08:45:36.062 INFO:teuthology.orchestra.run.smithi098.stdout: "addr_str": "192.168.0.1:0/1922868006", 2024-09-19T08:45:36.062 INFO:teuthology.orchestra.run.smithi098.stdout: "inode_count": 1, 2024-09-19T08:45:36.062 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_epoch": 74, 2024-09-19T08:45:36.062 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_epoch": 135, 2024-09-19T08:45:36.062 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_epoch_barrier": 0, 2024-09-19T08:45:36.062 INFO:teuthology.orchestra.run.smithi098.stdout: "blocklisted": false, 2024-09-19T08:45:36.062 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "cephfs" 2024-09-19T08:45:36.063 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:45:36.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:36.075 DEBUG:teuthology.orchestra.run.smithi098:> stat --file-system '--printf=%T 2024-09-19T08:45:36.075 DEBUG:teuthology.orchestra.run.smithi098:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T08:45:36.081 INFO:teuthology.orchestra.run.smithi098.stdout:fuseblk 2024-09-19T08:45:36.081 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T08:45:36.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:36.081 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T08:45:36.136 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T08:45:36.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.299+0000 7fc80c775640 1 Processor -- start 2024-09-19T08:45:36.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.299+0000 7fc80c775640 1 -- start start 2024-09-19T08:45:36.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.299+0000 7fc80c775640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc804158fd0 0x7fc8040b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:36.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.299+0000 7fc80c775640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8040b4360 0x7fc80406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:36.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.299+0000 7fc80c775640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8040ccc60 con 0x7fc80406c1d0 2024-09-19T08:45:36.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.299+0000 7fc80c775640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fc8041593b0 con 0x7fc804158fd0 2024-09-19T08:45:36.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.299+0000 7fc80c775640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc80406c600 con 0x7fc8040b4360 2024-09-19T08:45:36.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.299+0000 7fc80ad12640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fc80406c1d0 0x7fc80415efc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:44020/0 (socket says 172.21.15.98:44020) 2024-09-19T08:45:36.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.299+0000 7fc80ad12640 1 -- 172.21.15.98:0/253505524 learned_addr learned my addr 172.21.15.98:0/253505524 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:36.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.299+0000 7fc80a511640 1 --2- 172.21.15.98:0/253505524 >> v2:172.21.15.139:3300/0 conn(0x7fc804158fd0 0x7fc8040b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:36.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.299+0000 7fc809d10640 1 --2- 172.21.15.98:0/253505524 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8040b4360 0x7fc80406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:36.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.299+0000 7fc80950f640 1 -- 172.21.15.98:0/253505524 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2703194490 0 0) 0x7fc8040ccc60 con 0x7fc80406c1d0 2024-09-19T08:45:36.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.299+0000 7fc80950f640 1 -- 172.21.15.98:0/253505524 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7f0003530 con 0x7fc80406c1d0 2024-09-19T08:45:36.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.299+0000 7fc80950f640 1 -- 172.21.15.98:0/253505524 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1766155312 0 0) 0x7fc7f0003530 con 0x7fc80406c1d0 2024-09-19T08:45:36.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.299+0000 7fc80950f640 1 -- 172.21.15.98:0/253505524 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8040b4360 msgr2=0x7fc80406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:36.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.299+0000 7fc80950f640 1 --2- 172.21.15.98:0/253505524 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8040b4360 0x7fc80406bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.299+0000 7fc80950f640 1 -- 172.21.15.98:0/253505524 >> v2:172.21.15.139:3300/0 conn(0x7fc804158fd0 msgr2=0x7fc8040b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:36.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.299+0000 7fc80950f640 1 --2- 172.21.15.98:0/253505524 >> v2:172.21.15.139:3300/0 conn(0x7fc804158fd0 0x7fc8040b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.299+0000 7fc80950f640 1 -- 172.21.15.98:0/253505524 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8040ccf40 con 0x7fc80406c1d0 2024-09-19T08:45:36.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.299+0000 7fc80950f640 1 -- 172.21.15.98:0/253505524 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fc7fc001a00 con 0x7fc80406c1d0 2024-09-19T08:45:36.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.299+0000 7fc80950f640 1 -- 172.21.15.98:0/253505524 >> v1:172.21.15.98:6789/0 conn(0x7fc80406c1d0 legacy=0x7fc80415efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:36.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.299+0000 7fc80950f640 1 --2- 172.21.15.98:0/253505524 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8040b4360 0x7fc7f00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:36.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.299+0000 7fc80950f640 1 --2- 172.21.15.98:0/253505524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc7f0003f40 0x7fc7f0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:36.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.299+0000 7fc80950f640 1 --2- 172.21.15.98:0/253505524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7f0004d50 0x7fc7f00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:36.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.299+0000 7fc80950f640 1 -- 172.21.15.98:0/253505524 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc8041593b0 con 0x7fc7f0003f40 2024-09-19T08:45:36.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.299+0000 7fc80950f640 1 -- 172.21.15.98:0/253505524 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc80406c600 con 0x7fc7f0004d50 2024-09-19T08:45:36.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.299+0000 7fc80950f640 1 -- 172.21.15.98:0/253505524 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc7f000ba20 con 0x7fc8040b4360 2024-09-19T08:45:36.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.299+0000 7fc809d10640 1 --2- 172.21.15.98:0/253505524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc7f0003f40 0x7fc7f0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:36.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.299+0000 7fc80a511640 1 --2- 172.21.15.98:0/253505524 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8040b4360 0x7fc7f00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:36.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.299+0000 7fc80ad12640 1 --2- 172.21.15.98:0/253505524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7f0004d50 0x7fc7f00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:36.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.299+0000 7fc809d10640 1 -- 172.21.15.98:0/253505524 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8040b4360 msgr2=0x7fc7f00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:36.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.299+0000 7fc809d10640 1 --2- 172.21.15.98:0/253505524 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8040b4360 0x7fc7f00039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.299+0000 7fc809d10640 1 -- 172.21.15.98:0/253505524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7f0004d50 msgr2=0x7fc7f00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:36.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.299+0000 7fc809d10640 1 --2- 172.21.15.98:0/253505524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7f0004d50 0x7fc7f00051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.299+0000 7fc809d10640 1 -- 172.21.15.98:0/253505524 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f0003530 con 0x7fc7f0003f40 2024-09-19T08:45:36.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.303+0000 7fc809d10640 1 --2- 172.21.15.98:0/253505524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc7f0003f40 0x7fc7f0004810 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4009b10 tx=0x7fc7f401ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4001fdeb8585792 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:36.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.303+0000 7fc80950f640 1 -- 172.21.15.98:0/253505524 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f40b2070 con 0x7fc7f0003f40 2024-09-19T08:45:36.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.303+0000 7fc80950f640 1 -- 172.21.15.98:0/253505524 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7f40a55b0 con 0x7fc7f0003f40 2024-09-19T08:45:36.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.303+0000 7fc80950f640 1 -- 172.21.15.98:0/253505524 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f40096b0 con 0x7fc7f0003f40 2024-09-19T08:45:36.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.303+0000 7fc80c775640 1 -- 172.21.15.98:0/253505524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc7f0003f40 msgr2=0x7fc7f0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:36.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.303+0000 7fc80c775640 1 --2- 172.21.15.98:0/253505524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc7f0003f40 0x7fc7f0004810 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4009b10 tx=0x7fc7f401ca40 comp rx=0 tx=0).stop 2024-09-19T08:45:36.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.303+0000 7fc80ad12640 1 -- 172.21.15.98:0/253505524 reap_dead start 2024-09-19T08:45:36.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.303+0000 7fc80c775640 1 -- 172.21.15.98:0/253505524 shutdown_connections 2024-09-19T08:45:36.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.303+0000 7fc80c775640 1 -- 172.21.15.98:0/253505524 >> 172.21.15.98:0/253505524 conn(0x7fc8040b2010 msgr2=0x7fc8040b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:36.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.303+0000 7fc80c775640 1 -- 172.21.15.98:0/253505524 shutdown_connections 2024-09-19T08:45:36.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.303+0000 7fc80c775640 1 -- 172.21.15.98:0/253505524 wait complete. 2024-09-19T08:45:36.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.303+0000 7fc80c775640 1 Processor -- start 2024-09-19T08:45:36.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.303+0000 7fc80c775640 1 -- start start 2024-09-19T08:45:36.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.303+0000 7fc80c775640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7f0003f40 0x7fc804155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:36.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.303+0000 7fc80c775640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8040712a0 0x7fc8041488f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:36.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.303+0000 7fc80c775640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc80406d4f0 0x7fc80406d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:36.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.303+0000 7fc80c775640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc80415c920 con 0x7fc80406d4f0 2024-09-19T08:45:36.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.303+0000 7fc80c775640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc80415c7a0 con 0x7fc7f0003f40 2024-09-19T08:45:36.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.303+0000 7fc80c775640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc80406d170 con 0x7fc8040712a0 2024-09-19T08:45:36.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.307+0000 7fc80ad12640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc80406d4f0 0x7fc80406d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:36.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.307+0000 7fc80ad12640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc80406d4f0 0x7fc80406d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35686/0 (socket says 172.21.15.98:35686) 2024-09-19T08:45:36.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.307+0000 7fc80ad12640 1 -- 172.21.15.98:0/1254533132 learned_addr learned my addr 172.21.15.98:0/1254533132 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:36.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.307+0000 7fc809d10640 1 --2- 172.21.15.98:0/1254533132 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8040712a0 0x7fc8041488f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:36.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.307+0000 7fc80a511640 1 --2- 172.21.15.98:0/1254533132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7f0003f40 0x7fc804155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:36.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.307+0000 7fc80ad12640 1 -- 172.21.15.98:0/1254533132 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8040712a0 msgr2=0x7fc8041488f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:36.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.307+0000 7fc80ad12640 1 --2- 172.21.15.98:0/1254533132 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8040712a0 0x7fc8041488f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.307+0000 7fc80ad12640 1 -- 172.21.15.98:0/1254533132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7f0003f40 msgr2=0x7fc804155040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:36.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.307+0000 7fc80ad12640 1 --2- 172.21.15.98:0/1254533132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7f0003f40 0x7fc804155040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.307+0000 7fc80ad12640 1 -- 172.21.15.98:0/1254533132 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc804149340 con 0x7fc80406d4f0 2024-09-19T08:45:36.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.307+0000 7fc80ad12640 1 --2- 172.21.15.98:0/1254533132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc80406d4f0 0x7fc80406d970 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fc8040afe30 tx=0x7fc7fc00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:36.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.307+0000 7fc7faffd640 1 -- 172.21.15.98:0/1254533132 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc004d70 con 0x7fc80406d4f0 2024-09-19T08:45:36.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.307+0000 7fc7faffd640 1 -- 172.21.15.98:0/1254533132 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7fc00bb80 con 0x7fc80406d4f0 2024-09-19T08:45:36.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.307+0000 7fc7faffd640 1 -- 172.21.15.98:0/1254533132 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc007070 con 0x7fc80406d4f0 2024-09-19T08:45:36.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.307+0000 7fc80c775640 1 -- 172.21.15.98:0/1254533132 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8040718f0 con 0x7fc80406d4f0 2024-09-19T08:45:36.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.307+0000 7fc80c775640 1 -- 172.21.15.98:0/1254533132 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc804071eb0 con 0x7fc80406d4f0 2024-09-19T08:45:36.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.307+0000 7fc80c775640 1 -- 172.21.15.98:0/1254533132 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc804071680 con 0x7fc80406d4f0 2024-09-19T08:45:36.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.307+0000 7fc7faffd640 1 -- 172.21.15.98:0/1254533132 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fc7fc01b020 con 0x7fc80406d4f0 2024-09-19T08:45:36.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.307+0000 7fc7faffd640 1 --2- 172.21.15.98:0/1254533132 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc7e8067e60 0x7fc7e806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:36.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.307+0000 7fc7faffd640 1 -- 172.21.15.98:0/1254533132 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7fc7fc0869b0 con 0x7fc80406d4f0 2024-09-19T08:45:36.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.307+0000 7fc80a511640 1 --2- 172.21.15.98:0/1254533132 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc7e8067e60 0x7fc7e806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:36.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.307+0000 7fc80a511640 1 --2- 172.21.15.98:0/1254533132 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc7e8067e60 0x7fc7e806a320 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc800009ab0 tx=0x7fc800008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:36.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.311+0000 7fc7faffd640 1 -- 172.21.15.98:0/1254533132 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc804149340 con 0x7fc80406d4f0 2024-09-19T08:45:36.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.511+0000 7fc80c775640 1 -- 172.21.15.98:0/1254533132 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc804160080 con 0x7fc80406d4f0 2024-09-19T08:45:36.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.511+0000 7fc7faffd640 1 -- 172.21.15.98:0/1254533132 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v74) ==== 53+0+73 (secure 0 0 0) 0x7fc7fc056c10 con 0x7fc80406d4f0 2024-09-19T08:45:36.514 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T08:45:36.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.515+0000 7fc80c775640 1 -- 172.21.15.98:0/1254533132 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc7e8067e60 msgr2=0x7fc7e806a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:36.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.515+0000 7fc80c775640 1 --2- 172.21.15.98:0/1254533132 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc7e8067e60 0x7fc7e806a320 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc800009ab0 tx=0x7fc800008040 comp rx=0 tx=0).stop 2024-09-19T08:45:36.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.515+0000 7fc80c775640 1 -- 172.21.15.98:0/1254533132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc80406d4f0 msgr2=0x7fc80406d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:36.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.515+0000 7fc80c775640 1 --2- 172.21.15.98:0/1254533132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc80406d4f0 0x7fc80406d970 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fc8040afe30 tx=0x7fc7fc00b4a0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.515+0000 7fc80c775640 1 -- 172.21.15.98:0/1254533132 shutdown_connections 2024-09-19T08:45:36.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.515+0000 7fc80c775640 1 --2- 172.21.15.98:0/1254533132 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc7e8067e60 0x7fc7e806a320 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.515+0000 7fc80c775640 1 --2- 172.21.15.98:0/1254533132 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc80406d4f0 0x7fc80406d970 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.515+0000 7fc80c775640 1 --2- 172.21.15.98:0/1254533132 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8040712a0 0x7fc8041488f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.515+0000 7fc80c775640 1 --2- 172.21.15.98:0/1254533132 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7f0003f40 0x7fc804155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:36.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.515+0000 7fc80c775640 1 -- 172.21.15.98:0/1254533132 >> 172.21.15.98:0/1254533132 conn(0x7fc8040b2010 msgr2=0x7fc8040afed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:36.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.515+0000 7fc80c775640 1 -- 172.21.15.98:0/1254533132 shutdown_connections 2024-09-19T08:45:36.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:36.515+0000 7fc80c775640 1 -- 172.21.15.98:0/1254533132 wait complete. 2024-09-19T08:45:36.533 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T08:45:36.533 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T08:45:36.533 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2024-09-19T08:45:36.534 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-19T08:45:36.534 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-19T08:45:36.534 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T08:45:36.534 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T08:45:36.534 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T08:45:36.534 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T08:45:36.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:36.534 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2024-09-19T08:45:36.541 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T08:45:36.542 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T08:45:36.542 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T08:45:36.542 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:45:36.542 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-09-19T08:45:36.542 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:45:36.542 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:45:36.542 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:45:36.542 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2024-09-19T08:45:36.542 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:45:36.542 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:45:36.542 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2024-09-19T08:45:36.542 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:45:36.542 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:45:36.543 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T08:45:36.543 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:45:36.543 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T08:45:36.543 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 42083sec preferred_lft 42083sec 2024-09-19T08:45:36.543 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::ec4:7aff:fe88:7f59/64 scope link 2024-09-19T08:45:36.543 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:45:36.543 INFO:teuthology.orchestra.run.smithi139.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T08:45:36.543 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 02:42:74:ed:e4:60 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:45:36.543 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T08:45:36.543 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:45:36.543 INFO:teuthology.orchestra.run.smithi139.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T08:45:36.543 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-19T08:45:36.543 INFO:teuthology.orchestra.run.smithi139.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T08:45:36.543 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:45:36.544 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-19T08:45:36.544 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:45:36.544 INFO:teuthology.orchestra.run.smithi139.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:45:36.544 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:45:36.544 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-19T08:45:36.544 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:45:36.544 INFO:teuthology.orchestra.run.smithi139.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:45:36.544 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:45:36.544 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-19T08:45:36.544 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:45:36.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:36.545 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T08:45:36.594 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-09-19T08:45:36.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:36.596 DEBUG:teuthology.orchestra.run.smithi139:> route 2024-09-19T08:45:36.652 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2024-09-19T08:45:36.652 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T08:45:36.652 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T08:45:36.652 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T08:45:36.652 INFO:teuthology.orchestra.run.smithi139.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T08:45:36.652 INFO:teuthology.orchestra.run.smithi139.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T08:45:36.652 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T08:45:36.653 INFO:teuthology.orchestra.run.smithi139.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:45:36.653 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:45:36.653 INFO:teuthology.orchestra.run.smithi139.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:45:36.653 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T08:45:36.653 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:45:36.653 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-09-19T08:45:36.653 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T08:45:36.653 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T08:45:36.653 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T08:45:36.653 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-09-19T08:45:36.741 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:45:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:45:36.768 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:45:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:45:36.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:36.776 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-09-19T08:45:36.783 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-19T08:45:36.783 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-19T08:45:36.783 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T08:45:36.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:36.783 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.0 up 2024-09-19T08:45:36.839 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-19T08:45:36.839 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:45:36.839 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-19T08:45:36.888 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-19T08:45:36.888 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:45:36.888 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-19T08:45:36.935 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:45:36.994 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:45:37.040 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:45:37.040 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:45:37.040 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:45:37.041 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:45:37.041 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:45:37.041 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:45:37.041 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:45:37.041 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:45:37.041 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:45:37.041 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:45:37.041 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:45:37.041 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:45:37.041 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:45:37.041 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:45:37.041 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:45:37.041 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:45:37.041 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:45:37.042 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:45:37.042 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:45:37.042 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:45:37.042 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:45:37.042 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:45:37.042 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:45:37.042 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:45:37.042 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:45:37.042 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:45:37.042 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:45:37.042 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:45:37.042 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:45:37.042 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:45:37.042 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:45:37.042 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:45:37.043 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:45:37.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:37.044 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:45:37.091 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T08:45:37.092 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-19T08:45:37.136 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:45:37.151 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:45:37.199 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:45:37.200 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:45:37.200 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:45:37.200 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:45:37.200 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:45:37.200 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:45:37.200 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:45:37.200 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:45:37.200 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:45:37.200 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:45:37.200 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:45:37.200 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:45:37.201 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:45:37.201 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:45:37.201 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:45:37.201 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:45:37.201 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:45:37.201 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:45:37.201 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:45:37.201 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:45:37.201 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:45:37.201 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:45:37.201 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:45:37.201 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:45:37.201 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:45:37.202 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:45:37.202 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:45:37.202 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:45:37.202 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:45:37.202 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:45:37.202 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:45:37.202 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:45:37.202 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:45:37.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:37.203 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:45:37.219 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:2024-09-19T08:45:37.222+0000 7ffa47cbf580 -1 init, newargv = 0x55cc27b13f40 newargc=15 2024-09-19T08:45:37.219 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[36592]: starting ceph client 2024-09-19T08:45:37.226 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[36592]: starting fuse 2024-09-19T08:45:38.325 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:45:38.341 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:45:38.388 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:45:38.388 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:45:38.388 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:45:38.388 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:45:38.388 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:45:38.389 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:45:38.389 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:45:38.389 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:45:38.389 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:45:38.389 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:45:38.389 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:45:38.389 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:45:38.389 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:45:38.389 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:45:38.389 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:45:38.389 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:45:38.389 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:45:38.389 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:45:38.390 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:45:38.390 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:45:38.390 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:45:38.390 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:45:38.390 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:45:38.390 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:45:38.390 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:45:38.390 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:45:38.390 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:45:38.391 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:45:38.391 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:45:38.391 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:45:38.391 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:45:38.391 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:45:38.391 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:45:38.391 INFO:teuthology.orchestra.run.smithi139.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-19T08:45:38.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:38.392 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:45:38.440 INFO:teuthology.orchestra.run.smithi139.stdout:45 2024-09-19T08:45:38.440 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-19T08:45:38.440 DEBUG:teuthology.orchestra.run.smithi139:> sudo stdin-killer -- python3 -c ' 2024-09-19T08:45:38.441 DEBUG:teuthology.orchestra.run.smithi139:> import glob 2024-09-19T08:45:38.441 DEBUG:teuthology.orchestra.run.smithi139:> import re 2024-09-19T08:45:38.441 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-09-19T08:45:38.441 DEBUG:teuthology.orchestra.run.smithi139:> import subprocess 2024-09-19T08:45:38.441 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:45:38.441 DEBUG:teuthology.orchestra.run.smithi139:> def _find_admin_socket(client_name): 2024-09-19T08:45:38.441 DEBUG:teuthology.orchestra.run.smithi139:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-19T08:45:38.441 DEBUG:teuthology.orchestra.run.smithi139:> files = glob.glob(asok_path) 2024-09-19T08:45:38.441 DEBUG:teuthology.orchestra.run.smithi139:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-19T08:45:38.441 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:45:38.441 DEBUG:teuthology.orchestra.run.smithi139:> # Given a non-glob path, it better be there 2024-09-19T08:45:38.441 DEBUG:teuthology.orchestra.run.smithi139:> if "*" not in asok_path: 2024-09-19T08:45:38.441 DEBUG:teuthology.orchestra.run.smithi139:> assert(len(files) == 1) 2024-09-19T08:45:38.441 DEBUG:teuthology.orchestra.run.smithi139:> return files[0] 2024-09-19T08:45:38.442 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:45:38.442 DEBUG:teuthology.orchestra.run.smithi139:> for f in files: 2024-09-19T08:45:38.442 DEBUG:teuthology.orchestra.run.smithi139:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T08:45:38.442 DEBUG:teuthology.orchestra.run.smithi139:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T08:45:38.442 DEBUG:teuthology.orchestra.run.smithi139:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T08:45:38.442 DEBUG:teuthology.orchestra.run.smithi139:> contents = proc_f.read() 2024-09-19T08:45:38.442 DEBUG:teuthology.orchestra.run.smithi139:> if mountpoint in contents: 2024-09-19T08:45:38.442 DEBUG:teuthology.orchestra.run.smithi139:> return f 2024-09-19T08:45:38.442 DEBUG:teuthology.orchestra.run.smithi139:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T08:45:38.442 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:45:38.442 DEBUG:teuthology.orchestra.run.smithi139:> print(_find_admin_socket("client.1")) 2024-09-19T08:45:38.442 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-09-19T08:45:38.565 INFO:teuthology.orchestra.run.smithi139.stdout:/var/run/ceph/ceph-client.1.36592.asok 2024-09-19T08:45:38.569 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:45:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:45:38.577 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.36592.asok 2024-09-19T08:45:38.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:38.577 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.36592.asok status 2024-09-19T08:45:38.732 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-19T08:45:38.732 INFO:teuthology.orchestra.run.smithi139.stdout: "metadata": { 2024-09-19T08:45:38.733 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T08:45:38.733 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T08:45:38.733 INFO:teuthology.orchestra.run.smithi139.stdout: "entity_id": "1", 2024-09-19T08:45:38.733 INFO:teuthology.orchestra.run.smithi139.stdout: "hostname": "smithi139", 2024-09-19T08:45:38.733 INFO:teuthology.orchestra.run.smithi139.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-19T08:45:38.733 INFO:teuthology.orchestra.run.smithi139.stdout: "pid": "36592", 2024-09-19T08:45:38.733 INFO:teuthology.orchestra.run.smithi139.stdout: "root": "/" 2024-09-19T08:45:38.733 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:45:38.733 INFO:teuthology.orchestra.run.smithi139.stdout: "dentry_count": 0, 2024-09-19T08:45:38.733 INFO:teuthology.orchestra.run.smithi139.stdout: "dentry_pinned_count": 0, 2024-09-19T08:45:38.733 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 6673, 2024-09-19T08:45:38.733 INFO:teuthology.orchestra.run.smithi139.stdout: "inst": { 2024-09-19T08:45:38.734 INFO:teuthology.orchestra.run.smithi139.stdout: "name": { 2024-09-19T08:45:38.734 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "client", 2024-09-19T08:45:38.734 INFO:teuthology.orchestra.run.smithi139.stdout: "num": 6673 2024-09-19T08:45:38.734 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:45:38.734 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": { 2024-09-19T08:45:38.734 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-19T08:45:38.734 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:45:38.734 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3422088055 2024-09-19T08:45:38.734 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-19T08:45:38.734 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:45:38.734 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": { 2024-09-19T08:45:38.735 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-19T08:45:38.735 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:45:38.735 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3422088055 2024-09-19T08:45:38.735 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:45:38.735 INFO:teuthology.orchestra.run.smithi139.stdout: "inst_str": "client.6673 192.168.0.1:0/3422088055", 2024-09-19T08:45:38.735 INFO:teuthology.orchestra.run.smithi139.stdout: "addr_str": "192.168.0.1:0/3422088055", 2024-09-19T08:45:38.735 INFO:teuthology.orchestra.run.smithi139.stdout: "inode_count": 1, 2024-09-19T08:45:38.735 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_epoch": 74, 2024-09-19T08:45:38.735 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_epoch": 135, 2024-09-19T08:45:38.735 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_epoch_barrier": 0, 2024-09-19T08:45:38.735 INFO:teuthology.orchestra.run.smithi139.stdout: "blocklisted": false, 2024-09-19T08:45:38.735 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs" 2024-09-19T08:45:38.735 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-19T08:45:38.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:38.746 DEBUG:teuthology.orchestra.run.smithi139:> stat --file-system '--printf=%T 2024-09-19T08:45:38.746 DEBUG:teuthology.orchestra.run.smithi139:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-19T08:45:38.753 INFO:teuthology.orchestra.run.smithi139.stdout:fuseblk 2024-09-19T08:45:38.753 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-19T08:45:38.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:38.753 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-19T08:45:38.809 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-19T08:45:38.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.951+0000 7f8807f5d640 1 Processor -- start 2024-09-19T08:45:38.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.951+0000 7f8807f5d640 1 -- start start 2024-09-19T08:45:38.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.951+0000 7f8807f5d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f880006d4c0 0x7f880006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:38.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.951+0000 7f8807f5d640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f880006dde0 con 0x7f88000cf700 2024-09-19T08:45:38.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.951+0000 7f8807f5d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88001608c0 con 0x7f880015c290 2024-09-19T08:45:38.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.951+0000 7f8807f5d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f8800160e80 con 0x7f880006d4c0 2024-09-19T08:45:38.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.951+0000 7f88054f8640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f88000cf700 0x7f880015ff50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:53042/0 (socket says 172.21.15.98:53042) 2024-09-19T08:45:38.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.951+0000 7f88054f8640 1 -- 172.21.15.98:0/2511912270 learned_addr learned my addr 172.21.15.98:0/2511912270 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:38.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.951+0000 7f8804cf7640 1 -- 172.21.15.98:0/2511912270 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1398215389 0 0) 0x7f880006dde0 con 0x7f88000cf700 2024-09-19T08:45:38.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.951+0000 7f8804cf7640 1 -- 172.21.15.98:0/2511912270 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87ec003530 con 0x7f88000cf700 2024-09-19T08:45:38.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.951+0000 7f88064fa640 1 --2- 172.21.15.98:0/2511912270 >> v2:172.21.15.139:3300/0 conn(0x7f880006d4c0 0x7f880006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:38.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.951+0000 7f8804cf7640 1 -- 172.21.15.98:0/2511912270 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2952103239 0 0) 0x7f87ec003530 con 0x7f88000cf700 2024-09-19T08:45:38.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.951+0000 7f8804cf7640 1 -- 172.21.15.98:0/2511912270 >> v2:172.21.15.139:3300/0 conn(0x7f880006d4c0 msgr2=0x7f880006d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:38.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.951+0000 7f8804cf7640 1 --2- 172.21.15.98:0/2511912270 >> v2:172.21.15.139:3300/0 conn(0x7f880006d4c0 0x7f880006d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:38.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.951+0000 7f8804cf7640 1 -- 172.21.15.98:0/2511912270 >> v1:172.21.15.139:6789/0 conn(0x7f880015c290 legacy=0x7f88000cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:38.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.951+0000 7f8804cf7640 1 -- 172.21.15.98:0/2511912270 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8800160ba0 con 0x7f88000cf700 2024-09-19T08:45:38.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.951+0000 7f8804cf7640 1 -- 172.21.15.98:0/2511912270 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f87f0001680 con 0x7f88000cf700 2024-09-19T08:45:38.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.951+0000 7f8804cf7640 1 -- 172.21.15.98:0/2511912270 >> v1:172.21.15.98:6789/0 conn(0x7f88000cf700 legacy=0x7f880015ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:38.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.951+0000 7f8804cf7640 1 --2- 172.21.15.98:0/2511912270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f87ec0039c0 0x7f87ec003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:38.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.951+0000 7f8804cf7640 1 --2- 172.21.15.98:0/2511912270 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f87ec004810 0x7f87ec004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:38.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.951+0000 7f8804cf7640 1 --2- 172.21.15.98:0/2511912270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87ec0052f0 0x7f87ec00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:38.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.951+0000 7f8804cf7640 1 -- 172.21.15.98:0/2511912270 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f87ec00c190 con 0x7f87ec0052f0 2024-09-19T08:45:38.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.951+0000 7f8804cf7640 1 -- 172.21.15.98:0/2511912270 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f87ec00c340 con 0x7f87ec0039c0 2024-09-19T08:45:38.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.951+0000 7f8804cf7640 1 -- 172.21.15.98:0/2511912270 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f87ec00c540 con 0x7f87ec004810 2024-09-19T08:45:38.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.951+0000 7f88064fa640 1 --2- 172.21.15.98:0/2511912270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87ec0052f0 0x7f87ec00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:38.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.951+0000 7f8805cf9640 1 --2- 172.21.15.98:0/2511912270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f87ec0039c0 0x7f87ec003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:38.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.951+0000 7f88054f8640 1 --2- 172.21.15.98:0/2511912270 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f87ec004810 0x7f87ec004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:38.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.951+0000 7f88064fa640 1 -- 172.21.15.98:0/2511912270 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f87ec004810 msgr2=0x7f87ec004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:38.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.951+0000 7f88064fa640 1 --2- 172.21.15.98:0/2511912270 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f87ec004810 0x7f87ec004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:38.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.951+0000 7f88064fa640 1 -- 172.21.15.98:0/2511912270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f87ec0039c0 msgr2=0x7f87ec003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:38.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.951+0000 7f88064fa640 1 --2- 172.21.15.98:0/2511912270 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f87ec0039c0 0x7f87ec003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:38.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.951+0000 7f88064fa640 1 -- 172.21.15.98:0/2511912270 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87ec003530 con 0x7f87ec0052f0 2024-09-19T08:45:38.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.955+0000 7f88064fa640 1 -- 172.21.15.98:0/2511912270 reap_dead start 2024-09-19T08:45:38.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.955+0000 7f88064fa640 1 --2- 172.21.15.98:0/2511912270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87ec0052f0 0x7f87ec00b9d0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f87f8007330 tx=0x7f87f801ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc57c97bdd7d803a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:38.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.955+0000 7f8804cf7640 1 -- 172.21.15.98:0/2511912270 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f8009be0 con 0x7f87ec0052f0 2024-09-19T08:45:38.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.955+0000 7f8804cf7640 1 -- 172.21.15.98:0/2511912270 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87f80a5c50 con 0x7f87ec0052f0 2024-09-19T08:45:38.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.955+0000 7f8804cf7640 1 -- 172.21.15.98:0/2511912270 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f80039a0 con 0x7f87ec0052f0 2024-09-19T08:45:38.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.955+0000 7f8807f5d640 1 -- 172.21.15.98:0/2511912270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87ec0052f0 msgr2=0x7f87ec00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:38.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.955+0000 7f8807f5d640 1 --2- 172.21.15.98:0/2511912270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87ec0052f0 0x7f87ec00b9d0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f87f8007330 tx=0x7f87f801ce60 comp rx=0 tx=0).stop 2024-09-19T08:45:38.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.955+0000 7f8807f5d640 1 -- 172.21.15.98:0/2511912270 shutdown_connections 2024-09-19T08:45:38.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.955+0000 7f8807f5d640 1 --2- 172.21.15.98:0/2511912270 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f87ec0052f0 0x7f87ec00b9d0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:38.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.955+0000 7f8807f5d640 1 -- 172.21.15.98:0/2511912270 >> 172.21.15.98:0/2511912270 conn(0x7f880006c090 msgr2=0x7f880006c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:38.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.955+0000 7f8807f5d640 1 -- 172.21.15.98:0/2511912270 shutdown_connections 2024-09-19T08:45:38.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.955+0000 7f8807f5d640 1 -- 172.21.15.98:0/2511912270 wait complete. 2024-09-19T08:45:38.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.955+0000 7f8807f5d640 1 Processor -- start 2024-09-19T08:45:38.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.955+0000 7f8807f5d640 1 -- start start 2024-09-19T08:45:38.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.955+0000 7f8807f5d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f87ec0052f0 0x7f8800158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:38.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.955+0000 7f8807f5d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88000c5590 0x7f8800071d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:38.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.955+0000 7f8807f5d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f880015fa00 0x7f880015fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:38.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.955+0000 7f8807f5d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f880015c670 con 0x7f880015fa00 2024-09-19T08:45:38.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.955+0000 7f8807f5d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f88000cd510 con 0x7f88000c5590 2024-09-19T08:45:38.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.955+0000 7f8807f5d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f880015c7f0 con 0x7f87ec0052f0 2024-09-19T08:45:38.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.959+0000 7f88064fa640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f880015fa00 0x7f880015fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:38.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.959+0000 7f88064fa640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f880015fa00 0x7f880015fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58100/0 (socket says 172.21.15.98:58100) 2024-09-19T08:45:38.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.959+0000 7f88064fa640 1 -- 172.21.15.98:0/1690818576 learned_addr learned my addr 172.21.15.98:0/1690818576 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:38.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.959+0000 7f88054f8640 1 --2- 172.21.15.98:0/1690818576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88000c5590 0x7f8800071d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:38.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.959+0000 7f8805cf9640 1 --2- 172.21.15.98:0/1690818576 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f87ec0052f0 0x7f8800158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:38.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.959+0000 7f88064fa640 1 -- 172.21.15.98:0/1690818576 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f87ec0052f0 msgr2=0x7f8800158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:38.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.959+0000 7f88064fa640 1 --2- 172.21.15.98:0/1690818576 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f87ec0052f0 0x7f8800158510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:38.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.959+0000 7f88064fa640 1 -- 172.21.15.98:0/1690818576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88000c5590 msgr2=0x7f8800071d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:38.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.959+0000 7f88064fa640 1 --2- 172.21.15.98:0/1690818576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88000c5590 0x7f8800071d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:38.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.959+0000 7f88064fa640 1 -- 172.21.15.98:0/1690818576 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88001f3c70 con 0x7f880015fa00 2024-09-19T08:45:38.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.959+0000 7f88064fa640 1 --2- 172.21.15.98:0/1690818576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f880015fa00 0x7f880015fe80 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f87f801d700 tx=0x7f87f8019c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:38.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.959+0000 7f87f67fc640 1 -- 172.21.15.98:0/1690818576 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f8009be0 con 0x7f880015fa00 2024-09-19T08:45:38.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.959+0000 7f87f67fc640 1 -- 172.21.15.98:0/1690818576 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87f80094d0 con 0x7f880015fa00 2024-09-19T08:45:38.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.959+0000 7f8807f5d640 1 -- 172.21.15.98:0/1690818576 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88000c5b00 con 0x7f880015fa00 2024-09-19T08:45:38.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.959+0000 7f87f67fc640 1 -- 172.21.15.98:0/1690818576 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f80045e0 con 0x7f880015fa00 2024-09-19T08:45:38.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.959+0000 7f8807f5d640 1 -- 172.21.15.98:0/1690818576 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88000c60c0 con 0x7f880015fa00 2024-09-19T08:45:38.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.959+0000 7f8807f5d640 1 -- 172.21.15.98:0/1690818576 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f880006d8a0 con 0x7f880015fa00 2024-09-19T08:45:38.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.959+0000 7f87f67fc640 1 -- 172.21.15.98:0/1690818576 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f87f80bc060 con 0x7f880015fa00 2024-09-19T08:45:38.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.959+0000 7f87f67fc640 1 --2- 172.21.15.98:0/1690818576 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f87e4067e60 0x7f87e406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:38.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.959+0000 7f87f67fc640 1 -- 172.21.15.98:0/1690818576 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7f87f8120360 con 0x7f880015fa00 2024-09-19T08:45:38.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.959+0000 7f8805cf9640 1 --2- 172.21.15.98:0/1690818576 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f87e4067e60 0x7f87e406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:38.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.963+0000 7f8805cf9640 1 --2- 172.21.15.98:0/1690818576 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f87e4067e60 0x7f87e406a320 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f87fc000c10 tx=0x7f87fc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:38.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:38.963+0000 7f87f67fc640 1 -- 172.21.15.98:0/1690818576 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f87f8016b70 con 0x7f880015fa00 2024-09-19T08:45:39.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:39.155+0000 7f8807f5d640 1 -- 172.21.15.98:0/1690818576 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f880015df40 con 0x7f880015fa00 2024-09-19T08:45:39.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:39.983+0000 7f87f67fc640 1 -- 172.21.15.98:0/1690818576 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v75) ==== 136+0+0 (secure 0 0 0) 0x7f87f80f07a0 con 0x7f880015fa00 2024-09-19T08:45:39.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:39.991+0000 7f8807f5d640 1 -- 172.21.15.98:0/1690818576 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f87e4067e60 msgr2=0x7f87e406a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:39.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:39.991+0000 7f8807f5d640 1 --2- 172.21.15.98:0/1690818576 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f87e4067e60 0x7f87e406a320 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f87fc000c10 tx=0x7f87fc009040 comp rx=0 tx=0).stop 2024-09-19T08:45:39.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:39.991+0000 7f8807f5d640 1 -- 172.21.15.98:0/1690818576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f880015fa00 msgr2=0x7f880015fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:39.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:39.991+0000 7f8807f5d640 1 --2- 172.21.15.98:0/1690818576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f880015fa00 0x7f880015fe80 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f87f801d700 tx=0x7f87f8019c00 comp rx=0 tx=0).stop 2024-09-19T08:45:39.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:39.991+0000 7f8807f5d640 1 -- 172.21.15.98:0/1690818576 shutdown_connections 2024-09-19T08:45:39.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:39.991+0000 7f8807f5d640 1 --2- 172.21.15.98:0/1690818576 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f87e4067e60 0x7f87e406a320 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:39.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:39.991+0000 7f8807f5d640 1 --2- 172.21.15.98:0/1690818576 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f880015fa00 0x7f880015fe80 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:39.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:39.991+0000 7f8807f5d640 1 --2- 172.21.15.98:0/1690818576 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88000c5590 0x7f8800071d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:39.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:39.991+0000 7f8807f5d640 1 --2- 172.21.15.98:0/1690818576 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f87ec0052f0 0x7f8800158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:39.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:39.991+0000 7f8807f5d640 1 -- 172.21.15.98:0/1690818576 >> 172.21.15.98:0/1690818576 conn(0x7f880006c090 msgr2=0x7f880015e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:39.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:39.991+0000 7f8807f5d640 1 -- 172.21.15.98:0/1690818576 shutdown_connections 2024-09-19T08:45:39.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:39.991+0000 7f8807f5d640 1 -- 172.21.15.98:0/1690818576 wait complete. 2024-09-19T08:45:40.010 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-19T08:45:40.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.135+0000 7fae91dd8640 1 Processor -- start 2024-09-19T08:45:40.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.135+0000 7fae91dd8640 1 -- start start 2024-09-19T08:45:40.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.135+0000 7fae91dd8640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fae8c158fd0 0x7fae8c0b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:40.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.135+0000 7fae91dd8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae8c0b4360 0x7fae8c06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:40.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.135+0000 7fae91dd8640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae8c0ccc60 con 0x7fae8c06c1d0 2024-09-19T08:45:40.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.135+0000 7fae91dd8640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fae8c1593b0 con 0x7fae8c158fd0 2024-09-19T08:45:40.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.135+0000 7fae91dd8640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fae8c06c600 con 0x7fae8c0b4360 2024-09-19T08:45:40.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.135+0000 7fae8bfff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fae8c06c1d0 0x7fae8c15efc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:53050/0 (socket says 172.21.15.98:53050) 2024-09-19T08:45:40.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.135+0000 7fae8bfff640 1 -- 172.21.15.98:0/2884110130 learned_addr learned my addr 172.21.15.98:0/2884110130 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:40.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.135+0000 7fae8b7fe640 1 --2- 172.21.15.98:0/2884110130 >> v2:172.21.15.139:3300/0 conn(0x7fae8c158fd0 0x7fae8c0b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:40.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.135+0000 7fae8affd640 1 --2- 172.21.15.98:0/2884110130 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae8c0b4360 0x7fae8c06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:40.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.135+0000 7fae8a7fc640 1 -- 172.21.15.98:0/2884110130 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 113578757 0 0) 0x7fae8c0ccc60 con 0x7fae8c06c1d0 2024-09-19T08:45:40.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.135+0000 7fae8a7fc640 1 -- 172.21.15.98:0/2884110130 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae78003530 con 0x7fae8c06c1d0 2024-09-19T08:45:40.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.135+0000 7fae8a7fc640 1 -- 172.21.15.98:0/2884110130 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3156111522 0 0) 0x7fae78003530 con 0x7fae8c06c1d0 2024-09-19T08:45:40.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.135+0000 7fae8a7fc640 1 -- 172.21.15.98:0/2884110130 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae8c0b4360 msgr2=0x7fae8c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:40.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.135+0000 7fae8a7fc640 1 --2- 172.21.15.98:0/2884110130 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae8c0b4360 0x7fae8c06bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:40.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.135+0000 7fae8a7fc640 1 -- 172.21.15.98:0/2884110130 >> v2:172.21.15.139:3300/0 conn(0x7fae8c158fd0 msgr2=0x7fae8c0b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:40.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.135+0000 7fae8a7fc640 1 --2- 172.21.15.98:0/2884110130 >> v2:172.21.15.139:3300/0 conn(0x7fae8c158fd0 0x7fae8c0b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:40.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.135+0000 7fae8a7fc640 1 -- 172.21.15.98:0/2884110130 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae8c0ccf40 con 0x7fae8c06c1d0 2024-09-19T08:45:40.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.135+0000 7fae8a7fc640 1 -- 172.21.15.98:0/2884110130 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fae7c001a00 con 0x7fae8c06c1d0 2024-09-19T08:45:40.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.135+0000 7fae8a7fc640 1 -- 172.21.15.98:0/2884110130 >> v1:172.21.15.98:6789/0 conn(0x7fae8c06c1d0 legacy=0x7fae8c15efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:40.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.135+0000 7fae8a7fc640 1 --2- 172.21.15.98:0/2884110130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae780039c0 0x7fae78003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:40.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.135+0000 7fae8a7fc640 1 --2- 172.21.15.98:0/2884110130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae78004810 0x7fae78004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:40.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.135+0000 7fae8a7fc640 1 --2- 172.21.15.98:0/2884110130 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae8c0b4360 0x7fae780051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:40.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.135+0000 7fae8a7fc640 1 -- 172.21.15.98:0/2884110130 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c1593b0 con 0x7fae78004810 2024-09-19T08:45:40.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.135+0000 7fae8a7fc640 1 -- 172.21.15.98:0/2884110130 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c06c600 con 0x7fae780039c0 2024-09-19T08:45:40.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.135+0000 7fae8a7fc640 1 -- 172.21.15.98:0/2884110130 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fae7800ba00 con 0x7fae8c0b4360 2024-09-19T08:45:40.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.139+0000 7fae8affd640 1 --2- 172.21.15.98:0/2884110130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae78004810 0x7fae78004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:40.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.139+0000 7fae8b7fe640 1 --2- 172.21.15.98:0/2884110130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae780039c0 0x7fae78003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:40.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.139+0000 7fae8bfff640 1 --2- 172.21.15.98:0/2884110130 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae8c0b4360 0x7fae780051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:40.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.139+0000 7fae8affd640 1 -- 172.21.15.98:0/2884110130 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae8c0b4360 msgr2=0x7fae780051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:40.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.139+0000 7fae8affd640 1 --2- 172.21.15.98:0/2884110130 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae8c0b4360 0x7fae780051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:40.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.139+0000 7fae8affd640 1 -- 172.21.15.98:0/2884110130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae780039c0 msgr2=0x7fae78003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:40.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.139+0000 7fae8affd640 1 --2- 172.21.15.98:0/2884110130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae780039c0 0x7fae78003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:40.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.139+0000 7fae8affd640 1 -- 172.21.15.98:0/2884110130 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae78003530 con 0x7fae78004810 2024-09-19T08:45:40.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.139+0000 7fae8affd640 1 --2- 172.21.15.98:0/2884110130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae78004810 0x7fae78004c90 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fae74009ac0 tx=0x7fae7401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c0c016b0033f16a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:40.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.139+0000 7fae8a7fc640 1 -- 172.21.15.98:0/2884110130 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae740b2070 con 0x7fae78004810 2024-09-19T08:45:40.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.139+0000 7fae8a7fc640 1 -- 172.21.15.98:0/2884110130 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae740a5510 con 0x7fae78004810 2024-09-19T08:45:40.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.139+0000 7fae8a7fc640 1 -- 172.21.15.98:0/2884110130 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae74009690 con 0x7fae78004810 2024-09-19T08:45:40.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.139+0000 7fae91dd8640 1 -- 172.21.15.98:0/2884110130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae78004810 msgr2=0x7fae78004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:40.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.139+0000 7fae91dd8640 1 --2- 172.21.15.98:0/2884110130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae78004810 0x7fae78004c90 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fae74009ac0 tx=0x7fae7401c9c0 comp rx=0 tx=0).stop 2024-09-19T08:45:40.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.139+0000 7fae8bfff640 1 -- 172.21.15.98:0/2884110130 reap_dead start 2024-09-19T08:45:40.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.139+0000 7fae91dd8640 1 -- 172.21.15.98:0/2884110130 shutdown_connections 2024-09-19T08:45:40.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.139+0000 7fae91dd8640 1 -- 172.21.15.98:0/2884110130 >> 172.21.15.98:0/2884110130 conn(0x7fae8c0b2010 msgr2=0x7fae8c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:40.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.139+0000 7fae91dd8640 1 -- 172.21.15.98:0/2884110130 shutdown_connections 2024-09-19T08:45:40.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.139+0000 7fae91dd8640 1 -- 172.21.15.98:0/2884110130 wait complete. 2024-09-19T08:45:40.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.143+0000 7fae91dd8640 1 Processor -- start 2024-09-19T08:45:40.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.143+0000 7fae91dd8640 1 -- start start 2024-09-19T08:45:40.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.143+0000 7fae91dd8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae78004810 0x7fae8c14cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:40.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.143+0000 7fae91dd8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae8c073df0 0x7fae8c15d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:40.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.143+0000 7fae91dd8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae8c071aa0 0x7fae8c071f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:40.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.143+0000 7fae91dd8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c15c920 con 0x7fae8c073df0 2024-09-19T08:45:40.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.143+0000 7fae91dd8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c15c7a0 con 0x7fae8c071aa0 2024-09-19T08:45:40.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.143+0000 7fae91dd8640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fae8c166100 con 0x7fae78004810 2024-09-19T08:45:40.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.143+0000 7fae8affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae8c073df0 0x7fae8c15d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:40.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.143+0000 7fae8affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae8c073df0 0x7fae8c15d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58104/0 (socket says 172.21.15.98:58104) 2024-09-19T08:45:40.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.143+0000 7fae8affd640 1 -- 172.21.15.98:0/3678533456 learned_addr learned my addr 172.21.15.98:0/3678533456 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:40.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.143+0000 7fae8b7fe640 1 --2- 172.21.15.98:0/3678533456 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae78004810 0x7fae8c14cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:40.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.143+0000 7fae8affd640 1 -- 172.21.15.98:0/3678533456 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae78004810 msgr2=0x7fae8c14cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:40.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.143+0000 7fae8affd640 1 --2- 172.21.15.98:0/3678533456 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae78004810 0x7fae8c14cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:40.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.143+0000 7fae8affd640 1 -- 172.21.15.98:0/3678533456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae8c071aa0 msgr2=0x7fae8c071f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:40.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.143+0000 7fae8affd640 1 --2- 172.21.15.98:0/3678533456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae8c071aa0 0x7fae8c071f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:40.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.143+0000 7fae8affd640 1 -- 172.21.15.98:0/3678533456 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae8c15de80 con 0x7fae8c073df0 2024-09-19T08:45:40.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.143+0000 7fae8b7fe640 1 --2- 172.21.15.98:0/3678533456 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae78004810 0x7fae8c14cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:40.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.143+0000 7fae8affd640 1 --2- 172.21.15.98:0/3678533456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae8c073df0 0x7fae8c15d510 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fae7401c7f0 tx=0x7fae78005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:40.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.143+0000 7fae83fff640 1 -- 172.21.15.98:0/3678533456 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae740bc030 con 0x7fae8c073df0 2024-09-19T08:45:40.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.143+0000 7fae83fff640 1 -- 172.21.15.98:0/3678533456 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae7400f000 con 0x7fae8c073df0 2024-09-19T08:45:40.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.143+0000 7fae91dd8640 1 -- 172.21.15.98:0/3678533456 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae8c15e160 con 0x7fae8c073df0 2024-09-19T08:45:40.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.143+0000 7fae83fff640 1 -- 172.21.15.98:0/3678533456 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae740b2040 con 0x7fae8c073df0 2024-09-19T08:45:40.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.143+0000 7fae91dd8640 1 -- 172.21.15.98:0/3678533456 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae8c1549d0 con 0x7fae8c073df0 2024-09-19T08:45:40.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.143+0000 7fae91dd8640 1 -- 172.21.15.98:0/3678533456 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae8c0ccc80 con 0x7fae8c073df0 2024-09-19T08:45:40.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.143+0000 7fae83fff640 1 -- 172.21.15.98:0/3678533456 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fae740bf070 con 0x7fae8c073df0 2024-09-19T08:45:40.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.147+0000 7fae83fff640 1 --2- 172.21.15.98:0/3678533456 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fae6c067e60 0x7fae6c06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:40.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.147+0000 7fae83fff640 1 -- 172.21.15.98:0/3678533456 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7fae741200b0 con 0x7fae8c073df0 2024-09-19T08:45:40.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.147+0000 7fae8b7fe640 1 --2- 172.21.15.98:0/3678533456 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fae6c067e60 0x7fae6c06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:40.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.147+0000 7fae8b7fe640 1 --2- 172.21.15.98:0/3678533456 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fae6c067e60 0x7fae6c06a320 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fae8c15e840 tx=0x7fae84008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:40.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.151+0000 7fae83fff640 1 -- 172.21.15.98:0/3678533456 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fae74016760 con 0x7fae8c073df0 2024-09-19T08:45:40.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.319+0000 7fae91dd8640 1 -- 172.21.15.98:0/3678533456 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fae8c0d09c0 con 0x7fae6c067e60 2024-09-19T08:45:40.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.323+0000 7fae83fff640 1 -- 172.21.15.98:0/3678533456 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fae8c0d09c0 con 0x7fae6c067e60 2024-09-19T08:45:40.323 INFO:teuthology.orchestra.run.smithi098.stdout:[ 2024-09-19T08:45:40.323 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:45:40.324 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "cephfs" 2024-09-19T08:45:40.324 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:45:40.324 INFO:teuthology.orchestra.run.smithi098.stdout:] 2024-09-19T08:45:40.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.327+0000 7fae91dd8640 1 -- 172.21.15.98:0/3678533456 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fae6c067e60 msgr2=0x7fae6c06a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:40.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.327+0000 7fae91dd8640 1 --2- 172.21.15.98:0/3678533456 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fae6c067e60 0x7fae6c06a320 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fae8c15e840 tx=0x7fae84008040 comp rx=0 tx=0).stop 2024-09-19T08:45:40.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.327+0000 7fae91dd8640 1 -- 172.21.15.98:0/3678533456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae8c073df0 msgr2=0x7fae8c15d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:40.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.327+0000 7fae91dd8640 1 --2- 172.21.15.98:0/3678533456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae8c073df0 0x7fae8c15d510 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fae7401c7f0 tx=0x7fae78005840 comp rx=0 tx=0).stop 2024-09-19T08:45:40.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.327+0000 7fae91dd8640 1 -- 172.21.15.98:0/3678533456 shutdown_connections 2024-09-19T08:45:40.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.327+0000 7fae91dd8640 1 --2- 172.21.15.98:0/3678533456 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fae6c067e60 0x7fae6c06a320 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:40.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.327+0000 7fae91dd8640 1 --2- 172.21.15.98:0/3678533456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae8c071aa0 0x7fae8c071f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:40.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.327+0000 7fae91dd8640 1 --2- 172.21.15.98:0/3678533456 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae8c073df0 0x7fae8c15d510 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:40.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.327+0000 7fae91dd8640 1 --2- 172.21.15.98:0/3678533456 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae78004810 0x7fae8c14cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:40.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.327+0000 7fae91dd8640 1 -- 172.21.15.98:0/3678533456 >> 172.21.15.98:0/3678533456 conn(0x7fae8c0b2010 msgr2=0x7fae8c157e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:40.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.327+0000 7fae91dd8640 1 -- 172.21.15.98:0/3678533456 shutdown_connections 2024-09-19T08:45:40.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.327+0000 7fae91dd8640 1 -- 172.21.15.98:0/3678533456 wait complete. 2024-09-19T08:45:40.340 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T08:45:40.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.467+0000 7fb96db47640 1 Processor -- start 2024-09-19T08:45:40.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.467+0000 7fb96db47640 1 -- start start 2024-09-19T08:45:40.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.467+0000 7fb96db47640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9680b4420 0x7fb968071ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:40.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.467+0000 7fb96db47640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb968073530 con 0x7fb96806c3a0 2024-09-19T08:45:40.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.467+0000 7fb96db47640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb968073810 con 0x7fb968145b70 2024-09-19T08:45:40.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.467+0000 7fb96db47640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb968073af0 con 0x7fb9680b4420 2024-09-19T08:45:40.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.467+0000 7fb95ffff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fb96806c3a0 0x7fb9680713b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:53058/0 (socket says 172.21.15.98:53058) 2024-09-19T08:45:40.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.467+0000 7fb95ffff640 1 -- 172.21.15.98:0/2772363943 learned_addr learned my addr 172.21.15.98:0/2772363943 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:40.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.467+0000 7fb967fff640 1 --2- 172.21.15.98:0/2772363943 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9680b4420 0x7fb968071ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:40.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.467+0000 7fb966ffd640 1 -- 172.21.15.98:0/2772363943 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3094294293 0 0) 0x7fb968073530 con 0x7fb96806c3a0 2024-09-19T08:45:40.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.467+0000 7fb966ffd640 1 -- 172.21.15.98:0/2772363943 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb954003530 con 0x7fb96806c3a0 2024-09-19T08:45:40.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.467+0000 7fb966ffd640 1 -- 172.21.15.98:0/2772363943 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3790009496 0 0) 0x7fb968073810 con 0x7fb968145b70 2024-09-19T08:45:40.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.467+0000 7fb966ffd640 1 -- 172.21.15.98:0/2772363943 --> v1:172.21.15.139:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb968073530 con 0x7fb968145b70 2024-09-19T08:45:40.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.467+0000 7fb966ffd640 1 -- 172.21.15.98:0/2772363943 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1714778269 0 0) 0x7fb954003530 con 0x7fb96806c3a0 2024-09-19T08:45:40.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.467+0000 7fb966ffd640 1 -- 172.21.15.98:0/2772363943 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9680b4420 msgr2=0x7fb968071ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:40.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.467+0000 7fb966ffd640 1 --2- 172.21.15.98:0/2772363943 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9680b4420 0x7fb968071ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:40.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.467+0000 7fb966ffd640 1 -- 172.21.15.98:0/2772363943 >> v1:172.21.15.139:6789/0 conn(0x7fb968145b70 legacy=0x7fb96806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:40.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.467+0000 7fb966ffd640 1 -- 172.21.15.98:0/2772363943 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb968073dd0 con 0x7fb96806c3a0 2024-09-19T08:45:40.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.467+0000 7fb966ffd640 1 -- 172.21.15.98:0/2772363943 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fb950001680 con 0x7fb96806c3a0 2024-09-19T08:45:40.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.467+0000 7fb966ffd640 1 -- 172.21.15.98:0/2772363943 >> v1:172.21.15.98:6789/0 conn(0x7fb96806c3a0 legacy=0x7fb9680713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:40.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.467+0000 7fb966ffd640 1 --2- 172.21.15.98:0/2772363943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb954003b80 0x7fb954003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:40.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.467+0000 7fb966ffd640 1 --2- 172.21.15.98:0/2772363943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb954004810 0x7fb954004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:40.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.467+0000 7fb966ffd640 1 --2- 172.21.15.98:0/2772363943 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9680b4420 0x7fb9540051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:40.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.467+0000 7fb966ffd640 1 -- 172.21.15.98:0/2772363943 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb968073af0 con 0x7fb954004810 2024-09-19T08:45:40.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.467+0000 7fb966ffd640 1 -- 172.21.15.98:0/2772363943 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb95400bae0 con 0x7fb954003b80 2024-09-19T08:45:40.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.467+0000 7fb966ffd640 1 -- 172.21.15.98:0/2772363943 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb95400bce0 con 0x7fb9680b4420 2024-09-19T08:45:40.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.467+0000 7fb95ffff640 1 --2- 172.21.15.98:0/2772363943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb954004810 0x7fb954004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:40.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.467+0000 7fb9677fe640 1 --2- 172.21.15.98:0/2772363943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb954003b80 0x7fb954003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:40.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.467+0000 7fb967fff640 1 --2- 172.21.15.98:0/2772363943 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9680b4420 0x7fb9540051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:40.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.467+0000 7fb95ffff640 1 -- 172.21.15.98:0/2772363943 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9680b4420 msgr2=0x7fb9540051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:40.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.467+0000 7fb95ffff640 1 --2- 172.21.15.98:0/2772363943 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9680b4420 0x7fb9540051f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:40.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.467+0000 7fb95ffff640 1 -- 172.21.15.98:0/2772363943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb954003b80 msgr2=0x7fb954003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:40.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.467+0000 7fb95ffff640 1 --2- 172.21.15.98:0/2772363943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb954003b80 0x7fb954003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:40.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.467+0000 7fb95ffff640 1 -- 172.21.15.98:0/2772363943 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb954003530 con 0x7fb954004810 2024-09-19T08:45:40.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.467+0000 7fb967fff640 1 --2- 172.21.15.98:0/2772363943 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb9680b4420 0x7fb9540051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:40.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.471+0000 7fb95ffff640 1 --2- 172.21.15.98:0/2772363943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb954004810 0x7fb954004c90 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fb95000c190 tx=0x7fb95001cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ec5a0252651ba4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:40.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.471+0000 7fb966ffd640 1 -- 172.21.15.98:0/2772363943 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb950007cd0 con 0x7fb954004810 2024-09-19T08:45:40.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.471+0000 7fb966ffd640 1 -- 172.21.15.98:0/2772363943 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9500a9020 con 0x7fb954004810 2024-09-19T08:45:40.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.471+0000 7fb966ffd640 1 -- 172.21.15.98:0/2772363943 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb950005ae0 con 0x7fb954004810 2024-09-19T08:45:40.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.471+0000 7fb96db47640 1 -- 172.21.15.98:0/2772363943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb954004810 msgr2=0x7fb954004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:40.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.471+0000 7fb96db47640 1 --2- 172.21.15.98:0/2772363943 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb954004810 0x7fb954004c90 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fb95000c190 tx=0x7fb95001cc00 comp rx=0 tx=0).stop 2024-09-19T08:45:40.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.471+0000 7fb967fff640 1 -- 172.21.15.98:0/2772363943 reap_dead start 2024-09-19T08:45:40.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.471+0000 7fb96db47640 1 -- 172.21.15.98:0/2772363943 shutdown_connections 2024-09-19T08:45:40.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.471+0000 7fb96db47640 1 -- 172.21.15.98:0/2772363943 >> 172.21.15.98:0/2772363943 conn(0x7fb9680b2010 msgr2=0x7fb9680b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:40.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.471+0000 7fb96db47640 1 -- 172.21.15.98:0/2772363943 shutdown_connections 2024-09-19T08:45:40.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.471+0000 7fb96db47640 1 -- 172.21.15.98:0/2772363943 wait complete. 2024-09-19T08:45:40.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.471+0000 7fb96db47640 1 Processor -- start 2024-09-19T08:45:40.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.471+0000 7fb96db47640 1 -- start start 2024-09-19T08:45:40.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.475+0000 7fb96db47640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb954004810 0x7fb96813e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:40.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.475+0000 7fb96db47640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb96813e860 0x7fb96813ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:40.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.475+0000 7fb96db47640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb96813f200 0x7fb9680ab1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:40.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.475+0000 7fb96db47640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb968167a80 con 0x7fb954004810 2024-09-19T08:45:40.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.475+0000 7fb96db47640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb968071ed0 con 0x7fb96813e860 2024-09-19T08:45:40.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.475+0000 7fb96db47640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb968167c00 con 0x7fb96813f200 2024-09-19T08:45:40.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.475+0000 7fb9677fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb954004810 0x7fb96813e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:40.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.475+0000 7fb9677fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb954004810 0x7fb96813e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58114/0 (socket says 172.21.15.98:58114) 2024-09-19T08:45:40.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.475+0000 7fb9677fe640 1 -- 172.21.15.98:0/403348351 learned_addr learned my addr 172.21.15.98:0/403348351 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:40.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.475+0000 7fb9677fe640 1 -- 172.21.15.98:0/403348351 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb96813f200 msgr2=0x7fb9680ab1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:40.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.475+0000 7fb9677fe640 1 --2- 172.21.15.98:0/403348351 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb96813f200 0x7fb9680ab1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:40.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.475+0000 7fb9677fe640 1 -- 172.21.15.98:0/403348351 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb96813e860 msgr2=0x7fb96813ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:40.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.475+0000 7fb95ffff640 1 --2- 172.21.15.98:0/403348351 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb96813e860 0x7fb96813ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:40.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.475+0000 7fb9677fe640 1 --2- 172.21.15.98:0/403348351 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb96813e860 0x7fb96813ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:40.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.475+0000 7fb9677fe640 1 -- 172.21.15.98:0/403348351 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9680adad0 con 0x7fb954004810 2024-09-19T08:45:40.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.475+0000 7fb9677fe640 1 --2- 172.21.15.98:0/403348351 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb954004810 0x7fb96813e320 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fb95800e520 tx=0x7fb95800e9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:40.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.475+0000 7fb964ff9640 1 -- 172.21.15.98:0/403348351 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9580052e0 con 0x7fb954004810 2024-09-19T08:45:40.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.475+0000 7fb964ff9640 1 -- 172.21.15.98:0/403348351 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb958018070 con 0x7fb954004810 2024-09-19T08:45:40.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.475+0000 7fb96db47640 1 -- 172.21.15.98:0/403348351 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9680addb0 con 0x7fb954004810 2024-09-19T08:45:40.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.475+0000 7fb964ff9640 1 -- 172.21.15.98:0/403348351 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9580054a0 con 0x7fb954004810 2024-09-19T08:45:40.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.475+0000 7fb96db47640 1 -- 172.21.15.98:0/403348351 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9680ae370 con 0x7fb954004810 2024-09-19T08:45:40.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.475+0000 7fb96db47640 1 -- 172.21.15.98:0/403348351 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb96813f620 con 0x7fb954004810 2024-09-19T08:45:40.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.475+0000 7fb964ff9640 1 -- 172.21.15.98:0/403348351 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fb958003840 con 0x7fb954004810 2024-09-19T08:45:40.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.475+0000 7fb964ff9640 1 --2- 172.21.15.98:0/403348351 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb948067e60 0x7fb94806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:40.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.475+0000 7fb964ff9640 1 -- 172.21.15.98:0/403348351 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7fb958086fe0 con 0x7fb954004810 2024-09-19T08:45:40.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.475+0000 7fb95ffff640 1 --2- 172.21.15.98:0/403348351 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb948067e60 0x7fb94806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:40.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.479+0000 7fb95ffff640 1 --2- 172.21.15.98:0/403348351 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb948067e60 0x7fb94806a320 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb9500aa000 tx=0x7fb9500019d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:40.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.479+0000 7fb964ff9640 1 -- 172.21.15.98:0/403348351 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb9580572d0 con 0x7fb954004810 2024-09-19T08:45:40.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.643+0000 7fb96db47640 1 -- 172.21.15.98:0/403348351 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb9680713b0 con 0x7fb954004810 2024-09-19T08:45:40.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.643+0000 7fb964ff9640 1 -- 172.21.15.98:0/403348351 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7fb958056bd0 con 0x7fb954004810 2024-09-19T08:45:40.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.647+0000 7fb96db47640 1 -- 172.21.15.98:0/403348351 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb948067e60 msgr2=0x7fb94806a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:40.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.647+0000 7fb96db47640 1 --2- 172.21.15.98:0/403348351 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb948067e60 0x7fb94806a320 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb9500aa000 tx=0x7fb9500019d0 comp rx=0 tx=0).stop 2024-09-19T08:45:40.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.647+0000 7fb96db47640 1 -- 172.21.15.98:0/403348351 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb954004810 msgr2=0x7fb96813e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:40.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.647+0000 7fb96db47640 1 --2- 172.21.15.98:0/403348351 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb954004810 0x7fb96813e320 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fb95800e520 tx=0x7fb95800e9e0 comp rx=0 tx=0).stop 2024-09-19T08:45:40.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.647+0000 7fb96db47640 1 -- 172.21.15.98:0/403348351 shutdown_connections 2024-09-19T08:45:40.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.647+0000 7fb96db47640 1 --2- 172.21.15.98:0/403348351 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb948067e60 0x7fb94806a320 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:40.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.647+0000 7fb96db47640 1 --2- 172.21.15.98:0/403348351 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb96813f200 0x7fb9680ab1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:40.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.647+0000 7fb96db47640 1 --2- 172.21.15.98:0/403348351 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb96813e860 0x7fb96813ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:40.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.647+0000 7fb96db47640 1 --2- 172.21.15.98:0/403348351 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb954004810 0x7fb96813e320 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:40.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.647+0000 7fb96db47640 1 -- 172.21.15.98:0/403348351 >> 172.21.15.98:0/403348351 conn(0x7fb9680b2010 msgr2=0x7fb9680b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:40.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.647+0000 7fb96db47640 1 -- 172.21.15.98:0/403348351 shutdown_connections 2024-09-19T08:45:40.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.647+0000 7fb96db47640 1 -- 172.21.15.98:0/403348351 wait complete. 2024-09-19T08:45:40.662 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create non_existent_subvolume subvolume_1 2024-09-19T08:45:40.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.787+0000 7f957af39640 1 Processor -- start 2024-09-19T08:45:40.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.787+0000 7f957af39640 1 -- start start 2024-09-19T08:45:40.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.787+0000 7f957af39640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9574144d10 0x7f95740722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:40.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.787+0000 7f957af39640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9574072ea0 con 0x7f957406c3a0 2024-09-19T08:45:40.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.787+0000 7f957af39640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9574073180 con 0x7f9574072820 2024-09-19T08:45:40.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.787+0000 7f957af39640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f9574072c60 con 0x7f9574144d10 2024-09-19T08:45:40.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.787+0000 7f9578cd5640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9574144d10 0x7f95740722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:40.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.787+0000 7f95794d6640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f957406c3a0 0x7f95740cd250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:53062/0 (socket says 172.21.15.98:53062) 2024-09-19T08:45:40.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.787+0000 7f95794d6640 1 -- 172.21.15.98:0/2413508461 learned_addr learned my addr 172.21.15.98:0/2413508461 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:40.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.787+0000 7f95737fe640 1 -- 172.21.15.98:0/2413508461 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2687880299 0 0) 0x7f9574072ea0 con 0x7f957406c3a0 2024-09-19T08:45:40.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.787+0000 7f95737fe640 1 -- 172.21.15.98:0/2413508461 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f955c005740 con 0x7f957406c3a0 2024-09-19T08:45:40.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.787+0000 7f9578cd5640 1 -- 172.21.15.98:0/2413508461 >> v1:172.21.15.139:6789/0 conn(0x7f9574072820 legacy=0x7f957406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:40.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.787+0000 7f9578cd5640 1 -- 172.21.15.98:0/2413508461 >> v1:172.21.15.98:6789/0 conn(0x7f957406c3a0 legacy=0x7f95740cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:40.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.787+0000 7f9578cd5640 1 -- 172.21.15.98:0/2413508461 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9574073460 con 0x7f9574144d10 2024-09-19T08:45:40.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.791+0000 7f9578cd5640 1 --2- 172.21.15.98:0/2413508461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9574144d10 0x7f95740722e0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f95680097c0 tx=0x7f956801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47af4147673c155 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:40.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.791+0000 7f95737fe640 1 -- 172.21.15.98:0/2413508461 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95680b0070 con 0x7f9574144d10 2024-09-19T08:45:40.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.791+0000 7f95737fe640 1 -- 172.21.15.98:0/2413508461 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95680ab600 con 0x7f9574144d10 2024-09-19T08:45:40.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.791+0000 7f95737fe640 1 -- 172.21.15.98:0/2413508461 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95680ab9b0 con 0x7f9574144d10 2024-09-19T08:45:40.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.791+0000 7f957af39640 1 -- 172.21.15.98:0/2413508461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9574144d10 msgr2=0x7f95740722e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:40.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.791+0000 7f957af39640 1 --2- 172.21.15.98:0/2413508461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9574144d10 0x7f95740722e0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f95680097c0 tx=0x7f956801c4a0 comp rx=0 tx=0).stop 2024-09-19T08:45:40.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.791+0000 7f957af39640 1 -- 172.21.15.98:0/2413508461 shutdown_connections 2024-09-19T08:45:40.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.791+0000 7f957af39640 1 --2- 172.21.15.98:0/2413508461 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9574144d10 0x7f95740722e0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:40.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.791+0000 7f957af39640 1 -- 172.21.15.98:0/2413508461 >> 172.21.15.98:0/2413508461 conn(0x7f95740b2010 msgr2=0x7f95740b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:40.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.791+0000 7f957af39640 1 -- 172.21.15.98:0/2413508461 shutdown_connections 2024-09-19T08:45:40.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.791+0000 7f957af39640 1 -- 172.21.15.98:0/2413508461 wait complete. 2024-09-19T08:45:40.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.795+0000 7f957af39640 1 Processor -- start 2024-09-19T08:45:40.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.795+0000 7f957af39640 1 -- start start 2024-09-19T08:45:40.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.795+0000 7f957af39640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f957406c3a0 0x7f957415aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:40.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.795+0000 7f957af39640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9574072820 0x7f9574143d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:40.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.795+0000 7f957af39640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9574144d10 0x7f957413bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:40.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.795+0000 7f957af39640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f95740ce540 con 0x7f957406c3a0 2024-09-19T08:45:40.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.795+0000 7f957af39640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f95740ce3c0 con 0x7f9574144d10 2024-09-19T08:45:40.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.795+0000 7f957af39640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f95740b3410 con 0x7f9574072820 2024-09-19T08:45:40.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.795+0000 7f9578cd5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f957406c3a0 0x7f957415aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:40.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.795+0000 7f9578cd5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f957406c3a0 0x7f957415aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58122/0 (socket says 172.21.15.98:58122) 2024-09-19T08:45:40.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.795+0000 7f9578cd5640 1 -- 172.21.15.98:0/2158449844 learned_addr learned my addr 172.21.15.98:0/2158449844 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:40.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.795+0000 7f9573fff640 1 --2- 172.21.15.98:0/2158449844 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9574072820 0x7f9574143d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:40.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.795+0000 7f95794d6640 1 --2- 172.21.15.98:0/2158449844 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9574144d10 0x7f957413bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:40.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.795+0000 7f9578cd5640 1 -- 172.21.15.98:0/2158449844 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9574072820 msgr2=0x7f9574143d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:40.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.795+0000 7f9578cd5640 1 --2- 172.21.15.98:0/2158449844 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9574072820 0x7f9574143d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:40.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.795+0000 7f9578cd5640 1 -- 172.21.15.98:0/2158449844 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9574144d10 msgr2=0x7f957413bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:40.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.795+0000 7f9578cd5640 1 --2- 172.21.15.98:0/2158449844 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9574144d10 0x7f957413bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:40.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.795+0000 7f9578cd5640 1 -- 172.21.15.98:0/2158449844 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95741421e0 con 0x7f957406c3a0 2024-09-19T08:45:40.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.795+0000 7f9578cd5640 1 --2- 172.21.15.98:0/2158449844 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f957406c3a0 0x7f957415aea0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f95680ae040 tx=0x7f956801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:40.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.795+0000 7f95717fa640 1 -- 172.21.15.98:0/2158449844 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95680b0070 con 0x7f957406c3a0 2024-09-19T08:45:40.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.795+0000 7f95717fa640 1 -- 172.21.15.98:0/2158449844 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95680ba070 con 0x7f957406c3a0 2024-09-19T08:45:40.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.795+0000 7f957af39640 1 -- 172.21.15.98:0/2158449844 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95741424c0 con 0x7f957406c3a0 2024-09-19T08:45:40.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.795+0000 7f95717fa640 1 -- 172.21.15.98:0/2158449844 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95680abdc0 con 0x7f957406c3a0 2024-09-19T08:45:40.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.795+0000 7f957af39640 1 -- 172.21.15.98:0/2158449844 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9574142a80 con 0x7f957406c3a0 2024-09-19T08:45:40.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.795+0000 7f957af39640 1 -- 172.21.15.98:0/2158449844 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95740722e0 con 0x7f957406c3a0 2024-09-19T08:45:40.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.795+0000 7f95717fa640 1 -- 172.21.15.98:0/2158449844 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f95680ab4c0 con 0x7f957406c3a0 2024-09-19T08:45:40.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.799+0000 7f95717fa640 1 --2- 172.21.15.98:0/2158449844 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9560067e60 0x7f956006a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:40.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.799+0000 7f95717fa640 1 -- 172.21.15.98:0/2158449844 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7f9568120810 con 0x7f957406c3a0 2024-09-19T08:45:40.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.799+0000 7f9573fff640 1 --2- 172.21.15.98:0/2158449844 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9560067e60 0x7f956006a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:40.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.799+0000 7f9573fff640 1 --2- 172.21.15.98:0/2158449844 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9560067e60 0x7f956006a320 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f957413c850 tx=0x7f9564009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:40.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.803+0000 7f95717fa640 1 -- 172.21.15.98:0/2158449844 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9568016090 con 0x7f957406c3a0 2024-09-19T08:45:40.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.987+0000 7f957af39640 1 -- 172.21.15.98:0/2158449844 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7f95740757a0 con 0x7f9560067e60 2024-09-19T08:45:40.990 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:40.990+0000 7ff73a69c640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-19T08:45:40.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:40.991+0000 7f95717fa640 1 -- 172.21.15.98:0/2158449844 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f95740757a0 con 0x7f9560067e60 2024-09-19T08:45:40.991 INFO:teuthology.orchestra.run.smithi098.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-19T08:45:40.997 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-19T08:45:40.998 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm non_existent_subvolume subvolume_1 2024-09-19T08:45:41.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.119+0000 7ffb27000640 1 Processor -- start 2024-09-19T08:45:41.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.119+0000 7ffb27000640 1 -- start start 2024-09-19T08:45:41.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.119+0000 7ffb27000640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ffb20145b70 0x7ffb2006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.119+0000 7ffb27000640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ffb2006c1d0 0x7ffb200713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.119+0000 7ffb27000640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffb200718f0 0x7ffb200cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.119+0000 7ffb27000640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7ffb2006c620 con 0x7ffb20145b70 2024-09-19T08:45:41.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.119+0000 7ffb27000640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ffb20073530 con 0x7ffb2006c1d0 2024-09-19T08:45:41.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.119+0000 7ffb27000640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ffb20073810 con 0x7ffb200718f0 2024-09-19T08:45:41.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.119+0000 7ffb24d9c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ffb20145b70 0x7ffb2006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.119+0000 7ffb24d9c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ffb20145b70 0x7ffb2006bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58134/0 (socket says 172.21.15.98:58134) 2024-09-19T08:45:41.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.119+0000 7ffb24d9c640 1 -- 172.21.15.98:0/2551788009 learned_addr learned my addr 172.21.15.98:0/2551788009 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:41.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.119+0000 7ffb17fff640 1 --2- 172.21.15.98:0/2551788009 >> v2:172.21.15.139:3300/0 conn(0x7ffb2006c1d0 0x7ffb200713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.119+0000 7ffb2559d640 1 --2- 172.21.15.98:0/2551788009 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffb200718f0 0x7ffb200cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.119+0000 7ffb24d9c640 1 -- 172.21.15.98:0/2551788009 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffb200718f0 msgr2=0x7ffb200cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:41.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.119+0000 7ffb24d9c640 1 --2- 172.21.15.98:0/2551788009 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffb200718f0 0x7ffb200cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.119+0000 7ffb24d9c640 1 -- 172.21.15.98:0/2551788009 >> v2:172.21.15.139:3300/0 conn(0x7ffb2006c1d0 msgr2=0x7ffb200713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:41.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.119+0000 7ffb24d9c640 1 --2- 172.21.15.98:0/2551788009 >> v2:172.21.15.139:3300/0 conn(0x7ffb2006c1d0 0x7ffb200713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.119+0000 7ffb24d9c640 1 -- 172.21.15.98:0/2551788009 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb20073af0 con 0x7ffb20145b70 2024-09-19T08:45:41.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.123+0000 7ffb24d9c640 1 --2- 172.21.15.98:0/2551788009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb20145b70 0x7ffb2006bc90 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7ffb200ccd30 tx=0x7ffb1c01c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e6ef1eeb1ea72a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:41.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.123+0000 7ffb177fe640 1 -- 172.21.15.98:0/2551788009 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c0b0070 con 0x7ffb20145b70 2024-09-19T08:45:41.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.123+0000 7ffb177fe640 1 -- 172.21.15.98:0/2551788009 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb1c0ab600 con 0x7ffb20145b70 2024-09-19T08:45:41.121 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.123+0000 7ffb177fe640 1 -- 172.21.15.98:0/2551788009 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c0ab9d0 con 0x7ffb20145b70 2024-09-19T08:45:41.122 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.123+0000 7ffb27000640 1 -- 172.21.15.98:0/2551788009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb20145b70 msgr2=0x7ffb2006bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:41.122 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.123+0000 7ffb27000640 1 --2- 172.21.15.98:0/2551788009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb20145b70 0x7ffb2006bc90 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7ffb200ccd30 tx=0x7ffb1c01c1d0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.122 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.123+0000 7ffb27000640 1 -- 172.21.15.98:0/2551788009 shutdown_connections 2024-09-19T08:45:41.122 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.123+0000 7ffb27000640 1 --2- 172.21.15.98:0/2551788009 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffb200718f0 0x7ffb200cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.122 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.123+0000 7ffb27000640 1 --2- 172.21.15.98:0/2551788009 >> v2:172.21.15.139:3300/0 conn(0x7ffb2006c1d0 0x7ffb200713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.122 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.123+0000 7ffb27000640 1 --2- 172.21.15.98:0/2551788009 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb20145b70 0x7ffb2006bc90 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.122 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.123+0000 7ffb27000640 1 -- 172.21.15.98:0/2551788009 >> 172.21.15.98:0/2551788009 conn(0x7ffb200b2010 msgr2=0x7ffb200b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:41.123 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.123+0000 7ffb27000640 1 -- 172.21.15.98:0/2551788009 shutdown_connections 2024-09-19T08:45:41.123 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.123+0000 7ffb27000640 1 -- 172.21.15.98:0/2551788009 wait complete. 2024-09-19T08:45:41.123 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.123+0000 7ffb27000640 1 Processor -- start 2024-09-19T08:45:41.123 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.123+0000 7ffb27000640 1 -- start start 2024-09-19T08:45:41.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.127+0000 7ffb27000640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb2006c1d0 0x7ffb2013c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.127+0000 7ffb27000640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb200718f0 0x7ffb2013bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.127+0000 7ffb27000640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffb20145b70 0x7ffb2013b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.127+0000 7ffb27000640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ffb200b3410 con 0x7ffb2006c1d0 2024-09-19T08:45:41.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.127+0000 7ffb27000640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffb200b3590 con 0x7ffb200718f0 2024-09-19T08:45:41.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.127+0000 7ffb27000640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ffb200b3710 con 0x7ffb20145b70 2024-09-19T08:45:41.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.127+0000 7ffb24d9c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb2006c1d0 0x7ffb2013c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.127+0000 7ffb24d9c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb2006c1d0 0x7ffb2013c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58148/0 (socket says 172.21.15.98:58148) 2024-09-19T08:45:41.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.127+0000 7ffb24d9c640 1 -- 172.21.15.98:0/1388570410 learned_addr learned my addr 172.21.15.98:0/1388570410 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:41.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.127+0000 7ffb2559d640 1 --2- 172.21.15.98:0/1388570410 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffb20145b70 0x7ffb2013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.127+0000 7ffb17fff640 1 --2- 172.21.15.98:0/1388570410 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb200718f0 0x7ffb2013bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.127+0000 7ffb24d9c640 1 -- 172.21.15.98:0/1388570410 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffb20145b70 msgr2=0x7ffb2013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:41.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.127+0000 7ffb24d9c640 1 --2- 172.21.15.98:0/1388570410 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ffb20145b70 0x7ffb2013b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.127+0000 7ffb24d9c640 1 -- 172.21.15.98:0/1388570410 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb200718f0 msgr2=0x7ffb2013bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:41.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.127+0000 7ffb24d9c640 1 --2- 172.21.15.98:0/1388570410 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb200718f0 0x7ffb2013bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.127+0000 7ffb24d9c640 1 -- 172.21.15.98:0/1388570410 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb2013ed70 con 0x7ffb2006c1d0 2024-09-19T08:45:41.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.127+0000 7ffb24d9c640 1 --2- 172.21.15.98:0/1388570410 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ffb2006c1d0 0x7ffb2013c8d0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c0ae040 tx=0x7ffb1c018f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:41.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.127+0000 7ffb157fa640 1 -- 172.21.15.98:0/1388570410 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c0b0040 con 0x7ffb2006c1d0 2024-09-19T08:45:41.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.127+0000 7ffb157fa640 1 -- 172.21.15.98:0/1388570410 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb1c0b9070 con 0x7ffb2006c1d0 2024-09-19T08:45:41.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.127+0000 7ffb27000640 1 -- 172.21.15.98:0/1388570410 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb2013f050 con 0x7ffb2006c1d0 2024-09-19T08:45:41.127 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.127+0000 7ffb157fa640 1 -- 172.21.15.98:0/1388570410 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c0b46a0 con 0x7ffb2006c1d0 2024-09-19T08:45:41.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.127+0000 7ffb27000640 1 -- 172.21.15.98:0/1388570410 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb2013f8f0 con 0x7ffb2006c1d0 2024-09-19T08:45:41.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.127+0000 7ffb27000640 1 -- 172.21.15.98:0/1388570410 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb2013f220 con 0x7ffb2006c1d0 2024-09-19T08:45:41.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.127+0000 7ffb157fa640 1 -- 172.21.15.98:0/1388570410 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7ffb1c0b4840 con 0x7ffb2006c1d0 2024-09-19T08:45:41.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.127+0000 7ffb157fa640 1 --2- 172.21.15.98:0/1388570410 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ffb0c067e90 0x7ffb0c06a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.127+0000 7ffb157fa640 1 -- 172.21.15.98:0/1388570410 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7ffb1c1204f0 con 0x7ffb2006c1d0 2024-09-19T08:45:41.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.127+0000 7ffb17fff640 1 --2- 172.21.15.98:0/1388570410 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ffb0c067e90 0x7ffb0c06a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.129 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.131+0000 7ffb17fff640 1 --2- 172.21.15.98:0/1388570410 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ffb0c067e90 0x7ffb0c06a350 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ffb20141130 tx=0x7ffb10008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:41.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.131+0000 7ffb157fa640 1 -- 172.21.15.98:0/1388570410 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ffb1c016110 con 0x7ffb2006c1d0 2024-09-19T08:45:41.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.315+0000 7ffb27000640 1 -- 172.21.15.98:0/1388570410 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7ffb2013f450 con 0x7ffb0c067e90 2024-09-19T08:45:41.317 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:41.318+0000 7ff73a69c640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-19T08:45:41.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.319+0000 7ffb157fa640 1 -- 172.21.15.98:0/1388570410 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7ffb1000f830 con 0x7ffb0c067e90 2024-09-19T08:45:41.318 INFO:teuthology.orchestra.run.smithi098.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-19T08:45:41.323 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-19T08:45:41.324 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath non_existent_subvolume subvolume_1 2024-09-19T08:45:41.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.435+0000 7fda7151a640 1 Processor -- start 2024-09-19T08:45:41.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.435+0000 7fda7151a640 1 -- start start 2024-09-19T08:45:41.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.435+0000 7fda7151a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fda6c145b70 0x7fda6c06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.435+0000 7fda7151a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fda6c06c1d0 0x7fda6c0713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.435+0000 7fda7151a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda6c0718f0 0x7fda6c0cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.435+0000 7fda7151a640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fda6c06c620 con 0x7fda6c145b70 2024-09-19T08:45:41.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.435+0000 7fda7151a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fda6c073530 con 0x7fda6c06c1d0 2024-09-19T08:45:41.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.435+0000 7fda7151a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fda6c073810 con 0x7fda6c0718f0 2024-09-19T08:45:41.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.439+0000 7fda6affd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fda6c145b70 0x7fda6c06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.439+0000 7fda6affd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fda6c145b70 0x7fda6c06bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58160/0 (socket says 172.21.15.98:58160) 2024-09-19T08:45:41.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.439+0000 7fda6affd640 1 -- 172.21.15.98:0/304660778 learned_addr learned my addr 172.21.15.98:0/304660778 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:41.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.439+0000 7fda6a7fc640 1 --2- 172.21.15.98:0/304660778 >> v2:172.21.15.139:3300/0 conn(0x7fda6c06c1d0 0x7fda6c0713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.439+0000 7fda6b7fe640 1 --2- 172.21.15.98:0/304660778 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda6c0718f0 0x7fda6c0cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.439+0000 7fda6affd640 1 -- 172.21.15.98:0/304660778 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda6c0718f0 msgr2=0x7fda6c0cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:41.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.439+0000 7fda6affd640 1 --2- 172.21.15.98:0/304660778 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda6c0718f0 0x7fda6c0cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.439+0000 7fda6affd640 1 -- 172.21.15.98:0/304660778 >> v2:172.21.15.139:3300/0 conn(0x7fda6c06c1d0 msgr2=0x7fda6c0713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:41.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.439+0000 7fda6affd640 1 --2- 172.21.15.98:0/304660778 >> v2:172.21.15.139:3300/0 conn(0x7fda6c06c1d0 0x7fda6c0713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.439+0000 7fda6affd640 1 -- 172.21.15.98:0/304660778 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda6c073af0 con 0x7fda6c145b70 2024-09-19T08:45:41.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.439+0000 7fda6a7fc640 1 --2- 172.21.15.98:0/304660778 >> v2:172.21.15.139:3300/0 conn(0x7fda6c06c1d0 0x7fda6c0713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:41.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.439+0000 7fda6affd640 1 --2- 172.21.15.98:0/304660778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda6c145b70 0x7fda6c06bc90 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fda6c0ccd30 tx=0x7fda5c01c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1b363da89f7664f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:41.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.439+0000 7fda69ffb640 1 -- 172.21.15.98:0/304660778 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c0b0070 con 0x7fda6c145b70 2024-09-19T08:45:41.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.439+0000 7fda69ffb640 1 -- 172.21.15.98:0/304660778 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda5c0ab600 con 0x7fda6c145b70 2024-09-19T08:45:41.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.439+0000 7fda69ffb640 1 -- 172.21.15.98:0/304660778 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c0ab9d0 con 0x7fda6c145b70 2024-09-19T08:45:41.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.439+0000 7fda7151a640 1 -- 172.21.15.98:0/304660778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda6c145b70 msgr2=0x7fda6c06bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:41.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.439+0000 7fda7151a640 1 --2- 172.21.15.98:0/304660778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda6c145b70 0x7fda6c06bc90 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fda6c0ccd30 tx=0x7fda5c01c1d0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.439+0000 7fda7151a640 1 -- 172.21.15.98:0/304660778 shutdown_connections 2024-09-19T08:45:41.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.439+0000 7fda7151a640 1 --2- 172.21.15.98:0/304660778 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda6c0718f0 0x7fda6c0cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.439+0000 7fda7151a640 1 --2- 172.21.15.98:0/304660778 >> v2:172.21.15.139:3300/0 conn(0x7fda6c06c1d0 0x7fda6c0713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.439+0000 7fda7151a640 1 --2- 172.21.15.98:0/304660778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda6c145b70 0x7fda6c06bc90 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.439+0000 7fda7151a640 1 -- 172.21.15.98:0/304660778 >> 172.21.15.98:0/304660778 conn(0x7fda6c0b2010 msgr2=0x7fda6c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:41.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.439+0000 7fda7151a640 1 -- 172.21.15.98:0/304660778 shutdown_connections 2024-09-19T08:45:41.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.439+0000 7fda7151a640 1 -- 172.21.15.98:0/304660778 wait complete. 2024-09-19T08:45:41.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.443+0000 7fda7151a640 1 Processor -- start 2024-09-19T08:45:41.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.443+0000 7fda7151a640 1 -- start start 2024-09-19T08:45:41.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.443+0000 7fda7151a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda6c06c1d0 0x7fda6c13c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.443+0000 7fda7151a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda6c0718f0 0x7fda6c13bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.443+0000 7fda7151a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda6c145b70 0x7fda6c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.443+0000 7fda7151a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fda6c0b3410 con 0x7fda6c0718f0 2024-09-19T08:45:41.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.443+0000 7fda7151a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fda6c0b3590 con 0x7fda6c06c1d0 2024-09-19T08:45:41.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.443+0000 7fda7151a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fda6c0b3710 con 0x7fda6c145b70 2024-09-19T08:45:41.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.443+0000 7fda6a7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda6c0718f0 0x7fda6c13bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.443+0000 7fda6a7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda6c0718f0 0x7fda6c13bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58166/0 (socket says 172.21.15.98:58166) 2024-09-19T08:45:41.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.443+0000 7fda6a7fc640 1 -- 172.21.15.98:0/1642451915 learned_addr learned my addr 172.21.15.98:0/1642451915 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:41.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.443+0000 7fda6b7fe640 1 --2- 172.21.15.98:0/1642451915 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda6c145b70 0x7fda6c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.443+0000 7fda6affd640 1 --2- 172.21.15.98:0/1642451915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda6c06c1d0 0x7fda6c13c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.443+0000 7fda6a7fc640 1 -- 172.21.15.98:0/1642451915 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda6c145b70 msgr2=0x7fda6c13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:41.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.443+0000 7fda6a7fc640 1 --2- 172.21.15.98:0/1642451915 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda6c145b70 0x7fda6c13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.443+0000 7fda6a7fc640 1 -- 172.21.15.98:0/1642451915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda6c06c1d0 msgr2=0x7fda6c13c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:41.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.443+0000 7fda6a7fc640 1 --2- 172.21.15.98:0/1642451915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda6c06c1d0 0x7fda6c13c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.443+0000 7fda6a7fc640 1 -- 172.21.15.98:0/1642451915 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda6c13ed70 con 0x7fda6c0718f0 2024-09-19T08:45:41.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.443+0000 7fda6b7fe640 1 --2- 172.21.15.98:0/1642451915 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fda6c145b70 0x7fda6c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:41.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.443+0000 7fda6affd640 1 --2- 172.21.15.98:0/1642451915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fda6c06c1d0 0x7fda6c13c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:41.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.443+0000 7fda6a7fc640 1 --2- 172.21.15.98:0/1642451915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fda6c0718f0 0x7fda6c13bd40 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fda6000e810 tx=0x7fda6000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:41.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.443+0000 7fda537fe640 1 -- 172.21.15.98:0/1642451915 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6000f040 con 0x7fda6c0718f0 2024-09-19T08:45:41.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.443+0000 7fda537fe640 1 -- 172.21.15.98:0/1642451915 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda60004030 con 0x7fda6c0718f0 2024-09-19T08:45:41.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.443+0000 7fda7151a640 1 -- 172.21.15.98:0/1642451915 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda6c13f050 con 0x7fda6c0718f0 2024-09-19T08:45:41.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.443+0000 7fda537fe640 1 -- 172.21.15.98:0/1642451915 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda60004400 con 0x7fda6c0718f0 2024-09-19T08:45:41.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.443+0000 7fda7151a640 1 -- 172.21.15.98:0/1642451915 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda6c13f8f0 con 0x7fda6c0718f0 2024-09-19T08:45:41.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.447+0000 7fda537fe640 1 -- 172.21.15.98:0/1642451915 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fda6000c6d0 con 0x7fda6c0718f0 2024-09-19T08:45:41.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.447+0000 7fda7151a640 1 -- 172.21.15.98:0/1642451915 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda6c0cef70 con 0x7fda6c0718f0 2024-09-19T08:45:41.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.447+0000 7fda537fe640 1 --2- 172.21.15.98:0/1642451915 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fda58067e90 0x7fda5806a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.447+0000 7fda537fe640 1 -- 172.21.15.98:0/1642451915 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7fda600864d0 con 0x7fda6c0718f0 2024-09-19T08:45:41.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.447+0000 7fda6affd640 1 --2- 172.21.15.98:0/1642451915 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fda58067e90 0x7fda5806a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.447+0000 7fda6affd640 1 --2- 172.21.15.98:0/1642451915 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fda58067e90 0x7fda5806a350 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0ad000 tx=0x7fda5c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:41.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.451+0000 7fda537fe640 1 -- 172.21.15.98:0/1642451915 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fda60017020 con 0x7fda6c0718f0 2024-09-19T08:45:41.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.623+0000 7fda7151a640 1 -- 172.21.15.98:0/1642451915 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7fda6c143180 con 0x7fda58067e90 2024-09-19T08:45:41.626 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:41.626+0000 7ff73a69c640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-19T08:45:41.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.627+0000 7fda537fe640 1 -- 172.21.15.98:0/1642451915 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fda6c143180 con 0x7fda58067e90 2024-09-19T08:45:41.626 INFO:teuthology.orchestra.run.smithi098.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-19T08:45:41.632 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-19T08:45:41.632 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info non_existent_subvolume subvolume_1 2024-09-19T08:45:41.744 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.743+0000 7f145bed4640 1 Processor -- start 2024-09-19T08:45:41.744 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.747+0000 7f145bed4640 1 -- start start 2024-09-19T08:45:41.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.747+0000 7f145bed4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14540d27c0 0x7f14540d3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.747+0000 7f145bed4640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f145415eae0 con 0x7f14541415a0 2024-09-19T08:45:41.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.747+0000 7f145bed4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f145415edc0 con 0x7f14540d38a0 2024-09-19T08:45:41.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.747+0000 7f145bed4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f145415f0a0 con 0x7f14540d27c0 2024-09-19T08:45:41.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.747+0000 7f1459c70640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f14541415a0 0x7f14540d20b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:53066/0 (socket says 172.21.15.98:53066) 2024-09-19T08:45:41.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.747+0000 7f1459c70640 1 -- 172.21.15.98:0/202688191 learned_addr learned my addr 172.21.15.98:0/202688191 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:41.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.747+0000 7f145946f640 1 --2- 172.21.15.98:0/202688191 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14540d27c0 0x7f14540d3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.747+0000 7f1458c6e640 1 -- 172.21.15.98:0/202688191 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2640644440 0 0) 0x7f145415eae0 con 0x7f14541415a0 2024-09-19T08:45:41.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.747+0000 7f1458c6e640 1 -- 172.21.15.98:0/202688191 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1440003530 con 0x7f14541415a0 2024-09-19T08:45:41.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.747+0000 7f1458c6e640 1 -- 172.21.15.98:0/202688191 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 25764103 0 0) 0x7f1440003530 con 0x7f14541415a0 2024-09-19T08:45:41.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.747+0000 7f1458c6e640 1 -- 172.21.15.98:0/202688191 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14540d27c0 msgr2=0x7f14540d3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:41.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.747+0000 7f1458c6e640 1 --2- 172.21.15.98:0/202688191 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14540d27c0 0x7f14540d3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.747+0000 7f1458c6e640 1 -- 172.21.15.98:0/202688191 >> v1:172.21.15.139:6789/0 conn(0x7f14540d38a0 legacy=0x7f14540b8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:41.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.747+0000 7f1458c6e640 1 -- 172.21.15.98:0/202688191 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f145415f380 con 0x7f14541415a0 2024-09-19T08:45:41.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.747+0000 7f1458c6e640 1 -- 172.21.15.98:0/202688191 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f1450001680 con 0x7f14541415a0 2024-09-19T08:45:41.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.747+0000 7f1458c6e640 1 -- 172.21.15.98:0/202688191 >> v1:172.21.15.98:6789/0 conn(0x7f14541415a0 legacy=0x7f14540d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:41.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.747+0000 7f1458c6e640 1 --2- 172.21.15.98:0/202688191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14400039f0 0x7f1440003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.747+0000 7f1458c6e640 1 --2- 172.21.15.98:0/202688191 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14540d27c0 0x7f1440004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.747+0000 7f1458c6e640 1 --2- 172.21.15.98:0/202688191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1440004d50 0x7f14400051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.747+0000 7f1458c6e640 1 -- 172.21.15.98:0/202688191 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f145415f0a0 con 0x7f14400039f0 2024-09-19T08:45:41.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.747+0000 7f1458c6e640 1 -- 172.21.15.98:0/202688191 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f144000ba80 con 0x7f1440004d50 2024-09-19T08:45:41.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.747+0000 7f1458c6e640 1 -- 172.21.15.98:0/202688191 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f144000bc80 con 0x7f14540d27c0 2024-09-19T08:45:41.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.747+0000 7f1459c70640 1 --2- 172.21.15.98:0/202688191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14400039f0 0x7f1440003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.747+0000 7f145a471640 1 --2- 172.21.15.98:0/202688191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1440004d50 0x7f14400051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.747+0000 7f145946f640 1 --2- 172.21.15.98:0/202688191 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14540d27c0 0x7f1440004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.747+0000 7f1459c70640 1 -- 172.21.15.98:0/202688191 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14540d27c0 msgr2=0x7f1440004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:41.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.747+0000 7f1459c70640 1 --2- 172.21.15.98:0/202688191 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14540d27c0 0x7f1440004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.747+0000 7f1459c70640 1 -- 172.21.15.98:0/202688191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1440004d50 msgr2=0x7f14400051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:41.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.747+0000 7f1459c70640 1 --2- 172.21.15.98:0/202688191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1440004d50 0x7f14400051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.747+0000 7f1459c70640 1 -- 172.21.15.98:0/202688191 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1440003530 con 0x7f14400039f0 2024-09-19T08:45:41.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.747+0000 7f145a471640 1 --2- 172.21.15.98:0/202688191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1440004d50 0x7f14400051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:41.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.751+0000 7f1459c70640 1 --2- 172.21.15.98:0/202688191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14400039f0 0x7f1440003e10 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f1450001ec0 tx=0x7f145001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cb159474bf426b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:41.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.751+0000 7f1458c6e640 1 -- 172.21.15.98:0/202688191 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1450007d40 con 0x7f14400039f0 2024-09-19T08:45:41.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.751+0000 7f1458c6e640 1 -- 172.21.15.98:0/202688191 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14500a9070 con 0x7f14400039f0 2024-09-19T08:45:41.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.751+0000 7f1458c6e640 1 -- 172.21.15.98:0/202688191 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1450005ce0 con 0x7f14400039f0 2024-09-19T08:45:41.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.751+0000 7f145bed4640 1 -- 172.21.15.98:0/202688191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14400039f0 msgr2=0x7f1440003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:41.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.751+0000 7f145bed4640 1 --2- 172.21.15.98:0/202688191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14400039f0 0x7f1440003e10 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f1450001ec0 tx=0x7f145001cc60 comp rx=0 tx=0).stop 2024-09-19T08:45:41.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.751+0000 7f145a471640 1 -- 172.21.15.98:0/202688191 reap_dead start 2024-09-19T08:45:41.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.751+0000 7f145bed4640 1 -- 172.21.15.98:0/202688191 shutdown_connections 2024-09-19T08:45:41.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.751+0000 7f145bed4640 1 -- 172.21.15.98:0/202688191 >> 172.21.15.98:0/202688191 conn(0x7f1454074c70 msgr2=0x7f1454075080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:41.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.751+0000 7f145bed4640 1 -- 172.21.15.98:0/202688191 shutdown_connections 2024-09-19T08:45:41.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.751+0000 7f145bed4640 1 -- 172.21.15.98:0/202688191 wait complete. 2024-09-19T08:45:41.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.751+0000 7f145bed4640 1 Processor -- start 2024-09-19T08:45:41.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.751+0000 7f145bed4640 1 -- start start 2024-09-19T08:45:41.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.751+0000 7f145bed4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14400039f0 0x7f145414e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.751+0000 7f145bed4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14540cdae0 0x7f14541f8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.751+0000 7f145bed4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1454164120 0x7f14541645a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.751+0000 7f145bed4640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f145416a110 con 0x7f1454164120 2024-09-19T08:45:41.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.751+0000 7f145bed4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f14540d3ec0 con 0x7f14540cdae0 2024-09-19T08:45:41.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.751+0000 7f145bed4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f145416a290 con 0x7f14400039f0 2024-09-19T08:45:41.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.751+0000 7f145a471640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1454164120 0x7f14541645a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.751+0000 7f145a471640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1454164120 0x7f14541645a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58192/0 (socket says 172.21.15.98:58192) 2024-09-19T08:45:41.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.751+0000 7f145a471640 1 -- 172.21.15.98:0/3823025489 learned_addr learned my addr 172.21.15.98:0/3823025489 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:41.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.755+0000 7f1459c70640 1 --2- 172.21.15.98:0/3823025489 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14400039f0 0x7f145414e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.755+0000 7f145946f640 1 --2- 172.21.15.98:0/3823025489 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14540cdae0 0x7f14541f8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.755+0000 7f145a471640 1 -- 172.21.15.98:0/3823025489 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14400039f0 msgr2=0x7f145414e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:41.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.755+0000 7f145a471640 1 --2- 172.21.15.98:0/3823025489 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14400039f0 0x7f145414e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.755+0000 7f145a471640 1 -- 172.21.15.98:0/3823025489 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14540cdae0 msgr2=0x7f14541f8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:41.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.755+0000 7f145a471640 1 --2- 172.21.15.98:0/3823025489 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14540cdae0 0x7f14541f8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:41.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.755+0000 7f145a471640 1 -- 172.21.15.98:0/3823025489 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f145414ead0 con 0x7f1454164120 2024-09-19T08:45:41.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.755+0000 7f145a471640 1 --2- 172.21.15.98:0/3823025489 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1454164120 0x7f14541645a0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f145415e9a0 tx=0x7f144c00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:41.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.755+0000 7f144a7fc640 1 -- 172.21.15.98:0/3823025489 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c006b60 con 0x7f1454164120 2024-09-19T08:45:41.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.755+0000 7f144a7fc640 1 -- 172.21.15.98:0/3823025489 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f144c006d20 con 0x7f1454164120 2024-09-19T08:45:41.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.755+0000 7f145bed4640 1 -- 172.21.15.98:0/3823025489 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f145414edb0 con 0x7f1454164120 2024-09-19T08:45:41.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.755+0000 7f144a7fc640 1 -- 172.21.15.98:0/3823025489 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144c003980 con 0x7f1454164120 2024-09-19T08:45:41.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.755+0000 7f145bed4640 1 -- 172.21.15.98:0/3823025489 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14541f7e50 con 0x7f1454164120 2024-09-19T08:45:41.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.755+0000 7f145bed4640 1 -- 172.21.15.98:0/3823025489 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14541f8050 con 0x7f1454164120 2024-09-19T08:45:41.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.755+0000 7f144a7fc640 1 -- 172.21.15.98:0/3823025489 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f144c01a020 con 0x7f1454164120 2024-09-19T08:45:41.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.755+0000 7f144a7fc640 1 --2- 172.21.15.98:0/3823025489 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1438067e60 0x7f143806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:41.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.755+0000 7f144a7fc640 1 -- 172.21.15.98:0/3823025489 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7f144c085c20 con 0x7f1454164120 2024-09-19T08:45:41.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.755+0000 7f1459c70640 1 --2- 172.21.15.98:0/3823025489 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1438067e60 0x7f143806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:41.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.755+0000 7f1459c70640 1 --2- 172.21.15.98:0/3823025489 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1438067e60 0x7f143806a320 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f14500ab000 tx=0x7f14500b1040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:41.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.759+0000 7f144a7fc640 1 -- 172.21.15.98:0/3823025489 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f14541f8050 con 0x7f1454164120 2024-09-19T08:45:41.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.947+0000 7f145bed4640 1 -- 172.21.15.98:0/3823025489 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7f14540b8790 con 0x7f1438067e60 2024-09-19T08:45:41.949 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:41.950+0000 7ff73a69c640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-19T08:45:41.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:41.951+0000 7f144a7fc640 1 -- 172.21.15.98:0/3823025489 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f14540b8790 con 0x7f1438067e60 2024-09-19T08:45:41.950 INFO:teuthology.orchestra.run.smithi098.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-19T08:45:41.956 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-19T08:45:41.956 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume resize non_existent_subvolume subvolname_1 inf 2024-09-19T08:45:42.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.079+0000 7fe8e7978640 1 Processor -- start 2024-09-19T08:45:42.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.079+0000 7fe8e7978640 1 -- start start 2024-09-19T08:45:42.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.079+0000 7fe8e7978640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8e015c290 0x7fe8e00cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:42.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.079+0000 7fe8e7978640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe8e00cf700 0x7fe8e015ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:42.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.079+0000 7fe8e7978640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8e01608c0 con 0x7fe8e006d4c0 2024-09-19T08:45:42.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.079+0000 7fe8e7978640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe8e0160e80 con 0x7fe8e00cf700 2024-09-19T08:45:42.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.079+0000 7fe8e7978640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe8e00cd510 con 0x7fe8e015c290 2024-09-19T08:45:42.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.079+0000 7fe8e5f15640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fe8e006d4c0 0x7fe8e00b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:53074/0 (socket says 172.21.15.98:53074) 2024-09-19T08:45:42.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.079+0000 7fe8e5f15640 1 -- 172.21.15.98:0/605248466 learned_addr learned my addr 172.21.15.98:0/605248466 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:42.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.079+0000 7fe8e4f13640 1 --2- 172.21.15.98:0/605248466 >> v2:172.21.15.139:3300/0 conn(0x7fe8e00cf700 0x7fe8e015ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:42.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.079+0000 7fe8e5714640 1 --2- 172.21.15.98:0/605248466 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8e015c290 0x7fe8e00cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:42.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.079+0000 7fe8d7fff640 1 -- 172.21.15.98:0/605248466 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1510102293 0 0) 0x7fe8e01608c0 con 0x7fe8e006d4c0 2024-09-19T08:45:42.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.079+0000 7fe8d7fff640 1 -- 172.21.15.98:0/605248466 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe8cc003530 con 0x7fe8e006d4c0 2024-09-19T08:45:42.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.079+0000 7fe8d7fff640 1 -- 172.21.15.98:0/605248466 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 128605012 0 0) 0x7fe8cc003530 con 0x7fe8e006d4c0 2024-09-19T08:45:42.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.079+0000 7fe8d7fff640 1 -- 172.21.15.98:0/605248466 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8e015c290 msgr2=0x7fe8e00cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:42.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.079+0000 7fe8d7fff640 1 --2- 172.21.15.98:0/605248466 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8e015c290 0x7fe8e00cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:42.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.079+0000 7fe8d7fff640 1 -- 172.21.15.98:0/605248466 >> v2:172.21.15.139:3300/0 conn(0x7fe8e00cf700 msgr2=0x7fe8e015ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:42.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.079+0000 7fe8d7fff640 1 --2- 172.21.15.98:0/605248466 >> v2:172.21.15.139:3300/0 conn(0x7fe8e00cf700 0x7fe8e015ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:42.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.079+0000 7fe8d7fff640 1 -- 172.21.15.98:0/605248466 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e0160ba0 con 0x7fe8e006d4c0 2024-09-19T08:45:42.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.079+0000 7fe8d7fff640 1 -- 172.21.15.98:0/605248466 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fe8d8001a00 con 0x7fe8e006d4c0 2024-09-19T08:45:42.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.079+0000 7fe8d7fff640 1 -- 172.21.15.98:0/605248466 >> v1:172.21.15.98:6789/0 conn(0x7fe8e006d4c0 legacy=0x7fe8e00b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:42.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.079+0000 7fe8d7fff640 1 --2- 172.21.15.98:0/605248466 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8e015c290 0x7fe8cc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:42.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.079+0000 7fe8d7fff640 1 --2- 172.21.15.98:0/605248466 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8cc003f60 0x7fe8cc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:42.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.079+0000 7fe8d7fff640 1 --2- 172.21.15.98:0/605248466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8cc004d50 0x7fe8cc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:42.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.079+0000 7fe8d7fff640 1 -- 172.21.15.98:0/605248466 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e00cd510 con 0x7fe8cc004d50 2024-09-19T08:45:42.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.079+0000 7fe8d7fff640 1 -- 172.21.15.98:0/605248466 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe8cc00b9d0 con 0x7fe8cc003f60 2024-09-19T08:45:42.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.079+0000 7fe8d7fff640 1 -- 172.21.15.98:0/605248466 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe8cc00bbd0 con 0x7fe8e015c290 2024-09-19T08:45:42.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.079+0000 7fe8e4f13640 1 --2- 172.21.15.98:0/605248466 >> v2:172.21.15.139:3300/0 conn(0x7fe8e00cf700 0x7fe8e015ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:42.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.079+0000 7fe8e5f15640 1 --2- 172.21.15.98:0/605248466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8cc004d50 0x7fe8cc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:42.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.079+0000 7fe8e5f15640 1 -- 172.21.15.98:0/605248466 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8e015c290 msgr2=0x7fe8cc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:42.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.079+0000 7fe8e5f15640 1 --2- 172.21.15.98:0/605248466 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8e015c290 0x7fe8cc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:42.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.079+0000 7fe8e5f15640 1 -- 172.21.15.98:0/605248466 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8cc003f60 msgr2=0x7fe8cc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:42.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.079+0000 7fe8e5f15640 1 --2- 172.21.15.98:0/605248466 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8cc003f60 0x7fe8cc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:42.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.079+0000 7fe8e5f15640 1 -- 172.21.15.98:0/605248466 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8cc003530 con 0x7fe8cc004d50 2024-09-19T08:45:42.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.083+0000 7fe8e5f15640 1 --2- 172.21.15.98:0/605248466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8cc004d50 0x7fe8cc0051d0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fe8d800a880 tx=0x7fe8d801d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d39745a392705d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:42.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.083+0000 7fe8d7fff640 1 -- 172.21.15.98:0/605248466 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d800a4a0 con 0x7fe8cc004d50 2024-09-19T08:45:42.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.083+0000 7fe8d7fff640 1 -- 172.21.15.98:0/605248466 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d80ad630 con 0x7fe8cc004d50 2024-09-19T08:45:42.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.083+0000 7fe8d7fff640 1 -- 172.21.15.98:0/605248466 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d80ada20 con 0x7fe8cc004d50 2024-09-19T08:45:42.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.083+0000 7fe8e7978640 1 -- 172.21.15.98:0/605248466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8cc004d50 msgr2=0x7fe8cc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:42.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.083+0000 7fe8e7978640 1 --2- 172.21.15.98:0/605248466 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8cc004d50 0x7fe8cc0051d0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fe8d800a880 tx=0x7fe8d801d3a0 comp rx=0 tx=0).stop 2024-09-19T08:45:42.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.083+0000 7fe8e5f15640 1 -- 172.21.15.98:0/605248466 reap_dead start 2024-09-19T08:45:42.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.083+0000 7fe8e7978640 1 -- 172.21.15.98:0/605248466 shutdown_connections 2024-09-19T08:45:42.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.083+0000 7fe8e7978640 1 -- 172.21.15.98:0/605248466 >> 172.21.15.98:0/605248466 conn(0x7fe8e006c090 msgr2=0x7fe8e006c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:42.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.083+0000 7fe8e7978640 1 -- 172.21.15.98:0/605248466 shutdown_connections 2024-09-19T08:45:42.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.083+0000 7fe8e7978640 1 -- 172.21.15.98:0/605248466 wait complete. 2024-09-19T08:45:42.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.083+0000 7fe8e7978640 1 Processor -- start 2024-09-19T08:45:42.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.083+0000 7fe8e7978640 1 -- start start 2024-09-19T08:45:42.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.087+0000 7fe8e7978640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8cc004d50 0x7fe8e014ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:42.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.087+0000 7fe8e7978640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8e00ae8b0 0x7fe8e00aed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:42.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.087+0000 7fe8e7978640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8e0155c60 0x7fe8e01560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:42.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.087+0000 7fe8e7978640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e015c7f0 con 0x7fe8e00ae8b0 2024-09-19T08:45:42.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.087+0000 7fe8e7978640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e015c670 con 0x7fe8cc004d50 2024-09-19T08:45:42.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.087+0000 7fe8e7978640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe8e0160490 con 0x7fe8e0155c60 2024-09-19T08:45:42.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.087+0000 7fe8e4f13640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8e00ae8b0 0x7fe8e00aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:42.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.087+0000 7fe8e4f13640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8e00ae8b0 0x7fe8e00aed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58202/0 (socket says 172.21.15.98:58202) 2024-09-19T08:45:42.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.087+0000 7fe8e4f13640 1 -- 172.21.15.98:0/868387185 learned_addr learned my addr 172.21.15.98:0/868387185 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:42.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.087+0000 7fe8e5714640 1 --2- 172.21.15.98:0/868387185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8cc004d50 0x7fe8e014ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:42.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.087+0000 7fe8e4f13640 1 -- 172.21.15.98:0/868387185 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8e0155c60 msgr2=0x7fe8e01560e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:42.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.087+0000 7fe8e4f13640 1 --2- 172.21.15.98:0/868387185 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8e0155c60 0x7fe8e01560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:42.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.087+0000 7fe8e4f13640 1 -- 172.21.15.98:0/868387185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8cc004d50 msgr2=0x7fe8e014ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:42.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.087+0000 7fe8e4f13640 1 --2- 172.21.15.98:0/868387185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8cc004d50 0x7fe8e014ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:42.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.087+0000 7fe8e4f13640 1 -- 172.21.15.98:0/868387185 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e0156920 con 0x7fe8e00ae8b0 2024-09-19T08:45:42.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.087+0000 7fe8e4f13640 1 --2- 172.21.15.98:0/868387185 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8e00ae8b0 0x7fe8e00aed30 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fe8d000e7e0 tx=0x7fe8d000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:42.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.087+0000 7fe8d5ffb640 1 -- 172.21.15.98:0/868387185 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0017070 con 0x7fe8e00ae8b0 2024-09-19T08:45:42.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.087+0000 7fe8d5ffb640 1 -- 172.21.15.98:0/868387185 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d000c650 con 0x7fe8e00ae8b0 2024-09-19T08:45:42.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.087+0000 7fe8e7978640 1 -- 172.21.15.98:0/868387185 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e0156c00 con 0x7fe8e00ae8b0 2024-09-19T08:45:42.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.087+0000 7fe8d5ffb640 1 -- 172.21.15.98:0/868387185 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d000f040 con 0x7fe8e00ae8b0 2024-09-19T08:45:42.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.087+0000 7fe8e7978640 1 -- 172.21.15.98:0/868387185 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e01571c0 con 0x7fe8e00ae8b0 2024-09-19T08:45:42.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.087+0000 7fe8e7978640 1 -- 172.21.15.98:0/868387185 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8e00af270 con 0x7fe8e00ae8b0 2024-09-19T08:45:42.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.087+0000 7fe8d5ffb640 1 -- 172.21.15.98:0/868387185 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fe8d001a070 con 0x7fe8e00ae8b0 2024-09-19T08:45:42.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.087+0000 7fe8d5ffb640 1 --2- 172.21.15.98:0/868387185 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe8c4067e90 0x7fe8c406a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:42.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.087+0000 7fe8d5ffb640 1 -- 172.21.15.98:0/868387185 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7fe8d0086320 con 0x7fe8e00ae8b0 2024-09-19T08:45:42.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.087+0000 7fe8e5714640 1 --2- 172.21.15.98:0/868387185 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe8c4067e90 0x7fe8c406a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:42.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.087+0000 7fe8e5714640 1 --2- 172.21.15.98:0/868387185 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe8c4067e90 0x7fe8c406a350 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc0047c0 tx=0x7fe8dc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:42.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.091+0000 7fe8d5ffb640 1 -- 172.21.15.98:0/868387185 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe8d0056550 con 0x7fe8e00ae8b0 2024-09-19T08:45:42.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.283+0000 7fe8e7978640 1 -- 172.21.15.98:0/868387185 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume resize", "vol_name": "non_existent_subvolume", "sub_name": "subvolname_1", "new_size": "inf", "target": ["mon-mgr", ""]}) -- 0x7fe8e0160800 con 0x7fe8c4067e90 2024-09-19T08:45:42.286 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:42.286+0000 7ff73a69c640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-19T08:45:42.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.287+0000 7fe8d5ffb640 1 -- 172.21.15.98:0/868387185 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fe8e0160800 con 0x7fe8c4067e90 2024-09-19T08:45:42.286 INFO:teuthology.orchestra.run.smithi098.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-19T08:45:42.292 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-19T08:45:42.292 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume pin non_existent_subvolume subvolname_1 export 1 2024-09-19T08:45:42.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.411+0000 7f921e5ba640 1 Processor -- start 2024-09-19T08:45:42.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.411+0000 7f921e5ba640 1 -- start start 2024-09-19T08:45:42.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.411+0000 7f921e5ba640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f921815c290 0x7f92180cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:42.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.411+0000 7f921e5ba640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92181608c0 con 0x7f92180cf700 2024-09-19T08:45:42.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.411+0000 7f921e5ba640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9218160ba0 con 0x7f921806d4c0 2024-09-19T08:45:42.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.411+0000 7f921e5ba640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f9218160e80 con 0x7f921815c290 2024-09-19T08:45:42.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.415+0000 7f92177fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f92180cf700 0x7f921815ff50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:53078/0 (socket says 172.21.15.98:53078) 2024-09-19T08:45:42.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.415+0000 7f92177fe640 1 -- 172.21.15.98:0/2863285810 learned_addr learned my addr 172.21.15.98:0/2863285810 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:42.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.415+0000 7f9216ffd640 1 -- 172.21.15.98:0/2863285810 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1679316208 0 0) 0x7f92181608c0 con 0x7f92180cf700 2024-09-19T08:45:42.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.415+0000 7f9217fff640 1 --2- 172.21.15.98:0/2863285810 >> v2:172.21.15.139:3300/0 conn(0x7f921815c290 0x7f92180cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:42.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.415+0000 7f9216ffd640 1 -- 172.21.15.98:0/2863285810 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9200003530 con 0x7f92180cf700 2024-09-19T08:45:42.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.415+0000 7f9216ffd640 1 -- 172.21.15.98:0/2863285810 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3615091576 0 0) 0x7f9200003530 con 0x7f92180cf700 2024-09-19T08:45:42.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.415+0000 7f9216ffd640 1 -- 172.21.15.98:0/2863285810 >> v2:172.21.15.139:3300/0 conn(0x7f921815c290 msgr2=0x7f92180cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:42.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.415+0000 7f9216ffd640 1 --2- 172.21.15.98:0/2863285810 >> v2:172.21.15.139:3300/0 conn(0x7f921815c290 0x7f92180cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:42.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.415+0000 7f9216ffd640 1 -- 172.21.15.98:0/2863285810 >> v1:172.21.15.139:6789/0 conn(0x7f921806d4c0 legacy=0x7f921806d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:42.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.415+0000 7f9216ffd640 1 -- 172.21.15.98:0/2863285810 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92180cd1a0 con 0x7f92180cf700 2024-09-19T08:45:42.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.415+0000 7f9216ffd640 1 -- 172.21.15.98:0/2863285810 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f920c001680 con 0x7f92180cf700 2024-09-19T08:45:42.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.415+0000 7f9216ffd640 1 -- 172.21.15.98:0/2863285810 >> v1:172.21.15.98:6789/0 conn(0x7f92180cf700 legacy=0x7f921815ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:42.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.415+0000 7f9216ffd640 1 --2- 172.21.15.98:0/2863285810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92000039c0 0x7f9200003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:42.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.415+0000 7f9216ffd640 1 --2- 172.21.15.98:0/2863285810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9200004810 0x7f9200004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:42.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.415+0000 7f9216ffd640 1 --2- 172.21.15.98:0/2863285810 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92000051d0 0x7f920000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:42.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.415+0000 7f9216ffd640 1 -- 172.21.15.98:0/2863285810 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f920000c140 con 0x7f9200004810 2024-09-19T08:45:42.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.415+0000 7f9216ffd640 1 -- 172.21.15.98:0/2863285810 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f920000c2f0 con 0x7f92000039c0 2024-09-19T08:45:42.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.415+0000 7f9216ffd640 1 -- 172.21.15.98:0/2863285810 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f920000c4f0 con 0x7f92000051d0 2024-09-19T08:45:42.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.415+0000 7f92177fe640 1 --2- 172.21.15.98:0/2863285810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9200004810 0x7f9200004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:42.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.415+0000 7f921cb57640 1 --2- 172.21.15.98:0/2863285810 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92000051d0 0x7f920000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:42.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.415+0000 7f92177fe640 1 -- 172.21.15.98:0/2863285810 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92000051d0 msgr2=0x7f920000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:42.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.415+0000 7f92177fe640 1 --2- 172.21.15.98:0/2863285810 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f92000051d0 0x7f920000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:42.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.415+0000 7f92177fe640 1 -- 172.21.15.98:0/2863285810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92000039c0 msgr2=0x7f9200003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:42.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.415+0000 7f92177fe640 1 --2- 172.21.15.98:0/2863285810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f92000039c0 0x7f9200003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:42.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.415+0000 7f92177fe640 1 -- 172.21.15.98:0/2863285810 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9200003530 con 0x7f9200004810 2024-09-19T08:45:42.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.415+0000 7f921cb57640 1 -- 172.21.15.98:0/2863285810 reap_dead start 2024-09-19T08:45:42.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.415+0000 7f92177fe640 1 --2- 172.21.15.98:0/2863285810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9200004810 0x7f9200004c90 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f920c00c1c0 tx=0x7f920c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343e46e57848a7ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:42.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.415+0000 7f9216ffd640 1 -- 172.21.15.98:0/2863285810 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920c007d00 con 0x7f9200004810 2024-09-19T08:45:42.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.415+0000 7f9216ffd640 1 -- 172.21.15.98:0/2863285810 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f920c0a9020 con 0x7f9200004810 2024-09-19T08:45:42.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.415+0000 7f9216ffd640 1 -- 172.21.15.98:0/2863285810 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920c005b00 con 0x7f9200004810 2024-09-19T08:45:42.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.415+0000 7f921e5ba640 1 -- 172.21.15.98:0/2863285810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9200004810 msgr2=0x7f9200004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:42.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.415+0000 7f921e5ba640 1 --2- 172.21.15.98:0/2863285810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9200004810 0x7f9200004c90 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f920c00c1c0 tx=0x7f920c01cc60 comp rx=0 tx=0).stop 2024-09-19T08:45:42.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.415+0000 7f921e5ba640 1 -- 172.21.15.98:0/2863285810 shutdown_connections 2024-09-19T08:45:42.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.415+0000 7f921e5ba640 1 --2- 172.21.15.98:0/2863285810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9200004810 0x7f9200004c90 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:42.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.415+0000 7f921e5ba640 1 -- 172.21.15.98:0/2863285810 >> 172.21.15.98:0/2863285810 conn(0x7f921806c090 msgr2=0x7f921806c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:42.418 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.415+0000 7f921e5ba640 1 -- 172.21.15.98:0/2863285810 shutdown_connections 2024-09-19T08:45:42.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.419+0000 7f921e5ba640 1 -- 172.21.15.98:0/2863285810 wait complete. 2024-09-19T08:45:42.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.419+0000 7f921e5ba640 1 Processor -- start 2024-09-19T08:45:42.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.419+0000 7f921e5ba640 1 -- start start 2024-09-19T08:45:42.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.419+0000 7f921e5ba640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9200004810 0x7f92180af0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:42.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.419+0000 7f921e5ba640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9218150170 0x7f921815b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:42.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.419+0000 7f921e5ba640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f921815b5c0 0x7f9218154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:42.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.419+0000 7f921e5ba640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9218160320 con 0x7f9200004810 2024-09-19T08:45:42.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.419+0000 7f921e5ba640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f92181601a0 con 0x7f921815b5c0 2024-09-19T08:45:42.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.419+0000 7f921e5ba640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f92181604a0 con 0x7f9218150170 2024-09-19T08:45:42.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.419+0000 7f9217fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9200004810 0x7f92180af0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:42.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.419+0000 7f9217fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9200004810 0x7f92180af0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58214/0 (socket says 172.21.15.98:58214) 2024-09-19T08:45:42.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.419+0000 7f9217fff640 1 -- 172.21.15.98:0/3104109171 learned_addr learned my addr 172.21.15.98:0/3104109171 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:42.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.419+0000 7f921cb57640 1 --2- 172.21.15.98:0/3104109171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f921815b5c0 0x7f9218154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:42.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.419+0000 7f92177fe640 1 --2- 172.21.15.98:0/3104109171 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9218150170 0x7f921815b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:42.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.419+0000 7f9217fff640 1 -- 172.21.15.98:0/3104109171 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9218150170 msgr2=0x7f921815b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:42.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.419+0000 7f9217fff640 1 --2- 172.21.15.98:0/3104109171 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9218150170 0x7f921815b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:42.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.419+0000 7f9217fff640 1 -- 172.21.15.98:0/3104109171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f921815b5c0 msgr2=0x7f9218154700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:42.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.419+0000 7f9217fff640 1 --2- 172.21.15.98:0/3104109171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f921815b5c0 0x7f9218154700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:42.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.419+0000 7f9217fff640 1 -- 172.21.15.98:0/3104109171 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9218150660 con 0x7f9200004810 2024-09-19T08:45:42.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.419+0000 7f9217fff640 1 --2- 172.21.15.98:0/3104109171 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9200004810 0x7f92180af0b0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f920800e840 tx=0x7f920800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:42.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.419+0000 7f9214ff9640 1 -- 172.21.15.98:0/3104109171 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9208017070 con 0x7f9200004810 2024-09-19T08:45:42.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.419+0000 7f9214ff9640 1 -- 172.21.15.98:0/3104109171 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f920800c940 con 0x7f9200004810 2024-09-19T08:45:42.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.419+0000 7f9214ff9640 1 -- 172.21.15.98:0/3104109171 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920800cd30 con 0x7f9200004810 2024-09-19T08:45:42.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.419+0000 7f921e5ba640 1 -- 172.21.15.98:0/3104109171 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9218158820 con 0x7f9200004810 2024-09-19T08:45:42.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.419+0000 7f921e5ba640 1 -- 172.21.15.98:0/3104109171 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9218158dc0 con 0x7f9200004810 2024-09-19T08:45:42.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.419+0000 7f921e5ba640 1 -- 172.21.15.98:0/3104109171 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f921815b9d0 con 0x7f9200004810 2024-09-19T08:45:42.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.423+0000 7f9214ff9640 1 -- 172.21.15.98:0/3104109171 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f920801a070 con 0x7f9200004810 2024-09-19T08:45:42.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.423+0000 7f9214ff9640 1 --2- 172.21.15.98:0/3104109171 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9204067e60 0x7f920406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:42.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.423+0000 7f9214ff9640 1 -- 172.21.15.98:0/3104109171 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7f92080865b0 con 0x7f9200004810 2024-09-19T08:45:42.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.423+0000 7f92177fe640 1 --2- 172.21.15.98:0/3104109171 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9204067e60 0x7f920406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:42.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.423+0000 7f92177fe640 1 --2- 172.21.15.98:0/3104109171 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9204067e60 0x7f920406a320 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f920c0aa000 tx=0x7f92180ce510 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:42.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.427+0000 7f9214ff9640 1 -- 172.21.15.98:0/3104109171 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f921815b9d0 con 0x7f9200004810 2024-09-19T08:45:42.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.619+0000 7f921e5ba640 1 -- 172.21.15.98:0/3104109171 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume pin", "vol_name": "non_existent_subvolume", "sub_name": "subvolname_1", "pin_type": "export", "pin_setting": "1", "target": ["mon-mgr", ""]}) -- 0x7f921815f050 con 0x7f9204067e60 2024-09-19T08:45:42.621 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:42.622+0000 7ff73a69c640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-19T08:45:42.622 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.623+0000 7f9214ff9640 1 -- 172.21.15.98:0/3104109171 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f921815f050 con 0x7f9204067e60 2024-09-19T08:45:42.622 INFO:teuthology.orchestra.run.smithi098.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-19T08:45:42.628 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-19T08:45:42.629 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls non_existent_subvolume 2024-09-19T08:45:42.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.755+0000 7f2b22ee1640 1 Processor -- start 2024-09-19T08:45:42.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.755+0000 7f2b22ee1640 1 -- start start 2024-09-19T08:45:42.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.755+0000 7f2b22ee1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2b1c15fe50 0x7f2b1c0711c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:42.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.755+0000 7f2b22ee1640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b1c071700 con 0x7f2b1c160230 2024-09-19T08:45:42.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.755+0000 7f2b22ee1640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b1c15a1d0 con 0x7f2b1c0d1330 2024-09-19T08:45:42.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.755+0000 7f2b22ee1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f2b1c073000 con 0x7f2b1c15fe50 2024-09-19T08:45:42.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.755+0000 7f2b1bfff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f2b1c160230 0x7f2b1c15c830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:53082/0 (socket says 172.21.15.98:53082) 2024-09-19T08:45:42.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.755+0000 7f2b1bfff640 1 -- 172.21.15.98:0/2765758128 learned_addr learned my addr 172.21.15.98:0/2765758128 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:42.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.755+0000 7f2b20c7d640 1 --2- 172.21.15.98:0/2765758128 >> v2:172.21.15.139:3300/0 conn(0x7f2b1c15fe50 0x7f2b1c0711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:42.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.755+0000 7f2b1b7fe640 1 -- 172.21.15.98:0/2765758128 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2909941090 0 0) 0x7f2b1c071700 con 0x7f2b1c160230 2024-09-19T08:45:42.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.755+0000 7f2b1b7fe640 1 -- 172.21.15.98:0/2765758128 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b04003530 con 0x7f2b1c160230 2024-09-19T08:45:42.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.755+0000 7f2b1b7fe640 1 -- 172.21.15.98:0/2765758128 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1309832967 0 0) 0x7f2b04003530 con 0x7f2b1c160230 2024-09-19T08:45:42.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.755+0000 7f2b1b7fe640 1 -- 172.21.15.98:0/2765758128 >> v2:172.21.15.139:3300/0 conn(0x7f2b1c15fe50 msgr2=0x7f2b1c0711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:42.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.755+0000 7f2b1b7fe640 1 --2- 172.21.15.98:0/2765758128 >> v2:172.21.15.139:3300/0 conn(0x7f2b1c15fe50 0x7f2b1c0711c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:42.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.755+0000 7f2b1b7fe640 1 -- 172.21.15.98:0/2765758128 >> v1:172.21.15.139:6789/0 conn(0x7f2b1c0d1330 legacy=0x7f2b1c15cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:42.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.755+0000 7f2b1b7fe640 1 -- 172.21.15.98:0/2765758128 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b1c0732e0 con 0x7f2b1c160230 2024-09-19T08:45:42.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.755+0000 7f2b1b7fe640 1 -- 172.21.15.98:0/2765758128 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f2b10001680 con 0x7f2b1c160230 2024-09-19T08:45:42.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.755+0000 7f2b1b7fe640 1 -- 172.21.15.98:0/2765758128 >> v1:172.21.15.98:6789/0 conn(0x7f2b1c160230 legacy=0x7f2b1c15c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:42.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.755+0000 7f2b1b7fe640 1 --2- 172.21.15.98:0/2765758128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b040039c0 0x7f2b04003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:42.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.755+0000 7f2b1b7fe640 1 --2- 172.21.15.98:0/2765758128 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2b04004810 0x7f2b04004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:42.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.755+0000 7f2b1b7fe640 1 --2- 172.21.15.98:0/2765758128 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b040051d0 0x7f2b0400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:42.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.755+0000 7f2b1b7fe640 1 -- 172.21.15.98:0/2765758128 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c073000 con 0x7f2b040039c0 2024-09-19T08:45:42.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.755+0000 7f2b1b7fe640 1 -- 172.21.15.98:0/2765758128 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0400c070 con 0x7f2b040051d0 2024-09-19T08:45:42.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.755+0000 7f2b1b7fe640 1 -- 172.21.15.98:0/2765758128 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2b0400c270 con 0x7f2b04004810 2024-09-19T08:45:42.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.755+0000 7f2b20c7d640 1 --2- 172.21.15.98:0/2765758128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b040039c0 0x7f2b04003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:42.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.755+0000 7f2b2147e640 1 --2- 172.21.15.98:0/2765758128 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b040051d0 0x7f2b0400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:42.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.755+0000 7f2b1bfff640 1 --2- 172.21.15.98:0/2765758128 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2b04004810 0x7f2b04004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:42.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.755+0000 7f2b20c7d640 1 -- 172.21.15.98:0/2765758128 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2b04004810 msgr2=0x7f2b04004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:42.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.755+0000 7f2b20c7d640 1 --2- 172.21.15.98:0/2765758128 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2b04004810 0x7f2b04004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:42.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.755+0000 7f2b20c7d640 1 -- 172.21.15.98:0/2765758128 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b040051d0 msgr2=0x7f2b0400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:42.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.755+0000 7f2b20c7d640 1 --2- 172.21.15.98:0/2765758128 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b040051d0 0x7f2b0400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:42.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.755+0000 7f2b2147e640 1 -- 172.21.15.98:0/2765758128 reap_dead start 2024-09-19T08:45:42.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.755+0000 7f2b20c7d640 1 -- 172.21.15.98:0/2765758128 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b04003530 con 0x7f2b040039c0 2024-09-19T08:45:42.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.759+0000 7f2b20c7d640 1 --2- 172.21.15.98:0/2765758128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b040039c0 0x7f2b04003de0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c009fc0 tx=0x7f2b0c01cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=775b84edd57bf979 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:42.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.759+0000 7f2b1b7fe640 1 -- 172.21.15.98:0/2765758128 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c0b2070 con 0x7f2b040039c0 2024-09-19T08:45:42.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.759+0000 7f2b1b7fe640 1 -- 172.21.15.98:0/2765758128 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b0c0a5b50 con 0x7f2b040039c0 2024-09-19T08:45:42.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.759+0000 7f2b1b7fe640 1 -- 172.21.15.98:0/2765758128 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c009c10 con 0x7f2b040039c0 2024-09-19T08:45:42.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.759+0000 7f2b22ee1640 1 -- 172.21.15.98:0/2765758128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b040039c0 msgr2=0x7f2b04003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:42.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.759+0000 7f2b22ee1640 1 --2- 172.21.15.98:0/2765758128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b040039c0 0x7f2b04003de0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c009fc0 tx=0x7f2b0c01cc50 comp rx=0 tx=0).stop 2024-09-19T08:45:42.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.759+0000 7f2b22ee1640 1 -- 172.21.15.98:0/2765758128 shutdown_connections 2024-09-19T08:45:42.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.759+0000 7f2b22ee1640 1 --2- 172.21.15.98:0/2765758128 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b040039c0 0x7f2b04003de0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:42.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.759+0000 7f2b22ee1640 1 -- 172.21.15.98:0/2765758128 >> 172.21.15.98:0/2765758128 conn(0x7f2b1c0d23b0 msgr2=0x7f2b1c0d27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:42.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.759+0000 7f2b22ee1640 1 -- 172.21.15.98:0/2765758128 shutdown_connections 2024-09-19T08:45:42.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.759+0000 7f2b22ee1640 1 -- 172.21.15.98:0/2765758128 wait complete. 2024-09-19T08:45:42.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.759+0000 7f2b22ee1640 1 Processor -- start 2024-09-19T08:45:42.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.759+0000 7f2b22ee1640 1 -- start start 2024-09-19T08:45:42.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.763+0000 7f2b22ee1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b040039c0 0x7f2b1c0b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:42.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.763+0000 7f2b22ee1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b1c0ab1f0 0x7f2b1c156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:42.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.763+0000 7f2b22ee1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2b1c0ad910 0x7f2b1c0add90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:42.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.763+0000 7f2b22ee1640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c073e60 con 0x7f2b1c0ab1f0 2024-09-19T08:45:42.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.763+0000 7f2b22ee1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c06dca0 con 0x7f2b040039c0 2024-09-19T08:45:42.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.763+0000 7f2b22ee1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2b1c160610 con 0x7f2b1c0ad910 2024-09-19T08:45:42.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.763+0000 7f2b1bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b1c0ab1f0 0x7f2b1c156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:42.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.763+0000 7f2b1bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b1c0ab1f0 0x7f2b1c156a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58230/0 (socket says 172.21.15.98:58230) 2024-09-19T08:45:42.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.763+0000 7f2b1bfff640 1 -- 172.21.15.98:0/331047462 learned_addr learned my addr 172.21.15.98:0/331047462 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:42.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.763+0000 7f2b20c7d640 1 --2- 172.21.15.98:0/331047462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b040039c0 0x7f2b1c0b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:42.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.763+0000 7f2b2147e640 1 --2- 172.21.15.98:0/331047462 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2b1c0ad910 0x7f2b1c0add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:42.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.763+0000 7f2b1bfff640 1 -- 172.21.15.98:0/331047462 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2b1c0ad910 msgr2=0x7f2b1c0add90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:42.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.763+0000 7f2b1bfff640 1 --2- 172.21.15.98:0/331047462 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2b1c0ad910 0x7f2b1c0add90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:42.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.763+0000 7f2b1bfff640 1 -- 172.21.15.98:0/331047462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b040039c0 msgr2=0x7f2b1c0b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:42.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.763+0000 7f2b1bfff640 1 --2- 172.21.15.98:0/331047462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b040039c0 0x7f2b1c0b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:42.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.763+0000 7f2b1bfff640 1 -- 172.21.15.98:0/331047462 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b1c156f70 con 0x7f2b1c0ab1f0 2024-09-19T08:45:42.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.763+0000 7f2b2147e640 1 --2- 172.21.15.98:0/331047462 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2b1c0ad910 0x7f2b1c0add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:42.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.763+0000 7f2b1bfff640 1 --2- 172.21.15.98:0/331047462 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2b1c0ab1f0 0x7f2b1c156a30 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f2b10000c10 tx=0x7f2b1000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:42.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.763+0000 7f2b197fa640 1 -- 172.21.15.98:0/331047462 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10019070 con 0x7f2b1c0ab1f0 2024-09-19T08:45:42.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.763+0000 7f2b197fa640 1 -- 172.21.15.98:0/331047462 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b100074e0 con 0x7f2b1c0ab1f0 2024-09-19T08:45:42.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.763+0000 7f2b22ee1640 1 -- 172.21.15.98:0/331047462 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b1c0b0c80 con 0x7f2b1c0ab1f0 2024-09-19T08:45:42.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.763+0000 7f2b197fa640 1 -- 172.21.15.98:0/331047462 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b100049f0 con 0x7f2b1c0ab1f0 2024-09-19T08:45:42.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.763+0000 7f2b22ee1640 1 -- 172.21.15.98:0/331047462 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b1c0b1240 con 0x7f2b1c0ab1f0 2024-09-19T08:45:42.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.763+0000 7f2b22ee1640 1 -- 172.21.15.98:0/331047462 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b1c160ad0 con 0x7f2b1c0ab1f0 2024-09-19T08:45:42.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.763+0000 7f2b197fa640 1 -- 172.21.15.98:0/331047462 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f2b1001c070 con 0x7f2b1c0ab1f0 2024-09-19T08:45:42.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.763+0000 7f2b197fa640 1 --2- 172.21.15.98:0/331047462 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2b08067e90 0x7f2b0806a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:42.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.763+0000 7f2b197fa640 1 -- 172.21.15.98:0/331047462 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7f2b10086220 con 0x7f2b1c0ab1f0 2024-09-19T08:45:42.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.767+0000 7f2b20c7d640 1 --2- 172.21.15.98:0/331047462 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2b08067e90 0x7f2b0806a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:42.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.767+0000 7f2b20c7d640 1 --2- 172.21.15.98:0/331047462 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2b08067e90 0x7f2b0806a350 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c0af000 tx=0x7f2b0c004ac0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:42.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.767+0000 7f2b197fa640 1 -- 172.21.15.98:0/331047462 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2b1c156f70 con 0x7f2b1c0ab1f0 2024-09-19T08:45:42.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.943+0000 7f2b22ee1640 1 -- 172.21.15.98:0/331047462 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "non_existent_subvolume", "target": ["mon-mgr", ""]}) -- 0x7f2b1c0d21a0 con 0x7f2b08067e90 2024-09-19T08:45:42.946 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:42.950+0000 7ff73a69c640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-19T08:45:42.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:42.947+0000 7f2b197fa640 1 -- 172.21.15.98:0/331047462 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f2b1c0d21a0 con 0x7f2b08067e90 2024-09-19T08:45:42.947 INFO:teuthology.orchestra.run.smithi098.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-19T08:45:42.953 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-19T08:45:42.954 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create non_existent_subvolume subvolume_1 snapshot_1 2024-09-19T08:45:43.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.071+0000 7fd87af13640 1 Processor -- start 2024-09-19T08:45:43.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.071+0000 7fd87af13640 1 -- start start 2024-09-19T08:45:43.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.071+0000 7fd87af13640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd874144200 0x7fd8741445e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:43.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.071+0000 7fd87af13640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd874071470 0x7fd87415eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:43.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.071+0000 7fd87af13640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd874144b20 con 0x7fd874071030 2024-09-19T08:45:43.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.071+0000 7fd87af13640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fd874141940 con 0x7fd874071470 2024-09-19T08:45:43.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.071+0000 7fd87af13640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd874163060 con 0x7fd874144200 2024-09-19T08:45:43.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.071+0000 7fd873fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fd874071030 0x7fd87406d4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:53086/0 (socket says 172.21.15.98:53086) 2024-09-19T08:45:43.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.071+0000 7fd873fff640 1 -- 172.21.15.98:0/3762596249 learned_addr learned my addr 172.21.15.98:0/3762596249 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:43.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.075+0000 7fd8794b0640 1 --2- 172.21.15.98:0/3762596249 >> v2:172.21.15.139:3300/0 conn(0x7fd874071470 0x7fd87415eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:43.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.075+0000 7fd878caf640 1 --2- 172.21.15.98:0/3762596249 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd874144200 0x7fd8741445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:43.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.075+0000 7fd8737fe640 1 -- 172.21.15.98:0/3762596249 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4106228524 0 0) 0x7fd874144b20 con 0x7fd874071030 2024-09-19T08:45:43.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.075+0000 7fd8737fe640 1 -- 172.21.15.98:0/3762596249 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd860003530 con 0x7fd874071030 2024-09-19T08:45:43.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.075+0000 7fd8737fe640 1 -- 172.21.15.98:0/3762596249 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1272193921 0 0) 0x7fd860003530 con 0x7fd874071030 2024-09-19T08:45:43.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.075+0000 7fd8737fe640 1 -- 172.21.15.98:0/3762596249 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd874144200 msgr2=0x7fd8741445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:43.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.075+0000 7fd8737fe640 1 --2- 172.21.15.98:0/3762596249 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd874144200 0x7fd8741445e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:43.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.075+0000 7fd8737fe640 1 -- 172.21.15.98:0/3762596249 >> v2:172.21.15.139:3300/0 conn(0x7fd874071470 msgr2=0x7fd87415eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:43.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.075+0000 7fd8737fe640 1 --2- 172.21.15.98:0/3762596249 >> v2:172.21.15.139:3300/0 conn(0x7fd874071470 0x7fd87415eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:43.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.075+0000 7fd8737fe640 1 -- 172.21.15.98:0/3762596249 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8741622a0 con 0x7fd874071030 2024-09-19T08:45:43.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.075+0000 7fd8737fe640 1 -- 172.21.15.98:0/3762596249 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fd85c001a00 con 0x7fd874071030 2024-09-19T08:45:43.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.075+0000 7fd8737fe640 1 -- 172.21.15.98:0/3762596249 >> v1:172.21.15.98:6789/0 conn(0x7fd874071030 legacy=0x7fd87406d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:43.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.075+0000 7fd8737fe640 1 --2- 172.21.15.98:0/3762596249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8600039c0 0x7fd860003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:43.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.075+0000 7fd8737fe640 1 --2- 172.21.15.98:0/3762596249 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd874144200 0x7fd860004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:43.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.075+0000 7fd8737fe640 1 --2- 172.21.15.98:0/3762596249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd860004d50 0x7fd8600051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:43.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.075+0000 7fd8737fe640 1 -- 172.21.15.98:0/3762596249 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd874141940 con 0x7fd8600039c0 2024-09-19T08:45:43.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.075+0000 7fd8737fe640 1 -- 172.21.15.98:0/3762596249 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd874163060 con 0x7fd860004d50 2024-09-19T08:45:43.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.075+0000 7fd8737fe640 1 -- 172.21.15.98:0/3762596249 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd86000b9d0 con 0x7fd874144200 2024-09-19T08:45:43.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.075+0000 7fd878caf640 1 --2- 172.21.15.98:0/3762596249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8600039c0 0x7fd860003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:43.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.075+0000 7fd873fff640 1 --2- 172.21.15.98:0/3762596249 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd874144200 0x7fd860004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:43.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.075+0000 7fd878caf640 1 -- 172.21.15.98:0/3762596249 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd874144200 msgr2=0x7fd860004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:43.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.075+0000 7fd8794b0640 1 --2- 172.21.15.98:0/3762596249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd860004d50 0x7fd8600051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:43.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.075+0000 7fd878caf640 1 --2- 172.21.15.98:0/3762596249 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd874144200 0x7fd860004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:43.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.075+0000 7fd878caf640 1 -- 172.21.15.98:0/3762596249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd860004d50 msgr2=0x7fd8600051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:43.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.075+0000 7fd878caf640 1 --2- 172.21.15.98:0/3762596249 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd860004d50 0x7fd8600051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:43.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.075+0000 7fd878caf640 1 -- 172.21.15.98:0/3762596249 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd860003530 con 0x7fd8600039c0 2024-09-19T08:45:43.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.075+0000 7fd873fff640 1 --2- 172.21.15.98:0/3762596249 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd874144200 0x7fd860004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:43.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.075+0000 7fd878caf640 1 --2- 172.21.15.98:0/3762596249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8600039c0 0x7fd860003de0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fd864009ae0 tx=0x7fd86401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c93163a46e51dd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:43.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.075+0000 7fd8737fe640 1 -- 172.21.15.98:0/3762596249 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8640b2070 con 0x7fd8600039c0 2024-09-19T08:45:43.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.075+0000 7fd8737fe640 1 -- 172.21.15.98:0/3762596249 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8640a5530 con 0x7fd8600039c0 2024-09-19T08:45:43.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.075+0000 7fd8737fe640 1 -- 172.21.15.98:0/3762596249 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8640096b0 con 0x7fd8600039c0 2024-09-19T08:45:43.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.079+0000 7fd87af13640 1 -- 172.21.15.98:0/3762596249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8600039c0 msgr2=0x7fd860003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:43.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.079+0000 7fd87af13640 1 --2- 172.21.15.98:0/3762596249 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8600039c0 0x7fd860003de0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fd864009ae0 tx=0x7fd86401ca70 comp rx=0 tx=0).stop 2024-09-19T08:45:43.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.079+0000 7fd8794b0640 1 -- 172.21.15.98:0/3762596249 reap_dead start 2024-09-19T08:45:43.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.079+0000 7fd87af13640 1 -- 172.21.15.98:0/3762596249 shutdown_connections 2024-09-19T08:45:43.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.079+0000 7fd87af13640 1 -- 172.21.15.98:0/3762596249 >> 172.21.15.98:0/3762596249 conn(0x7fd87406c090 msgr2=0x7fd87406c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:43.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.079+0000 7fd87af13640 1 -- 172.21.15.98:0/3762596249 shutdown_connections 2024-09-19T08:45:43.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.079+0000 7fd87af13640 1 -- 172.21.15.98:0/3762596249 wait complete. 2024-09-19T08:45:43.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.079+0000 7fd87af13640 1 Processor -- start 2024-09-19T08:45:43.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.079+0000 7fd87af13640 1 -- start start 2024-09-19T08:45:43.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.079+0000 7fd87af13640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8600039c0 0x7fd87413b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:43.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.079+0000 7fd87af13640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8740b1a40 0x7fd874155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:43.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.079+0000 7fd87af13640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd8740b1ec0 0x7fd8740b0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:43.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.079+0000 7fd87af13640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd874072e90 con 0x7fd8600039c0 2024-09-19T08:45:43.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.079+0000 7fd87af13640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd874072d10 con 0x7fd8740b1a40 2024-09-19T08:45:43.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.079+0000 7fd87af13640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd87406dbd0 con 0x7fd8740b1ec0 2024-09-19T08:45:43.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.079+0000 7fd878caf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8600039c0 0x7fd87413b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:43.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.079+0000 7fd878caf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8600039c0 0x7fd87413b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58250/0 (socket says 172.21.15.98:58250) 2024-09-19T08:45:43.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.079+0000 7fd878caf640 1 -- 172.21.15.98:0/4135588173 learned_addr learned my addr 172.21.15.98:0/4135588173 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:43.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.079+0000 7fd873fff640 1 --2- 172.21.15.98:0/4135588173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8740b1a40 0x7fd874155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:43.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.079+0000 7fd8794b0640 1 --2- 172.21.15.98:0/4135588173 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd8740b1ec0 0x7fd8740b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:43.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.079+0000 7fd878caf640 1 -- 172.21.15.98:0/4135588173 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd8740b1ec0 msgr2=0x7fd8740b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:43.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.079+0000 7fd878caf640 1 --2- 172.21.15.98:0/4135588173 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd8740b1ec0 0x7fd8740b0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:43.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.079+0000 7fd878caf640 1 -- 172.21.15.98:0/4135588173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8740b1a40 msgr2=0x7fd874155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:43.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.079+0000 7fd878caf640 1 --2- 172.21.15.98:0/4135588173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd8740b1a40 0x7fd874155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:43.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.079+0000 7fd878caf640 1 -- 172.21.15.98:0/4135588173 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd874157230 con 0x7fd8600039c0 2024-09-19T08:45:43.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.079+0000 7fd878caf640 1 --2- 172.21.15.98:0/4135588173 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd8600039c0 0x7fd87413b1a0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fd864009ae0 tx=0x7fd860005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:43.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.079+0000 7fd8717fa640 1 -- 172.21.15.98:0/4135588173 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8640bc030 con 0x7fd8600039c0 2024-09-19T08:45:43.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.079+0000 7fd8717fa640 1 -- 172.21.15.98:0/4135588173 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd86400f000 con 0x7fd8600039c0 2024-09-19T08:45:43.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.079+0000 7fd87af13640 1 -- 172.21.15.98:0/4135588173 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd874157510 con 0x7fd8600039c0 2024-09-19T08:45:43.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.079+0000 7fd8717fa640 1 -- 172.21.15.98:0/4135588173 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8640b2040 con 0x7fd8600039c0 2024-09-19T08:45:43.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.079+0000 7fd87af13640 1 -- 172.21.15.98:0/4135588173 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd874156ee0 con 0x7fd8600039c0 2024-09-19T08:45:43.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.079+0000 7fd87af13640 1 -- 172.21.15.98:0/4135588173 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd87415f800 con 0x7fd8600039c0 2024-09-19T08:45:43.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.083+0000 7fd8717fa640 1 -- 172.21.15.98:0/4135588173 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fd8640bf070 con 0x7fd8600039c0 2024-09-19T08:45:43.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.083+0000 7fd8717fa640 1 --2- 172.21.15.98:0/4135588173 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd854067e90 0x7fd85406a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:43.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.083+0000 7fd8717fa640 1 -- 172.21.15.98:0/4135588173 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7fd8641205d0 con 0x7fd8600039c0 2024-09-19T08:45:43.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.083+0000 7fd873fff640 1 --2- 172.21.15.98:0/4135588173 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd854067e90 0x7fd85406a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:43.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.083+0000 7fd873fff640 1 --2- 172.21.15.98:0/4135588173 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd854067e90 0x7fd85406a350 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd85c001bf0 tx=0x7fd85c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:43.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.087+0000 7fd8717fa640 1 -- 172.21.15.98:0/4135588173 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd864016780 con 0x7fd8600039c0 2024-09-19T08:45:43.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.259+0000 7fd87af13640 1 -- 172.21.15.98:0/4135588173 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7fd874155a10 con 0x7fd854067e90 2024-09-19T08:45:43.262 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:43.262+0000 7ff73a69c640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-19T08:45:43.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.263+0000 7fd8717fa640 1 -- 172.21.15.98:0/4135588173 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fd874155a10 con 0x7fd854067e90 2024-09-19T08:45:43.263 INFO:teuthology.orchestra.run.smithi098.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-19T08:45:43.268 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-19T08:45:43.268 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm non_existent_subvolume subvolume_1 snapshot_1 2024-09-19T08:45:43.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.391+0000 7f86cc1fb640 1 Processor -- start 2024-09-19T08:45:43.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.391+0000 7f86cc1fb640 1 -- start start 2024-09-19T08:45:43.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.395+0000 7f86cc1fb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f86c415e720 0x7f86c40cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:43.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.395+0000 7f86cc1fb640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f86c40d3a60 0x7f86c40d3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:43.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.395+0000 7f86cc1fb640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86c40d29c0 con 0x7f86c40cfb50 2024-09-19T08:45:43.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.395+0000 7f86cc1fb640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f86c40d2ca0 con 0x7f86c40d3a60 2024-09-19T08:45:43.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.395+0000 7f86cc1fb640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f86c40d2f80 con 0x7f86c415e720 2024-09-19T08:45:43.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.395+0000 7f86ca798640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f86c40d3a60 0x7f86c40d3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:43.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.395+0000 7f86ca798640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f86c40d3a60 0x7f86c40d3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58262/0 (socket says 172.21.15.98:58262) 2024-09-19T08:45:43.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.395+0000 7f86ca798640 1 -- 172.21.15.98:0/1340563290 learned_addr learned my addr 172.21.15.98:0/1340563290 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:43.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.395+0000 7f86c9f97640 1 --2- 172.21.15.98:0/1340563290 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f86c415e720 0x7f86c40cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:43.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.395+0000 7f86ca798640 1 -- 172.21.15.98:0/1340563290 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f86c415e720 msgr2=0x7f86c40cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:43.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.395+0000 7f86ca798640 1 --2- 172.21.15.98:0/1340563290 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f86c415e720 0x7f86c40cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:43.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.395+0000 7f86ca798640 1 -- 172.21.15.98:0/1340563290 >> v1:172.21.15.139:6789/0 conn(0x7f86c40cfb50 legacy=0x7f86c40b7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:43.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.395+0000 7f86ca798640 1 -- 172.21.15.98:0/1340563290 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c40d3260 con 0x7f86c40d3a60 2024-09-19T08:45:43.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.395+0000 7f86ca798640 1 --2- 172.21.15.98:0/1340563290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86c40d3a60 0x7f86c40d3ef0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f86bc009e50 tx=0x7f86bc01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6e4e9c281cce284 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:43.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.395+0000 7f86c8f95640 1 -- 172.21.15.98:0/1340563290 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc0a5920 con 0x7f86c40d3a60 2024-09-19T08:45:43.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.395+0000 7f86c8f95640 1 -- 172.21.15.98:0/1340563290 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86bc0a5ac0 con 0x7f86c40d3a60 2024-09-19T08:45:43.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.395+0000 7f86c8f95640 1 -- 172.21.15.98:0/1340563290 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc0ad3f0 con 0x7f86c40d3a60 2024-09-19T08:45:43.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.399+0000 7f86cc1fb640 1 -- 172.21.15.98:0/1340563290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86c40d3a60 msgr2=0x7f86c40d3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:43.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.399+0000 7f86cc1fb640 1 --2- 172.21.15.98:0/1340563290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86c40d3a60 0x7f86c40d3ef0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f86bc009e50 tx=0x7f86bc01ca30 comp rx=0 tx=0).stop 2024-09-19T08:45:43.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.399+0000 7f86cc1fb640 1 -- 172.21.15.98:0/1340563290 shutdown_connections 2024-09-19T08:45:43.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.399+0000 7f86cc1fb640 1 --2- 172.21.15.98:0/1340563290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86c40d3a60 0x7f86c40d3ef0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:43.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.399+0000 7f86cc1fb640 1 --2- 172.21.15.98:0/1340563290 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f86c415e720 0x7f86c40cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:43.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.399+0000 7f86cc1fb640 1 -- 172.21.15.98:0/1340563290 >> 172.21.15.98:0/1340563290 conn(0x7f86c40d4640 msgr2=0x7f86c40d4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:43.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.399+0000 7f86cc1fb640 1 -- 172.21.15.98:0/1340563290 shutdown_connections 2024-09-19T08:45:43.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.399+0000 7f86cc1fb640 1 -- 172.21.15.98:0/1340563290 wait complete. 2024-09-19T08:45:43.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.399+0000 7f86cc1fb640 1 Processor -- start 2024-09-19T08:45:43.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.399+0000 7f86cc1fb640 1 -- start start 2024-09-19T08:45:43.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.399+0000 7f86cc1fb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f86c40cfb50 0x7f86c40b9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:43.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.399+0000 7f86cc1fb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86c40d3a60 0x7f86c415ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:43.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.399+0000 7f86cc1fb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86c415e720 0x7f86c415f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:43.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.399+0000 7f86cc1fb640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f86c40b8000 con 0x7f86c40d3a60 2024-09-19T08:45:43.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.399+0000 7f86cc1fb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f86c40b7e80 con 0x7f86c415e720 2024-09-19T08:45:43.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.399+0000 7f86cc1fb640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f86c416b6f0 con 0x7f86c40cfb50 2024-09-19T08:45:43.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.399+0000 7f86c9796640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86c40d3a60 0x7f86c415ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:43.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.399+0000 7f86c9796640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86c40d3a60 0x7f86c415ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58268/0 (socket says 172.21.15.98:58268) 2024-09-19T08:45:43.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.399+0000 7f86c9796640 1 -- 172.21.15.98:0/278425202 learned_addr learned my addr 172.21.15.98:0/278425202 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:43.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.399+0000 7f86ca798640 1 --2- 172.21.15.98:0/278425202 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86c415e720 0x7f86c415f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:43.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.399+0000 7f86c9f97640 1 --2- 172.21.15.98:0/278425202 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f86c40cfb50 0x7f86c40b9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:43.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.399+0000 7f86c9796640 1 -- 172.21.15.98:0/278425202 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f86c40cfb50 msgr2=0x7f86c40b9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:43.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.399+0000 7f86c9796640 1 --2- 172.21.15.98:0/278425202 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f86c40cfb50 0x7f86c40b9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:43.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.399+0000 7f86c9796640 1 -- 172.21.15.98:0/278425202 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86c415e720 msgr2=0x7f86c415f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:43.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.399+0000 7f86c9796640 1 --2- 172.21.15.98:0/278425202 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f86c415e720 0x7f86c415f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:43.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.399+0000 7f86c9796640 1 -- 172.21.15.98:0/278425202 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c40b8370 con 0x7f86c40d3a60 2024-09-19T08:45:43.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.399+0000 7f86c9796640 1 --2- 172.21.15.98:0/278425202 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f86c40d3a60 0x7f86c415ede0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f86b400c340 tx=0x7f86b400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:43.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.399+0000 7f86ba7fc640 1 -- 172.21.15.98:0/278425202 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b40052d0 con 0x7f86c40d3a60 2024-09-19T08:45:43.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.399+0000 7f86ba7fc640 1 -- 172.21.15.98:0/278425202 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86b4005470 con 0x7f86c40d3a60 2024-09-19T08:45:43.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.399+0000 7f86cc1fb640 1 -- 172.21.15.98:0/278425202 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86c40b8650 con 0x7f86c40d3a60 2024-09-19T08:45:43.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.399+0000 7f86ba7fc640 1 -- 172.21.15.98:0/278425202 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b4007260 con 0x7f86c40d3a60 2024-09-19T08:45:43.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.399+0000 7f86cc1fb640 1 -- 172.21.15.98:0/278425202 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86c40b8ef0 con 0x7f86c40d3a60 2024-09-19T08:45:43.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.403+0000 7f86cc1fb640 1 -- 172.21.15.98:0/278425202 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86c40b8820 con 0x7f86c40d3a60 2024-09-19T08:45:43.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.403+0000 7f86ba7fc640 1 -- 172.21.15.98:0/278425202 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f86b4007400 con 0x7f86c40d3a60 2024-09-19T08:45:43.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.403+0000 7f86ba7fc640 1 --2- 172.21.15.98:0/278425202 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f86a8067e60 0x7f86a806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:43.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.403+0000 7f86ba7fc640 1 -- 172.21.15.98:0/278425202 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7f86b4085b50 con 0x7f86c40d3a60 2024-09-19T08:45:43.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.403+0000 7f86c9f97640 1 --2- 172.21.15.98:0/278425202 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f86a8067e60 0x7f86a806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:43.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.403+0000 7f86c9f97640 1 --2- 172.21.15.98:0/278425202 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f86a8067e60 0x7f86a806a320 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f86c0009af0 tx=0x7f86c0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:43.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.407+0000 7f86ba7fc640 1 -- 172.21.15.98:0/278425202 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f86b4055e00 con 0x7f86c40d3a60 2024-09-19T08:45:43.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.583+0000 7f86cc1fb640 1 -- 172.21.15.98:0/278425202 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f86c40b90f0 con 0x7f86a8067e60 2024-09-19T08:45:43.586 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:43.586+0000 7ff73a69c640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-19T08:45:43.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.587+0000 7f86ba7fc640 1 -- 172.21.15.98:0/278425202 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f86c40b90f0 con 0x7f86a8067e60 2024-09-19T08:45:43.587 INFO:teuthology.orchestra.run.smithi098.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-19T08:45:43.592 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-19T08:45:43.593 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info non_existent_subvolume subvolume_1 snapshot_1 2024-09-19T08:45:43.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.707+0000 7f61560e2640 1 Processor -- start 2024-09-19T08:45:43.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.707+0000 7f61560e2640 1 -- start start 2024-09-19T08:45:43.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.711+0000 7f61560e2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f61501626a0 0x7f6150163990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:43.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.711+0000 7f61560e2640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6150162a80 0x7f615006d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:43.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.711+0000 7f61560e2640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f61500b4260 0x7f615006da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:43.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.711+0000 7f61560e2640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f6150163ed0 con 0x7f6150162a80 2024-09-19T08:45:43.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.711+0000 7f61560e2640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f61500b37a0 con 0x7f61500b4260 2024-09-19T08:45:43.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.711+0000 7f61560e2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f61500d0170 con 0x7f61501626a0 2024-09-19T08:45:43.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.711+0000 7f614effd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6150162a80 0x7f615006d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:43.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.711+0000 7f614effd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6150162a80 0x7f615006d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58278/0 (socket says 172.21.15.98:58278) 2024-09-19T08:45:43.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.711+0000 7f614effd640 1 -- 172.21.15.98:0/1419018439 learned_addr learned my addr 172.21.15.98:0/1419018439 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:43.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.711+0000 7f614ffff640 1 --2- 172.21.15.98:0/1419018439 >> v2:172.21.15.139:3300/0 conn(0x7f61500b4260 0x7f615006da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:43.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.711+0000 7f614f7fe640 1 --2- 172.21.15.98:0/1419018439 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f61501626a0 0x7f6150163990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:43.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.711+0000 7f614effd640 1 -- 172.21.15.98:0/1419018439 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f61501626a0 msgr2=0x7f6150163990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:43.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.711+0000 7f614effd640 1 --2- 172.21.15.98:0/1419018439 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f61501626a0 0x7f6150163990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:43.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.711+0000 7f614effd640 1 -- 172.21.15.98:0/1419018439 >> v2:172.21.15.139:3300/0 conn(0x7f61500b4260 msgr2=0x7f615006da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:43.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.711+0000 7f614effd640 1 --2- 172.21.15.98:0/1419018439 >> v2:172.21.15.139:3300/0 conn(0x7f61500b4260 0x7f615006da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:43.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.711+0000 7f614effd640 1 -- 172.21.15.98:0/1419018439 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61500b3a60 con 0x7f6150162a80 2024-09-19T08:45:43.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.711+0000 7f614effd640 1 --2- 172.21.15.98:0/1419018439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6150162a80 0x7f615006d4c0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f61500b5a80 tx=0x7f614401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7944c9f54163c0b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:43.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.711+0000 7f614e7fc640 1 -- 172.21.15.98:0/1419018439 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61440b2070 con 0x7f6150162a80 2024-09-19T08:45:43.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.715+0000 7f614e7fc640 1 -- 172.21.15.98:0/1419018439 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61440a9070 con 0x7f6150162a80 2024-09-19T08:45:43.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.715+0000 7f614e7fc640 1 -- 172.21.15.98:0/1419018439 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61440ad880 con 0x7f6150162a80 2024-09-19T08:45:43.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.715+0000 7f61560e2640 1 -- 172.21.15.98:0/1419018439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6150162a80 msgr2=0x7f615006d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:43.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.715+0000 7f61560e2640 1 --2- 172.21.15.98:0/1419018439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6150162a80 0x7f615006d4c0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f61500b5a80 tx=0x7f614401c490 comp rx=0 tx=0).stop 2024-09-19T08:45:43.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.715+0000 7f61560e2640 1 -- 172.21.15.98:0/1419018439 shutdown_connections 2024-09-19T08:45:43.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.715+0000 7f61560e2640 1 --2- 172.21.15.98:0/1419018439 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6150162a80 0x7f615006d4c0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:43.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.715+0000 7f61560e2640 1 --2- 172.21.15.98:0/1419018439 >> v2:172.21.15.139:3300/0 conn(0x7f61500b4260 0x7f615006da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:43.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.715+0000 7f61560e2640 1 --2- 172.21.15.98:0/1419018439 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f61501626a0 0x7f6150163990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:43.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.715+0000 7f61560e2640 1 -- 172.21.15.98:0/1419018439 >> 172.21.15.98:0/1419018439 conn(0x7f615006c090 msgr2=0x7f615006c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:43.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.715+0000 7f61560e2640 1 -- 172.21.15.98:0/1419018439 shutdown_connections 2024-09-19T08:45:43.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.715+0000 7f61560e2640 1 -- 172.21.15.98:0/1419018439 wait complete. 2024-09-19T08:45:43.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.715+0000 7f61560e2640 1 Processor -- start 2024-09-19T08:45:43.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.715+0000 7f61560e2640 1 -- start start 2024-09-19T08:45:43.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.715+0000 7f61560e2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f61500b4260 0x7f615015ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:43.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.715+0000 7f61560e2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61501626a0 0x7f61501660e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:43.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.715+0000 7f61560e2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6150162a80 0x7f61500aa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:43.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.715+0000 7f61560e2640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6150071c10 con 0x7f61501626a0 2024-09-19T08:45:43.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.715+0000 7f61560e2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6150071a90 con 0x7f6150162a80 2024-09-19T08:45:43.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.715+0000 7f61560e2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6150071d90 con 0x7f61500b4260 2024-09-19T08:45:43.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.715+0000 7f614effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61501626a0 0x7f61501660e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:43.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.715+0000 7f614effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61501626a0 0x7f61501660e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58288/0 (socket says 172.21.15.98:58288) 2024-09-19T08:45:43.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.715+0000 7f614effd640 1 -- 172.21.15.98:0/49953684 learned_addr learned my addr 172.21.15.98:0/49953684 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:43.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.715+0000 7f614ffff640 1 --2- 172.21.15.98:0/49953684 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6150162a80 0x7f61500aa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:43.720 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.715+0000 7f614f7fe640 1 --2- 172.21.15.98:0/49953684 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f61500b4260 0x7f615015ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:43.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.715+0000 7f614effd640 1 -- 172.21.15.98:0/49953684 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f61500b4260 msgr2=0x7f615015ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:43.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.715+0000 7f614effd640 1 --2- 172.21.15.98:0/49953684 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f61500b4260 0x7f615015ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:43.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.715+0000 7f614effd640 1 -- 172.21.15.98:0/49953684 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6150162a80 msgr2=0x7f61500aa4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:43.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.715+0000 7f614effd640 1 --2- 172.21.15.98:0/49953684 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6150162a80 0x7f61500aa4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:43.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.715+0000 7f614effd640 1 -- 172.21.15.98:0/49953684 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f615015dc00 con 0x7f61501626a0 2024-09-19T08:45:43.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.715+0000 7f614ffff640 1 --2- 172.21.15.98:0/49953684 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6150162a80 0x7f61500aa4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:43.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.715+0000 7f614effd640 1 --2- 172.21.15.98:0/49953684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f61501626a0 0x7f61501660e0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f61440b0040 tx=0x7f6144019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:43.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.715+0000 7f6137fff640 1 -- 172.21.15.98:0/49953684 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61440bc020 con 0x7f61501626a0 2024-09-19T08:45:43.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.715+0000 7f6137fff640 1 -- 172.21.15.98:0/49953684 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61440b8420 con 0x7f61501626a0 2024-09-19T08:45:43.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.715+0000 7f61560e2640 1 -- 172.21.15.98:0/49953684 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f615015dee0 con 0x7f61501626a0 2024-09-19T08:45:43.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.715+0000 7f6137fff640 1 -- 172.21.15.98:0/49953684 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61440b2040 con 0x7f61501626a0 2024-09-19T08:45:43.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.715+0000 7f61560e2640 1 -- 172.21.15.98:0/49953684 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f615015e4a0 con 0x7f61501626a0 2024-09-19T08:45:43.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.719+0000 7f61560e2640 1 -- 172.21.15.98:0/49953684 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6150163990 con 0x7f61501626a0 2024-09-19T08:45:43.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.719+0000 7f6137fff640 1 -- 172.21.15.98:0/49953684 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f61440bf020 con 0x7f61501626a0 2024-09-19T08:45:43.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.719+0000 7f6137fff640 1 --2- 172.21.15.98:0/49953684 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f613c067e60 0x7f613c06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:43.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.719+0000 7f6137fff640 1 -- 172.21.15.98:0/49953684 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7f6144120a70 con 0x7f61501626a0 2024-09-19T08:45:43.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.719+0000 7f614f7fe640 1 --2- 172.21.15.98:0/49953684 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f613c067e60 0x7f613c06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:43.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.719+0000 7f614f7fe640 1 --2- 172.21.15.98:0/49953684 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f613c067e60 0x7f613c06a320 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f615015db90 tx=0x7f6140008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:43.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.723+0000 7f6137fff640 1 -- 172.21.15.98:0/49953684 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f61440c6030 con 0x7f61501626a0 2024-09-19T08:45:43.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.903+0000 7f61560e2640 1 -- 172.21.15.98:0/49953684 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f61500ce5c0 con 0x7f613c067e60 2024-09-19T08:45:43.904 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:43.906+0000 7ff73a69c640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-19T08:45:43.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:43.907+0000 7f6137fff640 1 -- 172.21.15.98:0/49953684 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f61500ce5c0 con 0x7f613c067e60 2024-09-19T08:45:43.905 INFO:teuthology.orchestra.run.smithi098.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-19T08:45:43.910 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-19T08:45:43.910 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot protect non_existent_subvolume subvolume_1 snapshot_1 2024-09-19T08:45:44.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.035+0000 7f19096e8640 1 Processor -- start 2024-09-19T08:45:44.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.035+0000 7f19096e8640 1 -- start start 2024-09-19T08:45:44.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.039+0000 7f19096e8640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f190415a930 0x7f19040b3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:44.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.039+0000 7f19096e8640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f19040b63d0 0x7f19040cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:44.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.039+0000 7f19096e8640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19040b5110 con 0x7f19040b3b60 2024-09-19T08:45:44.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.039+0000 7f19096e8640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f19040b4070 con 0x7f19040b63d0 2024-09-19T08:45:44.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.039+0000 7f19096e8640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f19040b4350 con 0x7f190415a930 2024-09-19T08:45:44.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.039+0000 7f19037fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f19040b63d0 0x7f19040cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:44.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.039+0000 7f19037fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f19040b63d0 0x7f19040cdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58302/0 (socket says 172.21.15.98:58302) 2024-09-19T08:45:44.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.039+0000 7f19037fe640 1 -- 172.21.15.98:0/3398805096 learned_addr learned my addr 172.21.15.98:0/3398805096 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:44.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.039+0000 7f18fa7fc640 1 --1- 172.21.15.98:0/3398805096 >> v1:172.21.15.98:6789/0 conn(0x7f19040b3b60 0x7f19040b5cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:53100/0 (socket says 172.21.15.98:53100) 2024-09-19T08:45:44.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.039+0000 7f1902ffd640 1 --2- 172.21.15.98:0/3398805096 >> v2:172.21.15.139:3300/0 conn(0x7f190415a930 0x7f19040b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:44.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.039+0000 7f19037fe640 1 -- 172.21.15.98:0/3398805096 >> v2:172.21.15.139:3300/0 conn(0x7f190415a930 msgr2=0x7f19040b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:44.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.039+0000 7f19037fe640 1 --2- 172.21.15.98:0/3398805096 >> v2:172.21.15.139:3300/0 conn(0x7f190415a930 0x7f19040b3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:44.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.039+0000 7f19037fe640 1 -- 172.21.15.98:0/3398805096 >> v1:172.21.15.98:6789/0 conn(0x7f19040b3b60 legacy=0x7f19040b5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:44.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.039+0000 7f19027fc640 1 -- 172.21.15.98:0/3398805096 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 884715660 0 0) 0x7f19040b5110 con 0x7f19040b3b60 2024-09-19T08:45:44.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.039+0000 7f19037fe640 1 -- 172.21.15.98:0/3398805096 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19040b4630 con 0x7f19040b63d0 2024-09-19T08:45:44.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.039+0000 7f1902ffd640 1 --2- 172.21.15.98:0/3398805096 >> v2:172.21.15.139:3300/0 conn(0x7f190415a930 0x7f19040b3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:44.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.039+0000 7f19037fe640 1 --2- 172.21.15.98:0/3398805096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19040b63d0 0x7f19040cdb80 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f190406f340 tx=0x7f18fc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857ccc65f2ad9bf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:44.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.039+0000 7f19027fc640 1 -- 172.21.15.98:0/3398805096 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc0b1070 con 0x7f19040b63d0 2024-09-19T08:45:44.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.039+0000 7f19027fc640 1 -- 172.21.15.98:0/3398805096 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18fc0ac600 con 0x7f19040b63d0 2024-09-19T08:45:44.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.039+0000 7f19027fc640 1 -- 172.21.15.98:0/3398805096 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc0ac9d0 con 0x7f19040b63d0 2024-09-19T08:45:44.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.043+0000 7f19096e8640 1 -- 172.21.15.98:0/3398805096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19040b63d0 msgr2=0x7f19040cdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:44.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.043+0000 7f19096e8640 1 --2- 172.21.15.98:0/3398805096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19040b63d0 0x7f19040cdb80 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f190406f340 tx=0x7f18fc01c490 comp rx=0 tx=0).stop 2024-09-19T08:45:44.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.043+0000 7f19096e8640 1 -- 172.21.15.98:0/3398805096 shutdown_connections 2024-09-19T08:45:44.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.043+0000 7f19096e8640 1 --2- 172.21.15.98:0/3398805096 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19040b63d0 0x7f19040cdb80 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:44.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.043+0000 7f19096e8640 1 --2- 172.21.15.98:0/3398805096 >> v2:172.21.15.139:3300/0 conn(0x7f190415a930 0x7f19040b3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:44.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.043+0000 7f19096e8640 1 -- 172.21.15.98:0/3398805096 >> 172.21.15.98:0/3398805096 conn(0x7f190406c090 msgr2=0x7f190406c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:44.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.043+0000 7f19096e8640 1 -- 172.21.15.98:0/3398805096 shutdown_connections 2024-09-19T08:45:44.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.043+0000 7f19096e8640 1 -- 172.21.15.98:0/3398805096 wait complete. 2024-09-19T08:45:44.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.043+0000 7f19096e8640 1 Processor -- start 2024-09-19T08:45:44.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.043+0000 7f19096e8640 1 -- start start 2024-09-19T08:45:44.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.043+0000 7f19096e8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19040b3b60 0x7f190413fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:44.043 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.043+0000 7f19096e8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19040b63d0 0x7f190414f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:44.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.043+0000 7f19096e8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f190415a930 0x7f1904072950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:44.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.043+0000 7f19096e8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f19040cea50 con 0x7f190415a930 2024-09-19T08:45:44.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.043+0000 7f19096e8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f19040ce8d0 con 0x7f19040b3b60 2024-09-19T08:45:44.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.043+0000 7f19096e8640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f19040cebd0 con 0x7f19040b63d0 2024-09-19T08:45:44.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.043+0000 7f19037fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f190415a930 0x7f1904072950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:44.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.043+0000 7f19037fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f190415a930 0x7f1904072950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58318/0 (socket says 172.21.15.98:58318) 2024-09-19T08:45:44.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.043+0000 7f19037fe640 1 -- 172.21.15.98:0/659537340 learned_addr learned my addr 172.21.15.98:0/659537340 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:44.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.043+0000 7f19037fe640 1 -- 172.21.15.98:0/659537340 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19040b63d0 msgr2=0x7f190414f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:44.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.043+0000 7f19037fe640 1 --2- 172.21.15.98:0/659537340 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19040b63d0 0x7f190414f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:44.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.043+0000 7f19037fe640 1 -- 172.21.15.98:0/659537340 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19040b3b60 msgr2=0x7f190413fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:44.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.043+0000 7f19037fe640 1 --2- 172.21.15.98:0/659537340 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19040b3b60 0x7f190413fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:44.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.043+0000 7f19037fe640 1 -- 172.21.15.98:0/659537340 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19041401e0 con 0x7f190415a930 2024-09-19T08:45:44.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.043+0000 7f19037fe640 1 --2- 172.21.15.98:0/659537340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f190415a930 0x7f1904072950 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f18fc0190a0 tx=0x7f18fc01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:44.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.043+0000 7f18fbfff640 1 -- 172.21.15.98:0/659537340 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc0b1040 con 0x7f190415a930 2024-09-19T08:45:44.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.043+0000 7f18fbfff640 1 -- 172.21.15.98:0/659537340 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18fc0bc070 con 0x7f190415a930 2024-09-19T08:45:44.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.043+0000 7f19096e8640 1 -- 172.21.15.98:0/659537340 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1904150170 con 0x7f190415a930 2024-09-19T08:45:44.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.043+0000 7f18fbfff640 1 -- 172.21.15.98:0/659537340 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc0b7890 con 0x7f190415a930 2024-09-19T08:45:44.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.043+0000 7f19096e8640 1 -- 172.21.15.98:0/659537340 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1904155550 con 0x7f190415a930 2024-09-19T08:45:44.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.043+0000 7f19096e8640 1 -- 172.21.15.98:0/659537340 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19040cdb80 con 0x7f190415a930 2024-09-19T08:45:44.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.047+0000 7f18fbfff640 1 -- 172.21.15.98:0/659537340 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f18fc0b7a30 con 0x7f190415a930 2024-09-19T08:45:44.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.047+0000 7f18fbfff640 1 --2- 172.21.15.98:0/659537340 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f18e4067e60 0x7f18e406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:44.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.047+0000 7f18fbfff640 1 -- 172.21.15.98:0/659537340 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7f18fc1205d0 con 0x7f190415a930 2024-09-19T08:45:44.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.047+0000 7f1902ffd640 1 --2- 172.21.15.98:0/659537340 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f18e4067e60 0x7f18e406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:44.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.047+0000 7f1902ffd640 1 --2- 172.21.15.98:0/659537340 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f18e4067e60 0x7f18e406a320 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f18f4008e00 tx=0x7f18f4008330 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:44.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.051+0000 7f18fbfff640 1 -- 172.21.15.98:0/659537340 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f18fc0c5020 con 0x7f190415a930 2024-09-19T08:45:44.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.239+0000 7f19096e8640 1 -- 172.21.15.98:0/659537340 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot protect", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f1904000940 con 0x7f18e4067e60 2024-09-19T08:45:44.242 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:44.242+0000 7ff73a69c640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-19T08:45:44.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.243+0000 7f18fbfff640 1 -- 172.21.15.98:0/659537340 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f1904000940 con 0x7f18e4067e60 2024-09-19T08:45:44.243 INFO:teuthology.orchestra.run.smithi098.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-19T08:45:44.248 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-19T08:45:44.249 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot unprotect non_existent_subvolume subvolume_1 snapshot_1 2024-09-19T08:45:44.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.379+0000 7f7b874cb640 1 Processor -- start 2024-09-19T08:45:44.379 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.379+0000 7f7b874cb640 1 -- start start 2024-09-19T08:45:44.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.383+0000 7f7b874cb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b800cf700 0x7f7b8015ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:44.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.383+0000 7f7b874cb640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7b8006d4c0 0x7f7b8006d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:44.380 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.383+0000 7f7b874cb640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b801608c0 con 0x7f7b8015c290 2024-09-19T08:45:44.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.383+0000 7f7b874cb640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f7b8006de70 con 0x7f7b8006d4c0 2024-09-19T08:45:44.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.383+0000 7f7b874cb640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7b80160e80 con 0x7f7b800cf700 2024-09-19T08:45:44.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.383+0000 7f7b85a68640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7b8006d4c0 0x7f7b8006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:44.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.383+0000 7f7b85a68640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7b8006d4c0 0x7f7b8006d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58322/0 (socket says 172.21.15.98:58322) 2024-09-19T08:45:44.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.383+0000 7f7b85a68640 1 -- 172.21.15.98:0/432176291 learned_addr learned my addr 172.21.15.98:0/432176291 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:44.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.383+0000 7f7b84a66640 1 --2- 172.21.15.98:0/432176291 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b800cf700 0x7f7b8015ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:44.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.383+0000 7f7b85a68640 1 -- 172.21.15.98:0/432176291 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b800cf700 msgr2=0x7f7b8015ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:44.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.383+0000 7f7b85a68640 1 --2- 172.21.15.98:0/432176291 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b800cf700 0x7f7b8015ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:44.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.383+0000 7f7b85a68640 1 -- 172.21.15.98:0/432176291 >> v1:172.21.15.139:6789/0 conn(0x7f7b8015c290 legacy=0x7f7b800cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:44.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.383+0000 7f7b85a68640 1 -- 172.21.15.98:0/432176291 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b80160ba0 con 0x7f7b8006d4c0 2024-09-19T08:45:44.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.383+0000 7f7b85a68640 1 --2- 172.21.15.98:0/432176291 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b8006d4c0 0x7f7b8006d930 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f7b78009e50 tx=0x7f7b7801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d939c0cfd85f9b6f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:44.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.383+0000 7f7b77fff640 1 -- 172.21.15.98:0/432176291 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b780a5700 con 0x7f7b8006d4c0 2024-09-19T08:45:44.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.383+0000 7f7b77fff640 1 -- 172.21.15.98:0/432176291 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b780a58a0 con 0x7f7b8006d4c0 2024-09-19T08:45:44.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.383+0000 7f7b77fff640 1 -- 172.21.15.98:0/432176291 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b780a5c70 con 0x7f7b8006d4c0 2024-09-19T08:45:44.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.383+0000 7f7b874cb640 1 -- 172.21.15.98:0/432176291 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b8006d4c0 msgr2=0x7f7b8006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:44.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.383+0000 7f7b874cb640 1 --2- 172.21.15.98:0/432176291 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b8006d4c0 0x7f7b8006d930 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f7b78009e50 tx=0x7f7b7801ca30 comp rx=0 tx=0).stop 2024-09-19T08:45:44.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.383+0000 7f7b874cb640 1 -- 172.21.15.98:0/432176291 shutdown_connections 2024-09-19T08:45:44.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.383+0000 7f7b874cb640 1 --2- 172.21.15.98:0/432176291 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b8006d4c0 0x7f7b8006d930 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:44.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.383+0000 7f7b874cb640 1 --2- 172.21.15.98:0/432176291 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b800cf700 0x7f7b8015ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:44.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.383+0000 7f7b874cb640 1 -- 172.21.15.98:0/432176291 >> 172.21.15.98:0/432176291 conn(0x7f7b8006c090 msgr2=0x7f7b8006c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:44.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.383+0000 7f7b874cb640 1 -- 172.21.15.98:0/432176291 shutdown_connections 2024-09-19T08:45:44.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.383+0000 7f7b874cb640 1 -- 172.21.15.98:0/432176291 wait complete. 2024-09-19T08:45:44.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.387+0000 7f7b874cb640 1 Processor -- start 2024-09-19T08:45:44.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.387+0000 7f7b874cb640 1 -- start start 2024-09-19T08:45:44.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.387+0000 7f7b874cb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b8006d4c0 0x7f7b80154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:44.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.387+0000 7f7b874cb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b800cf700 0x7f7b801553f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:44.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.387+0000 7f7b874cb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b8015c290 0x7f7b80155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:44.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.387+0000 7f7b874cb640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7b800b5580 con 0x7f7b8015c290 2024-09-19T08:45:44.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.387+0000 7f7b874cb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7b800b5400 con 0x7f7b800cf700 2024-09-19T08:45:44.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.387+0000 7f7b874cb640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7b801690c0 con 0x7f7b8006d4c0 2024-09-19T08:45:44.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.387+0000 7f7b85a68640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b8015c290 0x7f7b80155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:44.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.387+0000 7f7b85a68640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b8015c290 0x7f7b80155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58338/0 (socket says 172.21.15.98:58338) 2024-09-19T08:45:44.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.387+0000 7f7b85a68640 1 -- 172.21.15.98:0/189988849 learned_addr learned my addr 172.21.15.98:0/189988849 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:44.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.387+0000 7f7b85267640 1 --2- 172.21.15.98:0/189988849 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b8006d4c0 0x7f7b80154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:44.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.387+0000 7f7b84a66640 1 --2- 172.21.15.98:0/189988849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b800cf700 0x7f7b801553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:44.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.387+0000 7f7b85a68640 1 -- 172.21.15.98:0/189988849 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b8006d4c0 msgr2=0x7f7b80154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:44.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.387+0000 7f7b85a68640 1 --2- 172.21.15.98:0/189988849 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7b8006d4c0 0x7f7b80154eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:44.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.387+0000 7f7b85a68640 1 -- 172.21.15.98:0/189988849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b800cf700 msgr2=0x7f7b801553f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:44.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.387+0000 7f7b85a68640 1 --2- 172.21.15.98:0/189988849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b800cf700 0x7f7b801553f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:44.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.387+0000 7f7b85a68640 1 -- 172.21.15.98:0/189988849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b80155e70 con 0x7f7b8015c290 2024-09-19T08:45:44.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.387+0000 7f7b84a66640 1 --2- 172.21.15.98:0/189988849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7b800cf700 0x7f7b801553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:44.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.387+0000 7f7b85a68640 1 --2- 172.21.15.98:0/189988849 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7b8015c290 0x7f7b80155930 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f7b780af040 tx=0x7f7b780197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:44.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.387+0000 7f7b75ffb640 1 -- 172.21.15.98:0/189988849 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b780b8670 con 0x7f7b8015c290 2024-09-19T08:45:44.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.387+0000 7f7b75ffb640 1 -- 172.21.15.98:0/189988849 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b780be070 con 0x7f7b8015c290 2024-09-19T08:45:44.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.387+0000 7f7b874cb640 1 -- 172.21.15.98:0/189988849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b800ae3c0 con 0x7f7b8015c290 2024-09-19T08:45:44.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.387+0000 7f7b75ffb640 1 -- 172.21.15.98:0/189988849 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b780b8a50 con 0x7f7b8015c290 2024-09-19T08:45:44.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.387+0000 7f7b874cb640 1 -- 172.21.15.98:0/189988849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b800ae980 con 0x7f7b8015c290 2024-09-19T08:45:44.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.387+0000 7f7b874cb640 1 -- 172.21.15.98:0/189988849 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b8006d930 con 0x7f7b8015c290 2024-09-19T08:45:44.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.387+0000 7f7b75ffb640 1 -- 172.21.15.98:0/189988849 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f7b780b8bf0 con 0x7f7b8015c290 2024-09-19T08:45:44.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.391+0000 7f7b75ffb640 1 --2- 172.21.15.98:0/189988849 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7b64067e60 0x7f7b6406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:44.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.391+0000 7f7b75ffb640 1 -- 172.21.15.98:0/189988849 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7f7b78121980 con 0x7f7b8015c290 2024-09-19T08:45:44.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.391+0000 7f7b85267640 1 --2- 172.21.15.98:0/189988849 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7b64067e60 0x7f7b6406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:44.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.391+0000 7f7b85267640 1 --2- 172.21.15.98:0/189988849 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7b64067e60 0x7f7b6406a320 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7b800b41f0 tx=0x7f7b7c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:44.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.395+0000 7f7b75ffb640 1 -- 172.21.15.98:0/189988849 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f7b78016620 con 0x7f7b8015c290 2024-09-19T08:45:44.580 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.583+0000 7f7b874cb640 1 -- 172.21.15.98:0/189988849 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot unprotect", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f7b800b0b40 con 0x7f7b64067e60 2024-09-19T08:45:44.583 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:44.586+0000 7ff73a69c640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-19T08:45:44.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.583+0000 7f7b75ffb640 1 -- 172.21.15.98:0/189988849 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f7b800b0b40 con 0x7f7b64067e60 2024-09-19T08:45:44.584 INFO:teuthology.orchestra.run.smithi098.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-19T08:45:44.589 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-19T08:45:44.590 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls non_existent_subvolume subvolume_1 2024-09-19T08:45:44.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.719+0000 7ff69eed7640 1 Processor -- start 2024-09-19T08:45:44.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.719+0000 7ff69eed7640 1 -- start start 2024-09-19T08:45:44.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.719+0000 7ff69eed7640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff69815a990 0x7ff69806e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:44.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.719+0000 7ff69eed7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff69806e8e0 0x7ff69806ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:44.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.719+0000 7ff69eed7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff6980cf850 0x7ff6980cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:44.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.719+0000 7ff69eed7640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7ff69806f240 con 0x7ff69815a990 2024-09-19T08:45:44.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.719+0000 7ff69eed7640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff6980d05f0 con 0x7ff6980cf850 2024-09-19T08:45:44.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.719+0000 7ff69eed7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff69806d780 con 0x7ff69806e8e0 2024-09-19T08:45:44.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.719+0000 7ff69cc73640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff69815a990 0x7ff69806e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:44.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.719+0000 7ff69cc73640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff69815a990 0x7ff69806e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58352/0 (socket says 172.21.15.98:58352) 2024-09-19T08:45:44.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.719+0000 7ff69cc73640 1 -- 172.21.15.98:0/3142407508 learned_addr learned my addr 172.21.15.98:0/3142407508 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:44.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.719+0000 7ff69cc73640 1 -- 172.21.15.98:0/3142407508 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff69806e8e0 msgr2=0x7ff69806ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:44.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.719+0000 7ff69cc73640 1 --2- 172.21.15.98:0/3142407508 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff69806e8e0 0x7ff69806ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:44.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.719+0000 7ff69cc73640 1 -- 172.21.15.98:0/3142407508 >> v2:172.21.15.139:3300/0 conn(0x7ff6980cf850 msgr2=0x7ff6980cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:44.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.719+0000 7ff69cc73640 1 --2- 172.21.15.98:0/3142407508 >> v2:172.21.15.139:3300/0 conn(0x7ff6980cf850 0x7ff6980cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:44.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.719+0000 7ff69cc73640 1 -- 172.21.15.98:0/3142407508 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69806da60 con 0x7ff69815a990 2024-09-19T08:45:44.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.723+0000 7ff69cc73640 1 --2- 172.21.15.98:0/3142407508 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff69815a990 0x7ff69806e260 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7ff6980cb500 tx=0x7ff69001eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e81af22061b45f1d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:44.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.723+0000 7ff6977fe640 1 -- 172.21.15.98:0/3142407508 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6900ad070 con 0x7ff69815a990 2024-09-19T08:45:44.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.723+0000 7ff6977fe640 1 -- 172.21.15.98:0/3142407508 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6900a7a80 con 0x7ff69815a990 2024-09-19T08:45:44.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.723+0000 7ff6977fe640 1 -- 172.21.15.98:0/3142407508 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6900a7e50 con 0x7ff69815a990 2024-09-19T08:45:44.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.723+0000 7ff69eed7640 1 -- 172.21.15.98:0/3142407508 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff69815a990 msgr2=0x7ff69806e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:44.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.723+0000 7ff69eed7640 1 --2- 172.21.15.98:0/3142407508 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff69815a990 0x7ff69806e260 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7ff6980cb500 tx=0x7ff69001eb60 comp rx=0 tx=0).stop 2024-09-19T08:45:44.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.723+0000 7ff69eed7640 1 -- 172.21.15.98:0/3142407508 shutdown_connections 2024-09-19T08:45:44.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.723+0000 7ff69eed7640 1 --2- 172.21.15.98:0/3142407508 >> v2:172.21.15.139:3300/0 conn(0x7ff6980cf850 0x7ff6980cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:44.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.723+0000 7ff69eed7640 1 --2- 172.21.15.98:0/3142407508 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff69806e8e0 0x7ff69806ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:44.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.723+0000 7ff69eed7640 1 --2- 172.21.15.98:0/3142407508 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff69815a990 0x7ff69806e260 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:44.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.723+0000 7ff69eed7640 1 -- 172.21.15.98:0/3142407508 >> 172.21.15.98:0/3142407508 conn(0x7ff6980b25c0 msgr2=0x7ff6980b29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:44.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.723+0000 7ff69eed7640 1 -- 172.21.15.98:0/3142407508 shutdown_connections 2024-09-19T08:45:44.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.723+0000 7ff69eed7640 1 -- 172.21.15.98:0/3142407508 wait complete. 2024-09-19T08:45:44.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.723+0000 7ff69eed7640 1 Processor -- start 2024-09-19T08:45:44.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.723+0000 7ff69eed7640 1 -- start start 2024-09-19T08:45:44.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.727+0000 7ff69eed7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff69806e8e0 0x7ff69814c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:44.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.727+0000 7ff69eed7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6980cf850 0x7ff69815b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:44.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.727+0000 7ff69eed7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff69815a990 0x7ff6980ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:44.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.727+0000 7ff69eed7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff6980719a0 con 0x7ff69806e8e0 2024-09-19T08:45:44.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.727+0000 7ff69eed7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff698071820 con 0x7ff69815a990 2024-09-19T08:45:44.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.727+0000 7ff69eed7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff698071b20 con 0x7ff6980cf850 2024-09-19T08:45:44.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.727+0000 7ff69cc73640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff69806e8e0 0x7ff69814c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:44.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.727+0000 7ff69cc73640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff69806e8e0 0x7ff69814c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58360/0 (socket says 172.21.15.98:58360) 2024-09-19T08:45:44.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.727+0000 7ff69cc73640 1 -- 172.21.15.98:0/2660244569 learned_addr learned my addr 172.21.15.98:0/2660244569 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:44.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.727+0000 7ff69cc73640 1 -- 172.21.15.98:0/2660244569 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6980cf850 msgr2=0x7ff69815b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:44.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.727+0000 7ff697fff640 1 --2- 172.21.15.98:0/2660244569 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6980cf850 0x7ff69815b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:44.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.727+0000 7ff69cc73640 1 --2- 172.21.15.98:0/2660244569 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff6980cf850 0x7ff69815b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:44.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.727+0000 7ff69cc73640 1 -- 172.21.15.98:0/2660244569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff69815a990 msgr2=0x7ff6980ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:44.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.727+0000 7ff69d474640 1 --2- 172.21.15.98:0/2660244569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff69815a990 0x7ff6980ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:44.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.727+0000 7ff69cc73640 1 --2- 172.21.15.98:0/2660244569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff69815a990 0x7ff6980ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:44.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.727+0000 7ff69cc73640 1 -- 172.21.15.98:0/2660244569 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69815b4b0 con 0x7ff69806e8e0 2024-09-19T08:45:44.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.727+0000 7ff69d474640 1 --2- 172.21.15.98:0/2660244569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff69815a990 0x7ff6980ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:44.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.727+0000 7ff69cc73640 1 --2- 172.21.15.98:0/2660244569 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff69806e8e0 0x7ff69814c9d0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7ff6900b3040 tx=0x7ff69001b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:44.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.727+0000 7ff6957fa640 1 -- 172.21.15.98:0/2660244569 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6900ad040 con 0x7ff69806e8e0 2024-09-19T08:45:44.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.727+0000 7ff6957fa640 1 -- 172.21.15.98:0/2660244569 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6900a7880 con 0x7ff69806e8e0 2024-09-19T08:45:44.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.727+0000 7ff69eed7640 1 -- 172.21.15.98:0/2660244569 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6981fa570 con 0x7ff69806e8e0 2024-09-19T08:45:44.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.727+0000 7ff6957fa640 1 -- 172.21.15.98:0/2660244569 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69000b370 con 0x7ff69806e8e0 2024-09-19T08:45:44.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.727+0000 7ff69eed7640 1 -- 172.21.15.98:0/2660244569 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff698165170 con 0x7ff69806e8e0 2024-09-19T08:45:44.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.727+0000 7ff69eed7640 1 -- 172.21.15.98:0/2660244569 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff69806ed00 con 0x7ff69806e8e0 2024-09-19T08:45:44.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.727+0000 7ff6957fa640 1 -- 172.21.15.98:0/2660244569 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7ff6900c2060 con 0x7ff69806e8e0 2024-09-19T08:45:44.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.727+0000 7ff6957fa640 1 --2- 172.21.15.98:0/2660244569 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff67c067e60 0x7ff67c06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:44.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.727+0000 7ff6957fa640 1 -- 172.21.15.98:0/2660244569 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7ff6901207c0 con 0x7ff69806e8e0 2024-09-19T08:45:44.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.727+0000 7ff697fff640 1 --2- 172.21.15.98:0/2660244569 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff67c067e60 0x7ff67c06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:44.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.731+0000 7ff697fff640 1 --2- 172.21.15.98:0/2660244569 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff67c067e60 0x7ff67c06a320 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff6980733c0 tx=0x7ff688008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:44.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.731+0000 7ff6957fa640 1 -- 172.21.15.98:0/2660244569 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff6900187e0 con 0x7ff69806e8e0 2024-09-19T08:45:44.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.927+0000 7ff69eed7640 1 -- 172.21.15.98:0/2660244569 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7ff698073550 con 0x7ff67c067e60 2024-09-19T08:45:44.928 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:44.930+0000 7ff73a69c640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-19T08:45:44.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:44.931+0000 7ff6957fa640 1 -- 172.21.15.98:0/2660244569 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7ff698073550 con 0x7ff67c067e60 2024-09-19T08:45:44.929 INFO:teuthology.orchestra.run.smithi098.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-19T08:45:44.934 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-19T08:45:44.935 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone non_existent_subvolume subvolume_1 snapshot_1 clone_1 2024-09-19T08:45:45.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.067+0000 7f96cf3a9640 1 Processor -- start 2024-09-19T08:45:45.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.067+0000 7f96cf3a9640 1 -- start start 2024-09-19T08:45:45.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.067+0000 7f96cf3a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96c8145b70 0x7f96c806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:45.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.067+0000 7f96cf3a9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f96c8071ac0 0x7f96c80b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:45.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.067+0000 7f96cf3a9640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96c806c5f0 con 0x7f96c806c1d0 2024-09-19T08:45:45.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.067+0000 7f96cf3a9640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f96c8071f20 con 0x7f96c8071ac0 2024-09-19T08:45:45.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.067+0000 7f96cf3a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f96c8073530 con 0x7f96c8145b70 2024-09-19T08:45:45.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.067+0000 7f96cc944640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f96c806c1d0 0x7f96c80713b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:53116/0 (socket says 172.21.15.98:53116) 2024-09-19T08:45:45.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.067+0000 7f96cc944640 1 -- 172.21.15.98:0/765785871 learned_addr learned my addr 172.21.15.98:0/765785871 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:45.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.067+0000 7f96cd946640 1 --2- 172.21.15.98:0/765785871 >> v2:172.21.15.139:3300/0 conn(0x7f96c8071ac0 0x7f96c80b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:45.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.067+0000 7f96cd145640 1 --2- 172.21.15.98:0/765785871 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96c8145b70 0x7f96c806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:45.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.067+0000 7f96bffff640 1 -- 172.21.15.98:0/765785871 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1948498310 0 0) 0x7f96c806c5f0 con 0x7f96c806c1d0 2024-09-19T08:45:45.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.067+0000 7f96bffff640 1 -- 172.21.15.98:0/765785871 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96b4003530 con 0x7f96c806c1d0 2024-09-19T08:45:45.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.067+0000 7f96bffff640 1 -- 172.21.15.98:0/765785871 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 341919660 0 0) 0x7f96b4003530 con 0x7f96c806c1d0 2024-09-19T08:45:45.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.067+0000 7f96bffff640 1 -- 172.21.15.98:0/765785871 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96c8145b70 msgr2=0x7f96c806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:45.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.067+0000 7f96bffff640 1 --2- 172.21.15.98:0/765785871 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96c8145b70 0x7f96c806bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:45.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.067+0000 7f96bffff640 1 -- 172.21.15.98:0/765785871 >> v2:172.21.15.139:3300/0 conn(0x7f96c8071ac0 msgr2=0x7f96c80b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:45.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.067+0000 7f96bffff640 1 --2- 172.21.15.98:0/765785871 >> v2:172.21.15.139:3300/0 conn(0x7f96c8071ac0 0x7f96c80b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:45.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.067+0000 7f96bffff640 1 -- 172.21.15.98:0/765785871 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c8073810 con 0x7f96c806c1d0 2024-09-19T08:45:45.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.067+0000 7f96bffff640 1 -- 172.21.15.98:0/765785871 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f96b8001680 con 0x7f96c806c1d0 2024-09-19T08:45:45.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.067+0000 7f96bffff640 1 -- 172.21.15.98:0/765785871 >> v1:172.21.15.98:6789/0 conn(0x7f96c806c1d0 legacy=0x7f96c80713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:45.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.067+0000 7f96bffff640 1 --2- 172.21.15.98:0/765785871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96b40039c0 0x7f96b4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:45.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.067+0000 7f96bffff640 1 --2- 172.21.15.98:0/765785871 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96c8145b70 0x7f96b4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:45.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.067+0000 7f96bffff640 1 --2- 172.21.15.98:0/765785871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96b4004d50 0x7f96b40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:45.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.067+0000 7f96bffff640 1 -- 172.21.15.98:0/765785871 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f96c8071f20 con 0x7f96b40039c0 2024-09-19T08:45:45.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.067+0000 7f96bffff640 1 -- 172.21.15.98:0/765785871 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f96c8073530 con 0x7f96b4004d50 2024-09-19T08:45:45.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.067+0000 7f96bffff640 1 -- 172.21.15.98:0/765785871 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f96b400b9d0 con 0x7f96c8145b70 2024-09-19T08:45:45.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.067+0000 7f96cd145640 1 --2- 172.21.15.98:0/765785871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96b40039c0 0x7f96b4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:45.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.067+0000 7f96cd145640 1 -- 172.21.15.98:0/765785871 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96c8145b70 msgr2=0x7f96b4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:45.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.067+0000 7f96cd145640 1 --2- 172.21.15.98:0/765785871 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96c8145b70 0x7f96b4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:45.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.067+0000 7f96cd145640 1 -- 172.21.15.98:0/765785871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96b4004d50 msgr2=0x7f96b40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:45.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.067+0000 7f96cd145640 1 --2- 172.21.15.98:0/765785871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96b4004d50 0x7f96b40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:45.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.067+0000 7f96cd145640 1 -- 172.21.15.98:0/765785871 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b4003530 con 0x7f96b40039c0 2024-09-19T08:45:45.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.071+0000 7f96cd145640 1 --2- 172.21.15.98:0/765785871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96b40039c0 0x7f96b4003de0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f96c4009ac0 tx=0x7f96c401ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d2632d8f92b0a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:45.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.071+0000 7f96bffff640 1 -- 172.21.15.98:0/765785871 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c4009690 con 0x7f96b40039c0 2024-09-19T08:45:45.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.071+0000 7f96bffff640 1 -- 172.21.15.98:0/765785871 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96c40a6470 con 0x7f96b40039c0 2024-09-19T08:45:45.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.071+0000 7f96bffff640 1 -- 172.21.15.98:0/765785871 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c40a6840 con 0x7f96b40039c0 2024-09-19T08:45:45.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.071+0000 7f96cf3a9640 1 -- 172.21.15.98:0/765785871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96b40039c0 msgr2=0x7f96b4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:45.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.071+0000 7f96cf3a9640 1 --2- 172.21.15.98:0/765785871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96b40039c0 0x7f96b4003de0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f96c4009ac0 tx=0x7f96c401ca90 comp rx=0 tx=0).stop 2024-09-19T08:45:45.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.071+0000 7f96cd946640 1 -- 172.21.15.98:0/765785871 reap_dead start 2024-09-19T08:45:45.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.071+0000 7f96cf3a9640 1 -- 172.21.15.98:0/765785871 shutdown_connections 2024-09-19T08:45:45.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.071+0000 7f96cf3a9640 1 -- 172.21.15.98:0/765785871 >> 172.21.15.98:0/765785871 conn(0x7f96c80b2010 msgr2=0x7f96c80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:45.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.071+0000 7f96cf3a9640 1 -- 172.21.15.98:0/765785871 shutdown_connections 2024-09-19T08:45:45.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.071+0000 7f96cf3a9640 1 -- 172.21.15.98:0/765785871 wait complete. 2024-09-19T08:45:45.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.071+0000 7f96cf3a9640 1 Processor -- start 2024-09-19T08:45:45.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.071+0000 7f96cf3a9640 1 -- start start 2024-09-19T08:45:45.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.071+0000 7f96cf3a9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96b40039c0 0x7f96c80aaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:45.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.071+0000 7f96cf3a9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96c80ce460 0x7f96c81fa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:45.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.071+0000 7f96cf3a9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96c80aa4a0 0x7f96c81fadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:45.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.071+0000 7f96cf3a9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f96c80b42e0 con 0x7f96c80ce460 2024-09-19T08:45:45.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.071+0000 7f96cf3a9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f96c80b4160 con 0x7f96b40039c0 2024-09-19T08:45:45.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.071+0000 7f96cf3a9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f96c80b4460 con 0x7f96c80aa4a0 2024-09-19T08:45:45.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.075+0000 7f96cc944640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96c80ce460 0x7f96c81fa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:45.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.075+0000 7f96cc944640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96c80ce460 0x7f96c81fa8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58376/0 (socket says 172.21.15.98:58376) 2024-09-19T08:45:45.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.075+0000 7f96cc944640 1 -- 172.21.15.98:0/501020676 learned_addr learned my addr 172.21.15.98:0/501020676 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:45.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.075+0000 7f96cd145640 1 --2- 172.21.15.98:0/501020676 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96b40039c0 0x7f96c80aaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:45.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.075+0000 7f96cd946640 1 --2- 172.21.15.98:0/501020676 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96c80aa4a0 0x7f96c81fadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:45.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.075+0000 7f96cc944640 1 -- 172.21.15.98:0/501020676 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96c80aa4a0 msgr2=0x7f96c81fadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:45.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.075+0000 7f96cc944640 1 --2- 172.21.15.98:0/501020676 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96c80aa4a0 0x7f96c81fadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:45.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.075+0000 7f96cc944640 1 -- 172.21.15.98:0/501020676 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96b40039c0 msgr2=0x7f96c80aaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:45.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.075+0000 7f96cc944640 1 --2- 172.21.15.98:0/501020676 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96b40039c0 0x7f96c80aaeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:45.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.075+0000 7f96cc944640 1 -- 172.21.15.98:0/501020676 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c80ab3f0 con 0x7f96c80ce460 2024-09-19T08:45:45.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.075+0000 7f96cd946640 1 --2- 172.21.15.98:0/501020676 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96c80aa4a0 0x7f96c81fadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:45.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.075+0000 7f96cc944640 1 --2- 172.21.15.98:0/501020676 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96c80ce460 0x7f96c81fa8b0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f96b800c730 tx=0x7f96b800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:45.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.075+0000 7f96bdffb640 1 -- 172.21.15.98:0/501020676 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b8018070 con 0x7f96c80ce460 2024-09-19T08:45:45.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.075+0000 7f96bdffb640 1 -- 172.21.15.98:0/501020676 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96b80072d0 con 0x7f96c80ce460 2024-09-19T08:45:45.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.075+0000 7f96bdffb640 1 -- 172.21.15.98:0/501020676 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b80047f0 con 0x7f96c80ce460 2024-09-19T08:45:45.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.075+0000 7f96cf3a9640 1 -- 172.21.15.98:0/501020676 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96c80aaa10 con 0x7f96c80ce460 2024-09-19T08:45:45.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.075+0000 7f96cf3a9640 1 -- 172.21.15.98:0/501020676 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96c80cdc60 con 0x7f96c80ce460 2024-09-19T08:45:45.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.075+0000 7f96cf3a9640 1 -- 172.21.15.98:0/501020676 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96c80cd770 con 0x7f96c80ce460 2024-09-19T08:45:45.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.075+0000 7f96bdffb640 1 -- 172.21.15.98:0/501020676 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f96b8004ba0 con 0x7f96c80ce460 2024-09-19T08:45:45.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.075+0000 7f96bdffb640 1 --2- 172.21.15.98:0/501020676 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f96ac067e90 0x7f96ac06a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:45.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.075+0000 7f96bdffb640 1 -- 172.21.15.98:0/501020676 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7f96b8085ca0 con 0x7f96c80ce460 2024-09-19T08:45:45.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.075+0000 7f96cd145640 1 --2- 172.21.15.98:0/501020676 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f96ac067e90 0x7f96ac06a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:45.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.079+0000 7f96cd145640 1 --2- 172.21.15.98:0/501020676 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f96ac067e90 0x7f96ac06a350 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f96c40ad000 tx=0x7f96c40b2040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:45.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.079+0000 7f96bdffb640 1 -- 172.21.15.98:0/501020676 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f96b8055fe0 con 0x7f96c80ce460 2024-09-19T08:45:45.263 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.263+0000 7f96cf3a9640 1 -- 172.21.15.98:0/501020676 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target_sub_name": "clone_1", "target": ["mon-mgr", ""]}) -- 0x7f96c80b3410 con 0x7f96ac067e90 2024-09-19T08:45:45.276 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:45.278+0000 7ff73a69c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:45:45.277 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:45.278+0000 7ff73a69c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:45:45.277 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:45.278+0000 7ff73a69c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:45:45.277 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:45.278+0000 7ff73a69c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:45:45.277 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:45.278+0000 7ff73a69c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:45:45.283 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:45.286+0000 7ff73a69c640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-19T08:45:45.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.283+0000 7f96bdffb640 1 -- 172.21.15.98:0/501020676 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f96c80b3410 con 0x7f96ac067e90 2024-09-19T08:45:45.284 INFO:teuthology.orchestra.run.smithi098.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-19T08:45:45.289 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-19T08:45:45.290 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status non_existent_subvolume clone_1 2024-09-19T08:45:45.414 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.415+0000 7fb787c10640 1 Processor -- start 2024-09-19T08:45:45.415 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.415+0000 7fb787c10640 1 -- start start 2024-09-19T08:45:45.416 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.415+0000 7fb787c10640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb780074c70 0x7fb7800750e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:45.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.415+0000 7fb787c10640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb78006ee40 con 0x7fb7801661c0 2024-09-19T08:45:45.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.415+0000 7fb787c10640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7800af670 con 0x7fb78015e7b0 2024-09-19T08:45:45.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.415+0000 7fb787c10640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fb780075620 con 0x7fb780074c70 2024-09-19T08:45:45.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.419+0000 7fb7861ad640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fb7801661c0 0x7fb7801685b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:53122/0 (socket says 172.21.15.98:53122) 2024-09-19T08:45:45.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.419+0000 7fb7861ad640 1 -- 172.21.15.98:0/440436636 learned_addr learned my addr 172.21.15.98:0/440436636 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:45.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.419+0000 7fb7851ab640 1 --2- 172.21.15.98:0/440436636 >> v2:172.21.15.139:3300/0 conn(0x7fb780074c70 0x7fb7800750e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:45.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.419+0000 7fb7849aa640 1 -- 172.21.15.98:0/440436636 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2044633765 0 0) 0x7fb78006ee40 con 0x7fb7801661c0 2024-09-19T08:45:45.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.419+0000 7fb7849aa640 1 -- 172.21.15.98:0/440436636 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb76c003530 con 0x7fb7801661c0 2024-09-19T08:45:45.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.419+0000 7fb7849aa640 1 -- 172.21.15.98:0/440436636 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1187655222 0 0) 0x7fb76c003530 con 0x7fb7801661c0 2024-09-19T08:45:45.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.419+0000 7fb7851ab640 1 -- 172.21.15.98:0/440436636 >> v1:172.21.15.139:6789/0 conn(0x7fb78015e7b0 legacy=0x7fb7800d0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:45.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.419+0000 7fb7851ab640 1 -- 172.21.15.98:0/440436636 >> v1:172.21.15.98:6789/0 conn(0x7fb7801661c0 legacy=0x7fb7801685b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:45.417 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.419+0000 7fb7851ab640 1 -- 172.21.15.98:0/440436636 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb78006e5a0 con 0x7fb780074c70 2024-09-19T08:45:45.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.419+0000 7fb7851ab640 1 --2- 172.21.15.98:0/440436636 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb780074c70 0x7fb7800750e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb7700097c0 tx=0x7fb77001c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71c55f3962c9b011 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:45.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.419+0000 7fb7849aa640 1 -- 172.21.15.98:0/440436636 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7700b0070 con 0x7fb780074c70 2024-09-19T08:45:45.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.419+0000 7fb7849aa640 1 -- 172.21.15.98:0/440436636 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7700ab400 con 0x7fb780074c70 2024-09-19T08:45:45.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.419+0000 7fb7849aa640 1 -- 172.21.15.98:0/440436636 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb778001680 con 0x7fb780074c70 2024-09-19T08:45:45.420 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.423+0000 7fb787c10640 1 -- 172.21.15.98:0/440436636 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb780074c70 msgr2=0x7fb7800750e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:45.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.423+0000 7fb787c10640 1 --2- 172.21.15.98:0/440436636 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb780074c70 0x7fb7800750e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb7700097c0 tx=0x7fb77001c3c0 comp rx=0 tx=0).stop 2024-09-19T08:45:45.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.423+0000 7fb787c10640 1 -- 172.21.15.98:0/440436636 shutdown_connections 2024-09-19T08:45:45.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.423+0000 7fb787c10640 1 --2- 172.21.15.98:0/440436636 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb780074c70 0x7fb7800750e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:45.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.423+0000 7fb787c10640 1 -- 172.21.15.98:0/440436636 >> 172.21.15.98:0/440436636 conn(0x7fb7800d1250 msgr2=0x7fb7800d1660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:45.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.423+0000 7fb787c10640 1 -- 172.21.15.98:0/440436636 shutdown_connections 2024-09-19T08:45:45.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.423+0000 7fb787c10640 1 -- 172.21.15.98:0/440436636 wait complete. 2024-09-19T08:45:45.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.423+0000 7fb787c10640 1 Processor -- start 2024-09-19T08:45:45.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.423+0000 7fb787c10640 1 -- start start 2024-09-19T08:45:45.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.423+0000 7fb787c10640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb780074c70 0x7fb7800cd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:45.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.423+0000 7fb787c10640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb78015e7b0 0x7fb78015b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:45.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.423+0000 7fb787c10640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7801661c0 0x7fb780158850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:45.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.423+0000 7fb787c10640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb7800d0790 con 0x7fb780074c70 2024-09-19T08:45:45.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.423+0000 7fb787c10640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb78006bc90 con 0x7fb7801661c0 2024-09-19T08:45:45.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.423+0000 7fb787c10640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb7800d0910 con 0x7fb78015e7b0 2024-09-19T08:45:45.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.423+0000 7fb7859ac640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb780074c70 0x7fb7800cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:45.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.423+0000 7fb7859ac640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb780074c70 0x7fb7800cd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58392/0 (socket says 172.21.15.98:58392) 2024-09-19T08:45:45.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.423+0000 7fb7859ac640 1 -- 172.21.15.98:0/919997856 learned_addr learned my addr 172.21.15.98:0/919997856 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:45.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.423+0000 7fb7861ad640 1 --2- 172.21.15.98:0/919997856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7801661c0 0x7fb780158850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:45.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.423+0000 7fb7851ab640 1 --2- 172.21.15.98:0/919997856 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb78015e7b0 0x7fb78015b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:45.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.423+0000 7fb7859ac640 1 -- 172.21.15.98:0/919997856 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb78015e7b0 msgr2=0x7fb78015b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:45.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.423+0000 7fb7859ac640 1 --2- 172.21.15.98:0/919997856 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb78015e7b0 0x7fb78015b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:45.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.423+0000 7fb7859ac640 1 -- 172.21.15.98:0/919997856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7801661c0 msgr2=0x7fb780158850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:45.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.423+0000 7fb7859ac640 1 --2- 172.21.15.98:0/919997856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7801661c0 0x7fb780158850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:45.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.423+0000 7fb7859ac640 1 -- 172.21.15.98:0/919997856 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb78015f7f0 con 0x7fb780074c70 2024-09-19T08:45:45.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.423+0000 7fb7861ad640 1 --2- 172.21.15.98:0/919997856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb7801661c0 0x7fb780158850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:45.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.423+0000 7fb7851ab640 1 --2- 172.21.15.98:0/919997856 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb78015e7b0 0x7fb78015b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:45.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.423+0000 7fb7859ac640 1 --2- 172.21.15.98:0/919997856 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb780074c70 0x7fb7800cd110 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fb77c00c7c0 tx=0x7fb77c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:45.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.423+0000 7fb7767fc640 1 -- 172.21.15.98:0/919997856 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c003ba0 con 0x7fb780074c70 2024-09-19T08:45:45.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.423+0000 7fb7767fc640 1 -- 172.21.15.98:0/919997856 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb77c003d40 con 0x7fb780074c70 2024-09-19T08:45:45.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.423+0000 7fb787c10640 1 -- 172.21.15.98:0/919997856 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb78015fa90 con 0x7fb780074c70 2024-09-19T08:45:45.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.423+0000 7fb7767fc640 1 -- 172.21.15.98:0/919997856 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c00aa90 con 0x7fb780074c70 2024-09-19T08:45:45.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.423+0000 7fb787c10640 1 -- 172.21.15.98:0/919997856 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7800b6010 con 0x7fb780074c70 2024-09-19T08:45:45.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.427+0000 7fb787c10640 1 -- 172.21.15.98:0/919997856 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7800750e0 con 0x7fb780074c70 2024-09-19T08:45:45.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.427+0000 7fb7767fc640 1 -- 172.21.15.98:0/919997856 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fb77c00ac30 con 0x7fb780074c70 2024-09-19T08:45:45.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.427+0000 7fb7767fc640 1 --2- 172.21.15.98:0/919997856 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb764067e60 0x7fb76406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:45.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.427+0000 7fb7767fc640 1 -- 172.21.15.98:0/919997856 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7fb77c0865d0 con 0x7fb780074c70 2024-09-19T08:45:45.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.427+0000 7fb7851ab640 1 --2- 172.21.15.98:0/919997856 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb764067e60 0x7fb76406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:45.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.427+0000 7fb7851ab640 1 --2- 172.21.15.98:0/919997856 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb764067e60 0x7fb76406a320 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb7700ad000 tx=0x7fb770008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:45.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.431+0000 7fb7767fc640 1 -- 172.21.15.98:0/919997856 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb77c056980 con 0x7fb780074c70 2024-09-19T08:45:45.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.615+0000 7fb787c10640 1 -- 172.21.15.98:0/919997856 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "non_existent_subvolume", "clone_name": "clone_1", "target": ["mon-mgr", ""]}) -- 0x7fb78006d250 con 0x7fb764067e60 2024-09-19T08:45:45.617 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:45.618+0000 7ff73a69c640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-19T08:45:45.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.619+0000 7fb7767fc640 1 -- 172.21.15.98:0/919997856 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fb78006d250 con 0x7fb764067e60 2024-09-19T08:45:45.618 INFO:teuthology.orchestra.run.smithi098.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-19T08:45:45.623 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-19T08:45:45.624 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create non_existent_subvolume group_1 2024-09-19T08:45:45.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.747+0000 7fb665a7b640 1 Processor -- start 2024-09-19T08:45:45.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.747+0000 7fb665a7b640 1 -- start start 2024-09-19T08:45:45.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.747+0000 7fb665a7b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6601415a0 0x7fb6600d20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:45.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.747+0000 7fb665a7b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb6600d25f0 0x7fb6600d3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:45.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.747+0000 7fb665a7b640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb66015eae0 con 0x7fb6600d38a0 2024-09-19T08:45:45.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.747+0000 7fb665a7b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fb6600d2a30 con 0x7fb6600d25f0 2024-09-19T08:45:45.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.747+0000 7fb665a7b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb66015edc0 con 0x7fb6601415a0 2024-09-19T08:45:45.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.747+0000 7fb65ffff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fb6600d38a0 0x7fb660161fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:53124/0 (socket says 172.21.15.98:53124) 2024-09-19T08:45:45.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.747+0000 7fb65ffff640 1 -- 172.21.15.98:0/1659380242 learned_addr learned my addr 172.21.15.98:0/1659380242 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:45.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.747+0000 7fb65effd640 1 --2- 172.21.15.98:0/1659380242 >> v2:172.21.15.139:3300/0 conn(0x7fb6600d25f0 0x7fb6600d3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:45.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.747+0000 7fb65f7fe640 1 --2- 172.21.15.98:0/1659380242 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6601415a0 0x7fb6600d20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:45.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.747+0000 7fb65e7fc640 1 -- 172.21.15.98:0/1659380242 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1292484347 0 0) 0x7fb66015eae0 con 0x7fb6600d38a0 2024-09-19T08:45:45.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.747+0000 7fb65e7fc640 1 -- 172.21.15.98:0/1659380242 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb648003530 con 0x7fb6600d38a0 2024-09-19T08:45:45.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.747+0000 7fb65e7fc640 1 -- 172.21.15.98:0/1659380242 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2712200465 0 0) 0x7fb648003530 con 0x7fb6600d38a0 2024-09-19T08:45:45.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.747+0000 7fb65e7fc640 1 -- 172.21.15.98:0/1659380242 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6601415a0 msgr2=0x7fb6600d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:45.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.747+0000 7fb65e7fc640 1 --2- 172.21.15.98:0/1659380242 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6601415a0 0x7fb6600d20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:45.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.747+0000 7fb65e7fc640 1 -- 172.21.15.98:0/1659380242 >> v2:172.21.15.139:3300/0 conn(0x7fb6600d25f0 msgr2=0x7fb6600d3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:45.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.747+0000 7fb65e7fc640 1 --2- 172.21.15.98:0/1659380242 >> v2:172.21.15.139:3300/0 conn(0x7fb6600d25f0 0x7fb6600d3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:45.748 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.747+0000 7fb65e7fc640 1 -- 172.21.15.98:0/1659380242 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb66015f0a0 con 0x7fb6600d38a0 2024-09-19T08:45:45.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.747+0000 7fb65e7fc640 1 -- 172.21.15.98:0/1659380242 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fb658001a00 con 0x7fb6600d38a0 2024-09-19T08:45:45.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.747+0000 7fb65e7fc640 1 -- 172.21.15.98:0/1659380242 >> v1:172.21.15.98:6789/0 conn(0x7fb6600d38a0 legacy=0x7fb660161fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:45.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.747+0000 7fb65e7fc640 1 --2- 172.21.15.98:0/1659380242 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6480039c0 0x7fb648003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:45.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.747+0000 7fb65e7fc640 1 --2- 172.21.15.98:0/1659380242 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6601415a0 0x7fb648004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:45.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.747+0000 7fb65e7fc640 1 --2- 172.21.15.98:0/1659380242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb648004d50 0x7fb6480051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:45.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.747+0000 7fb65e7fc640 1 -- 172.21.15.98:0/1659380242 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb6600d2a30 con 0x7fb6480039c0 2024-09-19T08:45:45.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.747+0000 7fb65e7fc640 1 -- 172.21.15.98:0/1659380242 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb66015edc0 con 0x7fb648004d50 2024-09-19T08:45:45.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.747+0000 7fb65e7fc640 1 -- 172.21.15.98:0/1659380242 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb64800b9d0 con 0x7fb6601415a0 2024-09-19T08:45:45.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.747+0000 7fb65f7fe640 1 --2- 172.21.15.98:0/1659380242 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6480039c0 0x7fb648003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:45.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.747+0000 7fb65f7fe640 1 -- 172.21.15.98:0/1659380242 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6601415a0 msgr2=0x7fb648004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:45.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.747+0000 7fb65f7fe640 1 --2- 172.21.15.98:0/1659380242 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6601415a0 0x7fb648004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:45.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.747+0000 7fb65f7fe640 1 -- 172.21.15.98:0/1659380242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb648004d50 msgr2=0x7fb6480051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:45.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.747+0000 7fb65f7fe640 1 --2- 172.21.15.98:0/1659380242 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb648004d50 0x7fb6480051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:45.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.747+0000 7fb65f7fe640 1 -- 172.21.15.98:0/1659380242 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb648003530 con 0x7fb6480039c0 2024-09-19T08:45:45.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.751+0000 7fb65f7fe640 1 --2- 172.21.15.98:0/1659380242 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6480039c0 0x7fb648003de0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fb650009ae0 tx=0x7fb65001ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf098109b5d17aa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:45.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.751+0000 7fb65e7fc640 1 -- 172.21.15.98:0/1659380242 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6500b2070 con 0x7fb6480039c0 2024-09-19T08:45:45.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.751+0000 7fb65e7fc640 1 -- 172.21.15.98:0/1659380242 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6500a5560 con 0x7fb6480039c0 2024-09-19T08:45:45.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.751+0000 7fb65e7fc640 1 -- 172.21.15.98:0/1659380242 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6500096b0 con 0x7fb6480039c0 2024-09-19T08:45:45.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.751+0000 7fb665a7b640 1 -- 172.21.15.98:0/1659380242 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6480039c0 msgr2=0x7fb648003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:45.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.751+0000 7fb665a7b640 1 --2- 172.21.15.98:0/1659380242 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6480039c0 0x7fb648003de0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fb650009ae0 tx=0x7fb65001ca80 comp rx=0 tx=0).stop 2024-09-19T08:45:45.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.751+0000 7fb65ffff640 1 -- 172.21.15.98:0/1659380242 reap_dead start 2024-09-19T08:45:45.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.751+0000 7fb665a7b640 1 -- 172.21.15.98:0/1659380242 shutdown_connections 2024-09-19T08:45:45.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.751+0000 7fb665a7b640 1 -- 172.21.15.98:0/1659380242 >> 172.21.15.98:0/1659380242 conn(0x7fb660074c70 msgr2=0x7fb660075080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:45.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.751+0000 7fb665a7b640 1 -- 172.21.15.98:0/1659380242 shutdown_connections 2024-09-19T08:45:45.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.751+0000 7fb665a7b640 1 -- 172.21.15.98:0/1659380242 wait complete. 2024-09-19T08:45:45.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.751+0000 7fb665a7b640 1 Processor -- start 2024-09-19T08:45:45.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.751+0000 7fb665a7b640 1 -- start start 2024-09-19T08:45:45.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.755+0000 7fb665a7b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6480039c0 0x7fb6601f8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:45.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.755+0000 7fb665a7b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6601fa490 0x7fb6600c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:45.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.755+0000 7fb665a7b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6600c9ea0 0x7fb6601f7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:45.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.755+0000 7fb665a7b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb6600d42a0 con 0x7fb6600c9ea0 2024-09-19T08:45:45.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.755+0000 7fb665a7b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb6600d4120 con 0x7fb6601fa490 2024-09-19T08:45:45.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.755+0000 7fb665a7b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb6600d3cf0 con 0x7fb6480039c0 2024-09-19T08:45:45.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.755+0000 7fb65ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6600c9ea0 0x7fb6601f7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:45.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.755+0000 7fb65ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6600c9ea0 0x7fb6601f7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58416/0 (socket says 172.21.15.98:58416) 2024-09-19T08:45:45.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.755+0000 7fb65ffff640 1 -- 172.21.15.98:0/1765248098 learned_addr learned my addr 172.21.15.98:0/1765248098 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:45.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.755+0000 7fb65f7fe640 1 --2- 172.21.15.98:0/1765248098 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6480039c0 0x7fb6601f8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:45.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.755+0000 7fb65effd640 1 --2- 172.21.15.98:0/1765248098 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6601fa490 0x7fb6600c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:45.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.755+0000 7fb65ffff640 1 -- 172.21.15.98:0/1765248098 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6480039c0 msgr2=0x7fb6601f8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:45.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.755+0000 7fb65ffff640 1 --2- 172.21.15.98:0/1765248098 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb6480039c0 0x7fb6601f8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:45.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.755+0000 7fb65ffff640 1 -- 172.21.15.98:0/1765248098 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6601fa490 msgr2=0x7fb6600c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:45.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.755+0000 7fb65ffff640 1 --2- 172.21.15.98:0/1765248098 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb6601fa490 0x7fb6600c9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:45.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.755+0000 7fb65ffff640 1 -- 172.21.15.98:0/1765248098 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6600ced40 con 0x7fb6600c9ea0 2024-09-19T08:45:45.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.755+0000 7fb65ffff640 1 --2- 172.21.15.98:0/1765248098 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb6600c9ea0 0x7fb6601f7a20 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fb6580013e0 tx=0x7fb65800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:45.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.755+0000 7fb647fff640 1 -- 172.21.15.98:0/1765248098 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb658007110 con 0x7fb6600c9ea0 2024-09-19T08:45:45.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.755+0000 7fb647fff640 1 -- 172.21.15.98:0/1765248098 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6580072b0 con 0x7fb6600c9ea0 2024-09-19T08:45:45.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.755+0000 7fb665a7b640 1 -- 172.21.15.98:0/1765248098 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6600cf020 con 0x7fb6600c9ea0 2024-09-19T08:45:45.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.755+0000 7fb647fff640 1 -- 172.21.15.98:0/1765248098 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6580076a0 con 0x7fb6600c9ea0 2024-09-19T08:45:45.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.755+0000 7fb665a7b640 1 -- 172.21.15.98:0/1765248098 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6600cf5e0 con 0x7fb6600c9ea0 2024-09-19T08:45:45.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.755+0000 7fb665a7b640 1 -- 172.21.15.98:0/1765248098 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6600d3360 con 0x7fb6600c9ea0 2024-09-19T08:45:45.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.755+0000 7fb647fff640 1 -- 172.21.15.98:0/1765248098 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fb658007840 con 0x7fb6600c9ea0 2024-09-19T08:45:45.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.755+0000 7fb647fff640 1 --2- 172.21.15.98:0/1765248098 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb64c067e60 0x7fb64c06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:45.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.755+0000 7fb647fff640 1 -- 172.21.15.98:0/1765248098 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7fb658086480 con 0x7fb6600c9ea0 2024-09-19T08:45:45.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.755+0000 7fb65f7fe640 1 --2- 172.21.15.98:0/1765248098 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb64c067e60 0x7fb64c06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:45.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.759+0000 7fb65f7fe640 1 --2- 172.21.15.98:0/1765248098 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb64c067e60 0x7fb64c06a320 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb6500af000 tx=0x7fb6500046d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:45.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.759+0000 7fb647fff640 1 -- 172.21.15.98:0/1765248098 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb658022080 con 0x7fb6600c9ea0 2024-09-19T08:45:45.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.947+0000 7fb665a7b640 1 -- 172.21.15.98:0/1765248098 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7fb6600cf7b0 con 0x7fb64c067e60 2024-09-19T08:45:45.948 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:45.950+0000 7ff73a69c640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-19T08:45:45.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:45.951+0000 7fb647fff640 1 -- 172.21.15.98:0/1765248098 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fb6600cf7b0 con 0x7fb64c067e60 2024-09-19T08:45:45.948 INFO:teuthology.orchestra.run.smithi098.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-19T08:45:45.953 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-19T08:45:45.954 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm non_existent_subvolume group_1 2024-09-19T08:45:46.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.075+0000 7f9b71b12640 1 Processor -- start 2024-09-19T08:45:46.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.075+0000 7f9b71b12640 1 -- start start 2024-09-19T08:45:46.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.079+0000 7f9b71b12640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9b6c06bc90 0x7f9b6c06c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:46.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.079+0000 7f9b71b12640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9b6c06f380 0x7f9b6c06f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:46.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.079+0000 7f9b71b12640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b6c06c800 con 0x7f9b6c15e680 2024-09-19T08:45:46.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.079+0000 7f9b71b12640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f9b6c15ea60 con 0x7f9b6c06f380 2024-09-19T08:45:46.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.079+0000 7f9b71b12640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9b6c0d08d0 con 0x7f9b6c06bc90 2024-09-19T08:45:46.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.079+0000 7f9b6bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9b6c06f380 0x7f9b6c06f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:46.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.079+0000 7f9b6affd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9b6c06bc90 0x7f9b6c06c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:46.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.079+0000 7f9b6bfff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9b6c06f380 0x7f9b6c06f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:47002/0 (socket says 172.21.15.98:47002) 2024-09-19T08:45:46.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.079+0000 7f9b6bfff640 1 -- 172.21.15.98:0/1015425362 learned_addr learned my addr 172.21.15.98:0/1015425362 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:46.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.079+0000 7f9b6affd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9b6c06bc90 0x7f9b6c06c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:43196/0 (socket says 172.21.15.98:43196) 2024-09-19T08:45:46.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.079+0000 7f9b6bfff640 1 -- 172.21.15.98:0/1015425362 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9b6c06bc90 msgr2=0x7f9b6c06c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:46.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.079+0000 7f9b6bfff640 1 --2- 172.21.15.98:0/1015425362 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9b6c06bc90 0x7f9b6c06c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:46.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.079+0000 7f9b6bfff640 1 -- 172.21.15.98:0/1015425362 >> v1:172.21.15.139:6789/0 conn(0x7f9b6c15e680 legacy=0x7f9b6c0d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:46.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.079+0000 7f9b6bfff640 1 -- 172.21.15.98:0/1015425362 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b6c0cffa0 con 0x7f9b6c06f380 2024-09-19T08:45:46.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.079+0000 7f9b6bfff640 1 --2- 172.21.15.98:0/1015425362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b6c06f380 0x7f9b6c06f810 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9b64009870 tx=0x7f9b6401c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=328daf721dbcc41b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:46.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.083+0000 7f9b6a7fc640 1 -- 172.21.15.98:0/1015425362 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b640b2070 con 0x7f9b6c06f380 2024-09-19T08:45:46.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.083+0000 7f9b6a7fc640 1 -- 172.21.15.98:0/1015425362 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b640a9070 con 0x7f9b6c06f380 2024-09-19T08:45:46.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.083+0000 7f9b6a7fc640 1 -- 172.21.15.98:0/1015425362 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b640ad880 con 0x7f9b6c06f380 2024-09-19T08:45:46.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.083+0000 7f9b71b12640 1 -- 172.21.15.98:0/1015425362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b6c06f380 msgr2=0x7f9b6c06f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:46.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.083+0000 7f9b71b12640 1 --2- 172.21.15.98:0/1015425362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b6c06f380 0x7f9b6c06f810 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9b64009870 tx=0x7f9b6401c560 comp rx=0 tx=0).stop 2024-09-19T08:45:46.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.083+0000 7f9b71b12640 1 -- 172.21.15.98:0/1015425362 shutdown_connections 2024-09-19T08:45:46.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.083+0000 7f9b71b12640 1 --2- 172.21.15.98:0/1015425362 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b6c06f380 0x7f9b6c06f810 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:46.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.083+0000 7f9b71b12640 1 --2- 172.21.15.98:0/1015425362 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9b6c06bc90 0x7f9b6c06c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:46.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.083+0000 7f9b71b12640 1 -- 172.21.15.98:0/1015425362 >> 172.21.15.98:0/1015425362 conn(0x7f9b6c0d23d0 msgr2=0x7f9b6c0d27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:46.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.083+0000 7f9b71b12640 1 -- 172.21.15.98:0/1015425362 shutdown_connections 2024-09-19T08:45:46.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.083+0000 7f9b71b12640 1 -- 172.21.15.98:0/1015425362 wait complete. 2024-09-19T08:45:46.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.083+0000 7f9b71b12640 1 Processor -- start 2024-09-19T08:45:46.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.083+0000 7f9b71b12640 1 -- start start 2024-09-19T08:45:46.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.083+0000 7f9b71b12640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9b6c06bc90 0x7f9b6c14b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:46.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.083+0000 7f9b71b12640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b6c06f380 0x7f9b6c0b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:46.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.083+0000 7f9b71b12640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b6c15e680 0x7f9b6c158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:46.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.083+0000 7f9b71b12640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6c073920 con 0x7f9b6c15e680 2024-09-19T08:45:46.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.083+0000 7f9b71b12640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9b6c0737a0 con 0x7f9b6c06f380 2024-09-19T08:45:46.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.083+0000 7f9b71b12640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9b6c073aa0 con 0x7f9b6c06bc90 2024-09-19T08:45:46.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.083+0000 7f9b6bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b6c15e680 0x7f9b6c158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:46.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.083+0000 7f9b6bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b6c15e680 0x7f9b6c158730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58432/0 (socket says 172.21.15.98:58432) 2024-09-19T08:45:46.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.083+0000 7f9b6bfff640 1 -- 172.21.15.98:0/2808495460 learned_addr learned my addr 172.21.15.98:0/2808495460 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:46.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.083+0000 7f9b6b7fe640 1 --2- 172.21.15.98:0/2808495460 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9b6c06bc90 0x7f9b6c14b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:46.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.083+0000 7f9b6affd640 1 --2- 172.21.15.98:0/2808495460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b6c06f380 0x7f9b6c0b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:46.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.083+0000 7f9b6bfff640 1 -- 172.21.15.98:0/2808495460 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9b6c06bc90 msgr2=0x7f9b6c14b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:46.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.083+0000 7f9b6bfff640 1 --2- 172.21.15.98:0/2808495460 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9b6c06bc90 0x7f9b6c14b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:46.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.083+0000 7f9b6bfff640 1 -- 172.21.15.98:0/2808495460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b6c06f380 msgr2=0x7f9b6c0b5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:46.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.083+0000 7f9b6bfff640 1 --2- 172.21.15.98:0/2808495460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b6c06f380 0x7f9b6c0b5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:46.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.083+0000 7f9b6bfff640 1 -- 172.21.15.98:0/2808495460 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b6c14bec0 con 0x7f9b6c15e680 2024-09-19T08:45:46.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.083+0000 7f9b6affd640 1 --2- 172.21.15.98:0/2808495460 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b6c06f380 0x7f9b6c0b5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:46.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.083+0000 7f9b6bfff640 1 --2- 172.21.15.98:0/2808495460 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b6c15e680 0x7f9b6c158730 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f9b640b0040 tx=0x7f9b6401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:46.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.083+0000 7f9b53fff640 1 -- 172.21.15.98:0/2808495460 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b640bc020 con 0x7f9b6c15e680 2024-09-19T08:45:46.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.083+0000 7f9b53fff640 1 -- 172.21.15.98:0/2808495460 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b640adda0 con 0x7f9b6c15e680 2024-09-19T08:45:46.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.083+0000 7f9b71b12640 1 -- 172.21.15.98:0/2808495460 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b6c152950 con 0x7f9b6c15e680 2024-09-19T08:45:46.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.083+0000 7f9b53fff640 1 -- 172.21.15.98:0/2808495460 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b640b2040 con 0x7f9b6c15e680 2024-09-19T08:45:46.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.083+0000 7f9b71b12640 1 -- 172.21.15.98:0/2808495460 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b6c156a00 con 0x7f9b6c15e680 2024-09-19T08:45:46.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.087+0000 7f9b71b12640 1 -- 172.21.15.98:0/2808495460 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b6c06c0d0 con 0x7f9b6c15e680 2024-09-19T08:45:46.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.087+0000 7f9b53fff640 1 -- 172.21.15.98:0/2808495460 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f9b640bf020 con 0x7f9b6c15e680 2024-09-19T08:45:46.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.087+0000 7f9b53fff640 1 --2- 172.21.15.98:0/2808495460 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9b58067e60 0x7f9b5806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:46.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.087+0000 7f9b53fff640 1 -- 172.21.15.98:0/2808495460 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7f9b64120960 con 0x7f9b6c15e680 2024-09-19T08:45:46.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.087+0000 7f9b6b7fe640 1 --2- 172.21.15.98:0/2808495460 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9b58067e60 0x7f9b5806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:46.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.087+0000 7f9b6b7fe640 1 --2- 172.21.15.98:0/2808495460 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9b58067e60 0x7f9b5806a320 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c001d50 tx=0x7f9b5c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:46.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.091+0000 7f9b53fff640 1 -- 172.21.15.98:0/2808495460 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9b640c6030 con 0x7f9b6c15e680 2024-09-19T08:45:46.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.259+0000 7f9b71b12640 1 -- 172.21.15.98:0/2808495460 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7f9b6c155490 con 0x7f9b58067e60 2024-09-19T08:45:46.262 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:46.262+0000 7ff73a69c640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-19T08:45:46.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.263+0000 7f9b53fff640 1 -- 172.21.15.98:0/2808495460 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f9b6c155490 con 0x7f9b58067e60 2024-09-19T08:45:46.262 INFO:teuthology.orchestra.run.smithi098.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-19T08:45:46.268 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-19T08:45:46.273 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup getpath non_existent_subvolume group_1 2024-09-19T08:45:46.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.395+0000 7f173973a640 1 Processor -- start 2024-09-19T08:45:46.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.395+0000 7f173973a640 1 -- start start 2024-09-19T08:45:46.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.395+0000 7f173973a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1734145b70 0x7f173406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:46.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.395+0000 7f173973a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f173406c1d0 0x7f17340713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:46.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.395+0000 7f173973a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f173406c5f0 con 0x7f17340718f0 2024-09-19T08:45:46.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.395+0000 7f173973a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f1734073530 con 0x7f173406c1d0 2024-09-19T08:45:46.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.395+0000 7f173973a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1734073810 con 0x7f1734145b70 2024-09-19T08:45:46.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.399+0000 7f1732ffd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1734145b70 0x7f173406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:46.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.399+0000 7f17327fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f173406c1d0 0x7f17340713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:46.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.399+0000 7f17327fc640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f173406c1d0 0x7f17340713b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:47012/0 (socket says 172.21.15.98:47012) 2024-09-19T08:45:46.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.399+0000 7f1732ffd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1734145b70 0x7f173406bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:43220/0 (socket says 172.21.15.98:43220) 2024-09-19T08:45:46.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.399+0000 7f17327fc640 1 -- 172.21.15.98:0/2185701214 learned_addr learned my addr 172.21.15.98:0/2185701214 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:46.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.399+0000 7f17327fc640 1 -- 172.21.15.98:0/2185701214 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1734145b70 msgr2=0x7f173406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:46.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.399+0000 7f17327fc640 1 --2- 172.21.15.98:0/2185701214 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1734145b70 0x7f173406bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:46.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.399+0000 7f17327fc640 1 -- 172.21.15.98:0/2185701214 >> v1:172.21.15.139:6789/0 conn(0x7f17340718f0 legacy=0x7f17340cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:46.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.399+0000 7f17327fc640 1 -- 172.21.15.98:0/2185701214 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1734073af0 con 0x7f173406c1d0 2024-09-19T08:45:46.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.399+0000 7f1732ffd640 1 --2- 172.21.15.98:0/2185701214 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1734145b70 0x7f173406bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:45:46.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.399+0000 7f17327fc640 1 --2- 172.21.15.98:0/2185701214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f173406c1d0 0x7f17340713b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f171c009870 tx=0x7f171c01c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e67aafd1678efaf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:46.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.403+0000 7f1731ffb640 1 -- 172.21.15.98:0/2185701214 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171c0b0070 con 0x7f173406c1d0 2024-09-19T08:45:46.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.403+0000 7f1731ffb640 1 -- 172.21.15.98:0/2185701214 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f171c0ab600 con 0x7f173406c1d0 2024-09-19T08:45:46.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.403+0000 7f1731ffb640 1 -- 172.21.15.98:0/2185701214 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171c0ab9d0 con 0x7f173406c1d0 2024-09-19T08:45:46.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.403+0000 7f173973a640 1 -- 172.21.15.98:0/2185701214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f173406c1d0 msgr2=0x7f17340713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:46.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.403+0000 7f173973a640 1 --2- 172.21.15.98:0/2185701214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f173406c1d0 0x7f17340713b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f171c009870 tx=0x7f171c01c4d0 comp rx=0 tx=0).stop 2024-09-19T08:45:46.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.403+0000 7f173973a640 1 -- 172.21.15.98:0/2185701214 shutdown_connections 2024-09-19T08:45:46.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.403+0000 7f173973a640 1 --2- 172.21.15.98:0/2185701214 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f173406c1d0 0x7f17340713b0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:46.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.403+0000 7f173973a640 1 --2- 172.21.15.98:0/2185701214 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1734145b70 0x7f173406bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:46.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.403+0000 7f173973a640 1 -- 172.21.15.98:0/2185701214 >> 172.21.15.98:0/2185701214 conn(0x7f17340b2010 msgr2=0x7f17340b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:46.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.403+0000 7f173973a640 1 -- 172.21.15.98:0/2185701214 shutdown_connections 2024-09-19T08:45:46.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.403+0000 7f173973a640 1 -- 172.21.15.98:0/2185701214 wait complete. 2024-09-19T08:45:46.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.403+0000 7f173973a640 1 Processor -- start 2024-09-19T08:45:46.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.403+0000 7f173973a640 1 -- start start 2024-09-19T08:45:46.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.403+0000 7f173973a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f173406c1d0 0x7f173413d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:46.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.403+0000 7f173973a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f17340718f0 0x7f173413b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:46.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.403+0000 7f173973a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1734145b70 0x7f173413e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:46.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.403+0000 7f173973a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1734169070 con 0x7f1734145b70 2024-09-19T08:45:46.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.403+0000 7f173973a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1734071f20 con 0x7f17340718f0 2024-09-19T08:45:46.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.403+0000 7f173973a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f17341691f0 con 0x7f173406c1d0 2024-09-19T08:45:46.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.403+0000 7f17337fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1734145b70 0x7f173413e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:46.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.403+0000 7f17337fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1734145b70 0x7f173413e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58442/0 (socket says 172.21.15.98:58442) 2024-09-19T08:45:46.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.403+0000 7f17337fe640 1 -- 172.21.15.98:0/1631506260 learned_addr learned my addr 172.21.15.98:0/1631506260 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:46.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.403+0000 7f1732ffd640 1 --2- 172.21.15.98:0/1631506260 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f173406c1d0 0x7f173413d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:46.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.403+0000 7f17327fc640 1 --2- 172.21.15.98:0/1631506260 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f17340718f0 0x7f173413b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:46.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.403+0000 7f17337fe640 1 -- 172.21.15.98:0/1631506260 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f173406c1d0 msgr2=0x7f173413d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:46.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.403+0000 7f17337fe640 1 --2- 172.21.15.98:0/1631506260 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f173406c1d0 0x7f173413d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:46.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.403+0000 7f17337fe640 1 -- 172.21.15.98:0/1631506260 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f17340718f0 msgr2=0x7f173413b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:46.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.403+0000 7f17337fe640 1 --2- 172.21.15.98:0/1631506260 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f17340718f0 0x7f173413b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:46.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.403+0000 7f17337fe640 1 -- 172.21.15.98:0/1631506260 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f173413bde0 con 0x7f1734145b70 2024-09-19T08:45:46.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.403+0000 7f17327fc640 1 --2- 172.21.15.98:0/1631506260 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f17340718f0 0x7f173413b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:46.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.403+0000 7f17337fe640 1 --2- 172.21.15.98:0/1631506260 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1734145b70 0x7f173413e9e0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f172400c7a0 tx=0x7f172400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:46.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.403+0000 7f172b7fe640 1 -- 172.21.15.98:0/1631506260 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1724003b80 con 0x7f1734145b70 2024-09-19T08:45:46.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.403+0000 7f172b7fe640 1 -- 172.21.15.98:0/1631506260 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1724003d20 con 0x7f1734145b70 2024-09-19T08:45:46.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.403+0000 7f172b7fe640 1 -- 172.21.15.98:0/1631506260 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172400aa30 con 0x7f1734145b70 2024-09-19T08:45:46.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.403+0000 7f173973a640 1 -- 172.21.15.98:0/1631506260 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f173413ef20 con 0x7f1734145b70 2024-09-19T08:45:46.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.403+0000 7f173973a640 1 -- 172.21.15.98:0/1631506260 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f173413f4c0 con 0x7f1734145b70 2024-09-19T08:45:46.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.407+0000 7f173973a640 1 -- 172.21.15.98:0/1631506260 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17340713b0 con 0x7f1734145b70 2024-09-19T08:45:46.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.407+0000 7f172b7fe640 1 -- 172.21.15.98:0/1631506260 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7f172400abd0 con 0x7f1734145b70 2024-09-19T08:45:46.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.407+0000 7f172b7fe640 1 --2- 172.21.15.98:0/1631506260 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1714067e60 0x7f171406a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:46.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.407+0000 7f172b7fe640 1 -- 172.21.15.98:0/1631506260 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7f1724086540 con 0x7f1734145b70 2024-09-19T08:45:46.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.407+0000 7f1732ffd640 1 --2- 172.21.15.98:0/1631506260 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1714067e60 0x7f171406a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:46.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.407+0000 7f1732ffd640 1 --2- 172.21.15.98:0/1631506260 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1714067e60 0x7f171406a320 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1734144380 tx=0x7f172c009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:46.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.411+0000 7f172b7fe640 1 -- 172.21.15.98:0/1631506260 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f17240567b0 con 0x7f1734145b70 2024-09-19T08:45:46.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.599+0000 7f173973a640 1 -- 172.21.15.98:0/1631506260 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup getpath", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7f1734142050 con 0x7f1714067e60 2024-09-19T08:45:46.601 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:46.602+0000 7ff73a69c640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-19T08:45:46.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.603+0000 7f172b7fe640 1 -- 172.21.15.98:0/1631506260 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f1734142050 con 0x7f1714067e60 2024-09-19T08:45:46.602 INFO:teuthology.orchestra.run.smithi098.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-19T08:45:46.607 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-19T08:45:46.608 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup pin non_existent_subvolume group_1 export 0 2024-09-19T08:45:46.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.731+0000 7fbad98cd640 1 Processor -- start 2024-09-19T08:45:46.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.731+0000 7fbad98cd640 1 -- start start 2024-09-19T08:45:46.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.735+0000 7fbad98cd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbad415a930 0x7fbad40b3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:46.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.735+0000 7fbad98cd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbad40b63d0 0x7fbad40cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:46.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.735+0000 7fbad98cd640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbad40b5110 con 0x7fbad40b3b60 2024-09-19T08:45:46.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.735+0000 7fbad98cd640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fbad40b4070 con 0x7fbad415a930 2024-09-19T08:45:46.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.735+0000 7fbad98cd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbad40b4350 con 0x7fbad40b63d0 2024-09-19T08:45:46.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.735+0000 7fbad2ffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbad415a930 0x7fbad40b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:46.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.735+0000 7fbad2ffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbad415a930 0x7fbad40b3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58446/0 (socket says 172.21.15.98:58446) 2024-09-19T08:45:46.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.735+0000 7fbad2ffd640 1 -- 172.21.15.98:0/331091765 learned_addr learned my addr 172.21.15.98:0/331091765 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:46.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.735+0000 7fbad37fe640 1 --2- 172.21.15.98:0/331091765 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbad40b63d0 0x7fbad40cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:46.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.735+0000 7fbad2ffd640 1 -- 172.21.15.98:0/331091765 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbad40b63d0 msgr2=0x7fbad40cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:46.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.735+0000 7fbad2ffd640 1 --2- 172.21.15.98:0/331091765 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbad40b63d0 0x7fbad40cdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:46.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.735+0000 7fbad2ffd640 1 -- 172.21.15.98:0/331091765 >> v1:172.21.15.139:6789/0 conn(0x7fbad40b3b60 legacy=0x7fbad40b5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:46.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.735+0000 7fbad2ffd640 1 -- 172.21.15.98:0/331091765 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad40b4630 con 0x7fbad415a930 2024-09-19T08:45:46.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.735+0000 7fbad2ffd640 1 --2- 172.21.15.98:0/331091765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbad415a930 0x7fbad40b3620 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fbac4009e50 tx=0x7fbac401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a55e3d7706d7c3c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:46.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.735+0000 7fbad1ffb640 1 -- 172.21.15.98:0/331091765 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac40a56f0 con 0x7fbad415a930 2024-09-19T08:45:46.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.735+0000 7fbad1ffb640 1 -- 172.21.15.98:0/331091765 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbac40a5890 con 0x7fbad415a930 2024-09-19T08:45:46.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.735+0000 7fbad1ffb640 1 -- 172.21.15.98:0/331091765 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac40a5c60 con 0x7fbad415a930 2024-09-19T08:45:46.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.735+0000 7fbad98cd640 1 -- 172.21.15.98:0/331091765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbad415a930 msgr2=0x7fbad40b3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:46.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.735+0000 7fbad98cd640 1 --2- 172.21.15.98:0/331091765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbad415a930 0x7fbad40b3620 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fbac4009e50 tx=0x7fbac401c960 comp rx=0 tx=0).stop 2024-09-19T08:45:46.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.735+0000 7fbad98cd640 1 -- 172.21.15.98:0/331091765 shutdown_connections 2024-09-19T08:45:46.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.735+0000 7fbad98cd640 1 --2- 172.21.15.98:0/331091765 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbad40b63d0 0x7fbad40cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:46.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.735+0000 7fbad98cd640 1 --2- 172.21.15.98:0/331091765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbad415a930 0x7fbad40b3620 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:46.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.735+0000 7fbad98cd640 1 -- 172.21.15.98:0/331091765 >> 172.21.15.98:0/331091765 conn(0x7fbad406c090 msgr2=0x7fbad406c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:46.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.735+0000 7fbad98cd640 1 -- 172.21.15.98:0/331091765 shutdown_connections 2024-09-19T08:45:46.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.735+0000 7fbad98cd640 1 -- 172.21.15.98:0/331091765 wait complete. 2024-09-19T08:45:46.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.739+0000 7fbad98cd640 1 Processor -- start 2024-09-19T08:45:46.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.739+0000 7fbad98cd640 1 -- start start 2024-09-19T08:45:46.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.739+0000 7fbad98cd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbad40b3b60 0x7fbad414aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:46.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.739+0000 7fbad98cd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbad40b63d0 0x7fbad415c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:46.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.739+0000 7fbad98cd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbad415a930 0x7fbad40729a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:46.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.739+0000 7fbad98cd640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbad40cea50 con 0x7fbad40b63d0 2024-09-19T08:45:46.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.739+0000 7fbad98cd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbad40ce8d0 con 0x7fbad40b3b60 2024-09-19T08:45:46.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.739+0000 7fbad98cd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbad40cebd0 con 0x7fbad415a930 2024-09-19T08:45:46.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.739+0000 7fbad27fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbad40b63d0 0x7fbad415c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:46.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.739+0000 7fbad27fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbad40b63d0 0x7fbad415c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58460/0 (socket says 172.21.15.98:58460) 2024-09-19T08:45:46.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.739+0000 7fbad27fc640 1 -- 172.21.15.98:0/1377174328 learned_addr learned my addr 172.21.15.98:0/1377174328 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:46.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.739+0000 7fbad2ffd640 1 --2- 172.21.15.98:0/1377174328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbad40b3b60 0x7fbad414aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:46.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.739+0000 7fbad27fc640 1 -- 172.21.15.98:0/1377174328 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbad415a930 msgr2=0x7fbad40729a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:46.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.739+0000 7fbad27fc640 1 --2- 172.21.15.98:0/1377174328 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbad415a930 0x7fbad40729a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:46.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.739+0000 7fbad27fc640 1 -- 172.21.15.98:0/1377174328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbad40b3b60 msgr2=0x7fbad414aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:46.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.739+0000 7fbad27fc640 1 --2- 172.21.15.98:0/1377174328 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbad40b3b60 0x7fbad414aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:46.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.739+0000 7fbad27fc640 1 -- 172.21.15.98:0/1377174328 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad4150310 con 0x7fbad40b63d0 2024-09-19T08:45:46.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.739+0000 7fbad27fc640 1 --2- 172.21.15.98:0/1377174328 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbad40b63d0 0x7fbad415c050 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fbabc00c450 tx=0x7fbabc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:46.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.739+0000 7fbacbfff640 1 -- 172.21.15.98:0/1377174328 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc012830 con 0x7fbad40b63d0 2024-09-19T08:45:46.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.739+0000 7fbacbfff640 1 -- 172.21.15.98:0/1377174328 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbabc017070 con 0x7fbad40b63d0 2024-09-19T08:45:46.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.739+0000 7fbad98cd640 1 -- 172.21.15.98:0/1377174328 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad41505f0 con 0x7fbad40b63d0 2024-09-19T08:45:46.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.739+0000 7fbacbfff640 1 -- 172.21.15.98:0/1377174328 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc012c30 con 0x7fbad40b63d0 2024-09-19T08:45:46.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.739+0000 7fbad98cd640 1 -- 172.21.15.98:0/1377174328 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad4150bb0 con 0x7fbad40b63d0 2024-09-19T08:45:46.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.739+0000 7fbad98cd640 1 -- 172.21.15.98:0/1377174328 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbad40cdb80 con 0x7fbad40b63d0 2024-09-19T08:45:46.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.739+0000 7fbacbfff640 1 -- 172.21.15.98:0/1377174328 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 21) ==== 87955+0+0 (secure 0 0 0) 0x7fbabc01c460 con 0x7fbad40b63d0 2024-09-19T08:45:46.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.743+0000 7fbacbfff640 1 --2- 172.21.15.98:0/1377174328 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbab4067e90 0x7fbab406a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:46.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.743+0000 7fbacbfff640 1 -- 172.21.15.98:0/1377174328 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7fbabc086490 con 0x7fbad40b63d0 2024-09-19T08:45:46.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.743+0000 7fbad2ffd640 1 --2- 172.21.15.98:0/1377174328 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbab4067e90 0x7fbab406a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:46.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.743+0000 7fbad2ffd640 1 --2- 172.21.15.98:0/1377174328 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbab4067e90 0x7fbab406a350 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbac40ae000 tx=0x7fbac40a7070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:46.744 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.747+0000 7fbacbfff640 1 -- 172.21.15.98:0/1377174328 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fbabc0567d0 con 0x7fbad40b63d0 2024-09-19T08:45:46.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.919+0000 7fbad98cd640 1 -- 172.21.15.98:0/1377174328 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "non_existent_subvolume", "group_name": "group_1", "pin_type": "export", "pin_setting": "0", "target": ["mon-mgr", ""]}) -- 0x7fbad414afe0 con 0x7fbab4067e90 2024-09-19T08:45:46.919 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:46.922+0000 7ff73a69c640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-19T08:45:46.920 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:46.919+0000 7fbacbfff640 1 -- 172.21.15.98:0/1377174328 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fbad414afe0 con 0x7fbab4067e90 2024-09-19T08:45:46.920 INFO:teuthology.orchestra.run.smithi098.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-19T08:45:46.925 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-19T08:45:46.925 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup ls non_existent_subvolume 2024-09-19T08:45:47.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.039+0000 7fbc57b96640 1 Processor -- start 2024-09-19T08:45:47.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.039+0000 7fbc57b96640 1 -- start start 2024-09-19T08:45:47.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.039+0000 7fbc57b96640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbc501455a0 0x7fbc50072370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:47.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.039+0000 7fbc57b96640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbc500728b0 0x7fbc5015fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:47.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.039+0000 7fbc57b96640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc500d0500 con 0x7fbc50142540 2024-09-19T08:45:47.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.039+0000 7fbc57b96640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fbc500cecf0 con 0x7fbc501455a0 2024-09-19T08:45:47.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.039+0000 7fbc57b96640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fbc500d02b0 con 0x7fbc500728b0 2024-09-19T08:45:47.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.039+0000 7fbc55131640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbc501455a0 0x7fbc50072370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:47.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.039+0000 7fbc55131640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbc501455a0 0x7fbc50072370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58464/0 (socket says 172.21.15.98:58464) 2024-09-19T08:45:47.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.039+0000 7fbc55932640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fbc50142540 0x7fbc500ce5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:53138/0 (socket says 172.21.15.98:53138) 2024-09-19T08:45:47.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.039+0000 7fbc55131640 1 -- 172.21.15.98:0/3336350231 learned_addr learned my addr 172.21.15.98:0/3336350231 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:47.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.039+0000 7fbc56133640 1 --2- 172.21.15.98:0/3336350231 >> v2:172.21.15.139:3300/0 conn(0x7fbc500728b0 0x7fbc5015fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:47.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.039+0000 7fbc55131640 1 -- 172.21.15.98:0/3336350231 >> v2:172.21.15.139:3300/0 conn(0x7fbc500728b0 msgr2=0x7fbc5015fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:47.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.039+0000 7fbc55131640 1 --2- 172.21.15.98:0/3336350231 >> v2:172.21.15.139:3300/0 conn(0x7fbc500728b0 0x7fbc5015fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.039+0000 7fbc55131640 1 -- 172.21.15.98:0/3336350231 >> v1:172.21.15.98:6789/0 conn(0x7fbc50142540 legacy=0x7fbc500ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:47.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.039+0000 7fbc55131640 1 -- 172.21.15.98:0/3336350231 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc500732d0 con 0x7fbc501455a0 2024-09-19T08:45:47.039 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.039+0000 7fbc54930640 1 -- 172.21.15.98:0/3336350231 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2746191579 0 0) 0x7fbc500d0500 con 0x7fbc50142540 2024-09-19T08:45:47.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.043+0000 7fbc55131640 1 --2- 172.21.15.98:0/3336350231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc501455a0 0x7fbc50072370 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fbc500cd2d0 tx=0x7fbc4001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d32432720e4a5405 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:47.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.043+0000 7fbc54930640 1 -- 172.21.15.98:0/3336350231 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc400b0070 con 0x7fbc501455a0 2024-09-19T08:45:47.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.043+0000 7fbc54930640 1 -- 172.21.15.98:0/3336350231 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc400ab600 con 0x7fbc501455a0 2024-09-19T08:45:47.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.043+0000 7fbc54930640 1 -- 172.21.15.98:0/3336350231 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc400ab9d0 con 0x7fbc501455a0 2024-09-19T08:45:47.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.043+0000 7fbc57b96640 1 -- 172.21.15.98:0/3336350231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc501455a0 msgr2=0x7fbc50072370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:47.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.043+0000 7fbc57b96640 1 --2- 172.21.15.98:0/3336350231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc501455a0 0x7fbc50072370 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fbc500cd2d0 tx=0x7fbc4001c520 comp rx=0 tx=0).stop 2024-09-19T08:45:47.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.043+0000 7fbc57b96640 1 -- 172.21.15.98:0/3336350231 shutdown_connections 2024-09-19T08:45:47.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.043+0000 7fbc57b96640 1 --2- 172.21.15.98:0/3336350231 >> v2:172.21.15.139:3300/0 conn(0x7fbc500728b0 0x7fbc5015fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.043+0000 7fbc57b96640 1 --2- 172.21.15.98:0/3336350231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc501455a0 0x7fbc50072370 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.043+0000 7fbc57b96640 1 -- 172.21.15.98:0/3336350231 >> 172.21.15.98:0/3336350231 conn(0x7fbc500d0a10 msgr2=0x7fbc500d0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:47.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.043+0000 7fbc57b96640 1 -- 172.21.15.98:0/3336350231 shutdown_connections 2024-09-19T08:45:47.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.043+0000 7fbc57b96640 1 -- 172.21.15.98:0/3336350231 wait complete. 2024-09-19T08:45:47.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.043+0000 7fbc57b96640 1 Processor -- start 2024-09-19T08:45:47.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.043+0000 7fbc57b96640 1 -- start start 2024-09-19T08:45:47.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.043+0000 7fbc57b96640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc500728b0 0x7fbc500b5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:47.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.043+0000 7fbc57b96640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc50142540 0x7fbc5006f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:47.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.043+0000 7fbc57b96640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc501455a0 0x7fbc5013f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:47.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.043+0000 7fbc57b96640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbc50169160 con 0x7fbc50142540 2024-09-19T08:45:47.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.043+0000 7fbc57b96640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbc50168fe0 con 0x7fbc500728b0 2024-09-19T08:45:47.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.043+0000 7fbc57b96640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbc501692e0 con 0x7fbc501455a0 2024-09-19T08:45:47.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.043+0000 7fbc55131640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc50142540 0x7fbc5006f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:47.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.043+0000 7fbc55131640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc50142540 0x7fbc5006f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58466/0 (socket says 172.21.15.98:58466) 2024-09-19T08:45:47.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.043+0000 7fbc55131640 1 -- 172.21.15.98:0/23667235 learned_addr learned my addr 172.21.15.98:0/23667235 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:47.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.043+0000 7fbc55932640 1 --2- 172.21.15.98:0/23667235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc500728b0 0x7fbc500b5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:47.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.043+0000 7fbc55131640 1 -- 172.21.15.98:0/23667235 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc501455a0 msgr2=0x7fbc5013f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:47.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.043+0000 7fbc55131640 1 --2- 172.21.15.98:0/23667235 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc501455a0 0x7fbc5013f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.043+0000 7fbc55131640 1 -- 172.21.15.98:0/23667235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc500728b0 msgr2=0x7fbc500b5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:47.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.043+0000 7fbc55131640 1 --2- 172.21.15.98:0/23667235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc500728b0 0x7fbc500b5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.043+0000 7fbc55131640 1 -- 172.21.15.98:0/23667235 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc500b4be0 con 0x7fbc50142540 2024-09-19T08:45:47.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.043+0000 7fbc55932640 1 --2- 172.21.15.98:0/23667235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc500728b0 0x7fbc500b5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:47.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.043+0000 7fbc55131640 1 --2- 172.21.15.98:0/23667235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc50142540 0x7fbc5006f5d0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fbc400ae040 tx=0x7fbc4001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:47.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.043+0000 7fbc467fc640 1 -- 172.21.15.98:0/23667235 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc400bc070 con 0x7fbc50142540 2024-09-19T08:45:47.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.043+0000 7fbc467fc640 1 -- 172.21.15.98:0/23667235 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc40004030 con 0x7fbc50142540 2024-09-19T08:45:47.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.047+0000 7fbc57b96640 1 -- 172.21.15.98:0/23667235 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc5015f310 con 0x7fbc50142540 2024-09-19T08:45:47.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.047+0000 7fbc467fc640 1 -- 172.21.15.98:0/23667235 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc400b0040 con 0x7fbc50142540 2024-09-19T08:45:47.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.047+0000 7fbc57b96640 1 -- 172.21.15.98:0/23667235 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc5006fb10 con 0x7fbc50142540 2024-09-19T08:45:47.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.047+0000 7fbc57b96640 1 -- 172.21.15.98:0/23667235 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc5015fe50 con 0x7fbc50142540 2024-09-19T08:45:47.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.047+0000 7fbc467fc640 1 -- 172.21.15.98:0/23667235 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fbc40003620 con 0x7fbc50142540 2024-09-19T08:45:47.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.047+0000 7fbc467fc640 1 --2- 172.21.15.98:0/23667235 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbc34067f20 0x7fbc3406a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:47.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.047+0000 7fbc467fc640 1 -- 172.21.15.98:0/23667235 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7fbc4011ffe0 con 0x7fbc50142540 2024-09-19T08:45:47.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.047+0000 7fbc55932640 1 --2- 172.21.15.98:0/23667235 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbc34067f20 0x7fbc3406a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:47.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.047+0000 7fbc55932640 1 --2- 172.21.15.98:0/23667235 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbc34067f20 0x7fbc3406a3e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbc48007fd0 tx=0x7fbc48009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:47.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.051+0000 7fbc467fc640 1 -- 172.21.15.98:0/23667235 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fbc40016110 con 0x7fbc50142540 2024-09-19T08:45:47.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.239+0000 7fbc57b96640 1 -- 172.21.15.98:0/23667235 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup ls", "vol_name": "non_existent_subvolume", "target": ["mon-mgr", ""]}) -- 0x7fbc500b5bc0 con 0x7fbc34067f20 2024-09-19T08:45:47.242 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:47.242+0000 7ff73a69c640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-19T08:45:47.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.243+0000 7fbc467fc640 1 -- 172.21.15.98:0/23667235 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fbc48001cc0 con 0x7fbc34067f20 2024-09-19T08:45:47.243 INFO:teuthology.orchestra.run.smithi098.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-19T08:45:47.249 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-19T08:45:47.250 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot create non_existent_subvolume group_1 snapshot_1 2024-09-19T08:45:47.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.375+0000 7f8abfb21640 1 Processor -- start 2024-09-19T08:45:47.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.375+0000 7f8abfb21640 1 -- start start 2024-09-19T08:45:47.375 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.375+0000 7f8abfb21640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ab815c060 0x7f8ab80afb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:47.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.375+0000 7f8abfb21640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ab80729f0 con 0x7f8ab80cd7f0 2024-09-19T08:45:47.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.375+0000 7f8abfb21640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ab8072cd0 con 0x7f8ab815c440 2024-09-19T08:45:47.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.375+0000 7f8abfb21640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8ab8068700 con 0x7f8ab815c060 2024-09-19T08:45:47.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.375+0000 7f8abe0be640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f8ab80cd7f0 0x7f8ab806d600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:53150/0 (socket says 172.21.15.98:53150) 2024-09-19T08:45:47.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.375+0000 7f8abe0be640 1 -- 172.21.15.98:0/2390340397 learned_addr learned my addr 172.21.15.98:0/2390340397 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:47.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.375+0000 7f8abc8bb640 1 -- 172.21.15.98:0/2390340397 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4117724433 0 0) 0x7f8ab80729f0 con 0x7f8ab80cd7f0 2024-09-19T08:45:47.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.375+0000 7f8abc8bb640 1 -- 172.21.15.98:0/2390340397 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8aa0003530 con 0x7f8ab80cd7f0 2024-09-19T08:45:47.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.375+0000 7f8abd8bd640 1 --2- 172.21.15.98:0/2390340397 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ab815c060 0x7f8ab80afb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:47.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.375+0000 7f8abc8bb640 1 -- 172.21.15.98:0/2390340397 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1299326119 0 0) 0x7f8aa0003530 con 0x7f8ab80cd7f0 2024-09-19T08:45:47.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.375+0000 7f8abc8bb640 1 -- 172.21.15.98:0/2390340397 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ab815c060 msgr2=0x7f8ab80afb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:47.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.375+0000 7f8abc8bb640 1 --2- 172.21.15.98:0/2390340397 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ab815c060 0x7f8ab80afb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.375+0000 7f8abc8bb640 1 -- 172.21.15.98:0/2390340397 >> v1:172.21.15.139:6789/0 conn(0x7f8ab815c440 legacy=0x7f8ab80cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:47.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.375+0000 7f8abc8bb640 1 -- 172.21.15.98:0/2390340397 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab8072fb0 con 0x7f8ab80cd7f0 2024-09-19T08:45:47.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.375+0000 7f8abc8bb640 1 -- 172.21.15.98:0/2390340397 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f8ab0001680 con 0x7f8ab80cd7f0 2024-09-19T08:45:47.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.375+0000 7f8abd8bd640 1 --2- 172.21.15.98:0/2390340397 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ab815c060 0x7f8ab80afb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:47.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.375+0000 7f8abc8bb640 1 -- 172.21.15.98:0/2390340397 >> v1:172.21.15.98:6789/0 conn(0x7f8ab80cd7f0 legacy=0x7f8ab806d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:47.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.375+0000 7f8abc8bb640 1 --2- 172.21.15.98:0/2390340397 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8aa00039f0 0x7f8aa0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:47.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.375+0000 7f8abc8bb640 1 --2- 172.21.15.98:0/2390340397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8aa0004810 0x7f8aa0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:47.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.375+0000 7f8abc8bb640 1 --2- 172.21.15.98:0/2390340397 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ab815c060 0x7f8aa00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:47.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.375+0000 7f8abc8bb640 1 -- 172.21.15.98:0/2390340397 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa000baa0 con 0x7f8aa0004810 2024-09-19T08:45:47.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.375+0000 7f8abc8bb640 1 -- 172.21.15.98:0/2390340397 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa000bc50 con 0x7f8aa00039f0 2024-09-19T08:45:47.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.375+0000 7f8abc8bb640 1 -- 172.21.15.98:0/2390340397 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8aa000be50 con 0x7f8ab815c060 2024-09-19T08:45:47.377 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.375+0000 7f8abd0bc640 1 --2- 172.21.15.98:0/2390340397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8aa0004810 0x7f8aa0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:47.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.375+0000 7f8abe0be640 1 --2- 172.21.15.98:0/2390340397 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ab815c060 0x7f8aa00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:47.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.375+0000 7f8abd8bd640 1 --2- 172.21.15.98:0/2390340397 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8aa00039f0 0x7f8aa0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:47.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.375+0000 7f8abd0bc640 1 -- 172.21.15.98:0/2390340397 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ab815c060 msgr2=0x7f8aa00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:47.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.375+0000 7f8abd0bc640 1 --2- 172.21.15.98:0/2390340397 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ab815c060 0x7f8aa00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.375+0000 7f8abd0bc640 1 -- 172.21.15.98:0/2390340397 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8aa00039f0 msgr2=0x7f8aa0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:47.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.375+0000 7f8abd0bc640 1 --2- 172.21.15.98:0/2390340397 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8aa00039f0 0x7f8aa0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.378 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.375+0000 7f8abd0bc640 1 -- 172.21.15.98:0/2390340397 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa0003530 con 0x7f8aa0004810 2024-09-19T08:45:47.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.379+0000 7f8abd0bc640 1 --2- 172.21.15.98:0/2390340397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8aa0004810 0x7f8aa0004c90 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f8aac001780 tx=0x7f8aac01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d752c8b21aa486cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:47.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.379+0000 7f8abc8bb640 1 -- 172.21.15.98:0/2390340397 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aac0013a0 con 0x7f8aa0004810 2024-09-19T08:45:47.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.379+0000 7f8abc8bb640 1 -- 172.21.15.98:0/2390340397 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8aac0052f0 con 0x7f8aa0004810 2024-09-19T08:45:47.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.379+0000 7f8abc8bb640 1 -- 172.21.15.98:0/2390340397 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aac003840 con 0x7f8aa0004810 2024-09-19T08:45:47.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.379+0000 7f8abfb21640 1 -- 172.21.15.98:0/2390340397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8aa0004810 msgr2=0x7f8aa0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:47.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.379+0000 7f8abfb21640 1 --2- 172.21.15.98:0/2390340397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8aa0004810 0x7f8aa0004c90 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f8aac001780 tx=0x7f8aac01c420 comp rx=0 tx=0).stop 2024-09-19T08:45:47.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.379+0000 7f8abe0be640 1 -- 172.21.15.98:0/2390340397 reap_dead start 2024-09-19T08:45:47.381 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.379+0000 7f8abfb21640 1 -- 172.21.15.98:0/2390340397 shutdown_connections 2024-09-19T08:45:47.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.379+0000 7f8abfb21640 1 -- 172.21.15.98:0/2390340397 >> 172.21.15.98:0/2390340397 conn(0x7f8ab80b25c0 msgr2=0x7f8ab80b29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:47.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.379+0000 7f8abfb21640 1 -- 172.21.15.98:0/2390340397 shutdown_connections 2024-09-19T08:45:47.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.379+0000 7f8abfb21640 1 -- 172.21.15.98:0/2390340397 wait complete. 2024-09-19T08:45:47.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.379+0000 7f8abfb21640 1 Processor -- start 2024-09-19T08:45:47.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.379+0000 7f8abfb21640 1 -- start start 2024-09-19T08:45:47.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.383+0000 7f8abfb21640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8aa0004810 0x7f8ab81f3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:47.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.383+0000 7f8abfb21640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ab81598a0 0x7f8ab8159d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:47.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.383+0000 7f8abfb21640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ab815a2c0 0x7f8ab815a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:47.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.383+0000 7f8abfb21640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab8072630 con 0x7f8ab815a2c0 2024-09-19T08:45:47.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.383+0000 7f8abfb21640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab80735b0 con 0x7f8aa0004810 2024-09-19T08:45:47.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.383+0000 7f8abfb21640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8ab80727b0 con 0x7f8ab81598a0 2024-09-19T08:45:47.382 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.383+0000 7f8abe0be640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ab815a2c0 0x7f8ab815a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:47.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.383+0000 7f8abe0be640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ab815a2c0 0x7f8ab815a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:58486/0 (socket says 172.21.15.98:58486) 2024-09-19T08:45:47.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.383+0000 7f8abe0be640 1 -- 172.21.15.98:0/3359035243 learned_addr learned my addr 172.21.15.98:0/3359035243 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:47.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.383+0000 7f8abd8bd640 1 --2- 172.21.15.98:0/3359035243 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8aa0004810 0x7f8ab81f3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:47.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.383+0000 7f8abd0bc640 1 --2- 172.21.15.98:0/3359035243 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ab81598a0 0x7f8ab8159d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:47.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.383+0000 7f8abe0be640 1 -- 172.21.15.98:0/3359035243 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ab81598a0 msgr2=0x7f8ab8159d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:47.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.383+0000 7f8abe0be640 1 --2- 172.21.15.98:0/3359035243 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8ab81598a0 0x7f8ab8159d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.383+0000 7f8abe0be640 1 -- 172.21.15.98:0/3359035243 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8aa0004810 msgr2=0x7f8ab81f3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:47.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.383+0000 7f8abe0be640 1 --2- 172.21.15.98:0/3359035243 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8aa0004810 0x7f8ab81f3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.383+0000 7f8abe0be640 1 -- 172.21.15.98:0/3359035243 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab8152200 con 0x7f8ab815a2c0 2024-09-19T08:45:47.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.383+0000 7f8abe0be640 1 --2- 172.21.15.98:0/3359035243 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8ab815a2c0 0x7f8ab815a740 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f8ab8073e10 tx=0x7f8ab00126f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:47.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.383+0000 7f8ab67fc640 1 -- 172.21.15.98:0/3359035243 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab001c070 con 0x7f8ab815a2c0 2024-09-19T08:45:47.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.383+0000 7f8ab67fc640 1 -- 172.21.15.98:0/3359035243 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ab00155e0 con 0x7f8ab815a2c0 2024-09-19T08:45:47.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.383+0000 7f8abfb21640 1 -- 172.21.15.98:0/3359035243 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ab8151960 con 0x7f8ab815a2c0 2024-09-19T08:45:47.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.383+0000 7f8ab67fc640 1 -- 172.21.15.98:0/3359035243 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab00159d0 con 0x7f8ab815a2c0 2024-09-19T08:45:47.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.383+0000 7f8abfb21640 1 -- 172.21.15.98:0/3359035243 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ab8151f20 con 0x7f8ab815a2c0 2024-09-19T08:45:47.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.383+0000 7f8abfb21640 1 -- 172.21.15.98:0/3359035243 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ab80afd60 con 0x7f8ab815a2c0 2024-09-19T08:45:47.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.383+0000 7f8ab67fc640 1 -- 172.21.15.98:0/3359035243 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f8ab001f070 con 0x7f8ab815a2c0 2024-09-19T08:45:47.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.383+0000 7f8ab67fc640 1 --2- 172.21.15.98:0/3359035243 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8aa4067f20 0x7f8aa406a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:47.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.383+0000 7f8ab67fc640 1 -- 172.21.15.98:0/3359035243 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7f8ab0088cb0 con 0x7f8ab815a2c0 2024-09-19T08:45:47.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.383+0000 7f8abd8bd640 1 --2- 172.21.15.98:0/3359035243 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8aa4067f20 0x7f8aa406a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:47.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.387+0000 7f8abd8bd640 1 --2- 172.21.15.98:0/3359035243 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8aa4067f20 0x7f8aa406a3e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8009900 tx=0x7f8aa800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:47.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.387+0000 7f8ab67fc640 1 -- 172.21.15.98:0/3359035243 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f8ab0058e90 con 0x7f8ab815a2c0 2024-09-19T08:45:47.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.563+0000 7f8abfb21640 1 -- 172.21.15.98:0/3359035243 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot create", "vol_name": "non_existent_subvolume", "group_name": "group_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f8ab80cf2c0 con 0x7f8aa4067f20 2024-09-19T08:45:47.565 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:47.566+0000 7ff73a69c640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-19T08:45:47.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.567+0000 7f8ab67fc640 1 -- 172.21.15.98:0/3359035243 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f8ab80cf2c0 con 0x7f8aa4067f20 2024-09-19T08:45:47.566 INFO:teuthology.orchestra.run.smithi098.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-19T08:45:47.571 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-19T08:45:47.572 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot rm non_existent_subvolume group_1 snapshot_1 2024-09-19T08:45:47.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.687+0000 7fc85f5c4640 1 Processor -- start 2024-09-19T08:45:47.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.687+0000 7fc85f5c4640 1 -- start start 2024-09-19T08:45:47.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.687+0000 7fc85f5c4640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc85815c290 0x7fc8580cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:47.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.687+0000 7fc85f5c4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8580cf700 0x7fc85815ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:47.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.687+0000 7fc85f5c4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc85806d4c0 0x7fc85806d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:47.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.687+0000 7fc85f5c4640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fc85806de70 con 0x7fc85815c290 2024-09-19T08:45:47.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.687+0000 7fc85f5c4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fc858160e80 con 0x7fc85806d4c0 2024-09-19T08:45:47.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.687+0000 7fc85f5c4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc8580cd510 con 0x7fc8580cf700 2024-09-19T08:45:47.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.687+0000 7fc85d360640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc85815c290 0x7fc8580cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:47.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.687+0000 7fc85d360640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc85815c290 0x7fc8580cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53758/0 (socket says 172.21.15.98:53758) 2024-09-19T08:45:47.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.687+0000 7fc85d360640 1 -- 172.21.15.98:0/1754630315 learned_addr learned my addr 172.21.15.98:0/1754630315 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:47.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.687+0000 7fc85db61640 1 --2- 172.21.15.98:0/1754630315 >> v2:172.21.15.139:3300/0 conn(0x7fc85806d4c0 0x7fc85806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:47.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.687+0000 7fc85cb5f640 1 --2- 172.21.15.98:0/1754630315 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8580cf700 0x7fc85815ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:47.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.687+0000 7fc85d360640 1 -- 172.21.15.98:0/1754630315 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8580cf700 msgr2=0x7fc85815ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:47.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.687+0000 7fc85d360640 1 --2- 172.21.15.98:0/1754630315 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8580cf700 0x7fc85815ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.687+0000 7fc85d360640 1 -- 172.21.15.98:0/1754630315 >> v2:172.21.15.139:3300/0 conn(0x7fc85806d4c0 msgr2=0x7fc85806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:47.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.687+0000 7fc85d360640 1 --2- 172.21.15.98:0/1754630315 >> v2:172.21.15.139:3300/0 conn(0x7fc85806d4c0 0x7fc85806d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.687+0000 7fc85d360640 1 -- 172.21.15.98:0/1754630315 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8581608c0 con 0x7fc85815c290 2024-09-19T08:45:47.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.687+0000 7fc85cb5f640 1 --2- 172.21.15.98:0/1754630315 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8580cf700 0x7fc85815ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:47.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.691+0000 7fc85d360640 1 --2- 172.21.15.98:0/1754630315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc85815c290 0x7fc8580cd9a0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fc8580b3b60 tx=0x7fc85401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c645e12112914d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:47.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.691+0000 7fc84ffff640 1 -- 172.21.15.98:0/1754630315 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8540b1070 con 0x7fc85815c290 2024-09-19T08:45:47.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.691+0000 7fc84ffff640 1 -- 172.21.15.98:0/1754630315 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8540ac630 con 0x7fc85815c290 2024-09-19T08:45:47.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.691+0000 7fc84ffff640 1 -- 172.21.15.98:0/1754630315 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8540a8050 con 0x7fc85815c290 2024-09-19T08:45:47.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.691+0000 7fc85f5c4640 1 -- 172.21.15.98:0/1754630315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc85815c290 msgr2=0x7fc8580cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:47.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.691+0000 7fc85f5c4640 1 --2- 172.21.15.98:0/1754630315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc85815c290 0x7fc8580cd9a0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fc8580b3b60 tx=0x7fc85401c520 comp rx=0 tx=0).stop 2024-09-19T08:45:47.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.691+0000 7fc85f5c4640 1 -- 172.21.15.98:0/1754630315 shutdown_connections 2024-09-19T08:45:47.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.691+0000 7fc85f5c4640 1 --2- 172.21.15.98:0/1754630315 >> v2:172.21.15.139:3300/0 conn(0x7fc85806d4c0 0x7fc85806d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.691+0000 7fc85f5c4640 1 --2- 172.21.15.98:0/1754630315 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc8580cf700 0x7fc85815ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.691+0000 7fc85f5c4640 1 --2- 172.21.15.98:0/1754630315 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc85815c290 0x7fc8580cd9a0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.691+0000 7fc85f5c4640 1 -- 172.21.15.98:0/1754630315 >> 172.21.15.98:0/1754630315 conn(0x7fc85806c090 msgr2=0x7fc85806c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:47.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.691+0000 7fc85f5c4640 1 -- 172.21.15.98:0/1754630315 shutdown_connections 2024-09-19T08:45:47.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.691+0000 7fc85f5c4640 1 -- 172.21.15.98:0/1754630315 wait complete. 2024-09-19T08:45:47.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.691+0000 7fc85f5c4640 1 Processor -- start 2024-09-19T08:45:47.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.691+0000 7fc85f5c4640 1 -- start start 2024-09-19T08:45:47.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.695+0000 7fc85f5c4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc85806d4c0 0x7fc858164540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:47.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.695+0000 7fc85f5c4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8580cf700 0x7fc8580b2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:47.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.695+0000 7fc85f5c4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc85815c290 0x7fc858159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:47.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.695+0000 7fc85f5c4640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc85815c7f0 con 0x7fc8580cf700 2024-09-19T08:45:47.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.695+0000 7fc85f5c4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc85815c670 con 0x7fc85806d4c0 2024-09-19T08:45:47.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.695+0000 7fc85f5c4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc858160490 con 0x7fc85815c290 2024-09-19T08:45:47.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.695+0000 7fc85cb5f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8580cf700 0x7fc8580b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:47.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.695+0000 7fc85cb5f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8580cf700 0x7fc8580b2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53766/0 (socket says 172.21.15.98:53766) 2024-09-19T08:45:47.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.695+0000 7fc85cb5f640 1 -- 172.21.15.98:0/4220053858 learned_addr learned my addr 172.21.15.98:0/4220053858 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:47.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.695+0000 7fc85d360640 1 --2- 172.21.15.98:0/4220053858 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc85806d4c0 0x7fc858164540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:47.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.695+0000 7fc85cb5f640 1 -- 172.21.15.98:0/4220053858 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc85815c290 msgr2=0x7fc858159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:47.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.695+0000 7fc85db61640 1 --2- 172.21.15.98:0/4220053858 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc85815c290 0x7fc858159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:47.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.695+0000 7fc85cb5f640 1 --2- 172.21.15.98:0/4220053858 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc85815c290 0x7fc858159b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.695+0000 7fc85cb5f640 1 -- 172.21.15.98:0/4220053858 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc85806d4c0 msgr2=0x7fc858164540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:47.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.695+0000 7fc85cb5f640 1 --2- 172.21.15.98:0/4220053858 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc85806d4c0 0x7fc858164540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:47.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.695+0000 7fc85cb5f640 1 -- 172.21.15.98:0/4220053858 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc858163a80 con 0x7fc8580cf700 2024-09-19T08:45:47.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.695+0000 7fc85cb5f640 1 --2- 172.21.15.98:0/4220053858 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc8580cf700 0x7fc8580b2550 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fc84800c7f0 tx=0x7fc84800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:47.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.695+0000 7fc84dffb640 1 -- 172.21.15.98:0/4220053858 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848015070 con 0x7fc8580cf700 2024-09-19T08:45:47.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.695+0000 7fc84dffb640 1 -- 172.21.15.98:0/4220053858 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc848007a30 con 0x7fc8580cf700 2024-09-19T08:45:47.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.695+0000 7fc85f5c4640 1 -- 172.21.15.98:0/4220053858 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8580af140 con 0x7fc8580cf700 2024-09-19T08:45:47.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.695+0000 7fc84dffb640 1 -- 172.21.15.98:0/4220053858 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc848007e00 con 0x7fc8580cf700 2024-09-19T08:45:47.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.695+0000 7fc85f5c4640 1 -- 172.21.15.98:0/4220053858 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc858163d40 con 0x7fc8580cf700 2024-09-19T08:45:47.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.695+0000 7fc85f5c4640 1 -- 172.21.15.98:0/4220053858 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8580b5400 con 0x7fc8580cf700 2024-09-19T08:45:47.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.695+0000 7fc84dffb640 1 -- 172.21.15.98:0/4220053858 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fc848018070 con 0x7fc8580cf700 2024-09-19T08:45:47.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.695+0000 7fc84dffb640 1 --2- 172.21.15.98:0/4220053858 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc83c067ef0 0x7fc83c06a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:47.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.695+0000 7fc84dffb640 1 -- 172.21.15.98:0/4220053858 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7fc848086850 con 0x7fc8580cf700 2024-09-19T08:45:47.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.695+0000 7fc85d360640 1 --2- 172.21.15.98:0/4220053858 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc83c067ef0 0x7fc83c06a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:47.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.699+0000 7fc85d360640 1 --2- 172.21.15.98:0/4220053858 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc83c067ef0 0x7fc83c06a3b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc8540ae000 tx=0x7fc854008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:47.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.699+0000 7fc84dffb640 1 -- 172.21.15.98:0/4220053858 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc8480569b0 con 0x7fc8580cf700 2024-09-19T08:45:47.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.871+0000 7fc85f5c4640 1 -- 172.21.15.98:0/4220053858 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot rm", "vol_name": "non_existent_subvolume", "group_name": "group_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7fc85815e140 con 0x7fc83c067ef0 2024-09-19T08:45:47.874 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:47.874+0000 7ff73a69c640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-19T08:45:47.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:47.875+0000 7fc84dffb640 1 -- 172.21.15.98:0/4220053858 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fc85815e140 con 0x7fc83c067ef0 2024-09-19T08:45:47.875 INFO:teuthology.orchestra.run.smithi098.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-19T08:45:47.881 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-19T08:45:47.882 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot ls non_existent_subvolume group_1 2024-09-19T08:45:48.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.007+0000 7fe9cfa89640 1 Processor -- start 2024-09-19T08:45:48.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.007+0000 7fe9cfa89640 1 -- start start 2024-09-19T08:45:48.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.007+0000 7fe9cfa89640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe9c8162a80 0x7fe9c806d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:48.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.007+0000 7fe9cfa89640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe9c80b4260 0x7fe9c806da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:48.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.007+0000 7fe9cfa89640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9c80b37a0 con 0x7fe9c81626a0 2024-09-19T08:45:48.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.007+0000 7fe9cfa89640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe9c80d0170 con 0x7fe9c8162a80 2024-09-19T08:45:48.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.007+0000 7fe9cfa89640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe9c80b3a60 con 0x7fe9c80b4260 2024-09-19T08:45:48.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.007+0000 7fe9cd024640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe9c8162a80 0x7fe9c806d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:48.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.007+0000 7fe9ce026640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe9c80b4260 0x7fe9c806da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:48.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.007+0000 7fe9cd024640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe9c8162a80 0x7fe9c806d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:44602/0 (socket says 172.21.15.98:44602) 2024-09-19T08:45:48.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.007+0000 7fe9cd024640 1 -- 172.21.15.98:0/1225450346 learned_addr learned my addr 172.21.15.98:0/1225450346 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:48.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.007+0000 7fe9cd024640 1 -- 172.21.15.98:0/1225450346 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe9c80b4260 msgr2=0x7fe9c806da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:48.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.007+0000 7fe9cd024640 1 --2- 172.21.15.98:0/1225450346 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe9c80b4260 0x7fe9c806da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:48.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.007+0000 7fe9cd024640 1 -- 172.21.15.98:0/1225450346 >> v1:172.21.15.139:6789/0 conn(0x7fe9c81626a0 legacy=0x7fe9c8163990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:48.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.007+0000 7fe9cd024640 1 -- 172.21.15.98:0/1225450346 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c80b56b0 con 0x7fe9c8162a80 2024-09-19T08:45:48.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.011+0000 7fe9cd024640 1 --2- 172.21.15.98:0/1225450346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9c8162a80 0x7fe9c806d4c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009870 tx=0x7fe9b801c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa9fd01899b16cca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:48.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.011+0000 7fe9cc823640 1 -- 172.21.15.98:0/1225450346 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b80b0070 con 0x7fe9c8162a80 2024-09-19T08:45:48.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.011+0000 7fe9cc823640 1 -- 172.21.15.98:0/1225450346 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9b8004230 con 0x7fe9c8162a80 2024-09-19T08:45:48.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.011+0000 7fe9cc823640 1 -- 172.21.15.98:0/1225450346 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8003620 con 0x7fe9c8162a80 2024-09-19T08:45:48.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.011+0000 7fe9cfa89640 1 -- 172.21.15.98:0/1225450346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9c8162a80 msgr2=0x7fe9c806d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:48.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.011+0000 7fe9cfa89640 1 --2- 172.21.15.98:0/1225450346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9c8162a80 0x7fe9c806d4c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009870 tx=0x7fe9b801c440 comp rx=0 tx=0).stop 2024-09-19T08:45:48.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.011+0000 7fe9cfa89640 1 -- 172.21.15.98:0/1225450346 shutdown_connections 2024-09-19T08:45:48.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.011+0000 7fe9cfa89640 1 --2- 172.21.15.98:0/1225450346 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe9c80b4260 0x7fe9c806da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:48.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.011+0000 7fe9cfa89640 1 --2- 172.21.15.98:0/1225450346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9c8162a80 0x7fe9c806d4c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:48.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.011+0000 7fe9cfa89640 1 -- 172.21.15.98:0/1225450346 >> 172.21.15.98:0/1225450346 conn(0x7fe9c806c090 msgr2=0x7fe9c806c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:48.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.011+0000 7fe9cfa89640 1 -- 172.21.15.98:0/1225450346 shutdown_connections 2024-09-19T08:45:48.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.011+0000 7fe9cfa89640 1 -- 172.21.15.98:0/1225450346 wait complete. 2024-09-19T08:45:48.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.015+0000 7fe9cfa89640 1 Processor -- start 2024-09-19T08:45:48.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.015+0000 7fe9cfa89640 1 -- start start 2024-09-19T08:45:48.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.015+0000 7fe9cfa89640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9c80b4260 0x7fe9c81579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:48.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.015+0000 7fe9cfa89640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9c81626a0 0x7fe9c8166390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:48.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.015+0000 7fe9cfa89640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe9c8162a80 0x7fe9c81668d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:48.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.015+0000 7fe9cfa89640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c8071bd0 con 0x7fe9c80b4260 2024-09-19T08:45:48.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.015+0000 7fe9cfa89640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe9c8071a50 con 0x7fe9c81626a0 2024-09-19T08:45:48.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.015+0000 7fe9cfa89640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe9c8071d50 con 0x7fe9c8162a80 2024-09-19T08:45:48.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.015+0000 7fe9cd825640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9c80b4260 0x7fe9c81579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:48.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.015+0000 7fe9cd825640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9c80b4260 0x7fe9c81579f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53782/0 (socket says 172.21.15.98:53782) 2024-09-19T08:45:48.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.015+0000 7fe9cd825640 1 -- 172.21.15.98:0/1504260248 learned_addr learned my addr 172.21.15.98:0/1504260248 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:48.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.015+0000 7fe9cd024640 1 --2- 172.21.15.98:0/1504260248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9c81626a0 0x7fe9c8166390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:48.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.015+0000 7fe9ce026640 1 --2- 172.21.15.98:0/1504260248 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe9c8162a80 0x7fe9c81668d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:48.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.015+0000 7fe9cd825640 1 -- 172.21.15.98:0/1504260248 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe9c8162a80 msgr2=0x7fe9c81668d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:48.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.015+0000 7fe9cd825640 1 --2- 172.21.15.98:0/1504260248 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe9c8162a80 0x7fe9c81668d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:48.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.015+0000 7fe9cd825640 1 -- 172.21.15.98:0/1504260248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9c81626a0 msgr2=0x7fe9c8166390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:48.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.015+0000 7fe9cd825640 1 --2- 172.21.15.98:0/1504260248 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9c81626a0 0x7fe9c8166390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:48.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.015+0000 7fe9cd825640 1 -- 172.21.15.98:0/1504260248 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9c815e490 con 0x7fe9c80b4260 2024-09-19T08:45:48.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.015+0000 7fe9cd825640 1 --2- 172.21.15.98:0/1504260248 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9c80b4260 0x7fe9c81579f0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000c370 tx=0x7fe9c000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:48.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.015+0000 7fe9be7fc640 1 -- 172.21.15.98:0/1504260248 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c00052d0 con 0x7fe9c80b4260 2024-09-19T08:45:48.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.015+0000 7fe9be7fc640 1 -- 172.21.15.98:0/1504260248 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9c0005470 con 0x7fe9c80b4260 2024-09-19T08:45:48.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.015+0000 7fe9cfa89640 1 -- 172.21.15.98:0/1504260248 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9c815e770 con 0x7fe9c80b4260 2024-09-19T08:45:48.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.015+0000 7fe9be7fc640 1 -- 172.21.15.98:0/1504260248 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0007260 con 0x7fe9c80b4260 2024-09-19T08:45:48.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.015+0000 7fe9cfa89640 1 -- 172.21.15.98:0/1504260248 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9c80b0ba0 con 0x7fe9c80b4260 2024-09-19T08:45:48.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.019+0000 7fe9cfa89640 1 -- 172.21.15.98:0/1504260248 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9c8163990 con 0x7fe9c80b4260 2024-09-19T08:45:48.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.019+0000 7fe9be7fc640 1 -- 172.21.15.98:0/1504260248 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fe9c0007400 con 0x7fe9c80b4260 2024-09-19T08:45:48.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.019+0000 7fe9be7fc640 1 --2- 172.21.15.98:0/1504260248 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe9ac067f20 0x7fe9ac06a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:48.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.019+0000 7fe9be7fc640 1 -- 172.21.15.98:0/1504260248 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7fe9c0085c50 con 0x7fe9c80b4260 2024-09-19T08:45:48.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.019+0000 7fe9cd024640 1 --2- 172.21.15.98:0/1504260248 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe9ac067f20 0x7fe9ac06a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:48.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.019+0000 7fe9cd024640 1 --2- 172.21.15.98:0/1504260248 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe9ac067f20 0x7fe9ac06a3e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe9b80ad000 tx=0x7fe9b80a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:48.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.023+0000 7fe9be7fc640 1 -- 172.21.15.98:0/1504260248 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe9c0055750 con 0x7fe9c80b4260 2024-09-19T08:45:48.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.195+0000 7fe9cfa89640 1 -- 172.21.15.98:0/1504260248 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot ls", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7fe9c80b0690 con 0x7fe9ac067f20 2024-09-19T08:45:48.198 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:45:48.198+0000 7ff73a69c640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-19T08:45:48.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.199+0000 7fe9be7fc640 1 -- 172.21.15.98:0/1504260248 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fe9c80b0690 con 0x7fe9ac067f20 2024-09-19T08:45:48.198 INFO:teuthology.orchestra.run.smithi098.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-19T08:45:48.204 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-19T08:45:48.205 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T08:45:48.205 DEBUG:teuthology.orchestra.run.smithi098:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:45:48.269 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:45:48.299 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:45:48.301 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:45:48.336 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T08:45:48.337 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:45:48.337 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:45:48.337 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:45:48.349 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T08:45:48.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.519+0000 7fbf19575640 1 Processor -- start 2024-09-19T08:45:48.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.519+0000 7fbf19575640 1 -- start start 2024-09-19T08:45:48.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.519+0000 7fbf19575640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbf1415c220 0x7fbf140733f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:48.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.519+0000 7fbf19575640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbf140d1700 0x7fbf140b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:48.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.519+0000 7fbf19575640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf140740b0 con 0x7fbf14073930 2024-09-19T08:45:48.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.519+0000 7fbf19575640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fbf14073dd0 con 0x7fbf1415c220 2024-09-19T08:45:48.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.519+0000 7fbf19575640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbf140b6460 con 0x7fbf140d1700 2024-09-19T08:45:48.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.519+0000 7fbf12ffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbf1415c220 0x7fbf140733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:48.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.519+0000 7fbf12ffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbf1415c220 0x7fbf140733f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53784/0 (socket says 172.21.15.98:53784) 2024-09-19T08:45:48.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.519+0000 7fbf12ffd640 1 -- 172.21.15.98:0/3567907217 learned_addr learned my addr 172.21.15.98:0/3567907217 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:48.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.519+0000 7fbf137fe640 1 --2- 172.21.15.98:0/3567907217 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbf140d1700 0x7fbf140b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:48.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.519+0000 7fbf12ffd640 1 -- 172.21.15.98:0/3567907217 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbf140d1700 msgr2=0x7fbf140b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:48.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.519+0000 7fbf12ffd640 1 --2- 172.21.15.98:0/3567907217 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbf140d1700 0x7fbf140b6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:48.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.519+0000 7fbf12ffd640 1 -- 172.21.15.98:0/3567907217 >> v1:172.21.15.98:6789/0 conn(0x7fbf14073930 legacy=0x7fbf140d0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:48.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.519+0000 7fbf12ffd640 1 -- 172.21.15.98:0/3567907217 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf140b6740 con 0x7fbf1415c220 2024-09-19T08:45:48.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.523+0000 7fbf12ffd640 1 --2- 172.21.15.98:0/3567907217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf1415c220 0x7fbf140733f0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fbf04009870 tx=0x7fbf0401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f49e2b79a617c00 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:48.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.523+0000 7fbf11ffb640 1 -- 172.21.15.98:0/3567907217 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf040a95f0 con 0x7fbf1415c220 2024-09-19T08:45:48.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.523+0000 7fbf11ffb640 1 -- 172.21.15.98:0/3567907217 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf040a9790 con 0x7fbf1415c220 2024-09-19T08:45:48.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.523+0000 7fbf11ffb640 1 -- 172.21.15.98:0/3567907217 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf040a9b60 con 0x7fbf1415c220 2024-09-19T08:45:48.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.523+0000 7fbf19575640 1 -- 172.21.15.98:0/3567907217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf1415c220 msgr2=0x7fbf140733f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:48.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.523+0000 7fbf19575640 1 --2- 172.21.15.98:0/3567907217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf1415c220 0x7fbf140733f0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fbf04009870 tx=0x7fbf0401c530 comp rx=0 tx=0).stop 2024-09-19T08:45:48.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.523+0000 7fbf19575640 1 -- 172.21.15.98:0/3567907217 shutdown_connections 2024-09-19T08:45:48.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.523+0000 7fbf19575640 1 --2- 172.21.15.98:0/3567907217 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbf140d1700 0x7fbf140b6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:48.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.523+0000 7fbf19575640 1 --2- 172.21.15.98:0/3567907217 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf1415c220 0x7fbf140733f0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:48.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.523+0000 7fbf19575640 1 -- 172.21.15.98:0/3567907217 >> 172.21.15.98:0/3567907217 conn(0x7fbf140b40a0 msgr2=0x7fbf140b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:48.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.523+0000 7fbf19575640 1 -- 172.21.15.98:0/3567907217 shutdown_connections 2024-09-19T08:45:48.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.523+0000 7fbf19575640 1 -- 172.21.15.98:0/3567907217 wait complete. 2024-09-19T08:45:48.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.523+0000 7fbf19575640 1 Processor -- start 2024-09-19T08:45:48.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.523+0000 7fbf19575640 1 -- start start 2024-09-19T08:45:48.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.523+0000 7fbf19575640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbf14073930 0x7fbf14151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:48.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.523+0000 7fbf19575640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf140d1700 0x7fbf1406c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:48.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.523+0000 7fbf19575640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf1415c220 0x7fbf1406cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:48.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.523+0000 7fbf19575640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1415ce10 con 0x7fbf140d1700 2024-09-19T08:45:48.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.523+0000 7fbf19575640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbf140d0e40 con 0x7fbf1415c220 2024-09-19T08:45:48.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.523+0000 7fbf19575640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbf1415cf90 con 0x7fbf14073930 2024-09-19T08:45:48.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.523+0000 7fbf127fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf140d1700 0x7fbf1406c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:48.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.523+0000 7fbf127fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf140d1700 0x7fbf1406c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53788/0 (socket says 172.21.15.98:53788) 2024-09-19T08:45:48.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.523+0000 7fbf127fc640 1 -- 172.21.15.98:0/2043568622 learned_addr learned my addr 172.21.15.98:0/2043568622 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:48.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.527+0000 7fbf12ffd640 1 --2- 172.21.15.98:0/2043568622 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbf14073930 0x7fbf14151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:48.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.527+0000 7fbf127fc640 1 -- 172.21.15.98:0/2043568622 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbf14073930 msgr2=0x7fbf14151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:48.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.527+0000 7fbf137fe640 1 --2- 172.21.15.98:0/2043568622 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf1415c220 0x7fbf1406cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:48.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.527+0000 7fbf127fc640 1 --2- 172.21.15.98:0/2043568622 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbf14073930 0x7fbf14151fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:48.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.527+0000 7fbf127fc640 1 -- 172.21.15.98:0/2043568622 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf1415c220 msgr2=0x7fbf1406cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:48.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.527+0000 7fbf127fc640 1 --2- 172.21.15.98:0/2043568622 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf1415c220 0x7fbf1406cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:48.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.527+0000 7fbf127fc640 1 -- 172.21.15.98:0/2043568622 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf1415e690 con 0x7fbf140d1700 2024-09-19T08:45:48.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.527+0000 7fbf12ffd640 1 --2- 172.21.15.98:0/2043568622 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbf14073930 0x7fbf14151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:48.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.527+0000 7fbf127fc640 1 --2- 172.21.15.98:0/2043568622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf140d1700 0x7fbf1406c6e0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fbf0800c450 tx=0x7fbf0800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:48.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.527+0000 7fbefb7fe640 1 -- 172.21.15.98:0/2043568622 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf08016020 con 0x7fbf140d1700 2024-09-19T08:45:48.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.527+0000 7fbefb7fe640 1 -- 172.21.15.98:0/2043568622 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf080052d0 con 0x7fbf140d1700 2024-09-19T08:45:48.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.527+0000 7fbf19575640 1 -- 172.21.15.98:0/2043568622 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf1406d250 con 0x7fbf140d1700 2024-09-19T08:45:48.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.527+0000 7fbefb7fe640 1 -- 172.21.15.98:0/2043568622 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf08007260 con 0x7fbf140d1700 2024-09-19T08:45:48.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.527+0000 7fbf19575640 1 -- 172.21.15.98:0/2043568622 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf1406d810 con 0x7fbf140d1700 2024-09-19T08:45:48.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.527+0000 7fbf19575640 1 -- 172.21.15.98:0/2043568622 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf140b6f40 con 0x7fbf140d1700 2024-09-19T08:45:48.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.527+0000 7fbefb7fe640 1 -- 172.21.15.98:0/2043568622 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fbf08007400 con 0x7fbf140d1700 2024-09-19T08:45:48.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.527+0000 7fbefb7fe640 1 --2- 172.21.15.98:0/2043568622 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbf00067ef0 0x7fbf0006a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:48.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.527+0000 7fbefb7fe640 1 -- 172.21.15.98:0/2043568622 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7fbf08086160 con 0x7fbf140d1700 2024-09-19T08:45:48.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.527+0000 7fbf12ffd640 1 --2- 172.21.15.98:0/2043568622 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbf00067ef0 0x7fbf0006a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:48.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.527+0000 7fbf12ffd640 1 --2- 172.21.15.98:0/2043568622 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbf00067ef0 0x7fbf0006a3b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbf040ae000 tx=0x7fbf040a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:48.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.531+0000 7fbefb7fe640 1 -- 172.21.15.98:0/2043568622 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fbf08020020 con 0x7fbf140d1700 2024-09-19T08:45:48.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.695+0000 7fbf19575640 1 -- 172.21.15.98:0/2043568622 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbf140d3150 con 0x7fbf140d1700 2024-09-19T08:45:48.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.695+0000 7fbefb7fe640 1 -- 172.21.15.98:0/2043568622 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v135) ==== 80+0+0 (secure 0 0 0) 0x7fbf08005480 con 0x7fbf140d1700 2024-09-19T08:45:48.696 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-19T08:45:48.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.699+0000 7fbf19575640 1 -- 172.21.15.98:0/2043568622 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbf00067ef0 msgr2=0x7fbf0006a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:48.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.699+0000 7fbf19575640 1 --2- 172.21.15.98:0/2043568622 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbf00067ef0 0x7fbf0006a3b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbf040ae000 tx=0x7fbf040a7000 comp rx=0 tx=0).stop 2024-09-19T08:45:48.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.699+0000 7fbf19575640 1 -- 172.21.15.98:0/2043568622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf140d1700 msgr2=0x7fbf1406c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:48.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.699+0000 7fbf19575640 1 --2- 172.21.15.98:0/2043568622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf140d1700 0x7fbf1406c6e0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fbf0800c450 tx=0x7fbf0800c910 comp rx=0 tx=0).stop 2024-09-19T08:45:48.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.699+0000 7fbf19575640 1 -- 172.21.15.98:0/2043568622 shutdown_connections 2024-09-19T08:45:48.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.699+0000 7fbf19575640 1 --2- 172.21.15.98:0/2043568622 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbf00067ef0 0x7fbf0006a3b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:48.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.699+0000 7fbf19575640 1 --2- 172.21.15.98:0/2043568622 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbf1415c220 0x7fbf1406cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:48.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.699+0000 7fbf19575640 1 --2- 172.21.15.98:0/2043568622 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbf140d1700 0x7fbf1406c6e0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:48.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.699+0000 7fbf19575640 1 --2- 172.21.15.98:0/2043568622 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbf14073930 0x7fbf14151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:48.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.699+0000 7fbf19575640 1 -- 172.21.15.98:0/2043568622 >> 172.21.15.98:0/2043568622 conn(0x7fbf140b40a0 msgr2=0x7fbf14153880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:48.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.699+0000 7fbf19575640 1 -- 172.21.15.98:0/2043568622 shutdown_connections 2024-09-19T08:45:48.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.699+0000 7fbf19575640 1 -- 172.21.15.98:0/2043568622 wait complete. 2024-09-19T08:45:48.714 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi098.front.sepia.ceph.com... 2024-09-19T08:45:48.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:48.714 DEBUG:teuthology.orchestra.run.smithi098:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T08:45:48.723 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:ceph-fuse[49695]: fuse finished with error 0 and tester_r 0 2024-09-19T08:45:48.724 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:45:48.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:48.724 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T08:45:48.800 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:45:48.800 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:45:48.800 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:45:48.807 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T08:45:48.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.967+0000 7f3f05b1b640 1 Processor -- start 2024-09-19T08:45:48.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.967+0000 7f3f05b1b640 1 -- start start 2024-09-19T08:45:48.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.967+0000 7f3f05b1b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3f00158fd0 0x7f3f000b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:48.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.967+0000 7f3f05b1b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3f000b4360 0x7f3f0006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:48.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.967+0000 7f3f05b1b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f0006c1d0 0x7f3f0015efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:48.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.967+0000 7f3f05b1b640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f3f0006c640 con 0x7f3f00158fd0 2024-09-19T08:45:48.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.967+0000 7f3f05b1b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f3f001593b0 con 0x7f3f000b4360 2024-09-19T08:45:48.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.967+0000 7f3f05b1b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3f000ccc60 con 0x7f3f0006c1d0 2024-09-19T08:45:48.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.967+0000 7f3eff7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3f00158fd0 0x7f3f000b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:48.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.967+0000 7f3eff7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3f00158fd0 0x7f3f000b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53804/0 (socket says 172.21.15.98:53804) 2024-09-19T08:45:48.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.967+0000 7f3eff7fe640 1 -- 172.21.15.98:0/980832459 learned_addr learned my addr 172.21.15.98:0/980832459 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:48.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.967+0000 7f3efeffd640 1 --2- 172.21.15.98:0/980832459 >> v2:172.21.15.139:3300/0 conn(0x7f3f000b4360 0x7f3f0006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:48.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.967+0000 7f3efffff640 1 --2- 172.21.15.98:0/980832459 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f0006c1d0 0x7f3f0015efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:48.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.967+0000 7f3eff7fe640 1 -- 172.21.15.98:0/980832459 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f0006c1d0 msgr2=0x7f3f0015efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:48.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.967+0000 7f3eff7fe640 1 --2- 172.21.15.98:0/980832459 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f0006c1d0 0x7f3f0015efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:48.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.967+0000 7f3eff7fe640 1 -- 172.21.15.98:0/980832459 >> v2:172.21.15.139:3300/0 conn(0x7f3f000b4360 msgr2=0x7f3f0006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:48.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.967+0000 7f3eff7fe640 1 --2- 172.21.15.98:0/980832459 >> v2:172.21.15.139:3300/0 conn(0x7f3f000b4360 0x7f3f0006bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:48.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.967+0000 7f3eff7fe640 1 -- 172.21.15.98:0/980832459 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f000ccf40 con 0x7f3f00158fd0 2024-09-19T08:45:48.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.967+0000 7f3efeffd640 1 --2- 172.21.15.98:0/980832459 >> v2:172.21.15.139:3300/0 conn(0x7f3f000b4360 0x7f3f0006bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:48.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.971+0000 7f3eff7fe640 1 --2- 172.21.15.98:0/980832459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f00158fd0 0x7f3f000b3e20 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f3f000cdef0 tx=0x7f3ef001c1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e6f5c59a8a760d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:48.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.971+0000 7f3efe7fc640 1 -- 172.21.15.98:0/980832459 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef00b0070 con 0x7f3f00158fd0 2024-09-19T08:45:48.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.971+0000 7f3efe7fc640 1 -- 172.21.15.98:0/980832459 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ef00ab600 con 0x7f3f00158fd0 2024-09-19T08:45:48.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.971+0000 7f3efe7fc640 1 -- 172.21.15.98:0/980832459 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef00ab9d0 con 0x7f3f00158fd0 2024-09-19T08:45:48.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.971+0000 7f3f05b1b640 1 -- 172.21.15.98:0/980832459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f00158fd0 msgr2=0x7f3f000b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:48.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.971+0000 7f3f05b1b640 1 --2- 172.21.15.98:0/980832459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f00158fd0 0x7f3f000b3e20 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f3f000cdef0 tx=0x7f3ef001c1a0 comp rx=0 tx=0).stop 2024-09-19T08:45:48.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.971+0000 7f3f05b1b640 1 -- 172.21.15.98:0/980832459 shutdown_connections 2024-09-19T08:45:48.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.971+0000 7f3f05b1b640 1 --2- 172.21.15.98:0/980832459 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f0006c1d0 0x7f3f0015efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:48.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.971+0000 7f3f05b1b640 1 --2- 172.21.15.98:0/980832459 >> v2:172.21.15.139:3300/0 conn(0x7f3f000b4360 0x7f3f0006bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:48.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.971+0000 7f3f05b1b640 1 --2- 172.21.15.98:0/980832459 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f00158fd0 0x7f3f000b3e20 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:48.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.971+0000 7f3f05b1b640 1 -- 172.21.15.98:0/980832459 >> 172.21.15.98:0/980832459 conn(0x7f3f000b2010 msgr2=0x7f3f000b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:48.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.971+0000 7f3f05b1b640 1 -- 172.21.15.98:0/980832459 shutdown_connections 2024-09-19T08:45:48.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.971+0000 7f3f05b1b640 1 -- 172.21.15.98:0/980832459 wait complete. 2024-09-19T08:45:48.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.971+0000 7f3f05b1b640 1 Processor -- start 2024-09-19T08:45:48.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.971+0000 7f3f05b1b640 1 -- start start 2024-09-19T08:45:48.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.975+0000 7f3f05b1b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f0006c1d0 0x7f3f000af190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:48.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.975+0000 7f3f05b1b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f000b4360 0x7f3f00150b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:48.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.975+0000 7f3f05b1b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f00158fd0 0x7f3f001519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:48.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.975+0000 7f3f05b1b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0015c920 con 0x7f3f0006c1d0 2024-09-19T08:45:48.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.975+0000 7f3f05b1b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0015c7a0 con 0x7f3f000b4360 2024-09-19T08:45:48.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.975+0000 7f3f05b1b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3f000ce840 con 0x7f3f00158fd0 2024-09-19T08:45:48.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.975+0000 7f3eff7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f0006c1d0 0x7f3f000af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:48.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.975+0000 7f3eff7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f0006c1d0 0x7f3f000af190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53812/0 (socket says 172.21.15.98:53812) 2024-09-19T08:45:48.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.975+0000 7f3eff7fe640 1 -- 172.21.15.98:0/2961750464 learned_addr learned my addr 172.21.15.98:0/2961750464 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:48.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.975+0000 7f3efeffd640 1 --2- 172.21.15.98:0/2961750464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f000b4360 0x7f3f00150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:48.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.975+0000 7f3eff7fe640 1 -- 172.21.15.98:0/2961750464 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f00158fd0 msgr2=0x7f3f001519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:48.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.975+0000 7f3efffff640 1 --2- 172.21.15.98:0/2961750464 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f00158fd0 0x7f3f001519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:48.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.975+0000 7f3eff7fe640 1 --2- 172.21.15.98:0/2961750464 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f00158fd0 0x7f3f001519d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:48.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.975+0000 7f3eff7fe640 1 -- 172.21.15.98:0/2961750464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f000b4360 msgr2=0x7f3f00150b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:48.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.975+0000 7f3eff7fe640 1 --2- 172.21.15.98:0/2961750464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f000b4360 0x7f3f00150b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:48.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.975+0000 7f3eff7fe640 1 -- 172.21.15.98:0/2961750464 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f00152220 con 0x7f3f0006c1d0 2024-09-19T08:45:48.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.975+0000 7f3eff7fe640 1 --2- 172.21.15.98:0/2961750464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f0006c1d0 0x7f3f000af190 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f3ef00ae040 tx=0x7f3ef0018f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:48.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.975+0000 7f3ee7fff640 1 -- 172.21.15.98:0/2961750464 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef00b0040 con 0x7f3f0006c1d0 2024-09-19T08:45:48.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.975+0000 7f3ee7fff640 1 -- 172.21.15.98:0/2961750464 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ef00b9070 con 0x7f3f0006c1d0 2024-09-19T08:45:48.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.975+0000 7f3ee7fff640 1 -- 172.21.15.98:0/2961750464 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef00b48a0 con 0x7f3f0006c1d0 2024-09-19T08:45:48.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.975+0000 7f3f05b1b640 1 -- 172.21.15.98:0/2961750464 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f00152500 con 0x7f3f0006c1d0 2024-09-19T08:45:48.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.975+0000 7f3f05b1b640 1 -- 172.21.15.98:0/2961750464 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f00152ac0 con 0x7f3f0006c1d0 2024-09-19T08:45:48.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.975+0000 7f3f05b1b640 1 -- 172.21.15.98:0/2961750464 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f0006bc90 con 0x7f3f0006c1d0 2024-09-19T08:45:48.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.975+0000 7f3ee7fff640 1 -- 172.21.15.98:0/2961750464 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f3ef00ab4c0 con 0x7f3f0006c1d0 2024-09-19T08:45:48.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.975+0000 7f3ee7fff640 1 --2- 172.21.15.98:0/2961750464 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3eec067ef0 0x7f3eec06a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:48.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.975+0000 7f3ee7fff640 1 -- 172.21.15.98:0/2961750464 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7f3ef0120530 con 0x7f3f0006c1d0 2024-09-19T08:45:48.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.975+0000 7f3efeffd640 1 --2- 172.21.15.98:0/2961750464 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3eec067ef0 0x7f3eec06a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:48.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.979+0000 7f3efeffd640 1 --2- 172.21.15.98:0/2961750464 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3eec067ef0 0x7f3eec06a3b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4006fd0 tx=0x7f3ef4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:48.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:48.979+0000 7f3ee7fff640 1 -- 172.21.15.98:0/2961750464 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3ef0016110 con 0x7f3f0006c1d0 2024-09-19T08:45:49.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.159+0000 7f3f05b1b640 1 -- 172.21.15.98:0/2961750464 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3f000b35f0 con 0x7f3f0006c1d0 2024-09-19T08:45:49.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.159+0000 7f3ee7fff640 1 -- 172.21.15.98:0/2961750464 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v135) ==== 80+0+0 (secure 0 0 0) 0x7f3ef00f0800 con 0x7f3f0006c1d0 2024-09-19T08:45:49.157 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-19T08:45:49.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.159+0000 7f3f05b1b640 1 -- 172.21.15.98:0/2961750464 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3eec067ef0 msgr2=0x7f3eec06a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:49.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.159+0000 7f3f05b1b640 1 --2- 172.21.15.98:0/2961750464 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3eec067ef0 0x7f3eec06a3b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4006fd0 tx=0x7f3ef4008040 comp rx=0 tx=0).stop 2024-09-19T08:45:49.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.159+0000 7f3f05b1b640 1 -- 172.21.15.98:0/2961750464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f0006c1d0 msgr2=0x7f3f000af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:49.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.159+0000 7f3f05b1b640 1 --2- 172.21.15.98:0/2961750464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f0006c1d0 0x7f3f000af190 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f3ef00ae040 tx=0x7f3ef0018f40 comp rx=0 tx=0).stop 2024-09-19T08:45:49.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.159+0000 7f3f05b1b640 1 -- 172.21.15.98:0/2961750464 shutdown_connections 2024-09-19T08:45:49.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.159+0000 7f3f05b1b640 1 --2- 172.21.15.98:0/2961750464 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3eec067ef0 0x7f3eec06a3b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:49.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.159+0000 7f3f05b1b640 1 --2- 172.21.15.98:0/2961750464 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f00158fd0 0x7f3f001519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:49.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.159+0000 7f3f05b1b640 1 --2- 172.21.15.98:0/2961750464 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f000b4360 0x7f3f00150b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:49.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.159+0000 7f3f05b1b640 1 --2- 172.21.15.98:0/2961750464 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f0006c1d0 0x7f3f000af190 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:49.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.159+0000 7f3f05b1b640 1 -- 172.21.15.98:0/2961750464 >> 172.21.15.98:0/2961750464 conn(0x7f3f000b2010 msgr2=0x7f3f0006d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:49.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.159+0000 7f3f05b1b640 1 -- 172.21.15.98:0/2961750464 shutdown_connections 2024-09-19T08:45:49.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.159+0000 7f3f05b1b640 1 -- 172.21.15.98:0/2961750464 wait complete. 2024-09-19T08:45:49.176 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi139.front.sepia.ceph.com... 2024-09-19T08:45:49.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:49.176 DEBUG:teuthology.orchestra.run.smithi139:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-19T08:45:49.189 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[36592]: fuse finished with error 0 and tester_r 0 2024-09-19T08:45:49.191 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:45:49.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:49.191 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-19T08:45:49.267 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:45:49.268 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:45:49.268 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:45:49.273 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15846 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T08:45:49.273 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:45:49.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:49.274 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-19T08:45:49.320 INFO:teuthology.orchestra.run.smithi098.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-19T08:45:49.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:45:49.321 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:45:49.321 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:45:49.321 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:45:49.327 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T08:45:49.328 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:45:49.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:45:49.328 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-19T08:45:49.375 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-19T08:45:49.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:45:49.376 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:45:49.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.503+0000 7fa0b6823640 1 Processor -- start 2024-09-19T08:45:49.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.503+0000 7fa0b6823640 1 -- start start 2024-09-19T08:45:49.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.507+0000 7fa0b6823640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa0b00d4820 0x7fa0b01621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:49.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.507+0000 7fa0b6823640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0b0068440 con 0x7fa0b00b7f80 2024-09-19T08:45:49.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.507+0000 7fa0b6823640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0b0162a90 con 0x7fa0b00b9970 2024-09-19T08:45:49.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.507+0000 7fa0b6823640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fa0b0163050 con 0x7fa0b00d4820 2024-09-19T08:45:49.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.507+0000 7fa0affff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fa0b00b7f80 0x7fa0b015f560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42440/0 (socket says 172.21.15.98:42440) 2024-09-19T08:45:49.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.507+0000 7fa0affff640 1 -- 172.21.15.98:0/3560428237 learned_addr learned my addr 172.21.15.98:0/3560428237 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:49.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.507+0000 7fa0af7fe640 1 --2- 172.21.15.98:0/3560428237 >> v2:172.21.15.139:3300/0 conn(0x7fa0b00d4820 0x7fa0b01621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:49.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.507+0000 7fa0aeffd640 1 -- 172.21.15.98:0/3560428237 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3880522084 0 0) 0x7fa0b0068440 con 0x7fa0b00b7f80 2024-09-19T08:45:49.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.507+0000 7fa0aeffd640 1 -- 172.21.15.98:0/3560428237 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa098003530 con 0x7fa0b00b7f80 2024-09-19T08:45:49.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.507+0000 7fa0aeffd640 1 -- 172.21.15.98:0/3560428237 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1581099162 0 0) 0x7fa098003530 con 0x7fa0b00b7f80 2024-09-19T08:45:49.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.507+0000 7fa0aeffd640 1 -- 172.21.15.98:0/3560428237 >> v2:172.21.15.139:3300/0 conn(0x7fa0b00d4820 msgr2=0x7fa0b01621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:49.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.507+0000 7fa0aeffd640 1 --2- 172.21.15.98:0/3560428237 >> v2:172.21.15.139:3300/0 conn(0x7fa0b00d4820 0x7fa0b01621b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:49.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.507+0000 7fa0aeffd640 1 -- 172.21.15.98:0/3560428237 >> v1:172.21.15.139:6789/0 conn(0x7fa0b00b9970 legacy=0x7fa0b00b9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:49.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.507+0000 7fa0aeffd640 1 -- 172.21.15.98:0/3560428237 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b0162d70 con 0x7fa0b00b7f80 2024-09-19T08:45:49.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.507+0000 7fa0aeffd640 1 -- 172.21.15.98:0/3560428237 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fa0a0001680 con 0x7fa0b00b7f80 2024-09-19T08:45:49.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.507+0000 7fa0aeffd640 1 -- 172.21.15.98:0/3560428237 >> v1:172.21.15.98:6789/0 conn(0x7fa0b00b7f80 legacy=0x7fa0b015f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:49.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.507+0000 7fa0aeffd640 1 --2- 172.21.15.98:0/3560428237 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0980039c0 0x7fa098003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:49.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.507+0000 7fa0aeffd640 1 --2- 172.21.15.98:0/3560428237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa098004810 0x7fa098004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:49.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.507+0000 7fa0aeffd640 1 --2- 172.21.15.98:0/3560428237 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0980051d0 0x7fa09800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:49.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.507+0000 7fa0aeffd640 1 -- 172.21.15.98:0/3560428237 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b0163050 con 0x7fa0980039c0 2024-09-19T08:45:49.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.507+0000 7fa0aeffd640 1 -- 172.21.15.98:0/3560428237 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa09800c170 con 0x7fa098004810 2024-09-19T08:45:49.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.507+0000 7fa0aeffd640 1 -- 172.21.15.98:0/3560428237 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa09800c370 con 0x7fa0980051d0 2024-09-19T08:45:49.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.507+0000 7fa0affff640 1 --2- 172.21.15.98:0/3560428237 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0980039c0 0x7fa098003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:49.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.507+0000 7fa0affff640 1 -- 172.21.15.98:0/3560428237 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0980051d0 msgr2=0x7fa09800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:49.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.507+0000 7fa0affff640 1 --2- 172.21.15.98:0/3560428237 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0980051d0 0x7fa09800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:49.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.507+0000 7fa0affff640 1 -- 172.21.15.98:0/3560428237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa098004810 msgr2=0x7fa098004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:49.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.507+0000 7fa0affff640 1 --2- 172.21.15.98:0/3560428237 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa098004810 0x7fa098004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:49.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.507+0000 7fa0affff640 1 -- 172.21.15.98:0/3560428237 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa098003530 con 0x7fa0980039c0 2024-09-19T08:45:49.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.507+0000 7fa0b4dc0640 1 -- 172.21.15.98:0/3560428237 reap_dead start 2024-09-19T08:45:49.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.507+0000 7fa0affff640 1 --2- 172.21.15.98:0/3560428237 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0980039c0 0x7fa098003de0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000c1c0 tx=0x7fa0a001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22402dc0be2bf39 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:49.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.507+0000 7fa0aeffd640 1 -- 172.21.15.98:0/3560428237 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0007d00 con 0x7fa0980039c0 2024-09-19T08:45:49.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.507+0000 7fa0aeffd640 1 -- 172.21.15.98:0/3560428237 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0a00a9020 con 0x7fa0980039c0 2024-09-19T08:45:49.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.507+0000 7fa0aeffd640 1 -- 172.21.15.98:0/3560428237 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0005b00 con 0x7fa0980039c0 2024-09-19T08:45:49.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.511+0000 7fa0b6823640 1 -- 172.21.15.98:0/3560428237 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0980039c0 msgr2=0x7fa098003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:49.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.511+0000 7fa0b6823640 1 --2- 172.21.15.98:0/3560428237 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0980039c0 0x7fa098003de0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000c1c0 tx=0x7fa0a001cc60 comp rx=0 tx=0).stop 2024-09-19T08:45:49.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.511+0000 7fa0b6823640 1 -- 172.21.15.98:0/3560428237 shutdown_connections 2024-09-19T08:45:49.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.511+0000 7fa0b6823640 1 --2- 172.21.15.98:0/3560428237 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0980039c0 0x7fa098003de0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:49.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.511+0000 7fa0b6823640 1 -- 172.21.15.98:0/3560428237 >> 172.21.15.98:0/3560428237 conn(0x7fa0b0074e40 msgr2=0x7fa0b0075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:49.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.511+0000 7fa0b6823640 1 -- 172.21.15.98:0/3560428237 shutdown_connections 2024-09-19T08:45:49.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.511+0000 7fa0b6823640 1 -- 172.21.15.98:0/3560428237 wait complete. 2024-09-19T08:45:49.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.511+0000 7fa0b6823640 1 Processor -- start 2024-09-19T08:45:49.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.511+0000 7fa0b6823640 1 -- start start 2024-09-19T08:45:49.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.511+0000 7fa0b6823640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0980039c0 0x7fa0b01f70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:49.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.511+0000 7fa0b6823640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0b01f7630 0x7fa0b013a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:49.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.511+0000 7fa0b6823640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa0b013aae0 0x7fa0b01fb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:49.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.511+0000 7fa0b6823640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b015f950 con 0x7fa0980039c0 2024-09-19T08:45:49.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.511+0000 7fa0b6823640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b00b8570 con 0x7fa0b013aae0 2024-09-19T08:45:49.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.511+0000 7fa0b6823640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa0b015fad0 con 0x7fa0b01f7630 2024-09-19T08:45:49.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.511+0000 7fa0affff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0980039c0 0x7fa0b01f70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:49.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.511+0000 7fa0affff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0980039c0 0x7fa0b01f70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53838/0 (socket says 172.21.15.98:53838) 2024-09-19T08:45:49.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.511+0000 7fa0affff640 1 -- 172.21.15.98:0/1221128364 learned_addr learned my addr 172.21.15.98:0/1221128364 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:49.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.511+0000 7fa0affff640 1 -- 172.21.15.98:0/1221128364 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0b01f7630 msgr2=0x7fa0b013a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:49.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.511+0000 7fa0affff640 1 --2- 172.21.15.98:0/1221128364 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0b01f7630 0x7fa0b013a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:49.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.511+0000 7fa0affff640 1 -- 172.21.15.98:0/1221128364 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa0b013aae0 msgr2=0x7fa0b01fb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:49.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.511+0000 7fa0affff640 1 --2- 172.21.15.98:0/1221128364 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa0b013aae0 0x7fa0b01fb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:49.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.511+0000 7fa0affff640 1 -- 172.21.15.98:0/1221128364 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b01f7a90 con 0x7fa0980039c0 2024-09-19T08:45:49.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.511+0000 7fa0affff640 1 --2- 172.21.15.98:0/1221128364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0980039c0 0x7fa0b01f70f0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fa0a001ca90 tx=0x7fa098005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:49.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.511+0000 7fa0acff9640 1 -- 172.21.15.98:0/1221128364 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a00a5a50 con 0x7fa0980039c0 2024-09-19T08:45:49.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.511+0000 7fa0acff9640 1 -- 172.21.15.98:0/1221128364 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0a00b2070 con 0x7fa0980039c0 2024-09-19T08:45:49.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.511+0000 7fa0b6823640 1 -- 172.21.15.98:0/1221128364 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b013af60 con 0x7fa0980039c0 2024-09-19T08:45:49.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.511+0000 7fa0acff9640 1 -- 172.21.15.98:0/1221128364 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a00a5df0 con 0x7fa0980039c0 2024-09-19T08:45:49.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.511+0000 7fa0b6823640 1 -- 172.21.15.98:0/1221128364 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b01585f0 con 0x7fa0980039c0 2024-09-19T08:45:49.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.515+0000 7fa0b6823640 1 -- 172.21.15.98:0/1221128364 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0b0143e10 con 0x7fa0980039c0 2024-09-19T08:45:49.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.515+0000 7fa0acff9640 1 -- 172.21.15.98:0/1221128364 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fa0a00047f0 con 0x7fa0980039c0 2024-09-19T08:45:49.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.515+0000 7fa0acff9640 1 --2- 172.21.15.98:0/1221128364 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa09c067f20 0x7fa09c06a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:49.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.515+0000 7fa0acff9640 1 -- 172.21.15.98:0/1221128364 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7fa0a011f9e0 con 0x7fa0980039c0 2024-09-19T08:45:49.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.515+0000 7fa0af7fe640 1 --2- 172.21.15.98:0/1221128364 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa09c067f20 0x7fa09c06a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:49.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.515+0000 7fa0af7fe640 1 --2- 172.21.15.98:0/1221128364 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa09c067f20 0x7fa09c06a3e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa0a40093f0 tx=0x7fa0a4009320 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:49.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.519+0000 7fa0acff9640 1 -- 172.21.15.98:0/1221128364 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa0b01f7a90 con 0x7fa0980039c0 2024-09-19T08:45:49.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.715+0000 7fa0b6823640 1 -- 172.21.15.98:0/1221128364 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0b01601a0 con 0x7fa0980039c0 2024-09-19T08:45:49.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.715+0000 7fa0acff9640 1 -- 172.21.15.98:0/1221128364 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5230 (secure 0 0 0) 0x7fa0a0016970 con 0x7fa0980039c0 2024-09-19T08:45:49.714 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:45:49.714 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":75,"btime":"2024-09-19T08:45:39:977951+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/1907681163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":1907681163},{"type":"v1","addr":"172.21.15.139:6834","nonce":1907681163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7896,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2481662340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2481662340},{"type":"v1","addr":"172.21.15.98:6833","nonce":2481662340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7902,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/286818221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":286818221},{"type":"v1","addr":"172.21.15.98:6835","nonce":286818221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:45:28.140764+0000","modified":"2024-09-19T08:45:30.144582+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7914},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7914":{"gid":7914,"name":"b","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6835/4255670104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6833","nonce":4255670104},{"type":"v1","addr":"172.21.15.139:6835","nonce":4255670104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7914,"qdb_cluster":[7914]},"id":10}]} 2024-09-19T08:45:49.714 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 75 2024-09-19T08:45:49.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.715+0000 7fa0b6823640 1 -- 172.21.15.98:0/1221128364 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa09c067f20 msgr2=0x7fa09c06a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:49.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.715+0000 7fa0b6823640 1 --2- 172.21.15.98:0/1221128364 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa09c067f20 0x7fa09c06a3e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa0a40093f0 tx=0x7fa0a4009320 comp rx=0 tx=0).stop 2024-09-19T08:45:49.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.715+0000 7fa0b6823640 1 -- 172.21.15.98:0/1221128364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0980039c0 msgr2=0x7fa0b01f70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:49.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.715+0000 7fa0b6823640 1 --2- 172.21.15.98:0/1221128364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0980039c0 0x7fa0b01f70f0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fa0a001ca90 tx=0x7fa098005840 comp rx=0 tx=0).stop 2024-09-19T08:45:49.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.715+0000 7fa0b6823640 1 -- 172.21.15.98:0/1221128364 shutdown_connections 2024-09-19T08:45:49.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.715+0000 7fa0b6823640 1 --2- 172.21.15.98:0/1221128364 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa09c067f20 0x7fa09c06a3e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:49.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.715+0000 7fa0b6823640 1 --2- 172.21.15.98:0/1221128364 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa0b013aae0 0x7fa0b01fb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:49.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.715+0000 7fa0b6823640 1 --2- 172.21.15.98:0/1221128364 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa0b01f7630 0x7fa0b013a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:49.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.715+0000 7fa0b6823640 1 --2- 172.21.15.98:0/1221128364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa0980039c0 0x7fa0b01f70f0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:49.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.715+0000 7fa0b6823640 1 -- 172.21.15.98:0/1221128364 >> 172.21.15.98:0/1221128364 conn(0x7fa0b0074e40 msgr2=0x7fa0b0160480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:49.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.715+0000 7fa0b6823640 1 -- 172.21.15.98:0/1221128364 shutdown_connections 2024-09-19T08:45:49.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.719+0000 7fa0b6823640 1 -- 172.21.15.98:0/1221128364 wait complete. 2024-09-19T08:45:49.731 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:45:49.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.851+0000 7fe80bdc0640 1 Processor -- start 2024-09-19T08:45:49.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.851+0000 7fe80bdc0640 1 -- start start 2024-09-19T08:45:49.853 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.855+0000 7fe80bdc0640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe80406d880 0x7fe8040b3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:49.853 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.855+0000 7fe80bdc0640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe80415ea10 con 0x7fe804141110 2024-09-19T08:45:49.853 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.855+0000 7fe80bdc0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe80415ecf0 con 0x7fe804140d30 2024-09-19T08:45:49.853 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.855+0000 7fe80bdc0640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fe80415e7c0 con 0x7fe80406d880 2024-09-19T08:45:49.853 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.855+0000 7fe80935b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe80406d880 0x7fe8040b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:49.853 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.855+0000 7fe80935b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe80406d880 0x7fe8040b3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53846/0 (socket says 172.21.15.98:53846) 2024-09-19T08:45:49.853 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.855+0000 7fe80935b640 1 -- 172.21.15.98:0/3270935814 learned_addr learned my addr 172.21.15.98:0/3270935814 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:49.853 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.855+0000 7fe80a35d640 1 --1- 172.21.15.98:0/3270935814 >> v1:172.21.15.98:6789/0 conn(0x7fe804141110 0x7fe80415b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42446/0 (socket says 172.21.15.98:42446) 2024-09-19T08:45:49.853 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.855+0000 7fe80935b640 1 -- 172.21.15.98:0/3270935814 >> v1:172.21.15.139:6789/0 conn(0x7fe804140d30 legacy=0x7fe80406d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:49.853 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.855+0000 7fe80935b640 1 -- 172.21.15.98:0/3270935814 >> v1:172.21.15.98:6789/0 conn(0x7fe804141110 legacy=0x7fe80415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:49.853 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.855+0000 7fe80935b640 1 -- 172.21.15.98:0/3270935814 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80415efd0 con 0x7fe80406d880 2024-09-19T08:45:49.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.855+0000 7fe80935b640 1 --2- 172.21.15.98:0/3270935814 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe80406d880 0x7fe8040b3310 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fe80415e940 tx=0x7fe7f801e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e22e029cef4984 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:49.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.855+0000 7fe808b5a640 1 -- 172.21.15.98:0/3270935814 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f80ad070 con 0x7fe80406d880 2024-09-19T08:45:49.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.855+0000 7fe808b5a640 1 -- 172.21.15.98:0/3270935814 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7f8004230 con 0x7fe80406d880 2024-09-19T08:45:49.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.855+0000 7fe808b5a640 1 -- 172.21.15.98:0/3270935814 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f80a7860 con 0x7fe80406d880 2024-09-19T08:45:49.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.855+0000 7fe80bdc0640 1 -- 172.21.15.98:0/3270935814 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe80406d880 msgr2=0x7fe8040b3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:49.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.859+0000 7fe80bdc0640 1 --2- 172.21.15.98:0/3270935814 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe80406d880 0x7fe8040b3310 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fe80415e940 tx=0x7fe7f801e9f0 comp rx=0 tx=0).stop 2024-09-19T08:45:49.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.859+0000 7fe80bdc0640 1 -- 172.21.15.98:0/3270935814 shutdown_connections 2024-09-19T08:45:49.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.859+0000 7fe80bdc0640 1 --2- 172.21.15.98:0/3270935814 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe80406d880 0x7fe8040b3310 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:49.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.859+0000 7fe80bdc0640 1 -- 172.21.15.98:0/3270935814 >> 172.21.15.98:0/3270935814 conn(0x7fe8040b2010 msgr2=0x7fe8040b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:49.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.859+0000 7fe80bdc0640 1 -- 172.21.15.98:0/3270935814 shutdown_connections 2024-09-19T08:45:49.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.859+0000 7fe80bdc0640 1 -- 172.21.15.98:0/3270935814 wait complete. 2024-09-19T08:45:49.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.859+0000 7fe80bdc0640 1 Processor -- start 2024-09-19T08:45:49.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.859+0000 7fe80bdc0640 1 -- start start 2024-09-19T08:45:49.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.859+0000 7fe80bdc0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe80406d880 0x7fe8041303c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:49.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.859+0000 7fe80bdc0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe804140d30 0x7fe804165250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:49.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.859+0000 7fe80bdc0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe804141110 0x7fe804130e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:49.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.859+0000 7fe80bdc0640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe80415ba80 con 0x7fe80406d880 2024-09-19T08:45:49.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.859+0000 7fe80bdc0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe80415b900 con 0x7fe804141110 2024-09-19T08:45:49.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.859+0000 7fe80bdc0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe80415bc00 con 0x7fe804140d30 2024-09-19T08:45:49.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.859+0000 7fe809b5c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe80406d880 0x7fe8041303c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:49.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.859+0000 7fe809b5c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe80406d880 0x7fe8041303c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53852/0 (socket says 172.21.15.98:53852) 2024-09-19T08:45:49.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.859+0000 7fe809b5c640 1 -- 172.21.15.98:0/82181556 learned_addr learned my addr 172.21.15.98:0/82181556 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:49.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.859+0000 7fe809b5c640 1 -- 172.21.15.98:0/82181556 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe804140d30 msgr2=0x7fe804165250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:49.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.859+0000 7fe809b5c640 1 --2- 172.21.15.98:0/82181556 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe804140d30 0x7fe804165250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:49.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.859+0000 7fe809b5c640 1 -- 172.21.15.98:0/82181556 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe804141110 msgr2=0x7fe804130e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:49.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.859+0000 7fe809b5c640 1 --2- 172.21.15.98:0/82181556 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe804141110 0x7fe804130e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:49.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.859+0000 7fe809b5c640 1 -- 172.21.15.98:0/82181556 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe804130900 con 0x7fe80406d880 2024-09-19T08:45:49.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.859+0000 7fe809b5c640 1 --2- 172.21.15.98:0/82181556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe80406d880 0x7fe8041303c0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc00c530 tx=0x7fe7fc00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:49.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.859+0000 7fe8027fc640 1 -- 172.21.15.98:0/82181556 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc016020 con 0x7fe80406d880 2024-09-19T08:45:49.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.859+0000 7fe8027fc640 1 -- 172.21.15.98:0/82181556 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7fc005220 con 0x7fe80406d880 2024-09-19T08:45:49.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.859+0000 7fe80bdc0640 1 -- 172.21.15.98:0/82181556 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe804165790 con 0x7fe80406d880 2024-09-19T08:45:49.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.859+0000 7fe8027fc640 1 -- 172.21.15.98:0/82181556 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc007030 con 0x7fe80406d880 2024-09-19T08:45:49.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.859+0000 7fe80bdc0640 1 -- 172.21.15.98:0/82181556 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe80412dfa0 con 0x7fe80406d880 2024-09-19T08:45:49.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.859+0000 7fe80bdc0640 1 -- 172.21.15.98:0/82181556 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe80415bfd0 con 0x7fe80406d880 2024-09-19T08:45:49.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.863+0000 7fe8027fc640 1 -- 172.21.15.98:0/82181556 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fe7fc0071d0 con 0x7fe80406d880 2024-09-19T08:45:49.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.863+0000 7fe8027fc640 1 --2- 172.21.15.98:0/82181556 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe7f0067f20 0x7fe7f006a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:49.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.863+0000 7fe8027fc640 1 -- 172.21.15.98:0/82181556 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7fe7fc086140 con 0x7fe80406d880 2024-09-19T08:45:49.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.863+0000 7fe80935b640 1 --2- 172.21.15.98:0/82181556 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe7f0067f20 0x7fe7f006a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:49.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.863+0000 7fe80935b640 1 --2- 172.21.15.98:0/82181556 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe7f0067f20 0x7fe7f006a3e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe7f80b1000 tx=0x7fe7f8004740 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:49.865 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:49.867+0000 7fe8027fc640 1 -- 172.21.15.98:0/82181556 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe7fc020020 con 0x7fe80406d880 2024-09-19T08:45:50.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.067+0000 7fe80bdc0640 1 -- 172.21.15.98:0/82181556 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8040cc990 con 0x7fe80406d880 2024-09-19T08:45:50.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.067+0000 7fe8027fc640 1 -- 172.21.15.98:0/82181556 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5230 (secure 0 0 0) 0x7fe804130900 con 0x7fe80406d880 2024-09-19T08:45:50.066 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:45:50.066 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":75,"btime":"2024-09-19T08:45:39:977951+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/1907681163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":1907681163},{"type":"v1","addr":"172.21.15.139:6834","nonce":1907681163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7896,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2481662340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2481662340},{"type":"v1","addr":"172.21.15.98:6833","nonce":2481662340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7902,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/286818221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":286818221},{"type":"v1","addr":"172.21.15.98:6835","nonce":286818221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:45:28.140764+0000","modified":"2024-09-19T08:45:30.144582+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7914},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7914":{"gid":7914,"name":"b","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6835/4255670104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6833","nonce":4255670104},{"type":"v1","addr":"172.21.15.139:6835","nonce":4255670104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7914,"qdb_cluster":[7914]},"id":10}]} 2024-09-19T08:45:50.066 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 75 2024-09-19T08:45:50.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.067+0000 7fe80bdc0640 1 -- 172.21.15.98:0/82181556 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe7f0067f20 msgr2=0x7fe7f006a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:50.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.067+0000 7fe80bdc0640 1 --2- 172.21.15.98:0/82181556 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe7f0067f20 0x7fe7f006a3e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe7f80b1000 tx=0x7fe7f8004740 comp rx=0 tx=0).stop 2024-09-19T08:45:50.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.067+0000 7fe80bdc0640 1 -- 172.21.15.98:0/82181556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe80406d880 msgr2=0x7fe8041303c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:50.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.067+0000 7fe80bdc0640 1 --2- 172.21.15.98:0/82181556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe80406d880 0x7fe8041303c0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc00c530 tx=0x7fe7fc00c9f0 comp rx=0 tx=0).stop 2024-09-19T08:45:50.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.067+0000 7fe80bdc0640 1 -- 172.21.15.98:0/82181556 shutdown_connections 2024-09-19T08:45:50.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.067+0000 7fe80bdc0640 1 --2- 172.21.15.98:0/82181556 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe7f0067f20 0x7fe7f006a3e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:50.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.067+0000 7fe80bdc0640 1 --2- 172.21.15.98:0/82181556 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe804141110 0x7fe804130e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:50.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.067+0000 7fe80bdc0640 1 --2- 172.21.15.98:0/82181556 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe804140d30 0x7fe804165250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:50.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.067+0000 7fe80bdc0640 1 --2- 172.21.15.98:0/82181556 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe80406d880 0x7fe8041303c0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:50.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.067+0000 7fe80bdc0640 1 -- 172.21.15.98:0/82181556 >> 172.21.15.98:0/82181556 conn(0x7fe8040b2010 msgr2=0x7fe8040b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:50.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.071+0000 7fe80bdc0640 1 -- 172.21.15.98:0/82181556 shutdown_connections 2024-09-19T08:45:50.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.071+0000 7fe80bdc0640 1 -- 172.21.15.98:0/82181556 wait complete. 2024-09-19T08:45:50.085 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:45:50.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.203+0000 7fbc1d65b640 1 Processor -- start 2024-09-19T08:45:50.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.203+0000 7fbc1d65b640 1 -- start start 2024-09-19T08:45:50.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.203+0000 7fbc1d65b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc18158fd0 0x7fbc180b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:50.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.203+0000 7fbc1d65b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbc180b4360 0x7fbc1806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:50.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.203+0000 7fbc1d65b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbc1806c1d0 0x7fbc1815efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:50.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.203+0000 7fbc1d65b640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fbc1806c620 con 0x7fbc1806c1d0 2024-09-19T08:45:50.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.203+0000 7fbc1d65b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fbc181593b0 con 0x7fbc180b4360 2024-09-19T08:45:50.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.203+0000 7fbc1d65b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbc180ccc60 con 0x7fbc18158fd0 2024-09-19T08:45:50.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.207+0000 7fbc177fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbc1806c1d0 0x7fbc1815efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:50.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.207+0000 7fbc177fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbc1806c1d0 0x7fbc1815efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53864/0 (socket says 172.21.15.98:53864) 2024-09-19T08:45:50.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.207+0000 7fbc177fe640 1 -- 172.21.15.98:0/2410373719 learned_addr learned my addr 172.21.15.98:0/2410373719 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:50.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.207+0000 7fbc16ffd640 1 --2- 172.21.15.98:0/2410373719 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc18158fd0 0x7fbc180b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:50.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.207+0000 7fbc0ffff640 1 --2- 172.21.15.98:0/2410373719 >> v2:172.21.15.139:3300/0 conn(0x7fbc180b4360 0x7fbc1806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:50.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.207+0000 7fbc177fe640 1 -- 172.21.15.98:0/2410373719 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc18158fd0 msgr2=0x7fbc180b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:50.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.207+0000 7fbc177fe640 1 --2- 172.21.15.98:0/2410373719 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc18158fd0 0x7fbc180b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:50.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.207+0000 7fbc177fe640 1 -- 172.21.15.98:0/2410373719 >> v2:172.21.15.139:3300/0 conn(0x7fbc180b4360 msgr2=0x7fbc1806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:50.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.207+0000 7fbc177fe640 1 --2- 172.21.15.98:0/2410373719 >> v2:172.21.15.139:3300/0 conn(0x7fbc180b4360 0x7fbc1806bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:50.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.207+0000 7fbc177fe640 1 -- 172.21.15.98:0/2410373719 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc180ccf40 con 0x7fbc1806c1d0 2024-09-19T08:45:50.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.207+0000 7fbc16ffd640 1 --2- 172.21.15.98:0/2410373719 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc18158fd0 0x7fbc180b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:50.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.207+0000 7fbc177fe640 1 --2- 172.21.15.98:0/2410373719 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc1806c1d0 0x7fbc1815efc0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fbc1806efc0 tx=0x7fbc1001c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd1433867e7a35 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:50.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.207+0000 7fbc167fc640 1 -- 172.21.15.98:0/2410373719 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc100a6500 con 0x7fbc1806c1d0 2024-09-19T08:45:50.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.207+0000 7fbc167fc640 1 -- 172.21.15.98:0/2410373719 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc100a66a0 con 0x7fbc1806c1d0 2024-09-19T08:45:50.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.207+0000 7fbc167fc640 1 -- 172.21.15.98:0/2410373719 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc100a6a70 con 0x7fbc1806c1d0 2024-09-19T08:45:50.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.207+0000 7fbc1d65b640 1 -- 172.21.15.98:0/2410373719 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc1806c1d0 msgr2=0x7fbc1815efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:50.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.207+0000 7fbc1d65b640 1 --2- 172.21.15.98:0/2410373719 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc1806c1d0 0x7fbc1815efc0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fbc1806efc0 tx=0x7fbc1001c670 comp rx=0 tx=0).stop 2024-09-19T08:45:50.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.211+0000 7fbc1d65b640 1 -- 172.21.15.98:0/2410373719 shutdown_connections 2024-09-19T08:45:50.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.211+0000 7fbc1d65b640 1 --2- 172.21.15.98:0/2410373719 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc1806c1d0 0x7fbc1815efc0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:50.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.211+0000 7fbc1d65b640 1 --2- 172.21.15.98:0/2410373719 >> v2:172.21.15.139:3300/0 conn(0x7fbc180b4360 0x7fbc1806bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:50.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.211+0000 7fbc1d65b640 1 --2- 172.21.15.98:0/2410373719 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc18158fd0 0x7fbc180b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:50.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.211+0000 7fbc1d65b640 1 -- 172.21.15.98:0/2410373719 >> 172.21.15.98:0/2410373719 conn(0x7fbc180b2010 msgr2=0x7fbc180b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:50.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.211+0000 7fbc1d65b640 1 -- 172.21.15.98:0/2410373719 shutdown_connections 2024-09-19T08:45:50.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.211+0000 7fbc1d65b640 1 -- 172.21.15.98:0/2410373719 wait complete. 2024-09-19T08:45:50.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.211+0000 7fbc1d65b640 1 Processor -- start 2024-09-19T08:45:50.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.211+0000 7fbc1d65b640 1 -- start start 2024-09-19T08:45:50.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.211+0000 7fbc1d65b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc1806c1d0 0x7fbc18153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:50.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.211+0000 7fbc1d65b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc180b4360 0x7fbc180732e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:50.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.211+0000 7fbc1d65b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc18158fd0 0x7fbc18073820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:50.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.211+0000 7fbc1d65b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1815c920 con 0x7fbc18158fd0 2024-09-19T08:45:50.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.211+0000 7fbc1d65b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1815c7a0 con 0x7fbc1806c1d0 2024-09-19T08:45:50.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.211+0000 7fbc1d65b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbc180ce840 con 0x7fbc180b4360 2024-09-19T08:45:50.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.211+0000 7fbc177fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc18158fd0 0x7fbc18073820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:50.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.211+0000 7fbc177fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc18158fd0 0x7fbc18073820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53874/0 (socket says 172.21.15.98:53874) 2024-09-19T08:45:50.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.211+0000 7fbc177fe640 1 -- 172.21.15.98:0/523506992 learned_addr learned my addr 172.21.15.98:0/523506992 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:50.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.211+0000 7fbc0ffff640 1 --2- 172.21.15.98:0/523506992 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc180b4360 0x7fbc180732e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:50.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.211+0000 7fbc16ffd640 1 --2- 172.21.15.98:0/523506992 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc1806c1d0 0x7fbc18153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:50.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.211+0000 7fbc177fe640 1 -- 172.21.15.98:0/523506992 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc180b4360 msgr2=0x7fbc180732e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:50.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.211+0000 7fbc177fe640 1 --2- 172.21.15.98:0/523506992 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc180b4360 0x7fbc180732e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:50.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.211+0000 7fbc177fe640 1 -- 172.21.15.98:0/523506992 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc1806c1d0 msgr2=0x7fbc18153f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:50.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.211+0000 7fbc177fe640 1 --2- 172.21.15.98:0/523506992 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc1806c1d0 0x7fbc18153f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:50.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.211+0000 7fbc177fe640 1 -- 172.21.15.98:0/523506992 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc18163d30 con 0x7fbc18158fd0 2024-09-19T08:45:50.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.211+0000 7fbc0ffff640 1 --2- 172.21.15.98:0/523506992 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc180b4360 0x7fbc180732e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:50.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.211+0000 7fbc177fe640 1 --2- 172.21.15.98:0/523506992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc18158fd0 0x7fbc18073820 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fbc1001ad20 tx=0x7fbc10019410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:50.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.211+0000 7fbc0f7fe640 1 -- 172.21.15.98:0/523506992 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc100bf070 con 0x7fbc18158fd0 2024-09-19T08:45:50.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.211+0000 7fbc0f7fe640 1 -- 172.21.15.98:0/523506992 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc100a6850 con 0x7fbc18158fd0 2024-09-19T08:45:50.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.211+0000 7fbc0f7fe640 1 -- 172.21.15.98:0/523506992 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc100a6c40 con 0x7fbc18158fd0 2024-09-19T08:45:50.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.211+0000 7fbc1d65b640 1 -- 172.21.15.98:0/523506992 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc18164010 con 0x7fbc18158fd0 2024-09-19T08:45:50.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.211+0000 7fbc1d65b640 1 -- 172.21.15.98:0/523506992 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc181645d0 con 0x7fbc18158fd0 2024-09-19T08:45:50.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.211+0000 7fbc1d65b640 1 -- 172.21.15.98:0/523506992 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc1806bc90 con 0x7fbc18158fd0 2024-09-19T08:45:50.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.215+0000 7fbc0f7fe640 1 -- 172.21.15.98:0/523506992 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fbc100c2070 con 0x7fbc18158fd0 2024-09-19T08:45:50.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.215+0000 7fbc0f7fe640 1 --2- 172.21.15.98:0/523506992 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbbf8067f20 0x7fbbf806a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:50.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.215+0000 7fbc0f7fe640 1 -- 172.21.15.98:0/523506992 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7fbc101205d0 con 0x7fbc18158fd0 2024-09-19T08:45:50.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.215+0000 7fbc16ffd640 1 --2- 172.21.15.98:0/523506992 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbbf8067f20 0x7fbbf806a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:50.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.215+0000 7fbc16ffd640 1 --2- 172.21.15.98:0/523506992 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbbf8067f20 0x7fbbf806a3e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbc18165370 tx=0x7fbc08008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:50.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.219+0000 7fbc0f7fe640 1 -- 172.21.15.98:0/523506992 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fbc100165e0 con 0x7fbc18158fd0 2024-09-19T08:45:50.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.371+0000 7fbc1d65b640 1 -- 172.21.15.98:0/523506992 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc180757a0 con 0x7fbc18158fd0 2024-09-19T08:45:50.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.371+0000 7fbc0f7fe640 1 -- 172.21.15.98:0/523506992 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) ==== 74+0+16783 (secure 0 0 0) 0x7fbc100f06a0 con 0x7fbc18158fd0 2024-09-19T08:45:50.370 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:45:50.370 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":135,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:45:29.773038+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:45:25.813513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-19T08:45:27.137275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:45:50.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.375+0000 7fbc1d65b640 1 -- 172.21.15.98:0/523506992 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbbf8067f20 msgr2=0x7fbbf806a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:50.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.375+0000 7fbc1d65b640 1 --2- 172.21.15.98:0/523506992 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbbf8067f20 0x7fbbf806a3e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbc18165370 tx=0x7fbc08008040 comp rx=0 tx=0).stop 2024-09-19T08:45:50.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.375+0000 7fbc1d65b640 1 -- 172.21.15.98:0/523506992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc18158fd0 msgr2=0x7fbc18073820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:50.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.375+0000 7fbc1d65b640 1 --2- 172.21.15.98:0/523506992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc18158fd0 0x7fbc18073820 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fbc1001ad20 tx=0x7fbc10019410 comp rx=0 tx=0).stop 2024-09-19T08:45:50.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.375+0000 7fbc1d65b640 1 -- 172.21.15.98:0/523506992 shutdown_connections 2024-09-19T08:45:50.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.375+0000 7fbc1d65b640 1 --2- 172.21.15.98:0/523506992 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbbf8067f20 0x7fbbf806a3e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:50.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.375+0000 7fbc1d65b640 1 --2- 172.21.15.98:0/523506992 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbc18158fd0 0x7fbc18073820 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:50.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.375+0000 7fbc1d65b640 1 --2- 172.21.15.98:0/523506992 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbc180b4360 0x7fbc180732e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:50.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.375+0000 7fbc1d65b640 1 --2- 172.21.15.98:0/523506992 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc1806c1d0 0x7fbc18153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:50.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.375+0000 7fbc1d65b640 1 -- 172.21.15.98:0/523506992 >> 172.21.15.98:0/523506992 conn(0x7fbc180b2010 msgr2=0x7fbc1815eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:50.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.375+0000 7fbc1d65b640 1 -- 172.21.15.98:0/523506992 shutdown_connections 2024-09-19T08:45:50.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.375+0000 7fbc1d65b640 1 -- 172.21.15.98:0/523506992 wait complete. 2024-09-19T08:45:50.389 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T08:45:50.389 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:45:50.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.507+0000 7f5dde3fa640 1 Processor -- start 2024-09-19T08:45:50.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.511+0000 7f5dde3fa640 1 -- start start 2024-09-19T08:45:50.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.511+0000 7f5dde3fa640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5dd8140d30 0x7f5dd806d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:50.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.511+0000 7f5dde3fa640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5dd80b3a20 0x7f5dd806d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:50.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.511+0000 7f5dde3fa640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dd815ea10 con 0x7f5dd8141110 2024-09-19T08:45:50.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.511+0000 7f5dde3fa640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f5dd815e7c0 con 0x7f5dd8140d30 2024-09-19T08:45:50.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.511+0000 7f5dde3fa640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5dd815ecf0 con 0x7f5dd80b3a20 2024-09-19T08:45:50.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.511+0000 7f5dd7fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5dd8140d30 0x7f5dd806d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:50.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.511+0000 7f5dd7fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5dd8140d30 0x7f5dd806d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53890/0 (socket says 172.21.15.98:53890) 2024-09-19T08:45:50.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.511+0000 7f5dd7fff640 1 -- 172.21.15.98:0/1715745727 learned_addr learned my addr 172.21.15.98:0/1715745727 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:50.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.511+0000 7f5dd7fff640 1 -- 172.21.15.98:0/1715745727 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5dd80b3a20 msgr2=0x7f5dd806d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:50.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.511+0000 7f5dd7fff640 1 --2- 172.21.15.98:0/1715745727 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5dd80b3a20 0x7f5dd806d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:50.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.511+0000 7f5dd7fff640 1 -- 172.21.15.98:0/1715745727 >> v1:172.21.15.139:6789/0 conn(0x7f5dd8141110 legacy=0x7f5dd80b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:50.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.511+0000 7f5dd7fff640 1 -- 172.21.15.98:0/1715745727 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dd815efd0 con 0x7f5dd8140d30 2024-09-19T08:45:50.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.511+0000 7f5dd7fff640 1 --2- 172.21.15.98:0/1715745727 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5dd8140d30 0x7f5dd806d170 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f5dd80d1370 tx=0x7f5dc801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e97693772b1079 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:50.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.515+0000 7f5dd77fe640 1 -- 172.21.15.98:0/1715745727 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc80ab650 con 0x7f5dd8140d30 2024-09-19T08:45:50.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.515+0000 7f5dd77fe640 1 -- 172.21.15.98:0/1715745727 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dc80b0070 con 0x7f5dd8140d30 2024-09-19T08:45:50.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.515+0000 7f5dd77fe640 1 -- 172.21.15.98:0/1715745727 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc80aba10 con 0x7f5dd8140d30 2024-09-19T08:45:50.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.515+0000 7f5dde3fa640 1 -- 172.21.15.98:0/1715745727 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5dd8140d30 msgr2=0x7f5dd806d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:50.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.515+0000 7f5dde3fa640 1 --2- 172.21.15.98:0/1715745727 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5dd8140d30 0x7f5dd806d170 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f5dd80d1370 tx=0x7f5dc801c470 comp rx=0 tx=0).stop 2024-09-19T08:45:50.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.515+0000 7f5dde3fa640 1 -- 172.21.15.98:0/1715745727 shutdown_connections 2024-09-19T08:45:50.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.515+0000 7f5dde3fa640 1 --2- 172.21.15.98:0/1715745727 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5dd80b3a20 0x7f5dd806d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:50.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.515+0000 7f5dde3fa640 1 --2- 172.21.15.98:0/1715745727 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5dd8140d30 0x7f5dd806d170 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:50.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.515+0000 7f5dde3fa640 1 -- 172.21.15.98:0/1715745727 >> 172.21.15.98:0/1715745727 conn(0x7f5dd80b2010 msgr2=0x7f5dd80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:50.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.515+0000 7f5dde3fa640 1 -- 172.21.15.98:0/1715745727 shutdown_connections 2024-09-19T08:45:50.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.515+0000 7f5dde3fa640 1 -- 172.21.15.98:0/1715745727 wait complete. 2024-09-19T08:45:50.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.515+0000 7f5dde3fa640 1 Processor -- start 2024-09-19T08:45:50.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.515+0000 7f5dde3fa640 1 -- start start 2024-09-19T08:45:50.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.515+0000 7f5dde3fa640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5dd80b3a20 0x7f5dd8155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:50.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.515+0000 7f5dde3fa640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5dd8140d30 0x7f5dd81faf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:50.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.515+0000 7f5dde3fa640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5dd8141110 0x7f5dd80aa580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:50.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.515+0000 7f5dde3fa640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd815bdd0 con 0x7f5dd8140d30 2024-09-19T08:45:50.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.515+0000 7f5dde3fa640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd815bc50 con 0x7f5dd8141110 2024-09-19T08:45:50.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.515+0000 7f5dde3fa640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5dd815bf50 con 0x7f5dd80b3a20 2024-09-19T08:45:50.516 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.515+0000 7f5dcffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5dd8140d30 0x7f5dd81faf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:50.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.515+0000 7f5dcffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5dd8140d30 0x7f5dd81faf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53898/0 (socket says 172.21.15.98:53898) 2024-09-19T08:45:50.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.515+0000 7f5dcffff640 1 -- 172.21.15.98:0/592946026 learned_addr learned my addr 172.21.15.98:0/592946026 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:50.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.515+0000 7f5dd7fff640 1 --2- 172.21.15.98:0/592946026 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5dd80b3a20 0x7f5dd8155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:50.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.515+0000 7f5ddc997640 1 --2- 172.21.15.98:0/592946026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5dd8141110 0x7f5dd80aa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:50.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.515+0000 7f5dcffff640 1 -- 172.21.15.98:0/592946026 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5dd80b3a20 msgr2=0x7f5dd8155110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:50.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.515+0000 7f5dcffff640 1 --2- 172.21.15.98:0/592946026 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5dd80b3a20 0x7f5dd8155110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:50.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.515+0000 7f5dcffff640 1 -- 172.21.15.98:0/592946026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5dd8141110 msgr2=0x7f5dd80aa580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:50.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.515+0000 7f5dcffff640 1 --2- 172.21.15.98:0/592946026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5dd8141110 0x7f5dd80aa580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:50.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.515+0000 7f5dcffff640 1 -- 172.21.15.98:0/592946026 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dd806e0d0 con 0x7f5dd8140d30 2024-09-19T08:45:50.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.515+0000 7f5dd7fff640 1 --2- 172.21.15.98:0/592946026 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5dd80b3a20 0x7f5dd8155110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:50.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.515+0000 7f5dcffff640 1 --2- 172.21.15.98:0/592946026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5dd8140d30 0x7f5dd81faf50 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f5dc000c440 tx=0x7f5dc000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:50.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.515+0000 7f5dd57fa640 1 -- 172.21.15.98:0/592946026 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0016020 con 0x7f5dd8140d30 2024-09-19T08:45:50.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.515+0000 7f5dd57fa640 1 -- 172.21.15.98:0/592946026 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dc0012830 con 0x7f5dd8140d30 2024-09-19T08:45:50.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.515+0000 7f5dde3fa640 1 -- 172.21.15.98:0/592946026 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dd8155730 con 0x7f5dd8140d30 2024-09-19T08:45:50.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.515+0000 7f5dd57fa640 1 -- 172.21.15.98:0/592946026 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0012c20 con 0x7f5dd8140d30 2024-09-19T08:45:50.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.515+0000 7f5dde3fa640 1 -- 172.21.15.98:0/592946026 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dd8155cf0 con 0x7f5dd8140d30 2024-09-19T08:45:50.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.519+0000 7f5dde3fa640 1 -- 172.21.15.98:0/592946026 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dd80b3310 con 0x7f5dd8140d30 2024-09-19T08:45:50.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.519+0000 7f5dd57fa640 1 -- 172.21.15.98:0/592946026 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f5dc0019020 con 0x7f5dd8140d30 2024-09-19T08:45:50.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.519+0000 7f5dd57fa640 1 --2- 172.21.15.98:0/592946026 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5db8067f20 0x7f5db806a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:50.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.519+0000 7f5dd57fa640 1 -- 172.21.15.98:0/592946026 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7f5dc0086480 con 0x7f5dd8140d30 2024-09-19T08:45:50.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.519+0000 7f5dd7fff640 1 --2- 172.21.15.98:0/592946026 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5db8067f20 0x7f5db806a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:50.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.519+0000 7f5dd7fff640 1 --2- 172.21.15.98:0/592946026 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5db8067f20 0x7f5db806a3e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5dc80af000 tx=0x7f5dc8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:50.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.523+0000 7f5dd57fa640 1 -- 172.21.15.98:0/592946026 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f5dc0020030 con 0x7f5dd8140d30 2024-09-19T08:45:50.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.723+0000 7f5dde3fa640 1 -- 172.21.15.98:0/592946026 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5dd806c490 con 0x7f5dd8140d30 2024-09-19T08:45:50.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.723+0000 7f5dd57fa640 1 -- 172.21.15.98:0/592946026 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5230 (secure 0 0 0) 0x7f5dc00182c0 con 0x7f5dd8140d30 2024-09-19T08:45:50.722 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:45:50.722 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":75,"btime":"2024-09-19T08:45:39:977951+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/1907681163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":1907681163},{"type":"v1","addr":"172.21.15.139:6834","nonce":1907681163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7896,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2481662340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2481662340},{"type":"v1","addr":"172.21.15.98:6833","nonce":2481662340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7902,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/286818221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":286818221},{"type":"v1","addr":"172.21.15.98:6835","nonce":286818221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:45:28.140764+0000","modified":"2024-09-19T08:45:30.144582+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7914},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7914":{"gid":7914,"name":"b","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6835/4255670104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6833","nonce":4255670104},{"type":"v1","addr":"172.21.15.139:6835","nonce":4255670104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7914,"qdb_cluster":[7914]},"id":10}]} 2024-09-19T08:45:50.722 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 75 2024-09-19T08:45:50.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.723+0000 7f5dde3fa640 1 -- 172.21.15.98:0/592946026 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5db8067f20 msgr2=0x7f5db806a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:50.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.727+0000 7f5dde3fa640 1 --2- 172.21.15.98:0/592946026 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5db8067f20 0x7f5db806a3e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5dc80af000 tx=0x7f5dc8008040 comp rx=0 tx=0).stop 2024-09-19T08:45:50.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.727+0000 7f5dde3fa640 1 -- 172.21.15.98:0/592946026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5dd8140d30 msgr2=0x7f5dd81faf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:50.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.727+0000 7f5dde3fa640 1 --2- 172.21.15.98:0/592946026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5dd8140d30 0x7f5dd81faf50 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f5dc000c440 tx=0x7f5dc000c900 comp rx=0 tx=0).stop 2024-09-19T08:45:50.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.727+0000 7f5dde3fa640 1 -- 172.21.15.98:0/592946026 shutdown_connections 2024-09-19T08:45:50.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.727+0000 7f5dde3fa640 1 --2- 172.21.15.98:0/592946026 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5db8067f20 0x7f5db806a3e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:50.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.727+0000 7f5dde3fa640 1 --2- 172.21.15.98:0/592946026 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5dd8141110 0x7f5dd80aa580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:50.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.727+0000 7f5dde3fa640 1 --2- 172.21.15.98:0/592946026 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5dd8140d30 0x7f5dd81faf50 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:50.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.727+0000 7f5dde3fa640 1 --2- 172.21.15.98:0/592946026 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5dd80b3a20 0x7f5dd8155110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:50.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.727+0000 7f5dde3fa640 1 -- 172.21.15.98:0/592946026 >> 172.21.15.98:0/592946026 conn(0x7f5dd80b2010 msgr2=0x7f5dd80ce440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:50.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.727+0000 7f5dde3fa640 1 -- 172.21.15.98:0/592946026 shutdown_connections 2024-09-19T08:45:50.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.727+0000 7f5dde3fa640 1 -- 172.21.15.98:0/592946026 wait complete. 2024-09-19T08:45:50.741 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:45:50.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.863+0000 7f49613eb640 1 Processor -- start 2024-09-19T08:45:50.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.863+0000 7f49613eb640 1 -- start start 2024-09-19T08:45:50.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.863+0000 7f49613eb640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f495c0b7f80 0x7f495c15f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:50.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.863+0000 7f49613eb640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f495c0d4820 0x7f495c1621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:50.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.863+0000 7f49613eb640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f495c068440 con 0x7f495c0b9970 2024-09-19T08:45:50.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.863+0000 7f49613eb640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f495c163050 con 0x7f495c0d4820 2024-09-19T08:45:50.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.863+0000 7f49613eb640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f495c162a90 con 0x7f495c0b7f80 2024-09-19T08:45:50.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.863+0000 7f495a7fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f495c0d4820 0x7f495c1621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:50.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.863+0000 7f495a7fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f495c0d4820 0x7f495c1621b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53910/0 (socket says 172.21.15.98:53910) 2024-09-19T08:45:50.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.863+0000 7f495a7fc640 1 -- 172.21.15.98:0/1861964119 learned_addr learned my addr 172.21.15.98:0/1861964119 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:50.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.863+0000 7f495affd640 1 --2- 172.21.15.98:0/1861964119 >> v2:172.21.15.139:3300/0 conn(0x7f495c0b7f80 0x7f495c15f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:50.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.863+0000 7f495a7fc640 1 -- 172.21.15.98:0/1861964119 >> v2:172.21.15.139:3300/0 conn(0x7f495c0b7f80 msgr2=0x7f495c15f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:50.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.863+0000 7f495a7fc640 1 --2- 172.21.15.98:0/1861964119 >> v2:172.21.15.139:3300/0 conn(0x7f495c0b7f80 0x7f495c15f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:50.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.863+0000 7f495a7fc640 1 -- 172.21.15.98:0/1861964119 >> v1:172.21.15.98:6789/0 conn(0x7f495c0b9970 legacy=0x7f495c0b9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:50.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.863+0000 7f495a7fc640 1 -- 172.21.15.98:0/1861964119 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495c162d70 con 0x7f495c0d4820 2024-09-19T08:45:50.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.863+0000 7f495affd640 1 --2- 172.21.15.98:0/1861964119 >> v2:172.21.15.139:3300/0 conn(0x7f495c0b7f80 0x7f495c15f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:50.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.867+0000 7f495a7fc640 1 --2- 172.21.15.98:0/1861964119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f495c0d4820 0x7f495c1621b0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f495c0d0dd0 tx=0x7f494c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2930b9abd7524b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:50.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.867+0000 7f4959ffb640 1 -- 172.21.15.98:0/1861964119 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494c0b2070 con 0x7f495c0d4820 2024-09-19T08:45:50.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.867+0000 7f4959ffb640 1 -- 172.21.15.98:0/1861964119 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f494c0a9070 con 0x7f495c0d4820 2024-09-19T08:45:50.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.867+0000 7f4959ffb640 1 -- 172.21.15.98:0/1861964119 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494c0ad880 con 0x7f495c0d4820 2024-09-19T08:45:50.865 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.867+0000 7f49613eb640 1 -- 172.21.15.98:0/1861964119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f495c0d4820 msgr2=0x7f495c1621b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:50.865 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.867+0000 7f49613eb640 1 --2- 172.21.15.98:0/1861964119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f495c0d4820 0x7f495c1621b0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f495c0d0dd0 tx=0x7f494c01c490 comp rx=0 tx=0).stop 2024-09-19T08:45:50.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.867+0000 7f49613eb640 1 -- 172.21.15.98:0/1861964119 shutdown_connections 2024-09-19T08:45:50.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.867+0000 7f49613eb640 1 --2- 172.21.15.98:0/1861964119 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f495c0d4820 0x7f495c1621b0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:50.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.867+0000 7f49613eb640 1 --2- 172.21.15.98:0/1861964119 >> v2:172.21.15.139:3300/0 conn(0x7f495c0b7f80 0x7f495c15f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:50.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.867+0000 7f49613eb640 1 -- 172.21.15.98:0/1861964119 >> 172.21.15.98:0/1861964119 conn(0x7f495c074e40 msgr2=0x7f495c075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:50.866 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.867+0000 7f49613eb640 1 -- 172.21.15.98:0/1861964119 shutdown_connections 2024-09-19T08:45:50.867 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.867+0000 7f49613eb640 1 -- 172.21.15.98:0/1861964119 wait complete. 2024-09-19T08:45:50.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.867+0000 7f49613eb640 1 Processor -- start 2024-09-19T08:45:50.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.867+0000 7f49613eb640 1 -- start start 2024-09-19T08:45:50.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.867+0000 7f49613eb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f495c0b7f80 0x7f495c13af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:50.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.867+0000 7f49613eb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f495c0b9970 0x7f495c13e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:50.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.867+0000 7f49613eb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f495c0d4820 0x7f495c1429a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:50.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.867+0000 7f49613eb640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f495c0b84e0 con 0x7f495c0b9970 2024-09-19T08:45:50.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.867+0000 7f49613eb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f495c0b8360 con 0x7f495c0b7f80 2024-09-19T08:45:50.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.867+0000 7f49613eb640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f495c144590 con 0x7f495c0d4820 2024-09-19T08:45:50.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.867+0000 7f495a7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f495c0b9970 0x7f495c13e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:50.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.867+0000 7f495a7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f495c0b9970 0x7f495c13e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53912/0 (socket says 172.21.15.98:53912) 2024-09-19T08:45:50.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.867+0000 7f495a7fc640 1 -- 172.21.15.98:0/4251532071 learned_addr learned my addr 172.21.15.98:0/4251532071 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:50.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.867+0000 7f495affd640 1 --2- 172.21.15.98:0/4251532071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f495c0b7f80 0x7f495c13af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:50.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.867+0000 7f495a7fc640 1 -- 172.21.15.98:0/4251532071 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f495c0d4820 msgr2=0x7f495c1429a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:50.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.867+0000 7f495a7fc640 1 --2- 172.21.15.98:0/4251532071 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f495c0d4820 0x7f495c1429a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:50.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.867+0000 7f495a7fc640 1 -- 172.21.15.98:0/4251532071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f495c0b7f80 msgr2=0x7f495c13af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:50.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.867+0000 7f495a7fc640 1 --2- 172.21.15.98:0/4251532071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f495c0b7f80 0x7f495c13af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:50.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.867+0000 7f495a7fc640 1 -- 172.21.15.98:0/4251532071 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495c13cef0 con 0x7f495c0b9970 2024-09-19T08:45:50.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.867+0000 7f495a7fc640 1 --2- 172.21.15.98:0/4251532071 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f495c0b9970 0x7f495c13e5f0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f494c000c10 tx=0x7f494c019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:50.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.867+0000 7f49437fe640 1 -- 172.21.15.98:0/4251532071 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494c0b2040 con 0x7f495c0b9970 2024-09-19T08:45:50.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.871+0000 7f49437fe640 1 -- 172.21.15.98:0/4251532071 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f494c0bb070 con 0x7f495c0b9970 2024-09-19T08:45:50.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.871+0000 7f49613eb640 1 -- 172.21.15.98:0/4251532071 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f495c1fafb0 con 0x7f495c0b9970 2024-09-19T08:45:50.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.871+0000 7f49437fe640 1 -- 172.21.15.98:0/4251532071 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494c0b6870 con 0x7f495c0b9970 2024-09-19T08:45:50.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.871+0000 7f49613eb640 1 -- 172.21.15.98:0/4251532071 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f495c1fb570 con 0x7f495c0b9970 2024-09-19T08:45:50.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.871+0000 7f49613eb640 1 -- 172.21.15.98:0/4251532071 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f495c068f50 con 0x7f495c0b9970 2024-09-19T08:45:50.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.871+0000 7f49437fe640 1 -- 172.21.15.98:0/4251532071 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f494c0ab070 con 0x7f495c0b9970 2024-09-19T08:45:50.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.871+0000 7f49437fe640 1 --2- 172.21.15.98:0/4251532071 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4948067ef0 0x7f494806a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:50.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.871+0000 7f49437fe640 1 -- 172.21.15.98:0/4251532071 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7f494c120660 con 0x7f495c0b9970 2024-09-19T08:45:50.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.871+0000 7f495affd640 1 --2- 172.21.15.98:0/4251532071 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4948067ef0 0x7f494806a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:50.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.871+0000 7f495affd640 1 --2- 172.21.15.98:0/4251532071 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4948067ef0 0x7f494806a3b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4950006fd0 tx=0x7f4950008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:50.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:50.875+0000 7f49437fe640 1 -- 172.21.15.98:0/4251532071 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f494c016110 con 0x7f495c0b9970 2024-09-19T08:45:51.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.059+0000 7f49613eb640 1 -- 172.21.15.98:0/4251532071 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f495c0d0900 con 0x7f495c0b9970 2024-09-19T08:45:51.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.059+0000 7f49437fe640 1 -- 172.21.15.98:0/4251532071 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5230 (secure 0 0 0) 0x7f494c0f0830 con 0x7f495c0b9970 2024-09-19T08:45:51.057 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:45:51.057 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":75,"btime":"2024-09-19T08:45:39:977951+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/1907681163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":1907681163},{"type":"v1","addr":"172.21.15.139:6834","nonce":1907681163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7896,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2481662340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2481662340},{"type":"v1","addr":"172.21.15.98:6833","nonce":2481662340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7902,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/286818221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":286818221},{"type":"v1","addr":"172.21.15.98:6835","nonce":286818221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:45:28.140764+0000","modified":"2024-09-19T08:45:30.144582+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7914},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7914":{"gid":7914,"name":"b","rank":0,"incarnation":72,"state":"up:active","state_seq":3,"addr":"172.21.15.139:6835/4255670104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6833","nonce":4255670104},{"type":"v1","addr":"172.21.15.139:6835","nonce":4255670104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7914,"qdb_cluster":[7914]},"id":10}]} 2024-09-19T08:45:51.057 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 75 2024-09-19T08:45:51.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.059+0000 7f49613eb640 1 -- 172.21.15.98:0/4251532071 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4948067ef0 msgr2=0x7f494806a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:51.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.059+0000 7f49613eb640 1 --2- 172.21.15.98:0/4251532071 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4948067ef0 0x7f494806a3b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4950006fd0 tx=0x7f4950008040 comp rx=0 tx=0).stop 2024-09-19T08:45:51.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.059+0000 7f49613eb640 1 -- 172.21.15.98:0/4251532071 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f495c0b9970 msgr2=0x7f495c13e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:51.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.059+0000 7f49613eb640 1 --2- 172.21.15.98:0/4251532071 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f495c0b9970 0x7f495c13e5f0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f494c000c10 tx=0x7f494c019230 comp rx=0 tx=0).stop 2024-09-19T08:45:51.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.059+0000 7f49613eb640 1 -- 172.21.15.98:0/4251532071 shutdown_connections 2024-09-19T08:45:51.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.059+0000 7f49613eb640 1 --2- 172.21.15.98:0/4251532071 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4948067ef0 0x7f494806a3b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.059+0000 7f49613eb640 1 --2- 172.21.15.98:0/4251532071 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f495c0d4820 0x7f495c1429a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.059+0000 7f49613eb640 1 --2- 172.21.15.98:0/4251532071 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f495c0b9970 0x7f495c13e5f0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.059+0000 7f49613eb640 1 --2- 172.21.15.98:0/4251532071 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f495c0b7f80 0x7f495c13af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.059+0000 7f49613eb640 1 -- 172.21.15.98:0/4251532071 >> 172.21.15.98:0/4251532071 conn(0x7f495c074e40 msgr2=0x7f495c0d0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:51.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.059+0000 7f49613eb640 1 -- 172.21.15.98:0/4251532071 shutdown_connections 2024-09-19T08:45:51.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.059+0000 7f49613eb640 1 -- 172.21.15.98:0/4251532071 wait complete. 2024-09-19T08:45:51.075 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:45:51.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.191+0000 7f68947c0640 1 Processor -- start 2024-09-19T08:45:51.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.195+0000 7f68947c0640 1 -- start start 2024-09-19T08:45:51.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.195+0000 7f68947c0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f688c06bc90 0x7f688c06c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:51.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.195+0000 7f68947c0640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f688c0ccc60 con 0x7f688c158fd0 2024-09-19T08:45:51.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.195+0000 7f68947c0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f688c0ccf40 con 0x7f688c0ce840 2024-09-19T08:45:51.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.195+0000 7f68947c0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f688c1593b0 con 0x7f688c06bc90 2024-09-19T08:45:51.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.195+0000 7f689255c640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f688c158fd0 0x7f688c0b3e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42462/0 (socket says 172.21.15.98:42462) 2024-09-19T08:45:51.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.195+0000 7f689255c640 1 -- 172.21.15.98:0/3424660156 learned_addr learned my addr 172.21.15.98:0/3424660156 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:51.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.195+0000 7f6891d5b640 1 --2- 172.21.15.98:0/3424660156 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f688c06bc90 0x7f688c06c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:51.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.195+0000 7f689155a640 1 -- 172.21.15.98:0/3424660156 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3796018183 0 0) 0x7f688c0ccc60 con 0x7f688c158fd0 2024-09-19T08:45:51.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.195+0000 7f689155a640 1 -- 172.21.15.98:0/3424660156 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6870005740 con 0x7f688c158fd0 2024-09-19T08:45:51.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.195+0000 7f689155a640 1 -- 172.21.15.98:0/3424660156 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 274651143 0 0) 0x7f6870005740 con 0x7f688c158fd0 2024-09-19T08:45:51.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.195+0000 7f689155a640 1 -- 172.21.15.98:0/3424660156 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f688c06bc90 msgr2=0x7f688c06c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:51.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.195+0000 7f689155a640 1 --2- 172.21.15.98:0/3424660156 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f688c06bc90 0x7f688c06c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.195+0000 7f689155a640 1 -- 172.21.15.98:0/3424660156 >> v1:172.21.15.139:6789/0 conn(0x7f688c0ce840 legacy=0x7f688c0cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:51.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.195+0000 7f689155a640 1 -- 172.21.15.98:0/3424660156 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f688c0cd220 con 0x7f688c158fd0 2024-09-19T08:45:51.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.195+0000 7f689155a640 1 -- 172.21.15.98:0/3424660156 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f687c001680 con 0x7f688c158fd0 2024-09-19T08:45:51.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.195+0000 7f689155a640 1 -- 172.21.15.98:0/3424660156 >> v1:172.21.15.98:6789/0 conn(0x7f688c158fd0 legacy=0x7f688c0b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:51.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.195+0000 7f689155a640 1 --2- 172.21.15.98:0/3424660156 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6870005c00 0x7f6870006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:51.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.195+0000 7f689155a640 1 --2- 172.21.15.98:0/3424660156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6870006a20 0x7f6870006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:51.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.195+0000 7f689155a640 1 --2- 172.21.15.98:0/3424660156 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6870007500 0x7f687000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:51.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.195+0000 7f689155a640 1 -- 172.21.15.98:0/3424660156 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f688c1593b0 con 0x7f6870006a20 2024-09-19T08:45:51.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.195+0000 7f689155a640 1 -- 172.21.15.98:0/3424660156 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f687000e330 con 0x7f6870005c00 2024-09-19T08:45:51.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.195+0000 7f689155a640 1 -- 172.21.15.98:0/3424660156 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f687000e530 con 0x7f6870007500 2024-09-19T08:45:51.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.195+0000 7f6891d5b640 1 --2- 172.21.15.98:0/3424660156 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f688c06bc90 0x7f688c06c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:51.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.195+0000 7f6891d5b640 1 --2- 172.21.15.98:0/3424660156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6870006a20 0x7f6870006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:51.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.195+0000 7f689255c640 1 --2- 172.21.15.98:0/3424660156 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6870005c00 0x7f6870006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:51.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.195+0000 7f6892d5d640 1 --2- 172.21.15.98:0/3424660156 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6870007500 0x7f687000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:51.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.195+0000 7f6891d5b640 1 -- 172.21.15.98:0/3424660156 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6870007500 msgr2=0x7f687000dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:51.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.195+0000 7f6891d5b640 1 --2- 172.21.15.98:0/3424660156 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6870007500 0x7f687000dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.195+0000 7f6891d5b640 1 -- 172.21.15.98:0/3424660156 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6870005c00 msgr2=0x7f6870006020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:51.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.195+0000 7f6891d5b640 1 --2- 172.21.15.98:0/3424660156 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6870005c00 0x7f6870006020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.195+0000 7f6891d5b640 1 -- 172.21.15.98:0/3424660156 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6870005740 con 0x7f6870006a20 2024-09-19T08:45:51.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.199+0000 7f6891d5b640 1 --2- 172.21.15.98:0/3424660156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6870006a20 0x7f6870006ea0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f68780096c0 tx=0x7f687801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2278e9e556a471da server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:51.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.199+0000 7f689155a640 1 -- 172.21.15.98:0/3424660156 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687800f040 con 0x7f6870006a20 2024-09-19T08:45:51.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.199+0000 7f689155a640 1 -- 172.21.15.98:0/3424660156 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68780b0070 con 0x7f6870006a20 2024-09-19T08:45:51.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.199+0000 7f689155a640 1 -- 172.21.15.98:0/3424660156 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68780ab810 con 0x7f6870006a20 2024-09-19T08:45:51.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.199+0000 7f68947c0640 1 -- 172.21.15.98:0/3424660156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6870006a20 msgr2=0x7f6870006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:51.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.199+0000 7f68947c0640 1 --2- 172.21.15.98:0/3424660156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6870006a20 0x7f6870006ea0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f68780096c0 tx=0x7f687801c3c0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.199+0000 7f6892d5d640 1 -- 172.21.15.98:0/3424660156 reap_dead start 2024-09-19T08:45:51.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.199+0000 7f68947c0640 1 -- 172.21.15.98:0/3424660156 shutdown_connections 2024-09-19T08:45:51.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.199+0000 7f68947c0640 1 -- 172.21.15.98:0/3424660156 >> 172.21.15.98:0/3424660156 conn(0x7f688c0b2010 msgr2=0x7f688c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:51.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.199+0000 7f68947c0640 1 -- 172.21.15.98:0/3424660156 shutdown_connections 2024-09-19T08:45:51.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.199+0000 7f68947c0640 1 -- 172.21.15.98:0/3424660156 wait complete. 2024-09-19T08:45:51.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.199+0000 7f68947c0640 1 Processor -- start 2024-09-19T08:45:51.199 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.199+0000 7f68947c0640 1 -- start start 2024-09-19T08:45:51.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.199+0000 7f68947c0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6870006a20 0x7f688c15d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:51.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.199+0000 7f68947c0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f688c15ed60 0x7f688c0729b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:51.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.199+0000 7f68947c0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f688c14dfb0 0x7f688c0affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:51.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.199+0000 7f68947c0640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f688c06c4b0 con 0x7f688c15ed60 2024-09-19T08:45:51.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.199+0000 7f68947c0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f688c15c7a0 con 0x7f688c14dfb0 2024-09-19T08:45:51.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.199+0000 7f68947c0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f688c15dcc0 con 0x7f6870006a20 2024-09-19T08:45:51.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.203+0000 7f6891d5b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f688c15ed60 0x7f688c0729b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:51.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.203+0000 7f6891d5b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f688c15ed60 0x7f688c0729b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53940/0 (socket says 172.21.15.98:53940) 2024-09-19T08:45:51.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.203+0000 7f6891d5b640 1 -- 172.21.15.98:0/1525553272 learned_addr learned my addr 172.21.15.98:0/1525553272 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:51.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.203+0000 7f689255c640 1 --2- 172.21.15.98:0/1525553272 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6870006a20 0x7f688c15d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:51.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.203+0000 7f6892d5d640 1 --2- 172.21.15.98:0/1525553272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f688c14dfb0 0x7f688c0affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:51.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.203+0000 7f6891d5b640 1 -- 172.21.15.98:0/1525553272 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6870006a20 msgr2=0x7f688c15d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:51.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.203+0000 7f6891d5b640 1 --2- 172.21.15.98:0/1525553272 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6870006a20 0x7f688c15d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.203+0000 7f6891d5b640 1 -- 172.21.15.98:0/1525553272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f688c14dfb0 msgr2=0x7f688c0affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:51.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.203+0000 7f6891d5b640 1 --2- 172.21.15.98:0/1525553272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f688c14dfb0 0x7f688c0affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.203+0000 7f6891d5b640 1 -- 172.21.15.98:0/1525553272 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f688c15dfa0 con 0x7f688c15ed60 2024-09-19T08:45:51.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.203+0000 7f6891d5b640 1 --2- 172.21.15.98:0/1525553272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f688c15ed60 0x7f688c0729b0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f68780ad040 tx=0x7f6870007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:51.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.203+0000 7f688affd640 1 -- 172.21.15.98:0/1525553272 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68780b5800 con 0x7f688c15ed60 2024-09-19T08:45:51.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.203+0000 7f688affd640 1 -- 172.21.15.98:0/1525553272 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68780b59a0 con 0x7f688c15ed60 2024-09-19T08:45:51.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.203+0000 7f688affd640 1 -- 172.21.15.98:0/1525553272 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68780b5d70 con 0x7f688c15ed60 2024-09-19T08:45:51.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.203+0000 7f68947c0640 1 -- 172.21.15.98:0/1525553272 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f688c15e280 con 0x7f688c15ed60 2024-09-19T08:45:51.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.203+0000 7f68947c0640 1 -- 172.21.15.98:0/1525553272 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f688c15e840 con 0x7f688c15ed60 2024-09-19T08:45:51.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.203+0000 7f68947c0640 1 -- 172.21.15.98:0/1525553272 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f688c0cec90 con 0x7f688c15ed60 2024-09-19T08:45:51.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.203+0000 7f688affd640 1 -- 172.21.15.98:0/1525553272 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f68780c93f0 con 0x7f688c15ed60 2024-09-19T08:45:51.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.203+0000 7f688affd640 1 --2- 172.21.15.98:0/1525553272 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6874067ef0 0x7f687406a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:51.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.203+0000 7f688affd640 1 -- 172.21.15.98:0/1525553272 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7f6878120b70 con 0x7f688c15ed60 2024-09-19T08:45:51.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.203+0000 7f689255c640 1 --2- 172.21.15.98:0/1525553272 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6874067ef0 0x7f687406a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:51.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.203+0000 7f689255c640 1 --2- 172.21.15.98:0/1525553272 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6874067ef0 0x7f687406a3b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f687c001cd0 tx=0x7f687c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:51.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.207+0000 7f688affd640 1 -- 172.21.15.98:0/1525553272 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6878016100 con 0x7f688c15ed60 2024-09-19T08:45:51.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.363+0000 7f68947c0640 1 -- 172.21.15.98:0/1525553272 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f688c0b3790 con 0x7f688c15ed60 2024-09-19T08:45:51.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.367+0000 7f688affd640 1 -- 172.21.15.98:0/1525553272 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) ==== 74+0+16783 (secure 0 0 0) 0x7f68780f0df0 con 0x7f688c15ed60 2024-09-19T08:45:51.365 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:45:51.366 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":135,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:45:29.773038+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:45:25.813513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-19T08:45:27.137275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:45:51.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.367+0000 7f68947c0640 1 -- 172.21.15.98:0/1525553272 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6874067ef0 msgr2=0x7f687406a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:51.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.367+0000 7f68947c0640 1 --2- 172.21.15.98:0/1525553272 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6874067ef0 0x7f687406a3b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f687c001cd0 tx=0x7f687c009040 comp rx=0 tx=0).stop 2024-09-19T08:45:51.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.367+0000 7f68947c0640 1 -- 172.21.15.98:0/1525553272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f688c15ed60 msgr2=0x7f688c0729b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:51.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.367+0000 7f68947c0640 1 --2- 172.21.15.98:0/1525553272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f688c15ed60 0x7f688c0729b0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f68780ad040 tx=0x7f6870007a50 comp rx=0 tx=0).stop 2024-09-19T08:45:51.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.367+0000 7f68947c0640 1 -- 172.21.15.98:0/1525553272 shutdown_connections 2024-09-19T08:45:51.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.367+0000 7f68947c0640 1 --2- 172.21.15.98:0/1525553272 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6874067ef0 0x7f687406a3b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.367+0000 7f68947c0640 1 --2- 172.21.15.98:0/1525553272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f688c14dfb0 0x7f688c0affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.367+0000 7f68947c0640 1 --2- 172.21.15.98:0/1525553272 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f688c15ed60 0x7f688c0729b0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.367+0000 7f68947c0640 1 --2- 172.21.15.98:0/1525553272 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6870006a20 0x7f688c15d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.367+0000 7f68947c0640 1 -- 172.21.15.98:0/1525553272 >> 172.21.15.98:0/1525553272 conn(0x7f688c0b2010 msgr2=0x7f688c06d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:51.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.367+0000 7f68947c0640 1 -- 172.21.15.98:0/1525553272 shutdown_connections 2024-09-19T08:45:51.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.367+0000 7f68947c0640 1 -- 172.21.15.98:0/1525553272 wait complete. 2024-09-19T08:45:51.383 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-19T08:45:51.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.503+0000 7fce77d4a640 1 Processor -- start 2024-09-19T08:45:51.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.503+0000 7fce77d4a640 1 -- start start 2024-09-19T08:45:51.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.503+0000 7fce77d4a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fce7015c550 0x7fce7006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:51.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.503+0000 7fce77d4a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce700cfad0 0x7fce700723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:51.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.503+0000 7fce77d4a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce7006d310 con 0x7fce7006c1d0 2024-09-19T08:45:51.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.503+0000 7fce77d4a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fce7006c640 con 0x7fce7015c550 2024-09-19T08:45:51.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.503+0000 7fce77d4a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fce7006d5f0 con 0x7fce700cfad0 2024-09-19T08:45:51.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.503+0000 7fce762e7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce700cfad0 0x7fce700723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:51.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.503+0000 7fce762e7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce700cfad0 0x7fce700723a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:45346/0 (socket says 172.21.15.98:45346) 2024-09-19T08:45:51.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.503+0000 7fce762e7640 1 -- 172.21.15.98:0/3937384627 learned_addr learned my addr 172.21.15.98:0/3937384627 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:51.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.503+0000 7fce75ae6640 1 --2- 172.21.15.98:0/3937384627 >> v2:172.21.15.139:3300/0 conn(0x7fce7015c550 0x7fce7006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:51.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.503+0000 7fce752e5640 1 --1- 172.21.15.98:0/3937384627 >> v1:172.21.15.139:6789/0 conn(0x7fce7006c1d0 0x7fce700cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:59784/0 (socket says 172.21.15.98:59784) 2024-09-19T08:45:51.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.503+0000 7fce762e7640 1 -- 172.21.15.98:0/3937384627 >> v2:172.21.15.139:3300/0 conn(0x7fce7015c550 msgr2=0x7fce7006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:51.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.503+0000 7fce762e7640 1 --2- 172.21.15.98:0/3937384627 >> v2:172.21.15.139:3300/0 conn(0x7fce7015c550 0x7fce7006bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.503+0000 7fce762e7640 1 -- 172.21.15.98:0/3937384627 >> v1:172.21.15.139:6789/0 conn(0x7fce7006c1d0 legacy=0x7fce700cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:51.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.503+0000 7fce762e7640 1 -- 172.21.15.98:0/3937384627 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce7006d8d0 con 0x7fce700cfad0 2024-09-19T08:45:51.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.507+0000 7fce762e7640 1 --2- 172.21.15.98:0/3937384627 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce700cfad0 0x7fce700723a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fce68009e50 tx=0x7fce6801ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63b99a76cdb2f84a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:51.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.507+0000 7fce74ae4640 1 -- 172.21.15.98:0/3937384627 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce680a5810 con 0x7fce700cfad0 2024-09-19T08:45:51.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.507+0000 7fce74ae4640 1 -- 172.21.15.98:0/3937384627 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce680a59b0 con 0x7fce700cfad0 2024-09-19T08:45:51.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.507+0000 7fce74ae4640 1 -- 172.21.15.98:0/3937384627 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce680a5d80 con 0x7fce700cfad0 2024-09-19T08:45:51.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.507+0000 7fce77d4a640 1 -- 172.21.15.98:0/3937384627 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce700cfad0 msgr2=0x7fce700723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:51.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.507+0000 7fce77d4a640 1 --2- 172.21.15.98:0/3937384627 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce700cfad0 0x7fce700723a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fce68009e50 tx=0x7fce6801ca30 comp rx=0 tx=0).stop 2024-09-19T08:45:51.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.507+0000 7fce77d4a640 1 -- 172.21.15.98:0/3937384627 shutdown_connections 2024-09-19T08:45:51.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.507+0000 7fce77d4a640 1 --2- 172.21.15.98:0/3937384627 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce700cfad0 0x7fce700723a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.507+0000 7fce77d4a640 1 --2- 172.21.15.98:0/3937384627 >> v2:172.21.15.139:3300/0 conn(0x7fce7015c550 0x7fce7006bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.507+0000 7fce77d4a640 1 -- 172.21.15.98:0/3937384627 >> 172.21.15.98:0/3937384627 conn(0x7fce700b2010 msgr2=0x7fce700b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:51.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.507+0000 7fce77d4a640 1 -- 172.21.15.98:0/3937384627 shutdown_connections 2024-09-19T08:45:51.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.507+0000 7fce77d4a640 1 -- 172.21.15.98:0/3937384627 wait complete. 2024-09-19T08:45:51.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.507+0000 7fce77d4a640 1 Processor -- start 2024-09-19T08:45:51.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.507+0000 7fce77d4a640 1 -- start start 2024-09-19T08:45:51.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.507+0000 7fce77d4a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce7006c1d0 0x7fce70147450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:51.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.507+0000 7fce77d4a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce700cfad0 0x7fce701531c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:51.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.507+0000 7fce77d4a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce7015c550 0x7fce7014ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:51.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.507+0000 7fce77d4a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fce70073500 con 0x7fce700cfad0 2024-09-19T08:45:51.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.507+0000 7fce77d4a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fce70073380 con 0x7fce7006c1d0 2024-09-19T08:45:51.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.507+0000 7fce77d4a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fce70073680 con 0x7fce7015c550 2024-09-19T08:45:51.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.507+0000 7fce752e5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce700cfad0 0x7fce701531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:51.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.507+0000 7fce752e5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce700cfad0 0x7fce701531c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53944/0 (socket says 172.21.15.98:53944) 2024-09-19T08:45:51.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.507+0000 7fce752e5640 1 -- 172.21.15.98:0/3056491488 learned_addr learned my addr 172.21.15.98:0/3056491488 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:51.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.507+0000 7fce762e7640 1 --2- 172.21.15.98:0/3056491488 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce7015c550 0x7fce7014ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:51.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.511+0000 7fce75ae6640 1 --2- 172.21.15.98:0/3056491488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce7006c1d0 0x7fce70147450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:51.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.511+0000 7fce752e5640 1 -- 172.21.15.98:0/3056491488 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce7015c550 msgr2=0x7fce7014ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:51.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.511+0000 7fce752e5640 1 --2- 172.21.15.98:0/3056491488 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce7015c550 0x7fce7014ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.511+0000 7fce752e5640 1 -- 172.21.15.98:0/3056491488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce7006c1d0 msgr2=0x7fce70147450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:51.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.511+0000 7fce752e5640 1 --2- 172.21.15.98:0/3056491488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce7006c1d0 0x7fce70147450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.511+0000 7fce752e5640 1 -- 172.21.15.98:0/3056491488 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce70154020 con 0x7fce700cfad0 2024-09-19T08:45:51.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.511+0000 7fce752e5640 1 --2- 172.21.15.98:0/3056491488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce700cfad0 0x7fce701531c0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fce6000c370 tx=0x7fce6000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:51.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.511+0000 7fce667fc640 1 -- 172.21.15.98:0/3056491488 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce600074e0 con 0x7fce700cfad0 2024-09-19T08:45:51.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.511+0000 7fce667fc640 1 -- 172.21.15.98:0/3056491488 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce60007680 con 0x7fce700cfad0 2024-09-19T08:45:51.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.511+0000 7fce667fc640 1 -- 172.21.15.98:0/3056491488 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce60004a30 con 0x7fce700cfad0 2024-09-19T08:45:51.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.511+0000 7fce77d4a640 1 -- 172.21.15.98:0/3056491488 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce70156420 con 0x7fce700cfad0 2024-09-19T08:45:51.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.511+0000 7fce77d4a640 1 -- 172.21.15.98:0/3056491488 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce701569e0 con 0x7fce700cfad0 2024-09-19T08:45:51.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.511+0000 7fce77d4a640 1 -- 172.21.15.98:0/3056491488 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce700723a0 con 0x7fce700cfad0 2024-09-19T08:45:51.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.511+0000 7fce667fc640 1 -- 172.21.15.98:0/3056491488 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fce6001a020 con 0x7fce700cfad0 2024-09-19T08:45:51.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.511+0000 7fce667fc640 1 --2- 172.21.15.98:0/3056491488 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fce54067ef0 0x7fce5406a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:51.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.511+0000 7fce667fc640 1 -- 172.21.15.98:0/3056491488 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5737+0+0 (secure 0 0 0) 0x7fce60085f70 con 0x7fce700cfad0 2024-09-19T08:45:51.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.511+0000 7fce75ae6640 1 --2- 172.21.15.98:0/3056491488 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fce54067ef0 0x7fce5406a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:51.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.511+0000 7fce75ae6640 1 --2- 172.21.15.98:0/3056491488 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fce54067ef0 0x7fce5406a3b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fce7014fbb0 tx=0x7fce6c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:51.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.515+0000 7fce667fc640 1 -- 172.21.15.98:0/3056491488 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fce60056150 con 0x7fce700cfad0 2024-09-19T08:45:51.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.707+0000 7fce77d4a640 1 -- 172.21.15.98:0/3056491488 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fce701518a0 con 0x7fce700cfad0 2024-09-19T08:45:51.707 INFO:tasks.ceph.mon.a.smithi098.stderr:2024-09-19T08:45:51.707+0000 7f5e49017640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T08:45:51.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.711+0000 7fce667fc640 1 -- 172.21.15.98:0/3056491488 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v76) ==== 161+0+0 (secure 0 0 0) 0x7fce60055a50 con 0x7fce700cfad0 2024-09-19T08:45:51.709 INFO:teuthology.orchestra.run.smithi098.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T08:45:51.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.715+0000 7fce77d4a640 1 -- 172.21.15.98:0/3056491488 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fce54067ef0 msgr2=0x7fce5406a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:51.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.715+0000 7fce77d4a640 1 --2- 172.21.15.98:0/3056491488 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fce54067ef0 0x7fce5406a3b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fce7014fbb0 tx=0x7fce6c008040 comp rx=0 tx=0).stop 2024-09-19T08:45:51.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.715+0000 7fce77d4a640 1 -- 172.21.15.98:0/3056491488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce700cfad0 msgr2=0x7fce701531c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:51.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.715+0000 7fce77d4a640 1 --2- 172.21.15.98:0/3056491488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce700cfad0 0x7fce701531c0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fce6000c370 tx=0x7fce6000c830 comp rx=0 tx=0).stop 2024-09-19T08:45:51.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.715+0000 7fce77d4a640 1 -- 172.21.15.98:0/3056491488 shutdown_connections 2024-09-19T08:45:51.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.715+0000 7fce77d4a640 1 --2- 172.21.15.98:0/3056491488 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fce54067ef0 0x7fce5406a3b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.715+0000 7fce77d4a640 1 --2- 172.21.15.98:0/3056491488 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce7015c550 0x7fce7014ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.715+0000 7fce77d4a640 1 --2- 172.21.15.98:0/3056491488 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce700cfad0 0x7fce701531c0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.715+0000 7fce77d4a640 1 --2- 172.21.15.98:0/3056491488 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce7006c1d0 0x7fce70147450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.715+0000 7fce77d4a640 1 -- 172.21.15.98:0/3056491488 >> 172.21.15.98:0/3056491488 conn(0x7fce700b2010 msgr2=0x7fce700cf3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:51.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.715+0000 7fce77d4a640 1 -- 172.21.15.98:0/3056491488 shutdown_connections 2024-09-19T08:45:51.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.715+0000 7fce77d4a640 1 -- 172.21.15.98:0/3056491488 wait complete. 2024-09-19T08:45:51.715 INFO:tasks.ceph.mds.b.smithi139.stderr: -2759> 2024-09-19T08:45:23.479+0000 7f2d73261640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-09-19T08:45:51.720 INFO:tasks.ceph.mds.b.smithi139.stderr: -1435> 2024-09-19T08:45:34.434+0000 7f2d6d255640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T08:45:51.728 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T08:45:51.773 INFO:tasks.ceph.mds.b.smithi139.stdout:starting mds.b at 2024-09-19T08:45:51.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.851+0000 7fd7183cd640 1 Processor -- start 2024-09-19T08:45:51.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.851+0000 7fd7183cd640 1 -- start start 2024-09-19T08:45:51.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.851+0000 7fd7183cd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd7101415a0 0x7fd7100d20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:51.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.851+0000 7fd7183cd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd7100d25f0 0x7fd7100d3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:51.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.851+0000 7fd7183cd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7100d38a0 0x7fd710161fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:51.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.851+0000 7fd7183cd640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fd7100d2a30 con 0x7fd7100d25f0 2024-09-19T08:45:51.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.851+0000 7fd7183cd640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fd71015eae0 con 0x7fd7101415a0 2024-09-19T08:45:51.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.851+0000 7fd7183cd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd71015edc0 con 0x7fd7100d38a0 2024-09-19T08:45:51.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.851+0000 7fd715968640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd7100d25f0 0x7fd7100d3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:51.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.851+0000 7fd715968640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd7100d25f0 0x7fd7100d3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53950/0 (socket says 172.21.15.98:53950) 2024-09-19T08:45:51.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.851+0000 7fd715968640 1 -- 172.21.15.98:0/3548560759 learned_addr learned my addr 172.21.15.98:0/3548560759 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:51.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.855+0000 7fd716169640 1 --2- 172.21.15.98:0/3548560759 >> v2:172.21.15.139:3300/0 conn(0x7fd7101415a0 0x7fd7100d20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:51.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.855+0000 7fd71696a640 1 --2- 172.21.15.98:0/3548560759 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7100d38a0 0x7fd710161fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:51.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.855+0000 7fd715968640 1 -- 172.21.15.98:0/3548560759 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7100d38a0 msgr2=0x7fd710161fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:51.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.855+0000 7fd715968640 1 --2- 172.21.15.98:0/3548560759 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7100d38a0 0x7fd710161fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.855+0000 7fd715968640 1 -- 172.21.15.98:0/3548560759 >> v2:172.21.15.139:3300/0 conn(0x7fd7101415a0 msgr2=0x7fd7100d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:51.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.855+0000 7fd715968640 1 --2- 172.21.15.98:0/3548560759 >> v2:172.21.15.139:3300/0 conn(0x7fd7101415a0 0x7fd7100d20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.855+0000 7fd715968640 1 -- 172.21.15.98:0/3548560759 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd71015f0a0 con 0x7fd7100d25f0 2024-09-19T08:45:51.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.855+0000 7fd71696a640 1 --2- 172.21.15.98:0/3548560759 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7100d38a0 0x7fd710161fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:51.855 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.855+0000 7fd715968640 1 --2- 172.21.15.98:0/3548560759 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7100d25f0 0x7fd7100d3360 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fd71006cca0 tx=0x7fd70001c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac82675518951bb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:51.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.855+0000 7fd715167640 1 -- 172.21.15.98:0/3548560759 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7000b0070 con 0x7fd7100d25f0 2024-09-19T08:45:51.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.855+0000 7fd715167640 1 -- 172.21.15.98:0/3548560759 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7000ab600 con 0x7fd7100d25f0 2024-09-19T08:45:51.856 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.855+0000 7fd715167640 1 -- 172.21.15.98:0/3548560759 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7000ab9d0 con 0x7fd7100d25f0 2024-09-19T08:45:51.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.855+0000 7fd7183cd640 1 -- 172.21.15.98:0/3548560759 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7100d25f0 msgr2=0x7fd7100d3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:51.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.855+0000 7fd7183cd640 1 --2- 172.21.15.98:0/3548560759 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7100d25f0 0x7fd7100d3360 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fd71006cca0 tx=0x7fd70001c230 comp rx=0 tx=0).stop 2024-09-19T08:45:51.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.855+0000 7fd7183cd640 1 -- 172.21.15.98:0/3548560759 shutdown_connections 2024-09-19T08:45:51.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.855+0000 7fd7183cd640 1 --2- 172.21.15.98:0/3548560759 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7100d25f0 0x7fd7100d3360 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.855+0000 7fd7183cd640 1 --2- 172.21.15.98:0/3548560759 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7100d38a0 0x7fd710161fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.855+0000 7fd7183cd640 1 --2- 172.21.15.98:0/3548560759 >> v2:172.21.15.139:3300/0 conn(0x7fd7101415a0 0x7fd7100d20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.855+0000 7fd7183cd640 1 -- 172.21.15.98:0/3548560759 >> 172.21.15.98:0/3548560759 conn(0x7fd710074c70 msgr2=0x7fd710075080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:51.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.855+0000 7fd7183cd640 1 -- 172.21.15.98:0/3548560759 shutdown_connections 2024-09-19T08:45:51.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.855+0000 7fd7183cd640 1 -- 172.21.15.98:0/3548560759 wait complete. 2024-09-19T08:45:51.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.859+0000 7fd7183cd640 1 Processor -- start 2024-09-19T08:45:51.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.859+0000 7fd7183cd640 1 -- start start 2024-09-19T08:45:51.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.859+0000 7fd7183cd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7100d25f0 0x7fd71013e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:51.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.859+0000 7fd7183cd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7100d38a0 0x7fd7100cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:51.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.859+0000 7fd7183cd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7101415a0 0x7fd71013f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:51.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.859+0000 7fd7183cd640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd7100d3e30 con 0x7fd7100d38a0 2024-09-19T08:45:51.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.859+0000 7fd7183cd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd7100d3cb0 con 0x7fd7100d25f0 2024-09-19T08:45:51.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.859+0000 7fd7183cd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd7100b8790 con 0x7fd7101415a0 2024-09-19T08:45:51.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.859+0000 7fd715968640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7100d38a0 0x7fd7100cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:51.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.859+0000 7fd715968640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7100d38a0 0x7fd7100cdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53962/0 (socket says 172.21.15.98:53962) 2024-09-19T08:45:51.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.859+0000 7fd715968640 1 -- 172.21.15.98:0/1262729147 learned_addr learned my addr 172.21.15.98:0/1262729147 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:51.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.859+0000 7fd716169640 1 --2- 172.21.15.98:0/1262729147 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7100d25f0 0x7fd71013e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:51.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.859+0000 7fd715968640 1 -- 172.21.15.98:0/1262729147 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7101415a0 msgr2=0x7fd71013f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:51.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.859+0000 7fd715968640 1 --2- 172.21.15.98:0/1262729147 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7101415a0 0x7fd71013f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.859+0000 7fd715968640 1 -- 172.21.15.98:0/1262729147 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7100d25f0 msgr2=0x7fd71013e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:51.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.859+0000 7fd715968640 1 --2- 172.21.15.98:0/1262729147 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7100d25f0 0x7fd71013e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:51.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.859+0000 7fd715968640 1 -- 172.21.15.98:0/1262729147 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7100cc9b0 con 0x7fd7100d38a0 2024-09-19T08:45:51.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.859+0000 7fd715968640 1 --2- 172.21.15.98:0/1262729147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7100d38a0 0x7fd7100cdd30 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fd700000c10 tx=0x7fd700018fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:51.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.859+0000 7fd706ffd640 1 -- 172.21.15.98:0/1262729147 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7000b0040 con 0x7fd7100d38a0 2024-09-19T08:45:51.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.859+0000 7fd706ffd640 1 -- 172.21.15.98:0/1262729147 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7000b9070 con 0x7fd7100d38a0 2024-09-19T08:45:51.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.859+0000 7fd7183cd640 1 -- 172.21.15.98:0/1262729147 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7100ccc90 con 0x7fd7100d38a0 2024-09-19T08:45:51.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.859+0000 7fd706ffd640 1 -- 172.21.15.98:0/1262729147 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7000b4830 con 0x7fd7100d38a0 2024-09-19T08:45:51.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.859+0000 7fd7183cd640 1 -- 172.21.15.98:0/1262729147 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7100cd250 con 0x7fd7100d38a0 2024-09-19T08:45:51.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.859+0000 7fd7183cd640 1 -- 172.21.15.98:0/1262729147 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd710161fc0 con 0x7fd7100d38a0 2024-09-19T08:45:51.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.859+0000 7fd706ffd640 1 -- 172.21.15.98:0/1262729147 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fd7000ab480 con 0x7fd7100d38a0 2024-09-19T08:45:51.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.859+0000 7fd706ffd640 1 --2- 172.21.15.98:0/1262729147 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd6f4067ef0 0x7fd6f406a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:51.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.859+0000 7fd706ffd640 1 -- 172.21.15.98:0/1262729147 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5823+0+0 (secure 0 0 0) 0x7fd700120570 con 0x7fd7100d38a0 2024-09-19T08:45:51.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.863+0000 7fd716169640 1 --2- 172.21.15.98:0/1262729147 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd6f4067ef0 0x7fd6f406a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:51.862 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.863+0000 7fd716169640 1 --2- 172.21.15.98:0/1262729147 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd6f4067ef0 0x7fd6f406a3b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd70c00b930 tx=0x7fd70c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:51.864 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:51.867+0000 7fd706ffd640 1 -- 172.21.15.98:0/1262729147 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd700016110 con 0x7fd7100d38a0 2024-09-19T08:45:52.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.051+0000 7fd7183cd640 1 -- 172.21.15.98:0/1262729147 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd7100757a0 con 0x7fd7100d38a0 2024-09-19T08:45:52.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.715+0000 7fd706ffd640 1 -- 172.21.15.98:0/1262729147 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v77) ==== 104+0+0 (secure 0 0 0) 0x7fd7000f0760 con 0x7fd7100d38a0 2024-09-19T08:45:52.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.719+0000 7fd7183cd640 1 -- 172.21.15.98:0/1262729147 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd6f4067ef0 msgr2=0x7fd6f406a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:52.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.719+0000 7fd7183cd640 1 --2- 172.21.15.98:0/1262729147 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd6f4067ef0 0x7fd6f406a3b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd70c00b930 tx=0x7fd70c008040 comp rx=0 tx=0).stop 2024-09-19T08:45:52.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.719+0000 7fd7183cd640 1 -- 172.21.15.98:0/1262729147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7100d38a0 msgr2=0x7fd7100cdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:52.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.719+0000 7fd7183cd640 1 --2- 172.21.15.98:0/1262729147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7100d38a0 0x7fd7100cdd30 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fd700000c10 tx=0x7fd700018fd0 comp rx=0 tx=0).stop 2024-09-19T08:45:52.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.719+0000 7fd7183cd640 1 -- 172.21.15.98:0/1262729147 shutdown_connections 2024-09-19T08:45:52.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.719+0000 7fd7183cd640 1 --2- 172.21.15.98:0/1262729147 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd6f4067ef0 0x7fd6f406a3b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:52.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.719+0000 7fd7183cd640 1 --2- 172.21.15.98:0/1262729147 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7101415a0 0x7fd71013f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:52.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.719+0000 7fd7183cd640 1 --2- 172.21.15.98:0/1262729147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7100d38a0 0x7fd7100cdd30 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:52.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.719+0000 7fd7183cd640 1 --2- 172.21.15.98:0/1262729147 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7100d25f0 0x7fd71013e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:52.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.719+0000 7fd7183cd640 1 -- 172.21.15.98:0/1262729147 >> 172.21.15.98:0/1262729147 conn(0x7fd710074c70 msgr2=0x7fd710157e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:52.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.719+0000 7fd7183cd640 1 -- 172.21.15.98:0/1262729147 shutdown_connections 2024-09-19T08:45:52.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.719+0000 7fd7183cd640 1 -- 172.21.15.98:0/1262729147 wait complete. 2024-09-19T08:45:52.738 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T08:45:52.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.871+0000 7fa859008640 1 Processor -- start 2024-09-19T08:45:52.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.871+0000 7fa859008640 1 -- start start 2024-09-19T08:45:52.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.871+0000 7fa859008640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa854144200 0x7fa8541445e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:52.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.871+0000 7fa859008640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa854071030 0x7fa85406d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:52.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.871+0000 7fa859008640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa85415fac0 con 0x7fa854071470 2024-09-19T08:45:52.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.871+0000 7fa859008640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fa854141940 con 0x7fa854071030 2024-09-19T08:45:52.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.871+0000 7fa859008640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa854163060 con 0x7fa854144200 2024-09-19T08:45:52.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.871+0000 7fa85359e640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fa854071470 0x7fa8540ae8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42472/0 (socket says 172.21.15.98:42472) 2024-09-19T08:45:52.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.871+0000 7fa85359e640 1 -- 172.21.15.98:0/199095300 learned_addr learned my addr 172.21.15.98:0/199095300 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:52.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.871+0000 7fa85259c640 1 --2- 172.21.15.98:0/199095300 >> v2:172.21.15.139:3300/0 conn(0x7fa854071030 0x7fa85406d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:52.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.871+0000 7fa852d9d640 1 --2- 172.21.15.98:0/199095300 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa854144200 0x7fa8541445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:52.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.871+0000 7fa851d9b640 1 -- 172.21.15.98:0/199095300 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 538680806 0 0) 0x7fa85415fac0 con 0x7fa854071470 2024-09-19T08:45:52.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.871+0000 7fa851d9b640 1 -- 172.21.15.98:0/199095300 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa83c003530 con 0x7fa854071470 2024-09-19T08:45:52.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.871+0000 7fa851d9b640 1 -- 172.21.15.98:0/199095300 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3271039314 0 0) 0x7fa83c003530 con 0x7fa854071470 2024-09-19T08:45:52.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.871+0000 7fa851d9b640 1 -- 172.21.15.98:0/199095300 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa854144200 msgr2=0x7fa8541445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:52.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.871+0000 7fa851d9b640 1 --2- 172.21.15.98:0/199095300 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa854144200 0x7fa8541445e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:52.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.871+0000 7fa851d9b640 1 -- 172.21.15.98:0/199095300 >> v2:172.21.15.139:3300/0 conn(0x7fa854071030 msgr2=0x7fa85406d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:52.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.871+0000 7fa851d9b640 1 --2- 172.21.15.98:0/199095300 >> v2:172.21.15.139:3300/0 conn(0x7fa854071030 0x7fa85406d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:52.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.871+0000 7fa851d9b640 1 -- 172.21.15.98:0/199095300 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa854144b20 con 0x7fa854071470 2024-09-19T08:45:52.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.871+0000 7fa851d9b640 1 -- 172.21.15.98:0/199095300 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fa844001680 con 0x7fa854071470 2024-09-19T08:45:52.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.871+0000 7fa851d9b640 1 -- 172.21.15.98:0/199095300 >> v1:172.21.15.98:6789/0 conn(0x7fa854071470 legacy=0x7fa8540ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:52.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.871+0000 7fa851d9b640 1 --2- 172.21.15.98:0/199095300 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa854144200 0x7fa83c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:52.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.871+0000 7fa851d9b640 1 --2- 172.21.15.98:0/199095300 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa83c003f40 0x7fa83c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:52.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.871+0000 7fa851d9b640 1 --2- 172.21.15.98:0/199095300 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa83c004d50 0x7fa83c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:52.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.871+0000 7fa851d9b640 1 -- 172.21.15.98:0/199095300 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa854141940 con 0x7fa83c003f40 2024-09-19T08:45:52.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.871+0000 7fa851d9b640 1 -- 172.21.15.98:0/199095300 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa854163060 con 0x7fa83c004d50 2024-09-19T08:45:52.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.871+0000 7fa851d9b640 1 -- 172.21.15.98:0/199095300 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa83c00b9d0 con 0x7fa854144200 2024-09-19T08:45:52.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.875+0000 7fa85259c640 1 --2- 172.21.15.98:0/199095300 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa83c003f40 0x7fa83c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:52.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.875+0000 7fa85359e640 1 --2- 172.21.15.98:0/199095300 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa83c004d50 0x7fa83c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:52.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.875+0000 7fa852d9d640 1 --2- 172.21.15.98:0/199095300 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa854144200 0x7fa83c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:52.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.875+0000 7fa85259c640 1 -- 172.21.15.98:0/199095300 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa854144200 msgr2=0x7fa83c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:52.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.875+0000 7fa85259c640 1 --2- 172.21.15.98:0/199095300 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa854144200 0x7fa83c0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:52.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.875+0000 7fa85259c640 1 -- 172.21.15.98:0/199095300 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa83c004d50 msgr2=0x7fa83c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:52.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.875+0000 7fa85259c640 1 --2- 172.21.15.98:0/199095300 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa83c004d50 0x7fa83c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:52.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.875+0000 7fa85259c640 1 -- 172.21.15.98:0/199095300 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa83c003530 con 0x7fa83c003f40 2024-09-19T08:45:52.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.875+0000 7fa85359e640 1 --2- 172.21.15.98:0/199095300 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa83c004d50 0x7fa83c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:52.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.875+0000 7fa85259c640 1 --2- 172.21.15.98:0/199095300 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa83c003f40 0x7fa83c004810 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fa848009ae0 tx=0x7fa84801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=364128090652e88d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:52.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.875+0000 7fa851d9b640 1 -- 172.21.15.98:0/199095300 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8480b2070 con 0x7fa83c003f40 2024-09-19T08:45:52.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.875+0000 7fa851d9b640 1 -- 172.21.15.98:0/199095300 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8480a56a0 con 0x7fa83c003f40 2024-09-19T08:45:52.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.875+0000 7fa851d9b640 1 -- 172.21.15.98:0/199095300 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8480096b0 con 0x7fa83c003f40 2024-09-19T08:45:52.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.875+0000 7fa859008640 1 -- 172.21.15.98:0/199095300 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa83c003f40 msgr2=0x7fa83c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:52.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.875+0000 7fa859008640 1 --2- 172.21.15.98:0/199095300 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa83c003f40 0x7fa83c004810 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fa848009ae0 tx=0x7fa84801ca50 comp rx=0 tx=0).stop 2024-09-19T08:45:52.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.875+0000 7fa85359e640 1 -- 172.21.15.98:0/199095300 reap_dead start 2024-09-19T08:45:52.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.875+0000 7fa859008640 1 -- 172.21.15.98:0/199095300 shutdown_connections 2024-09-19T08:45:52.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.875+0000 7fa859008640 1 -- 172.21.15.98:0/199095300 >> 172.21.15.98:0/199095300 conn(0x7fa85406c090 msgr2=0x7fa85406c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:52.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.875+0000 7fa859008640 1 -- 172.21.15.98:0/199095300 shutdown_connections 2024-09-19T08:45:52.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.875+0000 7fa859008640 1 -- 172.21.15.98:0/199095300 wait complete. 2024-09-19T08:45:52.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.879+0000 7fa859008640 1 Processor -- start 2024-09-19T08:45:52.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.879+0000 7fa859008640 1 -- start start 2024-09-19T08:45:52.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.879+0000 7fa859008640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa854141d20 0x7fa85413da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:52.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.879+0000 7fa859008640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa854142100 0x7fa85413df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:52.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.879+0000 7fa859008640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8541e8790 0x7fa8541e8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:52.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.879+0000 7fa859008640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa854162580 con 0x7fa854141d20 2024-09-19T08:45:52.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.879+0000 7fa859008640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa8540718f0 con 0x7fa854142100 2024-09-19T08:45:52.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.879+0000 7fa859008640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa854162860 con 0x7fa8541e8790 2024-09-19T08:45:52.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.879+0000 7fa852d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa854141d20 0x7fa85413da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:52.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.879+0000 7fa852d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa854141d20 0x7fa85413da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53974/0 (socket says 172.21.15.98:53974) 2024-09-19T08:45:52.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.879+0000 7fa852d9d640 1 -- 172.21.15.98:0/3325188340 learned_addr learned my addr 172.21.15.98:0/3325188340 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:52.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.879+0000 7fa85359e640 1 --2- 172.21.15.98:0/3325188340 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8541e8790 0x7fa8541e8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:52.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.879+0000 7fa852d9d640 1 -- 172.21.15.98:0/3325188340 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8541e8790 msgr2=0x7fa8541e8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:52.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.879+0000 7fa85259c640 1 --2- 172.21.15.98:0/3325188340 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa854142100 0x7fa85413df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:52.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.879+0000 7fa852d9d640 1 --2- 172.21.15.98:0/3325188340 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8541e8790 0x7fa8541e8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:52.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.879+0000 7fa852d9d640 1 -- 172.21.15.98:0/3325188340 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa854142100 msgr2=0x7fa85413df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:52.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.879+0000 7fa852d9d640 1 --2- 172.21.15.98:0/3325188340 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa854142100 0x7fa85413df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:52.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.879+0000 7fa852d9d640 1 -- 172.21.15.98:0/3325188340 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8541402c0 con 0x7fa854141d20 2024-09-19T08:45:52.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.879+0000 7fa852d9d640 1 --2- 172.21.15.98:0/3325188340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa854141d20 0x7fa85413da00 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fa84c007f30 tx=0x7fa84c011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:52.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.879+0000 7fa83b7fe640 1 -- 172.21.15.98:0/3325188340 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c01a020 con 0x7fa854141d20 2024-09-19T08:45:52.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.879+0000 7fa83b7fe640 1 -- 172.21.15.98:0/3325188340 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa84c016470 con 0x7fa854141d20 2024-09-19T08:45:52.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.879+0000 7fa859008640 1 -- 172.21.15.98:0/3325188340 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8541405a0 con 0x7fa854141d20 2024-09-19T08:45:52.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.879+0000 7fa83b7fe640 1 -- 172.21.15.98:0/3325188340 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c016860 con 0x7fa854141d20 2024-09-19T08:45:52.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.879+0000 7fa859008640 1 -- 172.21.15.98:0/3325188340 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8541e7f90 con 0x7fa854141d20 2024-09-19T08:45:52.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.879+0000 7fa859008640 1 -- 172.21.15.98:0/3325188340 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa854072d10 con 0x7fa854141d20 2024-09-19T08:45:52.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.883+0000 7fa83b7fe640 1 -- 172.21.15.98:0/3325188340 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fa84c01d020 con 0x7fa854141d20 2024-09-19T08:45:52.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.883+0000 7fa83b7fe640 1 --2- 172.21.15.98:0/3325188340 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa840067ef0 0x7fa84006a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:52.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.883+0000 7fa83b7fe640 1 -- 172.21.15.98:0/3325188340 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5823+0+0 (secure 0 0 0) 0x7fa84c0876b0 con 0x7fa854141d20 2024-09-19T08:45:52.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.883+0000 7fa85259c640 1 --2- 172.21.15.98:0/3325188340 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa840067ef0 0x7fa84006a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:52.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.883+0000 7fa85259c640 1 --2- 172.21.15.98:0/3325188340 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa840067ef0 0x7fa84006a3b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa8480af000 tx=0x7fa848008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:52.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:52.887+0000 7fa83b7fe640 1 -- 172.21.15.98:0/3325188340 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa84c024030 con 0x7fa854141d20 2024-09-19T08:45:53.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.039+0000 7fa859008640 1 -- 172.21.15.98:0/3325188340 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa8540b6070 con 0x7fa854141d20 2024-09-19T08:45:53.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.723+0000 7fa83b7fe640 1 -- 172.21.15.98:0/3325188340 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v137) ==== 181+0+0 (secure 0 0 0) 0x7fa84c057270 con 0x7fa854141d20 2024-09-19T08:45:53.721 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_metadata' removed 2024-09-19T08:45:53.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.727+0000 7fa859008640 1 -- 172.21.15.98:0/3325188340 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa840067ef0 msgr2=0x7fa84006a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:53.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.727+0000 7fa859008640 1 --2- 172.21.15.98:0/3325188340 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa840067ef0 0x7fa84006a3b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa8480af000 tx=0x7fa848008040 comp rx=0 tx=0).stop 2024-09-19T08:45:53.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.727+0000 7fa859008640 1 -- 172.21.15.98:0/3325188340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa854141d20 msgr2=0x7fa85413da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:53.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.727+0000 7fa859008640 1 --2- 172.21.15.98:0/3325188340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa854141d20 0x7fa85413da00 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fa84c007f30 tx=0x7fa84c011660 comp rx=0 tx=0).stop 2024-09-19T08:45:53.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.727+0000 7fa859008640 1 -- 172.21.15.98:0/3325188340 shutdown_connections 2024-09-19T08:45:53.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.727+0000 7fa859008640 1 --2- 172.21.15.98:0/3325188340 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa840067ef0 0x7fa84006a3b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:53.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.727+0000 7fa859008640 1 --2- 172.21.15.98:0/3325188340 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8541e8790 0x7fa8541e8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:53.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.727+0000 7fa859008640 1 --2- 172.21.15.98:0/3325188340 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa854142100 0x7fa85413df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:53.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.727+0000 7fa859008640 1 --2- 172.21.15.98:0/3325188340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa854141d20 0x7fa85413da00 secure :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fa84c007f30 tx=0x7fa84c011660 comp rx=0 tx=0).stop 2024-09-19T08:45:53.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.727+0000 7fa859008640 1 -- 172.21.15.98:0/3325188340 >> 172.21.15.98:0/3325188340 conn(0x7fa85406c090 msgr2=0x7fa8541619b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:53.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.727+0000 7fa859008640 1 -- 172.21.15.98:0/3325188340 shutdown_connections 2024-09-19T08:45:53.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.727+0000 7fa859008640 1 -- 172.21.15.98:0/3325188340 wait complete. 2024-09-19T08:45:53.747 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T08:45:53.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.875+0000 7f0707821640 1 Processor -- start 2024-09-19T08:45:53.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.875+0000 7f0707821640 1 -- start start 2024-09-19T08:45:53.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.875+0000 7f0707821640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f070015c290 0x7f07000cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:53.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.875+0000 7f0707821640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07000cf700 0x7f070015ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:53.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.875+0000 7f0707821640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07001608c0 con 0x7f070006d4c0 2024-09-19T08:45:53.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.875+0000 7f0707821640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f0700160e80 con 0x7f070015c290 2024-09-19T08:45:53.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.875+0000 7f0707821640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f07000cd510 con 0x7f07000cf700 2024-09-19T08:45:53.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.875+0000 7f0704dbc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07000cf700 0x7f070015ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:53.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.875+0000 7f07055bd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f070015c290 0x7f07000cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:53.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.875+0000 7f0705dbe640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f070006d4c0 0x7f07000b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:59786/0 (socket says 172.21.15.98:59786) 2024-09-19T08:45:53.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.875+0000 7f0705dbe640 1 -- 172.21.15.98:0/3773962144 learned_addr learned my addr 172.21.15.98:0/3773962144 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:53.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.875+0000 7f0704dbc640 1 -- 172.21.15.98:0/3773962144 >> v2:172.21.15.139:3300/0 conn(0x7f070015c290 msgr2=0x7f07000cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:53.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.875+0000 7f0704dbc640 1 --2- 172.21.15.98:0/3773962144 >> v2:172.21.15.139:3300/0 conn(0x7f070015c290 0x7f07000cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:53.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.875+0000 7f0704dbc640 1 -- 172.21.15.98:0/3773962144 >> v1:172.21.15.139:6789/0 conn(0x7f070006d4c0 legacy=0x7f07000b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:53.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.875+0000 7f0704dbc640 1 -- 172.21.15.98:0/3773962144 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0700160ba0 con 0x7f07000cf700 2024-09-19T08:45:53.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.875+0000 7f06f7fff640 1 -- 172.21.15.98:0/3773962144 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 643938688 0 0) 0x7f07001608c0 con 0x7f070006d4c0 2024-09-19T08:45:53.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.879+0000 7f0704dbc640 1 --2- 172.21.15.98:0/3773962144 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07000cf700 0x7f070015ff50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f07000cd690 tx=0x7f06f001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a446bdf4afa9bbdf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:53.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.879+0000 7f06f7fff640 1 -- 172.21.15.98:0/3773962144 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f00b0070 con 0x7f07000cf700 2024-09-19T08:45:53.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.879+0000 7f06f7fff640 1 -- 172.21.15.98:0/3773962144 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06f00ab600 con 0x7f07000cf700 2024-09-19T08:45:53.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.879+0000 7f06f7fff640 1 -- 172.21.15.98:0/3773962144 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f00ab9d0 con 0x7f07000cf700 2024-09-19T08:45:53.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.879+0000 7f0707821640 1 -- 172.21.15.98:0/3773962144 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07000cf700 msgr2=0x7f070015ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:53.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.879+0000 7f0707821640 1 --2- 172.21.15.98:0/3773962144 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07000cf700 0x7f070015ff50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f07000cd690 tx=0x7f06f001c4a0 comp rx=0 tx=0).stop 2024-09-19T08:45:53.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.879+0000 7f0707821640 1 -- 172.21.15.98:0/3773962144 shutdown_connections 2024-09-19T08:45:53.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.879+0000 7f0707821640 1 --2- 172.21.15.98:0/3773962144 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f07000cf700 0x7f070015ff50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:53.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.879+0000 7f0707821640 1 --2- 172.21.15.98:0/3773962144 >> v2:172.21.15.139:3300/0 conn(0x7f070015c290 0x7f07000cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:53.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.879+0000 7f0707821640 1 -- 172.21.15.98:0/3773962144 >> 172.21.15.98:0/3773962144 conn(0x7f070006c090 msgr2=0x7f070006c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:53.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.879+0000 7f0707821640 1 -- 172.21.15.98:0/3773962144 shutdown_connections 2024-09-19T08:45:53.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.879+0000 7f0707821640 1 -- 172.21.15.98:0/3773962144 wait complete. 2024-09-19T08:45:53.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.883+0000 7f0707821640 1 Processor -- start 2024-09-19T08:45:53.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.883+0000 7f0707821640 1 -- start start 2024-09-19T08:45:53.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.883+0000 7f0707821640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f070006d4c0 0x7f0700158a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:53.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.883+0000 7f0707821640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07000cf700 0x7f0700157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:53.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.883+0000 7f0707821640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f070015c290 0x7f0700157a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:53.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.883+0000 7f0707821640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f070015c7f0 con 0x7f070015c290 2024-09-19T08:45:53.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.883+0000 7f0707821640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f070015c670 con 0x7f07000cf700 2024-09-19T08:45:53.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.883+0000 7f0707821640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0700160490 con 0x7f070006d4c0 2024-09-19T08:45:53.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.883+0000 7f0705dbe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f070015c290 0x7f0700157a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:53.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.883+0000 7f0705dbe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f070015c290 0x7f0700157a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53984/0 (socket says 172.21.15.98:53984) 2024-09-19T08:45:53.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.883+0000 7f0705dbe640 1 -- 172.21.15.98:0/2604736954 learned_addr learned my addr 172.21.15.98:0/2604736954 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:53.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.883+0000 7f07055bd640 1 --2- 172.21.15.98:0/2604736954 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f070006d4c0 0x7f0700158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:53.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.883+0000 7f0704dbc640 1 --2- 172.21.15.98:0/2604736954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07000cf700 0x7f0700157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:53.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.883+0000 7f0705dbe640 1 -- 172.21.15.98:0/2604736954 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f070006d4c0 msgr2=0x7f0700158a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:53.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.883+0000 7f0705dbe640 1 --2- 172.21.15.98:0/2604736954 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f070006d4c0 0x7f0700158a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:53.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.883+0000 7f0705dbe640 1 -- 172.21.15.98:0/2604736954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07000cf700 msgr2=0x7f0700157550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:53.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.883+0000 7f0705dbe640 1 --2- 172.21.15.98:0/2604736954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07000cf700 0x7f0700157550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:53.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.883+0000 7f0705dbe640 1 -- 172.21.15.98:0/2604736954 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0700161b10 con 0x7f070015c290 2024-09-19T08:45:53.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.883+0000 7f07055bd640 1 --2- 172.21.15.98:0/2604736954 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f070006d4c0 0x7f0700158a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:45:53.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.883+0000 7f0705dbe640 1 --2- 172.21.15.98:0/2604736954 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f070015c290 0x7f0700157a90 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f06f800c700 tx=0x7f06f800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:53.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.883+0000 7f06f5ffb640 1 -- 172.21.15.98:0/2604736954 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f8018070 con 0x7f070015c290 2024-09-19T08:45:53.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.883+0000 7f06f5ffb640 1 -- 172.21.15.98:0/2604736954 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06f800ce10 con 0x7f070015c290 2024-09-19T08:45:53.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.883+0000 7f0707821640 1 -- 172.21.15.98:0/2604736954 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0700161d80 con 0x7f070015c290 2024-09-19T08:45:53.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.883+0000 7f06f5ffb640 1 -- 172.21.15.98:0/2604736954 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f800a860 con 0x7f070015c290 2024-09-19T08:45:53.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.883+0000 7f0707821640 1 -- 172.21.15.98:0/2604736954 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0700162620 con 0x7f070015c290 2024-09-19T08:45:53.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.883+0000 7f0707821640 1 -- 172.21.15.98:0/2604736954 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0700161f50 con 0x7f070015c290 2024-09-19T08:45:53.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.887+0000 7f06f5ffb640 1 -- 172.21.15.98:0/2604736954 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f06f800aa00 con 0x7f070015c290 2024-09-19T08:45:53.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.887+0000 7f06f5ffb640 1 --2- 172.21.15.98:0/2604736954 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f06e4067ef0 0x7f06e406a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:53.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.887+0000 7f06f5ffb640 1 -- 172.21.15.98:0/2604736954 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5385+0+0 (secure 0 0 0) 0x7f06f8085d10 con 0x7f070015c290 2024-09-19T08:45:53.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.887+0000 7f07055bd640 1 --2- 172.21.15.98:0/2604736954 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f06e4067ef0 0x7f06e406a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:53.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.887+0000 7f07055bd640 1 --2- 172.21.15.98:0/2604736954 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f06e4067ef0 0x7f06e406a3b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f06fc009bb0 tx=0x7f06fc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:53.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:53.891+0000 7f06f5ffb640 1 -- 172.21.15.98:0/2604736954 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f06f8055fd0 con 0x7f070015c290 2024-09-19T08:45:54.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.039+0000 7f0707821640 1 -- 172.21.15.98:0/2604736954 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f070015ff50 con 0x7f070015c290 2024-09-19T08:45:54.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.723+0000 7f06f5ffb640 1 -- 172.21.15.98:0/2604736954 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v138) ==== 169+0+0 (secure 0 0 0) 0x7f06f80558d0 con 0x7f070015c290 2024-09-19T08:45:54.724 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' removed 2024-09-19T08:45:54.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.727+0000 7f0707821640 1 -- 172.21.15.98:0/2604736954 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f06e4067ef0 msgr2=0x7f06e406a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:54.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.727+0000 7f0707821640 1 --2- 172.21.15.98:0/2604736954 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f06e4067ef0 0x7f06e406a3b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f06fc009bb0 tx=0x7f06fc008040 comp rx=0 tx=0).stop 2024-09-19T08:45:54.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.727+0000 7f0707821640 1 -- 172.21.15.98:0/2604736954 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f070015c290 msgr2=0x7f0700157a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:54.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.727+0000 7f0707821640 1 --2- 172.21.15.98:0/2604736954 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f070015c290 0x7f0700157a90 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f06f800c700 tx=0x7f06f800cbc0 comp rx=0 tx=0).stop 2024-09-19T08:45:54.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.727+0000 7f0707821640 1 -- 172.21.15.98:0/2604736954 shutdown_connections 2024-09-19T08:45:54.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.727+0000 7f0707821640 1 --2- 172.21.15.98:0/2604736954 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f06e4067ef0 0x7f06e406a3b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:54.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.727+0000 7f0707821640 1 --2- 172.21.15.98:0/2604736954 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f070015c290 0x7f0700157a90 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:54.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.727+0000 7f0707821640 1 --2- 172.21.15.98:0/2604736954 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f07000cf700 0x7f0700157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:54.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.727+0000 7f0707821640 1 --2- 172.21.15.98:0/2604736954 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f070006d4c0 0x7f0700158a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:54.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.727+0000 7f0707821640 1 -- 172.21.15.98:0/2604736954 >> 172.21.15.98:0/2604736954 conn(0x7f070006c090 msgr2=0x7f07000b61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:54.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.727+0000 7f0707821640 1 -- 172.21.15.98:0/2604736954 shutdown_connections 2024-09-19T08:45:54.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.727+0000 7f0707821640 1 -- 172.21.15.98:0/2604736954 wait complete. 2024-09-19T08:45:54.750 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-19T08:45:54.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.875+0000 7efcc742a640 1 Processor -- start 2024-09-19T08:45:54.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.875+0000 7efcc742a640 1 -- start start 2024-09-19T08:45:54.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.879+0000 7efcc742a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efcc00729f0 0x7efcc006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:54.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.879+0000 7efcc742a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7efcc006c1d0 0x7efcc00cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:54.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.879+0000 7efcc742a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcc0072ea0 con 0x7efcc0144d10 2024-09-19T08:45:54.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.879+0000 7efcc742a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7efcc0073ab0 con 0x7efcc006c1d0 2024-09-19T08:45:54.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.879+0000 7efcc742a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7efcc01450f0 con 0x7efcc00729f0 2024-09-19T08:45:54.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.879+0000 7efcc49c5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efcc00729f0 0x7efcc006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:54.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.879+0000 7efcc59c7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7efcc006c1d0 0x7efcc00cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:54.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.879+0000 7efcc51c6640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7efcc0144d10 0x7efcc00722e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:59802/0 (socket says 172.21.15.98:59802) 2024-09-19T08:45:54.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.879+0000 7efcc51c6640 1 -- 172.21.15.98:0/1031708689 learned_addr learned my addr 172.21.15.98:0/1031708689 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:54.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.879+0000 7efcb7fff640 1 -- 172.21.15.98:0/1031708689 <== mon.1 v1:172.21.15.139:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2398010870 0 0) 0x7efcc0072ea0 con 0x7efcc0144d10 2024-09-19T08:45:54.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.879+0000 7efcc49c5640 1 -- 172.21.15.98:0/1031708689 >> v2:172.21.15.139:3300/0 conn(0x7efcc006c1d0 msgr2=0x7efcc00cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:54.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.879+0000 7efcc49c5640 1 --2- 172.21.15.98:0/1031708689 >> v2:172.21.15.139:3300/0 conn(0x7efcc006c1d0 0x7efcc00cd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:54.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.879+0000 7efcc49c5640 1 -- 172.21.15.98:0/1031708689 >> v1:172.21.15.139:6789/0 conn(0x7efcc0144d10 legacy=0x7efcc00722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:54.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.879+0000 7efcc49c5640 1 -- 172.21.15.98:0/1031708689 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc0073180 con 0x7efcc00729f0 2024-09-19T08:45:54.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.883+0000 7efcc49c5640 1 --2- 172.21.15.98:0/1031708689 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efcc00729f0 0x7efcc006bc90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7efcb00097c0 tx=0x7efcb001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1205e647ba8bbcf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:54.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.883+0000 7efcb7fff640 1 -- 172.21.15.98:0/1031708689 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb00b2070 con 0x7efcc00729f0 2024-09-19T08:45:54.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.883+0000 7efcb7fff640 1 -- 172.21.15.98:0/1031708689 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcb00ad640 con 0x7efcc00729f0 2024-09-19T08:45:54.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.883+0000 7efcb7fff640 1 -- 172.21.15.98:0/1031708689 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb00ada10 con 0x7efcc00729f0 2024-09-19T08:45:54.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.883+0000 7efcc742a640 1 -- 172.21.15.98:0/1031708689 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efcc00729f0 msgr2=0x7efcc006bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:54.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.883+0000 7efcc742a640 1 --2- 172.21.15.98:0/1031708689 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efcc00729f0 0x7efcc006bc90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7efcb00097c0 tx=0x7efcb001c4a0 comp rx=0 tx=0).stop 2024-09-19T08:45:54.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.883+0000 7efcc742a640 1 -- 172.21.15.98:0/1031708689 shutdown_connections 2024-09-19T08:45:54.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.883+0000 7efcc742a640 1 --2- 172.21.15.98:0/1031708689 >> v2:172.21.15.139:3300/0 conn(0x7efcc006c1d0 0x7efcc00cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:54.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.883+0000 7efcc742a640 1 --2- 172.21.15.98:0/1031708689 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efcc00729f0 0x7efcc006bc90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:54.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.883+0000 7efcc742a640 1 -- 172.21.15.98:0/1031708689 >> 172.21.15.98:0/1031708689 conn(0x7efcc00b2010 msgr2=0x7efcc00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:54.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.883+0000 7efcc742a640 1 -- 172.21.15.98:0/1031708689 shutdown_connections 2024-09-19T08:45:54.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.883+0000 7efcc742a640 1 -- 172.21.15.98:0/1031708689 wait complete. 2024-09-19T08:45:54.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.883+0000 7efcc742a640 1 Processor -- start 2024-09-19T08:45:54.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.883+0000 7efcc742a640 1 -- start start 2024-09-19T08:45:54.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.887+0000 7efcc742a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efcc006c1d0 0x7efcc00ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:54.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.887+0000 7efcc742a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efcc00729f0 0x7efcc0142070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:54.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.887+0000 7efcc742a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcc0144d10 0x7efcc00b1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:54.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.887+0000 7efcc742a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efcc00ce120 con 0x7efcc006c1d0 2024-09-19T08:45:54.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.887+0000 7efcc742a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efcc00cdfa0 con 0x7efcc0144d10 2024-09-19T08:45:54.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.887+0000 7efcc742a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7efcc00ce2a0 con 0x7efcc00729f0 2024-09-19T08:45:54.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.887+0000 7efcc51c6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efcc006c1d0 0x7efcc00ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:54.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.887+0000 7efcc51c6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efcc006c1d0 0x7efcc00ac860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53992/0 (socket says 172.21.15.98:53992) 2024-09-19T08:45:54.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.887+0000 7efcc51c6640 1 -- 172.21.15.98:0/3929097072 learned_addr learned my addr 172.21.15.98:0/3929097072 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:54.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.887+0000 7efcc49c5640 1 --2- 172.21.15.98:0/3929097072 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efcc00729f0 0x7efcc0142070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:54.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.887+0000 7efcc59c7640 1 --2- 172.21.15.98:0/3929097072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcc0144d10 0x7efcc00b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:54.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.887+0000 7efcc51c6640 1 -- 172.21.15.98:0/3929097072 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efcc00729f0 msgr2=0x7efcc0142070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:54.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.887+0000 7efcc51c6640 1 --2- 172.21.15.98:0/3929097072 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efcc00729f0 0x7efcc0142070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:54.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.887+0000 7efcc51c6640 1 -- 172.21.15.98:0/3929097072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcc0144d10 msgr2=0x7efcc00b1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:54.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.887+0000 7efcc51c6640 1 --2- 172.21.15.98:0/3929097072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcc0144d10 0x7efcc00b1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:54.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.887+0000 7efcc51c6640 1 -- 172.21.15.98:0/3929097072 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc00ad0d0 con 0x7efcc006c1d0 2024-09-19T08:45:54.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.887+0000 7efcc49c5640 1 --2- 172.21.15.98:0/3929097072 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efcc00729f0 0x7efcc0142070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:54.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.887+0000 7efcc59c7640 1 --2- 172.21.15.98:0/3929097072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcc0144d10 0x7efcc00b1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:54.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.887+0000 7efcc51c6640 1 --2- 172.21.15.98:0/3929097072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efcc006c1d0 0x7efcc00ac860 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7efcbc00c4b0 tx=0x7efcbc00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:54.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.887+0000 7efcb5ffb640 1 -- 172.21.15.98:0/3929097072 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcbc018070 con 0x7efcc006c1d0 2024-09-19T08:45:54.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.887+0000 7efcb5ffb640 1 -- 172.21.15.98:0/3929097072 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcbc007500 con 0x7efcc006c1d0 2024-09-19T08:45:54.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.887+0000 7efcb5ffb640 1 -- 172.21.15.98:0/3929097072 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcbc004a30 con 0x7efcc006c1d0 2024-09-19T08:45:54.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.887+0000 7efcc742a640 1 -- 172.21.15.98:0/3929097072 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcc00ad3b0 con 0x7efcc006c1d0 2024-09-19T08:45:54.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.887+0000 7efcc742a640 1 -- 172.21.15.98:0/3929097072 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcc00adc50 con 0x7efcc006c1d0 2024-09-19T08:45:54.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.887+0000 7efcc742a640 1 -- 172.21.15.98:0/3929097072 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcc013c670 con 0x7efcc006c1d0 2024-09-19T08:45:54.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.887+0000 7efcb5ffb640 1 -- 172.21.15.98:0/3929097072 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7efcbc004bd0 con 0x7efcc006c1d0 2024-09-19T08:45:54.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.887+0000 7efcb5ffb640 1 --2- 172.21.15.98:0/3929097072 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7efca4067ef0 0x7efca406a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:54.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.887+0000 7efcb5ffb640 1 -- 172.21.15.98:0/3929097072 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 4971+0+0 (secure 0 0 0) 0x7efcbc085b60 con 0x7efcc006c1d0 2024-09-19T08:45:54.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.887+0000 7efcc49c5640 1 --2- 172.21.15.98:0/3929097072 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7efca4067ef0 0x7efca406a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:54.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.891+0000 7efcc49c5640 1 --2- 172.21.15.98:0/3929097072 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7efca4067ef0 0x7efca406a3b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efcb00af000 tx=0x7efcb0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:54.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:54.891+0000 7efcb5ffb640 1 -- 172.21.15.98:0/3929097072 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7efcbc056040 con 0x7efcc006c1d0 2024-09-19T08:45:55.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.055+0000 7efcc742a640 1 -- 172.21.15.98:0/3929097072 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7efcc00cd230 con 0x7efcc006c1d0 2024-09-19T08:45:55.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.059+0000 7efcb5ffb640 1 -- 172.21.15.98:0/3929097072 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) ==== 104+0+0 (secure 0 0 0) 0x7efcbc055940 con 0x7efcc006c1d0 2024-09-19T08:45:55.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.063+0000 7efcc742a640 1 -- 172.21.15.98:0/3929097072 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7efca4067ef0 msgr2=0x7efca406a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:55.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.063+0000 7efcc742a640 1 --2- 172.21.15.98:0/3929097072 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7efca4067ef0 0x7efca406a3b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efcb00af000 tx=0x7efcb0008040 comp rx=0 tx=0).stop 2024-09-19T08:45:55.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.063+0000 7efcc742a640 1 -- 172.21.15.98:0/3929097072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efcc006c1d0 msgr2=0x7efcc00ac860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:55.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.063+0000 7efcc742a640 1 --2- 172.21.15.98:0/3929097072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efcc006c1d0 0x7efcc00ac860 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7efcbc00c4b0 tx=0x7efcbc00c970 comp rx=0 tx=0).stop 2024-09-19T08:45:55.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.063+0000 7efcc742a640 1 -- 172.21.15.98:0/3929097072 shutdown_connections 2024-09-19T08:45:55.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.063+0000 7efcc742a640 1 --2- 172.21.15.98:0/3929097072 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7efca4067ef0 0x7efca406a3b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:55.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.063+0000 7efcc742a640 1 --2- 172.21.15.98:0/3929097072 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efcc0144d10 0x7efcc00b1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:55.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.063+0000 7efcc742a640 1 --2- 172.21.15.98:0/3929097072 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efcc00729f0 0x7efcc0142070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:55.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.063+0000 7efcc742a640 1 --2- 172.21.15.98:0/3929097072 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efcc006c1d0 0x7efcc00ac860 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:55.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.063+0000 7efcc742a640 1 -- 172.21.15.98:0/3929097072 >> 172.21.15.98:0/3929097072 conn(0x7efcc00b2010 msgr2=0x7efcc00b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:55.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.063+0000 7efcc742a640 1 -- 172.21.15.98:0/3929097072 shutdown_connections 2024-09-19T08:45:55.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.063+0000 7efcc742a640 1 -- 172.21.15.98:0/3929097072 wait complete. 2024-09-19T08:45:55.079 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol' 2024-09-19T08:45:55.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.203+0000 7f1036f4e640 1 Processor -- start 2024-09-19T08:45:55.201 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.203+0000 7f1036f4e640 1 -- start start 2024-09-19T08:45:55.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.203+0000 7f1036f4e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10300d4820 0x7f10301621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:55.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.203+0000 7f1036f4e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f10300b9970 0x7f10300b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:55.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.203+0000 7f1036f4e640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1030068440 con 0x7f10300b7f80 2024-09-19T08:45:55.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.203+0000 7f1036f4e640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f1030163050 con 0x7f10300b9970 2024-09-19T08:45:55.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.203+0000 7f1036f4e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f10300ba360 con 0x7f10300d4820 2024-09-19T08:45:55.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.203+0000 7f10354eb640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f10300b9970 0x7f10300b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:55.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.203+0000 7f1034cea640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f10300b7f80 0x7f103015f560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42482/0 (socket says 172.21.15.98:42482) 2024-09-19T08:45:55.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.203+0000 7f1034cea640 1 -- 172.21.15.98:0/2007868092 learned_addr learned my addr 172.21.15.98:0/2007868092 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:55.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.203+0000 7f10354eb640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f10300b9970 0x7f10300b9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:53994/0 (socket says 172.21.15.98:53994) 2024-09-19T08:45:55.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.203+0000 7f102ffff640 1 --2- 172.21.15.98:0/2007868092 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10300d4820 0x7f10301621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:55.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.203+0000 7f10354eb640 1 -- 172.21.15.98:0/2007868092 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10300d4820 msgr2=0x7f10301621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:55.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.203+0000 7f10354eb640 1 --2- 172.21.15.98:0/2007868092 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10300d4820 0x7f10301621b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:55.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.203+0000 7f10354eb640 1 -- 172.21.15.98:0/2007868092 >> v1:172.21.15.98:6789/0 conn(0x7f10300b7f80 legacy=0x7f103015f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:55.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.203+0000 7f10354eb640 1 -- 172.21.15.98:0/2007868092 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1030162a90 con 0x7f10300b9970 2024-09-19T08:45:55.203 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.203+0000 7f102ffff640 1 --2- 172.21.15.98:0/2007868092 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10300d4820 0x7f10301621b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:55.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.207+0000 7f10354eb640 1 --2- 172.21.15.98:0/2007868092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10300b9970 0x7f10300b9e20 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f1028009d90 tx=0x7f102801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=854b65679cb19ab0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:55.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.207+0000 7f102f7fe640 1 -- 172.21.15.98:0/2007868092 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10280b2070 con 0x7f10300b9970 2024-09-19T08:45:55.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.207+0000 7f102f7fe640 1 -- 172.21.15.98:0/2007868092 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10280a5920 con 0x7f10300b9970 2024-09-19T08:45:55.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.207+0000 7f102f7fe640 1 -- 172.21.15.98:0/2007868092 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10280a5cf0 con 0x7f10300b9970 2024-09-19T08:45:55.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.207+0000 7f1036f4e640 1 -- 172.21.15.98:0/2007868092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10300b9970 msgr2=0x7f10300b9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:55.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.207+0000 7f1036f4e640 1 --2- 172.21.15.98:0/2007868092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10300b9970 0x7f10300b9e20 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f1028009d90 tx=0x7f102801c970 comp rx=0 tx=0).stop 2024-09-19T08:45:55.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.207+0000 7f1036f4e640 1 -- 172.21.15.98:0/2007868092 shutdown_connections 2024-09-19T08:45:55.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.207+0000 7f1036f4e640 1 --2- 172.21.15.98:0/2007868092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10300b9970 0x7f10300b9e20 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:55.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.207+0000 7f1036f4e640 1 --2- 172.21.15.98:0/2007868092 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10300d4820 0x7f10301621b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:55.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.207+0000 7f1036f4e640 1 -- 172.21.15.98:0/2007868092 >> 172.21.15.98:0/2007868092 conn(0x7f1030074e40 msgr2=0x7f1030075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:55.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.207+0000 7f1036f4e640 1 -- 172.21.15.98:0/2007868092 shutdown_connections 2024-09-19T08:45:55.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.207+0000 7f1036f4e640 1 -- 172.21.15.98:0/2007868092 wait complete. 2024-09-19T08:45:55.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.207+0000 7f1036f4e640 1 Processor -- start 2024-09-19T08:45:55.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.207+0000 7f1036f4e640 1 -- start start 2024-09-19T08:45:55.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.207+0000 7f1036f4e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10300b7f80 0x7f1030161a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:55.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.207+0000 7f1036f4e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10300b9970 0x7f103014b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:55.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.207+0000 7f1036f4e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10300d4820 0x7f103014be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:55.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.207+0000 7f1036f4e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f10300b84e0 con 0x7f10300b7f80 2024-09-19T08:45:55.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.207+0000 7f1036f4e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f10300b8360 con 0x7f10300d4820 2024-09-19T08:45:55.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.207+0000 7f1036f4e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1030144590 con 0x7f10300b9970 2024-09-19T08:45:55.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.207+0000 7f1034cea640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10300b7f80 0x7f1030161a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:55.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.207+0000 7f1034cea640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10300b7f80 0x7f1030161a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54004/0 (socket says 172.21.15.98:54004) 2024-09-19T08:45:55.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.207+0000 7f1034cea640 1 -- 172.21.15.98:0/2725431790 learned_addr learned my addr 172.21.15.98:0/2725431790 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:55.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.207+0000 7f1034cea640 1 -- 172.21.15.98:0/2725431790 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10300b9970 msgr2=0x7f103014b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:55.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.207+0000 7f1034cea640 1 --2- 172.21.15.98:0/2725431790 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10300b9970 0x7f103014b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:55.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.207+0000 7f1034cea640 1 -- 172.21.15.98:0/2725431790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10300d4820 msgr2=0x7f103014be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:55.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.207+0000 7f1034cea640 1 --2- 172.21.15.98:0/2725431790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10300d4820 0x7f103014be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:55.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.207+0000 7f1034cea640 1 -- 172.21.15.98:0/2725431790 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f103014c460 con 0x7f10300b7f80 2024-09-19T08:45:55.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.207+0000 7f1034cea640 1 --2- 172.21.15.98:0/2725431790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10300b7f80 0x7f1030161a80 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f102000c3a0 tx=0x7f102000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:55.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.211+0000 7f102d7fa640 1 -- 172.21.15.98:0/2725431790 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1020017070 con 0x7f10300b7f80 2024-09-19T08:45:55.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.211+0000 7f102d7fa640 1 -- 172.21.15.98:0/2725431790 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1020005cf0 con 0x7f10300b7f80 2024-09-19T08:45:55.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.211+0000 7f1036f4e640 1 -- 172.21.15.98:0/2725431790 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10300d28a0 con 0x7f10300b7f80 2024-09-19T08:45:55.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.211+0000 7f102d7fa640 1 -- 172.21.15.98:0/2725431790 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10200072f0 con 0x7f10300b7f80 2024-09-19T08:45:55.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.211+0000 7f1036f4e640 1 -- 172.21.15.98:0/2725431790 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f103013be70 con 0x7f10300b7f80 2024-09-19T08:45:55.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.211+0000 7f1036f4e640 1 -- 172.21.15.98:0/2725431790 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10300b9e20 con 0x7f10300b7f80 2024-09-19T08:45:55.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.211+0000 7f102d7fa640 1 -- 172.21.15.98:0/2725431790 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f102001a070 con 0x7f10300b7f80 2024-09-19T08:45:55.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.211+0000 7f102d7fa640 1 --2- 172.21.15.98:0/2725431790 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f101c067ef0 0x7f101c06a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:55.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.211+0000 7f102d7fa640 1 -- 172.21.15.98:0/2725431790 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 4971+0+0 (secure 0 0 0) 0x7f1020085c20 con 0x7f10300b7f80 2024-09-19T08:45:55.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.211+0000 7f102ffff640 1 --2- 172.21.15.98:0/2725431790 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f101c067ef0 0x7f101c06a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:55.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.211+0000 7f102ffff640 1 --2- 172.21.15.98:0/2725431790 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f101c067ef0 0x7f101c06a3b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f10300cfae0 tx=0x7f1024008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:55.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.215+0000 7f102d7fa640 1 -- 172.21.15.98:0/2725431790 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f1020085e20 con 0x7f10300b7f80 2024-09-19T08:45:55.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.435+0000 7f1036f4e640 1 -- 172.21.15.98:0/2725431790 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]} v 0) -- 0x7f103013c160 con 0x7f10300b7f80 2024-09-19T08:45:55.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.731+0000 7f102d7fa640 1 -- 172.21.15.98:0/2725431790 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]}]=0 v398) ==== 151+0+0 (secure 0 0 0) 0x7f1020021020 con 0x7f10300b7f80 2024-09-19T08:45:55.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.735+0000 7f1036f4e640 1 -- 172.21.15.98:0/2725431790 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f101c067ef0 msgr2=0x7f101c06a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:55.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.735+0000 7f1036f4e640 1 --2- 172.21.15.98:0/2725431790 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f101c067ef0 0x7f101c06a3b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f10300cfae0 tx=0x7f1024008040 comp rx=0 tx=0).stop 2024-09-19T08:45:55.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.735+0000 7f1036f4e640 1 -- 172.21.15.98:0/2725431790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10300b7f80 msgr2=0x7f1030161a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:55.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.735+0000 7f1036f4e640 1 --2- 172.21.15.98:0/2725431790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10300b7f80 0x7f1030161a80 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f102000c3a0 tx=0x7f102000c860 comp rx=0 tx=0).stop 2024-09-19T08:45:55.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.735+0000 7f1036f4e640 1 -- 172.21.15.98:0/2725431790 shutdown_connections 2024-09-19T08:45:55.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.739+0000 7f1036f4e640 1 --2- 172.21.15.98:0/2725431790 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f101c067ef0 0x7f101c06a3b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:55.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.739+0000 7f1036f4e640 1 --2- 172.21.15.98:0/2725431790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10300d4820 0x7f103014be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:55.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.739+0000 7f1036f4e640 1 --2- 172.21.15.98:0/2725431790 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10300b9970 0x7f103014b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:55.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.739+0000 7f1036f4e640 1 --2- 172.21.15.98:0/2725431790 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10300b7f80 0x7f1030161a80 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:55.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.739+0000 7f1036f4e640 1 -- 172.21.15.98:0/2725431790 >> 172.21.15.98:0/2725431790 conn(0x7f1030074e40 msgr2=0x7f1030156300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:55.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.739+0000 7f1036f4e640 1 -- 172.21.15.98:0/2725431790 shutdown_connections 2024-09-19T08:45:55.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.739+0000 7f1036f4e640 1 -- 172.21.15.98:0/2725431790 wait complete. 2024-09-19T08:45:55.760 INFO:tasks.cephfs_test_runner:test_subvolume_ops_on_nonexistent_vol (tasks.cephfs.test_volumes.TestMisc) ... ok 2024-09-19T08:45:55.760 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_upgrade_legacy_to_v1 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- rather than going through a full upgrade cycle, 2024-09-19T08:45:55.760 INFO:tasks.cephfs_test_runner:test_subvolume_upgrade_legacy_to_v1 (tasks.cephfs.test_volumes.TestMisc) 2024-09-19T08:45:55.761 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:45:55.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.887+0000 7fcf362fb640 1 Processor -- start 2024-09-19T08:45:55.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.887+0000 7fcf362fb640 1 -- start start 2024-09-19T08:45:55.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.887+0000 7fcf362fb640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcf3006bfa0 0x7fcf3006c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:55.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.887+0000 7fcf362fb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcf30160e60 0x7fcf301612d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:55.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.887+0000 7fcf362fb640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf3015fb50 con 0x7fcf3015f590 2024-09-19T08:45:55.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.887+0000 7fcf362fb640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fcf3006c930 con 0x7fcf3006bfa0 2024-09-19T08:45:55.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.887+0000 7fcf362fb640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcf3015f970 con 0x7fcf30160e60 2024-09-19T08:45:55.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.887+0000 7fcf2f7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcf3006bfa0 0x7fcf3006c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:55.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.887+0000 7fcf2f7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcf3006bfa0 0x7fcf3006c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54010/0 (socket says 172.21.15.98:54010) 2024-09-19T08:45:55.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.887+0000 7fcf2f7fe640 1 -- 172.21.15.98:0/3277875955 learned_addr learned my addr 172.21.15.98:0/3277875955 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:55.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.887+0000 7fcf34898640 1 --2- 172.21.15.98:0/3277875955 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcf30160e60 0x7fcf301612d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:55.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.887+0000 7fcf2f7fe640 1 -- 172.21.15.98:0/3277875955 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcf30160e60 msgr2=0x7fcf301612d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:55.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.887+0000 7fcf2f7fe640 1 --2- 172.21.15.98:0/3277875955 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcf30160e60 0x7fcf301612d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:55.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.887+0000 7fcf2f7fe640 1 -- 172.21.15.98:0/3277875955 >> v1:172.21.15.139:6789/0 conn(0x7fcf3015f590 legacy=0x7fcf300cf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:55.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.887+0000 7fcf2f7fe640 1 -- 172.21.15.98:0/3277875955 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf3015fdc0 con 0x7fcf3006bfa0 2024-09-19T08:45:55.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.891+0000 7fcf2f7fe640 1 --2- 172.21.15.98:0/3277875955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf3006bfa0 0x7fcf3006c3f0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fcf24009870 tx=0x7fcf2401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fcf1ca26d8739d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:55.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.891+0000 7fcf2effd640 1 -- 172.21.15.98:0/3277875955 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf240b5070 con 0x7fcf3006bfa0 2024-09-19T08:45:55.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.891+0000 7fcf2effd640 1 -- 172.21.15.98:0/3277875955 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf240a89d0 con 0x7fcf3006bfa0 2024-09-19T08:45:55.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.891+0000 7fcf2effd640 1 -- 172.21.15.98:0/3277875955 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf240a8da0 con 0x7fcf3006bfa0 2024-09-19T08:45:55.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.891+0000 7fcf362fb640 1 -- 172.21.15.98:0/3277875955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf3006bfa0 msgr2=0x7fcf3006c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:55.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.891+0000 7fcf362fb640 1 --2- 172.21.15.98:0/3277875955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf3006bfa0 0x7fcf3006c3f0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fcf24009870 tx=0x7fcf2401ed10 comp rx=0 tx=0).stop 2024-09-19T08:45:55.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.891+0000 7fcf362fb640 1 -- 172.21.15.98:0/3277875955 shutdown_connections 2024-09-19T08:45:55.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.891+0000 7fcf362fb640 1 --2- 172.21.15.98:0/3277875955 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcf30160e60 0x7fcf301612d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:55.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.891+0000 7fcf362fb640 1 --2- 172.21.15.98:0/3277875955 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf3006bfa0 0x7fcf3006c3f0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:55.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.891+0000 7fcf362fb640 1 -- 172.21.15.98:0/3277875955 >> 172.21.15.98:0/3277875955 conn(0x7fcf30074c70 msgr2=0x7fcf30075080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:55.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.891+0000 7fcf362fb640 1 -- 172.21.15.98:0/3277875955 shutdown_connections 2024-09-19T08:45:55.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.891+0000 7fcf362fb640 1 -- 172.21.15.98:0/3277875955 wait complete. 2024-09-19T08:45:55.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.891+0000 7fcf362fb640 1 Processor -- start 2024-09-19T08:45:55.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.891+0000 7fcf362fb640 1 -- start start 2024-09-19T08:45:55.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.891+0000 7fcf362fb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf3006bfa0 0x7fcf30158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:55.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.891+0000 7fcf362fb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf3015f590 0x7fcf30153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:55.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.891+0000 7fcf362fb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcf30160e60 0x7fcf30162560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:55.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.891+0000 7fcf362fb640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcf30160660 con 0x7fcf3006bfa0 2024-09-19T08:45:55.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.891+0000 7fcf362fb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcf30160380 con 0x7fcf3015f590 2024-09-19T08:45:55.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.891+0000 7fcf362fb640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcf30160940 con 0x7fcf30160e60 2024-09-19T08:45:55.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.895+0000 7fcf2ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf3006bfa0 0x7fcf30158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:55.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.895+0000 7fcf2ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf3006bfa0 0x7fcf30158fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54024/0 (socket says 172.21.15.98:54024) 2024-09-19T08:45:55.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.895+0000 7fcf2ffff640 1 -- 172.21.15.98:0/1249656377 learned_addr learned my addr 172.21.15.98:0/1249656377 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:55.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.895+0000 7fcf2f7fe640 1 --2- 172.21.15.98:0/1249656377 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf3015f590 0x7fcf30153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:55.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.895+0000 7fcf2ffff640 1 -- 172.21.15.98:0/1249656377 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcf30160e60 msgr2=0x7fcf30162560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:55.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.895+0000 7fcf34898640 1 --2- 172.21.15.98:0/1249656377 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcf30160e60 0x7fcf30162560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:55.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.895+0000 7fcf2ffff640 1 --2- 172.21.15.98:0/1249656377 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcf30160e60 0x7fcf30162560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:55.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.895+0000 7fcf2ffff640 1 -- 172.21.15.98:0/1249656377 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf3015f590 msgr2=0x7fcf30153cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:55.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.895+0000 7fcf2ffff640 1 --2- 172.21.15.98:0/1249656377 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf3015f590 0x7fcf30153cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:55.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.895+0000 7fcf2ffff640 1 -- 172.21.15.98:0/1249656377 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf30156770 con 0x7fcf3006bfa0 2024-09-19T08:45:55.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.895+0000 7fcf2f7fe640 1 --2- 172.21.15.98:0/1249656377 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf3015f590 0x7fcf30153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:55.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.895+0000 7fcf2ffff640 1 --2- 172.21.15.98:0/1249656377 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf3006bfa0 0x7fcf30158fc0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fcf2000c450 tx=0x7fcf2000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:55.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.895+0000 7fcf2cff9640 1 -- 172.21.15.98:0/1249656377 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf20011790 con 0x7fcf3006bfa0 2024-09-19T08:45:55.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.895+0000 7fcf2cff9640 1 -- 172.21.15.98:0/1249656377 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf20017070 con 0x7fcf3006bfa0 2024-09-19T08:45:55.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.895+0000 7fcf362fb640 1 -- 172.21.15.98:0/1249656377 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf301569f0 con 0x7fcf3006bfa0 2024-09-19T08:45:55.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.895+0000 7fcf2cff9640 1 -- 172.21.15.98:0/1249656377 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf20011b90 con 0x7fcf3006bfa0 2024-09-19T08:45:55.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.895+0000 7fcf362fb640 1 -- 172.21.15.98:0/1249656377 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf3015b4b0 con 0x7fcf3006bfa0 2024-09-19T08:45:55.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.895+0000 7fcf362fb640 1 -- 172.21.15.98:0/1249656377 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf3015c5f0 con 0x7fcf3006bfa0 2024-09-19T08:45:55.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.895+0000 7fcf2cff9640 1 -- 172.21.15.98:0/1249656377 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fcf20011d30 con 0x7fcf3006bfa0 2024-09-19T08:45:55.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.895+0000 7fcf2cff9640 1 --2- 172.21.15.98:0/1249656377 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcf1c067f20 0x7fcf1c06a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:55.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.895+0000 7fcf2cff9640 1 -- 172.21.15.98:0/1249656377 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 4971+0+0 (secure 0 0 0) 0x7fcf200862a0 con 0x7fcf3006bfa0 2024-09-19T08:45:55.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.895+0000 7fcf2f7fe640 1 --2- 172.21.15.98:0/1249656377 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcf1c067f20 0x7fcf1c06a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:55.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.895+0000 7fcf2f7fe640 1 --2- 172.21.15.98:0/1249656377 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcf1c067f20 0x7fcf1c06a3e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fcf240b2000 tx=0x7fcf240aa070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:55.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:55.899+0000 7fcf2cff9640 1 -- 172.21.15.98:0/1249656377 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fcf30156770 con 0x7fcf3006bfa0 2024-09-19T08:45:56.061 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.063+0000 7fcf362fb640 1 -- 172.21.15.98:0/1249656377 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcf300b9770 con 0x7fcf3006bfa0 2024-09-19T08:45:56.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.063+0000 7fcf2cff9640 1 -- 172.21.15.98:0/1249656377 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) ==== 74+0+13088 (secure 0 0 0) 0x7fcf20056810 con 0x7fcf3006bfa0 2024-09-19T08:45:56.063 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:45:56.063 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":138,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:45:54.723917+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.139:6835/4255670104":"2024-09-20T08:45:51.710775+0000","172.21.15.139:6833/4255670104":"2024-09-20T08:45:51.710775+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:45:56.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.067+0000 7fcf362fb640 1 -- 172.21.15.98:0/1249656377 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcf1c067f20 msgr2=0x7fcf1c06a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:56.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.067+0000 7fcf362fb640 1 --2- 172.21.15.98:0/1249656377 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcf1c067f20 0x7fcf1c06a3e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fcf240b2000 tx=0x7fcf240aa070 comp rx=0 tx=0).stop 2024-09-19T08:45:56.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.067+0000 7fcf362fb640 1 -- 172.21.15.98:0/1249656377 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf3006bfa0 msgr2=0x7fcf30158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:56.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.067+0000 7fcf362fb640 1 --2- 172.21.15.98:0/1249656377 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf3006bfa0 0x7fcf30158fc0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fcf2000c450 tx=0x7fcf2000c910 comp rx=0 tx=0).stop 2024-09-19T08:45:56.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.067+0000 7fcf362fb640 1 -- 172.21.15.98:0/1249656377 shutdown_connections 2024-09-19T08:45:56.065 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.067+0000 7fcf362fb640 1 --2- 172.21.15.98:0/1249656377 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcf1c067f20 0x7fcf1c06a3e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.067+0000 7fcf362fb640 1 --2- 172.21.15.98:0/1249656377 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcf30160e60 0x7fcf30162560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.067+0000 7fcf362fb640 1 --2- 172.21.15.98:0/1249656377 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf3015f590 0x7fcf30153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.067+0000 7fcf362fb640 1 --2- 172.21.15.98:0/1249656377 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcf3006bfa0 0x7fcf30158fc0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.067+0000 7fcf362fb640 1 -- 172.21.15.98:0/1249656377 >> 172.21.15.98:0/1249656377 conn(0x7fcf30074c70 msgr2=0x7fcf300b9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:56.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.067+0000 7fcf362fb640 1 -- 172.21.15.98:0/1249656377 shutdown_connections 2024-09-19T08:45:56.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.067+0000 7fcf362fb640 1 -- 172.21.15.98:0/1249656377 wait complete. 2024-09-19T08:45:56.082 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T08:38:08.105684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T08:45:56.083 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T08:45:56.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.211+0000 7fa53f976640 1 Processor -- start 2024-09-19T08:45:56.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.211+0000 7fa53f976640 1 -- start start 2024-09-19T08:45:56.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.211+0000 7fa53f976640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa53815cf90 0x7fa53806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.211+0000 7fa53f976640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa53806c1d0 0x7fa53816c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.211+0000 7fa53f976640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5380ce3c0 con 0x7fa53815cbb0 2024-09-19T08:45:56.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.211+0000 7fa53f976640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fa5380cd090 con 0x7fa53815cf90 2024-09-19T08:45:56.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.211+0000 7fa53f976640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fa5380cd370 con 0x7fa53806c1d0 2024-09-19T08:45:56.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.211+0000 7fa53cf11640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa53815cf90 0x7fa53806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.211+0000 7fa53d712640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fa53815cbb0 0x7fa538163930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42494/0 (socket says 172.21.15.98:42494) 2024-09-19T08:45:56.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.211+0000 7fa53cf11640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa53815cf90 0x7fa53806bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54034/0 (socket says 172.21.15.98:54034) 2024-09-19T08:45:56.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.211+0000 7fa53d712640 1 -- 172.21.15.98:0/3143585252 learned_addr learned my addr 172.21.15.98:0/3143585252 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:56.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.211+0000 7fa53df13640 1 --2- 172.21.15.98:0/3143585252 >> v2:172.21.15.139:3300/0 conn(0x7fa53806c1d0 0x7fa53816c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.211+0000 7fa53cf11640 1 -- 172.21.15.98:0/3143585252 >> v2:172.21.15.139:3300/0 conn(0x7fa53806c1d0 msgr2=0x7fa53816c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:56.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.211+0000 7fa53cf11640 1 --2- 172.21.15.98:0/3143585252 >> v2:172.21.15.139:3300/0 conn(0x7fa53806c1d0 0x7fa53816c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.211+0000 7fa53cf11640 1 -- 172.21.15.98:0/3143585252 >> v1:172.21.15.98:6789/0 conn(0x7fa53815cbb0 legacy=0x7fa538163930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:56.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.211+0000 7fa52bfff640 1 -- 172.21.15.98:0/3143585252 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2124747090 0 0) 0x7fa5380ce3c0 con 0x7fa53815cbb0 2024-09-19T08:45:56.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.211+0000 7fa53cf11640 1 -- 172.21.15.98:0/3143585252 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5380cd650 con 0x7fa53815cf90 2024-09-19T08:45:56.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.211+0000 7fa53df13640 1 --2- 172.21.15.98:0/3143585252 >> v2:172.21.15.139:3300/0 conn(0x7fa53806c1d0 0x7fa53816c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:56.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.215+0000 7fa53cf11640 1 --2- 172.21.15.98:0/3143585252 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa53815cf90 0x7fa53806bc90 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fa5380cfc00 tx=0x7fa53401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d24a4a1fcd6b3572 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:56.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.215+0000 7fa52bfff640 1 -- 172.21.15.98:0/3143585252 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5340b0070 con 0x7fa53815cf90 2024-09-19T08:45:56.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.215+0000 7fa52bfff640 1 -- 172.21.15.98:0/3143585252 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5340ab600 con 0x7fa53815cf90 2024-09-19T08:45:56.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.215+0000 7fa52bfff640 1 -- 172.21.15.98:0/3143585252 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5340ab9d0 con 0x7fa53815cf90 2024-09-19T08:45:56.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.215+0000 7fa53f976640 1 -- 172.21.15.98:0/3143585252 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa53815cf90 msgr2=0x7fa53806bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:56.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.215+0000 7fa53f976640 1 --2- 172.21.15.98:0/3143585252 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa53815cf90 0x7fa53806bc90 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fa5380cfc00 tx=0x7fa53401c480 comp rx=0 tx=0).stop 2024-09-19T08:45:56.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.215+0000 7fa53f976640 1 -- 172.21.15.98:0/3143585252 shutdown_connections 2024-09-19T08:45:56.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.215+0000 7fa53f976640 1 --2- 172.21.15.98:0/3143585252 >> v2:172.21.15.139:3300/0 conn(0x7fa53806c1d0 0x7fa53816c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.215+0000 7fa53f976640 1 --2- 172.21.15.98:0/3143585252 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa53815cf90 0x7fa53806bc90 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.215+0000 7fa53f976640 1 -- 172.21.15.98:0/3143585252 >> 172.21.15.98:0/3143585252 conn(0x7fa5380b2010 msgr2=0x7fa5380b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:56.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.215+0000 7fa53f976640 1 -- 172.21.15.98:0/3143585252 shutdown_connections 2024-09-19T08:45:56.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.215+0000 7fa53f976640 1 -- 172.21.15.98:0/3143585252 wait complete. 2024-09-19T08:45:56.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.215+0000 7fa53f976640 1 Processor -- start 2024-09-19T08:45:56.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.215+0000 7fa53f976640 1 -- start start 2024-09-19T08:45:56.216 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.215+0000 7fa53f976640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa53806c1d0 0x7fa5381572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.216 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.215+0000 7fa53f976640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa53815cbb0 0x7fa5380b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.216 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.215+0000 7fa53f976640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa53815cf90 0x7fa538158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.216 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.215+0000 7fa53f976640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa5380cce70 con 0x7fa53806c1d0 2024-09-19T08:45:56.216 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.215+0000 7fa53f976640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa5380cccf0 con 0x7fa53815cf90 2024-09-19T08:45:56.216 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.215+0000 7fa53f976640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa53816f360 con 0x7fa53815cbb0 2024-09-19T08:45:56.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.215+0000 7fa53d712640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa53806c1d0 0x7fa5381572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.215+0000 7fa53d712640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa53806c1d0 0x7fa5381572f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54046/0 (socket says 172.21.15.98:54046) 2024-09-19T08:45:56.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.215+0000 7fa53d712640 1 -- 172.21.15.98:0/3546317542 learned_addr learned my addr 172.21.15.98:0/3546317542 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:56.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.215+0000 7fa53df13640 1 --2- 172.21.15.98:0/3546317542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa53815cf90 0x7fa538158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.215+0000 7fa53d712640 1 -- 172.21.15.98:0/3546317542 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa53815cbb0 msgr2=0x7fa5380b47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:56.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.215+0000 7fa53d712640 1 --2- 172.21.15.98:0/3546317542 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa53815cbb0 0x7fa5380b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.215+0000 7fa53d712640 1 -- 172.21.15.98:0/3546317542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa53815cf90 msgr2=0x7fa538158ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:56.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.215+0000 7fa53d712640 1 --2- 172.21.15.98:0/3546317542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa53815cf90 0x7fa538158ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.215+0000 7fa53d712640 1 -- 172.21.15.98:0/3546317542 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa538157a40 con 0x7fa53806c1d0 2024-09-19T08:45:56.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.215+0000 7fa53d712640 1 --2- 172.21.15.98:0/3546317542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa53806c1d0 0x7fa5381572f0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fa53000c890 tx=0x7fa53000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:56.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.215+0000 7fa529ffb640 1 -- 172.21.15.98:0/3546317542 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa530018070 con 0x7fa53806c1d0 2024-09-19T08:45:56.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.219+0000 7fa529ffb640 1 -- 172.21.15.98:0/3546317542 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa53000a640 con 0x7fa53806c1d0 2024-09-19T08:45:56.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.219+0000 7fa529ffb640 1 -- 172.21.15.98:0/3546317542 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53000aa30 con 0x7fa53806c1d0 2024-09-19T08:45:56.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.219+0000 7fa53f976640 1 -- 172.21.15.98:0/3546317542 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa538157d20 con 0x7fa53806c1d0 2024-09-19T08:45:56.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.219+0000 7fa53f976640 1 -- 172.21.15.98:0/3546317542 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5381582e0 con 0x7fa53806c1d0 2024-09-19T08:45:56.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.219+0000 7fa53f976640 1 -- 172.21.15.98:0/3546317542 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa53806bc90 con 0x7fa53806c1d0 2024-09-19T08:45:56.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.219+0000 7fa529ffb640 1 -- 172.21.15.98:0/3546317542 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fa53000abd0 con 0x7fa53806c1d0 2024-09-19T08:45:56.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.219+0000 7fa529ffb640 1 --2- 172.21.15.98:0/3546317542 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa51c067ef0 0x7fa51c06a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.219+0000 7fa529ffb640 1 -- 172.21.15.98:0/3546317542 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 4971+0+0 (secure 0 0 0) 0x7fa530085af0 con 0x7fa53806c1d0 2024-09-19T08:45:56.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.219+0000 7fa53cf11640 1 --2- 172.21.15.98:0/3546317542 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa51c067ef0 0x7fa51c06a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.219+0000 7fa53cf11640 1 --2- 172.21.15.98:0/3546317542 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa51c067ef0 0x7fa51c06a3b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa5340ad000 tx=0x7fa534008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:56.222 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.223+0000 7fa529ffb640 1 -- 172.21.15.98:0/3546317542 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa530056060 con 0x7fa53806c1d0 2024-09-19T08:45:56.385 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.387+0000 7fa53f976640 1 -- 172.21.15.98:0/3546317542 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa538161da0 con 0x7fa53806c1d0 2024-09-19T08:45:56.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.387+0000 7fa529ffb640 1 -- 172.21.15.98:0/3546317542 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v138) ==== 93+0+10 (secure 0 0 0) 0x7fa530055960 con 0x7fa53806c1d0 2024-09-19T08:45:56.386 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2024-09-19T08:45:56.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.391+0000 7fa53f976640 1 -- 172.21.15.98:0/3546317542 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa51c067ef0 msgr2=0x7fa51c06a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:56.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.391+0000 7fa53f976640 1 --2- 172.21.15.98:0/3546317542 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa51c067ef0 0x7fa51c06a3b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa5340ad000 tx=0x7fa534008040 comp rx=0 tx=0).stop 2024-09-19T08:45:56.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.391+0000 7fa53f976640 1 -- 172.21.15.98:0/3546317542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa53806c1d0 msgr2=0x7fa5381572f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:56.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.391+0000 7fa53f976640 1 --2- 172.21.15.98:0/3546317542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa53806c1d0 0x7fa5381572f0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fa53000c890 tx=0x7fa53000cd50 comp rx=0 tx=0).stop 2024-09-19T08:45:56.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.391+0000 7fa53f976640 1 -- 172.21.15.98:0/3546317542 shutdown_connections 2024-09-19T08:45:56.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.391+0000 7fa53f976640 1 --2- 172.21.15.98:0/3546317542 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa51c067ef0 0x7fa51c06a3b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.391+0000 7fa53f976640 1 --2- 172.21.15.98:0/3546317542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa53815cf90 0x7fa538158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.391+0000 7fa53f976640 1 --2- 172.21.15.98:0/3546317542 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa53815cbb0 0x7fa5380b47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.391+0000 7fa53f976640 1 --2- 172.21.15.98:0/3546317542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa53806c1d0 0x7fa5381572f0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.391+0000 7fa53f976640 1 -- 172.21.15.98:0/3546317542 >> 172.21.15.98:0/3546317542 conn(0x7fa5380b2010 msgr2=0x7fa538160cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:56.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.391+0000 7fa53f976640 1 -- 172.21.15.98:0/3546317542 shutdown_connections 2024-09-19T08:45:56.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.391+0000 7fa53f976640 1 -- 172.21.15.98:0/3546317542 wait complete. 2024-09-19T08:45:56.405 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1' 2024-09-19T08:45:56.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.519+0000 7fa8dd165640 1 Processor -- start 2024-09-19T08:45:56.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.519+0000 7fa8dd165640 1 -- start start 2024-09-19T08:45:56.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.523+0000 7fa8dd165640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8d815c550 0x7fa8d806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.523+0000 7fa8dd165640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa8d806c1d0 0x7fa8d80cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.523+0000 7fa8dd165640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8d806d310 con 0x7fa8d80cf900 2024-09-19T08:45:56.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.523+0000 7fa8dd165640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fa8d806c610 con 0x7fa8d806c1d0 2024-09-19T08:45:56.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.523+0000 7fa8dd165640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa8d806d5f0 con 0x7fa8d815c550 2024-09-19T08:45:56.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.523+0000 7fa8d659c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa8d806c1d0 0x7fa8d80cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.523+0000 7fa8d759e640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fa8d80cf900 0x7fa8d80ce1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42498/0 (socket says 172.21.15.98:42498) 2024-09-19T08:45:56.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.523+0000 7fa8d759e640 1 -- 172.21.15.98:0/3005657549 learned_addr learned my addr 172.21.15.98:0/3005657549 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:56.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.523+0000 7fa8d6d9d640 1 --2- 172.21.15.98:0/3005657549 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8d815c550 0x7fa8d806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.523+0000 7fa8d659c640 1 -- 172.21.15.98:0/3005657549 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8d815c550 msgr2=0x7fa8d806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:56.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.523+0000 7fa8d659c640 1 --2- 172.21.15.98:0/3005657549 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8d815c550 0x7fa8d806bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.523+0000 7fa8d659c640 1 -- 172.21.15.98:0/3005657549 >> v1:172.21.15.98:6789/0 conn(0x7fa8d80cf900 legacy=0x7fa8d80ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:56.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.523+0000 7fa8d5d9b640 1 -- 172.21.15.98:0/3005657549 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2352112901 0 0) 0x7fa8d806d310 con 0x7fa8d80cf900 2024-09-19T08:45:56.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.523+0000 7fa8d659c640 1 -- 172.21.15.98:0/3005657549 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d806d8d0 con 0x7fa8d806c1d0 2024-09-19T08:45:56.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.523+0000 7fa8d6d9d640 1 --2- 172.21.15.98:0/3005657549 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8d815c550 0x7fa8d806bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:56.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.523+0000 7fa8d659c640 1 --2- 172.21.15.98:0/3005657549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8d806c1d0 0x7fa8d80cf3c0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fa8c00097c0 tx=0x7fa8c001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0fd8933e68207c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:56.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.527+0000 7fa8d5d9b640 1 -- 172.21.15.98:0/3005657549 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c00b0070 con 0x7fa8d806c1d0 2024-09-19T08:45:56.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.527+0000 7fa8d5d9b640 1 -- 172.21.15.98:0/3005657549 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8c00ab600 con 0x7fa8d806c1d0 2024-09-19T08:45:56.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.527+0000 7fa8d5d9b640 1 -- 172.21.15.98:0/3005657549 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c00ab9d0 con 0x7fa8d806c1d0 2024-09-19T08:45:56.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.527+0000 7fa8dd165640 1 -- 172.21.15.98:0/3005657549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8d806c1d0 msgr2=0x7fa8d80cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:56.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.527+0000 7fa8dd165640 1 --2- 172.21.15.98:0/3005657549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8d806c1d0 0x7fa8d80cf3c0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fa8c00097c0 tx=0x7fa8c001c4a0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.527+0000 7fa8dd165640 1 -- 172.21.15.98:0/3005657549 shutdown_connections 2024-09-19T08:45:56.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.527+0000 7fa8dd165640 1 --2- 172.21.15.98:0/3005657549 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8d806c1d0 0x7fa8d80cf3c0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.527+0000 7fa8dd165640 1 --2- 172.21.15.98:0/3005657549 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8d815c550 0x7fa8d806bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.527+0000 7fa8dd165640 1 -- 172.21.15.98:0/3005657549 >> 172.21.15.98:0/3005657549 conn(0x7fa8d80b2010 msgr2=0x7fa8d80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:56.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.527+0000 7fa8dd165640 1 -- 172.21.15.98:0/3005657549 shutdown_connections 2024-09-19T08:45:56.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.527+0000 7fa8dd165640 1 -- 172.21.15.98:0/3005657549 wait complete. 2024-09-19T08:45:56.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.527+0000 7fa8dd165640 1 Processor -- start 2024-09-19T08:45:56.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.527+0000 7fa8dd165640 1 -- start start 2024-09-19T08:45:56.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.527+0000 7fa8dd165640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8d806c1d0 0x7fa8d8149a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.527+0000 7fa8dd165640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8d80cf900 0x7fa8d8151d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.527+0000 7fa8dd165640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8d815c550 0x7fa8d8158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.527+0000 7fa8dd165640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d8169230 con 0x7fa8d815c550 2024-09-19T08:45:56.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.527+0000 7fa8dd165640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d815cb40 con 0x7fa8d80cf900 2024-09-19T08:45:56.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.527+0000 7fa8dd165640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa8d81693b0 con 0x7fa8d806c1d0 2024-09-19T08:45:56.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.527+0000 7fa8d759e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8d815c550 0x7fa8d8158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.527+0000 7fa8d759e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8d815c550 0x7fa8d8158780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54070/0 (socket says 172.21.15.98:54070) 2024-09-19T08:45:56.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.527+0000 7fa8d759e640 1 -- 172.21.15.98:0/3024559628 learned_addr learned my addr 172.21.15.98:0/3024559628 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:56.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.527+0000 7fa8d659c640 1 --2- 172.21.15.98:0/3024559628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8d80cf900 0x7fa8d8151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.527+0000 7fa8d6d9d640 1 --2- 172.21.15.98:0/3024559628 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8d806c1d0 0x7fa8d8149a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.527+0000 7fa8d759e640 1 -- 172.21.15.98:0/3024559628 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8d806c1d0 msgr2=0x7fa8d8149a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:56.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.527+0000 7fa8d759e640 1 --2- 172.21.15.98:0/3024559628 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8d806c1d0 0x7fa8d8149a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.527+0000 7fa8d759e640 1 -- 172.21.15.98:0/3024559628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8d80cf900 msgr2=0x7fa8d8151d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:56.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.527+0000 7fa8d759e640 1 --2- 172.21.15.98:0/3024559628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8d80cf900 0x7fa8d8151d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:56.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.527+0000 7fa8d759e640 1 -- 172.21.15.98:0/3024559628 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d814e630 con 0x7fa8d815c550 2024-09-19T08:45:56.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.527+0000 7fa8d6d9d640 1 --2- 172.21.15.98:0/3024559628 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8d806c1d0 0x7fa8d8149a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:56.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.527+0000 7fa8d759e640 1 --2- 172.21.15.98:0/3024559628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8d815c550 0x7fa8d8158780 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fa8c800c7a0 tx=0x7fa8c800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:56.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.527+0000 7fa8cf7fe640 1 -- 172.21.15.98:0/3024559628 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c8018070 con 0x7fa8d815c550 2024-09-19T08:45:56.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.527+0000 7fa8cf7fe640 1 -- 172.21.15.98:0/3024559628 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8c800a640 con 0x7fa8d815c550 2024-09-19T08:45:56.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.527+0000 7fa8dd165640 1 -- 172.21.15.98:0/3024559628 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8d814e910 con 0x7fa8d815c550 2024-09-19T08:45:56.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.527+0000 7fa8cf7fe640 1 -- 172.21.15.98:0/3024559628 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c800aab0 con 0x7fa8d815c550 2024-09-19T08:45:56.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.527+0000 7fa8dd165640 1 -- 172.21.15.98:0/3024559628 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8d814eed0 con 0x7fa8d815c550 2024-09-19T08:45:56.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.531+0000 7fa8dd165640 1 -- 172.21.15.98:0/3024559628 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8d81f5a30 con 0x7fa8d815c550 2024-09-19T08:45:56.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.531+0000 7fa8cf7fe640 1 -- 172.21.15.98:0/3024559628 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fa8c801b070 con 0x7fa8d815c550 2024-09-19T08:45:56.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.531+0000 7fa8cf7fe640 1 --2- 172.21.15.98:0/3024559628 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa8b8067ef0 0x7fa8b806a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:56.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.531+0000 7fa8cf7fe640 1 -- 172.21.15.98:0/3024559628 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 4971+0+0 (secure 0 0 0) 0x7fa8c8085e20 con 0x7fa8d815c550 2024-09-19T08:45:56.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.531+0000 7fa8d6d9d640 1 --2- 172.21.15.98:0/3024559628 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa8b8067ef0 0x7fa8b806a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:56.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.531+0000 7fa8d6d9d640 1 --2- 172.21.15.98:0/3024559628 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa8b8067ef0 0x7fa8b806a3b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0008890 tx=0x7fa8d0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:56.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.535+0000 7fa8cf7fe640 1 -- 172.21.15.98:0/3024559628 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa8c8056400 con 0x7fa8d815c550 2024-09-19T08:45:56.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:56.739+0000 7fa8dd165640 1 -- 172.21.15.98:0/3024559628 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]} v 0) -- 0x7fa8d814f2b0 con 0x7fa8d815c550 2024-09-19T08:45:57.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.743+0000 7fa8cf7fe640 1 -- 172.21.15.98:0/3024559628 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]}]=0 v400) ==== 152+0+0 (secure 0 0 0) 0x7fa8c8055d00 con 0x7fa8d815c550 2024-09-19T08:45:57.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.747+0000 7fa8dd165640 1 -- 172.21.15.98:0/3024559628 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa8b8067ef0 msgr2=0x7fa8b806a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:57.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.747+0000 7fa8dd165640 1 --2- 172.21.15.98:0/3024559628 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa8b8067ef0 0x7fa8b806a3b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0008890 tx=0x7fa8d0008040 comp rx=0 tx=0).stop 2024-09-19T08:45:57.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.747+0000 7fa8dd165640 1 -- 172.21.15.98:0/3024559628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8d815c550 msgr2=0x7fa8d8158780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:57.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.747+0000 7fa8dd165640 1 --2- 172.21.15.98:0/3024559628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8d815c550 0x7fa8d8158780 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fa8c800c7a0 tx=0x7fa8c800cc60 comp rx=0 tx=0).stop 2024-09-19T08:45:57.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.747+0000 7fa8dd165640 1 -- 172.21.15.98:0/3024559628 shutdown_connections 2024-09-19T08:45:57.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.747+0000 7fa8dd165640 1 --2- 172.21.15.98:0/3024559628 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa8b8067ef0 0x7fa8b806a3b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.747+0000 7fa8dd165640 1 --2- 172.21.15.98:0/3024559628 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa8d815c550 0x7fa8d8158780 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.747+0000 7fa8dd165640 1 --2- 172.21.15.98:0/3024559628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa8d80cf900 0x7fa8d8151d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.747+0000 7fa8dd165640 1 --2- 172.21.15.98:0/3024559628 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa8d806c1d0 0x7fa8d8149a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.747+0000 7fa8dd165640 1 -- 172.21.15.98:0/3024559628 >> 172.21.15.98:0/3024559628 conn(0x7fa8d80b2010 msgr2=0x7fa8d8159f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:57.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.747+0000 7fa8dd165640 1 -- 172.21.15.98:0/3024559628 shutdown_connections 2024-09-19T08:45:57.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.747+0000 7fa8dd165640 1 -- 172.21.15.98:0/3024559628 wait complete. 2024-09-19T08:45:57.768 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T08:45:57.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.899+0000 7fd7a63b1640 1 Processor -- start 2024-09-19T08:45:57.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.899+0000 7fd7a63b1640 1 -- start start 2024-09-19T08:45:57.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.899+0000 7fd7a63b1640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd7a0144d10 0x7fd7a00722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:57.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.899+0000 7fd7a63b1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7a0072820 0x7fd7a006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:57.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.899+0000 7fd7a63b1640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7a0072ea0 con 0x7fd7a006c1d0 2024-09-19T08:45:57.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.899+0000 7fd7a63b1640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fd7a0072c60 con 0x7fd7a0144d10 2024-09-19T08:45:57.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.899+0000 7fd7a63b1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd7a0073ab0 con 0x7fd7a0072820 2024-09-19T08:45:57.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.899+0000 7fd79ffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd7a0144d10 0x7fd7a00722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:57.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.899+0000 7fd79ffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd7a0144d10 0x7fd7a00722e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46602/0 (socket says 172.21.15.98:46602) 2024-09-19T08:45:57.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.899+0000 7fd7a494e640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fd7a006c1d0 0x7fd7a00b4520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42358/0 (socket says 172.21.15.98:42358) 2024-09-19T08:45:57.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.899+0000 7fd79ffff640 1 -- 172.21.15.98:0/1276187858 learned_addr learned my addr 172.21.15.98:0/1276187858 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:57.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.899+0000 7fd79f7fe640 1 --2- 172.21.15.98:0/1276187858 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7a0072820 0x7fd7a006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:57.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.899+0000 7fd79ffff640 1 -- 172.21.15.98:0/1276187858 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7a0072820 msgr2=0x7fd7a006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:57.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.899+0000 7fd79ffff640 1 --2- 172.21.15.98:0/1276187858 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7a0072820 0x7fd7a006bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.899+0000 7fd79ffff640 1 -- 172.21.15.98:0/1276187858 >> v1:172.21.15.98:6789/0 conn(0x7fd7a006c1d0 legacy=0x7fd7a00b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:57.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.899+0000 7fd79ffff640 1 -- 172.21.15.98:0/1276187858 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a0073180 con 0x7fd7a0144d10 2024-09-19T08:45:57.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.903+0000 7fd79ffff640 1 --2- 172.21.15.98:0/1276187858 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7a0144d10 0x7fd7a00722e0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fd798009e50 tx=0x7fd79801c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7cc97c4d30651e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:57.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.903+0000 7fd79effd640 1 -- 172.21.15.98:0/1276187858 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7980a56e0 con 0x7fd7a0144d10 2024-09-19T08:45:57.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.903+0000 7fd79effd640 1 -- 172.21.15.98:0/1276187858 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7980a5880 con 0x7fd7a0144d10 2024-09-19T08:45:57.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.903+0000 7fd79effd640 1 -- 172.21.15.98:0/1276187858 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7980a5c50 con 0x7fd7a0144d10 2024-09-19T08:45:57.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.903+0000 7fd7a63b1640 1 -- 172.21.15.98:0/1276187858 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7a0144d10 msgr2=0x7fd7a00722e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:57.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.903+0000 7fd7a63b1640 1 --2- 172.21.15.98:0/1276187858 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7a0144d10 0x7fd7a00722e0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fd798009e50 tx=0x7fd79801c8c0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.903+0000 7fd7a63b1640 1 -- 172.21.15.98:0/1276187858 shutdown_connections 2024-09-19T08:45:57.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.903+0000 7fd7a63b1640 1 --2- 172.21.15.98:0/1276187858 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7a0072820 0x7fd7a006bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.903+0000 7fd7a63b1640 1 --2- 172.21.15.98:0/1276187858 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7a0144d10 0x7fd7a00722e0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.903+0000 7fd7a63b1640 1 -- 172.21.15.98:0/1276187858 >> 172.21.15.98:0/1276187858 conn(0x7fd7a00b2010 msgr2=0x7fd7a00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:57.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.903+0000 7fd7a63b1640 1 -- 172.21.15.98:0/1276187858 shutdown_connections 2024-09-19T08:45:57.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.903+0000 7fd7a63b1640 1 -- 172.21.15.98:0/1276187858 wait complete. 2024-09-19T08:45:57.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.903+0000 7fd7a63b1640 1 Processor -- start 2024-09-19T08:45:57.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.903+0000 7fd7a63b1640 1 -- start start 2024-09-19T08:45:57.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.903+0000 7fd7a63b1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7a006c1d0 0x7fd7a01f2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:57.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.903+0000 7fd7a63b1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7a0072820 0x7fd7a015ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:57.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.903+0000 7fd7a63b1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7a0144d10 0x7fd7a01f1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:57.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.903+0000 7fd7a63b1640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a0073740 con 0x7fd7a0072820 2024-09-19T08:45:57.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.903+0000 7fd7a63b1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a01450f0 con 0x7fd7a0144d10 2024-09-19T08:45:57.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.903+0000 7fd7a63b1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd7a00af8b0 con 0x7fd7a006c1d0 2024-09-19T08:45:57.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.903+0000 7fd79f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7a0072820 0x7fd7a015ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:57.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.903+0000 7fd79f7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7a0072820 0x7fd7a015ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46618/0 (socket says 172.21.15.98:46618) 2024-09-19T08:45:57.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.903+0000 7fd79f7fe640 1 -- 172.21.15.98:0/706544189 learned_addr learned my addr 172.21.15.98:0/706544189 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:57.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.903+0000 7fd79f7fe640 1 -- 172.21.15.98:0/706544189 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7a006c1d0 msgr2=0x7fd7a01f2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:57.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.903+0000 7fd79f7fe640 1 --2- 172.21.15.98:0/706544189 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7a006c1d0 0x7fd7a01f2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.903+0000 7fd7a494e640 1 --2- 172.21.15.98:0/706544189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7a0144d10 0x7fd7a01f1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:57.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.903+0000 7fd79f7fe640 1 -- 172.21.15.98:0/706544189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7a0144d10 msgr2=0x7fd7a01f1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:57.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.903+0000 7fd79f7fe640 1 --2- 172.21.15.98:0/706544189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7a0144d10 0x7fd7a01f1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:57.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.903+0000 7fd79f7fe640 1 -- 172.21.15.98:0/706544189 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a01f32e0 con 0x7fd7a0072820 2024-09-19T08:45:57.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.903+0000 7fd79f7fe640 1 --2- 172.21.15.98:0/706544189 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7a0072820 0x7fd7a015ad30 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fd78800d800 tx=0x7fd78800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:57.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.903+0000 7fd79cff9640 1 -- 172.21.15.98:0/706544189 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd788004020 con 0x7fd7a0072820 2024-09-19T08:45:57.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.903+0000 7fd79cff9640 1 -- 172.21.15.98:0/706544189 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7880041c0 con 0x7fd7a0072820 2024-09-19T08:45:57.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.903+0000 7fd7a63b1640 1 -- 172.21.15.98:0/706544189 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7a01f2530 con 0x7fd7a0072820 2024-09-19T08:45:57.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.903+0000 7fd79cff9640 1 -- 172.21.15.98:0/706544189 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7880043b0 con 0x7fd7a0072820 2024-09-19T08:45:57.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.903+0000 7fd7a63b1640 1 -- 172.21.15.98:0/706544189 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7a013e7f0 con 0x7fd7a0072820 2024-09-19T08:45:57.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.907+0000 7fd7a63b1640 1 -- 172.21.15.98:0/706544189 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7a006bc90 con 0x7fd7a0072820 2024-09-19T08:45:57.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.907+0000 7fd79cff9640 1 -- 172.21.15.98:0/706544189 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fd78801b020 con 0x7fd7a0072820 2024-09-19T08:45:57.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.907+0000 7fd79cff9640 1 --2- 172.21.15.98:0/706544189 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd780067ef0 0x7fd78006a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:57.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.907+0000 7fd79cff9640 1 -- 172.21.15.98:0/706544189 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 4971+0+0 (secure 0 0 0) 0x7fd788086d70 con 0x7fd7a0072820 2024-09-19T08:45:57.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.907+0000 7fd79ffff640 1 --2- 172.21.15.98:0/706544189 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd780067ef0 0x7fd78006a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:57.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.907+0000 7fd79ffff640 1 --2- 172.21.15.98:0/706544189 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd780067ef0 0x7fd78006a3b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd7980ae000 tx=0x7fd798004ac0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:57.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:57.911+0000 7fd79cff9640 1 -- 172.21.15.98:0/706544189 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd788057350 con 0x7fd7a0072820 2024-09-19T08:45:58.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.071+0000 7fd7a63b1640 1 -- 172.21.15.98:0/706544189 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fd7a00713b0 con 0x7fd7a0072820 2024-09-19T08:45:58.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.075+0000 7fd79cff9640 1 -- 172.21.15.98:0/706544189 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7fd788056c50 con 0x7fd7a0072820 2024-09-19T08:45:58.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.079+0000 7fd7a63b1640 1 -- 172.21.15.98:0/706544189 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd780067ef0 msgr2=0x7fd78006a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:58.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.079+0000 7fd7a63b1640 1 --2- 172.21.15.98:0/706544189 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd780067ef0 0x7fd78006a3b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd7980ae000 tx=0x7fd798004ac0 comp rx=0 tx=0).stop 2024-09-19T08:45:58.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.079+0000 7fd7a63b1640 1 -- 172.21.15.98:0/706544189 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7a0072820 msgr2=0x7fd7a015ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:58.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.079+0000 7fd7a63b1640 1 --2- 172.21.15.98:0/706544189 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7a0072820 0x7fd7a015ad30 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fd78800d800 tx=0x7fd78800dcc0 comp rx=0 tx=0).stop 2024-09-19T08:45:58.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.079+0000 7fd7a63b1640 1 -- 172.21.15.98:0/706544189 shutdown_connections 2024-09-19T08:45:58.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.079+0000 7fd7a63b1640 1 --2- 172.21.15.98:0/706544189 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd780067ef0 0x7fd78006a3b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:58.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.079+0000 7fd7a63b1640 1 --2- 172.21.15.98:0/706544189 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7a0144d10 0x7fd7a01f1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:58.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.079+0000 7fd7a63b1640 1 --2- 172.21.15.98:0/706544189 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd7a0072820 0x7fd7a015ad30 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:58.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.079+0000 7fd7a63b1640 1 --2- 172.21.15.98:0/706544189 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd7a006c1d0 0x7fd7a01f2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:58.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.079+0000 7fd7a63b1640 1 -- 172.21.15.98:0/706544189 >> 172.21.15.98:0/706544189 conn(0x7fd7a00b2010 msgr2=0x7fd7a00cd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:58.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.079+0000 7fd7a63b1640 1 -- 172.21.15.98:0/706544189 shutdown_connections 2024-09-19T08:45:58.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.079+0000 7fd7a63b1640 1 -- 172.21.15.98:0/706544189 wait complete. 2024-09-19T08:45:58.094 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T08:45:58.095 DEBUG:teuthology.orchestra.run.smithi098:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:45:58.161 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:45:58.244 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T08:45:58.244 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:45:58.244 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:45:58.250 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15846 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T08:45:58.251 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:45:58.251 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:45:58.257 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T08:45:58.257 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:45:58.257 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:45:58.297 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15846 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T08:45:58.297 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:45:58.297 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:45:58.303 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T08:45:58.303 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-19T08:45:58.303 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:45:58.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.467+0000 7f67b6f69640 1 Processor -- start 2024-09-19T08:45:58.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.467+0000 7f67b6f69640 1 -- start start 2024-09-19T08:45:58.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.467+0000 7f67b6f69640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f67b00d4820 0x7f67b01621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:58.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.467+0000 7f67b6f69640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67b0068440 con 0x7f67b00b9970 2024-09-19T08:45:58.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.467+0000 7f67b6f69640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67b0162a90 con 0x7f67b00b7f80 2024-09-19T08:45:58.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.467+0000 7f67b6f69640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f67b0163050 con 0x7f67b00d4820 2024-09-19T08:45:58.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.467+0000 7f67b5506640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f67b00b9970 0x7f67b00b9de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42372/0 (socket says 172.21.15.98:42372) 2024-09-19T08:45:58.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.467+0000 7f67b5506640 1 -- 172.21.15.98:0/382675695 learned_addr learned my addr 172.21.15.98:0/382675695 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:58.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.467+0000 7f67affff640 1 --2- 172.21.15.98:0/382675695 >> v2:172.21.15.139:3300/0 conn(0x7f67b00d4820 0x7f67b01621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:58.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.467+0000 7f67af7fe640 1 -- 172.21.15.98:0/382675695 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1267101579 0 0) 0x7f67b0068440 con 0x7f67b00b9970 2024-09-19T08:45:58.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.467+0000 7f67af7fe640 1 -- 172.21.15.98:0/382675695 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6798003530 con 0x7f67b00b9970 2024-09-19T08:45:58.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.467+0000 7f67af7fe640 1 -- 172.21.15.98:0/382675695 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2855844798 0 0) 0x7f6798003530 con 0x7f67b00b9970 2024-09-19T08:45:58.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.467+0000 7f67af7fe640 1 -- 172.21.15.98:0/382675695 >> v2:172.21.15.139:3300/0 conn(0x7f67b00d4820 msgr2=0x7f67b01621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:58.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.467+0000 7f67af7fe640 1 --2- 172.21.15.98:0/382675695 >> v2:172.21.15.139:3300/0 conn(0x7f67b00d4820 0x7f67b01621b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:58.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.467+0000 7f67af7fe640 1 -- 172.21.15.98:0/382675695 >> v1:172.21.15.139:6789/0 conn(0x7f67b00b7f80 legacy=0x7f67b015f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:58.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.467+0000 7f67af7fe640 1 -- 172.21.15.98:0/382675695 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b0162d70 con 0x7f67b00b9970 2024-09-19T08:45:58.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.467+0000 7f67af7fe640 1 -- 172.21.15.98:0/382675695 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f67a0001680 con 0x7f67b00b9970 2024-09-19T08:45:58.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.467+0000 7f67af7fe640 1 -- 172.21.15.98:0/382675695 >> v1:172.21.15.98:6789/0 conn(0x7f67b00b9970 legacy=0x7f67b00b9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:58.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.467+0000 7f67af7fe640 1 --2- 172.21.15.98:0/382675695 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67980039c0 0x7f6798003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:58.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.467+0000 7f67af7fe640 1 --2- 172.21.15.98:0/382675695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6798004810 0x7f6798004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:58.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.467+0000 7f67af7fe640 1 --2- 172.21.15.98:0/382675695 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f67980051d0 0x7f679800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:58.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.467+0000 7f67af7fe640 1 -- 172.21.15.98:0/382675695 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f67b0163050 con 0x7f6798004810 2024-09-19T08:45:58.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.467+0000 7f67af7fe640 1 -- 172.21.15.98:0/382675695 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f679800c170 con 0x7f67980039c0 2024-09-19T08:45:58.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.467+0000 7f67af7fe640 1 -- 172.21.15.98:0/382675695 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f679800c370 con 0x7f67980051d0 2024-09-19T08:45:58.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.467+0000 7f67affff640 1 --2- 172.21.15.98:0/382675695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6798004810 0x7f6798004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:58.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.467+0000 7f67b4d05640 1 --2- 172.21.15.98:0/382675695 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67980039c0 0x7f6798003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:58.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.467+0000 7f67b5506640 1 --2- 172.21.15.98:0/382675695 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f67980051d0 0x7f679800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:58.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.467+0000 7f67affff640 1 -- 172.21.15.98:0/382675695 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f67980051d0 msgr2=0x7f679800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:58.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.467+0000 7f67affff640 1 --2- 172.21.15.98:0/382675695 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f67980051d0 0x7f679800b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:58.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.467+0000 7f67affff640 1 -- 172.21.15.98:0/382675695 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67980039c0 msgr2=0x7f6798003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:58.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.467+0000 7f67affff640 1 --2- 172.21.15.98:0/382675695 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67980039c0 0x7f6798003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:58.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.467+0000 7f67affff640 1 -- 172.21.15.98:0/382675695 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6798003530 con 0x7f6798004810 2024-09-19T08:45:58.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.467+0000 7f67b4d05640 1 --2- 172.21.15.98:0/382675695 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67980039c0 0x7f6798003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:58.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.467+0000 7f67b5506640 1 -- 172.21.15.98:0/382675695 reap_dead start 2024-09-19T08:45:58.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.471+0000 7f67affff640 1 --2- 172.21.15.98:0/382675695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6798004810 0x7f6798004c90 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f67a4009fa0 tx=0x7f67a401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f7702de3ef5406 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:58.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.471+0000 7f67af7fe640 1 -- 172.21.15.98:0/382675695 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a40b2070 con 0x7f6798004810 2024-09-19T08:45:58.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.471+0000 7f67af7fe640 1 -- 172.21.15.98:0/382675695 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67a40a5c20 con 0x7f6798004810 2024-09-19T08:45:58.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.471+0000 7f67af7fe640 1 -- 172.21.15.98:0/382675695 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a4009bf0 con 0x7f6798004810 2024-09-19T08:45:58.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.471+0000 7f67b6f69640 1 -- 172.21.15.98:0/382675695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6798004810 msgr2=0x7f6798004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:58.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.471+0000 7f67b6f69640 1 --2- 172.21.15.98:0/382675695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6798004810 0x7f6798004c90 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f67a4009fa0 tx=0x7f67a401cc60 comp rx=0 tx=0).stop 2024-09-19T08:45:58.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.471+0000 7f67b6f69640 1 -- 172.21.15.98:0/382675695 shutdown_connections 2024-09-19T08:45:58.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.471+0000 7f67b6f69640 1 --2- 172.21.15.98:0/382675695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6798004810 0x7f6798004c90 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:58.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.471+0000 7f67b6f69640 1 -- 172.21.15.98:0/382675695 >> 172.21.15.98:0/382675695 conn(0x7f67b0074e40 msgr2=0x7f67b0075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:58.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.471+0000 7f67b6f69640 1 -- 172.21.15.98:0/382675695 shutdown_connections 2024-09-19T08:45:58.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.471+0000 7f67b6f69640 1 -- 172.21.15.98:0/382675695 wait complete. 2024-09-19T08:45:58.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.471+0000 7f67b6f69640 1 Processor -- start 2024-09-19T08:45:58.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.471+0000 7f67b6f69640 1 -- start start 2024-09-19T08:45:58.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.475+0000 7f67b6f69640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6798004810 0x7f67b0140850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:58.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.475+0000 7f67b6f69640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67b01eb220 0x7f67b0160d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:58.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.475+0000 7f67b6f69640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f67b0154a30 0x7f67b0161240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:58.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.475+0000 7f67b6f69640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f67b015f950 con 0x7f6798004810 2024-09-19T08:45:58.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.475+0000 7f67b6f69640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f67b00b8570 con 0x7f67b01eb220 2024-09-19T08:45:58.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.475+0000 7f67b6f69640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f67b015fad0 con 0x7f67b0154a30 2024-09-19T08:45:58.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.475+0000 7f67b4d05640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6798004810 0x7f67b0140850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:58.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.475+0000 7f67b4d05640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6798004810 0x7f67b0140850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46634/0 (socket says 172.21.15.98:46634) 2024-09-19T08:45:58.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.475+0000 7f67b4d05640 1 -- 172.21.15.98:0/2182348840 learned_addr learned my addr 172.21.15.98:0/2182348840 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:58.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.475+0000 7f67affff640 1 --2- 172.21.15.98:0/2182348840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67b01eb220 0x7f67b0160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:58.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.475+0000 7f67b5506640 1 --2- 172.21.15.98:0/2182348840 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f67b0154a30 0x7f67b0161240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:58.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.475+0000 7f67b4d05640 1 -- 172.21.15.98:0/2182348840 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f67b0154a30 msgr2=0x7f67b0161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:58.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.475+0000 7f67b4d05640 1 --2- 172.21.15.98:0/2182348840 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f67b0154a30 0x7f67b0161240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:58.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.475+0000 7f67b4d05640 1 -- 172.21.15.98:0/2182348840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67b01eb220 msgr2=0x7f67b0160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:58.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.475+0000 7f67b4d05640 1 --2- 172.21.15.98:0/2182348840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67b01eb220 0x7f67b0160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:58.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.475+0000 7f67b4d05640 1 -- 172.21.15.98:0/2182348840 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b01eb720 con 0x7f6798004810 2024-09-19T08:45:58.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.475+0000 7f67b4d05640 1 --2- 172.21.15.98:0/2182348840 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6798004810 0x7f67b0140850 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f67a800d7d0 tx=0x7f67a800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:58.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.475+0000 7f67ad7fa640 1 -- 172.21.15.98:0/2182348840 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a8005270 con 0x7f6798004810 2024-09-19T08:45:58.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.475+0000 7f67ad7fa640 1 -- 172.21.15.98:0/2182348840 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67a8017070 con 0x7f6798004810 2024-09-19T08:45:58.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.475+0000 7f67b6f69640 1 -- 172.21.15.98:0/2182348840 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67b01fb060 con 0x7f6798004810 2024-09-19T08:45:58.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.475+0000 7f67ad7fa640 1 -- 172.21.15.98:0/2182348840 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a800b6b0 con 0x7f6798004810 2024-09-19T08:45:58.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.475+0000 7f67b6f69640 1 -- 172.21.15.98:0/2182348840 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67b01fbb40 con 0x7f6798004810 2024-09-19T08:45:58.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.475+0000 7f67b6f69640 1 -- 172.21.15.98:0/2182348840 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67b0143d80 con 0x7f6798004810 2024-09-19T08:45:58.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.475+0000 7f67ad7fa640 1 -- 172.21.15.98:0/2182348840 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f67a800b850 con 0x7f6798004810 2024-09-19T08:45:58.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.475+0000 7f67ad7fa640 1 --2- 172.21.15.98:0/2182348840 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f679c067f20 0x7f679c06a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:58.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.475+0000 7f67ad7fa640 1 -- 172.21.15.98:0/2182348840 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 4971+0+0 (secure 0 0 0) 0x7f67a8086500 con 0x7f6798004810 2024-09-19T08:45:58.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.475+0000 7f67affff640 1 --2- 172.21.15.98:0/2182348840 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f679c067f20 0x7f679c06a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:58.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.475+0000 7f67affff640 1 --2- 172.21.15.98:0/2182348840 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f679c067f20 0x7f679c06a3e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f67a40af000 tx=0x7f67a4004ac0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:58.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.479+0000 7f67ad7fa640 1 -- 172.21.15.98:0/2182348840 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f67a80568d0 con 0x7f6798004810 2024-09-19T08:45:58.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.679+0000 7f67b6f69640 1 -- 172.21.15.98:0/2182348840 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67b0141e00 con 0x7f6798004810 2024-09-19T08:45:58.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.679+0000 7f67ad7fa640 1 -- 172.21.15.98:0/2182348840 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+3914 (secure 0 0 0) 0x7f67a801ae40 con 0x7f6798004810 2024-09-19T08:45:58.678 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:45:58.678 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":77,"btime":"2024-09-19T08:45:52:712896+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6604,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/1907681163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":1907681163},{"type":"v1","addr":"172.21.15.139:6834","nonce":1907681163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":6832,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1691540052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6833","nonce":1691540052},{"type":"v1","addr":"172.21.15.139:6835","nonce":1691540052}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7896,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/2481662340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":2481662340},{"type":"v1","addr":"172.21.15.98:6833","nonce":2481662340}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71},{"gid":7902,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/286818221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":286818221},{"type":"v1","addr":"172.21.15.98:6835","nonce":286818221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":71}],"filesystems":[]} 2024-09-19T08:45:58.679 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 77 2024-09-19T08:45:58.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.683+0000 7f67b6f69640 1 -- 172.21.15.98:0/2182348840 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f679c067f20 msgr2=0x7f679c06a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:58.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.683+0000 7f67b6f69640 1 --2- 172.21.15.98:0/2182348840 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f679c067f20 0x7f679c06a3e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f67a40af000 tx=0x7f67a4004ac0 comp rx=0 tx=0).stop 2024-09-19T08:45:58.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.683+0000 7f67b6f69640 1 -- 172.21.15.98:0/2182348840 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6798004810 msgr2=0x7f67b0140850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:58.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.683+0000 7f67b6f69640 1 --2- 172.21.15.98:0/2182348840 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6798004810 0x7f67b0140850 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f67a800d7d0 tx=0x7f67a800dc90 comp rx=0 tx=0).stop 2024-09-19T08:45:58.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.683+0000 7f67b6f69640 1 -- 172.21.15.98:0/2182348840 shutdown_connections 2024-09-19T08:45:58.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.683+0000 7f67b6f69640 1 --2- 172.21.15.98:0/2182348840 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f679c067f20 0x7f679c06a3e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:58.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.683+0000 7f67b6f69640 1 --2- 172.21.15.98:0/2182348840 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f67b0154a30 0x7f67b0161240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:58.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.683+0000 7f67b6f69640 1 --2- 172.21.15.98:0/2182348840 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67b01eb220 0x7f67b0160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:58.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.683+0000 7f67b6f69640 1 --2- 172.21.15.98:0/2182348840 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6798004810 0x7f67b0140850 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:58.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.683+0000 7f67b6f69640 1 -- 172.21.15.98:0/2182348840 >> 172.21.15.98:0/2182348840 conn(0x7f67b0074e40 msgr2=0x7f67b0143f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:58.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.683+0000 7f67b6f69640 1 -- 172.21.15.98:0/2182348840 shutdown_connections 2024-09-19T08:45:58.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.683+0000 7f67b6f69640 1 -- 172.21.15.98:0/2182348840 wait complete. 2024-09-19T08:45:58.697 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-19T08:45:58.697 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-19T08:45:58.698 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-19T08:45:58.698 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:45:58.698 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-19T08:45:58.698 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-19T08:45:58.698 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-19T08:45:58.698 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:45:58.698 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-19T08:45:58.698 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-19T08:45:58.699 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-19T08:45:58.699 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:45:58.699 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-19T08:45:58.699 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-19T08:45:58.699 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-19T08:45:58.699 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:45:58.708 INFO:tasks.ceph.mds.a:Stopped 2024-09-19T08:45:58.708 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-19T08:45:58.709 INFO:tasks.ceph.mds.c:Stopped 2024-09-19T08:45:58.709 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-19T08:45:58.711 INFO:tasks.ceph.mds.b:Stopped 2024-09-19T08:45:58.711 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-19T08:45:58.713 INFO:tasks.ceph.mds.d:Stopped 2024-09-19T08:45:58.713 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-19T08:45:58.756 INFO:tasks.ceph.mds.b:Started 2024-09-19T08:45:58.757 DEBUG:teuthology.parallel:result is None 2024-09-19T08:45:58.758 INFO:tasks.ceph.mds.d:Started 2024-09-19T08:45:58.759 DEBUG:teuthology.parallel:result is None 2024-09-19T08:45:58.760 INFO:tasks.ceph.mds.a:Started 2024-09-19T08:45:58.760 INFO:tasks.ceph.mds.c:Started 2024-09-19T08:45:58.761 DEBUG:teuthology.parallel:result is None 2024-09-19T08:45:58.761 DEBUG:teuthology.parallel:result is None 2024-09-19T08:45:58.762 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-19T08:45:58.818 INFO:tasks.ceph.mds.c.smithi098.stdout:starting mds.c at 2024-09-19T08:45:58.818 INFO:tasks.ceph.mds.a.smithi098.stdout:starting mds.a at 2024-09-19T08:45:58.826 INFO:tasks.ceph.mds.d.smithi139.stdout:starting mds.d at 2024-09-19T08:45:58.828 INFO:tasks.ceph.mds.b.smithi139.stdout:starting mds.b at 2024-09-19T08:45:58.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.875+0000 7f4b5da55640 1 Processor -- start 2024-09-19T08:45:58.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.875+0000 7f4b5da55640 1 -- start start 2024-09-19T08:45:58.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.875+0000 7f4b5da55640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b5815c550 0x7f4b5806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:58.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.875+0000 7f4b5da55640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4b5806c1d0 0x7f4b580cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:58.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.875+0000 7f4b5da55640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4b580cf900 0x7f4b580ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:58.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.875+0000 7f4b5da55640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f4b5806c610 con 0x7f4b5806c1d0 2024-09-19T08:45:58.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.875+0000 7f4b5da55640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f4b5806d310 con 0x7f4b580cf900 2024-09-19T08:45:58.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.875+0000 7f4b5da55640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4b5806d5f0 con 0x7f4b5815c550 2024-09-19T08:45:58.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.875+0000 7f4b567fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4b5806c1d0 0x7f4b580cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:58.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.875+0000 7f4b567fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4b5806c1d0 0x7f4b580cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46672/0 (socket says 172.21.15.98:46672) 2024-09-19T08:45:58.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.875+0000 7f4b567fc640 1 -- 172.21.15.98:0/3624399444 learned_addr learned my addr 172.21.15.98:0/3624399444 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:58.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.875+0000 7f4b567fc640 1 -- 172.21.15.98:0/3624399444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b5815c550 msgr2=0x7f4b5806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:58.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.875+0000 7f4b567fc640 1 --2- 172.21.15.98:0/3624399444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b5815c550 0x7f4b5806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:58.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.875+0000 7f4b567fc640 1 -- 172.21.15.98:0/3624399444 >> v2:172.21.15.139:3300/0 conn(0x7f4b580cf900 msgr2=0x7f4b580ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:58.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.875+0000 7f4b577fe640 1 --2- 172.21.15.98:0/3624399444 >> v2:172.21.15.139:3300/0 conn(0x7f4b580cf900 0x7f4b580ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:58.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.875+0000 7f4b567fc640 1 --2- 172.21.15.98:0/3624399444 >> v2:172.21.15.139:3300/0 conn(0x7f4b580cf900 0x7f4b580ce1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:58.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.875+0000 7f4b567fc640 1 -- 172.21.15.98:0/3624399444 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5806d8d0 con 0x7f4b5806c1d0 2024-09-19T08:45:58.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.875+0000 7f4b577fe640 1 --2- 172.21.15.98:0/3624399444 >> v2:172.21.15.139:3300/0 conn(0x7f4b580cf900 0x7f4b580ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:58.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.879+0000 7f4b567fc640 1 --2- 172.21.15.98:0/3624399444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b5806c1d0 0x7f4b580cf3c0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f4b58074090 tx=0x7f4b4801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae779ad3a2e587f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:58.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.879+0000 7f4b55ffb640 1 -- 172.21.15.98:0/3624399444 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b480b0070 con 0x7f4b5806c1d0 2024-09-19T08:45:58.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.879+0000 7f4b55ffb640 1 -- 172.21.15.98:0/3624399444 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b480ab600 con 0x7f4b5806c1d0 2024-09-19T08:45:58.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.879+0000 7f4b55ffb640 1 -- 172.21.15.98:0/3624399444 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b480ab9d0 con 0x7f4b5806c1d0 2024-09-19T08:45:58.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.879+0000 7f4b5da55640 1 -- 172.21.15.98:0/3624399444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b5806c1d0 msgr2=0x7f4b580cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:58.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.879+0000 7f4b5da55640 1 --2- 172.21.15.98:0/3624399444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b5806c1d0 0x7f4b580cf3c0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f4b58074090 tx=0x7f4b4801c260 comp rx=0 tx=0).stop 2024-09-19T08:45:58.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.879+0000 7f4b5da55640 1 -- 172.21.15.98:0/3624399444 shutdown_connections 2024-09-19T08:45:58.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.879+0000 7f4b5da55640 1 --2- 172.21.15.98:0/3624399444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b5806c1d0 0x7f4b580cf3c0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:58.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.879+0000 7f4b5da55640 1 --2- 172.21.15.98:0/3624399444 >> v2:172.21.15.139:3300/0 conn(0x7f4b580cf900 0x7f4b580ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:58.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.879+0000 7f4b5da55640 1 --2- 172.21.15.98:0/3624399444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b5815c550 0x7f4b5806bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:58.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.879+0000 7f4b5da55640 1 -- 172.21.15.98:0/3624399444 >> 172.21.15.98:0/3624399444 conn(0x7f4b580b2010 msgr2=0x7f4b580b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:58.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.879+0000 7f4b5da55640 1 -- 172.21.15.98:0/3624399444 shutdown_connections 2024-09-19T08:45:58.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.879+0000 7f4b5da55640 1 -- 172.21.15.98:0/3624399444 wait complete. 2024-09-19T08:45:58.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.879+0000 7f4b5da55640 1 Processor -- start 2024-09-19T08:45:58.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.879+0000 7f4b5da55640 1 -- start start 2024-09-19T08:45:58.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.883+0000 7f4b5da55640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b5806c1d0 0x7f4b5815ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:58.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.883+0000 7f4b5da55640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b580cf900 0x7f4b58072930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:58.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.883+0000 7f4b5da55640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b5815c550 0x7f4b580ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:58.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.883+0000 7f4b5da55640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4b580723a0 con 0x7f4b5806c1d0 2024-09-19T08:45:58.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.883+0000 7f4b5da55640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5815cb40 con 0x7f4b5815c550 2024-09-19T08:45:58.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.883+0000 7f4b5da55640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4b58072520 con 0x7f4b580cf900 2024-09-19T08:45:58.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.883+0000 7f4b56ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b5806c1d0 0x7f4b5815ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:58.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.883+0000 7f4b56ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b5806c1d0 0x7f4b5815ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46688/0 (socket says 172.21.15.98:46688) 2024-09-19T08:45:58.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.883+0000 7f4b56ffd640 1 -- 172.21.15.98:0/768545815 learned_addr learned my addr 172.21.15.98:0/768545815 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:58.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.883+0000 7f4b567fc640 1 --2- 172.21.15.98:0/768545815 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b580cf900 0x7f4b58072930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:58.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.883+0000 7f4b577fe640 1 --2- 172.21.15.98:0/768545815 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b5815c550 0x7f4b580ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:58.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.883+0000 7f4b56ffd640 1 -- 172.21.15.98:0/768545815 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b580cf900 msgr2=0x7f4b58072930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:58.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.883+0000 7f4b56ffd640 1 --2- 172.21.15.98:0/768545815 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b580cf900 0x7f4b58072930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:58.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.883+0000 7f4b56ffd640 1 -- 172.21.15.98:0/768545815 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b5815c550 msgr2=0x7f4b580ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:58.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.883+0000 7f4b56ffd640 1 --2- 172.21.15.98:0/768545815 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b5815c550 0x7f4b580ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:58.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.883+0000 7f4b56ffd640 1 -- 172.21.15.98:0/768545815 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b58072e70 con 0x7f4b5806c1d0 2024-09-19T08:45:58.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.883+0000 7f4b56ffd640 1 --2- 172.21.15.98:0/768545815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b5806c1d0 0x7f4b5815ccc0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00b410 tx=0x7f4b4c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:58.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.883+0000 7f4b3ffff640 1 -- 172.21.15.98:0/768545815 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c015020 con 0x7f4b5806c1d0 2024-09-19T08:45:58.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.883+0000 7f4b3ffff640 1 -- 172.21.15.98:0/768545815 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b4c011830 con 0x7f4b5806c1d0 2024-09-19T08:45:58.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.883+0000 7f4b5da55640 1 -- 172.21.15.98:0/768545815 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b58158190 con 0x7f4b5806c1d0 2024-09-19T08:45:58.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.883+0000 7f4b3ffff640 1 -- 172.21.15.98:0/768545815 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c011c20 con 0x7f4b5806c1d0 2024-09-19T08:45:58.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.883+0000 7f4b5da55640 1 -- 172.21.15.98:0/768545815 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b580b3cb0 con 0x7f4b5806c1d0 2024-09-19T08:45:58.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.883+0000 7f4b5da55640 1 -- 172.21.15.98:0/768545815 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b580ce1a0 con 0x7f4b5806c1d0 2024-09-19T08:45:58.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.883+0000 7f4b3ffff640 1 -- 172.21.15.98:0/768545815 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f4b4c018020 con 0x7f4b5806c1d0 2024-09-19T08:45:58.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.883+0000 7f4b3ffff640 1 --2- 172.21.15.98:0/768545815 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4b44067ef0 0x7f4b4406a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:58.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.883+0000 7f4b3ffff640 1 -- 172.21.15.98:0/768545815 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 4971+0+0 (secure 0 0 0) 0x7f4b4c086970 con 0x7f4b5806c1d0 2024-09-19T08:45:58.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.883+0000 7f4b567fc640 1 --2- 172.21.15.98:0/768545815 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4b44067ef0 0x7f4b4406a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:58.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.883+0000 7f4b567fc640 1 --2- 172.21.15.98:0/768545815 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4b44067ef0 0x7f4b4406a3b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4b480ad000 tx=0x7f4b48008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:58.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:58.887+0000 7f4b3ffff640 1 -- 172.21.15.98:0/768545815 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f4b4c01f020 con 0x7f4b5806c1d0 2024-09-19T08:45:59.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.055+0000 7f4b5da55640 1 -- 172.21.15.98:0/768545815 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f4b580cc7b0 con 0x7f4b5806c1d0 2024-09-19T08:45:59.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.763+0000 7f4b3ffff640 1 -- 172.21.15.98:0/768545815 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v140) ==== 97+0+0 (secure 0 0 0) 0x7f4b58072e70 con 0x7f4b5806c1d0 2024-09-19T08:45:59.763 INFO:teuthology.orchestra.run.smithi098.stderr: removed all blocklist entries 2024-09-19T08:45:59.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.767+0000 7f4b5da55640 1 -- 172.21.15.98:0/768545815 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4b44067ef0 msgr2=0x7f4b4406a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:59.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.767+0000 7f4b5da55640 1 --2- 172.21.15.98:0/768545815 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4b44067ef0 0x7f4b4406a3b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4b480ad000 tx=0x7f4b48008040 comp rx=0 tx=0).stop 2024-09-19T08:45:59.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.767+0000 7f4b5da55640 1 -- 172.21.15.98:0/768545815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b5806c1d0 msgr2=0x7f4b5815ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:59.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.767+0000 7f4b5da55640 1 --2- 172.21.15.98:0/768545815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b5806c1d0 0x7f4b5815ccc0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00b410 tx=0x7f4b4c00b8d0 comp rx=0 tx=0).stop 2024-09-19T08:45:59.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.767+0000 7f4b5da55640 1 -- 172.21.15.98:0/768545815 shutdown_connections 2024-09-19T08:45:59.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.767+0000 7f4b5da55640 1 --2- 172.21.15.98:0/768545815 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4b44067ef0 0x7f4b4406a3b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:59.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.767+0000 7f4b5da55640 1 --2- 172.21.15.98:0/768545815 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4b5815c550 0x7f4b580ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:59.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.767+0000 7f4b5da55640 1 --2- 172.21.15.98:0/768545815 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4b580cf900 0x7f4b58072930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:59.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.767+0000 7f4b5da55640 1 --2- 172.21.15.98:0/768545815 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4b5806c1d0 0x7f4b5815ccc0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:59.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.767+0000 7f4b5da55640 1 -- 172.21.15.98:0/768545815 >> 172.21.15.98:0/768545815 conn(0x7f4b580b2010 msgr2=0x7f4b581599c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:59.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.767+0000 7f4b5da55640 1 -- 172.21.15.98:0/768545815 shutdown_connections 2024-09-19T08:45:59.768 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.767+0000 7f4b5da55640 1 -- 172.21.15.98:0/768545815 wait complete. 2024-09-19T08:45:59.789 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-19T08:45:59.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.903+0000 7f10e54c7640 1 Processor -- start 2024-09-19T08:45:59.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.903+0000 7f10e54c7640 1 -- start start 2024-09-19T08:45:59.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.907+0000 7f10e54c7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10e0141110 0x7f10e0165940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:59.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.907+0000 7f10e54c7640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10e015ea10 con 0x7f10e0140d30 2024-09-19T08:45:59.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.907+0000 7f10e54c7640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10e015ecf0 con 0x7f10e006d880 2024-09-19T08:45:59.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.907+0000 7f10e54c7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f10e015e7c0 con 0x7f10e0141110 2024-09-19T08:45:59.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.907+0000 7f10deffd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f10e0140d30 0x7f10e006d170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42404/0 (socket says 172.21.15.98:42404) 2024-09-19T08:45:59.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.907+0000 7f10deffd640 1 -- 172.21.15.98:0/4160587842 learned_addr learned my addr 172.21.15.98:0/4160587842 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:45:59.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.907+0000 7f10df7fe640 1 --2- 172.21.15.98:0/4160587842 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10e0141110 0x7f10e0165940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:59.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.907+0000 7f10de7fc640 1 -- 172.21.15.98:0/4160587842 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2556587894 0 0) 0x7f10e015ea10 con 0x7f10e0140d30 2024-09-19T08:45:59.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.907+0000 7f10de7fc640 1 -- 172.21.15.98:0/4160587842 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10cc003530 con 0x7f10e0140d30 2024-09-19T08:45:59.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.907+0000 7f10de7fc640 1 -- 172.21.15.98:0/4160587842 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2724143511 0 0) 0x7f10cc003530 con 0x7f10e0140d30 2024-09-19T08:45:59.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.907+0000 7f10de7fc640 1 -- 172.21.15.98:0/4160587842 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10e0141110 msgr2=0x7f10e0165940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:59.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.907+0000 7f10de7fc640 1 --2- 172.21.15.98:0/4160587842 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10e0141110 0x7f10e0165940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:59.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.907+0000 7f10de7fc640 1 -- 172.21.15.98:0/4160587842 >> v1:172.21.15.139:6789/0 conn(0x7f10e006d880 legacy=0x7f10e00b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:59.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.907+0000 7f10de7fc640 1 -- 172.21.15.98:0/4160587842 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e015efd0 con 0x7f10e0140d30 2024-09-19T08:45:59.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.907+0000 7f10de7fc640 1 -- 172.21.15.98:0/4160587842 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f10d00016a0 con 0x7f10e0140d30 2024-09-19T08:45:59.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.907+0000 7f10de7fc640 1 -- 172.21.15.98:0/4160587842 >> v1:172.21.15.98:6789/0 conn(0x7f10e0140d30 legacy=0x7f10e006d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:59.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.907+0000 7f10de7fc640 1 --2- 172.21.15.98:0/4160587842 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10e0141110 0x7f10cc0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:59.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.907+0000 7f10de7fc640 1 --2- 172.21.15.98:0/4160587842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10cc003f30 0x7f10cc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:59.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.907+0000 7f10de7fc640 1 --2- 172.21.15.98:0/4160587842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10cc004d50 0x7f10cc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:59.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.907+0000 7f10de7fc640 1 -- 172.21.15.98:0/4160587842 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f10e015e7c0 con 0x7f10cc003f30 2024-09-19T08:45:59.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.907+0000 7f10de7fc640 1 -- 172.21.15.98:0/4160587842 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc00ba80 con 0x7f10cc004d50 2024-09-19T08:45:59.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.907+0000 7f10de7fc640 1 -- 172.21.15.98:0/4160587842 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f10cc00bc80 con 0x7f10e0141110 2024-09-19T08:45:59.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.907+0000 7f10d67fc640 1 --2- 172.21.15.98:0/4160587842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10cc003f30 0x7f10cc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:59.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.907+0000 7f10deffd640 1 --2- 172.21.15.98:0/4160587842 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10e0141110 0x7f10cc0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:59.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.907+0000 7f10d67fc640 1 -- 172.21.15.98:0/4160587842 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10e0141110 msgr2=0x7f10cc0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:59.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.907+0000 7f10d67fc640 1 --2- 172.21.15.98:0/4160587842 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10e0141110 0x7f10cc0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:59.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.907+0000 7f10d67fc640 1 -- 172.21.15.98:0/4160587842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10cc004d50 msgr2=0x7f10cc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:45:59.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.907+0000 7f10d67fc640 1 --2- 172.21.15.98:0/4160587842 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10cc004d50 0x7f10cc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:59.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.907+0000 7f10d67fc640 1 -- 172.21.15.98:0/4160587842 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10cc003530 con 0x7f10cc003f30 2024-09-19T08:45:59.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.907+0000 7f10d67fc640 1 --2- 172.21.15.98:0/4160587842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10cc003f30 0x7f10cc004810 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f10c8001860 tx=0x7f10c801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15284dbcbd6e86fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:59.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.907+0000 7f10de7fc640 1 -- 172.21.15.98:0/4160587842 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c80b5070 con 0x7f10cc003f30 2024-09-19T08:45:59.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.907+0000 7f10de7fc640 1 -- 172.21.15.98:0/4160587842 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10c80a8890 con 0x7f10cc003f30 2024-09-19T08:45:59.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.907+0000 7f10de7fc640 1 -- 172.21.15.98:0/4160587842 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c8001460 con 0x7f10cc003f30 2024-09-19T08:45:59.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.911+0000 7f10e54c7640 1 -- 172.21.15.98:0/4160587842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10cc003f30 msgr2=0x7f10cc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:59.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.911+0000 7f10e54c7640 1 --2- 172.21.15.98:0/4160587842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10cc003f30 0x7f10cc004810 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f10c8001860 tx=0x7f10c801ec70 comp rx=0 tx=0).stop 2024-09-19T08:45:59.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.911+0000 7f10df7fe640 1 -- 172.21.15.98:0/4160587842 reap_dead start 2024-09-19T08:45:59.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.911+0000 7f10e54c7640 1 -- 172.21.15.98:0/4160587842 shutdown_connections 2024-09-19T08:45:59.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.911+0000 7f10e54c7640 1 -- 172.21.15.98:0/4160587842 >> 172.21.15.98:0/4160587842 conn(0x7f10e00b2010 msgr2=0x7f10e00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:45:59.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.911+0000 7f10e54c7640 1 -- 172.21.15.98:0/4160587842 shutdown_connections 2024-09-19T08:45:59.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.911+0000 7f10e54c7640 1 -- 172.21.15.98:0/4160587842 wait complete. 2024-09-19T08:45:59.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.911+0000 7f10e54c7640 1 Processor -- start 2024-09-19T08:45:59.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.911+0000 7f10e54c7640 1 -- start start 2024-09-19T08:45:59.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.911+0000 7f10e54c7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10cc003f30 0x7f10e0157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:59.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.911+0000 7f10e54c7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10e015b940 0x7f10e015bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:59.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.911+0000 7f10e54c7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10e01e51a0 0x7f10e01e5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:59.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.911+0000 7f10e54c7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f10e00b3a20 con 0x7f10e01e51a0 2024-09-19T08:45:59.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.911+0000 7f10e54c7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f10e00ce1a0 con 0x7f10e015b940 2024-09-19T08:45:59.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.911+0000 7f10e54c7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f10e00b3ba0 con 0x7f10cc003f30 2024-09-19T08:45:59.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.911+0000 7f10df7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10e01e51a0 0x7f10e01e5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:59.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.911+0000 7f10df7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10e01e51a0 0x7f10e01e5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46718/0 (socket says 172.21.15.98:46718) 2024-09-19T08:45:59.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.911+0000 7f10df7fe640 1 -- 172.21.15.98:0/3930550966 learned_addr learned my addr 172.21.15.98:0/3930550966 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:45:59.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.911+0000 7f10deffd640 1 --2- 172.21.15.98:0/3930550966 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10cc003f30 0x7f10e0157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:59.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.911+0000 7f10d67fc640 1 --2- 172.21.15.98:0/3930550966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10e015b940 0x7f10e015bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:59.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.911+0000 7f10df7fe640 1 -- 172.21.15.98:0/3930550966 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10cc003f30 msgr2=0x7f10e0157f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:59.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.911+0000 7f10df7fe640 1 --2- 172.21.15.98:0/3930550966 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10cc003f30 0x7f10e0157f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:59.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.911+0000 7f10df7fe640 1 -- 172.21.15.98:0/3930550966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10e015b940 msgr2=0x7f10e015bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:45:59.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.911+0000 7f10df7fe640 1 --2- 172.21.15.98:0/3930550966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10e015b940 0x7f10e015bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:45:59.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.911+0000 7f10df7fe640 1 -- 172.21.15.98:0/3930550966 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e006d6b0 con 0x7f10e01e51a0 2024-09-19T08:45:59.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.911+0000 7f10d67fc640 1 --2- 172.21.15.98:0/3930550966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10e015b940 0x7f10e015bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:45:59.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.911+0000 7f10df7fe640 1 --2- 172.21.15.98:0/3930550966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10e01e51a0 0x7f10e01e5650 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f10e00b2d90 tx=0x7f10d80075c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:59.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.911+0000 7f10d7fff640 1 -- 172.21.15.98:0/3930550966 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8009e90 con 0x7f10e01e51a0 2024-09-19T08:45:59.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.911+0000 7f10d7fff640 1 -- 172.21.15.98:0/3930550966 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10d8007db0 con 0x7f10e01e51a0 2024-09-19T08:45:59.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.911+0000 7f10e54c7640 1 -- 172.21.15.98:0/3930550966 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e01402d0 con 0x7f10e01e51a0 2024-09-19T08:45:59.914 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.911+0000 7f10d7fff640 1 -- 172.21.15.98:0/3930550966 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8003be0 con 0x7f10e01e51a0 2024-09-19T08:45:59.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.911+0000 7f10e54c7640 1 -- 172.21.15.98:0/3930550966 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e015c7e0 con 0x7f10e01e51a0 2024-09-19T08:45:59.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.915+0000 7f10e54c7640 1 -- 172.21.15.98:0/3930550966 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10e00b3310 con 0x7f10e01e51a0 2024-09-19T08:45:59.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.915+0000 7f10d7fff640 1 -- 172.21.15.98:0/3930550966 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f10d8016470 con 0x7f10e01e51a0 2024-09-19T08:45:59.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.915+0000 7f10d7fff640 1 --2- 172.21.15.98:0/3930550966 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f10c0068090 0x7f10c006a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:45:59.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.915+0000 7f10d7fff640 1 -- 172.21.15.98:0/3930550966 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 4885+0+0 (secure 0 0 0) 0x7f10d8086ec0 con 0x7f10e01e51a0 2024-09-19T08:45:59.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.915+0000 7f10deffd640 1 --2- 172.21.15.98:0/3930550966 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f10c0068090 0x7f10c006a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:45:59.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.915+0000 7f10deffd640 1 --2- 172.21.15.98:0/3930550966 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f10c0068090 0x7f10c006a550 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f10d0001060 tx=0x7f10d0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:45:59.917 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:45:59.919+0000 7f10d7fff640 1 -- 172.21.15.98:0/3930550966 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f10d8024070 con 0x7f10e01e51a0 2024-09-19T08:46:00.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.195+0000 7f10e54c7640 1 -- 172.21.15.98:0/3930550966 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f10e01584a0 con 0x7f10e01e51a0 2024-09-19T08:46:00.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.195+0000 7f10d7fff640 1 -- 172.21.15.98:0/3930550966 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) ==== 80+0+5304 (secure 0 0 0) 0x7f10d80573e0 con 0x7f10e01e51a0 2024-09-19T08:46:00.196 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:46:00.196 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:46:00.196 INFO:teuthology.orchestra.run.smithi098.stdout: "auth_dump": [ 2024-09-19T08:46:00.197 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:00.197 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.a", 2024-09-19T08:46:00.197 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmuujGBRAAa2k/kW68NJI6nUKx+W+G2Q==", 2024-09-19T08:46:00.197 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:00.197 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:46:00.197 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:46:00.197 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:46:00.197 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:46:00.197 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:00.197 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:00.197 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:00.197 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.b", 2024-09-19T08:46:00.197 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmFoojFBAAaOZPpa41fdjBPEn3CGsJnA==", 2024-09-19T08:46:00.198 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:00.198 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:46:00.198 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:46:00.198 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:46:00.198 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:46:00.199 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:00.199 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:00.199 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:00.199 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.c", 2024-09-19T08:46:00.199 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmcQcDDhAAF0QJMlljMH+DiHfWW+/Sog==", 2024-09-19T08:46:00.199 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:00.199 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:46:00.199 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:46:00.199 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:46:00.199 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:46:00.199 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:00.199 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:00.200 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:00.200 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.d", 2024-09-19T08:46:00.200 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmwWmqHBAAf15VcrdeHf8AzQjGsN9LoA==", 2024-09-19T08:46:00.200 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:00.200 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:46:00.200 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:46:00.200 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:46:00.200 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:46:00.200 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:00.200 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:00.200 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:00.200 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.0", 2024-09-19T08:46:00.200 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDC4utm9PjNKxAAa6SG9AE4smPiu4azRAFKvg==", 2024-09-19T08:46:00.200 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:00.201 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:46:00.201 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:46:00.201 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:46:00.201 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:00.201 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:00.201 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:00.201 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.1", 2024-09-19T08:46:00.201 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDG4utmHy6UCxAAN6hpG4DJEp81iG/KoloTvQ==", 2024-09-19T08:46:00.201 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:00.201 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:46:00.201 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:46:00.201 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:46:00.201 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:00.202 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:00.202 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:00.202 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.2", 2024-09-19T08:46:00.202 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDJ4utmUVP4GhAAJNAL30sEAseoDG2fE6YbZQ==", 2024-09-19T08:46:00.202 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:00.202 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:46:00.202 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:46:00.202 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:46:00.202 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:00.202 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:00.202 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:00.202 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.3", 2024-09-19T08:46:00.202 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDM4utmTKgoLRAAiCrPRLML+wvjOt6RhKay3w==", 2024-09-19T08:46:00.203 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:00.203 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:46:00.203 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:46:00.203 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:46:00.203 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:00.203 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:00.203 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:00.203 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.4", 2024-09-19T08:46:00.203 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDS4utmAf6bMhAAI3uENuFoXLicUeuspgFjYA==", 2024-09-19T08:46:00.203 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:00.203 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:46:00.203 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:46:00.203 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:46:00.204 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:00.204 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:00.204 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:00.204 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.5", 2024-09-19T08:46:00.204 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDW4utmRjxwEhAAF1A2GtrZXVx9BlmjHjhUDQ==", 2024-09-19T08:46:00.204 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:00.204 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:46:00.204 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:46:00.204 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:46:00.204 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:00.204 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:00.204 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:00.205 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.6", 2024-09-19T08:46:00.205 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDZ4utmSk/XIRAAROSfzGL7smR88ZkDSx5jAg==", 2024-09-19T08:46:00.205 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:00.205 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:46:00.205 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:46:00.205 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:46:00.205 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:00.205 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:00.205 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:00.205 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.7", 2024-09-19T08:46:00.205 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDc4utmXfpbMxAAqEM4I/ACFvfu/sMA8bMcrQ==", 2024-09-19T08:46:00.205 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:00.205 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:46:00.205 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:46:00.206 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:46:00.206 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:00.206 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:00.206 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:00.206 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.0", 2024-09-19T08:46:00.206 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utm0vNpIhAA4S1E8dFZ0+ibmSKhmZ+bIA==", 2024-09-19T08:46:00.206 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:00.206 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:46:00.206 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow r", 2024-09-19T08:46:00.206 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-19T08:46:00.206 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:00.206 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:00.206 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:00.207 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.1", 2024-09-19T08:46:00.207 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmx7xyKhAAC84lykPEb01MiypXKQVREQ==", 2024-09-19T08:46:00.207 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:00.207 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:46:00.207 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-19T08:46:00.207 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-19T08:46:00.207 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-19T08:46:00.207 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:00.207 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:00.207 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:00.207 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.2", 2024-09-19T08:46:00.207 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmteOQJxAAtTF2Py21ZE2WRQjeQSrkKQ==", 2024-09-19T08:46:00.207 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:00.207 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:46:00.208 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-19T08:46:00.208 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-19T08:46:00.208 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-19T08:46:00.208 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:00.208 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:00.208 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:00.208 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.3", 2024-09-19T08:46:00.208 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmKN92LxAAMqraZOWRT6IIfx5AuDWS2g==", 2024-09-19T08:46:00.208 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:00.208 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:46:00.208 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-19T08:46:00.208 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-19T08:46:00.208 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-19T08:46:00.209 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:00.209 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:00.209 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:00.209 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.admin", 2024-09-19T08:46:00.209 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC+4utmLH97KhAA1DTYW5bTrGUG+7cb8+d7gA==", 2024-09-19T08:46:00.209 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:00.209 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-09-19T08:46:00.209 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:46:00.209 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:46:00.209 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:46:00.209 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:00.209 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:00.210 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:00.210 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mgr.x", 2024-09-19T08:46:00.210 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmbKuyAhAA0/XjHUIPOIV/YvNEhB8rEQ==", 2024-09-19T08:46:00.210 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:00.210 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-09-19T08:46:00.210 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile mgr", 2024-09-19T08:46:00.210 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:46:00.210 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:00.210 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:00.210 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:00.210 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mgr.y", 2024-09-19T08:46:00.210 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC+4utmU+ZrOxAAsvcwINH6qyxfN7N+rDrkqg==", 2024-09-19T08:46:00.210 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:00.211 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-09-19T08:46:00.211 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile mgr", 2024-09-19T08:46:00.211 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:46:00.211 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:00.211 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:00.211 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-19T08:46:00.211 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:46:00.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.199+0000 7f10e54c7640 1 -- 172.21.15.98:0/3930550966 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f10c0068090 msgr2=0x7f10c006a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:00.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.199+0000 7f10e54c7640 1 --2- 172.21.15.98:0/3930550966 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f10c0068090 0x7f10c006a550 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f10d0001060 tx=0x7f10d0009040 comp rx=0 tx=0).stop 2024-09-19T08:46:00.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.199+0000 7f10e54c7640 1 -- 172.21.15.98:0/3930550966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10e01e51a0 msgr2=0x7f10e01e5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:00.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.199+0000 7f10e54c7640 1 --2- 172.21.15.98:0/3930550966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10e01e51a0 0x7f10e01e5650 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f10e00b2d90 tx=0x7f10d80075c0 comp rx=0 tx=0).stop 2024-09-19T08:46:00.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.199+0000 7f10e54c7640 1 -- 172.21.15.98:0/3930550966 shutdown_connections 2024-09-19T08:46:00.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.199+0000 7f10e54c7640 1 --2- 172.21.15.98:0/3930550966 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f10c0068090 0x7f10c006a550 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:00.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.199+0000 7f10e54c7640 1 --2- 172.21.15.98:0/3930550966 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f10e01e51a0 0x7f10e01e5650 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:00.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.199+0000 7f10e54c7640 1 --2- 172.21.15.98:0/3930550966 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f10e015b940 0x7f10e015bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:00.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.199+0000 7f10e54c7640 1 --2- 172.21.15.98:0/3930550966 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f10cc003f30 0x7f10e0157f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:00.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.199+0000 7f10e54c7640 1 -- 172.21.15.98:0/3930550966 >> 172.21.15.98:0/3930550966 conn(0x7f10e00b2010 msgr2=0x7f10e0072ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:00.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.199+0000 7f10e54c7640 1 -- 172.21.15.98:0/3930550966 shutdown_connections 2024-09-19T08:46:00.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.199+0000 7f10e54c7640 1 -- 172.21.15.98:0/3930550966 wait complete. 2024-09-19T08:46:00.214 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-19T08:46:00.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.323+0000 7fd6ee060640 1 Processor -- start 2024-09-19T08:46:00.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.323+0000 7fd6ee060640 1 -- start start 2024-09-19T08:46:00.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.323+0000 7fd6ee060640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6e80733b0 0x7fd6e80cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:00.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.323+0000 7fd6ee060640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6e806d310 con 0x7fd6e815c550 2024-09-19T08:46:00.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.323+0000 7fd6ee060640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6e806d5f0 con 0x7fd6e806c3a0 2024-09-19T08:46:00.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.323+0000 7fd6ee060640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd6e806d8d0 con 0x7fd6e80733b0 2024-09-19T08:46:00.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.323+0000 7fd6e77fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fd6e815c550 0x7fd6e806bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42408/0 (socket says 172.21.15.98:42408) 2024-09-19T08:46:00.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.323+0000 7fd6e77fe640 1 -- 172.21.15.98:0/4245230582 learned_addr learned my addr 172.21.15.98:0/4245230582 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:00.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.323+0000 7fd6e6ffd640 1 -- 172.21.15.98:0/4245230582 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1738391989 0 0) 0x7fd6e806d310 con 0x7fd6e815c550 2024-09-19T08:46:00.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.323+0000 7fd6e6ffd640 1 -- 172.21.15.98:0/4245230582 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6d4003530 con 0x7fd6e815c550 2024-09-19T08:46:00.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.323+0000 7fd6e7fff640 1 --2- 172.21.15.98:0/4245230582 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6e80733b0 0x7fd6e80cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:00.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.323+0000 7fd6e6ffd640 1 -- 172.21.15.98:0/4245230582 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 698848426 0 0) 0x7fd6d4003530 con 0x7fd6e815c550 2024-09-19T08:46:00.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.323+0000 7fd6e6ffd640 1 -- 172.21.15.98:0/4245230582 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6e80733b0 msgr2=0x7fd6e80cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:00.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.323+0000 7fd6e6ffd640 1 --2- 172.21.15.98:0/4245230582 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6e80733b0 0x7fd6e80cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:00.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.323+0000 7fd6e6ffd640 1 -- 172.21.15.98:0/4245230582 >> v1:172.21.15.139:6789/0 conn(0x7fd6e806c3a0 legacy=0x7fd6e80cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:00.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.323+0000 7fd6e6ffd640 1 -- 172.21.15.98:0/4245230582 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e806dbb0 con 0x7fd6e815c550 2024-09-19T08:46:00.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.323+0000 7fd6e6ffd640 1 -- 172.21.15.98:0/4245230582 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fd6d8001680 con 0x7fd6e815c550 2024-09-19T08:46:00.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.323+0000 7fd6e6ffd640 1 -- 172.21.15.98:0/4245230582 >> v1:172.21.15.98:6789/0 conn(0x7fd6e815c550 legacy=0x7fd6e806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:00.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.323+0000 7fd6e6ffd640 1 --2- 172.21.15.98:0/4245230582 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6d40039f0 0x7fd6d4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:00.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.323+0000 7fd6e6ffd640 1 --2- 172.21.15.98:0/4245230582 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6e80733b0 0x7fd6d4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:00.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.323+0000 7fd6e6ffd640 1 --2- 172.21.15.98:0/4245230582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6d4004d50 0x7fd6d40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:00.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.323+0000 7fd6e6ffd640 1 -- 172.21.15.98:0/4245230582 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e806d8d0 con 0x7fd6d40039f0 2024-09-19T08:46:00.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.323+0000 7fd6e6ffd640 1 -- 172.21.15.98:0/4245230582 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d400ba80 con 0x7fd6d4004d50 2024-09-19T08:46:00.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.323+0000 7fd6e6ffd640 1 -- 172.21.15.98:0/4245230582 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd6d400bc80 con 0x7fd6e80733b0 2024-09-19T08:46:00.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.323+0000 7fd6e77fe640 1 --2- 172.21.15.98:0/4245230582 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6d40039f0 0x7fd6d4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:00.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.323+0000 7fd6deffd640 1 --2- 172.21.15.98:0/4245230582 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6e80733b0 0x7fd6d4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:00.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.323+0000 7fd6e77fe640 1 -- 172.21.15.98:0/4245230582 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6e80733b0 msgr2=0x7fd6d4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:00.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.323+0000 7fd6e77fe640 1 --2- 172.21.15.98:0/4245230582 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6e80733b0 0x7fd6d4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:00.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.323+0000 7fd6e77fe640 1 -- 172.21.15.98:0/4245230582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6d4004d50 msgr2=0x7fd6d40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:00.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.323+0000 7fd6e77fe640 1 --2- 172.21.15.98:0/4245230582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6d4004d50 0x7fd6d40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:00.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.323+0000 7fd6e77fe640 1 -- 172.21.15.98:0/4245230582 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d4003530 con 0x7fd6d40039f0 2024-09-19T08:46:00.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.327+0000 7fd6e77fe640 1 --2- 172.21.15.98:0/4245230582 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6d40039f0 0x7fd6d4003e10 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fd6d800c190 tx=0x7fd6d801cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd30eabcb850262 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:00.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.327+0000 7fd6e6ffd640 1 -- 172.21.15.98:0/4245230582 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8007d00 con 0x7fd6d40039f0 2024-09-19T08:46:00.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.327+0000 7fd6e6ffd640 1 -- 172.21.15.98:0/4245230582 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6d80a9020 con 0x7fd6d40039f0 2024-09-19T08:46:00.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.327+0000 7fd6e6ffd640 1 -- 172.21.15.98:0/4245230582 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8005ae0 con 0x7fd6d40039f0 2024-09-19T08:46:00.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.327+0000 7fd6ee060640 1 -- 172.21.15.98:0/4245230582 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6d40039f0 msgr2=0x7fd6d4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:00.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.327+0000 7fd6ee060640 1 --2- 172.21.15.98:0/4245230582 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6d40039f0 0x7fd6d4003e10 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fd6d800c190 tx=0x7fd6d801cc40 comp rx=0 tx=0).stop 2024-09-19T08:46:00.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.327+0000 7fd6e7fff640 1 -- 172.21.15.98:0/4245230582 reap_dead start 2024-09-19T08:46:00.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.327+0000 7fd6ee060640 1 -- 172.21.15.98:0/4245230582 shutdown_connections 2024-09-19T08:46:00.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.327+0000 7fd6ee060640 1 -- 172.21.15.98:0/4245230582 >> 172.21.15.98:0/4245230582 conn(0x7fd6e80b2010 msgr2=0x7fd6e80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:00.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.327+0000 7fd6ee060640 1 -- 172.21.15.98:0/4245230582 shutdown_connections 2024-09-19T08:46:00.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.327+0000 7fd6ee060640 1 -- 172.21.15.98:0/4245230582 wait complete. 2024-09-19T08:46:00.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.327+0000 7fd6ee060640 1 Processor -- start 2024-09-19T08:46:00.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.327+0000 7fd6ee060640 1 -- start start 2024-09-19T08:46:00.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.327+0000 7fd6ee060640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6d40039f0 0x7fd6e815f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:00.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.327+0000 7fd6ee060640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6e81f50f0 0x7fd6e81585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:00.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.327+0000 7fd6ee060640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6e815b0f0 0x7fd6e8142630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:00.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.327+0000 7fd6ee060640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e8167c40 con 0x7fd6d40039f0 2024-09-19T08:46:00.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.327+0000 7fd6ee060640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e815cb40 con 0x7fd6e815b0f0 2024-09-19T08:46:00.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.327+0000 7fd6ee060640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd6e8167dc0 con 0x7fd6e81f50f0 2024-09-19T08:46:00.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.327+0000 7fd6e77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6d40039f0 0x7fd6e815f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:00.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.327+0000 7fd6e77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6d40039f0 0x7fd6e815f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46746/0 (socket says 172.21.15.98:46746) 2024-09-19T08:46:00.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.327+0000 7fd6e77fe640 1 -- 172.21.15.98:0/939529384 learned_addr learned my addr 172.21.15.98:0/939529384 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:00.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.327+0000 7fd6e77fe640 1 -- 172.21.15.98:0/939529384 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6e81f50f0 msgr2=0x7fd6e81585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:00.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.327+0000 7fd6e77fe640 1 --2- 172.21.15.98:0/939529384 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6e81f50f0 0x7fd6e81585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:00.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.327+0000 7fd6e7fff640 1 --2- 172.21.15.98:0/939529384 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6e815b0f0 0x7fd6e8142630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:00.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.327+0000 7fd6e77fe640 1 -- 172.21.15.98:0/939529384 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6e815b0f0 msgr2=0x7fd6e8142630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:00.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.327+0000 7fd6e77fe640 1 --2- 172.21.15.98:0/939529384 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6e815b0f0 0x7fd6e8142630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:00.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.327+0000 7fd6e77fe640 1 -- 172.21.15.98:0/939529384 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e815f9a0 con 0x7fd6d40039f0 2024-09-19T08:46:00.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.327+0000 7fd6e77fe640 1 --2- 172.21.15.98:0/939529384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6d40039f0 0x7fd6e815f310 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fd6d80ab040 tx=0x7fd6d4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:00.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.331+0000 7fd6e4ff9640 1 -- 172.21.15.98:0/939529384 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8004c00 con 0x7fd6d40039f0 2024-09-19T08:46:00.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.331+0000 7fd6e4ff9640 1 -- 172.21.15.98:0/939529384 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6d8004da0 con 0x7fd6d40039f0 2024-09-19T08:46:00.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.331+0000 7fd6ee060640 1 -- 172.21.15.98:0/939529384 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e815ce60 con 0x7fd6d40039f0 2024-09-19T08:46:00.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.331+0000 7fd6e4ff9640 1 -- 172.21.15.98:0/939529384 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d80072d0 con 0x7fd6d40039f0 2024-09-19T08:46:00.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.331+0000 7fd6ee060640 1 -- 172.21.15.98:0/939529384 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e815d420 con 0x7fd6d40039f0 2024-09-19T08:46:00.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.331+0000 7fd6e4ff9640 1 -- 172.21.15.98:0/939529384 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fd6d8007470 con 0x7fd6d40039f0 2024-09-19T08:46:00.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.331+0000 7fd6ee060640 1 -- 172.21.15.98:0/939529384 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6e81f5530 con 0x7fd6d40039f0 2024-09-19T08:46:00.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.331+0000 7fd6e4ff9640 1 --2- 172.21.15.98:0/939529384 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd6c8067f20 0x7fd6c806a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:00.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.331+0000 7fd6e4ff9640 1 -- 172.21.15.98:0/939529384 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 4885+0+0 (secure 0 0 0) 0x7fd6d811fe80 con 0x7fd6d40039f0 2024-09-19T08:46:00.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.331+0000 7fd6deffd640 1 --2- 172.21.15.98:0/939529384 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd6c8067f20 0x7fd6c806a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:00.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.331+0000 7fd6deffd640 1 --2- 172.21.15.98:0/939529384 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd6c8067f20 0x7fd6c806a3e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0001060 tx=0x7fd6d0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:00.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.335+0000 7fd6e4ff9640 1 -- 172.21.15.98:0/939529384 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd6d8016950 con 0x7fd6d40039f0 2024-09-19T08:46:00.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.483+0000 7fd6ee060640 1 -- 172.21.15.98:0/939529384 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fd6e80cf3c0 con 0x7fd6d40039f0 2024-09-19T08:46:00.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.483+0000 7fd6e4ff9640 1 -- 172.21.15.98:0/939529384 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v142) ==== 84+0+65 (secure 0 0 0) 0x7fd6d80f03b0 con 0x7fd6d40039f0 2024-09-19T08:46:00.484 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:46:00.484 INFO:teuthology.orchestra.run.smithi098.stdout:[ 2024-09-19T08:46:00.484 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:00.485 INFO:teuthology.orchestra.run.smithi098.stdout: "poolnum": 1, 2024-09-19T08:46:00.485 INFO:teuthology.orchestra.run.smithi098.stdout: "poolname": ".mgr" 2024-09-19T08:46:00.485 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:00.485 INFO:teuthology.orchestra.run.smithi098.stdout:] 2024-09-19T08:46:00.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.487+0000 7fd6ee060640 1 -- 172.21.15.98:0/939529384 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd6c8067f20 msgr2=0x7fd6c806a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:00.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.487+0000 7fd6ee060640 1 --2- 172.21.15.98:0/939529384 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd6c8067f20 0x7fd6c806a3e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0001060 tx=0x7fd6d0009040 comp rx=0 tx=0).stop 2024-09-19T08:46:00.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.487+0000 7fd6ee060640 1 -- 172.21.15.98:0/939529384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6d40039f0 msgr2=0x7fd6e815f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:00.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.487+0000 7fd6ee060640 1 --2- 172.21.15.98:0/939529384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6d40039f0 0x7fd6e815f310 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fd6d80ab040 tx=0x7fd6d4005840 comp rx=0 tx=0).stop 2024-09-19T08:46:00.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.487+0000 7fd6ee060640 1 -- 172.21.15.98:0/939529384 shutdown_connections 2024-09-19T08:46:00.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.487+0000 7fd6ee060640 1 --2- 172.21.15.98:0/939529384 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd6c8067f20 0x7fd6c806a3e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:00.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.487+0000 7fd6ee060640 1 --2- 172.21.15.98:0/939529384 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6e815b0f0 0x7fd6e8142630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:00.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.487+0000 7fd6ee060640 1 --2- 172.21.15.98:0/939529384 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd6e81f50f0 0x7fd6e81585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:00.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.487+0000 7fd6ee060640 1 --2- 172.21.15.98:0/939529384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd6d40039f0 0x7fd6e815f310 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:00.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.487+0000 7fd6ee060640 1 -- 172.21.15.98:0/939529384 >> 172.21.15.98:0/939529384 conn(0x7fd6e80b2010 msgr2=0x7fd6e80cd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:00.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.487+0000 7fd6ee060640 1 -- 172.21.15.98:0/939529384 shutdown_connections 2024-09-19T08:46:00.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.487+0000 7fd6ee060640 1 -- 172.21.15.98:0/939529384 wait complete. 2024-09-19T08:46:00.501 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T08:46:00.502 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T08:46:00.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.627+0000 7f9100b54640 1 Processor -- start 2024-09-19T08:46:00.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.627+0000 7f9100b54640 1 -- start start 2024-09-19T08:46:00.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.627+0000 7f9100b54640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f90fc15c290 0x7f90fc0cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:00.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.627+0000 7f9100b54640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f90fc06d4c0 0x7f90fc06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:00.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.627+0000 7f9100b54640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90fc06dde0 con 0x7f90fc0cf700 2024-09-19T08:46:00.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.627+0000 7f9100b54640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f90fc160e80 con 0x7f90fc15c290 2024-09-19T08:46:00.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.627+0000 7f9100b54640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f90fc0cd510 con 0x7f90fc06d4c0 2024-09-19T08:46:00.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.627+0000 7f90fa59c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f90fc15c290 0x7f90fc0cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:00.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.627+0000 7f90fa59c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f90fc15c290 0x7f90fc0cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46752/0 (socket says 172.21.15.98:46752) 2024-09-19T08:46:00.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.627+0000 7f90fa59c640 1 -- 172.21.15.98:0/2175425225 learned_addr learned my addr 172.21.15.98:0/2175425225 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:00.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.627+0000 7f90fad9d640 1 --2- 172.21.15.98:0/2175425225 >> v2:172.21.15.139:3300/0 conn(0x7f90fc06d4c0 0x7f90fc06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:00.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.627+0000 7f90fa59c640 1 -- 172.21.15.98:0/2175425225 >> v2:172.21.15.139:3300/0 conn(0x7f90fc06d4c0 msgr2=0x7f90fc06d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:00.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.627+0000 7f90fa59c640 1 --2- 172.21.15.98:0/2175425225 >> v2:172.21.15.139:3300/0 conn(0x7f90fc06d4c0 0x7f90fc06d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:00.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.627+0000 7f90fa59c640 1 -- 172.21.15.98:0/2175425225 >> v1:172.21.15.139:6789/0 conn(0x7f90fc0cf700 legacy=0x7f90fc15ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:00.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.627+0000 7f90fa59c640 1 -- 172.21.15.98:0/2175425225 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90fc1608c0 con 0x7f90fc15c290 2024-09-19T08:46:00.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.631+0000 7f90fa59c640 1 --2- 172.21.15.98:0/2175425225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90fc15c290 0x7f90fc0cd9a0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f90fc06e0d0 tx=0x7f90ec01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21e50973000b3a4f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:00.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.631+0000 7f90f9d9b640 1 -- 172.21.15.98:0/2175425225 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ec0b0070 con 0x7f90fc15c290 2024-09-19T08:46:00.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.631+0000 7f90f9d9b640 1 -- 172.21.15.98:0/2175425225 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90ec0ab600 con 0x7f90fc15c290 2024-09-19T08:46:00.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.631+0000 7f90f9d9b640 1 -- 172.21.15.98:0/2175425225 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ec0ab9d0 con 0x7f90fc15c290 2024-09-19T08:46:00.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.631+0000 7f9100b54640 1 -- 172.21.15.98:0/2175425225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90fc15c290 msgr2=0x7f90fc0cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:00.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.631+0000 7f9100b54640 1 --2- 172.21.15.98:0/2175425225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90fc15c290 0x7f90fc0cd9a0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f90fc06e0d0 tx=0x7f90ec01c490 comp rx=0 tx=0).stop 2024-09-19T08:46:00.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.631+0000 7f9100b54640 1 -- 172.21.15.98:0/2175425225 shutdown_connections 2024-09-19T08:46:00.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.631+0000 7f9100b54640 1 --2- 172.21.15.98:0/2175425225 >> v2:172.21.15.139:3300/0 conn(0x7f90fc06d4c0 0x7f90fc06d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:00.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.631+0000 7f9100b54640 1 --2- 172.21.15.98:0/2175425225 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90fc15c290 0x7f90fc0cd9a0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:00.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.631+0000 7f9100b54640 1 -- 172.21.15.98:0/2175425225 >> 172.21.15.98:0/2175425225 conn(0x7f90fc06c090 msgr2=0x7f90fc06c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:00.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.631+0000 7f9100b54640 1 -- 172.21.15.98:0/2175425225 shutdown_connections 2024-09-19T08:46:00.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.631+0000 7f9100b54640 1 -- 172.21.15.98:0/2175425225 wait complete. 2024-09-19T08:46:00.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.631+0000 7f9100b54640 1 Processor -- start 2024-09-19T08:46:00.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.631+0000 7f9100b54640 1 -- start start 2024-09-19T08:46:00.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.631+0000 7f9100b54640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90fc06d4c0 0x7f90fc159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:00.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.631+0000 7f9100b54640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90fc0cf700 0x7f90fc0adb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:00.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.631+0000 7f9100b54640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f90fc15c290 0x7f90fc1581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:00.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.631+0000 7f9100b54640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f90fc15c7f0 con 0x7f90fc0cf700 2024-09-19T08:46:00.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.631+0000 7f9100b54640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f90fc15c670 con 0x7f90fc06d4c0 2024-09-19T08:46:00.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.631+0000 7f9100b54640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f90fc0b5400 con 0x7f90fc15c290 2024-09-19T08:46:00.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.631+0000 7f90f3fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90fc0cf700 0x7f90fc0adb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:00.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.631+0000 7f90f3fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90fc0cf700 0x7f90fc0adb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46768/0 (socket says 172.21.15.98:46768) 2024-09-19T08:46:00.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.631+0000 7f90f3fff640 1 -- 172.21.15.98:0/1676136087 learned_addr learned my addr 172.21.15.98:0/1676136087 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:00.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.635+0000 7f90fa59c640 1 --2- 172.21.15.98:0/1676136087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90fc06d4c0 0x7f90fc159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:00.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.635+0000 7f90f3fff640 1 -- 172.21.15.98:0/1676136087 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f90fc15c290 msgr2=0x7f90fc1581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:00.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.635+0000 7f90f3fff640 1 --2- 172.21.15.98:0/1676136087 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f90fc15c290 0x7f90fc1581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:00.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.635+0000 7f90f3fff640 1 -- 172.21.15.98:0/1676136087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90fc06d4c0 msgr2=0x7f90fc159e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:00.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.635+0000 7f90f3fff640 1 --2- 172.21.15.98:0/1676136087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90fc06d4c0 0x7f90fc159e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:00.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.635+0000 7f90f3fff640 1 -- 172.21.15.98:0/1676136087 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90fc0ae200 con 0x7f90fc0cf700 2024-09-19T08:46:00.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.635+0000 7f90fa59c640 1 --2- 172.21.15.98:0/1676136087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90fc06d4c0 0x7f90fc159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:00.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.635+0000 7f90f3fff640 1 --2- 172.21.15.98:0/1676136087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90fc0cf700 0x7f90fc0adb70 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f90e400c420 tx=0x7f90e400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:00.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.635+0000 7f90f2ffd640 1 -- 172.21.15.98:0/1676136087 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e40052d0 con 0x7f90fc0cf700 2024-09-19T08:46:00.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.635+0000 7f90f2ffd640 1 -- 172.21.15.98:0/1676136087 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90e4005470 con 0x7f90fc0cf700 2024-09-19T08:46:00.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.635+0000 7f9100b54640 1 -- 172.21.15.98:0/1676136087 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90fc0ac630 con 0x7f90fc0cf700 2024-09-19T08:46:00.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.635+0000 7f90f2ffd640 1 -- 172.21.15.98:0/1676136087 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e4007260 con 0x7f90fc0cf700 2024-09-19T08:46:00.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.635+0000 7f9100b54640 1 -- 172.21.15.98:0/1676136087 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90fc0acbb0 con 0x7f90fc0cf700 2024-09-19T08:46:00.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.635+0000 7f9100b54640 1 -- 172.21.15.98:0/1676136087 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90fc06d8a0 con 0x7f90fc0cf700 2024-09-19T08:46:00.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.635+0000 7f90f2ffd640 1 -- 172.21.15.98:0/1676136087 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f90e4007400 con 0x7f90fc0cf700 2024-09-19T08:46:00.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.635+0000 7f90f2ffd640 1 --2- 172.21.15.98:0/1676136087 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f90dc067f20 0x7f90dc06a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:00.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.635+0000 7f90f2ffd640 1 -- 172.21.15.98:0/1676136087 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 4885+0+0 (secure 0 0 0) 0x7f90e4085910 con 0x7f90fc0cf700 2024-09-19T08:46:00.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.635+0000 7f90fa59c640 1 --2- 172.21.15.98:0/1676136087 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f90dc067f20 0x7f90dc06a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:00.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.635+0000 7f90fa59c640 1 --2- 172.21.15.98:0/1676136087 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f90dc067f20 0x7f90dc06a3e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f90ec0ad000 tx=0x7f90ec008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:00.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.639+0000 7f90f2ffd640 1 -- 172.21.15.98:0/1676136087 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f90e4055ed0 con 0x7f90fc0cf700 2024-09-19T08:46:00.803 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:00.803+0000 7f9100b54640 1 -- 172.21.15.98:0/1676136087 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f90fc0b22f0 con 0x7f90fc0cf700 2024-09-19T08:46:01.770 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.771+0000 7f90f2ffd640 1 -- 172.21.15.98:0/1676136087 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v143) ==== 138+0+0 (secure 0 0 0) 0x7f90e40557d0 con 0x7f90fc0cf700 2024-09-19T08:46:01.770 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_metadata' created 2024-09-19T08:46:01.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.775+0000 7f9100b54640 1 -- 172.21.15.98:0/1676136087 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f90dc067f20 msgr2=0x7f90dc06a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:01.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.775+0000 7f9100b54640 1 --2- 172.21.15.98:0/1676136087 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f90dc067f20 0x7f90dc06a3e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f90ec0ad000 tx=0x7f90ec008040 comp rx=0 tx=0).stop 2024-09-19T08:46:01.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.775+0000 7f9100b54640 1 -- 172.21.15.98:0/1676136087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90fc0cf700 msgr2=0x7f90fc0adb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:01.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.775+0000 7f9100b54640 1 --2- 172.21.15.98:0/1676136087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90fc0cf700 0x7f90fc0adb70 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f90e400c420 tx=0x7f90e400c8e0 comp rx=0 tx=0).stop 2024-09-19T08:46:01.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.775+0000 7f9100b54640 1 -- 172.21.15.98:0/1676136087 shutdown_connections 2024-09-19T08:46:01.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.775+0000 7f9100b54640 1 --2- 172.21.15.98:0/1676136087 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f90dc067f20 0x7f90dc06a3e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:01.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.775+0000 7f9100b54640 1 --2- 172.21.15.98:0/1676136087 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f90fc15c290 0x7f90fc1581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:01.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.775+0000 7f9100b54640 1 --2- 172.21.15.98:0/1676136087 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90fc0cf700 0x7f90fc0adb70 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:01.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.775+0000 7f9100b54640 1 --2- 172.21.15.98:0/1676136087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f90fc06d4c0 0x7f90fc159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:01.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.775+0000 7f9100b54640 1 -- 172.21.15.98:0/1676136087 >> 172.21.15.98:0/1676136087 conn(0x7f90fc06c090 msgr2=0x7f90fc0cfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:01.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.775+0000 7f9100b54640 1 -- 172.21.15.98:0/1676136087 shutdown_connections 2024-09-19T08:46:01.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.775+0000 7f9100b54640 1 -- 172.21.15.98:0/1676136087 wait complete. 2024-09-19T08:46:01.797 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T08:46:01.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.927+0000 7f96cd3de640 1 Processor -- start 2024-09-19T08:46:01.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.927+0000 7f96cd3de640 1 -- start start 2024-09-19T08:46:01.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.931+0000 7f96cd3de640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f96c8075380 0x7f96c8073370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:01.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.931+0000 7f96cd3de640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96c80738e0 0x7f96c80b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:01.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.931+0000 7f96cd3de640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96c8075a40 con 0x7f96c815c510 2024-09-19T08:46:01.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.931+0000 7f96cd3de640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f96c80742b0 con 0x7f96c8075380 2024-09-19T08:46:01.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.931+0000 7f96cd3de640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f96c8073fd0 con 0x7f96c80738e0 2024-09-19T08:46:01.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.931+0000 7f96bffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f96c8075380 0x7f96c8073370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:01.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.931+0000 7f96bffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f96c8075380 0x7f96c8073370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46782/0 (socket says 172.21.15.98:46782) 2024-09-19T08:46:01.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.931+0000 7f96c6ffd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f96c815c510 0x7f96c8074c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42418/0 (socket says 172.21.15.98:42418) 2024-09-19T08:46:01.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.931+0000 7f96bffff640 1 -- 172.21.15.98:0/3659204940 learned_addr learned my addr 172.21.15.98:0/3659204940 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:01.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.931+0000 7f96c77fe640 1 --2- 172.21.15.98:0/3659204940 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96c80738e0 0x7f96c80b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:01.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.931+0000 7f96bffff640 1 -- 172.21.15.98:0/3659204940 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96c80738e0 msgr2=0x7f96c80b1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:01.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.931+0000 7f96bffff640 1 --2- 172.21.15.98:0/3659204940 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96c80738e0 0x7f96c80b1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:01.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.931+0000 7f96bffff640 1 -- 172.21.15.98:0/3659204940 >> v1:172.21.15.98:6789/0 conn(0x7f96c815c510 legacy=0x7f96c8074c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:01.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.931+0000 7f96c67fc640 1 -- 172.21.15.98:0/3659204940 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3193308874 0 0) 0x7f96c8075a40 con 0x7f96c815c510 2024-09-19T08:46:01.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.931+0000 7f96bffff640 1 -- 172.21.15.98:0/3659204940 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c80b6fa0 con 0x7f96c8075380 2024-09-19T08:46:01.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.931+0000 7f96c77fe640 1 --2- 172.21.15.98:0/3659204940 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96c80738e0 0x7f96c80b1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:01.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.931+0000 7f96bffff640 1 --2- 172.21.15.98:0/3659204940 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96c8075380 0x7f96c8073370 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f96b0009870 tx=0x7f96b001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed27d8eff18c931 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:01.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.931+0000 7f96c67fc640 1 -- 172.21.15.98:0/3659204940 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b00ab640 con 0x7f96c8075380 2024-09-19T08:46:01.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.931+0000 7f96c67fc640 1 -- 172.21.15.98:0/3659204940 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96b00b0070 con 0x7f96c8075380 2024-09-19T08:46:01.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.931+0000 7f96c67fc640 1 -- 172.21.15.98:0/3659204940 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b00aba00 con 0x7f96c8075380 2024-09-19T08:46:01.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.931+0000 7f96cd3de640 1 -- 172.21.15.98:0/3659204940 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96c8075380 msgr2=0x7f96c8073370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:01.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.931+0000 7f96cd3de640 1 --2- 172.21.15.98:0/3659204940 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96c8075380 0x7f96c8073370 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f96b0009870 tx=0x7f96b001c530 comp rx=0 tx=0).stop 2024-09-19T08:46:01.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.931+0000 7f96cd3de640 1 -- 172.21.15.98:0/3659204940 shutdown_connections 2024-09-19T08:46:01.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.931+0000 7f96cd3de640 1 --2- 172.21.15.98:0/3659204940 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96c80738e0 0x7f96c80b1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:01.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.931+0000 7f96cd3de640 1 --2- 172.21.15.98:0/3659204940 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96c8075380 0x7f96c8073370 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:01.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.931+0000 7f96cd3de640 1 -- 172.21.15.98:0/3659204940 >> 172.21.15.98:0/3659204940 conn(0x7f96c80b6130 msgr2=0x7f96c80b67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:01.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.931+0000 7f96cd3de640 1 -- 172.21.15.98:0/3659204940 shutdown_connections 2024-09-19T08:46:01.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.931+0000 7f96cd3de640 1 -- 172.21.15.98:0/3659204940 wait complete. 2024-09-19T08:46:01.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.935+0000 7f96cd3de640 1 Processor -- start 2024-09-19T08:46:01.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.935+0000 7f96cd3de640 1 -- start start 2024-09-19T08:46:01.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.935+0000 7f96cd3de640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96c80738e0 0x7f96c814d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:01.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.935+0000 7f96cd3de640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96c8075380 0x7f96c8158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:01.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.935+0000 7f96cd3de640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96c815c510 0x7f96c8153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:01.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.935+0000 7f96cd3de640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f96c8169270 con 0x7f96c80738e0 2024-09-19T08:46:01.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.935+0000 7f96cd3de640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f96c81690f0 con 0x7f96c815c510 2024-09-19T08:46:01.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.935+0000 7f96cd3de640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f96c81693f0 con 0x7f96c8075380 2024-09-19T08:46:01.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.935+0000 7f96c6ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96c80738e0 0x7f96c814d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:01.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.935+0000 7f96c6ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96c80738e0 0x7f96c814d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46788/0 (socket says 172.21.15.98:46788) 2024-09-19T08:46:01.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.935+0000 7f96c6ffd640 1 -- 172.21.15.98:0/3434026853 learned_addr learned my addr 172.21.15.98:0/3434026853 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:01.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.935+0000 7f96c6ffd640 1 -- 172.21.15.98:0/3434026853 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96c8075380 msgr2=0x7f96c8158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:01.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.935+0000 7f96c6ffd640 1 --2- 172.21.15.98:0/3434026853 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96c8075380 0x7f96c8158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:01.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.935+0000 7f96c6ffd640 1 -- 172.21.15.98:0/3434026853 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96c815c510 msgr2=0x7f96c8153030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:01.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.935+0000 7f96c6ffd640 1 --2- 172.21.15.98:0/3434026853 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96c815c510 0x7f96c8153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:01.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.935+0000 7f96c6ffd640 1 -- 172.21.15.98:0/3434026853 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c80b3e80 con 0x7f96c80738e0 2024-09-19T08:46:01.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.935+0000 7f96c6ffd640 1 --2- 172.21.15.98:0/3434026853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96c80738e0 0x7f96c814d9b0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f96b800c480 tx=0x7f96b800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:01.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.935+0000 7f96bf7fe640 1 -- 172.21.15.98:0/3434026853 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b8018070 con 0x7f96c80738e0 2024-09-19T08:46:01.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.935+0000 7f96bf7fe640 1 -- 172.21.15.98:0/3434026853 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96b8007500 con 0x7f96c80738e0 2024-09-19T08:46:01.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.935+0000 7f96cd3de640 1 -- 172.21.15.98:0/3434026853 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96c814e070 con 0x7f96c80738e0 2024-09-19T08:46:01.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.935+0000 7f96bf7fe640 1 -- 172.21.15.98:0/3434026853 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b8004a30 con 0x7f96c80738e0 2024-09-19T08:46:01.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.935+0000 7f96cd3de640 1 -- 172.21.15.98:0/3434026853 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96c8156100 con 0x7f96c80738e0 2024-09-19T08:46:01.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.935+0000 7f96cd3de640 1 -- 172.21.15.98:0/3434026853 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96c80b1c30 con 0x7f96c80738e0 2024-09-19T08:46:01.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.939+0000 7f96bf7fe640 1 -- 172.21.15.98:0/3434026853 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f96b801b070 con 0x7f96c80738e0 2024-09-19T08:46:01.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.939+0000 7f96bf7fe640 1 --2- 172.21.15.98:0/3434026853 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f96a8067ef0 0x7f96a806a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:01.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.939+0000 7f96bf7fe640 1 -- 172.21.15.98:0/3434026853 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5255+0+0 (secure 0 0 0) 0x7f96b8085960 con 0x7f96c80738e0 2024-09-19T08:46:01.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.939+0000 7f96bffff640 1 --2- 172.21.15.98:0/3434026853 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f96a8067ef0 0x7f96a806a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:01.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.939+0000 7f96bffff640 1 --2- 172.21.15.98:0/3434026853 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f96a8067ef0 0x7f96a806a3b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f96b00af000 tx=0x7f96b0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:01.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:01.943+0000 7f96bf7fe640 1 -- 172.21.15.98:0/3434026853 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f96b8055d70 con 0x7f96c80738e0 2024-09-19T08:46:02.101 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.103+0000 7f96cd3de640 1 -- 172.21.15.98:0/3434026853 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f96c8073430 con 0x7f96c80738e0 2024-09-19T08:46:02.774 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.775+0000 7f96bf7fe640 1 -- 172.21.15.98:0/3434026853 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v144) ==== 170+0+0 (secure 0 0 0) 0x7f96b8055670 con 0x7f96c80738e0 2024-09-19T08:46:02.775 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' created 2024-09-19T08:46:02.777 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.779+0000 7f96cd3de640 1 -- 172.21.15.98:0/3434026853 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f96a8067ef0 msgr2=0x7f96a806a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:02.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.779+0000 7f96cd3de640 1 --2- 172.21.15.98:0/3434026853 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f96a8067ef0 0x7f96a806a3b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f96b00af000 tx=0x7f96b0008040 comp rx=0 tx=0).stop 2024-09-19T08:46:02.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.779+0000 7f96cd3de640 1 -- 172.21.15.98:0/3434026853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96c80738e0 msgr2=0x7f96c814d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:02.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.779+0000 7f96cd3de640 1 --2- 172.21.15.98:0/3434026853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96c80738e0 0x7f96c814d9b0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f96b800c480 tx=0x7f96b800c940 comp rx=0 tx=0).stop 2024-09-19T08:46:02.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.779+0000 7f96cd3de640 1 -- 172.21.15.98:0/3434026853 shutdown_connections 2024-09-19T08:46:02.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.779+0000 7f96cd3de640 1 --2- 172.21.15.98:0/3434026853 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f96a8067ef0 0x7f96a806a3b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:02.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.779+0000 7f96cd3de640 1 --2- 172.21.15.98:0/3434026853 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96c815c510 0x7f96c8153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:02.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.779+0000 7f96cd3de640 1 --2- 172.21.15.98:0/3434026853 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f96c8075380 0x7f96c8158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:02.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.779+0000 7f96cd3de640 1 --2- 172.21.15.98:0/3434026853 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f96c80738e0 0x7f96c814d9b0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:02.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.779+0000 7f96cd3de640 1 -- 172.21.15.98:0/3434026853 >> 172.21.15.98:0/3434026853 conn(0x7f96c80b6130 msgr2=0x7f96c806e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:02.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.779+0000 7f96cd3de640 1 -- 172.21.15.98:0/3434026853 shutdown_connections 2024-09-19T08:46:02.778 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.779+0000 7f96cd3de640 1 -- 172.21.15.98:0/3434026853 wait complete. 2024-09-19T08:46:02.809 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T08:46:02.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.927+0000 7f7c9b347640 1 Processor -- start 2024-09-19T08:46:02.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.927+0000 7f7c9b347640 1 -- start start 2024-09-19T08:46:02.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.927+0000 7f7c9b347640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c94145b70 0x7f7c9406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:02.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.927+0000 7f7c9b347640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7c94071ac0 0x7f7c940b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:02.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.927+0000 7f7c9b347640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c9406c5f0 con 0x7f7c9406c1d0 2024-09-19T08:46:02.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.927+0000 7f7c9b347640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f7c94071f20 con 0x7f7c94071ac0 2024-09-19T08:46:02.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.927+0000 7f7c9b347640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7c94073530 con 0x7f7c94145b70 2024-09-19T08:46:02.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.927+0000 7f7c988e2640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f7c9406c1d0 0x7f7c940713b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42428/0 (socket says 172.21.15.98:42428) 2024-09-19T08:46:02.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.927+0000 7f7c998e4640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7c94071ac0 0x7f7c940b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:02.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.927+0000 7f7c988e2640 1 -- 172.21.15.98:0/3965499875 learned_addr learned my addr 172.21.15.98:0/3965499875 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:02.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.927+0000 7f7c8bfff640 1 -- 172.21.15.98:0/3965499875 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4233887624 0 0) 0x7f7c9406c5f0 con 0x7f7c9406c1d0 2024-09-19T08:46:02.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.927+0000 7f7c8bfff640 1 -- 172.21.15.98:0/3965499875 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c80003530 con 0x7f7c9406c1d0 2024-09-19T08:46:02.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.927+0000 7f7c990e3640 1 --2- 172.21.15.98:0/3965499875 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c94145b70 0x7f7c9406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:02.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.927+0000 7f7c998e4640 1 -- 172.21.15.98:0/3965499875 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c94145b70 msgr2=0x7f7c9406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:02.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.927+0000 7f7c998e4640 1 --2- 172.21.15.98:0/3965499875 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c94145b70 0x7f7c9406bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:02.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.927+0000 7f7c998e4640 1 -- 172.21.15.98:0/3965499875 >> v1:172.21.15.98:6789/0 conn(0x7f7c9406c1d0 legacy=0x7f7c940713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:02.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.927+0000 7f7c998e4640 1 -- 172.21.15.98:0/3965499875 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c94073810 con 0x7f7c94071ac0 2024-09-19T08:46:02.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.927+0000 7f7c990e3640 1 --2- 172.21.15.98:0/3965499875 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c94145b70 0x7f7c9406bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:02.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.931+0000 7f7c998e4640 1 --2- 172.21.15.98:0/3965499875 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c94071ac0 0x7f7c940b3410 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c009d90 tx=0x7f7c8c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2245907b010b5b02 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:02.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.931+0000 7f7c8bfff640 1 -- 172.21.15.98:0/3965499875 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c0b2070 con 0x7f7c94071ac0 2024-09-19T08:46:02.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.931+0000 7f7c8bfff640 1 -- 172.21.15.98:0/3965499875 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c8c0a57a0 con 0x7f7c94071ac0 2024-09-19T08:46:02.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.931+0000 7f7c8bfff640 1 -- 172.21.15.98:0/3965499875 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c0a5b50 con 0x7f7c94071ac0 2024-09-19T08:46:02.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.931+0000 7f7c9b347640 1 -- 172.21.15.98:0/3965499875 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c94071ac0 msgr2=0x7f7c940b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:02.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.931+0000 7f7c9b347640 1 --2- 172.21.15.98:0/3965499875 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c94071ac0 0x7f7c940b3410 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c009d90 tx=0x7f7c8c01c8e0 comp rx=0 tx=0).stop 2024-09-19T08:46:02.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.931+0000 7f7c9b347640 1 -- 172.21.15.98:0/3965499875 shutdown_connections 2024-09-19T08:46:02.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.931+0000 7f7c9b347640 1 --2- 172.21.15.98:0/3965499875 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c94071ac0 0x7f7c940b3410 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:02.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.931+0000 7f7c9b347640 1 --2- 172.21.15.98:0/3965499875 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c94145b70 0x7f7c9406bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:02.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.931+0000 7f7c9b347640 1 -- 172.21.15.98:0/3965499875 >> 172.21.15.98:0/3965499875 conn(0x7f7c940b2010 msgr2=0x7f7c940b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:02.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.931+0000 7f7c9b347640 1 -- 172.21.15.98:0/3965499875 shutdown_connections 2024-09-19T08:46:02.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.931+0000 7f7c9b347640 1 -- 172.21.15.98:0/3965499875 wait complete. 2024-09-19T08:46:02.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.931+0000 7f7c9b347640 1 Processor -- start 2024-09-19T08:46:02.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.931+0000 7f7c9b347640 1 -- start start 2024-09-19T08:46:02.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.931+0000 7f7c9b347640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c9406c1d0 0x7f7c9415b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:02.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.931+0000 7f7c9b347640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c94071ac0 0x7f7c9415a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:02.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.931+0000 7f7c9b347640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c94145b70 0x7f7c9415a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:02.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.931+0000 7f7c9b347640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7c940b42e0 con 0x7f7c94145b70 2024-09-19T08:46:02.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.931+0000 7f7c9b347640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7c940b4160 con 0x7f7c9406c1d0 2024-09-19T08:46:02.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.931+0000 7f7c9b347640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7c940b4460 con 0x7f7c94071ac0 2024-09-19T08:46:02.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.931+0000 7f7c998e4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c94145b70 0x7f7c9415a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:02.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.931+0000 7f7c998e4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c94145b70 0x7f7c9415a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46802/0 (socket says 172.21.15.98:46802) 2024-09-19T08:46:02.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.931+0000 7f7c998e4640 1 -- 172.21.15.98:0/3107243529 learned_addr learned my addr 172.21.15.98:0/3107243529 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:02.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.931+0000 7f7c990e3640 1 --2- 172.21.15.98:0/3107243529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c9406c1d0 0x7f7c9415b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:02.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.931+0000 7f7c988e2640 1 --2- 172.21.15.98:0/3107243529 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c94071ac0 0x7f7c9415a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:02.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.931+0000 7f7c998e4640 1 -- 172.21.15.98:0/3107243529 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c94071ac0 msgr2=0x7f7c9415a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:02.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.931+0000 7f7c998e4640 1 --2- 172.21.15.98:0/3107243529 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c94071ac0 0x7f7c9415a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:02.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.931+0000 7f7c998e4640 1 -- 172.21.15.98:0/3107243529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c9406c1d0 msgr2=0x7f7c9415b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:02.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.931+0000 7f7c998e4640 1 --2- 172.21.15.98:0/3107243529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c9406c1d0 0x7f7c9415b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:02.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.931+0000 7f7c998e4640 1 -- 172.21.15.98:0/3107243529 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9413e5d0 con 0x7f7c94145b70 2024-09-19T08:46:02.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.931+0000 7f7c990e3640 1 --2- 172.21.15.98:0/3107243529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c9406c1d0 0x7f7c9415b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:02.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.931+0000 7f7c998e4640 1 --2- 172.21.15.98:0/3107243529 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c94145b70 0x7f7c9415a7b0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c01af90 tx=0x7f7c8c019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:02.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.931+0000 7f7c89ffb640 1 -- 172.21.15.98:0/3107243529 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c0b2050 con 0x7f7c94145b70 2024-09-19T08:46:02.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.935+0000 7f7c89ffb640 1 -- 172.21.15.98:0/3107243529 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c8c0be070 con 0x7f7c94145b70 2024-09-19T08:46:02.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.935+0000 7f7c9b347640 1 -- 172.21.15.98:0/3107243529 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c9413e8b0 con 0x7f7c94145b70 2024-09-19T08:46:02.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.935+0000 7f7c89ffb640 1 -- 172.21.15.98:0/3107243529 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c0ad850 con 0x7f7c94145b70 2024-09-19T08:46:02.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.935+0000 7f7c9b347640 1 -- 172.21.15.98:0/3107243529 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c9413ee70 con 0x7f7c94145b70 2024-09-19T08:46:02.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.935+0000 7f7c9b347640 1 -- 172.21.15.98:0/3107243529 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c9413f0b0 con 0x7f7c94145b70 2024-09-19T08:46:02.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.935+0000 7f7c89ffb640 1 -- 172.21.15.98:0/3107243529 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f7c8c0ad9f0 con 0x7f7c94145b70 2024-09-19T08:46:02.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.935+0000 7f7c89ffb640 1 --2- 172.21.15.98:0/3107243529 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7c78067f20 0x7f7c7806a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:02.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.935+0000 7f7c89ffb640 1 -- 172.21.15.98:0/3107243529 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5637+0+0 (secure 0 0 0) 0x7f7c8c121e70 con 0x7f7c94145b70 2024-09-19T08:46:02.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.935+0000 7f7c990e3640 1 --2- 172.21.15.98:0/3107243529 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7c78067f20 0x7f7c7806a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:02.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.935+0000 7f7c990e3640 1 --2- 172.21.15.98:0/3107243529 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7c78067f20 0x7f7c7806a3e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7c9413f360 tx=0x7f7c90008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:02.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:02.939+0000 7f7c89ffb640 1 -- 172.21.15.98:0/3107243529 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f7c8c016560 con 0x7f7c94145b70 2024-09-19T08:46:03.119 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.119+0000 7f7c9b347640 1 -- 172.21.15.98:0/3107243529 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f7c940b3410 con 0x7f7c94145b70 2024-09-19T08:46:03.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.823+0000 7f7c89ffb640 1 -- 172.21.15.98:0/3107243529 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v80) ==== 174+0+0 (secure 0 0 0) 0x7f7c8c0f21b0 con 0x7f7c94145b70 2024-09-19T08:46:03.835 INFO:teuthology.orchestra.run.smithi098.stderr:new fs with metadata pool 22 and data pool 23 2024-09-19T08:46:03.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.843+0000 7f7c9b347640 1 -- 172.21.15.98:0/3107243529 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7c78067f20 msgr2=0x7f7c7806a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:03.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.843+0000 7f7c9b347640 1 --2- 172.21.15.98:0/3107243529 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7c78067f20 0x7f7c7806a3e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7c9413f360 tx=0x7f7c90008040 comp rx=0 tx=0).stop 2024-09-19T08:46:03.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.843+0000 7f7c9b347640 1 -- 172.21.15.98:0/3107243529 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c94145b70 msgr2=0x7f7c9415a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:03.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.843+0000 7f7c9b347640 1 --2- 172.21.15.98:0/3107243529 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c94145b70 0x7f7c9415a7b0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c01af90 tx=0x7f7c8c019680 comp rx=0 tx=0).stop 2024-09-19T08:46:03.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.843+0000 7f7c9b347640 1 -- 172.21.15.98:0/3107243529 shutdown_connections 2024-09-19T08:46:03.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.843+0000 7f7c9b347640 1 --2- 172.21.15.98:0/3107243529 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7c78067f20 0x7f7c7806a3e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:03.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.843+0000 7f7c9b347640 1 --2- 172.21.15.98:0/3107243529 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7c94145b70 0x7f7c9415a7b0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:03.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.843+0000 7f7c9b347640 1 --2- 172.21.15.98:0/3107243529 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7c94071ac0 0x7f7c9415a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:03.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.843+0000 7f7c9b347640 1 --2- 172.21.15.98:0/3107243529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7c9406c1d0 0x7f7c9415b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:03.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.843+0000 7f7c9b347640 1 -- 172.21.15.98:0/3107243529 >> 172.21.15.98:0/3107243529 conn(0x7f7c940b2010 msgr2=0x7f7c940cf790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:03.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.843+0000 7f7c9b347640 1 -- 172.21.15.98:0/3107243529 shutdown_connections 2024-09-19T08:46:03.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.843+0000 7f7c9b347640 1 -- 172.21.15.98:0/3107243529 wait complete. 2024-09-19T08:46:03.864 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:46:03.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.979+0000 7f130e0a1640 1 Processor -- start 2024-09-19T08:46:03.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.979+0000 7f130e0a1640 1 -- start start 2024-09-19T08:46:03.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.983+0000 7f130e0a1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1308158fd0 0x7f13080b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:03.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.983+0000 7f130e0a1640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f13080b4360 0x7f130806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:03.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.983+0000 7f130e0a1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f130806c1d0 0x7f130815efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:03.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.983+0000 7f130e0a1640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f130806c620 con 0x7f13080b4360 2024-09-19T08:46:03.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.983+0000 7f130e0a1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f13081593b0 con 0x7f130806c1d0 2024-09-19T08:46:03.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.983+0000 7f130e0a1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f13080ccc60 con 0x7f1308158fd0 2024-09-19T08:46:03.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.983+0000 7f1306ffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f13080b4360 0x7f130806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:03.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.983+0000 7f1306ffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f13080b4360 0x7f130806bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46804/0 (socket says 172.21.15.98:46804) 2024-09-19T08:46:03.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.983+0000 7f1306ffd640 1 -- 172.21.15.98:0/3254484273 learned_addr learned my addr 172.21.15.98:0/3254484273 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:03.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.983+0000 7f1306ffd640 1 -- 172.21.15.98:0/3254484273 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1308158fd0 msgr2=0x7f13080b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:03.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.983+0000 7f1306ffd640 1 --2- 172.21.15.98:0/3254484273 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1308158fd0 0x7f13080b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:03.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.983+0000 7f1306ffd640 1 -- 172.21.15.98:0/3254484273 >> v2:172.21.15.139:3300/0 conn(0x7f130806c1d0 msgr2=0x7f130815efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:03.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.983+0000 7f1306ffd640 1 --2- 172.21.15.98:0/3254484273 >> v2:172.21.15.139:3300/0 conn(0x7f130806c1d0 0x7f130815efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:03.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.983+0000 7f1306ffd640 1 -- 172.21.15.98:0/3254484273 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13080ccf40 con 0x7f13080b4360 2024-09-19T08:46:03.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.983+0000 7f1306ffd640 1 --2- 172.21.15.98:0/3254484273 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13080b4360 0x7f130806bc90 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f13080ce4d0 tx=0x7f12f001c1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c4eb3523483ce82 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:03.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.983+0000 7f13067fc640 1 -- 172.21.15.98:0/3254484273 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f00b0070 con 0x7f13080b4360 2024-09-19T08:46:03.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.983+0000 7f13067fc640 1 -- 172.21.15.98:0/3254484273 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12f00ab600 con 0x7f13080b4360 2024-09-19T08:46:03.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.983+0000 7f13067fc640 1 -- 172.21.15.98:0/3254484273 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f00ab9d0 con 0x7f13080b4360 2024-09-19T08:46:03.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.983+0000 7f130e0a1640 1 -- 172.21.15.98:0/3254484273 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13080b4360 msgr2=0x7f130806bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:03.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.983+0000 7f130e0a1640 1 --2- 172.21.15.98:0/3254484273 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13080b4360 0x7f130806bc90 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f13080ce4d0 tx=0x7f12f001c1a0 comp rx=0 tx=0).stop 2024-09-19T08:46:03.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.983+0000 7f130e0a1640 1 -- 172.21.15.98:0/3254484273 shutdown_connections 2024-09-19T08:46:03.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.983+0000 7f130e0a1640 1 --2- 172.21.15.98:0/3254484273 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13080b4360 0x7f130806bc90 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:03.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.983+0000 7f130e0a1640 1 --2- 172.21.15.98:0/3254484273 >> v2:172.21.15.139:3300/0 conn(0x7f130806c1d0 0x7f130815efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:03.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.983+0000 7f130e0a1640 1 --2- 172.21.15.98:0/3254484273 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1308158fd0 0x7f13080b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:03.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.983+0000 7f130e0a1640 1 -- 172.21.15.98:0/3254484273 >> 172.21.15.98:0/3254484273 conn(0x7f13080b2010 msgr2=0x7f13080b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:03.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.987+0000 7f130e0a1640 1 -- 172.21.15.98:0/3254484273 shutdown_connections 2024-09-19T08:46:03.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.987+0000 7f130e0a1640 1 -- 172.21.15.98:0/3254484273 wait complete. 2024-09-19T08:46:03.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.987+0000 7f130e0a1640 1 Processor -- start 2024-09-19T08:46:03.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.987+0000 7f130e0a1640 1 -- start start 2024-09-19T08:46:03.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.987+0000 7f130e0a1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f130806c1d0 0x7f13080d0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:03.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.987+0000 7f130e0a1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13080b4360 0x7f1308073470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:03.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.987+0000 7f130e0a1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1308158fd0 0x7f13080739b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:03.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.987+0000 7f130e0a1640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f130815c920 con 0x7f1308158fd0 2024-09-19T08:46:03.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.987+0000 7f130e0a1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f130815c7a0 con 0x7f130806c1d0 2024-09-19T08:46:03.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.987+0000 7f130e0a1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f13080ce840 con 0x7f13080b4360 2024-09-19T08:46:03.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.987+0000 7f1307fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1308158fd0 0x7f13080739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:03.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.987+0000 7f1307fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1308158fd0 0x7f13080739b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46818/0 (socket says 172.21.15.98:46818) 2024-09-19T08:46:03.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.987+0000 7f1307fff640 1 -- 172.21.15.98:0/788502173 learned_addr learned my addr 172.21.15.98:0/788502173 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:03.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.987+0000 7f13077fe640 1 --2- 172.21.15.98:0/788502173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f130806c1d0 0x7f13080d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:03.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.987+0000 7f1306ffd640 1 --2- 172.21.15.98:0/788502173 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13080b4360 0x7f1308073470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:03.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.987+0000 7f1307fff640 1 -- 172.21.15.98:0/788502173 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13080b4360 msgr2=0x7f1308073470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:03.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.987+0000 7f1307fff640 1 --2- 172.21.15.98:0/788502173 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13080b4360 0x7f1308073470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:03.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.987+0000 7f1307fff640 1 -- 172.21.15.98:0/788502173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f130806c1d0 msgr2=0x7f13080d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:03.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.987+0000 7f1307fff640 1 --2- 172.21.15.98:0/788502173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f130806c1d0 0x7f13080d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:03.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.987+0000 7f1307fff640 1 -- 172.21.15.98:0/788502173 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13080ae730 con 0x7f1308158fd0 2024-09-19T08:46:03.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.987+0000 7f1307fff640 1 --2- 172.21.15.98:0/788502173 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1308158fd0 0x7f13080739b0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f130000b970 tx=0x7f130000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:03.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.987+0000 7f12fffff640 1 -- 172.21.15.98:0/788502173 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1300019070 con 0x7f1308158fd0 2024-09-19T08:46:03.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.987+0000 7f12fffff640 1 -- 172.21.15.98:0/788502173 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f130000cb60 con 0x7f1308158fd0 2024-09-19T08:46:03.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.987+0000 7f130e0a1640 1 -- 172.21.15.98:0/788502173 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13080aea10 con 0x7f1308158fd0 2024-09-19T08:46:03.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.987+0000 7f12fffff640 1 -- 172.21.15.98:0/788502173 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1300014440 con 0x7f1308158fd0 2024-09-19T08:46:03.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.987+0000 7f130e0a1640 1 -- 172.21.15.98:0/788502173 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13080aefd0 con 0x7f1308158fd0 2024-09-19T08:46:03.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.987+0000 7f130e0a1640 1 -- 172.21.15.98:0/788502173 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13080af1d0 con 0x7f1308158fd0 2024-09-19T08:46:03.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.991+0000 7f12fffff640 1 -- 172.21.15.98:0/788502173 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f130001c070 con 0x7f1308158fd0 2024-09-19T08:46:03.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.991+0000 7f12fffff640 1 --2- 172.21.15.98:0/788502173 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f12e8067ef0 0x7f12e806a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:03.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.991+0000 7f12fffff640 1 -- 172.21.15.98:0/788502173 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5737+0+0 (secure 0 0 0) 0x7f1300086880 con 0x7f1308158fd0 2024-09-19T08:46:03.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.991+0000 7f13077fe640 1 --2- 172.21.15.98:0/788502173 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f12e8067ef0 0x7f12e806a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:03.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.991+0000 7f13077fe640 1 --2- 172.21.15.98:0/788502173 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f12e8067ef0 0x7f12e806a3b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f12f8006fd0 tx=0x7f12f8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:03.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:03.995+0000 7f12fffff640 1 -- 172.21.15.98:0/788502173 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f13080af1d0 con 0x7f1308158fd0 2024-09-19T08:46:04.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.155+0000 7f130e0a1640 1 -- 172.21.15.98:0/788502173 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13081657b0 con 0x7f1308158fd0 2024-09-19T08:46:04.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.159+0000 7f12fffff640 1 -- 172.21.15.98:0/788502173 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16788 (secure 0 0 0) 0x7f1300023020 con 0x7f1308158fd0 2024-09-19T08:46:04.157 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:46:04.157 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":145,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:46:03.123208+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:46:00.845215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-19T08:46:02.145134+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:46:04.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.159+0000 7f130e0a1640 1 -- 172.21.15.98:0/788502173 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f12e8067ef0 msgr2=0x7f12e806a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:04.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.159+0000 7f130e0a1640 1 --2- 172.21.15.98:0/788502173 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f12e8067ef0 0x7f12e806a3b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f12f8006fd0 tx=0x7f12f8008040 comp rx=0 tx=0).stop 2024-09-19T08:46:04.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.159+0000 7f130e0a1640 1 -- 172.21.15.98:0/788502173 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1308158fd0 msgr2=0x7f13080739b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:04.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.159+0000 7f130e0a1640 1 --2- 172.21.15.98:0/788502173 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1308158fd0 0x7f13080739b0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f130000b970 tx=0x7f130000be30 comp rx=0 tx=0).stop 2024-09-19T08:46:04.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.159+0000 7f130e0a1640 1 -- 172.21.15.98:0/788502173 shutdown_connections 2024-09-19T08:46:04.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.159+0000 7f130e0a1640 1 --2- 172.21.15.98:0/788502173 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f12e8067ef0 0x7f12e806a3b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:04.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.159+0000 7f130e0a1640 1 --2- 172.21.15.98:0/788502173 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1308158fd0 0x7f13080739b0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:04.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.159+0000 7f130e0a1640 1 --2- 172.21.15.98:0/788502173 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13080b4360 0x7f1308073470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:04.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.159+0000 7f130e0a1640 1 --2- 172.21.15.98:0/788502173 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f130806c1d0 0x7f13080d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:04.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.159+0000 7f130e0a1640 1 -- 172.21.15.98:0/788502173 >> 172.21.15.98:0/788502173 conn(0x7f13080b2010 msgr2=0x7f1308157210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:04.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.159+0000 7f130e0a1640 1 -- 172.21.15.98:0/788502173 shutdown_connections 2024-09-19T08:46:04.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.159+0000 7f130e0a1640 1 -- 172.21.15.98:0/788502173 wait complete. 2024-09-19T08:46:04.176 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:46:04.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.287+0000 7fb2b79e7640 1 Processor -- start 2024-09-19T08:46:04.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.287+0000 7fb2b79e7640 1 -- start start 2024-09-19T08:46:04.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.287+0000 7fb2b79e7640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb2b015c550 0x7fb2b006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:04.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.287+0000 7fb2b79e7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2b006c1d0 0x7fb2b00cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:04.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.287+0000 7fb2b79e7640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2b006d310 con 0x7fb2b00cf900 2024-09-19T08:46:04.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.287+0000 7fb2b79e7640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fb2b006c610 con 0x7fb2b015c550 2024-09-19T08:46:04.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.287+0000 7fb2b79e7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb2b006d5f0 con 0x7fb2b006c1d0 2024-09-19T08:46:04.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.287+0000 7fb2b5783640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb2b015c550 0x7fb2b006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:04.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.287+0000 7fb2b5f84640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fb2b00cf900 0x7fb2b00ce1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42442/0 (socket says 172.21.15.98:42442) 2024-09-19T08:46:04.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.287+0000 7fb2b5f84640 1 -- 172.21.15.98:0/170047084 learned_addr learned my addr 172.21.15.98:0/170047084 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:04.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.287+0000 7fb2b4f82640 1 --2- 172.21.15.98:0/170047084 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2b006c1d0 0x7fb2b00cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:04.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.287+0000 7fb2b5783640 1 -- 172.21.15.98:0/170047084 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2b006c1d0 msgr2=0x7fb2b00cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:04.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.287+0000 7fb2b5783640 1 --2- 172.21.15.98:0/170047084 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2b006c1d0 0x7fb2b00cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:04.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.287+0000 7fb2b5783640 1 -- 172.21.15.98:0/170047084 >> v1:172.21.15.98:6789/0 conn(0x7fb2b00cf900 legacy=0x7fb2b00ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:04.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.287+0000 7fb2a7fff640 1 -- 172.21.15.98:0/170047084 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1130911472 0 0) 0x7fb2b006d310 con 0x7fb2b00cf900 2024-09-19T08:46:04.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.287+0000 7fb2b5783640 1 -- 172.21.15.98:0/170047084 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b006d8d0 con 0x7fb2b015c550 2024-09-19T08:46:04.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2b5783640 1 --2- 172.21.15.98:0/170047084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2b015c550 0x7fb2b006bc90 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac0097c0 tx=0x7fb2ac01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0b0d66170f1caa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:04.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2a7fff640 1 -- 172.21.15.98:0/170047084 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac0b0070 con 0x7fb2b015c550 2024-09-19T08:46:04.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2a7fff640 1 -- 172.21.15.98:0/170047084 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2ac0ab600 con 0x7fb2b015c550 2024-09-19T08:46:04.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2a7fff640 1 -- 172.21.15.98:0/170047084 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac0ab9d0 con 0x7fb2b015c550 2024-09-19T08:46:04.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2b79e7640 1 -- 172.21.15.98:0/170047084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2b015c550 msgr2=0x7fb2b006bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:04.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2b79e7640 1 --2- 172.21.15.98:0/170047084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2b015c550 0x7fb2b006bc90 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac0097c0 tx=0x7fb2ac01c4a0 comp rx=0 tx=0).stop 2024-09-19T08:46:04.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2b79e7640 1 -- 172.21.15.98:0/170047084 shutdown_connections 2024-09-19T08:46:04.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2b79e7640 1 --2- 172.21.15.98:0/170047084 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2b006c1d0 0x7fb2b00cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:04.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2b79e7640 1 --2- 172.21.15.98:0/170047084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2b015c550 0x7fb2b006bc90 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:04.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2b79e7640 1 -- 172.21.15.98:0/170047084 >> 172.21.15.98:0/170047084 conn(0x7fb2b00b2010 msgr2=0x7fb2b00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:04.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2b79e7640 1 -- 172.21.15.98:0/170047084 shutdown_connections 2024-09-19T08:46:04.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2b79e7640 1 -- 172.21.15.98:0/170047084 wait complete. 2024-09-19T08:46:04.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2b79e7640 1 Processor -- start 2024-09-19T08:46:04.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2b79e7640 1 -- start start 2024-09-19T08:46:04.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2b79e7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2b006c1d0 0x7fb2b014d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:04.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2b79e7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2b00cf900 0x7fb2b01522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:04.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2b79e7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2b015c550 0x7fb2b01594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:04.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2b79e7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b0169230 con 0x7fb2b00cf900 2024-09-19T08:46:04.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2b79e7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b015cb40 con 0x7fb2b006c1d0 2024-09-19T08:46:04.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2b79e7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb2b01693b0 con 0x7fb2b015c550 2024-09-19T08:46:04.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2b4f82640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2b00cf900 0x7fb2b01522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:04.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2b4f82640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2b00cf900 0x7fb2b01522a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46826/0 (socket says 172.21.15.98:46826) 2024-09-19T08:46:04.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2b4f82640 1 -- 172.21.15.98:0/4114834810 learned_addr learned my addr 172.21.15.98:0/4114834810 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:04.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2b5783640 1 --2- 172.21.15.98:0/4114834810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2b006c1d0 0x7fb2b014d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:04.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2b5f84640 1 --2- 172.21.15.98:0/4114834810 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2b015c550 0x7fb2b01594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:04.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2b4f82640 1 -- 172.21.15.98:0/4114834810 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2b015c550 msgr2=0x7fb2b01594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:04.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2b4f82640 1 --2- 172.21.15.98:0/4114834810 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2b015c550 0x7fb2b01594b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:04.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2b4f82640 1 -- 172.21.15.98:0/4114834810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2b006c1d0 msgr2=0x7fb2b014d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:04.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2b4f82640 1 --2- 172.21.15.98:0/4114834810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2b006c1d0 0x7fb2b014d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:04.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2b4f82640 1 -- 172.21.15.98:0/4114834810 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b014e7b0 con 0x7fb2b00cf900 2024-09-19T08:46:04.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2b5783640 1 --2- 172.21.15.98:0/4114834810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2b006c1d0 0x7fb2b014d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:04.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2b4f82640 1 --2- 172.21.15.98:0/4114834810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2b00cf900 0x7fb2b01522a0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000d800 tx=0x7fb2a000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:04.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2a5ffb640 1 -- 172.21.15.98:0/4114834810 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a000b6c0 con 0x7fb2b00cf900 2024-09-19T08:46:04.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2a5ffb640 1 -- 172.21.15.98:0/4114834810 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2a0019070 con 0x7fb2b00cf900 2024-09-19T08:46:04.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2b79e7640 1 -- 172.21.15.98:0/4114834810 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2b014ea90 con 0x7fb2b00cf900 2024-09-19T08:46:04.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2a5ffb640 1 -- 172.21.15.98:0/4114834810 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a000bae0 con 0x7fb2b00cf900 2024-09-19T08:46:04.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.291+0000 7fb2b79e7640 1 -- 172.21.15.98:0/4114834810 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2b014f050 con 0x7fb2b00cf900 2024-09-19T08:46:04.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.295+0000 7fb2b79e7640 1 -- 172.21.15.98:0/4114834810 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2b014f430 con 0x7fb2b00cf900 2024-09-19T08:46:04.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.295+0000 7fb2a5ffb640 1 -- 172.21.15.98:0/4114834810 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fb2a000bc80 con 0x7fb2b00cf900 2024-09-19T08:46:04.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.295+0000 7fb2a5ffb640 1 --2- 172.21.15.98:0/4114834810 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb294067ef0 0x7fb29406a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:04.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.295+0000 7fb2a5ffb640 1 -- 172.21.15.98:0/4114834810 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5737+0+0 (secure 0 0 0) 0x7fb2a0086ce0 con 0x7fb2b00cf900 2024-09-19T08:46:04.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.295+0000 7fb2b5783640 1 --2- 172.21.15.98:0/4114834810 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb294067ef0 0x7fb29406a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:04.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.295+0000 7fb2b5783640 1 --2- 172.21.15.98:0/4114834810 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb294067ef0 0x7fb29406a3b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac0ad000 tx=0x7fb2ac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:04.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.299+0000 7fb2a5ffb640 1 -- 172.21.15.98:0/4114834810 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb2b014f430 con 0x7fb2b00cf900 2024-09-19T08:46:04.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.447+0000 7fb2b79e7640 1 -- 172.21.15.98:0/4114834810 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb2b015b9f0 con 0x7fb2b00cf900 2024-09-19T08:46:04.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.451+0000 7fb2a5ffb640 1 -- 172.21.15.98:0/4114834810 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16788 (secure 0 0 0) 0x7fb2a0056f50 con 0x7fb2b00cf900 2024-09-19T08:46:04.448 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:46:04.449 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":145,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:46:03.123208+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:46:00.845215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-19T08:46:02.145134+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:46:04.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.451+0000 7fb2b79e7640 1 -- 172.21.15.98:0/4114834810 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb294067ef0 msgr2=0x7fb29406a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:04.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.451+0000 7fb2b79e7640 1 --2- 172.21.15.98:0/4114834810 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb294067ef0 0x7fb29406a3b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac0ad000 tx=0x7fb2ac008040 comp rx=0 tx=0).stop 2024-09-19T08:46:04.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.451+0000 7fb2b79e7640 1 -- 172.21.15.98:0/4114834810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2b00cf900 msgr2=0x7fb2b01522a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:04.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.451+0000 7fb2b79e7640 1 --2- 172.21.15.98:0/4114834810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2b00cf900 0x7fb2b01522a0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000d800 tx=0x7fb2a000dcc0 comp rx=0 tx=0).stop 2024-09-19T08:46:04.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.451+0000 7fb2b79e7640 1 -- 172.21.15.98:0/4114834810 shutdown_connections 2024-09-19T08:46:04.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.451+0000 7fb2b79e7640 1 --2- 172.21.15.98:0/4114834810 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb294067ef0 0x7fb29406a3b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:04.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.451+0000 7fb2b79e7640 1 --2- 172.21.15.98:0/4114834810 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb2b015c550 0x7fb2b01594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:04.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.451+0000 7fb2b79e7640 1 --2- 172.21.15.98:0/4114834810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb2b00cf900 0x7fb2b01522a0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:04.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.451+0000 7fb2b79e7640 1 --2- 172.21.15.98:0/4114834810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb2b006c1d0 0x7fb2b014d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:04.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.451+0000 7fb2b79e7640 1 -- 172.21.15.98:0/4114834810 >> 172.21.15.98:0/4114834810 conn(0x7fb2b00b2010 msgr2=0x7fb2b00b0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:04.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.451+0000 7fb2b79e7640 1 -- 172.21.15.98:0/4114834810 shutdown_connections 2024-09-19T08:46:04.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.451+0000 7fb2b79e7640 1 -- 172.21.15.98:0/4114834810 wait complete. 2024-09-19T08:46:04.466 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-19T08:46:04.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.579+0000 7f4a7ccc3640 1 Processor -- start 2024-09-19T08:46:04.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.579+0000 7f4a7ccc3640 1 -- start start 2024-09-19T08:46:04.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.583+0000 7f4a7ccc3640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4a78145b70 0x7f4a7806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:04.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.583+0000 7f4a7ccc3640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f4a78071ac0 0x7f4a780b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:04.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.583+0000 7f4a7ccc3640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a78073530 con 0x7f4a7806c1d0 2024-09-19T08:46:04.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.583+0000 7f4a7ccc3640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f4a78071f10 con 0x7f4a78145b70 2024-09-19T08:46:04.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.583+0000 7f4a7ccc3640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f4a7806c620 con 0x7f4a78071ac0 2024-09-19T08:46:04.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.583+0000 7f4a7659c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4a78145b70 0x7f4a7806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:04.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.583+0000 7f4a75d9b640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f4a7806c1d0 0x7f4a780713b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42452/0 (socket says 172.21.15.98:42452) 2024-09-19T08:46:04.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.583+0000 7f4a75d9b640 1 -- 172.21.15.98:0/2705289813 learned_addr learned my addr 172.21.15.98:0/2705289813 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:04.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.583+0000 7f4a76d9d640 1 --2- 172.21.15.98:0/2705289813 >> v2:172.21.15.139:3300/0 conn(0x7f4a78071ac0 0x7f4a780b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:04.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.583+0000 7f4a7659c640 1 -- 172.21.15.98:0/2705289813 >> v2:172.21.15.139:3300/0 conn(0x7f4a78071ac0 msgr2=0x7f4a780b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:04.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.583+0000 7f4a7659c640 1 --2- 172.21.15.98:0/2705289813 >> v2:172.21.15.139:3300/0 conn(0x7f4a78071ac0 0x7f4a780b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:04.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.583+0000 7f4a7659c640 1 -- 172.21.15.98:0/2705289813 >> v1:172.21.15.98:6789/0 conn(0x7f4a7806c1d0 legacy=0x7f4a780713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:04.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.583+0000 7f4a7659c640 1 -- 172.21.15.98:0/2705289813 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a78073810 con 0x7f4a78145b70 2024-09-19T08:46:04.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.583+0000 7f4a7659c640 1 --2- 172.21.15.98:0/2705289813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a78145b70 0x7f4a7806bc90 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f4a78074450 tx=0x7f4a6801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=576f0677166823ae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:04.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.583+0000 7f4a7559a640 1 -- 172.21.15.98:0/2705289813 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a680b0070 con 0x7f4a78145b70 2024-09-19T08:46:04.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.583+0000 7f4a7559a640 1 -- 172.21.15.98:0/2705289813 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a680ab600 con 0x7f4a78145b70 2024-09-19T08:46:04.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.583+0000 7f4a7559a640 1 -- 172.21.15.98:0/2705289813 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a680ab9d0 con 0x7f4a78145b70 2024-09-19T08:46:04.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.587+0000 7f4a7ccc3640 1 -- 172.21.15.98:0/2705289813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a78145b70 msgr2=0x7f4a7806bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:04.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.587+0000 7f4a7ccc3640 1 --2- 172.21.15.98:0/2705289813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a78145b70 0x7f4a7806bc90 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f4a78074450 tx=0x7f4a6801c3f0 comp rx=0 tx=0).stop 2024-09-19T08:46:04.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.587+0000 7f4a7ccc3640 1 -- 172.21.15.98:0/2705289813 shutdown_connections 2024-09-19T08:46:04.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.587+0000 7f4a7ccc3640 1 --2- 172.21.15.98:0/2705289813 >> v2:172.21.15.139:3300/0 conn(0x7f4a78071ac0 0x7f4a780b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:04.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.587+0000 7f4a7ccc3640 1 --2- 172.21.15.98:0/2705289813 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a78145b70 0x7f4a7806bc90 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:04.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.587+0000 7f4a7ccc3640 1 -- 172.21.15.98:0/2705289813 >> 172.21.15.98:0/2705289813 conn(0x7f4a780b2010 msgr2=0x7f4a780b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:04.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.587+0000 7f4a7ccc3640 1 -- 172.21.15.98:0/2705289813 shutdown_connections 2024-09-19T08:46:04.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.587+0000 7f4a7ccc3640 1 -- 172.21.15.98:0/2705289813 wait complete. 2024-09-19T08:46:04.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.587+0000 7f4a7ccc3640 1 Processor -- start 2024-09-19T08:46:04.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.587+0000 7f4a7ccc3640 1 -- start start 2024-09-19T08:46:04.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.587+0000 7f4a7ccc3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a7806c1d0 0x7f4a781667e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:04.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.587+0000 7f4a7ccc3640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a78071ac0 0x7f4a7815fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:04.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.587+0000 7f4a7ccc3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a78145b70 0x7f4a7813e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:04.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.587+0000 7f4a7ccc3640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4a780b42e0 con 0x7f4a78145b70 2024-09-19T08:46:04.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.587+0000 7f4a7ccc3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4a780b4160 con 0x7f4a7806c1d0 2024-09-19T08:46:04.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.587+0000 7f4a7ccc3640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4a780b4460 con 0x7f4a78071ac0 2024-09-19T08:46:04.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.587+0000 7f4a76d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a78145b70 0x7f4a7813e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:04.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.587+0000 7f4a76d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a78145b70 0x7f4a7813e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46848/0 (socket says 172.21.15.98:46848) 2024-09-19T08:46:04.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.587+0000 7f4a76d9d640 1 -- 172.21.15.98:0/3207414863 learned_addr learned my addr 172.21.15.98:0/3207414863 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:04.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.587+0000 7f4a7659c640 1 --2- 172.21.15.98:0/3207414863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a7806c1d0 0x7f4a781667e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:04.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.587+0000 7f4a75d9b640 1 --2- 172.21.15.98:0/3207414863 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a78071ac0 0x7f4a7815fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:04.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.587+0000 7f4a76d9d640 1 -- 172.21.15.98:0/3207414863 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a78071ac0 msgr2=0x7f4a7815fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:04.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.587+0000 7f4a76d9d640 1 --2- 172.21.15.98:0/3207414863 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a78071ac0 0x7f4a7815fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:04.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.587+0000 7f4a76d9d640 1 -- 172.21.15.98:0/3207414863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a7806c1d0 msgr2=0x7f4a781667e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:04.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.587+0000 7f4a76d9d640 1 --2- 172.21.15.98:0/3207414863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a7806c1d0 0x7f4a781667e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:04.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.587+0000 7f4a76d9d640 1 -- 172.21.15.98:0/3207414863 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a7813d140 con 0x7f4a78145b70 2024-09-19T08:46:04.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.587+0000 7f4a76d9d640 1 --2- 172.21.15.98:0/3207414863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a78145b70 0x7f4a7813e1e0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f4a70007e40 tx=0x7f4a7000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:04.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.587+0000 7f4a66ffd640 1 -- 172.21.15.98:0/3207414863 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70014620 con 0x7f4a78145b70 2024-09-19T08:46:04.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.587+0000 7f4a66ffd640 1 -- 172.21.15.98:0/3207414863 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a70019070 con 0x7f4a78145b70 2024-09-19T08:46:04.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.587+0000 7f4a7ccc3640 1 -- 172.21.15.98:0/3207414863 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a7813d420 con 0x7f4a78145b70 2024-09-19T08:46:04.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.587+0000 7f4a66ffd640 1 -- 172.21.15.98:0/3207414863 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70014a20 con 0x7f4a78145b70 2024-09-19T08:46:04.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.587+0000 7f4a7ccc3640 1 -- 172.21.15.98:0/3207414863 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a7813d9e0 con 0x7f4a78145b70 2024-09-19T08:46:04.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.587+0000 7f4a7ccc3640 1 -- 172.21.15.98:0/3207414863 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a780b3410 con 0x7f4a78145b70 2024-09-19T08:46:04.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.591+0000 7f4a66ffd640 1 -- 172.21.15.98:0/3207414863 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f4a70014bc0 con 0x7f4a78145b70 2024-09-19T08:46:04.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.591+0000 7f4a66ffd640 1 --2- 172.21.15.98:0/3207414863 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4a58067ef0 0x7f4a5806a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:04.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.591+0000 7f4a66ffd640 1 -- 172.21.15.98:0/3207414863 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5737+0+0 (secure 0 0 0) 0x7f4a700870c0 con 0x7f4a78145b70 2024-09-19T08:46:04.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.591+0000 7f4a7659c640 1 --2- 172.21.15.98:0/3207414863 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4a58067ef0 0x7f4a5806a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:04.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.591+0000 7f4a7659c640 1 --2- 172.21.15.98:0/3207414863 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4a58067ef0 0x7f4a5806a3b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4a680ad000 tx=0x7f4a68008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:04.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.595+0000 7f4a66ffd640 1 -- 172.21.15.98:0/3207414863 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f4a700572a0 con 0x7f4a78145b70 2024-09-19T08:46:04.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.787+0000 7f4a7ccc3640 1 -- 172.21.15.98:0/3207414863 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f4a78145240 con 0x7f4a78145b70 2024-09-19T08:46:04.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.819+0000 7f4a66ffd640 1 -- 172.21.15.98:0/3207414863 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v81) ==== 118+0+0 (secure 0 0 0) 0x7f4a70056ba0 con 0x7f4a78145b70 2024-09-19T08:46:04.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.823+0000 7f4a7ccc3640 1 -- 172.21.15.98:0/3207414863 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4a58067ef0 msgr2=0x7f4a5806a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:04.821 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.823+0000 7f4a7ccc3640 1 --2- 172.21.15.98:0/3207414863 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4a58067ef0 0x7f4a5806a3b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4a680ad000 tx=0x7f4a68008040 comp rx=0 tx=0).stop 2024-09-19T08:46:04.822 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.823+0000 7f4a7ccc3640 1 -- 172.21.15.98:0/3207414863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a78145b70 msgr2=0x7f4a7813e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:04.822 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.823+0000 7f4a7ccc3640 1 --2- 172.21.15.98:0/3207414863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a78145b70 0x7f4a7813e1e0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f4a70007e40 tx=0x7f4a7000e4b0 comp rx=0 tx=0).stop 2024-09-19T08:46:04.822 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.823+0000 7f4a7ccc3640 1 -- 172.21.15.98:0/3207414863 shutdown_connections 2024-09-19T08:46:04.822 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.823+0000 7f4a7ccc3640 1 --2- 172.21.15.98:0/3207414863 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4a58067ef0 0x7f4a5806a3b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:04.822 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.823+0000 7f4a7ccc3640 1 --2- 172.21.15.98:0/3207414863 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4a78145b70 0x7f4a7813e1e0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:04.822 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.823+0000 7f4a7ccc3640 1 --2- 172.21.15.98:0/3207414863 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4a78071ac0 0x7f4a7815fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:04.822 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.823+0000 7f4a7ccc3640 1 --2- 172.21.15.98:0/3207414863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a7806c1d0 0x7f4a781667e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:04.822 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.823+0000 7f4a7ccc3640 1 -- 172.21.15.98:0/3207414863 >> 172.21.15.98:0/3207414863 conn(0x7f4a780b2010 msgr2=0x7f4a781434a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:04.822 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.823+0000 7f4a7ccc3640 1 -- 172.21.15.98:0/3207414863 shutdown_connections 2024-09-19T08:46:04.822 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.823+0000 7f4a7ccc3640 1 -- 172.21.15.98:0/3207414863 wait complete. 2024-09-19T08:46:04.838 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-19T08:46:04.838 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-19T08:46:04.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.955+0000 7f003a4c7640 1 Processor -- start 2024-09-19T08:46:04.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.955+0000 7f003a4c7640 1 -- start start 2024-09-19T08:46:04.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.955+0000 7f003a4c7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f003406bc90 0x7f003406c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:04.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.955+0000 7f003a4c7640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0034072d30 con 0x7f00340d2f00 2024-09-19T08:46:04.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.955+0000 7f003a4c7640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0034073010 con 0x7f0034161970 2024-09-19T08:46:04.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.955+0000 7f003a4c7640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f003406c600 con 0x7f003406bc90 2024-09-19T08:46:04.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.955+0000 7f0038a64640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f00340d2f00 0x7f00340d3330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42466/0 (socket says 172.21.15.98:42466) 2024-09-19T08:46:04.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.955+0000 7f0038a64640 1 -- 172.21.15.98:0/4193724928 learned_addr learned my addr 172.21.15.98:0/4193724928 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:04.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.955+0000 7f002bfff640 1 --2- 172.21.15.98:0/4193724928 >> v2:172.21.15.139:3300/0 conn(0x7f003406bc90 0x7f003406c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:04.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.955+0000 7f00337fe640 1 -- 172.21.15.98:0/4193724928 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2494594486 0 0) 0x7f0034072d30 con 0x7f00340d2f00 2024-09-19T08:46:04.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.955+0000 7f00337fe640 1 -- 172.21.15.98:0/4193724928 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0020003530 con 0x7f00340d2f00 2024-09-19T08:46:04.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.955+0000 7f00337fe640 1 -- 172.21.15.98:0/4193724928 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3040476393 0 0) 0x7f0020003530 con 0x7f00340d2f00 2024-09-19T08:46:04.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.955+0000 7f00337fe640 1 -- 172.21.15.98:0/4193724928 >> v2:172.21.15.139:3300/0 conn(0x7f003406bc90 msgr2=0x7f003406c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:04.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.955+0000 7f00337fe640 1 --2- 172.21.15.98:0/4193724928 >> v2:172.21.15.139:3300/0 conn(0x7f003406bc90 0x7f003406c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:04.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.955+0000 7f00337fe640 1 -- 172.21.15.98:0/4193724928 >> v1:172.21.15.139:6789/0 conn(0x7f0034161970 legacy=0x7f00340722b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:04.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.955+0000 7f00337fe640 1 -- 172.21.15.98:0/4193724928 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00340d2700 con 0x7f00340d2f00 2024-09-19T08:46:04.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.955+0000 7f00337fe640 1 -- 172.21.15.98:0/4193724928 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f002c001a00 con 0x7f00340d2f00 2024-09-19T08:46:04.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.955+0000 7f002bfff640 1 --2- 172.21.15.98:0/4193724928 >> v2:172.21.15.139:3300/0 conn(0x7f003406bc90 0x7f003406c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:04.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.955+0000 7f00337fe640 1 -- 172.21.15.98:0/4193724928 >> v1:172.21.15.98:6789/0 conn(0x7f00340d2f00 legacy=0x7f00340d3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:04.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.955+0000 7f00337fe640 1 --2- 172.21.15.98:0/4193724928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00200039c0 0x7f0020003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:04.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.955+0000 7f00337fe640 1 --2- 172.21.15.98:0/4193724928 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0020004810 0x7f0020004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:04.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.955+0000 7f00337fe640 1 --2- 172.21.15.98:0/4193724928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f00200051d0 0x7f002000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:04.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.955+0000 7f00337fe640 1 -- 172.21.15.98:0/4193724928 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f003406c600 con 0x7f00200051d0 2024-09-19T08:46:04.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.955+0000 7f00337fe640 1 -- 172.21.15.98:0/4193724928 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f002000c100 con 0x7f00200039c0 2024-09-19T08:46:04.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.955+0000 7f00337fe640 1 -- 172.21.15.98:0/4193724928 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f002000c300 con 0x7f0020004810 2024-09-19T08:46:04.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.955+0000 7f0038a64640 1 --2- 172.21.15.98:0/4193724928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f00200051d0 0x7f002000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:04.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.955+0000 7f0033fff640 1 --2- 172.21.15.98:0/4193724928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00200039c0 0x7f0020003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:04.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.955+0000 7f0038a64640 1 -- 172.21.15.98:0/4193724928 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0020004810 msgr2=0x7f0020004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:04.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.955+0000 7f0038a64640 1 --2- 172.21.15.98:0/4193724928 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0020004810 0x7f0020004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:04.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.955+0000 7f0038a64640 1 -- 172.21.15.98:0/4193724928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00200039c0 msgr2=0x7f0020003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:04.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.955+0000 7f0038a64640 1 --2- 172.21.15.98:0/4193724928 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00200039c0 0x7f0020003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:04.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.955+0000 7f0038a64640 1 -- 172.21.15.98:0/4193724928 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0020003530 con 0x7f00200051d0 2024-09-19T08:46:04.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.959+0000 7f0038a64640 1 -- 172.21.15.98:0/4193724928 reap_dead start 2024-09-19T08:46:04.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.959+0000 7f0038a64640 1 --2- 172.21.15.98:0/4193724928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f00200051d0 0x7f002000b9d0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f002c00a860 tx=0x7f002c01d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a00fe236e55bf8a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:04.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.959+0000 7f00337fe640 1 -- 172.21.15.98:0/4193724928 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c00a480 con 0x7f00200051d0 2024-09-19T08:46:04.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.959+0000 7f00337fe640 1 -- 172.21.15.98:0/4193724928 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f002c0ac620 con 0x7f00200051d0 2024-09-19T08:46:04.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.959+0000 7f00337fe640 1 -- 172.21.15.98:0/4193724928 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c0ac9f0 con 0x7f00200051d0 2024-09-19T08:46:04.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.959+0000 7f003a4c7640 1 -- 172.21.15.98:0/4193724928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f00200051d0 msgr2=0x7f002000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:04.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.959+0000 7f003a4c7640 1 --2- 172.21.15.98:0/4193724928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f00200051d0 0x7f002000b9d0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f002c00a860 tx=0x7f002c01d380 comp rx=0 tx=0).stop 2024-09-19T08:46:04.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.959+0000 7f003a4c7640 1 -- 172.21.15.98:0/4193724928 shutdown_connections 2024-09-19T08:46:04.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.959+0000 7f003a4c7640 1 --2- 172.21.15.98:0/4193724928 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f00200051d0 0x7f002000b9d0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:04.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.959+0000 7f003a4c7640 1 -- 172.21.15.98:0/4193724928 >> 172.21.15.98:0/4193724928 conn(0x7f00340d1200 msgr2=0x7f00340d1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:04.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.959+0000 7f003a4c7640 1 -- 172.21.15.98:0/4193724928 shutdown_connections 2024-09-19T08:46:04.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.959+0000 7f003a4c7640 1 -- 172.21.15.98:0/4193724928 wait complete. 2024-09-19T08:46:04.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.959+0000 7f003a4c7640 1 Processor -- start 2024-09-19T08:46:04.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.959+0000 7f003a4c7640 1 -- start start 2024-09-19T08:46:04.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.959+0000 7f003a4c7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f00200051d0 0x7f003415e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:04.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.959+0000 7f003a4c7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f003415b8f0 0x7f00340b9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:04.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.959+0000 7f003a4c7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00340b9740 0x7f0034149e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:04.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.959+0000 7f003a4c7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0034161d50 con 0x7f00200051d0 2024-09-19T08:46:04.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.959+0000 7f003a4c7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f00340cee20 con 0x7f00340b9740 2024-09-19T08:46:04.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.959+0000 7f003a4c7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0034161ed0 con 0x7f003415b8f0 2024-09-19T08:46:04.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.959+0000 7f0033fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f00200051d0 0x7f003415e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:04.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.959+0000 7f0033fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f00200051d0 0x7f003415e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46854/0 (socket says 172.21.15.98:46854) 2024-09-19T08:46:04.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.959+0000 7f0033fff640 1 -- 172.21.15.98:0/706731229 learned_addr learned my addr 172.21.15.98:0/706731229 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:04.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.959+0000 7f0033fff640 1 -- 172.21.15.98:0/706731229 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f003415b8f0 msgr2=0x7f00340b9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:04.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.959+0000 7f0038a64640 1 --2- 172.21.15.98:0/706731229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00340b9740 0x7f0034149e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:04.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.959+0000 7f002bfff640 1 --2- 172.21.15.98:0/706731229 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f003415b8f0 0x7f00340b9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:04.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.959+0000 7f0033fff640 1 --2- 172.21.15.98:0/706731229 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f003415b8f0 0x7f00340b9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:04.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.959+0000 7f0033fff640 1 -- 172.21.15.98:0/706731229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00340b9740 msgr2=0x7f0034149e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:04.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.959+0000 7f0033fff640 1 --2- 172.21.15.98:0/706731229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00340b9740 0x7f0034149e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:04.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.959+0000 7f0033fff640 1 -- 172.21.15.98:0/706731229 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f003415d180 con 0x7f00200051d0 2024-09-19T08:46:04.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.959+0000 7f0033fff640 1 --2- 172.21.15.98:0/706731229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f00200051d0 0x7f003415e1a0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f002400d520 tx=0x7f002400d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:04.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.959+0000 7f00317fa640 1 -- 172.21.15.98:0/706731229 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024005480 con 0x7f00200051d0 2024-09-19T08:46:04.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.959+0000 7f00317fa640 1 -- 172.21.15.98:0/706731229 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0024017070 con 0x7f00200051d0 2024-09-19T08:46:04.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.959+0000 7f003a4c7640 1 -- 172.21.15.98:0/706731229 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f003415d400 con 0x7f00200051d0 2024-09-19T08:46:04.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.959+0000 7f00317fa640 1 -- 172.21.15.98:0/706731229 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024003840 con 0x7f00200051d0 2024-09-19T08:46:04.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.959+0000 7f003a4c7640 1 -- 172.21.15.98:0/706731229 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f003415d9a0 con 0x7f00200051d0 2024-09-19T08:46:04.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.963+0000 7f003a4c7640 1 -- 172.21.15.98:0/706731229 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f003406c0c0 con 0x7f00200051d0 2024-09-19T08:46:04.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.963+0000 7f00317fa640 1 -- 172.21.15.98:0/706731229 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f00240039e0 con 0x7f00200051d0 2024-09-19T08:46:04.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.963+0000 7f00317fa640 1 --2- 172.21.15.98:0/706731229 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0014067f20 0x7f001406a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:04.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.963+0000 7f00317fa640 1 -- 172.21.15.98:0/706731229 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5737+0+0 (secure 0 0 0) 0x7f0024086230 con 0x7f00200051d0 2024-09-19T08:46:04.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.963+0000 7f002bfff640 1 --2- 172.21.15.98:0/706731229 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0014067f20 0x7f001406a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:04.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.963+0000 7f002bfff640 1 --2- 172.21.15.98:0/706731229 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0014067f20 0x7f001406a3e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f003415e770 tx=0x7f001c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:04.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:04.967+0000 7f00317fa640 1 -- 172.21.15.98:0/706731229 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f003415d180 con 0x7f00200051d0 2024-09-19T08:46:05.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.147+0000 7f003a4c7640 1 -- 172.21.15.98:0/706731229 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f00340b9b20 con 0x7f00200051d0 2024-09-19T08:46:05.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.835+0000 7f00317fa640 1 -- 172.21.15.98:0/706731229 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v82) ==== 122+0+0 (secure 0 0 0) 0x7f0024056370 con 0x7f00200051d0 2024-09-19T08:46:05.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.839+0000 7f003a4c7640 1 -- 172.21.15.98:0/706731229 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0014067f20 msgr2=0x7f001406a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:05.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.839+0000 7f003a4c7640 1 --2- 172.21.15.98:0/706731229 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0014067f20 0x7f001406a3e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f003415e770 tx=0x7f001c008040 comp rx=0 tx=0).stop 2024-09-19T08:46:05.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.839+0000 7f003a4c7640 1 -- 172.21.15.98:0/706731229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f00200051d0 msgr2=0x7f003415e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:05.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.839+0000 7f003a4c7640 1 --2- 172.21.15.98:0/706731229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f00200051d0 0x7f003415e1a0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f002400d520 tx=0x7f002400d9e0 comp rx=0 tx=0).stop 2024-09-19T08:46:05.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.839+0000 7f003a4c7640 1 -- 172.21.15.98:0/706731229 shutdown_connections 2024-09-19T08:46:05.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.839+0000 7f003a4c7640 1 --2- 172.21.15.98:0/706731229 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0014067f20 0x7f001406a3e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:05.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.839+0000 7f003a4c7640 1 --2- 172.21.15.98:0/706731229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f00340b9740 0x7f0034149e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:05.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.839+0000 7f003a4c7640 1 --2- 172.21.15.98:0/706731229 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f003415b8f0 0x7f00340b9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:05.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.839+0000 7f003a4c7640 1 --2- 172.21.15.98:0/706731229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f00200051d0 0x7f003415e1a0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:05.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.839+0000 7f003a4c7640 1 -- 172.21.15.98:0/706731229 >> 172.21.15.98:0/706731229 conn(0x7f00340d1200 msgr2=0x7f00340b7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:05.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.839+0000 7f003a4c7640 1 -- 172.21.15.98:0/706731229 shutdown_connections 2024-09-19T08:46:05.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.839+0000 7f003a4c7640 1 -- 172.21.15.98:0/706731229 wait complete. 2024-09-19T08:46:05.858 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-19T08:46:05.858 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:46:05.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.983+0000 7f62ba001640 1 Processor -- start 2024-09-19T08:46:05.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.983+0000 7f62ba001640 1 -- start start 2024-09-19T08:46:05.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.983+0000 7f62ba001640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f62b406c3a0 0x7f62b40ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:05.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.983+0000 7f62ba001640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f62b40ce6e0 0x7f62b40b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:05.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.983+0000 7f62ba001640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62b40ceb90 con 0x7f62b4158e40 2024-09-19T08:46:05.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.983+0000 7f62ba001640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f62b415fcf0 con 0x7f62b40ce6e0 2024-09-19T08:46:05.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.983+0000 7f62ba001640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f62b415ef30 con 0x7f62b406c3a0 2024-09-19T08:46:05.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.983+0000 7f62b37fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f62b4158e40 0x7f62b406bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42474/0 (socket says 172.21.15.98:42474) 2024-09-19T08:46:05.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.983+0000 7f62b37fe640 1 -- 172.21.15.98:0/4124662784 learned_addr learned my addr 172.21.15.98:0/4124662784 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:05.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.983+0000 7f62b3fff640 1 --2- 172.21.15.98:0/4124662784 >> v2:172.21.15.139:3300/0 conn(0x7f62b40ce6e0 0x7f62b40b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:05.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.983+0000 7f62b27fc640 1 -- 172.21.15.98:0/4124662784 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1469314016 0 0) 0x7f62b40ceb90 con 0x7f62b4158e40 2024-09-19T08:46:05.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.983+0000 7f62b2ffd640 1 --2- 172.21.15.98:0/4124662784 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f62b406c3a0 0x7f62b40ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:05.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.983+0000 7f62b27fc640 1 -- 172.21.15.98:0/4124662784 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f62a0003530 con 0x7f62b4158e40 2024-09-19T08:46:05.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.983+0000 7f62b27fc640 1 -- 172.21.15.98:0/4124662784 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1835631122 0 0) 0x7f62a0003530 con 0x7f62b4158e40 2024-09-19T08:46:05.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.983+0000 7f62b27fc640 1 -- 172.21.15.98:0/4124662784 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f62b406c3a0 msgr2=0x7f62b40ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:05.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.983+0000 7f62b27fc640 1 --2- 172.21.15.98:0/4124662784 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f62b406c3a0 0x7f62b40ce1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:05.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.983+0000 7f62b27fc640 1 -- 172.21.15.98:0/4124662784 >> v2:172.21.15.139:3300/0 conn(0x7f62b40ce6e0 msgr2=0x7f62b40b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:05.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.983+0000 7f62b27fc640 1 --2- 172.21.15.98:0/4124662784 >> v2:172.21.15.139:3300/0 conn(0x7f62b40ce6e0 0x7f62b40b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:05.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.983+0000 7f62b27fc640 1 -- 172.21.15.98:0/4124662784 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b415f210 con 0x7f62b4158e40 2024-09-19T08:46:05.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.983+0000 7f62b27fc640 1 -- 172.21.15.98:0/4124662784 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f62ac001680 con 0x7f62b4158e40 2024-09-19T08:46:05.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.983+0000 7f62b27fc640 1 -- 172.21.15.98:0/4124662784 >> v1:172.21.15.98:6789/0 conn(0x7f62b4158e40 legacy=0x7f62b406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:05.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.983+0000 7f62b27fc640 1 --2- 172.21.15.98:0/4124662784 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f62b406c3a0 0x7f62a00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:05.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.983+0000 7f62b27fc640 1 --2- 172.21.15.98:0/4124662784 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62a0003ff0 0x7f62a0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:05.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.983+0000 7f62b27fc640 1 --2- 172.21.15.98:0/4124662784 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62a0004d50 0x7f62a00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:05.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.983+0000 7f62b27fc640 1 -- 172.21.15.98:0/4124662784 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f62b415fcf0 con 0x7f62a0004d50 2024-09-19T08:46:05.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.983+0000 7f62b27fc640 1 -- 172.21.15.98:0/4124662784 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f62b415ef30 con 0x7f62a0003ff0 2024-09-19T08:46:05.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.983+0000 7f62b27fc640 1 -- 172.21.15.98:0/4124662784 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f62a000bb60 con 0x7f62b406c3a0 2024-09-19T08:46:05.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.983+0000 7f62b3fff640 1 --2- 172.21.15.98:0/4124662784 >> v2:172.21.15.139:3300/0 conn(0x7f62b40ce6e0 0x7f62b40b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:05.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.983+0000 7f62b3fff640 1 --2- 172.21.15.98:0/4124662784 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62a0004d50 0x7f62a00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:05.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.983+0000 7f62b37fe640 1 --2- 172.21.15.98:0/4124662784 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f62b406c3a0 0x7f62a00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:05.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.983+0000 7f62b2ffd640 1 --2- 172.21.15.98:0/4124662784 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62a0003ff0 0x7f62a0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:05.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.983+0000 7f62b3fff640 1 -- 172.21.15.98:0/4124662784 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f62b406c3a0 msgr2=0x7f62a00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:05.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.983+0000 7f62b3fff640 1 --2- 172.21.15.98:0/4124662784 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f62b406c3a0 0x7f62a00039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:05.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.983+0000 7f62b3fff640 1 -- 172.21.15.98:0/4124662784 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62a0003ff0 msgr2=0x7f62a0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:05.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.983+0000 7f62b3fff640 1 --2- 172.21.15.98:0/4124662784 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62a0003ff0 0x7f62a0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:05.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.983+0000 7f62b3fff640 1 -- 172.21.15.98:0/4124662784 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a0003530 con 0x7f62a0004d50 2024-09-19T08:46:05.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.987+0000 7f62b3fff640 1 --2- 172.21.15.98:0/4124662784 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62a0004d50 0x7f62a00051d0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f62a40073e0 tx=0x7f62a401cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c36db163a04027 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:05.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.987+0000 7f62b27fc640 1 -- 172.21.15.98:0/4124662784 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a40b2070 con 0x7f62a0004d50 2024-09-19T08:46:05.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.987+0000 7f62b27fc640 1 -- 172.21.15.98:0/4124662784 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62a40a5e40 con 0x7f62a0004d50 2024-09-19T08:46:05.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.987+0000 7f62b27fc640 1 -- 172.21.15.98:0/4124662784 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a4009c70 con 0x7f62a0004d50 2024-09-19T08:46:05.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.987+0000 7f62ba001640 1 -- 172.21.15.98:0/4124662784 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62a0004d50 msgr2=0x7f62a00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:05.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.987+0000 7f62ba001640 1 --2- 172.21.15.98:0/4124662784 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62a0004d50 0x7f62a00051d0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f62a40073e0 tx=0x7f62a401cf40 comp rx=0 tx=0).stop 2024-09-19T08:46:05.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.987+0000 7f62b3fff640 1 -- 172.21.15.98:0/4124662784 reap_dead start 2024-09-19T08:46:05.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.987+0000 7f62ba001640 1 -- 172.21.15.98:0/4124662784 shutdown_connections 2024-09-19T08:46:05.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.987+0000 7f62ba001640 1 -- 172.21.15.98:0/4124662784 >> 172.21.15.98:0/4124662784 conn(0x7f62b40b2010 msgr2=0x7f62b40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:05.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.987+0000 7f62ba001640 1 -- 172.21.15.98:0/4124662784 shutdown_connections 2024-09-19T08:46:05.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.987+0000 7f62ba001640 1 -- 172.21.15.98:0/4124662784 wait complete. 2024-09-19T08:46:05.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.987+0000 7f62ba001640 1 Processor -- start 2024-09-19T08:46:05.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.987+0000 7f62ba001640 1 -- start start 2024-09-19T08:46:05.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.991+0000 7f62ba001640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62b4148e40 0x7f62b4151df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:05.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.991+0000 7f62ba001640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62b4072a60 0x7f62b4152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:05.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.991+0000 7f62ba001640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f62b4152c80 0x7f62b4154760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:05.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.991+0000 7f62ba001640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f62b4159220 con 0x7f62b4072a60 2024-09-19T08:46:05.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.991+0000 7f62ba001640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f62b415f7d0 con 0x7f62b4148e40 2024-09-19T08:46:05.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.991+0000 7f62ba001640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f62b40cee20 con 0x7f62b4152c80 2024-09-19T08:46:05.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.991+0000 7f62b2ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62b4072a60 0x7f62b4152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:05.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.991+0000 7f62b2ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62b4072a60 0x7f62b4152740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46872/0 (socket says 172.21.15.98:46872) 2024-09-19T08:46:05.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.991+0000 7f62b2ffd640 1 -- 172.21.15.98:0/3925087838 learned_addr learned my addr 172.21.15.98:0/3925087838 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:05.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.991+0000 7f62b37fe640 1 --2- 172.21.15.98:0/3925087838 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62b4148e40 0x7f62b4151df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:05.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.991+0000 7f62b3fff640 1 --2- 172.21.15.98:0/3925087838 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f62b4152c80 0x7f62b4154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:05.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.991+0000 7f62b2ffd640 1 -- 172.21.15.98:0/3925087838 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f62b4152c80 msgr2=0x7f62b4154760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:05.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.991+0000 7f62b2ffd640 1 --2- 172.21.15.98:0/3925087838 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f62b4152c80 0x7f62b4154760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:05.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.991+0000 7f62b2ffd640 1 -- 172.21.15.98:0/3925087838 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62b4148e40 msgr2=0x7f62b4151df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:05.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.991+0000 7f62b2ffd640 1 --2- 172.21.15.98:0/3925087838 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62b4148e40 0x7f62b4151df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:05.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.991+0000 7f62b2ffd640 1 -- 172.21.15.98:0/3925087838 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b4072ec0 con 0x7f62b4072a60 2024-09-19T08:46:05.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.991+0000 7f62b3fff640 1 --2- 172.21.15.98:0/3925087838 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f62b4152c80 0x7f62b4154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:05.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.991+0000 7f62b2ffd640 1 --2- 172.21.15.98:0/3925087838 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62b4072a60 0x7f62b4152740 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f629c00f580 tx=0x7f629c00fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:05.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.991+0000 7f62abfff640 1 -- 172.21.15.98:0/3925087838 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c013810 con 0x7f62b4072a60 2024-09-19T08:46:05.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.991+0000 7f62abfff640 1 -- 172.21.15.98:0/3925087838 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f629c01a070 con 0x7f62b4072a60 2024-09-19T08:46:05.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.991+0000 7f62abfff640 1 -- 172.21.15.98:0/3925087838 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c013bf0 con 0x7f62b4072a60 2024-09-19T08:46:05.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.991+0000 7f62ba001640 1 -- 172.21.15.98:0/3925087838 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b4073160 con 0x7f62b4072a60 2024-09-19T08:46:05.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.991+0000 7f62ba001640 1 -- 172.21.15.98:0/3925087838 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b4073720 con 0x7f62b4072a60 2024-09-19T08:46:05.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.991+0000 7f62ba001640 1 -- 172.21.15.98:0/3925087838 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62b40ce1a0 con 0x7f62b4072a60 2024-09-19T08:46:05.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.991+0000 7f62abfff640 1 -- 172.21.15.98:0/3925087838 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f629c012070 con 0x7f62b4072a60 2024-09-19T08:46:05.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.991+0000 7f62abfff640 1 --2- 172.21.15.98:0/3925087838 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6294067f20 0x7f629406a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:05.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.991+0000 7f62abfff640 1 -- 172.21.15.98:0/3925087838 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5737+0+0 (secure 0 0 0) 0x7f629c086670 con 0x7f62b4072a60 2024-09-19T08:46:05.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.991+0000 7f62b37fe640 1 --2- 172.21.15.98:0/3925087838 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6294067f20 0x7f629406a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:05.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.995+0000 7f62b37fe640 1 --2- 172.21.15.98:0/3925087838 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6294067f20 0x7f629406a3e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f62ac005b60 tx=0x7f62ac009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:05.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:05.995+0000 7f62abfff640 1 -- 172.21.15.98:0/3925087838 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f629c01c310 con 0x7f62b4072a60 2024-09-19T08:46:06.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.187+0000 7f62ba001640 1 -- 172.21.15.98:0/3925087838 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62b4153090 con 0x7f62b4072a60 2024-09-19T08:46:06.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.187+0000 7f62abfff640 1 -- 172.21.15.98:0/3925087838 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5221 (secure 0 0 0) 0x7f629c056960 con 0x7f62b4072a60 2024-09-19T08:46:06.188 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:46:06.188 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":82,"btime":"2024-09-19T08:46:05:828977+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6886,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3238157999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3238157999},{"type":"v1","addr":"172.21.15.139:6833","nonce":3238157999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/209335177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":209335177},{"type":"v1","addr":"172.21.15.139:6835","nonce":209335177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8370,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6834/26914461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":26914461},{"type":"v1","addr":"172.21.15.98:6834","nonce":26914461}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:46:03.123986+0000","modified":"2024-09-19T08:46:05.150802+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8373":{"gid":8373,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/451004304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6833","nonce":451004304},{"type":"v1","addr":"172.21.15.98:6835","nonce":451004304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8373,"qdb_cluster":[8373]},"id":11}]} 2024-09-19T08:46:06.188 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 82 2024-09-19T08:46:06.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.191+0000 7f62ba001640 1 -- 172.21.15.98:0/3925087838 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6294067f20 msgr2=0x7f629406a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:06.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.191+0000 7f62ba001640 1 --2- 172.21.15.98:0/3925087838 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6294067f20 0x7f629406a3e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f62ac005b60 tx=0x7f62ac009040 comp rx=0 tx=0).stop 2024-09-19T08:46:06.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.191+0000 7f62ba001640 1 -- 172.21.15.98:0/3925087838 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62b4072a60 msgr2=0x7f62b4152740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:06.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.191+0000 7f62ba001640 1 --2- 172.21.15.98:0/3925087838 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62b4072a60 0x7f62b4152740 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f629c00f580 tx=0x7f629c00fa40 comp rx=0 tx=0).stop 2024-09-19T08:46:06.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.191+0000 7f62ba001640 1 -- 172.21.15.98:0/3925087838 shutdown_connections 2024-09-19T08:46:06.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.191+0000 7f62ba001640 1 --2- 172.21.15.98:0/3925087838 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6294067f20 0x7f629406a3e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:06.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.191+0000 7f62ba001640 1 --2- 172.21.15.98:0/3925087838 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f62b4152c80 0x7f62b4154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:06.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.191+0000 7f62ba001640 1 --2- 172.21.15.98:0/3925087838 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62b4072a60 0x7f62b4152740 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:06.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.191+0000 7f62ba001640 1 --2- 172.21.15.98:0/3925087838 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62b4148e40 0x7f62b4151df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:06.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.191+0000 7f62ba001640 1 -- 172.21.15.98:0/3925087838 >> 172.21.15.98:0/3925087838 conn(0x7f62b40b2010 msgr2=0x7f62b40b3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:06.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.191+0000 7f62ba001640 1 -- 172.21.15.98:0/3925087838 shutdown_connections 2024-09-19T08:46:06.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.191+0000 7f62ba001640 1 -- 172.21.15.98:0/3925087838 wait complete. 2024-09-19T08:46:06.205 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:46:06.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.311+0000 7f0af4ce5640 1 Processor -- start 2024-09-19T08:46:06.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.311+0000 7f0af4ce5640 1 -- start start 2024-09-19T08:46:06.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0af4ce5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0af00d4820 0x7f0af01621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:06.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0af4ce5640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0af0068440 con 0x7f0af00b9970 2024-09-19T08:46:06.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0af4ce5640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0af0162a90 con 0x7f0af00b7f80 2024-09-19T08:46:06.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0af4ce5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0af0163050 con 0x7f0af00d4820 2024-09-19T08:46:06.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0aeed9d640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f0af00b9970 0x7f0af00b9e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42484/0 (socket says 172.21.15.98:42484) 2024-09-19T08:46:06.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0aeed9d640 1 -- 172.21.15.98:0/4149110012 learned_addr learned my addr 172.21.15.98:0/4149110012 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:06.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0aed59a640 1 -- 172.21.15.98:0/4149110012 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2675307447 0 0) 0x7f0af0068440 con 0x7f0af00b9970 2024-09-19T08:46:06.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0aed59a640 1 -- 172.21.15.98:0/4149110012 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ad8003530 con 0x7f0af00b9970 2024-09-19T08:46:06.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0aedd9b640 1 --2- 172.21.15.98:0/4149110012 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0af00d4820 0x7f0af01621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:06.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0aed59a640 1 -- 172.21.15.98:0/4149110012 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4077276685 0 0) 0x7f0ad8003530 con 0x7f0af00b9970 2024-09-19T08:46:06.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0aed59a640 1 -- 172.21.15.98:0/4149110012 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0af00d4820 msgr2=0x7f0af01621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:06.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0aed59a640 1 --2- 172.21.15.98:0/4149110012 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0af00d4820 0x7f0af01621b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:06.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0aed59a640 1 -- 172.21.15.98:0/4149110012 >> v1:172.21.15.139:6789/0 conn(0x7f0af00b7f80 legacy=0x7f0af015f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:06.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0aed59a640 1 -- 172.21.15.98:0/4149110012 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af0162d70 con 0x7f0af00b9970 2024-09-19T08:46:06.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0aed59a640 1 -- 172.21.15.98:0/4149110012 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f0ae8001a00 con 0x7f0af00b9970 2024-09-19T08:46:06.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0aed59a640 1 -- 172.21.15.98:0/4149110012 >> v1:172.21.15.98:6789/0 conn(0x7f0af00b9970 legacy=0x7f0af00b9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:06.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0aed59a640 1 --2- 172.21.15.98:0/4149110012 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0af00d4820 0x7f0ad80039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:06.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0aed59a640 1 --2- 172.21.15.98:0/4149110012 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ad8004020 0x7f0ad8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:06.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0aed59a640 1 --2- 172.21.15.98:0/4149110012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ad8004d50 0x7f0ad80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:06.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0aed59a640 1 -- 172.21.15.98:0/4149110012 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0af0163050 con 0x7f0ad8004020 2024-09-19T08:46:06.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0aed59a640 1 -- 172.21.15.98:0/4149110012 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad800bb90 con 0x7f0ad8004d50 2024-09-19T08:46:06.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0aed59a640 1 -- 172.21.15.98:0/4149110012 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0ad800bd90 con 0x7f0af00d4820 2024-09-19T08:46:06.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0aedd9b640 1 --2- 172.21.15.98:0/4149110012 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ad8004020 0x7f0ad8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:06.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0aeed9d640 1 --2- 172.21.15.98:0/4149110012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ad8004d50 0x7f0ad80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:06.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0aee59c640 1 --2- 172.21.15.98:0/4149110012 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0af00d4820 0x7f0ad80039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:06.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0aedd9b640 1 -- 172.21.15.98:0/4149110012 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0af00d4820 msgr2=0x7f0ad80039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:06.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0aedd9b640 1 --2- 172.21.15.98:0/4149110012 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0af00d4820 0x7f0ad80039f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:06.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0aedd9b640 1 -- 172.21.15.98:0/4149110012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ad8004d50 msgr2=0x7f0ad80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:06.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0aedd9b640 1 --2- 172.21.15.98:0/4149110012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ad8004d50 0x7f0ad80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:06.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0aedd9b640 1 -- 172.21.15.98:0/4149110012 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ad8003530 con 0x7f0ad8004020 2024-09-19T08:46:06.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0aeed9d640 1 --2- 172.21.15.98:0/4149110012 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0ad8004d50 0x7f0ad80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:06.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0aedd9b640 1 --2- 172.21.15.98:0/4149110012 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ad8004020 0x7f0ad8004810 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f0ae400b9e0 tx=0x7f0ae401c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b759ebc60687c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:06.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0aed59a640 1 -- 172.21.15.98:0/4149110012 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae400b5e0 con 0x7f0ad8004020 2024-09-19T08:46:06.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0aed59a640 1 -- 172.21.15.98:0/4149110012 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ae40b0070 con 0x7f0ad8004020 2024-09-19T08:46:06.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0aed59a640 1 -- 172.21.15.98:0/4149110012 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae40ab8b0 con 0x7f0ad8004020 2024-09-19T08:46:06.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0af4ce5640 1 -- 172.21.15.98:0/4149110012 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ad8004020 msgr2=0x7f0ad8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:06.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0af4ce5640 1 --2- 172.21.15.98:0/4149110012 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0ad8004020 0x7f0ad8004810 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f0ae400b9e0 tx=0x7f0ae401c640 comp rx=0 tx=0).stop 2024-09-19T08:46:06.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.315+0000 7f0aeed9d640 1 -- 172.21.15.98:0/4149110012 reap_dead start 2024-09-19T08:46:06.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.319+0000 7f0af4ce5640 1 -- 172.21.15.98:0/4149110012 shutdown_connections 2024-09-19T08:46:06.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.319+0000 7f0af4ce5640 1 -- 172.21.15.98:0/4149110012 >> 172.21.15.98:0/4149110012 conn(0x7f0af0074e40 msgr2=0x7f0af0075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:06.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.319+0000 7f0af4ce5640 1 -- 172.21.15.98:0/4149110012 shutdown_connections 2024-09-19T08:46:06.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.319+0000 7f0af4ce5640 1 -- 172.21.15.98:0/4149110012 wait complete. 2024-09-19T08:46:06.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.319+0000 7f0af4ce5640 1 Processor -- start 2024-09-19T08:46:06.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.319+0000 7f0af4ce5640 1 -- start start 2024-09-19T08:46:06.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.319+0000 7f0af4ce5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ad8004020 0x7f0af0144390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:06.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.319+0000 7f0af4ce5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0af01f50c0 0x7f0af01551d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:06.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.319+0000 7f0af4ce5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0af00c9440 0x7f0af00c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:06.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.319+0000 7f0af4ce5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0af00b84e0 con 0x7f0af00c9440 2024-09-19T08:46:06.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.319+0000 7f0af4ce5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0af00b8360 con 0x7f0af01f50c0 2024-09-19T08:46:06.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.319+0000 7f0af4ce5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0af01621b0 con 0x7f0ad8004020 2024-09-19T08:46:06.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.319+0000 7f0aeed9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0af00c9440 0x7f0af00c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:06.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.319+0000 7f0aeed9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0af00c9440 0x7f0af00c98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46882/0 (socket says 172.21.15.98:46882) 2024-09-19T08:46:06.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.319+0000 7f0aeed9d640 1 -- 172.21.15.98:0/227243171 learned_addr learned my addr 172.21.15.98:0/227243171 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:06.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.319+0000 7f0aee59c640 1 --2- 172.21.15.98:0/227243171 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ad8004020 0x7f0af0144390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:06.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.319+0000 7f0aedd9b640 1 --2- 172.21.15.98:0/227243171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0af01f50c0 0x7f0af01551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:06.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.319+0000 7f0aeed9d640 1 -- 172.21.15.98:0/227243171 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ad8004020 msgr2=0x7f0af0144390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:06.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.319+0000 7f0aeed9d640 1 --2- 172.21.15.98:0/227243171 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ad8004020 0x7f0af0144390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:06.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.319+0000 7f0aeed9d640 1 -- 172.21.15.98:0/227243171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0af01f50c0 msgr2=0x7f0af01551d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:06.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.319+0000 7f0aeed9d640 1 --2- 172.21.15.98:0/227243171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0af01f50c0 0x7f0af01551d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:06.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.319+0000 7f0aeed9d640 1 -- 172.21.15.98:0/227243171 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af00cae70 con 0x7f0af00c9440 2024-09-19T08:46:06.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.319+0000 7f0aeed9d640 1 --2- 172.21.15.98:0/227243171 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0af00c9440 0x7f0af00c98c0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f0af00b6670 tx=0x7f0ae8010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:06.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.319+0000 7f0adeffd640 1 -- 172.21.15.98:0/227243171 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae801a070 con 0x7f0af00c9440 2024-09-19T08:46:06.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.319+0000 7f0adeffd640 1 -- 172.21.15.98:0/227243171 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ae8011040 con 0x7f0af00c9440 2024-09-19T08:46:06.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.319+0000 7f0af4ce5640 1 -- 172.21.15.98:0/227243171 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0af00cb150 con 0x7f0af00c9440 2024-09-19T08:46:06.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.319+0000 7f0adeffd640 1 -- 172.21.15.98:0/227243171 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae800ece0 con 0x7f0af00c9440 2024-09-19T08:46:06.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.319+0000 7f0af4ce5640 1 -- 172.21.15.98:0/227243171 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0af00cb710 con 0x7f0af00c9440 2024-09-19T08:46:06.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.319+0000 7f0af4ce5640 1 -- 172.21.15.98:0/227243171 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0af0162500 con 0x7f0af00c9440 2024-09-19T08:46:06.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.323+0000 7f0adeffd640 1 -- 172.21.15.98:0/227243171 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f0ae801d070 con 0x7f0af00c9440 2024-09-19T08:46:06.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.323+0000 7f0adeffd640 1 --2- 172.21.15.98:0/227243171 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0ad0067f20 0x7f0ad006a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:06.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.323+0000 7f0adeffd640 1 -- 172.21.15.98:0/227243171 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5737+0+0 (secure 0 0 0) 0x7f0ae8088f60 con 0x7f0af00c9440 2024-09-19T08:46:06.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.323+0000 7f0aee59c640 1 --2- 172.21.15.98:0/227243171 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0ad0067f20 0x7f0ad006a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:06.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.323+0000 7f0aee59c640 1 --2- 172.21.15.98:0/227243171 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0ad0067f20 0x7f0ad006a3e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0007fd0 tx=0x7f0ae0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:06.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.327+0000 7f0adeffd640 1 -- 172.21.15.98:0/227243171 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f0ae80591d0 con 0x7f0af00c9440 2024-09-19T08:46:06.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.479+0000 7f0af4ce5640 1 -- 172.21.15.98:0/227243171 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0af00cb8e0 con 0x7f0af00c9440 2024-09-19T08:46:06.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.483+0000 7f0adeffd640 1 -- 172.21.15.98:0/227243171 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) ==== 74+0+16775 (secure 0 0 0) 0x7f0ae8058ad0 con 0x7f0af00c9440 2024-09-19T08:46:06.481 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:46:06.481 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":146,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:46:04.805194+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:46:00.845215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-19T08:46:02.145134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:46:06.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.483+0000 7f0af4ce5640 1 -- 172.21.15.98:0/227243171 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0ad0067f20 msgr2=0x7f0ad006a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:06.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.483+0000 7f0af4ce5640 1 --2- 172.21.15.98:0/227243171 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0ad0067f20 0x7f0ad006a3e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0ae0007fd0 tx=0x7f0ae0009040 comp rx=0 tx=0).stop 2024-09-19T08:46:06.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.483+0000 7f0af4ce5640 1 -- 172.21.15.98:0/227243171 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0af00c9440 msgr2=0x7f0af00c98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:06.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.483+0000 7f0af4ce5640 1 --2- 172.21.15.98:0/227243171 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0af00c9440 0x7f0af00c98c0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f0af00b6670 tx=0x7f0ae8010d70 comp rx=0 tx=0).stop 2024-09-19T08:46:06.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.483+0000 7f0af4ce5640 1 -- 172.21.15.98:0/227243171 shutdown_connections 2024-09-19T08:46:06.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.483+0000 7f0af4ce5640 1 --2- 172.21.15.98:0/227243171 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0ad0067f20 0x7f0ad006a3e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:06.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.483+0000 7f0af4ce5640 1 --2- 172.21.15.98:0/227243171 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0af00c9440 0x7f0af00c98c0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:06.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.483+0000 7f0af4ce5640 1 --2- 172.21.15.98:0/227243171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0af01f50c0 0x7f0af01551d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:06.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.483+0000 7f0af4ce5640 1 --2- 172.21.15.98:0/227243171 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0ad8004020 0x7f0af0144390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:06.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.483+0000 7f0af4ce5640 1 -- 172.21.15.98:0/227243171 >> 172.21.15.98:0/227243171 conn(0x7f0af0074e40 msgr2=0x7f0af00cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:06.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.483+0000 7f0af4ce5640 1 -- 172.21.15.98:0/227243171 shutdown_connections 2024-09-19T08:46:06.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.483+0000 7f0af4ce5640 1 -- 172.21.15.98:0/227243171 wait complete. 2024-09-19T08:46:06.498 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T08:46:06.498 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:46:06.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.603+0000 7f33c01db640 1 Processor -- start 2024-09-19T08:46:06.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.603+0000 7f33c01db640 1 -- start start 2024-09-19T08:46:06.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.607+0000 7f33c01db640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f33b80d4820 0x7f33b81621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:06.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.607+0000 7f33c01db640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33b80b9970 0x7f33b80b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:06.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.607+0000 7f33c01db640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33b8068440 con 0x7f33b80b7f80 2024-09-19T08:46:06.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.607+0000 7f33c01db640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f33b8163050 con 0x7f33b80d4820 2024-09-19T08:46:06.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.607+0000 7f33c01db640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f33b8162a90 con 0x7f33b80b9970 2024-09-19T08:46:06.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.607+0000 7f33bd776640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f33b80d4820 0x7f33b81621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:06.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.607+0000 7f33be778640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33b80b9970 0x7f33b80b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:06.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.607+0000 7f33bd776640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f33b80d4820 0x7f33b81621b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:34100/0 (socket says 172.21.15.98:34100) 2024-09-19T08:46:06.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.607+0000 7f33bd776640 1 -- 172.21.15.98:0/2452481522 learned_addr learned my addr 172.21.15.98:0/2452481522 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:06.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.607+0000 7f33bdf77640 1 --1- 172.21.15.98:0/2452481522 >> v1:172.21.15.139:6789/0 conn(0x7f33b80b7f80 0x7f33b815f560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:37986/0 (socket says 172.21.15.98:37986) 2024-09-19T08:46:06.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.607+0000 7f33bd776640 1 -- 172.21.15.98:0/2452481522 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33b80b9970 msgr2=0x7f33b80b9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:06.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.607+0000 7f33bd776640 1 --2- 172.21.15.98:0/2452481522 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33b80b9970 0x7f33b80b9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:06.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.607+0000 7f33bd776640 1 -- 172.21.15.98:0/2452481522 >> v1:172.21.15.139:6789/0 conn(0x7f33b80b7f80 legacy=0x7f33b815f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:06.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.607+0000 7f33bd776640 1 -- 172.21.15.98:0/2452481522 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b8162d70 con 0x7f33b80d4820 2024-09-19T08:46:06.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.607+0000 7f33be778640 1 --2- 172.21.15.98:0/2452481522 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33b80b9970 0x7f33b80b9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:46:06.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.611+0000 7f33bd776640 1 --2- 172.21.15.98:0/2452481522 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33b80d4820 0x7f33b81621b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f33ac009e50 tx=0x7f33ac01ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=513ffd77bb3506a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:06.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.611+0000 7f33bcf75640 1 -- 172.21.15.98:0/2452481522 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ac0a5810 con 0x7f33b80d4820 2024-09-19T08:46:06.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.611+0000 7f33bcf75640 1 -- 172.21.15.98:0/2452481522 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33ac0a59b0 con 0x7f33b80d4820 2024-09-19T08:46:06.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.611+0000 7f33bcf75640 1 -- 172.21.15.98:0/2452481522 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ac0a5d80 con 0x7f33b80d4820 2024-09-19T08:46:06.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.611+0000 7f33c01db640 1 -- 172.21.15.98:0/2452481522 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33b80d4820 msgr2=0x7f33b81621b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:06.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.611+0000 7f33c01db640 1 --2- 172.21.15.98:0/2452481522 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33b80d4820 0x7f33b81621b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f33ac009e50 tx=0x7f33ac01ca30 comp rx=0 tx=0).stop 2024-09-19T08:46:06.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.611+0000 7f33c01db640 1 -- 172.21.15.98:0/2452481522 shutdown_connections 2024-09-19T08:46:06.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.611+0000 7f33c01db640 1 --2- 172.21.15.98:0/2452481522 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33b80b9970 0x7f33b80b9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:06.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.611+0000 7f33c01db640 1 --2- 172.21.15.98:0/2452481522 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33b80d4820 0x7f33b81621b0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:06.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.611+0000 7f33c01db640 1 -- 172.21.15.98:0/2452481522 >> 172.21.15.98:0/2452481522 conn(0x7f33b8074e40 msgr2=0x7f33b8075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:06.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.611+0000 7f33c01db640 1 -- 172.21.15.98:0/2452481522 shutdown_connections 2024-09-19T08:46:06.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.611+0000 7f33c01db640 1 -- 172.21.15.98:0/2452481522 wait complete. 2024-09-19T08:46:06.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.611+0000 7f33c01db640 1 Processor -- start 2024-09-19T08:46:06.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.611+0000 7f33c01db640 1 -- start start 2024-09-19T08:46:06.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.611+0000 7f33c01db640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33b80b7f80 0x7f33b8140520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:06.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.611+0000 7f33c01db640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33b80b9970 0x7f33b815a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:06.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.611+0000 7f33c01db640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33b80d4820 0x7f33b815abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:06.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.611+0000 7f33c01db640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f33b80b84e0 con 0x7f33b80b7f80 2024-09-19T08:46:06.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.611+0000 7f33c01db640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f33b80b8360 con 0x7f33b80b9970 2024-09-19T08:46:06.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.611+0000 7f33c01db640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f33b8144590 con 0x7f33b80d4820 2024-09-19T08:46:06.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.611+0000 7f33bdf77640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33b80b7f80 0x7f33b8140520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:06.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.611+0000 7f33bdf77640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33b80b7f80 0x7f33b8140520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46890/0 (socket says 172.21.15.98:46890) 2024-09-19T08:46:06.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.611+0000 7f33bdf77640 1 -- 172.21.15.98:0/1464649131 learned_addr learned my addr 172.21.15.98:0/1464649131 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:06.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.611+0000 7f33bd776640 1 --2- 172.21.15.98:0/1464649131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33b80b9970 0x7f33b815a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:06.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.611+0000 7f33be778640 1 --2- 172.21.15.98:0/1464649131 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33b80d4820 0x7f33b815abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:06.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.611+0000 7f33bdf77640 1 -- 172.21.15.98:0/1464649131 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33b80d4820 msgr2=0x7f33b815abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:06.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.611+0000 7f33bdf77640 1 --2- 172.21.15.98:0/1464649131 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33b80d4820 0x7f33b815abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:06.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.611+0000 7f33bdf77640 1 -- 172.21.15.98:0/1464649131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33b80b9970 msgr2=0x7f33b815a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:06.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.611+0000 7f33bdf77640 1 --2- 172.21.15.98:0/1464649131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33b80b9970 0x7f33b815a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:06.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.611+0000 7f33bdf77640 1 -- 172.21.15.98:0/1464649131 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b8159610 con 0x7f33b80b7f80 2024-09-19T08:46:06.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.611+0000 7f33be778640 1 --2- 172.21.15.98:0/1464649131 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33b80d4820 0x7f33b815abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:06.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.611+0000 7f33bd776640 1 --2- 172.21.15.98:0/1464649131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33b80b9970 0x7f33b815a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:06.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.615+0000 7f33bdf77640 1 --2- 172.21.15.98:0/1464649131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33b80b7f80 0x7f33b8140520 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f33b000c370 tx=0x7f33b000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:06.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.615+0000 7f33b67fc640 1 -- 172.21.15.98:0/1464649131 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b00074e0 con 0x7f33b80b7f80 2024-09-19T08:46:06.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.615+0000 7f33b67fc640 1 -- 172.21.15.98:0/1464649131 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33b00076a0 con 0x7f33b80b7f80 2024-09-19T08:46:06.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.615+0000 7f33c01db640 1 -- 172.21.15.98:0/1464649131 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33b81598f0 con 0x7f33b80b7f80 2024-09-19T08:46:06.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.615+0000 7f33b67fc640 1 -- 172.21.15.98:0/1464649131 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b0004a10 con 0x7f33b80b7f80 2024-09-19T08:46:06.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.615+0000 7f33c01db640 1 -- 172.21.15.98:0/1464649131 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33b8159eb0 con 0x7f33b80b7f80 2024-09-19T08:46:06.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.615+0000 7f33c01db640 1 -- 172.21.15.98:0/1464649131 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33b80b9de0 con 0x7f33b80b7f80 2024-09-19T08:46:06.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.615+0000 7f33b67fc640 1 -- 172.21.15.98:0/1464649131 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f33b001a020 con 0x7f33b80b7f80 2024-09-19T08:46:06.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.615+0000 7f33b67fc640 1 --2- 172.21.15.98:0/1464649131 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f33a4067ef0 0x7f33a406a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:06.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.615+0000 7f33b67fc640 1 -- 172.21.15.98:0/1464649131 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5737+0+0 (secure 0 0 0) 0x7f33b0086030 con 0x7f33b80b7f80 2024-09-19T08:46:06.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.615+0000 7f33bd776640 1 --2- 172.21.15.98:0/1464649131 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f33a4067ef0 0x7f33a406a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:06.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.615+0000 7f33bd776640 1 --2- 172.21.15.98:0/1464649131 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f33a4067ef0 0x7f33a406a3b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f33ac0ae000 tx=0x7f33ac0a7070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:06.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.619+0000 7f33b67fc640 1 -- 172.21.15.98:0/1464649131 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f33b0056190 con 0x7f33b80b7f80 2024-09-19T08:46:06.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.775+0000 7f33c01db640 1 -- 172.21.15.98:0/1464649131 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f33b8156a70 con 0x7f33a4067ef0 2024-09-19T08:46:06.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.795+0000 7f33b67fc640 1 -- 172.21.15.98:0/1464649131 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252414 (secure 0 0 0) 0x7f33b8156a70 con 0x7f33a4067ef0 2024-09-19T08:46:06.795 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:46:06.797 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:46:06.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.799+0000 7f33c01db640 1 -- 172.21.15.98:0/1464649131 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f33a4067ef0 msgr2=0x7f33a406a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:06.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.799+0000 7f33c01db640 1 --2- 172.21.15.98:0/1464649131 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f33a4067ef0 0x7f33a406a3b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f33ac0ae000 tx=0x7f33ac0a7070 comp rx=0 tx=0).stop 2024-09-19T08:46:06.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.799+0000 7f33c01db640 1 -- 172.21.15.98:0/1464649131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33b80b7f80 msgr2=0x7f33b8140520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:06.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.799+0000 7f33c01db640 1 --2- 172.21.15.98:0/1464649131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33b80b7f80 0x7f33b8140520 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f33b000c370 tx=0x7f33b000c830 comp rx=0 tx=0).stop 2024-09-19T08:46:06.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.799+0000 7f33c01db640 1 -- 172.21.15.98:0/1464649131 shutdown_connections 2024-09-19T08:46:06.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.799+0000 7f33c01db640 1 --2- 172.21.15.98:0/1464649131 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f33a4067ef0 0x7f33a406a3b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:06.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.799+0000 7f33c01db640 1 --2- 172.21.15.98:0/1464649131 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33b80d4820 0x7f33b815abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:06.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.799+0000 7f33c01db640 1 --2- 172.21.15.98:0/1464649131 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f33b80b9970 0x7f33b815a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:06.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.799+0000 7f33c01db640 1 --2- 172.21.15.98:0/1464649131 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33b80b7f80 0x7f33b8140520 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:06.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.799+0000 7f33c01db640 1 -- 172.21.15.98:0/1464649131 >> 172.21.15.98:0/1464649131 conn(0x7f33b8074e40 msgr2=0x7f33b8141bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:06.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.799+0000 7f33c01db640 1 -- 172.21.15.98:0/1464649131 shutdown_connections 2024-09-19T08:46:06.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.799+0000 7f33c01db640 1 -- 172.21.15.98:0/1464649131 wait complete. 2024-09-19T08:46:06.812 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2024-09-19T08:46:04.933214+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":268908,"kb_used_data":15240,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754705812,"statfs":{"total":773094113280,"available":772818751488,"internally_reserved":0,"allocated":15605760,"data_stored":7573192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001075"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.852535+0000","last_change":"2024-09-19T08:46:03.845875+0000","last_active":"2024-09-19T08:46:03.852535+0000","last_peered":"2024-09-19T08:46:03.852535+0000","last_clean":"2024-09-19T08:46:03.852535+0000","last_became_active":"2024-09-19T08:46:03.845116+0000","last_became_peered":"2024-09-19T08:46:03.845116+0000","last_unstale":"2024-09-19T08:46:03.852535+0000","last_undegraded":"2024-09-19T08:46:03.852535+0000","last_fullsized":"2024-09-19T08:46:03.852535+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:30.890134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.867942+0000","last_change":"2024-09-19T08:46:03.857639+0000","last_active":"2024-09-19T08:46:03.867942+0000","last_peered":"2024-09-19T08:46:03.867942+0000","last_clean":"2024-09-19T08:46:03.867942+0000","last_became_active":"2024-09-19T08:46:03.856855+0000","last_became_peered":"2024-09-19T08:46:03.856855+0000","last_unstale":"2024-09-19T08:46:03.867942+0000","last_undegraded":"2024-09-19T08:46:03.867942+0000","last_fullsized":"2024-09-19T08:46:03.867942+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:46.185253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.856293+0000","last_change":"2024-09-19T08:46:03.845735+0000","last_active":"2024-09-19T08:46:03.856293+0000","last_peered":"2024-09-19T08:46:03.856293+0000","last_clean":"2024-09-19T08:46:03.856293+0000","last_became_active":"2024-09-19T08:46:03.844420+0000","last_became_peered":"2024-09-19T08:46:03.844420+0000","last_unstale":"2024-09-19T08:46:03.856293+0000","last_undegraded":"2024-09-19T08:46:03.856293+0000","last_fullsized":"2024-09-19T08:46:03.856293+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:22:20.065648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.844155+0000","last_change":"2024-09-19T08:46:03.836280+0000","last_active":"2024-09-19T08:46:03.844155+0000","last_peered":"2024-09-19T08:46:03.844155+0000","last_clean":"2024-09-19T08:46:03.844155+0000","last_became_active":"2024-09-19T08:46:03.836014+0000","last_became_peered":"2024-09-19T08:46:03.836014+0000","last_unstale":"2024-09-19T08:46:03.844155+0000","last_undegraded":"2024-09-19T08:46:03.844155+0000","last_fullsized":"2024-09-19T08:46:03.844155+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:17:21.305649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.868428+0000","last_change":"2024-09-19T08:46:03.864933+0000","last_active":"2024-09-19T08:46:03.868428+0000","last_peered":"2024-09-19T08:46:03.868428+0000","last_clean":"2024-09-19T08:46:03.868428+0000","last_became_active":"2024-09-19T08:46:03.864122+0000","last_became_peered":"2024-09-19T08:46:03.864122+0000","last_unstale":"2024-09-19T08:46:03.868428+0000","last_undegraded":"2024-09-19T08:46:03.868428+0000","last_fullsized":"2024-09-19T08:46:03.868428+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:12.980219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.834242+0000","last_change":"2024-09-19T08:46:03.833166+0000","last_active":"2024-09-19T08:46:03.834242+0000","last_peered":"2024-09-19T08:46:03.834242+0000","last_clean":"2024-09-19T08:46:03.834242+0000","last_became_active":"2024-09-19T08:46:03.832822+0000","last_became_peered":"2024-09-19T08:46:03.832822+0000","last_unstale":"2024-09-19T08:46:03.834242+0000","last_undegraded":"2024-09-19T08:46:03.834242+0000","last_fullsized":"2024-09-19T08:46:03.834242+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:03:29.781045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.860401+0000","last_change":"2024-09-19T08:46:03.859952+0000","last_active":"2024-09-19T08:46:03.860401+0000","last_peered":"2024-09-19T08:46:03.860401+0000","last_clean":"2024-09-19T08:46:03.860401+0000","last_became_active":"2024-09-19T08:46:03.859681+0000","last_became_peered":"2024-09-19T08:46:03.859681+0000","last_unstale":"2024-09-19T08:46:03.860401+0000","last_undegraded":"2024-09-19T08:46:03.860401+0000","last_fullsized":"2024-09-19T08:46:03.860401+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:34.235923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.875411+0000","last_change":"2024-09-19T08:46:03.873085+0000","last_active":"2024-09-19T08:46:03.875411+0000","last_peered":"2024-09-19T08:46:03.875411+0000","last_clean":"2024-09-19T08:46:03.875411+0000","last_became_active":"2024-09-19T08:46:03.871402+0000","last_became_peered":"2024-09-19T08:46:03.871402+0000","last_unstale":"2024-09-19T08:46:03.875411+0000","last_undegraded":"2024-09-19T08:46:03.875411+0000","last_fullsized":"2024-09-19T08:46:03.875411+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:25:08.688723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.868594+0000","last_change":"2024-09-19T08:46:03.858143+0000","last_active":"2024-09-19T08:46:03.868594+0000","last_peered":"2024-09-19T08:46:03.868594+0000","last_clean":"2024-09-19T08:46:03.868594+0000","last_became_active":"2024-09-19T08:46:03.857185+0000","last_became_peered":"2024-09-19T08:46:03.857185+0000","last_unstale":"2024-09-19T08:46:03.868594+0000","last_undegraded":"2024-09-19T08:46:03.868594+0000","last_fullsized":"2024-09-19T08:46:03.868594+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:34:10.594759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.869595+0000","last_change":"2024-09-19T08:46:03.865464+0000","last_active":"2024-09-19T08:46:03.869595+0000","last_peered":"2024-09-19T08:46:03.869595+0000","last_clean":"2024-09-19T08:46:03.869595+0000","last_became_active":"2024-09-19T08:46:03.858453+0000","last_became_peered":"2024-09-19T08:46:03.858453+0000","last_unstale":"2024-09-19T08:46:03.869595+0000","last_undegraded":"2024-09-19T08:46:03.869595+0000","last_fullsized":"2024-09-19T08:46:03.869595+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:21:20.957798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.873065+0000","last_change":"2024-09-19T08:46:03.871081+0000","last_active":"2024-09-19T08:46:03.873065+0000","last_peered":"2024-09-19T08:46:03.873065+0000","last_clean":"2024-09-19T08:46:03.873065+0000","last_became_active":"2024-09-19T08:46:03.870238+0000","last_became_peered":"2024-09-19T08:46:03.870238+0000","last_unstale":"2024-09-19T08:46:03.873065+0000","last_undegraded":"2024-09-19T08:46:03.873065+0000","last_fullsized":"2024-09-19T08:46:03.873065+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:50.999221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.873145+0000","last_change":"2024-09-19T08:46:03.868381+0000","last_active":"2024-09-19T08:46:03.873145+0000","last_peered":"2024-09-19T08:46:03.873145+0000","last_clean":"2024-09-19T08:46:03.873145+0000","last_became_active":"2024-09-19T08:46:03.865474+0000","last_became_peered":"2024-09-19T08:46:03.865474+0000","last_unstale":"2024-09-19T08:46:03.873145+0000","last_undegraded":"2024-09-19T08:46:03.873145+0000","last_fullsized":"2024-09-19T08:46:03.873145+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:49.159131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.851262+0000","last_change":"2024-09-19T08:46:03.841212+0000","last_active":"2024-09-19T08:46:03.851262+0000","last_peered":"2024-09-19T08:46:03.851262+0000","last_clean":"2024-09-19T08:46:03.851262+0000","last_became_active":"2024-09-19T08:46:03.840346+0000","last_became_peered":"2024-09-19T08:46:03.840346+0000","last_unstale":"2024-09-19T08:46:03.851262+0000","last_undegraded":"2024-09-19T08:46:03.851262+0000","last_fullsized":"2024-09-19T08:46:03.851262+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:10:53.999170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.855503+0000","last_change":"2024-09-19T08:46:03.843064+0000","last_active":"2024-09-19T08:46:03.855503+0000","last_peered":"2024-09-19T08:46:03.855503+0000","last_clean":"2024-09-19T08:46:03.855503+0000","last_became_active":"2024-09-19T08:46:03.842288+0000","last_became_peered":"2024-09-19T08:46:03.842288+0000","last_unstale":"2024-09-19T08:46:03.855503+0000","last_undegraded":"2024-09-19T08:46:03.855503+0000","last_fullsized":"2024-09-19T08:46:03.855503+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:04:12.927685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.847444+0000","last_change":"2024-09-19T08:46:03.833833+0000","last_active":"2024-09-19T08:46:03.847444+0000","last_peered":"2024-09-19T08:46:03.847444+0000","last_clean":"2024-09-19T08:46:03.847444+0000","last_became_active":"2024-09-19T08:46:03.833001+0000","last_became_peered":"2024-09-19T08:46:03.833001+0000","last_unstale":"2024-09-19T08:46:03.847444+0000","last_undegraded":"2024-09-19T08:46:03.847444+0000","last_fullsized":"2024-09-19T08:46:03.847444+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:58.873284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.850386+0000","last_change":"2024-09-19T08:46:03.848995+0000","last_active":"2024-09-19T08:46:03.850386+0000","last_peered":"2024-09-19T08:46:03.850386+0000","last_clean":"2024-09-19T08:46:03.850386+0000","last_became_active":"2024-09-19T08:46:03.848517+0000","last_became_peered":"2024-09-19T08:46:03.848517+0000","last_unstale":"2024-09-19T08:46:03.850386+0000","last_undegraded":"2024-09-19T08:46:03.850386+0000","last_fullsized":"2024-09-19T08:46:03.850386+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:17.106192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.850332+0000","last_change":"2024-09-19T08:46:03.849711+0000","last_active":"2024-09-19T08:46:03.850332+0000","last_peered":"2024-09-19T08:46:03.850332+0000","last_clean":"2024-09-19T08:46:03.850332+0000","last_became_active":"2024-09-19T08:46:03.849459+0000","last_became_peered":"2024-09-19T08:46:03.849459+0000","last_unstale":"2024-09-19T08:46:03.850332+0000","last_undegraded":"2024-09-19T08:46:03.850332+0000","last_fullsized":"2024-09-19T08:46:03.850332+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:30:21.090683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.833538+0000","last_change":"2024-09-19T08:46:03.831264+0000","last_active":"2024-09-19T08:46:03.833538+0000","last_peered":"2024-09-19T08:46:03.833538+0000","last_clean":"2024-09-19T08:46:03.833538+0000","last_became_active":"2024-09-19T08:46:03.830966+0000","last_became_peered":"2024-09-19T08:46:03.830966+0000","last_unstale":"2024-09-19T08:46:03.833538+0000","last_undegraded":"2024-09-19T08:46:03.833538+0000","last_fullsized":"2024-09-19T08:46:03.833538+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:15:09.926688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.855564+0000","last_change":"2024-09-19T08:46:03.855087+0000","last_active":"2024-09-19T08:46:03.855564+0000","last_peered":"2024-09-19T08:46:03.855564+0000","last_clean":"2024-09-19T08:46:03.855564+0000","last_became_active":"2024-09-19T08:46:03.854750+0000","last_became_peered":"2024-09-19T08:46:03.854750+0000","last_unstale":"2024-09-19T08:46:03.855564+0000","last_undegraded":"2024-09-19T08:46:03.855564+0000","last_fullsized":"2024-09-19T08:46:03.855564+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:07:19.287192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.851279+0000","last_change":"2024-09-19T08:46:03.840987+0000","last_active":"2024-09-19T08:46:03.851279+0000","last_peered":"2024-09-19T08:46:03.851279+0000","last_clean":"2024-09-19T08:46:03.851279+0000","last_became_active":"2024-09-19T08:46:03.839219+0000","last_became_peered":"2024-09-19T08:46:03.839219+0000","last_unstale":"2024-09-19T08:46:03.851279+0000","last_undegraded":"2024-09-19T08:46:03.851279+0000","last_fullsized":"2024-09-19T08:46:03.851279+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:46.339270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.842992+0000","last_change":"2024-09-19T08:46:03.840744+0000","last_active":"2024-09-19T08:46:03.842992+0000","last_peered":"2024-09-19T08:46:03.842992+0000","last_clean":"2024-09-19T08:46:03.842992+0000","last_became_active":"2024-09-19T08:46:03.840409+0000","last_became_peered":"2024-09-19T08:46:03.840409+0000","last_unstale":"2024-09-19T08:46:03.842992+0000","last_undegraded":"2024-09-19T08:46:03.842992+0000","last_fullsized":"2024-09-19T08:46:03.842992+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:37:12.475418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.868761+0000","last_change":"2024-09-19T08:46:03.865010+0000","last_active":"2024-09-19T08:46:03.868761+0000","last_peered":"2024-09-19T08:46:03.868761+0000","last_clean":"2024-09-19T08:46:03.868761+0000","last_became_active":"2024-09-19T08:46:03.864546+0000","last_became_peered":"2024-09-19T08:46:03.864546+0000","last_unstale":"2024-09-19T08:46:03.868761+0000","last_undegraded":"2024-09-19T08:46:03.868761+0000","last_fullsized":"2024-09-19T08:46:03.868761+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:39:33.905171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.868563+0000","last_change":"2024-09-19T08:46:03.868051+0000","last_active":"2024-09-19T08:46:03.868563+0000","last_peered":"2024-09-19T08:46:03.868563+0000","last_clean":"2024-09-19T08:46:03.868563+0000","last_became_active":"2024-09-19T08:46:03.867232+0000","last_became_peered":"2024-09-19T08:46:03.867232+0000","last_unstale":"2024-09-19T08:46:03.868563+0000","last_undegraded":"2024-09-19T08:46:03.868563+0000","last_fullsized":"2024-09-19T08:46:03.868563+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:33.250058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.873471+0000","last_change":"2024-09-19T08:46:03.867851+0000","last_active":"2024-09-19T08:46:03.873471+0000","last_peered":"2024-09-19T08:46:03.873471+0000","last_clean":"2024-09-19T08:46:03.873471+0000","last_became_active":"2024-09-19T08:46:03.864868+0000","last_became_peered":"2024-09-19T08:46:03.864868+0000","last_unstale":"2024-09-19T08:46:03.873471+0000","last_undegraded":"2024-09-19T08:46:03.873471+0000","last_fullsized":"2024-09-19T08:46:03.873471+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:15:00.785757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.856127+0000","last_change":"2024-09-19T08:46:03.844467+0000","last_active":"2024-09-19T08:46:03.856127+0000","last_peered":"2024-09-19T08:46:03.856127+0000","last_clean":"2024-09-19T08:46:03.856127+0000","last_became_active":"2024-09-19T08:46:03.843735+0000","last_became_peered":"2024-09-19T08:46:03.843735+0000","last_unstale":"2024-09-19T08:46:03.856127+0000","last_undegraded":"2024-09-19T08:46:03.856127+0000","last_fullsized":"2024-09-19T08:46:03.856127+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:47:55.685173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.843339+0000","last_change":"2024-09-19T08:46:03.841631+0000","last_active":"2024-09-19T08:46:03.843339+0000","last_peered":"2024-09-19T08:46:03.843339+0000","last_clean":"2024-09-19T08:46:03.843339+0000","last_became_active":"2024-09-19T08:46:03.841058+0000","last_became_peered":"2024-09-19T08:46:03.841058+0000","last_unstale":"2024-09-19T08:46:03.843339+0000","last_undegraded":"2024-09-19T08:46:03.843339+0000","last_fullsized":"2024-09-19T08:46:03.843339+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:08:38.229382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.856248+0000","last_change":"2024-09-19T08:46:03.845137+0000","last_active":"2024-09-19T08:46:03.856248+0000","last_peered":"2024-09-19T08:46:03.856248+0000","last_clean":"2024-09-19T08:46:03.856248+0000","last_became_active":"2024-09-19T08:46:03.843422+0000","last_became_peered":"2024-09-19T08:46:03.843422+0000","last_unstale":"2024-09-19T08:46:03.856248+0000","last_undegraded":"2024-09-19T08:46:03.856248+0000","last_fullsized":"2024-09-19T08:46:03.856248+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:58:03.888372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.829223+0000","last_change":"2024-09-19T08:46:03.819236+0000","last_active":"2024-09-19T08:46:03.829223+0000","last_peered":"2024-09-19T08:46:03.829223+0000","last_clean":"2024-09-19T08:46:03.829223+0000","last_became_active":"2024-09-19T08:46:03.818523+0000","last_became_peered":"2024-09-19T08:46:03.818523+0000","last_unstale":"2024-09-19T08:46:03.829223+0000","last_undegraded":"2024-09-19T08:46:03.829223+0000","last_fullsized":"2024-09-19T08:46:03.829223+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:20.234366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.856877+0000","last_change":"2024-09-19T08:46:03.843968+0000","last_active":"2024-09-19T08:46:03.856877+0000","last_peered":"2024-09-19T08:46:03.856877+0000","last_clean":"2024-09-19T08:46:03.856877+0000","last_became_active":"2024-09-19T08:46:03.843208+0000","last_became_peered":"2024-09-19T08:46:03.843208+0000","last_unstale":"2024-09-19T08:46:03.856877+0000","last_undegraded":"2024-09-19T08:46:03.856877+0000","last_fullsized":"2024-09-19T08:46:03.856877+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:35:34.439099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.869811+0000","last_change":"2024-09-19T08:46:03.851027+0000","last_active":"2024-09-19T08:46:03.869811+0000","last_peered":"2024-09-19T08:46:03.869811+0000","last_clean":"2024-09-19T08:46:03.869811+0000","last_became_active":"2024-09-19T08:46:03.841235+0000","last_became_peered":"2024-09-19T08:46:03.841235+0000","last_unstale":"2024-09-19T08:46:03.869811+0000","last_undegraded":"2024-09-19T08:46:03.869811+0000","last_fullsized":"2024-09-19T08:46:03.869811+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:15:55.119248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.869011+0000","last_change":"2024-09-19T08:46:03.836883+0000","last_active":"2024-09-19T08:46:03.869011+0000","last_peered":"2024-09-19T08:46:03.869011+0000","last_clean":"2024-09-19T08:46:03.869011+0000","last_became_active":"2024-09-19T08:46:03.836064+0000","last_became_peered":"2024-09-19T08:46:03.836064+0000","last_unstale":"2024-09-19T08:46:03.869011+0000","last_undegraded":"2024-09-19T08:46:03.869011+0000","last_fullsized":"2024-09-19T08:46:03.869011+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:38:51.164522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.833514+0000","last_change":"2024-09-19T08:46:03.831260+0000","last_active":"2024-09-19T08:46:03.833514+0000","last_peered":"2024-09-19T08:46:03.833514+0000","last_clean":"2024-09-19T08:46:03.833514+0000","last_became_active":"2024-09-19T08:46:03.830941+0000","last_became_peered":"2024-09-19T08:46:03.830941+0000","last_unstale":"2024-09-19T08:46:03.833514+0000","last_undegraded":"2024-09-19T08:46:03.833514+0000","last_fullsized":"2024-09-19T08:46:03.833514+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:30.026574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.843361+0000","last_change":"2024-09-19T08:46:03.841782+0000","last_active":"2024-09-19T08:46:03.843361+0000","last_peered":"2024-09-19T08:46:03.843361+0000","last_clean":"2024-09-19T08:46:03.843361+0000","last_became_active":"2024-09-19T08:46:03.841341+0000","last_became_peered":"2024-09-19T08:46:03.841341+0000","last_unstale":"2024-09-19T08:46:03.843361+0000","last_undegraded":"2024-09-19T08:46:03.843361+0000","last_fullsized":"2024-09-19T08:46:03.843361+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:35:24.522000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"145'4","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.861835+0000","last_change":"2024-09-19T08:46:02.822927+0000","last_active":"2024-09-19T08:46:03.861835+0000","last_peered":"2024-09-19T08:46:03.861835+0000","last_clean":"2024-09-19T08:46:03.861835+0000","last_became_active":"2024-09-19T08:46:02.822228+0000","last_became_peered":"2024-09-19T08:46:02.822228+0000","last_unstale":"2024-09-19T08:46:03.861835+0000","last_undegraded":"2024-09-19T08:46:03.861835+0000","last_fullsized":"2024-09-19T08:46:03.861835+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:48:13.108449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.870021+0000","last_change":"2024-09-19T08:46:03.841386+0000","last_active":"2024-09-19T08:46:03.870021+0000","last_peered":"2024-09-19T08:46:03.870021+0000","last_clean":"2024-09-19T08:46:03.870021+0000","last_became_active":"2024-09-19T08:46:03.840366+0000","last_became_peered":"2024-09-19T08:46:03.840366+0000","last_unstale":"2024-09-19T08:46:03.870021+0000","last_undegraded":"2024-09-19T08:46:03.870021+0000","last_fullsized":"2024-09-19T08:46:03.870021+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:06:41.718917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"145'1","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.873109+0000","last_change":"2024-09-19T08:46:02.813607+0000","last_active":"2024-09-19T08:46:03.873109+0000","last_peered":"2024-09-19T08:46:03.873109+0000","last_clean":"2024-09-19T08:46:03.873109+0000","last_became_active":"2024-09-19T08:46:02.812005+0000","last_became_peered":"2024-09-19T08:46:02.812005+0000","last_unstale":"2024-09-19T08:46:03.873109+0000","last_undegraded":"2024-09-19T08:46:03.873109+0000","last_fullsized":"2024-09-19T08:46:03.873109+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:17:20.295044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.849649+0000","last_change":"2024-09-19T08:46:03.849084+0000","last_active":"2024-09-19T08:46:03.849649+0000","last_peered":"2024-09-19T08:46:03.849649+0000","last_clean":"2024-09-19T08:46:03.849649+0000","last_became_active":"2024-09-19T08:46:03.848714+0000","last_became_peered":"2024-09-19T08:46:03.848714+0000","last_unstale":"2024-09-19T08:46:03.849649+0000","last_undegraded":"2024-09-19T08:46:03.849649+0000","last_fullsized":"2024-09-19T08:46:03.849649+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:48:31.188244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"145'2","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.866784+0000","last_change":"2024-09-19T08:46:02.815918+0000","last_active":"2024-09-19T08:46:03.866784+0000","last_peered":"2024-09-19T08:46:03.866784+0000","last_clean":"2024-09-19T08:46:03.866784+0000","last_became_active":"2024-09-19T08:46:02.815594+0000","last_became_peered":"2024-09-19T08:46:02.815594+0000","last_unstale":"2024-09-19T08:46:03.866784+0000","last_undegraded":"2024-09-19T08:46:03.866784+0000","last_fullsized":"2024-09-19T08:46:03.866784+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:21:26.067928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.855184+0000","last_change":"2024-09-19T08:46:03.841447+0000","last_active":"2024-09-19T08:46:03.855184+0000","last_peered":"2024-09-19T08:46:03.855184+0000","last_clean":"2024-09-19T08:46:03.855184+0000","last_became_active":"2024-09-19T08:46:03.839658+0000","last_became_peered":"2024-09-19T08:46:03.839658+0000","last_unstale":"2024-09-19T08:46:03.855184+0000","last_undegraded":"2024-09-19T08:46:03.855184+0000","last_fullsized":"2024-09-19T08:46:03.855184+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:27:01.195371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"145'2","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.849201+0000","last_change":"2024-09-19T08:46:02.808816+0000","last_active":"2024-09-19T08:46:03.849201+0000","last_peered":"2024-09-19T08:46:03.849201+0000","last_clean":"2024-09-19T08:46:03.849201+0000","last_became_active":"2024-09-19T08:46:02.808425+0000","last_became_peered":"2024-09-19T08:46:02.808425+0000","last_unstale":"2024-09-19T08:46:03.849201+0000","last_undegraded":"2024-09-19T08:46:03.849201+0000","last_fullsized":"2024-09-19T08:46:03.849201+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:32:38.306210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.838032+0000","last_change":"2024-09-19T08:46:03.836871+0000","last_active":"2024-09-19T08:46:03.838032+0000","last_peered":"2024-09-19T08:46:03.838032+0000","last_clean":"2024-09-19T08:46:03.838032+0000","last_became_active":"2024-09-19T08:46:03.836207+0000","last_became_peered":"2024-09-19T08:46:03.836207+0000","last_unstale":"2024-09-19T08:46:03.838032+0000","last_undegraded":"2024-09-19T08:46:03.838032+0000","last_fullsized":"2024-09-19T08:46:03.838032+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:38.947995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"145'1","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.871250+0000","last_change":"2024-09-19T08:46:02.826293+0000","last_active":"2024-09-19T08:46:03.871250+0000","last_peered":"2024-09-19T08:46:03.871250+0000","last_clean":"2024-09-19T08:46:03.871250+0000","last_became_active":"2024-09-19T08:46:02.825521+0000","last_became_peered":"2024-09-19T08:46:02.825521+0000","last_unstale":"2024-09-19T08:46:03.871250+0000","last_undegraded":"2024-09-19T08:46:03.871250+0000","last_fullsized":"2024-09-19T08:46:03.871250+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:16:17.879970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.839003+0000","last_change":"2024-09-19T08:46:03.833178+0000","last_active":"2024-09-19T08:46:03.839003+0000","last_peered":"2024-09-19T08:46:03.839003+0000","last_clean":"2024-09-19T08:46:03.839003+0000","last_became_active":"2024-09-19T08:46:03.832950+0000","last_became_peered":"2024-09-19T08:46:03.832950+0000","last_unstale":"2024-09-19T08:46:03.839003+0000","last_undegraded":"2024-09-19T08:46:03.839003+0000","last_fullsized":"2024-09-19T08:46:03.839003+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:36:11.460653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"145'1","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.845297+0000","last_change":"2024-09-19T08:46:02.803300+0000","last_active":"2024-09-19T08:46:03.845297+0000","last_peered":"2024-09-19T08:46:03.845297+0000","last_clean":"2024-09-19T08:46:03.845297+0000","last_became_active":"2024-09-19T08:46:02.802421+0000","last_became_peered":"2024-09-19T08:46:02.802421+0000","last_unstale":"2024-09-19T08:46:03.845297+0000","last_undegraded":"2024-09-19T08:46:03.845297+0000","last_fullsized":"2024-09-19T08:46:03.845297+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:47:47.336303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.838354+0000","last_change":"2024-09-19T08:46:03.830673+0000","last_active":"2024-09-19T08:46:03.838354+0000","last_peered":"2024-09-19T08:46:03.838354+0000","last_clean":"2024-09-19T08:46:03.838354+0000","last_became_active":"2024-09-19T08:46:03.830401+0000","last_became_peered":"2024-09-19T08:46:03.830401+0000","last_unstale":"2024-09-19T08:46:03.838354+0000","last_undegraded":"2024-09-19T08:46:03.838354+0000","last_fullsized":"2024-09-19T08:46:03.838354+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:36:22.883202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.863448+0000","last_change":"2024-09-19T08:46:02.821450+0000","last_active":"2024-09-19T08:46:03.863448+0000","last_peered":"2024-09-19T08:46:03.863448+0000","last_clean":"2024-09-19T08:46:03.863448+0000","last_became_active":"2024-09-19T08:46:02.817925+0000","last_became_peered":"2024-09-19T08:46:02.817925+0000","last_unstale":"2024-09-19T08:46:03.863448+0000","last_undegraded":"2024-09-19T08:46:03.863448+0000","last_fullsized":"2024-09-19T08:46:03.863448+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:45.765147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.834221+0000","last_change":"2024-09-19T08:46:03.833192+0000","last_active":"2024-09-19T08:46:03.834221+0000","last_peered":"2024-09-19T08:46:03.834221+0000","last_clean":"2024-09-19T08:46:03.834221+0000","last_became_active":"2024-09-19T08:46:03.832771+0000","last_became_peered":"2024-09-19T08:46:03.832771+0000","last_unstale":"2024-09-19T08:46:03.834221+0000","last_undegraded":"2024-09-19T08:46:03.834221+0000","last_fullsized":"2024-09-19T08:46:03.834221+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:38:52.918550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.824218+0000","last_change":"2024-09-19T08:46:02.800644+0000","last_active":"2024-09-19T08:46:03.824218+0000","last_peered":"2024-09-19T08:46:03.824218+0000","last_clean":"2024-09-19T08:46:03.824218+0000","last_became_active":"2024-09-19T08:46:02.800326+0000","last_became_peered":"2024-09-19T08:46:02.800326+0000","last_unstale":"2024-09-19T08:46:03.824218+0000","last_undegraded":"2024-09-19T08:46:03.824218+0000","last_fullsized":"2024-09-19T08:46:03.824218+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:31.957926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.843605+0000","last_change":"2024-09-19T08:46:03.839842+0000","last_active":"2024-09-19T08:46:03.843605+0000","last_peered":"2024-09-19T08:46:03.843605+0000","last_clean":"2024-09-19T08:46:03.843605+0000","last_became_active":"2024-09-19T08:46:03.839245+0000","last_became_peered":"2024-09-19T08:46:03.839245+0000","last_unstale":"2024-09-19T08:46:03.843605+0000","last_undegraded":"2024-09-19T08:46:03.843605+0000","last_fullsized":"2024-09-19T08:46:03.843605+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:40:00.849087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.817786+0000","last_change":"2024-09-19T08:46:02.809804+0000","last_active":"2024-09-19T08:46:03.817786+0000","last_peered":"2024-09-19T08:46:03.817786+0000","last_clean":"2024-09-19T08:46:03.817786+0000","last_became_active":"2024-09-19T08:46:02.809517+0000","last_became_peered":"2024-09-19T08:46:02.809517+0000","last_unstale":"2024-09-19T08:46:03.817786+0000","last_undegraded":"2024-09-19T08:46:03.817786+0000","last_fullsized":"2024-09-19T08:46:03.817786+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:34:37.887250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.843494+0000","last_change":"2024-09-19T08:46:03.840107+0000","last_active":"2024-09-19T08:46:03.843494+0000","last_peered":"2024-09-19T08:46:03.843494+0000","last_clean":"2024-09-19T08:46:03.843494+0000","last_became_active":"2024-09-19T08:46:03.839663+0000","last_became_peered":"2024-09-19T08:46:03.839663+0000","last_unstale":"2024-09-19T08:46:03.843494+0000","last_undegraded":"2024-09-19T08:46:03.843494+0000","last_fullsized":"2024-09-19T08:46:03.843494+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:31:22.260734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"145'3","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.848880+0000","last_change":"2024-09-19T08:46:02.811541+0000","last_active":"2024-09-19T08:46:03.848880+0000","last_peered":"2024-09-19T08:46:03.848880+0000","last_clean":"2024-09-19T08:46:03.848880+0000","last_became_active":"2024-09-19T08:46:02.811241+0000","last_became_peered":"2024-09-19T08:46:02.811241+0000","last_unstale":"2024-09-19T08:46:03.848880+0000","last_undegraded":"2024-09-19T08:46:03.848880+0000","last_fullsized":"2024-09-19T08:46:03.848880+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:47:25.008234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.869974+0000","last_change":"2024-09-19T08:46:03.852952+0000","last_active":"2024-09-19T08:46:03.869974+0000","last_peered":"2024-09-19T08:46:03.869974+0000","last_clean":"2024-09-19T08:46:03.869974+0000","last_became_active":"2024-09-19T08:46:03.851921+0000","last_became_peered":"2024-09-19T08:46:03.851921+0000","last_unstale":"2024-09-19T08:46:03.869974+0000","last_undegraded":"2024-09-19T08:46:03.869974+0000","last_fullsized":"2024-09-19T08:46:03.869974+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:00:27.357328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"145'6","reported_seq":18,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.880589+0000","last_change":"2024-09-19T08:46:02.817203+0000","last_active":"2024-09-19T08:46:03.880589+0000","last_peered":"2024-09-19T08:46:03.880589+0000","last_clean":"2024-09-19T08:46:03.880589+0000","last_became_active":"2024-09-19T08:46:02.816489+0000","last_became_peered":"2024-09-19T08:46:02.816489+0000","last_unstale":"2024-09-19T08:46:03.880589+0000","last_undegraded":"2024-09-19T08:46:03.880589+0000","last_fullsized":"2024-09-19T08:46:03.880589+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:54:20.604808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.863991+0000","last_change":"2024-09-19T08:46:03.863261+0000","last_active":"2024-09-19T08:46:03.863991+0000","last_peered":"2024-09-19T08:46:03.863991+0000","last_clean":"2024-09-19T08:46:03.863991+0000","last_became_active":"2024-09-19T08:46:03.862984+0000","last_became_peered":"2024-09-19T08:46:03.862984+0000","last_unstale":"2024-09-19T08:46:03.863991+0000","last_undegraded":"2024-09-19T08:46:03.863991+0000","last_fullsized":"2024-09-19T08:46:03.863991+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:46:13.616664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"145'2","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.845674+0000","last_change":"2024-09-19T08:46:02.797025+0000","last_active":"2024-09-19T08:46:03.845674+0000","last_peered":"2024-09-19T08:46:03.845674+0000","last_clean":"2024-09-19T08:46:03.845674+0000","last_became_active":"2024-09-19T08:46:02.796737+0000","last_became_peered":"2024-09-19T08:46:02.796737+0000","last_unstale":"2024-09-19T08:46:03.845674+0000","last_undegraded":"2024-09-19T08:46:03.845674+0000","last_fullsized":"2024-09-19T08:46:03.845674+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:46:40.639107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.850682+0000","last_change":"2024-09-19T08:46:03.850150+0000","last_active":"2024-09-19T08:46:03.850682+0000","last_peered":"2024-09-19T08:46:03.850682+0000","last_clean":"2024-09-19T08:46:03.850682+0000","last_became_active":"2024-09-19T08:46:03.849815+0000","last_became_peered":"2024-09-19T08:46:03.849815+0000","last_unstale":"2024-09-19T08:46:03.850682+0000","last_undegraded":"2024-09-19T08:46:03.850682+0000","last_fullsized":"2024-09-19T08:46:03.850682+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:48:14.119055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.828228+0000","last_change":"2024-09-19T08:46:02.805022+0000","last_active":"2024-09-19T08:46:03.828228+0000","last_peered":"2024-09-19T08:46:03.828228+0000","last_clean":"2024-09-19T08:46:03.828228+0000","last_became_active":"2024-09-19T08:46:02.804472+0000","last_became_peered":"2024-09-19T08:46:02.804472+0000","last_unstale":"2024-09-19T08:46:03.828228+0000","last_undegraded":"2024-09-19T08:46:03.828228+0000","last_fullsized":"2024-09-19T08:46:03.828228+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:02:38.809125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.844364+0000","last_change":"2024-09-19T08:46:03.838208+0000","last_active":"2024-09-19T08:46:03.844364+0000","last_peered":"2024-09-19T08:46:03.844364+0000","last_clean":"2024-09-19T08:46:03.844364+0000","last_became_active":"2024-09-19T08:46:03.837720+0000","last_became_peered":"2024-09-19T08:46:03.837720+0000","last_unstale":"2024-09-19T08:46:03.844364+0000","last_undegraded":"2024-09-19T08:46:03.844364+0000","last_fullsized":"2024-09-19T08:46:03.844364+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:54:21.615414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.811429+0000","last_change":"2024-09-19T08:46:02.816068+0000","last_active":"2024-09-19T08:46:03.811429+0000","last_peered":"2024-09-19T08:46:03.811429+0000","last_clean":"2024-09-19T08:46:03.811429+0000","last_became_active":"2024-09-19T08:46:02.815686+0000","last_became_peered":"2024-09-19T08:46:02.815686+0000","last_unstale":"2024-09-19T08:46:03.811429+0000","last_undegraded":"2024-09-19T08:46:03.811429+0000","last_fullsized":"2024-09-19T08:46:03.811429+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:41.616069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.855564+0000","last_change":"2024-09-19T08:46:03.841385+0000","last_active":"2024-09-19T08:46:03.855564+0000","last_peered":"2024-09-19T08:46:03.855564+0000","last_clean":"2024-09-19T08:46:03.855564+0000","last_became_active":"2024-09-19T08:46:03.840602+0000","last_became_peered":"2024-09-19T08:46:03.840602+0000","last_unstale":"2024-09-19T08:46:03.855564+0000","last_undegraded":"2024-09-19T08:46:03.855564+0000","last_fullsized":"2024-09-19T08:46:03.855564+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:32:21.077746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"145'8","reported_seq":20,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.878810+0000","last_change":"2024-09-19T08:46:02.818995+0000","last_active":"2024-09-19T08:46:03.878810+0000","last_peered":"2024-09-19T08:46:03.878810+0000","last_clean":"2024-09-19T08:46:03.878810+0000","last_became_active":"2024-09-19T08:46:02.818302+0000","last_became_peered":"2024-09-19T08:46:02.818302+0000","last_unstale":"2024-09-19T08:46:03.878810+0000","last_undegraded":"2024-09-19T08:46:03.878810+0000","last_fullsized":"2024-09-19T08:46:03.878810+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:06:21.720276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.838274+0000","last_change":"2024-09-19T08:46:03.829615+0000","last_active":"2024-09-19T08:46:03.838274+0000","last_peered":"2024-09-19T08:46:03.838274+0000","last_clean":"2024-09-19T08:46:03.838274+0000","last_became_active":"2024-09-19T08:46:03.829337+0000","last_became_peered":"2024-09-19T08:46:03.829337+0000","last_unstale":"2024-09-19T08:46:03.838274+0000","last_undegraded":"2024-09-19T08:46:03.838274+0000","last_fullsized":"2024-09-19T08:46:03.838274+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:53.916495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"145'1","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.869335+0000","last_change":"2024-09-19T08:46:02.826073+0000","last_active":"2024-09-19T08:46:03.869335+0000","last_peered":"2024-09-19T08:46:03.869335+0000","last_clean":"2024-09-19T08:46:03.869335+0000","last_became_active":"2024-09-19T08:46:02.825423+0000","last_became_peered":"2024-09-19T08:46:02.825423+0000","last_unstale":"2024-09-19T08:46:03.869335+0000","last_undegraded":"2024-09-19T08:46:03.869335+0000","last_fullsized":"2024-09-19T08:46:03.869335+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:52.019014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.869380+0000","last_change":"2024-09-19T08:46:03.868989+0000","last_active":"2024-09-19T08:46:03.869380+0000","last_peered":"2024-09-19T08:46:03.869380+0000","last_clean":"2024-09-19T08:46:03.869380+0000","last_became_active":"2024-09-19T08:46:03.868667+0000","last_became_peered":"2024-09-19T08:46:03.868667+0000","last_unstale":"2024-09-19T08:46:03.869380+0000","last_undegraded":"2024-09-19T08:46:03.869380+0000","last_fullsized":"2024-09-19T08:46:03.869380+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:26.080637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"145'2","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.857068+0000","last_change":"2024-09-19T08:46:02.812569+0000","last_active":"2024-09-19T08:46:03.857068+0000","last_peered":"2024-09-19T08:46:03.857068+0000","last_clean":"2024-09-19T08:46:03.857068+0000","last_became_active":"2024-09-19T08:46:02.811674+0000","last_became_peered":"2024-09-19T08:46:02.811674+0000","last_unstale":"2024-09-19T08:46:03.857068+0000","last_undegraded":"2024-09-19T08:46:03.857068+0000","last_fullsized":"2024-09-19T08:46:03.857068+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:47.191119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.839031+0000","last_change":"2024-09-19T08:46:03.833176+0000","last_active":"2024-09-19T08:46:03.839031+0000","last_peered":"2024-09-19T08:46:03.839031+0000","last_clean":"2024-09-19T08:46:03.839031+0000","last_became_active":"2024-09-19T08:46:03.832906+0000","last_became_peered":"2024-09-19T08:46:03.832906+0000","last_unstale":"2024-09-19T08:46:03.839031+0000","last_undegraded":"2024-09-19T08:46:03.839031+0000","last_fullsized":"2024-09-19T08:46:03.839031+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:12.221337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"145'1","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.874149+0000","last_change":"2024-09-19T08:46:02.826237+0000","last_active":"2024-09-19T08:46:03.874149+0000","last_peered":"2024-09-19T08:46:03.874149+0000","last_clean":"2024-09-19T08:46:03.874149+0000","last_became_active":"2024-09-19T08:46:02.825550+0000","last_became_peered":"2024-09-19T08:46:02.825550+0000","last_unstale":"2024-09-19T08:46:03.874149+0000","last_undegraded":"2024-09-19T08:46:03.874149+0000","last_fullsized":"2024-09-19T08:46:03.874149+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:33.225318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.868273+0000","last_change":"2024-09-19T08:46:03.867671+0000","last_active":"2024-09-19T08:46:03.868273+0000","last_peered":"2024-09-19T08:46:03.868273+0000","last_clean":"2024-09-19T08:46:03.868273+0000","last_became_active":"2024-09-19T08:46:03.867346+0000","last_became_peered":"2024-09-19T08:46:03.867346+0000","last_unstale":"2024-09-19T08:46:03.868273+0000","last_undegraded":"2024-09-19T08:46:03.868273+0000","last_fullsized":"2024-09-19T08:46:03.868273+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:53.972805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.821153+0000","last_change":"2024-09-19T08:46:02.811610+0000","last_active":"2024-09-19T08:46:03.821153+0000","last_peered":"2024-09-19T08:46:03.821153+0000","last_clean":"2024-09-19T08:46:03.821153+0000","last_became_active":"2024-09-19T08:46:02.810589+0000","last_became_peered":"2024-09-19T08:46:02.810589+0000","last_unstale":"2024-09-19T08:46:03.821153+0000","last_undegraded":"2024-09-19T08:46:03.821153+0000","last_fullsized":"2024-09-19T08:46:03.821153+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:44:40.401216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.851355+0000","last_change":"2024-09-19T08:46:03.839733+0000","last_active":"2024-09-19T08:46:03.851355+0000","last_peered":"2024-09-19T08:46:03.851355+0000","last_clean":"2024-09-19T08:46:03.851355+0000","last_became_active":"2024-09-19T08:46:03.839000+0000","last_became_peered":"2024-09-19T08:46:03.839000+0000","last_unstale":"2024-09-19T08:46:03.851355+0000","last_undegraded":"2024-09-19T08:46:03.851355+0000","last_fullsized":"2024-09-19T08:46:03.851355+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:55.928756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"145'2","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.866597+0000","last_change":"2024-09-19T08:46:02.815895+0000","last_active":"2024-09-19T08:46:03.866597+0000","last_peered":"2024-09-19T08:46:03.866597+0000","last_clean":"2024-09-19T08:46:03.866597+0000","last_became_active":"2024-09-19T08:46:02.815528+0000","last_became_peered":"2024-09-19T08:46:02.815528+0000","last_unstale":"2024-09-19T08:46:03.866597+0000","last_undegraded":"2024-09-19T08:46:03.866597+0000","last_fullsized":"2024-09-19T08:46:03.866597+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:34:05.913711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.850382+0000","last_change":"2024-09-19T08:46:03.848970+0000","last_active":"2024-09-19T08:46:03.850382+0000","last_peered":"2024-09-19T08:46:03.850382+0000","last_clean":"2024-09-19T08:46:03.850382+0000","last_became_active":"2024-09-19T08:46:03.848492+0000","last_became_peered":"2024-09-19T08:46:03.848492+0000","last_unstale":"2024-09-19T08:46:03.850382+0000","last_undegraded":"2024-09-19T08:46:03.850382+0000","last_fullsized":"2024-09-19T08:46:03.850382+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:30.084530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.822483+0000","last_change":"2024-09-19T08:46:02.810879+0000","last_active":"2024-09-19T08:46:03.822483+0000","last_peered":"2024-09-19T08:46:03.822483+0000","last_clean":"2024-09-19T08:46:03.822483+0000","last_became_active":"2024-09-19T08:46:02.810110+0000","last_became_peered":"2024-09-19T08:46:02.810110+0000","last_unstale":"2024-09-19T08:46:03.822483+0000","last_undegraded":"2024-09-19T08:46:03.822483+0000","last_fullsized":"2024-09-19T08:46:03.822483+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:52.962200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.852620+0000","last_change":"2024-09-19T08:46:03.841458+0000","last_active":"2024-09-19T08:46:03.852620+0000","last_peered":"2024-09-19T08:46:03.852620+0000","last_clean":"2024-09-19T08:46:03.852620+0000","last_became_active":"2024-09-19T08:46:03.840677+0000","last_became_peered":"2024-09-19T08:46:03.840677+0000","last_unstale":"2024-09-19T08:46:03.852620+0000","last_undegraded":"2024-09-19T08:46:03.852620+0000","last_fullsized":"2024-09-19T08:46:03.852620+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:44:45.116396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.809975+0000","last_change":"2024-09-19T08:46:02.802952+0000","last_active":"2024-09-19T08:46:03.809975+0000","last_peered":"2024-09-19T08:46:03.809975+0000","last_clean":"2024-09-19T08:46:03.809975+0000","last_became_active":"2024-09-19T08:46:02.802520+0000","last_became_peered":"2024-09-19T08:46:02.802520+0000","last_unstale":"2024-09-19T08:46:03.809975+0000","last_undegraded":"2024-09-19T08:46:03.809975+0000","last_fullsized":"2024-09-19T08:46:03.809975+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:51.359353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.868173+0000","last_change":"2024-09-19T08:46:03.867618+0000","last_active":"2024-09-19T08:46:03.868173+0000","last_peered":"2024-09-19T08:46:03.868173+0000","last_clean":"2024-09-19T08:46:03.868173+0000","last_became_active":"2024-09-19T08:46:03.867224+0000","last_became_peered":"2024-09-19T08:46:03.867224+0000","last_unstale":"2024-09-19T08:46:03.868173+0000","last_undegraded":"2024-09-19T08:46:03.868173+0000","last_fullsized":"2024-09-19T08:46:03.868173+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:25.863652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.824437+0000","last_change":"2024-09-19T08:46:02.824281+0000","last_active":"2024-09-19T08:46:03.824437+0000","last_peered":"2024-09-19T08:46:03.824437+0000","last_clean":"2024-09-19T08:46:03.824437+0000","last_became_active":"2024-09-19T08:46:02.823571+0000","last_became_peered":"2024-09-19T08:46:02.823571+0000","last_unstale":"2024-09-19T08:46:03.824437+0000","last_undegraded":"2024-09-19T08:46:03.824437+0000","last_fullsized":"2024-09-19T08:46:03.824437+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:21:09.769250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.843446+0000","last_change":"2024-09-19T08:46:03.840107+0000","last_active":"2024-09-19T08:46:03.843446+0000","last_peered":"2024-09-19T08:46:03.843446+0000","last_clean":"2024-09-19T08:46:03.843446+0000","last_became_active":"2024-09-19T08:46:03.839530+0000","last_became_peered":"2024-09-19T08:46:03.839530+0000","last_unstale":"2024-09-19T08:46:03.843446+0000","last_undegraded":"2024-09-19T08:46:03.843446+0000","last_fullsized":"2024-09-19T08:46:03.843446+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:54:55.799914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"145'2","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.866837+0000","last_change":"2024-09-19T08:46:02.820093+0000","last_active":"2024-09-19T08:46:03.866837+0000","last_peered":"2024-09-19T08:46:03.866837+0000","last_clean":"2024-09-19T08:46:03.866837+0000","last_became_active":"2024-09-19T08:46:02.819624+0000","last_became_peered":"2024-09-19T08:46:02.819624+0000","last_unstale":"2024-09-19T08:46:03.866837+0000","last_undegraded":"2024-09-19T08:46:03.866837+0000","last_fullsized":"2024-09-19T08:46:03.866837+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:34:18.769816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.863858+0000","last_change":"2024-09-19T08:46:03.863241+0000","last_active":"2024-09-19T08:46:03.863858+0000","last_peered":"2024-09-19T08:46:03.863858+0000","last_clean":"2024-09-19T08:46:03.863858+0000","last_became_active":"2024-09-19T08:46:03.862946+0000","last_became_peered":"2024-09-19T08:46:03.862946+0000","last_unstale":"2024-09-19T08:46:03.863858+0000","last_undegraded":"2024-09-19T08:46:03.863858+0000","last_fullsized":"2024-09-19T08:46:03.863858+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:44:41.411821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.828759+0000","last_change":"2024-09-19T08:46:02.817930+0000","last_active":"2024-09-19T08:46:03.828759+0000","last_peered":"2024-09-19T08:46:03.828759+0000","last_clean":"2024-09-19T08:46:03.828759+0000","last_became_active":"2024-09-19T08:46:02.817157+0000","last_became_peered":"2024-09-19T08:46:02.817157+0000","last_unstale":"2024-09-19T08:46:03.828759+0000","last_undegraded":"2024-09-19T08:46:03.828759+0000","last_fullsized":"2024-09-19T08:46:03.828759+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:44:34.154817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.842880+0000","last_change":"2024-09-19T08:46:03.839805+0000","last_active":"2024-09-19T08:46:03.842880+0000","last_peered":"2024-09-19T08:46:03.842880+0000","last_clean":"2024-09-19T08:46:03.842880+0000","last_became_active":"2024-09-19T08:46:03.839507+0000","last_became_peered":"2024-09-19T08:46:03.839507+0000","last_unstale":"2024-09-19T08:46:03.842880+0000","last_undegraded":"2024-09-19T08:46:03.842880+0000","last_fullsized":"2024-09-19T08:46:03.842880+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:38.700161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.827464+0000","last_change":"2024-09-19T08:46:02.822328+0000","last_active":"2024-09-19T08:46:03.827464+0000","last_peered":"2024-09-19T08:46:03.827464+0000","last_clean":"2024-09-19T08:46:03.827464+0000","last_became_active":"2024-09-19T08:46:02.819128+0000","last_became_peered":"2024-09-19T08:46:02.819128+0000","last_unstale":"2024-09-19T08:46:03.827464+0000","last_undegraded":"2024-09-19T08:46:03.827464+0000","last_fullsized":"2024-09-19T08:46:03.827464+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:32.043000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.835392+0000","last_change":"2024-09-19T08:46:03.832092+0000","last_active":"2024-09-19T08:46:03.835392+0000","last_peered":"2024-09-19T08:46:03.835392+0000","last_clean":"2024-09-19T08:46:03.835392+0000","last_became_active":"2024-09-19T08:46:03.831768+0000","last_became_peered":"2024-09-19T08:46:03.831768+0000","last_unstale":"2024-09-19T08:46:03.835392+0000","last_undegraded":"2024-09-19T08:46:03.835392+0000","last_fullsized":"2024-09-19T08:46:03.835392+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:10.287914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.829511+0000","last_change":"2024-09-19T08:46:02.819293+0000","last_active":"2024-09-19T08:46:03.829511+0000","last_peered":"2024-09-19T08:46:03.829511+0000","last_clean":"2024-09-19T08:46:03.829511+0000","last_became_active":"2024-09-19T08:46:02.819040+0000","last_became_peered":"2024-09-19T08:46:02.819040+0000","last_unstale":"2024-09-19T08:46:03.829511+0000","last_undegraded":"2024-09-19T08:46:03.829511+0000","last_fullsized":"2024-09-19T08:46:03.829511+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:58.244723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.835442+0000","last_change":"2024-09-19T08:46:03.831610+0000","last_active":"2024-09-19T08:46:03.835442+0000","last_peered":"2024-09-19T08:46:03.835442+0000","last_clean":"2024-09-19T08:46:03.835442+0000","last_became_active":"2024-09-19T08:46:03.831233+0000","last_became_peered":"2024-09-19T08:46:03.831233+0000","last_unstale":"2024-09-19T08:46:03.835442+0000","last_undegraded":"2024-09-19T08:46:03.835442+0000","last_fullsized":"2024-09-19T08:46:03.835442+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:17:40.995948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.813954+0000","last_change":"2024-09-19T08:46:02.813739+0000","last_active":"2024-09-19T08:46:03.813954+0000","last_peered":"2024-09-19T08:46:03.813954+0000","last_clean":"2024-09-19T08:46:03.813954+0000","last_became_active":"2024-09-19T08:46:02.805993+0000","last_became_peered":"2024-09-19T08:46:02.805993+0000","last_unstale":"2024-09-19T08:46:03.813954+0000","last_undegraded":"2024-09-19T08:46:03.813954+0000","last_fullsized":"2024-09-19T08:46:03.813954+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:42.876907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.869362+0000","last_change":"2024-09-19T08:46:03.836909+0000","last_active":"2024-09-19T08:46:03.869362+0000","last_peered":"2024-09-19T08:46:03.869362+0000","last_clean":"2024-09-19T08:46:03.869362+0000","last_became_active":"2024-09-19T08:46:03.836141+0000","last_became_peered":"2024-09-19T08:46:03.836141+0000","last_unstale":"2024-09-19T08:46:03.869362+0000","last_undegraded":"2024-09-19T08:46:03.869362+0000","last_fullsized":"2024-09-19T08:46:03.869362+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:06:59.971302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.809591+0000","last_change":"2024-09-19T08:46:02.803230+0000","last_active":"2024-09-19T08:46:03.809591+0000","last_peered":"2024-09-19T08:46:03.809591+0000","last_clean":"2024-09-19T08:46:03.809591+0000","last_became_active":"2024-09-19T08:46:02.798750+0000","last_became_peered":"2024-09-19T08:46:02.798750+0000","last_unstale":"2024-09-19T08:46:03.809591+0000","last_undegraded":"2024-09-19T08:46:03.809591+0000","last_fullsized":"2024-09-19T08:46:03.809591+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:49.796040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.841978+0000","last_change":"2024-09-19T08:46:03.840788+0000","last_active":"2024-09-19T08:46:03.841978+0000","last_peered":"2024-09-19T08:46:03.841978+0000","last_clean":"2024-09-19T08:46:03.841978+0000","last_became_active":"2024-09-19T08:46:03.840509+0000","last_became_peered":"2024-09-19T08:46:03.840509+0000","last_unstale":"2024-09-19T08:46:03.841978+0000","last_undegraded":"2024-09-19T08:46:03.841978+0000","last_fullsized":"2024-09-19T08:46:03.841978+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:42.050789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.863445+0000","last_change":"2024-09-19T08:46:02.815898+0000","last_active":"2024-09-19T08:46:03.863445+0000","last_peered":"2024-09-19T08:46:03.863445+0000","last_clean":"2024-09-19T08:46:03.863445+0000","last_became_active":"2024-09-19T08:46:02.810982+0000","last_became_peered":"2024-09-19T08:46:02.810982+0000","last_unstale":"2024-09-19T08:46:03.863445+0000","last_undegraded":"2024-09-19T08:46:03.863445+0000","last_fullsized":"2024-09-19T08:46:03.863445+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:10.764379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.851193+0000","last_change":"2024-09-19T08:46:03.839794+0000","last_active":"2024-09-19T08:46:03.851193+0000","last_peered":"2024-09-19T08:46:03.851193+0000","last_clean":"2024-09-19T08:46:03.851193+0000","last_became_active":"2024-09-19T08:46:03.838903+0000","last_became_peered":"2024-09-19T08:46:03.838903+0000","last_unstale":"2024-09-19T08:46:03.851193+0000","last_undegraded":"2024-09-19T08:46:03.851193+0000","last_fullsized":"2024-09-19T08:46:03.851193+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:51:14.062744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.814989+0000","last_change":"2024-09-19T08:46:02.815409+0000","last_active":"2024-09-19T08:46:03.814989+0000","last_peered":"2024-09-19T08:46:03.814989+0000","last_clean":"2024-09-19T08:46:03.814989+0000","last_became_active":"2024-09-19T08:46:02.814663+0000","last_became_peered":"2024-09-19T08:46:02.814663+0000","last_unstale":"2024-09-19T08:46:03.814989+0000","last_undegraded":"2024-09-19T08:46:03.814989+0000","last_fullsized":"2024-09-19T08:46:03.814989+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:00.169631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.843468+0000","last_change":"2024-09-19T08:46:03.840754+0000","last_active":"2024-09-19T08:46:03.843468+0000","last_peered":"2024-09-19T08:46:03.843468+0000","last_clean":"2024-09-19T08:46:03.843468+0000","last_became_active":"2024-09-19T08:46:03.839653+0000","last_became_peered":"2024-09-19T08:46:03.839653+0000","last_unstale":"2024-09-19T08:46:03.843468+0000","last_undegraded":"2024-09-19T08:46:03.843468+0000","last_fullsized":"2024-09-19T08:46:03.843468+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:47.007104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.814063+0000","last_change":"2024-09-19T08:46:02.803881+0000","last_active":"2024-09-19T08:46:03.814063+0000","last_peered":"2024-09-19T08:46:03.814063+0000","last_clean":"2024-09-19T08:46:03.814063+0000","last_became_active":"2024-09-19T08:46:02.802876+0000","last_became_peered":"2024-09-19T08:46:02.802876+0000","last_unstale":"2024-09-19T08:46:03.814063+0000","last_undegraded":"2024-09-19T08:46:03.814063+0000","last_fullsized":"2024-09-19T08:46:03.814063+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:19:13.207333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"71'128","reported_seq":433,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.824575+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:46:03.824575+0000","last_peered":"2024-09-19T08:46:03.824575+0000","last_clean":"2024-09-19T08:46:03.824575+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:46:03.824575+0000","last_undegraded":"2024-09-19T08:46:03.824575+0000","last_fullsized":"2024-09-19T08:46:03.824575+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509538,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33528,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338312,"statfs":{"total":96636764160,"available":96602431488,"internally_reserved":0,"allocated":2396160,"data_stored":1389425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574947,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32952,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338888,"statfs":{"total":96636764160,"available":96603021312,"internally_reserved":0,"allocated":1802240,"data_stored":799057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509540,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32948,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338892,"statfs":{"total":96636764160,"available":96603025408,"internally_reserved":0,"allocated":1802240,"data_stored":799057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509540,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32948,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338892,"statfs":{"total":96636764160,"available":96603025408,"internally_reserved":0,"allocated":1802240,"data_stored":799057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542242,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37108,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334732,"statfs":{"total":96636764160,"available":96598765568,"internally_reserved":0,"allocated":1802240,"data_stored":799057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509537,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32948,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338892,"statfs":{"total":96636764160,"available":96603025408,"internally_reserved":0,"allocated":1802240,"data_stored":799057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509539,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32948,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338892,"statfs":{"total":96636764160,"available":96603025408,"internally_reserved":0,"allocated":1802240,"data_stored":799057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509537,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33528,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338312,"statfs":{"total":96636764160,"available":96602431488,"internally_reserved":0,"allocated":2396160,"data_stored":1389425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:46:06.816 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:46:06.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.967+0000 7fa811d3a640 1 Processor -- start 2024-09-19T08:46:06.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.967+0000 7fa811d3a640 1 -- start start 2024-09-19T08:46:06.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.967+0000 7fa811d3a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa80c15c550 0x7fa80c06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:06.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.967+0000 7fa811d3a640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa80c06d310 con 0x7fa80c0cfad0 2024-09-19T08:46:06.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.967+0000 7fa811d3a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa80c06d5f0 con 0x7fa80c06c1d0 2024-09-19T08:46:06.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.967+0000 7fa811d3a640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fa80c06c640 con 0x7fa80c15c550 2024-09-19T08:46:06.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.967+0000 7fa80b7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa80c15c550 0x7fa80c06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:06.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.967+0000 7fa80bfff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fa80c0cfad0 0x7fa80c0723a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42496/0 (socket says 172.21.15.98:42496) 2024-09-19T08:46:06.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.967+0000 7fa80b7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa80c15c550 0x7fa80c06bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46896/0 (socket says 172.21.15.98:46896) 2024-09-19T08:46:06.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.967+0000 7fa80bfff640 1 -- 172.21.15.98:0/174538423 learned_addr learned my addr 172.21.15.98:0/174538423 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:06.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.967+0000 7fa80b7fe640 1 -- 172.21.15.98:0/174538423 >> v1:172.21.15.139:6789/0 conn(0x7fa80c06c1d0 legacy=0x7fa80c0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:06.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.967+0000 7fa80b7fe640 1 -- 172.21.15.98:0/174538423 >> v1:172.21.15.98:6789/0 conn(0x7fa80c0cfad0 legacy=0x7fa80c0723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:06.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.967+0000 7fa80b7fe640 1 -- 172.21.15.98:0/174538423 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa80c06d8d0 con 0x7fa80c15c550 2024-09-19T08:46:06.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.971+0000 7fa80b7fe640 1 --2- 172.21.15.98:0/174538423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa80c15c550 0x7fa80c06bc90 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fa8000097c0 tx=0x7fa80001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a088e214734447b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:06.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.971+0000 7fa80a7fc640 1 -- 172.21.15.98:0/174538423 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8000b0070 con 0x7fa80c15c550 2024-09-19T08:46:06.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.971+0000 7fa80a7fc640 1 -- 172.21.15.98:0/174538423 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8000ab600 con 0x7fa80c15c550 2024-09-19T08:46:06.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.971+0000 7fa80a7fc640 1 -- 172.21.15.98:0/174538423 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8000ab9d0 con 0x7fa80c15c550 2024-09-19T08:46:06.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.971+0000 7fa811d3a640 1 -- 172.21.15.98:0/174538423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa80c15c550 msgr2=0x7fa80c06bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:06.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.971+0000 7fa811d3a640 1 --2- 172.21.15.98:0/174538423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa80c15c550 0x7fa80c06bc90 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fa8000097c0 tx=0x7fa80001c460 comp rx=0 tx=0).stop 2024-09-19T08:46:06.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.971+0000 7fa811d3a640 1 -- 172.21.15.98:0/174538423 shutdown_connections 2024-09-19T08:46:06.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.971+0000 7fa811d3a640 1 --2- 172.21.15.98:0/174538423 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa80c15c550 0x7fa80c06bc90 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:06.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.971+0000 7fa811d3a640 1 -- 172.21.15.98:0/174538423 >> 172.21.15.98:0/174538423 conn(0x7fa80c0b2010 msgr2=0x7fa80c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:06.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.971+0000 7fa811d3a640 1 -- 172.21.15.98:0/174538423 shutdown_connections 2024-09-19T08:46:06.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.971+0000 7fa811d3a640 1 -- 172.21.15.98:0/174538423 wait complete. 2024-09-19T08:46:06.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.971+0000 7fa811d3a640 1 Processor -- start 2024-09-19T08:46:06.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.971+0000 7fa811d3a640 1 -- start start 2024-09-19T08:46:06.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.971+0000 7fa811d3a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa80c06c1d0 0x7fa80c146720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:06.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.971+0000 7fa811d3a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa80c0cfad0 0x7fa80c135210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:06.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.971+0000 7fa811d3a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa80c15c550 0x7fa80c153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:06.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.971+0000 7fa811d3a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c073690 con 0x7fa80c15c550 2024-09-19T08:46:06.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.971+0000 7fa811d3a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c073510 con 0x7fa80c06c1d0 2024-09-19T08:46:06.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.971+0000 7fa811d3a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa80c16a240 con 0x7fa80c0cfad0 2024-09-19T08:46:06.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.971+0000 7fa80bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa80c15c550 0x7fa80c153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:06.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.975+0000 7fa80bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa80c15c550 0x7fa80c153360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46900/0 (socket says 172.21.15.98:46900) 2024-09-19T08:46:06.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.975+0000 7fa80bfff640 1 -- 172.21.15.98:0/3416050948 learned_addr learned my addr 172.21.15.98:0/3416050948 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:06.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.975+0000 7fa80b7fe640 1 --2- 172.21.15.98:0/3416050948 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa80c06c1d0 0x7fa80c146720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:06.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.975+0000 7fa80affd640 1 --2- 172.21.15.98:0/3416050948 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa80c0cfad0 0x7fa80c135210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:06.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.975+0000 7fa80bfff640 1 -- 172.21.15.98:0/3416050948 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa80c0cfad0 msgr2=0x7fa80c135210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:06.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.975+0000 7fa80bfff640 1 --2- 172.21.15.98:0/3416050948 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa80c0cfad0 0x7fa80c135210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:06.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.975+0000 7fa80bfff640 1 -- 172.21.15.98:0/3416050948 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa80c06c1d0 msgr2=0x7fa80c146720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:06.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.975+0000 7fa80bfff640 1 --2- 172.21.15.98:0/3416050948 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa80c06c1d0 0x7fa80c146720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:06.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.975+0000 7fa80bfff640 1 -- 172.21.15.98:0/3416050948 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa80c1556a0 con 0x7fa80c15c550 2024-09-19T08:46:06.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.975+0000 7fa80bfff640 1 --2- 172.21.15.98:0/3416050948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa80c15c550 0x7fa80c153360 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fa80400cbf0 tx=0x7fa80400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:06.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.975+0000 7fa7f3fff640 1 -- 172.21.15.98:0/3416050948 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa804017070 con 0x7fa80c15c550 2024-09-19T08:46:06.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.975+0000 7fa7f3fff640 1 -- 172.21.15.98:0/3416050948 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa80400ad40 con 0x7fa80c15c550 2024-09-19T08:46:06.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.975+0000 7fa7f3fff640 1 -- 172.21.15.98:0/3416050948 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa804012600 con 0x7fa80c15c550 2024-09-19T08:46:06.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.975+0000 7fa811d3a640 1 -- 172.21.15.98:0/3416050948 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa80c155980 con 0x7fa80c15c550 2024-09-19T08:46:06.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.975+0000 7fa811d3a640 1 -- 172.21.15.98:0/3416050948 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa80c155f40 con 0x7fa80c15c550 2024-09-19T08:46:06.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.975+0000 7fa811d3a640 1 -- 172.21.15.98:0/3416050948 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa80c06bc90 con 0x7fa80c15c550 2024-09-19T08:46:06.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.975+0000 7fa7f3fff640 1 -- 172.21.15.98:0/3416050948 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fa80401a070 con 0x7fa80c15c550 2024-09-19T08:46:06.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.975+0000 7fa7f3fff640 1 --2- 172.21.15.98:0/3416050948 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa7f8067ef0 0x7fa7f806a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:06.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.975+0000 7fa7f3fff640 1 -- 172.21.15.98:0/3416050948 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5737+0+0 (secure 0 0 0) 0x7fa804086480 con 0x7fa80c15c550 2024-09-19T08:46:06.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.975+0000 7fa80b7fe640 1 --2- 172.21.15.98:0/3416050948 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa7f8067ef0 0x7fa7f806a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:06.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.975+0000 7fa80b7fe640 1 --2- 172.21.15.98:0/3416050948 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa7f8067ef0 0x7fa7f806a3b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa8000ad000 tx=0x7fa800008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:06.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:06.979+0000 7fa7f3fff640 1 -- 172.21.15.98:0/3416050948 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa8040560a0 con 0x7fa80c15c550 2024-09-19T08:46:07.125 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.127+0000 7fa811d3a640 1 -- 172.21.15.98:0/3416050948 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa80c156440 con 0x7fa7f8067ef0 2024-09-19T08:46:07.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.143+0000 7fa7f3fff640 1 -- 172.21.15.98:0/3416050948 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252414 (secure 0 0 0) 0x7fa8000ab660 con 0x7fa7f8067ef0 2024-09-19T08:46:07.145 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:46:07.147 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:46:07.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.147+0000 7fa811d3a640 1 -- 172.21.15.98:0/3416050948 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa7f8067ef0 msgr2=0x7fa7f806a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:07.147 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.147+0000 7fa811d3a640 1 --2- 172.21.15.98:0/3416050948 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa7f8067ef0 0x7fa7f806a3b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa8000ad000 tx=0x7fa800008040 comp rx=0 tx=0).stop 2024-09-19T08:46:07.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.147+0000 7fa811d3a640 1 -- 172.21.15.98:0/3416050948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa80c15c550 msgr2=0x7fa80c153360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:07.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.147+0000 7fa811d3a640 1 --2- 172.21.15.98:0/3416050948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa80c15c550 0x7fa80c153360 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fa80400cbf0 tx=0x7fa80400a430 comp rx=0 tx=0).stop 2024-09-19T08:46:07.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.147+0000 7fa811d3a640 1 -- 172.21.15.98:0/3416050948 shutdown_connections 2024-09-19T08:46:07.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.147+0000 7fa811d3a640 1 --2- 172.21.15.98:0/3416050948 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa7f8067ef0 0x7fa7f806a3b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:07.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.147+0000 7fa811d3a640 1 --2- 172.21.15.98:0/3416050948 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa80c15c550 0x7fa80c153360 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:07.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.147+0000 7fa811d3a640 1 --2- 172.21.15.98:0/3416050948 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa80c0cfad0 0x7fa80c135210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:07.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.147+0000 7fa811d3a640 1 --2- 172.21.15.98:0/3416050948 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa80c06c1d0 0x7fa80c146720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:07.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.147+0000 7fa811d3a640 1 -- 172.21.15.98:0/3416050948 >> 172.21.15.98:0/3416050948 conn(0x7fa80c0b2010 msgr2=0x7fa80c0aff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:07.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.147+0000 7fa811d3a640 1 -- 172.21.15.98:0/3416050948 shutdown_connections 2024-09-19T08:46:07.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.147+0000 7fa811d3a640 1 -- 172.21.15.98:0/3416050948 wait complete. 2024-09-19T08:46:07.161 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2024-09-19T08:46:06.933802+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":223,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":166,"ondisk_log_size":166,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":268908,"kb_used_data":15240,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754705812,"statfs":{"total":773094113280,"available":772818751488,"internally_reserved":0,"allocated":15605760,"data_stored":7573192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.157266"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.852535+0000","last_change":"2024-09-19T08:46:03.845875+0000","last_active":"2024-09-19T08:46:03.852535+0000","last_peered":"2024-09-19T08:46:03.852535+0000","last_clean":"2024-09-19T08:46:03.852535+0000","last_became_active":"2024-09-19T08:46:03.845116+0000","last_became_peered":"2024-09-19T08:46:03.845116+0000","last_unstale":"2024-09-19T08:46:03.852535+0000","last_undegraded":"2024-09-19T08:46:03.852535+0000","last_fullsized":"2024-09-19T08:46:03.852535+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:30.890134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.867942+0000","last_change":"2024-09-19T08:46:03.857639+0000","last_active":"2024-09-19T08:46:03.867942+0000","last_peered":"2024-09-19T08:46:03.867942+0000","last_clean":"2024-09-19T08:46:03.867942+0000","last_became_active":"2024-09-19T08:46:03.856855+0000","last_became_peered":"2024-09-19T08:46:03.856855+0000","last_unstale":"2024-09-19T08:46:03.867942+0000","last_undegraded":"2024-09-19T08:46:03.867942+0000","last_fullsized":"2024-09-19T08:46:03.867942+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:19:46.185253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.856293+0000","last_change":"2024-09-19T08:46:03.845735+0000","last_active":"2024-09-19T08:46:03.856293+0000","last_peered":"2024-09-19T08:46:03.856293+0000","last_clean":"2024-09-19T08:46:03.856293+0000","last_became_active":"2024-09-19T08:46:03.844420+0000","last_became_peered":"2024-09-19T08:46:03.844420+0000","last_unstale":"2024-09-19T08:46:03.856293+0000","last_undegraded":"2024-09-19T08:46:03.856293+0000","last_fullsized":"2024-09-19T08:46:03.856293+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:22:20.065648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.844155+0000","last_change":"2024-09-19T08:46:03.836280+0000","last_active":"2024-09-19T08:46:03.844155+0000","last_peered":"2024-09-19T08:46:03.844155+0000","last_clean":"2024-09-19T08:46:03.844155+0000","last_became_active":"2024-09-19T08:46:03.836014+0000","last_became_peered":"2024-09-19T08:46:03.836014+0000","last_unstale":"2024-09-19T08:46:03.844155+0000","last_undegraded":"2024-09-19T08:46:03.844155+0000","last_fullsized":"2024-09-19T08:46:03.844155+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:17:21.305649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.868428+0000","last_change":"2024-09-19T08:46:03.864933+0000","last_active":"2024-09-19T08:46:03.868428+0000","last_peered":"2024-09-19T08:46:03.868428+0000","last_clean":"2024-09-19T08:46:03.868428+0000","last_became_active":"2024-09-19T08:46:03.864122+0000","last_became_peered":"2024-09-19T08:46:03.864122+0000","last_unstale":"2024-09-19T08:46:03.868428+0000","last_undegraded":"2024-09-19T08:46:03.868428+0000","last_fullsized":"2024-09-19T08:46:03.868428+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:22:12.980219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.834242+0000","last_change":"2024-09-19T08:46:03.833166+0000","last_active":"2024-09-19T08:46:03.834242+0000","last_peered":"2024-09-19T08:46:03.834242+0000","last_clean":"2024-09-19T08:46:03.834242+0000","last_became_active":"2024-09-19T08:46:03.832822+0000","last_became_peered":"2024-09-19T08:46:03.832822+0000","last_unstale":"2024-09-19T08:46:03.834242+0000","last_undegraded":"2024-09-19T08:46:03.834242+0000","last_fullsized":"2024-09-19T08:46:03.834242+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:03:29.781045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.860401+0000","last_change":"2024-09-19T08:46:03.859952+0000","last_active":"2024-09-19T08:46:03.860401+0000","last_peered":"2024-09-19T08:46:03.860401+0000","last_clean":"2024-09-19T08:46:03.860401+0000","last_became_active":"2024-09-19T08:46:03.859681+0000","last_became_peered":"2024-09-19T08:46:03.859681+0000","last_unstale":"2024-09-19T08:46:03.860401+0000","last_undegraded":"2024-09-19T08:46:03.860401+0000","last_fullsized":"2024-09-19T08:46:03.860401+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:34.235923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.875411+0000","last_change":"2024-09-19T08:46:03.873085+0000","last_active":"2024-09-19T08:46:03.875411+0000","last_peered":"2024-09-19T08:46:03.875411+0000","last_clean":"2024-09-19T08:46:03.875411+0000","last_became_active":"2024-09-19T08:46:03.871402+0000","last_became_peered":"2024-09-19T08:46:03.871402+0000","last_unstale":"2024-09-19T08:46:03.875411+0000","last_undegraded":"2024-09-19T08:46:03.875411+0000","last_fullsized":"2024-09-19T08:46:03.875411+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:25:08.688723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.868594+0000","last_change":"2024-09-19T08:46:03.858143+0000","last_active":"2024-09-19T08:46:03.868594+0000","last_peered":"2024-09-19T08:46:03.868594+0000","last_clean":"2024-09-19T08:46:03.868594+0000","last_became_active":"2024-09-19T08:46:03.857185+0000","last_became_peered":"2024-09-19T08:46:03.857185+0000","last_unstale":"2024-09-19T08:46:03.868594+0000","last_undegraded":"2024-09-19T08:46:03.868594+0000","last_fullsized":"2024-09-19T08:46:03.868594+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:34:10.594759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.869595+0000","last_change":"2024-09-19T08:46:03.865464+0000","last_active":"2024-09-19T08:46:03.869595+0000","last_peered":"2024-09-19T08:46:03.869595+0000","last_clean":"2024-09-19T08:46:03.869595+0000","last_became_active":"2024-09-19T08:46:03.858453+0000","last_became_peered":"2024-09-19T08:46:03.858453+0000","last_unstale":"2024-09-19T08:46:03.869595+0000","last_undegraded":"2024-09-19T08:46:03.869595+0000","last_fullsized":"2024-09-19T08:46:03.869595+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:21:20.957798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.873065+0000","last_change":"2024-09-19T08:46:03.871081+0000","last_active":"2024-09-19T08:46:03.873065+0000","last_peered":"2024-09-19T08:46:03.873065+0000","last_clean":"2024-09-19T08:46:03.873065+0000","last_became_active":"2024-09-19T08:46:03.870238+0000","last_became_peered":"2024-09-19T08:46:03.870238+0000","last_unstale":"2024-09-19T08:46:03.873065+0000","last_undegraded":"2024-09-19T08:46:03.873065+0000","last_fullsized":"2024-09-19T08:46:03.873065+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:50.999221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.873145+0000","last_change":"2024-09-19T08:46:03.868381+0000","last_active":"2024-09-19T08:46:03.873145+0000","last_peered":"2024-09-19T08:46:03.873145+0000","last_clean":"2024-09-19T08:46:03.873145+0000","last_became_active":"2024-09-19T08:46:03.865474+0000","last_became_peered":"2024-09-19T08:46:03.865474+0000","last_unstale":"2024-09-19T08:46:03.873145+0000","last_undegraded":"2024-09-19T08:46:03.873145+0000","last_fullsized":"2024-09-19T08:46:03.873145+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:49.159131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.851262+0000","last_change":"2024-09-19T08:46:03.841212+0000","last_active":"2024-09-19T08:46:03.851262+0000","last_peered":"2024-09-19T08:46:03.851262+0000","last_clean":"2024-09-19T08:46:03.851262+0000","last_became_active":"2024-09-19T08:46:03.840346+0000","last_became_peered":"2024-09-19T08:46:03.840346+0000","last_unstale":"2024-09-19T08:46:03.851262+0000","last_undegraded":"2024-09-19T08:46:03.851262+0000","last_fullsized":"2024-09-19T08:46:03.851262+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:10:53.999170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.855503+0000","last_change":"2024-09-19T08:46:03.843064+0000","last_active":"2024-09-19T08:46:03.855503+0000","last_peered":"2024-09-19T08:46:03.855503+0000","last_clean":"2024-09-19T08:46:03.855503+0000","last_became_active":"2024-09-19T08:46:03.842288+0000","last_became_peered":"2024-09-19T08:46:03.842288+0000","last_unstale":"2024-09-19T08:46:03.855503+0000","last_undegraded":"2024-09-19T08:46:03.855503+0000","last_fullsized":"2024-09-19T08:46:03.855503+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:04:12.927685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.847444+0000","last_change":"2024-09-19T08:46:03.833833+0000","last_active":"2024-09-19T08:46:03.847444+0000","last_peered":"2024-09-19T08:46:03.847444+0000","last_clean":"2024-09-19T08:46:03.847444+0000","last_became_active":"2024-09-19T08:46:03.833001+0000","last_became_peered":"2024-09-19T08:46:03.833001+0000","last_unstale":"2024-09-19T08:46:03.847444+0000","last_undegraded":"2024-09-19T08:46:03.847444+0000","last_fullsized":"2024-09-19T08:46:03.847444+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:58.873284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.850386+0000","last_change":"2024-09-19T08:46:03.848995+0000","last_active":"2024-09-19T08:46:03.850386+0000","last_peered":"2024-09-19T08:46:03.850386+0000","last_clean":"2024-09-19T08:46:03.850386+0000","last_became_active":"2024-09-19T08:46:03.848517+0000","last_became_peered":"2024-09-19T08:46:03.848517+0000","last_unstale":"2024-09-19T08:46:03.850386+0000","last_undegraded":"2024-09-19T08:46:03.850386+0000","last_fullsized":"2024-09-19T08:46:03.850386+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:17.106192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.850332+0000","last_change":"2024-09-19T08:46:03.849711+0000","last_active":"2024-09-19T08:46:03.850332+0000","last_peered":"2024-09-19T08:46:03.850332+0000","last_clean":"2024-09-19T08:46:03.850332+0000","last_became_active":"2024-09-19T08:46:03.849459+0000","last_became_peered":"2024-09-19T08:46:03.849459+0000","last_unstale":"2024-09-19T08:46:03.850332+0000","last_undegraded":"2024-09-19T08:46:03.850332+0000","last_fullsized":"2024-09-19T08:46:03.850332+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:30:21.090683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.833538+0000","last_change":"2024-09-19T08:46:03.831264+0000","last_active":"2024-09-19T08:46:03.833538+0000","last_peered":"2024-09-19T08:46:03.833538+0000","last_clean":"2024-09-19T08:46:03.833538+0000","last_became_active":"2024-09-19T08:46:03.830966+0000","last_became_peered":"2024-09-19T08:46:03.830966+0000","last_unstale":"2024-09-19T08:46:03.833538+0000","last_undegraded":"2024-09-19T08:46:03.833538+0000","last_fullsized":"2024-09-19T08:46:03.833538+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:15:09.926688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.855564+0000","last_change":"2024-09-19T08:46:03.855087+0000","last_active":"2024-09-19T08:46:03.855564+0000","last_peered":"2024-09-19T08:46:03.855564+0000","last_clean":"2024-09-19T08:46:03.855564+0000","last_became_active":"2024-09-19T08:46:03.854750+0000","last_became_peered":"2024-09-19T08:46:03.854750+0000","last_unstale":"2024-09-19T08:46:03.855564+0000","last_undegraded":"2024-09-19T08:46:03.855564+0000","last_fullsized":"2024-09-19T08:46:03.855564+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:07:19.287192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.851279+0000","last_change":"2024-09-19T08:46:03.840987+0000","last_active":"2024-09-19T08:46:03.851279+0000","last_peered":"2024-09-19T08:46:03.851279+0000","last_clean":"2024-09-19T08:46:03.851279+0000","last_became_active":"2024-09-19T08:46:03.839219+0000","last_became_peered":"2024-09-19T08:46:03.839219+0000","last_unstale":"2024-09-19T08:46:03.851279+0000","last_undegraded":"2024-09-19T08:46:03.851279+0000","last_fullsized":"2024-09-19T08:46:03.851279+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:46.339270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.842992+0000","last_change":"2024-09-19T08:46:03.840744+0000","last_active":"2024-09-19T08:46:03.842992+0000","last_peered":"2024-09-19T08:46:03.842992+0000","last_clean":"2024-09-19T08:46:03.842992+0000","last_became_active":"2024-09-19T08:46:03.840409+0000","last_became_peered":"2024-09-19T08:46:03.840409+0000","last_unstale":"2024-09-19T08:46:03.842992+0000","last_undegraded":"2024-09-19T08:46:03.842992+0000","last_fullsized":"2024-09-19T08:46:03.842992+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:37:12.475418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.868761+0000","last_change":"2024-09-19T08:46:03.865010+0000","last_active":"2024-09-19T08:46:03.868761+0000","last_peered":"2024-09-19T08:46:03.868761+0000","last_clean":"2024-09-19T08:46:03.868761+0000","last_became_active":"2024-09-19T08:46:03.864546+0000","last_became_peered":"2024-09-19T08:46:03.864546+0000","last_unstale":"2024-09-19T08:46:03.868761+0000","last_undegraded":"2024-09-19T08:46:03.868761+0000","last_fullsized":"2024-09-19T08:46:03.868761+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:39:33.905171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.868563+0000","last_change":"2024-09-19T08:46:03.868051+0000","last_active":"2024-09-19T08:46:03.868563+0000","last_peered":"2024-09-19T08:46:03.868563+0000","last_clean":"2024-09-19T08:46:03.868563+0000","last_became_active":"2024-09-19T08:46:03.867232+0000","last_became_peered":"2024-09-19T08:46:03.867232+0000","last_unstale":"2024-09-19T08:46:03.868563+0000","last_undegraded":"2024-09-19T08:46:03.868563+0000","last_fullsized":"2024-09-19T08:46:03.868563+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:33.250058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.873471+0000","last_change":"2024-09-19T08:46:03.867851+0000","last_active":"2024-09-19T08:46:03.873471+0000","last_peered":"2024-09-19T08:46:03.873471+0000","last_clean":"2024-09-19T08:46:03.873471+0000","last_became_active":"2024-09-19T08:46:03.864868+0000","last_became_peered":"2024-09-19T08:46:03.864868+0000","last_unstale":"2024-09-19T08:46:03.873471+0000","last_undegraded":"2024-09-19T08:46:03.873471+0000","last_fullsized":"2024-09-19T08:46:03.873471+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:15:00.785757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.856127+0000","last_change":"2024-09-19T08:46:03.844467+0000","last_active":"2024-09-19T08:46:03.856127+0000","last_peered":"2024-09-19T08:46:03.856127+0000","last_clean":"2024-09-19T08:46:03.856127+0000","last_became_active":"2024-09-19T08:46:03.843735+0000","last_became_peered":"2024-09-19T08:46:03.843735+0000","last_unstale":"2024-09-19T08:46:03.856127+0000","last_undegraded":"2024-09-19T08:46:03.856127+0000","last_fullsized":"2024-09-19T08:46:03.856127+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:47:55.685173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.843339+0000","last_change":"2024-09-19T08:46:03.841631+0000","last_active":"2024-09-19T08:46:03.843339+0000","last_peered":"2024-09-19T08:46:03.843339+0000","last_clean":"2024-09-19T08:46:03.843339+0000","last_became_active":"2024-09-19T08:46:03.841058+0000","last_became_peered":"2024-09-19T08:46:03.841058+0000","last_unstale":"2024-09-19T08:46:03.843339+0000","last_undegraded":"2024-09-19T08:46:03.843339+0000","last_fullsized":"2024-09-19T08:46:03.843339+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:08:38.229382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.856248+0000","last_change":"2024-09-19T08:46:03.845137+0000","last_active":"2024-09-19T08:46:03.856248+0000","last_peered":"2024-09-19T08:46:03.856248+0000","last_clean":"2024-09-19T08:46:03.856248+0000","last_became_active":"2024-09-19T08:46:03.843422+0000","last_became_peered":"2024-09-19T08:46:03.843422+0000","last_unstale":"2024-09-19T08:46:03.856248+0000","last_undegraded":"2024-09-19T08:46:03.856248+0000","last_fullsized":"2024-09-19T08:46:03.856248+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:58:03.888372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.829223+0000","last_change":"2024-09-19T08:46:03.819236+0000","last_active":"2024-09-19T08:46:03.829223+0000","last_peered":"2024-09-19T08:46:03.829223+0000","last_clean":"2024-09-19T08:46:03.829223+0000","last_became_active":"2024-09-19T08:46:03.818523+0000","last_became_peered":"2024-09-19T08:46:03.818523+0000","last_unstale":"2024-09-19T08:46:03.829223+0000","last_undegraded":"2024-09-19T08:46:03.829223+0000","last_fullsized":"2024-09-19T08:46:03.829223+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:20.234366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.856877+0000","last_change":"2024-09-19T08:46:03.843968+0000","last_active":"2024-09-19T08:46:03.856877+0000","last_peered":"2024-09-19T08:46:03.856877+0000","last_clean":"2024-09-19T08:46:03.856877+0000","last_became_active":"2024-09-19T08:46:03.843208+0000","last_became_peered":"2024-09-19T08:46:03.843208+0000","last_unstale":"2024-09-19T08:46:03.856877+0000","last_undegraded":"2024-09-19T08:46:03.856877+0000","last_fullsized":"2024-09-19T08:46:03.856877+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:35:34.439099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.869811+0000","last_change":"2024-09-19T08:46:03.851027+0000","last_active":"2024-09-19T08:46:03.869811+0000","last_peered":"2024-09-19T08:46:03.869811+0000","last_clean":"2024-09-19T08:46:03.869811+0000","last_became_active":"2024-09-19T08:46:03.841235+0000","last_became_peered":"2024-09-19T08:46:03.841235+0000","last_unstale":"2024-09-19T08:46:03.869811+0000","last_undegraded":"2024-09-19T08:46:03.869811+0000","last_fullsized":"2024-09-19T08:46:03.869811+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:15:55.119248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.869011+0000","last_change":"2024-09-19T08:46:03.836883+0000","last_active":"2024-09-19T08:46:03.869011+0000","last_peered":"2024-09-19T08:46:03.869011+0000","last_clean":"2024-09-19T08:46:03.869011+0000","last_became_active":"2024-09-19T08:46:03.836064+0000","last_became_peered":"2024-09-19T08:46:03.836064+0000","last_unstale":"2024-09-19T08:46:03.869011+0000","last_undegraded":"2024-09-19T08:46:03.869011+0000","last_fullsized":"2024-09-19T08:46:03.869011+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:38:51.164522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.833514+0000","last_change":"2024-09-19T08:46:03.831260+0000","last_active":"2024-09-19T08:46:03.833514+0000","last_peered":"2024-09-19T08:46:03.833514+0000","last_clean":"2024-09-19T08:46:03.833514+0000","last_became_active":"2024-09-19T08:46:03.830941+0000","last_became_peered":"2024-09-19T08:46:03.830941+0000","last_unstale":"2024-09-19T08:46:03.833514+0000","last_undegraded":"2024-09-19T08:46:03.833514+0000","last_fullsized":"2024-09-19T08:46:03.833514+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:49:30.026574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.843361+0000","last_change":"2024-09-19T08:46:03.841782+0000","last_active":"2024-09-19T08:46:03.843361+0000","last_peered":"2024-09-19T08:46:03.843361+0000","last_clean":"2024-09-19T08:46:03.843361+0000","last_became_active":"2024-09-19T08:46:03.841341+0000","last_became_peered":"2024-09-19T08:46:03.841341+0000","last_unstale":"2024-09-19T08:46:03.843361+0000","last_undegraded":"2024-09-19T08:46:03.843361+0000","last_fullsized":"2024-09-19T08:46:03.843361+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:35:24.522000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"145'4","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.861835+0000","last_change":"2024-09-19T08:46:02.822927+0000","last_active":"2024-09-19T08:46:03.861835+0000","last_peered":"2024-09-19T08:46:03.861835+0000","last_clean":"2024-09-19T08:46:03.861835+0000","last_became_active":"2024-09-19T08:46:02.822228+0000","last_became_peered":"2024-09-19T08:46:02.822228+0000","last_unstale":"2024-09-19T08:46:03.861835+0000","last_undegraded":"2024-09-19T08:46:03.861835+0000","last_fullsized":"2024-09-19T08:46:03.861835+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:48:13.108449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.870021+0000","last_change":"2024-09-19T08:46:03.841386+0000","last_active":"2024-09-19T08:46:03.870021+0000","last_peered":"2024-09-19T08:46:03.870021+0000","last_clean":"2024-09-19T08:46:03.870021+0000","last_became_active":"2024-09-19T08:46:03.840366+0000","last_became_peered":"2024-09-19T08:46:03.840366+0000","last_unstale":"2024-09-19T08:46:03.870021+0000","last_undegraded":"2024-09-19T08:46:03.870021+0000","last_fullsized":"2024-09-19T08:46:03.870021+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:06:41.718917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"145'1","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.873109+0000","last_change":"2024-09-19T08:46:02.813607+0000","last_active":"2024-09-19T08:46:03.873109+0000","last_peered":"2024-09-19T08:46:03.873109+0000","last_clean":"2024-09-19T08:46:03.873109+0000","last_became_active":"2024-09-19T08:46:02.812005+0000","last_became_peered":"2024-09-19T08:46:02.812005+0000","last_unstale":"2024-09-19T08:46:03.873109+0000","last_undegraded":"2024-09-19T08:46:03.873109+0000","last_fullsized":"2024-09-19T08:46:03.873109+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:17:20.295044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.849649+0000","last_change":"2024-09-19T08:46:03.849084+0000","last_active":"2024-09-19T08:46:03.849649+0000","last_peered":"2024-09-19T08:46:03.849649+0000","last_clean":"2024-09-19T08:46:03.849649+0000","last_became_active":"2024-09-19T08:46:03.848714+0000","last_became_peered":"2024-09-19T08:46:03.848714+0000","last_unstale":"2024-09-19T08:46:03.849649+0000","last_undegraded":"2024-09-19T08:46:03.849649+0000","last_fullsized":"2024-09-19T08:46:03.849649+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:48:31.188244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"145'2","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.866784+0000","last_change":"2024-09-19T08:46:02.815918+0000","last_active":"2024-09-19T08:46:03.866784+0000","last_peered":"2024-09-19T08:46:03.866784+0000","last_clean":"2024-09-19T08:46:03.866784+0000","last_became_active":"2024-09-19T08:46:02.815594+0000","last_became_peered":"2024-09-19T08:46:02.815594+0000","last_unstale":"2024-09-19T08:46:03.866784+0000","last_undegraded":"2024-09-19T08:46:03.866784+0000","last_fullsized":"2024-09-19T08:46:03.866784+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:21:26.067928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.855184+0000","last_change":"2024-09-19T08:46:03.841447+0000","last_active":"2024-09-19T08:46:03.855184+0000","last_peered":"2024-09-19T08:46:03.855184+0000","last_clean":"2024-09-19T08:46:03.855184+0000","last_became_active":"2024-09-19T08:46:03.839658+0000","last_became_peered":"2024-09-19T08:46:03.839658+0000","last_unstale":"2024-09-19T08:46:03.855184+0000","last_undegraded":"2024-09-19T08:46:03.855184+0000","last_fullsized":"2024-09-19T08:46:03.855184+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:27:01.195371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"145'2","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.849201+0000","last_change":"2024-09-19T08:46:02.808816+0000","last_active":"2024-09-19T08:46:03.849201+0000","last_peered":"2024-09-19T08:46:03.849201+0000","last_clean":"2024-09-19T08:46:03.849201+0000","last_became_active":"2024-09-19T08:46:02.808425+0000","last_became_peered":"2024-09-19T08:46:02.808425+0000","last_unstale":"2024-09-19T08:46:03.849201+0000","last_undegraded":"2024-09-19T08:46:03.849201+0000","last_fullsized":"2024-09-19T08:46:03.849201+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:32:38.306210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.838032+0000","last_change":"2024-09-19T08:46:03.836871+0000","last_active":"2024-09-19T08:46:03.838032+0000","last_peered":"2024-09-19T08:46:03.838032+0000","last_clean":"2024-09-19T08:46:03.838032+0000","last_became_active":"2024-09-19T08:46:03.836207+0000","last_became_peered":"2024-09-19T08:46:03.836207+0000","last_unstale":"2024-09-19T08:46:03.838032+0000","last_undegraded":"2024-09-19T08:46:03.838032+0000","last_fullsized":"2024-09-19T08:46:03.838032+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:38.947995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"145'1","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.871250+0000","last_change":"2024-09-19T08:46:02.826293+0000","last_active":"2024-09-19T08:46:03.871250+0000","last_peered":"2024-09-19T08:46:03.871250+0000","last_clean":"2024-09-19T08:46:03.871250+0000","last_became_active":"2024-09-19T08:46:02.825521+0000","last_became_peered":"2024-09-19T08:46:02.825521+0000","last_unstale":"2024-09-19T08:46:03.871250+0000","last_undegraded":"2024-09-19T08:46:03.871250+0000","last_fullsized":"2024-09-19T08:46:03.871250+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:16:17.879970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.839003+0000","last_change":"2024-09-19T08:46:03.833178+0000","last_active":"2024-09-19T08:46:03.839003+0000","last_peered":"2024-09-19T08:46:03.839003+0000","last_clean":"2024-09-19T08:46:03.839003+0000","last_became_active":"2024-09-19T08:46:03.832950+0000","last_became_peered":"2024-09-19T08:46:03.832950+0000","last_unstale":"2024-09-19T08:46:03.839003+0000","last_undegraded":"2024-09-19T08:46:03.839003+0000","last_fullsized":"2024-09-19T08:46:03.839003+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:36:11.460653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"145'1","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.845297+0000","last_change":"2024-09-19T08:46:02.803300+0000","last_active":"2024-09-19T08:46:03.845297+0000","last_peered":"2024-09-19T08:46:03.845297+0000","last_clean":"2024-09-19T08:46:03.845297+0000","last_became_active":"2024-09-19T08:46:02.802421+0000","last_became_peered":"2024-09-19T08:46:02.802421+0000","last_unstale":"2024-09-19T08:46:03.845297+0000","last_undegraded":"2024-09-19T08:46:03.845297+0000","last_fullsized":"2024-09-19T08:46:03.845297+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:47:47.336303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.838354+0000","last_change":"2024-09-19T08:46:03.830673+0000","last_active":"2024-09-19T08:46:03.838354+0000","last_peered":"2024-09-19T08:46:03.838354+0000","last_clean":"2024-09-19T08:46:03.838354+0000","last_became_active":"2024-09-19T08:46:03.830401+0000","last_became_peered":"2024-09-19T08:46:03.830401+0000","last_unstale":"2024-09-19T08:46:03.838354+0000","last_undegraded":"2024-09-19T08:46:03.838354+0000","last_fullsized":"2024-09-19T08:46:03.838354+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:36:22.883202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.863448+0000","last_change":"2024-09-19T08:46:02.821450+0000","last_active":"2024-09-19T08:46:03.863448+0000","last_peered":"2024-09-19T08:46:03.863448+0000","last_clean":"2024-09-19T08:46:03.863448+0000","last_became_active":"2024-09-19T08:46:02.817925+0000","last_became_peered":"2024-09-19T08:46:02.817925+0000","last_unstale":"2024-09-19T08:46:03.863448+0000","last_undegraded":"2024-09-19T08:46:03.863448+0000","last_fullsized":"2024-09-19T08:46:03.863448+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:45.765147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.834221+0000","last_change":"2024-09-19T08:46:03.833192+0000","last_active":"2024-09-19T08:46:03.834221+0000","last_peered":"2024-09-19T08:46:03.834221+0000","last_clean":"2024-09-19T08:46:03.834221+0000","last_became_active":"2024-09-19T08:46:03.832771+0000","last_became_peered":"2024-09-19T08:46:03.832771+0000","last_unstale":"2024-09-19T08:46:03.834221+0000","last_undegraded":"2024-09-19T08:46:03.834221+0000","last_fullsized":"2024-09-19T08:46:03.834221+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:38:52.918550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.824218+0000","last_change":"2024-09-19T08:46:02.800644+0000","last_active":"2024-09-19T08:46:03.824218+0000","last_peered":"2024-09-19T08:46:03.824218+0000","last_clean":"2024-09-19T08:46:03.824218+0000","last_became_active":"2024-09-19T08:46:02.800326+0000","last_became_peered":"2024-09-19T08:46:02.800326+0000","last_unstale":"2024-09-19T08:46:03.824218+0000","last_undegraded":"2024-09-19T08:46:03.824218+0000","last_fullsized":"2024-09-19T08:46:03.824218+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:51:31.957926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.843605+0000","last_change":"2024-09-19T08:46:03.839842+0000","last_active":"2024-09-19T08:46:03.843605+0000","last_peered":"2024-09-19T08:46:03.843605+0000","last_clean":"2024-09-19T08:46:03.843605+0000","last_became_active":"2024-09-19T08:46:03.839245+0000","last_became_peered":"2024-09-19T08:46:03.839245+0000","last_unstale":"2024-09-19T08:46:03.843605+0000","last_undegraded":"2024-09-19T08:46:03.843605+0000","last_fullsized":"2024-09-19T08:46:03.843605+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:40:00.849087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.817786+0000","last_change":"2024-09-19T08:46:02.809804+0000","last_active":"2024-09-19T08:46:03.817786+0000","last_peered":"2024-09-19T08:46:03.817786+0000","last_clean":"2024-09-19T08:46:03.817786+0000","last_became_active":"2024-09-19T08:46:02.809517+0000","last_became_peered":"2024-09-19T08:46:02.809517+0000","last_unstale":"2024-09-19T08:46:03.817786+0000","last_undegraded":"2024-09-19T08:46:03.817786+0000","last_fullsized":"2024-09-19T08:46:03.817786+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:34:37.887250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.843494+0000","last_change":"2024-09-19T08:46:03.840107+0000","last_active":"2024-09-19T08:46:03.843494+0000","last_peered":"2024-09-19T08:46:03.843494+0000","last_clean":"2024-09-19T08:46:03.843494+0000","last_became_active":"2024-09-19T08:46:03.839663+0000","last_became_peered":"2024-09-19T08:46:03.839663+0000","last_unstale":"2024-09-19T08:46:03.843494+0000","last_undegraded":"2024-09-19T08:46:03.843494+0000","last_fullsized":"2024-09-19T08:46:03.843494+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:31:22.260734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"145'3","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.848880+0000","last_change":"2024-09-19T08:46:02.811541+0000","last_active":"2024-09-19T08:46:03.848880+0000","last_peered":"2024-09-19T08:46:03.848880+0000","last_clean":"2024-09-19T08:46:03.848880+0000","last_became_active":"2024-09-19T08:46:02.811241+0000","last_became_peered":"2024-09-19T08:46:02.811241+0000","last_unstale":"2024-09-19T08:46:03.848880+0000","last_undegraded":"2024-09-19T08:46:03.848880+0000","last_fullsized":"2024-09-19T08:46:03.848880+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:47:25.008234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.869974+0000","last_change":"2024-09-19T08:46:03.852952+0000","last_active":"2024-09-19T08:46:03.869974+0000","last_peered":"2024-09-19T08:46:03.869974+0000","last_clean":"2024-09-19T08:46:03.869974+0000","last_became_active":"2024-09-19T08:46:03.851921+0000","last_became_peered":"2024-09-19T08:46:03.851921+0000","last_unstale":"2024-09-19T08:46:03.869974+0000","last_undegraded":"2024-09-19T08:46:03.869974+0000","last_fullsized":"2024-09-19T08:46:03.869974+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:00:27.357328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"145'6","reported_seq":18,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.880589+0000","last_change":"2024-09-19T08:46:02.817203+0000","last_active":"2024-09-19T08:46:03.880589+0000","last_peered":"2024-09-19T08:46:03.880589+0000","last_clean":"2024-09-19T08:46:03.880589+0000","last_became_active":"2024-09-19T08:46:02.816489+0000","last_became_peered":"2024-09-19T08:46:02.816489+0000","last_unstale":"2024-09-19T08:46:03.880589+0000","last_undegraded":"2024-09-19T08:46:03.880589+0000","last_fullsized":"2024-09-19T08:46:03.880589+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:54:20.604808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.863991+0000","last_change":"2024-09-19T08:46:03.863261+0000","last_active":"2024-09-19T08:46:03.863991+0000","last_peered":"2024-09-19T08:46:03.863991+0000","last_clean":"2024-09-19T08:46:03.863991+0000","last_became_active":"2024-09-19T08:46:03.862984+0000","last_became_peered":"2024-09-19T08:46:03.862984+0000","last_unstale":"2024-09-19T08:46:03.863991+0000","last_undegraded":"2024-09-19T08:46:03.863991+0000","last_fullsized":"2024-09-19T08:46:03.863991+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:46:13.616664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"145'2","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.845674+0000","last_change":"2024-09-19T08:46:02.797025+0000","last_active":"2024-09-19T08:46:03.845674+0000","last_peered":"2024-09-19T08:46:03.845674+0000","last_clean":"2024-09-19T08:46:03.845674+0000","last_became_active":"2024-09-19T08:46:02.796737+0000","last_became_peered":"2024-09-19T08:46:02.796737+0000","last_unstale":"2024-09-19T08:46:03.845674+0000","last_undegraded":"2024-09-19T08:46:03.845674+0000","last_fullsized":"2024-09-19T08:46:03.845674+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:46:40.639107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.850682+0000","last_change":"2024-09-19T08:46:03.850150+0000","last_active":"2024-09-19T08:46:03.850682+0000","last_peered":"2024-09-19T08:46:03.850682+0000","last_clean":"2024-09-19T08:46:03.850682+0000","last_became_active":"2024-09-19T08:46:03.849815+0000","last_became_peered":"2024-09-19T08:46:03.849815+0000","last_unstale":"2024-09-19T08:46:03.850682+0000","last_undegraded":"2024-09-19T08:46:03.850682+0000","last_fullsized":"2024-09-19T08:46:03.850682+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:48:14.119055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.828228+0000","last_change":"2024-09-19T08:46:02.805022+0000","last_active":"2024-09-19T08:46:03.828228+0000","last_peered":"2024-09-19T08:46:03.828228+0000","last_clean":"2024-09-19T08:46:03.828228+0000","last_became_active":"2024-09-19T08:46:02.804472+0000","last_became_peered":"2024-09-19T08:46:02.804472+0000","last_unstale":"2024-09-19T08:46:03.828228+0000","last_undegraded":"2024-09-19T08:46:03.828228+0000","last_fullsized":"2024-09-19T08:46:03.828228+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:02:38.809125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.844364+0000","last_change":"2024-09-19T08:46:03.838208+0000","last_active":"2024-09-19T08:46:03.844364+0000","last_peered":"2024-09-19T08:46:03.844364+0000","last_clean":"2024-09-19T08:46:03.844364+0000","last_became_active":"2024-09-19T08:46:03.837720+0000","last_became_peered":"2024-09-19T08:46:03.837720+0000","last_unstale":"2024-09-19T08:46:03.844364+0000","last_undegraded":"2024-09-19T08:46:03.844364+0000","last_fullsized":"2024-09-19T08:46:03.844364+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:54:21.615414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.811429+0000","last_change":"2024-09-19T08:46:02.816068+0000","last_active":"2024-09-19T08:46:03.811429+0000","last_peered":"2024-09-19T08:46:03.811429+0000","last_clean":"2024-09-19T08:46:03.811429+0000","last_became_active":"2024-09-19T08:46:02.815686+0000","last_became_peered":"2024-09-19T08:46:02.815686+0000","last_unstale":"2024-09-19T08:46:03.811429+0000","last_undegraded":"2024-09-19T08:46:03.811429+0000","last_fullsized":"2024-09-19T08:46:03.811429+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:41.616069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.855564+0000","last_change":"2024-09-19T08:46:03.841385+0000","last_active":"2024-09-19T08:46:03.855564+0000","last_peered":"2024-09-19T08:46:03.855564+0000","last_clean":"2024-09-19T08:46:03.855564+0000","last_became_active":"2024-09-19T08:46:03.840602+0000","last_became_peered":"2024-09-19T08:46:03.840602+0000","last_unstale":"2024-09-19T08:46:03.855564+0000","last_undegraded":"2024-09-19T08:46:03.855564+0000","last_fullsized":"2024-09-19T08:46:03.855564+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:32:21.077746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"145'8","reported_seq":20,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.878810+0000","last_change":"2024-09-19T08:46:02.818995+0000","last_active":"2024-09-19T08:46:03.878810+0000","last_peered":"2024-09-19T08:46:03.878810+0000","last_clean":"2024-09-19T08:46:03.878810+0000","last_became_active":"2024-09-19T08:46:02.818302+0000","last_became_peered":"2024-09-19T08:46:02.818302+0000","last_unstale":"2024-09-19T08:46:03.878810+0000","last_undegraded":"2024-09-19T08:46:03.878810+0000","last_fullsized":"2024-09-19T08:46:03.878810+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:06:21.720276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.838274+0000","last_change":"2024-09-19T08:46:03.829615+0000","last_active":"2024-09-19T08:46:03.838274+0000","last_peered":"2024-09-19T08:46:03.838274+0000","last_clean":"2024-09-19T08:46:03.838274+0000","last_became_active":"2024-09-19T08:46:03.829337+0000","last_became_peered":"2024-09-19T08:46:03.829337+0000","last_unstale":"2024-09-19T08:46:03.838274+0000","last_undegraded":"2024-09-19T08:46:03.838274+0000","last_fullsized":"2024-09-19T08:46:03.838274+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:16:53.916495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"145'1","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.869335+0000","last_change":"2024-09-19T08:46:02.826073+0000","last_active":"2024-09-19T08:46:03.869335+0000","last_peered":"2024-09-19T08:46:03.869335+0000","last_clean":"2024-09-19T08:46:03.869335+0000","last_became_active":"2024-09-19T08:46:02.825423+0000","last_became_peered":"2024-09-19T08:46:02.825423+0000","last_unstale":"2024-09-19T08:46:03.869335+0000","last_undegraded":"2024-09-19T08:46:03.869335+0000","last_fullsized":"2024-09-19T08:46:03.869335+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:58:52.019014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.869380+0000","last_change":"2024-09-19T08:46:03.868989+0000","last_active":"2024-09-19T08:46:03.869380+0000","last_peered":"2024-09-19T08:46:03.869380+0000","last_clean":"2024-09-19T08:46:03.869380+0000","last_became_active":"2024-09-19T08:46:03.868667+0000","last_became_peered":"2024-09-19T08:46:03.868667+0000","last_unstale":"2024-09-19T08:46:03.869380+0000","last_undegraded":"2024-09-19T08:46:03.869380+0000","last_fullsized":"2024-09-19T08:46:03.869380+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:18:26.080637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"145'2","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.857068+0000","last_change":"2024-09-19T08:46:02.812569+0000","last_active":"2024-09-19T08:46:03.857068+0000","last_peered":"2024-09-19T08:46:03.857068+0000","last_clean":"2024-09-19T08:46:03.857068+0000","last_became_active":"2024-09-19T08:46:02.811674+0000","last_became_peered":"2024-09-19T08:46:02.811674+0000","last_unstale":"2024-09-19T08:46:03.857068+0000","last_undegraded":"2024-09-19T08:46:03.857068+0000","last_fullsized":"2024-09-19T08:46:03.857068+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:20:47.191119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.839031+0000","last_change":"2024-09-19T08:46:03.833176+0000","last_active":"2024-09-19T08:46:03.839031+0000","last_peered":"2024-09-19T08:46:03.839031+0000","last_clean":"2024-09-19T08:46:03.839031+0000","last_became_active":"2024-09-19T08:46:03.832906+0000","last_became_peered":"2024-09-19T08:46:03.832906+0000","last_unstale":"2024-09-19T08:46:03.839031+0000","last_undegraded":"2024-09-19T08:46:03.839031+0000","last_fullsized":"2024-09-19T08:46:03.839031+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:12.221337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"145'1","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.874149+0000","last_change":"2024-09-19T08:46:02.826237+0000","last_active":"2024-09-19T08:46:03.874149+0000","last_peered":"2024-09-19T08:46:03.874149+0000","last_clean":"2024-09-19T08:46:03.874149+0000","last_became_active":"2024-09-19T08:46:02.825550+0000","last_became_peered":"2024-09-19T08:46:02.825550+0000","last_unstale":"2024-09-19T08:46:03.874149+0000","last_undegraded":"2024-09-19T08:46:03.874149+0000","last_fullsized":"2024-09-19T08:46:03.874149+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:33.225318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.868273+0000","last_change":"2024-09-19T08:46:03.867671+0000","last_active":"2024-09-19T08:46:03.868273+0000","last_peered":"2024-09-19T08:46:03.868273+0000","last_clean":"2024-09-19T08:46:03.868273+0000","last_became_active":"2024-09-19T08:46:03.867346+0000","last_became_peered":"2024-09-19T08:46:03.867346+0000","last_unstale":"2024-09-19T08:46:03.868273+0000","last_undegraded":"2024-09-19T08:46:03.868273+0000","last_fullsized":"2024-09-19T08:46:03.868273+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:53.972805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.821153+0000","last_change":"2024-09-19T08:46:02.811610+0000","last_active":"2024-09-19T08:46:03.821153+0000","last_peered":"2024-09-19T08:46:03.821153+0000","last_clean":"2024-09-19T08:46:03.821153+0000","last_became_active":"2024-09-19T08:46:02.810589+0000","last_became_peered":"2024-09-19T08:46:02.810589+0000","last_unstale":"2024-09-19T08:46:03.821153+0000","last_undegraded":"2024-09-19T08:46:03.821153+0000","last_fullsized":"2024-09-19T08:46:03.821153+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:44:40.401216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.851355+0000","last_change":"2024-09-19T08:46:03.839733+0000","last_active":"2024-09-19T08:46:03.851355+0000","last_peered":"2024-09-19T08:46:03.851355+0000","last_clean":"2024-09-19T08:46:03.851355+0000","last_became_active":"2024-09-19T08:46:03.839000+0000","last_became_peered":"2024-09-19T08:46:03.839000+0000","last_unstale":"2024-09-19T08:46:03.851355+0000","last_undegraded":"2024-09-19T08:46:03.851355+0000","last_fullsized":"2024-09-19T08:46:03.851355+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:35:55.928756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"145'2","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.866597+0000","last_change":"2024-09-19T08:46:02.815895+0000","last_active":"2024-09-19T08:46:03.866597+0000","last_peered":"2024-09-19T08:46:03.866597+0000","last_clean":"2024-09-19T08:46:03.866597+0000","last_became_active":"2024-09-19T08:46:02.815528+0000","last_became_peered":"2024-09-19T08:46:02.815528+0000","last_unstale":"2024-09-19T08:46:03.866597+0000","last_undegraded":"2024-09-19T08:46:03.866597+0000","last_fullsized":"2024-09-19T08:46:03.866597+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:34:05.913711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.850382+0000","last_change":"2024-09-19T08:46:03.848970+0000","last_active":"2024-09-19T08:46:03.850382+0000","last_peered":"2024-09-19T08:46:03.850382+0000","last_clean":"2024-09-19T08:46:03.850382+0000","last_became_active":"2024-09-19T08:46:03.848492+0000","last_became_peered":"2024-09-19T08:46:03.848492+0000","last_unstale":"2024-09-19T08:46:03.850382+0000","last_undegraded":"2024-09-19T08:46:03.850382+0000","last_fullsized":"2024-09-19T08:46:03.850382+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:30.084530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.822483+0000","last_change":"2024-09-19T08:46:02.810879+0000","last_active":"2024-09-19T08:46:03.822483+0000","last_peered":"2024-09-19T08:46:03.822483+0000","last_clean":"2024-09-19T08:46:03.822483+0000","last_became_active":"2024-09-19T08:46:02.810110+0000","last_became_peered":"2024-09-19T08:46:02.810110+0000","last_unstale":"2024-09-19T08:46:03.822483+0000","last_undegraded":"2024-09-19T08:46:03.822483+0000","last_fullsized":"2024-09-19T08:46:03.822483+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:52.962200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.852620+0000","last_change":"2024-09-19T08:46:03.841458+0000","last_active":"2024-09-19T08:46:03.852620+0000","last_peered":"2024-09-19T08:46:03.852620+0000","last_clean":"2024-09-19T08:46:03.852620+0000","last_became_active":"2024-09-19T08:46:03.840677+0000","last_became_peered":"2024-09-19T08:46:03.840677+0000","last_unstale":"2024-09-19T08:46:03.852620+0000","last_undegraded":"2024-09-19T08:46:03.852620+0000","last_fullsized":"2024-09-19T08:46:03.852620+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:44:45.116396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.809975+0000","last_change":"2024-09-19T08:46:02.802952+0000","last_active":"2024-09-19T08:46:03.809975+0000","last_peered":"2024-09-19T08:46:03.809975+0000","last_clean":"2024-09-19T08:46:03.809975+0000","last_became_active":"2024-09-19T08:46:02.802520+0000","last_became_peered":"2024-09-19T08:46:02.802520+0000","last_unstale":"2024-09-19T08:46:03.809975+0000","last_undegraded":"2024-09-19T08:46:03.809975+0000","last_fullsized":"2024-09-19T08:46:03.809975+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:51.359353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.868173+0000","last_change":"2024-09-19T08:46:03.867618+0000","last_active":"2024-09-19T08:46:03.868173+0000","last_peered":"2024-09-19T08:46:03.868173+0000","last_clean":"2024-09-19T08:46:03.868173+0000","last_became_active":"2024-09-19T08:46:03.867224+0000","last_became_peered":"2024-09-19T08:46:03.867224+0000","last_unstale":"2024-09-19T08:46:03.868173+0000","last_undegraded":"2024-09-19T08:46:03.868173+0000","last_fullsized":"2024-09-19T08:46:03.868173+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:25.863652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.824437+0000","last_change":"2024-09-19T08:46:02.824281+0000","last_active":"2024-09-19T08:46:03.824437+0000","last_peered":"2024-09-19T08:46:03.824437+0000","last_clean":"2024-09-19T08:46:03.824437+0000","last_became_active":"2024-09-19T08:46:02.823571+0000","last_became_peered":"2024-09-19T08:46:02.823571+0000","last_unstale":"2024-09-19T08:46:03.824437+0000","last_undegraded":"2024-09-19T08:46:03.824437+0000","last_fullsized":"2024-09-19T08:46:03.824437+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:21:09.769250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.843446+0000","last_change":"2024-09-19T08:46:03.840107+0000","last_active":"2024-09-19T08:46:03.843446+0000","last_peered":"2024-09-19T08:46:03.843446+0000","last_clean":"2024-09-19T08:46:03.843446+0000","last_became_active":"2024-09-19T08:46:03.839530+0000","last_became_peered":"2024-09-19T08:46:03.839530+0000","last_unstale":"2024-09-19T08:46:03.843446+0000","last_undegraded":"2024-09-19T08:46:03.843446+0000","last_fullsized":"2024-09-19T08:46:03.843446+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:54:55.799914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"145'2","reported_seq":14,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.866837+0000","last_change":"2024-09-19T08:46:02.820093+0000","last_active":"2024-09-19T08:46:03.866837+0000","last_peered":"2024-09-19T08:46:03.866837+0000","last_clean":"2024-09-19T08:46:03.866837+0000","last_became_active":"2024-09-19T08:46:02.819624+0000","last_became_peered":"2024-09-19T08:46:02.819624+0000","last_unstale":"2024-09-19T08:46:03.866837+0000","last_undegraded":"2024-09-19T08:46:03.866837+0000","last_fullsized":"2024-09-19T08:46:03.866837+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:34:18.769816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.863858+0000","last_change":"2024-09-19T08:46:03.863241+0000","last_active":"2024-09-19T08:46:03.863858+0000","last_peered":"2024-09-19T08:46:03.863858+0000","last_clean":"2024-09-19T08:46:03.863858+0000","last_became_active":"2024-09-19T08:46:03.862946+0000","last_became_peered":"2024-09-19T08:46:03.862946+0000","last_unstale":"2024-09-19T08:46:03.863858+0000","last_undegraded":"2024-09-19T08:46:03.863858+0000","last_fullsized":"2024-09-19T08:46:03.863858+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:44:41.411821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.828759+0000","last_change":"2024-09-19T08:46:02.817930+0000","last_active":"2024-09-19T08:46:03.828759+0000","last_peered":"2024-09-19T08:46:03.828759+0000","last_clean":"2024-09-19T08:46:03.828759+0000","last_became_active":"2024-09-19T08:46:02.817157+0000","last_became_peered":"2024-09-19T08:46:02.817157+0000","last_unstale":"2024-09-19T08:46:03.828759+0000","last_undegraded":"2024-09-19T08:46:03.828759+0000","last_fullsized":"2024-09-19T08:46:03.828759+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:44:34.154817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.842880+0000","last_change":"2024-09-19T08:46:03.839805+0000","last_active":"2024-09-19T08:46:03.842880+0000","last_peered":"2024-09-19T08:46:03.842880+0000","last_clean":"2024-09-19T08:46:03.842880+0000","last_became_active":"2024-09-19T08:46:03.839507+0000","last_became_peered":"2024-09-19T08:46:03.839507+0000","last_unstale":"2024-09-19T08:46:03.842880+0000","last_undegraded":"2024-09-19T08:46:03.842880+0000","last_fullsized":"2024-09-19T08:46:03.842880+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:38.700161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.827464+0000","last_change":"2024-09-19T08:46:02.822328+0000","last_active":"2024-09-19T08:46:03.827464+0000","last_peered":"2024-09-19T08:46:03.827464+0000","last_clean":"2024-09-19T08:46:03.827464+0000","last_became_active":"2024-09-19T08:46:02.819128+0000","last_became_peered":"2024-09-19T08:46:02.819128+0000","last_unstale":"2024-09-19T08:46:03.827464+0000","last_undegraded":"2024-09-19T08:46:03.827464+0000","last_fullsized":"2024-09-19T08:46:03.827464+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:32.043000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.835392+0000","last_change":"2024-09-19T08:46:03.832092+0000","last_active":"2024-09-19T08:46:03.835392+0000","last_peered":"2024-09-19T08:46:03.835392+0000","last_clean":"2024-09-19T08:46:03.835392+0000","last_became_active":"2024-09-19T08:46:03.831768+0000","last_became_peered":"2024-09-19T08:46:03.831768+0000","last_unstale":"2024-09-19T08:46:03.835392+0000","last_undegraded":"2024-09-19T08:46:03.835392+0000","last_fullsized":"2024-09-19T08:46:03.835392+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:10.287914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.829511+0000","last_change":"2024-09-19T08:46:02.819293+0000","last_active":"2024-09-19T08:46:03.829511+0000","last_peered":"2024-09-19T08:46:03.829511+0000","last_clean":"2024-09-19T08:46:03.829511+0000","last_became_active":"2024-09-19T08:46:02.819040+0000","last_became_peered":"2024-09-19T08:46:02.819040+0000","last_unstale":"2024-09-19T08:46:03.829511+0000","last_undegraded":"2024-09-19T08:46:03.829511+0000","last_fullsized":"2024-09-19T08:46:03.829511+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:58.244723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.835442+0000","last_change":"2024-09-19T08:46:03.831610+0000","last_active":"2024-09-19T08:46:03.835442+0000","last_peered":"2024-09-19T08:46:03.835442+0000","last_clean":"2024-09-19T08:46:03.835442+0000","last_became_active":"2024-09-19T08:46:03.831233+0000","last_became_peered":"2024-09-19T08:46:03.831233+0000","last_unstale":"2024-09-19T08:46:03.835442+0000","last_undegraded":"2024-09-19T08:46:03.835442+0000","last_fullsized":"2024-09-19T08:46:03.835442+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:17:40.995948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.813954+0000","last_change":"2024-09-19T08:46:02.813739+0000","last_active":"2024-09-19T08:46:03.813954+0000","last_peered":"2024-09-19T08:46:03.813954+0000","last_clean":"2024-09-19T08:46:03.813954+0000","last_became_active":"2024-09-19T08:46:02.805993+0000","last_became_peered":"2024-09-19T08:46:02.805993+0000","last_unstale":"2024-09-19T08:46:03.813954+0000","last_undegraded":"2024-09-19T08:46:03.813954+0000","last_fullsized":"2024-09-19T08:46:03.813954+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:52:42.876907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.869362+0000","last_change":"2024-09-19T08:46:03.836909+0000","last_active":"2024-09-19T08:46:03.869362+0000","last_peered":"2024-09-19T08:46:03.869362+0000","last_clean":"2024-09-19T08:46:03.869362+0000","last_became_active":"2024-09-19T08:46:03.836141+0000","last_became_peered":"2024-09-19T08:46:03.836141+0000","last_unstale":"2024-09-19T08:46:03.869362+0000","last_undegraded":"2024-09-19T08:46:03.869362+0000","last_fullsized":"2024-09-19T08:46:03.869362+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:06:59.971302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.809591+0000","last_change":"2024-09-19T08:46:02.803230+0000","last_active":"2024-09-19T08:46:03.809591+0000","last_peered":"2024-09-19T08:46:03.809591+0000","last_clean":"2024-09-19T08:46:03.809591+0000","last_became_active":"2024-09-19T08:46:02.798750+0000","last_became_peered":"2024-09-19T08:46:02.798750+0000","last_unstale":"2024-09-19T08:46:03.809591+0000","last_undegraded":"2024-09-19T08:46:03.809591+0000","last_fullsized":"2024-09-19T08:46:03.809591+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:49.796040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.841978+0000","last_change":"2024-09-19T08:46:03.840788+0000","last_active":"2024-09-19T08:46:03.841978+0000","last_peered":"2024-09-19T08:46:03.841978+0000","last_clean":"2024-09-19T08:46:03.841978+0000","last_became_active":"2024-09-19T08:46:03.840509+0000","last_became_peered":"2024-09-19T08:46:03.840509+0000","last_unstale":"2024-09-19T08:46:03.841978+0000","last_undegraded":"2024-09-19T08:46:03.841978+0000","last_fullsized":"2024-09-19T08:46:03.841978+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:42.050789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.863445+0000","last_change":"2024-09-19T08:46:02.815898+0000","last_active":"2024-09-19T08:46:03.863445+0000","last_peered":"2024-09-19T08:46:03.863445+0000","last_clean":"2024-09-19T08:46:03.863445+0000","last_became_active":"2024-09-19T08:46:02.810982+0000","last_became_peered":"2024-09-19T08:46:02.810982+0000","last_unstale":"2024-09-19T08:46:03.863445+0000","last_undegraded":"2024-09-19T08:46:03.863445+0000","last_fullsized":"2024-09-19T08:46:03.863445+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:19:10.764379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.851193+0000","last_change":"2024-09-19T08:46:03.839794+0000","last_active":"2024-09-19T08:46:03.851193+0000","last_peered":"2024-09-19T08:46:03.851193+0000","last_clean":"2024-09-19T08:46:03.851193+0000","last_became_active":"2024-09-19T08:46:03.838903+0000","last_became_peered":"2024-09-19T08:46:03.838903+0000","last_unstale":"2024-09-19T08:46:03.851193+0000","last_undegraded":"2024-09-19T08:46:03.851193+0000","last_fullsized":"2024-09-19T08:46:03.851193+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:51:14.062744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.814989+0000","last_change":"2024-09-19T08:46:02.815409+0000","last_active":"2024-09-19T08:46:03.814989+0000","last_peered":"2024-09-19T08:46:03.814989+0000","last_clean":"2024-09-19T08:46:03.814989+0000","last_became_active":"2024-09-19T08:46:02.814663+0000","last_became_peered":"2024-09-19T08:46:02.814663+0000","last_unstale":"2024-09-19T08:46:03.814989+0000","last_undegraded":"2024-09-19T08:46:03.814989+0000","last_fullsized":"2024-09-19T08:46:03.814989+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:00.169631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.843468+0000","last_change":"2024-09-19T08:46:03.840754+0000","last_active":"2024-09-19T08:46:03.843468+0000","last_peered":"2024-09-19T08:46:03.843468+0000","last_clean":"2024-09-19T08:46:03.843468+0000","last_became_active":"2024-09-19T08:46:03.839653+0000","last_became_peered":"2024-09-19T08:46:03.839653+0000","last_unstale":"2024-09-19T08:46:03.843468+0000","last_undegraded":"2024-09-19T08:46:03.843468+0000","last_fullsized":"2024-09-19T08:46:03.843468+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":144,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:02.772904+0000","last_clean_scrub_stamp":"2024-09-19T08:46:02.772904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:47.007104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.814063+0000","last_change":"2024-09-19T08:46:02.803881+0000","last_active":"2024-09-19T08:46:03.814063+0000","last_peered":"2024-09-19T08:46:03.814063+0000","last_clean":"2024-09-19T08:46:03.814063+0000","last_became_active":"2024-09-19T08:46:02.802876+0000","last_became_peered":"2024-09-19T08:46:02.802876+0000","last_unstale":"2024-09-19T08:46:03.814063+0000","last_undegraded":"2024-09-19T08:46:03.814063+0000","last_fullsized":"2024-09-19T08:46:03.814063+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:01.762298+0000","last_clean_scrub_stamp":"2024-09-19T08:46:01.762298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:19:13.207333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"71'128","reported_seq":433,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-19T08:46:03.824575+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:46:03.824575+0000","last_peered":"2024-09-19T08:46:03.824575+0000","last_clean":"2024-09-19T08:46:03.824575+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:46:03.824575+0000","last_undegraded":"2024-09-19T08:46:03.824575+0000","last_fullsized":"2024-09-19T08:46:03.824575+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509538,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33528,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338312,"statfs":{"total":96636764160,"available":96602431488,"internally_reserved":0,"allocated":2396160,"data_stored":1389425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574947,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32952,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338888,"statfs":{"total":96636764160,"available":96603021312,"internally_reserved":0,"allocated":1802240,"data_stored":799057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509540,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32948,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338892,"statfs":{"total":96636764160,"available":96603025408,"internally_reserved":0,"allocated":1802240,"data_stored":799057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509540,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32948,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338892,"statfs":{"total":96636764160,"available":96603025408,"internally_reserved":0,"allocated":1802240,"data_stored":799057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542242,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37108,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334732,"statfs":{"total":96636764160,"available":96598765568,"internally_reserved":0,"allocated":1802240,"data_stored":799057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509537,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32948,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338892,"statfs":{"total":96636764160,"available":96603025408,"internally_reserved":0,"allocated":1802240,"data_stored":799057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509539,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32948,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338892,"statfs":{"total":96636764160,"available":96603025408,"internally_reserved":0,"allocated":1802240,"data_stored":799057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509537,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33528,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338312,"statfs":{"total":96636764160,"available":96602431488,"internally_reserved":0,"allocated":2396160,"data_stored":1389425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:46:07.165 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T08:46:07.165 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-19T08:46:07.332 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.335+0000 7f6bc94f2640 1 Processor -- start 2024-09-19T08:46:07.333 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.335+0000 7f6bc94f2640 1 -- start start 2024-09-19T08:46:07.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.335+0000 7f6bc94f2640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6bc415cf90 0x7f6bc406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:07.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.335+0000 7f6bc94f2640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bc40ce3c0 con 0x7f6bc415cbb0 2024-09-19T08:46:07.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.335+0000 7f6bc94f2640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bc40cd090 con 0x7f6bc406c1d0 2024-09-19T08:46:07.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.335+0000 7f6bc94f2640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f6bc40cd370 con 0x7f6bc415cf90 2024-09-19T08:46:07.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.335+0000 7f6bc2ffd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f6bc415cbb0 0x7f6bc4163930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:42500/0 (socket says 172.21.15.98:42500) 2024-09-19T08:46:07.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.335+0000 7f6bc2ffd640 1 -- 172.21.15.98:0/1491220424 learned_addr learned my addr 172.21.15.98:0/1491220424 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:07.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.335+0000 7f6bba7fc640 1 --2- 172.21.15.98:0/1491220424 >> v2:172.21.15.98:3300/0 conn(0x7f6bc415cf90 0x7f6bc406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:07.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.335+0000 7f6bba7fc640 1 -- 172.21.15.98:0/1491220424 >> v1:172.21.15.139:6789/0 conn(0x7f6bc406c1d0 legacy=0x7f6bc416c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:07.334 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.335+0000 7f6bba7fc640 1 -- 172.21.15.98:0/1491220424 >> v1:172.21.15.98:6789/0 conn(0x7f6bc415cbb0 legacy=0x7f6bc4163930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:07.335 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.335+0000 7f6bba7fc640 1 -- 172.21.15.98:0/1491220424 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc40cd650 con 0x7f6bc415cf90 2024-09-19T08:46:07.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.339+0000 7f6bba7fc640 1 --2- 172.21.15.98:0/1491220424 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bc415cf90 0x7f6bc406bc90 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f6bb40097c0 tx=0x7f6bb401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7fa5768e952c11 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:07.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.339+0000 7f6bc27fc640 1 -- 172.21.15.98:0/1491220424 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb40b0070 con 0x7f6bc415cf90 2024-09-19T08:46:07.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.339+0000 7f6bc27fc640 1 -- 172.21.15.98:0/1491220424 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bb40ab600 con 0x7f6bc415cf90 2024-09-19T08:46:07.337 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.339+0000 7f6bc27fc640 1 -- 172.21.15.98:0/1491220424 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb40ab9d0 con 0x7f6bc415cf90 2024-09-19T08:46:07.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.339+0000 7f6bc94f2640 1 -- 172.21.15.98:0/1491220424 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bc415cf90 msgr2=0x7f6bc406bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:07.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.339+0000 7f6bc94f2640 1 --2- 172.21.15.98:0/1491220424 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bc415cf90 0x7f6bc406bc90 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f6bb40097c0 tx=0x7f6bb401c460 comp rx=0 tx=0).stop 2024-09-19T08:46:07.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.339+0000 7f6bc94f2640 1 -- 172.21.15.98:0/1491220424 shutdown_connections 2024-09-19T08:46:07.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.339+0000 7f6bc94f2640 1 --2- 172.21.15.98:0/1491220424 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bc415cf90 0x7f6bc406bc90 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:07.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.339+0000 7f6bc94f2640 1 -- 172.21.15.98:0/1491220424 >> 172.21.15.98:0/1491220424 conn(0x7f6bc40b2010 msgr2=0x7f6bc40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:07.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.339+0000 7f6bc94f2640 1 -- 172.21.15.98:0/1491220424 shutdown_connections 2024-09-19T08:46:07.338 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.339+0000 7f6bc94f2640 1 -- 172.21.15.98:0/1491220424 wait complete. 2024-09-19T08:46:07.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.339+0000 7f6bc94f2640 1 Processor -- start 2024-09-19T08:46:07.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.339+0000 7f6bc94f2640 1 -- start start 2024-09-19T08:46:07.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.339+0000 7f6bc94f2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6bc406c1d0 0x7f6bc40734c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:07.340 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.339+0000 7f6bc94f2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bc415cbb0 0x7f6bc40724e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:07.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.339+0000 7f6bc94f2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bc415cf90 0x7f6bc4144ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:07.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.339+0000 7f6bc94f2640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc40cce70 con 0x7f6bc415cf90 2024-09-19T08:46:07.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.339+0000 7f6bc94f2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc40cccf0 con 0x7f6bc415cbb0 2024-09-19T08:46:07.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.339+0000 7f6bc94f2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6bc4170540 con 0x7f6bc406c1d0 2024-09-19T08:46:07.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.339+0000 7f6bc37fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bc415cf90 0x7f6bc4144ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:07.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.339+0000 7f6bc37fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bc415cf90 0x7f6bc4144ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46916/0 (socket says 172.21.15.98:46916) 2024-09-19T08:46:07.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.339+0000 7f6bc37fe640 1 -- 172.21.15.98:0/2972720473 learned_addr learned my addr 172.21.15.98:0/2972720473 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:07.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.339+0000 7f6bc37fe640 1 -- 172.21.15.98:0/2972720473 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6bc406c1d0 msgr2=0x7f6bc40734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:07.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.343+0000 7f6bc37fe640 1 --2- 172.21.15.98:0/2972720473 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6bc406c1d0 0x7f6bc40734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:07.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.343+0000 7f6bc37fe640 1 -- 172.21.15.98:0/2972720473 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bc415cbb0 msgr2=0x7f6bc40724e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:07.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.343+0000 7f6bc37fe640 1 --2- 172.21.15.98:0/2972720473 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bc415cbb0 0x7f6bc40724e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:07.341 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.343+0000 7f6bc37fe640 1 -- 172.21.15.98:0/2972720473 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc415b160 con 0x7f6bc415cf90 2024-09-19T08:46:07.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.343+0000 7f6bc37fe640 1 --2- 172.21.15.98:0/2972720473 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bc415cf90 0x7f6bc4144ef0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc00c780 tx=0x7f6bbc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:07.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.343+0000 7f6bbbfff640 1 -- 172.21.15.98:0/2972720473 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc004b70 con 0x7f6bc415cf90 2024-09-19T08:46:07.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.343+0000 7f6bbbfff640 1 -- 172.21.15.98:0/2972720473 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bbc004d10 con 0x7f6bc415cf90 2024-09-19T08:46:07.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.343+0000 7f6bc94f2640 1 -- 172.21.15.98:0/2972720473 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bc415b440 con 0x7f6bc415cf90 2024-09-19T08:46:07.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.343+0000 7f6bbbfff640 1 -- 172.21.15.98:0/2972720473 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc00aa90 con 0x7f6bc415cf90 2024-09-19T08:46:07.342 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.343+0000 7f6bc94f2640 1 -- 172.21.15.98:0/2972720473 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bc415ba00 con 0x7f6bc415cf90 2024-09-19T08:46:07.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.343+0000 7f6bc94f2640 1 -- 172.21.15.98:0/2972720473 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bc406bc90 con 0x7f6bc415cf90 2024-09-19T08:46:07.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.343+0000 7f6bbbfff640 1 -- 172.21.15.98:0/2972720473 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f6bbc00ac30 con 0x7f6bc415cf90 2024-09-19T08:46:07.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.343+0000 7f6bbbfff640 1 --2- 172.21.15.98:0/2972720473 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6ba4067f20 0x7f6ba406a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:07.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.343+0000 7f6bbbfff640 1 -- 172.21.15.98:0/2972720473 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5737+0+0 (secure 0 0 0) 0x7f6bbc086a30 con 0x7f6bc415cf90 2024-09-19T08:46:07.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.343+0000 7f6bc2ffd640 1 --2- 172.21.15.98:0/2972720473 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6ba4067f20 0x7f6ba406a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:07.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.343+0000 7f6bc2ffd640 1 --2- 172.21.15.98:0/2972720473 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6ba4067f20 0x7f6ba406a3e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6bac000f70 tx=0x7f6bac007a90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:07.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.347+0000 7f6bbbfff640 1 -- 172.21.15.98:0/2972720473 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6bbc020070 con 0x7f6bc415cf90 2024-09-19T08:46:07.569 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.571+0000 7f6bc94f2640 1 -- 172.21.15.98:0/2972720473 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f6bc40b2770 con 0x7f6bc415cf90 2024-09-19T08:46:07.572 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.575+0000 7f6bbbfff640 1 -- 172.21.15.98:0/2972720473 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) ==== 190+0+147 (secure 0 0 0) 0x7f6bbc056ca0 con 0x7f6bc415cf90 2024-09-19T08:46:07.572 INFO:teuthology.orchestra.run.smithi098.stdout:[client.0] 2024-09-19T08:46:07.572 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQC/4utm0vNpIhAA4S1E8dFZ0+ibmSKhmZ+bIA== 2024-09-19T08:46:07.573 INFO:teuthology.orchestra.run.smithi098.stdout: caps mds = "allow" 2024-09-19T08:46:07.573 INFO:teuthology.orchestra.run.smithi098.stdout: caps mon = "allow r" 2024-09-19T08:46:07.573 INFO:teuthology.orchestra.run.smithi098.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-19T08:46:07.573 INFO:teuthology.orchestra.run.smithi098.stderr:updated caps for client.0 2024-09-19T08:46:07.574 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.575+0000 7f6bc94f2640 1 -- 172.21.15.98:0/2972720473 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6ba4067f20 msgr2=0x7f6ba406a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:07.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.575+0000 7f6bc94f2640 1 --2- 172.21.15.98:0/2972720473 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6ba4067f20 0x7f6ba406a3e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6bac000f70 tx=0x7f6bac007a90 comp rx=0 tx=0).stop 2024-09-19T08:46:07.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.575+0000 7f6bc94f2640 1 -- 172.21.15.98:0/2972720473 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bc415cf90 msgr2=0x7f6bc4144ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:07.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.575+0000 7f6bc94f2640 1 --2- 172.21.15.98:0/2972720473 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bc415cf90 0x7f6bc4144ef0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc00c780 tx=0x7f6bbc00cc40 comp rx=0 tx=0).stop 2024-09-19T08:46:07.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.575+0000 7f6bc94f2640 1 -- 172.21.15.98:0/2972720473 shutdown_connections 2024-09-19T08:46:07.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.575+0000 7f6bc94f2640 1 --2- 172.21.15.98:0/2972720473 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6ba4067f20 0x7f6ba406a3e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:07.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.575+0000 7f6bc94f2640 1 --2- 172.21.15.98:0/2972720473 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bc415cf90 0x7f6bc4144ef0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:07.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.575+0000 7f6bc94f2640 1 --2- 172.21.15.98:0/2972720473 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bc415cbb0 0x7f6bc40724e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:07.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.575+0000 7f6bc94f2640 1 --2- 172.21.15.98:0/2972720473 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6bc406c1d0 0x7f6bc40734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:07.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.575+0000 7f6bc94f2640 1 -- 172.21.15.98:0/2972720473 >> 172.21.15.98:0/2972720473 conn(0x7f6bc40b2010 msgr2=0x7f6bc40b4530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:07.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.575+0000 7f6bc94f2640 1 -- 172.21.15.98:0/2972720473 shutdown_connections 2024-09-19T08:46:07.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.575+0000 7f6bc94f2640 1 -- 172.21.15.98:0/2972720473 wait complete. 2024-09-19T08:46:07.590 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:46:07.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.715+0000 7f45e2b2e640 1 Processor -- start 2024-09-19T08:46:07.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.715+0000 7f45e2b2e640 1 -- start start 2024-09-19T08:46:07.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.715+0000 7f45e2b2e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45dc15c550 0x7f45dc06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:07.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.715+0000 7f45e2b2e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f45dc0cfad0 0x7f45dc0723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:07.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.715+0000 7f45e2b2e640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45dc06d310 con 0x7f45dc06c1d0 2024-09-19T08:46:07.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.715+0000 7f45e2b2e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f45dc06c610 con 0x7f45dc0cfad0 2024-09-19T08:46:07.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.715+0000 7f45e2b2e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f45dc06d5f0 con 0x7f45dc15c550 2024-09-19T08:46:07.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.715+0000 7f45dbfff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f45dc06c1d0 0x7f45dc0cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36874/0 (socket says 172.21.15.98:36874) 2024-09-19T08:46:07.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.715+0000 7f45dbfff640 1 -- 172.21.15.98:0/3194120089 learned_addr learned my addr 172.21.15.98:0/3194120089 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:07.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.715+0000 7f45e10cb640 1 --2- 172.21.15.98:0/3194120089 >> v2:172.21.15.139:3300/0 conn(0x7f45dc0cfad0 0x7f45dc0723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:07.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.715+0000 7f45e08ca640 1 --2- 172.21.15.98:0/3194120089 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45dc15c550 0x7f45dc06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:07.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.715+0000 7f45db7fe640 1 -- 172.21.15.98:0/3194120089 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3265110697 0 0) 0x7f45dc06d310 con 0x7f45dc06c1d0 2024-09-19T08:46:07.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.715+0000 7f45db7fe640 1 -- 172.21.15.98:0/3194120089 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45c4003530 con 0x7f45dc06c1d0 2024-09-19T08:46:07.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.715+0000 7f45db7fe640 1 -- 172.21.15.98:0/3194120089 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3087947436 0 0) 0x7f45c4003530 con 0x7f45dc06c1d0 2024-09-19T08:46:07.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.715+0000 7f45db7fe640 1 -- 172.21.15.98:0/3194120089 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45dc15c550 msgr2=0x7f45dc06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:07.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.715+0000 7f45db7fe640 1 --2- 172.21.15.98:0/3194120089 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45dc15c550 0x7f45dc06bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:07.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.715+0000 7f45db7fe640 1 -- 172.21.15.98:0/3194120089 >> v2:172.21.15.139:3300/0 conn(0x7f45dc0cfad0 msgr2=0x7f45dc0723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:07.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.715+0000 7f45db7fe640 1 --2- 172.21.15.98:0/3194120089 >> v2:172.21.15.139:3300/0 conn(0x7f45dc0cfad0 0x7f45dc0723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:07.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.715+0000 7f45db7fe640 1 -- 172.21.15.98:0/3194120089 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45dc06d8d0 con 0x7f45dc06c1d0 2024-09-19T08:46:07.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.715+0000 7f45db7fe640 1 -- 172.21.15.98:0/3194120089 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f45d0001680 con 0x7f45dc06c1d0 2024-09-19T08:46:07.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.715+0000 7f45db7fe640 1 -- 172.21.15.98:0/3194120089 >> v1:172.21.15.98:6789/0 conn(0x7f45dc06c1d0 legacy=0x7f45dc0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:07.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.715+0000 7f45db7fe640 1 --2- 172.21.15.98:0/3194120089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45c40039c0 0x7f45c4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:07.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.715+0000 7f45db7fe640 1 --2- 172.21.15.98:0/3194120089 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45dc15c550 0x7f45c4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:07.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.715+0000 7f45db7fe640 1 --2- 172.21.15.98:0/3194120089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45c4004d50 0x7f45c40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:07.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.715+0000 7f45db7fe640 1 -- 172.21.15.98:0/3194120089 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc06c610 con 0x7f45c40039c0 2024-09-19T08:46:07.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.715+0000 7f45db7fe640 1 -- 172.21.15.98:0/3194120089 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc06d5f0 con 0x7f45c4004d50 2024-09-19T08:46:07.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.715+0000 7f45db7fe640 1 -- 172.21.15.98:0/3194120089 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f45c400b9d0 con 0x7f45dc15c550 2024-09-19T08:46:07.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.715+0000 7f45e08ca640 1 --2- 172.21.15.98:0/3194120089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45c40039c0 0x7f45c4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:07.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.719+0000 7f45dbfff640 1 --2- 172.21.15.98:0/3194120089 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45dc15c550 0x7f45c4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:07.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.719+0000 7f45e10cb640 1 --2- 172.21.15.98:0/3194120089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45c4004d50 0x7f45c40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:07.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.719+0000 7f45e08ca640 1 -- 172.21.15.98:0/3194120089 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45dc15c550 msgr2=0x7f45c4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:07.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.719+0000 7f45e08ca640 1 --2- 172.21.15.98:0/3194120089 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45dc15c550 0x7f45c4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:07.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.719+0000 7f45e08ca640 1 -- 172.21.15.98:0/3194120089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45c4004d50 msgr2=0x7f45c40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:07.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.719+0000 7f45e08ca640 1 --2- 172.21.15.98:0/3194120089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45c4004d50 0x7f45c40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:07.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.719+0000 7f45e08ca640 1 -- 172.21.15.98:0/3194120089 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c4003530 con 0x7f45c40039c0 2024-09-19T08:46:07.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.719+0000 7f45e10cb640 1 --2- 172.21.15.98:0/3194120089 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45c4004d50 0x7f45c40051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:07.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.719+0000 7f45dbfff640 1 --2- 172.21.15.98:0/3194120089 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45dc15c550 0x7f45c4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:07.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.719+0000 7f45e08ca640 1 --2- 172.21.15.98:0/3194120089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45c40039c0 0x7f45c4003de0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f45cc009ae0 tx=0x7f45cc01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c07dd8800a4fc7e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:07.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.719+0000 7f45db7fe640 1 -- 172.21.15.98:0/3194120089 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc0096b0 con 0x7f45c40039c0 2024-09-19T08:46:07.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.719+0000 7f45db7fe640 1 -- 172.21.15.98:0/3194120089 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45cc0a6450 con 0x7f45c40039c0 2024-09-19T08:46:07.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.719+0000 7f45db7fe640 1 -- 172.21.15.98:0/3194120089 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc0a6840 con 0x7f45c40039c0 2024-09-19T08:46:07.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.719+0000 7f45e2b2e640 1 -- 172.21.15.98:0/3194120089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45c40039c0 msgr2=0x7f45c4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:07.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.719+0000 7f45e2b2e640 1 --2- 172.21.15.98:0/3194120089 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45c40039c0 0x7f45c4003de0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f45cc009ae0 tx=0x7f45cc01ca70 comp rx=0 tx=0).stop 2024-09-19T08:46:07.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.719+0000 7f45e10cb640 1 -- 172.21.15.98:0/3194120089 reap_dead start 2024-09-19T08:46:07.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.719+0000 7f45e2b2e640 1 -- 172.21.15.98:0/3194120089 shutdown_connections 2024-09-19T08:46:07.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.719+0000 7f45e2b2e640 1 -- 172.21.15.98:0/3194120089 >> 172.21.15.98:0/3194120089 conn(0x7f45dc0b2010 msgr2=0x7f45dc0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:07.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.719+0000 7f45e2b2e640 1 -- 172.21.15.98:0/3194120089 shutdown_connections 2024-09-19T08:46:07.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.719+0000 7f45e2b2e640 1 -- 172.21.15.98:0/3194120089 wait complete. 2024-09-19T08:46:07.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.723+0000 7f45e2b2e640 1 Processor -- start 2024-09-19T08:46:07.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.723+0000 7f45e2b2e640 1 -- start start 2024-09-19T08:46:07.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.723+0000 7f45e2b2e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45c40039c0 0x7f45dc144e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:07.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.723+0000 7f45e2b2e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45dc1560a0 0x7f45dc153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:07.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.723+0000 7f45e2b2e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45dc153f60 0x7f45dc151de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:07.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.723+0000 7f45e2b2e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc073300 con 0x7f45c40039c0 2024-09-19T08:46:07.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.723+0000 7f45e2b2e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc15cb40 con 0x7f45dc1560a0 2024-09-19T08:46:07.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.723+0000 7f45e2b2e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f45dc073480 con 0x7f45dc153f60 2024-09-19T08:46:07.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.723+0000 7f45e08ca640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45c40039c0 0x7f45dc144e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:07.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.723+0000 7f45e08ca640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45c40039c0 0x7f45dc144e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46426/0 (socket says 172.21.15.98:46426) 2024-09-19T08:46:07.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.723+0000 7f45e08ca640 1 -- 172.21.15.98:0/3054623660 learned_addr learned my addr 172.21.15.98:0/3054623660 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:07.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.723+0000 7f45dbfff640 1 --2- 172.21.15.98:0/3054623660 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45dc1560a0 0x7f45dc153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:07.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.723+0000 7f45e08ca640 1 -- 172.21.15.98:0/3054623660 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45dc153f60 msgr2=0x7f45dc151de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:07.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.723+0000 7f45e08ca640 1 --2- 172.21.15.98:0/3054623660 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45dc153f60 0x7f45dc151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:07.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.723+0000 7f45e08ca640 1 -- 172.21.15.98:0/3054623660 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45dc1560a0 msgr2=0x7f45dc153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:07.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.723+0000 7f45e08ca640 1 --2- 172.21.15.98:0/3054623660 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45dc1560a0 0x7f45dc153a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:07.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.723+0000 7f45e08ca640 1 -- 172.21.15.98:0/3054623660 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45dc154400 con 0x7f45c40039c0 2024-09-19T08:46:07.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.723+0000 7f45e08ca640 1 --2- 172.21.15.98:0/3054623660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45c40039c0 0x7f45dc144e90 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f45cc01c8a0 tx=0x7f45c4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:07.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.723+0000 7f45d97fa640 1 -- 172.21.15.98:0/3054623660 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc0be070 con 0x7f45c40039c0 2024-09-19T08:46:07.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.723+0000 7f45d97fa640 1 -- 172.21.15.98:0/3054623660 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45cc00f000 con 0x7f45c40039c0 2024-09-19T08:46:07.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.723+0000 7f45e2b2e640 1 -- 172.21.15.98:0/3054623660 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45dc1546e0 con 0x7f45c40039c0 2024-09-19T08:46:07.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.723+0000 7f45d97fa640 1 -- 172.21.15.98:0/3054623660 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc0a6e20 con 0x7f45c40039c0 2024-09-19T08:46:07.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.723+0000 7f45e2b2e640 1 -- 172.21.15.98:0/3054623660 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45dc154ca0 con 0x7f45c40039c0 2024-09-19T08:46:07.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.723+0000 7f45e2b2e640 1 -- 172.21.15.98:0/3054623660 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45dc0723a0 con 0x7f45c40039c0 2024-09-19T08:46:07.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.723+0000 7f45d97fa640 1 -- 172.21.15.98:0/3054623660 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f45cc0c1070 con 0x7f45c40039c0 2024-09-19T08:46:07.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.723+0000 7f45d97fa640 1 --2- 172.21.15.98:0/3054623660 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f45c8067ef0 0x7f45c806a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:07.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.723+0000 7f45d97fa640 1 -- 172.21.15.98:0/3054623660 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5737+0+0 (secure 0 0 0) 0x7f45cc1200d0 con 0x7f45c40039c0 2024-09-19T08:46:07.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.727+0000 7f45dbfff640 1 --2- 172.21.15.98:0/3054623660 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f45c8067ef0 0x7f45c806a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:07.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.727+0000 7f45dbfff640 1 --2- 172.21.15.98:0/3054623660 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f45c8067ef0 0x7f45c806a3b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f45dc0b17d0 tx=0x7f45d0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:07.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.731+0000 7f45d97fa640 1 -- 172.21.15.98:0/3054623660 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f45cc0ac020 con 0x7f45c40039c0 2024-09-19T08:46:07.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.911+0000 7f45e2b2e640 1 -- 172.21.15.98:0/3054623660 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45dc0d06f0 con 0x7f45c40039c0 2024-09-19T08:46:07.912 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.915+0000 7f45d97fa640 1 -- 172.21.15.98:0/3054623660 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5221 (secure 0 0 0) 0x7f45cc016780 con 0x7f45c40039c0 2024-09-19T08:46:07.912 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:46:07.912 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":82,"btime":"2024-09-19T08:46:05:828977+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6886,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3238157999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3238157999},{"type":"v1","addr":"172.21.15.139:6833","nonce":3238157999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/209335177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":209335177},{"type":"v1","addr":"172.21.15.139:6835","nonce":209335177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8370,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6834/26914461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":26914461},{"type":"v1","addr":"172.21.15.98:6834","nonce":26914461}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:46:03.123986+0000","modified":"2024-09-19T08:46:05.150802+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8373":{"gid":8373,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/451004304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6833","nonce":451004304},{"type":"v1","addr":"172.21.15.98:6835","nonce":451004304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8373,"qdb_cluster":[8373]},"id":11}]} 2024-09-19T08:46:07.913 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 82 2024-09-19T08:46:07.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.915+0000 7f45e2b2e640 1 -- 172.21.15.98:0/3054623660 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f45c8067ef0 msgr2=0x7f45c806a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:07.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.915+0000 7f45e2b2e640 1 --2- 172.21.15.98:0/3054623660 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f45c8067ef0 0x7f45c806a3b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f45dc0b17d0 tx=0x7f45d0009040 comp rx=0 tx=0).stop 2024-09-19T08:46:07.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.915+0000 7f45e2b2e640 1 -- 172.21.15.98:0/3054623660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45c40039c0 msgr2=0x7f45dc144e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:07.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.915+0000 7f45e2b2e640 1 --2- 172.21.15.98:0/3054623660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45c40039c0 0x7f45dc144e90 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f45cc01c8a0 tx=0x7f45c4005840 comp rx=0 tx=0).stop 2024-09-19T08:46:07.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.915+0000 7f45e2b2e640 1 -- 172.21.15.98:0/3054623660 shutdown_connections 2024-09-19T08:46:07.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.915+0000 7f45e2b2e640 1 --2- 172.21.15.98:0/3054623660 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f45c8067ef0 0x7f45c806a3b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:07.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.915+0000 7f45e2b2e640 1 --2- 172.21.15.98:0/3054623660 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45dc153f60 0x7f45dc151de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:07.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.915+0000 7f45e2b2e640 1 --2- 172.21.15.98:0/3054623660 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45dc1560a0 0x7f45dc153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:07.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.915+0000 7f45e2b2e640 1 --2- 172.21.15.98:0/3054623660 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45c40039c0 0x7f45dc144e90 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:07.915 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.915+0000 7f45e2b2e640 1 -- 172.21.15.98:0/3054623660 >> 172.21.15.98:0/3054623660 conn(0x7f45dc0b2010 msgr2=0x7f45dc073890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:07.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.915+0000 7f45e2b2e640 1 -- 172.21.15.98:0/3054623660 shutdown_connections 2024-09-19T08:46:07.916 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:07.915+0000 7f45e2b2e640 1 -- 172.21.15.98:0/3054623660 wait complete. 2024-09-19T08:46:08.332 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 82, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T08:46:03.123986+0000', 'modified': '2024-09-19T08:46:05.150802+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8373}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8373': {'gid': 8373, 'name': 'a', 'rank': 0, 'incarnation': 80, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.98:6835/451004304', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6833', 'nonce': 451004304}, {'type': 'v1', 'addr': '172.21.15.98:6835', 'nonce': 451004304}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8373, 'qdb_cluster': [8373]} 2024-09-19T08:46:08.332 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T08:46:08.333 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-19T08:46:08.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.463+0000 7f40e8f35640 1 Processor -- start 2024-09-19T08:46:08.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.463+0000 7f40e8f35640 1 -- start start 2024-09-19T08:46:08.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.463+0000 7f40e8f35640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40e406bc90 0x7f40e406c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:08.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.463+0000 7f40e8f35640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40e40ccc60 con 0x7f40e4158fd0 2024-09-19T08:46:08.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.463+0000 7f40e8f35640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40e40ccf40 con 0x7f40e40ce840 2024-09-19T08:46:08.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.463+0000 7f40e8f35640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f40e41593b0 con 0x7f40e406bc90 2024-09-19T08:46:08.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.463+0000 7f40e259c640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f40e4158fd0 0x7f40e40b3e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36882/0 (socket says 172.21.15.98:36882) 2024-09-19T08:46:08.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.463+0000 7f40e259c640 1 -- 172.21.15.98:0/4279274638 learned_addr learned my addr 172.21.15.98:0/4279274638 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:08.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.463+0000 7f40e1d9b640 1 --2- 172.21.15.98:0/4279274638 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40e406bc90 0x7f40e406c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:08.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.463+0000 7f40e159a640 1 -- 172.21.15.98:0/4279274638 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3210255162 0 0) 0x7f40e40ccc60 con 0x7f40e4158fd0 2024-09-19T08:46:08.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.463+0000 7f40e159a640 1 -- 172.21.15.98:0/4279274638 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40cc003530 con 0x7f40e4158fd0 2024-09-19T08:46:08.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.463+0000 7f40e159a640 1 -- 172.21.15.98:0/4279274638 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2455172095 0 0) 0x7f40cc003530 con 0x7f40e4158fd0 2024-09-19T08:46:08.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.463+0000 7f40e159a640 1 -- 172.21.15.98:0/4279274638 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40e406bc90 msgr2=0x7f40e406c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:08.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.463+0000 7f40e159a640 1 --2- 172.21.15.98:0/4279274638 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40e406bc90 0x7f40e406c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.463+0000 7f40e159a640 1 -- 172.21.15.98:0/4279274638 >> v1:172.21.15.139:6789/0 conn(0x7f40e40ce840 legacy=0x7f40e40cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:08.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.463+0000 7f40e159a640 1 -- 172.21.15.98:0/4279274638 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40e40cd220 con 0x7f40e4158fd0 2024-09-19T08:46:08.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.463+0000 7f40e159a640 1 -- 172.21.15.98:0/4279274638 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f40d4001a00 con 0x7f40e4158fd0 2024-09-19T08:46:08.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.463+0000 7f40e159a640 1 -- 172.21.15.98:0/4279274638 >> v1:172.21.15.98:6789/0 conn(0x7f40e4158fd0 legacy=0x7f40e40b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:08.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.463+0000 7f40e159a640 1 --2- 172.21.15.98:0/4279274638 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40e406bc90 0x7f40cc0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:08.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.467+0000 7f40e159a640 1 --2- 172.21.15.98:0/4279274638 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40cc003f70 0x7f40cc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:08.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.467+0000 7f40e159a640 1 --2- 172.21.15.98:0/4279274638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40cc004d50 0x7f40cc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:08.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.467+0000 7f40e159a640 1 -- 172.21.15.98:0/4279274638 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f40e41593b0 con 0x7f40cc003f70 2024-09-19T08:46:08.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.467+0000 7f40e159a640 1 -- 172.21.15.98:0/4279274638 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc00bb60 con 0x7f40cc004d50 2024-09-19T08:46:08.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.467+0000 7f40e159a640 1 -- 172.21.15.98:0/4279274638 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f40cc00bd60 con 0x7f40e406bc90 2024-09-19T08:46:08.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.467+0000 7f40e1d9b640 1 --2- 172.21.15.98:0/4279274638 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40cc003f70 0x7f40cc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:08.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.467+0000 7f40e259c640 1 --2- 172.21.15.98:0/4279274638 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40e406bc90 0x7f40cc0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:08.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.467+0000 7f40e2d9d640 1 --2- 172.21.15.98:0/4279274638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40cc004d50 0x7f40cc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:08.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.467+0000 7f40e1d9b640 1 -- 172.21.15.98:0/4279274638 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40e406bc90 msgr2=0x7f40cc0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:08.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.467+0000 7f40e1d9b640 1 --2- 172.21.15.98:0/4279274638 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40e406bc90 0x7f40cc0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.467+0000 7f40e1d9b640 1 -- 172.21.15.98:0/4279274638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40cc004d50 msgr2=0x7f40cc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:08.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.467+0000 7f40e1d9b640 1 --2- 172.21.15.98:0/4279274638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40cc004d50 0x7f40cc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.467+0000 7f40e1d9b640 1 -- 172.21.15.98:0/4279274638 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40cc003530 con 0x7f40cc003f70 2024-09-19T08:46:08.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.467+0000 7f40e2d9d640 1 --2- 172.21.15.98:0/4279274638 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40cc004d50 0x7f40cc0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:08.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.467+0000 7f40e1d9b640 1 --2- 172.21.15.98:0/4279274638 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40cc003f70 0x7f40cc004810 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f40d8009ac0 tx=0x7f40d801ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b1619b143a49cc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:08.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.467+0000 7f40e159a640 1 -- 172.21.15.98:0/4279274638 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d80b2070 con 0x7f40cc003f70 2024-09-19T08:46:08.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.467+0000 7f40e159a640 1 -- 172.21.15.98:0/4279274638 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40d80a5590 con 0x7f40cc003f70 2024-09-19T08:46:08.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.467+0000 7f40e159a640 1 -- 172.21.15.98:0/4279274638 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d8009690 con 0x7f40cc003f70 2024-09-19T08:46:08.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.467+0000 7f40e8f35640 1 -- 172.21.15.98:0/4279274638 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40cc003f70 msgr2=0x7f40cc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:08.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.467+0000 7f40e8f35640 1 --2- 172.21.15.98:0/4279274638 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40cc003f70 0x7f40cc004810 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f40d8009ac0 tx=0x7f40d801ca20 comp rx=0 tx=0).stop 2024-09-19T08:46:08.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.467+0000 7f40e2d9d640 1 -- 172.21.15.98:0/4279274638 reap_dead start 2024-09-19T08:46:08.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.467+0000 7f40e8f35640 1 -- 172.21.15.98:0/4279274638 shutdown_connections 2024-09-19T08:46:08.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.467+0000 7f40e8f35640 1 -- 172.21.15.98:0/4279274638 >> 172.21.15.98:0/4279274638 conn(0x7f40e40b2010 msgr2=0x7f40e40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:08.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.471+0000 7f40e8f35640 1 -- 172.21.15.98:0/4279274638 shutdown_connections 2024-09-19T08:46:08.468 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.471+0000 7f40e8f35640 1 -- 172.21.15.98:0/4279274638 wait complete. 2024-09-19T08:46:08.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.471+0000 7f40e8f35640 1 Processor -- start 2024-09-19T08:46:08.469 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.471+0000 7f40e8f35640 1 -- start start 2024-09-19T08:46:08.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.471+0000 7f40e8f35640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40cc003f70 0x7f40e415d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:08.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.471+0000 7f40e8f35640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40e4148960 0x7f40e4073850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:08.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.471+0000 7f40e8f35640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40e4073d90 0x7f40e40c52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:08.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.471+0000 7f40e8f35640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f40e406c600 con 0x7f40e4073d90 2024-09-19T08:46:08.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.471+0000 7f40e8f35640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f40e406c480 con 0x7f40cc003f70 2024-09-19T08:46:08.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.471+0000 7f40e8f35640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f40e415c7a0 con 0x7f40e4148960 2024-09-19T08:46:08.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.471+0000 7f40e2d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40e4073d90 0x7f40e40c52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:08.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.471+0000 7f40e2d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40e4073d90 0x7f40e40c52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46448/0 (socket says 172.21.15.98:46448) 2024-09-19T08:46:08.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.471+0000 7f40e2d9d640 1 -- 172.21.15.98:0/1898701005 learned_addr learned my addr 172.21.15.98:0/1898701005 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:08.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.471+0000 7f40e259c640 1 --2- 172.21.15.98:0/1898701005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40cc003f70 0x7f40e415d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:08.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.471+0000 7f40e1d9b640 1 --2- 172.21.15.98:0/1898701005 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40e4148960 0x7f40e4073850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:08.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.471+0000 7f40e2d9d640 1 -- 172.21.15.98:0/1898701005 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40e4148960 msgr2=0x7f40e4073850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:08.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.471+0000 7f40e2d9d640 1 --2- 172.21.15.98:0/1898701005 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40e4148960 0x7f40e4073850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.471+0000 7f40e2d9d640 1 -- 172.21.15.98:0/1898701005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40cc003f70 msgr2=0x7f40e415d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:08.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.471+0000 7f40e2d9d640 1 --2- 172.21.15.98:0/1898701005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40cc003f70 0x7f40e415d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.471+0000 7f40e2d9d640 1 -- 172.21.15.98:0/1898701005 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40e41492f0 con 0x7f40e4073d90 2024-09-19T08:46:08.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.471+0000 7f40e259c640 1 --2- 172.21.15.98:0/1898701005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40cc003f70 0x7f40e415d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:08.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.471+0000 7f40e2d9d640 1 --2- 172.21.15.98:0/1898701005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40e4073d90 0x7f40e40c52f0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f40e40d12d0 tx=0x7f40dc00d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:08.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.471+0000 7f40d2ffd640 1 -- 172.21.15.98:0/1898701005 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc005480 con 0x7f40e4073d90 2024-09-19T08:46:08.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.471+0000 7f40d2ffd640 1 -- 172.21.15.98:0/1898701005 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40dc007030 con 0x7f40e4073d90 2024-09-19T08:46:08.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.471+0000 7f40e8f35640 1 -- 172.21.15.98:0/1898701005 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40e41495d0 con 0x7f40e4073d90 2024-09-19T08:46:08.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.471+0000 7f40d2ffd640 1 -- 172.21.15.98:0/1898701005 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc007400 con 0x7f40e4073d90 2024-09-19T08:46:08.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.471+0000 7f40e8f35640 1 -- 172.21.15.98:0/1898701005 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40e4149b90 con 0x7f40e4073d90 2024-09-19T08:46:08.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.475+0000 7f40d2ffd640 1 -- 172.21.15.98:0/1898701005 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f40dc01b020 con 0x7f40e4073d90 2024-09-19T08:46:08.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.475+0000 7f40d2ffd640 1 --2- 172.21.15.98:0/1898701005 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f40c4067ef0 0x7f40c406a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:08.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.475+0000 7f40d2ffd640 1 -- 172.21.15.98:0/1898701005 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5737+0+0 (secure 0 0 0) 0x7f40dc086270 con 0x7f40e4073d90 2024-09-19T08:46:08.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.475+0000 7f40e259c640 1 --2- 172.21.15.98:0/1898701005 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f40c4067ef0 0x7f40c406a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:08.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.475+0000 7f40ebff1000 1 Processor -- start 2024-09-19T08:46:08.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.475+0000 7f40ebff1000 1 -- start start 2024-09-19T08:46:08.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.475+0000 7f40e259c640 1 --2- 172.21.15.98:0/1898701005 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f40c4067ef0 0x7f40c406a3b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f40d4004d10 tx=0x7f40d4006040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:08.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.475+0000 7f40ebff1000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcbee50b20 0x55bcbee50f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:08.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.475+0000 7f40ebff1000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcbee51440 0x55bcbee06d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:08.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.475+0000 7f40ebff1000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55bcbee07290 0x55bcbee07670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:08.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.475+0000 7f40ebff1000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55bcbeceeb60 con 0x55bcbee51440 2024-09-19T08:46:08.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.475+0000 7f40ebff1000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55bcbec55c70 con 0x55bcbee50b20 2024-09-19T08:46:08.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.475+0000 7f40ebff1000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x55bcbebfbed0 con 0x55bcbee07290 2024-09-19T08:46:08.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.475+0000 7f40e1d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcbee51440 0x55bcbee06d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:08.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.475+0000 7f40e1d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcbee51440 0x55bcbee06d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46456/0 (socket says 172.21.15.98:46456) 2024-09-19T08:46:08.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.475+0000 7f40e1d9b640 1 -- 172.21.15.98:0/1727040388 learned_addr learned my addr 172.21.15.98:0/1727040388 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:08.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.475+0000 7f40e259c640 1 --2- 172.21.15.98:0/1727040388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcbee50b20 0x55bcbee50f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:08.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.475+0000 7f40e2d9d640 1 --2- 172.21.15.98:0/1727040388 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55bcbee07290 0x55bcbee07670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:08.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.475+0000 7f40e1d9b640 1 -- 172.21.15.98:0/1727040388 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55bcbee07290 msgr2=0x55bcbee07670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:08.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.475+0000 7f40e1d9b640 1 --2- 172.21.15.98:0/1727040388 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55bcbee07290 0x55bcbee07670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.475+0000 7f40e1d9b640 1 -- 172.21.15.98:0/1727040388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcbee50b20 msgr2=0x55bcbee50f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:08.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.475+0000 7f40e1d9b640 1 --2- 172.21.15.98:0/1727040388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcbee50b20 0x55bcbee50f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.475+0000 7f40e1d9b640 1 -- 172.21.15.98:0/1727040388 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcbec0de50 con 0x55bcbee51440 2024-09-19T08:46:08.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.475+0000 7f40e1d9b640 1 --2- 172.21.15.98:0/1727040388 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcbee51440 0x55bcbee06d50 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f40d80b0040 tx=0x7f40d8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65628a93bb673960 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:08.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.475+0000 7f40c3fff640 1 -- 172.21.15.98:0/1727040388 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d80bd070 con 0x55bcbee51440 2024-09-19T08:46:08.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.475+0000 7f40c3fff640 1 -- 172.21.15.98:0/1727040388 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40d80a5790 con 0x55bcbee51440 2024-09-19T08:46:08.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.475+0000 7f40c3fff640 1 -- 172.21.15.98:0/1727040388 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d80b2040 con 0x55bcbee51440 2024-09-19T08:46:08.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.475+0000 7f40ebff1000 1 -- 172.21.15.98:0/1727040388 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcbee51440 msgr2=0x55bcbee06d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:08.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.475+0000 7f40ebff1000 1 --2- 172.21.15.98:0/1727040388 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcbee51440 0x55bcbee06d50 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f40d80b0040 tx=0x7f40d8004740 comp rx=0 tx=0).stop 2024-09-19T08:46:08.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40ebff1000 1 -- 172.21.15.98:0/1727040388 shutdown_connections 2024-09-19T08:46:08.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40ebff1000 1 --2- 172.21.15.98:0/1727040388 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55bcbee07290 0x55bcbee07670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40ebff1000 1 --2- 172.21.15.98:0/1727040388 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcbee51440 0x55bcbee06d50 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40ebff1000 1 --2- 172.21.15.98:0/1727040388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcbee50b20 0x55bcbee50f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40ebff1000 1 -- 172.21.15.98:0/1727040388 >> 172.21.15.98:0/1727040388 conn(0x55bcbee0d600 msgr2=0x55bcbee05630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:08.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40ebff1000 1 -- 172.21.15.98:0/1727040388 shutdown_connections 2024-09-19T08:46:08.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40ebff1000 1 -- 172.21.15.98:0/1727040388 wait complete. 2024-09-19T08:46:08.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40ebff1000 1 -- start start 2024-09-19T08:46:08.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40ebff1000 1 Processor -- start 2024-09-19T08:46:08.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40ebff1000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55bcbee50b20 0x55bcbee0be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:08.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40ebff1000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcbee51440 0x55bcbee11cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:08.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40ebff1000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcbee0c390 0x55bcbee12230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:08.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40ebff1000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55bcbecb7b00 con 0x55bcbee0c390 2024-09-19T08:46:08.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40ebff1000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55bcbec30360 con 0x55bcbee51440 2024-09-19T08:46:08.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40ebff1000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x55bcbec2d2d0 con 0x55bcbee50b20 2024-09-19T08:46:08.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40ebff1000 10 client.0 _ll_register_callbacks cb 0x55bcbedf2820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:08.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40e2d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcbee0c390 0x55bcbee12230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:08.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40e2d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcbee0c390 0x55bcbee12230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46472/0 (socket says 172.21.15.98:46472) 2024-09-19T08:46:08.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40e2d9d640 1 -- 172.21.15.98:0/1040445873 learned_addr learned my addr 172.21.15.98:0/1040445873 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:08.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40e1d9b640 1 --2- 172.21.15.98:0/1040445873 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcbee51440 0x55bcbee11cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:08.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40e259c640 1 --2- 172.21.15.98:0/1040445873 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55bcbee50b20 0x55bcbee0be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:08.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40e2d9d640 1 -- 172.21.15.98:0/1040445873 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55bcbee50b20 msgr2=0x55bcbee0be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:08.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40e2d9d640 1 --2- 172.21.15.98:0/1040445873 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55bcbee50b20 0x55bcbee0be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40e2d9d640 1 -- 172.21.15.98:0/1040445873 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcbee51440 msgr2=0x55bcbee11cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:08.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40e2d9d640 1 --2- 172.21.15.98:0/1040445873 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcbee51440 0x55bcbee11cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40e2d9d640 1 -- 172.21.15.98:0/1040445873 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcbebfc120 con 0x55bcbee0c390 2024-09-19T08:46:08.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40e259c640 1 --2- 172.21.15.98:0/1040445873 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55bcbee50b20 0x55bcbee0be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:08.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40e2d9d640 1 --2- 172.21.15.98:0/1040445873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcbee0c390 0x55bcbee12230 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f40dc084090 tx=0x7f40dc011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d51e9f4a46ce9a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:08.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-19T08:46:08.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40c17fa640 1 -- 172.21.15.98:0/1040445873 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc06c030 con 0x55bcbee0c390 2024-09-19T08:46:08.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40ebff1000 1 -- 172.21.15.98:0/1040445873 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40d80a5790 con 0x55bcbee0c390 2024-09-19T08:46:08.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40c17fa640 1 -- 172.21.15.98:0/1040445873 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 5737+0+0 (secure 0 0 0) 0x7f40dc07ed60 con 0x55bcbee0c390 2024-09-19T08:46:08.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40c17fa640 1 -- 172.21.15.98:0/1040445873 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f40dc05e820 con 0x55bcbee0c390 2024-09-19T08:46:08.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40ebff1000 10 client.8508 fetch_fsmap learned FSMap version 82 2024-09-19T08:46:08.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40ebff1000 1 -- 172.21.15.98:0/1040445873 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x55bcbebf5450 con 0x55bcbee0c390 2024-09-19T08:46:08.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40c17fa640 1 -- 172.21.15.98:0/1040445873 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40d80a5790 con 0x55bcbee0c390 2024-09-19T08:46:08.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40c17fa640 1 -- 172.21.15.98:0/1040445873 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 82) ==== 3502+0+0 (secure 0 0 0) 0x7f40dc05e390 con 0x55bcbee0c390 2024-09-19T08:46:08.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40ebff1000 10 client.8508 fetch_fsmap finished waiting for FSMap version 82 2024-09-19T08:46:08.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40ebff1000 10 client.8508 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T08:46:08.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40ebff1000 20 client.8508 populate_metadata read hostname 'smithi098' 2024-09-19T08:46:08.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40ebff1000 1 --2- 172.21.15.98:0/1040445873 >> [v2:172.21.15.98:6833/451004304,v1:172.21.15.98:6835/451004304] conn(0x55bcbee5f280 0x55bcbee61670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:08.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40ebff1000 4 client.8508 mds_command: new command op to 8373 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:46:08.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40ebff1000 1 -- 172.21.15.98:0/1040445873 --> [v2:172.21.15.98:6833/451004304,v1:172.21.15.98:6835/451004304] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcbebefc00 con 0x55bcbee5f280 2024-09-19T08:46:08.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40e259c640 1 --2- 172.21.15.98:0/1040445873 >> [v2:172.21.15.98:6833/451004304,v1:172.21.15.98:6835/451004304] conn(0x55bcbee5f280 0x55bcbee61670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:08.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40e259c640 1 --2- 172.21.15.98:0/1040445873 >> [v2:172.21.15.98:6833/451004304,v1:172.21.15.98:6835/451004304] conn(0x55bcbee5f280 0x55bcbee61670 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c88a22fba7d82d64 server_cookie=9269294e228e91c in_seq=0 out_seq=0 2024-09-19T08:46:08.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40c17fa640 10 client.8508 ms_handle_connect on v2:172.21.15.98:6833/451004304 2024-09-19T08:46:08.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40c17fa640 1 -- 172.21.15.98:0/1040445873 <== mds.0 v2:172.21.15.98:6833/451004304 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bcbebefc00 con 0x55bcbee5f280 2024-09-19T08:46:08.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40c17fa640 10 client.8508 handle_command_reply: tid=0 2024-09-19T08:46:08.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.479+0000 7f40ebff1000 1 client.8508 shutdown 2024-09-19T08:46:08.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.483+0000 7f40ebff1000 1 -- 172.21.15.98:0/1040445873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcbee0c390 msgr2=0x55bcbee12230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:08.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.483+0000 7f40ebff1000 1 --2- 172.21.15.98:0/1040445873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcbee0c390 0x55bcbee12230 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f40dc084090 tx=0x7f40dc011ed0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.483+0000 7f40ebff1000 1 -- 172.21.15.98:0/1040445873 shutdown_connections 2024-09-19T08:46:08.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.483+0000 7f40ebff1000 1 --2- 172.21.15.98:0/1040445873 >> [v2:172.21.15.98:6833/451004304,v1:172.21.15.98:6835/451004304] conn(0x55bcbee5f280 0x55bcbee61670 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.483+0000 7f40ebff1000 1 --2- 172.21.15.98:0/1040445873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcbee0c390 0x55bcbee12230 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.483+0000 7f40ebff1000 1 --2- 172.21.15.98:0/1040445873 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcbee51440 0x55bcbee11cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.483+0000 7f40ebff1000 1 --2- 172.21.15.98:0/1040445873 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55bcbee50b20 0x55bcbee0be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.483+0000 7f40ebff1000 1 -- 172.21.15.98:0/1040445873 >> 172.21.15.98:0/1040445873 conn(0x55bcbee07290 msgr2=0x55bcbee05d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:08.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.483+0000 7f40c17fa640 0 client.8508 ms_handle_reset on v2:172.21.15.98:6833/451004304 2024-09-19T08:46:08.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.483+0000 7f40ebff1000 1 -- 172.21.15.98:0/1040445873 shutdown_connections 2024-09-19T08:46:08.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.483+0000 7f40ebff1000 1 -- 172.21.15.98:0/1040445873 wait complete. 2024-09-19T08:46:08.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.483+0000 7f40ebff1000 20 client.8508 trim_cache size 0 max 16384 2024-09-19T08:46:08.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.499+0000 7f40ebff1000 1 Processor -- start 2024-09-19T08:46:08.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.499+0000 7f40ebff1000 1 -- start start 2024-09-19T08:46:08.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.499+0000 7f40ebff1000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcbee07290 0x55bcbee4fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:08.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.499+0000 7f40ebff1000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55bcbee0c390 0x55bcbee50230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:08.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.499+0000 7f40ebff1000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcbee50b20 0x55bcbee61290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:08.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.499+0000 7f40ebff1000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f40e42034b0 con 0x55bcbee50b20 2024-09-19T08:46:08.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.499+0000 7f40ebff1000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f40e40af430 con 0x55bcbee07290 2024-09-19T08:46:08.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.499+0000 7f40ebff1000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f40e4202e90 con 0x55bcbee0c390 2024-09-19T08:46:08.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.499+0000 7f40e2d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcbee50b20 0x55bcbee61290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:08.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.499+0000 7f40e2d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcbee50b20 0x55bcbee61290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46484/0 (socket says 172.21.15.98:46484) 2024-09-19T08:46:08.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.499+0000 7f40e2d9d640 1 -- 172.21.15.98:0/1777776420 learned_addr learned my addr 172.21.15.98:0/1777776420 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:08.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40e2d9d640 1 -- 172.21.15.98:0/1777776420 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55bcbee0c390 msgr2=0x55bcbee50230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:08.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40e2d9d640 1 --2- 172.21.15.98:0/1777776420 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55bcbee0c390 0x55bcbee50230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40e2d9d640 1 -- 172.21.15.98:0/1777776420 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcbee07290 msgr2=0x55bcbee4fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:08.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40e2d9d640 1 --2- 172.21.15.98:0/1777776420 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcbee07290 0x55bcbee4fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40e2d9d640 1 -- 172.21.15.98:0/1777776420 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcbebdcfd0 con 0x55bcbee50b20 2024-09-19T08:46:08.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40e2d9d640 1 --2- 172.21.15.98:0/1777776420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcbee50b20 0x55bcbee61290 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f40dc055bd0 tx=0x7f40dc059000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76abdb92bf0b7377 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:08.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40c37fe640 1 -- 172.21.15.98:0/1777776420 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc06c040 con 0x55bcbee50b20 2024-09-19T08:46:08.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40c37fe640 1 -- 172.21.15.98:0/1777776420 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40dc05c040 con 0x55bcbee50b20 2024-09-19T08:46:08.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40c37fe640 1 -- 172.21.15.98:0/1777776420 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc083d70 con 0x55bcbee50b20 2024-09-19T08:46:08.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40ebff1000 1 -- 172.21.15.98:0/1777776420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcbee50b20 msgr2=0x55bcbee61290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:08.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40ebff1000 1 --2- 172.21.15.98:0/1777776420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcbee50b20 0x55bcbee61290 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f40dc055bd0 tx=0x7f40dc059000 comp rx=0 tx=0).stop 2024-09-19T08:46:08.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40ebff1000 1 -- 172.21.15.98:0/1777776420 shutdown_connections 2024-09-19T08:46:08.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40ebff1000 1 --2- 172.21.15.98:0/1777776420 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcbee50b20 0x55bcbee61290 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40ebff1000 1 --2- 172.21.15.98:0/1777776420 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55bcbee0c390 0x55bcbee50230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40ebff1000 1 --2- 172.21.15.98:0/1777776420 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcbee07290 0x55bcbee4fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40ebff1000 1 -- 172.21.15.98:0/1777776420 >> 172.21.15.98:0/1777776420 conn(0x55bcbee5c270 msgr2=0x55bcbee0b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:08.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40ebff1000 1 -- 172.21.15.98:0/1777776420 shutdown_connections 2024-09-19T08:46:08.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40ebff1000 1 -- 172.21.15.98:0/1777776420 wait complete. 2024-09-19T08:46:08.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40ebff1000 1 -- start start 2024-09-19T08:46:08.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40ebff1000 1 Processor -- start 2024-09-19T08:46:08.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40ebff1000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55bcbee07290 0x55bcbee64120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:08.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40ebff1000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcbee0c390 0x55bcbee88170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:08.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40ebff1000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcbee50b20 0x55bcbee886b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:08.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40ebff1000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55bcbec54f80 con 0x55bcbee50b20 2024-09-19T08:46:08.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40ebff1000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55bcbec0dca0 con 0x55bcbee0c390 2024-09-19T08:46:08.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40ebff1000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x55bcbec5a2a0 con 0x55bcbee07290 2024-09-19T08:46:08.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40ebff1000 10 client.0 _ll_register_callbacks cb 0x7f40dc051f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:08.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40e2d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcbee50b20 0x55bcbee886b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:08.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40e2d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcbee50b20 0x55bcbee886b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46494/0 (socket says 172.21.15.98:46494) 2024-09-19T08:46:08.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40e2d9d640 1 -- 172.21.15.98:0/2049476341 learned_addr learned my addr 172.21.15.98:0/2049476341 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:08.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40e1d9b640 1 --2- 172.21.15.98:0/2049476341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcbee0c390 0x55bcbee88170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:08.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40e259c640 1 --2- 172.21.15.98:0/2049476341 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55bcbee07290 0x55bcbee64120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:08.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40e2d9d640 1 -- 172.21.15.98:0/2049476341 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55bcbee07290 msgr2=0x55bcbee64120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:08.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40e2d9d640 1 --2- 172.21.15.98:0/2049476341 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55bcbee07290 0x55bcbee64120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40e2d9d640 1 -- 172.21.15.98:0/2049476341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcbee0c390 msgr2=0x55bcbee88170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:08.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40e2d9d640 1 --2- 172.21.15.98:0/2049476341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcbee0c390 0x55bcbee88170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40e2d9d640 1 -- 172.21.15.98:0/2049476341 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcbecd8750 con 0x55bcbee50b20 2024-09-19T08:46:08.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40e259c640 1 --2- 172.21.15.98:0/2049476341 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55bcbee07290 0x55bcbee64120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:08.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40e2d9d640 1 --2- 172.21.15.98:0/2049476341 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcbee50b20 0x55bcbee886b0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f40dc05e9e0 tx=0x7f40dc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8bdceb3e7a6964f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:08.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-19T08:46:08.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40c17fa640 1 -- 172.21.15.98:0/2049476341 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc06c070 con 0x55bcbee50b20 2024-09-19T08:46:08.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40c17fa640 1 -- 172.21.15.98:0/2049476341 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 5737+0+0 (secure 0 0 0) 0x7f40dc075c20 con 0x55bcbee50b20 2024-09-19T08:46:08.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40ebff1000 1 -- 172.21.15.98:0/2049476341 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40dc05c040 con 0x55bcbee50b20 2024-09-19T08:46:08.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40c17fa640 1 -- 172.21.15.98:0/2049476341 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f40dc075e60 con 0x55bcbee50b20 2024-09-19T08:46:08.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40ebff1000 10 client.8514 fetch_fsmap learned FSMap version 82 2024-09-19T08:46:08.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40ebff1000 1 -- 172.21.15.98:0/2049476341 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) -- 0x55bcbebe3a30 con 0x55bcbee50b20 2024-09-19T08:46:08.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40c17fa640 1 -- 172.21.15.98:0/2049476341 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40dc05c040 con 0x55bcbee50b20 2024-09-19T08:46:08.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40c17fa640 1 -- 172.21.15.98:0/2049476341 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 82) ==== 3502+0+0 (secure 0 0 0) 0x7f40dc05c1f0 con 0x55bcbee50b20 2024-09-19T08:46:08.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40ebff1000 10 client.8514 fetch_fsmap finished waiting for FSMap version 82 2024-09-19T08:46:08.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40ebff1000 10 client.8514 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T08:46:08.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40ebff1000 20 client.8514 populate_metadata read hostname 'smithi098' 2024-09-19T08:46:08.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40ebff1000 1 --2- 172.21.15.98:0/2049476341 >> [v2:172.21.15.98:6833/451004304,v1:172.21.15.98:6835/451004304] conn(0x55bcbee51440 0x55bcbee88bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:08.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40ebff1000 4 client.8514 mds_command: new command op to 8373 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T08:46:08.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40ebff1000 1 -- 172.21.15.98:0/2049476341 --> [v2:172.21.15.98:6833/451004304,v1:172.21.15.98:6835/451004304] -- command(tid 0: {"prefix": "status"}) -- 0x55bcbec633a0 con 0x55bcbee51440 2024-09-19T08:46:08.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.503+0000 7f40e259c640 1 --2- 172.21.15.98:0/2049476341 >> [v2:172.21.15.98:6833/451004304,v1:172.21.15.98:6835/451004304] conn(0x55bcbee51440 0x55bcbee88bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:08.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.507+0000 7f40c17fa640 10 client.8514 ms_handle_connect on v2:172.21.15.98:6833/451004304 2024-09-19T08:46:08.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.507+0000 7f40e259c640 1 --2- 172.21.15.98:0/2049476341 >> [v2:172.21.15.98:6833/451004304,v1:172.21.15.98:6835/451004304] conn(0x55bcbee51440 0x55bcbee88bf0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=551ae8b9dedfd1a9 server_cookie=41dcda2a8d7c19f1 in_seq=0 out_seq=0 2024-09-19T08:46:08.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.507+0000 7f40c17fa640 1 -- 172.21.15.98:0/2049476341 <== mds.0 v2:172.21.15.98:6833/451004304 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55bcbec633a0 con 0x55bcbee51440 2024-09-19T08:46:08.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.507+0000 7f40c17fa640 10 client.8514 handle_command_reply: tid=0 2024-09-19T08:46:08.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.507+0000 7f40ebff1000 1 client.8514 shutdown 2024-09-19T08:46:08.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.507+0000 7f40ebff1000 1 -- 172.21.15.98:0/2049476341 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcbee50b20 msgr2=0x55bcbee886b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:08.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.507+0000 7f40ebff1000 1 --2- 172.21.15.98:0/2049476341 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcbee50b20 0x55bcbee886b0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f40dc05e9e0 tx=0x7f40dc01a000 comp rx=0 tx=0).stop 2024-09-19T08:46:08.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.507+0000 7f40ebff1000 1 -- 172.21.15.98:0/2049476341 shutdown_connections 2024-09-19T08:46:08.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.507+0000 7f40ebff1000 1 --2- 172.21.15.98:0/2049476341 >> [v2:172.21.15.98:6833/451004304,v1:172.21.15.98:6835/451004304] conn(0x55bcbee51440 0x55bcbee88bf0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.507+0000 7f40ebff1000 1 --2- 172.21.15.98:0/2049476341 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55bcbee50b20 0x55bcbee886b0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.507+0000 7f40c17fa640 0 client.8514 ms_handle_reset on v2:172.21.15.98:6833/451004304 2024-09-19T08:46:08.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.507+0000 7f40ebff1000 1 --2- 172.21.15.98:0/2049476341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55bcbee0c390 0x55bcbee88170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.507+0000 7f40ebff1000 1 --2- 172.21.15.98:0/2049476341 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55bcbee07290 0x55bcbee64120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.507+0000 7f40ebff1000 1 -- 172.21.15.98:0/2049476341 >> 172.21.15.98:0/2049476341 conn(0x55bcbee5c270 msgr2=0x55bcbee4ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:08.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.507+0000 7f40ebff1000 1 -- 172.21.15.98:0/2049476341 shutdown_connections 2024-09-19T08:46:08.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.507+0000 7f40ebff1000 1 -- 172.21.15.98:0/2049476341 wait complete. 2024-09-19T08:46:08.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.507+0000 7f40ebff1000 20 client.8514 trim_cache size 0 max 16384 2024-09-19T08:46:08.511 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:46:08.512 INFO:teuthology.orchestra.run.smithi098.stdout: "cluster_fsid": "552bf0ee-1f4f-4644-ba7c-d0931e55a451", 2024-09-19T08:46:08.512 INFO:teuthology.orchestra.run.smithi098.stdout: "whoami": 0, 2024-09-19T08:46:08.512 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 8373, 2024-09-19T08:46:08.512 INFO:teuthology.orchestra.run.smithi098.stdout: "want_state": "up:active", 2024-09-19T08:46:08.512 INFO:teuthology.orchestra.run.smithi098.stdout: "state": "up:active", 2024-09-19T08:46:08.512 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "cephfs", 2024-09-19T08:46:08.512 INFO:teuthology.orchestra.run.smithi098.stdout: "rank_uptime": 4.6830092040000002, 2024-09-19T08:46:08.512 INFO:teuthology.orchestra.run.smithi098.stdout: "mdsmap_epoch": 82, 2024-09-19T08:46:08.512 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch": 146, 2024-09-19T08:46:08.512 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch_barrier": 146, 2024-09-19T08:46:08.512 INFO:teuthology.orchestra.run.smithi098.stdout: "uptime": 9.6844165629999992 2024-09-19T08:46:08.512 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:46:08.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.507+0000 7f40e8f35640 1 -- 172.21.15.98:0/1898701005 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f40c4067ef0 msgr2=0x7f40c406a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:08.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.507+0000 7f40e8f35640 1 --2- 172.21.15.98:0/1898701005 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f40c4067ef0 0x7f40c406a3b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f40d4004d10 tx=0x7f40d4006040 comp rx=0 tx=0).stop 2024-09-19T08:46:08.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.507+0000 7f40e8f35640 1 -- 172.21.15.98:0/1898701005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40e4073d90 msgr2=0x7f40e40c52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:08.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.507+0000 7f40e8f35640 1 --2- 172.21.15.98:0/1898701005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40e4073d90 0x7f40e40c52f0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f40e40d12d0 tx=0x7f40dc00d890 comp rx=0 tx=0).stop 2024-09-19T08:46:08.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.507+0000 7f40e8f35640 1 -- 172.21.15.98:0/1898701005 shutdown_connections 2024-09-19T08:46:08.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.507+0000 7f40e8f35640 1 --2- 172.21.15.98:0/1898701005 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f40c4067ef0 0x7f40c406a3b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.507+0000 7f40e8f35640 1 --2- 172.21.15.98:0/1898701005 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f40e4073d90 0x7f40e40c52f0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.507+0000 7f40e8f35640 1 --2- 172.21.15.98:0/1898701005 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f40e4148960 0x7f40e4073850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.507+0000 7f40e8f35640 1 --2- 172.21.15.98:0/1898701005 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f40cc003f70 0x7f40e415d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.507+0000 7f40e8f35640 1 -- 172.21.15.98:0/1898701005 >> 172.21.15.98:0/1898701005 conn(0x7f40e40b2010 msgr2=0x7f40e40b05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:08.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.507+0000 7f40e8f35640 1 -- 172.21.15.98:0/1898701005 shutdown_connections 2024-09-19T08:46:08.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.507+0000 7f40e8f35640 1 -- 172.21.15.98:0/1898701005 wait complete. 2024-09-19T08:46:08.521 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T08:46:08.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.627+0000 7f1c52134640 1 Processor -- start 2024-09-19T08:46:08.624 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.627+0000 7f1c52134640 1 -- start start 2024-09-19T08:46:08.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.627+0000 7f1c52134640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1c4c06efa0 0x7f1c4c0ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:08.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.627+0000 7f1c52134640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1c4c06f380 0x7f1c4c0cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:08.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.627+0000 7f1c52134640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c4c0af4a0 con 0x7f1c4c0ae8c0 2024-09-19T08:46:08.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.627+0000 7f1c52134640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f1c4c15ed00 con 0x7f1c4c06efa0 2024-09-19T08:46:08.625 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.627+0000 7f1c52134640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1c4c0af780 con 0x7f1c4c06f380 2024-09-19T08:46:08.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.627+0000 7f1c4b7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1c4c06efa0 0x7f1c4c0ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:08.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.627+0000 7f1c4affd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1c4c06f380 0x7f1c4c0cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:08.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.627+0000 7f1c4b7fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1c4c06efa0 0x7f1c4c0ce550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:58310/0 (socket says 172.21.15.98:58310) 2024-09-19T08:46:08.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.627+0000 7f1c4bfff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f1c4c0ae8c0 0x7f1c4c15b5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:40146/0 (socket says 172.21.15.98:40146) 2024-09-19T08:46:08.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.627+0000 7f1c4b7fe640 1 -- 172.21.15.98:0/925659043 learned_addr learned my addr 172.21.15.98:0/925659043 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:08.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.627+0000 7f1c4affd640 1 -- 172.21.15.98:0/925659043 >> v2:172.21.15.139:3300/0 conn(0x7f1c4c06efa0 msgr2=0x7f1c4c0ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:08.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.627+0000 7f1c4affd640 1 --2- 172.21.15.98:0/925659043 >> v2:172.21.15.139:3300/0 conn(0x7f1c4c06efa0 0x7f1c4c0ce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.627+0000 7f1c4affd640 1 -- 172.21.15.98:0/925659043 >> v1:172.21.15.139:6789/0 conn(0x7f1c4c0ae8c0 legacy=0x7f1c4c15b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:08.627 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.627+0000 7f1c4affd640 1 -- 172.21.15.98:0/925659043 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c4c0afa60 con 0x7f1c4c06f380 2024-09-19T08:46:08.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.631+0000 7f1c4affd640 1 --2- 172.21.15.98:0/925659043 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1c4c06f380 0x7f1c4c0cea90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1c400097c0 tx=0x7f1c4001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f8bb9f4be8e1c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:08.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.631+0000 7f1c4a7fc640 1 -- 172.21.15.98:0/925659043 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c400b0070 con 0x7f1c4c06f380 2024-09-19T08:46:08.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.631+0000 7f1c4a7fc640 1 -- 172.21.15.98:0/925659043 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c400ab600 con 0x7f1c4c06f380 2024-09-19T08:46:08.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.631+0000 7f1c4a7fc640 1 -- 172.21.15.98:0/925659043 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c400ab9d0 con 0x7f1c4c06f380 2024-09-19T08:46:08.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.631+0000 7f1c52134640 1 -- 172.21.15.98:0/925659043 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1c4c06f380 msgr2=0x7f1c4c0cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:08.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.631+0000 7f1c52134640 1 --2- 172.21.15.98:0/925659043 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1c4c06f380 0x7f1c4c0cea90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1c400097c0 tx=0x7f1c4001c410 comp rx=0 tx=0).stop 2024-09-19T08:46:08.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.631+0000 7f1c52134640 1 -- 172.21.15.98:0/925659043 shutdown_connections 2024-09-19T08:46:08.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.631+0000 7f1c52134640 1 --2- 172.21.15.98:0/925659043 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1c4c06f380 0x7f1c4c0cea90 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.631+0000 7f1c52134640 1 --2- 172.21.15.98:0/925659043 >> v2:172.21.15.139:3300/0 conn(0x7f1c4c06efa0 0x7f1c4c0ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.631+0000 7f1c52134640 1 -- 172.21.15.98:0/925659043 >> 172.21.15.98:0/925659043 conn(0x7f1c4c0d0980 msgr2=0x7f1c4c0d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:08.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.631+0000 7f1c52134640 1 -- 172.21.15.98:0/925659043 shutdown_connections 2024-09-19T08:46:08.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.631+0000 7f1c52134640 1 -- 172.21.15.98:0/925659043 wait complete. 2024-09-19T08:46:08.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.631+0000 7f1c52134640 1 Processor -- start 2024-09-19T08:46:08.631 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.631+0000 7f1c52134640 1 -- start start 2024-09-19T08:46:08.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.631+0000 7f1c52134640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c4c06efa0 0x7f1c4c159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:08.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.631+0000 7f1c52134640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c4c0ae8c0 0x7f1c4c156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:08.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.631+0000 7f1c52134640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1c4c14dde0 0x7f1c4c0727d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:08.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.631+0000 7f1c52134640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c06d980 con 0x7f1c4c06efa0 2024-09-19T08:46:08.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.631+0000 7f1c52134640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c06d800 con 0x7f1c4c0ae8c0 2024-09-19T08:46:08.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.631+0000 7f1c52134640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1c4c06db00 con 0x7f1c4c14dde0 2024-09-19T08:46:08.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.631+0000 7f1c4b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c4c06efa0 0x7f1c4c159ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:08.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.631+0000 7f1c4b7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c4c06efa0 0x7f1c4c159ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46500/0 (socket says 172.21.15.98:46500) 2024-09-19T08:46:08.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.631+0000 7f1c4b7fe640 1 -- 172.21.15.98:0/3973460232 learned_addr learned my addr 172.21.15.98:0/3973460232 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:08.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.635+0000 7f1c4bfff640 1 --2- 172.21.15.98:0/3973460232 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1c4c14dde0 0x7f1c4c0727d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:08.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.635+0000 7f1c4affd640 1 --2- 172.21.15.98:0/3973460232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c4c0ae8c0 0x7f1c4c156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:08.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.635+0000 7f1c4b7fe640 1 -- 172.21.15.98:0/3973460232 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1c4c14dde0 msgr2=0x7f1c4c0727d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:08.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.635+0000 7f1c4b7fe640 1 --2- 172.21.15.98:0/3973460232 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1c4c14dde0 0x7f1c4c0727d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.635+0000 7f1c4b7fe640 1 -- 172.21.15.98:0/3973460232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c4c0ae8c0 msgr2=0x7f1c4c156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:08.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.635+0000 7f1c4b7fe640 1 --2- 172.21.15.98:0/3973460232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c4c0ae8c0 0x7f1c4c156680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.635+0000 7f1c4b7fe640 1 -- 172.21.15.98:0/3973460232 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c4c158d40 con 0x7f1c4c06efa0 2024-09-19T08:46:08.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.635+0000 7f1c4b7fe640 1 --2- 172.21.15.98:0/3973460232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c4c06efa0 0x7f1c4c159ae0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c00e930 tx=0x7f1c3c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:08.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.635+0000 7f1c33fff640 1 -- 172.21.15.98:0/3973460232 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c017070 con 0x7f1c4c06efa0 2024-09-19T08:46:08.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.635+0000 7f1c33fff640 1 -- 172.21.15.98:0/3973460232 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c3c00ca00 con 0x7f1c4c06efa0 2024-09-19T08:46:08.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.635+0000 7f1c52134640 1 -- 172.21.15.98:0/3973460232 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c4c159000 con 0x7f1c4c06efa0 2024-09-19T08:46:08.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.635+0000 7f1c33fff640 1 -- 172.21.15.98:0/3973460232 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c00cdf0 con 0x7f1c4c06efa0 2024-09-19T08:46:08.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.635+0000 7f1c52134640 1 -- 172.21.15.98:0/3973460232 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c4c1595c0 con 0x7f1c4c06efa0 2024-09-19T08:46:08.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.635+0000 7f1c52134640 1 -- 172.21.15.98:0/3973460232 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c4c06fee0 con 0x7f1c4c06efa0 2024-09-19T08:46:08.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.635+0000 7f1c33fff640 1 -- 172.21.15.98:0/3973460232 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f1c3c01a070 con 0x7f1c4c06efa0 2024-09-19T08:46:08.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.635+0000 7f1c33fff640 1 --2- 172.21.15.98:0/3973460232 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1c38067ef0 0x7f1c3806a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:08.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.635+0000 7f1c33fff640 1 -- 172.21.15.98:0/3973460232 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5737+0+0 (secure 0 0 0) 0x7f1c3c086810 con 0x7f1c4c06efa0 2024-09-19T08:46:08.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.635+0000 7f1c4affd640 1 --2- 172.21.15.98:0/3973460232 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1c38067ef0 0x7f1c3806a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:08.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.635+0000 7f1c4affd640 1 --2- 172.21.15.98:0/3973460232 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1c38067ef0 0x7f1c3806a3b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1c400ad000 tx=0x7f1c40008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:08.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.639+0000 7f1c33fff640 1 -- 172.21.15.98:0/3973460232 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f1c3c056a80 con 0x7f1c4c06efa0 2024-09-19T08:46:08.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.827+0000 7f1c52134640 1 -- 172.21.15.98:0/3973460232 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1c4c1545f0 con 0x7f1c4c06efa0 2024-09-19T08:46:08.826 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.827+0000 7f1c33fff640 1 -- 172.21.15.98:0/3973460232 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v82) ==== 53+0+73 (secure 0 0 0) 0x7f1c3c056380 con 0x7f1c4c06efa0 2024-09-19T08:46:08.827 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T08:46:08.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.831+0000 7f1c52134640 1 -- 172.21.15.98:0/3973460232 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1c38067ef0 msgr2=0x7f1c3806a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:08.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.831+0000 7f1c52134640 1 --2- 172.21.15.98:0/3973460232 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1c38067ef0 0x7f1c3806a3b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1c400ad000 tx=0x7f1c40008040 comp rx=0 tx=0).stop 2024-09-19T08:46:08.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.831+0000 7f1c52134640 1 -- 172.21.15.98:0/3973460232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c4c06efa0 msgr2=0x7f1c4c159ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:08.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.831+0000 7f1c52134640 1 --2- 172.21.15.98:0/3973460232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c4c06efa0 0x7f1c4c159ae0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c00e930 tx=0x7f1c3c00edf0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.831+0000 7f1c52134640 1 -- 172.21.15.98:0/3973460232 shutdown_connections 2024-09-19T08:46:08.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.831+0000 7f1c52134640 1 --2- 172.21.15.98:0/3973460232 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1c38067ef0 0x7f1c3806a3b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.831+0000 7f1c52134640 1 --2- 172.21.15.98:0/3973460232 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1c4c14dde0 0x7f1c4c0727d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.831+0000 7f1c52134640 1 --2- 172.21.15.98:0/3973460232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1c4c0ae8c0 0x7f1c4c156680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.831+0000 7f1c52134640 1 --2- 172.21.15.98:0/3973460232 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1c4c06efa0 0x7f1c4c159ae0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:08.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.831+0000 7f1c52134640 1 -- 172.21.15.98:0/3973460232 >> 172.21.15.98:0/3973460232 conn(0x7f1c4c0d0980 msgr2=0x7f1c4c158020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:08.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.831+0000 7f1c52134640 1 -- 172.21.15.98:0/3973460232 shutdown_connections 2024-09-19T08:46:08.829 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:08.831+0000 7f1c52134640 1 -- 172.21.15.98:0/3973460232 wait complete. 2024-09-19T08:46:08.843 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T08:46:08.844 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T08:46:08.844 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi098.front.sepia.ceph.com 2024-09-19T08:46:08.844 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T08:46:08.844 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T08:46:08.844 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T08:46:08.844 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T08:46:08.844 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T08:46:08.844 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:46:08.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:08.845 DEBUG:teuthology.orchestra.run.smithi098:> ip addr 2024-09-19T08:46:08.851 INFO:teuthology.orchestra.run.smithi098.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T08:46:08.851 INFO:teuthology.orchestra.run.smithi098.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T08:46:08.851 INFO:teuthology.orchestra.run.smithi098.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T08:46:08.851 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:46:08.851 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 ::1/128 scope host 2024-09-19T08:46:08.851 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:46:08.851 INFO:teuthology.orchestra.run.smithi098.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:46:08.851 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b8 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:46:08.851 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f0 2024-09-19T08:46:08.851 INFO:teuthology.orchestra.run.smithi098.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:46:08.852 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b9 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:46:08.852 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f1 2024-09-19T08:46:08.852 INFO:teuthology.orchestra.run.smithi098.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:46:08.852 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6a brd ff:ff:ff:ff:ff:ff 2024-09-19T08:46:08.852 INFO:teuthology.orchestra.run.smithi098.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T08:46:08.852 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6b brd ff:ff:ff:ff:ff:ff 2024-09-19T08:46:08.852 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.21.15.98/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T08:46:08.852 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft 42039sec preferred_lft 42039sec 2024-09-19T08:46:08.852 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::ec4:7aff:fe8f:d26b/64 scope link 2024-09-19T08:46:08.852 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:46:08.852 INFO:teuthology.orchestra.run.smithi098.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T08:46:08.852 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 02:42:98:dd:32:2d brd ff:ff:ff:ff:ff:ff 2024-09-19T08:46:08.852 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T08:46:08.853 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:46:08.853 INFO:teuthology.orchestra.run.smithi098.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T08:46:08.853 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-19T08:46:08.853 INFO:teuthology.orchestra.run.smithi098.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T08:46:08.853 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:46:08.853 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-19T08:46:08.853 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:46:08.853 INFO:teuthology.orchestra.run.smithi098.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:46:08.853 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:46:08.853 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-19T08:46:08.853 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:46:08.853 INFO:teuthology.orchestra.run.smithi098.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:46:08.853 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:46:08.854 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-19T08:46:08.854 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:46:08.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:08.854 DEBUG:teuthology.orchestra.run.smithi098:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T08:46:08.902 INFO:teuthology.orchestra.run.smithi098.stdout:1 2024-09-19T08:46:08.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:08.904 DEBUG:teuthology.orchestra.run.smithi098:> route 2024-09-19T08:46:08.956 INFO:teuthology.orchestra.run.smithi098.stdout:Kernel IP routing table 2024-09-19T08:46:08.957 INFO:teuthology.orchestra.run.smithi098.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T08:46:08.957 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T08:46:08.957 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T08:46:08.957 INFO:teuthology.orchestra.run.smithi098.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T08:46:08.957 INFO:teuthology.orchestra.run.smithi098.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T08:46:08.957 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T08:46:08.957 INFO:teuthology.orchestra.run.smithi098.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:46:08.957 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:46:08.957 INFO:teuthology.orchestra.run.smithi098.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:46:08.957 INFO:teuthology.orchestra.run.smithi098.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T08:46:08.958 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:46:08.958 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-09-19T08:46:08.958 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T08:46:08.958 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T08:46:08.958 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T08:46:08.958 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-09-19T08:46:09.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:46:09.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:46:09.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:09.075 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2024-09-19T08:46:09.120 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-19T08:46:09.120 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T08:46:09.121 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:46:09.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:09.121 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set brx.0 up 2024-09-19T08:46:09.176 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-19T08:46:09.176 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:46:09.176 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T08:46:09.224 INFO:teuthology.orchestra.run.smithi098.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:46:09.224 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:46:09.225 DEBUG:teuthology.orchestra.run.smithi098:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T08:46:09.272 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:46:09.330 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:46:09.377 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:46:09.377 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:46:09.378 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:46:09.378 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:46:09.378 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:46:09.378 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:46:09.378 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:46:09.378 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:46:09.378 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:46:09.378 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:46:09.378 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:46:09.378 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:46:09.378 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:46:09.378 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:46:09.378 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:46:09.379 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:46:09.379 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:46:09.379 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:46:09.379 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:46:09.379 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:46:09.379 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:46:09.379 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:46:09.379 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:46:09.379 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:46:09.379 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:46:09.379 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:46:09.379 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:46:09.379 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:46:09.379 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:46:09.380 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:46:09.380 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:46:09.380 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:46:09.380 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:46:09.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:09.381 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:46:09.429 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T08:46:09.429 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-19T08:46:09.473 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:46:09.488 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:46:09.536 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:46:09.537 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:46:09.537 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:46:09.537 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:46:09.537 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:46:09.537 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:46:09.537 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:46:09.537 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:46:09.537 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:46:09.537 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:46:09.537 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:46:09.537 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:46:09.537 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:46:09.537 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:46:09.537 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:46:09.538 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:46:09.538 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:46:09.538 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:46:09.538 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:46:09.538 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:46:09.538 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:46:09.538 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:46:09.538 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:46:09.538 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:46:09.538 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:46:09.538 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:46:09.538 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:46:09.538 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:46:09.538 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:46:09.539 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:46:09.539 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:46:09.539 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:46:09.539 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:46:09.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:09.540 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:46:09.556 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:2024-09-19T08:46:09.559+0000 7efdb2eaa580 -1 init, newargv = 0x559ab63d6120 newargc=15 2024-09-19T08:46:09.557 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:ceph-fuse[51532]: starting ceph client 2024-09-19T08:46:09.562 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:ceph-fuse[51532]: starting fuse 2024-09-19T08:46:09.769 INFO:tasks.ceph.mds.a.smithi098.stderr:2024-09-19T08:46:09.771+0000 7f622a094640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T08:46:10.662 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:46:10.677 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:46:10.727 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:46:10.727 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:46:10.727 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:46:10.728 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:46:10.728 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:46:10.728 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:46:10.728 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:46:10.728 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:46:10.728 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:46:10.728 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:46:10.728 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:46:10.728 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:46:10.728 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:46:10.728 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:46:10.728 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:46:10.728 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:46:10.729 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:46:10.729 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:46:10.729 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:46:10.729 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:46:10.729 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:46:10.729 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:46:10.729 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:46:10.729 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:46:10.729 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:46:10.729 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:46:10.729 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:46:10.729 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:46:10.729 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:46:10.729 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:46:10.730 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:46:10.730 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:46:10.730 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:46:10.730 INFO:teuthology.orchestra.run.smithi098.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-19T08:46:10.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:10.731 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:46:10.777 INFO:teuthology.orchestra.run.smithi098.stdout:45 2024-09-19T08:46:10.777 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-19T08:46:10.777 DEBUG:teuthology.orchestra.run.smithi098:> sudo stdin-killer -- python3 -c ' 2024-09-19T08:46:10.777 DEBUG:teuthology.orchestra.run.smithi098:> import glob 2024-09-19T08:46:10.777 DEBUG:teuthology.orchestra.run.smithi098:> import re 2024-09-19T08:46:10.777 DEBUG:teuthology.orchestra.run.smithi098:> import os 2024-09-19T08:46:10.777 DEBUG:teuthology.orchestra.run.smithi098:> import subprocess 2024-09-19T08:46:10.777 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:46:10.778 DEBUG:teuthology.orchestra.run.smithi098:> def _find_admin_socket(client_name): 2024-09-19T08:46:10.778 DEBUG:teuthology.orchestra.run.smithi098:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T08:46:10.778 DEBUG:teuthology.orchestra.run.smithi098:> files = glob.glob(asok_path) 2024-09-19T08:46:10.778 DEBUG:teuthology.orchestra.run.smithi098:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T08:46:10.778 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:46:10.778 DEBUG:teuthology.orchestra.run.smithi098:> # Given a non-glob path, it better be there 2024-09-19T08:46:10.778 DEBUG:teuthology.orchestra.run.smithi098:> if "*" not in asok_path: 2024-09-19T08:46:10.778 DEBUG:teuthology.orchestra.run.smithi098:> assert(len(files) == 1) 2024-09-19T08:46:10.778 DEBUG:teuthology.orchestra.run.smithi098:> return files[0] 2024-09-19T08:46:10.778 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:46:10.778 DEBUG:teuthology.orchestra.run.smithi098:> for f in files: 2024-09-19T08:46:10.778 DEBUG:teuthology.orchestra.run.smithi098:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T08:46:10.778 DEBUG:teuthology.orchestra.run.smithi098:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T08:46:10.778 DEBUG:teuthology.orchestra.run.smithi098:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T08:46:10.779 DEBUG:teuthology.orchestra.run.smithi098:> contents = proc_f.read() 2024-09-19T08:46:10.779 DEBUG:teuthology.orchestra.run.smithi098:> if mountpoint in contents: 2024-09-19T08:46:10.779 DEBUG:teuthology.orchestra.run.smithi098:> return f 2024-09-19T08:46:10.779 DEBUG:teuthology.orchestra.run.smithi098:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T08:46:10.779 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:46:10.779 DEBUG:teuthology.orchestra.run.smithi098:> print(_find_admin_socket("client.0")) 2024-09-19T08:46:10.779 DEBUG:teuthology.orchestra.run.smithi098:> ' 2024-09-19T08:46:10.901 INFO:teuthology.orchestra.run.smithi098.stdout:/var/run/ceph/ceph-client.0.51532.asok 2024-09-19T08:46:10.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:46:10.912 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.51532.asok 2024-09-19T08:46:10.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:10.912 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.51532.asok status 2024-09-19T08:46:11.064 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:46:11.064 INFO:teuthology.orchestra.run.smithi098.stdout: "metadata": { 2024-09-19T08:46:11.064 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T08:46:11.064 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T08:46:11.064 INFO:teuthology.orchestra.run.smithi098.stdout: "entity_id": "0", 2024-09-19T08:46:11.064 INFO:teuthology.orchestra.run.smithi098.stdout: "hostname": "smithi098", 2024-09-19T08:46:11.064 INFO:teuthology.orchestra.run.smithi098.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T08:46:11.064 INFO:teuthology.orchestra.run.smithi098.stdout: "pid": "51532", 2024-09-19T08:46:11.065 INFO:teuthology.orchestra.run.smithi098.stdout: "root": "/" 2024-09-19T08:46:11.065 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:11.065 INFO:teuthology.orchestra.run.smithi098.stdout: "dentry_count": 0, 2024-09-19T08:46:11.065 INFO:teuthology.orchestra.run.smithi098.stdout: "dentry_pinned_count": 0, 2024-09-19T08:46:11.065 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 8526, 2024-09-19T08:46:11.065 INFO:teuthology.orchestra.run.smithi098.stdout: "inst": { 2024-09-19T08:46:11.065 INFO:teuthology.orchestra.run.smithi098.stdout: "name": { 2024-09-19T08:46:11.065 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "client", 2024-09-19T08:46:11.065 INFO:teuthology.orchestra.run.smithi098.stdout: "num": 8526 2024-09-19T08:46:11.065 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:11.065 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": { 2024-09-19T08:46:11.065 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v1", 2024-09-19T08:46:11.065 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:46:11.066 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 311162546 2024-09-19T08:46:11.066 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:11.066 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:11.066 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": { 2024-09-19T08:46:11.066 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v1", 2024-09-19T08:46:11.066 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:46:11.066 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 311162546 2024-09-19T08:46:11.066 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:11.066 INFO:teuthology.orchestra.run.smithi098.stdout: "inst_str": "client.8526 192.168.0.1:0/311162546", 2024-09-19T08:46:11.066 INFO:teuthology.orchestra.run.smithi098.stdout: "addr_str": "192.168.0.1:0/311162546", 2024-09-19T08:46:11.066 INFO:teuthology.orchestra.run.smithi098.stdout: "inode_count": 1, 2024-09-19T08:46:11.066 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_epoch": 82, 2024-09-19T08:46:11.066 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_epoch": 146, 2024-09-19T08:46:11.066 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_epoch_barrier": 0, 2024-09-19T08:46:11.066 INFO:teuthology.orchestra.run.smithi098.stdout: "blocklisted": false, 2024-09-19T08:46:11.067 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "cephfs" 2024-09-19T08:46:11.067 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:46:11.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:11.078 DEBUG:teuthology.orchestra.run.smithi098:> stat --file-system '--printf=%T 2024-09-19T08:46:11.078 DEBUG:teuthology.orchestra.run.smithi098:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T08:46:11.084 INFO:teuthology.orchestra.run.smithi098.stdout:fuseblk 2024-09-19T08:46:11.084 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T08:46:11.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:11.085 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T08:46:11.141 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T08:46:11.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.307+0000 7fa2b75ca640 1 Processor -- start 2024-09-19T08:46:11.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.307+0000 7fa2b75ca640 1 -- start start 2024-09-19T08:46:11.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.307+0000 7fa2b75ca640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa2b0158fd0 0x7fa2b00b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:11.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.307+0000 7fa2b75ca640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2b00b4360 0x7fa2b006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:11.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.307+0000 7fa2b75ca640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa2b006c1d0 0x7fa2b015efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:11.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.307+0000 7fa2b75ca640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fa2b01593b0 con 0x7fa2b006c1d0 2024-09-19T08:46:11.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.307+0000 7fa2b75ca640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fa2b006c600 con 0x7fa2b0158fd0 2024-09-19T08:46:11.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.307+0000 7fa2b75ca640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa2b00ccc60 con 0x7fa2b00b4360 2024-09-19T08:46:11.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.307+0000 7fa2b5b67640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa2b006c1d0 0x7fa2b015efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:11.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.307+0000 7fa2b5b67640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa2b006c1d0 0x7fa2b015efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46508/0 (socket says 172.21.15.98:46508) 2024-09-19T08:46:11.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.307+0000 7fa2b5b67640 1 -- 172.21.15.98:0/2679586035 learned_addr learned my addr 172.21.15.98:0/2679586035 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:11.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.307+0000 7fa2b5366640 1 --2- 172.21.15.98:0/2679586035 >> v2:172.21.15.139:3300/0 conn(0x7fa2b0158fd0 0x7fa2b00b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:11.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.307+0000 7fa2b5b67640 1 -- 172.21.15.98:0/2679586035 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2b00b4360 msgr2=0x7fa2b006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:11.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.307+0000 7fa2b5b67640 1 --2- 172.21.15.98:0/2679586035 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2b00b4360 0x7fa2b006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.307+0000 7fa2b5b67640 1 -- 172.21.15.98:0/2679586035 >> v2:172.21.15.139:3300/0 conn(0x7fa2b0158fd0 msgr2=0x7fa2b00b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:11.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.307+0000 7fa2b5b67640 1 --2- 172.21.15.98:0/2679586035 >> v2:172.21.15.139:3300/0 conn(0x7fa2b0158fd0 0x7fa2b00b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.307+0000 7fa2b5b67640 1 -- 172.21.15.98:0/2679586035 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b00ccf40 con 0x7fa2b006c1d0 2024-09-19T08:46:11.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.307+0000 7fa2b5366640 1 --2- 172.21.15.98:0/2679586035 >> v2:172.21.15.139:3300/0 conn(0x7fa2b0158fd0 0x7fa2b00b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:11.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2b5b67640 1 --2- 172.21.15.98:0/2679586035 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2b006c1d0 0x7fa2b015efc0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fa2b006e310 tx=0x7fa2a801c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=871906b3dbac2081 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:11.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2a7fff640 1 -- 172.21.15.98:0/2679586035 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a80a6500 con 0x7fa2b006c1d0 2024-09-19T08:46:11.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2a7fff640 1 -- 172.21.15.98:0/2679586035 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2a80a66a0 con 0x7fa2b006c1d0 2024-09-19T08:46:11.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2a7fff640 1 -- 172.21.15.98:0/2679586035 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a80a6a70 con 0x7fa2b006c1d0 2024-09-19T08:46:11.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2b75ca640 1 -- 172.21.15.98:0/2679586035 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2b006c1d0 msgr2=0x7fa2b015efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:11.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2b75ca640 1 --2- 172.21.15.98:0/2679586035 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2b006c1d0 0x7fa2b015efc0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fa2b006e310 tx=0x7fa2a801c670 comp rx=0 tx=0).stop 2024-09-19T08:46:11.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2b75ca640 1 -- 172.21.15.98:0/2679586035 shutdown_connections 2024-09-19T08:46:11.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2b75ca640 1 --2- 172.21.15.98:0/2679586035 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2b006c1d0 0x7fa2b015efc0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2b75ca640 1 --2- 172.21.15.98:0/2679586035 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2b00b4360 0x7fa2b006bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2b75ca640 1 --2- 172.21.15.98:0/2679586035 >> v2:172.21.15.139:3300/0 conn(0x7fa2b0158fd0 0x7fa2b00b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2b75ca640 1 -- 172.21.15.98:0/2679586035 >> 172.21.15.98:0/2679586035 conn(0x7fa2b00b2010 msgr2=0x7fa2b00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:11.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2b75ca640 1 -- 172.21.15.98:0/2679586035 shutdown_connections 2024-09-19T08:46:11.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2b75ca640 1 -- 172.21.15.98:0/2679586035 wait complete. 2024-09-19T08:46:11.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2b75ca640 1 Processor -- start 2024-09-19T08:46:11.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2b75ca640 1 -- start start 2024-09-19T08:46:11.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2b75ca640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2b006c1d0 0x7fa2b0073d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:11.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2b75ca640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2b00b4360 0x7fa2b0149470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:11.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2b75ca640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2b0158fd0 0x7fa2b01499b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:11.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2b75ca640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b015c920 con 0x7fa2b006c1d0 2024-09-19T08:46:11.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2b75ca640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b015c7a0 con 0x7fa2b0158fd0 2024-09-19T08:46:11.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2b75ca640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa2b00ce840 con 0x7fa2b00b4360 2024-09-19T08:46:11.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2b5366640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2b006c1d0 0x7fa2b0073d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:11.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2b5366640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2b006c1d0 0x7fa2b0073d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46524/0 (socket says 172.21.15.98:46524) 2024-09-19T08:46:11.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2b5366640 1 -- 172.21.15.98:0/125994197 learned_addr learned my addr 172.21.15.98:0/125994197 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:11.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2b5366640 1 -- 172.21.15.98:0/125994197 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2b00b4360 msgr2=0x7fa2b0149470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:11.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2b4b65640 1 --2- 172.21.15.98:0/125994197 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2b00b4360 0x7fa2b0149470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:11.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2b5b67640 1 --2- 172.21.15.98:0/125994197 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2b0158fd0 0x7fa2b01499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:11.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2b5366640 1 --2- 172.21.15.98:0/125994197 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2b00b4360 0x7fa2b0149470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2b5366640 1 -- 172.21.15.98:0/125994197 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2b0158fd0 msgr2=0x7fa2b01499b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:11.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2b5366640 1 --2- 172.21.15.98:0/125994197 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2b0158fd0 0x7fa2b01499b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2b5366640 1 -- 172.21.15.98:0/125994197 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b0160200 con 0x7fa2b006c1d0 2024-09-19T08:46:11.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2b5366640 1 --2- 172.21.15.98:0/125994197 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2b006c1d0 0x7fa2b0073d70 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac00c820 tx=0x7fa2ac00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:11.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2a5ffb640 1 -- 172.21.15.98:0/125994197 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac015070 con 0x7fa2b006c1d0 2024-09-19T08:46:11.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2a5ffb640 1 -- 172.21.15.98:0/125994197 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2ac007a30 con 0x7fa2b006c1d0 2024-09-19T08:46:11.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2b75ca640 1 -- 172.21.15.98:0/125994197 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2b01604e0 con 0x7fa2b006c1d0 2024-09-19T08:46:11.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2a5ffb640 1 -- 172.21.15.98:0/125994197 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac007e40 con 0x7fa2b006c1d0 2024-09-19T08:46:11.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.311+0000 7fa2b75ca640 1 -- 172.21.15.98:0/125994197 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2b0144490 con 0x7fa2b006c1d0 2024-09-19T08:46:11.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.315+0000 7fa2b75ca640 1 -- 172.21.15.98:0/125994197 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2b006bc90 con 0x7fa2b006c1d0 2024-09-19T08:46:11.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.315+0000 7fa2a5ffb640 1 -- 172.21.15.98:0/125994197 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fa2ac018070 con 0x7fa2b006c1d0 2024-09-19T08:46:11.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.315+0000 7fa2a5ffb640 1 --2- 172.21.15.98:0/125994197 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa294067ef0 0x7fa29406a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:11.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.315+0000 7fa2a5ffb640 1 -- 172.21.15.98:0/125994197 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5737+0+0 (secure 0 0 0) 0x7fa2ac086bd0 con 0x7fa2b006c1d0 2024-09-19T08:46:11.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.315+0000 7fa2b4b65640 1 --2- 172.21.15.98:0/125994197 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa294067ef0 0x7fa29406a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:11.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.315+0000 7fa2b4b65640 1 --2- 172.21.15.98:0/125994197 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa294067ef0 0x7fa29406a3b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa2b00b0de0 tx=0x7fa2a0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:11.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.319+0000 7fa2a5ffb640 1 -- 172.21.15.98:0/125994197 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa2ac056db0 con 0x7fa2b006c1d0 2024-09-19T08:46:11.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.511+0000 7fa2b75ca640 1 -- 172.21.15.98:0/125994197 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa2b01658f0 con 0x7fa2b006c1d0 2024-09-19T08:46:11.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.511+0000 7fa2a5ffb640 1 -- 172.21.15.98:0/125994197 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v82) ==== 53+0+73 (secure 0 0 0) 0x7fa2ac0566b0 con 0x7fa2b006c1d0 2024-09-19T08:46:11.510 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T08:46:11.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.511+0000 7fa2b75ca640 1 -- 172.21.15.98:0/125994197 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa294067ef0 msgr2=0x7fa29406a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:11.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.511+0000 7fa2b75ca640 1 --2- 172.21.15.98:0/125994197 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa294067ef0 0x7fa29406a3b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa2b00b0de0 tx=0x7fa2a0008040 comp rx=0 tx=0).stop 2024-09-19T08:46:11.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.511+0000 7fa2b75ca640 1 -- 172.21.15.98:0/125994197 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2b006c1d0 msgr2=0x7fa2b0073d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:11.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.511+0000 7fa2b75ca640 1 --2- 172.21.15.98:0/125994197 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2b006c1d0 0x7fa2b0073d70 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac00c820 tx=0x7fa2ac00cce0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.511+0000 7fa2b75ca640 1 -- 172.21.15.98:0/125994197 shutdown_connections 2024-09-19T08:46:11.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.511+0000 7fa2b75ca640 1 --2- 172.21.15.98:0/125994197 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa294067ef0 0x7fa29406a3b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.511+0000 7fa2b75ca640 1 --2- 172.21.15.98:0/125994197 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa2b0158fd0 0x7fa2b01499b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.511+0000 7fa2b75ca640 1 --2- 172.21.15.98:0/125994197 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa2b00b4360 0x7fa2b0149470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.511+0000 7fa2b75ca640 1 --2- 172.21.15.98:0/125994197 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa2b006c1d0 0x7fa2b0073d70 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:11.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.511+0000 7fa2b75ca640 1 -- 172.21.15.98:0/125994197 >> 172.21.15.98:0/125994197 conn(0x7fa2b00b2010 msgr2=0x7fa2b00cf6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:11.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.511+0000 7fa2b75ca640 1 -- 172.21.15.98:0/125994197 shutdown_connections 2024-09-19T08:46:11.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:11.511+0000 7fa2b75ca640 1 -- 172.21.15.98:0/125994197 wait complete. 2024-09-19T08:46:11.528 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T08:46:11.528 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T08:46:11.528 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2024-09-19T08:46:11.528 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-19T08:46:11.528 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-19T08:46:11.528 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T08:46:11.528 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T08:46:11.528 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T08:46:11.528 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T08:46:11.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:11.529 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2024-09-19T08:46:11.537 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T08:46:11.537 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T08:46:11.537 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T08:46:11.538 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:46:11.538 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-09-19T08:46:11.538 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:46:11.538 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:46:11.538 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:46:11.538 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2024-09-19T08:46:11.538 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:46:11.538 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:46:11.538 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2024-09-19T08:46:11.539 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:46:11.539 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:46:11.539 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T08:46:11.539 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:46:11.539 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T08:46:11.539 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 42048sec preferred_lft 42048sec 2024-09-19T08:46:11.539 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::ec4:7aff:fe88:7f59/64 scope link 2024-09-19T08:46:11.539 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:46:11.539 INFO:teuthology.orchestra.run.smithi139.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T08:46:11.539 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 02:42:74:ed:e4:60 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:46:11.539 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T08:46:11.539 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:46:11.539 INFO:teuthology.orchestra.run.smithi139.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T08:46:11.540 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-19T08:46:11.540 INFO:teuthology.orchestra.run.smithi139.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T08:46:11.540 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:46:11.540 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-19T08:46:11.540 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:46:11.540 INFO:teuthology.orchestra.run.smithi139.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:46:11.540 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:46:11.540 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-19T08:46:11.540 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:46:11.540 INFO:teuthology.orchestra.run.smithi139.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:46:11.540 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:46:11.540 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-19T08:46:11.541 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:46:11.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:11.541 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T08:46:11.590 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-09-19T08:46:11.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:11.591 DEBUG:teuthology.orchestra.run.smithi139:> route 2024-09-19T08:46:11.646 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2024-09-19T08:46:11.647 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T08:46:11.647 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T08:46:11.647 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T08:46:11.647 INFO:teuthology.orchestra.run.smithi139.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T08:46:11.647 INFO:teuthology.orchestra.run.smithi139.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T08:46:11.647 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T08:46:11.647 INFO:teuthology.orchestra.run.smithi139.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:46:11.647 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:46:11.647 INFO:teuthology.orchestra.run.smithi139.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:46:11.647 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T08:46:11.647 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:46:11.648 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-09-19T08:46:11.648 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T08:46:11.648 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T08:46:11.648 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T08:46:11.648 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-09-19T08:46:11.742 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:46:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:46:11.771 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:46:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:46:11.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:11.780 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-09-19T08:46:11.785 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-19T08:46:11.785 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-19T08:46:11.785 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T08:46:11.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:11.785 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.0 up 2024-09-19T08:46:11.838 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-19T08:46:11.839 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:46:11.839 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-19T08:46:11.887 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-19T08:46:11.887 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:46:11.888 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-19T08:46:11.939 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:46:11.995 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:46:12.044 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:46:12.044 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:46:12.044 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:46:12.044 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:46:12.044 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:46:12.044 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:46:12.044 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:46:12.044 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:46:12.044 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:46:12.044 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:46:12.044 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:46:12.045 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:46:12.045 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:46:12.045 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:46:12.045 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:46:12.045 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:46:12.045 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:46:12.045 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:46:12.045 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:46:12.045 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:46:12.045 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:46:12.045 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:46:12.045 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:46:12.045 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:46:12.046 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:46:12.046 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:46:12.046 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:46:12.046 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:46:12.046 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:46:12.046 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:46:12.046 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:46:12.046 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:46:12.046 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:46:12.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:12.047 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:46:12.095 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T08:46:12.095 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-19T08:46:12.140 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:46:12.154 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:46:12.203 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:46:12.204 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:46:12.204 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:46:12.204 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:46:12.204 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:46:12.204 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:46:12.204 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:46:12.204 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:46:12.204 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:46:12.204 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:46:12.204 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:46:12.204 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:46:12.204 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:46:12.204 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:46:12.205 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:46:12.205 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:46:12.205 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:46:12.205 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:46:12.205 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:46:12.205 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:46:12.205 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:46:12.205 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:46:12.205 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:46:12.205 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:46:12.205 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:46:12.206 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:46:12.206 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:46:12.206 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:46:12.206 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:46:12.206 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:46:12.206 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:46:12.206 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:46:12.206 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:46:12.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:12.207 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:46:12.220 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:2024-09-19T08:46:12.222+0000 7fca78204580 -1 init, newargv = 0x559b495e1230 newargc=15 2024-09-19T08:46:12.221 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[36777]: starting ceph client 2024-09-19T08:46:12.227 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[36777]: starting fuse 2024-09-19T08:46:13.326 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:46:13.342 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:46:13.392 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:46:13.392 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:46:13.392 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:46:13.392 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:46:13.392 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:46:13.393 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:46:13.393 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:46:13.393 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:46:13.393 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:46:13.393 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:46:13.393 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:46:13.393 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:46:13.393 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:46:13.393 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:46:13.393 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:46:13.393 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:46:13.393 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:46:13.394 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:46:13.394 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:46:13.394 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:46:13.394 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:46:13.394 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:46:13.394 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:46:13.394 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:46:13.394 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:46:13.394 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:46:13.394 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:46:13.394 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:46:13.394 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:46:13.394 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:46:13.395 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:46:13.395 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:46:13.395 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:46:13.395 INFO:teuthology.orchestra.run.smithi139.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-19T08:46:13.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:13.396 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:46:13.447 INFO:teuthology.orchestra.run.smithi139.stdout:45 2024-09-19T08:46:13.447 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-19T08:46:13.447 DEBUG:teuthology.orchestra.run.smithi139:> sudo stdin-killer -- python3 -c ' 2024-09-19T08:46:13.448 DEBUG:teuthology.orchestra.run.smithi139:> import glob 2024-09-19T08:46:13.448 DEBUG:teuthology.orchestra.run.smithi139:> import re 2024-09-19T08:46:13.448 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-09-19T08:46:13.448 DEBUG:teuthology.orchestra.run.smithi139:> import subprocess 2024-09-19T08:46:13.448 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:46:13.448 DEBUG:teuthology.orchestra.run.smithi139:> def _find_admin_socket(client_name): 2024-09-19T08:46:13.448 DEBUG:teuthology.orchestra.run.smithi139:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-19T08:46:13.448 DEBUG:teuthology.orchestra.run.smithi139:> files = glob.glob(asok_path) 2024-09-19T08:46:13.448 DEBUG:teuthology.orchestra.run.smithi139:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-19T08:46:13.448 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:46:13.448 DEBUG:teuthology.orchestra.run.smithi139:> # Given a non-glob path, it better be there 2024-09-19T08:46:13.448 DEBUG:teuthology.orchestra.run.smithi139:> if "*" not in asok_path: 2024-09-19T08:46:13.448 DEBUG:teuthology.orchestra.run.smithi139:> assert(len(files) == 1) 2024-09-19T08:46:13.448 DEBUG:teuthology.orchestra.run.smithi139:> return files[0] 2024-09-19T08:46:13.449 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:46:13.449 DEBUG:teuthology.orchestra.run.smithi139:> for f in files: 2024-09-19T08:46:13.449 DEBUG:teuthology.orchestra.run.smithi139:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T08:46:13.449 DEBUG:teuthology.orchestra.run.smithi139:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T08:46:13.449 DEBUG:teuthology.orchestra.run.smithi139:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T08:46:13.449 DEBUG:teuthology.orchestra.run.smithi139:> contents = proc_f.read() 2024-09-19T08:46:13.449 DEBUG:teuthology.orchestra.run.smithi139:> if mountpoint in contents: 2024-09-19T08:46:13.449 DEBUG:teuthology.orchestra.run.smithi139:> return f 2024-09-19T08:46:13.449 DEBUG:teuthology.orchestra.run.smithi139:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T08:46:13.449 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:46:13.449 DEBUG:teuthology.orchestra.run.smithi139:> print(_find_admin_socket("client.1")) 2024-09-19T08:46:13.449 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-09-19T08:46:13.570 INFO:teuthology.orchestra.run.smithi139.stdout:/var/run/ceph/ceph-client.1.36777.asok 2024-09-19T08:46:13.573 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:46:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:46:13.581 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.36777.asok 2024-09-19T08:46:13.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:13.582 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.36777.asok status 2024-09-19T08:46:13.736 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-19T08:46:13.736 INFO:teuthology.orchestra.run.smithi139.stdout: "metadata": { 2024-09-19T08:46:13.736 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T08:46:13.736 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T08:46:13.736 INFO:teuthology.orchestra.run.smithi139.stdout: "entity_id": "1", 2024-09-19T08:46:13.736 INFO:teuthology.orchestra.run.smithi139.stdout: "hostname": "smithi139", 2024-09-19T08:46:13.736 INFO:teuthology.orchestra.run.smithi139.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-19T08:46:13.736 INFO:teuthology.orchestra.run.smithi139.stdout: "pid": "36777", 2024-09-19T08:46:13.736 INFO:teuthology.orchestra.run.smithi139.stdout: "root": "/" 2024-09-19T08:46:13.736 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:46:13.737 INFO:teuthology.orchestra.run.smithi139.stdout: "dentry_count": 0, 2024-09-19T08:46:13.737 INFO:teuthology.orchestra.run.smithi139.stdout: "dentry_pinned_count": 0, 2024-09-19T08:46:13.737 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 6967, 2024-09-19T08:46:13.737 INFO:teuthology.orchestra.run.smithi139.stdout: "inst": { 2024-09-19T08:46:13.737 INFO:teuthology.orchestra.run.smithi139.stdout: "name": { 2024-09-19T08:46:13.737 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "client", 2024-09-19T08:46:13.737 INFO:teuthology.orchestra.run.smithi139.stdout: "num": 6967 2024-09-19T08:46:13.737 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:46:13.737 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": { 2024-09-19T08:46:13.737 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-19T08:46:13.737 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:46:13.737 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2690916620 2024-09-19T08:46:13.737 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-19T08:46:13.738 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:46:13.738 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": { 2024-09-19T08:46:13.738 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-19T08:46:13.738 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:46:13.738 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 2690916620 2024-09-19T08:46:13.738 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:46:13.738 INFO:teuthology.orchestra.run.smithi139.stdout: "inst_str": "client.6967 192.168.0.1:0/2690916620", 2024-09-19T08:46:13.738 INFO:teuthology.orchestra.run.smithi139.stdout: "addr_str": "192.168.0.1:0/2690916620", 2024-09-19T08:46:13.738 INFO:teuthology.orchestra.run.smithi139.stdout: "inode_count": 1, 2024-09-19T08:46:13.738 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_epoch": 82, 2024-09-19T08:46:13.738 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_epoch": 146, 2024-09-19T08:46:13.738 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_epoch_barrier": 0, 2024-09-19T08:46:13.738 INFO:teuthology.orchestra.run.smithi139.stdout: "blocklisted": false, 2024-09-19T08:46:13.739 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs" 2024-09-19T08:46:13.739 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-19T08:46:13.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:13.749 DEBUG:teuthology.orchestra.run.smithi139:> stat --file-system '--printf=%T 2024-09-19T08:46:13.750 DEBUG:teuthology.orchestra.run.smithi139:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-19T08:46:13.755 INFO:teuthology.orchestra.run.smithi139.stdout:fuseblk 2024-09-19T08:46:13.756 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-19T08:46:13.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:13.756 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-19T08:46:13.810 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-19T08:46:13.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.947+0000 7fb43338d640 1 Processor -- start 2024-09-19T08:46:13.947 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.947+0000 7fb43338d640 1 -- start start 2024-09-19T08:46:13.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.951+0000 7fb43338d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb42c0cb2c0 0x7fb42c06d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:13.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.951+0000 7fb43338d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb42c075680 0x7fb42c0b6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:13.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.951+0000 7fb43338d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb42c0b69c0 0x7fb42c073290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:13.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.951+0000 7fb43338d640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fb42c0c83e0 con 0x7fb42c075680 2024-09-19T08:46:13.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.951+0000 7fb43338d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fb42c0c86c0 con 0x7fb42c0b69c0 2024-09-19T08:46:13.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.951+0000 7fb43338d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb42c0c89a0 con 0x7fb42c0cb2c0 2024-09-19T08:46:13.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.951+0000 7fb430928640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb42c075680 0x7fb42c0b6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:13.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.951+0000 7fb430928640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb42c075680 0x7fb42c0b6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46538/0 (socket says 172.21.15.98:46538) 2024-09-19T08:46:13.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.951+0000 7fb430928640 1 -- 172.21.15.98:0/3113964432 learned_addr learned my addr 172.21.15.98:0/3113964432 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:13.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.951+0000 7fb430928640 1 -- 172.21.15.98:0/3113964432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb42c0cb2c0 msgr2=0x7fb42c06d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:13.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.951+0000 7fb430928640 1 --2- 172.21.15.98:0/3113964432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb42c0cb2c0 0x7fb42c06d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.951+0000 7fb430928640 1 -- 172.21.15.98:0/3113964432 >> v2:172.21.15.139:3300/0 conn(0x7fb42c0b69c0 msgr2=0x7fb42c073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:13.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.951+0000 7fb430928640 1 --2- 172.21.15.98:0/3113964432 >> v2:172.21.15.139:3300/0 conn(0x7fb42c0b69c0 0x7fb42c073290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.951+0000 7fb430928640 1 -- 172.21.15.98:0/3113964432 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb42c0c8c80 con 0x7fb42c075680 2024-09-19T08:46:13.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.951+0000 7fb430928640 1 --2- 172.21.15.98:0/3113964432 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb42c075680 0x7fb42c0b6480 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fb42c0b4210 tx=0x7fb41c01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0ae0f038401d385 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:13.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.951+0000 7fb423fff640 1 -- 172.21.15.98:0/3113964432 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c0b0070 con 0x7fb42c075680 2024-09-19T08:46:13.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.951+0000 7fb423fff640 1 -- 172.21.15.98:0/3113964432 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb41c0ab600 con 0x7fb42c075680 2024-09-19T08:46:13.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.951+0000 7fb423fff640 1 -- 172.21.15.98:0/3113964432 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb41c0ab9d0 con 0x7fb42c075680 2024-09-19T08:46:13.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.955+0000 7fb43338d640 1 -- 172.21.15.98:0/3113964432 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb42c075680 msgr2=0x7fb42c0b6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:13.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.955+0000 7fb43338d640 1 --2- 172.21.15.98:0/3113964432 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb42c075680 0x7fb42c0b6480 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fb42c0b4210 tx=0x7fb41c01c260 comp rx=0 tx=0).stop 2024-09-19T08:46:13.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.955+0000 7fb43338d640 1 -- 172.21.15.98:0/3113964432 shutdown_connections 2024-09-19T08:46:13.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.955+0000 7fb43338d640 1 --2- 172.21.15.98:0/3113964432 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb42c075680 0x7fb42c0b6480 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.955+0000 7fb43338d640 1 --2- 172.21.15.98:0/3113964432 >> v2:172.21.15.139:3300/0 conn(0x7fb42c0b69c0 0x7fb42c073290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.955+0000 7fb43338d640 1 --2- 172.21.15.98:0/3113964432 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb42c0cb2c0 0x7fb42c06d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.955+0000 7fb43338d640 1 -- 172.21.15.98:0/3113964432 >> 172.21.15.98:0/3113964432 conn(0x7fb42c0b2e60 msgr2=0x7fb42c0b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:13.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.955+0000 7fb43338d640 1 -- 172.21.15.98:0/3113964432 shutdown_connections 2024-09-19T08:46:13.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.955+0000 7fb43338d640 1 -- 172.21.15.98:0/3113964432 wait complete. 2024-09-19T08:46:13.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.955+0000 7fb43338d640 1 Processor -- start 2024-09-19T08:46:13.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.955+0000 7fb43338d640 1 -- start start 2024-09-19T08:46:13.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.955+0000 7fb43338d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb42c075680 0x7fb42c155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:13.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.955+0000 7fb43338d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb42c0b69c0 0x7fb42c0cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:13.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.955+0000 7fb43338d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb42c0cb2c0 0x7fb42c0993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:13.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.955+0000 7fb43338d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c0c9810 con 0x7fb42c075680 2024-09-19T08:46:13.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.955+0000 7fb43338d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb42c0c9690 con 0x7fb42c0b69c0 2024-09-19T08:46:13.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.955+0000 7fb43338d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb42c0c9990 con 0x7fb42c0cb2c0 2024-09-19T08:46:13.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.955+0000 7fb431129640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb42c075680 0x7fb42c155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:13.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.955+0000 7fb431129640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb42c075680 0x7fb42c155b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46552/0 (socket says 172.21.15.98:46552) 2024-09-19T08:46:13.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.955+0000 7fb431129640 1 -- 172.21.15.98:0/787609231 learned_addr learned my addr 172.21.15.98:0/787609231 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:13.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.955+0000 7fb43192a640 1 --2- 172.21.15.98:0/787609231 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb42c0cb2c0 0x7fb42c0993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:13.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.955+0000 7fb430928640 1 --2- 172.21.15.98:0/787609231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb42c0b69c0 0x7fb42c0cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:13.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.955+0000 7fb431129640 1 -- 172.21.15.98:0/787609231 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb42c0cb2c0 msgr2=0x7fb42c0993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:13.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.955+0000 7fb431129640 1 --2- 172.21.15.98:0/787609231 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb42c0cb2c0 0x7fb42c0993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.955+0000 7fb431129640 1 -- 172.21.15.98:0/787609231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb42c0b69c0 msgr2=0x7fb42c0cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:13.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.955+0000 7fb431129640 1 --2- 172.21.15.98:0/787609231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb42c0b69c0 0x7fb42c0cb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:13.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.955+0000 7fb431129640 1 -- 172.21.15.98:0/787609231 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb42c152d20 con 0x7fb42c075680 2024-09-19T08:46:13.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.955+0000 7fb43192a640 1 --2- 172.21.15.98:0/787609231 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb42c0cb2c0 0x7fb42c0993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:13.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.955+0000 7fb430928640 1 --2- 172.21.15.98:0/787609231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb42c0b69c0 0x7fb42c0cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:13.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.955+0000 7fb431129640 1 --2- 172.21.15.98:0/787609231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb42c075680 0x7fb42c155b50 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fb42800b440 tx=0x7fb42800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:13.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.955+0000 7fb421ffb640 1 -- 172.21.15.98:0/787609231 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb428015020 con 0x7fb42c075680 2024-09-19T08:46:13.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.955+0000 7fb421ffb640 1 -- 172.21.15.98:0/787609231 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4280118b0 con 0x7fb42c075680 2024-09-19T08:46:13.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.955+0000 7fb421ffb640 1 -- 172.21.15.98:0/787609231 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb428011ca0 con 0x7fb42c075680 2024-09-19T08:46:13.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.955+0000 7fb43338d640 1 -- 172.21.15.98:0/787609231 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb42c153000 con 0x7fb42c075680 2024-09-19T08:46:13.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.955+0000 7fb43338d640 1 -- 172.21.15.98:0/787609231 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb42c098ec0 con 0x7fb42c075680 2024-09-19T08:46:13.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.959+0000 7fb43338d640 1 -- 172.21.15.98:0/787609231 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb42c0b6480 con 0x7fb42c075680 2024-09-19T08:46:13.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.959+0000 7fb421ffb640 1 -- 172.21.15.98:0/787609231 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fb428018020 con 0x7fb42c075680 2024-09-19T08:46:13.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.959+0000 7fb421ffb640 1 --2- 172.21.15.98:0/787609231 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb410067ef0 0x7fb41006a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:13.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.959+0000 7fb421ffb640 1 -- 172.21.15.98:0/787609231 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5737+0+0 (secure 0 0 0) 0x7fb428086a60 con 0x7fb42c075680 2024-09-19T08:46:13.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.959+0000 7fb430928640 1 --2- 172.21.15.98:0/787609231 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb410067ef0 0x7fb41006a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:13.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.959+0000 7fb430928640 1 --2- 172.21.15.98:0/787609231 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb410067ef0 0x7fb41006a3b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb41c0ad000 tx=0x7fb41c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:13.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:13.963+0000 7fb421ffb640 1 -- 172.21.15.98:0/787609231 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb42801f020 con 0x7fb42c075680 2024-09-19T08:46:14.162 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:14.163+0000 7fb43338d640 1 -- 172.21.15.98:0/787609231 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fb42c067ba0 con 0x7fb42c075680 2024-09-19T08:46:14.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:14.999+0000 7fb421ffb640 1 -- 172.21.15.98:0/787609231 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v83) ==== 136+0+0 (secure 0 0 0) 0x7fb42800d040 con 0x7fb42c075680 2024-09-19T08:46:15.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.003+0000 7fb43338d640 1 -- 172.21.15.98:0/787609231 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb410067ef0 msgr2=0x7fb41006a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:15.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.003+0000 7fb43338d640 1 --2- 172.21.15.98:0/787609231 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb410067ef0 0x7fb41006a3b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb41c0ad000 tx=0x7fb41c008040 comp rx=0 tx=0).stop 2024-09-19T08:46:15.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.003+0000 7fb43338d640 1 -- 172.21.15.98:0/787609231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb42c075680 msgr2=0x7fb42c155b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:15.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.003+0000 7fb43338d640 1 --2- 172.21.15.98:0/787609231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb42c075680 0x7fb42c155b50 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fb42800b440 tx=0x7fb42800b900 comp rx=0 tx=0).stop 2024-09-19T08:46:15.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.003+0000 7fb43338d640 1 -- 172.21.15.98:0/787609231 shutdown_connections 2024-09-19T08:46:15.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.003+0000 7fb43338d640 1 --2- 172.21.15.98:0/787609231 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb410067ef0 0x7fb41006a3b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:15.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.003+0000 7fb43338d640 1 --2- 172.21.15.98:0/787609231 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb42c0cb2c0 0x7fb42c0993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:15.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.003+0000 7fb43338d640 1 --2- 172.21.15.98:0/787609231 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb42c0b69c0 0x7fb42c0cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:15.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.003+0000 7fb43338d640 1 --2- 172.21.15.98:0/787609231 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb42c075680 0x7fb42c155b50 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:15.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.003+0000 7fb43338d640 1 -- 172.21.15.98:0/787609231 >> 172.21.15.98:0/787609231 conn(0x7fb42c0b2e60 msgr2=0x7fb42c09a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:15.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.003+0000 7fb43338d640 1 -- 172.21.15.98:0/787609231 shutdown_connections 2024-09-19T08:46:15.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.003+0000 7fb43338d640 1 -- 172.21.15.98:0/787609231 wait complete. 2024-09-19T08:46:15.021 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-19T08:46:15.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.151+0000 7fe60a17c640 1 Processor -- start 2024-09-19T08:46:15.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.151+0000 7fe60a17c640 1 -- start start 2024-09-19T08:46:15.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.151+0000 7fe60a17c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe604140d30 0x7fe60406d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:15.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.151+0000 7fe60a17c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe604141110 0x7fe60406d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:15.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.151+0000 7fe60a17c640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe60415ea10 con 0x7fe6040b3310 2024-09-19T08:46:15.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.151+0000 7fe60a17c640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fe60415e7c0 con 0x7fe604141110 2024-09-19T08:46:15.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.151+0000 7fe60a17c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe60415ecf0 con 0x7fe604140d30 2024-09-19T08:46:15.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.151+0000 7fe602ffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe604141110 0x7fe60406d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:15.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.151+0000 7fe603fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fe6040b3310 0x7fe6040b3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36886/0 (socket says 172.21.15.98:36886) 2024-09-19T08:46:15.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.151+0000 7fe602ffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe604141110 0x7fe60406d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46562/0 (socket says 172.21.15.98:46562) 2024-09-19T08:46:15.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.151+0000 7fe603fff640 1 -- 172.21.15.98:0/438118215 learned_addr learned my addr 172.21.15.98:0/438118215 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:15.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.151+0000 7fe6037fe640 1 --2- 172.21.15.98:0/438118215 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe604140d30 0x7fe60406d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:15.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.151+0000 7fe602ffd640 1 -- 172.21.15.98:0/438118215 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe604140d30 msgr2=0x7fe60406d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:15.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.151+0000 7fe602ffd640 1 --2- 172.21.15.98:0/438118215 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe604140d30 0x7fe60406d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:15.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.151+0000 7fe602ffd640 1 -- 172.21.15.98:0/438118215 >> v1:172.21.15.98:6789/0 conn(0x7fe6040b3310 legacy=0x7fe6040b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:15.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.151+0000 7fe602ffd640 1 -- 172.21.15.98:0/438118215 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe60415efd0 con 0x7fe604141110 2024-09-19T08:46:15.152 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.155+0000 7fe602ffd640 1 --2- 172.21.15.98:0/438118215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe604141110 0x7fe60406d6b0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fe5f80097c0 tx=0x7fe5f801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f744512170f3a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:15.152 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.155+0000 7fe6027fc640 1 -- 172.21.15.98:0/438118215 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f80aa620 con 0x7fe604141110 2024-09-19T08:46:15.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.155+0000 7fe6027fc640 1 -- 172.21.15.98:0/438118215 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5f80af070 con 0x7fe604141110 2024-09-19T08:46:15.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.155+0000 7fe6027fc640 1 -- 172.21.15.98:0/438118215 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f80aa9e0 con 0x7fe604141110 2024-09-19T08:46:15.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.155+0000 7fe60a17c640 1 -- 172.21.15.98:0/438118215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe604141110 msgr2=0x7fe60406d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:15.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.155+0000 7fe60a17c640 1 --2- 172.21.15.98:0/438118215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe604141110 0x7fe60406d6b0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fe5f80097c0 tx=0x7fe5f801c4a0 comp rx=0 tx=0).stop 2024-09-19T08:46:15.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.155+0000 7fe60a17c640 1 -- 172.21.15.98:0/438118215 shutdown_connections 2024-09-19T08:46:15.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.155+0000 7fe60a17c640 1 --2- 172.21.15.98:0/438118215 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe604141110 0x7fe60406d6b0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:15.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.155+0000 7fe60a17c640 1 --2- 172.21.15.98:0/438118215 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe604140d30 0x7fe60406d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:15.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.155+0000 7fe60a17c640 1 -- 172.21.15.98:0/438118215 >> 172.21.15.98:0/438118215 conn(0x7fe6040b2010 msgr2=0x7fe6040b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:15.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.155+0000 7fe60a17c640 1 -- 172.21.15.98:0/438118215 shutdown_connections 2024-09-19T08:46:15.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.155+0000 7fe60a17c640 1 -- 172.21.15.98:0/438118215 wait complete. 2024-09-19T08:46:15.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.155+0000 7fe60a17c640 1 Processor -- start 2024-09-19T08:46:15.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.155+0000 7fe60a17c640 1 -- start start 2024-09-19T08:46:15.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.159+0000 7fe60a17c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6040b3310 0x7fe60413fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:15.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.159+0000 7fe60a17c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe604140d30 0x7fe60415fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:15.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.159+0000 7fe60a17c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe604141110 0x7fe6041460d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:15.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.159+0000 7fe60a17c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe60415bdd0 con 0x7fe6040b3310 2024-09-19T08:46:15.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.159+0000 7fe60a17c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe60415bc50 con 0x7fe604141110 2024-09-19T08:46:15.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.159+0000 7fe60a17c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe60415bf50 con 0x7fe604140d30 2024-09-19T08:46:15.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.159+0000 7fe6037fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6040b3310 0x7fe60413fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:15.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.159+0000 7fe6037fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6040b3310 0x7fe60413fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46568/0 (socket says 172.21.15.98:46568) 2024-09-19T08:46:15.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.159+0000 7fe6037fe640 1 -- 172.21.15.98:0/486536542 learned_addr learned my addr 172.21.15.98:0/486536542 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:15.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.159+0000 7fe602ffd640 1 --2- 172.21.15.98:0/486536542 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe604140d30 0x7fe60415fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:15.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.159+0000 7fe603fff640 1 --2- 172.21.15.98:0/486536542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe604141110 0x7fe6041460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:15.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.159+0000 7fe6037fe640 1 -- 172.21.15.98:0/486536542 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe604140d30 msgr2=0x7fe60415fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:15.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.159+0000 7fe6037fe640 1 --2- 172.21.15.98:0/486536542 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe604140d30 0x7fe60415fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:15.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.159+0000 7fe6037fe640 1 -- 172.21.15.98:0/486536542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe604141110 msgr2=0x7fe6041460d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:15.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.159+0000 7fe6037fe640 1 --2- 172.21.15.98:0/486536542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe604141110 0x7fe6041460d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:15.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.159+0000 7fe6037fe640 1 -- 172.21.15.98:0/486536542 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe604145050 con 0x7fe6040b3310 2024-09-19T08:46:15.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.159+0000 7fe6037fe640 1 --2- 172.21.15.98:0/486536542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6040b3310 0x7fe60413fde0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fe5f400c820 tx=0x7fe5f400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:15.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.159+0000 7fe5ebfff640 1 -- 172.21.15.98:0/486536542 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4015070 con 0x7fe6040b3310 2024-09-19T08:46:15.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.159+0000 7fe5ebfff640 1 -- 172.21.15.98:0/486536542 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5f4004240 con 0x7fe6040b3310 2024-09-19T08:46:15.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.159+0000 7fe60a17c640 1 -- 172.21.15.98:0/486536542 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe604145330 con 0x7fe6040b3310 2024-09-19T08:46:15.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.159+0000 7fe5ebfff640 1 -- 172.21.15.98:0/486536542 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f400ce10 con 0x7fe6040b3310 2024-09-19T08:46:15.159 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.159+0000 7fe60a17c640 1 -- 172.21.15.98:0/486536542 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe604145bd0 con 0x7fe6040b3310 2024-09-19T08:46:15.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.159+0000 7fe60a17c640 1 -- 172.21.15.98:0/486536542 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe604145500 con 0x7fe6040b3310 2024-09-19T08:46:15.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.159+0000 7fe5ebfff640 1 -- 172.21.15.98:0/486536542 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fe5f4007830 con 0x7fe6040b3310 2024-09-19T08:46:15.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.159+0000 7fe5ebfff640 1 --2- 172.21.15.98:0/486536542 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe5f0067ef0 0x7fe5f006a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:15.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.159+0000 7fe5ebfff640 1 -- 172.21.15.98:0/486536542 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5737+0+0 (secure 0 0 0) 0x7fe5f40867e0 con 0x7fe6040b3310 2024-09-19T08:46:15.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.159+0000 7fe602ffd640 1 --2- 172.21.15.98:0/486536542 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe5f0067ef0 0x7fe5f006a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:15.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.163+0000 7fe602ffd640 1 --2- 172.21.15.98:0/486536542 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe5f0067ef0 0x7fe5f006a3b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe5f80ae000 tx=0x7fe5f80b5040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:15.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.163+0000 7fe5ebfff640 1 -- 172.21.15.98:0/486536542 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe5f40568b0 con 0x7fe6040b3310 2024-09-19T08:46:15.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.347+0000 7fe60a17c640 1 -- 172.21.15.98:0/486536542 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fe6040b1df0 con 0x7fe5f0067ef0 2024-09-19T08:46:15.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.351+0000 7fe5ebfff640 1 -- 172.21.15.98:0/486536542 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fe6040b1df0 con 0x7fe5f0067ef0 2024-09-19T08:46:15.351 INFO:teuthology.orchestra.run.smithi098.stdout:[ 2024-09-19T08:46:15.351 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:15.351 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "cephfs" 2024-09-19T08:46:15.351 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:15.351 INFO:teuthology.orchestra.run.smithi098.stdout:] 2024-09-19T08:46:15.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.355+0000 7fe60a17c640 1 -- 172.21.15.98:0/486536542 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe5f0067ef0 msgr2=0x7fe5f006a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:15.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.355+0000 7fe60a17c640 1 --2- 172.21.15.98:0/486536542 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe5f0067ef0 0x7fe5f006a3b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe5f80ae000 tx=0x7fe5f80b5040 comp rx=0 tx=0).stop 2024-09-19T08:46:15.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.355+0000 7fe60a17c640 1 -- 172.21.15.98:0/486536542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6040b3310 msgr2=0x7fe60413fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:15.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.355+0000 7fe60a17c640 1 --2- 172.21.15.98:0/486536542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6040b3310 0x7fe60413fde0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fe5f400c820 tx=0x7fe5f400cce0 comp rx=0 tx=0).stop 2024-09-19T08:46:15.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.355+0000 7fe60a17c640 1 -- 172.21.15.98:0/486536542 shutdown_connections 2024-09-19T08:46:15.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.355+0000 7fe60a17c640 1 --2- 172.21.15.98:0/486536542 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe5f0067ef0 0x7fe5f006a3b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:15.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.355+0000 7fe60a17c640 1 --2- 172.21.15.98:0/486536542 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe604141110 0x7fe6041460d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:15.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.355+0000 7fe60a17c640 1 --2- 172.21.15.98:0/486536542 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe604140d30 0x7fe60415fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:15.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.355+0000 7fe60a17c640 1 --2- 172.21.15.98:0/486536542 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe6040b3310 0x7fe60413fde0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:15.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.355+0000 7fe60a17c640 1 -- 172.21.15.98:0/486536542 >> 172.21.15.98:0/486536542 conn(0x7fe6040b2010 msgr2=0x7fe60415d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:15.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.355+0000 7fe60a17c640 1 -- 172.21.15.98:0/486536542 shutdown_connections 2024-09-19T08:46:15.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.355+0000 7fe60a17c640 1 -- 172.21.15.98:0/486536542 wait complete. 2024-09-19T08:46:15.370 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T08:46:15.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.487+0000 7fe1fff32640 1 Processor -- start 2024-09-19T08:46:15.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.487+0000 7fe1fff32640 1 -- start start 2024-09-19T08:46:15.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.487+0000 7fe1fff32640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe1f8144d10 0x7fe1f80722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:15.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.487+0000 7fe1fff32640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1f806c3a0 0x7fe1f80cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:15.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.487+0000 7fe1fff32640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1f8072ea0 con 0x7fe1f8072820 2024-09-19T08:46:15.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.487+0000 7fe1fff32640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe1f8072c60 con 0x7fe1f8144d10 2024-09-19T08:46:15.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.487+0000 7fe1fff32640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe1f8073ab0 con 0x7fe1f806c3a0 2024-09-19T08:46:15.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.487+0000 7fe1fe4cf640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1f806c3a0 0x7fe1f80cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:15.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.487+0000 7fe1fdcce640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe1f8144d10 0x7fe1f80722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:15.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.487+0000 7fe1fdcce640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe1f8144d10 0x7fe1f80722e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:58368/0 (socket says 172.21.15.98:58368) 2024-09-19T08:46:15.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.487+0000 7fe1fdcce640 1 -- 172.21.15.98:0/2789155202 learned_addr learned my addr 172.21.15.98:0/2789155202 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:15.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.487+0000 7fe1fdcce640 1 -- 172.21.15.98:0/2789155202 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1f806c3a0 msgr2=0x7fe1f80cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:15.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.487+0000 7fe1fdcce640 1 --2- 172.21.15.98:0/2789155202 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1f806c3a0 0x7fe1f80cd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:15.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.487+0000 7fe1fdcce640 1 -- 172.21.15.98:0/2789155202 >> v1:172.21.15.139:6789/0 conn(0x7fe1f8072820 legacy=0x7fe1f806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:15.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.487+0000 7fe1fdcce640 1 -- 172.21.15.98:0/2789155202 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f8073180 con 0x7fe1f8144d10 2024-09-19T08:46:15.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.491+0000 7fe1fdcce640 1 --2- 172.21.15.98:0/2789155202 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1f8144d10 0x7fe1f80722e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009e50 tx=0x7fe1f401ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a8496dc6388cade server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:15.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.491+0000 7fe1fcccc640 1 -- 172.21.15.98:0/2789155202 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f40a57f0 con 0x7fe1f8144d10 2024-09-19T08:46:15.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.491+0000 7fe1fcccc640 1 -- 172.21.15.98:0/2789155202 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1f40a5990 con 0x7fe1f8144d10 2024-09-19T08:46:15.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.491+0000 7fe1fcccc640 1 -- 172.21.15.98:0/2789155202 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f40a5d60 con 0x7fe1f8144d10 2024-09-19T08:46:15.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.491+0000 7fe1fff32640 1 -- 172.21.15.98:0/2789155202 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1f8144d10 msgr2=0x7fe1f80722e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:15.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.491+0000 7fe1fff32640 1 --2- 172.21.15.98:0/2789155202 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1f8144d10 0x7fe1f80722e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009e50 tx=0x7fe1f401ca30 comp rx=0 tx=0).stop 2024-09-19T08:46:15.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.491+0000 7fe1fff32640 1 -- 172.21.15.98:0/2789155202 shutdown_connections 2024-09-19T08:46:15.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.491+0000 7fe1fff32640 1 --2- 172.21.15.98:0/2789155202 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1f806c3a0 0x7fe1f80cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:15.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.491+0000 7fe1fff32640 1 --2- 172.21.15.98:0/2789155202 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1f8144d10 0x7fe1f80722e0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:15.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.491+0000 7fe1fff32640 1 -- 172.21.15.98:0/2789155202 >> 172.21.15.98:0/2789155202 conn(0x7fe1f80b2010 msgr2=0x7fe1f80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:15.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.491+0000 7fe1fff32640 1 -- 172.21.15.98:0/2789155202 shutdown_connections 2024-09-19T08:46:15.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.491+0000 7fe1fff32640 1 -- 172.21.15.98:0/2789155202 wait complete. 2024-09-19T08:46:15.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.491+0000 7fe1fff32640 1 Processor -- start 2024-09-19T08:46:15.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.491+0000 7fe1fff32640 1 -- start start 2024-09-19T08:46:15.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.495+0000 7fe1fff32640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1f806c3a0 0x7fe1f813a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:15.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.495+0000 7fe1fff32640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1f8072820 0x7fe1f8140490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:15.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.495+0000 7fe1fff32640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1f8144d10 0x7fe1f81550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:15.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.495+0000 7fe1fff32640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f80ce120 con 0x7fe1f806c3a0 2024-09-19T08:46:15.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.495+0000 7fe1fff32640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f80cdfa0 con 0x7fe1f8072820 2024-09-19T08:46:15.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.495+0000 7fe1fff32640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe1f80ce2a0 con 0x7fe1f8144d10 2024-09-19T08:46:15.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.495+0000 7fe1fdcce640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1f806c3a0 0x7fe1f813a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:15.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.495+0000 7fe1fdcce640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1f806c3a0 0x7fe1f813a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46574/0 (socket says 172.21.15.98:46574) 2024-09-19T08:46:15.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.495+0000 7fe1fdcce640 1 -- 172.21.15.98:0/1701552274 learned_addr learned my addr 172.21.15.98:0/1701552274 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:15.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.495+0000 7fe1fd4cd640 1 --2- 172.21.15.98:0/1701552274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1f8072820 0x7fe1f8140490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:15.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.495+0000 7fe1fe4cf640 1 --2- 172.21.15.98:0/1701552274 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1f8144d10 0x7fe1f81550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:15.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.495+0000 7fe1fdcce640 1 -- 172.21.15.98:0/1701552274 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1f8144d10 msgr2=0x7fe1f81550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:15.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.495+0000 7fe1fdcce640 1 --2- 172.21.15.98:0/1701552274 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1f8144d10 0x7fe1f81550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:15.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.495+0000 7fe1fdcce640 1 -- 172.21.15.98:0/1701552274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1f8072820 msgr2=0x7fe1f8140490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:15.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.495+0000 7fe1fdcce640 1 --2- 172.21.15.98:0/1701552274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1f8072820 0x7fe1f8140490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:15.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.495+0000 7fe1fdcce640 1 -- 172.21.15.98:0/1701552274 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f813ad40 con 0x7fe1f806c3a0 2024-09-19T08:46:15.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.495+0000 7fe1fe4cf640 1 --2- 172.21.15.98:0/1701552274 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1f8144d10 0x7fe1f81550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:15.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.495+0000 7fe1fdcce640 1 --2- 172.21.15.98:0/1701552274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1f806c3a0 0x7fe1f813a800 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fe1f401a660 tx=0x7fe1f40197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:15.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.495+0000 7fe1ee7fc640 1 -- 172.21.15.98:0/1701552274 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f40bc020 con 0x7fe1f806c3a0 2024-09-19T08:46:15.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.495+0000 7fe1ee7fc640 1 -- 172.21.15.98:0/1701552274 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1f40b8670 con 0x7fe1f806c3a0 2024-09-19T08:46:15.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.495+0000 7fe1fff32640 1 -- 172.21.15.98:0/1701552274 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1f8154330 con 0x7fe1f806c3a0 2024-09-19T08:46:15.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.495+0000 7fe1ee7fc640 1 -- 172.21.15.98:0/1701552274 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f40b8a40 con 0x7fe1f806c3a0 2024-09-19T08:46:15.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.495+0000 7fe1fff32640 1 -- 172.21.15.98:0/1701552274 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1f81548f0 con 0x7fe1f806c3a0 2024-09-19T08:46:15.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.495+0000 7fe1fff32640 1 -- 172.21.15.98:0/1701552274 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1f80cd250 con 0x7fe1f806c3a0 2024-09-19T08:46:15.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.495+0000 7fe1ee7fc640 1 -- 172.21.15.98:0/1701552274 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fe1f40bf020 con 0x7fe1f806c3a0 2024-09-19T08:46:15.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.495+0000 7fe1ee7fc640 1 --2- 172.21.15.98:0/1701552274 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe1dc067ef0 0x7fe1dc06a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:15.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.495+0000 7fe1ee7fc640 1 -- 172.21.15.98:0/1701552274 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5737+0+0 (secure 0 0 0) 0x7fe1f4121810 con 0x7fe1f806c3a0 2024-09-19T08:46:15.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.495+0000 7fe1fd4cd640 1 --2- 172.21.15.98:0/1701552274 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe1dc067ef0 0x7fe1dc06a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:15.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.499+0000 7fe1fd4cd640 1 --2- 172.21.15.98:0/1701552274 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe1dc067ef0 0x7fe1dc06a3b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe1f80b0850 tx=0x7fe1e8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:15.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.499+0000 7fe1ee7fc640 1 -- 172.21.15.98:0/1701552274 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe1f40c6030 con 0x7fe1f806c3a0 2024-09-19T08:46:15.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.659+0000 7fe1fff32640 1 -- 172.21.15.98:0/1701552274 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe1f80b05a0 con 0x7fe1f806c3a0 2024-09-19T08:46:15.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.659+0000 7fe1ee7fc640 1 -- 172.21.15.98:0/1701552274 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7fe1f4016620 con 0x7fe1f806c3a0 2024-09-19T08:46:15.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.659+0000 7fe1fff32640 1 -- 172.21.15.98:0/1701552274 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe1dc067ef0 msgr2=0x7fe1dc06a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:15.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.659+0000 7fe1fff32640 1 --2- 172.21.15.98:0/1701552274 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe1dc067ef0 0x7fe1dc06a3b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe1f80b0850 tx=0x7fe1e8009040 comp rx=0 tx=0).stop 2024-09-19T08:46:15.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.659+0000 7fe1fff32640 1 -- 172.21.15.98:0/1701552274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1f806c3a0 msgr2=0x7fe1f813a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:15.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.659+0000 7fe1fff32640 1 --2- 172.21.15.98:0/1701552274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1f806c3a0 0x7fe1f813a800 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fe1f401a660 tx=0x7fe1f40197d0 comp rx=0 tx=0).stop 2024-09-19T08:46:15.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.659+0000 7fe1fff32640 1 -- 172.21.15.98:0/1701552274 shutdown_connections 2024-09-19T08:46:15.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.659+0000 7fe1fff32640 1 --2- 172.21.15.98:0/1701552274 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe1dc067ef0 0x7fe1dc06a3b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:15.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.659+0000 7fe1fff32640 1 --2- 172.21.15.98:0/1701552274 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe1f8144d10 0x7fe1f81550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:15.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.659+0000 7fe1fff32640 1 --2- 172.21.15.98:0/1701552274 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1f8072820 0x7fe1f8140490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:15.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.659+0000 7fe1fff32640 1 --2- 172.21.15.98:0/1701552274 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1f806c3a0 0x7fe1f813a800 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:15.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.663+0000 7fe1fff32640 1 -- 172.21.15.98:0/1701552274 >> 172.21.15.98:0/1701552274 conn(0x7fe1f80b2010 msgr2=0x7fe1f80ce6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:15.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.663+0000 7fe1fff32640 1 -- 172.21.15.98:0/1701552274 shutdown_connections 2024-09-19T08:46:15.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.663+0000 7fe1fff32640 1 -- 172.21.15.98:0/1701552274 wait complete. 2024-09-19T08:46:15.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:15.681 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/_nogroup/subvol54610) 2024-09-19T08:46:15.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:15.704 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp53040/subvol64351) 2024-09-19T08:46:15.768 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol54610 2024-09-19T08:46:15.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.931+0000 7fa704099640 1 Processor -- start 2024-09-19T08:46:15.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.931+0000 7fa704099640 1 -- start start 2024-09-19T08:46:15.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.931+0000 7fa704099640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa6fc0af970 0x7fa6fc15ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:15.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.931+0000 7fa704099640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6fc0719f0 0x7fa6fc15fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:15.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.931+0000 7fa704099640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6fc1624a0 con 0x7fa6fc158850 2024-09-19T08:46:15.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.931+0000 7fa704099640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fa6fc162780 con 0x7fa6fc0af970 2024-09-19T08:46:15.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.931+0000 7fa704099640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa6fc162a60 con 0x7fa6fc0719f0 2024-09-19T08:46:15.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.931+0000 7fa701634640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fa6fc0af970 0x7fa6fc15ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:15.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.931+0000 7fa701e35640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fa6fc158850 0x7fa6fc0712e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36888/0 (socket says 172.21.15.98:36888) 2024-09-19T08:46:15.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.931+0000 7fa701e35640 1 -- 172.21.15.98:0/1147675781 learned_addr learned my addr 172.21.15.98:0/1147675781 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:15.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.931+0000 7fa701634640 1 -- 172.21.15.98:0/1147675781 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6fc0719f0 msgr2=0x7fa6fc15fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:15.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.931+0000 7fa701634640 1 --2- 172.21.15.98:0/1147675781 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6fc0719f0 0x7fa6fc15fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:15.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.931+0000 7fa701634640 1 -- 172.21.15.98:0/1147675781 >> v1:172.21.15.98:6789/0 conn(0x7fa6fc158850 legacy=0x7fa6fc0712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:15.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.931+0000 7fa701634640 1 -- 172.21.15.98:0/1147675781 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6fc162d40 con 0x7fa6fc0af970 2024-09-19T08:46:15.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.931+0000 7fa700e33640 1 -- 172.21.15.98:0/1147675781 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1747632141 0 0) 0x7fa6fc1624a0 con 0x7fa6fc158850 2024-09-19T08:46:15.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.935+0000 7fa701634640 1 --2- 172.21.15.98:0/1147675781 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6fc0af970 0x7fa6fc15ea40 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0097c0 tx=0x7fa6ec01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4278b971fdcc3bbc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:15.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.935+0000 7fa700e33640 1 -- 172.21.15.98:0/1147675781 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec0b0070 con 0x7fa6fc0af970 2024-09-19T08:46:15.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.935+0000 7fa700e33640 1 -- 172.21.15.98:0/1147675781 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6ec0ab600 con 0x7fa6fc0af970 2024-09-19T08:46:15.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.935+0000 7fa700e33640 1 -- 172.21.15.98:0/1147675781 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec0ab9d0 con 0x7fa6fc0af970 2024-09-19T08:46:15.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.935+0000 7fa704099640 1 -- 172.21.15.98:0/1147675781 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6fc0af970 msgr2=0x7fa6fc15ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:15.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.935+0000 7fa704099640 1 --2- 172.21.15.98:0/1147675781 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6fc0af970 0x7fa6fc15ea40 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0097c0 tx=0x7fa6ec01c4a0 comp rx=0 tx=0).stop 2024-09-19T08:46:15.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.935+0000 7fa704099640 1 -- 172.21.15.98:0/1147675781 shutdown_connections 2024-09-19T08:46:15.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.935+0000 7fa704099640 1 --2- 172.21.15.98:0/1147675781 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6fc0719f0 0x7fa6fc15fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:15.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.935+0000 7fa704099640 1 --2- 172.21.15.98:0/1147675781 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6fc0af970 0x7fa6fc15ea40 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:15.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.935+0000 7fa704099640 1 -- 172.21.15.98:0/1147675781 >> 172.21.15.98:0/1147675781 conn(0x7fa6fc075070 msgr2=0x7fa6fc06bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:15.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.935+0000 7fa704099640 1 -- 172.21.15.98:0/1147675781 shutdown_connections 2024-09-19T08:46:15.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.935+0000 7fa704099640 1 -- 172.21.15.98:0/1147675781 wait complete. 2024-09-19T08:46:15.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.935+0000 7fa704099640 1 Processor -- start 2024-09-19T08:46:15.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.935+0000 7fa704099640 1 -- start start 2024-09-19T08:46:15.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.935+0000 7fa704099640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6fc0719f0 0x7fa6fc14b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:15.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.935+0000 7fa704099640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6fc0af970 0x7fa6fc0b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:15.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.935+0000 7fa704099640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6fc158850 0x7fa6fc0cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:15.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.935+0000 7fa704099640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc06e860 con 0x7fa6fc0af970 2024-09-19T08:46:15.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.935+0000 7fa704099640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc06e6e0 con 0x7fa6fc158850 2024-09-19T08:46:15.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.935+0000 7fa704099640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa6fc162240 con 0x7fa6fc0719f0 2024-09-19T08:46:15.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.935+0000 7fa701634640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6fc0af970 0x7fa6fc0b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:15.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.935+0000 7fa701634640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6fc0af970 0x7fa6fc0b0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46598/0 (socket says 172.21.15.98:46598) 2024-09-19T08:46:15.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.935+0000 7fa701634640 1 -- 172.21.15.98:0/2171375326 learned_addr learned my addr 172.21.15.98:0/2171375326 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:15.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.935+0000 7fa701634640 1 -- 172.21.15.98:0/2171375326 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6fc0719f0 msgr2=0x7fa6fc14b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:15.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.935+0000 7fa701634640 1 --2- 172.21.15.98:0/2171375326 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6fc0719f0 0x7fa6fc14b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:15.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.935+0000 7fa702636640 1 --2- 172.21.15.98:0/2171375326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6fc158850 0x7fa6fc0cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:15.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.935+0000 7fa701634640 1 -- 172.21.15.98:0/2171375326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6fc158850 msgr2=0x7fa6fc0cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:15.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.935+0000 7fa701634640 1 --2- 172.21.15.98:0/2171375326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6fc158850 0x7fa6fc0cd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:15.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.935+0000 7fa701634640 1 -- 172.21.15.98:0/2171375326 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6fc0b11e0 con 0x7fa6fc0af970 2024-09-19T08:46:15.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.935+0000 7fa701634640 1 --2- 172.21.15.98:0/2171375326 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6fc0af970 0x7fa6fc0b0ca0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0097c0 tx=0x7fa6ec01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:15.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.935+0000 7fa6f27fc640 1 -- 172.21.15.98:0/2171375326 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec0bd070 con 0x7fa6fc0af970 2024-09-19T08:46:15.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.935+0000 7fa6f27fc640 1 -- 172.21.15.98:0/2171375326 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6ec004230 con 0x7fa6fc0af970 2024-09-19T08:46:15.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.939+0000 7fa704099640 1 -- 172.21.15.98:0/2171375326 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6fc14ff30 con 0x7fa6fc0af970 2024-09-19T08:46:15.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.939+0000 7fa6f27fc640 1 -- 172.21.15.98:0/2171375326 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec0b0040 con 0x7fa6fc0af970 2024-09-19T08:46:15.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.939+0000 7fa704099640 1 -- 172.21.15.98:0/2171375326 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6fc1504b0 con 0x7fa6fc0af970 2024-09-19T08:46:15.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.939+0000 7fa704099640 1 -- 172.21.15.98:0/2171375326 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6fc15fb50 con 0x7fa6fc0af970 2024-09-19T08:46:15.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.939+0000 7fa6f27fc640 1 -- 172.21.15.98:0/2171375326 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7fa6ec0c0070 con 0x7fa6fc0af970 2024-09-19T08:46:15.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.939+0000 7fa6f27fc640 1 --2- 172.21.15.98:0/2171375326 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa6e0067ef0 0x7fa6e006a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:15.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.939+0000 7fa6f27fc640 1 -- 172.21.15.98:0/2171375326 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5737+0+0 (secure 0 0 0) 0x7fa6ec120530 con 0x7fa6fc0af970 2024-09-19T08:46:15.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.939+0000 7fa701e35640 1 --2- 172.21.15.98:0/2171375326 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa6e0067ef0 0x7fa6e006a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:15.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.939+0000 7fa701e35640 1 --2- 172.21.15.98:0/2171375326 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa6e0067ef0 0x7fa6e006a3b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa6f80011f0 tx=0x7fa6f8007930 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:15.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:15.943+0000 7fa6f27fc640 1 -- 172.21.15.98:0/2171375326 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa6ec016090 con 0x7fa6fc0af970 2024-09-19T08:46:16.118 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.119+0000 7fa704099640 1 -- 172.21.15.98:0/2171375326 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol54610", "target": ["mon-mgr", ""]}) -- 0x7fa6fc0b0660 con 0x7fa6e0067ef0 2024-09-19T08:46:16.130 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:46:16.130+0000 7ff73a69c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:46:16.130 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:46:16.130+0000 7ff73a69c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:46:16.130 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:46:16.130+0000 7ff73a69c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:46:16.130 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:46:16.130+0000 7ff73a69c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:46:16.130 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:46:16.130+0000 7ff73a69c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:46:16.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.183+0000 7fa6f27fc640 1 -- 172.21.15.98:0/2171375326 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7fa6fc0b0660 con 0x7fa6e0067ef0 2024-09-19T08:46:16.180 INFO:teuthology.orchestra.run.smithi098.stdout:/volumes/_nogroup/subvol54610 2024-09-19T08:46:16.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.183+0000 7fa704099640 1 -- 172.21.15.98:0/2171375326 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa6e0067ef0 msgr2=0x7fa6e006a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:16.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.183+0000 7fa704099640 1 --2- 172.21.15.98:0/2171375326 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa6e0067ef0 0x7fa6e006a3b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa6f80011f0 tx=0x7fa6f8007930 comp rx=0 tx=0).stop 2024-09-19T08:46:16.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.183+0000 7fa704099640 1 -- 172.21.15.98:0/2171375326 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6fc0af970 msgr2=0x7fa6fc0b0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:16.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.183+0000 7fa704099640 1 --2- 172.21.15.98:0/2171375326 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6fc0af970 0x7fa6fc0b0ca0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0097c0 tx=0x7fa6ec01ace0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.183+0000 7fa704099640 1 -- 172.21.15.98:0/2171375326 shutdown_connections 2024-09-19T08:46:16.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.183+0000 7fa704099640 1 --2- 172.21.15.98:0/2171375326 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa6e0067ef0 0x7fa6e006a3b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.183+0000 7fa704099640 1 --2- 172.21.15.98:0/2171375326 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa6fc158850 0x7fa6fc0cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.183+0000 7fa704099640 1 --2- 172.21.15.98:0/2171375326 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa6fc0af970 0x7fa6fc0b0ca0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.183+0000 7fa704099640 1 --2- 172.21.15.98:0/2171375326 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa6fc0719f0 0x7fa6fc14b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.183+0000 7fa704099640 1 -- 172.21.15.98:0/2171375326 >> 172.21.15.98:0/2171375326 conn(0x7fa6fc075070 msgr2=0x7fa6fc0af230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:16.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.183+0000 7fa704099640 1 -- 172.21.15.98:0/2171375326 shutdown_connections 2024-09-19T08:46:16.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.183+0000 7fa704099640 1 -- 172.21.15.98:0/2171375326 wait complete. 2024-09-19T08:46:16.199 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol64351 subvol_grp53040 2024-09-19T08:46:16.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.315+0000 7f6750130640 1 Processor -- start 2024-09-19T08:46:16.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.315+0000 7f6750130640 1 -- start start 2024-09-19T08:46:16.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.315+0000 7f6750130640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f674815f6e0 0x7f674806d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:16.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.315+0000 7f6750130640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6748073d90 0x7f674806da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:16.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.315+0000 7f6750130640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f674815baf0 con 0x7f674815fac0 2024-09-19T08:46:16.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.315+0000 7f6750130640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f674815c8b0 con 0x7f6748073d90 2024-09-19T08:46:16.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.315+0000 7f6750130640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f67480742d0 con 0x7f674815f6e0 2024-09-19T08:46:16.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.315+0000 7f674e6cd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6748073d90 0x7f674806da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:16.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.315+0000 7f674e6cd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6748073d90 0x7f674806da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46610/0 (socket says 172.21.15.98:46610) 2024-09-19T08:46:16.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.315+0000 7f674e6cd640 1 -- 172.21.15.98:0/1084380876 learned_addr learned my addr 172.21.15.98:0/1084380876 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:16.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.315+0000 7f674decc640 1 --2- 172.21.15.98:0/1084380876 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f674815f6e0 0x7f674806d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:16.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.315+0000 7f674e6cd640 1 -- 172.21.15.98:0/1084380876 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f674815f6e0 msgr2=0x7f674806d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:16.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.315+0000 7f674e6cd640 1 --2- 172.21.15.98:0/1084380876 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f674815f6e0 0x7f674806d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.315+0000 7f674e6cd640 1 -- 172.21.15.98:0/1084380876 >> v1:172.21.15.139:6789/0 conn(0x7f674815fac0 legacy=0x7f6748073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:16.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.315+0000 7f674e6cd640 1 -- 172.21.15.98:0/1084380876 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f674815bdd0 con 0x7f6748073d90 2024-09-19T08:46:16.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.319+0000 7f674e6cd640 1 --2- 172.21.15.98:0/1084380876 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6748073d90 0x7f674806da00 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f6740009e50 tx=0x7f674001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a6671d8adf4267 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:16.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.319+0000 7f674ceca640 1 -- 172.21.15.98:0/1084380876 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67400a5750 con 0x7f6748073d90 2024-09-19T08:46:16.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.319+0000 7f674ceca640 1 -- 172.21.15.98:0/1084380876 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67400a58f0 con 0x7f6748073d90 2024-09-19T08:46:16.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.319+0000 7f674ceca640 1 -- 172.21.15.98:0/1084380876 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67400a5cc0 con 0x7f6748073d90 2024-09-19T08:46:16.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.319+0000 7f6750130640 1 -- 172.21.15.98:0/1084380876 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6748073d90 msgr2=0x7f674806da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:16.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.319+0000 7f6750130640 1 --2- 172.21.15.98:0/1084380876 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6748073d90 0x7f674806da00 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f6740009e50 tx=0x7f674001c9a0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.319+0000 7f6750130640 1 -- 172.21.15.98:0/1084380876 shutdown_connections 2024-09-19T08:46:16.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.319+0000 7f6750130640 1 --2- 172.21.15.98:0/1084380876 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6748073d90 0x7f674806da00 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.319+0000 7f6750130640 1 --2- 172.21.15.98:0/1084380876 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f674815f6e0 0x7f674806d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.319+0000 7f6750130640 1 -- 172.21.15.98:0/1084380876 >> 172.21.15.98:0/1084380876 conn(0x7f674806c090 msgr2=0x7f674806c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:16.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.319+0000 7f6750130640 1 -- 172.21.15.98:0/1084380876 shutdown_connections 2024-09-19T08:46:16.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.319+0000 7f6750130640 1 -- 172.21.15.98:0/1084380876 wait complete. 2024-09-19T08:46:16.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.319+0000 7f6750130640 1 Processor -- start 2024-09-19T08:46:16.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.319+0000 7f6750130640 1 -- start start 2024-09-19T08:46:16.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.323+0000 7f6750130640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6748073d90 0x7f6748149070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:16.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.323+0000 7f6750130640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f674815f6e0 0x7f6748146360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:16.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.323+0000 7f6750130640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f674815fac0 0x7f6748146b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:16.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.323+0000 7f6750130640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f67480cf000 con 0x7f674815fac0 2024-09-19T08:46:16.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.323+0000 7f6750130640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f67480cee80 con 0x7f6748073d90 2024-09-19T08:46:16.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.323+0000 7f6750130640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f67480cf180 con 0x7f674815f6e0 2024-09-19T08:46:16.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.323+0000 7f674e6cd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f674815fac0 0x7f6748146b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:16.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.323+0000 7f674e6cd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f674815fac0 0x7f6748146b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46620/0 (socket says 172.21.15.98:46620) 2024-09-19T08:46:16.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.323+0000 7f674e6cd640 1 -- 172.21.15.98:0/1317777278 learned_addr learned my addr 172.21.15.98:0/1317777278 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:16.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.323+0000 7f674decc640 1 --2- 172.21.15.98:0/1317777278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6748073d90 0x7f6748149070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:16.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.323+0000 7f674d6cb640 1 --2- 172.21.15.98:0/1317777278 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f674815f6e0 0x7f6748146360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:16.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.323+0000 7f674e6cd640 1 -- 172.21.15.98:0/1317777278 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f674815f6e0 msgr2=0x7f6748146360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:16.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.323+0000 7f674e6cd640 1 --2- 172.21.15.98:0/1317777278 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f674815f6e0 0x7f6748146360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.323+0000 7f674e6cd640 1 -- 172.21.15.98:0/1317777278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6748073d90 msgr2=0x7f6748149070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:16.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.323+0000 7f674e6cd640 1 --2- 172.21.15.98:0/1317777278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6748073d90 0x7f6748149070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.323+0000 7f674e6cd640 1 -- 172.21.15.98:0/1317777278 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67480b2500 con 0x7f674815fac0 2024-09-19T08:46:16.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.323+0000 7f674decc640 1 --2- 172.21.15.98:0/1317777278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6748073d90 0x7f6748149070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:16.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.323+0000 7f674e6cd640 1 --2- 172.21.15.98:0/1317777278 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f674815fac0 0x7f6748146b50 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f67400af040 tx=0x7f6740019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:16.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.323+0000 7f673e7fc640 1 -- 172.21.15.98:0/1317777278 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67400bc020 con 0x7f674815fac0 2024-09-19T08:46:16.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.323+0000 7f673e7fc640 1 -- 172.21.15.98:0/1317777278 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67400b8670 con 0x7f674815fac0 2024-09-19T08:46:16.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.323+0000 7f6750130640 1 -- 172.21.15.98:0/1317777278 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6748149830 con 0x7f674815fac0 2024-09-19T08:46:16.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.323+0000 7f673e7fc640 1 -- 172.21.15.98:0/1317777278 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67400b8a60 con 0x7f674815fac0 2024-09-19T08:46:16.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.323+0000 7f6750130640 1 -- 172.21.15.98:0/1317777278 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6748149c80 con 0x7f674815fac0 2024-09-19T08:46:16.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.323+0000 7f6750130640 1 -- 172.21.15.98:0/1317777278 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67480cf5c0 con 0x7f674815fac0 2024-09-19T08:46:16.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.323+0000 7f673e7fc640 1 -- 172.21.15.98:0/1317777278 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f67400bf020 con 0x7f674815fac0 2024-09-19T08:46:16.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.323+0000 7f673e7fc640 1 --2- 172.21.15.98:0/1317777278 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f672c067ef0 0x7f672c06a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:16.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.323+0000 7f673e7fc640 1 -- 172.21.15.98:0/1317777278 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5737+0+0 (secure 0 0 0) 0x7f6740121690 con 0x7f674815fac0 2024-09-19T08:46:16.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.323+0000 7f674decc640 1 --2- 172.21.15.98:0/1317777278 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f672c067ef0 0x7f672c06a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:16.324 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.327+0000 7f674decc640 1 --2- 172.21.15.98:0/1317777278 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f672c067ef0 0x7f672c06a3b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6744009840 tx=0x7f6744008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:16.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.327+0000 7f673e7fc640 1 -- 172.21.15.98:0/1317777278 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f67400c6030 con 0x7f674815fac0 2024-09-19T08:46:16.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.515+0000 7f6750130640 1 -- 172.21.15.98:0/1317777278 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol64351", "group_name": "subvol_grp53040", "target": ["mon-mgr", ""]}) -- 0x7f67480cfe70 con 0x7f672c067ef0 2024-09-19T08:46:16.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.551+0000 7f673e7fc640 1 -- 172.21.15.98:0/1317777278 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6744009e20 con 0x7f672c067ef0 2024-09-19T08:46:16.550 INFO:teuthology.orchestra.run.smithi098.stdout:/volumes/subvol_grp53040/subvol64351 2024-09-19T08:46:16.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.555+0000 7f6750130640 1 -- 172.21.15.98:0/1317777278 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f672c067ef0 msgr2=0x7f672c06a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:16.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.555+0000 7f6750130640 1 --2- 172.21.15.98:0/1317777278 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f672c067ef0 0x7f672c06a3b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6744009840 tx=0x7f6744008040 comp rx=0 tx=0).stop 2024-09-19T08:46:16.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.555+0000 7f6750130640 1 -- 172.21.15.98:0/1317777278 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f674815fac0 msgr2=0x7f6748146b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:16.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.555+0000 7f6750130640 1 --2- 172.21.15.98:0/1317777278 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f674815fac0 0x7f6748146b50 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f67400af040 tx=0x7f6740019740 comp rx=0 tx=0).stop 2024-09-19T08:46:16.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.555+0000 7f6750130640 1 -- 172.21.15.98:0/1317777278 shutdown_connections 2024-09-19T08:46:16.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.555+0000 7f6750130640 1 --2- 172.21.15.98:0/1317777278 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f672c067ef0 0x7f672c06a3b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.555+0000 7f6750130640 1 --2- 172.21.15.98:0/1317777278 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f674815fac0 0x7f6748146b50 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.555+0000 7f6750130640 1 --2- 172.21.15.98:0/1317777278 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f674815f6e0 0x7f6748146360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.555+0000 7f6750130640 1 --2- 172.21.15.98:0/1317777278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6748073d90 0x7f6748149070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.555+0000 7f6750130640 1 -- 172.21.15.98:0/1317777278 >> 172.21.15.98:0/1317777278 conn(0x7f674806c090 msgr2=0x7f674806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:16.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.555+0000 7f6750130640 1 -- 172.21.15.98:0/1317777278 shutdown_connections 2024-09-19T08:46:16.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.555+0000 7f6750130640 1 -- 172.21.15.98:0/1317777278 wait complete. 2024-09-19T08:46:16.573 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol54610 2024-09-19T08:46:16.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.695+0000 7f1dca109640 1 Processor -- start 2024-09-19T08:46:16.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.695+0000 7f1dca109640 1 -- start start 2024-09-19T08:46:16.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.695+0000 7f1dca109640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1dc4145b70 0x7f1dc406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:16.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.695+0000 7f1dca109640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dc406c5f0 con 0x7f1dc406c1d0 2024-09-19T08:46:16.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.695+0000 7f1dca109640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dc4073530 con 0x7f1dc4071ac0 2024-09-19T08:46:16.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.695+0000 7f1dca109640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1dc4071f20 con 0x7f1dc4145b70 2024-09-19T08:46:16.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.695+0000 7f1dbaffd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f1dc406c1d0 0x7f1dc40713b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36892/0 (socket says 172.21.15.98:36892) 2024-09-19T08:46:16.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.695+0000 7f1dbaffd640 1 -- 172.21.15.98:0/1454190610 learned_addr learned my addr 172.21.15.98:0/1454190610 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:16.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.695+0000 7f1dc37fe640 1 --2- 172.21.15.98:0/1454190610 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1dc4145b70 0x7f1dc406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:16.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.695+0000 7f1dc2ffd640 1 -- 172.21.15.98:0/1454190610 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1226378477 0 0) 0x7f1dc406c5f0 con 0x7f1dc406c1d0 2024-09-19T08:46:16.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.695+0000 7f1dc2ffd640 1 -- 172.21.15.98:0/1454190610 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1db0003530 con 0x7f1dc406c1d0 2024-09-19T08:46:16.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.695+0000 7f1dc2ffd640 1 -- 172.21.15.98:0/1454190610 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 20906455 0 0) 0x7f1db0003530 con 0x7f1dc406c1d0 2024-09-19T08:46:16.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.695+0000 7f1dc2ffd640 1 -- 172.21.15.98:0/1454190610 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1dc4145b70 msgr2=0x7f1dc406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:16.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.695+0000 7f1dc2ffd640 1 --2- 172.21.15.98:0/1454190610 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1dc4145b70 0x7f1dc406bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.695+0000 7f1dc2ffd640 1 -- 172.21.15.98:0/1454190610 >> v1:172.21.15.139:6789/0 conn(0x7f1dc4071ac0 legacy=0x7f1dc40b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:16.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.695+0000 7f1dc2ffd640 1 -- 172.21.15.98:0/1454190610 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc4073810 con 0x7f1dc406c1d0 2024-09-19T08:46:16.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.695+0000 7f1dc2ffd640 1 -- 172.21.15.98:0/1454190610 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f1dac001680 con 0x7f1dc406c1d0 2024-09-19T08:46:16.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.695+0000 7f1dc2ffd640 1 -- 172.21.15.98:0/1454190610 >> v1:172.21.15.98:6789/0 conn(0x7f1dc406c1d0 legacy=0x7f1dc40713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:16.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.695+0000 7f1dc2ffd640 1 --2- 172.21.15.98:0/1454190610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1db00039f0 0x7f1db0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:16.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.695+0000 7f1dc2ffd640 1 --2- 172.21.15.98:0/1454190610 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1dc4145b70 0x7f1db0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:16.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.695+0000 7f1dc2ffd640 1 --2- 172.21.15.98:0/1454190610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1db0004d50 0x7f1db00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:16.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.695+0000 7f1dc2ffd640 1 -- 172.21.15.98:0/1454190610 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc4071f20 con 0x7f1db0004d50 2024-09-19T08:46:16.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.695+0000 7f1dc2ffd640 1 -- 172.21.15.98:0/1454190610 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1db000bb00 con 0x7f1db00039f0 2024-09-19T08:46:16.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.695+0000 7f1dc2ffd640 1 -- 172.21.15.98:0/1454190610 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1db000bd00 con 0x7f1dc4145b70 2024-09-19T08:46:16.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.695+0000 7f1dc3fff640 1 --2- 172.21.15.98:0/1454190610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1db0004d50 0x7f1db00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:16.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.695+0000 7f1dbaffd640 1 --2- 172.21.15.98:0/1454190610 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1dc4145b70 0x7f1db0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:16.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.695+0000 7f1dc37fe640 1 --2- 172.21.15.98:0/1454190610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1db00039f0 0x7f1db0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:16.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.695+0000 7f1dc3fff640 1 -- 172.21.15.98:0/1454190610 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1dc4145b70 msgr2=0x7f1db0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:16.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.695+0000 7f1dc3fff640 1 --2- 172.21.15.98:0/1454190610 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1dc4145b70 0x7f1db0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.695+0000 7f1dc3fff640 1 -- 172.21.15.98:0/1454190610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1db00039f0 msgr2=0x7f1db0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:16.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.695+0000 7f1dc3fff640 1 --2- 172.21.15.98:0/1454190610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1db00039f0 0x7f1db0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.695+0000 7f1dc3fff640 1 -- 172.21.15.98:0/1454190610 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db0003530 con 0x7f1db0004d50 2024-09-19T08:46:16.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.695+0000 7f1dc37fe640 1 --2- 172.21.15.98:0/1454190610 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1db00039f0 0x7f1db0003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:16.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.699+0000 7f1dc3fff640 1 --2- 172.21.15.98:0/1454190610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1db0004d50 0x7f1db00051d0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc001e70 tx=0x7f1dbc01c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3bfc393f685f9f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:16.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.699+0000 7f1dc2ffd640 1 -- 172.21.15.98:0/1454190610 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc001a90 con 0x7f1db0004d50 2024-09-19T08:46:16.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.699+0000 7f1dc2ffd640 1 -- 172.21.15.98:0/1454190610 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dbc003b80 con 0x7f1db0004d50 2024-09-19T08:46:16.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.699+0000 7f1dc2ffd640 1 -- 172.21.15.98:0/1454190610 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc005080 con 0x7f1db0004d50 2024-09-19T08:46:16.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.699+0000 7f1dca109640 1 -- 172.21.15.98:0/1454190610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1db0004d50 msgr2=0x7f1db00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:16.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.699+0000 7f1dca109640 1 --2- 172.21.15.98:0/1454190610 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1db0004d50 0x7f1db00051d0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc001e70 tx=0x7f1dbc01c780 comp rx=0 tx=0).stop 2024-09-19T08:46:16.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.699+0000 7f1dc3fff640 1 -- 172.21.15.98:0/1454190610 reap_dead start 2024-09-19T08:46:16.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.699+0000 7f1dca109640 1 -- 172.21.15.98:0/1454190610 shutdown_connections 2024-09-19T08:46:16.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.699+0000 7f1dca109640 1 -- 172.21.15.98:0/1454190610 >> 172.21.15.98:0/1454190610 conn(0x7f1dc40b2010 msgr2=0x7f1dc40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:16.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.699+0000 7f1dca109640 1 -- 172.21.15.98:0/1454190610 shutdown_connections 2024-09-19T08:46:16.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.699+0000 7f1dca109640 1 -- 172.21.15.98:0/1454190610 wait complete. 2024-09-19T08:46:16.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.699+0000 7f1dca109640 1 Processor -- start 2024-09-19T08:46:16.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.699+0000 7f1dca109640 1 -- start start 2024-09-19T08:46:16.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.699+0000 7f1dca109640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1db0004d50 0x7f1dc41e6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:16.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.699+0000 7f1dca109640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1dc413df60 0x7f1dc413c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:16.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.699+0000 7f1dca109640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dc415bbf0 0x7f1dc413f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:16.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.699+0000 7f1dca109640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc4145f50 con 0x7f1dc415bbf0 2024-09-19T08:46:16.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.699+0000 7f1dca109640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc4073dd0 con 0x7f1dc413df60 2024-09-19T08:46:16.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.699+0000 7f1dca109640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1dc41460d0 con 0x7f1db0004d50 2024-09-19T08:46:16.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.699+0000 7f1dc3fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dc415bbf0 0x7f1dc413f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:16.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.699+0000 7f1dc3fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dc415bbf0 0x7f1dc413f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46626/0 (socket says 172.21.15.98:46626) 2024-09-19T08:46:16.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.699+0000 7f1dc3fff640 1 -- 172.21.15.98:0/290395498 learned_addr learned my addr 172.21.15.98:0/290395498 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:16.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.699+0000 7f1dc37fe640 1 --2- 172.21.15.98:0/290395498 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1db0004d50 0x7f1dc41e6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:16.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.699+0000 7f1dc3fff640 1 -- 172.21.15.98:0/290395498 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1db0004d50 msgr2=0x7f1dc41e6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:16.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.699+0000 7f1dc3fff640 1 --2- 172.21.15.98:0/290395498 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1db0004d50 0x7f1dc41e6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.699+0000 7f1dc3fff640 1 -- 172.21.15.98:0/290395498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1dc413df60 msgr2=0x7f1dc413c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:16.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.699+0000 7f1dc3fff640 1 --2- 172.21.15.98:0/290395498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1dc413df60 0x7f1dc413c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.703+0000 7f1dc3fff640 1 -- 172.21.15.98:0/290395498 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc413cec0 con 0x7f1dc415bbf0 2024-09-19T08:46:16.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.703+0000 7f1dc3fff640 1 --2- 172.21.15.98:0/290395498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dc415bbf0 0x7f1dc413f9b0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc01cc60 tx=0x7f1db0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:16.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.703+0000 7f1dc0ff9640 1 -- 172.21.15.98:0/290395498 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc007dc0 con 0x7f1dc415bbf0 2024-09-19T08:46:16.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.703+0000 7f1dc0ff9640 1 -- 172.21.15.98:0/290395498 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dbc0bc070 con 0x7f1dc415bbf0 2024-09-19T08:46:16.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.703+0000 7f1dc0ff9640 1 -- 172.21.15.98:0/290395498 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc005460 con 0x7f1dc415bbf0 2024-09-19T08:46:16.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.703+0000 7f1dca109640 1 -- 172.21.15.98:0/290395498 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dc413d1a0 con 0x7f1dc415bbf0 2024-09-19T08:46:16.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.703+0000 7f1dca109640 1 -- 172.21.15.98:0/290395498 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dc413d760 con 0x7f1dc415bbf0 2024-09-19T08:46:16.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.703+0000 7f1dca109640 1 -- 172.21.15.98:0/290395498 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dc413e3a0 con 0x7f1dc415bbf0 2024-09-19T08:46:16.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.703+0000 7f1dc0ff9640 1 -- 172.21.15.98:0/290395498 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 22) ==== 88006+0+0 (secure 0 0 0) 0x7f1dbc0b1040 con 0x7f1dc415bbf0 2024-09-19T08:46:16.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.703+0000 7f1dc0ff9640 1 --2- 172.21.15.98:0/290395498 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1da4067ef0 0x7f1da406a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:16.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.703+0000 7f1dc0ff9640 1 -- 172.21.15.98:0/290395498 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5737+0+0 (secure 0 0 0) 0x7f1dbc121030 con 0x7f1dc415bbf0 2024-09-19T08:46:16.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.703+0000 7f1dc37fe640 1 --2- 172.21.15.98:0/290395498 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1da4067ef0 0x7f1da406a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:16.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.703+0000 7f1dc37fe640 1 --2- 172.21.15.98:0/290395498 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1da4067ef0 0x7f1da406a3b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1db40099e0 tx=0x7f1db4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:16.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.707+0000 7f1dc0ff9640 1 -- 172.21.15.98:0/290395498 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f1dc413e3a0 con 0x7f1dc415bbf0 2024-09-19T08:46:16.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.883+0000 7f1dca109640 1 -- 172.21.15.98:0/290395498 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol54610", "target": ["mon-mgr", ""]}) -- 0x7f1dc4072420 con 0x7f1da4067ef0 2024-09-19T08:46:16.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.895+0000 7f1dc0ff9640 1 -- 172.21.15.98:0/290395498 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7f1dc4072420 con 0x7f1da4067ef0 2024-09-19T08:46:16.892 INFO:teuthology.orchestra.run.smithi098.stdout:/volumes/_nogroup/subvol54610 2024-09-19T08:46:16.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.895+0000 7f1dca109640 1 -- 172.21.15.98:0/290395498 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1da4067ef0 msgr2=0x7f1da406a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:16.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.895+0000 7f1dca109640 1 --2- 172.21.15.98:0/290395498 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1da4067ef0 0x7f1da406a3b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1db40099e0 tx=0x7f1db4008040 comp rx=0 tx=0).stop 2024-09-19T08:46:16.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.895+0000 7f1dca109640 1 -- 172.21.15.98:0/290395498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dc415bbf0 msgr2=0x7f1dc413f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:16.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.895+0000 7f1dca109640 1 --2- 172.21.15.98:0/290395498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dc415bbf0 0x7f1dc413f9b0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc01cc60 tx=0x7f1db0005840 comp rx=0 tx=0).stop 2024-09-19T08:46:16.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.895+0000 7f1dca109640 1 -- 172.21.15.98:0/290395498 shutdown_connections 2024-09-19T08:46:16.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.895+0000 7f1dca109640 1 --2- 172.21.15.98:0/290395498 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1da4067ef0 0x7f1da406a3b0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.895+0000 7f1dca109640 1 --2- 172.21.15.98:0/290395498 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dc415bbf0 0x7f1dc413f9b0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.895+0000 7f1dca109640 1 --2- 172.21.15.98:0/290395498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1dc413df60 0x7f1dc413c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.895+0000 7f1dca109640 1 --2- 172.21.15.98:0/290395498 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1db0004d50 0x7f1dc41e6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:16.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.895+0000 7f1dca109640 1 -- 172.21.15.98:0/290395498 >> 172.21.15.98:0/290395498 conn(0x7f1dc40b2010 msgr2=0x7f1dc406d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:16.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.895+0000 7f1dca109640 1 -- 172.21.15.98:0/290395498 shutdown_connections 2024-09-19T08:46:16.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:16.895+0000 7f1dca109640 1 -- 172.21.15.98:0/290395498 wait complete. 2024-09-19T08:46:16.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:16.910 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/631d30af32a5e36e3503377773a8570b.meta) 2024-09-19T08:46:18.444 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:46:18.447 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:46:18.837 INFO:teuthology.orchestra.run.smithi098.stdout:[GLOBAL] 2024-09-19T08:46:18.838 INFO:teuthology.orchestra.run.smithi098.stdout:version = 1 2024-09-19T08:46:18.838 INFO:teuthology.orchestra.run.smithi098.stdout:type = subvolume 2024-09-19T08:46:18.838 INFO:teuthology.orchestra.run.smithi098.stdout:path = /volumes/_nogroup/subvol54610 2024-09-19T08:46:18.838 INFO:teuthology.orchestra.run.smithi098.stdout:state = complete 2024-09-19T08:46:18.838 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:46:18.840 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol64351 subvol_grp53040 2024-09-19T08:46:18.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.979+0000 7fa4be6fe640 1 Processor -- start 2024-09-19T08:46:18.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.979+0000 7fa4be6fe640 1 -- start start 2024-09-19T08:46:18.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.979+0000 7fa4be6fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4b815c290 0x7fa4b80cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:18.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.979+0000 7fa4be6fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fa4b80cf700 0x7fa4b815ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:18.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.979+0000 7fa4be6fe640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4b81608c0 con 0x7fa4b806d4c0 2024-09-19T08:46:18.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.979+0000 7fa4be6fe640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fa4b8160e80 con 0x7fa4b80cf700 2024-09-19T08:46:18.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.979+0000 7fa4be6fe640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b80cd510 con 0x7fa4b815c290 2024-09-19T08:46:18.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.979+0000 7fa4bcc9b640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fa4b806d4c0 0x7fa4b80b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:44802/0 (socket says 172.21.15.98:44802) 2024-09-19T08:46:18.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.979+0000 7fa4bcc9b640 1 -- 172.21.15.98:0/3977445162 learned_addr learned my addr 172.21.15.98:0/3977445162 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:18.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.979+0000 7fa4b7fff640 1 --2- 172.21.15.98:0/3977445162 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4b815c290 0x7fa4b80cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:18.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.979+0000 7fa4b6ffd640 1 -- 172.21.15.98:0/3977445162 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2928123108 0 0) 0x7fa4b81608c0 con 0x7fa4b806d4c0 2024-09-19T08:46:18.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.979+0000 7fa4b77fe640 1 --2- 172.21.15.98:0/3977445162 >> v2:172.21.15.139:3300/0 conn(0x7fa4b80cf700 0x7fa4b815ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:18.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.979+0000 7fa4b6ffd640 1 -- 172.21.15.98:0/3977445162 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa4a0003530 con 0x7fa4b806d4c0 2024-09-19T08:46:18.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.979+0000 7fa4b6ffd640 1 -- 172.21.15.98:0/3977445162 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 340026035 0 0) 0x7fa4a0003530 con 0x7fa4b806d4c0 2024-09-19T08:46:18.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.979+0000 7fa4b6ffd640 1 -- 172.21.15.98:0/3977445162 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4b815c290 msgr2=0x7fa4b80cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:18.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.979+0000 7fa4b6ffd640 1 --2- 172.21.15.98:0/3977445162 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4b815c290 0x7fa4b80cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.979+0000 7fa4b6ffd640 1 -- 172.21.15.98:0/3977445162 >> v2:172.21.15.139:3300/0 conn(0x7fa4b80cf700 msgr2=0x7fa4b815ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:18.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.979+0000 7fa4b6ffd640 1 --2- 172.21.15.98:0/3977445162 >> v2:172.21.15.139:3300/0 conn(0x7fa4b80cf700 0x7fa4b815ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.979+0000 7fa4b6ffd640 1 -- 172.21.15.98:0/3977445162 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b8160ba0 con 0x7fa4b806d4c0 2024-09-19T08:46:18.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.979+0000 7fa4b6ffd640 1 -- 172.21.15.98:0/3977445162 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fa4b0001a00 con 0x7fa4b806d4c0 2024-09-19T08:46:18.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.979+0000 7fa4b77fe640 1 --2- 172.21.15.98:0/3977445162 >> v2:172.21.15.139:3300/0 conn(0x7fa4b80cf700 0x7fa4b815ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:18.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.979+0000 7fa4b6ffd640 1 -- 172.21.15.98:0/3977445162 >> v1:172.21.15.98:6789/0 conn(0x7fa4b806d4c0 legacy=0x7fa4b80b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:18.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.979+0000 7fa4b6ffd640 1 --2- 172.21.15.98:0/3977445162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a00039c0 0x7fa4a0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:18.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.979+0000 7fa4b6ffd640 1 --2- 172.21.15.98:0/3977445162 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4b815c290 0x7fa4a0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:18.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.979+0000 7fa4b6ffd640 1 --2- 172.21.15.98:0/3977445162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4a0004d50 0x7fa4a00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:18.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.979+0000 7fa4b6ffd640 1 -- 172.21.15.98:0/3977445162 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b80cd510 con 0x7fa4a0004d50 2024-09-19T08:46:18.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.979+0000 7fa4b6ffd640 1 -- 172.21.15.98:0/3977445162 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a000b9d0 con 0x7fa4a00039c0 2024-09-19T08:46:18.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.979+0000 7fa4b6ffd640 1 -- 172.21.15.98:0/3977445162 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa4a000bb80 con 0x7fa4b815c290 2024-09-19T08:46:18.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.979+0000 7fa4bcc9b640 1 --2- 172.21.15.98:0/3977445162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4a0004d50 0x7fa4a00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:18.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.979+0000 7fa4b7fff640 1 --2- 172.21.15.98:0/3977445162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a00039c0 0x7fa4a0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:18.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.979+0000 7fa4b77fe640 1 --2- 172.21.15.98:0/3977445162 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4b815c290 0x7fa4a0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:18.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.979+0000 7fa4bcc9b640 1 -- 172.21.15.98:0/3977445162 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4b815c290 msgr2=0x7fa4a0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:18.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.979+0000 7fa4bcc9b640 1 --2- 172.21.15.98:0/3977445162 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4b815c290 0x7fa4a0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.979+0000 7fa4bcc9b640 1 -- 172.21.15.98:0/3977445162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a00039c0 msgr2=0x7fa4a0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:18.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.979+0000 7fa4bcc9b640 1 --2- 172.21.15.98:0/3977445162 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a00039c0 0x7fa4a0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.979+0000 7fa4bcc9b640 1 -- 172.21.15.98:0/3977445162 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a0003530 con 0x7fa4a0004d50 2024-09-19T08:46:18.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.983+0000 7fa4bcc9b640 1 --2- 172.21.15.98:0/3977445162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4a0004d50 0x7fa4a00051d0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000a880 tx=0x7fa4b001d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1503c68801af5285 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:18.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.983+0000 7fa4b6ffd640 1 -- 172.21.15.98:0/3977445162 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b000a4a0 con 0x7fa4a0004d50 2024-09-19T08:46:18.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.983+0000 7fa4b6ffd640 1 -- 172.21.15.98:0/3977445162 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4b00ad650 con 0x7fa4a0004d50 2024-09-19T08:46:18.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.983+0000 7fa4b6ffd640 1 -- 172.21.15.98:0/3977445162 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b00ada80 con 0x7fa4a0004d50 2024-09-19T08:46:18.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.983+0000 7fa4be6fe640 1 -- 172.21.15.98:0/3977445162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4a0004d50 msgr2=0x7fa4a00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:18.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.983+0000 7fa4be6fe640 1 --2- 172.21.15.98:0/3977445162 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4a0004d50 0x7fa4a00051d0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000a880 tx=0x7fa4b001d3a0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.983+0000 7fa4bcc9b640 1 -- 172.21.15.98:0/3977445162 reap_dead start 2024-09-19T08:46:18.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.983+0000 7fa4be6fe640 1 -- 172.21.15.98:0/3977445162 shutdown_connections 2024-09-19T08:46:18.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.983+0000 7fa4be6fe640 1 -- 172.21.15.98:0/3977445162 >> 172.21.15.98:0/3977445162 conn(0x7fa4b806c090 msgr2=0x7fa4b806c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:18.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.983+0000 7fa4be6fe640 1 -- 172.21.15.98:0/3977445162 shutdown_connections 2024-09-19T08:46:18.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.983+0000 7fa4be6fe640 1 -- 172.21.15.98:0/3977445162 wait complete. 2024-09-19T08:46:18.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.983+0000 7fa4be6fe640 1 Processor -- start 2024-09-19T08:46:18.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.983+0000 7fa4be6fe640 1 -- start start 2024-09-19T08:46:18.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.987+0000 7fa4be6fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a0004d50 0x7fa4b8155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:18.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.987+0000 7fa4be6fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4b8159960 0x7fa4b8159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:18.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.987+0000 7fa4be6fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4b81f51d0 0x7fa4b81f5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:18.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.987+0000 7fa4be6fe640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b815c7f0 con 0x7fa4b8159960 2024-09-19T08:46:18.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.987+0000 7fa4be6fe640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b815c670 con 0x7fa4a0004d50 2024-09-19T08:46:18.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.987+0000 7fa4be6fe640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b8160490 con 0x7fa4b81f51d0 2024-09-19T08:46:18.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.987+0000 7fa4b77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4b8159960 0x7fa4b8159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:18.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.987+0000 7fa4b77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4b8159960 0x7fa4b8159de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59656/0 (socket says 172.21.15.98:59656) 2024-09-19T08:46:18.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.987+0000 7fa4b77fe640 1 -- 172.21.15.98:0/2364902522 learned_addr learned my addr 172.21.15.98:0/2364902522 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:18.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.987+0000 7fa4b77fe640 1 -- 172.21.15.98:0/2364902522 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4b81f51d0 msgr2=0x7fa4b81f5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:18.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.987+0000 7fa4b77fe640 1 --2- 172.21.15.98:0/2364902522 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4b81f51d0 0x7fa4b81f5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.987+0000 7fa4b77fe640 1 -- 172.21.15.98:0/2364902522 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a0004d50 msgr2=0x7fa4b8155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:18.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.987+0000 7fa4b77fe640 1 --2- 172.21.15.98:0/2364902522 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a0004d50 0x7fa4b8155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:18.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.987+0000 7fa4b77fe640 1 -- 172.21.15.98:0/2364902522 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b80d0300 con 0x7fa4b8159960 2024-09-19T08:46:18.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.987+0000 7fa4b77fe640 1 --2- 172.21.15.98:0/2364902522 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4b8159960 0x7fa4b8159de0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac010860 tx=0x7fa4ac010c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:18.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.987+0000 7fa4b4ff9640 1 -- 172.21.15.98:0/2364902522 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac01a020 con 0x7fa4b8159960 2024-09-19T08:46:18.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.987+0000 7fa4b4ff9640 1 -- 172.21.15.98:0/2364902522 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4ac00cac0 con 0x7fa4b8159960 2024-09-19T08:46:18.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.987+0000 7fa4be6fe640 1 -- 172.21.15.98:0/2364902522 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b81f5f90 con 0x7fa4b8159960 2024-09-19T08:46:18.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.987+0000 7fa4b4ff9640 1 -- 172.21.15.98:0/2364902522 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac00f040 con 0x7fa4b8159960 2024-09-19T08:46:18.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.987+0000 7fa4be6fe640 1 -- 172.21.15.98:0/2364902522 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b80cc340 con 0x7fa4b8159960 2024-09-19T08:46:18.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.987+0000 7fa4be6fe640 1 -- 172.21.15.98:0/2364902522 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4b815ff50 con 0x7fa4b8159960 2024-09-19T08:46:18.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.987+0000 7fa4b4ff9640 1 -- 172.21.15.98:0/2364902522 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 23) ==== 88057+0+0 (secure 0 0 0) 0x7fa4ac01d020 con 0x7fa4b8159960 2024-09-19T08:46:18.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.987+0000 7fa4b4ff9640 1 --2- 172.21.15.98:0/2364902522 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa4a4067f80 0x7fa4a406a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:18.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.987+0000 7fa4b4ff9640 1 -- 172.21.15.98:0/2364902522 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5737+0+0 (secure 0 0 0) 0x7fa4ac087060 con 0x7fa4b8159960 2024-09-19T08:46:18.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.987+0000 7fa4b7fff640 1 --2- 172.21.15.98:0/2364902522 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa4a4067f80 0x7fa4a406a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:18.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.991+0000 7fa4b7fff640 1 --2- 172.21.15.98:0/2364902522 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa4a4067f80 0x7fa4a406a440 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa4a80098b0 tx=0x7fa4a8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:18.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:18.991+0000 7fa4b4ff9640 1 -- 172.21.15.98:0/2364902522 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa4ac024020 con 0x7fa4b8159960 2024-09-19T08:46:19.180 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:19.183+0000 7fa4be6fe640 1 -- 172.21.15.98:0/2364902522 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol64351", "group_name": "subvol_grp53040", "target": ["mon-mgr", ""]}) -- 0x7fa4b80b2470 con 0x7fa4a4067f80 2024-09-19T08:46:19.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:19.191+0000 7fa4b4ff9640 1 -- 172.21.15.98:0/2364902522 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa4b80b2470 con 0x7fa4a4067f80 2024-09-19T08:46:19.191 INFO:teuthology.orchestra.run.smithi098.stdout:/volumes/subvol_grp53040/subvol64351 2024-09-19T08:46:19.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:19.195+0000 7fa4be6fe640 1 -- 172.21.15.98:0/2364902522 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa4a4067f80 msgr2=0x7fa4a406a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:19.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:19.195+0000 7fa4be6fe640 1 --2- 172.21.15.98:0/2364902522 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa4a4067f80 0x7fa4a406a440 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa4a80098b0 tx=0x7fa4a8008040 comp rx=0 tx=0).stop 2024-09-19T08:46:19.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:19.195+0000 7fa4be6fe640 1 -- 172.21.15.98:0/2364902522 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4b8159960 msgr2=0x7fa4b8159de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:19.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:19.195+0000 7fa4be6fe640 1 --2- 172.21.15.98:0/2364902522 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4b8159960 0x7fa4b8159de0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac010860 tx=0x7fa4ac010c60 comp rx=0 tx=0).stop 2024-09-19T08:46:19.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:19.195+0000 7fa4be6fe640 1 -- 172.21.15.98:0/2364902522 shutdown_connections 2024-09-19T08:46:19.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:19.195+0000 7fa4be6fe640 1 --2- 172.21.15.98:0/2364902522 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fa4a4067f80 0x7fa4a406a440 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:19.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:19.195+0000 7fa4be6fe640 1 --2- 172.21.15.98:0/2364902522 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fa4b81f51d0 0x7fa4b81f5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:19.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:19.195+0000 7fa4be6fe640 1 --2- 172.21.15.98:0/2364902522 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fa4b8159960 0x7fa4b8159de0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:19.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:19.195+0000 7fa4be6fe640 1 --2- 172.21.15.98:0/2364902522 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a0004d50 0x7fa4b8155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:19.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:19.195+0000 7fa4be6fe640 1 -- 172.21.15.98:0/2364902522 >> 172.21.15.98:0/2364902522 conn(0x7fa4b806c090 msgr2=0x7fa4b815a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:19.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:19.195+0000 7fa4be6fe640 1 -- 172.21.15.98:0/2364902522 shutdown_connections 2024-09-19T08:46:19.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:19.195+0000 7fa4be6fe640 1 -- 172.21.15.98:0/2364902522 wait complete. 2024-09-19T08:46:19.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:19.210 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/1a702dd34daa2fa6762a91def40db3ba.meta) 2024-09-19T08:46:23.837 INFO:teuthology.orchestra.run.smithi098.stdout:[GLOBAL] 2024-09-19T08:46:23.837 INFO:teuthology.orchestra.run.smithi098.stdout:version = 1 2024-09-19T08:46:23.837 INFO:teuthology.orchestra.run.smithi098.stdout:type = subvolume 2024-09-19T08:46:23.837 INFO:teuthology.orchestra.run.smithi098.stdout:path = /volumes/subvol_grp53040/subvol64351 2024-09-19T08:46:23.837 INFO:teuthology.orchestra.run.smithi098.stdout:state = complete 2024-09-19T08:46:23.837 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:46:23.839 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol54610 2024-09-19T08:46:23.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.975+0000 7f2f5d104640 1 Processor -- start 2024-09-19T08:46:23.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.975+0000 7f2f5d104640 1 -- start start 2024-09-19T08:46:23.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.975+0000 7f2f5d104640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f581415a0 0x7f2f580d20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:23.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.975+0000 7f2f5d104640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2f580d25f0 0x7f2f580d3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:23.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.975+0000 7f2f5d104640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f5815eae0 con 0x7f2f580d38a0 2024-09-19T08:46:23.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.975+0000 7f2f5d104640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f2f580d2a30 con 0x7f2f580d25f0 2024-09-19T08:46:23.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.975+0000 7f2f5d104640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2f5815edc0 con 0x7f2f581415a0 2024-09-19T08:46:23.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.975+0000 7f2f5659c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2f580d25f0 0x7f2f580d3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:23.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.975+0000 7f2f5759e640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f2f580d38a0 0x7f2f58161fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:44804/0 (socket says 172.21.15.98:44804) 2024-09-19T08:46:23.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.975+0000 7f2f5759e640 1 -- 172.21.15.98:0/1310084518 learned_addr learned my addr 172.21.15.98:0/1310084518 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:23.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.975+0000 7f2f56d9d640 1 --2- 172.21.15.98:0/1310084518 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f581415a0 0x7f2f580d20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:23.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.975+0000 7f2f5659c640 1 -- 172.21.15.98:0/1310084518 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f581415a0 msgr2=0x7f2f580d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:23.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.975+0000 7f2f5659c640 1 --2- 172.21.15.98:0/1310084518 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f581415a0 0x7f2f580d20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:23.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.975+0000 7f2f5659c640 1 -- 172.21.15.98:0/1310084518 >> v1:172.21.15.98:6789/0 conn(0x7f2f580d38a0 legacy=0x7f2f58161fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:23.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.975+0000 7f2f5659c640 1 -- 172.21.15.98:0/1310084518 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f5815f0a0 con 0x7f2f580d25f0 2024-09-19T08:46:23.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.975+0000 7f2f55d9b640 1 -- 172.21.15.98:0/1310084518 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 976966481 0 0) 0x7f2f5815eae0 con 0x7f2f580d38a0 2024-09-19T08:46:23.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.979+0000 7f2f5659c640 1 --2- 172.21.15.98:0/1310084518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f580d25f0 0x7f2f580d3360 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c0097c0 tx=0x7f2f4c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ae10b502de2ba1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:23.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.979+0000 7f2f55d9b640 1 -- 172.21.15.98:0/1310084518 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c0b0070 con 0x7f2f580d25f0 2024-09-19T08:46:23.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.979+0000 7f2f55d9b640 1 -- 172.21.15.98:0/1310084518 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f4c0ab600 con 0x7f2f580d25f0 2024-09-19T08:46:23.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.979+0000 7f2f55d9b640 1 -- 172.21.15.98:0/1310084518 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c0ab9d0 con 0x7f2f580d25f0 2024-09-19T08:46:23.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.979+0000 7f2f5d104640 1 -- 172.21.15.98:0/1310084518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f580d25f0 msgr2=0x7f2f580d3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:23.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.979+0000 7f2f5d104640 1 --2- 172.21.15.98:0/1310084518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f580d25f0 0x7f2f580d3360 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c0097c0 tx=0x7f2f4c01c470 comp rx=0 tx=0).stop 2024-09-19T08:46:23.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.979+0000 7f2f5d104640 1 -- 172.21.15.98:0/1310084518 shutdown_connections 2024-09-19T08:46:23.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.979+0000 7f2f5d104640 1 --2- 172.21.15.98:0/1310084518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f580d25f0 0x7f2f580d3360 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:23.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.979+0000 7f2f5d104640 1 --2- 172.21.15.98:0/1310084518 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f581415a0 0x7f2f580d20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:23.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.979+0000 7f2f5d104640 1 -- 172.21.15.98:0/1310084518 >> 172.21.15.98:0/1310084518 conn(0x7f2f58074c70 msgr2=0x7f2f58075080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:23.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.979+0000 7f2f5d104640 1 -- 172.21.15.98:0/1310084518 shutdown_connections 2024-09-19T08:46:23.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.979+0000 7f2f5d104640 1 -- 172.21.15.98:0/1310084518 wait complete. 2024-09-19T08:46:23.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.979+0000 7f2f5d104640 1 Processor -- start 2024-09-19T08:46:23.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.979+0000 7f2f5d104640 1 -- start start 2024-09-19T08:46:23.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.979+0000 7f2f5d104640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f580d25f0 0x7f2f5813cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:23.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.979+0000 7f2f5d104640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f580d38a0 0x7f2f5813d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:23.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.979+0000 7f2f5d104640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f581415a0 0x7f2f5813e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:23.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.979+0000 7f2f5d104640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2f580d42a0 con 0x7f2f580d38a0 2024-09-19T08:46:23.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.979+0000 7f2f5d104640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2f580d4120 con 0x7f2f581415a0 2024-09-19T08:46:23.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.979+0000 7f2f5d104640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2f580d3d10 con 0x7f2f580d25f0 2024-09-19T08:46:23.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.979+0000 7f2f5659c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f580d38a0 0x7f2f5813d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:23.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.979+0000 7f2f5659c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f580d38a0 0x7f2f5813d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59662/0 (socket says 172.21.15.98:59662) 2024-09-19T08:46:23.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.979+0000 7f2f5659c640 1 -- 172.21.15.98:0/2370050810 learned_addr learned my addr 172.21.15.98:0/2370050810 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:23.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.979+0000 7f2f56d9d640 1 --2- 172.21.15.98:0/2370050810 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f580d25f0 0x7f2f5813cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:23.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.979+0000 7f2f5759e640 1 --2- 172.21.15.98:0/2370050810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f581415a0 0x7f2f5813e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:23.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.979+0000 7f2f5659c640 1 -- 172.21.15.98:0/2370050810 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f580d25f0 msgr2=0x7f2f5813cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:23.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.979+0000 7f2f5659c640 1 --2- 172.21.15.98:0/2370050810 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f580d25f0 0x7f2f5813cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:23.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.979+0000 7f2f5659c640 1 -- 172.21.15.98:0/2370050810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f581415a0 msgr2=0x7f2f5813e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:23.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.979+0000 7f2f5659c640 1 --2- 172.21.15.98:0/2370050810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f581415a0 0x7f2f5813e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:23.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.979+0000 7f2f5659c640 1 -- 172.21.15.98:0/2370050810 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f5813da00 con 0x7f2f580d38a0 2024-09-19T08:46:23.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.979+0000 7f2f56d9d640 1 --2- 172.21.15.98:0/2370050810 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f580d25f0 0x7f2f5813cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:23.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.979+0000 7f2f5659c640 1 --2- 172.21.15.98:0/2370050810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f580d38a0 0x7f2f5813d180 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c0ae040 tx=0x7f2f4c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:23.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.983+0000 7f2f3f7fe640 1 -- 172.21.15.98:0/2370050810 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c0b0050 con 0x7f2f580d38a0 2024-09-19T08:46:23.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.983+0000 7f2f3f7fe640 1 -- 172.21.15.98:0/2370050810 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f4c0b1040 con 0x7f2f580d38a0 2024-09-19T08:46:23.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.983+0000 7f2f5d104640 1 -- 172.21.15.98:0/2370050810 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f5813dce0 con 0x7f2f580d38a0 2024-09-19T08:46:23.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.983+0000 7f2f3f7fe640 1 -- 172.21.15.98:0/2370050810 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c004220 con 0x7f2f580d38a0 2024-09-19T08:46:23.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.983+0000 7f2f5d104640 1 -- 172.21.15.98:0/2370050810 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f581426d0 con 0x7f2f580d38a0 2024-09-19T08:46:23.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.983+0000 7f2f5d104640 1 -- 172.21.15.98:0/2370050810 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f580d3360 con 0x7f2f580d38a0 2024-09-19T08:46:23.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.983+0000 7f2f3f7fe640 1 -- 172.21.15.98:0/2370050810 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 23) ==== 88057+0+0 (secure 0 0 0) 0x7f2f4c0be060 con 0x7f2f580d38a0 2024-09-19T08:46:23.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.983+0000 7f2f3f7fe640 1 --2- 172.21.15.98:0/2370050810 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2f44067f80 0x7f2f4406a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:23.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.983+0000 7f2f3f7fe640 1 -- 172.21.15.98:0/2370050810 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5737+0+0 (secure 0 0 0) 0x7f2f4c120480 con 0x7f2f580d38a0 2024-09-19T08:46:23.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.983+0000 7f2f56d9d640 1 --2- 172.21.15.98:0/2370050810 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2f44067f80 0x7f2f4406a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:23.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.983+0000 7f2f56d9d640 1 --2- 172.21.15.98:0/2370050810 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2f44067f80 0x7f2f4406a440 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2f480097a0 tx=0x7f2f48008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:23.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:23.987+0000 7f2f3f7fe640 1 -- 172.21.15.98:0/2370050810 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2f4c016090 con 0x7f2f580d38a0 2024-09-19T08:46:24.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.175+0000 7f2f5d104640 1 -- 172.21.15.98:0/2370050810 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol54610", "target": ["mon-mgr", ""]}) -- 0x7f2f580b8880 con 0x7f2f44067f80 2024-09-19T08:46:24.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.211+0000 7f2f3f7fe640 1 -- 172.21.15.98:0/2370050810 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2f580b8880 con 0x7f2f44067f80 2024-09-19T08:46:24.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.215+0000 7f2f5d104640 1 -- 172.21.15.98:0/2370050810 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2f44067f80 msgr2=0x7f2f4406a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:24.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.215+0000 7f2f5d104640 1 --2- 172.21.15.98:0/2370050810 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2f44067f80 0x7f2f4406a440 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2f480097a0 tx=0x7f2f48008040 comp rx=0 tx=0).stop 2024-09-19T08:46:24.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.215+0000 7f2f5d104640 1 -- 172.21.15.98:0/2370050810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f580d38a0 msgr2=0x7f2f5813d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:24.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.215+0000 7f2f5d104640 1 --2- 172.21.15.98:0/2370050810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f580d38a0 0x7f2f5813d180 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c0ae040 tx=0x7f2f4c01acb0 comp rx=0 tx=0).stop 2024-09-19T08:46:24.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.215+0000 7f2f5d104640 1 -- 172.21.15.98:0/2370050810 shutdown_connections 2024-09-19T08:46:24.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.215+0000 7f2f5d104640 1 --2- 172.21.15.98:0/2370050810 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2f44067f80 0x7f2f4406a440 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:24.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.215+0000 7f2f5d104640 1 --2- 172.21.15.98:0/2370050810 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2f581415a0 0x7f2f5813e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:24.216 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.215+0000 7f2f5d104640 1 --2- 172.21.15.98:0/2370050810 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2f580d38a0 0x7f2f5813d180 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:24.216 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.215+0000 7f2f5d104640 1 --2- 172.21.15.98:0/2370050810 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2f580d25f0 0x7f2f5813cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:24.216 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.215+0000 7f2f5d104640 1 -- 172.21.15.98:0/2370050810 >> 172.21.15.98:0/2370050810 conn(0x7f2f58074c70 msgr2=0x7f2f5815fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:24.216 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.215+0000 7f2f5d104640 1 -- 172.21.15.98:0/2370050810 shutdown_connections 2024-09-19T08:46:24.216 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.215+0000 7f2f5d104640 1 -- 172.21.15.98:0/2370050810 wait complete. 2024-09-19T08:46:24.223 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:46:24.226+0000 7ff73c6a0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:46:24.223 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:46:24.226+0000 7ff73c6a0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:46:24.223 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:46:24.226+0000 7ff73c6a0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:46:24.223 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:46:24.226+0000 7ff73c6a0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:46:24.223 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:46:24.226+0000 7ff73c6a0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:46:24.231 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol64351 subvol_grp53040 2024-09-19T08:46:24.240 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:46:24.242+0000 7ff73cea1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:46:24.240 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:46:24.242+0000 7ff73cea1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:46:24.240 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:46:24.242+0000 7ff73cea1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:46:24.240 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:46:24.242+0000 7ff73cea1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:46:24.240 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:46:24.242+0000 7ff73cea1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:46:24.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.347+0000 7f14ad389640 1 Processor -- start 2024-09-19T08:46:24.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.347+0000 7f14ad389640 1 -- start start 2024-09-19T08:46:24.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.347+0000 7f14ad389640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f14a80cf430 0x7f14a8166050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:24.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.347+0000 7f14ad389640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14a815baf0 con 0x7f14a815f6e0 2024-09-19T08:46:24.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.347+0000 7f14ad389640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14a815bdd0 con 0x7f14a806dbd0 2024-09-19T08:46:24.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.347+0000 7f14ad389640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f14a815c8b0 con 0x7f14a80cf430 2024-09-19T08:46:24.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.347+0000 7f14a6ffd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f14a815f6e0 0x7f14a806d4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:44820/0 (socket says 172.21.15.98:44820) 2024-09-19T08:46:24.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.347+0000 7f14a6ffd640 1 -- 172.21.15.98:0/1271666142 learned_addr learned my addr 172.21.15.98:0/1271666142 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:24.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.347+0000 7f14a5ffb640 1 -- 172.21.15.98:0/1271666142 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1483280826 0 0) 0x7f14a815baf0 con 0x7f14a815f6e0 2024-09-19T08:46:24.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.347+0000 7f14a5ffb640 1 -- 172.21.15.98:0/1271666142 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1490005740 con 0x7f14a815f6e0 2024-09-19T08:46:24.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.347+0000 7f14a77fe640 1 --2- 172.21.15.98:0/1271666142 >> v2:172.21.15.139:3300/0 conn(0x7f14a80cf430 0x7f14a8166050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:24.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.347+0000 7f14a5ffb640 1 -- 172.21.15.98:0/1271666142 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1715707848 0 0) 0x7f1490005740 con 0x7f14a815f6e0 2024-09-19T08:46:24.348 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.347+0000 7f14a5ffb640 1 -- 172.21.15.98:0/1271666142 >> v2:172.21.15.139:3300/0 conn(0x7f14a80cf430 msgr2=0x7f14a8166050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:24.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.347+0000 7f14a5ffb640 1 --2- 172.21.15.98:0/1271666142 >> v2:172.21.15.139:3300/0 conn(0x7f14a80cf430 0x7f14a8166050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:24.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.347+0000 7f14a5ffb640 1 -- 172.21.15.98:0/1271666142 >> v1:172.21.15.139:6789/0 conn(0x7f14a806dbd0 legacy=0x7f14a8073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:24.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.347+0000 7f14a5ffb640 1 -- 172.21.15.98:0/1271666142 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a815c0b0 con 0x7f14a815f6e0 2024-09-19T08:46:24.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.347+0000 7f14a5ffb640 1 -- 172.21.15.98:0/1271666142 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f1498001680 con 0x7f14a815f6e0 2024-09-19T08:46:24.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.347+0000 7f14a5ffb640 1 -- 172.21.15.98:0/1271666142 >> v1:172.21.15.98:6789/0 conn(0x7f14a815f6e0 legacy=0x7f14a806d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:24.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.347+0000 7f14a5ffb640 1 --2- 172.21.15.98:0/1271666142 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1490005bd0 0x7f1490005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:24.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.347+0000 7f14a5ffb640 1 --2- 172.21.15.98:0/1271666142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1490006a20 0x7f1490006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:24.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.347+0000 7f14a5ffb640 1 --2- 172.21.15.98:0/1271666142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14900073e0 0x7f149000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:24.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.347+0000 7f14a5ffb640 1 -- 172.21.15.98:0/1271666142 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f14a815c8b0 con 0x7f14900073e0 2024-09-19T08:46:24.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.347+0000 7f14a5ffb640 1 -- 172.21.15.98:0/1271666142 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f149000e330 con 0x7f1490006a20 2024-09-19T08:46:24.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.347+0000 7f14a5ffb640 1 -- 172.21.15.98:0/1271666142 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f149000e530 con 0x7f1490005bd0 2024-09-19T08:46:24.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.347+0000 7f14a77fe640 1 --2- 172.21.15.98:0/1271666142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14900073e0 0x7f149000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:24.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.347+0000 7f14a67fc640 1 --2- 172.21.15.98:0/1271666142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1490006a20 0x7f1490006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:24.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.347+0000 7f14a6ffd640 1 --2- 172.21.15.98:0/1271666142 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1490005bd0 0x7f1490005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:24.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.347+0000 7f14a77fe640 1 -- 172.21.15.98:0/1271666142 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1490005bd0 msgr2=0x7f1490005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:24.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.347+0000 7f14a77fe640 1 --2- 172.21.15.98:0/1271666142 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1490005bd0 0x7f1490005ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:24.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.347+0000 7f14a77fe640 1 -- 172.21.15.98:0/1271666142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1490006a20 msgr2=0x7f1490006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:24.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.347+0000 7f14a77fe640 1 --2- 172.21.15.98:0/1271666142 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1490006a20 0x7f1490006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:24.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.347+0000 7f14a77fe640 1 -- 172.21.15.98:0/1271666142 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1490005740 con 0x7f14900073e0 2024-09-19T08:46:24.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.351+0000 7f14a77fe640 1 -- 172.21.15.98:0/1271666142 reap_dead start 2024-09-19T08:46:24.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.351+0000 7f14a77fe640 1 --2- 172.21.15.98:0/1271666142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14900073e0 0x7f149000dbe0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f14a0007360 tx=0x7f14a001ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f488d300b38b134 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:24.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.351+0000 7f14a5ffb640 1 -- 172.21.15.98:0/1271666142 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a0009be0 con 0x7f14900073e0 2024-09-19T08:46:24.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.351+0000 7f14a5ffb640 1 -- 172.21.15.98:0/1271666142 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14a00a5dc0 con 0x7f14900073e0 2024-09-19T08:46:24.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.351+0000 7f14a5ffb640 1 -- 172.21.15.98:0/1271666142 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a0003be0 con 0x7f14900073e0 2024-09-19T08:46:24.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.351+0000 7f14ad389640 1 -- 172.21.15.98:0/1271666142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14900073e0 msgr2=0x7f149000dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:24.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.351+0000 7f14ad389640 1 --2- 172.21.15.98:0/1271666142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14900073e0 0x7f149000dbe0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f14a0007360 tx=0x7f14a001ce80 comp rx=0 tx=0).stop 2024-09-19T08:46:24.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.351+0000 7f14ad389640 1 -- 172.21.15.98:0/1271666142 shutdown_connections 2024-09-19T08:46:24.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.351+0000 7f14ad389640 1 --2- 172.21.15.98:0/1271666142 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14900073e0 0x7f149000dbe0 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:24.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.351+0000 7f14ad389640 1 -- 172.21.15.98:0/1271666142 >> 172.21.15.98:0/1271666142 conn(0x7f14a806c090 msgr2=0x7f14a806c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:24.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.351+0000 7f14ad389640 1 -- 172.21.15.98:0/1271666142 shutdown_connections 2024-09-19T08:46:24.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.351+0000 7f14ad389640 1 -- 172.21.15.98:0/1271666142 wait complete. 2024-09-19T08:46:24.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.351+0000 7f14ad389640 1 Processor -- start 2024-09-19T08:46:24.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.351+0000 7f14ad389640 1 -- start start 2024-09-19T08:46:24.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.351+0000 7f14ad389640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14900073e0 0x7f14a8147b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:24.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.351+0000 7f14ad389640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14a80ae8b0 0x7f14a80aed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:24.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.351+0000 7f14ad389640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14a8141bf0 0x7f14a81420a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:24.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.351+0000 7f14ad389640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f14a815fac0 con 0x7f14a8141bf0 2024-09-19T08:46:24.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.351+0000 7f14ad389640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f14a815b890 con 0x7f14900073e0 2024-09-19T08:46:24.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.351+0000 7f14ad389640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f14a815fc40 con 0x7f14a80ae8b0 2024-09-19T08:46:24.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.351+0000 7f14a77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14a8141bf0 0x7f14a81420a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:24.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.351+0000 7f14a77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14a8141bf0 0x7f14a81420a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59672/0 (socket says 172.21.15.98:59672) 2024-09-19T08:46:24.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.351+0000 7f14a77fe640 1 -- 172.21.15.98:0/4134314000 learned_addr learned my addr 172.21.15.98:0/4134314000 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:24.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.351+0000 7f14a6ffd640 1 --2- 172.21.15.98:0/4134314000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14900073e0 0x7f14a8147b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:24.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.351+0000 7f14a67fc640 1 --2- 172.21.15.98:0/4134314000 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14a80ae8b0 0x7f14a80aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:24.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.355+0000 7f14a77fe640 1 -- 172.21.15.98:0/4134314000 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14a80ae8b0 msgr2=0x7f14a80aed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:24.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.355+0000 7f14a77fe640 1 --2- 172.21.15.98:0/4134314000 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14a80ae8b0 0x7f14a80aed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:24.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.355+0000 7f14a77fe640 1 -- 172.21.15.98:0/4134314000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14900073e0 msgr2=0x7f14a8147b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:24.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.355+0000 7f14a77fe640 1 --2- 172.21.15.98:0/4134314000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14900073e0 0x7f14a8147b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:24.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.355+0000 7f14a77fe640 1 -- 172.21.15.98:0/4134314000 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a8142820 con 0x7f14a8141bf0 2024-09-19T08:46:24.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.355+0000 7f14a67fc640 1 --2- 172.21.15.98:0/4134314000 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14a80ae8b0 0x7f14a80aed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:24.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.355+0000 7f14a77fe640 1 --2- 172.21.15.98:0/4134314000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14a8141bf0 0x7f14a81420a0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f14a001d720 tx=0x7f14a0019c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:24.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.355+0000 7f148f7fe640 1 -- 172.21.15.98:0/4134314000 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a0009be0 con 0x7f14a8141bf0 2024-09-19T08:46:24.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.355+0000 7f148f7fe640 1 -- 172.21.15.98:0/4134314000 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14a0003e80 con 0x7f14a8141bf0 2024-09-19T08:46:24.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.355+0000 7f14ad389640 1 -- 172.21.15.98:0/4134314000 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14a8142b00 con 0x7f14a8141bf0 2024-09-19T08:46:24.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.355+0000 7f148f7fe640 1 -- 172.21.15.98:0/4134314000 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a00ad430 con 0x7f14a8141bf0 2024-09-19T08:46:24.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.355+0000 7f14ad389640 1 -- 172.21.15.98:0/4134314000 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14a81430c0 con 0x7f14a8141bf0 2024-09-19T08:46:24.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.355+0000 7f14ad389640 1 -- 172.21.15.98:0/4134314000 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14a80af270 con 0x7f14a8141bf0 2024-09-19T08:46:24.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.355+0000 7f148f7fe640 1 -- 172.21.15.98:0/4134314000 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 23) ==== 88057+0+0 (secure 0 0 0) 0x7f14a00b4070 con 0x7f14a8141bf0 2024-09-19T08:46:24.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.355+0000 7f148f7fe640 1 --2- 172.21.15.98:0/4134314000 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1494067fb0 0x7f149406a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:24.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.355+0000 7f148f7fe640 1 -- 172.21.15.98:0/4134314000 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5737+0+0 (secure 0 0 0) 0x7f14a0120570 con 0x7f14a8141bf0 2024-09-19T08:46:24.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.355+0000 7f14a6ffd640 1 --2- 172.21.15.98:0/4134314000 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1494067fb0 0x7f149406a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:24.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.355+0000 7f14a6ffd640 1 --2- 172.21.15.98:0/4134314000 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1494067fb0 0x7f149406a470 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1498001ec0 tx=0x7f1498009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:24.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.359+0000 7f148f7fe640 1 -- 172.21.15.98:0/4134314000 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f14a00bc060 con 0x7f14a8141bf0 2024-09-19T08:46:24.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.531+0000 7f14ad389640 1 -- 172.21.15.98:0/4134314000 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol64351", "group_name": "subvol_grp53040", "target": ["mon-mgr", ""]}) -- 0x7f14a806d4c0 con 0x7f1494067fb0 2024-09-19T08:46:24.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.567+0000 7f148f7fe640 1 -- 172.21.15.98:0/4134314000 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f14a806d4c0 con 0x7f1494067fb0 2024-09-19T08:46:24.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.567+0000 7f14ad389640 1 -- 172.21.15.98:0/4134314000 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1494067fb0 msgr2=0x7f149406a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:24.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.567+0000 7f14ad389640 1 --2- 172.21.15.98:0/4134314000 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1494067fb0 0x7f149406a470 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1498001ec0 tx=0x7f1498009040 comp rx=0 tx=0).stop 2024-09-19T08:46:24.567 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.567+0000 7f14ad389640 1 -- 172.21.15.98:0/4134314000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14a8141bf0 msgr2=0x7f14a81420a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:24.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.567+0000 7f14ad389640 1 --2- 172.21.15.98:0/4134314000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14a8141bf0 0x7f14a81420a0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f14a001d720 tx=0x7f14a0019c20 comp rx=0 tx=0).stop 2024-09-19T08:46:24.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.567+0000 7f14ad389640 1 -- 172.21.15.98:0/4134314000 shutdown_connections 2024-09-19T08:46:24.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.567+0000 7f14ad389640 1 --2- 172.21.15.98:0/4134314000 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1494067fb0 0x7f149406a470 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:24.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.567+0000 7f14ad389640 1 --2- 172.21.15.98:0/4134314000 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f14a8141bf0 0x7f14a81420a0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:24.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.567+0000 7f14ad389640 1 --2- 172.21.15.98:0/4134314000 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f14a80ae8b0 0x7f14a80aed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:24.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.567+0000 7f14ad389640 1 --2- 172.21.15.98:0/4134314000 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f14900073e0 0x7f14a8147b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:24.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.567+0000 7f14ad389640 1 -- 172.21.15.98:0/4134314000 >> 172.21.15.98:0/4134314000 conn(0x7f14a806c090 msgr2=0x7f14a80715c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:24.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.567+0000 7f14ad389640 1 -- 172.21.15.98:0/4134314000 shutdown_connections 2024-09-19T08:46:24.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.567+0000 7f14ad389640 1 -- 172.21.15.98:0/4134314000 wait complete. 2024-09-19T08:46:24.584 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-19T08:46:24.628 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T08:46:24.632 INFO:teuthology.orchestra.run.smithi098.stdout:2 2024-09-19T08:46:24.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:46:24.638 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp53040 2024-09-19T08:46:24.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.755+0000 7f7392889640 1 Processor -- start 2024-09-19T08:46:24.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.755+0000 7f7392889640 1 -- start start 2024-09-19T08:46:24.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.755+0000 7f7392889640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f738c144d10 0x7f738c0722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:24.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.755+0000 7f7392889640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f738c06c3a0 0x7f738c0cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:24.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.755+0000 7f7392889640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f738c072ea0 con 0x7f738c072820 2024-09-19T08:46:24.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.755+0000 7f7392889640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f738c072c60 con 0x7f738c06c3a0 2024-09-19T08:46:24.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.755+0000 7f7392889640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f738c073ab0 con 0x7f738c144d10 2024-09-19T08:46:24.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.755+0000 7f7390e26640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f738c06c3a0 0x7f738c0cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:24.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.755+0000 7f7390e26640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f738c06c3a0 0x7f738c0cd250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59684/0 (socket says 172.21.15.98:59684) 2024-09-19T08:46:24.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.755+0000 7f7390e26640 1 -- 172.21.15.98:0/697174982 learned_addr learned my addr 172.21.15.98:0/697174982 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:24.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.755+0000 7f7390e26640 1 -- 172.21.15.98:0/697174982 >> v2:172.21.15.139:3300/0 conn(0x7f738c144d10 msgr2=0x7f738c0722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:24.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.755+0000 7f7390e26640 1 --2- 172.21.15.98:0/697174982 >> v2:172.21.15.139:3300/0 conn(0x7f738c144d10 0x7f738c0722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:24.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.755+0000 7f7390e26640 1 -- 172.21.15.98:0/697174982 >> v1:172.21.15.139:6789/0 conn(0x7f738c072820 legacy=0x7f738c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:24.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.755+0000 7f7390e26640 1 -- 172.21.15.98:0/697174982 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f738c073180 con 0x7f738c06c3a0 2024-09-19T08:46:24.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.759+0000 7f7390e26640 1 --2- 172.21.15.98:0/697174982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f738c06c3a0 0x7f738c0cd250 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f738c0684a0 tx=0x7f738401ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=554de4cf849f061 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:24.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.759+0000 7f738affd640 1 -- 172.21.15.98:0/697174982 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f738400b2d0 con 0x7f738c06c3a0 2024-09-19T08:46:24.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.759+0000 7f738affd640 1 -- 172.21.15.98:0/697174982 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f738400b470 con 0x7f738c06c3a0 2024-09-19T08:46:24.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.759+0000 7f738affd640 1 -- 172.21.15.98:0/697174982 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73840a7b90 con 0x7f738c06c3a0 2024-09-19T08:46:24.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.759+0000 7f7392889640 1 -- 172.21.15.98:0/697174982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f738c06c3a0 msgr2=0x7f738c0cd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:24.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.759+0000 7f7392889640 1 --2- 172.21.15.98:0/697174982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f738c06c3a0 0x7f738c0cd250 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f738c0684a0 tx=0x7f738401ebb0 comp rx=0 tx=0).stop 2024-09-19T08:46:24.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.759+0000 7f7392889640 1 -- 172.21.15.98:0/697174982 shutdown_connections 2024-09-19T08:46:24.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.759+0000 7f7392889640 1 --2- 172.21.15.98:0/697174982 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f738c06c3a0 0x7f738c0cd250 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:24.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.759+0000 7f7392889640 1 --2- 172.21.15.98:0/697174982 >> v2:172.21.15.139:3300/0 conn(0x7f738c144d10 0x7f738c0722e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:24.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.759+0000 7f7392889640 1 -- 172.21.15.98:0/697174982 >> 172.21.15.98:0/697174982 conn(0x7f738c0b2010 msgr2=0x7f738c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:24.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.759+0000 7f7392889640 1 -- 172.21.15.98:0/697174982 shutdown_connections 2024-09-19T08:46:24.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.759+0000 7f7392889640 1 -- 172.21.15.98:0/697174982 wait complete. 2024-09-19T08:46:24.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.759+0000 7f7392889640 1 Processor -- start 2024-09-19T08:46:24.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.759+0000 7f7392889640 1 -- start start 2024-09-19T08:46:24.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.763+0000 7f7392889640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f738c06c3a0 0x7f738c15b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:24.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.763+0000 7f7392889640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f738c072820 0x7f738c13a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:24.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.763+0000 7f7392889640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f738c144d10 0x7f738c0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:24.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.763+0000 7f7392889640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f738c0ce120 con 0x7f738c144d10 2024-09-19T08:46:24.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.763+0000 7f7392889640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f738c0cdfa0 con 0x7f738c072820 2024-09-19T08:46:24.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.763+0000 7f7392889640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f738c0ce2a0 con 0x7f738c06c3a0 2024-09-19T08:46:24.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.763+0000 7f7390e26640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f738c144d10 0x7f738c0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:24.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.763+0000 7f7390e26640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f738c144d10 0x7f738c0ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59696/0 (socket says 172.21.15.98:59696) 2024-09-19T08:46:24.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.763+0000 7f7390e26640 1 -- 172.21.15.98:0/2935168976 learned_addr learned my addr 172.21.15.98:0/2935168976 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:24.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.763+0000 7f738b7fe640 1 --2- 172.21.15.98:0/2935168976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f738c072820 0x7f738c13a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:24.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.763+0000 7f7390e26640 1 -- 172.21.15.98:0/2935168976 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f738c06c3a0 msgr2=0x7f738c15b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:24.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.763+0000 7f7390e26640 1 --2- 172.21.15.98:0/2935168976 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f738c06c3a0 0x7f738c15b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:24.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.763+0000 7f7390e26640 1 -- 172.21.15.98:0/2935168976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f738c072820 msgr2=0x7f738c13a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:24.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.763+0000 7f7390e26640 1 --2- 172.21.15.98:0/2935168976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f738c072820 0x7f738c13a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:24.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.763+0000 7f7390e26640 1 -- 172.21.15.98:0/2935168976 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f738c1436c0 con 0x7f738c144d10 2024-09-19T08:46:24.763 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.763+0000 7f738b7fe640 1 --2- 172.21.15.98:0/2935168976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f738c072820 0x7f738c13a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:24.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.763+0000 7f7390e26640 1 --2- 172.21.15.98:0/2935168976 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f738c144d10 0x7f738c0ae8b0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f73840b1040 tx=0x7f738401b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:24.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.763+0000 7f7388ff9640 1 -- 172.21.15.98:0/2935168976 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7384009800 con 0x7f738c144d10 2024-09-19T08:46:24.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.763+0000 7f7388ff9640 1 -- 172.21.15.98:0/2935168976 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73840be070 con 0x7f738c144d10 2024-09-19T08:46:24.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.763+0000 7f7392889640 1 -- 172.21.15.98:0/2935168976 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f738c1439a0 con 0x7f738c144d10 2024-09-19T08:46:24.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.763+0000 7f7388ff9640 1 -- 172.21.15.98:0/2935168976 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7384009be0 con 0x7f738c144d10 2024-09-19T08:46:24.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.763+0000 7f7392889640 1 -- 172.21.15.98:0/2935168976 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f738c143f60 con 0x7f738c144d10 2024-09-19T08:46:24.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.763+0000 7f7392889640 1 -- 172.21.15.98:0/2935168976 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f738c0cd250 con 0x7f738c144d10 2024-09-19T08:46:24.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.763+0000 7f7388ff9640 1 -- 172.21.15.98:0/2935168976 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 23) ==== 88057+0+0 (secure 0 0 0) 0x7f73840b6070 con 0x7f738c144d10 2024-09-19T08:46:24.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.763+0000 7f7388ff9640 1 --2- 172.21.15.98:0/2935168976 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7378067f80 0x7f737806a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:24.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.763+0000 7f7388ff9640 1 -- 172.21.15.98:0/2935168976 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5737+0+0 (secure 0 0 0) 0x7f7384121a40 con 0x7f738c144d10 2024-09-19T08:46:24.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.763+0000 7f738bfff640 1 --2- 172.21.15.98:0/2935168976 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7378067f80 0x7f737806a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:24.766 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.767+0000 7f738bfff640 1 --2- 172.21.15.98:0/2935168976 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7378067f80 0x7f737806a440 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f738c15aab0 tx=0x7f737c006c00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:24.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.767+0000 7f7388ff9640 1 -- 172.21.15.98:0/2935168976 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f73840187a0 con 0x7f738c144d10 2024-09-19T08:46:24.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.947+0000 7f7392889640 1 -- 172.21.15.98:0/2935168976 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp53040", "target": ["mon-mgr", ""]}) -- 0x7f738c13d8a0 con 0x7f7378067f80 2024-09-19T08:46:24.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.955+0000 7f7388ff9640 1 -- 172.21.15.98:0/2935168976 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f738c13d8a0 con 0x7f7378067f80 2024-09-19T08:46:24.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.959+0000 7f7392889640 1 -- 172.21.15.98:0/2935168976 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7378067f80 msgr2=0x7f737806a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:24.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.959+0000 7f7392889640 1 --2- 172.21.15.98:0/2935168976 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7378067f80 0x7f737806a440 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f738c15aab0 tx=0x7f737c006c00 comp rx=0 tx=0).stop 2024-09-19T08:46:24.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.959+0000 7f7392889640 1 -- 172.21.15.98:0/2935168976 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f738c144d10 msgr2=0x7f738c0ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:24.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.959+0000 7f7392889640 1 --2- 172.21.15.98:0/2935168976 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f738c144d10 0x7f738c0ae8b0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f73840b1040 tx=0x7f738401b950 comp rx=0 tx=0).stop 2024-09-19T08:46:24.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.959+0000 7f7392889640 1 -- 172.21.15.98:0/2935168976 shutdown_connections 2024-09-19T08:46:24.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.959+0000 7f7392889640 1 --2- 172.21.15.98:0/2935168976 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7378067f80 0x7f737806a440 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:24.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.959+0000 7f7392889640 1 --2- 172.21.15.98:0/2935168976 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f738c144d10 0x7f738c0ae8b0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:24.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.959+0000 7f7392889640 1 --2- 172.21.15.98:0/2935168976 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f738c072820 0x7f738c13a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:24.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.959+0000 7f7392889640 1 --2- 172.21.15.98:0/2935168976 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f738c06c3a0 0x7f738c15b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:24.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.959+0000 7f7392889640 1 -- 172.21.15.98:0/2935168976 >> 172.21.15.98:0/2935168976 conn(0x7f738c0b2010 msgr2=0x7f738c0b4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:24.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.959+0000 7f7392889640 1 -- 172.21.15.98:0/2935168976 shutdown_connections 2024-09-19T08:46:24.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:24.959+0000 7f7392889640 1 -- 172.21.15.98:0/2935168976 wait complete. 2024-09-19T08:46:24.971 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T08:46:24.972 DEBUG:teuthology.orchestra.run.smithi098:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:46:25.045 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:46:25.128 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T08:46:25.128 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:46:25.128 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:46:25.129 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:46:25.135 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T08:46:25.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.303+0000 7fb1b66f9640 1 Processor -- start 2024-09-19T08:46:25.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.303+0000 7fb1b66f9640 1 -- start start 2024-09-19T08:46:25.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.303+0000 7fb1b66f9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1b0158fd0 0x7fb1b00b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:25.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.303+0000 7fb1b66f9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb1b00b4360 0x7fb1b006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:25.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.303+0000 7fb1b66f9640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb1b006c1d0 0x7fb1b015efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:25.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.303+0000 7fb1b66f9640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fb1b006c620 con 0x7fb1b006c1d0 2024-09-19T08:46:25.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.303+0000 7fb1b66f9640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fb1b01593b0 con 0x7fb1b00b4360 2024-09-19T08:46:25.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.303+0000 7fb1b66f9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb1b00ccc60 con 0x7fb1b0158fd0 2024-09-19T08:46:25.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.303+0000 7fb1b4c96640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb1b006c1d0 0x7fb1b015efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:25.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.303+0000 7fb1b4c96640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb1b006c1d0 0x7fb1b015efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59700/0 (socket says 172.21.15.98:59700) 2024-09-19T08:46:25.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.303+0000 7fb1b4c96640 1 -- 172.21.15.98:0/776193914 learned_addr learned my addr 172.21.15.98:0/776193914 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:25.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.303+0000 7fb1af7fe640 1 --2- 172.21.15.98:0/776193914 >> v2:172.21.15.139:3300/0 conn(0x7fb1b00b4360 0x7fb1b006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:25.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.303+0000 7fb1affff640 1 --2- 172.21.15.98:0/776193914 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1b0158fd0 0x7fb1b00b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:25.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.303+0000 7fb1b4c96640 1 -- 172.21.15.98:0/776193914 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1b0158fd0 msgr2=0x7fb1b00b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:25.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.303+0000 7fb1b4c96640 1 --2- 172.21.15.98:0/776193914 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1b0158fd0 0x7fb1b00b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:25.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.303+0000 7fb1b4c96640 1 -- 172.21.15.98:0/776193914 >> v2:172.21.15.139:3300/0 conn(0x7fb1b00b4360 msgr2=0x7fb1b006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:25.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.303+0000 7fb1b4c96640 1 --2- 172.21.15.98:0/776193914 >> v2:172.21.15.139:3300/0 conn(0x7fb1b00b4360 0x7fb1b006bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:25.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.303+0000 7fb1b4c96640 1 -- 172.21.15.98:0/776193914 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b00ccf40 con 0x7fb1b006c1d0 2024-09-19T08:46:25.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.307+0000 7fb1b4c96640 1 --2- 172.21.15.98:0/776193914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1b006c1d0 0x7fb1b015efc0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fb1b006efc0 tx=0x7fb1a001c1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988a0f573dc49dce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:25.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.307+0000 7fb1aeffd640 1 -- 172.21.15.98:0/776193914 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a00b0070 con 0x7fb1b006c1d0 2024-09-19T08:46:25.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.307+0000 7fb1aeffd640 1 -- 172.21.15.98:0/776193914 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1a00ab600 con 0x7fb1b006c1d0 2024-09-19T08:46:25.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.307+0000 7fb1aeffd640 1 -- 172.21.15.98:0/776193914 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a00ab9d0 con 0x7fb1b006c1d0 2024-09-19T08:46:25.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.307+0000 7fb1b66f9640 1 -- 172.21.15.98:0/776193914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1b006c1d0 msgr2=0x7fb1b015efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:25.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.307+0000 7fb1b66f9640 1 --2- 172.21.15.98:0/776193914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1b006c1d0 0x7fb1b015efc0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fb1b006efc0 tx=0x7fb1a001c1a0 comp rx=0 tx=0).stop 2024-09-19T08:46:25.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.307+0000 7fb1b66f9640 1 -- 172.21.15.98:0/776193914 shutdown_connections 2024-09-19T08:46:25.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.307+0000 7fb1b66f9640 1 --2- 172.21.15.98:0/776193914 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1b006c1d0 0x7fb1b015efc0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:25.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.307+0000 7fb1b66f9640 1 --2- 172.21.15.98:0/776193914 >> v2:172.21.15.139:3300/0 conn(0x7fb1b00b4360 0x7fb1b006bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:25.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.307+0000 7fb1b66f9640 1 --2- 172.21.15.98:0/776193914 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1b0158fd0 0x7fb1b00b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:25.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.307+0000 7fb1b66f9640 1 -- 172.21.15.98:0/776193914 >> 172.21.15.98:0/776193914 conn(0x7fb1b00b2010 msgr2=0x7fb1b00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:25.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.307+0000 7fb1b66f9640 1 -- 172.21.15.98:0/776193914 shutdown_connections 2024-09-19T08:46:25.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.307+0000 7fb1b66f9640 1 -- 172.21.15.98:0/776193914 wait complete. 2024-09-19T08:46:25.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.307+0000 7fb1b66f9640 1 Processor -- start 2024-09-19T08:46:25.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.307+0000 7fb1b66f9640 1 -- start start 2024-09-19T08:46:25.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.307+0000 7fb1b66f9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1b006c1d0 0x7fb1b015dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:25.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.307+0000 7fb1b66f9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1b00b4360 0x7fb1b014de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:25.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.307+0000 7fb1b66f9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1b0158fd0 0x7fb1b00d08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:25.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.307+0000 7fb1b66f9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b015c920 con 0x7fb1b006c1d0 2024-09-19T08:46:25.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.307+0000 7fb1b66f9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b015c7a0 con 0x7fb1b00b4360 2024-09-19T08:46:25.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.307+0000 7fb1b66f9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb1b00ce840 con 0x7fb1b0158fd0 2024-09-19T08:46:25.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.307+0000 7fb1affff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1b006c1d0 0x7fb1b015dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:25.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.307+0000 7fb1affff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1b006c1d0 0x7fb1b015dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59714/0 (socket says 172.21.15.98:59714) 2024-09-19T08:46:25.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.307+0000 7fb1affff640 1 -- 172.21.15.98:0/3689781915 learned_addr learned my addr 172.21.15.98:0/3689781915 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:25.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.307+0000 7fb1af7fe640 1 --2- 172.21.15.98:0/3689781915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1b00b4360 0x7fb1b014de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:25.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.307+0000 7fb1b4c96640 1 --2- 172.21.15.98:0/3689781915 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1b0158fd0 0x7fb1b00d08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:25.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.307+0000 7fb1affff640 1 -- 172.21.15.98:0/3689781915 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1b0158fd0 msgr2=0x7fb1b00d08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:25.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.307+0000 7fb1affff640 1 --2- 172.21.15.98:0/3689781915 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1b0158fd0 0x7fb1b00d08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:25.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.311+0000 7fb1affff640 1 -- 172.21.15.98:0/3689781915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1b00b4360 msgr2=0x7fb1b014de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:25.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.311+0000 7fb1affff640 1 --2- 172.21.15.98:0/3689781915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1b00b4360 0x7fb1b014de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:25.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.311+0000 7fb1affff640 1 -- 172.21.15.98:0/3689781915 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b00cece0 con 0x7fb1b006c1d0 2024-09-19T08:46:25.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.311+0000 7fb1affff640 1 --2- 172.21.15.98:0/3689781915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1b006c1d0 0x7fb1b015dc40 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8007e70 tx=0x7fb1a800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:25.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.311+0000 7fb1acff9640 1 -- 172.21.15.98:0/3689781915 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a8014620 con 0x7fb1b006c1d0 2024-09-19T08:46:25.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.311+0000 7fb1acff9640 1 -- 172.21.15.98:0/3689781915 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1a8019070 con 0x7fb1b006c1d0 2024-09-19T08:46:25.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.311+0000 7fb1b66f9640 1 -- 172.21.15.98:0/3689781915 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1b014e3b0 con 0x7fb1b006c1d0 2024-09-19T08:46:25.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.311+0000 7fb1acff9640 1 -- 172.21.15.98:0/3689781915 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a8014a00 con 0x7fb1b006c1d0 2024-09-19T08:46:25.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.311+0000 7fb1b66f9640 1 -- 172.21.15.98:0/3689781915 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1b00cf230 con 0x7fb1b006c1d0 2024-09-19T08:46:25.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.311+0000 7fb1b66f9640 1 -- 172.21.15.98:0/3689781915 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1b006bc90 con 0x7fb1b006c1d0 2024-09-19T08:46:25.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.311+0000 7fb1acff9640 1 -- 172.21.15.98:0/3689781915 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 23) ==== 88057+0+0 (secure 0 0 0) 0x7fb1a8014ba0 con 0x7fb1b006c1d0 2024-09-19T08:46:25.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.311+0000 7fb1acff9640 1 --2- 172.21.15.98:0/3689781915 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb19c067f80 0x7fb19c06a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:25.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.311+0000 7fb1acff9640 1 -- 172.21.15.98:0/3689781915 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5737+0+0 (secure 0 0 0) 0x7fb1a8087020 con 0x7fb1b006c1d0 2024-09-19T08:46:25.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.311+0000 7fb1af7fe640 1 --2- 172.21.15.98:0/3689781915 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb19c067f80 0x7fb19c06a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:25.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.311+0000 7fb1af7fe640 1 --2- 172.21.15.98:0/3689781915 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb19c067f80 0x7fb19c06a440 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0157f70 tx=0x7fb1a4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:25.315 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.315+0000 7fb1acff9640 1 -- 172.21.15.98:0/3689781915 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb1a8011260 con 0x7fb1b006c1d0 2024-09-19T08:46:25.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.491+0000 7fb1b66f9640 1 -- 172.21.15.98:0/3689781915 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb1b00757a0 con 0x7fb1b006c1d0 2024-09-19T08:46:25.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.491+0000 7fb1acff9640 1 -- 172.21.15.98:0/3689781915 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v146) ==== 80+0+0 (secure 0 0 0) 0x7fb1a80572b0 con 0x7fb1b006c1d0 2024-09-19T08:46:25.489 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-19T08:46:25.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.491+0000 7fb1b66f9640 1 -- 172.21.15.98:0/3689781915 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb19c067f80 msgr2=0x7fb19c06a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:25.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.491+0000 7fb1b66f9640 1 --2- 172.21.15.98:0/3689781915 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb19c067f80 0x7fb19c06a440 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0157f70 tx=0x7fb1a4008040 comp rx=0 tx=0).stop 2024-09-19T08:46:25.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.491+0000 7fb1b66f9640 1 -- 172.21.15.98:0/3689781915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1b006c1d0 msgr2=0x7fb1b015dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:25.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.491+0000 7fb1b66f9640 1 --2- 172.21.15.98:0/3689781915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1b006c1d0 0x7fb1b015dc40 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8007e70 tx=0x7fb1a800e4b0 comp rx=0 tx=0).stop 2024-09-19T08:46:25.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.491+0000 7fb1b66f9640 1 -- 172.21.15.98:0/3689781915 shutdown_connections 2024-09-19T08:46:25.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.491+0000 7fb1b66f9640 1 --2- 172.21.15.98:0/3689781915 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb19c067f80 0x7fb19c06a440 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:25.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.491+0000 7fb1b66f9640 1 --2- 172.21.15.98:0/3689781915 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb1b0158fd0 0x7fb1b00d08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:25.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.491+0000 7fb1b66f9640 1 --2- 172.21.15.98:0/3689781915 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb1b00b4360 0x7fb1b014de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:25.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.491+0000 7fb1b66f9640 1 --2- 172.21.15.98:0/3689781915 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb1b006c1d0 0x7fb1b015dc40 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:25.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.491+0000 7fb1b66f9640 1 -- 172.21.15.98:0/3689781915 >> 172.21.15.98:0/3689781915 conn(0x7fb1b00b2010 msgr2=0x7fb1b0159b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:25.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.491+0000 7fb1b66f9640 1 -- 172.21.15.98:0/3689781915 shutdown_connections 2024-09-19T08:46:25.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.491+0000 7fb1b66f9640 1 -- 172.21.15.98:0/3689781915 wait complete. 2024-09-19T08:46:25.507 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi098.front.sepia.ceph.com... 2024-09-19T08:46:25.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:25.508 DEBUG:teuthology.orchestra.run.smithi098:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T08:46:25.518 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:ceph-fuse[51532]: fuse finished with error 0 and tester_r 0 2024-09-19T08:46:25.520 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:46:25.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:25.520 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T08:46:25.569 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:46:25.569 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:46:25.569 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:46:25.576 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T08:46:25.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.731+0000 7fb844f9f640 1 Processor -- start 2024-09-19T08:46:25.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.731+0000 7fb844f9f640 1 -- start start 2024-09-19T08:46:25.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.731+0000 7fb844f9f640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb840158e40 0x7fb84006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:25.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.731+0000 7fb844f9f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb84006c1d0 0x7fb8400ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:25.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.731+0000 7fb844f9f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8400ceb70 con 0x7fb8400ce6e0 2024-09-19T08:46:25.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.731+0000 7fb844f9f640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fb84006c610 con 0x7fb840158e40 2024-09-19T08:46:25.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.731+0000 7fb844f9f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb84015fcf0 con 0x7fb84006c1d0 2024-09-19T08:46:25.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.731+0000 7fb83e59c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb840158e40 0x7fb84006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:25.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.731+0000 7fb83e59c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb840158e40 0x7fb84006bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59730/0 (socket says 172.21.15.98:59730) 2024-09-19T08:46:25.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.731+0000 7fb83e59c640 1 -- 172.21.15.98:0/1451352282 learned_addr learned my addr 172.21.15.98:0/1451352282 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:25.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.731+0000 7fb83dd9b640 1 --2- 172.21.15.98:0/1451352282 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb84006c1d0 0x7fb8400ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:25.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.731+0000 7fb83e59c640 1 -- 172.21.15.98:0/1451352282 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb84006c1d0 msgr2=0x7fb8400ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:25.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.731+0000 7fb83e59c640 1 --2- 172.21.15.98:0/1451352282 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb84006c1d0 0x7fb8400ce1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:25.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.731+0000 7fb83e59c640 1 -- 172.21.15.98:0/1451352282 >> v1:172.21.15.139:6789/0 conn(0x7fb8400ce6e0 legacy=0x7fb84006d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:25.732 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.731+0000 7fb83e59c640 1 -- 172.21.15.98:0/1451352282 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb84015ef30 con 0x7fb840158e40 2024-09-19T08:46:25.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.735+0000 7fb83e59c640 1 --2- 172.21.15.98:0/1451352282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb840158e40 0x7fb84006bc90 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fb830009870 tx=0x7fb83001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de54db5f30ccbca7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:25.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.735+0000 7fb83d59a640 1 -- 172.21.15.98:0/1451352282 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8300b5070 con 0x7fb840158e40 2024-09-19T08:46:25.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.735+0000 7fb83d59a640 1 -- 172.21.15.98:0/1451352282 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8300a89d0 con 0x7fb840158e40 2024-09-19T08:46:25.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.735+0000 7fb83d59a640 1 -- 172.21.15.98:0/1451352282 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8300a8da0 con 0x7fb840158e40 2024-09-19T08:46:25.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.735+0000 7fb844f9f640 1 -- 172.21.15.98:0/1451352282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb840158e40 msgr2=0x7fb84006bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:25.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.735+0000 7fb844f9f640 1 --2- 172.21.15.98:0/1451352282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb840158e40 0x7fb84006bc90 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fb830009870 tx=0x7fb83001ed10 comp rx=0 tx=0).stop 2024-09-19T08:46:25.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.735+0000 7fb844f9f640 1 -- 172.21.15.98:0/1451352282 shutdown_connections 2024-09-19T08:46:25.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.735+0000 7fb844f9f640 1 --2- 172.21.15.98:0/1451352282 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb84006c1d0 0x7fb8400ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:25.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.735+0000 7fb844f9f640 1 --2- 172.21.15.98:0/1451352282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb840158e40 0x7fb84006bc90 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:25.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.735+0000 7fb844f9f640 1 -- 172.21.15.98:0/1451352282 >> 172.21.15.98:0/1451352282 conn(0x7fb8400b2010 msgr2=0x7fb8400b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:25.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.735+0000 7fb844f9f640 1 -- 172.21.15.98:0/1451352282 shutdown_connections 2024-09-19T08:46:25.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.735+0000 7fb844f9f640 1 -- 172.21.15.98:0/1451352282 wait complete. 2024-09-19T08:46:25.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.739+0000 7fb844f9f640 1 Processor -- start 2024-09-19T08:46:25.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.739+0000 7fb844f9f640 1 -- start start 2024-09-19T08:46:25.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.739+0000 7fb844f9f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb84006c1d0 0x7fb840143bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:25.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.739+0000 7fb844f9f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8400ce6e0 0x7fb8400b0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:25.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.739+0000 7fb844f9f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb840158e40 0x7fb8400b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:25.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.739+0000 7fb844f9f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb84015f7d0 con 0x7fb84006c1d0 2024-09-19T08:46:25.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.739+0000 7fb844f9f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb84015f4f0 con 0x7fb8400ce6e0 2024-09-19T08:46:25.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.739+0000 7fb844f9f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb840159220 con 0x7fb840158e40 2024-09-19T08:46:25.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.739+0000 7fb83e59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb84006c1d0 0x7fb840143bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:25.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.739+0000 7fb83e59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb84006c1d0 0x7fb840143bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59736/0 (socket says 172.21.15.98:59736) 2024-09-19T08:46:25.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.739+0000 7fb83e59c640 1 -- 172.21.15.98:0/999256781 learned_addr learned my addr 172.21.15.98:0/999256781 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:25.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.739+0000 7fb83ed9d640 1 --2- 172.21.15.98:0/999256781 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb840158e40 0x7fb8400b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:25.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.739+0000 7fb83dd9b640 1 --2- 172.21.15.98:0/999256781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8400ce6e0 0x7fb8400b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:25.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.739+0000 7fb83e59c640 1 -- 172.21.15.98:0/999256781 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb840158e40 msgr2=0x7fb8400b0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:25.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.739+0000 7fb83e59c640 1 --2- 172.21.15.98:0/999256781 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb840158e40 0x7fb8400b0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:25.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.739+0000 7fb83e59c640 1 -- 172.21.15.98:0/999256781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8400ce6e0 msgr2=0x7fb8400b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:25.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.739+0000 7fb83e59c640 1 --2- 172.21.15.98:0/999256781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8400ce6e0 0x7fb8400b0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:25.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.739+0000 7fb83e59c640 1 -- 172.21.15.98:0/999256781 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8400af960 con 0x7fb84006c1d0 2024-09-19T08:46:25.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.739+0000 7fb83e59c640 1 --2- 172.21.15.98:0/999256781 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb84006c1d0 0x7fb840143bc0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fb8300b3040 tx=0x7fb83001bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:25.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.739+0000 7fb82effd640 1 -- 172.21.15.98:0/999256781 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8300bf070 con 0x7fb84006c1d0 2024-09-19T08:46:25.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.739+0000 7fb82effd640 1 -- 172.21.15.98:0/999256781 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8300b0880 con 0x7fb84006c1d0 2024-09-19T08:46:25.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.739+0000 7fb844f9f640 1 -- 172.21.15.98:0/999256781 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb840159fc0 con 0x7fb84006c1d0 2024-09-19T08:46:25.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.739+0000 7fb82effd640 1 -- 172.21.15.98:0/999256781 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8300b5040 con 0x7fb84006c1d0 2024-09-19T08:46:25.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.739+0000 7fb844f9f640 1 -- 172.21.15.98:0/999256781 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb84015a580 con 0x7fb84006c1d0 2024-09-19T08:46:25.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.739+0000 7fb844f9f640 1 -- 172.21.15.98:0/999256781 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8400cee20 con 0x7fb84006c1d0 2024-09-19T08:46:25.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.743+0000 7fb82effd640 1 -- 172.21.15.98:0/999256781 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 23) ==== 88057+0+0 (secure 0 0 0) 0x7fb8300c2070 con 0x7fb84006c1d0 2024-09-19T08:46:25.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.743+0000 7fb82effd640 1 --2- 172.21.15.98:0/999256781 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb820067f80 0x7fb82006a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:25.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.743+0000 7fb82effd640 1 -- 172.21.15.98:0/999256781 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5737+0+0 (secure 0 0 0) 0x7fb830123180 con 0x7fb84006c1d0 2024-09-19T08:46:25.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.743+0000 7fb83dd9b640 1 --2- 172.21.15.98:0/999256781 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb820067f80 0x7fb82006a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:25.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.743+0000 7fb83dd9b640 1 --2- 172.21.15.98:0/999256781 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb820067f80 0x7fb82006a440 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb8400b4670 tx=0x7fb834008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:25.744 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.747+0000 7fb82effd640 1 -- 172.21.15.98:0/999256781 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb830018870 con 0x7fb84006c1d0 2024-09-19T08:46:25.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.907+0000 7fb844f9f640 1 -- 172.21.15.98:0/999256781 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb8400b40a0 con 0x7fb84006c1d0 2024-09-19T08:46:25.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.907+0000 7fb82effd640 1 -- 172.21.15.98:0/999256781 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v146) ==== 80+0+0 (secure 0 0 0) 0x7fb8300f3380 con 0x7fb84006c1d0 2024-09-19T08:46:25.906 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-19T08:46:25.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.907+0000 7fb844f9f640 1 -- 172.21.15.98:0/999256781 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb820067f80 msgr2=0x7fb82006a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:25.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.907+0000 7fb844f9f640 1 --2- 172.21.15.98:0/999256781 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb820067f80 0x7fb82006a440 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb8400b4670 tx=0x7fb834008040 comp rx=0 tx=0).stop 2024-09-19T08:46:25.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.907+0000 7fb844f9f640 1 -- 172.21.15.98:0/999256781 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb84006c1d0 msgr2=0x7fb840143bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:25.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.907+0000 7fb844f9f640 1 --2- 172.21.15.98:0/999256781 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb84006c1d0 0x7fb840143bc0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fb8300b3040 tx=0x7fb83001bab0 comp rx=0 tx=0).stop 2024-09-19T08:46:25.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.907+0000 7fb844f9f640 1 -- 172.21.15.98:0/999256781 shutdown_connections 2024-09-19T08:46:25.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.907+0000 7fb844f9f640 1 --2- 172.21.15.98:0/999256781 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb820067f80 0x7fb82006a440 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:25.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.907+0000 7fb844f9f640 1 --2- 172.21.15.98:0/999256781 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb840158e40 0x7fb8400b0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:25.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.907+0000 7fb844f9f640 1 --2- 172.21.15.98:0/999256781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8400ce6e0 0x7fb8400b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:25.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.907+0000 7fb844f9f640 1 --2- 172.21.15.98:0/999256781 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb84006c1d0 0x7fb840143bc0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:25.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.907+0000 7fb844f9f640 1 -- 172.21.15.98:0/999256781 >> 172.21.15.98:0/999256781 conn(0x7fb8400b2010 msgr2=0x7fb840072f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:25.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.907+0000 7fb844f9f640 1 -- 172.21.15.98:0/999256781 shutdown_connections 2024-09-19T08:46:25.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:25.907+0000 7fb844f9f640 1 -- 172.21.15.98:0/999256781 wait complete. 2024-09-19T08:46:25.923 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi139.front.sepia.ceph.com... 2024-09-19T08:46:25.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:25.923 DEBUG:teuthology.orchestra.run.smithi139:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-19T08:46:25.936 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[36777]: fuse finished with error 0 and tester_r 0 2024-09-19T08:46:25.938 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:46:25.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:25.938 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-19T08:46:26.032 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:46:26.032 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:46:26.032 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:46:26.038 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15846 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T08:46:26.038 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:46:26.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:26.038 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-19T08:46:26.089 INFO:teuthology.orchestra.run.smithi098.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-19T08:46:26.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:46:26.091 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:46:26.091 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:46:26.091 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:46:26.097 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T08:46:26.098 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:46:26.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:26.098 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-19T08:46:26.143 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-19T08:46:26.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:46:26.144 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:46:26.267 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.267+0000 7fad8781d640 1 Processor -- start 2024-09-19T08:46:26.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.267+0000 7fad8781d640 1 -- start start 2024-09-19T08:46:26.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.271+0000 7fad8781d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fad8015e800 0x7fad8015f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:26.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.271+0000 7fad8781d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad800b78b0 0x7fad800b7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:26.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.271+0000 7fad8781d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad8006c6e0 con 0x7fad8015e420 2024-09-19T08:46:26.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.271+0000 7fad8781d640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fad800b8240 con 0x7fad8015e800 2024-09-19T08:46:26.269 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.271+0000 7fad8781d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fad8006c9c0 con 0x7fad800b78b0 2024-09-19T08:46:26.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.271+0000 7fad84db8640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fad8015e800 0x7fad8015f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:26.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.271+0000 7fad84db8640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fad8015e800 0x7fad8015f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59738/0 (socket says 172.21.15.98:59738) 2024-09-19T08:46:26.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.271+0000 7fad84db8640 1 -- 172.21.15.98:0/581825644 learned_addr learned my addr 172.21.15.98:0/581825644 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:26.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.271+0000 7fad85dba640 1 --2- 172.21.15.98:0/581825644 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad800b78b0 0x7fad800b7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:26.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.271+0000 7fad84db8640 1 -- 172.21.15.98:0/581825644 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad800b78b0 msgr2=0x7fad800b7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:26.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.271+0000 7fad84db8640 1 --2- 172.21.15.98:0/581825644 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad800b78b0 0x7fad800b7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:26.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.271+0000 7fad84db8640 1 -- 172.21.15.98:0/581825644 >> v1:172.21.15.139:6789/0 conn(0x7fad8015e420 legacy=0x7fad8006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:26.270 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.271+0000 7fad84db8640 1 -- 172.21.15.98:0/581825644 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad8015ed60 con 0x7fad8015e800 2024-09-19T08:46:26.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.271+0000 7fad84db8640 1 --2- 172.21.15.98:0/581825644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad8015e800 0x7fad8015f560 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fad80070350 tx=0x7fad7401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe132e16c24d72e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:26.271 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.271+0000 7fad73fff640 1 -- 172.21.15.98:0/581825644 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad740b0070 con 0x7fad8015e800 2024-09-19T08:46:26.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.271+0000 7fad73fff640 1 -- 172.21.15.98:0/581825644 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad740ab600 con 0x7fad8015e800 2024-09-19T08:46:26.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.271+0000 7fad73fff640 1 -- 172.21.15.98:0/581825644 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad740ab9d0 con 0x7fad8015e800 2024-09-19T08:46:26.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.275+0000 7fad8781d640 1 -- 172.21.15.98:0/581825644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad8015e800 msgr2=0x7fad8015f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:26.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.275+0000 7fad8781d640 1 --2- 172.21.15.98:0/581825644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad8015e800 0x7fad8015f560 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fad80070350 tx=0x7fad7401c4e0 comp rx=0 tx=0).stop 2024-09-19T08:46:26.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.275+0000 7fad8781d640 1 -- 172.21.15.98:0/581825644 shutdown_connections 2024-09-19T08:46:26.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.275+0000 7fad8781d640 1 --2- 172.21.15.98:0/581825644 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad800b78b0 0x7fad800b7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:26.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.275+0000 7fad8781d640 1 --2- 172.21.15.98:0/581825644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad8015e800 0x7fad8015f560 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:26.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.275+0000 7fad8781d640 1 -- 172.21.15.98:0/581825644 >> 172.21.15.98:0/581825644 conn(0x7fad800b40a0 msgr2=0x7fad800b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:26.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.275+0000 7fad8781d640 1 -- 172.21.15.98:0/581825644 shutdown_connections 2024-09-19T08:46:26.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.275+0000 7fad8781d640 1 -- 172.21.15.98:0/581825644 wait complete. 2024-09-19T08:46:26.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.275+0000 7fad8781d640 1 Processor -- start 2024-09-19T08:46:26.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.275+0000 7fad8781d640 1 -- start start 2024-09-19T08:46:26.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.275+0000 7fad8781d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad800b78b0 0x7fad801f73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:26.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.275+0000 7fad8781d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad8015e420 0x7fad801615d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:26.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.275+0000 7fad8781d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad8015e800 0x7fad80156da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:26.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.275+0000 7fad8781d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fad80162a00 con 0x7fad8015e800 2024-09-19T08:46:26.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.275+0000 7fad8781d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fad80162880 con 0x7fad800b78b0 2024-09-19T08:46:26.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.275+0000 7fad8781d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fad80162b80 con 0x7fad8015e420 2024-09-19T08:46:26.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.275+0000 7fad85dba640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad8015e800 0x7fad80156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:26.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.275+0000 7fad85dba640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad8015e800 0x7fad80156da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59742/0 (socket says 172.21.15.98:59742) 2024-09-19T08:46:26.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.275+0000 7fad85dba640 1 -- 172.21.15.98:0/1470472371 learned_addr learned my addr 172.21.15.98:0/1470472371 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:26.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.275+0000 7fad855b9640 1 --2- 172.21.15.98:0/1470472371 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad800b78b0 0x7fad801f73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:26.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.275+0000 7fad84db8640 1 --2- 172.21.15.98:0/1470472371 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad8015e420 0x7fad801615d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:26.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.275+0000 7fad85dba640 1 -- 172.21.15.98:0/1470472371 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad8015e420 msgr2=0x7fad801615d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:26.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.275+0000 7fad85dba640 1 --2- 172.21.15.98:0/1470472371 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad8015e420 0x7fad801615d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:26.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.275+0000 7fad85dba640 1 -- 172.21.15.98:0/1470472371 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad800b78b0 msgr2=0x7fad801f73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:26.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.275+0000 7fad85dba640 1 --2- 172.21.15.98:0/1470472371 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad800b78b0 0x7fad801f73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:26.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.275+0000 7fad85dba640 1 -- 172.21.15.98:0/1470472371 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad8015d640 con 0x7fad8015e800 2024-09-19T08:46:26.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.275+0000 7fad85dba640 1 --2- 172.21.15.98:0/1470472371 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad8015e800 0x7fad80156da0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fad7c00d590 tx=0x7fad7c00da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:26.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.279+0000 7fad71ffb640 1 -- 172.21.15.98:0/1470472371 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7c0125f0 con 0x7fad8015e800 2024-09-19T08:46:26.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.279+0000 7fad71ffb640 1 -- 172.21.15.98:0/1470472371 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad7c019070 con 0x7fad8015e800 2024-09-19T08:46:26.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.279+0000 7fad8781d640 1 -- 172.21.15.98:0/1470472371 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad801672a0 con 0x7fad8015e800 2024-09-19T08:46:26.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.279+0000 7fad71ffb640 1 -- 172.21.15.98:0/1470472371 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7c0129f0 con 0x7fad8015e800 2024-09-19T08:46:26.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.279+0000 7fad8781d640 1 -- 172.21.15.98:0/1470472371 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad80163600 con 0x7fad8015e800 2024-09-19T08:46:26.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.279+0000 7fad8781d640 1 -- 172.21.15.98:0/1470472371 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad800b7d00 con 0x7fad8015e800 2024-09-19T08:46:26.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.279+0000 7fad71ffb640 1 -- 172.21.15.98:0/1470472371 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7fad7c012b90 con 0x7fad8015e800 2024-09-19T08:46:26.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.279+0000 7fad71ffb640 1 --2- 172.21.15.98:0/1470472371 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fad640680a0 0x7fad6406a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:26.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.279+0000 7fad71ffb640 1 -- 172.21.15.98:0/1470472371 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5737+0+0 (secure 0 0 0) 0x7fad7c0873c0 con 0x7fad8015e800 2024-09-19T08:46:26.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.279+0000 7fad855b9640 1 --2- 172.21.15.98:0/1470472371 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fad640680a0 0x7fad6406a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:26.279 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.279+0000 7fad855b9640 1 --2- 172.21.15.98:0/1470472371 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fad640680a0 0x7fad6406a560 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fad78009f20 tx=0x7fad780094d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:26.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.283+0000 7fad71ffb640 1 -- 172.21.15.98:0/1470472371 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fad7c056ec0 con 0x7fad8015e800 2024-09-19T08:46:26.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.467+0000 7fad8781d640 1 -- 172.21.15.98:0/1470472371 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad8006dc80 con 0x7fad8015e800 2024-09-19T08:46:26.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.467+0000 7fad71ffb640 1 -- 172.21.15.98:0/1470472371 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5221 (secure 0 0 0) 0x7fad7c0567c0 con 0x7fad8015e800 2024-09-19T08:46:26.468 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:46:26.468 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":83,"btime":"2024-09-19T08:46:14:993710+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6886,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3238157999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3238157999},{"type":"v1","addr":"172.21.15.139:6833","nonce":3238157999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/209335177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":209335177},{"type":"v1","addr":"172.21.15.139:6835","nonce":209335177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8370,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6834/26914461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":26914461},{"type":"v1","addr":"172.21.15.98:6834","nonce":26914461}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:46:03.123986+0000","modified":"2024-09-19T08:46:05.150802+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8373":{"gid":8373,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/451004304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6833","nonce":451004304},{"type":"v1","addr":"172.21.15.98:6835","nonce":451004304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8373,"qdb_cluster":[8373]},"id":11}]} 2024-09-19T08:46:26.468 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 83 2024-09-19T08:46:26.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.471+0000 7fad8781d640 1 -- 172.21.15.98:0/1470472371 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fad640680a0 msgr2=0x7fad6406a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:26.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.471+0000 7fad8781d640 1 --2- 172.21.15.98:0/1470472371 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fad640680a0 0x7fad6406a560 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fad78009f20 tx=0x7fad780094d0 comp rx=0 tx=0).stop 2024-09-19T08:46:26.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.471+0000 7fad8781d640 1 -- 172.21.15.98:0/1470472371 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad8015e800 msgr2=0x7fad80156da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:26.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.471+0000 7fad8781d640 1 --2- 172.21.15.98:0/1470472371 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad8015e800 0x7fad80156da0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fad7c00d590 tx=0x7fad7c00da50 comp rx=0 tx=0).stop 2024-09-19T08:46:26.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.471+0000 7fad8781d640 1 -- 172.21.15.98:0/1470472371 shutdown_connections 2024-09-19T08:46:26.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.471+0000 7fad8781d640 1 --2- 172.21.15.98:0/1470472371 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fad640680a0 0x7fad6406a560 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:26.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.471+0000 7fad8781d640 1 --2- 172.21.15.98:0/1470472371 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fad8015e800 0x7fad80156da0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:26.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.471+0000 7fad8781d640 1 --2- 172.21.15.98:0/1470472371 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fad8015e420 0x7fad801615d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:26.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.471+0000 7fad8781d640 1 --2- 172.21.15.98:0/1470472371 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fad800b78b0 0x7fad801f73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:26.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.471+0000 7fad8781d640 1 -- 172.21.15.98:0/1470472371 >> 172.21.15.98:0/1470472371 conn(0x7fad800b40a0 msgr2=0x7fad800b6750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:26.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.471+0000 7fad8781d640 1 -- 172.21.15.98:0/1470472371 shutdown_connections 2024-09-19T08:46:26.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.471+0000 7fad8781d640 1 -- 172.21.15.98:0/1470472371 wait complete. 2024-09-19T08:46:26.485 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:46:26.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.595+0000 7f2fdfcd4640 1 Processor -- start 2024-09-19T08:46:26.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.595+0000 7f2fdfcd4640 1 -- start start 2024-09-19T08:46:26.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.595+0000 7f2fdfcd4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2fd815c550 0x7f2fd806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:26.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.595+0000 7f2fdfcd4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fd80cfad0 0x7f2fd80723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:26.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.595+0000 7f2fdfcd4640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fd806d310 con 0x7f2fd806c1d0 2024-09-19T08:46:26.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.595+0000 7f2fdfcd4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f2fd806c640 con 0x7f2fd815c550 2024-09-19T08:46:26.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.595+0000 7f2fdfcd4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2fd806d5f0 con 0x7f2fd80cfad0 2024-09-19T08:46:26.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.595+0000 7f2fdd26f640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f2fd806c1d0 0x7f2fd80cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:44822/0 (socket says 172.21.15.98:44822) 2024-09-19T08:46:26.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.595+0000 7f2fdd26f640 1 -- 172.21.15.98:0/1287097484 learned_addr learned my addr 172.21.15.98:0/1287097484 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:26.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.595+0000 7f2fde271640 1 --2- 172.21.15.98:0/1287097484 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fd80cfad0 0x7f2fd80723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:26.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.595+0000 7f2fdda70640 1 --2- 172.21.15.98:0/1287097484 >> v2:172.21.15.139:3300/0 conn(0x7f2fd815c550 0x7f2fd806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:26.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.595+0000 7f2fdca6e640 1 -- 172.21.15.98:0/1287097484 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 478339497 0 0) 0x7f2fd806d310 con 0x7f2fd806c1d0 2024-09-19T08:46:26.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.595+0000 7f2fdca6e640 1 -- 172.21.15.98:0/1287097484 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fc4003530 con 0x7f2fd806c1d0 2024-09-19T08:46:26.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.595+0000 7f2fdca6e640 1 -- 172.21.15.98:0/1287097484 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1668953465 0 0) 0x7f2fc4003530 con 0x7f2fd806c1d0 2024-09-19T08:46:26.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.595+0000 7f2fdca6e640 1 -- 172.21.15.98:0/1287097484 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fd80cfad0 msgr2=0x7f2fd80723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:26.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.595+0000 7f2fdca6e640 1 --2- 172.21.15.98:0/1287097484 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fd80cfad0 0x7f2fd80723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:26.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.595+0000 7f2fdca6e640 1 -- 172.21.15.98:0/1287097484 >> v2:172.21.15.139:3300/0 conn(0x7f2fd815c550 msgr2=0x7f2fd806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:26.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.595+0000 7f2fdca6e640 1 --2- 172.21.15.98:0/1287097484 >> v2:172.21.15.139:3300/0 conn(0x7f2fd815c550 0x7f2fd806bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:26.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.595+0000 7f2fdca6e640 1 -- 172.21.15.98:0/1287097484 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd806d8d0 con 0x7f2fd806c1d0 2024-09-19T08:46:26.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.595+0000 7f2fdca6e640 1 -- 172.21.15.98:0/1287097484 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f2fc8001680 con 0x7f2fd806c1d0 2024-09-19T08:46:26.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.595+0000 7f2fdca6e640 1 -- 172.21.15.98:0/1287097484 >> v1:172.21.15.98:6789/0 conn(0x7f2fd806c1d0 legacy=0x7f2fd80cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:26.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.595+0000 7f2fdca6e640 1 --2- 172.21.15.98:0/1287097484 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fd80cfad0 0x7f2fc40039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:26.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.595+0000 7f2fdca6e640 1 --2- 172.21.15.98:0/1287097484 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fc4003f40 0x7f2fc4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:26.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.595+0000 7f2fdca6e640 1 --2- 172.21.15.98:0/1287097484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fc4004d50 0x7f2fc40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:26.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.595+0000 7f2fdca6e640 1 -- 172.21.15.98:0/1287097484 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd806c640 con 0x7f2fc4004d50 2024-09-19T08:46:26.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.595+0000 7f2fdca6e640 1 -- 172.21.15.98:0/1287097484 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd806d5f0 con 0x7f2fc4003f40 2024-09-19T08:46:26.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.595+0000 7f2fdca6e640 1 -- 172.21.15.98:0/1287097484 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2fc400ba20 con 0x7f2fd80cfad0 2024-09-19T08:46:26.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.595+0000 7f2fdda70640 1 --2- 172.21.15.98:0/1287097484 >> v2:172.21.15.139:3300/0 conn(0x7f2fd815c550 0x7f2fd806bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:26.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.595+0000 7f2fde271640 1 --2- 172.21.15.98:0/1287097484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fc4004d50 0x7f2fc40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:26.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.595+0000 7f2fdd26f640 1 --2- 172.21.15.98:0/1287097484 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fc4003f40 0x7f2fc4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:26.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.595+0000 7f2fde271640 1 -- 172.21.15.98:0/1287097484 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fd80cfad0 msgr2=0x7f2fc40039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:26.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.595+0000 7f2fde271640 1 --2- 172.21.15.98:0/1287097484 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fd80cfad0 0x7f2fc40039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:26.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.595+0000 7f2fde271640 1 -- 172.21.15.98:0/1287097484 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fc4003f40 msgr2=0x7f2fc4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:26.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.595+0000 7f2fde271640 1 --2- 172.21.15.98:0/1287097484 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fc4003f40 0x7f2fc4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:26.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.595+0000 7f2fde271640 1 -- 172.21.15.98:0/1287097484 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc4003530 con 0x7f2fc4004d50 2024-09-19T08:46:26.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.595+0000 7f2fdd26f640 1 --2- 172.21.15.98:0/1287097484 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fc4003f40 0x7f2fc4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:26.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.599+0000 7f2fde271640 1 --2- 172.21.15.98:0/1287097484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fc4004d50 0x7f2fc40051d0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0009fa0 tx=0x7f2fd001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5982a197b877f43 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:26.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.599+0000 7f2fdca6e640 1 -- 172.21.15.98:0/1287097484 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd00b2070 con 0x7f2fc4004d50 2024-09-19T08:46:26.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.599+0000 7f2fdca6e640 1 -- 172.21.15.98:0/1287097484 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fd00a5d50 con 0x7f2fc4004d50 2024-09-19T08:46:26.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.599+0000 7f2fdca6e640 1 -- 172.21.15.98:0/1287097484 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0009bf0 con 0x7f2fc4004d50 2024-09-19T08:46:26.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.599+0000 7f2fdfcd4640 1 -- 172.21.15.98:0/1287097484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fc4004d50 msgr2=0x7f2fc40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:26.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.599+0000 7f2fdfcd4640 1 --2- 172.21.15.98:0/1287097484 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fc4004d50 0x7f2fc40051d0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0009fa0 tx=0x7f2fd001cc80 comp rx=0 tx=0).stop 2024-09-19T08:46:26.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.599+0000 7f2fde271640 1 -- 172.21.15.98:0/1287097484 reap_dead start 2024-09-19T08:46:26.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.599+0000 7f2fdfcd4640 1 -- 172.21.15.98:0/1287097484 shutdown_connections 2024-09-19T08:46:26.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.599+0000 7f2fdfcd4640 1 -- 172.21.15.98:0/1287097484 >> 172.21.15.98:0/1287097484 conn(0x7f2fd80b2010 msgr2=0x7f2fd80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:26.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.599+0000 7f2fdfcd4640 1 -- 172.21.15.98:0/1287097484 shutdown_connections 2024-09-19T08:46:26.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.599+0000 7f2fdfcd4640 1 -- 172.21.15.98:0/1287097484 wait complete. 2024-09-19T08:46:26.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.599+0000 7f2fdfcd4640 1 Processor -- start 2024-09-19T08:46:26.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.603+0000 7f2fdfcd4640 1 -- start start 2024-09-19T08:46:26.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.603+0000 7f2fdfcd4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fd814de90 0x7f2fd815e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:26.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.603+0000 7f2fdfcd4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fd814e270 0x7f2fd80b0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:26.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.603+0000 7f2fdfcd4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fd80cc920 0x7f2fd80b0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:26.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.603+0000 7f2fdfcd4640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd815cab0 con 0x7f2fd814e270 2024-09-19T08:46:26.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.603+0000 7f2fdfcd4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd815c930 con 0x7f2fd814de90 2024-09-19T08:46:26.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.603+0000 7f2fdfcd4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2fd80723a0 con 0x7f2fd80cc920 2024-09-19T08:46:26.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.603+0000 7f2fdd26f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fd814e270 0x7f2fd80b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:26.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.603+0000 7f2fdd26f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fd814e270 0x7f2fd80b0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59750/0 (socket says 172.21.15.98:59750) 2024-09-19T08:46:26.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.603+0000 7f2fdd26f640 1 -- 172.21.15.98:0/1643467519 learned_addr learned my addr 172.21.15.98:0/1643467519 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:26.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.603+0000 7f2fdda70640 1 --2- 172.21.15.98:0/1643467519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fd814de90 0x7f2fd815e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:26.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.603+0000 7f2fde271640 1 --2- 172.21.15.98:0/1643467519 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fd80cc920 0x7f2fd80b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:26.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.603+0000 7f2fdd26f640 1 -- 172.21.15.98:0/1643467519 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fd80cc920 msgr2=0x7f2fd80b0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:26.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.603+0000 7f2fdd26f640 1 --2- 172.21.15.98:0/1643467519 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fd80cc920 0x7f2fd80b0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:26.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.603+0000 7f2fdd26f640 1 -- 172.21.15.98:0/1643467519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fd814de90 msgr2=0x7f2fd815e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:26.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.603+0000 7f2fdd26f640 1 --2- 172.21.15.98:0/1643467519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fd814de90 0x7f2fd815e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:26.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.603+0000 7f2fdd26f640 1 -- 172.21.15.98:0/1643467519 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd815ea40 con 0x7f2fd814e270 2024-09-19T08:46:26.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.603+0000 7f2fdda70640 1 --2- 172.21.15.98:0/1643467519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fd814de90 0x7f2fd815e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:26.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.603+0000 7f2fdd26f640 1 --2- 172.21.15.98:0/1643467519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fd814e270 0x7f2fd80b0120 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f2fc800dae0 tx=0x7f2fc8007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:26.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.603+0000 7f2fce7fc640 1 -- 172.21.15.98:0/1643467519 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc8019070 con 0x7f2fd814e270 2024-09-19T08:46:26.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.603+0000 7f2fce7fc640 1 -- 172.21.15.98:0/1643467519 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fc8004cc0 con 0x7f2fd814e270 2024-09-19T08:46:26.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.603+0000 7f2fdfcd4640 1 -- 172.21.15.98:0/1643467519 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fd815ed20 con 0x7f2fd814e270 2024-09-19T08:46:26.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.603+0000 7f2fce7fc640 1 -- 172.21.15.98:0/1643467519 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc80072d0 con 0x7f2fd814e270 2024-09-19T08:46:26.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.603+0000 7f2fdfcd4640 1 -- 172.21.15.98:0/1643467519 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fd815f2e0 con 0x7f2fd814e270 2024-09-19T08:46:26.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.603+0000 7f2fdfcd4640 1 -- 172.21.15.98:0/1643467519 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fd80cff60 con 0x7f2fd814e270 2024-09-19T08:46:26.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.603+0000 7f2fce7fc640 1 -- 172.21.15.98:0/1643467519 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f2fc801c070 con 0x7f2fd814e270 2024-09-19T08:46:26.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.603+0000 7f2fce7fc640 1 --2- 172.21.15.98:0/1643467519 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2fbc0680d0 0x7f2fbc06a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:26.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.603+0000 7f2fce7fc640 1 -- 172.21.15.98:0/1643467519 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5737+0+0 (secure 0 0 0) 0x7f2fc80842a0 con 0x7f2fd814e270 2024-09-19T08:46:26.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.607+0000 7f2fdda70640 1 --2- 172.21.15.98:0/1643467519 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2fbc0680d0 0x7f2fbc06a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:26.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.607+0000 7f2fdda70640 1 --2- 172.21.15.98:0/1643467519 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2fbc0680d0 0x7f2fbc06a590 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4009970 tx=0x7f2fd4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:26.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.611+0000 7f2fce7fc640 1 -- 172.21.15.98:0/1643467519 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2fc8056790 con 0x7f2fd814e270 2024-09-19T08:46:26.800 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.803+0000 7f2fdfcd4640 1 -- 172.21.15.98:0/1643467519 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2fd8159eb0 con 0x7f2fd814e270 2024-09-19T08:46:26.801 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:46:26.805 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":83,"btime":"2024-09-19T08:46:14:993710+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6886,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3238157999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3238157999},{"type":"v1","addr":"172.21.15.139:6833","nonce":3238157999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/209335177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":209335177},{"type":"v1","addr":"172.21.15.139:6835","nonce":209335177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8370,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6834/26914461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":26914461},{"type":"v1","addr":"172.21.15.98:6834","nonce":26914461}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:46:03.123986+0000","modified":"2024-09-19T08:46:05.150802+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8373":{"gid":8373,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/451004304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6833","nonce":451004304},{"type":"v1","addr":"172.21.15.98:6835","nonce":451004304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8373,"qdb_cluster":[8373]},"id":11}]} 2024-09-19T08:46:26.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.803+0000 7f2fce7fc640 1 -- 172.21.15.98:0/1643467519 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5221 (secure 0 0 0) 0x7f2fc8056090 con 0x7f2fd814e270 2024-09-19T08:46:26.806 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 83 2024-09-19T08:46:26.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.803+0000 7f2fdfcd4640 1 -- 172.21.15.98:0/1643467519 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2fbc0680d0 msgr2=0x7f2fbc06a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:26.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.803+0000 7f2fdfcd4640 1 --2- 172.21.15.98:0/1643467519 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2fbc0680d0 0x7f2fbc06a590 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4009970 tx=0x7f2fd4008040 comp rx=0 tx=0).stop 2024-09-19T08:46:26.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.803+0000 7f2fdfcd4640 1 -- 172.21.15.98:0/1643467519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fd814e270 msgr2=0x7f2fd80b0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:26.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.803+0000 7f2fdfcd4640 1 --2- 172.21.15.98:0/1643467519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fd814e270 0x7f2fd80b0120 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f2fc800dae0 tx=0x7f2fc8007ea0 comp rx=0 tx=0).stop 2024-09-19T08:46:26.806 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.803+0000 7f2fdfcd4640 1 -- 172.21.15.98:0/1643467519 shutdown_connections 2024-09-19T08:46:26.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.803+0000 7f2fdfcd4640 1 --2- 172.21.15.98:0/1643467519 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2fbc0680d0 0x7f2fbc06a590 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:26.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.803+0000 7f2fdfcd4640 1 --2- 172.21.15.98:0/1643467519 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2fd80cc920 0x7f2fd80b0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:26.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.803+0000 7f2fdfcd4640 1 --2- 172.21.15.98:0/1643467519 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2fd814e270 0x7f2fd80b0120 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:26.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.803+0000 7f2fdfcd4640 1 --2- 172.21.15.98:0/1643467519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2fd814de90 0x7f2fd815e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:26.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.803+0000 7f2fdfcd4640 1 -- 172.21.15.98:0/1643467519 >> 172.21.15.98:0/1643467519 conn(0x7f2fd80b2010 msgr2=0x7f2fd80cd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:26.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.803+0000 7f2fdfcd4640 1 -- 172.21.15.98:0/1643467519 shutdown_connections 2024-09-19T08:46:26.807 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.803+0000 7f2fdfcd4640 1 -- 172.21.15.98:0/1643467519 wait complete. 2024-09-19T08:46:26.820 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:46:26.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.927+0000 7f7d3cb93640 1 Processor -- start 2024-09-19T08:46:26.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.927+0000 7f7d3cb93640 1 -- start start 2024-09-19T08:46:26.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.931+0000 7f7d3cb93640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7d38158fd0 0x7f7d380b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:26.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.931+0000 7f7d3cb93640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7d380b4360 0x7f7d3806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:26.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.931+0000 7f7d3cb93640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d380ccc60 con 0x7f7d3806c1d0 2024-09-19T08:46:26.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.931+0000 7f7d3cb93640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f7d381593b0 con 0x7f7d38158fd0 2024-09-19T08:46:26.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.931+0000 7f7d3cb93640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7d3806c600 con 0x7f7d380b4360 2024-09-19T08:46:26.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.931+0000 7f7d3659c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7d38158fd0 0x7f7d380b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:26.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.931+0000 7f7d3659c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f7d38158fd0 0x7f7d380b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59752/0 (socket says 172.21.15.98:59752) 2024-09-19T08:46:26.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.931+0000 7f7d3659c640 1 -- 172.21.15.98:0/1016574588 learned_addr learned my addr 172.21.15.98:0/1016574588 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:26.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.931+0000 7f7d35d9b640 1 --2- 172.21.15.98:0/1016574588 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7d380b4360 0x7f7d3806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:26.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.931+0000 7f7d3659c640 1 -- 172.21.15.98:0/1016574588 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7d380b4360 msgr2=0x7f7d3806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:26.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.931+0000 7f7d3659c640 1 --2- 172.21.15.98:0/1016574588 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7d380b4360 0x7f7d3806bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:26.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.931+0000 7f7d3659c640 1 -- 172.21.15.98:0/1016574588 >> v1:172.21.15.139:6789/0 conn(0x7f7d3806c1d0 legacy=0x7f7d3815efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:26.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.931+0000 7f7d3659c640 1 -- 172.21.15.98:0/1016574588 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d380ccf40 con 0x7f7d38158fd0 2024-09-19T08:46:26.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.931+0000 7f7d3659c640 1 --2- 172.21.15.98:0/1016574588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d38158fd0 0x7f7d380b3e20 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f7d28009870 tx=0x7f7d2801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae11ee297e39171 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:26.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.931+0000 7f7d3559a640 1 -- 172.21.15.98:0/1016574588 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d280ab620 con 0x7f7d38158fd0 2024-09-19T08:46:26.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.931+0000 7f7d3559a640 1 -- 172.21.15.98:0/1016574588 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d280b0070 con 0x7f7d38158fd0 2024-09-19T08:46:26.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.931+0000 7f7d3559a640 1 -- 172.21.15.98:0/1016574588 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d280ab9e0 con 0x7f7d38158fd0 2024-09-19T08:46:26.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.935+0000 7f7d3cb93640 1 -- 172.21.15.98:0/1016574588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d38158fd0 msgr2=0x7f7d380b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:26.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.935+0000 7f7d3cb93640 1 --2- 172.21.15.98:0/1016574588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d38158fd0 0x7f7d380b3e20 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f7d28009870 tx=0x7f7d2801c460 comp rx=0 tx=0).stop 2024-09-19T08:46:26.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.935+0000 7f7d3cb93640 1 -- 172.21.15.98:0/1016574588 shutdown_connections 2024-09-19T08:46:26.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.935+0000 7f7d3cb93640 1 --2- 172.21.15.98:0/1016574588 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7d380b4360 0x7f7d3806bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:26.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.935+0000 7f7d3cb93640 1 --2- 172.21.15.98:0/1016574588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d38158fd0 0x7f7d380b3e20 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:26.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.935+0000 7f7d3cb93640 1 -- 172.21.15.98:0/1016574588 >> 172.21.15.98:0/1016574588 conn(0x7f7d380b2010 msgr2=0x7f7d380b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:26.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.935+0000 7f7d3cb93640 1 -- 172.21.15.98:0/1016574588 shutdown_connections 2024-09-19T08:46:26.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.935+0000 7f7d3cb93640 1 -- 172.21.15.98:0/1016574588 wait complete. 2024-09-19T08:46:26.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.935+0000 7f7d3cb93640 1 Processor -- start 2024-09-19T08:46:26.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.935+0000 7f7d3cb93640 1 -- start start 2024-09-19T08:46:26.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.935+0000 7f7d3cb93640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7d3806c1d0 0x7f7d380712f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:26.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.935+0000 7f7d3cb93640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d380b4360 0x7f7d38152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:26.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.935+0000 7f7d3cb93640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d38158fd0 0x7f7d3806d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:26.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.935+0000 7f7d3cb93640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3815c920 con 0x7f7d38158fd0 2024-09-19T08:46:26.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.935+0000 7f7d3cb93640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3815c7a0 con 0x7f7d380b4360 2024-09-19T08:46:26.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.935+0000 7f7d3cb93640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7d38168110 con 0x7f7d3806c1d0 2024-09-19T08:46:26.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.935+0000 7f7d36d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d38158fd0 0x7f7d3806d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:26.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.935+0000 7f7d36d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d38158fd0 0x7f7d3806d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59758/0 (socket says 172.21.15.98:59758) 2024-09-19T08:46:26.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.935+0000 7f7d36d9d640 1 -- 172.21.15.98:0/3842631256 learned_addr learned my addr 172.21.15.98:0/3842631256 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:26.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.935+0000 7f7d3659c640 1 --2- 172.21.15.98:0/3842631256 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7d3806c1d0 0x7f7d380712f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:26.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.935+0000 7f7d35d9b640 1 --2- 172.21.15.98:0/3842631256 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d380b4360 0x7f7d38152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:26.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.935+0000 7f7d36d9d640 1 -- 172.21.15.98:0/3842631256 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7d3806c1d0 msgr2=0x7f7d380712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:26.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.935+0000 7f7d36d9d640 1 --2- 172.21.15.98:0/3842631256 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7d3806c1d0 0x7f7d380712f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:26.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.935+0000 7f7d36d9d640 1 -- 172.21.15.98:0/3842631256 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d380b4360 msgr2=0x7f7d38152970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:26.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.935+0000 7f7d36d9d640 1 --2- 172.21.15.98:0/3842631256 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d380b4360 0x7f7d38152970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:26.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.935+0000 7f7d36d9d640 1 -- 172.21.15.98:0/3842631256 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d38163860 con 0x7f7d38158fd0 2024-09-19T08:46:26.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.935+0000 7f7d3659c640 1 --2- 172.21.15.98:0/3842631256 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7d3806c1d0 0x7f7d380712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:26.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.935+0000 7f7d35d9b640 1 --2- 172.21.15.98:0/3842631256 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d380b4360 0x7f7d38152970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:26.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.935+0000 7f7d36d9d640 1 --2- 172.21.15.98:0/3842631256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d38158fd0 0x7f7d3806d170 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000c7b0 tx=0x7f7d3000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:26.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.935+0000 7f7d26ffd640 1 -- 172.21.15.98:0/3842631256 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d30004b70 con 0x7f7d38158fd0 2024-09-19T08:46:26.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.935+0000 7f7d26ffd640 1 -- 172.21.15.98:0/3842631256 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d30004d10 con 0x7f7d38158fd0 2024-09-19T08:46:26.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.935+0000 7f7d3cb93640 1 -- 172.21.15.98:0/3842631256 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d38163b40 con 0x7f7d38158fd0 2024-09-19T08:46:26.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.935+0000 7f7d26ffd640 1 -- 172.21.15.98:0/3842631256 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3000aa90 con 0x7f7d38158fd0 2024-09-19T08:46:26.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.935+0000 7f7d3cb93640 1 -- 172.21.15.98:0/3842631256 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d38164100 con 0x7f7d38158fd0 2024-09-19T08:46:26.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.935+0000 7f7d3cb93640 1 -- 172.21.15.98:0/3842631256 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d3806bc90 con 0x7f7d38158fd0 2024-09-19T08:46:26.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.939+0000 7f7d26ffd640 1 -- 172.21.15.98:0/3842631256 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f7d3000ac30 con 0x7f7d38158fd0 2024-09-19T08:46:26.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.939+0000 7f7d26ffd640 1 --2- 172.21.15.98:0/3842631256 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7d180680a0 0x7f7d1806a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:26.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.939+0000 7f7d26ffd640 1 -- 172.21.15.98:0/3842631256 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5737+0+0 (secure 0 0 0) 0x7f7d30084910 con 0x7f7d38158fd0 2024-09-19T08:46:26.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.939+0000 7f7d3659c640 1 --2- 172.21.15.98:0/3842631256 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7d180680a0 0x7f7d1806a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:26.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.939+0000 7f7d3659c640 1 --2- 172.21.15.98:0/3842631256 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7d180680a0 0x7f7d1806a560 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7d280af000 tx=0x7f7d28008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:26.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:26.943+0000 7f7d26ffd640 1 -- 172.21.15.98:0/3842631256 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f7d30020070 con 0x7f7d38158fd0 2024-09-19T08:46:27.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.099+0000 7f7d3cb93640 1 -- 172.21.15.98:0/3842631256 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d38153450 con 0x7f7d38158fd0 2024-09-19T08:46:27.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.099+0000 7f7d26ffd640 1 -- 172.21.15.98:0/3842631256 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) ==== 74+0+16775 (secure 0 0 0) 0x7f7d30056d40 con 0x7f7d38158fd0 2024-09-19T08:46:27.099 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:46:27.099 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":146,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:46:04.805194+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:46:00.845215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-19T08:46:02.145134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:46:27.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.103+0000 7f7d3cb93640 1 -- 172.21.15.98:0/3842631256 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7d180680a0 msgr2=0x7f7d1806a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:27.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.103+0000 7f7d3cb93640 1 --2- 172.21.15.98:0/3842631256 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7d180680a0 0x7f7d1806a560 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7d280af000 tx=0x7f7d28008040 comp rx=0 tx=0).stop 2024-09-19T08:46:27.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.103+0000 7f7d3cb93640 1 -- 172.21.15.98:0/3842631256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d38158fd0 msgr2=0x7f7d3806d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:27.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.103+0000 7f7d3cb93640 1 --2- 172.21.15.98:0/3842631256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d38158fd0 0x7f7d3806d170 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f7d3000c7b0 tx=0x7f7d3000cc70 comp rx=0 tx=0).stop 2024-09-19T08:46:27.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.103+0000 7f7d3cb93640 1 -- 172.21.15.98:0/3842631256 shutdown_connections 2024-09-19T08:46:27.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.103+0000 7f7d3cb93640 1 --2- 172.21.15.98:0/3842631256 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f7d180680a0 0x7f7d1806a560 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:27.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.103+0000 7f7d3cb93640 1 --2- 172.21.15.98:0/3842631256 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7d38158fd0 0x7f7d3806d170 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:27.102 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.103+0000 7f7d3cb93640 1 --2- 172.21.15.98:0/3842631256 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7d380b4360 0x7f7d38152970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:27.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.103+0000 7f7d3cb93640 1 --2- 172.21.15.98:0/3842631256 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7d3806c1d0 0x7f7d380712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:27.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.103+0000 7f7d3cb93640 1 -- 172.21.15.98:0/3842631256 >> 172.21.15.98:0/3842631256 conn(0x7f7d380b2010 msgr2=0x7f7d380d02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:27.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.103+0000 7f7d3cb93640 1 -- 172.21.15.98:0/3842631256 shutdown_connections 2024-09-19T08:46:27.103 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.103+0000 7f7d3cb93640 1 -- 172.21.15.98:0/3842631256 wait complete. 2024-09-19T08:46:27.117 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T08:46:27.118 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:46:27.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.235+0000 7f1f46638640 1 Processor -- start 2024-09-19T08:46:27.236 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.239+0000 7f1f46638640 1 -- start start 2024-09-19T08:46:27.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.239+0000 7f1f46638640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f4006d880 0x7f1f400b3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:27.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.239+0000 7f1f46638640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f1f40141110 0x7f1f400b3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:27.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.239+0000 7f1f46638640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f4015ea10 con 0x7f1f40140d30 2024-09-19T08:46:27.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.239+0000 7f1f46638640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f1f4015e7c0 con 0x7f1f40141110 2024-09-19T08:46:27.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.239+0000 7f1f46638640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1f4015ecf0 con 0x7f1f4006d880 2024-09-19T08:46:27.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.239+0000 7f1f3ffff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f1f40140d30 0x7f1f4006d170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:44828/0 (socket says 172.21.15.98:44828) 2024-09-19T08:46:27.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.239+0000 7f1f3ffff640 1 -- 172.21.15.98:0/2686362560 learned_addr learned my addr 172.21.15.98:0/2686362560 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:27.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.239+0000 7f1f44bd5640 1 --2- 172.21.15.98:0/2686362560 >> v2:172.21.15.139:3300/0 conn(0x7f1f40141110 0x7f1f400b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:27.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.239+0000 7f1f3f7fe640 1 -- 172.21.15.98:0/2686362560 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3785701878 0 0) 0x7f1f4015ea10 con 0x7f1f40140d30 2024-09-19T08:46:27.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.239+0000 7f1f37fff640 1 --2- 172.21.15.98:0/2686362560 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f4006d880 0x7f1f400b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:27.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.239+0000 7f1f3f7fe640 1 -- 172.21.15.98:0/2686362560 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f2c003450 con 0x7f1f40140d30 2024-09-19T08:46:27.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.239+0000 7f1f3f7fe640 1 -- 172.21.15.98:0/2686362560 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 92212187 0 0) 0x7f1f2c003450 con 0x7f1f40140d30 2024-09-19T08:46:27.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.239+0000 7f1f3f7fe640 1 -- 172.21.15.98:0/2686362560 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f4006d880 msgr2=0x7f1f400b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:27.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.239+0000 7f1f3f7fe640 1 --2- 172.21.15.98:0/2686362560 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f4006d880 0x7f1f400b3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:27.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.239+0000 7f1f3f7fe640 1 -- 172.21.15.98:0/2686362560 >> v2:172.21.15.139:3300/0 conn(0x7f1f40141110 msgr2=0x7f1f400b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:27.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.239+0000 7f1f3f7fe640 1 --2- 172.21.15.98:0/2686362560 >> v2:172.21.15.139:3300/0 conn(0x7f1f40141110 0x7f1f400b3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:27.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.239+0000 7f1f3f7fe640 1 -- 172.21.15.98:0/2686362560 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f4015efd0 con 0x7f1f40140d30 2024-09-19T08:46:27.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.239+0000 7f1f3f7fe640 1 -- 172.21.15.98:0/2686362560 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f1f30001680 con 0x7f1f40140d30 2024-09-19T08:46:27.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.239+0000 7f1f3f7fe640 1 -- 172.21.15.98:0/2686362560 >> v1:172.21.15.98:6789/0 conn(0x7f1f40140d30 legacy=0x7f1f4006d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:27.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.239+0000 7f1f3f7fe640 1 --2- 172.21.15.98:0/2686362560 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f4006d880 0x7f1f2c003ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:27.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.239+0000 7f1f3f7fe640 1 --2- 172.21.15.98:0/2686362560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f2c004050 0x7f1f2c00a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:27.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.239+0000 7f1f3f7fe640 1 --2- 172.21.15.98:0/2686362560 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f2c00ae50 0x7f1f2c00d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:27.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.239+0000 7f1f3f7fe640 1 -- 172.21.15.98:0/2686362560 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4015e7c0 con 0x7f1f2c00ae50 2024-09-19T08:46:27.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.239+0000 7f1f3f7fe640 1 -- 172.21.15.98:0/2686362560 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4015ecf0 con 0x7f1f2c004050 2024-09-19T08:46:27.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.239+0000 7f1f3f7fe640 1 -- 172.21.15.98:0/2686362560 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1f2c00dba0 con 0x7f1f4006d880 2024-09-19T08:46:27.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.239+0000 7f1f44bd5640 1 --2- 172.21.15.98:0/2686362560 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f2c00ae50 0x7f1f2c00d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:27.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.239+0000 7f1f3ffff640 1 --2- 172.21.15.98:0/2686362560 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f4006d880 0x7f1f2c003ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:27.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.239+0000 7f1f37fff640 1 --2- 172.21.15.98:0/2686362560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f2c004050 0x7f1f2c00a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:27.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.239+0000 7f1f44bd5640 1 -- 172.21.15.98:0/2686362560 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f4006d880 msgr2=0x7f1f2c003ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:27.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.239+0000 7f1f44bd5640 1 --2- 172.21.15.98:0/2686362560 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f4006d880 0x7f1f2c003ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:27.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.239+0000 7f1f44bd5640 1 -- 172.21.15.98:0/2686362560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f2c004050 msgr2=0x7f1f2c00a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:27.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.239+0000 7f1f44bd5640 1 --2- 172.21.15.98:0/2686362560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f2c004050 0x7f1f2c00a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:27.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.239+0000 7f1f44bd5640 1 -- 172.21.15.98:0/2686362560 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f2c003450 con 0x7f1f2c00ae50 2024-09-19T08:46:27.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.239+0000 7f1f3ffff640 1 --2- 172.21.15.98:0/2686362560 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f4006d880 0x7f1f2c003ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:27.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.243+0000 7f1f44bd5640 1 --2- 172.21.15.98:0/2686362560 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f2c00ae50 0x7f1f2c00d2e0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f1f4006cd60 tx=0x7f1f3801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d0b5b86f53b3fa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:27.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.243+0000 7f1f3f7fe640 1 -- 172.21.15.98:0/2686362560 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f380b2070 con 0x7f1f2c00ae50 2024-09-19T08:46:27.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.243+0000 7f1f3f7fe640 1 -- 172.21.15.98:0/2686362560 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f380a5c70 con 0x7f1f2c00ae50 2024-09-19T08:46:27.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.243+0000 7f1f3f7fe640 1 -- 172.21.15.98:0/2686362560 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38009c10 con 0x7f1f2c00ae50 2024-09-19T08:46:27.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.243+0000 7f1f46638640 1 -- 172.21.15.98:0/2686362560 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f2c00ae50 msgr2=0x7f1f2c00d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:27.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.243+0000 7f1f46638640 1 --2- 172.21.15.98:0/2686362560 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f2c00ae50 0x7f1f2c00d2e0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f1f4006cd60 tx=0x7f1f3801cc60 comp rx=0 tx=0).stop 2024-09-19T08:46:27.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.243+0000 7f1f44bd5640 1 -- 172.21.15.98:0/2686362560 reap_dead start 2024-09-19T08:46:27.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.243+0000 7f1f46638640 1 -- 172.21.15.98:0/2686362560 shutdown_connections 2024-09-19T08:46:27.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.243+0000 7f1f46638640 1 -- 172.21.15.98:0/2686362560 >> 172.21.15.98:0/2686362560 conn(0x7f1f400b2010 msgr2=0x7f1f400b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:27.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.243+0000 7f1f46638640 1 -- 172.21.15.98:0/2686362560 shutdown_connections 2024-09-19T08:46:27.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.243+0000 7f1f46638640 1 -- 172.21.15.98:0/2686362560 wait complete. 2024-09-19T08:46:27.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.243+0000 7f1f46638640 1 Processor -- start 2024-09-19T08:46:27.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.243+0000 7f1f46638640 1 -- start start 2024-09-19T08:46:27.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.243+0000 7f1f46638640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f2c00ae50 0x7f1f400ae760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:27.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.243+0000 7f1f46638640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f40158cd0 0x7f1f4015db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:27.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.243+0000 7f1f46638640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f4015e0d0 0x7f1f40072590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:27.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.243+0000 7f1f46638640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1f400b3310 con 0x7f1f40158cd0 2024-09-19T08:46:27.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.243+0000 7f1f46638640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1f400ce1a0 con 0x7f1f2c00ae50 2024-09-19T08:46:27.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.243+0000 7f1f46638640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1f400b3490 con 0x7f1f4015e0d0 2024-09-19T08:46:27.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.243+0000 7f1f37fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f40158cd0 0x7f1f4015db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:27.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.243+0000 7f1f37fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f40158cd0 0x7f1f4015db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:59790/0 (socket says 172.21.15.98:59790) 2024-09-19T08:46:27.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.243+0000 7f1f37fff640 1 -- 172.21.15.98:0/1270857481 learned_addr learned my addr 172.21.15.98:0/1270857481 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:27.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.243+0000 7f1f3ffff640 1 --2- 172.21.15.98:0/1270857481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f2c00ae50 0x7f1f400ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:27.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.243+0000 7f1f37fff640 1 -- 172.21.15.98:0/1270857481 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f4015e0d0 msgr2=0x7f1f40072590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:27.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.247+0000 7f1f37fff640 1 --2- 172.21.15.98:0/1270857481 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f4015e0d0 0x7f1f40072590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:27.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.247+0000 7f1f37fff640 1 -- 172.21.15.98:0/1270857481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f2c00ae50 msgr2=0x7f1f400ae760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:27.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.247+0000 7f1f37fff640 1 --2- 172.21.15.98:0/1270857481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f2c00ae50 0x7f1f400ae760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:27.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.247+0000 7f1f37fff640 1 -- 172.21.15.98:0/1270857481 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f400ad6c0 con 0x7f1f40158cd0 2024-09-19T08:46:27.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.247+0000 7f1f37fff640 1 --2- 172.21.15.98:0/1270857481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f40158cd0 0x7f1f4015db90 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f1f28010840 tx=0x7f1f28010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:27.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.247+0000 7f1f3d7fa640 1 -- 172.21.15.98:0/1270857481 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2800f040 con 0x7f1f40158cd0 2024-09-19T08:46:27.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.247+0000 7f1f3d7fa640 1 -- 172.21.15.98:0/1270857481 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f2800eb60 con 0x7f1f40158cd0 2024-09-19T08:46:27.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.247+0000 7f1f46638640 1 -- 172.21.15.98:0/1270857481 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f400ad9a0 con 0x7f1f40158cd0 2024-09-19T08:46:27.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.247+0000 7f1f3d7fa640 1 -- 172.21.15.98:0/1270857481 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2801c400 con 0x7f1f40158cd0 2024-09-19T08:46:27.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.247+0000 7f1f46638640 1 -- 172.21.15.98:0/1270857481 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f400adf60 con 0x7f1f40158cd0 2024-09-19T08:46:27.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.247+0000 7f1f46638640 1 -- 172.21.15.98:0/1270857481 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f400d0dd0 con 0x7f1f40158cd0 2024-09-19T08:46:27.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.247+0000 7f1f3d7fa640 1 -- 172.21.15.98:0/1270857481 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f1f2801c5a0 con 0x7f1f40158cd0 2024-09-19T08:46:27.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.247+0000 7f1f3d7fa640 1 --2- 172.21.15.98:0/1270857481 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1f200680a0 0x7f1f2006a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:27.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.247+0000 7f1f3d7fa640 1 -- 172.21.15.98:0/1270857481 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5737+0+0 (secure 0 0 0) 0x7f1f28084d30 con 0x7f1f40158cd0 2024-09-19T08:46:27.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.247+0000 7f1f3ffff640 1 --2- 172.21.15.98:0/1270857481 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1f200680a0 0x7f1f2006a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:27.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.247+0000 7f1f3ffff640 1 --2- 172.21.15.98:0/1270857481 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1f200680a0 0x7f1f2006a560 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1f300017f0 tx=0x7f1f30009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:27.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.251+0000 7f1f3d7fa640 1 -- 172.21.15.98:0/1270857481 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f1f28018030 con 0x7f1f40158cd0 2024-09-19T08:46:27.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.447+0000 7f1f46638640 1 -- 172.21.15.98:0/1270857481 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f4006d0f0 con 0x7f1f40158cd0 2024-09-19T08:46:27.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.447+0000 7f1f3d7fa640 1 -- 172.21.15.98:0/1270857481 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5221 (secure 0 0 0) 0x7f1f28057160 con 0x7f1f40158cd0 2024-09-19T08:46:27.446 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:46:27.446 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":83,"btime":"2024-09-19T08:46:14:993710+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6886,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3238157999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3238157999},{"type":"v1","addr":"172.21.15.139:6833","nonce":3238157999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/209335177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":209335177},{"type":"v1","addr":"172.21.15.139:6835","nonce":209335177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8370,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6834/26914461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":26914461},{"type":"v1","addr":"172.21.15.98:6834","nonce":26914461}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:46:03.123986+0000","modified":"2024-09-19T08:46:05.150802+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8373":{"gid":8373,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/451004304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6833","nonce":451004304},{"type":"v1","addr":"172.21.15.98:6835","nonce":451004304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8373,"qdb_cluster":[8373]},"id":11}]} 2024-09-19T08:46:27.447 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 83 2024-09-19T08:46:27.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.451+0000 7f1f46638640 1 -- 172.21.15.98:0/1270857481 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1f200680a0 msgr2=0x7f1f2006a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:27.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.451+0000 7f1f46638640 1 --2- 172.21.15.98:0/1270857481 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1f200680a0 0x7f1f2006a560 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1f300017f0 tx=0x7f1f30009040 comp rx=0 tx=0).stop 2024-09-19T08:46:27.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.451+0000 7f1f46638640 1 -- 172.21.15.98:0/1270857481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f40158cd0 msgr2=0x7f1f4015db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:27.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.451+0000 7f1f46638640 1 --2- 172.21.15.98:0/1270857481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f40158cd0 0x7f1f4015db90 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f1f28010840 tx=0x7f1f28010d00 comp rx=0 tx=0).stop 2024-09-19T08:46:27.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.451+0000 7f1f46638640 1 -- 172.21.15.98:0/1270857481 shutdown_connections 2024-09-19T08:46:27.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.451+0000 7f1f46638640 1 --2- 172.21.15.98:0/1270857481 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1f200680a0 0x7f1f2006a560 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:27.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.451+0000 7f1f46638640 1 --2- 172.21.15.98:0/1270857481 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1f4015e0d0 0x7f1f40072590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:27.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.451+0000 7f1f46638640 1 --2- 172.21.15.98:0/1270857481 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1f40158cd0 0x7f1f4015db90 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:27.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.451+0000 7f1f46638640 1 --2- 172.21.15.98:0/1270857481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1f2c00ae50 0x7f1f400ae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:27.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.451+0000 7f1f46638640 1 -- 172.21.15.98:0/1270857481 >> 172.21.15.98:0/1270857481 conn(0x7f1f400b2010 msgr2=0x7f1f400ce6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:27.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.451+0000 7f1f46638640 1 -- 172.21.15.98:0/1270857481 shutdown_connections 2024-09-19T08:46:27.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.451+0000 7f1f46638640 1 -- 172.21.15.98:0/1270857481 wait complete. 2024-09-19T08:46:27.467 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:46:27.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.579+0000 7f98e3daf640 1 Processor -- start 2024-09-19T08:46:27.579 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.579+0000 7f98e3daf640 1 -- start start 2024-09-19T08:46:27.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.583+0000 7f98e3daf640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f98dc06c120 0x7f98dc0cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:27.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.583+0000 7f98e3daf640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98dc0ccc60 con 0x7f98dc06bc90 2024-09-19T08:46:27.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.583+0000 7f98e3daf640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98dc0ccf40 con 0x7f98dc158fd0 2024-09-19T08:46:27.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.583+0000 7f98e3daf640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f98dc1593b0 con 0x7f98dc06c120 2024-09-19T08:46:27.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.583+0000 7f98e234c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f98dc06c120 0x7f98dc0cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:27.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.583+0000 7f98e134a640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f98dc06bc90 0x7f98dc0ce840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36296/0 (socket says 172.21.15.98:36296) 2024-09-19T08:46:27.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.583+0000 7f98e134a640 1 -- 172.21.15.98:0/3052943381 learned_addr learned my addr 172.21.15.98:0/3052943381 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:27.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.583+0000 7f98e234c640 1 -- 172.21.15.98:0/3052943381 >> v1:172.21.15.139:6789/0 conn(0x7f98dc158fd0 legacy=0x7f98dc0b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:27.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.583+0000 7f98e234c640 1 -- 172.21.15.98:0/3052943381 >> v1:172.21.15.98:6789/0 conn(0x7f98dc06bc90 legacy=0x7f98dc0ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:27.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.583+0000 7f98e234c640 1 -- 172.21.15.98:0/3052943381 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98dc0cd220 con 0x7f98dc06c120 2024-09-19T08:46:27.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.583+0000 7f98e234c640 1 --2- 172.21.15.98:0/3052943381 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98dc06c120 0x7f98dc0cef50 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f98cc009d90 tx=0x7f98cc01c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6fac323b680d2f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:27.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.583+0000 7f98e0b49640 1 -- 172.21.15.98:0/3052943381 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc0b2070 con 0x7f98dc06c120 2024-09-19T08:46:27.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.583+0000 7f98e0b49640 1 -- 172.21.15.98:0/3052943381 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98cc0a5780 con 0x7f98dc06c120 2024-09-19T08:46:27.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.583+0000 7f98e0b49640 1 -- 172.21.15.98:0/3052943381 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc0a5b50 con 0x7f98dc06c120 2024-09-19T08:46:27.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.583+0000 7f98e3daf640 1 -- 172.21.15.98:0/3052943381 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98dc06c120 msgr2=0x7f98dc0cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:27.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.583+0000 7f98e3daf640 1 --2- 172.21.15.98:0/3052943381 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98dc06c120 0x7f98dc0cef50 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f98cc009d90 tx=0x7f98cc01c700 comp rx=0 tx=0).stop 2024-09-19T08:46:27.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.583+0000 7f98e3daf640 1 -- 172.21.15.98:0/3052943381 shutdown_connections 2024-09-19T08:46:27.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.583+0000 7f98e3daf640 1 --2- 172.21.15.98:0/3052943381 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98dc06c120 0x7f98dc0cef50 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:27.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.583+0000 7f98e3daf640 1 -- 172.21.15.98:0/3052943381 >> 172.21.15.98:0/3052943381 conn(0x7f98dc0b2010 msgr2=0x7f98dc0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:27.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.583+0000 7f98e3daf640 1 -- 172.21.15.98:0/3052943381 shutdown_connections 2024-09-19T08:46:27.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.583+0000 7f98e3daf640 1 -- 172.21.15.98:0/3052943381 wait complete. 2024-09-19T08:46:27.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.587+0000 7f98e3daf640 1 Processor -- start 2024-09-19T08:46:27.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.587+0000 7f98e3daf640 1 -- start start 2024-09-19T08:46:27.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.587+0000 7f98e3daf640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98dc06bc90 0x7f98dc0b0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:27.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.587+0000 7f98e3daf640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98dc06c120 0x7f98dc154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:27.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.587+0000 7f98e3daf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98dc158fd0 0x7f98dc073870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:27.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.587+0000 7f98e3daf640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc15c7a0 con 0x7f98dc158fd0 2024-09-19T08:46:27.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.587+0000 7f98e3daf640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc0b4530 con 0x7f98dc06bc90 2024-09-19T08:46:27.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.587+0000 7f98e3daf640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f98dc15c920 con 0x7f98dc06c120 2024-09-19T08:46:27.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.587+0000 7f98e234c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98dc158fd0 0x7f98dc073870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:27.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.587+0000 7f98e234c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98dc158fd0 0x7f98dc073870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:43174/0 (socket says 172.21.15.98:43174) 2024-09-19T08:46:27.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.587+0000 7f98e234c640 1 -- 172.21.15.98:0/3815827436 learned_addr learned my addr 172.21.15.98:0/3815827436 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:27.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.587+0000 7f98e234c640 1 -- 172.21.15.98:0/3815827436 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98dc06c120 msgr2=0x7f98dc154490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:27.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.587+0000 7f98e234c640 1 --2- 172.21.15.98:0/3815827436 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98dc06c120 0x7f98dc154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:27.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.587+0000 7f98e234c640 1 -- 172.21.15.98:0/3815827436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98dc06bc90 msgr2=0x7f98dc0b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:27.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.587+0000 7f98e234c640 1 --2- 172.21.15.98:0/3815827436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98dc06bc90 0x7f98dc0b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:27.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.587+0000 7f98e234c640 1 -- 172.21.15.98:0/3815827436 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98dc14d150 con 0x7f98dc158fd0 2024-09-19T08:46:27.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.587+0000 7f98e234c640 1 --2- 172.21.15.98:0/3815827436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98dc158fd0 0x7f98dc073870 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0b0040 tx=0x7f98cc0194a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:27.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.587+0000 7f98da7fc640 1 -- 172.21.15.98:0/3815827436 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc0bd020 con 0x7f98dc158fd0 2024-09-19T08:46:27.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.587+0000 7f98da7fc640 1 -- 172.21.15.98:0/3815827436 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98cc0b8670 con 0x7f98dc158fd0 2024-09-19T08:46:27.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.587+0000 7f98e3daf640 1 -- 172.21.15.98:0/3815827436 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98dc14e950 con 0x7f98dc158fd0 2024-09-19T08:46:27.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.587+0000 7f98da7fc640 1 -- 172.21.15.98:0/3815827436 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc0b2040 con 0x7f98dc158fd0 2024-09-19T08:46:27.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.587+0000 7f98e3daf640 1 -- 172.21.15.98:0/3815827436 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98dc14f7e0 con 0x7f98dc158fd0 2024-09-19T08:46:27.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.587+0000 7f98e3daf640 1 -- 172.21.15.98:0/3815827436 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98dc156fa0 con 0x7f98dc158fd0 2024-09-19T08:46:27.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.587+0000 7f98da7fc640 1 -- 172.21.15.98:0/3815827436 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f98cc0c0020 con 0x7f98dc158fd0 2024-09-19T08:46:27.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.591+0000 7f98da7fc640 1 --2- 172.21.15.98:0/3815827436 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f98c80680a0 0x7f98c806a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:27.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.591+0000 7f98da7fc640 1 -- 172.21.15.98:0/3815827436 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5737+0+0 (secure 0 0 0) 0x7f98cc11f5e0 con 0x7f98dc158fd0 2024-09-19T08:46:27.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.591+0000 7f98e1b4b640 1 --2- 172.21.15.98:0/3815827436 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f98c80680a0 0x7f98c806a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:27.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.591+0000 7f98e1b4b640 1 --2- 172.21.15.98:0/3815827436 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f98c80680a0 0x7f98c806a560 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f98d4009670 tx=0x7f98d4006bd0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:27.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.595+0000 7f98da7fc640 1 -- 172.21.15.98:0/3815827436 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f98cc0c7030 con 0x7f98dc158fd0 2024-09-19T08:46:27.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.787+0000 7f98e3daf640 1 -- 172.21.15.98:0/3815827436 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98dc153d30 con 0x7f98dc158fd0 2024-09-19T08:46:27.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.791+0000 7f98da7fc640 1 -- 172.21.15.98:0/3815827436 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5221 (secure 0 0 0) 0x7f98cc016480 con 0x7f98dc158fd0 2024-09-19T08:46:27.789 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:46:27.789 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":83,"btime":"2024-09-19T08:46:14:993710+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6886,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3238157999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3238157999},{"type":"v1","addr":"172.21.15.139:6833","nonce":3238157999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/209335177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":209335177},{"type":"v1","addr":"172.21.15.139:6835","nonce":209335177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8370,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6834/26914461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":26914461},{"type":"v1","addr":"172.21.15.98:6834","nonce":26914461}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:46:03.123986+0000","modified":"2024-09-19T08:46:05.150802+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8373},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8373":{"gid":8373,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/451004304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6833","nonce":451004304},{"type":"v1","addr":"172.21.15.98:6835","nonce":451004304}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8373,"qdb_cluster":[8373]},"id":11}]} 2024-09-19T08:46:27.790 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 83 2024-09-19T08:46:27.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.791+0000 7f98e3daf640 1 -- 172.21.15.98:0/3815827436 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f98c80680a0 msgr2=0x7f98c806a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:27.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.791+0000 7f98e3daf640 1 --2- 172.21.15.98:0/3815827436 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f98c80680a0 0x7f98c806a560 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f98d4009670 tx=0x7f98d4006bd0 comp rx=0 tx=0).stop 2024-09-19T08:46:27.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.791+0000 7f98e3daf640 1 -- 172.21.15.98:0/3815827436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98dc158fd0 msgr2=0x7f98dc073870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:27.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.791+0000 7f98e3daf640 1 --2- 172.21.15.98:0/3815827436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98dc158fd0 0x7f98dc073870 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0b0040 tx=0x7f98cc0194a0 comp rx=0 tx=0).stop 2024-09-19T08:46:27.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.791+0000 7f98e3daf640 1 -- 172.21.15.98:0/3815827436 shutdown_connections 2024-09-19T08:46:27.793 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.791+0000 7f98e3daf640 1 --2- 172.21.15.98:0/3815827436 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f98c80680a0 0x7f98c806a560 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:27.793 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.791+0000 7f98e3daf640 1 --2- 172.21.15.98:0/3815827436 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f98dc158fd0 0x7f98dc073870 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:27.793 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.791+0000 7f98e3daf640 1 --2- 172.21.15.98:0/3815827436 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f98dc06c120 0x7f98dc154490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:27.793 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.791+0000 7f98e3daf640 1 --2- 172.21.15.98:0/3815827436 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98dc06bc90 0x7f98dc0b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:27.793 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.791+0000 7f98e3daf640 1 -- 172.21.15.98:0/3815827436 >> 172.21.15.98:0/3815827436 conn(0x7f98dc0b2010 msgr2=0x7f98dc156890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:27.793 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.791+0000 7f98e3daf640 1 -- 172.21.15.98:0/3815827436 shutdown_connections 2024-09-19T08:46:27.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.791+0000 7f98e3daf640 1 -- 172.21.15.98:0/3815827436 wait complete. 2024-09-19T08:46:27.807 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:46:27.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.927+0000 7f146871d640 1 Processor -- start 2024-09-19T08:46:27.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.927+0000 7f146871d640 1 -- start start 2024-09-19T08:46:27.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.927+0000 7f146871d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f146006bc90 0x7f146006c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:27.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.927+0000 7f146871d640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14600ccc60 con 0x7f1460158fd0 2024-09-19T08:46:27.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.927+0000 7f146871d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14600ccf40 con 0x7f14600ce840 2024-09-19T08:46:27.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.927+0000 7f146871d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f14601593b0 con 0x7f146006bc90 2024-09-19T08:46:27.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.927+0000 7f14664b9640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f1460158fd0 0x7f14600b3e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36308/0 (socket says 172.21.15.98:36308) 2024-09-19T08:46:27.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.927+0000 7f14664b9640 1 -- 172.21.15.98:0/2449875886 learned_addr learned my addr 172.21.15.98:0/2449875886 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:27.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.927+0000 7f1465cb8640 1 --2- 172.21.15.98:0/2449875886 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f146006bc90 0x7f146006c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:27.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.927+0000 7f14654b7640 1 -- 172.21.15.98:0/2449875886 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2061559800 0 0) 0x7f14600ccc60 con 0x7f1460158fd0 2024-09-19T08:46:27.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.927+0000 7f14654b7640 1 -- 172.21.15.98:0/2449875886 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f144c003530 con 0x7f1460158fd0 2024-09-19T08:46:27.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.927+0000 7f14654b7640 1 -- 172.21.15.98:0/2449875886 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 554272552 0 0) 0x7f144c003530 con 0x7f1460158fd0 2024-09-19T08:46:27.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.927+0000 7f14654b7640 1 -- 172.21.15.98:0/2449875886 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f146006bc90 msgr2=0x7f146006c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:27.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.927+0000 7f14654b7640 1 --2- 172.21.15.98:0/2449875886 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f146006bc90 0x7f146006c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:27.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.927+0000 7f14654b7640 1 -- 172.21.15.98:0/2449875886 >> v1:172.21.15.139:6789/0 conn(0x7f14600ce840 legacy=0x7f14600cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:27.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.927+0000 7f14654b7640 1 -- 172.21.15.98:0/2449875886 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14600cd220 con 0x7f1460158fd0 2024-09-19T08:46:27.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.927+0000 7f14654b7640 1 -- 172.21.15.98:0/2449875886 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f145c001680 con 0x7f1460158fd0 2024-09-19T08:46:27.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.927+0000 7f14654b7640 1 -- 172.21.15.98:0/2449875886 >> v1:172.21.15.98:6789/0 conn(0x7f1460158fd0 legacy=0x7f14600b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:27.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.927+0000 7f14654b7640 1 --2- 172.21.15.98:0/2449875886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f144c0039f0 0x7f144c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:27.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.927+0000 7f14654b7640 1 --2- 172.21.15.98:0/2449875886 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f146006bc90 0x7f144c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:27.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.927+0000 7f14654b7640 1 --2- 172.21.15.98:0/2449875886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f144c004d50 0x7f144c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:27.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.927+0000 7f14654b7640 1 -- 172.21.15.98:0/2449875886 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f14601593b0 con 0x7f144c004d50 2024-09-19T08:46:27.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.927+0000 7f14654b7640 1 -- 172.21.15.98:0/2449875886 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f144c00bb30 con 0x7f144c0039f0 2024-09-19T08:46:27.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.927+0000 7f14654b7640 1 -- 172.21.15.98:0/2449875886 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f144c00bd30 con 0x7f146006bc90 2024-09-19T08:46:27.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.927+0000 7f1466cba640 1 --2- 172.21.15.98:0/2449875886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f144c004d50 0x7f144c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:27.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.927+0000 7f1465cb8640 1 --2- 172.21.15.98:0/2449875886 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f146006bc90 0x7f144c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:27.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.927+0000 7f14664b9640 1 --2- 172.21.15.98:0/2449875886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f144c0039f0 0x7f144c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:27.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.927+0000 7f1466cba640 1 -- 172.21.15.98:0/2449875886 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f146006bc90 msgr2=0x7f144c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:27.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.927+0000 7f1466cba640 1 --2- 172.21.15.98:0/2449875886 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f146006bc90 0x7f144c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:27.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.927+0000 7f1466cba640 1 -- 172.21.15.98:0/2449875886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f144c0039f0 msgr2=0x7f144c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:27.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.927+0000 7f1466cba640 1 --2- 172.21.15.98:0/2449875886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f144c0039f0 0x7f144c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:27.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.927+0000 7f1466cba640 1 -- 172.21.15.98:0/2449875886 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f144c003530 con 0x7f144c004d50 2024-09-19T08:46:27.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.927+0000 7f14664b9640 1 --2- 172.21.15.98:0/2449875886 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f144c0039f0 0x7f144c003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:27.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.931+0000 7f1466cba640 1 --2- 172.21.15.98:0/2449875886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f144c004d50 0x7f144c0051d0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f1458001e70 tx=0x7f145801c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d90f90c1b921d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:27.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.931+0000 7f14654b7640 1 -- 172.21.15.98:0/2449875886 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1458001a90 con 0x7f144c004d50 2024-09-19T08:46:27.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.931+0000 7f14654b7640 1 -- 172.21.15.98:0/2449875886 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1458003b80 con 0x7f144c004d50 2024-09-19T08:46:27.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.931+0000 7f14654b7640 1 -- 172.21.15.98:0/2449875886 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1458005080 con 0x7f144c004d50 2024-09-19T08:46:27.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.931+0000 7f146871d640 1 -- 172.21.15.98:0/2449875886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f144c004d50 msgr2=0x7f144c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:27.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.931+0000 7f146871d640 1 --2- 172.21.15.98:0/2449875886 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f144c004d50 0x7f144c0051d0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f1458001e70 tx=0x7f145801c780 comp rx=0 tx=0).stop 2024-09-19T08:46:27.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.931+0000 7f1466cba640 1 -- 172.21.15.98:0/2449875886 reap_dead start 2024-09-19T08:46:27.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.931+0000 7f146871d640 1 -- 172.21.15.98:0/2449875886 shutdown_connections 2024-09-19T08:46:27.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.931+0000 7f146871d640 1 -- 172.21.15.98:0/2449875886 >> 172.21.15.98:0/2449875886 conn(0x7f14600b2010 msgr2=0x7f14600b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:27.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.931+0000 7f146871d640 1 -- 172.21.15.98:0/2449875886 shutdown_connections 2024-09-19T08:46:27.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.931+0000 7f146871d640 1 -- 172.21.15.98:0/2449875886 wait complete. 2024-09-19T08:46:27.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.931+0000 7f146871d640 1 Processor -- start 2024-09-19T08:46:27.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.931+0000 7f146871d640 1 -- start start 2024-09-19T08:46:27.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.931+0000 7f146871d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1460152060 0x7f146015cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:27.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.931+0000 7f146871d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1460071cf0 0x7f1460159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:27.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.931+0000 7f146871d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f146015a520 0x7f1460153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:27.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.931+0000 7f146871d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f14600b4530 con 0x7f1460152060 2024-09-19T08:46:27.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.931+0000 7f146871d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f14600cd7e0 con 0x7f1460071cf0 2024-09-19T08:46:27.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.931+0000 7f146871d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f146015c7a0 con 0x7f146015a520 2024-09-19T08:46:27.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.931+0000 7f14664b9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1460152060 0x7f146015cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:27.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.931+0000 7f14664b9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1460152060 0x7f146015cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:43188/0 (socket says 172.21.15.98:43188) 2024-09-19T08:46:27.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.931+0000 7f14664b9640 1 -- 172.21.15.98:0/1910477137 learned_addr learned my addr 172.21.15.98:0/1910477137 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:27.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.931+0000 7f1465cb8640 1 --2- 172.21.15.98:0/1910477137 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1460071cf0 0x7f1460159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:27.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.931+0000 7f1466cba640 1 --2- 172.21.15.98:0/1910477137 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f146015a520 0x7f1460153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:27.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.931+0000 7f14664b9640 1 -- 172.21.15.98:0/1910477137 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f146015a520 msgr2=0x7f1460153770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:27.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.931+0000 7f14664b9640 1 --2- 172.21.15.98:0/1910477137 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f146015a520 0x7f1460153770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:27.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.931+0000 7f14664b9640 1 -- 172.21.15.98:0/1910477137 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1460071cf0 msgr2=0x7f1460159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:27.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.931+0000 7f14664b9640 1 --2- 172.21.15.98:0/1910477137 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1460071cf0 0x7f1460159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:27.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.931+0000 7f14664b9640 1 -- 172.21.15.98:0/1910477137 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f146015d820 con 0x7f1460152060 2024-09-19T08:46:27.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.931+0000 7f1466cba640 1 --2- 172.21.15.98:0/1910477137 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f146015a520 0x7f1460153770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:27.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.931+0000 7f14664b9640 1 --2- 172.21.15.98:0/1910477137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1460152060 0x7f146015cc40 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f145c010c00 tx=0x7f145c00e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:27.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.935+0000 7f1456ffd640 1 -- 172.21.15.98:0/1910477137 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145c01b070 con 0x7f1460152060 2024-09-19T08:46:27.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.935+0000 7f1456ffd640 1 -- 172.21.15.98:0/1910477137 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f145c011040 con 0x7f1460152060 2024-09-19T08:46:27.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.935+0000 7f146871d640 1 -- 172.21.15.98:0/1910477137 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f146015d540 con 0x7f1460152060 2024-09-19T08:46:27.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.935+0000 7f1456ffd640 1 -- 172.21.15.98:0/1910477137 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145c0143a0 con 0x7f1460152060 2024-09-19T08:46:27.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.935+0000 7f146871d640 1 -- 172.21.15.98:0/1910477137 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f146015f210 con 0x7f1460152060 2024-09-19T08:46:27.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.935+0000 7f146871d640 1 -- 172.21.15.98:0/1910477137 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f146015a930 con 0x7f1460152060 2024-09-19T08:46:27.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.935+0000 7f1456ffd640 1 -- 172.21.15.98:0/1910477137 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f145c01e070 con 0x7f1460152060 2024-09-19T08:46:27.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.935+0000 7f1456ffd640 1 --2- 172.21.15.98:0/1910477137 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f14440680d0 0x7f144406a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:27.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.935+0000 7f1456ffd640 1 -- 172.21.15.98:0/1910477137 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5737+0+0 (secure 0 0 0) 0x7f145c0871f0 con 0x7f1460152060 2024-09-19T08:46:27.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.935+0000 7f1465cb8640 1 --2- 172.21.15.98:0/1910477137 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f14440680d0 0x7f144406a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:27.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.935+0000 7f1465cb8640 1 --2- 172.21.15.98:0/1910477137 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f14440680d0 0x7f144406a590 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f146015cbb0 tx=0x7f145000b3a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:27.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:27.939+0000 7f1456ffd640 1 -- 172.21.15.98:0/1910477137 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f146015a930 con 0x7f1460152060 2024-09-19T08:46:28.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.087+0000 7f146871d640 1 -- 172.21.15.98:0/1910477137 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f146006c120 con 0x7f1460152060 2024-09-19T08:46:28.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.091+0000 7f1456ffd640 1 -- 172.21.15.98:0/1910477137 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) ==== 74+0+16775 (secure 0 0 0) 0x7f145c059620 con 0x7f1460152060 2024-09-19T08:46:28.090 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:46:28.090 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":146,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:46:04.805194+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:46:00.845215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-19T08:46:02.145134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:46:28.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.095+0000 7f146871d640 1 -- 172.21.15.98:0/1910477137 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f14440680d0 msgr2=0x7f144406a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:28.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.095+0000 7f146871d640 1 --2- 172.21.15.98:0/1910477137 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f14440680d0 0x7f144406a590 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f146015cbb0 tx=0x7f145000b3a0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.095+0000 7f146871d640 1 -- 172.21.15.98:0/1910477137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1460152060 msgr2=0x7f146015cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:28.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.095+0000 7f146871d640 1 --2- 172.21.15.98:0/1910477137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1460152060 0x7f146015cc40 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f145c010c00 tx=0x7f145c00e4d0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.095+0000 7f146871d640 1 -- 172.21.15.98:0/1910477137 shutdown_connections 2024-09-19T08:46:28.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.095+0000 7f146871d640 1 --2- 172.21.15.98:0/1910477137 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f14440680d0 0x7f144406a590 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.095+0000 7f146871d640 1 --2- 172.21.15.98:0/1910477137 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f146015a520 0x7f1460153770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.095+0000 7f146871d640 1 --2- 172.21.15.98:0/1910477137 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1460071cf0 0x7f1460159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.095+0000 7f146871d640 1 --2- 172.21.15.98:0/1910477137 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1460152060 0x7f146015cc40 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.095+0000 7f146871d640 1 -- 172.21.15.98:0/1910477137 >> 172.21.15.98:0/1910477137 conn(0x7f14600b2010 msgr2=0x7f14600b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:28.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.095+0000 7f146871d640 1 -- 172.21.15.98:0/1910477137 shutdown_connections 2024-09-19T08:46:28.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.095+0000 7f146871d640 1 -- 172.21.15.98:0/1910477137 wait complete. 2024-09-19T08:46:28.107 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-19T08:46:28.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.227+0000 7fed8e79c640 1 Processor -- start 2024-09-19T08:46:28.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.227+0000 7fed8e79c640 1 -- start start 2024-09-19T08:46:28.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.227+0000 7fed8e79c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fed880b6f60 0x7fed8806d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:28.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.227+0000 7fed8e79c640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed88160350 con 0x7fed880b0280 2024-09-19T08:46:28.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.227+0000 7fed8e79c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed88160630 con 0x7fed88074c70 2024-09-19T08:46:28.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.227+0000 7fed8e79c640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fed88160910 con 0x7fed880b6f60 2024-09-19T08:46:28.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.227+0000 7fed877fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fed880b6f60 0x7fed8806d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:28.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.227+0000 7fed877fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fed880b6f60 0x7fed8806d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:43202/0 (socket says 172.21.15.98:43202) 2024-09-19T08:46:28.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.227+0000 7fed877fe640 1 -- 172.21.15.98:0/289408159 learned_addr learned my addr 172.21.15.98:0/289408159 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:28.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.227+0000 7fed87fff640 1 --1- 172.21.15.98:0/289408159 >> v1:172.21.15.98:6789/0 conn(0x7fed880b0280 0x7fed880b6850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36314/0 (socket says 172.21.15.98:36314) 2024-09-19T08:46:28.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.227+0000 7fed877fe640 1 -- 172.21.15.98:0/289408159 >> v1:172.21.15.139:6789/0 conn(0x7fed88074c70 legacy=0x7fed880ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:28.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.227+0000 7fed877fe640 1 -- 172.21.15.98:0/289408159 >> v1:172.21.15.98:6789/0 conn(0x7fed880b0280 legacy=0x7fed880b6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:28.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.227+0000 7fed877fe640 1 -- 172.21.15.98:0/289408159 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed88160bf0 con 0x7fed880b6f60 2024-09-19T08:46:28.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.231+0000 7fed877fe640 1 --2- 172.21.15.98:0/289408159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed880b6f60 0x7fed8806d1d0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fed7c009870 tx=0x7fed7c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc9ea5bef4d3ffd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:28.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.231+0000 7fed86ffd640 1 -- 172.21.15.98:0/289408159 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c0b0070 con 0x7fed880b6f60 2024-09-19T08:46:28.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.231+0000 7fed86ffd640 1 -- 172.21.15.98:0/289408159 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed7c0ab600 con 0x7fed880b6f60 2024-09-19T08:46:28.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.231+0000 7fed86ffd640 1 -- 172.21.15.98:0/289408159 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c0ab9d0 con 0x7fed880b6f60 2024-09-19T08:46:28.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.231+0000 7fed8e79c640 1 -- 172.21.15.98:0/289408159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed880b6f60 msgr2=0x7fed8806d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:28.229 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.231+0000 7fed8e79c640 1 --2- 172.21.15.98:0/289408159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed880b6f60 0x7fed8806d1d0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fed7c009870 tx=0x7fed7c01c490 comp rx=0 tx=0).stop 2024-09-19T08:46:28.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.231+0000 7fed8e79c640 1 -- 172.21.15.98:0/289408159 shutdown_connections 2024-09-19T08:46:28.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.231+0000 7fed8e79c640 1 --2- 172.21.15.98:0/289408159 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed880b6f60 0x7fed8806d1d0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.231+0000 7fed8e79c640 1 -- 172.21.15.98:0/289408159 >> 172.21.15.98:0/289408159 conn(0x7fed88073170 msgr2=0x7fed88073580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:28.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.231+0000 7fed8e79c640 1 -- 172.21.15.98:0/289408159 shutdown_connections 2024-09-19T08:46:28.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.231+0000 7fed8e79c640 1 -- 172.21.15.98:0/289408159 wait complete. 2024-09-19T08:46:28.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.231+0000 7fed8e79c640 1 Processor -- start 2024-09-19T08:46:28.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.231+0000 7fed8e79c640 1 -- start start 2024-09-19T08:46:28.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.231+0000 7fed8e79c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed88074c70 0x7fed8815c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:28.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.231+0000 7fed8e79c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed880b0280 0x7fed8815dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:28.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.231+0000 7fed8e79c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed880b6f60 0x7fed8815e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:28.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.231+0000 7fed8e79c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fed880af670 con 0x7fed88074c70 2024-09-19T08:46:28.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.231+0000 7fed8e79c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fed880af4f0 con 0x7fed880b6f60 2024-09-19T08:46:28.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.231+0000 7fed8e79c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fed8816a390 con 0x7fed880b0280 2024-09-19T08:46:28.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.231+0000 7fed87fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed88074c70 0x7fed8815c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:28.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.231+0000 7fed87fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed88074c70 0x7fed8815c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:43204/0 (socket says 172.21.15.98:43204) 2024-09-19T08:46:28.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.231+0000 7fed87fff640 1 -- 172.21.15.98:0/1965823418 learned_addr learned my addr 172.21.15.98:0/1965823418 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:28.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.231+0000 7fed877fe640 1 --2- 172.21.15.98:0/1965823418 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed880b0280 0x7fed8815dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:28.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.231+0000 7fed87fff640 1 -- 172.21.15.98:0/1965823418 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed880b0280 msgr2=0x7fed8815dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:28.237 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.231+0000 7fed8cd39640 1 --2- 172.21.15.98:0/1965823418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed880b6f60 0x7fed8815e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:28.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.231+0000 7fed87fff640 1 --2- 172.21.15.98:0/1965823418 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed880b0280 0x7fed8815dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.231+0000 7fed87fff640 1 -- 172.21.15.98:0/1965823418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed880b6f60 msgr2=0x7fed8815e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:28.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.231+0000 7fed87fff640 1 --2- 172.21.15.98:0/1965823418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed880b6f60 0x7fed8815e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.231+0000 7fed87fff640 1 -- 172.21.15.98:0/1965823418 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed880cb8a0 con 0x7fed88074c70 2024-09-19T08:46:28.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.231+0000 7fed87fff640 1 --2- 172.21.15.98:0/1965823418 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed88074c70 0x7fed8815c270 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fed7800c420 tx=0x7fed7800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:28.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.231+0000 7fed84ff9640 1 -- 172.21.15.98:0/1965823418 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed780074e0 con 0x7fed88074c70 2024-09-19T08:46:28.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.235+0000 7fed84ff9640 1 -- 172.21.15.98:0/1965823418 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed78007680 con 0x7fed88074c70 2024-09-19T08:46:28.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.235+0000 7fed8e79c640 1 -- 172.21.15.98:0/1965823418 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed880cbb80 con 0x7fed88074c70 2024-09-19T08:46:28.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.235+0000 7fed84ff9640 1 -- 172.21.15.98:0/1965823418 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78004a30 con 0x7fed88074c70 2024-09-19T08:46:28.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.235+0000 7fed8e79c640 1 -- 172.21.15.98:0/1965823418 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed880ce8b0 con 0x7fed88074c70 2024-09-19T08:46:28.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.235+0000 7fed8e79c640 1 -- 172.21.15.98:0/1965823418 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed8806d1d0 con 0x7fed88074c70 2024-09-19T08:46:28.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.235+0000 7fed84ff9640 1 -- 172.21.15.98:0/1965823418 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7fed7801a020 con 0x7fed88074c70 2024-09-19T08:46:28.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.235+0000 7fed84ff9640 1 --2- 172.21.15.98:0/1965823418 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fed740680d0 0x7fed7406a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:28.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.235+0000 7fed84ff9640 1 -- 172.21.15.98:0/1965823418 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5737+0+0 (secure 0 0 0) 0x7fed78086670 con 0x7fed88074c70 2024-09-19T08:46:28.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.235+0000 7fed877fe640 1 --2- 172.21.15.98:0/1965823418 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fed740680d0 0x7fed7406a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:28.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.235+0000 7fed877fe640 1 --2- 172.21.15.98:0/1965823418 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fed740680d0 0x7fed7406a590 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fed88069910 tx=0x7fed7c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:28.241 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.239+0000 7fed84ff9640 1 -- 172.21.15.98:0/1965823418 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fed780562b0 con 0x7fed88074c70 2024-09-19T08:46:28.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.423+0000 7fed8e79c640 1 -- 172.21.15.98:0/1965823418 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fed88072060 con 0x7fed88074c70 2024-09-19T08:46:28.422 INFO:tasks.ceph.mon.a.smithi098.stderr:2024-09-19T08:46:28.423+0000 7f5e49017640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T08:46:28.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.427+0000 7fed84ff9640 1 -- 172.21.15.98:0/1965823418 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v84) ==== 161+0+0 (secure 0 0 0) 0x7fed78055bb0 con 0x7fed88074c70 2024-09-19T08:46:28.424 INFO:teuthology.orchestra.run.smithi098.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T08:46:28.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.431+0000 7fed8e79c640 1 -- 172.21.15.98:0/1965823418 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fed740680d0 msgr2=0x7fed7406a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:28.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.431+0000 7fed8e79c640 1 --2- 172.21.15.98:0/1965823418 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fed740680d0 0x7fed7406a590 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fed88069910 tx=0x7fed7c008040 comp rx=0 tx=0).stop 2024-09-19T08:46:28.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.431+0000 7fed8e79c640 1 -- 172.21.15.98:0/1965823418 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed88074c70 msgr2=0x7fed8815c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:28.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.431+0000 7fed8e79c640 1 --2- 172.21.15.98:0/1965823418 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed88074c70 0x7fed8815c270 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fed7800c420 tx=0x7fed7800c8e0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.431+0000 7fed8e79c640 1 -- 172.21.15.98:0/1965823418 shutdown_connections 2024-09-19T08:46:28.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.431+0000 7fed8e79c640 1 --2- 172.21.15.98:0/1965823418 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fed740680d0 0x7fed7406a590 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.431+0000 7fed8e79c640 1 --2- 172.21.15.98:0/1965823418 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed880b6f60 0x7fed8815e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.431+0000 7fed8e79c640 1 --2- 172.21.15.98:0/1965823418 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed880b0280 0x7fed8815dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.431+0000 7fed8e79c640 1 --2- 172.21.15.98:0/1965823418 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed88074c70 0x7fed8815c270 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.431+0000 7fed8e79c640 1 -- 172.21.15.98:0/1965823418 >> 172.21.15.98:0/1965823418 conn(0x7fed88073170 msgr2=0x7fed88158690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:28.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.431+0000 7fed8e79c640 1 -- 172.21.15.98:0/1965823418 shutdown_connections 2024-09-19T08:46:28.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.431+0000 7fed8e79c640 1 -- 172.21.15.98:0/1965823418 wait complete. 2024-09-19T08:46:28.444 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T08:46:28.490 INFO:tasks.ceph.mds.a.smithi098.stdout:starting mds.a at 2024-09-19T08:46:28.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.555+0000 7f4749bad640 1 Processor -- start 2024-09-19T08:46:28.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.555+0000 7f4749bad640 1 -- start start 2024-09-19T08:46:28.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.555+0000 7f4749bad640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f474406c3a0 0x7f47440713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:28.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.555+0000 7f4749bad640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f47440718f0 0x7f47440b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:28.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.555+0000 7f4749bad640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4744073530 con 0x7f4744145b70 2024-09-19T08:46:28.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.555+0000 7f4749bad640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f4744073810 con 0x7f47440718f0 2024-09-19T08:46:28.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.555+0000 7f4749bad640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f4744073af0 con 0x7f474406c3a0 2024-09-19T08:46:28.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.555+0000 7f4743fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f47440718f0 0x7f47440b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:28.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.555+0000 7f47437fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f4744145b70 0x7f474406bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36336/0 (socket says 172.21.15.98:36336) 2024-09-19T08:46:28.556 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.555+0000 7f47437fe640 1 -- 172.21.15.98:0/3341090696 learned_addr learned my addr 172.21.15.98:0/3341090696 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:28.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.559+0000 7f4743fff640 1 -- 172.21.15.98:0/3341090696 >> v2:172.21.15.139:3300/0 conn(0x7f474406c3a0 msgr2=0x7f47440713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:28.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.559+0000 7f4742ffd640 1 --2- 172.21.15.98:0/3341090696 >> v2:172.21.15.139:3300/0 conn(0x7f474406c3a0 0x7f47440713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:28.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.559+0000 7f47427fc640 1 -- 172.21.15.98:0/3341090696 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1839482574 0 0) 0x7f4744073530 con 0x7f4744145b70 2024-09-19T08:46:28.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.559+0000 7f4743fff640 1 --2- 172.21.15.98:0/3341090696 >> v2:172.21.15.139:3300/0 conn(0x7f474406c3a0 0x7f47440713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.559+0000 7f4743fff640 1 -- 172.21.15.98:0/3341090696 >> v1:172.21.15.98:6789/0 conn(0x7f4744145b70 legacy=0x7f474406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:28.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.559+0000 7f4743fff640 1 -- 172.21.15.98:0/3341090696 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4744073dd0 con 0x7f47440718f0 2024-09-19T08:46:28.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.559+0000 7f4742ffd640 1 --2- 172.21.15.98:0/3341090696 >> v2:172.21.15.139:3300/0 conn(0x7f474406c3a0 0x7f47440713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:28.558 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.559+0000 7f4743fff640 1 --2- 172.21.15.98:0/3341090696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47440718f0 0x7f47440b3410 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f4744074420 tx=0x7f473401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532d1e5c21953682 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:28.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.559+0000 7f47427fc640 1 -- 172.21.15.98:0/3341090696 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47340b0070 con 0x7f47440718f0 2024-09-19T08:46:28.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.559+0000 7f47427fc640 1 -- 172.21.15.98:0/3341090696 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47340ab600 con 0x7f47440718f0 2024-09-19T08:46:28.559 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.559+0000 7f47427fc640 1 -- 172.21.15.98:0/3341090696 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47340ab9d0 con 0x7f47440718f0 2024-09-19T08:46:28.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.559+0000 7f4749bad640 1 -- 172.21.15.98:0/3341090696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47440718f0 msgr2=0x7f47440b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:28.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.559+0000 7f4749bad640 1 --2- 172.21.15.98:0/3341090696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47440718f0 0x7f47440b3410 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f4744074420 tx=0x7f473401c3f0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.559+0000 7f4749bad640 1 -- 172.21.15.98:0/3341090696 shutdown_connections 2024-09-19T08:46:28.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.559+0000 7f4749bad640 1 --2- 172.21.15.98:0/3341090696 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f47440718f0 0x7f47440b3410 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.559+0000 7f4749bad640 1 --2- 172.21.15.98:0/3341090696 >> v2:172.21.15.139:3300/0 conn(0x7f474406c3a0 0x7f47440713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.559+0000 7f4749bad640 1 -- 172.21.15.98:0/3341090696 >> 172.21.15.98:0/3341090696 conn(0x7f47440b2010 msgr2=0x7f47440b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:28.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.559+0000 7f4749bad640 1 -- 172.21.15.98:0/3341090696 shutdown_connections 2024-09-19T08:46:28.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.559+0000 7f4749bad640 1 -- 172.21.15.98:0/3341090696 wait complete. 2024-09-19T08:46:28.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.563+0000 7f4749bad640 1 Processor -- start 2024-09-19T08:46:28.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.563+0000 7f4749bad640 1 -- start start 2024-09-19T08:46:28.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.563+0000 7f4749bad640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f474406c3a0 0x7f474415b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:28.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.563+0000 7f4749bad640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47440718f0 0x7f474415e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:28.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.563+0000 7f4749bad640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4744145b70 0x7f47441f4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:28.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.563+0000 7f4749bad640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f47440b44f0 con 0x7f4744145b70 2024-09-19T08:46:28.563 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.563+0000 7f4749bad640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f47440b4370 con 0x7f47440718f0 2024-09-19T08:46:28.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.563+0000 7f4749bad640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f47440b4670 con 0x7f474406c3a0 2024-09-19T08:46:28.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.563+0000 7f4743fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4744145b70 0x7f47441f4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:28.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.563+0000 7f4743fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4744145b70 0x7f47441f4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:43238/0 (socket says 172.21.15.98:43238) 2024-09-19T08:46:28.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.563+0000 7f4743fff640 1 -- 172.21.15.98:0/1694504703 learned_addr learned my addr 172.21.15.98:0/1694504703 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:28.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.563+0000 7f4743fff640 1 -- 172.21.15.98:0/1694504703 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f474406c3a0 msgr2=0x7f474415b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:28.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.563+0000 7f4743fff640 1 --2- 172.21.15.98:0/1694504703 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f474406c3a0 0x7f474415b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.563+0000 7f4743fff640 1 -- 172.21.15.98:0/1694504703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47440718f0 msgr2=0x7f474415e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:28.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.563+0000 7f4743fff640 1 --2- 172.21.15.98:0/1694504703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47440718f0 0x7f474415e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:28.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.563+0000 7f4743fff640 1 -- 172.21.15.98:0/1694504703 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f474415bc80 con 0x7f4744145b70 2024-09-19T08:46:28.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.563+0000 7f4743fff640 1 --2- 172.21.15.98:0/1694504703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4744145b70 0x7f47441f4f10 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f47340ae040 tx=0x7f4734019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:28.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.563+0000 7f472bfff640 1 -- 172.21.15.98:0/1694504703 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47340b8020 con 0x7f4744145b70 2024-09-19T08:46:28.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.563+0000 7f472bfff640 1 -- 172.21.15.98:0/1694504703 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47340b4410 con 0x7f4744145b70 2024-09-19T08:46:28.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.563+0000 7f4749bad640 1 -- 172.21.15.98:0/1694504703 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f474415f1a0 con 0x7f4744145b70 2024-09-19T08:46:28.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.563+0000 7f472bfff640 1 -- 172.21.15.98:0/1694504703 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47340b0040 con 0x7f4744145b70 2024-09-19T08:46:28.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.563+0000 7f4749bad640 1 -- 172.21.15.98:0/1694504703 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f474415f760 con 0x7f4744145b70 2024-09-19T08:46:28.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.563+0000 7f4749bad640 1 -- 172.21.15.98:0/1694504703 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47440713b0 con 0x7f4744145b70 2024-09-19T08:46:28.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.567+0000 7f472bfff640 1 -- 172.21.15.98:0/1694504703 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f47340bb020 con 0x7f4744145b70 2024-09-19T08:46:28.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.567+0000 7f472bfff640 1 --2- 172.21.15.98:0/1694504703 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f47300680d0 0x7f473006a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:28.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.567+0000 7f472bfff640 1 -- 172.21.15.98:0/1694504703 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5823+0+0 (secure 0 0 0) 0x7f4734120d10 con 0x7f4744145b70 2024-09-19T08:46:28.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.567+0000 7f47437fe640 1 --2- 172.21.15.98:0/1694504703 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f47300680d0 0x7f473006a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:28.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.567+0000 7f47437fe640 1 --2- 172.21.15.98:0/1694504703 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f47300680d0 0x7f473006a590 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f47440afb50 tx=0x7f4738007970 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:28.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.571+0000 7f472bfff640 1 -- 172.21.15.98:0/1694504703 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f47340c2030 con 0x7f4744145b70 2024-09-19T08:46:28.764 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:28.767+0000 7f4749bad640 1 -- 172.21.15.98:0/1694504703 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f4744143cb0 con 0x7f4744145b70 2024-09-19T08:46:29.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.427+0000 7f472bfff640 1 -- 172.21.15.98:0/1694504703 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v85) ==== 104+0+0 (secure 0 0 0) 0x7f4734016070 con 0x7f4744145b70 2024-09-19T08:46:29.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.431+0000 7f4749bad640 1 -- 172.21.15.98:0/1694504703 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f47300680d0 msgr2=0x7f473006a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:29.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.431+0000 7f4749bad640 1 --2- 172.21.15.98:0/1694504703 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f47300680d0 0x7f473006a590 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f47440afb50 tx=0x7f4738007970 comp rx=0 tx=0).stop 2024-09-19T08:46:29.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.431+0000 7f4749bad640 1 -- 172.21.15.98:0/1694504703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4744145b70 msgr2=0x7f47441f4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:29.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.431+0000 7f4749bad640 1 --2- 172.21.15.98:0/1694504703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4744145b70 0x7f47441f4f10 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f47340ae040 tx=0x7f4734019190 comp rx=0 tx=0).stop 2024-09-19T08:46:29.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.431+0000 7f4749bad640 1 -- 172.21.15.98:0/1694504703 shutdown_connections 2024-09-19T08:46:29.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.431+0000 7f4749bad640 1 --2- 172.21.15.98:0/1694504703 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f47300680d0 0x7f473006a590 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:29.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.431+0000 7f4749bad640 1 --2- 172.21.15.98:0/1694504703 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4744145b70 0x7f47441f4f10 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:29.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.431+0000 7f4749bad640 1 --2- 172.21.15.98:0/1694504703 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f47440718f0 0x7f474415e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:29.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.431+0000 7f4749bad640 1 --2- 172.21.15.98:0/1694504703 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f474406c3a0 0x7f474415b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:29.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.431+0000 7f4749bad640 1 -- 172.21.15.98:0/1694504703 >> 172.21.15.98:0/1694504703 conn(0x7f47440b2010 msgr2=0x7f47440aff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:29.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.431+0000 7f4749bad640 1 -- 172.21.15.98:0/1694504703 shutdown_connections 2024-09-19T08:46:29.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.431+0000 7f4749bad640 1 -- 172.21.15.98:0/1694504703 wait complete. 2024-09-19T08:46:29.453 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T08:46:29.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.583+0000 7f363cb9f640 1 Processor -- start 2024-09-19T08:46:29.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.583+0000 7f363cb9f640 1 -- start start 2024-09-19T08:46:29.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.583+0000 7f363cb9f640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f363815c290 0x7f36380cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:29.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.583+0000 7f363cb9f640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36381608c0 con 0x7f36380cf700 2024-09-19T08:46:29.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.583+0000 7f363cb9f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3638160ba0 con 0x7f363806d4c0 2024-09-19T08:46:29.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.583+0000 7f363cb9f640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f3638160e80 con 0x7f363815c290 2024-09-19T08:46:29.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.583+0000 7f363659c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f363815c290 0x7f36380cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:29.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.583+0000 7f3635d9b640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f36380cf700 0x7f363815ff50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36346/0 (socket says 172.21.15.98:36346) 2024-09-19T08:46:29.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.583+0000 7f3635d9b640 1 -- 172.21.15.98:0/1977337508 learned_addr learned my addr 172.21.15.98:0/1977337508 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:29.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.583+0000 7f363559a640 1 -- 172.21.15.98:0/1977337508 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3852543249 0 0) 0x7f36381608c0 con 0x7f36380cf700 2024-09-19T08:46:29.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.583+0000 7f363559a640 1 -- 172.21.15.98:0/1977337508 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3620003530 con 0x7f36380cf700 2024-09-19T08:46:29.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.583+0000 7f363659c640 1 -- 172.21.15.98:0/1977337508 >> v1:172.21.15.139:6789/0 conn(0x7f363806d4c0 legacy=0x7f363806d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:29.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.583+0000 7f363659c640 1 -- 172.21.15.98:0/1977337508 >> v1:172.21.15.98:6789/0 conn(0x7f36380cf700 legacy=0x7f363815ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:29.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.583+0000 7f363659c640 1 -- 172.21.15.98:0/1977337508 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36380cd1a0 con 0x7f363815c290 2024-09-19T08:46:29.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.587+0000 7f363659c640 1 --2- 172.21.15.98:0/1977337508 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f363815c290 0x7f36380cd9a0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f36280097c0 tx=0x7f362801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc658bb06bd9878d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:29.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.587+0000 7f363559a640 1 -- 172.21.15.98:0/1977337508 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36280b0070 con 0x7f363815c290 2024-09-19T08:46:29.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.587+0000 7f363559a640 1 -- 172.21.15.98:0/1977337508 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36280ab600 con 0x7f363815c290 2024-09-19T08:46:29.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.587+0000 7f363559a640 1 -- 172.21.15.98:0/1977337508 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36280a4e60 con 0x7f363815c290 2024-09-19T08:46:29.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.587+0000 7f363cb9f640 1 -- 172.21.15.98:0/1977337508 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f363815c290 msgr2=0x7f36380cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:29.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.587+0000 7f363cb9f640 1 --2- 172.21.15.98:0/1977337508 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f363815c290 0x7f36380cd9a0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f36280097c0 tx=0x7f362801c3d0 comp rx=0 tx=0).stop 2024-09-19T08:46:29.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.587+0000 7f363cb9f640 1 -- 172.21.15.98:0/1977337508 shutdown_connections 2024-09-19T08:46:29.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.587+0000 7f363cb9f640 1 --2- 172.21.15.98:0/1977337508 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f363815c290 0x7f36380cd9a0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:29.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.587+0000 7f363cb9f640 1 -- 172.21.15.98:0/1977337508 >> 172.21.15.98:0/1977337508 conn(0x7f363806c090 msgr2=0x7f363806c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:29.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.587+0000 7f363cb9f640 1 -- 172.21.15.98:0/1977337508 shutdown_connections 2024-09-19T08:46:29.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.587+0000 7f363cb9f640 1 -- 172.21.15.98:0/1977337508 wait complete. 2024-09-19T08:46:29.587 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.591+0000 7f363cb9f640 1 Processor -- start 2024-09-19T08:46:29.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.591+0000 7f363cb9f640 1 -- start start 2024-09-19T08:46:29.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.591+0000 7f363cb9f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f363806d4c0 0x7f3638146830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:29.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.591+0000 7f363cb9f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36380cf700 0x7f3638149ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:29.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.591+0000 7f363cb9f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f363815c290 0x7f3638165e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:29.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.591+0000 7f363cb9f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3638160660 con 0x7f36380cf700 2024-09-19T08:46:29.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.591+0000 7f363cb9f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f36380b5690 con 0x7f363815c290 2024-09-19T08:46:29.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.591+0000 7f363cb9f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f363816a0d0 con 0x7f363806d4c0 2024-09-19T08:46:29.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.591+0000 7f3635d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36380cf700 0x7f3638149ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:29.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.591+0000 7f3635d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36380cf700 0x7f3638149ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:43250/0 (socket says 172.21.15.98:43250) 2024-09-19T08:46:29.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.591+0000 7f3635d9b640 1 -- 172.21.15.98:0/616396880 learned_addr learned my addr 172.21.15.98:0/616396880 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:29.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.591+0000 7f363659c640 1 --2- 172.21.15.98:0/616396880 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f363806d4c0 0x7f3638146830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:29.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.591+0000 7f3636d9d640 1 --2- 172.21.15.98:0/616396880 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f363815c290 0x7f3638165e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:29.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.591+0000 7f3635d9b640 1 -- 172.21.15.98:0/616396880 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f363806d4c0 msgr2=0x7f3638146830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:29.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.591+0000 7f3635d9b640 1 --2- 172.21.15.98:0/616396880 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f363806d4c0 0x7f3638146830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:29.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.591+0000 7f3635d9b640 1 -- 172.21.15.98:0/616396880 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f363815c290 msgr2=0x7f3638165e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:29.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.591+0000 7f3635d9b640 1 --2- 172.21.15.98:0/616396880 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f363815c290 0x7f3638165e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:29.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.591+0000 7f3635d9b640 1 -- 172.21.15.98:0/616396880 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36381666d0 con 0x7f36380cf700 2024-09-19T08:46:29.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.591+0000 7f363659c640 1 --2- 172.21.15.98:0/616396880 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f363806d4c0 0x7f3638146830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:29.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.591+0000 7f3636d9d640 1 --2- 172.21.15.98:0/616396880 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f363815c290 0x7f3638165e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:29.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.591+0000 7f3635d9b640 1 --2- 172.21.15.98:0/616396880 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36380cf700 0x7f3638149ac0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f362c00c4d0 tx=0x7f362c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:29.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.591+0000 7f3626ffd640 1 -- 172.21.15.98:0/616396880 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c017070 con 0x7f36380cf700 2024-09-19T08:46:29.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.591+0000 7f3626ffd640 1 -- 172.21.15.98:0/616396880 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f362c005cf0 con 0x7f36380cf700 2024-09-19T08:46:29.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.591+0000 7f363cb9f640 1 -- 172.21.15.98:0/616396880 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36381669b0 con 0x7f36380cf700 2024-09-19T08:46:29.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.591+0000 7f3626ffd640 1 -- 172.21.15.98:0/616396880 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c007270 con 0x7f36380cf700 2024-09-19T08:46:29.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.591+0000 7f363cb9f640 1 -- 172.21.15.98:0/616396880 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36381470c0 con 0x7f36380cf700 2024-09-19T08:46:29.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.591+0000 7f363cb9f640 1 -- 172.21.15.98:0/616396880 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3638146ed0 con 0x7f36380cf700 2024-09-19T08:46:29.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.591+0000 7f3626ffd640 1 -- 172.21.15.98:0/616396880 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f362c01a070 con 0x7f36380cf700 2024-09-19T08:46:29.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.591+0000 7f3626ffd640 1 --2- 172.21.15.98:0/616396880 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f36180680a0 0x7f361806a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:29.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.591+0000 7f3626ffd640 1 -- 172.21.15.98:0/616396880 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5823+0+0 (secure 0 0 0) 0x7f362c083960 con 0x7f36380cf700 2024-09-19T08:46:29.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.595+0000 7f363659c640 1 --2- 172.21.15.98:0/616396880 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f36180680a0 0x7f361806a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:29.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.595+0000 7f363659c640 1 --2- 172.21.15.98:0/616396880 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f36180680a0 0x7f361806a560 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f36280ad000 tx=0x7f3628008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:29.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.595+0000 7f3626ffd640 1 -- 172.21.15.98:0/616396880 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f362c055740 con 0x7f36380cf700 2024-09-19T08:46:29.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:29.763+0000 7f363cb9f640 1 -- 172.21.15.98:0/616396880 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f36380cd9a0 con 0x7f36380cf700 2024-09-19T08:46:30.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.438+0000 7f3626ffd640 1 -- 172.21.15.98:0/616396880 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v148) ==== 181+0+0 (secure 0 0 0) 0x7f362c059a40 con 0x7f36380cf700 2024-09-19T08:46:30.437 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_metadata' removed 2024-09-19T08:46:30.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.438+0000 7f363cb9f640 1 -- 172.21.15.98:0/616396880 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f36180680a0 msgr2=0x7f361806a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:30.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.438+0000 7f363cb9f640 1 --2- 172.21.15.98:0/616396880 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f36180680a0 0x7f361806a560 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f36280ad000 tx=0x7f3628008040 comp rx=0 tx=0).stop 2024-09-19T08:46:30.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.438+0000 7f363cb9f640 1 -- 172.21.15.98:0/616396880 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36380cf700 msgr2=0x7f3638149ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:30.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.438+0000 7f363cb9f640 1 --2- 172.21.15.98:0/616396880 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36380cf700 0x7f3638149ac0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f362c00c4d0 tx=0x7f362c00c990 comp rx=0 tx=0).stop 2024-09-19T08:46:30.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.438+0000 7f363cb9f640 1 -- 172.21.15.98:0/616396880 shutdown_connections 2024-09-19T08:46:30.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.438+0000 7f363cb9f640 1 --2- 172.21.15.98:0/616396880 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f36180680a0 0x7f361806a560 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:30.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.438+0000 7f363cb9f640 1 --2- 172.21.15.98:0/616396880 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f363815c290 0x7f3638165e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:30.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.438+0000 7f363cb9f640 1 --2- 172.21.15.98:0/616396880 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36380cf700 0x7f3638149ac0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:30.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.438+0000 7f363cb9f640 1 --2- 172.21.15.98:0/616396880 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f363806d4c0 0x7f3638146830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:30.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.438+0000 7f363cb9f640 1 -- 172.21.15.98:0/616396880 >> 172.21.15.98:0/616396880 conn(0x7f363806c090 msgr2=0x7f363815f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:30.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.438+0000 7f363cb9f640 1 -- 172.21.15.98:0/616396880 shutdown_connections 2024-09-19T08:46:30.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.438+0000 7f363cb9f640 1 -- 172.21.15.98:0/616396880 wait complete. 2024-09-19T08:46:30.464 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T08:46:30.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.594+0000 7f713f587640 1 Processor -- start 2024-09-19T08:46:30.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.594+0000 7f713f587640 1 -- start start 2024-09-19T08:46:30.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.594+0000 7f713f587640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f713815f6e0 0x7f713806d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:30.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.594+0000 7f713f587640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f713815fac0 0x7f713806da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:30.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.594+0000 7f713f587640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f713815baf0 con 0x7f7138073680 2024-09-19T08:46:30.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.594+0000 7f713f587640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f713815c8b0 con 0x7f713815f6e0 2024-09-19T08:46:30.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.594+0000 7f713f587640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f71380742d0 con 0x7f713815fac0 2024-09-19T08:46:30.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.594+0000 7f713d323640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f713815f6e0 0x7f713806d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:30.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.594+0000 7f713db24640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f7138073680 0x7f7138073a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36356/0 (socket says 172.21.15.98:36356) 2024-09-19T08:46:30.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.594+0000 7f713db24640 1 -- 172.21.15.98:0/2379923679 learned_addr learned my addr 172.21.15.98:0/2379923679 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:30.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.594+0000 7f713cb22640 1 --2- 172.21.15.98:0/2379923679 >> v2:172.21.15.139:3300/0 conn(0x7f713815fac0 0x7f713806da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:30.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.594+0000 7f713d323640 1 -- 172.21.15.98:0/2379923679 >> v2:172.21.15.139:3300/0 conn(0x7f713815fac0 msgr2=0x7f713806da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:30.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.594+0000 7f713d323640 1 --2- 172.21.15.98:0/2379923679 >> v2:172.21.15.139:3300/0 conn(0x7f713815fac0 0x7f713806da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:30.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.594+0000 7f713d323640 1 -- 172.21.15.98:0/2379923679 >> v1:172.21.15.98:6789/0 conn(0x7f7138073680 legacy=0x7f7138073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:30.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.594+0000 7f712ffff640 1 -- 172.21.15.98:0/2379923679 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3133941546 0 0) 0x7f713815baf0 con 0x7f7138073680 2024-09-19T08:46:30.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.594+0000 7f713d323640 1 -- 172.21.15.98:0/2379923679 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f713815bdd0 con 0x7f713815f6e0 2024-09-19T08:46:30.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.594+0000 7f713cb22640 1 --2- 172.21.15.98:0/2379923679 >> v2:172.21.15.139:3300/0 conn(0x7f713815fac0 0x7f713806da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:30.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.598+0000 7f713d323640 1 --2- 172.21.15.98:0/2379923679 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f713815f6e0 0x7f713806d4c0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f71380af7f0 tx=0x7f713401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0096e6196387e08 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:30.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.598+0000 7f712ffff640 1 -- 172.21.15.98:0/2379923679 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71340b0070 con 0x7f713815f6e0 2024-09-19T08:46:30.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.598+0000 7f712ffff640 1 -- 172.21.15.98:0/2379923679 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71340ab600 con 0x7f713815f6e0 2024-09-19T08:46:30.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.598+0000 7f712ffff640 1 -- 172.21.15.98:0/2379923679 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71340ab9d0 con 0x7f713815f6e0 2024-09-19T08:46:30.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.598+0000 7f713f587640 1 -- 172.21.15.98:0/2379923679 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f713815f6e0 msgr2=0x7f713806d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:30.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.598+0000 7f713f587640 1 --2- 172.21.15.98:0/2379923679 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f713815f6e0 0x7f713806d4c0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f71380af7f0 tx=0x7f713401c3f0 comp rx=0 tx=0).stop 2024-09-19T08:46:30.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.598+0000 7f713f587640 1 -- 172.21.15.98:0/2379923679 shutdown_connections 2024-09-19T08:46:30.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.598+0000 7f713f587640 1 --2- 172.21.15.98:0/2379923679 >> v2:172.21.15.139:3300/0 conn(0x7f713815fac0 0x7f713806da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:30.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.598+0000 7f713f587640 1 --2- 172.21.15.98:0/2379923679 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f713815f6e0 0x7f713806d4c0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:30.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.598+0000 7f713f587640 1 -- 172.21.15.98:0/2379923679 >> 172.21.15.98:0/2379923679 conn(0x7f713806c090 msgr2=0x7f713806c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:30.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.598+0000 7f713f587640 1 -- 172.21.15.98:0/2379923679 shutdown_connections 2024-09-19T08:46:30.599 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.598+0000 7f713f587640 1 -- 172.21.15.98:0/2379923679 wait complete. 2024-09-19T08:46:30.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.602+0000 7f713f587640 1 Processor -- start 2024-09-19T08:46:30.600 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.602+0000 7f713f587640 1 -- start start 2024-09-19T08:46:30.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.602+0000 7f713f587640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7138073680 0x7f7138149280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:30.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.602+0000 7f713f587640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f713815f6e0 0x7f7138153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:30.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.602+0000 7f713f587640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f713815fac0 0x7f7138154520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:30.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.602+0000 7f713f587640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f71380cefc0 con 0x7f713815fac0 2024-09-19T08:46:30.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.602+0000 7f713f587640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f71380cee40 con 0x7f713815f6e0 2024-09-19T08:46:30.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.602+0000 7f713f587640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f71380cf140 con 0x7f7138073680 2024-09-19T08:46:30.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.602+0000 7f713db24640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f713815fac0 0x7f7138154520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:30.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.602+0000 7f713db24640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f713815fac0 0x7f7138154520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:43266/0 (socket says 172.21.15.98:43266) 2024-09-19T08:46:30.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.602+0000 7f713db24640 1 -- 172.21.15.98:0/2106996235 learned_addr learned my addr 172.21.15.98:0/2106996235 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:30.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.602+0000 7f713cb22640 1 --2- 172.21.15.98:0/2106996235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f713815f6e0 0x7f7138153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:30.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.602+0000 7f713db24640 1 -- 172.21.15.98:0/2106996235 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7138073680 msgr2=0x7f7138149280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:30.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.602+0000 7f713d323640 1 --2- 172.21.15.98:0/2106996235 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7138073680 0x7f7138149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:30.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.602+0000 7f713db24640 1 --2- 172.21.15.98:0/2106996235 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7138073680 0x7f7138149280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:30.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.602+0000 7f713db24640 1 -- 172.21.15.98:0/2106996235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f713815f6e0 msgr2=0x7f7138153fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:30.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.602+0000 7f713db24640 1 --2- 172.21.15.98:0/2106996235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f713815f6e0 0x7f7138153fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:30.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.602+0000 7f713db24640 1 -- 172.21.15.98:0/2106996235 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71380b2450 con 0x7f713815fac0 2024-09-19T08:46:30.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.602+0000 7f713db24640 1 --2- 172.21.15.98:0/2106996235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f713815fac0 0x7f7138154520 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f713000c7b0 tx=0x7f713000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:30.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.602+0000 7f712dffb640 1 -- 172.21.15.98:0/2106996235 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7130018070 con 0x7f713815fac0 2024-09-19T08:46:30.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.602+0000 7f712dffb640 1 -- 172.21.15.98:0/2106996235 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f713000a640 con 0x7f713815fac0 2024-09-19T08:46:30.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.602+0000 7f713f587640 1 -- 172.21.15.98:0/2106996235 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7138153540 con 0x7f713815fac0 2024-09-19T08:46:30.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.602+0000 7f712dffb640 1 -- 172.21.15.98:0/2106996235 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713000aa30 con 0x7f713815fac0 2024-09-19T08:46:30.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.602+0000 7f713f587640 1 -- 172.21.15.98:0/2106996235 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71381537e0 con 0x7f713815fac0 2024-09-19T08:46:30.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.602+0000 7f713f587640 1 -- 172.21.15.98:0/2106996235 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71380cf530 con 0x7f713815fac0 2024-09-19T08:46:30.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.602+0000 7f712dffb640 1 -- 172.21.15.98:0/2106996235 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f713001b070 con 0x7f713815fac0 2024-09-19T08:46:30.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.606+0000 7f712dffb640 1 --2- 172.21.15.98:0/2106996235 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f711c0680a0 0x7f711c06a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:30.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.606+0000 7f712dffb640 1 -- 172.21.15.98:0/2106996235 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5385+0+0 (secure 0 0 0) 0x7f7130083c10 con 0x7f713815fac0 2024-09-19T08:46:30.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.606+0000 7f713d323640 1 --2- 172.21.15.98:0/2106996235 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f711c0680a0 0x7f711c06a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:30.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.606+0000 7f713d323640 1 --2- 172.21.15.98:0/2106996235 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f711c0680a0 0x7f711c06a560 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f71340ad000 tx=0x7f71340a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:30.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.610+0000 7f712dffb640 1 -- 172.21.15.98:0/2106996235 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f7130056040 con 0x7f713815fac0 2024-09-19T08:46:30.767 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:30.766+0000 7f713f587640 1 -- 172.21.15.98:0/2106996235 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7138073cc0 con 0x7f713815fac0 2024-09-19T08:46:31.445 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.446+0000 7f712dffb640 1 -- 172.21.15.98:0/2106996235 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v149) ==== 169+0+0 (secure 0 0 0) 0x7f7130055940 con 0x7f713815fac0 2024-09-19T08:46:31.445 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' removed 2024-09-19T08:46:31.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.450+0000 7f713f587640 1 -- 172.21.15.98:0/2106996235 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f711c0680a0 msgr2=0x7f711c06a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:31.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.450+0000 7f713f587640 1 --2- 172.21.15.98:0/2106996235 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f711c0680a0 0x7f711c06a560 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f71340ad000 tx=0x7f71340a6000 comp rx=0 tx=0).stop 2024-09-19T08:46:31.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.450+0000 7f713f587640 1 -- 172.21.15.98:0/2106996235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f713815fac0 msgr2=0x7f7138154520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:31.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.450+0000 7f713f587640 1 --2- 172.21.15.98:0/2106996235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f713815fac0 0x7f7138154520 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f713000c7b0 tx=0x7f713000cc70 comp rx=0 tx=0).stop 2024-09-19T08:46:31.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.450+0000 7f713f587640 1 -- 172.21.15.98:0/2106996235 shutdown_connections 2024-09-19T08:46:31.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.450+0000 7f713f587640 1 --2- 172.21.15.98:0/2106996235 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f711c0680a0 0x7f711c06a560 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:31.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.450+0000 7f713f587640 1 --2- 172.21.15.98:0/2106996235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f713815fac0 0x7f7138154520 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:31.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.450+0000 7f713f587640 1 --2- 172.21.15.98:0/2106996235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f713815f6e0 0x7f7138153fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:31.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.450+0000 7f713f587640 1 --2- 172.21.15.98:0/2106996235 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f7138073680 0x7f7138149280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:31.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.450+0000 7f713f587640 1 -- 172.21.15.98:0/2106996235 >> 172.21.15.98:0/2106996235 conn(0x7f713806c090 msgr2=0x7f71380b3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:31.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.450+0000 7f713f587640 1 -- 172.21.15.98:0/2106996235 shutdown_connections 2024-09-19T08:46:31.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.450+0000 7f713f587640 1 -- 172.21.15.98:0/2106996235 wait complete. 2024-09-19T08:46:31.476 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-19T08:46:31.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.602+0000 7f36f6367640 1 Processor -- start 2024-09-19T08:46:31.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.602+0000 7f36f6367640 1 -- start start 2024-09-19T08:46:31.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36f6367640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f36f006c3a0 0x7f36f0165f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:31.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36f6367640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36f0072ea0 con 0x7f36f00729f0 2024-09-19T08:46:31.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36f6367640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36f0073180 con 0x7f36f0144d10 2024-09-19T08:46:31.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36f6367640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f36f0073ab0 con 0x7f36f006c3a0 2024-09-19T08:46:31.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36e77fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f36f00729f0 0x7f36f006bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36362/0 (socket says 172.21.15.98:36362) 2024-09-19T08:46:31.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36e77fe640 1 -- 172.21.15.98:0/2494782283 learned_addr learned my addr 172.21.15.98:0/2494782283 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:31.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36f4904640 1 --2- 172.21.15.98:0/2494782283 >> v2:172.21.15.139:3300/0 conn(0x7f36f006c3a0 0x7f36f0165f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:31.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36ef7fe640 1 -- 172.21.15.98:0/2494782283 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 596673345 0 0) 0x7f36f0072ea0 con 0x7f36f00729f0 2024-09-19T08:46:31.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36ef7fe640 1 -- 172.21.15.98:0/2494782283 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f36dc003530 con 0x7f36f00729f0 2024-09-19T08:46:31.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36ef7fe640 1 -- 172.21.15.98:0/2494782283 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 33642839 0 0) 0x7f36dc003530 con 0x7f36f00729f0 2024-09-19T08:46:31.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36ef7fe640 1 -- 172.21.15.98:0/2494782283 >> v2:172.21.15.139:3300/0 conn(0x7f36f006c3a0 msgr2=0x7f36f0165f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:31.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36ef7fe640 1 --2- 172.21.15.98:0/2494782283 >> v2:172.21.15.139:3300/0 conn(0x7f36f006c3a0 0x7f36f0165f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:31.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36ef7fe640 1 -- 172.21.15.98:0/2494782283 >> v1:172.21.15.139:6789/0 conn(0x7f36f0144d10 legacy=0x7f36f00722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:31.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36ef7fe640 1 -- 172.21.15.98:0/2494782283 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f0073460 con 0x7f36f00729f0 2024-09-19T08:46:31.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36ef7fe640 1 -- 172.21.15.98:0/2494782283 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f36d8001680 con 0x7f36f00729f0 2024-09-19T08:46:31.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36ef7fe640 1 -- 172.21.15.98:0/2494782283 >> v1:172.21.15.98:6789/0 conn(0x7f36f00729f0 legacy=0x7f36f006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:31.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36ef7fe640 1 --2- 172.21.15.98:0/2494782283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36dc003a10 0x7f36dc003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:31.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36ef7fe640 1 --2- 172.21.15.98:0/2494782283 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36dc004810 0x7f36dc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:31.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36ef7fe640 1 --2- 172.21.15.98:0/2494782283 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36dc0051d0 0x7f36dc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:31.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36ef7fe640 1 -- 172.21.15.98:0/2494782283 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f36f0073ab0 con 0x7f36dc003a10 2024-09-19T08:46:31.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36ef7fe640 1 -- 172.21.15.98:0/2494782283 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc00c0d0 con 0x7f36dc0051d0 2024-09-19T08:46:31.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36ef7fe640 1 -- 172.21.15.98:0/2494782283 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f36dc00c2d0 con 0x7f36dc004810 2024-09-19T08:46:31.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36effff640 1 --2- 172.21.15.98:0/2494782283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36dc003a10 0x7f36dc003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:31.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36f4904640 1 --2- 172.21.15.98:0/2494782283 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36dc0051d0 0x7f36dc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:31.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36e77fe640 1 --2- 172.21.15.98:0/2494782283 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36dc004810 0x7f36dc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:31.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36effff640 1 -- 172.21.15.98:0/2494782283 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36dc004810 msgr2=0x7f36dc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:31.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36effff640 1 --2- 172.21.15.98:0/2494782283 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36dc004810 0x7f36dc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:31.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36effff640 1 -- 172.21.15.98:0/2494782283 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36dc0051d0 msgr2=0x7f36dc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:31.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36effff640 1 --2- 172.21.15.98:0/2494782283 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36dc0051d0 0x7f36dc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:31.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36effff640 1 -- 172.21.15.98:0/2494782283 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36dc003530 con 0x7f36dc003a10 2024-09-19T08:46:31.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36e77fe640 1 --2- 172.21.15.98:0/2494782283 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36dc004810 0x7f36dc004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:31.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36f4904640 1 --2- 172.21.15.98:0/2494782283 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36dc0051d0 0x7f36dc00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:31.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36f4904640 1 -- 172.21.15.98:0/2494782283 reap_dead start 2024-09-19T08:46:31.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36effff640 1 --2- 172.21.15.98:0/2494782283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36dc003a10 0x7f36dc003e30 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f36e0001780 tx=0x7f36e001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57df274497d2f11 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:31.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36ef7fe640 1 -- 172.21.15.98:0/2494782283 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e00013a0 con 0x7f36dc003a10 2024-09-19T08:46:31.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36ef7fe640 1 -- 172.21.15.98:0/2494782283 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36e00a4e40 con 0x7f36dc003a10 2024-09-19T08:46:31.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.606+0000 7f36ef7fe640 1 -- 172.21.15.98:0/2494782283 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e0005310 con 0x7f36dc003a10 2024-09-19T08:46:31.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.610+0000 7f36f6367640 1 -- 172.21.15.98:0/2494782283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36dc003a10 msgr2=0x7f36dc003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:31.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.610+0000 7f36f6367640 1 --2- 172.21.15.98:0/2494782283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36dc003a10 0x7f36dc003e30 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f36e0001780 tx=0x7f36e001c3c0 comp rx=0 tx=0).stop 2024-09-19T08:46:31.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.610+0000 7f36f6367640 1 -- 172.21.15.98:0/2494782283 shutdown_connections 2024-09-19T08:46:31.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.610+0000 7f36f6367640 1 --2- 172.21.15.98:0/2494782283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36dc003a10 0x7f36dc003e30 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:31.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.610+0000 7f36f6367640 1 -- 172.21.15.98:0/2494782283 >> 172.21.15.98:0/2494782283 conn(0x7f36f00b2010 msgr2=0x7f36f00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:31.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.610+0000 7f36f6367640 1 -- 172.21.15.98:0/2494782283 shutdown_connections 2024-09-19T08:46:31.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.610+0000 7f36f6367640 1 -- 172.21.15.98:0/2494782283 wait complete. 2024-09-19T08:46:31.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.610+0000 7f36f6367640 1 Processor -- start 2024-09-19T08:46:31.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.610+0000 7f36f6367640 1 -- start start 2024-09-19T08:46:31.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.610+0000 7f36f6367640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36dc003a10 0x7f36f015afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:31.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.610+0000 7f36f6367640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36f01fbe30 0x7f36f013da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:31.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.610+0000 7f36f6367640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36f01406f0 0x7f36f00ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:31.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.610+0000 7f36f6367640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f36f006c220 con 0x7f36dc003a10 2024-09-19T08:46:31.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.610+0000 7f36f6367640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f36f006c0a0 con 0x7f36f01fbe30 2024-09-19T08:46:31.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.610+0000 7f36f6367640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f36f00b4d30 con 0x7f36f01406f0 2024-09-19T08:46:31.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.610+0000 7f36effff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36dc003a10 0x7f36f015afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:31.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.610+0000 7f36effff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36dc003a10 0x7f36f015afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:43286/0 (socket says 172.21.15.98:43286) 2024-09-19T08:46:31.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.610+0000 7f36effff640 1 -- 172.21.15.98:0/947076848 learned_addr learned my addr 172.21.15.98:0/947076848 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:31.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.610+0000 7f36f4904640 1 --2- 172.21.15.98:0/947076848 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36f01406f0 0x7f36f00ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:31.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.610+0000 7f36effff640 1 -- 172.21.15.98:0/947076848 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36f01406f0 msgr2=0x7f36f00ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:31.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.610+0000 7f36e77fe640 1 --2- 172.21.15.98:0/947076848 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36f01fbe30 0x7f36f013da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:31.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.610+0000 7f36effff640 1 --2- 172.21.15.98:0/947076848 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36f01406f0 0x7f36f00ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:31.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.610+0000 7f36effff640 1 -- 172.21.15.98:0/947076848 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36f01fbe30 msgr2=0x7f36f013da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:31.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.610+0000 7f36effff640 1 --2- 172.21.15.98:0/947076848 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36f01fbe30 0x7f36f013da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:31.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.610+0000 7f36effff640 1 -- 172.21.15.98:0/947076848 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f015b500 con 0x7f36dc003a10 2024-09-19T08:46:31.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.610+0000 7f36f4904640 1 --2- 172.21.15.98:0/947076848 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36f01406f0 0x7f36f00ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:31.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.610+0000 7f36effff640 1 --2- 172.21.15.98:0/947076848 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36dc003a10 0x7f36f015afc0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f36e0005040 tx=0x7f36dc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:31.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.610+0000 7f36ed7fa640 1 -- 172.21.15.98:0/947076848 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e0007030 con 0x7f36dc003a10 2024-09-19T08:46:31.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.610+0000 7f36ed7fa640 1 -- 172.21.15.98:0/947076848 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36e00071d0 con 0x7f36dc003a10 2024-09-19T08:46:31.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.610+0000 7f36ed7fa640 1 -- 172.21.15.98:0/947076848 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e00073a0 con 0x7f36dc003a10 2024-09-19T08:46:31.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.610+0000 7f36f6367640 1 -- 172.21.15.98:0/947076848 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36f013f650 con 0x7f36dc003a10 2024-09-19T08:46:31.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.610+0000 7f36f6367640 1 -- 172.21.15.98:0/947076848 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36f013fc10 con 0x7f36dc003a10 2024-09-19T08:46:31.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.614+0000 7f36f6367640 1 -- 172.21.15.98:0/947076848 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36f00b50c0 con 0x7f36dc003a10 2024-09-19T08:46:31.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.614+0000 7f36ed7fa640 1 -- 172.21.15.98:0/947076848 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f36e00bb020 con 0x7f36dc003a10 2024-09-19T08:46:31.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.614+0000 7f36ed7fa640 1 --2- 172.21.15.98:0/947076848 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f36d00680a0 0x7f36d006a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:31.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.614+0000 7f36ed7fa640 1 -- 172.21.15.98:0/947076848 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4971+0+0 (secure 0 0 0) 0x7f36e011d5a0 con 0x7f36dc003a10 2024-09-19T08:46:31.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.614+0000 7f36e77fe640 1 --2- 172.21.15.98:0/947076848 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f36d00680a0 0x7f36d006a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:31.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.614+0000 7f36e77fe640 1 --2- 172.21.15.98:0/947076848 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f36d00680a0 0x7f36d006a560 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f36d8000c10 tx=0x7f36d8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:31.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.618+0000 7f36ed7fa640 1 -- 172.21.15.98:0/947076848 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f36e00c2020 con 0x7f36dc003a10 2024-09-19T08:46:31.779 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.778+0000 7f36f6367640 1 -- 172.21.15.98:0/947076848 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f36f00b15c0 con 0x7f36dc003a10 2024-09-19T08:46:31.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.786+0000 7f36ed7fa640 1 -- 172.21.15.98:0/947076848 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) ==== 104+0+0 (secure 0 0 0) 0x7f36f015b500 con 0x7f36dc003a10 2024-09-19T08:46:31.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.786+0000 7f36f6367640 1 -- 172.21.15.98:0/947076848 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f36d00680a0 msgr2=0x7f36d006a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:31.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.786+0000 7f36f6367640 1 --2- 172.21.15.98:0/947076848 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f36d00680a0 0x7f36d006a560 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f36d8000c10 tx=0x7f36d8009040 comp rx=0 tx=0).stop 2024-09-19T08:46:31.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.786+0000 7f36f6367640 1 -- 172.21.15.98:0/947076848 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36dc003a10 msgr2=0x7f36f015afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:31.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.786+0000 7f36f6367640 1 --2- 172.21.15.98:0/947076848 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36dc003a10 0x7f36f015afc0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f36e0005040 tx=0x7f36dc005840 comp rx=0 tx=0).stop 2024-09-19T08:46:31.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.786+0000 7f36f6367640 1 -- 172.21.15.98:0/947076848 shutdown_connections 2024-09-19T08:46:31.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.786+0000 7f36f6367640 1 --2- 172.21.15.98:0/947076848 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f36d00680a0 0x7f36d006a560 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:31.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.786+0000 7f36f6367640 1 --2- 172.21.15.98:0/947076848 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f36f01406f0 0x7f36f00ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:31.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.786+0000 7f36f6367640 1 --2- 172.21.15.98:0/947076848 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36f01fbe30 0x7f36f013da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:31.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.786+0000 7f36f6367640 1 --2- 172.21.15.98:0/947076848 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f36dc003a10 0x7f36f015afc0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:31.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.786+0000 7f36f6367640 1 -- 172.21.15.98:0/947076848 >> 172.21.15.98:0/947076848 conn(0x7f36f00b2010 msgr2=0x7f36f00b33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:31.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.790+0000 7f36f6367640 1 -- 172.21.15.98:0/947076848 shutdown_connections 2024-09-19T08:46:31.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.790+0000 7f36f6367640 1 -- 172.21.15.98:0/947076848 wait complete. 2024-09-19T08:46:31.805 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1' 2024-09-19T08:46:31.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.930+0000 7fb378dfc640 1 Processor -- start 2024-09-19T08:46:31.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.930+0000 7fb378dfc640 1 -- start start 2024-09-19T08:46:31.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.930+0000 7fb378dfc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3740733b0 0x7fb3740cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:31.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.930+0000 7fb378dfc640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb37406d310 con 0x7fb37415c550 2024-09-19T08:46:31.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.930+0000 7fb378dfc640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb37406d5f0 con 0x7fb37406c3a0 2024-09-19T08:46:31.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.930+0000 7fb378dfc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb37406d8d0 con 0x7fb3740733b0 2024-09-19T08:46:31.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.930+0000 7fb37259c640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fb37415c550 0x7fb37406bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36366/0 (socket says 172.21.15.98:36366) 2024-09-19T08:46:31.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.930+0000 7fb37259c640 1 -- 172.21.15.98:0/2561335282 learned_addr learned my addr 172.21.15.98:0/2561335282 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:31.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.930+0000 7fb372d9d640 1 --2- 172.21.15.98:0/2561335282 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3740733b0 0x7fb3740cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:31.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.930+0000 7fb371d9b640 1 -- 172.21.15.98:0/2561335282 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3129716371 0 0) 0x7fb37406d310 con 0x7fb37415c550 2024-09-19T08:46:31.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.930+0000 7fb371d9b640 1 -- 172.21.15.98:0/2561335282 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb360003530 con 0x7fb37415c550 2024-09-19T08:46:31.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.930+0000 7fb371d9b640 1 -- 172.21.15.98:0/2561335282 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1541489657 0 0) 0x7fb360003530 con 0x7fb37415c550 2024-09-19T08:46:31.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.930+0000 7fb371d9b640 1 -- 172.21.15.98:0/2561335282 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3740733b0 msgr2=0x7fb3740cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:31.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.930+0000 7fb371d9b640 1 --2- 172.21.15.98:0/2561335282 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3740733b0 0x7fb3740cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:31.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.930+0000 7fb371d9b640 1 -- 172.21.15.98:0/2561335282 >> v1:172.21.15.139:6789/0 conn(0x7fb37406c3a0 legacy=0x7fb3740cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:31.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.930+0000 7fb371d9b640 1 -- 172.21.15.98:0/2561335282 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37406dbb0 con 0x7fb37415c550 2024-09-19T08:46:31.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.930+0000 7fb371d9b640 1 -- 172.21.15.98:0/2561335282 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fb364001680 con 0x7fb37415c550 2024-09-19T08:46:31.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.930+0000 7fb371d9b640 1 -- 172.21.15.98:0/2561335282 >> v1:172.21.15.98:6789/0 conn(0x7fb37415c550 legacy=0x7fb37406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:31.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.930+0000 7fb371d9b640 1 --2- 172.21.15.98:0/2561335282 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3600039f0 0x7fb360003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:31.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.930+0000 7fb371d9b640 1 --2- 172.21.15.98:0/2561335282 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3740733b0 0x7fb360004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:31.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.930+0000 7fb371d9b640 1 --2- 172.21.15.98:0/2561335282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb360004d50 0x7fb3600051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:31.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.930+0000 7fb371d9b640 1 -- 172.21.15.98:0/2561335282 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb37406d8d0 con 0x7fb360004d50 2024-09-19T08:46:31.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.930+0000 7fb371d9b640 1 -- 172.21.15.98:0/2561335282 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb36000bab0 con 0x7fb3600039f0 2024-09-19T08:46:31.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.930+0000 7fb371d9b640 1 -- 172.21.15.98:0/2561335282 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb36000bcb0 con 0x7fb3740733b0 2024-09-19T08:46:31.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.930+0000 7fb372d9d640 1 --2- 172.21.15.98:0/2561335282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb360004d50 0x7fb3600051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:31.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.930+0000 7fb37259c640 1 --2- 172.21.15.98:0/2561335282 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3600039f0 0x7fb360003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:31.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.930+0000 7fb369d9b640 1 --2- 172.21.15.98:0/2561335282 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3740733b0 0x7fb360004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:31.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.930+0000 7fb372d9d640 1 -- 172.21.15.98:0/2561335282 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3740733b0 msgr2=0x7fb360004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:31.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.930+0000 7fb372d9d640 1 --2- 172.21.15.98:0/2561335282 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3740733b0 0x7fb360004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:31.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.930+0000 7fb372d9d640 1 -- 172.21.15.98:0/2561335282 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3600039f0 msgr2=0x7fb360003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:31.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.930+0000 7fb372d9d640 1 --2- 172.21.15.98:0/2561335282 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3600039f0 0x7fb360003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:31.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.930+0000 7fb372d9d640 1 -- 172.21.15.98:0/2561335282 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb360003530 con 0x7fb360004d50 2024-09-19T08:46:31.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.930+0000 7fb37259c640 1 --2- 172.21.15.98:0/2561335282 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb3600039f0 0x7fb360003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:31.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.934+0000 7fb372d9d640 1 --2- 172.21.15.98:0/2561335282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb360004d50 0x7fb3600051d0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fb36c009f90 tx=0x7fb36c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b78288f157d1f3bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:31.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.934+0000 7fb371d9b640 1 -- 172.21.15.98:0/2561335282 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c0b2070 con 0x7fb360004d50 2024-09-19T08:46:31.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.934+0000 7fb371d9b640 1 -- 172.21.15.98:0/2561335282 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb36c0a5c50 con 0x7fb360004d50 2024-09-19T08:46:31.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.934+0000 7fb371d9b640 1 -- 172.21.15.98:0/2561335282 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c009c10 con 0x7fb360004d50 2024-09-19T08:46:31.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.934+0000 7fb378dfc640 1 -- 172.21.15.98:0/2561335282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb360004d50 msgr2=0x7fb3600051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:31.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.934+0000 7fb378dfc640 1 --2- 172.21.15.98:0/2561335282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb360004d50 0x7fb3600051d0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fb36c009f90 tx=0x7fb36c01cc40 comp rx=0 tx=0).stop 2024-09-19T08:46:31.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.934+0000 7fb372d9d640 1 -- 172.21.15.98:0/2561335282 reap_dead start 2024-09-19T08:46:31.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.934+0000 7fb378dfc640 1 -- 172.21.15.98:0/2561335282 shutdown_connections 2024-09-19T08:46:31.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.934+0000 7fb378dfc640 1 -- 172.21.15.98:0/2561335282 >> 172.21.15.98:0/2561335282 conn(0x7fb3740b2010 msgr2=0x7fb3740b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:31.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.934+0000 7fb378dfc640 1 -- 172.21.15.98:0/2561335282 shutdown_connections 2024-09-19T08:46:31.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.934+0000 7fb378dfc640 1 -- 172.21.15.98:0/2561335282 wait complete. 2024-09-19T08:46:31.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.934+0000 7fb378dfc640 1 Processor -- start 2024-09-19T08:46:31.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.938+0000 7fb378dfc640 1 -- start start 2024-09-19T08:46:31.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.938+0000 7fb378dfc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb360004d50 0x7fb3741f1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:31.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.938+0000 7fb378dfc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3741472e0 0x7fb3741547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:31.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.938+0000 7fb378dfc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb37415f090 0x7fb37415f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:31.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.938+0000 7fb378dfc640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb3740cfc50 con 0x7fb37415f090 2024-09-19T08:46:31.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.938+0000 7fb378dfc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb3740cfad0 con 0x7fb360004d50 2024-09-19T08:46:31.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.938+0000 7fb378dfc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb3740cfdd0 con 0x7fb3741472e0 2024-09-19T08:46:31.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.938+0000 7fb372d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb37415f090 0x7fb37415f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:31.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.938+0000 7fb372d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb37415f090 0x7fb37415f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:43296/0 (socket says 172.21.15.98:43296) 2024-09-19T08:46:31.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.938+0000 7fb372d9d640 1 -- 172.21.15.98:0/916206250 learned_addr learned my addr 172.21.15.98:0/916206250 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:31.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.938+0000 7fb369d9b640 1 --2- 172.21.15.98:0/916206250 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3741472e0 0x7fb3741547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:31.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.938+0000 7fb37259c640 1 --2- 172.21.15.98:0/916206250 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb360004d50 0x7fb3741f1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:31.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.938+0000 7fb372d9d640 1 -- 172.21.15.98:0/916206250 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3741472e0 msgr2=0x7fb3741547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:31.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.938+0000 7fb372d9d640 1 --2- 172.21.15.98:0/916206250 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3741472e0 0x7fb3741547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:31.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.938+0000 7fb372d9d640 1 -- 172.21.15.98:0/916206250 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb360004d50 msgr2=0x7fb3741f1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:31.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.938+0000 7fb372d9d640 1 --2- 172.21.15.98:0/916206250 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb360004d50 0x7fb3741f1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:31.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.938+0000 7fb372d9d640 1 -- 172.21.15.98:0/916206250 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb374155120 con 0x7fb37415f090 2024-09-19T08:46:31.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.938+0000 7fb372d9d640 1 --2- 172.21.15.98:0/916206250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb37415f090 0x7fb37415f510 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fb36c0b0040 tx=0x7fb36c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:31.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.938+0000 7fb36b7fe640 1 -- 172.21.15.98:0/916206250 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c0bd070 con 0x7fb37415f090 2024-09-19T08:46:31.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.938+0000 7fb36b7fe640 1 -- 172.21.15.98:0/916206250 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb36c0a5a50 con 0x7fb37415f090 2024-09-19T08:46:31.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.938+0000 7fb378dfc640 1 -- 172.21.15.98:0/916206250 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb374155400 con 0x7fb37415f090 2024-09-19T08:46:31.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.938+0000 7fb36b7fe640 1 -- 172.21.15.98:0/916206250 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c0b2040 con 0x7fb37415f090 2024-09-19T08:46:31.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.938+0000 7fb378dfc640 1 -- 172.21.15.98:0/916206250 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb37415cd90 con 0x7fb37415f090 2024-09-19T08:46:31.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.938+0000 7fb378dfc640 1 -- 172.21.15.98:0/916206250 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3740cf3c0 con 0x7fb37415f090 2024-09-19T08:46:31.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.938+0000 7fb36b7fe640 1 -- 172.21.15.98:0/916206250 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7fb36c0c0070 con 0x7fb37415f090 2024-09-19T08:46:31.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.938+0000 7fb36b7fe640 1 --2- 172.21.15.98:0/916206250 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb3540680d0 0x7fb35406a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:31.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.938+0000 7fb36b7fe640 1 -- 172.21.15.98:0/916206250 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4971+0+0 (secure 0 0 0) 0x7fb36c11e1e0 con 0x7fb37415f090 2024-09-19T08:46:31.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.938+0000 7fb37259c640 1 --2- 172.21.15.98:0/916206250 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb3540680d0 0x7fb35406a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:31.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.942+0000 7fb37259c640 1 --2- 172.21.15.98:0/916206250 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb3540680d0 0x7fb35406a590 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb364000c10 tx=0x7fb364009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:31.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:31.942+0000 7fb36b7fe640 1 -- 172.21.15.98:0/916206250 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb36c016950 con 0x7fb37415f090 2024-09-19T08:46:32.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.166+0000 7fb378dfc640 1 -- 172.21.15.98:0/916206250 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]} v 0) -- 0x7fb37406be10 con 0x7fb37415f090 2024-09-19T08:46:32.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.446+0000 7fb36b7fe640 1 -- 172.21.15.98:0/916206250 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]}]=0 v429) ==== 149+0+0 (secure 0 0 0) 0x7fb36c0a5c20 con 0x7fb37415f090 2024-09-19T08:46:32.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.450+0000 7fb378dfc640 1 -- 172.21.15.98:0/916206250 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb3540680d0 msgr2=0x7fb35406a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:32.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.450+0000 7fb378dfc640 1 --2- 172.21.15.98:0/916206250 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb3540680d0 0x7fb35406a590 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb364000c10 tx=0x7fb364009040 comp rx=0 tx=0).stop 2024-09-19T08:46:32.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.450+0000 7fb378dfc640 1 -- 172.21.15.98:0/916206250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb37415f090 msgr2=0x7fb37415f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:32.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.450+0000 7fb378dfc640 1 --2- 172.21.15.98:0/916206250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb37415f090 0x7fb37415f510 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fb36c0b0040 tx=0x7fb36c009750 comp rx=0 tx=0).stop 2024-09-19T08:46:32.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.450+0000 7fb378dfc640 1 -- 172.21.15.98:0/916206250 shutdown_connections 2024-09-19T08:46:32.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.450+0000 7fb378dfc640 1 --2- 172.21.15.98:0/916206250 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb3540680d0 0x7fb35406a590 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.450+0000 7fb378dfc640 1 --2- 172.21.15.98:0/916206250 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb37415f090 0x7fb37415f510 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.450+0000 7fb378dfc640 1 --2- 172.21.15.98:0/916206250 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb3741472e0 0x7fb3741547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.450+0000 7fb378dfc640 1 --2- 172.21.15.98:0/916206250 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb360004d50 0x7fb3741f1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.450+0000 7fb378dfc640 1 -- 172.21.15.98:0/916206250 >> 172.21.15.98:0/916206250 conn(0x7fb3740b2010 msgr2=0x7fb3740ce1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:32.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.450+0000 7fb378dfc640 1 -- 172.21.15.98:0/916206250 shutdown_connections 2024-09-19T08:46:32.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.450+0000 7fb378dfc640 1 -- 172.21.15.98:0/916206250 wait complete. 2024-09-19T08:46:32.473 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- rather than going through a full upgrade cycle, ... ok 2024-09-19T08:46:32.473 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2024-09-19T08:46:32.473 INFO:tasks.cephfs_test_runner:test_subvolume_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) 2024-09-19T08:46:32.474 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:46:32.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.602+0000 7f6f2ff2f640 1 Processor -- start 2024-09-19T08:46:32.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.602+0000 7f6f2ff2f640 1 -- start start 2024-09-19T08:46:32.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.602+0000 7f6f2ff2f640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6f2806d880 0x7f6f280b3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:32.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.602+0000 7f6f2ff2f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f28141110 0x7f6f280b3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:32.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.602+0000 7f6f2ff2f640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f2815ea10 con 0x7f6f28140d30 2024-09-19T08:46:32.603 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.602+0000 7f6f2ff2f640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f6f2815e7c0 con 0x7f6f2806d880 2024-09-19T08:46:32.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.602+0000 7f6f2ff2f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6f2815ecf0 con 0x7f6f28141110 2024-09-19T08:46:32.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.606+0000 7f6f2d4ca640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6f2806d880 0x7f6f280b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:32.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.606+0000 7f6f2e4cc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f28141110 0x7f6f280b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:32.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.606+0000 7f6f2dccb640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f6f28140d30 0x7f6f2806d170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:57144/0 (socket says 172.21.15.98:57144) 2024-09-19T08:46:32.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.606+0000 7f6f2dccb640 1 -- 172.21.15.98:0/1630013380 learned_addr learned my addr 172.21.15.98:0/1630013380 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:32.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.606+0000 7f6f2e4cc640 1 --2- 172.21.15.98:0/1630013380 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f28141110 0x7f6f280b3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:60880/0 (socket says 172.21.15.98:60880) 2024-09-19T08:46:32.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.606+0000 7f6f2e4cc640 1 -- 172.21.15.98:0/1630013380 >> v2:172.21.15.139:3300/0 conn(0x7f6f2806d880 msgr2=0x7f6f280b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:32.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.606+0000 7f6f2e4cc640 1 --2- 172.21.15.98:0/1630013380 >> v2:172.21.15.139:3300/0 conn(0x7f6f2806d880 0x7f6f280b3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.606+0000 7f6f2e4cc640 1 -- 172.21.15.98:0/1630013380 >> v1:172.21.15.139:6789/0 conn(0x7f6f28140d30 legacy=0x7f6f2806d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:32.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.606+0000 7f6f2e4cc640 1 -- 172.21.15.98:0/1630013380 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f2815efd0 con 0x7f6f28141110 2024-09-19T08:46:32.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.606+0000 7f6f2e4cc640 1 --2- 172.21.15.98:0/1630013380 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f28141110 0x7f6f280b3850 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6f20009d90 tx=0x7f6f2001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3035dd84ffedcc32 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:32.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.606+0000 7f6f2ccc9640 1 -- 172.21.15.98:0/1630013380 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f200b2070 con 0x7f6f28141110 2024-09-19T08:46:32.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.606+0000 7f6f2ccc9640 1 -- 172.21.15.98:0/1630013380 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f200a59d0 con 0x7f6f28141110 2024-09-19T08:46:32.607 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.606+0000 7f6f2ccc9640 1 -- 172.21.15.98:0/1630013380 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f200a5da0 con 0x7f6f28141110 2024-09-19T08:46:32.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.610+0000 7f6f2ff2f640 1 -- 172.21.15.98:0/1630013380 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f28141110 msgr2=0x7f6f280b3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:32.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.610+0000 7f6f2ff2f640 1 --2- 172.21.15.98:0/1630013380 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f28141110 0x7f6f280b3850 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6f20009d90 tx=0x7f6f2001c970 comp rx=0 tx=0).stop 2024-09-19T08:46:32.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.610+0000 7f6f2ff2f640 1 -- 172.21.15.98:0/1630013380 shutdown_connections 2024-09-19T08:46:32.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.610+0000 7f6f2ff2f640 1 --2- 172.21.15.98:0/1630013380 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f28141110 0x7f6f280b3850 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.610+0000 7f6f2ff2f640 1 --2- 172.21.15.98:0/1630013380 >> v2:172.21.15.139:3300/0 conn(0x7f6f2806d880 0x7f6f280b3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.610+0000 7f6f2ff2f640 1 -- 172.21.15.98:0/1630013380 >> 172.21.15.98:0/1630013380 conn(0x7f6f280b2010 msgr2=0x7f6f280b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:32.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.610+0000 7f6f2ff2f640 1 -- 172.21.15.98:0/1630013380 shutdown_connections 2024-09-19T08:46:32.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.610+0000 7f6f2ff2f640 1 -- 172.21.15.98:0/1630013380 wait complete. 2024-09-19T08:46:32.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.610+0000 7f6f2ff2f640 1 Processor -- start 2024-09-19T08:46:32.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.610+0000 7f6f2ff2f640 1 -- start start 2024-09-19T08:46:32.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.610+0000 7f6f2ff2f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f2806d880 0x7f6f28156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:32.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.610+0000 7f6f2ff2f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f28140d30 0x7f6f281639e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:32.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.610+0000 7f6f2ff2f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f28141110 0x7f6f281f4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:32.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.610+0000 7f6f2ff2f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2815b580 con 0x7f6f28140d30 2024-09-19T08:46:32.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.610+0000 7f6f2ff2f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2815b400 con 0x7f6f2806d880 2024-09-19T08:46:32.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.610+0000 7f6f2ff2f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6f2815b700 con 0x7f6f28141110 2024-09-19T08:46:32.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.610+0000 7f6f2d4ca640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f28140d30 0x7f6f281639e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:32.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.610+0000 7f6f2d4ca640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f28140d30 0x7f6f281639e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:43300/0 (socket says 172.21.15.98:43300) 2024-09-19T08:46:32.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.610+0000 7f6f2d4ca640 1 -- 172.21.15.98:0/1381869644 learned_addr learned my addr 172.21.15.98:0/1381869644 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:32.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.610+0000 7f6f2e4cc640 1 --2- 172.21.15.98:0/1381869644 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f28141110 0x7f6f281f4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:32.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.610+0000 7f6f2dccb640 1 --2- 172.21.15.98:0/1381869644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f2806d880 0x7f6f28156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:32.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.610+0000 7f6f2d4ca640 1 -- 172.21.15.98:0/1381869644 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f28141110 msgr2=0x7f6f281f4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:32.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.610+0000 7f6f2d4ca640 1 --2- 172.21.15.98:0/1381869644 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f28141110 0x7f6f281f4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.610+0000 7f6f2d4ca640 1 -- 172.21.15.98:0/1381869644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f2806d880 msgr2=0x7f6f28156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:32.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.610+0000 7f6f2d4ca640 1 --2- 172.21.15.98:0/1381869644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f2806d880 0x7f6f28156e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.610+0000 7f6f2d4ca640 1 -- 172.21.15.98:0/1381869644 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f280736a0 con 0x7f6f28140d30 2024-09-19T08:46:32.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.610+0000 7f6f2d4ca640 1 --2- 172.21.15.98:0/1381869644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f28140d30 0x7f6f281639e0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f6f1800e930 tx=0x7f6f1800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:32.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.610+0000 7f6f1e7fc640 1 -- 172.21.15.98:0/1381869644 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f18017070 con 0x7f6f28140d30 2024-09-19T08:46:32.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.610+0000 7f6f1e7fc640 1 -- 172.21.15.98:0/1381869644 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f1800ca00 con 0x7f6f28140d30 2024-09-19T08:46:32.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.610+0000 7f6f1e7fc640 1 -- 172.21.15.98:0/1381869644 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1800cdb0 con 0x7f6f28140d30 2024-09-19T08:46:32.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.610+0000 7f6f2ff2f640 1 -- 172.21.15.98:0/1381869644 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f28073ae0 con 0x7f6f28140d30 2024-09-19T08:46:32.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.610+0000 7f6f2ff2f640 1 -- 172.21.15.98:0/1381869644 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f280740a0 con 0x7f6f28140d30 2024-09-19T08:46:32.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.614+0000 7f6f2ff2f640 1 -- 172.21.15.98:0/1381869644 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f280cc770 con 0x7f6f28140d30 2024-09-19T08:46:32.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.614+0000 7f6f1e7fc640 1 -- 172.21.15.98:0/1381869644 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f6f1801a070 con 0x7f6f28140d30 2024-09-19T08:46:32.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.614+0000 7f6f1e7fc640 1 --2- 172.21.15.98:0/1381869644 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6f0c0680a0 0x7f6f0c06a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:32.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.614+0000 7f6f1e7fc640 1 -- 172.21.15.98:0/1381869644 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4971+0+0 (secure 0 0 0) 0x7f6f18084a70 con 0x7f6f28140d30 2024-09-19T08:46:32.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.614+0000 7f6f2dccb640 1 --2- 172.21.15.98:0/1381869644 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6f0c0680a0 0x7f6f0c06a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:32.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.614+0000 7f6f2dccb640 1 --2- 172.21.15.98:0/1381869644 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6f0c0680a0 0x7f6f0c06a560 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6f24000f70 tx=0x7f6f24009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:32.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.618+0000 7f6f1e7fc640 1 -- 172.21.15.98:0/1381869644 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6f18056f60 con 0x7f6f28140d30 2024-09-19T08:46:32.782 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.782+0000 7f6f2ff2f640 1 -- 172.21.15.98:0/1381869644 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6f280af7e0 con 0x7f6f28140d30 2024-09-19T08:46:32.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.786+0000 7f6f1e7fc640 1 -- 172.21.15.98:0/1381869644 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) ==== 74+0+13084 (secure 0 0 0) 0x7f6f18056860 con 0x7f6f28140d30 2024-09-19T08:46:32.784 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:46:32.785 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":149,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:46:31.440324+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.98:6835/451004304":"2024-09-20T08:46:28.425146+0000","172.21.15.98:6833/451004304":"2024-09-20T08:46:28.425146+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:46:32.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.786+0000 7f6f2ff2f640 1 -- 172.21.15.98:0/1381869644 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6f0c0680a0 msgr2=0x7f6f0c06a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:32.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.786+0000 7f6f2ff2f640 1 --2- 172.21.15.98:0/1381869644 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6f0c0680a0 0x7f6f0c06a560 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6f24000f70 tx=0x7f6f24009040 comp rx=0 tx=0).stop 2024-09-19T08:46:32.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.786+0000 7f6f2ff2f640 1 -- 172.21.15.98:0/1381869644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f28140d30 msgr2=0x7f6f281639e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:32.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.786+0000 7f6f2ff2f640 1 --2- 172.21.15.98:0/1381869644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f28140d30 0x7f6f281639e0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f6f1800e930 tx=0x7f6f1800edf0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.786+0000 7f6f2ff2f640 1 -- 172.21.15.98:0/1381869644 shutdown_connections 2024-09-19T08:46:32.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.786+0000 7f6f2ff2f640 1 --2- 172.21.15.98:0/1381869644 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6f0c0680a0 0x7f6f0c06a560 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.786+0000 7f6f2ff2f640 1 --2- 172.21.15.98:0/1381869644 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6f28141110 0x7f6f281f4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.786+0000 7f6f2ff2f640 1 --2- 172.21.15.98:0/1381869644 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6f28140d30 0x7f6f281639e0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.786+0000 7f6f2ff2f640 1 --2- 172.21.15.98:0/1381869644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f2806d880 0x7f6f28156e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.786+0000 7f6f2ff2f640 1 -- 172.21.15.98:0/1381869644 >> 172.21.15.98:0/1381869644 conn(0x7f6f280b2010 msgr2=0x7f6f280ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:32.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.786+0000 7f6f2ff2f640 1 -- 172.21.15.98:0/1381869644 shutdown_connections 2024-09-19T08:46:32.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.786+0000 7f6f2ff2f640 1 -- 172.21.15.98:0/1381869644 wait complete. 2024-09-19T08:46:32.807 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T08:38:08.105684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T08:46:32.807 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T08:46:32.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.934+0000 7fe62097d640 1 Processor -- start 2024-09-19T08:46:32.934 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.934+0000 7fe62097d640 1 -- start start 2024-09-19T08:46:32.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.934+0000 7fe62097d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe61c06bc90 0x7fe61c06c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:32.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.934+0000 7fe62097d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe61c0d2f00 0x7fe61c0d3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:32.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.934+0000 7fe62097d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe61c072d30 con 0x7fe61c161970 2024-09-19T08:46:32.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.934+0000 7fe62097d640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fe61c06c600 con 0x7fe61c0d2f00 2024-09-19T08:46:32.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.934+0000 7fe62097d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe61c073010 con 0x7fe61c06bc90 2024-09-19T08:46:32.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.934+0000 7fe61ad9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe61c0d2f00 0x7fe61c0d3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:32.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.934+0000 7fe61ad9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe61c0d2f00 0x7fe61c0d3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:43304/0 (socket says 172.21.15.98:43304) 2024-09-19T08:46:32.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.934+0000 7fe61ad9d640 1 -- 172.21.15.98:0/765179778 learned_addr learned my addr 172.21.15.98:0/765179778 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:32.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.934+0000 7fe619d9b640 1 --2- 172.21.15.98:0/765179778 >> v2:172.21.15.139:3300/0 conn(0x7fe61c06bc90 0x7fe61c06c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:32.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.934+0000 7fe61ad9d640 1 -- 172.21.15.98:0/765179778 >> v2:172.21.15.139:3300/0 conn(0x7fe61c06bc90 msgr2=0x7fe61c06c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:32.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.934+0000 7fe61ad9d640 1 --2- 172.21.15.98:0/765179778 >> v2:172.21.15.139:3300/0 conn(0x7fe61c06bc90 0x7fe61c06c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.934+0000 7fe61ad9d640 1 -- 172.21.15.98:0/765179778 >> v1:172.21.15.139:6789/0 conn(0x7fe61c161970 legacy=0x7fe61c0722b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:32.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.934+0000 7fe61ad9d640 1 -- 172.21.15.98:0/765179778 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe61c0d2700 con 0x7fe61c0d2f00 2024-09-19T08:46:32.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.938+0000 7fe61ad9d640 1 --2- 172.21.15.98:0/765179778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe61c0d2f00 0x7fe61c0d3330 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fe614009e50 tx=0x7fe61401f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a9b9d96529fedf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:32.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.938+0000 7fe61959a640 1 -- 172.21.15.98:0/765179778 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6140b4070 con 0x7fe61c0d2f00 2024-09-19T08:46:32.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.938+0000 7fe61959a640 1 -- 172.21.15.98:0/765179778 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6140af400 con 0x7fe61c0d2f00 2024-09-19T08:46:32.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.938+0000 7fe61959a640 1 -- 172.21.15.98:0/765179778 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6140af7d0 con 0x7fe61c0d2f00 2024-09-19T08:46:32.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.938+0000 7fe62097d640 1 -- 172.21.15.98:0/765179778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe61c0d2f00 msgr2=0x7fe61c0d3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:32.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.938+0000 7fe62097d640 1 --2- 172.21.15.98:0/765179778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe61c0d2f00 0x7fe61c0d3330 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fe614009e50 tx=0x7fe61401f1e0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.938+0000 7fe62097d640 1 -- 172.21.15.98:0/765179778 shutdown_connections 2024-09-19T08:46:32.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.938+0000 7fe62097d640 1 --2- 172.21.15.98:0/765179778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe61c0d2f00 0x7fe61c0d3330 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.938+0000 7fe62097d640 1 --2- 172.21.15.98:0/765179778 >> v2:172.21.15.139:3300/0 conn(0x7fe61c06bc90 0x7fe61c06c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.938+0000 7fe62097d640 1 -- 172.21.15.98:0/765179778 >> 172.21.15.98:0/765179778 conn(0x7fe61c0d1200 msgr2=0x7fe61c0d1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:32.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.938+0000 7fe62097d640 1 -- 172.21.15.98:0/765179778 shutdown_connections 2024-09-19T08:46:32.939 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.938+0000 7fe62097d640 1 -- 172.21.15.98:0/765179778 wait complete. 2024-09-19T08:46:32.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.942+0000 7fe62097d640 1 Processor -- start 2024-09-19T08:46:32.940 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.942+0000 7fe62097d640 1 -- start start 2024-09-19T08:46:32.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.942+0000 7fe62097d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe61c06bc90 0x7fe61c0ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:32.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.942+0000 7fe62097d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe61c0d2f00 0x7fe61c0ce550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:32.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.942+0000 7fe62097d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe61c161970 0x7fe61c0b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:32.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.942+0000 7fe62097d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c0729c0 con 0x7fe61c0d2f00 2024-09-19T08:46:32.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.942+0000 7fe62097d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c0cee20 con 0x7fe61c06bc90 2024-09-19T08:46:32.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.942+0000 7fe62097d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe61c161d50 con 0x7fe61c161970 2024-09-19T08:46:32.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.942+0000 7fe619d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe61c0d2f00 0x7fe61c0ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:32.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.942+0000 7fe619d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe61c0d2f00 0x7fe61c0ce550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:43312/0 (socket says 172.21.15.98:43312) 2024-09-19T08:46:32.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.942+0000 7fe619d9b640 1 -- 172.21.15.98:0/4111974608 learned_addr learned my addr 172.21.15.98:0/4111974608 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:32.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.942+0000 7fe61ad9d640 1 --2- 172.21.15.98:0/4111974608 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe61c161970 0x7fe61c0b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:32.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.942+0000 7fe61a59c640 1 --2- 172.21.15.98:0/4111974608 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe61c06bc90 0x7fe61c0ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:32.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.942+0000 7fe619d9b640 1 -- 172.21.15.98:0/4111974608 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe61c161970 msgr2=0x7fe61c0b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:32.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.942+0000 7fe619d9b640 1 --2- 172.21.15.98:0/4111974608 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe61c161970 0x7fe61c0b10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.942+0000 7fe619d9b640 1 -- 172.21.15.98:0/4111974608 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe61c06bc90 msgr2=0x7fe61c0ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:32.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.942+0000 7fe619d9b640 1 --2- 172.21.15.98:0/4111974608 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe61c06bc90 0x7fe61c0ce010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:32.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.942+0000 7fe619d9b640 1 -- 172.21.15.98:0/4111974608 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe61c1f5530 con 0x7fe61c0d2f00 2024-09-19T08:46:32.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.942+0000 7fe61a59c640 1 --2- 172.21.15.98:0/4111974608 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe61c06bc90 0x7fe61c0ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:32.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.942+0000 7fe61ad9d640 1 --2- 172.21.15.98:0/4111974608 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe61c161970 0x7fe61c0b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:32.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.942+0000 7fe619d9b640 1 --2- 172.21.15.98:0/4111974608 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe61c0d2f00 0x7fe61c0ce550 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fe61000d5f0 tx=0x7fe61000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:32.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.942+0000 7fe60affd640 1 -- 172.21.15.98:0/4111974608 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6100125f0 con 0x7fe61c0d2f00 2024-09-19T08:46:32.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.942+0000 7fe60affd640 1 -- 172.21.15.98:0/4111974608 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe610019070 con 0x7fe61c0d2f00 2024-09-19T08:46:32.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.942+0000 7fe62097d640 1 -- 172.21.15.98:0/4111974608 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe61c0b1dc0 con 0x7fe61c0d2f00 2024-09-19T08:46:32.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.942+0000 7fe60affd640 1 -- 172.21.15.98:0/4111974608 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6100129b0 con 0x7fe61c0d2f00 2024-09-19T08:46:32.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.942+0000 7fe62097d640 1 -- 172.21.15.98:0/4111974608 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe61c0b2380 con 0x7fe61c0d2f00 2024-09-19T08:46:32.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.942+0000 7fe62097d640 1 -- 172.21.15.98:0/4111974608 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe61c06c0c0 con 0x7fe61c0d2f00 2024-09-19T08:46:32.944 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.942+0000 7fe60affd640 1 -- 172.21.15.98:0/4111974608 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7fe610012b50 con 0x7fe61c0d2f00 2024-09-19T08:46:32.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.942+0000 7fe60affd640 1 --2- 172.21.15.98:0/4111974608 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe5fc0680a0 0x7fe5fc06a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:32.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.942+0000 7fe60affd640 1 -- 172.21.15.98:0/4111974608 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4971+0+0 (secure 0 0 0) 0x7fe610084a80 con 0x7fe61c0d2f00 2024-09-19T08:46:32.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.946+0000 7fe61a59c640 1 --2- 172.21.15.98:0/4111974608 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe5fc0680a0 0x7fe5fc06a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:32.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.946+0000 7fe61a59c640 1 --2- 172.21.15.98:0/4111974608 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe5fc0680a0 0x7fe5fc06a560 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe61c072690 tx=0x7fe60c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:32.948 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:32.950+0000 7fe60affd640 1 -- 172.21.15.98:0/4111974608 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe610056eb0 con 0x7fe61c0d2f00 2024-09-19T08:46:33.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.110+0000 7fe62097d640 1 -- 172.21.15.98:0/4111974608 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe61c0ba530 con 0x7fe61c0d2f00 2024-09-19T08:46:33.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.114+0000 7fe60affd640 1 -- 172.21.15.98:0/4111974608 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v149) ==== 93+0+10 (secure 0 0 0) 0x7fe6100567b0 con 0x7fe61c0d2f00 2024-09-19T08:46:33.112 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2024-09-19T08:46:33.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.114+0000 7fe62097d640 1 -- 172.21.15.98:0/4111974608 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe5fc0680a0 msgr2=0x7fe5fc06a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:33.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.114+0000 7fe62097d640 1 --2- 172.21.15.98:0/4111974608 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe5fc0680a0 0x7fe5fc06a560 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe61c072690 tx=0x7fe60c009040 comp rx=0 tx=0).stop 2024-09-19T08:46:33.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.114+0000 7fe62097d640 1 -- 172.21.15.98:0/4111974608 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe61c0d2f00 msgr2=0x7fe61c0ce550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:33.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.114+0000 7fe62097d640 1 --2- 172.21.15.98:0/4111974608 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe61c0d2f00 0x7fe61c0ce550 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fe61000d5f0 tx=0x7fe61000dab0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.115 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.114+0000 7fe62097d640 1 -- 172.21.15.98:0/4111974608 shutdown_connections 2024-09-19T08:46:33.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.114+0000 7fe62097d640 1 --2- 172.21.15.98:0/4111974608 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe5fc0680a0 0x7fe5fc06a560 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.114+0000 7fe62097d640 1 --2- 172.21.15.98:0/4111974608 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe61c161970 0x7fe61c0b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.114+0000 7fe62097d640 1 --2- 172.21.15.98:0/4111974608 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe61c0d2f00 0x7fe61c0ce550 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.114+0000 7fe62097d640 1 --2- 172.21.15.98:0/4111974608 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe61c06bc90 0x7fe61c0ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.114+0000 7fe62097d640 1 -- 172.21.15.98:0/4111974608 >> 172.21.15.98:0/4111974608 conn(0x7fe61c0d1200 msgr2=0x7fe61c0b9190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:33.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.114+0000 7fe62097d640 1 -- 172.21.15.98:0/4111974608 shutdown_connections 2024-09-19T08:46:33.116 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.114+0000 7fe62097d640 1 -- 172.21.15.98:0/4111974608 wait complete. 2024-09-19T08:46:33.132 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2' 2024-09-19T08:46:33.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.250+0000 7f03d1169640 1 Processor -- start 2024-09-19T08:46:33.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.250+0000 7f03d1169640 1 -- start start 2024-09-19T08:46:33.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.254+0000 7f03d1169640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f03cc0d1d70 0x7f03cc0d1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:33.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.254+0000 7f03d1169640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f03cc0b6200 0x7f03cc0b6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:33.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.254+0000 7f03d1169640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03cc0b8060 con 0x7f03cc147530 2024-09-19T08:46:33.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.254+0000 7f03d1169640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f03cc0d2230 con 0x7f03cc0b6200 2024-09-19T08:46:33.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.254+0000 7f03d1169640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f03cc0b8340 con 0x7f03cc0d1d70 2024-09-19T08:46:33.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.254+0000 7f03cad9d640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f03cc147530 0x7f03cc15ec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36370/0 (socket says 172.21.15.98:36370) 2024-09-19T08:46:33.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.254+0000 7f03cad9d640 1 -- 172.21.15.98:0/1762137749 learned_addr learned my addr 172.21.15.98:0/1762137749 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:33.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.254+0000 7f03cb59e640 1 --2- 172.21.15.98:0/1762137749 >> v2:172.21.15.98:3300/0 conn(0x7f03cc0b6200 0x7f03cc0b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:33.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.254+0000 7f03ca59c640 1 --2- 172.21.15.98:0/1762137749 >> v2:172.21.15.139:3300/0 conn(0x7f03cc0d1d70 0x7f03cc0d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:33.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.254+0000 7f03cb59e640 1 -- 172.21.15.98:0/1762137749 >> v2:172.21.15.139:3300/0 conn(0x7f03cc0d1d70 msgr2=0x7f03cc0d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:33.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.254+0000 7f03cb59e640 1 --2- 172.21.15.98:0/1762137749 >> v2:172.21.15.139:3300/0 conn(0x7f03cc0d1d70 0x7f03cc0d1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.254+0000 7f03cb59e640 1 -- 172.21.15.98:0/1762137749 >> v1:172.21.15.98:6789/0 conn(0x7f03cc147530 legacy=0x7f03cc15ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:33.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.254+0000 7f03cb59e640 1 -- 172.21.15.98:0/1762137749 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03cc0b8620 con 0x7f03cc0b6200 2024-09-19T08:46:33.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.254+0000 7f03cb59e640 1 --2- 172.21.15.98:0/1762137749 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03cc0b6200 0x7f03cc0b6690 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f03cc148100 tx=0x7f03c401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b940abeba716bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:33.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.254+0000 7f03c9d9b640 1 -- 172.21.15.98:0/1762137749 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c40b1070 con 0x7f03cc0b6200 2024-09-19T08:46:33.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.254+0000 7f03c9d9b640 1 -- 172.21.15.98:0/1762137749 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03c40ac5f0 con 0x7f03cc0b6200 2024-09-19T08:46:33.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.254+0000 7f03c9d9b640 1 -- 172.21.15.98:0/1762137749 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c40ac9c0 con 0x7f03cc0b6200 2024-09-19T08:46:33.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.258+0000 7f03d1169640 1 -- 172.21.15.98:0/1762137749 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03cc0b6200 msgr2=0x7f03cc0b6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:33.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.258+0000 7f03d1169640 1 --2- 172.21.15.98:0/1762137749 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03cc0b6200 0x7f03cc0b6690 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f03cc148100 tx=0x7f03c401c480 comp rx=0 tx=0).stop 2024-09-19T08:46:33.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.258+0000 7f03d1169640 1 -- 172.21.15.98:0/1762137749 shutdown_connections 2024-09-19T08:46:33.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.258+0000 7f03d1169640 1 --2- 172.21.15.98:0/1762137749 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03cc0b6200 0x7f03cc0b6690 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.258+0000 7f03d1169640 1 --2- 172.21.15.98:0/1762137749 >> v2:172.21.15.139:3300/0 conn(0x7f03cc0d1d70 0x7f03cc0d1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.258+0000 7f03d1169640 1 -- 172.21.15.98:0/1762137749 >> 172.21.15.98:0/1762137749 conn(0x7f03cc075070 msgr2=0x7f03cc0722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:33.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.258+0000 7f03d1169640 1 -- 172.21.15.98:0/1762137749 shutdown_connections 2024-09-19T08:46:33.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.258+0000 7f03d1169640 1 -- 172.21.15.98:0/1762137749 wait complete. 2024-09-19T08:46:33.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.258+0000 7f03d1169640 1 Processor -- start 2024-09-19T08:46:33.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.258+0000 7f03d1169640 1 -- start start 2024-09-19T08:46:33.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.258+0000 7f03d1169640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03cc0b6200 0x7f03cc15cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:33.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.258+0000 7f03d1169640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03cc0d1d70 0x7f03cc15db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:33.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.258+0000 7f03d1169640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03cc147530 0x7f03cc15af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:33.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.258+0000 7f03d1169640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc0b97b0 con 0x7f03cc0b6200 2024-09-19T08:46:33.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.258+0000 7f03d1169640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc0b9630 con 0x7f03cc147530 2024-09-19T08:46:33.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.258+0000 7f03d1169640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f03cc0b9930 con 0x7f03cc0d1d70 2024-09-19T08:46:33.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.258+0000 7f03cad9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03cc0b6200 0x7f03cc15cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:33.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.258+0000 7f03cad9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03cc0b6200 0x7f03cc15cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:43334/0 (socket says 172.21.15.98:43334) 2024-09-19T08:46:33.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.258+0000 7f03cad9d640 1 -- 172.21.15.98:0/2181171520 learned_addr learned my addr 172.21.15.98:0/2181171520 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:33.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.258+0000 7f03cad9d640 1 -- 172.21.15.98:0/2181171520 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03cc0d1d70 msgr2=0x7f03cc15db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:33.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.258+0000 7f03cad9d640 1 --2- 172.21.15.98:0/2181171520 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03cc0d1d70 0x7f03cc15db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.258+0000 7f03cad9d640 1 -- 172.21.15.98:0/2181171520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03cc147530 msgr2=0x7f03cc15af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:33.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.258+0000 7f03cad9d640 1 --2- 172.21.15.98:0/2181171520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03cc147530 0x7f03cc15af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:33.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.258+0000 7f03cad9d640 1 -- 172.21.15.98:0/2181171520 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03cc15c6b0 con 0x7f03cc0b6200 2024-09-19T08:46:33.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.258+0000 7f03cad9d640 1 --2- 172.21.15.98:0/2181171520 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03cc0b6200 0x7f03cc15cf50 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f03bc00cb40 tx=0x7f03bc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:33.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.258+0000 7f03c37fe640 1 -- 172.21.15.98:0/2181171520 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc017070 con 0x7f03cc0b6200 2024-09-19T08:46:33.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.258+0000 7f03c37fe640 1 -- 172.21.15.98:0/2181171520 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03bc00aca0 con 0x7f03cc0b6200 2024-09-19T08:46:33.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.258+0000 7f03d1169640 1 -- 172.21.15.98:0/2181171520 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03cc15c990 con 0x7f03cc0b6200 2024-09-19T08:46:33.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.258+0000 7f03c37fe640 1 -- 172.21.15.98:0/2181171520 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc012620 con 0x7f03cc0b6200 2024-09-19T08:46:33.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.258+0000 7f03d1169640 1 -- 172.21.15.98:0/2181171520 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03cc15ee50 con 0x7f03cc0b6200 2024-09-19T08:46:33.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.262+0000 7f03d1169640 1 -- 172.21.15.98:0/2181171520 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03cc15ec70 con 0x7f03cc0b6200 2024-09-19T08:46:33.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.262+0000 7f03c37fe640 1 -- 172.21.15.98:0/2181171520 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f03bc01a070 con 0x7f03cc0b6200 2024-09-19T08:46:33.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.262+0000 7f03c37fe640 1 --2- 172.21.15.98:0/2181171520 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f03ac0680a0 0x7f03ac06a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:33.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.262+0000 7f03c37fe640 1 -- 172.21.15.98:0/2181171520 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4971+0+0 (secure 0 0 0) 0x7f03bc084470 con 0x7f03cc0b6200 2024-09-19T08:46:33.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.262+0000 7f03ca59c640 1 --2- 172.21.15.98:0/2181171520 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f03ac0680a0 0x7f03ac06a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:33.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.262+0000 7f03ca59c640 1 --2- 172.21.15.98:0/2181171520 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f03ac0680a0 0x7f03ac06a560 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f03cc1453c0 tx=0x7f03b4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:33.265 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.266+0000 7f03c37fe640 1 -- 172.21.15.98:0/2181171520 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f03cc15ec70 con 0x7f03cc0b6200 2024-09-19T08:46:33.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:33.482+0000 7f03d1169640 1 -- 172.21.15.98:0/2181171520 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]} v 0) -- 0x7f03cc0b6690 con 0x7f03cc0b6200 2024-09-19T08:46:34.458 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.458+0000 7f03c37fe640 1 -- 172.21.15.98:0/2181171520 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]}]=0 v431) ==== 148+0+0 (secure 0 0 0) 0x7f03bc0568a0 con 0x7f03cc0b6200 2024-09-19T08:46:34.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.462+0000 7f03d1169640 1 -- 172.21.15.98:0/2181171520 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f03ac0680a0 msgr2=0x7f03ac06a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:34.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.462+0000 7f03d1169640 1 --2- 172.21.15.98:0/2181171520 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f03ac0680a0 0x7f03ac06a560 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f03cc1453c0 tx=0x7f03b4008040 comp rx=0 tx=0).stop 2024-09-19T08:46:34.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.462+0000 7f03d1169640 1 -- 172.21.15.98:0/2181171520 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03cc0b6200 msgr2=0x7f03cc15cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:34.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.462+0000 7f03d1169640 1 --2- 172.21.15.98:0/2181171520 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03cc0b6200 0x7f03cc15cf50 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f03bc00cb40 tx=0x7f03bc00a060 comp rx=0 tx=0).stop 2024-09-19T08:46:34.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.462+0000 7f03d1169640 1 -- 172.21.15.98:0/2181171520 shutdown_connections 2024-09-19T08:46:34.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.462+0000 7f03d1169640 1 --2- 172.21.15.98:0/2181171520 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f03ac0680a0 0x7f03ac06a560 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:34.461 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.462+0000 7f03d1169640 1 --2- 172.21.15.98:0/2181171520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03cc147530 0x7f03cc15af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:34.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.462+0000 7f03d1169640 1 --2- 172.21.15.98:0/2181171520 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f03cc0d1d70 0x7f03cc15db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:34.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.462+0000 7f03d1169640 1 --2- 172.21.15.98:0/2181171520 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f03cc0b6200 0x7f03cc15cf50 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:34.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.462+0000 7f03d1169640 1 -- 172.21.15.98:0/2181171520 >> 172.21.15.98:0/2181171520 conn(0x7f03cc075070 msgr2=0x7f03cc13ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:34.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.462+0000 7f03d1169640 1 -- 172.21.15.98:0/2181171520 shutdown_connections 2024-09-19T08:46:34.462 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.462+0000 7f03d1169640 1 -- 172.21.15.98:0/2181171520 wait complete. 2024-09-19T08:46:34.480 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T08:46:34.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01da16a640 1 Processor -- start 2024-09-19T08:46:34.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01da16a640 1 -- start start 2024-09-19T08:46:34.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01da16a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f01d415b4f0 0x7f01d40b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:34.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01da16a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f01d40b0a70 0x7f01d406c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:34.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01da16a640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01d40af4a0 con 0x7f01d406c9c0 2024-09-19T08:46:34.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01da16a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f01d40af780 con 0x7f01d415b4f0 2024-09-19T08:46:34.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01da16a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f01d40afa60 con 0x7f01d40b0a70 2024-09-19T08:46:34.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01d3fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f01d406c9c0 0x7f01d40ae0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36382/0 (socket says 172.21.15.98:36382) 2024-09-19T08:46:34.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01d3fff640 1 -- 172.21.15.98:0/1247252528 learned_addr learned my addr 172.21.15.98:0/1247252528 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:34.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01cbfff640 1 --2- 172.21.15.98:0/1247252528 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f01d40b0a70 0x7f01d406c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:34.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01d37fe640 1 --2- 172.21.15.98:0/1247252528 >> v2:172.21.15.139:3300/0 conn(0x7f01d415b4f0 0x7f01d40b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:34.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01d2ffd640 1 -- 172.21.15.98:0/1247252528 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 604505800 0 0) 0x7f01d40af4a0 con 0x7f01d406c9c0 2024-09-19T08:46:34.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01d2ffd640 1 -- 172.21.15.98:0/1247252528 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01c0003530 con 0x7f01d406c9c0 2024-09-19T08:46:34.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01d2ffd640 1 -- 172.21.15.98:0/1247252528 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 136610740 0 0) 0x7f01c0003530 con 0x7f01d406c9c0 2024-09-19T08:46:34.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01d2ffd640 1 -- 172.21.15.98:0/1247252528 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f01d40b0a70 msgr2=0x7f01d406c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:34.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01d2ffd640 1 --2- 172.21.15.98:0/1247252528 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f01d40b0a70 0x7f01d406c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:34.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01d2ffd640 1 -- 172.21.15.98:0/1247252528 >> v2:172.21.15.139:3300/0 conn(0x7f01d415b4f0 msgr2=0x7f01d40b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:34.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01d2ffd640 1 --2- 172.21.15.98:0/1247252528 >> v2:172.21.15.139:3300/0 conn(0x7f01d415b4f0 0x7f01d40b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:34.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01d2ffd640 1 -- 172.21.15.98:0/1247252528 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d40afd40 con 0x7f01d406c9c0 2024-09-19T08:46:34.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01d2ffd640 1 -- 172.21.15.98:0/1247252528 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f01cc001a00 con 0x7f01d406c9c0 2024-09-19T08:46:34.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01d2ffd640 1 -- 172.21.15.98:0/1247252528 >> v1:172.21.15.98:6789/0 conn(0x7f01d406c9c0 legacy=0x7f01d40ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:34.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01d2ffd640 1 --2- 172.21.15.98:0/1247252528 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01c00039c0 0x7f01c0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:34.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01d2ffd640 1 --2- 172.21.15.98:0/1247252528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01c0004810 0x7f01c0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:34.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01d2ffd640 1 --2- 172.21.15.98:0/1247252528 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f01d40b0a70 0x7f01c00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:34.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01d2ffd640 1 -- 172.21.15.98:0/1247252528 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f01d40af780 con 0x7f01c0004810 2024-09-19T08:46:34.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01d2ffd640 1 -- 172.21.15.98:0/1247252528 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f01d40afa60 con 0x7f01c00039c0 2024-09-19T08:46:34.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01d2ffd640 1 -- 172.21.15.98:0/1247252528 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f01c000b9d0 con 0x7f01d40b0a70 2024-09-19T08:46:34.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01cbfff640 1 --2- 172.21.15.98:0/1247252528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01c0004810 0x7f01c0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:34.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01d37fe640 1 --2- 172.21.15.98:0/1247252528 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01c00039c0 0x7f01c0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:34.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01d3fff640 1 --2- 172.21.15.98:0/1247252528 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f01d40b0a70 0x7f01c00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:34.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01cbfff640 1 -- 172.21.15.98:0/1247252528 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f01d40b0a70 msgr2=0x7f01c00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:34.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01cbfff640 1 --2- 172.21.15.98:0/1247252528 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f01d40b0a70 0x7f01c00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:34.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01cbfff640 1 -- 172.21.15.98:0/1247252528 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01c00039c0 msgr2=0x7f01c0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:34.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01cbfff640 1 --2- 172.21.15.98:0/1247252528 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01c00039c0 0x7f01c0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:34.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01cbfff640 1 -- 172.21.15.98:0/1247252528 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c0003530 con 0x7f01c0004810 2024-09-19T08:46:34.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01d37fe640 1 --2- 172.21.15.98:0/1247252528 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01c00039c0 0x7f01c0003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:34.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.610+0000 7f01d3fff640 1 --2- 172.21.15.98:0/1247252528 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f01d40b0a70 0x7f01c00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:34.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.614+0000 7f01cbfff640 1 --2- 172.21.15.98:0/1247252528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01c0004810 0x7f01c0004c90 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f01c4009ab0 tx=0x7f01c401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=219ae3287903c754 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:34.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.614+0000 7f01d2ffd640 1 -- 172.21.15.98:0/1247252528 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c40b2070 con 0x7f01c0004810 2024-09-19T08:46:34.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.614+0000 7f01d2ffd640 1 -- 172.21.15.98:0/1247252528 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01c40a5560 con 0x7f01c0004810 2024-09-19T08:46:34.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.614+0000 7f01d2ffd640 1 -- 172.21.15.98:0/1247252528 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c40096b0 con 0x7f01c0004810 2024-09-19T08:46:34.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.614+0000 7f01da16a640 1 -- 172.21.15.98:0/1247252528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01c0004810 msgr2=0x7f01c0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:34.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.614+0000 7f01da16a640 1 --2- 172.21.15.98:0/1247252528 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01c0004810 0x7f01c0004c90 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f01c4009ab0 tx=0x7f01c401ca50 comp rx=0 tx=0).stop 2024-09-19T08:46:34.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.614+0000 7f01d3fff640 1 -- 172.21.15.98:0/1247252528 reap_dead start 2024-09-19T08:46:34.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.614+0000 7f01da16a640 1 -- 172.21.15.98:0/1247252528 shutdown_connections 2024-09-19T08:46:34.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.614+0000 7f01da16a640 1 -- 172.21.15.98:0/1247252528 >> 172.21.15.98:0/1247252528 conn(0x7f01d40b5f50 msgr2=0x7f01d40b6650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:34.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.614+0000 7f01da16a640 1 -- 172.21.15.98:0/1247252528 shutdown_connections 2024-09-19T08:46:34.615 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.614+0000 7f01da16a640 1 -- 172.21.15.98:0/1247252528 wait complete. 2024-09-19T08:46:34.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.614+0000 7f01da16a640 1 Processor -- start 2024-09-19T08:46:34.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.614+0000 7f01da16a640 1 -- start start 2024-09-19T08:46:34.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.614+0000 7f01da16a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01c0004810 0x7f01d415d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:34.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.618+0000 7f01da16a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01d406f760 0x7f01d41465f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:34.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.618+0000 7f01da16a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f01d415ba50 0x7f01d415bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:34.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.618+0000 7f01da16a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f01d40b2bd0 con 0x7f01c0004810 2024-09-19T08:46:34.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.618+0000 7f01da16a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f01d415b8d0 con 0x7f01d406f760 2024-09-19T08:46:34.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.618+0000 7f01da16a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f01d40b2d50 con 0x7f01d415ba50 2024-09-19T08:46:34.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.618+0000 7f01d37fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01c0004810 0x7f01d415d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:34.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.618+0000 7f01d37fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01c0004810 0x7f01d415d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:43362/0 (socket says 172.21.15.98:43362) 2024-09-19T08:46:34.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.618+0000 7f01d37fe640 1 -- 172.21.15.98:0/1646688451 learned_addr learned my addr 172.21.15.98:0/1646688451 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:34.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.618+0000 7f01cbfff640 1 --2- 172.21.15.98:0/1646688451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01d406f760 0x7f01d41465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:34.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.618+0000 7f01d3fff640 1 --2- 172.21.15.98:0/1646688451 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f01d415ba50 0x7f01d415bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:34.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.618+0000 7f01d37fe640 1 -- 172.21.15.98:0/1646688451 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f01d415ba50 msgr2=0x7f01d415bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:34.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.618+0000 7f01d37fe640 1 --2- 172.21.15.98:0/1646688451 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f01d415ba50 0x7f01d415bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:34.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.618+0000 7f01d37fe640 1 -- 172.21.15.98:0/1646688451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01d406f760 msgr2=0x7f01d41465f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:34.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.618+0000 7f01d37fe640 1 --2- 172.21.15.98:0/1646688451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01d406f760 0x7f01d41465f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:34.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.618+0000 7f01d37fe640 1 -- 172.21.15.98:0/1646688451 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d415e000 con 0x7f01c0004810 2024-09-19T08:46:34.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.618+0000 7f01d37fe640 1 --2- 172.21.15.98:0/1646688451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01c0004810 0x7f01d415d920 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f01bc011900 tx=0x7f01bc011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:34.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.618+0000 7f01d0ff9640 1 -- 172.21.15.98:0/1646688451 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01bc004220 con 0x7f01c0004810 2024-09-19T08:46:34.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.618+0000 7f01d0ff9640 1 -- 172.21.15.98:0/1646688451 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01bc0043c0 con 0x7f01c0004810 2024-09-19T08:46:34.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.618+0000 7f01d0ff9640 1 -- 172.21.15.98:0/1646688451 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01bc00fbd0 con 0x7f01c0004810 2024-09-19T08:46:34.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.618+0000 7f01da16a640 1 -- 172.21.15.98:0/1646688451 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01d415e2e0 con 0x7f01c0004810 2024-09-19T08:46:34.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.618+0000 7f01da16a640 1 -- 172.21.15.98:0/1646688451 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01d415e8a0 con 0x7f01c0004810 2024-09-19T08:46:34.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.618+0000 7f01da16a640 1 -- 172.21.15.98:0/1646688451 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01d40b0530 con 0x7f01c0004810 2024-09-19T08:46:34.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.618+0000 7f01d0ff9640 1 -- 172.21.15.98:0/1646688451 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f01bc003620 con 0x7f01c0004810 2024-09-19T08:46:34.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.618+0000 7f01d0ff9640 1 --2- 172.21.15.98:0/1646688451 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f01b40680d0 0x7f01b406a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:34.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.618+0000 7f01d0ff9640 1 -- 172.21.15.98:0/1646688451 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4971+0+0 (secure 0 0 0) 0x7f01bc084e00 con 0x7f01c0004810 2024-09-19T08:46:34.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.618+0000 7f01cbfff640 1 --2- 172.21.15.98:0/1646688451 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f01b40680d0 0x7f01b406a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:34.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.622+0000 7f01cbfff640 1 --2- 172.21.15.98:0/1646688451 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f01b40680d0 0x7f01b406a590 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f01c40af000 tx=0x7f01c40046d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:34.623 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.622+0000 7f01d0ff9640 1 -- 172.21.15.98:0/1646688451 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f01bc019030 con 0x7f01c0004810 2024-09-19T08:46:34.780 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.782+0000 7f01da16a640 1 -- 172.21.15.98:0/1646688451 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f01d40cb8f0 con 0x7f01c0004810 2024-09-19T08:46:34.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.786+0000 7f01d0ff9640 1 -- 172.21.15.98:0/1646688451 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7f01bc057230 con 0x7f01c0004810 2024-09-19T08:46:34.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.790+0000 7f01da16a640 1 -- 172.21.15.98:0/1646688451 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f01b40680d0 msgr2=0x7f01b406a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:34.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.790+0000 7f01da16a640 1 --2- 172.21.15.98:0/1646688451 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f01b40680d0 0x7f01b406a590 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f01c40af000 tx=0x7f01c40046d0 comp rx=0 tx=0).stop 2024-09-19T08:46:34.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.790+0000 7f01da16a640 1 -- 172.21.15.98:0/1646688451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01c0004810 msgr2=0x7f01d415d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:34.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.790+0000 7f01da16a640 1 --2- 172.21.15.98:0/1646688451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01c0004810 0x7f01d415d920 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f01bc011900 tx=0x7f01bc011dc0 comp rx=0 tx=0).stop 2024-09-19T08:46:34.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.790+0000 7f01da16a640 1 -- 172.21.15.98:0/1646688451 shutdown_connections 2024-09-19T08:46:34.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.790+0000 7f01da16a640 1 --2- 172.21.15.98:0/1646688451 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f01b40680d0 0x7f01b406a590 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:34.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.790+0000 7f01da16a640 1 --2- 172.21.15.98:0/1646688451 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f01d415ba50 0x7f01d415bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:34.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.790+0000 7f01da16a640 1 --2- 172.21.15.98:0/1646688451 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01d406f760 0x7f01d41465f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:34.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.790+0000 7f01da16a640 1 --2- 172.21.15.98:0/1646688451 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01c0004810 0x7f01d415d920 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:34.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.790+0000 7f01da16a640 1 -- 172.21.15.98:0/1646688451 >> 172.21.15.98:0/1646688451 conn(0x7f01d40b5f50 msgr2=0x7f01d4074c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:34.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.790+0000 7f01da16a640 1 -- 172.21.15.98:0/1646688451 shutdown_connections 2024-09-19T08:46:34.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:34.790+0000 7f01da16a640 1 -- 172.21.15.98:0/1646688451 wait complete. 2024-09-19T08:46:34.805 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T08:46:34.805 DEBUG:teuthology.orchestra.run.smithi098:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:46:34.877 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:46:34.960 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T08:46:34.960 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:46:34.960 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:46:34.966 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15846 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T08:46:34.966 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:46:34.966 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:46:34.973 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T08:46:34.973 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:46:34.973 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:46:35.013 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15846 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T08:46:35.014 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:46:35.014 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:46:35.022 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T08:46:35.022 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-19T08:46:35.023 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:46:35.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.182+0000 7fd08ff60640 1 Processor -- start 2024-09-19T08:46:35.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.182+0000 7fd08ff60640 1 -- start start 2024-09-19T08:46:35.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.186+0000 7fd08ff60640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd08806c3a0 0x7fd0880cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:35.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.186+0000 7fd08ff60640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd08806d310 con 0x7fd0880cf900 2024-09-19T08:46:35.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.186+0000 7fd08ff60640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd08806d5f0 con 0x7fd08815c550 2024-09-19T08:46:35.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.186+0000 7fd08ff60640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fd08806d8d0 con 0x7fd08806c3a0 2024-09-19T08:46:35.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.186+0000 7fd08e4fd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fd0880cf900 0x7fd0880723a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36384/0 (socket says 172.21.15.98:36384) 2024-09-19T08:46:35.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.186+0000 7fd08e4fd640 1 -- 172.21.15.98:0/1978962761 learned_addr learned my addr 172.21.15.98:0/1978962761 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:35.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.186+0000 7fd08d4fb640 1 --2- 172.21.15.98:0/1978962761 >> v2:172.21.15.139:3300/0 conn(0x7fd08806c3a0 0x7fd0880cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:35.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.186+0000 7fd08ccfa640 1 -- 172.21.15.98:0/1978962761 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1761863984 0 0) 0x7fd08806d310 con 0x7fd0880cf900 2024-09-19T08:46:35.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.186+0000 7fd08ccfa640 1 -- 172.21.15.98:0/1978962761 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd074003530 con 0x7fd0880cf900 2024-09-19T08:46:35.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.186+0000 7fd08ccfa640 1 -- 172.21.15.98:0/1978962761 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3157222073 0 0) 0x7fd074003530 con 0x7fd0880cf900 2024-09-19T08:46:35.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.186+0000 7fd08ccfa640 1 -- 172.21.15.98:0/1978962761 >> v2:172.21.15.139:3300/0 conn(0x7fd08806c3a0 msgr2=0x7fd0880cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:35.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.186+0000 7fd08ccfa640 1 --2- 172.21.15.98:0/1978962761 >> v2:172.21.15.139:3300/0 conn(0x7fd08806c3a0 0x7fd0880cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:35.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.186+0000 7fd08ccfa640 1 -- 172.21.15.98:0/1978962761 >> v1:172.21.15.139:6789/0 conn(0x7fd08815c550 legacy=0x7fd08806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:35.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.186+0000 7fd08ccfa640 1 -- 172.21.15.98:0/1978962761 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd08806dbb0 con 0x7fd0880cf900 2024-09-19T08:46:35.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.186+0000 7fd08ccfa640 1 -- 172.21.15.98:0/1978962761 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fd080001a00 con 0x7fd0880cf900 2024-09-19T08:46:35.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.186+0000 7fd08ccfa640 1 -- 172.21.15.98:0/1978962761 >> v1:172.21.15.98:6789/0 conn(0x7fd0880cf900 legacy=0x7fd0880723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:35.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.186+0000 7fd08ccfa640 1 --2- 172.21.15.98:0/1978962761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0740039c0 0x7fd074003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:35.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.186+0000 7fd08ccfa640 1 --2- 172.21.15.98:0/1978962761 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd074004810 0x7fd074004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:35.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.186+0000 7fd08ccfa640 1 --2- 172.21.15.98:0/1978962761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0740051d0 0x7fd07400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:35.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.186+0000 7fd08ccfa640 1 -- 172.21.15.98:0/1978962761 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd08806d8d0 con 0x7fd0740039c0 2024-09-19T08:46:35.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.186+0000 7fd08ccfa640 1 -- 172.21.15.98:0/1978962761 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd07400c070 con 0x7fd0740051d0 2024-09-19T08:46:35.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.186+0000 7fd08ccfa640 1 -- 172.21.15.98:0/1978962761 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd07400c270 con 0x7fd074004810 2024-09-19T08:46:35.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.186+0000 7fd08dcfc640 1 --2- 172.21.15.98:0/1978962761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0740039c0 0x7fd074003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:35.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.186+0000 7fd08e4fd640 1 --2- 172.21.15.98:0/1978962761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0740051d0 0x7fd07400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:35.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.186+0000 7fd08dcfc640 1 -- 172.21.15.98:0/1978962761 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd074004810 msgr2=0x7fd074004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:35.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.186+0000 7fd08dcfc640 1 --2- 172.21.15.98:0/1978962761 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd074004810 0x7fd074004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:35.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.186+0000 7fd08dcfc640 1 -- 172.21.15.98:0/1978962761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0740051d0 msgr2=0x7fd07400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:35.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.186+0000 7fd08dcfc640 1 --2- 172.21.15.98:0/1978962761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0740051d0 0x7fd07400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:35.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.186+0000 7fd08e4fd640 1 -- 172.21.15.98:0/1978962761 reap_dead start 2024-09-19T08:46:35.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.186+0000 7fd08dcfc640 1 -- 172.21.15.98:0/1978962761 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd074003530 con 0x7fd0740039c0 2024-09-19T08:46:35.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.186+0000 7fd08dcfc640 1 --2- 172.21.15.98:0/1978962761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0740039c0 0x7fd074003de0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fd084001780 tx=0x7fd08401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c12162bbe92fd34a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:35.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.186+0000 7fd08ccfa640 1 -- 172.21.15.98:0/1978962761 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0840013a0 con 0x7fd0740039c0 2024-09-19T08:46:35.189 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.186+0000 7fd08ccfa640 1 -- 172.21.15.98:0/1978962761 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0840052f0 con 0x7fd0740039c0 2024-09-19T08:46:35.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.186+0000 7fd08ccfa640 1 -- 172.21.15.98:0/1978962761 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd084003840 con 0x7fd0740039c0 2024-09-19T08:46:35.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.190+0000 7fd08ff60640 1 -- 172.21.15.98:0/1978962761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0740039c0 msgr2=0x7fd074003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:35.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.190+0000 7fd08ff60640 1 --2- 172.21.15.98:0/1978962761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0740039c0 0x7fd074003de0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fd084001780 tx=0x7fd08401c400 comp rx=0 tx=0).stop 2024-09-19T08:46:35.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.190+0000 7fd08ff60640 1 -- 172.21.15.98:0/1978962761 shutdown_connections 2024-09-19T08:46:35.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.190+0000 7fd08ff60640 1 --2- 172.21.15.98:0/1978962761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd0740039c0 0x7fd074003de0 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:35.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.190+0000 7fd08ff60640 1 -- 172.21.15.98:0/1978962761 >> 172.21.15.98:0/1978962761 conn(0x7fd0880b2010 msgr2=0x7fd0880b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:35.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.190+0000 7fd08ff60640 1 -- 172.21.15.98:0/1978962761 shutdown_connections 2024-09-19T08:46:35.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.190+0000 7fd08ff60640 1 -- 172.21.15.98:0/1978962761 wait complete. 2024-09-19T08:46:35.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.190+0000 7fd08ff60640 1 Processor -- start 2024-09-19T08:46:35.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.190+0000 7fd08ff60640 1 -- start start 2024-09-19T08:46:35.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.190+0000 7fd08ff60640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0740039c0 0x7fd08815cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:35.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.190+0000 7fd08ff60640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0881f9c30 0x7fd088151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:35.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.190+0000 7fd08ff60640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd08815f7a0 0x7fd088072780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:35.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.190+0000 7fd08ff60640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd0880ce9b0 con 0x7fd08815f7a0 2024-09-19T08:46:35.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.190+0000 7fd08ff60640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd08815cb40 con 0x7fd0740039c0 2024-09-19T08:46:35.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.190+0000 7fd08ff60640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd0880ceb30 con 0x7fd0881f9c30 2024-09-19T08:46:35.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.190+0000 7fd08e4fd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd08815f7a0 0x7fd088072780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:35.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.190+0000 7fd08e4fd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd08815f7a0 0x7fd088072780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:43386/0 (socket says 172.21.15.98:43386) 2024-09-19T08:46:35.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.190+0000 7fd08e4fd640 1 -- 172.21.15.98:0/2322126283 learned_addr learned my addr 172.21.15.98:0/2322126283 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:35.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.190+0000 7fd08dcfc640 1 --2- 172.21.15.98:0/2322126283 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0740039c0 0x7fd08815cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:35.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.190+0000 7fd08d4fb640 1 --2- 172.21.15.98:0/2322126283 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0881f9c30 0x7fd088151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:35.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.190+0000 7fd08e4fd640 1 -- 172.21.15.98:0/2322126283 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0881f9c30 msgr2=0x7fd088151560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:35.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.190+0000 7fd08e4fd640 1 --2- 172.21.15.98:0/2322126283 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0881f9c30 0x7fd088151560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:35.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.190+0000 7fd08e4fd640 1 -- 172.21.15.98:0/2322126283 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0740039c0 msgr2=0x7fd08815cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:35.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.190+0000 7fd08e4fd640 1 --2- 172.21.15.98:0/2322126283 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0740039c0 0x7fd08815cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:35.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.190+0000 7fd08e4fd640 1 -- 172.21.15.98:0/2322126283 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0881f91c0 con 0x7fd08815f7a0 2024-09-19T08:46:35.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.190+0000 7fd08e4fd640 1 --2- 172.21.15.98:0/2322126283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd08815f7a0 0x7fd088072780 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fd088073b20 tx=0x7fd08000b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:35.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.190+0000 7fd07e7fc640 1 -- 172.21.15.98:0/2322126283 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd080019070 con 0x7fd08815f7a0 2024-09-19T08:46:35.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.190+0000 7fd07e7fc640 1 -- 172.21.15.98:0/2322126283 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd08000b810 con 0x7fd08815f7a0 2024-09-19T08:46:35.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.190+0000 7fd08ff60640 1 -- 172.21.15.98:0/2322126283 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd08815fc20 con 0x7fd08815f7a0 2024-09-19T08:46:35.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.190+0000 7fd07e7fc640 1 -- 172.21.15.98:0/2322126283 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08000bbc0 con 0x7fd08815f7a0 2024-09-19T08:46:35.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.190+0000 7fd08ff60640 1 -- 172.21.15.98:0/2322126283 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0881f9430 con 0x7fd08815f7a0 2024-09-19T08:46:35.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.194+0000 7fd08ff60640 1 -- 172.21.15.98:0/2322126283 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0880ceec0 con 0x7fd08815f7a0 2024-09-19T08:46:35.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.194+0000 7fd07e7fc640 1 -- 172.21.15.98:0/2322126283 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7fd08001c070 con 0x7fd08815f7a0 2024-09-19T08:46:35.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.194+0000 7fd07e7fc640 1 --2- 172.21.15.98:0/2322126283 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd06c0680a0 0x7fd06c06a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:35.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.194+0000 7fd07e7fc640 1 -- 172.21.15.98:0/2322126283 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4971+0+0 (secure 0 0 0) 0x7fd0800842d0 con 0x7fd08815f7a0 2024-09-19T08:46:35.195 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.194+0000 7fd08dcfc640 1 --2- 172.21.15.98:0/2322126283 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd06c0680a0 0x7fd06c06a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:35.196 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.194+0000 7fd08dcfc640 1 --2- 172.21.15.98:0/2322126283 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd06c0680a0 0x7fd06c06a560 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd084001750 tx=0x7fd0840011a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:35.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.198+0000 7fd07e7fc640 1 -- 172.21.15.98:0/2322126283 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd080056700 con 0x7fd08815f7a0 2024-09-19T08:46:35.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.394+0000 7fd08ff60640 1 -- 172.21.15.98:0/2322126283 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0880b0020 con 0x7fd08815f7a0 2024-09-19T08:46:35.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.394+0000 7fd07e7fc640 1 -- 172.21.15.98:0/2322126283 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+3908 (secure 0 0 0) 0x7fd080056000 con 0x7fd08815f7a0 2024-09-19T08:46:35.395 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:46:35.395 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":85,"btime":"2024-09-19T08:46:29:427645+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6886,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3238157999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3238157999},{"type":"v1","addr":"172.21.15.139:6833","nonce":3238157999}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/209335177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":209335177},{"type":"v1","addr":"172.21.15.139:6835","nonce":209335177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8370,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6834/26914461","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":26914461},{"type":"v1","addr":"172.21.15.98:6834","nonce":26914461}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":8685,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/4034590852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6833","nonce":4034590852},{"type":"v1","addr":"172.21.15.98:6835","nonce":4034590852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[]} 2024-09-19T08:46:35.395 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 85 2024-09-19T08:46:35.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.398+0000 7fd08ff60640 1 -- 172.21.15.98:0/2322126283 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd06c0680a0 msgr2=0x7fd06c06a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:35.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.398+0000 7fd08ff60640 1 --2- 172.21.15.98:0/2322126283 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd06c0680a0 0x7fd06c06a560 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd084001750 tx=0x7fd0840011a0 comp rx=0 tx=0).stop 2024-09-19T08:46:35.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.398+0000 7fd08ff60640 1 -- 172.21.15.98:0/2322126283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd08815f7a0 msgr2=0x7fd088072780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:35.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.398+0000 7fd08ff60640 1 --2- 172.21.15.98:0/2322126283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd08815f7a0 0x7fd088072780 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fd088073b20 tx=0x7fd08000b0d0 comp rx=0 tx=0).stop 2024-09-19T08:46:35.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.398+0000 7fd08ff60640 1 -- 172.21.15.98:0/2322126283 shutdown_connections 2024-09-19T08:46:35.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.398+0000 7fd08ff60640 1 --2- 172.21.15.98:0/2322126283 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd06c0680a0 0x7fd06c06a560 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:35.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.398+0000 7fd08ff60640 1 --2- 172.21.15.98:0/2322126283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd08815f7a0 0x7fd088072780 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:35.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.398+0000 7fd08ff60640 1 --2- 172.21.15.98:0/2322126283 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd0881f9c30 0x7fd088151560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:35.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.398+0000 7fd08ff60640 1 --2- 172.21.15.98:0/2322126283 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd0740039c0 0x7fd08815cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:35.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.398+0000 7fd08ff60640 1 -- 172.21.15.98:0/2322126283 >> 172.21.15.98:0/2322126283 conn(0x7fd0880b2010 msgr2=0x7fd08815ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:35.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.398+0000 7fd08ff60640 1 -- 172.21.15.98:0/2322126283 shutdown_connections 2024-09-19T08:46:35.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.398+0000 7fd08ff60640 1 -- 172.21.15.98:0/2322126283 wait complete. 2024-09-19T08:46:35.413 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-19T08:46:35.414 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-19T08:46:35.414 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-19T08:46:35.414 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:46:35.414 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-19T08:46:35.414 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-19T08:46:35.414 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-19T08:46:35.415 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:46:35.415 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-19T08:46:35.415 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-19T08:46:35.415 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-19T08:46:35.415 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:46:35.415 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-19T08:46:35.415 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-19T08:46:35.416 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-19T08:46:35.416 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:46:35.427 INFO:tasks.ceph.mds.a:Stopped 2024-09-19T08:46:35.427 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-19T08:46:35.428 INFO:tasks.ceph.mds.c:Stopped 2024-09-19T08:46:35.428 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-19T08:46:35.430 INFO:tasks.ceph.mds.b:Stopped 2024-09-19T08:46:35.430 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-19T08:46:35.431 INFO:tasks.ceph.mds.d:Stopped 2024-09-19T08:46:35.431 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-19T08:46:35.473 INFO:tasks.ceph.mds.a:Started 2024-09-19T08:46:35.473 DEBUG:teuthology.parallel:result is None 2024-09-19T08:46:35.474 INFO:tasks.ceph.mds.c:Started 2024-09-19T08:46:35.474 DEBUG:teuthology.parallel:result is None 2024-09-19T08:46:35.479 INFO:tasks.ceph.mds.b:Started 2024-09-19T08:46:35.480 DEBUG:teuthology.parallel:result is None 2024-09-19T08:46:35.480 INFO:tasks.ceph.mds.d:Started 2024-09-19T08:46:35.480 DEBUG:teuthology.parallel:result is None 2024-09-19T08:46:35.481 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-19T08:46:35.538 INFO:tasks.ceph.mds.a.smithi098.stdout:starting mds.a at 2024-09-19T08:46:35.538 INFO:tasks.ceph.mds.c.smithi098.stdout:starting mds.c at 2024-09-19T08:46:35.556 INFO:tasks.ceph.mds.b.smithi139.stdout:starting mds.b at 2024-09-19T08:46:35.556 INFO:tasks.ceph.mds.d.smithi139.stdout:starting mds.d at 2024-09-19T08:46:35.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.590+0000 7fd959564640 1 Processor -- start 2024-09-19T08:46:35.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.590+0000 7fd959564640 1 -- start start 2024-09-19T08:46:35.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.590+0000 7fd959564640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd9540d1d70 0x7fd9540d1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:35.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.590+0000 7fd959564640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9540b6200 0x7fd9540b6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:35.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.590+0000 7fd959564640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9540b8060 con 0x7fd954147530 2024-09-19T08:46:35.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.590+0000 7fd959564640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fd9540b6ba0 con 0x7fd9540d1d70 2024-09-19T08:46:35.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.590+0000 7fd959564640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd9540d2230 con 0x7fd9540b6200 2024-09-19T08:46:35.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.590+0000 7fd9527fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fd9540d1d70 0x7fd9540d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:35.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.590+0000 7fd952ffd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fd954147530 0x7fd95415ec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36404/0 (socket says 172.21.15.98:36404) 2024-09-19T08:46:35.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.590+0000 7fd952ffd640 1 -- 172.21.15.98:0/1030799226 learned_addr learned my addr 172.21.15.98:0/1030799226 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:35.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.590+0000 7fd9537fe640 1 --2- 172.21.15.98:0/1030799226 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9540b6200 0x7fd9540b6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:35.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.590+0000 7fd9527fc640 1 -- 172.21.15.98:0/1030799226 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9540b6200 msgr2=0x7fd9540b6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:35.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.590+0000 7fd9527fc640 1 --2- 172.21.15.98:0/1030799226 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9540b6200 0x7fd9540b6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:35.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.590+0000 7fd9527fc640 1 -- 172.21.15.98:0/1030799226 >> v1:172.21.15.98:6789/0 conn(0x7fd954147530 legacy=0x7fd95415ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:35.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.590+0000 7fd9527fc640 1 -- 172.21.15.98:0/1030799226 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9540b8340 con 0x7fd9540d1d70 2024-09-19T08:46:35.590 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.590+0000 7fd951ffb640 1 -- 172.21.15.98:0/1030799226 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3173536833 0 0) 0x7fd9540b8060 con 0x7fd954147530 2024-09-19T08:46:35.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.590+0000 7fd9527fc640 1 --2- 172.21.15.98:0/1030799226 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd9540d1d70 0x7fd9540d1510 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fd9480097c0 tx=0x7fd94801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9999a88087a6dfbe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:35.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.590+0000 7fd951ffb640 1 -- 172.21.15.98:0/1030799226 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9480a7070 con 0x7fd9540d1d70 2024-09-19T08:46:35.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.590+0000 7fd951ffb640 1 -- 172.21.15.98:0/1030799226 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9480b0070 con 0x7fd9540d1d70 2024-09-19T08:46:35.591 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.590+0000 7fd951ffb640 1 -- 172.21.15.98:0/1030799226 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9480ab890 con 0x7fd9540d1d70 2024-09-19T08:46:35.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.590+0000 7fd959564640 1 -- 172.21.15.98:0/1030799226 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd9540d1d70 msgr2=0x7fd9540d1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:35.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.590+0000 7fd959564640 1 --2- 172.21.15.98:0/1030799226 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd9540d1d70 0x7fd9540d1510 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fd9480097c0 tx=0x7fd94801c470 comp rx=0 tx=0).stop 2024-09-19T08:46:35.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.590+0000 7fd959564640 1 -- 172.21.15.98:0/1030799226 shutdown_connections 2024-09-19T08:46:35.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.590+0000 7fd959564640 1 --2- 172.21.15.98:0/1030799226 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9540b6200 0x7fd9540b6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:35.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.590+0000 7fd959564640 1 --2- 172.21.15.98:0/1030799226 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd9540d1d70 0x7fd9540d1510 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:35.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.594+0000 7fd959564640 1 -- 172.21.15.98:0/1030799226 >> 172.21.15.98:0/1030799226 conn(0x7fd954075070 msgr2=0x7fd9540722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:35.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.594+0000 7fd959564640 1 -- 172.21.15.98:0/1030799226 shutdown_connections 2024-09-19T08:46:35.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.594+0000 7fd959564640 1 -- 172.21.15.98:0/1030799226 wait complete. 2024-09-19T08:46:35.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.594+0000 7fd959564640 1 Processor -- start 2024-09-19T08:46:35.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.594+0000 7fd959564640 1 -- start start 2024-09-19T08:46:35.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.594+0000 7fd959564640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9540b6200 0x7fd9541581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:35.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.594+0000 7fd959564640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9540d1d70 0x7fd95413ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:35.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.594+0000 7fd959564640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd954147530 0x7fd95413cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:35.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.594+0000 7fd959564640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd954147a90 con 0x7fd954147530 2024-09-19T08:46:35.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.594+0000 7fd959564640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd954147910 con 0x7fd9540b6200 2024-09-19T08:46:35.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.594+0000 7fd959564640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd9540b9630 con 0x7fd9540d1d70 2024-09-19T08:46:35.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.594+0000 7fd9537fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd954147530 0x7fd95413cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:35.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.594+0000 7fd9537fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd954147530 0x7fd95413cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:43428/0 (socket says 172.21.15.98:43428) 2024-09-19T08:46:35.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.594+0000 7fd9537fe640 1 -- 172.21.15.98:0/1400260347 learned_addr learned my addr 172.21.15.98:0/1400260347 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:35.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.594+0000 7fd9527fc640 1 --2- 172.21.15.98:0/1400260347 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9540d1d70 0x7fd95413ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:35.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.594+0000 7fd952ffd640 1 --2- 172.21.15.98:0/1400260347 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9540b6200 0x7fd9541581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:35.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.594+0000 7fd9537fe640 1 -- 172.21.15.98:0/1400260347 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9540d1d70 msgr2=0x7fd95413ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:35.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.594+0000 7fd9537fe640 1 --2- 172.21.15.98:0/1400260347 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9540d1d70 0x7fd95413ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:35.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.594+0000 7fd9537fe640 1 -- 172.21.15.98:0/1400260347 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9540b6200 msgr2=0x7fd9541581b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:35.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.594+0000 7fd9537fe640 1 --2- 172.21.15.98:0/1400260347 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9540b6200 0x7fd9541581b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:35.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.594+0000 7fd9537fe640 1 -- 172.21.15.98:0/1400260347 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd95413daa0 con 0x7fd954147530 2024-09-19T08:46:35.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.594+0000 7fd952ffd640 1 --2- 172.21.15.98:0/1400260347 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9540b6200 0x7fd9541581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:35.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.594+0000 7fd9537fe640 1 --2- 172.21.15.98:0/1400260347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd954147530 0x7fd95413cb90 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fd94400d800 tx=0x7fd94400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:35.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.594+0000 7fd93b7fe640 1 -- 172.21.15.98:0/1400260347 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94400b6c0 con 0x7fd954147530 2024-09-19T08:46:35.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.594+0000 7fd93b7fe640 1 -- 172.21.15.98:0/1400260347 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd944019070 con 0x7fd954147530 2024-09-19T08:46:35.596 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.594+0000 7fd93b7fe640 1 -- 172.21.15.98:0/1400260347 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94400baa0 con 0x7fd954147530 2024-09-19T08:46:35.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.594+0000 7fd959564640 1 -- 172.21.15.98:0/1400260347 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd95413d7c0 con 0x7fd954147530 2024-09-19T08:46:35.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.594+0000 7fd959564640 1 -- 172.21.15.98:0/1400260347 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd95415aa90 con 0x7fd954147530 2024-09-19T08:46:35.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.594+0000 7fd959564640 1 -- 172.21.15.98:0/1400260347 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9540b6660 con 0x7fd954147530 2024-09-19T08:46:35.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.598+0000 7fd93b7fe640 1 -- 172.21.15.98:0/1400260347 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7fd94400bc40 con 0x7fd954147530 2024-09-19T08:46:35.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.598+0000 7fd93b7fe640 1 --2- 172.21.15.98:0/1400260347 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd9400680a0 0x7fd94006a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:35.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.598+0000 7fd93b7fe640 1 -- 172.21.15.98:0/1400260347 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4971+0+0 (secure 0 0 0) 0x7fd9440849e0 con 0x7fd954147530 2024-09-19T08:46:35.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.598+0000 7fd952ffd640 1 --2- 172.21.15.98:0/1400260347 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd9400680a0 0x7fd94006a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:35.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.598+0000 7fd952ffd640 1 --2- 172.21.15.98:0/1400260347 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd9400680a0 0x7fd94006a560 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd94c007fd0 tx=0x7fd94c005b80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:35.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.602+0000 7fd93b7fe640 1 -- 172.21.15.98:0/1400260347 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd944056e10 con 0x7fd954147530 2024-09-19T08:46:35.775 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:35.774+0000 7fd959564640 1 -- 172.21.15.98:0/1400260347 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fd954148040 con 0x7fd954147530 2024-09-19T08:46:36.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.814+0000 7fd93b7fe640 1 -- 172.21.15.98:0/1400260347 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v154) ==== 97+0+0 (secure 0 0 0) 0x7fd944056710 con 0x7fd954147530 2024-09-19T08:46:36.813 INFO:teuthology.orchestra.run.smithi098.stderr: removed all blocklist entries 2024-09-19T08:46:36.816 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.818+0000 7fd959564640 1 -- 172.21.15.98:0/1400260347 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd9400680a0 msgr2=0x7fd94006a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:36.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.818+0000 7fd959564640 1 --2- 172.21.15.98:0/1400260347 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd9400680a0 0x7fd94006a560 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd94c007fd0 tx=0x7fd94c005b80 comp rx=0 tx=0).stop 2024-09-19T08:46:36.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.818+0000 7fd959564640 1 -- 172.21.15.98:0/1400260347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd954147530 msgr2=0x7fd95413cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:36.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.818+0000 7fd959564640 1 --2- 172.21.15.98:0/1400260347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd954147530 0x7fd95413cb90 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fd94400d800 tx=0x7fd94400dcc0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.818+0000 7fd959564640 1 -- 172.21.15.98:0/1400260347 shutdown_connections 2024-09-19T08:46:36.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.818+0000 7fd959564640 1 --2- 172.21.15.98:0/1400260347 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd9400680a0 0x7fd94006a560 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.818+0000 7fd959564640 1 --2- 172.21.15.98:0/1400260347 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd954147530 0x7fd95413cb90 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.818+0000 7fd959564640 1 --2- 172.21.15.98:0/1400260347 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd9540d1d70 0x7fd95413ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.818+0000 7fd959564640 1 --2- 172.21.15.98:0/1400260347 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd9540b6200 0x7fd9541581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.818+0000 7fd959564640 1 -- 172.21.15.98:0/1400260347 >> 172.21.15.98:0/1400260347 conn(0x7fd954075070 msgr2=0x7fd9540d4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:36.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.818+0000 7fd959564640 1 -- 172.21.15.98:0/1400260347 shutdown_connections 2024-09-19T08:46:36.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.818+0000 7fd959564640 1 -- 172.21.15.98:0/1400260347 wait complete. 2024-09-19T08:46:36.836 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-19T08:46:36.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.962+0000 7effa5a1c640 1 Processor -- start 2024-09-19T08:46:36.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.962+0000 7effa5a1c640 1 -- start start 2024-09-19T08:46:36.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.962+0000 7effa5a1c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7effa00ce8b0 0x7effa01693e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:36.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.962+0000 7effa5a1c640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effa015fcf0 con 0x7effa0158e40 2024-09-19T08:46:36.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.962+0000 7effa5a1c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effa015ef30 con 0x7effa006c3a0 2024-09-19T08:46:36.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.962+0000 7effa5a1c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7effa015f210 con 0x7effa00ce8b0 2024-09-19T08:46:36.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.962+0000 7eff9effd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7effa0158e40 0x7effa006bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36418/0 (socket says 172.21.15.98:36418) 2024-09-19T08:46:36.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.962+0000 7eff9effd640 1 -- 172.21.15.98:0/4178794045 learned_addr learned my addr 172.21.15.98:0/4178794045 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:36.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.962+0000 7eff9e7fc640 1 -- 172.21.15.98:0/4178794045 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 626592077 0 0) 0x7effa015fcf0 con 0x7effa0158e40 2024-09-19T08:46:36.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.962+0000 7eff9e7fc640 1 -- 172.21.15.98:0/4178794045 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7eff8c003530 con 0x7effa0158e40 2024-09-19T08:46:36.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.962+0000 7eff9f7fe640 1 --2- 172.21.15.98:0/4178794045 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7effa00ce8b0 0x7effa01693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:36.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.962+0000 7eff9e7fc640 1 -- 172.21.15.98:0/4178794045 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1368556657 0 0) 0x7eff8c003530 con 0x7effa0158e40 2024-09-19T08:46:36.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.962+0000 7eff9e7fc640 1 -- 172.21.15.98:0/4178794045 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7effa00ce8b0 msgr2=0x7effa01693e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:36.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.962+0000 7eff9e7fc640 1 --2- 172.21.15.98:0/4178794045 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7effa00ce8b0 0x7effa01693e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.962+0000 7eff9e7fc640 1 -- 172.21.15.98:0/4178794045 >> v1:172.21.15.139:6789/0 conn(0x7effa006c3a0 legacy=0x7effa00ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:36.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.962+0000 7eff9e7fc640 1 -- 172.21.15.98:0/4178794045 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa015f4f0 con 0x7effa0158e40 2024-09-19T08:46:36.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.962+0000 7eff9e7fc640 1 -- 172.21.15.98:0/4178794045 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7eff90001680 con 0x7effa0158e40 2024-09-19T08:46:36.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.962+0000 7eff9e7fc640 1 -- 172.21.15.98:0/4178794045 >> v1:172.21.15.98:6789/0 conn(0x7effa0158e40 legacy=0x7effa006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:36.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.962+0000 7eff9e7fc640 1 --2- 172.21.15.98:0/4178794045 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7effa00ce8b0 0x7eff8c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:36.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.962+0000 7eff9e7fc640 1 --2- 172.21.15.98:0/4178794045 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff8c003f70 0x7eff8c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:36.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.962+0000 7eff9e7fc640 1 --2- 172.21.15.98:0/4178794045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff8c004d50 0x7eff8c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:36.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.962+0000 7eff9e7fc640 1 -- 172.21.15.98:0/4178794045 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7effa015f210 con 0x7eff8c004d50 2024-09-19T08:46:36.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.962+0000 7eff9e7fc640 1 -- 172.21.15.98:0/4178794045 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c00bb40 con 0x7eff8c003f70 2024-09-19T08:46:36.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.962+0000 7eff9e7fc640 1 -- 172.21.15.98:0/4178794045 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7eff8c00bd40 con 0x7effa00ce8b0 2024-09-19T08:46:36.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.962+0000 7eff9f7fe640 1 --2- 172.21.15.98:0/4178794045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff8c004d50 0x7eff8c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:36.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.962+0000 7eff9effd640 1 --2- 172.21.15.98:0/4178794045 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7effa00ce8b0 0x7eff8c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:36.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.962+0000 7eff97fff640 1 --2- 172.21.15.98:0/4178794045 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff8c003f70 0x7eff8c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:36.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.962+0000 7eff9f7fe640 1 -- 172.21.15.98:0/4178794045 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7effa00ce8b0 msgr2=0x7eff8c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:36.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.962+0000 7eff9f7fe640 1 --2- 172.21.15.98:0/4178794045 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7effa00ce8b0 0x7eff8c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.962+0000 7eff9f7fe640 1 -- 172.21.15.98:0/4178794045 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff8c003f70 msgr2=0x7eff8c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:36.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.962+0000 7eff9f7fe640 1 --2- 172.21.15.98:0/4178794045 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff8c003f70 0x7eff8c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.962+0000 7eff9f7fe640 1 -- 172.21.15.98:0/4178794045 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff8c003530 con 0x7eff8c004d50 2024-09-19T08:46:36.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.962+0000 7eff97fff640 1 --2- 172.21.15.98:0/4178794045 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7eff8c003f70 0x7eff8c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:36.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.966+0000 7eff9f7fe640 1 --2- 172.21.15.98:0/4178794045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff8c004d50 0x7eff8c0051d0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7eff98009fc0 tx=0x7eff9801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8167903366e8f88 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:36.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.966+0000 7eff9e7fc640 1 -- 172.21.15.98:0/4178794045 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff980b2070 con 0x7eff8c004d50 2024-09-19T08:46:36.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.966+0000 7eff9e7fc640 1 -- 172.21.15.98:0/4178794045 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff980a5c90 con 0x7eff8c004d50 2024-09-19T08:46:36.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.966+0000 7eff9e7fc640 1 -- 172.21.15.98:0/4178794045 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff98009c10 con 0x7eff8c004d50 2024-09-19T08:46:36.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.966+0000 7effa5a1c640 1 -- 172.21.15.98:0/4178794045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff8c004d50 msgr2=0x7eff8c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:36.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.966+0000 7effa5a1c640 1 --2- 172.21.15.98:0/4178794045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7eff8c004d50 0x7eff8c0051d0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7eff98009fc0 tx=0x7eff9801cc80 comp rx=0 tx=0).stop 2024-09-19T08:46:36.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.966+0000 7eff9f7fe640 1 -- 172.21.15.98:0/4178794045 reap_dead start 2024-09-19T08:46:36.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.966+0000 7effa5a1c640 1 -- 172.21.15.98:0/4178794045 shutdown_connections 2024-09-19T08:46:36.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.966+0000 7effa5a1c640 1 -- 172.21.15.98:0/4178794045 >> 172.21.15.98:0/4178794045 conn(0x7effa00b2010 msgr2=0x7effa00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:36.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.966+0000 7effa5a1c640 1 -- 172.21.15.98:0/4178794045 shutdown_connections 2024-09-19T08:46:36.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.966+0000 7effa5a1c640 1 -- 172.21.15.98:0/4178794045 wait complete. 2024-09-19T08:46:36.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.966+0000 7effa5a1c640 1 Processor -- start 2024-09-19T08:46:36.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.966+0000 7effa5a1c640 1 -- start start 2024-09-19T08:46:36.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.966+0000 7effa5a1c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7effa0152820 0x7effa0152c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:36.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.966+0000 7effa5a1c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7effa01f8380 0x7effa0144b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:36.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.966+0000 7effa5a1c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effa01f8800 0x7effa0145090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:36.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.966+0000 7effa5a1c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7effa00cee20 con 0x7effa0152820 2024-09-19T08:46:36.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.966+0000 7effa5a1c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7effa0159220 con 0x7effa01f8800 2024-09-19T08:46:36.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.966+0000 7effa5a1c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7effa00b4570 con 0x7effa01f8380 2024-09-19T08:46:36.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.966+0000 7eff9effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7effa0152820 0x7effa0152c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:36.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.966+0000 7eff9effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7effa0152820 0x7effa0152c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:43442/0 (socket says 172.21.15.98:43442) 2024-09-19T08:46:36.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.966+0000 7eff9effd640 1 -- 172.21.15.98:0/1485301302 learned_addr learned my addr 172.21.15.98:0/1485301302 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:36.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.970+0000 7eff97fff640 1 --2- 172.21.15.98:0/1485301302 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7effa01f8380 0x7effa0144b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:36.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.970+0000 7eff9f7fe640 1 --2- 172.21.15.98:0/1485301302 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effa01f8800 0x7effa0145090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:36.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.970+0000 7eff9effd640 1 -- 172.21.15.98:0/1485301302 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7effa01f8380 msgr2=0x7effa0144b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:36.970 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.970+0000 7eff9effd640 1 --2- 172.21.15.98:0/1485301302 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7effa01f8380 0x7effa0144b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.970+0000 7eff9effd640 1 -- 172.21.15.98:0/1485301302 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effa01f8800 msgr2=0x7effa0145090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:36.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.970+0000 7eff9effd640 1 --2- 172.21.15.98:0/1485301302 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effa01f8800 0x7effa0145090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:36.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.970+0000 7eff9effd640 1 -- 172.21.15.98:0/1485301302 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa0153140 con 0x7effa0152820 2024-09-19T08:46:36.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.970+0000 7eff9f7fe640 1 --2- 172.21.15.98:0/1485301302 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effa01f8800 0x7effa0145090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:36.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.970+0000 7eff9effd640 1 --2- 172.21.15.98:0/1485301302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7effa0152820 0x7effa0152c00 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7eff9000ead0 tx=0x7eff9000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:36.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.970+0000 7eff9cff9640 1 -- 172.21.15.98:0/1485301302 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9000c850 con 0x7effa0152820 2024-09-19T08:46:36.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.970+0000 7eff9cff9640 1 -- 172.21.15.98:0/1485301302 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff90018070 con 0x7effa0152820 2024-09-19T08:46:36.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.970+0000 7effa5a1c640 1 -- 172.21.15.98:0/1485301302 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa0154c80 con 0x7effa0152820 2024-09-19T08:46:36.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.970+0000 7eff9cff9640 1 -- 172.21.15.98:0/1485301302 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9000cc50 con 0x7effa0152820 2024-09-19T08:46:36.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.970+0000 7effa5a1c640 1 -- 172.21.15.98:0/1485301302 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa0155240 con 0x7effa0152820 2024-09-19T08:46:36.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.970+0000 7effa5a1c640 1 -- 172.21.15.98:0/1485301302 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effa00ce1a0 con 0x7effa0152820 2024-09-19T08:46:36.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.970+0000 7eff9cff9640 1 -- 172.21.15.98:0/1485301302 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7eff90013420 con 0x7effa0152820 2024-09-19T08:46:36.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.970+0000 7eff9cff9640 1 --2- 172.21.15.98:0/1485301302 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7eff800680d0 0x7eff8006a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:36.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.970+0000 7eff9cff9640 1 -- 172.21.15.98:0/1485301302 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 4885+0+0 (secure 0 0 0) 0x7eff90084330 con 0x7effa0152820 2024-09-19T08:46:36.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.970+0000 7eff97fff640 1 --2- 172.21.15.98:0/1485301302 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7eff800680d0 0x7eff8006a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:36.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.970+0000 7eff97fff640 1 --2- 172.21.15.98:0/1485301302 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7eff800680d0 0x7eff8006a590 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7effa0073d90 tx=0x7eff88009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:36.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:36.974+0000 7eff9cff9640 1 -- 172.21.15.98:0/1485301302 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7eff90056760 con 0x7effa0152820 2024-09-19T08:46:37.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.182+0000 7effa5a1c640 1 -- 172.21.15.98:0/1485301302 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7effa015c520 con 0x7effa0152820 2024-09-19T08:46:37.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.182+0000 7eff9cff9640 1 -- 172.21.15.98:0/1485301302 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v24) ==== 80+0+5304 (secure 0 0 0) 0x7eff90056060 con 0x7effa0152820 2024-09-19T08:46:37.182 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:46:37.183 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:46:37.183 INFO:teuthology.orchestra.run.smithi098.stdout: "auth_dump": [ 2024-09-19T08:46:37.183 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:37.183 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.a", 2024-09-19T08:46:37.183 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmuujGBRAAa2k/kW68NJI6nUKx+W+G2Q==", 2024-09-19T08:46:37.183 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:37.183 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:46:37.183 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:46:37.183 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:46:37.183 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:46:37.183 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:37.183 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:37.184 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:37.184 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.b", 2024-09-19T08:46:37.184 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmFoojFBAAaOZPpa41fdjBPEn3CGsJnA==", 2024-09-19T08:46:37.184 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:37.184 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:46:37.184 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:46:37.184 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:46:37.184 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:46:37.184 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:37.184 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:37.184 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:37.184 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.c", 2024-09-19T08:46:37.184 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmcQcDDhAAF0QJMlljMH+DiHfWW+/Sog==", 2024-09-19T08:46:37.185 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:37.185 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:46:37.185 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:46:37.185 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:46:37.185 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:46:37.185 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:37.185 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:37.185 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:37.185 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.d", 2024-09-19T08:46:37.185 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmwWmqHBAAf15VcrdeHf8AzQjGsN9LoA==", 2024-09-19T08:46:37.185 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:37.185 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:46:37.185 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:46:37.186 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:46:37.186 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:46:37.186 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:37.186 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:37.186 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:37.186 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.0", 2024-09-19T08:46:37.186 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDC4utm9PjNKxAAa6SG9AE4smPiu4azRAFKvg==", 2024-09-19T08:46:37.186 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:37.186 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:46:37.186 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:46:37.186 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:46:37.186 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:37.187 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:37.187 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:37.187 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.1", 2024-09-19T08:46:37.187 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDG4utmHy6UCxAAN6hpG4DJEp81iG/KoloTvQ==", 2024-09-19T08:46:37.187 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:37.187 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:46:37.187 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:46:37.187 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:46:37.187 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:37.187 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:37.187 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:37.187 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.2", 2024-09-19T08:46:37.187 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDJ4utmUVP4GhAAJNAL30sEAseoDG2fE6YbZQ==", 2024-09-19T08:46:37.188 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:37.188 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:46:37.188 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:46:37.188 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:46:37.188 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:37.188 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:37.188 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:37.188 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.3", 2024-09-19T08:46:37.188 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDM4utmTKgoLRAAiCrPRLML+wvjOt6RhKay3w==", 2024-09-19T08:46:37.188 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:37.188 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:46:37.188 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:46:37.188 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:46:37.189 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:37.189 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:37.189 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:37.189 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.4", 2024-09-19T08:46:37.189 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDS4utmAf6bMhAAI3uENuFoXLicUeuspgFjYA==", 2024-09-19T08:46:37.189 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:37.189 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:46:37.189 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:46:37.189 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:46:37.189 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:37.189 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:37.189 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:37.189 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.5", 2024-09-19T08:46:37.190 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDW4utmRjxwEhAAF1A2GtrZXVx9BlmjHjhUDQ==", 2024-09-19T08:46:37.190 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:37.190 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:46:37.190 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:46:37.190 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:46:37.190 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:37.190 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:37.190 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:37.190 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.6", 2024-09-19T08:46:37.190 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDZ4utmSk/XIRAAROSfzGL7smR88ZkDSx5jAg==", 2024-09-19T08:46:37.190 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:37.190 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:46:37.190 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:46:37.190 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:46:37.191 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:37.191 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:37.191 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:37.191 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.7", 2024-09-19T08:46:37.191 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDc4utmXfpbMxAAqEM4I/ACFvfu/sMA8bMcrQ==", 2024-09-19T08:46:37.191 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:37.191 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:46:37.191 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:46:37.191 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:46:37.191 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:37.191 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:37.191 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:37.191 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.0", 2024-09-19T08:46:37.191 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utm0vNpIhAA4S1E8dFZ0+ibmSKhmZ+bIA==", 2024-09-19T08:46:37.192 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:37.192 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:46:37.192 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow r", 2024-09-19T08:46:37.192 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-19T08:46:37.192 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:37.192 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:37.192 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:37.192 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.1", 2024-09-19T08:46:37.192 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmx7xyKhAAC84lykPEb01MiypXKQVREQ==", 2024-09-19T08:46:37.192 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:37.192 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:46:37.192 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-19T08:46:37.192 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-19T08:46:37.193 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-19T08:46:37.193 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:37.193 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:37.193 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:37.193 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.2", 2024-09-19T08:46:37.193 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmteOQJxAAtTF2Py21ZE2WRQjeQSrkKQ==", 2024-09-19T08:46:37.193 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:37.193 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:46:37.193 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-19T08:46:37.193 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-19T08:46:37.193 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-19T08:46:37.193 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:37.193 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:37.193 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:37.193 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.3", 2024-09-19T08:46:37.194 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmKN92LxAAMqraZOWRT6IIfx5AuDWS2g==", 2024-09-19T08:46:37.194 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:37.194 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:46:37.194 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-19T08:46:37.194 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-19T08:46:37.194 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-19T08:46:37.194 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:37.194 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:37.194 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:37.194 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.admin", 2024-09-19T08:46:37.194 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC+4utmLH97KhAA1DTYW5bTrGUG+7cb8+d7gA==", 2024-09-19T08:46:37.194 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:37.194 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-09-19T08:46:37.194 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:46:37.195 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:46:37.195 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:46:37.195 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:37.195 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:37.195 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:37.195 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mgr.x", 2024-09-19T08:46:37.195 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmbKuyAhAA0/XjHUIPOIV/YvNEhB8rEQ==", 2024-09-19T08:46:37.195 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:37.195 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-09-19T08:46:37.195 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile mgr", 2024-09-19T08:46:37.195 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:46:37.195 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:37.195 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:37.195 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:37.196 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mgr.y", 2024-09-19T08:46:37.196 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC+4utmU+ZrOxAAsvcwINH6qyxfN7N+rDrkqg==", 2024-09-19T08:46:37.196 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:46:37.196 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-09-19T08:46:37.196 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile mgr", 2024-09-19T08:46:37.196 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:46:37.196 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:37.196 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:37.196 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-19T08:46:37.196 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:46:37.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.186+0000 7effa5a1c640 1 -- 172.21.15.98:0/1485301302 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7eff800680d0 msgr2=0x7eff8006a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:37.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.186+0000 7effa5a1c640 1 --2- 172.21.15.98:0/1485301302 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7eff800680d0 0x7eff8006a590 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7effa0073d90 tx=0x7eff88009040 comp rx=0 tx=0).stop 2024-09-19T08:46:37.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.186+0000 7effa5a1c640 1 -- 172.21.15.98:0/1485301302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7effa0152820 msgr2=0x7effa0152c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:37.197 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.186+0000 7effa5a1c640 1 --2- 172.21.15.98:0/1485301302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7effa0152820 0x7effa0152c00 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7eff9000ead0 tx=0x7eff9000ef90 comp rx=0 tx=0).stop 2024-09-19T08:46:37.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.186+0000 7effa5a1c640 1 -- 172.21.15.98:0/1485301302 shutdown_connections 2024-09-19T08:46:37.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.186+0000 7effa5a1c640 1 --2- 172.21.15.98:0/1485301302 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7eff800680d0 0x7eff8006a590 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.186+0000 7effa5a1c640 1 --2- 172.21.15.98:0/1485301302 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7effa01f8800 0x7effa0145090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.186+0000 7effa5a1c640 1 --2- 172.21.15.98:0/1485301302 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7effa01f8380 0x7effa0144b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.186+0000 7effa5a1c640 1 --2- 172.21.15.98:0/1485301302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7effa0152820 0x7effa0152c00 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.186+0000 7effa5a1c640 1 -- 172.21.15.98:0/1485301302 >> 172.21.15.98:0/1485301302 conn(0x7effa00b2010 msgr2=0x7effa015e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:37.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.186+0000 7effa5a1c640 1 -- 172.21.15.98:0/1485301302 shutdown_connections 2024-09-19T08:46:37.198 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.186+0000 7effa5a1c640 1 -- 172.21.15.98:0/1485301302 wait complete. 2024-09-19T08:46:37.200 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-19T08:46:37.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.306+0000 7f18be65c640 1 Processor -- start 2024-09-19T08:46:37.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.310+0000 7f18be65c640 1 -- start start 2024-09-19T08:46:37.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.310+0000 7f18be65c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f18b806d880 0x7f18b80b3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:37.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.310+0000 7f18be65c640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18b815ea10 con 0x7f18b8141110 2024-09-19T08:46:37.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.310+0000 7f18be65c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18b815ecf0 con 0x7f18b8140d30 2024-09-19T08:46:37.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.310+0000 7f18be65c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f18b815e7c0 con 0x7f18b806d880 2024-09-19T08:46:37.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.310+0000 7f18bcbf9640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f18b8141110 0x7f18b815b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:36420/0 (socket says 172.21.15.98:36420) 2024-09-19T08:46:37.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.310+0000 7f18bcbf9640 1 -- 172.21.15.98:0/388854166 learned_addr learned my addr 172.21.15.98:0/388854166 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:37.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.310+0000 7f18b77fe640 1 --2- 172.21.15.98:0/388854166 >> v2:172.21.15.139:3300/0 conn(0x7f18b806d880 0x7f18b80b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:37.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.310+0000 7f18b6ffd640 1 -- 172.21.15.98:0/388854166 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2113063574 0 0) 0x7f18b815ea10 con 0x7f18b8141110 2024-09-19T08:46:37.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.310+0000 7f18b6ffd640 1 -- 172.21.15.98:0/388854166 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f18a0003530 con 0x7f18b8141110 2024-09-19T08:46:37.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.310+0000 7f18b6ffd640 1 -- 172.21.15.98:0/388854166 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2451807693 0 0) 0x7f18a0003530 con 0x7f18b8141110 2024-09-19T08:46:37.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.310+0000 7f18b6ffd640 1 -- 172.21.15.98:0/388854166 >> v2:172.21.15.139:3300/0 conn(0x7f18b806d880 msgr2=0x7f18b80b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:37.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.310+0000 7f18b6ffd640 1 --2- 172.21.15.98:0/388854166 >> v2:172.21.15.139:3300/0 conn(0x7f18b806d880 0x7f18b80b3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.310+0000 7f18b6ffd640 1 -- 172.21.15.98:0/388854166 >> v1:172.21.15.139:6789/0 conn(0x7f18b8140d30 legacy=0x7f18b806d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:37.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.310+0000 7f18b6ffd640 1 -- 172.21.15.98:0/388854166 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b815efd0 con 0x7f18b8141110 2024-09-19T08:46:37.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.310+0000 7f18b6ffd640 1 -- 172.21.15.98:0/388854166 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f18b0001a00 con 0x7f18b8141110 2024-09-19T08:46:37.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.310+0000 7f18b6ffd640 1 -- 172.21.15.98:0/388854166 >> v1:172.21.15.98:6789/0 conn(0x7f18b8141110 legacy=0x7f18b815b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:37.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.310+0000 7f18b6ffd640 1 --2- 172.21.15.98:0/388854166 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18a00039c0 0x7f18a0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:37.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.310+0000 7f18b6ffd640 1 --2- 172.21.15.98:0/388854166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18a0004810 0x7f18a0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:37.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.310+0000 7f18b6ffd640 1 --2- 172.21.15.98:0/388854166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18a00051d0 0x7f18a000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:37.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.310+0000 7f18b6ffd640 1 -- 172.21.15.98:0/388854166 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f18b815e7c0 con 0x7f18a0004810 2024-09-19T08:46:37.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.310+0000 7f18b6ffd640 1 -- 172.21.15.98:0/388854166 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f18a000c0a0 con 0x7f18a00051d0 2024-09-19T08:46:37.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.310+0000 7f18b6ffd640 1 -- 172.21.15.98:0/388854166 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f18a000c2a0 con 0x7f18a00039c0 2024-09-19T08:46:37.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.310+0000 7f18b77fe640 1 --2- 172.21.15.98:0/388854166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18a0004810 0x7f18a0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:37.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.310+0000 7f18bcbf9640 1 --2- 172.21.15.98:0/388854166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18a00051d0 0x7f18a000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:37.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.310+0000 7f18b77fe640 1 -- 172.21.15.98:0/388854166 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18a00039c0 msgr2=0x7f18a0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:37.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.310+0000 7f18b7fff640 1 --2- 172.21.15.98:0/388854166 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18a00039c0 0x7f18a0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:37.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.310+0000 7f18b77fe640 1 --2- 172.21.15.98:0/388854166 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18a00039c0 0x7f18a0003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.310+0000 7f18b77fe640 1 -- 172.21.15.98:0/388854166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18a00051d0 msgr2=0x7f18a000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:37.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.310+0000 7f18b77fe640 1 --2- 172.21.15.98:0/388854166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18a00051d0 0x7f18a000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.310+0000 7f18b77fe640 1 -- 172.21.15.98:0/388854166 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a0003530 con 0x7f18a0004810 2024-09-19T08:46:37.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.310+0000 7f18bcbf9640 1 --2- 172.21.15.98:0/388854166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18a00051d0 0x7f18a000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:37.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.310+0000 7f18bcbf9640 1 -- 172.21.15.98:0/388854166 reap_dead start 2024-09-19T08:46:37.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.314+0000 7f18b77fe640 1 --2- 172.21.15.98:0/388854166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18a0004810 0x7f18a0004c90 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f18ac00aae0 tx=0x7f18ac01dfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18093afa2bd99638 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:37.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.314+0000 7f18b6ffd640 1 -- 172.21.15.98:0/388854166 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac0b2070 con 0x7f18a0004810 2024-09-19T08:46:37.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.314+0000 7f18b6ffd640 1 -- 172.21.15.98:0/388854166 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18ac0ad600 con 0x7f18a0004810 2024-09-19T08:46:37.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.314+0000 7f18b6ffd640 1 -- 172.21.15.98:0/388854166 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac00b040 con 0x7f18a0004810 2024-09-19T08:46:37.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.314+0000 7f18be65c640 1 -- 172.21.15.98:0/388854166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18a0004810 msgr2=0x7f18a0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:37.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.314+0000 7f18be65c640 1 --2- 172.21.15.98:0/388854166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18a0004810 0x7f18a0004c90 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f18ac00aae0 tx=0x7f18ac01dfc0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.314+0000 7f18be65c640 1 -- 172.21.15.98:0/388854166 shutdown_connections 2024-09-19T08:46:37.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.314+0000 7f18be65c640 1 --2- 172.21.15.98:0/388854166 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18a0004810 0x7f18a0004c90 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.314+0000 7f18be65c640 1 -- 172.21.15.98:0/388854166 >> 172.21.15.98:0/388854166 conn(0x7f18b80b2010 msgr2=0x7f18b80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:37.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.314+0000 7f18be65c640 1 -- 172.21.15.98:0/388854166 shutdown_connections 2024-09-19T08:46:37.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.314+0000 7f18be65c640 1 -- 172.21.15.98:0/388854166 wait complete. 2024-09-19T08:46:37.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.314+0000 7f18be65c640 1 Processor -- start 2024-09-19T08:46:37.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.314+0000 7f18be65c640 1 -- start start 2024-09-19T08:46:37.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.314+0000 7f18be65c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18a0004810 0x7f18b813c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:37.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.314+0000 7f18be65c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18b813ee40 0x7f18b80cc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:37.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.314+0000 7f18be65c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18b81f3d00 0x7f18b80ceba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:37.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.314+0000 7f18be65c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f18b806d170 con 0x7f18b813ee40 2024-09-19T08:46:37.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.314+0000 7f18be65c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f18b80ce1a0 con 0x7f18a0004810 2024-09-19T08:46:37.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.314+0000 7f18be65c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f18b806d2f0 con 0x7f18b81f3d00 2024-09-19T08:46:37.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.314+0000 7f18b77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18b813ee40 0x7f18b80cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:37.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.314+0000 7f18b77fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18b813ee40 0x7f18b80cc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:43454/0 (socket says 172.21.15.98:43454) 2024-09-19T08:46:37.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.314+0000 7f18b77fe640 1 -- 172.21.15.98:0/1356577476 learned_addr learned my addr 172.21.15.98:0/1356577476 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:37.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.314+0000 7f18bcbf9640 1 --2- 172.21.15.98:0/1356577476 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18b81f3d00 0x7f18b80ceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:37.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.314+0000 7f18b7fff640 1 --2- 172.21.15.98:0/1356577476 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18a0004810 0x7f18b813c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:37.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.314+0000 7f18b77fe640 1 -- 172.21.15.98:0/1356577476 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18b81f3d00 msgr2=0x7f18b80ceba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:37.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.314+0000 7f18b77fe640 1 --2- 172.21.15.98:0/1356577476 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18b81f3d00 0x7f18b80ceba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.314+0000 7f18b77fe640 1 -- 172.21.15.98:0/1356577476 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18a0004810 msgr2=0x7f18b813c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:37.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.314+0000 7f18b77fe640 1 --2- 172.21.15.98:0/1356577476 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18a0004810 0x7f18b813c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.314+0000 7f18b77fe640 1 -- 172.21.15.98:0/1356577476 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b80cc870 con 0x7f18b813ee40 2024-09-19T08:46:37.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.314+0000 7f18b77fe640 1 --2- 172.21.15.98:0/1356577476 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18b813ee40 0x7f18b80cc330 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f18ac0b0040 tx=0x7f18ac01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:37.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.318+0000 7f18b4ff9640 1 -- 172.21.15.98:0/1356577476 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac0bb020 con 0x7f18b813ee40 2024-09-19T08:46:37.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.318+0000 7f18b4ff9640 1 -- 172.21.15.98:0/1356577476 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18ac0b3040 con 0x7f18b813ee40 2024-09-19T08:46:37.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.318+0000 7f18be65c640 1 -- 172.21.15.98:0/1356577476 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b8072300 con 0x7f18b813ee40 2024-09-19T08:46:37.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.318+0000 7f18b4ff9640 1 -- 172.21.15.98:0/1356577476 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac0b2070 con 0x7f18b813ee40 2024-09-19T08:46:37.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.318+0000 7f18be65c640 1 -- 172.21.15.98:0/1356577476 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b80728c0 con 0x7f18b813ee40 2024-09-19T08:46:37.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.318+0000 7f18be65c640 1 -- 172.21.15.98:0/1356577476 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18b806bc90 con 0x7f18b813ee40 2024-09-19T08:46:37.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.318+0000 7f18b4ff9640 1 -- 172.21.15.98:0/1356577476 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f18ac0be020 con 0x7f18b813ee40 2024-09-19T08:46:37.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.318+0000 7f18b4ff9640 1 --2- 172.21.15.98:0/1356577476 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f18a40680d0 0x7f18a406a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:37.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.318+0000 7f18b4ff9640 1 -- 172.21.15.98:0/1356577476 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 4885+0+0 (secure 0 0 0) 0x7f18ac11ee60 con 0x7f18b813ee40 2024-09-19T08:46:37.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.318+0000 7f18b7fff640 1 --2- 172.21.15.98:0/1356577476 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f18a40680d0 0x7f18a406a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:37.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.318+0000 7f18b7fff640 1 --2- 172.21.15.98:0/1356577476 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f18a40680d0 0x7f18a406a590 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f18b8069400 tx=0x7f18a8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:37.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.322+0000 7f18b4ff9640 1 -- 172.21.15.98:0/1356577476 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f18ac0c5070 con 0x7f18b813ee40 2024-09-19T08:46:37.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.490+0000 7f18be65c640 1 -- 172.21.15.98:0/1356577476 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f18b8159c80 con 0x7f18b813ee40 2024-09-19T08:46:37.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.490+0000 7f18b4ff9640 1 -- 172.21.15.98:0/1356577476 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v154) ==== 84+0+65 (secure 0 0 0) 0x7f18b80cc870 con 0x7f18b813ee40 2024-09-19T08:46:37.491 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:46:37.491 INFO:teuthology.orchestra.run.smithi098.stdout:[ 2024-09-19T08:46:37.491 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:37.491 INFO:teuthology.orchestra.run.smithi098.stdout: "poolnum": 1, 2024-09-19T08:46:37.491 INFO:teuthology.orchestra.run.smithi098.stdout: "poolname": ".mgr" 2024-09-19T08:46:37.491 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:37.492 INFO:teuthology.orchestra.run.smithi098.stdout:] 2024-09-19T08:46:37.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.494+0000 7f18be65c640 1 -- 172.21.15.98:0/1356577476 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f18a40680d0 msgr2=0x7f18a406a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:37.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.494+0000 7f18be65c640 1 --2- 172.21.15.98:0/1356577476 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f18a40680d0 0x7f18a406a590 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f18b8069400 tx=0x7f18a8009040 comp rx=0 tx=0).stop 2024-09-19T08:46:37.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.494+0000 7f18be65c640 1 -- 172.21.15.98:0/1356577476 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18b813ee40 msgr2=0x7f18b80cc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:37.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.494+0000 7f18be65c640 1 --2- 172.21.15.98:0/1356577476 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18b813ee40 0x7f18b80cc330 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f18ac0b0040 tx=0x7f18ac01ad60 comp rx=0 tx=0).stop 2024-09-19T08:46:37.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.494+0000 7f18be65c640 1 -- 172.21.15.98:0/1356577476 shutdown_connections 2024-09-19T08:46:37.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.494+0000 7f18be65c640 1 --2- 172.21.15.98:0/1356577476 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f18a40680d0 0x7f18a406a590 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.494+0000 7f18be65c640 1 --2- 172.21.15.98:0/1356577476 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18b81f3d00 0x7f18b80ceba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.494+0000 7f18be65c640 1 --2- 172.21.15.98:0/1356577476 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18b813ee40 0x7f18b80cc330 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.494+0000 7f18be65c640 1 --2- 172.21.15.98:0/1356577476 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18a0004810 0x7f18b813c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.494+0000 7f18be65c640 1 -- 172.21.15.98:0/1356577476 >> 172.21.15.98:0/1356577476 conn(0x7f18b80b2010 msgr2=0x7f18b80b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:37.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.494+0000 7f18be65c640 1 -- 172.21.15.98:0/1356577476 shutdown_connections 2024-09-19T08:46:37.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.494+0000 7f18be65c640 1 -- 172.21.15.98:0/1356577476 wait complete. 2024-09-19T08:46:37.510 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T08:46:37.510 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T08:46:37.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.630+0000 7fe11ef13640 1 Processor -- start 2024-09-19T08:46:37.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.630+0000 7fe11ef13640 1 -- start start 2024-09-19T08:46:37.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.630+0000 7fe11ef13640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe1180713b0 0x7fe11815d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:37.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.630+0000 7fe11ef13640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1180d2150 con 0x7fe1180d1d70 2024-09-19T08:46:37.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.630+0000 7fe11ef13640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe118071a90 con 0x7fe11806e5b0 2024-09-19T08:46:37.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.630+0000 7fe11ef13640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe118071840 con 0x7fe1180713b0 2024-09-19T08:46:37.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.630+0000 7fe11ccaf640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fe1180d1d70 0x7fe11806dea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:54896/0 (socket says 172.21.15.98:54896) 2024-09-19T08:46:37.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.630+0000 7fe11ccaf640 1 -- 172.21.15.98:0/642278917 learned_addr learned my addr 172.21.15.98:0/642278917 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:37.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.630+0000 7fe1177fe640 1 -- 172.21.15.98:0/642278917 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2403259990 0 0) 0x7fe1180d2150 con 0x7fe1180d1d70 2024-09-19T08:46:37.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.630+0000 7fe11d4b0640 1 --2- 172.21.15.98:0/642278917 >> v2:172.21.15.139:3300/0 conn(0x7fe1180713b0 0x7fe11815d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:37.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.630+0000 7fe1177fe640 1 -- 172.21.15.98:0/642278917 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe100003530 con 0x7fe1180d1d70 2024-09-19T08:46:37.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.630+0000 7fe1177fe640 1 -- 172.21.15.98:0/642278917 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 114878969 0 0) 0x7fe100003530 con 0x7fe1180d1d70 2024-09-19T08:46:37.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.630+0000 7fe1177fe640 1 -- 172.21.15.98:0/642278917 >> v2:172.21.15.139:3300/0 conn(0x7fe1180713b0 msgr2=0x7fe11815d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:37.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.630+0000 7fe1177fe640 1 --2- 172.21.15.98:0/642278917 >> v2:172.21.15.139:3300/0 conn(0x7fe1180713b0 0x7fe11815d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.632 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.630+0000 7fe1177fe640 1 -- 172.21.15.98:0/642278917 >> v1:172.21.15.139:6789/0 conn(0x7fe11806e5b0 legacy=0x7fe1180d0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:37.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.630+0000 7fe1177fe640 1 -- 172.21.15.98:0/642278917 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe11815ff30 con 0x7fe1180d1d70 2024-09-19T08:46:37.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.630+0000 7fe1177fe640 1 -- 172.21.15.98:0/642278917 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fe108001680 con 0x7fe1180d1d70 2024-09-19T08:46:37.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.630+0000 7fe1177fe640 1 -- 172.21.15.98:0/642278917 >> v1:172.21.15.98:6789/0 conn(0x7fe1180d1d70 legacy=0x7fe11806dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:37.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.630+0000 7fe1177fe640 1 --2- 172.21.15.98:0/642278917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1000039c0 0x7fe100003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:37.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.630+0000 7fe1177fe640 1 --2- 172.21.15.98:0/642278917 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe100004810 0x7fe100004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:37.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.630+0000 7fe1177fe640 1 --2- 172.21.15.98:0/642278917 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1000051d0 0x7fe10000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:37.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.630+0000 7fe1177fe640 1 -- 172.21.15.98:0/642278917 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe118071840 con 0x7fe1000039c0 2024-09-19T08:46:37.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.630+0000 7fe1177fe640 1 -- 172.21.15.98:0/642278917 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe10000c120 con 0x7fe1000051d0 2024-09-19T08:46:37.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.630+0000 7fe1177fe640 1 -- 172.21.15.98:0/642278917 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe10000c320 con 0x7fe100004810 2024-09-19T08:46:37.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.630+0000 7fe11ccaf640 1 --2- 172.21.15.98:0/642278917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1000039c0 0x7fe100003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:37.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.630+0000 7fe11d4b0640 1 --2- 172.21.15.98:0/642278917 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1000051d0 0x7fe10000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:37.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.630+0000 7fe117fff640 1 --2- 172.21.15.98:0/642278917 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe100004810 0x7fe100004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:37.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.630+0000 7fe11ccaf640 1 -- 172.21.15.98:0/642278917 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe100004810 msgr2=0x7fe100004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:37.633 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.630+0000 7fe11ccaf640 1 --2- 172.21.15.98:0/642278917 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe100004810 0x7fe100004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.630+0000 7fe11ccaf640 1 -- 172.21.15.98:0/642278917 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1000051d0 msgr2=0x7fe10000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:37.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.630+0000 7fe11ccaf640 1 --2- 172.21.15.98:0/642278917 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1000051d0 0x7fe10000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.630+0000 7fe11ccaf640 1 -- 172.21.15.98:0/642278917 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe100003530 con 0x7fe1000039c0 2024-09-19T08:46:37.634 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.630+0000 7fe11d4b0640 1 -- 172.21.15.98:0/642278917 reap_dead start 2024-09-19T08:46:37.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.634+0000 7fe11ccaf640 1 --2- 172.21.15.98:0/642278917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1000039c0 0x7fe100003de0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fe10800c1c0 tx=0x7fe10801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b04a8a211a5fef8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:37.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.634+0000 7fe1177fe640 1 -- 172.21.15.98:0/642278917 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe108007d00 con 0x7fe1000039c0 2024-09-19T08:46:37.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.634+0000 7fe1177fe640 1 -- 172.21.15.98:0/642278917 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1080a9020 con 0x7fe1000039c0 2024-09-19T08:46:37.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.634+0000 7fe1177fe640 1 -- 172.21.15.98:0/642278917 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe108005ae0 con 0x7fe1000039c0 2024-09-19T08:46:37.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.634+0000 7fe11ef13640 1 -- 172.21.15.98:0/642278917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1000039c0 msgr2=0x7fe100003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:37.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.634+0000 7fe11ef13640 1 --2- 172.21.15.98:0/642278917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1000039c0 0x7fe100003de0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fe10800c1c0 tx=0x7fe10801cc60 comp rx=0 tx=0).stop 2024-09-19T08:46:37.635 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.634+0000 7fe11ef13640 1 -- 172.21.15.98:0/642278917 shutdown_connections 2024-09-19T08:46:37.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.634+0000 7fe11ef13640 1 --2- 172.21.15.98:0/642278917 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1000039c0 0x7fe100003de0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.634+0000 7fe11ef13640 1 -- 172.21.15.98:0/642278917 >> 172.21.15.98:0/642278917 conn(0x7fe1180b4180 msgr2=0x7fe1180b4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:37.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.634+0000 7fe11ef13640 1 -- 172.21.15.98:0/642278917 shutdown_connections 2024-09-19T08:46:37.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.634+0000 7fe11ef13640 1 -- 172.21.15.98:0/642278917 wait complete. 2024-09-19T08:46:37.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.634+0000 7fe11ef13640 1 Processor -- start 2024-09-19T08:46:37.636 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.634+0000 7fe11ef13640 1 -- start start 2024-09-19T08:46:37.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.638+0000 7fe11ef13640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1000039c0 0x7fe118159850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:37.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.638+0000 7fe11ef13640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe118157ab0 0x7fe11814d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:37.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.638+0000 7fe11ef13640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1180b64d0 0x7fe1180b6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:37.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.638+0000 7fe11ef13640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe11815d730 con 0x7fe1000039c0 2024-09-19T08:46:37.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.638+0000 7fe11ef13640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe118160cd0 con 0x7fe1180b64d0 2024-09-19T08:46:37.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.638+0000 7fe11ef13640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe11806e2b0 con 0x7fe118157ab0 2024-09-19T08:46:37.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.638+0000 7fe11ccaf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1000039c0 0x7fe118159850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:37.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.638+0000 7fe11ccaf640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1000039c0 0x7fe118159850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54154/0 (socket says 172.21.15.98:54154) 2024-09-19T08:46:37.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.638+0000 7fe11ccaf640 1 -- 172.21.15.98:0/1615006570 learned_addr learned my addr 172.21.15.98:0/1615006570 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:37.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.638+0000 7fe11d4b0640 1 --2- 172.21.15.98:0/1615006570 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1180b64d0 0x7fe1180b6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:37.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.638+0000 7fe117fff640 1 --2- 172.21.15.98:0/1615006570 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe118157ab0 0x7fe11814d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:37.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.638+0000 7fe11ccaf640 1 -- 172.21.15.98:0/1615006570 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe118157ab0 msgr2=0x7fe11814d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:37.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.638+0000 7fe11ccaf640 1 --2- 172.21.15.98:0/1615006570 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe118157ab0 0x7fe11814d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.638+0000 7fe11ccaf640 1 -- 172.21.15.98:0/1615006570 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1180b64d0 msgr2=0x7fe1180b6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:37.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.638+0000 7fe11ccaf640 1 --2- 172.21.15.98:0/1615006570 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1180b64d0 0x7fe1180b6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:37.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.638+0000 7fe11ccaf640 1 -- 172.21.15.98:0/1615006570 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe11815c660 con 0x7fe1000039c0 2024-09-19T08:46:37.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.638+0000 7fe11ccaf640 1 --2- 172.21.15.98:0/1615006570 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1000039c0 0x7fe118159850 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fe1080ab040 tx=0x7fe100005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:37.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.638+0000 7fe1157fa640 1 -- 172.21.15.98:0/1615006570 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe108007d00 con 0x7fe1000039c0 2024-09-19T08:46:37.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.638+0000 7fe1157fa640 1 -- 172.21.15.98:0/1615006570 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe108000ee0 con 0x7fe1000039c0 2024-09-19T08:46:37.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.638+0000 7fe11ef13640 1 -- 172.21.15.98:0/1615006570 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe118158020 con 0x7fe1000039c0 2024-09-19T08:46:37.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.638+0000 7fe1157fa640 1 -- 172.21.15.98:0/1615006570 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe108001c40 con 0x7fe1000039c0 2024-09-19T08:46:37.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.638+0000 7fe11ef13640 1 -- 172.21.15.98:0/1615006570 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1181e9190 con 0x7fe1000039c0 2024-09-19T08:46:37.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.638+0000 7fe11ef13640 1 -- 172.21.15.98:0/1615006570 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe118073950 con 0x7fe1000039c0 2024-09-19T08:46:37.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.638+0000 7fe1157fa640 1 -- 172.21.15.98:0/1615006570 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7fe1080bd060 con 0x7fe1000039c0 2024-09-19T08:46:37.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.638+0000 7fe1157fa640 1 --2- 172.21.15.98:0/1615006570 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe1040680d0 0x7fe10406a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:37.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.638+0000 7fe1157fa640 1 -- 172.21.15.98:0/1615006570 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 4885+0+0 (secure 0 0 0) 0x7fe10811d4d0 con 0x7fe1000039c0 2024-09-19T08:46:37.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.638+0000 7fe117fff640 1 --2- 172.21.15.98:0/1615006570 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe1040680d0 0x7fe10406a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:37.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.638+0000 7fe117fff640 1 --2- 172.21.15.98:0/1615006570 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe1040680d0 0x7fe10406a590 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe118074010 tx=0x7fe10c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:37.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.642+0000 7fe1157fa640 1 -- 172.21.15.98:0/1615006570 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe108016970 con 0x7fe1000039c0 2024-09-19T08:46:37.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:37.794+0000 7fe11ef13640 1 -- 172.21.15.98:0/1615006570 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe1180d0f00 con 0x7fe1000039c0 2024-09-19T08:46:38.480 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.482+0000 7fe1157fa640 1 -- 172.21.15.98:0/1615006570 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v155) ==== 138+0+0 (secure 0 0 0) 0x7fe1080ef900 con 0x7fe1000039c0 2024-09-19T08:46:38.480 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_metadata' created 2024-09-19T08:46:38.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.482+0000 7fe11ef13640 1 -- 172.21.15.98:0/1615006570 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe1040680d0 msgr2=0x7fe10406a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:38.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.482+0000 7fe11ef13640 1 --2- 172.21.15.98:0/1615006570 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe1040680d0 0x7fe10406a590 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe118074010 tx=0x7fe10c009040 comp rx=0 tx=0).stop 2024-09-19T08:46:38.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.482+0000 7fe11ef13640 1 -- 172.21.15.98:0/1615006570 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1000039c0 msgr2=0x7fe118159850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:38.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.482+0000 7fe11ef13640 1 --2- 172.21.15.98:0/1615006570 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1000039c0 0x7fe118159850 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fe1080ab040 tx=0x7fe100005840 comp rx=0 tx=0).stop 2024-09-19T08:46:38.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.482+0000 7fe11ef13640 1 -- 172.21.15.98:0/1615006570 shutdown_connections 2024-09-19T08:46:38.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.482+0000 7fe11ef13640 1 --2- 172.21.15.98:0/1615006570 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe1040680d0 0x7fe10406a590 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:38.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.482+0000 7fe11ef13640 1 --2- 172.21.15.98:0/1615006570 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe1180b64d0 0x7fe1180b6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:38.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.482+0000 7fe11ef13640 1 --2- 172.21.15.98:0/1615006570 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe118157ab0 0x7fe11814d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:38.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.482+0000 7fe11ef13640 1 --2- 172.21.15.98:0/1615006570 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe1000039c0 0x7fe118159850 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:38.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.482+0000 7fe11ef13640 1 -- 172.21.15.98:0/1615006570 >> 172.21.15.98:0/1615006570 conn(0x7fe1180b4180 msgr2=0x7fe11815e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:38.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.482+0000 7fe11ef13640 1 -- 172.21.15.98:0/1615006570 shutdown_connections 2024-09-19T08:46:38.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.482+0000 7fe11ef13640 1 -- 172.21.15.98:0/1615006570 wait complete. 2024-09-19T08:46:38.510 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T08:46:38.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.642+0000 7fe30eec7640 1 Processor -- start 2024-09-19T08:46:38.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.642+0000 7fe30eec7640 1 -- start start 2024-09-19T08:46:38.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.642+0000 7fe30eec7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe308145b70 0x7fe30806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:38.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.642+0000 7fe30eec7640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe308071ac0 0x7fe3080b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:38.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.642+0000 7fe30eec7640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe308073530 con 0x7fe30806c1d0 2024-09-19T08:46:38.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.642+0000 7fe30eec7640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fe308071f10 con 0x7fe308071ac0 2024-09-19T08:46:38.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.642+0000 7fe30eec7640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe30806c620 con 0x7fe308145b70 2024-09-19T08:46:38.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.642+0000 7fe30d464640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe308071ac0 0x7fe3080b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:38.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.642+0000 7fe307fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fe30806c1d0 0x7fe3080713b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:54912/0 (socket says 172.21.15.98:54912) 2024-09-19T08:46:38.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.642+0000 7fe307fff640 1 -- 172.21.15.98:0/3958246265 learned_addr learned my addr 172.21.15.98:0/3958246265 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:38.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.642+0000 7fe30cc63640 1 --2- 172.21.15.98:0/3958246265 >> v2:172.21.15.139:3300/0 conn(0x7fe308145b70 0x7fe30806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:38.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.642+0000 7fe30d464640 1 -- 172.21.15.98:0/3958246265 >> v2:172.21.15.139:3300/0 conn(0x7fe308145b70 msgr2=0x7fe30806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:38.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.642+0000 7fe30d464640 1 --2- 172.21.15.98:0/3958246265 >> v2:172.21.15.139:3300/0 conn(0x7fe308145b70 0x7fe30806bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:38.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.642+0000 7fe30d464640 1 -- 172.21.15.98:0/3958246265 >> v1:172.21.15.98:6789/0 conn(0x7fe30806c1d0 legacy=0x7fe3080713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:38.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.642+0000 7fe30d464640 1 -- 172.21.15.98:0/3958246265 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe308073810 con 0x7fe308071ac0 2024-09-19T08:46:38.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.646+0000 7fe30d464640 1 --2- 172.21.15.98:0/3958246265 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe308071ac0 0x7fe3080b3410 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fe3080b2d30 tx=0x7fe2f801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c72d16b3a5a9c72e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:38.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.646+0000 7fe3077fe640 1 -- 172.21.15.98:0/3958246265 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f80b0070 con 0x7fe308071ac0 2024-09-19T08:46:38.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.646+0000 7fe3077fe640 1 -- 172.21.15.98:0/3958246265 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2f80ab600 con 0x7fe308071ac0 2024-09-19T08:46:38.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.646+0000 7fe3077fe640 1 -- 172.21.15.98:0/3958246265 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f80ab9d0 con 0x7fe308071ac0 2024-09-19T08:46:38.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.646+0000 7fe30eec7640 1 -- 172.21.15.98:0/3958246265 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe308071ac0 msgr2=0x7fe3080b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:38.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.646+0000 7fe30eec7640 1 --2- 172.21.15.98:0/3958246265 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe308071ac0 0x7fe3080b3410 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fe3080b2d30 tx=0x7fe2f801c3f0 comp rx=0 tx=0).stop 2024-09-19T08:46:38.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.646+0000 7fe30eec7640 1 -- 172.21.15.98:0/3958246265 shutdown_connections 2024-09-19T08:46:38.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.646+0000 7fe30eec7640 1 --2- 172.21.15.98:0/3958246265 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe308071ac0 0x7fe3080b3410 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:38.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.646+0000 7fe30eec7640 1 --2- 172.21.15.98:0/3958246265 >> v2:172.21.15.139:3300/0 conn(0x7fe308145b70 0x7fe30806bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:38.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.646+0000 7fe30eec7640 1 -- 172.21.15.98:0/3958246265 >> 172.21.15.98:0/3958246265 conn(0x7fe3080b2010 msgr2=0x7fe3080b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:38.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.646+0000 7fe30eec7640 1 -- 172.21.15.98:0/3958246265 shutdown_connections 2024-09-19T08:46:38.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.646+0000 7fe30eec7640 1 -- 172.21.15.98:0/3958246265 wait complete. 2024-09-19T08:46:38.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.646+0000 7fe30eec7640 1 Processor -- start 2024-09-19T08:46:38.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.646+0000 7fe30eec7640 1 -- start start 2024-09-19T08:46:38.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.646+0000 7fe30eec7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe30806c1d0 0x7fe308140730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:38.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.646+0000 7fe30eec7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe308071ac0 0x7fe30815b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:38.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.650+0000 7fe30eec7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe308145b70 0x7fe308140e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:38.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.650+0000 7fe30eec7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe3080b42e0 con 0x7fe308071ac0 2024-09-19T08:46:38.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.650+0000 7fe30eec7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe3080b4160 con 0x7fe308145b70 2024-09-19T08:46:38.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.650+0000 7fe30eec7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe3080b4460 con 0x7fe30806c1d0 2024-09-19T08:46:38.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.650+0000 7fe307fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe308071ac0 0x7fe30815b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:38.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.650+0000 7fe307fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe308071ac0 0x7fe30815b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54176/0 (socket says 172.21.15.98:54176) 2024-09-19T08:46:38.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.650+0000 7fe307fff640 1 -- 172.21.15.98:0/794978240 learned_addr learned my addr 172.21.15.98:0/794978240 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:38.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.650+0000 7fe307fff640 1 -- 172.21.15.98:0/794978240 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe30806c1d0 msgr2=0x7fe308140730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:38.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.650+0000 7fe30cc63640 1 --2- 172.21.15.98:0/794978240 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe30806c1d0 0x7fe308140730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:38.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.650+0000 7fe30d464640 1 --2- 172.21.15.98:0/794978240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe308145b70 0x7fe308140e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:38.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.650+0000 7fe307fff640 1 --2- 172.21.15.98:0/794978240 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe30806c1d0 0x7fe308140730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:38.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.650+0000 7fe307fff640 1 -- 172.21.15.98:0/794978240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe308145b70 msgr2=0x7fe308140e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:38.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.650+0000 7fe307fff640 1 --2- 172.21.15.98:0/794978240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe308145b70 0x7fe308140e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:38.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.650+0000 7fe307fff640 1 -- 172.21.15.98:0/794978240 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3080726d0 con 0x7fe308071ac0 2024-09-19T08:46:38.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.650+0000 7fe307fff640 1 --2- 172.21.15.98:0/794978240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe308071ac0 0x7fe30815b4f0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fe2f000c420 tx=0x7fe2f000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:38.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.650+0000 7fe3057fa640 1 -- 172.21.15.98:0/794978240 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f0017070 con 0x7fe308071ac0 2024-09-19T08:46:38.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.650+0000 7fe3057fa640 1 -- 172.21.15.98:0/794978240 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2f0005cf0 con 0x7fe308071ac0 2024-09-19T08:46:38.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.650+0000 7fe30eec7640 1 -- 172.21.15.98:0/794978240 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3081418f0 con 0x7fe308071ac0 2024-09-19T08:46:38.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.650+0000 7fe3057fa640 1 -- 172.21.15.98:0/794978240 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f00072f0 con 0x7fe308071ac0 2024-09-19T08:46:38.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.650+0000 7fe30eec7640 1 -- 172.21.15.98:0/794978240 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe308140210 con 0x7fe308071ac0 2024-09-19T08:46:38.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.650+0000 7fe30eec7640 1 -- 172.21.15.98:0/794978240 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe308140410 con 0x7fe308071ac0 2024-09-19T08:46:38.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.650+0000 7fe3057fa640 1 -- 172.21.15.98:0/794978240 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7fe2f001a070 con 0x7fe308071ac0 2024-09-19T08:46:38.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.650+0000 7fe3057fa640 1 --2- 172.21.15.98:0/794978240 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe2e80680a0 0x7fe2e806a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:38.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.650+0000 7fe3057fa640 1 -- 172.21.15.98:0/794978240 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5255+0+0 (secure 0 0 0) 0x7fe2f00864f0 con 0x7fe308071ac0 2024-09-19T08:46:38.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.650+0000 7fe30cc63640 1 --2- 172.21.15.98:0/794978240 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe2e80680a0 0x7fe2e806a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:38.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.654+0000 7fe30cc63640 1 --2- 172.21.15.98:0/794978240 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe2e80680a0 0x7fe2e806a560 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe308142150 tx=0x7fe3000083c0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:38.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.654+0000 7fe3057fa640 1 -- 172.21.15.98:0/794978240 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe308140410 con 0x7fe308071ac0 2024-09-19T08:46:38.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:38.818+0000 7fe30eec7640 1 -- 172.21.15.98:0/794978240 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe3080b3410 con 0x7fe308071ac0 2024-09-19T08:46:39.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.490+0000 7fe3057fa640 1 -- 172.21.15.98:0/794978240 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v156) ==== 170+0+0 (secure 0 0 0) 0x7fe2f0021020 con 0x7fe308071ac0 2024-09-19T08:46:39.492 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' created 2024-09-19T08:46:39.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.494+0000 7fe30eec7640 1 -- 172.21.15.98:0/794978240 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe2e80680a0 msgr2=0x7fe2e806a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:39.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.494+0000 7fe30eec7640 1 --2- 172.21.15.98:0/794978240 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe2e80680a0 0x7fe2e806a560 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe308142150 tx=0x7fe3000083c0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.494+0000 7fe30eec7640 1 -- 172.21.15.98:0/794978240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe308071ac0 msgr2=0x7fe30815b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:39.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.494+0000 7fe30eec7640 1 --2- 172.21.15.98:0/794978240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe308071ac0 0x7fe30815b4f0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fe2f000c420 tx=0x7fe2f000c8e0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.494+0000 7fe30eec7640 1 -- 172.21.15.98:0/794978240 shutdown_connections 2024-09-19T08:46:39.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.494+0000 7fe30eec7640 1 --2- 172.21.15.98:0/794978240 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe2e80680a0 0x7fe2e806a560 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.494+0000 7fe30eec7640 1 --2- 172.21.15.98:0/794978240 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe308145b70 0x7fe308140e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.494+0000 7fe30eec7640 1 --2- 172.21.15.98:0/794978240 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe308071ac0 0x7fe30815b4f0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.494+0000 7fe30eec7640 1 --2- 172.21.15.98:0/794978240 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe30806c1d0 0x7fe308140730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.494+0000 7fe30eec7640 1 -- 172.21.15.98:0/794978240 >> 172.21.15.98:0/794978240 conn(0x7fe3080b2010 msgr2=0x7fe3080b18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:39.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.494+0000 7fe30eec7640 1 -- 172.21.15.98:0/794978240 shutdown_connections 2024-09-19T08:46:39.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.494+0000 7fe30eec7640 1 -- 172.21.15.98:0/794978240 wait complete. 2024-09-19T08:46:39.527 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T08:46:39.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.650+0000 7ff14b28f640 1 Processor -- start 2024-09-19T08:46:39.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.650+0000 7ff14b28f640 1 -- start start 2024-09-19T08:46:39.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.654+0000 7ff14b28f640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff1440729f0 0x7ff14406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:39.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.654+0000 7ff14b28f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff14406c1d0 0x7ff1440cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:39.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.654+0000 7ff14b28f640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff144072ea0 con 0x7ff144144d10 2024-09-19T08:46:39.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.654+0000 7ff14b28f640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7ff144073ab0 con 0x7ff1440729f0 2024-09-19T08:46:39.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.654+0000 7ff14b28f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff1441450f0 con 0x7ff14406c1d0 2024-09-19T08:46:39.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.654+0000 7ff14902b640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7ff144144d10 0x7ff1440722e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:54928/0 (socket says 172.21.15.98:54928) 2024-09-19T08:46:39.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.654+0000 7ff14902b640 1 -- 172.21.15.98:0/3782455419 learned_addr learned my addr 172.21.15.98:0/3782455419 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:39.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.654+0000 7ff14882a640 1 --2- 172.21.15.98:0/3782455419 >> v2:172.21.15.98:3300/0 conn(0x7ff1440729f0 0x7ff14406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:39.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.654+0000 7ff13bfff640 1 -- 172.21.15.98:0/3782455419 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2020364169 0 0) 0x7ff144072ea0 con 0x7ff144144d10 2024-09-19T08:46:39.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.654+0000 7ff14982c640 1 --2- 172.21.15.98:0/3782455419 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff14406c1d0 0x7ff1440cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:39.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.654+0000 7ff13bfff640 1 -- 172.21.15.98:0/3782455419 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff130003530 con 0x7ff144144d10 2024-09-19T08:46:39.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.654+0000 7ff14882a640 1 -- 172.21.15.98:0/3782455419 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff14406c1d0 msgr2=0x7ff1440cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:39.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.654+0000 7ff14882a640 1 --2- 172.21.15.98:0/3782455419 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff14406c1d0 0x7ff1440cd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.654+0000 7ff14882a640 1 -- 172.21.15.98:0/3782455419 >> v1:172.21.15.98:6789/0 conn(0x7ff144144d10 legacy=0x7ff1440722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:39.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.654+0000 7ff14882a640 1 -- 172.21.15.98:0/3782455419 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff144073180 con 0x7ff1440729f0 2024-09-19T08:46:39.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.654+0000 7ff14882a640 1 --2- 172.21.15.98:0/3782455419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1440729f0 0x7ff14406bc90 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7ff140009d90 tx=0x7ff14001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b8988d976ca91ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:39.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.654+0000 7ff13bfff640 1 -- 172.21.15.98:0/3782455419 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1400b2070 con 0x7ff1440729f0 2024-09-19T08:46:39.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.654+0000 7ff13bfff640 1 -- 172.21.15.98:0/3782455419 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1400a5920 con 0x7ff1440729f0 2024-09-19T08:46:39.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.654+0000 7ff13bfff640 1 -- 172.21.15.98:0/3782455419 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1400a5cf0 con 0x7ff1440729f0 2024-09-19T08:46:39.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.658+0000 7ff14b28f640 1 -- 172.21.15.98:0/3782455419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1440729f0 msgr2=0x7ff14406bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:39.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.658+0000 7ff14b28f640 1 --2- 172.21.15.98:0/3782455419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1440729f0 0x7ff14406bc90 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7ff140009d90 tx=0x7ff14001c970 comp rx=0 tx=0).stop 2024-09-19T08:46:39.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.658+0000 7ff14b28f640 1 -- 172.21.15.98:0/3782455419 shutdown_connections 2024-09-19T08:46:39.656 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.658+0000 7ff14b28f640 1 --2- 172.21.15.98:0/3782455419 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff14406c1d0 0x7ff1440cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.658+0000 7ff14b28f640 1 --2- 172.21.15.98:0/3782455419 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1440729f0 0x7ff14406bc90 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.658+0000 7ff14b28f640 1 -- 172.21.15.98:0/3782455419 >> 172.21.15.98:0/3782455419 conn(0x7ff1440b2010 msgr2=0x7ff1440b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:39.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.658+0000 7ff14b28f640 1 -- 172.21.15.98:0/3782455419 shutdown_connections 2024-09-19T08:46:39.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.658+0000 7ff14b28f640 1 -- 172.21.15.98:0/3782455419 wait complete. 2024-09-19T08:46:39.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.658+0000 7ff14b28f640 1 Processor -- start 2024-09-19T08:46:39.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.658+0000 7ff14b28f640 1 -- start start 2024-09-19T08:46:39.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.658+0000 7ff14b28f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff14406c1d0 0x7ff1441407a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:39.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.658+0000 7ff14b28f640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1440729f0 0x7ff1440aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:39.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.658+0000 7ff14b28f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff144144d10 0x7ff14413dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:39.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.658+0000 7ff14b28f640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff1440ce120 con 0x7ff14406c1d0 2024-09-19T08:46:39.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.658+0000 7ff14b28f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff1440cdfa0 con 0x7ff144144d10 2024-09-19T08:46:39.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.658+0000 7ff14b28f640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff1440ce2a0 con 0x7ff1440729f0 2024-09-19T08:46:39.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.658+0000 7ff14902b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff14406c1d0 0x7ff1441407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:39.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.658+0000 7ff14902b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff14406c1d0 0x7ff1441407a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54192/0 (socket says 172.21.15.98:54192) 2024-09-19T08:46:39.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.658+0000 7ff14902b640 1 -- 172.21.15.98:0/1110252973 learned_addr learned my addr 172.21.15.98:0/1110252973 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:39.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.658+0000 7ff14882a640 1 --2- 172.21.15.98:0/1110252973 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1440729f0 0x7ff1440aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:39.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.658+0000 7ff14982c640 1 --2- 172.21.15.98:0/1110252973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff144144d10 0x7ff14413dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:39.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.658+0000 7ff14902b640 1 -- 172.21.15.98:0/1110252973 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1440729f0 msgr2=0x7ff1440aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:39.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.658+0000 7ff14902b640 1 --2- 172.21.15.98:0/1110252973 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1440729f0 0x7ff1440aeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.658+0000 7ff14902b640 1 -- 172.21.15.98:0/1110252973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff144144d10 msgr2=0x7ff14413dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:39.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.658+0000 7ff14902b640 1 --2- 172.21.15.98:0/1110252973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff144144d10 0x7ff14413dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:39.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.658+0000 7ff14902b640 1 -- 172.21.15.98:0/1110252973 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff14413fa40 con 0x7ff14406c1d0 2024-09-19T08:46:39.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.658+0000 7ff14902b640 1 --2- 172.21.15.98:0/1110252973 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff14406c1d0 0x7ff1441407a0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7ff13400c3c0 tx=0x7ff13400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:39.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.658+0000 7ff139ffb640 1 -- 172.21.15.98:0/1110252973 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff134017070 con 0x7ff14406c1d0 2024-09-19T08:46:39.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.658+0000 7ff139ffb640 1 -- 172.21.15.98:0/1110252973 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff134005cf0 con 0x7ff14406c1d0 2024-09-19T08:46:39.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.658+0000 7ff14b28f640 1 -- 172.21.15.98:0/1110252973 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1441f76a0 con 0x7ff14406c1d0 2024-09-19T08:46:39.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.658+0000 7ff139ffb640 1 -- 172.21.15.98:0/1110252973 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1340072f0 con 0x7ff14406c1d0 2024-09-19T08:46:39.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.658+0000 7ff14b28f640 1 -- 172.21.15.98:0/1110252973 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff14413ffa0 con 0x7ff14406c1d0 2024-09-19T08:46:39.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.662+0000 7ff14b28f640 1 -- 172.21.15.98:0/1110252973 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1440cd250 con 0x7ff14406c1d0 2024-09-19T08:46:39.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.662+0000 7ff139ffb640 1 -- 172.21.15.98:0/1110252973 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7ff13401a070 con 0x7ff14406c1d0 2024-09-19T08:46:39.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.662+0000 7ff139ffb640 1 --2- 172.21.15.98:0/1110252973 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff1280680a0 0x7ff12806a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:39.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.662+0000 7ff139ffb640 1 -- 172.21.15.98:0/1110252973 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5637+0+0 (secure 0 0 0) 0x7ff134086630 con 0x7ff14406c1d0 2024-09-19T08:46:39.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.662+0000 7ff14882a640 1 --2- 172.21.15.98:0/1110252973 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff1280680a0 0x7ff12806a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:39.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.662+0000 7ff14882a640 1 --2- 172.21.15.98:0/1110252973 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff1280680a0 0x7ff12806a560 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff1400af000 tx=0x7ff1400a7070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:39.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.666+0000 7ff139ffb640 1 -- 172.21.15.98:0/1110252973 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff134021020 con 0x7ff14406c1d0 2024-09-19T08:46:39.863 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:39.862+0000 7ff14b28f640 1 -- 172.21.15.98:0/1110252973 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ff14413caa0 con 0x7ff14406c1d0 2024-09-19T08:46:40.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.502+0000 7ff139ffb640 1 -- 172.21.15.98:0/1110252973 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 25 v87) ==== 174+0+0 (secure 0 0 0) 0x7ff14413fa40 con 0x7ff14406c1d0 2024-09-19T08:46:40.504 INFO:teuthology.orchestra.run.smithi098.stderr:new fs with metadata pool 24 and data pool 25 2024-09-19T08:46:40.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.506+0000 7ff14b28f640 1 -- 172.21.15.98:0/1110252973 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff1280680a0 msgr2=0x7ff12806a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:40.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.506+0000 7ff14b28f640 1 --2- 172.21.15.98:0/1110252973 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff1280680a0 0x7ff12806a560 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff1400af000 tx=0x7ff1400a7070 comp rx=0 tx=0).stop 2024-09-19T08:46:40.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.506+0000 7ff14b28f640 1 -- 172.21.15.98:0/1110252973 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff14406c1d0 msgr2=0x7ff1441407a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:40.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.506+0000 7ff14b28f640 1 --2- 172.21.15.98:0/1110252973 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff14406c1d0 0x7ff1441407a0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7ff13400c3c0 tx=0x7ff13400c880 comp rx=0 tx=0).stop 2024-09-19T08:46:40.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.506+0000 7ff14b28f640 1 -- 172.21.15.98:0/1110252973 shutdown_connections 2024-09-19T08:46:40.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.506+0000 7ff14b28f640 1 --2- 172.21.15.98:0/1110252973 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff1280680a0 0x7ff12806a560 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.506+0000 7ff14b28f640 1 --2- 172.21.15.98:0/1110252973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff144144d10 0x7ff14413dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.506+0000 7ff14b28f640 1 --2- 172.21.15.98:0/1110252973 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1440729f0 0x7ff1440aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.506+0000 7ff14b28f640 1 --2- 172.21.15.98:0/1110252973 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff14406c1d0 0x7ff1441407a0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.506+0000 7ff14b28f640 1 -- 172.21.15.98:0/1110252973 >> 172.21.15.98:0/1110252973 conn(0x7ff1440b2010 msgr2=0x7ff14415ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:40.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.506+0000 7ff14b28f640 1 -- 172.21.15.98:0/1110252973 shutdown_connections 2024-09-19T08:46:40.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.510+0000 7ff14b28f640 1 -- 172.21.15.98:0/1110252973 wait complete. 2024-09-19T08:46:40.535 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:46:40.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.654+0000 7f1dca82d640 1 Processor -- start 2024-09-19T08:46:40.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.654+0000 7f1dca82d640 1 -- start start 2024-09-19T08:46:40.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.654+0000 7f1dca82d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f1dc40b7f80 0x7f1dc415f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:40.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.654+0000 7f1dca82d640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dc4068440 con 0x7f1dc40d4820 2024-09-19T08:46:40.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.654+0000 7f1dca82d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dc4162a90 con 0x7f1dc40b9970 2024-09-19T08:46:40.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.654+0000 7f1dca82d640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f1dc4163050 con 0x7f1dc40b7f80 2024-09-19T08:46:40.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.654+0000 7f1dc3fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f1dc40b7f80 0x7f1dc415f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:40.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.654+0000 7f1dc37fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f1dc40d4820 0x7f1dc41621b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:54942/0 (socket says 172.21.15.98:54942) 2024-09-19T08:46:40.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.654+0000 7f1dc37fe640 1 -- 172.21.15.98:0/489276728 learned_addr learned my addr 172.21.15.98:0/489276728 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:40.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.654+0000 7f1dc2ffd640 1 -- 172.21.15.98:0/489276728 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 709528135 0 0) 0x7f1dc4068440 con 0x7f1dc40d4820 2024-09-19T08:46:40.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.654+0000 7f1dc2ffd640 1 -- 172.21.15.98:0/489276728 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1dac003530 con 0x7f1dc40d4820 2024-09-19T08:46:40.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.654+0000 7f1dc3fff640 1 -- 172.21.15.98:0/489276728 >> v1:172.21.15.139:6789/0 conn(0x7f1dc40b9970 legacy=0x7f1dc40b9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:40.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.654+0000 7f1dc3fff640 1 -- 172.21.15.98:0/489276728 >> v1:172.21.15.98:6789/0 conn(0x7f1dc40d4820 legacy=0x7f1dc41621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:40.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.654+0000 7f1dc2ffd640 1 -- 172.21.15.98:0/489276728 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3849625669 0 0) 0x7f1dac003530 con 0x7f1dc40d4820 2024-09-19T08:46:40.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.654+0000 7f1dc3fff640 1 -- 172.21.15.98:0/489276728 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc4162d70 con 0x7f1dc40b7f80 2024-09-19T08:46:40.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.658+0000 7f1dc3fff640 1 --2- 172.21.15.98:0/489276728 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dc40b7f80 0x7f1dc415f560 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f1db40097c0 tx=0x7f1db401c2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22aabf2b6557f21 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:40.657 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.658+0000 7f1dc2ffd640 1 -- 172.21.15.98:0/489276728 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db40b0070 con 0x7f1dc40b7f80 2024-09-19T08:46:40.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.658+0000 7f1dc2ffd640 1 -- 172.21.15.98:0/489276728 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db40ab400 con 0x7f1dc40b7f80 2024-09-19T08:46:40.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.658+0000 7f1dc2ffd640 1 -- 172.21.15.98:0/489276728 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8001680 con 0x7f1dc40b7f80 2024-09-19T08:46:40.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.658+0000 7f1dca82d640 1 -- 172.21.15.98:0/489276728 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dc40b7f80 msgr2=0x7f1dc415f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:40.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.658+0000 7f1dca82d640 1 --2- 172.21.15.98:0/489276728 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dc40b7f80 0x7f1dc415f560 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f1db40097c0 tx=0x7f1db401c2f0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.658+0000 7f1dca82d640 1 -- 172.21.15.98:0/489276728 shutdown_connections 2024-09-19T08:46:40.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.658+0000 7f1dca82d640 1 --2- 172.21.15.98:0/489276728 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dc40b7f80 0x7f1dc415f560 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.658+0000 7f1dca82d640 1 -- 172.21.15.98:0/489276728 >> 172.21.15.98:0/489276728 conn(0x7f1dc4074e40 msgr2=0x7f1dc4075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:40.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.658+0000 7f1dca82d640 1 -- 172.21.15.98:0/489276728 shutdown_connections 2024-09-19T08:46:40.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.658+0000 7f1dca82d640 1 -- 172.21.15.98:0/489276728 wait complete. 2024-09-19T08:46:40.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.658+0000 7f1dca82d640 1 Processor -- start 2024-09-19T08:46:40.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.662+0000 7f1dca82d640 1 -- start start 2024-09-19T08:46:40.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.662+0000 7f1dca82d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1dc40b7f80 0x7f1dc40d38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:40.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.662+0000 7f1dca82d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dc40b9970 0x7f1dc414fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:40.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.662+0000 7f1dca82d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1dc40d4820 0x7f1dc4138d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:40.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.662+0000 7f1dca82d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc4144710 con 0x7f1dc40b9970 2024-09-19T08:46:40.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.662+0000 7f1dca82d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc4144590 con 0x7f1dc40d4820 2024-09-19T08:46:40.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.662+0000 7f1dca82d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1dc4144890 con 0x7f1dc40b7f80 2024-09-19T08:46:40.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.662+0000 7f1dc37fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dc40b9970 0x7f1dc414fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:40.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.662+0000 7f1dc37fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dc40b9970 0x7f1dc414fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54208/0 (socket says 172.21.15.98:54208) 2024-09-19T08:46:40.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.662+0000 7f1dc37fe640 1 -- 172.21.15.98:0/2559287254 learned_addr learned my addr 172.21.15.98:0/2559287254 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:40.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.662+0000 7f1dc3fff640 1 --2- 172.21.15.98:0/2559287254 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1dc40b7f80 0x7f1dc40d38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:40.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.662+0000 7f1dc8dca640 1 --2- 172.21.15.98:0/2559287254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1dc40d4820 0x7f1dc4138d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:40.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.662+0000 7f1dc37fe640 1 -- 172.21.15.98:0/2559287254 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1dc40b7f80 msgr2=0x7f1dc40d38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:40.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.662+0000 7f1dc37fe640 1 --2- 172.21.15.98:0/2559287254 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1dc40b7f80 0x7f1dc40d38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.662+0000 7f1dc37fe640 1 -- 172.21.15.98:0/2559287254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1dc40d4820 msgr2=0x7f1dc4138d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:40.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.662+0000 7f1dc37fe640 1 --2- 172.21.15.98:0/2559287254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1dc40d4820 0x7f1dc4138d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.662+0000 7f1dc37fe640 1 -- 172.21.15.98:0/2559287254 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc41539b0 con 0x7f1dc40b9970 2024-09-19T08:46:40.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.662+0000 7f1dc37fe640 1 --2- 172.21.15.98:0/2559287254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dc40b9970 0x7f1dc414fe90 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f1db800c780 tx=0x7f1db800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:40.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.662+0000 7f1dc0ff9640 1 -- 172.21.15.98:0/2559287254 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8005ae0 con 0x7f1dc40b9970 2024-09-19T08:46:40.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.662+0000 7f1dc0ff9640 1 -- 172.21.15.98:0/2559287254 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db8005ca0 con 0x7f1dc40b9970 2024-09-19T08:46:40.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.662+0000 7f1dca82d640 1 -- 172.21.15.98:0/2559287254 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dc4153c90 con 0x7f1dc40b9970 2024-09-19T08:46:40.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.662+0000 7f1dc0ff9640 1 -- 172.21.15.98:0/2559287254 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db80072b0 con 0x7f1dc40b9970 2024-09-19T08:46:40.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.662+0000 7f1dca82d640 1 -- 172.21.15.98:0/2559287254 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dc40d1650 con 0x7f1dc40b9970 2024-09-19T08:46:40.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.662+0000 7f1dca82d640 1 -- 172.21.15.98:0/2559287254 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dc413cb00 con 0x7f1dc40b9970 2024-09-19T08:46:40.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.662+0000 7f1dc0ff9640 1 -- 172.21.15.98:0/2559287254 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f1db8007450 con 0x7f1dc40b9970 2024-09-19T08:46:40.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.662+0000 7f1dc0ff9640 1 --2- 172.21.15.98:0/2559287254 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1db00680d0 0x7f1db006a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:40.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.662+0000 7f1dc0ff9640 1 -- 172.21.15.98:0/2559287254 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 5737+0+0 (secure 0 0 0) 0x7f1db80845c0 con 0x7f1dc40b9970 2024-09-19T08:46:40.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.662+0000 7f1dc3fff640 1 --2- 172.21.15.98:0/2559287254 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1db00680d0 0x7f1db006a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:40.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.666+0000 7f1dc3fff640 1 --2- 172.21.15.98:0/2559287254 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1db00680d0 0x7f1db006a590 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1db40ad000 tx=0x7f1db4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:40.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.666+0000 7f1dc0ff9640 1 -- 172.21.15.98:0/2559287254 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f1db8010300 con 0x7f1dc40b9970 2024-09-19T08:46:40.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.834+0000 7f1dca82d640 1 -- 172.21.15.98:0/2559287254 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1dc415f540 con 0x7f1dc40b9970 2024-09-19T08:46:40.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.834+0000 7f1dc0ff9640 1 -- 172.21.15.98:0/2559287254 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v157) ==== 74+0+16796 (secure 0 0 0) 0x7f1db8056960 con 0x7f1dc40b9970 2024-09-19T08:46:40.836 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:46:40.836 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":157,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:46:39.868005+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:46:37.833554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-19T08:46:38.858268+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:46:40.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.838+0000 7f1dca82d640 1 -- 172.21.15.98:0/2559287254 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1db00680d0 msgr2=0x7f1db006a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:40.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.838+0000 7f1dca82d640 1 --2- 172.21.15.98:0/2559287254 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1db00680d0 0x7f1db006a590 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1db40ad000 tx=0x7f1db4008040 comp rx=0 tx=0).stop 2024-09-19T08:46:40.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.838+0000 7f1dca82d640 1 -- 172.21.15.98:0/2559287254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dc40b9970 msgr2=0x7f1dc414fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:40.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.838+0000 7f1dca82d640 1 --2- 172.21.15.98:0/2559287254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dc40b9970 0x7f1dc414fe90 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f1db800c780 tx=0x7f1db800cc40 comp rx=0 tx=0).stop 2024-09-19T08:46:40.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.838+0000 7f1dca82d640 1 -- 172.21.15.98:0/2559287254 shutdown_connections 2024-09-19T08:46:40.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.838+0000 7f1dca82d640 1 --2- 172.21.15.98:0/2559287254 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1db00680d0 0x7f1db006a590 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.838+0000 7f1dca82d640 1 --2- 172.21.15.98:0/2559287254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1dc40d4820 0x7f1dc4138d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.838+0000 7f1dca82d640 1 --2- 172.21.15.98:0/2559287254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f1dc40b9970 0x7f1dc414fe90 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.838+0000 7f1dca82d640 1 --2- 172.21.15.98:0/2559287254 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f1dc40b7f80 0x7f1dc40d38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.838+0000 7f1dca82d640 1 -- 172.21.15.98:0/2559287254 >> 172.21.15.98:0/2559287254 conn(0x7f1dc4074e40 msgr2=0x7f1dc40d0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:40.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.838+0000 7f1dca82d640 1 -- 172.21.15.98:0/2559287254 shutdown_connections 2024-09-19T08:46:40.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.838+0000 7f1dca82d640 1 -- 172.21.15.98:0/2559287254 wait complete. 2024-09-19T08:46:40.855 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:46:40.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.974+0000 7f19cb610640 1 Processor -- start 2024-09-19T08:46:40.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.974+0000 7f19cb610640 1 -- start start 2024-09-19T08:46:40.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.974+0000 7f19cb610640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19c415c220 0x7f19c40733f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:40.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.974+0000 7f19cb610640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f19c4073930 0x7f19c40d0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:40.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.974+0000 7f19cb610640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19c40740b0 con 0x7f19c40d1530 2024-09-19T08:46:40.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.974+0000 7f19cb610640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f19c4073dd0 con 0x7f19c4073930 2024-09-19T08:46:40.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.974+0000 7f19cb610640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f19c40b6460 con 0x7f19c415c220 2024-09-19T08:46:40.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.974+0000 7f19c8bab640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f19c4073930 0x7f19c40d0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:40.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.974+0000 7f19c9bad640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f19c40d1530 0x7f19c40b6f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:54944/0 (socket says 172.21.15.98:54944) 2024-09-19T08:46:40.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.974+0000 7f19c9bad640 1 -- 172.21.15.98:0/1590417110 learned_addr learned my addr 172.21.15.98:0/1590417110 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:40.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.974+0000 7f19c93ac640 1 --2- 172.21.15.98:0/1590417110 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19c415c220 0x7f19c40733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:40.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.974+0000 7f19c8bab640 1 -- 172.21.15.98:0/1590417110 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19c415c220 msgr2=0x7f19c40733f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:40.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.974+0000 7f19c8bab640 1 --2- 172.21.15.98:0/1590417110 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19c415c220 0x7f19c40733f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.974+0000 7f19c8bab640 1 -- 172.21.15.98:0/1590417110 >> v1:172.21.15.98:6789/0 conn(0x7f19c40d1530 legacy=0x7f19c40b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:40.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.974+0000 7f19c8bab640 1 -- 172.21.15.98:0/1590417110 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19c40b6740 con 0x7f19c4073930 2024-09-19T08:46:40.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.974+0000 7f19bbfff640 1 -- 172.21.15.98:0/1590417110 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1204039798 0 0) 0x7f19c40740b0 con 0x7f19c40d1530 2024-09-19T08:46:40.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.974+0000 7f19c93ac640 1 --2- 172.21.15.98:0/1590417110 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19c415c220 0x7f19c40733f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:40.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.978+0000 7f19c8bab640 1 --2- 172.21.15.98:0/1590417110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19c4073930 0x7f19c40d0ff0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f19b40097c0 tx=0x7f19b401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=601b7041d261a242 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:40.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.978+0000 7f19bbfff640 1 -- 172.21.15.98:0/1590417110 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b40b1070 con 0x7f19c4073930 2024-09-19T08:46:40.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.978+0000 7f19bbfff640 1 -- 172.21.15.98:0/1590417110 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19b40ac410 con 0x7f19c4073930 2024-09-19T08:46:40.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.978+0000 7f19bbfff640 1 -- 172.21.15.98:0/1590417110 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b40ac7e0 con 0x7f19c4073930 2024-09-19T08:46:40.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.978+0000 7f19cb610640 1 -- 172.21.15.98:0/1590417110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19c4073930 msgr2=0x7f19c40d0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:40.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.978+0000 7f19cb610640 1 --2- 172.21.15.98:0/1590417110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19c4073930 0x7f19c40d0ff0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f19b40097c0 tx=0x7f19b401c4a0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.978+0000 7f19cb610640 1 -- 172.21.15.98:0/1590417110 shutdown_connections 2024-09-19T08:46:40.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.978+0000 7f19cb610640 1 --2- 172.21.15.98:0/1590417110 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19c4073930 0x7f19c40d0ff0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.978+0000 7f19cb610640 1 --2- 172.21.15.98:0/1590417110 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19c415c220 0x7f19c40733f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.978+0000 7f19cb610640 1 -- 172.21.15.98:0/1590417110 >> 172.21.15.98:0/1590417110 conn(0x7f19c40b40a0 msgr2=0x7f19c40b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:40.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.978+0000 7f19cb610640 1 -- 172.21.15.98:0/1590417110 shutdown_connections 2024-09-19T08:46:40.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.978+0000 7f19cb610640 1 -- 172.21.15.98:0/1590417110 wait complete. 2024-09-19T08:46:40.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.978+0000 7f19cb610640 1 Processor -- start 2024-09-19T08:46:40.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.978+0000 7f19cb610640 1 -- start start 2024-09-19T08:46:40.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.978+0000 7f19cb610640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19c4073930 0x7f19c41515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:40.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.978+0000 7f19cb610640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19c40d1530 0x7f19c40d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:40.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.978+0000 7f19cb610640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19c415c220 0x7f19c414d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:40.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.978+0000 7f19cb610640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f19c4168ea0 con 0x7f19c4073930 2024-09-19T08:46:40.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.978+0000 7f19cb610640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f19c415d0a0 con 0x7f19c415c220 2024-09-19T08:46:40.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.978+0000 7f19cb610640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f19c4169020 con 0x7f19c40d1530 2024-09-19T08:46:40.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.978+0000 7f19c93ac640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19c4073930 0x7f19c41515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:40.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.978+0000 7f19c93ac640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19c4073930 0x7f19c41515a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54236/0 (socket says 172.21.15.98:54236) 2024-09-19T08:46:40.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.978+0000 7f19c93ac640 1 -- 172.21.15.98:0/3252117150 learned_addr learned my addr 172.21.15.98:0/3252117150 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:40.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.978+0000 7f19c9bad640 1 --2- 172.21.15.98:0/3252117150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19c415c220 0x7f19c414d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:40.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.978+0000 7f19c8bab640 1 --2- 172.21.15.98:0/3252117150 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19c40d1530 0x7f19c40d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:40.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.978+0000 7f19c93ac640 1 -- 172.21.15.98:0/3252117150 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19c40d1530 msgr2=0x7f19c40d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:40.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.978+0000 7f19c93ac640 1 --2- 172.21.15.98:0/3252117150 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19c40d1530 0x7f19c40d29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.978+0000 7f19c93ac640 1 -- 172.21.15.98:0/3252117150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19c415c220 msgr2=0x7f19c414d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:40.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.978+0000 7f19c93ac640 1 --2- 172.21.15.98:0/3252117150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19c415c220 0x7f19c414d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:40.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.978+0000 7f19c93ac640 1 -- 172.21.15.98:0/3252117150 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19c41520d0 con 0x7f19c4073930 2024-09-19T08:46:40.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.978+0000 7f19c9bad640 1 --2- 172.21.15.98:0/3252117150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19c415c220 0x7f19c414d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:40.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.982+0000 7f19c93ac640 1 --2- 172.21.15.98:0/3252117150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19c4073930 0x7f19c41515a0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f19c000c7c0 tx=0x7f19c000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:40.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.982+0000 7f19b9ffb640 1 -- 172.21.15.98:0/3252117150 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c0007a30 con 0x7f19c4073930 2024-09-19T08:46:40.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.982+0000 7f19b9ffb640 1 -- 172.21.15.98:0/3252117150 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19c0007bd0 con 0x7f19c4073930 2024-09-19T08:46:40.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.982+0000 7f19cb610640 1 -- 172.21.15.98:0/3252117150 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19c4154170 con 0x7f19c4073930 2024-09-19T08:46:40.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.982+0000 7f19b9ffb640 1 -- 172.21.15.98:0/3252117150 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c00113f0 con 0x7f19c4073930 2024-09-19T08:46:40.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.982+0000 7f19cb610640 1 -- 172.21.15.98:0/3252117150 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19c4154410 con 0x7f19c4073930 2024-09-19T08:46:40.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.982+0000 7f19cb610640 1 -- 172.21.15.98:0/3252117150 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19c40d0ff0 con 0x7f19c4073930 2024-09-19T08:46:40.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.982+0000 7f19b9ffb640 1 -- 172.21.15.98:0/3252117150 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f19c0011590 con 0x7f19c4073930 2024-09-19T08:46:40.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.982+0000 7f19b9ffb640 1 --2- 172.21.15.98:0/3252117150 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f19a80680a0 0x7f19a806a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:40.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.982+0000 7f19b9ffb640 1 -- 172.21.15.98:0/3252117150 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 5737+0+0 (secure 0 0 0) 0x7f19c0084820 con 0x7f19c4073930 2024-09-19T08:46:40.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.982+0000 7f19c8bab640 1 --2- 172.21.15.98:0/3252117150 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f19a80680a0 0x7f19a806a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:40.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.982+0000 7f19c8bab640 1 --2- 172.21.15.98:0/3252117150 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f19a80680a0 0x7f19a806a560 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f19b40ae000 tx=0x7f19b40a6040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:40.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:40.986+0000 7f19b9ffb640 1 -- 172.21.15.98:0/3252117150 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f19c0015020 con 0x7f19c4073930 2024-09-19T08:46:41.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.150+0000 7f19cb610640 1 -- 172.21.15.98:0/3252117150 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f19c4157180 con 0x7f19c4073930 2024-09-19T08:46:41.152 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.154+0000 7f19b9ffb640 1 -- 172.21.15.98:0/3252117150 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v157) ==== 74+0+16796 (secure 0 0 0) 0x7f19c0056c50 con 0x7f19c4073930 2024-09-19T08:46:41.152 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:46:41.152 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":157,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:46:39.868005+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:46:37.833554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-19T08:46:38.858268+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:46:41.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.154+0000 7f19cb610640 1 -- 172.21.15.98:0/3252117150 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f19a80680a0 msgr2=0x7f19a806a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:41.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.154+0000 7f19cb610640 1 --2- 172.21.15.98:0/3252117150 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f19a80680a0 0x7f19a806a560 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f19b40ae000 tx=0x7f19b40a6040 comp rx=0 tx=0).stop 2024-09-19T08:46:41.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.154+0000 7f19cb610640 1 -- 172.21.15.98:0/3252117150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19c4073930 msgr2=0x7f19c41515a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:41.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.154+0000 7f19cb610640 1 --2- 172.21.15.98:0/3252117150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19c4073930 0x7f19c41515a0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f19c000c7c0 tx=0x7f19c000cc80 comp rx=0 tx=0).stop 2024-09-19T08:46:41.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.154+0000 7f19cb610640 1 -- 172.21.15.98:0/3252117150 shutdown_connections 2024-09-19T08:46:41.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.154+0000 7f19cb610640 1 --2- 172.21.15.98:0/3252117150 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f19a80680a0 0x7f19a806a560 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:41.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.154+0000 7f19cb610640 1 --2- 172.21.15.98:0/3252117150 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19c415c220 0x7f19c414d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:41.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.154+0000 7f19cb610640 1 --2- 172.21.15.98:0/3252117150 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19c40d1530 0x7f19c40d29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:41.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.154+0000 7f19cb610640 1 --2- 172.21.15.98:0/3252117150 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19c4073930 0x7f19c41515a0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:41.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.154+0000 7f19cb610640 1 -- 172.21.15.98:0/3252117150 >> 172.21.15.98:0/3252117150 conn(0x7f19c40b40a0 msgr2=0x7f19c4152f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:41.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.154+0000 7f19cb610640 1 -- 172.21.15.98:0/3252117150 shutdown_connections 2024-09-19T08:46:41.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.154+0000 7f19cb610640 1 -- 172.21.15.98:0/3252117150 wait complete. 2024-09-19T08:46:41.171 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-19T08:46:41.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.298+0000 7fc722198640 1 Processor -- start 2024-09-19T08:46:41.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.298+0000 7fc722198640 1 -- start start 2024-09-19T08:46:41.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.298+0000 7fc722198640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc71c144d10 0x7fc71c0722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:41.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.298+0000 7fc722198640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc71c072820 0x7fc71c06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:41.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.298+0000 7fc722198640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc71c072ea0 con 0x7fc71c06c1d0 2024-09-19T08:46:41.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.298+0000 7fc722198640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fc71c072c60 con 0x7fc71c072820 2024-09-19T08:46:41.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.298+0000 7fc722198640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc71c073ab0 con 0x7fc71c144d10 2024-09-19T08:46:41.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.298+0000 7fc71bfff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fc71c06c1d0 0x7fc71c0b4520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:54960/0 (socket says 172.21.15.98:54960) 2024-09-19T08:46:41.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.298+0000 7fc71bfff640 1 -- 172.21.15.98:0/1213971013 learned_addr learned my addr 172.21.15.98:0/1213971013 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:41.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.298+0000 7fc712ffd640 1 --2- 172.21.15.98:0/1213971013 >> v2:172.21.15.139:3300/0 conn(0x7fc71c072820 0x7fc71c06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:41.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.298+0000 7fc71b7fe640 1 --2- 172.21.15.98:0/1213971013 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc71c144d10 0x7fc71c0722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:41.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.302+0000 7fc71affd640 1 -- 172.21.15.98:0/1213971013 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4239514550 0 0) 0x7fc71c072ea0 con 0x7fc71c06c1d0 2024-09-19T08:46:41.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.302+0000 7fc71affd640 1 -- 172.21.15.98:0/1213971013 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc708003530 con 0x7fc71c06c1d0 2024-09-19T08:46:41.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.302+0000 7fc71affd640 1 -- 172.21.15.98:0/1213971013 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3171491251 0 0) 0x7fc708003530 con 0x7fc71c06c1d0 2024-09-19T08:46:41.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.302+0000 7fc71affd640 1 -- 172.21.15.98:0/1213971013 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc71c144d10 msgr2=0x7fc71c0722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:41.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.302+0000 7fc71affd640 1 --2- 172.21.15.98:0/1213971013 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc71c144d10 0x7fc71c0722e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:41.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.302+0000 7fc71affd640 1 -- 172.21.15.98:0/1213971013 >> v2:172.21.15.139:3300/0 conn(0x7fc71c072820 msgr2=0x7fc71c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:41.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.302+0000 7fc71affd640 1 --2- 172.21.15.98:0/1213971013 >> v2:172.21.15.139:3300/0 conn(0x7fc71c072820 0x7fc71c06bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:41.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.302+0000 7fc71affd640 1 -- 172.21.15.98:0/1213971013 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc71c073180 con 0x7fc71c06c1d0 2024-09-19T08:46:41.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.302+0000 7fc71affd640 1 -- 172.21.15.98:0/1213971013 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fc714001a00 con 0x7fc71c06c1d0 2024-09-19T08:46:41.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.302+0000 7fc71affd640 1 -- 172.21.15.98:0/1213971013 >> v1:172.21.15.98:6789/0 conn(0x7fc71c06c1d0 legacy=0x7fc71c0b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:41.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.302+0000 7fc71affd640 1 --2- 172.21.15.98:0/1213971013 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7080039c0 0x7fc708003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:41.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.302+0000 7fc71affd640 1 --2- 172.21.15.98:0/1213971013 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc71c144d10 0x7fc708004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:41.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.302+0000 7fc71affd640 1 --2- 172.21.15.98:0/1213971013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc708004d50 0x7fc7080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:41.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.302+0000 7fc71affd640 1 -- 172.21.15.98:0/1213971013 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c072c60 con 0x7fc708004d50 2024-09-19T08:46:41.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.302+0000 7fc71affd640 1 -- 172.21.15.98:0/1213971013 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c073ab0 con 0x7fc7080039c0 2024-09-19T08:46:41.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.302+0000 7fc71affd640 1 -- 172.21.15.98:0/1213971013 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc70800b9d0 con 0x7fc71c144d10 2024-09-19T08:46:41.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.302+0000 7fc71bfff640 1 --2- 172.21.15.98:0/1213971013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc708004d50 0x7fc7080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:41.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.302+0000 7fc712ffd640 1 --2- 172.21.15.98:0/1213971013 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc71c144d10 0x7fc708004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:41.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.302+0000 7fc71b7fe640 1 --2- 172.21.15.98:0/1213971013 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7080039c0 0x7fc708003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:41.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.302+0000 7fc71bfff640 1 -- 172.21.15.98:0/1213971013 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc71c144d10 msgr2=0x7fc708004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:41.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.302+0000 7fc71bfff640 1 --2- 172.21.15.98:0/1213971013 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc71c144d10 0x7fc708004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:41.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.302+0000 7fc71bfff640 1 -- 172.21.15.98:0/1213971013 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7080039c0 msgr2=0x7fc708003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:41.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.302+0000 7fc71bfff640 1 --2- 172.21.15.98:0/1213971013 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc7080039c0 0x7fc708003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:41.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.302+0000 7fc71bfff640 1 -- 172.21.15.98:0/1213971013 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc708003530 con 0x7fc708004d50 2024-09-19T08:46:41.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.302+0000 7fc71bfff640 1 --2- 172.21.15.98:0/1213971013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc708004d50 0x7fc7080051d0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fc7140013e0 tx=0x7fc71401d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37060a8bff256c35 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:41.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.302+0000 7fc71affd640 1 -- 172.21.15.98:0/1213971013 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71400a4e0 con 0x7fc708004d50 2024-09-19T08:46:41.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.302+0000 7fc71affd640 1 -- 172.21.15.98:0/1213971013 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7140072f0 con 0x7fc708004d50 2024-09-19T08:46:41.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.302+0000 7fc71affd640 1 -- 172.21.15.98:0/1213971013 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7140076e0 con 0x7fc708004d50 2024-09-19T08:46:41.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.302+0000 7fc722198640 1 -- 172.21.15.98:0/1213971013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc708004d50 msgr2=0x7fc7080051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:41.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.302+0000 7fc722198640 1 --2- 172.21.15.98:0/1213971013 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc708004d50 0x7fc7080051d0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fc7140013e0 tx=0x7fc71401d3a0 comp rx=0 tx=0).stop 2024-09-19T08:46:41.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.302+0000 7fc71bfff640 1 -- 172.21.15.98:0/1213971013 reap_dead start 2024-09-19T08:46:41.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.302+0000 7fc722198640 1 -- 172.21.15.98:0/1213971013 shutdown_connections 2024-09-19T08:46:41.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.302+0000 7fc722198640 1 -- 172.21.15.98:0/1213971013 >> 172.21.15.98:0/1213971013 conn(0x7fc71c0b2010 msgr2=0x7fc71c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:41.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.302+0000 7fc722198640 1 -- 172.21.15.98:0/1213971013 shutdown_connections 2024-09-19T08:46:41.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.302+0000 7fc722198640 1 -- 172.21.15.98:0/1213971013 wait complete. 2024-09-19T08:46:41.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.306+0000 7fc722198640 1 Processor -- start 2024-09-19T08:46:41.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.306+0000 7fc722198640 1 -- start start 2024-09-19T08:46:41.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.306+0000 7fc722198640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc708004d50 0x7fc71c1f5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:41.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.306+0000 7fc722198640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc71c1f5790 0x7fc71c0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:41.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.306+0000 7fc722198640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc71c0b3950 0x7fc71c0ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:41.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.306+0000 7fc722198640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c073740 con 0x7fc71c0b3950 2024-09-19T08:46:41.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.306+0000 7fc722198640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc71c1450f0 con 0x7fc71c1f5790 2024-09-19T08:46:41.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.306+0000 7fc722198640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc71c0af8b0 con 0x7fc708004d50 2024-09-19T08:46:41.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.306+0000 7fc71bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc71c0b3950 0x7fc71c0ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:41.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.306+0000 7fc71bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc71c0b3950 0x7fc71c0ae6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54260/0 (socket says 172.21.15.98:54260) 2024-09-19T08:46:41.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.306+0000 7fc71bfff640 1 -- 172.21.15.98:0/1840914575 learned_addr learned my addr 172.21.15.98:0/1840914575 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:41.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.306+0000 7fc71b7fe640 1 --2- 172.21.15.98:0/1840914575 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc708004d50 0x7fc71c1f5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:41.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.306+0000 7fc712ffd640 1 --2- 172.21.15.98:0/1840914575 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc71c1f5790 0x7fc71c0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:41.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.306+0000 7fc71bfff640 1 -- 172.21.15.98:0/1840914575 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc708004d50 msgr2=0x7fc71c1f5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:41.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.306+0000 7fc71bfff640 1 --2- 172.21.15.98:0/1840914575 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc708004d50 0x7fc71c1f5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:41.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.306+0000 7fc71bfff640 1 -- 172.21.15.98:0/1840914575 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc71c1f5790 msgr2=0x7fc71c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:41.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.306+0000 7fc71bfff640 1 --2- 172.21.15.98:0/1840914575 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc71c1f5790 0x7fc71c0b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:41.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.306+0000 7fc71bfff640 1 -- 172.21.15.98:0/1840914575 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc71c1f2000 con 0x7fc71c0b3950 2024-09-19T08:46:41.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.306+0000 7fc71bfff640 1 --2- 172.21.15.98:0/1840914575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc71c0b3950 0x7fc71c0ae6a0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fc714007040 tx=0x7fc708005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:41.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.306+0000 7fc718ff9640 1 -- 172.21.15.98:0/1840914575 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7140b2040 con 0x7fc71c0b3950 2024-09-19T08:46:41.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.306+0000 7fc718ff9640 1 -- 172.21.15.98:0/1840914575 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc71400a4c0 con 0x7fc71c0b3950 2024-09-19T08:46:41.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.306+0000 7fc722198640 1 -- 172.21.15.98:0/1840914575 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc71c0b3ff0 con 0x7fc71c0b3950 2024-09-19T08:46:41.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.306+0000 7fc718ff9640 1 -- 172.21.15.98:0/1840914575 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7140017c0 con 0x7fc71c0b3950 2024-09-19T08:46:41.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.306+0000 7fc722198640 1 -- 172.21.15.98:0/1840914575 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc71c0b45b0 con 0x7fc71c0b3950 2024-09-19T08:46:41.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.310+0000 7fc722198640 1 -- 172.21.15.98:0/1840914575 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc71c06bc90 con 0x7fc71c0b3950 2024-09-19T08:46:41.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.310+0000 7fc718ff9640 1 -- 172.21.15.98:0/1840914575 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7fc7140bd020 con 0x7fc71c0b3950 2024-09-19T08:46:41.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.310+0000 7fc718ff9640 1 --2- 172.21.15.98:0/1840914575 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc6fc0680d0 0x7fc6fc06a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:41.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.310+0000 7fc718ff9640 1 -- 172.21.15.98:0/1840914575 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5737+0+0 (secure 0 0 0) 0x7fc71411e6e0 con 0x7fc71c0b3950 2024-09-19T08:46:41.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.310+0000 7fc71b7fe640 1 --2- 172.21.15.98:0/1840914575 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc6fc0680d0 0x7fc6fc06a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:41.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.310+0000 7fc71b7fe640 1 --2- 172.21.15.98:0/1840914575 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc6fc0680d0 0x7fc6fc06a590 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc70c0099b0 tx=0x7fc70c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:41.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.314+0000 7fc718ff9640 1 -- 172.21.15.98:0/1840914575 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc71c1f2000 con 0x7fc71c0b3950 2024-09-19T08:46:41.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:41.518+0000 7fc722198640 1 -- 172.21.15.98:0/1840914575 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc71c1f5b70 con 0x7fc71c0b3950 2024-09-19T08:46:42.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.518+0000 7fc718ff9640 1 -- 172.21.15.98:0/1840914575 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v90) ==== 118+0+0 (secure 0 0 0) 0x7fc7140170b0 con 0x7fc71c0b3950 2024-09-19T08:46:42.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.526+0000 7fc722198640 1 -- 172.21.15.98:0/1840914575 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc6fc0680d0 msgr2=0x7fc6fc06a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:42.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.526+0000 7fc722198640 1 --2- 172.21.15.98:0/1840914575 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc6fc0680d0 0x7fc6fc06a590 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc70c0099b0 tx=0x7fc70c008040 comp rx=0 tx=0).stop 2024-09-19T08:46:42.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.526+0000 7fc722198640 1 -- 172.21.15.98:0/1840914575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc71c0b3950 msgr2=0x7fc71c0ae6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:42.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.526+0000 7fc722198640 1 --2- 172.21.15.98:0/1840914575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc71c0b3950 0x7fc71c0ae6a0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fc714007040 tx=0x7fc708005840 comp rx=0 tx=0).stop 2024-09-19T08:46:42.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.526+0000 7fc722198640 1 -- 172.21.15.98:0/1840914575 shutdown_connections 2024-09-19T08:46:42.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.526+0000 7fc722198640 1 --2- 172.21.15.98:0/1840914575 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc6fc0680d0 0x7fc6fc06a590 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:42.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.526+0000 7fc722198640 1 --2- 172.21.15.98:0/1840914575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc71c0b3950 0x7fc71c0ae6a0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:42.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.526+0000 7fc722198640 1 --2- 172.21.15.98:0/1840914575 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc71c1f5790 0x7fc71c0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:42.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.526+0000 7fc722198640 1 --2- 172.21.15.98:0/1840914575 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc708004d50 0x7fc71c1f5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:42.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.526+0000 7fc722198640 1 -- 172.21.15.98:0/1840914575 >> 172.21.15.98:0/1840914575 conn(0x7fc71c0b2010 msgr2=0x7fc71c0ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:42.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.526+0000 7fc722198640 1 -- 172.21.15.98:0/1840914575 shutdown_connections 2024-09-19T08:46:42.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.526+0000 7fc722198640 1 -- 172.21.15.98:0/1840914575 wait complete. 2024-09-19T08:46:42.549 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-19T08:46:42.549 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-19T08:46:42.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.682+0000 7fdf52357640 1 Processor -- start 2024-09-19T08:46:42.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.682+0000 7fdf52357640 1 -- start start 2024-09-19T08:46:42.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.682+0000 7fdf52357640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fdf4c06c3a0 0x7fdf4c0713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:42.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.682+0000 7fdf52357640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf4c0718f0 0x7fdf4c0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:42.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.682+0000 7fdf52357640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf4c073530 con 0x7fdf4c145b70 2024-09-19T08:46:42.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.682+0000 7fdf52357640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fdf4c073810 con 0x7fdf4c06c3a0 2024-09-19T08:46:42.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.682+0000 7fdf52357640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdf4c073af0 con 0x7fdf4c0718f0 2024-09-19T08:46:42.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.682+0000 7fdf4b7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fdf4c06c3a0 0x7fdf4c0713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:42.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.682+0000 7fdf4b7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fdf4c06c3a0 0x7fdf4c0713b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54276/0 (socket says 172.21.15.98:54276) 2024-09-19T08:46:42.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.682+0000 7fdf4b7fe640 1 -- 172.21.15.98:0/1547333283 learned_addr learned my addr 172.21.15.98:0/1547333283 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:42.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.682+0000 7fdf508f4640 1 --2- 172.21.15.98:0/1547333283 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf4c0718f0 0x7fdf4c0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:42.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.682+0000 7fdf4b7fe640 1 -- 172.21.15.98:0/1547333283 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf4c0718f0 msgr2=0x7fdf4c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:42.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.682+0000 7fdf4b7fe640 1 --2- 172.21.15.98:0/1547333283 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf4c0718f0 0x7fdf4c0b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:42.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.682+0000 7fdf4b7fe640 1 -- 172.21.15.98:0/1547333283 >> v1:172.21.15.139:6789/0 conn(0x7fdf4c145b70 legacy=0x7fdf4c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:42.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.682+0000 7fdf4b7fe640 1 -- 172.21.15.98:0/1547333283 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf4c073dd0 con 0x7fdf4c06c3a0 2024-09-19T08:46:42.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.682+0000 7fdf508f4640 1 --2- 172.21.15.98:0/1547333283 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf4c0718f0 0x7fdf4c0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:42.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.686+0000 7fdf4b7fe640 1 --2- 172.21.15.98:0/1547333283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf4c06c3a0 0x7fdf4c0713b0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009870 tx=0x7fdf3c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83156cbfdad9bfdb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:42.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.686+0000 7fdf4affd640 1 -- 172.21.15.98:0/1547333283 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c0b0070 con 0x7fdf4c06c3a0 2024-09-19T08:46:42.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.686+0000 7fdf4affd640 1 -- 172.21.15.98:0/1547333283 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf3c0ab600 con 0x7fdf4c06c3a0 2024-09-19T08:46:42.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.686+0000 7fdf4affd640 1 -- 172.21.15.98:0/1547333283 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c0ab9d0 con 0x7fdf4c06c3a0 2024-09-19T08:46:42.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.686+0000 7fdf52357640 1 -- 172.21.15.98:0/1547333283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf4c06c3a0 msgr2=0x7fdf4c0713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:42.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.686+0000 7fdf52357640 1 --2- 172.21.15.98:0/1547333283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf4c06c3a0 0x7fdf4c0713b0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009870 tx=0x7fdf3c01c490 comp rx=0 tx=0).stop 2024-09-19T08:46:42.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.686+0000 7fdf52357640 1 -- 172.21.15.98:0/1547333283 shutdown_connections 2024-09-19T08:46:42.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.686+0000 7fdf52357640 1 --2- 172.21.15.98:0/1547333283 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf4c0718f0 0x7fdf4c0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:42.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.686+0000 7fdf52357640 1 --2- 172.21.15.98:0/1547333283 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf4c06c3a0 0x7fdf4c0713b0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:42.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.686+0000 7fdf52357640 1 -- 172.21.15.98:0/1547333283 >> 172.21.15.98:0/1547333283 conn(0x7fdf4c0b2010 msgr2=0x7fdf4c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:42.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.686+0000 7fdf52357640 1 -- 172.21.15.98:0/1547333283 shutdown_connections 2024-09-19T08:46:42.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.686+0000 7fdf52357640 1 -- 172.21.15.98:0/1547333283 wait complete. 2024-09-19T08:46:42.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.690+0000 7fdf52357640 1 Processor -- start 2024-09-19T08:46:42.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.690+0000 7fdf52357640 1 -- start start 2024-09-19T08:46:42.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.690+0000 7fdf52357640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf4c06c3a0 0x7fdf4c144880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:42.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.690+0000 7fdf52357640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf4c0718f0 0x7fdf4c13ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:42.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.690+0000 7fdf52357640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf4c145b70 0x7fdf4c13b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:42.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.690+0000 7fdf52357640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4c0b44f0 con 0x7fdf4c145b70 2024-09-19T08:46:42.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.690+0000 7fdf52357640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4c0b4370 con 0x7fdf4c06c3a0 2024-09-19T08:46:42.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.690+0000 7fdf52357640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fdf4c0b4670 con 0x7fdf4c0718f0 2024-09-19T08:46:42.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.690+0000 7fdf508f4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf4c145b70 0x7fdf4c13b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:42.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.690+0000 7fdf508f4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf4c145b70 0x7fdf4c13b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54292/0 (socket says 172.21.15.98:54292) 2024-09-19T08:46:42.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.690+0000 7fdf508f4640 1 -- 172.21.15.98:0/4028627873 learned_addr learned my addr 172.21.15.98:0/4028627873 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:42.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.690+0000 7fdf4b7fe640 1 --2- 172.21.15.98:0/4028627873 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf4c0718f0 0x7fdf4c13ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:42.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.690+0000 7fdf508f4640 1 -- 172.21.15.98:0/4028627873 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf4c0718f0 msgr2=0x7fdf4c13ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:42.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.690+0000 7fdf508f4640 1 --2- 172.21.15.98:0/4028627873 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf4c0718f0 0x7fdf4c13ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:42.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.690+0000 7fdf508f4640 1 -- 172.21.15.98:0/4028627873 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf4c06c3a0 msgr2=0x7fdf4c144880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:42.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.690+0000 7fdf508f4640 1 --2- 172.21.15.98:0/4028627873 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf4c06c3a0 0x7fdf4c144880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:42.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.690+0000 7fdf508f4640 1 -- 172.21.15.98:0/4028627873 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf4c15f790 con 0x7fdf4c145b70 2024-09-19T08:46:42.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.690+0000 7fdf508f4640 1 --2- 172.21.15.98:0/4028627873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf4c145b70 0x7fdf4c13b760 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400cef0 tx=0x7fdf4400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:42.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.690+0000 7fdf48ff9640 1 -- 172.21.15.98:0/4028627873 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4401a070 con 0x7fdf4c145b70 2024-09-19T08:46:42.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.690+0000 7fdf48ff9640 1 -- 172.21.15.98:0/4028627873 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf4400d860 con 0x7fdf4c145b70 2024-09-19T08:46:42.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.690+0000 7fdf52357640 1 -- 172.21.15.98:0/4028627873 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf4c15fa70 con 0x7fdf4c145b70 2024-09-19T08:46:42.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.690+0000 7fdf48ff9640 1 -- 172.21.15.98:0/4028627873 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4400dc50 con 0x7fdf4c145b70 2024-09-19T08:46:42.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.690+0000 7fdf52357640 1 -- 172.21.15.98:0/4028627873 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf4c158db0 con 0x7fdf4c145b70 2024-09-19T08:46:42.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.690+0000 7fdf52357640 1 -- 172.21.15.98:0/4028627873 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf4c0713b0 con 0x7fdf4c145b70 2024-09-19T08:46:42.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.690+0000 7fdf48ff9640 1 -- 172.21.15.98:0/4028627873 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7fdf4401d070 con 0x7fdf4c145b70 2024-09-19T08:46:42.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.694+0000 7fdf48ff9640 1 --2- 172.21.15.98:0/4028627873 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fdf380680a0 0x7fdf3806a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:42.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.694+0000 7fdf48ff9640 1 -- 172.21.15.98:0/4028627873 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5737+0+0 (secure 0 0 0) 0x7fdf440876b0 con 0x7fdf4c145b70 2024-09-19T08:46:42.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.694+0000 7fdf4bfff640 1 --2- 172.21.15.98:0/4028627873 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fdf380680a0 0x7fdf3806a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:42.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.694+0000 7fdf4bfff640 1 --2- 172.21.15.98:0/4028627873 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fdf380680a0 0x7fdf3806a560 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdf40009670 tx=0x7fdf40006bd0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:42.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.698+0000 7fdf48ff9640 1 -- 172.21.15.98:0/4028627873 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fdf440573b0 con 0x7fdf4c145b70 2024-09-19T08:46:42.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:42.894+0000 7fdf52357640 1 -- 172.21.15.98:0/4028627873 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fdf4c0b1ac0 con 0x7fdf4c145b70 2024-09-19T08:46:43.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.530+0000 7fdf48ff9640 1 -- 172.21.15.98:0/4028627873 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v91) ==== 122+0+0 (secure 0 0 0) 0x7fdf44056cb0 con 0x7fdf4c145b70 2024-09-19T08:46:43.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.534+0000 7fdf52357640 1 -- 172.21.15.98:0/4028627873 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fdf380680a0 msgr2=0x7fdf3806a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:43.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.534+0000 7fdf52357640 1 --2- 172.21.15.98:0/4028627873 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fdf380680a0 0x7fdf3806a560 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdf40009670 tx=0x7fdf40006bd0 comp rx=0 tx=0).stop 2024-09-19T08:46:43.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.534+0000 7fdf52357640 1 -- 172.21.15.98:0/4028627873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf4c145b70 msgr2=0x7fdf4c13b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:43.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.534+0000 7fdf52357640 1 --2- 172.21.15.98:0/4028627873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf4c145b70 0x7fdf4c13b760 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400cef0 tx=0x7fdf4400a710 comp rx=0 tx=0).stop 2024-09-19T08:46:43.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.534+0000 7fdf52357640 1 -- 172.21.15.98:0/4028627873 shutdown_connections 2024-09-19T08:46:43.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.534+0000 7fdf52357640 1 --2- 172.21.15.98:0/4028627873 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fdf380680a0 0x7fdf3806a560 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:43.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.534+0000 7fdf52357640 1 --2- 172.21.15.98:0/4028627873 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fdf4c145b70 0x7fdf4c13b760 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:43.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.534+0000 7fdf52357640 1 --2- 172.21.15.98:0/4028627873 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fdf4c0718f0 0x7fdf4c13ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:43.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.534+0000 7fdf52357640 1 --2- 172.21.15.98:0/4028627873 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdf4c06c3a0 0x7fdf4c144880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:43.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.534+0000 7fdf52357640 1 -- 172.21.15.98:0/4028627873 >> 172.21.15.98:0/4028627873 conn(0x7fdf4c0b2010 msgr2=0x7fdf4c141ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:43.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.534+0000 7fdf52357640 1 -- 172.21.15.98:0/4028627873 shutdown_connections 2024-09-19T08:46:43.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.534+0000 7fdf52357640 1 -- 172.21.15.98:0/4028627873 wait complete. 2024-09-19T08:46:43.554 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-19T08:46:43.554 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:46:43.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.678+0000 7fea03b94640 1 Processor -- start 2024-09-19T08:46:43.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.678+0000 7fea03b94640 1 -- start start 2024-09-19T08:46:43.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.682+0000 7fea03b94640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe9fc158fd0 0x7fe9fc0b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:43.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.682+0000 7fea03b94640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe9fc06c3a0 0x7fe9fc0ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:43.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.682+0000 7fea03b94640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9fc0ccc60 con 0x7fe9fc0b4360 2024-09-19T08:46:43.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.682+0000 7fea03b94640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fe9fc1593b0 con 0x7fe9fc158fd0 2024-09-19T08:46:43.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.682+0000 7fea03b94640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe9fc0ccf40 con 0x7fe9fc06c3a0 2024-09-19T08:46:43.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.682+0000 7fea01930640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe9fc158fd0 0x7fe9fc0b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:43.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.682+0000 7fea0112f640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fe9fc0b4360 0x7fe9fc06bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:54972/0 (socket says 172.21.15.98:54972) 2024-09-19T08:46:43.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.682+0000 7fea0112f640 1 -- 172.21.15.98:0/1290779079 learned_addr learned my addr 172.21.15.98:0/1290779079 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:43.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.682+0000 7fea01930640 1 -- 172.21.15.98:0/1290779079 >> v2:172.21.15.139:3300/0 conn(0x7fe9fc06c3a0 msgr2=0x7fe9fc0ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:43.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.682+0000 7fea02131640 1 --2- 172.21.15.98:0/1290779079 >> v2:172.21.15.139:3300/0 conn(0x7fe9fc06c3a0 0x7fe9fc0ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:43.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.682+0000 7fea0092e640 1 -- 172.21.15.98:0/1290779079 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 669894493 0 0) 0x7fe9fc0ccc60 con 0x7fe9fc0b4360 2024-09-19T08:46:43.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.682+0000 7fea01930640 1 --2- 172.21.15.98:0/1290779079 >> v2:172.21.15.139:3300/0 conn(0x7fe9fc06c3a0 0x7fe9fc0ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:43.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.682+0000 7fea01930640 1 -- 172.21.15.98:0/1290779079 >> v1:172.21.15.98:6789/0 conn(0x7fe9fc0b4360 legacy=0x7fe9fc06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:43.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.682+0000 7fea01930640 1 -- 172.21.15.98:0/1290779079 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9fc0cd220 con 0x7fe9fc158fd0 2024-09-19T08:46:43.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.682+0000 7fea02131640 1 --2- 172.21.15.98:0/1290779079 >> v2:172.21.15.139:3300/0 conn(0x7fe9fc06c3a0 0x7fe9fc0ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:43.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.682+0000 7fea01930640 1 --2- 172.21.15.98:0/1290779079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9fc158fd0 0x7fe9fc0b3e20 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc0afe30 tx=0x7fe9f801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d608f81cf591fb6b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:43.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.682+0000 7fea0092e640 1 -- 172.21.15.98:0/1290779079 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f80b1070 con 0x7fe9fc158fd0 2024-09-19T08:46:43.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.682+0000 7fea0092e640 1 -- 172.21.15.98:0/1290779079 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9f80ac620 con 0x7fe9fc158fd0 2024-09-19T08:46:43.683 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.682+0000 7fea0092e640 1 -- 172.21.15.98:0/1290779079 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f80ac9f0 con 0x7fe9fc158fd0 2024-09-19T08:46:43.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.686+0000 7fea03b94640 1 -- 172.21.15.98:0/1290779079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9fc158fd0 msgr2=0x7fe9fc0b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:43.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.686+0000 7fea03b94640 1 --2- 172.21.15.98:0/1290779079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9fc158fd0 0x7fe9fc0b3e20 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc0afe30 tx=0x7fe9f801c3c0 comp rx=0 tx=0).stop 2024-09-19T08:46:43.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.686+0000 7fea03b94640 1 -- 172.21.15.98:0/1290779079 shutdown_connections 2024-09-19T08:46:43.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.686+0000 7fea03b94640 1 --2- 172.21.15.98:0/1290779079 >> v2:172.21.15.139:3300/0 conn(0x7fe9fc06c3a0 0x7fe9fc0ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:43.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.686+0000 7fea03b94640 1 --2- 172.21.15.98:0/1290779079 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9fc158fd0 0x7fe9fc0b3e20 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:43.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.686+0000 7fea03b94640 1 -- 172.21.15.98:0/1290779079 >> 172.21.15.98:0/1290779079 conn(0x7fe9fc0b2010 msgr2=0x7fe9fc0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:43.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.686+0000 7fea03b94640 1 -- 172.21.15.98:0/1290779079 shutdown_connections 2024-09-19T08:46:43.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.686+0000 7fea03b94640 1 -- 172.21.15.98:0/1290779079 wait complete. 2024-09-19T08:46:43.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.686+0000 7fea03b94640 1 Processor -- start 2024-09-19T08:46:43.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.686+0000 7fea03b94640 1 -- start start 2024-09-19T08:46:43.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.686+0000 7fea03b94640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe9fc06c3a0 0x7fe9fc0b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:43.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.686+0000 7fea03b94640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9fc0b4360 0x7fe9fc144750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:43.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.686+0000 7fea03b94640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9fc158fd0 0x7fe9fc1f5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:43.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.686+0000 7fea03b94640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc0cf710 con 0x7fe9fc0b4360 2024-09-19T08:46:43.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.686+0000 7fea03b94640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc0cf590 con 0x7fe9fc158fd0 2024-09-19T08:46:43.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.686+0000 7fea03b94640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe9fc0cf890 con 0x7fe9fc06c3a0 2024-09-19T08:46:43.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.686+0000 7fea0112f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9fc0b4360 0x7fe9fc144750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:43.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.686+0000 7fea0112f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9fc0b4360 0x7fe9fc144750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54314/0 (socket says 172.21.15.98:54314) 2024-09-19T08:46:43.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.686+0000 7fea0112f640 1 -- 172.21.15.98:0/246526669 learned_addr learned my addr 172.21.15.98:0/246526669 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:43.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.686+0000 7fea01930640 1 --2- 172.21.15.98:0/246526669 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe9fc06c3a0 0x7fe9fc0b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:43.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.686+0000 7fea02131640 1 --2- 172.21.15.98:0/246526669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9fc158fd0 0x7fe9fc1f5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:43.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.686+0000 7fea0112f640 1 -- 172.21.15.98:0/246526669 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe9fc06c3a0 msgr2=0x7fe9fc0b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:43.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.686+0000 7fea0112f640 1 --2- 172.21.15.98:0/246526669 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe9fc06c3a0 0x7fe9fc0b0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:43.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.686+0000 7fea0112f640 1 -- 172.21.15.98:0/246526669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9fc158fd0 msgr2=0x7fe9fc1f5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:43.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.686+0000 7fea0112f640 1 --2- 172.21.15.98:0/246526669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9fc158fd0 0x7fe9fc1f5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:43.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.686+0000 7fea0112f640 1 -- 172.21.15.98:0/246526669 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9fc0b1220 con 0x7fe9fc0b4360 2024-09-19T08:46:43.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.686+0000 7fea02131640 1 --2- 172.21.15.98:0/246526669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9fc158fd0 0x7fe9fc1f5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:43.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.686+0000 7fea0112f640 1 --2- 172.21.15.98:0/246526669 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9fc0b4360 0x7fe9fc144750 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec0011f0 tx=0x7fe9ec00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:43.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.686+0000 7fe9f27fc640 1 -- 172.21.15.98:0/246526669 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec018070 con 0x7fe9fc0b4360 2024-09-19T08:46:43.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.686+0000 7fe9f27fc640 1 -- 172.21.15.98:0/246526669 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9ec0074e0 con 0x7fe9fc0b4360 2024-09-19T08:46:43.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.686+0000 7fea03b94640 1 -- 172.21.15.98:0/246526669 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9fc1451f0 con 0x7fe9fc0b4360 2024-09-19T08:46:43.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.686+0000 7fe9f27fc640 1 -- 172.21.15.98:0/246526669 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec004a10 con 0x7fe9fc0b4360 2024-09-19T08:46:43.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.686+0000 7fea03b94640 1 -- 172.21.15.98:0/246526669 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9fc1457b0 con 0x7fe9fc0b4360 2024-09-19T08:46:43.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.690+0000 7fea03b94640 1 -- 172.21.15.98:0/246526669 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9fc0008e0 con 0x7fe9fc0b4360 2024-09-19T08:46:43.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.690+0000 7fe9f27fc640 1 -- 172.21.15.98:0/246526669 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7fe9ec01b070 con 0x7fe9fc0b4360 2024-09-19T08:46:43.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.690+0000 7fe9f27fc640 1 --2- 172.21.15.98:0/246526669 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe9e00680a0 0x7fe9e006a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:43.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.690+0000 7fe9f27fc640 1 -- 172.21.15.98:0/246526669 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5737+0+0 (secure 0 0 0) 0x7fe9ec0839a0 con 0x7fe9fc0b4360 2024-09-19T08:46:43.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.690+0000 7fea01930640 1 --2- 172.21.15.98:0/246526669 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe9e00680a0 0x7fe9e006a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:43.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.690+0000 7fea01930640 1 --2- 172.21.15.98:0/246526669 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe9e00680a0 0x7fe9e006a560 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe9f80ae000 tx=0x7fe9f80a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:43.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.694+0000 7fe9f27fc640 1 -- 172.21.15.98:0/246526669 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe9fc0b1220 con 0x7fe9fc0b4360 2024-09-19T08:46:43.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.886+0000 7fea03b94640 1 -- 172.21.15.98:0/246526669 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9fc1f6820 con 0x7fe9fc0b4360 2024-09-19T08:46:43.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.886+0000 7fe9f27fc640 1 -- 172.21.15.98:0/246526669 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5233 (secure 0 0 0) 0x7fe9ec055dd0 con 0x7fe9fc0b4360 2024-09-19T08:46:43.887 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:46:43.887 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":91,"btime":"2024-09-19T08:46:43:523818+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/1752784745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":1752784745},{"type":"v1","addr":"172.21.15.139:6834","nonce":1752784745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":6212,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3564493643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6833","nonce":3564493643},{"type":"v1","addr":"172.21.15.139:6835","nonce":3564493643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":8763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6834/3374714194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3374714194},{"type":"v1","addr":"172.21.15.98:6834","nonce":3374714194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:46:39.868735+0000","modified":"2024-09-19T08:46:42.899681+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8766},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8766":{"gid":8766,"name":"c","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/3734544897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6833","nonce":3734544897},{"type":"v1","addr":"172.21.15.98:6835","nonce":3734544897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8766,"qdb_cluster":[8766]},"id":12}]} 2024-09-19T08:46:43.887 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 91 2024-09-19T08:46:43.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.890+0000 7fea03b94640 1 -- 172.21.15.98:0/246526669 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe9e00680a0 msgr2=0x7fe9e006a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:43.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.890+0000 7fea03b94640 1 --2- 172.21.15.98:0/246526669 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe9e00680a0 0x7fe9e006a560 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe9f80ae000 tx=0x7fe9f80a7000 comp rx=0 tx=0).stop 2024-09-19T08:46:43.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.890+0000 7fea03b94640 1 -- 172.21.15.98:0/246526669 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9fc0b4360 msgr2=0x7fe9fc144750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:43.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.890+0000 7fea03b94640 1 --2- 172.21.15.98:0/246526669 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9fc0b4360 0x7fe9fc144750 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec0011f0 tx=0x7fe9ec00c9b0 comp rx=0 tx=0).stop 2024-09-19T08:46:43.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.890+0000 7fea03b94640 1 -- 172.21.15.98:0/246526669 shutdown_connections 2024-09-19T08:46:43.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.890+0000 7fea03b94640 1 --2- 172.21.15.98:0/246526669 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe9e00680a0 0x7fe9e006a560 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:43.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.890+0000 7fea03b94640 1 --2- 172.21.15.98:0/246526669 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe9fc158fd0 0x7fe9fc1f5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:43.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.890+0000 7fea03b94640 1 --2- 172.21.15.98:0/246526669 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe9fc0b4360 0x7fe9fc144750 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:43.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.890+0000 7fea03b94640 1 --2- 172.21.15.98:0/246526669 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe9fc06c3a0 0x7fe9fc0b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:43.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.890+0000 7fea03b94640 1 -- 172.21.15.98:0/246526669 >> 172.21.15.98:0/246526669 conn(0x7fe9fc0b2010 msgr2=0x7fe9fc0cfc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:43.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.890+0000 7fea03b94640 1 -- 172.21.15.98:0/246526669 shutdown_connections 2024-09-19T08:46:43.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:43.890+0000 7fea03b94640 1 -- 172.21.15.98:0/246526669 wait complete. 2024-09-19T08:46:43.904 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:46:44.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.026+0000 7f088adfd640 1 Processor -- start 2024-09-19T08:46:44.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.026+0000 7f088adfd640 1 -- start start 2024-09-19T08:46:44.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.026+0000 7f088adfd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f088415c550 0x7f088406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:44.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.026+0000 7f088adfd640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f088406d310 con 0x7f088406c1d0 2024-09-19T08:46:44.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.026+0000 7f088adfd640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f088406d5f0 con 0x7f08840cfad0 2024-09-19T08:46:44.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.026+0000 7f088adfd640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f088406c640 con 0x7f088415c550 2024-09-19T08:46:44.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.026+0000 7f0888b99640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f088415c550 0x7f088406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:44.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.026+0000 7f0888b99640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f088415c550 0x7f088406bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54330/0 (socket says 172.21.15.98:54330) 2024-09-19T08:46:44.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.026+0000 7f0888b99640 1 -- 172.21.15.98:0/4422683 learned_addr learned my addr 172.21.15.98:0/4422683 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:44.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.026+0000 7f0888b99640 1 -- 172.21.15.98:0/4422683 >> v1:172.21.15.139:6789/0 conn(0x7f08840cfad0 legacy=0x7f08840723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:44.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.026+0000 7f0888b99640 1 -- 172.21.15.98:0/4422683 >> v1:172.21.15.98:6789/0 conn(0x7f088406c1d0 legacy=0x7f08840cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:44.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.026+0000 7f0888b99640 1 -- 172.21.15.98:0/4422683 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f088406d8d0 con 0x7f088415c550 2024-09-19T08:46:44.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.026+0000 7f08837fe640 1 -- 172.21.15.98:0/4422683 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 929223886 0 0) 0x7f088406d310 con 0x7f088406c1d0 2024-09-19T08:46:44.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.030+0000 7f0888b99640 1 --2- 172.21.15.98:0/4422683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f088415c550 0x7f088406bc90 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f0878009870 tx=0x7f087801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=907ad0d21c5e3272 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:44.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.030+0000 7f08837fe640 1 -- 172.21.15.98:0/4422683 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08780b2070 con 0x7f088415c550 2024-09-19T08:46:44.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.030+0000 7f08837fe640 1 -- 172.21.15.98:0/4422683 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08780a9070 con 0x7f088415c550 2024-09-19T08:46:44.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.030+0000 7f08837fe640 1 -- 172.21.15.98:0/4422683 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08780ad880 con 0x7f088415c550 2024-09-19T08:46:44.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.030+0000 7f088adfd640 1 -- 172.21.15.98:0/4422683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f088415c550 msgr2=0x7f088406bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:44.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.030+0000 7f088adfd640 1 --2- 172.21.15.98:0/4422683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f088415c550 0x7f088406bc90 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f0878009870 tx=0x7f087801c560 comp rx=0 tx=0).stop 2024-09-19T08:46:44.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.030+0000 7f088adfd640 1 -- 172.21.15.98:0/4422683 shutdown_connections 2024-09-19T08:46:44.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.030+0000 7f088adfd640 1 --2- 172.21.15.98:0/4422683 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f088415c550 0x7f088406bc90 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:44.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.030+0000 7f088adfd640 1 -- 172.21.15.98:0/4422683 >> 172.21.15.98:0/4422683 conn(0x7f08840b2010 msgr2=0x7f08840b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:44.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.030+0000 7f088adfd640 1 -- 172.21.15.98:0/4422683 shutdown_connections 2024-09-19T08:46:44.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.030+0000 7f088adfd640 1 -- 172.21.15.98:0/4422683 wait complete. 2024-09-19T08:46:44.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.030+0000 7f088adfd640 1 Processor -- start 2024-09-19T08:46:44.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.030+0000 7f088adfd640 1 -- start start 2024-09-19T08:46:44.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.030+0000 7f088adfd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f088406c1d0 0x7f088414c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:44.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.030+0000 7f088adfd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08840cfad0 0x7f0884138210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:44.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.030+0000 7f088adfd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f088415c550 0x7f0884135910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:44.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.030+0000 7f088adfd640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0884073690 con 0x7f08840cfad0 2024-09-19T08:46:44.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.030+0000 7f088adfd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0884073510 con 0x7f088415c550 2024-09-19T08:46:44.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.030+0000 7f088adfd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f088416a240 con 0x7f088406c1d0 2024-09-19T08:46:44.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.030+0000 7f0883fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08840cfad0 0x7f0884138210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:44.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.030+0000 7f0883fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08840cfad0 0x7f0884138210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54332/0 (socket says 172.21.15.98:54332) 2024-09-19T08:46:44.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.030+0000 7f0883fff640 1 -- 172.21.15.98:0/874576338 learned_addr learned my addr 172.21.15.98:0/874576338 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:44.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.030+0000 7f088939a640 1 --2- 172.21.15.98:0/874576338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f088415c550 0x7f0884135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:44.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.030+0000 7f0888b99640 1 --2- 172.21.15.98:0/874576338 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f088406c1d0 0x7f088414c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:44.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.030+0000 7f0883fff640 1 -- 172.21.15.98:0/874576338 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f088406c1d0 msgr2=0x7f088414c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:44.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.030+0000 7f0883fff640 1 --2- 172.21.15.98:0/874576338 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f088406c1d0 0x7f088414c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:44.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.030+0000 7f0883fff640 1 -- 172.21.15.98:0/874576338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f088415c550 msgr2=0x7f0884135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:44.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.030+0000 7f0883fff640 1 --2- 172.21.15.98:0/874576338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f088415c550 0x7f0884135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:44.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.030+0000 7f0883fff640 1 -- 172.21.15.98:0/874576338 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f088414df90 con 0x7f08840cfad0 2024-09-19T08:46:44.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.030+0000 7f088939a640 1 --2- 172.21.15.98:0/874576338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f088415c550 0x7f0884135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:44.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.030+0000 7f0883fff640 1 --2- 172.21.15.98:0/874576338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08840cfad0 0x7f0884138210 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f087400c480 tx=0x7f087400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:44.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.034+0000 7f08817fa640 1 -- 172.21.15.98:0/874576338 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0874005480 con 0x7f08840cfad0 2024-09-19T08:46:44.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.034+0000 7f08817fa640 1 -- 172.21.15.98:0/874576338 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0874003600 con 0x7f08840cfad0 2024-09-19T08:46:44.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.034+0000 7f088adfd640 1 -- 172.21.15.98:0/874576338 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f088414cb10 con 0x7f08840cfad0 2024-09-19T08:46:44.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.034+0000 7f08817fa640 1 -- 172.21.15.98:0/874576338 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08740039f0 con 0x7f08840cfad0 2024-09-19T08:46:44.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.034+0000 7f088adfd640 1 -- 172.21.15.98:0/874576338 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f088414e510 con 0x7f08840cfad0 2024-09-19T08:46:44.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.034+0000 7f088adfd640 1 -- 172.21.15.98:0/874576338 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f088406bc90 con 0x7f08840cfad0 2024-09-19T08:46:44.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.034+0000 7f08817fa640 1 -- 172.21.15.98:0/874576338 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f0874003b90 con 0x7f08840cfad0 2024-09-19T08:46:44.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.034+0000 7f08817fa640 1 --2- 172.21.15.98:0/874576338 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f08700680a0 0x7f087006a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:44.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.034+0000 7f08817fa640 1 -- 172.21.15.98:0/874576338 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5737+0+0 (secure 0 0 0) 0x7f08740842d0 con 0x7f08840cfad0 2024-09-19T08:46:44.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.034+0000 7f0888b99640 1 --2- 172.21.15.98:0/874576338 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f08700680a0 0x7f087006a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:44.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.034+0000 7f0888b99640 1 --2- 172.21.15.98:0/874576338 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f08700680a0 0x7f087006a560 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f08780af000 tx=0x7f08780a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:44.038 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.038+0000 7f08817fa640 1 -- 172.21.15.98:0/874576338 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f0874056700 con 0x7f08840cfad0 2024-09-19T08:46:44.200 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.202+0000 7f088adfd640 1 -- 172.21.15.98:0/874576338 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f08840cf880 con 0x7f08840cfad0 2024-09-19T08:46:44.202 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.202+0000 7f08817fa640 1 -- 172.21.15.98:0/874576338 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) ==== 74+0+16783 (secure 0 0 0) 0x7f0874056000 con 0x7f08840cfad0 2024-09-19T08:46:44.202 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:46:44.202 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":158,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:46:41.262235+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:46:37.833554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-19T08:46:38.858268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:46:44.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.206+0000 7f088adfd640 1 -- 172.21.15.98:0/874576338 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f08700680a0 msgr2=0x7f087006a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:44.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.206+0000 7f088adfd640 1 --2- 172.21.15.98:0/874576338 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f08700680a0 0x7f087006a560 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f08780af000 tx=0x7f08780a7000 comp rx=0 tx=0).stop 2024-09-19T08:46:44.204 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.206+0000 7f088adfd640 1 -- 172.21.15.98:0/874576338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08840cfad0 msgr2=0x7f0884138210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:44.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.206+0000 7f088adfd640 1 --2- 172.21.15.98:0/874576338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08840cfad0 0x7f0884138210 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f087400c480 tx=0x7f087400c940 comp rx=0 tx=0).stop 2024-09-19T08:46:44.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.206+0000 7f088adfd640 1 -- 172.21.15.98:0/874576338 shutdown_connections 2024-09-19T08:46:44.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.206+0000 7f088adfd640 1 --2- 172.21.15.98:0/874576338 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f08700680a0 0x7f087006a560 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:44.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.206+0000 7f088adfd640 1 --2- 172.21.15.98:0/874576338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f088415c550 0x7f0884135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:44.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.206+0000 7f088adfd640 1 --2- 172.21.15.98:0/874576338 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f08840cfad0 0x7f0884138210 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:44.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.206+0000 7f088adfd640 1 --2- 172.21.15.98:0/874576338 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f088406c1d0 0x7f088414c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:44.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.206+0000 7f088adfd640 1 -- 172.21.15.98:0/874576338 >> 172.21.15.98:0/874576338 conn(0x7f08840b2010 msgr2=0x7f08840cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:44.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.206+0000 7f088adfd640 1 -- 172.21.15.98:0/874576338 shutdown_connections 2024-09-19T08:46:44.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.206+0000 7f088adfd640 1 -- 172.21.15.98:0/874576338 wait complete. 2024-09-19T08:46:44.221 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T08:46:44.221 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:46:44.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.342+0000 7f7768a9e640 1 Processor -- start 2024-09-19T08:46:44.344 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.342+0000 7f7768a9e640 1 -- start start 2024-09-19T08:46:44.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.346+0000 7f7768a9e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f776415c550 0x7f776406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:44.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.346+0000 7f7768a9e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f776406c1d0 0x7f77640cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:44.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.346+0000 7f7768a9e640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f776406d310 con 0x7f77640cf900 2024-09-19T08:46:44.345 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.346+0000 7f7768a9e640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f776406c610 con 0x7f776406c1d0 2024-09-19T08:46:44.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.346+0000 7f7768a9e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f776406d5f0 con 0x7f776415c550 2024-09-19T08:46:44.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.346+0000 7f7761d9b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f776406c1d0 0x7f77640cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:44.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.346+0000 7f7762d9d640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f77640cf900 0x7f77640ce1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:54984/0 (socket says 172.21.15.98:54984) 2024-09-19T08:46:44.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.346+0000 7f7762d9d640 1 -- 172.21.15.98:0/2953408751 learned_addr learned my addr 172.21.15.98:0/2953408751 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:44.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.346+0000 7f776259c640 1 --2- 172.21.15.98:0/2953408751 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f776415c550 0x7f776406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:44.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.346+0000 7f7761d9b640 1 -- 172.21.15.98:0/2953408751 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f776415c550 msgr2=0x7f776406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:44.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.346+0000 7f7761d9b640 1 --2- 172.21.15.98:0/2953408751 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f776415c550 0x7f776406bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:44.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.346+0000 7f7761d9b640 1 -- 172.21.15.98:0/2953408751 >> v1:172.21.15.98:6789/0 conn(0x7f77640cf900 legacy=0x7f77640ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:44.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.346+0000 7f776159a640 1 -- 172.21.15.98:0/2953408751 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3804250199 0 0) 0x7f776406d310 con 0x7f77640cf900 2024-09-19T08:46:44.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.346+0000 7f7761d9b640 1 -- 172.21.15.98:0/2953408751 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776406d8d0 con 0x7f776406c1d0 2024-09-19T08:46:44.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.350+0000 7f7761d9b640 1 --2- 172.21.15.98:0/2953408751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f776406c1d0 0x7f77640cf3c0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f774c0097c0 tx=0x7f774c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600e7085a7c281dd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:44.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.350+0000 7f776159a640 1 -- 172.21.15.98:0/2953408751 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c0b0070 con 0x7f776406c1d0 2024-09-19T08:46:44.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.350+0000 7f776159a640 1 -- 172.21.15.98:0/2953408751 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f774c0ab600 con 0x7f776406c1d0 2024-09-19T08:46:44.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.350+0000 7f776159a640 1 -- 172.21.15.98:0/2953408751 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c0ab9d0 con 0x7f776406c1d0 2024-09-19T08:46:44.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.350+0000 7f7768a9e640 1 -- 172.21.15.98:0/2953408751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f776406c1d0 msgr2=0x7f77640cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:44.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.350+0000 7f7768a9e640 1 --2- 172.21.15.98:0/2953408751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f776406c1d0 0x7f77640cf3c0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f774c0097c0 tx=0x7f774c01c4a0 comp rx=0 tx=0).stop 2024-09-19T08:46:44.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.350+0000 7f7768a9e640 1 -- 172.21.15.98:0/2953408751 shutdown_connections 2024-09-19T08:46:44.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.350+0000 7f7768a9e640 1 --2- 172.21.15.98:0/2953408751 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f776406c1d0 0x7f77640cf3c0 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:44.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.350+0000 7f7768a9e640 1 --2- 172.21.15.98:0/2953408751 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f776415c550 0x7f776406bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:44.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.350+0000 7f7768a9e640 1 -- 172.21.15.98:0/2953408751 >> 172.21.15.98:0/2953408751 conn(0x7f77640b2010 msgr2=0x7f77640b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:44.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.350+0000 7f7768a9e640 1 -- 172.21.15.98:0/2953408751 shutdown_connections 2024-09-19T08:46:44.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.350+0000 7f7768a9e640 1 -- 172.21.15.98:0/2953408751 wait complete. 2024-09-19T08:46:44.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.354+0000 7f7768a9e640 1 Processor -- start 2024-09-19T08:46:44.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.354+0000 7f7768a9e640 1 -- start start 2024-09-19T08:46:44.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.354+0000 7f7768a9e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f776406c1d0 0x7f77640b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:44.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.354+0000 7f7768a9e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77640cf900 0x7f77640b1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:44.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.354+0000 7f7768a9e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f776415c550 0x7f77640b1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:44.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.354+0000 7f7768a9e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7764169230 con 0x7f77640cf900 2024-09-19T08:46:44.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.354+0000 7f7768a9e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f776415cb40 con 0x7f776406c1d0 2024-09-19T08:46:44.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.354+0000 7f7768a9e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f77641693b0 con 0x7f776415c550 2024-09-19T08:46:44.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.354+0000 7f7761d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77640cf900 0x7f77640b1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:44.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.354+0000 7f7761d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77640cf900 0x7f77640b1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54350/0 (socket says 172.21.15.98:54350) 2024-09-19T08:46:44.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.354+0000 7f7761d9b640 1 -- 172.21.15.98:0/1826573092 learned_addr learned my addr 172.21.15.98:0/1826573092 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:44.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.354+0000 7f7762d9d640 1 --2- 172.21.15.98:0/1826573092 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f776415c550 0x7f77640b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:44.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.354+0000 7f776259c640 1 --2- 172.21.15.98:0/1826573092 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f776406c1d0 0x7f77640b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:44.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.354+0000 7f7761d9b640 1 -- 172.21.15.98:0/1826573092 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f776415c550 msgr2=0x7f77640b1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:44.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.354+0000 7f7761d9b640 1 --2- 172.21.15.98:0/1826573092 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f776415c550 0x7f77640b1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:44.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.354+0000 7f7761d9b640 1 -- 172.21.15.98:0/1826573092 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f776406c1d0 msgr2=0x7f77640b4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:44.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.354+0000 7f7761d9b640 1 --2- 172.21.15.98:0/1826573092 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f776406c1d0 0x7f77640b4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:44.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.354+0000 7f7761d9b640 1 -- 172.21.15.98:0/1826573092 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7764151920 con 0x7f77640cf900 2024-09-19T08:46:44.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.354+0000 7f7762d9d640 1 --2- 172.21.15.98:0/1826573092 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f776415c550 0x7f77640b1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:44.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.354+0000 7f7761d9b640 1 --2- 172.21.15.98:0/1826573092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77640cf900 0x7f77640b1000 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f774c0ae040 tx=0x7f774c01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:44.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.354+0000 7f775affd640 1 -- 172.21.15.98:0/1826573092 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c0bd070 con 0x7f77640cf900 2024-09-19T08:46:44.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.354+0000 7f775affd640 1 -- 172.21.15.98:0/1826573092 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f774c004030 con 0x7f77640cf900 2024-09-19T08:46:44.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.354+0000 7f775affd640 1 -- 172.21.15.98:0/1826573092 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c0b0040 con 0x7f77640cf900 2024-09-19T08:46:44.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.354+0000 7f7768a9e640 1 -- 172.21.15.98:0/1826573092 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7764151c00 con 0x7f77640cf900 2024-09-19T08:46:44.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.354+0000 7f7768a9e640 1 -- 172.21.15.98:0/1826573092 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77641521c0 con 0x7f77640cf900 2024-09-19T08:46:44.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.354+0000 7f7768a9e640 1 -- 172.21.15.98:0/1826573092 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7764152fe0 con 0x7f77640cf900 2024-09-19T08:46:44.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.358+0000 7f775affd640 1 -- 172.21.15.98:0/1826573092 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f774c003620 con 0x7f77640cf900 2024-09-19T08:46:44.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.358+0000 7f775affd640 1 --2- 172.21.15.98:0/1826573092 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f77440680a0 0x7f774406a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:44.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.358+0000 7f775affd640 1 -- 172.21.15.98:0/1826573092 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5737+0+0 (secure 0 0 0) 0x7f774c1208e0 con 0x7f77640cf900 2024-09-19T08:46:44.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.358+0000 7f776259c640 1 --2- 172.21.15.98:0/1826573092 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f77440680a0 0x7f774406a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:44.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.358+0000 7f776259c640 1 --2- 172.21.15.98:0/1826573092 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f77440680a0 0x7f774406a560 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f775400b930 tx=0x7f7754008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:44.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.362+0000 7f775affd640 1 -- 172.21.15.98:0/1826573092 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f7764152fe0 con 0x7f77640cf900 2024-09-19T08:46:44.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.506+0000 7f7768a9e640 1 -- 172.21.15.98:0/1826573092 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f77640cf3c0 con 0x7f77440680a0 2024-09-19T08:46:44.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.522+0000 7f775affd640 1 -- 172.21.15.98:0/1826573092 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240281 (secure 0 0 0) 0x7f77640cf3c0 con 0x7f77440680a0 2024-09-19T08:46:44.526 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:46:44.528 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:46:44.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.526+0000 7f7768a9e640 1 -- 172.21.15.98:0/1826573092 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f77440680a0 msgr2=0x7f774406a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:44.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.526+0000 7f7768a9e640 1 --2- 172.21.15.98:0/1826573092 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f77440680a0 0x7f774406a560 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f775400b930 tx=0x7f7754008040 comp rx=0 tx=0).stop 2024-09-19T08:46:44.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.526+0000 7f7768a9e640 1 -- 172.21.15.98:0/1826573092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77640cf900 msgr2=0x7f77640b1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:44.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.526+0000 7f7768a9e640 1 --2- 172.21.15.98:0/1826573092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77640cf900 0x7f77640b1000 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f774c0ae040 tx=0x7f774c01ace0 comp rx=0 tx=0).stop 2024-09-19T08:46:44.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.526+0000 7f7768a9e640 1 -- 172.21.15.98:0/1826573092 shutdown_connections 2024-09-19T08:46:44.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.526+0000 7f7768a9e640 1 --2- 172.21.15.98:0/1826573092 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f77440680a0 0x7f774406a560 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:44.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.526+0000 7f7768a9e640 1 --2- 172.21.15.98:0/1826573092 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f776415c550 0x7f77640b1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:44.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.526+0000 7f7768a9e640 1 --2- 172.21.15.98:0/1826573092 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f77640cf900 0x7f77640b1000 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:44.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.526+0000 7f7768a9e640 1 --2- 172.21.15.98:0/1826573092 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f776406c1d0 0x7f77640b4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:44.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.526+0000 7f7768a9e640 1 -- 172.21.15.98:0/1826573092 >> 172.21.15.98:0/1826573092 conn(0x7f77640b2010 msgr2=0x7f77641551c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:44.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.526+0000 7f7768a9e640 1 -- 172.21.15.98:0/1826573092 shutdown_connections 2024-09-19T08:46:44.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.526+0000 7f7768a9e640 1 -- 172.21.15.98:0/1826573092 wait complete. 2024-09-19T08:46:44.543 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-09-19T08:46:42.946765+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":4,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":269988,"kb_used_data":16276,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754704732,"statfs":{"total":773094113280,"available":772817645568,"internally_reserved":0,"allocated":16666624,"data_stored":8033351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.458801"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.492943+0000","last_change":"2024-09-19T08:46:38.492943+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.492943+0000","last_undegraded":"2024-09-19T08:46:38.492943+0000","last_fullsized":"2024-09-19T08:46:38.492943+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.491369+0000","last_change":"2024-09-19T08:46:38.491369+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.491369+0000","last_undegraded":"2024-09-19T08:46:38.491369+0000","last_fullsized":"2024-09-19T08:46:38.491369+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.489231+0000","last_change":"2024-09-19T08:46:38.489231+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.489231+0000","last_undegraded":"2024-09-19T08:46:38.489231+0000","last_fullsized":"2024-09-19T08:46:38.489231+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.496736+0000","last_change":"2024-09-19T08:46:38.496736+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.496736+0000","last_undegraded":"2024-09-19T08:46:38.496736+0000","last_fullsized":"2024-09-19T08:46:38.496736+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.491922+0000","last_change":"2024-09-19T08:46:38.491922+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.491922+0000","last_undegraded":"2024-09-19T08:46:38.491922+0000","last_fullsized":"2024-09-19T08:46:38.491922+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.495832+0000","last_change":"2024-09-19T08:46:38.495832+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.495832+0000","last_undegraded":"2024-09-19T08:46:38.495832+0000","last_fullsized":"2024-09-19T08:46:38.495832+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.498463+0000","last_change":"2024-09-19T08:46:38.498463+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.498463+0000","last_undegraded":"2024-09-19T08:46:38.498463+0000","last_fullsized":"2024-09-19T08:46:38.498463+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.498650+0000","last_change":"2024-09-19T08:46:38.498650+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.498650+0000","last_undegraded":"2024-09-19T08:46:38.498650+0000","last_fullsized":"2024-09-19T08:46:38.498650+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.491953+0000","last_change":"2024-09-19T08:46:38.491953+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.491953+0000","last_undegraded":"2024-09-19T08:46:38.491953+0000","last_fullsized":"2024-09-19T08:46:38.491953+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.491912+0000","last_change":"2024-09-19T08:46:38.491912+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.491912+0000","last_undegraded":"2024-09-19T08:46:38.491912+0000","last_fullsized":"2024-09-19T08:46:38.491912+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.489662+0000","last_change":"2024-09-19T08:46:38.489662+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.489662+0000","last_undegraded":"2024-09-19T08:46:38.489662+0000","last_fullsized":"2024-09-19T08:46:38.489662+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.496075+0000","last_change":"2024-09-19T08:46:38.496075+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.496075+0000","last_undegraded":"2024-09-19T08:46:38.496075+0000","last_fullsized":"2024-09-19T08:46:38.496075+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.495715+0000","last_change":"2024-09-19T08:46:38.495715+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.495715+0000","last_undegraded":"2024-09-19T08:46:38.495715+0000","last_fullsized":"2024-09-19T08:46:38.495715+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.495818+0000","last_change":"2024-09-19T08:46:38.495818+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.495818+0000","last_undegraded":"2024-09-19T08:46:38.495818+0000","last_fullsized":"2024-09-19T08:46:38.495818+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.497303+0000","last_change":"2024-09-19T08:46:38.497303+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.497303+0000","last_undegraded":"2024-09-19T08:46:38.497303+0000","last_fullsized":"2024-09-19T08:46:38.497303+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.491859+0000","last_change":"2024-09-19T08:46:38.491859+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.491859+0000","last_undegraded":"2024-09-19T08:46:38.491859+0000","last_fullsized":"2024-09-19T08:46:38.491859+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.496728+0000","last_change":"2024-09-19T08:46:38.496728+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.496728+0000","last_undegraded":"2024-09-19T08:46:38.496728+0000","last_fullsized":"2024-09-19T08:46:38.496728+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.496092+0000","last_change":"2024-09-19T08:46:38.496092+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.496092+0000","last_undegraded":"2024-09-19T08:46:38.496092+0000","last_fullsized":"2024-09-19T08:46:38.496092+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.498670+0000","last_change":"2024-09-19T08:46:38.498670+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.498670+0000","last_undegraded":"2024-09-19T08:46:38.498670+0000","last_fullsized":"2024-09-19T08:46:38.498670+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.489377+0000","last_change":"2024-09-19T08:46:38.489377+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.489377+0000","last_undegraded":"2024-09-19T08:46:38.489377+0000","last_fullsized":"2024-09-19T08:46:38.489377+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.492312+0000","last_change":"2024-09-19T08:46:38.492312+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.492312+0000","last_undegraded":"2024-09-19T08:46:38.492312+0000","last_fullsized":"2024-09-19T08:46:38.492312+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.489470+0000","last_change":"2024-09-19T08:46:38.489470+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.489470+0000","last_undegraded":"2024-09-19T08:46:38.489470+0000","last_fullsized":"2024-09-19T08:46:38.489470+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.491301+0000","last_change":"2024-09-19T08:46:38.491301+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.491301+0000","last_undegraded":"2024-09-19T08:46:38.491301+0000","last_fullsized":"2024-09-19T08:46:38.491301+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.497453+0000","last_change":"2024-09-19T08:46:38.497453+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.497453+0000","last_undegraded":"2024-09-19T08:46:38.497453+0000","last_fullsized":"2024-09-19T08:46:38.497453+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.494357+0000","last_change":"2024-09-19T08:46:38.494357+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.494357+0000","last_undegraded":"2024-09-19T08:46:38.494357+0000","last_fullsized":"2024-09-19T08:46:38.494357+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.493516+0000","last_change":"2024-09-19T08:46:38.493516+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.493516+0000","last_undegraded":"2024-09-19T08:46:38.493516+0000","last_fullsized":"2024-09-19T08:46:38.493516+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.496162+0000","last_change":"2024-09-19T08:46:38.496162+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.496162+0000","last_undegraded":"2024-09-19T08:46:38.496162+0000","last_fullsized":"2024-09-19T08:46:38.496162+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.491412+0000","last_change":"2024-09-19T08:46:38.491412+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.491412+0000","last_undegraded":"2024-09-19T08:46:38.491412+0000","last_fullsized":"2024-09-19T08:46:38.491412+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.491303+0000","last_change":"2024-09-19T08:46:38.491303+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.491303+0000","last_undegraded":"2024-09-19T08:46:38.491303+0000","last_fullsized":"2024-09-19T08:46:38.491303+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.498187+0000","last_change":"2024-09-19T08:46:38.498187+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.498187+0000","last_undegraded":"2024-09-19T08:46:38.498187+0000","last_fullsized":"2024-09-19T08:46:38.498187+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.489149+0000","last_change":"2024-09-19T08:46:38.489149+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.489149+0000","last_undegraded":"2024-09-19T08:46:38.489149+0000","last_fullsized":"2024-09-19T08:46:38.489149+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.496071+0000","last_change":"2024-09-19T08:46:38.496071+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.496071+0000","last_undegraded":"2024-09-19T08:46:38.496071+0000","last_fullsized":"2024-09-19T08:46:38.496071+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"71'128","reported_seq":451,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-19T08:46:38.499513+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:46:38.499513+0000","last_peered":"2024-09-19T08:46:38.499513+0000","last_clean":"2024-09-19T08:46:38.499513+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:46:38.499513+0000","last_undegraded":"2024-09-19T08:46:38.499513+0000","last_fullsized":"2024-09-19T08:46:38.499513+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509545,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33660,"kb_used_data":2468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338180,"statfs":{"total":96636764160,"available":96602296320,"internally_reserved":0,"allocated":2527232,"data_stored":1446128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574954,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33088,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338752,"statfs":{"total":96636764160,"available":96602882048,"internally_reserved":0,"allocated":1945600,"data_stored":861605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33080,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338760,"statfs":{"total":96636764160,"available":96602890240,"internally_reserved":0,"allocated":1933312,"data_stored":855760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509547,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33092,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338748,"statfs":{"total":96636764160,"available":96602877952,"internally_reserved":0,"allocated":1933312,"data_stored":855760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542249,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37228,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334612,"statfs":{"total":96636764160,"available":96598642688,"internally_reserved":0,"allocated":1921024,"data_stored":850605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509544,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33092,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338748,"statfs":{"total":96636764160,"available":96602877952,"internally_reserved":0,"allocated":1945600,"data_stored":861605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509546,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33080,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338760,"statfs":{"total":96636764160,"available":96602890240,"internally_reserved":0,"allocated":1933312,"data_stored":855760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509544,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33668,"kb_used_data":2468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338172,"statfs":{"total":96636764160,"available":96602288128,"internally_reserved":0,"allocated":2527232,"data_stored":1446128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:46:44.547 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:46:44.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.702+0000 7fcd83fb6640 1 Processor -- start 2024-09-19T08:46:44.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.702+0000 7fcd83fb6640 1 -- start start 2024-09-19T08:46:44.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.702+0000 7fcd83fb6640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcd7c0733b0 0x7fcd7c0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:44.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.702+0000 7fcd83fb6640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd7c06d310 con 0x7fcd7c15c550 2024-09-19T08:46:44.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.702+0000 7fcd83fb6640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd7c06d5f0 con 0x7fcd7c06c3a0 2024-09-19T08:46:44.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.702+0000 7fcd83fb6640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fcd7c06d8d0 con 0x7fcd7c0733b0 2024-09-19T08:46:44.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.702+0000 7fcd82553640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcd7c0733b0 0x7fcd7c0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:44.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.702+0000 7fcd81d52640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fcd7c15c550 0x7fcd7c06bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:55000/0 (socket says 172.21.15.98:55000) 2024-09-19T08:46:44.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.702+0000 7fcd81d52640 1 -- 172.21.15.98:0/3579340364 learned_addr learned my addr 172.21.15.98:0/3579340364 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:44.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.702+0000 7fcd82553640 1 -- 172.21.15.98:0/3579340364 >> v1:172.21.15.139:6789/0 conn(0x7fcd7c06c3a0 legacy=0x7fcd7c0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:44.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.702+0000 7fcd82553640 1 -- 172.21.15.98:0/3579340364 >> v1:172.21.15.98:6789/0 conn(0x7fcd7c15c550 legacy=0x7fcd7c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:44.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.702+0000 7fcd82553640 1 -- 172.21.15.98:0/3579340364 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7c06dbb0 con 0x7fcd7c0733b0 2024-09-19T08:46:44.702 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.702+0000 7fcd80d50640 1 -- 172.21.15.98:0/3579340364 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1741713221 0 0) 0x7fcd7c06d310 con 0x7fcd7c15c550 2024-09-19T08:46:44.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.706+0000 7fcd82553640 1 --2- 172.21.15.98:0/3579340364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd7c0733b0 0x7fcd7c0cfad0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fcd74009d90 tx=0x7fcd7401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=572a50d2d998b600 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:44.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.706+0000 7fcd80d50640 1 -- 172.21.15.98:0/3579340364 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd740b2070 con 0x7fcd7c0733b0 2024-09-19T08:46:44.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.706+0000 7fcd80d50640 1 -- 172.21.15.98:0/3579340364 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd740a57f0 con 0x7fcd7c0733b0 2024-09-19T08:46:44.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.706+0000 7fcd80d50640 1 -- 172.21.15.98:0/3579340364 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd740a5bc0 con 0x7fcd7c0733b0 2024-09-19T08:46:44.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.706+0000 7fcd83fb6640 1 -- 172.21.15.98:0/3579340364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd7c0733b0 msgr2=0x7fcd7c0cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:44.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.706+0000 7fcd83fb6640 1 --2- 172.21.15.98:0/3579340364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd7c0733b0 0x7fcd7c0cfad0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fcd74009d90 tx=0x7fcd7401c930 comp rx=0 tx=0).stop 2024-09-19T08:46:44.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.706+0000 7fcd83fb6640 1 -- 172.21.15.98:0/3579340364 shutdown_connections 2024-09-19T08:46:44.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.706+0000 7fcd83fb6640 1 --2- 172.21.15.98:0/3579340364 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd7c0733b0 0x7fcd7c0cfad0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:44.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.706+0000 7fcd83fb6640 1 -- 172.21.15.98:0/3579340364 >> 172.21.15.98:0/3579340364 conn(0x7fcd7c0b2010 msgr2=0x7fcd7c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:44.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.706+0000 7fcd83fb6640 1 -- 172.21.15.98:0/3579340364 shutdown_connections 2024-09-19T08:46:44.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.706+0000 7fcd83fb6640 1 -- 172.21.15.98:0/3579340364 wait complete. 2024-09-19T08:46:44.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.706+0000 7fcd83fb6640 1 Processor -- start 2024-09-19T08:46:44.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.706+0000 7fcd83fb6640 1 -- start start 2024-09-19T08:46:44.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.706+0000 7fcd83fb6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd7c06c3a0 0x7fcd7c1534c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:44.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.706+0000 7fcd83fb6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd7c0733b0 0x7fcd7c143a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:44.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.706+0000 7fcd83fb6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd7c15c550 0x7fcd7c1496b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:44.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.706+0000 7fcd83fb6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7c169dd0 con 0x7fcd7c0733b0 2024-09-19T08:46:44.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.706+0000 7fcd83fb6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7c169c50 con 0x7fcd7c06c3a0 2024-09-19T08:46:44.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.706+0000 7fcd83fb6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcd7c169f50 con 0x7fcd7c15c550 2024-09-19T08:46:44.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.706+0000 7fcd81551640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd7c0733b0 0x7fcd7c143a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:44.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.706+0000 7fcd81551640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd7c0733b0 0x7fcd7c143a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54374/0 (socket says 172.21.15.98:54374) 2024-09-19T08:46:44.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.706+0000 7fcd81551640 1 -- 172.21.15.98:0/2487807889 learned_addr learned my addr 172.21.15.98:0/2487807889 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:44.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.706+0000 7fcd81551640 1 -- 172.21.15.98:0/2487807889 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd7c15c550 msgr2=0x7fcd7c1496b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:44.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.706+0000 7fcd81551640 1 --2- 172.21.15.98:0/2487807889 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd7c15c550 0x7fcd7c1496b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:44.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.706+0000 7fcd81551640 1 -- 172.21.15.98:0/2487807889 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd7c06c3a0 msgr2=0x7fcd7c1534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:44.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.706+0000 7fcd81551640 1 --2- 172.21.15.98:0/2487807889 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd7c06c3a0 0x7fcd7c1534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:44.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.706+0000 7fcd81551640 1 -- 172.21.15.98:0/2487807889 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7c14a530 con 0x7fcd7c0733b0 2024-09-19T08:46:44.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.706+0000 7fcd81551640 1 --2- 172.21.15.98:0/2487807889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd7c0733b0 0x7fcd7c143a30 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00b3e0 tx=0x7fcd6c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:44.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.706+0000 7fcd727fc640 1 -- 172.21.15.98:0/2487807889 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c015020 con 0x7fcd7c0733b0 2024-09-19T08:46:44.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.706+0000 7fcd727fc640 1 -- 172.21.15.98:0/2487807889 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd6c011850 con 0x7fcd7c0733b0 2024-09-19T08:46:44.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.706+0000 7fcd83fb6640 1 -- 172.21.15.98:0/2487807889 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd7c14a810 con 0x7fcd7c0733b0 2024-09-19T08:46:44.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.706+0000 7fcd727fc640 1 -- 172.21.15.98:0/2487807889 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c011c40 con 0x7fcd7c0733b0 2024-09-19T08:46:44.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.706+0000 7fcd83fb6640 1 -- 172.21.15.98:0/2487807889 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd7c153a00 con 0x7fcd7c0733b0 2024-09-19T08:46:44.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.710+0000 7fcd83fb6640 1 -- 172.21.15.98:0/2487807889 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd7c14b1b0 con 0x7fcd7c0733b0 2024-09-19T08:46:44.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.710+0000 7fcd727fc640 1 -- 172.21.15.98:0/2487807889 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7fcd6c018020 con 0x7fcd7c0733b0 2024-09-19T08:46:44.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.710+0000 7fcd727fc640 1 --2- 172.21.15.98:0/2487807889 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcd600680a0 0x7fcd6006a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:44.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.710+0000 7fcd727fc640 1 -- 172.21.15.98:0/2487807889 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5737+0+0 (secure 0 0 0) 0x7fcd6c084600 con 0x7fcd7c0733b0 2024-09-19T08:46:44.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.710+0000 7fcd81d52640 1 --2- 172.21.15.98:0/2487807889 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcd600680a0 0x7fcd6006a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:44.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.710+0000 7fcd81d52640 1 --2- 172.21.15.98:0/2487807889 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcd600680a0 0x7fcd6006a560 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fcd780011f0 tx=0x7fcd78007930 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:44.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.714+0000 7fcd727fc640 1 -- 172.21.15.98:0/2487807889 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fcd7c14b1b0 con 0x7fcd7c0733b0 2024-09-19T08:46:44.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.858+0000 7fcd83fb6640 1 -- 172.21.15.98:0/2487807889 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcd7c06d310 con 0x7fcd600680a0 2024-09-19T08:46:44.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.874+0000 7fcd727fc640 1 -- 172.21.15.98:0/2487807889 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240281 (secure 0 0 0) 0x7fcd7c06d310 con 0x7fcd600680a0 2024-09-19T08:46:44.877 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:46:44.878 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:46:44.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.878+0000 7fcd83fb6640 1 -- 172.21.15.98:0/2487807889 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcd600680a0 msgr2=0x7fcd6006a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:44.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.878+0000 7fcd83fb6640 1 --2- 172.21.15.98:0/2487807889 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcd600680a0 0x7fcd6006a560 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fcd780011f0 tx=0x7fcd78007930 comp rx=0 tx=0).stop 2024-09-19T08:46:44.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.878+0000 7fcd83fb6640 1 -- 172.21.15.98:0/2487807889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd7c0733b0 msgr2=0x7fcd7c143a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:44.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.878+0000 7fcd83fb6640 1 --2- 172.21.15.98:0/2487807889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd7c0733b0 0x7fcd7c143a30 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00b3e0 tx=0x7fcd6c00b8a0 comp rx=0 tx=0).stop 2024-09-19T08:46:44.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.878+0000 7fcd83fb6640 1 -- 172.21.15.98:0/2487807889 shutdown_connections 2024-09-19T08:46:44.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.878+0000 7fcd83fb6640 1 --2- 172.21.15.98:0/2487807889 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcd600680a0 0x7fcd6006a560 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:44.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.878+0000 7fcd83fb6640 1 --2- 172.21.15.98:0/2487807889 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcd7c15c550 0x7fcd7c1496b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:44.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.878+0000 7fcd83fb6640 1 --2- 172.21.15.98:0/2487807889 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcd7c0733b0 0x7fcd7c143a30 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:44.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.878+0000 7fcd83fb6640 1 --2- 172.21.15.98:0/2487807889 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd7c06c3a0 0x7fcd7c1534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:44.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.878+0000 7fcd83fb6640 1 -- 172.21.15.98:0/2487807889 >> 172.21.15.98:0/2487807889 conn(0x7fcd7c0b2010 msgr2=0x7fcd7c15b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:44.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.878+0000 7fcd83fb6640 1 -- 172.21.15.98:0/2487807889 shutdown_connections 2024-09-19T08:46:44.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:44.878+0000 7fcd83fb6640 1 -- 172.21.15.98:0/2487807889 wait complete. 2024-09-19T08:46:44.893 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-09-19T08:46:42.946765+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":4,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":269988,"kb_used_data":16276,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754704732,"statfs":{"total":773094113280,"available":772817645568,"internally_reserved":0,"allocated":16666624,"data_stored":8033351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":259575344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.458801"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.492943+0000","last_change":"2024-09-19T08:46:38.492943+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.492943+0000","last_undegraded":"2024-09-19T08:46:38.492943+0000","last_fullsized":"2024-09-19T08:46:38.492943+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.491369+0000","last_change":"2024-09-19T08:46:38.491369+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.491369+0000","last_undegraded":"2024-09-19T08:46:38.491369+0000","last_fullsized":"2024-09-19T08:46:38.491369+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.489231+0000","last_change":"2024-09-19T08:46:38.489231+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.489231+0000","last_undegraded":"2024-09-19T08:46:38.489231+0000","last_fullsized":"2024-09-19T08:46:38.489231+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.496736+0000","last_change":"2024-09-19T08:46:38.496736+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.496736+0000","last_undegraded":"2024-09-19T08:46:38.496736+0000","last_fullsized":"2024-09-19T08:46:38.496736+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.491922+0000","last_change":"2024-09-19T08:46:38.491922+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.491922+0000","last_undegraded":"2024-09-19T08:46:38.491922+0000","last_fullsized":"2024-09-19T08:46:38.491922+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.495832+0000","last_change":"2024-09-19T08:46:38.495832+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.495832+0000","last_undegraded":"2024-09-19T08:46:38.495832+0000","last_fullsized":"2024-09-19T08:46:38.495832+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.498463+0000","last_change":"2024-09-19T08:46:38.498463+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.498463+0000","last_undegraded":"2024-09-19T08:46:38.498463+0000","last_fullsized":"2024-09-19T08:46:38.498463+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.498650+0000","last_change":"2024-09-19T08:46:38.498650+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.498650+0000","last_undegraded":"2024-09-19T08:46:38.498650+0000","last_fullsized":"2024-09-19T08:46:38.498650+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.491953+0000","last_change":"2024-09-19T08:46:38.491953+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.491953+0000","last_undegraded":"2024-09-19T08:46:38.491953+0000","last_fullsized":"2024-09-19T08:46:38.491953+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.491912+0000","last_change":"2024-09-19T08:46:38.491912+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.491912+0000","last_undegraded":"2024-09-19T08:46:38.491912+0000","last_fullsized":"2024-09-19T08:46:38.491912+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.489662+0000","last_change":"2024-09-19T08:46:38.489662+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.489662+0000","last_undegraded":"2024-09-19T08:46:38.489662+0000","last_fullsized":"2024-09-19T08:46:38.489662+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.496075+0000","last_change":"2024-09-19T08:46:38.496075+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.496075+0000","last_undegraded":"2024-09-19T08:46:38.496075+0000","last_fullsized":"2024-09-19T08:46:38.496075+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.495715+0000","last_change":"2024-09-19T08:46:38.495715+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.495715+0000","last_undegraded":"2024-09-19T08:46:38.495715+0000","last_fullsized":"2024-09-19T08:46:38.495715+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.495818+0000","last_change":"2024-09-19T08:46:38.495818+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.495818+0000","last_undegraded":"2024-09-19T08:46:38.495818+0000","last_fullsized":"2024-09-19T08:46:38.495818+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.497303+0000","last_change":"2024-09-19T08:46:38.497303+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.497303+0000","last_undegraded":"2024-09-19T08:46:38.497303+0000","last_fullsized":"2024-09-19T08:46:38.497303+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.491859+0000","last_change":"2024-09-19T08:46:38.491859+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.491859+0000","last_undegraded":"2024-09-19T08:46:38.491859+0000","last_fullsized":"2024-09-19T08:46:38.491859+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.496728+0000","last_change":"2024-09-19T08:46:38.496728+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.496728+0000","last_undegraded":"2024-09-19T08:46:38.496728+0000","last_fullsized":"2024-09-19T08:46:38.496728+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.496092+0000","last_change":"2024-09-19T08:46:38.496092+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.496092+0000","last_undegraded":"2024-09-19T08:46:38.496092+0000","last_fullsized":"2024-09-19T08:46:38.496092+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.498670+0000","last_change":"2024-09-19T08:46:38.498670+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.498670+0000","last_undegraded":"2024-09-19T08:46:38.498670+0000","last_fullsized":"2024-09-19T08:46:38.498670+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.489377+0000","last_change":"2024-09-19T08:46:38.489377+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.489377+0000","last_undegraded":"2024-09-19T08:46:38.489377+0000","last_fullsized":"2024-09-19T08:46:38.489377+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.492312+0000","last_change":"2024-09-19T08:46:38.492312+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.492312+0000","last_undegraded":"2024-09-19T08:46:38.492312+0000","last_fullsized":"2024-09-19T08:46:38.492312+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.489470+0000","last_change":"2024-09-19T08:46:38.489470+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.489470+0000","last_undegraded":"2024-09-19T08:46:38.489470+0000","last_fullsized":"2024-09-19T08:46:38.489470+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.491301+0000","last_change":"2024-09-19T08:46:38.491301+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.491301+0000","last_undegraded":"2024-09-19T08:46:38.491301+0000","last_fullsized":"2024-09-19T08:46:38.491301+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.497453+0000","last_change":"2024-09-19T08:46:38.497453+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.497453+0000","last_undegraded":"2024-09-19T08:46:38.497453+0000","last_fullsized":"2024-09-19T08:46:38.497453+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.494357+0000","last_change":"2024-09-19T08:46:38.494357+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.494357+0000","last_undegraded":"2024-09-19T08:46:38.494357+0000","last_fullsized":"2024-09-19T08:46:38.494357+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.493516+0000","last_change":"2024-09-19T08:46:38.493516+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.493516+0000","last_undegraded":"2024-09-19T08:46:38.493516+0000","last_fullsized":"2024-09-19T08:46:38.493516+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.496162+0000","last_change":"2024-09-19T08:46:38.496162+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.496162+0000","last_undegraded":"2024-09-19T08:46:38.496162+0000","last_fullsized":"2024-09-19T08:46:38.496162+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.491412+0000","last_change":"2024-09-19T08:46:38.491412+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.491412+0000","last_undegraded":"2024-09-19T08:46:38.491412+0000","last_fullsized":"2024-09-19T08:46:38.491412+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.491303+0000","last_change":"2024-09-19T08:46:38.491303+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.491303+0000","last_undegraded":"2024-09-19T08:46:38.491303+0000","last_fullsized":"2024-09-19T08:46:38.491303+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.498187+0000","last_change":"2024-09-19T08:46:38.498187+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.498187+0000","last_undegraded":"2024-09-19T08:46:38.498187+0000","last_fullsized":"2024-09-19T08:46:38.498187+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.489149+0000","last_change":"2024-09-19T08:46:38.489149+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.489149+0000","last_undegraded":"2024-09-19T08:46:38.489149+0000","last_fullsized":"2024-09-19T08:46:38.489149+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T08:46:39.484150+0000","last_change":"2024-09-19T08:46:39.484150+0000","last_active":"2024-09-19T08:46:39.484150+0000","last_peered":"2024-09-19T08:46:39.484150+0000","last_clean":"2024-09-19T08:46:39.484150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:39.484150+0000","last_undegraded":"2024-09-19T08:46:39.484150+0000","last_fullsized":"2024-09-19T08:46:39.484150+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":1,"reported_epoch":155,"state":"creating+peering","last_fresh":"2024-09-19T08:46:38.496071+0000","last_change":"2024-09-19T08:46:38.496071+0000","last_active":"2024-09-19T08:46:38.474849+0000","last_peered":"2024-09-19T08:46:38.474849+0000","last_clean":"2024-09-19T08:46:38.474849+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:46:38.496071+0000","last_undegraded":"2024-09-19T08:46:38.496071+0000","last_fullsized":"2024-09-19T08:46:38.496071+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"71'128","reported_seq":451,"reported_epoch":155,"state":"active+clean","last_fresh":"2024-09-19T08:46:38.499513+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:46:38.499513+0000","last_peered":"2024-09-19T08:46:38.499513+0000","last_clean":"2024-09-19T08:46:38.499513+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:46:38.499513+0000","last_undegraded":"2024-09-19T08:46:38.499513+0000","last_fullsized":"2024-09-19T08:46:38.499513+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509545,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33660,"kb_used_data":2468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338180,"statfs":{"total":96636764160,"available":96602296320,"internally_reserved":0,"allocated":2527232,"data_stored":1446128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574954,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33088,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338752,"statfs":{"total":96636764160,"available":96602882048,"internally_reserved":0,"allocated":1945600,"data_stored":861605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33080,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338760,"statfs":{"total":96636764160,"available":96602890240,"internally_reserved":0,"allocated":1933312,"data_stored":855760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509547,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33092,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338748,"statfs":{"total":96636764160,"available":96602877952,"internally_reserved":0,"allocated":1933312,"data_stored":855760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542249,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37228,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334612,"statfs":{"total":96636764160,"available":96598642688,"internally_reserved":0,"allocated":1921024,"data_stored":850605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509544,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33092,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338748,"statfs":{"total":96636764160,"available":96602877952,"internally_reserved":0,"allocated":1945600,"data_stored":861605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509546,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33080,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338760,"statfs":{"total":96636764160,"available":96602890240,"internally_reserved":0,"allocated":1933312,"data_stored":855760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509544,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33668,"kb_used_data":2468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338172,"statfs":{"total":96636764160,"available":96602288128,"internally_reserved":0,"allocated":2527232,"data_stored":1446128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:46:44.897 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:46:45.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.054+0000 7f18c3757640 1 Processor -- start 2024-09-19T08:46:45.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.054+0000 7f18c3757640 1 -- start start 2024-09-19T08:46:45.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.058+0000 7f18c3757640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f18bc158fd0 0x7f18bc0b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:45.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.058+0000 7f18c3757640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18bc0ccc60 con 0x7f18bc06c3a0 2024-09-19T08:46:45.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.058+0000 7f18c3757640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18bc0ccf40 con 0x7f18bc0b4360 2024-09-19T08:46:45.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.058+0000 7f18c3757640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f18bc1593b0 con 0x7f18bc158fd0 2024-09-19T08:46:45.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.058+0000 7f18c1cf4640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f18bc06c3a0 0x7f18bc0ce840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:55016/0 (socket says 172.21.15.98:55016) 2024-09-19T08:46:45.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.058+0000 7f18c14f3640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f18bc158fd0 0x7f18bc0b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:45.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.058+0000 7f18c1cf4640 1 -- 172.21.15.98:0/876010384 learned_addr learned my addr 172.21.15.98:0/876010384 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:45.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.058+0000 7f18b3fff640 1 -- 172.21.15.98:0/876010384 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1743574592 0 0) 0x7f18bc0ccc60 con 0x7f18bc06c3a0 2024-09-19T08:46:45.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.058+0000 7f18b3fff640 1 -- 172.21.15.98:0/876010384 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f18a8003530 con 0x7f18bc06c3a0 2024-09-19T08:46:45.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.058+0000 7f18c14f3640 1 -- 172.21.15.98:0/876010384 >> v1:172.21.15.139:6789/0 conn(0x7f18bc0b4360 legacy=0x7f18bc06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:45.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.058+0000 7f18c14f3640 1 -- 172.21.15.98:0/876010384 >> v1:172.21.15.98:6789/0 conn(0x7f18bc06c3a0 legacy=0x7f18bc0ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:45.060 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.058+0000 7f18c14f3640 1 -- 172.21.15.98:0/876010384 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18bc0cd220 con 0x7f18bc158fd0 2024-09-19T08:46:45.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.058+0000 7f18c14f3640 1 --2- 172.21.15.98:0/876010384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18bc158fd0 0x7f18bc0b3e20 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f18b80097c0 tx=0x7f18b801e5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f936fa692c5fcb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:45.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.058+0000 7f18b3fff640 1 -- 172.21.15.98:0/876010384 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b80aa620 con 0x7f18bc158fd0 2024-09-19T08:46:45.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.062+0000 7f18b3fff640 1 -- 172.21.15.98:0/876010384 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18b80aa7c0 con 0x7f18bc158fd0 2024-09-19T08:46:45.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.062+0000 7f18b3fff640 1 -- 172.21.15.98:0/876010384 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b80aab90 con 0x7f18bc158fd0 2024-09-19T08:46:45.068 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.062+0000 7f18c3757640 1 -- 172.21.15.98:0/876010384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18bc158fd0 msgr2=0x7f18bc0b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:45.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.062+0000 7f18c3757640 1 --2- 172.21.15.98:0/876010384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18bc158fd0 0x7f18bc0b3e20 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f18b80097c0 tx=0x7f18b801e5f0 comp rx=0 tx=0).stop 2024-09-19T08:46:45.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.062+0000 7f18c3757640 1 -- 172.21.15.98:0/876010384 shutdown_connections 2024-09-19T08:46:45.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.062+0000 7f18c3757640 1 --2- 172.21.15.98:0/876010384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18bc158fd0 0x7f18bc0b3e20 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:45.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.062+0000 7f18c3757640 1 -- 172.21.15.98:0/876010384 >> 172.21.15.98:0/876010384 conn(0x7f18bc0b2010 msgr2=0x7f18bc0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:45.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.062+0000 7f18c3757640 1 -- 172.21.15.98:0/876010384 shutdown_connections 2024-09-19T08:46:45.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.062+0000 7f18c3757640 1 -- 172.21.15.98:0/876010384 wait complete. 2024-09-19T08:46:45.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.062+0000 7f18c3757640 1 Processor -- start 2024-09-19T08:46:45.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.062+0000 7f18c3757640 1 -- start start 2024-09-19T08:46:45.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.062+0000 7f18c3757640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18bc06c3a0 0x7f18bc150200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:45.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.062+0000 7f18c3757640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18bc0b4360 0x7f18bc14f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:45.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.062+0000 7f18c3757640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18bc158fd0 0x7f18bc0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:45.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.062+0000 7f18c3757640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc0cf8e0 con 0x7f18bc158fd0 2024-09-19T08:46:45.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.062+0000 7f18c3757640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc0cf760 con 0x7f18bc0b4360 2024-09-19T08:46:45.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.062+0000 7f18c3757640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f18bc0cfa60 con 0x7f18bc06c3a0 2024-09-19T08:46:45.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.062+0000 7f18c1cf4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18bc158fd0 0x7f18bc0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:45.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.062+0000 7f18c1cf4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18bc158fd0 0x7f18bc0ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54392/0 (socket says 172.21.15.98:54392) 2024-09-19T08:46:45.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.062+0000 7f18c1cf4640 1 -- 172.21.15.98:0/2441380178 learned_addr learned my addr 172.21.15.98:0/2441380178 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:45.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.062+0000 7f18c0cf2640 1 --2- 172.21.15.98:0/2441380178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18bc0b4360 0x7f18bc14f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:45.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.062+0000 7f18c14f3640 1 --2- 172.21.15.98:0/2441380178 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18bc06c3a0 0x7f18bc150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:45.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.062+0000 7f18c1cf4640 1 -- 172.21.15.98:0/2441380178 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18bc06c3a0 msgr2=0x7f18bc150200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:45.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.062+0000 7f18c1cf4640 1 --2- 172.21.15.98:0/2441380178 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18bc06c3a0 0x7f18bc150200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:45.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.062+0000 7f18c1cf4640 1 -- 172.21.15.98:0/2441380178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18bc0b4360 msgr2=0x7f18bc14f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:45.072 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.062+0000 7f18c1cf4640 1 --2- 172.21.15.98:0/2441380178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18bc0b4360 0x7f18bc14f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:45.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.062+0000 7f18c1cf4640 1 -- 172.21.15.98:0/2441380178 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18bc152120 con 0x7f18bc158fd0 2024-09-19T08:46:45.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.062+0000 7f18c0cf2640 1 --2- 172.21.15.98:0/2441380178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18bc0b4360 0x7f18bc14f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:45.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.062+0000 7f18c1cf4640 1 --2- 172.21.15.98:0/2441380178 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18bc158fd0 0x7f18bc0ae8b0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f18b400cbb0 tx=0x7f18b400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:45.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.062+0000 7f18b1ffb640 1 -- 172.21.15.98:0/2441380178 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b4017070 con 0x7f18bc158fd0 2024-09-19T08:46:45.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.062+0000 7f18b1ffb640 1 -- 172.21.15.98:0/2441380178 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18b400ad40 con 0x7f18bc158fd0 2024-09-19T08:46:45.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.062+0000 7f18c3757640 1 -- 172.21.15.98:0/2441380178 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18bc15f670 con 0x7f18bc158fd0 2024-09-19T08:46:45.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.062+0000 7f18b1ffb640 1 -- 172.21.15.98:0/2441380178 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b4012600 con 0x7f18bc158fd0 2024-09-19T08:46:45.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.062+0000 7f18c3757640 1 -- 172.21.15.98:0/2441380178 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18bc15fc30 con 0x7f18bc158fd0 2024-09-19T08:46:45.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.066+0000 7f18c3757640 1 -- 172.21.15.98:0/2441380178 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18bc0b1a00 con 0x7f18bc158fd0 2024-09-19T08:46:45.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.066+0000 7f18b1ffb640 1 -- 172.21.15.98:0/2441380178 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f18b401a070 con 0x7f18bc158fd0 2024-09-19T08:46:45.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.066+0000 7f18b1ffb640 1 --2- 172.21.15.98:0/2441380178 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f18a00680a0 0x7f18a006a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:45.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.066+0000 7f18b1ffb640 1 -- 172.21.15.98:0/2441380178 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5737+0+0 (secure 0 0 0) 0x7f18b4086c20 con 0x7f18bc158fd0 2024-09-19T08:46:45.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.066+0000 7f18c14f3640 1 --2- 172.21.15.98:0/2441380178 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f18a00680a0 0x7f18a006a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:45.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.066+0000 7f18c14f3640 1 --2- 172.21.15.98:0/2441380178 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f18a00680a0 0x7f18a006a560 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f18b80b1000 tx=0x7f18b80b4040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:45.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.070+0000 7f18b1ffb640 1 -- 172.21.15.98:0/2441380178 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f18b40560d0 con 0x7f18bc158fd0 2024-09-19T08:46:45.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.214+0000 7f18c3757640 1 -- 172.21.15.98:0/2441380178 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f18bc0757a0 con 0x7f18a00680a0 2024-09-19T08:46:45.232 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.234+0000 7f18b1ffb640 1 -- 172.21.15.98:0/2441380178 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252484 (secure 0 0 0) 0x7f18bc0757a0 con 0x7f18a00680a0 2024-09-19T08:46:45.234 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:46:45.236 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:46:45.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.238+0000 7f18c3757640 1 -- 172.21.15.98:0/2441380178 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f18a00680a0 msgr2=0x7f18a006a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:45.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.238+0000 7f18c3757640 1 --2- 172.21.15.98:0/2441380178 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f18a00680a0 0x7f18a006a560 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f18b80b1000 tx=0x7f18b80b4040 comp rx=0 tx=0).stop 2024-09-19T08:46:45.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.238+0000 7f18c3757640 1 -- 172.21.15.98:0/2441380178 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18bc158fd0 msgr2=0x7f18bc0ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:45.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.238+0000 7f18c3757640 1 --2- 172.21.15.98:0/2441380178 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18bc158fd0 0x7f18bc0ae8b0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f18b400cbb0 tx=0x7f18b400a430 comp rx=0 tx=0).stop 2024-09-19T08:46:45.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.238+0000 7f18c3757640 1 -- 172.21.15.98:0/2441380178 shutdown_connections 2024-09-19T08:46:45.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.238+0000 7f18c3757640 1 --2- 172.21.15.98:0/2441380178 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f18a00680a0 0x7f18a006a560 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:45.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.238+0000 7f18c3757640 1 --2- 172.21.15.98:0/2441380178 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18bc158fd0 0x7f18bc0ae8b0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:45.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.238+0000 7f18c3757640 1 --2- 172.21.15.98:0/2441380178 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18bc0b4360 0x7f18bc14f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:45.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.238+0000 7f18c3757640 1 --2- 172.21.15.98:0/2441380178 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18bc06c3a0 0x7f18bc150200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:45.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.238+0000 7f18c3757640 1 -- 172.21.15.98:0/2441380178 >> 172.21.15.98:0/2441380178 conn(0x7f18bc0b2010 msgr2=0x7f18bc06bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:45.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.238+0000 7f18c3757640 1 -- 172.21.15.98:0/2441380178 shutdown_connections 2024-09-19T08:46:45.238 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.238+0000 7f18c3757640 1 -- 172.21.15.98:0/2441380178 wait complete. 2024-09-19T08:46:45.252 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":202,"stamp":"2024-09-19T08:46:44.947364+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":159,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":274588,"kb_used_data":16712,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754700132,"statfs":{"total":773094113280,"available":772812935168,"internally_reserved":0,"allocated":17113088,"data_stored":8233868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001975"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.300021+0000","last_change":"2024-09-19T08:46:40.322807+0000","last_active":"2024-09-19T08:46:41.300021+0000","last_peered":"2024-09-19T08:46:41.300021+0000","last_clean":"2024-09-19T08:46:41.300021+0000","last_became_active":"2024-09-19T08:46:40.322530+0000","last_became_peered":"2024-09-19T08:46:40.322530+0000","last_unstale":"2024-09-19T08:46:41.300021+0000","last_undegraded":"2024-09-19T08:46:41.300021+0000","last_fullsized":"2024-09-19T08:46:41.300021+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:14:04.162980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.275876+0000","last_change":"2024-09-19T08:46:40.308583+0000","last_active":"2024-09-19T08:46:41.275876+0000","last_peered":"2024-09-19T08:46:41.275876+0000","last_clean":"2024-09-19T08:46:41.275876+0000","last_became_active":"2024-09-19T08:46:40.307894+0000","last_became_peered":"2024-09-19T08:46:40.307894+0000","last_unstale":"2024-09-19T08:46:41.275876+0000","last_undegraded":"2024-09-19T08:46:41.275876+0000","last_fullsized":"2024-09-19T08:46:41.275876+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:48.398635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.294222+0000","last_change":"2024-09-19T08:46:40.305267+0000","last_active":"2024-09-19T08:46:41.294222+0000","last_peered":"2024-09-19T08:46:41.294222+0000","last_clean":"2024-09-19T08:46:41.294222+0000","last_became_active":"2024-09-19T08:46:40.304553+0000","last_became_peered":"2024-09-19T08:46:40.304553+0000","last_unstale":"2024-09-19T08:46:41.294222+0000","last_undegraded":"2024-09-19T08:46:41.294222+0000","last_fullsized":"2024-09-19T08:46:41.294222+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:49.743970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.275107+0000","last_change":"2024-09-19T08:46:40.307985+0000","last_active":"2024-09-19T08:46:41.275107+0000","last_peered":"2024-09-19T08:46:41.275107+0000","last_clean":"2024-09-19T08:46:41.275107+0000","last_became_active":"2024-09-19T08:46:40.307321+0000","last_became_peered":"2024-09-19T08:46:40.307321+0000","last_unstale":"2024-09-19T08:46:41.275107+0000","last_undegraded":"2024-09-19T08:46:41.275107+0000","last_fullsized":"2024-09-19T08:46:41.275107+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:37:44.052309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271115+0000","last_change":"2024-09-19T08:46:40.313691+0000","last_active":"2024-09-19T08:46:41.271115+0000","last_peered":"2024-09-19T08:46:41.271115+0000","last_clean":"2024-09-19T08:46:41.271115+0000","last_became_active":"2024-09-19T08:46:40.313152+0000","last_became_peered":"2024-09-19T08:46:40.313152+0000","last_unstale":"2024-09-19T08:46:41.271115+0000","last_undegraded":"2024-09-19T08:46:41.271115+0000","last_fullsized":"2024-09-19T08:46:41.271115+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:17:30.627741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.293469+0000","last_change":"2024-09-19T08:46:40.310239+0000","last_active":"2024-09-19T08:46:41.293469+0000","last_peered":"2024-09-19T08:46:41.293469+0000","last_clean":"2024-09-19T08:46:41.293469+0000","last_became_active":"2024-09-19T08:46:40.309610+0000","last_became_peered":"2024-09-19T08:46:40.309610+0000","last_unstale":"2024-09-19T08:46:41.293469+0000","last_undegraded":"2024-09-19T08:46:41.293469+0000","last_fullsized":"2024-09-19T08:46:41.293469+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:10.797977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.297686+0000","last_change":"2024-09-19T08:46:40.300397+0000","last_active":"2024-09-19T08:46:41.297686+0000","last_peered":"2024-09-19T08:46:41.297686+0000","last_clean":"2024-09-19T08:46:41.297686+0000","last_became_active":"2024-09-19T08:46:40.299734+0000","last_became_peered":"2024-09-19T08:46:40.299734+0000","last_unstale":"2024-09-19T08:46:41.297686+0000","last_undegraded":"2024-09-19T08:46:41.297686+0000","last_fullsized":"2024-09-19T08:46:41.297686+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:20.271146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295020+0000","last_change":"2024-09-19T08:46:40.282539+0000","last_active":"2024-09-19T08:46:41.295020+0000","last_peered":"2024-09-19T08:46:41.295020+0000","last_clean":"2024-09-19T08:46:41.295020+0000","last_became_active":"2024-09-19T08:46:40.281719+0000","last_became_peered":"2024-09-19T08:46:40.281719+0000","last_unstale":"2024-09-19T08:46:41.295020+0000","last_undegraded":"2024-09-19T08:46:41.295020+0000","last_fullsized":"2024-09-19T08:46:41.295020+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:54:49.652718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.302625+0000","last_change":"2024-09-19T08:46:40.313125+0000","last_active":"2024-09-19T08:46:41.302625+0000","last_peered":"2024-09-19T08:46:41.302625+0000","last_clean":"2024-09-19T08:46:41.302625+0000","last_became_active":"2024-09-19T08:46:40.312593+0000","last_became_peered":"2024-09-19T08:46:40.312593+0000","last_unstale":"2024-09-19T08:46:41.302625+0000","last_undegraded":"2024-09-19T08:46:41.302625+0000","last_fullsized":"2024-09-19T08:46:41.302625+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:18.762035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295826+0000","last_change":"2024-09-19T08:46:40.299010+0000","last_active":"2024-09-19T08:46:41.295826+0000","last_peered":"2024-09-19T08:46:41.295826+0000","last_clean":"2024-09-19T08:46:41.295826+0000","last_became_active":"2024-09-19T08:46:40.298263+0000","last_became_peered":"2024-09-19T08:46:40.298263+0000","last_unstale":"2024-09-19T08:46:41.295826+0000","last_undegraded":"2024-09-19T08:46:41.295826+0000","last_fullsized":"2024-09-19T08:46:41.295826+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:29:29.482503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.298285+0000","last_change":"2024-09-19T08:46:40.312068+0000","last_active":"2024-09-19T08:46:41.298285+0000","last_peered":"2024-09-19T08:46:41.298285+0000","last_clean":"2024-09-19T08:46:41.298285+0000","last_became_active":"2024-09-19T08:46:40.311698+0000","last_became_peered":"2024-09-19T08:46:40.311698+0000","last_unstale":"2024-09-19T08:46:41.298285+0000","last_undegraded":"2024-09-19T08:46:41.298285+0000","last_fullsized":"2024-09-19T08:46:41.298285+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:37:24.320667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.275730+0000","last_change":"2024-09-19T08:46:40.318436+0000","last_active":"2024-09-19T08:46:41.275730+0000","last_peered":"2024-09-19T08:46:41.275730+0000","last_clean":"2024-09-19T08:46:41.275730+0000","last_became_active":"2024-09-19T08:46:40.318012+0000","last_became_peered":"2024-09-19T08:46:40.318012+0000","last_unstale":"2024-09-19T08:46:41.275730+0000","last_undegraded":"2024-09-19T08:46:41.275730+0000","last_fullsized":"2024-09-19T08:46:41.275730+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:08.849222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.274801+0000","last_change":"2024-09-19T08:46:40.291508+0000","last_active":"2024-09-19T08:46:41.274801+0000","last_peered":"2024-09-19T08:46:41.274801+0000","last_clean":"2024-09-19T08:46:41.274801+0000","last_became_active":"2024-09-19T08:46:40.291252+0000","last_became_peered":"2024-09-19T08:46:40.291252+0000","last_unstale":"2024-09-19T08:46:41.274801+0000","last_undegraded":"2024-09-19T08:46:41.274801+0000","last_fullsized":"2024-09-19T08:46:41.274801+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:42.984066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.302365+0000","last_change":"2024-09-19T08:46:40.318237+0000","last_active":"2024-09-19T08:46:41.302365+0000","last_peered":"2024-09-19T08:46:41.302365+0000","last_clean":"2024-09-19T08:46:41.302365+0000","last_became_active":"2024-09-19T08:46:40.317621+0000","last_became_peered":"2024-09-19T08:46:40.317621+0000","last_unstale":"2024-09-19T08:46:41.302365+0000","last_undegraded":"2024-09-19T08:46:41.302365+0000","last_fullsized":"2024-09-19T08:46:41.302365+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:09.679777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.275004+0000","last_change":"2024-09-19T08:46:40.295471+0000","last_active":"2024-09-19T08:46:41.275004+0000","last_peered":"2024-09-19T08:46:41.275004+0000","last_clean":"2024-09-19T08:46:41.275004+0000","last_became_active":"2024-09-19T08:46:40.294889+0000","last_became_peered":"2024-09-19T08:46:40.294889+0000","last_unstale":"2024-09-19T08:46:41.275004+0000","last_undegraded":"2024-09-19T08:46:41.275004+0000","last_fullsized":"2024-09-19T08:46:41.275004+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:44.098158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295327+0000","last_change":"2024-09-19T08:46:40.306178+0000","last_active":"2024-09-19T08:46:41.295327+0000","last_peered":"2024-09-19T08:46:41.295327+0000","last_clean":"2024-09-19T08:46:41.295327+0000","last_became_active":"2024-09-19T08:46:40.304073+0000","last_became_peered":"2024-09-19T08:46:40.304073+0000","last_unstale":"2024-09-19T08:46:41.295327+0000","last_undegraded":"2024-09-19T08:46:41.295327+0000","last_fullsized":"2024-09-19T08:46:41.295327+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:22.830837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271822+0000","last_change":"2024-09-19T08:46:40.287586+0000","last_active":"2024-09-19T08:46:41.271822+0000","last_peered":"2024-09-19T08:46:41.271822+0000","last_clean":"2024-09-19T08:46:41.271822+0000","last_became_active":"2024-09-19T08:46:40.286802+0000","last_became_peered":"2024-09-19T08:46:40.286802+0000","last_unstale":"2024-09-19T08:46:41.271822+0000","last_undegraded":"2024-09-19T08:46:41.271822+0000","last_fullsized":"2024-09-19T08:46:41.271822+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:53:20.598758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.299555+0000","last_change":"2024-09-19T08:46:40.325550+0000","last_active":"2024-09-19T08:46:41.299555+0000","last_peered":"2024-09-19T08:46:41.299555+0000","last_clean":"2024-09-19T08:46:41.299555+0000","last_became_active":"2024-09-19T08:46:40.325309+0000","last_became_peered":"2024-09-19T08:46:40.325309+0000","last_unstale":"2024-09-19T08:46:41.299555+0000","last_undegraded":"2024-09-19T08:46:41.299555+0000","last_fullsized":"2024-09-19T08:46:41.299555+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:21:31.832739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295824+0000","last_change":"2024-09-19T08:46:40.312961+0000","last_active":"2024-09-19T08:46:41.295824+0000","last_peered":"2024-09-19T08:46:41.295824+0000","last_clean":"2024-09-19T08:46:41.295824+0000","last_became_active":"2024-09-19T08:46:40.312097+0000","last_became_peered":"2024-09-19T08:46:40.312097+0000","last_unstale":"2024-09-19T08:46:41.295824+0000","last_undegraded":"2024-09-19T08:46:41.295824+0000","last_fullsized":"2024-09-19T08:46:41.295824+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:58:38.046334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271499+0000","last_change":"2024-09-19T08:46:40.286511+0000","last_active":"2024-09-19T08:46:41.271499+0000","last_peered":"2024-09-19T08:46:41.271499+0000","last_clean":"2024-09-19T08:46:41.271499+0000","last_became_active":"2024-09-19T08:46:40.286045+0000","last_became_peered":"2024-09-19T08:46:40.286045+0000","last_unstale":"2024-09-19T08:46:41.271499+0000","last_undegraded":"2024-09-19T08:46:41.271499+0000","last_fullsized":"2024-09-19T08:46:41.271499+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:48.932583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271520+0000","last_change":"2024-09-19T08:46:40.312590+0000","last_active":"2024-09-19T08:46:41.271520+0000","last_peered":"2024-09-19T08:46:41.271520+0000","last_clean":"2024-09-19T08:46:41.271520+0000","last_became_active":"2024-09-19T08:46:40.312010+0000","last_became_peered":"2024-09-19T08:46:40.312010+0000","last_unstale":"2024-09-19T08:46:41.271520+0000","last_undegraded":"2024-09-19T08:46:41.271520+0000","last_fullsized":"2024-09-19T08:46:41.271520+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:17:58.016895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.294442+0000","last_change":"2024-09-19T08:46:40.314682+0000","last_active":"2024-09-19T08:46:41.294442+0000","last_peered":"2024-09-19T08:46:41.294442+0000","last_clean":"2024-09-19T08:46:41.294442+0000","last_became_active":"2024-09-19T08:46:40.313976+0000","last_became_peered":"2024-09-19T08:46:40.313976+0000","last_unstale":"2024-09-19T08:46:41.294442+0000","last_undegraded":"2024-09-19T08:46:41.294442+0000","last_fullsized":"2024-09-19T08:46:41.294442+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:27:03.146587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.293441+0000","last_change":"2024-09-19T08:46:40.300547+0000","last_active":"2024-09-19T08:46:41.293441+0000","last_peered":"2024-09-19T08:46:41.293441+0000","last_clean":"2024-09-19T08:46:41.293441+0000","last_became_active":"2024-09-19T08:46:40.299848+0000","last_became_peered":"2024-09-19T08:46:40.299848+0000","last_unstale":"2024-09-19T08:46:41.293441+0000","last_undegraded":"2024-09-19T08:46:41.293441+0000","last_fullsized":"2024-09-19T08:46:41.293441+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:29:23.072767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295172+0000","last_change":"2024-09-19T08:46:40.296781+0000","last_active":"2024-09-19T08:46:41.295172+0000","last_peered":"2024-09-19T08:46:41.295172+0000","last_clean":"2024-09-19T08:46:41.295172+0000","last_became_active":"2024-09-19T08:46:40.296159+0000","last_became_peered":"2024-09-19T08:46:40.296159+0000","last_unstale":"2024-09-19T08:46:41.295172+0000","last_undegraded":"2024-09-19T08:46:41.295172+0000","last_fullsized":"2024-09-19T08:46:41.295172+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:36:31.828168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.275725+0000","last_change":"2024-09-19T08:46:40.311172+0000","last_active":"2024-09-19T08:46:41.275725+0000","last_peered":"2024-09-19T08:46:41.275725+0000","last_clean":"2024-09-19T08:46:41.275725+0000","last_became_active":"2024-09-19T08:46:40.310768+0000","last_became_peered":"2024-09-19T08:46:40.310768+0000","last_unstale":"2024-09-19T08:46:41.275725+0000","last_undegraded":"2024-09-19T08:46:41.275725+0000","last_fullsized":"2024-09-19T08:46:41.275725+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:26:01.059827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.298510+0000","last_change":"2024-09-19T08:46:40.304105+0000","last_active":"2024-09-19T08:46:41.298510+0000","last_peered":"2024-09-19T08:46:41.298510+0000","last_clean":"2024-09-19T08:46:41.298510+0000","last_became_active":"2024-09-19T08:46:40.303618+0000","last_became_peered":"2024-09-19T08:46:40.303618+0000","last_unstale":"2024-09-19T08:46:41.298510+0000","last_undegraded":"2024-09-19T08:46:41.298510+0000","last_fullsized":"2024-09-19T08:46:41.298510+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:15:05.241341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.294336+0000","last_change":"2024-09-19T08:46:40.310327+0000","last_active":"2024-09-19T08:46:41.294336+0000","last_peered":"2024-09-19T08:46:41.294336+0000","last_clean":"2024-09-19T08:46:41.294336+0000","last_became_active":"2024-09-19T08:46:40.309561+0000","last_became_peered":"2024-09-19T08:46:40.309561+0000","last_unstale":"2024-09-19T08:46:41.294336+0000","last_undegraded":"2024-09-19T08:46:41.294336+0000","last_fullsized":"2024-09-19T08:46:41.294336+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:39:18.317103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271831+0000","last_change":"2024-09-19T08:46:40.297612+0000","last_active":"2024-09-19T08:46:41.271831+0000","last_peered":"2024-09-19T08:46:41.271831+0000","last_clean":"2024-09-19T08:46:41.271831+0000","last_became_active":"2024-09-19T08:46:40.296522+0000","last_became_peered":"2024-09-19T08:46:40.296522+0000","last_unstale":"2024-09-19T08:46:41.271831+0000","last_undegraded":"2024-09-19T08:46:41.271831+0000","last_fullsized":"2024-09-19T08:46:41.271831+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:19:50.929185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.275232+0000","last_change":"2024-09-19T08:46:40.319198+0000","last_active":"2024-09-19T08:46:41.275232+0000","last_peered":"2024-09-19T08:46:41.275232+0000","last_clean":"2024-09-19T08:46:41.275232+0000","last_became_active":"2024-09-19T08:46:40.318778+0000","last_became_peered":"2024-09-19T08:46:40.318778+0000","last_unstale":"2024-09-19T08:46:41.275232+0000","last_undegraded":"2024-09-19T08:46:41.275232+0000","last_fullsized":"2024-09-19T08:46:41.275232+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:17:12.366818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.301582+0000","last_change":"2024-09-19T08:46:40.324681+0000","last_active":"2024-09-19T08:46:41.301582+0000","last_peered":"2024-09-19T08:46:41.301582+0000","last_clean":"2024-09-19T08:46:41.301582+0000","last_became_active":"2024-09-19T08:46:40.321740+0000","last_became_peered":"2024-09-19T08:46:40.321740+0000","last_unstale":"2024-09-19T08:46:41.301582+0000","last_undegraded":"2024-09-19T08:46:41.301582+0000","last_fullsized":"2024-09-19T08:46:41.301582+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:44:45.487714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.293584+0000","last_change":"2024-09-19T08:46:40.295308+0000","last_active":"2024-09-19T08:46:41.293584+0000","last_peered":"2024-09-19T08:46:41.293584+0000","last_clean":"2024-09-19T08:46:41.293584+0000","last_became_active":"2024-09-19T08:46:40.290389+0000","last_became_peered":"2024-09-19T08:46:40.290389+0000","last_unstale":"2024-09-19T08:46:41.293584+0000","last_undegraded":"2024-09-19T08:46:41.293584+0000","last_fullsized":"2024-09-19T08:46:41.293584+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:09:25.970759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271075+0000","last_change":"2024-09-19T08:46:40.287160+0000","last_active":"2024-09-19T08:46:41.271075+0000","last_peered":"2024-09-19T08:46:41.271075+0000","last_clean":"2024-09-19T08:46:41.271075+0000","last_became_active":"2024-09-19T08:46:40.286383+0000","last_became_peered":"2024-09-19T08:46:40.286383+0000","last_unstale":"2024-09-19T08:46:41.271075+0000","last_undegraded":"2024-09-19T08:46:41.271075+0000","last_fullsized":"2024-09-19T08:46:41.271075+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:56.945612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.296321+0000","last_change":"2024-09-19T08:46:40.305212+0000","last_active":"2024-09-19T08:46:41.296321+0000","last_peered":"2024-09-19T08:46:41.296321+0000","last_clean":"2024-09-19T08:46:41.296321+0000","last_became_active":"2024-09-19T08:46:40.304643+0000","last_became_peered":"2024-09-19T08:46:40.304643+0000","last_unstale":"2024-09-19T08:46:41.296321+0000","last_undegraded":"2024-09-19T08:46:41.296321+0000","last_fullsized":"2024-09-19T08:46:41.296321+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:57:56.539906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"157'4","reported_seq":18,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271785+0000","last_change":"2024-09-19T08:46:39.533871+0000","last_active":"2024-09-19T08:46:41.271785+0000","last_peered":"2024-09-19T08:46:41.271785+0000","last_clean":"2024-09-19T08:46:41.271785+0000","last_became_active":"2024-09-19T08:46:39.533575+0000","last_became_peered":"2024-09-19T08:46:39.533575+0000","last_unstale":"2024-09-19T08:46:41.271785+0000","last_undegraded":"2024-09-19T08:46:41.271785+0000","last_fullsized":"2024-09-19T08:46:41.271785+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:36:08.646357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.296566+0000","last_change":"2024-09-19T08:46:40.299343+0000","last_active":"2024-09-19T08:46:41.296566+0000","last_peered":"2024-09-19T08:46:41.296566+0000","last_clean":"2024-09-19T08:46:41.296566+0000","last_became_active":"2024-09-19T08:46:40.296360+0000","last_became_peered":"2024-09-19T08:46:40.296360+0000","last_unstale":"2024-09-19T08:46:41.296566+0000","last_undegraded":"2024-09-19T08:46:41.296566+0000","last_fullsized":"2024-09-19T08:46:41.296566+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:27:03.803525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"157'1","reported_seq":15,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.302393+0000","last_change":"2024-09-19T08:46:39.524090+0000","last_active":"2024-09-19T08:46:41.302393+0000","last_peered":"2024-09-19T08:46:41.302393+0000","last_clean":"2024-09-19T08:46:41.302393+0000","last_became_active":"2024-09-19T08:46:39.523389+0000","last_became_peered":"2024-09-19T08:46:39.523389+0000","last_unstale":"2024-09-19T08:46:41.302393+0000","last_undegraded":"2024-09-19T08:46:41.302393+0000","last_fullsized":"2024-09-19T08:46:41.302393+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:54:57.317359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.299684+0000","last_change":"2024-09-19T08:46:40.287661+0000","last_active":"2024-09-19T08:46:41.299684+0000","last_peered":"2024-09-19T08:46:41.299684+0000","last_clean":"2024-09-19T08:46:41.299684+0000","last_became_active":"2024-09-19T08:46:40.287397+0000","last_became_peered":"2024-09-19T08:46:40.287397+0000","last_unstale":"2024-09-19T08:46:41.299684+0000","last_undegraded":"2024-09-19T08:46:41.299684+0000","last_fullsized":"2024-09-19T08:46:41.299684+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:15.351548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"157'2","reported_seq":16,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.294377+0000","last_change":"2024-09-19T08:46:39.517378+0000","last_active":"2024-09-19T08:46:41.294377+0000","last_peered":"2024-09-19T08:46:41.294377+0000","last_clean":"2024-09-19T08:46:41.294377+0000","last_became_active":"2024-09-19T08:46:39.517112+0000","last_became_peered":"2024-09-19T08:46:39.517112+0000","last_unstale":"2024-09-19T08:46:41.294377+0000","last_undegraded":"2024-09-19T08:46:41.294377+0000","last_fullsized":"2024-09-19T08:46:41.294377+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:21:46.481801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.302427+0000","last_change":"2024-09-19T08:46:40.305888+0000","last_active":"2024-09-19T08:46:41.302427+0000","last_peered":"2024-09-19T08:46:41.302427+0000","last_clean":"2024-09-19T08:46:41.302427+0000","last_became_active":"2024-09-19T08:46:40.305283+0000","last_became_peered":"2024-09-19T08:46:40.305283+0000","last_unstale":"2024-09-19T08:46:41.302427+0000","last_undegraded":"2024-09-19T08:46:41.302427+0000","last_fullsized":"2024-09-19T08:46:41.302427+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:15.659241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"157'2","reported_seq":16,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.275270+0000","last_change":"2024-09-19T08:46:39.525313+0000","last_active":"2024-09-19T08:46:41.275270+0000","last_peered":"2024-09-19T08:46:41.275270+0000","last_clean":"2024-09-19T08:46:41.275270+0000","last_became_active":"2024-09-19T08:46:39.524576+0000","last_became_peered":"2024-09-19T08:46:39.524576+0000","last_unstale":"2024-09-19T08:46:41.275270+0000","last_undegraded":"2024-09-19T08:46:41.275270+0000","last_fullsized":"2024-09-19T08:46:41.275270+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:57.140465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.294535+0000","last_change":"2024-09-19T08:46:40.311346+0000","last_active":"2024-09-19T08:46:41.294535+0000","last_peered":"2024-09-19T08:46:41.294535+0000","last_clean":"2024-09-19T08:46:41.294535+0000","last_became_active":"2024-09-19T08:46:40.310733+0000","last_became_peered":"2024-09-19T08:46:40.310733+0000","last_unstale":"2024-09-19T08:46:41.294535+0000","last_undegraded":"2024-09-19T08:46:41.294535+0000","last_fullsized":"2024-09-19T08:46:41.294535+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:36:50.649569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"157'1","reported_seq":15,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271232+0000","last_change":"2024-09-19T08:46:39.526200+0000","last_active":"2024-09-19T08:46:41.271232+0000","last_peered":"2024-09-19T08:46:41.271232+0000","last_clean":"2024-09-19T08:46:41.271232+0000","last_became_active":"2024-09-19T08:46:39.525883+0000","last_became_peered":"2024-09-19T08:46:39.525883+0000","last_unstale":"2024-09-19T08:46:41.271232+0000","last_undegraded":"2024-09-19T08:46:41.271232+0000","last_fullsized":"2024-09-19T08:46:41.271232+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:52.132068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295513+0000","last_change":"2024-09-19T08:46:40.305229+0000","last_active":"2024-09-19T08:46:41.295513+0000","last_peered":"2024-09-19T08:46:41.295513+0000","last_clean":"2024-09-19T08:46:41.295513+0000","last_became_active":"2024-09-19T08:46:40.304492+0000","last_became_peered":"2024-09-19T08:46:40.304492+0000","last_unstale":"2024-09-19T08:46:41.295513+0000","last_undegraded":"2024-09-19T08:46:41.295513+0000","last_fullsized":"2024-09-19T08:46:41.295513+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:32.712559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"157'1","reported_seq":15,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295463+0000","last_change":"2024-09-19T08:46:39.519337+0000","last_active":"2024-09-19T08:46:41.295463+0000","last_peered":"2024-09-19T08:46:41.295463+0000","last_clean":"2024-09-19T08:46:41.295463+0000","last_became_active":"2024-09-19T08:46:39.518792+0000","last_became_peered":"2024-09-19T08:46:39.518792+0000","last_unstale":"2024-09-19T08:46:41.295463+0000","last_undegraded":"2024-09-19T08:46:41.295463+0000","last_fullsized":"2024-09-19T08:46:41.295463+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:30:05.550797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.297805+0000","last_change":"2024-09-19T08:46:40.311303+0000","last_active":"2024-09-19T08:46:41.297805+0000","last_peered":"2024-09-19T08:46:41.297805+0000","last_clean":"2024-09-19T08:46:41.297805+0000","last_became_active":"2024-09-19T08:46:40.310999+0000","last_became_peered":"2024-09-19T08:46:40.310999+0000","last_unstale":"2024-09-19T08:46:41.297805+0000","last_undegraded":"2024-09-19T08:46:41.297805+0000","last_fullsized":"2024-09-19T08:46:41.297805+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:49.743970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.298406+0000","last_change":"2024-09-19T08:46:39.532881+0000","last_active":"2024-09-19T08:46:41.298406+0000","last_peered":"2024-09-19T08:46:41.298406+0000","last_clean":"2024-09-19T08:46:41.298406+0000","last_became_active":"2024-09-19T08:46:39.532112+0000","last_became_peered":"2024-09-19T08:46:39.532112+0000","last_unstale":"2024-09-19T08:46:41.298406+0000","last_undegraded":"2024-09-19T08:46:41.298406+0000","last_fullsized":"2024-09-19T08:46:41.298406+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:32.581664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.294399+0000","last_change":"2024-09-19T08:46:40.296034+0000","last_active":"2024-09-19T08:46:41.294399+0000","last_peered":"2024-09-19T08:46:41.294399+0000","last_clean":"2024-09-19T08:46:41.294399+0000","last_became_active":"2024-09-19T08:46:40.294878+0000","last_became_peered":"2024-09-19T08:46:40.294878+0000","last_unstale":"2024-09-19T08:46:41.294399+0000","last_undegraded":"2024-09-19T08:46:41.294399+0000","last_fullsized":"2024-09-19T08:46:41.294399+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:44.855570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.299039+0000","last_change":"2024-09-19T08:46:39.540052+0000","last_active":"2024-09-19T08:46:41.299039+0000","last_peered":"2024-09-19T08:46:41.299039+0000","last_clean":"2024-09-19T08:46:41.299039+0000","last_became_active":"2024-09-19T08:46:39.539378+0000","last_became_peered":"2024-09-19T08:46:39.539378+0000","last_unstale":"2024-09-19T08:46:41.299039+0000","last_undegraded":"2024-09-19T08:46:41.299039+0000","last_fullsized":"2024-09-19T08:46:41.299039+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:06.592079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.296558+0000","last_change":"2024-09-19T08:46:40.302139+0000","last_active":"2024-09-19T08:46:41.296558+0000","last_peered":"2024-09-19T08:46:41.296558+0000","last_clean":"2024-09-19T08:46:41.296558+0000","last_became_active":"2024-09-19T08:46:40.301363+0000","last_became_peered":"2024-09-19T08:46:40.301363+0000","last_unstale":"2024-09-19T08:46:41.296558+0000","last_undegraded":"2024-09-19T08:46:41.296558+0000","last_fullsized":"2024-09-19T08:46:41.296558+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:13.914441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271475+0000","last_change":"2024-09-19T08:46:39.529992+0000","last_active":"2024-09-19T08:46:41.271475+0000","last_peered":"2024-09-19T08:46:41.271475+0000","last_clean":"2024-09-19T08:46:41.271475+0000","last_became_active":"2024-09-19T08:46:39.529500+0000","last_became_peered":"2024-09-19T08:46:39.529500+0000","last_unstale":"2024-09-19T08:46:41.271475+0000","last_undegraded":"2024-09-19T08:46:41.271475+0000","last_fullsized":"2024-09-19T08:46:41.271475+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:34.276215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.298137+0000","last_change":"2024-09-19T08:46:40.306814+0000","last_active":"2024-09-19T08:46:41.298137+0000","last_peered":"2024-09-19T08:46:41.298137+0000","last_clean":"2024-09-19T08:46:41.298137+0000","last_became_active":"2024-09-19T08:46:40.306279+0000","last_became_peered":"2024-09-19T08:46:40.306279+0000","last_unstale":"2024-09-19T08:46:41.298137+0000","last_undegraded":"2024-09-19T08:46:41.298137+0000","last_fullsized":"2024-09-19T08:46:41.298137+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:45.155900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"157'3","reported_seq":17,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271615+0000","last_change":"2024-09-19T08:46:39.530600+0000","last_active":"2024-09-19T08:46:41.271615+0000","last_peered":"2024-09-19T08:46:41.271615+0000","last_clean":"2024-09-19T08:46:41.271615+0000","last_became_active":"2024-09-19T08:46:39.526884+0000","last_became_peered":"2024-09-19T08:46:39.526884+0000","last_unstale":"2024-09-19T08:46:41.271615+0000","last_undegraded":"2024-09-19T08:46:41.271615+0000","last_fullsized":"2024-09-19T08:46:41.271615+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:01:13.258546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295715+0000","last_change":"2024-09-19T08:46:40.310166+0000","last_active":"2024-09-19T08:46:41.295715+0000","last_peered":"2024-09-19T08:46:41.295715+0000","last_clean":"2024-09-19T08:46:41.295715+0000","last_became_active":"2024-09-19T08:46:40.309720+0000","last_became_peered":"2024-09-19T08:46:40.309720+0000","last_unstale":"2024-09-19T08:46:41.295715+0000","last_undegraded":"2024-09-19T08:46:41.295715+0000","last_fullsized":"2024-09-19T08:46:41.295715+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:53:13.406407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"157'6","reported_seq":20,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.294318+0000","last_change":"2024-09-19T08:46:39.533705+0000","last_active":"2024-09-19T08:46:41.294318+0000","last_peered":"2024-09-19T08:46:41.294318+0000","last_clean":"2024-09-19T08:46:41.294318+0000","last_became_active":"2024-09-19T08:46:39.533388+0000","last_became_peered":"2024-09-19T08:46:39.533388+0000","last_unstale":"2024-09-19T08:46:41.294318+0000","last_undegraded":"2024-09-19T08:46:41.294318+0000","last_fullsized":"2024-09-19T08:46:41.294318+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:29.674750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.293672+0000","last_change":"2024-09-19T08:46:40.298398+0000","last_active":"2024-09-19T08:46:41.293672+0000","last_peered":"2024-09-19T08:46:41.293672+0000","last_clean":"2024-09-19T08:46:41.293672+0000","last_became_active":"2024-09-19T08:46:40.297708+0000","last_became_peered":"2024-09-19T08:46:40.297708+0000","last_unstale":"2024-09-19T08:46:41.293672+0000","last_undegraded":"2024-09-19T08:46:41.293672+0000","last_fullsized":"2024-09-19T08:46:41.293672+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:20.271146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"157'2","reported_seq":16,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.298905+0000","last_change":"2024-09-19T08:46:39.532544+0000","last_active":"2024-09-19T08:46:41.298905+0000","last_peered":"2024-09-19T08:46:41.298905+0000","last_clean":"2024-09-19T08:46:41.298905+0000","last_became_active":"2024-09-19T08:46:39.531951+0000","last_became_peered":"2024-09-19T08:46:39.531951+0000","last_unstale":"2024-09-19T08:46:41.298905+0000","last_undegraded":"2024-09-19T08:46:41.298905+0000","last_fullsized":"2024-09-19T08:46:41.298905+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:44:23.781566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.297970+0000","last_change":"2024-09-19T08:46:40.302762+0000","last_active":"2024-09-19T08:46:41.297970+0000","last_peered":"2024-09-19T08:46:41.297970+0000","last_clean":"2024-09-19T08:46:41.297970+0000","last_became_active":"2024-09-19T08:46:40.301495+0000","last_became_peered":"2024-09-19T08:46:40.301495+0000","last_unstale":"2024-09-19T08:46:41.297970+0000","last_undegraded":"2024-09-19T08:46:41.297970+0000","last_fullsized":"2024-09-19T08:46:41.297970+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:39:18.317103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.294654+0000","last_change":"2024-09-19T08:46:39.523177+0000","last_active":"2024-09-19T08:46:41.294654+0000","last_peered":"2024-09-19T08:46:41.294654+0000","last_clean":"2024-09-19T08:46:41.294654+0000","last_became_active":"2024-09-19T08:46:39.522823+0000","last_became_peered":"2024-09-19T08:46:39.522823+0000","last_unstale":"2024-09-19T08:46:41.294654+0000","last_undegraded":"2024-09-19T08:46:41.294654+0000","last_fullsized":"2024-09-19T08:46:41.294654+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:32.581664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.301407+0000","last_change":"2024-09-19T08:46:40.319863+0000","last_active":"2024-09-19T08:46:41.301407+0000","last_peered":"2024-09-19T08:46:41.301407+0000","last_clean":"2024-09-19T08:46:41.301407+0000","last_became_active":"2024-09-19T08:46:40.319407+0000","last_became_peered":"2024-09-19T08:46:40.319407+0000","last_unstale":"2024-09-19T08:46:41.301407+0000","last_undegraded":"2024-09-19T08:46:41.301407+0000","last_fullsized":"2024-09-19T08:46:41.301407+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:34.896081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295262+0000","last_change":"2024-09-19T08:46:39.528911+0000","last_active":"2024-09-19T08:46:41.295262+0000","last_peered":"2024-09-19T08:46:41.295262+0000","last_clean":"2024-09-19T08:46:41.295262+0000","last_became_active":"2024-09-19T08:46:39.528519+0000","last_became_peered":"2024-09-19T08:46:39.528519+0000","last_unstale":"2024-09-19T08:46:41.295262+0000","last_undegraded":"2024-09-19T08:46:41.295262+0000","last_fullsized":"2024-09-19T08:46:41.295262+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:05.280493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.294063+0000","last_change":"2024-09-19T08:46:40.290329+0000","last_active":"2024-09-19T08:46:41.294063+0000","last_peered":"2024-09-19T08:46:41.294063+0000","last_clean":"2024-09-19T08:46:41.294063+0000","last_became_active":"2024-09-19T08:46:40.289851+0000","last_became_peered":"2024-09-19T08:46:40.289851+0000","last_unstale":"2024-09-19T08:46:41.294063+0000","last_undegraded":"2024-09-19T08:46:41.294063+0000","last_fullsized":"2024-09-19T08:46:41.294063+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:36:51.887939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"157'8","reported_seq":22,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.298681+0000","last_change":"2024-09-19T08:46:39.534632+0000","last_active":"2024-09-19T08:46:41.298681+0000","last_peered":"2024-09-19T08:46:41.298681+0000","last_clean":"2024-09-19T08:46:41.298681+0000","last_became_active":"2024-09-19T08:46:39.533906+0000","last_became_peered":"2024-09-19T08:46:39.533906+0000","last_unstale":"2024-09-19T08:46:41.298681+0000","last_undegraded":"2024-09-19T08:46:41.298681+0000","last_fullsized":"2024-09-19T08:46:41.298681+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:42.284571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.296092+0000","last_change":"2024-09-19T08:46:40.294805+0000","last_active":"2024-09-19T08:46:41.296092+0000","last_peered":"2024-09-19T08:46:41.296092+0000","last_clean":"2024-09-19T08:46:41.296092+0000","last_became_active":"2024-09-19T08:46:40.294338+0000","last_became_peered":"2024-09-19T08:46:40.294338+0000","last_unstale":"2024-09-19T08:46:41.296092+0000","last_undegraded":"2024-09-19T08:46:41.296092+0000","last_fullsized":"2024-09-19T08:46:41.296092+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:10:19.078157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"157'1","reported_seq":15,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271881+0000","last_change":"2024-09-19T08:46:39.529025+0000","last_active":"2024-09-19T08:46:41.271881+0000","last_peered":"2024-09-19T08:46:41.271881+0000","last_clean":"2024-09-19T08:46:41.271881+0000","last_became_active":"2024-09-19T08:46:39.526531+0000","last_became_peered":"2024-09-19T08:46:39.526531+0000","last_unstale":"2024-09-19T08:46:41.271881+0000","last_undegraded":"2024-09-19T08:46:41.271881+0000","last_fullsized":"2024-09-19T08:46:41.271881+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:18:57.366963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.302806+0000","last_change":"2024-09-19T08:46:40.320361+0000","last_active":"2024-09-19T08:46:41.302806+0000","last_peered":"2024-09-19T08:46:41.302806+0000","last_clean":"2024-09-19T08:46:41.302806+0000","last_became_active":"2024-09-19T08:46:40.319637+0000","last_became_peered":"2024-09-19T08:46:40.319637+0000","last_unstale":"2024-09-19T08:46:41.302806+0000","last_undegraded":"2024-09-19T08:46:41.302806+0000","last_fullsized":"2024-09-19T08:46:41.302806+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:09:14.940629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"157'2","reported_seq":16,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.296238+0000","last_change":"2024-09-19T08:46:39.528693+0000","last_active":"2024-09-19T08:46:41.296238+0000","last_peered":"2024-09-19T08:46:41.296238+0000","last_clean":"2024-09-19T08:46:41.296238+0000","last_became_active":"2024-09-19T08:46:39.528063+0000","last_became_peered":"2024-09-19T08:46:39.528063+0000","last_unstale":"2024-09-19T08:46:41.296238+0000","last_undegraded":"2024-09-19T08:46:41.296238+0000","last_fullsized":"2024-09-19T08:46:41.296238+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:04:47.931376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.301874+0000","last_change":"2024-09-19T08:46:40.324664+0000","last_active":"2024-09-19T08:46:41.301874+0000","last_peered":"2024-09-19T08:46:41.301874+0000","last_clean":"2024-09-19T08:46:41.301874+0000","last_became_active":"2024-09-19T08:46:40.321357+0000","last_became_peered":"2024-09-19T08:46:40.321357+0000","last_unstale":"2024-09-19T08:46:41.301874+0000","last_undegraded":"2024-09-19T08:46:41.301874+0000","last_fullsized":"2024-09-19T08:46:41.301874+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:59.869941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"157'1","reported_seq":15,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.299335+0000","last_change":"2024-09-19T08:46:39.534210+0000","last_active":"2024-09-19T08:46:41.299335+0000","last_peered":"2024-09-19T08:46:41.299335+0000","last_clean":"2024-09-19T08:46:41.299335+0000","last_became_active":"2024-09-19T08:46:39.533548+0000","last_became_peered":"2024-09-19T08:46:39.533548+0000","last_unstale":"2024-09-19T08:46:41.299335+0000","last_undegraded":"2024-09-19T08:46:41.299335+0000","last_fullsized":"2024-09-19T08:46:41.299335+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:50.291503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.275460+0000","last_change":"2024-09-19T08:46:40.308301+0000","last_active":"2024-09-19T08:46:41.275460+0000","last_peered":"2024-09-19T08:46:41.275460+0000","last_clean":"2024-09-19T08:46:41.275460+0000","last_became_active":"2024-09-19T08:46:40.307793+0000","last_became_peered":"2024-09-19T08:46:40.307793+0000","last_unstale":"2024-09-19T08:46:41.275460+0000","last_undegraded":"2024-09-19T08:46:41.275460+0000","last_fullsized":"2024-09-19T08:46:41.275460+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:21.144985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.298624+0000","last_change":"2024-09-19T08:46:39.536189+0000","last_active":"2024-09-19T08:46:41.298624+0000","last_peered":"2024-09-19T08:46:41.298624+0000","last_clean":"2024-09-19T08:46:41.298624+0000","last_became_active":"2024-09-19T08:46:39.535387+0000","last_became_peered":"2024-09-19T08:46:39.535387+0000","last_unstale":"2024-09-19T08:46:41.298624+0000","last_undegraded":"2024-09-19T08:46:41.298624+0000","last_fullsized":"2024-09-19T08:46:41.298624+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:00:43.262958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.294622+0000","last_change":"2024-09-19T08:46:40.308240+0000","last_active":"2024-09-19T08:46:41.294622+0000","last_peered":"2024-09-19T08:46:41.294622+0000","last_clean":"2024-09-19T08:46:41.294622+0000","last_became_active":"2024-09-19T08:46:40.307131+0000","last_became_peered":"2024-09-19T08:46:40.307131+0000","last_unstale":"2024-09-19T08:46:41.294622+0000","last_undegraded":"2024-09-19T08:46:41.294622+0000","last_fullsized":"2024-09-19T08:46:41.294622+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:40:32.674934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"157'2","reported_seq":16,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.293923+0000","last_change":"2024-09-19T08:46:39.529933+0000","last_active":"2024-09-19T08:46:41.293923+0000","last_peered":"2024-09-19T08:46:41.293923+0000","last_clean":"2024-09-19T08:46:41.293923+0000","last_became_active":"2024-09-19T08:46:39.529602+0000","last_became_peered":"2024-09-19T08:46:39.529602+0000","last_unstale":"2024-09-19T08:46:41.293923+0000","last_undegraded":"2024-09-19T08:46:41.293923+0000","last_fullsized":"2024-09-19T08:46:41.293923+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:45:17.113767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.298023+0000","last_change":"2024-09-19T08:46:40.278369+0000","last_active":"2024-09-19T08:46:41.298023+0000","last_peered":"2024-09-19T08:46:41.298023+0000","last_clean":"2024-09-19T08:46:41.298023+0000","last_became_active":"2024-09-19T08:46:40.277767+0000","last_became_peered":"2024-09-19T08:46:40.277767+0000","last_unstale":"2024-09-19T08:46:41.298023+0000","last_undegraded":"2024-09-19T08:46:41.298023+0000","last_fullsized":"2024-09-19T08:46:41.298023+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:44.855570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271109+0000","last_change":"2024-09-19T08:46:39.526194+0000","last_active":"2024-09-19T08:46:41.271109+0000","last_peered":"2024-09-19T08:46:41.271109+0000","last_clean":"2024-09-19T08:46:41.271109+0000","last_became_active":"2024-09-19T08:46:39.525582+0000","last_became_peered":"2024-09-19T08:46:39.525582+0000","last_unstale":"2024-09-19T08:46:41.271109+0000","last_undegraded":"2024-09-19T08:46:41.271109+0000","last_fullsized":"2024-09-19T08:46:41.271109+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:54.940787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.274690+0000","last_change":"2024-09-19T08:46:40.311768+0000","last_active":"2024-09-19T08:46:41.274690+0000","last_peered":"2024-09-19T08:46:41.274690+0000","last_clean":"2024-09-19T08:46:41.274690+0000","last_became_active":"2024-09-19T08:46:40.311464+0000","last_became_peered":"2024-09-19T08:46:40.311464+0000","last_unstale":"2024-09-19T08:46:41.274690+0000","last_undegraded":"2024-09-19T08:46:41.274690+0000","last_fullsized":"2024-09-19T08:46:41.274690+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:08:48.768860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.293578+0000","last_change":"2024-09-19T08:46:39.531582+0000","last_active":"2024-09-19T08:46:41.293578+0000","last_peered":"2024-09-19T08:46:41.293578+0000","last_clean":"2024-09-19T08:46:41.293578+0000","last_became_active":"2024-09-19T08:46:39.531166+0000","last_became_peered":"2024-09-19T08:46:39.531166+0000","last_unstale":"2024-09-19T08:46:41.293578+0000","last_undegraded":"2024-09-19T08:46:41.293578+0000","last_fullsized":"2024-09-19T08:46:41.293578+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:45:20.818342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271332+0000","last_change":"2024-09-19T08:46:40.299301+0000","last_active":"2024-09-19T08:46:41.271332+0000","last_peered":"2024-09-19T08:46:41.271332+0000","last_clean":"2024-09-19T08:46:41.271332+0000","last_became_active":"2024-09-19T08:46:40.298235+0000","last_became_peered":"2024-09-19T08:46:40.298235+0000","last_unstale":"2024-09-19T08:46:41.271332+0000","last_undegraded":"2024-09-19T08:46:41.271332+0000","last_fullsized":"2024-09-19T08:46:41.271332+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:36:59.594448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.301286+0000","last_change":"2024-09-19T08:46:39.529718+0000","last_active":"2024-09-19T08:46:41.301286+0000","last_peered":"2024-09-19T08:46:41.301286+0000","last_clean":"2024-09-19T08:46:41.301286+0000","last_became_active":"2024-09-19T08:46:39.529000+0000","last_became_peered":"2024-09-19T08:46:39.529000+0000","last_unstale":"2024-09-19T08:46:41.301286+0000","last_undegraded":"2024-09-19T08:46:41.301286+0000","last_fullsized":"2024-09-19T08:46:41.301286+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:47.923282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271607+0000","last_change":"2024-09-19T08:46:40.296371+0000","last_active":"2024-09-19T08:46:41.271607+0000","last_peered":"2024-09-19T08:46:41.271607+0000","last_clean":"2024-09-19T08:46:41.271607+0000","last_became_active":"2024-09-19T08:46:40.295860+0000","last_became_peered":"2024-09-19T08:46:40.295860+0000","last_unstale":"2024-09-19T08:46:41.271607+0000","last_undegraded":"2024-09-19T08:46:41.271607+0000","last_fullsized":"2024-09-19T08:46:41.271607+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:54:58.326660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"157'2","reported_seq":16,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.299767+0000","last_change":"2024-09-19T08:46:39.533469+0000","last_active":"2024-09-19T08:46:41.299767+0000","last_peered":"2024-09-19T08:46:41.299767+0000","last_clean":"2024-09-19T08:46:41.299767+0000","last_became_active":"2024-09-19T08:46:39.532381+0000","last_became_peered":"2024-09-19T08:46:39.532381+0000","last_unstale":"2024-09-19T08:46:41.299767+0000","last_undegraded":"2024-09-19T08:46:41.299767+0000","last_fullsized":"2024-09-19T08:46:41.299767+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:38.871312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.296434+0000","last_change":"2024-09-19T08:46:40.316938+0000","last_active":"2024-09-19T08:46:41.296434+0000","last_peered":"2024-09-19T08:46:41.296434+0000","last_clean":"2024-09-19T08:46:41.296434+0000","last_became_active":"2024-09-19T08:46:40.316355+0000","last_became_peered":"2024-09-19T08:46:40.316355+0000","last_unstale":"2024-09-19T08:46:41.296434+0000","last_undegraded":"2024-09-19T08:46:41.296434+0000","last_fullsized":"2024-09-19T08:46:41.296434+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:20:14.319040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"158'1","reported_seq":15,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.296024+0000","last_change":"2024-09-19T08:46:39.517540+0000","last_active":"2024-09-19T08:46:41.296024+0000","last_peered":"2024-09-19T08:46:41.296024+0000","last_clean":"2024-09-19T08:46:41.296024+0000","last_became_active":"2024-09-19T08:46:39.517247+0000","last_became_peered":"2024-09-19T08:46:39.517247+0000","last_unstale":"2024-09-19T08:46:41.296024+0000","last_undegraded":"2024-09-19T08:46:41.296024+0000","last_fullsized":"2024-09-19T08:46:41.296024+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:42.770912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295742+0000","last_change":"2024-09-19T08:46:40.292589+0000","last_active":"2024-09-19T08:46:41.295742+0000","last_peered":"2024-09-19T08:46:41.295742+0000","last_clean":"2024-09-19T08:46:41.295742+0000","last_became_active":"2024-09-19T08:46:40.292129+0000","last_became_peered":"2024-09-19T08:46:40.292129+0000","last_unstale":"2024-09-19T08:46:41.295742+0000","last_undegraded":"2024-09-19T08:46:41.295742+0000","last_fullsized":"2024-09-19T08:46:41.295742+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:04.629751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295780+0000","last_change":"2024-09-19T08:46:39.517207+0000","last_active":"2024-09-19T08:46:41.295780+0000","last_peered":"2024-09-19T08:46:41.295780+0000","last_clean":"2024-09-19T08:46:41.295780+0000","last_became_active":"2024-09-19T08:46:39.516924+0000","last_became_peered":"2024-09-19T08:46:39.516924+0000","last_unstale":"2024-09-19T08:46:41.295780+0000","last_undegraded":"2024-09-19T08:46:41.295780+0000","last_fullsized":"2024-09-19T08:46:41.295780+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:55:04.131994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295871+0000","last_change":"2024-09-19T08:46:40.313527+0000","last_active":"2024-09-19T08:46:41.295871+0000","last_peered":"2024-09-19T08:46:41.295871+0000","last_clean":"2024-09-19T08:46:41.295871+0000","last_became_active":"2024-09-19T08:46:40.312733+0000","last_became_peered":"2024-09-19T08:46:40.312733+0000","last_unstale":"2024-09-19T08:46:41.295871+0000","last_undegraded":"2024-09-19T08:46:41.295871+0000","last_fullsized":"2024-09-19T08:46:41.295871+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:30.018530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295815+0000","last_change":"2024-09-19T08:46:39.533782+0000","last_active":"2024-09-19T08:46:41.295815+0000","last_peered":"2024-09-19T08:46:41.295815+0000","last_clean":"2024-09-19T08:46:41.295815+0000","last_became_active":"2024-09-19T08:46:39.533401+0000","last_became_peered":"2024-09-19T08:46:39.533401+0000","last_unstale":"2024-09-19T08:46:41.295815+0000","last_undegraded":"2024-09-19T08:46:41.295815+0000","last_fullsized":"2024-09-19T08:46:41.295815+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:14:49.514714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.298822+0000","last_change":"2024-09-19T08:46:40.317873+0000","last_active":"2024-09-19T08:46:41.298822+0000","last_peered":"2024-09-19T08:46:41.298822+0000","last_clean":"2024-09-19T08:46:41.298822+0000","last_became_active":"2024-09-19T08:46:40.316906+0000","last_became_peered":"2024-09-19T08:46:40.316906+0000","last_unstale":"2024-09-19T08:46:41.298822+0000","last_undegraded":"2024-09-19T08:46:41.298822+0000","last_fullsized":"2024-09-19T08:46:41.298822+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:08.299341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.301923+0000","last_change":"2024-09-19T08:46:39.521611+0000","last_active":"2024-09-19T08:46:41.301923+0000","last_peered":"2024-09-19T08:46:41.301923+0000","last_clean":"2024-09-19T08:46:41.301923+0000","last_became_active":"2024-09-19T08:46:39.520863+0000","last_became_peered":"2024-09-19T08:46:39.520863+0000","last_unstale":"2024-09-19T08:46:41.301923+0000","last_undegraded":"2024-09-19T08:46:41.301923+0000","last_fullsized":"2024-09-19T08:46:41.301923+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:53:19.589458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.298638+0000","last_change":"2024-09-19T08:46:40.317455+0000","last_active":"2024-09-19T08:46:41.298638+0000","last_peered":"2024-09-19T08:46:41.298638+0000","last_clean":"2024-09-19T08:46:41.298638+0000","last_became_active":"2024-09-19T08:46:40.317109+0000","last_became_peered":"2024-09-19T08:46:40.317109+0000","last_unstale":"2024-09-19T08:46:41.298638+0000","last_undegraded":"2024-09-19T08:46:41.298638+0000","last_fullsized":"2024-09-19T08:46:41.298638+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:06.068010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.301669+0000","last_change":"2024-09-19T08:46:39.536563+0000","last_active":"2024-09-19T08:46:41.301669+0000","last_peered":"2024-09-19T08:46:41.301669+0000","last_clean":"2024-09-19T08:46:41.301669+0000","last_became_active":"2024-09-19T08:46:39.535725+0000","last_became_peered":"2024-09-19T08:46:39.535725+0000","last_unstale":"2024-09-19T08:46:41.301669+0000","last_undegraded":"2024-09-19T08:46:41.301669+0000","last_fullsized":"2024-09-19T08:46:41.301669+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:17:57.007595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.274677+0000","last_change":"2024-09-19T08:46:40.300496+0000","last_active":"2024-09-19T08:46:41.274677+0000","last_peered":"2024-09-19T08:46:41.274677+0000","last_clean":"2024-09-19T08:46:41.274677+0000","last_became_active":"2024-09-19T08:46:40.299999+0000","last_became_peered":"2024-09-19T08:46:40.299999+0000","last_unstale":"2024-09-19T08:46:41.274677+0000","last_undegraded":"2024-09-19T08:46:41.274677+0000","last_fullsized":"2024-09-19T08:46:41.274677+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:57:03.311516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.274968+0000","last_change":"2024-09-19T08:46:39.539941+0000","last_active":"2024-09-19T08:46:41.274968+0000","last_peered":"2024-09-19T08:46:41.274968+0000","last_clean":"2024-09-19T08:46:41.274968+0000","last_became_active":"2024-09-19T08:46:39.539225+0000","last_became_peered":"2024-09-19T08:46:39.539225+0000","last_unstale":"2024-09-19T08:46:41.274968+0000","last_undegraded":"2024-09-19T08:46:41.274968+0000","last_fullsized":"2024-09-19T08:46:41.274968+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:46:06.121419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271306+0000","last_change":"2024-09-19T08:46:40.293254+0000","last_active":"2024-09-19T08:46:41.271306+0000","last_peered":"2024-09-19T08:46:41.271306+0000","last_clean":"2024-09-19T08:46:41.271306+0000","last_became_active":"2024-09-19T08:46:40.292460+0000","last_became_peered":"2024-09-19T08:46:40.292460+0000","last_unstale":"2024-09-19T08:46:41.271306+0000","last_undegraded":"2024-09-19T08:46:41.271306+0000","last_fullsized":"2024-09-19T08:46:41.271306+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:36:48.171899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.293216+0000","last_change":"2024-09-19T08:46:39.529973+0000","last_active":"2024-09-19T08:46:41.293216+0000","last_peered":"2024-09-19T08:46:41.293216+0000","last_clean":"2024-09-19T08:46:41.293216+0000","last_became_active":"2024-09-19T08:46:39.529691+0000","last_became_peered":"2024-09-19T08:46:39.529691+0000","last_unstale":"2024-09-19T08:46:41.293216+0000","last_undegraded":"2024-09-19T08:46:41.293216+0000","last_fullsized":"2024-09-19T08:46:41.293216+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:34.575229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.293890+0000","last_change":"2024-09-19T08:46:40.292254+0000","last_active":"2024-09-19T08:46:41.293890+0000","last_peered":"2024-09-19T08:46:41.293890+0000","last_clean":"2024-09-19T08:46:41.293890+0000","last_became_active":"2024-09-19T08:46:40.290492+0000","last_became_peered":"2024-09-19T08:46:40.290492+0000","last_unstale":"2024-09-19T08:46:41.293890+0000","last_undegraded":"2024-09-19T08:46:41.293890+0000","last_fullsized":"2024-09-19T08:46:41.293890+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:14:04.162980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.298903+0000","last_change":"2024-09-19T08:46:39.527908+0000","last_active":"2024-09-19T08:46:41.298903+0000","last_peered":"2024-09-19T08:46:41.298903+0000","last_clean":"2024-09-19T08:46:41.298903+0000","last_became_active":"2024-09-19T08:46:39.527338+0000","last_became_peered":"2024-09-19T08:46:39.527338+0000","last_unstale":"2024-09-19T08:46:41.298903+0000","last_undegraded":"2024-09-19T08:46:41.298903+0000","last_fullsized":"2024-09-19T08:46:41.298903+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:42.770912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"71'128","reported_seq":457,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.298973+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:46:41.298973+0000","last_peered":"2024-09-19T08:46:41.298973+0000","last_clean":"2024-09-19T08:46:41.298973+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:46:41.298973+0000","last_undegraded":"2024-09-19T08:46:41.298973+0000","last_fullsized":"2024-09-19T08:46:41.298973+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509546,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33724,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338116,"statfs":{"total":96636764160,"available":96602230784,"internally_reserved":0,"allocated":2584576,"data_stored":1471511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574955,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33128,"kb_used_data":1936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338712,"statfs":{"total":96636764160,"available":96602841088,"internally_reserved":0,"allocated":1982464,"data_stored":881039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509548,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33136,"kb_used_data":1936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338704,"statfs":{"total":96636764160,"available":96602832896,"internally_reserved":0,"allocated":1982464,"data_stored":881039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509548,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33140,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338700,"statfs":{"total":96636764160,"available":96602828800,"internally_reserved":0,"allocated":1998848,"data_stored":883233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542250,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37296,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334544,"statfs":{"total":96636764160,"available":96598573056,"internally_reserved":0,"allocated":1994752,"data_stored":881233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509545,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33144,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338696,"statfs":{"total":96636764160,"available":96602824704,"internally_reserved":0,"allocated":1994752,"data_stored":881733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509547,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33144,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338696,"statfs":{"total":96636764160,"available":96602824704,"internally_reserved":0,"allocated":1994752,"data_stored":882651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509545,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37876,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333964,"statfs":{"total":96636764160,"available":96597979136,"internally_reserved":0,"allocated":2580480,"data_stored":1471429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:46:45.256 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T08:46:45.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.418+0000 7fe6863a1640 1 Processor -- start 2024-09-19T08:46:45.419 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.418+0000 7fe6863a1640 1 -- start start 2024-09-19T08:46:45.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.422+0000 7fe6863a1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe68015bfe0 0x7fe680074c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:45.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.422+0000 7fe6863a1640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe68015f960 con 0x7fe6800751b0 2024-09-19T08:46:45.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.422+0000 7fe6863a1640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe68006bd30 con 0x7fe68006caf0 2024-09-19T08:46:45.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.422+0000 7fe6863a1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe68006c660 con 0x7fe68015bfe0 2024-09-19T08:46:45.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.422+0000 7fe68493e640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fe6800751b0 0x7fe68006dd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:55032/0 (socket says 172.21.15.98:55032) 2024-09-19T08:46:45.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.422+0000 7fe68493e640 1 -- 172.21.15.98:0/1509964511 learned_addr learned my addr 172.21.15.98:0/1509964511 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:45.422 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.422+0000 7fe67effd640 1 -- 172.21.15.98:0/1509964511 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3859579865 0 0) 0x7fe68015f960 con 0x7fe6800751b0 2024-09-19T08:46:45.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.422+0000 7fe67ffff640 1 --2- 172.21.15.98:0/1509964511 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe68015bfe0 0x7fe680074c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:45.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.422+0000 7fe67effd640 1 -- 172.21.15.98:0/1509964511 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe668003530 con 0x7fe6800751b0 2024-09-19T08:46:45.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.422+0000 7fe67effd640 1 -- 172.21.15.98:0/1509964511 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3918894571 0 0) 0x7fe668003530 con 0x7fe6800751b0 2024-09-19T08:46:45.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.422+0000 7fe67effd640 1 -- 172.21.15.98:0/1509964511 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe68015bfe0 msgr2=0x7fe680074c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:45.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.422+0000 7fe67effd640 1 --2- 172.21.15.98:0/1509964511 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe68015bfe0 0x7fe680074c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:45.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.422+0000 7fe67effd640 1 -- 172.21.15.98:0/1509964511 >> v1:172.21.15.139:6789/0 conn(0x7fe68006caf0 legacy=0x7fe68015eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:45.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.422+0000 7fe67effd640 1 -- 172.21.15.98:0/1509964511 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68006c010 con 0x7fe6800751b0 2024-09-19T08:46:45.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.422+0000 7fe67effd640 1 -- 172.21.15.98:0/1509964511 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fe678001a00 con 0x7fe6800751b0 2024-09-19T08:46:45.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.422+0000 7fe67effd640 1 -- 172.21.15.98:0/1509964511 >> v1:172.21.15.98:6789/0 conn(0x7fe6800751b0 legacy=0x7fe68006dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:45.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.422+0000 7fe67effd640 1 --2- 172.21.15.98:0/1509964511 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe68015bfe0 0x7fe6680039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:45.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.422+0000 7fe67effd640 1 --2- 172.21.15.98:0/1509964511 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe668003f70 0x7fe668004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:45.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.422+0000 7fe67effd640 1 --2- 172.21.15.98:0/1509964511 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe668004d50 0x7fe6680051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:45.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.422+0000 7fe67effd640 1 -- 172.21.15.98:0/1509964511 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe68006c660 con 0x7fe668004d50 2024-09-19T08:46:45.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.422+0000 7fe67effd640 1 -- 172.21.15.98:0/1509964511 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe66800bb40 con 0x7fe668003f70 2024-09-19T08:46:45.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.422+0000 7fe67effd640 1 -- 172.21.15.98:0/1509964511 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe66800bd40 con 0x7fe68015bfe0 2024-09-19T08:46:45.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.422+0000 7fe68493e640 1 --2- 172.21.15.98:0/1509964511 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe668004d50 0x7fe6680051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:45.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.422+0000 7fe67f7fe640 1 --2- 172.21.15.98:0/1509964511 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe668003f70 0x7fe668004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:45.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.422+0000 7fe67ffff640 1 --2- 172.21.15.98:0/1509964511 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe68015bfe0 0x7fe6680039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:45.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.422+0000 7fe68493e640 1 -- 172.21.15.98:0/1509964511 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe68015bfe0 msgr2=0x7fe6680039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:45.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.422+0000 7fe68493e640 1 --2- 172.21.15.98:0/1509964511 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe68015bfe0 0x7fe6680039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:45.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.422+0000 7fe68493e640 1 -- 172.21.15.98:0/1509964511 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe668003f70 msgr2=0x7fe668004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:45.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.422+0000 7fe68493e640 1 --2- 172.21.15.98:0/1509964511 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe668003f70 0x7fe668004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:45.424 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.422+0000 7fe68493e640 1 -- 172.21.15.98:0/1509964511 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe668003530 con 0x7fe668004d50 2024-09-19T08:46:45.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.426+0000 7fe68493e640 1 --2- 172.21.15.98:0/1509964511 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe668004d50 0x7fe6680051d0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fe67800a840 tx=0x7fe67801d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e2659c4f70c9c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:45.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.426+0000 7fe67effd640 1 -- 172.21.15.98:0/1509964511 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67800a460 con 0x7fe668004d50 2024-09-19T08:46:45.425 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.426+0000 7fe67effd640 1 -- 172.21.15.98:0/1509964511 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6780ad650 con 0x7fe668004d50 2024-09-19T08:46:45.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.426+0000 7fe67effd640 1 -- 172.21.15.98:0/1509964511 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6780ada80 con 0x7fe668004d50 2024-09-19T08:46:45.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.426+0000 7fe6863a1640 1 -- 172.21.15.98:0/1509964511 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe668004d50 msgr2=0x7fe6680051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:45.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.426+0000 7fe6863a1640 1 --2- 172.21.15.98:0/1509964511 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe668004d50 0x7fe6680051d0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fe67800a840 tx=0x7fe67801d360 comp rx=0 tx=0).stop 2024-09-19T08:46:45.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.426+0000 7fe68493e640 1 -- 172.21.15.98:0/1509964511 reap_dead start 2024-09-19T08:46:45.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.426+0000 7fe6863a1640 1 -- 172.21.15.98:0/1509964511 shutdown_connections 2024-09-19T08:46:45.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.426+0000 7fe6863a1640 1 -- 172.21.15.98:0/1509964511 >> 172.21.15.98:0/1509964511 conn(0x7fe6800b3500 msgr2=0x7fe6800b3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:45.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.426+0000 7fe6863a1640 1 -- 172.21.15.98:0/1509964511 shutdown_connections 2024-09-19T08:46:45.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.426+0000 7fe6863a1640 1 -- 172.21.15.98:0/1509964511 wait complete. 2024-09-19T08:46:45.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.426+0000 7fe6863a1640 1 Processor -- start 2024-09-19T08:46:45.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.426+0000 7fe6863a1640 1 -- start start 2024-09-19T08:46:45.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.426+0000 7fe6863a1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe668004d50 0x7fe68015c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:45.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.426+0000 7fe6863a1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe680146e10 0x7fe68015e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:45.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.426+0000 7fe6863a1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6801529d0 0x7fe680152e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:45.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.426+0000 7fe6863a1640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe680074c70 con 0x7fe668004d50 2024-09-19T08:46:45.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.426+0000 7fe6863a1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe68015c3c0 con 0x7fe6801529d0 2024-09-19T08:46:45.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.426+0000 7fe6863a1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe680074df0 con 0x7fe680146e10 2024-09-19T08:46:45.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.426+0000 7fe67ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe668004d50 0x7fe68015c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:45.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.426+0000 7fe67ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe668004d50 0x7fe68015c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54414/0 (socket says 172.21.15.98:54414) 2024-09-19T08:46:45.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.426+0000 7fe67ffff640 1 -- 172.21.15.98:0/267100670 learned_addr learned my addr 172.21.15.98:0/267100670 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:45.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.426+0000 7fe67f7fe640 1 --2- 172.21.15.98:0/267100670 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe680146e10 0x7fe68015e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:45.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.426+0000 7fe68493e640 1 --2- 172.21.15.98:0/267100670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6801529d0 0x7fe680152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:45.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.426+0000 7fe67ffff640 1 -- 172.21.15.98:0/267100670 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe680146e10 msgr2=0x7fe68015e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:45.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.430+0000 7fe67ffff640 1 --2- 172.21.15.98:0/267100670 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe680146e10 0x7fe68015e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:45.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.430+0000 7fe67ffff640 1 -- 172.21.15.98:0/267100670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6801529d0 msgr2=0x7fe680152e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:45.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.430+0000 7fe67ffff640 1 --2- 172.21.15.98:0/267100670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6801529d0 0x7fe680152e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:45.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.430+0000 7fe67ffff640 1 -- 172.21.15.98:0/267100670 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe680147290 con 0x7fe668004d50 2024-09-19T08:46:45.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.430+0000 7fe67f7fe640 1 --2- 172.21.15.98:0/267100670 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe680146e10 0x7fe68015e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:45.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.430+0000 7fe67ffff640 1 --2- 172.21.15.98:0/267100670 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe668004d50 0x7fe68015c540 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fe670011610 tx=0x7fe670011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:45.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.430+0000 7fe67cff9640 1 -- 172.21.15.98:0/267100670 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe670015870 con 0x7fe668004d50 2024-09-19T08:46:45.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.430+0000 7fe67cff9640 1 -- 172.21.15.98:0/267100670 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe670015a10 con 0x7fe668004d50 2024-09-19T08:46:45.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.430+0000 7fe6863a1640 1 -- 172.21.15.98:0/267100670 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe68015e7b0 con 0x7fe668004d50 2024-09-19T08:46:45.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.430+0000 7fe67cff9640 1 -- 172.21.15.98:0/267100670 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe670015dc0 con 0x7fe668004d50 2024-09-19T08:46:45.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.430+0000 7fe6863a1640 1 -- 172.21.15.98:0/267100670 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe68015ce00 con 0x7fe668004d50 2024-09-19T08:46:45.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.430+0000 7fe6863a1640 1 -- 172.21.15.98:0/267100670 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe68015eba0 con 0x7fe668004d50 2024-09-19T08:46:45.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.430+0000 7fe67cff9640 1 -- 172.21.15.98:0/267100670 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7fe67000f480 con 0x7fe668004d50 2024-09-19T08:46:45.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.430+0000 7fe67cff9640 1 --2- 172.21.15.98:0/267100670 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe66c0680d0 0x7fe66c06a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:45.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.430+0000 7fe67cff9640 1 -- 172.21.15.98:0/267100670 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5737+0+0 (secure 0 0 0) 0x7fe670084f50 con 0x7fe668004d50 2024-09-19T08:46:45.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.430+0000 7fe67f7fe640 1 --2- 172.21.15.98:0/267100670 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe66c0680d0 0x7fe66c06a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:45.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.430+0000 7fe67f7fe640 1 --2- 172.21.15.98:0/267100670 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe66c0680d0 0x7fe66c06a590 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe680153d80 tx=0x7fe674009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:45.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.434+0000 7fe67cff9640 1 -- 172.21.15.98:0/267100670 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe670019030 con 0x7fe668004d50 2024-09-19T08:46:45.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.646+0000 7fe6863a1640 1 -- 172.21.15.98:0/267100670 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fe68006e980 con 0x7fe668004d50 2024-09-19T08:46:45.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.646+0000 7fe67cff9640 1 -- 172.21.15.98:0/267100670 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1478 (secure 0 0 0) 0x7fe670057380 con 0x7fe668004d50 2024-09-19T08:46:45.646 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:46:45.646 INFO:teuthology.orchestra.run.smithi098.stdout:{"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":530,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":158,"num_osds":8,"num_up_osds":8,"osd_up_since":1726735088,"num_in_osds":8,"osd_in_since":1726735077,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":281178112,"bytes_avail":772812935168,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":91,"btime":"2024-09-19T08:46:43:523818+0000","id":12,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":12,"rank":0,"name":"c","status":"up:active","gid":8766}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":15,"modified":"2024-09-19T08:46:28.942581+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-19T08:46:45.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.650+0000 7fe6863a1640 1 -- 172.21.15.98:0/267100670 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe66c0680d0 msgr2=0x7fe66c06a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:45.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.650+0000 7fe6863a1640 1 --2- 172.21.15.98:0/267100670 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe66c0680d0 0x7fe66c06a590 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe680153d80 tx=0x7fe674009040 comp rx=0 tx=0).stop 2024-09-19T08:46:45.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.650+0000 7fe6863a1640 1 -- 172.21.15.98:0/267100670 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe668004d50 msgr2=0x7fe68015c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:45.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.650+0000 7fe6863a1640 1 --2- 172.21.15.98:0/267100670 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe668004d50 0x7fe68015c540 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fe670011610 tx=0x7fe670011ad0 comp rx=0 tx=0).stop 2024-09-19T08:46:45.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.650+0000 7fe6863a1640 1 -- 172.21.15.98:0/267100670 shutdown_connections 2024-09-19T08:46:45.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.650+0000 7fe6863a1640 1 --2- 172.21.15.98:0/267100670 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe66c0680d0 0x7fe66c06a590 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:45.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.650+0000 7fe6863a1640 1 --2- 172.21.15.98:0/267100670 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe6801529d0 0x7fe680152e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:45.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.650+0000 7fe6863a1640 1 --2- 172.21.15.98:0/267100670 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe680146e10 0x7fe68015e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:45.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.650+0000 7fe6863a1640 1 --2- 172.21.15.98:0/267100670 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe668004d50 0x7fe68015c540 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:45.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.650+0000 7fe6863a1640 1 -- 172.21.15.98:0/267100670 >> 172.21.15.98:0/267100670 conn(0x7fe6800b3500 msgr2=0x7fe68006e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:45.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.650+0000 7fe6863a1640 1 -- 172.21.15.98:0/267100670 shutdown_connections 2024-09-19T08:46:45.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.650+0000 7fe6863a1640 1 -- 172.21.15.98:0/267100670 wait complete. 2024-09-19T08:46:45.663 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-19T08:46:45.664 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:46:45.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.782+0000 7f8230af6640 1 Processor -- start 2024-09-19T08:46:45.783 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.782+0000 7f8230af6640 1 -- start start 2024-09-19T08:46:45.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.786+0000 7f8230af6640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f822c0d4820 0x7f822c1621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:45.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.786+0000 7f8230af6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f822c0b9970 0x7f822c0b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:45.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.786+0000 7f8230af6640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f822c068440 con 0x7f822c0b7f80 2024-09-19T08:46:45.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.786+0000 7f8230af6640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f822c163050 con 0x7f822c0d4820 2024-09-19T08:46:45.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.786+0000 7f8230af6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f822c162a90 con 0x7f822c0b9970 2024-09-19T08:46:45.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.786+0000 7f8229d9b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f822c0d4820 0x7f822c1621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:45.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.786+0000 7f822a59c640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f822c0b7f80 0x7f822c15f560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:55040/0 (socket says 172.21.15.98:55040) 2024-09-19T08:46:45.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.786+0000 7f822a59c640 1 -- 172.21.15.98:0/3100556156 learned_addr learned my addr 172.21.15.98:0/3100556156 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:45.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.786+0000 7f822ad9d640 1 --2- 172.21.15.98:0/3100556156 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f822c0b9970 0x7f822c0b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:45.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.786+0000 7f8229d9b640 1 -- 172.21.15.98:0/3100556156 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f822c0b9970 msgr2=0x7f822c0b9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:45.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.786+0000 7f8229d9b640 1 --2- 172.21.15.98:0/3100556156 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f822c0b9970 0x7f822c0b9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:45.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.786+0000 7f8229d9b640 1 -- 172.21.15.98:0/3100556156 >> v1:172.21.15.98:6789/0 conn(0x7f822c0b7f80 legacy=0x7f822c15f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:45.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.786+0000 7f822959a640 1 -- 172.21.15.98:0/3100556156 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1385646080 0 0) 0x7f822c068440 con 0x7f822c0b7f80 2024-09-19T08:46:45.786 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.786+0000 7f8229d9b640 1 -- 172.21.15.98:0/3100556156 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f822c162d70 con 0x7f822c0d4820 2024-09-19T08:46:45.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.786+0000 7f8229d9b640 1 --2- 172.21.15.98:0/3100556156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f822c0d4820 0x7f822c1621b0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f82140097c0 tx=0x7f821401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=515045f2150067f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:45.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.786+0000 7f822959a640 1 -- 172.21.15.98:0/3100556156 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82140b1070 con 0x7f822c0d4820 2024-09-19T08:46:45.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.786+0000 7f822959a640 1 -- 172.21.15.98:0/3100556156 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82140ac410 con 0x7f822c0d4820 2024-09-19T08:46:45.787 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.786+0000 7f822959a640 1 -- 172.21.15.98:0/3100556156 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82140ac7e0 con 0x7f822c0d4820 2024-09-19T08:46:45.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.786+0000 7f8230af6640 1 -- 172.21.15.98:0/3100556156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f822c0d4820 msgr2=0x7f822c1621b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:45.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.786+0000 7f8230af6640 1 --2- 172.21.15.98:0/3100556156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f822c0d4820 0x7f822c1621b0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f82140097c0 tx=0x7f821401c4a0 comp rx=0 tx=0).stop 2024-09-19T08:46:45.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.786+0000 7f8230af6640 1 -- 172.21.15.98:0/3100556156 shutdown_connections 2024-09-19T08:46:45.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.786+0000 7f8230af6640 1 --2- 172.21.15.98:0/3100556156 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f822c0b9970 0x7f822c0b9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:45.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.786+0000 7f8230af6640 1 --2- 172.21.15.98:0/3100556156 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f822c0d4820 0x7f822c1621b0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:45.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.786+0000 7f8230af6640 1 -- 172.21.15.98:0/3100556156 >> 172.21.15.98:0/3100556156 conn(0x7f822c074e40 msgr2=0x7f822c075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:45.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.786+0000 7f8230af6640 1 -- 172.21.15.98:0/3100556156 shutdown_connections 2024-09-19T08:46:45.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.786+0000 7f8230af6640 1 -- 172.21.15.98:0/3100556156 wait complete. 2024-09-19T08:46:45.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.790+0000 7f8230af6640 1 Processor -- start 2024-09-19T08:46:45.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.790+0000 7f8230af6640 1 -- start start 2024-09-19T08:46:45.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.790+0000 7f8230af6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f822c0b7f80 0x7f822c1606d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:45.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.790+0000 7f8230af6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f822c0b9970 0x7f822c15fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:45.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.790+0000 7f8230af6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f822c0d4820 0x7f822c13f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:45.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.790+0000 7f8230af6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f822c0b84e0 con 0x7f822c0b9970 2024-09-19T08:46:45.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.790+0000 7f8230af6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f822c0b8360 con 0x7f822c0b7f80 2024-09-19T08:46:45.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.790+0000 7f8230af6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f822c144590 con 0x7f822c0d4820 2024-09-19T08:46:45.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.790+0000 7f8229d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f822c0b9970 0x7f822c15fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:45.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.790+0000 7f8229d9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f822c0b9970 0x7f822c15fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54446/0 (socket says 172.21.15.98:54446) 2024-09-19T08:46:45.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.790+0000 7f8229d9b640 1 -- 172.21.15.98:0/4089010465 learned_addr learned my addr 172.21.15.98:0/4089010465 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:45.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.790+0000 7f822a59c640 1 --2- 172.21.15.98:0/4089010465 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f822c0b7f80 0x7f822c1606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:45.791 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.790+0000 7f8229d9b640 1 -- 172.21.15.98:0/4089010465 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f822c0d4820 msgr2=0x7f822c13f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:45.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.790+0000 7f822ad9d640 1 --2- 172.21.15.98:0/4089010465 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f822c0d4820 0x7f822c13f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:45.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.790+0000 7f8229d9b640 1 --2- 172.21.15.98:0/4089010465 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f822c0d4820 0x7f822c13f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:45.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.790+0000 7f8229d9b640 1 -- 172.21.15.98:0/4089010465 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f822c0b7f80 msgr2=0x7f822c1606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:45.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.790+0000 7f8229d9b640 1 --2- 172.21.15.98:0/4089010465 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f822c0b7f80 0x7f822c1606d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:45.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.790+0000 7f8229d9b640 1 -- 172.21.15.98:0/4089010465 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f822c160e40 con 0x7f822c0b9970 2024-09-19T08:46:45.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.790+0000 7f822a59c640 1 --2- 172.21.15.98:0/4089010465 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f822c0b7f80 0x7f822c1606d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:45.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.790+0000 7f8229d9b640 1 --2- 172.21.15.98:0/4089010465 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f822c0b9970 0x7f822c15fe90 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f82140af040 tx=0x7f8214019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:45.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.790+0000 7f8222ffd640 1 -- 172.21.15.98:0/4089010465 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82140bb070 con 0x7f822c0b9970 2024-09-19T08:46:45.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.790+0000 7f8222ffd640 1 -- 172.21.15.98:0/4089010465 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82140ace20 con 0x7f822c0b9970 2024-09-19T08:46:45.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.790+0000 7f8230af6640 1 -- 172.21.15.98:0/4089010465 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f822c161120 con 0x7f822c0b9970 2024-09-19T08:46:45.792 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.790+0000 7f8222ffd640 1 -- 172.21.15.98:0/4089010465 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82140b1040 con 0x7f822c0b9970 2024-09-19T08:46:45.793 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.790+0000 7f8230af6640 1 -- 172.21.15.98:0/4089010465 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f822c1603d0 con 0x7f822c0b9970 2024-09-19T08:46:45.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.790+0000 7f8230af6640 1 -- 172.21.15.98:0/4089010465 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f822c068f50 con 0x7f822c0b9970 2024-09-19T08:46:45.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.794+0000 7f8222ffd640 1 -- 172.21.15.98:0/4089010465 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f8214004220 con 0x7f822c0b9970 2024-09-19T08:46:45.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.794+0000 7f8222ffd640 1 --2- 172.21.15.98:0/4089010465 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f820c0680a0 0x7f820c06a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:45.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.794+0000 7f8222ffd640 1 -- 172.21.15.98:0/4089010465 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5737+0+0 (secure 0 0 0) 0x7f821411e8c0 con 0x7f822c0b9970 2024-09-19T08:46:45.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.794+0000 7f822a59c640 1 --2- 172.21.15.98:0/4089010465 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f820c0680a0 0x7f820c06a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:45.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.794+0000 7f822a59c640 1 --2- 172.21.15.98:0/4089010465 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f820c0680a0 0x7f820c06a560 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f821c000f70 tx=0x7f821c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:45.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.798+0000 7f8222ffd640 1 -- 172.21.15.98:0/4089010465 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f8214016090 con 0x7f822c0b9970 2024-09-19T08:46:45.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.958+0000 7f8230af6640 1 -- 172.21.15.98:0/4089010465 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f822c141f70 con 0x7f820c0680a0 2024-09-19T08:46:45.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.974+0000 7f8222ffd640 1 -- 172.21.15.98:0/4089010465 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252484 (secure 0 0 0) 0x7f822c141f70 con 0x7f820c0680a0 2024-09-19T08:46:45.978 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:46:45.980 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:46:45.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.982+0000 7f8230af6640 1 -- 172.21.15.98:0/4089010465 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f820c0680a0 msgr2=0x7f820c06a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:45.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.982+0000 7f8230af6640 1 --2- 172.21.15.98:0/4089010465 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f820c0680a0 0x7f820c06a560 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f821c000f70 tx=0x7f821c009040 comp rx=0 tx=0).stop 2024-09-19T08:46:45.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.982+0000 7f8230af6640 1 -- 172.21.15.98:0/4089010465 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f822c0b9970 msgr2=0x7f822c15fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:45.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.982+0000 7f8230af6640 1 --2- 172.21.15.98:0/4089010465 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f822c0b9970 0x7f822c15fe90 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f82140af040 tx=0x7f8214019240 comp rx=0 tx=0).stop 2024-09-19T08:46:45.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.982+0000 7f8230af6640 1 -- 172.21.15.98:0/4089010465 shutdown_connections 2024-09-19T08:46:45.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.982+0000 7f8230af6640 1 --2- 172.21.15.98:0/4089010465 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f820c0680a0 0x7f820c06a560 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:45.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.982+0000 7f8230af6640 1 --2- 172.21.15.98:0/4089010465 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f822c0d4820 0x7f822c13f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:45.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.982+0000 7f8230af6640 1 --2- 172.21.15.98:0/4089010465 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f822c0b9970 0x7f822c15fe90 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:45.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.982+0000 7f8230af6640 1 --2- 172.21.15.98:0/4089010465 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f822c0b7f80 0x7f822c1606d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:45.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.982+0000 7f8230af6640 1 -- 172.21.15.98:0/4089010465 >> 172.21.15.98:0/4089010465 conn(0x7f822c074e40 msgr2=0x7f822c142bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:45.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.982+0000 7f8230af6640 1 -- 172.21.15.98:0/4089010465 shutdown_connections 2024-09-19T08:46:45.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:45.982+0000 7f8230af6640 1 -- 172.21.15.98:0/4089010465 wait complete. 2024-09-19T08:46:45.997 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":202,"stamp":"2024-09-19T08:46:44.947364+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":159,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":274588,"kb_used_data":16712,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754700132,"statfs":{"total":773094113280,"available":772812935168,"internally_reserved":0,"allocated":17113088,"data_stored":8233868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001975"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.300021+0000","last_change":"2024-09-19T08:46:40.322807+0000","last_active":"2024-09-19T08:46:41.300021+0000","last_peered":"2024-09-19T08:46:41.300021+0000","last_clean":"2024-09-19T08:46:41.300021+0000","last_became_active":"2024-09-19T08:46:40.322530+0000","last_became_peered":"2024-09-19T08:46:40.322530+0000","last_unstale":"2024-09-19T08:46:41.300021+0000","last_undegraded":"2024-09-19T08:46:41.300021+0000","last_fullsized":"2024-09-19T08:46:41.300021+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:14:04.162980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.275876+0000","last_change":"2024-09-19T08:46:40.308583+0000","last_active":"2024-09-19T08:46:41.275876+0000","last_peered":"2024-09-19T08:46:41.275876+0000","last_clean":"2024-09-19T08:46:41.275876+0000","last_became_active":"2024-09-19T08:46:40.307894+0000","last_became_peered":"2024-09-19T08:46:40.307894+0000","last_unstale":"2024-09-19T08:46:41.275876+0000","last_undegraded":"2024-09-19T08:46:41.275876+0000","last_fullsized":"2024-09-19T08:46:41.275876+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:48.398635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.294222+0000","last_change":"2024-09-19T08:46:40.305267+0000","last_active":"2024-09-19T08:46:41.294222+0000","last_peered":"2024-09-19T08:46:41.294222+0000","last_clean":"2024-09-19T08:46:41.294222+0000","last_became_active":"2024-09-19T08:46:40.304553+0000","last_became_peered":"2024-09-19T08:46:40.304553+0000","last_unstale":"2024-09-19T08:46:41.294222+0000","last_undegraded":"2024-09-19T08:46:41.294222+0000","last_fullsized":"2024-09-19T08:46:41.294222+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:49.743970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.275107+0000","last_change":"2024-09-19T08:46:40.307985+0000","last_active":"2024-09-19T08:46:41.275107+0000","last_peered":"2024-09-19T08:46:41.275107+0000","last_clean":"2024-09-19T08:46:41.275107+0000","last_became_active":"2024-09-19T08:46:40.307321+0000","last_became_peered":"2024-09-19T08:46:40.307321+0000","last_unstale":"2024-09-19T08:46:41.275107+0000","last_undegraded":"2024-09-19T08:46:41.275107+0000","last_fullsized":"2024-09-19T08:46:41.275107+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:37:44.052309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271115+0000","last_change":"2024-09-19T08:46:40.313691+0000","last_active":"2024-09-19T08:46:41.271115+0000","last_peered":"2024-09-19T08:46:41.271115+0000","last_clean":"2024-09-19T08:46:41.271115+0000","last_became_active":"2024-09-19T08:46:40.313152+0000","last_became_peered":"2024-09-19T08:46:40.313152+0000","last_unstale":"2024-09-19T08:46:41.271115+0000","last_undegraded":"2024-09-19T08:46:41.271115+0000","last_fullsized":"2024-09-19T08:46:41.271115+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:17:30.627741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.293469+0000","last_change":"2024-09-19T08:46:40.310239+0000","last_active":"2024-09-19T08:46:41.293469+0000","last_peered":"2024-09-19T08:46:41.293469+0000","last_clean":"2024-09-19T08:46:41.293469+0000","last_became_active":"2024-09-19T08:46:40.309610+0000","last_became_peered":"2024-09-19T08:46:40.309610+0000","last_unstale":"2024-09-19T08:46:41.293469+0000","last_undegraded":"2024-09-19T08:46:41.293469+0000","last_fullsized":"2024-09-19T08:46:41.293469+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:10.797977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.297686+0000","last_change":"2024-09-19T08:46:40.300397+0000","last_active":"2024-09-19T08:46:41.297686+0000","last_peered":"2024-09-19T08:46:41.297686+0000","last_clean":"2024-09-19T08:46:41.297686+0000","last_became_active":"2024-09-19T08:46:40.299734+0000","last_became_peered":"2024-09-19T08:46:40.299734+0000","last_unstale":"2024-09-19T08:46:41.297686+0000","last_undegraded":"2024-09-19T08:46:41.297686+0000","last_fullsized":"2024-09-19T08:46:41.297686+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:20.271146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295020+0000","last_change":"2024-09-19T08:46:40.282539+0000","last_active":"2024-09-19T08:46:41.295020+0000","last_peered":"2024-09-19T08:46:41.295020+0000","last_clean":"2024-09-19T08:46:41.295020+0000","last_became_active":"2024-09-19T08:46:40.281719+0000","last_became_peered":"2024-09-19T08:46:40.281719+0000","last_unstale":"2024-09-19T08:46:41.295020+0000","last_undegraded":"2024-09-19T08:46:41.295020+0000","last_fullsized":"2024-09-19T08:46:41.295020+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:54:49.652718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.302625+0000","last_change":"2024-09-19T08:46:40.313125+0000","last_active":"2024-09-19T08:46:41.302625+0000","last_peered":"2024-09-19T08:46:41.302625+0000","last_clean":"2024-09-19T08:46:41.302625+0000","last_became_active":"2024-09-19T08:46:40.312593+0000","last_became_peered":"2024-09-19T08:46:40.312593+0000","last_unstale":"2024-09-19T08:46:41.302625+0000","last_undegraded":"2024-09-19T08:46:41.302625+0000","last_fullsized":"2024-09-19T08:46:41.302625+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:18.762035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295826+0000","last_change":"2024-09-19T08:46:40.299010+0000","last_active":"2024-09-19T08:46:41.295826+0000","last_peered":"2024-09-19T08:46:41.295826+0000","last_clean":"2024-09-19T08:46:41.295826+0000","last_became_active":"2024-09-19T08:46:40.298263+0000","last_became_peered":"2024-09-19T08:46:40.298263+0000","last_unstale":"2024-09-19T08:46:41.295826+0000","last_undegraded":"2024-09-19T08:46:41.295826+0000","last_fullsized":"2024-09-19T08:46:41.295826+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:29:29.482503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.298285+0000","last_change":"2024-09-19T08:46:40.312068+0000","last_active":"2024-09-19T08:46:41.298285+0000","last_peered":"2024-09-19T08:46:41.298285+0000","last_clean":"2024-09-19T08:46:41.298285+0000","last_became_active":"2024-09-19T08:46:40.311698+0000","last_became_peered":"2024-09-19T08:46:40.311698+0000","last_unstale":"2024-09-19T08:46:41.298285+0000","last_undegraded":"2024-09-19T08:46:41.298285+0000","last_fullsized":"2024-09-19T08:46:41.298285+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:37:24.320667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.275730+0000","last_change":"2024-09-19T08:46:40.318436+0000","last_active":"2024-09-19T08:46:41.275730+0000","last_peered":"2024-09-19T08:46:41.275730+0000","last_clean":"2024-09-19T08:46:41.275730+0000","last_became_active":"2024-09-19T08:46:40.318012+0000","last_became_peered":"2024-09-19T08:46:40.318012+0000","last_unstale":"2024-09-19T08:46:41.275730+0000","last_undegraded":"2024-09-19T08:46:41.275730+0000","last_fullsized":"2024-09-19T08:46:41.275730+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:08.849222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.274801+0000","last_change":"2024-09-19T08:46:40.291508+0000","last_active":"2024-09-19T08:46:41.274801+0000","last_peered":"2024-09-19T08:46:41.274801+0000","last_clean":"2024-09-19T08:46:41.274801+0000","last_became_active":"2024-09-19T08:46:40.291252+0000","last_became_peered":"2024-09-19T08:46:40.291252+0000","last_unstale":"2024-09-19T08:46:41.274801+0000","last_undegraded":"2024-09-19T08:46:41.274801+0000","last_fullsized":"2024-09-19T08:46:41.274801+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:42.984066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.302365+0000","last_change":"2024-09-19T08:46:40.318237+0000","last_active":"2024-09-19T08:46:41.302365+0000","last_peered":"2024-09-19T08:46:41.302365+0000","last_clean":"2024-09-19T08:46:41.302365+0000","last_became_active":"2024-09-19T08:46:40.317621+0000","last_became_peered":"2024-09-19T08:46:40.317621+0000","last_unstale":"2024-09-19T08:46:41.302365+0000","last_undegraded":"2024-09-19T08:46:41.302365+0000","last_fullsized":"2024-09-19T08:46:41.302365+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:09.679777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.275004+0000","last_change":"2024-09-19T08:46:40.295471+0000","last_active":"2024-09-19T08:46:41.275004+0000","last_peered":"2024-09-19T08:46:41.275004+0000","last_clean":"2024-09-19T08:46:41.275004+0000","last_became_active":"2024-09-19T08:46:40.294889+0000","last_became_peered":"2024-09-19T08:46:40.294889+0000","last_unstale":"2024-09-19T08:46:41.275004+0000","last_undegraded":"2024-09-19T08:46:41.275004+0000","last_fullsized":"2024-09-19T08:46:41.275004+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:44.098158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295327+0000","last_change":"2024-09-19T08:46:40.306178+0000","last_active":"2024-09-19T08:46:41.295327+0000","last_peered":"2024-09-19T08:46:41.295327+0000","last_clean":"2024-09-19T08:46:41.295327+0000","last_became_active":"2024-09-19T08:46:40.304073+0000","last_became_peered":"2024-09-19T08:46:40.304073+0000","last_unstale":"2024-09-19T08:46:41.295327+0000","last_undegraded":"2024-09-19T08:46:41.295327+0000","last_fullsized":"2024-09-19T08:46:41.295327+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:22.830837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271822+0000","last_change":"2024-09-19T08:46:40.287586+0000","last_active":"2024-09-19T08:46:41.271822+0000","last_peered":"2024-09-19T08:46:41.271822+0000","last_clean":"2024-09-19T08:46:41.271822+0000","last_became_active":"2024-09-19T08:46:40.286802+0000","last_became_peered":"2024-09-19T08:46:40.286802+0000","last_unstale":"2024-09-19T08:46:41.271822+0000","last_undegraded":"2024-09-19T08:46:41.271822+0000","last_fullsized":"2024-09-19T08:46:41.271822+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:53:20.598758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.299555+0000","last_change":"2024-09-19T08:46:40.325550+0000","last_active":"2024-09-19T08:46:41.299555+0000","last_peered":"2024-09-19T08:46:41.299555+0000","last_clean":"2024-09-19T08:46:41.299555+0000","last_became_active":"2024-09-19T08:46:40.325309+0000","last_became_peered":"2024-09-19T08:46:40.325309+0000","last_unstale":"2024-09-19T08:46:41.299555+0000","last_undegraded":"2024-09-19T08:46:41.299555+0000","last_fullsized":"2024-09-19T08:46:41.299555+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:21:31.832739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295824+0000","last_change":"2024-09-19T08:46:40.312961+0000","last_active":"2024-09-19T08:46:41.295824+0000","last_peered":"2024-09-19T08:46:41.295824+0000","last_clean":"2024-09-19T08:46:41.295824+0000","last_became_active":"2024-09-19T08:46:40.312097+0000","last_became_peered":"2024-09-19T08:46:40.312097+0000","last_unstale":"2024-09-19T08:46:41.295824+0000","last_undegraded":"2024-09-19T08:46:41.295824+0000","last_fullsized":"2024-09-19T08:46:41.295824+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:58:38.046334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271499+0000","last_change":"2024-09-19T08:46:40.286511+0000","last_active":"2024-09-19T08:46:41.271499+0000","last_peered":"2024-09-19T08:46:41.271499+0000","last_clean":"2024-09-19T08:46:41.271499+0000","last_became_active":"2024-09-19T08:46:40.286045+0000","last_became_peered":"2024-09-19T08:46:40.286045+0000","last_unstale":"2024-09-19T08:46:41.271499+0000","last_undegraded":"2024-09-19T08:46:41.271499+0000","last_fullsized":"2024-09-19T08:46:41.271499+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:48.932583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271520+0000","last_change":"2024-09-19T08:46:40.312590+0000","last_active":"2024-09-19T08:46:41.271520+0000","last_peered":"2024-09-19T08:46:41.271520+0000","last_clean":"2024-09-19T08:46:41.271520+0000","last_became_active":"2024-09-19T08:46:40.312010+0000","last_became_peered":"2024-09-19T08:46:40.312010+0000","last_unstale":"2024-09-19T08:46:41.271520+0000","last_undegraded":"2024-09-19T08:46:41.271520+0000","last_fullsized":"2024-09-19T08:46:41.271520+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:17:58.016895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.294442+0000","last_change":"2024-09-19T08:46:40.314682+0000","last_active":"2024-09-19T08:46:41.294442+0000","last_peered":"2024-09-19T08:46:41.294442+0000","last_clean":"2024-09-19T08:46:41.294442+0000","last_became_active":"2024-09-19T08:46:40.313976+0000","last_became_peered":"2024-09-19T08:46:40.313976+0000","last_unstale":"2024-09-19T08:46:41.294442+0000","last_undegraded":"2024-09-19T08:46:41.294442+0000","last_fullsized":"2024-09-19T08:46:41.294442+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:27:03.146587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.293441+0000","last_change":"2024-09-19T08:46:40.300547+0000","last_active":"2024-09-19T08:46:41.293441+0000","last_peered":"2024-09-19T08:46:41.293441+0000","last_clean":"2024-09-19T08:46:41.293441+0000","last_became_active":"2024-09-19T08:46:40.299848+0000","last_became_peered":"2024-09-19T08:46:40.299848+0000","last_unstale":"2024-09-19T08:46:41.293441+0000","last_undegraded":"2024-09-19T08:46:41.293441+0000","last_fullsized":"2024-09-19T08:46:41.293441+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:29:23.072767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295172+0000","last_change":"2024-09-19T08:46:40.296781+0000","last_active":"2024-09-19T08:46:41.295172+0000","last_peered":"2024-09-19T08:46:41.295172+0000","last_clean":"2024-09-19T08:46:41.295172+0000","last_became_active":"2024-09-19T08:46:40.296159+0000","last_became_peered":"2024-09-19T08:46:40.296159+0000","last_unstale":"2024-09-19T08:46:41.295172+0000","last_undegraded":"2024-09-19T08:46:41.295172+0000","last_fullsized":"2024-09-19T08:46:41.295172+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:36:31.828168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.275725+0000","last_change":"2024-09-19T08:46:40.311172+0000","last_active":"2024-09-19T08:46:41.275725+0000","last_peered":"2024-09-19T08:46:41.275725+0000","last_clean":"2024-09-19T08:46:41.275725+0000","last_became_active":"2024-09-19T08:46:40.310768+0000","last_became_peered":"2024-09-19T08:46:40.310768+0000","last_unstale":"2024-09-19T08:46:41.275725+0000","last_undegraded":"2024-09-19T08:46:41.275725+0000","last_fullsized":"2024-09-19T08:46:41.275725+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:26:01.059827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.298510+0000","last_change":"2024-09-19T08:46:40.304105+0000","last_active":"2024-09-19T08:46:41.298510+0000","last_peered":"2024-09-19T08:46:41.298510+0000","last_clean":"2024-09-19T08:46:41.298510+0000","last_became_active":"2024-09-19T08:46:40.303618+0000","last_became_peered":"2024-09-19T08:46:40.303618+0000","last_unstale":"2024-09-19T08:46:41.298510+0000","last_undegraded":"2024-09-19T08:46:41.298510+0000","last_fullsized":"2024-09-19T08:46:41.298510+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:15:05.241341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.294336+0000","last_change":"2024-09-19T08:46:40.310327+0000","last_active":"2024-09-19T08:46:41.294336+0000","last_peered":"2024-09-19T08:46:41.294336+0000","last_clean":"2024-09-19T08:46:41.294336+0000","last_became_active":"2024-09-19T08:46:40.309561+0000","last_became_peered":"2024-09-19T08:46:40.309561+0000","last_unstale":"2024-09-19T08:46:41.294336+0000","last_undegraded":"2024-09-19T08:46:41.294336+0000","last_fullsized":"2024-09-19T08:46:41.294336+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:39:18.317103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271831+0000","last_change":"2024-09-19T08:46:40.297612+0000","last_active":"2024-09-19T08:46:41.271831+0000","last_peered":"2024-09-19T08:46:41.271831+0000","last_clean":"2024-09-19T08:46:41.271831+0000","last_became_active":"2024-09-19T08:46:40.296522+0000","last_became_peered":"2024-09-19T08:46:40.296522+0000","last_unstale":"2024-09-19T08:46:41.271831+0000","last_undegraded":"2024-09-19T08:46:41.271831+0000","last_fullsized":"2024-09-19T08:46:41.271831+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:19:50.929185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.275232+0000","last_change":"2024-09-19T08:46:40.319198+0000","last_active":"2024-09-19T08:46:41.275232+0000","last_peered":"2024-09-19T08:46:41.275232+0000","last_clean":"2024-09-19T08:46:41.275232+0000","last_became_active":"2024-09-19T08:46:40.318778+0000","last_became_peered":"2024-09-19T08:46:40.318778+0000","last_unstale":"2024-09-19T08:46:41.275232+0000","last_undegraded":"2024-09-19T08:46:41.275232+0000","last_fullsized":"2024-09-19T08:46:41.275232+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:17:12.366818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.301582+0000","last_change":"2024-09-19T08:46:40.324681+0000","last_active":"2024-09-19T08:46:41.301582+0000","last_peered":"2024-09-19T08:46:41.301582+0000","last_clean":"2024-09-19T08:46:41.301582+0000","last_became_active":"2024-09-19T08:46:40.321740+0000","last_became_peered":"2024-09-19T08:46:40.321740+0000","last_unstale":"2024-09-19T08:46:41.301582+0000","last_undegraded":"2024-09-19T08:46:41.301582+0000","last_fullsized":"2024-09-19T08:46:41.301582+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:44:45.487714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.293584+0000","last_change":"2024-09-19T08:46:40.295308+0000","last_active":"2024-09-19T08:46:41.293584+0000","last_peered":"2024-09-19T08:46:41.293584+0000","last_clean":"2024-09-19T08:46:41.293584+0000","last_became_active":"2024-09-19T08:46:40.290389+0000","last_became_peered":"2024-09-19T08:46:40.290389+0000","last_unstale":"2024-09-19T08:46:41.293584+0000","last_undegraded":"2024-09-19T08:46:41.293584+0000","last_fullsized":"2024-09-19T08:46:41.293584+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:09:25.970759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271075+0000","last_change":"2024-09-19T08:46:40.287160+0000","last_active":"2024-09-19T08:46:41.271075+0000","last_peered":"2024-09-19T08:46:41.271075+0000","last_clean":"2024-09-19T08:46:41.271075+0000","last_became_active":"2024-09-19T08:46:40.286383+0000","last_became_peered":"2024-09-19T08:46:40.286383+0000","last_unstale":"2024-09-19T08:46:41.271075+0000","last_undegraded":"2024-09-19T08:46:41.271075+0000","last_fullsized":"2024-09-19T08:46:41.271075+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:56.945612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.296321+0000","last_change":"2024-09-19T08:46:40.305212+0000","last_active":"2024-09-19T08:46:41.296321+0000","last_peered":"2024-09-19T08:46:41.296321+0000","last_clean":"2024-09-19T08:46:41.296321+0000","last_became_active":"2024-09-19T08:46:40.304643+0000","last_became_peered":"2024-09-19T08:46:40.304643+0000","last_unstale":"2024-09-19T08:46:41.296321+0000","last_undegraded":"2024-09-19T08:46:41.296321+0000","last_fullsized":"2024-09-19T08:46:41.296321+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:57:56.539906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"157'4","reported_seq":18,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271785+0000","last_change":"2024-09-19T08:46:39.533871+0000","last_active":"2024-09-19T08:46:41.271785+0000","last_peered":"2024-09-19T08:46:41.271785+0000","last_clean":"2024-09-19T08:46:41.271785+0000","last_became_active":"2024-09-19T08:46:39.533575+0000","last_became_peered":"2024-09-19T08:46:39.533575+0000","last_unstale":"2024-09-19T08:46:41.271785+0000","last_undegraded":"2024-09-19T08:46:41.271785+0000","last_fullsized":"2024-09-19T08:46:41.271785+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:36:08.646357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.296566+0000","last_change":"2024-09-19T08:46:40.299343+0000","last_active":"2024-09-19T08:46:41.296566+0000","last_peered":"2024-09-19T08:46:41.296566+0000","last_clean":"2024-09-19T08:46:41.296566+0000","last_became_active":"2024-09-19T08:46:40.296360+0000","last_became_peered":"2024-09-19T08:46:40.296360+0000","last_unstale":"2024-09-19T08:46:41.296566+0000","last_undegraded":"2024-09-19T08:46:41.296566+0000","last_fullsized":"2024-09-19T08:46:41.296566+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:27:03.803525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"157'1","reported_seq":15,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.302393+0000","last_change":"2024-09-19T08:46:39.524090+0000","last_active":"2024-09-19T08:46:41.302393+0000","last_peered":"2024-09-19T08:46:41.302393+0000","last_clean":"2024-09-19T08:46:41.302393+0000","last_became_active":"2024-09-19T08:46:39.523389+0000","last_became_peered":"2024-09-19T08:46:39.523389+0000","last_unstale":"2024-09-19T08:46:41.302393+0000","last_undegraded":"2024-09-19T08:46:41.302393+0000","last_fullsized":"2024-09-19T08:46:41.302393+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:54:57.317359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.299684+0000","last_change":"2024-09-19T08:46:40.287661+0000","last_active":"2024-09-19T08:46:41.299684+0000","last_peered":"2024-09-19T08:46:41.299684+0000","last_clean":"2024-09-19T08:46:41.299684+0000","last_became_active":"2024-09-19T08:46:40.287397+0000","last_became_peered":"2024-09-19T08:46:40.287397+0000","last_unstale":"2024-09-19T08:46:41.299684+0000","last_undegraded":"2024-09-19T08:46:41.299684+0000","last_fullsized":"2024-09-19T08:46:41.299684+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:15.351548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"157'2","reported_seq":16,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.294377+0000","last_change":"2024-09-19T08:46:39.517378+0000","last_active":"2024-09-19T08:46:41.294377+0000","last_peered":"2024-09-19T08:46:41.294377+0000","last_clean":"2024-09-19T08:46:41.294377+0000","last_became_active":"2024-09-19T08:46:39.517112+0000","last_became_peered":"2024-09-19T08:46:39.517112+0000","last_unstale":"2024-09-19T08:46:41.294377+0000","last_undegraded":"2024-09-19T08:46:41.294377+0000","last_fullsized":"2024-09-19T08:46:41.294377+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:21:46.481801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.302427+0000","last_change":"2024-09-19T08:46:40.305888+0000","last_active":"2024-09-19T08:46:41.302427+0000","last_peered":"2024-09-19T08:46:41.302427+0000","last_clean":"2024-09-19T08:46:41.302427+0000","last_became_active":"2024-09-19T08:46:40.305283+0000","last_became_peered":"2024-09-19T08:46:40.305283+0000","last_unstale":"2024-09-19T08:46:41.302427+0000","last_undegraded":"2024-09-19T08:46:41.302427+0000","last_fullsized":"2024-09-19T08:46:41.302427+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:15.659241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"157'2","reported_seq":16,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.275270+0000","last_change":"2024-09-19T08:46:39.525313+0000","last_active":"2024-09-19T08:46:41.275270+0000","last_peered":"2024-09-19T08:46:41.275270+0000","last_clean":"2024-09-19T08:46:41.275270+0000","last_became_active":"2024-09-19T08:46:39.524576+0000","last_became_peered":"2024-09-19T08:46:39.524576+0000","last_unstale":"2024-09-19T08:46:41.275270+0000","last_undegraded":"2024-09-19T08:46:41.275270+0000","last_fullsized":"2024-09-19T08:46:41.275270+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:57.140465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.294535+0000","last_change":"2024-09-19T08:46:40.311346+0000","last_active":"2024-09-19T08:46:41.294535+0000","last_peered":"2024-09-19T08:46:41.294535+0000","last_clean":"2024-09-19T08:46:41.294535+0000","last_became_active":"2024-09-19T08:46:40.310733+0000","last_became_peered":"2024-09-19T08:46:40.310733+0000","last_unstale":"2024-09-19T08:46:41.294535+0000","last_undegraded":"2024-09-19T08:46:41.294535+0000","last_fullsized":"2024-09-19T08:46:41.294535+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:36:50.649569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"157'1","reported_seq":15,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271232+0000","last_change":"2024-09-19T08:46:39.526200+0000","last_active":"2024-09-19T08:46:41.271232+0000","last_peered":"2024-09-19T08:46:41.271232+0000","last_clean":"2024-09-19T08:46:41.271232+0000","last_became_active":"2024-09-19T08:46:39.525883+0000","last_became_peered":"2024-09-19T08:46:39.525883+0000","last_unstale":"2024-09-19T08:46:41.271232+0000","last_undegraded":"2024-09-19T08:46:41.271232+0000","last_fullsized":"2024-09-19T08:46:41.271232+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:52.132068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295513+0000","last_change":"2024-09-19T08:46:40.305229+0000","last_active":"2024-09-19T08:46:41.295513+0000","last_peered":"2024-09-19T08:46:41.295513+0000","last_clean":"2024-09-19T08:46:41.295513+0000","last_became_active":"2024-09-19T08:46:40.304492+0000","last_became_peered":"2024-09-19T08:46:40.304492+0000","last_unstale":"2024-09-19T08:46:41.295513+0000","last_undegraded":"2024-09-19T08:46:41.295513+0000","last_fullsized":"2024-09-19T08:46:41.295513+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:32.712559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"157'1","reported_seq":15,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295463+0000","last_change":"2024-09-19T08:46:39.519337+0000","last_active":"2024-09-19T08:46:41.295463+0000","last_peered":"2024-09-19T08:46:41.295463+0000","last_clean":"2024-09-19T08:46:41.295463+0000","last_became_active":"2024-09-19T08:46:39.518792+0000","last_became_peered":"2024-09-19T08:46:39.518792+0000","last_unstale":"2024-09-19T08:46:41.295463+0000","last_undegraded":"2024-09-19T08:46:41.295463+0000","last_fullsized":"2024-09-19T08:46:41.295463+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:30:05.550797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.297805+0000","last_change":"2024-09-19T08:46:40.311303+0000","last_active":"2024-09-19T08:46:41.297805+0000","last_peered":"2024-09-19T08:46:41.297805+0000","last_clean":"2024-09-19T08:46:41.297805+0000","last_became_active":"2024-09-19T08:46:40.310999+0000","last_became_peered":"2024-09-19T08:46:40.310999+0000","last_unstale":"2024-09-19T08:46:41.297805+0000","last_undegraded":"2024-09-19T08:46:41.297805+0000","last_fullsized":"2024-09-19T08:46:41.297805+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:49.743970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.298406+0000","last_change":"2024-09-19T08:46:39.532881+0000","last_active":"2024-09-19T08:46:41.298406+0000","last_peered":"2024-09-19T08:46:41.298406+0000","last_clean":"2024-09-19T08:46:41.298406+0000","last_became_active":"2024-09-19T08:46:39.532112+0000","last_became_peered":"2024-09-19T08:46:39.532112+0000","last_unstale":"2024-09-19T08:46:41.298406+0000","last_undegraded":"2024-09-19T08:46:41.298406+0000","last_fullsized":"2024-09-19T08:46:41.298406+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:32.581664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.294399+0000","last_change":"2024-09-19T08:46:40.296034+0000","last_active":"2024-09-19T08:46:41.294399+0000","last_peered":"2024-09-19T08:46:41.294399+0000","last_clean":"2024-09-19T08:46:41.294399+0000","last_became_active":"2024-09-19T08:46:40.294878+0000","last_became_peered":"2024-09-19T08:46:40.294878+0000","last_unstale":"2024-09-19T08:46:41.294399+0000","last_undegraded":"2024-09-19T08:46:41.294399+0000","last_fullsized":"2024-09-19T08:46:41.294399+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:44.855570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.299039+0000","last_change":"2024-09-19T08:46:39.540052+0000","last_active":"2024-09-19T08:46:41.299039+0000","last_peered":"2024-09-19T08:46:41.299039+0000","last_clean":"2024-09-19T08:46:41.299039+0000","last_became_active":"2024-09-19T08:46:39.539378+0000","last_became_peered":"2024-09-19T08:46:39.539378+0000","last_unstale":"2024-09-19T08:46:41.299039+0000","last_undegraded":"2024-09-19T08:46:41.299039+0000","last_fullsized":"2024-09-19T08:46:41.299039+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:06.592079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.296558+0000","last_change":"2024-09-19T08:46:40.302139+0000","last_active":"2024-09-19T08:46:41.296558+0000","last_peered":"2024-09-19T08:46:41.296558+0000","last_clean":"2024-09-19T08:46:41.296558+0000","last_became_active":"2024-09-19T08:46:40.301363+0000","last_became_peered":"2024-09-19T08:46:40.301363+0000","last_unstale":"2024-09-19T08:46:41.296558+0000","last_undegraded":"2024-09-19T08:46:41.296558+0000","last_fullsized":"2024-09-19T08:46:41.296558+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:13.914441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271475+0000","last_change":"2024-09-19T08:46:39.529992+0000","last_active":"2024-09-19T08:46:41.271475+0000","last_peered":"2024-09-19T08:46:41.271475+0000","last_clean":"2024-09-19T08:46:41.271475+0000","last_became_active":"2024-09-19T08:46:39.529500+0000","last_became_peered":"2024-09-19T08:46:39.529500+0000","last_unstale":"2024-09-19T08:46:41.271475+0000","last_undegraded":"2024-09-19T08:46:41.271475+0000","last_fullsized":"2024-09-19T08:46:41.271475+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:34.276215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.298137+0000","last_change":"2024-09-19T08:46:40.306814+0000","last_active":"2024-09-19T08:46:41.298137+0000","last_peered":"2024-09-19T08:46:41.298137+0000","last_clean":"2024-09-19T08:46:41.298137+0000","last_became_active":"2024-09-19T08:46:40.306279+0000","last_became_peered":"2024-09-19T08:46:40.306279+0000","last_unstale":"2024-09-19T08:46:41.298137+0000","last_undegraded":"2024-09-19T08:46:41.298137+0000","last_fullsized":"2024-09-19T08:46:41.298137+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:45.155900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"157'3","reported_seq":17,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271615+0000","last_change":"2024-09-19T08:46:39.530600+0000","last_active":"2024-09-19T08:46:41.271615+0000","last_peered":"2024-09-19T08:46:41.271615+0000","last_clean":"2024-09-19T08:46:41.271615+0000","last_became_active":"2024-09-19T08:46:39.526884+0000","last_became_peered":"2024-09-19T08:46:39.526884+0000","last_unstale":"2024-09-19T08:46:41.271615+0000","last_undegraded":"2024-09-19T08:46:41.271615+0000","last_fullsized":"2024-09-19T08:46:41.271615+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:01:13.258546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295715+0000","last_change":"2024-09-19T08:46:40.310166+0000","last_active":"2024-09-19T08:46:41.295715+0000","last_peered":"2024-09-19T08:46:41.295715+0000","last_clean":"2024-09-19T08:46:41.295715+0000","last_became_active":"2024-09-19T08:46:40.309720+0000","last_became_peered":"2024-09-19T08:46:40.309720+0000","last_unstale":"2024-09-19T08:46:41.295715+0000","last_undegraded":"2024-09-19T08:46:41.295715+0000","last_fullsized":"2024-09-19T08:46:41.295715+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:53:13.406407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"157'6","reported_seq":20,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.294318+0000","last_change":"2024-09-19T08:46:39.533705+0000","last_active":"2024-09-19T08:46:41.294318+0000","last_peered":"2024-09-19T08:46:41.294318+0000","last_clean":"2024-09-19T08:46:41.294318+0000","last_became_active":"2024-09-19T08:46:39.533388+0000","last_became_peered":"2024-09-19T08:46:39.533388+0000","last_unstale":"2024-09-19T08:46:41.294318+0000","last_undegraded":"2024-09-19T08:46:41.294318+0000","last_fullsized":"2024-09-19T08:46:41.294318+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:29.674750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.293672+0000","last_change":"2024-09-19T08:46:40.298398+0000","last_active":"2024-09-19T08:46:41.293672+0000","last_peered":"2024-09-19T08:46:41.293672+0000","last_clean":"2024-09-19T08:46:41.293672+0000","last_became_active":"2024-09-19T08:46:40.297708+0000","last_became_peered":"2024-09-19T08:46:40.297708+0000","last_unstale":"2024-09-19T08:46:41.293672+0000","last_undegraded":"2024-09-19T08:46:41.293672+0000","last_fullsized":"2024-09-19T08:46:41.293672+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:20.271146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"157'2","reported_seq":16,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.298905+0000","last_change":"2024-09-19T08:46:39.532544+0000","last_active":"2024-09-19T08:46:41.298905+0000","last_peered":"2024-09-19T08:46:41.298905+0000","last_clean":"2024-09-19T08:46:41.298905+0000","last_became_active":"2024-09-19T08:46:39.531951+0000","last_became_peered":"2024-09-19T08:46:39.531951+0000","last_unstale":"2024-09-19T08:46:41.298905+0000","last_undegraded":"2024-09-19T08:46:41.298905+0000","last_fullsized":"2024-09-19T08:46:41.298905+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:44:23.781566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.297970+0000","last_change":"2024-09-19T08:46:40.302762+0000","last_active":"2024-09-19T08:46:41.297970+0000","last_peered":"2024-09-19T08:46:41.297970+0000","last_clean":"2024-09-19T08:46:41.297970+0000","last_became_active":"2024-09-19T08:46:40.301495+0000","last_became_peered":"2024-09-19T08:46:40.301495+0000","last_unstale":"2024-09-19T08:46:41.297970+0000","last_undegraded":"2024-09-19T08:46:41.297970+0000","last_fullsized":"2024-09-19T08:46:41.297970+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:39:18.317103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.294654+0000","last_change":"2024-09-19T08:46:39.523177+0000","last_active":"2024-09-19T08:46:41.294654+0000","last_peered":"2024-09-19T08:46:41.294654+0000","last_clean":"2024-09-19T08:46:41.294654+0000","last_became_active":"2024-09-19T08:46:39.522823+0000","last_became_peered":"2024-09-19T08:46:39.522823+0000","last_unstale":"2024-09-19T08:46:41.294654+0000","last_undegraded":"2024-09-19T08:46:41.294654+0000","last_fullsized":"2024-09-19T08:46:41.294654+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:32.581664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.301407+0000","last_change":"2024-09-19T08:46:40.319863+0000","last_active":"2024-09-19T08:46:41.301407+0000","last_peered":"2024-09-19T08:46:41.301407+0000","last_clean":"2024-09-19T08:46:41.301407+0000","last_became_active":"2024-09-19T08:46:40.319407+0000","last_became_peered":"2024-09-19T08:46:40.319407+0000","last_unstale":"2024-09-19T08:46:41.301407+0000","last_undegraded":"2024-09-19T08:46:41.301407+0000","last_fullsized":"2024-09-19T08:46:41.301407+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:34.896081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295262+0000","last_change":"2024-09-19T08:46:39.528911+0000","last_active":"2024-09-19T08:46:41.295262+0000","last_peered":"2024-09-19T08:46:41.295262+0000","last_clean":"2024-09-19T08:46:41.295262+0000","last_became_active":"2024-09-19T08:46:39.528519+0000","last_became_peered":"2024-09-19T08:46:39.528519+0000","last_unstale":"2024-09-19T08:46:41.295262+0000","last_undegraded":"2024-09-19T08:46:41.295262+0000","last_fullsized":"2024-09-19T08:46:41.295262+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:05.280493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.294063+0000","last_change":"2024-09-19T08:46:40.290329+0000","last_active":"2024-09-19T08:46:41.294063+0000","last_peered":"2024-09-19T08:46:41.294063+0000","last_clean":"2024-09-19T08:46:41.294063+0000","last_became_active":"2024-09-19T08:46:40.289851+0000","last_became_peered":"2024-09-19T08:46:40.289851+0000","last_unstale":"2024-09-19T08:46:41.294063+0000","last_undegraded":"2024-09-19T08:46:41.294063+0000","last_fullsized":"2024-09-19T08:46:41.294063+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:36:51.887939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"157'8","reported_seq":22,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.298681+0000","last_change":"2024-09-19T08:46:39.534632+0000","last_active":"2024-09-19T08:46:41.298681+0000","last_peered":"2024-09-19T08:46:41.298681+0000","last_clean":"2024-09-19T08:46:41.298681+0000","last_became_active":"2024-09-19T08:46:39.533906+0000","last_became_peered":"2024-09-19T08:46:39.533906+0000","last_unstale":"2024-09-19T08:46:41.298681+0000","last_undegraded":"2024-09-19T08:46:41.298681+0000","last_fullsized":"2024-09-19T08:46:41.298681+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:42.284571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.296092+0000","last_change":"2024-09-19T08:46:40.294805+0000","last_active":"2024-09-19T08:46:41.296092+0000","last_peered":"2024-09-19T08:46:41.296092+0000","last_clean":"2024-09-19T08:46:41.296092+0000","last_became_active":"2024-09-19T08:46:40.294338+0000","last_became_peered":"2024-09-19T08:46:40.294338+0000","last_unstale":"2024-09-19T08:46:41.296092+0000","last_undegraded":"2024-09-19T08:46:41.296092+0000","last_fullsized":"2024-09-19T08:46:41.296092+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:10:19.078157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"157'1","reported_seq":15,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271881+0000","last_change":"2024-09-19T08:46:39.529025+0000","last_active":"2024-09-19T08:46:41.271881+0000","last_peered":"2024-09-19T08:46:41.271881+0000","last_clean":"2024-09-19T08:46:41.271881+0000","last_became_active":"2024-09-19T08:46:39.526531+0000","last_became_peered":"2024-09-19T08:46:39.526531+0000","last_unstale":"2024-09-19T08:46:41.271881+0000","last_undegraded":"2024-09-19T08:46:41.271881+0000","last_fullsized":"2024-09-19T08:46:41.271881+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:18:57.366963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.302806+0000","last_change":"2024-09-19T08:46:40.320361+0000","last_active":"2024-09-19T08:46:41.302806+0000","last_peered":"2024-09-19T08:46:41.302806+0000","last_clean":"2024-09-19T08:46:41.302806+0000","last_became_active":"2024-09-19T08:46:40.319637+0000","last_became_peered":"2024-09-19T08:46:40.319637+0000","last_unstale":"2024-09-19T08:46:41.302806+0000","last_undegraded":"2024-09-19T08:46:41.302806+0000","last_fullsized":"2024-09-19T08:46:41.302806+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:09:14.940629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"157'2","reported_seq":16,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.296238+0000","last_change":"2024-09-19T08:46:39.528693+0000","last_active":"2024-09-19T08:46:41.296238+0000","last_peered":"2024-09-19T08:46:41.296238+0000","last_clean":"2024-09-19T08:46:41.296238+0000","last_became_active":"2024-09-19T08:46:39.528063+0000","last_became_peered":"2024-09-19T08:46:39.528063+0000","last_unstale":"2024-09-19T08:46:41.296238+0000","last_undegraded":"2024-09-19T08:46:41.296238+0000","last_fullsized":"2024-09-19T08:46:41.296238+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:04:47.931376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.301874+0000","last_change":"2024-09-19T08:46:40.324664+0000","last_active":"2024-09-19T08:46:41.301874+0000","last_peered":"2024-09-19T08:46:41.301874+0000","last_clean":"2024-09-19T08:46:41.301874+0000","last_became_active":"2024-09-19T08:46:40.321357+0000","last_became_peered":"2024-09-19T08:46:40.321357+0000","last_unstale":"2024-09-19T08:46:41.301874+0000","last_undegraded":"2024-09-19T08:46:41.301874+0000","last_fullsized":"2024-09-19T08:46:41.301874+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:59.869941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"157'1","reported_seq":15,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.299335+0000","last_change":"2024-09-19T08:46:39.534210+0000","last_active":"2024-09-19T08:46:41.299335+0000","last_peered":"2024-09-19T08:46:41.299335+0000","last_clean":"2024-09-19T08:46:41.299335+0000","last_became_active":"2024-09-19T08:46:39.533548+0000","last_became_peered":"2024-09-19T08:46:39.533548+0000","last_unstale":"2024-09-19T08:46:41.299335+0000","last_undegraded":"2024-09-19T08:46:41.299335+0000","last_fullsized":"2024-09-19T08:46:41.299335+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:50.291503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.275460+0000","last_change":"2024-09-19T08:46:40.308301+0000","last_active":"2024-09-19T08:46:41.275460+0000","last_peered":"2024-09-19T08:46:41.275460+0000","last_clean":"2024-09-19T08:46:41.275460+0000","last_became_active":"2024-09-19T08:46:40.307793+0000","last_became_peered":"2024-09-19T08:46:40.307793+0000","last_unstale":"2024-09-19T08:46:41.275460+0000","last_undegraded":"2024-09-19T08:46:41.275460+0000","last_fullsized":"2024-09-19T08:46:41.275460+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:21.144985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.298624+0000","last_change":"2024-09-19T08:46:39.536189+0000","last_active":"2024-09-19T08:46:41.298624+0000","last_peered":"2024-09-19T08:46:41.298624+0000","last_clean":"2024-09-19T08:46:41.298624+0000","last_became_active":"2024-09-19T08:46:39.535387+0000","last_became_peered":"2024-09-19T08:46:39.535387+0000","last_unstale":"2024-09-19T08:46:41.298624+0000","last_undegraded":"2024-09-19T08:46:41.298624+0000","last_fullsized":"2024-09-19T08:46:41.298624+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:00:43.262958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.294622+0000","last_change":"2024-09-19T08:46:40.308240+0000","last_active":"2024-09-19T08:46:41.294622+0000","last_peered":"2024-09-19T08:46:41.294622+0000","last_clean":"2024-09-19T08:46:41.294622+0000","last_became_active":"2024-09-19T08:46:40.307131+0000","last_became_peered":"2024-09-19T08:46:40.307131+0000","last_unstale":"2024-09-19T08:46:41.294622+0000","last_undegraded":"2024-09-19T08:46:41.294622+0000","last_fullsized":"2024-09-19T08:46:41.294622+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:40:32.674934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"157'2","reported_seq":16,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.293923+0000","last_change":"2024-09-19T08:46:39.529933+0000","last_active":"2024-09-19T08:46:41.293923+0000","last_peered":"2024-09-19T08:46:41.293923+0000","last_clean":"2024-09-19T08:46:41.293923+0000","last_became_active":"2024-09-19T08:46:39.529602+0000","last_became_peered":"2024-09-19T08:46:39.529602+0000","last_unstale":"2024-09-19T08:46:41.293923+0000","last_undegraded":"2024-09-19T08:46:41.293923+0000","last_fullsized":"2024-09-19T08:46:41.293923+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:45:17.113767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.298023+0000","last_change":"2024-09-19T08:46:40.278369+0000","last_active":"2024-09-19T08:46:41.298023+0000","last_peered":"2024-09-19T08:46:41.298023+0000","last_clean":"2024-09-19T08:46:41.298023+0000","last_became_active":"2024-09-19T08:46:40.277767+0000","last_became_peered":"2024-09-19T08:46:40.277767+0000","last_unstale":"2024-09-19T08:46:41.298023+0000","last_undegraded":"2024-09-19T08:46:41.298023+0000","last_fullsized":"2024-09-19T08:46:41.298023+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:44.855570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271109+0000","last_change":"2024-09-19T08:46:39.526194+0000","last_active":"2024-09-19T08:46:41.271109+0000","last_peered":"2024-09-19T08:46:41.271109+0000","last_clean":"2024-09-19T08:46:41.271109+0000","last_became_active":"2024-09-19T08:46:39.525582+0000","last_became_peered":"2024-09-19T08:46:39.525582+0000","last_unstale":"2024-09-19T08:46:41.271109+0000","last_undegraded":"2024-09-19T08:46:41.271109+0000","last_fullsized":"2024-09-19T08:46:41.271109+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:54.940787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.274690+0000","last_change":"2024-09-19T08:46:40.311768+0000","last_active":"2024-09-19T08:46:41.274690+0000","last_peered":"2024-09-19T08:46:41.274690+0000","last_clean":"2024-09-19T08:46:41.274690+0000","last_became_active":"2024-09-19T08:46:40.311464+0000","last_became_peered":"2024-09-19T08:46:40.311464+0000","last_unstale":"2024-09-19T08:46:41.274690+0000","last_undegraded":"2024-09-19T08:46:41.274690+0000","last_fullsized":"2024-09-19T08:46:41.274690+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:08:48.768860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.293578+0000","last_change":"2024-09-19T08:46:39.531582+0000","last_active":"2024-09-19T08:46:41.293578+0000","last_peered":"2024-09-19T08:46:41.293578+0000","last_clean":"2024-09-19T08:46:41.293578+0000","last_became_active":"2024-09-19T08:46:39.531166+0000","last_became_peered":"2024-09-19T08:46:39.531166+0000","last_unstale":"2024-09-19T08:46:41.293578+0000","last_undegraded":"2024-09-19T08:46:41.293578+0000","last_fullsized":"2024-09-19T08:46:41.293578+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:45:20.818342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271332+0000","last_change":"2024-09-19T08:46:40.299301+0000","last_active":"2024-09-19T08:46:41.271332+0000","last_peered":"2024-09-19T08:46:41.271332+0000","last_clean":"2024-09-19T08:46:41.271332+0000","last_became_active":"2024-09-19T08:46:40.298235+0000","last_became_peered":"2024-09-19T08:46:40.298235+0000","last_unstale":"2024-09-19T08:46:41.271332+0000","last_undegraded":"2024-09-19T08:46:41.271332+0000","last_fullsized":"2024-09-19T08:46:41.271332+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:36:59.594448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.301286+0000","last_change":"2024-09-19T08:46:39.529718+0000","last_active":"2024-09-19T08:46:41.301286+0000","last_peered":"2024-09-19T08:46:41.301286+0000","last_clean":"2024-09-19T08:46:41.301286+0000","last_became_active":"2024-09-19T08:46:39.529000+0000","last_became_peered":"2024-09-19T08:46:39.529000+0000","last_unstale":"2024-09-19T08:46:41.301286+0000","last_undegraded":"2024-09-19T08:46:41.301286+0000","last_fullsized":"2024-09-19T08:46:41.301286+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:47.923282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271607+0000","last_change":"2024-09-19T08:46:40.296371+0000","last_active":"2024-09-19T08:46:41.271607+0000","last_peered":"2024-09-19T08:46:41.271607+0000","last_clean":"2024-09-19T08:46:41.271607+0000","last_became_active":"2024-09-19T08:46:40.295860+0000","last_became_peered":"2024-09-19T08:46:40.295860+0000","last_unstale":"2024-09-19T08:46:41.271607+0000","last_undegraded":"2024-09-19T08:46:41.271607+0000","last_fullsized":"2024-09-19T08:46:41.271607+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:54:58.326660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"157'2","reported_seq":16,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.299767+0000","last_change":"2024-09-19T08:46:39.533469+0000","last_active":"2024-09-19T08:46:41.299767+0000","last_peered":"2024-09-19T08:46:41.299767+0000","last_clean":"2024-09-19T08:46:41.299767+0000","last_became_active":"2024-09-19T08:46:39.532381+0000","last_became_peered":"2024-09-19T08:46:39.532381+0000","last_unstale":"2024-09-19T08:46:41.299767+0000","last_undegraded":"2024-09-19T08:46:41.299767+0000","last_fullsized":"2024-09-19T08:46:41.299767+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:38.871312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.296434+0000","last_change":"2024-09-19T08:46:40.316938+0000","last_active":"2024-09-19T08:46:41.296434+0000","last_peered":"2024-09-19T08:46:41.296434+0000","last_clean":"2024-09-19T08:46:41.296434+0000","last_became_active":"2024-09-19T08:46:40.316355+0000","last_became_peered":"2024-09-19T08:46:40.316355+0000","last_unstale":"2024-09-19T08:46:41.296434+0000","last_undegraded":"2024-09-19T08:46:41.296434+0000","last_fullsized":"2024-09-19T08:46:41.296434+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:20:14.319040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"158'1","reported_seq":15,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.296024+0000","last_change":"2024-09-19T08:46:39.517540+0000","last_active":"2024-09-19T08:46:41.296024+0000","last_peered":"2024-09-19T08:46:41.296024+0000","last_clean":"2024-09-19T08:46:41.296024+0000","last_became_active":"2024-09-19T08:46:39.517247+0000","last_became_peered":"2024-09-19T08:46:39.517247+0000","last_unstale":"2024-09-19T08:46:41.296024+0000","last_undegraded":"2024-09-19T08:46:41.296024+0000","last_fullsized":"2024-09-19T08:46:41.296024+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:42.770912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295742+0000","last_change":"2024-09-19T08:46:40.292589+0000","last_active":"2024-09-19T08:46:41.295742+0000","last_peered":"2024-09-19T08:46:41.295742+0000","last_clean":"2024-09-19T08:46:41.295742+0000","last_became_active":"2024-09-19T08:46:40.292129+0000","last_became_peered":"2024-09-19T08:46:40.292129+0000","last_unstale":"2024-09-19T08:46:41.295742+0000","last_undegraded":"2024-09-19T08:46:41.295742+0000","last_fullsized":"2024-09-19T08:46:41.295742+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:04.629751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295780+0000","last_change":"2024-09-19T08:46:39.517207+0000","last_active":"2024-09-19T08:46:41.295780+0000","last_peered":"2024-09-19T08:46:41.295780+0000","last_clean":"2024-09-19T08:46:41.295780+0000","last_became_active":"2024-09-19T08:46:39.516924+0000","last_became_peered":"2024-09-19T08:46:39.516924+0000","last_unstale":"2024-09-19T08:46:41.295780+0000","last_undegraded":"2024-09-19T08:46:41.295780+0000","last_fullsized":"2024-09-19T08:46:41.295780+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:55:04.131994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295871+0000","last_change":"2024-09-19T08:46:40.313527+0000","last_active":"2024-09-19T08:46:41.295871+0000","last_peered":"2024-09-19T08:46:41.295871+0000","last_clean":"2024-09-19T08:46:41.295871+0000","last_became_active":"2024-09-19T08:46:40.312733+0000","last_became_peered":"2024-09-19T08:46:40.312733+0000","last_unstale":"2024-09-19T08:46:41.295871+0000","last_undegraded":"2024-09-19T08:46:41.295871+0000","last_fullsized":"2024-09-19T08:46:41.295871+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:30.018530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295815+0000","last_change":"2024-09-19T08:46:39.533782+0000","last_active":"2024-09-19T08:46:41.295815+0000","last_peered":"2024-09-19T08:46:41.295815+0000","last_clean":"2024-09-19T08:46:41.295815+0000","last_became_active":"2024-09-19T08:46:39.533401+0000","last_became_peered":"2024-09-19T08:46:39.533401+0000","last_unstale":"2024-09-19T08:46:41.295815+0000","last_undegraded":"2024-09-19T08:46:41.295815+0000","last_fullsized":"2024-09-19T08:46:41.295815+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:14:49.514714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.298822+0000","last_change":"2024-09-19T08:46:40.317873+0000","last_active":"2024-09-19T08:46:41.298822+0000","last_peered":"2024-09-19T08:46:41.298822+0000","last_clean":"2024-09-19T08:46:41.298822+0000","last_became_active":"2024-09-19T08:46:40.316906+0000","last_became_peered":"2024-09-19T08:46:40.316906+0000","last_unstale":"2024-09-19T08:46:41.298822+0000","last_undegraded":"2024-09-19T08:46:41.298822+0000","last_fullsized":"2024-09-19T08:46:41.298822+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:08.299341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.301923+0000","last_change":"2024-09-19T08:46:39.521611+0000","last_active":"2024-09-19T08:46:41.301923+0000","last_peered":"2024-09-19T08:46:41.301923+0000","last_clean":"2024-09-19T08:46:41.301923+0000","last_became_active":"2024-09-19T08:46:39.520863+0000","last_became_peered":"2024-09-19T08:46:39.520863+0000","last_unstale":"2024-09-19T08:46:41.301923+0000","last_undegraded":"2024-09-19T08:46:41.301923+0000","last_fullsized":"2024-09-19T08:46:41.301923+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:53:19.589458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.298638+0000","last_change":"2024-09-19T08:46:40.317455+0000","last_active":"2024-09-19T08:46:41.298638+0000","last_peered":"2024-09-19T08:46:41.298638+0000","last_clean":"2024-09-19T08:46:41.298638+0000","last_became_active":"2024-09-19T08:46:40.317109+0000","last_became_peered":"2024-09-19T08:46:40.317109+0000","last_unstale":"2024-09-19T08:46:41.298638+0000","last_undegraded":"2024-09-19T08:46:41.298638+0000","last_fullsized":"2024-09-19T08:46:41.298638+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:06.068010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.301669+0000","last_change":"2024-09-19T08:46:39.536563+0000","last_active":"2024-09-19T08:46:41.301669+0000","last_peered":"2024-09-19T08:46:41.301669+0000","last_clean":"2024-09-19T08:46:41.301669+0000","last_became_active":"2024-09-19T08:46:39.535725+0000","last_became_peered":"2024-09-19T08:46:39.535725+0000","last_unstale":"2024-09-19T08:46:41.301669+0000","last_undegraded":"2024-09-19T08:46:41.301669+0000","last_fullsized":"2024-09-19T08:46:41.301669+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:17:57.007595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.274677+0000","last_change":"2024-09-19T08:46:40.300496+0000","last_active":"2024-09-19T08:46:41.274677+0000","last_peered":"2024-09-19T08:46:41.274677+0000","last_clean":"2024-09-19T08:46:41.274677+0000","last_became_active":"2024-09-19T08:46:40.299999+0000","last_became_peered":"2024-09-19T08:46:40.299999+0000","last_unstale":"2024-09-19T08:46:41.274677+0000","last_undegraded":"2024-09-19T08:46:41.274677+0000","last_fullsized":"2024-09-19T08:46:41.274677+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:57:03.311516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.274968+0000","last_change":"2024-09-19T08:46:39.539941+0000","last_active":"2024-09-19T08:46:41.274968+0000","last_peered":"2024-09-19T08:46:41.274968+0000","last_clean":"2024-09-19T08:46:41.274968+0000","last_became_active":"2024-09-19T08:46:39.539225+0000","last_became_peered":"2024-09-19T08:46:39.539225+0000","last_unstale":"2024-09-19T08:46:41.274968+0000","last_undegraded":"2024-09-19T08:46:41.274968+0000","last_fullsized":"2024-09-19T08:46:41.274968+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:46:06.121419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271306+0000","last_change":"2024-09-19T08:46:40.293254+0000","last_active":"2024-09-19T08:46:41.271306+0000","last_peered":"2024-09-19T08:46:41.271306+0000","last_clean":"2024-09-19T08:46:41.271306+0000","last_became_active":"2024-09-19T08:46:40.292460+0000","last_became_peered":"2024-09-19T08:46:40.292460+0000","last_unstale":"2024-09-19T08:46:41.271306+0000","last_undegraded":"2024-09-19T08:46:41.271306+0000","last_fullsized":"2024-09-19T08:46:41.271306+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:36:48.171899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.293216+0000","last_change":"2024-09-19T08:46:39.529973+0000","last_active":"2024-09-19T08:46:41.293216+0000","last_peered":"2024-09-19T08:46:41.293216+0000","last_clean":"2024-09-19T08:46:41.293216+0000","last_became_active":"2024-09-19T08:46:39.529691+0000","last_became_peered":"2024-09-19T08:46:39.529691+0000","last_unstale":"2024-09-19T08:46:41.293216+0000","last_undegraded":"2024-09-19T08:46:41.293216+0000","last_fullsized":"2024-09-19T08:46:41.293216+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:34.575229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.293890+0000","last_change":"2024-09-19T08:46:40.292254+0000","last_active":"2024-09-19T08:46:41.293890+0000","last_peered":"2024-09-19T08:46:41.293890+0000","last_clean":"2024-09-19T08:46:41.293890+0000","last_became_active":"2024-09-19T08:46:40.290492+0000","last_became_peered":"2024-09-19T08:46:40.290492+0000","last_unstale":"2024-09-19T08:46:41.293890+0000","last_undegraded":"2024-09-19T08:46:41.293890+0000","last_fullsized":"2024-09-19T08:46:41.293890+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:14:04.162980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.298903+0000","last_change":"2024-09-19T08:46:39.527908+0000","last_active":"2024-09-19T08:46:41.298903+0000","last_peered":"2024-09-19T08:46:41.298903+0000","last_clean":"2024-09-19T08:46:41.298903+0000","last_became_active":"2024-09-19T08:46:39.527338+0000","last_became_peered":"2024-09-19T08:46:39.527338+0000","last_unstale":"2024-09-19T08:46:41.298903+0000","last_undegraded":"2024-09-19T08:46:41.298903+0000","last_fullsized":"2024-09-19T08:46:41.298903+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:42.770912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"71'128","reported_seq":457,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.298973+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:46:41.298973+0000","last_peered":"2024-09-19T08:46:41.298973+0000","last_clean":"2024-09-19T08:46:41.298973+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:46:41.298973+0000","last_undegraded":"2024-09-19T08:46:41.298973+0000","last_fullsized":"2024-09-19T08:46:41.298973+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509546,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33724,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338116,"statfs":{"total":96636764160,"available":96602230784,"internally_reserved":0,"allocated":2584576,"data_stored":1471511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574955,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33128,"kb_used_data":1936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338712,"statfs":{"total":96636764160,"available":96602841088,"internally_reserved":0,"allocated":1982464,"data_stored":881039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509548,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33136,"kb_used_data":1936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338704,"statfs":{"total":96636764160,"available":96602832896,"internally_reserved":0,"allocated":1982464,"data_stored":881039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509548,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33140,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338700,"statfs":{"total":96636764160,"available":96602828800,"internally_reserved":0,"allocated":1998848,"data_stored":883233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542250,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37296,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334544,"statfs":{"total":96636764160,"available":96598573056,"internally_reserved":0,"allocated":1994752,"data_stored":881233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509545,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33144,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338696,"statfs":{"total":96636764160,"available":96602824704,"internally_reserved":0,"allocated":1994752,"data_stored":881733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509547,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33144,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338696,"statfs":{"total":96636764160,"available":96602824704,"internally_reserved":0,"allocated":1994752,"data_stored":882651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509545,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37876,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333964,"statfs":{"total":96636764160,"available":96597979136,"internally_reserved":0,"allocated":2580480,"data_stored":1471429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:46:46.480 INFO:tasks.ceph.mds.c.smithi098.stderr:2024-09-19T08:46:46.482+0000 7fab4e5b7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T08:46:48.608 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:46:48.611 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:46:49.002 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:46:49.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.134+0000 7f5f069c8640 1 Processor -- start 2024-09-19T08:46:49.133 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.134+0000 7f5f069c8640 1 -- start start 2024-09-19T08:46:49.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.134+0000 7f5f069c8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f000d4820 0x7f5f001621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:49.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.134+0000 7f5f069c8640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5f000b9970 0x7f5f000b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:49.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.134+0000 7f5f069c8640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f00068440 con 0x7f5f000b7f80 2024-09-19T08:46:49.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.134+0000 7f5f069c8640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f5f00163050 con 0x7f5f000b9970 2024-09-19T08:46:49.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.134+0000 7f5f069c8640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5f000ba360 con 0x7f5f000d4820 2024-09-19T08:46:49.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.134+0000 7f5f04f65640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5f000b9970 0x7f5f000b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:49.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.134+0000 7f5f04f65640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5f000b9970 0x7f5f000b9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54282/0 (socket says 172.21.15.98:54282) 2024-09-19T08:46:49.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.134+0000 7f5f04f65640 1 -- 172.21.15.98:0/2554605833 learned_addr learned my addr 172.21.15.98:0/2554605833 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:49.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.134+0000 7f5eff7fe640 1 --2- 172.21.15.98:0/2554605833 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f000d4820 0x7f5f001621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:49.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.134+0000 7f5f04f65640 1 -- 172.21.15.98:0/2554605833 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f000d4820 msgr2=0x7f5f001621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:49.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.134+0000 7f5f04f65640 1 --2- 172.21.15.98:0/2554605833 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f000d4820 0x7f5f001621b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:49.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.134+0000 7f5f04f65640 1 -- 172.21.15.98:0/2554605833 >> v1:172.21.15.139:6789/0 conn(0x7f5f000b7f80 legacy=0x7f5f0015f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:49.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.134+0000 7f5f04f65640 1 -- 172.21.15.98:0/2554605833 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f00162a90 con 0x7f5f000b9970 2024-09-19T08:46:49.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.138+0000 7f5f04f65640 1 --2- 172.21.15.98:0/2554605833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f000b9970 0x7f5f000b9e20 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8009e50 tx=0x7f5ef801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f3d5ad78d3a4324 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:49.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.138+0000 7f5efeffd640 1 -- 172.21.15.98:0/2554605833 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef80a57f0 con 0x7f5f000b9970 2024-09-19T08:46:49.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.138+0000 7f5efeffd640 1 -- 172.21.15.98:0/2554605833 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ef80a5990 con 0x7f5f000b9970 2024-09-19T08:46:49.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.138+0000 7f5efeffd640 1 -- 172.21.15.98:0/2554605833 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef80a5d60 con 0x7f5f000b9970 2024-09-19T08:46:49.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.138+0000 7f5f069c8640 1 -- 172.21.15.98:0/2554605833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f000b9970 msgr2=0x7f5f000b9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:49.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.138+0000 7f5f069c8640 1 --2- 172.21.15.98:0/2554605833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f000b9970 0x7f5f000b9e20 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8009e50 tx=0x7f5ef801ca30 comp rx=0 tx=0).stop 2024-09-19T08:46:49.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.138+0000 7f5f069c8640 1 -- 172.21.15.98:0/2554605833 shutdown_connections 2024-09-19T08:46:49.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.138+0000 7f5f069c8640 1 --2- 172.21.15.98:0/2554605833 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f000b9970 0x7f5f000b9e20 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:49.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.138+0000 7f5f069c8640 1 --2- 172.21.15.98:0/2554605833 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f000d4820 0x7f5f001621b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:49.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.138+0000 7f5f069c8640 1 -- 172.21.15.98:0/2554605833 >> 172.21.15.98:0/2554605833 conn(0x7f5f00074e40 msgr2=0x7f5f00075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:49.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.138+0000 7f5f069c8640 1 -- 172.21.15.98:0/2554605833 shutdown_connections 2024-09-19T08:46:49.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.138+0000 7f5f069c8640 1 -- 172.21.15.98:0/2554605833 wait complete. 2024-09-19T08:46:49.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.138+0000 7f5f069c8640 1 Processor -- start 2024-09-19T08:46:49.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.138+0000 7f5f069c8640 1 -- start start 2024-09-19T08:46:49.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.138+0000 7f5f069c8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f000b7f80 0x7f5f0013fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:49.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.138+0000 7f5f069c8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f000b9970 0x7f5f000d21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:49.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.138+0000 7f5f069c8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f000d4820 0x7f5f00142c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:49.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.138+0000 7f5f069c8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5f000b84e0 con 0x7f5f000b7f80 2024-09-19T08:46:49.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.138+0000 7f5f069c8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5f000b8360 con 0x7f5f000b9970 2024-09-19T08:46:49.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.138+0000 7f5f069c8640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5f00144590 con 0x7f5f000d4820 2024-09-19T08:46:49.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.138+0000 7f5efffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f000b7f80 0x7f5f0013fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:49.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.138+0000 7f5efffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f000b7f80 0x7f5f0013fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54294/0 (socket says 172.21.15.98:54294) 2024-09-19T08:46:49.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.138+0000 7f5efffff640 1 -- 172.21.15.98:0/2702522884 learned_addr learned my addr 172.21.15.98:0/2702522884 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:49.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.138+0000 7f5f04f65640 1 --2- 172.21.15.98:0/2702522884 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f000d4820 0x7f5f00142c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:49.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.138+0000 7f5eff7fe640 1 --2- 172.21.15.98:0/2702522884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f000b9970 0x7f5f000d21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:49.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.138+0000 7f5efffff640 1 -- 172.21.15.98:0/2702522884 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f000d4820 msgr2=0x7f5f00142c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:49.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.138+0000 7f5efffff640 1 --2- 172.21.15.98:0/2702522884 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f000d4820 0x7f5f00142c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:49.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.138+0000 7f5efffff640 1 -- 172.21.15.98:0/2702522884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f000b9970 msgr2=0x7f5f000d21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:49.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.138+0000 7f5efffff640 1 --2- 172.21.15.98:0/2702522884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f000b9970 0x7f5f000d21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:49.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.138+0000 7f5efffff640 1 -- 172.21.15.98:0/2702522884 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f00140400 con 0x7f5f000b7f80 2024-09-19T08:46:49.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.138+0000 7f5eff7fe640 1 --2- 172.21.15.98:0/2702522884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f000b9970 0x7f5f000d21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:49.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.142+0000 7f5efffff640 1 --2- 172.21.15.98:0/2702522884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f000b7f80 0x7f5f0013fa40 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f5ef000c3a0 tx=0x7f5ef000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:49.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.142+0000 7f5efcff9640 1 -- 172.21.15.98:0/2702522884 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef00052d0 con 0x7f5f000b7f80 2024-09-19T08:46:49.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.142+0000 7f5efcff9640 1 -- 172.21.15.98:0/2702522884 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ef0005470 con 0x7f5f000b7f80 2024-09-19T08:46:49.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.142+0000 7f5f069c8640 1 -- 172.21.15.98:0/2702522884 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f00155550 con 0x7f5f000b7f80 2024-09-19T08:46:49.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.142+0000 7f5efcff9640 1 -- 172.21.15.98:0/2702522884 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0007260 con 0x7f5f000b7f80 2024-09-19T08:46:49.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.142+0000 7f5f069c8640 1 -- 172.21.15.98:0/2702522884 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f00150760 con 0x7f5f000b7f80 2024-09-19T08:46:49.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.142+0000 7f5f069c8640 1 -- 172.21.15.98:0/2702522884 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f000b9e20 con 0x7f5f000b7f80 2024-09-19T08:46:49.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.142+0000 7f5efcff9640 1 -- 172.21.15.98:0/2702522884 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f5ef0007400 con 0x7f5f000b7f80 2024-09-19T08:46:49.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.142+0000 7f5efcff9640 1 --2- 172.21.15.98:0/2702522884 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5eec0680a0 0x7f5eec06a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:49.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.142+0000 7f5efcff9640 1 -- 172.21.15.98:0/2702522884 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5737+0+0 (secure 0 0 0) 0x7f5ef0083b00 con 0x7f5f000b7f80 2024-09-19T08:46:49.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.142+0000 7f5eff7fe640 1 --2- 172.21.15.98:0/2702522884 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5eec0680a0 0x7f5eec06a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:49.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.142+0000 7f5eff7fe640 1 --2- 172.21.15.98:0/2702522884 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5eec0680a0 0x7f5eec06a560 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5f00155100 tx=0x7f5ef4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:49.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.146+0000 7f5efcff9640 1 -- 172.21.15.98:0/2702522884 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f5ef0055f30 con 0x7f5f000b7f80 2024-09-19T08:46:49.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.306+0000 7f5f069c8640 1 -- 172.21.15.98:0/2702522884 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5f00155830 con 0x7f5eec0680a0 2024-09-19T08:46:49.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.322+0000 7f5efcff9640 1 -- 172.21.15.98:0/2702522884 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252373 (secure 0 0 0) 0x7f5f00155830 con 0x7f5eec0680a0 2024-09-19T08:46:49.327 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:46:49.328 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:46:49.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.326+0000 7f5f069c8640 1 -- 172.21.15.98:0/2702522884 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5eec0680a0 msgr2=0x7f5eec06a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:49.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.326+0000 7f5f069c8640 1 --2- 172.21.15.98:0/2702522884 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5eec0680a0 0x7f5eec06a560 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5f00155100 tx=0x7f5ef4008040 comp rx=0 tx=0).stop 2024-09-19T08:46:49.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.326+0000 7f5f069c8640 1 -- 172.21.15.98:0/2702522884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f000b7f80 msgr2=0x7f5f0013fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:49.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.326+0000 7f5f069c8640 1 --2- 172.21.15.98:0/2702522884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f000b7f80 0x7f5f0013fa40 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f5ef000c3a0 tx=0x7f5ef000c860 comp rx=0 tx=0).stop 2024-09-19T08:46:49.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.326+0000 7f5f069c8640 1 -- 172.21.15.98:0/2702522884 shutdown_connections 2024-09-19T08:46:49.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.326+0000 7f5f069c8640 1 --2- 172.21.15.98:0/2702522884 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5eec0680a0 0x7f5eec06a560 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:49.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.326+0000 7f5f069c8640 1 --2- 172.21.15.98:0/2702522884 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5f000d4820 0x7f5f00142c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:49.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.326+0000 7f5f069c8640 1 --2- 172.21.15.98:0/2702522884 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5f000b9970 0x7f5f000d21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:49.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.326+0000 7f5f069c8640 1 --2- 172.21.15.98:0/2702522884 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5f000b7f80 0x7f5f0013fa40 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:49.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.326+0000 7f5f069c8640 1 -- 172.21.15.98:0/2702522884 >> 172.21.15.98:0/2702522884 conn(0x7f5f00074e40 msgr2=0x7f5f00140690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:49.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.326+0000 7f5f069c8640 1 -- 172.21.15.98:0/2702522884 shutdown_connections 2024-09-19T08:46:49.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.330+0000 7f5f069c8640 1 -- 172.21.15.98:0/2702522884 wait complete. 2024-09-19T08:46:49.344 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":204,"stamp":"2024-09-19T08:46:48.948744+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":274588,"kb_used_data":16712,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754700132,"statfs":{"total":773094113280,"available":772812935168,"internally_reserved":0,"allocated":17113088,"data_stored":8233868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.681200"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.300021+0000","last_change":"2024-09-19T08:46:40.322807+0000","last_active":"2024-09-19T08:46:41.300021+0000","last_peered":"2024-09-19T08:46:41.300021+0000","last_clean":"2024-09-19T08:46:41.300021+0000","last_became_active":"2024-09-19T08:46:40.322530+0000","last_became_peered":"2024-09-19T08:46:40.322530+0000","last_unstale":"2024-09-19T08:46:41.300021+0000","last_undegraded":"2024-09-19T08:46:41.300021+0000","last_fullsized":"2024-09-19T08:46:41.300021+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:14:04.162980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.275876+0000","last_change":"2024-09-19T08:46:40.308583+0000","last_active":"2024-09-19T08:46:41.275876+0000","last_peered":"2024-09-19T08:46:41.275876+0000","last_clean":"2024-09-19T08:46:41.275876+0000","last_became_active":"2024-09-19T08:46:40.307894+0000","last_became_peered":"2024-09-19T08:46:40.307894+0000","last_unstale":"2024-09-19T08:46:41.275876+0000","last_undegraded":"2024-09-19T08:46:41.275876+0000","last_fullsized":"2024-09-19T08:46:41.275876+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:48.398635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.294222+0000","last_change":"2024-09-19T08:46:40.305267+0000","last_active":"2024-09-19T08:46:41.294222+0000","last_peered":"2024-09-19T08:46:41.294222+0000","last_clean":"2024-09-19T08:46:41.294222+0000","last_became_active":"2024-09-19T08:46:40.304553+0000","last_became_peered":"2024-09-19T08:46:40.304553+0000","last_unstale":"2024-09-19T08:46:41.294222+0000","last_undegraded":"2024-09-19T08:46:41.294222+0000","last_fullsized":"2024-09-19T08:46:41.294222+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:49.743970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.275107+0000","last_change":"2024-09-19T08:46:40.307985+0000","last_active":"2024-09-19T08:46:41.275107+0000","last_peered":"2024-09-19T08:46:41.275107+0000","last_clean":"2024-09-19T08:46:41.275107+0000","last_became_active":"2024-09-19T08:46:40.307321+0000","last_became_peered":"2024-09-19T08:46:40.307321+0000","last_unstale":"2024-09-19T08:46:41.275107+0000","last_undegraded":"2024-09-19T08:46:41.275107+0000","last_fullsized":"2024-09-19T08:46:41.275107+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:37:44.052309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271115+0000","last_change":"2024-09-19T08:46:40.313691+0000","last_active":"2024-09-19T08:46:41.271115+0000","last_peered":"2024-09-19T08:46:41.271115+0000","last_clean":"2024-09-19T08:46:41.271115+0000","last_became_active":"2024-09-19T08:46:40.313152+0000","last_became_peered":"2024-09-19T08:46:40.313152+0000","last_unstale":"2024-09-19T08:46:41.271115+0000","last_undegraded":"2024-09-19T08:46:41.271115+0000","last_fullsized":"2024-09-19T08:46:41.271115+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:17:30.627741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.293469+0000","last_change":"2024-09-19T08:46:40.310239+0000","last_active":"2024-09-19T08:46:41.293469+0000","last_peered":"2024-09-19T08:46:41.293469+0000","last_clean":"2024-09-19T08:46:41.293469+0000","last_became_active":"2024-09-19T08:46:40.309610+0000","last_became_peered":"2024-09-19T08:46:40.309610+0000","last_unstale":"2024-09-19T08:46:41.293469+0000","last_undegraded":"2024-09-19T08:46:41.293469+0000","last_fullsized":"2024-09-19T08:46:41.293469+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:10.797977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.297686+0000","last_change":"2024-09-19T08:46:40.300397+0000","last_active":"2024-09-19T08:46:41.297686+0000","last_peered":"2024-09-19T08:46:41.297686+0000","last_clean":"2024-09-19T08:46:41.297686+0000","last_became_active":"2024-09-19T08:46:40.299734+0000","last_became_peered":"2024-09-19T08:46:40.299734+0000","last_unstale":"2024-09-19T08:46:41.297686+0000","last_undegraded":"2024-09-19T08:46:41.297686+0000","last_fullsized":"2024-09-19T08:46:41.297686+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:20.271146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295020+0000","last_change":"2024-09-19T08:46:40.282539+0000","last_active":"2024-09-19T08:46:41.295020+0000","last_peered":"2024-09-19T08:46:41.295020+0000","last_clean":"2024-09-19T08:46:41.295020+0000","last_became_active":"2024-09-19T08:46:40.281719+0000","last_became_peered":"2024-09-19T08:46:40.281719+0000","last_unstale":"2024-09-19T08:46:41.295020+0000","last_undegraded":"2024-09-19T08:46:41.295020+0000","last_fullsized":"2024-09-19T08:46:41.295020+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:54:49.652718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.302625+0000","last_change":"2024-09-19T08:46:40.313125+0000","last_active":"2024-09-19T08:46:41.302625+0000","last_peered":"2024-09-19T08:46:41.302625+0000","last_clean":"2024-09-19T08:46:41.302625+0000","last_became_active":"2024-09-19T08:46:40.312593+0000","last_became_peered":"2024-09-19T08:46:40.312593+0000","last_unstale":"2024-09-19T08:46:41.302625+0000","last_undegraded":"2024-09-19T08:46:41.302625+0000","last_fullsized":"2024-09-19T08:46:41.302625+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:18.762035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295826+0000","last_change":"2024-09-19T08:46:40.299010+0000","last_active":"2024-09-19T08:46:41.295826+0000","last_peered":"2024-09-19T08:46:41.295826+0000","last_clean":"2024-09-19T08:46:41.295826+0000","last_became_active":"2024-09-19T08:46:40.298263+0000","last_became_peered":"2024-09-19T08:46:40.298263+0000","last_unstale":"2024-09-19T08:46:41.295826+0000","last_undegraded":"2024-09-19T08:46:41.295826+0000","last_fullsized":"2024-09-19T08:46:41.295826+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:29:29.482503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.298285+0000","last_change":"2024-09-19T08:46:40.312068+0000","last_active":"2024-09-19T08:46:41.298285+0000","last_peered":"2024-09-19T08:46:41.298285+0000","last_clean":"2024-09-19T08:46:41.298285+0000","last_became_active":"2024-09-19T08:46:40.311698+0000","last_became_peered":"2024-09-19T08:46:40.311698+0000","last_unstale":"2024-09-19T08:46:41.298285+0000","last_undegraded":"2024-09-19T08:46:41.298285+0000","last_fullsized":"2024-09-19T08:46:41.298285+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:37:24.320667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.275730+0000","last_change":"2024-09-19T08:46:40.318436+0000","last_active":"2024-09-19T08:46:41.275730+0000","last_peered":"2024-09-19T08:46:41.275730+0000","last_clean":"2024-09-19T08:46:41.275730+0000","last_became_active":"2024-09-19T08:46:40.318012+0000","last_became_peered":"2024-09-19T08:46:40.318012+0000","last_unstale":"2024-09-19T08:46:41.275730+0000","last_undegraded":"2024-09-19T08:46:41.275730+0000","last_fullsized":"2024-09-19T08:46:41.275730+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:08.849222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.274801+0000","last_change":"2024-09-19T08:46:40.291508+0000","last_active":"2024-09-19T08:46:41.274801+0000","last_peered":"2024-09-19T08:46:41.274801+0000","last_clean":"2024-09-19T08:46:41.274801+0000","last_became_active":"2024-09-19T08:46:40.291252+0000","last_became_peered":"2024-09-19T08:46:40.291252+0000","last_unstale":"2024-09-19T08:46:41.274801+0000","last_undegraded":"2024-09-19T08:46:41.274801+0000","last_fullsized":"2024-09-19T08:46:41.274801+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:42.984066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.302365+0000","last_change":"2024-09-19T08:46:40.318237+0000","last_active":"2024-09-19T08:46:41.302365+0000","last_peered":"2024-09-19T08:46:41.302365+0000","last_clean":"2024-09-19T08:46:41.302365+0000","last_became_active":"2024-09-19T08:46:40.317621+0000","last_became_peered":"2024-09-19T08:46:40.317621+0000","last_unstale":"2024-09-19T08:46:41.302365+0000","last_undegraded":"2024-09-19T08:46:41.302365+0000","last_fullsized":"2024-09-19T08:46:41.302365+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:09.679777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.275004+0000","last_change":"2024-09-19T08:46:40.295471+0000","last_active":"2024-09-19T08:46:41.275004+0000","last_peered":"2024-09-19T08:46:41.275004+0000","last_clean":"2024-09-19T08:46:41.275004+0000","last_became_active":"2024-09-19T08:46:40.294889+0000","last_became_peered":"2024-09-19T08:46:40.294889+0000","last_unstale":"2024-09-19T08:46:41.275004+0000","last_undegraded":"2024-09-19T08:46:41.275004+0000","last_fullsized":"2024-09-19T08:46:41.275004+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:44.098158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295327+0000","last_change":"2024-09-19T08:46:40.306178+0000","last_active":"2024-09-19T08:46:41.295327+0000","last_peered":"2024-09-19T08:46:41.295327+0000","last_clean":"2024-09-19T08:46:41.295327+0000","last_became_active":"2024-09-19T08:46:40.304073+0000","last_became_peered":"2024-09-19T08:46:40.304073+0000","last_unstale":"2024-09-19T08:46:41.295327+0000","last_undegraded":"2024-09-19T08:46:41.295327+0000","last_fullsized":"2024-09-19T08:46:41.295327+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:22.830837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271822+0000","last_change":"2024-09-19T08:46:40.287586+0000","last_active":"2024-09-19T08:46:41.271822+0000","last_peered":"2024-09-19T08:46:41.271822+0000","last_clean":"2024-09-19T08:46:41.271822+0000","last_became_active":"2024-09-19T08:46:40.286802+0000","last_became_peered":"2024-09-19T08:46:40.286802+0000","last_unstale":"2024-09-19T08:46:41.271822+0000","last_undegraded":"2024-09-19T08:46:41.271822+0000","last_fullsized":"2024-09-19T08:46:41.271822+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:53:20.598758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.299555+0000","last_change":"2024-09-19T08:46:40.325550+0000","last_active":"2024-09-19T08:46:41.299555+0000","last_peered":"2024-09-19T08:46:41.299555+0000","last_clean":"2024-09-19T08:46:41.299555+0000","last_became_active":"2024-09-19T08:46:40.325309+0000","last_became_peered":"2024-09-19T08:46:40.325309+0000","last_unstale":"2024-09-19T08:46:41.299555+0000","last_undegraded":"2024-09-19T08:46:41.299555+0000","last_fullsized":"2024-09-19T08:46:41.299555+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:21:31.832739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295824+0000","last_change":"2024-09-19T08:46:40.312961+0000","last_active":"2024-09-19T08:46:41.295824+0000","last_peered":"2024-09-19T08:46:41.295824+0000","last_clean":"2024-09-19T08:46:41.295824+0000","last_became_active":"2024-09-19T08:46:40.312097+0000","last_became_peered":"2024-09-19T08:46:40.312097+0000","last_unstale":"2024-09-19T08:46:41.295824+0000","last_undegraded":"2024-09-19T08:46:41.295824+0000","last_fullsized":"2024-09-19T08:46:41.295824+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:58:38.046334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271499+0000","last_change":"2024-09-19T08:46:40.286511+0000","last_active":"2024-09-19T08:46:41.271499+0000","last_peered":"2024-09-19T08:46:41.271499+0000","last_clean":"2024-09-19T08:46:41.271499+0000","last_became_active":"2024-09-19T08:46:40.286045+0000","last_became_peered":"2024-09-19T08:46:40.286045+0000","last_unstale":"2024-09-19T08:46:41.271499+0000","last_undegraded":"2024-09-19T08:46:41.271499+0000","last_fullsized":"2024-09-19T08:46:41.271499+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:48.932583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271520+0000","last_change":"2024-09-19T08:46:40.312590+0000","last_active":"2024-09-19T08:46:41.271520+0000","last_peered":"2024-09-19T08:46:41.271520+0000","last_clean":"2024-09-19T08:46:41.271520+0000","last_became_active":"2024-09-19T08:46:40.312010+0000","last_became_peered":"2024-09-19T08:46:40.312010+0000","last_unstale":"2024-09-19T08:46:41.271520+0000","last_undegraded":"2024-09-19T08:46:41.271520+0000","last_fullsized":"2024-09-19T08:46:41.271520+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:17:58.016895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.294442+0000","last_change":"2024-09-19T08:46:40.314682+0000","last_active":"2024-09-19T08:46:41.294442+0000","last_peered":"2024-09-19T08:46:41.294442+0000","last_clean":"2024-09-19T08:46:41.294442+0000","last_became_active":"2024-09-19T08:46:40.313976+0000","last_became_peered":"2024-09-19T08:46:40.313976+0000","last_unstale":"2024-09-19T08:46:41.294442+0000","last_undegraded":"2024-09-19T08:46:41.294442+0000","last_fullsized":"2024-09-19T08:46:41.294442+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:27:03.146587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.293441+0000","last_change":"2024-09-19T08:46:40.300547+0000","last_active":"2024-09-19T08:46:41.293441+0000","last_peered":"2024-09-19T08:46:41.293441+0000","last_clean":"2024-09-19T08:46:41.293441+0000","last_became_active":"2024-09-19T08:46:40.299848+0000","last_became_peered":"2024-09-19T08:46:40.299848+0000","last_unstale":"2024-09-19T08:46:41.293441+0000","last_undegraded":"2024-09-19T08:46:41.293441+0000","last_fullsized":"2024-09-19T08:46:41.293441+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:29:23.072767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295172+0000","last_change":"2024-09-19T08:46:40.296781+0000","last_active":"2024-09-19T08:46:41.295172+0000","last_peered":"2024-09-19T08:46:41.295172+0000","last_clean":"2024-09-19T08:46:41.295172+0000","last_became_active":"2024-09-19T08:46:40.296159+0000","last_became_peered":"2024-09-19T08:46:40.296159+0000","last_unstale":"2024-09-19T08:46:41.295172+0000","last_undegraded":"2024-09-19T08:46:41.295172+0000","last_fullsized":"2024-09-19T08:46:41.295172+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:36:31.828168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.275725+0000","last_change":"2024-09-19T08:46:40.311172+0000","last_active":"2024-09-19T08:46:41.275725+0000","last_peered":"2024-09-19T08:46:41.275725+0000","last_clean":"2024-09-19T08:46:41.275725+0000","last_became_active":"2024-09-19T08:46:40.310768+0000","last_became_peered":"2024-09-19T08:46:40.310768+0000","last_unstale":"2024-09-19T08:46:41.275725+0000","last_undegraded":"2024-09-19T08:46:41.275725+0000","last_fullsized":"2024-09-19T08:46:41.275725+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:26:01.059827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.298510+0000","last_change":"2024-09-19T08:46:40.304105+0000","last_active":"2024-09-19T08:46:41.298510+0000","last_peered":"2024-09-19T08:46:41.298510+0000","last_clean":"2024-09-19T08:46:41.298510+0000","last_became_active":"2024-09-19T08:46:40.303618+0000","last_became_peered":"2024-09-19T08:46:40.303618+0000","last_unstale":"2024-09-19T08:46:41.298510+0000","last_undegraded":"2024-09-19T08:46:41.298510+0000","last_fullsized":"2024-09-19T08:46:41.298510+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:15:05.241341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.294336+0000","last_change":"2024-09-19T08:46:40.310327+0000","last_active":"2024-09-19T08:46:41.294336+0000","last_peered":"2024-09-19T08:46:41.294336+0000","last_clean":"2024-09-19T08:46:41.294336+0000","last_became_active":"2024-09-19T08:46:40.309561+0000","last_became_peered":"2024-09-19T08:46:40.309561+0000","last_unstale":"2024-09-19T08:46:41.294336+0000","last_undegraded":"2024-09-19T08:46:41.294336+0000","last_fullsized":"2024-09-19T08:46:41.294336+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:39:18.317103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271831+0000","last_change":"2024-09-19T08:46:40.297612+0000","last_active":"2024-09-19T08:46:41.271831+0000","last_peered":"2024-09-19T08:46:41.271831+0000","last_clean":"2024-09-19T08:46:41.271831+0000","last_became_active":"2024-09-19T08:46:40.296522+0000","last_became_peered":"2024-09-19T08:46:40.296522+0000","last_unstale":"2024-09-19T08:46:41.271831+0000","last_undegraded":"2024-09-19T08:46:41.271831+0000","last_fullsized":"2024-09-19T08:46:41.271831+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:19:50.929185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.275232+0000","last_change":"2024-09-19T08:46:40.319198+0000","last_active":"2024-09-19T08:46:41.275232+0000","last_peered":"2024-09-19T08:46:41.275232+0000","last_clean":"2024-09-19T08:46:41.275232+0000","last_became_active":"2024-09-19T08:46:40.318778+0000","last_became_peered":"2024-09-19T08:46:40.318778+0000","last_unstale":"2024-09-19T08:46:41.275232+0000","last_undegraded":"2024-09-19T08:46:41.275232+0000","last_fullsized":"2024-09-19T08:46:41.275232+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:17:12.366818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.301582+0000","last_change":"2024-09-19T08:46:40.324681+0000","last_active":"2024-09-19T08:46:41.301582+0000","last_peered":"2024-09-19T08:46:41.301582+0000","last_clean":"2024-09-19T08:46:41.301582+0000","last_became_active":"2024-09-19T08:46:40.321740+0000","last_became_peered":"2024-09-19T08:46:40.321740+0000","last_unstale":"2024-09-19T08:46:41.301582+0000","last_undegraded":"2024-09-19T08:46:41.301582+0000","last_fullsized":"2024-09-19T08:46:41.301582+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:44:45.487714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.293584+0000","last_change":"2024-09-19T08:46:40.295308+0000","last_active":"2024-09-19T08:46:41.293584+0000","last_peered":"2024-09-19T08:46:41.293584+0000","last_clean":"2024-09-19T08:46:41.293584+0000","last_became_active":"2024-09-19T08:46:40.290389+0000","last_became_peered":"2024-09-19T08:46:40.290389+0000","last_unstale":"2024-09-19T08:46:41.293584+0000","last_undegraded":"2024-09-19T08:46:41.293584+0000","last_fullsized":"2024-09-19T08:46:41.293584+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:09:25.970759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271075+0000","last_change":"2024-09-19T08:46:40.287160+0000","last_active":"2024-09-19T08:46:41.271075+0000","last_peered":"2024-09-19T08:46:41.271075+0000","last_clean":"2024-09-19T08:46:41.271075+0000","last_became_active":"2024-09-19T08:46:40.286383+0000","last_became_peered":"2024-09-19T08:46:40.286383+0000","last_unstale":"2024-09-19T08:46:41.271075+0000","last_undegraded":"2024-09-19T08:46:41.271075+0000","last_fullsized":"2024-09-19T08:46:41.271075+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:11:56.945612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.296321+0000","last_change":"2024-09-19T08:46:40.305212+0000","last_active":"2024-09-19T08:46:41.296321+0000","last_peered":"2024-09-19T08:46:41.296321+0000","last_clean":"2024-09-19T08:46:41.296321+0000","last_became_active":"2024-09-19T08:46:40.304643+0000","last_became_peered":"2024-09-19T08:46:40.304643+0000","last_unstale":"2024-09-19T08:46:41.296321+0000","last_undegraded":"2024-09-19T08:46:41.296321+0000","last_fullsized":"2024-09-19T08:46:41.296321+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:57:56.539906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"157'4","reported_seq":18,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271785+0000","last_change":"2024-09-19T08:46:39.533871+0000","last_active":"2024-09-19T08:46:41.271785+0000","last_peered":"2024-09-19T08:46:41.271785+0000","last_clean":"2024-09-19T08:46:41.271785+0000","last_became_active":"2024-09-19T08:46:39.533575+0000","last_became_peered":"2024-09-19T08:46:39.533575+0000","last_unstale":"2024-09-19T08:46:41.271785+0000","last_undegraded":"2024-09-19T08:46:41.271785+0000","last_fullsized":"2024-09-19T08:46:41.271785+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:36:08.646357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.296566+0000","last_change":"2024-09-19T08:46:40.299343+0000","last_active":"2024-09-19T08:46:41.296566+0000","last_peered":"2024-09-19T08:46:41.296566+0000","last_clean":"2024-09-19T08:46:41.296566+0000","last_became_active":"2024-09-19T08:46:40.296360+0000","last_became_peered":"2024-09-19T08:46:40.296360+0000","last_unstale":"2024-09-19T08:46:41.296566+0000","last_undegraded":"2024-09-19T08:46:41.296566+0000","last_fullsized":"2024-09-19T08:46:41.296566+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:27:03.803525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"157'1","reported_seq":15,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.302393+0000","last_change":"2024-09-19T08:46:39.524090+0000","last_active":"2024-09-19T08:46:41.302393+0000","last_peered":"2024-09-19T08:46:41.302393+0000","last_clean":"2024-09-19T08:46:41.302393+0000","last_became_active":"2024-09-19T08:46:39.523389+0000","last_became_peered":"2024-09-19T08:46:39.523389+0000","last_unstale":"2024-09-19T08:46:41.302393+0000","last_undegraded":"2024-09-19T08:46:41.302393+0000","last_fullsized":"2024-09-19T08:46:41.302393+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:54:57.317359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.299684+0000","last_change":"2024-09-19T08:46:40.287661+0000","last_active":"2024-09-19T08:46:41.299684+0000","last_peered":"2024-09-19T08:46:41.299684+0000","last_clean":"2024-09-19T08:46:41.299684+0000","last_became_active":"2024-09-19T08:46:40.287397+0000","last_became_peered":"2024-09-19T08:46:40.287397+0000","last_unstale":"2024-09-19T08:46:41.299684+0000","last_undegraded":"2024-09-19T08:46:41.299684+0000","last_fullsized":"2024-09-19T08:46:41.299684+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:15.351548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"157'2","reported_seq":16,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.294377+0000","last_change":"2024-09-19T08:46:39.517378+0000","last_active":"2024-09-19T08:46:41.294377+0000","last_peered":"2024-09-19T08:46:41.294377+0000","last_clean":"2024-09-19T08:46:41.294377+0000","last_became_active":"2024-09-19T08:46:39.517112+0000","last_became_peered":"2024-09-19T08:46:39.517112+0000","last_unstale":"2024-09-19T08:46:41.294377+0000","last_undegraded":"2024-09-19T08:46:41.294377+0000","last_fullsized":"2024-09-19T08:46:41.294377+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:21:46.481801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.302427+0000","last_change":"2024-09-19T08:46:40.305888+0000","last_active":"2024-09-19T08:46:41.302427+0000","last_peered":"2024-09-19T08:46:41.302427+0000","last_clean":"2024-09-19T08:46:41.302427+0000","last_became_active":"2024-09-19T08:46:40.305283+0000","last_became_peered":"2024-09-19T08:46:40.305283+0000","last_unstale":"2024-09-19T08:46:41.302427+0000","last_undegraded":"2024-09-19T08:46:41.302427+0000","last_fullsized":"2024-09-19T08:46:41.302427+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:15.659241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"157'2","reported_seq":16,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.275270+0000","last_change":"2024-09-19T08:46:39.525313+0000","last_active":"2024-09-19T08:46:41.275270+0000","last_peered":"2024-09-19T08:46:41.275270+0000","last_clean":"2024-09-19T08:46:41.275270+0000","last_became_active":"2024-09-19T08:46:39.524576+0000","last_became_peered":"2024-09-19T08:46:39.524576+0000","last_unstale":"2024-09-19T08:46:41.275270+0000","last_undegraded":"2024-09-19T08:46:41.275270+0000","last_fullsized":"2024-09-19T08:46:41.275270+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:57.140465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.294535+0000","last_change":"2024-09-19T08:46:40.311346+0000","last_active":"2024-09-19T08:46:41.294535+0000","last_peered":"2024-09-19T08:46:41.294535+0000","last_clean":"2024-09-19T08:46:41.294535+0000","last_became_active":"2024-09-19T08:46:40.310733+0000","last_became_peered":"2024-09-19T08:46:40.310733+0000","last_unstale":"2024-09-19T08:46:41.294535+0000","last_undegraded":"2024-09-19T08:46:41.294535+0000","last_fullsized":"2024-09-19T08:46:41.294535+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:36:50.649569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"157'1","reported_seq":15,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271232+0000","last_change":"2024-09-19T08:46:39.526200+0000","last_active":"2024-09-19T08:46:41.271232+0000","last_peered":"2024-09-19T08:46:41.271232+0000","last_clean":"2024-09-19T08:46:41.271232+0000","last_became_active":"2024-09-19T08:46:39.525883+0000","last_became_peered":"2024-09-19T08:46:39.525883+0000","last_unstale":"2024-09-19T08:46:41.271232+0000","last_undegraded":"2024-09-19T08:46:41.271232+0000","last_fullsized":"2024-09-19T08:46:41.271232+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:52.132068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295513+0000","last_change":"2024-09-19T08:46:40.305229+0000","last_active":"2024-09-19T08:46:41.295513+0000","last_peered":"2024-09-19T08:46:41.295513+0000","last_clean":"2024-09-19T08:46:41.295513+0000","last_became_active":"2024-09-19T08:46:40.304492+0000","last_became_peered":"2024-09-19T08:46:40.304492+0000","last_unstale":"2024-09-19T08:46:41.295513+0000","last_undegraded":"2024-09-19T08:46:41.295513+0000","last_fullsized":"2024-09-19T08:46:41.295513+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:47:32.712559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"157'1","reported_seq":15,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295463+0000","last_change":"2024-09-19T08:46:39.519337+0000","last_active":"2024-09-19T08:46:41.295463+0000","last_peered":"2024-09-19T08:46:41.295463+0000","last_clean":"2024-09-19T08:46:41.295463+0000","last_became_active":"2024-09-19T08:46:39.518792+0000","last_became_peered":"2024-09-19T08:46:39.518792+0000","last_unstale":"2024-09-19T08:46:41.295463+0000","last_undegraded":"2024-09-19T08:46:41.295463+0000","last_fullsized":"2024-09-19T08:46:41.295463+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:30:05.550797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.297805+0000","last_change":"2024-09-19T08:46:40.311303+0000","last_active":"2024-09-19T08:46:41.297805+0000","last_peered":"2024-09-19T08:46:41.297805+0000","last_clean":"2024-09-19T08:46:41.297805+0000","last_became_active":"2024-09-19T08:46:40.310999+0000","last_became_peered":"2024-09-19T08:46:40.310999+0000","last_unstale":"2024-09-19T08:46:41.297805+0000","last_undegraded":"2024-09-19T08:46:41.297805+0000","last_fullsized":"2024-09-19T08:46:41.297805+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:49.743970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.298406+0000","last_change":"2024-09-19T08:46:39.532881+0000","last_active":"2024-09-19T08:46:41.298406+0000","last_peered":"2024-09-19T08:46:41.298406+0000","last_clean":"2024-09-19T08:46:41.298406+0000","last_became_active":"2024-09-19T08:46:39.532112+0000","last_became_peered":"2024-09-19T08:46:39.532112+0000","last_unstale":"2024-09-19T08:46:41.298406+0000","last_undegraded":"2024-09-19T08:46:41.298406+0000","last_fullsized":"2024-09-19T08:46:41.298406+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:32.581664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.294399+0000","last_change":"2024-09-19T08:46:40.296034+0000","last_active":"2024-09-19T08:46:41.294399+0000","last_peered":"2024-09-19T08:46:41.294399+0000","last_clean":"2024-09-19T08:46:41.294399+0000","last_became_active":"2024-09-19T08:46:40.294878+0000","last_became_peered":"2024-09-19T08:46:40.294878+0000","last_unstale":"2024-09-19T08:46:41.294399+0000","last_undegraded":"2024-09-19T08:46:41.294399+0000","last_fullsized":"2024-09-19T08:46:41.294399+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:44.855570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.299039+0000","last_change":"2024-09-19T08:46:39.540052+0000","last_active":"2024-09-19T08:46:41.299039+0000","last_peered":"2024-09-19T08:46:41.299039+0000","last_clean":"2024-09-19T08:46:41.299039+0000","last_became_active":"2024-09-19T08:46:39.539378+0000","last_became_peered":"2024-09-19T08:46:39.539378+0000","last_unstale":"2024-09-19T08:46:41.299039+0000","last_undegraded":"2024-09-19T08:46:41.299039+0000","last_fullsized":"2024-09-19T08:46:41.299039+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:06.592079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.296558+0000","last_change":"2024-09-19T08:46:40.302139+0000","last_active":"2024-09-19T08:46:41.296558+0000","last_peered":"2024-09-19T08:46:41.296558+0000","last_clean":"2024-09-19T08:46:41.296558+0000","last_became_active":"2024-09-19T08:46:40.301363+0000","last_became_peered":"2024-09-19T08:46:40.301363+0000","last_unstale":"2024-09-19T08:46:41.296558+0000","last_undegraded":"2024-09-19T08:46:41.296558+0000","last_fullsized":"2024-09-19T08:46:41.296558+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:13.914441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271475+0000","last_change":"2024-09-19T08:46:39.529992+0000","last_active":"2024-09-19T08:46:41.271475+0000","last_peered":"2024-09-19T08:46:41.271475+0000","last_clean":"2024-09-19T08:46:41.271475+0000","last_became_active":"2024-09-19T08:46:39.529500+0000","last_became_peered":"2024-09-19T08:46:39.529500+0000","last_unstale":"2024-09-19T08:46:41.271475+0000","last_undegraded":"2024-09-19T08:46:41.271475+0000","last_fullsized":"2024-09-19T08:46:41.271475+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:42:34.276215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.298137+0000","last_change":"2024-09-19T08:46:40.306814+0000","last_active":"2024-09-19T08:46:41.298137+0000","last_peered":"2024-09-19T08:46:41.298137+0000","last_clean":"2024-09-19T08:46:41.298137+0000","last_became_active":"2024-09-19T08:46:40.306279+0000","last_became_peered":"2024-09-19T08:46:40.306279+0000","last_unstale":"2024-09-19T08:46:41.298137+0000","last_undegraded":"2024-09-19T08:46:41.298137+0000","last_fullsized":"2024-09-19T08:46:41.298137+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:45.155900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"157'3","reported_seq":17,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271615+0000","last_change":"2024-09-19T08:46:39.530600+0000","last_active":"2024-09-19T08:46:41.271615+0000","last_peered":"2024-09-19T08:46:41.271615+0000","last_clean":"2024-09-19T08:46:41.271615+0000","last_became_active":"2024-09-19T08:46:39.526884+0000","last_became_peered":"2024-09-19T08:46:39.526884+0000","last_unstale":"2024-09-19T08:46:41.271615+0000","last_undegraded":"2024-09-19T08:46:41.271615+0000","last_fullsized":"2024-09-19T08:46:41.271615+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:01:13.258546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295715+0000","last_change":"2024-09-19T08:46:40.310166+0000","last_active":"2024-09-19T08:46:41.295715+0000","last_peered":"2024-09-19T08:46:41.295715+0000","last_clean":"2024-09-19T08:46:41.295715+0000","last_became_active":"2024-09-19T08:46:40.309720+0000","last_became_peered":"2024-09-19T08:46:40.309720+0000","last_unstale":"2024-09-19T08:46:41.295715+0000","last_undegraded":"2024-09-19T08:46:41.295715+0000","last_fullsized":"2024-09-19T08:46:41.295715+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:53:13.406407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"157'6","reported_seq":20,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.294318+0000","last_change":"2024-09-19T08:46:39.533705+0000","last_active":"2024-09-19T08:46:41.294318+0000","last_peered":"2024-09-19T08:46:41.294318+0000","last_clean":"2024-09-19T08:46:41.294318+0000","last_became_active":"2024-09-19T08:46:39.533388+0000","last_became_peered":"2024-09-19T08:46:39.533388+0000","last_unstale":"2024-09-19T08:46:41.294318+0000","last_undegraded":"2024-09-19T08:46:41.294318+0000","last_fullsized":"2024-09-19T08:46:41.294318+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:29.674750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.293672+0000","last_change":"2024-09-19T08:46:40.298398+0000","last_active":"2024-09-19T08:46:41.293672+0000","last_peered":"2024-09-19T08:46:41.293672+0000","last_clean":"2024-09-19T08:46:41.293672+0000","last_became_active":"2024-09-19T08:46:40.297708+0000","last_became_peered":"2024-09-19T08:46:40.297708+0000","last_unstale":"2024-09-19T08:46:41.293672+0000","last_undegraded":"2024-09-19T08:46:41.293672+0000","last_fullsized":"2024-09-19T08:46:41.293672+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:20.271146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"157'2","reported_seq":16,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.298905+0000","last_change":"2024-09-19T08:46:39.532544+0000","last_active":"2024-09-19T08:46:41.298905+0000","last_peered":"2024-09-19T08:46:41.298905+0000","last_clean":"2024-09-19T08:46:41.298905+0000","last_became_active":"2024-09-19T08:46:39.531951+0000","last_became_peered":"2024-09-19T08:46:39.531951+0000","last_unstale":"2024-09-19T08:46:41.298905+0000","last_undegraded":"2024-09-19T08:46:41.298905+0000","last_fullsized":"2024-09-19T08:46:41.298905+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:44:23.781566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.297970+0000","last_change":"2024-09-19T08:46:40.302762+0000","last_active":"2024-09-19T08:46:41.297970+0000","last_peered":"2024-09-19T08:46:41.297970+0000","last_clean":"2024-09-19T08:46:41.297970+0000","last_became_active":"2024-09-19T08:46:40.301495+0000","last_became_peered":"2024-09-19T08:46:40.301495+0000","last_unstale":"2024-09-19T08:46:41.297970+0000","last_undegraded":"2024-09-19T08:46:41.297970+0000","last_fullsized":"2024-09-19T08:46:41.297970+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:39:18.317103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.294654+0000","last_change":"2024-09-19T08:46:39.523177+0000","last_active":"2024-09-19T08:46:41.294654+0000","last_peered":"2024-09-19T08:46:41.294654+0000","last_clean":"2024-09-19T08:46:41.294654+0000","last_became_active":"2024-09-19T08:46:39.522823+0000","last_became_peered":"2024-09-19T08:46:39.522823+0000","last_unstale":"2024-09-19T08:46:41.294654+0000","last_undegraded":"2024-09-19T08:46:41.294654+0000","last_fullsized":"2024-09-19T08:46:41.294654+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:32.581664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.301407+0000","last_change":"2024-09-19T08:46:40.319863+0000","last_active":"2024-09-19T08:46:41.301407+0000","last_peered":"2024-09-19T08:46:41.301407+0000","last_clean":"2024-09-19T08:46:41.301407+0000","last_became_active":"2024-09-19T08:46:40.319407+0000","last_became_peered":"2024-09-19T08:46:40.319407+0000","last_unstale":"2024-09-19T08:46:41.301407+0000","last_undegraded":"2024-09-19T08:46:41.301407+0000","last_fullsized":"2024-09-19T08:46:41.301407+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:34.896081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295262+0000","last_change":"2024-09-19T08:46:39.528911+0000","last_active":"2024-09-19T08:46:41.295262+0000","last_peered":"2024-09-19T08:46:41.295262+0000","last_clean":"2024-09-19T08:46:41.295262+0000","last_became_active":"2024-09-19T08:46:39.528519+0000","last_became_peered":"2024-09-19T08:46:39.528519+0000","last_unstale":"2024-09-19T08:46:41.295262+0000","last_undegraded":"2024-09-19T08:46:41.295262+0000","last_fullsized":"2024-09-19T08:46:41.295262+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:49:05.280493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.294063+0000","last_change":"2024-09-19T08:46:40.290329+0000","last_active":"2024-09-19T08:46:41.294063+0000","last_peered":"2024-09-19T08:46:41.294063+0000","last_clean":"2024-09-19T08:46:41.294063+0000","last_became_active":"2024-09-19T08:46:40.289851+0000","last_became_peered":"2024-09-19T08:46:40.289851+0000","last_unstale":"2024-09-19T08:46:41.294063+0000","last_undegraded":"2024-09-19T08:46:41.294063+0000","last_fullsized":"2024-09-19T08:46:41.294063+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:36:51.887939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"157'8","reported_seq":22,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.298681+0000","last_change":"2024-09-19T08:46:39.534632+0000","last_active":"2024-09-19T08:46:41.298681+0000","last_peered":"2024-09-19T08:46:41.298681+0000","last_clean":"2024-09-19T08:46:41.298681+0000","last_became_active":"2024-09-19T08:46:39.533906+0000","last_became_peered":"2024-09-19T08:46:39.533906+0000","last_unstale":"2024-09-19T08:46:41.298681+0000","last_undegraded":"2024-09-19T08:46:41.298681+0000","last_fullsized":"2024-09-19T08:46:41.298681+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:46:42.284571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.296092+0000","last_change":"2024-09-19T08:46:40.294805+0000","last_active":"2024-09-19T08:46:41.296092+0000","last_peered":"2024-09-19T08:46:41.296092+0000","last_clean":"2024-09-19T08:46:41.296092+0000","last_became_active":"2024-09-19T08:46:40.294338+0000","last_became_peered":"2024-09-19T08:46:40.294338+0000","last_unstale":"2024-09-19T08:46:41.296092+0000","last_undegraded":"2024-09-19T08:46:41.296092+0000","last_fullsized":"2024-09-19T08:46:41.296092+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:10:19.078157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"157'1","reported_seq":15,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271881+0000","last_change":"2024-09-19T08:46:39.529025+0000","last_active":"2024-09-19T08:46:41.271881+0000","last_peered":"2024-09-19T08:46:41.271881+0000","last_clean":"2024-09-19T08:46:41.271881+0000","last_became_active":"2024-09-19T08:46:39.526531+0000","last_became_peered":"2024-09-19T08:46:39.526531+0000","last_unstale":"2024-09-19T08:46:41.271881+0000","last_undegraded":"2024-09-19T08:46:41.271881+0000","last_fullsized":"2024-09-19T08:46:41.271881+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:18:57.366963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.302806+0000","last_change":"2024-09-19T08:46:40.320361+0000","last_active":"2024-09-19T08:46:41.302806+0000","last_peered":"2024-09-19T08:46:41.302806+0000","last_clean":"2024-09-19T08:46:41.302806+0000","last_became_active":"2024-09-19T08:46:40.319637+0000","last_became_peered":"2024-09-19T08:46:40.319637+0000","last_unstale":"2024-09-19T08:46:41.302806+0000","last_undegraded":"2024-09-19T08:46:41.302806+0000","last_fullsized":"2024-09-19T08:46:41.302806+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:09:14.940629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"157'2","reported_seq":16,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.296238+0000","last_change":"2024-09-19T08:46:39.528693+0000","last_active":"2024-09-19T08:46:41.296238+0000","last_peered":"2024-09-19T08:46:41.296238+0000","last_clean":"2024-09-19T08:46:41.296238+0000","last_became_active":"2024-09-19T08:46:39.528063+0000","last_became_peered":"2024-09-19T08:46:39.528063+0000","last_unstale":"2024-09-19T08:46:41.296238+0000","last_undegraded":"2024-09-19T08:46:41.296238+0000","last_fullsized":"2024-09-19T08:46:41.296238+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:04:47.931376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.301874+0000","last_change":"2024-09-19T08:46:40.324664+0000","last_active":"2024-09-19T08:46:41.301874+0000","last_peered":"2024-09-19T08:46:41.301874+0000","last_clean":"2024-09-19T08:46:41.301874+0000","last_became_active":"2024-09-19T08:46:40.321357+0000","last_became_peered":"2024-09-19T08:46:40.321357+0000","last_unstale":"2024-09-19T08:46:41.301874+0000","last_undegraded":"2024-09-19T08:46:41.301874+0000","last_fullsized":"2024-09-19T08:46:41.301874+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:56:59.869941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"157'1","reported_seq":15,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.299335+0000","last_change":"2024-09-19T08:46:39.534210+0000","last_active":"2024-09-19T08:46:41.299335+0000","last_peered":"2024-09-19T08:46:41.299335+0000","last_clean":"2024-09-19T08:46:41.299335+0000","last_became_active":"2024-09-19T08:46:39.533548+0000","last_became_peered":"2024-09-19T08:46:39.533548+0000","last_unstale":"2024-09-19T08:46:41.299335+0000","last_undegraded":"2024-09-19T08:46:41.299335+0000","last_fullsized":"2024-09-19T08:46:41.299335+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:21:50.291503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.275460+0000","last_change":"2024-09-19T08:46:40.308301+0000","last_active":"2024-09-19T08:46:41.275460+0000","last_peered":"2024-09-19T08:46:41.275460+0000","last_clean":"2024-09-19T08:46:41.275460+0000","last_became_active":"2024-09-19T08:46:40.307793+0000","last_became_peered":"2024-09-19T08:46:40.307793+0000","last_unstale":"2024-09-19T08:46:41.275460+0000","last_undegraded":"2024-09-19T08:46:41.275460+0000","last_fullsized":"2024-09-19T08:46:41.275460+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:28:21.144985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.298624+0000","last_change":"2024-09-19T08:46:39.536189+0000","last_active":"2024-09-19T08:46:41.298624+0000","last_peered":"2024-09-19T08:46:41.298624+0000","last_clean":"2024-09-19T08:46:41.298624+0000","last_became_active":"2024-09-19T08:46:39.535387+0000","last_became_peered":"2024-09-19T08:46:39.535387+0000","last_unstale":"2024-09-19T08:46:41.298624+0000","last_undegraded":"2024-09-19T08:46:41.298624+0000","last_fullsized":"2024-09-19T08:46:41.298624+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:00:43.262958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.294622+0000","last_change":"2024-09-19T08:46:40.308240+0000","last_active":"2024-09-19T08:46:41.294622+0000","last_peered":"2024-09-19T08:46:41.294622+0000","last_clean":"2024-09-19T08:46:41.294622+0000","last_became_active":"2024-09-19T08:46:40.307131+0000","last_became_peered":"2024-09-19T08:46:40.307131+0000","last_unstale":"2024-09-19T08:46:41.294622+0000","last_undegraded":"2024-09-19T08:46:41.294622+0000","last_fullsized":"2024-09-19T08:46:41.294622+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:40:32.674934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"157'2","reported_seq":16,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.293923+0000","last_change":"2024-09-19T08:46:39.529933+0000","last_active":"2024-09-19T08:46:41.293923+0000","last_peered":"2024-09-19T08:46:41.293923+0000","last_clean":"2024-09-19T08:46:41.293923+0000","last_became_active":"2024-09-19T08:46:39.529602+0000","last_became_peered":"2024-09-19T08:46:39.529602+0000","last_unstale":"2024-09-19T08:46:41.293923+0000","last_undegraded":"2024-09-19T08:46:41.293923+0000","last_fullsized":"2024-09-19T08:46:41.293923+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:45:17.113767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.298023+0000","last_change":"2024-09-19T08:46:40.278369+0000","last_active":"2024-09-19T08:46:41.298023+0000","last_peered":"2024-09-19T08:46:41.298023+0000","last_clean":"2024-09-19T08:46:41.298023+0000","last_became_active":"2024-09-19T08:46:40.277767+0000","last_became_peered":"2024-09-19T08:46:40.277767+0000","last_unstale":"2024-09-19T08:46:41.298023+0000","last_undegraded":"2024-09-19T08:46:41.298023+0000","last_fullsized":"2024-09-19T08:46:41.298023+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:04:44.855570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271109+0000","last_change":"2024-09-19T08:46:39.526194+0000","last_active":"2024-09-19T08:46:41.271109+0000","last_peered":"2024-09-19T08:46:41.271109+0000","last_clean":"2024-09-19T08:46:41.271109+0000","last_became_active":"2024-09-19T08:46:39.525582+0000","last_became_peered":"2024-09-19T08:46:39.525582+0000","last_unstale":"2024-09-19T08:46:41.271109+0000","last_undegraded":"2024-09-19T08:46:41.271109+0000","last_fullsized":"2024-09-19T08:46:41.271109+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:54.940787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.274690+0000","last_change":"2024-09-19T08:46:40.311768+0000","last_active":"2024-09-19T08:46:41.274690+0000","last_peered":"2024-09-19T08:46:41.274690+0000","last_clean":"2024-09-19T08:46:41.274690+0000","last_became_active":"2024-09-19T08:46:40.311464+0000","last_became_peered":"2024-09-19T08:46:40.311464+0000","last_unstale":"2024-09-19T08:46:41.274690+0000","last_undegraded":"2024-09-19T08:46:41.274690+0000","last_fullsized":"2024-09-19T08:46:41.274690+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:08:48.768860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.293578+0000","last_change":"2024-09-19T08:46:39.531582+0000","last_active":"2024-09-19T08:46:41.293578+0000","last_peered":"2024-09-19T08:46:41.293578+0000","last_clean":"2024-09-19T08:46:41.293578+0000","last_became_active":"2024-09-19T08:46:39.531166+0000","last_became_peered":"2024-09-19T08:46:39.531166+0000","last_unstale":"2024-09-19T08:46:41.293578+0000","last_undegraded":"2024-09-19T08:46:41.293578+0000","last_fullsized":"2024-09-19T08:46:41.293578+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:45:20.818342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271332+0000","last_change":"2024-09-19T08:46:40.299301+0000","last_active":"2024-09-19T08:46:41.271332+0000","last_peered":"2024-09-19T08:46:41.271332+0000","last_clean":"2024-09-19T08:46:41.271332+0000","last_became_active":"2024-09-19T08:46:40.298235+0000","last_became_peered":"2024-09-19T08:46:40.298235+0000","last_unstale":"2024-09-19T08:46:41.271332+0000","last_undegraded":"2024-09-19T08:46:41.271332+0000","last_fullsized":"2024-09-19T08:46:41.271332+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:36:59.594448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.301286+0000","last_change":"2024-09-19T08:46:39.529718+0000","last_active":"2024-09-19T08:46:41.301286+0000","last_peered":"2024-09-19T08:46:41.301286+0000","last_clean":"2024-09-19T08:46:41.301286+0000","last_became_active":"2024-09-19T08:46:39.529000+0000","last_became_peered":"2024-09-19T08:46:39.529000+0000","last_unstale":"2024-09-19T08:46:41.301286+0000","last_undegraded":"2024-09-19T08:46:41.301286+0000","last_fullsized":"2024-09-19T08:46:41.301286+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:03:47.923282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271607+0000","last_change":"2024-09-19T08:46:40.296371+0000","last_active":"2024-09-19T08:46:41.271607+0000","last_peered":"2024-09-19T08:46:41.271607+0000","last_clean":"2024-09-19T08:46:41.271607+0000","last_became_active":"2024-09-19T08:46:40.295860+0000","last_became_peered":"2024-09-19T08:46:40.295860+0000","last_unstale":"2024-09-19T08:46:41.271607+0000","last_undegraded":"2024-09-19T08:46:41.271607+0000","last_fullsized":"2024-09-19T08:46:41.271607+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:54:58.326660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"157'2","reported_seq":16,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.299767+0000","last_change":"2024-09-19T08:46:39.533469+0000","last_active":"2024-09-19T08:46:41.299767+0000","last_peered":"2024-09-19T08:46:41.299767+0000","last_clean":"2024-09-19T08:46:41.299767+0000","last_became_active":"2024-09-19T08:46:39.532381+0000","last_became_peered":"2024-09-19T08:46:39.532381+0000","last_unstale":"2024-09-19T08:46:41.299767+0000","last_undegraded":"2024-09-19T08:46:41.299767+0000","last_fullsized":"2024-09-19T08:46:41.299767+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:38.871312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.296434+0000","last_change":"2024-09-19T08:46:40.316938+0000","last_active":"2024-09-19T08:46:41.296434+0000","last_peered":"2024-09-19T08:46:41.296434+0000","last_clean":"2024-09-19T08:46:41.296434+0000","last_became_active":"2024-09-19T08:46:40.316355+0000","last_became_peered":"2024-09-19T08:46:40.316355+0000","last_unstale":"2024-09-19T08:46:41.296434+0000","last_undegraded":"2024-09-19T08:46:41.296434+0000","last_fullsized":"2024-09-19T08:46:41.296434+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:20:14.319040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"158'1","reported_seq":15,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.296024+0000","last_change":"2024-09-19T08:46:39.517540+0000","last_active":"2024-09-19T08:46:41.296024+0000","last_peered":"2024-09-19T08:46:41.296024+0000","last_clean":"2024-09-19T08:46:41.296024+0000","last_became_active":"2024-09-19T08:46:39.517247+0000","last_became_peered":"2024-09-19T08:46:39.517247+0000","last_unstale":"2024-09-19T08:46:41.296024+0000","last_undegraded":"2024-09-19T08:46:41.296024+0000","last_fullsized":"2024-09-19T08:46:41.296024+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:42.770912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295742+0000","last_change":"2024-09-19T08:46:40.292589+0000","last_active":"2024-09-19T08:46:41.295742+0000","last_peered":"2024-09-19T08:46:41.295742+0000","last_clean":"2024-09-19T08:46:41.295742+0000","last_became_active":"2024-09-19T08:46:40.292129+0000","last_became_peered":"2024-09-19T08:46:40.292129+0000","last_unstale":"2024-09-19T08:46:41.295742+0000","last_undegraded":"2024-09-19T08:46:41.295742+0000","last_fullsized":"2024-09-19T08:46:41.295742+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:04.629751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295780+0000","last_change":"2024-09-19T08:46:39.517207+0000","last_active":"2024-09-19T08:46:41.295780+0000","last_peered":"2024-09-19T08:46:41.295780+0000","last_clean":"2024-09-19T08:46:41.295780+0000","last_became_active":"2024-09-19T08:46:39.516924+0000","last_became_peered":"2024-09-19T08:46:39.516924+0000","last_unstale":"2024-09-19T08:46:41.295780+0000","last_undegraded":"2024-09-19T08:46:41.295780+0000","last_fullsized":"2024-09-19T08:46:41.295780+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:55:04.131994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295871+0000","last_change":"2024-09-19T08:46:40.313527+0000","last_active":"2024-09-19T08:46:41.295871+0000","last_peered":"2024-09-19T08:46:41.295871+0000","last_clean":"2024-09-19T08:46:41.295871+0000","last_became_active":"2024-09-19T08:46:40.312733+0000","last_became_peered":"2024-09-19T08:46:40.312733+0000","last_unstale":"2024-09-19T08:46:41.295871+0000","last_undegraded":"2024-09-19T08:46:41.295871+0000","last_fullsized":"2024-09-19T08:46:41.295871+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:30.018530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.295815+0000","last_change":"2024-09-19T08:46:39.533782+0000","last_active":"2024-09-19T08:46:41.295815+0000","last_peered":"2024-09-19T08:46:41.295815+0000","last_clean":"2024-09-19T08:46:41.295815+0000","last_became_active":"2024-09-19T08:46:39.533401+0000","last_became_peered":"2024-09-19T08:46:39.533401+0000","last_unstale":"2024-09-19T08:46:41.295815+0000","last_undegraded":"2024-09-19T08:46:41.295815+0000","last_fullsized":"2024-09-19T08:46:41.295815+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:14:49.514714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.298822+0000","last_change":"2024-09-19T08:46:40.317873+0000","last_active":"2024-09-19T08:46:41.298822+0000","last_peered":"2024-09-19T08:46:41.298822+0000","last_clean":"2024-09-19T08:46:41.298822+0000","last_became_active":"2024-09-19T08:46:40.316906+0000","last_became_peered":"2024-09-19T08:46:40.316906+0000","last_unstale":"2024-09-19T08:46:41.298822+0000","last_undegraded":"2024-09-19T08:46:41.298822+0000","last_fullsized":"2024-09-19T08:46:41.298822+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:08.299341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.301923+0000","last_change":"2024-09-19T08:46:39.521611+0000","last_active":"2024-09-19T08:46:41.301923+0000","last_peered":"2024-09-19T08:46:41.301923+0000","last_clean":"2024-09-19T08:46:41.301923+0000","last_became_active":"2024-09-19T08:46:39.520863+0000","last_became_peered":"2024-09-19T08:46:39.520863+0000","last_unstale":"2024-09-19T08:46:41.301923+0000","last_undegraded":"2024-09-19T08:46:41.301923+0000","last_fullsized":"2024-09-19T08:46:41.301923+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:53:19.589458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.298638+0000","last_change":"2024-09-19T08:46:40.317455+0000","last_active":"2024-09-19T08:46:41.298638+0000","last_peered":"2024-09-19T08:46:41.298638+0000","last_clean":"2024-09-19T08:46:41.298638+0000","last_became_active":"2024-09-19T08:46:40.317109+0000","last_became_peered":"2024-09-19T08:46:40.317109+0000","last_unstale":"2024-09-19T08:46:41.298638+0000","last_undegraded":"2024-09-19T08:46:41.298638+0000","last_fullsized":"2024-09-19T08:46:41.298638+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:06.068010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.301669+0000","last_change":"2024-09-19T08:46:39.536563+0000","last_active":"2024-09-19T08:46:41.301669+0000","last_peered":"2024-09-19T08:46:41.301669+0000","last_clean":"2024-09-19T08:46:41.301669+0000","last_became_active":"2024-09-19T08:46:39.535725+0000","last_became_peered":"2024-09-19T08:46:39.535725+0000","last_unstale":"2024-09-19T08:46:41.301669+0000","last_undegraded":"2024-09-19T08:46:41.301669+0000","last_fullsized":"2024-09-19T08:46:41.301669+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:17:57.007595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.274677+0000","last_change":"2024-09-19T08:46:40.300496+0000","last_active":"2024-09-19T08:46:41.274677+0000","last_peered":"2024-09-19T08:46:41.274677+0000","last_clean":"2024-09-19T08:46:41.274677+0000","last_became_active":"2024-09-19T08:46:40.299999+0000","last_became_peered":"2024-09-19T08:46:40.299999+0000","last_unstale":"2024-09-19T08:46:41.274677+0000","last_undegraded":"2024-09-19T08:46:41.274677+0000","last_fullsized":"2024-09-19T08:46:41.274677+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:57:03.311516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.274968+0000","last_change":"2024-09-19T08:46:39.539941+0000","last_active":"2024-09-19T08:46:41.274968+0000","last_peered":"2024-09-19T08:46:41.274968+0000","last_clean":"2024-09-19T08:46:41.274968+0000","last_became_active":"2024-09-19T08:46:39.539225+0000","last_became_peered":"2024-09-19T08:46:39.539225+0000","last_unstale":"2024-09-19T08:46:41.274968+0000","last_undegraded":"2024-09-19T08:46:41.274968+0000","last_fullsized":"2024-09-19T08:46:41.274968+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:46:06.121419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.271306+0000","last_change":"2024-09-19T08:46:40.293254+0000","last_active":"2024-09-19T08:46:41.271306+0000","last_peered":"2024-09-19T08:46:41.271306+0000","last_clean":"2024-09-19T08:46:41.271306+0000","last_became_active":"2024-09-19T08:46:40.292460+0000","last_became_peered":"2024-09-19T08:46:40.292460+0000","last_unstale":"2024-09-19T08:46:41.271306+0000","last_undegraded":"2024-09-19T08:46:41.271306+0000","last_fullsized":"2024-09-19T08:46:41.271306+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:36:48.171899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.293216+0000","last_change":"2024-09-19T08:46:39.529973+0000","last_active":"2024-09-19T08:46:41.293216+0000","last_peered":"2024-09-19T08:46:41.293216+0000","last_clean":"2024-09-19T08:46:41.293216+0000","last_became_active":"2024-09-19T08:46:39.529691+0000","last_became_peered":"2024-09-19T08:46:39.529691+0000","last_unstale":"2024-09-19T08:46:41.293216+0000","last_undegraded":"2024-09-19T08:46:41.293216+0000","last_fullsized":"2024-09-19T08:46:41.293216+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:10:34.575229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.293890+0000","last_change":"2024-09-19T08:46:40.292254+0000","last_active":"2024-09-19T08:46:41.293890+0000","last_peered":"2024-09-19T08:46:41.293890+0000","last_clean":"2024-09-19T08:46:41.293890+0000","last_became_active":"2024-09-19T08:46:40.290492+0000","last_became_peered":"2024-09-19T08:46:40.290492+0000","last_unstale":"2024-09-19T08:46:41.293890+0000","last_undegraded":"2024-09-19T08:46:41.293890+0000","last_fullsized":"2024-09-19T08:46:41.293890+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:39.484150+0000","last_clean_scrub_stamp":"2024-09-19T08:46:39.484150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:14:04.162980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.298903+0000","last_change":"2024-09-19T08:46:39.527908+0000","last_active":"2024-09-19T08:46:41.298903+0000","last_peered":"2024-09-19T08:46:41.298903+0000","last_clean":"2024-09-19T08:46:41.298903+0000","last_became_active":"2024-09-19T08:46:39.527338+0000","last_became_peered":"2024-09-19T08:46:39.527338+0000","last_unstale":"2024-09-19T08:46:41.298903+0000","last_undegraded":"2024-09-19T08:46:41.298903+0000","last_fullsized":"2024-09-19T08:46:41.298903+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:46:38.474849+0000","last_clean_scrub_stamp":"2024-09-19T08:46:38.474849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:42.770912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"71'128","reported_seq":457,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-19T08:46:41.298973+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:46:41.298973+0000","last_peered":"2024-09-19T08:46:41.298973+0000","last_clean":"2024-09-19T08:46:41.298973+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:46:41.298973+0000","last_undegraded":"2024-09-19T08:46:41.298973+0000","last_fullsized":"2024-09-19T08:46:41.298973+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509547,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33724,"kb_used_data":2524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338116,"statfs":{"total":96636764160,"available":96602230784,"internally_reserved":0,"allocated":2584576,"data_stored":1471511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574956,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33128,"kb_used_data":1936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338712,"statfs":{"total":96636764160,"available":96602841088,"internally_reserved":0,"allocated":1982464,"data_stored":881039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509549,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33136,"kb_used_data":1936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338704,"statfs":{"total":96636764160,"available":96602832896,"internally_reserved":0,"allocated":1982464,"data_stored":881039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509549,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33140,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338700,"statfs":{"total":96636764160,"available":96602828800,"internally_reserved":0,"allocated":1998848,"data_stored":883233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542251,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37296,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334544,"statfs":{"total":96636764160,"available":96598573056,"internally_reserved":0,"allocated":1994752,"data_stored":881233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509546,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33144,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338696,"statfs":{"total":96636764160,"available":96602824704,"internally_reserved":0,"allocated":1994752,"data_stored":881733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509548,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33144,"kb_used_data":1948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338696,"statfs":{"total":96636764160,"available":96602824704,"internally_reserved":0,"allocated":1994752,"data_stored":882651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509546,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37876,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333964,"statfs":{"total":96636764160,"available":96597979136,"internally_reserved":0,"allocated":2580480,"data_stored":1471429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:46:49.348 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T08:46:49.349 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-19T08:46:49.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.518+0000 7ff89b7b7640 1 Processor -- start 2024-09-19T08:46:49.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.518+0000 7ff89b7b7640 1 -- start start 2024-09-19T08:46:49.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.518+0000 7ff89b7b7640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff894145b70 0x7ff89406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:49.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.518+0000 7ff89b7b7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff89406c1d0 0x7ff8940713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:49.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.518+0000 7ff89b7b7640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff894073530 con 0x7ff8940718f0 2024-09-19T08:46:49.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.518+0000 7ff89b7b7640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7ff89406c620 con 0x7ff894145b70 2024-09-19T08:46:49.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.518+0000 7ff89b7b7640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff894073810 con 0x7ff89406c1d0 2024-09-19T08:46:49.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.518+0000 7ff899553640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff894145b70 0x7ff89406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:49.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.518+0000 7ff899d54640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7ff8940718f0 0x7ff8940cef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:53046/0 (socket says 172.21.15.98:53046) 2024-09-19T08:46:49.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.518+0000 7ff899d54640 1 -- 172.21.15.98:0/2575208906 learned_addr learned my addr 172.21.15.98:0/2575208906 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:49.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.518+0000 7ff898d52640 1 --2- 172.21.15.98:0/2575208906 >> v2:172.21.15.139:3300/0 conn(0x7ff89406c1d0 0x7ff8940713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:49.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.518+0000 7ff899553640 1 -- 172.21.15.98:0/2575208906 >> v2:172.21.15.139:3300/0 conn(0x7ff89406c1d0 msgr2=0x7ff8940713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:49.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.518+0000 7ff899553640 1 --2- 172.21.15.98:0/2575208906 >> v2:172.21.15.139:3300/0 conn(0x7ff89406c1d0 0x7ff8940713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:49.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.518+0000 7ff899553640 1 -- 172.21.15.98:0/2575208906 >> v1:172.21.15.98:6789/0 conn(0x7ff8940718f0 legacy=0x7ff8940cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:49.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.518+0000 7ff899553640 1 -- 172.21.15.98:0/2575208906 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff894073af0 con 0x7ff894145b70 2024-09-19T08:46:49.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.518+0000 7ff898d52640 1 --2- 172.21.15.98:0/2575208906 >> v2:172.21.15.139:3300/0 conn(0x7ff89406c1d0 0x7ff8940713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:49.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.522+0000 7ff899553640 1 --2- 172.21.15.98:0/2575208906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff894145b70 0x7ff89406bc90 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7ff8940ccd30 tx=0x7ff88801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1855d072f2ad004 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:49.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.522+0000 7ff887fff640 1 -- 172.21.15.98:0/2575208906 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8880b0070 con 0x7ff894145b70 2024-09-19T08:46:49.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.522+0000 7ff887fff640 1 -- 172.21.15.98:0/2575208906 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8880ab600 con 0x7ff894145b70 2024-09-19T08:46:49.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.522+0000 7ff887fff640 1 -- 172.21.15.98:0/2575208906 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8880ab9d0 con 0x7ff894145b70 2024-09-19T08:46:49.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.522+0000 7ff89b7b7640 1 -- 172.21.15.98:0/2575208906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff894145b70 msgr2=0x7ff89406bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:49.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.522+0000 7ff89b7b7640 1 --2- 172.21.15.98:0/2575208906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff894145b70 0x7ff89406bc90 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7ff8940ccd30 tx=0x7ff88801c3f0 comp rx=0 tx=0).stop 2024-09-19T08:46:49.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.522+0000 7ff89b7b7640 1 -- 172.21.15.98:0/2575208906 shutdown_connections 2024-09-19T08:46:49.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.522+0000 7ff89b7b7640 1 --2- 172.21.15.98:0/2575208906 >> v2:172.21.15.139:3300/0 conn(0x7ff89406c1d0 0x7ff8940713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:49.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.522+0000 7ff89b7b7640 1 --2- 172.21.15.98:0/2575208906 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff894145b70 0x7ff89406bc90 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:49.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.522+0000 7ff89b7b7640 1 -- 172.21.15.98:0/2575208906 >> 172.21.15.98:0/2575208906 conn(0x7ff8940b2010 msgr2=0x7ff8940b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:49.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.522+0000 7ff89b7b7640 1 -- 172.21.15.98:0/2575208906 shutdown_connections 2024-09-19T08:46:49.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.522+0000 7ff89b7b7640 1 -- 172.21.15.98:0/2575208906 wait complete. 2024-09-19T08:46:49.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.522+0000 7ff89b7b7640 1 Processor -- start 2024-09-19T08:46:49.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.522+0000 7ff89b7b7640 1 -- start start 2024-09-19T08:46:49.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.526+0000 7ff89b7b7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff89406c1d0 0x7ff89415fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:49.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.526+0000 7ff89b7b7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8940718f0 0x7ff89415ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:49.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.526+0000 7ff89b7b7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff894145b70 0x7ff894161010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:49.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.526+0000 7ff89b7b7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff894169070 con 0x7ff89406c1d0 2024-09-19T08:46:49.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.526+0000 7ff89b7b7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff8941691f0 con 0x7ff8940718f0 2024-09-19T08:46:49.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.526+0000 7ff89b7b7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff894169370 con 0x7ff894145b70 2024-09-19T08:46:49.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.526+0000 7ff899553640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff89406c1d0 0x7ff89415fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:49.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.526+0000 7ff899553640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff89406c1d0 0x7ff89415fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54310/0 (socket says 172.21.15.98:54310) 2024-09-19T08:46:49.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.526+0000 7ff899553640 1 -- 172.21.15.98:0/699187112 learned_addr learned my addr 172.21.15.98:0/699187112 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:49.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.526+0000 7ff898d52640 1 --2- 172.21.15.98:0/699187112 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8940718f0 0x7ff89415ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:49.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.526+0000 7ff899d54640 1 --2- 172.21.15.98:0/699187112 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff894145b70 0x7ff894161010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:49.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.526+0000 7ff899553640 1 -- 172.21.15.98:0/699187112 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff894145b70 msgr2=0x7ff894161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:49.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.526+0000 7ff899553640 1 --2- 172.21.15.98:0/699187112 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff894145b70 0x7ff894161010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:49.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.526+0000 7ff899553640 1 -- 172.21.15.98:0/699187112 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8940718f0 msgr2=0x7ff89415ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:49.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.526+0000 7ff899553640 1 --2- 172.21.15.98:0/699187112 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8940718f0 0x7ff89415ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:49.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.526+0000 7ff899553640 1 -- 172.21.15.98:0/699187112 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8941629e0 con 0x7ff89406c1d0 2024-09-19T08:46:49.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.526+0000 7ff898d52640 1 --2- 172.21.15.98:0/699187112 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8940718f0 0x7ff89415ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:49.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.526+0000 7ff899d54640 1 --2- 172.21.15.98:0/699187112 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff894145b70 0x7ff894161010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:49.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.526+0000 7ff899553640 1 --2- 172.21.15.98:0/699187112 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff89406c1d0 0x7ff89415fe20 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7ff8880ae040 tx=0x7ff888019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:49.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.526+0000 7ff885ffb640 1 -- 172.21.15.98:0/699187112 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8880b8020 con 0x7ff89406c1d0 2024-09-19T08:46:49.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.526+0000 7ff885ffb640 1 -- 172.21.15.98:0/699187112 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8880b4610 con 0x7ff89406c1d0 2024-09-19T08:46:49.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.526+0000 7ff89b7b7640 1 -- 172.21.15.98:0/699187112 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff894162c80 con 0x7ff89406c1d0 2024-09-19T08:46:49.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.526+0000 7ff885ffb640 1 -- 172.21.15.98:0/699187112 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8880b0040 con 0x7ff89406c1d0 2024-09-19T08:46:49.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.526+0000 7ff89b7b7640 1 -- 172.21.15.98:0/699187112 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff894163240 con 0x7ff89406c1d0 2024-09-19T08:46:49.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.526+0000 7ff89b7b7640 1 -- 172.21.15.98:0/699187112 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8940713b0 con 0x7ff89406c1d0 2024-09-19T08:46:49.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.526+0000 7ff885ffb640 1 -- 172.21.15.98:0/699187112 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7ff8880bb020 con 0x7ff89406c1d0 2024-09-19T08:46:49.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.526+0000 7ff885ffb640 1 --2- 172.21.15.98:0/699187112 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff8780680a0 0x7ff87806a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:49.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.526+0000 7ff885ffb640 1 -- 172.21.15.98:0/699187112 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5737+0+0 (secure 0 0 0) 0x7ff88811e360 con 0x7ff89406c1d0 2024-09-19T08:46:49.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.526+0000 7ff898d52640 1 --2- 172.21.15.98:0/699187112 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff8780680a0 0x7ff87806a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:49.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.530+0000 7ff898d52640 1 --2- 172.21.15.98:0/699187112 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff8780680a0 0x7ff87806a560 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff8941f93f0 tx=0x7ff890008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:49.531 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.530+0000 7ff885ffb640 1 -- 172.21.15.98:0/699187112 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff8880c2030 con 0x7ff89406c1d0 2024-09-19T08:46:49.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.754+0000 7ff89b7b7640 1 -- 172.21.15.98:0/699187112 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7ff8940ccae0 con 0x7ff89406c1d0 2024-09-19T08:46:49.758 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.758+0000 7ff885ffb640 1 -- 172.21.15.98:0/699187112 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v25) ==== 190+0+147 (secure 0 0 0) 0x7ff888016070 con 0x7ff89406c1d0 2024-09-19T08:46:49.759 INFO:teuthology.orchestra.run.smithi098.stdout:[client.0] 2024-09-19T08:46:49.759 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQC/4utm0vNpIhAA4S1E8dFZ0+ibmSKhmZ+bIA== 2024-09-19T08:46:49.759 INFO:teuthology.orchestra.run.smithi098.stdout: caps mds = "allow" 2024-09-19T08:46:49.759 INFO:teuthology.orchestra.run.smithi098.stdout: caps mon = "allow r" 2024-09-19T08:46:49.759 INFO:teuthology.orchestra.run.smithi098.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-19T08:46:49.759 INFO:teuthology.orchestra.run.smithi098.stderr:updated caps for client.0 2024-09-19T08:46:49.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.762+0000 7ff89b7b7640 1 -- 172.21.15.98:0/699187112 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff8780680a0 msgr2=0x7ff87806a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:49.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.762+0000 7ff89b7b7640 1 --2- 172.21.15.98:0/699187112 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff8780680a0 0x7ff87806a560 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff8941f93f0 tx=0x7ff890008040 comp rx=0 tx=0).stop 2024-09-19T08:46:49.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.762+0000 7ff89b7b7640 1 -- 172.21.15.98:0/699187112 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff89406c1d0 msgr2=0x7ff89415fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:49.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.762+0000 7ff89b7b7640 1 --2- 172.21.15.98:0/699187112 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff89406c1d0 0x7ff89415fe20 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7ff8880ae040 tx=0x7ff888019190 comp rx=0 tx=0).stop 2024-09-19T08:46:49.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.762+0000 7ff89b7b7640 1 -- 172.21.15.98:0/699187112 shutdown_connections 2024-09-19T08:46:49.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.762+0000 7ff89b7b7640 1 --2- 172.21.15.98:0/699187112 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff8780680a0 0x7ff87806a560 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:49.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.762+0000 7ff89b7b7640 1 --2- 172.21.15.98:0/699187112 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff894145b70 0x7ff894161010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:49.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.762+0000 7ff89b7b7640 1 --2- 172.21.15.98:0/699187112 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff8940718f0 0x7ff89415ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:49.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.762+0000 7ff89b7b7640 1 --2- 172.21.15.98:0/699187112 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff89406c1d0 0x7ff89415fe20 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:49.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.762+0000 7ff89b7b7640 1 -- 172.21.15.98:0/699187112 >> 172.21.15.98:0/699187112 conn(0x7ff8940b2010 msgr2=0x7ff894142900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:49.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.762+0000 7ff89b7b7640 1 -- 172.21.15.98:0/699187112 shutdown_connections 2024-09-19T08:46:49.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.762+0000 7ff89b7b7640 1 -- 172.21.15.98:0/699187112 wait complete. 2024-09-19T08:46:49.778 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:46:49.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.898+0000 7f2ef54a1640 1 Processor -- start 2024-09-19T08:46:49.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.898+0000 7f2ef54a1640 1 -- start start 2024-09-19T08:46:49.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.898+0000 7f2ef54a1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2ef006d880 0x7f2ef00b3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:49.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.898+0000 7f2ef54a1640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2ef0141110 0x7f2ef00b3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:49.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.898+0000 7f2ef54a1640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ef015ea10 con 0x7f2ef0140d30 2024-09-19T08:46:49.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.898+0000 7f2ef54a1640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f2ef015e7c0 con 0x7f2ef0141110 2024-09-19T08:46:49.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.898+0000 7f2ef54a1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f2ef015ecf0 con 0x7f2ef006d880 2024-09-19T08:46:49.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.898+0000 7f2eef7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2ef0141110 0x7f2ef00b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:49.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.898+0000 7f2eef7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f2ef0141110 0x7f2ef00b3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54326/0 (socket says 172.21.15.98:54326) 2024-09-19T08:46:49.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.898+0000 7f2eef7fe640 1 -- 172.21.15.98:0/2649008340 learned_addr learned my addr 172.21.15.98:0/2649008340 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:49.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.898+0000 7f2eee7fc640 1 --2- 172.21.15.98:0/2649008340 >> v2:172.21.15.139:3300/0 conn(0x7f2ef006d880 0x7f2ef00b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:49.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.898+0000 7f2eef7fe640 1 -- 172.21.15.98:0/2649008340 >> v2:172.21.15.139:3300/0 conn(0x7f2ef006d880 msgr2=0x7f2ef00b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:49.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.898+0000 7f2eef7fe640 1 --2- 172.21.15.98:0/2649008340 >> v2:172.21.15.139:3300/0 conn(0x7f2ef006d880 0x7f2ef00b3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:49.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.898+0000 7f2eef7fe640 1 -- 172.21.15.98:0/2649008340 >> v1:172.21.15.139:6789/0 conn(0x7f2ef0140d30 legacy=0x7f2ef006d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:49.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.898+0000 7f2eef7fe640 1 -- 172.21.15.98:0/2649008340 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef015efd0 con 0x7f2ef0141110 2024-09-19T08:46:49.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.898+0000 7f2eef7fe640 1 --2- 172.21.15.98:0/2649008340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ef0141110 0x7f2ef00b3850 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f2ef006e040 tx=0x7f2ee801dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1cbe9101d61502 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:49.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.902+0000 7f2eedffb640 1 -- 172.21.15.98:0/2649008340 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee80a6940 con 0x7f2ef0141110 2024-09-19T08:46:49.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.902+0000 7f2eedffb640 1 -- 172.21.15.98:0/2649008340 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ee80a6ae0 con 0x7f2ef0141110 2024-09-19T08:46:49.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.902+0000 7f2eedffb640 1 -- 172.21.15.98:0/2649008340 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee80ae3f0 con 0x7f2ef0141110 2024-09-19T08:46:49.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.902+0000 7f2ef54a1640 1 -- 172.21.15.98:0/2649008340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ef0141110 msgr2=0x7f2ef00b3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:49.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.902+0000 7f2ef54a1640 1 --2- 172.21.15.98:0/2649008340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ef0141110 0x7f2ef00b3850 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f2ef006e040 tx=0x7f2ee801dac0 comp rx=0 tx=0).stop 2024-09-19T08:46:49.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.902+0000 7f2ef54a1640 1 -- 172.21.15.98:0/2649008340 shutdown_connections 2024-09-19T08:46:49.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.902+0000 7f2ef54a1640 1 --2- 172.21.15.98:0/2649008340 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ef0141110 0x7f2ef00b3850 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:49.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.902+0000 7f2ef54a1640 1 --2- 172.21.15.98:0/2649008340 >> v2:172.21.15.139:3300/0 conn(0x7f2ef006d880 0x7f2ef00b3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:49.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.902+0000 7f2ef54a1640 1 -- 172.21.15.98:0/2649008340 >> 172.21.15.98:0/2649008340 conn(0x7f2ef00b2010 msgr2=0x7f2ef00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:49.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.902+0000 7f2ef54a1640 1 -- 172.21.15.98:0/2649008340 shutdown_connections 2024-09-19T08:46:49.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.902+0000 7f2ef54a1640 1 -- 172.21.15.98:0/2649008340 wait complete. 2024-09-19T08:46:49.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.902+0000 7f2ef54a1640 1 Processor -- start 2024-09-19T08:46:49.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.902+0000 7f2ef54a1640 1 -- start start 2024-09-19T08:46:49.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.902+0000 7f2ef54a1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ef006d880 0x7f2ef013fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:49.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.902+0000 7f2ef54a1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ef0140d30 0x7f2ef0073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:49.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.902+0000 7f2ef54a1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ef0141110 0x7f2ef015c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:49.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.902+0000 7f2ef54a1640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef015b580 con 0x7f2ef0141110 2024-09-19T08:46:49.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.902+0000 7f2ef54a1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef015b400 con 0x7f2ef006d880 2024-09-19T08:46:49.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.902+0000 7f2ef54a1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2ef015b700 con 0x7f2ef0140d30 2024-09-19T08:46:49.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.902+0000 7f2eef7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ef0141110 0x7f2ef015c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:49.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.902+0000 7f2eef7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ef0141110 0x7f2ef015c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54336/0 (socket says 172.21.15.98:54336) 2024-09-19T08:46:49.907 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.902+0000 7f2eef7fe640 1 -- 172.21.15.98:0/2096403944 learned_addr learned my addr 172.21.15.98:0/2096403944 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:49.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.902+0000 7f2eeeffd640 1 --2- 172.21.15.98:0/2096403944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ef006d880 0x7f2ef013fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:49.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.902+0000 7f2eee7fc640 1 --2- 172.21.15.98:0/2096403944 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ef0140d30 0x7f2ef0073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:49.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.902+0000 7f2eef7fe640 1 -- 172.21.15.98:0/2096403944 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ef0140d30 msgr2=0x7f2ef0073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:49.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.902+0000 7f2eef7fe640 1 --2- 172.21.15.98:0/2096403944 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ef0140d30 0x7f2ef0073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:49.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.902+0000 7f2eef7fe640 1 -- 172.21.15.98:0/2096403944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ef006d880 msgr2=0x7f2ef013fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:49.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.902+0000 7f2eef7fe640 1 --2- 172.21.15.98:0/2096403944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ef006d880 0x7f2ef013fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:49.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.902+0000 7f2eef7fe640 1 -- 172.21.15.98:0/2096403944 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef0072270 con 0x7f2ef0141110 2024-09-19T08:46:49.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.902+0000 7f2eee7fc640 1 --2- 172.21.15.98:0/2096403944 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ef0140d30 0x7f2ef0073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:49.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.902+0000 7f2eef7fe640 1 --2- 172.21.15.98:0/2096403944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ef0141110 0x7f2ef015c840 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f2ee801b6f0 tx=0x7f2ee801c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:49.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.902+0000 7f2ed77fe640 1 -- 172.21.15.98:0/2096403944 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee80a6940 con 0x7f2ef0141110 2024-09-19T08:46:49.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.902+0000 7f2ed77fe640 1 -- 172.21.15.98:0/2096403944 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ee80bd070 con 0x7f2ef0141110 2024-09-19T08:46:49.908 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.902+0000 7f2ef54a1640 1 -- 172.21.15.98:0/2096403944 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ef0072550 con 0x7f2ef0141110 2024-09-19T08:46:49.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.902+0000 7f2ed77fe640 1 -- 172.21.15.98:0/2096403944 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8009510 con 0x7f2ef0141110 2024-09-19T08:46:49.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.902+0000 7f2ef54a1640 1 -- 172.21.15.98:0/2096403944 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ef0140360 con 0x7f2ef0141110 2024-09-19T08:46:49.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.906+0000 7f2ef54a1640 1 -- 172.21.15.98:0/2096403944 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ef00008e0 con 0x7f2ef0141110 2024-09-19T08:46:49.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.906+0000 7f2ed77fe640 1 -- 172.21.15.98:0/2096403944 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f2ee80b8660 con 0x7f2ef0141110 2024-09-19T08:46:49.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.906+0000 7f2ed77fe640 1 --2- 172.21.15.98:0/2096403944 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2edc0680d0 0x7f2edc06a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:49.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.906+0000 7f2ed77fe640 1 -- 172.21.15.98:0/2096403944 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5737+0+0 (secure 0 0 0) 0x7f2ee811f1c0 con 0x7f2ef0141110 2024-09-19T08:46:49.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.906+0000 7f2eeeffd640 1 --2- 172.21.15.98:0/2096403944 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2edc0680d0 0x7f2edc06a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:49.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.906+0000 7f2eeeffd640 1 --2- 172.21.15.98:0/2096403944 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2edc0680d0 0x7f2edc06a590 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4003d60 tx=0x7f2ee4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:49.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:49.910+0000 7f2ed77fe640 1 -- 172.21.15.98:0/2096403944 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2ee80176b0 con 0x7f2ef0141110 2024-09-19T08:46:50.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.094+0000 7f2ef54a1640 1 -- 172.21.15.98:0/2096403944 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ef00757a0 con 0x7f2ef0141110 2024-09-19T08:46:50.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.094+0000 7f2ed77fe640 1 -- 172.21.15.98:0/2096403944 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5233 (secure 0 0 0) 0x7f2ee80f1710 con 0x7f2ef0141110 2024-09-19T08:46:50.094 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:46:50.094 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":91,"btime":"2024-09-19T08:46:43:523818+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/1752784745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":1752784745},{"type":"v1","addr":"172.21.15.139:6834","nonce":1752784745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":6212,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3564493643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6833","nonce":3564493643},{"type":"v1","addr":"172.21.15.139:6835","nonce":3564493643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":8763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6834/3374714194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3374714194},{"type":"v1","addr":"172.21.15.98:6834","nonce":3374714194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:46:39.868735+0000","modified":"2024-09-19T08:46:42.899681+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8766},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8766":{"gid":8766,"name":"c","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/3734544897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6833","nonce":3734544897},{"type":"v1","addr":"172.21.15.98:6835","nonce":3734544897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8766,"qdb_cluster":[8766]},"id":12}]} 2024-09-19T08:46:50.094 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 91 2024-09-19T08:46:50.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.094+0000 7f2ef54a1640 1 -- 172.21.15.98:0/2096403944 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2edc0680d0 msgr2=0x7f2edc06a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:50.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.094+0000 7f2ef54a1640 1 --2- 172.21.15.98:0/2096403944 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2edc0680d0 0x7f2edc06a590 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4003d60 tx=0x7f2ee4009040 comp rx=0 tx=0).stop 2024-09-19T08:46:50.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.098+0000 7f2ef54a1640 1 -- 172.21.15.98:0/2096403944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ef0141110 msgr2=0x7f2ef015c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:50.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.098+0000 7f2ef54a1640 1 --2- 172.21.15.98:0/2096403944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ef0141110 0x7f2ef015c840 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f2ee801b6f0 tx=0x7f2ee801c300 comp rx=0 tx=0).stop 2024-09-19T08:46:50.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.098+0000 7f2ef54a1640 1 -- 172.21.15.98:0/2096403944 shutdown_connections 2024-09-19T08:46:50.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.098+0000 7f2ef54a1640 1 --2- 172.21.15.98:0/2096403944 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2edc0680d0 0x7f2edc06a590 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.098+0000 7f2ef54a1640 1 --2- 172.21.15.98:0/2096403944 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ef0141110 0x7f2ef015c840 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.098+0000 7f2ef54a1640 1 --2- 172.21.15.98:0/2096403944 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ef0140d30 0x7f2ef0073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.098+0000 7f2ef54a1640 1 --2- 172.21.15.98:0/2096403944 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ef006d880 0x7f2ef013fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.098+0000 7f2ef54a1640 1 -- 172.21.15.98:0/2096403944 >> 172.21.15.98:0/2096403944 conn(0x7f2ef00b2010 msgr2=0x7f2ef0158440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:50.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.098+0000 7f2ef54a1640 1 -- 172.21.15.98:0/2096403944 shutdown_connections 2024-09-19T08:46:50.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.098+0000 7f2ef54a1640 1 -- 172.21.15.98:0/2096403944 wait complete. 2024-09-19T08:46:50.514 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 91, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T08:46:39.868735+0000', 'modified': '2024-09-19T08:46:42.899681+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8766}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8766': {'gid': 8766, 'name': 'c', 'rank': 0, 'incarnation': 87, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.98:6835/3734544897', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6833', 'nonce': 3734544897}, {'type': 'v1', 'addr': '172.21.15.98:6835', 'nonce': 3734544897}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [25], 'metadata_pool': 24, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8766, 'qdb_cluster': [8766]} 2024-09-19T08:46:50.514 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T08:46:50.514 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-19T08:46:50.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.646+0000 7f22e1fac640 1 Processor -- start 2024-09-19T08:46:50.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.646+0000 7f22e1fac640 1 -- start start 2024-09-19T08:46:50.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.650+0000 7f22e1fac640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22dc15e650 0x7f22dc06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:50.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.650+0000 7f22e1fac640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22dc162fc0 con 0x7f22dc0d2e00 2024-09-19T08:46:50.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.650+0000 7f22e1fac640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22dc06c1d0 con 0x7f22dc070ec0 2024-09-19T08:46:50.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.650+0000 7f22e1fac640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f22dc162440 con 0x7f22dc15e650 2024-09-19T08:46:50.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.650+0000 7f22daffd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f22dc0d2e00 0x7f22dc0707b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:53054/0 (socket says 172.21.15.98:53054) 2024-09-19T08:46:50.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.650+0000 7f22daffd640 1 -- 172.21.15.98:0/2095172971 learned_addr learned my addr 172.21.15.98:0/2095172971 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:50.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.650+0000 7f22db7fe640 1 --2- 172.21.15.98:0/2095172971 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22dc15e650 0x7f22dc06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:50.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.650+0000 7f22da7fc640 1 -- 172.21.15.98:0/2095172971 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2735548453 0 0) 0x7f22dc162fc0 con 0x7f22dc0d2e00 2024-09-19T08:46:50.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.650+0000 7f22da7fc640 1 -- 172.21.15.98:0/2095172971 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22c8003530 con 0x7f22dc0d2e00 2024-09-19T08:46:50.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.650+0000 7f22da7fc640 1 -- 172.21.15.98:0/2095172971 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 501771948 0 0) 0x7f22c8003530 con 0x7f22dc0d2e00 2024-09-19T08:46:50.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.650+0000 7f22da7fc640 1 -- 172.21.15.98:0/2095172971 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22dc15e650 msgr2=0x7f22dc06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:50.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.650+0000 7f22da7fc640 1 --2- 172.21.15.98:0/2095172971 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22dc15e650 0x7f22dc06bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.650+0000 7f22da7fc640 1 -- 172.21.15.98:0/2095172971 >> v1:172.21.15.139:6789/0 conn(0x7f22dc070ec0 legacy=0x7f22dc073310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:50.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.650+0000 7f22da7fc640 1 -- 172.21.15.98:0/2095172971 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22dc162720 con 0x7f22dc0d2e00 2024-09-19T08:46:50.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.650+0000 7f22da7fc640 1 -- 172.21.15.98:0/2095172971 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f22c4001a00 con 0x7f22dc0d2e00 2024-09-19T08:46:50.651 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.650+0000 7f22da7fc640 1 -- 172.21.15.98:0/2095172971 >> v1:172.21.15.98:6789/0 conn(0x7f22dc0d2e00 legacy=0x7f22dc0707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:50.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.650+0000 7f22da7fc640 1 --2- 172.21.15.98:0/2095172971 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22dc15e650 0x7f22c80039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:50.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.650+0000 7f22da7fc640 1 --2- 172.21.15.98:0/2095172971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22c8003f70 0x7f22c8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:50.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.650+0000 7f22da7fc640 1 --2- 172.21.15.98:0/2095172971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22c8004d50 0x7f22c80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:50.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.650+0000 7f22da7fc640 1 -- 172.21.15.98:0/2095172971 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc162440 con 0x7f22c8004d50 2024-09-19T08:46:50.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.650+0000 7f22da7fc640 1 -- 172.21.15.98:0/2095172971 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f22c800bab0 con 0x7f22c8003f70 2024-09-19T08:46:50.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.650+0000 7f22da7fc640 1 -- 172.21.15.98:0/2095172971 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f22c800bcb0 con 0x7f22dc15e650 2024-09-19T08:46:50.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.650+0000 7f22dbfff640 1 --2- 172.21.15.98:0/2095172971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22c8004d50 0x7f22c80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:50.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.650+0000 7f22daffd640 1 --2- 172.21.15.98:0/2095172971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22c8003f70 0x7f22c8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:50.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.650+0000 7f22db7fe640 1 --2- 172.21.15.98:0/2095172971 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22dc15e650 0x7f22c80039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:50.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.650+0000 7f22dbfff640 1 -- 172.21.15.98:0/2095172971 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22dc15e650 msgr2=0x7f22c80039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:50.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.650+0000 7f22dbfff640 1 --2- 172.21.15.98:0/2095172971 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22dc15e650 0x7f22c80039f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.652 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.650+0000 7f22dbfff640 1 -- 172.21.15.98:0/2095172971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22c8003f70 msgr2=0x7f22c8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:50.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.650+0000 7f22dbfff640 1 --2- 172.21.15.98:0/2095172971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22c8003f70 0x7f22c8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.650+0000 7f22dbfff640 1 -- 172.21.15.98:0/2095172971 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22c8003530 con 0x7f22c8004d50 2024-09-19T08:46:50.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.654+0000 7f22dbfff640 1 --2- 172.21.15.98:0/2095172971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22c8004d50 0x7f22c80051d0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f22dc069910 tx=0x7f22d401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90556aed6e6ad0bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:50.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.654+0000 7f22da7fc640 1 -- 172.21.15.98:0/2095172971 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d40013a0 con 0x7f22c8004d50 2024-09-19T08:46:50.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.654+0000 7f22da7fc640 1 -- 172.21.15.98:0/2095172971 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22d40052f0 con 0x7f22c8004d50 2024-09-19T08:46:50.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.654+0000 7f22da7fc640 1 -- 172.21.15.98:0/2095172971 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d4003840 con 0x7f22c8004d50 2024-09-19T08:46:50.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.654+0000 7f22e1fac640 1 -- 172.21.15.98:0/2095172971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22c8004d50 msgr2=0x7f22c80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:50.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.654+0000 7f22e1fac640 1 --2- 172.21.15.98:0/2095172971 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22c8004d50 0x7f22c80051d0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f22dc069910 tx=0x7f22d401c380 comp rx=0 tx=0).stop 2024-09-19T08:46:50.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.654+0000 7f22dbfff640 1 -- 172.21.15.98:0/2095172971 reap_dead start 2024-09-19T08:46:50.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.654+0000 7f22e1fac640 1 -- 172.21.15.98:0/2095172971 shutdown_connections 2024-09-19T08:46:50.654 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.654+0000 7f22e1fac640 1 -- 172.21.15.98:0/2095172971 >> 172.21.15.98:0/2095172971 conn(0x7f22dc0d2310 msgr2=0x7f22dc0d26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:50.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.654+0000 7f22e1fac640 1 -- 172.21.15.98:0/2095172971 shutdown_connections 2024-09-19T08:46:50.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.654+0000 7f22e1fac640 1 -- 172.21.15.98:0/2095172971 wait complete. 2024-09-19T08:46:50.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.654+0000 7f22e1fac640 1 Processor -- start 2024-09-19T08:46:50.655 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.654+0000 7f22e1fac640 1 -- start start 2024-09-19T08:46:50.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.654+0000 7f22e1fac640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22c8004d50 0x7f22dc0ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:50.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.654+0000 7f22e1fac640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22dc158280 0x7f22dc157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:50.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.654+0000 7f22e1fac640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22dc0b0a80 0x7f22dc0b0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:50.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.654+0000 7f22e1fac640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc162060 con 0x7f22dc158280 2024-09-19T08:46:50.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.654+0000 7f22e1fac640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc162ce0 con 0x7f22c8004d50 2024-09-19T08:46:50.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.654+0000 7f22e1fac640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f22dc1621e0 con 0x7f22dc0b0a80 2024-09-19T08:46:50.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.654+0000 7f22daffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22dc158280 0x7f22dc157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:50.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.654+0000 7f22daffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22dc158280 0x7f22dc157a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54354/0 (socket says 172.21.15.98:54354) 2024-09-19T08:46:50.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.654+0000 7f22daffd640 1 -- 172.21.15.98:0/3583738702 learned_addr learned my addr 172.21.15.98:0/3583738702 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:50.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.654+0000 7f22db7fe640 1 --2- 172.21.15.98:0/3583738702 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22c8004d50 0x7f22dc0ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:50.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.654+0000 7f22dbfff640 1 --2- 172.21.15.98:0/3583738702 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22dc0b0a80 0x7f22dc0b0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:50.658 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.654+0000 7f22daffd640 1 -- 172.21.15.98:0/3583738702 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22dc0b0a80 msgr2=0x7f22dc0b0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:50.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.654+0000 7f22daffd640 1 --2- 172.21.15.98:0/3583738702 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22dc0b0a80 0x7f22dc0b0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.654+0000 7f22daffd640 1 -- 172.21.15.98:0/3583738702 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22c8004d50 msgr2=0x7f22dc0ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:50.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.654+0000 7f22daffd640 1 --2- 172.21.15.98:0/3583738702 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22c8004d50 0x7f22dc0ce4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.654+0000 7f22daffd640 1 -- 172.21.15.98:0/3583738702 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22dc157f70 con 0x7f22dc158280 2024-09-19T08:46:50.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.654+0000 7f22db7fe640 1 --2- 172.21.15.98:0/3583738702 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22c8004d50 0x7f22dc0ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:50.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.654+0000 7f22daffd640 1 --2- 172.21.15.98:0/3583738702 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22dc158280 0x7f22dc157a30 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f22c4007a40 tx=0x7f22c400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:50.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.654+0000 7f22d3fff640 1 -- 172.21.15.98:0/3583738702 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c400ee50 con 0x7f22dc158280 2024-09-19T08:46:50.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.654+0000 7f22d3fff640 1 -- 172.21.15.98:0/3583738702 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22c4018070 con 0x7f22dc158280 2024-09-19T08:46:50.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.654+0000 7f22e1fac640 1 -- 172.21.15.98:0/3583738702 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22dc15af60 con 0x7f22dc158280 2024-09-19T08:46:50.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.658+0000 7f22d3fff640 1 -- 172.21.15.98:0/3583738702 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c4013660 con 0x7f22dc158280 2024-09-19T08:46:50.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.658+0000 7f22e1fac640 1 -- 172.21.15.98:0/3583738702 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22dc15d390 con 0x7f22dc158280 2024-09-19T08:46:50.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.658+0000 7f22d3fff640 1 -- 172.21.15.98:0/3583738702 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f22c4013800 con 0x7f22dc158280 2024-09-19T08:46:50.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.658+0000 7f22d3fff640 1 --2- 172.21.15.98:0/3583738702 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f22bc0680d0 0x7f22bc06a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:50.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.658+0000 7f22d3fff640 1 -- 172.21.15.98:0/3583738702 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5737+0+0 (secure 0 0 0) 0x7f22c4084d00 con 0x7f22dc158280 2024-09-19T08:46:50.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.658+0000 7f22db7fe640 1 --2- 172.21.15.98:0/3583738702 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f22bc0680d0 0x7f22bc06a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:50.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.658+0000 7f22db7fe640 1 --2- 172.21.15.98:0/3583738702 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f22bc0680d0 0x7f22bc06a590 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f22cc009af0 tx=0x7f22cc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:50.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.658+0000 7f22e5068000 1 Processor -- start 2024-09-19T08:46:50.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.658+0000 7f22e5068000 1 -- start start 2024-09-19T08:46:50.666 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.658+0000 7f22e5068000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c363cccb20 0x55c363cccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:50.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.658+0000 7f22e5068000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c363ccd440 0x55c363c82d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:50.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.658+0000 7f22e5068000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c363c83290 0x55c363c83670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:50.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.658+0000 7f22e5068000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c363b6ab60 con 0x55c363ccd440 2024-09-19T08:46:50.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.658+0000 7f22e5068000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c363ad1c70 con 0x55c363c83290 2024-09-19T08:46:50.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.658+0000 7f22e5068000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x55c363a77ed0 con 0x55c363cccb20 2024-09-19T08:46:50.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22daffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c363ccd440 0x55c363c82d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:50.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22daffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c363ccd440 0x55c363c82d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54362/0 (socket says 172.21.15.98:54362) 2024-09-19T08:46:50.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22daffd640 1 -- 172.21.15.98:0/1735862697 learned_addr learned my addr 172.21.15.98:0/1735862697 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:50.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22dbfff640 1 --2- 172.21.15.98:0/1735862697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c363c83290 0x55c363c83670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:50.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22db7fe640 1 --2- 172.21.15.98:0/1735862697 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c363cccb20 0x55c363cccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:50.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22daffd640 1 -- 172.21.15.98:0/1735862697 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c363cccb20 msgr2=0x55c363cccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:50.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22daffd640 1 --2- 172.21.15.98:0/1735862697 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c363cccb20 0x55c363cccf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22daffd640 1 -- 172.21.15.98:0/1735862697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c363c83290 msgr2=0x55c363c83670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:50.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22daffd640 1 --2- 172.21.15.98:0/1735862697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c363c83290 0x55c363c83670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22daffd640 1 -- 172.21.15.98:0/1735862697 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c363a89e50 con 0x55c363ccd440 2024-09-19T08:46:50.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22daffd640 1 --2- 172.21.15.98:0/1735862697 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c363ccd440 0x55c363c82d50 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f22c40569c0 tx=0x7f22c4056480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e7b09de859f7fcc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:50.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22d17fa640 1 -- 172.21.15.98:0/1735862697 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c405f610 con 0x55c363ccd440 2024-09-19T08:46:50.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22d17fa640 1 -- 172.21.15.98:0/1735862697 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22c4011dc0 con 0x55c363ccd440 2024-09-19T08:46:50.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22d17fa640 1 -- 172.21.15.98:0/1735862697 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c405f180 con 0x55c363ccd440 2024-09-19T08:46:50.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22e5068000 1 -- 172.21.15.98:0/1735862697 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c363ccd440 msgr2=0x55c363c82d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:50.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22e5068000 1 --2- 172.21.15.98:0/1735862697 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c363ccd440 0x55c363c82d50 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f22c40569c0 tx=0x7f22c4056480 comp rx=0 tx=0).stop 2024-09-19T08:46:50.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22e5068000 1 -- 172.21.15.98:0/1735862697 shutdown_connections 2024-09-19T08:46:50.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22e5068000 1 --2- 172.21.15.98:0/1735862697 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c363c83290 0x55c363c83670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22e5068000 1 --2- 172.21.15.98:0/1735862697 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c363ccd440 0x55c363c82d50 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22e5068000 1 --2- 172.21.15.98:0/1735862697 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c363cccb20 0x55c363cccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22e5068000 1 -- 172.21.15.98:0/1735862697 >> 172.21.15.98:0/1735862697 conn(0x55c363c89600 msgr2=0x55c363c81630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:50.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22e5068000 1 -- 172.21.15.98:0/1735862697 shutdown_connections 2024-09-19T08:46:50.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22e5068000 1 -- 172.21.15.98:0/1735862697 wait complete. 2024-09-19T08:46:50.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22e5068000 1 -- start start 2024-09-19T08:46:50.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22e5068000 1 Processor -- start 2024-09-19T08:46:50.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22e5068000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c363cccb20 0x55c363c87e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:50.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22e5068000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c363ccd440 0x55c363c8dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:50.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22e5068000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c363c88390 0x55c363c8e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:50.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22e5068000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c363b33b00 con 0x55c363c88390 2024-09-19T08:46:50.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22e5068000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c363aac360 con 0x55c363cccb20 2024-09-19T08:46:50.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22e5068000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x55c363aa92d0 con 0x55c363ccd440 2024-09-19T08:46:50.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22e5068000 10 client.0 _ll_register_callbacks cb 0x55c363c6e820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:50.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22dbfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c363c88390 0x55c363c8e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:50.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22dbfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c363c88390 0x55c363c8e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54372/0 (socket says 172.21.15.98:54372) 2024-09-19T08:46:50.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22dbfff640 1 -- 172.21.15.98:0/1449546591 learned_addr learned my addr 172.21.15.98:0/1449546591 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:50.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22daffd640 1 --2- 172.21.15.98:0/1449546591 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c363ccd440 0x55c363c8dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:50.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22dbfff640 1 -- 172.21.15.98:0/1449546591 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c363ccd440 msgr2=0x55c363c8dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:50.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22db7fe640 1 --2- 172.21.15.98:0/1449546591 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c363cccb20 0x55c363c87e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:50.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22dbfff640 1 --2- 172.21.15.98:0/1449546591 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c363ccd440 0x55c363c8dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22dbfff640 1 -- 172.21.15.98:0/1449546591 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c363cccb20 msgr2=0x55c363c87e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:50.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22dbfff640 1 --2- 172.21.15.98:0/1449546591 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c363cccb20 0x55c363c87e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22dbfff640 1 -- 172.21.15.98:0/1449546591 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c363a78120 con 0x55c363c88390 2024-09-19T08:46:50.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22daffd640 1 --2- 172.21.15.98:0/1449546591 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c363ccd440 0x55c363c8dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:50.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22dbfff640 1 --2- 172.21.15.98:0/1449546591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c363c88390 0x55c363c8e230 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f22d401cc20 tx=0x7f22d4019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0d305b65919c0f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:50.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22ba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-19T08:46:50.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22ba7fc640 1 -- 172.21.15.98:0/1449546591 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d40bb070 con 0x55c363c88390 2024-09-19T08:46:50.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22ba7fc640 1 -- 172.21.15.98:0/1449546591 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(158..158 src has 1..158) ==== 5737+0+0 (secure 0 0 0) 0x7f22d40013a0 con 0x55c363c88390 2024-09-19T08:46:50.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22e5068000 1 -- 172.21.15.98:0/1449546591 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22c4011dc0 con 0x55c363c88390 2024-09-19T08:46:50.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22ba7fc640 1 -- 172.21.15.98:0/1449546591 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f22d40c6050 con 0x55c363c88390 2024-09-19T08:46:50.672 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22e5068000 10 client.8943 fetch_fsmap learned FSMap version 91 2024-09-19T08:46:50.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22e5068000 1 -- 172.21.15.98:0/1449546591 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x55c363a71450 con 0x55c363c88390 2024-09-19T08:46:50.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22ba7fc640 1 -- 172.21.15.98:0/1449546591 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22c4011dc0 con 0x55c363c88390 2024-09-19T08:46:50.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22ba7fc640 1 -- 172.21.15.98:0/1449546591 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 91) ==== 3502+0+0 (secure 0 0 0) 0x7f22d40c9020 con 0x55c363c88390 2024-09-19T08:46:50.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.662+0000 7f22e5068000 10 client.8943 fetch_fsmap finished waiting for FSMap version 91 2024-09-19T08:46:50.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.666+0000 7f22e5068000 10 client.8943 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T08:46:50.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.666+0000 7f22e5068000 20 client.8943 populate_metadata read hostname 'smithi098' 2024-09-19T08:46:50.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.666+0000 7f22e5068000 1 --2- 172.21.15.98:0/1449546591 >> [v2:172.21.15.98:6833/3734544897,v1:172.21.15.98:6835/3734544897] conn(0x55c363cd2f50 0x55c363cd3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:50.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.666+0000 7f22e5068000 4 client.8943 mds_command: new command op to 8766 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:46:50.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.666+0000 7f22e5068000 1 -- 172.21.15.98:0/1449546591 --> [v2:172.21.15.98:6833/3734544897,v1:172.21.15.98:6835/3734544897] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c363a6bc00 con 0x55c363cd2f50 2024-09-19T08:46:50.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.666+0000 7f22db7fe640 1 --2- 172.21.15.98:0/1449546591 >> [v2:172.21.15.98:6833/3734544897,v1:172.21.15.98:6835/3734544897] conn(0x55c363cd2f50 0x55c363cd3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:50.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.666+0000 7f22ba7fc640 10 client.8943 ms_handle_connect on v2:172.21.15.98:6833/3734544897 2024-09-19T08:46:50.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.666+0000 7f22db7fe640 1 --2- 172.21.15.98:0/1449546591 >> [v2:172.21.15.98:6833/3734544897,v1:172.21.15.98:6835/3734544897] conn(0x55c363cd2f50 0x55c363cd3330 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b345a39474def17 server_cookie=ddf750a6b7c63972 in_seq=0 out_seq=0 2024-09-19T08:46:50.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.666+0000 7f22ba7fc640 1 -- 172.21.15.98:0/1449546591 <== mds.0 v2:172.21.15.98:6833/3734544897 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c363a6bc00 con 0x55c363cd2f50 2024-09-19T08:46:50.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.666+0000 7f22ba7fc640 10 client.8943 handle_command_reply: tid=0 2024-09-19T08:46:50.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.666+0000 7f22e5068000 1 client.8943 shutdown 2024-09-19T08:46:50.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.666+0000 7f22e5068000 1 -- 172.21.15.98:0/1449546591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c363c88390 msgr2=0x55c363c8e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:50.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.666+0000 7f22e5068000 1 --2- 172.21.15.98:0/1449546591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c363c88390 0x55c363c8e230 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f22d401cc20 tx=0x7f22d4019120 comp rx=0 tx=0).stop 2024-09-19T08:46:50.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.666+0000 7f22e5068000 1 -- 172.21.15.98:0/1449546591 shutdown_connections 2024-09-19T08:46:50.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.666+0000 7f22e5068000 1 --2- 172.21.15.98:0/1449546591 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c363c88390 0x55c363c8e230 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.666+0000 7f22e5068000 1 --2- 172.21.15.98:0/1449546591 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c363ccd440 0x55c363c8dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.666+0000 7f22e5068000 1 --2- 172.21.15.98:0/1449546591 >> [v2:172.21.15.98:6833/3734544897,v1:172.21.15.98:6835/3734544897] conn(0x55c363cd2f50 0x55c363cd3330 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.666+0000 7f22e5068000 1 --2- 172.21.15.98:0/1449546591 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c363cccb20 0x55c363c87e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.666+0000 7f22e5068000 1 -- 172.21.15.98:0/1449546591 >> 172.21.15.98:0/1449546591 conn(0x55c363c83290 msgr2=0x55c363c81d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:50.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.666+0000 7f22ba7fc640 0 client.8943 ms_handle_reset on v2:172.21.15.98:6833/3734544897 2024-09-19T08:46:50.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.666+0000 7f22e5068000 1 -- 172.21.15.98:0/1449546591 shutdown_connections 2024-09-19T08:46:50.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.666+0000 7f22e5068000 1 -- 172.21.15.98:0/1449546591 wait complete. 2024-09-19T08:46:50.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.666+0000 7f22e5068000 20 client.8943 trim_cache size 0 max 16384 2024-09-19T08:46:50.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.682+0000 7f22e5068000 1 Processor -- start 2024-09-19T08:46:50.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.682+0000 7f22e5068000 1 -- start start 2024-09-19T08:46:50.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22e5068000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c363c88390 0x55c363ce3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:50.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22e5068000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c363cccb20 0x55c363ccbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:50.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22e5068000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c363ccd440 0x55c363ccc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:50.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22e5068000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc200270 con 0x55c363c88390 2024-09-19T08:46:50.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22e5068000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc0d0cb0 con 0x55c363cccb20 2024-09-19T08:46:50.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22e5068000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f22dc0d1300 con 0x55c363ccd440 2024-09-19T08:46:50.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22db7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c363c88390 0x55c363ce3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:50.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22db7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c363c88390 0x55c363ce3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54378/0 (socket says 172.21.15.98:54378) 2024-09-19T08:46:50.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22db7fe640 1 -- 172.21.15.98:0/4137895933 learned_addr learned my addr 172.21.15.98:0/4137895933 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:50.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22db7fe640 1 -- 172.21.15.98:0/4137895933 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c363ccd440 msgr2=0x55c363ccc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:50.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22db7fe640 1 --2- 172.21.15.98:0/4137895933 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c363ccd440 0x55c363ccc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22daffd640 1 --2- 172.21.15.98:0/4137895933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c363cccb20 0x55c363ccbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:50.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22db7fe640 1 -- 172.21.15.98:0/4137895933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c363cccb20 msgr2=0x55c363ccbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:50.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22db7fe640 1 --2- 172.21.15.98:0/4137895933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c363cccb20 0x55c363ccbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22db7fe640 1 -- 172.21.15.98:0/4137895933 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c363a58fd0 con 0x55c363c88390 2024-09-19T08:46:50.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22db7fe640 1 --2- 172.21.15.98:0/4137895933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c363c88390 0x55c363ce3640 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f22cc012bf0 tx=0x7f22cc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948da554ec8ead7e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:50.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22d0ff9640 1 -- 172.21.15.98:0/4137895933 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22cc00f040 con 0x55c363c88390 2024-09-19T08:46:50.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22d0ff9640 1 -- 172.21.15.98:0/4137895933 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22cc02f070 con 0x55c363c88390 2024-09-19T08:46:50.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22d0ff9640 1 -- 172.21.15.98:0/4137895933 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22cc02a4c0 con 0x55c363c88390 2024-09-19T08:46:50.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22e5068000 1 -- 172.21.15.98:0/4137895933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c363c88390 msgr2=0x55c363ce3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:50.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22e5068000 1 --2- 172.21.15.98:0/4137895933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c363c88390 0x55c363ce3640 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f22cc012bf0 tx=0x7f22cc018000 comp rx=0 tx=0).stop 2024-09-19T08:46:50.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22e5068000 1 -- 172.21.15.98:0/4137895933 shutdown_connections 2024-09-19T08:46:50.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22e5068000 1 --2- 172.21.15.98:0/4137895933 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c363ccd440 0x55c363ccc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22e5068000 1 --2- 172.21.15.98:0/4137895933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c363cccb20 0x55c363ccbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22e5068000 1 --2- 172.21.15.98:0/4137895933 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c363c88390 0x55c363ce3640 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22e5068000 1 -- 172.21.15.98:0/4137895933 >> 172.21.15.98:0/4137895933 conn(0x55c363c83290 msgr2=0x55c363c87650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:50.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22e5068000 1 -- 172.21.15.98:0/4137895933 shutdown_connections 2024-09-19T08:46:50.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22e5068000 1 -- 172.21.15.98:0/4137895933 wait complete. 2024-09-19T08:46:50.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22e5068000 1 -- start start 2024-09-19T08:46:50.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22e5068000 1 Processor -- start 2024-09-19T08:46:50.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22e5068000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c363c88390 0x55c363d3fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:50.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22e5068000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c363cccb20 0x55c363d40110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:50.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22e5068000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c363ccd440 0x55c363cd6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:50.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22e5068000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55c363ad0f80 con 0x55c363cccb20 2024-09-19T08:46:50.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22e5068000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c363a89ca0 con 0x55c363c88390 2024-09-19T08:46:50.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22e5068000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x55c363ad62a0 con 0x55c363ccd440 2024-09-19T08:46:50.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22e5068000 10 client.0 _ll_register_callbacks cb 0x7f22d40a4f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:46:50.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22daffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c363cccb20 0x55c363d40110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:50.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22daffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c363cccb20 0x55c363d40110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54390/0 (socket says 172.21.15.98:54390) 2024-09-19T08:46:50.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22daffd640 1 -- 172.21.15.98:0/3477076782 learned_addr learned my addr 172.21.15.98:0/3477076782 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:50.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22daffd640 1 -- 172.21.15.98:0/3477076782 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c363ccd440 msgr2=0x55c363cd6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:50.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22daffd640 1 --2- 172.21.15.98:0/3477076782 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c363ccd440 0x55c363cd6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22daffd640 1 -- 172.21.15.98:0/3477076782 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c363c88390 msgr2=0x55c363d3fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:50.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22daffd640 1 --2- 172.21.15.98:0/3477076782 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c363c88390 0x55c363d3fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.686+0000 7f22daffd640 1 -- 172.21.15.98:0/3477076782 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c363b54750 con 0x55c363cccb20 2024-09-19T08:46:50.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22daffd640 1 --2- 172.21.15.98:0/3477076782 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c363cccb20 0x55c363d40110 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f22c40569c0 tx=0x7f22c4015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=898101e0c342c4fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:50.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22ba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-19T08:46:50.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22ba7fc640 1 -- 172.21.15.98:0/3477076782 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c4072070 con 0x55c363cccb20 2024-09-19T08:46:50.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22ba7fc640 1 -- 172.21.15.98:0/3477076782 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(158..158 src has 1..158) ==== 5737+0+0 (secure 0 0 0) 0x7f22c405f180 con 0x55c363cccb20 2024-09-19T08:46:50.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22e5068000 1 -- 172.21.15.98:0/3477076782 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22cc02f070 con 0x55c363cccb20 2024-09-19T08:46:50.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22ba7fc640 1 -- 172.21.15.98:0/3477076782 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f22c4056390 con 0x55c363cccb20 2024-09-19T08:46:50.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22e5068000 10 client.8949 fetch_fsmap learned FSMap version 91 2024-09-19T08:46:50.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22e5068000 1 -- 172.21.15.98:0/3477076782 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x55c363a5fa30 con 0x55c363cccb20 2024-09-19T08:46:50.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22ba7fc640 1 -- 172.21.15.98:0/3477076782 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22cc02f070 con 0x55c363cccb20 2024-09-19T08:46:50.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22ba7fc640 1 -- 172.21.15.98:0/3477076782 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 91) ==== 3502+0+0 (secure 0 0 0) 0x7f22c4055e80 con 0x55c363cccb20 2024-09-19T08:46:50.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22e5068000 10 client.8949 fetch_fsmap finished waiting for FSMap version 91 2024-09-19T08:46:50.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22e5068000 10 client.8949 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T08:46:50.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22e5068000 20 client.8949 populate_metadata read hostname 'smithi098' 2024-09-19T08:46:50.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22e5068000 1 --2- 172.21.15.98:0/3477076782 >> [v2:172.21.15.98:6833/3734544897,v1:172.21.15.98:6835/3734544897] conn(0x55c363cd68d0 0x55c363cd6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:50.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22e5068000 4 client.8949 mds_command: new command op to 8766 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T08:46:50.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22e5068000 1 -- 172.21.15.98:0/3477076782 --> [v2:172.21.15.98:6833/3734544897,v1:172.21.15.98:6835/3734544897] -- command(tid 0: {"prefix": "status"}) -- 0x55c363adf3a0 con 0x55c363cd68d0 2024-09-19T08:46:50.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22db7fe640 1 --2- 172.21.15.98:0/3477076782 >> [v2:172.21.15.98:6833/3734544897,v1:172.21.15.98:6835/3734544897] conn(0x55c363cd68d0 0x55c363cd6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:50.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22ba7fc640 10 client.8949 ms_handle_connect on v2:172.21.15.98:6833/3734544897 2024-09-19T08:46:50.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22db7fe640 1 --2- 172.21.15.98:0/3477076782 >> [v2:172.21.15.98:6833/3734544897,v1:172.21.15.98:6835/3734544897] conn(0x55c363cd68d0 0x55c363cd6cb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29ab852350b61328 server_cookie=2382f19f457edcaf in_seq=0 out_seq=0 2024-09-19T08:46:50.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22ba7fc640 1 -- 172.21.15.98:0/3477076782 <== mds.0 v2:172.21.15.98:6833/3734544897 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x55c363adf3a0 con 0x55c363cd68d0 2024-09-19T08:46:50.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22ba7fc640 10 client.8949 handle_command_reply: tid=0 2024-09-19T08:46:50.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22e5068000 1 client.8949 shutdown 2024-09-19T08:46:50.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22e5068000 1 -- 172.21.15.98:0/3477076782 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c363cccb20 msgr2=0x55c363d40110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:50.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22e5068000 1 --2- 172.21.15.98:0/3477076782 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c363cccb20 0x55c363d40110 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f22c40569c0 tx=0x7f22c4015040 comp rx=0 tx=0).stop 2024-09-19T08:46:50.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22e5068000 1 -- 172.21.15.98:0/3477076782 shutdown_connections 2024-09-19T08:46:50.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22e5068000 1 --2- 172.21.15.98:0/3477076782 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55c363ccd440 0x55c363cd6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22e5068000 1 --2- 172.21.15.98:0/3477076782 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55c363cccb20 0x55c363d40110 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22e5068000 1 --2- 172.21.15.98:0/3477076782 >> [v2:172.21.15.98:6833/3734544897,v1:172.21.15.98:6835/3734544897] conn(0x55c363cd68d0 0x55c363cd6cb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22e5068000 1 --2- 172.21.15.98:0/3477076782 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c363c88390 0x55c363d3fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22ba7fc640 0 client.8949 ms_handle_reset on v2:172.21.15.98:6833/3734544897 2024-09-19T08:46:50.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22e5068000 1 -- 172.21.15.98:0/3477076782 >> 172.21.15.98:0/3477076782 conn(0x55c363c83290 msgr2=0x55c363ccace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:50.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22e5068000 1 -- 172.21.15.98:0/3477076782 shutdown_connections 2024-09-19T08:46:50.695 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22e5068000 1 -- 172.21.15.98:0/3477076782 wait complete. 2024-09-19T08:46:50.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22e5068000 20 client.8949 trim_cache size 0 max 16384 2024-09-19T08:46:50.696 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:46:50.696 INFO:teuthology.orchestra.run.smithi098.stdout: "cluster_fsid": "552bf0ee-1f4f-4644-ba7c-d0931e55a451", 2024-09-19T08:46:50.696 INFO:teuthology.orchestra.run.smithi098.stdout: "whoami": 0, 2024-09-19T08:46:50.696 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 8766, 2024-09-19T08:46:50.696 INFO:teuthology.orchestra.run.smithi098.stdout: "want_state": "up:active", 2024-09-19T08:46:50.696 INFO:teuthology.orchestra.run.smithi098.stdout: "state": "up:active", 2024-09-19T08:46:50.696 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "cephfs", 2024-09-19T08:46:50.696 INFO:teuthology.orchestra.run.smithi098.stdout: "rank_uptime": 10.184696634, 2024-09-19T08:46:50.697 INFO:teuthology.orchestra.run.smithi098.stdout: "mdsmap_epoch": 91, 2024-09-19T08:46:50.697 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch": 158, 2024-09-19T08:46:50.697 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch_barrier": 158, 2024-09-19T08:46:50.697 INFO:teuthology.orchestra.run.smithi098.stdout: "uptime": 15.150096522 2024-09-19T08:46:50.697 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:46:50.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22e1fac640 1 -- 172.21.15.98:0/3583738702 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f22bc0680d0 msgr2=0x7f22bc06a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:50.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22e1fac640 1 --2- 172.21.15.98:0/3583738702 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f22bc0680d0 0x7f22bc06a590 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f22cc009af0 tx=0x7f22cc008040 comp rx=0 tx=0).stop 2024-09-19T08:46:50.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22e1fac640 1 -- 172.21.15.98:0/3583738702 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22dc158280 msgr2=0x7f22dc157a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:50.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22e1fac640 1 --2- 172.21.15.98:0/3583738702 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22dc158280 0x7f22dc157a30 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f22c4007a40 tx=0x7f22c400e530 comp rx=0 tx=0).stop 2024-09-19T08:46:50.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22e1fac640 1 -- 172.21.15.98:0/3583738702 shutdown_connections 2024-09-19T08:46:50.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22e1fac640 1 --2- 172.21.15.98:0/3583738702 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f22bc0680d0 0x7f22bc06a590 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22e1fac640 1 --2- 172.21.15.98:0/3583738702 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f22dc0b0a80 0x7f22dc0b0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22e1fac640 1 --2- 172.21.15.98:0/3583738702 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f22dc158280 0x7f22dc157a30 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22e1fac640 1 --2- 172.21.15.98:0/3583738702 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f22c8004d50 0x7f22dc0ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.699 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22e1fac640 1 -- 172.21.15.98:0/3583738702 >> 172.21.15.98:0/3583738702 conn(0x7f22dc0d2310 msgr2=0x7f22dc0b2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:50.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22e1fac640 1 -- 172.21.15.98:0/3583738702 shutdown_connections 2024-09-19T08:46:50.700 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.690+0000 7f22e1fac640 1 -- 172.21.15.98:0/3583738702 wait complete. 2024-09-19T08:46:50.706 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T08:46:50.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.810+0000 7f0c17c92640 1 Processor -- start 2024-09-19T08:46:50.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.810+0000 7f0c17c92640 1 -- start start 2024-09-19T08:46:50.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.810+0000 7f0c17c92640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0c1006f9a0 0x7f0c1006da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:50.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.810+0000 7f0c17c92640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0c100aff00 0x7f0c100cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:50.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.810+0000 7f0c17c92640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c10072ad0 con 0x7f0c100cfba0 2024-09-19T08:46:50.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.810+0000 7f0c17c92640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f0c100d0000 con 0x7f0c1006f9a0 2024-09-19T08:46:50.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.810+0000 7f0c17c92640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0c1006dfb0 con 0x7f0c100aff00 2024-09-19T08:46:50.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.810+0000 7f0c15a2e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0c1006f9a0 0x7f0c1006da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:50.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.810+0000 7f0c15a2e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0c1006f9a0 0x7f0c1006da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54402/0 (socket says 172.21.15.98:54402) 2024-09-19T08:46:50.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.810+0000 7f0c15a2e640 1 -- 172.21.15.98:0/3808426099 learned_addr learned my addr 172.21.15.98:0/3808426099 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:50.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.810+0000 7f0c1522d640 1 --2- 172.21.15.98:0/3808426099 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0c100aff00 0x7f0c100cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:50.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.810+0000 7f0c15a2e640 1 -- 172.21.15.98:0/3808426099 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0c100aff00 msgr2=0x7f0c100cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:50.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.810+0000 7f0c15a2e640 1 --2- 172.21.15.98:0/3808426099 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0c100aff00 0x7f0c100cf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.810+0000 7f0c15a2e640 1 -- 172.21.15.98:0/3808426099 >> v1:172.21.15.139:6789/0 conn(0x7f0c100cfba0 legacy=0x7f0c100cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:50.811 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.810+0000 7f0c15a2e640 1 -- 172.21.15.98:0/3808426099 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c100cb440 con 0x7f0c1006f9a0 2024-09-19T08:46:50.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.814+0000 7f0c15a2e640 1 --2- 172.21.15.98:0/3808426099 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c1006f9a0 0x7f0c1006da70 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f0c00009e50 tx=0x7f0c0001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51fe7f016e8fd5a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:50.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.814+0000 7f0c14a2c640 1 -- 172.21.15.98:0/3808426099 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c000a55d0 con 0x7f0c1006f9a0 2024-09-19T08:46:50.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.814+0000 7f0c14a2c640 1 -- 172.21.15.98:0/3808426099 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c000a5770 con 0x7f0c1006f9a0 2024-09-19T08:46:50.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.814+0000 7f0c14a2c640 1 -- 172.21.15.98:0/3808426099 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c000a5b40 con 0x7f0c1006f9a0 2024-09-19T08:46:50.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.814+0000 7f0c17c92640 1 -- 172.21.15.98:0/3808426099 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c1006f9a0 msgr2=0x7f0c1006da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:50.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.814+0000 7f0c17c92640 1 --2- 172.21.15.98:0/3808426099 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c1006f9a0 0x7f0c1006da70 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f0c00009e50 tx=0x7f0c0001c910 comp rx=0 tx=0).stop 2024-09-19T08:46:50.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.814+0000 7f0c17c92640 1 -- 172.21.15.98:0/3808426099 shutdown_connections 2024-09-19T08:46:50.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.814+0000 7f0c17c92640 1 --2- 172.21.15.98:0/3808426099 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0c100aff00 0x7f0c100cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.814+0000 7f0c17c92640 1 --2- 172.21.15.98:0/3808426099 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c1006f9a0 0x7f0c1006da70 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.814+0000 7f0c17c92640 1 -- 172.21.15.98:0/3808426099 >> 172.21.15.98:0/3808426099 conn(0x7f0c100d0980 msgr2=0x7f0c100d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:50.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.814+0000 7f0c17c92640 1 -- 172.21.15.98:0/3808426099 shutdown_connections 2024-09-19T08:46:50.814 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.814+0000 7f0c17c92640 1 -- 172.21.15.98:0/3808426099 wait complete. 2024-09-19T08:46:50.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.814+0000 7f0c17c92640 1 Processor -- start 2024-09-19T08:46:50.815 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.814+0000 7f0c17c92640 1 -- start start 2024-09-19T08:46:50.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.814+0000 7f0c17c92640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0c1006f9a0 0x7f0c10156cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:50.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.814+0000 7f0c17c92640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c100aff00 0x7f0c100aee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:50.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.814+0000 7f0c17c92640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c100cfba0 0x7f0c100af350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:50.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.814+0000 7f0c17c92640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0c100683a0 con 0x7f0c100aff00 2024-09-19T08:46:50.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.814+0000 7f0c17c92640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0c10068220 con 0x7f0c100cfba0 2024-09-19T08:46:50.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.814+0000 7f0c17c92640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0c100cd000 con 0x7f0c1006f9a0 2024-09-19T08:46:50.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.818+0000 7f0c1522d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c100aff00 0x7f0c100aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:50.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.818+0000 7f0c1522d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c100aff00 0x7f0c100aee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54410/0 (socket says 172.21.15.98:54410) 2024-09-19T08:46:50.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.818+0000 7f0c1522d640 1 -- 172.21.15.98:0/3271916069 learned_addr learned my addr 172.21.15.98:0/3271916069 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:50.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.818+0000 7f0c15a2e640 1 --2- 172.21.15.98:0/3271916069 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0c1006f9a0 0x7f0c10156cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:50.817 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.818+0000 7f0c1522d640 1 -- 172.21.15.98:0/3271916069 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0c1006f9a0 msgr2=0x7f0c10156cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:50.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.818+0000 7f0c1522d640 1 --2- 172.21.15.98:0/3271916069 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0c1006f9a0 0x7f0c10156cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.818+0000 7f0c1522d640 1 -- 172.21.15.98:0/3271916069 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c100cfba0 msgr2=0x7f0c100af350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:50.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.818+0000 7f0c1622f640 1 --2- 172.21.15.98:0/3271916069 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c100cfba0 0x7f0c100af350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:50.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.818+0000 7f0c1522d640 1 --2- 172.21.15.98:0/3271916069 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c100cfba0 0x7f0c100af350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:50.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.818+0000 7f0c1522d640 1 -- 172.21.15.98:0/3271916069 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c100c7bf0 con 0x7f0c100aff00 2024-09-19T08:46:50.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.818+0000 7f0c1522d640 1 --2- 172.21.15.98:0/3271916069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c100aff00 0x7f0c100aee10 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400d5f0 tx=0x7f0c0400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:50.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.818+0000 7f0c0e7fc640 1 -- 172.21.15.98:0/3271916069 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c040125f0 con 0x7f0c100aff00 2024-09-19T08:46:50.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.818+0000 7f0c0e7fc640 1 -- 172.21.15.98:0/3271916069 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c04019070 con 0x7f0c100aff00 2024-09-19T08:46:50.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.818+0000 7f0c17c92640 1 -- 172.21.15.98:0/3271916069 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c100c7ed0 con 0x7f0c100aff00 2024-09-19T08:46:50.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.818+0000 7f0c0e7fc640 1 -- 172.21.15.98:0/3271916069 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c040129d0 con 0x7f0c100aff00 2024-09-19T08:46:50.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.818+0000 7f0c17c92640 1 -- 172.21.15.98:0/3271916069 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c100c8490 con 0x7f0c100aff00 2024-09-19T08:46:50.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.818+0000 7f0c17c92640 1 -- 172.21.15.98:0/3271916069 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c100c8690 con 0x7f0c100aff00 2024-09-19T08:46:50.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.818+0000 7f0c0e7fc640 1 -- 172.21.15.98:0/3271916069 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f0c04012b70 con 0x7f0c100aff00 2024-09-19T08:46:50.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.818+0000 7f0c0e7fc640 1 --2- 172.21.15.98:0/3271916069 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0bfc0680a0 0x7f0bfc06a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:50.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.818+0000 7f0c0e7fc640 1 -- 172.21.15.98:0/3271916069 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5737+0+0 (secure 0 0 0) 0x7f0c04084b30 con 0x7f0c100aff00 2024-09-19T08:46:50.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.818+0000 7f0c15a2e640 1 --2- 172.21.15.98:0/3271916069 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0bfc0680a0 0x7f0bfc06a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:50.819 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.818+0000 7f0c15a2e640 1 --2- 172.21.15.98:0/3271916069 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0bfc0680a0 0x7f0bfc06a560 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0c000ae000 tx=0x7f0c000a7070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:50.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:50.822+0000 7f0c0e7fc640 1 -- 172.21.15.98:0/3271916069 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f0c100c8690 con 0x7f0c100aff00 2024-09-19T08:46:51.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:51.018+0000 7f0c17c92640 1 -- 172.21.15.98:0/3271916069 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0c1006fd80 con 0x7f0c100aff00 2024-09-19T08:46:51.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:51.018+0000 7f0c0e7fc640 1 -- 172.21.15.98:0/3271916069 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v91) ==== 53+0+73 (secure 0 0 0) 0x7f0c04056f60 con 0x7f0c100aff00 2024-09-19T08:46:51.018 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T08:46:51.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:51.022+0000 7f0c17c92640 1 -- 172.21.15.98:0/3271916069 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0bfc0680a0 msgr2=0x7f0bfc06a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:51.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:51.022+0000 7f0c17c92640 1 --2- 172.21.15.98:0/3271916069 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0bfc0680a0 0x7f0bfc06a560 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0c000ae000 tx=0x7f0c000a7070 comp rx=0 tx=0).stop 2024-09-19T08:46:51.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:51.022+0000 7f0c17c92640 1 -- 172.21.15.98:0/3271916069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c100aff00 msgr2=0x7f0c100aee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:51.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:51.022+0000 7f0c17c92640 1 --2- 172.21.15.98:0/3271916069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c100aff00 0x7f0c100aee10 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400d5f0 tx=0x7f0c0400dab0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:51.022+0000 7f0c17c92640 1 -- 172.21.15.98:0/3271916069 shutdown_connections 2024-09-19T08:46:51.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:51.022+0000 7f0c17c92640 1 --2- 172.21.15.98:0/3271916069 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0bfc0680a0 0x7f0bfc06a560 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:51.022+0000 7f0c17c92640 1 --2- 172.21.15.98:0/3271916069 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c100cfba0 0x7f0c100af350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:51.022+0000 7f0c17c92640 1 --2- 172.21.15.98:0/3271916069 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0c100aff00 0x7f0c100aee10 secure :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400d5f0 tx=0x7f0c0400dab0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:51.022+0000 7f0c17c92640 1 --2- 172.21.15.98:0/3271916069 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0c1006f9a0 0x7f0c10156cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:51.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:51.022+0000 7f0c17c92640 1 -- 172.21.15.98:0/3271916069 >> 172.21.15.98:0/3271916069 conn(0x7f0c100d0980 msgr2=0x7f0c10071f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:51.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:51.022+0000 7f0c17c92640 1 -- 172.21.15.98:0/3271916069 shutdown_connections 2024-09-19T08:46:51.021 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:51.022+0000 7f0c17c92640 1 -- 172.21.15.98:0/3271916069 wait complete. 2024-09-19T08:46:51.036 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T08:46:51.036 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T08:46:51.036 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi098.front.sepia.ceph.com 2024-09-19T08:46:51.036 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T08:46:51.036 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T08:46:51.036 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T08:46:51.036 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T08:46:51.036 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T08:46:51.036 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:46:51.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:51.037 DEBUG:teuthology.orchestra.run.smithi098:> ip addr 2024-09-19T08:46:51.043 INFO:teuthology.orchestra.run.smithi098.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T08:46:51.043 INFO:teuthology.orchestra.run.smithi098.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T08:46:51.043 INFO:teuthology.orchestra.run.smithi098.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T08:46:51.043 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:46:51.043 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 ::1/128 scope host 2024-09-19T08:46:51.043 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:46:51.043 INFO:teuthology.orchestra.run.smithi098.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:46:51.043 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b8 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:46:51.044 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f0 2024-09-19T08:46:51.044 INFO:teuthology.orchestra.run.smithi098.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:46:51.044 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b9 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:46:51.044 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f1 2024-09-19T08:46:51.044 INFO:teuthology.orchestra.run.smithi098.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:46:51.044 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6a brd ff:ff:ff:ff:ff:ff 2024-09-19T08:46:51.044 INFO:teuthology.orchestra.run.smithi098.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T08:46:51.044 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6b brd ff:ff:ff:ff:ff:ff 2024-09-19T08:46:51.044 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.21.15.98/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T08:46:51.044 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft 41997sec preferred_lft 41997sec 2024-09-19T08:46:51.044 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::ec4:7aff:fe8f:d26b/64 scope link 2024-09-19T08:46:51.044 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:46:51.045 INFO:teuthology.orchestra.run.smithi098.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T08:46:51.045 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 02:42:98:dd:32:2d brd ff:ff:ff:ff:ff:ff 2024-09-19T08:46:51.045 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T08:46:51.045 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:46:51.045 INFO:teuthology.orchestra.run.smithi098.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T08:46:51.045 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-19T08:46:51.045 INFO:teuthology.orchestra.run.smithi098.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T08:46:51.045 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:46:51.045 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-19T08:46:51.045 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:46:51.045 INFO:teuthology.orchestra.run.smithi098.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:46:51.046 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:46:51.046 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-19T08:46:51.046 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:46:51.046 INFO:teuthology.orchestra.run.smithi098.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:46:51.046 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:46:51.046 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-19T08:46:51.046 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:46:51.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:51.048 DEBUG:teuthology.orchestra.run.smithi098:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T08:46:51.092 INFO:teuthology.orchestra.run.smithi098.stdout:1 2024-09-19T08:46:51.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:51.093 DEBUG:teuthology.orchestra.run.smithi098:> route 2024-09-19T08:46:51.142 INFO:teuthology.orchestra.run.smithi098.stdout:Kernel IP routing table 2024-09-19T08:46:51.143 INFO:teuthology.orchestra.run.smithi098.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T08:46:51.143 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T08:46:51.143 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T08:46:51.143 INFO:teuthology.orchestra.run.smithi098.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T08:46:51.143 INFO:teuthology.orchestra.run.smithi098.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T08:46:51.143 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T08:46:51.143 INFO:teuthology.orchestra.run.smithi098.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:46:51.143 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:46:51.143 INFO:teuthology.orchestra.run.smithi098.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:46:51.143 INFO:teuthology.orchestra.run.smithi098.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T08:46:51.144 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:46:51.144 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-09-19T08:46:51.144 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T08:46:51.144 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T08:46:51.144 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T08:46:51.144 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-09-19T08:46:51.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:46:51.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:46:51.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:51.258 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2024-09-19T08:46:51.305 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-19T08:46:51.305 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T08:46:51.305 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:46:51.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:51.305 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set brx.0 up 2024-09-19T08:46:51.360 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-19T08:46:51.360 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:46:51.360 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T08:46:51.408 INFO:teuthology.orchestra.run.smithi098.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:46:51.409 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:46:51.409 DEBUG:teuthology.orchestra.run.smithi098:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T08:46:51.460 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:46:51.518 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:46:51.565 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:46:51.566 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:46:51.566 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:46:51.566 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:46:51.566 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:46:51.566 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:46:51.566 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:46:51.567 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:46:51.567 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:46:51.567 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:46:51.567 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:46:51.567 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:46:51.567 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:46:51.568 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:46:51.568 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:46:51.568 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:46:51.568 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:46:51.568 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:46:51.568 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:46:51.569 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:46:51.569 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:46:51.569 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:46:51.569 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:46:51.569 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:46:51.569 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:46:51.569 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:46:51.570 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:46:51.570 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:46:51.570 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:46:51.570 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:46:51.570 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:46:51.571 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:46:51.571 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:46:51.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:51.572 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:46:51.621 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T08:46:51.621 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-19T08:46:51.665 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:46:51.679 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:46:51.724 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:46:51.725 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:46:51.725 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:46:51.725 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:46:51.725 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:46:51.725 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:46:51.725 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:46:51.725 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:46:51.725 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:46:51.725 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:46:51.725 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:46:51.725 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:46:51.725 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:46:51.725 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:46:51.726 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:46:51.726 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:46:51.726 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:46:51.726 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:46:51.726 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:46:51.726 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:46:51.726 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:46:51.726 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:46:51.726 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:46:51.726 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:46:51.726 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:46:51.726 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:46:51.726 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:46:51.727 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:46:51.727 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:46:51.727 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:46:51.727 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:46:51.727 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:46:51.727 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:46:51.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:51.728 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:46:51.745 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:2024-09-19T08:46:51.746+0000 7f2686f12580 -1 init, newargv = 0x55e0045175d0 newargc=15 2024-09-19T08:46:51.745 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:ceph-fuse[53115]: starting ceph client 2024-09-19T08:46:51.751 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:ceph-fuse[53115]: starting fuse 2024-09-19T08:46:52.851 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:46:52.867 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:46:52.915 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:46:52.915 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:46:52.915 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:46:52.915 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:46:52.915 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:46:52.915 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:46:52.915 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:46:52.916 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:46:52.916 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:46:52.916 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:46:52.916 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:46:52.916 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:46:52.916 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:46:52.916 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:46:52.916 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:46:52.916 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:46:52.916 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:46:52.916 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:46:52.916 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:46:52.917 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:46:52.917 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:46:52.917 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:46:52.917 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:46:52.917 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:46:52.917 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:46:52.917 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:46:52.917 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:46:52.917 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:46:52.917 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:46:52.917 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:46:52.917 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:46:52.918 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:46:52.918 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:46:52.918 INFO:teuthology.orchestra.run.smithi098.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-19T08:46:52.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:52.919 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:46:52.965 INFO:teuthology.orchestra.run.smithi098.stdout:45 2024-09-19T08:46:52.966 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-19T08:46:52.966 DEBUG:teuthology.orchestra.run.smithi098:> sudo stdin-killer -- python3 -c ' 2024-09-19T08:46:52.966 DEBUG:teuthology.orchestra.run.smithi098:> import glob 2024-09-19T08:46:52.966 DEBUG:teuthology.orchestra.run.smithi098:> import re 2024-09-19T08:46:52.966 DEBUG:teuthology.orchestra.run.smithi098:> import os 2024-09-19T08:46:52.966 DEBUG:teuthology.orchestra.run.smithi098:> import subprocess 2024-09-19T08:46:52.967 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:46:52.967 DEBUG:teuthology.orchestra.run.smithi098:> def _find_admin_socket(client_name): 2024-09-19T08:46:52.967 DEBUG:teuthology.orchestra.run.smithi098:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T08:46:52.967 DEBUG:teuthology.orchestra.run.smithi098:> files = glob.glob(asok_path) 2024-09-19T08:46:52.967 DEBUG:teuthology.orchestra.run.smithi098:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T08:46:52.967 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:46:52.967 DEBUG:teuthology.orchestra.run.smithi098:> # Given a non-glob path, it better be there 2024-09-19T08:46:52.967 DEBUG:teuthology.orchestra.run.smithi098:> if "*" not in asok_path: 2024-09-19T08:46:52.967 DEBUG:teuthology.orchestra.run.smithi098:> assert(len(files) == 1) 2024-09-19T08:46:52.967 DEBUG:teuthology.orchestra.run.smithi098:> return files[0] 2024-09-19T08:46:52.967 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:46:52.968 DEBUG:teuthology.orchestra.run.smithi098:> for f in files: 2024-09-19T08:46:52.968 DEBUG:teuthology.orchestra.run.smithi098:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T08:46:52.968 DEBUG:teuthology.orchestra.run.smithi098:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T08:46:52.968 DEBUG:teuthology.orchestra.run.smithi098:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T08:46:52.968 DEBUG:teuthology.orchestra.run.smithi098:> contents = proc_f.read() 2024-09-19T08:46:52.968 DEBUG:teuthology.orchestra.run.smithi098:> if mountpoint in contents: 2024-09-19T08:46:52.968 DEBUG:teuthology.orchestra.run.smithi098:> return f 2024-09-19T08:46:52.968 DEBUG:teuthology.orchestra.run.smithi098:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T08:46:52.968 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:46:52.968 DEBUG:teuthology.orchestra.run.smithi098:> print(_find_admin_socket("client.0")) 2024-09-19T08:46:52.968 DEBUG:teuthology.orchestra.run.smithi098:> ' 2024-09-19T08:46:53.087 INFO:teuthology.orchestra.run.smithi098.stdout:/var/run/ceph/ceph-client.0.53115.asok 2024-09-19T08:46:53.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:46:53.099 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.53115.asok 2024-09-19T08:46:53.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:53.099 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.53115.asok status 2024-09-19T08:46:53.246 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:46:53.246 INFO:teuthology.orchestra.run.smithi098.stdout: "metadata": { 2024-09-19T08:46:53.246 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T08:46:53.247 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T08:46:53.247 INFO:teuthology.orchestra.run.smithi098.stdout: "entity_id": "0", 2024-09-19T08:46:53.247 INFO:teuthology.orchestra.run.smithi098.stdout: "hostname": "smithi098", 2024-09-19T08:46:53.247 INFO:teuthology.orchestra.run.smithi098.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T08:46:53.247 INFO:teuthology.orchestra.run.smithi098.stdout: "pid": "53115", 2024-09-19T08:46:53.247 INFO:teuthology.orchestra.run.smithi098.stdout: "root": "/" 2024-09-19T08:46:53.247 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:53.247 INFO:teuthology.orchestra.run.smithi098.stdout: "dentry_count": 0, 2024-09-19T08:46:53.248 INFO:teuthology.orchestra.run.smithi098.stdout: "dentry_pinned_count": 0, 2024-09-19T08:46:53.248 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 8961, 2024-09-19T08:46:53.248 INFO:teuthology.orchestra.run.smithi098.stdout: "inst": { 2024-09-19T08:46:53.248 INFO:teuthology.orchestra.run.smithi098.stdout: "name": { 2024-09-19T08:46:53.248 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "client", 2024-09-19T08:46:53.248 INFO:teuthology.orchestra.run.smithi098.stdout: "num": 8961 2024-09-19T08:46:53.248 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:53.248 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": { 2024-09-19T08:46:53.248 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v1", 2024-09-19T08:46:53.248 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:46:53.249 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 313607843 2024-09-19T08:46:53.249 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:53.249 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:53.249 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": { 2024-09-19T08:46:53.249 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v1", 2024-09-19T08:46:53.249 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:46:53.249 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 313607843 2024-09-19T08:46:53.249 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:46:53.249 INFO:teuthology.orchestra.run.smithi098.stdout: "inst_str": "client.8961 192.168.0.1:0/313607843", 2024-09-19T08:46:53.249 INFO:teuthology.orchestra.run.smithi098.stdout: "addr_str": "192.168.0.1:0/313607843", 2024-09-19T08:46:53.249 INFO:teuthology.orchestra.run.smithi098.stdout: "inode_count": 1, 2024-09-19T08:46:53.249 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_epoch": 91, 2024-09-19T08:46:53.249 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_epoch": 158, 2024-09-19T08:46:53.250 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_epoch_barrier": 0, 2024-09-19T08:46:53.250 INFO:teuthology.orchestra.run.smithi098.stdout: "blocklisted": false, 2024-09-19T08:46:53.250 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "cephfs" 2024-09-19T08:46:53.250 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:46:53.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:53.260 DEBUG:teuthology.orchestra.run.smithi098:> stat --file-system '--printf=%T 2024-09-19T08:46:53.261 DEBUG:teuthology.orchestra.run.smithi098:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T08:46:53.266 INFO:teuthology.orchestra.run.smithi098.stdout:fuseblk 2024-09-19T08:46:53.267 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T08:46:53.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:53.267 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T08:46:53.322 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T08:46:53.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.490+0000 7efc9b083640 1 Processor -- start 2024-09-19T08:46:53.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.490+0000 7efc9b083640 1 -- start start 2024-09-19T08:46:53.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.490+0000 7efc9b083640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7efc9415e650 0x7efc9406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:53.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.490+0000 7efc9b083640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7efc940d2e00 0x7efc940707b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:53.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.490+0000 7efc9b083640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc940b73d0 0x7efc94070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:53.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.490+0000 7efc9b083640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7efc94162fc0 con 0x7efc9415e650 2024-09-19T08:46:53.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.490+0000 7efc9b083640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7efc9406c1d0 con 0x7efc940d2e00 2024-09-19T08:46:53.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.490+0000 7efc9b083640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7efc94162440 con 0x7efc940b73d0 2024-09-19T08:46:53.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.490+0000 7efc98e1f640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7efc9415e650 0x7efc9406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:53.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.490+0000 7efc98e1f640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7efc9415e650 0x7efc9406bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54418/0 (socket says 172.21.15.98:54418) 2024-09-19T08:46:53.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.490+0000 7efc98e1f640 1 -- 172.21.15.98:0/2898801122 learned_addr learned my addr 172.21.15.98:0/2898801122 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:53.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.490+0000 7efc99620640 1 --2- 172.21.15.98:0/2898801122 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc940b73d0 0x7efc94070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:53.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.490+0000 7efc93fff640 1 --2- 172.21.15.98:0/2898801122 >> v2:172.21.15.139:3300/0 conn(0x7efc940d2e00 0x7efc940707b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:53.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.490+0000 7efc98e1f640 1 -- 172.21.15.98:0/2898801122 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc940b73d0 msgr2=0x7efc94070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:53.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.490+0000 7efc98e1f640 1 --2- 172.21.15.98:0/2898801122 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc940b73d0 0x7efc94070cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.490+0000 7efc98e1f640 1 -- 172.21.15.98:0/2898801122 >> v2:172.21.15.139:3300/0 conn(0x7efc940d2e00 msgr2=0x7efc940707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:53.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.490+0000 7efc98e1f640 1 --2- 172.21.15.98:0/2898801122 >> v2:172.21.15.139:3300/0 conn(0x7efc940d2e00 0x7efc940707b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.490+0000 7efc98e1f640 1 -- 172.21.15.98:0/2898801122 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc94162720 con 0x7efc9415e650 2024-09-19T08:46:53.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.490+0000 7efc93fff640 1 --2- 172.21.15.98:0/2898801122 >> v2:172.21.15.139:3300/0 conn(0x7efc940d2e00 0x7efc940707b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:53.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.494+0000 7efc98e1f640 1 --2- 172.21.15.98:0/2898801122 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc9415e650 0x7efc9406bc90 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7efc84009870 tx=0x7efc8401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d49c27c85d828408 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:53.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.494+0000 7efc937fe640 1 -- 172.21.15.98:0/2898801122 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc840aa610 con 0x7efc9415e650 2024-09-19T08:46:53.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.494+0000 7efc937fe640 1 -- 172.21.15.98:0/2898801122 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc840af070 con 0x7efc9415e650 2024-09-19T08:46:53.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.494+0000 7efc937fe640 1 -- 172.21.15.98:0/2898801122 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc840aa9d0 con 0x7efc9415e650 2024-09-19T08:46:53.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.494+0000 7efc9b083640 1 -- 172.21.15.98:0/2898801122 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc9415e650 msgr2=0x7efc9406bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:53.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.494+0000 7efc9b083640 1 --2- 172.21.15.98:0/2898801122 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc9415e650 0x7efc9406bc90 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7efc84009870 tx=0x7efc8401c4f0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.494+0000 7efc9b083640 1 -- 172.21.15.98:0/2898801122 shutdown_connections 2024-09-19T08:46:53.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.494+0000 7efc9b083640 1 --2- 172.21.15.98:0/2898801122 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc940b73d0 0x7efc94070cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.494+0000 7efc9b083640 1 --2- 172.21.15.98:0/2898801122 >> v2:172.21.15.139:3300/0 conn(0x7efc940d2e00 0x7efc940707b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.494+0000 7efc9b083640 1 --2- 172.21.15.98:0/2898801122 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc9415e650 0x7efc9406bc90 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.494+0000 7efc9b083640 1 -- 172.21.15.98:0/2898801122 >> 172.21.15.98:0/2898801122 conn(0x7efc940d2310 msgr2=0x7efc940d26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:53.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.494+0000 7efc9b083640 1 -- 172.21.15.98:0/2898801122 shutdown_connections 2024-09-19T08:46:53.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.494+0000 7efc9b083640 1 -- 172.21.15.98:0/2898801122 wait complete. 2024-09-19T08:46:53.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.494+0000 7efc9b083640 1 Processor -- start 2024-09-19T08:46:53.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.494+0000 7efc9b083640 1 -- start start 2024-09-19T08:46:53.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.498+0000 7efc9b083640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc940b73d0 0x7efc9414ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:53.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.498+0000 7efc9b083640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc940d2e00 0x7efc941fa610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:53.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.498+0000 7efc9b083640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc9415e650 0x7efc941fab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:53.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.498+0000 7efc9b083640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7efc94073ca0 con 0x7efc940b73d0 2024-09-19T08:46:53.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.498+0000 7efc9b083640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efc94073b20 con 0x7efc940d2e00 2024-09-19T08:46:53.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.498+0000 7efc9b083640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7efc94073e20 con 0x7efc9415e650 2024-09-19T08:46:53.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.498+0000 7efc98e1f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc940b73d0 0x7efc9414ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:53.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.498+0000 7efc98e1f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc940b73d0 0x7efc9414ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54424/0 (socket says 172.21.15.98:54424) 2024-09-19T08:46:53.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.498+0000 7efc98e1f640 1 -- 172.21.15.98:0/3242060441 learned_addr learned my addr 172.21.15.98:0/3242060441 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:53.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.498+0000 7efc93fff640 1 --2- 172.21.15.98:0/3242060441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc940d2e00 0x7efc941fa610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:53.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.498+0000 7efc99620640 1 --2- 172.21.15.98:0/3242060441 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc9415e650 0x7efc941fab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:53.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.498+0000 7efc98e1f640 1 -- 172.21.15.98:0/3242060441 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc9415e650 msgr2=0x7efc941fab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:53.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.498+0000 7efc98e1f640 1 --2- 172.21.15.98:0/3242060441 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc9415e650 0x7efc941fab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.498+0000 7efc98e1f640 1 -- 172.21.15.98:0/3242060441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc940d2e00 msgr2=0x7efc941fa610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:53.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.498+0000 7efc98e1f640 1 --2- 172.21.15.98:0/3242060441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc940d2e00 0x7efc941fa610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.498+0000 7efc98e1f640 1 -- 172.21.15.98:0/3242060441 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc94147360 con 0x7efc940b73d0 2024-09-19T08:46:53.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.498+0000 7efc93fff640 1 --2- 172.21.15.98:0/3242060441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc940d2e00 0x7efc941fa610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:53.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.498+0000 7efc98e1f640 1 --2- 172.21.15.98:0/3242060441 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc940b73d0 0x7efc9414ea70 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7efc840ac040 tx=0x7efc8401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:53.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.498+0000 7efc917fa640 1 -- 172.21.15.98:0/3242060441 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc840bd5f0 con 0x7efc940b73d0 2024-09-19T08:46:53.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.498+0000 7efc917fa640 1 -- 172.21.15.98:0/3242060441 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc840bd7b0 con 0x7efc940b73d0 2024-09-19T08:46:53.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.498+0000 7efc9b083640 1 -- 172.21.15.98:0/3242060441 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc94147640 con 0x7efc940b73d0 2024-09-19T08:46:53.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.498+0000 7efc917fa640 1 -- 172.21.15.98:0/3242060441 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc840bdba0 con 0x7efc940b73d0 2024-09-19T08:46:53.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.498+0000 7efc9b083640 1 -- 172.21.15.98:0/3242060441 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc94147c00 con 0x7efc940b73d0 2024-09-19T08:46:53.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.498+0000 7efc9b083640 1 -- 172.21.15.98:0/3242060441 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc9406bc90 con 0x7efc940b73d0 2024-09-19T08:46:53.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.498+0000 7efc917fa640 1 -- 172.21.15.98:0/3242060441 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7efc840c7420 con 0x7efc940b73d0 2024-09-19T08:46:53.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.498+0000 7efc917fa640 1 --2- 172.21.15.98:0/3242060441 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7efc800680d0 0x7efc8006a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:53.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.498+0000 7efc917fa640 1 -- 172.21.15.98:0/3242060441 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5737+0+0 (secure 0 0 0) 0x7efc84121590 con 0x7efc940b73d0 2024-09-19T08:46:53.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.498+0000 7efc93fff640 1 --2- 172.21.15.98:0/3242060441 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7efc800680d0 0x7efc8006a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:53.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.502+0000 7efc93fff640 1 --2- 172.21.15.98:0/3242060441 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7efc800680d0 0x7efc8006a590 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7efc940ce6a0 tx=0x7efc88008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:53.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.502+0000 7efc917fa640 1 -- 172.21.15.98:0/3242060441 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7efc840c3060 con 0x7efc940b73d0 2024-09-19T08:46:53.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.706+0000 7efc9b083640 1 -- 172.21.15.98:0/3242060441 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7efc940ceb60 con 0x7efc940b73d0 2024-09-19T08:46:53.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.706+0000 7efc917fa640 1 -- 172.21.15.98:0/3242060441 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v91) ==== 53+0+73 (secure 0 0 0) 0x7efc840a7070 con 0x7efc940b73d0 2024-09-19T08:46:53.705 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T08:46:53.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.706+0000 7efc9b083640 1 -- 172.21.15.98:0/3242060441 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7efc800680d0 msgr2=0x7efc8006a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:53.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.706+0000 7efc9b083640 1 --2- 172.21.15.98:0/3242060441 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7efc800680d0 0x7efc8006a590 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7efc940ce6a0 tx=0x7efc88008040 comp rx=0 tx=0).stop 2024-09-19T08:46:53.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.706+0000 7efc9b083640 1 -- 172.21.15.98:0/3242060441 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc940b73d0 msgr2=0x7efc9414ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:53.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.706+0000 7efc9b083640 1 --2- 172.21.15.98:0/3242060441 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc940b73d0 0x7efc9414ea70 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7efc840ac040 tx=0x7efc8401ad30 comp rx=0 tx=0).stop 2024-09-19T08:46:53.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.706+0000 7efc9b083640 1 -- 172.21.15.98:0/3242060441 shutdown_connections 2024-09-19T08:46:53.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.706+0000 7efc9b083640 1 --2- 172.21.15.98:0/3242060441 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7efc800680d0 0x7efc8006a590 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.706+0000 7efc9b083640 1 --2- 172.21.15.98:0/3242060441 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7efc9415e650 0x7efc941fab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.706+0000 7efc9b083640 1 --2- 172.21.15.98:0/3242060441 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efc940d2e00 0x7efc941fa610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.706+0000 7efc9b083640 1 --2- 172.21.15.98:0/3242060441 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7efc940b73d0 0x7efc9414ea70 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:53.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.706+0000 7efc9b083640 1 -- 172.21.15.98:0/3242060441 >> 172.21.15.98:0/3242060441 conn(0x7efc940d2310 msgr2=0x7efc9406c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:53.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.706+0000 7efc9b083640 1 -- 172.21.15.98:0/3242060441 shutdown_connections 2024-09-19T08:46:53.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:53.706+0000 7efc9b083640 1 -- 172.21.15.98:0/3242060441 wait complete. 2024-09-19T08:46:53.724 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T08:46:53.724 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T08:46:53.725 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2024-09-19T08:46:53.725 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-19T08:46:53.725 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-19T08:46:53.725 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T08:46:53.725 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T08:46:53.725 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T08:46:53.726 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T08:46:53.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:53.726 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2024-09-19T08:46:53.734 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T08:46:53.734 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T08:46:53.734 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T08:46:53.734 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:46:53.735 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-09-19T08:46:53.735 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:46:53.735 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:46:53.735 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:46:53.735 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2024-09-19T08:46:53.735 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:46:53.736 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:46:53.736 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2024-09-19T08:46:53.736 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:46:53.736 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:46:53.736 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T08:46:53.736 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:46:53.736 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T08:46:53.737 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 42006sec preferred_lft 42006sec 2024-09-19T08:46:53.737 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::ec4:7aff:fe88:7f59/64 scope link 2024-09-19T08:46:53.737 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:46:53.737 INFO:teuthology.orchestra.run.smithi139.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T08:46:53.737 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 02:42:74:ed:e4:60 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:46:53.737 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T08:46:53.737 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:46:53.737 INFO:teuthology.orchestra.run.smithi139.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T08:46:53.737 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-19T08:46:53.737 INFO:teuthology.orchestra.run.smithi139.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T08:46:53.737 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:46:53.737 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-19T08:46:53.737 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:46:53.738 INFO:teuthology.orchestra.run.smithi139.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:46:53.738 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:46:53.738 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-19T08:46:53.738 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:46:53.738 INFO:teuthology.orchestra.run.smithi139.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:46:53.738 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:46:53.738 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-19T08:46:53.738 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:46:53.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:53.738 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T08:46:53.790 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-09-19T08:46:53.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:53.792 DEBUG:teuthology.orchestra.run.smithi139:> route 2024-09-19T08:46:53.844 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2024-09-19T08:46:53.844 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T08:46:53.845 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T08:46:53.845 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T08:46:53.845 INFO:teuthology.orchestra.run.smithi139.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T08:46:53.845 INFO:teuthology.orchestra.run.smithi139.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T08:46:53.845 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T08:46:53.845 INFO:teuthology.orchestra.run.smithi139.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:46:53.845 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:46:53.845 INFO:teuthology.orchestra.run.smithi139.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:46:53.845 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T08:46:53.845 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:46:53.846 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-09-19T08:46:53.846 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T08:46:53.846 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T08:46:53.846 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T08:46:53.846 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-09-19T08:46:53.938 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:46:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:46:53.966 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:46:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:46:53.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:53.973 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-09-19T08:46:53.978 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-19T08:46:53.978 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-19T08:46:53.978 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T08:46:53.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:53.978 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.0 up 2024-09-19T08:46:54.032 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-19T08:46:54.032 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:46:54.032 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-19T08:46:54.078 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-19T08:46:54.078 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:46:54.078 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-19T08:46:54.127 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:46:54.184 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:46:54.231 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:46:54.231 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:46:54.231 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:46:54.232 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:46:54.232 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:46:54.232 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:46:54.232 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:46:54.232 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:46:54.232 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:46:54.232 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:46:54.232 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:46:54.232 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:46:54.232 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:46:54.232 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:46:54.232 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:46:54.233 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:46:54.233 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:46:54.233 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:46:54.233 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:46:54.233 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:46:54.233 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:46:54.233 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:46:54.233 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:46:54.233 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:46:54.233 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:46:54.233 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:46:54.233 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:46:54.234 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:46:54.234 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:46:54.234 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:46:54.234 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:46:54.234 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:46:54.234 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:46:54.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:54.235 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:46:54.282 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T08:46:54.283 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-19T08:46:54.328 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:46:54.343 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:46:54.390 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:46:54.391 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:46:54.391 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:46:54.391 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:46:54.391 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:46:54.391 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:46:54.391 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:46:54.391 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:46:54.391 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:46:54.391 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:46:54.391 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:46:54.391 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:46:54.392 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:46:54.392 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:46:54.392 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:46:54.392 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:46:54.392 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:46:54.392 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:46:54.392 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:46:54.392 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:46:54.392 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:46:54.392 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:46:54.392 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:46:54.392 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:46:54.393 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:46:54.393 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:46:54.393 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:46:54.393 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:46:54.393 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:46:54.393 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:46:54.393 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:46:54.393 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:46:54.393 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:46:54.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:54.394 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:46:54.407 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:2024-09-19T08:46:54.410+0000 7f9acd8c3580 -1 init, newargv = 0x55694453d080 newargc=15 2024-09-19T08:46:54.408 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[36968]: starting ceph client 2024-09-19T08:46:54.414 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[36968]: starting fuse 2024-09-19T08:46:55.513 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:46:55.529 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:46:55.579 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:46:55.580 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:46:55.580 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:46:55.580 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:46:55.580 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:46:55.580 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:46:55.580 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:46:55.580 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:46:55.580 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:46:55.580 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:46:55.580 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:46:55.580 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:46:55.581 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:46:55.581 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:46:55.581 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:46:55.581 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:46:55.581 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:46:55.581 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:46:55.581 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:46:55.581 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:46:55.581 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:46:55.581 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:46:55.581 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:46:55.581 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:46:55.581 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:46:55.581 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:46:55.582 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:46:55.582 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:46:55.582 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:46:55.582 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:46:55.582 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:46:55.582 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:46:55.582 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:46:55.582 INFO:teuthology.orchestra.run.smithi139.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-19T08:46:55.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:55.583 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:46:55.630 INFO:teuthology.orchestra.run.smithi139.stdout:45 2024-09-19T08:46:55.631 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-19T08:46:55.631 DEBUG:teuthology.orchestra.run.smithi139:> sudo stdin-killer -- python3 -c ' 2024-09-19T08:46:55.631 DEBUG:teuthology.orchestra.run.smithi139:> import glob 2024-09-19T08:46:55.631 DEBUG:teuthology.orchestra.run.smithi139:> import re 2024-09-19T08:46:55.631 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-09-19T08:46:55.631 DEBUG:teuthology.orchestra.run.smithi139:> import subprocess 2024-09-19T08:46:55.631 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:46:55.631 DEBUG:teuthology.orchestra.run.smithi139:> def _find_admin_socket(client_name): 2024-09-19T08:46:55.631 DEBUG:teuthology.orchestra.run.smithi139:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-19T08:46:55.632 DEBUG:teuthology.orchestra.run.smithi139:> files = glob.glob(asok_path) 2024-09-19T08:46:55.632 DEBUG:teuthology.orchestra.run.smithi139:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-19T08:46:55.632 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:46:55.632 DEBUG:teuthology.orchestra.run.smithi139:> # Given a non-glob path, it better be there 2024-09-19T08:46:55.632 DEBUG:teuthology.orchestra.run.smithi139:> if "*" not in asok_path: 2024-09-19T08:46:55.632 DEBUG:teuthology.orchestra.run.smithi139:> assert(len(files) == 1) 2024-09-19T08:46:55.632 DEBUG:teuthology.orchestra.run.smithi139:> return files[0] 2024-09-19T08:46:55.632 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:46:55.632 DEBUG:teuthology.orchestra.run.smithi139:> for f in files: 2024-09-19T08:46:55.632 DEBUG:teuthology.orchestra.run.smithi139:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T08:46:55.632 DEBUG:teuthology.orchestra.run.smithi139:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T08:46:55.632 DEBUG:teuthology.orchestra.run.smithi139:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T08:46:55.632 DEBUG:teuthology.orchestra.run.smithi139:> contents = proc_f.read() 2024-09-19T08:46:55.633 DEBUG:teuthology.orchestra.run.smithi139:> if mountpoint in contents: 2024-09-19T08:46:55.633 DEBUG:teuthology.orchestra.run.smithi139:> return f 2024-09-19T08:46:55.633 DEBUG:teuthology.orchestra.run.smithi139:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T08:46:55.633 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:46:55.633 DEBUG:teuthology.orchestra.run.smithi139:> print(_find_admin_socket("client.1")) 2024-09-19T08:46:55.633 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-09-19T08:46:55.757 INFO:teuthology.orchestra.run.smithi139.stdout:/var/run/ceph/ceph-client.1.36968.asok 2024-09-19T08:46:55.761 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:46:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:46:55.769 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.36968.asok 2024-09-19T08:46:55.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:55.769 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.36968.asok status 2024-09-19T08:46:55.922 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-19T08:46:55.922 INFO:teuthology.orchestra.run.smithi139.stdout: "metadata": { 2024-09-19T08:46:55.922 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T08:46:55.922 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T08:46:55.923 INFO:teuthology.orchestra.run.smithi139.stdout: "entity_id": "1", 2024-09-19T08:46:55.923 INFO:teuthology.orchestra.run.smithi139.stdout: "hostname": "smithi139", 2024-09-19T08:46:55.923 INFO:teuthology.orchestra.run.smithi139.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-19T08:46:55.923 INFO:teuthology.orchestra.run.smithi139.stdout: "pid": "36968", 2024-09-19T08:46:55.923 INFO:teuthology.orchestra.run.smithi139.stdout: "root": "/" 2024-09-19T08:46:55.923 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:46:55.923 INFO:teuthology.orchestra.run.smithi139.stdout: "dentry_count": 0, 2024-09-19T08:46:55.923 INFO:teuthology.orchestra.run.smithi139.stdout: "dentry_pinned_count": 0, 2024-09-19T08:46:55.923 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 7186, 2024-09-19T08:46:55.923 INFO:teuthology.orchestra.run.smithi139.stdout: "inst": { 2024-09-19T08:46:55.923 INFO:teuthology.orchestra.run.smithi139.stdout: "name": { 2024-09-19T08:46:55.923 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "client", 2024-09-19T08:46:55.923 INFO:teuthology.orchestra.run.smithi139.stdout: "num": 7186 2024-09-19T08:46:55.923 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:46:55.923 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": { 2024-09-19T08:46:55.924 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-19T08:46:55.924 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:46:55.924 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3080370725 2024-09-19T08:46:55.924 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-19T08:46:55.924 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:46:55.924 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": { 2024-09-19T08:46:55.924 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-19T08:46:55.924 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:46:55.924 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 3080370725 2024-09-19T08:46:55.924 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:46:55.924 INFO:teuthology.orchestra.run.smithi139.stdout: "inst_str": "client.7186 192.168.0.1:0/3080370725", 2024-09-19T08:46:55.924 INFO:teuthology.orchestra.run.smithi139.stdout: "addr_str": "192.168.0.1:0/3080370725", 2024-09-19T08:46:55.924 INFO:teuthology.orchestra.run.smithi139.stdout: "inode_count": 1, 2024-09-19T08:46:55.924 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_epoch": 91, 2024-09-19T08:46:55.924 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_epoch": 158, 2024-09-19T08:46:55.925 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_epoch_barrier": 0, 2024-09-19T08:46:55.925 INFO:teuthology.orchestra.run.smithi139.stdout: "blocklisted": false, 2024-09-19T08:46:55.925 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs" 2024-09-19T08:46:55.925 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-19T08:46:55.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:55.937 DEBUG:teuthology.orchestra.run.smithi139:> stat --file-system '--printf=%T 2024-09-19T08:46:55.937 DEBUG:teuthology.orchestra.run.smithi139:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-19T08:46:55.943 INFO:teuthology.orchestra.run.smithi139.stdout:fuseblk 2024-09-19T08:46:55.943 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-19T08:46:55.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:55.943 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-19T08:46:55.996 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-19T08:46:56.134 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.134+0000 7fceb3414640 1 Processor -- start 2024-09-19T08:46:56.135 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.134+0000 7fceb3414640 1 -- start start 2024-09-19T08:46:56.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.134+0000 7fceb3414640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fceac15e720 0x7fceac0cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:56.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.134+0000 7fceb3414640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fceac0d29c0 con 0x7fceac0cfb50 2024-09-19T08:46:56.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.134+0000 7fceb3414640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fceac0d2ca0 con 0x7fceac0d3a60 2024-09-19T08:46:56.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.134+0000 7fceb3414640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fceac0d2f80 con 0x7fceac15e720 2024-09-19T08:46:56.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.134+0000 7fceb09af640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fceac0cfb50 0x7fceac0b7770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:53062/0 (socket says 172.21.15.98:53062) 2024-09-19T08:46:56.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.134+0000 7fceb09af640 1 -- 172.21.15.98:0/1929834763 learned_addr learned my addr 172.21.15.98:0/1929834763 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:56.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.138+0000 7fceb11b0640 1 --2- 172.21.15.98:0/1929834763 >> v2:172.21.15.139:3300/0 conn(0x7fceac15e720 0x7fceac0cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:56.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.138+0000 7fcea3fff640 1 -- 172.21.15.98:0/1929834763 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 709299148 0 0) 0x7fceac0d29c0 con 0x7fceac0cfb50 2024-09-19T08:46:56.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.138+0000 7fcea3fff640 1 -- 172.21.15.98:0/1929834763 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce98003530 con 0x7fceac0cfb50 2024-09-19T08:46:56.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.138+0000 7fcea3fff640 1 -- 172.21.15.98:0/1929834763 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2183119234 0 0) 0x7fce98003530 con 0x7fceac0cfb50 2024-09-19T08:46:56.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.138+0000 7fcea3fff640 1 -- 172.21.15.98:0/1929834763 >> v2:172.21.15.139:3300/0 conn(0x7fceac15e720 msgr2=0x7fceac0cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:56.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.138+0000 7fcea3fff640 1 --2- 172.21.15.98:0/1929834763 >> v2:172.21.15.139:3300/0 conn(0x7fceac15e720 0x7fceac0cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.138+0000 7fcea3fff640 1 -- 172.21.15.98:0/1929834763 >> v1:172.21.15.139:6789/0 conn(0x7fceac0d3a60 legacy=0x7fceac1623b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:56.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.138+0000 7fcea3fff640 1 -- 172.21.15.98:0/1929834763 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceac0d3260 con 0x7fceac0cfb50 2024-09-19T08:46:56.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.138+0000 7fcea3fff640 1 -- 172.21.15.98:0/1929834763 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fce9c001680 con 0x7fceac0cfb50 2024-09-19T08:46:56.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.138+0000 7fcea3fff640 1 -- 172.21.15.98:0/1929834763 >> v1:172.21.15.98:6789/0 conn(0x7fceac0cfb50 legacy=0x7fceac0b7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:56.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.138+0000 7fcea3fff640 1 --2- 172.21.15.98:0/1929834763 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce980039c0 0x7fce98003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:56.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.138+0000 7fcea3fff640 1 --2- 172.21.15.98:0/1929834763 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce98004810 0x7fce98004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:56.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.138+0000 7fcea3fff640 1 --2- 172.21.15.98:0/1929834763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce980051d0 0x7fce9800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:56.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.138+0000 7fcea3fff640 1 -- 172.21.15.98:0/1929834763 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fceac0d2f80 con 0x7fce980039c0 2024-09-19T08:46:56.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.138+0000 7fcea3fff640 1 -- 172.21.15.98:0/1929834763 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fce9800c070 con 0x7fce980051d0 2024-09-19T08:46:56.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.138+0000 7fcea3fff640 1 -- 172.21.15.98:0/1929834763 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fce9800c270 con 0x7fce98004810 2024-09-19T08:46:56.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.138+0000 7fceb11b0640 1 --2- 172.21.15.98:0/1929834763 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce980039c0 0x7fce98003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:56.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.138+0000 7fceb19b1640 1 --2- 172.21.15.98:0/1929834763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce980051d0 0x7fce9800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:56.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.138+0000 7fceb09af640 1 --2- 172.21.15.98:0/1929834763 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce98004810 0x7fce98004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:56.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.138+0000 7fceb11b0640 1 -- 172.21.15.98:0/1929834763 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce98004810 msgr2=0x7fce98004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:56.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.138+0000 7fceb11b0640 1 --2- 172.21.15.98:0/1929834763 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce98004810 0x7fce98004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.138+0000 7fceb11b0640 1 -- 172.21.15.98:0/1929834763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce980051d0 msgr2=0x7fce9800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:56.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.138+0000 7fceb11b0640 1 --2- 172.21.15.98:0/1929834763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce980051d0 0x7fce9800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.138+0000 7fceb11b0640 1 -- 172.21.15.98:0/1929834763 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce98003530 con 0x7fce980039c0 2024-09-19T08:46:56.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.138+0000 7fceb09af640 1 --2- 172.21.15.98:0/1929834763 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fce98004810 0x7fce98004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:56.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.138+0000 7fceb19b1640 1 -- 172.21.15.98:0/1929834763 reap_dead start 2024-09-19T08:46:56.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.138+0000 7fceb11b0640 1 --2- 172.21.15.98:0/1929834763 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce980039c0 0x7fce98003de0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fcea8009ac0 tx=0x7fcea801ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d09d4534c7f7042 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:56.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.138+0000 7fcea3fff640 1 -- 172.21.15.98:0/1929834763 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea8009690 con 0x7fce980039c0 2024-09-19T08:46:56.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.138+0000 7fcea3fff640 1 -- 172.21.15.98:0/1929834763 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcea80a6510 con 0x7fce980039c0 2024-09-19T08:46:56.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.138+0000 7fcea3fff640 1 -- 172.21.15.98:0/1929834763 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea80a68e0 con 0x7fce980039c0 2024-09-19T08:46:56.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.142+0000 7fceb3414640 1 -- 172.21.15.98:0/1929834763 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce980039c0 msgr2=0x7fce98003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:56.148 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.142+0000 7fceb3414640 1 --2- 172.21.15.98:0/1929834763 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce980039c0 0x7fce98003de0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fcea8009ac0 tx=0x7fcea801ca20 comp rx=0 tx=0).stop 2024-09-19T08:46:56.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.142+0000 7fceb3414640 1 -- 172.21.15.98:0/1929834763 shutdown_connections 2024-09-19T08:46:56.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.142+0000 7fceb3414640 1 --2- 172.21.15.98:0/1929834763 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fce980039c0 0x7fce98003de0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.142+0000 7fceb3414640 1 -- 172.21.15.98:0/1929834763 >> 172.21.15.98:0/1929834763 conn(0x7fceac0d4640 msgr2=0x7fceac0d4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:56.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.142+0000 7fceb3414640 1 -- 172.21.15.98:0/1929834763 shutdown_connections 2024-09-19T08:46:56.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.142+0000 7fceb3414640 1 -- 172.21.15.98:0/1929834763 wait complete. 2024-09-19T08:46:56.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.142+0000 7fceb3414640 1 Processor -- start 2024-09-19T08:46:56.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.142+0000 7fceb3414640 1 -- start start 2024-09-19T08:46:56.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.142+0000 7fceb3414640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce980039c0 0x7fceac15ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:56.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.142+0000 7fceb3414640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fceac15f3a0 0x7fceac160e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:56.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.142+0000 7fceb3414640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fceac161350 0x7fceac1617d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:56.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.142+0000 7fceb3414640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fceac15eb00 con 0x7fceac161350 2024-09-19T08:46:56.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.142+0000 7fceb3414640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fceac0b7f80 con 0x7fce980039c0 2024-09-19T08:46:56.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.142+0000 7fceb3414640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fceac15ec80 con 0x7fceac15f3a0 2024-09-19T08:46:56.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.142+0000 7fceb19b1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fceac161350 0x7fceac1617d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:56.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.142+0000 7fceb19b1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fceac161350 0x7fceac1617d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54448/0 (socket says 172.21.15.98:54448) 2024-09-19T08:46:56.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.142+0000 7fceb19b1640 1 -- 172.21.15.98:0/1268203634 learned_addr learned my addr 172.21.15.98:0/1268203634 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:56.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.142+0000 7fceb11b0640 1 --2- 172.21.15.98:0/1268203634 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce980039c0 0x7fceac15ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:56.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.142+0000 7fceb09af640 1 --2- 172.21.15.98:0/1268203634 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fceac15f3a0 0x7fceac160e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:56.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.142+0000 7fceb19b1640 1 -- 172.21.15.98:0/1268203634 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fceac15f3a0 msgr2=0x7fceac160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:56.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.142+0000 7fceb19b1640 1 --2- 172.21.15.98:0/1268203634 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fceac15f3a0 0x7fceac160e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.142+0000 7fceb19b1640 1 -- 172.21.15.98:0/1268203634 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce980039c0 msgr2=0x7fceac15ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:56.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.142+0000 7fceb19b1640 1 --2- 172.21.15.98:0/1268203634 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce980039c0 0x7fceac15ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.142+0000 7fceb19b1640 1 -- 172.21.15.98:0/1268203634 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceac161d10 con 0x7fceac161350 2024-09-19T08:46:56.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.142+0000 7fceb09af640 1 --2- 172.21.15.98:0/1268203634 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fceac15f3a0 0x7fceac160e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:56.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.142+0000 7fceb19b1640 1 --2- 172.21.15.98:0/1268203634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fceac161350 0x7fceac1617d0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fceac06cc20 tx=0x7fcea400e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:56.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.146+0000 7fcea1ffb640 1 -- 172.21.15.98:0/1268203634 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea4019070 con 0x7fceac161350 2024-09-19T08:46:56.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.146+0000 7fcea1ffb640 1 -- 172.21.15.98:0/1268203634 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcea4006e30 con 0x7fceac161350 2024-09-19T08:46:56.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.146+0000 7fcea1ffb640 1 -- 172.21.15.98:0/1268203634 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea4003b80 con 0x7fceac161350 2024-09-19T08:46:56.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.146+0000 7fceb3414640 1 -- 172.21.15.98:0/1268203634 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceac1ecb20 con 0x7fceac161350 2024-09-19T08:46:56.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.146+0000 7fceb3414640 1 -- 172.21.15.98:0/1268203634 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceac0c8cc0 con 0x7fceac161350 2024-09-19T08:46:56.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.146+0000 7fceb3414640 1 -- 172.21.15.98:0/1268203634 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fceac1623b0 con 0x7fceac161350 2024-09-19T08:46:56.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.146+0000 7fcea1ffb640 1 -- 172.21.15.98:0/1268203634 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7fcea4005110 con 0x7fceac161350 2024-09-19T08:46:56.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.146+0000 7fcea1ffb640 1 --2- 172.21.15.98:0/1268203634 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fce900680a0 0x7fce9006a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:56.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.146+0000 7fcea1ffb640 1 -- 172.21.15.98:0/1268203634 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5737+0+0 (secure 0 0 0) 0x7fcea4083b20 con 0x7fceac161350 2024-09-19T08:46:56.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.146+0000 7fceb11b0640 1 --2- 172.21.15.98:0/1268203634 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fce900680a0 0x7fce9006a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:56.151 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.146+0000 7fceb11b0640 1 --2- 172.21.15.98:0/1268203634 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fce900680a0 0x7fce9006a560 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fcea80af000 tx=0x7fcea8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:56.152 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.150+0000 7fcea1ffb640 1 -- 172.21.15.98:0/1268203634 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fcea4055f50 con 0x7fceac161350 2024-09-19T08:46:56.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.346+0000 7fceb3414640 1 -- 172.21.15.98:0/1268203634 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fceac15f780 con 0x7fceac161350 2024-09-19T08:46:56.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.582+0000 7fcea1ffb640 1 -- 172.21.15.98:0/1268203634 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v92) ==== 136+0+0 (secure 0 0 0) 0x7fcea4055850 con 0x7fceac161350 2024-09-19T08:46:56.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.586+0000 7fceb3414640 1 -- 172.21.15.98:0/1268203634 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fce900680a0 msgr2=0x7fce9006a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:56.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.586+0000 7fceb3414640 1 --2- 172.21.15.98:0/1268203634 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fce900680a0 0x7fce9006a560 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fcea80af000 tx=0x7fcea8008040 comp rx=0 tx=0).stop 2024-09-19T08:46:56.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.586+0000 7fceb3414640 1 -- 172.21.15.98:0/1268203634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fceac161350 msgr2=0x7fceac1617d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:56.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.586+0000 7fceb3414640 1 --2- 172.21.15.98:0/1268203634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fceac161350 0x7fceac1617d0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fceac06cc20 tx=0x7fcea400e820 comp rx=0 tx=0).stop 2024-09-19T08:46:56.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.586+0000 7fceb3414640 1 -- 172.21.15.98:0/1268203634 shutdown_connections 2024-09-19T08:46:56.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.586+0000 7fceb3414640 1 --2- 172.21.15.98:0/1268203634 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fce900680a0 0x7fce9006a560 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.586+0000 7fceb3414640 1 --2- 172.21.15.98:0/1268203634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fceac161350 0x7fceac1617d0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.586+0000 7fceb3414640 1 --2- 172.21.15.98:0/1268203634 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fceac15f3a0 0x7fceac160e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.586+0000 7fceb3414640 1 --2- 172.21.15.98:0/1268203634 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce980039c0 0x7fceac15ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.586+0000 7fceb3414640 1 -- 172.21.15.98:0/1268203634 >> 172.21.15.98:0/1268203634 conn(0x7fceac0d4640 msgr2=0x7fceac0b4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:56.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.586+0000 7fceb3414640 1 -- 172.21.15.98:0/1268203634 shutdown_connections 2024-09-19T08:46:56.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.586+0000 7fceb3414640 1 -- 172.21.15.98:0/1268203634 wait complete. 2024-09-19T08:46:56.606 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-19T08:46:56.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.734+0000 7f79c9d3b640 1 Processor -- start 2024-09-19T08:46:56.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.734+0000 7f79c9d3b640 1 -- start start 2024-09-19T08:46:56.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.734+0000 7f79c9d3b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79c406c3a0 0x7f79c40cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:56.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.734+0000 7f79c9d3b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f79c40cf900 0x7f79c40723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:56.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.734+0000 7f79c9d3b640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79c406d310 con 0x7f79c415c550 2024-09-19T08:46:56.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.734+0000 7f79c9d3b640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f79c406d5f0 con 0x7f79c40cf900 2024-09-19T08:46:56.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.734+0000 7f79c9d3b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f79c406d8d0 con 0x7f79c406c3a0 2024-09-19T08:46:56.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.734+0000 7f79c3fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f79c40cf900 0x7f79c40723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:56.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.734+0000 7f79c37fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f79c415c550 0x7f79c406bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:53078/0 (socket says 172.21.15.98:53078) 2024-09-19T08:46:56.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.734+0000 7f79c37fe640 1 -- 172.21.15.98:0/2681385634 learned_addr learned my addr 172.21.15.98:0/2681385634 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:56.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.734+0000 7f79c2ffd640 1 --2- 172.21.15.98:0/2681385634 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79c406c3a0 0x7f79c40cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:56.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.734+0000 7f79c3fff640 1 -- 172.21.15.98:0/2681385634 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79c406c3a0 msgr2=0x7f79c40cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:56.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.734+0000 7f79c3fff640 1 --2- 172.21.15.98:0/2681385634 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79c406c3a0 0x7f79c40cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.734+0000 7f79c3fff640 1 -- 172.21.15.98:0/2681385634 >> v1:172.21.15.98:6789/0 conn(0x7f79c415c550 legacy=0x7f79c406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:56.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.734+0000 7f79c27fc640 1 -- 172.21.15.98:0/2681385634 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 489967935 0 0) 0x7f79c406d310 con 0x7f79c415c550 2024-09-19T08:46:56.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.734+0000 7f79c3fff640 1 -- 172.21.15.98:0/2681385634 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c406dbb0 con 0x7f79c40cf900 2024-09-19T08:46:56.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.734+0000 7f79c2ffd640 1 --2- 172.21.15.98:0/2681385634 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79c406c3a0 0x7f79c40cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:56.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.738+0000 7f79c3fff640 1 --2- 172.21.15.98:0/2681385634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79c40cf900 0x7f79c40723a0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f79b4009d90 tx=0x7f79b401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c47a5e296d8b1a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:56.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.738+0000 7f79c27fc640 1 -- 172.21.15.98:0/2681385634 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b40b2070 con 0x7f79c40cf900 2024-09-19T08:46:56.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.738+0000 7f79c27fc640 1 -- 172.21.15.98:0/2681385634 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79b40a5940 con 0x7f79c40cf900 2024-09-19T08:46:56.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.738+0000 7f79c27fc640 1 -- 172.21.15.98:0/2681385634 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b40a5d10 con 0x7f79c40cf900 2024-09-19T08:46:56.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.738+0000 7f79c9d3b640 1 -- 172.21.15.98:0/2681385634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79c40cf900 msgr2=0x7f79c40723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:56.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.738+0000 7f79c9d3b640 1 --2- 172.21.15.98:0/2681385634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79c40cf900 0x7f79c40723a0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f79b4009d90 tx=0x7f79b401c970 comp rx=0 tx=0).stop 2024-09-19T08:46:56.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.738+0000 7f79c9d3b640 1 -- 172.21.15.98:0/2681385634 shutdown_connections 2024-09-19T08:46:56.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.738+0000 7f79c9d3b640 1 --2- 172.21.15.98:0/2681385634 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79c40cf900 0x7f79c40723a0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.738+0000 7f79c9d3b640 1 --2- 172.21.15.98:0/2681385634 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79c406c3a0 0x7f79c40cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.738+0000 7f79c9d3b640 1 -- 172.21.15.98:0/2681385634 >> 172.21.15.98:0/2681385634 conn(0x7f79c40b2010 msgr2=0x7f79c40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:56.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.738+0000 7f79c9d3b640 1 -- 172.21.15.98:0/2681385634 shutdown_connections 2024-09-19T08:46:56.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.738+0000 7f79c9d3b640 1 -- 172.21.15.98:0/2681385634 wait complete. 2024-09-19T08:46:56.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.738+0000 7f79c9d3b640 1 Processor -- start 2024-09-19T08:46:56.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.738+0000 7f79c9d3b640 1 -- start start 2024-09-19T08:46:56.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.742+0000 7f79c9d3b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79c406c3a0 0x7f79c414ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:56.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.742+0000 7f79c9d3b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79c40cf900 0x7f79c4138700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:56.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.742+0000 7f79c9d3b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79c415c550 0x7f79c414dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:56.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.742+0000 7f79c9d3b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f79c4073480 con 0x7f79c40cf900 2024-09-19T08:46:56.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.742+0000 7f79c9d3b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f79c4073300 con 0x7f79c415c550 2024-09-19T08:46:56.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.742+0000 7f79c9d3b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f79c4073600 con 0x7f79c406c3a0 2024-09-19T08:46:56.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.742+0000 7f79c2ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79c40cf900 0x7f79c4138700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:56.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.742+0000 7f79c2ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79c40cf900 0x7f79c4138700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54460/0 (socket says 172.21.15.98:54460) 2024-09-19T08:46:56.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.742+0000 7f79c2ffd640 1 -- 172.21.15.98:0/3475629684 learned_addr learned my addr 172.21.15.98:0/3475629684 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:56.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.742+0000 7f79c37fe640 1 --2- 172.21.15.98:0/3475629684 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79c406c3a0 0x7f79c414ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:56.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.742+0000 7f79c3fff640 1 --2- 172.21.15.98:0/3475629684 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79c415c550 0x7f79c414dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:56.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.742+0000 7f79c2ffd640 1 -- 172.21.15.98:0/3475629684 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79c406c3a0 msgr2=0x7f79c414ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:56.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.742+0000 7f79c2ffd640 1 --2- 172.21.15.98:0/3475629684 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79c406c3a0 0x7f79c414ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.742+0000 7f79c2ffd640 1 -- 172.21.15.98:0/3475629684 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79c415c550 msgr2=0x7f79c414dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:56.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.742+0000 7f79c2ffd640 1 --2- 172.21.15.98:0/3475629684 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79c415c550 0x7f79c414dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.742+0000 7f79c2ffd640 1 -- 172.21.15.98:0/3475629684 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c414d520 con 0x7f79c40cf900 2024-09-19T08:46:56.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.742+0000 7f79c2ffd640 1 --2- 172.21.15.98:0/3475629684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79c40cf900 0x7f79c4138700 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f79ac00d800 tx=0x7f79ac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:56.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.742+0000 7f79bbfff640 1 -- 172.21.15.98:0/3475629684 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac018020 con 0x7f79c40cf900 2024-09-19T08:46:56.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.742+0000 7f79bbfff640 1 -- 172.21.15.98:0/3475629684 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79ac009660 con 0x7f79c40cf900 2024-09-19T08:46:56.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.742+0000 7f79c9d3b640 1 -- 172.21.15.98:0/3475629684 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79c4138f50 con 0x7f79c40cf900 2024-09-19T08:46:56.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.742+0000 7f79c9d3b640 1 -- 172.21.15.98:0/3475629684 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79c4139510 con 0x7f79c40cf900 2024-09-19T08:46:56.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.742+0000 7f79bbfff640 1 -- 172.21.15.98:0/3475629684 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ac009a50 con 0x7f79c40cf900 2024-09-19T08:46:56.744 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.742+0000 7f79c9d3b640 1 -- 172.21.15.98:0/3475629684 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79a80052a0 con 0x7f79c40cf900 2024-09-19T08:46:56.744 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.742+0000 7f79bbfff640 1 -- 172.21.15.98:0/3475629684 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f79ac01b020 con 0x7f79c40cf900 2024-09-19T08:46:56.744 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.742+0000 7f79bbfff640 1 --2- 172.21.15.98:0/3475629684 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f79a40680a0 0x7f79a406a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:56.744 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.742+0000 7f79bbfff640 1 -- 172.21.15.98:0/3475629684 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5737+0+0 (secure 0 0 0) 0x7f79ac084b20 con 0x7f79c40cf900 2024-09-19T08:46:56.744 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.742+0000 7f79c37fe640 1 --2- 172.21.15.98:0/3475629684 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f79a40680a0 0x7f79a406a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:56.744 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.746+0000 7f79c37fe640 1 --2- 172.21.15.98:0/3475629684 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f79a40680a0 0x7f79a406a560 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f79bc000f70 tx=0x7f79bc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:56.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.746+0000 7f79bbfff640 1 -- 172.21.15.98:0/3475629684 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f79ac022020 con 0x7f79c40cf900 2024-09-19T08:46:56.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.934+0000 7f79c9d3b640 1 -- 172.21.15.98:0/3475629684 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f79a8000bd0 con 0x7f79a40680a0 2024-09-19T08:46:56.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.934+0000 7f79bbfff640 1 -- 172.21.15.98:0/3475629684 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f79a8000bd0 con 0x7f79a40680a0 2024-09-19T08:46:56.935 INFO:teuthology.orchestra.run.smithi098.stdout:[ 2024-09-19T08:46:56.935 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:46:56.935 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "cephfs" 2024-09-19T08:46:56.936 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:46:56.936 INFO:teuthology.orchestra.run.smithi098.stdout:] 2024-09-19T08:46:56.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.938+0000 7f79c9d3b640 1 -- 172.21.15.98:0/3475629684 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f79a40680a0 msgr2=0x7f79a406a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:56.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.938+0000 7f79c9d3b640 1 --2- 172.21.15.98:0/3475629684 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f79a40680a0 0x7f79a406a560 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f79bc000f70 tx=0x7f79bc009040 comp rx=0 tx=0).stop 2024-09-19T08:46:56.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.938+0000 7f79c9d3b640 1 -- 172.21.15.98:0/3475629684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79c40cf900 msgr2=0x7f79c4138700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:56.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.938+0000 7f79c9d3b640 1 --2- 172.21.15.98:0/3475629684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79c40cf900 0x7f79c4138700 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f79ac00d800 tx=0x7f79ac00dcc0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.938+0000 7f79c9d3b640 1 -- 172.21.15.98:0/3475629684 shutdown_connections 2024-09-19T08:46:56.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.938+0000 7f79c9d3b640 1 --2- 172.21.15.98:0/3475629684 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f79a40680a0 0x7f79a406a560 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.938+0000 7f79c9d3b640 1 --2- 172.21.15.98:0/3475629684 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79c415c550 0x7f79c414dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.938+0000 7f79c9d3b640 1 --2- 172.21.15.98:0/3475629684 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f79c40cf900 0x7f79c4138700 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.938+0000 7f79c9d3b640 1 --2- 172.21.15.98:0/3475629684 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f79c406c3a0 0x7f79c414ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:56.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.938+0000 7f79c9d3b640 1 -- 172.21.15.98:0/3475629684 >> 172.21.15.98:0/3475629684 conn(0x7f79c40b2010 msgr2=0x7f79c4156950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:56.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.938+0000 7f79c9d3b640 1 -- 172.21.15.98:0/3475629684 shutdown_connections 2024-09-19T08:46:56.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:56.938+0000 7f79c9d3b640 1 -- 172.21.15.98:0/3475629684 wait complete. 2024-09-19T08:46:56.954 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T08:46:57.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.082+0000 7f5ca97f6640 1 Processor -- start 2024-09-19T08:46:57.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.082+0000 7f5ca97f6640 1 -- start start 2024-09-19T08:46:57.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.082+0000 7f5ca97f6640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5ca406c3a0 0x7f5ca416d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:57.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.082+0000 7f5ca97f6640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ca40ce3c0 con 0x7f5ca415cf90 2024-09-19T08:46:57.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.082+0000 7f5ca97f6640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ca40cd090 con 0x7f5ca415cbb0 2024-09-19T08:46:57.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.082+0000 7f5ca97f6640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f5ca40cd370 con 0x7f5ca406c3a0 2024-09-19T08:46:57.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.082+0000 7f5ca37fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5ca406c3a0 0x7f5ca416d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:57.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.082+0000 7f5c9bfff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f5ca415cf90 0x7f5ca406bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:53084/0 (socket says 172.21.15.98:53084) 2024-09-19T08:46:57.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.082+0000 7f5c9bfff640 1 -- 172.21.15.98:0/1359086874 learned_addr learned my addr 172.21.15.98:0/1359086874 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:57.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.082+0000 7f5ca37fe640 1 -- 172.21.15.98:0/1359086874 >> v1:172.21.15.139:6789/0 conn(0x7f5ca415cbb0 legacy=0x7f5ca4163930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:57.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.082+0000 7f5ca37fe640 1 -- 172.21.15.98:0/1359086874 >> v1:172.21.15.98:6789/0 conn(0x7f5ca415cf90 legacy=0x7f5ca406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:57.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.082+0000 7f5ca37fe640 1 -- 172.21.15.98:0/1359086874 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca40cd650 con 0x7f5ca406c3a0 2024-09-19T08:46:57.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.082+0000 7f5ca27fc640 1 -- 172.21.15.98:0/1359086874 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1227955167 0 0) 0x7f5ca40ce3c0 con 0x7f5ca415cf90 2024-09-19T08:46:57.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.086+0000 7f5ca37fe640 1 --2- 172.21.15.98:0/1359086874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ca406c3a0 0x7f5ca416d610 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009d90 tx=0x7f5c9c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4cb33429dfa8ef3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:57.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.086+0000 7f5ca27fc640 1 -- 172.21.15.98:0/1359086874 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c0b2070 con 0x7f5ca406c3a0 2024-09-19T08:46:57.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.086+0000 7f5ca27fc640 1 -- 172.21.15.98:0/1359086874 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c9c0a5830 con 0x7f5ca406c3a0 2024-09-19T08:46:57.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.086+0000 7f5ca27fc640 1 -- 172.21.15.98:0/1359086874 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c0a5c00 con 0x7f5ca406c3a0 2024-09-19T08:46:57.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.086+0000 7f5ca97f6640 1 -- 172.21.15.98:0/1359086874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ca406c3a0 msgr2=0x7f5ca416d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:57.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.086+0000 7f5ca97f6640 1 --2- 172.21.15.98:0/1359086874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ca406c3a0 0x7f5ca416d610 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009d90 tx=0x7f5c9c01c970 comp rx=0 tx=0).stop 2024-09-19T08:46:57.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.086+0000 7f5ca97f6640 1 -- 172.21.15.98:0/1359086874 shutdown_connections 2024-09-19T08:46:57.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.086+0000 7f5ca97f6640 1 --2- 172.21.15.98:0/1359086874 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ca406c3a0 0x7f5ca416d610 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:57.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.086+0000 7f5ca97f6640 1 -- 172.21.15.98:0/1359086874 >> 172.21.15.98:0/1359086874 conn(0x7f5ca40b2010 msgr2=0x7f5ca40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:57.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.086+0000 7f5ca97f6640 1 -- 172.21.15.98:0/1359086874 shutdown_connections 2024-09-19T08:46:57.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.086+0000 7f5ca97f6640 1 -- 172.21.15.98:0/1359086874 wait complete. 2024-09-19T08:46:57.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.086+0000 7f5ca97f6640 1 Processor -- start 2024-09-19T08:46:57.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.086+0000 7f5ca97f6640 1 -- start start 2024-09-19T08:46:57.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.086+0000 7f5ca97f6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ca406c3a0 0x7f5ca415bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:57.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.086+0000 7f5ca97f6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ca415cbb0 0x7f5ca415f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:57.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.086+0000 7f5ca97f6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ca415cf90 0x7f5ca415f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:57.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.086+0000 7f5ca97f6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca40cce70 con 0x7f5ca415cbb0 2024-09-19T08:46:57.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.086+0000 7f5ca97f6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca40cccf0 con 0x7f5ca406c3a0 2024-09-19T08:46:57.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.086+0000 7f5ca97f6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5ca4170780 con 0x7f5ca415cf90 2024-09-19T08:46:57.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.086+0000 7f5c9bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ca415cbb0 0x7f5ca415f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:57.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.086+0000 7f5c9bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ca415cbb0 0x7f5ca415f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:54474/0 (socket says 172.21.15.98:54474) 2024-09-19T08:46:57.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.086+0000 7f5c9bfff640 1 -- 172.21.15.98:0/113804962 learned_addr learned my addr 172.21.15.98:0/113804962 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:57.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.086+0000 7f5ca2ffd640 1 --2- 172.21.15.98:0/113804962 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ca406c3a0 0x7f5ca415bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:57.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.086+0000 7f5ca37fe640 1 --2- 172.21.15.98:0/113804962 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ca415cf90 0x7f5ca415f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:57.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.086+0000 7f5c9bfff640 1 -- 172.21.15.98:0/113804962 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ca415cf90 msgr2=0x7f5ca415f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:57.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.086+0000 7f5c9bfff640 1 --2- 172.21.15.98:0/113804962 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ca415cf90 0x7f5ca415f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:57.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.086+0000 7f5c9bfff640 1 -- 172.21.15.98:0/113804962 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ca406c3a0 msgr2=0x7f5ca415bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:57.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.086+0000 7f5c9bfff640 1 --2- 172.21.15.98:0/113804962 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ca406c3a0 0x7f5ca415bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:57.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.086+0000 7f5c9bfff640 1 -- 172.21.15.98:0/113804962 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca4162bf0 con 0x7f5ca415cbb0 2024-09-19T08:46:57.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.086+0000 7f5ca37fe640 1 --2- 172.21.15.98:0/113804962 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ca415cf90 0x7f5ca415f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:57.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.086+0000 7f5c9bfff640 1 --2- 172.21.15.98:0/113804962 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ca415cbb0 0x7f5ca415f270 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00c440 tx=0x7f5c8c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:57.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.086+0000 7f5c9b7fe640 1 -- 172.21.15.98:0/113804962 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c018070 con 0x7f5ca415cbb0 2024-09-19T08:46:57.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.086+0000 7f5c9b7fe640 1 -- 172.21.15.98:0/113804962 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c8c0074e0 con 0x7f5ca415cbb0 2024-09-19T08:46:57.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.086+0000 7f5ca97f6640 1 -- 172.21.15.98:0/113804962 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ca40712d0 con 0x7f5ca415cbb0 2024-09-19T08:46:57.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.086+0000 7f5c9b7fe640 1 -- 172.21.15.98:0/113804962 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c004a10 con 0x7f5ca415cbb0 2024-09-19T08:46:57.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.086+0000 7f5ca97f6640 1 -- 172.21.15.98:0/113804962 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ca4071b30 con 0x7f5ca415cbb0 2024-09-19T08:46:57.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.090+0000 7f5ca97f6640 1 -- 172.21.15.98:0/113804962 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ca406bc90 con 0x7f5ca415cbb0 2024-09-19T08:46:57.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.090+0000 7f5c9b7fe640 1 -- 172.21.15.98:0/113804962 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f5c8c01b070 con 0x7f5ca415cbb0 2024-09-19T08:46:57.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.090+0000 7f5c9b7fe640 1 --2- 172.21.15.98:0/113804962 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5c840680a0 0x7f5c8406a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:57.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.090+0000 7f5c9b7fe640 1 -- 172.21.15.98:0/113804962 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5737+0+0 (secure 0 0 0) 0x7f5c8c0863d0 con 0x7f5ca415cbb0 2024-09-19T08:46:57.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.090+0000 7f5ca2ffd640 1 --2- 172.21.15.98:0/113804962 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5c840680a0 0x7f5c8406a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:57.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.090+0000 7f5ca2ffd640 1 --2- 172.21.15.98:0/113804962 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5c840680a0 0x7f5c8406a560 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5c94000f70 tx=0x7f5c94009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:57.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.094+0000 7f5c9b7fe640 1 -- 172.21.15.98:0/113804962 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f5c8c0560a0 con 0x7f5ca415cbb0 2024-09-19T08:46:57.242 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.242+0000 7f5ca97f6640 1 -- 172.21.15.98:0/113804962 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f5ca4165760 con 0x7f5ca415cbb0 2024-09-19T08:46:57.243 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.242+0000 7f5c9b7fe640 1 -- 172.21.15.98:0/113804962 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7f5c8c0559a0 con 0x7f5ca415cbb0 2024-09-19T08:46:57.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.246+0000 7f5ca97f6640 1 -- 172.21.15.98:0/113804962 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5c840680a0 msgr2=0x7f5c8406a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:57.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.246+0000 7f5ca97f6640 1 --2- 172.21.15.98:0/113804962 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5c840680a0 0x7f5c8406a560 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5c94000f70 tx=0x7f5c94009040 comp rx=0 tx=0).stop 2024-09-19T08:46:57.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.246+0000 7f5ca97f6640 1 -- 172.21.15.98:0/113804962 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ca415cbb0 msgr2=0x7f5ca415f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:57.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.246+0000 7f5ca97f6640 1 --2- 172.21.15.98:0/113804962 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ca415cbb0 0x7f5ca415f270 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00c440 tx=0x7f5c8c00c900 comp rx=0 tx=0).stop 2024-09-19T08:46:57.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.246+0000 7f5ca97f6640 1 -- 172.21.15.98:0/113804962 shutdown_connections 2024-09-19T08:46:57.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.246+0000 7f5ca97f6640 1 --2- 172.21.15.98:0/113804962 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5c840680a0 0x7f5c8406a560 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:57.245 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.246+0000 7f5ca97f6640 1 --2- 172.21.15.98:0/113804962 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5ca415cf90 0x7f5ca415f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:57.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.246+0000 7f5ca97f6640 1 --2- 172.21.15.98:0/113804962 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5ca415cbb0 0x7f5ca415f270 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:57.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.246+0000 7f5ca97f6640 1 --2- 172.21.15.98:0/113804962 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ca406c3a0 0x7f5ca415bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:57.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.246+0000 7f5ca97f6640 1 -- 172.21.15.98:0/113804962 >> 172.21.15.98:0/113804962 conn(0x7f5ca40b2010 msgr2=0x7f5ca40ce640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:57.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.246+0000 7f5ca97f6640 1 -- 172.21.15.98:0/113804962 shutdown_connections 2024-09-19T08:46:57.246 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.246+0000 7f5ca97f6640 1 -- 172.21.15.98:0/113804962 wait complete. 2024-09-19T08:46:57.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:57.261 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol28130/ffee075f-3de5-4e5d-82f4-b9cb6aa29483) 2024-09-19T08:46:57.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:57.287 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-19T08:46:57.339 INFO:teuthology.orchestra.run.smithi098.stdout:cephfs_data 2024-09-19T08:46:57.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:57.340 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol28130/ffee075f-3de5-4e5d-82f4-b9cb6aa29483) 2024-09-19T08:46:57.405 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:46:57.405 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol28130/.meta 2024-09-19T08:46:57.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:57.468 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/subvol_grp95760/subvol51/c05c69ae-8679-4707-8343-aef36c6a4070) 2024-09-19T08:46:57.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:57.534 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-19T08:46:57.583 INFO:teuthology.orchestra.run.smithi098.stdout:cephfs_data 2024-09-19T08:46:57.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:57.583 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/subvol_grp95760/subvol51/c05c69ae-8679-4707-8343-aef36c6a4070) 2024-09-19T08:46:57.652 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:46:57.652 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/subvol_grp95760/subvol51/.meta 2024-09-19T08:46:57.711 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol28130 2024-09-19T08:46:57.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.886+0000 7fcbe67fb640 1 Processor -- start 2024-09-19T08:46:57.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.886+0000 7fcbe67fb640 1 -- start start 2024-09-19T08:46:57.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.886+0000 7fcbe67fb640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcbe015c440 0x7fcbe00706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:57.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.886+0000 7fcbe67fb640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcbe0072f90 0x7fcbe00ce400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:57.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.886+0000 7fcbe67fb640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbe0070c20 con 0x7fcbe00ce940 2024-09-19T08:46:57.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.886+0000 7fcbe67fb640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fcbe015c820 con 0x7fcbe015c440 2024-09-19T08:46:57.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.886+0000 7fcbe67fb640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fcbe0070ec0 con 0x7fcbe0072f90 2024-09-19T08:46:57.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.886+0000 7fcbdffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcbe015c440 0x7fcbe00706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:57.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.886+0000 7fcbdffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcbe015c440 0x7fcbe00706e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46226/0 (socket says 172.21.15.98:46226) 2024-09-19T08:46:57.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.886+0000 7fcbdffff640 1 -- 172.21.15.98:0/461813630 learned_addr learned my addr 172.21.15.98:0/461813630 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:57.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.886+0000 7fcbdffff640 1 -- 172.21.15.98:0/461813630 >> v2:172.21.15.139:3300/0 conn(0x7fcbe0072f90 msgr2=0x7fcbe00ce400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:57.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.886+0000 7fcbdffff640 1 --2- 172.21.15.98:0/461813630 >> v2:172.21.15.139:3300/0 conn(0x7fcbe0072f90 0x7fcbe00ce400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:57.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.886+0000 7fcbdffff640 1 -- 172.21.15.98:0/461813630 >> v1:172.21.15.139:6789/0 conn(0x7fcbe00ce940 legacy=0x7fcbe00b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:57.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.886+0000 7fcbdffff640 1 -- 172.21.15.98:0/461813630 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe00711a0 con 0x7fcbe015c440 2024-09-19T08:46:57.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.890+0000 7fcbdffff640 1 --2- 172.21.15.98:0/461813630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbe015c440 0x7fcbe00706e0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0069830 tx=0x7fcbd001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d19e0e5279d632 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:57.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.890+0000 7fcbdeffd640 1 -- 172.21.15.98:0/461813630 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd00aa610 con 0x7fcbe015c440 2024-09-19T08:46:57.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.890+0000 7fcbdeffd640 1 -- 172.21.15.98:0/461813630 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbd00af070 con 0x7fcbe015c440 2024-09-19T08:46:57.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.890+0000 7fcbdeffd640 1 -- 172.21.15.98:0/461813630 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd00aa9d0 con 0x7fcbe015c440 2024-09-19T08:46:57.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.890+0000 7fcbe67fb640 1 -- 172.21.15.98:0/461813630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbe015c440 msgr2=0x7fcbe00706e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:57.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.890+0000 7fcbe67fb640 1 --2- 172.21.15.98:0/461813630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbe015c440 0x7fcbe00706e0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0069830 tx=0x7fcbd001c4b0 comp rx=0 tx=0).stop 2024-09-19T08:46:57.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.890+0000 7fcbe67fb640 1 -- 172.21.15.98:0/461813630 shutdown_connections 2024-09-19T08:46:57.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.890+0000 7fcbe67fb640 1 --2- 172.21.15.98:0/461813630 >> v2:172.21.15.139:3300/0 conn(0x7fcbe0072f90 0x7fcbe00ce400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:57.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.890+0000 7fcbe67fb640 1 --2- 172.21.15.98:0/461813630 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbe015c440 0x7fcbe00706e0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:57.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.890+0000 7fcbe67fb640 1 -- 172.21.15.98:0/461813630 >> 172.21.15.98:0/461813630 conn(0x7fcbe00d23d0 msgr2=0x7fcbe00d27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:57.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.890+0000 7fcbe67fb640 1 -- 172.21.15.98:0/461813630 shutdown_connections 2024-09-19T08:46:57.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.890+0000 7fcbe67fb640 1 -- 172.21.15.98:0/461813630 wait complete. 2024-09-19T08:46:57.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.890+0000 7fcbe67fb640 1 Processor -- start 2024-09-19T08:46:57.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.890+0000 7fcbe67fb640 1 -- start start 2024-09-19T08:46:57.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.890+0000 7fcbe67fb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbe0072f90 0x7fcbe00b71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:57.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.890+0000 7fcbe67fb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbe00ce940 0x7fcbe0158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:57.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.890+0000 7fcbe67fb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcbe015c440 0x7fcbe0152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:57.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.890+0000 7fcbe67fb640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe00b5ab0 con 0x7fcbe00ce940 2024-09-19T08:46:57.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.890+0000 7fcbe67fb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe0168fa0 con 0x7fcbe0072f90 2024-09-19T08:46:57.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.890+0000 7fcbe67fb640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcbe00c7880 con 0x7fcbe015c440 2024-09-19T08:46:57.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.890+0000 7fcbdf7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbe00ce940 0x7fcbe0158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:57.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.890+0000 7fcbdf7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbe00ce940 0x7fcbe0158880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46238/0 (socket says 172.21.15.98:46238) 2024-09-19T08:46:57.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.890+0000 7fcbdf7fe640 1 -- 172.21.15.98:0/1993748704 learned_addr learned my addr 172.21.15.98:0/1993748704 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:57.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.890+0000 7fcbdffff640 1 --2- 172.21.15.98:0/1993748704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbe0072f90 0x7fcbe00b71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:57.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.890+0000 7fcbdf7fe640 1 -- 172.21.15.98:0/1993748704 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcbe015c440 msgr2=0x7fcbe0152360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:57.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.890+0000 7fcbdf7fe640 1 --2- 172.21.15.98:0/1993748704 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcbe015c440 0x7fcbe0152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:57.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.890+0000 7fcbdf7fe640 1 -- 172.21.15.98:0/1993748704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbe0072f90 msgr2=0x7fcbe00b71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:57.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.890+0000 7fcbdf7fe640 1 --2- 172.21.15.98:0/1993748704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbe0072f90 0x7fcbe00b71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:57.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.890+0000 7fcbdf7fe640 1 -- 172.21.15.98:0/1993748704 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe00c7b60 con 0x7fcbe00ce940 2024-09-19T08:46:57.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.890+0000 7fcbdffff640 1 --2- 172.21.15.98:0/1993748704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbe0072f90 0x7fcbe00b71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:57.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.890+0000 7fcbdf7fe640 1 --2- 172.21.15.98:0/1993748704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbe00ce940 0x7fcbe0158880 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fcbd400b410 tx=0x7fcbd400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:57.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.890+0000 7fcbdcff9640 1 -- 172.21.15.98:0/1993748704 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4015020 con 0x7fcbe00ce940 2024-09-19T08:46:57.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.890+0000 7fcbdcff9640 1 -- 172.21.15.98:0/1993748704 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbd4011830 con 0x7fcbe00ce940 2024-09-19T08:46:57.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.890+0000 7fcbe67fb640 1 -- 172.21.15.98:0/1993748704 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbe00c7e40 con 0x7fcbe00ce940 2024-09-19T08:46:57.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.894+0000 7fcbdcff9640 1 -- 172.21.15.98:0/1993748704 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4011c20 con 0x7fcbe00ce940 2024-09-19T08:46:57.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.894+0000 7fcbe67fb640 1 -- 172.21.15.98:0/1993748704 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbe00c8400 con 0x7fcbe00ce940 2024-09-19T08:46:57.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.894+0000 7fcbe67fb640 1 -- 172.21.15.98:0/1993748704 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbe00ce400 con 0x7fcbe00ce940 2024-09-19T08:46:57.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.894+0000 7fcbdcff9640 1 -- 172.21.15.98:0/1993748704 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7fcbd4018020 con 0x7fcbe00ce940 2024-09-19T08:46:57.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.894+0000 7fcbdcff9640 1 --2- 172.21.15.98:0/1993748704 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcbcc0680a0 0x7fcbcc06a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:57.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.894+0000 7fcbdcff9640 1 -- 172.21.15.98:0/1993748704 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5737+0+0 (secure 0 0 0) 0x7fcbd4084600 con 0x7fcbe00ce940 2024-09-19T08:46:57.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.894+0000 7fcbdffff640 1 --2- 172.21.15.98:0/1993748704 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcbcc0680a0 0x7fcbcc06a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:57.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.894+0000 7fcbdffff640 1 --2- 172.21.15.98:0/1993748704 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcbcc0680a0 0x7fcbcc06a560 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcbd00ae000 tx=0x7fcbd0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:57.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:57.898+0000 7fcbdcff9640 1 -- 172.21.15.98:0/1993748704 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fcbd408b070 con 0x7fcbe00ce940 2024-09-19T08:46:58.086 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.086+0000 7fcbe67fb640 1 -- 172.21.15.98:0/1993748704 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol28130", "target": ["mon-mgr", ""]}) -- 0x7fcbe00b7820 con 0x7fcbcc0680a0 2024-09-19T08:46:58.095 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:46:58.098+0000 7ff73a69c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:46:58.095 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:46:58.098+0000 7ff73a69c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:46:58.095 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:46:58.098+0000 7ff73a69c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:46:58.095 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:46:58.098+0000 7ff73a69c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:46:58.095 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:46:58.098+0000 7ff73a69c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:46:58.172 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.174+0000 7fcbdcff9640 1 -- 172.21.15.98:0/1993748704 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7fcbe00b7820 con 0x7fcbcc0680a0 2024-09-19T08:46:58.173 INFO:teuthology.orchestra.run.smithi098.stdout:/volumes/_nogroup/subvol28130/ffee075f-3de5-4e5d-82f4-b9cb6aa29483 2024-09-19T08:46:58.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.174+0000 7fcbe67fb640 1 -- 172.21.15.98:0/1993748704 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcbcc0680a0 msgr2=0x7fcbcc06a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:58.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.174+0000 7fcbe67fb640 1 --2- 172.21.15.98:0/1993748704 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcbcc0680a0 0x7fcbcc06a560 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcbd00ae000 tx=0x7fcbd0008040 comp rx=0 tx=0).stop 2024-09-19T08:46:58.175 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.174+0000 7fcbe67fb640 1 -- 172.21.15.98:0/1993748704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbe00ce940 msgr2=0x7fcbe0158880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:58.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.174+0000 7fcbe67fb640 1 --2- 172.21.15.98:0/1993748704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbe00ce940 0x7fcbe0158880 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fcbd400b410 tx=0x7fcbd400b8d0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.174+0000 7fcbe67fb640 1 -- 172.21.15.98:0/1993748704 shutdown_connections 2024-09-19T08:46:58.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.174+0000 7fcbe67fb640 1 --2- 172.21.15.98:0/1993748704 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcbcc0680a0 0x7fcbcc06a560 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.174+0000 7fcbe67fb640 1 --2- 172.21.15.98:0/1993748704 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcbe015c440 0x7fcbe0152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.174+0000 7fcbe67fb640 1 --2- 172.21.15.98:0/1993748704 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcbe00ce940 0x7fcbe0158880 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.174+0000 7fcbe67fb640 1 --2- 172.21.15.98:0/1993748704 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcbe0072f90 0x7fcbe00b71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.174+0000 7fcbe67fb640 1 -- 172.21.15.98:0/1993748704 >> 172.21.15.98:0/1993748704 conn(0x7fcbe00d23d0 msgr2=0x7fcbe015a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:58.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.174+0000 7fcbe67fb640 1 -- 172.21.15.98:0/1993748704 shutdown_connections 2024-09-19T08:46:58.176 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.174+0000 7fcbe67fb640 1 -- 172.21.15.98:0/1993748704 wait complete. 2024-09-19T08:46:58.193 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol51 subvol_grp95760 2024-09-19T08:46:58.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.318+0000 7f992fab5640 1 Processor -- start 2024-09-19T08:46:58.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.318+0000 7f992fab5640 1 -- start start 2024-09-19T08:46:58.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.318+0000 7f992fab5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f992815f6e0 0x7f992806d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:58.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.318+0000 7f992fab5640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f992815fac0 0x7f992806da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:58.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.318+0000 7f992fab5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9928073680 0x7f9928073a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:58.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.318+0000 7f992fab5640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f992815c8b0 con 0x7f992815fac0 2024-09-19T08:46:58.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.318+0000 7f992fab5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f99280742d0 con 0x7f992815f6e0 2024-09-19T08:46:58.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.318+0000 7f992fab5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f992815b890 con 0x7f9928073680 2024-09-19T08:46:58.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.318+0000 7f992d050640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f992815fac0 0x7f992806da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:58.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.318+0000 7f992d050640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f992815fac0 0x7f992806da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46244/0 (socket says 172.21.15.98:46244) 2024-09-19T08:46:58.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.318+0000 7f992d050640 1 -- 172.21.15.98:0/1211710598 learned_addr learned my addr 172.21.15.98:0/1211710598 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:58.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.318+0000 7f992e052640 1 --2- 172.21.15.98:0/1211710598 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9928073680 0x7f9928073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:58.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.318+0000 7f992d851640 1 --2- 172.21.15.98:0/1211710598 >> v2:172.21.15.139:3300/0 conn(0x7f992815f6e0 0x7f992806d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:58.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.318+0000 7f992d050640 1 -- 172.21.15.98:0/1211710598 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9928073680 msgr2=0x7f9928073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:58.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.318+0000 7f992d050640 1 --2- 172.21.15.98:0/1211710598 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9928073680 0x7f9928073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.318+0000 7f992d050640 1 -- 172.21.15.98:0/1211710598 >> v2:172.21.15.139:3300/0 conn(0x7f992815f6e0 msgr2=0x7f992806d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:58.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.318+0000 7f992d050640 1 --2- 172.21.15.98:0/1211710598 >> v2:172.21.15.139:3300/0 conn(0x7f992815f6e0 0x7f992806d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.318+0000 7f992d050640 1 -- 172.21.15.98:0/1211710598 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f992815baf0 con 0x7f992815fac0 2024-09-19T08:46:58.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.318+0000 7f992d851640 1 --2- 172.21.15.98:0/1211710598 >> v2:172.21.15.139:3300/0 conn(0x7f992815f6e0 0x7f992806d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:58.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.322+0000 7f992d050640 1 --2- 172.21.15.98:0/1211710598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f992815fac0 0x7f992806da00 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f992806e080 tx=0x7f991801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b890e9b7817f400d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:58.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.322+0000 7f992c84f640 1 -- 172.21.15.98:0/1211710598 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99180aa610 con 0x7f992815fac0 2024-09-19T08:46:58.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.322+0000 7f992c84f640 1 -- 172.21.15.98:0/1211710598 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99180af070 con 0x7f992815fac0 2024-09-19T08:46:58.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.322+0000 7f992c84f640 1 -- 172.21.15.98:0/1211710598 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99180aa9d0 con 0x7f992815fac0 2024-09-19T08:46:58.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.322+0000 7f992fab5640 1 -- 172.21.15.98:0/1211710598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f992815fac0 msgr2=0x7f992806da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:58.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.322+0000 7f992fab5640 1 --2- 172.21.15.98:0/1211710598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f992815fac0 0x7f992806da00 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f992806e080 tx=0x7f991801c4f0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.322+0000 7f992fab5640 1 -- 172.21.15.98:0/1211710598 shutdown_connections 2024-09-19T08:46:58.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.322+0000 7f992fab5640 1 --2- 172.21.15.98:0/1211710598 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f992815fac0 0x7f992806da00 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.322+0000 7f992fab5640 1 --2- 172.21.15.98:0/1211710598 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9928073680 0x7f9928073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.322+0000 7f992fab5640 1 --2- 172.21.15.98:0/1211710598 >> v2:172.21.15.139:3300/0 conn(0x7f992815f6e0 0x7f992806d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.322+0000 7f992fab5640 1 -- 172.21.15.98:0/1211710598 >> 172.21.15.98:0/1211710598 conn(0x7f992806c090 msgr2=0x7f992806c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:58.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.322+0000 7f992fab5640 1 -- 172.21.15.98:0/1211710598 shutdown_connections 2024-09-19T08:46:58.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.322+0000 7f992fab5640 1 -- 172.21.15.98:0/1211710598 wait complete. 2024-09-19T08:46:58.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.322+0000 7f992fab5640 1 Processor -- start 2024-09-19T08:46:58.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.322+0000 7f992fab5640 1 -- start start 2024-09-19T08:46:58.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.326+0000 7f992fab5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9928073680 0x7f992814f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:58.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.326+0000 7f992fab5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f992815f6e0 0x7f992814c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:58.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.326+0000 7f992fab5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f992815fac0 0x7f9928071f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:58.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.326+0000 7f992fab5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f99280b5040 con 0x7f992815fac0 2024-09-19T08:46:58.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.326+0000 7f992fab5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9928073fd0 con 0x7f992815f6e0 2024-09-19T08:46:58.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.326+0000 7f992fab5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f99280b51c0 con 0x7f9928073680 2024-09-19T08:46:58.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.326+0000 7f992e052640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f992815fac0 0x7f9928071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:58.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.326+0000 7f992e052640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f992815fac0 0x7f9928071f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46254/0 (socket says 172.21.15.98:46254) 2024-09-19T08:46:58.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.326+0000 7f992e052640 1 -- 172.21.15.98:0/4265025353 learned_addr learned my addr 172.21.15.98:0/4265025353 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:58.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.326+0000 7f992d851640 1 --2- 172.21.15.98:0/4265025353 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9928073680 0x7f992814f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:58.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.326+0000 7f992d050640 1 --2- 172.21.15.98:0/4265025353 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f992815f6e0 0x7f992814c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:58.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.326+0000 7f992e052640 1 -- 172.21.15.98:0/4265025353 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9928073680 msgr2=0x7f992814f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:58.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.326+0000 7f992e052640 1 --2- 172.21.15.98:0/4265025353 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9928073680 0x7f992814f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.326+0000 7f992e052640 1 -- 172.21.15.98:0/4265025353 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f992815f6e0 msgr2=0x7f992814c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:58.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.326+0000 7f992e052640 1 --2- 172.21.15.98:0/4265025353 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f992815f6e0 0x7f992814c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.326 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.326+0000 7f992e052640 1 -- 172.21.15.98:0/4265025353 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f992814f760 con 0x7f992815fac0 2024-09-19T08:46:58.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.326+0000 7f992e052640 1 --2- 172.21.15.98:0/4265025353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f992815fac0 0x7f9928071f60 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f9920007d50 tx=0x7f992000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:58.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.326+0000 7f991e7fc640 1 -- 172.21.15.98:0/4265025353 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9920014420 con 0x7f992815fac0 2024-09-19T08:46:58.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.326+0000 7f991e7fc640 1 -- 172.21.15.98:0/4265025353 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9920019070 con 0x7f992815fac0 2024-09-19T08:46:58.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.326+0000 7f992fab5640 1 -- 172.21.15.98:0/4265025353 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f992814fa40 con 0x7f992815fac0 2024-09-19T08:46:58.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.326+0000 7f991e7fc640 1 -- 172.21.15.98:0/4265025353 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9920014820 con 0x7f992815fac0 2024-09-19T08:46:58.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.326+0000 7f992fab5640 1 -- 172.21.15.98:0/4265025353 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9928150000 con 0x7f992815fac0 2024-09-19T08:46:58.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.326+0000 7f992fab5640 1 -- 172.21.15.98:0/4265025353 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99280b55a0 con 0x7f992815fac0 2024-09-19T08:46:58.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.326+0000 7f991e7fc640 1 -- 172.21.15.98:0/4265025353 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f99200149c0 con 0x7f992815fac0 2024-09-19T08:46:58.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.326+0000 7f991e7fc640 1 --2- 172.21.15.98:0/4265025353 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f990c0680a0 0x7f990c06a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:58.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.326+0000 7f991e7fc640 1 -- 172.21.15.98:0/4265025353 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5737+0+0 (secure 0 0 0) 0x7f9920087660 con 0x7f992815fac0 2024-09-19T08:46:58.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.326+0000 7f992d851640 1 --2- 172.21.15.98:0/4265025353 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f990c0680a0 0x7f990c06a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:58.328 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.330+0000 7f992d851640 1 --2- 172.21.15.98:0/4265025353 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f990c0680a0 0x7f990c06a560 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f992400c900 tx=0x7f9924008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:58.331 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.330+0000 7f991e7fc640 1 -- 172.21.15.98:0/4265025353 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f99200572a0 con 0x7f992815fac0 2024-09-19T08:46:58.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.522+0000 7f992fab5640 1 -- 172.21.15.98:0/4265025353 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol51", "group_name": "subvol_grp95760", "target": ["mon-mgr", ""]}) -- 0x7f992815e640 con 0x7f990c0680a0 2024-09-19T08:46:58.626 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.626+0000 7f991e7fc640 1 -- 172.21.15.98:0/4265025353 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+71 (secure 0 0 0) 0x7f992815e640 con 0x7f990c0680a0 2024-09-19T08:46:58.626 INFO:teuthology.orchestra.run.smithi098.stdout:/volumes/subvol_grp95760/subvol51/c05c69ae-8679-4707-8343-aef36c6a4070 2024-09-19T08:46:58.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.630+0000 7f992fab5640 1 -- 172.21.15.98:0/4265025353 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f990c0680a0 msgr2=0x7f990c06a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:58.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.630+0000 7f992fab5640 1 --2- 172.21.15.98:0/4265025353 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f990c0680a0 0x7f990c06a560 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f992400c900 tx=0x7f9924008040 comp rx=0 tx=0).stop 2024-09-19T08:46:58.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.630+0000 7f992fab5640 1 -- 172.21.15.98:0/4265025353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f992815fac0 msgr2=0x7f9928071f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:58.629 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.630+0000 7f992fab5640 1 --2- 172.21.15.98:0/4265025353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f992815fac0 0x7f9928071f60 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f9920007d50 tx=0x7f992000e4b0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.630+0000 7f992fab5640 1 -- 172.21.15.98:0/4265025353 shutdown_connections 2024-09-19T08:46:58.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.630+0000 7f992fab5640 1 --2- 172.21.15.98:0/4265025353 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f990c0680a0 0x7f990c06a560 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.630+0000 7f992fab5640 1 --2- 172.21.15.98:0/4265025353 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f992815fac0 0x7f9928071f60 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.630+0000 7f992fab5640 1 --2- 172.21.15.98:0/4265025353 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f992815f6e0 0x7f992814c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.630+0000 7f992fab5640 1 --2- 172.21.15.98:0/4265025353 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9928073680 0x7f992814f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.630+0000 7f992fab5640 1 -- 172.21.15.98:0/4265025353 >> 172.21.15.98:0/4265025353 conn(0x7f992806c090 msgr2=0x7f99280cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:58.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.630+0000 7f992fab5640 1 -- 172.21.15.98:0/4265025353 shutdown_connections 2024-09-19T08:46:58.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.630+0000 7f992fab5640 1 -- 172.21.15.98:0/4265025353 wait complete. 2024-09-19T08:46:58.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:58.650 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol28130/.meta) 2024-09-19T08:46:58.668 INFO:teuthology.orchestra.run.smithi098.stdout:[GLOBAL] 2024-09-19T08:46:58.668 INFO:teuthology.orchestra.run.smithi098.stdout:version = 2 2024-09-19T08:46:58.668 INFO:teuthology.orchestra.run.smithi098.stdout:type = subvolume 2024-09-19T08:46:58.668 INFO:teuthology.orchestra.run.smithi098.stdout:path = /volumes/_nogroup/subvol28130/ffee075f-3de5-4e5d-82f4-b9cb6aa29483 2024-09-19T08:46:58.668 INFO:teuthology.orchestra.run.smithi098.stdout:state = complete 2024-09-19T08:46:58.668 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:46:58.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:46:58.670 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/subvol_grp95760/subvol51/.meta) 2024-09-19T08:46:58.727 INFO:teuthology.orchestra.run.smithi098.stdout:[GLOBAL] 2024-09-19T08:46:58.727 INFO:teuthology.orchestra.run.smithi098.stdout:version = 2 2024-09-19T08:46:58.727 INFO:teuthology.orchestra.run.smithi098.stdout:type = subvolume 2024-09-19T08:46:58.727 INFO:teuthology.orchestra.run.smithi098.stdout:path = /volumes/subvol_grp95760/subvol51/c05c69ae-8679-4707-8343-aef36c6a4070 2024-09-19T08:46:58.727 INFO:teuthology.orchestra.run.smithi098.stdout:state = complete 2024-09-19T08:46:58.727 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:46:58.728 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol28130 2024-09-19T08:46:58.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.902+0000 7f3368832640 1 Processor -- start 2024-09-19T08:46:58.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.902+0000 7f3368832640 1 -- start start 2024-09-19T08:46:58.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.902+0000 7f3368832640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f336406c3a0 0x7f33640713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:58.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.902+0000 7f3368832640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f33640718f0 0x7f33640b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:58.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.902+0000 7f3368832640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3364073530 con 0x7f3364145b70 2024-09-19T08:46:58.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.902+0000 7f3368832640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f3364073810 con 0x7f33640718f0 2024-09-19T08:46:58.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.902+0000 7f3368832640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f3364073af0 con 0x7f336406c3a0 2024-09-19T08:46:58.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.902+0000 7f336259c640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f3364145b70 0x7f336406bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:60972/0 (socket says 172.21.15.98:60972) 2024-09-19T08:46:58.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.902+0000 7f336259c640 1 -- 172.21.15.98:0/2953540212 learned_addr learned my addr 172.21.15.98:0/2953540212 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:58.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.902+0000 7f3362d9d640 1 --2- 172.21.15.98:0/2953540212 >> v2:172.21.15.98:3300/0 conn(0x7f33640718f0 0x7f33640b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:58.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.902+0000 7f3359d9b640 1 --2- 172.21.15.98:0/2953540212 >> v2:172.21.15.139:3300/0 conn(0x7f336406c3a0 0x7f33640713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:58.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.902+0000 7f3361d9b640 1 -- 172.21.15.98:0/2953540212 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3776898072 0 0) 0x7f3364073530 con 0x7f3364145b70 2024-09-19T08:46:58.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.902+0000 7f3362d9d640 1 -- 172.21.15.98:0/2953540212 >> v2:172.21.15.139:3300/0 conn(0x7f336406c3a0 msgr2=0x7f33640713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:58.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.902+0000 7f3362d9d640 1 --2- 172.21.15.98:0/2953540212 >> v2:172.21.15.139:3300/0 conn(0x7f336406c3a0 0x7f33640713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.902+0000 7f3362d9d640 1 -- 172.21.15.98:0/2953540212 >> v1:172.21.15.98:6789/0 conn(0x7f3364145b70 legacy=0x7f336406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:58.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.902+0000 7f3362d9d640 1 -- 172.21.15.98:0/2953540212 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3364073dd0 con 0x7f33640718f0 2024-09-19T08:46:58.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.906+0000 7f3362d9d640 1 --2- 172.21.15.98:0/2953540212 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33640718f0 0x7f33640b3410 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f3364074420 tx=0x7f335c01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a9ee4c841360ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:58.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.906+0000 7f3361d9b640 1 -- 172.21.15.98:0/2953540212 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c0b2070 con 0x7f33640718f0 2024-09-19T08:46:58.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.906+0000 7f3361d9b640 1 -- 172.21.15.98:0/2953540212 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f335c0a5760 con 0x7f33640718f0 2024-09-19T08:46:58.904 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.906+0000 7f3361d9b640 1 -- 172.21.15.98:0/2953540212 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c0a5b30 con 0x7f33640718f0 2024-09-19T08:46:58.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.906+0000 7f3368832640 1 -- 172.21.15.98:0/2953540212 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33640718f0 msgr2=0x7f33640b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:58.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.906+0000 7f3368832640 1 --2- 172.21.15.98:0/2953540212 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33640718f0 0x7f33640b3410 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f3364074420 tx=0x7f335c01c8a0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.906+0000 7f3368832640 1 -- 172.21.15.98:0/2953540212 shutdown_connections 2024-09-19T08:46:58.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.906+0000 7f3368832640 1 --2- 172.21.15.98:0/2953540212 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f33640718f0 0x7f33640b3410 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.906+0000 7f3368832640 1 --2- 172.21.15.98:0/2953540212 >> v2:172.21.15.139:3300/0 conn(0x7f336406c3a0 0x7f33640713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.906+0000 7f3368832640 1 -- 172.21.15.98:0/2953540212 >> 172.21.15.98:0/2953540212 conn(0x7f33640b2010 msgr2=0x7f33640b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:58.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.906+0000 7f3368832640 1 -- 172.21.15.98:0/2953540212 shutdown_connections 2024-09-19T08:46:58.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.906+0000 7f3368832640 1 -- 172.21.15.98:0/2953540212 wait complete. 2024-09-19T08:46:58.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.906+0000 7f3368832640 1 Processor -- start 2024-09-19T08:46:58.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.906+0000 7f3368832640 1 -- start start 2024-09-19T08:46:58.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.906+0000 7f3368832640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f336406c3a0 0x7f3364162980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:58.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.906+0000 7f3368832640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33640718f0 0x7f336415b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:58.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.906+0000 7f3368832640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3364145b70 0x7f336413e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:58.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.906+0000 7f3368832640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f33640b44f0 con 0x7f336406c3a0 2024-09-19T08:46:58.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.906+0000 7f3368832640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f33640b4370 con 0x7f3364145b70 2024-09-19T08:46:58.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.906+0000 7f3368832640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f33640b4670 con 0x7f33640718f0 2024-09-19T08:46:58.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.906+0000 7f336259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f336406c3a0 0x7f3364162980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:58.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.906+0000 7f336259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f336406c3a0 0x7f3364162980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46268/0 (socket says 172.21.15.98:46268) 2024-09-19T08:46:58.909 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.906+0000 7f336259c640 1 -- 172.21.15.98:0/365430667 learned_addr learned my addr 172.21.15.98:0/365430667 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:58.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.906+0000 7f3362d9d640 1 --2- 172.21.15.98:0/365430667 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3364145b70 0x7f336413e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:58.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.906+0000 7f336259c640 1 -- 172.21.15.98:0/365430667 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33640718f0 msgr2=0x7f336415b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:58.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.906+0000 7f336259c640 1 --2- 172.21.15.98:0/365430667 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33640718f0 0x7f336415b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.906+0000 7f336259c640 1 -- 172.21.15.98:0/365430667 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3364145b70 msgr2=0x7f336413e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:58.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.906+0000 7f336259c640 1 --2- 172.21.15.98:0/365430667 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3364145b70 0x7f336413e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:58.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.906+0000 7f336259c640 1 -- 172.21.15.98:0/365430667 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3364161bc0 con 0x7f336406c3a0 2024-09-19T08:46:58.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.906+0000 7f336259c640 1 --2- 172.21.15.98:0/365430667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f336406c3a0 0x7f3364162980 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f334c00c480 tx=0x7f334c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:58.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.906+0000 7f335b7fe640 1 -- 172.21.15.98:0/365430667 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334c018070 con 0x7f336406c3a0 2024-09-19T08:46:58.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.906+0000 7f335b7fe640 1 -- 172.21.15.98:0/365430667 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f334c007500 con 0x7f336406c3a0 2024-09-19T08:46:58.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.906+0000 7f3368832640 1 -- 172.21.15.98:0/365430667 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3364161ea0 con 0x7f336406c3a0 2024-09-19T08:46:58.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.906+0000 7f335b7fe640 1 -- 172.21.15.98:0/365430667 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334c004a30 con 0x7f336406c3a0 2024-09-19T08:46:58.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.910+0000 7f3368832640 1 -- 172.21.15.98:0/365430667 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3364162460 con 0x7f336406c3a0 2024-09-19T08:46:58.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.910+0000 7f3368832640 1 -- 172.21.15.98:0/365430667 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3364166730 con 0x7f336406c3a0 2024-09-19T08:46:58.910 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.910+0000 7f335b7fe640 1 -- 172.21.15.98:0/365430667 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7f334c01b070 con 0x7f336406c3a0 2024-09-19T08:46:58.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.910+0000 7f335b7fe640 1 --2- 172.21.15.98:0/365430667 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f33480680a0 0x7f334806a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:58.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.910+0000 7f335b7fe640 1 -- 172.21.15.98:0/365430667 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5737+0+0 (secure 0 0 0) 0x7f334c0839d0 con 0x7f336406c3a0 2024-09-19T08:46:58.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.910+0000 7f3359d9b640 1 --2- 172.21.15.98:0/365430667 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f33480680a0 0x7f334806a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:58.911 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.910+0000 7f3359d9b640 1 --2- 172.21.15.98:0/365430667 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f33480680a0 0x7f334806a560 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f33640cf2e0 tx=0x7f3354008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:58.913 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:58.914+0000 7f335b7fe640 1 -- 172.21.15.98:0/365430667 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f334c055e00 con 0x7f336406c3a0 2024-09-19T08:46:59.092 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.094+0000 7f3368832640 1 -- 172.21.15.98:0/365430667 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol28130", "target": ["mon-mgr", ""]}) -- 0x7f33640757a0 con 0x7f33480680a0 2024-09-19T08:46:59.128 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.130+0000 7f335b7fe640 1 -- 172.21.15.98:0/365430667 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f33640757a0 con 0x7f33480680a0 2024-09-19T08:46:59.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.130+0000 7f3368832640 1 -- 172.21.15.98:0/365430667 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f33480680a0 msgr2=0x7f334806a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:59.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.130+0000 7f3368832640 1 --2- 172.21.15.98:0/365430667 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f33480680a0 0x7f334806a560 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f33640cf2e0 tx=0x7f3354008040 comp rx=0 tx=0).stop 2024-09-19T08:46:59.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.130+0000 7f3368832640 1 -- 172.21.15.98:0/365430667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f336406c3a0 msgr2=0x7f3364162980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:59.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.130+0000 7f3368832640 1 --2- 172.21.15.98:0/365430667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f336406c3a0 0x7f3364162980 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f334c00c480 tx=0x7f334c00c940 comp rx=0 tx=0).stop 2024-09-19T08:46:59.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.130+0000 7f3368832640 1 -- 172.21.15.98:0/365430667 shutdown_connections 2024-09-19T08:46:59.131 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.130+0000 7f3368832640 1 --2- 172.21.15.98:0/365430667 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f33480680a0 0x7f334806a560 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:59.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.130+0000 7f3368832640 1 --2- 172.21.15.98:0/365430667 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3364145b70 0x7f336413e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:59.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.130+0000 7f3368832640 1 --2- 172.21.15.98:0/365430667 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f33640718f0 0x7f336415b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:59.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.130+0000 7f3368832640 1 --2- 172.21.15.98:0/365430667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f336406c3a0 0x7f3364162980 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:59.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.130+0000 7f3368832640 1 -- 172.21.15.98:0/365430667 >> 172.21.15.98:0/365430667 conn(0x7f33640b2010 msgr2=0x7f33640b1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:59.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.130+0000 7f3368832640 1 -- 172.21.15.98:0/365430667 shutdown_connections 2024-09-19T08:46:59.132 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.130+0000 7f3368832640 1 -- 172.21.15.98:0/365430667 wait complete. 2024-09-19T08:46:59.139 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:46:59.142+0000 7ff73d6a2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:46:59.139 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:46:59.142+0000 7ff73d6a2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:46:59.139 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:46:59.142+0000 7ff73d6a2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:46:59.139 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:46:59.142+0000 7ff73d6a2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:46:59.140 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:46:59.142+0000 7ff73d6a2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:46:59.147 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol51 subvol_grp95760 2024-09-19T08:46:59.157 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:46:59.158+0000 7ff73cea1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:46:59.157 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:46:59.158+0000 7ff73cea1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:46:59.157 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:46:59.158+0000 7ff73cea1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:46:59.157 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:46:59.158+0000 7ff73cea1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:46:59.157 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:46:59.158+0000 7ff73cea1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:46:59.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.270+0000 7fed8fb8a640 1 Processor -- start 2024-09-19T08:46:59.268 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.270+0000 7fed8fb8a640 1 -- start start 2024-09-19T08:46:59.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.270+0000 7fed8fb8a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fed8815f6e0 0x7fed8806d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:59.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.270+0000 7fed8fb8a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed8815fac0 0x7fed8806da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:59.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.270+0000 7fed8fb8a640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed8815baf0 con 0x7fed88073680 2024-09-19T08:46:59.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.270+0000 7fed8fb8a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fed8815c8b0 con 0x7fed8815f6e0 2024-09-19T08:46:59.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.270+0000 7fed8fb8a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fed880742d0 con 0x7fed8815fac0 2024-09-19T08:46:59.272 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.270+0000 7fed8e127640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fed88073680 0x7fed88073ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:60978/0 (socket says 172.21.15.98:60978) 2024-09-19T08:46:59.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.270+0000 7fed8e127640 1 -- 172.21.15.98:0/975392565 learned_addr learned my addr 172.21.15.98:0/975392565 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:46:59.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.270+0000 7fed8c924640 1 -- 172.21.15.98:0/975392565 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3783960474 0 0) 0x7fed8815baf0 con 0x7fed88073680 2024-09-19T08:46:59.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.270+0000 7fed8d926640 1 --2- 172.21.15.98:0/975392565 >> v2:172.21.15.139:3300/0 conn(0x7fed8815f6e0 0x7fed8806d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:59.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.270+0000 7fed8d125640 1 --2- 172.21.15.98:0/975392565 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed8815fac0 0x7fed8806da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:59.273 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.270+0000 7fed8c924640 1 -- 172.21.15.98:0/975392565 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed74003530 con 0x7fed88073680 2024-09-19T08:46:59.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.270+0000 7fed8c924640 1 -- 172.21.15.98:0/975392565 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 147012929 0 0) 0x7fed74003530 con 0x7fed88073680 2024-09-19T08:46:59.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.270+0000 7fed8c924640 1 -- 172.21.15.98:0/975392565 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed8815fac0 msgr2=0x7fed8806da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:59.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.270+0000 7fed8c924640 1 --2- 172.21.15.98:0/975392565 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed8815fac0 0x7fed8806da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:59.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.270+0000 7fed8c924640 1 -- 172.21.15.98:0/975392565 >> v2:172.21.15.139:3300/0 conn(0x7fed8815f6e0 msgr2=0x7fed8806d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:59.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.270+0000 7fed8c924640 1 --2- 172.21.15.98:0/975392565 >> v2:172.21.15.139:3300/0 conn(0x7fed8815f6e0 0x7fed8806d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:59.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.270+0000 7fed8c924640 1 -- 172.21.15.98:0/975392565 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8815bdd0 con 0x7fed88073680 2024-09-19T08:46:59.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.270+0000 7fed8c924640 1 -- 172.21.15.98:0/975392565 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fed80001a00 con 0x7fed88073680 2024-09-19T08:46:59.274 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.270+0000 7fed8c924640 1 -- 172.21.15.98:0/975392565 >> v1:172.21.15.98:6789/0 conn(0x7fed88073680 legacy=0x7fed88073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:59.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.270+0000 7fed8c924640 1 --2- 172.21.15.98:0/975392565 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed8815fac0 0x7fed740039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:59.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.270+0000 7fed8c924640 1 --2- 172.21.15.98:0/975392565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed74003f40 0x7fed74004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:59.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.270+0000 7fed8c924640 1 --2- 172.21.15.98:0/975392565 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed74004d50 0x7fed740051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:59.275 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.270+0000 7fed8c924640 1 -- 172.21.15.98:0/975392565 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fed8815c8b0 con 0x7fed74003f40 2024-09-19T08:46:59.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.270+0000 7fed8c924640 1 -- 172.21.15.98:0/975392565 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fed880742d0 con 0x7fed74004d50 2024-09-19T08:46:59.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.270+0000 7fed8c924640 1 -- 172.21.15.98:0/975392565 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fed7400bad0 con 0x7fed8815fac0 2024-09-19T08:46:59.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.270+0000 7fed8d926640 1 --2- 172.21.15.98:0/975392565 >> v2:172.21.15.139:3300/0 conn(0x7fed8815f6e0 0x7fed8806d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:46:59.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.270+0000 7fed8d125640 1 --2- 172.21.15.98:0/975392565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed74003f40 0x7fed74004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:59.276 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.270+0000 7fed8d125640 1 -- 172.21.15.98:0/975392565 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed8815fac0 msgr2=0x7fed740039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:59.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.270+0000 7fed8d125640 1 --2- 172.21.15.98:0/975392565 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed8815fac0 0x7fed740039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:59.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.270+0000 7fed8d125640 1 -- 172.21.15.98:0/975392565 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed74004d50 msgr2=0x7fed740051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:46:59.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.270+0000 7fed8e127640 1 --2- 172.21.15.98:0/975392565 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed74004d50 0x7fed740051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:59.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.270+0000 7fed8d125640 1 --2- 172.21.15.98:0/975392565 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed74004d50 0x7fed740051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:59.277 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.270+0000 7fed8d125640 1 -- 172.21.15.98:0/975392565 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed74003530 con 0x7fed74003f40 2024-09-19T08:46:59.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.274+0000 7fed8d125640 1 --2- 172.21.15.98:0/975392565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed74003f40 0x7fed74004810 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fed78009670 tx=0x7fed7801ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e41879a7f557deab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:59.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.274+0000 7fed8c924640 1 -- 172.21.15.98:0/975392565 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed780b4070 con 0x7fed74003f40 2024-09-19T08:46:59.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.274+0000 7fed8c924640 1 -- 172.21.15.98:0/975392565 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed780a75a0 con 0x7fed74003f40 2024-09-19T08:46:59.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.274+0000 7fed8c924640 1 -- 172.21.15.98:0/975392565 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed780096b0 con 0x7fed74003f40 2024-09-19T08:46:59.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.274+0000 7fed8fb8a640 1 -- 172.21.15.98:0/975392565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed74003f40 msgr2=0x7fed74004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:59.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.274+0000 7fed8fb8a640 1 --2- 172.21.15.98:0/975392565 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed74003f40 0x7fed74004810 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fed78009670 tx=0x7fed7801ecc0 comp rx=0 tx=0).stop 2024-09-19T08:46:59.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.274+0000 7fed8e127640 1 -- 172.21.15.98:0/975392565 reap_dead start 2024-09-19T08:46:59.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.274+0000 7fed8fb8a640 1 -- 172.21.15.98:0/975392565 shutdown_connections 2024-09-19T08:46:59.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.274+0000 7fed8fb8a640 1 -- 172.21.15.98:0/975392565 >> 172.21.15.98:0/975392565 conn(0x7fed8806c090 msgr2=0x7fed8806c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:46:59.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.274+0000 7fed8fb8a640 1 -- 172.21.15.98:0/975392565 shutdown_connections 2024-09-19T08:46:59.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.274+0000 7fed8fb8a640 1 -- 172.21.15.98:0/975392565 wait complete. 2024-09-19T08:46:59.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.274+0000 7fed8fb8a640 1 Processor -- start 2024-09-19T08:46:59.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.274+0000 7fed8fb8a640 1 -- start start 2024-09-19T08:46:59.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.274+0000 7fed8fb8a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed74003f40 0x7fed881427f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:59.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.274+0000 7fed8fb8a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed880aa4a0 0x7fed880aa920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:59.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.274+0000 7fed8fb8a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed88145410 0x7fed881458a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:59.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.274+0000 7fed8fb8a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fed880b42c0 con 0x7fed74003f40 2024-09-19T08:46:59.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.274+0000 7fed8fb8a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fed880b4140 con 0x7fed880aa4a0 2024-09-19T08:46:59.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.274+0000 7fed8fb8a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fed880b4440 con 0x7fed88145410 2024-09-19T08:46:59.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.274+0000 7fed8d926640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed74003f40 0x7fed881427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:59.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.274+0000 7fed8d926640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed74003f40 0x7fed881427f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46274/0 (socket says 172.21.15.98:46274) 2024-09-19T08:46:59.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.274+0000 7fed8d926640 1 -- 172.21.15.98:0/2256610767 learned_addr learned my addr 172.21.15.98:0/2256610767 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:46:59.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.274+0000 7fed8d125640 1 --2- 172.21.15.98:0/2256610767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed880aa4a0 0x7fed880aa920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:59.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.274+0000 7fed8e127640 1 --2- 172.21.15.98:0/2256610767 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed88145410 0x7fed881458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:59.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.278+0000 7fed8d926640 1 -- 172.21.15.98:0/2256610767 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed88145410 msgr2=0x7fed881458a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:59.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.278+0000 7fed8d926640 1 --2- 172.21.15.98:0/2256610767 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed88145410 0x7fed881458a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:59.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.278+0000 7fed8d926640 1 -- 172.21.15.98:0/2256610767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed880aa4a0 msgr2=0x7fed880aa920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:46:59.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.278+0000 7fed8d926640 1 --2- 172.21.15.98:0/2256610767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed880aa4a0 0x7fed880aa920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:46:59.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.278+0000 7fed8d926640 1 -- 172.21.15.98:0/2256610767 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed88143260 con 0x7fed74003f40 2024-09-19T08:46:59.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.278+0000 7fed8e127640 1 --2- 172.21.15.98:0/2256610767 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed88145410 0x7fed881458a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:46:59.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.278+0000 7fed8d926640 1 --2- 172.21.15.98:0/2256610767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed74003f40 0x7fed881427f0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fed8400e810 tx=0x7fed8400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:59.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.278+0000 7fed7e7fc640 1 -- 172.21.15.98:0/2256610767 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed84017070 con 0x7fed74003f40 2024-09-19T08:46:59.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.278+0000 7fed7e7fc640 1 -- 172.21.15.98:0/2256610767 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed8400c8d0 con 0x7fed74003f40 2024-09-19T08:46:59.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.278+0000 7fed8fb8a640 1 -- 172.21.15.98:0/2256610767 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed88143520 con 0x7fed74003f40 2024-09-19T08:46:59.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.278+0000 7fed7e7fc640 1 -- 172.21.15.98:0/2256610767 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed8400f040 con 0x7fed74003f40 2024-09-19T08:46:59.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.278+0000 7fed8fb8a640 1 -- 172.21.15.98:0/2256610767 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed88142d30 con 0x7fed74003f40 2024-09-19T08:46:59.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.278+0000 7fed8fb8a640 1 -- 172.21.15.98:0/2256610767 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed8806d4c0 con 0x7fed74003f40 2024-09-19T08:46:59.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.278+0000 7fed7e7fc640 1 -- 172.21.15.98:0/2256610767 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 24) ==== 88159+0+0 (secure 0 0 0) 0x7fed8401a070 con 0x7fed74003f40 2024-09-19T08:46:59.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.278+0000 7fed7e7fc640 1 --2- 172.21.15.98:0/2256610767 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fed6c0680a0 0x7fed6c06a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:46:59.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.278+0000 7fed7e7fc640 1 -- 172.21.15.98:0/2256610767 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5737+0+0 (secure 0 0 0) 0x7fed840842f0 con 0x7fed74003f40 2024-09-19T08:46:59.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.278+0000 7fed8d125640 1 --2- 172.21.15.98:0/2256610767 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fed6c0680a0 0x7fed6c06a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:46:59.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.278+0000 7fed8d125640 1 --2- 172.21.15.98:0/2256610767 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fed6c0680a0 0x7fed6c06a560 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fed780b1000 tx=0x7fed78010040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:46:59.288 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.282+0000 7fed7e7fc640 1 -- 172.21.15.98:0/2256610767 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fed84056720 con 0x7fed74003f40 2024-09-19T08:46:59.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.466+0000 7fed8fb8a640 1 -- 172.21.15.98:0/2256610767 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol51", "group_name": "subvol_grp95760", "target": ["mon-mgr", ""]}) -- 0x7fed8815dd30 con 0x7fed6c0680a0 2024-09-19T08:46:59.619 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:46:59.618+0000 7fed7e7fc640 1 -- 172.21.15.98:0/2256610767 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7fed84052690 con 0x7fed74003f40 2024-09-19T08:47:00.557 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.558+0000 7fed7e7fc640 1 -- 172.21.15.98:0/2256610767 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fed8815dd30 con 0x7fed6c0680a0 2024-09-19T08:47:00.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.562+0000 7fed8fb8a640 1 -- 172.21.15.98:0/2256610767 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fed6c0680a0 msgr2=0x7fed6c06a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:00.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.562+0000 7fed8fb8a640 1 --2- 172.21.15.98:0/2256610767 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fed6c0680a0 0x7fed6c06a560 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fed780b1000 tx=0x7fed78010040 comp rx=0 tx=0).stop 2024-09-19T08:47:00.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.562+0000 7fed8fb8a640 1 -- 172.21.15.98:0/2256610767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed74003f40 msgr2=0x7fed881427f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:00.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.562+0000 7fed8fb8a640 1 --2- 172.21.15.98:0/2256610767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed74003f40 0x7fed881427f0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fed8400e810 tx=0x7fed8400ecd0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.562+0000 7fed8fb8a640 1 -- 172.21.15.98:0/2256610767 shutdown_connections 2024-09-19T08:47:00.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.562+0000 7fed8fb8a640 1 --2- 172.21.15.98:0/2256610767 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fed6c0680a0 0x7fed6c06a560 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.562+0000 7fed8fb8a640 1 --2- 172.21.15.98:0/2256610767 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed88145410 0x7fed881458a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.562+0000 7fed8fb8a640 1 --2- 172.21.15.98:0/2256610767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed880aa4a0 0x7fed880aa920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.562+0000 7fed8fb8a640 1 --2- 172.21.15.98:0/2256610767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed74003f40 0x7fed881427f0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.562+0000 7fed8fb8a640 1 -- 172.21.15.98:0/2256610767 >> 172.21.15.98:0/2256610767 conn(0x7fed8806c090 msgr2=0x7fed8806c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:00.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.562+0000 7fed8fb8a640 1 -- 172.21.15.98:0/2256610767 shutdown_connections 2024-09-19T08:47:00.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.562+0000 7fed8fb8a640 1 -- 172.21.15.98:0/2256610767 wait complete. 2024-09-19T08:47:00.581 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-19T08:47:00.630 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T08:47:00.638 INFO:teuthology.orchestra.run.smithi098.stdout:2 2024-09-19T08:47:00.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:47:00.644 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T08:47:00.645 DEBUG:teuthology.orchestra.run.smithi098:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:47:00.729 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:47:00.799 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T08:47:00.800 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:47:00.800 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:47:00.800 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:47:00.807 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T08:47:00.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.982+0000 7f6e8e5c1640 1 Processor -- start 2024-09-19T08:47:00.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.982+0000 7f6e8e5c1640 1 -- start start 2024-09-19T08:47:00.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.982+0000 7f6e8e5c1640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6e880d0ff0 0x7f6e880d1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:00.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.982+0000 7f6e8e5c1640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6e880b6f40 0x7f6e880b73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:00.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.982+0000 7f6e8e5c1640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e880b78e0 con 0x7f6e8815c220 2024-09-19T08:47:00.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.982+0000 7f6e8e5c1640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f6e880d19a0 con 0x7f6e880b6f40 2024-09-19T08:47:00.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.982+0000 7f6e8e5c1640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f6e880740b0 con 0x7f6e880d0ff0 2024-09-19T08:47:00.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.982+0000 7f6e8cb5e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6e880b6f40 0x7f6e880b73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:00.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.982+0000 7f6e8cb5e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6e880b6f40 0x7f6e880b73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46280/0 (socket says 172.21.15.98:46280) 2024-09-19T08:47:00.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.982+0000 7f6e8cb5e640 1 -- 172.21.15.98:0/624814078 learned_addr learned my addr 172.21.15.98:0/624814078 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:00.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.982+0000 7f6e877fe640 1 --2- 172.21.15.98:0/624814078 >> v2:172.21.15.139:3300/0 conn(0x7f6e880d0ff0 0x7f6e880d1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:00.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.982+0000 7f6e8cb5e640 1 -- 172.21.15.98:0/624814078 >> v2:172.21.15.139:3300/0 conn(0x7f6e880d0ff0 msgr2=0x7f6e880d1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:00.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.982+0000 7f6e8cb5e640 1 --2- 172.21.15.98:0/624814078 >> v2:172.21.15.139:3300/0 conn(0x7f6e880d0ff0 0x7f6e880d1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.982+0000 7f6e8cb5e640 1 -- 172.21.15.98:0/624814078 >> v1:172.21.15.139:6789/0 conn(0x7f6e8815c220 legacy=0x7f6e880733f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:00.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.982+0000 7f6e8cb5e640 1 -- 172.21.15.98:0/624814078 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e88073b00 con 0x7f6e880b6f40 2024-09-19T08:47:00.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.982+0000 7f6e877fe640 1 --2- 172.21.15.98:0/624814078 >> v2:172.21.15.139:3300/0 conn(0x7f6e880d0ff0 0x7f6e880d1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:00.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.986+0000 7f6e8cb5e640 1 --2- 172.21.15.98:0/624814078 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e880b6f40 0x7f6e880b73a0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f6e88073090 tx=0x7f6e8001c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdaf760e1e25620c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:00.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.986+0000 7f6e86ffd640 1 -- 172.21.15.98:0/624814078 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e800a5660 con 0x7f6e880b6f40 2024-09-19T08:47:00.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.986+0000 7f6e86ffd640 1 -- 172.21.15.98:0/624814078 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e800a5800 con 0x7f6e880b6f40 2024-09-19T08:47:00.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.986+0000 7f6e86ffd640 1 -- 172.21.15.98:0/624814078 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e800a5bd0 con 0x7f6e880b6f40 2024-09-19T08:47:00.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.986+0000 7f6e8e5c1640 1 -- 172.21.15.98:0/624814078 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e880b6f40 msgr2=0x7f6e880b73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:00.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.986+0000 7f6e8e5c1640 1 --2- 172.21.15.98:0/624814078 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e880b6f40 0x7f6e880b73a0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f6e88073090 tx=0x7f6e8001c7a0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.986+0000 7f6e8e5c1640 1 -- 172.21.15.98:0/624814078 shutdown_connections 2024-09-19T08:47:00.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.986+0000 7f6e8e5c1640 1 --2- 172.21.15.98:0/624814078 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e880b6f40 0x7f6e880b73a0 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.986+0000 7f6e8e5c1640 1 --2- 172.21.15.98:0/624814078 >> v2:172.21.15.139:3300/0 conn(0x7f6e880d0ff0 0x7f6e880d1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.986+0000 7f6e8e5c1640 1 -- 172.21.15.98:0/624814078 >> 172.21.15.98:0/624814078 conn(0x7f6e880b40a0 msgr2=0x7f6e880b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:00.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.986+0000 7f6e8e5c1640 1 -- 172.21.15.98:0/624814078 shutdown_connections 2024-09-19T08:47:00.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.986+0000 7f6e8e5c1640 1 -- 172.21.15.98:0/624814078 wait complete. 2024-09-19T08:47:00.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.990+0000 7f6e8e5c1640 1 Processor -- start 2024-09-19T08:47:00.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.990+0000 7f6e8e5c1640 1 -- start start 2024-09-19T08:47:00.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.990+0000 7f6e8e5c1640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6e880b6f40 0x7f6e8806d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:00.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.990+0000 7f6e8e5c1640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e880d0ff0 0x7f6e8814fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:00.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.990+0000 7f6e8e5c1640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e8815c220 0x7f6e88154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:00.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.990+0000 7f6e8e5c1640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6e880d0c30 con 0x7f6e8815c220 2024-09-19T08:47:00.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.990+0000 7f6e8e5c1640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6e880b6a20 con 0x7f6e880d0ff0 2024-09-19T08:47:00.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.990+0000 7f6e8e5c1640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6e880d0db0 con 0x7f6e880b6f40 2024-09-19T08:47:00.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.990+0000 7f6e8cb5e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e8815c220 0x7f6e88154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:00.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.990+0000 7f6e8cb5e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e8815c220 0x7f6e88154ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46292/0 (socket says 172.21.15.98:46292) 2024-09-19T08:47:00.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.990+0000 7f6e8cb5e640 1 -- 172.21.15.98:0/358108239 learned_addr learned my addr 172.21.15.98:0/358108239 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:00.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.990+0000 7f6e877fe640 1 --2- 172.21.15.98:0/358108239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e880d0ff0 0x7f6e8814fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:00.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.990+0000 7f6e87fff640 1 --2- 172.21.15.98:0/358108239 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6e880b6f40 0x7f6e8806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:00.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.990+0000 7f6e8cb5e640 1 -- 172.21.15.98:0/358108239 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6e880b6f40 msgr2=0x7f6e8806d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:00.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.990+0000 7f6e8cb5e640 1 --2- 172.21.15.98:0/358108239 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6e880b6f40 0x7f6e8806d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.990+0000 7f6e8cb5e640 1 -- 172.21.15.98:0/358108239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e880d0ff0 msgr2=0x7f6e8814fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:00.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.990+0000 7f6e8cb5e640 1 --2- 172.21.15.98:0/358108239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e880d0ff0 0x7f6e8814fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:00.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.990+0000 7f6e8cb5e640 1 -- 172.21.15.98:0/358108239 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e8815aff0 con 0x7f6e8815c220 2024-09-19T08:47:00.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.990+0000 7f6e877fe640 1 --2- 172.21.15.98:0/358108239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e880d0ff0 0x7f6e8814fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:00.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.990+0000 7f6e8cb5e640 1 --2- 172.21.15.98:0/358108239 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e8815c220 0x7f6e88154ad0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f6e8001a3d0 tx=0x7f6e80019540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:00.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.990+0000 7f6e84ff9640 1 -- 172.21.15.98:0/358108239 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e800bc020 con 0x7f6e8815c220 2024-09-19T08:47:00.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.990+0000 7f6e84ff9640 1 -- 172.21.15.98:0/358108239 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e800b8690 con 0x7f6e8815c220 2024-09-19T08:47:00.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.990+0000 7f6e84ff9640 1 -- 172.21.15.98:0/358108239 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e800b8a80 con 0x7f6e8815c220 2024-09-19T08:47:00.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.990+0000 7f6e8e5c1640 1 -- 172.21.15.98:0/358108239 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e880b84c0 con 0x7f6e8815c220 2024-09-19T08:47:00.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.990+0000 7f6e8e5c1640 1 -- 172.21.15.98:0/358108239 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e881556c0 con 0x7f6e8815c220 2024-09-19T08:47:00.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.990+0000 7f6e8e5c1640 1 -- 172.21.15.98:0/358108239 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e88156da0 con 0x7f6e8815c220 2024-09-19T08:47:00.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.994+0000 7f6e84ff9640 1 -- 172.21.15.98:0/358108239 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f6e800bf020 con 0x7f6e8815c220 2024-09-19T08:47:00.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.994+0000 7f6e84ff9640 1 --2- 172.21.15.98:0/358108239 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6e74068250 0x7f6e7406a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:00.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.994+0000 7f6e84ff9640 1 -- 172.21.15.98:0/358108239 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5737+0+0 (secure 0 0 0) 0x7f6e8011f520 con 0x7f6e8815c220 2024-09-19T08:47:00.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.994+0000 7f6e87fff640 1 --2- 172.21.15.98:0/358108239 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6e74068250 0x7f6e7406a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:00.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.994+0000 7f6e87fff640 1 --2- 172.21.15.98:0/358108239 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6e74068250 0x7f6e7406a710 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6e7800a750 tx=0x7f6e78008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:00.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:00.998+0000 7f6e84ff9640 1 -- 172.21.15.98:0/358108239 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6e800c6030 con 0x7f6e8815c220 2024-09-19T08:47:01.169 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.170+0000 7f6e8e5c1640 1 -- 172.21.15.98:0/358108239 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6e880688c0 con 0x7f6e8815c220 2024-09-19T08:47:01.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.170+0000 7f6e84ff9640 1 -- 172.21.15.98:0/358108239 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v158) ==== 80+0+0 (secure 0 0 0) 0x7f6e880b78e0 con 0x7f6e8815c220 2024-09-19T08:47:01.170 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-19T08:47:01.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.174+0000 7f6e8e5c1640 1 -- 172.21.15.98:0/358108239 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6e74068250 msgr2=0x7f6e7406a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:01.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.174+0000 7f6e8e5c1640 1 --2- 172.21.15.98:0/358108239 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6e74068250 0x7f6e7406a710 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6e7800a750 tx=0x7f6e78008040 comp rx=0 tx=0).stop 2024-09-19T08:47:01.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.174+0000 7f6e8e5c1640 1 -- 172.21.15.98:0/358108239 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e8815c220 msgr2=0x7f6e88154ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:01.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.174+0000 7f6e8e5c1640 1 --2- 172.21.15.98:0/358108239 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e8815c220 0x7f6e88154ad0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f6e8001a3d0 tx=0x7f6e80019540 comp rx=0 tx=0).stop 2024-09-19T08:47:01.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.174+0000 7f6e8e5c1640 1 -- 172.21.15.98:0/358108239 shutdown_connections 2024-09-19T08:47:01.173 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.174+0000 7f6e8e5c1640 1 --2- 172.21.15.98:0/358108239 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6e74068250 0x7f6e7406a710 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:01.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.174+0000 7f6e8e5c1640 1 --2- 172.21.15.98:0/358108239 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6e8815c220 0x7f6e88154ad0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:01.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.174+0000 7f6e8e5c1640 1 --2- 172.21.15.98:0/358108239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6e880d0ff0 0x7f6e8814fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:01.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.174+0000 7f6e8e5c1640 1 --2- 172.21.15.98:0/358108239 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6e880b6f40 0x7f6e8806d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:01.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.174+0000 7f6e8e5c1640 1 -- 172.21.15.98:0/358108239 >> 172.21.15.98:0/358108239 conn(0x7f6e880b40a0 msgr2=0x7f6e880733f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:01.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.174+0000 7f6e8e5c1640 1 -- 172.21.15.98:0/358108239 shutdown_connections 2024-09-19T08:47:01.174 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.174+0000 7f6e8e5c1640 1 -- 172.21.15.98:0/358108239 wait complete. 2024-09-19T08:47:01.190 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi098.front.sepia.ceph.com... 2024-09-19T08:47:01.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:01.190 DEBUG:teuthology.orchestra.run.smithi098:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T08:47:01.201 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:ceph-fuse[53115]: fuse finished with error 0 and tester_r 0 2024-09-19T08:47:01.203 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:47:01.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:01.203 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T08:47:01.348 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:47:01.349 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:47:01.349 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:47:01.356 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T08:47:01.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.482+0000 7f2a86bfd640 1 Processor -- start 2024-09-19T08:47:01.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.482+0000 7f2a86bfd640 1 -- start start 2024-09-19T08:47:01.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.482+0000 7f2a86bfd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a8015c550 0x7f2a8006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:01.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.482+0000 7f2a86bfd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2a800cfad0 0x7f2a800723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:01.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.482+0000 7f2a86bfd640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a8006d310 con 0x7f2a8006c1d0 2024-09-19T08:47:01.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.482+0000 7f2a86bfd640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f2a8006c610 con 0x7f2a800cfad0 2024-09-19T08:47:01.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.482+0000 7f2a86bfd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2a8006d5f0 con 0x7f2a8015c550 2024-09-19T08:47:01.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.482+0000 7f2a7ffff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f2a8006c1d0 0x7f2a800cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:60994/0 (socket says 172.21.15.98:60994) 2024-09-19T08:47:01.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.482+0000 7f2a7ffff640 1 -- 172.21.15.98:0/3400174659 learned_addr learned my addr 172.21.15.98:0/3400174659 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:01.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.482+0000 7f2a84999640 1 --2- 172.21.15.98:0/3400174659 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a8015c550 0x7f2a8006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:01.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.482+0000 7f2a8519a640 1 --2- 172.21.15.98:0/3400174659 >> v2:172.21.15.139:3300/0 conn(0x7f2a800cfad0 0x7f2a800723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:01.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.482+0000 7f2a7f7fe640 1 -- 172.21.15.98:0/3400174659 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 465059939 0 0) 0x7f2a8006d310 con 0x7f2a8006c1d0 2024-09-19T08:47:01.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.482+0000 7f2a7f7fe640 1 -- 172.21.15.98:0/3400174659 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a6c003530 con 0x7f2a8006c1d0 2024-09-19T08:47:01.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.482+0000 7f2a7f7fe640 1 -- 172.21.15.98:0/3400174659 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4011167107 0 0) 0x7f2a6c003530 con 0x7f2a8006c1d0 2024-09-19T08:47:01.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.482+0000 7f2a7f7fe640 1 -- 172.21.15.98:0/3400174659 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a8015c550 msgr2=0x7f2a8006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:01.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.482+0000 7f2a7f7fe640 1 --2- 172.21.15.98:0/3400174659 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a8015c550 0x7f2a8006bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:01.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.482+0000 7f2a7f7fe640 1 -- 172.21.15.98:0/3400174659 >> v2:172.21.15.139:3300/0 conn(0x7f2a800cfad0 msgr2=0x7f2a800723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:01.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.482+0000 7f2a7f7fe640 1 --2- 172.21.15.98:0/3400174659 >> v2:172.21.15.139:3300/0 conn(0x7f2a800cfad0 0x7f2a800723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:01.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.482+0000 7f2a7f7fe640 1 -- 172.21.15.98:0/3400174659 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a8006d8d0 con 0x7f2a8006c1d0 2024-09-19T08:47:01.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.482+0000 7f2a7f7fe640 1 -- 172.21.15.98:0/3400174659 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f2a68001680 con 0x7f2a8006c1d0 2024-09-19T08:47:01.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.482+0000 7f2a7f7fe640 1 -- 172.21.15.98:0/3400174659 >> v1:172.21.15.98:6789/0 conn(0x7f2a8006c1d0 legacy=0x7f2a800cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:01.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.482+0000 7f2a7f7fe640 1 --2- 172.21.15.98:0/3400174659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a6c0039c0 0x7f2a6c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:01.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.482+0000 7f2a7f7fe640 1 --2- 172.21.15.98:0/3400174659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a6c004810 0x7f2a6c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:01.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.482+0000 7f2a7f7fe640 1 --2- 172.21.15.98:0/3400174659 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a8015c550 0x7f2a6c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:01.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.482+0000 7f2a7f7fe640 1 -- 172.21.15.98:0/3400174659 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8006c610 con 0x7f2a6c004810 2024-09-19T08:47:01.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.482+0000 7f2a7f7fe640 1 -- 172.21.15.98:0/3400174659 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8006d5f0 con 0x7f2a6c0039c0 2024-09-19T08:47:01.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.482+0000 7f2a7f7fe640 1 -- 172.21.15.98:0/3400174659 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2a6c00b9d0 con 0x7f2a8015c550 2024-09-19T08:47:01.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.482+0000 7f2a7ffff640 1 --2- 172.21.15.98:0/3400174659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a6c004810 0x7f2a6c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:01.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.482+0000 7f2a84999640 1 --2- 172.21.15.98:0/3400174659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a6c0039c0 0x7f2a6c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:01.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.482+0000 7f2a7ffff640 1 -- 172.21.15.98:0/3400174659 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a8015c550 msgr2=0x7f2a6c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:01.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.482+0000 7f2a8519a640 1 --2- 172.21.15.98:0/3400174659 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a8015c550 0x7f2a6c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:01.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.482+0000 7f2a7ffff640 1 --2- 172.21.15.98:0/3400174659 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a8015c550 0x7f2a6c0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:01.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.482+0000 7f2a7ffff640 1 -- 172.21.15.98:0/3400174659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a6c0039c0 msgr2=0x7f2a6c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:01.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.482+0000 7f2a7ffff640 1 --2- 172.21.15.98:0/3400174659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a6c0039c0 0x7f2a6c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:01.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.482+0000 7f2a7ffff640 1 -- 172.21.15.98:0/3400174659 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a6c003530 con 0x7f2a6c004810 2024-09-19T08:47:01.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.482+0000 7f2a84999640 1 --2- 172.21.15.98:0/3400174659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a6c0039c0 0x7f2a6c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:01.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.486+0000 7f2a7ffff640 1 --2- 172.21.15.98:0/3400174659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a6c004810 0x7f2a6c004c90 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800c210 tx=0x7f2a6801f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899c81565d6eff34 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:01.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.486+0000 7f2a7f7fe640 1 -- 172.21.15.98:0/3400174659 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68007d40 con 0x7f2a6c004810 2024-09-19T08:47:01.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.486+0000 7f2a7f7fe640 1 -- 172.21.15.98:0/3400174659 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a680ac020 con 0x7f2a6c004810 2024-09-19T08:47:01.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.486+0000 7f2a7f7fe640 1 -- 172.21.15.98:0/3400174659 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a680058e0 con 0x7f2a6c004810 2024-09-19T08:47:01.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.486+0000 7f2a86bfd640 1 -- 172.21.15.98:0/3400174659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a6c004810 msgr2=0x7f2a6c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:01.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.486+0000 7f2a86bfd640 1 --2- 172.21.15.98:0/3400174659 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a6c004810 0x7f2a6c004c90 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800c210 tx=0x7f2a6801f3d0 comp rx=0 tx=0).stop 2024-09-19T08:47:01.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.486+0000 7f2a8519a640 1 -- 172.21.15.98:0/3400174659 reap_dead start 2024-09-19T08:47:01.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.486+0000 7f2a86bfd640 1 -- 172.21.15.98:0/3400174659 shutdown_connections 2024-09-19T08:47:01.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.486+0000 7f2a86bfd640 1 -- 172.21.15.98:0/3400174659 >> 172.21.15.98:0/3400174659 conn(0x7f2a800b2010 msgr2=0x7f2a800b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:01.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.486+0000 7f2a86bfd640 1 -- 172.21.15.98:0/3400174659 shutdown_connections 2024-09-19T08:47:01.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.486+0000 7f2a86bfd640 1 -- 172.21.15.98:0/3400174659 wait complete. 2024-09-19T08:47:01.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.486+0000 7f2a86bfd640 1 Processor -- start 2024-09-19T08:47:01.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.486+0000 7f2a86bfd640 1 -- start start 2024-09-19T08:47:01.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.490+0000 7f2a86bfd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a6c004810 0x7f2a80148db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:01.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.490+0000 7f2a86bfd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a801576a0 0x7f2a801552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:01.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.490+0000 7f2a86bfd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a801497a0 0x7f2a801545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:01.490 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.490+0000 7f2a86bfd640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8015cab0 con 0x7f2a801497a0 2024-09-19T08:47:01.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.490+0000 7f2a86bfd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8015c930 con 0x7f2a6c004810 2024-09-19T08:47:01.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.490+0000 7f2a86bfd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2a8006bc90 con 0x7f2a801576a0 2024-09-19T08:47:01.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.490+0000 7f2a8519a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a801497a0 0x7f2a801545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:01.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.490+0000 7f2a8519a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a801497a0 0x7f2a801545b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46306/0 (socket says 172.21.15.98:46306) 2024-09-19T08:47:01.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.490+0000 7f2a8519a640 1 -- 172.21.15.98:0/420891359 learned_addr learned my addr 172.21.15.98:0/420891359 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:01.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.490+0000 7f2a7ffff640 1 --2- 172.21.15.98:0/420891359 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a801576a0 0x7f2a801552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:01.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.490+0000 7f2a84999640 1 --2- 172.21.15.98:0/420891359 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a6c004810 0x7f2a80148db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:01.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.490+0000 7f2a8519a640 1 -- 172.21.15.98:0/420891359 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a801576a0 msgr2=0x7f2a801552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:01.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.490+0000 7f2a8519a640 1 --2- 172.21.15.98:0/420891359 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a801576a0 0x7f2a801552e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:01.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.490+0000 7f2a8519a640 1 -- 172.21.15.98:0/420891359 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a6c004810 msgr2=0x7f2a80148db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:01.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.490+0000 7f2a8519a640 1 --2- 172.21.15.98:0/420891359 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a6c004810 0x7f2a80148db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:01.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.490+0000 7f2a8519a640 1 -- 172.21.15.98:0/420891359 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a80155be0 con 0x7f2a801497a0 2024-09-19T08:47:01.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.490+0000 7f2a8519a640 1 --2- 172.21.15.98:0/420891359 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a801497a0 0x7f2a801545b0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f2a8006f2d0 tx=0x7f2a78010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:01.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.490+0000 7f2a7d7fa640 1 -- 172.21.15.98:0/420891359 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7800f040 con 0x7f2a801497a0 2024-09-19T08:47:01.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.490+0000 7f2a7d7fa640 1 -- 172.21.15.98:0/420891359 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a7800e890 con 0x7f2a801497a0 2024-09-19T08:47:01.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.490+0000 7f2a7d7fa640 1 -- 172.21.15.98:0/420891359 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7800ec80 con 0x7f2a801497a0 2024-09-19T08:47:01.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.490+0000 7f2a86bfd640 1 -- 172.21.15.98:0/420891359 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a80155ec0 con 0x7f2a801497a0 2024-09-19T08:47:01.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.490+0000 7f2a86bfd640 1 -- 172.21.15.98:0/420891359 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a800b3e60 con 0x7f2a801497a0 2024-09-19T08:47:01.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.490+0000 7f2a86bfd640 1 -- 172.21.15.98:0/420891359 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a800cf3c0 con 0x7f2a801497a0 2024-09-19T08:47:01.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.490+0000 7f2a7d7fa640 1 -- 172.21.15.98:0/420891359 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f2a780143f0 con 0x7f2a801497a0 2024-09-19T08:47:01.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.490+0000 7f2a7d7fa640 1 --2- 172.21.15.98:0/420891359 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2a60068250 0x7f2a6006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:01.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.490+0000 7f2a7d7fa640 1 -- 172.21.15.98:0/420891359 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5737+0+0 (secure 0 0 0) 0x7f2a78084380 con 0x7f2a801497a0 2024-09-19T08:47:01.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.490+0000 7f2a84999640 1 --2- 172.21.15.98:0/420891359 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2a60068250 0x7f2a6006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:01.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.494+0000 7f2a84999640 1 --2- 172.21.15.98:0/420891359 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2a60068250 0x7f2a6006a710 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2a70007440 tx=0x7f2a7000d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:01.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.494+0000 7f2a7d7fa640 1 -- 172.21.15.98:0/420891359 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2a78018030 con 0x7f2a801497a0 2024-09-19T08:47:01.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.674+0000 7f2a86bfd640 1 -- 172.21.15.98:0/420891359 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2a80159090 con 0x7f2a801497a0 2024-09-19T08:47:01.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.674+0000 7f2a7d7fa640 1 -- 172.21.15.98:0/420891359 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v158) ==== 80+0+0 (secure 0 0 0) 0x7f2a78056a40 con 0x7f2a801497a0 2024-09-19T08:47:01.674 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-19T08:47:01.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.678+0000 7f2a86bfd640 1 -- 172.21.15.98:0/420891359 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2a60068250 msgr2=0x7f2a6006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:01.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.678+0000 7f2a86bfd640 1 --2- 172.21.15.98:0/420891359 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2a60068250 0x7f2a6006a710 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2a70007440 tx=0x7f2a7000d040 comp rx=0 tx=0).stop 2024-09-19T08:47:01.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.678+0000 7f2a86bfd640 1 -- 172.21.15.98:0/420891359 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a801497a0 msgr2=0x7f2a801545b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:01.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.678+0000 7f2a86bfd640 1 --2- 172.21.15.98:0/420891359 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a801497a0 0x7f2a801545b0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f2a8006f2d0 tx=0x7f2a78010c20 comp rx=0 tx=0).stop 2024-09-19T08:47:01.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.678+0000 7f2a86bfd640 1 -- 172.21.15.98:0/420891359 shutdown_connections 2024-09-19T08:47:01.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.678+0000 7f2a86bfd640 1 --2- 172.21.15.98:0/420891359 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2a60068250 0x7f2a6006a710 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:01.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.678+0000 7f2a86bfd640 1 --2- 172.21.15.98:0/420891359 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2a801497a0 0x7f2a801545b0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:01.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.678+0000 7f2a86bfd640 1 --2- 172.21.15.98:0/420891359 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2a801576a0 0x7f2a801552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:01.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.678+0000 7f2a86bfd640 1 --2- 172.21.15.98:0/420891359 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a6c004810 0x7f2a80148db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:01.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.678+0000 7f2a86bfd640 1 -- 172.21.15.98:0/420891359 >> 172.21.15.98:0/420891359 conn(0x7f2a800b2010 msgr2=0x7f2a800cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:01.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.678+0000 7f2a86bfd640 1 -- 172.21.15.98:0/420891359 shutdown_connections 2024-09-19T08:47:01.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:01.678+0000 7f2a86bfd640 1 -- 172.21.15.98:0/420891359 wait complete. 2024-09-19T08:47:01.693 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi139.front.sepia.ceph.com... 2024-09-19T08:47:01.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:01.693 DEBUG:teuthology.orchestra.run.smithi139:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-19T08:47:01.707 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[36968]: fuse finished with error 0 and tester_r 0 2024-09-19T08:47:01.708 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:47:01.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:01.708 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-19T08:47:01.805 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:47:01.806 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:47:01.806 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:47:01.812 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15846 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T08:47:01.812 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:47:01.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:01.812 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-19T08:47:01.861 INFO:teuthology.orchestra.run.smithi098.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-19T08:47:01.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:47:01.862 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:47:01.862 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:47:01.862 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:47:01.868 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T08:47:01.868 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:47:01.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:01.868 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-19T08:47:01.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:47:01.915 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-19T08:47:01.915 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:47:02.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.046+0000 7f993f003640 1 Processor -- start 2024-09-19T08:47:02.044 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.046+0000 7f993f003640 1 -- start start 2024-09-19T08:47:02.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.046+0000 7f993f003640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f993806c3a0 0x7f99380cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:02.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.046+0000 7f993f003640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99380cf900 0x7f99380723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:02.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.046+0000 7f993f003640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f993806d310 con 0x7f993815c550 2024-09-19T08:47:02.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.046+0000 7f993f003640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f993806d5f0 con 0x7f993806c3a0 2024-09-19T08:47:02.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.046+0000 7f993f003640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f993806d8d0 con 0x7f99380cf900 2024-09-19T08:47:02.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.046+0000 7f9937fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f993806c3a0 0x7f99380cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:02.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.046+0000 7f993d5a0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99380cf900 0x7f99380723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:02.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.046+0000 7f993cd9f640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f993815c550 0x7f993806bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:53720/0 (socket says 172.21.15.98:53720) 2024-09-19T08:47:02.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.046+0000 7f993d5a0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99380cf900 0x7f99380723a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:49564/0 (socket says 172.21.15.98:49564) 2024-09-19T08:47:02.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.046+0000 7f9937fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f993806c3a0 0x7f99380cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:41654/0 (socket says 172.21.15.98:41654) 2024-09-19T08:47:02.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.046+0000 7f993cd9f640 1 -- 172.21.15.98:0/1955568903 learned_addr learned my addr 172.21.15.98:0/1955568903 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:02.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.046+0000 7f993d5a0640 1 -- 172.21.15.98:0/1955568903 >> v2:172.21.15.139:3300/0 conn(0x7f993806c3a0 msgr2=0x7f99380cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:02.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.046+0000 7f993d5a0640 1 --2- 172.21.15.98:0/1955568903 >> v2:172.21.15.139:3300/0 conn(0x7f993806c3a0 0x7f99380cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:02.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.046+0000 7f993d5a0640 1 -- 172.21.15.98:0/1955568903 >> v1:172.21.15.139:6789/0 conn(0x7f993815c550 legacy=0x7f993806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:02.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.046+0000 7f993d5a0640 1 -- 172.21.15.98:0/1955568903 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f993806dbb0 con 0x7f99380cf900 2024-09-19T08:47:02.048 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.050+0000 7f993d5a0640 1 --2- 172.21.15.98:0/1955568903 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99380cf900 0x7f99380723a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9930009d90 tx=0x7f993001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10e421d59f425b31 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:02.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.050+0000 7f99377fe640 1 -- 172.21.15.98:0/1955568903 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99300b2070 con 0x7f99380cf900 2024-09-19T08:47:02.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.050+0000 7f99377fe640 1 -- 172.21.15.98:0/1955568903 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99300a5940 con 0x7f99380cf900 2024-09-19T08:47:02.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.050+0000 7f99377fe640 1 -- 172.21.15.98:0/1955568903 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99300a5d10 con 0x7f99380cf900 2024-09-19T08:47:02.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.050+0000 7f993f003640 1 -- 172.21.15.98:0/1955568903 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99380cf900 msgr2=0x7f99380723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:02.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.050+0000 7f993f003640 1 --2- 172.21.15.98:0/1955568903 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99380cf900 0x7f99380723a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9930009d90 tx=0x7f993001c970 comp rx=0 tx=0).stop 2024-09-19T08:47:02.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.050+0000 7f993f003640 1 -- 172.21.15.98:0/1955568903 shutdown_connections 2024-09-19T08:47:02.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.050+0000 7f993f003640 1 --2- 172.21.15.98:0/1955568903 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f99380cf900 0x7f99380723a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:02.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.050+0000 7f993f003640 1 --2- 172.21.15.98:0/1955568903 >> v2:172.21.15.139:3300/0 conn(0x7f993806c3a0 0x7f99380cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:02.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.050+0000 7f993f003640 1 -- 172.21.15.98:0/1955568903 >> 172.21.15.98:0/1955568903 conn(0x7f99380b2010 msgr2=0x7f99380b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:02.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.050+0000 7f993f003640 1 -- 172.21.15.98:0/1955568903 shutdown_connections 2024-09-19T08:47:02.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.050+0000 7f993f003640 1 -- 172.21.15.98:0/1955568903 wait complete. 2024-09-19T08:47:02.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.050+0000 7f993f003640 1 Processor -- start 2024-09-19T08:47:02.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.050+0000 7f993f003640 1 -- start start 2024-09-19T08:47:02.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.054+0000 7f993f003640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f993806c3a0 0x7f993815a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:02.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.054+0000 7f993f003640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99380cf900 0x7f993813dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:02.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.054+0000 7f993f003640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f993815c550 0x7f99380ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:02.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.054+0000 7f993f003640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f99380734c0 con 0x7f99380cf900 2024-09-19T08:47:02.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.054+0000 7f993f003640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9938073340 con 0x7f993806c3a0 2024-09-19T08:47:02.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.054+0000 7f993f003640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9938073640 con 0x7f993815c550 2024-09-19T08:47:02.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.054+0000 7f9937fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99380cf900 0x7f993813dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:02.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.054+0000 7f9937fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99380cf900 0x7f993813dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46308/0 (socket says 172.21.15.98:46308) 2024-09-19T08:47:02.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.054+0000 7f9937fff640 1 -- 172.21.15.98:0/1533827607 learned_addr learned my addr 172.21.15.98:0/1533827607 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:02.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.054+0000 7f993cd9f640 1 --2- 172.21.15.98:0/1533827607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f993806c3a0 0x7f993815a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:02.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.054+0000 7f993d5a0640 1 --2- 172.21.15.98:0/1533827607 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f993815c550 0x7f99380ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:02.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.054+0000 7f9937fff640 1 -- 172.21.15.98:0/1533827607 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f993815c550 msgr2=0x7f99380ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:02.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.054+0000 7f9937fff640 1 --2- 172.21.15.98:0/1533827607 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f993815c550 0x7f99380ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:02.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.054+0000 7f9937fff640 1 -- 172.21.15.98:0/1533827607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f993806c3a0 msgr2=0x7f993815a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:02.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.054+0000 7f9937fff640 1 --2- 172.21.15.98:0/1533827607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f993806c3a0 0x7f993815a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:02.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.054+0000 7f9937fff640 1 -- 172.21.15.98:0/1533827607 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9938159150 con 0x7f99380cf900 2024-09-19T08:47:02.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.054+0000 7f9937fff640 1 --2- 172.21.15.98:0/1533827607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99380cf900 0x7f993813dc00 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f992c00e8a0 tx=0x7f992c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:02.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.054+0000 7f99357fa640 1 -- 172.21.15.98:0/1533827607 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c017070 con 0x7f99380cf900 2024-09-19T08:47:02.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.054+0000 7f99357fa640 1 -- 172.21.15.98:0/1533827607 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f992c00c940 con 0x7f99380cf900 2024-09-19T08:47:02.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.054+0000 7f993f003640 1 -- 172.21.15.98:0/1533827607 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9938159430 con 0x7f99380cf900 2024-09-19T08:47:02.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.054+0000 7f99357fa640 1 -- 172.21.15.98:0/1533827607 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c00ccf0 con 0x7f99380cf900 2024-09-19T08:47:02.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.054+0000 7f993f003640 1 -- 172.21.15.98:0/1533827607 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99381599f0 con 0x7f99380cf900 2024-09-19T08:47:02.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.054+0000 7f993f003640 1 -- 172.21.15.98:0/1533827607 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99380cf3c0 con 0x7f99380cf900 2024-09-19T08:47:02.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.054+0000 7f99357fa640 1 -- 172.21.15.98:0/1533827607 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f992c01a070 con 0x7f99380cf900 2024-09-19T08:47:02.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.058+0000 7f99357fa640 1 --2- 172.21.15.98:0/1533827607 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9924068250 0x7f992406a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:02.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.058+0000 7f99357fa640 1 -- 172.21.15.98:0/1533827607 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5737+0+0 (secure 0 0 0) 0x7f992c084aa0 con 0x7f99380cf900 2024-09-19T08:47:02.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.058+0000 7f993cd9f640 1 --2- 172.21.15.98:0/1533827607 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9924068250 0x7f992406a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:02.057 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.058+0000 7f993cd9f640 1 --2- 172.21.15.98:0/1533827607 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9924068250 0x7f992406a710 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9928000f70 tx=0x7f9928007e90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:02.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.058+0000 7f99357fa640 1 -- 172.21.15.98:0/1533827607 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f992c056b00 con 0x7f99380cf900 2024-09-19T08:47:02.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.254+0000 7f993f003640 1 -- 172.21.15.98:0/1533827607 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f993813e140 con 0x7f99380cf900 2024-09-19T08:47:02.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.258+0000 7f99357fa640 1 -- 172.21.15.98:0/1533827607 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5233 (secure 0 0 0) 0x7f992c05ae00 con 0x7f99380cf900 2024-09-19T08:47:02.256 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:47:02.257 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":92,"btime":"2024-09-19T08:46:56:581962+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/1752784745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":1752784745},{"type":"v1","addr":"172.21.15.139:6834","nonce":1752784745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":6212,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3564493643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6833","nonce":3564493643},{"type":"v1","addr":"172.21.15.139:6835","nonce":3564493643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":8763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6834/3374714194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3374714194},{"type":"v1","addr":"172.21.15.98:6834","nonce":3374714194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:46:39.868735+0000","modified":"2024-09-19T08:46:42.899681+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8766},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8766":{"gid":8766,"name":"c","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/3734544897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6833","nonce":3734544897},{"type":"v1","addr":"172.21.15.98:6835","nonce":3734544897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8766,"qdb_cluster":[8766]},"id":12}]} 2024-09-19T08:47:02.257 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 92 2024-09-19T08:47:02.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.258+0000 7f993f003640 1 -- 172.21.15.98:0/1533827607 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9924068250 msgr2=0x7f992406a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:02.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.258+0000 7f993f003640 1 --2- 172.21.15.98:0/1533827607 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9924068250 0x7f992406a710 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9928000f70 tx=0x7f9928007e90 comp rx=0 tx=0).stop 2024-09-19T08:47:02.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.258+0000 7f993f003640 1 -- 172.21.15.98:0/1533827607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99380cf900 msgr2=0x7f993813dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:02.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.258+0000 7f993f003640 1 --2- 172.21.15.98:0/1533827607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99380cf900 0x7f993813dc00 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f992c00e8a0 tx=0x7f992c00eca0 comp rx=0 tx=0).stop 2024-09-19T08:47:02.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.258+0000 7f993f003640 1 -- 172.21.15.98:0/1533827607 shutdown_connections 2024-09-19T08:47:02.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.258+0000 7f993f003640 1 --2- 172.21.15.98:0/1533827607 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9924068250 0x7f992406a710 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:02.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.258+0000 7f993f003640 1 --2- 172.21.15.98:0/1533827607 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f993815c550 0x7f99380ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:02.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.258+0000 7f993f003640 1 --2- 172.21.15.98:0/1533827607 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f99380cf900 0x7f993813dc00 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:02.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.258+0000 7f993f003640 1 --2- 172.21.15.98:0/1533827607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f993806c3a0 0x7f993815a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:02.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.258+0000 7f993f003640 1 -- 172.21.15.98:0/1533827607 >> 172.21.15.98:0/1533827607 conn(0x7f99380b2010 msgr2=0x7f993806bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:02.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.258+0000 7f993f003640 1 -- 172.21.15.98:0/1533827607 shutdown_connections 2024-09-19T08:47:02.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.258+0000 7f993f003640 1 -- 172.21.15.98:0/1533827607 wait complete. 2024-09-19T08:47:02.276 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:47:02.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.398+0000 7f6bd28fe640 1 Processor -- start 2024-09-19T08:47:02.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.398+0000 7f6bd28fe640 1 -- start start 2024-09-19T08:47:02.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.398+0000 7f6bd28fe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f6bcc15c550 0x7f6bcc06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:02.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.398+0000 7f6bd28fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6bcc06c1d0 0x7f6bcc0cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:02.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.398+0000 7f6bd28fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6bcc0cf900 0x7f6bcc0ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:02.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.398+0000 7f6bd28fe640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f6bcc06c610 con 0x7f6bcc0cf900 2024-09-19T08:47:02.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.398+0000 7f6bd28fe640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f6bcc06d310 con 0x7f6bcc15c550 2024-09-19T08:47:02.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.398+0000 7f6bd28fe640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6bcc06d5f0 con 0x7f6bcc06c1d0 2024-09-19T08:47:02.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.398+0000 7f6bd0e9b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6bcc0cf900 0x7f6bcc0ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:02.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.398+0000 7f6bd0e9b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6bcc0cf900 0x7f6bcc0ce1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46312/0 (socket says 172.21.15.98:46312) 2024-09-19T08:47:02.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.398+0000 7f6bd0e9b640 1 -- 172.21.15.98:0/1749363214 learned_addr learned my addr 172.21.15.98:0/1749363214 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:02.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.398+0000 7f6bd0e9b640 1 -- 172.21.15.98:0/1749363214 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6bcc06c1d0 msgr2=0x7f6bcc0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:02.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.398+0000 7f6bd0e9b640 1 --2- 172.21.15.98:0/1749363214 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6bcc06c1d0 0x7f6bcc0cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:02.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.398+0000 7f6bcbfff640 1 --2- 172.21.15.98:0/1749363214 >> v2:172.21.15.139:3300/0 conn(0x7f6bcc15c550 0x7f6bcc06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:02.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.398+0000 7f6bd0e9b640 1 -- 172.21.15.98:0/1749363214 >> v2:172.21.15.139:3300/0 conn(0x7f6bcc15c550 msgr2=0x7f6bcc06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:02.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.398+0000 7f6bd0e9b640 1 --2- 172.21.15.98:0/1749363214 >> v2:172.21.15.139:3300/0 conn(0x7f6bcc15c550 0x7f6bcc06bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:02.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.398+0000 7f6bd0e9b640 1 -- 172.21.15.98:0/1749363214 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bcc06d8d0 con 0x7f6bcc0cf900 2024-09-19T08:47:02.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.398+0000 7f6bd0e9b640 1 --2- 172.21.15.98:0/1749363214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bcc0cf900 0x7f6bcc0ce1a0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc0b56b0 tx=0x7f6bc401c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7bea1feca036c71 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:02.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.398+0000 7f6bcaffd640 1 -- 172.21.15.98:0/1749363214 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc40a64d0 con 0x7f6bcc0cf900 2024-09-19T08:47:02.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.398+0000 7f6bcaffd640 1 -- 172.21.15.98:0/1749363214 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bc40a6670 con 0x7f6bcc0cf900 2024-09-19T08:47:02.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.398+0000 7f6bcaffd640 1 -- 172.21.15.98:0/1749363214 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc40a6a40 con 0x7f6bcc0cf900 2024-09-19T08:47:02.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.402+0000 7f6bd28fe640 1 -- 172.21.15.98:0/1749363214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bcc0cf900 msgr2=0x7f6bcc0ce1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:02.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.402+0000 7f6bd28fe640 1 --2- 172.21.15.98:0/1749363214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bcc0cf900 0x7f6bcc0ce1a0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc0b56b0 tx=0x7f6bc401c730 comp rx=0 tx=0).stop 2024-09-19T08:47:02.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.402+0000 7f6bd28fe640 1 -- 172.21.15.98:0/1749363214 shutdown_connections 2024-09-19T08:47:02.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.402+0000 7f6bd28fe640 1 --2- 172.21.15.98:0/1749363214 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bcc0cf900 0x7f6bcc0ce1a0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:02.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.402+0000 7f6bd28fe640 1 --2- 172.21.15.98:0/1749363214 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6bcc06c1d0 0x7f6bcc0cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:02.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.402+0000 7f6bd28fe640 1 --2- 172.21.15.98:0/1749363214 >> v2:172.21.15.139:3300/0 conn(0x7f6bcc15c550 0x7f6bcc06bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:02.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.402+0000 7f6bd28fe640 1 -- 172.21.15.98:0/1749363214 >> 172.21.15.98:0/1749363214 conn(0x7f6bcc0b2010 msgr2=0x7f6bcc0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:02.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.402+0000 7f6bd28fe640 1 -- 172.21.15.98:0/1749363214 shutdown_connections 2024-09-19T08:47:02.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.402+0000 7f6bd28fe640 1 -- 172.21.15.98:0/1749363214 wait complete. 2024-09-19T08:47:02.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.402+0000 7f6bd28fe640 1 Processor -- start 2024-09-19T08:47:02.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.402+0000 7f6bd28fe640 1 -- start start 2024-09-19T08:47:02.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.402+0000 7f6bd28fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bcc06c1d0 0x7f6bcc15d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:02.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.402+0000 7f6bd28fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bcc0cf900 0x7f6bcc14b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:02.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.402+0000 7f6bd28fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6bcc15c550 0x7f6bcc0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:02.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.402+0000 7f6bd28fe640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6bcc0723a0 con 0x7f6bcc0cf900 2024-09-19T08:47:02.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.402+0000 7f6bd28fe640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6bcc15cb40 con 0x7f6bcc06c1d0 2024-09-19T08:47:02.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.402+0000 7f6bd28fe640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6bcc072520 con 0x7f6bcc15c550 2024-09-19T08:47:02.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.402+0000 7f6bcb7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bcc0cf900 0x7f6bcc14b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:02.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.402+0000 7f6bcb7fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bcc0cf900 0x7f6bcc14b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46324/0 (socket says 172.21.15.98:46324) 2024-09-19T08:47:02.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.402+0000 7f6bcb7fe640 1 -- 172.21.15.98:0/370617562 learned_addr learned my addr 172.21.15.98:0/370617562 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:02.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.402+0000 7f6bd0e9b640 1 --2- 172.21.15.98:0/370617562 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6bcc15c550 0x7f6bcc0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:02.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.402+0000 7f6bcbfff640 1 --2- 172.21.15.98:0/370617562 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bcc06c1d0 0x7f6bcc15d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:02.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.402+0000 7f6bcb7fe640 1 -- 172.21.15.98:0/370617562 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6bcc15c550 msgr2=0x7f6bcc0ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:02.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.402+0000 7f6bcb7fe640 1 --2- 172.21.15.98:0/370617562 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6bcc15c550 0x7f6bcc0ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:02.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.402+0000 7f6bcb7fe640 1 -- 172.21.15.98:0/370617562 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bcc06c1d0 msgr2=0x7f6bcc15d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:02.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.402+0000 7f6bcb7fe640 1 --2- 172.21.15.98:0/370617562 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bcc06c1d0 0x7f6bcc15d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:02.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.402+0000 7f6bcb7fe640 1 -- 172.21.15.98:0/370617562 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bcc15f3f0 con 0x7f6bcc0cf900 2024-09-19T08:47:02.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.402+0000 7f6bcb7fe640 1 --2- 172.21.15.98:0/370617562 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bcc0cf900 0x7f6bcc14b6f0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f6bc000b580 tx=0x7f6bc000ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:02.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.402+0000 7f6bc8ff9640 1 -- 172.21.15.98:0/370617562 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0014070 con 0x7f6bcc0cf900 2024-09-19T08:47:02.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.402+0000 7f6bc8ff9640 1 -- 172.21.15.98:0/370617562 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bc0010890 con 0x7f6bcc0cf900 2024-09-19T08:47:02.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.402+0000 7f6bd28fe640 1 -- 172.21.15.98:0/370617562 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bcc15f6d0 con 0x7f6bcc0cf900 2024-09-19T08:47:02.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.402+0000 7f6bc8ff9640 1 -- 172.21.15.98:0/370617562 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0010c80 con 0x7f6bcc0cf900 2024-09-19T08:47:02.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.402+0000 7f6bd28fe640 1 -- 172.21.15.98:0/370617562 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bcc15cf60 con 0x7f6bcc0cf900 2024-09-19T08:47:02.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.406+0000 7f6bd28fe640 1 -- 172.21.15.98:0/370617562 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bcc0ce1a0 con 0x7f6bcc0cf900 2024-09-19T08:47:02.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.406+0000 7f6bc8ff9640 1 -- 172.21.15.98:0/370617562 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f6bc0017070 con 0x7f6bcc0cf900 2024-09-19T08:47:02.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.406+0000 7f6bc8ff9640 1 --2- 172.21.15.98:0/370617562 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6bb8068250 0x7f6bb806a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:02.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.406+0000 7f6bc8ff9640 1 -- 172.21.15.98:0/370617562 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5737+0+0 (secure 0 0 0) 0x7f6bc00842c0 con 0x7f6bcc0cf900 2024-09-19T08:47:02.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.406+0000 7f6bcbfff640 1 --2- 172.21.15.98:0/370617562 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6bb8068250 0x7f6bb806a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:02.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.406+0000 7f6bcbfff640 1 --2- 172.21.15.98:0/370617562 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6bb8068250 0x7f6bb806a710 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc00a8d0 tx=0x7f6bbc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:02.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.410+0000 7f6bc8ff9640 1 -- 172.21.15.98:0/370617562 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6bc0056980 con 0x7f6bcc0cf900 2024-09-19T08:47:02.594 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.594+0000 7f6bd28fe640 1 -- 172.21.15.98:0/370617562 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6bcc15d3b0 con 0x7f6bcc0cf900 2024-09-19T08:47:02.595 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.594+0000 7f6bc8ff9640 1 -- 172.21.15.98:0/370617562 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5233 (secure 0 0 0) 0x7f6bc0010e20 con 0x7f6bcc0cf900 2024-09-19T08:47:02.595 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:47:02.595 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":92,"btime":"2024-09-19T08:46:56:581962+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/1752784745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":1752784745},{"type":"v1","addr":"172.21.15.139:6834","nonce":1752784745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":6212,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3564493643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6833","nonce":3564493643},{"type":"v1","addr":"172.21.15.139:6835","nonce":3564493643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":8763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6834/3374714194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3374714194},{"type":"v1","addr":"172.21.15.98:6834","nonce":3374714194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:46:39.868735+0000","modified":"2024-09-19T08:46:42.899681+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8766},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8766":{"gid":8766,"name":"c","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/3734544897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6833","nonce":3734544897},{"type":"v1","addr":"172.21.15.98:6835","nonce":3734544897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8766,"qdb_cluster":[8766]},"id":12}]} 2024-09-19T08:47:02.596 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 92 2024-09-19T08:47:02.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.598+0000 7f6bd28fe640 1 -- 172.21.15.98:0/370617562 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6bb8068250 msgr2=0x7f6bb806a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:02.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.598+0000 7f6bd28fe640 1 --2- 172.21.15.98:0/370617562 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6bb8068250 0x7f6bb806a710 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc00a8d0 tx=0x7f6bbc008040 comp rx=0 tx=0).stop 2024-09-19T08:47:02.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.598+0000 7f6bd28fe640 1 -- 172.21.15.98:0/370617562 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bcc0cf900 msgr2=0x7f6bcc14b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:02.597 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.598+0000 7f6bd28fe640 1 --2- 172.21.15.98:0/370617562 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bcc0cf900 0x7f6bcc14b6f0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f6bc000b580 tx=0x7f6bc000ba40 comp rx=0 tx=0).stop 2024-09-19T08:47:02.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.598+0000 7f6bd28fe640 1 -- 172.21.15.98:0/370617562 shutdown_connections 2024-09-19T08:47:02.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.598+0000 7f6bd28fe640 1 --2- 172.21.15.98:0/370617562 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6bb8068250 0x7f6bb806a710 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:02.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.598+0000 7f6bd28fe640 1 --2- 172.21.15.98:0/370617562 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6bcc15c550 0x7f6bcc0ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:02.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.598+0000 7f6bd28fe640 1 --2- 172.21.15.98:0/370617562 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6bcc0cf900 0x7f6bcc14b6f0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:02.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.598+0000 7f6bd28fe640 1 --2- 172.21.15.98:0/370617562 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6bcc06c1d0 0x7f6bcc15d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:02.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.598+0000 7f6bd28fe640 1 -- 172.21.15.98:0/370617562 >> 172.21.15.98:0/370617562 conn(0x7f6bcc0b2010 msgr2=0x7f6bcc159da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:02.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.598+0000 7f6bd28fe640 1 -- 172.21.15.98:0/370617562 shutdown_connections 2024-09-19T08:47:02.598 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.598+0000 7f6bd28fe640 1 -- 172.21.15.98:0/370617562 wait complete. 2024-09-19T08:47:02.614 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:47:02.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.734+0000 7f560e966640 1 Processor -- start 2024-09-19T08:47:02.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.734+0000 7f560e966640 1 -- start start 2024-09-19T08:47:02.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.738+0000 7f560e966640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f560806c3a0 0x7f56080cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:02.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.738+0000 7f560e966640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f560806d310 con 0x7f560815c550 2024-09-19T08:47:02.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.738+0000 7f560e966640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f560806d5f0 con 0x7f56080cf900 2024-09-19T08:47:02.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.738+0000 7f560e966640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f560806d8d0 con 0x7f560806c3a0 2024-09-19T08:47:02.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.738+0000 7f5607fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f560815c550 0x7f560806bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:32776/0 (socket says 172.21.15.98:32776) 2024-09-19T08:47:02.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.738+0000 7f5607fff640 1 -- 172.21.15.98:0/1377822287 learned_addr learned my addr 172.21.15.98:0/1377822287 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:02.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.738+0000 7f5606ffd640 1 -- 172.21.15.98:0/1377822287 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3075827932 0 0) 0x7f560806d310 con 0x7f560815c550 2024-09-19T08:47:02.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.738+0000 7f5606ffd640 1 -- 172.21.15.98:0/1377822287 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55f0003530 con 0x7f560815c550 2024-09-19T08:47:02.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.738+0000 7f56077fe640 1 --2- 172.21.15.98:0/1377822287 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f560806c3a0 0x7f56080cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:02.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.738+0000 7f5606ffd640 1 -- 172.21.15.98:0/1377822287 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3170064627 0 0) 0x7f55f0003530 con 0x7f560815c550 2024-09-19T08:47:02.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.738+0000 7f5606ffd640 1 -- 172.21.15.98:0/1377822287 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f560806c3a0 msgr2=0x7f56080cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:02.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.738+0000 7f5606ffd640 1 --2- 172.21.15.98:0/1377822287 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f560806c3a0 0x7f56080cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:02.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.738+0000 7f5606ffd640 1 -- 172.21.15.98:0/1377822287 >> v1:172.21.15.139:6789/0 conn(0x7f56080cf900 legacy=0x7f56080723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:02.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.738+0000 7f5606ffd640 1 -- 172.21.15.98:0/1377822287 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560806dbb0 con 0x7f560815c550 2024-09-19T08:47:02.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.738+0000 7f5606ffd640 1 -- 172.21.15.98:0/1377822287 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f55f8001680 con 0x7f560815c550 2024-09-19T08:47:02.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.738+0000 7f5606ffd640 1 -- 172.21.15.98:0/1377822287 >> v1:172.21.15.98:6789/0 conn(0x7f560815c550 legacy=0x7f560806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:02.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.738+0000 7f5606ffd640 1 --2- 172.21.15.98:0/1377822287 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55f00039f0 0x7f55f0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:02.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.738+0000 7f5606ffd640 1 --2- 172.21.15.98:0/1377822287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55f0004810 0x7f55f0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:02.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.738+0000 7f5606ffd640 1 --2- 172.21.15.98:0/1377822287 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f560806c3a0 0x7f55f00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:02.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.738+0000 7f5606ffd640 1 -- 172.21.15.98:0/1377822287 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f560806d8d0 con 0x7f55f00039f0 2024-09-19T08:47:02.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.738+0000 7f5606ffd640 1 -- 172.21.15.98:0/1377822287 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f55f000ba80 con 0x7f55f0004810 2024-09-19T08:47:02.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.738+0000 7f5606ffd640 1 -- 172.21.15.98:0/1377822287 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f55f000bc80 con 0x7f560806c3a0 2024-09-19T08:47:02.742 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.738+0000 7f5607fff640 1 --2- 172.21.15.98:0/1377822287 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55f00039f0 0x7f55f0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:02.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.738+0000 7f56077fe640 1 --2- 172.21.15.98:0/1377822287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55f0004810 0x7f55f0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:02.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.738+0000 7f5607fff640 1 -- 172.21.15.98:0/1377822287 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f560806c3a0 msgr2=0x7f55f00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:02.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.738+0000 7f560cf03640 1 --2- 172.21.15.98:0/1377822287 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f560806c3a0 0x7f55f00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:02.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.738+0000 7f5607fff640 1 --2- 172.21.15.98:0/1377822287 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f560806c3a0 0x7f55f00051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:02.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.738+0000 7f5607fff640 1 -- 172.21.15.98:0/1377822287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55f0004810 msgr2=0x7f55f0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:02.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.738+0000 7f5607fff640 1 --2- 172.21.15.98:0/1377822287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55f0004810 0x7f55f0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:02.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.738+0000 7f5607fff640 1 -- 172.21.15.98:0/1377822287 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f0003530 con 0x7f55f00039f0 2024-09-19T08:47:02.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.738+0000 7f56077fe640 1 --2- 172.21.15.98:0/1377822287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f55f0004810 0x7f55f0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:02.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.742+0000 7f5607fff640 1 --2- 172.21.15.98:0/1377822287 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55f00039f0 0x7f55f0003e10 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f55f800c190 tx=0x7f55f801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f1b6b5925f31a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:02.744 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.742+0000 7f5606ffd640 1 -- 172.21.15.98:0/1377822287 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f8007d00 con 0x7f55f00039f0 2024-09-19T08:47:02.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.742+0000 7f5606ffd640 1 -- 172.21.15.98:0/1377822287 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55f80a9020 con 0x7f55f00039f0 2024-09-19T08:47:02.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.742+0000 7f5606ffd640 1 -- 172.21.15.98:0/1377822287 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f8005b00 con 0x7f55f00039f0 2024-09-19T08:47:02.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.742+0000 7f560e966640 1 -- 172.21.15.98:0/1377822287 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55f00039f0 msgr2=0x7f55f0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:02.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.742+0000 7f560e966640 1 --2- 172.21.15.98:0/1377822287 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f55f00039f0 0x7f55f0003e10 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f55f800c190 tx=0x7f55f801cc20 comp rx=0 tx=0).stop 2024-09-19T08:47:02.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.742+0000 7f560cf03640 1 -- 172.21.15.98:0/1377822287 reap_dead start 2024-09-19T08:47:02.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.742+0000 7f560e966640 1 -- 172.21.15.98:0/1377822287 shutdown_connections 2024-09-19T08:47:02.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.742+0000 7f560e966640 1 -- 172.21.15.98:0/1377822287 >> 172.21.15.98:0/1377822287 conn(0x7f56080b2010 msgr2=0x7f56080b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:02.749 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.742+0000 7f560e966640 1 -- 172.21.15.98:0/1377822287 shutdown_connections 2024-09-19T08:47:02.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.742+0000 7f560e966640 1 -- 172.21.15.98:0/1377822287 wait complete. 2024-09-19T08:47:02.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.742+0000 7f560e966640 1 Processor -- start 2024-09-19T08:47:02.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.742+0000 7f560e966640 1 -- start start 2024-09-19T08:47:02.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.742+0000 7f560e966640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f55f00039f0 0x7f56081533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:02.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.742+0000 7f560e966640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f560814cd30 0x7f5608141b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:02.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.742+0000 7f560e966640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56081522b0 0x7f5608152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:02.750 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.742+0000 7f560e966640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f56080734d0 con 0x7f560814cd30 2024-09-19T08:47:02.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.742+0000 7f560e966640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f560815cb40 con 0x7f56081522b0 2024-09-19T08:47:02.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.742+0000 7f560e966640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5608073650 con 0x7f55f00039f0 2024-09-19T08:47:02.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.742+0000 7f56077fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f560814cd30 0x7f5608141b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:02.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.742+0000 7f56077fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f560814cd30 0x7f5608141b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46332/0 (socket says 172.21.15.98:46332) 2024-09-19T08:47:02.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.742+0000 7f56077fe640 1 -- 172.21.15.98:0/3355444512 learned_addr learned my addr 172.21.15.98:0/3355444512 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:02.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.742+0000 7f560cf03640 1 --2- 172.21.15.98:0/3355444512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56081522b0 0x7f5608152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:02.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.742+0000 7f56077fe640 1 -- 172.21.15.98:0/3355444512 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f55f00039f0 msgr2=0x7f56081533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:02.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.742+0000 7f5607fff640 1 --2- 172.21.15.98:0/3355444512 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f55f00039f0 0x7f56081533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:02.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.742+0000 7f56077fe640 1 --2- 172.21.15.98:0/3355444512 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f55f00039f0 0x7f56081533e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:02.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.742+0000 7f56077fe640 1 -- 172.21.15.98:0/3355444512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56081522b0 msgr2=0x7f5608152730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:02.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.742+0000 7f56077fe640 1 --2- 172.21.15.98:0/3355444512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56081522b0 0x7f5608152730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:02.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.742+0000 7f56077fe640 1 -- 172.21.15.98:0/3355444512 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5608142070 con 0x7f560814cd30 2024-09-19T08:47:02.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.742+0000 7f56077fe640 1 --2- 172.21.15.98:0/3355444512 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f560814cd30 0x7f5608141b30 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f55fc010890 tx=0x7f55fc010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:02.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.742+0000 7f5604ff9640 1 -- 172.21.15.98:0/3355444512 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc00f040 con 0x7f560814cd30 2024-09-19T08:47:02.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.742+0000 7f5604ff9640 1 -- 172.21.15.98:0/3355444512 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55fc00eb60 con 0x7f560814cd30 2024-09-19T08:47:02.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.742+0000 7f560e966640 1 -- 172.21.15.98:0/3355444512 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5608153ae0 con 0x7f560814cd30 2024-09-19T08:47:02.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.742+0000 7f5604ff9640 1 -- 172.21.15.98:0/3355444512 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc01c400 con 0x7f560814cd30 2024-09-19T08:47:02.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.742+0000 7f560e966640 1 -- 172.21.15.98:0/3355444512 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56081540a0 con 0x7f560814cd30 2024-09-19T08:47:02.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.746+0000 7f560e966640 1 -- 172.21.15.98:0/3355444512 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56080d0820 con 0x7f560814cd30 2024-09-19T08:47:02.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.746+0000 7f5604ff9640 1 -- 172.21.15.98:0/3355444512 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f55fc01c5a0 con 0x7f560814cd30 2024-09-19T08:47:02.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.746+0000 7f5604ff9640 1 --2- 172.21.15.98:0/3355444512 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f55f4068250 0x7f55f406a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:02.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.746+0000 7f5604ff9640 1 -- 172.21.15.98:0/3355444512 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5737+0+0 (secure 0 0 0) 0x7f55fc0849d0 con 0x7f560814cd30 2024-09-19T08:47:02.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.746+0000 7f5607fff640 1 --2- 172.21.15.98:0/3355444512 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f55f4068250 0x7f55f406a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:02.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.746+0000 7f5607fff640 1 --2- 172.21.15.98:0/3355444512 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f55f4068250 0x7f55f406a710 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f55f80aa000 tx=0x7f560806d4e0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:02.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.750+0000 7f5604ff9640 1 -- 172.21.15.98:0/3355444512 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f55fc018030 con 0x7f560814cd30 2024-09-19T08:47:02.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.898+0000 7f560e966640 1 -- 172.21.15.98:0/3355444512 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56080d0540 con 0x7f560814cd30 2024-09-19T08:47:02.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.902+0000 7f5604ff9640 1 -- 172.21.15.98:0/3355444512 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) ==== 74+0+16783 (secure 0 0 0) 0x7f55fc057080 con 0x7f560814cd30 2024-09-19T08:47:02.902 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:47:02.902 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":158,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:46:41.262235+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:46:37.833554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-19T08:46:38.858268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:47:02.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.906+0000 7f560e966640 1 -- 172.21.15.98:0/3355444512 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f55f4068250 msgr2=0x7f55f406a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:02.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.906+0000 7f560e966640 1 --2- 172.21.15.98:0/3355444512 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f55f4068250 0x7f55f406a710 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f55f80aa000 tx=0x7f560806d4e0 comp rx=0 tx=0).stop 2024-09-19T08:47:02.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.906+0000 7f560e966640 1 -- 172.21.15.98:0/3355444512 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f560814cd30 msgr2=0x7f5608141b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:02.905 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.906+0000 7f560e966640 1 --2- 172.21.15.98:0/3355444512 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f560814cd30 0x7f5608141b30 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f55fc010890 tx=0x7f55fc010d50 comp rx=0 tx=0).stop 2024-09-19T08:47:02.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.906+0000 7f560e966640 1 -- 172.21.15.98:0/3355444512 shutdown_connections 2024-09-19T08:47:02.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.906+0000 7f560e966640 1 --2- 172.21.15.98:0/3355444512 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f55f4068250 0x7f55f406a710 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:02.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.906+0000 7f560e966640 1 --2- 172.21.15.98:0/3355444512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56081522b0 0x7f5608152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:02.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.906+0000 7f560e966640 1 --2- 172.21.15.98:0/3355444512 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f560814cd30 0x7f5608141b30 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:02.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.906+0000 7f560e966640 1 --2- 172.21.15.98:0/3355444512 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f55f00039f0 0x7f56081533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:02.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.906+0000 7f560e966640 1 -- 172.21.15.98:0/3355444512 >> 172.21.15.98:0/3355444512 conn(0x7f56080b2010 msgr2=0x7f56081599c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:02.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.906+0000 7f560e966640 1 -- 172.21.15.98:0/3355444512 shutdown_connections 2024-09-19T08:47:02.906 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:02.906+0000 7f560e966640 1 -- 172.21.15.98:0/3355444512 wait complete. 2024-09-19T08:47:02.921 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T08:47:02.921 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:47:03.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.038+0000 7f418affe640 1 Processor -- start 2024-09-19T08:47:03.040 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.038+0000 7f418affe640 1 -- start start 2024-09-19T08:47:03.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.042+0000 7f418affe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41840cd350 0x7f41840b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.042+0000 7f418affe640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f41840b6f50 0x7f41840cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.042+0000 7f418affe640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41840cd730 con 0x7f41840ce880 2024-09-19T08:47:03.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.042+0000 7f418affe640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f41840b5f60 con 0x7f41840b6f50 2024-09-19T08:47:03.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.042+0000 7f418affe640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f41840b6190 con 0x7f41840cd350 2024-09-19T08:47:03.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.042+0000 7f4183fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f41840b6f50 0x7f41840cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.042+0000 7f4188d9a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41840cd350 0x7f41840b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.042+0000 7f4188d9a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41840cd350 0x7f41840b24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:49640/0 (socket says 172.21.15.98:49640) 2024-09-19T08:47:03.041 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.042+0000 7f4188d9a640 1 -- 172.21.15.98:0/5269508 learned_addr learned my addr 172.21.15.98:0/5269508 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:03.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.042+0000 7f418959b640 1 --1- 172.21.15.98:0/5269508 >> v1:172.21.15.139:6789/0 conn(0x7f41840ce880 0x7f418406c030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:53730/0 (socket says 172.21.15.98:53730) 2024-09-19T08:47:03.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.042+0000 7f4183fff640 1 -- 172.21.15.98:0/5269508 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41840cd350 msgr2=0x7f41840b24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:03.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.042+0000 7f4183fff640 1 --2- 172.21.15.98:0/5269508 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41840cd350 0x7f41840b24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.042+0000 7f4183fff640 1 -- 172.21.15.98:0/5269508 >> v1:172.21.15.139:6789/0 conn(0x7f41840ce880 legacy=0x7f418406c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:03.042 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.042+0000 7f4183fff640 1 -- 172.21.15.98:0/5269508 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41840b6470 con 0x7f41840b6f50 2024-09-19T08:47:03.045 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.046+0000 7f4183fff640 1 --2- 172.21.15.98:0/5269508 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41840b6f50 0x7f41840cdc70 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f41780097c0 tx=0x7f417801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b32dae56ab6d6d7c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:03.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.046+0000 7f41837fe640 1 -- 172.21.15.98:0/5269508 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41780b2070 con 0x7f41840b6f50 2024-09-19T08:47:03.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.046+0000 7f41837fe640 1 -- 172.21.15.98:0/5269508 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41780ad680 con 0x7f41840b6f50 2024-09-19T08:47:03.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.046+0000 7f41837fe640 1 -- 172.21.15.98:0/5269508 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41780ada50 con 0x7f41840b6f50 2024-09-19T08:47:03.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.046+0000 7f418affe640 1 -- 172.21.15.98:0/5269508 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41840b6f50 msgr2=0x7f41840cdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:03.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.046+0000 7f418affe640 1 --2- 172.21.15.98:0/5269508 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41840b6f50 0x7f41840cdc70 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f41780097c0 tx=0x7f417801c470 comp rx=0 tx=0).stop 2024-09-19T08:47:03.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.046+0000 7f418affe640 1 -- 172.21.15.98:0/5269508 shutdown_connections 2024-09-19T08:47:03.046 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.046+0000 7f418affe640 1 --2- 172.21.15.98:0/5269508 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41840b6f50 0x7f41840cdc70 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.046+0000 7f418affe640 1 --2- 172.21.15.98:0/5269508 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41840cd350 0x7f41840b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.046+0000 7f418affe640 1 -- 172.21.15.98:0/5269508 >> 172.21.15.98:0/5269508 conn(0x7f4184073170 msgr2=0x7f4184073580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:03.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.046+0000 7f418affe640 1 -- 172.21.15.98:0/5269508 shutdown_connections 2024-09-19T08:47:03.047 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.046+0000 7f418affe640 1 -- 172.21.15.98:0/5269508 wait complete. 2024-09-19T08:47:03.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.046+0000 7f418affe640 1 Processor -- start 2024-09-19T08:47:03.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.046+0000 7f418affe640 1 -- start start 2024-09-19T08:47:03.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.046+0000 7f418affe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41840b6f50 0x7f4184099380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.046+0000 7f418affe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41840cd350 0x7f41840c3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.046+0000 7f418affe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41840ce880 0x7f41840c4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.050 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.046+0000 7f418affe640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f4184168190 con 0x7f41840ce880 2024-09-19T08:47:03.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.046+0000 7f418affe640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4184168010 con 0x7f41840cd350 2024-09-19T08:47:03.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.046+0000 7f418affe640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f4184168310 con 0x7f41840b6f50 2024-09-19T08:47:03.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.046+0000 7f418959b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41840ce880 0x7f41840c4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.046+0000 7f418959b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41840ce880 0x7f41840c4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46342/0 (socket says 172.21.15.98:46342) 2024-09-19T08:47:03.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.046+0000 7f418959b640 1 -- 172.21.15.98:0/3247258760 learned_addr learned my addr 172.21.15.98:0/3247258760 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:03.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.046+0000 7f4183fff640 1 --2- 172.21.15.98:0/3247258760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41840cd350 0x7f41840c3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.046+0000 7f4188d9a640 1 --2- 172.21.15.98:0/3247258760 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41840b6f50 0x7f4184099380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.046+0000 7f418959b640 1 -- 172.21.15.98:0/3247258760 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41840b6f50 msgr2=0x7f4184099380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:03.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.046+0000 7f418959b640 1 --2- 172.21.15.98:0/3247258760 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41840b6f50 0x7f4184099380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.046+0000 7f418959b640 1 -- 172.21.15.98:0/3247258760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41840cd350 msgr2=0x7f41840c3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:03.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.046+0000 7f418959b640 1 --2- 172.21.15.98:0/3247258760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41840cd350 0x7f41840c3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.046+0000 7f418959b640 1 -- 172.21.15.98:0/3247258760 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f418415da90 con 0x7f41840ce880 2024-09-19T08:47:03.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.050+0000 7f418959b640 1 --2- 172.21.15.98:0/3247258760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41840ce880 0x7f41840c4190 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f417c00c7a0 tx=0x7f417c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:03.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.050+0000 7f41817fa640 1 -- 172.21.15.98:0/3247258760 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417c017070 con 0x7f41840ce880 2024-09-19T08:47:03.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.050+0000 7f41817fa640 1 -- 172.21.15.98:0/3247258760 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f417c00a640 con 0x7f41840ce880 2024-09-19T08:47:03.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.050+0000 7f41817fa640 1 -- 172.21.15.98:0/3247258760 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417c00a9f0 con 0x7f41840ce880 2024-09-19T08:47:03.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.050+0000 7f418affe640 1 -- 172.21.15.98:0/3247258760 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f418409a8b0 con 0x7f41840ce880 2024-09-19T08:47:03.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.050+0000 7f418affe640 1 -- 172.21.15.98:0/3247258760 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f418415d7b0 con 0x7f41840ce880 2024-09-19T08:47:03.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.050+0000 7f418affe640 1 -- 172.21.15.98:0/3247258760 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41840cdc70 con 0x7f41840ce880 2024-09-19T08:47:03.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.050+0000 7f41817fa640 1 -- 172.21.15.98:0/3247258760 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f417c01a070 con 0x7f41840ce880 2024-09-19T08:47:03.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.050+0000 7f41817fa640 1 --2- 172.21.15.98:0/3247258760 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4170068250 0x7f417006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.050+0000 7f41817fa640 1 -- 172.21.15.98:0/3247258760 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5737+0+0 (secure 0 0 0) 0x7f417c083960 con 0x7f41840ce880 2024-09-19T08:47:03.052 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.050+0000 7f4188d9a640 1 --2- 172.21.15.98:0/3247258760 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4170068250 0x7f417006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.053 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.050+0000 7f4188d9a640 1 --2- 172.21.15.98:0/3247258760 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4170068250 0x7f417006a710 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4174009ae0 tx=0x7f4174008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:03.054 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.054+0000 7f41817fa640 1 -- 172.21.15.98:0/3247258760 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f417c055a50 con 0x7f41840ce880 2024-09-19T08:47:03.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.254+0000 7f418affe640 1 -- 172.21.15.98:0/3247258760 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f418415a310 con 0x7f41840ce880 2024-09-19T08:47:03.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.254+0000 7f41817fa640 1 -- 172.21.15.98:0/3247258760 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5233 (secure 0 0 0) 0x7f417c059d50 con 0x7f41840ce880 2024-09-19T08:47:03.254 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:47:03.255 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":92,"btime":"2024-09-19T08:46:56:581962+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/1752784745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":1752784745},{"type":"v1","addr":"172.21.15.139:6834","nonce":1752784745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":6212,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3564493643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6833","nonce":3564493643},{"type":"v1","addr":"172.21.15.139:6835","nonce":3564493643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":8763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6834/3374714194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3374714194},{"type":"v1","addr":"172.21.15.98:6834","nonce":3374714194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:46:39.868735+0000","modified":"2024-09-19T08:46:42.899681+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8766},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8766":{"gid":8766,"name":"c","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/3734544897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6833","nonce":3734544897},{"type":"v1","addr":"172.21.15.98:6835","nonce":3734544897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8766,"qdb_cluster":[8766]},"id":12}]} 2024-09-19T08:47:03.255 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 92 2024-09-19T08:47:03.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.258+0000 7f418affe640 1 -- 172.21.15.98:0/3247258760 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4170068250 msgr2=0x7f417006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:03.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.258+0000 7f418affe640 1 --2- 172.21.15.98:0/3247258760 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4170068250 0x7f417006a710 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4174009ae0 tx=0x7f4174008040 comp rx=0 tx=0).stop 2024-09-19T08:47:03.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.258+0000 7f418affe640 1 -- 172.21.15.98:0/3247258760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41840ce880 msgr2=0x7f41840c4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:03.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.258+0000 7f418affe640 1 --2- 172.21.15.98:0/3247258760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41840ce880 0x7f41840c4190 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f417c00c7a0 tx=0x7f417c00cc60 comp rx=0 tx=0).stop 2024-09-19T08:47:03.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.258+0000 7f418affe640 1 -- 172.21.15.98:0/3247258760 shutdown_connections 2024-09-19T08:47:03.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.258+0000 7f418affe640 1 --2- 172.21.15.98:0/3247258760 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4170068250 0x7f417006a710 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.258+0000 7f418affe640 1 --2- 172.21.15.98:0/3247258760 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f41840ce880 0x7f41840c4190 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.258+0000 7f418affe640 1 --2- 172.21.15.98:0/3247258760 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41840cd350 0x7f41840c3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.258+0000 7f418affe640 1 --2- 172.21.15.98:0/3247258760 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f41840b6f50 0x7f4184099380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.258+0000 7f418affe640 1 -- 172.21.15.98:0/3247258760 >> 172.21.15.98:0/3247258760 conn(0x7f4184073170 msgr2=0x7f41841565f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:03.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.258+0000 7f418affe640 1 -- 172.21.15.98:0/3247258760 shutdown_connections 2024-09-19T08:47:03.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.258+0000 7f418affe640 1 -- 172.21.15.98:0/3247258760 wait complete. 2024-09-19T08:47:03.273 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:47:03.383 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.382+0000 7fd4eb4f7640 1 Processor -- start 2024-09-19T08:47:03.384 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.382+0000 7fd4eb4f7640 1 -- start start 2024-09-19T08:47:03.386 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.386+0000 7fd4eb4f7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fd4e415c550 0x7fd4e406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.386+0000 7fd4eb4f7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd4e40cfad0 0x7fd4e40723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.386+0000 7fd4eb4f7640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4e406d310 con 0x7fd4e406c1d0 2024-09-19T08:47:03.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.386+0000 7fd4eb4f7640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fd4e406c640 con 0x7fd4e415c550 2024-09-19T08:47:03.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.386+0000 7fd4eb4f7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd4e406d5f0 con 0x7fd4e40cfad0 2024-09-19T08:47:03.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.386+0000 7fd4e8a92640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fd4e406c1d0 0x7fd4e40cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:32792/0 (socket says 172.21.15.98:32792) 2024-09-19T08:47:03.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.386+0000 7fd4e8a92640 1 -- 172.21.15.98:0/2759337613 learned_addr learned my addr 172.21.15.98:0/2759337613 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:03.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.386+0000 7fd4e9a94640 1 --2- 172.21.15.98:0/2759337613 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd4e40cfad0 0x7fd4e40723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.386+0000 7fd4e9293640 1 --2- 172.21.15.98:0/2759337613 >> v2:172.21.15.139:3300/0 conn(0x7fd4e415c550 0x7fd4e406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.386+0000 7fd4dbfff640 1 -- 172.21.15.98:0/2759337613 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3740600900 0 0) 0x7fd4e406d310 con 0x7fd4e406c1d0 2024-09-19T08:47:03.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.386+0000 7fd4dbfff640 1 -- 172.21.15.98:0/2759337613 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd4d0003530 con 0x7fd4e406c1d0 2024-09-19T08:47:03.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.386+0000 7fd4dbfff640 1 -- 172.21.15.98:0/2759337613 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4019422592 0 0) 0x7fd4d0003530 con 0x7fd4e406c1d0 2024-09-19T08:47:03.387 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.386+0000 7fd4dbfff640 1 -- 172.21.15.98:0/2759337613 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd4e40cfad0 msgr2=0x7fd4e40723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:03.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.386+0000 7fd4dbfff640 1 --2- 172.21.15.98:0/2759337613 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd4e40cfad0 0x7fd4e40723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.386+0000 7fd4dbfff640 1 -- 172.21.15.98:0/2759337613 >> v2:172.21.15.139:3300/0 conn(0x7fd4e415c550 msgr2=0x7fd4e406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:03.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.386+0000 7fd4dbfff640 1 --2- 172.21.15.98:0/2759337613 >> v2:172.21.15.139:3300/0 conn(0x7fd4e415c550 0x7fd4e406bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.386+0000 7fd4dbfff640 1 -- 172.21.15.98:0/2759337613 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4e406d8d0 con 0x7fd4e406c1d0 2024-09-19T08:47:03.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.386+0000 7fd4dbfff640 1 -- 172.21.15.98:0/2759337613 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fd4d4001680 con 0x7fd4e406c1d0 2024-09-19T08:47:03.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.386+0000 7fd4dbfff640 1 -- 172.21.15.98:0/2759337613 >> v1:172.21.15.98:6789/0 conn(0x7fd4e406c1d0 legacy=0x7fd4e40cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:03.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.386+0000 7fd4dbfff640 1 --2- 172.21.15.98:0/2759337613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4d00039c0 0x7fd4d0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.386+0000 7fd4dbfff640 1 --2- 172.21.15.98:0/2759337613 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd4e40cfad0 0x7fd4d0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.386+0000 7fd4dbfff640 1 --2- 172.21.15.98:0/2759337613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4d0004d50 0x7fd4d00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.386+0000 7fd4dbfff640 1 -- 172.21.15.98:0/2759337613 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e406c640 con 0x7fd4d0004d50 2024-09-19T08:47:03.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.386+0000 7fd4dbfff640 1 -- 172.21.15.98:0/2759337613 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e406d5f0 con 0x7fd4d00039c0 2024-09-19T08:47:03.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.386+0000 7fd4dbfff640 1 -- 172.21.15.98:0/2759337613 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd4d000b9d0 con 0x7fd4e40cfad0 2024-09-19T08:47:03.388 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.386+0000 7fd4e9a94640 1 --2- 172.21.15.98:0/2759337613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4d0004d50 0x7fd4d00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.386+0000 7fd4e9293640 1 --2- 172.21.15.98:0/2759337613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4d00039c0 0x7fd4d0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.386+0000 7fd4e8a92640 1 --2- 172.21.15.98:0/2759337613 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd4e40cfad0 0x7fd4d0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.386+0000 7fd4e9a94640 1 -- 172.21.15.98:0/2759337613 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd4e40cfad0 msgr2=0x7fd4d0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:03.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.386+0000 7fd4e9a94640 1 --2- 172.21.15.98:0/2759337613 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd4e40cfad0 0x7fd4d0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.386+0000 7fd4e9a94640 1 -- 172.21.15.98:0/2759337613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4d00039c0 msgr2=0x7fd4d0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:03.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.386+0000 7fd4e9a94640 1 --2- 172.21.15.98:0/2759337613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4d00039c0 0x7fd4d0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.386+0000 7fd4e9a94640 1 -- 172.21.15.98:0/2759337613 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d0003530 con 0x7fd4d0004d50 2024-09-19T08:47:03.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.386+0000 7fd4e8a92640 1 --2- 172.21.15.98:0/2759337613 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd4e40cfad0 0x7fd4d0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:03.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.390+0000 7fd4e9a94640 1 --2- 172.21.15.98:0/2759337613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4d0004d50 0x7fd4d00051d0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc009fa0 tx=0x7fd4dc01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a914e0b68f0be51 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:03.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.390+0000 7fd4dbfff640 1 -- 172.21.15.98:0/2759337613 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc0b2070 con 0x7fd4d0004d50 2024-09-19T08:47:03.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.390+0000 7fd4dbfff640 1 -- 172.21.15.98:0/2759337613 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4dc0a5d10 con 0x7fd4d0004d50 2024-09-19T08:47:03.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.390+0000 7fd4dbfff640 1 -- 172.21.15.98:0/2759337613 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc009bf0 con 0x7fd4d0004d50 2024-09-19T08:47:03.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.390+0000 7fd4eb4f7640 1 -- 172.21.15.98:0/2759337613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4d0004d50 msgr2=0x7fd4d00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:03.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.390+0000 7fd4eb4f7640 1 --2- 172.21.15.98:0/2759337613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4d0004d50 0x7fd4d00051d0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc009fa0 tx=0x7fd4dc01cc40 comp rx=0 tx=0).stop 2024-09-19T08:47:03.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.390+0000 7fd4e9a94640 1 -- 172.21.15.98:0/2759337613 reap_dead start 2024-09-19T08:47:03.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.390+0000 7fd4eb4f7640 1 -- 172.21.15.98:0/2759337613 shutdown_connections 2024-09-19T08:47:03.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.390+0000 7fd4eb4f7640 1 -- 172.21.15.98:0/2759337613 >> 172.21.15.98:0/2759337613 conn(0x7fd4e40b2010 msgr2=0x7fd4e40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:03.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.390+0000 7fd4eb4f7640 1 -- 172.21.15.98:0/2759337613 shutdown_connections 2024-09-19T08:47:03.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.390+0000 7fd4eb4f7640 1 -- 172.21.15.98:0/2759337613 wait complete. 2024-09-19T08:47:03.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.390+0000 7fd4eb4f7640 1 Processor -- start 2024-09-19T08:47:03.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.390+0000 7fd4eb4f7640 1 -- start start 2024-09-19T08:47:03.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.390+0000 7fd4eb4f7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4d0004d50 0x7fd4e40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.390+0000 7fd4eb4f7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd4e40b4680 0x7fd4e414f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.390+0000 7fd4eb4f7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4e415ecd0 0x7fd4e415f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.390+0000 7fd4eb4f7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e415cab0 con 0x7fd4d0004d50 2024-09-19T08:47:03.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.390+0000 7fd4eb4f7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e415c930 con 0x7fd4e415ecd0 2024-09-19T08:47:03.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.390+0000 7fd4eb4f7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd4e40723a0 con 0x7fd4e40b4680 2024-09-19T08:47:03.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.390+0000 7fd4e9293640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4d0004d50 0x7fd4e40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.390+0000 7fd4e9293640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4d0004d50 0x7fd4e40b4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46358/0 (socket says 172.21.15.98:46358) 2024-09-19T08:47:03.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.390+0000 7fd4e9293640 1 -- 172.21.15.98:0/3397548862 learned_addr learned my addr 172.21.15.98:0/3397548862 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:03.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.390+0000 7fd4e8a92640 1 --2- 172.21.15.98:0/3397548862 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd4e40b4680 0x7fd4e414f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.390+0000 7fd4e9a94640 1 --2- 172.21.15.98:0/3397548862 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4e415ecd0 0x7fd4e415f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.390+0000 7fd4e9293640 1 -- 172.21.15.98:0/3397548862 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd4e40b4680 msgr2=0x7fd4e414f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:03.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.390+0000 7fd4e9293640 1 --2- 172.21.15.98:0/3397548862 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd4e40b4680 0x7fd4e414f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.390+0000 7fd4e9293640 1 -- 172.21.15.98:0/3397548862 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4e415ecd0 msgr2=0x7fd4e415f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:03.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.390+0000 7fd4e9293640 1 --2- 172.21.15.98:0/3397548862 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4e415ecd0 0x7fd4e415f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.390+0000 7fd4e9293640 1 -- 172.21.15.98:0/3397548862 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4e4153470 con 0x7fd4d0004d50 2024-09-19T08:47:03.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.390+0000 7fd4e9293640 1 --2- 172.21.15.98:0/3397548862 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4d0004d50 0x7fd4e40b4e10 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0011640 tx=0x7fd4e0011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:03.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.390+0000 7fd4d9ffb640 1 -- 172.21.15.98:0/3397548862 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e000f920 con 0x7fd4d0004d50 2024-09-19T08:47:03.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.394+0000 7fd4d9ffb640 1 -- 172.21.15.98:0/3397548862 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4e000fac0 con 0x7fd4d0004d50 2024-09-19T08:47:03.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.394+0000 7fd4eb4f7640 1 -- 172.21.15.98:0/3397548862 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4e40b35b0 con 0x7fd4d0004d50 2024-09-19T08:47:03.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.394+0000 7fd4d9ffb640 1 -- 172.21.15.98:0/3397548862 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e000fe70 con 0x7fd4d0004d50 2024-09-19T08:47:03.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.394+0000 7fd4eb4f7640 1 -- 172.21.15.98:0/3397548862 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4e40b3b30 con 0x7fd4d0004d50 2024-09-19T08:47:03.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.394+0000 7fd4eb4f7640 1 -- 172.21.15.98:0/3397548862 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4e415f690 con 0x7fd4d0004d50 2024-09-19T08:47:03.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.394+0000 7fd4d9ffb640 1 -- 172.21.15.98:0/3397548862 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7fd4e0015460 con 0x7fd4d0004d50 2024-09-19T08:47:03.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.394+0000 7fd4d9ffb640 1 --2- 172.21.15.98:0/3397548862 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd4c8068250 0x7fd4c806a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.394+0000 7fd4d9ffb640 1 -- 172.21.15.98:0/3397548862 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5737+0+0 (secure 0 0 0) 0x7fd4e0084db0 con 0x7fd4d0004d50 2024-09-19T08:47:03.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.394+0000 7fd4e8a92640 1 --2- 172.21.15.98:0/3397548862 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd4c8068250 0x7fd4c806a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.394+0000 7fd4e8a92640 1 --2- 172.21.15.98:0/3397548862 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd4c8068250 0x7fd4c806a710 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4001ec0 tx=0x7fd4d4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:03.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.398+0000 7fd4d9ffb640 1 -- 172.21.15.98:0/3397548862 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd4e0019030 con 0x7fd4d0004d50 2024-09-19T08:47:03.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.582+0000 7fd4eb4f7640 1 -- 172.21.15.98:0/3397548862 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4e406bc90 con 0x7fd4d0004d50 2024-09-19T08:47:03.583 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.582+0000 7fd4d9ffb640 1 -- 172.21.15.98:0/3397548862 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5233 (secure 0 0 0) 0x7fd4e00574f0 con 0x7fd4d0004d50 2024-09-19T08:47:03.583 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:47:03.584 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":92,"btime":"2024-09-19T08:46:56:581962+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/1752784745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":1752784745},{"type":"v1","addr":"172.21.15.139:6834","nonce":1752784745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":6212,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3564493643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6833","nonce":3564493643},{"type":"v1","addr":"172.21.15.139:6835","nonce":3564493643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":8763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6834/3374714194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3374714194},{"type":"v1","addr":"172.21.15.98:6834","nonce":3374714194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:46:39.868735+0000","modified":"2024-09-19T08:46:42.899681+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8766},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8766":{"gid":8766,"name":"c","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/3734544897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6833","nonce":3734544897},{"type":"v1","addr":"172.21.15.98:6835","nonce":3734544897}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8766,"qdb_cluster":[8766]},"id":12}]} 2024-09-19T08:47:03.584 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 92 2024-09-19T08:47:03.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.586+0000 7fd4eb4f7640 1 -- 172.21.15.98:0/3397548862 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd4c8068250 msgr2=0x7fd4c806a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:03.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.586+0000 7fd4eb4f7640 1 --2- 172.21.15.98:0/3397548862 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd4c8068250 0x7fd4c806a710 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4001ec0 tx=0x7fd4d4009040 comp rx=0 tx=0).stop 2024-09-19T08:47:03.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.586+0000 7fd4eb4f7640 1 -- 172.21.15.98:0/3397548862 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4d0004d50 msgr2=0x7fd4e40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:03.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.586+0000 7fd4eb4f7640 1 --2- 172.21.15.98:0/3397548862 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4d0004d50 0x7fd4e40b4e10 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0011640 tx=0x7fd4e0011b00 comp rx=0 tx=0).stop 2024-09-19T08:47:03.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.586+0000 7fd4eb4f7640 1 -- 172.21.15.98:0/3397548862 shutdown_connections 2024-09-19T08:47:03.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.586+0000 7fd4eb4f7640 1 --2- 172.21.15.98:0/3397548862 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd4c8068250 0x7fd4c806a710 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.586+0000 7fd4eb4f7640 1 --2- 172.21.15.98:0/3397548862 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd4e415ecd0 0x7fd4e415f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.586+0000 7fd4eb4f7640 1 --2- 172.21.15.98:0/3397548862 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd4e40b4680 0x7fd4e414f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.586+0000 7fd4eb4f7640 1 --2- 172.21.15.98:0/3397548862 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd4d0004d50 0x7fd4e40b4e10 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.586+0000 7fd4eb4f7640 1 -- 172.21.15.98:0/3397548862 >> 172.21.15.98:0/3397548862 conn(0x7fd4e40b2010 msgr2=0x7fd4e4073390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:03.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.586+0000 7fd4eb4f7640 1 -- 172.21.15.98:0/3397548862 shutdown_connections 2024-09-19T08:47:03.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.586+0000 7fd4eb4f7640 1 -- 172.21.15.98:0/3397548862 wait complete. 2024-09-19T08:47:03.601 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:47:03.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.714+0000 7fb98427e640 1 Processor -- start 2024-09-19T08:47:03.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.714+0000 7fb98427e640 1 -- start start 2024-09-19T08:47:03.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.714+0000 7fb98427e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb97c15c550 0x7fb97c06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.714+0000 7fb98427e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb97c0cfad0 0x7fb97c0723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.714+0000 7fb98427e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb97c06d310 con 0x7fb97c06c1d0 2024-09-19T08:47:03.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.714+0000 7fb98427e640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fb97c06c610 con 0x7fb97c0cfad0 2024-09-19T08:47:03.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.714+0000 7fb98427e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb97c06d5f0 con 0x7fb97c15c550 2024-09-19T08:47:03.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.714+0000 7fb98281b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb97c0cfad0 0x7fb97c0723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.714+0000 7fb98281b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fb97c0cfad0 0x7fb97c0723a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46368/0 (socket says 172.21.15.98:46368) 2024-09-19T08:47:03.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.714+0000 7fb98281b640 1 -- 172.21.15.98:0/2501036575 learned_addr learned my addr 172.21.15.98:0/2501036575 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:03.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.714+0000 7fb98281b640 1 -- 172.21.15.98:0/2501036575 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb97c15c550 msgr2=0x7fb97c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:03.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.714+0000 7fb98201a640 1 --2- 172.21.15.98:0/2501036575 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb97c15c550 0x7fb97c06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.714+0000 7fb98281b640 1 --2- 172.21.15.98:0/2501036575 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb97c15c550 0x7fb97c06bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.714+0000 7fb98281b640 1 -- 172.21.15.98:0/2501036575 >> v1:172.21.15.139:6789/0 conn(0x7fb97c06c1d0 legacy=0x7fb97c0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:03.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.714+0000 7fb98281b640 1 -- 172.21.15.98:0/2501036575 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb97c06d8d0 con 0x7fb97c0cfad0 2024-09-19T08:47:03.716 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.718+0000 7fb98281b640 1 --2- 172.21.15.98:0/2501036575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb97c0cfad0 0x7fb97c0723a0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fb974009e50 tx=0x7fb97401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=748caf4eae61d9a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:03.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.718+0000 7fb981018640 1 -- 172.21.15.98:0/2501036575 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9740a5780 con 0x7fb97c0cfad0 2024-09-19T08:47:03.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.718+0000 7fb981018640 1 -- 172.21.15.98:0/2501036575 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9740a5920 con 0x7fb97c0cfad0 2024-09-19T08:47:03.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.718+0000 7fb981018640 1 -- 172.21.15.98:0/2501036575 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9740a5cf0 con 0x7fb97c0cfad0 2024-09-19T08:47:03.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.718+0000 7fb98427e640 1 -- 172.21.15.98:0/2501036575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb97c0cfad0 msgr2=0x7fb97c0723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:03.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.718+0000 7fb98427e640 1 --2- 172.21.15.98:0/2501036575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb97c0cfad0 0x7fb97c0723a0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fb974009e50 tx=0x7fb97401c9f0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.718+0000 7fb98427e640 1 -- 172.21.15.98:0/2501036575 shutdown_connections 2024-09-19T08:47:03.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.718+0000 7fb98427e640 1 --2- 172.21.15.98:0/2501036575 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb97c0cfad0 0x7fb97c0723a0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.718+0000 7fb98427e640 1 --2- 172.21.15.98:0/2501036575 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb97c15c550 0x7fb97c06bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.718+0000 7fb98427e640 1 -- 172.21.15.98:0/2501036575 >> 172.21.15.98:0/2501036575 conn(0x7fb97c0b2010 msgr2=0x7fb97c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:03.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.718+0000 7fb98427e640 1 -- 172.21.15.98:0/2501036575 shutdown_connections 2024-09-19T08:47:03.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.718+0000 7fb98427e640 1 -- 172.21.15.98:0/2501036575 wait complete. 2024-09-19T08:47:03.718 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.718+0000 7fb98427e640 1 Processor -- start 2024-09-19T08:47:03.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.718+0000 7fb98427e640 1 -- start start 2024-09-19T08:47:03.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.718+0000 7fb98427e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb97c06c1d0 0x7fb97c15ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.718+0000 7fb98427e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb97c0cfad0 0x7fb97c0b1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.718+0000 7fb98427e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb97c15c550 0x7fb97c0b1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.718+0000 7fb98427e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c073480 con 0x7fb97c15c550 2024-09-19T08:47:03.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.718+0000 7fb98427e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c073300 con 0x7fb97c06c1d0 2024-09-19T08:47:03.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.718+0000 7fb98427e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb97c073600 con 0x7fb97c0cfad0 2024-09-19T08:47:03.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.718+0000 7fb98281b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb97c15c550 0x7fb97c0b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.718+0000 7fb98281b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb97c15c550 0x7fb97c0b1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46376/0 (socket says 172.21.15.98:46376) 2024-09-19T08:47:03.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.718+0000 7fb98281b640 1 -- 172.21.15.98:0/1584918123 learned_addr learned my addr 172.21.15.98:0/1584918123 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:03.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.718+0000 7fb98201a640 1 --2- 172.21.15.98:0/1584918123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb97c06c1d0 0x7fb97c15ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.721 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.718+0000 7fb981819640 1 --2- 172.21.15.98:0/1584918123 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb97c0cfad0 0x7fb97c0b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.722+0000 7fb98281b640 1 -- 172.21.15.98:0/1584918123 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb97c0cfad0 msgr2=0x7fb97c0b1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:03.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.722+0000 7fb98281b640 1 --2- 172.21.15.98:0/1584918123 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb97c0cfad0 0x7fb97c0b1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.722+0000 7fb98281b640 1 -- 172.21.15.98:0/1584918123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb97c06c1d0 msgr2=0x7fb97c15ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:03.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.722+0000 7fb98281b640 1 --2- 172.21.15.98:0/1584918123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb97c06c1d0 0x7fb97c15ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.722+0000 7fb98281b640 1 -- 172.21.15.98:0/1584918123 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb97c152df0 con 0x7fb97c15c550 2024-09-19T08:47:03.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.722+0000 7fb98281b640 1 --2- 172.21.15.98:0/1584918123 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb97c15c550 0x7fb97c0b1690 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fb9740af040 tx=0x7fb974019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:03.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.722+0000 7fb972ffd640 1 -- 172.21.15.98:0/1584918123 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9740bc020 con 0x7fb97c15c550 2024-09-19T08:47:03.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.722+0000 7fb972ffd640 1 -- 172.21.15.98:0/1584918123 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9740b8670 con 0x7fb97c15c550 2024-09-19T08:47:03.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.722+0000 7fb98427e640 1 -- 172.21.15.98:0/1584918123 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb97c15d430 con 0x7fb97c15c550 2024-09-19T08:47:03.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.722+0000 7fb972ffd640 1 -- 172.21.15.98:0/1584918123 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9740b8a60 con 0x7fb97c15c550 2024-09-19T08:47:03.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.722+0000 7fb98427e640 1 -- 172.21.15.98:0/1584918123 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb97c15d9f0 con 0x7fb97c15c550 2024-09-19T08:47:03.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.722+0000 7fb98427e640 1 -- 172.21.15.98:0/1584918123 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb97c0723a0 con 0x7fb97c15c550 2024-09-19T08:47:03.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.722+0000 7fb972ffd640 1 -- 172.21.15.98:0/1584918123 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7fb9740bf020 con 0x7fb97c15c550 2024-09-19T08:47:03.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.722+0000 7fb972ffd640 1 --2- 172.21.15.98:0/1584918123 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb960068250 0x7fb96006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:03.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.722+0000 7fb972ffd640 1 -- 172.21.15.98:0/1584918123 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5737+0+0 (secure 0 0 0) 0x7fb97411f600 con 0x7fb97c15c550 2024-09-19T08:47:03.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.722+0000 7fb98201a640 1 --2- 172.21.15.98:0/1584918123 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb960068250 0x7fb96006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:03.723 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.722+0000 7fb98201a640 1 --2- 172.21.15.98:0/1584918123 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb960068250 0x7fb96006a710 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb97c0b55f0 tx=0x7fb978008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:03.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.726+0000 7fb972ffd640 1 -- 172.21.15.98:0/1584918123 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb9740c6030 con 0x7fb97c15c550 2024-09-19T08:47:03.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.890+0000 7fb98427e640 1 -- 172.21.15.98:0/1584918123 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb97c0cd370 con 0x7fb97c15c550 2024-09-19T08:47:03.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.890+0000 7fb972ffd640 1 -- 172.21.15.98:0/1584918123 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) ==== 74+0+16783 (secure 0 0 0) 0x7fb9740165e0 con 0x7fb97c15c550 2024-09-19T08:47:03.891 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:47:03.891 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":158,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:46:41.262235+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:46:37.833554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-19T08:46:38.858268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:47:03.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.894+0000 7fb98427e640 1 -- 172.21.15.98:0/1584918123 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb960068250 msgr2=0x7fb96006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:03.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.894+0000 7fb98427e640 1 --2- 172.21.15.98:0/1584918123 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb960068250 0x7fb96006a710 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb97c0b55f0 tx=0x7fb978008040 comp rx=0 tx=0).stop 2024-09-19T08:47:03.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.894+0000 7fb98427e640 1 -- 172.21.15.98:0/1584918123 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb97c15c550 msgr2=0x7fb97c0b1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:03.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.894+0000 7fb98427e640 1 --2- 172.21.15.98:0/1584918123 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb97c15c550 0x7fb97c0b1690 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fb9740af040 tx=0x7fb974019790 comp rx=0 tx=0).stop 2024-09-19T08:47:03.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.894+0000 7fb98427e640 1 -- 172.21.15.98:0/1584918123 shutdown_connections 2024-09-19T08:47:03.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.894+0000 7fb98427e640 1 --2- 172.21.15.98:0/1584918123 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb960068250 0x7fb96006a710 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.894+0000 7fb98427e640 1 --2- 172.21.15.98:0/1584918123 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb97c15c550 0x7fb97c0b1690 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.894+0000 7fb98427e640 1 --2- 172.21.15.98:0/1584918123 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb97c0cfad0 0x7fb97c0b1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.894+0000 7fb98427e640 1 --2- 172.21.15.98:0/1584918123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb97c06c1d0 0x7fb97c15ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:03.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.894+0000 7fb98427e640 1 -- 172.21.15.98:0/1584918123 >> 172.21.15.98:0/1584918123 conn(0x7fb97c0b2010 msgr2=0x7fb97c1599c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:03.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.894+0000 7fb98427e640 1 -- 172.21.15.98:0/1584918123 shutdown_connections 2024-09-19T08:47:03.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:03.894+0000 7fb98427e640 1 -- 172.21.15.98:0/1584918123 wait complete. 2024-09-19T08:47:03.910 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-19T08:47:04.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.018+0000 7f1984bf3640 1 Processor -- start 2024-09-19T08:47:04.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.018+0000 7f1984bf3640 1 -- start start 2024-09-19T08:47:04.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.018+0000 7f1984bf3640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f198015c550 0x7f198006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:04.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.018+0000 7f1984bf3640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19800cfad0 0x7f19800723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:04.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.018+0000 7f1984bf3640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f198006d310 con 0x7f198006c1d0 2024-09-19T08:47:04.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.018+0000 7f1984bf3640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f198006c640 con 0x7f198015c550 2024-09-19T08:47:04.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.018+0000 7f1984bf3640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f198006d5f0 con 0x7f19800cfad0 2024-09-19T08:47:04.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.018+0000 7f197e59c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f198015c550 0x7f198006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:04.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.018+0000 7f197dd9b640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f198006c1d0 0x7f19800cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:32808/0 (socket says 172.21.15.98:32808) 2024-09-19T08:47:04.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.018+0000 7f197e59c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f198015c550 0x7f198006bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46392/0 (socket says 172.21.15.98:46392) 2024-09-19T08:47:04.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.018+0000 7f197dd9b640 1 -- 172.21.15.98:0/4176006829 learned_addr learned my addr 172.21.15.98:0/4176006829 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:04.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.018+0000 7f197e59c640 1 -- 172.21.15.98:0/4176006829 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19800cfad0 msgr2=0x7f19800723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:04.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.018+0000 7f197ed9d640 1 --2- 172.21.15.98:0/4176006829 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19800cfad0 0x7f19800723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:04.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.018+0000 7f197e59c640 1 --2- 172.21.15.98:0/4176006829 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19800cfad0 0x7f19800723a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.018+0000 7f197e59c640 1 -- 172.21.15.98:0/4176006829 >> v1:172.21.15.98:6789/0 conn(0x7f198006c1d0 legacy=0x7f19800cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:04.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.018+0000 7f197e59c640 1 -- 172.21.15.98:0/4176006829 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198006d8d0 con 0x7f198015c550 2024-09-19T08:47:04.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.022+0000 7f197e59c640 1 --2- 172.21.15.98:0/4176006829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f198015c550 0x7f198006bc90 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f19700097c0 tx=0x7f197001c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12270f23bc0a20ae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:04.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.022+0000 7f197d59a640 1 -- 172.21.15.98:0/4176006829 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19700b0070 con 0x7f198015c550 2024-09-19T08:47:04.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.022+0000 7f197d59a640 1 -- 172.21.15.98:0/4176006829 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19700ab600 con 0x7f198015c550 2024-09-19T08:47:04.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.022+0000 7f197d59a640 1 -- 172.21.15.98:0/4176006829 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19700ab9d0 con 0x7f198015c550 2024-09-19T08:47:04.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.022+0000 7f1984bf3640 1 -- 172.21.15.98:0/4176006829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f198015c550 msgr2=0x7f198006bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:04.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.022+0000 7f1984bf3640 1 --2- 172.21.15.98:0/4176006829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f198015c550 0x7f198006bc90 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f19700097c0 tx=0x7f197001c330 comp rx=0 tx=0).stop 2024-09-19T08:47:04.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.022+0000 7f1984bf3640 1 -- 172.21.15.98:0/4176006829 shutdown_connections 2024-09-19T08:47:04.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.022+0000 7f1984bf3640 1 --2- 172.21.15.98:0/4176006829 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19800cfad0 0x7f19800723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.022+0000 7f1984bf3640 1 --2- 172.21.15.98:0/4176006829 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f198015c550 0x7f198006bc90 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.022+0000 7f1984bf3640 1 -- 172.21.15.98:0/4176006829 >> 172.21.15.98:0/4176006829 conn(0x7f19800b2010 msgr2=0x7f19800b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:04.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.022+0000 7f1984bf3640 1 -- 172.21.15.98:0/4176006829 shutdown_connections 2024-09-19T08:47:04.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.022+0000 7f1984bf3640 1 -- 172.21.15.98:0/4176006829 wait complete. 2024-09-19T08:47:04.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.022+0000 7f1984bf3640 1 Processor -- start 2024-09-19T08:47:04.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.022+0000 7f1984bf3640 1 -- start start 2024-09-19T08:47:04.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.026+0000 7f1984bf3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f198006c1d0 0x7f198014ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:04.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.026+0000 7f1984bf3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19800cfad0 0x7f19801e63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:04.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.026+0000 7f1984bf3640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f198015c550 0x7f19800ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:04.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.026+0000 7f1984bf3640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f198015cab0 con 0x7f19800cfad0 2024-09-19T08:47:04.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.026+0000 7f1984bf3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f198015c930 con 0x7f198006c1d0 2024-09-19T08:47:04.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.026+0000 7f1984bf3640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f19800728e0 con 0x7f198015c550 2024-09-19T08:47:04.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.026+0000 7f197dd9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19800cfad0 0x7f19801e63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:04.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.026+0000 7f197dd9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19800cfad0 0x7f19801e63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46408/0 (socket says 172.21.15.98:46408) 2024-09-19T08:47:04.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.026+0000 7f197dd9b640 1 -- 172.21.15.98:0/913794823 learned_addr learned my addr 172.21.15.98:0/913794823 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:04.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.026+0000 7f197e59c640 1 --2- 172.21.15.98:0/913794823 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f198006c1d0 0x7f198014ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:04.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.026+0000 7f197ed9d640 1 --2- 172.21.15.98:0/913794823 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f198015c550 0x7f19800ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:04.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.026+0000 7f197dd9b640 1 -- 172.21.15.98:0/913794823 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f198015c550 msgr2=0x7f19800ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:04.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.026+0000 7f197dd9b640 1 --2- 172.21.15.98:0/913794823 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f198015c550 0x7f19800ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.026+0000 7f197dd9b640 1 -- 172.21.15.98:0/913794823 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f198006c1d0 msgr2=0x7f198014ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:04.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.026+0000 7f197dd9b640 1 --2- 172.21.15.98:0/913794823 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f198006c1d0 0x7f198014ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.026+0000 7f197dd9b640 1 -- 172.21.15.98:0/913794823 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198015fc00 con 0x7f19800cfad0 2024-09-19T08:47:04.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.026+0000 7f197dd9b640 1 --2- 172.21.15.98:0/913794823 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19800cfad0 0x7f19801e63d0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f1974009ef0 tx=0x7f197400c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:04.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.026+0000 7f196effd640 1 -- 172.21.15.98:0/913794823 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1974019070 con 0x7f19800cfad0 2024-09-19T08:47:04.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.026+0000 7f196effd640 1 -- 172.21.15.98:0/913794823 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1974007af0 con 0x7f19800cfad0 2024-09-19T08:47:04.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.026+0000 7f1984bf3640 1 -- 172.21.15.98:0/913794823 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19801e6910 con 0x7f19800cfad0 2024-09-19T08:47:04.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.026+0000 7f196effd640 1 -- 172.21.15.98:0/913794823 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197400ce10 con 0x7f19800cfad0 2024-09-19T08:47:04.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.026+0000 7f1984bf3640 1 -- 172.21.15.98:0/913794823 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19801e6eb0 con 0x7f19800cfad0 2024-09-19T08:47:04.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.026+0000 7f1984bf3640 1 -- 172.21.15.98:0/913794823 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19800723a0 con 0x7f19800cfad0 2024-09-19T08:47:04.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.026+0000 7f196effd640 1 -- 172.21.15.98:0/913794823 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f197401c070 con 0x7f19800cfad0 2024-09-19T08:47:04.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.026+0000 7f196effd640 1 --2- 172.21.15.98:0/913794823 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1960068250 0x7f196006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:04.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.026+0000 7f196effd640 1 -- 172.21.15.98:0/913794823 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5737+0+0 (secure 0 0 0) 0x7f1974086050 con 0x7f19800cfad0 2024-09-19T08:47:04.032 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.026+0000 7f197e59c640 1 --2- 172.21.15.98:0/913794823 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1960068250 0x7f196006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:04.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.030+0000 7f197e59c640 1 --2- 172.21.15.98:0/913794823 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1960068250 0x7f196006a710 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f19700ad000 tx=0x7f1970008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:04.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.030+0000 7f196effd640 1 -- 172.21.15.98:0/913794823 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f1974058140 con 0x7f19800cfad0 2024-09-19T08:47:04.218 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.218+0000 7f1984bf3640 1 -- 172.21.15.98:0/913794823 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f19800757a0 con 0x7f19800cfad0 2024-09-19T08:47:04.219 INFO:tasks.ceph.mon.a.smithi098.stderr:2024-09-19T08:47:04.218+0000 7f5e49017640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T08:47:04.221 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.222+0000 7f196effd640 1 -- 172.21.15.98:0/913794823 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v93) ==== 161+0+0 (secure 0 0 0) 0x7f197405c440 con 0x7f19800cfad0 2024-09-19T08:47:04.221 INFO:teuthology.orchestra.run.smithi098.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T08:47:04.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.222+0000 7f1984bf3640 1 -- 172.21.15.98:0/913794823 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1960068250 msgr2=0x7f196006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:04.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.222+0000 7f1984bf3640 1 --2- 172.21.15.98:0/913794823 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1960068250 0x7f196006a710 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f19700ad000 tx=0x7f1970008040 comp rx=0 tx=0).stop 2024-09-19T08:47:04.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.222+0000 7f1984bf3640 1 -- 172.21.15.98:0/913794823 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19800cfad0 msgr2=0x7f19801e63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:04.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.222+0000 7f1984bf3640 1 --2- 172.21.15.98:0/913794823 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19800cfad0 0x7f19801e63d0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f1974009ef0 tx=0x7f197400c430 comp rx=0 tx=0).stop 2024-09-19T08:47:04.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.226+0000 7f1984bf3640 1 -- 172.21.15.98:0/913794823 shutdown_connections 2024-09-19T08:47:04.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.226+0000 7f1984bf3640 1 --2- 172.21.15.98:0/913794823 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1960068250 0x7f196006a710 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.226+0000 7f1984bf3640 1 --2- 172.21.15.98:0/913794823 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f198015c550 0x7f19800ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.225 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.226+0000 7f1984bf3640 1 --2- 172.21.15.98:0/913794823 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19800cfad0 0x7f19801e63d0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.226+0000 7f1984bf3640 1 --2- 172.21.15.98:0/913794823 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f198006c1d0 0x7f198014ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.226+0000 7f1984bf3640 1 -- 172.21.15.98:0/913794823 >> 172.21.15.98:0/913794823 conn(0x7f19800b2010 msgr2=0x7f19800afe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:04.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.226+0000 7f1984bf3640 1 -- 172.21.15.98:0/913794823 shutdown_connections 2024-09-19T08:47:04.226 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.226+0000 7f1984bf3640 1 -- 172.21.15.98:0/913794823 wait complete. 2024-09-19T08:47:04.244 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T08:47:04.263 INFO:tasks.ceph.mds.c.smithi098.stdout:starting mds.c at 2024-09-19T08:47:04.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.354+0000 7f902fae5640 1 Processor -- start 2024-09-19T08:47:04.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.354+0000 7f902fae5640 1 -- start start 2024-09-19T08:47:04.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.358+0000 7f902fae5640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f90280cea90 0x7f90280cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:04.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.358+0000 7f902fae5640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f902815ebe0 0x7f9028165610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:04.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.358+0000 7f902fae5640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90280b4320 con 0x7f902815b7e0 2024-09-19T08:47:04.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.358+0000 7f902fae5640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f90280cd600 con 0x7f90280cea90 2024-09-19T08:47:04.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.358+0000 7f902fae5640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f9028158870 con 0x7f902815ebe0 2024-09-19T08:47:04.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.358+0000 7f902d881640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f90280cea90 0x7f90280cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:04.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.358+0000 7f902d080640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f902815b7e0 0x7f902815e4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:32824/0 (socket says 172.21.15.98:32824) 2024-09-19T08:47:04.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.358+0000 7f902d881640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f90280cea90 0x7f90280cee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46428/0 (socket says 172.21.15.98:46428) 2024-09-19T08:47:04.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.358+0000 7f902d080640 1 -- 172.21.15.98:0/652763651 learned_addr learned my addr 172.21.15.98:0/652763651 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:04.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.358+0000 7f902e082640 1 --2- 172.21.15.98:0/652763651 >> v2:172.21.15.139:3300/0 conn(0x7f902815ebe0 0x7f9028165610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:04.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.358+0000 7f902d881640 1 -- 172.21.15.98:0/652763651 >> v2:172.21.15.139:3300/0 conn(0x7f902815ebe0 msgr2=0x7f9028165610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:04.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.358+0000 7f902d881640 1 --2- 172.21.15.98:0/652763651 >> v2:172.21.15.139:3300/0 conn(0x7f902815ebe0 0x7f9028165610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.358+0000 7f902d881640 1 -- 172.21.15.98:0/652763651 >> v1:172.21.15.98:6789/0 conn(0x7f902815b7e0 legacy=0x7f902815e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:04.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.358+0000 7f902d881640 1 -- 172.21.15.98:0/652763651 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90280ccd10 con 0x7f90280cea90 2024-09-19T08:47:04.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.358+0000 7f902c87f640 1 -- 172.21.15.98:0/652763651 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1896416 0 0) 0x7f9018001120 con 0x7f902815b7e0 2024-09-19T08:47:04.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.358+0000 7f902d881640 1 --2- 172.21.15.98:0/652763651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90280cea90 0x7f90280cee70 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f90280b5320 tx=0x7f902401eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc1e23e90d265dc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:04.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.358+0000 7f902c87f640 1 -- 172.21.15.98:0/652763651 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90240b4070 con 0x7f90280cea90 2024-09-19T08:47:04.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.358+0000 7f902c87f640 1 -- 172.21.15.98:0/652763651 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90240a7c20 con 0x7f90280cea90 2024-09-19T08:47:04.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.358+0000 7f902c87f640 1 -- 172.21.15.98:0/652763651 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90240af440 con 0x7f90280cea90 2024-09-19T08:47:04.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.358+0000 7f902fae5640 1 -- 172.21.15.98:0/652763651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90280cea90 msgr2=0x7f90280cee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:04.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.358+0000 7f902fae5640 1 --2- 172.21.15.98:0/652763651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90280cea90 0x7f90280cee70 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f90280b5320 tx=0x7f902401eb50 comp rx=0 tx=0).stop 2024-09-19T08:47:04.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.358+0000 7f902fae5640 1 -- 172.21.15.98:0/652763651 shutdown_connections 2024-09-19T08:47:04.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.358+0000 7f902fae5640 1 --2- 172.21.15.98:0/652763651 >> v2:172.21.15.139:3300/0 conn(0x7f902815ebe0 0x7f9028165610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.358+0000 7f902fae5640 1 --2- 172.21.15.98:0/652763651 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f90280cea90 0x7f90280cee70 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.358+0000 7f902fae5640 1 -- 172.21.15.98:0/652763651 >> 172.21.15.98:0/652763651 conn(0x7f90280b2010 msgr2=0x7f90280b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:04.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.358+0000 7f902fae5640 1 -- 172.21.15.98:0/652763651 shutdown_connections 2024-09-19T08:47:04.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.358+0000 7f902fae5640 1 -- 172.21.15.98:0/652763651 wait complete. 2024-09-19T08:47:04.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.362+0000 7f902fae5640 1 Processor -- start 2024-09-19T08:47:04.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.362+0000 7f902fae5640 1 -- start start 2024-09-19T08:47:04.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.362+0000 7f902fae5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f90280cea90 0x7f9028147c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:04.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.362+0000 7f902fae5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f902815b7e0 0x7f9028153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:04.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.362+0000 7f902fae5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f902815ebe0 0x7f90281542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:04.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.362+0000 7f902fae5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f90280ce4c0 con 0x7f902815ebe0 2024-09-19T08:47:04.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.362+0000 7f902fae5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f902806cba0 con 0x7f902815b7e0 2024-09-19T08:47:04.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.362+0000 7f902fae5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f90280cda90 con 0x7f90280cea90 2024-09-19T08:47:04.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.362+0000 7f902e082640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f902815ebe0 0x7f90281542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:04.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.362+0000 7f902e082640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f902815ebe0 0x7f90281542a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46440/0 (socket says 172.21.15.98:46440) 2024-09-19T08:47:04.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.362+0000 7f902e082640 1 -- 172.21.15.98:0/638724551 learned_addr learned my addr 172.21.15.98:0/638724551 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:04.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.362+0000 7f902d881640 1 --2- 172.21.15.98:0/638724551 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f90280cea90 0x7f9028147c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:04.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.362+0000 7f902d080640 1 --2- 172.21.15.98:0/638724551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f902815b7e0 0x7f9028153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:04.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.362+0000 7f902e082640 1 -- 172.21.15.98:0/638724551 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f90280cea90 msgr2=0x7f9028147c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:04.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.362+0000 7f902e082640 1 --2- 172.21.15.98:0/638724551 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f90280cea90 0x7f9028147c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.362+0000 7f902e082640 1 -- 172.21.15.98:0/638724551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f902815b7e0 msgr2=0x7f9028153d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:04.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.362+0000 7f902e082640 1 --2- 172.21.15.98:0/638724551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f902815b7e0 0x7f9028153d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:04.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.362+0000 7f902e082640 1 -- 172.21.15.98:0/638724551 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9028156400 con 0x7f902815ebe0 2024-09-19T08:47:04.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.362+0000 7f902d881640 1 --2- 172.21.15.98:0/638724551 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f90280cea90 0x7f9028147c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:04.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.362+0000 7f902d080640 1 --2- 172.21.15.98:0/638724551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f902815b7e0 0x7f9028153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:04.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.362+0000 7f902e082640 1 --2- 172.21.15.98:0/638724551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f902815ebe0 0x7f90281542a0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f9020007e70 tx=0x7f902000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:04.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.362+0000 7f901e7fc640 1 -- 172.21.15.98:0/638724551 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9020014420 con 0x7f902815ebe0 2024-09-19T08:47:04.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.362+0000 7f901e7fc640 1 -- 172.21.15.98:0/638724551 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9020019070 con 0x7f902815ebe0 2024-09-19T08:47:04.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.362+0000 7f901e7fc640 1 -- 172.21.15.98:0/638724551 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9020014820 con 0x7f902815ebe0 2024-09-19T08:47:04.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.362+0000 7f902fae5640 1 -- 172.21.15.98:0/638724551 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90281566e0 con 0x7f902815ebe0 2024-09-19T08:47:04.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.362+0000 7f902fae5640 1 -- 172.21.15.98:0/638724551 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9028156ca0 con 0x7f902815ebe0 2024-09-19T08:47:04.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.362+0000 7f902fae5640 1 -- 172.21.15.98:0/638724551 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f902806c730 con 0x7f902815ebe0 2024-09-19T08:47:04.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.366+0000 7f901e7fc640 1 -- 172.21.15.98:0/638724551 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f90200149c0 con 0x7f902815ebe0 2024-09-19T08:47:04.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.366+0000 7f901e7fc640 1 --2- 172.21.15.98:0/638724551 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f900c068250 0x7f900c06a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:04.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.366+0000 7f901e7fc640 1 -- 172.21.15.98:0/638724551 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5881+0+0 (secure 0 0 0) 0x7f9020084c40 con 0x7f902815ebe0 2024-09-19T08:47:04.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.366+0000 7f902d881640 1 --2- 172.21.15.98:0/638724551 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f900c068250 0x7f900c06a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:04.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.366+0000 7f902d881640 1 --2- 172.21.15.98:0/638724551 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f900c068250 0x7f900c06a710 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f90240b1000 tx=0x7f90240a9070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:04.369 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.370+0000 7f901e7fc640 1 -- 172.21.15.98:0/638724551 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9020011260 con 0x7f902815ebe0 2024-09-19T08:47:04.560 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:04.562+0000 7f902fae5640 1 -- 172.21.15.98:0/638724551 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f9028151c90 con 0x7f902815ebe0 2024-09-19T08:47:05.224 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.222+0000 7f901e7fc640 1 -- 172.21.15.98:0/638724551 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v94) ==== 104+0+0 (secure 0 0 0) 0x7f9020057380 con 0x7f902815ebe0 2024-09-19T08:47:05.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.226+0000 7f902fae5640 1 -- 172.21.15.98:0/638724551 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f900c068250 msgr2=0x7f900c06a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:05.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.226+0000 7f902fae5640 1 --2- 172.21.15.98:0/638724551 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f900c068250 0x7f900c06a710 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f90240b1000 tx=0x7f90240a9070 comp rx=0 tx=0).stop 2024-09-19T08:47:05.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.226+0000 7f902fae5640 1 -- 172.21.15.98:0/638724551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f902815ebe0 msgr2=0x7f90281542a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:05.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.226+0000 7f902fae5640 1 --2- 172.21.15.98:0/638724551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f902815ebe0 0x7f90281542a0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f9020007e70 tx=0x7f902000e4b0 comp rx=0 tx=0).stop 2024-09-19T08:47:05.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.226+0000 7f902fae5640 1 -- 172.21.15.98:0/638724551 shutdown_connections 2024-09-19T08:47:05.227 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.226+0000 7f902fae5640 1 --2- 172.21.15.98:0/638724551 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f900c068250 0x7f900c06a710 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:05.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.226+0000 7f902fae5640 1 --2- 172.21.15.98:0/638724551 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f902815ebe0 0x7f90281542a0 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:05.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.226+0000 7f902fae5640 1 --2- 172.21.15.98:0/638724551 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f902815b7e0 0x7f9028153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:05.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.226+0000 7f902fae5640 1 --2- 172.21.15.98:0/638724551 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f90280cea90 0x7f9028147c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:05.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.226+0000 7f902fae5640 1 -- 172.21.15.98:0/638724551 >> 172.21.15.98:0/638724551 conn(0x7f90280b2010 msgr2=0x7f90281521b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:05.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.226+0000 7f902fae5640 1 -- 172.21.15.98:0/638724551 shutdown_connections 2024-09-19T08:47:05.228 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.226+0000 7f902fae5640 1 -- 172.21.15.98:0/638724551 wait complete. 2024-09-19T08:47:05.252 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T08:47:05.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.426+0000 7f9ec4ed7640 1 Processor -- start 2024-09-19T08:47:05.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.426+0000 7f9ec4ed7640 1 -- start start 2024-09-19T08:47:05.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.426+0000 7f9ec4ed7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ec015a930 0x7f9ec00b3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:05.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.426+0000 7f9ec4ed7640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f9ec00b63d0 0x7f9ec00cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:05.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.426+0000 7f9ec4ed7640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ec00b5110 con 0x7f9ec00b3b60 2024-09-19T08:47:05.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.426+0000 7f9ec4ed7640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f9ec00b4070 con 0x7f9ec00b63d0 2024-09-19T08:47:05.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.426+0000 7f9ec4ed7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9ec00b4350 con 0x7f9ec015a930 2024-09-19T08:47:05.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.426+0000 7f9ebdd9b640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f9ec00b3b60 0x7f9ec00b5cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:32840/0 (socket says 172.21.15.98:32840) 2024-09-19T08:47:05.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.426+0000 7f9ebdd9b640 1 -- 172.21.15.98:0/1496435499 learned_addr learned my addr 172.21.15.98:0/1496435499 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:05.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.430+0000 7f9ebed9d640 1 --2- 172.21.15.98:0/1496435499 >> v2:172.21.15.139:3300/0 conn(0x7f9ec00b63d0 0x7f9ec00cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:05.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.430+0000 7f9ebe59c640 1 --2- 172.21.15.98:0/1496435499 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ec015a930 0x7f9ec00b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:05.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.430+0000 7f9ebd59a640 1 -- 172.21.15.98:0/1496435499 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3040480477 0 0) 0x7f9ec00b5110 con 0x7f9ec00b3b60 2024-09-19T08:47:05.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.430+0000 7f9ebd59a640 1 -- 172.21.15.98:0/1496435499 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9eac003530 con 0x7f9ec00b3b60 2024-09-19T08:47:05.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.430+0000 7f9ebd59a640 1 -- 172.21.15.98:0/1496435499 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2988052709 0 0) 0x7f9eac003530 con 0x7f9ec00b3b60 2024-09-19T08:47:05.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.430+0000 7f9ebd59a640 1 -- 172.21.15.98:0/1496435499 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ec015a930 msgr2=0x7f9ec00b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:05.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.430+0000 7f9ebd59a640 1 --2- 172.21.15.98:0/1496435499 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ec015a930 0x7f9ec00b3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:05.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.430+0000 7f9ebd59a640 1 -- 172.21.15.98:0/1496435499 >> v2:172.21.15.139:3300/0 conn(0x7f9ec00b63d0 msgr2=0x7f9ec00cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:05.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.430+0000 7f9ebd59a640 1 --2- 172.21.15.98:0/1496435499 >> v2:172.21.15.139:3300/0 conn(0x7f9ec00b63d0 0x7f9ec00cdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:05.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.430+0000 7f9ebd59a640 1 -- 172.21.15.98:0/1496435499 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec00b4630 con 0x7f9ec00b3b60 2024-09-19T08:47:05.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.430+0000 7f9ebd59a640 1 -- 172.21.15.98:0/1496435499 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f9ea8001680 con 0x7f9ec00b3b60 2024-09-19T08:47:05.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.430+0000 7f9ebd59a640 1 -- 172.21.15.98:0/1496435499 >> v1:172.21.15.98:6789/0 conn(0x7f9ec00b3b60 legacy=0x7f9ec00b5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:05.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.430+0000 7f9ebd59a640 1 --2- 172.21.15.98:0/1496435499 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9eac0039c0 0x7f9eac003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:05.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.430+0000 7f9ebd59a640 1 --2- 172.21.15.98:0/1496435499 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ec015a930 0x7f9eac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:05.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.430+0000 7f9ebd59a640 1 --2- 172.21.15.98:0/1496435499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9eac004d50 0x7f9eac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:05.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.430+0000 7f9ebd59a640 1 -- 172.21.15.98:0/1496435499 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec00b4070 con 0x7f9eac004d50 2024-09-19T08:47:05.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.430+0000 7f9ebd59a640 1 -- 172.21.15.98:0/1496435499 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec00b4350 con 0x7f9eac0039c0 2024-09-19T08:47:05.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.430+0000 7f9ebd59a640 1 -- 172.21.15.98:0/1496435499 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9eac00b9d0 con 0x7f9ec015a930 2024-09-19T08:47:05.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.430+0000 7f9ebed9d640 1 --2- 172.21.15.98:0/1496435499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9eac004d50 0x7f9eac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:05.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.430+0000 7f9ebdd9b640 1 --2- 172.21.15.98:0/1496435499 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ec015a930 0x7f9eac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:05.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.430+0000 7f9ebe59c640 1 --2- 172.21.15.98:0/1496435499 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9eac0039c0 0x7f9eac003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:05.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.430+0000 7f9ebed9d640 1 -- 172.21.15.98:0/1496435499 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ec015a930 msgr2=0x7f9eac004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:05.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.430+0000 7f9ebed9d640 1 --2- 172.21.15.98:0/1496435499 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ec015a930 0x7f9eac004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:05.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.430+0000 7f9ebed9d640 1 -- 172.21.15.98:0/1496435499 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9eac0039c0 msgr2=0x7f9eac003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:05.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.430+0000 7f9ebed9d640 1 --2- 172.21.15.98:0/1496435499 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9eac0039c0 0x7f9eac003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:05.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.430+0000 7f9ebed9d640 1 -- 172.21.15.98:0/1496435499 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eac003530 con 0x7f9eac004d50 2024-09-19T08:47:05.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.430+0000 7f9ebed9d640 1 --2- 172.21.15.98:0/1496435499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9eac004d50 0x7f9eac0051d0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f9ec00b4c30 tx=0x7f9eb801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=636d913f4f00e771 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:05.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.430+0000 7f9ebd59a640 1 -- 172.21.15.98:0/1496435499 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb80b2070 con 0x7f9eac004d50 2024-09-19T08:47:05.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.430+0000 7f9ebd59a640 1 -- 172.21.15.98:0/1496435499 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9eb80a5530 con 0x7f9eac004d50 2024-09-19T08:47:05.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.430+0000 7f9ebd59a640 1 -- 172.21.15.98:0/1496435499 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb80096b0 con 0x7f9eac004d50 2024-09-19T08:47:05.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.434+0000 7f9ec4ed7640 1 -- 172.21.15.98:0/1496435499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9eac004d50 msgr2=0x7f9eac0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:05.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.434+0000 7f9ec4ed7640 1 --2- 172.21.15.98:0/1496435499 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9eac004d50 0x7f9eac0051d0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f9ec00b4c30 tx=0x7f9eb801ca50 comp rx=0 tx=0).stop 2024-09-19T08:47:05.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.434+0000 7f9ebed9d640 1 -- 172.21.15.98:0/1496435499 reap_dead start 2024-09-19T08:47:05.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.434+0000 7f9ec4ed7640 1 -- 172.21.15.98:0/1496435499 shutdown_connections 2024-09-19T08:47:05.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.434+0000 7f9ec4ed7640 1 -- 172.21.15.98:0/1496435499 >> 172.21.15.98:0/1496435499 conn(0x7f9ec006c090 msgr2=0x7f9ec006c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:05.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.434+0000 7f9ec4ed7640 1 -- 172.21.15.98:0/1496435499 shutdown_connections 2024-09-19T08:47:05.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.434+0000 7f9ec4ed7640 1 -- 172.21.15.98:0/1496435499 wait complete. 2024-09-19T08:47:05.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.434+0000 7f9ec4ed7640 1 Processor -- start 2024-09-19T08:47:05.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.434+0000 7f9ec4ed7640 1 -- start start 2024-09-19T08:47:05.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.434+0000 7f9ec4ed7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ec014a530 0x7f9ec0071f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:05.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.434+0000 7f9ec4ed7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ec014ae60 0x7f9ec014de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:05.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.434+0000 7f9ec4ed7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ec014a910 0x7f9ec015d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:05.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.434+0000 7f9ec4ed7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec015ad10 con 0x7f9ec014a910 2024-09-19T08:47:05.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.434+0000 7f9ec4ed7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec00724c0 con 0x7f9ec014ae60 2024-09-19T08:47:05.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.434+0000 7f9ec4ed7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9ec015ae90 con 0x7f9ec014a530 2024-09-19T08:47:05.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.434+0000 7f9ebed9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ec014a910 0x7f9ec015d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:05.440 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.434+0000 7f9ebed9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ec014a910 0x7f9ec015d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46444/0 (socket says 172.21.15.98:46444) 2024-09-19T08:47:05.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.434+0000 7f9ebed9d640 1 -- 172.21.15.98:0/3983087564 learned_addr learned my addr 172.21.15.98:0/3983087564 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:05.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.434+0000 7f9ebe59c640 1 --2- 172.21.15.98:0/3983087564 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ec014a530 0x7f9ec0071f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:05.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.434+0000 7f9ebdd9b640 1 --2- 172.21.15.98:0/3983087564 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ec014ae60 0x7f9ec014de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:05.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.434+0000 7f9ebed9d640 1 -- 172.21.15.98:0/3983087564 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ec014a530 msgr2=0x7f9ec0071f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:05.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.434+0000 7f9ebed9d640 1 --2- 172.21.15.98:0/3983087564 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ec014a530 0x7f9ec0071f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:05.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.434+0000 7f9ebed9d640 1 -- 172.21.15.98:0/3983087564 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ec014ae60 msgr2=0x7f9ec014de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:05.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.434+0000 7f9ebed9d640 1 --2- 172.21.15.98:0/3983087564 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ec014ae60 0x7f9ec014de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:05.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.434+0000 7f9ebed9d640 1 -- 172.21.15.98:0/3983087564 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec00af9c0 con 0x7f9ec014a910 2024-09-19T08:47:05.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.434+0000 7f9ebed9d640 1 --2- 172.21.15.98:0/3983087564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ec014a910 0x7f9ec015d220 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f9eb80b0040 tx=0x7f9eac005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:05.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.434+0000 7f9eb6ffd640 1 -- 172.21.15.98:0/3983087564 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb80b2040 con 0x7f9ec014a910 2024-09-19T08:47:05.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.434+0000 7f9eb6ffd640 1 -- 172.21.15.98:0/3983087564 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9eb80bb070 con 0x7f9ec014a910 2024-09-19T08:47:05.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.434+0000 7f9ec4ed7640 1 -- 172.21.15.98:0/3983087564 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ec0150a10 con 0x7f9ec014a910 2024-09-19T08:47:05.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.434+0000 7f9eb6ffd640 1 -- 172.21.15.98:0/3983087564 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb80096b0 con 0x7f9ec014a910 2024-09-19T08:47:05.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.434+0000 7f9ec4ed7640 1 -- 172.21.15.98:0/3983087564 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ec0150730 con 0x7f9ec014a910 2024-09-19T08:47:05.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.438+0000 7f9ec4ed7640 1 -- 172.21.15.98:0/3983087564 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ec00b3620 con 0x7f9ec014a910 2024-09-19T08:47:05.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.438+0000 7f9eb6ffd640 1 -- 172.21.15.98:0/3983087564 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f9eb80c0460 con 0x7f9ec014a910 2024-09-19T08:47:05.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.438+0000 7f9eb6ffd640 1 --2- 172.21.15.98:0/3983087564 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9ea0068280 0x7f9ea006a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:05.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.438+0000 7f9eb6ffd640 1 -- 172.21.15.98:0/3983087564 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 5881+0+0 (secure 0 0 0) 0x7f9eb811db50 con 0x7f9ec014a910 2024-09-19T08:47:05.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.438+0000 7f9ebe59c640 1 --2- 172.21.15.98:0/3983087564 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9ea0068280 0x7f9ea006a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:05.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.438+0000 7f9ebe59c640 1 --2- 172.21.15.98:0/3983087564 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9ea0068280 0x7f9ea006a740 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0009fd0 tx=0x7f9eb000b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:05.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.442+0000 7f9eb6ffd640 1 -- 172.21.15.98:0/3983087564 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9eb8016760 con 0x7f9ec014a910 2024-09-19T08:47:05.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:05.594+0000 7f9ec4ed7640 1 -- 172.21.15.98:0/3983087564 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9ec0070fd0 con 0x7f9ec014a910 2024-09-19T08:47:06.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.234+0000 7f9eb6ffd640 1 -- 172.21.15.98:0/3983087564 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v161) ==== 181+0+0 (secure 0 0 0) 0x7f9eb80f0200 con 0x7f9ec014a910 2024-09-19T08:47:06.234 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_metadata' removed 2024-09-19T08:47:06.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.238+0000 7f9ec4ed7640 1 -- 172.21.15.98:0/3983087564 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9ea0068280 msgr2=0x7f9ea006a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:06.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.238+0000 7f9ec4ed7640 1 --2- 172.21.15.98:0/3983087564 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9ea0068280 0x7f9ea006a740 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0009fd0 tx=0x7f9eb000b040 comp rx=0 tx=0).stop 2024-09-19T08:47:06.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.238+0000 7f9ec4ed7640 1 -- 172.21.15.98:0/3983087564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ec014a910 msgr2=0x7f9ec015d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:06.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.238+0000 7f9ec4ed7640 1 --2- 172.21.15.98:0/3983087564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ec014a910 0x7f9ec015d220 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f9eb80b0040 tx=0x7f9eac005840 comp rx=0 tx=0).stop 2024-09-19T08:47:06.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.238+0000 7f9ec4ed7640 1 -- 172.21.15.98:0/3983087564 shutdown_connections 2024-09-19T08:47:06.239 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.238+0000 7f9ec4ed7640 1 --2- 172.21.15.98:0/3983087564 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9ea0068280 0x7f9ea006a740 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:06.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.238+0000 7f9ec4ed7640 1 --2- 172.21.15.98:0/3983087564 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9ec014a910 0x7f9ec015d220 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:06.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.238+0000 7f9ec4ed7640 1 --2- 172.21.15.98:0/3983087564 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ec014ae60 0x7f9ec014de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:06.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.238+0000 7f9ec4ed7640 1 --2- 172.21.15.98:0/3983087564 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9ec014a530 0x7f9ec0071f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:06.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.238+0000 7f9ec4ed7640 1 -- 172.21.15.98:0/3983087564 >> 172.21.15.98:0/3983087564 conn(0x7f9ec006c090 msgr2=0x7f9ec0151bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:06.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.242+0000 7f9ec4ed7640 1 -- 172.21.15.98:0/3983087564 shutdown_connections 2024-09-19T08:47:06.240 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.242+0000 7f9ec4ed7640 1 -- 172.21.15.98:0/3983087564 wait complete. 2024-09-19T08:47:06.260 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T08:47:06.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.386+0000 7f504c3fe640 1 Processor -- start 2024-09-19T08:47:06.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.386+0000 7f504c3fe640 1 -- start start 2024-09-19T08:47:06.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.386+0000 7f504c3fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50440cf700 0x7f504415ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:06.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.386+0000 7f504c3fe640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50441608c0 con 0x7f504406d4c0 2024-09-19T08:47:06.389 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.386+0000 7f504c3fe640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5044160ba0 con 0x7f504415c290 2024-09-19T08:47:06.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.386+0000 7f504c3fe640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5044160e80 con 0x7f50440cf700 2024-09-19T08:47:06.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.386+0000 7f504a99b640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f504406d4c0 0x7f50440b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:32850/0 (socket says 172.21.15.98:32850) 2024-09-19T08:47:06.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.386+0000 7f504a99b640 1 -- 172.21.15.98:0/4157630155 learned_addr learned my addr 172.21.15.98:0/4157630155 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:06.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.386+0000 7f5049999640 1 --2- 172.21.15.98:0/4157630155 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50440cf700 0x7f504415ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:06.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.386+0000 7f5049198640 1 -- 172.21.15.98:0/4157630155 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2345661139 0 0) 0x7f50441608c0 con 0x7f504406d4c0 2024-09-19T08:47:06.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.386+0000 7f5049198640 1 -- 172.21.15.98:0/4157630155 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5030003530 con 0x7f504406d4c0 2024-09-19T08:47:06.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.386+0000 7f5049198640 1 -- 172.21.15.98:0/4157630155 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3801736903 0 0) 0x7f5030003530 con 0x7f504406d4c0 2024-09-19T08:47:06.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.386+0000 7f5049198640 1 -- 172.21.15.98:0/4157630155 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50440cf700 msgr2=0x7f504415ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:06.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.386+0000 7f5049198640 1 --2- 172.21.15.98:0/4157630155 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50440cf700 0x7f504415ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:06.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.386+0000 7f5049198640 1 -- 172.21.15.98:0/4157630155 >> v1:172.21.15.139:6789/0 conn(0x7f504415c290 legacy=0x7f50440cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:06.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.386+0000 7f5049198640 1 -- 172.21.15.98:0/4157630155 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50440cd1a0 con 0x7f504406d4c0 2024-09-19T08:47:06.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.386+0000 7f5049198640 1 -- 172.21.15.98:0/4157630155 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f503c001680 con 0x7f504406d4c0 2024-09-19T08:47:06.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.386+0000 7f5049198640 1 -- 172.21.15.98:0/4157630155 >> v1:172.21.15.98:6789/0 conn(0x7f504406d4c0 legacy=0x7f50440b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:06.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.390+0000 7f5049198640 1 --2- 172.21.15.98:0/4157630155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50300039f0 0x7f5030003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:06.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.390+0000 7f5049198640 1 --2- 172.21.15.98:0/4157630155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5030004810 0x7f5030004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:06.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.390+0000 7f5049198640 1 --2- 172.21.15.98:0/4157630155 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50440cf700 0x7f50300051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:06.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.390+0000 7f5049198640 1 -- 172.21.15.98:0/4157630155 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f503000ba80 con 0x7f5030004810 2024-09-19T08:47:06.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.390+0000 7f5049198640 1 -- 172.21.15.98:0/4157630155 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f503000bc30 con 0x7f50300039f0 2024-09-19T08:47:06.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.390+0000 7f5049198640 1 -- 172.21.15.98:0/4157630155 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f503000be30 con 0x7f50440cf700 2024-09-19T08:47:06.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.390+0000 7f5049999640 1 --2- 172.21.15.98:0/4157630155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5030004810 0x7f5030004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:06.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.390+0000 7f504a19a640 1 --2- 172.21.15.98:0/4157630155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50300039f0 0x7f5030003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:06.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.390+0000 7f504a99b640 1 --2- 172.21.15.98:0/4157630155 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50440cf700 0x7f50300051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:06.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.390+0000 7f5049999640 1 -- 172.21.15.98:0/4157630155 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50440cf700 msgr2=0x7f50300051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:06.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.390+0000 7f5049999640 1 --2- 172.21.15.98:0/4157630155 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f50440cf700 0x7f50300051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:06.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.390+0000 7f5049999640 1 -- 172.21.15.98:0/4157630155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50300039f0 msgr2=0x7f5030003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:06.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.390+0000 7f5049999640 1 --2- 172.21.15.98:0/4157630155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50300039f0 0x7f5030003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:06.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.390+0000 7f5049999640 1 -- 172.21.15.98:0/4157630155 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5030003530 con 0x7f5030004810 2024-09-19T08:47:06.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.390+0000 7f504a19a640 1 --2- 172.21.15.98:0/4157630155 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50300039f0 0x7f5030003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:06.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.390+0000 7f5049999640 1 --2- 172.21.15.98:0/4157630155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5030004810 0x7f5030004c90 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f504415feb0 tx=0x7f503401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12517058f832a93 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:06.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.390+0000 7f5049198640 1 -- 172.21.15.98:0/4157630155 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50340b2070 con 0x7f5030004810 2024-09-19T08:47:06.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.390+0000 7f5049198640 1 -- 172.21.15.98:0/4157630155 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50340a55e0 con 0x7f5030004810 2024-09-19T08:47:06.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.390+0000 7f5049198640 1 -- 172.21.15.98:0/4157630155 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50340096b0 con 0x7f5030004810 2024-09-19T08:47:06.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.390+0000 7f504c3fe640 1 -- 172.21.15.98:0/4157630155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5030004810 msgr2=0x7f5030004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:06.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.390+0000 7f504c3fe640 1 --2- 172.21.15.98:0/4157630155 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5030004810 0x7f5030004c90 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f504415feb0 tx=0x7f503401c9f0 comp rx=0 tx=0).stop 2024-09-19T08:47:06.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.390+0000 7f504a99b640 1 -- 172.21.15.98:0/4157630155 reap_dead start 2024-09-19T08:47:06.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.390+0000 7f504c3fe640 1 -- 172.21.15.98:0/4157630155 shutdown_connections 2024-09-19T08:47:06.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.390+0000 7f504c3fe640 1 -- 172.21.15.98:0/4157630155 >> 172.21.15.98:0/4157630155 conn(0x7f504406c090 msgr2=0x7f504406c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:06.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.390+0000 7f504c3fe640 1 -- 172.21.15.98:0/4157630155 shutdown_connections 2024-09-19T08:47:06.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.390+0000 7f504c3fe640 1 -- 172.21.15.98:0/4157630155 wait complete. 2024-09-19T08:47:06.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.394+0000 7f504c3fe640 1 Processor -- start 2024-09-19T08:47:06.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.394+0000 7f504c3fe640 1 -- start start 2024-09-19T08:47:06.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.394+0000 7f504c3fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5030004810 0x7f504414f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:06.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.394+0000 7f504c3fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50440ad080 0x7f50441f7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:06.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.394+0000 7f504c3fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50441f7a80 0x7f50441f7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:06.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.394+0000 7f504c3fe640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f50440b5300 con 0x7f50440ad080 2024-09-19T08:47:06.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.394+0000 7f504c3fe640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f50440cd510 con 0x7f50441f7a80 2024-09-19T08:47:06.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.394+0000 7f504c3fe640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f50440b5480 con 0x7f5030004810 2024-09-19T08:47:06.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.394+0000 7f5049999640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50440ad080 0x7f50441f7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:06.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.394+0000 7f5049999640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50440ad080 0x7f50441f7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46460/0 (socket says 172.21.15.98:46460) 2024-09-19T08:47:06.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.394+0000 7f5049999640 1 -- 172.21.15.98:0/3847685883 learned_addr learned my addr 172.21.15.98:0/3847685883 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:06.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.394+0000 7f504a19a640 1 --2- 172.21.15.98:0/3847685883 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5030004810 0x7f504414f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:06.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.394+0000 7f504a99b640 1 --2- 172.21.15.98:0/3847685883 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50441f7a80 0x7f50441f7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:06.397 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.394+0000 7f5049999640 1 -- 172.21.15.98:0/3847685883 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5030004810 msgr2=0x7f504414f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:06.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.394+0000 7f5049999640 1 --2- 172.21.15.98:0/3847685883 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5030004810 0x7f504414f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:06.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.394+0000 7f5049999640 1 -- 172.21.15.98:0/3847685883 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50441f7a80 msgr2=0x7f50441f7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:06.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.394+0000 7f5049999640 1 --2- 172.21.15.98:0/3847685883 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50441f7a80 0x7f50441f7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:06.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.394+0000 7f5049999640 1 -- 172.21.15.98:0/3847685883 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50440acd30 con 0x7f50440ad080 2024-09-19T08:47:06.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.394+0000 7f5049999640 1 --2- 172.21.15.98:0/3847685883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50440ad080 0x7f50441f7540 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f503401c820 tx=0x7f5030005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:06.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.394+0000 7f503affd640 1 -- 172.21.15.98:0/3847685883 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50340bc020 con 0x7f50440ad080 2024-09-19T08:47:06.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.394+0000 7f503affd640 1 -- 172.21.15.98:0/3847685883 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50340a5d90 con 0x7f50440ad080 2024-09-19T08:47:06.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.394+0000 7f504c3fe640 1 -- 172.21.15.98:0/3847685883 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50440af6b0 con 0x7f50440ad080 2024-09-19T08:47:06.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.394+0000 7f503affd640 1 -- 172.21.15.98:0/3847685883 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50340b2040 con 0x7f50440ad080 2024-09-19T08:47:06.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.394+0000 7f504c3fe640 1 -- 172.21.15.98:0/3847685883 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50440ac810 con 0x7f50440ad080 2024-09-19T08:47:06.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.394+0000 7f504c3fe640 1 -- 172.21.15.98:0/3847685883 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50440afbb0 con 0x7f50440ad080 2024-09-19T08:47:06.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.394+0000 7f503affd640 1 -- 172.21.15.98:0/3847685883 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f5034004260 con 0x7f50440ad080 2024-09-19T08:47:06.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.394+0000 7f503affd640 1 --2- 172.21.15.98:0/3847685883 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5028068250 0x7f502806a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:06.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.398+0000 7f503affd640 1 -- 172.21.15.98:0/3847685883 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 5443+0+0 (secure 0 0 0) 0x7f503411e6f0 con 0x7f50440ad080 2024-09-19T08:47:06.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.398+0000 7f504a19a640 1 --2- 172.21.15.98:0/3847685883 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5028068250 0x7f502806a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:06.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.398+0000 7f504a19a640 1 --2- 172.21.15.98:0/3847685883 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5028068250 0x7f502806a710 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5040007fd0 tx=0x7f5040009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:06.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.402+0000 7f503affd640 1 -- 172.21.15.98:0/3847685883 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f50440afbb0 con 0x7f50440ad080 2024-09-19T08:47:06.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:06.566+0000 7f504c3fe640 1 -- 172.21.15.98:0/3847685883 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f504406d930 con 0x7f50440ad080 2024-09-19T08:47:07.247 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.246+0000 7f503affd640 1 -- 172.21.15.98:0/3847685883 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v162) ==== 169+0+0 (secure 0 0 0) 0x7f50340c6020 con 0x7f50440ad080 2024-09-19T08:47:07.247 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' removed 2024-09-19T08:47:07.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.250+0000 7f504c3fe640 1 -- 172.21.15.98:0/3847685883 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5028068250 msgr2=0x7f502806a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:07.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.250+0000 7f504c3fe640 1 --2- 172.21.15.98:0/3847685883 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5028068250 0x7f502806a710 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5040007fd0 tx=0x7f5040009040 comp rx=0 tx=0).stop 2024-09-19T08:47:07.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.250+0000 7f504c3fe640 1 -- 172.21.15.98:0/3847685883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50440ad080 msgr2=0x7f50441f7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:07.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.250+0000 7f504c3fe640 1 --2- 172.21.15.98:0/3847685883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50440ad080 0x7f50441f7540 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f503401c820 tx=0x7f5030005840 comp rx=0 tx=0).stop 2024-09-19T08:47:07.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.250+0000 7f504c3fe640 1 -- 172.21.15.98:0/3847685883 shutdown_connections 2024-09-19T08:47:07.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.250+0000 7f504c3fe640 1 --2- 172.21.15.98:0/3847685883 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5028068250 0x7f502806a710 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:07.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.250+0000 7f504c3fe640 1 --2- 172.21.15.98:0/3847685883 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f50441f7a80 0x7f50441f7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:07.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.250+0000 7f504c3fe640 1 --2- 172.21.15.98:0/3847685883 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f50440ad080 0x7f50441f7540 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:07.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.250+0000 7f504c3fe640 1 --2- 172.21.15.98:0/3847685883 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5030004810 0x7f504414f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:07.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.250+0000 7f504c3fe640 1 -- 172.21.15.98:0/3847685883 >> 172.21.15.98:0/3847685883 conn(0x7f504406c090 msgr2=0x7f504415ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:07.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.250+0000 7f504c3fe640 1 -- 172.21.15.98:0/3847685883 shutdown_connections 2024-09-19T08:47:07.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.250+0000 7f504c3fe640 1 -- 172.21.15.98:0/3847685883 wait complete. 2024-09-19T08:47:07.273 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-19T08:47:07.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.398+0000 7fcae16c0640 1 Processor -- start 2024-09-19T08:47:07.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.398+0000 7fcae16c0640 1 -- start start 2024-09-19T08:47:07.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.398+0000 7fcae16c0640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcadc147cb0 0x7fcadc06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:07.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.398+0000 7fcae16c0640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcadc162f90 con 0x7fcadc06c1d0 2024-09-19T08:47:07.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.398+0000 7fcae16c0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcadc0cf040 con 0x7fcadc162910 2024-09-19T08:47:07.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.398+0000 7fcae16c0640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fcadc06c640 con 0x7fcadc147cb0 2024-09-19T08:47:07.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.398+0000 7fcadaffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcadc147cb0 0x7fcadc06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:07.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.398+0000 7fcada7fc640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fcadc06c1d0 0x7fcadc162200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:32864/0 (socket says 172.21.15.98:32864) 2024-09-19T08:47:07.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.398+0000 7fcada7fc640 1 -- 172.21.15.98:0/3698981835 learned_addr learned my addr 172.21.15.98:0/3698981835 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:07.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.402+0000 7fcadaffd640 1 -- 172.21.15.98:0/3698981835 >> v1:172.21.15.139:6789/0 conn(0x7fcadc162910 legacy=0x7fcadc0b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:07.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.402+0000 7fcadaffd640 1 -- 172.21.15.98:0/3698981835 >> v1:172.21.15.98:6789/0 conn(0x7fcadc06c1d0 legacy=0x7fcadc162200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:07.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.402+0000 7fcadaffd640 1 -- 172.21.15.98:0/3698981835 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcadc0cf2e0 con 0x7fcadc147cb0 2024-09-19T08:47:07.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.402+0000 7fcadaffd640 1 --2- 172.21.15.98:0/3698981835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcadc147cb0 0x7fcadc06bc90 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fcacc0097c0 tx=0x7fcacc01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f175efc928ac1e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:07.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.402+0000 7fcad9ffb640 1 -- 172.21.15.98:0/3698981835 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc0b0070 con 0x7fcadc147cb0 2024-09-19T08:47:07.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.402+0000 7fcad9ffb640 1 -- 172.21.15.98:0/3698981835 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcacc0ab400 con 0x7fcadc147cb0 2024-09-19T08:47:07.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.402+0000 7fcad9ffb640 1 -- 172.21.15.98:0/3698981835 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc0a4e60 con 0x7fcadc147cb0 2024-09-19T08:47:07.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.402+0000 7fcae16c0640 1 -- 172.21.15.98:0/3698981835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcadc147cb0 msgr2=0x7fcadc06bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:07.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.402+0000 7fcae16c0640 1 --2- 172.21.15.98:0/3698981835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcadc147cb0 0x7fcadc06bc90 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fcacc0097c0 tx=0x7fcacc01c3d0 comp rx=0 tx=0).stop 2024-09-19T08:47:07.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.402+0000 7fcae16c0640 1 -- 172.21.15.98:0/3698981835 shutdown_connections 2024-09-19T08:47:07.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.402+0000 7fcae16c0640 1 --2- 172.21.15.98:0/3698981835 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcadc147cb0 0x7fcadc06bc90 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:07.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.402+0000 7fcae16c0640 1 -- 172.21.15.98:0/3698981835 >> 172.21.15.98:0/3698981835 conn(0x7fcadc0b2010 msgr2=0x7fcadc0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:07.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.402+0000 7fcae16c0640 1 -- 172.21.15.98:0/3698981835 shutdown_connections 2024-09-19T08:47:07.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.402+0000 7fcae16c0640 1 -- 172.21.15.98:0/3698981835 wait complete. 2024-09-19T08:47:07.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.406+0000 7fcae16c0640 1 Processor -- start 2024-09-19T08:47:07.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.406+0000 7fcae16c0640 1 -- start start 2024-09-19T08:47:07.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.406+0000 7fcae16c0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcadc06c1d0 0x7fcadc15e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:07.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.406+0000 7fcae16c0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcadc147cb0 0x7fcadc144c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:07.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.406+0000 7fcae16c0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcadc162910 0x7fcadc139570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:07.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.406+0000 7fcae16c0640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcadc162d40 con 0x7fcadc147cb0 2024-09-19T08:47:07.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.406+0000 7fcae16c0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcadc0cffd0 con 0x7fcadc06c1d0 2024-09-19T08:47:07.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.406+0000 7fcae16c0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcadc16c3e0 con 0x7fcadc162910 2024-09-19T08:47:07.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.406+0000 7fcada7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcadc147cb0 0x7fcadc144c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:07.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.406+0000 7fcada7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcadc147cb0 0x7fcadc144c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:46486/0 (socket says 172.21.15.98:46486) 2024-09-19T08:47:07.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.406+0000 7fcada7fc640 1 -- 172.21.15.98:0/2313998475 learned_addr learned my addr 172.21.15.98:0/2313998475 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:07.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.406+0000 7fcadaffd640 1 --2- 172.21.15.98:0/2313998475 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcadc06c1d0 0x7fcadc15e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:07.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.406+0000 7fcadb7fe640 1 --2- 172.21.15.98:0/2313998475 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcadc162910 0x7fcadc139570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:07.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.406+0000 7fcada7fc640 1 -- 172.21.15.98:0/2313998475 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcadc162910 msgr2=0x7fcadc139570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:07.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.406+0000 7fcada7fc640 1 --2- 172.21.15.98:0/2313998475 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcadc162910 0x7fcadc139570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:07.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.406+0000 7fcada7fc640 1 -- 172.21.15.98:0/2313998475 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcadc06c1d0 msgr2=0x7fcadc15e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:07.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.406+0000 7fcada7fc640 1 --2- 172.21.15.98:0/2313998475 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcadc06c1d0 0x7fcadc15e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:07.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.406+0000 7fcada7fc640 1 -- 172.21.15.98:0/2313998475 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcadc143b70 con 0x7fcadc147cb0 2024-09-19T08:47:07.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.406+0000 7fcada7fc640 1 --2- 172.21.15.98:0/2313998475 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcadc147cb0 0x7fcadc144c10 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fcad000c370 tx=0x7fcad000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:07.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.406+0000 7fcac37fe640 1 -- 172.21.15.98:0/2313998475 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad0017070 con 0x7fcadc147cb0 2024-09-19T08:47:07.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.406+0000 7fcac37fe640 1 -- 172.21.15.98:0/2313998475 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcad0005cf0 con 0x7fcadc147cb0 2024-09-19T08:47:07.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.406+0000 7fcae16c0640 1 -- 172.21.15.98:0/2313998475 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcadc143e50 con 0x7fcadc147cb0 2024-09-19T08:47:07.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.406+0000 7fcac37fe640 1 -- 172.21.15.98:0/2313998475 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad00072f0 con 0x7fcadc147cb0 2024-09-19T08:47:07.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.406+0000 7fcae16c0640 1 -- 172.21.15.98:0/2313998475 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcadc144410 con 0x7fcadc147cb0 2024-09-19T08:47:07.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.406+0000 7fcae16c0640 1 -- 172.21.15.98:0/2313998475 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcadc06bc90 con 0x7fcadc147cb0 2024-09-19T08:47:07.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.406+0000 7fcac37fe640 1 -- 172.21.15.98:0/2313998475 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7fcad001a070 con 0x7fcadc147cb0 2024-09-19T08:47:07.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.410+0000 7fcac37fe640 1 --2- 172.21.15.98:0/2313998475 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcac8068250 0x7fcac806a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:07.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.410+0000 7fcac37fe640 1 -- 172.21.15.98:0/2313998475 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 4971+0+0 (secure 0 0 0) 0x7fcad0083720 con 0x7fcadc147cb0 2024-09-19T08:47:07.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.410+0000 7fcadaffd640 1 --2- 172.21.15.98:0/2313998475 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcac8068250 0x7fcac806a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:07.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.410+0000 7fcadaffd640 1 --2- 172.21.15.98:0/2313998475 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcac8068250 0x7fcac806a710 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcacc0ad000 tx=0x7fcacc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:07.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.414+0000 7fcac37fe640 1 -- 172.21.15.98:0/2313998475 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fcad0055e60 con 0x7fcadc147cb0 2024-09-19T08:47:07.576 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.578+0000 7fcae16c0640 1 -- 172.21.15.98:0/2313998475 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fcadc13e250 con 0x7fcadc147cb0 2024-09-19T08:47:07.581 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.582+0000 7fcac37fe640 1 -- 172.21.15.98:0/2313998475 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7fcad0055760 con 0x7fcadc147cb0 2024-09-19T08:47:07.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.586+0000 7fcae16c0640 1 -- 172.21.15.98:0/2313998475 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcac8068250 msgr2=0x7fcac806a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:07.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.586+0000 7fcae16c0640 1 --2- 172.21.15.98:0/2313998475 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcac8068250 0x7fcac806a710 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcacc0ad000 tx=0x7fcacc008040 comp rx=0 tx=0).stop 2024-09-19T08:47:07.584 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.586+0000 7fcae16c0640 1 -- 172.21.15.98:0/2313998475 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcadc147cb0 msgr2=0x7fcadc144c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:07.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.586+0000 7fcae16c0640 1 --2- 172.21.15.98:0/2313998475 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcadc147cb0 0x7fcadc144c10 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fcad000c370 tx=0x7fcad000c830 comp rx=0 tx=0).stop 2024-09-19T08:47:07.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.586+0000 7fcae16c0640 1 -- 172.21.15.98:0/2313998475 shutdown_connections 2024-09-19T08:47:07.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.586+0000 7fcae16c0640 1 --2- 172.21.15.98:0/2313998475 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcac8068250 0x7fcac806a710 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:07.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.586+0000 7fcae16c0640 1 --2- 172.21.15.98:0/2313998475 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcadc162910 0x7fcadc139570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:07.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.586+0000 7fcae16c0640 1 --2- 172.21.15.98:0/2313998475 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcadc147cb0 0x7fcadc144c10 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:07.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.586+0000 7fcae16c0640 1 --2- 172.21.15.98:0/2313998475 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcadc06c1d0 0x7fcadc15e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:07.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.586+0000 7fcae16c0640 1 -- 172.21.15.98:0/2313998475 >> 172.21.15.98:0/2313998475 conn(0x7fcadc0b2010 msgr2=0x7fcadc13e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:07.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.586+0000 7fcae16c0640 1 -- 172.21.15.98:0/2313998475 shutdown_connections 2024-09-19T08:47:07.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.586+0000 7fcae16c0640 1 -- 172.21.15.98:0/2313998475 wait complete. 2024-09-19T08:47:07.602 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2' 2024-09-19T08:47:07.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.726+0000 7ff9c77bc640 1 Processor -- start 2024-09-19T08:47:07.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.726+0000 7ff9c77bc640 1 -- start start 2024-09-19T08:47:07.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.726+0000 7ff9c77bc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9c015c550 0x7ff9c006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:07.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.726+0000 7ff9c77bc640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9c006d310 con 0x7ff9c006c1d0 2024-09-19T08:47:07.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.726+0000 7ff9c77bc640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9c006d5f0 con 0x7ff9c00cfad0 2024-09-19T08:47:07.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.726+0000 7ff9c77bc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff9c006c610 con 0x7ff9c015c550 2024-09-19T08:47:07.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.726+0000 7ff9c4d57640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7ff9c006c1d0 0x7ff9c00cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40542/0 (socket says 172.21.15.98:40542) 2024-09-19T08:47:07.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.726+0000 7ff9c4d57640 1 -- 172.21.15.98:0/3155613682 learned_addr learned my addr 172.21.15.98:0/3155613682 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:07.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.726+0000 7ff9c5558640 1 --2- 172.21.15.98:0/3155613682 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9c015c550 0x7ff9c006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:07.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.726+0000 7ff9b7fff640 1 -- 172.21.15.98:0/3155613682 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3213783579 0 0) 0x7ff9c006d310 con 0x7ff9c006c1d0 2024-09-19T08:47:07.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.726+0000 7ff9b7fff640 1 -- 172.21.15.98:0/3155613682 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9ac003530 con 0x7ff9c006c1d0 2024-09-19T08:47:07.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.726+0000 7ff9b7fff640 1 -- 172.21.15.98:0/3155613682 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3129329729 0 0) 0x7ff9ac003530 con 0x7ff9c006c1d0 2024-09-19T08:47:07.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.726+0000 7ff9b7fff640 1 -- 172.21.15.98:0/3155613682 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9c015c550 msgr2=0x7ff9c006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:07.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.726+0000 7ff9b7fff640 1 --2- 172.21.15.98:0/3155613682 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9c015c550 0x7ff9c006bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:07.727 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.726+0000 7ff9b7fff640 1 -- 172.21.15.98:0/3155613682 >> v1:172.21.15.139:6789/0 conn(0x7ff9c00cfad0 legacy=0x7ff9c00723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:07.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.726+0000 7ff9b7fff640 1 -- 172.21.15.98:0/3155613682 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c006d8d0 con 0x7ff9c006c1d0 2024-09-19T08:47:07.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.726+0000 7ff9b7fff640 1 -- 172.21.15.98:0/3155613682 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7ff9b0001680 con 0x7ff9c006c1d0 2024-09-19T08:47:07.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.726+0000 7ff9b7fff640 1 -- 172.21.15.98:0/3155613682 >> v1:172.21.15.98:6789/0 conn(0x7ff9c006c1d0 legacy=0x7ff9c00cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:07.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.726+0000 7ff9b7fff640 1 --2- 172.21.15.98:0/3155613682 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9c015c550 0x7ff9ac0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:07.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.726+0000 7ff9b7fff640 1 --2- 172.21.15.98:0/3155613682 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9ac004020 0x7ff9ac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:07.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.726+0000 7ff9b7fff640 1 --2- 172.21.15.98:0/3155613682 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9ac004d50 0x7ff9ac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:07.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.726+0000 7ff9b7fff640 1 -- 172.21.15.98:0/3155613682 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c006c610 con 0x7ff9ac004d50 2024-09-19T08:47:07.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.726+0000 7ff9b7fff640 1 -- 172.21.15.98:0/3155613682 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff9ac00bb90 con 0x7ff9ac004020 2024-09-19T08:47:07.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.726+0000 7ff9b7fff640 1 -- 172.21.15.98:0/3155613682 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff9ac00bd90 con 0x7ff9c015c550 2024-09-19T08:47:07.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.726+0000 7ff9c5d59640 1 --2- 172.21.15.98:0/3155613682 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9ac004d50 0x7ff9ac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:07.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.726+0000 7ff9c5558640 1 --2- 172.21.15.98:0/3155613682 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9c015c550 0x7ff9ac0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:07.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.726+0000 7ff9c4d57640 1 --2- 172.21.15.98:0/3155613682 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9ac004020 0x7ff9ac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:07.728 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.726+0000 7ff9c5d59640 1 -- 172.21.15.98:0/3155613682 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9c015c550 msgr2=0x7ff9ac0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:07.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.726+0000 7ff9c5d59640 1 --2- 172.21.15.98:0/3155613682 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9c015c550 0x7ff9ac0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:07.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.726+0000 7ff9c5d59640 1 -- 172.21.15.98:0/3155613682 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9ac004020 msgr2=0x7ff9ac004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:07.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.726+0000 7ff9c5d59640 1 --2- 172.21.15.98:0/3155613682 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9ac004020 0x7ff9ac004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:07.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.726+0000 7ff9c5d59640 1 -- 172.21.15.98:0/3155613682 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9ac003530 con 0x7ff9ac004d50 2024-09-19T08:47:07.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.730+0000 7ff9c5d59640 1 --2- 172.21.15.98:0/3155613682 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9ac004d50 0x7ff9ac0051d0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8001e70 tx=0x7ff9b801c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e7e164df5f944e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:07.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.730+0000 7ff9b7fff640 1 -- 172.21.15.98:0/3155613682 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8001a90 con 0x7ff9ac004d50 2024-09-19T08:47:07.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.730+0000 7ff9b7fff640 1 -- 172.21.15.98:0/3155613682 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9b8003b80 con 0x7ff9ac004d50 2024-09-19T08:47:07.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.730+0000 7ff9b7fff640 1 -- 172.21.15.98:0/3155613682 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8005080 con 0x7ff9ac004d50 2024-09-19T08:47:07.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.730+0000 7ff9c77bc640 1 -- 172.21.15.98:0/3155613682 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9ac004d50 msgr2=0x7ff9ac0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:07.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.730+0000 7ff9c77bc640 1 --2- 172.21.15.98:0/3155613682 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9ac004d50 0x7ff9ac0051d0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8001e70 tx=0x7ff9b801c780 comp rx=0 tx=0).stop 2024-09-19T08:47:07.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.730+0000 7ff9c5d59640 1 -- 172.21.15.98:0/3155613682 reap_dead start 2024-09-19T08:47:07.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.730+0000 7ff9c77bc640 1 -- 172.21.15.98:0/3155613682 shutdown_connections 2024-09-19T08:47:07.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.730+0000 7ff9c77bc640 1 -- 172.21.15.98:0/3155613682 >> 172.21.15.98:0/3155613682 conn(0x7ff9c00b2010 msgr2=0x7ff9c00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:07.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.730+0000 7ff9c77bc640 1 -- 172.21.15.98:0/3155613682 shutdown_connections 2024-09-19T08:47:07.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.730+0000 7ff9c77bc640 1 -- 172.21.15.98:0/3155613682 wait complete. 2024-09-19T08:47:07.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.730+0000 7ff9c77bc640 1 Processor -- start 2024-09-19T08:47:07.731 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.730+0000 7ff9c77bc640 1 -- start start 2024-09-19T08:47:07.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.734+0000 7ff9c77bc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9ac004d50 0x7ff9c014f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:07.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.734+0000 7ff9c77bc640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9c014faa0 0x7ff9c01549c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:07.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.734+0000 7ff9c77bc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9c0154f00 0x7ff9c01553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:07.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.734+0000 7ff9c77bc640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c015cab0 con 0x7ff9c0154f00 2024-09-19T08:47:07.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.734+0000 7ff9c77bc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c015c930 con 0x7ff9ac004d50 2024-09-19T08:47:07.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.734+0000 7ff9c77bc640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff9c006bc90 con 0x7ff9c014faa0 2024-09-19T08:47:07.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.734+0000 7ff9c5d59640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9c0154f00 0x7ff9c01553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:07.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.734+0000 7ff9c5d59640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9c0154f00 0x7ff9c01553b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34956/0 (socket says 172.21.15.98:34956) 2024-09-19T08:47:07.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.734+0000 7ff9c5d59640 1 -- 172.21.15.98:0/2909006919 learned_addr learned my addr 172.21.15.98:0/2909006919 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:07.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.734+0000 7ff9c4d57640 1 --2- 172.21.15.98:0/2909006919 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9c014faa0 0x7ff9c01549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:07.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.734+0000 7ff9c5558640 1 --2- 172.21.15.98:0/2909006919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9ac004d50 0x7ff9c014f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:07.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.734+0000 7ff9c5d59640 1 -- 172.21.15.98:0/2909006919 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9c014faa0 msgr2=0x7ff9c01549c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:07.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.734+0000 7ff9c5d59640 1 --2- 172.21.15.98:0/2909006919 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9c014faa0 0x7ff9c01549c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:07.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.734+0000 7ff9c5d59640 1 -- 172.21.15.98:0/2909006919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9ac004d50 msgr2=0x7ff9c014f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:07.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.734+0000 7ff9c5d59640 1 --2- 172.21.15.98:0/2909006919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9ac004d50 0x7ff9c014f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:07.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.734+0000 7ff9c5d59640 1 -- 172.21.15.98:0/2909006919 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c015cd60 con 0x7ff9c0154f00 2024-09-19T08:47:07.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.734+0000 7ff9c5d59640 1 --2- 172.21.15.98:0/2909006919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9c0154f00 0x7ff9c01553b0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8001e70 tx=0x7ff9ac005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:07.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.734+0000 7ff9b5ffb640 1 -- 172.21.15.98:0/2909006919 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b80b9020 con 0x7ff9c0154f00 2024-09-19T08:47:07.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.734+0000 7ff9b5ffb640 1 -- 172.21.15.98:0/2909006919 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9b80a6480 con 0x7ff9c0154f00 2024-09-19T08:47:07.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.734+0000 7ff9c77bc640 1 -- 172.21.15.98:0/2909006919 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c015d040 con 0x7ff9c0154f00 2024-09-19T08:47:07.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.734+0000 7ff9b5ffb640 1 -- 172.21.15.98:0/2909006919 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8005840 con 0x7ff9c0154f00 2024-09-19T08:47:07.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.734+0000 7ff9c77bc640 1 -- 172.21.15.98:0/2909006919 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c015d600 con 0x7ff9c0154f00 2024-09-19T08:47:07.735 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.734+0000 7ff9c77bc640 1 -- 172.21.15.98:0/2909006919 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9c00cff80 con 0x7ff9c0154f00 2024-09-19T08:47:07.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.734+0000 7ff9b5ffb640 1 -- 172.21.15.98:0/2909006919 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7ff9b80bc070 con 0x7ff9c0154f00 2024-09-19T08:47:07.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.734+0000 7ff9b5ffb640 1 --2- 172.21.15.98:0/2909006919 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff9a4068280 0x7ff9a406a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:07.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.734+0000 7ff9b5ffb640 1 -- 172.21.15.98:0/2909006919 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 4971+0+0 (secure 0 0 0) 0x7ff9b811d740 con 0x7ff9c0154f00 2024-09-19T08:47:07.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.734+0000 7ff9c5558640 1 --2- 172.21.15.98:0/2909006919 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff9a4068280 0x7ff9a406a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:07.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.738+0000 7ff9c5558640 1 --2- 172.21.15.98:0/2909006919 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff9a4068280 0x7ff9a406a740 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff9c01f5880 tx=0x7ff9bc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:07.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.738+0000 7ff9b5ffb640 1 -- 172.21.15.98:0/2909006919 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff9b80ac020 con 0x7ff9c0154f00 2024-09-19T08:47:07.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:07.958+0000 7ff9c77bc640 1 -- 172.21.15.98:0/2909006919 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]} v 0) -- 0x7ff9c00d0fa0 con 0x7ff9c0154f00 2024-09-19T08:47:08.249 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.250+0000 7ff9b5ffb640 1 -- 172.21.15.98:0/2909006919 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]}]=0 v461) ==== 145+0+0 (secure 0 0 0) 0x7ff9b8016490 con 0x7ff9c0154f00 2024-09-19T08:47:08.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.254+0000 7ff9c77bc640 1 -- 172.21.15.98:0/2909006919 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff9a4068280 msgr2=0x7ff9a406a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:08.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.254+0000 7ff9c77bc640 1 --2- 172.21.15.98:0/2909006919 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff9a4068280 0x7ff9a406a740 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff9c01f5880 tx=0x7ff9bc008040 comp rx=0 tx=0).stop 2024-09-19T08:47:08.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.254+0000 7ff9c77bc640 1 -- 172.21.15.98:0/2909006919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9c0154f00 msgr2=0x7ff9c01553b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:08.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.254+0000 7ff9c77bc640 1 --2- 172.21.15.98:0/2909006919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9c0154f00 0x7ff9c01553b0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8001e70 tx=0x7ff9ac005840 comp rx=0 tx=0).stop 2024-09-19T08:47:08.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.254+0000 7ff9c77bc640 1 -- 172.21.15.98:0/2909006919 shutdown_connections 2024-09-19T08:47:08.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.254+0000 7ff9c77bc640 1 --2- 172.21.15.98:0/2909006919 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff9a4068280 0x7ff9a406a740 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:08.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.254+0000 7ff9c77bc640 1 --2- 172.21.15.98:0/2909006919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9c0154f00 0x7ff9c01553b0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:08.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.254+0000 7ff9c77bc640 1 --2- 172.21.15.98:0/2909006919 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9c014faa0 0x7ff9c01549c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:08.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.254+0000 7ff9c77bc640 1 --2- 172.21.15.98:0/2909006919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff9ac004d50 0x7ff9c014f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:08.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.254+0000 7ff9c77bc640 1 -- 172.21.15.98:0/2909006919 >> 172.21.15.98:0/2909006919 conn(0x7ff9c00b2010 msgr2=0x7ff9c0159a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:08.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.254+0000 7ff9c77bc640 1 -- 172.21.15.98:0/2909006919 shutdown_connections 2024-09-19T08:47:08.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.254+0000 7ff9c77bc640 1 -- 172.21.15.98:0/2909006919 wait complete. 2024-09-19T08:47:08.271 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2024-09-19T08:47:08.271 INFO:tasks.cephfs_test_runner:Starting test: test_unparseable_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate unparseable text .meta file on legacy subvol root doesn't break the system 2024-09-19T08:47:08.271 INFO:tasks.cephfs_test_runner:test_unparseable_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-09-19T08:47:08.271 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:47:08.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.394+0000 7f9bb5f58640 1 Processor -- start 2024-09-19T08:47:08.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.394+0000 7f9bb5f58640 1 -- start start 2024-09-19T08:47:08.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.398+0000 7f9bb5f58640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9bb006c3a0 0x7f9bb00cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:08.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.398+0000 7f9bb5f58640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bb006d310 con 0x7f9bb00cf900 2024-09-19T08:47:08.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.398+0000 7f9bb5f58640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bb006d5f0 con 0x7f9bb015c550 2024-09-19T08:47:08.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.398+0000 7f9bb5f58640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9bb006d8d0 con 0x7f9bb006c3a0 2024-09-19T08:47:08.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.398+0000 7f9baffff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f9bb00cf900 0x7f9bb00723a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40546/0 (socket says 172.21.15.98:40546) 2024-09-19T08:47:08.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.398+0000 7f9baffff640 1 -- 172.21.15.98:0/1231007767 learned_addr learned my addr 172.21.15.98:0/1231007767 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:08.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.398+0000 7f9ba6ffd640 1 --2- 172.21.15.98:0/1231007767 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9bb006c3a0 0x7f9bb00cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:08.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.398+0000 7f9baeffd640 1 -- 172.21.15.98:0/1231007767 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3068445319 0 0) 0x7f9bb006d310 con 0x7f9bb00cf900 2024-09-19T08:47:08.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.398+0000 7f9baeffd640 1 -- 172.21.15.98:0/1231007767 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b9c003530 con 0x7f9bb00cf900 2024-09-19T08:47:08.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.398+0000 7f9baeffd640 1 -- 172.21.15.98:0/1231007767 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 642791759 0 0) 0x7f9b9c003530 con 0x7f9bb00cf900 2024-09-19T08:47:08.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.398+0000 7f9baeffd640 1 -- 172.21.15.98:0/1231007767 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9bb006c3a0 msgr2=0x7f9bb00cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:08.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.398+0000 7f9baeffd640 1 --2- 172.21.15.98:0/1231007767 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9bb006c3a0 0x7f9bb00cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:08.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.398+0000 7f9baeffd640 1 -- 172.21.15.98:0/1231007767 >> v1:172.21.15.139:6789/0 conn(0x7f9bb015c550 legacy=0x7f9bb006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:08.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.398+0000 7f9baeffd640 1 -- 172.21.15.98:0/1231007767 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bb006dbb0 con 0x7f9bb00cf900 2024-09-19T08:47:08.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.398+0000 7f9baeffd640 1 -- 172.21.15.98:0/1231007767 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f9ba8001a00 con 0x7f9bb00cf900 2024-09-19T08:47:08.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.398+0000 7f9baeffd640 1 -- 172.21.15.98:0/1231007767 >> v1:172.21.15.98:6789/0 conn(0x7f9bb00cf900 legacy=0x7f9bb00723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:08.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.398+0000 7f9baeffd640 1 --2- 172.21.15.98:0/1231007767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b9c0039f0 0x7f9b9c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:08.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.398+0000 7f9baeffd640 1 --2- 172.21.15.98:0/1231007767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b9c004810 0x7f9b9c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:08.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.398+0000 7f9baeffd640 1 --2- 172.21.15.98:0/1231007767 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9bb006c3a0 0x7f9b9c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:08.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.398+0000 7f9baeffd640 1 -- 172.21.15.98:0/1231007767 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb006d8d0 con 0x7f9b9c0039f0 2024-09-19T08:47:08.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.398+0000 7f9baeffd640 1 -- 172.21.15.98:0/1231007767 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9c00ba80 con 0x7f9b9c004810 2024-09-19T08:47:08.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.398+0000 7f9baeffd640 1 -- 172.21.15.98:0/1231007767 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9b9c00bc80 con 0x7f9bb006c3a0 2024-09-19T08:47:08.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.398+0000 7f9baf7fe640 1 --2- 172.21.15.98:0/1231007767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b9c0039f0 0x7f9b9c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:08.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.398+0000 7f9baf7fe640 1 -- 172.21.15.98:0/1231007767 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9bb006c3a0 msgr2=0x7f9b9c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:08.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.398+0000 7f9baffff640 1 --2- 172.21.15.98:0/1231007767 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9bb006c3a0 0x7f9b9c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:08.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.398+0000 7f9ba6ffd640 1 --2- 172.21.15.98:0/1231007767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b9c004810 0x7f9b9c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:08.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.398+0000 7f9baf7fe640 1 --2- 172.21.15.98:0/1231007767 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9bb006c3a0 0x7f9b9c0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:08.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.398+0000 7f9baf7fe640 1 -- 172.21.15.98:0/1231007767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b9c004810 msgr2=0x7f9b9c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:08.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.398+0000 7f9baf7fe640 1 --2- 172.21.15.98:0/1231007767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9b9c004810 0x7f9b9c004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:08.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.398+0000 7f9baf7fe640 1 -- 172.21.15.98:0/1231007767 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b9c003530 con 0x7f9b9c0039f0 2024-09-19T08:47:08.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.402+0000 7f9baf7fe640 1 --2- 172.21.15.98:0/1231007767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b9c0039f0 0x7f9b9c003e10 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0001750 tx=0x7f9ba001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b3ee417af0c575 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:08.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.402+0000 7f9baeffd640 1 -- 172.21.15.98:0/1231007767 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba00013a0 con 0x7f9b9c0039f0 2024-09-19T08:47:08.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.402+0000 7f9baeffd640 1 -- 172.21.15.98:0/1231007767 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ba00052f0 con 0x7f9b9c0039f0 2024-09-19T08:47:08.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.402+0000 7f9baeffd640 1 -- 172.21.15.98:0/1231007767 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba0003840 con 0x7f9b9c0039f0 2024-09-19T08:47:08.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.402+0000 7f9bb5f58640 1 -- 172.21.15.98:0/1231007767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b9c0039f0 msgr2=0x7f9b9c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:08.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.402+0000 7f9bb5f58640 1 --2- 172.21.15.98:0/1231007767 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9b9c0039f0 0x7f9b9c003e10 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0001750 tx=0x7f9ba001c3c0 comp rx=0 tx=0).stop 2024-09-19T08:47:08.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.402+0000 7f9baffff640 1 -- 172.21.15.98:0/1231007767 reap_dead start 2024-09-19T08:47:08.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.402+0000 7f9bb5f58640 1 -- 172.21.15.98:0/1231007767 shutdown_connections 2024-09-19T08:47:08.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.402+0000 7f9bb5f58640 1 -- 172.21.15.98:0/1231007767 >> 172.21.15.98:0/1231007767 conn(0x7f9bb00b2010 msgr2=0x7f9bb00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:08.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.402+0000 7f9bb5f58640 1 -- 172.21.15.98:0/1231007767 shutdown_connections 2024-09-19T08:47:08.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.402+0000 7f9bb5f58640 1 -- 172.21.15.98:0/1231007767 wait complete. 2024-09-19T08:47:08.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.402+0000 7f9bb5f58640 1 Processor -- start 2024-09-19T08:47:08.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.402+0000 7f9bb5f58640 1 -- start start 2024-09-19T08:47:08.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.402+0000 7f9bb5f58640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9b9c0039f0 0x7f9bb01f5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:08.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.402+0000 7f9bb5f58640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bb015f560 0x7f9bb01ea9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:08.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.402+0000 7f9bb5f58640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9bb015f9e0 0x7f9bb01eaee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:08.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.402+0000 7f9bb5f58640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb00734d0 con 0x7f9bb015f560 2024-09-19T08:47:08.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.402+0000 7f9bb5f58640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb015cb40 con 0x7f9bb015f9e0 2024-09-19T08:47:08.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.402+0000 7f9bb5f58640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9bb0073650 con 0x7f9b9c0039f0 2024-09-19T08:47:08.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.402+0000 7f9ba6ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bb015f560 0x7f9bb01ea9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:08.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.402+0000 7f9ba6ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bb015f560 0x7f9bb01ea9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34980/0 (socket says 172.21.15.98:34980) 2024-09-19T08:47:08.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.402+0000 7f9ba6ffd640 1 -- 172.21.15.98:0/3563129748 learned_addr learned my addr 172.21.15.98:0/3563129748 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:08.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.402+0000 7f9ba6ffd640 1 -- 172.21.15.98:0/3563129748 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9b9c0039f0 msgr2=0x7f9bb01f5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:08.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.402+0000 7f9baf7fe640 1 --2- 172.21.15.98:0/3563129748 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9b9c0039f0 0x7f9bb01f5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:08.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.402+0000 7f9ba6ffd640 1 --2- 172.21.15.98:0/3563129748 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9b9c0039f0 0x7f9bb01f5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:08.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.402+0000 7f9ba6ffd640 1 -- 172.21.15.98:0/3563129748 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9bb015f9e0 msgr2=0x7f9bb01eaee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:08.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.402+0000 7f9baffff640 1 --2- 172.21.15.98:0/3563129748 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9bb015f9e0 0x7f9bb01eaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:08.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.402+0000 7f9ba6ffd640 1 --2- 172.21.15.98:0/3563129748 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9bb015f9e0 0x7f9bb01eaee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:08.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.402+0000 7f9ba6ffd640 1 -- 172.21.15.98:0/3563129748 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bb015e4c0 con 0x7f9bb015f560 2024-09-19T08:47:08.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.402+0000 7f9ba6ffd640 1 --2- 172.21.15.98:0/3563129748 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bb015f560 0x7f9bb01ea9a0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f9b980108f0 tx=0x7f9b98010db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:08.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.402+0000 7f9bacff9640 1 -- 172.21.15.98:0/3563129748 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9800f040 con 0x7f9bb015f560 2024-09-19T08:47:08.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.406+0000 7f9bacff9640 1 -- 172.21.15.98:0/3563129748 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b9800eb00 con 0x7f9bb015f560 2024-09-19T08:47:08.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.406+0000 7f9bb5f58640 1 -- 172.21.15.98:0/3563129748 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bb015e7a0 con 0x7f9bb015f560 2024-09-19T08:47:08.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.406+0000 7f9bacff9640 1 -- 172.21.15.98:0/3563129748 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9801c400 con 0x7f9bb015f560 2024-09-19T08:47:08.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.406+0000 7f9bb5f58640 1 -- 172.21.15.98:0/3563129748 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bb015ed60 con 0x7f9bb015f560 2024-09-19T08:47:08.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.406+0000 7f9bb5f58640 1 -- 172.21.15.98:0/3563129748 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bb00723a0 con 0x7f9bb015f560 2024-09-19T08:47:08.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.406+0000 7f9bacff9640 1 -- 172.21.15.98:0/3563129748 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f9b9801c5a0 con 0x7f9bb015f560 2024-09-19T08:47:08.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.406+0000 7f9bacff9640 1 --2- 172.21.15.98:0/3563129748 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9b90068280 0x7f9b9006a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:08.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.406+0000 7f9bacff9640 1 -- 172.21.15.98:0/3563129748 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 4971+0+0 (secure 0 0 0) 0x7f9b98084910 con 0x7f9bb015f560 2024-09-19T08:47:08.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.406+0000 7f9baf7fe640 1 --2- 172.21.15.98:0/3563129748 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9b90068280 0x7f9b9006a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:08.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.406+0000 7f9baf7fe640 1 --2- 172.21.15.98:0/3563129748 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9b90068280 0x7f9b9006a740 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c0057f0 tx=0x7f9ba00011a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:08.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.410+0000 7f9bacff9640 1 -- 172.21.15.98:0/3563129748 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9b98018030 con 0x7f9bb015f560 2024-09-19T08:47:08.561 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.562+0000 7f9bb5f58640 1 -- 172.21.15.98:0/3563129748 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9bb00d0b40 con 0x7f9bb015f560 2024-09-19T08:47:08.562 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.562+0000 7f9bacff9640 1 -- 172.21.15.98:0/3563129748 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) ==== 74+0+13086 (secure 0 0 0) 0x7f9b98057050 con 0x7f9bb015f560 2024-09-19T08:47:08.563 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:47:08.563 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":162,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:47:07.237351+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.98:6835/3734544897":"2024-09-20T08:47:04.222575+0000","172.21.15.98:6833/3734544897":"2024-09-20T08:47:04.222575+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:47:08.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.566+0000 7f9bb5f58640 1 -- 172.21.15.98:0/3563129748 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9b90068280 msgr2=0x7f9b9006a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:08.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.566+0000 7f9bb5f58640 1 --2- 172.21.15.98:0/3563129748 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9b90068280 0x7f9b9006a740 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c0057f0 tx=0x7f9ba00011a0 comp rx=0 tx=0).stop 2024-09-19T08:47:08.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.566+0000 7f9bb5f58640 1 -- 172.21.15.98:0/3563129748 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bb015f560 msgr2=0x7f9bb01ea9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:08.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.566+0000 7f9bb5f58640 1 --2- 172.21.15.98:0/3563129748 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bb015f560 0x7f9bb01ea9a0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f9b980108f0 tx=0x7f9b98010db0 comp rx=0 tx=0).stop 2024-09-19T08:47:08.565 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.566+0000 7f9bb5f58640 1 -- 172.21.15.98:0/3563129748 shutdown_connections 2024-09-19T08:47:08.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.566+0000 7f9bb5f58640 1 --2- 172.21.15.98:0/3563129748 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9b90068280 0x7f9b9006a740 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:08.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.566+0000 7f9bb5f58640 1 --2- 172.21.15.98:0/3563129748 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9bb015f9e0 0x7f9bb01eaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:08.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.566+0000 7f9bb5f58640 1 --2- 172.21.15.98:0/3563129748 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9bb015f560 0x7f9bb01ea9a0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:08.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.566+0000 7f9bb5f58640 1 --2- 172.21.15.98:0/3563129748 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9b9c0039f0 0x7f9bb01f5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:08.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.566+0000 7f9bb5f58640 1 -- 172.21.15.98:0/3563129748 >> 172.21.15.98:0/3563129748 conn(0x7f9bb00b2010 msgr2=0x7f9bb015b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:08.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.566+0000 7f9bb5f58640 1 -- 172.21.15.98:0/3563129748 shutdown_connections 2024-09-19T08:47:08.566 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.566+0000 7f9bb5f58640 1 -- 172.21.15.98:0/3563129748 wait complete. 2024-09-19T08:47:08.581 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T08:38:08.105684+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T08:47:08.581 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T08:47:08.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.698+0000 7fb85d1ee640 1 Processor -- start 2024-09-19T08:47:08.696 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.698+0000 7fb85d1ee640 1 -- start start 2024-09-19T08:47:08.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.698+0000 7fb85d1ee640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb8580cea90 0x7fb8580cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:08.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.698+0000 7fb85d1ee640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8580b4320 con 0x7fb85815b7e0 2024-09-19T08:47:08.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.698+0000 7fb85d1ee640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb858158870 con 0x7fb85815ebe0 2024-09-19T08:47:08.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.698+0000 7fb85d1ee640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fb8580cd600 con 0x7fb8580cea90 2024-09-19T08:47:08.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.698+0000 7fb85659c640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fb85815b7e0 0x7fb85815e4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40560/0 (socket says 172.21.15.98:40560) 2024-09-19T08:47:08.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.698+0000 7fb85659c640 1 -- 172.21.15.98:0/597356588 learned_addr learned my addr 172.21.15.98:0/597356588 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:08.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.698+0000 7fb855d9b640 1 -- 172.21.15.98:0/597356588 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 65566236 0 0) 0x7fb84c001120 con 0x7fb85815b7e0 2024-09-19T08:47:08.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.698+0000 7fb855d9b640 1 -- 172.21.15.98:0/597356588 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb840003530 con 0x7fb85815b7e0 2024-09-19T08:47:08.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.698+0000 7fb856d9d640 1 --2- 172.21.15.98:0/597356588 >> v2:172.21.15.139:3300/0 conn(0x7fb8580cea90 0x7fb8580cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:08.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.698+0000 7fb855d9b640 1 -- 172.21.15.98:0/597356588 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 185778155 0 0) 0x7fb840003530 con 0x7fb85815b7e0 2024-09-19T08:47:08.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.698+0000 7fb855d9b640 1 -- 172.21.15.98:0/597356588 >> v2:172.21.15.139:3300/0 conn(0x7fb8580cea90 msgr2=0x7fb8580cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:08.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.698+0000 7fb855d9b640 1 --2- 172.21.15.98:0/597356588 >> v2:172.21.15.139:3300/0 conn(0x7fb8580cea90 0x7fb8580cee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:08.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.698+0000 7fb855d9b640 1 -- 172.21.15.98:0/597356588 >> v1:172.21.15.139:6789/0 conn(0x7fb85815ebe0 legacy=0x7fb858165610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:08.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.698+0000 7fb855d9b640 1 -- 172.21.15.98:0/597356588 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8580ccd10 con 0x7fb85815b7e0 2024-09-19T08:47:08.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.698+0000 7fb855d9b640 1 -- 172.21.15.98:0/597356588 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fb84c001850 con 0x7fb85815b7e0 2024-09-19T08:47:08.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.698+0000 7fb856d9d640 1 --2- 172.21.15.98:0/597356588 >> v2:172.21.15.139:3300/0 conn(0x7fb8580cea90 0x7fb8580cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:47:08.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.698+0000 7fb855d9b640 1 -- 172.21.15.98:0/597356588 >> v1:172.21.15.98:6789/0 conn(0x7fb85815b7e0 legacy=0x7fb85815e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:08.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.698+0000 7fb855d9b640 1 --2- 172.21.15.98:0/597356588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8400039c0 0x7fb840003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:08.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.698+0000 7fb855d9b640 1 --2- 172.21.15.98:0/597356588 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb840004810 0x7fb840004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:08.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.698+0000 7fb855d9b640 1 --2- 172.21.15.98:0/597356588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8400051d0 0x7fb84000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:08.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.698+0000 7fb855d9b640 1 -- 172.21.15.98:0/597356588 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb8580cd600 con 0x7fb8400051d0 2024-09-19T08:47:08.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.698+0000 7fb855d9b640 1 -- 172.21.15.98:0/597356588 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb84000d210 con 0x7fb8400039c0 2024-09-19T08:47:08.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.698+0000 7fb855d9b640 1 -- 172.21.15.98:0/597356588 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb84000d410 con 0x7fb840004810 2024-09-19T08:47:08.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.698+0000 7fb85759e640 1 --2- 172.21.15.98:0/597356588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8400051d0 0x7fb84000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:08.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.698+0000 7fb856d9d640 1 --2- 172.21.15.98:0/597356588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8400039c0 0x7fb840003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:08.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.698+0000 7fb85659c640 1 --2- 172.21.15.98:0/597356588 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb840004810 0x7fb840004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:08.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.698+0000 7fb85759e640 1 -- 172.21.15.98:0/597356588 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb840004810 msgr2=0x7fb840004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:08.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.698+0000 7fb85759e640 1 --2- 172.21.15.98:0/597356588 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb840004810 0x7fb840004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:08.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.698+0000 7fb85759e640 1 -- 172.21.15.98:0/597356588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8400039c0 msgr2=0x7fb840003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:08.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.698+0000 7fb85759e640 1 --2- 172.21.15.98:0/597356588 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8400039c0 0x7fb840003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:08.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.698+0000 7fb85759e640 1 -- 172.21.15.98:0/597356588 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb840003530 con 0x7fb8400051d0 2024-09-19T08:47:08.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.698+0000 7fb85659c640 1 --2- 172.21.15.98:0/597356588 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb840004810 0x7fb840004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:08.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.702+0000 7fb85759e640 1 -- 172.21.15.98:0/597356588 reap_dead start 2024-09-19T08:47:08.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.702+0000 7fb85759e640 1 --2- 172.21.15.98:0/597356588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8400051d0 0x7fb84000b9d0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fb850007fd0 tx=0x7fb85001c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=373f50aa5e954cc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:08.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.702+0000 7fb855d9b640 1 -- 172.21.15.98:0/597356588 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8500040b0 con 0x7fb8400051d0 2024-09-19T08:47:08.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.702+0000 7fb855d9b640 1 -- 172.21.15.98:0/597356588 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8500a9680 con 0x7fb8400051d0 2024-09-19T08:47:08.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.702+0000 7fb855d9b640 1 -- 172.21.15.98:0/597356588 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8500a9a70 con 0x7fb8400051d0 2024-09-19T08:47:08.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.702+0000 7fb85d1ee640 1 -- 172.21.15.98:0/597356588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8400051d0 msgr2=0x7fb84000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:08.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.702+0000 7fb85d1ee640 1 --2- 172.21.15.98:0/597356588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8400051d0 0x7fb84000b9d0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fb850007fd0 tx=0x7fb85001c680 comp rx=0 tx=0).stop 2024-09-19T08:47:08.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.702+0000 7fb85d1ee640 1 -- 172.21.15.98:0/597356588 shutdown_connections 2024-09-19T08:47:08.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.702+0000 7fb85d1ee640 1 --2- 172.21.15.98:0/597356588 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb8400051d0 0x7fb84000b9d0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:08.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.702+0000 7fb85d1ee640 1 -- 172.21.15.98:0/597356588 >> 172.21.15.98:0/597356588 conn(0x7fb8580b2010 msgr2=0x7fb8580b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:08.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.702+0000 7fb85d1ee640 1 -- 172.21.15.98:0/597356588 shutdown_connections 2024-09-19T08:47:08.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.702+0000 7fb85d1ee640 1 -- 172.21.15.98:0/597356588 wait complete. 2024-09-19T08:47:08.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.702+0000 7fb85d1ee640 1 Processor -- start 2024-09-19T08:47:08.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.702+0000 7fb85d1ee640 1 -- start start 2024-09-19T08:47:08.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.702+0000 7fb85d1ee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8400051d0 0x7fb858155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:08.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.702+0000 7fb85d1ee640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb858152d90 0x7fb858154b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:08.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.702+0000 7fb85d1ee640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb858155090 0x7fb858155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:08.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.702+0000 7fb85d1ee640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb8580ce4c0 con 0x7fb858155090 2024-09-19T08:47:08.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.702+0000 7fb85d1ee640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb8580cd290 con 0x7fb8400051d0 2024-09-19T08:47:08.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.702+0000 7fb85d1ee640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb8580cda90 con 0x7fb858152d90 2024-09-19T08:47:08.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.702+0000 7fb85759e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb858155090 0x7fb858155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:08.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.702+0000 7fb85759e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb858155090 0x7fb858155540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:34996/0 (socket says 172.21.15.98:34996) 2024-09-19T08:47:08.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.702+0000 7fb85759e640 1 -- 172.21.15.98:0/1732989001 learned_addr learned my addr 172.21.15.98:0/1732989001 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:08.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.702+0000 7fb856d9d640 1 --2- 172.21.15.98:0/1732989001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8400051d0 0x7fb858155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:08.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.702+0000 7fb85659c640 1 --2- 172.21.15.98:0/1732989001 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb858152d90 0x7fb858154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:08.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.702+0000 7fb85759e640 1 -- 172.21.15.98:0/1732989001 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb858152d90 msgr2=0x7fb858154b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:08.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.702+0000 7fb85759e640 1 --2- 172.21.15.98:0/1732989001 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb858152d90 0x7fb858154b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:08.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.702+0000 7fb85759e640 1 -- 172.21.15.98:0/1732989001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8400051d0 msgr2=0x7fb858155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:08.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.702+0000 7fb85759e640 1 --2- 172.21.15.98:0/1732989001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8400051d0 0x7fb858155cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:08.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.702+0000 7fb85759e640 1 -- 172.21.15.98:0/1732989001 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb858153500 con 0x7fb858155090 2024-09-19T08:47:08.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.706+0000 7fb85759e640 1 --2- 172.21.15.98:0/1732989001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb858155090 0x7fb858155540 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fb850000f90 tx=0x7fb840005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:08.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.706+0000 7fb83f7fe640 1 -- 172.21.15.98:0/1732989001 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8500b15f0 con 0x7fb858155090 2024-09-19T08:47:08.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.706+0000 7fb83f7fe640 1 -- 172.21.15.98:0/1732989001 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8500b7070 con 0x7fb858155090 2024-09-19T08:47:08.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.706+0000 7fb83f7fe640 1 -- 172.21.15.98:0/1732989001 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8500b19d0 con 0x7fb858155090 2024-09-19T08:47:08.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.706+0000 7fb85d1ee640 1 -- 172.21.15.98:0/1732989001 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb858159320 con 0x7fb858155090 2024-09-19T08:47:08.710 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.706+0000 7fb85d1ee640 1 -- 172.21.15.98:0/1732989001 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8581598e0 con 0x7fb858155090 2024-09-19T08:47:08.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.706+0000 7fb85d1ee640 1 -- 172.21.15.98:0/1732989001 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb85815bc40 con 0x7fb858155090 2024-09-19T08:47:08.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.706+0000 7fb83f7fe640 1 -- 172.21.15.98:0/1732989001 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7fb8500a7070 con 0x7fb858155090 2024-09-19T08:47:08.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.706+0000 7fb83f7fe640 1 --2- 172.21.15.98:0/1732989001 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb844068280 0x7fb84406a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:08.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.706+0000 7fb83f7fe640 1 -- 172.21.15.98:0/1732989001 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 4971+0+0 (secure 0 0 0) 0x7fb85011e600 con 0x7fb858155090 2024-09-19T08:47:08.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.706+0000 7fb856d9d640 1 --2- 172.21.15.98:0/1732989001 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb844068280 0x7fb84406a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:08.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.706+0000 7fb856d9d640 1 --2- 172.21.15.98:0/1732989001 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb844068280 0x7fb84406a740 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb85815a5b0 tx=0x7fb848008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:08.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.710+0000 7fb83f7fe640 1 -- 172.21.15.98:0/1732989001 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb850016390 con 0x7fb858155090 2024-09-19T08:47:08.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.874+0000 7fb85d1ee640 1 -- 172.21.15.98:0/1732989001 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb85806dcc0 con 0x7fb858155090 2024-09-19T08:47:08.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.874+0000 7fb83f7fe640 1 -- 172.21.15.98:0/1732989001 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v162) ==== 93+0+10 (secure 0 0 0) 0x7fb8500f0d40 con 0x7fb858155090 2024-09-19T08:47:08.873 INFO:teuthology.orchestra.run.smithi098.stdout:pg_num: 1 2024-09-19T08:47:08.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.874+0000 7fb85d1ee640 1 -- 172.21.15.98:0/1732989001 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb844068280 msgr2=0x7fb84406a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:08.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.874+0000 7fb85d1ee640 1 --2- 172.21.15.98:0/1732989001 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb844068280 0x7fb84406a740 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb85815a5b0 tx=0x7fb848008040 comp rx=0 tx=0).stop 2024-09-19T08:47:08.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.874+0000 7fb85d1ee640 1 -- 172.21.15.98:0/1732989001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb858155090 msgr2=0x7fb858155540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:08.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.874+0000 7fb85d1ee640 1 --2- 172.21.15.98:0/1732989001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb858155090 0x7fb858155540 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fb850000f90 tx=0x7fb840005840 comp rx=0 tx=0).stop 2024-09-19T08:47:08.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.874+0000 7fb85d1ee640 1 -- 172.21.15.98:0/1732989001 shutdown_connections 2024-09-19T08:47:08.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.874+0000 7fb85d1ee640 1 --2- 172.21.15.98:0/1732989001 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb844068280 0x7fb84406a740 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:08.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.874+0000 7fb85d1ee640 1 --2- 172.21.15.98:0/1732989001 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb858155090 0x7fb858155540 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:08.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.874+0000 7fb85d1ee640 1 --2- 172.21.15.98:0/1732989001 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb858152d90 0x7fb858154b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:08.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.874+0000 7fb85d1ee640 1 --2- 172.21.15.98:0/1732989001 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8400051d0 0x7fb858155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:08.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.874+0000 7fb85d1ee640 1 -- 172.21.15.98:0/1732989001 >> 172.21.15.98:0/1732989001 conn(0x7fb8580b2010 msgr2=0x7fb85815e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:08.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.874+0000 7fb85d1ee640 1 -- 172.21.15.98:0/1732989001 shutdown_connections 2024-09-19T08:47:08.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:08.874+0000 7fb85d1ee640 1 -- 172.21.15.98:0/1732989001 wait complete. 2024-09-19T08:47:08.892 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade' 2024-09-19T08:47:09.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.014+0000 7fb52dd04640 1 Processor -- start 2024-09-19T08:47:09.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.014+0000 7fb52dd04640 1 -- start start 2024-09-19T08:47:09.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.014+0000 7fb52dd04640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fb528141110 0x7fb528165940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:09.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.014+0000 7fb52dd04640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb52815ea10 con 0x7fb528140d30 2024-09-19T08:47:09.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.014+0000 7fb52dd04640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb52815ecf0 con 0x7fb52806d880 2024-09-19T08:47:09.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.014+0000 7fb52dd04640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fb52815e7c0 con 0x7fb528141110 2024-09-19T08:47:09.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.014+0000 7fb5277fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fb528140d30 0x7fb52806d170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40566/0 (socket says 172.21.15.98:40566) 2024-09-19T08:47:09.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.014+0000 7fb5277fe640 1 -- 172.21.15.98:0/790800645 learned_addr learned my addr 172.21.15.98:0/790800645 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:09.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.014+0000 7fb527fff640 1 --2- 172.21.15.98:0/790800645 >> v2:172.21.15.139:3300/0 conn(0x7fb528141110 0x7fb528165940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:09.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.014+0000 7fb5267fc640 1 -- 172.21.15.98:0/790800645 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 647935433 0 0) 0x7fb52815ea10 con 0x7fb528140d30 2024-09-19T08:47:09.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.014+0000 7fb5267fc640 1 -- 172.21.15.98:0/790800645 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb510003530 con 0x7fb528140d30 2024-09-19T08:47:09.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.014+0000 7fb5267fc640 1 -- 172.21.15.98:0/790800645 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 54651227 0 0) 0x7fb510003530 con 0x7fb528140d30 2024-09-19T08:47:09.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.014+0000 7fb5267fc640 1 -- 172.21.15.98:0/790800645 >> v2:172.21.15.139:3300/0 conn(0x7fb528141110 msgr2=0x7fb528165940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:09.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.014+0000 7fb5267fc640 1 --2- 172.21.15.98:0/790800645 >> v2:172.21.15.139:3300/0 conn(0x7fb528141110 0x7fb528165940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:09.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.014+0000 7fb5267fc640 1 -- 172.21.15.98:0/790800645 >> v1:172.21.15.139:6789/0 conn(0x7fb52806d880 legacy=0x7fb5280b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:09.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.014+0000 7fb5267fc640 1 -- 172.21.15.98:0/790800645 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb52815efd0 con 0x7fb528140d30 2024-09-19T08:47:09.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.014+0000 7fb5267fc640 1 -- 172.21.15.98:0/790800645 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fb51c001680 con 0x7fb528140d30 2024-09-19T08:47:09.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.014+0000 7fb5267fc640 1 -- 172.21.15.98:0/790800645 >> v1:172.21.15.98:6789/0 conn(0x7fb528140d30 legacy=0x7fb52806d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:09.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.014+0000 7fb5267fc640 1 --2- 172.21.15.98:0/790800645 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5100039c0 0x7fb510003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:09.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.014+0000 7fb5267fc640 1 --2- 172.21.15.98:0/790800645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb510004810 0x7fb510004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:09.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.014+0000 7fb5267fc640 1 --2- 172.21.15.98:0/790800645 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb5100051d0 0x7fb51000cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:09.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.014+0000 7fb5267fc640 1 -- 172.21.15.98:0/790800645 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb52815e7c0 con 0x7fb510004810 2024-09-19T08:47:09.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.014+0000 7fb5267fc640 1 -- 172.21.15.98:0/790800645 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb51000d1b0 con 0x7fb5100039c0 2024-09-19T08:47:09.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.014+0000 7fb5267fc640 1 -- 172.21.15.98:0/790800645 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb51000d3b0 con 0x7fb5100051d0 2024-09-19T08:47:09.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.014+0000 7fb526ffd640 1 --2- 172.21.15.98:0/790800645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb510004810 0x7fb510004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:09.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.014+0000 7fb5277fe640 1 --2- 172.21.15.98:0/790800645 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5100039c0 0x7fb510003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:09.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.014+0000 7fb527fff640 1 --2- 172.21.15.98:0/790800645 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb5100051d0 0x7fb51000cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:09.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.014+0000 7fb526ffd640 1 -- 172.21.15.98:0/790800645 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb5100051d0 msgr2=0x7fb51000cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:09.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.014+0000 7fb526ffd640 1 --2- 172.21.15.98:0/790800645 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb5100051d0 0x7fb51000cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:09.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.014+0000 7fb526ffd640 1 -- 172.21.15.98:0/790800645 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5100039c0 msgr2=0x7fb510003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:09.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.014+0000 7fb526ffd640 1 --2- 172.21.15.98:0/790800645 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5100039c0 0x7fb510003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:09.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.014+0000 7fb526ffd640 1 -- 172.21.15.98:0/790800645 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb510003530 con 0x7fb510004810 2024-09-19T08:47:09.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.014+0000 7fb527fff640 1 -- 172.21.15.98:0/790800645 reap_dead start 2024-09-19T08:47:09.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.014+0000 7fb5277fe640 1 --2- 172.21.15.98:0/790800645 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5100039c0 0x7fb510003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:09.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.018+0000 7fb526ffd640 1 --2- 172.21.15.98:0/790800645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb510004810 0x7fb510004c90 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fb518001780 tx=0x7fb51801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8123ebbcb605e9ff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:09.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.018+0000 7fb5267fc640 1 -- 172.21.15.98:0/790800645 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5180013a0 con 0x7fb510004810 2024-09-19T08:47:09.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.018+0000 7fb5267fc640 1 -- 172.21.15.98:0/790800645 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5180052f0 con 0x7fb510004810 2024-09-19T08:47:09.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.018+0000 7fb5267fc640 1 -- 172.21.15.98:0/790800645 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb518003840 con 0x7fb510004810 2024-09-19T08:47:09.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.018+0000 7fb52dd04640 1 -- 172.21.15.98:0/790800645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb510004810 msgr2=0x7fb510004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:09.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.018+0000 7fb52dd04640 1 --2- 172.21.15.98:0/790800645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb510004810 0x7fb510004c90 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fb518001780 tx=0x7fb51801c400 comp rx=0 tx=0).stop 2024-09-19T08:47:09.022 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.018+0000 7fb52dd04640 1 -- 172.21.15.98:0/790800645 shutdown_connections 2024-09-19T08:47:09.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.018+0000 7fb52dd04640 1 --2- 172.21.15.98:0/790800645 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb510004810 0x7fb510004c90 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:09.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.018+0000 7fb52dd04640 1 -- 172.21.15.98:0/790800645 >> 172.21.15.98:0/790800645 conn(0x7fb5280b2010 msgr2=0x7fb5280b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:09.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.018+0000 7fb52dd04640 1 -- 172.21.15.98:0/790800645 shutdown_connections 2024-09-19T08:47:09.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.018+0000 7fb52dd04640 1 -- 172.21.15.98:0/790800645 wait complete. 2024-09-19T08:47:09.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.018+0000 7fb52dd04640 1 Processor -- start 2024-09-19T08:47:09.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.018+0000 7fb52dd04640 1 -- start start 2024-09-19T08:47:09.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.018+0000 7fb52dd04640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb510004810 0x7fb5280cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:09.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.018+0000 7fb52dd04640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb5281f2de0 0x7fb5280c5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:09.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.018+0000 7fb52dd04640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5280c5a60 0x7fb52813e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:09.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.018+0000 7fb52dd04640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fb52806d170 con 0x7fb510004810 2024-09-19T08:47:09.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.018+0000 7fb52dd04640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb5280ce1a0 con 0x7fb5280c5a60 2024-09-19T08:47:09.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.018+0000 7fb52dd04640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fb52806d2f0 con 0x7fb5281f2de0 2024-09-19T08:47:09.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.018+0000 7fb5277fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb510004810 0x7fb5280cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:09.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.018+0000 7fb5277fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb510004810 0x7fb5280cc850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35012/0 (socket says 172.21.15.98:35012) 2024-09-19T08:47:09.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.018+0000 7fb5277fe640 1 -- 172.21.15.98:0/398371996 learned_addr learned my addr 172.21.15.98:0/398371996 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:09.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.022+0000 7fb5277fe640 1 -- 172.21.15.98:0/398371996 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb5281f2de0 msgr2=0x7fb5280c5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:09.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.022+0000 7fb5277fe640 1 --2- 172.21.15.98:0/398371996 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb5281f2de0 0x7fb5280c5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:09.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.022+0000 7fb5277fe640 1 -- 172.21.15.98:0/398371996 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5280c5a60 msgr2=0x7fb52813e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:09.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.022+0000 7fb5277fe640 1 --2- 172.21.15.98:0/398371996 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5280c5a60 0x7fb52813e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:09.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.022+0000 7fb5277fe640 1 -- 172.21.15.98:0/398371996 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5280c60a0 con 0x7fb510004810 2024-09-19T08:47:09.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.022+0000 7fb5277fe640 1 --2- 172.21.15.98:0/398371996 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb510004810 0x7fb5280cc850 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fb51c010b40 tx=0x7fb51c0056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:09.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.022+0000 7fb50ffff640 1 -- 172.21.15.98:0/398371996 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51c01b070 con 0x7fb510004810 2024-09-19T08:47:09.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.022+0000 7fb50ffff640 1 -- 172.21.15.98:0/398371996 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb51c011040 con 0x7fb510004810 2024-09-19T08:47:09.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.022+0000 7fb50ffff640 1 -- 172.21.15.98:0/398371996 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51c00edf0 con 0x7fb510004810 2024-09-19T08:47:09.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.022+0000 7fb52dd04640 1 -- 172.21.15.98:0/398371996 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5281f32f0 con 0x7fb510004810 2024-09-19T08:47:09.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.022+0000 7fb52dd04640 1 -- 172.21.15.98:0/398371996 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5281f38b0 con 0x7fb510004810 2024-09-19T08:47:09.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.022+0000 7fb52dd04640 1 -- 172.21.15.98:0/398371996 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb52806bc90 con 0x7fb510004810 2024-09-19T08:47:09.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.022+0000 7fb50ffff640 1 -- 172.21.15.98:0/398371996 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7fb51c0143a0 con 0x7fb510004810 2024-09-19T08:47:09.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.022+0000 7fb50ffff640 1 --2- 172.21.15.98:0/398371996 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb514068250 0x7fb51406a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:09.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.022+0000 7fb50ffff640 1 -- 172.21.15.98:0/398371996 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 4971+0+0 (secure 0 0 0) 0x7fb51c086ca0 con 0x7fb510004810 2024-09-19T08:47:09.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.022+0000 7fb526ffd640 1 --2- 172.21.15.98:0/398371996 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb514068250 0x7fb51406a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:09.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.022+0000 7fb526ffd640 1 --2- 172.21.15.98:0/398371996 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb514068250 0x7fb51406a710 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb518001750 tx=0x7fb518001130 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:09.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.026+0000 7fb50ffff640 1 -- 172.21.15.98:0/398371996 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb51c0593e0 con 0x7fb510004810 2024-09-19T08:47:09.230 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.230+0000 7fb52dd04640 1 -- 172.21.15.98:0/398371996 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7fb528157d70 con 0x7fb510004810 2024-09-19T08:47:09.259 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.258+0000 7fb50ffff640 1 -- 172.21.15.98:0/398371996 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]}]=0 v462) ==== 166+0+0 (secure 0 0 0) 0x7fb51c058ce0 con 0x7fb510004810 2024-09-19T08:47:09.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.262+0000 7fb52dd04640 1 -- 172.21.15.98:0/398371996 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb514068250 msgr2=0x7fb51406a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:09.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.262+0000 7fb52dd04640 1 --2- 172.21.15.98:0/398371996 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb514068250 0x7fb51406a710 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb518001750 tx=0x7fb518001130 comp rx=0 tx=0).stop 2024-09-19T08:47:09.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.262+0000 7fb52dd04640 1 -- 172.21.15.98:0/398371996 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb510004810 msgr2=0x7fb5280cc850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:09.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.262+0000 7fb52dd04640 1 --2- 172.21.15.98:0/398371996 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb510004810 0x7fb5280cc850 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fb51c010b40 tx=0x7fb51c0056e0 comp rx=0 tx=0).stop 2024-09-19T08:47:09.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.262+0000 7fb52dd04640 1 -- 172.21.15.98:0/398371996 shutdown_connections 2024-09-19T08:47:09.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.262+0000 7fb52dd04640 1 --2- 172.21.15.98:0/398371996 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fb514068250 0x7fb51406a710 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:09.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.262+0000 7fb52dd04640 1 --2- 172.21.15.98:0/398371996 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5280c5a60 0x7fb52813e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:09.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.262+0000 7fb52dd04640 1 --2- 172.21.15.98:0/398371996 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fb5281f2de0 0x7fb5280c5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:09.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.262+0000 7fb52dd04640 1 --2- 172.21.15.98:0/398371996 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fb510004810 0x7fb5280cc850 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:09.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.262+0000 7fb52dd04640 1 -- 172.21.15.98:0/398371996 >> 172.21.15.98:0/398371996 conn(0x7fb5280b2010 msgr2=0x7fb52813fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:09.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.262+0000 7fb52dd04640 1 -- 172.21.15.98:0/398371996 shutdown_connections 2024-09-19T08:47:09.262 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.262+0000 7fb52dd04640 1 -- 172.21.15.98:0/398371996 wait complete. 2024-09-19T08:47:09.277 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T08:47:09.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.398+0000 7f75d3eb4640 1 Processor -- start 2024-09-19T08:47:09.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.398+0000 7f75d3eb4640 1 -- start start 2024-09-19T08:47:09.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.402+0000 7f75d3eb4640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f75cc0d3a70 0x7f75cc169100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:09.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.402+0000 7f75d3eb4640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75cc15eae0 con 0x7f75cc0d27c0 2024-09-19T08:47:09.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.402+0000 7f75d3eb4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75cc15edc0 con 0x7f75cc1415a0 2024-09-19T08:47:09.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.402+0000 7f75d3eb4640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f75cc0d3ee0 con 0x7f75cc0d3a70 2024-09-19T08:47:09.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.402+0000 7f75d2451640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f75cc0d3a70 0x7f75cc169100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:09.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.402+0000 7f75d144f640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f75cc0d27c0 0x7f75cc0d3360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40582/0 (socket says 172.21.15.98:40582) 2024-09-19T08:47:09.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.402+0000 7f75d144f640 1 -- 172.21.15.98:0/2379614229 learned_addr learned my addr 172.21.15.98:0/2379614229 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:09.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.402+0000 7f75d2451640 1 -- 172.21.15.98:0/2379614229 >> v1:172.21.15.139:6789/0 conn(0x7f75cc1415a0 legacy=0x7f75cc0d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:09.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.402+0000 7f75d2451640 1 -- 172.21.15.98:0/2379614229 >> v1:172.21.15.98:6789/0 conn(0x7f75cc0d27c0 legacy=0x7f75cc0d3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:09.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.402+0000 7f75d2451640 1 -- 172.21.15.98:0/2379614229 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75cc15f0a0 con 0x7f75cc0d3a70 2024-09-19T08:47:09.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.402+0000 7f75d2451640 1 --2- 172.21.15.98:0/2379614229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75cc0d3a70 0x7f75cc169100 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f75c4009d90 tx=0x7f75c401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447c536beaa16c6b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:09.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.402+0000 7f75d0c4e640 1 -- 172.21.15.98:0/2379614229 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c40b2070 con 0x7f75cc0d3a70 2024-09-19T08:47:09.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.402+0000 7f75d0c4e640 1 -- 172.21.15.98:0/2379614229 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75c40a5960 con 0x7f75cc0d3a70 2024-09-19T08:47:09.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.402+0000 7f75d0c4e640 1 -- 172.21.15.98:0/2379614229 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c40a5d30 con 0x7f75cc0d3a70 2024-09-19T08:47:09.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.402+0000 7f75d3eb4640 1 -- 172.21.15.98:0/2379614229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75cc0d3a70 msgr2=0x7f75cc169100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:09.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.402+0000 7f75d3eb4640 1 --2- 172.21.15.98:0/2379614229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75cc0d3a70 0x7f75cc169100 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f75c4009d90 tx=0x7f75c401c900 comp rx=0 tx=0).stop 2024-09-19T08:47:09.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.402+0000 7f75d3eb4640 1 -- 172.21.15.98:0/2379614229 shutdown_connections 2024-09-19T08:47:09.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.402+0000 7f75d3eb4640 1 --2- 172.21.15.98:0/2379614229 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75cc0d3a70 0x7f75cc169100 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:09.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.402+0000 7f75d3eb4640 1 -- 172.21.15.98:0/2379614229 >> 172.21.15.98:0/2379614229 conn(0x7f75cc074c70 msgr2=0x7f75cc075080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:09.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.402+0000 7f75d3eb4640 1 -- 172.21.15.98:0/2379614229 shutdown_connections 2024-09-19T08:47:09.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.402+0000 7f75d3eb4640 1 -- 172.21.15.98:0/2379614229 wait complete. 2024-09-19T08:47:09.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.406+0000 7f75d3eb4640 1 Processor -- start 2024-09-19T08:47:09.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.406+0000 7f75d3eb4640 1 -- start start 2024-09-19T08:47:09.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.406+0000 7f75d3eb4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75cc0d27c0 0x7f75cc0d1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:09.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.406+0000 7f75d3eb4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75cc0d3a70 0x7f75cc15ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:09.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.406+0000 7f75d3eb4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75cc1415a0 0x7f75cc154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:09.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.406+0000 7f75d3eb4640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f75cc0d42a0 con 0x7f75cc1415a0 2024-09-19T08:47:09.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.406+0000 7f75d3eb4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f75cc0d4120 con 0x7f75cc0d3a70 2024-09-19T08:47:09.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.406+0000 7f75d3eb4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f75cc16be60 con 0x7f75cc0d27c0 2024-09-19T08:47:09.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.406+0000 7f75d2451640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75cc1415a0 0x7f75cc154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:09.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.406+0000 7f75d2451640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75cc1415a0 0x7f75cc154020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35036/0 (socket says 172.21.15.98:35036) 2024-09-19T08:47:09.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.406+0000 7f75d2451640 1 -- 172.21.15.98:0/1486531003 learned_addr learned my addr 172.21.15.98:0/1486531003 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:09.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.406+0000 7f75d1c50640 1 --2- 172.21.15.98:0/1486531003 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75cc0d27c0 0x7f75cc0d1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:09.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.406+0000 7f75d2451640 1 -- 172.21.15.98:0/1486531003 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75cc0d27c0 msgr2=0x7f75cc0d1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:09.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.406+0000 7f75d144f640 1 --2- 172.21.15.98:0/1486531003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75cc0d3a70 0x7f75cc15ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:09.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.406+0000 7f75d2451640 1 --2- 172.21.15.98:0/1486531003 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75cc0d27c0 0x7f75cc0d1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:09.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.406+0000 7f75d2451640 1 -- 172.21.15.98:0/1486531003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75cc0d3a70 msgr2=0x7f75cc15ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:09.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.406+0000 7f75d2451640 1 --2- 172.21.15.98:0/1486531003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75cc0d3a70 0x7f75cc15ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:09.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.406+0000 7f75d2451640 1 -- 172.21.15.98:0/1486531003 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75cc153260 con 0x7f75cc1415a0 2024-09-19T08:47:09.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.406+0000 7f75d144f640 1 --2- 172.21.15.98:0/1486531003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75cc0d3a70 0x7f75cc15ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:09.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.406+0000 7f75d2451640 1 --2- 172.21.15.98:0/1486531003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75cc1415a0 0x7f75cc154020 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f75c401a530 tx=0x7f75c40196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:09.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.406+0000 7f75c27fc640 1 -- 172.21.15.98:0/1486531003 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c40b2050 con 0x7f75cc1415a0 2024-09-19T08:47:09.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.406+0000 7f75c27fc640 1 -- 172.21.15.98:0/1486531003 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75c40be070 con 0x7f75cc1415a0 2024-09-19T08:47:09.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.406+0000 7f75d3eb4640 1 -- 172.21.15.98:0/1486531003 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75cc153540 con 0x7f75cc1415a0 2024-09-19T08:47:09.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.406+0000 7f75c27fc640 1 -- 172.21.15.98:0/1486531003 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c40adb50 con 0x7f75cc1415a0 2024-09-19T08:47:09.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.406+0000 7f75d3eb4640 1 -- 172.21.15.98:0/1486531003 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75cc153b00 con 0x7f75cc1415a0 2024-09-19T08:47:09.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.406+0000 7f75d3eb4640 1 -- 172.21.15.98:0/1486531003 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75cc0d3360 con 0x7f75cc1415a0 2024-09-19T08:47:09.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.410+0000 7f75c27fc640 1 -- 172.21.15.98:0/1486531003 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f75c40adcf0 con 0x7f75cc1415a0 2024-09-19T08:47:09.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.410+0000 7f75c27fc640 1 --2- 172.21.15.98:0/1486531003 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f75b0068250 0x7f75b006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:09.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.410+0000 7f75c27fc640 1 -- 172.21.15.98:0/1486531003 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 4971+0+0 (secure 0 0 0) 0x7f75c411f830 con 0x7f75cc1415a0 2024-09-19T08:47:09.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.410+0000 7f75d1c50640 1 --2- 172.21.15.98:0/1486531003 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f75b0068250 0x7f75b006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:09.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.410+0000 7f75d1c50640 1 --2- 172.21.15.98:0/1486531003 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f75b0068250 0x7f75b006a710 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f75c800a750 tx=0x7f75c8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:09.413 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.414+0000 7f75c27fc640 1 -- 172.21.15.98:0/1486531003 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f75c4016520 con 0x7f75cc1415a0 2024-09-19T08:47:09.575 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.574+0000 7f75d3eb4640 1 -- 172.21.15.98:0/1486531003 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f75cc1693e0 con 0x7f75cc1415a0 2024-09-19T08:47:09.582 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.582+0000 7f75c27fc640 1 -- 172.21.15.98:0/1486531003 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) ==== 122+0+0 (secure 0 0 0) 0x7f75c40f1f70 con 0x7f75cc1415a0 2024-09-19T08:47:09.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.586+0000 7f75d3eb4640 1 -- 172.21.15.98:0/1486531003 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f75b0068250 msgr2=0x7f75b006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:09.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.586+0000 7f75d3eb4640 1 --2- 172.21.15.98:0/1486531003 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f75b0068250 0x7f75b006a710 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f75c800a750 tx=0x7f75c8008040 comp rx=0 tx=0).stop 2024-09-19T08:47:09.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.586+0000 7f75d3eb4640 1 -- 172.21.15.98:0/1486531003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75cc1415a0 msgr2=0x7f75cc154020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:09.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.586+0000 7f75d3eb4640 1 --2- 172.21.15.98:0/1486531003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75cc1415a0 0x7f75cc154020 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f75c401a530 tx=0x7f75c40196a0 comp rx=0 tx=0).stop 2024-09-19T08:47:09.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.586+0000 7f75d3eb4640 1 -- 172.21.15.98:0/1486531003 shutdown_connections 2024-09-19T08:47:09.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.586+0000 7f75d3eb4640 1 --2- 172.21.15.98:0/1486531003 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f75b0068250 0x7f75b006a710 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:09.585 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.586+0000 7f75d3eb4640 1 --2- 172.21.15.98:0/1486531003 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f75cc1415a0 0x7f75cc154020 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:09.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.586+0000 7f75d3eb4640 1 --2- 172.21.15.98:0/1486531003 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f75cc0d3a70 0x7f75cc15ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:09.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.586+0000 7f75d3eb4640 1 --2- 172.21.15.98:0/1486531003 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f75cc0d27c0 0x7f75cc0d1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:09.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.586+0000 7f75d3eb4640 1 -- 172.21.15.98:0/1486531003 >> 172.21.15.98:0/1486531003 conn(0x7f75cc074c70 msgr2=0x7f75cc06c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:09.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.586+0000 7f75d3eb4640 1 -- 172.21.15.98:0/1486531003 shutdown_connections 2024-09-19T08:47:09.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.586+0000 7f75d3eb4640 1 -- 172.21.15.98:0/1486531003 wait complete. 2024-09-19T08:47:09.602 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T08:47:09.602 DEBUG:teuthology.orchestra.run.smithi098:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:47:09.677 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:47:09.759 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T08:47:09.759 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:47:09.760 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:47:09.766 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15846 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T08:47:09.766 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:47:09.766 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:47:09.771 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T08:47:09.772 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:47:09.772 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:47:09.813 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15846 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T08:47:09.813 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:47:09.813 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:47:09.822 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T08:47:09.823 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-19T08:47:09.823 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:47:09.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.982+0000 7ff1edf2c640 1 Processor -- start 2024-09-19T08:47:09.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.982+0000 7ff1edf2c640 1 -- start start 2024-09-19T08:47:09.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.986+0000 7ff1edf2c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1e815c550 0x7ff1e806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:09.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.986+0000 7ff1edf2c640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1e806d310 con 0x7ff1e80cfad0 2024-09-19T08:47:09.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.986+0000 7ff1edf2c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1e806d5f0 con 0x7ff1e806c1d0 2024-09-19T08:47:09.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.986+0000 7ff1edf2c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff1e806c610 con 0x7ff1e815c550 2024-09-19T08:47:09.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.986+0000 7ff1e7fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7ff1e80cfad0 0x7ff1e80723a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40590/0 (socket says 172.21.15.98:40590) 2024-09-19T08:47:09.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.986+0000 7ff1e7fff640 1 -- 172.21.15.98:0/2915717182 learned_addr learned my addr 172.21.15.98:0/2915717182 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:09.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.986+0000 7ff1e77fe640 1 --2- 172.21.15.98:0/2915717182 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1e815c550 0x7ff1e806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:09.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.986+0000 7ff1e67fc640 1 -- 172.21.15.98:0/2915717182 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 484878941 0 0) 0x7ff1e806d310 con 0x7ff1e80cfad0 2024-09-19T08:47:09.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.986+0000 7ff1e67fc640 1 -- 172.21.15.98:0/2915717182 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1d0003530 con 0x7ff1e80cfad0 2024-09-19T08:47:09.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.986+0000 7ff1e67fc640 1 -- 172.21.15.98:0/2915717182 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1452389689 0 0) 0x7ff1d0003530 con 0x7ff1e80cfad0 2024-09-19T08:47:09.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.986+0000 7ff1e67fc640 1 -- 172.21.15.98:0/2915717182 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1e815c550 msgr2=0x7ff1e806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:09.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.986+0000 7ff1e67fc640 1 --2- 172.21.15.98:0/2915717182 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1e815c550 0x7ff1e806bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:09.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.986+0000 7ff1e67fc640 1 -- 172.21.15.98:0/2915717182 >> v1:172.21.15.139:6789/0 conn(0x7ff1e806c1d0 legacy=0x7ff1e80cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:09.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.986+0000 7ff1e67fc640 1 -- 172.21.15.98:0/2915717182 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1e806d8d0 con 0x7ff1e80cfad0 2024-09-19T08:47:09.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.986+0000 7ff1e67fc640 1 -- 172.21.15.98:0/2915717182 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7ff1e0001a00 con 0x7ff1e80cfad0 2024-09-19T08:47:09.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.986+0000 7ff1e67fc640 1 -- 172.21.15.98:0/2915717182 >> v1:172.21.15.98:6789/0 conn(0x7ff1e80cfad0 legacy=0x7ff1e80723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:09.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.986+0000 7ff1e67fc640 1 --2- 172.21.15.98:0/2915717182 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1e815c550 0x7ff1d00039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:09.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.986+0000 7ff1e67fc640 1 --2- 172.21.15.98:0/2915717182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1d0003f70 0x7ff1d0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:09.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.986+0000 7ff1e67fc640 1 --2- 172.21.15.98:0/2915717182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1d0004d50 0x7ff1d00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:09.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.986+0000 7ff1e67fc640 1 -- 172.21.15.98:0/2915717182 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e806c610 con 0x7ff1d0004d50 2024-09-19T08:47:09.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.986+0000 7ff1e67fc640 1 -- 172.21.15.98:0/2915717182 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d000bab0 con 0x7ff1d0003f70 2024-09-19T08:47:09.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.986+0000 7ff1e67fc640 1 -- 172.21.15.98:0/2915717182 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff1d000bcb0 con 0x7ff1e815c550 2024-09-19T08:47:09.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.986+0000 7ff1e7fff640 1 --2- 172.21.15.98:0/2915717182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1d0004d50 0x7ff1d00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:09.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.986+0000 7ff1e77fe640 1 --2- 172.21.15.98:0/2915717182 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1e815c550 0x7ff1d00039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:09.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.986+0000 7ff1e6ffd640 1 --2- 172.21.15.98:0/2915717182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1d0003f70 0x7ff1d0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:09.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.986+0000 7ff1e7fff640 1 -- 172.21.15.98:0/2915717182 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1e815c550 msgr2=0x7ff1d00039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:09.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.986+0000 7ff1e7fff640 1 --2- 172.21.15.98:0/2915717182 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1e815c550 0x7ff1d00039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:09.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.986+0000 7ff1e7fff640 1 -- 172.21.15.98:0/2915717182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1d0003f70 msgr2=0x7ff1d0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:09.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.986+0000 7ff1e7fff640 1 --2- 172.21.15.98:0/2915717182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1d0003f70 0x7ff1d0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:09.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.986+0000 7ff1e7fff640 1 -- 172.21.15.98:0/2915717182 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d0003530 con 0x7ff1d0004d50 2024-09-19T08:47:09.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.986+0000 7ff1e6ffd640 1 --2- 172.21.15.98:0/2915717182 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1d0003f70 0x7ff1d0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:09.990 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.986+0000 7ff1e77fe640 1 --2- 172.21.15.98:0/2915717182 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1e815c550 0x7ff1d00039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:47:09.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.990+0000 7ff1e7fff640 1 --2- 172.21.15.98:0/2915717182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1d0004d50 0x7ff1d00051d0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7ff1e000a840 tx=0x7ff1e001d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25646c8632909a63 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:09.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.990+0000 7ff1e67fc640 1 -- 172.21.15.98:0/2915717182 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e000a460 con 0x7ff1d0004d50 2024-09-19T08:47:09.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.990+0000 7ff1e67fc640 1 -- 172.21.15.98:0/2915717182 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1e00ad630 con 0x7ff1d0004d50 2024-09-19T08:47:09.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.990+0000 7ff1e67fc640 1 -- 172.21.15.98:0/2915717182 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e00ada60 con 0x7ff1d0004d50 2024-09-19T08:47:09.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.990+0000 7ff1edf2c640 1 -- 172.21.15.98:0/2915717182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1d0004d50 msgr2=0x7ff1d00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:09.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.990+0000 7ff1edf2c640 1 --2- 172.21.15.98:0/2915717182 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1d0004d50 0x7ff1d00051d0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7ff1e000a840 tx=0x7ff1e001d360 comp rx=0 tx=0).stop 2024-09-19T08:47:09.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.990+0000 7ff1e7fff640 1 -- 172.21.15.98:0/2915717182 reap_dead start 2024-09-19T08:47:09.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.990+0000 7ff1edf2c640 1 -- 172.21.15.98:0/2915717182 shutdown_connections 2024-09-19T08:47:09.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.990+0000 7ff1edf2c640 1 -- 172.21.15.98:0/2915717182 >> 172.21.15.98:0/2915717182 conn(0x7ff1e80b2010 msgr2=0x7ff1e80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:09.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.990+0000 7ff1edf2c640 1 -- 172.21.15.98:0/2915717182 shutdown_connections 2024-09-19T08:47:09.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.990+0000 7ff1edf2c640 1 -- 172.21.15.98:0/2915717182 wait complete. 2024-09-19T08:47:09.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.990+0000 7ff1edf2c640 1 Processor -- start 2024-09-19T08:47:09.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.990+0000 7ff1edf2c640 1 -- start start 2024-09-19T08:47:09.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.994+0000 7ff1edf2c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1d0004d50 0x7ff1e8073630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:09.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.994+0000 7ff1edf2c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1e8142850 0x7ff1e8142cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:09.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.994+0000 7ff1edf2c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1e8152bc0 0x7ff1e8153040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:09.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.994+0000 7ff1edf2c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e815cab0 con 0x7ff1e8142850 2024-09-19T08:47:09.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.994+0000 7ff1edf2c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e815c930 con 0x7ff1d0004d50 2024-09-19T08:47:09.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.994+0000 7ff1edf2c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff1e806bc90 con 0x7ff1e8152bc0 2024-09-19T08:47:09.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.994+0000 7ff1e6ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1e8142850 0x7ff1e8142cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:09.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.994+0000 7ff1e6ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1e8142850 0x7ff1e8142cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35060/0 (socket says 172.21.15.98:35060) 2024-09-19T08:47:09.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.994+0000 7ff1e6ffd640 1 -- 172.21.15.98:0/3007818230 learned_addr learned my addr 172.21.15.98:0/3007818230 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:09.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.994+0000 7ff1e77fe640 1 --2- 172.21.15.98:0/3007818230 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1d0004d50 0x7ff1e8073630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:09.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.994+0000 7ff1e7fff640 1 --2- 172.21.15.98:0/3007818230 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1e8152bc0 0x7ff1e8153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:09.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.994+0000 7ff1e6ffd640 1 -- 172.21.15.98:0/3007818230 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1e8152bc0 msgr2=0x7ff1e8153040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:09.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.994+0000 7ff1e6ffd640 1 --2- 172.21.15.98:0/3007818230 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1e8152bc0 0x7ff1e8153040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:09.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.994+0000 7ff1e6ffd640 1 -- 172.21.15.98:0/3007818230 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1d0004d50 msgr2=0x7ff1e8073630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:09.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.994+0000 7ff1e6ffd640 1 --2- 172.21.15.98:0/3007818230 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1d0004d50 0x7ff1e8073630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:09.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.994+0000 7ff1e6ffd640 1 -- 172.21.15.98:0/3007818230 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1e81431f0 con 0x7ff1e8142850 2024-09-19T08:47:09.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.994+0000 7ff1e6ffd640 1 --2- 172.21.15.98:0/3007818230 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1e8142850 0x7ff1e8142cb0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00d5a0 tx=0x7ff1dc00da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:09.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.994+0000 7ff1cffff640 1 -- 172.21.15.98:0/3007818230 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc003620 con 0x7ff1e8142850 2024-09-19T08:47:09.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.994+0000 7ff1cffff640 1 -- 172.21.15.98:0/3007818230 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1dc017070 con 0x7ff1e8142850 2024-09-19T08:47:09.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.994+0000 7ff1cffff640 1 -- 172.21.15.98:0/3007818230 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc003a00 con 0x7ff1e8142850 2024-09-19T08:47:09.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.994+0000 7ff1edf2c640 1 -- 172.21.15.98:0/3007818230 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1e8151760 con 0x7ff1e8142850 2024-09-19T08:47:09.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.994+0000 7ff1edf2c640 1 -- 172.21.15.98:0/3007818230 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1e8151d20 con 0x7ff1e8142850 2024-09-19T08:47:09.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.994+0000 7ff1edf2c640 1 -- 172.21.15.98:0/3007818230 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1e80cff80 con 0x7ff1e8142850 2024-09-19T08:47:09.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.994+0000 7ff1cffff640 1 -- 172.21.15.98:0/3007818230 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7ff1dc003ba0 con 0x7ff1e8142850 2024-09-19T08:47:09.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.994+0000 7ff1cffff640 1 --2- 172.21.15.98:0/3007818230 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff1d4068280 0x7ff1d406a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:09.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.994+0000 7ff1cffff640 1 -- 172.21.15.98:0/3007818230 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 4971+0+0 (secure 0 0 0) 0x7ff1dc0841d0 con 0x7ff1e8142850 2024-09-19T08:47:09.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.994+0000 7ff1e77fe640 1 --2- 172.21.15.98:0/3007818230 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff1d4068280 0x7ff1d406a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:09.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.998+0000 7ff1e77fe640 1 --2- 172.21.15.98:0/3007818230 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff1d4068280 0x7ff1d406a740 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8009ab0 tx=0x7ff1d8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:09.999 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:09.998+0000 7ff1cffff640 1 -- 172.21.15.98:0/3007818230 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff1dc0569d0 con 0x7ff1e8142850 2024-09-19T08:47:10.190 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.190+0000 7ff1edf2c640 1 -- 172.21.15.98:0/3007818230 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1e815a580 con 0x7ff1e8142850 2024-09-19T08:47:10.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.190+0000 7ff1cffff640 1 -- 172.21.15.98:0/3007818230 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+3917 (secure 0 0 0) 0x7ff1dc0562d0 con 0x7ff1e8142850 2024-09-19T08:47:10.191 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:47:10.191 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":94,"btime":"2024-09-19T08:47:05:224967+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/1752784745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":1752784745},{"type":"v1","addr":"172.21.15.139:6834","nonce":1752784745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":6212,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3564493643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6833","nonce":3564493643},{"type":"v1","addr":"172.21.15.139:6835","nonce":3564493643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":8763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6834/3374714194","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":3374714194},{"type":"v1","addr":"172.21.15.98:6834","nonce":3374714194}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":9096,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/3579672973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6833","nonce":3579672973},{"type":"v1","addr":"172.21.15.98:6835","nonce":3579672973}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[]} 2024-09-19T08:47:10.192 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 94 2024-09-19T08:47:10.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.194+0000 7ff1edf2c640 1 -- 172.21.15.98:0/3007818230 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff1d4068280 msgr2=0x7ff1d406a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:10.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.194+0000 7ff1edf2c640 1 --2- 172.21.15.98:0/3007818230 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff1d4068280 0x7ff1d406a740 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8009ab0 tx=0x7ff1d8008040 comp rx=0 tx=0).stop 2024-09-19T08:47:10.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.194+0000 7ff1edf2c640 1 -- 172.21.15.98:0/3007818230 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1e8142850 msgr2=0x7ff1e8142cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:10.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.194+0000 7ff1edf2c640 1 --2- 172.21.15.98:0/3007818230 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1e8142850 0x7ff1e8142cb0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00d5a0 tx=0x7ff1dc00da60 comp rx=0 tx=0).stop 2024-09-19T08:47:10.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.194+0000 7ff1edf2c640 1 -- 172.21.15.98:0/3007818230 shutdown_connections 2024-09-19T08:47:10.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.194+0000 7ff1edf2c640 1 --2- 172.21.15.98:0/3007818230 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff1d4068280 0x7ff1d406a740 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:10.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.194+0000 7ff1edf2c640 1 --2- 172.21.15.98:0/3007818230 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff1e8152bc0 0x7ff1e8153040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:10.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.194+0000 7ff1edf2c640 1 --2- 172.21.15.98:0/3007818230 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff1e8142850 0x7ff1e8142cb0 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:10.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.194+0000 7ff1edf2c640 1 --2- 172.21.15.98:0/3007818230 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff1d0004d50 0x7ff1e8073630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:10.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.194+0000 7ff1edf2c640 1 -- 172.21.15.98:0/3007818230 >> 172.21.15.98:0/3007818230 conn(0x7ff1e80b2010 msgr2=0x7ff1e80cc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:10.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.194+0000 7ff1edf2c640 1 -- 172.21.15.98:0/3007818230 shutdown_connections 2024-09-19T08:47:10.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.194+0000 7ff1edf2c640 1 -- 172.21.15.98:0/3007818230 wait complete. 2024-09-19T08:47:10.210 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-19T08:47:10.210 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-19T08:47:10.210 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-19T08:47:10.210 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:47:10.211 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-19T08:47:10.211 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-19T08:47:10.211 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-19T08:47:10.211 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:47:10.211 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-19T08:47:10.212 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-19T08:47:10.212 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-19T08:47:10.212 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:47:10.212 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-19T08:47:10.212 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-19T08:47:10.212 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-19T08:47:10.213 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:47:10.219 INFO:tasks.ceph.mds.a:Stopped 2024-09-19T08:47:10.219 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-19T08:47:10.224 INFO:tasks.ceph.mds.c:Stopped 2024-09-19T08:47:10.224 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-19T08:47:10.226 INFO:tasks.ceph.mds.b:Stopped 2024-09-19T08:47:10.226 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-19T08:47:10.231 INFO:tasks.ceph.mds.d:Stopped 2024-09-19T08:47:10.231 DEBUG:teuthology.orchestra.run.smithi139:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-19T08:47:10.268 INFO:tasks.ceph.mds.a:Started 2024-09-19T08:47:10.269 DEBUG:teuthology.parallel:result is None 2024-09-19T08:47:10.269 INFO:tasks.ceph.mds.c:Started 2024-09-19T08:47:10.270 DEBUG:teuthology.parallel:result is None 2024-09-19T08:47:10.278 INFO:tasks.ceph.mds.b:Started 2024-09-19T08:47:10.279 DEBUG:teuthology.parallel:result is None 2024-09-19T08:47:10.280 INFO:tasks.ceph.mds.d:Started 2024-09-19T08:47:10.280 DEBUG:teuthology.parallel:result is None 2024-09-19T08:47:10.280 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-19T08:47:10.328 INFO:tasks.ceph.mds.c.smithi098.stdout:starting mds.c at 2024-09-19T08:47:10.333 INFO:tasks.ceph.mds.a.smithi098.stdout:starting mds.a at 2024-09-19T08:47:10.349 INFO:tasks.ceph.mds.d.smithi139.stdout:starting mds.d at 2024-09-19T08:47:10.349 INFO:tasks.ceph.mds.b.smithi139.stdout:starting mds.b at 2024-09-19T08:47:10.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.398+0000 7f881824e640 1 Processor -- start 2024-09-19T08:47:10.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.398+0000 7f881824e640 1 -- start start 2024-09-19T08:47:10.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.402+0000 7f881824e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f881015c550 0x7f881006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:10.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.402+0000 7f881824e640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f881006d310 con 0x7f881006c1d0 2024-09-19T08:47:10.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.402+0000 7f881824e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f881006d5f0 con 0x7f88100cfad0 2024-09-19T08:47:10.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.402+0000 7f881824e640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f881006c640 con 0x7f881015c550 2024-09-19T08:47:10.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.402+0000 7f8815fea640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f881015c550 0x7f881006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:10.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.402+0000 7f88157e9640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f881006c1d0 0x7f88100cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40608/0 (socket says 172.21.15.98:40608) 2024-09-19T08:47:10.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.402+0000 7f88157e9640 1 -- 172.21.15.98:0/2663382083 learned_addr learned my addr 172.21.15.98:0/2663382083 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:10.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.402+0000 7f8815fea640 1 -- 172.21.15.98:0/2663382083 >> v1:172.21.15.139:6789/0 conn(0x7f88100cfad0 legacy=0x7f88100723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:10.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.402+0000 7f8815fea640 1 -- 172.21.15.98:0/2663382083 >> v1:172.21.15.98:6789/0 conn(0x7f881006c1d0 legacy=0x7f88100cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:10.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.402+0000 7f8815fea640 1 -- 172.21.15.98:0/2663382083 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881006d8d0 con 0x7f881015c550 2024-09-19T08:47:10.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.402+0000 7f8814fe8640 1 -- 172.21.15.98:0/2663382083 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 649629177 0 0) 0x7f881006d310 con 0x7f881006c1d0 2024-09-19T08:47:10.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.402+0000 7f8815fea640 1 --2- 172.21.15.98:0/2663382083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f881015c550 0x7f881006bc90 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f880c0097c0 tx=0x7f880c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5db5c4251c97ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:10.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.402+0000 7f8814fe8640 1 -- 172.21.15.98:0/2663382083 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c0b0070 con 0x7f881015c550 2024-09-19T08:47:10.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.402+0000 7f8814fe8640 1 -- 172.21.15.98:0/2663382083 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f880c0ab600 con 0x7f881015c550 2024-09-19T08:47:10.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.402+0000 7f8814fe8640 1 -- 172.21.15.98:0/2663382083 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c0ab9d0 con 0x7f881015c550 2024-09-19T08:47:10.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.402+0000 7f881824e640 1 -- 172.21.15.98:0/2663382083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f881015c550 msgr2=0x7f881006bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:10.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.402+0000 7f881824e640 1 --2- 172.21.15.98:0/2663382083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f881015c550 0x7f881006bc90 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f880c0097c0 tx=0x7f880c01c460 comp rx=0 tx=0).stop 2024-09-19T08:47:10.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.402+0000 7f881824e640 1 -- 172.21.15.98:0/2663382083 shutdown_connections 2024-09-19T08:47:10.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.402+0000 7f881824e640 1 --2- 172.21.15.98:0/2663382083 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f881015c550 0x7f881006bc90 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:10.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.402+0000 7f881824e640 1 -- 172.21.15.98:0/2663382083 >> 172.21.15.98:0/2663382083 conn(0x7f88100b2010 msgr2=0x7f88100b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:10.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.402+0000 7f881824e640 1 -- 172.21.15.98:0/2663382083 shutdown_connections 2024-09-19T08:47:10.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.402+0000 7f881824e640 1 -- 172.21.15.98:0/2663382083 wait complete. 2024-09-19T08:47:10.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.406+0000 7f881824e640 1 Processor -- start 2024-09-19T08:47:10.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.406+0000 7f881824e640 1 -- start start 2024-09-19T08:47:10.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.406+0000 7f881824e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f881006c1d0 0x7f881014bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:10.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.406+0000 7f881824e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88100cfad0 0x7f8810158a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:10.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.406+0000 7f881824e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f881015c550 0x7f88101f74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:10.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.406+0000 7f881824e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8810073690 con 0x7f881006c1d0 2024-09-19T08:47:10.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.406+0000 7f881824e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8810073510 con 0x7f88100cfad0 2024-09-19T08:47:10.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.406+0000 7f881824e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f881016a240 con 0x7f881015c550 2024-09-19T08:47:10.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.406+0000 7f8815fea640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f881006c1d0 0x7f881014bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:10.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.406+0000 7f8815fea640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f881006c1d0 0x7f881014bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35096/0 (socket says 172.21.15.98:35096) 2024-09-19T08:47:10.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.406+0000 7f8815fea640 1 -- 172.21.15.98:0/1903593647 learned_addr learned my addr 172.21.15.98:0/1903593647 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:10.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.406+0000 7f88157e9640 1 --2- 172.21.15.98:0/1903593647 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88100cfad0 0x7f8810158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:10.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.406+0000 7f88167eb640 1 --2- 172.21.15.98:0/1903593647 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f881015c550 0x7f88101f74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:10.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.406+0000 7f8815fea640 1 -- 172.21.15.98:0/1903593647 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f881015c550 msgr2=0x7f88101f74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:10.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.406+0000 7f8815fea640 1 --2- 172.21.15.98:0/1903593647 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f881015c550 0x7f88101f74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:10.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.406+0000 7f8815fea640 1 -- 172.21.15.98:0/1903593647 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88100cfad0 msgr2=0x7f8810158a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:10.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.406+0000 7f8815fea640 1 --2- 172.21.15.98:0/1903593647 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88100cfad0 0x7f8810158a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:10.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.406+0000 7f8815fea640 1 -- 172.21.15.98:0/1903593647 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8810159440 con 0x7f881006c1d0 2024-09-19T08:47:10.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.406+0000 7f8815fea640 1 --2- 172.21.15.98:0/1903593647 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f881006c1d0 0x7f881014bf00 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f880c0ae040 tx=0x7f880c01aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:10.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.406+0000 7f88067fc640 1 -- 172.21.15.98:0/1903593647 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c0b0050 con 0x7f881006c1d0 2024-09-19T08:47:10.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.406+0000 7f88067fc640 1 -- 172.21.15.98:0/1903593647 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f880c0bc070 con 0x7f881006c1d0 2024-09-19T08:47:10.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.406+0000 7f88067fc640 1 -- 172.21.15.98:0/1903593647 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c0b5a70 con 0x7f881006c1d0 2024-09-19T08:47:10.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.406+0000 7f881824e640 1 -- 172.21.15.98:0/1903593647 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88100b3540 con 0x7f881006c1d0 2024-09-19T08:47:10.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.406+0000 7f881824e640 1 -- 172.21.15.98:0/1903593647 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88100b3b00 con 0x7f881006c1d0 2024-09-19T08:47:10.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.406+0000 7f881824e640 1 -- 172.21.15.98:0/1903593647 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f881006bc90 con 0x7f881006c1d0 2024-09-19T08:47:10.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.406+0000 7f88067fc640 1 -- 172.21.15.98:0/1903593647 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f880c0ab4c0 con 0x7f881006c1d0 2024-09-19T08:47:10.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.410+0000 7f88067fc640 1 --2- 172.21.15.98:0/1903593647 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f87f4068250 0x7f87f406a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:10.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.410+0000 7f88067fc640 1 -- 172.21.15.98:0/1903593647 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 4971+0+0 (secure 0 0 0) 0x7f880c11ecf0 con 0x7f881006c1d0 2024-09-19T08:47:10.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.410+0000 7f88157e9640 1 --2- 172.21.15.98:0/1903593647 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f87f4068250 0x7f87f406a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:10.409 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.410+0000 7f88157e9640 1 --2- 172.21.15.98:0/1903593647 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f87f4068250 0x7f87f406a710 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f88100cd8a0 tx=0x7f8800009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:10.412 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.414+0000 7f88067fc640 1 -- 172.21.15.98:0/1903593647 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f880c016050 con 0x7f881006c1d0 2024-09-19T08:47:10.586 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:10.586+0000 7f881824e640 1 -- 172.21.15.98:0/1903593647 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f8810073e50 con 0x7f881006c1d0 2024-09-19T08:47:11.278 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.278+0000 7f88067fc640 1 -- 172.21.15.98:0/1903593647 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v164) ==== 97+0+0 (secure 0 0 0) 0x7f880c0f1430 con 0x7f881006c1d0 2024-09-19T08:47:11.278 INFO:teuthology.orchestra.run.smithi098.stderr: removed all blocklist entries 2024-09-19T08:47:11.282 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.282+0000 7f881824e640 1 -- 172.21.15.98:0/1903593647 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f87f4068250 msgr2=0x7f87f406a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:11.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.282+0000 7f881824e640 1 --2- 172.21.15.98:0/1903593647 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f87f4068250 0x7f87f406a710 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f88100cd8a0 tx=0x7f8800009040 comp rx=0 tx=0).stop 2024-09-19T08:47:11.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.282+0000 7f881824e640 1 -- 172.21.15.98:0/1903593647 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f881006c1d0 msgr2=0x7f881014bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:11.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.282+0000 7f881824e640 1 --2- 172.21.15.98:0/1903593647 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f881006c1d0 0x7f881014bf00 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f880c0ae040 tx=0x7f880c01aca0 comp rx=0 tx=0).stop 2024-09-19T08:47:11.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.282+0000 7f881824e640 1 -- 172.21.15.98:0/1903593647 shutdown_connections 2024-09-19T08:47:11.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.282+0000 7f881824e640 1 --2- 172.21.15.98:0/1903593647 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f87f4068250 0x7f87f406a710 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:11.283 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.282+0000 7f881824e640 1 --2- 172.21.15.98:0/1903593647 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f881015c550 0x7f88101f74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:11.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.282+0000 7f881824e640 1 --2- 172.21.15.98:0/1903593647 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88100cfad0 0x7f8810158a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:11.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.282+0000 7f881824e640 1 --2- 172.21.15.98:0/1903593647 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f881006c1d0 0x7f881014bf00 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:11.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.282+0000 7f881824e640 1 -- 172.21.15.98:0/1903593647 >> 172.21.15.98:0/1903593647 conn(0x7f88100b2010 msgr2=0x7f8810152800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:11.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.282+0000 7f881824e640 1 -- 172.21.15.98:0/1903593647 shutdown_connections 2024-09-19T08:47:11.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.282+0000 7f881824e640 1 -- 172.21.15.98:0/1903593647 wait complete. 2024-09-19T08:47:11.304 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-19T08:47:11.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.422+0000 7ff3aab6a640 1 Processor -- start 2024-09-19T08:47:11.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.422+0000 7ff3aab6a640 1 -- start start 2024-09-19T08:47:11.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.426+0000 7ff3aab6a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff3a4158fd0 0x7ff3a40b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:11.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.426+0000 7ff3aab6a640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3a40ccc60 con 0x7ff3a40b4360 2024-09-19T08:47:11.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.426+0000 7ff3aab6a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3a40ccf40 con 0x7ff3a406c3a0 2024-09-19T08:47:11.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.426+0000 7ff3aab6a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff3a41593b0 con 0x7ff3a4158fd0 2024-09-19T08:47:11.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.426+0000 7ff3a3fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7ff3a40b4360 0x7ff3a406bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40622/0 (socket says 172.21.15.98:40622) 2024-09-19T08:47:11.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.426+0000 7ff3a3fff640 1 -- 172.21.15.98:0/3897963427 learned_addr learned my addr 172.21.15.98:0/3897963427 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:11.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.426+0000 7ff3a8906640 1 --2- 172.21.15.98:0/3897963427 >> v2:172.21.15.139:3300/0 conn(0x7ff3a4158fd0 0x7ff3a40b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:11.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.426+0000 7ff3a37fe640 1 -- 172.21.15.98:0/3897963427 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3017703311 0 0) 0x7ff3a40ccc60 con 0x7ff3a40b4360 2024-09-19T08:47:11.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.426+0000 7ff3a37fe640 1 -- 172.21.15.98:0/3897963427 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff38c003530 con 0x7ff3a40b4360 2024-09-19T08:47:11.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.426+0000 7ff3a37fe640 1 -- 172.21.15.98:0/3897963427 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4095634159 0 0) 0x7ff38c003530 con 0x7ff3a40b4360 2024-09-19T08:47:11.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.426+0000 7ff3a37fe640 1 -- 172.21.15.98:0/3897963427 >> v2:172.21.15.139:3300/0 conn(0x7ff3a4158fd0 msgr2=0x7ff3a40b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:11.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.426+0000 7ff3a37fe640 1 --2- 172.21.15.98:0/3897963427 >> v2:172.21.15.139:3300/0 conn(0x7ff3a4158fd0 0x7ff3a40b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:11.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.426+0000 7ff3a37fe640 1 -- 172.21.15.98:0/3897963427 >> v1:172.21.15.139:6789/0 conn(0x7ff3a406c3a0 legacy=0x7ff3a40ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:11.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.426+0000 7ff3a37fe640 1 -- 172.21.15.98:0/3897963427 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a40cd220 con 0x7ff3a40b4360 2024-09-19T08:47:11.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.426+0000 7ff3a37fe640 1 -- 172.21.15.98:0/3897963427 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7ff398001680 con 0x7ff3a40b4360 2024-09-19T08:47:11.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.426+0000 7ff3a37fe640 1 -- 172.21.15.98:0/3897963427 >> v1:172.21.15.98:6789/0 conn(0x7ff3a40b4360 legacy=0x7ff3a406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:11.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.426+0000 7ff3a37fe640 1 --2- 172.21.15.98:0/3897963427 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff38c0039c0 0x7ff38c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:11.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.426+0000 7ff3a37fe640 1 --2- 172.21.15.98:0/3897963427 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff38c004810 0x7ff38c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:11.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.426+0000 7ff3a37fe640 1 --2- 172.21.15.98:0/3897963427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff38c0051d0 0x7ff38c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:11.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.426+0000 7ff3a37fe640 1 -- 172.21.15.98:0/3897963427 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a41593b0 con 0x7ff38c0039c0 2024-09-19T08:47:11.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.426+0000 7ff3a37fe640 1 -- 172.21.15.98:0/3897963427 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c00c170 con 0x7ff38c0051d0 2024-09-19T08:47:11.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.426+0000 7ff3a37fe640 1 -- 172.21.15.98:0/3897963427 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff38c00c370 con 0x7ff38c004810 2024-09-19T08:47:11.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.426+0000 7ff3a8906640 1 --2- 172.21.15.98:0/3897963427 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff38c0039c0 0x7ff38c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:11.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.426+0000 7ff3a8906640 1 -- 172.21.15.98:0/3897963427 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff38c004810 msgr2=0x7ff38c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:11.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.426+0000 7ff3a3fff640 1 --2- 172.21.15.98:0/3897963427 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff38c004810 0x7ff38c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:11.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.426+0000 7ff3a8906640 1 --2- 172.21.15.98:0/3897963427 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff38c004810 0x7ff38c004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:11.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.426+0000 7ff3a8906640 1 -- 172.21.15.98:0/3897963427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff38c0051d0 msgr2=0x7ff38c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:11.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.426+0000 7ff3a9107640 1 --2- 172.21.15.98:0/3897963427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff38c0051d0 0x7ff38c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:11.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.426+0000 7ff3a8906640 1 --2- 172.21.15.98:0/3897963427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff38c0051d0 0x7ff38c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:11.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.426+0000 7ff3a8906640 1 -- 172.21.15.98:0/3897963427 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38c003530 con 0x7ff38c0039c0 2024-09-19T08:47:11.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.426+0000 7ff3a9107640 1 --2- 172.21.15.98:0/3897963427 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff38c0051d0 0x7ff38c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:11.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.426+0000 7ff3a9107640 1 -- 172.21.15.98:0/3897963427 reap_dead start 2024-09-19T08:47:11.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.430+0000 7ff3a8906640 1 --2- 172.21.15.98:0/3897963427 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff38c0039c0 0x7ff38c003de0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009ae0 tx=0x7ff39c01ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=316df67e8eecb211 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:11.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.430+0000 7ff3a37fe640 1 -- 172.21.15.98:0/3897963427 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c0b2070 con 0x7ff38c0039c0 2024-09-19T08:47:11.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.430+0000 7ff3a37fe640 1 -- 172.21.15.98:0/3897963427 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff39c0a5520 con 0x7ff38c0039c0 2024-09-19T08:47:11.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.430+0000 7ff3a37fe640 1 -- 172.21.15.98:0/3897963427 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39c0096b0 con 0x7ff38c0039c0 2024-09-19T08:47:11.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.430+0000 7ff3aab6a640 1 -- 172.21.15.98:0/3897963427 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff38c0039c0 msgr2=0x7ff38c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:11.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.430+0000 7ff3aab6a640 1 --2- 172.21.15.98:0/3897963427 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff38c0039c0 0x7ff38c003de0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009ae0 tx=0x7ff39c01ca40 comp rx=0 tx=0).stop 2024-09-19T08:47:11.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.430+0000 7ff3aab6a640 1 -- 172.21.15.98:0/3897963427 shutdown_connections 2024-09-19T08:47:11.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.430+0000 7ff3aab6a640 1 --2- 172.21.15.98:0/3897963427 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff38c0039c0 0x7ff38c003de0 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:11.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.430+0000 7ff3aab6a640 1 -- 172.21.15.98:0/3897963427 >> 172.21.15.98:0/3897963427 conn(0x7ff3a40b2010 msgr2=0x7ff3a40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:11.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.430+0000 7ff3aab6a640 1 -- 172.21.15.98:0/3897963427 shutdown_connections 2024-09-19T08:47:11.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.430+0000 7ff3aab6a640 1 -- 172.21.15.98:0/3897963427 wait complete. 2024-09-19T08:47:11.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.430+0000 7ff3aab6a640 1 Processor -- start 2024-09-19T08:47:11.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.430+0000 7ff3aab6a640 1 -- start start 2024-09-19T08:47:11.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.430+0000 7ff3aab6a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff38c0039c0 0x7ff3a41447a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:11.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.430+0000 7ff3aab6a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3a41413f0 0x7ff3a4157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:11.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.430+0000 7ff3aab6a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3a4157ec0 0x7ff3a40cf890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:11.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.430+0000 7ff3aab6a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a415c920 con 0x7ff3a41413f0 2024-09-19T08:47:11.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.430+0000 7ff3aab6a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a415c7a0 con 0x7ff3a4157ec0 2024-09-19T08:47:11.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.430+0000 7ff3aab6a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff3a415f7d0 con 0x7ff38c0039c0 2024-09-19T08:47:11.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.430+0000 7ff3a3fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3a41413f0 0x7ff3a4157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:11.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.430+0000 7ff3a3fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3a41413f0 0x7ff3a4157980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35118/0 (socket says 172.21.15.98:35118) 2024-09-19T08:47:11.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.430+0000 7ff3a3fff640 1 -- 172.21.15.98:0/301441413 learned_addr learned my addr 172.21.15.98:0/301441413 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:11.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.430+0000 7ff3a9107640 1 --2- 172.21.15.98:0/301441413 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3a4157ec0 0x7ff3a40cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:11.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.430+0000 7ff3a8906640 1 --2- 172.21.15.98:0/301441413 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff38c0039c0 0x7ff3a41447a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:11.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.430+0000 7ff3a3fff640 1 -- 172.21.15.98:0/301441413 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff38c0039c0 msgr2=0x7ff3a41447a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:11.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.430+0000 7ff3a3fff640 1 --2- 172.21.15.98:0/301441413 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff38c0039c0 0x7ff3a41447a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:11.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.430+0000 7ff3a3fff640 1 -- 172.21.15.98:0/301441413 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3a4157ec0 msgr2=0x7ff3a40cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:11.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.430+0000 7ff3a3fff640 1 --2- 172.21.15.98:0/301441413 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3a4157ec0 0x7ff3a40cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:11.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.430+0000 7ff3a3fff640 1 -- 172.21.15.98:0/301441413 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a4144ce0 con 0x7ff3a41413f0 2024-09-19T08:47:11.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.430+0000 7ff3a3fff640 1 --2- 172.21.15.98:0/301441413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3a41413f0 0x7ff3a4157980 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7ff39800d830 tx=0x7ff39800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:11.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.434+0000 7ff3a17fa640 1 -- 172.21.15.98:0/301441413 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff398019070 con 0x7ff3a41413f0 2024-09-19T08:47:11.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.434+0000 7ff3a17fa640 1 -- 172.21.15.98:0/301441413 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3980074e0 con 0x7ff3a41413f0 2024-09-19T08:47:11.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.434+0000 7ff3aab6a640 1 -- 172.21.15.98:0/301441413 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3a4154e10 con 0x7ff3a41413f0 2024-09-19T08:47:11.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.434+0000 7ff3a17fa640 1 -- 172.21.15.98:0/301441413 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3980049f0 con 0x7ff3a41413f0 2024-09-19T08:47:11.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.434+0000 7ff3aab6a640 1 -- 172.21.15.98:0/301441413 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3a41553d0 con 0x7ff3a41413f0 2024-09-19T08:47:11.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.434+0000 7ff3a17fa640 1 -- 172.21.15.98:0/301441413 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7ff39801c070 con 0x7ff3a41413f0 2024-09-19T08:47:11.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.434+0000 7ff3aab6a640 1 -- 172.21.15.98:0/301441413 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3a415fb00 con 0x7ff3a41413f0 2024-09-19T08:47:11.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.434+0000 7ff3a17fa640 1 --2- 172.21.15.98:0/301441413 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff390068280 0x7ff39006a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:11.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.434+0000 7ff3a17fa640 1 -- 172.21.15.98:0/301441413 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 4885+0+0 (secure 0 0 0) 0x7ff398083df0 con 0x7ff3a41413f0 2024-09-19T08:47:11.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.434+0000 7ff3a8906640 1 --2- 172.21.15.98:0/301441413 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff390068280 0x7ff39006a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:11.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.434+0000 7ff3a8906640 1 --2- 172.21.15.98:0/301441413 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff390068280 0x7ff39006a740 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff39c0af000 tx=0x7ff39c0046d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:11.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.438+0000 7ff3a17fa640 1 -- 172.21.15.98:0/301441413 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff3a4144ce0 con 0x7ff3a41413f0 2024-09-19T08:47:11.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.642+0000 7ff3aab6a640 1 -- 172.21.15.98:0/301441413 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7ff3a40af880 con 0x7ff3a41413f0 2024-09-19T08:47:11.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.642+0000 7ff3a17fa640 1 -- 172.21.15.98:0/301441413 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v25) ==== 80+0+5304 (secure 0 0 0) 0x7ff398055fa0 con 0x7ff3a41413f0 2024-09-19T08:47:11.643 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:47:11.644 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:47:11.644 INFO:teuthology.orchestra.run.smithi098.stdout: "auth_dump": [ 2024-09-19T08:47:11.644 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:47:11.644 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.a", 2024-09-19T08:47:11.644 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmuujGBRAAa2k/kW68NJI6nUKx+W+G2Q==", 2024-09-19T08:47:11.644 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:47:11.644 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:47:11.644 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:47:11.644 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:47:11.644 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:47:11.645 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:47:11.645 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:47:11.645 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:47:11.645 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.b", 2024-09-19T08:47:11.645 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmFoojFBAAaOZPpa41fdjBPEn3CGsJnA==", 2024-09-19T08:47:11.645 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:47:11.645 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:47:11.645 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:47:11.645 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:47:11.645 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:47:11.645 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:47:11.646 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:47:11.646 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:47:11.646 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.c", 2024-09-19T08:47:11.646 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmcQcDDhAAF0QJMlljMH+DiHfWW+/Sog==", 2024-09-19T08:47:11.646 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:47:11.646 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:47:11.646 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:47:11.646 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:47:11.646 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:47:11.646 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:47:11.646 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:47:11.646 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:47:11.646 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mds.d", 2024-09-19T08:47:11.646 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmwWmqHBAAf15VcrdeHf8AzQjGsN9LoA==", 2024-09-19T08:47:11.647 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:47:11.647 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:47:11.647 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:47:11.647 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:47:11.647 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:47:11.647 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:47:11.647 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:47:11.647 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:47:11.647 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.0", 2024-09-19T08:47:11.647 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDC4utm9PjNKxAAa6SG9AE4smPiu4azRAFKvg==", 2024-09-19T08:47:11.647 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:47:11.648 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:47:11.648 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:47:11.648 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:47:11.648 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:47:11.648 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:47:11.648 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:47:11.648 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.1", 2024-09-19T08:47:11.649 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDG4utmHy6UCxAAN6hpG4DJEp81iG/KoloTvQ==", 2024-09-19T08:47:11.649 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:47:11.649 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:47:11.649 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:47:11.649 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:47:11.649 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:47:11.649 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:47:11.649 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:47:11.649 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.2", 2024-09-19T08:47:11.650 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDJ4utmUVP4GhAAJNAL30sEAseoDG2fE6YbZQ==", 2024-09-19T08:47:11.650 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:47:11.650 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:47:11.650 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:47:11.650 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:47:11.650 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:47:11.650 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:47:11.650 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:47:11.650 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.3", 2024-09-19T08:47:11.650 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDM4utmTKgoLRAAiCrPRLML+wvjOt6RhKay3w==", 2024-09-19T08:47:11.650 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:47:11.650 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:47:11.651 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:47:11.651 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:47:11.651 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:47:11.651 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:47:11.651 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:47:11.651 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.4", 2024-09-19T08:47:11.651 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDS4utmAf6bMhAAI3uENuFoXLicUeuspgFjYA==", 2024-09-19T08:47:11.651 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:47:11.651 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:47:11.651 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:47:11.651 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:47:11.651 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:47:11.651 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:47:11.652 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:47:11.652 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.5", 2024-09-19T08:47:11.652 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDW4utmRjxwEhAAF1A2GtrZXVx9BlmjHjhUDQ==", 2024-09-19T08:47:11.652 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:47:11.652 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:47:11.652 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:47:11.652 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:47:11.652 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:47:11.652 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:47:11.652 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:47:11.652 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.6", 2024-09-19T08:47:11.653 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDZ4utmSk/XIRAAROSfzGL7smR88ZkDSx5jAg==", 2024-09-19T08:47:11.653 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:47:11.653 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:47:11.653 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:47:11.653 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:47:11.653 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:47:11.654 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:47:11.654 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:47:11.654 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "osd.7", 2024-09-19T08:47:11.654 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQDc4utmXfpbMxAAqEM4I/ACFvfu/sMA8bMcrQ==", 2024-09-19T08:47:11.654 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:47:11.654 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow profile osd", 2024-09-19T08:47:11.654 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile osd", 2024-09-19T08:47:11.654 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:47:11.654 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:47:11.654 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:47:11.654 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:47:11.655 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.0", 2024-09-19T08:47:11.655 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utm0vNpIhAA4S1E8dFZ0+ibmSKhmZ+bIA==", 2024-09-19T08:47:11.655 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:47:11.655 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:47:11.655 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow r", 2024-09-19T08:47:11.655 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-19T08:47:11.655 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:47:11.655 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:47:11.655 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:47:11.655 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.1", 2024-09-19T08:47:11.655 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmx7xyKhAAC84lykPEb01MiypXKQVREQ==", 2024-09-19T08:47:11.655 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:47:11.655 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:47:11.656 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-19T08:47:11.656 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-19T08:47:11.656 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-19T08:47:11.656 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:47:11.656 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:47:11.656 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:47:11.656 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.2", 2024-09-19T08:47:11.656 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmteOQJxAAtTF2Py21ZE2WRQjeQSrkKQ==", 2024-09-19T08:47:11.656 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:47:11.656 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:47:11.657 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-19T08:47:11.657 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-19T08:47:11.657 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-19T08:47:11.657 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:47:11.657 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:47:11.657 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:47:11.657 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.3", 2024-09-19T08:47:11.657 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmKN92LxAAMqraZOWRT6IIfx5AuDWS2g==", 2024-09-19T08:47:11.657 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:47:11.657 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow", 2024-09-19T08:47:11.658 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow r", 2024-09-19T08:47:11.658 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow rw", 2024-09-19T08:47:11.658 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow rwx" 2024-09-19T08:47:11.658 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:47:11.658 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:47:11.658 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:47:11.658 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "client.admin", 2024-09-19T08:47:11.658 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC+4utmLH97KhAA1DTYW5bTrGUG+7cb8+d7gA==", 2024-09-19T08:47:11.658 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:47:11.659 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-09-19T08:47:11.659 INFO:teuthology.orchestra.run.smithi098.stdout: "mgr": "allow *", 2024-09-19T08:47:11.659 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow *", 2024-09-19T08:47:11.659 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:47:11.659 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:47:11.659 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:47:11.659 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:47:11.659 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mgr.x", 2024-09-19T08:47:11.659 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC/4utmbKuyAhAA0/XjHUIPOIV/YvNEhB8rEQ==", 2024-09-19T08:47:11.660 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:47:11.660 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-09-19T08:47:11.660 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile mgr", 2024-09-19T08:47:11.660 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:47:11.660 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:47:11.660 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:47:11.660 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:47:11.660 INFO:teuthology.orchestra.run.smithi098.stdout: "entity": "mgr.y", 2024-09-19T08:47:11.660 INFO:teuthology.orchestra.run.smithi098.stdout: "key": "AQC+4utmU+ZrOxAAsvcwINH6qyxfN7N+rDrkqg==", 2024-09-19T08:47:11.660 INFO:teuthology.orchestra.run.smithi098.stdout: "caps": { 2024-09-19T08:47:11.661 INFO:teuthology.orchestra.run.smithi098.stdout: "mds": "allow *", 2024-09-19T08:47:11.661 INFO:teuthology.orchestra.run.smithi098.stdout: "mon": "allow profile mgr", 2024-09-19T08:47:11.661 INFO:teuthology.orchestra.run.smithi098.stdout: "osd": "allow *" 2024-09-19T08:47:11.661 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:47:11.661 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:47:11.661 INFO:teuthology.orchestra.run.smithi098.stdout: ] 2024-09-19T08:47:11.661 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:47:11.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.646+0000 7ff3aab6a640 1 -- 172.21.15.98:0/301441413 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff390068280 msgr2=0x7ff39006a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:11.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.646+0000 7ff3aab6a640 1 --2- 172.21.15.98:0/301441413 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff390068280 0x7ff39006a740 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff39c0af000 tx=0x7ff39c0046d0 comp rx=0 tx=0).stop 2024-09-19T08:47:11.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.646+0000 7ff3aab6a640 1 -- 172.21.15.98:0/301441413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3a41413f0 msgr2=0x7ff3a4157980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:11.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.646+0000 7ff3aab6a640 1 --2- 172.21.15.98:0/301441413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3a41413f0 0x7ff3a4157980 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7ff39800d830 tx=0x7ff39800dcf0 comp rx=0 tx=0).stop 2024-09-19T08:47:11.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.646+0000 7ff3aab6a640 1 -- 172.21.15.98:0/301441413 shutdown_connections 2024-09-19T08:47:11.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.646+0000 7ff3aab6a640 1 --2- 172.21.15.98:0/301441413 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff390068280 0x7ff39006a740 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:11.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.646+0000 7ff3aab6a640 1 --2- 172.21.15.98:0/301441413 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3a4157ec0 0x7ff3a40cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:11.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.646+0000 7ff3aab6a640 1 --2- 172.21.15.98:0/301441413 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff3a41413f0 0x7ff3a4157980 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:11.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.646+0000 7ff3aab6a640 1 --2- 172.21.15.98:0/301441413 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff38c0039c0 0x7ff3a41447a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:11.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.646+0000 7ff3aab6a640 1 -- 172.21.15.98:0/301441413 >> 172.21.15.98:0/301441413 conn(0x7ff3a40b2010 msgr2=0x7ff3a415a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:11.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.646+0000 7ff3aab6a640 1 -- 172.21.15.98:0/301441413 shutdown_connections 2024-09-19T08:47:11.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.646+0000 7ff3aab6a640 1 -- 172.21.15.98:0/301441413 wait complete. 2024-09-19T08:47:11.664 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-19T08:47:11.784 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.786+0000 7f50993d6640 1 Processor -- start 2024-09-19T08:47:11.785 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.786+0000 7f50993d6640 1 -- start start 2024-09-19T08:47:11.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.786+0000 7f50993d6640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f50940b9970 0x7f50940b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:11.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.786+0000 7f50993d6640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5094068440 con 0x7f50940d4820 2024-09-19T08:47:11.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.786+0000 7f50993d6640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5094162a90 con 0x7f50940b7f80 2024-09-19T08:47:11.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.786+0000 7f50993d6640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f5094163050 con 0x7f50940b9970 2024-09-19T08:47:11.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.786+0000 7f50927fc640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f50940d4820 0x7f50941621b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40632/0 (socket says 172.21.15.98:40632) 2024-09-19T08:47:11.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.786+0000 7f50927fc640 1 -- 172.21.15.98:0/294948864 learned_addr learned my addr 172.21.15.98:0/294948864 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:11.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.786+0000 7f50937fe640 1 --2- 172.21.15.98:0/294948864 >> v2:172.21.15.139:3300/0 conn(0x7f50940b9970 0x7f50940b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:11.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.786+0000 7f5091ffb640 1 -- 172.21.15.98:0/294948864 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 850649348 0 0) 0x7f5094068440 con 0x7f50940d4820 2024-09-19T08:47:11.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.786+0000 7f5091ffb640 1 -- 172.21.15.98:0/294948864 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f507c003530 con 0x7f50940d4820 2024-09-19T08:47:11.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.786+0000 7f5091ffb640 1 -- 172.21.15.98:0/294948864 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1639397123 0 0) 0x7f507c003530 con 0x7f50940d4820 2024-09-19T08:47:11.788 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.786+0000 7f5091ffb640 1 -- 172.21.15.98:0/294948864 >> v2:172.21.15.139:3300/0 conn(0x7f50940b9970 msgr2=0x7f50940b9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:11.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.786+0000 7f5091ffb640 1 --2- 172.21.15.98:0/294948864 >> v2:172.21.15.139:3300/0 conn(0x7f50940b9970 0x7f50940b9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:11.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.786+0000 7f5091ffb640 1 -- 172.21.15.98:0/294948864 >> v1:172.21.15.139:6789/0 conn(0x7f50940b7f80 legacy=0x7f509415f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:11.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.786+0000 7f5091ffb640 1 -- 172.21.15.98:0/294948864 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5094162d70 con 0x7f50940d4820 2024-09-19T08:47:11.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.786+0000 7f5091ffb640 1 -- 172.21.15.98:0/294948864 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f5088001680 con 0x7f50940d4820 2024-09-19T08:47:11.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.786+0000 7f5091ffb640 1 -- 172.21.15.98:0/294948864 >> v1:172.21.15.98:6789/0 conn(0x7f50940d4820 legacy=0x7f50941621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:11.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.786+0000 7f5091ffb640 1 --2- 172.21.15.98:0/294948864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f507c0039c0 0x7f507c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:11.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.786+0000 7f5091ffb640 1 --2- 172.21.15.98:0/294948864 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f507c004810 0x7f507c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:11.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.786+0000 7f5091ffb640 1 --2- 172.21.15.98:0/294948864 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f507c0051d0 0x7f507c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:11.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.786+0000 7f5091ffb640 1 -- 172.21.15.98:0/294948864 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5094163050 con 0x7f507c004810 2024-09-19T08:47:11.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.786+0000 7f5091ffb640 1 -- 172.21.15.98:0/294948864 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f507c00c070 con 0x7f507c0039c0 2024-09-19T08:47:11.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.786+0000 7f5091ffb640 1 -- 172.21.15.98:0/294948864 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f507c00c270 con 0x7f507c0051d0 2024-09-19T08:47:11.789 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.786+0000 7f50927fc640 1 --2- 172.21.15.98:0/294948864 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f507c004810 0x7f507c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:11.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.786+0000 7f5092ffd640 1 --2- 172.21.15.98:0/294948864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f507c0039c0 0x7f507c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:11.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.786+0000 7f50937fe640 1 --2- 172.21.15.98:0/294948864 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f507c0051d0 0x7f507c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:11.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.786+0000 7f50927fc640 1 -- 172.21.15.98:0/294948864 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f507c0051d0 msgr2=0x7f507c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:11.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.786+0000 7f50927fc640 1 --2- 172.21.15.98:0/294948864 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f507c0051d0 0x7f507c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:11.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.786+0000 7f50927fc640 1 -- 172.21.15.98:0/294948864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f507c0039c0 msgr2=0x7f507c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:11.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.786+0000 7f50927fc640 1 --2- 172.21.15.98:0/294948864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f507c0039c0 0x7f507c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:11.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.786+0000 7f50927fc640 1 -- 172.21.15.98:0/294948864 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f507c003530 con 0x7f507c004810 2024-09-19T08:47:11.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.786+0000 7f50937fe640 1 --2- 172.21.15.98:0/294948864 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f507c0051d0 0x7f507c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:11.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.786+0000 7f50937fe640 1 -- 172.21.15.98:0/294948864 reap_dead start 2024-09-19T08:47:11.793 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.790+0000 7f50927fc640 1 --2- 172.21.15.98:0/294948864 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f507c004810 0x7f507c004c90 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f508800c1c0 tx=0x7f508801f2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e561a05643481848 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:11.793 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.790+0000 7f5091ffb640 1 -- 172.21.15.98:0/294948864 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5088007d00 con 0x7f507c004810 2024-09-19T08:47:11.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.790+0000 7f5091ffb640 1 -- 172.21.15.98:0/294948864 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50880ac020 con 0x7f507c004810 2024-09-19T08:47:11.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.790+0000 7f5091ffb640 1 -- 172.21.15.98:0/294948864 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50880058e0 con 0x7f507c004810 2024-09-19T08:47:11.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.790+0000 7f50993d6640 1 -- 172.21.15.98:0/294948864 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f507c004810 msgr2=0x7f507c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:11.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.790+0000 7f50993d6640 1 --2- 172.21.15.98:0/294948864 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f507c004810 0x7f507c004c90 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f508800c1c0 tx=0x7f508801f2d0 comp rx=0 tx=0).stop 2024-09-19T08:47:11.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.790+0000 7f50993d6640 1 -- 172.21.15.98:0/294948864 shutdown_connections 2024-09-19T08:47:11.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.790+0000 7f50993d6640 1 --2- 172.21.15.98:0/294948864 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f507c004810 0x7f507c004c90 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:11.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.790+0000 7f50993d6640 1 -- 172.21.15.98:0/294948864 >> 172.21.15.98:0/294948864 conn(0x7f5094074e40 msgr2=0x7f5094075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:11.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.790+0000 7f50993d6640 1 -- 172.21.15.98:0/294948864 shutdown_connections 2024-09-19T08:47:11.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.790+0000 7f50993d6640 1 -- 172.21.15.98:0/294948864 wait complete. 2024-09-19T08:47:11.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.790+0000 7f50993d6640 1 Processor -- start 2024-09-19T08:47:11.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.790+0000 7f50993d6640 1 -- start start 2024-09-19T08:47:11.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.794+0000 7f50993d6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f507c004810 0x7f5094151f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:11.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.794+0000 7f50993d6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5094159d20 0x7f509415a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:11.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.794+0000 7f50993d6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5094154190 0x7f5094154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:11.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.794+0000 7f50993d6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f50940b84e0 con 0x7f5094159d20 2024-09-19T08:47:11.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.794+0000 7f50993d6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f50940b8360 con 0x7f5094154190 2024-09-19T08:47:11.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.794+0000 7f50993d6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f509415f560 con 0x7f507c004810 2024-09-19T08:47:11.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.794+0000 7f50927fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5094159d20 0x7f509415a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:11.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.794+0000 7f50927fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5094159d20 0x7f509415a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35132/0 (socket says 172.21.15.98:35132) 2024-09-19T08:47:11.795 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.794+0000 7f50927fc640 1 -- 172.21.15.98:0/2694091662 learned_addr learned my addr 172.21.15.98:0/2694091662 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:11.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.794+0000 7f50937fe640 1 --2- 172.21.15.98:0/2694091662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5094154190 0x7f5094154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:11.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.794+0000 7f5092ffd640 1 --2- 172.21.15.98:0/2694091662 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f507c004810 0x7f5094151f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:11.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.794+0000 7f50927fc640 1 -- 172.21.15.98:0/2694091662 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f507c004810 msgr2=0x7f5094151f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:11.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.794+0000 7f50927fc640 1 --2- 172.21.15.98:0/2694091662 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f507c004810 0x7f5094151f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:11.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.794+0000 7f50927fc640 1 -- 172.21.15.98:0/2694091662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5094154190 msgr2=0x7f5094154610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:11.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.794+0000 7f50927fc640 1 --2- 172.21.15.98:0/2694091662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5094154190 0x7f5094154610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:11.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.794+0000 7f50927fc640 1 -- 172.21.15.98:0/2694091662 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5094140fd0 con 0x7f5094159d20 2024-09-19T08:47:11.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.794+0000 7f50927fc640 1 --2- 172.21.15.98:0/2694091662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5094159d20 0x7f509415a1a0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f508800c1c0 tx=0x7f507c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:11.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.794+0000 7f507b7fe640 1 -- 172.21.15.98:0/2694091662 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50880049f0 con 0x7f5094159d20 2024-09-19T08:47:11.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.794+0000 7f507b7fe640 1 -- 172.21.15.98:0/2694091662 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50880b5070 con 0x7f5094159d20 2024-09-19T08:47:11.796 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.794+0000 7f50993d6640 1 -- 172.21.15.98:0/2694091662 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5094143790 con 0x7f5094159d20 2024-09-19T08:47:11.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.794+0000 7f507b7fe640 1 -- 172.21.15.98:0/2694091662 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5088004d90 con 0x7f5094159d20 2024-09-19T08:47:11.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.794+0000 7f50993d6640 1 -- 172.21.15.98:0/2694091662 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5094159520 con 0x7f5094159d20 2024-09-19T08:47:11.797 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.794+0000 7f50993d6640 1 -- 172.21.15.98:0/2694091662 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f509415f860 con 0x7f5094159d20 2024-09-19T08:47:11.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.794+0000 7f507b7fe640 1 -- 172.21.15.98:0/2694091662 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f50880b1430 con 0x7f5094159d20 2024-09-19T08:47:11.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.794+0000 7f507b7fe640 1 --2- 172.21.15.98:0/2694091662 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5080068250 0x7f508006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:11.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.794+0000 7f507b7fe640 1 -- 172.21.15.98:0/2694091662 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 4885+0+0 (secure 0 0 0) 0x7f5088120e10 con 0x7f5094159d20 2024-09-19T08:47:11.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.794+0000 7f5092ffd640 1 --2- 172.21.15.98:0/2694091662 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5080068250 0x7f508006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:11.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.794+0000 7f5092ffd640 1 --2- 172.21.15.98:0/2694091662 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5080068250 0x7f508006a710 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5084000c10 tx=0x7f5084009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:11.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.798+0000 7f507b7fe640 1 -- 172.21.15.98:0/2694091662 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f5088018fe0 con 0x7f5094159d20 2024-09-19T08:47:11.960 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.962+0000 7f50993d6640 1 -- 172.21.15.98:0/2694091662 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f50940d51d0 con 0x7f5094159d20 2024-09-19T08:47:11.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.962+0000 7f507b7fe640 1 -- 172.21.15.98:0/2694091662 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v166) ==== 84+0+65 (secure 0 0 0) 0x7f50880f3550 con 0x7f5094159d20 2024-09-19T08:47:11.961 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:47:11.961 INFO:teuthology.orchestra.run.smithi098.stdout:[ 2024-09-19T08:47:11.961 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:47:11.961 INFO:teuthology.orchestra.run.smithi098.stdout: "poolnum": 1, 2024-09-19T08:47:11.962 INFO:teuthology.orchestra.run.smithi098.stdout: "poolname": ".mgr" 2024-09-19T08:47:11.962 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:47:11.962 INFO:teuthology.orchestra.run.smithi098.stdout:] 2024-09-19T08:47:11.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.962+0000 7f50993d6640 1 -- 172.21.15.98:0/2694091662 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5080068250 msgr2=0x7f508006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:11.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.962+0000 7f50993d6640 1 --2- 172.21.15.98:0/2694091662 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5080068250 0x7f508006a710 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5084000c10 tx=0x7f5084009040 comp rx=0 tx=0).stop 2024-09-19T08:47:11.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.962+0000 7f50993d6640 1 -- 172.21.15.98:0/2694091662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5094159d20 msgr2=0x7f509415a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:11.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.962+0000 7f50993d6640 1 --2- 172.21.15.98:0/2694091662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5094159d20 0x7f509415a1a0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f508800c1c0 tx=0x7f507c005840 comp rx=0 tx=0).stop 2024-09-19T08:47:11.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.962+0000 7f50993d6640 1 -- 172.21.15.98:0/2694091662 shutdown_connections 2024-09-19T08:47:11.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.962+0000 7f50993d6640 1 --2- 172.21.15.98:0/2694091662 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5080068250 0x7f508006a710 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:11.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.962+0000 7f50993d6640 1 --2- 172.21.15.98:0/2694091662 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5094154190 0x7f5094154610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:11.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.962+0000 7f50993d6640 1 --2- 172.21.15.98:0/2694091662 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5094159d20 0x7f509415a1a0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:11.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.962+0000 7f50993d6640 1 --2- 172.21.15.98:0/2694091662 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f507c004810 0x7f5094151f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:11.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.962+0000 7f50993d6640 1 -- 172.21.15.98:0/2694091662 >> 172.21.15.98:0/2694091662 conn(0x7f5094074e40 msgr2=0x7f509415fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:11.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.962+0000 7f50993d6640 1 -- 172.21.15.98:0/2694091662 shutdown_connections 2024-09-19T08:47:11.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:11.962+0000 7f50993d6640 1 -- 172.21.15.98:0/2694091662 wait complete. 2024-09-19T08:47:11.979 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T08:47:11.980 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T08:47:12.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.094+0000 7f13e147c640 1 Processor -- start 2024-09-19T08:47:12.094 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.094+0000 7f13e147c640 1 -- start start 2024-09-19T08:47:12.096 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.094+0000 7f13e147c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13dc15a930 0x7f13dc0b3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:12.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.094+0000 7f13e147c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f13dc0b3b60 0x7f13dc0b5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:12.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.094+0000 7f13e147c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13dc0b5110 con 0x7f13dc0b6200 2024-09-19T08:47:12.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.094+0000 7f13e147c640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f13dc0b4070 con 0x7f13dc0b3b60 2024-09-19T08:47:12.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.094+0000 7f13e147c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f13dc0b4350 con 0x7f13dc15a930 2024-09-19T08:47:12.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.094+0000 7f13da7fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f13dc0b3b60 0x7f13dc0b5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:12.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.094+0000 7f13da7fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f13dc0b3b60 0x7f13dc0b5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35138/0 (socket says 172.21.15.98:35138) 2024-09-19T08:47:12.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.094+0000 7f13da7fc640 1 -- 172.21.15.98:0/4010962373 learned_addr learned my addr 172.21.15.98:0/4010962373 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:12.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.094+0000 7f13daffd640 1 --2- 172.21.15.98:0/4010962373 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13dc15a930 0x7f13dc0b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:12.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.094+0000 7f13da7fc640 1 -- 172.21.15.98:0/4010962373 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13dc15a930 msgr2=0x7f13dc0b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:12.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.094+0000 7f13da7fc640 1 --2- 172.21.15.98:0/4010962373 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13dc15a930 0x7f13dc0b3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:12.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.098+0000 7f13da7fc640 1 -- 172.21.15.98:0/4010962373 >> v1:172.21.15.139:6789/0 conn(0x7f13dc0b6200 legacy=0x7f13dc071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:12.099 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.098+0000 7f13da7fc640 1 -- 172.21.15.98:0/4010962373 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13dc0b4630 con 0x7f13dc0b3b60 2024-09-19T08:47:12.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.098+0000 7f13da7fc640 1 --2- 172.21.15.98:0/4010962373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13dc0b3b60 0x7f13dc0b5cc0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f13d000aa10 tx=0x7f13d001d8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc48a232b8021c21 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:12.105 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.098+0000 7f13d9ffb640 1 -- 172.21.15.98:0/4010962373 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d000b040 con 0x7f13dc0b3b60 2024-09-19T08:47:12.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.098+0000 7f13d9ffb640 1 -- 172.21.15.98:0/4010962373 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13d00a7700 con 0x7f13dc0b3b60 2024-09-19T08:47:12.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.098+0000 7f13d9ffb640 1 -- 172.21.15.98:0/4010962373 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d00a7ad0 con 0x7f13dc0b3b60 2024-09-19T08:47:12.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.098+0000 7f13e147c640 1 -- 172.21.15.98:0/4010962373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13dc0b3b60 msgr2=0x7f13dc0b5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:12.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.098+0000 7f13e147c640 1 --2- 172.21.15.98:0/4010962373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13dc0b3b60 0x7f13dc0b5cc0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f13d000aa10 tx=0x7f13d001d8a0 comp rx=0 tx=0).stop 2024-09-19T08:47:12.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.098+0000 7f13e147c640 1 -- 172.21.15.98:0/4010962373 shutdown_connections 2024-09-19T08:47:12.106 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.098+0000 7f13e147c640 1 --2- 172.21.15.98:0/4010962373 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13dc0b3b60 0x7f13dc0b5cc0 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:12.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.098+0000 7f13e147c640 1 --2- 172.21.15.98:0/4010962373 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13dc15a930 0x7f13dc0b3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:12.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.098+0000 7f13e147c640 1 -- 172.21.15.98:0/4010962373 >> 172.21.15.98:0/4010962373 conn(0x7f13dc06c090 msgr2=0x7f13dc06c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:12.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.098+0000 7f13e147c640 1 -- 172.21.15.98:0/4010962373 shutdown_connections 2024-09-19T08:47:12.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.098+0000 7f13e147c640 1 -- 172.21.15.98:0/4010962373 wait complete. 2024-09-19T08:47:12.107 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.102+0000 7f13e147c640 1 Processor -- start 2024-09-19T08:47:12.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.102+0000 7f13e147c640 1 -- start start 2024-09-19T08:47:12.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.102+0000 7f13e147c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13dc0b3b60 0x7f13dc14bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:12.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.102+0000 7f13e147c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13dc0b6200 0x7f13dc157f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:12.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.102+0000 7f13e147c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13dc15a930 0x7f13dc158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:12.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.102+0000 7f13e147c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f13dc0ced10 con 0x7f13dc0b6200 2024-09-19T08:47:12.108 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.102+0000 7f13e147c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f13dc0ceb90 con 0x7f13dc15a930 2024-09-19T08:47:12.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.102+0000 7f13e147c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f13dc071740 con 0x7f13dc0b3b60 2024-09-19T08:47:12.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.102+0000 7f13da7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13dc0b6200 0x7f13dc157f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:12.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.102+0000 7f13da7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13dc0b6200 0x7f13dc157f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35154/0 (socket says 172.21.15.98:35154) 2024-09-19T08:47:12.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.102+0000 7f13da7fc640 1 -- 172.21.15.98:0/157055983 learned_addr learned my addr 172.21.15.98:0/157055983 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:12.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.102+0000 7f13daffd640 1 --2- 172.21.15.98:0/157055983 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13dc0b3b60 0x7f13dc14bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:12.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.102+0000 7f13db7fe640 1 --2- 172.21.15.98:0/157055983 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13dc15a930 0x7f13dc158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:12.109 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.102+0000 7f13da7fc640 1 -- 172.21.15.98:0/157055983 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13dc0b3b60 msgr2=0x7f13dc14bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:12.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.102+0000 7f13da7fc640 1 --2- 172.21.15.98:0/157055983 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13dc0b3b60 0x7f13dc14bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:12.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.102+0000 7f13da7fc640 1 -- 172.21.15.98:0/157055983 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13dc15a930 msgr2=0x7f13dc158490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:12.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.102+0000 7f13da7fc640 1 --2- 172.21.15.98:0/157055983 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13dc15a930 0x7f13dc158490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:12.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.102+0000 7f13da7fc640 1 -- 172.21.15.98:0/157055983 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13dc150ba0 con 0x7f13dc0b6200 2024-09-19T08:47:12.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.102+0000 7f13da7fc640 1 --2- 172.21.15.98:0/157055983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13dc0b6200 0x7f13dc157f50 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f13d001b4d0 tx=0x7f13d001c0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:12.110 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.102+0000 7f13c37fe640 1 -- 172.21.15.98:0/157055983 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d00bd020 con 0x7f13dc0b6200 2024-09-19T08:47:12.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.102+0000 7f13c37fe640 1 -- 172.21.15.98:0/157055983 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13d00af850 con 0x7f13dc0b6200 2024-09-19T08:47:12.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.102+0000 7f13c37fe640 1 -- 172.21.15.98:0/157055983 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d00afc40 con 0x7f13dc0b6200 2024-09-19T08:47:12.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.102+0000 7f13e147c640 1 -- 172.21.15.98:0/157055983 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13dc157060 con 0x7f13dc0b6200 2024-09-19T08:47:12.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.102+0000 7f13e147c640 1 -- 172.21.15.98:0/157055983 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13dc157620 con 0x7f13dc0b6200 2024-09-19T08:47:12.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.102+0000 7f13e147c640 1 -- 172.21.15.98:0/157055983 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13dc150d70 con 0x7f13dc0b6200 2024-09-19T08:47:12.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.102+0000 7f13c37fe640 1 -- 172.21.15.98:0/157055983 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f13d00c0020 con 0x7f13dc0b6200 2024-09-19T08:47:12.111 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.102+0000 7f13c37fe640 1 --2- 172.21.15.98:0/157055983 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f13c8068250 0x7f13c806a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:12.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.106+0000 7f13c37fe640 1 -- 172.21.15.98:0/157055983 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 4885+0+0 (secure 0 0 0) 0x7f13d011f230 con 0x7f13dc0b6200 2024-09-19T08:47:12.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.106+0000 7f13daffd640 1 --2- 172.21.15.98:0/157055983 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f13c8068250 0x7f13c806a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:12.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.106+0000 7f13daffd640 1 --2- 172.21.15.98:0/157055983 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f13c8068250 0x7f13c806a710 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f13dc14f9b0 tx=0x7f13cc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:12.112 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.110+0000 7f13c37fe640 1 -- 172.21.15.98:0/157055983 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f13d00c7030 con 0x7f13dc0b6200 2024-09-19T08:47:12.258 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.258+0000 7f13e147c640 1 -- 172.21.15.98:0/157055983 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f13dc0b5cc0 con 0x7f13dc0b6200 2024-09-19T08:47:12.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.350+0000 7f13c37fe640 1 -- 172.21.15.98:0/157055983 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v167) ==== 138+0+0 (secure 0 0 0) 0x7f13d0017520 con 0x7f13dc0b6200 2024-09-19T08:47:12.351 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_metadata' created 2024-09-19T08:47:12.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.354+0000 7f13e147c640 1 -- 172.21.15.98:0/157055983 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f13c8068250 msgr2=0x7f13c806a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:12.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.354+0000 7f13e147c640 1 --2- 172.21.15.98:0/157055983 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f13c8068250 0x7f13c806a710 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f13dc14f9b0 tx=0x7f13cc008040 comp rx=0 tx=0).stop 2024-09-19T08:47:12.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.354+0000 7f13e147c640 1 -- 172.21.15.98:0/157055983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13dc0b6200 msgr2=0x7f13dc157f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:12.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.354+0000 7f13e147c640 1 --2- 172.21.15.98:0/157055983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13dc0b6200 0x7f13dc157f50 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f13d001b4d0 tx=0x7f13d001c0e0 comp rx=0 tx=0).stop 2024-09-19T08:47:12.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.354+0000 7f13e147c640 1 -- 172.21.15.98:0/157055983 shutdown_connections 2024-09-19T08:47:12.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.354+0000 7f13e147c640 1 --2- 172.21.15.98:0/157055983 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f13c8068250 0x7f13c806a710 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:12.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.354+0000 7f13e147c640 1 --2- 172.21.15.98:0/157055983 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13dc15a930 0x7f13dc158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:12.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.354+0000 7f13e147c640 1 --2- 172.21.15.98:0/157055983 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f13dc0b6200 0x7f13dc157f50 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:12.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.354+0000 7f13e147c640 1 --2- 172.21.15.98:0/157055983 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f13dc0b3b60 0x7f13dc14bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:12.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.354+0000 7f13e147c640 1 -- 172.21.15.98:0/157055983 >> 172.21.15.98:0/157055983 conn(0x7f13dc06c090 msgr2=0x7f13dc15af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:12.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.354+0000 7f13e147c640 1 -- 172.21.15.98:0/157055983 shutdown_connections 2024-09-19T08:47:12.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.354+0000 7f13e147c640 1 -- 172.21.15.98:0/157055983 wait complete. 2024-09-19T08:47:12.378 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T08:47:12.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.502+0000 7f9787e6d640 1 Processor -- start 2024-09-19T08:47:12.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.502+0000 7f9787e6d640 1 -- start start 2024-09-19T08:47:12.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.502+0000 7f9787e6d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f978015e680 0x7f97800d0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:12.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.502+0000 7f9787e6d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f978006fa90 0x7f978006c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:12.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.502+0000 7f9787e6d640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97800d12a0 con 0x7f978006bc90 2024-09-19T08:47:12.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.502+0000 7f9787e6d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f978015ea60 con 0x7f978015e680 2024-09-19T08:47:12.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.502+0000 7f9787e6d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f97800d08d0 con 0x7f978006fa90 2024-09-19T08:47:12.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.502+0000 7f978640a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f978006fa90 0x7f978006c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:12.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.502+0000 7f9785c09640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f978015e680 0x7f97800d0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:12.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.502+0000 7f978640a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f978006fa90 0x7f978006c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:37828/0 (socket says 172.21.15.98:37828) 2024-09-19T08:47:12.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.502+0000 7f978640a640 1 -- 172.21.15.98:0/2112640751 learned_addr learned my addr 172.21.15.98:0/2112640751 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:12.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.502+0000 7f9785c09640 1 -- 172.21.15.98:0/2112640751 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f978006fa90 msgr2=0x7f978006c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:12.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.502+0000 7f9785c09640 1 --2- 172.21.15.98:0/2112640751 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f978006fa90 0x7f978006c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:12.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.502+0000 7f9785c09640 1 -- 172.21.15.98:0/2112640751 >> v1:172.21.15.139:6789/0 conn(0x7f978006bc90 legacy=0x7f978006f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:12.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.502+0000 7f9785c09640 1 -- 172.21.15.98:0/2112640751 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f978006c800 con 0x7f978015e680 2024-09-19T08:47:12.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.502+0000 7f978640a640 1 --2- 172.21.15.98:0/2112640751 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f978006fa90 0x7f978006c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T08:47:12.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.506+0000 7f9785c09640 1 --2- 172.21.15.98:0/2112640751 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f978015e680 0x7f97800d0d60 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f977c0097f0 tx=0x7f977c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3db925e39611e70 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:12.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.506+0000 7f9784c07640 1 -- 172.21.15.98:0/2112640751 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977c0aa600 con 0x7f978015e680 2024-09-19T08:47:12.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.506+0000 7f9784c07640 1 -- 172.21.15.98:0/2112640751 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f977c0af070 con 0x7f978015e680 2024-09-19T08:47:12.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.506+0000 7f9784c07640 1 -- 172.21.15.98:0/2112640751 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977c0aa9c0 con 0x7f978015e680 2024-09-19T08:47:12.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.506+0000 7f9787e6d640 1 -- 172.21.15.98:0/2112640751 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f978015e680 msgr2=0x7f97800d0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:12.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.506+0000 7f9787e6d640 1 --2- 172.21.15.98:0/2112640751 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f978015e680 0x7f97800d0d60 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f977c0097f0 tx=0x7f977c01c4a0 comp rx=0 tx=0).stop 2024-09-19T08:47:12.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.506+0000 7f9787e6d640 1 -- 172.21.15.98:0/2112640751 shutdown_connections 2024-09-19T08:47:12.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.506+0000 7f9787e6d640 1 --2- 172.21.15.98:0/2112640751 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f978006fa90 0x7f978006c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:12.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.506+0000 7f9787e6d640 1 --2- 172.21.15.98:0/2112640751 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f978015e680 0x7f97800d0d60 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:12.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.506+0000 7f9787e6d640 1 -- 172.21.15.98:0/2112640751 >> 172.21.15.98:0/2112640751 conn(0x7f97800d23d0 msgr2=0x7f97800d27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:12.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.506+0000 7f9787e6d640 1 -- 172.21.15.98:0/2112640751 shutdown_connections 2024-09-19T08:47:12.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.506+0000 7f9787e6d640 1 -- 172.21.15.98:0/2112640751 wait complete. 2024-09-19T08:47:12.509 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.506+0000 7f9787e6d640 1 Processor -- start 2024-09-19T08:47:12.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.506+0000 7f9787e6d640 1 -- start start 2024-09-19T08:47:12.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.510+0000 7f9787e6d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f978006bc90 0x7f97801522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:12.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.510+0000 7f9787e6d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f978006fa90 0x7f978014a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:12.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.510+0000 7f9787e6d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f978015e680 0x7f978014f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:12.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.510+0000 7f9787e6d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f97800737a0 con 0x7f978006fa90 2024-09-19T08:47:12.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.510+0000 7f9787e6d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9780070970 con 0x7f978006bc90 2024-09-19T08:47:12.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.510+0000 7f9787e6d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9780073920 con 0x7f978015e680 2024-09-19T08:47:12.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.510+0000 7f9785408640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f978006fa90 0x7f978014a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:12.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.510+0000 7f9785408640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f978006fa90 0x7f978014a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35166/0 (socket says 172.21.15.98:35166) 2024-09-19T08:47:12.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.510+0000 7f9785408640 1 -- 172.21.15.98:0/3882822317 learned_addr learned my addr 172.21.15.98:0/3882822317 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:12.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.510+0000 7f978640a640 1 --2- 172.21.15.98:0/3882822317 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f978015e680 0x7f978014f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:12.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.510+0000 7f9785c09640 1 --2- 172.21.15.98:0/3882822317 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f978006bc90 0x7f97801522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:12.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.510+0000 7f9785408640 1 -- 172.21.15.98:0/3882822317 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f978015e680 msgr2=0x7f978014f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:12.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.510+0000 7f9785408640 1 --2- 172.21.15.98:0/3882822317 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f978015e680 0x7f978014f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:12.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.510+0000 7f9785408640 1 -- 172.21.15.98:0/3882822317 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f978006bc90 msgr2=0x7f97801522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:12.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.510+0000 7f9785408640 1 --2- 172.21.15.98:0/3882822317 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f978006bc90 0x7f97801522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:12.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.510+0000 7f9785408640 1 -- 172.21.15.98:0/3882822317 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f978014b2e0 con 0x7f978006fa90 2024-09-19T08:47:12.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.510+0000 7f9785408640 1 --2- 172.21.15.98:0/3882822317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f978006fa90 0x7f978014a920 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f977000c420 tx=0x7f977000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:12.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.510+0000 7f97767fc640 1 -- 172.21.15.98:0/3882822317 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97700052d0 con 0x7f978006fa90 2024-09-19T08:47:12.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.510+0000 7f97767fc640 1 -- 172.21.15.98:0/3882822317 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9770005470 con 0x7f978006fa90 2024-09-19T08:47:12.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.510+0000 7f97767fc640 1 -- 172.21.15.98:0/3882822317 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9770007260 con 0x7f978006fa90 2024-09-19T08:47:12.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.510+0000 7f9787e6d640 1 -- 172.21.15.98:0/3882822317 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f978014b5c0 con 0x7f978006fa90 2024-09-19T08:47:12.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.510+0000 7f9787e6d640 1 -- 172.21.15.98:0/3882822317 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f978014bb80 con 0x7f978006fa90 2024-09-19T08:47:12.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.510+0000 7f9787e6d640 1 -- 172.21.15.98:0/3882822317 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f978006c100 con 0x7f978006fa90 2024-09-19T08:47:12.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.510+0000 7f97767fc640 1 -- 172.21.15.98:0/3882822317 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f9770007400 con 0x7f978006fa90 2024-09-19T08:47:12.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.510+0000 7f97767fc640 1 --2- 172.21.15.98:0/3882822317 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9764068250 0x7f976406a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:12.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.510+0000 7f97767fc640 1 -- 172.21.15.98:0/3882822317 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5255+0+0 (secure 0 0 0) 0x7f97700837e0 con 0x7f978006fa90 2024-09-19T08:47:12.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.510+0000 7f9785c09640 1 --2- 172.21.15.98:0/3882822317 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9764068250 0x7f976406a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:12.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.514+0000 7f9785c09640 1 --2- 172.21.15.98:0/3882822317 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9764068250 0x7f976406a710 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f977c0ae000 tx=0x7f977c0a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:12.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.518+0000 7f97767fc640 1 -- 172.21.15.98:0/3882822317 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f97700558d0 con 0x7f978006fa90 2024-09-19T08:47:12.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:12.678+0000 7f9787e6d640 1 -- 172.21.15.98:0/3882822317 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f9780158c60 con 0x7f978006fa90 2024-09-19T08:47:13.361 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.362+0000 7f97767fc640 1 -- 172.21.15.98:0/3882822317 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v168) ==== 170+0+0 (secure 0 0 0) 0x7f9770059bd0 con 0x7f978006fa90 2024-09-19T08:47:13.361 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' created 2024-09-19T08:47:13.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.366+0000 7f9787e6d640 1 -- 172.21.15.98:0/3882822317 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9764068250 msgr2=0x7f976406a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:13.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.366+0000 7f9787e6d640 1 --2- 172.21.15.98:0/3882822317 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9764068250 0x7f976406a710 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f977c0ae000 tx=0x7f977c0a7000 comp rx=0 tx=0).stop 2024-09-19T08:47:13.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.366+0000 7f9787e6d640 1 -- 172.21.15.98:0/3882822317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f978006fa90 msgr2=0x7f978014a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:13.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.366+0000 7f9787e6d640 1 --2- 172.21.15.98:0/3882822317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f978006fa90 0x7f978014a920 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f977000c420 tx=0x7f977000c8e0 comp rx=0 tx=0).stop 2024-09-19T08:47:13.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.366+0000 7f9787e6d640 1 -- 172.21.15.98:0/3882822317 shutdown_connections 2024-09-19T08:47:13.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.366+0000 7f9787e6d640 1 --2- 172.21.15.98:0/3882822317 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9764068250 0x7f976406a710 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:13.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.366+0000 7f9787e6d640 1 --2- 172.21.15.98:0/3882822317 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f978015e680 0x7f978014f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:13.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.366+0000 7f9787e6d640 1 --2- 172.21.15.98:0/3882822317 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f978006fa90 0x7f978014a920 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:13.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.366+0000 7f9787e6d640 1 --2- 172.21.15.98:0/3882822317 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f978006bc90 0x7f97801522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:13.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.366+0000 7f9787e6d640 1 -- 172.21.15.98:0/3882822317 >> 172.21.15.98:0/3882822317 conn(0x7f97800d23d0 msgr2=0x7f97800cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:13.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.370+0000 7f9787e6d640 1 -- 172.21.15.98:0/3882822317 shutdown_connections 2024-09-19T08:47:13.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.370+0000 7f9787e6d640 1 -- 172.21.15.98:0/3882822317 wait complete. 2024-09-19T08:47:13.404 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T08:47:13.524 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.526+0000 7f327979b640 1 Processor -- start 2024-09-19T08:47:13.525 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.526+0000 7f327979b640 1 -- start start 2024-09-19T08:47:13.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.526+0000 7f327979b640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3274145b70 0x7f327406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:13.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.526+0000 7f327979b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f327406c1d0 0x7f32740713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:13.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.526+0000 7f327979b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32740718f0 0x7f32740cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:13.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.526+0000 7f327979b640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f327406c620 con 0x7f327406c1d0 2024-09-19T08:47:13.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.526+0000 7f327979b640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f3274073530 con 0x7f3274145b70 2024-09-19T08:47:13.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.526+0000 7f327979b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3274073810 con 0x7f32740718f0 2024-09-19T08:47:13.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.526+0000 7f32727fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f327406c1d0 0x7f32740713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:13.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.526+0000 7f32727fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f327406c1d0 0x7f32740713b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35168/0 (socket says 172.21.15.98:35168) 2024-09-19T08:47:13.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.526+0000 7f32727fc640 1 -- 172.21.15.98:0/2774505397 learned_addr learned my addr 172.21.15.98:0/2774505397 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:13.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.526+0000 7f32737fe640 1 --2- 172.21.15.98:0/2774505397 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32740718f0 0x7f32740cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:13.528 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.526+0000 7f32727fc640 1 -- 172.21.15.98:0/2774505397 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32740718f0 msgr2=0x7f32740cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:13.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.526+0000 7f3272ffd640 1 --2- 172.21.15.98:0/2774505397 >> v2:172.21.15.139:3300/0 conn(0x7f3274145b70 0x7f327406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:13.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.526+0000 7f32727fc640 1 --2- 172.21.15.98:0/2774505397 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32740718f0 0x7f32740cef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:13.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.526+0000 7f32727fc640 1 -- 172.21.15.98:0/2774505397 >> v2:172.21.15.139:3300/0 conn(0x7f3274145b70 msgr2=0x7f327406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:13.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.526+0000 7f32727fc640 1 --2- 172.21.15.98:0/2774505397 >> v2:172.21.15.139:3300/0 conn(0x7f3274145b70 0x7f327406bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:13.529 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.526+0000 7f32727fc640 1 -- 172.21.15.98:0/2774505397 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3274073af0 con 0x7f327406c1d0 2024-09-19T08:47:13.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.530+0000 7f32727fc640 1 --2- 172.21.15.98:0/2774505397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f327406c1d0 0x7f32740713b0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f32740b2d30 tx=0x7f326801c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31bd5befc6c919b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:13.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.530+0000 7f3271ffb640 1 -- 172.21.15.98:0/2774505397 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32680b0070 con 0x7f327406c1d0 2024-09-19T08:47:13.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.530+0000 7f3271ffb640 1 -- 172.21.15.98:0/2774505397 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32680ab600 con 0x7f327406c1d0 2024-09-19T08:47:13.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.530+0000 7f3271ffb640 1 -- 172.21.15.98:0/2774505397 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32680ab9d0 con 0x7f327406c1d0 2024-09-19T08:47:13.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.530+0000 7f327979b640 1 -- 172.21.15.98:0/2774505397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f327406c1d0 msgr2=0x7f32740713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:13.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.530+0000 7f327979b640 1 --2- 172.21.15.98:0/2774505397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f327406c1d0 0x7f32740713b0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f32740b2d30 tx=0x7f326801c1d0 comp rx=0 tx=0).stop 2024-09-19T08:47:13.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.530+0000 7f327979b640 1 -- 172.21.15.98:0/2774505397 shutdown_connections 2024-09-19T08:47:13.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.530+0000 7f327979b640 1 --2- 172.21.15.98:0/2774505397 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f327406c1d0 0x7f32740713b0 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:13.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.530+0000 7f327979b640 1 --2- 172.21.15.98:0/2774505397 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32740718f0 0x7f32740cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:13.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.530+0000 7f327979b640 1 --2- 172.21.15.98:0/2774505397 >> v2:172.21.15.139:3300/0 conn(0x7f3274145b70 0x7f327406bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:13.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.530+0000 7f327979b640 1 -- 172.21.15.98:0/2774505397 >> 172.21.15.98:0/2774505397 conn(0x7f32740b2010 msgr2=0x7f32740b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:13.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.530+0000 7f327979b640 1 -- 172.21.15.98:0/2774505397 shutdown_connections 2024-09-19T08:47:13.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.530+0000 7f327979b640 1 -- 172.21.15.98:0/2774505397 wait complete. 2024-09-19T08:47:13.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.530+0000 7f327979b640 1 Processor -- start 2024-09-19T08:47:13.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.530+0000 7f327979b640 1 -- start start 2024-09-19T08:47:13.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.530+0000 7f327979b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f327406c1d0 0x7f327413bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:13.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.530+0000 7f327979b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32740718f0 0x7f32740b4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:13.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.530+0000 7f327979b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3274145b70 0x7f3274165420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:13.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.530+0000 7f327979b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f32740b3410 con 0x7f32740718f0 2024-09-19T08:47:13.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.530+0000 7f327979b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f32740b3590 con 0x7f3274145b70 2024-09-19T08:47:13.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.530+0000 7f327979b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f32740b3710 con 0x7f327406c1d0 2024-09-19T08:47:13.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.530+0000 7f32727fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32740718f0 0x7f32740b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:13.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.530+0000 7f32727fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32740718f0 0x7f32740b4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35170/0 (socket says 172.21.15.98:35170) 2024-09-19T08:47:13.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.530+0000 7f32727fc640 1 -- 172.21.15.98:0/210375536 learned_addr learned my addr 172.21.15.98:0/210375536 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:13.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.530+0000 7f3272ffd640 1 --2- 172.21.15.98:0/210375536 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f327406c1d0 0x7f327413bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:13.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.530+0000 7f32737fe640 1 --2- 172.21.15.98:0/210375536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3274145b70 0x7f3274165420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:13.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.530+0000 7f32727fc640 1 -- 172.21.15.98:0/210375536 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f327406c1d0 msgr2=0x7f327413bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:13.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.530+0000 7f32727fc640 1 --2- 172.21.15.98:0/210375536 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f327406c1d0 0x7f327413bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:13.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.530+0000 7f32727fc640 1 -- 172.21.15.98:0/210375536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3274145b70 msgr2=0x7f3274165420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:13.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.530+0000 7f32727fc640 1 --2- 172.21.15.98:0/210375536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3274145b70 0x7f3274165420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:13.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.530+0000 7f32727fc640 1 -- 172.21.15.98:0/210375536 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f327415a670 con 0x7f32740718f0 2024-09-19T08:47:13.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.530+0000 7f3272ffd640 1 --2- 172.21.15.98:0/210375536 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f327406c1d0 0x7f327413bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:13.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.534+0000 7f32727fc640 1 --2- 172.21.15.98:0/210375536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32740718f0 0x7f32740b4360 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f32680ae040 tx=0x7f3268018f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:13.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.534+0000 7f325b7fe640 1 -- 172.21.15.98:0/210375536 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32680b9070 con 0x7f32740718f0 2024-09-19T08:47:13.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.534+0000 7f325b7fe640 1 -- 172.21.15.98:0/210375536 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32680b4600 con 0x7f32740718f0 2024-09-19T08:47:13.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.534+0000 7f327979b640 1 -- 172.21.15.98:0/210375536 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f327415a0b0 con 0x7f32740718f0 2024-09-19T08:47:13.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.534+0000 7f325b7fe640 1 -- 172.21.15.98:0/210375536 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32680b0040 con 0x7f32740718f0 2024-09-19T08:47:13.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.534+0000 7f327979b640 1 -- 172.21.15.98:0/210375536 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32740ce750 con 0x7f32740718f0 2024-09-19T08:47:13.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.534+0000 7f327979b640 1 -- 172.21.15.98:0/210375536 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32740ce540 con 0x7f32740718f0 2024-09-19T08:47:13.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.534+0000 7f325b7fe640 1 -- 172.21.15.98:0/210375536 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f32680bc070 con 0x7f32740718f0 2024-09-19T08:47:13.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.534+0000 7f325b7fe640 1 --2- 172.21.15.98:0/210375536 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3260068250 0x7f326006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:13.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.534+0000 7f325b7fe640 1 -- 172.21.15.98:0/210375536 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5637+0+0 (secure 0 0 0) 0x7f326811dad0 con 0x7f32740718f0 2024-09-19T08:47:13.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.534+0000 7f3272ffd640 1 --2- 172.21.15.98:0/210375536 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3260068250 0x7f326006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:13.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.534+0000 7f3272ffd640 1 --2- 172.21.15.98:0/210375536 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3260068250 0x7f326006a710 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3264006fd0 tx=0x7f3264008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:13.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.538+0000 7f325b7fe640 1 -- 172.21.15.98:0/210375536 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3268016110 con 0x7f32740718f0 2024-09-19T08:47:13.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:13.738+0000 7f327979b640 1 -- 172.21.15.98:0/210375536 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f32740724b0 con 0x7f32740718f0 2024-09-19T08:47:14.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.366+0000 7f325b7fe640 1 -- 172.21.15.98:0/210375536 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 26 and data pool 27 v97) ==== 174+0+0 (secure 0 0 0) 0x7f32680f0210 con 0x7f32740718f0 2024-09-19T08:47:14.367 INFO:teuthology.orchestra.run.smithi098.stderr:new fs with metadata pool 26 and data pool 27 2024-09-19T08:47:14.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.370+0000 7f327979b640 1 -- 172.21.15.98:0/210375536 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3260068250 msgr2=0x7f326006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:14.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.370+0000 7f327979b640 1 --2- 172.21.15.98:0/210375536 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3260068250 0x7f326006a710 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3264006fd0 tx=0x7f3264008040 comp rx=0 tx=0).stop 2024-09-19T08:47:14.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.370+0000 7f327979b640 1 -- 172.21.15.98:0/210375536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32740718f0 msgr2=0x7f32740b4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:14.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.370+0000 7f327979b640 1 --2- 172.21.15.98:0/210375536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32740718f0 0x7f32740b4360 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f32680ae040 tx=0x7f3268018f70 comp rx=0 tx=0).stop 2024-09-19T08:47:14.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.370+0000 7f327979b640 1 -- 172.21.15.98:0/210375536 shutdown_connections 2024-09-19T08:47:14.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.370+0000 7f327979b640 1 --2- 172.21.15.98:0/210375536 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3260068250 0x7f326006a710 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:14.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.370+0000 7f327979b640 1 --2- 172.21.15.98:0/210375536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3274145b70 0x7f3274165420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:14.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.370+0000 7f327979b640 1 --2- 172.21.15.98:0/210375536 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32740718f0 0x7f32740b4360 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:14.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.370+0000 7f327979b640 1 --2- 172.21.15.98:0/210375536 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f327406c1d0 0x7f327413bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:14.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.370+0000 7f327979b640 1 -- 172.21.15.98:0/210375536 >> 172.21.15.98:0/210375536 conn(0x7f32740b2010 msgr2=0x7f32740b0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:14.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.370+0000 7f327979b640 1 -- 172.21.15.98:0/210375536 shutdown_connections 2024-09-19T08:47:14.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.370+0000 7f327979b640 1 -- 172.21.15.98:0/210375536 wait complete. 2024-09-19T08:47:14.395 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:47:14.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.506+0000 7fcc367cb640 1 Processor -- start 2024-09-19T08:47:14.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.506+0000 7fcc367cb640 1 -- start start 2024-09-19T08:47:14.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.510+0000 7fcc367cb640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcc30147770 0x7fcc300b9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:14.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.510+0000 7fcc367cb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc300b6200 0x7fcc300b66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:14.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.510+0000 7fcc367cb640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc300b7fe0 con 0x7fcc300b97d0 2024-09-19T08:47:14.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.510+0000 7fcc367cb640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fcc300b85a0 con 0x7fcc30147770 2024-09-19T08:47:14.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.510+0000 7fcc367cb640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcc300b82c0 con 0x7fcc300b6200 2024-09-19T08:47:14.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.510+0000 7fcc2f7fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fcc300b97d0 0x7fcc300b7520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40636/0 (socket says 172.21.15.98:40636) 2024-09-19T08:47:14.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.510+0000 7fcc2f7fe640 1 -- 172.21.15.98:0/665700999 learned_addr learned my addr 172.21.15.98:0/665700999 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:14.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.510+0000 7fcc34d68640 1 --2- 172.21.15.98:0/665700999 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc300b6200 0x7fcc300b66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:14.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.510+0000 7fcc2ffff640 1 --2- 172.21.15.98:0/665700999 >> v2:172.21.15.139:3300/0 conn(0x7fcc30147770 0x7fcc300b9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:14.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.510+0000 7fcc2effd640 1 -- 172.21.15.98:0/665700999 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 569917550 0 0) 0x7fcc300b7fe0 con 0x7fcc300b97d0 2024-09-19T08:47:14.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.510+0000 7fcc2effd640 1 -- 172.21.15.98:0/665700999 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc18003530 con 0x7fcc300b97d0 2024-09-19T08:47:14.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.510+0000 7fcc2effd640 1 -- 172.21.15.98:0/665700999 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4114087737 0 0) 0x7fcc18003530 con 0x7fcc300b97d0 2024-09-19T08:47:14.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.510+0000 7fcc2effd640 1 -- 172.21.15.98:0/665700999 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc300b6200 msgr2=0x7fcc300b66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:14.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.510+0000 7fcc2effd640 1 --2- 172.21.15.98:0/665700999 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc300b6200 0x7fcc300b66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:14.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.510+0000 7fcc2effd640 1 -- 172.21.15.98:0/665700999 >> v2:172.21.15.139:3300/0 conn(0x7fcc30147770 msgr2=0x7fcc300b9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:14.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.510+0000 7fcc2effd640 1 --2- 172.21.15.98:0/665700999 >> v2:172.21.15.139:3300/0 conn(0x7fcc30147770 0x7fcc300b9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:14.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.510+0000 7fcc2effd640 1 -- 172.21.15.98:0/665700999 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc300b8a90 con 0x7fcc300b97d0 2024-09-19T08:47:14.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.510+0000 7fcc2effd640 1 -- 172.21.15.98:0/665700999 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fcc24001680 con 0x7fcc300b97d0 2024-09-19T08:47:14.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.510+0000 7fcc2effd640 1 -- 172.21.15.98:0/665700999 >> v1:172.21.15.98:6789/0 conn(0x7fcc300b97d0 legacy=0x7fcc300b7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:14.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.510+0000 7fcc2effd640 1 --2- 172.21.15.98:0/665700999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc180039c0 0x7fcc18003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:14.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.510+0000 7fcc2effd640 1 --2- 172.21.15.98:0/665700999 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc300b6200 0x7fcc18004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:14.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.510+0000 7fcc2effd640 1 --2- 172.21.15.98:0/665700999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc18004d50 0x7fcc180051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:14.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.510+0000 7fcc2effd640 1 -- 172.21.15.98:0/665700999 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcc300b85a0 con 0x7fcc180039c0 2024-09-19T08:47:14.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.510+0000 7fcc2effd640 1 -- 172.21.15.98:0/665700999 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcc300b82c0 con 0x7fcc18004d50 2024-09-19T08:47:14.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.510+0000 7fcc2effd640 1 -- 172.21.15.98:0/665700999 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcc1800b9d0 con 0x7fcc300b6200 2024-09-19T08:47:14.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.510+0000 7fcc2ffff640 1 --2- 172.21.15.98:0/665700999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc180039c0 0x7fcc18003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:14.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.510+0000 7fcc34d68640 1 --2- 172.21.15.98:0/665700999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc18004d50 0x7fcc180051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:14.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.510+0000 7fcc2f7fe640 1 --2- 172.21.15.98:0/665700999 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc300b6200 0x7fcc18004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:14.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.510+0000 7fcc2ffff640 1 -- 172.21.15.98:0/665700999 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc300b6200 msgr2=0x7fcc18004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:14.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.510+0000 7fcc2ffff640 1 --2- 172.21.15.98:0/665700999 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc300b6200 0x7fcc18004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:14.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.510+0000 7fcc2ffff640 1 -- 172.21.15.98:0/665700999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc18004d50 msgr2=0x7fcc180051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:14.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.510+0000 7fcc2ffff640 1 --2- 172.21.15.98:0/665700999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc18004d50 0x7fcc180051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:14.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.510+0000 7fcc2ffff640 1 -- 172.21.15.98:0/665700999 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc18003530 con 0x7fcc180039c0 2024-09-19T08:47:14.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.514+0000 7fcc2ffff640 1 --2- 172.21.15.98:0/665700999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc180039c0 0x7fcc18003de0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fcc20009ae0 tx=0x7fcc2001ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d20e6bb4197116b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:14.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.514+0000 7fcc2effd640 1 -- 172.21.15.98:0/665700999 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc200b2070 con 0x7fcc180039c0 2024-09-19T08:47:14.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.514+0000 7fcc2effd640 1 -- 172.21.15.98:0/665700999 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc200a5670 con 0x7fcc180039c0 2024-09-19T08:47:14.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.514+0000 7fcc2effd640 1 -- 172.21.15.98:0/665700999 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc200096b0 con 0x7fcc180039c0 2024-09-19T08:47:14.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.514+0000 7fcc367cb640 1 -- 172.21.15.98:0/665700999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc180039c0 msgr2=0x7fcc18003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:14.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.514+0000 7fcc367cb640 1 --2- 172.21.15.98:0/665700999 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc180039c0 0x7fcc18003de0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fcc20009ae0 tx=0x7fcc2001ca80 comp rx=0 tx=0).stop 2024-09-19T08:47:14.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.514+0000 7fcc34d68640 1 -- 172.21.15.98:0/665700999 reap_dead start 2024-09-19T08:47:14.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.514+0000 7fcc367cb640 1 -- 172.21.15.98:0/665700999 shutdown_connections 2024-09-19T08:47:14.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.514+0000 7fcc367cb640 1 -- 172.21.15.98:0/665700999 >> 172.21.15.98:0/665700999 conn(0x7fcc30074e40 msgr2=0x7fcc300722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:14.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.514+0000 7fcc367cb640 1 -- 172.21.15.98:0/665700999 shutdown_connections 2024-09-19T08:47:14.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.514+0000 7fcc367cb640 1 -- 172.21.15.98:0/665700999 wait complete. 2024-09-19T08:47:14.514 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.514+0000 7fcc367cb640 1 Processor -- start 2024-09-19T08:47:14.515 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.514+0000 7fcc367cb640 1 -- start start 2024-09-19T08:47:14.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.514+0000 7fcc367cb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc180039c0 0x7fcc301453e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:14.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.514+0000 7fcc367cb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc30140f60 0x7fcc301413c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:14.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.514+0000 7fcc367cb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc30160e70 0x7fcc301612f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:14.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.514+0000 7fcc367cb640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcc300b6a20 con 0x7fcc30160e70 2024-09-19T08:47:14.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.514+0000 7fcc367cb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcc30147b50 con 0x7fcc180039c0 2024-09-19T08:47:14.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.514+0000 7fcc367cb640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcc30141900 con 0x7fcc30140f60 2024-09-19T08:47:14.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.514+0000 7fcc34d68640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc30160e70 0x7fcc301612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:14.517 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.514+0000 7fcc34d68640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc30160e70 0x7fcc301612f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35194/0 (socket says 172.21.15.98:35194) 2024-09-19T08:47:14.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.514+0000 7fcc34d68640 1 -- 172.21.15.98:0/25944403 learned_addr learned my addr 172.21.15.98:0/25944403 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:14.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.514+0000 7fcc2f7fe640 1 --2- 172.21.15.98:0/25944403 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc30140f60 0x7fcc301413c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:14.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.514+0000 7fcc2ffff640 1 --2- 172.21.15.98:0/25944403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc180039c0 0x7fcc301453e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:14.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.514+0000 7fcc34d68640 1 -- 172.21.15.98:0/25944403 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc30140f60 msgr2=0x7fcc301413c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:14.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.514+0000 7fcc34d68640 1 --2- 172.21.15.98:0/25944403 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc30140f60 0x7fcc301413c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:14.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.514+0000 7fcc34d68640 1 -- 172.21.15.98:0/25944403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc180039c0 msgr2=0x7fcc301453e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:14.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.514+0000 7fcc34d68640 1 --2- 172.21.15.98:0/25944403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc180039c0 0x7fcc301453e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:14.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.514+0000 7fcc34d68640 1 -- 172.21.15.98:0/25944403 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc30161c10 con 0x7fcc30160e70 2024-09-19T08:47:14.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.514+0000 7fcc34d68640 1 --2- 172.21.15.98:0/25944403 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc30160e70 0x7fcc301612f0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fcc300c9740 tx=0x7fcc280115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:14.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.514+0000 7fcc2cff9640 1 -- 172.21.15.98:0/25944403 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc28011e10 con 0x7fcc30160e70 2024-09-19T08:47:14.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.514+0000 7fcc2cff9640 1 -- 172.21.15.98:0/25944403 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc2801b070 con 0x7fcc30160e70 2024-09-19T08:47:14.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.514+0000 7fcc2cff9640 1 -- 172.21.15.98:0/25944403 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc280039a0 con 0x7fcc30160e70 2024-09-19T08:47:14.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.514+0000 7fcc367cb640 1 -- 172.21.15.98:0/25944403 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc301439e0 con 0x7fcc30160e70 2024-09-19T08:47:14.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.514+0000 7fcc367cb640 1 -- 172.21.15.98:0/25944403 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc30143fa0 con 0x7fcc30160e70 2024-09-19T08:47:14.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.518+0000 7fcc367cb640 1 -- 172.21.15.98:0/25944403 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc300b9290 con 0x7fcc30160e70 2024-09-19T08:47:14.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.518+0000 7fcc2cff9640 1 -- 172.21.15.98:0/25944403 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7fcc28003b60 con 0x7fcc30160e70 2024-09-19T08:47:14.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.518+0000 7fcc2cff9640 1 --2- 172.21.15.98:0/25944403 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcc1c068250 0x7fcc1c06a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:14.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.518+0000 7fcc2cff9640 1 -- 172.21.15.98:0/25944403 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 5737+0+0 (secure 0 0 0) 0x7fcc28084d00 con 0x7fcc30160e70 2024-09-19T08:47:14.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.518+0000 7fcc2ffff640 1 --2- 172.21.15.98:0/25944403 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcc1c068250 0x7fcc1c06a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:14.520 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.518+0000 7fcc2ffff640 1 --2- 172.21.15.98:0/25944403 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcc1c068250 0x7fcc1c06a710 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fcc200af000 tx=0x7fcc200046d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:14.521 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.522+0000 7fcc2cff9640 1 -- 172.21.15.98:0/25944403 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fcc28025020 con 0x7fcc30160e70 2024-09-19T08:47:14.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.686+0000 7fcc367cb640 1 -- 172.21.15.98:0/25944403 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcc3015e010 con 0x7fcc30160e70 2024-09-19T08:47:14.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.690+0000 7fcc2cff9640 1 -- 172.21.15.98:0/25944403 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) ==== 74+0+16800 (secure 0 0 0) 0x7fcc280573b0 con 0x7fcc30160e70 2024-09-19T08:47:14.688 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:47:14.688 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":169,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:47:13.742666+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:47:12.300410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-19T08:47:12.721062+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:47:14.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.690+0000 7fcc367cb640 1 -- 172.21.15.98:0/25944403 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcc1c068250 msgr2=0x7fcc1c06a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:14.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.690+0000 7fcc367cb640 1 --2- 172.21.15.98:0/25944403 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcc1c068250 0x7fcc1c06a710 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fcc200af000 tx=0x7fcc200046d0 comp rx=0 tx=0).stop 2024-09-19T08:47:14.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.690+0000 7fcc367cb640 1 -- 172.21.15.98:0/25944403 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc30160e70 msgr2=0x7fcc301612f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:14.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.690+0000 7fcc367cb640 1 --2- 172.21.15.98:0/25944403 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc30160e70 0x7fcc301612f0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fcc300c9740 tx=0x7fcc280115a0 comp rx=0 tx=0).stop 2024-09-19T08:47:14.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.690+0000 7fcc367cb640 1 -- 172.21.15.98:0/25944403 shutdown_connections 2024-09-19T08:47:14.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.690+0000 7fcc367cb640 1 --2- 172.21.15.98:0/25944403 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcc1c068250 0x7fcc1c06a710 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:14.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.690+0000 7fcc367cb640 1 --2- 172.21.15.98:0/25944403 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc30160e70 0x7fcc301612f0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:14.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.690+0000 7fcc367cb640 1 --2- 172.21.15.98:0/25944403 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc30140f60 0x7fcc301413c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:14.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.690+0000 7fcc367cb640 1 --2- 172.21.15.98:0/25944403 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc180039c0 0x7fcc301453e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:14.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.690+0000 7fcc367cb640 1 -- 172.21.15.98:0/25944403 >> 172.21.15.98:0/25944403 conn(0x7fcc30074e40 msgr2=0x7fcc300d0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:14.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.690+0000 7fcc367cb640 1 -- 172.21.15.98:0/25944403 shutdown_connections 2024-09-19T08:47:14.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.690+0000 7fcc367cb640 1 -- 172.21.15.98:0/25944403 wait complete. 2024-09-19T08:47:14.708 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:47:14.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.830+0000 7f560acb9640 1 Processor -- start 2024-09-19T08:47:14.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.830+0000 7f560acb9640 1 -- start start 2024-09-19T08:47:14.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.830+0000 7f560acb9640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f56040b7f80 0x7f560415f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:14.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.830+0000 7f560acb9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f56040b9970 0x7f56040b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:14.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.830+0000 7f560acb9640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5604068440 con 0x7f56040d4820 2024-09-19T08:47:14.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.830+0000 7f560acb9640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f5604163050 con 0x7f56040b7f80 2024-09-19T08:47:14.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.830+0000 7f560acb9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5604162a90 con 0x7f56040b9970 2024-09-19T08:47:14.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.830+0000 7f5608a55640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f56040b7f80 0x7f560415f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:14.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.830+0000 7f5608a55640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f56040b7f80 0x7f560415f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35202/0 (socket says 172.21.15.98:35202) 2024-09-19T08:47:14.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.830+0000 7f5608a55640 1 -- 172.21.15.98:0/3546831611 learned_addr learned my addr 172.21.15.98:0/3546831611 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:14.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.830+0000 7f5603fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f56040d4820 0x7f56041621b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40642/0 (socket says 172.21.15.98:40642) 2024-09-19T08:47:14.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.830+0000 7f5609256640 1 --2- 172.21.15.98:0/3546831611 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f56040b9970 0x7f56040b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:14.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.830+0000 7f56037fe640 1 -- 172.21.15.98:0/3546831611 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1744567493 0 0) 0x7f5604068440 con 0x7f56040d4820 2024-09-19T08:47:14.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.830+0000 7f56037fe640 1 -- 172.21.15.98:0/3546831611 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f55f0003530 con 0x7f56040d4820 2024-09-19T08:47:14.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.830+0000 7f56037fe640 1 -- 172.21.15.98:0/3546831611 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2704194363 0 0) 0x7f55f0003530 con 0x7f56040d4820 2024-09-19T08:47:14.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.830+0000 7f5609256640 1 -- 172.21.15.98:0/3546831611 >> v2:172.21.15.98:3300/0 conn(0x7f56040b7f80 msgr2=0x7f560415f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:14.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.830+0000 7f5609256640 1 --2- 172.21.15.98:0/3546831611 >> v2:172.21.15.98:3300/0 conn(0x7f56040b7f80 0x7f560415f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:14.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.830+0000 7f5609256640 1 -- 172.21.15.98:0/3546831611 >> v1:172.21.15.98:6789/0 conn(0x7f56040d4820 legacy=0x7f56041621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:14.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.830+0000 7f5609256640 1 -- 172.21.15.98:0/3546831611 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5604162d70 con 0x7f56040b9970 2024-09-19T08:47:14.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.834+0000 7f5609256640 1 --2- 172.21.15.98:0/3546831611 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f56040b9970 0x7f56040b9de0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f55fc009d90 tx=0x7f55fc01c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbc744afbfc2b8ae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:14.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.834+0000 7f56037fe640 1 -- 172.21.15.98:0/3546831611 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc0ac070 con 0x7f56040b9970 2024-09-19T08:47:14.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.834+0000 7f56037fe640 1 -- 172.21.15.98:0/3546831611 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55fc0a64a0 con 0x7f56040b9970 2024-09-19T08:47:14.833 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.834+0000 7f56037fe640 1 -- 172.21.15.98:0/3546831611 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ec0016a0 con 0x7f56040b9970 2024-09-19T08:47:14.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.834+0000 7f560acb9640 1 -- 172.21.15.98:0/3546831611 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f56040b9970 msgr2=0x7f56040b9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:14.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.834+0000 7f560acb9640 1 --2- 172.21.15.98:0/3546831611 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f56040b9970 0x7f56040b9de0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f55fc009d90 tx=0x7f55fc01c890 comp rx=0 tx=0).stop 2024-09-19T08:47:14.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.834+0000 7f560acb9640 1 -- 172.21.15.98:0/3546831611 shutdown_connections 2024-09-19T08:47:14.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.834+0000 7f560acb9640 1 --2- 172.21.15.98:0/3546831611 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f56040b9970 0x7f56040b9de0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:14.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.834+0000 7f560acb9640 1 --2- 172.21.15.98:0/3546831611 >> v2:172.21.15.98:3300/0 conn(0x7f56040b7f80 0x7f560415f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:14.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.834+0000 7f560acb9640 1 -- 172.21.15.98:0/3546831611 >> 172.21.15.98:0/3546831611 conn(0x7f5604074e40 msgr2=0x7f5604075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:14.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.834+0000 7f560acb9640 1 -- 172.21.15.98:0/3546831611 shutdown_connections 2024-09-19T08:47:14.834 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.834+0000 7f560acb9640 1 -- 172.21.15.98:0/3546831611 wait complete. 2024-09-19T08:47:14.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.834+0000 7f560acb9640 1 Processor -- start 2024-09-19T08:47:14.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.834+0000 7f560acb9640 1 -- start start 2024-09-19T08:47:14.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.838+0000 7f560acb9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f56040b7f80 0x7f5604140930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:14.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.838+0000 7f560acb9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56040b9970 0x7f5604154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:14.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.838+0000 7f560acb9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56040d4820 0x7f5604142480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:14.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.838+0000 7f560acb9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f56040b84e0 con 0x7f56040d4820 2024-09-19T08:47:14.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.838+0000 7f560acb9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f56040b8360 con 0x7f56040b9970 2024-09-19T08:47:14.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.838+0000 7f560acb9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5604144590 con 0x7f56040b7f80 2024-09-19T08:47:14.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.838+0000 7f5609256640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56040d4820 0x7f5604142480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:14.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.838+0000 7f5609256640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56040d4820 0x7f5604142480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35208/0 (socket says 172.21.15.98:35208) 2024-09-19T08:47:14.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.838+0000 7f5609256640 1 -- 172.21.15.98:0/2435074438 learned_addr learned my addr 172.21.15.98:0/2435074438 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:14.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.838+0000 7f5608a55640 1 --2- 172.21.15.98:0/2435074438 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f56040b7f80 0x7f5604140930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:14.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.838+0000 7f5603fff640 1 --2- 172.21.15.98:0/2435074438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56040b9970 0x7f5604154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:14.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.838+0000 7f5609256640 1 -- 172.21.15.98:0/2435074438 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f56040b7f80 msgr2=0x7f5604140930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:14.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.838+0000 7f5609256640 1 --2- 172.21.15.98:0/2435074438 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f56040b7f80 0x7f5604140930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:14.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.838+0000 7f5609256640 1 -- 172.21.15.98:0/2435074438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56040b9970 msgr2=0x7f5604154110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:14.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.838+0000 7f5609256640 1 --2- 172.21.15.98:0/2435074438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56040b9970 0x7f5604154110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:14.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.838+0000 7f5609256640 1 -- 172.21.15.98:0/2435074438 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560415fd50 con 0x7f56040d4820 2024-09-19T08:47:14.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.838+0000 7f5609256640 1 --2- 172.21.15.98:0/2435074438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56040d4820 0x7f5604142480 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f55fc01d130 tx=0x7f55fc019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:14.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.838+0000 7f56017fa640 1 -- 172.21.15.98:0/2435074438 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc0ac070 con 0x7f56040d4820 2024-09-19T08:47:14.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.838+0000 7f56017fa640 1 -- 172.21.15.98:0/2435074438 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55fc0a6860 con 0x7f56040d4820 2024-09-19T08:47:14.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.838+0000 7f560acb9640 1 -- 172.21.15.98:0/2435074438 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5604140e70 con 0x7f56040d4820 2024-09-19T08:47:14.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.838+0000 7f56017fa640 1 -- 172.21.15.98:0/2435074438 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc0a6c10 con 0x7f56040d4820 2024-09-19T08:47:14.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.838+0000 7f560acb9640 1 -- 172.21.15.98:0/2435074438 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f560415fff0 con 0x7f56040d4820 2024-09-19T08:47:14.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.838+0000 7f560acb9640 1 -- 172.21.15.98:0/2435074438 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f560415f560 con 0x7f56040d4820 2024-09-19T08:47:14.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.838+0000 7f56017fa640 1 -- 172.21.15.98:0/2435074438 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f55fc0af070 con 0x7f56040d4820 2024-09-19T08:47:14.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.838+0000 7f56017fa640 1 --2- 172.21.15.98:0/2435074438 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f55e4068250 0x7f55e406a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:14.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.838+0000 7f56017fa640 1 -- 172.21.15.98:0/2435074438 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 5737+0+0 (secure 0 0 0) 0x7f55fc11e5c0 con 0x7f56040d4820 2024-09-19T08:47:14.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.838+0000 7f5608a55640 1 --2- 172.21.15.98:0/2435074438 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f55e4068250 0x7f55e406a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:14.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.842+0000 7f5608a55640 1 --2- 172.21.15.98:0/2435074438 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f55e4068250 0x7f55e406a710 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f55f40045f0 tx=0x7f55f4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:14.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:14.842+0000 7f56017fa640 1 -- 172.21.15.98:0/2435074438 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f55fc0c1060 con 0x7f56040d4820 2024-09-19T08:47:15.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.006+0000 7f560acb9640 1 -- 172.21.15.98:0/2435074438 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56040755a0 con 0x7f56040d4820 2024-09-19T08:47:15.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.010+0000 7f56017fa640 1 -- 172.21.15.98:0/2435074438 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) ==== 74+0+16800 (secure 0 0 0) 0x7f55fc016480 con 0x7f56040d4820 2024-09-19T08:47:15.008 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:47:15.009 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":169,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:47:13.742666+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:47:12.300410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-19T08:47:12.721062+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:47:15.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.010+0000 7f560acb9640 1 -- 172.21.15.98:0/2435074438 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f55e4068250 msgr2=0x7f55e406a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:15.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.010+0000 7f560acb9640 1 --2- 172.21.15.98:0/2435074438 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f55e4068250 0x7f55e406a710 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f55f40045f0 tx=0x7f55f4008040 comp rx=0 tx=0).stop 2024-09-19T08:47:15.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.010+0000 7f560acb9640 1 -- 172.21.15.98:0/2435074438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56040d4820 msgr2=0x7f5604142480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:15.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.010+0000 7f560acb9640 1 --2- 172.21.15.98:0/2435074438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56040d4820 0x7f5604142480 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f55fc01d130 tx=0x7f55fc019630 comp rx=0 tx=0).stop 2024-09-19T08:47:15.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.010+0000 7f560acb9640 1 -- 172.21.15.98:0/2435074438 shutdown_connections 2024-09-19T08:47:15.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.010+0000 7f560acb9640 1 --2- 172.21.15.98:0/2435074438 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f55e4068250 0x7f55e406a710 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:15.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.010+0000 7f560acb9640 1 --2- 172.21.15.98:0/2435074438 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f56040d4820 0x7f5604142480 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:15.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.010+0000 7f560acb9640 1 --2- 172.21.15.98:0/2435074438 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56040b9970 0x7f5604154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:15.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.010+0000 7f560acb9640 1 --2- 172.21.15.98:0/2435074438 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f56040b7f80 0x7f5604140930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:15.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.010+0000 7f560acb9640 1 -- 172.21.15.98:0/2435074438 >> 172.21.15.98:0/2435074438 conn(0x7f5604074e40 msgr2=0x7f56040d0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:15.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.010+0000 7f560acb9640 1 -- 172.21.15.98:0/2435074438 shutdown_connections 2024-09-19T08:47:15.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.010+0000 7f560acb9640 1 -- 172.21.15.98:0/2435074438 wait complete. 2024-09-19T08:47:15.027 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-19T08:47:15.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.154+0000 7f84851b4640 1 Processor -- start 2024-09-19T08:47:15.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.154+0000 7f84851b4640 1 -- start start 2024-09-19T08:47:15.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.154+0000 7f84851b4640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8480145b70 0x7f848006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:15.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.154+0000 7f84851b4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f848006c1d0 0x7f84800713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:15.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.154+0000 7f84851b4640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8480073530 con 0x7f84800718f0 2024-09-19T08:47:15.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.154+0000 7f84851b4640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f848006c620 con 0x7f8480145b70 2024-09-19T08:47:15.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.154+0000 7f84851b4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f8480073810 con 0x7f848006c1d0 2024-09-19T08:47:15.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.154+0000 7f847ed9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f8480145b70 0x7f848006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:15.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.154+0000 7f847f59e640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f84800718f0 0x7f84800cef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40650/0 (socket says 172.21.15.98:40650) 2024-09-19T08:47:15.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.154+0000 7f847f59e640 1 -- 172.21.15.98:0/1943035775 learned_addr learned my addr 172.21.15.98:0/1943035775 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:15.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.154+0000 7f847e59c640 1 --2- 172.21.15.98:0/1943035775 >> v2:172.21.15.139:3300/0 conn(0x7f848006c1d0 0x7f84800713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:15.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.154+0000 7f847ed9d640 1 -- 172.21.15.98:0/1943035775 >> v2:172.21.15.139:3300/0 conn(0x7f848006c1d0 msgr2=0x7f84800713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:15.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.154+0000 7f847ed9d640 1 --2- 172.21.15.98:0/1943035775 >> v2:172.21.15.139:3300/0 conn(0x7f848006c1d0 0x7f84800713b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:15.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.154+0000 7f847ed9d640 1 -- 172.21.15.98:0/1943035775 >> v1:172.21.15.98:6789/0 conn(0x7f84800718f0 legacy=0x7f84800cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:15.155 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.154+0000 7f847ed9d640 1 -- 172.21.15.98:0/1943035775 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8480073af0 con 0x7f8480145b70 2024-09-19T08:47:15.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.158+0000 7f847ed9d640 1 --2- 172.21.15.98:0/1943035775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8480145b70 0x7f848006bc90 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f84800ccd30 tx=0x7f847001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfcd20f1a92ad202 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:15.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.158+0000 7f847dd9b640 1 -- 172.21.15.98:0/1943035775 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84700b0070 con 0x7f8480145b70 2024-09-19T08:47:15.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.158+0000 7f847dd9b640 1 -- 172.21.15.98:0/1943035775 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84700ab600 con 0x7f8480145b70 2024-09-19T08:47:15.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.158+0000 7f847dd9b640 1 -- 172.21.15.98:0/1943035775 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84700ab9d0 con 0x7f8480145b70 2024-09-19T08:47:15.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.158+0000 7f84851b4640 1 -- 172.21.15.98:0/1943035775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8480145b70 msgr2=0x7f848006bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:15.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.158+0000 7f84851b4640 1 --2- 172.21.15.98:0/1943035775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8480145b70 0x7f848006bc90 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f84800ccd30 tx=0x7f847001c3f0 comp rx=0 tx=0).stop 2024-09-19T08:47:15.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.158+0000 7f84851b4640 1 -- 172.21.15.98:0/1943035775 shutdown_connections 2024-09-19T08:47:15.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.158+0000 7f84851b4640 1 --2- 172.21.15.98:0/1943035775 >> v2:172.21.15.139:3300/0 conn(0x7f848006c1d0 0x7f84800713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:15.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.158+0000 7f84851b4640 1 --2- 172.21.15.98:0/1943035775 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8480145b70 0x7f848006bc90 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:15.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.158+0000 7f84851b4640 1 -- 172.21.15.98:0/1943035775 >> 172.21.15.98:0/1943035775 conn(0x7f84800b2010 msgr2=0x7f84800b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:15.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.158+0000 7f84851b4640 1 -- 172.21.15.98:0/1943035775 shutdown_connections 2024-09-19T08:47:15.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.158+0000 7f84851b4640 1 -- 172.21.15.98:0/1943035775 wait complete. 2024-09-19T08:47:15.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.158+0000 7f84851b4640 1 Processor -- start 2024-09-19T08:47:15.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.158+0000 7f84851b4640 1 -- start start 2024-09-19T08:47:15.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.158+0000 7f84851b4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f848006c1d0 0x7f848013b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:15.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.158+0000 7f84851b4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84800718f0 0x7f848013b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:15.160 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.158+0000 7f84851b4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8480145b70 0x7f848015c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:15.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.158+0000 7f84851b4640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8480169070 con 0x7f84800718f0 2024-09-19T08:47:15.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.158+0000 7f84851b4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f84801691f0 con 0x7f848006c1d0 2024-09-19T08:47:15.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.158+0000 7f84851b4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8480169370 con 0x7f8480145b70 2024-09-19T08:47:15.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.158+0000 7f847e59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84800718f0 0x7f848013b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:15.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.158+0000 7f847e59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84800718f0 0x7f848013b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35228/0 (socket says 172.21.15.98:35228) 2024-09-19T08:47:15.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.158+0000 7f847e59c640 1 -- 172.21.15.98:0/3159530975 learned_addr learned my addr 172.21.15.98:0/3159530975 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:15.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.158+0000 7f847f59e640 1 --2- 172.21.15.98:0/3159530975 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8480145b70 0x7f848015c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:15.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.158+0000 7f847ed9d640 1 --2- 172.21.15.98:0/3159530975 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f848006c1d0 0x7f848013b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:15.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.162+0000 7f847e59c640 1 -- 172.21.15.98:0/3159530975 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8480145b70 msgr2=0x7f848015c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:15.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.162+0000 7f847e59c640 1 --2- 172.21.15.98:0/3159530975 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8480145b70 0x7f848015c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:15.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.162+0000 7f847e59c640 1 -- 172.21.15.98:0/3159530975 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f848006c1d0 msgr2=0x7f848013b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:15.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.162+0000 7f847e59c640 1 --2- 172.21.15.98:0/3159530975 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f848006c1d0 0x7f848013b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:15.161 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.162+0000 7f847e59c640 1 -- 172.21.15.98:0/3159530975 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f848013fd60 con 0x7f84800718f0 2024-09-19T08:47:15.162 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.162+0000 7f847ed9d640 1 --2- 172.21.15.98:0/3159530975 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f848006c1d0 0x7f848013b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:15.162 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.162+0000 7f847e59c640 1 --2- 172.21.15.98:0/3159530975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84800718f0 0x7f848013b750 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f847400c7f0 tx=0x7f847400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:15.162 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.162+0000 7f84677fe640 1 -- 172.21.15.98:0/3159530975 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8474015070 con 0x7f84800718f0 2024-09-19T08:47:15.162 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.162+0000 7f84677fe640 1 -- 172.21.15.98:0/3159530975 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8474004240 con 0x7f84800718f0 2024-09-19T08:47:15.162 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.162+0000 7f84851b4640 1 -- 172.21.15.98:0/3159530975 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f848013c670 con 0x7f84800718f0 2024-09-19T08:47:15.162 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.162+0000 7f84677fe640 1 -- 172.21.15.98:0/3159530975 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8474003620 con 0x7f84800718f0 2024-09-19T08:47:15.162 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.162+0000 7f84851b4640 1 -- 172.21.15.98:0/3159530975 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f848013cc30 con 0x7f84800718f0 2024-09-19T08:47:15.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.162+0000 7f84851b4640 1 -- 172.21.15.98:0/3159530975 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84800713b0 con 0x7f84800718f0 2024-09-19T08:47:15.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.162+0000 7f84677fe640 1 -- 172.21.15.98:0/3159530975 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f84740039d0 con 0x7f84800718f0 2024-09-19T08:47:15.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.162+0000 7f84677fe640 1 --2- 172.21.15.98:0/3159530975 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f846c068250 0x7f846c06a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:15.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.162+0000 7f84677fe640 1 -- 172.21.15.98:0/3159530975 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 5737+0+0 (secure 0 0 0) 0x7f8474084450 con 0x7f84800718f0 2024-09-19T08:47:15.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.162+0000 7f847ed9d640 1 --2- 172.21.15.98:0/3159530975 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f846c068250 0x7f846c06a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:15.163 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.162+0000 7f847ed9d640 1 --2- 172.21.15.98:0/3159530975 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f846c068250 0x7f846c06a710 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f84700ad000 tx=0x7f8470008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:15.166 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.166+0000 7f84677fe640 1 -- 172.21.15.98:0/3159530975 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f8474056b90 con 0x7f84800718f0 2024-09-19T08:47:15.365 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:15.366+0000 7f84851b4640 1 -- 172.21.15.98:0/3159530975 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f84800af920 con 0x7f84800718f0 2024-09-19T08:47:16.325 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.326+0000 7f84677fe640 1 -- 172.21.15.98:0/3159530975 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v99) ==== 118+0+0 (secure 0 0 0) 0x7f8474056490 con 0x7f84800718f0 2024-09-19T08:47:16.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.330+0000 7f84851b4640 1 -- 172.21.15.98:0/3159530975 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f846c068250 msgr2=0x7f846c06a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:16.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.330+0000 7f84851b4640 1 --2- 172.21.15.98:0/3159530975 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f846c068250 0x7f846c06a710 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f84700ad000 tx=0x7f8470008040 comp rx=0 tx=0).stop 2024-09-19T08:47:16.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.330+0000 7f84851b4640 1 -- 172.21.15.98:0/3159530975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84800718f0 msgr2=0x7f848013b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:16.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.330+0000 7f84851b4640 1 --2- 172.21.15.98:0/3159530975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84800718f0 0x7f848013b750 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f847400c7f0 tx=0x7f847400ccb0 comp rx=0 tx=0).stop 2024-09-19T08:47:16.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.330+0000 7f84851b4640 1 -- 172.21.15.98:0/3159530975 shutdown_connections 2024-09-19T08:47:16.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.330+0000 7f84851b4640 1 --2- 172.21.15.98:0/3159530975 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f846c068250 0x7f846c06a710 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:16.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.330+0000 7f84851b4640 1 --2- 172.21.15.98:0/3159530975 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8480145b70 0x7f848015c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:16.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.330+0000 7f84851b4640 1 --2- 172.21.15.98:0/3159530975 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f84800718f0 0x7f848013b750 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:16.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.330+0000 7f84851b4640 1 --2- 172.21.15.98:0/3159530975 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f848006c1d0 0x7f848013b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:16.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.330+0000 7f84851b4640 1 -- 172.21.15.98:0/3159530975 >> 172.21.15.98:0/3159530975 conn(0x7f84800b2010 msgr2=0x7f84800b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:16.329 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.330+0000 7f84851b4640 1 -- 172.21.15.98:0/3159530975 shutdown_connections 2024-09-19T08:47:16.330 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.330+0000 7f84851b4640 1 -- 172.21.15.98:0/3159530975 wait complete. 2024-09-19T08:47:16.353 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-19T08:47:16.353 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-19T08:47:16.476 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.478+0000 7fe85b72c640 1 Processor -- start 2024-09-19T08:47:16.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.478+0000 7fe85b72c640 1 -- start start 2024-09-19T08:47:16.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.478+0000 7fe85b72c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fe8540d27c0 0x7fe8540d3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:16.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.478+0000 7fe85b72c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe8540d38a0 0x7fe8540b8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:16.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.478+0000 7fe85b72c640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe85415eae0 con 0x7fe8541415a0 2024-09-19T08:47:16.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.478+0000 7fe85b72c640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fe85415edc0 con 0x7fe8540d38a0 2024-09-19T08:47:16.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.478+0000 7fe85b72c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fe85415f0a0 con 0x7fe8540d27c0 2024-09-19T08:47:16.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.478+0000 7fe8594c8640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fe8541415a0 0x7fe8540d20b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40654/0 (socket says 172.21.15.98:40654) 2024-09-19T08:47:16.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.478+0000 7fe8594c8640 1 -- 172.21.15.98:0/2690565518 learned_addr learned my addr 172.21.15.98:0/2690565518 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:16.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.478+0000 7fe859cc9640 1 --2- 172.21.15.98:0/2690565518 >> v2:172.21.15.98:3300/0 conn(0x7fe8540d38a0 0x7fe8540b8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:16.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.478+0000 7fe847fff640 1 -- 172.21.15.98:0/2690565518 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 75264441 0 0) 0x7fe85415eae0 con 0x7fe8541415a0 2024-09-19T08:47:16.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.478+0000 7fe847fff640 1 -- 172.21.15.98:0/2690565518 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe840003530 con 0x7fe8541415a0 2024-09-19T08:47:16.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.478+0000 7fe858cc7640 1 --2- 172.21.15.98:0/2690565518 >> v2:172.21.15.139:3300/0 conn(0x7fe8540d27c0 0x7fe8540d3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:16.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.478+0000 7fe859cc9640 1 -- 172.21.15.98:0/2690565518 >> v2:172.21.15.139:3300/0 conn(0x7fe8540d27c0 msgr2=0x7fe8540d3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:16.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.478+0000 7fe859cc9640 1 --2- 172.21.15.98:0/2690565518 >> v2:172.21.15.139:3300/0 conn(0x7fe8540d27c0 0x7fe8540d3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:16.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.478+0000 7fe859cc9640 1 -- 172.21.15.98:0/2690565518 >> v1:172.21.15.98:6789/0 conn(0x7fe8541415a0 legacy=0x7fe8540d20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:16.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.478+0000 7fe847fff640 1 -- 172.21.15.98:0/2690565518 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3726421646 0 0) 0x7fe840003530 con 0x7fe8541415a0 2024-09-19T08:47:16.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.478+0000 7fe859cc9640 1 -- 172.21.15.98:0/2690565518 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe85415f380 con 0x7fe8540d38a0 2024-09-19T08:47:16.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.482+0000 7fe859cc9640 1 --2- 172.21.15.98:0/2690565518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8540d38a0 0x7fe8540b8790 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fe85415c5c0 tx=0x7fe84c01c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e0cacbf8c4d464 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:16.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.482+0000 7fe847fff640 1 -- 172.21.15.98:0/2690565518 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c0b2070 con 0x7fe8540d38a0 2024-09-19T08:47:16.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.482+0000 7fe847fff640 1 -- 172.21.15.98:0/2690565518 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe84c0a5620 con 0x7fe8540d38a0 2024-09-19T08:47:16.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.482+0000 7fe847fff640 1 -- 172.21.15.98:0/2690565518 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe850001680 con 0x7fe8540d38a0 2024-09-19T08:47:16.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.482+0000 7fe85b72c640 1 -- 172.21.15.98:0/2690565518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8540d38a0 msgr2=0x7fe8540b8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:16.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.482+0000 7fe85b72c640 1 --2- 172.21.15.98:0/2690565518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8540d38a0 0x7fe8540b8790 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fe85415c5c0 tx=0x7fe84c01c7c0 comp rx=0 tx=0).stop 2024-09-19T08:47:16.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.482+0000 7fe85b72c640 1 -- 172.21.15.98:0/2690565518 shutdown_connections 2024-09-19T08:47:16.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.482+0000 7fe85b72c640 1 --2- 172.21.15.98:0/2690565518 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8540d38a0 0x7fe8540b8790 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:16.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.482+0000 7fe85b72c640 1 --2- 172.21.15.98:0/2690565518 >> v2:172.21.15.139:3300/0 conn(0x7fe8540d27c0 0x7fe8540d3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:16.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.482+0000 7fe85b72c640 1 -- 172.21.15.98:0/2690565518 >> 172.21.15.98:0/2690565518 conn(0x7fe854074c70 msgr2=0x7fe854075080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:16.482 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.482+0000 7fe85b72c640 1 -- 172.21.15.98:0/2690565518 shutdown_connections 2024-09-19T08:47:16.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.482+0000 7fe85b72c640 1 -- 172.21.15.98:0/2690565518 wait complete. 2024-09-19T08:47:16.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.482+0000 7fe85b72c640 1 Processor -- start 2024-09-19T08:47:16.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.482+0000 7fe85b72c640 1 -- start start 2024-09-19T08:47:16.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.486+0000 7fe85b72c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8540d27c0 0x7fe8541613b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:16.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.486+0000 7fe85b72c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8540d38a0 0x7fe85415fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:16.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.486+0000 7fe85b72c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8541415a0 0x7fe854160330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:16.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.486+0000 7fe85b72c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe8540b9660 con 0x7fe8540d27c0 2024-09-19T08:47:16.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.486+0000 7fe85b72c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe8540b94e0 con 0x7fe8541415a0 2024-09-19T08:47:16.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.486+0000 7fe85b72c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe8540b97e0 con 0x7fe8540d38a0 2024-09-19T08:47:16.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.486+0000 7fe8594c8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8540d27c0 0x7fe8541613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:16.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.486+0000 7fe8594c8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8540d27c0 0x7fe8541613b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35242/0 (socket says 172.21.15.98:35242) 2024-09-19T08:47:16.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.486+0000 7fe8594c8640 1 -- 172.21.15.98:0/3825004763 learned_addr learned my addr 172.21.15.98:0/3825004763 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:16.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.486+0000 7fe859cc9640 1 --2- 172.21.15.98:0/3825004763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8541415a0 0x7fe854160330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:16.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.486+0000 7fe858cc7640 1 --2- 172.21.15.98:0/3825004763 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8540d38a0 0x7fe85415fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:16.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.486+0000 7fe8594c8640 1 -- 172.21.15.98:0/3825004763 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8540d38a0 msgr2=0x7fe85415fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:16.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.486+0000 7fe8594c8640 1 --2- 172.21.15.98:0/3825004763 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8540d38a0 0x7fe85415fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:16.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.486+0000 7fe8594c8640 1 -- 172.21.15.98:0/3825004763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8541415a0 msgr2=0x7fe854160330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:16.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.486+0000 7fe8594c8640 1 --2- 172.21.15.98:0/3825004763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8541415a0 0x7fe854160330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:16.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.486+0000 7fe8594c8640 1 -- 172.21.15.98:0/3825004763 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8541619d0 con 0x7fe8540d27c0 2024-09-19T08:47:16.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.486+0000 7fe859cc9640 1 --2- 172.21.15.98:0/3825004763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8541415a0 0x7fe854160330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:47:16.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.486+0000 7fe8594c8640 1 --2- 172.21.15.98:0/3825004763 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8540d27c0 0x7fe8541613b0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fe8500017f0 tx=0x7fe85000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:16.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.486+0000 7fe845ffb640 1 -- 172.21.15.98:0/3825004763 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe850005ae0 con 0x7fe8540d27c0 2024-09-19T08:47:16.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.486+0000 7fe845ffb640 1 -- 172.21.15.98:0/3825004763 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe850005ca0 con 0x7fe8540d27c0 2024-09-19T08:47:16.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.486+0000 7fe85b72c640 1 -- 172.21.15.98:0/3825004763 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe854161cb0 con 0x7fe8540d27c0 2024-09-19T08:47:16.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.486+0000 7fe845ffb640 1 -- 172.21.15.98:0/3825004763 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8500072b0 con 0x7fe8540d27c0 2024-09-19T08:47:16.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.486+0000 7fe85b72c640 1 -- 172.21.15.98:0/3825004763 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe854162270 con 0x7fe8540d27c0 2024-09-19T08:47:16.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.486+0000 7fe85b72c640 1 -- 172.21.15.98:0/3825004763 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8540d3360 con 0x7fe8540d27c0 2024-09-19T08:47:16.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.486+0000 7fe845ffb640 1 -- 172.21.15.98:0/3825004763 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7fe850007450 con 0x7fe8540d27c0 2024-09-19T08:47:16.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.486+0000 7fe845ffb640 1 --2- 172.21.15.98:0/3825004763 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe838068280 0x7fe83806a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:16.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.486+0000 7fe845ffb640 1 -- 172.21.15.98:0/3825004763 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5737+0+0 (secure 0 0 0) 0x7fe850084c90 con 0x7fe8540d27c0 2024-09-19T08:47:16.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.490+0000 7fe858cc7640 1 --2- 172.21.15.98:0/3825004763 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe838068280 0x7fe83806a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:16.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.490+0000 7fe858cc7640 1 --2- 172.21.15.98:0/3825004763 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe838068280 0x7fe83806a740 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe8540d06a0 tx=0x7fe848008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:16.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.490+0000 7fe845ffb640 1 -- 172.21.15.98:0/3825004763 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe8500573d0 con 0x7fe8540d27c0 2024-09-19T08:47:16.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:16.690+0000 7fe85b72c640 1 -- 172.21.15.98:0/3825004763 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe85413ff20 con 0x7fe8540d27c0 2024-09-19T08:47:17.317 INFO:tasks.ceph.mds.a.smithi098.stderr:2024-09-19T08:47:17.318+0000 7f87a13d3640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T08:47:17.343 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.342+0000 7fe845ffb640 1 -- 172.21.15.98:0/3825004763 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v100) ==== 122+0+0 (secure 0 0 0) 0x7fe850056cd0 con 0x7fe8540d27c0 2024-09-19T08:47:17.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.346+0000 7fe85b72c640 1 -- 172.21.15.98:0/3825004763 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe838068280 msgr2=0x7fe83806a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:17.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.346+0000 7fe85b72c640 1 --2- 172.21.15.98:0/3825004763 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe838068280 0x7fe83806a740 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe8540d06a0 tx=0x7fe848008040 comp rx=0 tx=0).stop 2024-09-19T08:47:17.346 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.346+0000 7fe85b72c640 1 -- 172.21.15.98:0/3825004763 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8540d27c0 msgr2=0x7fe8541613b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:17.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.346+0000 7fe85b72c640 1 --2- 172.21.15.98:0/3825004763 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8540d27c0 0x7fe8541613b0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fe8500017f0 tx=0x7fe85000cbe0 comp rx=0 tx=0).stop 2024-09-19T08:47:17.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.346+0000 7fe85b72c640 1 -- 172.21.15.98:0/3825004763 shutdown_connections 2024-09-19T08:47:17.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.346+0000 7fe85b72c640 1 --2- 172.21.15.98:0/3825004763 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe838068280 0x7fe83806a740 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:17.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.346+0000 7fe85b72c640 1 --2- 172.21.15.98:0/3825004763 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe8541415a0 0x7fe854160330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:17.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.346+0000 7fe85b72c640 1 --2- 172.21.15.98:0/3825004763 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe8540d38a0 0x7fe85415fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:17.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.346+0000 7fe85b72c640 1 --2- 172.21.15.98:0/3825004763 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe8540d27c0 0x7fe8541613b0 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:17.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.346+0000 7fe85b72c640 1 -- 172.21.15.98:0/3825004763 >> 172.21.15.98:0/3825004763 conn(0x7fe854074c70 msgr2=0x7fe85413f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:17.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.346+0000 7fe85b72c640 1 -- 172.21.15.98:0/3825004763 shutdown_connections 2024-09-19T08:47:17.347 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.346+0000 7fe85b72c640 1 -- 172.21.15.98:0/3825004763 wait complete. 2024-09-19T08:47:17.365 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-19T08:47:17.365 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:47:17.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.494+0000 7f80b90b2640 1 Processor -- start 2024-09-19T08:47:17.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.494+0000 7f80b90b2640 1 -- start start 2024-09-19T08:47:17.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.494+0000 7f80b90b2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80b406bfa0 0x7f80b406c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:17.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.494+0000 7f80b90b2640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80b415fb50 con 0x7f80b4160e60 2024-09-19T08:47:17.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.494+0000 7f80b90b2640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80b406c900 con 0x7f80b415f590 2024-09-19T08:47:17.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.494+0000 7f80b90b2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f80b415f970 con 0x7f80b406bfa0 2024-09-19T08:47:17.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.494+0000 7f80b359e640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f80b4160e60 0x7f80b41612d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:40670/0 (socket says 172.21.15.98:40670) 2024-09-19T08:47:17.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.494+0000 7f80b359e640 1 -- 172.21.15.98:0/153838290 learned_addr learned my addr 172.21.15.98:0/153838290 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:17.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.494+0000 7f80b259c640 1 --2- 172.21.15.98:0/153838290 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80b406bfa0 0x7f80b406c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:17.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.494+0000 7f80b1d9b640 1 -- 172.21.15.98:0/153838290 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1083824239 0 0) 0x7f80b415fb50 con 0x7f80b4160e60 2024-09-19T08:47:17.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.494+0000 7f80b1d9b640 1 -- 172.21.15.98:0/153838290 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f809c003530 con 0x7f80b4160e60 2024-09-19T08:47:17.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.494+0000 7f80b1d9b640 1 -- 172.21.15.98:0/153838290 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 520824205 0 0) 0x7f809c003530 con 0x7f80b4160e60 2024-09-19T08:47:17.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.494+0000 7f80b1d9b640 1 -- 172.21.15.98:0/153838290 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80b406bfa0 msgr2=0x7f80b406c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:17.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.494+0000 7f80b1d9b640 1 --2- 172.21.15.98:0/153838290 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80b406bfa0 0x7f80b406c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:17.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.494+0000 7f80b1d9b640 1 -- 172.21.15.98:0/153838290 >> v1:172.21.15.139:6789/0 conn(0x7f80b415f590 legacy=0x7f80b40cf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:17.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.494+0000 7f80b1d9b640 1 -- 172.21.15.98:0/153838290 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b415fdc0 con 0x7f80b4160e60 2024-09-19T08:47:17.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.494+0000 7f80b1d9b640 1 -- 172.21.15.98:0/153838290 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f80ac001a00 con 0x7f80b4160e60 2024-09-19T08:47:17.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.494+0000 7f80b1d9b640 1 -- 172.21.15.98:0/153838290 >> v1:172.21.15.98:6789/0 conn(0x7f80b4160e60 legacy=0x7f80b41612d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:17.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.494+0000 7f80b1d9b640 1 --2- 172.21.15.98:0/153838290 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80b406bfa0 0x7f809c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:17.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.494+0000 7f80b1d9b640 1 --2- 172.21.15.98:0/153838290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f809c003f70 0x7f809c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:17.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.494+0000 7f80b1d9b640 1 --2- 172.21.15.98:0/153838290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f809c004d50 0x7f809c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:17.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.494+0000 7f80b1d9b640 1 -- 172.21.15.98:0/153838290 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f80b415f970 con 0x7f809c003f70 2024-09-19T08:47:17.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.494+0000 7f80b1d9b640 1 -- 172.21.15.98:0/153838290 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f809c00bb40 con 0x7f809c004d50 2024-09-19T08:47:17.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.494+0000 7f80b1d9b640 1 -- 172.21.15.98:0/153838290 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f809c00bd40 con 0x7f80b406bfa0 2024-09-19T08:47:17.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.494+0000 7f80b259c640 1 --2- 172.21.15.98:0/153838290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f809c003f70 0x7f809c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:17.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.494+0000 7f80b2d9d640 1 --2- 172.21.15.98:0/153838290 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80b406bfa0 0x7f809c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:17.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.494+0000 7f80b359e640 1 --2- 172.21.15.98:0/153838290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f809c004d50 0x7f809c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:17.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.494+0000 7f80b259c640 1 -- 172.21.15.98:0/153838290 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80b406bfa0 msgr2=0x7f809c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:17.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.494+0000 7f80b259c640 1 --2- 172.21.15.98:0/153838290 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f80b406bfa0 0x7f809c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:17.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.494+0000 7f80b259c640 1 -- 172.21.15.98:0/153838290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f809c004d50 msgr2=0x7f809c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:17.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.494+0000 7f80b259c640 1 --2- 172.21.15.98:0/153838290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f809c004d50 0x7f809c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:17.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.494+0000 7f80b259c640 1 -- 172.21.15.98:0/153838290 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f809c003530 con 0x7f809c003f70 2024-09-19T08:47:17.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.494+0000 7f80b359e640 1 --2- 172.21.15.98:0/153838290 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f809c004d50 0x7f809c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:17.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.498+0000 7f80b259c640 1 --2- 172.21.15.98:0/153838290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f809c003f70 0x7f809c004810 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f80a8009ae0 tx=0x7f80a801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913eacd90e936241 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:17.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.498+0000 7f80b1d9b640 1 -- 172.21.15.98:0/153838290 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a80b2070 con 0x7f809c003f70 2024-09-19T08:47:17.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.498+0000 7f80b1d9b640 1 -- 172.21.15.98:0/153838290 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80a80a5660 con 0x7f809c003f70 2024-09-19T08:47:17.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.498+0000 7f80b1d9b640 1 -- 172.21.15.98:0/153838290 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a80096b0 con 0x7f809c003f70 2024-09-19T08:47:17.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.498+0000 7f80b90b2640 1 -- 172.21.15.98:0/153838290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f809c003f70 msgr2=0x7f809c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:17.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.498+0000 7f80b90b2640 1 --2- 172.21.15.98:0/153838290 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f809c003f70 0x7f809c004810 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f80a8009ae0 tx=0x7f80a801ca70 comp rx=0 tx=0).stop 2024-09-19T08:47:17.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.498+0000 7f80b359e640 1 -- 172.21.15.98:0/153838290 reap_dead start 2024-09-19T08:47:17.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.498+0000 7f80b90b2640 1 -- 172.21.15.98:0/153838290 shutdown_connections 2024-09-19T08:47:17.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.498+0000 7f80b90b2640 1 -- 172.21.15.98:0/153838290 >> 172.21.15.98:0/153838290 conn(0x7f80b4074c70 msgr2=0x7f80b4075080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:17.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.498+0000 7f80b90b2640 1 -- 172.21.15.98:0/153838290 shutdown_connections 2024-09-19T08:47:17.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.498+0000 7f80b90b2640 1 -- 172.21.15.98:0/153838290 wait complete. 2024-09-19T08:47:17.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.498+0000 7f80b90b2640 1 Processor -- start 2024-09-19T08:47:17.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.498+0000 7f80b90b2640 1 -- start start 2024-09-19T08:47:17.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.498+0000 7f80b90b2640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f809c003f70 0x7f80b4155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:17.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.498+0000 7f80b90b2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80b414aed0 0x7f80b4158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:17.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.498+0000 7f80b90b2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80b4152fc0 0x7f80b4153420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:17.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.498+0000 7f80b90b2640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f80b4160660 con 0x7f80b414aed0 2024-09-19T08:47:17.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.498+0000 7f80b90b2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f80b4160380 con 0x7f80b4152fc0 2024-09-19T08:47:17.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.498+0000 7f80b90b2640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f80b4160940 con 0x7f809c003f70 2024-09-19T08:47:17.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.498+0000 7f80b259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80b414aed0 0x7f80b4158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:17.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.498+0000 7f80b259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80b414aed0 0x7f80b4158fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:35272/0 (socket says 172.21.15.98:35272) 2024-09-19T08:47:17.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.498+0000 7f80b259c640 1 -- 172.21.15.98:0/4039277287 learned_addr learned my addr 172.21.15.98:0/4039277287 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:17.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.498+0000 7f80b2d9d640 1 --2- 172.21.15.98:0/4039277287 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f809c003f70 0x7f80b4155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:17.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.502+0000 7f80b359e640 1 --2- 172.21.15.98:0/4039277287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80b4152fc0 0x7f80b4153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:17.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.502+0000 7f80b259c640 1 -- 172.21.15.98:0/4039277287 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f809c003f70 msgr2=0x7f80b4155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:17.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.502+0000 7f80b259c640 1 --2- 172.21.15.98:0/4039277287 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f809c003f70 0x7f80b4155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:17.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.502+0000 7f80b259c640 1 -- 172.21.15.98:0/4039277287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80b4152fc0 msgr2=0x7f80b4153420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:17.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.502+0000 7f80b259c640 1 --2- 172.21.15.98:0/4039277287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80b4152fc0 0x7f80b4153420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:17.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.502+0000 7f80b259c640 1 -- 172.21.15.98:0/4039277287 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b415a0b0 con 0x7f80b414aed0 2024-09-19T08:47:17.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.502+0000 7f80b2d9d640 1 --2- 172.21.15.98:0/4039277287 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f809c003f70 0x7f80b4155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:17.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.502+0000 7f80b359e640 1 --2- 172.21.15.98:0/4039277287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80b4152fc0 0x7f80b4153420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:17.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.502+0000 7f80b259c640 1 --2- 172.21.15.98:0/4039277287 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80b414aed0 0x7f80b4158fc0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f80a801c8a0 tx=0x7f809c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:17.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.502+0000 7f809b7fe640 1 -- 172.21.15.98:0/4039277287 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a80b2040 con 0x7f80b414aed0 2024-09-19T08:47:17.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.502+0000 7f809b7fe640 1 -- 172.21.15.98:0/4039277287 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80a8004020 con 0x7f80b414aed0 2024-09-19T08:47:17.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.502+0000 7f80b90b2640 1 -- 172.21.15.98:0/4039277287 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80b40b93f0 con 0x7f80b414aed0 2024-09-19T08:47:17.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.502+0000 7f809b7fe640 1 -- 172.21.15.98:0/4039277287 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a80043d0 con 0x7f80b414aed0 2024-09-19T08:47:17.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.502+0000 7f80b90b2640 1 -- 172.21.15.98:0/4039277287 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80b4159af0 con 0x7f80b414aed0 2024-09-19T08:47:17.503 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.502+0000 7f80b90b2640 1 -- 172.21.15.98:0/4039277287 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80b415d770 con 0x7f80b414aed0 2024-09-19T08:47:17.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.502+0000 7f809b7fe640 1 -- 172.21.15.98:0/4039277287 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f80a80bd020 con 0x7f80b414aed0 2024-09-19T08:47:17.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.502+0000 7f809b7fe640 1 --2- 172.21.15.98:0/4039277287 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f80a0068250 0x7f80a006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:17.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.502+0000 7f809b7fe640 1 -- 172.21.15.98:0/4039277287 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5737+0+0 (secure 0 0 0) 0x7f80a811dff0 con 0x7f80b414aed0 2024-09-19T08:47:17.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.502+0000 7f80b2d9d640 1 --2- 172.21.15.98:0/4039277287 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f80a0068250 0x7f80a006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:17.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.502+0000 7f80b2d9d640 1 --2- 172.21.15.98:0/4039277287 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f80a0068250 0x7f80a006a710 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f80a40016e0 tx=0x7f80a4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:17.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.506+0000 7f809b7fe640 1 -- 172.21.15.98:0/4039277287 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f80a8016780 con 0x7f80b414aed0 2024-09-19T08:47:17.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.706+0000 7f80b90b2640 1 -- 172.21.15.98:0/4039277287 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80b4153960 con 0x7f80b414aed0 2024-09-19T08:47:17.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.706+0000 7f809b7fe640 1 -- 172.21.15.98:0/4039277287 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5235 (secure 0 0 0) 0x7f80b4153960 con 0x7f80b414aed0 2024-09-19T08:47:17.705 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:47:17.705 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":100,"btime":"2024-09-19T08:47:17:337562+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6392,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/1027012777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":1027012777},{"type":"v1","addr":"172.21.15.139:6834","nonce":1027012777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7339,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3216462399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6833","nonce":3216462399},{"type":"v1","addr":"172.21.15.139:6835","nonce":3216462399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":9174,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1629579852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1629579852},{"type":"v1","addr":"172.21.15.98:6833","nonce":1629579852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:47:13.743399+0000","modified":"2024-09-19T08:47:16.695205+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9177":{"gid":9177,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/3407902663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3407902663},{"type":"v1","addr":"172.21.15.98:6835","nonce":3407902663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9177,"qdb_cluster":[9177]},"id":13}]} 2024-09-19T08:47:17.705 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 100 2024-09-19T08:47:17.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.706+0000 7f80b90b2640 1 -- 172.21.15.98:0/4039277287 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f80a0068250 msgr2=0x7f80a006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:17.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.706+0000 7f80b90b2640 1 --2- 172.21.15.98:0/4039277287 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f80a0068250 0x7f80a006a710 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f80a40016e0 tx=0x7f80a4009040 comp rx=0 tx=0).stop 2024-09-19T08:47:17.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.706+0000 7f80b90b2640 1 -- 172.21.15.98:0/4039277287 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80b414aed0 msgr2=0x7f80b4158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:17.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.706+0000 7f80b90b2640 1 --2- 172.21.15.98:0/4039277287 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80b414aed0 0x7f80b4158fc0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f80a801c8a0 tx=0x7f809c005840 comp rx=0 tx=0).stop 2024-09-19T08:47:17.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.706+0000 7f80b90b2640 1 -- 172.21.15.98:0/4039277287 shutdown_connections 2024-09-19T08:47:17.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.706+0000 7f80b90b2640 1 --2- 172.21.15.98:0/4039277287 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f80a0068250 0x7f80a006a710 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:17.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.706+0000 7f80b90b2640 1 --2- 172.21.15.98:0/4039277287 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f80b4152fc0 0x7f80b4153420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:17.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.706+0000 7f80b90b2640 1 --2- 172.21.15.98:0/4039277287 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f80b414aed0 0x7f80b4158fc0 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:17.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.706+0000 7f80b90b2640 1 --2- 172.21.15.98:0/4039277287 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f809c003f70 0x7f80b4155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:17.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.706+0000 7f80b90b2640 1 -- 172.21.15.98:0/4039277287 >> 172.21.15.98:0/4039277287 conn(0x7f80b4074c70 msgr2=0x7f80b40d29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:17.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.706+0000 7f80b90b2640 1 -- 172.21.15.98:0/4039277287 shutdown_connections 2024-09-19T08:47:17.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.706+0000 7f80b90b2640 1 -- 172.21.15.98:0/4039277287 wait complete. 2024-09-19T08:47:17.724 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:47:17.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.838+0000 7f858562b640 1 Processor -- start 2024-09-19T08:47:17.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.838+0000 7f858562b640 1 -- start start 2024-09-19T08:47:17.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.838+0000 7f858562b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f858006efa0 0x7f85800ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:17.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.838+0000 7f858562b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f858006f380 0x7f85800cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:17.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.838+0000 7f858562b640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85800af4a0 con 0x7f85800ae8c0 2024-09-19T08:47:17.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.838+0000 7f858562b640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f858015ed00 con 0x7f858006f380 2024-09-19T08:47:17.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.838+0000 7f858562b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f85800af780 con 0x7f858006efa0 2024-09-19T08:47:17.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.838+0000 7f8577fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f858006f380 0x7f85800cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:17.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.838+0000 7f857f7fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f85800ae8c0 0x7f858015b5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:45970/0 (socket says 172.21.15.98:45970) 2024-09-19T08:47:17.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.838+0000 7f857f7fe640 1 -- 172.21.15.98:0/3068970980 learned_addr learned my addr 172.21.15.98:0/3068970980 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:17.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.838+0000 7f857e7fc640 1 -- 172.21.15.98:0/3068970980 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3233449161 0 0) 0x7f85800af4a0 con 0x7f85800ae8c0 2024-09-19T08:47:17.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.838+0000 7f857effd640 1 --2- 172.21.15.98:0/3068970980 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f858006efa0 0x7f85800ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:17.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.838+0000 7f857e7fc640 1 -- 172.21.15.98:0/3068970980 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f856c003530 con 0x7f85800ae8c0 2024-09-19T08:47:17.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.838+0000 7f8577fff640 1 -- 172.21.15.98:0/3068970980 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f858006efa0 msgr2=0x7f85800ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:17.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.838+0000 7f8577fff640 1 --2- 172.21.15.98:0/3068970980 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f858006efa0 0x7f85800ce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:17.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.838+0000 7f8577fff640 1 -- 172.21.15.98:0/3068970980 >> v1:172.21.15.98:6789/0 conn(0x7f85800ae8c0 legacy=0x7f858015b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:17.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.838+0000 7f8577fff640 1 -- 172.21.15.98:0/3068970980 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85800afa60 con 0x7f858006f380 2024-09-19T08:47:17.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.842+0000 7f8577fff640 1 --2- 172.21.15.98:0/3068970980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f858006f380 0x7f85800cea90 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f85680097c0 tx=0x7f856801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecbaa052ee0095f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:17.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.842+0000 7f857e7fc640 1 -- 172.21.15.98:0/3068970980 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85680b0070 con 0x7f858006f380 2024-09-19T08:47:17.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.842+0000 7f857e7fc640 1 -- 172.21.15.98:0/3068970980 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85680ab600 con 0x7f858006f380 2024-09-19T08:47:17.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.842+0000 7f857e7fc640 1 -- 172.21.15.98:0/3068970980 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85680ab9d0 con 0x7f858006f380 2024-09-19T08:47:17.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.842+0000 7f858562b640 1 -- 172.21.15.98:0/3068970980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f858006f380 msgr2=0x7f85800cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:17.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.842+0000 7f858562b640 1 --2- 172.21.15.98:0/3068970980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f858006f380 0x7f85800cea90 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f85680097c0 tx=0x7f856801c410 comp rx=0 tx=0).stop 2024-09-19T08:47:17.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.842+0000 7f858562b640 1 -- 172.21.15.98:0/3068970980 shutdown_connections 2024-09-19T08:47:17.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.842+0000 7f858562b640 1 --2- 172.21.15.98:0/3068970980 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f858006f380 0x7f85800cea90 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:17.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.842+0000 7f858562b640 1 --2- 172.21.15.98:0/3068970980 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f858006efa0 0x7f85800ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:17.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.842+0000 7f858562b640 1 -- 172.21.15.98:0/3068970980 >> 172.21.15.98:0/3068970980 conn(0x7f85800d0980 msgr2=0x7f85800d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:17.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.842+0000 7f858562b640 1 -- 172.21.15.98:0/3068970980 shutdown_connections 2024-09-19T08:47:17.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.842+0000 7f858562b640 1 -- 172.21.15.98:0/3068970980 wait complete. 2024-09-19T08:47:17.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.842+0000 7f858562b640 1 Processor -- start 2024-09-19T08:47:17.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.842+0000 7f858562b640 1 -- start start 2024-09-19T08:47:17.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.842+0000 7f858562b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f858006efa0 0x7f85801fb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:17.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.842+0000 7f858562b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85800ae8c0 0x7f8580155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:17.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.842+0000 7f858562b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85801f8780 0x7f8580161090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:17.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.842+0000 7f858562b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f858006d980 con 0x7f858006efa0 2024-09-19T08:47:17.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.842+0000 7f858562b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f858006d800 con 0x7f85801f8780 2024-09-19T08:47:17.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.842+0000 7f858562b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f858006db00 con 0x7f85800ae8c0 2024-09-19T08:47:17.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.842+0000 7f857effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f858006efa0 0x7f85801fb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:17.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.842+0000 7f857effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f858006efa0 0x7f85801fb920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41814/0 (socket says 172.21.15.98:41814) 2024-09-19T08:47:17.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.842+0000 7f857effd640 1 -- 172.21.15.98:0/4044336443 learned_addr learned my addr 172.21.15.98:0/4044336443 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:17.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.842+0000 7f857effd640 1 -- 172.21.15.98:0/4044336443 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85800ae8c0 msgr2=0x7f8580155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:17.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.842+0000 7f857effd640 1 --2- 172.21.15.98:0/4044336443 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85800ae8c0 0x7f8580155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:17.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.842+0000 7f857effd640 1 -- 172.21.15.98:0/4044336443 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85801f8780 msgr2=0x7f8580161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:17.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.842+0000 7f857effd640 1 --2- 172.21.15.98:0/4044336443 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85801f8780 0x7f8580161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:17.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.842+0000 7f857effd640 1 -- 172.21.15.98:0/4044336443 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8580153cc0 con 0x7f858006efa0 2024-09-19T08:47:17.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.842+0000 7f857effd640 1 --2- 172.21.15.98:0/4044336443 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f858006efa0 0x7f85801fb920 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f857000d800 tx=0x7f857000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:17.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.842+0000 7f85777fe640 1 -- 172.21.15.98:0/4044336443 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857000b6c0 con 0x7f858006efa0 2024-09-19T08:47:17.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.842+0000 7f85777fe640 1 -- 172.21.15.98:0/4044336443 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8570019070 con 0x7f858006efa0 2024-09-19T08:47:17.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.842+0000 7f858562b640 1 -- 172.21.15.98:0/4044336443 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8580153300 con 0x7f858006efa0 2024-09-19T08:47:17.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.842+0000 7f85777fe640 1 -- 172.21.15.98:0/4044336443 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857000ba60 con 0x7f858006efa0 2024-09-19T08:47:17.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.842+0000 7f858562b640 1 -- 172.21.15.98:0/4044336443 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85801f7f80 con 0x7f858006efa0 2024-09-19T08:47:17.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.846+0000 7f858562b640 1 -- 172.21.15.98:0/4044336443 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f858015b5e0 con 0x7f858006efa0 2024-09-19T08:47:17.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.846+0000 7f85777fe640 1 -- 172.21.15.98:0/4044336443 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f857000bc00 con 0x7f858006efa0 2024-09-19T08:47:17.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.846+0000 7f85777fe640 1 --2- 172.21.15.98:0/4044336443 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8560068280 0x7f856006a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:17.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.846+0000 7f85777fe640 1 -- 172.21.15.98:0/4044336443 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5737+0+0 (secure 0 0 0) 0x7f8570084890 con 0x7f858006efa0 2024-09-19T08:47:17.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.846+0000 7f8577fff640 1 --2- 172.21.15.98:0/4044336443 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8560068280 0x7f856006a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:17.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.846+0000 7f8577fff640 1 --2- 172.21.15.98:0/4044336443 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8560068280 0x7f856006a740 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f85680ad000 tx=0x7f8568004740 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:17.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:17.850+0000 7f85777fe640 1 -- 172.21.15.98:0/4044336443 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f8580153cc0 con 0x7f858006efa0 2024-09-19T08:47:18.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.010+0000 7f858562b640 1 -- 172.21.15.98:0/4044336443 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85800cfde0 con 0x7f858006efa0 2024-09-19T08:47:18.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.014+0000 7f85777fe640 1 -- 172.21.15.98:0/4044336443 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) ==== 74+0+16787 (secure 0 0 0) 0x7f8570056fd0 con 0x7f858006efa0 2024-09-19T08:47:18.014 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:47:18.014 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":170,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:47:15.308260+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:47:12.300410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-19T08:47:12.721062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:47:18.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.018+0000 7f858562b640 1 -- 172.21.15.98:0/4044336443 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8560068280 msgr2=0x7f856006a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:18.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.018+0000 7f858562b640 1 --2- 172.21.15.98:0/4044336443 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8560068280 0x7f856006a740 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f85680ad000 tx=0x7f8568004740 comp rx=0 tx=0).stop 2024-09-19T08:47:18.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.018+0000 7f858562b640 1 -- 172.21.15.98:0/4044336443 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f858006efa0 msgr2=0x7f85801fb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:18.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.018+0000 7f858562b640 1 --2- 172.21.15.98:0/4044336443 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f858006efa0 0x7f85801fb920 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f857000d800 tx=0x7f857000dcc0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.018+0000 7f858562b640 1 -- 172.21.15.98:0/4044336443 shutdown_connections 2024-09-19T08:47:18.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.018+0000 7f858562b640 1 --2- 172.21.15.98:0/4044336443 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8560068280 0x7f856006a740 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.018+0000 7f858562b640 1 --2- 172.21.15.98:0/4044336443 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f85801f8780 0x7f8580161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.018+0000 7f858562b640 1 --2- 172.21.15.98:0/4044336443 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f85800ae8c0 0x7f8580155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.018+0000 7f858562b640 1 --2- 172.21.15.98:0/4044336443 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f858006efa0 0x7f85801fb920 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.018+0000 7f858562b640 1 -- 172.21.15.98:0/4044336443 >> 172.21.15.98:0/4044336443 conn(0x7f85800d0980 msgr2=0x7f858015df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:18.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.018+0000 7f858562b640 1 -- 172.21.15.98:0/4044336443 shutdown_connections 2024-09-19T08:47:18.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.018+0000 7f858562b640 1 -- 172.21.15.98:0/4044336443 wait complete. 2024-09-19T08:47:18.032 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T08:47:18.032 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:47:18.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.138+0000 7ff241c98640 1 Processor -- start 2024-09-19T08:47:18.136 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.138+0000 7ff241c98640 1 -- start start 2024-09-19T08:47:18.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.138+0000 7ff241c98640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff23c15cc10 0x7ff23c071520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:18.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.138+0000 7ff241c98640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff23c163d90 0x7ff23c166180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:18.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.138+0000 7ff241c98640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff23c06ff30 con 0x7ff23c0d01b0 2024-09-19T08:47:18.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.138+0000 7ff241c98640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7ff23c15d3a0 con 0x7ff23c163d90 2024-09-19T08:47:18.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.138+0000 7ff241c98640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff23c073080 con 0x7ff23c15cc10 2024-09-19T08:47:18.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.138+0000 7ff23bfff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff23c163d90 0x7ff23c166180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:18.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.138+0000 7ff23bfff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff23c163d90 0x7ff23c166180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41828/0 (socket says 172.21.15.98:41828) 2024-09-19T08:47:18.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.138+0000 7ff23bfff640 1 -- 172.21.15.98:0/1250167500 learned_addr learned my addr 172.21.15.98:0/1250167500 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:18.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.138+0000 7ff23b7fe640 1 --2- 172.21.15.98:0/1250167500 >> v2:172.21.15.139:3300/0 conn(0x7ff23c15cc10 0x7ff23c071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:18.138 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.138+0000 7ff23bfff640 1 -- 172.21.15.98:0/1250167500 >> v2:172.21.15.139:3300/0 conn(0x7ff23c15cc10 msgr2=0x7ff23c071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:18.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.138+0000 7ff23bfff640 1 --2- 172.21.15.98:0/1250167500 >> v2:172.21.15.139:3300/0 conn(0x7ff23c15cc10 0x7ff23c071520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.138+0000 7ff23bfff640 1 -- 172.21.15.98:0/1250167500 >> v1:172.21.15.139:6789/0 conn(0x7ff23c0d01b0 legacy=0x7ff23c071a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:18.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.138+0000 7ff23bfff640 1 -- 172.21.15.98:0/1250167500 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff23c073360 con 0x7ff23c163d90 2024-09-19T08:47:18.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.138+0000 7ff23b7fe640 1 --2- 172.21.15.98:0/1250167500 >> v2:172.21.15.139:3300/0 conn(0x7ff23c15cc10 0x7ff23c071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:18.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.142+0000 7ff23bfff640 1 --2- 172.21.15.98:0/1250167500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff23c163d90 0x7ff23c166180 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7ff23c0cd2f0 tx=0x7ff23401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=957cf3fa26fb217 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:18.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.142+0000 7ff23a7fc640 1 -- 172.21.15.98:0/1250167500 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2340ab640 con 0x7ff23c163d90 2024-09-19T08:47:18.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.142+0000 7ff23a7fc640 1 -- 172.21.15.98:0/1250167500 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2340b0070 con 0x7ff23c163d90 2024-09-19T08:47:18.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.142+0000 7ff23a7fc640 1 -- 172.21.15.98:0/1250167500 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2340aba00 con 0x7ff23c163d90 2024-09-19T08:47:18.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.142+0000 7ff241c98640 1 -- 172.21.15.98:0/1250167500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff23c163d90 msgr2=0x7ff23c166180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:18.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.142+0000 7ff241c98640 1 --2- 172.21.15.98:0/1250167500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff23c163d90 0x7ff23c166180 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7ff23c0cd2f0 tx=0x7ff23401c4b0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.142+0000 7ff241c98640 1 -- 172.21.15.98:0/1250167500 shutdown_connections 2024-09-19T08:47:18.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.142+0000 7ff241c98640 1 --2- 172.21.15.98:0/1250167500 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff23c163d90 0x7ff23c166180 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.142+0000 7ff241c98640 1 --2- 172.21.15.98:0/1250167500 >> v2:172.21.15.139:3300/0 conn(0x7ff23c15cc10 0x7ff23c071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.142+0000 7ff241c98640 1 -- 172.21.15.98:0/1250167500 >> 172.21.15.98:0/1250167500 conn(0x7ff23c0d0980 msgr2=0x7ff23c0d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:18.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.142+0000 7ff241c98640 1 -- 172.21.15.98:0/1250167500 shutdown_connections 2024-09-19T08:47:18.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.142+0000 7ff241c98640 1 -- 172.21.15.98:0/1250167500 wait complete. 2024-09-19T08:47:18.141 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.142+0000 7ff241c98640 1 Processor -- start 2024-09-19T08:47:18.142 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.142+0000 7ff241c98640 1 -- start start 2024-09-19T08:47:18.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.142+0000 7ff241c98640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff23c0d01b0 0x7ff23c152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:18.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.142+0000 7ff241c98640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff23c15cc10 0x7ff23c071950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:18.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.142+0000 7ff241c98640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff23c163d90 0x7ff23c1647d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:18.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.142+0000 7ff241c98640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c15cff0 con 0x7ff23c163d90 2024-09-19T08:47:18.143 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.142+0000 7ff241c98640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c0ce550 con 0x7ff23c0d01b0 2024-09-19T08:47:18.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.142+0000 7ff241c98640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff23c15c060 con 0x7ff23c15cc10 2024-09-19T08:47:18.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.142+0000 7ff23bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff23c163d90 0x7ff23c1647d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:18.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.142+0000 7ff23bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff23c163d90 0x7ff23c1647d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41836/0 (socket says 172.21.15.98:41836) 2024-09-19T08:47:18.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.142+0000 7ff23bfff640 1 -- 172.21.15.98:0/1840892973 learned_addr learned my addr 172.21.15.98:0/1840892973 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:18.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.142+0000 7ff23b7fe640 1 --2- 172.21.15.98:0/1840892973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff23c0d01b0 0x7ff23c152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:18.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.142+0000 7ff23affd640 1 --2- 172.21.15.98:0/1840892973 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff23c15cc10 0x7ff23c071950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:18.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.142+0000 7ff23bfff640 1 -- 172.21.15.98:0/1840892973 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff23c15cc10 msgr2=0x7ff23c071950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:18.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.142+0000 7ff23bfff640 1 --2- 172.21.15.98:0/1840892973 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff23c15cc10 0x7ff23c071950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.142+0000 7ff23bfff640 1 -- 172.21.15.98:0/1840892973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff23c0d01b0 msgr2=0x7ff23c152d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:18.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.142+0000 7ff23bfff640 1 --2- 172.21.15.98:0/1840892973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff23c0d01b0 0x7ff23c152d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.142+0000 7ff23bfff640 1 -- 172.21.15.98:0/1840892973 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff23c15b450 con 0x7ff23c163d90 2024-09-19T08:47:18.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.142+0000 7ff23b7fe640 1 --2- 172.21.15.98:0/1840892973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff23c0d01b0 0x7ff23c152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:18.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.142+0000 7ff23bfff640 1 --2- 172.21.15.98:0/1840892973 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff23c163d90 0x7ff23c1647d0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7ff23401a0e0 tx=0x7ff23401acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:18.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.142+0000 7ff233fff640 1 -- 172.21.15.98:0/1840892973 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2340be3f0 con 0x7ff23c163d90 2024-09-19T08:47:18.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.142+0000 7ff233fff640 1 -- 172.21.15.98:0/1840892973 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2340be590 con 0x7ff23c163d90 2024-09-19T08:47:18.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.142+0000 7ff233fff640 1 -- 172.21.15.98:0/1840892973 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2340be980 con 0x7ff23c163d90 2024-09-19T08:47:18.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.146+0000 7ff241c98640 1 -- 172.21.15.98:0/1840892973 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff23c15b730 con 0x7ff23c163d90 2024-09-19T08:47:18.145 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.146+0000 7ff241c98640 1 -- 172.21.15.98:0/1840892973 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff23c15bcf0 con 0x7ff23c163d90 2024-09-19T08:47:18.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.146+0000 7ff241c98640 1 -- 172.21.15.98:0/1840892973 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff23c071520 con 0x7ff23c163d90 2024-09-19T08:47:18.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.146+0000 7ff233fff640 1 -- 172.21.15.98:0/1840892973 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7ff2340b43f0 con 0x7ff23c163d90 2024-09-19T08:47:18.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.146+0000 7ff233fff640 1 --2- 172.21.15.98:0/1840892973 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff21c068280 0x7ff21c06a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:18.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.146+0000 7ff233fff640 1 -- 172.21.15.98:0/1840892973 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5737+0+0 (secure 0 0 0) 0x7ff23411e660 con 0x7ff23c163d90 2024-09-19T08:47:18.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.146+0000 7ff23b7fe640 1 --2- 172.21.15.98:0/1840892973 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff21c068280 0x7ff21c06a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:18.146 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.146+0000 7ff23b7fe640 1 --2- 172.21.15.98:0/1840892973 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff21c068280 0x7ff21c06a740 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00d430 tx=0x7ff22c007530 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:18.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.150+0000 7ff233fff640 1 -- 172.21.15.98:0/1840892973 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff234124030 con 0x7ff23c163d90 2024-09-19T08:47:18.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.294+0000 7ff241c98640 1 -- 172.21.15.98:0/1840892973 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff23c1561e0 con 0x7ff21c068280 2024-09-19T08:47:18.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.314+0000 7ff233fff640 1 -- 172.21.15.98:0/1840892973 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247089 (secure 0 0 0) 0x7ff23c1561e0 con 0x7ff21c068280 2024-09-19T08:47:18.315 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:47:18.317 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:47:18.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.318+0000 7ff241c98640 1 -- 172.21.15.98:0/1840892973 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff21c068280 msgr2=0x7ff21c06a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:18.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.318+0000 7ff241c98640 1 --2- 172.21.15.98:0/1840892973 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff21c068280 0x7ff21c06a740 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00d430 tx=0x7ff22c007530 comp rx=0 tx=0).stop 2024-09-19T08:47:18.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.318+0000 7ff241c98640 1 -- 172.21.15.98:0/1840892973 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff23c163d90 msgr2=0x7ff23c1647d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:18.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.318+0000 7ff241c98640 1 --2- 172.21.15.98:0/1840892973 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff23c163d90 0x7ff23c1647d0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7ff23401a0e0 tx=0x7ff23401acf0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.318+0000 7ff241c98640 1 -- 172.21.15.98:0/1840892973 shutdown_connections 2024-09-19T08:47:18.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.318+0000 7ff241c98640 1 --2- 172.21.15.98:0/1840892973 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff21c068280 0x7ff21c06a740 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.318+0000 7ff241c98640 1 --2- 172.21.15.98:0/1840892973 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff23c163d90 0x7ff23c1647d0 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.318+0000 7ff241c98640 1 --2- 172.21.15.98:0/1840892973 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff23c15cc10 0x7ff23c071950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.318+0000 7ff241c98640 1 --2- 172.21.15.98:0/1840892973 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff23c0d01b0 0x7ff23c152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.318+0000 7ff241c98640 1 -- 172.21.15.98:0/1840892973 >> 172.21.15.98:0/1840892973 conn(0x7ff23c0d0980 msgr2=0x7ff23c15ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:18.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.318+0000 7ff241c98640 1 -- 172.21.15.98:0/1840892973 shutdown_connections 2024-09-19T08:47:18.319 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.318+0000 7ff241c98640 1 -- 172.21.15.98:0/1840892973 wait complete. 2024-09-19T08:47:18.332 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2024-09-19T08:47:16.958217+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":275500,"kb_used_data":17636,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754699220,"statfs":{"total":773094113280,"available":772812001280,"internally_reserved":0,"allocated":18059264,"data_stored":8663595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.599406"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.375621+0000","last_change":"2024-09-19T08:47:13.375621+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.375621+0000","last_undegraded":"2024-09-19T08:47:13.375621+0000","last_fullsized":"2024-09-19T08:47:13.375621+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.393844+0000","last_change":"2024-09-19T08:47:13.393844+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.393844+0000","last_undegraded":"2024-09-19T08:47:13.393844+0000","last_fullsized":"2024-09-19T08:47:13.393844+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.380106+0000","last_change":"2024-09-19T08:47:13.380106+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.380106+0000","last_undegraded":"2024-09-19T08:47:13.380106+0000","last_fullsized":"2024-09-19T08:47:13.380106+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.383331+0000","last_change":"2024-09-19T08:47:13.383331+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.383331+0000","last_undegraded":"2024-09-19T08:47:13.383331+0000","last_fullsized":"2024-09-19T08:47:13.383331+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.374122+0000","last_change":"2024-09-19T08:47:13.374122+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.374122+0000","last_undegraded":"2024-09-19T08:47:13.374122+0000","last_fullsized":"2024-09-19T08:47:13.374122+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.374054+0000","last_change":"2024-09-19T08:47:13.374054+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.374054+0000","last_undegraded":"2024-09-19T08:47:13.374054+0000","last_fullsized":"2024-09-19T08:47:13.374054+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.377663+0000","last_change":"2024-09-19T08:47:13.377663+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.377663+0000","last_undegraded":"2024-09-19T08:47:13.377663+0000","last_fullsized":"2024-09-19T08:47:13.377663+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.373860+0000","last_change":"2024-09-19T08:47:13.373860+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.373860+0000","last_undegraded":"2024-09-19T08:47:13.373860+0000","last_fullsized":"2024-09-19T08:47:13.373860+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.392064+0000","last_change":"2024-09-19T08:47:13.392064+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.392064+0000","last_undegraded":"2024-09-19T08:47:13.392064+0000","last_fullsized":"2024-09-19T08:47:13.392064+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.374995+0000","last_change":"2024-09-19T08:47:13.374995+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.374995+0000","last_undegraded":"2024-09-19T08:47:13.374995+0000","last_fullsized":"2024-09-19T08:47:13.374995+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.375676+0000","last_change":"2024-09-19T08:47:13.375676+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.375676+0000","last_undegraded":"2024-09-19T08:47:13.375676+0000","last_fullsized":"2024-09-19T08:47:13.375676+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.373586+0000","last_change":"2024-09-19T08:47:13.373586+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.373586+0000","last_undegraded":"2024-09-19T08:47:13.373586+0000","last_fullsized":"2024-09-19T08:47:13.373586+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.373583+0000","last_change":"2024-09-19T08:47:13.373583+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.373583+0000","last_undegraded":"2024-09-19T08:47:13.373583+0000","last_fullsized":"2024-09-19T08:47:13.373583+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.377730+0000","last_change":"2024-09-19T08:47:13.377730+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.377730+0000","last_undegraded":"2024-09-19T08:47:13.377730+0000","last_fullsized":"2024-09-19T08:47:13.377730+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.376178+0000","last_change":"2024-09-19T08:47:13.376178+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.376178+0000","last_undegraded":"2024-09-19T08:47:13.376178+0000","last_fullsized":"2024-09-19T08:47:13.376178+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.375416+0000","last_change":"2024-09-19T08:47:13.375416+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.375416+0000","last_undegraded":"2024-09-19T08:47:13.375416+0000","last_fullsized":"2024-09-19T08:47:13.375416+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.375076+0000","last_change":"2024-09-19T08:47:13.375076+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.375076+0000","last_undegraded":"2024-09-19T08:47:13.375076+0000","last_fullsized":"2024-09-19T08:47:13.375076+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.374529+0000","last_change":"2024-09-19T08:47:13.374529+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.374529+0000","last_undegraded":"2024-09-19T08:47:13.374529+0000","last_fullsized":"2024-09-19T08:47:13.374529+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.375270+0000","last_change":"2024-09-19T08:47:13.375270+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.375270+0000","last_undegraded":"2024-09-19T08:47:13.375270+0000","last_fullsized":"2024-09-19T08:47:13.375270+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.373724+0000","last_change":"2024-09-19T08:47:13.373724+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.373724+0000","last_undegraded":"2024-09-19T08:47:13.373724+0000","last_fullsized":"2024-09-19T08:47:13.373724+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.384490+0000","last_change":"2024-09-19T08:47:13.384490+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.384490+0000","last_undegraded":"2024-09-19T08:47:13.384490+0000","last_fullsized":"2024-09-19T08:47:13.384490+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.388649+0000","last_change":"2024-09-19T08:47:13.388649+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.388649+0000","last_undegraded":"2024-09-19T08:47:13.388649+0000","last_fullsized":"2024-09-19T08:47:13.388649+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.386532+0000","last_change":"2024-09-19T08:47:13.386532+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.386532+0000","last_undegraded":"2024-09-19T08:47:13.386532+0000","last_fullsized":"2024-09-19T08:47:13.386532+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.381686+0000","last_change":"2024-09-19T08:47:13.381686+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.381686+0000","last_undegraded":"2024-09-19T08:47:13.381686+0000","last_fullsized":"2024-09-19T08:47:13.381686+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.397263+0000","last_change":"2024-09-19T08:47:13.397263+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.397263+0000","last_undegraded":"2024-09-19T08:47:13.397263+0000","last_fullsized":"2024-09-19T08:47:13.397263+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.376365+0000","last_change":"2024-09-19T08:47:13.376365+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.376365+0000","last_undegraded":"2024-09-19T08:47:13.376365+0000","last_fullsized":"2024-09-19T08:47:13.376365+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.374470+0000","last_change":"2024-09-19T08:47:13.374470+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.374470+0000","last_undegraded":"2024-09-19T08:47:13.374470+0000","last_fullsized":"2024-09-19T08:47:13.374470+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.375256+0000","last_change":"2024-09-19T08:47:13.375256+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.375256+0000","last_undegraded":"2024-09-19T08:47:13.375256+0000","last_fullsized":"2024-09-19T08:47:13.375256+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.376057+0000","last_change":"2024-09-19T08:47:13.376057+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.376057+0000","last_undegraded":"2024-09-19T08:47:13.376057+0000","last_fullsized":"2024-09-19T08:47:13.376057+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.381136+0000","last_change":"2024-09-19T08:47:13.381136+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.381136+0000","last_undegraded":"2024-09-19T08:47:13.381136+0000","last_fullsized":"2024-09-19T08:47:13.381136+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.376688+0000","last_change":"2024-09-19T08:47:13.376688+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.376688+0000","last_undegraded":"2024-09-19T08:47:13.376688+0000","last_fullsized":"2024-09-19T08:47:13.376688+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.376109+0000","last_change":"2024-09-19T08:47:13.376109+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.376109+0000","last_undegraded":"2024-09-19T08:47:13.376109+0000","last_fullsized":"2024-09-19T08:47:13.376109+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.391812+0000","last_change":"2024-09-19T08:47:13.391812+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.391812+0000","last_undegraded":"2024-09-19T08:47:13.391812+0000","last_fullsized":"2024-09-19T08:47:13.391812+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.404594+0000","last_change":"2024-09-19T08:47:13.401720+0000","last_active":"2024-09-19T08:47:13.404594+0000","last_peered":"2024-09-19T08:47:13.404594+0000","last_clean":"2024-09-19T08:47:13.404594+0000","last_became_active":"2024-09-19T08:47:13.400995+0000","last_became_peered":"2024-09-19T08:47:13.400995+0000","last_unstale":"2024-09-19T08:47:13.404594+0000","last_undegraded":"2024-09-19T08:47:13.404594+0000","last_fullsized":"2024-09-19T08:47:13.404594+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:31:30.868216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.389838+0000","last_change":"2024-09-19T08:47:13.389838+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.389838+0000","last_undegraded":"2024-09-19T08:47:13.389838+0000","last_fullsized":"2024-09-19T08:47:13.389838+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.412531+0000","last_change":"2024-09-19T08:47:13.411165+0000","last_active":"2024-09-19T08:47:13.412531+0000","last_peered":"2024-09-19T08:47:13.412531+0000","last_clean":"2024-09-19T08:47:13.412531+0000","last_became_active":"2024-09-19T08:47:13.410431+0000","last_became_peered":"2024-09-19T08:47:13.410431+0000","last_unstale":"2024-09-19T08:47:13.412531+0000","last_undegraded":"2024-09-19T08:47:13.412531+0000","last_fullsized":"2024-09-19T08:47:13.412531+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:17.138674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.373947+0000","last_change":"2024-09-19T08:47:13.373947+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.373947+0000","last_undegraded":"2024-09-19T08:47:13.373947+0000","last_fullsized":"2024-09-19T08:47:13.373947+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.397842+0000","last_change":"2024-09-19T08:47:13.393085+0000","last_active":"2024-09-19T08:47:13.397842+0000","last_peered":"2024-09-19T08:47:13.397842+0000","last_clean":"2024-09-19T08:47:13.397842+0000","last_became_active":"2024-09-19T08:47:13.392776+0000","last_became_peered":"2024-09-19T08:47:13.392776+0000","last_unstale":"2024-09-19T08:47:13.397842+0000","last_undegraded":"2024-09-19T08:47:13.397842+0000","last_fullsized":"2024-09-19T08:47:13.397842+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:48.525656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.379453+0000","last_change":"2024-09-19T08:47:13.379453+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.379453+0000","last_undegraded":"2024-09-19T08:47:13.379453+0000","last_fullsized":"2024-09-19T08:47:13.379453+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.410308+0000","last_change":"2024-09-19T08:47:13.403896+0000","last_active":"2024-09-19T08:47:13.410308+0000","last_peered":"2024-09-19T08:47:13.410308+0000","last_clean":"2024-09-19T08:47:13.410308+0000","last_became_active":"2024-09-19T08:47:13.403168+0000","last_became_peered":"2024-09-19T08:47:13.403168+0000","last_unstale":"2024-09-19T08:47:13.410308+0000","last_undegraded":"2024-09-19T08:47:13.410308+0000","last_fullsized":"2024-09-19T08:47:13.410308+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:41.705016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.383435+0000","last_change":"2024-09-19T08:47:13.383435+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.383435+0000","last_undegraded":"2024-09-19T08:47:13.383435+0000","last_fullsized":"2024-09-19T08:47:13.383435+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.401038+0000","last_change":"2024-09-19T08:47:13.400567+0000","last_active":"2024-09-19T08:47:13.401038+0000","last_peered":"2024-09-19T08:47:13.401038+0000","last_clean":"2024-09-19T08:47:13.401038+0000","last_became_active":"2024-09-19T08:47:13.400297+0000","last_became_peered":"2024-09-19T08:47:13.400297+0000","last_unstale":"2024-09-19T08:47:13.401038+0000","last_undegraded":"2024-09-19T08:47:13.401038+0000","last_fullsized":"2024-09-19T08:47:13.401038+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:52.284449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.372347+0000","last_change":"2024-09-19T08:47:13.372347+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.372347+0000","last_undegraded":"2024-09-19T08:47:13.372347+0000","last_fullsized":"2024-09-19T08:47:13.372347+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.386785+0000","last_change":"2024-09-19T08:47:13.382448+0000","last_active":"2024-09-19T08:47:13.386785+0000","last_peered":"2024-09-19T08:47:13.386785+0000","last_clean":"2024-09-19T08:47:13.386785+0000","last_became_active":"2024-09-19T08:47:13.382136+0000","last_became_peered":"2024-09-19T08:47:13.382136+0000","last_unstale":"2024-09-19T08:47:13.386785+0000","last_undegraded":"2024-09-19T08:47:13.386785+0000","last_fullsized":"2024-09-19T08:47:13.386785+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:47:16.160287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.380645+0000","last_change":"2024-09-19T08:47:13.380645+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.380645+0000","last_undegraded":"2024-09-19T08:47:13.380645+0000","last_fullsized":"2024-09-19T08:47:13.380645+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.402491+0000","last_change":"2024-09-19T08:47:13.394882+0000","last_active":"2024-09-19T08:47:13.402491+0000","last_peered":"2024-09-19T08:47:13.402491+0000","last_clean":"2024-09-19T08:47:13.402491+0000","last_became_active":"2024-09-19T08:47:13.393932+0000","last_became_peered":"2024-09-19T08:47:13.393932+0000","last_unstale":"2024-09-19T08:47:13.402491+0000","last_undegraded":"2024-09-19T08:47:13.402491+0000","last_fullsized":"2024-09-19T08:47:13.402491+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:54:16.646628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.381875+0000","last_change":"2024-09-19T08:47:13.381875+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.381875+0000","last_undegraded":"2024-09-19T08:47:13.381875+0000","last_fullsized":"2024-09-19T08:47:13.381875+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.400581+0000","last_change":"2024-09-19T08:47:13.398554+0000","last_active":"2024-09-19T08:47:13.400581+0000","last_peered":"2024-09-19T08:47:13.400581+0000","last_clean":"2024-09-19T08:47:13.400581+0000","last_became_active":"2024-09-19T08:47:13.397871+0000","last_became_peered":"2024-09-19T08:47:13.397871+0000","last_unstale":"2024-09-19T08:47:13.400581+0000","last_undegraded":"2024-09-19T08:47:13.400581+0000","last_fullsized":"2024-09-19T08:47:13.400581+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:40.000754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.375234+0000","last_change":"2024-09-19T08:47:13.375234+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.375234+0000","last_undegraded":"2024-09-19T08:47:13.375234+0000","last_fullsized":"2024-09-19T08:47:13.375234+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.412003+0000","last_change":"2024-09-19T08:47:13.407203+0000","last_active":"2024-09-19T08:47:13.412003+0000","last_peered":"2024-09-19T08:47:13.412003+0000","last_clean":"2024-09-19T08:47:13.412003+0000","last_became_active":"2024-09-19T08:47:13.406419+0000","last_became_peered":"2024-09-19T08:47:13.406419+0000","last_unstale":"2024-09-19T08:47:13.412003+0000","last_undegraded":"2024-09-19T08:47:13.412003+0000","last_fullsized":"2024-09-19T08:47:13.412003+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:40.000754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.382196+0000","last_change":"2024-09-19T08:47:13.382196+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.382196+0000","last_undegraded":"2024-09-19T08:47:13.382196+0000","last_fullsized":"2024-09-19T08:47:13.382196+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.386404+0000","last_change":"2024-09-19T08:47:13.382865+0000","last_active":"2024-09-19T08:47:13.386404+0000","last_peered":"2024-09-19T08:47:13.386404+0000","last_clean":"2024-09-19T08:47:13.386404+0000","last_became_active":"2024-09-19T08:47:13.382528+0000","last_became_peered":"2024-09-19T08:47:13.382528+0000","last_unstale":"2024-09-19T08:47:13.386404+0000","last_undegraded":"2024-09-19T08:47:13.386404+0000","last_fullsized":"2024-09-19T08:47:13.386404+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:16.220719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.375815+0000","last_change":"2024-09-19T08:47:13.375815+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.375815+0000","last_undegraded":"2024-09-19T08:47:13.375815+0000","last_fullsized":"2024-09-19T08:47:13.375815+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.411608+0000","last_change":"2024-09-19T08:47:13.409867+0000","last_active":"2024-09-19T08:47:13.411608+0000","last_peered":"2024-09-19T08:47:13.411608+0000","last_clean":"2024-09-19T08:47:13.411608+0000","last_became_active":"2024-09-19T08:47:13.408810+0000","last_became_peered":"2024-09-19T08:47:13.408810+0000","last_unstale":"2024-09-19T08:47:13.411608+0000","last_undegraded":"2024-09-19T08:47:13.411608+0000","last_fullsized":"2024-09-19T08:47:13.411608+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:18:39.135827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.373137+0000","last_change":"2024-09-19T08:47:13.373137+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.373137+0000","last_undegraded":"2024-09-19T08:47:13.373137+0000","last_fullsized":"2024-09-19T08:47:13.373137+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.405795+0000","last_change":"2024-09-19T08:47:13.404129+0000","last_active":"2024-09-19T08:47:13.405795+0000","last_peered":"2024-09-19T08:47:13.405795+0000","last_clean":"2024-09-19T08:47:13.405795+0000","last_became_active":"2024-09-19T08:47:13.403371+0000","last_became_peered":"2024-09-19T08:47:13.403371+0000","last_unstale":"2024-09-19T08:47:13.405795+0000","last_undegraded":"2024-09-19T08:47:13.405795+0000","last_fullsized":"2024-09-19T08:47:13.405795+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:44:57.657282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.379376+0000","last_change":"2024-09-19T08:47:13.379376+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.379376+0000","last_undegraded":"2024-09-19T08:47:13.379376+0000","last_fullsized":"2024-09-19T08:47:13.379376+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.409306+0000","last_change":"2024-09-19T08:47:13.405188+0000","last_active":"2024-09-19T08:47:13.409306+0000","last_peered":"2024-09-19T08:47:13.409306+0000","last_clean":"2024-09-19T08:47:13.409306+0000","last_became_active":"2024-09-19T08:47:13.404512+0000","last_became_peered":"2024-09-19T08:47:13.404512+0000","last_unstale":"2024-09-19T08:47:13.409306+0000","last_undegraded":"2024-09-19T08:47:13.409306+0000","last_fullsized":"2024-09-19T08:47:13.409306+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:22:24.167219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.372698+0000","last_change":"2024-09-19T08:47:13.372698+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.372698+0000","last_undegraded":"2024-09-19T08:47:13.372698+0000","last_fullsized":"2024-09-19T08:47:13.372698+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.397985+0000","last_change":"2024-09-19T08:47:13.393731+0000","last_active":"2024-09-19T08:47:13.397985+0000","last_peered":"2024-09-19T08:47:13.397985+0000","last_clean":"2024-09-19T08:47:13.397985+0000","last_became_active":"2024-09-19T08:47:13.393445+0000","last_became_peered":"2024-09-19T08:47:13.393445+0000","last_unstale":"2024-09-19T08:47:13.397985+0000","last_undegraded":"2024-09-19T08:47:13.397985+0000","last_fullsized":"2024-09-19T08:47:13.397985+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:25:43.296666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.373004+0000","last_change":"2024-09-19T08:47:13.373004+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.373004+0000","last_undegraded":"2024-09-19T08:47:13.373004+0000","last_fullsized":"2024-09-19T08:47:13.373004+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.397746+0000","last_change":"2024-09-19T08:47:13.393111+0000","last_active":"2024-09-19T08:47:13.397746+0000","last_peered":"2024-09-19T08:47:13.397746+0000","last_clean":"2024-09-19T08:47:13.397746+0000","last_became_active":"2024-09-19T08:47:13.392754+0000","last_became_peered":"2024-09-19T08:47:13.392754+0000","last_unstale":"2024-09-19T08:47:13.397746+0000","last_undegraded":"2024-09-19T08:47:13.397746+0000","last_fullsized":"2024-09-19T08:47:13.397746+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:51.628450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.390767+0000","last_change":"2024-09-19T08:47:13.390767+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.390767+0000","last_undegraded":"2024-09-19T08:47:13.390767+0000","last_fullsized":"2024-09-19T08:47:13.390767+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.386834+0000","last_change":"2024-09-19T08:47:13.383220+0000","last_active":"2024-09-19T08:47:13.386834+0000","last_peered":"2024-09-19T08:47:13.386834+0000","last_clean":"2024-09-19T08:47:13.386834+0000","last_became_active":"2024-09-19T08:47:13.382953+0000","last_became_peered":"2024-09-19T08:47:13.382953+0000","last_unstale":"2024-09-19T08:47:13.386834+0000","last_undegraded":"2024-09-19T08:47:13.386834+0000","last_fullsized":"2024-09-19T08:47:13.386834+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:54:16.646628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.396193+0000","last_change":"2024-09-19T08:47:13.396193+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.396193+0000","last_undegraded":"2024-09-19T08:47:13.396193+0000","last_fullsized":"2024-09-19T08:47:13.396193+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.412044+0000","last_change":"2024-09-19T08:47:13.408636+0000","last_active":"2024-09-19T08:47:13.412044+0000","last_peered":"2024-09-19T08:47:13.412044+0000","last_clean":"2024-09-19T08:47:13.412044+0000","last_became_active":"2024-09-19T08:47:13.407243+0000","last_became_peered":"2024-09-19T08:47:13.407243+0000","last_unstale":"2024-09-19T08:47:13.412044+0000","last_undegraded":"2024-09-19T08:47:13.412044+0000","last_fullsized":"2024-09-19T08:47:13.412044+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:40.467795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.372029+0000","last_change":"2024-09-19T08:47:13.372029+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.372029+0000","last_undegraded":"2024-09-19T08:47:13.372029+0000","last_fullsized":"2024-09-19T08:47:13.372029+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.386432+0000","last_change":"2024-09-19T08:47:13.382929+0000","last_active":"2024-09-19T08:47:13.386432+0000","last_peered":"2024-09-19T08:47:13.386432+0000","last_clean":"2024-09-19T08:47:13.386432+0000","last_became_active":"2024-09-19T08:47:13.382644+0000","last_became_peered":"2024-09-19T08:47:13.382644+0000","last_unstale":"2024-09-19T08:47:13.386432+0000","last_undegraded":"2024-09-19T08:47:13.386432+0000","last_fullsized":"2024-09-19T08:47:13.386432+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:41.025166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.375784+0000","last_change":"2024-09-19T08:47:13.375784+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.375784+0000","last_undegraded":"2024-09-19T08:47:13.375784+0000","last_fullsized":"2024-09-19T08:47:13.375784+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.386306+0000","last_change":"2024-09-19T08:47:13.382212+0000","last_active":"2024-09-19T08:47:13.386306+0000","last_peered":"2024-09-19T08:47:13.386306+0000","last_clean":"2024-09-19T08:47:13.386306+0000","last_became_active":"2024-09-19T08:47:13.381878+0000","last_became_peered":"2024-09-19T08:47:13.381878+0000","last_unstale":"2024-09-19T08:47:13.386306+0000","last_undegraded":"2024-09-19T08:47:13.386306+0000","last_fullsized":"2024-09-19T08:47:13.386306+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:42.546192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.376790+0000","last_change":"2024-09-19T08:47:13.376790+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.376790+0000","last_undegraded":"2024-09-19T08:47:13.376790+0000","last_fullsized":"2024-09-19T08:47:13.376790+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.386944+0000","last_change":"2024-09-19T08:47:13.384584+0000","last_active":"2024-09-19T08:47:13.386944+0000","last_peered":"2024-09-19T08:47:13.386944+0000","last_clean":"2024-09-19T08:47:13.386944+0000","last_became_active":"2024-09-19T08:47:13.384215+0000","last_became_peered":"2024-09-19T08:47:13.384215+0000","last_unstale":"2024-09-19T08:47:13.386944+0000","last_undegraded":"2024-09-19T08:47:13.386944+0000","last_fullsized":"2024-09-19T08:47:13.386944+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:08.459798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.371932+0000","last_change":"2024-09-19T08:47:13.371932+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.371932+0000","last_undegraded":"2024-09-19T08:47:13.371932+0000","last_fullsized":"2024-09-19T08:47:13.371932+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.398410+0000","last_change":"2024-09-19T08:47:13.393941+0000","last_active":"2024-09-19T08:47:13.398410+0000","last_peered":"2024-09-19T08:47:13.398410+0000","last_clean":"2024-09-19T08:47:13.398410+0000","last_became_active":"2024-09-19T08:47:13.393683+0000","last_became_peered":"2024-09-19T08:47:13.393683+0000","last_unstale":"2024-09-19T08:47:13.398410+0000","last_undegraded":"2024-09-19T08:47:13.398410+0000","last_fullsized":"2024-09-19T08:47:13.398410+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:34:09.089055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.380239+0000","last_change":"2024-09-19T08:47:13.380239+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.380239+0000","last_undegraded":"2024-09-19T08:47:13.380239+0000","last_fullsized":"2024-09-19T08:47:13.380239+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.405027+0000","last_change":"2024-09-19T08:47:13.404413+0000","last_active":"2024-09-19T08:47:13.405027+0000","last_peered":"2024-09-19T08:47:13.405027+0000","last_clean":"2024-09-19T08:47:13.405027+0000","last_became_active":"2024-09-19T08:47:13.404206+0000","last_became_peered":"2024-09-19T08:47:13.404206+0000","last_unstale":"2024-09-19T08:47:13.405027+0000","last_undegraded":"2024-09-19T08:47:13.405027+0000","last_fullsized":"2024-09-19T08:47:13.405027+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:54:30.371240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.385386+0000","last_change":"2024-09-19T08:47:13.385386+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.385386+0000","last_undegraded":"2024-09-19T08:47:13.385386+0000","last_fullsized":"2024-09-19T08:47:13.385386+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.392249+0000","last_change":"2024-09-19T08:47:13.387174+0000","last_active":"2024-09-19T08:47:13.392249+0000","last_peered":"2024-09-19T08:47:13.392249+0000","last_clean":"2024-09-19T08:47:13.392249+0000","last_became_active":"2024-09-19T08:47:13.386875+0000","last_became_peered":"2024-09-19T08:47:13.386875+0000","last_unstale":"2024-09-19T08:47:13.392249+0000","last_undegraded":"2024-09-19T08:47:13.392249+0000","last_fullsized":"2024-09-19T08:47:13.392249+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:18:12.747028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.374099+0000","last_change":"2024-09-19T08:47:13.374099+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.374099+0000","last_undegraded":"2024-09-19T08:47:13.374099+0000","last_fullsized":"2024-09-19T08:47:13.374099+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.398842+0000","last_change":"2024-09-19T08:47:13.398116+0000","last_active":"2024-09-19T08:47:13.398842+0000","last_peered":"2024-09-19T08:47:13.398842+0000","last_clean":"2024-09-19T08:47:13.398842+0000","last_became_active":"2024-09-19T08:47:13.397684+0000","last_became_peered":"2024-09-19T08:47:13.397684+0000","last_unstale":"2024-09-19T08:47:13.398842+0000","last_undegraded":"2024-09-19T08:47:13.398842+0000","last_fullsized":"2024-09-19T08:47:13.398842+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:09:47.807044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.374542+0000","last_change":"2024-09-19T08:47:13.374542+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.374542+0000","last_undegraded":"2024-09-19T08:47:13.374542+0000","last_fullsized":"2024-09-19T08:47:13.374542+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.409827+0000","last_change":"2024-09-19T08:47:13.402409+0000","last_active":"2024-09-19T08:47:13.409827+0000","last_peered":"2024-09-19T08:47:13.409827+0000","last_clean":"2024-09-19T08:47:13.409827+0000","last_became_active":"2024-09-19T08:47:13.401624+0000","last_became_peered":"2024-09-19T08:47:13.401624+0000","last_unstale":"2024-09-19T08:47:13.409827+0000","last_undegraded":"2024-09-19T08:47:13.409827+0000","last_fullsized":"2024-09-19T08:47:13.409827+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:59:08.197023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.380966+0000","last_change":"2024-09-19T08:47:13.380966+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.380966+0000","last_undegraded":"2024-09-19T08:47:13.380966+0000","last_fullsized":"2024-09-19T08:47:13.380966+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.389403+0000","last_change":"2024-09-19T08:47:13.385451+0000","last_active":"2024-09-19T08:47:13.389403+0000","last_peered":"2024-09-19T08:47:13.389403+0000","last_clean":"2024-09-19T08:47:13.389403+0000","last_became_active":"2024-09-19T08:47:13.384496+0000","last_became_peered":"2024-09-19T08:47:13.384496+0000","last_unstale":"2024-09-19T08:47:13.389403+0000","last_undegraded":"2024-09-19T08:47:13.389403+0000","last_fullsized":"2024-09-19T08:47:13.389403+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:22:24.167219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.381828+0000","last_change":"2024-09-19T08:47:13.381828+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.381828+0000","last_undegraded":"2024-09-19T08:47:13.381828+0000","last_fullsized":"2024-09-19T08:47:13.381828+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.398847+0000","last_change":"2024-09-19T08:47:13.398325+0000","last_active":"2024-09-19T08:47:13.398847+0000","last_peered":"2024-09-19T08:47:13.398847+0000","last_clean":"2024-09-19T08:47:13.398847+0000","last_became_active":"2024-09-19T08:47:13.397450+0000","last_became_peered":"2024-09-19T08:47:13.397450+0000","last_unstale":"2024-09-19T08:47:13.398847+0000","last_undegraded":"2024-09-19T08:47:13.398847+0000","last_fullsized":"2024-09-19T08:47:13.398847+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:34.099661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.376127+0000","last_change":"2024-09-19T08:47:13.376127+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.376127+0000","last_undegraded":"2024-09-19T08:47:13.376127+0000","last_fullsized":"2024-09-19T08:47:13.376127+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.409439+0000","last_change":"2024-09-19T08:47:13.399625+0000","last_active":"2024-09-19T08:47:13.409439+0000","last_peered":"2024-09-19T08:47:13.409439+0000","last_clean":"2024-09-19T08:47:13.409439+0000","last_became_active":"2024-09-19T08:47:13.398838+0000","last_became_peered":"2024-09-19T08:47:13.398838+0000","last_unstale":"2024-09-19T08:47:13.409439+0000","last_undegraded":"2024-09-19T08:47:13.409439+0000","last_fullsized":"2024-09-19T08:47:13.409439+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:33:27.811970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.374929+0000","last_change":"2024-09-19T08:47:13.374929+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.374929+0000","last_undegraded":"2024-09-19T08:47:13.374929+0000","last_fullsized":"2024-09-19T08:47:13.374929+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.389369+0000","last_change":"2024-09-19T08:47:13.384922+0000","last_active":"2024-09-19T08:47:13.389369+0000","last_peered":"2024-09-19T08:47:13.389369+0000","last_clean":"2024-09-19T08:47:13.389369+0000","last_became_active":"2024-09-19T08:47:13.384555+0000","last_became_peered":"2024-09-19T08:47:13.384555+0000","last_unstale":"2024-09-19T08:47:13.389369+0000","last_undegraded":"2024-09-19T08:47:13.389369+0000","last_fullsized":"2024-09-19T08:47:13.389369+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:44:57.657282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.372134+0000","last_change":"2024-09-19T08:47:13.372134+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.372134+0000","last_undegraded":"2024-09-19T08:47:13.372134+0000","last_fullsized":"2024-09-19T08:47:13.372134+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.411931+0000","last_change":"2024-09-19T08:47:13.407182+0000","last_active":"2024-09-19T08:47:13.411931+0000","last_peered":"2024-09-19T08:47:13.411931+0000","last_clean":"2024-09-19T08:47:13.411931+0000","last_became_active":"2024-09-19T08:47:13.405978+0000","last_became_peered":"2024-09-19T08:47:13.405978+0000","last_unstale":"2024-09-19T08:47:13.411931+0000","last_undegraded":"2024-09-19T08:47:13.411931+0000","last_fullsized":"2024-09-19T08:47:13.411931+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:18:12.747028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.386744+0000","last_change":"2024-09-19T08:47:13.386744+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.386744+0000","last_undegraded":"2024-09-19T08:47:13.386744+0000","last_fullsized":"2024-09-19T08:47:13.386744+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.393663+0000","last_change":"2024-09-19T08:47:13.386481+0000","last_active":"2024-09-19T08:47:13.393663+0000","last_peered":"2024-09-19T08:47:13.393663+0000","last_clean":"2024-09-19T08:47:13.393663+0000","last_became_active":"2024-09-19T08:47:13.385716+0000","last_became_peered":"2024-09-19T08:47:13.385716+0000","last_unstale":"2024-09-19T08:47:13.393663+0000","last_undegraded":"2024-09-19T08:47:13.393663+0000","last_fullsized":"2024-09-19T08:47:13.393663+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:27.685629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.377235+0000","last_change":"2024-09-19T08:47:13.377235+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.377235+0000","last_undegraded":"2024-09-19T08:47:13.377235+0000","last_fullsized":"2024-09-19T08:47:13.377235+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.395411+0000","last_change":"2024-09-19T08:47:13.388601+0000","last_active":"2024-09-19T08:47:13.395411+0000","last_peered":"2024-09-19T08:47:13.395411+0000","last_clean":"2024-09-19T08:47:13.395411+0000","last_became_active":"2024-09-19T08:47:13.387981+0000","last_became_peered":"2024-09-19T08:47:13.387981+0000","last_unstale":"2024-09-19T08:47:13.395411+0000","last_undegraded":"2024-09-19T08:47:13.395411+0000","last_fullsized":"2024-09-19T08:47:13.395411+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:43:37.808820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"71'128","reported_seq":474,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.385151+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:47:13.385151+0000","last_peered":"2024-09-19T08:47:13.385151+0000","last_clean":"2024-09-19T08:47:13.385151+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:47:13.385151+0000","last_undegraded":"2024-09-19T08:47:13.385151+0000","last_fullsized":"2024-09-19T08:47:13.385151+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509552,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33824,"kb_used_data":2632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338016,"statfs":{"total":96636764160,"available":96602128384,"internally_reserved":0,"allocated":2695168,"data_stored":1521786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574961,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33252,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338588,"statfs":{"total":96636764160,"available":96602714112,"internally_reserved":0,"allocated":2113536,"data_stored":937721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509554,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33244,"kb_used_data":2052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338596,"statfs":{"total":96636764160,"available":96602722304,"internally_reserved":0,"allocated":2101248,"data_stored":931418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509554,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33268,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338572,"statfs":{"total":96636764160,"available":96602697728,"internally_reserved":0,"allocated":2113536,"data_stored":937721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542256,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37416,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334424,"statfs":{"total":96636764160,"available":96598450176,"internally_reserved":0,"allocated":2113536,"data_stored":937721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509551,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33256,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338584,"statfs":{"total":96636764160,"available":96602710016,"internally_reserved":0,"allocated":2113536,"data_stored":937721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509553,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33244,"kb_used_data":2052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338596,"statfs":{"total":96636764160,"available":96602722304,"internally_reserved":0,"allocated":2101248,"data_stored":931418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509551,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37996,"kb_used_data":2644,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333844,"statfs":{"total":96636764160,"available":96597856256,"internally_reserved":0,"allocated":2707456,"data_stored":1528089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:47:18.337 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:47:18.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.482+0000 7f521e858640 1 Processor -- start 2024-09-19T08:47:18.483 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.482+0000 7f521e858640 1 -- start start 2024-09-19T08:47:18.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.486+0000 7f521e858640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f521815c550 0x7f521806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:18.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.486+0000 7f521e858640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f521806c1d0 0x7f52180cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:18.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.486+0000 7f521e858640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f52180cf900 0x7f52180ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:18.484 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.486+0000 7f521e858640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f521806c640 con 0x7f521806c1d0 2024-09-19T08:47:18.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.486+0000 7f521e858640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f521806d310 con 0x7f521815c550 2024-09-19T08:47:18.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.486+0000 7f521e858640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f521806d5f0 con 0x7f52180cf900 2024-09-19T08:47:18.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.486+0000 7f520f7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f521806c1d0 0x7f52180cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:18.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.486+0000 7f520f7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f521806c1d0 0x7f52180cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41838/0 (socket says 172.21.15.98:41838) 2024-09-19T08:47:18.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.486+0000 7f520f7fe640 1 -- 172.21.15.98:0/2229994281 learned_addr learned my addr 172.21.15.98:0/2229994281 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:18.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.486+0000 7f520f7fe640 1 -- 172.21.15.98:0/2229994281 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f52180cf900 msgr2=0x7f52180ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:18.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.486+0000 7f520f7fe640 1 --2- 172.21.15.98:0/2229994281 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f52180cf900 0x7f52180ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.486+0000 7f520f7fe640 1 -- 172.21.15.98:0/2229994281 >> v2:172.21.15.139:3300/0 conn(0x7f521815c550 msgr2=0x7f521806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:18.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.486+0000 7f520f7fe640 1 --2- 172.21.15.98:0/2229994281 >> v2:172.21.15.139:3300/0 conn(0x7f521815c550 0x7f521806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.485 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.486+0000 7f520f7fe640 1 -- 172.21.15.98:0/2229994281 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f521806d8d0 con 0x7f521806c1d0 2024-09-19T08:47:18.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.486+0000 7f520f7fe640 1 --2- 172.21.15.98:0/2229994281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f521806c1d0 0x7f52180cf3c0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f52180b56b0 tx=0x7f520001c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ecaf7756359694 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:18.486 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.486+0000 7f52177fe640 1 -- 172.21.15.98:0/2229994281 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52000b0070 con 0x7f521806c1d0 2024-09-19T08:47:18.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.486+0000 7f52177fe640 1 -- 172.21.15.98:0/2229994281 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52000ab600 con 0x7f521806c1d0 2024-09-19T08:47:18.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.486+0000 7f52177fe640 1 -- 172.21.15.98:0/2229994281 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52000ab9d0 con 0x7f521806c1d0 2024-09-19T08:47:18.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.486+0000 7f521e858640 1 -- 172.21.15.98:0/2229994281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f521806c1d0 msgr2=0x7f52180cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:18.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.486+0000 7f521e858640 1 --2- 172.21.15.98:0/2229994281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f521806c1d0 0x7f52180cf3c0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f52180b56b0 tx=0x7f520001c260 comp rx=0 tx=0).stop 2024-09-19T08:47:18.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.486+0000 7f521e858640 1 -- 172.21.15.98:0/2229994281 shutdown_connections 2024-09-19T08:47:18.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.486+0000 7f521e858640 1 --2- 172.21.15.98:0/2229994281 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f521806c1d0 0x7f52180cf3c0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.486+0000 7f521e858640 1 --2- 172.21.15.98:0/2229994281 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f52180cf900 0x7f52180ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.486+0000 7f521e858640 1 --2- 172.21.15.98:0/2229994281 >> v2:172.21.15.139:3300/0 conn(0x7f521815c550 0x7f521806bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.486+0000 7f521e858640 1 -- 172.21.15.98:0/2229994281 >> 172.21.15.98:0/2229994281 conn(0x7f52180b2010 msgr2=0x7f52180b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:18.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.486+0000 7f521e858640 1 -- 172.21.15.98:0/2229994281 shutdown_connections 2024-09-19T08:47:18.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.486+0000 7f521e858640 1 -- 172.21.15.98:0/2229994281 wait complete. 2024-09-19T08:47:18.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.490+0000 7f521e858640 1 Processor -- start 2024-09-19T08:47:18.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.490+0000 7f521e858640 1 -- start start 2024-09-19T08:47:18.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.490+0000 7f521e858640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f521806c1d0 0x7f521815b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:18.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.490+0000 7f521e858640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f52180cf900 0x7f52180b4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:18.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.490+0000 7f521e858640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f521815c550 0x7f521813f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:18.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.490+0000 7f521e858640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5218072520 con 0x7f521806c1d0 2024-09-19T08:47:18.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.490+0000 7f521e858640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f52180723a0 con 0x7f52180cf900 2024-09-19T08:47:18.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.490+0000 7f521e858640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f52180726a0 con 0x7f521815c550 2024-09-19T08:47:18.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.490+0000 7f5217fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f521806c1d0 0x7f521815b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:18.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.490+0000 7f5217fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f521806c1d0 0x7f521815b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41846/0 (socket says 172.21.15.98:41846) 2024-09-19T08:47:18.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.490+0000 7f5217fff640 1 -- 172.21.15.98:0/1453579754 learned_addr learned my addr 172.21.15.98:0/1453579754 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:18.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.490+0000 7f521cdf5640 1 --2- 172.21.15.98:0/1453579754 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f521815c550 0x7f521813f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:18.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.490+0000 7f520f7fe640 1 --2- 172.21.15.98:0/1453579754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f52180cf900 0x7f52180b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:18.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.490+0000 7f5217fff640 1 -- 172.21.15.98:0/1453579754 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f521815c550 msgr2=0x7f521813f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:18.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.490+0000 7f5217fff640 1 --2- 172.21.15.98:0/1453579754 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f521815c550 0x7f521813f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.490+0000 7f5217fff640 1 -- 172.21.15.98:0/1453579754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f52180cf900 msgr2=0x7f52180b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:18.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.490+0000 7f5217fff640 1 --2- 172.21.15.98:0/1453579754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f52180cf900 0x7f52180b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.490+0000 7f5217fff640 1 -- 172.21.15.98:0/1453579754 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52180b4db0 con 0x7f521806c1d0 2024-09-19T08:47:18.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.490+0000 7f520f7fe640 1 --2- 172.21.15.98:0/1453579754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f52180cf900 0x7f52180b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:18.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.490+0000 7f5217fff640 1 --2- 172.21.15.98:0/1453579754 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f521806c1d0 0x7f521815b0f0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f520800b440 tx=0x7f520800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:18.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.490+0000 7f52157fa640 1 -- 172.21.15.98:0/1453579754 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5208015020 con 0x7f521806c1d0 2024-09-19T08:47:18.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.490+0000 7f52157fa640 1 -- 172.21.15.98:0/1453579754 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52080118b0 con 0x7f521806c1d0 2024-09-19T08:47:18.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.490+0000 7f521e858640 1 -- 172.21.15.98:0/1453579754 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5218151220 con 0x7f521806c1d0 2024-09-19T08:47:18.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.490+0000 7f52157fa640 1 -- 172.21.15.98:0/1453579754 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5208011ca0 con 0x7f521806c1d0 2024-09-19T08:47:18.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.490+0000 7f521e858640 1 -- 172.21.15.98:0/1453579754 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52181517e0 con 0x7f521806c1d0 2024-09-19T08:47:18.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.490+0000 7f521e858640 1 -- 172.21.15.98:0/1453579754 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52180ce1a0 con 0x7f521806c1d0 2024-09-19T08:47:18.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.494+0000 7f52157fa640 1 -- 172.21.15.98:0/1453579754 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f5208018020 con 0x7f521806c1d0 2024-09-19T08:47:18.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.494+0000 7f52157fa640 1 --2- 172.21.15.98:0/1453579754 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f51f8068250 0x7f51f806a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:18.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.494+0000 7f52157fa640 1 -- 172.21.15.98:0/1453579754 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5737+0+0 (secure 0 0 0) 0x7f52080845b0 con 0x7f521806c1d0 2024-09-19T08:47:18.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.494+0000 7f520f7fe640 1 --2- 172.21.15.98:0/1453579754 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f51f8068250 0x7f51f806a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:18.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.494+0000 7f520f7fe640 1 --2- 172.21.15.98:0/1453579754 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f51f8068250 0x7f51f806a710 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f52000ad000 tx=0x7f5200008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:18.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.498+0000 7f52157fa640 1 -- 172.21.15.98:0/1453579754 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f520801f020 con 0x7f521806c1d0 2024-09-19T08:47:18.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.642+0000 7f521e858640 1 -- 172.21.15.98:0/1453579754 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5218154fd0 con 0x7f51f8068250 2024-09-19T08:47:18.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.658+0000 7f52157fa640 1 -- 172.21.15.98:0/1453579754 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247089 (secure 0 0 0) 0x7f5218154fd0 con 0x7f51f8068250 2024-09-19T08:47:18.661 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:47:18.663 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:47:18.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.662+0000 7f521e858640 1 -- 172.21.15.98:0/1453579754 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f51f8068250 msgr2=0x7f51f806a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:18.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.662+0000 7f521e858640 1 --2- 172.21.15.98:0/1453579754 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f51f8068250 0x7f51f806a710 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f52000ad000 tx=0x7f5200008040 comp rx=0 tx=0).stop 2024-09-19T08:47:18.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.666+0000 7f521e858640 1 -- 172.21.15.98:0/1453579754 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f521806c1d0 msgr2=0x7f521815b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:18.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.666+0000 7f521e858640 1 --2- 172.21.15.98:0/1453579754 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f521806c1d0 0x7f521815b0f0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f520800b440 tx=0x7f520800b900 comp rx=0 tx=0).stop 2024-09-19T08:47:18.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.666+0000 7f521e858640 1 -- 172.21.15.98:0/1453579754 shutdown_connections 2024-09-19T08:47:18.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.666+0000 7f521e858640 1 --2- 172.21.15.98:0/1453579754 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f51f8068250 0x7f51f806a710 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.666+0000 7f521e858640 1 --2- 172.21.15.98:0/1453579754 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f521815c550 0x7f521813f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.666+0000 7f521e858640 1 --2- 172.21.15.98:0/1453579754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f52180cf900 0x7f52180b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.666+0000 7f521e858640 1 --2- 172.21.15.98:0/1453579754 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f521806c1d0 0x7f521815b0f0 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.666+0000 7f521e858640 1 -- 172.21.15.98:0/1453579754 >> 172.21.15.98:0/1453579754 conn(0x7f52180b2010 msgr2=0x7f521815a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:18.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.666+0000 7f521e858640 1 -- 172.21.15.98:0/1453579754 shutdown_connections 2024-09-19T08:47:18.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.666+0000 7f521e858640 1 -- 172.21.15.98:0/1453579754 wait complete. 2024-09-19T08:47:18.675 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:47:18.677 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:47:18.678 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2024-09-19T08:47:16.958217+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":275500,"kb_used_data":17636,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754699220,"statfs":{"total":773094113280,"available":772812001280,"internally_reserved":0,"allocated":18059264,"data_stored":8663595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.599406"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.375621+0000","last_change":"2024-09-19T08:47:13.375621+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.375621+0000","last_undegraded":"2024-09-19T08:47:13.375621+0000","last_fullsized":"2024-09-19T08:47:13.375621+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.393844+0000","last_change":"2024-09-19T08:47:13.393844+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.393844+0000","last_undegraded":"2024-09-19T08:47:13.393844+0000","last_fullsized":"2024-09-19T08:47:13.393844+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.380106+0000","last_change":"2024-09-19T08:47:13.380106+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.380106+0000","last_undegraded":"2024-09-19T08:47:13.380106+0000","last_fullsized":"2024-09-19T08:47:13.380106+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.383331+0000","last_change":"2024-09-19T08:47:13.383331+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.383331+0000","last_undegraded":"2024-09-19T08:47:13.383331+0000","last_fullsized":"2024-09-19T08:47:13.383331+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.374122+0000","last_change":"2024-09-19T08:47:13.374122+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.374122+0000","last_undegraded":"2024-09-19T08:47:13.374122+0000","last_fullsized":"2024-09-19T08:47:13.374122+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.374054+0000","last_change":"2024-09-19T08:47:13.374054+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.374054+0000","last_undegraded":"2024-09-19T08:47:13.374054+0000","last_fullsized":"2024-09-19T08:47:13.374054+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.377663+0000","last_change":"2024-09-19T08:47:13.377663+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.377663+0000","last_undegraded":"2024-09-19T08:47:13.377663+0000","last_fullsized":"2024-09-19T08:47:13.377663+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.373860+0000","last_change":"2024-09-19T08:47:13.373860+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.373860+0000","last_undegraded":"2024-09-19T08:47:13.373860+0000","last_fullsized":"2024-09-19T08:47:13.373860+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.392064+0000","last_change":"2024-09-19T08:47:13.392064+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.392064+0000","last_undegraded":"2024-09-19T08:47:13.392064+0000","last_fullsized":"2024-09-19T08:47:13.392064+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.374995+0000","last_change":"2024-09-19T08:47:13.374995+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.374995+0000","last_undegraded":"2024-09-19T08:47:13.374995+0000","last_fullsized":"2024-09-19T08:47:13.374995+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.375676+0000","last_change":"2024-09-19T08:47:13.375676+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.375676+0000","last_undegraded":"2024-09-19T08:47:13.375676+0000","last_fullsized":"2024-09-19T08:47:13.375676+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.373586+0000","last_change":"2024-09-19T08:47:13.373586+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.373586+0000","last_undegraded":"2024-09-19T08:47:13.373586+0000","last_fullsized":"2024-09-19T08:47:13.373586+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.373583+0000","last_change":"2024-09-19T08:47:13.373583+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.373583+0000","last_undegraded":"2024-09-19T08:47:13.373583+0000","last_fullsized":"2024-09-19T08:47:13.373583+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.377730+0000","last_change":"2024-09-19T08:47:13.377730+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.377730+0000","last_undegraded":"2024-09-19T08:47:13.377730+0000","last_fullsized":"2024-09-19T08:47:13.377730+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.376178+0000","last_change":"2024-09-19T08:47:13.376178+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.376178+0000","last_undegraded":"2024-09-19T08:47:13.376178+0000","last_fullsized":"2024-09-19T08:47:13.376178+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.375416+0000","last_change":"2024-09-19T08:47:13.375416+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.375416+0000","last_undegraded":"2024-09-19T08:47:13.375416+0000","last_fullsized":"2024-09-19T08:47:13.375416+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.375076+0000","last_change":"2024-09-19T08:47:13.375076+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.375076+0000","last_undegraded":"2024-09-19T08:47:13.375076+0000","last_fullsized":"2024-09-19T08:47:13.375076+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.374529+0000","last_change":"2024-09-19T08:47:13.374529+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.374529+0000","last_undegraded":"2024-09-19T08:47:13.374529+0000","last_fullsized":"2024-09-19T08:47:13.374529+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.375270+0000","last_change":"2024-09-19T08:47:13.375270+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.375270+0000","last_undegraded":"2024-09-19T08:47:13.375270+0000","last_fullsized":"2024-09-19T08:47:13.375270+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.373724+0000","last_change":"2024-09-19T08:47:13.373724+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.373724+0000","last_undegraded":"2024-09-19T08:47:13.373724+0000","last_fullsized":"2024-09-19T08:47:13.373724+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.384490+0000","last_change":"2024-09-19T08:47:13.384490+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.384490+0000","last_undegraded":"2024-09-19T08:47:13.384490+0000","last_fullsized":"2024-09-19T08:47:13.384490+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.388649+0000","last_change":"2024-09-19T08:47:13.388649+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.388649+0000","last_undegraded":"2024-09-19T08:47:13.388649+0000","last_fullsized":"2024-09-19T08:47:13.388649+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.386532+0000","last_change":"2024-09-19T08:47:13.386532+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.386532+0000","last_undegraded":"2024-09-19T08:47:13.386532+0000","last_fullsized":"2024-09-19T08:47:13.386532+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.381686+0000","last_change":"2024-09-19T08:47:13.381686+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.381686+0000","last_undegraded":"2024-09-19T08:47:13.381686+0000","last_fullsized":"2024-09-19T08:47:13.381686+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.397263+0000","last_change":"2024-09-19T08:47:13.397263+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.397263+0000","last_undegraded":"2024-09-19T08:47:13.397263+0000","last_fullsized":"2024-09-19T08:47:13.397263+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.376365+0000","last_change":"2024-09-19T08:47:13.376365+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.376365+0000","last_undegraded":"2024-09-19T08:47:13.376365+0000","last_fullsized":"2024-09-19T08:47:13.376365+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.374470+0000","last_change":"2024-09-19T08:47:13.374470+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.374470+0000","last_undegraded":"2024-09-19T08:47:13.374470+0000","last_fullsized":"2024-09-19T08:47:13.374470+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.375256+0000","last_change":"2024-09-19T08:47:13.375256+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.375256+0000","last_undegraded":"2024-09-19T08:47:13.375256+0000","last_fullsized":"2024-09-19T08:47:13.375256+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.376057+0000","last_change":"2024-09-19T08:47:13.376057+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.376057+0000","last_undegraded":"2024-09-19T08:47:13.376057+0000","last_fullsized":"2024-09-19T08:47:13.376057+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.381136+0000","last_change":"2024-09-19T08:47:13.381136+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.381136+0000","last_undegraded":"2024-09-19T08:47:13.381136+0000","last_fullsized":"2024-09-19T08:47:13.381136+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.376688+0000","last_change":"2024-09-19T08:47:13.376688+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.376688+0000","last_undegraded":"2024-09-19T08:47:13.376688+0000","last_fullsized":"2024-09-19T08:47:13.376688+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.376109+0000","last_change":"2024-09-19T08:47:13.376109+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.376109+0000","last_undegraded":"2024-09-19T08:47:13.376109+0000","last_fullsized":"2024-09-19T08:47:13.376109+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.391812+0000","last_change":"2024-09-19T08:47:13.391812+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.391812+0000","last_undegraded":"2024-09-19T08:47:13.391812+0000","last_fullsized":"2024-09-19T08:47:13.391812+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.404594+0000","last_change":"2024-09-19T08:47:13.401720+0000","last_active":"2024-09-19T08:47:13.404594+0000","last_peered":"2024-09-19T08:47:13.404594+0000","last_clean":"2024-09-19T08:47:13.404594+0000","last_became_active":"2024-09-19T08:47:13.400995+0000","last_became_peered":"2024-09-19T08:47:13.400995+0000","last_unstale":"2024-09-19T08:47:13.404594+0000","last_undegraded":"2024-09-19T08:47:13.404594+0000","last_fullsized":"2024-09-19T08:47:13.404594+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:31:30.868216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.389838+0000","last_change":"2024-09-19T08:47:13.389838+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.389838+0000","last_undegraded":"2024-09-19T08:47:13.389838+0000","last_fullsized":"2024-09-19T08:47:13.389838+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.412531+0000","last_change":"2024-09-19T08:47:13.411165+0000","last_active":"2024-09-19T08:47:13.412531+0000","last_peered":"2024-09-19T08:47:13.412531+0000","last_clean":"2024-09-19T08:47:13.412531+0000","last_became_active":"2024-09-19T08:47:13.410431+0000","last_became_peered":"2024-09-19T08:47:13.410431+0000","last_unstale":"2024-09-19T08:47:13.412531+0000","last_undegraded":"2024-09-19T08:47:13.412531+0000","last_fullsized":"2024-09-19T08:47:13.412531+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:17.138674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.373947+0000","last_change":"2024-09-19T08:47:13.373947+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.373947+0000","last_undegraded":"2024-09-19T08:47:13.373947+0000","last_fullsized":"2024-09-19T08:47:13.373947+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.397842+0000","last_change":"2024-09-19T08:47:13.393085+0000","last_active":"2024-09-19T08:47:13.397842+0000","last_peered":"2024-09-19T08:47:13.397842+0000","last_clean":"2024-09-19T08:47:13.397842+0000","last_became_active":"2024-09-19T08:47:13.392776+0000","last_became_peered":"2024-09-19T08:47:13.392776+0000","last_unstale":"2024-09-19T08:47:13.397842+0000","last_undegraded":"2024-09-19T08:47:13.397842+0000","last_fullsized":"2024-09-19T08:47:13.397842+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:48.525656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.379453+0000","last_change":"2024-09-19T08:47:13.379453+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.379453+0000","last_undegraded":"2024-09-19T08:47:13.379453+0000","last_fullsized":"2024-09-19T08:47:13.379453+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.410308+0000","last_change":"2024-09-19T08:47:13.403896+0000","last_active":"2024-09-19T08:47:13.410308+0000","last_peered":"2024-09-19T08:47:13.410308+0000","last_clean":"2024-09-19T08:47:13.410308+0000","last_became_active":"2024-09-19T08:47:13.403168+0000","last_became_peered":"2024-09-19T08:47:13.403168+0000","last_unstale":"2024-09-19T08:47:13.410308+0000","last_undegraded":"2024-09-19T08:47:13.410308+0000","last_fullsized":"2024-09-19T08:47:13.410308+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:41.705016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.383435+0000","last_change":"2024-09-19T08:47:13.383435+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.383435+0000","last_undegraded":"2024-09-19T08:47:13.383435+0000","last_fullsized":"2024-09-19T08:47:13.383435+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.401038+0000","last_change":"2024-09-19T08:47:13.400567+0000","last_active":"2024-09-19T08:47:13.401038+0000","last_peered":"2024-09-19T08:47:13.401038+0000","last_clean":"2024-09-19T08:47:13.401038+0000","last_became_active":"2024-09-19T08:47:13.400297+0000","last_became_peered":"2024-09-19T08:47:13.400297+0000","last_unstale":"2024-09-19T08:47:13.401038+0000","last_undegraded":"2024-09-19T08:47:13.401038+0000","last_fullsized":"2024-09-19T08:47:13.401038+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:52.284449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.372347+0000","last_change":"2024-09-19T08:47:13.372347+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.372347+0000","last_undegraded":"2024-09-19T08:47:13.372347+0000","last_fullsized":"2024-09-19T08:47:13.372347+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.386785+0000","last_change":"2024-09-19T08:47:13.382448+0000","last_active":"2024-09-19T08:47:13.386785+0000","last_peered":"2024-09-19T08:47:13.386785+0000","last_clean":"2024-09-19T08:47:13.386785+0000","last_became_active":"2024-09-19T08:47:13.382136+0000","last_became_peered":"2024-09-19T08:47:13.382136+0000","last_unstale":"2024-09-19T08:47:13.386785+0000","last_undegraded":"2024-09-19T08:47:13.386785+0000","last_fullsized":"2024-09-19T08:47:13.386785+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:47:16.160287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.380645+0000","last_change":"2024-09-19T08:47:13.380645+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.380645+0000","last_undegraded":"2024-09-19T08:47:13.380645+0000","last_fullsized":"2024-09-19T08:47:13.380645+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.402491+0000","last_change":"2024-09-19T08:47:13.394882+0000","last_active":"2024-09-19T08:47:13.402491+0000","last_peered":"2024-09-19T08:47:13.402491+0000","last_clean":"2024-09-19T08:47:13.402491+0000","last_became_active":"2024-09-19T08:47:13.393932+0000","last_became_peered":"2024-09-19T08:47:13.393932+0000","last_unstale":"2024-09-19T08:47:13.402491+0000","last_undegraded":"2024-09-19T08:47:13.402491+0000","last_fullsized":"2024-09-19T08:47:13.402491+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:54:16.646628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.381875+0000","last_change":"2024-09-19T08:47:13.381875+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.381875+0000","last_undegraded":"2024-09-19T08:47:13.381875+0000","last_fullsized":"2024-09-19T08:47:13.381875+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.400581+0000","last_change":"2024-09-19T08:47:13.398554+0000","last_active":"2024-09-19T08:47:13.400581+0000","last_peered":"2024-09-19T08:47:13.400581+0000","last_clean":"2024-09-19T08:47:13.400581+0000","last_became_active":"2024-09-19T08:47:13.397871+0000","last_became_peered":"2024-09-19T08:47:13.397871+0000","last_unstale":"2024-09-19T08:47:13.400581+0000","last_undegraded":"2024-09-19T08:47:13.400581+0000","last_fullsized":"2024-09-19T08:47:13.400581+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:40.000754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.375234+0000","last_change":"2024-09-19T08:47:13.375234+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.375234+0000","last_undegraded":"2024-09-19T08:47:13.375234+0000","last_fullsized":"2024-09-19T08:47:13.375234+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.412003+0000","last_change":"2024-09-19T08:47:13.407203+0000","last_active":"2024-09-19T08:47:13.412003+0000","last_peered":"2024-09-19T08:47:13.412003+0000","last_clean":"2024-09-19T08:47:13.412003+0000","last_became_active":"2024-09-19T08:47:13.406419+0000","last_became_peered":"2024-09-19T08:47:13.406419+0000","last_unstale":"2024-09-19T08:47:13.412003+0000","last_undegraded":"2024-09-19T08:47:13.412003+0000","last_fullsized":"2024-09-19T08:47:13.412003+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:40.000754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.382196+0000","last_change":"2024-09-19T08:47:13.382196+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.382196+0000","last_undegraded":"2024-09-19T08:47:13.382196+0000","last_fullsized":"2024-09-19T08:47:13.382196+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.386404+0000","last_change":"2024-09-19T08:47:13.382865+0000","last_active":"2024-09-19T08:47:13.386404+0000","last_peered":"2024-09-19T08:47:13.386404+0000","last_clean":"2024-09-19T08:47:13.386404+0000","last_became_active":"2024-09-19T08:47:13.382528+0000","last_became_peered":"2024-09-19T08:47:13.382528+0000","last_unstale":"2024-09-19T08:47:13.386404+0000","last_undegraded":"2024-09-19T08:47:13.386404+0000","last_fullsized":"2024-09-19T08:47:13.386404+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:16.220719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.375815+0000","last_change":"2024-09-19T08:47:13.375815+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.375815+0000","last_undegraded":"2024-09-19T08:47:13.375815+0000","last_fullsized":"2024-09-19T08:47:13.375815+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.411608+0000","last_change":"2024-09-19T08:47:13.409867+0000","last_active":"2024-09-19T08:47:13.411608+0000","last_peered":"2024-09-19T08:47:13.411608+0000","last_clean":"2024-09-19T08:47:13.411608+0000","last_became_active":"2024-09-19T08:47:13.408810+0000","last_became_peered":"2024-09-19T08:47:13.408810+0000","last_unstale":"2024-09-19T08:47:13.411608+0000","last_undegraded":"2024-09-19T08:47:13.411608+0000","last_fullsized":"2024-09-19T08:47:13.411608+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:18:39.135827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.373137+0000","last_change":"2024-09-19T08:47:13.373137+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.373137+0000","last_undegraded":"2024-09-19T08:47:13.373137+0000","last_fullsized":"2024-09-19T08:47:13.373137+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.405795+0000","last_change":"2024-09-19T08:47:13.404129+0000","last_active":"2024-09-19T08:47:13.405795+0000","last_peered":"2024-09-19T08:47:13.405795+0000","last_clean":"2024-09-19T08:47:13.405795+0000","last_became_active":"2024-09-19T08:47:13.403371+0000","last_became_peered":"2024-09-19T08:47:13.403371+0000","last_unstale":"2024-09-19T08:47:13.405795+0000","last_undegraded":"2024-09-19T08:47:13.405795+0000","last_fullsized":"2024-09-19T08:47:13.405795+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:44:57.657282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.379376+0000","last_change":"2024-09-19T08:47:13.379376+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.379376+0000","last_undegraded":"2024-09-19T08:47:13.379376+0000","last_fullsized":"2024-09-19T08:47:13.379376+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.409306+0000","last_change":"2024-09-19T08:47:13.405188+0000","last_active":"2024-09-19T08:47:13.409306+0000","last_peered":"2024-09-19T08:47:13.409306+0000","last_clean":"2024-09-19T08:47:13.409306+0000","last_became_active":"2024-09-19T08:47:13.404512+0000","last_became_peered":"2024-09-19T08:47:13.404512+0000","last_unstale":"2024-09-19T08:47:13.409306+0000","last_undegraded":"2024-09-19T08:47:13.409306+0000","last_fullsized":"2024-09-19T08:47:13.409306+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:22:24.167219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.372698+0000","last_change":"2024-09-19T08:47:13.372698+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.372698+0000","last_undegraded":"2024-09-19T08:47:13.372698+0000","last_fullsized":"2024-09-19T08:47:13.372698+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.397985+0000","last_change":"2024-09-19T08:47:13.393731+0000","last_active":"2024-09-19T08:47:13.397985+0000","last_peered":"2024-09-19T08:47:13.397985+0000","last_clean":"2024-09-19T08:47:13.397985+0000","last_became_active":"2024-09-19T08:47:13.393445+0000","last_became_peered":"2024-09-19T08:47:13.393445+0000","last_unstale":"2024-09-19T08:47:13.397985+0000","last_undegraded":"2024-09-19T08:47:13.397985+0000","last_fullsized":"2024-09-19T08:47:13.397985+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:25:43.296666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.373004+0000","last_change":"2024-09-19T08:47:13.373004+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.373004+0000","last_undegraded":"2024-09-19T08:47:13.373004+0000","last_fullsized":"2024-09-19T08:47:13.373004+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.397746+0000","last_change":"2024-09-19T08:47:13.393111+0000","last_active":"2024-09-19T08:47:13.397746+0000","last_peered":"2024-09-19T08:47:13.397746+0000","last_clean":"2024-09-19T08:47:13.397746+0000","last_became_active":"2024-09-19T08:47:13.392754+0000","last_became_peered":"2024-09-19T08:47:13.392754+0000","last_unstale":"2024-09-19T08:47:13.397746+0000","last_undegraded":"2024-09-19T08:47:13.397746+0000","last_fullsized":"2024-09-19T08:47:13.397746+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:29:51.628450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.390767+0000","last_change":"2024-09-19T08:47:13.390767+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.390767+0000","last_undegraded":"2024-09-19T08:47:13.390767+0000","last_fullsized":"2024-09-19T08:47:13.390767+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.386834+0000","last_change":"2024-09-19T08:47:13.383220+0000","last_active":"2024-09-19T08:47:13.386834+0000","last_peered":"2024-09-19T08:47:13.386834+0000","last_clean":"2024-09-19T08:47:13.386834+0000","last_became_active":"2024-09-19T08:47:13.382953+0000","last_became_peered":"2024-09-19T08:47:13.382953+0000","last_unstale":"2024-09-19T08:47:13.386834+0000","last_undegraded":"2024-09-19T08:47:13.386834+0000","last_fullsized":"2024-09-19T08:47:13.386834+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:54:16.646628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.396193+0000","last_change":"2024-09-19T08:47:13.396193+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.396193+0000","last_undegraded":"2024-09-19T08:47:13.396193+0000","last_fullsized":"2024-09-19T08:47:13.396193+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.412044+0000","last_change":"2024-09-19T08:47:13.408636+0000","last_active":"2024-09-19T08:47:13.412044+0000","last_peered":"2024-09-19T08:47:13.412044+0000","last_clean":"2024-09-19T08:47:13.412044+0000","last_became_active":"2024-09-19T08:47:13.407243+0000","last_became_peered":"2024-09-19T08:47:13.407243+0000","last_unstale":"2024-09-19T08:47:13.412044+0000","last_undegraded":"2024-09-19T08:47:13.412044+0000","last_fullsized":"2024-09-19T08:47:13.412044+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:40.467795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.372029+0000","last_change":"2024-09-19T08:47:13.372029+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.372029+0000","last_undegraded":"2024-09-19T08:47:13.372029+0000","last_fullsized":"2024-09-19T08:47:13.372029+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.386432+0000","last_change":"2024-09-19T08:47:13.382929+0000","last_active":"2024-09-19T08:47:13.386432+0000","last_peered":"2024-09-19T08:47:13.386432+0000","last_clean":"2024-09-19T08:47:13.386432+0000","last_became_active":"2024-09-19T08:47:13.382644+0000","last_became_peered":"2024-09-19T08:47:13.382644+0000","last_unstale":"2024-09-19T08:47:13.386432+0000","last_undegraded":"2024-09-19T08:47:13.386432+0000","last_fullsized":"2024-09-19T08:47:13.386432+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:59:41.025166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.375784+0000","last_change":"2024-09-19T08:47:13.375784+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.375784+0000","last_undegraded":"2024-09-19T08:47:13.375784+0000","last_fullsized":"2024-09-19T08:47:13.375784+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.386306+0000","last_change":"2024-09-19T08:47:13.382212+0000","last_active":"2024-09-19T08:47:13.386306+0000","last_peered":"2024-09-19T08:47:13.386306+0000","last_clean":"2024-09-19T08:47:13.386306+0000","last_became_active":"2024-09-19T08:47:13.381878+0000","last_became_peered":"2024-09-19T08:47:13.381878+0000","last_unstale":"2024-09-19T08:47:13.386306+0000","last_undegraded":"2024-09-19T08:47:13.386306+0000","last_fullsized":"2024-09-19T08:47:13.386306+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:42.546192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.376790+0000","last_change":"2024-09-19T08:47:13.376790+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.376790+0000","last_undegraded":"2024-09-19T08:47:13.376790+0000","last_fullsized":"2024-09-19T08:47:13.376790+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.386944+0000","last_change":"2024-09-19T08:47:13.384584+0000","last_active":"2024-09-19T08:47:13.386944+0000","last_peered":"2024-09-19T08:47:13.386944+0000","last_clean":"2024-09-19T08:47:13.386944+0000","last_became_active":"2024-09-19T08:47:13.384215+0000","last_became_peered":"2024-09-19T08:47:13.384215+0000","last_unstale":"2024-09-19T08:47:13.386944+0000","last_undegraded":"2024-09-19T08:47:13.386944+0000","last_fullsized":"2024-09-19T08:47:13.386944+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:08.459798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.371932+0000","last_change":"2024-09-19T08:47:13.371932+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.371932+0000","last_undegraded":"2024-09-19T08:47:13.371932+0000","last_fullsized":"2024-09-19T08:47:13.371932+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.398410+0000","last_change":"2024-09-19T08:47:13.393941+0000","last_active":"2024-09-19T08:47:13.398410+0000","last_peered":"2024-09-19T08:47:13.398410+0000","last_clean":"2024-09-19T08:47:13.398410+0000","last_became_active":"2024-09-19T08:47:13.393683+0000","last_became_peered":"2024-09-19T08:47:13.393683+0000","last_unstale":"2024-09-19T08:47:13.398410+0000","last_undegraded":"2024-09-19T08:47:13.398410+0000","last_fullsized":"2024-09-19T08:47:13.398410+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:34:09.089055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.380239+0000","last_change":"2024-09-19T08:47:13.380239+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.380239+0000","last_undegraded":"2024-09-19T08:47:13.380239+0000","last_fullsized":"2024-09-19T08:47:13.380239+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.405027+0000","last_change":"2024-09-19T08:47:13.404413+0000","last_active":"2024-09-19T08:47:13.405027+0000","last_peered":"2024-09-19T08:47:13.405027+0000","last_clean":"2024-09-19T08:47:13.405027+0000","last_became_active":"2024-09-19T08:47:13.404206+0000","last_became_peered":"2024-09-19T08:47:13.404206+0000","last_unstale":"2024-09-19T08:47:13.405027+0000","last_undegraded":"2024-09-19T08:47:13.405027+0000","last_fullsized":"2024-09-19T08:47:13.405027+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:54:30.371240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.385386+0000","last_change":"2024-09-19T08:47:13.385386+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.385386+0000","last_undegraded":"2024-09-19T08:47:13.385386+0000","last_fullsized":"2024-09-19T08:47:13.385386+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.392249+0000","last_change":"2024-09-19T08:47:13.387174+0000","last_active":"2024-09-19T08:47:13.392249+0000","last_peered":"2024-09-19T08:47:13.392249+0000","last_clean":"2024-09-19T08:47:13.392249+0000","last_became_active":"2024-09-19T08:47:13.386875+0000","last_became_peered":"2024-09-19T08:47:13.386875+0000","last_unstale":"2024-09-19T08:47:13.392249+0000","last_undegraded":"2024-09-19T08:47:13.392249+0000","last_fullsized":"2024-09-19T08:47:13.392249+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:18:12.747028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.374099+0000","last_change":"2024-09-19T08:47:13.374099+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.374099+0000","last_undegraded":"2024-09-19T08:47:13.374099+0000","last_fullsized":"2024-09-19T08:47:13.374099+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.398842+0000","last_change":"2024-09-19T08:47:13.398116+0000","last_active":"2024-09-19T08:47:13.398842+0000","last_peered":"2024-09-19T08:47:13.398842+0000","last_clean":"2024-09-19T08:47:13.398842+0000","last_became_active":"2024-09-19T08:47:13.397684+0000","last_became_peered":"2024-09-19T08:47:13.397684+0000","last_unstale":"2024-09-19T08:47:13.398842+0000","last_undegraded":"2024-09-19T08:47:13.398842+0000","last_fullsized":"2024-09-19T08:47:13.398842+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:09:47.807044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.374542+0000","last_change":"2024-09-19T08:47:13.374542+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.374542+0000","last_undegraded":"2024-09-19T08:47:13.374542+0000","last_fullsized":"2024-09-19T08:47:13.374542+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.409827+0000","last_change":"2024-09-19T08:47:13.402409+0000","last_active":"2024-09-19T08:47:13.409827+0000","last_peered":"2024-09-19T08:47:13.409827+0000","last_clean":"2024-09-19T08:47:13.409827+0000","last_became_active":"2024-09-19T08:47:13.401624+0000","last_became_peered":"2024-09-19T08:47:13.401624+0000","last_unstale":"2024-09-19T08:47:13.409827+0000","last_undegraded":"2024-09-19T08:47:13.409827+0000","last_fullsized":"2024-09-19T08:47:13.409827+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:59:08.197023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.380966+0000","last_change":"2024-09-19T08:47:13.380966+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.380966+0000","last_undegraded":"2024-09-19T08:47:13.380966+0000","last_fullsized":"2024-09-19T08:47:13.380966+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.389403+0000","last_change":"2024-09-19T08:47:13.385451+0000","last_active":"2024-09-19T08:47:13.389403+0000","last_peered":"2024-09-19T08:47:13.389403+0000","last_clean":"2024-09-19T08:47:13.389403+0000","last_became_active":"2024-09-19T08:47:13.384496+0000","last_became_peered":"2024-09-19T08:47:13.384496+0000","last_unstale":"2024-09-19T08:47:13.389403+0000","last_undegraded":"2024-09-19T08:47:13.389403+0000","last_fullsized":"2024-09-19T08:47:13.389403+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:22:24.167219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.381828+0000","last_change":"2024-09-19T08:47:13.381828+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.381828+0000","last_undegraded":"2024-09-19T08:47:13.381828+0000","last_fullsized":"2024-09-19T08:47:13.381828+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.398847+0000","last_change":"2024-09-19T08:47:13.398325+0000","last_active":"2024-09-19T08:47:13.398847+0000","last_peered":"2024-09-19T08:47:13.398847+0000","last_clean":"2024-09-19T08:47:13.398847+0000","last_became_active":"2024-09-19T08:47:13.397450+0000","last_became_peered":"2024-09-19T08:47:13.397450+0000","last_unstale":"2024-09-19T08:47:13.398847+0000","last_undegraded":"2024-09-19T08:47:13.398847+0000","last_fullsized":"2024-09-19T08:47:13.398847+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:36:34.099661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.376127+0000","last_change":"2024-09-19T08:47:13.376127+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.376127+0000","last_undegraded":"2024-09-19T08:47:13.376127+0000","last_fullsized":"2024-09-19T08:47:13.376127+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.409439+0000","last_change":"2024-09-19T08:47:13.399625+0000","last_active":"2024-09-19T08:47:13.409439+0000","last_peered":"2024-09-19T08:47:13.409439+0000","last_clean":"2024-09-19T08:47:13.409439+0000","last_became_active":"2024-09-19T08:47:13.398838+0000","last_became_peered":"2024-09-19T08:47:13.398838+0000","last_unstale":"2024-09-19T08:47:13.409439+0000","last_undegraded":"2024-09-19T08:47:13.409439+0000","last_fullsized":"2024-09-19T08:47:13.409439+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:33:27.811970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.374929+0000","last_change":"2024-09-19T08:47:13.374929+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.374929+0000","last_undegraded":"2024-09-19T08:47:13.374929+0000","last_fullsized":"2024-09-19T08:47:13.374929+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.389369+0000","last_change":"2024-09-19T08:47:13.384922+0000","last_active":"2024-09-19T08:47:13.389369+0000","last_peered":"2024-09-19T08:47:13.389369+0000","last_clean":"2024-09-19T08:47:13.389369+0000","last_became_active":"2024-09-19T08:47:13.384555+0000","last_became_peered":"2024-09-19T08:47:13.384555+0000","last_unstale":"2024-09-19T08:47:13.389369+0000","last_undegraded":"2024-09-19T08:47:13.389369+0000","last_fullsized":"2024-09-19T08:47:13.389369+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:44:57.657282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.372134+0000","last_change":"2024-09-19T08:47:13.372134+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.372134+0000","last_undegraded":"2024-09-19T08:47:13.372134+0000","last_fullsized":"2024-09-19T08:47:13.372134+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.411931+0000","last_change":"2024-09-19T08:47:13.407182+0000","last_active":"2024-09-19T08:47:13.411931+0000","last_peered":"2024-09-19T08:47:13.411931+0000","last_clean":"2024-09-19T08:47:13.411931+0000","last_became_active":"2024-09-19T08:47:13.405978+0000","last_became_peered":"2024-09-19T08:47:13.405978+0000","last_unstale":"2024-09-19T08:47:13.411931+0000","last_undegraded":"2024-09-19T08:47:13.411931+0000","last_fullsized":"2024-09-19T08:47:13.411931+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:18:12.747028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.386744+0000","last_change":"2024-09-19T08:47:13.386744+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.386744+0000","last_undegraded":"2024-09-19T08:47:13.386744+0000","last_fullsized":"2024-09-19T08:47:13.386744+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.393663+0000","last_change":"2024-09-19T08:47:13.386481+0000","last_active":"2024-09-19T08:47:13.393663+0000","last_peered":"2024-09-19T08:47:13.393663+0000","last_clean":"2024-09-19T08:47:13.393663+0000","last_became_active":"2024-09-19T08:47:13.385716+0000","last_became_peered":"2024-09-19T08:47:13.385716+0000","last_unstale":"2024-09-19T08:47:13.393663+0000","last_undegraded":"2024-09-19T08:47:13.393663+0000","last_fullsized":"2024-09-19T08:47:13.393663+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:27.685629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":1,"reported_epoch":168,"state":"creating+peering","last_fresh":"2024-09-19T08:47:13.377235+0000","last_change":"2024-09-19T08:47:13.377235+0000","last_active":"2024-09-19T08:47:13.355035+0000","last_peered":"2024-09-19T08:47:13.355035+0000","last_clean":"2024-09-19T08:47:13.355035+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T08:47:13.377235+0000","last_undegraded":"2024-09-19T08:47:13.377235+0000","last_fullsized":"2024-09-19T08:47:13.377235+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":9,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.395411+0000","last_change":"2024-09-19T08:47:13.388601+0000","last_active":"2024-09-19T08:47:13.395411+0000","last_peered":"2024-09-19T08:47:13.395411+0000","last_clean":"2024-09-19T08:47:13.395411+0000","last_became_active":"2024-09-19T08:47:13.387981+0000","last_became_peered":"2024-09-19T08:47:13.387981+0000","last_unstale":"2024-09-19T08:47:13.395411+0000","last_undegraded":"2024-09-19T08:47:13.395411+0000","last_fullsized":"2024-09-19T08:47:13.395411+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:43:37.808820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"71'128","reported_seq":474,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-19T08:47:13.385151+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:47:13.385151+0000","last_peered":"2024-09-19T08:47:13.385151+0000","last_clean":"2024-09-19T08:47:13.385151+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:47:13.385151+0000","last_undegraded":"2024-09-19T08:47:13.385151+0000","last_fullsized":"2024-09-19T08:47:13.385151+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509552,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33824,"kb_used_data":2632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338016,"statfs":{"total":96636764160,"available":96602128384,"internally_reserved":0,"allocated":2695168,"data_stored":1521786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574961,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33252,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338588,"statfs":{"total":96636764160,"available":96602714112,"internally_reserved":0,"allocated":2113536,"data_stored":937721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509554,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33244,"kb_used_data":2052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338596,"statfs":{"total":96636764160,"available":96602722304,"internally_reserved":0,"allocated":2101248,"data_stored":931418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509554,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33268,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338572,"statfs":{"total":96636764160,"available":96602697728,"internally_reserved":0,"allocated":2113536,"data_stored":937721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542256,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37416,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334424,"statfs":{"total":96636764160,"available":96598450176,"internally_reserved":0,"allocated":2113536,"data_stored":937721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509551,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33256,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338584,"statfs":{"total":96636764160,"available":96602710016,"internally_reserved":0,"allocated":2113536,"data_stored":937721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509553,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33244,"kb_used_data":2052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338596,"statfs":{"total":96636764160,"available":96602722304,"internally_reserved":0,"allocated":2101248,"data_stored":931418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509551,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37996,"kb_used_data":2644,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333844,"statfs":{"total":96636764160,"available":96597856256,"internally_reserved":0,"allocated":2707456,"data_stored":1528089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:47:18.683 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:47:18.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.834+0000 7f9f264af640 1 Processor -- start 2024-09-19T08:47:18.835 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.834+0000 7f9f264af640 1 -- start start 2024-09-19T08:47:18.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.838+0000 7f9f264af640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9f2015c550 0x7f9f2006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:18.836 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.838+0000 7f9f264af640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f2006c1d0 0x7f9f200cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:18.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.838+0000 7f9f264af640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f2006d310 con 0x7f9f200cf900 2024-09-19T08:47:18.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.838+0000 7f9f264af640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f9f2006c610 con 0x7f9f2015c550 2024-09-19T08:47:18.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.838+0000 7f9f264af640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9f2006d5f0 con 0x7f9f2006c1d0 2024-09-19T08:47:18.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.838+0000 7f9f1ffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9f2015c550 0x7f9f2006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:18.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.838+0000 7f9f1ffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f9f2015c550 0x7f9f2006bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41856/0 (socket says 172.21.15.98:41856) 2024-09-19T08:47:18.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.838+0000 7f9f1ffff640 1 -- 172.21.15.98:0/4097038377 learned_addr learned my addr 172.21.15.98:0/4097038377 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:18.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.838+0000 7f9f24a4c640 1 --1- 172.21.15.98:0/4097038377 >> v1:172.21.15.98:6789/0 conn(0x7f9f200cf900 0x7f9f200ce1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:45986/0 (socket says 172.21.15.98:45986) 2024-09-19T08:47:18.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.838+0000 7f9f1ffff640 1 -- 172.21.15.98:0/4097038377 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f2006c1d0 msgr2=0x7f9f200cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:18.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.838+0000 7f9f1ffff640 1 --2- 172.21.15.98:0/4097038377 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f2006c1d0 0x7f9f200cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.838+0000 7f9f1ffff640 1 -- 172.21.15.98:0/4097038377 >> v1:172.21.15.98:6789/0 conn(0x7f9f200cf900 legacy=0x7f9f200ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:18.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.838+0000 7f9f1effd640 1 -- 172.21.15.98:0/4097038377 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3672313719 0 0) 0x7f9f2006d310 con 0x7f9f200cf900 2024-09-19T08:47:18.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.838+0000 7f9f1ffff640 1 -- 172.21.15.98:0/4097038377 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f2006d8d0 con 0x7f9f2015c550 2024-09-19T08:47:18.838 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.838+0000 7f9f1ffff640 1 --2- 172.21.15.98:0/4097038377 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f2015c550 0x7f9f2006bc90 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f9f10009870 tx=0x7f9f1001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b7e1e4c7bf64577 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:18.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.838+0000 7f9f1effd640 1 -- 172.21.15.98:0/4097038377 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f100b2070 con 0x7f9f2015c550 2024-09-19T08:47:18.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.838+0000 7f9f1effd640 1 -- 172.21.15.98:0/4097038377 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f100a9070 con 0x7f9f2015c550 2024-09-19T08:47:18.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.838+0000 7f9f1effd640 1 -- 172.21.15.98:0/4097038377 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f100ad880 con 0x7f9f2015c550 2024-09-19T08:47:18.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.838+0000 7f9f264af640 1 -- 172.21.15.98:0/4097038377 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f2015c550 msgr2=0x7f9f2006bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:18.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.838+0000 7f9f264af640 1 --2- 172.21.15.98:0/4097038377 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f2015c550 0x7f9f2006bc90 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f9f10009870 tx=0x7f9f1001c560 comp rx=0 tx=0).stop 2024-09-19T08:47:18.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.838+0000 7f9f264af640 1 -- 172.21.15.98:0/4097038377 shutdown_connections 2024-09-19T08:47:18.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.838+0000 7f9f264af640 1 --2- 172.21.15.98:0/4097038377 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f2006c1d0 0x7f9f200cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.838+0000 7f9f264af640 1 --2- 172.21.15.98:0/4097038377 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f2015c550 0x7f9f2006bc90 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.838+0000 7f9f264af640 1 -- 172.21.15.98:0/4097038377 >> 172.21.15.98:0/4097038377 conn(0x7f9f200b2010 msgr2=0x7f9f200b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:18.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.838+0000 7f9f264af640 1 -- 172.21.15.98:0/4097038377 shutdown_connections 2024-09-19T08:47:18.840 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.838+0000 7f9f264af640 1 -- 172.21.15.98:0/4097038377 wait complete. 2024-09-19T08:47:18.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.842+0000 7f9f264af640 1 Processor -- start 2024-09-19T08:47:18.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.842+0000 7f9f264af640 1 -- start start 2024-09-19T08:47:18.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.842+0000 7f9f264af640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f2006c1d0 0x7f9f2014a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:18.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.842+0000 7f9f264af640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f200cf900 0x7f9f2014f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:18.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.842+0000 7f9f264af640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f2015c550 0x7f9f2014fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:18.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.842+0000 7f9f264af640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f9f20169230 con 0x7f9f2006c1d0 2024-09-19T08:47:18.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.842+0000 7f9f264af640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2015cb40 con 0x7f9f200cf900 2024-09-19T08:47:18.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.842+0000 7f9f264af640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f9f201693b0 con 0x7f9f2015c550 2024-09-19T08:47:18.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.842+0000 7f9f1ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f2006c1d0 0x7f9f2014a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:18.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.842+0000 7f9f1ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f2006c1d0 0x7f9f2014a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41862/0 (socket says 172.21.15.98:41862) 2024-09-19T08:47:18.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.842+0000 7f9f1ffff640 1 -- 172.21.15.98:0/1260234602 learned_addr learned my addr 172.21.15.98:0/1260234602 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:18.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.842+0000 7f9f1ffff640 1 -- 172.21.15.98:0/1260234602 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f2015c550 msgr2=0x7f9f2014fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:18.843 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.842+0000 7f9f1ffff640 1 --2- 172.21.15.98:0/1260234602 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f2015c550 0x7f9f2014fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.842+0000 7f9f1ffff640 1 -- 172.21.15.98:0/1260234602 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f200cf900 msgr2=0x7f9f2014f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:18.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.842+0000 7f9f1ffff640 1 --2- 172.21.15.98:0/1260234602 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f200cf900 0x7f9f2014f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:18.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.842+0000 7f9f1ffff640 1 -- 172.21.15.98:0/1260234602 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f201579b0 con 0x7f9f2006c1d0 2024-09-19T08:47:18.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.842+0000 7f9f1ffff640 1 --2- 172.21.15.98:0/1260234602 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f2006c1d0 0x7f9f2014a6b0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f9f1001ac10 tx=0x7f9f1001ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:18.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.842+0000 7f9f1cff9640 1 -- 172.21.15.98:0/1260234602 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f100bc020 con 0x7f9f2006c1d0 2024-09-19T08:47:18.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.842+0000 7f9f1cff9640 1 -- 172.21.15.98:0/1260234602 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f100b8420 con 0x7f9f2006c1d0 2024-09-19T08:47:18.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.842+0000 7f9f1cff9640 1 -- 172.21.15.98:0/1260234602 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f100b2040 con 0x7f9f2006c1d0 2024-09-19T08:47:18.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.842+0000 7f9f264af640 1 -- 172.21.15.98:0/1260234602 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f20157c90 con 0x7f9f2006c1d0 2024-09-19T08:47:18.844 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.842+0000 7f9f264af640 1 -- 172.21.15.98:0/1260234602 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f2014b2c0 con 0x7f9f2006c1d0 2024-09-19T08:47:18.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.842+0000 7f9f264af640 1 -- 172.21.15.98:0/1260234602 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f200cf3c0 con 0x7f9f2006c1d0 2024-09-19T08:47:18.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.846+0000 7f9f1cff9640 1 -- 172.21.15.98:0/1260234602 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f9f100bf020 con 0x7f9f2006c1d0 2024-09-19T08:47:18.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.846+0000 7f9f1cff9640 1 --2- 172.21.15.98:0/1260234602 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9f00068250 0x7f9f0006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:18.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.846+0000 7f9f1cff9640 1 -- 172.21.15.98:0/1260234602 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5737+0+0 (secure 0 0 0) 0x7f9f1011e780 con 0x7f9f2006c1d0 2024-09-19T08:47:18.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.846+0000 7f9f1f7fe640 1 --2- 172.21.15.98:0/1260234602 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9f00068250 0x7f9f0006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:18.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.846+0000 7f9f1f7fe640 1 --2- 172.21.15.98:0/1260234602 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9f00068250 0x7f9f0006a710 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9f200af630 tx=0x7f9f08006ca0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:18.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.850+0000 7f9f1cff9640 1 -- 172.21.15.98:0/1260234602 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9f100c6030 con 0x7f9f2006c1d0 2024-09-19T08:47:18.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:18.994+0000 7f9f264af640 1 -- 172.21.15.98:0/1260234602 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9f2015b850 con 0x7f9f00068250 2024-09-19T08:47:19.013 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.014+0000 7f9f1cff9640 1 -- 172.21.15.98:0/1260234602 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252496 (secure 0 0 0) 0x7f9f2015b850 con 0x7f9f00068250 2024-09-19T08:47:19.015 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:47:19.017 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:47:19.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.018+0000 7f9f264af640 1 -- 172.21.15.98:0/1260234602 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9f00068250 msgr2=0x7f9f0006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:19.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.018+0000 7f9f264af640 1 --2- 172.21.15.98:0/1260234602 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9f00068250 0x7f9f0006a710 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9f200af630 tx=0x7f9f08006ca0 comp rx=0 tx=0).stop 2024-09-19T08:47:19.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.018+0000 7f9f264af640 1 -- 172.21.15.98:0/1260234602 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f2006c1d0 msgr2=0x7f9f2014a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:19.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.018+0000 7f9f264af640 1 --2- 172.21.15.98:0/1260234602 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f2006c1d0 0x7f9f2014a6b0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f9f1001ac10 tx=0x7f9f1001ada0 comp rx=0 tx=0).stop 2024-09-19T08:47:19.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.018+0000 7f9f264af640 1 -- 172.21.15.98:0/1260234602 shutdown_connections 2024-09-19T08:47:19.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.018+0000 7f9f264af640 1 --2- 172.21.15.98:0/1260234602 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f9f00068250 0x7f9f0006a710 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:19.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.018+0000 7f9f264af640 1 --2- 172.21.15.98:0/1260234602 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f9f2015c550 0x7f9f2014fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:19.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.018+0000 7f9f264af640 1 --2- 172.21.15.98:0/1260234602 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9f200cf900 0x7f9f2014f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:19.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.018+0000 7f9f264af640 1 --2- 172.21.15.98:0/1260234602 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f9f2006c1d0 0x7f9f2014a6b0 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:19.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.018+0000 7f9f264af640 1 -- 172.21.15.98:0/1260234602 >> 172.21.15.98:0/1260234602 conn(0x7f9f200b2010 msgr2=0x7f9f200cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:19.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.018+0000 7f9f264af640 1 -- 172.21.15.98:0/1260234602 shutdown_connections 2024-09-19T08:47:19.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.018+0000 7f9f264af640 1 -- 172.21.15.98:0/1260234602 wait complete. 2024-09-19T08:47:19.032 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2024-09-19T08:47:18.958819+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":275768,"kb_used_data":17928,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754698952,"statfs":{"total":773094113280,"available":772811726848,"internally_reserved":0,"allocated":18358272,"data_stored":8792372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":58,"apply_latency_ms":58,"commit_latency_ns":58000000,"apply_latency_ns":58000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001878"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833628+0000","last_change":"2024-09-19T08:47:14.359460+0000","last_active":"2024-09-19T08:47:15.833628+0000","last_peered":"2024-09-19T08:47:15.833628+0000","last_clean":"2024-09-19T08:47:15.833628+0000","last_became_active":"2024-09-19T08:47:14.352467+0000","last_became_peered":"2024-09-19T08:47:14.352467+0000","last_unstale":"2024-09-19T08:47:15.833628+0000","last_undegraded":"2024-09-19T08:47:15.833628+0000","last_fullsized":"2024-09-19T08:47:15.833628+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:12.331382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.834037+0000","last_change":"2024-09-19T08:47:14.363174+0000","last_active":"2024-09-19T08:47:15.834037+0000","last_peered":"2024-09-19T08:47:15.834037+0000","last_clean":"2024-09-19T08:47:15.834037+0000","last_became_active":"2024-09-19T08:47:14.362109+0000","last_became_peered":"2024-09-19T08:47:14.362109+0000","last_unstale":"2024-09-19T08:47:15.834037+0000","last_undegraded":"2024-09-19T08:47:15.834037+0000","last_fullsized":"2024-09-19T08:47:15.834037+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:34.366623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.701137+0000","last_change":"2024-09-19T08:47:14.365200+0000","last_active":"2024-09-19T08:47:15.701137+0000","last_peered":"2024-09-19T08:47:15.701137+0000","last_clean":"2024-09-19T08:47:15.701137+0000","last_became_active":"2024-09-19T08:47:14.357837+0000","last_became_peered":"2024-09-19T08:47:14.357837+0000","last_unstale":"2024-09-19T08:47:15.701137+0000","last_undegraded":"2024-09-19T08:47:15.701137+0000","last_fullsized":"2024-09-19T08:47:15.701137+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:54.142030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.383795+0000","last_change":"2024-09-19T08:47:14.354043+0000","last_active":"2024-09-19T08:47:15.383795+0000","last_peered":"2024-09-19T08:47:15.383795+0000","last_clean":"2024-09-19T08:47:15.383795+0000","last_became_active":"2024-09-19T08:47:14.352832+0000","last_became_peered":"2024-09-19T08:47:14.352832+0000","last_unstale":"2024-09-19T08:47:15.383795+0000","last_undegraded":"2024-09-19T08:47:15.383795+0000","last_fullsized":"2024-09-19T08:47:15.383795+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:38:10.816584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.315069+0000","last_change":"2024-09-19T08:47:14.342189+0000","last_active":"2024-09-19T08:47:15.315069+0000","last_peered":"2024-09-19T08:47:15.315069+0000","last_clean":"2024-09-19T08:47:15.315069+0000","last_became_active":"2024-09-19T08:47:14.340626+0000","last_became_peered":"2024-09-19T08:47:14.340626+0000","last_unstale":"2024-09-19T08:47:15.315069+0000","last_undegraded":"2024-09-19T08:47:15.315069+0000","last_fullsized":"2024-09-19T08:47:15.315069+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:42.709485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.314735+0000","last_change":"2024-09-19T08:47:14.361414+0000","last_active":"2024-09-19T08:47:15.314735+0000","last_peered":"2024-09-19T08:47:15.314735+0000","last_clean":"2024-09-19T08:47:15.314735+0000","last_became_active":"2024-09-19T08:47:14.360629+0000","last_became_peered":"2024-09-19T08:47:14.360629+0000","last_unstale":"2024-09-19T08:47:15.314735+0000","last_undegraded":"2024-09-19T08:47:15.314735+0000","last_fullsized":"2024-09-19T08:47:15.314735+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:33:28.816440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.832761+0000","last_change":"2024-09-19T08:47:14.360193+0000","last_active":"2024-09-19T08:47:15.832761+0000","last_peered":"2024-09-19T08:47:15.832761+0000","last_clean":"2024-09-19T08:47:15.832761+0000","last_became_active":"2024-09-19T08:47:14.356098+0000","last_became_peered":"2024-09-19T08:47:14.356098+0000","last_unstale":"2024-09-19T08:47:15.832761+0000","last_undegraded":"2024-09-19T08:47:15.832761+0000","last_fullsized":"2024-09-19T08:47:15.832761+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:53:48.646448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.314425+0000","last_change":"2024-09-19T08:47:14.351408+0000","last_active":"2024-09-19T08:47:15.314425+0000","last_peered":"2024-09-19T08:47:15.314425+0000","last_clean":"2024-09-19T08:47:15.314425+0000","last_became_active":"2024-09-19T08:47:14.350440+0000","last_became_peered":"2024-09-19T08:47:14.350440+0000","last_unstale":"2024-09-19T08:47:15.314425+0000","last_undegraded":"2024-09-19T08:47:15.314425+0000","last_fullsized":"2024-09-19T08:47:15.314425+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:20:48.189924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.834432+0000","last_change":"2024-09-19T08:47:14.357553+0000","last_active":"2024-09-19T08:47:15.834432+0000","last_peered":"2024-09-19T08:47:15.834432+0000","last_clean":"2024-09-19T08:47:15.834432+0000","last_became_active":"2024-09-19T08:47:14.355255+0000","last_became_peered":"2024-09-19T08:47:14.355255+0000","last_unstale":"2024-09-19T08:47:15.834432+0000","last_undegraded":"2024-09-19T08:47:15.834432+0000","last_fullsized":"2024-09-19T08:47:15.834432+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:55:39.012179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.831464+0000","last_change":"2024-09-19T08:47:14.350592+0000","last_active":"2024-09-19T08:47:15.831464+0000","last_peered":"2024-09-19T08:47:15.831464+0000","last_clean":"2024-09-19T08:47:15.831464+0000","last_became_active":"2024-09-19T08:47:14.349843+0000","last_became_peered":"2024-09-19T08:47:14.349843+0000","last_unstale":"2024-09-19T08:47:15.831464+0000","last_undegraded":"2024-09-19T08:47:15.831464+0000","last_fullsized":"2024-09-19T08:47:15.831464+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:36.184294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.701152+0000","last_change":"2024-09-19T08:47:14.347087+0000","last_active":"2024-09-19T08:47:15.701152+0000","last_peered":"2024-09-19T08:47:15.701152+0000","last_clean":"2024-09-19T08:47:15.701152+0000","last_became_active":"2024-09-19T08:47:14.346271+0000","last_became_peered":"2024-09-19T08:47:14.346271+0000","last_unstale":"2024-09-19T08:47:15.701152+0000","last_undegraded":"2024-09-19T08:47:15.701152+0000","last_fullsized":"2024-09-19T08:47:15.701152+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:23.614854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.388582+0000","last_change":"2024-09-19T08:47:14.344723+0000","last_active":"2024-09-19T08:47:15.388582+0000","last_peered":"2024-09-19T08:47:15.388582+0000","last_clean":"2024-09-19T08:47:15.388582+0000","last_became_active":"2024-09-19T08:47:14.343948+0000","last_became_peered":"2024-09-19T08:47:14.343948+0000","last_unstale":"2024-09-19T08:47:15.388582+0000","last_undegraded":"2024-09-19T08:47:15.388582+0000","last_fullsized":"2024-09-19T08:47:15.388582+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:16.685634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.388746+0000","last_change":"2024-09-19T08:47:14.352237+0000","last_active":"2024-09-19T08:47:15.388746+0000","last_peered":"2024-09-19T08:47:15.388746+0000","last_clean":"2024-09-19T08:47:15.388746+0000","last_became_active":"2024-09-19T08:47:14.351466+0000","last_became_peered":"2024-09-19T08:47:14.351466+0000","last_unstale":"2024-09-19T08:47:15.388746+0000","last_undegraded":"2024-09-19T08:47:15.388746+0000","last_fullsized":"2024-09-19T08:47:15.388746+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:44:09.637091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.832981+0000","last_change":"2024-09-19T08:47:14.352279+0000","last_active":"2024-09-19T08:47:15.832981+0000","last_peered":"2024-09-19T08:47:15.832981+0000","last_clean":"2024-09-19T08:47:15.832981+0000","last_became_active":"2024-09-19T08:47:14.348493+0000","last_became_peered":"2024-09-19T08:47:14.348493+0000","last_unstale":"2024-09-19T08:47:15.832981+0000","last_undegraded":"2024-09-19T08:47:15.832981+0000","last_fullsized":"2024-09-19T08:47:15.832981+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:57:37.877782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.832961+0000","last_change":"2024-09-19T08:47:14.360081+0000","last_active":"2024-09-19T08:47:15.832961+0000","last_peered":"2024-09-19T08:47:15.832961+0000","last_clean":"2024-09-19T08:47:15.832961+0000","last_became_active":"2024-09-19T08:47:14.355887+0000","last_became_peered":"2024-09-19T08:47:14.355887+0000","last_unstale":"2024-09-19T08:47:15.832961+0000","last_undegraded":"2024-09-19T08:47:15.832961+0000","last_fullsized":"2024-09-19T08:47:15.832961+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:47.995264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.830850+0000","last_change":"2024-09-19T08:47:14.337255+0000","last_active":"2024-09-19T08:47:15.830850+0000","last_peered":"2024-09-19T08:47:15.830850+0000","last_clean":"2024-09-19T08:47:15.830850+0000","last_became_active":"2024-09-19T08:47:14.336468+0000","last_became_peered":"2024-09-19T08:47:14.336468+0000","last_unstale":"2024-09-19T08:47:15.830850+0000","last_undegraded":"2024-09-19T08:47:15.830850+0000","last_fullsized":"2024-09-19T08:47:15.830850+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:12:56.723940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.834027+0000","last_change":"2024-09-19T08:47:14.353222+0000","last_active":"2024-09-19T08:47:15.834027+0000","last_peered":"2024-09-19T08:47:15.834027+0000","last_clean":"2024-09-19T08:47:15.834027+0000","last_became_active":"2024-09-19T08:47:14.351562+0000","last_became_peered":"2024-09-19T08:47:14.351562+0000","last_unstale":"2024-09-19T08:47:15.834027+0000","last_undegraded":"2024-09-19T08:47:15.834027+0000","last_fullsized":"2024-09-19T08:47:15.834027+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:58:45.045360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.831561+0000","last_change":"2024-09-19T08:47:14.340655+0000","last_active":"2024-09-19T08:47:15.831561+0000","last_peered":"2024-09-19T08:47:15.831561+0000","last_clean":"2024-09-19T08:47:15.831561+0000","last_became_active":"2024-09-19T08:47:14.339893+0000","last_became_peered":"2024-09-19T08:47:14.339893+0000","last_unstale":"2024-09-19T08:47:15.831561+0000","last_undegraded":"2024-09-19T08:47:15.831561+0000","last_fullsized":"2024-09-19T08:47:15.831561+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:56.648047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.831544+0000","last_change":"2024-09-19T08:47:14.350896+0000","last_active":"2024-09-19T08:47:15.831544+0000","last_peered":"2024-09-19T08:47:15.831544+0000","last_clean":"2024-09-19T08:47:15.831544+0000","last_became_active":"2024-09-19T08:47:14.349975+0000","last_became_peered":"2024-09-19T08:47:14.349975+0000","last_unstale":"2024-09-19T08:47:15.831544+0000","last_undegraded":"2024-09-19T08:47:15.831544+0000","last_fullsized":"2024-09-19T08:47:15.831544+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:50.758257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.315869+0000","last_change":"2024-09-19T08:47:14.338182+0000","last_active":"2024-09-19T08:47:15.315869+0000","last_peered":"2024-09-19T08:47:15.315869+0000","last_clean":"2024-09-19T08:47:15.315869+0000","last_became_active":"2024-09-19T08:47:14.336028+0000","last_became_peered":"2024-09-19T08:47:14.336028+0000","last_unstale":"2024-09-19T08:47:15.315869+0000","last_undegraded":"2024-09-19T08:47:15.315869+0000","last_fullsized":"2024-09-19T08:47:15.315869+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:18:40.140297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.383323+0000","last_change":"2024-09-19T08:47:14.348401+0000","last_active":"2024-09-19T08:47:15.383323+0000","last_peered":"2024-09-19T08:47:15.383323+0000","last_clean":"2024-09-19T08:47:15.383323+0000","last_became_active":"2024-09-19T08:47:14.346885+0000","last_became_peered":"2024-09-19T08:47:14.346885+0000","last_unstale":"2024-09-19T08:47:15.383323+0000","last_undegraded":"2024-09-19T08:47:15.383323+0000","last_fullsized":"2024-09-19T08:47:15.383323+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:23:37.200270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.318153+0000","last_change":"2024-09-19T08:47:14.338348+0000","last_active":"2024-09-19T08:47:15.318153+0000","last_peered":"2024-09-19T08:47:15.318153+0000","last_clean":"2024-09-19T08:47:15.318153+0000","last_became_active":"2024-09-19T08:47:14.337676+0000","last_became_peered":"2024-09-19T08:47:14.337676+0000","last_unstale":"2024-09-19T08:47:15.318153+0000","last_undegraded":"2024-09-19T08:47:15.318153+0000","last_fullsized":"2024-09-19T08:47:15.318153+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:53:47.277291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.318101+0000","last_change":"2024-09-19T08:47:14.337863+0000","last_active":"2024-09-19T08:47:15.318101+0000","last_peered":"2024-09-19T08:47:15.318101+0000","last_clean":"2024-09-19T08:47:15.318101+0000","last_became_active":"2024-09-19T08:47:14.337170+0000","last_became_peered":"2024-09-19T08:47:14.337170+0000","last_unstale":"2024-09-19T08:47:15.318101+0000","last_undegraded":"2024-09-19T08:47:15.318101+0000","last_fullsized":"2024-09-19T08:47:15.318101+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:41:06.545819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.383167+0000","last_change":"2024-09-19T08:47:14.349384+0000","last_active":"2024-09-19T08:47:15.383167+0000","last_peered":"2024-09-19T08:47:15.383167+0000","last_clean":"2024-09-19T08:47:15.383167+0000","last_became_active":"2024-09-19T08:47:14.346975+0000","last_became_peered":"2024-09-19T08:47:14.346975+0000","last_unstale":"2024-09-19T08:47:15.383167+0000","last_undegraded":"2024-09-19T08:47:15.383167+0000","last_fullsized":"2024-09-19T08:47:15.383167+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:15.674519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.834210+0000","last_change":"2024-09-19T08:47:14.358211+0000","last_active":"2024-09-19T08:47:15.834210+0000","last_peered":"2024-09-19T08:47:15.834210+0000","last_clean":"2024-09-19T08:47:15.834210+0000","last_became_active":"2024-09-19T08:47:14.356031+0000","last_became_peered":"2024-09-19T08:47:14.356031+0000","last_unstale":"2024-09-19T08:47:15.834210+0000","last_undegraded":"2024-09-19T08:47:15.834210+0000","last_fullsized":"2024-09-19T08:47:15.834210+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:14:56.115719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833588+0000","last_change":"2024-09-19T08:47:14.351542+0000","last_active":"2024-09-19T08:47:15.833588+0000","last_peered":"2024-09-19T08:47:15.833588+0000","last_clean":"2024-09-19T08:47:15.833588+0000","last_became_active":"2024-09-19T08:47:14.348510+0000","last_became_peered":"2024-09-19T08:47:14.348510+0000","last_unstale":"2024-09-19T08:47:15.833588+0000","last_undegraded":"2024-09-19T08:47:15.833588+0000","last_fullsized":"2024-09-19T08:47:15.833588+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:06:30.241912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.388376+0000","last_change":"2024-09-19T08:47:14.343722+0000","last_active":"2024-09-19T08:47:15.388376+0000","last_peered":"2024-09-19T08:47:15.388376+0000","last_clean":"2024-09-19T08:47:15.388376+0000","last_became_active":"2024-09-19T08:47:14.342950+0000","last_became_peered":"2024-09-19T08:47:14.342950+0000","last_unstale":"2024-09-19T08:47:15.388376+0000","last_undegraded":"2024-09-19T08:47:15.388376+0000","last_fullsized":"2024-09-19T08:47:15.388376+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:52.353505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.700591+0000","last_change":"2024-09-19T08:47:14.354023+0000","last_active":"2024-09-19T08:47:15.700591+0000","last_peered":"2024-09-19T08:47:15.700591+0000","last_clean":"2024-09-19T08:47:15.700591+0000","last_became_active":"2024-09-19T08:47:14.347846+0000","last_became_peered":"2024-09-19T08:47:14.347846+0000","last_unstale":"2024-09-19T08:47:15.700591+0000","last_undegraded":"2024-09-19T08:47:15.700591+0000","last_fullsized":"2024-09-19T08:47:15.700591+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:29:56.943652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.388148+0000","last_change":"2024-09-19T08:47:14.343483+0000","last_active":"2024-09-19T08:47:15.388148+0000","last_peered":"2024-09-19T08:47:15.388148+0000","last_clean":"2024-09-19T08:47:15.388148+0000","last_became_active":"2024-09-19T08:47:14.342737+0000","last_became_peered":"2024-09-19T08:47:14.342737+0000","last_unstale":"2024-09-19T08:47:15.388148+0000","last_undegraded":"2024-09-19T08:47:15.388148+0000","last_fullsized":"2024-09-19T08:47:15.388148+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:23:35.024819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833688+0000","last_change":"2024-09-19T08:47:14.354661+0000","last_active":"2024-09-19T08:47:15.833688+0000","last_peered":"2024-09-19T08:47:15.833688+0000","last_clean":"2024-09-19T08:47:15.833688+0000","last_became_active":"2024-09-19T08:47:14.353824+0000","last_became_peered":"2024-09-19T08:47:14.353824+0000","last_unstale":"2024-09-19T08:47:15.833688+0000","last_undegraded":"2024-09-19T08:47:15.833688+0000","last_fullsized":"2024-09-19T08:47:15.833688+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:15:39.112225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.830773+0000","last_change":"2024-09-19T08:47:14.346187+0000","last_active":"2024-09-19T08:47:15.830773+0000","last_peered":"2024-09-19T08:47:15.830773+0000","last_clean":"2024-09-19T08:47:15.830773+0000","last_became_active":"2024-09-19T08:47:14.339726+0000","last_became_peered":"2024-09-19T08:47:14.339726+0000","last_unstale":"2024-09-19T08:47:15.830773+0000","last_undegraded":"2024-09-19T08:47:15.830773+0000","last_fullsized":"2024-09-19T08:47:15.830773+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:01.753982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833104+0000","last_change":"2024-09-19T08:47:14.352760+0000","last_active":"2024-09-19T08:47:15.833104+0000","last_peered":"2024-09-19T08:47:15.833104+0000","last_clean":"2024-09-19T08:47:15.833104+0000","last_became_active":"2024-09-19T08:47:14.352060+0000","last_became_peered":"2024-09-19T08:47:14.352060+0000","last_unstale":"2024-09-19T08:47:15.833104+0000","last_undegraded":"2024-09-19T08:47:15.833104+0000","last_fullsized":"2024-09-19T08:47:15.833104+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:35:03.456650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.834160+0000","last_change":"2024-09-19T08:47:14.362807+0000","last_active":"2024-09-19T08:47:15.834160+0000","last_peered":"2024-09-19T08:47:15.834160+0000","last_clean":"2024-09-19T08:47:15.834160+0000","last_became_active":"2024-09-19T08:47:14.361708+0000","last_became_peered":"2024-09-19T08:47:14.361708+0000","last_unstale":"2024-09-19T08:47:15.834160+0000","last_undegraded":"2024-09-19T08:47:15.834160+0000","last_fullsized":"2024-09-19T08:47:15.834160+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:58:18.390110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"169'4","reported_seq":18,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.388872+0000","last_change":"2024-09-19T08:47:13.401720+0000","last_active":"2024-09-19T08:47:15.388872+0000","last_peered":"2024-09-19T08:47:15.388872+0000","last_clean":"2024-09-19T08:47:15.388872+0000","last_became_active":"2024-09-19T08:47:13.400995+0000","last_became_peered":"2024-09-19T08:47:13.400995+0000","last_unstale":"2024-09-19T08:47:15.388872+0000","last_undegraded":"2024-09-19T08:47:15.388872+0000","last_fullsized":"2024-09-19T08:47:15.388872+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:55.697252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.319123+0000","last_change":"2024-09-19T08:47:14.334414+0000","last_active":"2024-09-19T08:47:15.319123+0000","last_peered":"2024-09-19T08:47:15.319123+0000","last_clean":"2024-09-19T08:47:15.319123+0000","last_became_active":"2024-09-19T08:47:14.333618+0000","last_became_peered":"2024-09-19T08:47:14.333618+0000","last_unstale":"2024-09-19T08:47:15.319123+0000","last_undegraded":"2024-09-19T08:47:15.319123+0000","last_fullsized":"2024-09-19T08:47:15.319123+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:10:52.949042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"169'1","reported_seq":15,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.834325+0000","last_change":"2024-09-19T08:47:13.411165+0000","last_active":"2024-09-19T08:47:15.834325+0000","last_peered":"2024-09-19T08:47:15.834325+0000","last_clean":"2024-09-19T08:47:15.834325+0000","last_became_active":"2024-09-19T08:47:13.410431+0000","last_became_peered":"2024-09-19T08:47:13.410431+0000","last_unstale":"2024-09-19T08:47:15.834325+0000","last_undegraded":"2024-09-19T08:47:15.834325+0000","last_fullsized":"2024-09-19T08:47:15.834325+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:04.142471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.834365+0000","last_change":"2024-09-19T08:47:14.347732+0000","last_active":"2024-09-19T08:47:15.834365+0000","last_peered":"2024-09-19T08:47:15.834365+0000","last_clean":"2024-09-19T08:47:15.834365+0000","last_became_active":"2024-09-19T08:47:14.346863+0000","last_became_peered":"2024-09-19T08:47:14.346863+0000","last_unstale":"2024-09-19T08:47:15.834365+0000","last_undegraded":"2024-09-19T08:47:15.834365+0000","last_fullsized":"2024-09-19T08:47:15.834365+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:31:49.532264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"169'2","reported_seq":16,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.701304+0000","last_change":"2024-09-19T08:47:13.393085+0000","last_active":"2024-09-19T08:47:15.701304+0000","last_peered":"2024-09-19T08:47:15.701304+0000","last_clean":"2024-09-19T08:47:15.701304+0000","last_became_active":"2024-09-19T08:47:13.392776+0000","last_became_peered":"2024-09-19T08:47:13.392776+0000","last_unstale":"2024-09-19T08:47:15.701304+0000","last_undegraded":"2024-09-19T08:47:15.701304+0000","last_fullsized":"2024-09-19T08:47:15.701304+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:45:50.989483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.831442+0000","last_change":"2024-09-19T08:47:14.345730+0000","last_active":"2024-09-19T08:47:15.831442+0000","last_peered":"2024-09-19T08:47:15.831442+0000","last_clean":"2024-09-19T08:47:15.831442+0000","last_became_active":"2024-09-19T08:47:14.345014+0000","last_became_peered":"2024-09-19T08:47:14.345014+0000","last_unstale":"2024-09-19T08:47:15.831442+0000","last_undegraded":"2024-09-19T08:47:15.831442+0000","last_fullsized":"2024-09-19T08:47:15.831442+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:17.675241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"169'2","reported_seq":16,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.384194+0000","last_change":"2024-09-19T08:47:13.403896+0000","last_active":"2024-09-19T08:47:15.384194+0000","last_peered":"2024-09-19T08:47:15.384194+0000","last_clean":"2024-09-19T08:47:15.384194+0000","last_became_active":"2024-09-19T08:47:13.403168+0000","last_became_peered":"2024-09-19T08:47:13.403168+0000","last_unstale":"2024-09-19T08:47:15.384194+0000","last_undegraded":"2024-09-19T08:47:15.384194+0000","last_fullsized":"2024-09-19T08:47:15.384194+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:50:14.051718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.383345+0000","last_change":"2024-09-19T08:47:14.351675+0000","last_active":"2024-09-19T08:47:15.383345+0000","last_peered":"2024-09-19T08:47:15.383345+0000","last_clean":"2024-09-19T08:47:15.383345+0000","last_became_active":"2024-09-19T08:47:14.350955+0000","last_became_peered":"2024-09-19T08:47:14.350955+0000","last_unstale":"2024-09-19T08:47:15.383345+0000","last_undegraded":"2024-09-19T08:47:15.383345+0000","last_fullsized":"2024-09-19T08:47:15.383345+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:44.489711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"169'1","reported_seq":15,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.314986+0000","last_change":"2024-09-19T08:47:13.400567+0000","last_active":"2024-09-19T08:47:15.314986+0000","last_peered":"2024-09-19T08:47:15.314986+0000","last_clean":"2024-09-19T08:47:15.314986+0000","last_became_active":"2024-09-19T08:47:13.400297+0000","last_became_peered":"2024-09-19T08:47:13.400297+0000","last_unstale":"2024-09-19T08:47:15.314986+0000","last_undegraded":"2024-09-19T08:47:15.314986+0000","last_fullsized":"2024-09-19T08:47:15.314986+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:14:37.029395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.314387+0000","last_change":"2024-09-19T08:47:14.341230+0000","last_active":"2024-09-19T08:47:15.314387+0000","last_peered":"2024-09-19T08:47:15.314387+0000","last_clean":"2024-09-19T08:47:15.314387+0000","last_became_active":"2024-09-19T08:47:14.340408+0000","last_became_peered":"2024-09-19T08:47:14.340408+0000","last_unstale":"2024-09-19T08:47:15.314387+0000","last_undegraded":"2024-09-19T08:47:15.314387+0000","last_fullsized":"2024-09-19T08:47:15.314387+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:27:37.674410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.314321+0000","last_change":"2024-09-19T08:47:13.382448+0000","last_active":"2024-09-19T08:47:15.314321+0000","last_peered":"2024-09-19T08:47:15.314321+0000","last_clean":"2024-09-19T08:47:15.314321+0000","last_became_active":"2024-09-19T08:47:13.382136+0000","last_became_peered":"2024-09-19T08:47:13.382136+0000","last_unstale":"2024-09-19T08:47:15.314321+0000","last_undegraded":"2024-09-19T08:47:15.314321+0000","last_fullsized":"2024-09-19T08:47:15.314321+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:31.792660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.383313+0000","last_change":"2024-09-19T08:47:14.354758+0000","last_active":"2024-09-19T08:47:15.383313+0000","last_peered":"2024-09-19T08:47:15.383313+0000","last_clean":"2024-09-19T08:47:15.383313+0000","last_became_active":"2024-09-19T08:47:14.353999+0000","last_became_peered":"2024-09-19T08:47:14.353999+0000","last_unstale":"2024-09-19T08:47:15.383313+0000","last_undegraded":"2024-09-19T08:47:15.383313+0000","last_fullsized":"2024-09-19T08:47:15.383313+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:18.467994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.317703+0000","last_change":"2024-09-19T08:47:13.394882+0000","last_active":"2024-09-19T08:47:15.317703+0000","last_peered":"2024-09-19T08:47:15.317703+0000","last_clean":"2024-09-19T08:47:15.317703+0000","last_became_active":"2024-09-19T08:47:13.393932+0000","last_became_peered":"2024-09-19T08:47:13.393932+0000","last_unstale":"2024-09-19T08:47:15.317703+0000","last_undegraded":"2024-09-19T08:47:15.317703+0000","last_fullsized":"2024-09-19T08:47:15.317703+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:52.284449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.384004+0000","last_change":"2024-09-19T08:47:14.365557+0000","last_active":"2024-09-19T08:47:15.384004+0000","last_peered":"2024-09-19T08:47:15.384004+0000","last_clean":"2024-09-19T08:47:15.384004+0000","last_became_active":"2024-09-19T08:47:14.364879+0000","last_became_peered":"2024-09-19T08:47:14.364879+0000","last_unstale":"2024-09-19T08:47:15.384004+0000","last_undegraded":"2024-09-19T08:47:15.384004+0000","last_fullsized":"2024-09-19T08:47:15.384004+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:51:01.879120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.314681+0000","last_change":"2024-09-19T08:47:13.398554+0000","last_active":"2024-09-19T08:47:15.314681+0000","last_peered":"2024-09-19T08:47:15.314681+0000","last_clean":"2024-09-19T08:47:15.314681+0000","last_became_active":"2024-09-19T08:47:13.397871+0000","last_became_peered":"2024-09-19T08:47:13.397871+0000","last_unstale":"2024-09-19T08:47:15.314681+0000","last_undegraded":"2024-09-19T08:47:15.314681+0000","last_fullsized":"2024-09-19T08:47:15.314681+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:41.165756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.701266+0000","last_change":"2024-09-19T08:47:14.346956+0000","last_active":"2024-09-19T08:47:15.701266+0000","last_peered":"2024-09-19T08:47:15.701266+0000","last_clean":"2024-09-19T08:47:15.701266+0000","last_became_active":"2024-09-19T08:47:14.346229+0000","last_became_peered":"2024-09-19T08:47:14.346229+0000","last_unstale":"2024-09-19T08:47:15.701266+0000","last_undegraded":"2024-09-19T08:47:15.701266+0000","last_fullsized":"2024-09-19T08:47:15.701266+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:39:52.187988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.319341+0000","last_change":"2024-09-19T08:47:13.407203+0000","last_active":"2024-09-19T08:47:15.319341+0000","last_peered":"2024-09-19T08:47:15.319341+0000","last_clean":"2024-09-19T08:47:15.319341+0000","last_became_active":"2024-09-19T08:47:13.406419+0000","last_became_peered":"2024-09-19T08:47:13.406419+0000","last_unstale":"2024-09-19T08:47:15.319341+0000","last_undegraded":"2024-09-19T08:47:15.319341+0000","last_fullsized":"2024-09-19T08:47:15.319341+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:31.792660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.384218+0000","last_change":"2024-09-19T08:47:14.349482+0000","last_active":"2024-09-19T08:47:15.384218+0000","last_peered":"2024-09-19T08:47:15.384218+0000","last_clean":"2024-09-19T08:47:15.384218+0000","last_became_active":"2024-09-19T08:47:14.348750+0000","last_became_peered":"2024-09-19T08:47:14.348750+0000","last_unstale":"2024-09-19T08:47:15.384218+0000","last_undegraded":"2024-09-19T08:47:15.384218+0000","last_fullsized":"2024-09-19T08:47:15.384218+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:47:56.893479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"169'3","reported_seq":17,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.831832+0000","last_change":"2024-09-19T08:47:13.382865+0000","last_active":"2024-09-19T08:47:15.831832+0000","last_peered":"2024-09-19T08:47:15.831832+0000","last_clean":"2024-09-19T08:47:15.831832+0000","last_became_active":"2024-09-19T08:47:13.382528+0000","last_became_peered":"2024-09-19T08:47:13.382528+0000","last_unstale":"2024-09-19T08:47:15.831832+0000","last_undegraded":"2024-09-19T08:47:15.831832+0000","last_fullsized":"2024-09-19T08:47:15.831832+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:21.216354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833935+0000","last_change":"2024-09-19T08:47:14.347539+0000","last_active":"2024-09-19T08:47:15.833935+0000","last_peered":"2024-09-19T08:47:15.833935+0000","last_clean":"2024-09-19T08:47:15.833935+0000","last_became_active":"2024-09-19T08:47:14.346707+0000","last_became_peered":"2024-09-19T08:47:14.346707+0000","last_unstale":"2024-09-19T08:47:15.833935+0000","last_undegraded":"2024-09-19T08:47:15.833935+0000","last_fullsized":"2024-09-19T08:47:15.833935+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:28:44.021408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"169'6","reported_seq":20,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.384347+0000","last_change":"2024-09-19T08:47:13.409867+0000","last_active":"2024-09-19T08:47:15.384347+0000","last_peered":"2024-09-19T08:47:15.384347+0000","last_clean":"2024-09-19T08:47:15.384347+0000","last_became_active":"2024-09-19T08:47:13.408810+0000","last_became_peered":"2024-09-19T08:47:13.408810+0000","last_unstale":"2024-09-19T08:47:15.384347+0000","last_undegraded":"2024-09-19T08:47:15.384347+0000","last_fullsized":"2024-09-19T08:47:15.384347+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:04.792972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.314861+0000","last_change":"2024-09-19T08:47:14.352505+0000","last_active":"2024-09-19T08:47:15.314861+0000","last_peered":"2024-09-19T08:47:15.314861+0000","last_clean":"2024-09-19T08:47:15.314861+0000","last_became_active":"2024-09-19T08:47:14.351672+0000","last_became_peered":"2024-09-19T08:47:14.351672+0000","last_unstale":"2024-09-19T08:47:15.314861+0000","last_undegraded":"2024-09-19T08:47:15.314861+0000","last_fullsized":"2024-09-19T08:47:15.314861+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:59:09.201492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"169'2","reported_seq":16,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.318763+0000","last_change":"2024-09-19T08:47:13.404129+0000","last_active":"2024-09-19T08:47:15.318763+0000","last_peered":"2024-09-19T08:47:15.318763+0000","last_clean":"2024-09-19T08:47:15.318763+0000","last_became_active":"2024-09-19T08:47:13.403371+0000","last_became_peered":"2024-09-19T08:47:13.403371+0000","last_unstale":"2024-09-19T08:47:15.318763+0000","last_undegraded":"2024-09-19T08:47:15.318763+0000","last_fullsized":"2024-09-19T08:47:15.318763+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:17.535117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.831322+0000","last_change":"2024-09-19T08:47:14.359096+0000","last_active":"2024-09-19T08:47:15.831322+0000","last_peered":"2024-09-19T08:47:15.831322+0000","last_clean":"2024-09-19T08:47:15.831322+0000","last_became_active":"2024-09-19T08:47:14.358410+0000","last_became_peered":"2024-09-19T08:47:14.358410+0000","last_unstale":"2024-09-19T08:47:15.831322+0000","last_undegraded":"2024-09-19T08:47:15.831322+0000","last_fullsized":"2024-09-19T08:47:15.831322+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:10:15.182744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.318529+0000","last_change":"2024-09-19T08:47:13.405188+0000","last_active":"2024-09-19T08:47:15.318529+0000","last_peered":"2024-09-19T08:47:15.318529+0000","last_clean":"2024-09-19T08:47:15.318529+0000","last_became_active":"2024-09-19T08:47:13.404512+0000","last_became_peered":"2024-09-19T08:47:13.404512+0000","last_unstale":"2024-09-19T08:47:15.318529+0000","last_undegraded":"2024-09-19T08:47:15.318529+0000","last_fullsized":"2024-09-19T08:47:15.318529+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:38.934425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.700267+0000","last_change":"2024-09-19T08:47:14.347839+0000","last_active":"2024-09-19T08:47:15.700267+0000","last_peered":"2024-09-19T08:47:15.700267+0000","last_clean":"2024-09-19T08:47:15.700267+0000","last_became_active":"2024-09-19T08:47:14.346533+0000","last_became_peered":"2024-09-19T08:47:14.346533+0000","last_unstale":"2024-09-19T08:47:15.700267+0000","last_undegraded":"2024-09-19T08:47:15.700267+0000","last_fullsized":"2024-09-19T08:47:15.700267+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:09:46.261549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.830605+0000","last_change":"2024-09-19T08:47:13.393731+0000","last_active":"2024-09-19T08:47:15.830605+0000","last_peered":"2024-09-19T08:47:15.830605+0000","last_clean":"2024-09-19T08:47:15.830605+0000","last_became_active":"2024-09-19T08:47:13.393445+0000","last_became_peered":"2024-09-19T08:47:13.393445+0000","last_unstale":"2024-09-19T08:47:15.830605+0000","last_undegraded":"2024-09-19T08:47:15.830605+0000","last_fullsized":"2024-09-19T08:47:15.830605+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:17:50.481692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833216+0000","last_change":"2024-09-19T08:47:14.362136+0000","last_active":"2024-09-19T08:47:15.833216+0000","last_peered":"2024-09-19T08:47:15.833216+0000","last_clean":"2024-09-19T08:47:15.833216+0000","last_became_active":"2024-09-19T08:47:14.360937+0000","last_became_peered":"2024-09-19T08:47:14.360937+0000","last_unstale":"2024-09-19T08:47:15.833216+0000","last_undegraded":"2024-09-19T08:47:15.833216+0000","last_fullsized":"2024-09-19T08:47:15.833216+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:09:33.724240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"169'8","reported_seq":22,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.700621+0000","last_change":"2024-09-19T08:47:13.393111+0000","last_active":"2024-09-19T08:47:15.700621+0000","last_peered":"2024-09-19T08:47:15.700621+0000","last_clean":"2024-09-19T08:47:15.700621+0000","last_became_active":"2024-09-19T08:47:13.392754+0000","last_became_peered":"2024-09-19T08:47:13.392754+0000","last_unstale":"2024-09-19T08:47:15.700621+0000","last_undegraded":"2024-09-19T08:47:15.700621+0000","last_fullsized":"2024-09-19T08:47:15.700621+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:47:39.662192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.834342+0000","last_change":"2024-09-19T08:47:14.363789+0000","last_active":"2024-09-19T08:47:15.834342+0000","last_peered":"2024-09-19T08:47:15.834342+0000","last_clean":"2024-09-19T08:47:15.834342+0000","last_became_active":"2024-09-19T08:47:14.362514+0000","last_became_peered":"2024-09-19T08:47:14.362514+0000","last_unstale":"2024-09-19T08:47:15.834342+0000","last_undegraded":"2024-09-19T08:47:15.834342+0000","last_fullsized":"2024-09-19T08:47:15.834342+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:45.701737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"169'1","reported_seq":15,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.314766+0000","last_change":"2024-09-19T08:47:13.383220+0000","last_active":"2024-09-19T08:47:15.314766+0000","last_peered":"2024-09-19T08:47:15.314766+0000","last_clean":"2024-09-19T08:47:15.314766+0000","last_became_active":"2024-09-19T08:47:13.382953+0000","last_became_peered":"2024-09-19T08:47:13.382953+0000","last_unstale":"2024-09-19T08:47:15.314766+0000","last_undegraded":"2024-09-19T08:47:15.314766+0000","last_fullsized":"2024-09-19T08:47:15.314766+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:24.754354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.834207+0000","last_change":"2024-09-19T08:47:14.358774+0000","last_active":"2024-09-19T08:47:15.834207+0000","last_peered":"2024-09-19T08:47:15.834207+0000","last_clean":"2024-09-19T08:47:15.834207+0000","last_became_active":"2024-09-19T08:47:14.356848+0000","last_became_peered":"2024-09-19T08:47:14.356848+0000","last_unstale":"2024-09-19T08:47:15.834207+0000","last_undegraded":"2024-09-19T08:47:15.834207+0000","last_fullsized":"2024-09-19T08:47:15.834207+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:32:24.678912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"169'2","reported_seq":16,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.834488+0000","last_change":"2024-09-19T08:47:13.408636+0000","last_active":"2024-09-19T08:47:15.834488+0000","last_peered":"2024-09-19T08:47:15.834488+0000","last_clean":"2024-09-19T08:47:15.834488+0000","last_became_active":"2024-09-19T08:47:13.407243+0000","last_became_peered":"2024-09-19T08:47:13.407243+0000","last_unstale":"2024-09-19T08:47:15.834488+0000","last_undegraded":"2024-09-19T08:47:15.834488+0000","last_fullsized":"2024-09-19T08:47:15.834488+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:59.000541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.700602+0000","last_change":"2024-09-19T08:47:14.355537+0000","last_active":"2024-09-19T08:47:15.700602+0000","last_peered":"2024-09-19T08:47:15.700602+0000","last_clean":"2024-09-19T08:47:15.700602+0000","last_became_active":"2024-09-19T08:47:14.354555+0000","last_became_peered":"2024-09-19T08:47:14.354555+0000","last_unstale":"2024-09-19T08:47:15.700602+0000","last_undegraded":"2024-09-19T08:47:15.700602+0000","last_fullsized":"2024-09-19T08:47:15.700602+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:44.668862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"169'1","reported_seq":15,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.831941+0000","last_change":"2024-09-19T08:47:13.382929+0000","last_active":"2024-09-19T08:47:15.831941+0000","last_peered":"2024-09-19T08:47:15.831941+0000","last_clean":"2024-09-19T08:47:15.831941+0000","last_became_active":"2024-09-19T08:47:13.382644+0000","last_became_peered":"2024-09-19T08:47:13.382644+0000","last_unstale":"2024-09-19T08:47:15.831941+0000","last_undegraded":"2024-09-19T08:47:15.831941+0000","last_fullsized":"2024-09-19T08:47:15.831941+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:08:23.245952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.831118+0000","last_change":"2024-09-19T08:47:14.345326+0000","last_active":"2024-09-19T08:47:15.831118+0000","last_peered":"2024-09-19T08:47:15.831118+0000","last_clean":"2024-09-19T08:47:15.831118+0000","last_became_active":"2024-09-19T08:47:14.340690+0000","last_became_peered":"2024-09-19T08:47:14.340690+0000","last_unstale":"2024-09-19T08:47:15.831118+0000","last_undegraded":"2024-09-19T08:47:15.831118+0000","last_fullsized":"2024-09-19T08:47:15.831118+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:34.978283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.700881+0000","last_change":"2024-09-19T08:47:13.382212+0000","last_active":"2024-09-19T08:47:15.700881+0000","last_peered":"2024-09-19T08:47:15.700881+0000","last_clean":"2024-09-19T08:47:15.700881+0000","last_became_active":"2024-09-19T08:47:13.381878+0000","last_became_peered":"2024-09-19T08:47:13.381878+0000","last_unstale":"2024-09-19T08:47:15.700881+0000","last_undegraded":"2024-09-19T08:47:15.700881+0000","last_fullsized":"2024-09-19T08:47:15.700881+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:43.813585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.831842+0000","last_change":"2024-09-19T08:47:14.346182+0000","last_active":"2024-09-19T08:47:15.831842+0000","last_peered":"2024-09-19T08:47:15.831842+0000","last_clean":"2024-09-19T08:47:15.831842+0000","last_became_active":"2024-09-19T08:47:14.345377+0000","last_became_peered":"2024-09-19T08:47:14.345377+0000","last_unstale":"2024-09-19T08:47:15.831842+0000","last_undegraded":"2024-09-19T08:47:15.831842+0000","last_fullsized":"2024-09-19T08:47:15.831842+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:59.978626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"169'2","reported_seq":16,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.831403+0000","last_change":"2024-09-19T08:47:13.384584+0000","last_active":"2024-09-19T08:47:15.831403+0000","last_peered":"2024-09-19T08:47:15.831403+0000","last_clean":"2024-09-19T08:47:15.831403+0000","last_became_active":"2024-09-19T08:47:13.384215+0000","last_became_peered":"2024-09-19T08:47:13.384215+0000","last_unstale":"2024-09-19T08:47:15.831403+0000","last_undegraded":"2024-09-19T08:47:15.831403+0000","last_fullsized":"2024-09-19T08:47:15.831403+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:19.499642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.700643+0000","last_change":"2024-09-19T08:47:14.353176+0000","last_active":"2024-09-19T08:47:15.700643+0000","last_peered":"2024-09-19T08:47:15.700643+0000","last_clean":"2024-09-19T08:47:15.700643+0000","last_became_active":"2024-09-19T08:47:14.347339+0000","last_became_peered":"2024-09-19T08:47:14.347339+0000","last_unstale":"2024-09-19T08:47:15.700643+0000","last_undegraded":"2024-09-19T08:47:15.700643+0000","last_fullsized":"2024-09-19T08:47:15.700643+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:09:59.841644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833567+0000","last_change":"2024-09-19T08:47:13.393941+0000","last_active":"2024-09-19T08:47:15.833567+0000","last_peered":"2024-09-19T08:47:15.833567+0000","last_clean":"2024-09-19T08:47:15.833567+0000","last_became_active":"2024-09-19T08:47:13.393683+0000","last_became_peered":"2024-09-19T08:47:13.393683+0000","last_unstale":"2024-09-19T08:47:15.833567+0000","last_undegraded":"2024-09-19T08:47:15.833567+0000","last_fullsized":"2024-09-19T08:47:15.833567+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:29.678095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833498+0000","last_change":"2024-09-19T08:47:14.354662+0000","last_active":"2024-09-19T08:47:15.833498+0000","last_peered":"2024-09-19T08:47:15.833498+0000","last_clean":"2024-09-19T08:47:15.833498+0000","last_became_active":"2024-09-19T08:47:14.353887+0000","last_became_peered":"2024-09-19T08:47:14.353887+0000","last_unstale":"2024-09-19T08:47:15.833498+0000","last_undegraded":"2024-09-19T08:47:15.833498+0000","last_fullsized":"2024-09-19T08:47:15.833498+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:19.026785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833264+0000","last_change":"2024-09-19T08:47:13.404413+0000","last_active":"2024-09-19T08:47:15.833264+0000","last_peered":"2024-09-19T08:47:15.833264+0000","last_clean":"2024-09-19T08:47:15.833264+0000","last_became_active":"2024-09-19T08:47:13.404206+0000","last_became_peered":"2024-09-19T08:47:13.404206+0000","last_unstale":"2024-09-19T08:47:15.833264+0000","last_undegraded":"2024-09-19T08:47:15.833264+0000","last_fullsized":"2024-09-19T08:47:15.833264+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:09.485498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.318156+0000","last_change":"2024-09-19T08:47:14.340136+0000","last_active":"2024-09-19T08:47:15.318156+0000","last_peered":"2024-09-19T08:47:15.318156+0000","last_clean":"2024-09-19T08:47:15.318156+0000","last_became_active":"2024-09-19T08:47:14.337946+0000","last_became_peered":"2024-09-19T08:47:14.337946+0000","last_unstale":"2024-09-19T08:47:15.318156+0000","last_undegraded":"2024-09-19T08:47:15.318156+0000","last_fullsized":"2024-09-19T08:47:15.318156+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:03.353387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.314696+0000","last_change":"2024-09-19T08:47:13.387174+0000","last_active":"2024-09-19T08:47:15.314696+0000","last_peered":"2024-09-19T08:47:15.314696+0000","last_clean":"2024-09-19T08:47:15.314696+0000","last_became_active":"2024-09-19T08:47:13.386875+0000","last_became_peered":"2024-09-19T08:47:13.386875+0000","last_unstale":"2024-09-19T08:47:15.314696+0000","last_undegraded":"2024-09-19T08:47:15.314696+0000","last_fullsized":"2024-09-19T08:47:15.314696+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:22:04.699154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.388462+0000","last_change":"2024-09-19T08:47:14.344815+0000","last_active":"2024-09-19T08:47:15.388462+0000","last_peered":"2024-09-19T08:47:15.388462+0000","last_clean":"2024-09-19T08:47:15.388462+0000","last_became_active":"2024-09-19T08:47:14.343956+0000","last_became_peered":"2024-09-19T08:47:14.343956+0000","last_unstale":"2024-09-19T08:47:15.388462+0000","last_undegraded":"2024-09-19T08:47:15.388462+0000","last_fullsized":"2024-09-19T08:47:15.388462+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:17:40.271999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"169'2","reported_seq":16,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.701112+0000","last_change":"2024-09-19T08:47:13.398116+0000","last_active":"2024-09-19T08:47:15.701112+0000","last_peered":"2024-09-19T08:47:15.701112+0000","last_clean":"2024-09-19T08:47:15.701112+0000","last_became_active":"2024-09-19T08:47:13.397684+0000","last_became_peered":"2024-09-19T08:47:13.397684+0000","last_unstale":"2024-09-19T08:47:15.701112+0000","last_undegraded":"2024-09-19T08:47:15.701112+0000","last_fullsized":"2024-09-19T08:47:15.701112+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:10:26.683853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833962+0000","last_change":"2024-09-19T08:47:14.360783+0000","last_active":"2024-09-19T08:47:15.833962+0000","last_peered":"2024-09-19T08:47:15.833962+0000","last_clean":"2024-09-19T08:47:15.833962+0000","last_became_active":"2024-09-19T08:47:14.348853+0000","last_became_peered":"2024-09-19T08:47:14.348853+0000","last_unstale":"2024-09-19T08:47:15.833962+0000","last_undegraded":"2024-09-19T08:47:15.833962+0000","last_fullsized":"2024-09-19T08:47:15.833962+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:46:22.684299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"170'1","reported_seq":15,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.384227+0000","last_change":"2024-09-19T08:47:13.402409+0000","last_active":"2024-09-19T08:47:15.384227+0000","last_peered":"2024-09-19T08:47:15.384227+0000","last_clean":"2024-09-19T08:47:15.384227+0000","last_became_active":"2024-09-19T08:47:13.401624+0000","last_became_peered":"2024-09-19T08:47:13.401624+0000","last_unstale":"2024-09-19T08:47:15.384227+0000","last_undegraded":"2024-09-19T08:47:15.384227+0000","last_fullsized":"2024-09-19T08:47:15.384227+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:45.064523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833923+0000","last_change":"2024-09-19T08:47:14.354272+0000","last_active":"2024-09-19T08:47:15.833923+0000","last_peered":"2024-09-19T08:47:15.833923+0000","last_clean":"2024-09-19T08:47:15.833923+0000","last_became_active":"2024-09-19T08:47:14.353523+0000","last_became_peered":"2024-09-19T08:47:14.353523+0000","last_unstale":"2024-09-19T08:47:15.833923+0000","last_undegraded":"2024-09-19T08:47:15.833923+0000","last_fullsized":"2024-09-19T08:47:15.833923+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:49.222433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.315399+0000","last_change":"2024-09-19T08:47:13.385451+0000","last_active":"2024-09-19T08:47:15.315399+0000","last_peered":"2024-09-19T08:47:15.315399+0000","last_clean":"2024-09-19T08:47:15.315399+0000","last_became_active":"2024-09-19T08:47:13.384496+0000","last_became_peered":"2024-09-19T08:47:13.384496+0000","last_unstale":"2024-09-19T08:47:15.315399+0000","last_undegraded":"2024-09-19T08:47:15.315399+0000","last_fullsized":"2024-09-19T08:47:15.315399+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:38.934425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.318949+0000","last_change":"2024-09-19T08:47:14.346827+0000","last_active":"2024-09-19T08:47:15.318949+0000","last_peered":"2024-09-19T08:47:15.318949+0000","last_clean":"2024-09-19T08:47:15.318949+0000","last_became_active":"2024-09-19T08:47:14.346102+0000","last_became_peered":"2024-09-19T08:47:14.346102+0000","last_unstale":"2024-09-19T08:47:15.318949+0000","last_undegraded":"2024-09-19T08:47:15.318949+0000","last_fullsized":"2024-09-19T08:47:15.318949+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:11.917219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.700870+0000","last_change":"2024-09-19T08:47:13.398325+0000","last_active":"2024-09-19T08:47:15.700870+0000","last_peered":"2024-09-19T08:47:15.700870+0000","last_clean":"2024-09-19T08:47:15.700870+0000","last_became_active":"2024-09-19T08:47:13.397450+0000","last_became_peered":"2024-09-19T08:47:13.397450+0000","last_unstale":"2024-09-19T08:47:15.700870+0000","last_undegraded":"2024-09-19T08:47:15.700870+0000","last_fullsized":"2024-09-19T08:47:15.700870+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:49:23.696716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833467+0000","last_change":"2024-09-19T08:47:14.344698+0000","last_active":"2024-09-19T08:47:15.833467+0000","last_peered":"2024-09-19T08:47:15.833467+0000","last_clean":"2024-09-19T08:47:15.833467+0000","last_became_active":"2024-09-19T08:47:14.343844+0000","last_became_peered":"2024-09-19T08:47:14.343844+0000","last_unstale":"2024-09-19T08:47:15.833467+0000","last_undegraded":"2024-09-19T08:47:15.833467+0000","last_fullsized":"2024-09-19T08:47:15.833467+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:25:42.320274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.383623+0000","last_change":"2024-09-19T08:47:13.399625+0000","last_active":"2024-09-19T08:47:15.383623+0000","last_peered":"2024-09-19T08:47:15.383623+0000","last_clean":"2024-09-19T08:47:15.383623+0000","last_became_active":"2024-09-19T08:47:13.398838+0000","last_became_peered":"2024-09-19T08:47:13.398838+0000","last_unstale":"2024-09-19T08:47:15.383623+0000","last_undegraded":"2024-09-19T08:47:15.383623+0000","last_fullsized":"2024-09-19T08:47:15.383623+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:26:50.957727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.388653+0000","last_change":"2024-09-19T08:47:14.343451+0000","last_active":"2024-09-19T08:47:15.388653+0000","last_peered":"2024-09-19T08:47:15.388653+0000","last_clean":"2024-09-19T08:47:15.388653+0000","last_became_active":"2024-09-19T08:47:14.342662+0000","last_became_peered":"2024-09-19T08:47:14.342662+0000","last_unstale":"2024-09-19T08:47:15.388653+0000","last_undegraded":"2024-09-19T08:47:15.388653+0000","last_fullsized":"2024-09-19T08:47:15.388653+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:02:13.929162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.315041+0000","last_change":"2024-09-19T08:47:13.384922+0000","last_active":"2024-09-19T08:47:15.315041+0000","last_peered":"2024-09-19T08:47:15.315041+0000","last_clean":"2024-09-19T08:47:15.315041+0000","last_became_active":"2024-09-19T08:47:13.384555+0000","last_became_peered":"2024-09-19T08:47:13.384555+0000","last_unstale":"2024-09-19T08:47:15.315041+0000","last_undegraded":"2024-09-19T08:47:15.315041+0000","last_fullsized":"2024-09-19T08:47:15.315041+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:27:36.013002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.700392+0000","last_change":"2024-09-19T08:47:14.347684+0000","last_active":"2024-09-19T08:47:15.700392+0000","last_peered":"2024-09-19T08:47:15.700392+0000","last_clean":"2024-09-19T08:47:15.700392+0000","last_became_active":"2024-09-19T08:47:14.345594+0000","last_became_peered":"2024-09-19T08:47:14.345594+0000","last_unstale":"2024-09-19T08:47:15.700392+0000","last_undegraded":"2024-09-19T08:47:15.700392+0000","last_fullsized":"2024-09-19T08:47:15.700392+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:11.450025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.318402+0000","last_change":"2024-09-19T08:47:13.407182+0000","last_active":"2024-09-19T08:47:15.318402+0000","last_peered":"2024-09-19T08:47:15.318402+0000","last_clean":"2024-09-19T08:47:15.318402+0000","last_became_active":"2024-09-19T08:47:13.405978+0000","last_became_peered":"2024-09-19T08:47:13.405978+0000","last_unstale":"2024-09-19T08:47:15.318402+0000","last_undegraded":"2024-09-19T08:47:15.318402+0000","last_fullsized":"2024-09-19T08:47:15.318402+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:41.165756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833516+0000","last_change":"2024-09-19T08:47:14.354669+0000","last_active":"2024-09-19T08:47:15.833516+0000","last_peered":"2024-09-19T08:47:15.833516+0000","last_clean":"2024-09-19T08:47:15.833516+0000","last_became_active":"2024-09-19T08:47:14.353479+0000","last_became_peered":"2024-09-19T08:47:14.353479+0000","last_unstale":"2024-09-19T08:47:15.833516+0000","last_undegraded":"2024-09-19T08:47:15.833516+0000","last_fullsized":"2024-09-19T08:47:15.833516+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:37:58.191552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.387876+0000","last_change":"2024-09-19T08:47:13.386481+0000","last_active":"2024-09-19T08:47:15.387876+0000","last_peered":"2024-09-19T08:47:15.387876+0000","last_clean":"2024-09-19T08:47:15.387876+0000","last_became_active":"2024-09-19T08:47:13.385716+0000","last_became_peered":"2024-09-19T08:47:13.385716+0000","last_unstale":"2024-09-19T08:47:15.387876+0000","last_undegraded":"2024-09-19T08:47:15.387876+0000","last_fullsized":"2024-09-19T08:47:15.387876+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:22.519133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833065+0000","last_change":"2024-09-19T08:47:14.354191+0000","last_active":"2024-09-19T08:47:15.833065+0000","last_peered":"2024-09-19T08:47:15.833065+0000","last_clean":"2024-09-19T08:47:15.833065+0000","last_became_active":"2024-09-19T08:47:14.348271+0000","last_became_peered":"2024-09-19T08:47:14.348271+0000","last_unstale":"2024-09-19T08:47:15.833065+0000","last_undegraded":"2024-09-19T08:47:15.833065+0000","last_fullsized":"2024-09-19T08:47:15.833065+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:34:12.965823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"169'1","reported_seq":15,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.388912+0000","last_change":"2024-09-19T08:47:13.388601+0000","last_active":"2024-09-19T08:47:15.388912+0000","last_peered":"2024-09-19T08:47:15.388912+0000","last_clean":"2024-09-19T08:47:15.388912+0000","last_became_active":"2024-09-19T08:47:13.387981+0000","last_became_peered":"2024-09-19T08:47:13.387981+0000","last_unstale":"2024-09-19T08:47:15.388912+0000","last_undegraded":"2024-09-19T08:47:15.388912+0000","last_fullsized":"2024-09-19T08:47:15.388912+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:04.694583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"71'128","reported_seq":478,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.383426+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:47:15.383426+0000","last_peered":"2024-09-19T08:47:15.383426+0000","last_clean":"2024-09-19T08:47:15.383426+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:47:15.383426+0000","last_undegraded":"2024-09-19T08:47:15.383426+0000","last_fullsized":"2024-09-19T08:47:15.383426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509553,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33868,"kb_used_data":2680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337972,"statfs":{"total":96636764160,"available":96602083328,"internally_reserved":0,"allocated":2744320,"data_stored":1542832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574962,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33288,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2142208,"data_stored":951434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509555,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33280,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338560,"statfs":{"total":96636764160,"available":96602685440,"internally_reserved":0,"allocated":2142208,"data_stored":951434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509555,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33300,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338540,"statfs":{"total":96636764160,"available":96602664960,"internally_reserved":0,"allocated":2158592,"data_stored":952568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542257,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37448,"kb_used_data":2100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334392,"statfs":{"total":96636764160,"available":96598417408,"internally_reserved":0,"allocated":2150400,"data_stored":951034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509552,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33276,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338564,"statfs":{"total":96636764160,"available":96602689536,"internally_reserved":0,"allocated":2138112,"data_stored":950852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509554,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33288,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2146304,"data_stored":950908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509552,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38020,"kb_used_data":2672,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333820,"statfs":{"total":96636764160,"available":96597831680,"internally_reserved":0,"allocated":2736128,"data_stored":1541310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:47:19.038 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T08:47:19.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.182+0000 7fc21f569640 1 Processor -- start 2024-09-19T08:47:19.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.182+0000 7fc21f569640 1 -- start start 2024-09-19T08:47:19.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.182+0000 7fc21f569640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc21815f180 0x7fc21815f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:19.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.182+0000 7fc21f569640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc2180d35e0 0x7fc2180d3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:19.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.182+0000 7fc21f569640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2180d41a0 con 0x7fc2180d48b0 2024-09-19T08:47:19.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.182+0000 7fc21f569640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fc2180d3f40 con 0x7fc2180d35e0 2024-09-19T08:47:19.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.182+0000 7fc21f569640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc2180d0df0 con 0x7fc21815f180 2024-09-19T08:47:19.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.182+0000 7fc21db06640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fc2180d48b0 0x7fc218162080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:45994/0 (socket says 172.21.15.98:45994) 2024-09-19T08:47:19.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.182+0000 7fc21db06640 1 -- 172.21.15.98:0/3534632093 learned_addr learned my addr 172.21.15.98:0/3534632093 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:19.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.182+0000 7fc21cb04640 1 --2- 172.21.15.98:0/3534632093 >> v2:172.21.15.98:3300/0 conn(0x7fc2180d35e0 0x7fc2180d3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:19.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.182+0000 7fc21d305640 1 --2- 172.21.15.98:0/3534632093 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc21815f180 0x7fc21815f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:19.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.182+0000 7fc20ffff640 1 -- 172.21.15.98:0/3534632093 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 997366424 0 0) 0x7fc2180d41a0 con 0x7fc2180d48b0 2024-09-19T08:47:19.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.182+0000 7fc20ffff640 1 -- 172.21.15.98:0/3534632093 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc204003530 con 0x7fc2180d48b0 2024-09-19T08:47:19.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.182+0000 7fc21cb04640 1 -- 172.21.15.98:0/3534632093 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc21815f180 msgr2=0x7fc21815f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:19.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.182+0000 7fc21cb04640 1 --2- 172.21.15.98:0/3534632093 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc21815f180 0x7fc21815f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:19.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.182+0000 7fc21cb04640 1 -- 172.21.15.98:0/3534632093 >> v1:172.21.15.98:6789/0 conn(0x7fc2180d48b0 legacy=0x7fc218162080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:19.184 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.182+0000 7fc21cb04640 1 -- 172.21.15.98:0/3534632093 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2180d10d0 con 0x7fc2180d35e0 2024-09-19T08:47:19.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.186+0000 7fc21cb04640 1 --2- 172.21.15.98:0/3534632093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2180d35e0 0x7fc2180d3a00 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fc2080097c0 tx=0x7fc20801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf29ec1955f2b0dc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:19.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.186+0000 7fc20ffff640 1 -- 172.21.15.98:0/3534632093 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2080aa600 con 0x7fc2180d35e0 2024-09-19T08:47:19.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.186+0000 7fc20ffff640 1 -- 172.21.15.98:0/3534632093 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2080af070 con 0x7fc2180d35e0 2024-09-19T08:47:19.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.186+0000 7fc20ffff640 1 -- 172.21.15.98:0/3534632093 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2080aa9a0 con 0x7fc2180d35e0 2024-09-19T08:47:19.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.186+0000 7fc21f569640 1 -- 172.21.15.98:0/3534632093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2180d35e0 msgr2=0x7fc2180d3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:19.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.186+0000 7fc21f569640 1 --2- 172.21.15.98:0/3534632093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2180d35e0 0x7fc2180d3a00 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fc2080097c0 tx=0x7fc20801c4a0 comp rx=0 tx=0).stop 2024-09-19T08:47:19.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.186+0000 7fc21f569640 1 -- 172.21.15.98:0/3534632093 shutdown_connections 2024-09-19T08:47:19.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.186+0000 7fc21f569640 1 --2- 172.21.15.98:0/3534632093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2180d35e0 0x7fc2180d3a00 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:19.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.186+0000 7fc21f569640 1 --2- 172.21.15.98:0/3534632093 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc21815f180 0x7fc21815f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:19.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.186+0000 7fc21f569640 1 -- 172.21.15.98:0/3534632093 >> 172.21.15.98:0/3534632093 conn(0x7fc218074c70 msgr2=0x7fc218075080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:19.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.186+0000 7fc21f569640 1 -- 172.21.15.98:0/3534632093 shutdown_connections 2024-09-19T08:47:19.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.186+0000 7fc21f569640 1 -- 172.21.15.98:0/3534632093 wait complete. 2024-09-19T08:47:19.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.186+0000 7fc21f569640 1 Processor -- start 2024-09-19T08:47:19.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.186+0000 7fc21f569640 1 -- start start 2024-09-19T08:47:19.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.190+0000 7fc21f569640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2180d35e0 0x7fc21814d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:19.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.190+0000 7fc21f569640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc2180d48b0 0x7fc21814afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:19.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.190+0000 7fc21f569640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc21815f180 0x7fc218155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:19.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.190+0000 7fc21f569640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc2180d1d30 con 0x7fc2180d35e0 2024-09-19T08:47:19.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.190+0000 7fc21f569640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc2180d1bb0 con 0x7fc21815f180 2024-09-19T08:47:19.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.190+0000 7fc21f569640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc21816b1d0 con 0x7fc2180d48b0 2024-09-19T08:47:19.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.190+0000 7fc21d305640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2180d35e0 0x7fc21814d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:19.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.190+0000 7fc21d305640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2180d35e0 0x7fc21814d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41876/0 (socket says 172.21.15.98:41876) 2024-09-19T08:47:19.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.190+0000 7fc21d305640 1 -- 172.21.15.98:0/917677910 learned_addr learned my addr 172.21.15.98:0/917677910 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:19.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.190+0000 7fc21d305640 1 -- 172.21.15.98:0/917677910 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc2180d48b0 msgr2=0x7fc21814afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:19.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.190+0000 7fc21d305640 1 --2- 172.21.15.98:0/917677910 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc2180d48b0 0x7fc21814afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:19.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.190+0000 7fc21d305640 1 -- 172.21.15.98:0/917677910 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc21815f180 msgr2=0x7fc218155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:19.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.190+0000 7fc21d305640 1 --2- 172.21.15.98:0/917677910 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc21815f180 0x7fc218155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:19.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.190+0000 7fc21d305640 1 -- 172.21.15.98:0/917677910 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc21814b510 con 0x7fc2180d35e0 2024-09-19T08:47:19.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.190+0000 7fc21d305640 1 --2- 172.21.15.98:0/917677910 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2180d35e0 0x7fc21814d500 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fc21000d5f0 tx=0x7fc21000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:19.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.190+0000 7fc20dffb640 1 -- 172.21.15.98:0/917677910 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2100125f0 con 0x7fc2180d35e0 2024-09-19T08:47:19.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.190+0000 7fc20dffb640 1 -- 172.21.15.98:0/917677910 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc210019070 con 0x7fc2180d35e0 2024-09-19T08:47:19.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.190+0000 7fc21f569640 1 -- 172.21.15.98:0/917677910 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc21815a340 con 0x7fc2180d35e0 2024-09-19T08:47:19.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.190+0000 7fc20dffb640 1 -- 172.21.15.98:0/917677910 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2100129d0 con 0x7fc2180d35e0 2024-09-19T08:47:19.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.190+0000 7fc21f569640 1 -- 172.21.15.98:0/917677910 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc21814e620 con 0x7fc2180d35e0 2024-09-19T08:47:19.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.190+0000 7fc21f569640 1 -- 172.21.15.98:0/917677910 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2180d3a00 con 0x7fc2180d35e0 2024-09-19T08:47:19.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.190+0000 7fc20dffb640 1 -- 172.21.15.98:0/917677910 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7fc210012b70 con 0x7fc2180d35e0 2024-09-19T08:47:19.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.190+0000 7fc20dffb640 1 --2- 172.21.15.98:0/917677910 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc1fc068250 0x7fc1fc06a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:19.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.190+0000 7fc20dffb640 1 -- 172.21.15.98:0/917677910 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5737+0+0 (secure 0 0 0) 0x7fc2100848f0 con 0x7fc2180d35e0 2024-09-19T08:47:19.192 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.190+0000 7fc21cb04640 1 --2- 172.21.15.98:0/917677910 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc1fc068250 0x7fc1fc06a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:19.193 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.190+0000 7fc21cb04640 1 --2- 172.21.15.98:0/917677910 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc1fc068250 0x7fc1fc06a710 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc2080ae000 tx=0x7fc208004740 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:19.194 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.194+0000 7fc20dffb640 1 -- 172.21.15.98:0/917677910 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc210057030 con 0x7fc2180d35e0 2024-09-19T08:47:19.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.402+0000 7fc21f569640 1 -- 172.21.15.98:0/917677910 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fc218160f30 con 0x7fc2180d35e0 2024-09-19T08:47:19.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.402+0000 7fc20dffb640 1 -- 172.21.15.98:0/917677910 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1479 (secure 0 0 0) 0x7fc210056930 con 0x7fc2180d35e0 2024-09-19T08:47:19.401 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:47:19.401 INFO:teuthology.orchestra.run.smithi098.stdout:{"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":563,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":170,"num_osds":8,"num_up_osds":8,"osd_up_since":1726735088,"num_in_osds":8,"osd_in_since":1726735077,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":282386432,"bytes_avail":772811726848,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":100,"btime":"2024-09-19T08:47:17:337562+0000","id":13,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":13,"rank":0,"name":"a","status":"up:active","gid":9177}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-09-19T08:47:04.954826+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-19T08:47:19.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.402+0000 7fc21f569640 1 -- 172.21.15.98:0/917677910 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc1fc068250 msgr2=0x7fc1fc06a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:19.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.402+0000 7fc21f569640 1 --2- 172.21.15.98:0/917677910 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc1fc068250 0x7fc1fc06a710 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc2080ae000 tx=0x7fc208004740 comp rx=0 tx=0).stop 2024-09-19T08:47:19.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.402+0000 7fc21f569640 1 -- 172.21.15.98:0/917677910 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2180d35e0 msgr2=0x7fc21814d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:19.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.402+0000 7fc21f569640 1 --2- 172.21.15.98:0/917677910 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2180d35e0 0x7fc21814d500 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fc21000d5f0 tx=0x7fc21000dab0 comp rx=0 tx=0).stop 2024-09-19T08:47:19.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.402+0000 7fc21f569640 1 -- 172.21.15.98:0/917677910 shutdown_connections 2024-09-19T08:47:19.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.402+0000 7fc21f569640 1 --2- 172.21.15.98:0/917677910 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc1fc068250 0x7fc1fc06a710 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:19.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.402+0000 7fc21f569640 1 --2- 172.21.15.98:0/917677910 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc21815f180 0x7fc218155b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:19.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.402+0000 7fc21f569640 1 --2- 172.21.15.98:0/917677910 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc2180d48b0 0x7fc21814afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:19.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.402+0000 7fc21f569640 1 --2- 172.21.15.98:0/917677910 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc2180d35e0 0x7fc21814d500 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:19.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.402+0000 7fc21f569640 1 -- 172.21.15.98:0/917677910 >> 172.21.15.98:0/917677910 conn(0x7fc218074c70 msgr2=0x7fc21814bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:19.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.402+0000 7fc21f569640 1 -- 172.21.15.98:0/917677910 shutdown_connections 2024-09-19T08:47:19.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.402+0000 7fc21f569640 1 -- 172.21.15.98:0/917677910 wait complete. 2024-09-19T08:47:19.419 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-19T08:47:19.419 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:47:19.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.534+0000 7fd1fec3a640 1 Processor -- start 2024-09-19T08:47:19.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.534+0000 7fd1fec3a640 1 -- start start 2024-09-19T08:47:19.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.534+0000 7fd1fec3a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1f8158e40 0x7fd1f806bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:19.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.534+0000 7fd1fec3a640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1f815fcf0 con 0x7fd1f806c1d0 2024-09-19T08:47:19.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.534+0000 7fd1fec3a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1f815ef30 con 0x7fd1f80ce8b0 2024-09-19T08:47:19.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.534+0000 7fd1fec3a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd1f806c610 con 0x7fd1f8158e40 2024-09-19T08:47:19.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.534+0000 7fd1f7fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fd1f806c1d0 0x7fd1f80ce1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:46010/0 (socket says 172.21.15.98:46010) 2024-09-19T08:47:19.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.534+0000 7fd1f7fff640 1 -- 172.21.15.98:0/238418384 learned_addr learned my addr 172.21.15.98:0/238418384 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:19.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.534+0000 7fd1fc9d6640 1 --2- 172.21.15.98:0/238418384 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1f8158e40 0x7fd1f806bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:19.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.534+0000 7fd1f77fe640 1 -- 172.21.15.98:0/238418384 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 99688483 0 0) 0x7fd1f815fcf0 con 0x7fd1f806c1d0 2024-09-19T08:47:19.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.534+0000 7fd1f77fe640 1 -- 172.21.15.98:0/238418384 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd1e8003530 con 0x7fd1f806c1d0 2024-09-19T08:47:19.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.534+0000 7fd1f77fe640 1 -- 172.21.15.98:0/238418384 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 739392759 0 0) 0x7fd1e8003530 con 0x7fd1f806c1d0 2024-09-19T08:47:19.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.534+0000 7fd1f77fe640 1 -- 172.21.15.98:0/238418384 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1f8158e40 msgr2=0x7fd1f806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:19.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.534+0000 7fd1f77fe640 1 --2- 172.21.15.98:0/238418384 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1f8158e40 0x7fd1f806bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:19.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.534+0000 7fd1f77fe640 1 -- 172.21.15.98:0/238418384 >> v1:172.21.15.139:6789/0 conn(0x7fd1f80ce8b0 legacy=0x7fd1f80b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:19.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.534+0000 7fd1f77fe640 1 -- 172.21.15.98:0/238418384 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f815f210 con 0x7fd1f806c1d0 2024-09-19T08:47:19.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.534+0000 7fd1f77fe640 1 -- 172.21.15.98:0/238418384 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7fd1ec001680 con 0x7fd1f806c1d0 2024-09-19T08:47:19.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.534+0000 7fd1f77fe640 1 -- 172.21.15.98:0/238418384 >> v1:172.21.15.98:6789/0 conn(0x7fd1f806c1d0 legacy=0x7fd1f80ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:19.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.534+0000 7fd1f77fe640 1 --2- 172.21.15.98:0/238418384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1e80039f0 0x7fd1e8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:19.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.534+0000 7fd1f77fe640 1 --2- 172.21.15.98:0/238418384 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1e8004810 0x7fd1e8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:19.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.534+0000 7fd1f77fe640 1 --2- 172.21.15.98:0/238418384 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1f8158e40 0x7fd1e80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:19.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.534+0000 7fd1f77fe640 1 -- 172.21.15.98:0/238418384 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f806c610 con 0x7fd1e80039f0 2024-09-19T08:47:19.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.534+0000 7fd1f77fe640 1 -- 172.21.15.98:0/238418384 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e800bb10 con 0x7fd1e8004810 2024-09-19T08:47:19.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.534+0000 7fd1f77fe640 1 -- 172.21.15.98:0/238418384 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd1e800bd10 con 0x7fd1f8158e40 2024-09-19T08:47:19.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.534+0000 7fd1fc9d6640 1 --2- 172.21.15.98:0/238418384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1e80039f0 0x7fd1e8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:19.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.534+0000 7fd1fc9d6640 1 -- 172.21.15.98:0/238418384 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1f8158e40 msgr2=0x7fd1e80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:19.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.534+0000 7fd1fc9d6640 1 --2- 172.21.15.98:0/238418384 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1f8158e40 0x7fd1e80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:19.536 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.534+0000 7fd1f7fff640 1 --2- 172.21.15.98:0/238418384 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1e8004810 0x7fd1e8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:19.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.534+0000 7fd1fc9d6640 1 -- 172.21.15.98:0/238418384 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1e8004810 msgr2=0x7fd1e8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:19.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.534+0000 7fd1fc9d6640 1 --2- 172.21.15.98:0/238418384 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1e8004810 0x7fd1e8004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:19.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.534+0000 7fd1fc9d6640 1 -- 172.21.15.98:0/238418384 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e8003530 con 0x7fd1e80039f0 2024-09-19T08:47:19.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.538+0000 7fd1fc9d6640 1 --2- 172.21.15.98:0/238418384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1e80039f0 0x7fd1e8003e10 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4009ab0 tx=0x7fd1e401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71505eb6e1b4c260 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:19.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.538+0000 7fd1f77fe640 1 -- 172.21.15.98:0/238418384 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e40b2070 con 0x7fd1e80039f0 2024-09-19T08:47:19.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.538+0000 7fd1f77fe640 1 -- 172.21.15.98:0/238418384 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1e40a5660 con 0x7fd1e80039f0 2024-09-19T08:47:19.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.538+0000 7fd1f77fe640 1 -- 172.21.15.98:0/238418384 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e40096b0 con 0x7fd1e80039f0 2024-09-19T08:47:19.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.538+0000 7fd1fec3a640 1 -- 172.21.15.98:0/238418384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1e80039f0 msgr2=0x7fd1e8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:19.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.538+0000 7fd1fec3a640 1 --2- 172.21.15.98:0/238418384 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1e80039f0 0x7fd1e8003e10 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4009ab0 tx=0x7fd1e401ca70 comp rx=0 tx=0).stop 2024-09-19T08:47:19.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.538+0000 7fd1fd1d7640 1 -- 172.21.15.98:0/238418384 reap_dead start 2024-09-19T08:47:19.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.538+0000 7fd1fec3a640 1 -- 172.21.15.98:0/238418384 shutdown_connections 2024-09-19T08:47:19.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.538+0000 7fd1fec3a640 1 -- 172.21.15.98:0/238418384 >> 172.21.15.98:0/238418384 conn(0x7fd1f80b2010 msgr2=0x7fd1f80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:19.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.538+0000 7fd1fec3a640 1 -- 172.21.15.98:0/238418384 shutdown_connections 2024-09-19T08:47:19.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.538+0000 7fd1fec3a640 1 -- 172.21.15.98:0/238418384 wait complete. 2024-09-19T08:47:19.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.538+0000 7fd1fec3a640 1 Processor -- start 2024-09-19T08:47:19.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.538+0000 7fd1fec3a640 1 -- start start 2024-09-19T08:47:19.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.538+0000 7fd1fec3a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1e80039f0 0x7fd1f814dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:19.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.538+0000 7fd1fec3a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1f814e0e0 0x7fd1f8159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:19.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.538+0000 7fd1fec3a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1f815c320 0x7fd1f8159970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:19.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.538+0000 7fd1fec3a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f8159220 con 0x7fd1f814e0e0 2024-09-19T08:47:19.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.538+0000 7fd1fec3a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f815f7d0 con 0x7fd1f815c320 2024-09-19T08:47:19.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.538+0000 7fd1fec3a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fd1f80cee20 con 0x7fd1e80039f0 2024-09-19T08:47:19.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.538+0000 7fd1f7fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1f814e0e0 0x7fd1f8159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:19.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.538+0000 7fd1f7fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1f814e0e0 0x7fd1f8159430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41902/0 (socket says 172.21.15.98:41902) 2024-09-19T08:47:19.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.538+0000 7fd1f7fff640 1 -- 172.21.15.98:0/2847364078 learned_addr learned my addr 172.21.15.98:0/2847364078 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:19.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.538+0000 7fd1fc9d6640 1 --2- 172.21.15.98:0/2847364078 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1e80039f0 0x7fd1f814dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:19.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.538+0000 7fd1f7fff640 1 -- 172.21.15.98:0/2847364078 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1e80039f0 msgr2=0x7fd1f814dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:19.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.538+0000 7fd1f7fff640 1 --2- 172.21.15.98:0/2847364078 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1e80039f0 0x7fd1f814dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:19.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.538+0000 7fd1fd1d7640 1 --2- 172.21.15.98:0/2847364078 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1f815c320 0x7fd1f8159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:19.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.538+0000 7fd1f7fff640 1 -- 172.21.15.98:0/2847364078 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1f815c320 msgr2=0x7fd1f8159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:19.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.538+0000 7fd1f7fff640 1 --2- 172.21.15.98:0/2847364078 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1f815c320 0x7fd1f8159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:19.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.538+0000 7fd1f7fff640 1 -- 172.21.15.98:0/2847364078 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f814e560 con 0x7fd1f814e0e0 2024-09-19T08:47:19.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.542+0000 7fd1f7fff640 1 --2- 172.21.15.98:0/2847364078 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1f814e0e0 0x7fd1f8159430 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec010b40 tx=0x7fd1ec005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:19.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.542+0000 7fd1f57fa640 1 -- 172.21.15.98:0/2847364078 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec01b070 con 0x7fd1f814e0e0 2024-09-19T08:47:19.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.542+0000 7fd1f57fa640 1 -- 172.21.15.98:0/2847364078 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1ec011040 con 0x7fd1f814e0e0 2024-09-19T08:47:19.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.542+0000 7fd1fec3a640 1 -- 172.21.15.98:0/2847364078 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1f814e840 con 0x7fd1f814e0e0 2024-09-19T08:47:19.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.542+0000 7fd1f57fa640 1 -- 172.21.15.98:0/2847364078 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec00ede0 con 0x7fd1f814e0e0 2024-09-19T08:47:19.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.542+0000 7fd1fec3a640 1 -- 172.21.15.98:0/2847364078 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1f8071dc0 con 0x7fd1f814e0e0 2024-09-19T08:47:19.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.542+0000 7fd1fec3a640 1 -- 172.21.15.98:0/2847364078 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1f806bc90 con 0x7fd1f814e0e0 2024-09-19T08:47:19.543 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.542+0000 7fd1f57fa640 1 -- 172.21.15.98:0/2847364078 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7fd1ec01e070 con 0x7fd1f814e0e0 2024-09-19T08:47:19.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.542+0000 7fd1f57fa640 1 --2- 172.21.15.98:0/2847364078 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd1dc068250 0x7fd1dc06a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:19.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.542+0000 7fd1f57fa640 1 -- 172.21.15.98:0/2847364078 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5737+0+0 (secure 0 0 0) 0x7fd1ec086ec0 con 0x7fd1f814e0e0 2024-09-19T08:47:19.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.542+0000 7fd1fc9d6640 1 --2- 172.21.15.98:0/2847364078 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd1dc068250 0x7fd1dc06a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:19.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.542+0000 7fd1fc9d6640 1 --2- 172.21.15.98:0/2847364078 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd1dc068250 0x7fd1dc06a710 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd1e40af000 tx=0x7fd1e40046d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:19.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.546+0000 7fd1f57fa640 1 -- 172.21.15.98:0/2847364078 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd1ec059600 con 0x7fd1f814e0e0 2024-09-19T08:47:19.701 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.702+0000 7fd1fec3a640 1 -- 172.21.15.98:0/2847364078 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd1f80b1b70 con 0x7fd1dc068250 2024-09-19T08:47:19.719 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.718+0000 7fd1f57fa640 1 -- 172.21.15.98:0/2847364078 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252496 (secure 0 0 0) 0x7fd1f80b1b70 con 0x7fd1dc068250 2024-09-19T08:47:19.721 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:47:19.723 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:47:19.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.722+0000 7fd1fec3a640 1 -- 172.21.15.98:0/2847364078 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd1dc068250 msgr2=0x7fd1dc06a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:19.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.722+0000 7fd1fec3a640 1 --2- 172.21.15.98:0/2847364078 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd1dc068250 0x7fd1dc06a710 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd1e40af000 tx=0x7fd1e40046d0 comp rx=0 tx=0).stop 2024-09-19T08:47:19.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.722+0000 7fd1fec3a640 1 -- 172.21.15.98:0/2847364078 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1f814e0e0 msgr2=0x7fd1f8159430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:19.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.722+0000 7fd1fec3a640 1 --2- 172.21.15.98:0/2847364078 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1f814e0e0 0x7fd1f8159430 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec010b40 tx=0x7fd1ec005780 comp rx=0 tx=0).stop 2024-09-19T08:47:19.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.722+0000 7fd1fec3a640 1 -- 172.21.15.98:0/2847364078 shutdown_connections 2024-09-19T08:47:19.724 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.722+0000 7fd1fec3a640 1 --2- 172.21.15.98:0/2847364078 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fd1dc068250 0x7fd1dc06a710 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:19.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.722+0000 7fd1fec3a640 1 --2- 172.21.15.98:0/2847364078 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd1f815c320 0x7fd1f8159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:19.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.722+0000 7fd1fec3a640 1 --2- 172.21.15.98:0/2847364078 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fd1f814e0e0 0x7fd1f8159430 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:19.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.722+0000 7fd1fec3a640 1 --2- 172.21.15.98:0/2847364078 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fd1e80039f0 0x7fd1f814dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:19.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.722+0000 7fd1fec3a640 1 -- 172.21.15.98:0/2847364078 >> 172.21.15.98:0/2847364078 conn(0x7fd1f80b2010 msgr2=0x7fd1f80b3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:19.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.722+0000 7fd1fec3a640 1 -- 172.21.15.98:0/2847364078 shutdown_connections 2024-09-19T08:47:19.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:19.722+0000 7fd1fec3a640 1 -- 172.21.15.98:0/2847364078 wait complete. 2024-09-19T08:47:19.739 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2024-09-19T08:47:18.958819+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":275768,"kb_used_data":17928,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754698952,"statfs":{"total":773094113280,"available":772811726848,"internally_reserved":0,"allocated":18358272,"data_stored":8792372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":58,"apply_latency_ms":58,"commit_latency_ns":58000000,"apply_latency_ns":58000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001878"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833628+0000","last_change":"2024-09-19T08:47:14.359460+0000","last_active":"2024-09-19T08:47:15.833628+0000","last_peered":"2024-09-19T08:47:15.833628+0000","last_clean":"2024-09-19T08:47:15.833628+0000","last_became_active":"2024-09-19T08:47:14.352467+0000","last_became_peered":"2024-09-19T08:47:14.352467+0000","last_unstale":"2024-09-19T08:47:15.833628+0000","last_undegraded":"2024-09-19T08:47:15.833628+0000","last_fullsized":"2024-09-19T08:47:15.833628+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:12.331382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.834037+0000","last_change":"2024-09-19T08:47:14.363174+0000","last_active":"2024-09-19T08:47:15.834037+0000","last_peered":"2024-09-19T08:47:15.834037+0000","last_clean":"2024-09-19T08:47:15.834037+0000","last_became_active":"2024-09-19T08:47:14.362109+0000","last_became_peered":"2024-09-19T08:47:14.362109+0000","last_unstale":"2024-09-19T08:47:15.834037+0000","last_undegraded":"2024-09-19T08:47:15.834037+0000","last_fullsized":"2024-09-19T08:47:15.834037+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:34.366623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.701137+0000","last_change":"2024-09-19T08:47:14.365200+0000","last_active":"2024-09-19T08:47:15.701137+0000","last_peered":"2024-09-19T08:47:15.701137+0000","last_clean":"2024-09-19T08:47:15.701137+0000","last_became_active":"2024-09-19T08:47:14.357837+0000","last_became_peered":"2024-09-19T08:47:14.357837+0000","last_unstale":"2024-09-19T08:47:15.701137+0000","last_undegraded":"2024-09-19T08:47:15.701137+0000","last_fullsized":"2024-09-19T08:47:15.701137+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:54.142030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.383795+0000","last_change":"2024-09-19T08:47:14.354043+0000","last_active":"2024-09-19T08:47:15.383795+0000","last_peered":"2024-09-19T08:47:15.383795+0000","last_clean":"2024-09-19T08:47:15.383795+0000","last_became_active":"2024-09-19T08:47:14.352832+0000","last_became_peered":"2024-09-19T08:47:14.352832+0000","last_unstale":"2024-09-19T08:47:15.383795+0000","last_undegraded":"2024-09-19T08:47:15.383795+0000","last_fullsized":"2024-09-19T08:47:15.383795+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:38:10.816584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.315069+0000","last_change":"2024-09-19T08:47:14.342189+0000","last_active":"2024-09-19T08:47:15.315069+0000","last_peered":"2024-09-19T08:47:15.315069+0000","last_clean":"2024-09-19T08:47:15.315069+0000","last_became_active":"2024-09-19T08:47:14.340626+0000","last_became_peered":"2024-09-19T08:47:14.340626+0000","last_unstale":"2024-09-19T08:47:15.315069+0000","last_undegraded":"2024-09-19T08:47:15.315069+0000","last_fullsized":"2024-09-19T08:47:15.315069+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:42.709485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.314735+0000","last_change":"2024-09-19T08:47:14.361414+0000","last_active":"2024-09-19T08:47:15.314735+0000","last_peered":"2024-09-19T08:47:15.314735+0000","last_clean":"2024-09-19T08:47:15.314735+0000","last_became_active":"2024-09-19T08:47:14.360629+0000","last_became_peered":"2024-09-19T08:47:14.360629+0000","last_unstale":"2024-09-19T08:47:15.314735+0000","last_undegraded":"2024-09-19T08:47:15.314735+0000","last_fullsized":"2024-09-19T08:47:15.314735+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:33:28.816440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.832761+0000","last_change":"2024-09-19T08:47:14.360193+0000","last_active":"2024-09-19T08:47:15.832761+0000","last_peered":"2024-09-19T08:47:15.832761+0000","last_clean":"2024-09-19T08:47:15.832761+0000","last_became_active":"2024-09-19T08:47:14.356098+0000","last_became_peered":"2024-09-19T08:47:14.356098+0000","last_unstale":"2024-09-19T08:47:15.832761+0000","last_undegraded":"2024-09-19T08:47:15.832761+0000","last_fullsized":"2024-09-19T08:47:15.832761+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:53:48.646448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.314425+0000","last_change":"2024-09-19T08:47:14.351408+0000","last_active":"2024-09-19T08:47:15.314425+0000","last_peered":"2024-09-19T08:47:15.314425+0000","last_clean":"2024-09-19T08:47:15.314425+0000","last_became_active":"2024-09-19T08:47:14.350440+0000","last_became_peered":"2024-09-19T08:47:14.350440+0000","last_unstale":"2024-09-19T08:47:15.314425+0000","last_undegraded":"2024-09-19T08:47:15.314425+0000","last_fullsized":"2024-09-19T08:47:15.314425+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:20:48.189924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.834432+0000","last_change":"2024-09-19T08:47:14.357553+0000","last_active":"2024-09-19T08:47:15.834432+0000","last_peered":"2024-09-19T08:47:15.834432+0000","last_clean":"2024-09-19T08:47:15.834432+0000","last_became_active":"2024-09-19T08:47:14.355255+0000","last_became_peered":"2024-09-19T08:47:14.355255+0000","last_unstale":"2024-09-19T08:47:15.834432+0000","last_undegraded":"2024-09-19T08:47:15.834432+0000","last_fullsized":"2024-09-19T08:47:15.834432+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:55:39.012179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.831464+0000","last_change":"2024-09-19T08:47:14.350592+0000","last_active":"2024-09-19T08:47:15.831464+0000","last_peered":"2024-09-19T08:47:15.831464+0000","last_clean":"2024-09-19T08:47:15.831464+0000","last_became_active":"2024-09-19T08:47:14.349843+0000","last_became_peered":"2024-09-19T08:47:14.349843+0000","last_unstale":"2024-09-19T08:47:15.831464+0000","last_undegraded":"2024-09-19T08:47:15.831464+0000","last_fullsized":"2024-09-19T08:47:15.831464+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:36.184294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.701152+0000","last_change":"2024-09-19T08:47:14.347087+0000","last_active":"2024-09-19T08:47:15.701152+0000","last_peered":"2024-09-19T08:47:15.701152+0000","last_clean":"2024-09-19T08:47:15.701152+0000","last_became_active":"2024-09-19T08:47:14.346271+0000","last_became_peered":"2024-09-19T08:47:14.346271+0000","last_unstale":"2024-09-19T08:47:15.701152+0000","last_undegraded":"2024-09-19T08:47:15.701152+0000","last_fullsized":"2024-09-19T08:47:15.701152+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:23.614854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.388582+0000","last_change":"2024-09-19T08:47:14.344723+0000","last_active":"2024-09-19T08:47:15.388582+0000","last_peered":"2024-09-19T08:47:15.388582+0000","last_clean":"2024-09-19T08:47:15.388582+0000","last_became_active":"2024-09-19T08:47:14.343948+0000","last_became_peered":"2024-09-19T08:47:14.343948+0000","last_unstale":"2024-09-19T08:47:15.388582+0000","last_undegraded":"2024-09-19T08:47:15.388582+0000","last_fullsized":"2024-09-19T08:47:15.388582+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:16.685634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.388746+0000","last_change":"2024-09-19T08:47:14.352237+0000","last_active":"2024-09-19T08:47:15.388746+0000","last_peered":"2024-09-19T08:47:15.388746+0000","last_clean":"2024-09-19T08:47:15.388746+0000","last_became_active":"2024-09-19T08:47:14.351466+0000","last_became_peered":"2024-09-19T08:47:14.351466+0000","last_unstale":"2024-09-19T08:47:15.388746+0000","last_undegraded":"2024-09-19T08:47:15.388746+0000","last_fullsized":"2024-09-19T08:47:15.388746+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:44:09.637091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.832981+0000","last_change":"2024-09-19T08:47:14.352279+0000","last_active":"2024-09-19T08:47:15.832981+0000","last_peered":"2024-09-19T08:47:15.832981+0000","last_clean":"2024-09-19T08:47:15.832981+0000","last_became_active":"2024-09-19T08:47:14.348493+0000","last_became_peered":"2024-09-19T08:47:14.348493+0000","last_unstale":"2024-09-19T08:47:15.832981+0000","last_undegraded":"2024-09-19T08:47:15.832981+0000","last_fullsized":"2024-09-19T08:47:15.832981+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:57:37.877782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.832961+0000","last_change":"2024-09-19T08:47:14.360081+0000","last_active":"2024-09-19T08:47:15.832961+0000","last_peered":"2024-09-19T08:47:15.832961+0000","last_clean":"2024-09-19T08:47:15.832961+0000","last_became_active":"2024-09-19T08:47:14.355887+0000","last_became_peered":"2024-09-19T08:47:14.355887+0000","last_unstale":"2024-09-19T08:47:15.832961+0000","last_undegraded":"2024-09-19T08:47:15.832961+0000","last_fullsized":"2024-09-19T08:47:15.832961+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:47.995264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.830850+0000","last_change":"2024-09-19T08:47:14.337255+0000","last_active":"2024-09-19T08:47:15.830850+0000","last_peered":"2024-09-19T08:47:15.830850+0000","last_clean":"2024-09-19T08:47:15.830850+0000","last_became_active":"2024-09-19T08:47:14.336468+0000","last_became_peered":"2024-09-19T08:47:14.336468+0000","last_unstale":"2024-09-19T08:47:15.830850+0000","last_undegraded":"2024-09-19T08:47:15.830850+0000","last_fullsized":"2024-09-19T08:47:15.830850+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:12:56.723940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.834027+0000","last_change":"2024-09-19T08:47:14.353222+0000","last_active":"2024-09-19T08:47:15.834027+0000","last_peered":"2024-09-19T08:47:15.834027+0000","last_clean":"2024-09-19T08:47:15.834027+0000","last_became_active":"2024-09-19T08:47:14.351562+0000","last_became_peered":"2024-09-19T08:47:14.351562+0000","last_unstale":"2024-09-19T08:47:15.834027+0000","last_undegraded":"2024-09-19T08:47:15.834027+0000","last_fullsized":"2024-09-19T08:47:15.834027+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:58:45.045360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.831561+0000","last_change":"2024-09-19T08:47:14.340655+0000","last_active":"2024-09-19T08:47:15.831561+0000","last_peered":"2024-09-19T08:47:15.831561+0000","last_clean":"2024-09-19T08:47:15.831561+0000","last_became_active":"2024-09-19T08:47:14.339893+0000","last_became_peered":"2024-09-19T08:47:14.339893+0000","last_unstale":"2024-09-19T08:47:15.831561+0000","last_undegraded":"2024-09-19T08:47:15.831561+0000","last_fullsized":"2024-09-19T08:47:15.831561+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:56.648047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.831544+0000","last_change":"2024-09-19T08:47:14.350896+0000","last_active":"2024-09-19T08:47:15.831544+0000","last_peered":"2024-09-19T08:47:15.831544+0000","last_clean":"2024-09-19T08:47:15.831544+0000","last_became_active":"2024-09-19T08:47:14.349975+0000","last_became_peered":"2024-09-19T08:47:14.349975+0000","last_unstale":"2024-09-19T08:47:15.831544+0000","last_undegraded":"2024-09-19T08:47:15.831544+0000","last_fullsized":"2024-09-19T08:47:15.831544+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:50.758257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.315869+0000","last_change":"2024-09-19T08:47:14.338182+0000","last_active":"2024-09-19T08:47:15.315869+0000","last_peered":"2024-09-19T08:47:15.315869+0000","last_clean":"2024-09-19T08:47:15.315869+0000","last_became_active":"2024-09-19T08:47:14.336028+0000","last_became_peered":"2024-09-19T08:47:14.336028+0000","last_unstale":"2024-09-19T08:47:15.315869+0000","last_undegraded":"2024-09-19T08:47:15.315869+0000","last_fullsized":"2024-09-19T08:47:15.315869+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:18:40.140297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.383323+0000","last_change":"2024-09-19T08:47:14.348401+0000","last_active":"2024-09-19T08:47:15.383323+0000","last_peered":"2024-09-19T08:47:15.383323+0000","last_clean":"2024-09-19T08:47:15.383323+0000","last_became_active":"2024-09-19T08:47:14.346885+0000","last_became_peered":"2024-09-19T08:47:14.346885+0000","last_unstale":"2024-09-19T08:47:15.383323+0000","last_undegraded":"2024-09-19T08:47:15.383323+0000","last_fullsized":"2024-09-19T08:47:15.383323+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:23:37.200270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.318153+0000","last_change":"2024-09-19T08:47:14.338348+0000","last_active":"2024-09-19T08:47:15.318153+0000","last_peered":"2024-09-19T08:47:15.318153+0000","last_clean":"2024-09-19T08:47:15.318153+0000","last_became_active":"2024-09-19T08:47:14.337676+0000","last_became_peered":"2024-09-19T08:47:14.337676+0000","last_unstale":"2024-09-19T08:47:15.318153+0000","last_undegraded":"2024-09-19T08:47:15.318153+0000","last_fullsized":"2024-09-19T08:47:15.318153+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:53:47.277291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.318101+0000","last_change":"2024-09-19T08:47:14.337863+0000","last_active":"2024-09-19T08:47:15.318101+0000","last_peered":"2024-09-19T08:47:15.318101+0000","last_clean":"2024-09-19T08:47:15.318101+0000","last_became_active":"2024-09-19T08:47:14.337170+0000","last_became_peered":"2024-09-19T08:47:14.337170+0000","last_unstale":"2024-09-19T08:47:15.318101+0000","last_undegraded":"2024-09-19T08:47:15.318101+0000","last_fullsized":"2024-09-19T08:47:15.318101+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:41:06.545819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.383167+0000","last_change":"2024-09-19T08:47:14.349384+0000","last_active":"2024-09-19T08:47:15.383167+0000","last_peered":"2024-09-19T08:47:15.383167+0000","last_clean":"2024-09-19T08:47:15.383167+0000","last_became_active":"2024-09-19T08:47:14.346975+0000","last_became_peered":"2024-09-19T08:47:14.346975+0000","last_unstale":"2024-09-19T08:47:15.383167+0000","last_undegraded":"2024-09-19T08:47:15.383167+0000","last_fullsized":"2024-09-19T08:47:15.383167+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:15.674519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.834210+0000","last_change":"2024-09-19T08:47:14.358211+0000","last_active":"2024-09-19T08:47:15.834210+0000","last_peered":"2024-09-19T08:47:15.834210+0000","last_clean":"2024-09-19T08:47:15.834210+0000","last_became_active":"2024-09-19T08:47:14.356031+0000","last_became_peered":"2024-09-19T08:47:14.356031+0000","last_unstale":"2024-09-19T08:47:15.834210+0000","last_undegraded":"2024-09-19T08:47:15.834210+0000","last_fullsized":"2024-09-19T08:47:15.834210+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:14:56.115719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833588+0000","last_change":"2024-09-19T08:47:14.351542+0000","last_active":"2024-09-19T08:47:15.833588+0000","last_peered":"2024-09-19T08:47:15.833588+0000","last_clean":"2024-09-19T08:47:15.833588+0000","last_became_active":"2024-09-19T08:47:14.348510+0000","last_became_peered":"2024-09-19T08:47:14.348510+0000","last_unstale":"2024-09-19T08:47:15.833588+0000","last_undegraded":"2024-09-19T08:47:15.833588+0000","last_fullsized":"2024-09-19T08:47:15.833588+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:06:30.241912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.388376+0000","last_change":"2024-09-19T08:47:14.343722+0000","last_active":"2024-09-19T08:47:15.388376+0000","last_peered":"2024-09-19T08:47:15.388376+0000","last_clean":"2024-09-19T08:47:15.388376+0000","last_became_active":"2024-09-19T08:47:14.342950+0000","last_became_peered":"2024-09-19T08:47:14.342950+0000","last_unstale":"2024-09-19T08:47:15.388376+0000","last_undegraded":"2024-09-19T08:47:15.388376+0000","last_fullsized":"2024-09-19T08:47:15.388376+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:52.353505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.700591+0000","last_change":"2024-09-19T08:47:14.354023+0000","last_active":"2024-09-19T08:47:15.700591+0000","last_peered":"2024-09-19T08:47:15.700591+0000","last_clean":"2024-09-19T08:47:15.700591+0000","last_became_active":"2024-09-19T08:47:14.347846+0000","last_became_peered":"2024-09-19T08:47:14.347846+0000","last_unstale":"2024-09-19T08:47:15.700591+0000","last_undegraded":"2024-09-19T08:47:15.700591+0000","last_fullsized":"2024-09-19T08:47:15.700591+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:29:56.943652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.388148+0000","last_change":"2024-09-19T08:47:14.343483+0000","last_active":"2024-09-19T08:47:15.388148+0000","last_peered":"2024-09-19T08:47:15.388148+0000","last_clean":"2024-09-19T08:47:15.388148+0000","last_became_active":"2024-09-19T08:47:14.342737+0000","last_became_peered":"2024-09-19T08:47:14.342737+0000","last_unstale":"2024-09-19T08:47:15.388148+0000","last_undegraded":"2024-09-19T08:47:15.388148+0000","last_fullsized":"2024-09-19T08:47:15.388148+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:23:35.024819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833688+0000","last_change":"2024-09-19T08:47:14.354661+0000","last_active":"2024-09-19T08:47:15.833688+0000","last_peered":"2024-09-19T08:47:15.833688+0000","last_clean":"2024-09-19T08:47:15.833688+0000","last_became_active":"2024-09-19T08:47:14.353824+0000","last_became_peered":"2024-09-19T08:47:14.353824+0000","last_unstale":"2024-09-19T08:47:15.833688+0000","last_undegraded":"2024-09-19T08:47:15.833688+0000","last_fullsized":"2024-09-19T08:47:15.833688+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:15:39.112225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.830773+0000","last_change":"2024-09-19T08:47:14.346187+0000","last_active":"2024-09-19T08:47:15.830773+0000","last_peered":"2024-09-19T08:47:15.830773+0000","last_clean":"2024-09-19T08:47:15.830773+0000","last_became_active":"2024-09-19T08:47:14.339726+0000","last_became_peered":"2024-09-19T08:47:14.339726+0000","last_unstale":"2024-09-19T08:47:15.830773+0000","last_undegraded":"2024-09-19T08:47:15.830773+0000","last_fullsized":"2024-09-19T08:47:15.830773+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:01.753982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833104+0000","last_change":"2024-09-19T08:47:14.352760+0000","last_active":"2024-09-19T08:47:15.833104+0000","last_peered":"2024-09-19T08:47:15.833104+0000","last_clean":"2024-09-19T08:47:15.833104+0000","last_became_active":"2024-09-19T08:47:14.352060+0000","last_became_peered":"2024-09-19T08:47:14.352060+0000","last_unstale":"2024-09-19T08:47:15.833104+0000","last_undegraded":"2024-09-19T08:47:15.833104+0000","last_fullsized":"2024-09-19T08:47:15.833104+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:35:03.456650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.834160+0000","last_change":"2024-09-19T08:47:14.362807+0000","last_active":"2024-09-19T08:47:15.834160+0000","last_peered":"2024-09-19T08:47:15.834160+0000","last_clean":"2024-09-19T08:47:15.834160+0000","last_became_active":"2024-09-19T08:47:14.361708+0000","last_became_peered":"2024-09-19T08:47:14.361708+0000","last_unstale":"2024-09-19T08:47:15.834160+0000","last_undegraded":"2024-09-19T08:47:15.834160+0000","last_fullsized":"2024-09-19T08:47:15.834160+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:58:18.390110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"169'4","reported_seq":18,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.388872+0000","last_change":"2024-09-19T08:47:13.401720+0000","last_active":"2024-09-19T08:47:15.388872+0000","last_peered":"2024-09-19T08:47:15.388872+0000","last_clean":"2024-09-19T08:47:15.388872+0000","last_became_active":"2024-09-19T08:47:13.400995+0000","last_became_peered":"2024-09-19T08:47:13.400995+0000","last_unstale":"2024-09-19T08:47:15.388872+0000","last_undegraded":"2024-09-19T08:47:15.388872+0000","last_fullsized":"2024-09-19T08:47:15.388872+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:55.697252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.319123+0000","last_change":"2024-09-19T08:47:14.334414+0000","last_active":"2024-09-19T08:47:15.319123+0000","last_peered":"2024-09-19T08:47:15.319123+0000","last_clean":"2024-09-19T08:47:15.319123+0000","last_became_active":"2024-09-19T08:47:14.333618+0000","last_became_peered":"2024-09-19T08:47:14.333618+0000","last_unstale":"2024-09-19T08:47:15.319123+0000","last_undegraded":"2024-09-19T08:47:15.319123+0000","last_fullsized":"2024-09-19T08:47:15.319123+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:10:52.949042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"169'1","reported_seq":15,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.834325+0000","last_change":"2024-09-19T08:47:13.411165+0000","last_active":"2024-09-19T08:47:15.834325+0000","last_peered":"2024-09-19T08:47:15.834325+0000","last_clean":"2024-09-19T08:47:15.834325+0000","last_became_active":"2024-09-19T08:47:13.410431+0000","last_became_peered":"2024-09-19T08:47:13.410431+0000","last_unstale":"2024-09-19T08:47:15.834325+0000","last_undegraded":"2024-09-19T08:47:15.834325+0000","last_fullsized":"2024-09-19T08:47:15.834325+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:04.142471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.834365+0000","last_change":"2024-09-19T08:47:14.347732+0000","last_active":"2024-09-19T08:47:15.834365+0000","last_peered":"2024-09-19T08:47:15.834365+0000","last_clean":"2024-09-19T08:47:15.834365+0000","last_became_active":"2024-09-19T08:47:14.346863+0000","last_became_peered":"2024-09-19T08:47:14.346863+0000","last_unstale":"2024-09-19T08:47:15.834365+0000","last_undegraded":"2024-09-19T08:47:15.834365+0000","last_fullsized":"2024-09-19T08:47:15.834365+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:31:49.532264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"169'2","reported_seq":16,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.701304+0000","last_change":"2024-09-19T08:47:13.393085+0000","last_active":"2024-09-19T08:47:15.701304+0000","last_peered":"2024-09-19T08:47:15.701304+0000","last_clean":"2024-09-19T08:47:15.701304+0000","last_became_active":"2024-09-19T08:47:13.392776+0000","last_became_peered":"2024-09-19T08:47:13.392776+0000","last_unstale":"2024-09-19T08:47:15.701304+0000","last_undegraded":"2024-09-19T08:47:15.701304+0000","last_fullsized":"2024-09-19T08:47:15.701304+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:45:50.989483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.831442+0000","last_change":"2024-09-19T08:47:14.345730+0000","last_active":"2024-09-19T08:47:15.831442+0000","last_peered":"2024-09-19T08:47:15.831442+0000","last_clean":"2024-09-19T08:47:15.831442+0000","last_became_active":"2024-09-19T08:47:14.345014+0000","last_became_peered":"2024-09-19T08:47:14.345014+0000","last_unstale":"2024-09-19T08:47:15.831442+0000","last_undegraded":"2024-09-19T08:47:15.831442+0000","last_fullsized":"2024-09-19T08:47:15.831442+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:17.675241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"169'2","reported_seq":16,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.384194+0000","last_change":"2024-09-19T08:47:13.403896+0000","last_active":"2024-09-19T08:47:15.384194+0000","last_peered":"2024-09-19T08:47:15.384194+0000","last_clean":"2024-09-19T08:47:15.384194+0000","last_became_active":"2024-09-19T08:47:13.403168+0000","last_became_peered":"2024-09-19T08:47:13.403168+0000","last_unstale":"2024-09-19T08:47:15.384194+0000","last_undegraded":"2024-09-19T08:47:15.384194+0000","last_fullsized":"2024-09-19T08:47:15.384194+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:50:14.051718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.383345+0000","last_change":"2024-09-19T08:47:14.351675+0000","last_active":"2024-09-19T08:47:15.383345+0000","last_peered":"2024-09-19T08:47:15.383345+0000","last_clean":"2024-09-19T08:47:15.383345+0000","last_became_active":"2024-09-19T08:47:14.350955+0000","last_became_peered":"2024-09-19T08:47:14.350955+0000","last_unstale":"2024-09-19T08:47:15.383345+0000","last_undegraded":"2024-09-19T08:47:15.383345+0000","last_fullsized":"2024-09-19T08:47:15.383345+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:44.489711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"169'1","reported_seq":15,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.314986+0000","last_change":"2024-09-19T08:47:13.400567+0000","last_active":"2024-09-19T08:47:15.314986+0000","last_peered":"2024-09-19T08:47:15.314986+0000","last_clean":"2024-09-19T08:47:15.314986+0000","last_became_active":"2024-09-19T08:47:13.400297+0000","last_became_peered":"2024-09-19T08:47:13.400297+0000","last_unstale":"2024-09-19T08:47:15.314986+0000","last_undegraded":"2024-09-19T08:47:15.314986+0000","last_fullsized":"2024-09-19T08:47:15.314986+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:14:37.029395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.314387+0000","last_change":"2024-09-19T08:47:14.341230+0000","last_active":"2024-09-19T08:47:15.314387+0000","last_peered":"2024-09-19T08:47:15.314387+0000","last_clean":"2024-09-19T08:47:15.314387+0000","last_became_active":"2024-09-19T08:47:14.340408+0000","last_became_peered":"2024-09-19T08:47:14.340408+0000","last_unstale":"2024-09-19T08:47:15.314387+0000","last_undegraded":"2024-09-19T08:47:15.314387+0000","last_fullsized":"2024-09-19T08:47:15.314387+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:27:37.674410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.314321+0000","last_change":"2024-09-19T08:47:13.382448+0000","last_active":"2024-09-19T08:47:15.314321+0000","last_peered":"2024-09-19T08:47:15.314321+0000","last_clean":"2024-09-19T08:47:15.314321+0000","last_became_active":"2024-09-19T08:47:13.382136+0000","last_became_peered":"2024-09-19T08:47:13.382136+0000","last_unstale":"2024-09-19T08:47:15.314321+0000","last_undegraded":"2024-09-19T08:47:15.314321+0000","last_fullsized":"2024-09-19T08:47:15.314321+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:31.792660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.383313+0000","last_change":"2024-09-19T08:47:14.354758+0000","last_active":"2024-09-19T08:47:15.383313+0000","last_peered":"2024-09-19T08:47:15.383313+0000","last_clean":"2024-09-19T08:47:15.383313+0000","last_became_active":"2024-09-19T08:47:14.353999+0000","last_became_peered":"2024-09-19T08:47:14.353999+0000","last_unstale":"2024-09-19T08:47:15.383313+0000","last_undegraded":"2024-09-19T08:47:15.383313+0000","last_fullsized":"2024-09-19T08:47:15.383313+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:18.467994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.317703+0000","last_change":"2024-09-19T08:47:13.394882+0000","last_active":"2024-09-19T08:47:15.317703+0000","last_peered":"2024-09-19T08:47:15.317703+0000","last_clean":"2024-09-19T08:47:15.317703+0000","last_became_active":"2024-09-19T08:47:13.393932+0000","last_became_peered":"2024-09-19T08:47:13.393932+0000","last_unstale":"2024-09-19T08:47:15.317703+0000","last_undegraded":"2024-09-19T08:47:15.317703+0000","last_fullsized":"2024-09-19T08:47:15.317703+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:52.284449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.384004+0000","last_change":"2024-09-19T08:47:14.365557+0000","last_active":"2024-09-19T08:47:15.384004+0000","last_peered":"2024-09-19T08:47:15.384004+0000","last_clean":"2024-09-19T08:47:15.384004+0000","last_became_active":"2024-09-19T08:47:14.364879+0000","last_became_peered":"2024-09-19T08:47:14.364879+0000","last_unstale":"2024-09-19T08:47:15.384004+0000","last_undegraded":"2024-09-19T08:47:15.384004+0000","last_fullsized":"2024-09-19T08:47:15.384004+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:51:01.879120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.314681+0000","last_change":"2024-09-19T08:47:13.398554+0000","last_active":"2024-09-19T08:47:15.314681+0000","last_peered":"2024-09-19T08:47:15.314681+0000","last_clean":"2024-09-19T08:47:15.314681+0000","last_became_active":"2024-09-19T08:47:13.397871+0000","last_became_peered":"2024-09-19T08:47:13.397871+0000","last_unstale":"2024-09-19T08:47:15.314681+0000","last_undegraded":"2024-09-19T08:47:15.314681+0000","last_fullsized":"2024-09-19T08:47:15.314681+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:41.165756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.701266+0000","last_change":"2024-09-19T08:47:14.346956+0000","last_active":"2024-09-19T08:47:15.701266+0000","last_peered":"2024-09-19T08:47:15.701266+0000","last_clean":"2024-09-19T08:47:15.701266+0000","last_became_active":"2024-09-19T08:47:14.346229+0000","last_became_peered":"2024-09-19T08:47:14.346229+0000","last_unstale":"2024-09-19T08:47:15.701266+0000","last_undegraded":"2024-09-19T08:47:15.701266+0000","last_fullsized":"2024-09-19T08:47:15.701266+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:39:52.187988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.319341+0000","last_change":"2024-09-19T08:47:13.407203+0000","last_active":"2024-09-19T08:47:15.319341+0000","last_peered":"2024-09-19T08:47:15.319341+0000","last_clean":"2024-09-19T08:47:15.319341+0000","last_became_active":"2024-09-19T08:47:13.406419+0000","last_became_peered":"2024-09-19T08:47:13.406419+0000","last_unstale":"2024-09-19T08:47:15.319341+0000","last_undegraded":"2024-09-19T08:47:15.319341+0000","last_fullsized":"2024-09-19T08:47:15.319341+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:31.792660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.384218+0000","last_change":"2024-09-19T08:47:14.349482+0000","last_active":"2024-09-19T08:47:15.384218+0000","last_peered":"2024-09-19T08:47:15.384218+0000","last_clean":"2024-09-19T08:47:15.384218+0000","last_became_active":"2024-09-19T08:47:14.348750+0000","last_became_peered":"2024-09-19T08:47:14.348750+0000","last_unstale":"2024-09-19T08:47:15.384218+0000","last_undegraded":"2024-09-19T08:47:15.384218+0000","last_fullsized":"2024-09-19T08:47:15.384218+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:47:56.893479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"169'3","reported_seq":17,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.831832+0000","last_change":"2024-09-19T08:47:13.382865+0000","last_active":"2024-09-19T08:47:15.831832+0000","last_peered":"2024-09-19T08:47:15.831832+0000","last_clean":"2024-09-19T08:47:15.831832+0000","last_became_active":"2024-09-19T08:47:13.382528+0000","last_became_peered":"2024-09-19T08:47:13.382528+0000","last_unstale":"2024-09-19T08:47:15.831832+0000","last_undegraded":"2024-09-19T08:47:15.831832+0000","last_fullsized":"2024-09-19T08:47:15.831832+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:21.216354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833935+0000","last_change":"2024-09-19T08:47:14.347539+0000","last_active":"2024-09-19T08:47:15.833935+0000","last_peered":"2024-09-19T08:47:15.833935+0000","last_clean":"2024-09-19T08:47:15.833935+0000","last_became_active":"2024-09-19T08:47:14.346707+0000","last_became_peered":"2024-09-19T08:47:14.346707+0000","last_unstale":"2024-09-19T08:47:15.833935+0000","last_undegraded":"2024-09-19T08:47:15.833935+0000","last_fullsized":"2024-09-19T08:47:15.833935+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:28:44.021408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"169'6","reported_seq":20,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.384347+0000","last_change":"2024-09-19T08:47:13.409867+0000","last_active":"2024-09-19T08:47:15.384347+0000","last_peered":"2024-09-19T08:47:15.384347+0000","last_clean":"2024-09-19T08:47:15.384347+0000","last_became_active":"2024-09-19T08:47:13.408810+0000","last_became_peered":"2024-09-19T08:47:13.408810+0000","last_unstale":"2024-09-19T08:47:15.384347+0000","last_undegraded":"2024-09-19T08:47:15.384347+0000","last_fullsized":"2024-09-19T08:47:15.384347+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:04.792972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.314861+0000","last_change":"2024-09-19T08:47:14.352505+0000","last_active":"2024-09-19T08:47:15.314861+0000","last_peered":"2024-09-19T08:47:15.314861+0000","last_clean":"2024-09-19T08:47:15.314861+0000","last_became_active":"2024-09-19T08:47:14.351672+0000","last_became_peered":"2024-09-19T08:47:14.351672+0000","last_unstale":"2024-09-19T08:47:15.314861+0000","last_undegraded":"2024-09-19T08:47:15.314861+0000","last_fullsized":"2024-09-19T08:47:15.314861+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:59:09.201492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"169'2","reported_seq":16,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.318763+0000","last_change":"2024-09-19T08:47:13.404129+0000","last_active":"2024-09-19T08:47:15.318763+0000","last_peered":"2024-09-19T08:47:15.318763+0000","last_clean":"2024-09-19T08:47:15.318763+0000","last_became_active":"2024-09-19T08:47:13.403371+0000","last_became_peered":"2024-09-19T08:47:13.403371+0000","last_unstale":"2024-09-19T08:47:15.318763+0000","last_undegraded":"2024-09-19T08:47:15.318763+0000","last_fullsized":"2024-09-19T08:47:15.318763+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:17.535117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.831322+0000","last_change":"2024-09-19T08:47:14.359096+0000","last_active":"2024-09-19T08:47:15.831322+0000","last_peered":"2024-09-19T08:47:15.831322+0000","last_clean":"2024-09-19T08:47:15.831322+0000","last_became_active":"2024-09-19T08:47:14.358410+0000","last_became_peered":"2024-09-19T08:47:14.358410+0000","last_unstale":"2024-09-19T08:47:15.831322+0000","last_undegraded":"2024-09-19T08:47:15.831322+0000","last_fullsized":"2024-09-19T08:47:15.831322+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:10:15.182744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.318529+0000","last_change":"2024-09-19T08:47:13.405188+0000","last_active":"2024-09-19T08:47:15.318529+0000","last_peered":"2024-09-19T08:47:15.318529+0000","last_clean":"2024-09-19T08:47:15.318529+0000","last_became_active":"2024-09-19T08:47:13.404512+0000","last_became_peered":"2024-09-19T08:47:13.404512+0000","last_unstale":"2024-09-19T08:47:15.318529+0000","last_undegraded":"2024-09-19T08:47:15.318529+0000","last_fullsized":"2024-09-19T08:47:15.318529+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:38.934425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.700267+0000","last_change":"2024-09-19T08:47:14.347839+0000","last_active":"2024-09-19T08:47:15.700267+0000","last_peered":"2024-09-19T08:47:15.700267+0000","last_clean":"2024-09-19T08:47:15.700267+0000","last_became_active":"2024-09-19T08:47:14.346533+0000","last_became_peered":"2024-09-19T08:47:14.346533+0000","last_unstale":"2024-09-19T08:47:15.700267+0000","last_undegraded":"2024-09-19T08:47:15.700267+0000","last_fullsized":"2024-09-19T08:47:15.700267+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:09:46.261549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.830605+0000","last_change":"2024-09-19T08:47:13.393731+0000","last_active":"2024-09-19T08:47:15.830605+0000","last_peered":"2024-09-19T08:47:15.830605+0000","last_clean":"2024-09-19T08:47:15.830605+0000","last_became_active":"2024-09-19T08:47:13.393445+0000","last_became_peered":"2024-09-19T08:47:13.393445+0000","last_unstale":"2024-09-19T08:47:15.830605+0000","last_undegraded":"2024-09-19T08:47:15.830605+0000","last_fullsized":"2024-09-19T08:47:15.830605+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:17:50.481692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833216+0000","last_change":"2024-09-19T08:47:14.362136+0000","last_active":"2024-09-19T08:47:15.833216+0000","last_peered":"2024-09-19T08:47:15.833216+0000","last_clean":"2024-09-19T08:47:15.833216+0000","last_became_active":"2024-09-19T08:47:14.360937+0000","last_became_peered":"2024-09-19T08:47:14.360937+0000","last_unstale":"2024-09-19T08:47:15.833216+0000","last_undegraded":"2024-09-19T08:47:15.833216+0000","last_fullsized":"2024-09-19T08:47:15.833216+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:09:33.724240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"169'8","reported_seq":22,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.700621+0000","last_change":"2024-09-19T08:47:13.393111+0000","last_active":"2024-09-19T08:47:15.700621+0000","last_peered":"2024-09-19T08:47:15.700621+0000","last_clean":"2024-09-19T08:47:15.700621+0000","last_became_active":"2024-09-19T08:47:13.392754+0000","last_became_peered":"2024-09-19T08:47:13.392754+0000","last_unstale":"2024-09-19T08:47:15.700621+0000","last_undegraded":"2024-09-19T08:47:15.700621+0000","last_fullsized":"2024-09-19T08:47:15.700621+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:47:39.662192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.834342+0000","last_change":"2024-09-19T08:47:14.363789+0000","last_active":"2024-09-19T08:47:15.834342+0000","last_peered":"2024-09-19T08:47:15.834342+0000","last_clean":"2024-09-19T08:47:15.834342+0000","last_became_active":"2024-09-19T08:47:14.362514+0000","last_became_peered":"2024-09-19T08:47:14.362514+0000","last_unstale":"2024-09-19T08:47:15.834342+0000","last_undegraded":"2024-09-19T08:47:15.834342+0000","last_fullsized":"2024-09-19T08:47:15.834342+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:45.701737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"169'1","reported_seq":15,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.314766+0000","last_change":"2024-09-19T08:47:13.383220+0000","last_active":"2024-09-19T08:47:15.314766+0000","last_peered":"2024-09-19T08:47:15.314766+0000","last_clean":"2024-09-19T08:47:15.314766+0000","last_became_active":"2024-09-19T08:47:13.382953+0000","last_became_peered":"2024-09-19T08:47:13.382953+0000","last_unstale":"2024-09-19T08:47:15.314766+0000","last_undegraded":"2024-09-19T08:47:15.314766+0000","last_fullsized":"2024-09-19T08:47:15.314766+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:24.754354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.834207+0000","last_change":"2024-09-19T08:47:14.358774+0000","last_active":"2024-09-19T08:47:15.834207+0000","last_peered":"2024-09-19T08:47:15.834207+0000","last_clean":"2024-09-19T08:47:15.834207+0000","last_became_active":"2024-09-19T08:47:14.356848+0000","last_became_peered":"2024-09-19T08:47:14.356848+0000","last_unstale":"2024-09-19T08:47:15.834207+0000","last_undegraded":"2024-09-19T08:47:15.834207+0000","last_fullsized":"2024-09-19T08:47:15.834207+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:32:24.678912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"169'2","reported_seq":16,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.834488+0000","last_change":"2024-09-19T08:47:13.408636+0000","last_active":"2024-09-19T08:47:15.834488+0000","last_peered":"2024-09-19T08:47:15.834488+0000","last_clean":"2024-09-19T08:47:15.834488+0000","last_became_active":"2024-09-19T08:47:13.407243+0000","last_became_peered":"2024-09-19T08:47:13.407243+0000","last_unstale":"2024-09-19T08:47:15.834488+0000","last_undegraded":"2024-09-19T08:47:15.834488+0000","last_fullsized":"2024-09-19T08:47:15.834488+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:59.000541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.700602+0000","last_change":"2024-09-19T08:47:14.355537+0000","last_active":"2024-09-19T08:47:15.700602+0000","last_peered":"2024-09-19T08:47:15.700602+0000","last_clean":"2024-09-19T08:47:15.700602+0000","last_became_active":"2024-09-19T08:47:14.354555+0000","last_became_peered":"2024-09-19T08:47:14.354555+0000","last_unstale":"2024-09-19T08:47:15.700602+0000","last_undegraded":"2024-09-19T08:47:15.700602+0000","last_fullsized":"2024-09-19T08:47:15.700602+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:44.668862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"169'1","reported_seq":15,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.831941+0000","last_change":"2024-09-19T08:47:13.382929+0000","last_active":"2024-09-19T08:47:15.831941+0000","last_peered":"2024-09-19T08:47:15.831941+0000","last_clean":"2024-09-19T08:47:15.831941+0000","last_became_active":"2024-09-19T08:47:13.382644+0000","last_became_peered":"2024-09-19T08:47:13.382644+0000","last_unstale":"2024-09-19T08:47:15.831941+0000","last_undegraded":"2024-09-19T08:47:15.831941+0000","last_fullsized":"2024-09-19T08:47:15.831941+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:08:23.245952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.831118+0000","last_change":"2024-09-19T08:47:14.345326+0000","last_active":"2024-09-19T08:47:15.831118+0000","last_peered":"2024-09-19T08:47:15.831118+0000","last_clean":"2024-09-19T08:47:15.831118+0000","last_became_active":"2024-09-19T08:47:14.340690+0000","last_became_peered":"2024-09-19T08:47:14.340690+0000","last_unstale":"2024-09-19T08:47:15.831118+0000","last_undegraded":"2024-09-19T08:47:15.831118+0000","last_fullsized":"2024-09-19T08:47:15.831118+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:34.978283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.700881+0000","last_change":"2024-09-19T08:47:13.382212+0000","last_active":"2024-09-19T08:47:15.700881+0000","last_peered":"2024-09-19T08:47:15.700881+0000","last_clean":"2024-09-19T08:47:15.700881+0000","last_became_active":"2024-09-19T08:47:13.381878+0000","last_became_peered":"2024-09-19T08:47:13.381878+0000","last_unstale":"2024-09-19T08:47:15.700881+0000","last_undegraded":"2024-09-19T08:47:15.700881+0000","last_fullsized":"2024-09-19T08:47:15.700881+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:43.813585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.831842+0000","last_change":"2024-09-19T08:47:14.346182+0000","last_active":"2024-09-19T08:47:15.831842+0000","last_peered":"2024-09-19T08:47:15.831842+0000","last_clean":"2024-09-19T08:47:15.831842+0000","last_became_active":"2024-09-19T08:47:14.345377+0000","last_became_peered":"2024-09-19T08:47:14.345377+0000","last_unstale":"2024-09-19T08:47:15.831842+0000","last_undegraded":"2024-09-19T08:47:15.831842+0000","last_fullsized":"2024-09-19T08:47:15.831842+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:59.978626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"169'2","reported_seq":16,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.831403+0000","last_change":"2024-09-19T08:47:13.384584+0000","last_active":"2024-09-19T08:47:15.831403+0000","last_peered":"2024-09-19T08:47:15.831403+0000","last_clean":"2024-09-19T08:47:15.831403+0000","last_became_active":"2024-09-19T08:47:13.384215+0000","last_became_peered":"2024-09-19T08:47:13.384215+0000","last_unstale":"2024-09-19T08:47:15.831403+0000","last_undegraded":"2024-09-19T08:47:15.831403+0000","last_fullsized":"2024-09-19T08:47:15.831403+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:19.499642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.700643+0000","last_change":"2024-09-19T08:47:14.353176+0000","last_active":"2024-09-19T08:47:15.700643+0000","last_peered":"2024-09-19T08:47:15.700643+0000","last_clean":"2024-09-19T08:47:15.700643+0000","last_became_active":"2024-09-19T08:47:14.347339+0000","last_became_peered":"2024-09-19T08:47:14.347339+0000","last_unstale":"2024-09-19T08:47:15.700643+0000","last_undegraded":"2024-09-19T08:47:15.700643+0000","last_fullsized":"2024-09-19T08:47:15.700643+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:09:59.841644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833567+0000","last_change":"2024-09-19T08:47:13.393941+0000","last_active":"2024-09-19T08:47:15.833567+0000","last_peered":"2024-09-19T08:47:15.833567+0000","last_clean":"2024-09-19T08:47:15.833567+0000","last_became_active":"2024-09-19T08:47:13.393683+0000","last_became_peered":"2024-09-19T08:47:13.393683+0000","last_unstale":"2024-09-19T08:47:15.833567+0000","last_undegraded":"2024-09-19T08:47:15.833567+0000","last_fullsized":"2024-09-19T08:47:15.833567+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:29.678095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833498+0000","last_change":"2024-09-19T08:47:14.354662+0000","last_active":"2024-09-19T08:47:15.833498+0000","last_peered":"2024-09-19T08:47:15.833498+0000","last_clean":"2024-09-19T08:47:15.833498+0000","last_became_active":"2024-09-19T08:47:14.353887+0000","last_became_peered":"2024-09-19T08:47:14.353887+0000","last_unstale":"2024-09-19T08:47:15.833498+0000","last_undegraded":"2024-09-19T08:47:15.833498+0000","last_fullsized":"2024-09-19T08:47:15.833498+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:19.026785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833264+0000","last_change":"2024-09-19T08:47:13.404413+0000","last_active":"2024-09-19T08:47:15.833264+0000","last_peered":"2024-09-19T08:47:15.833264+0000","last_clean":"2024-09-19T08:47:15.833264+0000","last_became_active":"2024-09-19T08:47:13.404206+0000","last_became_peered":"2024-09-19T08:47:13.404206+0000","last_unstale":"2024-09-19T08:47:15.833264+0000","last_undegraded":"2024-09-19T08:47:15.833264+0000","last_fullsized":"2024-09-19T08:47:15.833264+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:09.485498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.318156+0000","last_change":"2024-09-19T08:47:14.340136+0000","last_active":"2024-09-19T08:47:15.318156+0000","last_peered":"2024-09-19T08:47:15.318156+0000","last_clean":"2024-09-19T08:47:15.318156+0000","last_became_active":"2024-09-19T08:47:14.337946+0000","last_became_peered":"2024-09-19T08:47:14.337946+0000","last_unstale":"2024-09-19T08:47:15.318156+0000","last_undegraded":"2024-09-19T08:47:15.318156+0000","last_fullsized":"2024-09-19T08:47:15.318156+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:03.353387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.314696+0000","last_change":"2024-09-19T08:47:13.387174+0000","last_active":"2024-09-19T08:47:15.314696+0000","last_peered":"2024-09-19T08:47:15.314696+0000","last_clean":"2024-09-19T08:47:15.314696+0000","last_became_active":"2024-09-19T08:47:13.386875+0000","last_became_peered":"2024-09-19T08:47:13.386875+0000","last_unstale":"2024-09-19T08:47:15.314696+0000","last_undegraded":"2024-09-19T08:47:15.314696+0000","last_fullsized":"2024-09-19T08:47:15.314696+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:22:04.699154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.388462+0000","last_change":"2024-09-19T08:47:14.344815+0000","last_active":"2024-09-19T08:47:15.388462+0000","last_peered":"2024-09-19T08:47:15.388462+0000","last_clean":"2024-09-19T08:47:15.388462+0000","last_became_active":"2024-09-19T08:47:14.343956+0000","last_became_peered":"2024-09-19T08:47:14.343956+0000","last_unstale":"2024-09-19T08:47:15.388462+0000","last_undegraded":"2024-09-19T08:47:15.388462+0000","last_fullsized":"2024-09-19T08:47:15.388462+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:17:40.271999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"169'2","reported_seq":16,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.701112+0000","last_change":"2024-09-19T08:47:13.398116+0000","last_active":"2024-09-19T08:47:15.701112+0000","last_peered":"2024-09-19T08:47:15.701112+0000","last_clean":"2024-09-19T08:47:15.701112+0000","last_became_active":"2024-09-19T08:47:13.397684+0000","last_became_peered":"2024-09-19T08:47:13.397684+0000","last_unstale":"2024-09-19T08:47:15.701112+0000","last_undegraded":"2024-09-19T08:47:15.701112+0000","last_fullsized":"2024-09-19T08:47:15.701112+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:10:26.683853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833962+0000","last_change":"2024-09-19T08:47:14.360783+0000","last_active":"2024-09-19T08:47:15.833962+0000","last_peered":"2024-09-19T08:47:15.833962+0000","last_clean":"2024-09-19T08:47:15.833962+0000","last_became_active":"2024-09-19T08:47:14.348853+0000","last_became_peered":"2024-09-19T08:47:14.348853+0000","last_unstale":"2024-09-19T08:47:15.833962+0000","last_undegraded":"2024-09-19T08:47:15.833962+0000","last_fullsized":"2024-09-19T08:47:15.833962+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:46:22.684299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"170'1","reported_seq":15,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.384227+0000","last_change":"2024-09-19T08:47:13.402409+0000","last_active":"2024-09-19T08:47:15.384227+0000","last_peered":"2024-09-19T08:47:15.384227+0000","last_clean":"2024-09-19T08:47:15.384227+0000","last_became_active":"2024-09-19T08:47:13.401624+0000","last_became_peered":"2024-09-19T08:47:13.401624+0000","last_unstale":"2024-09-19T08:47:15.384227+0000","last_undegraded":"2024-09-19T08:47:15.384227+0000","last_fullsized":"2024-09-19T08:47:15.384227+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:45.064523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833923+0000","last_change":"2024-09-19T08:47:14.354272+0000","last_active":"2024-09-19T08:47:15.833923+0000","last_peered":"2024-09-19T08:47:15.833923+0000","last_clean":"2024-09-19T08:47:15.833923+0000","last_became_active":"2024-09-19T08:47:14.353523+0000","last_became_peered":"2024-09-19T08:47:14.353523+0000","last_unstale":"2024-09-19T08:47:15.833923+0000","last_undegraded":"2024-09-19T08:47:15.833923+0000","last_fullsized":"2024-09-19T08:47:15.833923+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:49.222433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.315399+0000","last_change":"2024-09-19T08:47:13.385451+0000","last_active":"2024-09-19T08:47:15.315399+0000","last_peered":"2024-09-19T08:47:15.315399+0000","last_clean":"2024-09-19T08:47:15.315399+0000","last_became_active":"2024-09-19T08:47:13.384496+0000","last_became_peered":"2024-09-19T08:47:13.384496+0000","last_unstale":"2024-09-19T08:47:15.315399+0000","last_undegraded":"2024-09-19T08:47:15.315399+0000","last_fullsized":"2024-09-19T08:47:15.315399+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:38.934425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.318949+0000","last_change":"2024-09-19T08:47:14.346827+0000","last_active":"2024-09-19T08:47:15.318949+0000","last_peered":"2024-09-19T08:47:15.318949+0000","last_clean":"2024-09-19T08:47:15.318949+0000","last_became_active":"2024-09-19T08:47:14.346102+0000","last_became_peered":"2024-09-19T08:47:14.346102+0000","last_unstale":"2024-09-19T08:47:15.318949+0000","last_undegraded":"2024-09-19T08:47:15.318949+0000","last_fullsized":"2024-09-19T08:47:15.318949+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:11.917219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.700870+0000","last_change":"2024-09-19T08:47:13.398325+0000","last_active":"2024-09-19T08:47:15.700870+0000","last_peered":"2024-09-19T08:47:15.700870+0000","last_clean":"2024-09-19T08:47:15.700870+0000","last_became_active":"2024-09-19T08:47:13.397450+0000","last_became_peered":"2024-09-19T08:47:13.397450+0000","last_unstale":"2024-09-19T08:47:15.700870+0000","last_undegraded":"2024-09-19T08:47:15.700870+0000","last_fullsized":"2024-09-19T08:47:15.700870+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:49:23.696716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833467+0000","last_change":"2024-09-19T08:47:14.344698+0000","last_active":"2024-09-19T08:47:15.833467+0000","last_peered":"2024-09-19T08:47:15.833467+0000","last_clean":"2024-09-19T08:47:15.833467+0000","last_became_active":"2024-09-19T08:47:14.343844+0000","last_became_peered":"2024-09-19T08:47:14.343844+0000","last_unstale":"2024-09-19T08:47:15.833467+0000","last_undegraded":"2024-09-19T08:47:15.833467+0000","last_fullsized":"2024-09-19T08:47:15.833467+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:25:42.320274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.383623+0000","last_change":"2024-09-19T08:47:13.399625+0000","last_active":"2024-09-19T08:47:15.383623+0000","last_peered":"2024-09-19T08:47:15.383623+0000","last_clean":"2024-09-19T08:47:15.383623+0000","last_became_active":"2024-09-19T08:47:13.398838+0000","last_became_peered":"2024-09-19T08:47:13.398838+0000","last_unstale":"2024-09-19T08:47:15.383623+0000","last_undegraded":"2024-09-19T08:47:15.383623+0000","last_fullsized":"2024-09-19T08:47:15.383623+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:26:50.957727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.388653+0000","last_change":"2024-09-19T08:47:14.343451+0000","last_active":"2024-09-19T08:47:15.388653+0000","last_peered":"2024-09-19T08:47:15.388653+0000","last_clean":"2024-09-19T08:47:15.388653+0000","last_became_active":"2024-09-19T08:47:14.342662+0000","last_became_peered":"2024-09-19T08:47:14.342662+0000","last_unstale":"2024-09-19T08:47:15.388653+0000","last_undegraded":"2024-09-19T08:47:15.388653+0000","last_fullsized":"2024-09-19T08:47:15.388653+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:02:13.929162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.315041+0000","last_change":"2024-09-19T08:47:13.384922+0000","last_active":"2024-09-19T08:47:15.315041+0000","last_peered":"2024-09-19T08:47:15.315041+0000","last_clean":"2024-09-19T08:47:15.315041+0000","last_became_active":"2024-09-19T08:47:13.384555+0000","last_became_peered":"2024-09-19T08:47:13.384555+0000","last_unstale":"2024-09-19T08:47:15.315041+0000","last_undegraded":"2024-09-19T08:47:15.315041+0000","last_fullsized":"2024-09-19T08:47:15.315041+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:27:36.013002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.700392+0000","last_change":"2024-09-19T08:47:14.347684+0000","last_active":"2024-09-19T08:47:15.700392+0000","last_peered":"2024-09-19T08:47:15.700392+0000","last_clean":"2024-09-19T08:47:15.700392+0000","last_became_active":"2024-09-19T08:47:14.345594+0000","last_became_peered":"2024-09-19T08:47:14.345594+0000","last_unstale":"2024-09-19T08:47:15.700392+0000","last_undegraded":"2024-09-19T08:47:15.700392+0000","last_fullsized":"2024-09-19T08:47:15.700392+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:11.450025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.318402+0000","last_change":"2024-09-19T08:47:13.407182+0000","last_active":"2024-09-19T08:47:15.318402+0000","last_peered":"2024-09-19T08:47:15.318402+0000","last_clean":"2024-09-19T08:47:15.318402+0000","last_became_active":"2024-09-19T08:47:13.405978+0000","last_became_peered":"2024-09-19T08:47:13.405978+0000","last_unstale":"2024-09-19T08:47:15.318402+0000","last_undegraded":"2024-09-19T08:47:15.318402+0000","last_fullsized":"2024-09-19T08:47:15.318402+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:41.165756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833516+0000","last_change":"2024-09-19T08:47:14.354669+0000","last_active":"2024-09-19T08:47:15.833516+0000","last_peered":"2024-09-19T08:47:15.833516+0000","last_clean":"2024-09-19T08:47:15.833516+0000","last_became_active":"2024-09-19T08:47:14.353479+0000","last_became_peered":"2024-09-19T08:47:14.353479+0000","last_unstale":"2024-09-19T08:47:15.833516+0000","last_undegraded":"2024-09-19T08:47:15.833516+0000","last_fullsized":"2024-09-19T08:47:15.833516+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:37:58.191552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.387876+0000","last_change":"2024-09-19T08:47:13.386481+0000","last_active":"2024-09-19T08:47:15.387876+0000","last_peered":"2024-09-19T08:47:15.387876+0000","last_clean":"2024-09-19T08:47:15.387876+0000","last_became_active":"2024-09-19T08:47:13.385716+0000","last_became_peered":"2024-09-19T08:47:13.385716+0000","last_unstale":"2024-09-19T08:47:15.387876+0000","last_undegraded":"2024-09-19T08:47:15.387876+0000","last_fullsized":"2024-09-19T08:47:15.387876+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:22.519133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833065+0000","last_change":"2024-09-19T08:47:14.354191+0000","last_active":"2024-09-19T08:47:15.833065+0000","last_peered":"2024-09-19T08:47:15.833065+0000","last_clean":"2024-09-19T08:47:15.833065+0000","last_became_active":"2024-09-19T08:47:14.348271+0000","last_became_peered":"2024-09-19T08:47:14.348271+0000","last_unstale":"2024-09-19T08:47:15.833065+0000","last_undegraded":"2024-09-19T08:47:15.833065+0000","last_fullsized":"2024-09-19T08:47:15.833065+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:34:12.965823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"169'1","reported_seq":15,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.388912+0000","last_change":"2024-09-19T08:47:13.388601+0000","last_active":"2024-09-19T08:47:15.388912+0000","last_peered":"2024-09-19T08:47:15.388912+0000","last_clean":"2024-09-19T08:47:15.388912+0000","last_became_active":"2024-09-19T08:47:13.387981+0000","last_became_peered":"2024-09-19T08:47:13.387981+0000","last_unstale":"2024-09-19T08:47:15.388912+0000","last_undegraded":"2024-09-19T08:47:15.388912+0000","last_fullsized":"2024-09-19T08:47:15.388912+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:04.694583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"71'128","reported_seq":478,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.383426+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:47:15.383426+0000","last_peered":"2024-09-19T08:47:15.383426+0000","last_clean":"2024-09-19T08:47:15.383426+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:47:15.383426+0000","last_undegraded":"2024-09-19T08:47:15.383426+0000","last_fullsized":"2024-09-19T08:47:15.383426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509553,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33868,"kb_used_data":2680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337972,"statfs":{"total":96636764160,"available":96602083328,"internally_reserved":0,"allocated":2744320,"data_stored":1542832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574962,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33288,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2142208,"data_stored":951434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509555,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33280,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338560,"statfs":{"total":96636764160,"available":96602685440,"internally_reserved":0,"allocated":2142208,"data_stored":951434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509555,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33300,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338540,"statfs":{"total":96636764160,"available":96602664960,"internally_reserved":0,"allocated":2158592,"data_stored":952568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542257,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37448,"kb_used_data":2100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334392,"statfs":{"total":96636764160,"available":96598417408,"internally_reserved":0,"allocated":2150400,"data_stored":951034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509552,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33276,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338564,"statfs":{"total":96636764160,"available":96602689536,"internally_reserved":0,"allocated":2138112,"data_stored":950852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509554,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33288,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2146304,"data_stored":950908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509552,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38020,"kb_used_data":2672,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333820,"statfs":{"total":96636764160,"available":96597831680,"internally_reserved":0,"allocated":2736128,"data_stored":1541310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:47:22.743 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:47:22.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.890+0000 7fe57a8e9640 1 Processor -- start 2024-09-19T08:47:22.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.890+0000 7fe57a8e9640 1 -- start start 2024-09-19T08:47:22.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.890+0000 7fe57a8e9640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe57406c3a0 0x7fe5740cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:22.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.890+0000 7fe57a8e9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5740cf900 0x7fe5740723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:22.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.890+0000 7fe57a8e9640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe57406d310 con 0x7fe57415c550 2024-09-19T08:47:22.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.890+0000 7fe57a8e9640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fe57406d5f0 con 0x7fe57406c3a0 2024-09-19T08:47:22.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.890+0000 7fe57a8e9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe57406d8d0 con 0x7fe5740cf900 2024-09-19T08:47:22.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.890+0000 7fe56bfff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe57406c3a0 0x7fe5740cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:22.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.890+0000 7fe56bfff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe57406c3a0 0x7fe5740cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41910/0 (socket says 172.21.15.98:41910) 2024-09-19T08:47:22.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.890+0000 7fe56bfff640 1 -- 172.21.15.98:0/4004812444 learned_addr learned my addr 172.21.15.98:0/4004812444 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:22.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.890+0000 7fe578e86640 1 --2- 172.21.15.98:0/4004812444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5740cf900 0x7fe5740723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:22.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.890+0000 7fe56bfff640 1 -- 172.21.15.98:0/4004812444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5740cf900 msgr2=0x7fe5740723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:22.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.890+0000 7fe56bfff640 1 --2- 172.21.15.98:0/4004812444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5740cf900 0x7fe5740723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:22.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.890+0000 7fe56bfff640 1 -- 172.21.15.98:0/4004812444 >> v1:172.21.15.139:6789/0 conn(0x7fe57415c550 legacy=0x7fe57406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:22.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.890+0000 7fe56bfff640 1 -- 172.21.15.98:0/4004812444 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe57406dbb0 con 0x7fe57406c3a0 2024-09-19T08:47:22.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.894+0000 7fe56bfff640 1 --2- 172.21.15.98:0/4004812444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe57406c3a0 0x7fe5740cf3c0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fe55c009870 tx=0x7fe55c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29dee3c728fd5ac5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:22.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.894+0000 7fe5737fe640 1 -- 172.21.15.98:0/4004812444 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c0a7070 con 0x7fe57406c3a0 2024-09-19T08:47:22.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.894+0000 7fe5737fe640 1 -- 172.21.15.98:0/4004812444 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe55c0b0070 con 0x7fe57406c3a0 2024-09-19T08:47:22.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.894+0000 7fe5737fe640 1 -- 172.21.15.98:0/4004812444 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c0ab890 con 0x7fe57406c3a0 2024-09-19T08:47:22.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.894+0000 7fe57a8e9640 1 -- 172.21.15.98:0/4004812444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe57406c3a0 msgr2=0x7fe5740cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:22.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.894+0000 7fe57a8e9640 1 --2- 172.21.15.98:0/4004812444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe57406c3a0 0x7fe5740cf3c0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fe55c009870 tx=0x7fe55c01c560 comp rx=0 tx=0).stop 2024-09-19T08:47:22.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.894+0000 7fe57a8e9640 1 -- 172.21.15.98:0/4004812444 shutdown_connections 2024-09-19T08:47:22.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.894+0000 7fe57a8e9640 1 --2- 172.21.15.98:0/4004812444 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe5740cf900 0x7fe5740723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:22.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.894+0000 7fe57a8e9640 1 --2- 172.21.15.98:0/4004812444 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe57406c3a0 0x7fe5740cf3c0 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:22.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.894+0000 7fe57a8e9640 1 -- 172.21.15.98:0/4004812444 >> 172.21.15.98:0/4004812444 conn(0x7fe5740b2010 msgr2=0x7fe5740b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:22.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.894+0000 7fe57a8e9640 1 -- 172.21.15.98:0/4004812444 shutdown_connections 2024-09-19T08:47:22.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.894+0000 7fe57a8e9640 1 -- 172.21.15.98:0/4004812444 wait complete. 2024-09-19T08:47:22.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.894+0000 7fe57a8e9640 1 Processor -- start 2024-09-19T08:47:22.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.894+0000 7fe57a8e9640 1 -- start start 2024-09-19T08:47:22.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.894+0000 7fe57a8e9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5740cf900 0x7fe574152ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:22.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.894+0000 7fe57a8e9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe57415c550 0x7fe574157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:22.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.894+0000 7fe57a8e9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe574159800 0x7fe5741549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:22.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.894+0000 7fe57a8e9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe574073500 con 0x7fe57415c550 2024-09-19T08:47:22.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.894+0000 7fe57a8e9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe574073380 con 0x7fe5740cf900 2024-09-19T08:47:22.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.894+0000 7fe57a8e9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe574073680 con 0x7fe574159800 2024-09-19T08:47:22.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.894+0000 7fe56bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe57415c550 0x7fe574157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:22.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.894+0000 7fe56bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe57415c550 0x7fe574157ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41924/0 (socket says 172.21.15.98:41924) 2024-09-19T08:47:22.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.894+0000 7fe56bfff640 1 -- 172.21.15.98:0/122142804 learned_addr learned my addr 172.21.15.98:0/122142804 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:22.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.894+0000 7fe578e86640 1 --2- 172.21.15.98:0/122142804 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe574159800 0x7fe5741549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:22.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.894+0000 7fe573fff640 1 --2- 172.21.15.98:0/122142804 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5740cf900 0x7fe574152ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:22.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.894+0000 7fe56bfff640 1 -- 172.21.15.98:0/122142804 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe574159800 msgr2=0x7fe5741549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:22.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.894+0000 7fe56bfff640 1 --2- 172.21.15.98:0/122142804 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe574159800 0x7fe5741549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:22.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.894+0000 7fe56bfff640 1 -- 172.21.15.98:0/122142804 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5740cf900 msgr2=0x7fe574152ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:22.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.894+0000 7fe56bfff640 1 --2- 172.21.15.98:0/122142804 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5740cf900 0x7fe574152ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:22.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.894+0000 7fe56bfff640 1 -- 172.21.15.98:0/122142804 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe574158760 con 0x7fe57415c550 2024-09-19T08:47:22.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.894+0000 7fe573fff640 1 --2- 172.21.15.98:0/122142804 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5740cf900 0x7fe574152ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:22.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.898+0000 7fe56bfff640 1 --2- 172.21.15.98:0/122142804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe57415c550 0x7fe574157ff0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fe55c0ad040 tx=0x7fe55c01ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:22.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.898+0000 7fe5717fa640 1 -- 172.21.15.98:0/122142804 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c004220 con 0x7fe57415c550 2024-09-19T08:47:22.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.898+0000 7fe5717fa640 1 -- 172.21.15.98:0/122142804 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe55c0043c0 con 0x7fe57415c550 2024-09-19T08:47:22.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.898+0000 7fe5717fa640 1 -- 172.21.15.98:0/122142804 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c003820 con 0x7fe57415c550 2024-09-19T08:47:22.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.898+0000 7fe57a8e9640 1 -- 172.21.15.98:0/122142804 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe574158a40 con 0x7fe57415c550 2024-09-19T08:47:22.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.898+0000 7fe57a8e9640 1 -- 172.21.15.98:0/122142804 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe574159000 con 0x7fe57415c550 2024-09-19T08:47:22.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.898+0000 7fe57a8e9640 1 -- 172.21.15.98:0/122142804 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5740cf3c0 con 0x7fe57415c550 2024-09-19T08:47:22.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.898+0000 7fe5717fa640 1 -- 172.21.15.98:0/122142804 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7fe55c0039c0 con 0x7fe57415c550 2024-09-19T08:47:22.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.898+0000 7fe5717fa640 1 --2- 172.21.15.98:0/122142804 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe554068250 0x7fe55406a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:22.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.898+0000 7fe5717fa640 1 -- 172.21.15.98:0/122142804 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5737+0+0 (secure 0 0 0) 0x7fe55c11e5e0 con 0x7fe57415c550 2024-09-19T08:47:22.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.898+0000 7fe573fff640 1 --2- 172.21.15.98:0/122142804 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe554068250 0x7fe55406a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:22.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.898+0000 7fe573fff640 1 --2- 172.21.15.98:0/122142804 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe554068250 0x7fe55406a710 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe574153910 tx=0x7fe564009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:22.902 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:22.902+0000 7fe5717fa640 1 -- 172.21.15.98:0/122142804 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe55c016150 con 0x7fe57415c550 2024-09-19T08:47:23.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.074+0000 7fe57a8e9640 1 -- 172.21.15.98:0/122142804 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe5740cd1c0 con 0x7fe554068250 2024-09-19T08:47:23.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.094+0000 7fe5717fa640 1 -- 172.21.15.98:0/122142804 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252496 (secure 0 0 0) 0x7fe5740cd1c0 con 0x7fe554068250 2024-09-19T08:47:23.095 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:47:23.096 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:47:23.097 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.098+0000 7fe57a8e9640 1 -- 172.21.15.98:0/122142804 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe554068250 msgr2=0x7fe55406a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:23.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.098+0000 7fe57a8e9640 1 --2- 172.21.15.98:0/122142804 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe554068250 0x7fe55406a710 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe574153910 tx=0x7fe564009040 comp rx=0 tx=0).stop 2024-09-19T08:47:23.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.098+0000 7fe57a8e9640 1 -- 172.21.15.98:0/122142804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe57415c550 msgr2=0x7fe574157ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:23.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.098+0000 7fe57a8e9640 1 --2- 172.21.15.98:0/122142804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe57415c550 0x7fe574157ff0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fe55c0ad040 tx=0x7fe55c01ada0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.098+0000 7fe57a8e9640 1 -- 172.21.15.98:0/122142804 shutdown_connections 2024-09-19T08:47:23.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.098+0000 7fe57a8e9640 1 --2- 172.21.15.98:0/122142804 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe554068250 0x7fe55406a710 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.098+0000 7fe57a8e9640 1 --2- 172.21.15.98:0/122142804 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe574159800 0x7fe5741549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.098+0000 7fe57a8e9640 1 --2- 172.21.15.98:0/122142804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe57415c550 0x7fe574157ff0 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.098+0000 7fe57a8e9640 1 --2- 172.21.15.98:0/122142804 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5740cf900 0x7fe574152ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.098+0000 7fe57a8e9640 1 -- 172.21.15.98:0/122142804 >> 172.21.15.98:0/122142804 conn(0x7fe5740b2010 msgr2=0x7fe574155ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:23.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.098+0000 7fe57a8e9640 1 -- 172.21.15.98:0/122142804 shutdown_connections 2024-09-19T08:47:23.098 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.098+0000 7fe57a8e9640 1 -- 172.21.15.98:0/122142804 wait complete. 2024-09-19T08:47:23.113 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":233,"stamp":"2024-09-19T08:47:22.960284+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":275768,"kb_used_data":17928,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754698952,"statfs":{"total":773094113280,"available":772811726848,"internally_reserved":0,"allocated":18358272,"data_stored":8792372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":58,"apply_latency_ms":58,"commit_latency_ns":58000000,"apply_latency_ns":58000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.646868"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833628+0000","last_change":"2024-09-19T08:47:14.359460+0000","last_active":"2024-09-19T08:47:15.833628+0000","last_peered":"2024-09-19T08:47:15.833628+0000","last_clean":"2024-09-19T08:47:15.833628+0000","last_became_active":"2024-09-19T08:47:14.352467+0000","last_became_peered":"2024-09-19T08:47:14.352467+0000","last_unstale":"2024-09-19T08:47:15.833628+0000","last_undegraded":"2024-09-19T08:47:15.833628+0000","last_fullsized":"2024-09-19T08:47:15.833628+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:49:12.331382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.834037+0000","last_change":"2024-09-19T08:47:14.363174+0000","last_active":"2024-09-19T08:47:15.834037+0000","last_peered":"2024-09-19T08:47:15.834037+0000","last_clean":"2024-09-19T08:47:15.834037+0000","last_became_active":"2024-09-19T08:47:14.362109+0000","last_became_peered":"2024-09-19T08:47:14.362109+0000","last_unstale":"2024-09-19T08:47:15.834037+0000","last_undegraded":"2024-09-19T08:47:15.834037+0000","last_fullsized":"2024-09-19T08:47:15.834037+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:34.366623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.701137+0000","last_change":"2024-09-19T08:47:14.365200+0000","last_active":"2024-09-19T08:47:15.701137+0000","last_peered":"2024-09-19T08:47:15.701137+0000","last_clean":"2024-09-19T08:47:15.701137+0000","last_became_active":"2024-09-19T08:47:14.357837+0000","last_became_peered":"2024-09-19T08:47:14.357837+0000","last_unstale":"2024-09-19T08:47:15.701137+0000","last_undegraded":"2024-09-19T08:47:15.701137+0000","last_fullsized":"2024-09-19T08:47:15.701137+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:54.142030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.383795+0000","last_change":"2024-09-19T08:47:14.354043+0000","last_active":"2024-09-19T08:47:15.383795+0000","last_peered":"2024-09-19T08:47:15.383795+0000","last_clean":"2024-09-19T08:47:15.383795+0000","last_became_active":"2024-09-19T08:47:14.352832+0000","last_became_peered":"2024-09-19T08:47:14.352832+0000","last_unstale":"2024-09-19T08:47:15.383795+0000","last_undegraded":"2024-09-19T08:47:15.383795+0000","last_fullsized":"2024-09-19T08:47:15.383795+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:38:10.816584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.315069+0000","last_change":"2024-09-19T08:47:14.342189+0000","last_active":"2024-09-19T08:47:15.315069+0000","last_peered":"2024-09-19T08:47:15.315069+0000","last_clean":"2024-09-19T08:47:15.315069+0000","last_became_active":"2024-09-19T08:47:14.340626+0000","last_became_peered":"2024-09-19T08:47:14.340626+0000","last_unstale":"2024-09-19T08:47:15.315069+0000","last_undegraded":"2024-09-19T08:47:15.315069+0000","last_fullsized":"2024-09-19T08:47:15.315069+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:42.709485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.314735+0000","last_change":"2024-09-19T08:47:14.361414+0000","last_active":"2024-09-19T08:47:15.314735+0000","last_peered":"2024-09-19T08:47:15.314735+0000","last_clean":"2024-09-19T08:47:15.314735+0000","last_became_active":"2024-09-19T08:47:14.360629+0000","last_became_peered":"2024-09-19T08:47:14.360629+0000","last_unstale":"2024-09-19T08:47:15.314735+0000","last_undegraded":"2024-09-19T08:47:15.314735+0000","last_fullsized":"2024-09-19T08:47:15.314735+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:33:28.816440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.832761+0000","last_change":"2024-09-19T08:47:14.360193+0000","last_active":"2024-09-19T08:47:15.832761+0000","last_peered":"2024-09-19T08:47:15.832761+0000","last_clean":"2024-09-19T08:47:15.832761+0000","last_became_active":"2024-09-19T08:47:14.356098+0000","last_became_peered":"2024-09-19T08:47:14.356098+0000","last_unstale":"2024-09-19T08:47:15.832761+0000","last_undegraded":"2024-09-19T08:47:15.832761+0000","last_fullsized":"2024-09-19T08:47:15.832761+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:53:48.646448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.314425+0000","last_change":"2024-09-19T08:47:14.351408+0000","last_active":"2024-09-19T08:47:15.314425+0000","last_peered":"2024-09-19T08:47:15.314425+0000","last_clean":"2024-09-19T08:47:15.314425+0000","last_became_active":"2024-09-19T08:47:14.350440+0000","last_became_peered":"2024-09-19T08:47:14.350440+0000","last_unstale":"2024-09-19T08:47:15.314425+0000","last_undegraded":"2024-09-19T08:47:15.314425+0000","last_fullsized":"2024-09-19T08:47:15.314425+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:20:48.189924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.834432+0000","last_change":"2024-09-19T08:47:14.357553+0000","last_active":"2024-09-19T08:47:15.834432+0000","last_peered":"2024-09-19T08:47:15.834432+0000","last_clean":"2024-09-19T08:47:15.834432+0000","last_became_active":"2024-09-19T08:47:14.355255+0000","last_became_peered":"2024-09-19T08:47:14.355255+0000","last_unstale":"2024-09-19T08:47:15.834432+0000","last_undegraded":"2024-09-19T08:47:15.834432+0000","last_fullsized":"2024-09-19T08:47:15.834432+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:55:39.012179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.831464+0000","last_change":"2024-09-19T08:47:14.350592+0000","last_active":"2024-09-19T08:47:15.831464+0000","last_peered":"2024-09-19T08:47:15.831464+0000","last_clean":"2024-09-19T08:47:15.831464+0000","last_became_active":"2024-09-19T08:47:14.349843+0000","last_became_peered":"2024-09-19T08:47:14.349843+0000","last_unstale":"2024-09-19T08:47:15.831464+0000","last_undegraded":"2024-09-19T08:47:15.831464+0000","last_fullsized":"2024-09-19T08:47:15.831464+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:34:36.184294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.701152+0000","last_change":"2024-09-19T08:47:14.347087+0000","last_active":"2024-09-19T08:47:15.701152+0000","last_peered":"2024-09-19T08:47:15.701152+0000","last_clean":"2024-09-19T08:47:15.701152+0000","last_became_active":"2024-09-19T08:47:14.346271+0000","last_became_peered":"2024-09-19T08:47:14.346271+0000","last_unstale":"2024-09-19T08:47:15.701152+0000","last_undegraded":"2024-09-19T08:47:15.701152+0000","last_fullsized":"2024-09-19T08:47:15.701152+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:13:23.614854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.388582+0000","last_change":"2024-09-19T08:47:14.344723+0000","last_active":"2024-09-19T08:47:15.388582+0000","last_peered":"2024-09-19T08:47:15.388582+0000","last_clean":"2024-09-19T08:47:15.388582+0000","last_became_active":"2024-09-19T08:47:14.343948+0000","last_became_peered":"2024-09-19T08:47:14.343948+0000","last_unstale":"2024-09-19T08:47:15.388582+0000","last_undegraded":"2024-09-19T08:47:15.388582+0000","last_fullsized":"2024-09-19T08:47:15.388582+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:05:16.685634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.388746+0000","last_change":"2024-09-19T08:47:14.352237+0000","last_active":"2024-09-19T08:47:15.388746+0000","last_peered":"2024-09-19T08:47:15.388746+0000","last_clean":"2024-09-19T08:47:15.388746+0000","last_became_active":"2024-09-19T08:47:14.351466+0000","last_became_peered":"2024-09-19T08:47:14.351466+0000","last_unstale":"2024-09-19T08:47:15.388746+0000","last_undegraded":"2024-09-19T08:47:15.388746+0000","last_fullsized":"2024-09-19T08:47:15.388746+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:44:09.637091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.832981+0000","last_change":"2024-09-19T08:47:14.352279+0000","last_active":"2024-09-19T08:47:15.832981+0000","last_peered":"2024-09-19T08:47:15.832981+0000","last_clean":"2024-09-19T08:47:15.832981+0000","last_became_active":"2024-09-19T08:47:14.348493+0000","last_became_peered":"2024-09-19T08:47:14.348493+0000","last_unstale":"2024-09-19T08:47:15.832981+0000","last_undegraded":"2024-09-19T08:47:15.832981+0000","last_fullsized":"2024-09-19T08:47:15.832981+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:57:37.877782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.832961+0000","last_change":"2024-09-19T08:47:14.360081+0000","last_active":"2024-09-19T08:47:15.832961+0000","last_peered":"2024-09-19T08:47:15.832961+0000","last_clean":"2024-09-19T08:47:15.832961+0000","last_became_active":"2024-09-19T08:47:14.355887+0000","last_became_peered":"2024-09-19T08:47:14.355887+0000","last_unstale":"2024-09-19T08:47:15.832961+0000","last_undegraded":"2024-09-19T08:47:15.832961+0000","last_fullsized":"2024-09-19T08:47:15.832961+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:48:47.995264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.830850+0000","last_change":"2024-09-19T08:47:14.337255+0000","last_active":"2024-09-19T08:47:15.830850+0000","last_peered":"2024-09-19T08:47:15.830850+0000","last_clean":"2024-09-19T08:47:15.830850+0000","last_became_active":"2024-09-19T08:47:14.336468+0000","last_became_peered":"2024-09-19T08:47:14.336468+0000","last_unstale":"2024-09-19T08:47:15.830850+0000","last_undegraded":"2024-09-19T08:47:15.830850+0000","last_fullsized":"2024-09-19T08:47:15.830850+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:12:56.723940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.834027+0000","last_change":"2024-09-19T08:47:14.353222+0000","last_active":"2024-09-19T08:47:15.834027+0000","last_peered":"2024-09-19T08:47:15.834027+0000","last_clean":"2024-09-19T08:47:15.834027+0000","last_became_active":"2024-09-19T08:47:14.351562+0000","last_became_peered":"2024-09-19T08:47:14.351562+0000","last_unstale":"2024-09-19T08:47:15.834027+0000","last_undegraded":"2024-09-19T08:47:15.834027+0000","last_fullsized":"2024-09-19T08:47:15.834027+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:58:45.045360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.831561+0000","last_change":"2024-09-19T08:47:14.340655+0000","last_active":"2024-09-19T08:47:15.831561+0000","last_peered":"2024-09-19T08:47:15.831561+0000","last_clean":"2024-09-19T08:47:15.831561+0000","last_became_active":"2024-09-19T08:47:14.339893+0000","last_became_peered":"2024-09-19T08:47:14.339893+0000","last_unstale":"2024-09-19T08:47:15.831561+0000","last_undegraded":"2024-09-19T08:47:15.831561+0000","last_fullsized":"2024-09-19T08:47:15.831561+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:56.648047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.831544+0000","last_change":"2024-09-19T08:47:14.350896+0000","last_active":"2024-09-19T08:47:15.831544+0000","last_peered":"2024-09-19T08:47:15.831544+0000","last_clean":"2024-09-19T08:47:15.831544+0000","last_became_active":"2024-09-19T08:47:14.349975+0000","last_became_peered":"2024-09-19T08:47:14.349975+0000","last_unstale":"2024-09-19T08:47:15.831544+0000","last_undegraded":"2024-09-19T08:47:15.831544+0000","last_fullsized":"2024-09-19T08:47:15.831544+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:50.758257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.315869+0000","last_change":"2024-09-19T08:47:14.338182+0000","last_active":"2024-09-19T08:47:15.315869+0000","last_peered":"2024-09-19T08:47:15.315869+0000","last_clean":"2024-09-19T08:47:15.315869+0000","last_became_active":"2024-09-19T08:47:14.336028+0000","last_became_peered":"2024-09-19T08:47:14.336028+0000","last_unstale":"2024-09-19T08:47:15.315869+0000","last_undegraded":"2024-09-19T08:47:15.315869+0000","last_fullsized":"2024-09-19T08:47:15.315869+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:18:40.140297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.383323+0000","last_change":"2024-09-19T08:47:14.348401+0000","last_active":"2024-09-19T08:47:15.383323+0000","last_peered":"2024-09-19T08:47:15.383323+0000","last_clean":"2024-09-19T08:47:15.383323+0000","last_became_active":"2024-09-19T08:47:14.346885+0000","last_became_peered":"2024-09-19T08:47:14.346885+0000","last_unstale":"2024-09-19T08:47:15.383323+0000","last_undegraded":"2024-09-19T08:47:15.383323+0000","last_fullsized":"2024-09-19T08:47:15.383323+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:23:37.200270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.318153+0000","last_change":"2024-09-19T08:47:14.338348+0000","last_active":"2024-09-19T08:47:15.318153+0000","last_peered":"2024-09-19T08:47:15.318153+0000","last_clean":"2024-09-19T08:47:15.318153+0000","last_became_active":"2024-09-19T08:47:14.337676+0000","last_became_peered":"2024-09-19T08:47:14.337676+0000","last_unstale":"2024-09-19T08:47:15.318153+0000","last_undegraded":"2024-09-19T08:47:15.318153+0000","last_fullsized":"2024-09-19T08:47:15.318153+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:53:47.277291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.318101+0000","last_change":"2024-09-19T08:47:14.337863+0000","last_active":"2024-09-19T08:47:15.318101+0000","last_peered":"2024-09-19T08:47:15.318101+0000","last_clean":"2024-09-19T08:47:15.318101+0000","last_became_active":"2024-09-19T08:47:14.337170+0000","last_became_peered":"2024-09-19T08:47:14.337170+0000","last_unstale":"2024-09-19T08:47:15.318101+0000","last_undegraded":"2024-09-19T08:47:15.318101+0000","last_fullsized":"2024-09-19T08:47:15.318101+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:41:06.545819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.383167+0000","last_change":"2024-09-19T08:47:14.349384+0000","last_active":"2024-09-19T08:47:15.383167+0000","last_peered":"2024-09-19T08:47:15.383167+0000","last_clean":"2024-09-19T08:47:15.383167+0000","last_became_active":"2024-09-19T08:47:14.346975+0000","last_became_peered":"2024-09-19T08:47:14.346975+0000","last_unstale":"2024-09-19T08:47:15.383167+0000","last_undegraded":"2024-09-19T08:47:15.383167+0000","last_fullsized":"2024-09-19T08:47:15.383167+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:50:15.674519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.834210+0000","last_change":"2024-09-19T08:47:14.358211+0000","last_active":"2024-09-19T08:47:15.834210+0000","last_peered":"2024-09-19T08:47:15.834210+0000","last_clean":"2024-09-19T08:47:15.834210+0000","last_became_active":"2024-09-19T08:47:14.356031+0000","last_became_peered":"2024-09-19T08:47:14.356031+0000","last_unstale":"2024-09-19T08:47:15.834210+0000","last_undegraded":"2024-09-19T08:47:15.834210+0000","last_fullsized":"2024-09-19T08:47:15.834210+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:14:56.115719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833588+0000","last_change":"2024-09-19T08:47:14.351542+0000","last_active":"2024-09-19T08:47:15.833588+0000","last_peered":"2024-09-19T08:47:15.833588+0000","last_clean":"2024-09-19T08:47:15.833588+0000","last_became_active":"2024-09-19T08:47:14.348510+0000","last_became_peered":"2024-09-19T08:47:14.348510+0000","last_unstale":"2024-09-19T08:47:15.833588+0000","last_undegraded":"2024-09-19T08:47:15.833588+0000","last_fullsized":"2024-09-19T08:47:15.833588+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:06:30.241912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.388376+0000","last_change":"2024-09-19T08:47:14.343722+0000","last_active":"2024-09-19T08:47:15.388376+0000","last_peered":"2024-09-19T08:47:15.388376+0000","last_clean":"2024-09-19T08:47:15.388376+0000","last_became_active":"2024-09-19T08:47:14.342950+0000","last_became_peered":"2024-09-19T08:47:14.342950+0000","last_unstale":"2024-09-19T08:47:15.388376+0000","last_undegraded":"2024-09-19T08:47:15.388376+0000","last_fullsized":"2024-09-19T08:47:15.388376+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:52.353505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.700591+0000","last_change":"2024-09-19T08:47:14.354023+0000","last_active":"2024-09-19T08:47:15.700591+0000","last_peered":"2024-09-19T08:47:15.700591+0000","last_clean":"2024-09-19T08:47:15.700591+0000","last_became_active":"2024-09-19T08:47:14.347846+0000","last_became_peered":"2024-09-19T08:47:14.347846+0000","last_unstale":"2024-09-19T08:47:15.700591+0000","last_undegraded":"2024-09-19T08:47:15.700591+0000","last_fullsized":"2024-09-19T08:47:15.700591+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:29:56.943652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.388148+0000","last_change":"2024-09-19T08:47:14.343483+0000","last_active":"2024-09-19T08:47:15.388148+0000","last_peered":"2024-09-19T08:47:15.388148+0000","last_clean":"2024-09-19T08:47:15.388148+0000","last_became_active":"2024-09-19T08:47:14.342737+0000","last_became_peered":"2024-09-19T08:47:14.342737+0000","last_unstale":"2024-09-19T08:47:15.388148+0000","last_undegraded":"2024-09-19T08:47:15.388148+0000","last_fullsized":"2024-09-19T08:47:15.388148+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:23:35.024819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833688+0000","last_change":"2024-09-19T08:47:14.354661+0000","last_active":"2024-09-19T08:47:15.833688+0000","last_peered":"2024-09-19T08:47:15.833688+0000","last_clean":"2024-09-19T08:47:15.833688+0000","last_became_active":"2024-09-19T08:47:14.353824+0000","last_became_peered":"2024-09-19T08:47:14.353824+0000","last_unstale":"2024-09-19T08:47:15.833688+0000","last_undegraded":"2024-09-19T08:47:15.833688+0000","last_fullsized":"2024-09-19T08:47:15.833688+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:15:39.112225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.830773+0000","last_change":"2024-09-19T08:47:14.346187+0000","last_active":"2024-09-19T08:47:15.830773+0000","last_peered":"2024-09-19T08:47:15.830773+0000","last_clean":"2024-09-19T08:47:15.830773+0000","last_became_active":"2024-09-19T08:47:14.339726+0000","last_became_peered":"2024-09-19T08:47:14.339726+0000","last_unstale":"2024-09-19T08:47:15.830773+0000","last_undegraded":"2024-09-19T08:47:15.830773+0000","last_fullsized":"2024-09-19T08:47:15.830773+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:50:01.753982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833104+0000","last_change":"2024-09-19T08:47:14.352760+0000","last_active":"2024-09-19T08:47:15.833104+0000","last_peered":"2024-09-19T08:47:15.833104+0000","last_clean":"2024-09-19T08:47:15.833104+0000","last_became_active":"2024-09-19T08:47:14.352060+0000","last_became_peered":"2024-09-19T08:47:14.352060+0000","last_unstale":"2024-09-19T08:47:15.833104+0000","last_undegraded":"2024-09-19T08:47:15.833104+0000","last_fullsized":"2024-09-19T08:47:15.833104+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:35:03.456650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.834160+0000","last_change":"2024-09-19T08:47:14.362807+0000","last_active":"2024-09-19T08:47:15.834160+0000","last_peered":"2024-09-19T08:47:15.834160+0000","last_clean":"2024-09-19T08:47:15.834160+0000","last_became_active":"2024-09-19T08:47:14.361708+0000","last_became_peered":"2024-09-19T08:47:14.361708+0000","last_unstale":"2024-09-19T08:47:15.834160+0000","last_undegraded":"2024-09-19T08:47:15.834160+0000","last_fullsized":"2024-09-19T08:47:15.834160+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:58:18.390110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"169'4","reported_seq":18,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.388872+0000","last_change":"2024-09-19T08:47:13.401720+0000","last_active":"2024-09-19T08:47:15.388872+0000","last_peered":"2024-09-19T08:47:15.388872+0000","last_clean":"2024-09-19T08:47:15.388872+0000","last_became_active":"2024-09-19T08:47:13.400995+0000","last_became_peered":"2024-09-19T08:47:13.400995+0000","last_unstale":"2024-09-19T08:47:15.388872+0000","last_undegraded":"2024-09-19T08:47:15.388872+0000","last_fullsized":"2024-09-19T08:47:15.388872+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:55.697252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.319123+0000","last_change":"2024-09-19T08:47:14.334414+0000","last_active":"2024-09-19T08:47:15.319123+0000","last_peered":"2024-09-19T08:47:15.319123+0000","last_clean":"2024-09-19T08:47:15.319123+0000","last_became_active":"2024-09-19T08:47:14.333618+0000","last_became_peered":"2024-09-19T08:47:14.333618+0000","last_unstale":"2024-09-19T08:47:15.319123+0000","last_undegraded":"2024-09-19T08:47:15.319123+0000","last_fullsized":"2024-09-19T08:47:15.319123+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:10:52.949042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"169'1","reported_seq":15,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.834325+0000","last_change":"2024-09-19T08:47:13.411165+0000","last_active":"2024-09-19T08:47:15.834325+0000","last_peered":"2024-09-19T08:47:15.834325+0000","last_clean":"2024-09-19T08:47:15.834325+0000","last_became_active":"2024-09-19T08:47:13.410431+0000","last_became_peered":"2024-09-19T08:47:13.410431+0000","last_unstale":"2024-09-19T08:47:15.834325+0000","last_undegraded":"2024-09-19T08:47:15.834325+0000","last_fullsized":"2024-09-19T08:47:15.834325+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:03:04.142471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.834365+0000","last_change":"2024-09-19T08:47:14.347732+0000","last_active":"2024-09-19T08:47:15.834365+0000","last_peered":"2024-09-19T08:47:15.834365+0000","last_clean":"2024-09-19T08:47:15.834365+0000","last_became_active":"2024-09-19T08:47:14.346863+0000","last_became_peered":"2024-09-19T08:47:14.346863+0000","last_unstale":"2024-09-19T08:47:15.834365+0000","last_undegraded":"2024-09-19T08:47:15.834365+0000","last_fullsized":"2024-09-19T08:47:15.834365+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:31:49.532264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"169'2","reported_seq":16,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.701304+0000","last_change":"2024-09-19T08:47:13.393085+0000","last_active":"2024-09-19T08:47:15.701304+0000","last_peered":"2024-09-19T08:47:15.701304+0000","last_clean":"2024-09-19T08:47:15.701304+0000","last_became_active":"2024-09-19T08:47:13.392776+0000","last_became_peered":"2024-09-19T08:47:13.392776+0000","last_unstale":"2024-09-19T08:47:15.701304+0000","last_undegraded":"2024-09-19T08:47:15.701304+0000","last_fullsized":"2024-09-19T08:47:15.701304+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:45:50.989483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.831442+0000","last_change":"2024-09-19T08:47:14.345730+0000","last_active":"2024-09-19T08:47:15.831442+0000","last_peered":"2024-09-19T08:47:15.831442+0000","last_clean":"2024-09-19T08:47:15.831442+0000","last_became_active":"2024-09-19T08:47:14.345014+0000","last_became_peered":"2024-09-19T08:47:14.345014+0000","last_unstale":"2024-09-19T08:47:15.831442+0000","last_undegraded":"2024-09-19T08:47:15.831442+0000","last_fullsized":"2024-09-19T08:47:15.831442+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:17.675241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"169'2","reported_seq":16,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.384194+0000","last_change":"2024-09-19T08:47:13.403896+0000","last_active":"2024-09-19T08:47:15.384194+0000","last_peered":"2024-09-19T08:47:15.384194+0000","last_clean":"2024-09-19T08:47:15.384194+0000","last_became_active":"2024-09-19T08:47:13.403168+0000","last_became_peered":"2024-09-19T08:47:13.403168+0000","last_unstale":"2024-09-19T08:47:15.384194+0000","last_undegraded":"2024-09-19T08:47:15.384194+0000","last_fullsized":"2024-09-19T08:47:15.384194+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:50:14.051718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.383345+0000","last_change":"2024-09-19T08:47:14.351675+0000","last_active":"2024-09-19T08:47:15.383345+0000","last_peered":"2024-09-19T08:47:15.383345+0000","last_clean":"2024-09-19T08:47:15.383345+0000","last_became_active":"2024-09-19T08:47:14.350955+0000","last_became_peered":"2024-09-19T08:47:14.350955+0000","last_unstale":"2024-09-19T08:47:15.383345+0000","last_undegraded":"2024-09-19T08:47:15.383345+0000","last_fullsized":"2024-09-19T08:47:15.383345+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:44.489711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"169'1","reported_seq":15,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.314986+0000","last_change":"2024-09-19T08:47:13.400567+0000","last_active":"2024-09-19T08:47:15.314986+0000","last_peered":"2024-09-19T08:47:15.314986+0000","last_clean":"2024-09-19T08:47:15.314986+0000","last_became_active":"2024-09-19T08:47:13.400297+0000","last_became_peered":"2024-09-19T08:47:13.400297+0000","last_unstale":"2024-09-19T08:47:15.314986+0000","last_undegraded":"2024-09-19T08:47:15.314986+0000","last_fullsized":"2024-09-19T08:47:15.314986+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:14:37.029395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.314387+0000","last_change":"2024-09-19T08:47:14.341230+0000","last_active":"2024-09-19T08:47:15.314387+0000","last_peered":"2024-09-19T08:47:15.314387+0000","last_clean":"2024-09-19T08:47:15.314387+0000","last_became_active":"2024-09-19T08:47:14.340408+0000","last_became_peered":"2024-09-19T08:47:14.340408+0000","last_unstale":"2024-09-19T08:47:15.314387+0000","last_undegraded":"2024-09-19T08:47:15.314387+0000","last_fullsized":"2024-09-19T08:47:15.314387+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:27:37.674410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.314321+0000","last_change":"2024-09-19T08:47:13.382448+0000","last_active":"2024-09-19T08:47:15.314321+0000","last_peered":"2024-09-19T08:47:15.314321+0000","last_clean":"2024-09-19T08:47:15.314321+0000","last_became_active":"2024-09-19T08:47:13.382136+0000","last_became_peered":"2024-09-19T08:47:13.382136+0000","last_unstale":"2024-09-19T08:47:15.314321+0000","last_undegraded":"2024-09-19T08:47:15.314321+0000","last_fullsized":"2024-09-19T08:47:15.314321+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:31.792660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.383313+0000","last_change":"2024-09-19T08:47:14.354758+0000","last_active":"2024-09-19T08:47:15.383313+0000","last_peered":"2024-09-19T08:47:15.383313+0000","last_clean":"2024-09-19T08:47:15.383313+0000","last_became_active":"2024-09-19T08:47:14.353999+0000","last_became_peered":"2024-09-19T08:47:14.353999+0000","last_unstale":"2024-09-19T08:47:15.383313+0000","last_undegraded":"2024-09-19T08:47:15.383313+0000","last_fullsized":"2024-09-19T08:47:15.383313+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:18.467994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.317703+0000","last_change":"2024-09-19T08:47:13.394882+0000","last_active":"2024-09-19T08:47:15.317703+0000","last_peered":"2024-09-19T08:47:15.317703+0000","last_clean":"2024-09-19T08:47:15.317703+0000","last_became_active":"2024-09-19T08:47:13.393932+0000","last_became_peered":"2024-09-19T08:47:13.393932+0000","last_unstale":"2024-09-19T08:47:15.317703+0000","last_undegraded":"2024-09-19T08:47:15.317703+0000","last_fullsized":"2024-09-19T08:47:15.317703+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:47:52.284449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.384004+0000","last_change":"2024-09-19T08:47:14.365557+0000","last_active":"2024-09-19T08:47:15.384004+0000","last_peered":"2024-09-19T08:47:15.384004+0000","last_clean":"2024-09-19T08:47:15.384004+0000","last_became_active":"2024-09-19T08:47:14.364879+0000","last_became_peered":"2024-09-19T08:47:14.364879+0000","last_unstale":"2024-09-19T08:47:15.384004+0000","last_undegraded":"2024-09-19T08:47:15.384004+0000","last_fullsized":"2024-09-19T08:47:15.384004+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:51:01.879120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.314681+0000","last_change":"2024-09-19T08:47:13.398554+0000","last_active":"2024-09-19T08:47:15.314681+0000","last_peered":"2024-09-19T08:47:15.314681+0000","last_clean":"2024-09-19T08:47:15.314681+0000","last_became_active":"2024-09-19T08:47:13.397871+0000","last_became_peered":"2024-09-19T08:47:13.397871+0000","last_unstale":"2024-09-19T08:47:15.314681+0000","last_undegraded":"2024-09-19T08:47:15.314681+0000","last_fullsized":"2024-09-19T08:47:15.314681+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:41.165756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.701266+0000","last_change":"2024-09-19T08:47:14.346956+0000","last_active":"2024-09-19T08:47:15.701266+0000","last_peered":"2024-09-19T08:47:15.701266+0000","last_clean":"2024-09-19T08:47:15.701266+0000","last_became_active":"2024-09-19T08:47:14.346229+0000","last_became_peered":"2024-09-19T08:47:14.346229+0000","last_unstale":"2024-09-19T08:47:15.701266+0000","last_undegraded":"2024-09-19T08:47:15.701266+0000","last_fullsized":"2024-09-19T08:47:15.701266+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:39:52.187988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.319341+0000","last_change":"2024-09-19T08:47:13.407203+0000","last_active":"2024-09-19T08:47:15.319341+0000","last_peered":"2024-09-19T08:47:15.319341+0000","last_clean":"2024-09-19T08:47:15.319341+0000","last_became_active":"2024-09-19T08:47:13.406419+0000","last_became_peered":"2024-09-19T08:47:13.406419+0000","last_unstale":"2024-09-19T08:47:15.319341+0000","last_undegraded":"2024-09-19T08:47:15.319341+0000","last_fullsized":"2024-09-19T08:47:15.319341+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:31.792660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.384218+0000","last_change":"2024-09-19T08:47:14.349482+0000","last_active":"2024-09-19T08:47:15.384218+0000","last_peered":"2024-09-19T08:47:15.384218+0000","last_clean":"2024-09-19T08:47:15.384218+0000","last_became_active":"2024-09-19T08:47:14.348750+0000","last_became_peered":"2024-09-19T08:47:14.348750+0000","last_unstale":"2024-09-19T08:47:15.384218+0000","last_undegraded":"2024-09-19T08:47:15.384218+0000","last_fullsized":"2024-09-19T08:47:15.384218+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:47:56.893479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"169'3","reported_seq":17,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.831832+0000","last_change":"2024-09-19T08:47:13.382865+0000","last_active":"2024-09-19T08:47:15.831832+0000","last_peered":"2024-09-19T08:47:15.831832+0000","last_clean":"2024-09-19T08:47:15.831832+0000","last_became_active":"2024-09-19T08:47:13.382528+0000","last_became_peered":"2024-09-19T08:47:13.382528+0000","last_unstale":"2024-09-19T08:47:15.831832+0000","last_undegraded":"2024-09-19T08:47:15.831832+0000","last_fullsized":"2024-09-19T08:47:15.831832+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:09:21.216354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833935+0000","last_change":"2024-09-19T08:47:14.347539+0000","last_active":"2024-09-19T08:47:15.833935+0000","last_peered":"2024-09-19T08:47:15.833935+0000","last_clean":"2024-09-19T08:47:15.833935+0000","last_became_active":"2024-09-19T08:47:14.346707+0000","last_became_peered":"2024-09-19T08:47:14.346707+0000","last_unstale":"2024-09-19T08:47:15.833935+0000","last_undegraded":"2024-09-19T08:47:15.833935+0000","last_fullsized":"2024-09-19T08:47:15.833935+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:28:44.021408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"169'6","reported_seq":20,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.384347+0000","last_change":"2024-09-19T08:47:13.409867+0000","last_active":"2024-09-19T08:47:15.384347+0000","last_peered":"2024-09-19T08:47:15.384347+0000","last_clean":"2024-09-19T08:47:15.384347+0000","last_became_active":"2024-09-19T08:47:13.408810+0000","last_became_peered":"2024-09-19T08:47:13.408810+0000","last_unstale":"2024-09-19T08:47:15.384347+0000","last_undegraded":"2024-09-19T08:47:15.384347+0000","last_fullsized":"2024-09-19T08:47:15.384347+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:27:04.792972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.314861+0000","last_change":"2024-09-19T08:47:14.352505+0000","last_active":"2024-09-19T08:47:15.314861+0000","last_peered":"2024-09-19T08:47:15.314861+0000","last_clean":"2024-09-19T08:47:15.314861+0000","last_became_active":"2024-09-19T08:47:14.351672+0000","last_became_peered":"2024-09-19T08:47:14.351672+0000","last_unstale":"2024-09-19T08:47:15.314861+0000","last_undegraded":"2024-09-19T08:47:15.314861+0000","last_fullsized":"2024-09-19T08:47:15.314861+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:59:09.201492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"169'2","reported_seq":16,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.318763+0000","last_change":"2024-09-19T08:47:13.404129+0000","last_active":"2024-09-19T08:47:15.318763+0000","last_peered":"2024-09-19T08:47:15.318763+0000","last_clean":"2024-09-19T08:47:15.318763+0000","last_became_active":"2024-09-19T08:47:13.403371+0000","last_became_peered":"2024-09-19T08:47:13.403371+0000","last_unstale":"2024-09-19T08:47:15.318763+0000","last_undegraded":"2024-09-19T08:47:15.318763+0000","last_fullsized":"2024-09-19T08:47:15.318763+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:32:17.535117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.831322+0000","last_change":"2024-09-19T08:47:14.359096+0000","last_active":"2024-09-19T08:47:15.831322+0000","last_peered":"2024-09-19T08:47:15.831322+0000","last_clean":"2024-09-19T08:47:15.831322+0000","last_became_active":"2024-09-19T08:47:14.358410+0000","last_became_peered":"2024-09-19T08:47:14.358410+0000","last_unstale":"2024-09-19T08:47:15.831322+0000","last_undegraded":"2024-09-19T08:47:15.831322+0000","last_fullsized":"2024-09-19T08:47:15.831322+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:10:15.182744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.318529+0000","last_change":"2024-09-19T08:47:13.405188+0000","last_active":"2024-09-19T08:47:15.318529+0000","last_peered":"2024-09-19T08:47:15.318529+0000","last_clean":"2024-09-19T08:47:15.318529+0000","last_became_active":"2024-09-19T08:47:13.404512+0000","last_became_peered":"2024-09-19T08:47:13.404512+0000","last_unstale":"2024-09-19T08:47:15.318529+0000","last_undegraded":"2024-09-19T08:47:15.318529+0000","last_fullsized":"2024-09-19T08:47:15.318529+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:38.934425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.700267+0000","last_change":"2024-09-19T08:47:14.347839+0000","last_active":"2024-09-19T08:47:15.700267+0000","last_peered":"2024-09-19T08:47:15.700267+0000","last_clean":"2024-09-19T08:47:15.700267+0000","last_became_active":"2024-09-19T08:47:14.346533+0000","last_became_peered":"2024-09-19T08:47:14.346533+0000","last_unstale":"2024-09-19T08:47:15.700267+0000","last_undegraded":"2024-09-19T08:47:15.700267+0000","last_fullsized":"2024-09-19T08:47:15.700267+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:09:46.261549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.830605+0000","last_change":"2024-09-19T08:47:13.393731+0000","last_active":"2024-09-19T08:47:15.830605+0000","last_peered":"2024-09-19T08:47:15.830605+0000","last_clean":"2024-09-19T08:47:15.830605+0000","last_became_active":"2024-09-19T08:47:13.393445+0000","last_became_peered":"2024-09-19T08:47:13.393445+0000","last_unstale":"2024-09-19T08:47:15.830605+0000","last_undegraded":"2024-09-19T08:47:15.830605+0000","last_fullsized":"2024-09-19T08:47:15.830605+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:17:50.481692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833216+0000","last_change":"2024-09-19T08:47:14.362136+0000","last_active":"2024-09-19T08:47:15.833216+0000","last_peered":"2024-09-19T08:47:15.833216+0000","last_clean":"2024-09-19T08:47:15.833216+0000","last_became_active":"2024-09-19T08:47:14.360937+0000","last_became_peered":"2024-09-19T08:47:14.360937+0000","last_unstale":"2024-09-19T08:47:15.833216+0000","last_undegraded":"2024-09-19T08:47:15.833216+0000","last_fullsized":"2024-09-19T08:47:15.833216+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:09:33.724240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"169'8","reported_seq":22,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.700621+0000","last_change":"2024-09-19T08:47:13.393111+0000","last_active":"2024-09-19T08:47:15.700621+0000","last_peered":"2024-09-19T08:47:15.700621+0000","last_clean":"2024-09-19T08:47:15.700621+0000","last_became_active":"2024-09-19T08:47:13.392754+0000","last_became_peered":"2024-09-19T08:47:13.392754+0000","last_unstale":"2024-09-19T08:47:15.700621+0000","last_undegraded":"2024-09-19T08:47:15.700621+0000","last_fullsized":"2024-09-19T08:47:15.700621+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:47:39.662192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.834342+0000","last_change":"2024-09-19T08:47:14.363789+0000","last_active":"2024-09-19T08:47:15.834342+0000","last_peered":"2024-09-19T08:47:15.834342+0000","last_clean":"2024-09-19T08:47:15.834342+0000","last_became_active":"2024-09-19T08:47:14.362514+0000","last_became_peered":"2024-09-19T08:47:14.362514+0000","last_unstale":"2024-09-19T08:47:15.834342+0000","last_undegraded":"2024-09-19T08:47:15.834342+0000","last_fullsized":"2024-09-19T08:47:15.834342+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:45.701737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"169'1","reported_seq":15,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.314766+0000","last_change":"2024-09-19T08:47:13.383220+0000","last_active":"2024-09-19T08:47:15.314766+0000","last_peered":"2024-09-19T08:47:15.314766+0000","last_clean":"2024-09-19T08:47:15.314766+0000","last_became_active":"2024-09-19T08:47:13.382953+0000","last_became_peered":"2024-09-19T08:47:13.382953+0000","last_unstale":"2024-09-19T08:47:15.314766+0000","last_undegraded":"2024-09-19T08:47:15.314766+0000","last_fullsized":"2024-09-19T08:47:15.314766+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:24.754354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.834207+0000","last_change":"2024-09-19T08:47:14.358774+0000","last_active":"2024-09-19T08:47:15.834207+0000","last_peered":"2024-09-19T08:47:15.834207+0000","last_clean":"2024-09-19T08:47:15.834207+0000","last_became_active":"2024-09-19T08:47:14.356848+0000","last_became_peered":"2024-09-19T08:47:14.356848+0000","last_unstale":"2024-09-19T08:47:15.834207+0000","last_undegraded":"2024-09-19T08:47:15.834207+0000","last_fullsized":"2024-09-19T08:47:15.834207+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:32:24.678912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"169'2","reported_seq":16,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.834488+0000","last_change":"2024-09-19T08:47:13.408636+0000","last_active":"2024-09-19T08:47:15.834488+0000","last_peered":"2024-09-19T08:47:15.834488+0000","last_clean":"2024-09-19T08:47:15.834488+0000","last_became_active":"2024-09-19T08:47:13.407243+0000","last_became_peered":"2024-09-19T08:47:13.407243+0000","last_unstale":"2024-09-19T08:47:15.834488+0000","last_undegraded":"2024-09-19T08:47:15.834488+0000","last_fullsized":"2024-09-19T08:47:15.834488+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:43:59.000541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.700602+0000","last_change":"2024-09-19T08:47:14.355537+0000","last_active":"2024-09-19T08:47:15.700602+0000","last_peered":"2024-09-19T08:47:15.700602+0000","last_clean":"2024-09-19T08:47:15.700602+0000","last_became_active":"2024-09-19T08:47:14.354555+0000","last_became_peered":"2024-09-19T08:47:14.354555+0000","last_unstale":"2024-09-19T08:47:15.700602+0000","last_undegraded":"2024-09-19T08:47:15.700602+0000","last_fullsized":"2024-09-19T08:47:15.700602+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:44.668862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"169'1","reported_seq":15,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.831941+0000","last_change":"2024-09-19T08:47:13.382929+0000","last_active":"2024-09-19T08:47:15.831941+0000","last_peered":"2024-09-19T08:47:15.831941+0000","last_clean":"2024-09-19T08:47:15.831941+0000","last_became_active":"2024-09-19T08:47:13.382644+0000","last_became_peered":"2024-09-19T08:47:13.382644+0000","last_unstale":"2024-09-19T08:47:15.831941+0000","last_undegraded":"2024-09-19T08:47:15.831941+0000","last_fullsized":"2024-09-19T08:47:15.831941+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:08:23.245952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.831118+0000","last_change":"2024-09-19T08:47:14.345326+0000","last_active":"2024-09-19T08:47:15.831118+0000","last_peered":"2024-09-19T08:47:15.831118+0000","last_clean":"2024-09-19T08:47:15.831118+0000","last_became_active":"2024-09-19T08:47:14.340690+0000","last_became_peered":"2024-09-19T08:47:14.340690+0000","last_unstale":"2024-09-19T08:47:15.831118+0000","last_undegraded":"2024-09-19T08:47:15.831118+0000","last_fullsized":"2024-09-19T08:47:15.831118+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:34.978283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.700881+0000","last_change":"2024-09-19T08:47:13.382212+0000","last_active":"2024-09-19T08:47:15.700881+0000","last_peered":"2024-09-19T08:47:15.700881+0000","last_clean":"2024-09-19T08:47:15.700881+0000","last_became_active":"2024-09-19T08:47:13.381878+0000","last_became_peered":"2024-09-19T08:47:13.381878+0000","last_unstale":"2024-09-19T08:47:15.700881+0000","last_undegraded":"2024-09-19T08:47:15.700881+0000","last_fullsized":"2024-09-19T08:47:15.700881+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:43.813585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.831842+0000","last_change":"2024-09-19T08:47:14.346182+0000","last_active":"2024-09-19T08:47:15.831842+0000","last_peered":"2024-09-19T08:47:15.831842+0000","last_clean":"2024-09-19T08:47:15.831842+0000","last_became_active":"2024-09-19T08:47:14.345377+0000","last_became_peered":"2024-09-19T08:47:14.345377+0000","last_unstale":"2024-09-19T08:47:15.831842+0000","last_undegraded":"2024-09-19T08:47:15.831842+0000","last_fullsized":"2024-09-19T08:47:15.831842+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:23:59.978626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"169'2","reported_seq":16,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.831403+0000","last_change":"2024-09-19T08:47:13.384584+0000","last_active":"2024-09-19T08:47:15.831403+0000","last_peered":"2024-09-19T08:47:15.831403+0000","last_clean":"2024-09-19T08:47:15.831403+0000","last_became_active":"2024-09-19T08:47:13.384215+0000","last_became_peered":"2024-09-19T08:47:13.384215+0000","last_unstale":"2024-09-19T08:47:15.831403+0000","last_undegraded":"2024-09-19T08:47:15.831403+0000","last_fullsized":"2024-09-19T08:47:15.831403+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:19.499642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.700643+0000","last_change":"2024-09-19T08:47:14.353176+0000","last_active":"2024-09-19T08:47:15.700643+0000","last_peered":"2024-09-19T08:47:15.700643+0000","last_clean":"2024-09-19T08:47:15.700643+0000","last_became_active":"2024-09-19T08:47:14.347339+0000","last_became_peered":"2024-09-19T08:47:14.347339+0000","last_unstale":"2024-09-19T08:47:15.700643+0000","last_undegraded":"2024-09-19T08:47:15.700643+0000","last_fullsized":"2024-09-19T08:47:15.700643+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:09:59.841644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833567+0000","last_change":"2024-09-19T08:47:13.393941+0000","last_active":"2024-09-19T08:47:15.833567+0000","last_peered":"2024-09-19T08:47:15.833567+0000","last_clean":"2024-09-19T08:47:15.833567+0000","last_became_active":"2024-09-19T08:47:13.393683+0000","last_became_peered":"2024-09-19T08:47:13.393683+0000","last_unstale":"2024-09-19T08:47:15.833567+0000","last_undegraded":"2024-09-19T08:47:15.833567+0000","last_fullsized":"2024-09-19T08:47:15.833567+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:25:29.678095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833498+0000","last_change":"2024-09-19T08:47:14.354662+0000","last_active":"2024-09-19T08:47:15.833498+0000","last_peered":"2024-09-19T08:47:15.833498+0000","last_clean":"2024-09-19T08:47:15.833498+0000","last_became_active":"2024-09-19T08:47:14.353887+0000","last_became_peered":"2024-09-19T08:47:14.353887+0000","last_unstale":"2024-09-19T08:47:15.833498+0000","last_undegraded":"2024-09-19T08:47:15.833498+0000","last_fullsized":"2024-09-19T08:47:15.833498+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:19.026785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833264+0000","last_change":"2024-09-19T08:47:13.404413+0000","last_active":"2024-09-19T08:47:15.833264+0000","last_peered":"2024-09-19T08:47:15.833264+0000","last_clean":"2024-09-19T08:47:15.833264+0000","last_became_active":"2024-09-19T08:47:13.404206+0000","last_became_peered":"2024-09-19T08:47:13.404206+0000","last_unstale":"2024-09-19T08:47:15.833264+0000","last_undegraded":"2024-09-19T08:47:15.833264+0000","last_fullsized":"2024-09-19T08:47:15.833264+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:05:09.485498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.318156+0000","last_change":"2024-09-19T08:47:14.340136+0000","last_active":"2024-09-19T08:47:15.318156+0000","last_peered":"2024-09-19T08:47:15.318156+0000","last_clean":"2024-09-19T08:47:15.318156+0000","last_became_active":"2024-09-19T08:47:14.337946+0000","last_became_peered":"2024-09-19T08:47:14.337946+0000","last_unstale":"2024-09-19T08:47:15.318156+0000","last_undegraded":"2024-09-19T08:47:15.318156+0000","last_fullsized":"2024-09-19T08:47:15.318156+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:03.353387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.314696+0000","last_change":"2024-09-19T08:47:13.387174+0000","last_active":"2024-09-19T08:47:15.314696+0000","last_peered":"2024-09-19T08:47:15.314696+0000","last_clean":"2024-09-19T08:47:15.314696+0000","last_became_active":"2024-09-19T08:47:13.386875+0000","last_became_peered":"2024-09-19T08:47:13.386875+0000","last_unstale":"2024-09-19T08:47:15.314696+0000","last_undegraded":"2024-09-19T08:47:15.314696+0000","last_fullsized":"2024-09-19T08:47:15.314696+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:22:04.699154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.388462+0000","last_change":"2024-09-19T08:47:14.344815+0000","last_active":"2024-09-19T08:47:15.388462+0000","last_peered":"2024-09-19T08:47:15.388462+0000","last_clean":"2024-09-19T08:47:15.388462+0000","last_became_active":"2024-09-19T08:47:14.343956+0000","last_became_peered":"2024-09-19T08:47:14.343956+0000","last_unstale":"2024-09-19T08:47:15.388462+0000","last_undegraded":"2024-09-19T08:47:15.388462+0000","last_fullsized":"2024-09-19T08:47:15.388462+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:17:40.271999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"169'2","reported_seq":16,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.701112+0000","last_change":"2024-09-19T08:47:13.398116+0000","last_active":"2024-09-19T08:47:15.701112+0000","last_peered":"2024-09-19T08:47:15.701112+0000","last_clean":"2024-09-19T08:47:15.701112+0000","last_became_active":"2024-09-19T08:47:13.397684+0000","last_became_peered":"2024-09-19T08:47:13.397684+0000","last_unstale":"2024-09-19T08:47:15.701112+0000","last_undegraded":"2024-09-19T08:47:15.701112+0000","last_fullsized":"2024-09-19T08:47:15.701112+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:10:26.683853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833962+0000","last_change":"2024-09-19T08:47:14.360783+0000","last_active":"2024-09-19T08:47:15.833962+0000","last_peered":"2024-09-19T08:47:15.833962+0000","last_clean":"2024-09-19T08:47:15.833962+0000","last_became_active":"2024-09-19T08:47:14.348853+0000","last_became_peered":"2024-09-19T08:47:14.348853+0000","last_unstale":"2024-09-19T08:47:15.833962+0000","last_undegraded":"2024-09-19T08:47:15.833962+0000","last_fullsized":"2024-09-19T08:47:15.833962+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:46:22.684299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"170'1","reported_seq":15,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.384227+0000","last_change":"2024-09-19T08:47:13.402409+0000","last_active":"2024-09-19T08:47:15.384227+0000","last_peered":"2024-09-19T08:47:15.384227+0000","last_clean":"2024-09-19T08:47:15.384227+0000","last_became_active":"2024-09-19T08:47:13.401624+0000","last_became_peered":"2024-09-19T08:47:13.401624+0000","last_unstale":"2024-09-19T08:47:15.384227+0000","last_undegraded":"2024-09-19T08:47:15.384227+0000","last_fullsized":"2024-09-19T08:47:15.384227+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:45.064523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833923+0000","last_change":"2024-09-19T08:47:14.354272+0000","last_active":"2024-09-19T08:47:15.833923+0000","last_peered":"2024-09-19T08:47:15.833923+0000","last_clean":"2024-09-19T08:47:15.833923+0000","last_became_active":"2024-09-19T08:47:14.353523+0000","last_became_peered":"2024-09-19T08:47:14.353523+0000","last_unstale":"2024-09-19T08:47:15.833923+0000","last_undegraded":"2024-09-19T08:47:15.833923+0000","last_fullsized":"2024-09-19T08:47:15.833923+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:49.222433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.315399+0000","last_change":"2024-09-19T08:47:13.385451+0000","last_active":"2024-09-19T08:47:15.315399+0000","last_peered":"2024-09-19T08:47:15.315399+0000","last_clean":"2024-09-19T08:47:15.315399+0000","last_became_active":"2024-09-19T08:47:13.384496+0000","last_became_peered":"2024-09-19T08:47:13.384496+0000","last_unstale":"2024-09-19T08:47:15.315399+0000","last_undegraded":"2024-09-19T08:47:15.315399+0000","last_fullsized":"2024-09-19T08:47:15.315399+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:44:38.934425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.318949+0000","last_change":"2024-09-19T08:47:14.346827+0000","last_active":"2024-09-19T08:47:15.318949+0000","last_peered":"2024-09-19T08:47:15.318949+0000","last_clean":"2024-09-19T08:47:15.318949+0000","last_became_active":"2024-09-19T08:47:14.346102+0000","last_became_peered":"2024-09-19T08:47:14.346102+0000","last_unstale":"2024-09-19T08:47:15.318949+0000","last_undegraded":"2024-09-19T08:47:15.318949+0000","last_fullsized":"2024-09-19T08:47:15.318949+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:11.917219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.700870+0000","last_change":"2024-09-19T08:47:13.398325+0000","last_active":"2024-09-19T08:47:15.700870+0000","last_peered":"2024-09-19T08:47:15.700870+0000","last_clean":"2024-09-19T08:47:15.700870+0000","last_became_active":"2024-09-19T08:47:13.397450+0000","last_became_peered":"2024-09-19T08:47:13.397450+0000","last_unstale":"2024-09-19T08:47:15.700870+0000","last_undegraded":"2024-09-19T08:47:15.700870+0000","last_fullsized":"2024-09-19T08:47:15.700870+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:49:23.696716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833467+0000","last_change":"2024-09-19T08:47:14.344698+0000","last_active":"2024-09-19T08:47:15.833467+0000","last_peered":"2024-09-19T08:47:15.833467+0000","last_clean":"2024-09-19T08:47:15.833467+0000","last_became_active":"2024-09-19T08:47:14.343844+0000","last_became_peered":"2024-09-19T08:47:14.343844+0000","last_unstale":"2024-09-19T08:47:15.833467+0000","last_undegraded":"2024-09-19T08:47:15.833467+0000","last_fullsized":"2024-09-19T08:47:15.833467+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:25:42.320274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.383623+0000","last_change":"2024-09-19T08:47:13.399625+0000","last_active":"2024-09-19T08:47:15.383623+0000","last_peered":"2024-09-19T08:47:15.383623+0000","last_clean":"2024-09-19T08:47:15.383623+0000","last_became_active":"2024-09-19T08:47:13.398838+0000","last_became_peered":"2024-09-19T08:47:13.398838+0000","last_unstale":"2024-09-19T08:47:15.383623+0000","last_undegraded":"2024-09-19T08:47:15.383623+0000","last_fullsized":"2024-09-19T08:47:15.383623+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:26:50.957727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.388653+0000","last_change":"2024-09-19T08:47:14.343451+0000","last_active":"2024-09-19T08:47:15.388653+0000","last_peered":"2024-09-19T08:47:15.388653+0000","last_clean":"2024-09-19T08:47:15.388653+0000","last_became_active":"2024-09-19T08:47:14.342662+0000","last_became_peered":"2024-09-19T08:47:14.342662+0000","last_unstale":"2024-09-19T08:47:15.388653+0000","last_undegraded":"2024-09-19T08:47:15.388653+0000","last_fullsized":"2024-09-19T08:47:15.388653+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:02:13.929162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.315041+0000","last_change":"2024-09-19T08:47:13.384922+0000","last_active":"2024-09-19T08:47:15.315041+0000","last_peered":"2024-09-19T08:47:15.315041+0000","last_clean":"2024-09-19T08:47:15.315041+0000","last_became_active":"2024-09-19T08:47:13.384555+0000","last_became_peered":"2024-09-19T08:47:13.384555+0000","last_unstale":"2024-09-19T08:47:15.315041+0000","last_undegraded":"2024-09-19T08:47:15.315041+0000","last_fullsized":"2024-09-19T08:47:15.315041+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:27:36.013002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.700392+0000","last_change":"2024-09-19T08:47:14.347684+0000","last_active":"2024-09-19T08:47:15.700392+0000","last_peered":"2024-09-19T08:47:15.700392+0000","last_clean":"2024-09-19T08:47:15.700392+0000","last_became_active":"2024-09-19T08:47:14.345594+0000","last_became_peered":"2024-09-19T08:47:14.345594+0000","last_unstale":"2024-09-19T08:47:15.700392+0000","last_undegraded":"2024-09-19T08:47:15.700392+0000","last_fullsized":"2024-09-19T08:47:15.700392+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:11.450025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.318402+0000","last_change":"2024-09-19T08:47:13.407182+0000","last_active":"2024-09-19T08:47:15.318402+0000","last_peered":"2024-09-19T08:47:15.318402+0000","last_clean":"2024-09-19T08:47:15.318402+0000","last_became_active":"2024-09-19T08:47:13.405978+0000","last_became_peered":"2024-09-19T08:47:13.405978+0000","last_unstale":"2024-09-19T08:47:15.318402+0000","last_undegraded":"2024-09-19T08:47:15.318402+0000","last_fullsized":"2024-09-19T08:47:15.318402+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:54:41.165756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833516+0000","last_change":"2024-09-19T08:47:14.354669+0000","last_active":"2024-09-19T08:47:15.833516+0000","last_peered":"2024-09-19T08:47:15.833516+0000","last_clean":"2024-09-19T08:47:15.833516+0000","last_became_active":"2024-09-19T08:47:14.353479+0000","last_became_peered":"2024-09-19T08:47:14.353479+0000","last_unstale":"2024-09-19T08:47:15.833516+0000","last_undegraded":"2024-09-19T08:47:15.833516+0000","last_fullsized":"2024-09-19T08:47:15.833516+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T19:37:58.191552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.387876+0000","last_change":"2024-09-19T08:47:13.386481+0000","last_active":"2024-09-19T08:47:15.387876+0000","last_peered":"2024-09-19T08:47:15.387876+0000","last_clean":"2024-09-19T08:47:15.387876+0000","last_became_active":"2024-09-19T08:47:13.385716+0000","last_became_peered":"2024-09-19T08:47:13.385716+0000","last_unstale":"2024-09-19T08:47:15.387876+0000","last_undegraded":"2024-09-19T08:47:15.387876+0000","last_fullsized":"2024-09-19T08:47:15.387876+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:22.519133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.833065+0000","last_change":"2024-09-19T08:47:14.354191+0000","last_active":"2024-09-19T08:47:15.833065+0000","last_peered":"2024-09-19T08:47:15.833065+0000","last_clean":"2024-09-19T08:47:15.833065+0000","last_became_active":"2024-09-19T08:47:14.348271+0000","last_became_peered":"2024-09-19T08:47:14.348271+0000","last_unstale":"2024-09-19T08:47:15.833065+0000","last_undegraded":"2024-09-19T08:47:15.833065+0000","last_fullsized":"2024-09-19T08:47:15.833065+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:13.355035+0000","last_clean_scrub_stamp":"2024-09-19T08:47:13.355035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:34:12.965823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"169'1","reported_seq":15,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.388912+0000","last_change":"2024-09-19T08:47:13.388601+0000","last_active":"2024-09-19T08:47:15.388912+0000","last_peered":"2024-09-19T08:47:15.388912+0000","last_clean":"2024-09-19T08:47:15.388912+0000","last_became_active":"2024-09-19T08:47:13.387981+0000","last_became_peered":"2024-09-19T08:47:13.387981+0000","last_unstale":"2024-09-19T08:47:15.388912+0000","last_undegraded":"2024-09-19T08:47:15.388912+0000","last_fullsized":"2024-09-19T08:47:15.388912+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:47:12.350565+0000","last_clean_scrub_stamp":"2024-09-19T08:47:12.350565+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:37:04.694583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"71'128","reported_seq":478,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.383426+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:47:15.383426+0000","last_peered":"2024-09-19T08:47:15.383426+0000","last_clean":"2024-09-19T08:47:15.383426+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:47:15.383426+0000","last_undegraded":"2024-09-19T08:47:15.383426+0000","last_fullsized":"2024-09-19T08:47:15.383426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509553,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33868,"kb_used_data":2680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337972,"statfs":{"total":96636764160,"available":96602083328,"internally_reserved":0,"allocated":2744320,"data_stored":1542832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574962,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33288,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2142208,"data_stored":951434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509555,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33280,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338560,"statfs":{"total":96636764160,"available":96602685440,"internally_reserved":0,"allocated":2142208,"data_stored":951434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509555,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33300,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338540,"statfs":{"total":96636764160,"available":96602664960,"internally_reserved":0,"allocated":2158592,"data_stored":952568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542257,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37448,"kb_used_data":2100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334392,"statfs":{"total":96636764160,"available":96598417408,"internally_reserved":0,"allocated":2150400,"data_stored":951034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509552,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33276,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338564,"statfs":{"total":96636764160,"available":96602689536,"internally_reserved":0,"allocated":2138112,"data_stored":950852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509554,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33288,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2146304,"data_stored":950908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509552,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38020,"kb_used_data":2672,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333820,"statfs":{"total":96636764160,"available":96597831680,"internally_reserved":0,"allocated":2736128,"data_stored":1541310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:47:23.117 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T08:47:23.117 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-19T08:47:23.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.290+0000 7fc5e3127640 1 Processor -- start 2024-09-19T08:47:23.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.290+0000 7fc5e3127640 1 -- start start 2024-09-19T08:47:23.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.290+0000 7fc5e3127640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc5dc145b70 0x7fc5dc06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:23.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.290+0000 7fc5e3127640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc5dc06c1d0 0x7fc5dc0713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:23.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.290+0000 7fc5e3127640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5dc06c5f0 con 0x7fc5dc0718f0 2024-09-19T08:47:23.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.290+0000 7fc5e3127640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fc5dc073530 con 0x7fc5dc145b70 2024-09-19T08:47:23.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.290+0000 7fc5e3127640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc5dc073810 con 0x7fc5dc06c1d0 2024-09-19T08:47:23.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.290+0000 7fc5e0ec3640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc5dc145b70 0x7fc5dc06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:23.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.290+0000 7fc5e0ec3640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc5dc145b70 0x7fc5dc06bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41938/0 (socket says 172.21.15.98:41938) 2024-09-19T08:47:23.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.290+0000 7fc5e0ec3640 1 -- 172.21.15.98:0/2104236774 learned_addr learned my addr 172.21.15.98:0/2104236774 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:23.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.290+0000 7fc5e16c4640 1 --1- 172.21.15.98:0/2104236774 >> v1:172.21.15.98:6789/0 conn(0x7fc5dc0718f0 0x7fc5dc0cef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:46022/0 (socket says 172.21.15.98:46022) 2024-09-19T08:47:23.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.290+0000 7fc5dbfff640 1 --2- 172.21.15.98:0/2104236774 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc5dc06c1d0 0x7fc5dc0713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:23.291 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.290+0000 7fc5e0ec3640 1 -- 172.21.15.98:0/2104236774 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc5dc06c1d0 msgr2=0x7fc5dc0713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:23.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.290+0000 7fc5e0ec3640 1 --2- 172.21.15.98:0/2104236774 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc5dc06c1d0 0x7fc5dc0713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.290+0000 7fc5e0ec3640 1 -- 172.21.15.98:0/2104236774 >> v1:172.21.15.98:6789/0 conn(0x7fc5dc0718f0 legacy=0x7fc5dc0cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:23.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.290+0000 7fc5e0ec3640 1 -- 172.21.15.98:0/2104236774 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5dc073af0 con 0x7fc5dc145b70 2024-09-19T08:47:23.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.290+0000 7fc5db7fe640 1 -- 172.21.15.98:0/2104236774 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1527246158 0 0) 0x7fc5dc06c5f0 con 0x7fc5dc0718f0 2024-09-19T08:47:23.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.294+0000 7fc5e0ec3640 1 --2- 172.21.15.98:0/2104236774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5dc145b70 0x7fc5dc06bc90 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4009870 tx=0x7fc5c401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3903e54d7977aeaf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:23.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.294+0000 7fc5db7fe640 1 -- 172.21.15.98:0/2104236774 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c40b0070 con 0x7fc5dc145b70 2024-09-19T08:47:23.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.294+0000 7fc5db7fe640 1 -- 172.21.15.98:0/2104236774 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5c40ab600 con 0x7fc5dc145b70 2024-09-19T08:47:23.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.294+0000 7fc5db7fe640 1 -- 172.21.15.98:0/2104236774 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c40ab9d0 con 0x7fc5dc145b70 2024-09-19T08:47:23.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.294+0000 7fc5e3127640 1 -- 172.21.15.98:0/2104236774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5dc145b70 msgr2=0x7fc5dc06bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:23.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.294+0000 7fc5e3127640 1 --2- 172.21.15.98:0/2104236774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5dc145b70 0x7fc5dc06bc90 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4009870 tx=0x7fc5c401c4d0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.294+0000 7fc5e3127640 1 -- 172.21.15.98:0/2104236774 shutdown_connections 2024-09-19T08:47:23.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.294+0000 7fc5e3127640 1 --2- 172.21.15.98:0/2104236774 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc5dc06c1d0 0x7fc5dc0713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.294+0000 7fc5e3127640 1 --2- 172.21.15.98:0/2104236774 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5dc145b70 0x7fc5dc06bc90 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.294+0000 7fc5e3127640 1 -- 172.21.15.98:0/2104236774 >> 172.21.15.98:0/2104236774 conn(0x7fc5dc0b2010 msgr2=0x7fc5dc0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:23.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.294+0000 7fc5e3127640 1 -- 172.21.15.98:0/2104236774 shutdown_connections 2024-09-19T08:47:23.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.294+0000 7fc5e3127640 1 -- 172.21.15.98:0/2104236774 wait complete. 2024-09-19T08:47:23.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.294+0000 7fc5e3127640 1 Processor -- start 2024-09-19T08:47:23.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.294+0000 7fc5e3127640 1 -- start start 2024-09-19T08:47:23.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.298+0000 7fc5e3127640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5dc06c1d0 0x7fc5dc160300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:23.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.298+0000 7fc5e3127640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5dc0718f0 0x7fc5dc13c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:23.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.298+0000 7fc5e3127640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc5dc145b70 0x7fc5dc159230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:23.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.298+0000 7fc5e3127640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc169070 con 0x7fc5dc0718f0 2024-09-19T08:47:23.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.298+0000 7fc5e3127640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc071f00 con 0x7fc5dc06c1d0 2024-09-19T08:47:23.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.298+0000 7fc5e3127640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc5dc1691f0 con 0x7fc5dc145b70 2024-09-19T08:47:23.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.298+0000 7fc5dbfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5dc0718f0 0x7fc5dc13c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:23.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.298+0000 7fc5dbfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5dc0718f0 0x7fc5dc13c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41946/0 (socket says 172.21.15.98:41946) 2024-09-19T08:47:23.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.298+0000 7fc5dbfff640 1 -- 172.21.15.98:0/2322925929 learned_addr learned my addr 172.21.15.98:0/2322925929 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:23.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.298+0000 7fc5e16c4640 1 --2- 172.21.15.98:0/2322925929 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc5dc145b70 0x7fc5dc159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:23.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.298+0000 7fc5e0ec3640 1 --2- 172.21.15.98:0/2322925929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5dc06c1d0 0x7fc5dc160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:23.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.298+0000 7fc5dbfff640 1 -- 172.21.15.98:0/2322925929 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc5dc145b70 msgr2=0x7fc5dc159230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:23.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.298+0000 7fc5dbfff640 1 --2- 172.21.15.98:0/2322925929 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc5dc145b70 0x7fc5dc159230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.298+0000 7fc5dbfff640 1 -- 172.21.15.98:0/2322925929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5dc06c1d0 msgr2=0x7fc5dc160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:23.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.298+0000 7fc5dbfff640 1 --2- 172.21.15.98:0/2322925929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5dc06c1d0 0x7fc5dc160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.298+0000 7fc5dbfff640 1 -- 172.21.15.98:0/2322925929 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5dc158790 con 0x7fc5dc0718f0 2024-09-19T08:47:23.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.298+0000 7fc5e0ec3640 1 --2- 172.21.15.98:0/2322925929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5dc06c1d0 0x7fc5dc160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:23.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.298+0000 7fc5dbfff640 1 --2- 172.21.15.98:0/2322925929 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5dc0718f0 0x7fc5dc13c670 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc00d830 tx=0x7fc5cc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:23.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.298+0000 7fc5d97fa640 1 -- 172.21.15.98:0/2322925929 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc00b6c0 con 0x7fc5dc0718f0 2024-09-19T08:47:23.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.298+0000 7fc5d97fa640 1 -- 172.21.15.98:0/2322925929 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5cc019070 con 0x7fc5dc0718f0 2024-09-19T08:47:23.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.298+0000 7fc5e3127640 1 -- 172.21.15.98:0/2322925929 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5dc15d810 con 0x7fc5dc0718f0 2024-09-19T08:47:23.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.298+0000 7fc5d97fa640 1 -- 172.21.15.98:0/2322925929 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc00bae0 con 0x7fc5dc0718f0 2024-09-19T08:47:23.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.298+0000 7fc5e3127640 1 -- 172.21.15.98:0/2322925929 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5dc158a30 con 0x7fc5dc0718f0 2024-09-19T08:47:23.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.298+0000 7fc5e3127640 1 -- 172.21.15.98:0/2322925929 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5dc0713b0 con 0x7fc5dc0718f0 2024-09-19T08:47:23.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.298+0000 7fc5d97fa640 1 -- 172.21.15.98:0/2322925929 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7fc5cc01e460 con 0x7fc5dc0718f0 2024-09-19T08:47:23.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.298+0000 7fc5d97fa640 1 --2- 172.21.15.98:0/2322925929 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc5bc068250 0x7fc5bc06a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:23.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.298+0000 7fc5d97fa640 1 -- 172.21.15.98:0/2322925929 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5737+0+0 (secure 0 0 0) 0x7fc5cc084b50 con 0x7fc5dc0718f0 2024-09-19T08:47:23.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.298+0000 7fc5e0ec3640 1 --2- 172.21.15.98:0/2322925929 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc5bc068250 0x7fc5bc06a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:23.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.302+0000 7fc5e0ec3640 1 --2- 172.21.15.98:0/2322925929 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc5bc068250 0x7fc5bc06a710 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc5c40ad000 tx=0x7fc5c4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:23.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.302+0000 7fc5d97fa640 1 -- 172.21.15.98:0/2322925929 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc5cc057290 con 0x7fc5dc0718f0 2024-09-19T08:47:23.526 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.526+0000 7fc5e3127640 1 -- 172.21.15.98:0/2322925929 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fc5dc158ed0 con 0x7fc5dc0718f0 2024-09-19T08:47:23.530 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.530+0000 7fc5d97fa640 1 -- 172.21.15.98:0/2322925929 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v26) ==== 190+0+147 (secure 0 0 0) 0x7fc5cc056b90 con 0x7fc5dc0718f0 2024-09-19T08:47:23.530 INFO:teuthology.orchestra.run.smithi098.stdout:[client.0] 2024-09-19T08:47:23.530 INFO:teuthology.orchestra.run.smithi098.stdout: key = AQC/4utm0vNpIhAA4S1E8dFZ0+ibmSKhmZ+bIA== 2024-09-19T08:47:23.530 INFO:teuthology.orchestra.run.smithi098.stdout: caps mds = "allow" 2024-09-19T08:47:23.531 INFO:teuthology.orchestra.run.smithi098.stdout: caps mon = "allow r" 2024-09-19T08:47:23.531 INFO:teuthology.orchestra.run.smithi098.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-19T08:47:23.531 INFO:teuthology.orchestra.run.smithi098.stderr:updated caps for client.0 2024-09-19T08:47:23.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.534+0000 7fc5e3127640 1 -- 172.21.15.98:0/2322925929 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc5bc068250 msgr2=0x7fc5bc06a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:23.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.534+0000 7fc5e3127640 1 --2- 172.21.15.98:0/2322925929 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc5bc068250 0x7fc5bc06a710 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc5c40ad000 tx=0x7fc5c4008040 comp rx=0 tx=0).stop 2024-09-19T08:47:23.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.534+0000 7fc5e3127640 1 -- 172.21.15.98:0/2322925929 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5dc0718f0 msgr2=0x7fc5dc13c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:23.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.534+0000 7fc5e3127640 1 --2- 172.21.15.98:0/2322925929 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5dc0718f0 0x7fc5dc13c670 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc00d830 tx=0x7fc5cc00dcf0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.534+0000 7fc5e3127640 1 -- 172.21.15.98:0/2322925929 shutdown_connections 2024-09-19T08:47:23.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.534+0000 7fc5e3127640 1 --2- 172.21.15.98:0/2322925929 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc5bc068250 0x7fc5bc06a710 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.534+0000 7fc5e3127640 1 --2- 172.21.15.98:0/2322925929 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc5dc145b70 0x7fc5dc159230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.534+0000 7fc5e3127640 1 --2- 172.21.15.98:0/2322925929 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc5dc0718f0 0x7fc5dc13c670 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.533 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.534+0000 7fc5e3127640 1 --2- 172.21.15.98:0/2322925929 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5dc06c1d0 0x7fc5dc160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.534+0000 7fc5e3127640 1 -- 172.21.15.98:0/2322925929 >> 172.21.15.98:0/2322925929 conn(0x7fc5dc0b2010 msgr2=0x7fc5dc1427e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:23.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.534+0000 7fc5e3127640 1 -- 172.21.15.98:0/2322925929 shutdown_connections 2024-09-19T08:47:23.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.534+0000 7fc5e3127640 1 -- 172.21.15.98:0/2322925929 wait complete. 2024-09-19T08:47:23.549 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:47:23.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.670+0000 7f58d9709640 1 Processor -- start 2024-09-19T08:47:23.671 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.670+0000 7f58d9709640 1 -- start start 2024-09-19T08:47:23.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.674+0000 7f58d9709640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58d415c550 0x7f58d406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:23.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.674+0000 7f58d9709640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f58d406c1d0 0x7f58d40cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:23.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.674+0000 7f58d9709640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58d406d310 con 0x7f58d40cf900 2024-09-19T08:47:23.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.674+0000 7f58d9709640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f58d406c610 con 0x7f58d406c1d0 2024-09-19T08:47:23.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.674+0000 7f58d9709640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f58d406d5f0 con 0x7f58d415c550 2024-09-19T08:47:23.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.674+0000 7f58d27fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f58d406c1d0 0x7f58d40cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:23.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.674+0000 7f58d27fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f58d406c1d0 0x7f58d40cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41948/0 (socket says 172.21.15.98:41948) 2024-09-19T08:47:23.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.674+0000 7f58d27fc640 1 -- 172.21.15.98:0/3230842595 learned_addr learned my addr 172.21.15.98:0/3230842595 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:23.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.674+0000 7f58d2ffd640 1 --2- 172.21.15.98:0/3230842595 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58d415c550 0x7f58d406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:23.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.674+0000 7f58d27fc640 1 -- 172.21.15.98:0/3230842595 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58d415c550 msgr2=0x7f58d406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:23.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.674+0000 7f58d27fc640 1 --2- 172.21.15.98:0/3230842595 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58d415c550 0x7f58d406bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.674+0000 7f58d27fc640 1 -- 172.21.15.98:0/3230842595 >> v1:172.21.15.139:6789/0 conn(0x7f58d40cf900 legacy=0x7f58d40ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:23.674 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.674+0000 7f58d27fc640 1 -- 172.21.15.98:0/3230842595 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d406d8d0 con 0x7f58d406c1d0 2024-09-19T08:47:23.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.674+0000 7f58d27fc640 1 --2- 172.21.15.98:0/3230842595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58d406c1d0 0x7f58d40cf3c0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f58c8009870 tx=0x7f58c801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835de64c860009e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:23.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.674+0000 7f58d1ffb640 1 -- 172.21.15.98:0/3230842595 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c80b2070 con 0x7f58d406c1d0 2024-09-19T08:47:23.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.674+0000 7f58d1ffb640 1 -- 172.21.15.98:0/3230842595 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58c80a9070 con 0x7f58d406c1d0 2024-09-19T08:47:23.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.674+0000 7f58d1ffb640 1 -- 172.21.15.98:0/3230842595 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c80ad880 con 0x7f58d406c1d0 2024-09-19T08:47:23.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.678+0000 7f58d9709640 1 -- 172.21.15.98:0/3230842595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58d406c1d0 msgr2=0x7f58d40cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:23.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.678+0000 7f58d9709640 1 --2- 172.21.15.98:0/3230842595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58d406c1d0 0x7f58d40cf3c0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f58c8009870 tx=0x7f58c801c560 comp rx=0 tx=0).stop 2024-09-19T08:47:23.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.678+0000 7f58d9709640 1 -- 172.21.15.98:0/3230842595 shutdown_connections 2024-09-19T08:47:23.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.678+0000 7f58d9709640 1 --2- 172.21.15.98:0/3230842595 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58d406c1d0 0x7f58d40cf3c0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.678+0000 7f58d9709640 1 --2- 172.21.15.98:0/3230842595 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58d415c550 0x7f58d406bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.678+0000 7f58d9709640 1 -- 172.21.15.98:0/3230842595 >> 172.21.15.98:0/3230842595 conn(0x7f58d40b2010 msgr2=0x7f58d40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:23.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.678+0000 7f58d9709640 1 -- 172.21.15.98:0/3230842595 shutdown_connections 2024-09-19T08:47:23.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.678+0000 7f58d9709640 1 -- 172.21.15.98:0/3230842595 wait complete. 2024-09-19T08:47:23.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.678+0000 7f58d9709640 1 Processor -- start 2024-09-19T08:47:23.677 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.678+0000 7f58d9709640 1 -- start start 2024-09-19T08:47:23.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.678+0000 7f58d9709640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58d406c1d0 0x7f58d4151a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:23.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.678+0000 7f58d9709640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58d40cf900 0x7f58d41581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:23.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.678+0000 7f58d9709640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58d415c550 0x7f58d414e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:23.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.678+0000 7f58d9709640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f58d4169230 con 0x7f58d415c550 2024-09-19T08:47:23.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.678+0000 7f58d9709640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f58d415cb40 con 0x7f58d40cf900 2024-09-19T08:47:23.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.678+0000 7f58d9709640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f58d41693b0 con 0x7f58d406c1d0 2024-09-19T08:47:23.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.678+0000 7f58d37fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58d415c550 0x7f58d414e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:23.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.678+0000 7f58d37fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58d415c550 0x7f58d414e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41962/0 (socket says 172.21.15.98:41962) 2024-09-19T08:47:23.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.678+0000 7f58d37fe640 1 -- 172.21.15.98:0/3580231045 learned_addr learned my addr 172.21.15.98:0/3580231045 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:23.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.678+0000 7f58d2ffd640 1 --2- 172.21.15.98:0/3580231045 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58d406c1d0 0x7f58d4151a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:23.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.678+0000 7f58d27fc640 1 --2- 172.21.15.98:0/3580231045 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58d40cf900 0x7f58d41581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:23.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.678+0000 7f58d37fe640 1 -- 172.21.15.98:0/3580231045 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58d406c1d0 msgr2=0x7f58d4151a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:23.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.678+0000 7f58d37fe640 1 --2- 172.21.15.98:0/3580231045 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58d406c1d0 0x7f58d4151a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.678+0000 7f58d37fe640 1 -- 172.21.15.98:0/3580231045 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58d40cf900 msgr2=0x7f58d41581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:23.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.678+0000 7f58d37fe640 1 --2- 172.21.15.98:0/3580231045 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58d40cf900 0x7f58d41581d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.678+0000 7f58d37fe640 1 -- 172.21.15.98:0/3580231045 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d4158c90 con 0x7f58d415c550 2024-09-19T08:47:23.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.678+0000 7f58d37fe640 1 --2- 172.21.15.98:0/3580231045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58d415c550 0x7f58d414e150 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00c770 tx=0x7f58cc00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:23.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.678+0000 7f58bb7fe640 1 -- 172.21.15.98:0/3580231045 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc003b80 con 0x7f58d415c550 2024-09-19T08:47:23.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.678+0000 7f58bb7fe640 1 -- 172.21.15.98:0/3580231045 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58cc003d20 con 0x7f58d415c550 2024-09-19T08:47:23.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.678+0000 7f58d9709640 1 -- 172.21.15.98:0/3580231045 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58d4158f70 con 0x7f58d415c550 2024-09-19T08:47:23.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.678+0000 7f58bb7fe640 1 -- 172.21.15.98:0/3580231045 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc00aa90 con 0x7f58d415c550 2024-09-19T08:47:23.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.678+0000 7f58d9709640 1 -- 172.21.15.98:0/3580231045 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58d4149da0 con 0x7f58d415c550 2024-09-19T08:47:23.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.682+0000 7f58d9709640 1 -- 172.21.15.98:0/3580231045 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58d414d740 con 0x7f58d415c550 2024-09-19T08:47:23.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.682+0000 7f58bb7fe640 1 -- 172.21.15.98:0/3580231045 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f58cc00ac30 con 0x7f58d415c550 2024-09-19T08:47:23.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.682+0000 7f58bb7fe640 1 --2- 172.21.15.98:0/3580231045 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f58c0068250 0x7f58c006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:23.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.682+0000 7f58bb7fe640 1 -- 172.21.15.98:0/3580231045 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5737+0+0 (secure 0 0 0) 0x7f58cc084370 con 0x7f58d415c550 2024-09-19T08:47:23.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.682+0000 7f58d2ffd640 1 --2- 172.21.15.98:0/3580231045 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f58c0068250 0x7f58c006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:23.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.682+0000 7f58d2ffd640 1 --2- 172.21.15.98:0/3580231045 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f58c0068250 0x7f58c006a710 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f58c400b930 tx=0x7f58c4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:23.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.686+0000 7f58bb7fe640 1 -- 172.21.15.98:0/3580231045 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f58cc056ab0 con 0x7f58d415c550 2024-09-19T08:47:23.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.870+0000 7f58d9709640 1 -- 172.21.15.98:0/3580231045 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58d40cf3c0 con 0x7f58d415c550 2024-09-19T08:47:23.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.874+0000 7f58bb7fe640 1 -- 172.21.15.98:0/3580231045 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5235 (secure 0 0 0) 0x7f58cc0563b0 con 0x7f58d415c550 2024-09-19T08:47:23.872 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:47:23.873 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":100,"btime":"2024-09-19T08:47:17:337562+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6392,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/1027012777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":1027012777},{"type":"v1","addr":"172.21.15.139:6834","nonce":1027012777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7339,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3216462399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6833","nonce":3216462399},{"type":"v1","addr":"172.21.15.139:6835","nonce":3216462399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":9174,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1629579852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1629579852},{"type":"v1","addr":"172.21.15.98:6833","nonce":1629579852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:47:13.743399+0000","modified":"2024-09-19T08:47:16.695205+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9177":{"gid":9177,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/3407902663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3407902663},{"type":"v1","addr":"172.21.15.98:6835","nonce":3407902663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9177,"qdb_cluster":[9177]},"id":13}]} 2024-09-19T08:47:23.873 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 100 2024-09-19T08:47:23.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.874+0000 7f58d9709640 1 -- 172.21.15.98:0/3580231045 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f58c0068250 msgr2=0x7f58c006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:23.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.874+0000 7f58d9709640 1 --2- 172.21.15.98:0/3580231045 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f58c0068250 0x7f58c006a710 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f58c400b930 tx=0x7f58c4008040 comp rx=0 tx=0).stop 2024-09-19T08:47:23.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.874+0000 7f58d9709640 1 -- 172.21.15.98:0/3580231045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58d415c550 msgr2=0x7f58d414e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:23.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.874+0000 7f58d9709640 1 --2- 172.21.15.98:0/3580231045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58d415c550 0x7f58d414e150 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00c770 tx=0x7f58cc00cc30 comp rx=0 tx=0).stop 2024-09-19T08:47:23.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.874+0000 7f58d9709640 1 -- 172.21.15.98:0/3580231045 shutdown_connections 2024-09-19T08:47:23.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.874+0000 7f58d9709640 1 --2- 172.21.15.98:0/3580231045 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f58c0068250 0x7f58c006a710 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.874+0000 7f58d9709640 1 --2- 172.21.15.98:0/3580231045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f58d415c550 0x7f58d414e150 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.874+0000 7f58d9709640 1 --2- 172.21.15.98:0/3580231045 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f58d40cf900 0x7f58d41581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.874+0000 7f58d9709640 1 --2- 172.21.15.98:0/3580231045 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f58d406c1d0 0x7f58d4151a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:23.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.874+0000 7f58d9709640 1 -- 172.21.15.98:0/3580231045 >> 172.21.15.98:0/3580231045 conn(0x7f58d40b2010 msgr2=0x7f58d4156f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:23.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.874+0000 7f58d9709640 1 -- 172.21.15.98:0/3580231045 shutdown_connections 2024-09-19T08:47:23.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:23.874+0000 7f58d9709640 1 -- 172.21.15.98:0/3580231045 wait complete. 2024-09-19T08:47:24.293 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 100, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T08:47:13.743399+0000', 'modified': '2024-09-19T08:47:16.695205+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9177}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9177': {'gid': 9177, 'name': 'a', 'rank': 0, 'incarnation': 97, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.98:6835/3407902663', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.98:6834', 'nonce': 3407902663}, {'type': 'v1', 'addr': '172.21.15.98:6835', 'nonce': 3407902663}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [27], 'metadata_pool': 26, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9177, 'qdb_cluster': [9177]} 2024-09-19T08:47:24.294 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T08:47:24.294 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-19T08:47:24.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.426+0000 7f256502a640 1 Processor -- start 2024-09-19T08:47:24.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.426+0000 7f256502a640 1 -- start start 2024-09-19T08:47:24.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.430+0000 7f256502a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f256006c3a0 0x7f25600cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:24.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.430+0000 7f256502a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f25600cf900 0x7f25600723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:24.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.430+0000 7f256502a640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f256006d310 con 0x7f256015c550 2024-09-19T08:47:24.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.430+0000 7f256502a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f256006d5f0 con 0x7f25600cf900 2024-09-19T08:47:24.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.430+0000 7f256502a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f256006d8d0 con 0x7f256006c3a0 2024-09-19T08:47:24.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.430+0000 7f255ed9d640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f256015c550 0x7f256006bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:46024/0 (socket says 172.21.15.98:46024) 2024-09-19T08:47:24.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.430+0000 7f255ed9d640 1 -- 172.21.15.98:0/1517624457 learned_addr learned my addr 172.21.15.98:0/1517624457 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:24.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.430+0000 7f255dd9b640 1 -- 172.21.15.98:0/1517624457 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 986196774 0 0) 0x7f256006d310 con 0x7f256015c550 2024-09-19T08:47:24.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.430+0000 7f255dd9b640 1 -- 172.21.15.98:0/1517624457 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2548003530 con 0x7f256015c550 2024-09-19T08:47:24.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.430+0000 7f255f59e640 1 --2- 172.21.15.98:0/1517624457 >> v2:172.21.15.139:3300/0 conn(0x7f25600cf900 0x7f25600723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:24.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.430+0000 7f255e59c640 1 --2- 172.21.15.98:0/1517624457 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f256006c3a0 0x7f25600cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:24.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.430+0000 7f255dd9b640 1 -- 172.21.15.98:0/1517624457 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3179691439 0 0) 0x7f2548003530 con 0x7f256015c550 2024-09-19T08:47:24.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.430+0000 7f255dd9b640 1 -- 172.21.15.98:0/1517624457 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f256006c3a0 msgr2=0x7f25600cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:24.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.430+0000 7f255dd9b640 1 --2- 172.21.15.98:0/1517624457 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f256006c3a0 0x7f25600cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.430+0000 7f255dd9b640 1 -- 172.21.15.98:0/1517624457 >> v2:172.21.15.139:3300/0 conn(0x7f25600cf900 msgr2=0x7f25600723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:24.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.430+0000 7f255dd9b640 1 --2- 172.21.15.98:0/1517624457 >> v2:172.21.15.139:3300/0 conn(0x7f25600cf900 0x7f25600723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.430+0000 7f255dd9b640 1 -- 172.21.15.98:0/1517624457 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f256006dbb0 con 0x7f256015c550 2024-09-19T08:47:24.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.430+0000 7f255dd9b640 1 -- 172.21.15.98:0/1517624457 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f2550001680 con 0x7f256015c550 2024-09-19T08:47:24.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.430+0000 7f255dd9b640 1 -- 172.21.15.98:0/1517624457 >> v1:172.21.15.98:6789/0 conn(0x7f256015c550 legacy=0x7f256006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:24.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.430+0000 7f255dd9b640 1 --2- 172.21.15.98:0/1517624457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25480039c0 0x7f2548003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:24.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.430+0000 7f255dd9b640 1 --2- 172.21.15.98:0/1517624457 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f256006c3a0 0x7f2548004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:24.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.430+0000 7f255dd9b640 1 --2- 172.21.15.98:0/1517624457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2548004d50 0x7f25480051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:24.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.430+0000 7f255dd9b640 1 -- 172.21.15.98:0/1517624457 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f256006d5f0 con 0x7f2548004d50 2024-09-19T08:47:24.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.430+0000 7f255dd9b640 1 -- 172.21.15.98:0/1517624457 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f256006d8d0 con 0x7f25480039c0 2024-09-19T08:47:24.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.430+0000 7f255dd9b640 1 -- 172.21.15.98:0/1517624457 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f254800b9d0 con 0x7f256006c3a0 2024-09-19T08:47:24.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.430+0000 7f255f59e640 1 --2- 172.21.15.98:0/1517624457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2548004d50 0x7f25480051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:24.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.430+0000 7f255ed9d640 1 --2- 172.21.15.98:0/1517624457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25480039c0 0x7f2548003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:24.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.430+0000 7f255e59c640 1 --2- 172.21.15.98:0/1517624457 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f256006c3a0 0x7f2548004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:24.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.430+0000 7f255f59e640 1 -- 172.21.15.98:0/1517624457 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f256006c3a0 msgr2=0x7f2548004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:24.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.430+0000 7f255f59e640 1 --2- 172.21.15.98:0/1517624457 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f256006c3a0 0x7f2548004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.430+0000 7f255f59e640 1 -- 172.21.15.98:0/1517624457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25480039c0 msgr2=0x7f2548003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:24.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.430+0000 7f255f59e640 1 --2- 172.21.15.98:0/1517624457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f25480039c0 0x7f2548003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.430+0000 7f255f59e640 1 -- 172.21.15.98:0/1517624457 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2548003530 con 0x7f2548004d50 2024-09-19T08:47:24.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.434+0000 7f255f59e640 1 --2- 172.21.15.98:0/1517624457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2548004d50 0x7f25480051d0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f2558009fc0 tx=0x7f255801ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e2d6378932b4c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:24.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.434+0000 7f255dd9b640 1 -- 172.21.15.98:0/1517624457 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25580b2070 con 0x7f2548004d50 2024-09-19T08:47:24.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.434+0000 7f255dd9b640 1 -- 172.21.15.98:0/1517624457 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25580a5cd0 con 0x7f2548004d50 2024-09-19T08:47:24.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.434+0000 7f255dd9b640 1 -- 172.21.15.98:0/1517624457 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2558009c10 con 0x7f2548004d50 2024-09-19T08:47:24.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.434+0000 7f256502a640 1 -- 172.21.15.98:0/1517624457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2548004d50 msgr2=0x7f25480051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:24.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.434+0000 7f256502a640 1 --2- 172.21.15.98:0/1517624457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2548004d50 0x7f25480051d0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f2558009fc0 tx=0x7f255801ccc0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.434+0000 7f255f59e640 1 -- 172.21.15.98:0/1517624457 reap_dead start 2024-09-19T08:47:24.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.434+0000 7f256502a640 1 -- 172.21.15.98:0/1517624457 shutdown_connections 2024-09-19T08:47:24.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.434+0000 7f256502a640 1 -- 172.21.15.98:0/1517624457 >> 172.21.15.98:0/1517624457 conn(0x7f25600b2010 msgr2=0x7f25600b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:24.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.434+0000 7f256502a640 1 -- 172.21.15.98:0/1517624457 shutdown_connections 2024-09-19T08:47:24.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.434+0000 7f256502a640 1 -- 172.21.15.98:0/1517624457 wait complete. 2024-09-19T08:47:24.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.434+0000 7f256502a640 1 Processor -- start 2024-09-19T08:47:24.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.434+0000 7f256502a640 1 -- start start 2024-09-19T08:47:24.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.434+0000 7f256502a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2548004d50 0x7f256015f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:24.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.434+0000 7f256502a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2560145ca0 0x7f2560150450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:24.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.434+0000 7f256502a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f256014f5f0 0x7f256014fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:24.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.434+0000 7f256502a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f256015cab0 con 0x7f2548004d50 2024-09-19T08:47:24.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.434+0000 7f256502a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f256015c930 con 0x7f2560145ca0 2024-09-19T08:47:24.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.434+0000 7f256502a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f25600cf3c0 con 0x7f256014f5f0 2024-09-19T08:47:24.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.434+0000 7f255ed9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2548004d50 0x7f256015f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:24.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.434+0000 7f255ed9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2548004d50 0x7f256015f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41976/0 (socket says 172.21.15.98:41976) 2024-09-19T08:47:24.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.434+0000 7f255ed9d640 1 -- 172.21.15.98:0/1989197093 learned_addr learned my addr 172.21.15.98:0/1989197093 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:24.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.434+0000 7f255e59c640 1 --2- 172.21.15.98:0/1989197093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2560145ca0 0x7f2560150450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:24.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.434+0000 7f255f59e640 1 --2- 172.21.15.98:0/1989197093 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f256014f5f0 0x7f256014fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:24.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.434+0000 7f255ed9d640 1 -- 172.21.15.98:0/1989197093 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f256014f5f0 msgr2=0x7f256014fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:24.437 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.434+0000 7f255ed9d640 1 --2- 172.21.15.98:0/1989197093 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f256014f5f0 0x7f256014fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.434+0000 7f255ed9d640 1 -- 172.21.15.98:0/1989197093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2560145ca0 msgr2=0x7f2560150450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:24.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.434+0000 7f255ed9d640 1 --2- 172.21.15.98:0/1989197093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2560145ca0 0x7f2560150450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.434+0000 7f255ed9d640 1 -- 172.21.15.98:0/1989197093 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2560150990 con 0x7f2548004d50 2024-09-19T08:47:24.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.434+0000 7f255f59e640 1 --2- 172.21.15.98:0/1989197093 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f256014f5f0 0x7f256014fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:24.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.434+0000 7f255ed9d640 1 --2- 172.21.15.98:0/1989197093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2548004d50 0x7f256015f810 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f255000e820 tx=0x7f255000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:24.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.438+0000 7f25477fe640 1 -- 172.21.15.98:0/1989197093 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255000c8d0 con 0x7f2548004d50 2024-09-19T08:47:24.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.438+0000 7f25477fe640 1 -- 172.21.15.98:0/1989197093 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2550018070 con 0x7f2548004d50 2024-09-19T08:47:24.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.438+0000 7f256502a640 1 -- 172.21.15.98:0/1989197093 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2560150c30 con 0x7f2548004d50 2024-09-19T08:47:24.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.438+0000 7f25477fe640 1 -- 172.21.15.98:0/1989197093 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255000cc90 con 0x7f2548004d50 2024-09-19T08:47:24.438 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.438+0000 7f256502a640 1 -- 172.21.15.98:0/1989197093 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25601511f0 con 0x7f2548004d50 2024-09-19T08:47:24.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.438+0000 7f25477fe640 1 -- 172.21.15.98:0/1989197093 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f255001d460 con 0x7f2548004d50 2024-09-19T08:47:24.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.438+0000 7f25477fe640 1 --2- 172.21.15.98:0/1989197093 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f254c068280 0x7f254c06a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:24.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.438+0000 7f25477fe640 1 -- 172.21.15.98:0/1989197093 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5737+0+0 (secure 0 0 0) 0x7f25500842f0 con 0x7f2548004d50 2024-09-19T08:47:24.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.438+0000 7f255e59c640 1 --2- 172.21.15.98:0/1989197093 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f254c068280 0x7f254c06a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:24.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.438+0000 7f25680e6000 1 Processor -- start 2024-09-19T08:47:24.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.438+0000 7f25680e6000 1 -- start start 2024-09-19T08:47:24.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.438+0000 7f255e59c640 1 --2- 172.21.15.98:0/1989197093 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f254c068280 0x7f254c06a740 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2560159730 tx=0x7f2554008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:24.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.438+0000 7f25680e6000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ed49e10d70 0x55ed49e11150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:24.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.438+0000 7f25680e6000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed49e11690 0x55ed49dc7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:24.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.438+0000 7f25680e6000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed49dc76a0 0x55ed49dc7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:24.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.438+0000 7f25680e6000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ed49caab60 con 0x55ed49e11690 2024-09-19T08:47:24.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.438+0000 7f25680e6000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ed49c11c70 con 0x55ed49dc76a0 2024-09-19T08:47:24.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.438+0000 7f25680e6000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x55ed49bb7ed0 con 0x55ed49e10d70 2024-09-19T08:47:24.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.438+0000 7f255e59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed49e11690 0x55ed49dc7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:24.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.438+0000 7f255e59c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed49e11690 0x55ed49dc7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41978/0 (socket says 172.21.15.98:41978) 2024-09-19T08:47:24.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.438+0000 7f255e59c640 1 -- 172.21.15.98:0/1489919765 learned_addr learned my addr 172.21.15.98:0/1489919765 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:24.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.438+0000 7f255ed9d640 1 --2- 172.21.15.98:0/1489919765 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ed49e10d70 0x55ed49e11150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:24.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.438+0000 7f255f59e640 1 --2- 172.21.15.98:0/1489919765 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed49dc76a0 0x55ed49dc7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:24.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.438+0000 7f255e59c640 1 -- 172.21.15.98:0/1489919765 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ed49e10d70 msgr2=0x55ed49e11150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:24.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.438+0000 7f255e59c640 1 --2- 172.21.15.98:0/1489919765 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ed49e10d70 0x55ed49e11150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.438+0000 7f255e59c640 1 -- 172.21.15.98:0/1489919765 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed49dc76a0 msgr2=0x55ed49dc7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:24.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.438+0000 7f255e59c640 1 --2- 172.21.15.98:0/1489919765 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed49dc76a0 0x55ed49dc7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.438+0000 7f255e59c640 1 -- 172.21.15.98:0/1489919765 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed49bc9e50 con 0x55ed49e11690 2024-09-19T08:47:24.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.438+0000 7f255f59e640 1 --2- 172.21.15.98:0/1489919765 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed49dc76a0 0x55ed49dc7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:24.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f255e59c640 1 --2- 172.21.15.98:0/1489919765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed49e11690 0x55ed49dc7160 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f25540043b0 tx=0x7f2554016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ea888027a51ef81 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:24.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f2544ff9640 1 -- 172.21.15.98:0/1489919765 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2554029070 con 0x55ed49e11690 2024-09-19T08:47:24.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f2544ff9640 1 -- 172.21.15.98:0/1489919765 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f255401f050 con 0x55ed49e11690 2024-09-19T08:47:24.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f2544ff9640 1 -- 172.21.15.98:0/1489919765 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25540096a0 con 0x55ed49e11690 2024-09-19T08:47:24.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f25680e6000 1 -- 172.21.15.98:0/1489919765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed49e11690 msgr2=0x55ed49dc7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:24.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f25680e6000 1 --2- 172.21.15.98:0/1489919765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed49e11690 0x55ed49dc7160 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f25540043b0 tx=0x7f2554016000 comp rx=0 tx=0).stop 2024-09-19T08:47:24.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f25680e6000 1 -- 172.21.15.98:0/1489919765 shutdown_connections 2024-09-19T08:47:24.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f25680e6000 1 --2- 172.21.15.98:0/1489919765 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed49dc76a0 0x55ed49dc7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f25680e6000 1 --2- 172.21.15.98:0/1489919765 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed49e11690 0x55ed49dc7160 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f25680e6000 1 --2- 172.21.15.98:0/1489919765 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ed49e10d70 0x55ed49e11150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f25680e6000 1 -- 172.21.15.98:0/1489919765 >> 172.21.15.98:0/1489919765 conn(0x55ed49dc9600 msgr2=0x55ed49dc1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:24.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f25680e6000 1 -- 172.21.15.98:0/1489919765 shutdown_connections 2024-09-19T08:47:24.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f25680e6000 1 -- 172.21.15.98:0/1489919765 wait complete. 2024-09-19T08:47:24.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f25680e6000 1 -- start start 2024-09-19T08:47:24.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f25680e6000 1 Processor -- start 2024-09-19T08:47:24.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f25680e6000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ed49e10d70 0x55ed49e16eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:24.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f25680e6000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed49e11690 0x55ed49e173f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:24.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f25680e6000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed49dfd490 0x55ed49dfd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:24.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f25680e6000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ed49c73b00 con 0x55ed49dfd490 2024-09-19T08:47:24.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f25680e6000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ed49bec360 con 0x55ed49e11690 2024-09-19T08:47:24.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f25680e6000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x55ed49be92d0 con 0x55ed49e10d70 2024-09-19T08:47:24.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f25680e6000 10 client.0 _ll_register_callbacks cb 0x55ed49dae820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:47:24.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f255f59e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed49dfd490 0x55ed49dfd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:24.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f255f59e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed49dfd490 0x55ed49dfd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41992/0 (socket says 172.21.15.98:41992) 2024-09-19T08:47:24.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f255f59e640 1 -- 172.21.15.98:0/1216254319 learned_addr learned my addr 172.21.15.98:0/1216254319 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:24.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f255ed9d640 1 --2- 172.21.15.98:0/1216254319 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ed49e10d70 0x55ed49e16eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:24.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f255e59c640 1 --2- 172.21.15.98:0/1216254319 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed49e11690 0x55ed49e173f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:24.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f255f59e640 1 -- 172.21.15.98:0/1216254319 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ed49e10d70 msgr2=0x55ed49e16eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:24.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f255f59e640 1 --2- 172.21.15.98:0/1216254319 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ed49e10d70 0x55ed49e16eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f255f59e640 1 -- 172.21.15.98:0/1216254319 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed49e11690 msgr2=0x55ed49e173f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:24.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f255f59e640 1 --2- 172.21.15.98:0/1216254319 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed49e11690 0x55ed49e173f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f255f59e640 1 -- 172.21.15.98:0/1216254319 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed49bb8120 con 0x55ed49dfd490 2024-09-19T08:47:24.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f255e59c640 1 --2- 172.21.15.98:0/1216254319 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed49e11690 0x55ed49e173f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:24.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f255f59e640 1 --2- 172.21.15.98:0/1216254319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed49dfd490 0x55ed49dfd870 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f255801caf0 tx=0x7f2548005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f16dec0909b4376 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:24.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f253dffb640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-19T08:47:24.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f253dffb640 1 -- 172.21.15.98:0/1216254319 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25580bd060 con 0x55ed49dfd490 2024-09-19T08:47:24.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f25680e6000 1 -- 172.21.15.98:0/1216254319 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f255401f050 con 0x55ed49dfd490 2024-09-19T08:47:24.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f253dffb640 1 -- 172.21.15.98:0/1216254319 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(170..170 src has 1..170) ==== 5737+0+0 (secure 0 0 0) 0x7f2558009c10 con 0x55ed49dfd490 2024-09-19T08:47:24.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f253dffb640 1 -- 172.21.15.98:0/1216254319 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f25580c3050 con 0x55ed49dfd490 2024-09-19T08:47:24.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f25680e6000 10 client.9342 fetch_fsmap learned FSMap version 100 2024-09-19T08:47:24.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f25680e6000 1 -- 172.21.15.98:0/1216254319 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) -- 0x55ed49bb1450 con 0x55ed49dfd490 2024-09-19T08:47:24.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f253dffb640 1 -- 172.21.15.98:0/1216254319 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f255401f050 con 0x55ed49dfd490 2024-09-19T08:47:24.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f253dffb640 1 -- 172.21.15.98:0/1216254319 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 100) ==== 3502+0+0 (secure 0 0 0) 0x7f25580c6020 con 0x55ed49dfd490 2024-09-19T08:47:24.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f25680e6000 10 client.9342 fetch_fsmap finished waiting for FSMap version 100 2024-09-19T08:47:24.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f25680e6000 10 client.9342 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T08:47:24.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f25680e6000 20 client.9342 populate_metadata read hostname 'smithi098' 2024-09-19T08:47:24.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f25680e6000 1 --2- 172.21.15.98:0/1216254319 >> [v2:172.21.15.98:6834/3407902663,v1:172.21.15.98:6835/3407902663] conn(0x55ed49dfe280 0x55ed49dfe660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:24.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f25680e6000 4 client.9342 mds_command: new command op to 9177 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T08:47:24.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.442+0000 7f25680e6000 1 -- 172.21.15.98:0/1216254319 --> [v2:172.21.15.98:6834/3407902663,v1:172.21.15.98:6835/3407902663] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed49babc00 con 0x55ed49dfe280 2024-09-19T08:47:24.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.446+0000 7f255ed9d640 1 --2- 172.21.15.98:0/1216254319 >> [v2:172.21.15.98:6834/3407902663,v1:172.21.15.98:6835/3407902663] conn(0x55ed49dfe280 0x55ed49dfe660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:24.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.446+0000 7f253dffb640 10 client.9342 ms_handle_connect on v2:172.21.15.98:6834/3407902663 2024-09-19T08:47:24.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.446+0000 7f255ed9d640 1 --2- 172.21.15.98:0/1216254319 >> [v2:172.21.15.98:6834/3407902663,v1:172.21.15.98:6835/3407902663] conn(0x55ed49dfe280 0x55ed49dfe660 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b44c520f2c338f0 server_cookie=87a27f475c1c7631 in_seq=0 out_seq=0 2024-09-19T08:47:24.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.446+0000 7f253dffb640 1 -- 172.21.15.98:0/1216254319 <== mds.0 v2:172.21.15.98:6834/3407902663 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ed49babc00 con 0x55ed49dfe280 2024-09-19T08:47:24.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.446+0000 7f253dffb640 10 client.9342 handle_command_reply: tid=0 2024-09-19T08:47:24.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.446+0000 7f25680e6000 1 client.9342 shutdown 2024-09-19T08:47:24.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.446+0000 7f25680e6000 1 -- 172.21.15.98:0/1216254319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed49dfd490 msgr2=0x55ed49dfd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:24.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.446+0000 7f25680e6000 1 --2- 172.21.15.98:0/1216254319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed49dfd490 0x55ed49dfd870 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f255801caf0 tx=0x7f2548005840 comp rx=0 tx=0).stop 2024-09-19T08:47:24.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.446+0000 7f25680e6000 1 -- 172.21.15.98:0/1216254319 shutdown_connections 2024-09-19T08:47:24.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.446+0000 7f25680e6000 1 --2- 172.21.15.98:0/1216254319 >> [v2:172.21.15.98:6834/3407902663,v1:172.21.15.98:6835/3407902663] conn(0x55ed49dfe280 0x55ed49dfe660 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.446+0000 7f25680e6000 1 --2- 172.21.15.98:0/1216254319 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed49dfd490 0x55ed49dfd870 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.446+0000 7f253dffb640 0 client.9342 ms_handle_reset on v2:172.21.15.98:6834/3407902663 2024-09-19T08:47:24.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.446+0000 7f25680e6000 1 --2- 172.21.15.98:0/1216254319 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed49e11690 0x55ed49e173f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.446+0000 7f25680e6000 1 --2- 172.21.15.98:0/1216254319 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ed49e10d70 0x55ed49e16eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.446+0000 7f25680e6000 1 -- 172.21.15.98:0/1216254319 >> 172.21.15.98:0/1216254319 conn(0x55ed49dc76a0 msgr2=0x55ed49dc89b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:24.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.446+0000 7f25680e6000 1 -- 172.21.15.98:0/1216254319 shutdown_connections 2024-09-19T08:47:24.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.446+0000 7f25680e6000 1 -- 172.21.15.98:0/1216254319 wait complete. 2024-09-19T08:47:24.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.446+0000 7f25680e6000 20 client.9342 trim_cache size 0 max 16384 2024-09-19T08:47:24.463 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f25680e6000 1 Processor -- start 2024-09-19T08:47:24.464 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f25680e6000 1 -- start start 2024-09-19T08:47:24.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f25680e6000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed49dfd490 0x55ed49e27260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:24.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f25680e6000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ed49e10d70 0x55ed49e277a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:24.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f25680e6000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed49e11690 0x55ed49e15ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:24.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f25680e6000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f25600b4840 con 0x55ed49e11690 2024-09-19T08:47:24.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f25680e6000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2560154f40 con 0x55ed49dfd490 2024-09-19T08:47:24.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f25680e6000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2560155590 con 0x55ed49e10d70 2024-09-19T08:47:24.465 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f255f59e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed49e11690 0x55ed49e15ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:24.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f255f59e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed49e11690 0x55ed49e15ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:41998/0 (socket says 172.21.15.98:41998) 2024-09-19T08:47:24.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f255f59e640 1 -- 172.21.15.98:0/2288160738 learned_addr learned my addr 172.21.15.98:0/2288160738 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:24.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f255f59e640 1 -- 172.21.15.98:0/2288160738 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ed49e10d70 msgr2=0x55ed49e277a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:24.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f255f59e640 1 --2- 172.21.15.98:0/2288160738 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ed49e10d70 0x55ed49e277a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f255f59e640 1 -- 172.21.15.98:0/2288160738 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed49dfd490 msgr2=0x55ed49e27260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:24.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f255f59e640 1 --2- 172.21.15.98:0/2288160738 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed49dfd490 0x55ed49e27260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f255f59e640 1 -- 172.21.15.98:0/2288160738 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed49dfdbe0 con 0x55ed49e11690 2024-09-19T08:47:24.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f255f59e640 1 --2- 172.21.15.98:0/2288160738 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed49e11690 0x55ed49e15ca0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f255801cf30 tx=0x7f25580b6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d9b7f98e17ecb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:24.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f253ffff640 1 -- 172.21.15.98:0/2288160738 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25580b2040 con 0x55ed49e11690 2024-09-19T08:47:24.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f253ffff640 1 -- 172.21.15.98:0/2288160738 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25580039a0 con 0x55ed49e11690 2024-09-19T08:47:24.466 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f253ffff640 1 -- 172.21.15.98:0/2288160738 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2558003d50 con 0x55ed49e11690 2024-09-19T08:47:24.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f25680e6000 1 -- 172.21.15.98:0/2288160738 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed49e11690 msgr2=0x55ed49e15ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:24.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f25680e6000 1 --2- 172.21.15.98:0/2288160738 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed49e11690 0x55ed49e15ca0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f255801cf30 tx=0x7f25580b6000 comp rx=0 tx=0).stop 2024-09-19T08:47:24.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f25680e6000 1 -- 172.21.15.98:0/2288160738 shutdown_connections 2024-09-19T08:47:24.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f25680e6000 1 --2- 172.21.15.98:0/2288160738 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed49e11690 0x55ed49e15ca0 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f25680e6000 1 --2- 172.21.15.98:0/2288160738 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ed49e10d70 0x55ed49e277a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f25680e6000 1 --2- 172.21.15.98:0/2288160738 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed49dfd490 0x55ed49e27260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f25680e6000 1 -- 172.21.15.98:0/2288160738 >> 172.21.15.98:0/2288160738 conn(0x55ed49dc76a0 msgr2=0x55ed49dc1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:24.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f25680e6000 1 -- 172.21.15.98:0/2288160738 shutdown_connections 2024-09-19T08:47:24.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f25680e6000 1 -- 172.21.15.98:0/2288160738 wait complete. 2024-09-19T08:47:24.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f25680e6000 1 -- start start 2024-09-19T08:47:24.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f25680e6000 1 Processor -- start 2024-09-19T08:47:24.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f25680e6000 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed49dfd490 0x55ed49e89210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:24.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f25680e6000 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ed49e10d70 0x55ed49e89750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:24.471 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f25680e6000 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed49e11690 0x55ed49e89c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:24.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f25680e6000 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x55ed49c10f80 con 0x55ed49dfd490 2024-09-19T08:47:24.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f25680e6000 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ed49bc9ca0 con 0x55ed49e11690 2024-09-19T08:47:24.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f25680e6000 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x55ed49c162a0 con 0x55ed49e10d70 2024-09-19T08:47:24.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f25680e6000 10 client.0 _ll_register_callbacks cb 0x7f25580b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T08:47:24.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f255ed9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed49dfd490 0x55ed49e89210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:24.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f255ed9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed49dfd490 0x55ed49e89210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42014/0 (socket says 172.21.15.98:42014) 2024-09-19T08:47:24.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f255ed9d640 1 -- 172.21.15.98:0/866937103 learned_addr learned my addr 172.21.15.98:0/866937103 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:24.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f255f59e640 1 --2- 172.21.15.98:0/866937103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed49e11690 0x55ed49e89c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:24.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f255e59c640 1 --2- 172.21.15.98:0/866937103 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ed49e10d70 0x55ed49e89750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:24.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f255ed9d640 1 -- 172.21.15.98:0/866937103 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ed49e10d70 msgr2=0x55ed49e89750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:24.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f255ed9d640 1 --2- 172.21.15.98:0/866937103 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ed49e10d70 0x55ed49e89750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f255ed9d640 1 -- 172.21.15.98:0/866937103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed49e11690 msgr2=0x55ed49e89c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:24.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f255ed9d640 1 --2- 172.21.15.98:0/866937103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed49e11690 0x55ed49e89c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.466+0000 7f255ed9d640 1 -- 172.21.15.98:0/866937103 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed49b98fd0 con 0x55ed49dfd490 2024-09-19T08:47:24.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f255ed9d640 1 --2- 172.21.15.98:0/866937103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed49dfd490 0x55ed49e89210 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f2550052790 tx=0x7f2550056af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98f499e361764eb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:24.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f253dffb640 10 client.0 ms_handle_connect on v2:172.21.15.98:3300/0 2024-09-19T08:47:24.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f253dffb640 1 -- 172.21.15.98:0/866937103 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255005ef10 con 0x55ed49dfd490 2024-09-19T08:47:24.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f253dffb640 1 -- 172.21.15.98:0/866937103 <== mon.0 v2:172.21.15.98:3300/0 2 ==== osd_map(170..170 src has 1..170) ==== 5737+0+0 (secure 0 0 0) 0x7f255006fc60 con 0x55ed49dfd490 2024-09-19T08:47:24.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f25680e6000 1 -- 172.21.15.98:0/866937103 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25580039a0 con 0x55ed49dfd490 2024-09-19T08:47:24.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f253dffb640 1 -- 172.21.15.98:0/866937103 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f255006fe80 con 0x55ed49dfd490 2024-09-19T08:47:24.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f25680e6000 10 client.9348 fetch_fsmap learned FSMap version 100 2024-09-19T08:47:24.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f25680e6000 1 -- 172.21.15.98:0/866937103 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) -- 0x55ed49c94750 con 0x55ed49dfd490 2024-09-19T08:47:24.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f253dffb640 1 -- 172.21.15.98:0/866937103 <== mon.0 v2:172.21.15.98:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25580039a0 con 0x55ed49dfd490 2024-09-19T08:47:24.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f253dffb640 1 -- 172.21.15.98:0/866937103 <== mon.0 v2:172.21.15.98:3300/0 5 ==== fsmap(e 100) ==== 3502+0+0 (secure 0 0 0) 0x7f2550056590 con 0x55ed49dfd490 2024-09-19T08:47:24.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f25680e6000 10 client.9348 fetch_fsmap finished waiting for FSMap version 100 2024-09-19T08:47:24.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f25680e6000 10 client.9348 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T08:47:24.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f25680e6000 20 client.9348 populate_metadata read hostname 'smithi098' 2024-09-19T08:47:24.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f25680e6000 1 --2- 172.21.15.98:0/866937103 >> [v2:172.21.15.98:6834/3407902663,v1:172.21.15.98:6835/3407902663] conn(0x55ed49e90460 0x55ed49e18490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:24.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f25680e6000 4 client.9348 mds_command: new command op to 9177 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T08:47:24.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f25680e6000 1 -- 172.21.15.98:0/866937103 --> [v2:172.21.15.98:6834/3407902663,v1:172.21.15.98:6835/3407902663] -- command(tid 0: {"prefix": "status"}) -- 0x55ed49c1f3a0 con 0x55ed49e90460 2024-09-19T08:47:24.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f255e59c640 1 --2- 172.21.15.98:0/866937103 >> [v2:172.21.15.98:6834/3407902663,v1:172.21.15.98:6835/3407902663] conn(0x55ed49e90460 0x55ed49e18490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:24.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f253dffb640 10 client.9348 ms_handle_connect on v2:172.21.15.98:6834/3407902663 2024-09-19T08:47:24.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f255e59c640 1 --2- 172.21.15.98:0/866937103 >> [v2:172.21.15.98:6834/3407902663,v1:172.21.15.98:6835/3407902663] conn(0x55ed49e90460 0x55ed49e18490 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee516fe08fcf05a server_cookie=bb01f5140efce0cb in_seq=0 out_seq=0 2024-09-19T08:47:24.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f253dffb640 1 -- 172.21.15.98:0/866937103 <== mds.0 v2:172.21.15.98:6834/3407902663 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55ed49c1f3a0 con 0x55ed49e90460 2024-09-19T08:47:24.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f253dffb640 10 client.9348 handle_command_reply: tid=0 2024-09-19T08:47:24.474 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f25680e6000 1 client.9348 shutdown 2024-09-19T08:47:24.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f25680e6000 1 -- 172.21.15.98:0/866937103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed49dfd490 msgr2=0x55ed49e89210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:24.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f25680e6000 1 --2- 172.21.15.98:0/866937103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed49dfd490 0x55ed49e89210 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f2550052790 tx=0x7f2550056af0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f25680e6000 1 -- 172.21.15.98:0/866937103 shutdown_connections 2024-09-19T08:47:24.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f25680e6000 1 --2- 172.21.15.98:0/866937103 >> [v2:172.21.15.98:6834/3407902663,v1:172.21.15.98:6835/3407902663] conn(0x55ed49e90460 0x55ed49e18490 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f25680e6000 1 --2- 172.21.15.98:0/866937103 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ed49e11690 0x55ed49e89c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f253dffb640 0 client.9348 ms_handle_reset on v2:172.21.15.98:6834/3407902663 2024-09-19T08:47:24.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f25680e6000 1 --2- 172.21.15.98:0/866937103 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x55ed49e10d70 0x55ed49e89750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f25680e6000 1 --2- 172.21.15.98:0/866937103 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x55ed49dfd490 0x55ed49e89210 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f25680e6000 1 -- 172.21.15.98:0/866937103 >> 172.21.15.98:0/866937103 conn(0x55ed49dc76a0 msgr2=0x55ed49e26250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:24.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f25680e6000 1 -- 172.21.15.98:0/866937103 shutdown_connections 2024-09-19T08:47:24.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f25680e6000 1 -- 172.21.15.98:0/866937103 wait complete. 2024-09-19T08:47:24.475 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f25680e6000 20 client.9348 trim_cache size 0 max 16384 2024-09-19T08:47:24.476 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:47:24.476 INFO:teuthology.orchestra.run.smithi098.stdout: "cluster_fsid": "552bf0ee-1f4f-4644-ba7c-d0931e55a451", 2024-09-19T08:47:24.476 INFO:teuthology.orchestra.run.smithi098.stdout: "whoami": 0, 2024-09-19T08:47:24.476 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 9177, 2024-09-19T08:47:24.476 INFO:teuthology.orchestra.run.smithi098.stdout: "want_state": "up:active", 2024-09-19T08:47:24.476 INFO:teuthology.orchestra.run.smithi098.stdout: "state": "up:active", 2024-09-19T08:47:24.476 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "cephfs", 2024-09-19T08:47:24.476 INFO:teuthology.orchestra.run.smithi098.stdout: "rank_uptime": 10.101683918000001, 2024-09-19T08:47:24.476 INFO:teuthology.orchestra.run.smithi098.stdout: "mdsmap_epoch": 100, 2024-09-19T08:47:24.476 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch": 170, 2024-09-19T08:47:24.476 INFO:teuthology.orchestra.run.smithi098.stdout: "osdmap_epoch_barrier": 170, 2024-09-19T08:47:24.476 INFO:teuthology.orchestra.run.smithi098.stdout: "uptime": 14.134622823000001 2024-09-19T08:47:24.476 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:47:24.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f256502a640 1 -- 172.21.15.98:0/1989197093 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f254c068280 msgr2=0x7f254c06a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:24.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f256502a640 1 --2- 172.21.15.98:0/1989197093 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f254c068280 0x7f254c06a740 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2560159730 tx=0x7f2554008040 comp rx=0 tx=0).stop 2024-09-19T08:47:24.477 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f256502a640 1 -- 172.21.15.98:0/1989197093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2548004d50 msgr2=0x7f256015f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:24.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f256502a640 1 --2- 172.21.15.98:0/1989197093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2548004d50 0x7f256015f810 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f255000e820 tx=0x7f255000ece0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f256502a640 1 -- 172.21.15.98:0/1989197093 shutdown_connections 2024-09-19T08:47:24.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f256502a640 1 --2- 172.21.15.98:0/1989197093 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f254c068280 0x7f254c06a740 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f256502a640 1 --2- 172.21.15.98:0/1989197093 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f256014f5f0 0x7f256014fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f256502a640 1 --2- 172.21.15.98:0/1989197093 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2560145ca0 0x7f2560150450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f256502a640 1 --2- 172.21.15.98:0/1989197093 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2548004d50 0x7f256015f810 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f256502a640 1 -- 172.21.15.98:0/1989197093 >> 172.21.15.98:0/1989197093 conn(0x7f25600b2010 msgr2=0x7f256015b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:24.478 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f256502a640 1 -- 172.21.15.98:0/1989197093 shutdown_connections 2024-09-19T08:47:24.479 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.470+0000 7f256502a640 1 -- 172.21.15.98:0/1989197093 wait complete. 2024-09-19T08:47:24.486 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T08:47:24.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.598+0000 7ff863305640 1 Processor -- start 2024-09-19T08:47:24.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.598+0000 7ff863305640 1 -- start start 2024-09-19T08:47:24.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.602+0000 7ff863305640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff85c15c250 0x7ff85c0d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:24.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.602+0000 7ff863305640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff85c070ec0 0x7ff85c0d0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:24.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.602+0000 7ff863305640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff85c0d2ea0 con 0x7ff85c0d1860 2024-09-19T08:47:24.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.602+0000 7ff863305640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7ff85c0cffe0 con 0x7ff85c070ec0 2024-09-19T08:47:24.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.602+0000 7ff863305640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff85c073f70 con 0x7ff85c15c250 2024-09-19T08:47:24.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.602+0000 7ff8618a2640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff85c070ec0 0x7ff85c0d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:24.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.602+0000 7ff8608a0640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7ff85c0d1860 0x7ff85c0707b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:46028/0 (socket says 172.21.15.98:46028) 2024-09-19T08:47:24.601 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.602+0000 7ff8608a0640 1 -- 172.21.15.98:0/3251585206 learned_addr learned my addr 172.21.15.98:0/3251585206 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:24.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.602+0000 7ff8610a1640 1 --2- 172.21.15.98:0/3251585206 >> v2:172.21.15.139:3300/0 conn(0x7ff85c15c250 0x7ff85c0d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:24.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.602+0000 7ff8618a2640 1 -- 172.21.15.98:0/3251585206 >> v2:172.21.15.139:3300/0 conn(0x7ff85c15c250 msgr2=0x7ff85c0d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:24.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.602+0000 7ff8618a2640 1 --2- 172.21.15.98:0/3251585206 >> v2:172.21.15.139:3300/0 conn(0x7ff85c15c250 0x7ff85c0d1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.602+0000 7ff8618a2640 1 -- 172.21.15.98:0/3251585206 >> v1:172.21.15.98:6789/0 conn(0x7ff85c0d1860 legacy=0x7ff85c0707b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:24.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.602+0000 7ff8618a2640 1 -- 172.21.15.98:0/3251585206 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff85c0739d0 con 0x7ff85c070ec0 2024-09-19T08:47:24.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.602+0000 7ff853fff640 1 -- 172.21.15.98:0/3251585206 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 106468641 0 0) 0x7ff85c0d2ea0 con 0x7ff85c0d1860 2024-09-19T08:47:24.602 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.602+0000 7ff8610a1640 1 --2- 172.21.15.98:0/3251585206 >> v2:172.21.15.139:3300/0 conn(0x7ff85c15c250 0x7ff85c0d1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:24.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.606+0000 7ff8618a2640 1 --2- 172.21.15.98:0/3251585206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff85c070ec0 0x7ff85c0d0450 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7ff85c050030 tx=0x7ff85401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4235c6be283ff76f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:24.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.606+0000 7ff853fff640 1 -- 172.21.15.98:0/3251585206 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8540b2070 con 0x7ff85c070ec0 2024-09-19T08:47:24.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.606+0000 7ff853fff640 1 -- 172.21.15.98:0/3251585206 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8540a59c0 con 0x7ff85c070ec0 2024-09-19T08:47:24.604 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.606+0000 7ff853fff640 1 -- 172.21.15.98:0/3251585206 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8540a5d90 con 0x7ff85c070ec0 2024-09-19T08:47:24.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.606+0000 7ff863305640 1 -- 172.21.15.98:0/3251585206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff85c070ec0 msgr2=0x7ff85c0d0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:24.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.606+0000 7ff863305640 1 --2- 172.21.15.98:0/3251585206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff85c070ec0 0x7ff85c0d0450 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7ff85c050030 tx=0x7ff85401c900 comp rx=0 tx=0).stop 2024-09-19T08:47:24.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.606+0000 7ff863305640 1 -- 172.21.15.98:0/3251585206 shutdown_connections 2024-09-19T08:47:24.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.606+0000 7ff863305640 1 --2- 172.21.15.98:0/3251585206 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff85c070ec0 0x7ff85c0d0450 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.606+0000 7ff863305640 1 --2- 172.21.15.98:0/3251585206 >> v2:172.21.15.139:3300/0 conn(0x7ff85c15c250 0x7ff85c0d1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.606+0000 7ff863305640 1 -- 172.21.15.98:0/3251585206 >> 172.21.15.98:0/3251585206 conn(0x7ff85c0d2310 msgr2=0x7ff85c0d26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:24.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.606+0000 7ff863305640 1 -- 172.21.15.98:0/3251585206 shutdown_connections 2024-09-19T08:47:24.605 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.606+0000 7ff863305640 1 -- 172.21.15.98:0/3251585206 wait complete. 2024-09-19T08:47:24.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.606+0000 7ff863305640 1 Processor -- start 2024-09-19T08:47:24.606 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.606+0000 7ff863305640 1 -- start start 2024-09-19T08:47:24.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.606+0000 7ff863305640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff85c070ec0 0x7ff85c15f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:24.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.606+0000 7ff863305640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff85c0d1860 0x7ff85c15e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:24.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.606+0000 7ff863305640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff85c15c250 0x7ff85c14e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:24.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.606+0000 7ff863305640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c0b53a0 con 0x7ff85c15c250 2024-09-19T08:47:24.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.606+0000 7ff863305640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c0b1d10 con 0x7ff85c0d1860 2024-09-19T08:47:24.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.606+0000 7ff863305640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff85c0b5520 con 0x7ff85c070ec0 2024-09-19T08:47:24.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.606+0000 7ff8618a2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff85c15c250 0x7ff85c14e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:24.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.606+0000 7ff8618a2640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff85c15c250 0x7ff85c14e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42034/0 (socket says 172.21.15.98:42034) 2024-09-19T08:47:24.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.606+0000 7ff8618a2640 1 -- 172.21.15.98:0/1434906200 learned_addr learned my addr 172.21.15.98:0/1434906200 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:24.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.606+0000 7ff8610a1640 1 --2- 172.21.15.98:0/1434906200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff85c070ec0 0x7ff85c15f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:24.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.606+0000 7ff8618a2640 1 -- 172.21.15.98:0/1434906200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff85c070ec0 msgr2=0x7ff85c15f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:24.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.606+0000 7ff8608a0640 1 --2- 172.21.15.98:0/1434906200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff85c0d1860 0x7ff85c15e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:24.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.606+0000 7ff8618a2640 1 --2- 172.21.15.98:0/1434906200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff85c070ec0 0x7ff85c15f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.606+0000 7ff8618a2640 1 -- 172.21.15.98:0/1434906200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff85c0d1860 msgr2=0x7ff85c15e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:24.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.606+0000 7ff8618a2640 1 --2- 172.21.15.98:0/1434906200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff85c0d1860 0x7ff85c15e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.606+0000 7ff8618a2640 1 -- 172.21.15.98:0/1434906200 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff85c152d50 con 0x7ff85c15c250 2024-09-19T08:47:24.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.606+0000 7ff8610a1640 1 --2- 172.21.15.98:0/1434906200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff85c070ec0 0x7ff85c15f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:24.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.606+0000 7ff8608a0640 1 --2- 172.21.15.98:0/1434906200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff85c0d1860 0x7ff85c15e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:24.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.606+0000 7ff8618a2640 1 --2- 172.21.15.98:0/1434906200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff85c15c250 0x7ff85c14e7a0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7ff854019510 tx=0x7ff8540196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:24.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.606+0000 7ff851ffb640 1 -- 172.21.15.98:0/1434906200 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8540b2050 con 0x7ff85c15c250 2024-09-19T08:47:24.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.606+0000 7ff851ffb640 1 -- 172.21.15.98:0/1434906200 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8540be070 con 0x7ff85c15c250 2024-09-19T08:47:24.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.606+0000 7ff851ffb640 1 -- 172.21.15.98:0/1434906200 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8540adbf0 con 0x7ff85c15c250 2024-09-19T08:47:24.609 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.610+0000 7ff863305640 1 -- 172.21.15.98:0/1434906200 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff85c14fc60 con 0x7ff85c15c250 2024-09-19T08:47:24.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.610+0000 7ff863305640 1 -- 172.21.15.98:0/1434906200 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff85c155c50 con 0x7ff85c15c250 2024-09-19T08:47:24.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.610+0000 7ff863305640 1 -- 172.21.15.98:0/1434906200 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff85c155760 con 0x7ff85c15c250 2024-09-19T08:47:24.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.610+0000 7ff851ffb640 1 -- 172.21.15.98:0/1434906200 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7ff8540c3460 con 0x7ff85c15c250 2024-09-19T08:47:24.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.610+0000 7ff851ffb640 1 --2- 172.21.15.98:0/1434906200 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff840068250 0x7ff84006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:24.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.610+0000 7ff851ffb640 1 -- 172.21.15.98:0/1434906200 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5737+0+0 (secure 0 0 0) 0x7ff85411f850 con 0x7ff85c15c250 2024-09-19T08:47:24.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.610+0000 7ff8610a1640 1 --2- 172.21.15.98:0/1434906200 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff840068250 0x7ff84006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:24.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.610+0000 7ff8610a1640 1 --2- 172.21.15.98:0/1434906200 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff840068250 0x7ff84006a710 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff85c0b30a0 tx=0x7ff858008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:24.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.614+0000 7ff851ffb640 1 -- 172.21.15.98:0/1434906200 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff854016520 con 0x7ff85c15c250 2024-09-19T08:47:24.804 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.806+0000 7ff863305640 1 -- 172.21.15.98:0/1434906200 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff85c0d11a0 con 0x7ff85c15c250 2024-09-19T08:47:24.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.806+0000 7ff851ffb640 1 -- 172.21.15.98:0/1434906200 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v100) ==== 53+0+73 (secure 0 0 0) 0x7ff8540f1f90 con 0x7ff85c15c250 2024-09-19T08:47:24.805 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T08:47:24.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.806+0000 7ff863305640 1 -- 172.21.15.98:0/1434906200 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff840068250 msgr2=0x7ff84006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:24.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.806+0000 7ff863305640 1 --2- 172.21.15.98:0/1434906200 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff840068250 0x7ff84006a710 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff85c0b30a0 tx=0x7ff858008040 comp rx=0 tx=0).stop 2024-09-19T08:47:24.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.806+0000 7ff863305640 1 -- 172.21.15.98:0/1434906200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff85c15c250 msgr2=0x7ff85c14e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:24.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.806+0000 7ff863305640 1 --2- 172.21.15.98:0/1434906200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff85c15c250 0x7ff85c14e7a0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7ff854019510 tx=0x7ff8540196a0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.806+0000 7ff863305640 1 -- 172.21.15.98:0/1434906200 shutdown_connections 2024-09-19T08:47:24.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.806+0000 7ff863305640 1 --2- 172.21.15.98:0/1434906200 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff840068250 0x7ff84006a710 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.806+0000 7ff863305640 1 --2- 172.21.15.98:0/1434906200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff85c15c250 0x7ff85c14e7a0 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.806+0000 7ff863305640 1 --2- 172.21.15.98:0/1434906200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff85c0d1860 0x7ff85c15e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.806+0000 7ff863305640 1 --2- 172.21.15.98:0/1434906200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff85c070ec0 0x7ff85c15f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:24.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.806+0000 7ff863305640 1 -- 172.21.15.98:0/1434906200 >> 172.21.15.98:0/1434906200 conn(0x7ff85c0d2310 msgr2=0x7ff85c0b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:24.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.806+0000 7ff863305640 1 -- 172.21.15.98:0/1434906200 shutdown_connections 2024-09-19T08:47:24.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:24.806+0000 7ff863305640 1 -- 172.21.15.98:0/1434906200 wait complete. 2024-09-19T08:47:24.823 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T08:47:24.824 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T08:47:24.824 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi098.front.sepia.ceph.com 2024-09-19T08:47:24.824 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T08:47:24.824 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T08:47:24.824 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T08:47:24.824 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T08:47:24.824 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T08:47:24.824 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:47:24.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:24.825 DEBUG:teuthology.orchestra.run.smithi098:> ip addr 2024-09-19T08:47:24.831 INFO:teuthology.orchestra.run.smithi098.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T08:47:24.831 INFO:teuthology.orchestra.run.smithi098.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T08:47:24.831 INFO:teuthology.orchestra.run.smithi098.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T08:47:24.831 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:47:24.831 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 ::1/128 scope host 2024-09-19T08:47:24.831 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:47:24.831 INFO:teuthology.orchestra.run.smithi098.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:47:24.831 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b8 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:47:24.832 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f0 2024-09-19T08:47:24.832 INFO:teuthology.orchestra.run.smithi098.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:47:24.832 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:d8:d9:b9 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:47:24.832 INFO:teuthology.orchestra.run.smithi098.stdout: altname enp7s0f1 2024-09-19T08:47:24.832 INFO:teuthology.orchestra.run.smithi098.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:47:24.832 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6a brd ff:ff:ff:ff:ff:ff 2024-09-19T08:47:24.832 INFO:teuthology.orchestra.run.smithi098.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T08:47:24.832 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0c:c4:7a:8f:d2:6b brd ff:ff:ff:ff:ff:ff 2024-09-19T08:47:24.832 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.21.15.98/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T08:47:24.832 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft 41963sec preferred_lft 41963sec 2024-09-19T08:47:24.832 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::ec4:7aff:fe8f:d26b/64 scope link 2024-09-19T08:47:24.832 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:47:24.832 INFO:teuthology.orchestra.run.smithi098.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T08:47:24.832 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 02:42:98:dd:32:2d brd ff:ff:ff:ff:ff:ff 2024-09-19T08:47:24.833 INFO:teuthology.orchestra.run.smithi098.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T08:47:24.833 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:47:24.833 INFO:teuthology.orchestra.run.smithi098.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T08:47:24.833 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-19T08:47:24.833 INFO:teuthology.orchestra.run.smithi098.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T08:47:24.833 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:47:24.833 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-19T08:47:24.833 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:47:24.833 INFO:teuthology.orchestra.run.smithi098.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:47:24.833 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:47:24.833 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-19T08:47:24.833 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:47:24.833 INFO:teuthology.orchestra.run.smithi098.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:47:24.834 INFO:teuthology.orchestra.run.smithi098.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:47:24.834 INFO:teuthology.orchestra.run.smithi098.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-19T08:47:24.834 INFO:teuthology.orchestra.run.smithi098.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:47:24.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:24.834 DEBUG:teuthology.orchestra.run.smithi098:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T08:47:24.886 INFO:teuthology.orchestra.run.smithi098.stdout:1 2024-09-19T08:47:24.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:24.888 DEBUG:teuthology.orchestra.run.smithi098:> route 2024-09-19T08:47:24.941 INFO:teuthology.orchestra.run.smithi098.stdout:Kernel IP routing table 2024-09-19T08:47:24.941 INFO:teuthology.orchestra.run.smithi098.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T08:47:24.941 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T08:47:24.941 INFO:teuthology.orchestra.run.smithi098.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T08:47:24.941 INFO:teuthology.orchestra.run.smithi098.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T08:47:24.941 INFO:teuthology.orchestra.run.smithi098.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T08:47:24.942 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T08:47:24.942 INFO:teuthology.orchestra.run.smithi098.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:47:24.942 INFO:teuthology.orchestra.run.smithi098.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:47:24.942 INFO:teuthology.orchestra.run.smithi098.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:47:24.942 INFO:teuthology.orchestra.run.smithi098.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T08:47:24.942 DEBUG:teuthology.orchestra.run.smithi098:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:47:24.942 DEBUG:teuthology.orchestra.run.smithi098:> set -e 2024-09-19T08:47:24.942 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T08:47:24.943 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T08:47:24.943 DEBUG:teuthology.orchestra.run.smithi098:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T08:47:24.943 DEBUG:teuthology.orchestra.run.smithi098:> ') 2024-09-19T08:47:25.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:47:25.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:47:25.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:25.062 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2024-09-19T08:47:25.108 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-19T08:47:25.108 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T08:47:25.108 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:47:25.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:25.109 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link set brx.0 up 2024-09-19T08:47:25.164 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-19T08:47:25.165 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:47:25.165 DEBUG:teuthology.orchestra.run.smithi098:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T08:47:25.212 INFO:teuthology.orchestra.run.smithi098.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T08:47:25.213 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:47:25.213 DEBUG:teuthology.orchestra.run.smithi098:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T08:47:25.260 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:47:25.318 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:47:25.369 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:47:25.369 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:47:25.370 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:47:25.370 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:47:25.370 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:47:25.370 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:47:25.370 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:47:25.370 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:47:25.370 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:47:25.370 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:47:25.370 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:47:25.371 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:47:25.371 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:47:25.371 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:47:25.371 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:47:25.371 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:47:25.371 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:47:25.371 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:47:25.371 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:47:25.371 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:47:25.372 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:47:25.372 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:47:25.372 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:47:25.372 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:47:25.372 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:47:25.372 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:47:25.372 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:47:25.372 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:47:25.372 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:47:25.372 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:47:25.372 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:47:25.373 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:47:25.373 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:47:25.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:25.374 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:47:25.421 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T08:47:25.421 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-19T08:47:25.469 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:47:25.486 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:47:25.536 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:47:25.536 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:47:25.537 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:47:25.537 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:47:25.537 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:47:25.537 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:47:25.537 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:47:25.537 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:47:25.537 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:47:25.537 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:47:25.537 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:47:25.537 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:47:25.537 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:47:25.537 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:47:25.538 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:47:25.538 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:47:25.538 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:47:25.538 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:47:25.538 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:47:25.538 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:47:25.538 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:47:25.538 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:47:25.538 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:47:25.538 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:47:25.538 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:47:25.538 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:47:25.538 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:47:25.538 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:47:25.539 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:47:25.539 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:47:25.539 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:47:25.539 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:47:25.539 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:47:25.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:25.540 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:47:25.549 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:2024-09-19T08:47:25.550+0000 7f9b483f8580 -1 init, newargv = 0x55c59e1f72b0 newargc=15 2024-09-19T08:47:25.549 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:ceph-fuse[54634]: starting ceph client 2024-09-19T08:47:25.556 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:ceph-fuse[54634]: starting fuse 2024-09-19T08:47:26.654 DEBUG:teuthology.orchestra.run.smithi098:> sudo modprobe fuse 2024-09-19T08:47:26.670 DEBUG:teuthology.orchestra.run.smithi098:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:47:26.719 INFO:teuthology.orchestra.run.smithi098.stdout:/sys 2024-09-19T08:47:26.719 INFO:teuthology.orchestra.run.smithi098.stdout:/proc 2024-09-19T08:47:26.719 INFO:teuthology.orchestra.run.smithi098.stdout:/dev 2024-09-19T08:47:26.719 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/pts 2024-09-19T08:47:26.719 INFO:teuthology.orchestra.run.smithi098.stdout:/run 2024-09-19T08:47:26.719 INFO:teuthology.orchestra.run.smithi098.stdout:/ 2024-09-19T08:47:26.720 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/security 2024-09-19T08:47:26.720 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/shm 2024-09-19T08:47:26.720 INFO:teuthology.orchestra.run.smithi098.stdout:/run/lock 2024-09-19T08:47:26.720 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/cgroup 2024-09-19T08:47:26.720 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/pstore 2024-09-19T08:47:26.720 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/bpf 2024-09-19T08:47:26.720 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:47:26.720 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/hugepages 2024-09-19T08:47:26.720 INFO:teuthology.orchestra.run.smithi098.stdout:/dev/mqueue 2024-09-19T08:47:26.720 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/debug 2024-09-19T08:47:26.720 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/tracing 2024-09-19T08:47:26.720 INFO:teuthology.orchestra.run.smithi098.stdout:/proc/fs/nfsd 2024-09-19T08:47:26.720 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/fs/fuse/connections 2024-09-19T08:47:26.721 INFO:teuthology.orchestra.run.smithi098.stdout:/sys/kernel/config 2024-09-19T08:47:26.721 INFO:teuthology.orchestra.run.smithi098.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:47:26.721 INFO:teuthology.orchestra.run.smithi098.stdout:/run/rpc_pipefs 2024-09-19T08:47:26.721 INFO:teuthology.orchestra.run.smithi098.stdout:/run/user/1000 2024-09-19T08:47:26.721 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph 2024-09-19T08:47:26.721 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T08:47:26.721 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T08:47:26.721 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T08:47:26.721 INFO:teuthology.orchestra.run.smithi098.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T08:47:26.721 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns 2024-09-19T08:47:26.721 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:47:26.721 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:47:26.721 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:47:26.721 INFO:teuthology.orchestra.run.smithi098.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:47:26.722 INFO:teuthology.orchestra.run.smithi098.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-19T08:47:26.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:26.723 DEBUG:teuthology.orchestra.run.smithi098:> ls /sys/fs/fuse/connections 2024-09-19T08:47:26.769 INFO:teuthology.orchestra.run.smithi098.stdout:45 2024-09-19T08:47:26.769 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-19T08:47:26.769 DEBUG:teuthology.orchestra.run.smithi098:> sudo stdin-killer -- python3 -c ' 2024-09-19T08:47:26.769 DEBUG:teuthology.orchestra.run.smithi098:> import glob 2024-09-19T08:47:26.769 DEBUG:teuthology.orchestra.run.smithi098:> import re 2024-09-19T08:47:26.769 DEBUG:teuthology.orchestra.run.smithi098:> import os 2024-09-19T08:47:26.769 DEBUG:teuthology.orchestra.run.smithi098:> import subprocess 2024-09-19T08:47:26.769 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:47:26.770 DEBUG:teuthology.orchestra.run.smithi098:> def _find_admin_socket(client_name): 2024-09-19T08:47:26.770 DEBUG:teuthology.orchestra.run.smithi098:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T08:47:26.770 DEBUG:teuthology.orchestra.run.smithi098:> files = glob.glob(asok_path) 2024-09-19T08:47:26.770 DEBUG:teuthology.orchestra.run.smithi098:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T08:47:26.770 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:47:26.770 DEBUG:teuthology.orchestra.run.smithi098:> # Given a non-glob path, it better be there 2024-09-19T08:47:26.770 DEBUG:teuthology.orchestra.run.smithi098:> if "*" not in asok_path: 2024-09-19T08:47:26.770 DEBUG:teuthology.orchestra.run.smithi098:> assert(len(files) == 1) 2024-09-19T08:47:26.771 DEBUG:teuthology.orchestra.run.smithi098:> return files[0] 2024-09-19T08:47:26.771 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:47:26.771 DEBUG:teuthology.orchestra.run.smithi098:> for f in files: 2024-09-19T08:47:26.771 DEBUG:teuthology.orchestra.run.smithi098:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T08:47:26.771 DEBUG:teuthology.orchestra.run.smithi098:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T08:47:26.771 DEBUG:teuthology.orchestra.run.smithi098:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T08:47:26.771 DEBUG:teuthology.orchestra.run.smithi098:> contents = proc_f.read() 2024-09-19T08:47:26.771 DEBUG:teuthology.orchestra.run.smithi098:> if mountpoint in contents: 2024-09-19T08:47:26.771 DEBUG:teuthology.orchestra.run.smithi098:> return f 2024-09-19T08:47:26.771 DEBUG:teuthology.orchestra.run.smithi098:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T08:47:26.771 DEBUG:teuthology.orchestra.run.smithi098:> 2024-09-19T08:47:26.771 DEBUG:teuthology.orchestra.run.smithi098:> print(_find_admin_socket("client.0")) 2024-09-19T08:47:26.771 DEBUG:teuthology.orchestra.run.smithi098:> ' 2024-09-19T08:47:26.893 INFO:teuthology.orchestra.run.smithi098.stdout:/var/run/ceph/ceph-client.0.54634.asok 2024-09-19T08:47:26.897 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:47:26.906 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.54634.asok 2024-09-19T08:47:26.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:26.907 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.54634.asok status 2024-09-19T08:47:27.062 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:47:27.062 INFO:teuthology.orchestra.run.smithi098.stdout: "metadata": { 2024-09-19T08:47:27.063 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T08:47:27.063 INFO:teuthology.orchestra.run.smithi098.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T08:47:27.063 INFO:teuthology.orchestra.run.smithi098.stdout: "entity_id": "0", 2024-09-19T08:47:27.063 INFO:teuthology.orchestra.run.smithi098.stdout: "hostname": "smithi098", 2024-09-19T08:47:27.063 INFO:teuthology.orchestra.run.smithi098.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T08:47:27.063 INFO:teuthology.orchestra.run.smithi098.stdout: "pid": "54634", 2024-09-19T08:47:27.063 INFO:teuthology.orchestra.run.smithi098.stdout: "root": "/" 2024-09-19T08:47:27.063 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:47:27.063 INFO:teuthology.orchestra.run.smithi098.stdout: "dentry_count": 0, 2024-09-19T08:47:27.063 INFO:teuthology.orchestra.run.smithi098.stdout: "dentry_pinned_count": 0, 2024-09-19T08:47:27.063 INFO:teuthology.orchestra.run.smithi098.stdout: "id": 9363, 2024-09-19T08:47:27.063 INFO:teuthology.orchestra.run.smithi098.stdout: "inst": { 2024-09-19T08:47:27.064 INFO:teuthology.orchestra.run.smithi098.stdout: "name": { 2024-09-19T08:47:27.064 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "client", 2024-09-19T08:47:27.064 INFO:teuthology.orchestra.run.smithi098.stdout: "num": 9363 2024-09-19T08:47:27.064 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:47:27.064 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": { 2024-09-19T08:47:27.064 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v1", 2024-09-19T08:47:27.064 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:47:27.064 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 3544782286 2024-09-19T08:47:27.064 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:47:27.064 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:47:27.064 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": { 2024-09-19T08:47:27.064 INFO:teuthology.orchestra.run.smithi098.stdout: "type": "v1", 2024-09-19T08:47:27.064 INFO:teuthology.orchestra.run.smithi098.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:47:27.065 INFO:teuthology.orchestra.run.smithi098.stdout: "nonce": 3544782286 2024-09-19T08:47:27.065 INFO:teuthology.orchestra.run.smithi098.stdout: }, 2024-09-19T08:47:27.065 INFO:teuthology.orchestra.run.smithi098.stdout: "inst_str": "client.9363 192.168.0.1:0/3544782286", 2024-09-19T08:47:27.065 INFO:teuthology.orchestra.run.smithi098.stdout: "addr_str": "192.168.0.1:0/3544782286", 2024-09-19T08:47:27.065 INFO:teuthology.orchestra.run.smithi098.stdout: "inode_count": 1, 2024-09-19T08:47:27.065 INFO:teuthology.orchestra.run.smithi098.stdout: "mds_epoch": 100, 2024-09-19T08:47:27.065 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_epoch": 170, 2024-09-19T08:47:27.065 INFO:teuthology.orchestra.run.smithi098.stdout: "osd_epoch_barrier": 0, 2024-09-19T08:47:27.065 INFO:teuthology.orchestra.run.smithi098.stdout: "blocklisted": false, 2024-09-19T08:47:27.065 INFO:teuthology.orchestra.run.smithi098.stdout: "fs_name": "cephfs" 2024-09-19T08:47:27.065 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:47:27.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:27.076 DEBUG:teuthology.orchestra.run.smithi098:> stat --file-system '--printf=%T 2024-09-19T08:47:27.076 DEBUG:teuthology.orchestra.run.smithi098:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T08:47:27.082 INFO:teuthology.orchestra.run.smithi098.stdout:fuseblk 2024-09-19T08:47:27.083 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T08:47:27.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:27.083 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T08:47:27.137 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T08:47:27.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.302+0000 7f63a9e87640 1 Processor -- start 2024-09-19T08:47:27.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.302+0000 7f63a9e87640 1 -- start start 2024-09-19T08:47:27.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.306+0000 7f63a9e87640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f63a415c550 0x7f63a406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:27.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.306+0000 7f63a9e87640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f63a406c1d0 0x7f63a40cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:27.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.306+0000 7f63a9e87640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63a40cf900 0x7f63a40ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:27.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.306+0000 7f63a9e87640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f63a406c640 con 0x7f63a415c550 2024-09-19T08:47:27.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.306+0000 7f63a9e87640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f63a406d310 con 0x7f63a406c1d0 2024-09-19T08:47:27.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.306+0000 7f63a9e87640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f63a406d5f0 con 0x7f63a40cf900 2024-09-19T08:47:27.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.306+0000 7f63a37fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f63a415c550 0x7f63a406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:27.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.306+0000 7f63a37fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f63a415c550 0x7f63a406bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42044/0 (socket says 172.21.15.98:42044) 2024-09-19T08:47:27.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.306+0000 7f63a37fe640 1 -- 172.21.15.98:0/2123103694 learned_addr learned my addr 172.21.15.98:0/2123103694 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:27.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.306+0000 7f63a3fff640 1 --2- 172.21.15.98:0/2123103694 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63a40cf900 0x7f63a40ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:27.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.306+0000 7f63a2ffd640 1 --2- 172.21.15.98:0/2123103694 >> v2:172.21.15.139:3300/0 conn(0x7f63a406c1d0 0x7f63a40cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:27.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.306+0000 7f63a37fe640 1 -- 172.21.15.98:0/2123103694 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63a40cf900 msgr2=0x7f63a40ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:27.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.306+0000 7f63a37fe640 1 --2- 172.21.15.98:0/2123103694 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63a40cf900 0x7f63a40ce1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:27.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.306+0000 7f63a37fe640 1 -- 172.21.15.98:0/2123103694 >> v2:172.21.15.139:3300/0 conn(0x7f63a406c1d0 msgr2=0x7f63a40cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:27.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.306+0000 7f63a37fe640 1 --2- 172.21.15.98:0/2123103694 >> v2:172.21.15.139:3300/0 conn(0x7f63a406c1d0 0x7f63a40cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:27.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.306+0000 7f63a37fe640 1 -- 172.21.15.98:0/2123103694 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a406d8d0 con 0x7f63a415c550 2024-09-19T08:47:27.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.306+0000 7f63a37fe640 1 --2- 172.21.15.98:0/2123103694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63a415c550 0x7f63a406bc90 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f63a40ccc60 tx=0x7f639401c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea4dfe5623b54955 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:27.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.306+0000 7f63a27fc640 1 -- 172.21.15.98:0/2123103694 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63940b0070 con 0x7f63a415c550 2024-09-19T08:47:27.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.310+0000 7f63a27fc640 1 -- 172.21.15.98:0/2123103694 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63940ab600 con 0x7f63a415c550 2024-09-19T08:47:27.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.310+0000 7f63a27fc640 1 -- 172.21.15.98:0/2123103694 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63940ab9d0 con 0x7f63a415c550 2024-09-19T08:47:27.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.310+0000 7f63a9e87640 1 -- 172.21.15.98:0/2123103694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63a415c550 msgr2=0x7f63a406bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:27.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.310+0000 7f63a9e87640 1 --2- 172.21.15.98:0/2123103694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63a415c550 0x7f63a406bc90 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f63a40ccc60 tx=0x7f639401c260 comp rx=0 tx=0).stop 2024-09-19T08:47:27.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.310+0000 7f63a9e87640 1 -- 172.21.15.98:0/2123103694 shutdown_connections 2024-09-19T08:47:27.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.310+0000 7f63a9e87640 1 --2- 172.21.15.98:0/2123103694 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63a40cf900 0x7f63a40ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:27.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.310+0000 7f63a9e87640 1 --2- 172.21.15.98:0/2123103694 >> v2:172.21.15.139:3300/0 conn(0x7f63a406c1d0 0x7f63a40cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:27.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.310+0000 7f63a9e87640 1 --2- 172.21.15.98:0/2123103694 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63a415c550 0x7f63a406bc90 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:27.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.310+0000 7f63a9e87640 1 -- 172.21.15.98:0/2123103694 >> 172.21.15.98:0/2123103694 conn(0x7f63a40b2010 msgr2=0x7f63a40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:27.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.310+0000 7f63a9e87640 1 -- 172.21.15.98:0/2123103694 shutdown_connections 2024-09-19T08:47:27.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.310+0000 7f63a9e87640 1 -- 172.21.15.98:0/2123103694 wait complete. 2024-09-19T08:47:27.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.310+0000 7f63a9e87640 1 Processor -- start 2024-09-19T08:47:27.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.310+0000 7f63a9e87640 1 -- start start 2024-09-19T08:47:27.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.310+0000 7f63a9e87640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63a406c1d0 0x7f63a40ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:27.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.310+0000 7f63a9e87640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63a40cf900 0x7f63a40aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:27.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.310+0000 7f63a9e87640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63a415c550 0x7f63a41f7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:27.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.310+0000 7f63a9e87640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f63a4072520 con 0x7f63a415c550 2024-09-19T08:47:27.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.310+0000 7f63a9e87640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f63a40723a0 con 0x7f63a406c1d0 2024-09-19T08:47:27.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.310+0000 7f63a9e87640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f63a40726a0 con 0x7f63a40cf900 2024-09-19T08:47:27.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.310+0000 7f63a3fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63a415c550 0x7f63a41f7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:27.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.310+0000 7f63a3fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63a415c550 0x7f63a41f7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:42060/0 (socket says 172.21.15.98:42060) 2024-09-19T08:47:27.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.310+0000 7f63a3fff640 1 -- 172.21.15.98:0/2533731609 learned_addr learned my addr 172.21.15.98:0/2533731609 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:27.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.310+0000 7f63a2ffd640 1 --2- 172.21.15.98:0/2533731609 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63a40cf900 0x7f63a40aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:27.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.310+0000 7f63a3fff640 1 -- 172.21.15.98:0/2533731609 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63a40cf900 msgr2=0x7f63a40aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:27.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.310+0000 7f63a37fe640 1 --2- 172.21.15.98:0/2533731609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63a406c1d0 0x7f63a40ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:27.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.310+0000 7f63a3fff640 1 --2- 172.21.15.98:0/2533731609 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63a40cf900 0x7f63a40aedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:27.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.310+0000 7f63a3fff640 1 -- 172.21.15.98:0/2533731609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63a406c1d0 msgr2=0x7f63a40ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:27.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.310+0000 7f63a3fff640 1 --2- 172.21.15.98:0/2533731609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63a406c1d0 0x7f63a40ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:27.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.310+0000 7f63a3fff640 1 -- 172.21.15.98:0/2533731609 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a40b5430 con 0x7f63a415c550 2024-09-19T08:47:27.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.310+0000 7f63a3fff640 1 --2- 172.21.15.98:0/2533731609 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63a415c550 0x7f63a41f7380 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f639c00efc0 tx=0x7f639c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:27.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.310+0000 7f638bfff640 1 -- 172.21.15.98:0/2533731609 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639c018020 con 0x7f63a415c550 2024-09-19T08:47:27.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.310+0000 7f638bfff640 1 -- 172.21.15.98:0/2533731609 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f639c014480 con 0x7f63a415c550 2024-09-19T08:47:27.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.310+0000 7f63a9e87640 1 -- 172.21.15.98:0/2533731609 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63a40b4670 con 0x7f63a415c550 2024-09-19T08:47:27.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.310+0000 7f638bfff640 1 -- 172.21.15.98:0/2533731609 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639c014870 con 0x7f63a415c550 2024-09-19T08:47:27.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.310+0000 7f63a9e87640 1 -- 172.21.15.98:0/2533731609 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63a40b4c30 con 0x7f63a415c550 2024-09-19T08:47:27.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.314+0000 7f63a9e87640 1 -- 172.21.15.98:0/2533731609 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63a40ce1a0 con 0x7f63a415c550 2024-09-19T08:47:27.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.314+0000 7f638bfff640 1 -- 172.21.15.98:0/2533731609 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f639c01b020 con 0x7f63a415c550 2024-09-19T08:47:27.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.314+0000 7f638bfff640 1 --2- 172.21.15.98:0/2533731609 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6390068250 0x7f639006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:27.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.314+0000 7f638bfff640 1 -- 172.21.15.98:0/2533731609 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5737+0+0 (secure 0 0 0) 0x7f639c085180 con 0x7f63a415c550 2024-09-19T08:47:27.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.314+0000 7f63a37fe640 1 --2- 172.21.15.98:0/2533731609 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6390068250 0x7f639006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:27.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.314+0000 7f63a37fe640 1 --2- 172.21.15.98:0/2533731609 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6390068250 0x7f639006a710 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f63940ad000 tx=0x7f6394008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:27.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.318+0000 7f638bfff640 1 -- 172.21.15.98:0/2533731609 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f639c022030 con 0x7f63a415c550 2024-09-19T08:47:27.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.518+0000 7f63a9e87640 1 -- 172.21.15.98:0/2533731609 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f63a41545a0 con 0x7f63a415c550 2024-09-19T08:47:27.519 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.522+0000 7f638bfff640 1 -- 172.21.15.98:0/2533731609 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v100) ==== 53+0+73 (secure 0 0 0) 0x7f639c010040 con 0x7f63a415c550 2024-09-19T08:47:27.520 INFO:teuthology.orchestra.run.smithi098.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T08:47:27.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.522+0000 7f63a9e87640 1 -- 172.21.15.98:0/2533731609 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6390068250 msgr2=0x7f639006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:27.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.522+0000 7f63a9e87640 1 --2- 172.21.15.98:0/2533731609 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6390068250 0x7f639006a710 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f63940ad000 tx=0x7f6394008040 comp rx=0 tx=0).stop 2024-09-19T08:47:27.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.522+0000 7f63a9e87640 1 -- 172.21.15.98:0/2533731609 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63a415c550 msgr2=0x7f63a41f7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:27.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.522+0000 7f63a9e87640 1 --2- 172.21.15.98:0/2533731609 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63a415c550 0x7f63a41f7380 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f639c00efc0 tx=0x7f639c00c490 comp rx=0 tx=0).stop 2024-09-19T08:47:27.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.522+0000 7f63a9e87640 1 -- 172.21.15.98:0/2533731609 shutdown_connections 2024-09-19T08:47:27.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.522+0000 7f63a9e87640 1 --2- 172.21.15.98:0/2533731609 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6390068250 0x7f639006a710 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:27.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.522+0000 7f63a9e87640 1 --2- 172.21.15.98:0/2533731609 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f63a415c550 0x7f63a41f7380 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:27.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.522+0000 7f63a9e87640 1 --2- 172.21.15.98:0/2533731609 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f63a40cf900 0x7f63a40aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:27.522 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.522+0000 7f63a9e87640 1 --2- 172.21.15.98:0/2533731609 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63a406c1d0 0x7f63a40ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:27.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.522+0000 7f63a9e87640 1 -- 172.21.15.98:0/2533731609 >> 172.21.15.98:0/2533731609 conn(0x7f63a40b2010 msgr2=0x7f63a40aff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:27.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.522+0000 7f63a9e87640 1 -- 172.21.15.98:0/2533731609 shutdown_connections 2024-09-19T08:47:27.523 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:27.522+0000 7f63a9e87640 1 -- 172.21.15.98:0/2533731609 wait complete. 2024-09-19T08:47:27.538 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T08:47:27.539 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T08:47:27.539 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi139.front.sepia.ceph.com 2024-09-19T08:47:27.539 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-19T08:47:27.539 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-19T08:47:27.539 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T08:47:27.539 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T08:47:27.539 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T08:47:27.539 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T08:47:27.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:27.539 DEBUG:teuthology.orchestra.run.smithi139:> ip addr 2024-09-19T08:47:27.546 INFO:teuthology.orchestra.run.smithi139.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T08:47:27.546 INFO:teuthology.orchestra.run.smithi139.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T08:47:27.547 INFO:teuthology.orchestra.run.smithi139.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T08:47:27.547 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:47:27.547 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 ::1/128 scope host 2024-09-19T08:47:27.547 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:47:27.547 INFO:teuthology.orchestra.run.smithi139.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:47:27.547 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e8 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:47:27.547 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f0 2024-09-19T08:47:27.547 INFO:teuthology.orchestra.run.smithi139.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:47:27.547 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:d9:36:e9 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:47:27.547 INFO:teuthology.orchestra.run.smithi139.stdout: altname enp7s0f1 2024-09-19T08:47:27.547 INFO:teuthology.orchestra.run.smithi139.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T08:47:27.547 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:58 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:47:27.547 INFO:teuthology.orchestra.run.smithi139.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T08:47:27.547 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0c:c4:7a:88:7f:59 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:47:27.548 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.21.15.139/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T08:47:27.548 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft 41972sec preferred_lft 41972sec 2024-09-19T08:47:27.548 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::ec4:7aff:fe88:7f59/64 scope link 2024-09-19T08:47:27.548 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:47:27.548 INFO:teuthology.orchestra.run.smithi139.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T08:47:27.548 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 02:42:74:ed:e4:60 brd ff:ff:ff:ff:ff:ff 2024-09-19T08:47:27.548 INFO:teuthology.orchestra.run.smithi139.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T08:47:27.548 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:47:27.548 INFO:teuthology.orchestra.run.smithi139.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T08:47:27.548 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-19T08:47:27.548 INFO:teuthology.orchestra.run.smithi139.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T08:47:27.548 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:47:27.548 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-19T08:47:27.549 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:47:27.549 INFO:teuthology.orchestra.run.smithi139.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:47:27.549 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:47:27.549 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-19T08:47:27.549 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:47:27.549 INFO:teuthology.orchestra.run.smithi139.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T08:47:27.549 INFO:teuthology.orchestra.run.smithi139.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:47:27.549 INFO:teuthology.orchestra.run.smithi139.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-19T08:47:27.549 INFO:teuthology.orchestra.run.smithi139.stdout: valid_lft forever preferred_lft forever 2024-09-19T08:47:27.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:27.549 DEBUG:teuthology.orchestra.run.smithi139:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T08:47:27.601 INFO:teuthology.orchestra.run.smithi139.stdout:1 2024-09-19T08:47:27.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:27.603 DEBUG:teuthology.orchestra.run.smithi139:> route 2024-09-19T08:47:27.655 INFO:teuthology.orchestra.run.smithi139.stdout:Kernel IP routing table 2024-09-19T08:47:27.656 INFO:teuthology.orchestra.run.smithi139.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T08:47:27.656 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T08:47:27.656 INFO:teuthology.orchestra.run.smithi139.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T08:47:27.656 INFO:teuthology.orchestra.run.smithi139.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T08:47:27.656 INFO:teuthology.orchestra.run.smithi139.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T08:47:27.656 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T08:47:27.656 INFO:teuthology.orchestra.run.smithi139.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:47:27.656 INFO:teuthology.orchestra.run.smithi139.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:47:27.656 INFO:teuthology.orchestra.run.smithi139.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T08:47:27.656 INFO:teuthology.orchestra.run.smithi139.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T08:47:27.657 DEBUG:teuthology.orchestra.run.smithi139:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T08:47:27.657 DEBUG:teuthology.orchestra.run.smithi139:> set -e 2024-09-19T08:47:27.657 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T08:47:27.657 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T08:47:27.657 DEBUG:teuthology.orchestra.run.smithi139:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T08:47:27.657 DEBUG:teuthology.orchestra.run.smithi139:> ') 2024-09-19T08:47:27.744 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:47:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T08:47:27.774 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:47:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:47:27.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:27.785 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-09-19T08:47:27.790 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-19T08:47:27.790 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-19T08:47:27.790 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T08:47:27.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:27.791 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link set brx.0 up 2024-09-19T08:47:27.846 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-19T08:47:27.846 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:47:27.846 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-19T08:47:27.894 INFO:teuthology.orchestra.run.smithi139.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-19T08:47:27.894 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T08:47:27.894 DEBUG:teuthology.orchestra.run.smithi139:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-19T08:47:27.941 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:47:27.999 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:47:28.048 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:47:28.048 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:47:28.048 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:47:28.049 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:47:28.049 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:47:28.049 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:47:28.049 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:47:28.049 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:47:28.049 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:47:28.049 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:47:28.049 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:47:28.049 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:47:28.049 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:47:28.049 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:47:28.049 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:47:28.050 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:47:28.050 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:47:28.050 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:47:28.050 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:47:28.050 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:47:28.050 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:47:28.050 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:47:28.050 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:47:28.050 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:47:28.050 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:47:28.050 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:47:28.050 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:47:28.050 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:47:28.050 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:47:28.050 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:47:28.051 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:47:28.051 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:47:28.051 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:47:28.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:28.052 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:47:28.098 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T08:47:28.098 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-19T08:47:28.143 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:47:28.157 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:47:28.206 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:47:28.206 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:47:28.206 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:47:28.207 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:47:28.207 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:47:28.207 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:47:28.207 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:47:28.207 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:47:28.207 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:47:28.207 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:47:28.207 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:47:28.207 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:47:28.207 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:47:28.207 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:47:28.207 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:47:28.208 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:47:28.208 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:47:28.208 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:47:28.208 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:47:28.208 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:47:28.208 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:47:28.208 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:47:28.208 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:47:28.208 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:47:28.208 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:47:28.209 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:47:28.209 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:47:28.209 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:47:28.209 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:47:28.209 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:47:28.209 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:47:28.209 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:47:28.209 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:47:28.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:28.210 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:47:28.223 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:2024-09-19T08:47:28.225+0000 7f84f1ab3580 -1 init, newargv = 0x562121c55210 newargc=15 2024-09-19T08:47:28.224 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[37158]: starting ceph client 2024-09-19T08:47:28.229 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[37158]: starting fuse 2024-09-19T08:47:29.329 DEBUG:teuthology.orchestra.run.smithi139:> sudo modprobe fuse 2024-09-19T08:47:29.344 DEBUG:teuthology.orchestra.run.smithi139:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T08:47:29.393 INFO:teuthology.orchestra.run.smithi139.stdout:/sys 2024-09-19T08:47:29.393 INFO:teuthology.orchestra.run.smithi139.stdout:/proc 2024-09-19T08:47:29.393 INFO:teuthology.orchestra.run.smithi139.stdout:/dev 2024-09-19T08:47:29.393 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/pts 2024-09-19T08:47:29.393 INFO:teuthology.orchestra.run.smithi139.stdout:/run 2024-09-19T08:47:29.393 INFO:teuthology.orchestra.run.smithi139.stdout:/ 2024-09-19T08:47:29.393 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/security 2024-09-19T08:47:29.394 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/shm 2024-09-19T08:47:29.394 INFO:teuthology.orchestra.run.smithi139.stdout:/run/lock 2024-09-19T08:47:29.394 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/cgroup 2024-09-19T08:47:29.394 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/pstore 2024-09-19T08:47:29.394 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/bpf 2024-09-19T08:47:29.394 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T08:47:29.394 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/hugepages 2024-09-19T08:47:29.394 INFO:teuthology.orchestra.run.smithi139.stdout:/dev/mqueue 2024-09-19T08:47:29.394 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/debug 2024-09-19T08:47:29.394 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/tracing 2024-09-19T08:47:29.394 INFO:teuthology.orchestra.run.smithi139.stdout:/proc/fs/nfsd 2024-09-19T08:47:29.394 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/fs/fuse/connections 2024-09-19T08:47:29.395 INFO:teuthology.orchestra.run.smithi139.stdout:/sys/kernel/config 2024-09-19T08:47:29.395 INFO:teuthology.orchestra.run.smithi139.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T08:47:29.395 INFO:teuthology.orchestra.run.smithi139.stdout:/run/rpc_pipefs 2024-09-19T08:47:29.395 INFO:teuthology.orchestra.run.smithi139.stdout:/run/user/1000 2024-09-19T08:47:29.395 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph 2024-09-19T08:47:29.395 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T08:47:29.395 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T08:47:29.395 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T08:47:29.395 INFO:teuthology.orchestra.run.smithi139.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T08:47:29.395 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns 2024-09-19T08:47:29.395 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:47:29.395 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:47:29.395 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:47:29.395 INFO:teuthology.orchestra.run.smithi139.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:47:29.396 INFO:teuthology.orchestra.run.smithi139.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-19T08:47:29.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:29.397 DEBUG:teuthology.orchestra.run.smithi139:> ls /sys/fs/fuse/connections 2024-09-19T08:47:29.442 INFO:teuthology.orchestra.run.smithi139.stdout:45 2024-09-19T08:47:29.443 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-19T08:47:29.443 DEBUG:teuthology.orchestra.run.smithi139:> sudo stdin-killer -- python3 -c ' 2024-09-19T08:47:29.443 DEBUG:teuthology.orchestra.run.smithi139:> import glob 2024-09-19T08:47:29.443 DEBUG:teuthology.orchestra.run.smithi139:> import re 2024-09-19T08:47:29.444 DEBUG:teuthology.orchestra.run.smithi139:> import os 2024-09-19T08:47:29.444 DEBUG:teuthology.orchestra.run.smithi139:> import subprocess 2024-09-19T08:47:29.444 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:47:29.444 DEBUG:teuthology.orchestra.run.smithi139:> def _find_admin_socket(client_name): 2024-09-19T08:47:29.444 DEBUG:teuthology.orchestra.run.smithi139:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-19T08:47:29.444 DEBUG:teuthology.orchestra.run.smithi139:> files = glob.glob(asok_path) 2024-09-19T08:47:29.444 DEBUG:teuthology.orchestra.run.smithi139:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-19T08:47:29.444 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:47:29.444 DEBUG:teuthology.orchestra.run.smithi139:> # Given a non-glob path, it better be there 2024-09-19T08:47:29.444 DEBUG:teuthology.orchestra.run.smithi139:> if "*" not in asok_path: 2024-09-19T08:47:29.444 DEBUG:teuthology.orchestra.run.smithi139:> assert(len(files) == 1) 2024-09-19T08:47:29.444 DEBUG:teuthology.orchestra.run.smithi139:> return files[0] 2024-09-19T08:47:29.444 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:47:29.445 DEBUG:teuthology.orchestra.run.smithi139:> for f in files: 2024-09-19T08:47:29.445 DEBUG:teuthology.orchestra.run.smithi139:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T08:47:29.445 DEBUG:teuthology.orchestra.run.smithi139:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T08:47:29.445 DEBUG:teuthology.orchestra.run.smithi139:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T08:47:29.445 DEBUG:teuthology.orchestra.run.smithi139:> contents = proc_f.read() 2024-09-19T08:47:29.445 DEBUG:teuthology.orchestra.run.smithi139:> if mountpoint in contents: 2024-09-19T08:47:29.445 DEBUG:teuthology.orchestra.run.smithi139:> return f 2024-09-19T08:47:29.445 DEBUG:teuthology.orchestra.run.smithi139:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T08:47:29.445 DEBUG:teuthology.orchestra.run.smithi139:> 2024-09-19T08:47:29.445 DEBUG:teuthology.orchestra.run.smithi139:> print(_find_admin_socket("client.1")) 2024-09-19T08:47:29.445 DEBUG:teuthology.orchestra.run.smithi139:> ' 2024-09-19T08:47:29.567 INFO:teuthology.orchestra.run.smithi139.stdout:/var/run/ceph/ceph-client.1.37158.asok 2024-09-19T08:47:29.570 INFO:teuthology.orchestra.run.smithi139.stderr:2024-09-19T08:47:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:47:29.578 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.37158.asok 2024-09-19T08:47:29.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:29.579 DEBUG:teuthology.orchestra.run.smithi139:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.37158.asok status 2024-09-19T08:47:29.734 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-09-19T08:47:29.734 INFO:teuthology.orchestra.run.smithi139.stdout: "metadata": { 2024-09-19T08:47:29.734 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T08:47:29.734 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T08:47:29.734 INFO:teuthology.orchestra.run.smithi139.stdout: "entity_id": "1", 2024-09-19T08:47:29.735 INFO:teuthology.orchestra.run.smithi139.stdout: "hostname": "smithi139", 2024-09-19T08:47:29.735 INFO:teuthology.orchestra.run.smithi139.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-19T08:47:29.735 INFO:teuthology.orchestra.run.smithi139.stdout: "pid": "37158", 2024-09-19T08:47:29.735 INFO:teuthology.orchestra.run.smithi139.stdout: "root": "/" 2024-09-19T08:47:29.735 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:47:29.735 INFO:teuthology.orchestra.run.smithi139.stdout: "dentry_count": 0, 2024-09-19T08:47:29.735 INFO:teuthology.orchestra.run.smithi139.stdout: "dentry_pinned_count": 0, 2024-09-19T08:47:29.735 INFO:teuthology.orchestra.run.smithi139.stdout: "id": 6479, 2024-09-19T08:47:29.735 INFO:teuthology.orchestra.run.smithi139.stdout: "inst": { 2024-09-19T08:47:29.735 INFO:teuthology.orchestra.run.smithi139.stdout: "name": { 2024-09-19T08:47:29.735 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "client", 2024-09-19T08:47:29.735 INFO:teuthology.orchestra.run.smithi139.stdout: "num": 6479 2024-09-19T08:47:29.735 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:47:29.735 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": { 2024-09-19T08:47:29.736 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-19T08:47:29.736 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:47:29.736 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 169798316 2024-09-19T08:47:29.736 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-09-19T08:47:29.736 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:47:29.736 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": { 2024-09-19T08:47:29.736 INFO:teuthology.orchestra.run.smithi139.stdout: "type": "v1", 2024-09-19T08:47:29.736 INFO:teuthology.orchestra.run.smithi139.stdout: "addr": "192.168.0.1:0", 2024-09-19T08:47:29.736 INFO:teuthology.orchestra.run.smithi139.stdout: "nonce": 169798316 2024-09-19T08:47:29.736 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-09-19T08:47:29.736 INFO:teuthology.orchestra.run.smithi139.stdout: "inst_str": "client.6479 192.168.0.1:0/169798316", 2024-09-19T08:47:29.736 INFO:teuthology.orchestra.run.smithi139.stdout: "addr_str": "192.168.0.1:0/169798316", 2024-09-19T08:47:29.736 INFO:teuthology.orchestra.run.smithi139.stdout: "inode_count": 1, 2024-09-19T08:47:29.737 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_epoch": 100, 2024-09-19T08:47:29.737 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_epoch": 170, 2024-09-19T08:47:29.737 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_epoch_barrier": 0, 2024-09-19T08:47:29.737 INFO:teuthology.orchestra.run.smithi139.stdout: "blocklisted": false, 2024-09-19T08:47:29.737 INFO:teuthology.orchestra.run.smithi139.stdout: "fs_name": "cephfs" 2024-09-19T08:47:29.737 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-09-19T08:47:29.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:29.748 DEBUG:teuthology.orchestra.run.smithi139:> stat --file-system '--printf=%T 2024-09-19T08:47:29.748 DEBUG:teuthology.orchestra.run.smithi139:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-19T08:47:29.753 INFO:teuthology.orchestra.run.smithi139.stdout:fuseblk 2024-09-19T08:47:29.753 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-19T08:47:29.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:29.754 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-19T08:47:29.810 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-19T08:47:29.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.950+0000 7f2ee192a640 1 Processor -- start 2024-09-19T08:47:29.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.950+0000 7f2ee192a640 1 -- start start 2024-09-19T08:47:29.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.954+0000 7f2ee192a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f2edc06dbd0 0x7f2edc073680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:29.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.954+0000 7f2ee192a640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2edc15baf0 con 0x7f2edc15f6e0 2024-09-19T08:47:29.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.954+0000 7f2ee192a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2edc15bdd0 con 0x7f2edc15fac0 2024-09-19T08:47:29.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.954+0000 7f2ee192a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f2edc15c8b0 con 0x7f2edc06dbd0 2024-09-19T08:47:29.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.954+0000 7f2edaffd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f2edc15f6e0 0x7f2edc06d4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:44446/0 (socket says 172.21.15.98:44446) 2024-09-19T08:47:29.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.954+0000 7f2edaffd640 1 -- 172.21.15.98:0/4039177199 learned_addr learned my addr 172.21.15.98:0/4039177199 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:29.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.954+0000 7f2eda7fc640 1 --2- 172.21.15.98:0/4039177199 >> v2:172.21.15.139:3300/0 conn(0x7f2edc06dbd0 0x7f2edc073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:29.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.954+0000 7f2ed9ffb640 1 -- 172.21.15.98:0/4039177199 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3679721140 0 0) 0x7f2edc15baf0 con 0x7f2edc15f6e0 2024-09-19T08:47:29.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.954+0000 7f2ed9ffb640 1 -- 172.21.15.98:0/4039177199 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ec4005740 con 0x7f2edc15f6e0 2024-09-19T08:47:29.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.954+0000 7f2ed9ffb640 1 -- 172.21.15.98:0/4039177199 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 324041519 0 0) 0x7f2ec4005740 con 0x7f2edc15f6e0 2024-09-19T08:47:29.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.954+0000 7f2ed9ffb640 1 -- 172.21.15.98:0/4039177199 >> v2:172.21.15.139:3300/0 conn(0x7f2edc06dbd0 msgr2=0x7f2edc073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:29.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.954+0000 7f2ed9ffb640 1 --2- 172.21.15.98:0/4039177199 >> v2:172.21.15.139:3300/0 conn(0x7f2edc06dbd0 0x7f2edc073680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:29.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.954+0000 7f2ed9ffb640 1 -- 172.21.15.98:0/4039177199 >> v1:172.21.15.139:6789/0 conn(0x7f2edc15fac0 legacy=0x7f2edc0ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:29.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.954+0000 7f2ed9ffb640 1 -- 172.21.15.98:0/4039177199 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2edc15c0b0 con 0x7f2edc15f6e0 2024-09-19T08:47:29.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.954+0000 7f2ed9ffb640 1 -- 172.21.15.98:0/4039177199 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f2ecc001680 con 0x7f2edc15f6e0 2024-09-19T08:47:29.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.954+0000 7f2ed9ffb640 1 -- 172.21.15.98:0/4039177199 >> v1:172.21.15.98:6789/0 conn(0x7f2edc15f6e0 legacy=0x7f2edc06d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:29.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.954+0000 7f2ed9ffb640 1 --2- 172.21.15.98:0/4039177199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ec4005bd0 0x7f2ec4005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:29.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.954+0000 7f2ed9ffb640 1 --2- 172.21.15.98:0/4039177199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ec4006a20 0x7f2ec4006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:29.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.954+0000 7f2ed9ffb640 1 --2- 172.21.15.98:0/4039177199 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ec40073e0 0x7f2ec400dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:29.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.954+0000 7f2ed9ffb640 1 -- 172.21.15.98:0/4039177199 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc15c8b0 con 0x7f2ec4006a20 2024-09-19T08:47:29.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.954+0000 7f2ed9ffb640 1 -- 172.21.15.98:0/4039177199 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec400e280 con 0x7f2ec4005bd0 2024-09-19T08:47:29.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.954+0000 7f2ed9ffb640 1 -- 172.21.15.98:0/4039177199 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2ec400e480 con 0x7f2ec40073e0 2024-09-19T08:47:29.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.954+0000 7f2eda7fc640 1 --2- 172.21.15.98:0/4039177199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ec4006a20 0x7f2ec4006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:29.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.954+0000 7f2edaffd640 1 --2- 172.21.15.98:0/4039177199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ec4005bd0 0x7f2ec4005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:29.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.954+0000 7f2edb7fe640 1 --2- 172.21.15.98:0/4039177199 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ec40073e0 0x7f2ec400dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:29.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.954+0000 7f2eda7fc640 1 -- 172.21.15.98:0/4039177199 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ec40073e0 msgr2=0x7f2ec400dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:29.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.954+0000 7f2eda7fc640 1 --2- 172.21.15.98:0/4039177199 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2ec40073e0 0x7f2ec400dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:29.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.954+0000 7f2eda7fc640 1 -- 172.21.15.98:0/4039177199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ec4005bd0 msgr2=0x7f2ec4005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:29.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.954+0000 7f2eda7fc640 1 --2- 172.21.15.98:0/4039177199 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ec4005bd0 0x7f2ec4005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:29.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.954+0000 7f2eda7fc640 1 -- 172.21.15.98:0/4039177199 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec4005740 con 0x7f2ec4006a20 2024-09-19T08:47:29.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.954+0000 7f2edb7fe640 1 -- 172.21.15.98:0/4039177199 reap_dead start 2024-09-19T08:47:29.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.958+0000 7f2eda7fc640 1 --2- 172.21.15.98:0/4039177199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ec4006a20 0x7f2ec4006ea0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0009ab0 tx=0x7f2ed001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d245a75eeccb66a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:29.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.958+0000 7f2ed9ffb640 1 -- 172.21.15.98:0/4039177199 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed00b2070 con 0x7f2ec4006a20 2024-09-19T08:47:29.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.958+0000 7f2ed9ffb640 1 -- 172.21.15.98:0/4039177199 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ed00a5530 con 0x7f2ec4006a20 2024-09-19T08:47:29.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.958+0000 7f2ed9ffb640 1 -- 172.21.15.98:0/4039177199 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed00096b0 con 0x7f2ec4006a20 2024-09-19T08:47:29.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.958+0000 7f2ee192a640 1 -- 172.21.15.98:0/4039177199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ec4006a20 msgr2=0x7f2ec4006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:29.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.958+0000 7f2ee192a640 1 --2- 172.21.15.98:0/4039177199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ec4006a20 0x7f2ec4006ea0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0009ab0 tx=0x7f2ed001ca50 comp rx=0 tx=0).stop 2024-09-19T08:47:29.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.958+0000 7f2ee192a640 1 -- 172.21.15.98:0/4039177199 shutdown_connections 2024-09-19T08:47:29.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.958+0000 7f2ee192a640 1 --2- 172.21.15.98:0/4039177199 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2ec4006a20 0x7f2ec4006ea0 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:29.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.958+0000 7f2ee192a640 1 -- 172.21.15.98:0/4039177199 >> 172.21.15.98:0/4039177199 conn(0x7f2edc06c090 msgr2=0x7f2edc06c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:29.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.958+0000 7f2ee192a640 1 -- 172.21.15.98:0/4039177199 shutdown_connections 2024-09-19T08:47:29.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.958+0000 7f2ee192a640 1 -- 172.21.15.98:0/4039177199 wait complete. 2024-09-19T08:47:29.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.958+0000 7f2ee192a640 1 Processor -- start 2024-09-19T08:47:29.959 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.958+0000 7f2ee192a640 1 -- start start 2024-09-19T08:47:29.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.958+0000 7f2ee192a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ec4006a20 0x7f2edc0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:29.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.958+0000 7f2ee192a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2edc15b280 0x7f2edc14cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:29.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.958+0000 7f2ee192a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2edc153310 0x7f2edc1537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:29.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.958+0000 7f2ee192a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc15b890 con 0x7f2edc15b280 2024-09-19T08:47:29.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.958+0000 7f2ee192a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2edc0742d0 con 0x7f2ec4006a20 2024-09-19T08:47:29.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.958+0000 7f2ee192a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f2edc06d4c0 con 0x7f2edc153310 2024-09-19T08:47:29.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.958+0000 7f2eda7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2edc15b280 0x7f2edc14cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:29.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.958+0000 7f2eda7fc640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2edc15b280 0x7f2edc14cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39190/0 (socket says 172.21.15.98:39190) 2024-09-19T08:47:29.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.958+0000 7f2eda7fc640 1 -- 172.21.15.98:0/3883475650 learned_addr learned my addr 172.21.15.98:0/3883475650 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:29.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.958+0000 7f2edaffd640 1 --2- 172.21.15.98:0/3883475650 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ec4006a20 0x7f2edc0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:29.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.958+0000 7f2edb7fe640 1 --2- 172.21.15.98:0/3883475650 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2edc153310 0x7f2edc1537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:29.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.958+0000 7f2eda7fc640 1 -- 172.21.15.98:0/3883475650 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2edc153310 msgr2=0x7f2edc1537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:29.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.958+0000 7f2eda7fc640 1 --2- 172.21.15.98:0/3883475650 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2edc153310 0x7f2edc1537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:29.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.958+0000 7f2eda7fc640 1 -- 172.21.15.98:0/3883475650 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ec4006a20 msgr2=0x7f2edc0ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:29.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.958+0000 7f2eda7fc640 1 --2- 172.21.15.98:0/3883475650 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ec4006a20 0x7f2edc0ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:29.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.958+0000 7f2eda7fc640 1 -- 172.21.15.98:0/3883475650 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2edc15ae30 con 0x7f2edc15b280 2024-09-19T08:47:29.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.962+0000 7f2eda7fc640 1 --2- 172.21.15.98:0/3883475650 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2edc15b280 0x7f2edc14cf10 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f2ed001c880 tx=0x7f2ed00197f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:29.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.962+0000 7f2ec3fff640 1 -- 172.21.15.98:0/3883475650 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed00b2040 con 0x7f2edc15b280 2024-09-19T08:47:29.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.962+0000 7f2ec3fff640 1 -- 172.21.15.98:0/3883475650 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ed000f000 con 0x7f2edc15b280 2024-09-19T08:47:29.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.962+0000 7f2ee192a640 1 -- 172.21.15.98:0/3883475650 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2edc0af2e0 con 0x7f2edc15b280 2024-09-19T08:47:29.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.962+0000 7f2ec3fff640 1 -- 172.21.15.98:0/3883475650 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0004260 con 0x7f2edc15b280 2024-09-19T08:47:29.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.962+0000 7f2ee192a640 1 -- 172.21.15.98:0/3883475650 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2edc0af000 con 0x7f2edc15b280 2024-09-19T08:47:29.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.962+0000 7f2ee192a640 1 -- 172.21.15.98:0/3883475650 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2edc06d850 con 0x7f2edc15b280 2024-09-19T08:47:29.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.962+0000 7f2ec3fff640 1 -- 172.21.15.98:0/3883475650 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f2ed00bf030 con 0x7f2edc15b280 2024-09-19T08:47:29.963 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.962+0000 7f2ec3fff640 1 --2- 172.21.15.98:0/3883475650 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2ec8068250 0x7f2ec806a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:29.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.962+0000 7f2ec3fff640 1 -- 172.21.15.98:0/3883475650 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5737+0+0 (secure 0 0 0) 0x7f2ed011e290 con 0x7f2edc15b280 2024-09-19T08:47:29.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.962+0000 7f2edaffd640 1 --2- 172.21.15.98:0/3883475650 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2ec8068250 0x7f2ec806a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:29.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.962+0000 7f2edaffd640 1 --2- 172.21.15.98:0/3883475650 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2ec8068250 0x7f2ec806a710 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2edc0512d0 tx=0x7f2ecc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:29.966 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:29.966+0000 7f2ec3fff640 1 -- 172.21.15.98:0/3883475650 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2ed00c6060 con 0x7f2edc15b280 2024-09-19T08:47:30.164 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.166+0000 7f2ee192a640 1 -- 172.21.15.98:0/3883475650 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f2edc0ccea0 con 0x7f2edc15b280 2024-09-19T08:47:30.390 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.390+0000 7f2ec3fff640 1 -- 172.21.15.98:0/3883475650 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v101) ==== 136+0+0 (secure 0 0 0) 0x7f2edc15ae30 con 0x7f2edc15b280 2024-09-19T08:47:30.393 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.394+0000 7f2ee192a640 1 -- 172.21.15.98:0/3883475650 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2ec8068250 msgr2=0x7f2ec806a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:30.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.394+0000 7f2ee192a640 1 --2- 172.21.15.98:0/3883475650 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2ec8068250 0x7f2ec806a710 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2edc0512d0 tx=0x7f2ecc009040 comp rx=0 tx=0).stop 2024-09-19T08:47:30.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.394+0000 7f2ee192a640 1 -- 172.21.15.98:0/3883475650 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2edc15b280 msgr2=0x7f2edc14cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:30.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.394+0000 7f2ee192a640 1 --2- 172.21.15.98:0/3883475650 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2edc15b280 0x7f2edc14cf10 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f2ed001c880 tx=0x7f2ed00197f0 comp rx=0 tx=0).stop 2024-09-19T08:47:30.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.394+0000 7f2ee192a640 1 -- 172.21.15.98:0/3883475650 shutdown_connections 2024-09-19T08:47:30.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.394+0000 7f2ee192a640 1 --2- 172.21.15.98:0/3883475650 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f2ec8068250 0x7f2ec806a710 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:30.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.394+0000 7f2ee192a640 1 --2- 172.21.15.98:0/3883475650 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f2edc153310 0x7f2edc1537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:30.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.394+0000 7f2ee192a640 1 --2- 172.21.15.98:0/3883475650 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f2edc15b280 0x7f2edc14cf10 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:30.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.394+0000 7f2ee192a640 1 --2- 172.21.15.98:0/3883475650 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2ec4006a20 0x7f2edc0ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:30.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.394+0000 7f2ee192a640 1 -- 172.21.15.98:0/3883475650 >> 172.21.15.98:0/3883475650 conn(0x7f2edc06c090 msgr2=0x7f2edc0717f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:30.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.394+0000 7f2ee192a640 1 -- 172.21.15.98:0/3883475650 shutdown_connections 2024-09-19T08:47:30.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.394+0000 7f2ee192a640 1 -- 172.21.15.98:0/3883475650 wait complete. 2024-09-19T08:47:30.415 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-19T08:47:30.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.538+0000 7f0f8b113640 1 Processor -- start 2024-09-19T08:47:30.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.538+0000 7f0f8b113640 1 -- start start 2024-09-19T08:47:30.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.538+0000 7f0f8b113640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0f8415c550 0x7f0f8406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:30.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.538+0000 7f0f8b113640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0f8406c1d0 0x7f0f840cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:30.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.538+0000 7f0f8b113640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f8406d310 con 0x7f0f840cf900 2024-09-19T08:47:30.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.538+0000 7f0f8b113640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f0f8406c610 con 0x7f0f8406c1d0 2024-09-19T08:47:30.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.538+0000 7f0f8b113640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0f8406d5f0 con 0x7f0f8415c550 2024-09-19T08:47:30.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.538+0000 7f0f88eaf640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0f8415c550 0x7f0f8406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:30.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.538+0000 7f0f83fff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0f8406c1d0 0x7f0f840cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:30.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.538+0000 7f0f88eaf640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0f8415c550 0x7f0f8406bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:38872/0 (socket says 172.21.15.98:38872) 2024-09-19T08:47:30.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.538+0000 7f0f88eaf640 1 -- 172.21.15.98:0/2101478651 learned_addr learned my addr 172.21.15.98:0/2101478651 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:30.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.538+0000 7f0f83fff640 1 -- 172.21.15.98:0/2101478651 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0f8415c550 msgr2=0x7f0f8406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:30.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.538+0000 7f0f83fff640 1 --2- 172.21.15.98:0/2101478651 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0f8415c550 0x7f0f8406bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:30.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.538+0000 7f0f83fff640 1 -- 172.21.15.98:0/2101478651 >> v1:172.21.15.139:6789/0 conn(0x7f0f840cf900 legacy=0x7f0f840ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:30.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.538+0000 7f0f83fff640 1 -- 172.21.15.98:0/2101478651 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f8406d8d0 con 0x7f0f8406c1d0 2024-09-19T08:47:30.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.542+0000 7f0f83fff640 1 --2- 172.21.15.98:0/2101478651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f8406c1d0 0x7f0f840cf3c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f0f780097c0 tx=0x7f0f7801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a77964b987c1bd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:30.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.542+0000 7f0f837fe640 1 -- 172.21.15.98:0/2101478651 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f780b0070 con 0x7f0f8406c1d0 2024-09-19T08:47:30.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.542+0000 7f0f837fe640 1 -- 172.21.15.98:0/2101478651 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f780ab600 con 0x7f0f8406c1d0 2024-09-19T08:47:30.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.542+0000 7f0f837fe640 1 -- 172.21.15.98:0/2101478651 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f780ab9d0 con 0x7f0f8406c1d0 2024-09-19T08:47:30.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.542+0000 7f0f8b113640 1 -- 172.21.15.98:0/2101478651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f8406c1d0 msgr2=0x7f0f840cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:30.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.542+0000 7f0f8b113640 1 --2- 172.21.15.98:0/2101478651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f8406c1d0 0x7f0f840cf3c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f0f780097c0 tx=0x7f0f7801c4a0 comp rx=0 tx=0).stop 2024-09-19T08:47:30.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.542+0000 7f0f8b113640 1 -- 172.21.15.98:0/2101478651 shutdown_connections 2024-09-19T08:47:30.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.542+0000 7f0f8b113640 1 --2- 172.21.15.98:0/2101478651 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f8406c1d0 0x7f0f840cf3c0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:30.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.542+0000 7f0f8b113640 1 --2- 172.21.15.98:0/2101478651 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0f8415c550 0x7f0f8406bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:30.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.542+0000 7f0f8b113640 1 -- 172.21.15.98:0/2101478651 >> 172.21.15.98:0/2101478651 conn(0x7f0f840b2010 msgr2=0x7f0f840b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:30.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.542+0000 7f0f8b113640 1 -- 172.21.15.98:0/2101478651 shutdown_connections 2024-09-19T08:47:30.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.542+0000 7f0f8b113640 1 -- 172.21.15.98:0/2101478651 wait complete. 2024-09-19T08:47:30.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.546+0000 7f0f8b113640 1 Processor -- start 2024-09-19T08:47:30.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.546+0000 7f0f8b113640 1 -- start start 2024-09-19T08:47:30.551 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.546+0000 7f0f8b113640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f8406c1d0 0x7f0f84158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:30.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.546+0000 7f0f8b113640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f840cf900 0x7f0f841496b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:30.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.546+0000 7f0f8b113640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0f8415c550 0x7f0f840d0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:30.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.546+0000 7f0f8b113640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0f84169230 con 0x7f0f840cf900 2024-09-19T08:47:30.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.546+0000 7f0f8b113640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8415cb40 con 0x7f0f8406c1d0 2024-09-19T08:47:30.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.546+0000 7f0f8b113640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0f841693b0 con 0x7f0f8415c550 2024-09-19T08:47:30.552 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.546+0000 7f0f83fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f840cf900 0x7f0f841496b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:30.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.546+0000 7f0f83fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f840cf900 0x7f0f841496b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39194/0 (socket says 172.21.15.98:39194) 2024-09-19T08:47:30.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.546+0000 7f0f83fff640 1 -- 172.21.15.98:0/3572614235 learned_addr learned my addr 172.21.15.98:0/3572614235 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:30.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.546+0000 7f0f88eaf640 1 --2- 172.21.15.98:0/3572614235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f8406c1d0 0x7f0f84158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:30.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.546+0000 7f0f896b0640 1 --2- 172.21.15.98:0/3572614235 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0f8415c550 0x7f0f840d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:30.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.546+0000 7f0f83fff640 1 -- 172.21.15.98:0/3572614235 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0f8415c550 msgr2=0x7f0f840d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:30.553 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.546+0000 7f0f83fff640 1 --2- 172.21.15.98:0/3572614235 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0f8415c550 0x7f0f840d0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:30.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.546+0000 7f0f83fff640 1 -- 172.21.15.98:0/3572614235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f8406c1d0 msgr2=0x7f0f84158090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:30.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.546+0000 7f0f83fff640 1 --2- 172.21.15.98:0/3572614235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f8406c1d0 0x7f0f84158090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:30.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.546+0000 7f0f83fff640 1 -- 172.21.15.98:0/3572614235 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f841585d0 con 0x7f0f840cf900 2024-09-19T08:47:30.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.546+0000 7f0f88eaf640 1 --2- 172.21.15.98:0/3572614235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f8406c1d0 0x7f0f84158090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:47:30.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.546+0000 7f0f83fff640 1 --2- 172.21.15.98:0/3572614235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f840cf900 0x7f0f841496b0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f0f7801a0d0 tx=0x7f0f7801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:30.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.546+0000 7f0f817fa640 1 -- 172.21.15.98:0/3572614235 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f780b0050 con 0x7f0f840cf900 2024-09-19T08:47:30.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.546+0000 7f0f8b113640 1 -- 172.21.15.98:0/3572614235 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f84159910 con 0x7f0f840cf900 2024-09-19T08:47:30.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.546+0000 7f0f817fa640 1 -- 172.21.15.98:0/3572614235 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f780b1040 con 0x7f0f840cf900 2024-09-19T08:47:30.554 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.546+0000 7f0f817fa640 1 -- 172.21.15.98:0/3572614235 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f780b5610 con 0x7f0f840cf900 2024-09-19T08:47:30.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.546+0000 7f0f8b113640 1 -- 172.21.15.98:0/3572614235 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f84158b50 con 0x7f0f840cf900 2024-09-19T08:47:30.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.546+0000 7f0f8b113640 1 -- 172.21.15.98:0/3572614235 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f84149bf0 con 0x7f0f840cf900 2024-09-19T08:47:30.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.546+0000 7f0f817fa640 1 -- 172.21.15.98:0/3572614235 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f0f780be030 con 0x7f0f840cf900 2024-09-19T08:47:30.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.546+0000 7f0f817fa640 1 --2- 172.21.15.98:0/3572614235 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0f70068250 0x7f0f7006a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:30.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.550+0000 7f0f817fa640 1 -- 172.21.15.98:0/3572614235 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5737+0+0 (secure 0 0 0) 0x7f0f7811e140 con 0x7f0f840cf900 2024-09-19T08:47:30.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.550+0000 7f0f88eaf640 1 --2- 172.21.15.98:0/3572614235 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0f70068250 0x7f0f7006a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:30.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.550+0000 7f0f88eaf640 1 --2- 172.21.15.98:0/3572614235 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0f70068250 0x7f0f7006a710 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0f84156380 tx=0x7f0f74008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:30.555 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.554+0000 7f0f817fa640 1 -- 172.21.15.98:0/3572614235 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f0f780c5060 con 0x7f0f840cf900 2024-09-19T08:47:30.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.734+0000 7f0f8b113640 1 -- 172.21.15.98:0/3572614235 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f0f8415b5f0 con 0x7f0f70068250 2024-09-19T08:47:30.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.738+0000 7f0f817fa640 1 -- 172.21.15.98:0/3572614235 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f0f8415b5f0 con 0x7f0f70068250 2024-09-19T08:47:30.738 INFO:teuthology.orchestra.run.smithi098.stdout:[ 2024-09-19T08:47:30.738 INFO:teuthology.orchestra.run.smithi098.stdout: { 2024-09-19T08:47:30.738 INFO:teuthology.orchestra.run.smithi098.stdout: "name": "cephfs" 2024-09-19T08:47:30.738 INFO:teuthology.orchestra.run.smithi098.stdout: } 2024-09-19T08:47:30.738 INFO:teuthology.orchestra.run.smithi098.stdout:] 2024-09-19T08:47:30.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.738+0000 7f0f8b113640 1 -- 172.21.15.98:0/3572614235 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0f70068250 msgr2=0x7f0f7006a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:30.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.738+0000 7f0f8b113640 1 --2- 172.21.15.98:0/3572614235 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0f70068250 0x7f0f7006a710 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0f84156380 tx=0x7f0f74008040 comp rx=0 tx=0).stop 2024-09-19T08:47:30.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.742+0000 7f0f8b113640 1 -- 172.21.15.98:0/3572614235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f840cf900 msgr2=0x7f0f841496b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:30.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.742+0000 7f0f8b113640 1 --2- 172.21.15.98:0/3572614235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f840cf900 0x7f0f841496b0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f0f7801a0d0 tx=0x7f0f7801ace0 comp rx=0 tx=0).stop 2024-09-19T08:47:30.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.742+0000 7f0f8b113640 1 -- 172.21.15.98:0/3572614235 shutdown_connections 2024-09-19T08:47:30.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.742+0000 7f0f8b113640 1 --2- 172.21.15.98:0/3572614235 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0f70068250 0x7f0f7006a710 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:30.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.742+0000 7f0f8b113640 1 --2- 172.21.15.98:0/3572614235 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0f8415c550 0x7f0f840d0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:30.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.742+0000 7f0f8b113640 1 --2- 172.21.15.98:0/3572614235 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0f840cf900 0x7f0f841496b0 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:30.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.742+0000 7f0f8b113640 1 --2- 172.21.15.98:0/3572614235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0f8406c1d0 0x7f0f84158090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:30.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.742+0000 7f0f8b113640 1 -- 172.21.15.98:0/3572614235 >> 172.21.15.98:0/3572614235 conn(0x7f0f840b2010 msgr2=0x7f0f84157020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:30.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.742+0000 7f0f8b113640 1 -- 172.21.15.98:0/3572614235 shutdown_connections 2024-09-19T08:47:30.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.742+0000 7f0f8b113640 1 -- 172.21.15.98:0/3572614235 wait complete. 2024-09-19T08:47:30.756 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T08:47:30.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.882+0000 7f0233e29640 1 Processor -- start 2024-09-19T08:47:30.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.882+0000 7f0233e29640 1 -- start start 2024-09-19T08:47:30.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.882+0000 7f0233e29640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f022c145b70 0x7f022c06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:30.881 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.882+0000 7f0233e29640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f022c071ac0 0x7f022c0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:30.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.882+0000 7f0233e29640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f022c073530 con 0x7f022c06c1d0 2024-09-19T08:47:30.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.882+0000 7f0233e29640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f022c071f10 con 0x7f022c145b70 2024-09-19T08:47:30.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.882+0000 7f0233e29640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f022c06c620 con 0x7f022c071ac0 2024-09-19T08:47:30.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.882+0000 7f0231bc5640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f022c145b70 0x7f022c06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:30.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.882+0000 7f0231bc5640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f022c145b70 0x7f022c06bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39200/0 (socket says 172.21.15.98:39200) 2024-09-19T08:47:30.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.882+0000 7f0231bc5640 1 -- 172.21.15.98:0/774909842 learned_addr learned my addr 172.21.15.98:0/774909842 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:30.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.882+0000 7f0231bc5640 1 -- 172.21.15.98:0/774909842 >> v2:172.21.15.139:3300/0 conn(0x7f022c071ac0 msgr2=0x7f022c0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:30.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.882+0000 7f0231bc5640 1 --2- 172.21.15.98:0/774909842 >> v2:172.21.15.139:3300/0 conn(0x7f022c071ac0 0x7f022c0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:30.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.882+0000 7f0231bc5640 1 -- 172.21.15.98:0/774909842 >> v1:172.21.15.139:6789/0 conn(0x7f022c06c1d0 legacy=0x7f022c0713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:30.882 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.882+0000 7f0231bc5640 1 -- 172.21.15.98:0/774909842 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f022c073810 con 0x7f022c145b70 2024-09-19T08:47:30.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.882+0000 7f0231bc5640 1 --2- 172.21.15.98:0/774909842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f022c145b70 0x7f022c06bc90 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f022c074450 tx=0x7f022801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2315c41dc7a67050 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:30.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.886+0000 7f0230bc3640 1 -- 172.21.15.98:0/774909842 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02280b0070 con 0x7f022c145b70 2024-09-19T08:47:30.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.886+0000 7f0230bc3640 1 -- 172.21.15.98:0/774909842 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02280ab600 con 0x7f022c145b70 2024-09-19T08:47:30.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.886+0000 7f0230bc3640 1 -- 172.21.15.98:0/774909842 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02280ab9d0 con 0x7f022c145b70 2024-09-19T08:47:30.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.886+0000 7f0233e29640 1 -- 172.21.15.98:0/774909842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f022c145b70 msgr2=0x7f022c06bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:30.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.886+0000 7f0233e29640 1 --2- 172.21.15.98:0/774909842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f022c145b70 0x7f022c06bc90 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f022c074450 tx=0x7f022801c450 comp rx=0 tx=0).stop 2024-09-19T08:47:30.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.886+0000 7f0233e29640 1 -- 172.21.15.98:0/774909842 shutdown_connections 2024-09-19T08:47:30.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.886+0000 7f0233e29640 1 --2- 172.21.15.98:0/774909842 >> v2:172.21.15.139:3300/0 conn(0x7f022c071ac0 0x7f022c0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:30.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.886+0000 7f0233e29640 1 --2- 172.21.15.98:0/774909842 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f022c145b70 0x7f022c06bc90 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:30.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.886+0000 7f0233e29640 1 -- 172.21.15.98:0/774909842 >> 172.21.15.98:0/774909842 conn(0x7f022c0b2010 msgr2=0x7f022c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:30.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.886+0000 7f0233e29640 1 -- 172.21.15.98:0/774909842 shutdown_connections 2024-09-19T08:47:30.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.886+0000 7f0233e29640 1 -- 172.21.15.98:0/774909842 wait complete. 2024-09-19T08:47:30.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.886+0000 7f0233e29640 1 Processor -- start 2024-09-19T08:47:30.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.886+0000 7f0233e29640 1 -- start start 2024-09-19T08:47:30.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.886+0000 7f0233e29640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f022c06c1d0 0x7f022c158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:30.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.886+0000 7f0233e29640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f022c071ac0 0x7f022c0cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:30.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.886+0000 7f0233e29640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f022c1faa60 0x7f022c140ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:30.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.886+0000 7f0233e29640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f022c0b42e0 con 0x7f022c1faa60 2024-09-19T08:47:30.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.886+0000 7f0233e29640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f022c0b4160 con 0x7f022c06c1d0 2024-09-19T08:47:30.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.886+0000 7f0233e29640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f022c0b4460 con 0x7f022c071ac0 2024-09-19T08:47:30.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.886+0000 7f02323c6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f022c1faa60 0x7f022c140ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:30.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.886+0000 7f02323c6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f022c1faa60 0x7f022c140ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39204/0 (socket says 172.21.15.98:39204) 2024-09-19T08:47:30.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.886+0000 7f02323c6640 1 -- 172.21.15.98:0/151770118 learned_addr learned my addr 172.21.15.98:0/151770118 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:30.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.886+0000 7f02313c4640 1 --2- 172.21.15.98:0/151770118 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f022c071ac0 0x7f022c0cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:30.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.886+0000 7f02323c6640 1 -- 172.21.15.98:0/151770118 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f022c071ac0 msgr2=0x7f022c0cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:30.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.886+0000 7f0231bc5640 1 --2- 172.21.15.98:0/151770118 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f022c06c1d0 0x7f022c158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:30.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.886+0000 7f02323c6640 1 --2- 172.21.15.98:0/151770118 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f022c071ac0 0x7f022c0cd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:30.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.886+0000 7f02323c6640 1 -- 172.21.15.98:0/151770118 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f022c06c1d0 msgr2=0x7f022c158710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:30.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.886+0000 7f02323c6640 1 --2- 172.21.15.98:0/151770118 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f022c06c1d0 0x7f022c158710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:30.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.886+0000 7f02323c6640 1 -- 172.21.15.98:0/151770118 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f022c1fb4b0 con 0x7f022c1faa60 2024-09-19T08:47:30.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.886+0000 7f02323c6640 1 --2- 172.21.15.98:0/151770118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f022c1faa60 0x7f022c140ab0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f022400b970 tx=0x7f022400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:30.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.886+0000 7f02227fc640 1 -- 172.21.15.98:0/151770118 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0224019070 con 0x7f022c1faa60 2024-09-19T08:47:30.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.886+0000 7f02227fc640 1 -- 172.21.15.98:0/151770118 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f022400cb00 con 0x7f022c1faa60 2024-09-19T08:47:30.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.886+0000 7f0233e29640 1 -- 172.21.15.98:0/151770118 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f022c1fb790 con 0x7f022c1faa60 2024-09-19T08:47:30.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.886+0000 7f02227fc640 1 -- 172.21.15.98:0/151770118 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0224014440 con 0x7f022c1faa60 2024-09-19T08:47:30.889 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.886+0000 7f0233e29640 1 -- 172.21.15.98:0/151770118 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f022c158e70 con 0x7f022c1faa60 2024-09-19T08:47:30.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.890+0000 7f0233e29640 1 -- 172.21.15.98:0/151770118 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f022c06bc90 con 0x7f022c1faa60 2024-09-19T08:47:30.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.890+0000 7f02227fc640 1 -- 172.21.15.98:0/151770118 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f022401c070 con 0x7f022c1faa60 2024-09-19T08:47:30.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.890+0000 7f02227fc640 1 --2- 172.21.15.98:0/151770118 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0210068280 0x7f021006a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:30.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.890+0000 7f02227fc640 1 -- 172.21.15.98:0/151770118 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5737+0+0 (secure 0 0 0) 0x7f0224084050 con 0x7f022c1faa60 2024-09-19T08:47:30.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.890+0000 7f0231bc5640 1 --2- 172.21.15.98:0/151770118 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0210068280 0x7f021006a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:30.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.890+0000 7f0231bc5640 1 --2- 172.21.15.98:0/151770118 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0210068280 0x7f021006a740 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f02280ad000 tx=0x7f0228008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:30.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:30.894+0000 7f02227fc640 1 -- 172.21.15.98:0/151770118 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f0224056790 con 0x7f022c1faa60 2024-09-19T08:47:31.055 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.054+0000 7f0233e29640 1 -- 172.21.15.98:0/151770118 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f022c0af890 con 0x7f022c1faa60 2024-09-19T08:47:31.056 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.058+0000 7f02227fc640 1 -- 172.21.15.98:0/151770118 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) ==== 122+0+0 (secure 0 0 0) 0x7f0224056090 con 0x7f022c1faa60 2024-09-19T08:47:31.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.058+0000 7f0233e29640 1 -- 172.21.15.98:0/151770118 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0210068280 msgr2=0x7f021006a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:31.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.058+0000 7f0233e29640 1 --2- 172.21.15.98:0/151770118 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0210068280 0x7f021006a740 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f02280ad000 tx=0x7f0228008040 comp rx=0 tx=0).stop 2024-09-19T08:47:31.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.058+0000 7f0233e29640 1 -- 172.21.15.98:0/151770118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f022c1faa60 msgr2=0x7f022c140ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:31.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.058+0000 7f0233e29640 1 --2- 172.21.15.98:0/151770118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f022c1faa60 0x7f022c140ab0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f022400b970 tx=0x7f022400be30 comp rx=0 tx=0).stop 2024-09-19T08:47:31.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.058+0000 7f0233e29640 1 -- 172.21.15.98:0/151770118 shutdown_connections 2024-09-19T08:47:31.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.058+0000 7f0233e29640 1 --2- 172.21.15.98:0/151770118 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0210068280 0x7f021006a740 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:31.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.058+0000 7f0233e29640 1 --2- 172.21.15.98:0/151770118 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f022c1faa60 0x7f022c140ab0 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:31.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.058+0000 7f0233e29640 1 --2- 172.21.15.98:0/151770118 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f022c071ac0 0x7f022c0cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:31.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.058+0000 7f0233e29640 1 --2- 172.21.15.98:0/151770118 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f022c06c1d0 0x7f022c158710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:31.058 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.058+0000 7f0233e29640 1 -- 172.21.15.98:0/151770118 >> 172.21.15.98:0/151770118 conn(0x7f022c0b2010 msgr2=0x7f022c15fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:31.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.058+0000 7f0233e29640 1 -- 172.21.15.98:0/151770118 shutdown_connections 2024-09-19T08:47:31.059 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.058+0000 7f0233e29640 1 -- 172.21.15.98:0/151770118 wait complete. 2024-09-19T08:47:31.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:31.074 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp15790/subvol45710) 2024-09-19T08:47:31.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:31.098 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-19T08:47:31.146 INFO:teuthology.orchestra.run.smithi098.stdout:cephfs_data 2024-09-19T08:47:31.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:31.147 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/subvol_grp15790/subvol45710) 2024-09-19T08:47:31.207 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:47:31.207 DEBUG:teuthology.orchestra.run.smithi098:> sudo dd of=/home/ubuntu/cephtest/mnt.0/./volumes/subvol_grp15790/subvol45710/.meta 2024-09-19T08:47:31.267 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol45710 subvol_grp15790 2024-09-19T08:47:31.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.442+0000 7f45c9d0a640 1 Processor -- start 2024-09-19T08:47:31.441 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.442+0000 7f45c9d0a640 1 -- start start 2024-09-19T08:47:31.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.442+0000 7f45c9d0a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45c415c290 0x7f45c40cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:31.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.442+0000 7f45c9d0a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f45c40cf700 0x7f45c415ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:31.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.442+0000 7f45c9d0a640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45c41608c0 con 0x7f45c406d4c0 2024-09-19T08:47:31.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.442+0000 7f45c9d0a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f45c4160e80 con 0x7f45c40cf700 2024-09-19T08:47:31.442 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.442+0000 7f45c9d0a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f45c40cd510 con 0x7f45c415c290 2024-09-19T08:47:31.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.442+0000 7f45c37fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45c415c290 0x7f45c40cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:31.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.442+0000 7f45c2ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f45c40cf700 0x7f45c415ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:31.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.442+0000 7f45c2ffd640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f45c40cf700 0x7f45c415ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.98:55662/0 (socket says 172.21.15.98:55662) 2024-09-19T08:47:31.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.442+0000 7f45c37fe640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45c415c290 0x7f45c40cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:38898/0 (socket says 172.21.15.98:38898) 2024-09-19T08:47:31.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.442+0000 7f45c2ffd640 1 -- 172.21.15.98:0/3271715575 learned_addr learned my addr 172.21.15.98:0/3271715575 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:31.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.442+0000 7f45c3fff640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7f45c406d4c0 0x7f45c40b4bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:44290/0 (socket says 172.21.15.98:44290) 2024-09-19T08:47:31.443 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.442+0000 7f45c37fe640 1 -- 172.21.15.98:0/3271715575 >> v2:172.21.15.139:3300/0 conn(0x7f45c40cf700 msgr2=0x7f45c415ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:31.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.442+0000 7f45c37fe640 1 --2- 172.21.15.98:0/3271715575 >> v2:172.21.15.139:3300/0 conn(0x7f45c40cf700 0x7f45c415ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:31.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.442+0000 7f45c37fe640 1 -- 172.21.15.98:0/3271715575 >> v1:172.21.15.139:6789/0 conn(0x7f45c406d4c0 legacy=0x7f45c40b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:31.444 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.442+0000 7f45c37fe640 1 -- 172.21.15.98:0/3271715575 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c4160ba0 con 0x7f45c415c290 2024-09-19T08:47:31.446 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.446+0000 7f45c37fe640 1 --2- 172.21.15.98:0/3271715575 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45c415c290 0x7f45c40cd9a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f45c40cd690 tx=0x7f45b401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30e594be1d7aee60 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:31.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.446+0000 7f45c27fc640 1 -- 172.21.15.98:0/3271715575 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b40b0070 con 0x7f45c415c290 2024-09-19T08:47:31.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.446+0000 7f45c27fc640 1 -- 172.21.15.98:0/3271715575 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45b40ab600 con 0x7f45c415c290 2024-09-19T08:47:31.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.446+0000 7f45c27fc640 1 -- 172.21.15.98:0/3271715575 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b40ab9d0 con 0x7f45c415c290 2024-09-19T08:47:31.447 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.446+0000 7f45c9d0a640 1 -- 172.21.15.98:0/3271715575 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45c415c290 msgr2=0x7f45c40cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:31.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.446+0000 7f45c9d0a640 1 --2- 172.21.15.98:0/3271715575 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45c415c290 0x7f45c40cd9a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f45c40cd690 tx=0x7f45b401c4a0 comp rx=0 tx=0).stop 2024-09-19T08:47:31.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.446+0000 7f45c9d0a640 1 -- 172.21.15.98:0/3271715575 shutdown_connections 2024-09-19T08:47:31.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.446+0000 7f45c9d0a640 1 --2- 172.21.15.98:0/3271715575 >> v2:172.21.15.139:3300/0 conn(0x7f45c40cf700 0x7f45c415ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:31.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.446+0000 7f45c9d0a640 1 --2- 172.21.15.98:0/3271715575 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45c415c290 0x7f45c40cd9a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:31.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.446+0000 7f45c9d0a640 1 -- 172.21.15.98:0/3271715575 >> 172.21.15.98:0/3271715575 conn(0x7f45c406c090 msgr2=0x7f45c406c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:31.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.446+0000 7f45c9d0a640 1 -- 172.21.15.98:0/3271715575 shutdown_connections 2024-09-19T08:47:31.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.446+0000 7f45c9d0a640 1 -- 172.21.15.98:0/3271715575 wait complete. 2024-09-19T08:47:31.448 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.450+0000 7f45c9d0a640 1 Processor -- start 2024-09-19T08:47:31.449 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.450+0000 7f45c9d0a640 1 -- start start 2024-09-19T08:47:31.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.450+0000 7f45c9d0a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45c406d4c0 0x7f45c4166990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:31.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.450+0000 7f45c9d0a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45c40cf700 0x7f45c40aeff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:31.450 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.450+0000 7f45c9d0a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45c415c290 0x7f45c40acf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:31.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.450+0000 7f45c9d0a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f45c415c7f0 con 0x7f45c40cf700 2024-09-19T08:47:31.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.450+0000 7f45c9d0a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f45c415c670 con 0x7f45c406d4c0 2024-09-19T08:47:31.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.450+0000 7f45c9d0a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f45c4160490 con 0x7f45c415c290 2024-09-19T08:47:31.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.450+0000 7f45c2ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45c40cf700 0x7f45c40aeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:31.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.450+0000 7f45c2ffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45c40cf700 0x7f45c40aeff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39220/0 (socket says 172.21.15.98:39220) 2024-09-19T08:47:31.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.450+0000 7f45c2ffd640 1 -- 172.21.15.98:0/1813128112 learned_addr learned my addr 172.21.15.98:0/1813128112 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:31.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.450+0000 7f45c37fe640 1 --2- 172.21.15.98:0/1813128112 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45c406d4c0 0x7f45c4166990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:31.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.450+0000 7f45c3fff640 1 --2- 172.21.15.98:0/1813128112 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45c415c290 0x7f45c40acf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:31.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.450+0000 7f45c2ffd640 1 -- 172.21.15.98:0/1813128112 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45c415c290 msgr2=0x7f45c40acf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:31.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.450+0000 7f45c2ffd640 1 --2- 172.21.15.98:0/1813128112 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45c415c290 0x7f45c40acf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:31.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.450+0000 7f45c2ffd640 1 -- 172.21.15.98:0/1813128112 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45c406d4c0 msgr2=0x7f45c4166990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:31.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.450+0000 7f45c2ffd640 1 --2- 172.21.15.98:0/1813128112 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45c406d4c0 0x7f45c4166990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:31.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.450+0000 7f45c2ffd640 1 -- 172.21.15.98:0/1813128112 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c4155570 con 0x7f45c40cf700 2024-09-19T08:47:31.451 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.450+0000 7f45c37fe640 1 --2- 172.21.15.98:0/1813128112 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45c406d4c0 0x7f45c4166990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:47:31.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.450+0000 7f45c2ffd640 1 --2- 172.21.15.98:0/1813128112 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45c40cf700 0x7f45c40aeff0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00e8d0 tx=0x7f45ac00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:31.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.450+0000 7f45bbfff640 1 -- 172.21.15.98:0/1813128112 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac017070 con 0x7f45c40cf700 2024-09-19T08:47:31.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.450+0000 7f45bbfff640 1 -- 172.21.15.98:0/1813128112 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45ac00c940 con 0x7f45c40cf700 2024-09-19T08:47:31.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.450+0000 7f45c9d0a640 1 -- 172.21.15.98:0/1813128112 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45c4166ed0 con 0x7f45c40cf700 2024-09-19T08:47:31.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.450+0000 7f45bbfff640 1 -- 172.21.15.98:0/1813128112 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac00cd10 con 0x7f45c40cf700 2024-09-19T08:47:31.452 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.450+0000 7f45c9d0a640 1 -- 172.21.15.98:0/1813128112 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45c4158310 con 0x7f45c40cf700 2024-09-19T08:47:31.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.450+0000 7f45c9d0a640 1 -- 172.21.15.98:0/1813128112 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45c415ff50 con 0x7f45c40cf700 2024-09-19T08:47:31.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.454+0000 7f45bbfff640 1 -- 172.21.15.98:0/1813128112 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f45ac01a070 con 0x7f45c40cf700 2024-09-19T08:47:31.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.454+0000 7f45bbfff640 1 --2- 172.21.15.98:0/1813128112 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f45a4068250 0x7f45a406a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:31.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.454+0000 7f45bbfff640 1 -- 172.21.15.98:0/1813128112 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5737+0+0 (secure 0 0 0) 0x7f45ac084510 con 0x7f45c40cf700 2024-09-19T08:47:31.453 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.454+0000 7f45c37fe640 1 --2- 172.21.15.98:0/1813128112 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f45a4068250 0x7f45a406a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:31.454 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.454+0000 7f45c37fe640 1 --2- 172.21.15.98:0/1813128112 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f45a4068250 0x7f45a406a710 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f45b40ad000 tx=0x7f45b40a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:31.456 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.458+0000 7f45bbfff640 1 -- 172.21.15.98:0/1813128112 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f45c4166ed0 con 0x7f45c40cf700 2024-09-19T08:47:31.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.658+0000 7f45c9d0a640 1 -- 172.21.15.98:0/1813128112 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol45710", "group_name": "subvol_grp15790", "target": ["mon-mgr", ""]}) -- 0x7f45c40b5000 con 0x7f45a4068250 2024-09-19T08:47:31.670 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:47:31.673+0000 7ff73a69c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:47:31.670 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:47:31.673+0000 7ff73a69c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:47:31.670 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:47:31.673+0000 7ff73a69c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:47:31.670 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:47:31.673+0000 7ff73a69c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:47:31.670 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:47:31.673+0000 7ff73a69c640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:47:31.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.726+0000 7f45bbfff640 1 -- 172.21.15.98:0/1813128112 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f45c40b5000 con 0x7f45a4068250 2024-09-19T08:47:31.726 INFO:teuthology.orchestra.run.smithi098.stdout:/volumes/subvol_grp15790/subvol45710 2024-09-19T08:47:31.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.730+0000 7f45c9d0a640 1 -- 172.21.15.98:0/1813128112 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f45a4068250 msgr2=0x7f45a406a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:31.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.730+0000 7f45c9d0a640 1 --2- 172.21.15.98:0/1813128112 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f45a4068250 0x7f45a406a710 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f45b40ad000 tx=0x7f45b40a6000 comp rx=0 tx=0).stop 2024-09-19T08:47:31.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.730+0000 7f45c9d0a640 1 -- 172.21.15.98:0/1813128112 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45c40cf700 msgr2=0x7f45c40aeff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:31.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.730+0000 7f45c9d0a640 1 --2- 172.21.15.98:0/1813128112 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45c40cf700 0x7f45c40aeff0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00e8d0 tx=0x7f45ac00ed90 comp rx=0 tx=0).stop 2024-09-19T08:47:31.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.730+0000 7f45c9d0a640 1 -- 172.21.15.98:0/1813128112 shutdown_connections 2024-09-19T08:47:31.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.730+0000 7f45c9d0a640 1 --2- 172.21.15.98:0/1813128112 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f45a4068250 0x7f45a406a710 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:31.729 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.730+0000 7f45c9d0a640 1 --2- 172.21.15.98:0/1813128112 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f45c415c290 0x7f45c40acf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:31.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.730+0000 7f45c9d0a640 1 --2- 172.21.15.98:0/1813128112 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f45c40cf700 0x7f45c40aeff0 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:31.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.730+0000 7f45c9d0a640 1 --2- 172.21.15.98:0/1813128112 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45c406d4c0 0x7f45c4166990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:31.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.730+0000 7f45c9d0a640 1 -- 172.21.15.98:0/1813128112 >> 172.21.15.98:0/1813128112 conn(0x7f45c406c090 msgr2=0x7f45c415eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:31.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.730+0000 7f45c9d0a640 1 -- 172.21.15.98:0/1813128112 shutdown_connections 2024-09-19T08:47:31.730 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.730+0000 7f45c9d0a640 1 -- 172.21.15.98:0/1813128112 wait complete. 2024-09-19T08:47:31.746 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol45710 subvol_grp15790 2024-09-19T08:47:31.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.874+0000 7f970cc01640 1 Processor -- start 2024-09-19T08:47:31.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.874+0000 7f970cc01640 1 -- start start 2024-09-19T08:47:31.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.874+0000 7f970cc01640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f97080cf700 0x7f970815ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:31.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.874+0000 7f970cc01640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f970806d4c0 0x7f970806d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:31.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.874+0000 7f970cc01640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97081608c0 con 0x7f970815c290 2024-09-19T08:47:31.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.874+0000 7f970cc01640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f9708160e80 con 0x7f97080cf700 2024-09-19T08:47:31.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.874+0000 7f970cc01640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f97080cd510 con 0x7f970806d4c0 2024-09-19T08:47:31.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.874+0000 7f96fffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f97080cf700 0x7f970815ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:31.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.874+0000 7f96fffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f97080cf700 0x7f970815ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39232/0 (socket says 172.21.15.98:39232) 2024-09-19T08:47:31.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.874+0000 7f96fffff640 1 -- 172.21.15.98:0/251574804 learned_addr learned my addr 172.21.15.98:0/251574804 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:31.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.874+0000 7f9706d9d640 1 --2- 172.21.15.98:0/251574804 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f970806d4c0 0x7f970806d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:31.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.874+0000 7f96fffff640 1 -- 172.21.15.98:0/251574804 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f970806d4c0 msgr2=0x7f970806d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:31.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.874+0000 7f96fffff640 1 --2- 172.21.15.98:0/251574804 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f970806d4c0 0x7f970806d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:31.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.874+0000 7f96fffff640 1 -- 172.21.15.98:0/251574804 >> v1:172.21.15.139:6789/0 conn(0x7f970815c290 legacy=0x7f97080cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:31.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.874+0000 7f96fffff640 1 -- 172.21.15.98:0/251574804 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9708160ba0 con 0x7f97080cf700 2024-09-19T08:47:31.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.878+0000 7f96fffff640 1 --2- 172.21.15.98:0/251574804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97080cf700 0x7f970815ff50 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f96f0009870 tx=0x7f96f001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8445918c559909b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:31.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.878+0000 7f9705d9b640 1 -- 172.21.15.98:0/251574804 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f00b2070 con 0x7f97080cf700 2024-09-19T08:47:31.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.878+0000 7f9705d9b640 1 -- 172.21.15.98:0/251574804 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96f00a9070 con 0x7f97080cf700 2024-09-19T08:47:31.877 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.878+0000 7f9705d9b640 1 -- 172.21.15.98:0/251574804 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f00ad880 con 0x7f97080cf700 2024-09-19T08:47:31.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.878+0000 7f970cc01640 1 -- 172.21.15.98:0/251574804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97080cf700 msgr2=0x7f970815ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:31.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.878+0000 7f970cc01640 1 --2- 172.21.15.98:0/251574804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97080cf700 0x7f970815ff50 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f96f0009870 tx=0x7f96f001c560 comp rx=0 tx=0).stop 2024-09-19T08:47:31.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.878+0000 7f970cc01640 1 -- 172.21.15.98:0/251574804 shutdown_connections 2024-09-19T08:47:31.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.878+0000 7f970cc01640 1 --2- 172.21.15.98:0/251574804 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f970806d4c0 0x7f970806d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:31.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.878+0000 7f970cc01640 1 --2- 172.21.15.98:0/251574804 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97080cf700 0x7f970815ff50 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:31.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.878+0000 7f970cc01640 1 -- 172.21.15.98:0/251574804 >> 172.21.15.98:0/251574804 conn(0x7f970806c090 msgr2=0x7f970806c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:31.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.878+0000 7f970cc01640 1 -- 172.21.15.98:0/251574804 shutdown_connections 2024-09-19T08:47:31.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.878+0000 7f970cc01640 1 -- 172.21.15.98:0/251574804 wait complete. 2024-09-19T08:47:31.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.878+0000 7f970cc01640 1 Processor -- start 2024-09-19T08:47:31.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.878+0000 7f970cc01640 1 -- start start 2024-09-19T08:47:31.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.882+0000 7f970cc01640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f970806d4c0 0x7f9708161b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:31.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.882+0000 7f970cc01640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97080cf700 0x7f97080adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:31.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.882+0000 7f970cc01640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f970815c290 0x7f97080ae030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:31.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.882+0000 7f970cc01640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f97080b5580 con 0x7f97080cf700 2024-09-19T08:47:31.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.882+0000 7f970cc01640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f97080b5400 con 0x7f970806d4c0 2024-09-19T08:47:31.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.882+0000 7f970cc01640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f97081690c0 con 0x7f970815c290 2024-09-19T08:47:31.883 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.882+0000 7f96fffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97080cf700 0x7f97080adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:31.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.882+0000 7f96fffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97080cf700 0x7f97080adaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39248/0 (socket says 172.21.15.98:39248) 2024-09-19T08:47:31.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.882+0000 7f96fffff640 1 -- 172.21.15.98:0/3506772958 learned_addr learned my addr 172.21.15.98:0/3506772958 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:31.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.882+0000 7f9706d9d640 1 --2- 172.21.15.98:0/3506772958 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f970815c290 0x7f97080ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:31.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.882+0000 7f970659c640 1 --2- 172.21.15.98:0/3506772958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f970806d4c0 0x7f9708161b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:31.884 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.882+0000 7f96fffff640 1 -- 172.21.15.98:0/3506772958 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f970815c290 msgr2=0x7f97080ae030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:31.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.882+0000 7f96fffff640 1 --2- 172.21.15.98:0/3506772958 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f970815c290 0x7f97080ae030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:31.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.882+0000 7f96fffff640 1 -- 172.21.15.98:0/3506772958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f970806d4c0 msgr2=0x7f9708161b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:31.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.882+0000 7f96fffff640 1 --2- 172.21.15.98:0/3506772958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f970806d4c0 0x7f9708161b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:31.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.882+0000 7f96fffff640 1 -- 172.21.15.98:0/3506772958 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97080ae6d0 con 0x7f97080cf700 2024-09-19T08:47:31.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.882+0000 7f970659c640 1 --2- 172.21.15.98:0/3506772958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f970806d4c0 0x7f9708161b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:31.885 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.882+0000 7f96fffff640 1 --2- 172.21.15.98:0/3506772958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97080cf700 0x7f97080adaf0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f96f001ac10 tx=0x7f96f0019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:31.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.882+0000 7f96feffd640 1 -- 172.21.15.98:0/3506772958 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f00bb070 con 0x7f97080cf700 2024-09-19T08:47:31.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.882+0000 7f96feffd640 1 -- 172.21.15.98:0/3506772958 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96f00addb0 con 0x7f97080cf700 2024-09-19T08:47:31.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.882+0000 7f970cc01640 1 -- 172.21.15.98:0/3506772958 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97080ae9b0 con 0x7f97080cf700 2024-09-19T08:47:31.886 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.882+0000 7f96feffd640 1 -- 172.21.15.98:0/3506772958 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f00b2040 con 0x7f97080cf700 2024-09-19T08:47:31.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.882+0000 7f970cc01640 1 -- 172.21.15.98:0/3506772958 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97080aef70 con 0x7f97080cf700 2024-09-19T08:47:31.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.882+0000 7f970cc01640 1 -- 172.21.15.98:0/3506772958 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f970806d8f0 con 0x7f97080cf700 2024-09-19T08:47:31.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.882+0000 7f96feffd640 1 -- 172.21.15.98:0/3506772958 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 25) ==== 88312+0+0 (secure 0 0 0) 0x7f96f00be070 con 0x7f97080cf700 2024-09-19T08:47:31.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.882+0000 7f96feffd640 1 --2- 172.21.15.98:0/3506772958 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f96e8068250 0x7f96e806a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:31.887 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.882+0000 7f96feffd640 1 -- 172.21.15.98:0/3506772958 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5737+0+0 (secure 0 0 0) 0x7f96f011d620 con 0x7f97080cf700 2024-09-19T08:47:31.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.882+0000 7f970659c640 1 --2- 172.21.15.98:0/3506772958 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f96e8068250 0x7f96e806a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:31.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.886+0000 7f970659c640 1 --2- 172.21.15.98:0/3506772958 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f96e8068250 0x7f96e806a710 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f96f8000f70 tx=0x7f96f8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:31.888 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:31.886+0000 7f96feffd640 1 -- 172.21.15.98:0/3506772958 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f96f0016150 con 0x7f97080cf700 2024-09-19T08:47:32.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:32.062+0000 7f970cc01640 1 -- 172.21.15.98:0/3506772958 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol45710", "group_name": "subvol_grp15790", "target": ["mon-mgr", ""]}) -- 0x7f970815de50 con 0x7f96e8068250 2024-09-19T08:47:32.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:32.074+0000 7f96feffd640 1 -- 172.21.15.98:0/3506772958 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f970815de50 con 0x7f96e8068250 2024-09-19T08:47:32.073 INFO:teuthology.orchestra.run.smithi098.stdout:/volumes/subvol_grp15790/subvol45710 2024-09-19T08:47:32.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:32.074+0000 7f970cc01640 1 -- 172.21.15.98:0/3506772958 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f96e8068250 msgr2=0x7f96e806a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:32.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:32.074+0000 7f970cc01640 1 --2- 172.21.15.98:0/3506772958 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f96e8068250 0x7f96e806a710 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f96f8000f70 tx=0x7f96f8009040 comp rx=0 tx=0).stop 2024-09-19T08:47:32.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:32.074+0000 7f970cc01640 1 -- 172.21.15.98:0/3506772958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97080cf700 msgr2=0x7f97080adaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:32.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:32.074+0000 7f970cc01640 1 --2- 172.21.15.98:0/3506772958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97080cf700 0x7f97080adaf0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f96f001ac10 tx=0x7f96f0019300 comp rx=0 tx=0).stop 2024-09-19T08:47:32.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:32.074+0000 7f970cc01640 1 -- 172.21.15.98:0/3506772958 shutdown_connections 2024-09-19T08:47:32.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:32.074+0000 7f970cc01640 1 --2- 172.21.15.98:0/3506772958 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f96e8068250 0x7f96e806a710 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:32.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:32.074+0000 7f970cc01640 1 --2- 172.21.15.98:0/3506772958 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f970815c290 0x7f97080ae030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:32.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:32.074+0000 7f970cc01640 1 --2- 172.21.15.98:0/3506772958 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f97080cf700 0x7f97080adaf0 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:32.075 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:32.074+0000 7f970cc01640 1 --2- 172.21.15.98:0/3506772958 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f970806d4c0 0x7f9708161b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:32.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:32.074+0000 7f970cc01640 1 -- 172.21.15.98:0/3506772958 >> 172.21.15.98:0/3506772958 conn(0x7f970806c090 msgr2=0x7f97080b5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:32.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:32.074+0000 7f970cc01640 1 -- 172.21.15.98:0/3506772958 shutdown_connections 2024-09-19T08:47:32.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:32.074+0000 7f970cc01640 1 -- 172.21.15.98:0/3506772958 wait complete. 2024-09-19T08:47:32.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:32.091 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/9f51202e4e60eee56c9a563e6cb5b75f.meta) 2024-09-19T08:47:35.346 INFO:teuthology.orchestra.run.smithi098.stdout:[GLOBAL] 2024-09-19T08:47:35.346 INFO:teuthology.orchestra.run.smithi098.stdout:version = 1 2024-09-19T08:47:35.346 INFO:teuthology.orchestra.run.smithi098.stdout:type = subvolume 2024-09-19T08:47:35.346 INFO:teuthology.orchestra.run.smithi098.stdout:path = /volumes/subvol_grp15790/subvol45710 2024-09-19T08:47:35.346 INFO:teuthology.orchestra.run.smithi098.stdout:state = complete 2024-09-19T08:47:35.346 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:47:35.349 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol45710 subvol_grp15790 2024-09-19T08:47:35.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.486+0000 7f732e9af640 1 Processor -- start 2024-09-19T08:47:35.487 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.486+0000 7f732e9af640 1 -- start start 2024-09-19T08:47:35.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.490+0000 7f732e9af640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f732806dbd0 0x7f7328073680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:35.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.490+0000 7f732e9af640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f732815fac0 0x7f7328073bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:35.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.490+0000 7f732e9af640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f732815baf0 con 0x7f732815f6e0 2024-09-19T08:47:35.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.490+0000 7f732e9af640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f732815c8b0 con 0x7f732806dbd0 2024-09-19T08:47:35.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.490+0000 7f732e9af640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f73280742d0 con 0x7f732815fac0 2024-09-19T08:47:35.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.490+0000 7f73277fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f732806dbd0 0x7f7328073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:35.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.490+0000 7f73277fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f732806dbd0 0x7f7328073680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39254/0 (socket says 172.21.15.98:39254) 2024-09-19T08:47:35.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.490+0000 7f73277fe640 1 -- 172.21.15.98:0/3374253695 learned_addr learned my addr 172.21.15.98:0/3374253695 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:35.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.490+0000 7f732cf4c640 1 --2- 172.21.15.98:0/3374253695 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f732815fac0 0x7f7328073bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:35.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.490+0000 7f73277fe640 1 -- 172.21.15.98:0/3374253695 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f732815fac0 msgr2=0x7f7328073bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:35.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.490+0000 7f73277fe640 1 --2- 172.21.15.98:0/3374253695 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f732815fac0 0x7f7328073bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:35.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.490+0000 7f73277fe640 1 -- 172.21.15.98:0/3374253695 >> v1:172.21.15.139:6789/0 conn(0x7f732815f6e0 legacy=0x7f732806d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:35.489 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.490+0000 7f73277fe640 1 -- 172.21.15.98:0/3374253695 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732815bdd0 con 0x7f732806dbd0 2024-09-19T08:47:35.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.490+0000 7f73277fe640 1 --2- 172.21.15.98:0/3374253695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f732806dbd0 0x7f7328073680 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f731c009870 tx=0x7f731c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa8880f445aa87a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:35.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.490+0000 7f7326ffd640 1 -- 172.21.15.98:0/3374253695 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c0b0070 con 0x7f732806dbd0 2024-09-19T08:47:35.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.490+0000 7f7326ffd640 1 -- 172.21.15.98:0/3374253695 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f731c0ab600 con 0x7f732806dbd0 2024-09-19T08:47:35.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.490+0000 7f7326ffd640 1 -- 172.21.15.98:0/3374253695 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c0ab9d0 con 0x7f732806dbd0 2024-09-19T08:47:35.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.494+0000 7f732e9af640 1 -- 172.21.15.98:0/3374253695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f732806dbd0 msgr2=0x7f7328073680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:35.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.494+0000 7f732e9af640 1 --2- 172.21.15.98:0/3374253695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f732806dbd0 0x7f7328073680 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f731c009870 tx=0x7f731c01c520 comp rx=0 tx=0).stop 2024-09-19T08:47:35.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.494+0000 7f732e9af640 1 -- 172.21.15.98:0/3374253695 shutdown_connections 2024-09-19T08:47:35.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.494+0000 7f732e9af640 1 --2- 172.21.15.98:0/3374253695 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f732815fac0 0x7f7328073bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:35.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.494+0000 7f732e9af640 1 --2- 172.21.15.98:0/3374253695 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f732806dbd0 0x7f7328073680 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:35.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.494+0000 7f732e9af640 1 -- 172.21.15.98:0/3374253695 >> 172.21.15.98:0/3374253695 conn(0x7f732806c090 msgr2=0x7f732806c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:35.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.494+0000 7f732e9af640 1 -- 172.21.15.98:0/3374253695 shutdown_connections 2024-09-19T08:47:35.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.494+0000 7f732e9af640 1 -- 172.21.15.98:0/3374253695 wait complete. 2024-09-19T08:47:35.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.494+0000 7f732e9af640 1 Processor -- start 2024-09-19T08:47:35.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.494+0000 7f732e9af640 1 -- start start 2024-09-19T08:47:35.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.494+0000 7f732e9af640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f732806dbd0 0x7f7328152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:35.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.494+0000 7f732e9af640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f732815f6e0 0x7f73280b3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:35.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.494+0000 7f732e9af640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f732815fac0 0x7f73280b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:35.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.494+0000 7f732e9af640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f73280cefc0 con 0x7f732815f6e0 2024-09-19T08:47:35.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.494+0000 7f732e9af640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f73280cee40 con 0x7f732806dbd0 2024-09-19T08:47:35.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.494+0000 7f732e9af640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f73280cf140 con 0x7f732815fac0 2024-09-19T08:47:35.495 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.494+0000 7f73277fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f732815f6e0 0x7f73280b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:35.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.494+0000 7f73277fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f732815f6e0 0x7f73280b3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39264/0 (socket says 172.21.15.98:39264) 2024-09-19T08:47:35.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.494+0000 7f73277fe640 1 -- 172.21.15.98:0/619701986 learned_addr learned my addr 172.21.15.98:0/619701986 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:35.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.494+0000 7f7327fff640 1 --2- 172.21.15.98:0/619701986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f732806dbd0 0x7f7328152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:35.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.494+0000 7f732cf4c640 1 --2- 172.21.15.98:0/619701986 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f732815fac0 0x7f73280b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:35.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.494+0000 7f73277fe640 1 -- 172.21.15.98:0/619701986 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f732815fac0 msgr2=0x7f73280b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:35.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.494+0000 7f73277fe640 1 --2- 172.21.15.98:0/619701986 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f732815fac0 0x7f73280b3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:35.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.494+0000 7f73277fe640 1 -- 172.21.15.98:0/619701986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f732806dbd0 msgr2=0x7f7328152a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:35.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.494+0000 7f73277fe640 1 --2- 172.21.15.98:0/619701986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f732806dbd0 0x7f7328152a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:35.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.494+0000 7f73277fe640 1 -- 172.21.15.98:0/619701986 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73280b1600 con 0x7f732815f6e0 2024-09-19T08:47:35.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.494+0000 7f732cf4c640 1 --2- 172.21.15.98:0/619701986 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f732815fac0 0x7f73280b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:35.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.494+0000 7f73277fe640 1 --2- 172.21.15.98:0/619701986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f732815f6e0 0x7f73280b3720 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f731c0ae040 tx=0x7f731c0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:35.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.494+0000 7f7324ff9640 1 -- 172.21.15.98:0/619701986 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c0b0040 con 0x7f732815f6e0 2024-09-19T08:47:35.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.494+0000 7f7324ff9640 1 -- 172.21.15.98:0/619701986 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f731c0b9070 con 0x7f732815f6e0 2024-09-19T08:47:35.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.494+0000 7f732e9af640 1 -- 172.21.15.98:0/619701986 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7328150640 con 0x7f732815f6e0 2024-09-19T08:47:35.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.494+0000 7f7324ff9640 1 -- 172.21.15.98:0/619701986 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c0b4660 con 0x7f732815f6e0 2024-09-19T08:47:35.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.494+0000 7f732e9af640 1 -- 172.21.15.98:0/619701986 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73280b18e0 con 0x7f732815f6e0 2024-09-19T08:47:35.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.498+0000 7f732e9af640 1 -- 172.21.15.98:0/619701986 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73280cf5a0 con 0x7f732815f6e0 2024-09-19T08:47:35.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.498+0000 7f7324ff9640 1 -- 172.21.15.98:0/619701986 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 26) ==== 88363+0+0 (secure 0 0 0) 0x7f731c0b4800 con 0x7f732815f6e0 2024-09-19T08:47:35.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.498+0000 7f7324ff9640 1 --2- 172.21.15.98:0/619701986 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f73140682e0 0x7f731406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:35.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.498+0000 7f7324ff9640 1 -- 172.21.15.98:0/619701986 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5737+0+0 (secure 0 0 0) 0x7f731c11e490 con 0x7f732815f6e0 2024-09-19T08:47:35.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.498+0000 7f7327fff640 1 --2- 172.21.15.98:0/619701986 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f73140682e0 0x7f731406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:35.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.498+0000 7f7327fff640 1 --2- 172.21.15.98:0/619701986 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f73140682e0 0x7f731406a7a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f732815b520 tx=0x7f7318009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:35.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.502+0000 7f7324ff9640 1 -- 172.21.15.98:0/619701986 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f731c016110 con 0x7f732815f6e0 2024-09-19T08:47:35.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.690+0000 7f732e9af640 1 -- 172.21.15.98:0/619701986 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol45710", "group_name": "subvol_grp15790", "target": ["mon-mgr", ""]}) -- 0x7f73280d0370 con 0x7f73140682e0 2024-09-19T08:47:35.722 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.722+0000 7f7324ff9640 1 -- 172.21.15.98:0/619701986 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f73280d0370 con 0x7f73140682e0 2024-09-19T08:47:35.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.726+0000 7f732e9af640 1 -- 172.21.15.98:0/619701986 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f73140682e0 msgr2=0x7f731406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:35.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.726+0000 7f732e9af640 1 --2- 172.21.15.98:0/619701986 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f73140682e0 0x7f731406a7a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f732815b520 tx=0x7f7318009040 comp rx=0 tx=0).stop 2024-09-19T08:47:35.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.726+0000 7f732e9af640 1 -- 172.21.15.98:0/619701986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f732815f6e0 msgr2=0x7f73280b3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:35.725 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.726+0000 7f732e9af640 1 --2- 172.21.15.98:0/619701986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f732815f6e0 0x7f73280b3720 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f731c0ae040 tx=0x7f731c0192c0 comp rx=0 tx=0).stop 2024-09-19T08:47:35.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.726+0000 7f732e9af640 1 -- 172.21.15.98:0/619701986 shutdown_connections 2024-09-19T08:47:35.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.726+0000 7f732e9af640 1 --2- 172.21.15.98:0/619701986 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f73140682e0 0x7f731406a7a0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:35.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.726+0000 7f732e9af640 1 --2- 172.21.15.98:0/619701986 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f732815fac0 0x7f73280b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:35.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.726+0000 7f732e9af640 1 --2- 172.21.15.98:0/619701986 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f732815f6e0 0x7f73280b3720 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:35.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.726+0000 7f732e9af640 1 --2- 172.21.15.98:0/619701986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f732806dbd0 0x7f7328152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:35.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.726+0000 7f732e9af640 1 -- 172.21.15.98:0/619701986 >> 172.21.15.98:0/619701986 conn(0x7f732806c090 msgr2=0x7f732806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:35.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.726+0000 7f732e9af640 1 -- 172.21.15.98:0/619701986 shutdown_connections 2024-09-19T08:47:35.726 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.726+0000 7f732e9af640 1 -- 172.21.15.98:0/619701986 wait complete. 2024-09-19T08:47:35.733 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:47:35.733+0000 7ff73d6a2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:47:35.733 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:47:35.737+0000 7ff73d6a2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:47:35.733 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:47:35.737+0000 7ff73d6a2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:47:35.733 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:47:35.737+0000 7ff73d6a2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:47:35.733 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:47:35.737+0000 7ff73d6a2640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:47:35.743 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-19T08:47:35.755 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:47:35.757+0000 7ff73c6a0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:47:35.755 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:47:35.757+0000 7ff73c6a0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:47:35.755 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:47:35.757+0000 7ff73c6a0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:47:35.755 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:47:35.757+0000 7ff73c6a0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:47:35.755 INFO:tasks.ceph.mgr.x.smithi139.stderr:2024-09-19T08:47:35.757+0000 7ff73c6a0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T08:47:35.790 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T08:47:35.794 INFO:teuthology.orchestra.run.smithi098.stdout:2 2024-09-19T08:47:35.794 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T08:47:35.800 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp15790 2024-09-19T08:47:35.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.922+0000 7fab9988d640 1 Processor -- start 2024-09-19T08:47:35.922 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.922+0000 7fab9988d640 1 -- start start 2024-09-19T08:47:35.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.922+0000 7fab9988d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fab94144d10 0x7fab940722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:35.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.922+0000 7fab9988d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fab9406c3a0 0x7fab940cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:35.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.922+0000 7fab9988d640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab94072ea0 con 0x7fab94072820 2024-09-19T08:47:35.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.922+0000 7fab9988d640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fab94072c60 con 0x7fab9406c3a0 2024-09-19T08:47:35.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.922+0000 7fab9988d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fab94073ab0 con 0x7fab94144d10 2024-09-19T08:47:35.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.922+0000 7fab937fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fab9406c3a0 0x7fab940cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:35.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.922+0000 7fab927fc640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fab94072820 0x7fab9406bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:44456/0 (socket says 172.21.15.98:44456) 2024-09-19T08:47:35.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.922+0000 7fab927fc640 1 -- 172.21.15.98:0/2505196801 learned_addr learned my addr 172.21.15.98:0/2505196801 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:35.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.922+0000 7fab92ffd640 1 --2- 172.21.15.98:0/2505196801 >> v2:172.21.15.139:3300/0 conn(0x7fab94144d10 0x7fab940722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:35.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.926+0000 7fab937fe640 1 -- 172.21.15.98:0/2505196801 >> v2:172.21.15.139:3300/0 conn(0x7fab94144d10 msgr2=0x7fab940722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:35.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.926+0000 7fab937fe640 1 --2- 172.21.15.98:0/2505196801 >> v2:172.21.15.139:3300/0 conn(0x7fab94144d10 0x7fab940722e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:35.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.926+0000 7fab937fe640 1 -- 172.21.15.98:0/2505196801 >> v1:172.21.15.98:6789/0 conn(0x7fab94072820 legacy=0x7fab9406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:35.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.926+0000 7fab937fe640 1 -- 172.21.15.98:0/2505196801 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab94073180 con 0x7fab9406c3a0 2024-09-19T08:47:35.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.926+0000 7fab91ffb640 1 -- 172.21.15.98:0/2505196801 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 745086093 0 0) 0x7fab94072ea0 con 0x7fab94072820 2024-09-19T08:47:35.925 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.926+0000 7fab92ffd640 1 --2- 172.21.15.98:0/2505196801 >> v2:172.21.15.139:3300/0 conn(0x7fab94144d10 0x7fab940722e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:35.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.926+0000 7fab937fe640 1 --2- 172.21.15.98:0/2505196801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab9406c3a0 0x7fab940cd250 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fab940684a0 tx=0x7fab8c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e857a5bb03285984 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:35.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.926+0000 7fab91ffb640 1 -- 172.21.15.98:0/2505196801 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c0b2070 con 0x7fab9406c3a0 2024-09-19T08:47:35.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.926+0000 7fab91ffb640 1 -- 172.21.15.98:0/2505196801 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab8c0a58e0 con 0x7fab9406c3a0 2024-09-19T08:47:35.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.926+0000 7fab91ffb640 1 -- 172.21.15.98:0/2505196801 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c0a5cb0 con 0x7fab9406c3a0 2024-09-19T08:47:35.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.926+0000 7fab9988d640 1 -- 172.21.15.98:0/2505196801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab9406c3a0 msgr2=0x7fab940cd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:35.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.926+0000 7fab9988d640 1 --2- 172.21.15.98:0/2505196801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab9406c3a0 0x7fab940cd250 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fab940684a0 tx=0x7fab8c01c930 comp rx=0 tx=0).stop 2024-09-19T08:47:35.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.926+0000 7fab9988d640 1 -- 172.21.15.98:0/2505196801 shutdown_connections 2024-09-19T08:47:35.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.926+0000 7fab9988d640 1 --2- 172.21.15.98:0/2505196801 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab9406c3a0 0x7fab940cd250 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:35.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.926+0000 7fab9988d640 1 --2- 172.21.15.98:0/2505196801 >> v2:172.21.15.139:3300/0 conn(0x7fab94144d10 0x7fab940722e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:35.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.926+0000 7fab9988d640 1 -- 172.21.15.98:0/2505196801 >> 172.21.15.98:0/2505196801 conn(0x7fab940b2010 msgr2=0x7fab940b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:35.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.926+0000 7fab9988d640 1 -- 172.21.15.98:0/2505196801 shutdown_connections 2024-09-19T08:47:35.927 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.926+0000 7fab9988d640 1 -- 172.21.15.98:0/2505196801 wait complete. 2024-09-19T08:47:35.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.930+0000 7fab9988d640 1 Processor -- start 2024-09-19T08:47:35.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.930+0000 7fab9988d640 1 -- start start 2024-09-19T08:47:35.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.930+0000 7fab9988d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab9406c3a0 0x7fab9413c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:35.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.930+0000 7fab9988d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab94072820 0x7fab940c5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:35.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.930+0000 7fab9988d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab94144d10 0x7fab940c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:35.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.930+0000 7fab9988d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fab940ce120 con 0x7fab94144d10 2024-09-19T08:47:35.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.930+0000 7fab9988d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fab940cdfa0 con 0x7fab9406c3a0 2024-09-19T08:47:35.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.930+0000 7fab9988d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fab940ce2a0 con 0x7fab94072820 2024-09-19T08:47:35.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.930+0000 7fab937fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab94144d10 0x7fab940c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:35.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.930+0000 7fab937fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab94144d10 0x7fab940c9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39284/0 (socket says 172.21.15.98:39284) 2024-09-19T08:47:35.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.930+0000 7fab937fe640 1 -- 172.21.15.98:0/1534925909 learned_addr learned my addr 172.21.15.98:0/1534925909 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:35.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.930+0000 7fab92ffd640 1 --2- 172.21.15.98:0/1534925909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab9406c3a0 0x7fab9413c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:35.930 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.930+0000 7fab927fc640 1 --2- 172.21.15.98:0/1534925909 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab94072820 0x7fab940c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:35.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.930+0000 7fab937fe640 1 -- 172.21.15.98:0/1534925909 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab94072820 msgr2=0x7fab940c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:35.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.930+0000 7fab937fe640 1 --2- 172.21.15.98:0/1534925909 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab94072820 0x7fab940c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:35.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.930+0000 7fab937fe640 1 -- 172.21.15.98:0/1534925909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab9406c3a0 msgr2=0x7fab9413c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:35.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.930+0000 7fab937fe640 1 --2- 172.21.15.98:0/1534925909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab9406c3a0 0x7fab9413c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:35.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.930+0000 7fab937fe640 1 -- 172.21.15.98:0/1534925909 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab940c6080 con 0x7fab94144d10 2024-09-19T08:47:35.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.930+0000 7fab937fe640 1 --2- 172.21.15.98:0/1534925909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab94144d10 0x7fab940c9d40 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0b0040 tx=0x7fab8c0196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:35.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.930+0000 7fab7bfff640 1 -- 172.21.15.98:0/1534925909 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c0b2050 con 0x7fab94144d10 2024-09-19T08:47:35.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.930+0000 7fab7bfff640 1 -- 172.21.15.98:0/1534925909 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab8c0b3040 con 0x7fab94144d10 2024-09-19T08:47:35.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.930+0000 7fab9988d640 1 -- 172.21.15.98:0/1534925909 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab940c6360 con 0x7fab94144d10 2024-09-19T08:47:35.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.930+0000 7fab7bfff640 1 -- 172.21.15.98:0/1534925909 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c0ad850 con 0x7fab94144d10 2024-09-19T08:47:35.931 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.930+0000 7fab9988d640 1 -- 172.21.15.98:0/1534925909 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab940c6920 con 0x7fab94144d10 2024-09-19T08:47:35.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.930+0000 7fab9988d640 1 -- 172.21.15.98:0/1534925909 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab940cd250 con 0x7fab94144d10 2024-09-19T08:47:35.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.930+0000 7fab7bfff640 1 -- 172.21.15.98:0/1534925909 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 26) ==== 88363+0+0 (secure 0 0 0) 0x7fab8c0c0020 con 0x7fab94144d10 2024-09-19T08:47:35.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.934+0000 7fab7bfff640 1 --2- 172.21.15.98:0/1534925909 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fab800682e0 0x7fab8006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:35.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.934+0000 7fab7bfff640 1 -- 172.21.15.98:0/1534925909 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5737+0+0 (secure 0 0 0) 0x7fab8c11f200 con 0x7fab94144d10 2024-09-19T08:47:35.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.934+0000 7fab92ffd640 1 --2- 172.21.15.98:0/1534925909 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fab800682e0 0x7fab8006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:35.933 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.934+0000 7fab92ffd640 1 --2- 172.21.15.98:0/1534925909 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fab800682e0 0x7fab8006a7a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fab84006fd0 tx=0x7fab84008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:35.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:35.938+0000 7fab7bfff640 1 -- 172.21.15.98:0/1534925909 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fab8c016520 con 0x7fab94144d10 2024-09-19T08:47:36.126 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.126+0000 7fab9988d640 1 -- 172.21.15.98:0/1534925909 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp15790", "target": ["mon-mgr", ""]}) -- 0x7fab940c6c70 con 0x7fab800682e0 2024-09-19T08:47:36.137 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.138+0000 7fab7bfff640 1 -- 172.21.15.98:0/1534925909 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fab940c6c70 con 0x7fab800682e0 2024-09-19T08:47:36.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.138+0000 7fab9988d640 1 -- 172.21.15.98:0/1534925909 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fab800682e0 msgr2=0x7fab8006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:36.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.138+0000 7fab9988d640 1 --2- 172.21.15.98:0/1534925909 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fab800682e0 0x7fab8006a7a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fab84006fd0 tx=0x7fab84008040 comp rx=0 tx=0).stop 2024-09-19T08:47:36.139 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.138+0000 7fab9988d640 1 -- 172.21.15.98:0/1534925909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab94144d10 msgr2=0x7fab940c9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:36.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.138+0000 7fab9988d640 1 --2- 172.21.15.98:0/1534925909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab94144d10 0x7fab940c9d40 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0b0040 tx=0x7fab8c0196d0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.138+0000 7fab9988d640 1 -- 172.21.15.98:0/1534925909 shutdown_connections 2024-09-19T08:47:36.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.138+0000 7fab9988d640 1 --2- 172.21.15.98:0/1534925909 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fab800682e0 0x7fab8006a7a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.138+0000 7fab9988d640 1 --2- 172.21.15.98:0/1534925909 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fab94144d10 0x7fab940c9d40 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.138+0000 7fab9988d640 1 --2- 172.21.15.98:0/1534925909 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fab94072820 0x7fab940c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.138+0000 7fab9988d640 1 --2- 172.21.15.98:0/1534925909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab9406c3a0 0x7fab9413c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.138+0000 7fab9988d640 1 -- 172.21.15.98:0/1534925909 >> 172.21.15.98:0/1534925909 conn(0x7fab940b2010 msgr2=0x7fab9406d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:36.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.138+0000 7fab9988d640 1 -- 172.21.15.98:0/1534925909 shutdown_connections 2024-09-19T08:47:36.140 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.142+0000 7fab9988d640 1 -- 172.21.15.98:0/1534925909 wait complete. 2024-09-19T08:47:36.156 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T08:47:36.157 DEBUG:teuthology.orchestra.run.smithi098:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:47:36.238 DEBUG:teuthology.orchestra.run.smithi139:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T08:47:36.319 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T08:47:36.319 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:47:36.320 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:47:36.320 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:47:36.326 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T08:47:36.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.502+0000 7f625b213640 1 Processor -- start 2024-09-19T08:47:36.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.502+0000 7f625b213640 1 -- start start 2024-09-19T08:47:36.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.502+0000 7f625b213640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f62540b9970 0x7f62540b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:36.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.502+0000 7f625b213640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6254068440 con 0x7f62540d4820 2024-09-19T08:47:36.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.502+0000 7f625b213640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6254162a90 con 0x7f62540b7f80 2024-09-19T08:47:36.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.502+0000 7f625b213640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6254163050 con 0x7f62540b9970 2024-09-19T08:47:36.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.502+0000 7f6253fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f62540d4820 0x7f62541621b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:44462/0 (socket says 172.21.15.98:44462) 2024-09-19T08:47:36.504 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.502+0000 7f6253fff640 1 -- 172.21.15.98:0/184464769 learned_addr learned my addr 172.21.15.98:0/184464769 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:36.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.502+0000 7f62597b0640 1 --2- 172.21.15.98:0/184464769 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f62540b9970 0x7f62540b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:36.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.502+0000 7f62537fe640 1 -- 172.21.15.98:0/184464769 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3496687544 0 0) 0x7f6254068440 con 0x7f62540d4820 2024-09-19T08:47:36.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.502+0000 7f62537fe640 1 -- 172.21.15.98:0/184464769 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6244003530 con 0x7f62540d4820 2024-09-19T08:47:36.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.502+0000 7f62537fe640 1 -- 172.21.15.98:0/184464769 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1479218382 0 0) 0x7f6244003530 con 0x7f62540d4820 2024-09-19T08:47:36.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.502+0000 7f62537fe640 1 -- 172.21.15.98:0/184464769 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f62540b9970 msgr2=0x7f62540b9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:36.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.502+0000 7f62537fe640 1 --2- 172.21.15.98:0/184464769 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f62540b9970 0x7f62540b9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.502+0000 7f62537fe640 1 -- 172.21.15.98:0/184464769 >> v1:172.21.15.139:6789/0 conn(0x7f62540b7f80 legacy=0x7f625415f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:36.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.502+0000 7f62537fe640 1 -- 172.21.15.98:0/184464769 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6254162d70 con 0x7f62540d4820 2024-09-19T08:47:36.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.502+0000 7f62537fe640 1 -- 172.21.15.98:0/184464769 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f62400016a0 con 0x7f62540d4820 2024-09-19T08:47:36.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.502+0000 7f62537fe640 1 -- 172.21.15.98:0/184464769 >> v1:172.21.15.98:6789/0 conn(0x7f62540d4820 legacy=0x7f62541621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:36.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.506+0000 7f62537fe640 1 --2- 172.21.15.98:0/184464769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62440039f0 0x7f6244003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:36.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.506+0000 7f62537fe640 1 --2- 172.21.15.98:0/184464769 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f62540b9970 0x7f6244004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:36.505 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.506+0000 7f62537fe640 1 --2- 172.21.15.98:0/184464769 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6244004d50 0x7f62440051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:36.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.506+0000 7f62537fe640 1 -- 172.21.15.98:0/184464769 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6254163050 con 0x7f6244004d50 2024-09-19T08:47:36.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.506+0000 7f62537fe640 1 -- 172.21.15.98:0/184464769 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f624400bab0 con 0x7f62440039f0 2024-09-19T08:47:36.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.506+0000 7f62537fe640 1 -- 172.21.15.98:0/184464769 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f624400bcb0 con 0x7f62540b9970 2024-09-19T08:47:36.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.506+0000 7f62597b0640 1 --2- 172.21.15.98:0/184464769 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6244004d50 0x7f62440051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:36.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.506+0000 7f6253fff640 1 --2- 172.21.15.98:0/184464769 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f62540b9970 0x7f6244004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:36.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.506+0000 7f6258faf640 1 --2- 172.21.15.98:0/184464769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62440039f0 0x7f6244003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:36.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.506+0000 7f62597b0640 1 -- 172.21.15.98:0/184464769 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f62540b9970 msgr2=0x7f6244004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:36.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.506+0000 7f62597b0640 1 --2- 172.21.15.98:0/184464769 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f62540b9970 0x7f6244004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.506+0000 7f62597b0640 1 -- 172.21.15.98:0/184464769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62440039f0 msgr2=0x7f6244003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:36.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.506+0000 7f62597b0640 1 --2- 172.21.15.98:0/184464769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f62440039f0 0x7f6244003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.506 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.506+0000 7f62597b0640 1 -- 172.21.15.98:0/184464769 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6244003530 con 0x7f6244004d50 2024-09-19T08:47:36.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.506+0000 7f62597b0640 1 --2- 172.21.15.98:0/184464769 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6244004d50 0x7f62440051d0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f624c009f90 tx=0x7f624c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3adbba13f3bcc574 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:36.507 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.506+0000 7f62537fe640 1 -- 172.21.15.98:0/184464769 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c0b2070 con 0x7f6244004d50 2024-09-19T08:47:36.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.506+0000 7f62537fe640 1 -- 172.21.15.98:0/184464769 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f624c0a5c50 con 0x7f6244004d50 2024-09-19T08:47:36.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.506+0000 7f62537fe640 1 -- 172.21.15.98:0/184464769 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c009c10 con 0x7f6244004d50 2024-09-19T08:47:36.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.506+0000 7f625b213640 1 -- 172.21.15.98:0/184464769 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6244004d50 msgr2=0x7f62440051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:36.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.506+0000 7f625b213640 1 --2- 172.21.15.98:0/184464769 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6244004d50 0x7f62440051d0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f624c009f90 tx=0x7f624c01cc40 comp rx=0 tx=0).stop 2024-09-19T08:47:36.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.506+0000 7f62597b0640 1 -- 172.21.15.98:0/184464769 reap_dead start 2024-09-19T08:47:36.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.506+0000 7f625b213640 1 -- 172.21.15.98:0/184464769 shutdown_connections 2024-09-19T08:47:36.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.506+0000 7f625b213640 1 -- 172.21.15.98:0/184464769 >> 172.21.15.98:0/184464769 conn(0x7f6254074e40 msgr2=0x7f6254075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:36.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.506+0000 7f625b213640 1 -- 172.21.15.98:0/184464769 shutdown_connections 2024-09-19T08:47:36.508 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.506+0000 7f625b213640 1 -- 172.21.15.98:0/184464769 wait complete. 2024-09-19T08:47:36.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.510+0000 7f625b213640 1 Processor -- start 2024-09-19T08:47:36.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.510+0000 7f625b213640 1 -- start start 2024-09-19T08:47:36.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.510+0000 7f625b213640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6244004d50 0x7f6254156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:36.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.510+0000 7f625b213640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62541507d0 0x7f6254140db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:36.510 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.510+0000 7f625b213640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6254157db0 0x7f6254158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:36.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.510+0000 7f625b213640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f62540b84e0 con 0x7f62541507d0 2024-09-19T08:47:36.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.510+0000 7f625b213640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f62540b8360 con 0x7f6244004d50 2024-09-19T08:47:36.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.510+0000 7f625b213640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f62540b9de0 con 0x7f6254157db0 2024-09-19T08:47:36.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.510+0000 7f6253fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62541507d0 0x7f6254140db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:36.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.510+0000 7f6253fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62541507d0 0x7f6254140db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39300/0 (socket says 172.21.15.98:39300) 2024-09-19T08:47:36.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.510+0000 7f6253fff640 1 -- 172.21.15.98:0/3512000919 learned_addr learned my addr 172.21.15.98:0/3512000919 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:36.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.510+0000 7f62597b0640 1 --2- 172.21.15.98:0/3512000919 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6254157db0 0x7f6254158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:36.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.510+0000 7f6258faf640 1 --2- 172.21.15.98:0/3512000919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6244004d50 0x7f6254156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:36.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.510+0000 7f6253fff640 1 -- 172.21.15.98:0/3512000919 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6254157db0 msgr2=0x7f6254158230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:36.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.510+0000 7f6253fff640 1 --2- 172.21.15.98:0/3512000919 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6254157db0 0x7f6254158230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.510+0000 7f6253fff640 1 -- 172.21.15.98:0/3512000919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6244004d50 msgr2=0x7f6254156730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:36.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.510+0000 7f6253fff640 1 --2- 172.21.15.98:0/3512000919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6244004d50 0x7f6254156730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.510+0000 7f6253fff640 1 -- 172.21.15.98:0/3512000919 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6254156d10 con 0x7f62541507d0 2024-09-19T08:47:36.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.510+0000 7f62597b0640 1 --2- 172.21.15.98:0/3512000919 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6254157db0 0x7f6254158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:47:36.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.510+0000 7f6258faf640 1 --2- 172.21.15.98:0/3512000919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6244004d50 0x7f6254156730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:47:36.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.510+0000 7f6253fff640 1 --2- 172.21.15.98:0/3512000919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62541507d0 0x7f6254140db0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f624000e7f0 tx=0x7f624000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:36.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.510+0000 7f62517fa640 1 -- 172.21.15.98:0/3512000919 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624000eea0 con 0x7f62541507d0 2024-09-19T08:47:36.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.510+0000 7f62517fa640 1 -- 172.21.15.98:0/3512000919 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6240018070 con 0x7f62541507d0 2024-09-19T08:47:36.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.510+0000 7f62517fa640 1 -- 172.21.15.98:0/3512000919 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624000ca90 con 0x7f62541507d0 2024-09-19T08:47:36.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.510+0000 7f625b213640 1 -- 172.21.15.98:0/3512000919 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6254156ff0 con 0x7f62541507d0 2024-09-19T08:47:36.512 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.510+0000 7f625b213640 1 -- 172.21.15.98:0/3512000919 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62541575b0 con 0x7f62541507d0 2024-09-19T08:47:36.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.510+0000 7f625b213640 1 -- 172.21.15.98:0/3512000919 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62540ba0e0 con 0x7f62541507d0 2024-09-19T08:47:36.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.514+0000 7f62517fa640 1 -- 172.21.15.98:0/3512000919 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 26) ==== 88363+0+0 (secure 0 0 0) 0x7f624000cc30 con 0x7f62541507d0 2024-09-19T08:47:36.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.514+0000 7f62517fa640 1 --2- 172.21.15.98:0/3512000919 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f62380682e0 0x7f623806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:36.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.514+0000 7f62517fa640 1 -- 172.21.15.98:0/3512000919 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5737+0+0 (secure 0 0 0) 0x7f6240084600 con 0x7f62541507d0 2024-09-19T08:47:36.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.514+0000 7f6258faf640 1 --2- 172.21.15.98:0/3512000919 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f62380682e0 0x7f623806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:36.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.514+0000 7f6258faf640 1 --2- 172.21.15.98:0/3512000919 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f62380682e0 0x7f623806a7a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6248005630 tx=0x7f6248009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:36.518 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.518+0000 7f62517fa640 1 -- 172.21.15.98:0/3512000919 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f62400569d0 con 0x7f62541507d0 2024-09-19T08:47:36.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.694+0000 7f625b213640 1 -- 172.21.15.98:0/3512000919 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f62540d1bd0 con 0x7f62541507d0 2024-09-19T08:47:36.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.694+0000 7f62517fa640 1 -- 172.21.15.98:0/3512000919 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v170) ==== 80+0+0 (secure 0 0 0) 0x7f62400562d0 con 0x7f62541507d0 2024-09-19T08:47:36.695 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-19T08:47:36.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.698+0000 7f625b213640 1 -- 172.21.15.98:0/3512000919 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f62380682e0 msgr2=0x7f623806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:36.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.698+0000 7f625b213640 1 --2- 172.21.15.98:0/3512000919 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f62380682e0 0x7f623806a7a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6248005630 tx=0x7f6248009040 comp rx=0 tx=0).stop 2024-09-19T08:47:36.697 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.698+0000 7f625b213640 1 -- 172.21.15.98:0/3512000919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62541507d0 msgr2=0x7f6254140db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:36.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.698+0000 7f625b213640 1 --2- 172.21.15.98:0/3512000919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62541507d0 0x7f6254140db0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f624000e7f0 tx=0x7f624000ecb0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.698+0000 7f625b213640 1 -- 172.21.15.98:0/3512000919 shutdown_connections 2024-09-19T08:47:36.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.698+0000 7f625b213640 1 --2- 172.21.15.98:0/3512000919 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f62380682e0 0x7f623806a7a0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.698+0000 7f625b213640 1 --2- 172.21.15.98:0/3512000919 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f6254157db0 0x7f6254158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.698+0000 7f625b213640 1 --2- 172.21.15.98:0/3512000919 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f62541507d0 0x7f6254140db0 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.698+0000 7f625b213640 1 --2- 172.21.15.98:0/3512000919 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6244004d50 0x7f6254156730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:36.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.698+0000 7f625b213640 1 -- 172.21.15.98:0/3512000919 >> 172.21.15.98:0/3512000919 conn(0x7f6254074e40 msgr2=0x7f62540d39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:36.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.698+0000 7f625b213640 1 -- 172.21.15.98:0/3512000919 shutdown_connections 2024-09-19T08:47:36.698 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:36.698+0000 7f625b213640 1 -- 172.21.15.98:0/3512000919 wait complete. 2024-09-19T08:47:36.713 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi098.front.sepia.ceph.com... 2024-09-19T08:47:36.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:36.714 DEBUG:teuthology.orchestra.run.smithi098:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T08:47:36.724 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi098.stderr:ceph-fuse[54634]: fuse finished with error 0 and tester_r 0 2024-09-19T08:47:36.726 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:47:36.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:36.726 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T08:47:36.899 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:47:36.899 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:47:36.899 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:47:36.906 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T08:47:37.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.066+0000 7face8e1d640 1 Processor -- start 2024-09-19T08:47:37.064 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.066+0000 7face8e1d640 1 -- start start 2024-09-19T08:47:37.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.066+0000 7face8e1d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7face415c550 0x7face406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:37.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.066+0000 7face8e1d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7face406c1d0 0x7face40cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:37.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.066+0000 7face8e1d640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7face40cf900 0x7face40ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:37.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.066+0000 7face8e1d640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7face406c610 con 0x7face406c1d0 2024-09-19T08:47:37.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.066+0000 7face8e1d640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7face406d310 con 0x7face40cf900 2024-09-19T08:47:37.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.066+0000 7face8e1d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7face406d5f0 con 0x7face415c550 2024-09-19T08:47:37.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.066+0000 7facdbfff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7face406c1d0 0x7face40cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:37.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.066+0000 7facdbfff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7face406c1d0 0x7face40cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39316/0 (socket says 172.21.15.98:39316) 2024-09-19T08:47:37.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.066+0000 7facdbfff640 1 -- 172.21.15.98:0/687523298 learned_addr learned my addr 172.21.15.98:0/687523298 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:37.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.066+0000 7face259c640 1 --2- 172.21.15.98:0/687523298 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7face415c550 0x7face406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:37.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.066+0000 7face2d9d640 1 --2- 172.21.15.98:0/687523298 >> v2:172.21.15.139:3300/0 conn(0x7face40cf900 0x7face40ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:37.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.066+0000 7facdbfff640 1 -- 172.21.15.98:0/687523298 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7face415c550 msgr2=0x7face406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:37.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.066+0000 7facdbfff640 1 --2- 172.21.15.98:0/687523298 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7face415c550 0x7face406bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:37.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.066+0000 7facdbfff640 1 -- 172.21.15.98:0/687523298 >> v2:172.21.15.139:3300/0 conn(0x7face40cf900 msgr2=0x7face40ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:37.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.066+0000 7facdbfff640 1 --2- 172.21.15.98:0/687523298 >> v2:172.21.15.139:3300/0 conn(0x7face40cf900 0x7face40ce1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:37.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.066+0000 7facdbfff640 1 -- 172.21.15.98:0/687523298 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face406d8d0 con 0x7face406c1d0 2024-09-19T08:47:37.069 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.070+0000 7facdbfff640 1 --2- 172.21.15.98:0/687523298 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7face406c1d0 0x7face40cf3c0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7face4074090 tx=0x7facd401c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee739fd29838b113 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:37.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.070+0000 7face1d9b640 1 -- 172.21.15.98:0/687523298 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd40b0070 con 0x7face406c1d0 2024-09-19T08:47:37.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.070+0000 7face1d9b640 1 -- 172.21.15.98:0/687523298 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facd40ab600 con 0x7face406c1d0 2024-09-19T08:47:37.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.070+0000 7face1d9b640 1 -- 172.21.15.98:0/687523298 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd40ab9d0 con 0x7face406c1d0 2024-09-19T08:47:37.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.070+0000 7face8e1d640 1 -- 172.21.15.98:0/687523298 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7face406c1d0 msgr2=0x7face40cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:37.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.070+0000 7face8e1d640 1 --2- 172.21.15.98:0/687523298 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7face406c1d0 0x7face40cf3c0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7face4074090 tx=0x7facd401c260 comp rx=0 tx=0).stop 2024-09-19T08:47:37.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.070+0000 7face8e1d640 1 -- 172.21.15.98:0/687523298 shutdown_connections 2024-09-19T08:47:37.070 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.070+0000 7face8e1d640 1 --2- 172.21.15.98:0/687523298 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7face406c1d0 0x7face40cf3c0 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:37.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.070+0000 7face8e1d640 1 --2- 172.21.15.98:0/687523298 >> v2:172.21.15.139:3300/0 conn(0x7face40cf900 0x7face40ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:37.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.070+0000 7face8e1d640 1 --2- 172.21.15.98:0/687523298 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7face415c550 0x7face406bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:37.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.070+0000 7face8e1d640 1 -- 172.21.15.98:0/687523298 >> 172.21.15.98:0/687523298 conn(0x7face40b2010 msgr2=0x7face40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:37.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.070+0000 7face8e1d640 1 -- 172.21.15.98:0/687523298 shutdown_connections 2024-09-19T08:47:37.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.070+0000 7face8e1d640 1 -- 172.21.15.98:0/687523298 wait complete. 2024-09-19T08:47:37.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.070+0000 7face8e1d640 1 Processor -- start 2024-09-19T08:47:37.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.070+0000 7face8e1d640 1 -- start start 2024-09-19T08:47:37.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.070+0000 7face8e1d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7face406c1d0 0x7face4148e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:37.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.070+0000 7face8e1d640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7face40cf900 0x7face40ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:37.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.070+0000 7face8e1d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7face415c550 0x7face40aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:37.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.070+0000 7face8e1d640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7face40723a0 con 0x7face415c550 2024-09-19T08:47:37.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.070+0000 7face8e1d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7face415cb40 con 0x7face406c1d0 2024-09-19T08:47:37.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.070+0000 7face8e1d640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7face4072520 con 0x7face40cf900 2024-09-19T08:47:37.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.070+0000 7face2d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7face415c550 0x7face40aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:37.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.070+0000 7face2d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7face415c550 0x7face40aedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:39318/0 (socket says 172.21.15.98:39318) 2024-09-19T08:47:37.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.070+0000 7face2d9d640 1 -- 172.21.15.98:0/599175121 learned_addr learned my addr 172.21.15.98:0/599175121 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:37.078 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.070+0000 7face259c640 1 --2- 172.21.15.98:0/599175121 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7face406c1d0 0x7face4148e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:37.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.070+0000 7facdbfff640 1 --2- 172.21.15.98:0/599175121 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7face40cf900 0x7face40ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:37.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.070+0000 7face2d9d640 1 -- 172.21.15.98:0/599175121 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7face40cf900 msgr2=0x7face40ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:37.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.070+0000 7face2d9d640 1 --2- 172.21.15.98:0/599175121 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7face40cf900 0x7face40ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:37.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.070+0000 7face2d9d640 1 -- 172.21.15.98:0/599175121 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7face406c1d0 msgr2=0x7face4148e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:37.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.070+0000 7face2d9d640 1 --2- 172.21.15.98:0/599175121 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7face406c1d0 0x7face4148e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:37.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.070+0000 7face2d9d640 1 -- 172.21.15.98:0/599175121 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face40b3410 con 0x7face415c550 2024-09-19T08:47:37.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.070+0000 7facdbfff640 1 --2- 172.21.15.98:0/599175121 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7face40cf900 0x7face40ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:37.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.070+0000 7face2d9d640 1 --2- 172.21.15.98:0/599175121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7face415c550 0x7face40aedf0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7facdc007e90 tx=0x7facdc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:37.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.070+0000 7facdaffd640 1 -- 172.21.15.98:0/599175121 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc014420 con 0x7face415c550 2024-09-19T08:47:37.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.070+0000 7facdaffd640 1 -- 172.21.15.98:0/599175121 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facdc019070 con 0x7face415c550 2024-09-19T08:47:37.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.070+0000 7facdaffd640 1 -- 172.21.15.98:0/599175121 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc014820 con 0x7face415c550 2024-09-19T08:47:37.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.074+0000 7face8e1d640 1 -- 172.21.15.98:0/599175121 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7face40b36b0 con 0x7face415c550 2024-09-19T08:47:37.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.074+0000 7face8e1d640 1 -- 172.21.15.98:0/599175121 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7face40b3c70 con 0x7face415c550 2024-09-19T08:47:37.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.074+0000 7face8e1d640 1 -- 172.21.15.98:0/599175121 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7face40ce1a0 con 0x7face415c550 2024-09-19T08:47:37.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.074+0000 7facdaffd640 1 -- 172.21.15.98:0/599175121 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 26) ==== 88363+0+0 (secure 0 0 0) 0x7facdc0149c0 con 0x7face415c550 2024-09-19T08:47:37.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.074+0000 7facdaffd640 1 --2- 172.21.15.98:0/599175121 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7facc40682e0 0x7facc406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:37.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.074+0000 7facdaffd640 1 -- 172.21.15.98:0/599175121 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5737+0+0 (secure 0 0 0) 0x7facdc085020 con 0x7face415c550 2024-09-19T08:47:37.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.074+0000 7face259c640 1 --2- 172.21.15.98:0/599175121 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7facc40682e0 0x7facc406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:37.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.074+0000 7face259c640 1 --2- 172.21.15.98:0/599175121 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7facc40682e0 0x7facc406a7a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7face40af470 tx=0x7faccc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:37.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.078+0000 7facdaffd640 1 -- 172.21.15.98:0/599175121 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7facdc0573f0 con 0x7face415c550 2024-09-19T08:47:37.251 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.250+0000 7face8e1d640 1 -- 172.21.15.98:0/599175121 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7face40b0440 con 0x7face415c550 2024-09-19T08:47:37.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.254+0000 7facdaffd640 1 -- 172.21.15.98:0/599175121 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v170) ==== 80+0+0 (secure 0 0 0) 0x7facdc056cf0 con 0x7face415c550 2024-09-19T08:47:37.252 INFO:teuthology.orchestra.run.smithi098.stderr:listed 0 entries 2024-09-19T08:47:37.254 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.254+0000 7face8e1d640 1 -- 172.21.15.98:0/599175121 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7facc40682e0 msgr2=0x7facc406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:37.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.254+0000 7face8e1d640 1 --2- 172.21.15.98:0/599175121 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7facc40682e0 0x7facc406a7a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7face40af470 tx=0x7faccc008040 comp rx=0 tx=0).stop 2024-09-19T08:47:37.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.254+0000 7face8e1d640 1 -- 172.21.15.98:0/599175121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7face415c550 msgr2=0x7face40aedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:37.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.254+0000 7face8e1d640 1 --2- 172.21.15.98:0/599175121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7face415c550 0x7face40aedf0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7facdc007e90 tx=0x7facdc00e4b0 comp rx=0 tx=0).stop 2024-09-19T08:47:37.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.254+0000 7face8e1d640 1 -- 172.21.15.98:0/599175121 shutdown_connections 2024-09-19T08:47:37.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.254+0000 7face8e1d640 1 --2- 172.21.15.98:0/599175121 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7facc40682e0 0x7facc406a7a0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:37.255 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.254+0000 7face8e1d640 1 --2- 172.21.15.98:0/599175121 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7face415c550 0x7face40aedf0 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:37.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.254+0000 7face8e1d640 1 --2- 172.21.15.98:0/599175121 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7face40cf900 0x7face40ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:37.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.254+0000 7face8e1d640 1 --2- 172.21.15.98:0/599175121 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7face406c1d0 0x7face4148e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:37.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.254+0000 7face8e1d640 1 -- 172.21.15.98:0/599175121 >> 172.21.15.98:0/599175121 conn(0x7face40b2010 msgr2=0x7face4149590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:37.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.254+0000 7face8e1d640 1 -- 172.21.15.98:0/599175121 shutdown_connections 2024-09-19T08:47:37.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.254+0000 7face8e1d640 1 -- 172.21.15.98:0/599175121 wait complete. 2024-09-19T08:47:37.271 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi139.front.sepia.ceph.com... 2024-09-19T08:47:37.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:37.272 DEBUG:teuthology.orchestra.run.smithi139:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-19T08:47:37.285 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi139.stderr:ceph-fuse[37158]: fuse finished with error 0 and tester_r 0 2024-09-19T08:47:37.287 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:47:37.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:37.287 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-19T08:47:37.429 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:47:37.429 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:47:37.429 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:47:37.435 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15846 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T08:47:37.435 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:47:37.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:37.436 DEBUG:teuthology.orchestra.run.smithi098:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-19T08:47:37.485 INFO:teuthology.orchestra.run.smithi098.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-19T08:47:37.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:47:37.486 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T08:47:37.486 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:47:37.486 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:47:37.492 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T08:47:37.492 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:47:37.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:37.492 DEBUG:teuthology.orchestra.run.smithi139:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-19T08:47:37.541 INFO:teuthology.orchestra.run.smithi139.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-19T08:47:37.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:47:37.542 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:47:37.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.674+0000 7fae545ff640 1 Processor -- start 2024-09-19T08:47:37.675 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.674+0000 7fae545ff640 1 -- start start 2024-09-19T08:47:37.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.678+0000 7fae545ff640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fae4c15c550 0x7fae4c06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:37.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.678+0000 7fae545ff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae4c06c1d0 0x7fae4c0cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:37.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.678+0000 7fae545ff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fae4c0cf900 0x7fae4c0ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:37.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.678+0000 7fae545ff640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fae4c06c610 con 0x7fae4c0cf900 2024-09-19T08:47:37.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.678+0000 7fae545ff640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fae4c06d310 con 0x7fae4c15c550 2024-09-19T08:47:37.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.678+0000 7fae545ff640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fae4c06d5f0 con 0x7fae4c06c1d0 2024-09-19T08:47:37.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.678+0000 7fae52b9c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fae4c0cf900 0x7fae4c0ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:37.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.678+0000 7fae52b9c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fae4c0cf900 0x7fae4c0ce1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37170/0 (socket says 172.21.15.98:37170) 2024-09-19T08:47:37.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.678+0000 7fae52b9c640 1 -- 172.21.15.98:0/3526813730 learned_addr learned my addr 172.21.15.98:0/3526813730 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:37.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.678+0000 7fae5239b640 1 --2- 172.21.15.98:0/3526813730 >> v2:172.21.15.139:3300/0 conn(0x7fae4c15c550 0x7fae4c06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:37.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.678+0000 7fae51b9a640 1 --2- 172.21.15.98:0/3526813730 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae4c06c1d0 0x7fae4c0cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:37.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.678+0000 7fae52b9c640 1 -- 172.21.15.98:0/3526813730 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae4c06c1d0 msgr2=0x7fae4c0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:37.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.678+0000 7fae52b9c640 1 --2- 172.21.15.98:0/3526813730 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae4c06c1d0 0x7fae4c0cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:37.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.678+0000 7fae52b9c640 1 -- 172.21.15.98:0/3526813730 >> v2:172.21.15.139:3300/0 conn(0x7fae4c15c550 msgr2=0x7fae4c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:37.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.678+0000 7fae52b9c640 1 --2- 172.21.15.98:0/3526813730 >> v2:172.21.15.139:3300/0 conn(0x7fae4c15c550 0x7fae4c06bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:37.680 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.678+0000 7fae52b9c640 1 -- 172.21.15.98:0/3526813730 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae4c06d8d0 con 0x7fae4c0cf900 2024-09-19T08:47:37.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.678+0000 7fae52b9c640 1 --2- 172.21.15.98:0/3526813730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae4c0cf900 0x7fae4c0ce1a0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fae4c0b56b0 tx=0x7fae4401c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ff954aad479989 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:37.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.678+0000 7fae51399640 1 -- 172.21.15.98:0/3526813730 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae440a64d0 con 0x7fae4c0cf900 2024-09-19T08:47:37.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.678+0000 7fae51399640 1 -- 172.21.15.98:0/3526813730 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae440a6670 con 0x7fae4c0cf900 2024-09-19T08:47:37.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.678+0000 7fae51399640 1 -- 172.21.15.98:0/3526813730 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae440a6a40 con 0x7fae4c0cf900 2024-09-19T08:47:37.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.682+0000 7fae545ff640 1 -- 172.21.15.98:0/3526813730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae4c0cf900 msgr2=0x7fae4c0ce1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:37.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.682+0000 7fae545ff640 1 --2- 172.21.15.98:0/3526813730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae4c0cf900 0x7fae4c0ce1a0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fae4c0b56b0 tx=0x7fae4401c730 comp rx=0 tx=0).stop 2024-09-19T08:47:37.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.682+0000 7fae545ff640 1 -- 172.21.15.98:0/3526813730 shutdown_connections 2024-09-19T08:47:37.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.682+0000 7fae545ff640 1 --2- 172.21.15.98:0/3526813730 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae4c0cf900 0x7fae4c0ce1a0 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:37.688 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.682+0000 7fae545ff640 1 --2- 172.21.15.98:0/3526813730 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae4c06c1d0 0x7fae4c0cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:37.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.682+0000 7fae545ff640 1 --2- 172.21.15.98:0/3526813730 >> v2:172.21.15.139:3300/0 conn(0x7fae4c15c550 0x7fae4c06bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:37.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.682+0000 7fae545ff640 1 -- 172.21.15.98:0/3526813730 >> 172.21.15.98:0/3526813730 conn(0x7fae4c0b2010 msgr2=0x7fae4c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:37.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.682+0000 7fae545ff640 1 -- 172.21.15.98:0/3526813730 shutdown_connections 2024-09-19T08:47:37.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.682+0000 7fae545ff640 1 -- 172.21.15.98:0/3526813730 wait complete. 2024-09-19T08:47:37.689 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.682+0000 7fae545ff640 1 Processor -- start 2024-09-19T08:47:37.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.682+0000 7fae545ff640 1 -- start start 2024-09-19T08:47:37.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.682+0000 7fae545ff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae4c06c1d0 0x7fae4c14efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:37.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.682+0000 7fae545ff640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae4c0cf900 0x7fae4c152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:37.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.682+0000 7fae545ff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae4c15c550 0x7fae4c158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:37.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.682+0000 7fae545ff640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c0723a0 con 0x7fae4c06c1d0 2024-09-19T08:47:37.690 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.682+0000 7fae545ff640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c15cb40 con 0x7fae4c15c550 2024-09-19T08:47:37.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.682+0000 7fae545ff640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fae4c072520 con 0x7fae4c0cf900 2024-09-19T08:47:37.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.682+0000 7fae5239b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae4c06c1d0 0x7fae4c14efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:37.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.682+0000 7fae5239b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae4c06c1d0 0x7fae4c14efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37182/0 (socket says 172.21.15.98:37182) 2024-09-19T08:47:37.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.682+0000 7fae5239b640 1 -- 172.21.15.98:0/2579356289 learned_addr learned my addr 172.21.15.98:0/2579356289 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:37.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.682+0000 7fae51b9a640 1 --2- 172.21.15.98:0/2579356289 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae4c0cf900 0x7fae4c152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:37.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.682+0000 7fae52b9c640 1 --2- 172.21.15.98:0/2579356289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae4c15c550 0x7fae4c158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:37.691 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.682+0000 7fae5239b640 1 -- 172.21.15.98:0/2579356289 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae4c0cf900 msgr2=0x7fae4c152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:37.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.682+0000 7fae5239b640 1 --2- 172.21.15.98:0/2579356289 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae4c0cf900 0x7fae4c152750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:37.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.682+0000 7fae5239b640 1 -- 172.21.15.98:0/2579356289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae4c15c550 msgr2=0x7fae4c158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:37.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.682+0000 7fae5239b640 1 --2- 172.21.15.98:0/2579356289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae4c15c550 0x7fae4c158760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:37.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.682+0000 7fae5239b640 1 -- 172.21.15.98:0/2579356289 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae4c14f520 con 0x7fae4c06c1d0 2024-09-19T08:47:37.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.682+0000 7fae52b9c640 1 --2- 172.21.15.98:0/2579356289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae4c15c550 0x7fae4c158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:37.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.682+0000 7fae5239b640 1 --2- 172.21.15.98:0/2579356289 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae4c06c1d0 0x7fae4c14efe0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fae4800d5f0 tx=0x7fae4800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:37.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.682+0000 7fae42ffd640 1 -- 172.21.15.98:0/2579356289 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae480125f0 con 0x7fae4c06c1d0 2024-09-19T08:47:37.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.682+0000 7fae42ffd640 1 -- 172.21.15.98:0/2579356289 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae48019070 con 0x7fae4c06c1d0 2024-09-19T08:47:37.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.682+0000 7fae545ff640 1 -- 172.21.15.98:0/2579356289 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae4c14bc30 con 0x7fae4c06c1d0 2024-09-19T08:47:37.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.682+0000 7fae42ffd640 1 -- 172.21.15.98:0/2579356289 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae480129b0 con 0x7fae4c06c1d0 2024-09-19T08:47:37.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.682+0000 7fae545ff640 1 -- 172.21.15.98:0/2579356289 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae4c14c1b0 con 0x7fae4c06c1d0 2024-09-19T08:47:37.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.682+0000 7fae545ff640 1 -- 172.21.15.98:0/2579356289 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae4c0ce1a0 con 0x7fae4c06c1d0 2024-09-19T08:47:37.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.686+0000 7fae42ffd640 1 -- 172.21.15.98:0/2579356289 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 26) ==== 88363+0+0 (secure 0 0 0) 0x7fae48012b50 con 0x7fae4c06c1d0 2024-09-19T08:47:37.693 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.686+0000 7fae42ffd640 1 --2- 172.21.15.98:0/2579356289 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fae300682e0 0x7fae3006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:37.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.686+0000 7fae42ffd640 1 -- 172.21.15.98:0/2579356289 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5737+0+0 (secure 0 0 0) 0x7fae48084c50 con 0x7fae4c06c1d0 2024-09-19T08:47:37.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.686+0000 7fae51b9a640 1 --2- 172.21.15.98:0/2579356289 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fae300682e0 0x7fae3006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:37.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.686+0000 7fae51b9a640 1 --2- 172.21.15.98:0/2579356289 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fae300682e0 0x7fae3006a7a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fae4c071dc0 tx=0x7fae3c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:37.694 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.690+0000 7fae42ffd640 1 -- 172.21.15.98:0/2579356289 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fae4c14f520 con 0x7fae4c06c1d0 2024-09-19T08:47:37.875 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.874+0000 7fae545ff640 1 -- 172.21.15.98:0/2579356289 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae4c0757a0 con 0x7fae4c06c1d0 2024-09-19T08:47:37.876 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.878+0000 7fae42ffd640 1 -- 172.21.15.98:0/2579356289 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) ==== 95+0+5235 (secure 0 0 0) 0x7fae48056f80 con 0x7fae4c06c1d0 2024-09-19T08:47:37.876 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:47:37.876 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":101,"btime":"2024-09-19T08:47:30:391433+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6392,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/1027012777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":1027012777},{"type":"v1","addr":"172.21.15.139:6834","nonce":1027012777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7339,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3216462399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6833","nonce":3216462399},{"type":"v1","addr":"172.21.15.139:6835","nonce":3216462399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":9174,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1629579852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1629579852},{"type":"v1","addr":"172.21.15.98:6833","nonce":1629579852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:47:13.743399+0000","modified":"2024-09-19T08:47:16.695205+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9177":{"gid":9177,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/3407902663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3407902663},{"type":"v1","addr":"172.21.15.98:6835","nonce":3407902663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9177,"qdb_cluster":[9177]},"id":13}]} 2024-09-19T08:47:37.877 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 101 2024-09-19T08:47:37.878 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.878+0000 7fae545ff640 1 -- 172.21.15.98:0/2579356289 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fae300682e0 msgr2=0x7fae3006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:37.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.878+0000 7fae545ff640 1 --2- 172.21.15.98:0/2579356289 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fae300682e0 0x7fae3006a7a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fae4c071dc0 tx=0x7fae3c008040 comp rx=0 tx=0).stop 2024-09-19T08:47:37.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.878+0000 7fae545ff640 1 -- 172.21.15.98:0/2579356289 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae4c06c1d0 msgr2=0x7fae4c14efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:37.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.878+0000 7fae545ff640 1 --2- 172.21.15.98:0/2579356289 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae4c06c1d0 0x7fae4c14efe0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fae4800d5f0 tx=0x7fae4800dab0 comp rx=0 tx=0).stop 2024-09-19T08:47:37.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.878+0000 7fae545ff640 1 -- 172.21.15.98:0/2579356289 shutdown_connections 2024-09-19T08:47:37.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.878+0000 7fae545ff640 1 --2- 172.21.15.98:0/2579356289 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fae300682e0 0x7fae3006a7a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:37.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.878+0000 7fae545ff640 1 --2- 172.21.15.98:0/2579356289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fae4c15c550 0x7fae4c158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:37.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.878+0000 7fae545ff640 1 --2- 172.21.15.98:0/2579356289 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fae4c0cf900 0x7fae4c152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:37.879 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.878+0000 7fae545ff640 1 --2- 172.21.15.98:0/2579356289 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fae4c06c1d0 0x7fae4c14efe0 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:37.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.878+0000 7fae545ff640 1 -- 172.21.15.98:0/2579356289 >> 172.21.15.98:0/2579356289 conn(0x7fae4c0b2010 msgr2=0x7fae4c153a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:37.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.878+0000 7fae545ff640 1 -- 172.21.15.98:0/2579356289 shutdown_connections 2024-09-19T08:47:37.880 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:37.878+0000 7fae545ff640 1 -- 172.21.15.98:0/2579356289 wait complete. 2024-09-19T08:47:37.893 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:47:38.014 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.014+0000 7f0657243640 1 Processor -- start 2024-09-19T08:47:38.015 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.014+0000 7f0657243640 1 -- start start 2024-09-19T08:47:38.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.014+0000 7f0657243640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0650158fd0 0x7f06500b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.014+0000 7f0657243640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f06500b4360 0x7f065006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.014+0000 7f0657243640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06500ccc60 con 0x7f065006c1d0 2024-09-19T08:47:38.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.014+0000 7f0657243640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f065006c640 con 0x7f0650158fd0 2024-09-19T08:47:38.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.014+0000 7f0657243640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f06501593b0 con 0x7f06500b4360 2024-09-19T08:47:38.016 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.018+0000 7f0654fdf640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0650158fd0 0x7f06500b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.018+0000 7f06557e0640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f065006c1d0 0x7f065015efc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:58778/0 (socket says 172.21.15.98:58778) 2024-09-19T08:47:38.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.018+0000 7f06557e0640 1 -- 172.21.15.98:0/1638124303 learned_addr learned my addr 172.21.15.98:0/1638124303 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:38.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.018+0000 7f0654fdf640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0650158fd0 0x7f06500b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37198/0 (socket says 172.21.15.98:37198) 2024-09-19T08:47:38.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.018+0000 7f064ffff640 1 --2- 172.21.15.98:0/1638124303 >> v2:172.21.15.139:3300/0 conn(0x7f06500b4360 0x7f065006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.018+0000 7f0654fdf640 1 -- 172.21.15.98:0/1638124303 >> v2:172.21.15.139:3300/0 conn(0x7f06500b4360 msgr2=0x7f065006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:38.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.018+0000 7f0654fdf640 1 --2- 172.21.15.98:0/1638124303 >> v2:172.21.15.139:3300/0 conn(0x7f06500b4360 0x7f065006bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.018+0000 7f0654fdf640 1 -- 172.21.15.98:0/1638124303 >> v1:172.21.15.98:6789/0 conn(0x7f065006c1d0 legacy=0x7f065015efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:38.017 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.018+0000 7f0654fdf640 1 -- 172.21.15.98:0/1638124303 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06500ccf40 con 0x7f0650158fd0 2024-09-19T08:47:38.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.018+0000 7f0654fdf640 1 --2- 172.21.15.98:0/1638124303 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0650158fd0 0x7f06500b3e20 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f06500cdef0 tx=0x7f064401e5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67c093c309493209 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:38.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.018+0000 7f064f7fe640 1 -- 172.21.15.98:0/1638124303 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06440b2600 con 0x7f0650158fd0 2024-09-19T08:47:38.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.018+0000 7f064f7fe640 1 -- 172.21.15.98:0/1638124303 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06440b27a0 con 0x7f0650158fd0 2024-09-19T08:47:38.018 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.018+0000 7f064f7fe640 1 -- 172.21.15.98:0/1638124303 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06440b2b70 con 0x7f0650158fd0 2024-09-19T08:47:38.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.018+0000 7f0657243640 1 -- 172.21.15.98:0/1638124303 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0650158fd0 msgr2=0x7f06500b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:38.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.018+0000 7f0657243640 1 --2- 172.21.15.98:0/1638124303 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0650158fd0 0x7f06500b3e20 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f06500cdef0 tx=0x7f064401e5d0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.018+0000 7f0657243640 1 -- 172.21.15.98:0/1638124303 shutdown_connections 2024-09-19T08:47:38.019 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.018+0000 7f0657243640 1 --2- 172.21.15.98:0/1638124303 >> v2:172.21.15.139:3300/0 conn(0x7f06500b4360 0x7f065006bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.018+0000 7f0657243640 1 --2- 172.21.15.98:0/1638124303 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0650158fd0 0x7f06500b3e20 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.018+0000 7f0657243640 1 -- 172.21.15.98:0/1638124303 >> 172.21.15.98:0/1638124303 conn(0x7f06500b2010 msgr2=0x7f06500b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:38.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.018+0000 7f0657243640 1 -- 172.21.15.98:0/1638124303 shutdown_connections 2024-09-19T08:47:38.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.018+0000 7f0657243640 1 -- 172.21.15.98:0/1638124303 wait complete. 2024-09-19T08:47:38.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.022+0000 7f0657243640 1 Processor -- start 2024-09-19T08:47:38.020 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.022+0000 7f0657243640 1 -- start start 2024-09-19T08:47:38.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.022+0000 7f0657243640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f065006c1d0 0x7f0650148cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.022+0000 7f0657243640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06500b4360 0x7f0650157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.022+0000 7f0657243640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0650158fd0 0x7f06501583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.022+0000 7f0657243640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f065015c920 con 0x7f06500b4360 2024-09-19T08:47:38.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.022+0000 7f0657243640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f065015c7a0 con 0x7f0650158fd0 2024-09-19T08:47:38.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.022+0000 7f0657243640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f065006d170 con 0x7f065006c1d0 2024-09-19T08:47:38.023 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.022+0000 7f064ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06500b4360 0x7f0650157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.022+0000 7f064ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06500b4360 0x7f0650157eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37210/0 (socket says 172.21.15.98:37210) 2024-09-19T08:47:38.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.022+0000 7f064ffff640 1 -- 172.21.15.98:0/2966771755 learned_addr learned my addr 172.21.15.98:0/2966771755 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:38.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.022+0000 7f06557e0640 1 --2- 172.21.15.98:0/2966771755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0650158fd0 0x7f06501583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.022+0000 7f0654fdf640 1 --2- 172.21.15.98:0/2966771755 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f065006c1d0 0x7f0650148cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.022+0000 7f064ffff640 1 -- 172.21.15.98:0/2966771755 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f065006c1d0 msgr2=0x7f0650148cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:38.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.022+0000 7f064ffff640 1 --2- 172.21.15.98:0/2966771755 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f065006c1d0 0x7f0650148cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.022+0000 7f064ffff640 1 -- 172.21.15.98:0/2966771755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0650158fd0 msgr2=0x7f06501583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:38.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.022+0000 7f064ffff640 1 --2- 172.21.15.98:0/2966771755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0650158fd0 0x7f06501583f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.022+0000 7f064ffff640 1 -- 172.21.15.98:0/2966771755 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06501551f0 con 0x7f06500b4360 2024-09-19T08:47:38.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.022+0000 7f0654fdf640 1 --2- 172.21.15.98:0/2966771755 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f065006c1d0 0x7f0650148cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:38.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.022+0000 7f06557e0640 1 --2- 172.21.15.98:0/2966771755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0650158fd0 0x7f06501583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:38.024 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.022+0000 7f064ffff640 1 --2- 172.21.15.98:0/2966771755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06500b4360 0x7f0650157eb0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f063c00d800 tx=0x7f063c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:38.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.022+0000 7f064d7fa640 1 -- 172.21.15.98:0/2966771755 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c00b6c0 con 0x7f06500b4360 2024-09-19T08:47:38.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.022+0000 7f064d7fa640 1 -- 172.21.15.98:0/2966771755 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f063c019070 con 0x7f06500b4360 2024-09-19T08:47:38.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.022+0000 7f0657243640 1 -- 172.21.15.98:0/2966771755 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06501512b0 con 0x7f06500b4360 2024-09-19T08:47:38.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.022+0000 7f064d7fa640 1 -- 172.21.15.98:0/2966771755 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c00ba60 con 0x7f06500b4360 2024-09-19T08:47:38.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.022+0000 7f0657243640 1 -- 172.21.15.98:0/2966771755 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0650151870 con 0x7f06500b4360 2024-09-19T08:47:38.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.022+0000 7f0657243640 1 -- 172.21.15.98:0/2966771755 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06501577a0 con 0x7f06500b4360 2024-09-19T08:47:38.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.022+0000 7f064d7fa640 1 -- 172.21.15.98:0/2966771755 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 26) ==== 88363+0+0 (secure 0 0 0) 0x7f063c00bc00 con 0x7f06500b4360 2024-09-19T08:47:38.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.026+0000 7f064d7fa640 1 --2- 172.21.15.98:0/2966771755 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f06340682e0 0x7f063406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.026+0000 7f064d7fa640 1 -- 172.21.15.98:0/2966771755 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5737+0+0 (secure 0 0 0) 0x7f063c084c00 con 0x7f06500b4360 2024-09-19T08:47:38.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.026+0000 7f0654fdf640 1 --2- 172.21.15.98:0/2966771755 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f06340682e0 0x7f063406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.026 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.026+0000 7f0654fdf640 1 --2- 172.21.15.98:0/2966771755 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f06340682e0 0x7f063406a7a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f06440b1000 tx=0x7f06440a9000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:38.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.030+0000 7f064d7fa640 1 -- 172.21.15.98:0/2966771755 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f063c056fd0 con 0x7f06500b4360 2024-09-19T08:47:38.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.210+0000 7f0657243640 1 -- 172.21.15.98:0/2966771755 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f065006bc90 con 0x7f06500b4360 2024-09-19T08:47:38.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.210+0000 7f064d7fa640 1 -- 172.21.15.98:0/2966771755 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) ==== 95+0+5235 (secure 0 0 0) 0x7f063c0568d0 con 0x7f06500b4360 2024-09-19T08:47:38.210 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:47:38.210 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":101,"btime":"2024-09-19T08:47:30:391433+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6392,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/1027012777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":1027012777},{"type":"v1","addr":"172.21.15.139:6834","nonce":1027012777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7339,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3216462399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6833","nonce":3216462399},{"type":"v1","addr":"172.21.15.139:6835","nonce":3216462399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":9174,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1629579852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1629579852},{"type":"v1","addr":"172.21.15.98:6833","nonce":1629579852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:47:13.743399+0000","modified":"2024-09-19T08:47:16.695205+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9177":{"gid":9177,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/3407902663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3407902663},{"type":"v1","addr":"172.21.15.98:6835","nonce":3407902663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9177,"qdb_cluster":[9177]},"id":13}]} 2024-09-19T08:47:38.210 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 101 2024-09-19T08:47:38.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.210+0000 7f0657243640 1 -- 172.21.15.98:0/2966771755 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f06340682e0 msgr2=0x7f063406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:38.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.210+0000 7f0657243640 1 --2- 172.21.15.98:0/2966771755 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f06340682e0 0x7f063406a7a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f06440b1000 tx=0x7f06440a9000 comp rx=0 tx=0).stop 2024-09-19T08:47:38.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.210+0000 7f0657243640 1 -- 172.21.15.98:0/2966771755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06500b4360 msgr2=0x7f0650157eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:38.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.210+0000 7f0657243640 1 --2- 172.21.15.98:0/2966771755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06500b4360 0x7f0650157eb0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f063c00d800 tx=0x7f063c00dcc0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.212 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.210+0000 7f0657243640 1 -- 172.21.15.98:0/2966771755 shutdown_connections 2024-09-19T08:47:38.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.210+0000 7f0657243640 1 --2- 172.21.15.98:0/2966771755 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f06340682e0 0x7f063406a7a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.210+0000 7f0657243640 1 --2- 172.21.15.98:0/2966771755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0650158fd0 0x7f06501583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.210+0000 7f0657243640 1 --2- 172.21.15.98:0/2966771755 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f06500b4360 0x7f0650157eb0 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.214+0000 7f0657243640 1 --2- 172.21.15.98:0/2966771755 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f065006c1d0 0x7f0650148cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.214+0000 7f0657243640 1 -- 172.21.15.98:0/2966771755 >> 172.21.15.98:0/2966771755 conn(0x7f06500b2010 msgr2=0x7f0650159530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:38.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.214+0000 7f0657243640 1 -- 172.21.15.98:0/2966771755 shutdown_connections 2024-09-19T08:47:38.213 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.214+0000 7f0657243640 1 -- 172.21.15.98:0/2966771755 wait complete. 2024-09-19T08:47:38.227 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:47:38.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.350+0000 7f7416ee8640 1 Processor -- start 2024-09-19T08:47:38.349 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.350+0000 7f7416ee8640 1 -- start start 2024-09-19T08:47:38.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.350+0000 7f7416ee8640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f74100cea90 0x7f7410067a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.350+0000 7f7416ee8640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f74100d0230 0x7f74100cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.350+0000 7f7416ee8640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f741015c710 con 0x7f74100cf3b0 2024-09-19T08:47:38.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.350+0000 7f7416ee8640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f7410068090 con 0x7f74100cea90 2024-09-19T08:47:38.350 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.350+0000 7f7416ee8640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f741006e1a0 con 0x7f74100d0230 2024-09-19T08:47:38.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.350+0000 7f7414c84640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f74100cea90 0x7f7410067a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.350+0000 7f7414c84640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f74100cea90 0x7f7410067a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37216/0 (socket says 172.21.15.98:37216) 2024-09-19T08:47:38.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.350+0000 7f7414c84640 1 -- 172.21.15.98:0/2946043524 learned_addr learned my addr 172.21.15.98:0/2946043524 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:38.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.350+0000 7f740ffff640 1 --2- 172.21.15.98:0/2946043524 >> v2:172.21.15.139:3300/0 conn(0x7f74100d0230 0x7f74100cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.350+0000 7f7414c84640 1 -- 172.21.15.98:0/2946043524 >> v2:172.21.15.139:3300/0 conn(0x7f74100d0230 msgr2=0x7f74100cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:38.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.350+0000 7f7414c84640 1 --2- 172.21.15.98:0/2946043524 >> v2:172.21.15.139:3300/0 conn(0x7f74100d0230 0x7f74100cee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.350+0000 7f7414c84640 1 -- 172.21.15.98:0/2946043524 >> v1:172.21.15.139:6789/0 conn(0x7f74100cf3b0 legacy=0x7f7410071590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:38.351 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.350+0000 7f7414c84640 1 -- 172.21.15.98:0/2946043524 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f741006e440 con 0x7f74100cea90 2024-09-19T08:47:38.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.354+0000 7f7414c84640 1 --2- 172.21.15.98:0/2946043524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f74100cea90 0x7f7410067a00 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f7410073e40 tx=0x7f740401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e7941fa1bd3543 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:38.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.354+0000 7f740f7fe640 1 -- 172.21.15.98:0/2946043524 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74040aa650 con 0x7f74100cea90 2024-09-19T08:47:38.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.354+0000 7f740f7fe640 1 -- 172.21.15.98:0/2946043524 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74040af070 con 0x7f74100cea90 2024-09-19T08:47:38.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.354+0000 7f740f7fe640 1 -- 172.21.15.98:0/2946043524 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74040aaa10 con 0x7f74100cea90 2024-09-19T08:47:38.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.354+0000 7f7416ee8640 1 -- 172.21.15.98:0/2946043524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f74100cea90 msgr2=0x7f7410067a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:38.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.354+0000 7f7416ee8640 1 --2- 172.21.15.98:0/2946043524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f74100cea90 0x7f7410067a00 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f7410073e40 tx=0x7f740401c4f0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.354+0000 7f7416ee8640 1 -- 172.21.15.98:0/2946043524 shutdown_connections 2024-09-19T08:47:38.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.354+0000 7f7416ee8640 1 --2- 172.21.15.98:0/2946043524 >> v2:172.21.15.139:3300/0 conn(0x7f74100d0230 0x7f74100cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.354+0000 7f7416ee8640 1 --2- 172.21.15.98:0/2946043524 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f74100cea90 0x7f7410067a00 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.354+0000 7f7416ee8640 1 -- 172.21.15.98:0/2946043524 >> 172.21.15.98:0/2946043524 conn(0x7f74100b26a0 msgr2=0x7f74100b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:38.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.354+0000 7f7416ee8640 1 -- 172.21.15.98:0/2946043524 shutdown_connections 2024-09-19T08:47:38.354 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.354+0000 7f7416ee8640 1 -- 172.21.15.98:0/2946043524 wait complete. 2024-09-19T08:47:38.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.354+0000 7f7416ee8640 1 Processor -- start 2024-09-19T08:47:38.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.354+0000 7f7416ee8640 1 -- start start 2024-09-19T08:47:38.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.354+0000 7f7416ee8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f74100cf3b0 0x7f74100aa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.354+0000 7f7416ee8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74100d0230 0x7f7410158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.354+0000 7f7416ee8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f741014e370 0x7f7410155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.354+0000 7f7416ee8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7410168eb0 con 0x7f74100cf3b0 2024-09-19T08:47:38.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.354+0000 7f7416ee8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f741006ef20 con 0x7f741014e370 2024-09-19T08:47:38.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.354+0000 7f7416ee8640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7410169030 con 0x7f74100d0230 2024-09-19T08:47:38.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.354+0000 7f7414c84640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f74100cf3b0 0x7f74100aa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.358+0000 7f7414c84640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f74100cf3b0 0x7f74100aa4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37220/0 (socket says 172.21.15.98:37220) 2024-09-19T08:47:38.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.358+0000 7f7414c84640 1 -- 172.21.15.98:0/3428581959 learned_addr learned my addr 172.21.15.98:0/3428581959 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:38.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.358+0000 7f740ffff640 1 --2- 172.21.15.98:0/3428581959 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74100d0230 0x7f7410158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.358+0000 7f7415485640 1 --2- 172.21.15.98:0/3428581959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f741014e370 0x7f7410155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.358+0000 7f7414c84640 1 -- 172.21.15.98:0/3428581959 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74100d0230 msgr2=0x7f7410158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:38.357 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.358+0000 7f7414c84640 1 --2- 172.21.15.98:0/3428581959 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74100d0230 0x7f7410158370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.358+0000 7f7414c84640 1 -- 172.21.15.98:0/3428581959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f741014e370 msgr2=0x7f7410155d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:38.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.358+0000 7f7414c84640 1 --2- 172.21.15.98:0/3428581959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f741014e370 0x7f7410155d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.358+0000 7f7414c84640 1 -- 172.21.15.98:0/3428581959 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7410153fb0 con 0x7f74100cf3b0 2024-09-19T08:47:38.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.358+0000 7f7415485640 1 --2- 172.21.15.98:0/3428581959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f741014e370 0x7f7410155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:38.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.358+0000 7f7414c84640 1 --2- 172.21.15.98:0/3428581959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f74100cf3b0 0x7f74100aa4a0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f740401c9d0 tx=0x7f740401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:38.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.358+0000 7f740d7fa640 1 -- 172.21.15.98:0/3428581959 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74040bd5f0 con 0x7f74100cf3b0 2024-09-19T08:47:38.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.358+0000 7f740d7fa640 1 -- 172.21.15.98:0/3428581959 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74040a6040 con 0x7f74100cf3b0 2024-09-19T08:47:38.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.358+0000 7f7416ee8640 1 -- 172.21.15.98:0/3428581959 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7410154290 con 0x7f74100cf3b0 2024-09-19T08:47:38.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.358+0000 7f740d7fa640 1 -- 172.21.15.98:0/3428581959 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74040bd9d0 con 0x7f74100cf3b0 2024-09-19T08:47:38.358 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.358+0000 7f7416ee8640 1 -- 172.21.15.98:0/3428581959 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7410154850 con 0x7f74100cf3b0 2024-09-19T08:47:38.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.358+0000 7f7416ee8640 1 -- 172.21.15.98:0/3428581959 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7410067a00 con 0x7f74100cf3b0 2024-09-19T08:47:38.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.358+0000 7f740d7fa640 1 -- 172.21.15.98:0/3428581959 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 26) ==== 88363+0+0 (secure 0 0 0) 0x7f74040b33f0 con 0x7f74100cf3b0 2024-09-19T08:47:38.359 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.358+0000 7f740d7fa640 1 --2- 172.21.15.98:0/3428581959 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f73f40682e0 0x7f73f406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.358+0000 7f740d7fa640 1 -- 172.21.15.98:0/3428581959 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5737+0+0 (secure 0 0 0) 0x7f740411f2b0 con 0x7f74100cf3b0 2024-09-19T08:47:38.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.358+0000 7f740ffff640 1 --2- 172.21.15.98:0/3428581959 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f73f40682e0 0x7f73f406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.360 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.358+0000 7f740ffff640 1 --2- 172.21.15.98:0/3428581959 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f73f40682e0 0x7f73f406a7a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7410166e30 tx=0x7f73fc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:38.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.362+0000 7f740d7fa640 1 -- 172.21.15.98:0/3428581959 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f74040c3060 con 0x7f74100cf3b0 2024-09-19T08:47:38.481 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.482+0000 7f740d7fa640 1 -- 172.21.15.98:0/3428581959 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f74040ed810 con 0x7f74100cf3b0 2024-09-19T08:47:38.532 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.534+0000 7f7416ee8640 1 -- 172.21.15.98:0/3428581959 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f74100b1300 con 0x7f74100cf3b0 2024-09-19T08:47:38.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.534+0000 7f740d7fa640 1 -- 172.21.15.98:0/3428581959 <== mon.0 v2:172.21.15.98:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) ==== 74+0+16787 (secure 0 0 0) 0x7f7404016110 con 0x7f74100cf3b0 2024-09-19T08:47:38.535 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:47:38.535 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":170,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:47:15.308260+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:47:12.300410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-19T08:47:12.721062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:47:38.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.538+0000 7f7416ee8640 1 -- 172.21.15.98:0/3428581959 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f73f40682e0 msgr2=0x7f73f406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:38.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.538+0000 7f7416ee8640 1 --2- 172.21.15.98:0/3428581959 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f73f40682e0 0x7f73f406a7a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7410166e30 tx=0x7f73fc008040 comp rx=0 tx=0).stop 2024-09-19T08:47:38.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.538+0000 7f7416ee8640 1 -- 172.21.15.98:0/3428581959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f74100cf3b0 msgr2=0x7f74100aa4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:38.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.538+0000 7f7416ee8640 1 --2- 172.21.15.98:0/3428581959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f74100cf3b0 0x7f74100aa4a0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f740401c9d0 tx=0x7f740401ad30 comp rx=0 tx=0).stop 2024-09-19T08:47:38.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.538+0000 7f7416ee8640 1 -- 172.21.15.98:0/3428581959 shutdown_connections 2024-09-19T08:47:38.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.538+0000 7f7416ee8640 1 --2- 172.21.15.98:0/3428581959 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f73f40682e0 0x7f73f406a7a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.538+0000 7f7416ee8640 1 --2- 172.21.15.98:0/3428581959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f741014e370 0x7f7410155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.538+0000 7f7416ee8640 1 --2- 172.21.15.98:0/3428581959 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f74100d0230 0x7f7410158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.538+0000 7f7416ee8640 1 --2- 172.21.15.98:0/3428581959 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f74100cf3b0 0x7f74100aa4a0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.538+0000 7f7416ee8640 1 -- 172.21.15.98:0/3428581959 >> 172.21.15.98:0/3428581959 conn(0x7f74100b26a0 msgr2=0x7f74100727c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:38.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.538+0000 7f7416ee8640 1 -- 172.21.15.98:0/3428581959 shutdown_connections 2024-09-19T08:47:38.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.538+0000 7f7416ee8640 1 -- 172.21.15.98:0/3428581959 wait complete. 2024-09-19T08:47:38.552 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T08:47:38.552 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:47:38.659 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.658+0000 7f1138a9b640 1 Processor -- start 2024-09-19T08:47:38.660 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.658+0000 7f1138a9b640 1 -- start start 2024-09-19T08:47:38.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.662+0000 7f1138a9b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f11340b9970 0x7f11340b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.662+0000 7f1138a9b640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1134068440 con 0x7f11340d4820 2024-09-19T08:47:38.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.662+0000 7f1138a9b640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1134162a90 con 0x7f11340b7f80 2024-09-19T08:47:38.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.662+0000 7f1138a9b640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f1134163050 con 0x7f11340b9970 2024-09-19T08:47:38.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.662+0000 7f1131d9b640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f11340d4820 0x7f11341621b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:58784/0 (socket says 172.21.15.98:58784) 2024-09-19T08:47:38.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.662+0000 7f1132d9d640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f11340b9970 0x7f11340b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.662+0000 7f1131d9b640 1 -- 172.21.15.98:0/3905673066 learned_addr learned my addr 172.21.15.98:0/3905673066 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:38.661 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.662+0000 7f113159a640 1 -- 172.21.15.98:0/3905673066 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1127032321 0 0) 0x7f1134068440 con 0x7f11340d4820 2024-09-19T08:47:38.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.662+0000 7f113159a640 1 -- 172.21.15.98:0/3905673066 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f111c003530 con 0x7f11340d4820 2024-09-19T08:47:38.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.662+0000 7f1132d9d640 1 -- 172.21.15.98:0/3905673066 >> v1:172.21.15.139:6789/0 conn(0x7f11340b7f80 legacy=0x7f113415f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:38.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.662+0000 7f1132d9d640 1 -- 172.21.15.98:0/3905673066 >> v1:172.21.15.98:6789/0 conn(0x7f11340d4820 legacy=0x7f11341621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:38.662 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.662+0000 7f1132d9d640 1 -- 172.21.15.98:0/3905673066 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1134162d70 con 0x7f11340b9970 2024-09-19T08:47:38.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.662+0000 7f1132d9d640 1 --2- 172.21.15.98:0/3905673066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11340b9970 0x7f11340b9de0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f112c009d90 tx=0x7f112c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca27918d2eee999b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:38.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.662+0000 7f113159a640 1 -- 172.21.15.98:0/3905673066 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112c0b2070 con 0x7f11340b9970 2024-09-19T08:47:38.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.662+0000 7f113159a640 1 -- 172.21.15.98:0/3905673066 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f112c0a57f0 con 0x7f11340b9970 2024-09-19T08:47:38.663 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.662+0000 7f113159a640 1 -- 172.21.15.98:0/3905673066 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112c0a5ba0 con 0x7f11340b9970 2024-09-19T08:47:38.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.666+0000 7f1138a9b640 1 -- 172.21.15.98:0/3905673066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11340b9970 msgr2=0x7f11340b9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:38.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.666+0000 7f1138a9b640 1 --2- 172.21.15.98:0/3905673066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11340b9970 0x7f11340b9de0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f112c009d90 tx=0x7f112c01c930 comp rx=0 tx=0).stop 2024-09-19T08:47:38.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.666+0000 7f1138a9b640 1 -- 172.21.15.98:0/3905673066 shutdown_connections 2024-09-19T08:47:38.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.666+0000 7f1138a9b640 1 --2- 172.21.15.98:0/3905673066 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11340b9970 0x7f11340b9de0 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.664 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.666+0000 7f1138a9b640 1 -- 172.21.15.98:0/3905673066 >> 172.21.15.98:0/3905673066 conn(0x7f1134074e40 msgr2=0x7f1134075250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:38.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.666+0000 7f1138a9b640 1 -- 172.21.15.98:0/3905673066 shutdown_connections 2024-09-19T08:47:38.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.666+0000 7f1138a9b640 1 -- 172.21.15.98:0/3905673066 wait complete. 2024-09-19T08:47:38.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.666+0000 7f1138a9b640 1 Processor -- start 2024-09-19T08:47:38.665 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.666+0000 7f1138a9b640 1 -- start start 2024-09-19T08:47:38.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.666+0000 7f1138a9b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11340b7f80 0x7f11341565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.666+0000 7f1138a9b640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f11340b9970 0x7f1134158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.666+0000 7f1138a9b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11340d4820 0x7f113413d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.666+0000 7f1138a9b640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f1134144710 con 0x7f11340d4820 2024-09-19T08:47:38.667 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.666+0000 7f1138a9b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1134144590 con 0x7f11340b7f80 2024-09-19T08:47:38.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.666+0000 7f1138a9b640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f1134144890 con 0x7f11340b9970 2024-09-19T08:47:38.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.666+0000 7f1132d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11340d4820 0x7f113413d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.666+0000 7f1132d9d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11340d4820 0x7f113413d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37230/0 (socket says 172.21.15.98:37230) 2024-09-19T08:47:38.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.666+0000 7f1132d9d640 1 -- 172.21.15.98:0/2960415088 learned_addr learned my addr 172.21.15.98:0/2960415088 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:38.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.666+0000 7f1131d9b640 1 --2- 172.21.15.98:0/2960415088 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f11340b9970 0x7f1134158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.666+0000 7f113259c640 1 --2- 172.21.15.98:0/2960415088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11340b7f80 0x7f11341565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.666+0000 7f1132d9d640 1 -- 172.21.15.98:0/2960415088 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f11340b9970 msgr2=0x7f1134158830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:38.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.666+0000 7f1132d9d640 1 --2- 172.21.15.98:0/2960415088 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f11340b9970 0x7f1134158830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.666+0000 7f1132d9d640 1 -- 172.21.15.98:0/2960415088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11340b7f80 msgr2=0x7f11341565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:38.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.666+0000 7f1132d9d640 1 --2- 172.21.15.98:0/2960415088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11340b7f80 0x7f11341565c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.666+0000 7f1132d9d640 1 -- 172.21.15.98:0/2960415088 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1134156ea0 con 0x7f11340d4820 2024-09-19T08:47:38.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.666+0000 7f1132d9d640 1 --2- 172.21.15.98:0/2960415088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11340d4820 0x7f113413d520 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f112c019540 tx=0x7f112c0196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:38.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.666+0000 7f1122ffd640 1 -- 172.21.15.98:0/2960415088 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112c0b2050 con 0x7f11340d4820 2024-09-19T08:47:38.668 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.666+0000 7f1122ffd640 1 -- 172.21.15.98:0/2960415088 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f112c0be070 con 0x7f11340d4820 2024-09-19T08:47:38.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.666+0000 7f1138a9b640 1 -- 172.21.15.98:0/2960415088 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11340b3d70 con 0x7f11340d4820 2024-09-19T08:47:38.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.666+0000 7f1122ffd640 1 -- 172.21.15.98:0/2960415088 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112c003db0 con 0x7f11340d4820 2024-09-19T08:47:38.669 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.666+0000 7f1138a9b640 1 -- 172.21.15.98:0/2960415088 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11340b4330 con 0x7f11340d4820 2024-09-19T08:47:38.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.670+0000 7f1138a9b640 1 -- 172.21.15.98:0/2960415088 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11340b9de0 con 0x7f11340d4820 2024-09-19T08:47:38.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.670+0000 7f1122ffd640 1 -- 172.21.15.98:0/2960415088 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f112c004040 con 0x7f11340d4820 2024-09-19T08:47:38.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.670+0000 7f1122ffd640 1 --2- 172.21.15.98:0/2960415088 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1114068400 0x7f111406a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.670+0000 7f1122ffd640 1 -- 172.21.15.98:0/2960415088 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5737+0+0 (secure 0 0 0) 0x7f112c11f600 con 0x7f11340d4820 2024-09-19T08:47:38.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.670+0000 7f113259c640 1 --2- 172.21.15.98:0/2960415088 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1114068400 0x7f111406a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.670 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.670+0000 7f113259c640 1 --2- 172.21.15.98:0/2960415088 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1114068400 0x7f111406a8c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f112400a750 tx=0x7f1124008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:38.673 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.674+0000 7f1122ffd640 1 -- 172.21.15.98:0/2960415088 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f112c127020 con 0x7f11340d4820 2024-09-19T08:47:38.853 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.854+0000 7f1138a9b640 1 -- 172.21.15.98:0/2960415088 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11340ce8e0 con 0x7f11340d4820 2024-09-19T08:47:38.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.854+0000 7f1122ffd640 1 -- 172.21.15.98:0/2960415088 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) ==== 95+0+5235 (secure 0 0 0) 0x7f112c016520 con 0x7f11340d4820 2024-09-19T08:47:38.854 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:47:38.855 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":101,"btime":"2024-09-19T08:47:30:391433+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6392,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/1027012777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":1027012777},{"type":"v1","addr":"172.21.15.139:6834","nonce":1027012777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7339,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3216462399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6833","nonce":3216462399},{"type":"v1","addr":"172.21.15.139:6835","nonce":3216462399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":9174,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1629579852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1629579852},{"type":"v1","addr":"172.21.15.98:6833","nonce":1629579852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:47:13.743399+0000","modified":"2024-09-19T08:47:16.695205+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9177":{"gid":9177,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/3407902663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3407902663},{"type":"v1","addr":"172.21.15.98:6835","nonce":3407902663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9177,"qdb_cluster":[9177]},"id":13}]} 2024-09-19T08:47:38.855 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 101 2024-09-19T08:47:38.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.858+0000 7f1138a9b640 1 -- 172.21.15.98:0/2960415088 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1114068400 msgr2=0x7f111406a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:38.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.858+0000 7f1138a9b640 1 --2- 172.21.15.98:0/2960415088 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1114068400 0x7f111406a8c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f112400a750 tx=0x7f1124008040 comp rx=0 tx=0).stop 2024-09-19T08:47:38.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.858+0000 7f1138a9b640 1 -- 172.21.15.98:0/2960415088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11340d4820 msgr2=0x7f113413d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:38.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.858+0000 7f1138a9b640 1 --2- 172.21.15.98:0/2960415088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11340d4820 0x7f113413d520 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f112c019540 tx=0x7f112c0196d0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.858+0000 7f1138a9b640 1 -- 172.21.15.98:0/2960415088 shutdown_connections 2024-09-19T08:47:38.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.858+0000 7f1138a9b640 1 --2- 172.21.15.98:0/2960415088 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1114068400 0x7f111406a8c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.858+0000 7f1138a9b640 1 --2- 172.21.15.98:0/2960415088 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f11340d4820 0x7f113413d520 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.858+0000 7f1138a9b640 1 --2- 172.21.15.98:0/2960415088 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f11340b9970 0x7f1134158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.858+0000 7f1138a9b640 1 --2- 172.21.15.98:0/2960415088 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f11340b7f80 0x7f11341565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.858+0000 7f1138a9b640 1 -- 172.21.15.98:0/2960415088 >> 172.21.15.98:0/2960415088 conn(0x7f1134074e40 msgr2=0x7f11340d0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:38.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.858+0000 7f1138a9b640 1 -- 172.21.15.98:0/2960415088 shutdown_connections 2024-09-19T08:47:38.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.858+0000 7f1138a9b640 1 -- 172.21.15.98:0/2960415088 wait complete. 2024-09-19T08:47:38.872 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:47:38.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.982+0000 7fc149c95640 1 Processor -- start 2024-09-19T08:47:38.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.982+0000 7fc149c95640 1 -- start start 2024-09-19T08:47:38.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.986+0000 7fc149c95640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc144158fd0 0x7fc1440b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.986+0000 7fc149c95640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc14406c3a0 0x7fc1440ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.986+0000 7fc149c95640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1440ccc60 con 0x7fc1440b4360 2024-09-19T08:47:38.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.986+0000 7fc149c95640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fc1441593b0 con 0x7fc144158fd0 2024-09-19T08:47:38.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.986+0000 7fc149c95640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc1440ccf40 con 0x7fc14406c3a0 2024-09-19T08:47:38.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.986+0000 7fc1437fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc144158fd0 0x7fc1440b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.986+0000 7fc142ffd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fc1440b4360 0x7fc14406bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:58794/0 (socket says 172.21.15.98:58794) 2024-09-19T08:47:38.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.986+0000 7fc142ffd640 1 -- 172.21.15.98:0/4158132839 learned_addr learned my addr 172.21.15.98:0/4158132839 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:38.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.986+0000 7fc143fff640 1 --2- 172.21.15.98:0/4158132839 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc14406c3a0 0x7fc1440ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.986+0000 7fc1437fe640 1 -- 172.21.15.98:0/4158132839 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc14406c3a0 msgr2=0x7fc1440ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:38.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.986+0000 7fc1437fe640 1 --2- 172.21.15.98:0/4158132839 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc14406c3a0 0x7fc1440ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.986+0000 7fc1437fe640 1 -- 172.21.15.98:0/4158132839 >> v1:172.21.15.98:6789/0 conn(0x7fc1440b4360 legacy=0x7fc14406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:38.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.986+0000 7fc1437fe640 1 -- 172.21.15.98:0/4158132839 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1440cd220 con 0x7fc144158fd0 2024-09-19T08:47:38.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.986+0000 7fc1427fc640 1 -- 172.21.15.98:0/4158132839 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1188279595 0 0) 0x7fc1440ccc60 con 0x7fc1440b4360 2024-09-19T08:47:38.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.986+0000 7fc1437fe640 1 --2- 172.21.15.98:0/4158132839 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc144158fd0 0x7fc1440b3e20 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fc13c0097c0 tx=0x7fc13c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80289ad552fa0f92 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:38.987 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.986+0000 7fc1427fc640 1 -- 172.21.15.98:0/4158132839 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c0b1070 con 0x7fc144158fd0 2024-09-19T08:47:38.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.986+0000 7fc1427fc640 1 -- 172.21.15.98:0/4158132839 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc13c0a8050 con 0x7fc144158fd0 2024-09-19T08:47:38.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.990+0000 7fc1427fc640 1 -- 172.21.15.98:0/4158132839 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c0ac860 con 0x7fc144158fd0 2024-09-19T08:47:38.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.990+0000 7fc149c95640 1 -- 172.21.15.98:0/4158132839 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc144158fd0 msgr2=0x7fc1440b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:38.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.990+0000 7fc149c95640 1 --2- 172.21.15.98:0/4158132839 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc144158fd0 0x7fc1440b3e20 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fc13c0097c0 tx=0x7fc13c01c3e0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.990+0000 7fc149c95640 1 -- 172.21.15.98:0/4158132839 shutdown_connections 2024-09-19T08:47:38.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.990+0000 7fc149c95640 1 --2- 172.21.15.98:0/4158132839 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc14406c3a0 0x7fc1440ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.990+0000 7fc149c95640 1 --2- 172.21.15.98:0/4158132839 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc144158fd0 0x7fc1440b3e20 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.990+0000 7fc149c95640 1 -- 172.21.15.98:0/4158132839 >> 172.21.15.98:0/4158132839 conn(0x7fc1440b2010 msgr2=0x7fc1440b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:38.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.990+0000 7fc149c95640 1 -- 172.21.15.98:0/4158132839 shutdown_connections 2024-09-19T08:47:38.989 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.990+0000 7fc149c95640 1 -- 172.21.15.98:0/4158132839 wait complete. 2024-09-19T08:47:38.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.990+0000 7fc149c95640 1 Processor -- start 2024-09-19T08:47:38.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.990+0000 7fc149c95640 1 -- start start 2024-09-19T08:47:38.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.990+0000 7fc149c95640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc14406c3a0 0x7fc1440b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.990+0000 7fc149c95640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc1440b4360 0x7fc144161be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.990+0000 7fc149c95640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc144158fd0 0x7fc14415a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.990+0000 7fc149c95640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc1440cf710 con 0x7fc14406c3a0 2024-09-19T08:47:38.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.990+0000 7fc149c95640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc1440cf590 con 0x7fc144158fd0 2024-09-19T08:47:38.991 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.990+0000 7fc149c95640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc1440cf890 con 0x7fc1440b4360 2024-09-19T08:47:38.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.990+0000 7fc1437fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc14406c3a0 0x7fc1440b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.990+0000 7fc1437fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc14406c3a0 0x7fc1440b0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37248/0 (socket says 172.21.15.98:37248) 2024-09-19T08:47:38.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.990+0000 7fc1437fe640 1 -- 172.21.15.98:0/1116523869 learned_addr learned my addr 172.21.15.98:0/1116523869 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:38.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.990+0000 7fc143fff640 1 --2- 172.21.15.98:0/1116523869 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc144158fd0 0x7fc14415a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.990+0000 7fc142ffd640 1 --2- 172.21.15.98:0/1116523869 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc1440b4360 0x7fc144161be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.990+0000 7fc1437fe640 1 -- 172.21.15.98:0/1116523869 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc1440b4360 msgr2=0x7fc144161be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:38.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.990+0000 7fc1437fe640 1 --2- 172.21.15.98:0/1116523869 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc1440b4360 0x7fc144161be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.990+0000 7fc1437fe640 1 -- 172.21.15.98:0/1116523869 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc144158fd0 msgr2=0x7fc14415a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:38.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.990+0000 7fc1437fe640 1 --2- 172.21.15.98:0/1116523869 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc144158fd0 0x7fc14415a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:38.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.990+0000 7fc1437fe640 1 -- 172.21.15.98:0/1116523869 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc144160b80 con 0x7fc14406c3a0 2024-09-19T08:47:38.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.990+0000 7fc1437fe640 1 --2- 172.21.15.98:0/1116523869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc14406c3a0 0x7fc1440b0fd0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fc13c01aa90 tx=0x7fc13c01ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:38.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.990+0000 7fc13bfff640 1 -- 172.21.15.98:0/1116523869 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c0b1040 con 0x7fc14406c3a0 2024-09-19T08:47:38.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.990+0000 7fc13bfff640 1 -- 172.21.15.98:0/1116523869 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc13c0bd070 con 0x7fc14406c3a0 2024-09-19T08:47:38.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.990+0000 7fc13bfff640 1 -- 172.21.15.98:0/1116523869 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c0b7660 con 0x7fc14406c3a0 2024-09-19T08:47:38.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.990+0000 7fc149c95640 1 -- 172.21.15.98:0/1116523869 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc144160e20 con 0x7fc14406c3a0 2024-09-19T08:47:38.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.990+0000 7fc149c95640 1 -- 172.21.15.98:0/1116523869 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1441613e0 con 0x7fc14406c3a0 2024-09-19T08:47:38.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.994+0000 7fc149c95640 1 -- 172.21.15.98:0/1116523869 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1440cc6a0 con 0x7fc14406c3a0 2024-09-19T08:47:38.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.994+0000 7fc13bfff640 1 -- 172.21.15.98:0/1116523869 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7fc13c0b7800 con 0x7fc14406c3a0 2024-09-19T08:47:38.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.994+0000 7fc13bfff640 1 --2- 172.21.15.98:0/1116523869 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc124068430 0x7fc12406a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:38.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.994+0000 7fc13bfff640 1 -- 172.21.15.98:0/1116523869 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5737+0+0 (secure 0 0 0) 0x7fc13c11ea50 con 0x7fc14406c3a0 2024-09-19T08:47:38.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.994+0000 7fc142ffd640 1 --2- 172.21.15.98:0/1116523869 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc124068430 0x7fc12406a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:38.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.994+0000 7fc142ffd640 1 --2- 172.21.15.98:0/1116523869 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc124068430 0x7fc12406a8f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc144073630 tx=0x7fc12c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:38.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:38.998+0000 7fc13bfff640 1 -- 172.21.15.98:0/1116523869 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc13c016090 con 0x7fc14406c3a0 2024-09-19T08:47:39.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.178+0000 7fc149c95640 1 -- 172.21.15.98:0/1116523869 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1441617c0 con 0x7fc14406c3a0 2024-09-19T08:47:39.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.178+0000 7fc13bfff640 1 -- 172.21.15.98:0/1116523869 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) ==== 95+0+5235 (secure 0 0 0) 0x7fc13c0f1040 con 0x7fc14406c3a0 2024-09-19T08:47:39.179 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:47:39.179 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":101,"btime":"2024-09-19T08:47:30:391433+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6392,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/1027012777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":1027012777},{"type":"v1","addr":"172.21.15.139:6834","nonce":1027012777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7339,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3216462399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6833","nonce":3216462399},{"type":"v1","addr":"172.21.15.139:6835","nonce":3216462399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":9174,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1629579852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1629579852},{"type":"v1","addr":"172.21.15.98:6833","nonce":1629579852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T08:47:13.743399+0000","modified":"2024-09-19T08:47:16.695205+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9177":{"gid":9177,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.98:6835/3407902663","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":3407902663},{"type":"v1","addr":"172.21.15.98:6835","nonce":3407902663}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9177,"qdb_cluster":[9177]},"id":13}]} 2024-09-19T08:47:39.179 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 101 2024-09-19T08:47:39.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.182+0000 7fc149c95640 1 -- 172.21.15.98:0/1116523869 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc124068430 msgr2=0x7fc12406a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:39.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.182+0000 7fc149c95640 1 --2- 172.21.15.98:0/1116523869 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc124068430 0x7fc12406a8f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc144073630 tx=0x7fc12c009040 comp rx=0 tx=0).stop 2024-09-19T08:47:39.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.182+0000 7fc149c95640 1 -- 172.21.15.98:0/1116523869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc14406c3a0 msgr2=0x7fc1440b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:39.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.182+0000 7fc149c95640 1 --2- 172.21.15.98:0/1116523869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc14406c3a0 0x7fc1440b0fd0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fc13c01aa90 tx=0x7fc13c01ac20 comp rx=0 tx=0).stop 2024-09-19T08:47:39.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.182+0000 7fc149c95640 1 -- 172.21.15.98:0/1116523869 shutdown_connections 2024-09-19T08:47:39.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.182+0000 7fc149c95640 1 --2- 172.21.15.98:0/1116523869 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc124068430 0x7fc12406a8f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:39.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.182+0000 7fc149c95640 1 --2- 172.21.15.98:0/1116523869 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc144158fd0 0x7fc14415a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:39.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.182+0000 7fc149c95640 1 --2- 172.21.15.98:0/1116523869 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc1440b4360 0x7fc144161be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:39.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.182+0000 7fc149c95640 1 --2- 172.21.15.98:0/1116523869 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc14406c3a0 0x7fc1440b0fd0 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:39.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.182+0000 7fc149c95640 1 -- 172.21.15.98:0/1116523869 >> 172.21.15.98:0/1116523869 conn(0x7fc1440b2010 msgr2=0x7fc144159a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:39.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.182+0000 7fc149c95640 1 -- 172.21.15.98:0/1116523869 shutdown_connections 2024-09-19T08:47:39.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.182+0000 7fc149c95640 1 -- 172.21.15.98:0/1116523869 wait complete. 2024-09-19T08:47:39.197 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:47:39.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.302+0000 7ff99834a640 1 Processor -- start 2024-09-19T08:47:39.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.302+0000 7ff99834a640 1 -- start start 2024-09-19T08:47:39.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.306+0000 7ff99834a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7ff99015c550 0x7ff99006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:39.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.306+0000 7ff99834a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff99006c1d0 0x7ff9900cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:39.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.306+0000 7ff99834a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff9900cf900 0x7ff9900ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:39.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.306+0000 7ff99834a640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7ff99006c610 con 0x7ff9900cf900 2024-09-19T08:47:39.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.306+0000 7ff99834a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7ff99006d310 con 0x7ff99015c550 2024-09-19T08:47:39.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.306+0000 7ff99834a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff99006d5f0 con 0x7ff99006c1d0 2024-09-19T08:47:39.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.306+0000 7ff9968e7640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff9900cf900 0x7ff9900ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:39.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.306+0000 7ff9968e7640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7ff9900cf900 0x7ff9900ce1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37264/0 (socket says 172.21.15.98:37264) 2024-09-19T08:47:39.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.306+0000 7ff9968e7640 1 -- 172.21.15.98:0/3565925828 learned_addr learned my addr 172.21.15.98:0/3565925828 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:39.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.306+0000 7ff9958e5640 1 --2- 172.21.15.98:0/3565925828 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff99006c1d0 0x7ff9900cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:39.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.306+0000 7ff9960e6640 1 --2- 172.21.15.98:0/3565925828 >> v2:172.21.15.139:3300/0 conn(0x7ff99015c550 0x7ff99006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:39.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.306+0000 7ff9968e7640 1 -- 172.21.15.98:0/3565925828 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff99006c1d0 msgr2=0x7ff9900cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:39.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.306+0000 7ff9968e7640 1 --2- 172.21.15.98:0/3565925828 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff99006c1d0 0x7ff9900cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:39.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.306+0000 7ff9968e7640 1 -- 172.21.15.98:0/3565925828 >> v2:172.21.15.139:3300/0 conn(0x7ff99015c550 msgr2=0x7ff99006bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:39.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.306+0000 7ff9968e7640 1 --2- 172.21.15.98:0/3565925828 >> v2:172.21.15.139:3300/0 conn(0x7ff99015c550 0x7ff99006bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:39.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.306+0000 7ff9968e7640 1 -- 172.21.15.98:0/3565925828 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff99006d8d0 con 0x7ff9900cf900 2024-09-19T08:47:39.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.306+0000 7ff9968e7640 1 --2- 172.21.15.98:0/3565925828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9900cf900 0x7ff9900ce1a0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7ff9900b56b0 tx=0x7ff98801c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677c538f08b406bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:39.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.306+0000 7ff9950e4640 1 -- 172.21.15.98:0/3565925828 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9880a64d0 con 0x7ff9900cf900 2024-09-19T08:47:39.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.306+0000 7ff9950e4640 1 -- 172.21.15.98:0/3565925828 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9880a6670 con 0x7ff9900cf900 2024-09-19T08:47:39.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.306+0000 7ff9950e4640 1 -- 172.21.15.98:0/3565925828 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9880a6a40 con 0x7ff9900cf900 2024-09-19T08:47:39.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.310+0000 7ff99834a640 1 -- 172.21.15.98:0/3565925828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9900cf900 msgr2=0x7ff9900ce1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:39.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.310+0000 7ff99834a640 1 --2- 172.21.15.98:0/3565925828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9900cf900 0x7ff9900ce1a0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7ff9900b56b0 tx=0x7ff98801c730 comp rx=0 tx=0).stop 2024-09-19T08:47:39.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.310+0000 7ff99834a640 1 -- 172.21.15.98:0/3565925828 shutdown_connections 2024-09-19T08:47:39.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.310+0000 7ff99834a640 1 --2- 172.21.15.98:0/3565925828 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff9900cf900 0x7ff9900ce1a0 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:39.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.310+0000 7ff99834a640 1 --2- 172.21.15.98:0/3565925828 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff99006c1d0 0x7ff9900cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:39.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.310+0000 7ff99834a640 1 --2- 172.21.15.98:0/3565925828 >> v2:172.21.15.139:3300/0 conn(0x7ff99015c550 0x7ff99006bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:39.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.310+0000 7ff99834a640 1 -- 172.21.15.98:0/3565925828 >> 172.21.15.98:0/3565925828 conn(0x7ff9900b2010 msgr2=0x7ff9900b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:39.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.310+0000 7ff99834a640 1 -- 172.21.15.98:0/3565925828 shutdown_connections 2024-09-19T08:47:39.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.310+0000 7ff99834a640 1 -- 172.21.15.98:0/3565925828 wait complete. 2024-09-19T08:47:39.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.310+0000 7ff99834a640 1 Processor -- start 2024-09-19T08:47:39.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.310+0000 7ff99834a640 1 -- start start 2024-09-19T08:47:39.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.310+0000 7ff99834a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff99006c1d0 0x7ff99014efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:39.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.310+0000 7ff99834a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9900cf900 0x7ff990152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:39.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.310+0000 7ff99834a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff99015c550 0x7ff990158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:39.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.310+0000 7ff99834a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff9900723a0 con 0x7ff99015c550 2024-09-19T08:47:39.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.310+0000 7ff99834a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff99015cb40 con 0x7ff99006c1d0 2024-09-19T08:47:39.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.310+0000 7ff99834a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff990072520 con 0x7ff9900cf900 2024-09-19T08:47:39.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.310+0000 7ff9968e7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff99015c550 0x7ff990158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:39.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.310+0000 7ff9968e7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff99015c550 0x7ff990158760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37280/0 (socket says 172.21.15.98:37280) 2024-09-19T08:47:39.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.310+0000 7ff9968e7640 1 -- 172.21.15.98:0/214416552 learned_addr learned my addr 172.21.15.98:0/214416552 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:39.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.310+0000 7ff9958e5640 1 --2- 172.21.15.98:0/214416552 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9900cf900 0x7ff990152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:39.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.310+0000 7ff9960e6640 1 --2- 172.21.15.98:0/214416552 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff99006c1d0 0x7ff99014efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:39.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.310+0000 7ff9968e7640 1 -- 172.21.15.98:0/214416552 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9900cf900 msgr2=0x7ff990152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:39.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.310+0000 7ff9968e7640 1 --2- 172.21.15.98:0/214416552 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9900cf900 0x7ff990152750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:39.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.310+0000 7ff9968e7640 1 -- 172.21.15.98:0/214416552 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff99006c1d0 msgr2=0x7ff99014efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:39.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.310+0000 7ff9968e7640 1 --2- 172.21.15.98:0/214416552 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff99006c1d0 0x7ff99014efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:39.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.310+0000 7ff9968e7640 1 -- 172.21.15.98:0/214416552 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff99014f520 con 0x7ff99015c550 2024-09-19T08:47:39.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.310+0000 7ff9960e6640 1 --2- 172.21.15.98:0/214416552 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff99006c1d0 0x7ff99014efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:39.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.310+0000 7ff9968e7640 1 --2- 172.21.15.98:0/214416552 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff99015c550 0x7ff990158760 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7ff98801cfd0 tx=0x7ff9880194d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:39.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.310+0000 7ff986ffd640 1 -- 172.21.15.98:0/214416552 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9880bf070 con 0x7ff99015c550 2024-09-19T08:47:39.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.310+0000 7ff986ffd640 1 -- 172.21.15.98:0/214416552 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff988009150 con 0x7ff99015c550 2024-09-19T08:47:39.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.310+0000 7ff99834a640 1 -- 172.21.15.98:0/214416552 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff99014bc30 con 0x7ff99015c550 2024-09-19T08:47:39.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.310+0000 7ff986ffd640 1 -- 172.21.15.98:0/214416552 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9880a6a60 con 0x7ff99015c550 2024-09-19T08:47:39.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.310+0000 7ff99834a640 1 -- 172.21.15.98:0/214416552 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff99014c1b0 con 0x7ff99015c550 2024-09-19T08:47:39.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.310+0000 7ff99834a640 1 -- 172.21.15.98:0/214416552 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9900ce1a0 con 0x7ff99015c550 2024-09-19T08:47:39.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.314+0000 7ff986ffd640 1 -- 172.21.15.98:0/214416552 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7ff9880c2070 con 0x7ff99015c550 2024-09-19T08:47:39.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.314+0000 7ff986ffd640 1 --2- 172.21.15.98:0/214416552 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff974068400 0x7ff97406a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:39.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.314+0000 7ff986ffd640 1 -- 172.21.15.98:0/214416552 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5737+0+0 (secure 0 0 0) 0x7ff98811e5e0 con 0x7ff99015c550 2024-09-19T08:47:39.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.314+0000 7ff9960e6640 1 --2- 172.21.15.98:0/214416552 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff974068400 0x7ff97406a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:39.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.314+0000 7ff9960e6640 1 --2- 172.21.15.98:0/214416552 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff974068400 0x7ff97406a8c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff99014d760 tx=0x7ff98c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:39.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.318+0000 7ff986ffd640 1 -- 172.21.15.98:0/214416552 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff9880c9080 con 0x7ff99015c550 2024-09-19T08:47:39.467 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.466+0000 7ff99834a640 1 -- 172.21.15.98:0/214416552 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9900757a0 con 0x7ff99015c550 2024-09-19T08:47:39.470 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.470+0000 7ff986ffd640 1 -- 172.21.15.98:0/214416552 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) ==== 74+0+16787 (secure 0 0 0) 0x7ff99014f520 con 0x7ff99015c550 2024-09-19T08:47:39.470 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:47:39.470 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":170,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:47:15.308260+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-19T08:47:12.300410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-19T08:47:12.721062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:47:39.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.474+0000 7ff99834a640 1 -- 172.21.15.98:0/214416552 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff974068400 msgr2=0x7ff97406a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:39.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.474+0000 7ff99834a640 1 --2- 172.21.15.98:0/214416552 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff974068400 0x7ff97406a8c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff99014d760 tx=0x7ff98c008040 comp rx=0 tx=0).stop 2024-09-19T08:47:39.472 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.474+0000 7ff99834a640 1 -- 172.21.15.98:0/214416552 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff99015c550 msgr2=0x7ff990158760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:39.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.474+0000 7ff99834a640 1 --2- 172.21.15.98:0/214416552 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff99015c550 0x7ff990158760 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7ff98801cfd0 tx=0x7ff9880194d0 comp rx=0 tx=0).stop 2024-09-19T08:47:39.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.474+0000 7ff99834a640 1 -- 172.21.15.98:0/214416552 shutdown_connections 2024-09-19T08:47:39.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.474+0000 7ff99834a640 1 --2- 172.21.15.98:0/214416552 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff974068400 0x7ff97406a8c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:39.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.474+0000 7ff99834a640 1 --2- 172.21.15.98:0/214416552 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff99015c550 0x7ff990158760 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:39.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.474+0000 7ff99834a640 1 --2- 172.21.15.98:0/214416552 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9900cf900 0x7ff990152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:39.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.474+0000 7ff99834a640 1 --2- 172.21.15.98:0/214416552 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff99006c1d0 0x7ff99014efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:39.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.474+0000 7ff99834a640 1 -- 172.21.15.98:0/214416552 >> 172.21.15.98:0/214416552 conn(0x7ff9900b2010 msgr2=0x7ff990153a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:39.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.474+0000 7ff99834a640 1 -- 172.21.15.98:0/214416552 shutdown_connections 2024-09-19T08:47:39.473 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.474+0000 7ff99834a640 1 -- 172.21.15.98:0/214416552 wait complete. 2024-09-19T08:47:39.489 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-19T08:47:39.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.610+0000 7f452656c640 1 Processor -- start 2024-09-19T08:47:39.608 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.610+0000 7f452656c640 1 -- start start 2024-09-19T08:47:39.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.610+0000 7f452656c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4520140d30 0x7f452006d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:39.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.610+0000 7f452656c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4520141110 0x7f452006d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:39.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.610+0000 7f452656c640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f452015ea10 con 0x7f452006bc90 2024-09-19T08:47:39.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.610+0000 7f452656c640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f452015e7c0 con 0x7f4520141110 2024-09-19T08:47:39.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.610+0000 7f452656c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f452015ecf0 con 0x7f4520140d30 2024-09-19T08:47:39.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.610+0000 7f451f7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f4520141110 0x7f452006d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:39.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.610+0000 7f4524b09640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f452006bc90 0x7f45200b3310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:58804/0 (socket says 172.21.15.98:58804) 2024-09-19T08:47:39.610 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.610+0000 7f4524b09640 1 -- 172.21.15.98:0/296785969 learned_addr learned my addr 172.21.15.98:0/296785969 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:39.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.610+0000 7f451ffff640 1 --2- 172.21.15.98:0/296785969 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4520140d30 0x7f452006d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:39.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.610+0000 7f451f7fe640 1 -- 172.21.15.98:0/296785969 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4520140d30 msgr2=0x7f452006d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:39.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.610+0000 7f451f7fe640 1 --2- 172.21.15.98:0/296785969 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4520140d30 0x7f452006d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:39.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.610+0000 7f451f7fe640 1 -- 172.21.15.98:0/296785969 >> v1:172.21.15.98:6789/0 conn(0x7f452006bc90 legacy=0x7f45200b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:39.611 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.610+0000 7f451f7fe640 1 -- 172.21.15.98:0/296785969 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452015efd0 con 0x7f4520141110 2024-09-19T08:47:39.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.614+0000 7f451f7fe640 1 --2- 172.21.15.98:0/296785969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4520141110 0x7f452006d6b0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f45080097c0 tx=0x7f450801e6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=360e4c977154650a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:39.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.614+0000 7f451effd640 1 -- 172.21.15.98:0/296785969 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45080b2600 con 0x7f4520141110 2024-09-19T08:47:39.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.614+0000 7f451effd640 1 -- 172.21.15.98:0/296785969 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45080b27a0 con 0x7f4520141110 2024-09-19T08:47:39.612 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.614+0000 7f451effd640 1 -- 172.21.15.98:0/296785969 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45080b2b70 con 0x7f4520141110 2024-09-19T08:47:39.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.614+0000 7f452656c640 1 -- 172.21.15.98:0/296785969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4520141110 msgr2=0x7f452006d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:39.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.614+0000 7f452656c640 1 --2- 172.21.15.98:0/296785969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4520141110 0x7f452006d6b0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f45080097c0 tx=0x7f450801e6b0 comp rx=0 tx=0).stop 2024-09-19T08:47:39.613 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.614+0000 7f452656c640 1 -- 172.21.15.98:0/296785969 shutdown_connections 2024-09-19T08:47:39.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.614+0000 7f452656c640 1 --2- 172.21.15.98:0/296785969 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f4520141110 0x7f452006d6b0 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:39.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.614+0000 7f452656c640 1 --2- 172.21.15.98:0/296785969 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4520140d30 0x7f452006d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:39.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.614+0000 7f452656c640 1 -- 172.21.15.98:0/296785969 >> 172.21.15.98:0/296785969 conn(0x7f45200b2010 msgr2=0x7f45200b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:39.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.614+0000 7f452656c640 1 -- 172.21.15.98:0/296785969 shutdown_connections 2024-09-19T08:47:39.614 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.614+0000 7f452656c640 1 -- 172.21.15.98:0/296785969 wait complete. 2024-09-19T08:47:39.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.614+0000 7f452656c640 1 Processor -- start 2024-09-19T08:47:39.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.614+0000 7f452656c640 1 -- start start 2024-09-19T08:47:39.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.614+0000 7f452656c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f452006bc90 0x7f4520072180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:39.616 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.614+0000 7f452656c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4520140d30 0x7f452013fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:39.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.614+0000 7f452656c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4520141110 0x7f45201f8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:39.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.614+0000 7f452656c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f452015bdd0 con 0x7f452006bc90 2024-09-19T08:47:39.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.614+0000 7f452656c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f452015bc50 con 0x7f4520141110 2024-09-19T08:47:39.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.614+0000 7f452656c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f452015bf50 con 0x7f4520140d30 2024-09-19T08:47:39.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.614+0000 7f451ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f452006bc90 0x7f4520072180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:39.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.614+0000 7f451ffff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f452006bc90 0x7f4520072180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37300/0 (socket says 172.21.15.98:37300) 2024-09-19T08:47:39.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.614+0000 7f451ffff640 1 -- 172.21.15.98:0/2567484667 learned_addr learned my addr 172.21.15.98:0/2567484667 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:39.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.614+0000 7f451ffff640 1 -- 172.21.15.98:0/2567484667 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4520140d30 msgr2=0x7f452013fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:39.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.614+0000 7f451ffff640 1 --2- 172.21.15.98:0/2567484667 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4520140d30 0x7f452013fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:39.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.614+0000 7f451ffff640 1 -- 172.21.15.98:0/2567484667 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4520141110 msgr2=0x7f45201f8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:39.617 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.614+0000 7f451ffff640 1 --2- 172.21.15.98:0/2567484667 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4520141110 0x7f45201f8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:39.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.614+0000 7f451ffff640 1 -- 172.21.15.98:0/2567484667 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452015fb40 con 0x7f452006bc90 2024-09-19T08:47:39.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.614+0000 7f451ffff640 1 --2- 172.21.15.98:0/2567484667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f452006bc90 0x7f4520072180 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f451000d5f0 tx=0x7f451000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:39.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.614+0000 7f451cff9640 1 -- 172.21.15.98:0/2567484667 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45100125f0 con 0x7f452006bc90 2024-09-19T08:47:39.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.614+0000 7f451cff9640 1 -- 172.21.15.98:0/2567484667 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4510019070 con 0x7f452006bc90 2024-09-19T08:47:39.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.614+0000 7f452656c640 1 -- 172.21.15.98:0/2567484667 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f452015fe20 con 0x7f452006bc90 2024-09-19T08:47:39.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.614+0000 7f451cff9640 1 -- 172.21.15.98:0/2567484667 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45100129b0 con 0x7f452006bc90 2024-09-19T08:47:39.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.614+0000 7f452656c640 1 -- 172.21.15.98:0/2567484667 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45201f8db0 con 0x7f452006bc90 2024-09-19T08:47:39.618 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.618+0000 7f452656c640 1 -- 172.21.15.98:0/2567484667 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45200b3310 con 0x7f452006bc90 2024-09-19T08:47:39.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.618+0000 7f451cff9640 1 -- 172.21.15.98:0/2567484667 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f4510012b50 con 0x7f452006bc90 2024-09-19T08:47:39.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.618+0000 7f451cff9640 1 --2- 172.21.15.98:0/2567484667 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4500068400 0x7f450006a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:39.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.618+0000 7f451cff9640 1 -- 172.21.15.98:0/2567484667 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5737+0+0 (secure 0 0 0) 0x7f4510084aa0 con 0x7f452006bc90 2024-09-19T08:47:39.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.618+0000 7f451f7fe640 1 --2- 172.21.15.98:0/2567484667 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4500068400 0x7f450006a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:39.620 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.618+0000 7f451f7fe640 1 --2- 172.21.15.98:0/2567484667 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4500068400 0x7f450006a8c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f45080b1000 tx=0x7f4508004740 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:39.621 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.622+0000 7f451cff9640 1 -- 172.21.15.98:0/2567484667 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f4510056ff0 con 0x7f452006bc90 2024-09-19T08:47:39.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.802+0000 7f452656c640 1 -- 172.21.15.98:0/2567484667 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f4520165700 con 0x7f452006bc90 2024-09-19T08:47:39.803 INFO:tasks.ceph.mon.a.smithi098.stderr:2024-09-19T08:47:39.802+0000 7f5e49017640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T08:47:39.805 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.806+0000 7f451cff9640 1 -- 172.21.15.98:0/2567484667 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v102) ==== 161+0+0 (secure 0 0 0) 0x7f45100568f0 con 0x7f452006bc90 2024-09-19T08:47:39.806 INFO:teuthology.orchestra.run.smithi098.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T08:47:39.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.806+0000 7f452656c640 1 -- 172.21.15.98:0/2567484667 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4500068400 msgr2=0x7f450006a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:39.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.806+0000 7f452656c640 1 --2- 172.21.15.98:0/2567484667 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4500068400 0x7f450006a8c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f45080b1000 tx=0x7f4508004740 comp rx=0 tx=0).stop 2024-09-19T08:47:39.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.810+0000 7f452656c640 1 -- 172.21.15.98:0/2567484667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f452006bc90 msgr2=0x7f4520072180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:39.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.810+0000 7f452656c640 1 --2- 172.21.15.98:0/2567484667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f452006bc90 0x7f4520072180 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f451000d5f0 tx=0x7f451000dab0 comp rx=0 tx=0).stop 2024-09-19T08:47:39.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.810+0000 7f452656c640 1 -- 172.21.15.98:0/2567484667 shutdown_connections 2024-09-19T08:47:39.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.810+0000 7f452656c640 1 --2- 172.21.15.98:0/2567484667 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f4500068400 0x7f450006a8c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:39.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.810+0000 7f452656c640 1 --2- 172.21.15.98:0/2567484667 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4520141110 0x7f45201f8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:39.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.810+0000 7f452656c640 1 --2- 172.21.15.98:0/2567484667 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f4520140d30 0x7f452013fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:39.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.810+0000 7f452656c640 1 --2- 172.21.15.98:0/2567484667 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f452006bc90 0x7f4520072180 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:39.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.810+0000 7f452656c640 1 -- 172.21.15.98:0/2567484667 >> 172.21.15.98:0/2567484667 conn(0x7f45200b2010 msgr2=0x7f452015c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:39.809 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.810+0000 7f452656c640 1 -- 172.21.15.98:0/2567484667 shutdown_connections 2024-09-19T08:47:39.810 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.810+0000 7f452656c640 1 -- 172.21.15.98:0/2567484667 wait complete. 2024-09-19T08:47:39.826 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T08:47:39.880 INFO:tasks.ceph.mds.a.smithi098.stdout:starting mds.a at 2024-09-19T08:47:39.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.942+0000 7f81eab61640 1 Processor -- start 2024-09-19T08:47:39.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.942+0000 7f81eab61640 1 -- start start 2024-09-19T08:47:39.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.942+0000 7f81eab61640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81e40739a0 0x7f81e415eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:39.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.942+0000 7f81eab61640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81e4074060 con 0x7f81e4158ae0 2024-09-19T08:47:39.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.942+0000 7f81eab61640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81e40b20c0 con 0x7f81e40b1a00 2024-09-19T08:47:39.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.942+0000 7f81eab61640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f81e40b2cd0 con 0x7f81e40739a0 2024-09-19T08:47:39.949 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.942+0000 7f81e88fd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f81e4158ae0 0x7f81e4073290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:58816/0 (socket says 172.21.15.98:58816) 2024-09-19T08:47:39.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.942+0000 7f81e88fd640 1 -- 172.21.15.98:0/402618772 learned_addr learned my addr 172.21.15.98:0/402618772 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:39.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.942+0000 7f81e3fff640 1 --2- 172.21.15.98:0/402618772 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81e40739a0 0x7f81e415eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:39.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.942+0000 7f81e37fe640 1 -- 172.21.15.98:0/402618772 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 852622665 0 0) 0x7f81e4074060 con 0x7f81e4158ae0 2024-09-19T08:47:39.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.942+0000 7f81e37fe640 1 -- 172.21.15.98:0/402618772 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f81cc003530 con 0x7f81e4158ae0 2024-09-19T08:47:39.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.942+0000 7f81e37fe640 1 -- 172.21.15.98:0/402618772 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 530557054 0 0) 0x7f81cc003530 con 0x7f81e4158ae0 2024-09-19T08:47:39.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.942+0000 7f81e37fe640 1 -- 172.21.15.98:0/402618772 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81e40739a0 msgr2=0x7f81e415eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:39.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.942+0000 7f81e37fe640 1 --2- 172.21.15.98:0/402618772 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81e40739a0 0x7f81e415eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:39.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.942+0000 7f81e37fe640 1 -- 172.21.15.98:0/402618772 >> v1:172.21.15.139:6789/0 conn(0x7f81e40b1a00 legacy=0x7f81e406c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:39.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.942+0000 7f81e37fe640 1 -- 172.21.15.98:0/402618772 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81e40b23a0 con 0x7f81e4158ae0 2024-09-19T08:47:39.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.942+0000 7f81e37fe640 1 -- 172.21.15.98:0/402618772 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f81d4001680 con 0x7f81e4158ae0 2024-09-19T08:47:39.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.942+0000 7f81e37fe640 1 -- 172.21.15.98:0/402618772 >> v1:172.21.15.98:6789/0 conn(0x7f81e4158ae0 legacy=0x7f81e4073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:39.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.942+0000 7f81e37fe640 1 --2- 172.21.15.98:0/402618772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81cc0039f0 0x7f81cc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:39.950 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.942+0000 7f81e37fe640 1 --2- 172.21.15.98:0/402618772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81cc004810 0x7f81cc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:39.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.942+0000 7f81e37fe640 1 --2- 172.21.15.98:0/402618772 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81e40739a0 0x7f81cc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:39.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.942+0000 7f81e37fe640 1 -- 172.21.15.98:0/402618772 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc00ba80 con 0x7f81cc0039f0 2024-09-19T08:47:39.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.942+0000 7f81e37fe640 1 -- 172.21.15.98:0/402618772 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc00bc30 con 0x7f81cc004810 2024-09-19T08:47:39.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.942+0000 7f81e37fe640 1 -- 172.21.15.98:0/402618772 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f81cc00be30 con 0x7f81e40739a0 2024-09-19T08:47:39.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.942+0000 7f81e88fd640 1 --2- 172.21.15.98:0/402618772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81cc0039f0 0x7f81cc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:39.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.942+0000 7f81e3fff640 1 --2- 172.21.15.98:0/402618772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81cc004810 0x7f81cc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:39.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.942+0000 7f81e88fd640 1 -- 172.21.15.98:0/402618772 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81e40739a0 msgr2=0x7f81cc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:39.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.942+0000 7f81e88fd640 1 --2- 172.21.15.98:0/402618772 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81e40739a0 0x7f81cc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:39.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.942+0000 7f81e88fd640 1 -- 172.21.15.98:0/402618772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81cc004810 msgr2=0x7f81cc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:39.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.942+0000 7f81e88fd640 1 --2- 172.21.15.98:0/402618772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81cc004810 0x7f81cc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:39.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.942+0000 7f81e88fd640 1 -- 172.21.15.98:0/402618772 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81cc003530 con 0x7f81cc0039f0 2024-09-19T08:47:39.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.942+0000 7f81e3fff640 1 --2- 172.21.15.98:0/402618772 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81cc004810 0x7f81cc004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:39.951 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.946+0000 7f81e88fd640 1 --2- 172.21.15.98:0/402618772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81cc0039f0 0x7f81cc003e10 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f81d400c190 tx=0x7f81d401cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc1ba1b9a3c20df9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:39.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.946+0000 7f81e37fe640 1 -- 172.21.15.98:0/402618772 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d4007d00 con 0x7f81cc0039f0 2024-09-19T08:47:39.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.946+0000 7f81e37fe640 1 -- 172.21.15.98:0/402618772 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81d40a9020 con 0x7f81cc0039f0 2024-09-19T08:47:39.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.946+0000 7f81e37fe640 1 -- 172.21.15.98:0/402618772 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d4005b00 con 0x7f81cc0039f0 2024-09-19T08:47:39.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.946+0000 7f81eab61640 1 -- 172.21.15.98:0/402618772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81cc0039f0 msgr2=0x7f81cc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:39.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.946+0000 7f81eab61640 1 --2- 172.21.15.98:0/402618772 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81cc0039f0 0x7f81cc003e10 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f81d400c190 tx=0x7f81d401cc20 comp rx=0 tx=0).stop 2024-09-19T08:47:39.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.946+0000 7f81e90fe640 1 -- 172.21.15.98:0/402618772 reap_dead start 2024-09-19T08:47:39.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.946+0000 7f81eab61640 1 -- 172.21.15.98:0/402618772 shutdown_connections 2024-09-19T08:47:39.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.946+0000 7f81eab61640 1 -- 172.21.15.98:0/402618772 >> 172.21.15.98:0/402618772 conn(0x7f81e40b4d60 msgr2=0x7f81e40b5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:39.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.946+0000 7f81eab61640 1 -- 172.21.15.98:0/402618772 shutdown_connections 2024-09-19T08:47:39.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.946+0000 7f81eab61640 1 -- 172.21.15.98:0/402618772 wait complete. 2024-09-19T08:47:39.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.946+0000 7f81eab61640 1 Processor -- start 2024-09-19T08:47:39.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.946+0000 7f81eab61640 1 -- start start 2024-09-19T08:47:39.952 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.946+0000 7f81eab61640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81cc0039f0 0x7f81e4159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:39.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.946+0000 7f81eab61640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81e406f450 0x7f81e409b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:39.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.946+0000 7f81eab61640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81e406f8d0 0x7f81e409b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:39.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.946+0000 7f81eab61640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f81e415fe70 con 0x7f81e406f450 2024-09-19T08:47:39.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.946+0000 7f81eab61640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f81e415fcf0 con 0x7f81e406f8d0 2024-09-19T08:47:39.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.946+0000 7f81eab61640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f81e416a170 con 0x7f81cc0039f0 2024-09-19T08:47:39.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.950+0000 7f81e3fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81e406f450 0x7f81e409b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:39.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.950+0000 7f81e3fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81e406f450 0x7f81e409b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37340/0 (socket says 172.21.15.98:37340) 2024-09-19T08:47:39.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.950+0000 7f81e3fff640 1 -- 172.21.15.98:0/1298688139 learned_addr learned my addr 172.21.15.98:0/1298688139 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:39.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.950+0000 7f81e90fe640 1 --2- 172.21.15.98:0/1298688139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81e406f8d0 0x7f81e409b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:39.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.950+0000 7f81e88fd640 1 --2- 172.21.15.98:0/1298688139 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81cc0039f0 0x7f81e4159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:39.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.950+0000 7f81e3fff640 1 -- 172.21.15.98:0/1298688139 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81cc0039f0 msgr2=0x7f81e4159750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:39.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.950+0000 7f81e3fff640 1 --2- 172.21.15.98:0/1298688139 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81cc0039f0 0x7f81e4159750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:39.953 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.950+0000 7f81e3fff640 1 -- 172.21.15.98:0/1298688139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81e406f8d0 msgr2=0x7f81e409b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:39.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.950+0000 7f81e3fff640 1 --2- 172.21.15.98:0/1298688139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81e406f8d0 0x7f81e409b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:39.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.950+0000 7f81e3fff640 1 -- 172.21.15.98:0/1298688139 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81e409d360 con 0x7f81e406f450 2024-09-19T08:47:39.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.950+0000 7f81e3fff640 1 --2- 172.21.15.98:0/1298688139 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81e406f450 0x7f81e409b2c0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f81d80108c0 tx=0x7f81d8010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:39.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.950+0000 7f81e17fa640 1 -- 172.21.15.98:0/1298688139 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d800f040 con 0x7f81e406f450 2024-09-19T08:47:39.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.950+0000 7f81e17fa640 1 -- 172.21.15.98:0/1298688139 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81d8004220 con 0x7f81e406f450 2024-09-19T08:47:39.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.950+0000 7f81eab61640 1 -- 172.21.15.98:0/1298688139 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81e409d640 con 0x7f81e406f450 2024-09-19T08:47:39.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.950+0000 7f81e17fa640 1 -- 172.21.15.98:0/1298688139 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d800e900 con 0x7f81e406f450 2024-09-19T08:47:39.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.950+0000 7f81eab61640 1 -- 172.21.15.98:0/1298688139 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81e409dc00 con 0x7f81e406f450 2024-09-19T08:47:39.954 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.950+0000 7f81eab61640 1 -- 172.21.15.98:0/1298688139 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81e406c180 con 0x7f81e406f450 2024-09-19T08:47:39.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.950+0000 7f81e17fa640 1 -- 172.21.15.98:0/1298688139 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f81d800eaa0 con 0x7f81e406f450 2024-09-19T08:47:39.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.950+0000 7f81e17fa640 1 --2- 172.21.15.98:0/1298688139 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f81d0068400 0x7f81d006a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:39.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.950+0000 7f81e17fa640 1 -- 172.21.15.98:0/1298688139 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 5823+0+0 (secure 0 0 0) 0x7f81d8084e60 con 0x7f81e406f450 2024-09-19T08:47:39.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.950+0000 7f81e88fd640 1 --2- 172.21.15.98:0/1298688139 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f81d0068400 0x7f81d006a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:39.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.950+0000 7f81e88fd640 1 --2- 172.21.15.98:0/1298688139 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f81d0068400 0x7f81d006a8c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f81d40aa000 tx=0x7f81d40b0040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:39.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:39.954+0000 7f81e17fa640 1 -- 172.21.15.98:0/1298688139 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f81d8018030 con 0x7f81e406f450 2024-09-19T08:47:40.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.150+0000 7f81eab61640 1 -- 172.21.15.98:0/1298688139 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f81e415d9b0 con 0x7f81e406f450 2024-09-19T08:47:40.808 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.810+0000 7f81e17fa640 1 -- 172.21.15.98:0/1298688139 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v103) ==== 104+0+0 (secure 0 0 0) 0x7f81d80572a0 con 0x7f81e406f450 2024-09-19T08:47:40.812 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.814+0000 7f81eab61640 1 -- 172.21.15.98:0/1298688139 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f81d0068400 msgr2=0x7f81d006a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:40.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.814+0000 7f81eab61640 1 --2- 172.21.15.98:0/1298688139 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f81d0068400 0x7f81d006a8c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f81d40aa000 tx=0x7f81d40b0040 comp rx=0 tx=0).stop 2024-09-19T08:47:40.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.814+0000 7f81eab61640 1 -- 172.21.15.98:0/1298688139 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81e406f450 msgr2=0x7f81e409b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:40.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.814+0000 7f81eab61640 1 --2- 172.21.15.98:0/1298688139 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81e406f450 0x7f81e409b2c0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f81d80108c0 tx=0x7f81d8010d80 comp rx=0 tx=0).stop 2024-09-19T08:47:40.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.814+0000 7f81eab61640 1 -- 172.21.15.98:0/1298688139 shutdown_connections 2024-09-19T08:47:40.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.814+0000 7f81eab61640 1 --2- 172.21.15.98:0/1298688139 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f81d0068400 0x7f81d006a8c0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:40.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.814+0000 7f81eab61640 1 --2- 172.21.15.98:0/1298688139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f81e406f8d0 0x7f81e409b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:40.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.814+0000 7f81eab61640 1 --2- 172.21.15.98:0/1298688139 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f81e406f450 0x7f81e409b2c0 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:40.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.814+0000 7f81eab61640 1 --2- 172.21.15.98:0/1298688139 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f81cc0039f0 0x7f81e4159750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:40.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.814+0000 7f81eab61640 1 -- 172.21.15.98:0/1298688139 >> 172.21.15.98:0/1298688139 conn(0x7f81e40b4d60 msgr2=0x7f81e415c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:40.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.814+0000 7f81eab61640 1 -- 172.21.15.98:0/1298688139 shutdown_connections 2024-09-19T08:47:40.813 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.814+0000 7f81eab61640 1 -- 172.21.15.98:0/1298688139 wait complete. 2024-09-19T08:47:40.833 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T08:47:40.961 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.962+0000 7f42fea56640 1 Processor -- start 2024-09-19T08:47:40.962 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.962+0000 7f42fea56640 1 -- start start 2024-09-19T08:47:40.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.962+0000 7f42fea56640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f42f80cf700 0x7f42f815ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:40.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.962+0000 7f42fea56640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42f806d4c0 0x7f42f806d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:40.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.962+0000 7f42fea56640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42f81608c0 con 0x7f42f815c290 2024-09-19T08:47:40.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.962+0000 7f42fea56640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f42f8160e80 con 0x7f42f80cf700 2024-09-19T08:47:40.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.962+0000 7f42fea56640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f42f80cd510 con 0x7f42f806d4c0 2024-09-19T08:47:40.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.962+0000 7f42f77fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f42f80cf700 0x7f42f815ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:40.964 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.962+0000 7f42f77fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f42f80cf700 0x7f42f815ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37344/0 (socket says 172.21.15.98:37344) 2024-09-19T08:47:40.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.962+0000 7f42f77fe640 1 -- 172.21.15.98:0/2332550282 learned_addr learned my addr 172.21.15.98:0/2332550282 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:40.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.962+0000 7f42fcff3640 1 --2- 172.21.15.98:0/2332550282 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42f806d4c0 0x7f42f806d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:40.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.962+0000 7f42f77fe640 1 -- 172.21.15.98:0/2332550282 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42f806d4c0 msgr2=0x7f42f806d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:40.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.962+0000 7f42f77fe640 1 --2- 172.21.15.98:0/2332550282 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42f806d4c0 0x7f42f806d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:40.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.962+0000 7f42f77fe640 1 -- 172.21.15.98:0/2332550282 >> v1:172.21.15.139:6789/0 conn(0x7f42f815c290 legacy=0x7f42f80cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:40.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.962+0000 7f42f77fe640 1 -- 172.21.15.98:0/2332550282 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f8160ba0 con 0x7f42f80cf700 2024-09-19T08:47:40.965 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.962+0000 7f42fcff3640 1 --2- 172.21.15.98:0/2332550282 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42f806d4c0 0x7f42f806d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:40.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.966+0000 7f42f77fe640 1 --2- 172.21.15.98:0/2332550282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42f80cf700 0x7f42f815ff50 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f42f815feb0 tx=0x7f42ec01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4fd95775505f5a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:40.967 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.966+0000 7f42f6ffd640 1 -- 172.21.15.98:0/2332550282 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ec0b0070 con 0x7f42f80cf700 2024-09-19T08:47:40.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.966+0000 7f42f6ffd640 1 -- 172.21.15.98:0/2332550282 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42ec0ab600 con 0x7f42f80cf700 2024-09-19T08:47:40.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.966+0000 7f42f6ffd640 1 -- 172.21.15.98:0/2332550282 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ec0ab9d0 con 0x7f42f80cf700 2024-09-19T08:47:40.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.966+0000 7f42fea56640 1 -- 172.21.15.98:0/2332550282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42f80cf700 msgr2=0x7f42f815ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:40.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.966+0000 7f42fea56640 1 --2- 172.21.15.98:0/2332550282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42f80cf700 0x7f42f815ff50 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f42f815feb0 tx=0x7f42ec01c4e0 comp rx=0 tx=0).stop 2024-09-19T08:47:40.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.966+0000 7f42fea56640 1 -- 172.21.15.98:0/2332550282 shutdown_connections 2024-09-19T08:47:40.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.966+0000 7f42fea56640 1 --2- 172.21.15.98:0/2332550282 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42f806d4c0 0x7f42f806d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:40.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.966+0000 7f42fea56640 1 --2- 172.21.15.98:0/2332550282 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42f80cf700 0x7f42f815ff50 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:40.968 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.966+0000 7f42fea56640 1 -- 172.21.15.98:0/2332550282 >> 172.21.15.98:0/2332550282 conn(0x7f42f806c090 msgr2=0x7f42f806c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:40.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.966+0000 7f42fea56640 1 -- 172.21.15.98:0/2332550282 shutdown_connections 2024-09-19T08:47:40.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.966+0000 7f42fea56640 1 -- 172.21.15.98:0/2332550282 wait complete. 2024-09-19T08:47:40.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.966+0000 7f42fea56640 1 Processor -- start 2024-09-19T08:47:40.969 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.966+0000 7f42fea56640 1 -- start start 2024-09-19T08:47:40.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.970+0000 7f42fea56640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42f806d4c0 0x7f42f81588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:40.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.970+0000 7f42fea56640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42f80cf700 0x7f42f815df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:40.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.970+0000 7f42fea56640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42f815c290 0x7f42f80b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:40.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.970+0000 7f42fea56640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f42f80b5580 con 0x7f42f806d4c0 2024-09-19T08:47:40.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.970+0000 7f42fea56640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f42f80b5400 con 0x7f42f80cf700 2024-09-19T08:47:40.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.970+0000 7f42fea56640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f42f81690c0 con 0x7f42f815c290 2024-09-19T08:47:40.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.970+0000 7f42f7fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42f806d4c0 0x7f42f81588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:40.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.970+0000 7f42f7fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42f806d4c0 0x7f42f81588a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37352/0 (socket says 172.21.15.98:37352) 2024-09-19T08:47:40.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.970+0000 7f42f7fff640 1 -- 172.21.15.98:0/2313888635 learned_addr learned my addr 172.21.15.98:0/2313888635 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:40.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.970+0000 7f42f77fe640 1 --2- 172.21.15.98:0/2313888635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42f80cf700 0x7f42f815df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:40.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.970+0000 7f42fcff3640 1 --2- 172.21.15.98:0/2313888635 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42f815c290 0x7f42f80b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:40.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.970+0000 7f42f7fff640 1 -- 172.21.15.98:0/2313888635 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42f815c290 msgr2=0x7f42f80b0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:40.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.970+0000 7f42f7fff640 1 --2- 172.21.15.98:0/2313888635 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42f815c290 0x7f42f80b0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:40.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.970+0000 7f42f7fff640 1 -- 172.21.15.98:0/2313888635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42f80cf700 msgr2=0x7f42f815df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:40.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.970+0000 7f42f7fff640 1 --2- 172.21.15.98:0/2313888635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42f80cf700 0x7f42f815df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:40.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.970+0000 7f42f7fff640 1 -- 172.21.15.98:0/2313888635 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f8166050 con 0x7f42f806d4c0 2024-09-19T08:47:40.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.970+0000 7f42fcff3640 1 --2- 172.21.15.98:0/2313888635 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42f815c290 0x7f42f80b0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:40.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.970+0000 7f42f77fe640 1 --2- 172.21.15.98:0/2313888635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42f80cf700 0x7f42f815df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:40.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.970+0000 7f42f7fff640 1 --2- 172.21.15.98:0/2313888635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42f806d4c0 0x7f42f81588a0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f42e800b440 tx=0x7f42e800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:40.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.970+0000 7f42f4ff9640 1 -- 172.21.15.98:0/2313888635 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e8015020 con 0x7f42f806d4c0 2024-09-19T08:47:40.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.970+0000 7f42f4ff9640 1 -- 172.21.15.98:0/2313888635 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42e8011890 con 0x7f42f806d4c0 2024-09-19T08:47:40.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.970+0000 7f42fea56640 1 -- 172.21.15.98:0/2313888635 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42f8165440 con 0x7f42f806d4c0 2024-09-19T08:47:40.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.970+0000 7f42f4ff9640 1 -- 172.21.15.98:0/2313888635 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e8011c80 con 0x7f42f806d4c0 2024-09-19T08:47:40.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.970+0000 7f42fea56640 1 -- 172.21.15.98:0/2313888635 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42f8165a00 con 0x7f42f806d4c0 2024-09-19T08:47:40.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.970+0000 7f42fea56640 1 -- 172.21.15.98:0/2313888635 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42f806d8f0 con 0x7f42f806d4c0 2024-09-19T08:47:40.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.970+0000 7f42f4ff9640 1 -- 172.21.15.98:0/2313888635 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f42e8018020 con 0x7f42f806d4c0 2024-09-19T08:47:40.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.970+0000 7f42f4ff9640 1 --2- 172.21.15.98:0/2313888635 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f42e4068400 0x7f42e406a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:40.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.970+0000 7f42f4ff9640 1 -- 172.21.15.98:0/2313888635 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 5823+0+0 (secure 0 0 0) 0x7f42e8084770 con 0x7f42f806d4c0 2024-09-19T08:47:40.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.970+0000 7f42f77fe640 1 --2- 172.21.15.98:0/2313888635 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f42e4068400 0x7f42e406a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:40.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.974+0000 7f42f77fe640 1 --2- 172.21.15.98:0/2313888635 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f42e4068400 0x7f42e406a8c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f42ec0ad000 tx=0x7f42ec008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:40.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:40.974+0000 7f42f4ff9640 1 -- 172.21.15.98:0/2313888635 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f42e801f020 con 0x7f42f806d4c0 2024-09-19T08:47:41.144 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.146+0000 7f42fea56640 1 -- 172.21.15.98:0/2313888635 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f42f80b1b90 con 0x7f42f806d4c0 2024-09-19T08:47:41.818 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.818+0000 7f42f4ff9640 1 -- 172.21.15.98:0/2313888635 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v172) ==== 181+0+0 (secure 0 0 0) 0x7f42e800d040 con 0x7f42f806d4c0 2024-09-19T08:47:41.818 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_metadata' removed 2024-09-19T08:47:41.822 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.822+0000 7f42fea56640 1 -- 172.21.15.98:0/2313888635 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f42e4068400 msgr2=0x7f42e406a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:41.822 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.822+0000 7f42fea56640 1 --2- 172.21.15.98:0/2313888635 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f42e4068400 0x7f42e406a8c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f42ec0ad000 tx=0x7f42ec008040 comp rx=0 tx=0).stop 2024-09-19T08:47:41.822 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.822+0000 7f42fea56640 1 -- 172.21.15.98:0/2313888635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42f806d4c0 msgr2=0x7f42f81588a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:41.822 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.822+0000 7f42fea56640 1 --2- 172.21.15.98:0/2313888635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42f806d4c0 0x7f42f81588a0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f42e800b440 tx=0x7f42e800b900 comp rx=0 tx=0).stop 2024-09-19T08:47:41.822 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.822+0000 7f42fea56640 1 -- 172.21.15.98:0/2313888635 shutdown_connections 2024-09-19T08:47:41.822 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.822+0000 7f42fea56640 1 --2- 172.21.15.98:0/2313888635 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f42e4068400 0x7f42e406a8c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:41.822 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.822+0000 7f42fea56640 1 --2- 172.21.15.98:0/2313888635 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f42f815c290 0x7f42f80b0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:41.822 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.822+0000 7f42fea56640 1 --2- 172.21.15.98:0/2313888635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f42f80cf700 0x7f42f815df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:41.822 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.822+0000 7f42fea56640 1 --2- 172.21.15.98:0/2313888635 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f42f806d4c0 0x7f42f81588a0 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:41.822 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.822+0000 7f42fea56640 1 -- 172.21.15.98:0/2313888635 >> 172.21.15.98:0/2313888635 conn(0x7f42f806c090 msgr2=0x7f42f80b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:41.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.822+0000 7f42fea56640 1 -- 172.21.15.98:0/2313888635 shutdown_connections 2024-09-19T08:47:41.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.822+0000 7f42fea56640 1 -- 172.21.15.98:0/2313888635 wait complete. 2024-09-19T08:47:41.844 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T08:47:41.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.974+0000 7fcdf3203640 1 Processor -- start 2024-09-19T08:47:41.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.974+0000 7fcdf3203640 1 -- start start 2024-09-19T08:47:41.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.978+0000 7fcdf3203640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcdec144200 0x7fcdec1445e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:41.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.978+0000 7fcdf3203640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcdec071030 0x7fcdec06d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:41.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.978+0000 7fcdf3203640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcdec06da00 0x7fcdec0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:41.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.978+0000 7fcdf3203640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fcdec141940 con 0x7fcdec071030 2024-09-19T08:47:41.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.978+0000 7fcdf3203640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fcdec06de70 con 0x7fcdec144200 2024-09-19T08:47:41.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.978+0000 7fcdf3203640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcdec163060 con 0x7fcdec06da00 2024-09-19T08:47:41.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.978+0000 7fcde3fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcdec071030 0x7fcdec06d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:41.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.978+0000 7fcde3fff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fcdec071030 0x7fcdec06d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37362/0 (socket says 172.21.15.98:37362) 2024-09-19T08:47:41.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.978+0000 7fcde3fff640 1 -- 172.21.15.98:0/2296920345 learned_addr learned my addr 172.21.15.98:0/2296920345 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:41.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.978+0000 7fcdf17a0640 1 --2- 172.21.15.98:0/2296920345 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcdec06da00 0x7fcdec0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:41.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.978+0000 7fcdf0f9f640 1 --2- 172.21.15.98:0/2296920345 >> v2:172.21.15.139:3300/0 conn(0x7fcdec144200 0x7fcdec1445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:41.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.978+0000 7fcde3fff640 1 -- 172.21.15.98:0/2296920345 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcdec06da00 msgr2=0x7fcdec0ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:41.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.978+0000 7fcde3fff640 1 --2- 172.21.15.98:0/2296920345 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcdec06da00 0x7fcdec0ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:41.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.978+0000 7fcde3fff640 1 -- 172.21.15.98:0/2296920345 >> v2:172.21.15.139:3300/0 conn(0x7fcdec144200 msgr2=0x7fcdec1445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:41.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.978+0000 7fcde3fff640 1 --2- 172.21.15.98:0/2296920345 >> v2:172.21.15.139:3300/0 conn(0x7fcdec144200 0x7fcdec1445e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:41.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.978+0000 7fcde3fff640 1 -- 172.21.15.98:0/2296920345 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdec144b20 con 0x7fcdec071030 2024-09-19T08:47:41.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.978+0000 7fcdf0f9f640 1 --2- 172.21.15.98:0/2296920345 >> v2:172.21.15.139:3300/0 conn(0x7fcdec144200 0x7fcdec1445e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:41.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.978+0000 7fcdf17a0640 1 --2- 172.21.15.98:0/2296920345 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcdec06da00 0x7fcdec0ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:41.978 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.978+0000 7fcde3fff640 1 --2- 172.21.15.98:0/2296920345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdec071030 0x7fcdec06d4c0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fcdec06e080 tx=0x7fcddc01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=953ecd5a473f104b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:41.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.978+0000 7fcde37fe640 1 -- 172.21.15.98:0/2296920345 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc0b0070 con 0x7fcdec071030 2024-09-19T08:47:41.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.978+0000 7fcde37fe640 1 -- 172.21.15.98:0/2296920345 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcddc0ab600 con 0x7fcdec071030 2024-09-19T08:47:41.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.978+0000 7fcde37fe640 1 -- 172.21.15.98:0/2296920345 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc0ab9d0 con 0x7fcdec071030 2024-09-19T08:47:41.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.978+0000 7fcdf3203640 1 -- 172.21.15.98:0/2296920345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdec071030 msgr2=0x7fcdec06d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:41.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.978+0000 7fcdf3203640 1 --2- 172.21.15.98:0/2296920345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdec071030 0x7fcdec06d4c0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fcdec06e080 tx=0x7fcddc01c260 comp rx=0 tx=0).stop 2024-09-19T08:47:41.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.978+0000 7fcdf3203640 1 -- 172.21.15.98:0/2296920345 shutdown_connections 2024-09-19T08:47:41.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.978+0000 7fcdf3203640 1 --2- 172.21.15.98:0/2296920345 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdec071030 0x7fcdec06d4c0 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:41.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.978+0000 7fcdf3203640 1 --2- 172.21.15.98:0/2296920345 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcdec06da00 0x7fcdec0ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:41.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.978+0000 7fcdf3203640 1 --2- 172.21.15.98:0/2296920345 >> v2:172.21.15.139:3300/0 conn(0x7fcdec144200 0x7fcdec1445e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:41.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.978+0000 7fcdf3203640 1 -- 172.21.15.98:0/2296920345 >> 172.21.15.98:0/2296920345 conn(0x7fcdec06c090 msgr2=0x7fcdec06c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:41.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.978+0000 7fcdf3203640 1 -- 172.21.15.98:0/2296920345 shutdown_connections 2024-09-19T08:47:41.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.978+0000 7fcdf3203640 1 -- 172.21.15.98:0/2296920345 wait complete. 2024-09-19T08:47:41.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.982+0000 7fcdf3203640 1 Processor -- start 2024-09-19T08:47:41.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.982+0000 7fcdf3203640 1 -- start start 2024-09-19T08:47:41.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.982+0000 7fcdf3203640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcdec06da00 0x7fcdec156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:41.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.982+0000 7fcdf3203640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdec071030 0x7fcdec1585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:41.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.982+0000 7fcdf3203640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdec144200 0x7fcdec141ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:41.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.982+0000 7fcdf3203640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec072d10 con 0x7fcdec071030 2024-09-19T08:47:41.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.982+0000 7fcdf3203640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec162b40 con 0x7fcdec144200 2024-09-19T08:47:41.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.982+0000 7fcdf3203640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcdec072e90 con 0x7fcdec06da00 2024-09-19T08:47:41.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.982+0000 7fcde3fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdec071030 0x7fcdec1585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:41.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.982+0000 7fcde3fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdec071030 0x7fcdec1585a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37370/0 (socket says 172.21.15.98:37370) 2024-09-19T08:47:41.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.982+0000 7fcde3fff640 1 -- 172.21.15.98:0/2208457881 learned_addr learned my addr 172.21.15.98:0/2208457881 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:41.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.982+0000 7fcdf0f9f640 1 --2- 172.21.15.98:0/2208457881 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcdec06da00 0x7fcdec156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:41.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.982+0000 7fcdf17a0640 1 --2- 172.21.15.98:0/2208457881 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdec144200 0x7fcdec141ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:41.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.982+0000 7fcde3fff640 1 -- 172.21.15.98:0/2208457881 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcdec06da00 msgr2=0x7fcdec156a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:41.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.982+0000 7fcde3fff640 1 --2- 172.21.15.98:0/2208457881 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcdec06da00 0x7fcdec156a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:41.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.982+0000 7fcde3fff640 1 -- 172.21.15.98:0/2208457881 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdec144200 msgr2=0x7fcdec141ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:41.983 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.982+0000 7fcde3fff640 1 --2- 172.21.15.98:0/2208457881 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdec144200 0x7fcdec141ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:41.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.982+0000 7fcde3fff640 1 -- 172.21.15.98:0/2208457881 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdec15fb10 con 0x7fcdec071030 2024-09-19T08:47:41.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.982+0000 7fcdf17a0640 1 --2- 172.21.15.98:0/2208457881 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdec144200 0x7fcdec141ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:41.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.982+0000 7fcde3fff640 1 --2- 172.21.15.98:0/2208457881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdec071030 0x7fcdec1585a0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fcddc01a910 tx=0x7fcddc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:41.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.982+0000 7fcde17fa640 1 -- 172.21.15.98:0/2208457881 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc0b9070 con 0x7fcdec071030 2024-09-19T08:47:41.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.982+0000 7fcde17fa640 1 -- 172.21.15.98:0/2208457881 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcddc0b4600 con 0x7fcdec071030 2024-09-19T08:47:41.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.982+0000 7fcdf3203640 1 -- 172.21.15.98:0/2208457881 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdec155cd0 con 0x7fcdec071030 2024-09-19T08:47:41.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.982+0000 7fcde17fa640 1 -- 172.21.15.98:0/2208457881 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc0b0040 con 0x7fcdec071030 2024-09-19T08:47:41.984 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.982+0000 7fcdf3203640 1 -- 172.21.15.98:0/2208457881 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdec156290 con 0x7fcdec071030 2024-09-19T08:47:41.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.982+0000 7fcdf3203640 1 -- 172.21.15.98:0/2208457881 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdec071610 con 0x7fcdec071030 2024-09-19T08:47:41.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.986+0000 7fcde17fa640 1 -- 172.21.15.98:0/2208457881 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7fcddc0bc070 con 0x7fcdec071030 2024-09-19T08:47:41.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.986+0000 7fcde17fa640 1 --2- 172.21.15.98:0/2208457881 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcdd8068400 0x7fcdd806a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:41.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.986+0000 7fcde17fa640 1 -- 172.21.15.98:0/2208457881 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(172..172 src has 1..172) ==== 5385+0+0 (secure 0 0 0) 0x7fcddc11d9f0 con 0x7fcdec071030 2024-09-19T08:47:41.985 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.986+0000 7fcdf0f9f640 1 --2- 172.21.15.98:0/2208457881 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcdd8068400 0x7fcdd806a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:41.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.986+0000 7fcdf0f9f640 1 --2- 172.21.15.98:0/2208457881 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcdd8068400 0x7fcdd806a8c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fcde8006fd0 tx=0x7fcde8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:41.988 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:41.990+0000 7fcde17fa640 1 -- 172.21.15.98:0/2208457881 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fcddc016110 con 0x7fcdec071030 2024-09-19T08:47:42.149 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.150+0000 7fcdf3203640 1 -- 172.21.15.98:0/2208457881 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fcdec142840 con 0x7fcdec071030 2024-09-19T08:47:42.820 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.822+0000 7fcde17fa640 1 -- 172.21.15.98:0/2208457881 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v173) ==== 169+0+0 (secure 0 0 0) 0x7fcddc0efd90 con 0x7fcdec071030 2024-09-19T08:47:42.821 INFO:teuthology.orchestra.run.smithi098.stderr:pool 'cephfs_data' removed 2024-09-19T08:47:42.823 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.822+0000 7fcdf3203640 1 -- 172.21.15.98:0/2208457881 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcdd8068400 msgr2=0x7fcdd806a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:42.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.822+0000 7fcdf3203640 1 --2- 172.21.15.98:0/2208457881 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcdd8068400 0x7fcdd806a8c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fcde8006fd0 tx=0x7fcde8008040 comp rx=0 tx=0).stop 2024-09-19T08:47:42.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.822+0000 7fcdf3203640 1 -- 172.21.15.98:0/2208457881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdec071030 msgr2=0x7fcdec1585a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:42.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.822+0000 7fcdf3203640 1 --2- 172.21.15.98:0/2208457881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdec071030 0x7fcdec1585a0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fcddc01a910 tx=0x7fcddc019000 comp rx=0 tx=0).stop 2024-09-19T08:47:42.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.822+0000 7fcdf3203640 1 -- 172.21.15.98:0/2208457881 shutdown_connections 2024-09-19T08:47:42.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.822+0000 7fcdf3203640 1 --2- 172.21.15.98:0/2208457881 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcdd8068400 0x7fcdd806a8c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:42.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.822+0000 7fcdf3203640 1 --2- 172.21.15.98:0/2208457881 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcdec144200 0x7fcdec141ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:42.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.822+0000 7fcdf3203640 1 --2- 172.21.15.98:0/2208457881 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcdec071030 0x7fcdec1585a0 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:42.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.822+0000 7fcdf3203640 1 --2- 172.21.15.98:0/2208457881 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcdec06da00 0x7fcdec156a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:42.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.822+0000 7fcdf3203640 1 -- 172.21.15.98:0/2208457881 >> 172.21.15.98:0/2208457881 conn(0x7fcdec06c090 msgr2=0x7fcdec0b5da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:42.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.826+0000 7fcdf3203640 1 -- 172.21.15.98:0/2208457881 shutdown_connections 2024-09-19T08:47:42.824 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.826+0000 7fcdf3203640 1 -- 172.21.15.98:0/2208457881 wait complete. 2024-09-19T08:47:42.850 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-19T08:47:42.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.970+0000 7fec830a7640 1 Processor -- start 2024-09-19T08:47:42.971 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.970+0000 7fec830a7640 1 -- start start 2024-09-19T08:47:42.972 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.974+0000 7fec830a7640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fec7c147cb0 0x7fec7c06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:42.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.974+0000 7fec830a7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec7c06c1d0 0x7fec7c162200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:42.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.974+0000 7fec830a7640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec7c162f90 con 0x7fec7c162740 2024-09-19T08:47:42.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.974+0000 7fec830a7640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fec7c06c640 con 0x7fec7c147cb0 2024-09-19T08:47:42.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.974+0000 7fec830a7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fec7c0cee10 con 0x7fec7c06c1d0 2024-09-19T08:47:42.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.974+0000 7fec81644640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fec7c162740 0x7fec7c0ae8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:58826/0 (socket says 172.21.15.98:58826) 2024-09-19T08:47:42.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.974+0000 7fec81644640 1 -- 172.21.15.98:0/4000343085 learned_addr learned my addr 172.21.15.98:0/4000343085 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:42.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.974+0000 7fec80e43640 1 --2- 172.21.15.98:0/4000343085 >> v2:172.21.15.98:3300/0 conn(0x7fec7c147cb0 0x7fec7c06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:42.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.974+0000 7fec7bfff640 1 --2- 172.21.15.98:0/4000343085 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec7c06c1d0 0x7fec7c162200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:42.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.974+0000 7fec7b7fe640 1 -- 172.21.15.98:0/4000343085 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2721283088 0 0) 0x7fec7c162f90 con 0x7fec7c162740 2024-09-19T08:47:42.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.974+0000 7fec7b7fe640 1 -- 172.21.15.98:0/4000343085 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec68003530 con 0x7fec7c162740 2024-09-19T08:47:42.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.974+0000 7fec80e43640 1 -- 172.21.15.98:0/4000343085 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec7c06c1d0 msgr2=0x7fec7c162200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:42.973 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.974+0000 7fec80e43640 1 --2- 172.21.15.98:0/4000343085 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec7c06c1d0 0x7fec7c162200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:42.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.974+0000 7fec80e43640 1 -- 172.21.15.98:0/4000343085 >> v1:172.21.15.98:6789/0 conn(0x7fec7c162740 legacy=0x7fec7c0ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:42.974 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.974+0000 7fec80e43640 1 -- 172.21.15.98:0/4000343085 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec7c0cf040 con 0x7fec7c147cb0 2024-09-19T08:47:42.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.974+0000 7fec80e43640 1 --2- 172.21.15.98:0/4000343085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec7c147cb0 0x7fec7c06bc90 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7fec740097c0 tx=0x7fec7401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=decb5bea76d5480 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:42.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.974+0000 7fec7b7fe640 1 -- 172.21.15.98:0/4000343085 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec740b0070 con 0x7fec7c147cb0 2024-09-19T08:47:42.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.974+0000 7fec7b7fe640 1 -- 172.21.15.98:0/4000343085 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec740ab600 con 0x7fec7c147cb0 2024-09-19T08:47:42.975 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.974+0000 7fec7b7fe640 1 -- 172.21.15.98:0/4000343085 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec740ab9d0 con 0x7fec7c147cb0 2024-09-19T08:47:42.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.978+0000 7fec830a7640 1 -- 172.21.15.98:0/4000343085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec7c147cb0 msgr2=0x7fec7c06bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:42.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.978+0000 7fec830a7640 1 --2- 172.21.15.98:0/4000343085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec7c147cb0 0x7fec7c06bc90 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7fec740097c0 tx=0x7fec7401c410 comp rx=0 tx=0).stop 2024-09-19T08:47:42.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.978+0000 7fec830a7640 1 -- 172.21.15.98:0/4000343085 shutdown_connections 2024-09-19T08:47:42.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.978+0000 7fec830a7640 1 --2- 172.21.15.98:0/4000343085 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec7c06c1d0 0x7fec7c162200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:42.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.978+0000 7fec830a7640 1 --2- 172.21.15.98:0/4000343085 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec7c147cb0 0x7fec7c06bc90 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:42.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.978+0000 7fec830a7640 1 -- 172.21.15.98:0/4000343085 >> 172.21.15.98:0/4000343085 conn(0x7fec7c0b2010 msgr2=0x7fec7c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:42.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.978+0000 7fec830a7640 1 -- 172.21.15.98:0/4000343085 shutdown_connections 2024-09-19T08:47:42.976 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.978+0000 7fec830a7640 1 -- 172.21.15.98:0/4000343085 wait complete. 2024-09-19T08:47:42.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.978+0000 7fec830a7640 1 Processor -- start 2024-09-19T08:47:42.977 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.978+0000 7fec830a7640 1 -- start start 2024-09-19T08:47:42.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.978+0000 7fec830a7640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec7c06c1d0 0x7fec7c160310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:42.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.978+0000 7fec830a7640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec7c147cb0 0x7fec7c13afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:42.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.978+0000 7fec830a7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec7c162740 0x7fec7c15d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:42.979 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.978+0000 7fec830a7640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c162cd0 con 0x7fec7c147cb0 2024-09-19T08:47:42.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.978+0000 7fec830a7640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c162b50 con 0x7fec7c162740 2024-09-19T08:47:42.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.978+0000 7fec830a7640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fec7c0cfdc0 con 0x7fec7c06c1d0 2024-09-19T08:47:42.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.978+0000 7fec7bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec7c147cb0 0x7fec7c13afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:42.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.978+0000 7fec7bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec7c147cb0 0x7fec7c13afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37388/0 (socket says 172.21.15.98:37388) 2024-09-19T08:47:42.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.978+0000 7fec7bfff640 1 -- 172.21.15.98:0/2787473209 learned_addr learned my addr 172.21.15.98:0/2787473209 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:42.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.978+0000 7fec80e43640 1 --2- 172.21.15.98:0/2787473209 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec7c06c1d0 0x7fec7c160310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:42.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.978+0000 7fec81644640 1 --2- 172.21.15.98:0/2787473209 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec7c162740 0x7fec7c15d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:42.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.978+0000 7fec7bfff640 1 -- 172.21.15.98:0/2787473209 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec7c06c1d0 msgr2=0x7fec7c160310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:42.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.978+0000 7fec7bfff640 1 --2- 172.21.15.98:0/2787473209 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec7c06c1d0 0x7fec7c160310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:42.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.978+0000 7fec7bfff640 1 -- 172.21.15.98:0/2787473209 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec7c162740 msgr2=0x7fec7c15d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:42.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.978+0000 7fec7bfff640 1 --2- 172.21.15.98:0/2787473209 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec7c162740 0x7fec7c15d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:42.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.978+0000 7fec7bfff640 1 -- 172.21.15.98:0/2787473209 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec7c13b7f0 con 0x7fec7c147cb0 2024-09-19T08:47:42.980 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.978+0000 7fec81644640 1 --2- 172.21.15.98:0/2787473209 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec7c162740 0x7fec7c15d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:42.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.978+0000 7fec7bfff640 1 --2- 172.21.15.98:0/2787473209 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec7c147cb0 0x7fec7c13afc0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fec64009d30 tx=0x7fec6400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:42.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.978+0000 7fec797fa640 1 -- 172.21.15.98:0/2787473209 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec64018020 con 0x7fec7c147cb0 2024-09-19T08:47:42.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.978+0000 7fec797fa640 1 -- 172.21.15.98:0/2787473209 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec64014480 con 0x7fec7c147cb0 2024-09-19T08:47:42.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.978+0000 7fec830a7640 1 -- 172.21.15.98:0/2787473209 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec7c13bad0 con 0x7fec7c147cb0 2024-09-19T08:47:42.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.978+0000 7fec797fa640 1 -- 172.21.15.98:0/2787473209 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec64014870 con 0x7fec7c147cb0 2024-09-19T08:47:42.981 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.978+0000 7fec830a7640 1 -- 172.21.15.98:0/2787473209 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec7c13c090 con 0x7fec7c147cb0 2024-09-19T08:47:42.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.982+0000 7fec830a7640 1 -- 172.21.15.98:0/2787473209 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec7c162200 con 0x7fec7c147cb0 2024-09-19T08:47:42.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.982+0000 7fec797fa640 1 -- 172.21.15.98:0/2787473209 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7fec6401b020 con 0x7fec7c147cb0 2024-09-19T08:47:42.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.982+0000 7fec797fa640 1 --2- 172.21.15.98:0/2787473209 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fec5c068400 0x7fec5c06a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:42.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.982+0000 7fec797fa640 1 -- 172.21.15.98:0/2787473209 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7fec640859e0 con 0x7fec7c147cb0 2024-09-19T08:47:42.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.982+0000 7fec80e43640 1 --2- 172.21.15.98:0/2787473209 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fec5c068400 0x7fec5c06a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:42.982 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.982+0000 7fec80e43640 1 --2- 172.21.15.98:0/2787473209 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fec5c068400 0x7fec5c06a8c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fec740ad000 tx=0x7fec74008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:42.986 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:42.986+0000 7fec797fa640 1 -- 172.21.15.98:0/2787473209 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fec64022030 con 0x7fec7c147cb0 2024-09-19T08:47:43.150 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.150+0000 7fec830a7640 1 -- 172.21.15.98:0/2787473209 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fec7c0b05a0 con 0x7fec7c147cb0 2024-09-19T08:47:43.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.154+0000 7fec797fa640 1 -- 172.21.15.98:0/2787473209 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v31) ==== 104+0+0 (secure 0 0 0) 0x7fec64010040 con 0x7fec7c147cb0 2024-09-19T08:47:43.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.158+0000 7fec830a7640 1 -- 172.21.15.98:0/2787473209 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fec5c068400 msgr2=0x7fec5c06a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:43.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.158+0000 7fec830a7640 1 --2- 172.21.15.98:0/2787473209 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fec5c068400 0x7fec5c06a8c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fec740ad000 tx=0x7fec74008040 comp rx=0 tx=0).stop 2024-09-19T08:47:43.156 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.158+0000 7fec830a7640 1 -- 172.21.15.98:0/2787473209 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec7c147cb0 msgr2=0x7fec7c13afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:43.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.158+0000 7fec830a7640 1 --2- 172.21.15.98:0/2787473209 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec7c147cb0 0x7fec7c13afc0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fec64009d30 tx=0x7fec6400c490 comp rx=0 tx=0).stop 2024-09-19T08:47:43.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.158+0000 7fec830a7640 1 -- 172.21.15.98:0/2787473209 shutdown_connections 2024-09-19T08:47:43.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.158+0000 7fec830a7640 1 --2- 172.21.15.98:0/2787473209 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fec5c068400 0x7fec5c06a8c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:43.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.158+0000 7fec830a7640 1 --2- 172.21.15.98:0/2787473209 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec7c162740 0x7fec7c15d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:43.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.158+0000 7fec830a7640 1 --2- 172.21.15.98:0/2787473209 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec7c147cb0 0x7fec7c13afc0 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:43.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.158+0000 7fec830a7640 1 --2- 172.21.15.98:0/2787473209 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec7c06c1d0 0x7fec7c160310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:43.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.158+0000 7fec830a7640 1 -- 172.21.15.98:0/2787473209 >> 172.21.15.98:0/2787473209 conn(0x7fec7c0b2010 msgr2=0x7fec7c145d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:43.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.158+0000 7fec830a7640 1 -- 172.21.15.98:0/2787473209 shutdown_connections 2024-09-19T08:47:43.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.158+0000 7fec830a7640 1 -- 172.21.15.98:0/2787473209 wait complete. 2024-09-19T08:47:43.174 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade' 2024-09-19T08:47:43.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.298+0000 7f0b1aeee640 1 Processor -- start 2024-09-19T08:47:43.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.298+0000 7f0b1aeee640 1 -- start start 2024-09-19T08:47:43.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.298+0000 7f0b1aeee640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0b14158fd0 0x7f0b140b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:43.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.298+0000 7f0b1aeee640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0b1406c3a0 0x7f0b140ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:43.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.298+0000 7f0b1aeee640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b140ccc60 con 0x7f0b140b4360 2024-09-19T08:47:43.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.298+0000 7f0b1aeee640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f0b141593b0 con 0x7f0b1406c3a0 2024-09-19T08:47:43.297 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.298+0000 7f0b1aeee640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0b140ccf40 con 0x7f0b14158fd0 2024-09-19T08:47:43.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.298+0000 7f0b18c8a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0b14158fd0 0x7f0b140b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:43.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.298+0000 7f0b18c8a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0b14158fd0 0x7f0b140b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:33242/0 (socket says 172.21.15.98:33242) 2024-09-19T08:47:43.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.298+0000 7f0b18c8a640 1 -- 172.21.15.98:0/2409372075 learned_addr learned my addr 172.21.15.98:0/2409372075 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:43.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.298+0000 7f0b1948b640 1 --2- 172.21.15.98:0/2409372075 >> v2:172.21.15.139:3300/0 conn(0x7f0b1406c3a0 0x7f0b140ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:43.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.298+0000 7f0b18c8a640 1 -- 172.21.15.98:0/2409372075 >> v2:172.21.15.139:3300/0 conn(0x7f0b1406c3a0 msgr2=0x7f0b140ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:43.298 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.298+0000 7f0b18c8a640 1 --2- 172.21.15.98:0/2409372075 >> v2:172.21.15.139:3300/0 conn(0x7f0b1406c3a0 0x7f0b140ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:43.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.298+0000 7f0b18c8a640 1 -- 172.21.15.98:0/2409372075 >> v1:172.21.15.139:6789/0 conn(0x7f0b140b4360 legacy=0x7f0b1406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:43.299 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.298+0000 7f0b18c8a640 1 -- 172.21.15.98:0/2409372075 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b140cd220 con 0x7f0b14158fd0 2024-09-19T08:47:43.300 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.302+0000 7f0b18c8a640 1 --2- 172.21.15.98:0/2409372075 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0b14158fd0 0x7f0b140b3e20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0b0400bb10 tx=0x7f0b0401eb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84cff5dc4eeef95e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:43.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.302+0000 7f0b137fe640 1 -- 172.21.15.98:0/2409372075 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b040b4070 con 0x7f0b14158fd0 2024-09-19T08:47:43.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.302+0000 7f0b137fe640 1 -- 172.21.15.98:0/2409372075 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b040a7c40 con 0x7f0b14158fd0 2024-09-19T08:47:43.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.302+0000 7f0b137fe640 1 -- 172.21.15.98:0/2409372075 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b040af440 con 0x7f0b14158fd0 2024-09-19T08:47:43.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.302+0000 7f0b1aeee640 1 -- 172.21.15.98:0/2409372075 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0b14158fd0 msgr2=0x7f0b140b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:43.301 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.302+0000 7f0b1aeee640 1 --2- 172.21.15.98:0/2409372075 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0b14158fd0 0x7f0b140b3e20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0b0400bb10 tx=0x7f0b0401eb70 comp rx=0 tx=0).stop 2024-09-19T08:47:43.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.302+0000 7f0b1aeee640 1 -- 172.21.15.98:0/2409372075 shutdown_connections 2024-09-19T08:47:43.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.302+0000 7f0b1aeee640 1 --2- 172.21.15.98:0/2409372075 >> v2:172.21.15.139:3300/0 conn(0x7f0b1406c3a0 0x7f0b140ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:43.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.302+0000 7f0b1aeee640 1 --2- 172.21.15.98:0/2409372075 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0b14158fd0 0x7f0b140b3e20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:43.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.302+0000 7f0b1aeee640 1 -- 172.21.15.98:0/2409372075 >> 172.21.15.98:0/2409372075 conn(0x7f0b140b2010 msgr2=0x7f0b140b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:43.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.302+0000 7f0b1aeee640 1 -- 172.21.15.98:0/2409372075 shutdown_connections 2024-09-19T08:47:43.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.302+0000 7f0b1aeee640 1 -- 172.21.15.98:0/2409372075 wait complete. 2024-09-19T08:47:43.302 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.302+0000 7f0b1aeee640 1 Processor -- start 2024-09-19T08:47:43.303 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.302+0000 7f0b1aeee640 1 -- start start 2024-09-19T08:47:43.304 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.302+0000 7f0b1aeee640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0b1406c3a0 0x7f0b1415c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:43.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.302+0000 7f0b1aeee640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b140b4360 0x7f0b14160140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:43.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.302+0000 7f0b1aeee640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b14158fd0 0x7f0b14160680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:43.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.302+0000 7f0b1aeee640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0b140cf710 con 0x7f0b14158fd0 2024-09-19T08:47:43.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.302+0000 7f0b1aeee640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0b140cf590 con 0x7f0b140b4360 2024-09-19T08:47:43.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.302+0000 7f0b1aeee640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0b140cf890 con 0x7f0b1406c3a0 2024-09-19T08:47:43.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.306+0000 7f0b1948b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b14158fd0 0x7f0b14160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:43.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.306+0000 7f0b1948b640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b14158fd0 0x7f0b14160680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37396/0 (socket says 172.21.15.98:37396) 2024-09-19T08:47:43.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.306+0000 7f0b1948b640 1 -- 172.21.15.98:0/992218074 learned_addr learned my addr 172.21.15.98:0/992218074 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:43.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.306+0000 7f0b18c8a640 1 --2- 172.21.15.98:0/992218074 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0b1406c3a0 0x7f0b1415c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:43.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.306+0000 7f0b13fff640 1 --2- 172.21.15.98:0/992218074 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b140b4360 0x7f0b14160140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:43.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.306+0000 7f0b1948b640 1 -- 172.21.15.98:0/992218074 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0b1406c3a0 msgr2=0x7f0b1415c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:43.305 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.306+0000 7f0b1948b640 1 --2- 172.21.15.98:0/992218074 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0b1406c3a0 0x7f0b1415c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:43.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.306+0000 7f0b1948b640 1 -- 172.21.15.98:0/992218074 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b140b4360 msgr2=0x7f0b14160140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:43.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.306+0000 7f0b1948b640 1 --2- 172.21.15.98:0/992218074 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b140b4360 0x7f0b14160140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:43.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.306+0000 7f0b1948b640 1 -- 172.21.15.98:0/992218074 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b1415feb0 con 0x7f0b14158fd0 2024-09-19T08:47:43.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.306+0000 7f0b1948b640 1 --2- 172.21.15.98:0/992218074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b14158fd0 0x7f0b14160680 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c00efc0 tx=0x7f0b0c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:43.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.306+0000 7f0b117fa640 1 -- 172.21.15.98:0/992218074 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c018020 con 0x7f0b14158fd0 2024-09-19T08:47:43.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.306+0000 7f0b117fa640 1 -- 172.21.15.98:0/992218074 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b0c014480 con 0x7f0b14158fd0 2024-09-19T08:47:43.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.306+0000 7f0b1aeee640 1 -- 172.21.15.98:0/992218074 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b14160bc0 con 0x7f0b14158fd0 2024-09-19T08:47:43.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.306+0000 7f0b117fa640 1 -- 172.21.15.98:0/992218074 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c014870 con 0x7f0b14158fd0 2024-09-19T08:47:43.306 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.306+0000 7f0b1aeee640 1 -- 172.21.15.98:0/992218074 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b14161150 con 0x7f0b14158fd0 2024-09-19T08:47:43.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.306+0000 7f0b1aeee640 1 -- 172.21.15.98:0/992218074 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b14164980 con 0x7f0b14158fd0 2024-09-19T08:47:43.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.306+0000 7f0b117fa640 1 -- 172.21.15.98:0/992218074 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f0b0c01b020 con 0x7f0b14158fd0 2024-09-19T08:47:43.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.306+0000 7f0b117fa640 1 --2- 172.21.15.98:0/992218074 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0b00068400 0x7f0b0006a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:43.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.306+0000 7f0b117fa640 1 -- 172.21.15.98:0/992218074 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7f0b0c085040 con 0x7f0b14158fd0 2024-09-19T08:47:43.307 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.306+0000 7f0b18c8a640 1 --2- 172.21.15.98:0/992218074 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0b00068400 0x7f0b0006a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:43.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.306+0000 7f0b18c8a640 1 --2- 172.21.15.98:0/992218074 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0b00068400 0x7f0b0006a8c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0b040b1000 tx=0x7f0b040a9070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:43.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.310+0000 7f0b117fa640 1 -- 172.21.15.98:0/992218074 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f0b0c022070 con 0x7f0b14158fd0 2024-09-19T08:47:43.527 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.526+0000 7f0b1aeee640 1 -- 172.21.15.98:0/992218074 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7f0b140d0670 con 0x7f0b14158fd0 2024-09-19T08:47:43.828 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.830+0000 7f0b117fa640 1 -- 172.21.15.98:0/992218074 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]}]=0 v493) ==== 163+0+0 (secure 0 0 0) 0x7f0b0c010040 con 0x7f0b14158fd0 2024-09-19T08:47:43.830 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.830+0000 7f0b1aeee640 1 -- 172.21.15.98:0/992218074 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0b00068400 msgr2=0x7f0b0006a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:43.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.830+0000 7f0b1aeee640 1 --2- 172.21.15.98:0/992218074 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0b00068400 0x7f0b0006a8c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0b040b1000 tx=0x7f0b040a9070 comp rx=0 tx=0).stop 2024-09-19T08:47:43.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.830+0000 7f0b1aeee640 1 -- 172.21.15.98:0/992218074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b14158fd0 msgr2=0x7f0b14160680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:43.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.830+0000 7f0b1aeee640 1 --2- 172.21.15.98:0/992218074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b14158fd0 0x7f0b14160680 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c00efc0 tx=0x7f0b0c00c490 comp rx=0 tx=0).stop 2024-09-19T08:47:43.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.830+0000 7f0b1aeee640 1 -- 172.21.15.98:0/992218074 shutdown_connections 2024-09-19T08:47:43.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.830+0000 7f0b1aeee640 1 --2- 172.21.15.98:0/992218074 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0b00068400 0x7f0b0006a8c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:43.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.830+0000 7f0b1aeee640 1 --2- 172.21.15.98:0/992218074 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0b14158fd0 0x7f0b14160680 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:43.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.830+0000 7f0b1aeee640 1 --2- 172.21.15.98:0/992218074 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b140b4360 0x7f0b14160140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:43.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.830+0000 7f0b1aeee640 1 --2- 172.21.15.98:0/992218074 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0b1406c3a0 0x7f0b1415c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:43.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.830+0000 7f0b1aeee640 1 -- 172.21.15.98:0/992218074 >> 172.21.15.98:0/992218074 conn(0x7f0b140b2010 msgr2=0x7f0b14071220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:43.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.830+0000 7f0b1aeee640 1 -- 172.21.15.98:0/992218074 shutdown_connections 2024-09-19T08:47:43.831 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:43.830+0000 7f0b1aeee640 1 -- 172.21.15.98:0/992218074 wait complete. 2024-09-19T08:47:43.851 INFO:tasks.cephfs_test_runner:Validate unparseable text .meta file on legacy subvol root doesn't break the system ... ok 2024-09-19T08:47:43.851 INFO:tasks.cephfs_test_runner: 2024-09-19T08:47:43.851 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-19T08:47:43.852 INFO:tasks.cephfs_test_runner:Ran 11 tests in 536.444s 2024-09-19T08:47:43.852 INFO:tasks.cephfs_test_runner: 2024-09-19T08:47:43.852 INFO:tasks.cephfs_test_runner:OK 2024-09-19T08:47:43.852 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-19T08:47:43.864 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-19T08:47:43.880 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-19T08:47:43.880 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:47:43.880 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:47:43.885 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15846 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T08:47:43.886 DEBUG:teuthology.orchestra.run.smithi098:> set -ex 2024-09-19T08:47:43.886 DEBUG:teuthology.orchestra.run.smithi098:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:47:43.933 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15846 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T08:47:43.934 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:47:43.934 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:47:43.940 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T08:47:43.941 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-09-19T08:47:43.941 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T08:47:43.986 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22673 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T08:47:43.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:43.987 DEBUG:teuthology.orchestra.run.smithi139:> ip netns list 2024-09-19T08:47:44.035 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-19T08:47:44.035 INFO:teuthology.orchestra.run.smithi139.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-19T08:47:44.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:44.035 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T08:47:44.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:44.091 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T08:47:44.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:44.146 DEBUG:teuthology.orchestra.run.smithi139:> sudo ip link delete ceph-brx 2024-09-19T08:47:44.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:44.331 DEBUG:teuthology.orchestra.run.smithi098:> ip netns list 2024-09-19T08:47:44.337 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-19T08:47:44.338 INFO:teuthology.orchestra.run.smithi098.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T08:47:44.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:44.338 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T08:47:44.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:44.394 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T08:47:44.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T08:47:44.449 DEBUG:teuthology.orchestra.run.smithi098:> sudo ip link delete ceph-brx 2024-09-19T08:47:44.613 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-19T08:47:44.623 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T08:47:44.623 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:47:44.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.750+0000 7fcc97f10640 1 Processor -- start 2024-09-19T08:47:44.751 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.750+0000 7fcc97f10640 1 -- start start 2024-09-19T08:47:44.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.754+0000 7fcc97f10640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcc900d01b0 0x7fcc90071c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:44.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.754+0000 7fcc97f10640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc90163d90 0x7fcc90166180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:44.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.754+0000 7fcc97f10640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc9006ff30 con 0x7fcc9015cc10 2024-09-19T08:47:44.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.754+0000 7fcc97f10640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fcc9015d3a0 con 0x7fcc900d01b0 2024-09-19T08:47:44.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.754+0000 7fcc97f10640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcc90073080 con 0x7fcc90163d90 2024-09-19T08:47:44.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.754+0000 7fcc964ad640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc90163d90 0x7fcc90166180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:44.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.754+0000 7fcc954ab640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fcc900d01b0 0x7fcc90071c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:44.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.754+0000 7fcc964ad640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc90163d90 0x7fcc90166180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:33258/0 (socket says 172.21.15.98:33258) 2024-09-19T08:47:44.752 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.754+0000 7fcc964ad640 1 -- 172.21.15.98:0/534659637 learned_addr learned my addr 172.21.15.98:0/534659637 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:44.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.754+0000 7fcc954ab640 1 -- 172.21.15.98:0/534659637 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc90163d90 msgr2=0x7fcc90166180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:44.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.754+0000 7fcc954ab640 1 --2- 172.21.15.98:0/534659637 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc90163d90 0x7fcc90166180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:44.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.754+0000 7fcc954ab640 1 -- 172.21.15.98:0/534659637 >> v1:172.21.15.139:6789/0 conn(0x7fcc9015cc10 legacy=0x7fcc90071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:44.753 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.754+0000 7fcc954ab640 1 -- 172.21.15.98:0/534659637 --> v2:172.21.15.139:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc90073360 con 0x7fcc900d01b0 2024-09-19T08:47:44.754 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.754+0000 7fcc954ab640 1 --2- 172.21.15.98:0/534659637 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc900d01b0 0x7fcc90071c30 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fcc840097c0 tx=0x7fcc8401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b7a064d42808d46 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:44.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.754+0000 7fcc94caa640 1 -- 172.21.15.98:0/534659637 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc840ab600 con 0x7fcc900d01b0 2024-09-19T08:47:44.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.754+0000 7fcc94caa640 1 -- 172.21.15.98:0/534659637 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc840b0070 con 0x7fcc900d01b0 2024-09-19T08:47:44.755 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.754+0000 7fcc94caa640 1 -- 172.21.15.98:0/534659637 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc840ab9c0 con 0x7fcc900d01b0 2024-09-19T08:47:44.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.758+0000 7fcc97f10640 1 -- 172.21.15.98:0/534659637 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc900d01b0 msgr2=0x7fcc90071c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:44.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.758+0000 7fcc97f10640 1 --2- 172.21.15.98:0/534659637 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc900d01b0 0x7fcc90071c30 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fcc840097c0 tx=0x7fcc8401c470 comp rx=0 tx=0).stop 2024-09-19T08:47:44.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.758+0000 7fcc97f10640 1 -- 172.21.15.98:0/534659637 shutdown_connections 2024-09-19T08:47:44.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.758+0000 7fcc97f10640 1 --2- 172.21.15.98:0/534659637 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc90163d90 0x7fcc90166180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:44.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.758+0000 7fcc97f10640 1 --2- 172.21.15.98:0/534659637 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc900d01b0 0x7fcc90071c30 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:44.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.758+0000 7fcc97f10640 1 -- 172.21.15.98:0/534659637 >> 172.21.15.98:0/534659637 conn(0x7fcc900d0980 msgr2=0x7fcc900d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:44.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.758+0000 7fcc97f10640 1 -- 172.21.15.98:0/534659637 shutdown_connections 2024-09-19T08:47:44.756 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.758+0000 7fcc97f10640 1 -- 172.21.15.98:0/534659637 wait complete. 2024-09-19T08:47:44.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.758+0000 7fcc97f10640 1 Processor -- start 2024-09-19T08:47:44.757 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.758+0000 7fcc97f10640 1 -- start start 2024-09-19T08:47:44.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.758+0000 7fcc97f10640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc900d01b0 0x7fcc90145d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:44.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.758+0000 7fcc97f10640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc9015cc10 0x7fcc90154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:44.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.758+0000 7fcc97f10640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc90163d90 0x7fcc901547a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:44.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.758+0000 7fcc97f10640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9015cff0 con 0x7fcc9015cc10 2024-09-19T08:47:44.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.758+0000 7fcc97f10640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcc900ce550 con 0x7fcc90163d90 2024-09-19T08:47:44.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.758+0000 7fcc97f10640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fcc900cb680 con 0x7fcc900d01b0 2024-09-19T08:47:44.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.758+0000 7fcc954ab640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc9015cc10 0x7fcc90154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:44.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.758+0000 7fcc954ab640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc9015cc10 0x7fcc90154260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37404/0 (socket says 172.21.15.98:37404) 2024-09-19T08:47:44.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.758+0000 7fcc954ab640 1 -- 172.21.15.98:0/1602951754 learned_addr learned my addr 172.21.15.98:0/1602951754 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:44.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.758+0000 7fcc964ad640 1 --2- 172.21.15.98:0/1602951754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc90163d90 0x7fcc901547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:44.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.758+0000 7fcc95cac640 1 --2- 172.21.15.98:0/1602951754 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc900d01b0 0x7fcc90145d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:44.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.758+0000 7fcc954ab640 1 -- 172.21.15.98:0/1602951754 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc900d01b0 msgr2=0x7fcc90145d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:44.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.758+0000 7fcc954ab640 1 --2- 172.21.15.98:0/1602951754 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc900d01b0 0x7fcc90145d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:44.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.758+0000 7fcc954ab640 1 -- 172.21.15.98:0/1602951754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc90163d90 msgr2=0x7fcc901547a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:44.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.758+0000 7fcc954ab640 1 --2- 172.21.15.98:0/1602951754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc90163d90 0x7fcc901547a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:44.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.758+0000 7fcc954ab640 1 -- 172.21.15.98:0/1602951754 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc901462a0 con 0x7fcc9015cc10 2024-09-19T08:47:44.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.758+0000 7fcc954ab640 1 --2- 172.21.15.98:0/1602951754 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc9015cc10 0x7fcc90154260 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fcc8401a0a0 tx=0x7fcc8401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:44.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.758+0000 7fcc8e7fc640 1 -- 172.21.15.98:0/1602951754 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc840bf610 con 0x7fcc9015cc10 2024-09-19T08:47:44.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.758+0000 7fcc8e7fc640 1 -- 172.21.15.98:0/1602951754 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc840bf7b0 con 0x7fcc9015cc10 2024-09-19T08:47:44.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.758+0000 7fcc8e7fc640 1 -- 172.21.15.98:0/1602951754 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc840bfba0 con 0x7fcc9015cc10 2024-09-19T08:47:44.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.758+0000 7fcc97f10640 1 -- 172.21.15.98:0/1602951754 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc900cb070 con 0x7fcc9015cc10 2024-09-19T08:47:44.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.758+0000 7fcc97f10640 1 -- 172.21.15.98:0/1602951754 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc900cb310 con 0x7fcc9015cc10 2024-09-19T08:47:44.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.762+0000 7fcc97f10640 1 -- 172.21.15.98:0/1602951754 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc90071520 con 0x7fcc9015cc10 2024-09-19T08:47:44.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.762+0000 7fcc8e7fc640 1 -- 172.21.15.98:0/1602951754 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7fcc840bd020 con 0x7fcc9015cc10 2024-09-19T08:47:44.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.762+0000 7fcc8e7fc640 1 --2- 172.21.15.98:0/1602951754 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcc7c068430 0x7fcc7c06a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:44.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.762+0000 7fcc8e7fc640 1 -- 172.21.15.98:0/1602951754 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7fcc8411e6b0 con 0x7fcc9015cc10 2024-09-19T08:47:44.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.762+0000 7fcc95cac640 1 --2- 172.21.15.98:0/1602951754 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcc7c068430 0x7fcc7c06a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:44.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.762+0000 7fcc95cac640 1 --2- 172.21.15.98:0/1602951754 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcc7c068430 0x7fcc7c06a8f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fcc800012f0 tx=0x7fcc80009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:44.765 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.766+0000 7fcc8e7fc640 1 -- 172.21.15.98:0/1602951754 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fcc840c6030 con 0x7fcc9015cc10 2024-09-19T08:47:44.924 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.926+0000 7fcc97f10640 1 -- 172.21.15.98:0/1602951754 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcc90072220 con 0x7fcc7c068430 2024-09-19T08:47:44.926 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.926+0000 7fcc8e7fc640 1 -- 172.21.15.98:0/1602951754 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13523 (secure 0 0 0) 0x7fcc90072220 con 0x7fcc7c068430 2024-09-19T08:47:44.926 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:47:44.926 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:47:44.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.930+0000 7fcc97f10640 1 -- 172.21.15.98:0/1602951754 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcc7c068430 msgr2=0x7fcc7c06a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:44.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.930+0000 7fcc97f10640 1 --2- 172.21.15.98:0/1602951754 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcc7c068430 0x7fcc7c06a8f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fcc800012f0 tx=0x7fcc80009040 comp rx=0 tx=0).stop 2024-09-19T08:47:44.928 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.930+0000 7fcc97f10640 1 -- 172.21.15.98:0/1602951754 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc9015cc10 msgr2=0x7fcc90154260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:44.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.930+0000 7fcc97f10640 1 --2- 172.21.15.98:0/1602951754 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc9015cc10 0x7fcc90154260 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fcc8401a0a0 tx=0x7fcc8401acb0 comp rx=0 tx=0).stop 2024-09-19T08:47:44.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.930+0000 7fcc97f10640 1 -- 172.21.15.98:0/1602951754 shutdown_connections 2024-09-19T08:47:44.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.930+0000 7fcc97f10640 1 --2- 172.21.15.98:0/1602951754 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fcc7c068430 0x7fcc7c06a8f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:44.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.930+0000 7fcc97f10640 1 --2- 172.21.15.98:0/1602951754 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcc90163d90 0x7fcc901547a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:44.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.930+0000 7fcc97f10640 1 --2- 172.21.15.98:0/1602951754 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fcc9015cc10 0x7fcc90154260 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:44.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.930+0000 7fcc97f10640 1 --2- 172.21.15.98:0/1602951754 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fcc900d01b0 0x7fcc90145d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:44.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.930+0000 7fcc97f10640 1 -- 172.21.15.98:0/1602951754 >> 172.21.15.98:0/1602951754 conn(0x7fcc900d0980 msgr2=0x7fcc901562b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:44.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.930+0000 7fcc97f10640 1 -- 172.21.15.98:0/1602951754 shutdown_connections 2024-09-19T08:47:44.929 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:44.930+0000 7fcc97f10640 1 -- 172.21.15.98:0/1602951754 wait complete. 2024-09-19T08:47:44.945 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2024-09-19T08:47:42.967947+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":275828,"kb_used_data":18036,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754698892,"statfs":{"total":773094113280,"available":772811665408,"internally_reserved":0,"allocated":18468864,"data_stored":8894354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":58,"apply_latency_ms":58,"commit_latency_ns":58000000,"apply_latency_ns":58000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-30426,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-62,"num_write_kb":-51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002277"},"pg_stats":[{"pgid":"1.0","version":"71'128","reported_seq":478,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-19T08:47:15.383426+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:47:15.383426+0000","last_peered":"2024-09-19T08:47:15.383426+0000","last_clean":"2024-09-19T08:47:15.383426+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:47:15.383426+0000","last_undegraded":"2024-09-19T08:47:15.383426+0000","last_fullsized":"2024-09-19T08:47:15.383426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509557,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33912,"kb_used_data":2728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337928,"statfs":{"total":96636764160,"available":96602038272,"internally_reserved":0,"allocated":2793472,"data_stored":1593571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574966,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33280,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338560,"statfs":{"total":96636764160,"available":96602685440,"internally_reserved":0,"allocated":2146304,"data_stored":951636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509559,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33276,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338564,"statfs":{"total":96636764160,"available":96602689536,"internally_reserved":0,"allocated":2142208,"data_stored":951434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509559,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33344,"kb_used_data":2160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338496,"statfs":{"total":96636764160,"available":96602619904,"internally_reserved":0,"allocated":2211840,"data_stored":1003509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542261,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37444,"kb_used_data":2100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334396,"statfs":{"total":96636764160,"available":96598421504,"internally_reserved":0,"allocated":2150400,"data_stored":951034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509556,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33272,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338568,"statfs":{"total":96636764160,"available":96602693632,"internally_reserved":0,"allocated":2138112,"data_stored":950852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509558,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33284,"kb_used_data":2100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338556,"statfs":{"total":96636764160,"available":96602681344,"internally_reserved":0,"allocated":2150400,"data_stored":951008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509556,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38016,"kb_used_data":2672,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333824,"statfs":{"total":96636764160,"available":96597835776,"internally_reserved":0,"allocated":2736128,"data_stored":1541310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:47:44.946 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:47:45.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.070+0000 7f013bcc5640 1 Processor -- start 2024-09-19T08:47:45.071 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.070+0000 7f013bcc5640 1 -- start start 2024-09-19T08:47:45.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.074+0000 7f013bcc5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f013406bc90 0x7f013406c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:45.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.074+0000 7f013bcc5640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f01340ce840 0x7f01340cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:45.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.074+0000 7f013bcc5640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01340ccc60 con 0x7f0134158fd0 2024-09-19T08:47:45.073 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.074+0000 7f013bcc5640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f01341593b0 con 0x7f01340ce840 2024-09-19T08:47:45.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.074+0000 7f013bcc5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f01340ccf40 con 0x7f013406bc90 2024-09-19T08:47:45.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.074+0000 7f013a262640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f01340ce840 0x7f01340cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:45.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.074+0000 7f013a262640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f01340ce840 0x7f01340cec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37412/0 (socket says 172.21.15.98:37412) 2024-09-19T08:47:45.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.074+0000 7f013a262640 1 -- 172.21.15.98:0/1858902321 learned_addr learned my addr 172.21.15.98:0/1858902321 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:45.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.074+0000 7f0139a61640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f0134158fd0 0x7f01340b3e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:58840/0 (socket says 172.21.15.98:58840) 2024-09-19T08:47:45.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.074+0000 7f0139260640 1 --2- 172.21.15.98:0/1858902321 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f013406bc90 0x7f013406c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:45.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.074+0000 7f013a262640 1 -- 172.21.15.98:0/1858902321 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f013406bc90 msgr2=0x7f013406c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:45.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.074+0000 7f013a262640 1 --2- 172.21.15.98:0/1858902321 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f013406bc90 0x7f013406c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:45.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.074+0000 7f013a262640 1 -- 172.21.15.98:0/1858902321 >> v1:172.21.15.98:6789/0 conn(0x7f0134158fd0 legacy=0x7f01340b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:45.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.074+0000 7f0138a5f640 1 -- 172.21.15.98:0/1858902321 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 811115093 0 0) 0x7f01340ccc60 con 0x7f0134158fd0 2024-09-19T08:47:45.074 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.074+0000 7f013a262640 1 -- 172.21.15.98:0/1858902321 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01340cd220 con 0x7f01340ce840 2024-09-19T08:47:45.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.074+0000 7f013a262640 1 --2- 172.21.15.98:0/1858902321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01340ce840 0x7f01340cec90 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f012c009e50 tx=0x7f012c01c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d70ad1bd32780659 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:45.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.074+0000 7f0138a5f640 1 -- 172.21.15.98:0/1858902321 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c0ac020 con 0x7f01340ce840 2024-09-19T08:47:45.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.074+0000 7f0138a5f640 1 -- 172.21.15.98:0/1858902321 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f012c0a64a0 con 0x7f01340ce840 2024-09-19T08:47:45.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.074+0000 7f0138a5f640 1 -- 172.21.15.98:0/1858902321 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012c0a6870 con 0x7f01340ce840 2024-09-19T08:47:45.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.078+0000 7f013bcc5640 1 -- 172.21.15.98:0/1858902321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01340ce840 msgr2=0x7f01340cec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:45.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.078+0000 7f013bcc5640 1 --2- 172.21.15.98:0/1858902321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01340ce840 0x7f01340cec90 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f012c009e50 tx=0x7f012c01c6e0 comp rx=0 tx=0).stop 2024-09-19T08:47:45.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.078+0000 7f013bcc5640 1 -- 172.21.15.98:0/1858902321 shutdown_connections 2024-09-19T08:47:45.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.078+0000 7f013bcc5640 1 --2- 172.21.15.98:0/1858902321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f01340ce840 0x7f01340cec90 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:45.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.078+0000 7f013bcc5640 1 --2- 172.21.15.98:0/1858902321 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f013406bc90 0x7f013406c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:45.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.078+0000 7f013bcc5640 1 -- 172.21.15.98:0/1858902321 >> 172.21.15.98:0/1858902321 conn(0x7f01340b2010 msgr2=0x7f01340b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:45.076 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.078+0000 7f013bcc5640 1 -- 172.21.15.98:0/1858902321 shutdown_connections 2024-09-19T08:47:45.077 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.078+0000 7f013bcc5640 1 -- 172.21.15.98:0/1858902321 wait complete. 2024-09-19T08:47:45.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.078+0000 7f013bcc5640 1 Processor -- start 2024-09-19T08:47:45.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.078+0000 7f013bcc5640 1 -- start start 2024-09-19T08:47:45.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.078+0000 7f013bcc5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f013406bc90 0x7f01341f7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:45.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.078+0000 7f013bcc5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01340ce840 0x7f0134154660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:45.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.078+0000 7f013bcc5640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0134158fd0 0x7f0134072c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:45.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.078+0000 7f013bcc5640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f013415c920 con 0x7f013406bc90 2024-09-19T08:47:45.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.078+0000 7f013bcc5640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f013415c7a0 con 0x7f01340ce840 2024-09-19T08:47:45.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.078+0000 7f013bcc5640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0134166100 con 0x7f0134158fd0 2024-09-19T08:47:45.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.078+0000 7f0139a61640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f013406bc90 0x7f01341f7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:45.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.078+0000 7f0139a61640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f013406bc90 0x7f01341f7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37418/0 (socket says 172.21.15.98:37418) 2024-09-19T08:47:45.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.078+0000 7f0139a61640 1 -- 172.21.15.98:0/3601522147 learned_addr learned my addr 172.21.15.98:0/3601522147 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:45.079 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.078+0000 7f0139a61640 1 -- 172.21.15.98:0/3601522147 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0134158fd0 msgr2=0x7f0134072c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:45.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.078+0000 7f013a262640 1 --2- 172.21.15.98:0/3601522147 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0134158fd0 0x7f0134072c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:45.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.078+0000 7f0139260640 1 --2- 172.21.15.98:0/3601522147 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01340ce840 0x7f0134154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:45.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.078+0000 7f0139a61640 1 --2- 172.21.15.98:0/3601522147 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0134158fd0 0x7f0134072c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:45.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.078+0000 7f0139a61640 1 -- 172.21.15.98:0/3601522147 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01340ce840 msgr2=0x7f0134154660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:45.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.078+0000 7f0139a61640 1 --2- 172.21.15.98:0/3601522147 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01340ce840 0x7f0134154660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:45.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.078+0000 7f0139a61640 1 -- 172.21.15.98:0/3601522147 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01341f71b0 con 0x7f013406bc90 2024-09-19T08:47:45.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.078+0000 7f0139a61640 1 --2- 172.21.15.98:0/3601522147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f013406bc90 0x7f01341f7c70 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f013000c480 tx=0x7f013000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:45.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.078+0000 7f012a7fc640 1 -- 172.21.15.98:0/3601522147 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0130018070 con 0x7f013406bc90 2024-09-19T08:47:45.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.078+0000 7f012a7fc640 1 -- 172.21.15.98:0/3601522147 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0130007500 con 0x7f013406bc90 2024-09-19T08:47:45.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.078+0000 7f012a7fc640 1 -- 172.21.15.98:0/3601522147 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0130004a30 con 0x7f013406bc90 2024-09-19T08:47:45.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.078+0000 7f013bcc5640 1 -- 172.21.15.98:0/3601522147 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0134073580 con 0x7f013406bc90 2024-09-19T08:47:45.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.078+0000 7f013bcc5640 1 -- 172.21.15.98:0/3601522147 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01341f7470 con 0x7f013406bc90 2024-09-19T08:47:45.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.082+0000 7f013bcc5640 1 -- 172.21.15.98:0/3601522147 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f013406c120 con 0x7f013406bc90 2024-09-19T08:47:45.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.082+0000 7f012a7fc640 1 -- 172.21.15.98:0/3601522147 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f013001b070 con 0x7f013406bc90 2024-09-19T08:47:45.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.082+0000 7f012a7fc640 1 --2- 172.21.15.98:0/3601522147 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0118068400 0x7f011806a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:45.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.082+0000 7f012a7fc640 1 -- 172.21.15.98:0/3601522147 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7f0130083b10 con 0x7f013406bc90 2024-09-19T08:47:45.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.082+0000 7f0139260640 1 --2- 172.21.15.98:0/3601522147 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0118068400 0x7f011806a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:45.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.082+0000 7f0139260640 1 --2- 172.21.15.98:0/3601522147 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0118068400 0x7f011806a8c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f0124009790 tx=0x7f0124008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:45.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.086+0000 7f012a7fc640 1 -- 172.21.15.98:0/3601522147 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f0130055eb0 con 0x7f013406bc90 2024-09-19T08:47:45.248 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.250+0000 7f013bcc5640 1 -- 172.21.15.98:0/3601522147 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f013415b8d0 con 0x7f0118068400 2024-09-19T08:47:45.250 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.250+0000 7f012a7fc640 1 -- 172.21.15.98:0/3601522147 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13510 (secure 0 0 0) 0x7f013415b8d0 con 0x7f0118068400 2024-09-19T08:47:45.250 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:47:45.250 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:47:45.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.254+0000 7f013bcc5640 1 -- 172.21.15.98:0/3601522147 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0118068400 msgr2=0x7f011806a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:45.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.254+0000 7f013bcc5640 1 --2- 172.21.15.98:0/3601522147 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0118068400 0x7f011806a8c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f0124009790 tx=0x7f0124008040 comp rx=0 tx=0).stop 2024-09-19T08:47:45.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.254+0000 7f013bcc5640 1 -- 172.21.15.98:0/3601522147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f013406bc90 msgr2=0x7f01341f7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:45.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.254+0000 7f013bcc5640 1 --2- 172.21.15.98:0/3601522147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f013406bc90 0x7f01341f7c70 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f013000c480 tx=0x7f013000c940 comp rx=0 tx=0).stop 2024-09-19T08:47:45.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.254+0000 7f013bcc5640 1 -- 172.21.15.98:0/3601522147 shutdown_connections 2024-09-19T08:47:45.252 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.254+0000 7f013bcc5640 1 --2- 172.21.15.98:0/3601522147 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0118068400 0x7f011806a8c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:45.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.254+0000 7f013bcc5640 1 --2- 172.21.15.98:0/3601522147 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0134158fd0 0x7f0134072c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:45.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.254+0000 7f013bcc5640 1 --2- 172.21.15.98:0/3601522147 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f01340ce840 0x7f0134154660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:45.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.254+0000 7f013bcc5640 1 --2- 172.21.15.98:0/3601522147 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f013406bc90 0x7f01341f7c70 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:45.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.254+0000 7f013bcc5640 1 -- 172.21.15.98:0/3601522147 >> 172.21.15.98:0/3601522147 conn(0x7f01340b2010 msgr2=0x7f01340b0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:45.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.254+0000 7f013bcc5640 1 -- 172.21.15.98:0/3601522147 shutdown_connections 2024-09-19T08:47:45.253 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.254+0000 7f013bcc5640 1 -- 172.21.15.98:0/3601522147 wait complete. 2024-09-19T08:47:45.268 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":247,"stamp":"2024-09-19T08:47:44.968475+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":171,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":275960,"kb_used_data":18120,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754698760,"statfs":{"total":773094113280,"available":772811530240,"internally_reserved":0,"allocated":18554880,"data_stored":8895858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-53351,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-3,"num_read_kb":-3,"num_write":-77,"num_write_kb":-82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002131"},"pg_stats":[{"pgid":"1.0","version":"71'128","reported_seq":482,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-19T08:47:41.939848+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:47:41.939848+0000","last_peered":"2024-09-19T08:47:41.939848+0000","last_clean":"2024-09-19T08:47:41.939848+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:47:41.939848+0000","last_undegraded":"2024-09-19T08:47:41.939848+0000","last_fullsized":"2024-09-19T08:47:41.939848+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509558,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33884,"kb_used_data":2696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337956,"statfs":{"total":96636764160,"available":96602066944,"internally_reserved":0,"allocated":2760704,"data_stored":1553360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574967,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33308,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338532,"statfs":{"total":96636764160,"available":96602656768,"internally_reserved":0,"allocated":2166784,"data_stored":962992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509560,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33304,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338536,"statfs":{"total":96636764160,"available":96602660864,"internally_reserved":0,"allocated":2166784,"data_stored":962992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509560,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33324,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338516,"statfs":{"total":96636764160,"available":96602640384,"internally_reserved":0,"allocated":2183168,"data_stored":968585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542262,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37464,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334376,"statfs":{"total":96636764160,"available":96598401024,"internally_reserved":0,"allocated":2166784,"data_stored":962992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509557,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33308,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338532,"statfs":{"total":96636764160,"available":96602656768,"internally_reserved":0,"allocated":2166784,"data_stored":962992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509559,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33304,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338536,"statfs":{"total":96636764160,"available":96602660864,"internally_reserved":0,"allocated":2166784,"data_stored":962992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509557,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38064,"kb_used_data":2712,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333776,"statfs":{"total":96636764160,"available":96597786624,"internally_reserved":0,"allocated":2777088,"data_stored":1558953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:47:45.268 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T08:47:45.269 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:47:45.391 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.390+0000 7f664ae4c640 1 Processor -- start 2024-09-19T08:47:45.392 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.390+0000 7f664ae4c640 1 -- start start 2024-09-19T08:47:45.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.394+0000 7f664ae4c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f664415c550 0x7f664406bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:45.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.394+0000 7f664ae4c640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f664406d310 con 0x7f664406c1d0 2024-09-19T08:47:45.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.394+0000 7f664ae4c640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f664406d5f0 con 0x7f66440cfad0 2024-09-19T08:47:45.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.394+0000 7f664ae4c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f664406c610 con 0x7f664415c550 2024-09-19T08:47:45.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.394+0000 7f6643fff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f664406c1d0 0x7f66440cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:58848/0 (socket says 172.21.15.98:58848) 2024-09-19T08:47:45.394 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.394+0000 7f6643fff640 1 -- 172.21.15.98:0/2847132210 learned_addr learned my addr 172.21.15.98:0/2847132210 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:45.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.394+0000 7f6648be8640 1 --2- 172.21.15.98:0/2847132210 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f664415c550 0x7f664406bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:45.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.394+0000 7f66437fe640 1 -- 172.21.15.98:0/2847132210 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 61931827 0 0) 0x7f664406d310 con 0x7f664406c1d0 2024-09-19T08:47:45.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.394+0000 7f66437fe640 1 -- 172.21.15.98:0/2847132210 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6630003530 con 0x7f664406c1d0 2024-09-19T08:47:45.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.394+0000 7f66437fe640 1 -- 172.21.15.98:0/2847132210 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3907672568 0 0) 0x7f6630003530 con 0x7f664406c1d0 2024-09-19T08:47:45.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.394+0000 7f66437fe640 1 -- 172.21.15.98:0/2847132210 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f664415c550 msgr2=0x7f664406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:45.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.394+0000 7f66437fe640 1 --2- 172.21.15.98:0/2847132210 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f664415c550 0x7f664406bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:45.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.394+0000 7f66437fe640 1 -- 172.21.15.98:0/2847132210 >> v1:172.21.15.139:6789/0 conn(0x7f66440cfad0 legacy=0x7f66440723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:45.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.394+0000 7f66437fe640 1 -- 172.21.15.98:0/2847132210 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f664406d8d0 con 0x7f664406c1d0 2024-09-19T08:47:45.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.394+0000 7f66437fe640 1 -- 172.21.15.98:0/2847132210 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f662c001680 con 0x7f664406c1d0 2024-09-19T08:47:45.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.394+0000 7f66437fe640 1 -- 172.21.15.98:0/2847132210 >> v1:172.21.15.98:6789/0 conn(0x7f664406c1d0 legacy=0x7f66440cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:45.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.394+0000 7f66437fe640 1 --2- 172.21.15.98:0/2847132210 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66300039f0 0x7f6630003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:45.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.394+0000 7f66437fe640 1 --2- 172.21.15.98:0/2847132210 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6630004810 0x7f6630004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:45.395 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.394+0000 7f66437fe640 1 --2- 172.21.15.98:0/2847132210 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f664415c550 0x7f66300051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:45.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.394+0000 7f66437fe640 1 -- 172.21.15.98:0/2847132210 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f664406c610 con 0x7f66300039f0 2024-09-19T08:47:45.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.394+0000 7f66437fe640 1 -- 172.21.15.98:0/2847132210 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f663000ba80 con 0x7f6630004810 2024-09-19T08:47:45.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.394+0000 7f66437fe640 1 -- 172.21.15.98:0/2847132210 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f663000bc80 con 0x7f664415c550 2024-09-19T08:47:45.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.394+0000 7f6648be8640 1 --2- 172.21.15.98:0/2847132210 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66300039f0 0x7f6630003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:45.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.394+0000 7f6643fff640 1 --2- 172.21.15.98:0/2847132210 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6630004810 0x7f6630004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:45.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.394+0000 7f66493e9640 1 --2- 172.21.15.98:0/2847132210 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f664415c550 0x7f66300051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:45.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.394+0000 7f6648be8640 1 -- 172.21.15.98:0/2847132210 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f664415c550 msgr2=0x7f66300051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:45.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.394+0000 7f6648be8640 1 --2- 172.21.15.98:0/2847132210 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f664415c550 0x7f66300051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:45.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.394+0000 7f6648be8640 1 -- 172.21.15.98:0/2847132210 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6630004810 msgr2=0x7f6630004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:45.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.394+0000 7f6648be8640 1 --2- 172.21.15.98:0/2847132210 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6630004810 0x7f6630004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:45.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.394+0000 7f6648be8640 1 -- 172.21.15.98:0/2847132210 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6630003530 con 0x7f66300039f0 2024-09-19T08:47:45.396 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.394+0000 7f66493e9640 1 --2- 172.21.15.98:0/2847132210 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f664415c550 0x7f66300051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:45.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.398+0000 7f6648be8640 1 --2- 172.21.15.98:0/2847132210 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66300039f0 0x7f6630003e10 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f663c009ab0 tx=0x7f663c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1add587f0c0967 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:45.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.398+0000 7f66437fe640 1 -- 172.21.15.98:0/2847132210 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c0096b0 con 0x7f66300039f0 2024-09-19T08:47:45.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.398+0000 7f66437fe640 1 -- 172.21.15.98:0/2847132210 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f663c0a6410 con 0x7f66300039f0 2024-09-19T08:47:45.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.398+0000 7f66437fe640 1 -- 172.21.15.98:0/2847132210 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c0a6800 con 0x7f66300039f0 2024-09-19T08:47:45.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.398+0000 7f664ae4c640 1 -- 172.21.15.98:0/2847132210 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66300039f0 msgr2=0x7f6630003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:45.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.398+0000 7f664ae4c640 1 --2- 172.21.15.98:0/2847132210 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66300039f0 0x7f6630003e10 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f663c009ab0 tx=0x7f663c01ca30 comp rx=0 tx=0).stop 2024-09-19T08:47:45.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.398+0000 7f66493e9640 1 -- 172.21.15.98:0/2847132210 reap_dead start 2024-09-19T08:47:45.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.398+0000 7f664ae4c640 1 -- 172.21.15.98:0/2847132210 shutdown_connections 2024-09-19T08:47:45.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.398+0000 7f664ae4c640 1 -- 172.21.15.98:0/2847132210 >> 172.21.15.98:0/2847132210 conn(0x7f66440b2010 msgr2=0x7f66440b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:45.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.398+0000 7f664ae4c640 1 -- 172.21.15.98:0/2847132210 shutdown_connections 2024-09-19T08:47:45.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.398+0000 7f664ae4c640 1 -- 172.21.15.98:0/2847132210 wait complete. 2024-09-19T08:47:45.398 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.398+0000 7f664ae4c640 1 Processor -- start 2024-09-19T08:47:45.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.398+0000 7f664ae4c640 1 -- start start 2024-09-19T08:47:45.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.398+0000 7f664ae4c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66300039f0 0x7f6644143cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:45.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.398+0000 7f664ae4c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66441550e0 0x7f6644155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:45.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.398+0000 7f664ae4c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f664415fc40 0x7f66441600c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:45.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.398+0000 7f664ae4c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f6644073510 con 0x7f66441550e0 2024-09-19T08:47:45.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.398+0000 7f664ae4c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f664415cb40 con 0x7f66300039f0 2024-09-19T08:47:45.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.398+0000 7f664ae4c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6644073690 con 0x7f664415fc40 2024-09-19T08:47:45.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.398+0000 7f6643fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66441550e0 0x7f6644155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:45.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.398+0000 7f6643fff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66441550e0 0x7f6644155f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37426/0 (socket says 172.21.15.98:37426) 2024-09-19T08:47:45.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.398+0000 7f6643fff640 1 -- 172.21.15.98:0/2867481388 learned_addr learned my addr 172.21.15.98:0/2867481388 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:45.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.398+0000 7f6648be8640 1 --2- 172.21.15.98:0/2867481388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66300039f0 0x7f6644143cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:45.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.398+0000 7f66493e9640 1 --2- 172.21.15.98:0/2867481388 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f664415fc40 0x7f66441600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:45.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.398+0000 7f6643fff640 1 -- 172.21.15.98:0/2867481388 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f664415fc40 msgr2=0x7f66441600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:45.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.398+0000 7f6643fff640 1 --2- 172.21.15.98:0/2867481388 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f664415fc40 0x7f66441600c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:45.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.398+0000 7f6643fff640 1 -- 172.21.15.98:0/2867481388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66300039f0 msgr2=0x7f6644143cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:45.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.398+0000 7f6643fff640 1 --2- 172.21.15.98:0/2867481388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66300039f0 0x7f6644143cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:45.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.398+0000 7f6643fff640 1 -- 172.21.15.98:0/2867481388 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f664415d330 con 0x7f66441550e0 2024-09-19T08:47:45.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.398+0000 7f6648be8640 1 --2- 172.21.15.98:0/2867481388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66300039f0 0x7f6644143cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:45.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.398+0000 7f6643fff640 1 --2- 172.21.15.98:0/2867481388 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66441550e0 0x7f6644155f70 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f662c000fc0 tx=0x7f662c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:45.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.398+0000 7f66417fa640 1 -- 172.21.15.98:0/2867481388 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c00c870 con 0x7f66441550e0 2024-09-19T08:47:45.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.398+0000 7f66417fa640 1 -- 172.21.15.98:0/2867481388 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f662c018070 con 0x7f66441550e0 2024-09-19T08:47:45.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.398+0000 7f664ae4c640 1 -- 172.21.15.98:0/2867481388 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6644144230 con 0x7f66441550e0 2024-09-19T08:47:45.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.398+0000 7f66417fa640 1 -- 172.21.15.98:0/2867481388 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c00cc50 con 0x7f66441550e0 2024-09-19T08:47:45.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.398+0000 7f664ae4c640 1 -- 172.21.15.98:0/2867481388 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f664415d880 con 0x7f66441550e0 2024-09-19T08:47:45.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.402+0000 7f664ae4c640 1 -- 172.21.15.98:0/2867481388 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66440cf3c0 con 0x7f66441550e0 2024-09-19T08:47:45.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.402+0000 7f66417fa640 1 -- 172.21.15.98:0/2867481388 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f662c013420 con 0x7f66441550e0 2024-09-19T08:47:45.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.402+0000 7f66417fa640 1 --2- 172.21.15.98:0/2867481388 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6624068430 0x7f662406a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:45.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.402+0000 7f66417fa640 1 -- 172.21.15.98:0/2867481388 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7f662c084110 con 0x7f66441550e0 2024-09-19T08:47:45.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.402+0000 7f6648be8640 1 --2- 172.21.15.98:0/2867481388 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6624068430 0x7f662406a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:45.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.402+0000 7f6648be8640 1 --2- 172.21.15.98:0/2867481388 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6624068430 0x7f662406a8f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f663c0ad000 tx=0x7f663c0046d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:45.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.406+0000 7f66417fa640 1 -- 172.21.15.98:0/2867481388 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f662c056700 con 0x7f66441550e0 2024-09-19T08:47:45.564 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.566+0000 7f664ae4c640 1 -- 172.21.15.98:0/2867481388 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f664415a9b0 con 0x7f6624068430 2024-09-19T08:47:45.568 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.566+0000 7f66417fa640 1 -- 172.21.15.98:0/2867481388 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13510 (secure 0 0 0) 0x7f664415a9b0 con 0x7f6624068430 2024-09-19T08:47:45.569 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:47:45.570 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:47:45.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.570+0000 7f664ae4c640 1 -- 172.21.15.98:0/2867481388 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6624068430 msgr2=0x7f662406a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:45.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.570+0000 7f664ae4c640 1 --2- 172.21.15.98:0/2867481388 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6624068430 0x7f662406a8f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f663c0ad000 tx=0x7f663c0046d0 comp rx=0 tx=0).stop 2024-09-19T08:47:45.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.570+0000 7f664ae4c640 1 -- 172.21.15.98:0/2867481388 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66441550e0 msgr2=0x7f6644155f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:45.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.570+0000 7f664ae4c640 1 --2- 172.21.15.98:0/2867481388 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66441550e0 0x7f6644155f70 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f662c000fc0 tx=0x7f662c00ece0 comp rx=0 tx=0).stop 2024-09-19T08:47:45.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.570+0000 7f664ae4c640 1 -- 172.21.15.98:0/2867481388 shutdown_connections 2024-09-19T08:47:45.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.570+0000 7f664ae4c640 1 --2- 172.21.15.98:0/2867481388 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f6624068430 0x7f662406a8f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:45.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.570+0000 7f664ae4c640 1 --2- 172.21.15.98:0/2867481388 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f664415fc40 0x7f66441600c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:45.570 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.570+0000 7f664ae4c640 1 --2- 172.21.15.98:0/2867481388 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f66441550e0 0x7f6644155f70 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:45.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.570+0000 7f664ae4c640 1 --2- 172.21.15.98:0/2867481388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66300039f0 0x7f6644143cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:45.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.570+0000 7f664ae4c640 1 -- 172.21.15.98:0/2867481388 >> 172.21.15.98:0/2867481388 conn(0x7f66440b2010 msgr2=0x7f66440cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:45.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.570+0000 7f664ae4c640 1 -- 172.21.15.98:0/2867481388 shutdown_connections 2024-09-19T08:47:45.571 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.570+0000 7f664ae4c640 1 -- 172.21.15.98:0/2867481388 wait complete. 2024-09-19T08:47:45.586 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":247,"stamp":"2024-09-19T08:47:44.968475+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":171,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":275960,"kb_used_data":18120,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754698760,"statfs":{"total":773094113280,"available":772811530240,"internally_reserved":0,"allocated":18554880,"data_stored":8895858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-53351,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-3,"num_read_kb":-3,"num_write":-77,"num_write_kb":-82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002131"},"pg_stats":[{"pgid":"1.0","version":"71'128","reported_seq":482,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-19T08:47:41.939848+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:47:41.939848+0000","last_peered":"2024-09-19T08:47:41.939848+0000","last_clean":"2024-09-19T08:47:41.939848+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:47:41.939848+0000","last_undegraded":"2024-09-19T08:47:41.939848+0000","last_fullsized":"2024-09-19T08:47:41.939848+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509558,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33884,"kb_used_data":2696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337956,"statfs":{"total":96636764160,"available":96602066944,"internally_reserved":0,"allocated":2760704,"data_stored":1553360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574967,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33308,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338532,"statfs":{"total":96636764160,"available":96602656768,"internally_reserved":0,"allocated":2166784,"data_stored":962992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509560,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33304,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338536,"statfs":{"total":96636764160,"available":96602660864,"internally_reserved":0,"allocated":2166784,"data_stored":962992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509560,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33324,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338516,"statfs":{"total":96636764160,"available":96602640384,"internally_reserved":0,"allocated":2183168,"data_stored":968585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542262,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37464,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334376,"statfs":{"total":96636764160,"available":96598401024,"internally_reserved":0,"allocated":2166784,"data_stored":962992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509557,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33308,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338532,"statfs":{"total":96636764160,"available":96602656768,"internally_reserved":0,"allocated":2166784,"data_stored":962992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509559,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33304,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338536,"statfs":{"total":96636764160,"available":96602660864,"internally_reserved":0,"allocated":2166784,"data_stored":962992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509557,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38064,"kb_used_data":2712,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333776,"statfs":{"total":96636764160,"available":96597786624,"internally_reserved":0,"allocated":2777088,"data_stored":1558953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:47:45.586 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T08:47:45.703 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.702+0000 7f71671e0640 1 Processor -- start 2024-09-19T08:47:45.704 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.702+0000 7f71671e0640 1 -- start start 2024-09-19T08:47:45.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.706+0000 7f71671e0640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f71600d1d70 0x7f71600d1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:45.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.706+0000 7f71671e0640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71600b8060 con 0x7f71600b6200 2024-09-19T08:47:45.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.706+0000 7f71671e0640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71600b8340 con 0x7f7160147530 2024-09-19T08:47:45.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.706+0000 7f71671e0640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f71600d2230 con 0x7f71600d1d70 2024-09-19T08:47:45.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.706+0000 7f715ffff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f71600d1d70 0x7f71600d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:45.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.706+0000 7f716577d640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f71600b6200 0x7f71600b8e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:58854/0 (socket says 172.21.15.98:58854) 2024-09-19T08:47:45.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.706+0000 7f716577d640 1 -- 172.21.15.98:0/2943644921 learned_addr learned my addr 172.21.15.98:0/2943644921 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:45.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.706+0000 7f715ffff640 1 -- 172.21.15.98:0/2943644921 >> v1:172.21.15.139:6789/0 conn(0x7f7160147530 legacy=0x7f716015ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:45.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.706+0000 7f715ffff640 1 -- 172.21.15.98:0/2943644921 >> v1:172.21.15.98:6789/0 conn(0x7f71600b6200 legacy=0x7f71600b8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:45.705 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.706+0000 7f715ffff640 1 -- 172.21.15.98:0/2943644921 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71600b8620 con 0x7f71600d1d70 2024-09-19T08:47:45.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.706+0000 7f715ffff640 1 --2- 172.21.15.98:0/2943644921 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71600d1d70 0x7f71600d1510 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f71540097c0 tx=0x7f715401c2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8392fadc3edd6a19 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:45.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.706+0000 7f715f7fe640 1 -- 172.21.15.98:0/2943644921 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71540b0070 con 0x7f71600d1d70 2024-09-19T08:47:45.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.706+0000 7f715f7fe640 1 -- 172.21.15.98:0/2943644921 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71540ab600 con 0x7f71600d1d70 2024-09-19T08:47:45.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.706+0000 7f715f7fe640 1 -- 172.21.15.98:0/2943644921 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71540ab9d0 con 0x7f71600d1d70 2024-09-19T08:47:45.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.706+0000 7f71671e0640 1 -- 172.21.15.98:0/2943644921 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71600d1d70 msgr2=0x7f71600d1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:45.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.706+0000 7f71671e0640 1 --2- 172.21.15.98:0/2943644921 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71600d1d70 0x7f71600d1510 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f71540097c0 tx=0x7f715401c2f0 comp rx=0 tx=0).stop 2024-09-19T08:47:45.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.710+0000 7f71671e0640 1 -- 172.21.15.98:0/2943644921 shutdown_connections 2024-09-19T08:47:45.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.710+0000 7f71671e0640 1 --2- 172.21.15.98:0/2943644921 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f71600d1d70 0x7f71600d1510 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:45.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.710+0000 7f71671e0640 1 -- 172.21.15.98:0/2943644921 >> 172.21.15.98:0/2943644921 conn(0x7f7160075070 msgr2=0x7f71600722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:45.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.710+0000 7f71671e0640 1 -- 172.21.15.98:0/2943644921 shutdown_connections 2024-09-19T08:47:45.708 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.710+0000 7f71671e0640 1 -- 172.21.15.98:0/2943644921 wait complete. 2024-09-19T08:47:45.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.710+0000 7f71671e0640 1 Processor -- start 2024-09-19T08:47:45.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.710+0000 7f71671e0640 1 -- start start 2024-09-19T08:47:45.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.710+0000 7f71671e0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71600b6200 0x7f716015e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:45.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.710+0000 7f71671e0640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71600d1d70 0x7f716015ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:45.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.710+0000 7f71671e0640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7160147530 0x7f716015f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:45.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.710+0000 7f71671e0640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f7160147910 con 0x7f7160147530 2024-09-19T08:47:45.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.710+0000 7f71671e0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f71601483a0 con 0x7f71600b6200 2024-09-19T08:47:45.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.710+0000 7f71671e0640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f7160147a90 con 0x7f71600d1d70 2024-09-19T08:47:45.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.710+0000 7f716577d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7160147530 0x7f716015f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:45.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.710+0000 7f716577d640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7160147530 0x7f716015f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37438/0 (socket says 172.21.15.98:37438) 2024-09-19T08:47:45.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.710+0000 7f716577d640 1 -- 172.21.15.98:0/3204098330 learned_addr learned my addr 172.21.15.98:0/3204098330 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:45.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.710+0000 7f7164f7c640 1 --2- 172.21.15.98:0/3204098330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71600b6200 0x7f716015e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:45.711 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.710+0000 7f715ffff640 1 --2- 172.21.15.98:0/3204098330 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71600d1d70 0x7f716015ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:45.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.710+0000 7f716577d640 1 -- 172.21.15.98:0/3204098330 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71600d1d70 msgr2=0x7f716015ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:45.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.710+0000 7f716577d640 1 --2- 172.21.15.98:0/3204098330 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71600d1d70 0x7f716015ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:45.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.710+0000 7f716577d640 1 -- 172.21.15.98:0/3204098330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71600b6200 msgr2=0x7f716015e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:45.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.710+0000 7f716577d640 1 --2- 172.21.15.98:0/3204098330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71600b6200 0x7f716015e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:45.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.710+0000 7f716577d640 1 -- 172.21.15.98:0/3204098330 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71600b3980 con 0x7f7160147530 2024-09-19T08:47:45.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.710+0000 7f716577d640 1 --2- 172.21.15.98:0/3204098330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7160147530 0x7f716015f1b0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f7150009e40 tx=0x7f715000c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:45.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.710+0000 7f715d7fa640 1 -- 172.21.15.98:0/3204098330 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7150019070 con 0x7f7160147530 2024-09-19T08:47:45.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.710+0000 7f715d7fa640 1 -- 172.21.15.98:0/3204098330 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71500078f0 con 0x7f7160147530 2024-09-19T08:47:45.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.710+0000 7f71671e0640 1 -- 172.21.15.98:0/3204098330 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71600b3c60 con 0x7f7160147530 2024-09-19T08:47:45.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.710+0000 7f715d7fa640 1 -- 172.21.15.98:0/3204098330 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7150007cc0 con 0x7f7160147530 2024-09-19T08:47:45.712 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.710+0000 7f71671e0640 1 -- 172.21.15.98:0/3204098330 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71600b4220 con 0x7f7160147530 2024-09-19T08:47:45.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.710+0000 7f71671e0640 1 -- 172.21.15.98:0/3204098330 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71600d1510 con 0x7f7160147530 2024-09-19T08:47:45.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.714+0000 7f715d7fa640 1 -- 172.21.15.98:0/3204098330 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f715001c070 con 0x7f7160147530 2024-09-19T08:47:45.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.714+0000 7f715d7fa640 1 --2- 172.21.15.98:0/3204098330 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f714c068430 0x7f714c06a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:45.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.714+0000 7f715d7fa640 1 -- 172.21.15.98:0/3204098330 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7f7150086530 con 0x7f7160147530 2024-09-19T08:47:45.713 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.714+0000 7f7164f7c640 1 --2- 172.21.15.98:0/3204098330 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f714c068430 0x7f714c06a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:45.714 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.714+0000 7f7164f7c640 1 --2- 172.21.15.98:0/3204098330 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f714c068430 0x7f714c06a8f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7158009f20 tx=0x7f71580094d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:45.717 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.718+0000 7f715d7fa640 1 -- 172.21.15.98:0/3204098330 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f715008f080 con 0x7f7160147530 2024-09-19T08:47:45.869 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.870+0000 7f71671e0640 1 -- 172.21.15.98:0/3204098330 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f716013dba0 con 0x7f7160147530 2024-09-19T08:47:45.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.870+0000 7f715d7fa640 1 -- 172.21.15.98:0/3204098330 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v173) ==== 74+0+13086 (secure 0 0 0) 0x7f71500588d0 con 0x7f7160147530 2024-09-19T08:47:45.870 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:47:45.871 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":173,"fsid":"552bf0ee-1f4f-4644-ba7c-d0931e55a451","created":"2024-09-19T08:37:55.448033+0000","modified":"2024-09-19T08:47:42.821259+0000","last_up_change":"2024-09-19T08:38:08.696389+0000","last_in_change":"2024-09-19T08:37:57.685486+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T08:38:08.105684+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ac912a7b-9671-49b2-bc86-b3eae286c89f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6816","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6818","nonce":2191254527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6819","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6821","nonce":2191254527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6826","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6827","nonce":2191254527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6822","nonce":2191254527},{"type":"v1","addr":"172.21.15.98:6824","nonce":2191254527}]},"public_addr":"172.21.15.98:6818/2191254527","cluster_addr":"172.21.15.98:6821/2191254527","heartbeat_back_addr":"172.21.15.98:6827/2191254527","heartbeat_front_addr":"172.21.15.98:6824/2191254527","state":["exists","up"]},{"osd":1,"uuid":"ed3494e5-e909-4d1a-a4f2-9080dbed7574","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6817","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6820","nonce":1538482309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6823","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6825","nonce":1538482309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6830","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6831","nonce":1538482309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6828","nonce":1538482309},{"type":"v1","addr":"172.21.15.98:6829","nonce":1538482309}]},"public_addr":"172.21.15.98:6820/1538482309","cluster_addr":"172.21.15.98:6825/1538482309","heartbeat_back_addr":"172.21.15.98:6831/1538482309","heartbeat_front_addr":"172.21.15.98:6829/1538482309","state":["exists","up"]},{"osd":2,"uuid":"9760b7fd-c73e-4668-9cc9-a1efe5d5767c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6808","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6809","nonce":2466005517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6810","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6811","nonce":2466005517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6814","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6815","nonce":2466005517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6812","nonce":2466005517},{"type":"v1","addr":"172.21.15.98:6813","nonce":2466005517}]},"public_addr":"172.21.15.98:6809/2466005517","cluster_addr":"172.21.15.98:6811/2466005517","heartbeat_back_addr":"172.21.15.98:6815/2466005517","heartbeat_front_addr":"172.21.15.98:6813/2466005517","state":["exists","up"]},{"osd":3,"uuid":"cf08f97e-9975-4b66-ac1a-7c6eb6a24e12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6800","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6801","nonce":3643795173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6802","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6803","nonce":3643795173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6806","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6807","nonce":3643795173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6804","nonce":3643795173},{"type":"v1","addr":"172.21.15.98:6805","nonce":3643795173}]},"public_addr":"172.21.15.98:6801/3643795173","cluster_addr":"172.21.15.98:6803/3643795173","heartbeat_back_addr":"172.21.15.98:6807/3643795173","heartbeat_front_addr":"172.21.15.98:6805/3643795173","state":["exists","up"]},{"osd":4,"uuid":"af96ffe3-e63d-48be-a561-6e3d1bf005c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6825","nonce":1412125138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6827","nonce":1412125138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6831","nonce":1412125138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1412125138},{"type":"v1","addr":"172.21.15.139:6829","nonce":1412125138}]},"public_addr":"172.21.15.139:6825/1412125138","cluster_addr":"172.21.15.139:6827/1412125138","heartbeat_back_addr":"172.21.15.139:6831/1412125138","heartbeat_front_addr":"172.21.15.139:6829/1412125138","state":["exists","up"]},{"osd":5,"uuid":"6225119f-e471-4453-a3c5-853dd8a58eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6809","nonce":520689888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6811","nonce":520689888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6815","nonce":520689888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":520689888},{"type":"v1","addr":"172.21.15.139:6813","nonce":520689888}]},"public_addr":"172.21.15.139:6809/520689888","cluster_addr":"172.21.15.139:6811/520689888","heartbeat_back_addr":"172.21.15.139:6815/520689888","heartbeat_front_addr":"172.21.15.139:6813/520689888","state":["exists","up"]},{"osd":6,"uuid":"da297124-d02d-4cba-ba1c-9a6f53c62929","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6801","nonce":2917689006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6803","nonce":2917689006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6807","nonce":2917689006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":2917689006},{"type":"v1","addr":"172.21.15.139:6805","nonce":2917689006}]},"public_addr":"172.21.15.139:6801/2917689006","cluster_addr":"172.21.15.139:6803/2917689006","heartbeat_back_addr":"172.21.15.139:6807/2917689006","heartbeat_front_addr":"172.21.15.139:6805/2917689006","state":["exists","up"]},{"osd":7,"uuid":"4ce2f72e-c241-44c1-b4cc-7616cde855fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6817","nonce":688014474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6819","nonce":688014474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6823","nonce":688014474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":688014474},{"type":"v1","addr":"172.21.15.139:6821","nonce":688014474}]},"public_addr":"172.21.15.139:6817/688014474","cluster_addr":"172.21.15.139:6819/688014474","heartbeat_back_addr":"172.21.15.139:6823/688014474","heartbeat_front_addr":"172.21.15.139:6821/688014474","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.182577+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.252535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.321441+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.742403+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.482896+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.809436+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:04.969327+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T08:38:05.700283+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.98:6835/3407902663":"2024-09-20T08:47:39.806778+0000","172.21.15.98:6834/3407902663":"2024-09-20T08:47:39.806778+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T08:47:45.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.874+0000 7f71671e0640 1 -- 172.21.15.98:0/3204098330 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f714c068430 msgr2=0x7f714c06a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:45.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.874+0000 7f71671e0640 1 --2- 172.21.15.98:0/3204098330 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f714c068430 0x7f714c06a8f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7158009f20 tx=0x7f71580094d0 comp rx=0 tx=0).stop 2024-09-19T08:47:45.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.874+0000 7f71671e0640 1 -- 172.21.15.98:0/3204098330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7160147530 msgr2=0x7f716015f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:45.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.874+0000 7f71671e0640 1 --2- 172.21.15.98:0/3204098330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7160147530 0x7f716015f1b0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f7150009e40 tx=0x7f715000c430 comp rx=0 tx=0).stop 2024-09-19T08:47:45.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.874+0000 7f71671e0640 1 -- 172.21.15.98:0/3204098330 shutdown_connections 2024-09-19T08:47:45.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.874+0000 7f71671e0640 1 --2- 172.21.15.98:0/3204098330 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f714c068430 0x7f714c06a8f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:45.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.874+0000 7f71671e0640 1 --2- 172.21.15.98:0/3204098330 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f7160147530 0x7f716015f1b0 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:45.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.874+0000 7f71671e0640 1 --2- 172.21.15.98:0/3204098330 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f71600d1d70 0x7f716015ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:45.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.874+0000 7f71671e0640 1 --2- 172.21.15.98:0/3204098330 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f71600b6200 0x7f716015e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:45.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.874+0000 7f71671e0640 1 -- 172.21.15.98:0/3204098330 >> 172.21.15.98:0/3204098330 conn(0x7f7160075070 msgr2=0x7f71600b8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:45.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.874+0000 7f71671e0640 1 -- 172.21.15.98:0/3204098330 shutdown_connections 2024-09-19T08:47:45.874 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:45.874+0000 7f71671e0640 1 -- 172.21.15.98:0/3204098330 wait complete. 2024-09-19T08:47:46.888 INFO:tasks.ceph:Scrubbing osd.0 2024-09-19T08:47:46.888 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-19T08:47:47.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.026+0000 7f0e5bdfa640 1 Processor -- start 2024-09-19T08:47:47.025 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.026+0000 7f0e5bdfa640 1 -- start start 2024-09-19T08:47:47.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.026+0000 7f0e5bdfa640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f0e5415c290 0x7f0e540cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.026+0000 7f0e5bdfa640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0e540cf700 0x7f0e5415ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.026+0000 7f0e5bdfa640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e541608c0 con 0x7f0e5406d4c0 2024-09-19T08:47:47.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.026+0000 7f0e5bdfa640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f0e54160e80 con 0x7f0e540cf700 2024-09-19T08:47:47.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.026+0000 7f0e5bdfa640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f0e540cd510 con 0x7f0e5415c290 2024-09-19T08:47:47.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.026+0000 7f0e59395640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f0e540cf700 0x7f0e5415ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.026+0000 7f0e5a397640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f0e5406d4c0 0x7f0e5406d8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:58868/0 (socket says 172.21.15.98:58868) 2024-09-19T08:47:47.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.026+0000 7f0e5a397640 1 -- 172.21.15.98:0/379095836 learned_addr learned my addr 172.21.15.98:0/379095836 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:47.027 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.026+0000 7f0e59b96640 1 --2- 172.21.15.98:0/379095836 >> v2:172.21.15.139:3300/0 conn(0x7f0e5415c290 0x7f0e540cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.026+0000 7f0e59395640 1 -- 172.21.15.98:0/379095836 >> v2:172.21.15.139:3300/0 conn(0x7f0e5415c290 msgr2=0x7f0e540cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:47.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.026+0000 7f0e59395640 1 --2- 172.21.15.98:0/379095836 >> v2:172.21.15.139:3300/0 conn(0x7f0e5415c290 0x7f0e540cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.026+0000 7f0e59395640 1 -- 172.21.15.98:0/379095836 >> v1:172.21.15.98:6789/0 conn(0x7f0e5406d4c0 legacy=0x7f0e5406d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:47.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.026+0000 7f0e59395640 1 -- 172.21.15.98:0/379095836 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e54160ba0 con 0x7f0e540cf700 2024-09-19T08:47:47.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.026+0000 7f0e58b94640 1 -- 172.21.15.98:0/379095836 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1738808218 0 0) 0x7f0e541608c0 con 0x7f0e5406d4c0 2024-09-19T08:47:47.028 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.026+0000 7f0e59b96640 1 --2- 172.21.15.98:0/379095836 >> v2:172.21.15.139:3300/0 conn(0x7f0e5415c290 0x7f0e540cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:47.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.030+0000 7f0e59395640 1 --2- 172.21.15.98:0/379095836 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e540cf700 0x7f0e5415ff50 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f0e540742e0 tx=0x7f0e5001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368855e2075c3490 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:47.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.030+0000 7f0e58b94640 1 -- 172.21.15.98:0/379095836 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e500b0070 con 0x7f0e540cf700 2024-09-19T08:47:47.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.030+0000 7f0e58b94640 1 -- 172.21.15.98:0/379095836 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e500ab600 con 0x7f0e540cf700 2024-09-19T08:47:47.029 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.030+0000 7f0e58b94640 1 -- 172.21.15.98:0/379095836 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e500ab9d0 con 0x7f0e540cf700 2024-09-19T08:47:47.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.030+0000 7f0e5bdfa640 1 -- 172.21.15.98:0/379095836 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e540cf700 msgr2=0x7f0e5415ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:47.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.030+0000 7f0e5bdfa640 1 --2- 172.21.15.98:0/379095836 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e540cf700 0x7f0e5415ff50 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f0e540742e0 tx=0x7f0e5001c3f0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.030+0000 7f0e5bdfa640 1 -- 172.21.15.98:0/379095836 shutdown_connections 2024-09-19T08:47:47.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.030+0000 7f0e5bdfa640 1 --2- 172.21.15.98:0/379095836 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e540cf700 0x7f0e5415ff50 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.030+0000 7f0e5bdfa640 1 --2- 172.21.15.98:0/379095836 >> v2:172.21.15.139:3300/0 conn(0x7f0e5415c290 0x7f0e540cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.030+0000 7f0e5bdfa640 1 -- 172.21.15.98:0/379095836 >> 172.21.15.98:0/379095836 conn(0x7f0e5406c090 msgr2=0x7f0e5406c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:47.030 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.030+0000 7f0e5bdfa640 1 -- 172.21.15.98:0/379095836 shutdown_connections 2024-09-19T08:47:47.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.030+0000 7f0e5bdfa640 1 -- 172.21.15.98:0/379095836 wait complete. 2024-09-19T08:47:47.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.030+0000 7f0e5bdfa640 1 Processor -- start 2024-09-19T08:47:47.031 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.030+0000 7f0e5bdfa640 1 -- start start 2024-09-19T08:47:47.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.034+0000 7f0e5bdfa640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e5406d4c0 0x7f0e5414f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.034+0000 7f0e5bdfa640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e540cf700 0x7f0e541547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.034+0000 7f0e5bdfa640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e5415c290 0x7f0e54152e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.034+0000 7f0e5bdfa640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f0e54160610 con 0x7f0e540cf700 2024-09-19T08:47:47.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.034+0000 7f0e5bdfa640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0e54160490 con 0x7f0e5415c290 2024-09-19T08:47:47.033 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.034+0000 7f0e5bdfa640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f0e540b5400 con 0x7f0e5406d4c0 2024-09-19T08:47:47.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.034+0000 7f0e59395640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e540cf700 0x7f0e541547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.034+0000 7f0e59395640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e540cf700 0x7f0e541547b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37446/0 (socket says 172.21.15.98:37446) 2024-09-19T08:47:47.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.034+0000 7f0e59395640 1 -- 172.21.15.98:0/3752775472 learned_addr learned my addr 172.21.15.98:0/3752775472 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:47.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.034+0000 7f0e59b96640 1 --2- 172.21.15.98:0/3752775472 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e5406d4c0 0x7f0e5414f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.034+0000 7f0e59395640 1 -- 172.21.15.98:0/3752775472 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e5406d4c0 msgr2=0x7f0e5414f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:47.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.034+0000 7f0e59395640 1 --2- 172.21.15.98:0/3752775472 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f0e5406d4c0 0x7f0e5414f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.034+0000 7f0e5a397640 1 --2- 172.21.15.98:0/3752775472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e5415c290 0x7f0e54152e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.034+0000 7f0e59395640 1 -- 172.21.15.98:0/3752775472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e5415c290 msgr2=0x7f0e54152e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:47.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.034+0000 7f0e59395640 1 --2- 172.21.15.98:0/3752775472 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0e5415c290 0x7f0e54152e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.034+0000 7f0e59395640 1 -- 172.21.15.98:0/3752775472 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e54155200 con 0x7f0e540cf700 2024-09-19T08:47:47.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.034+0000 7f0e59395640 1 --2- 172.21.15.98:0/3752775472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e540cf700 0x7f0e541547b0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f0e500ae040 tx=0x7f0e50019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:47.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.034+0000 7f0e4a7fc640 1 -- 172.21.15.98:0/3752775472 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e500b8020 con 0x7f0e540cf700 2024-09-19T08:47:47.034 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.034+0000 7f0e4a7fc640 1 -- 172.21.15.98:0/3752775472 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e500b4630 con 0x7f0e540cf700 2024-09-19T08:47:47.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.034+0000 7f0e5bdfa640 1 -- 172.21.15.98:0/3752775472 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e541554e0 con 0x7f0e540cf700 2024-09-19T08:47:47.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.034+0000 7f0e5bdfa640 1 -- 172.21.15.98:0/3752775472 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e5414fac0 con 0x7f0e540cf700 2024-09-19T08:47:47.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.034+0000 7f0e4a7fc640 1 -- 172.21.15.98:0/3752775472 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e500b0040 con 0x7f0e540cf700 2024-09-19T08:47:47.035 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.034+0000 7f0e37fff640 1 -- 172.21.15.98:0/3752775472 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0e30001040 con 0x7f0e540cf700 2024-09-19T08:47:47.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.034+0000 7f0e4a7fc640 1 -- 172.21.15.98:0/3752775472 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f0e500bb020 con 0x7f0e540cf700 2024-09-19T08:47:47.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.034+0000 7f0e4a7fc640 1 --2- 172.21.15.98:0/3752775472 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0e38068400 0x7f0e3806a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.034+0000 7f0e4a7fc640 1 -- 172.21.15.98:0/3752775472 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7f0e5011e930 con 0x7f0e540cf700 2024-09-19T08:47:47.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.034+0000 7f0e4a7fc640 1 --2- 172.21.15.98:0/3752775472 >> [v2:172.21.15.98:6816/2191254527,v1:172.21.15.98:6818/2191254527] conn(0x7f0e3806e2c0 0x7f0e380706e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.036 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.034+0000 7f0e4a7fc640 1 -- 172.21.15.98:0/3752775472 --> [v2:172.21.15.98:6816/2191254527,v1:172.21.15.98:6818/2191254527] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0e5000af30 con 0x7f0e3806e2c0 2024-09-19T08:47:47.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.034+0000 7f0e4a7fc640 1 -- 172.21.15.98:0/3752775472 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=173) ==== 24+0+0 (secure 0 0 0) 0x7f0e50121170 con 0x7f0e540cf700 2024-09-19T08:47:47.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.034+0000 7f0e59b96640 1 --2- 172.21.15.98:0/3752775472 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0e38068400 0x7f0e3806a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.034+0000 7f0e5a397640 1 --2- 172.21.15.98:0/3752775472 >> [v2:172.21.15.98:6816/2191254527,v1:172.21.15.98:6818/2191254527] conn(0x7f0e3806e2c0 0x7f0e380706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.034+0000 7f0e5a397640 1 --2- 172.21.15.98:0/3752775472 >> [v2:172.21.15.98:6816/2191254527,v1:172.21.15.98:6818/2191254527] conn(0x7f0e3806e2c0 0x7f0e380706e0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:47.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.038+0000 7f0e59b96640 1 --2- 172.21.15.98:0/3752775472 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0e38068400 0x7f0e3806a8c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0e44006fd0 tx=0x7f0e44008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:47.037 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.038+0000 7f0e4a7fc640 1 -- 172.21.15.98:0/3752775472 <== osd.0 v2:172.21.15.98:6816/2191254527 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0e5000af30 con 0x7f0e3806e2c0 2024-09-19T08:47:47.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.050+0000 7f0e37fff640 1 -- 172.21.15.98:0/3752775472 --> [v2:172.21.15.98:6816/2191254527,v1:172.21.15.98:6818/2191254527] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0e30002e40 con 0x7f0e3806e2c0 2024-09-19T08:47:47.049 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.050+0000 7f0e4a7fc640 1 -- 172.21.15.98:0/3752775472 <== osd.0 v2:172.21.15.98:6816/2191254527 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0e30002e40 con 0x7f0e3806e2c0 2024-09-19T08:47:47.049 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:47:47.050 INFO:teuthology.orchestra.run.smithi098.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T08:47:47.050 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:47:47.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.050+0000 7f0e37fff640 1 -- 172.21.15.98:0/3752775472 >> [v2:172.21.15.98:6816/2191254527,v1:172.21.15.98:6818/2191254527] conn(0x7f0e3806e2c0 msgr2=0x7f0e380706e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:47.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.050+0000 7f0e37fff640 1 --2- 172.21.15.98:0/3752775472 >> [v2:172.21.15.98:6816/2191254527,v1:172.21.15.98:6818/2191254527] conn(0x7f0e3806e2c0 0x7f0e380706e0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.050+0000 7f0e37fff640 1 -- 172.21.15.98:0/3752775472 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0e38068400 msgr2=0x7f0e3806a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:47.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.050+0000 7f0e37fff640 1 --2- 172.21.15.98:0/3752775472 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f0e38068400 0x7f0e3806a8c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0e44006fd0 tx=0x7f0e44008040 comp rx=0 tx=0).stop 2024-09-19T08:47:47.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.050+0000 7f0e37fff640 1 -- 172.21.15.98:0/3752775472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e540cf700 msgr2=0x7f0e541547b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:47.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.050+0000 7f0e37fff640 1 --2- 172.21.15.98:0/3752775472 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f0e540cf700 0x7f0e541547b0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f0e500ae040 tx=0x7f0e50019190 comp rx=0 tx=0).stop 2024-09-19T08:47:47.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.050+0000 7f0e5a397640 1 -- 172.21.15.98:0/3752775472 reap_dead start 2024-09-19T08:47:47.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.050+0000 7f0e37fff640 1 -- 172.21.15.98:0/3752775472 shutdown_connections 2024-09-19T08:47:47.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.050+0000 7f0e37fff640 1 -- 172.21.15.98:0/3752775472 >> 172.21.15.98:0/3752775472 conn(0x7f0e5406c090 msgr2=0x7f0e540b0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:47.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.050+0000 7f0e37fff640 1 -- 172.21.15.98:0/3752775472 shutdown_connections 2024-09-19T08:47:47.051 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.050+0000 7f0e37fff640 1 -- 172.21.15.98:0/3752775472 wait complete. 2024-09-19T08:47:47.066 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-19T08:47:47.177 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.178+0000 7f5ca1f2e640 1 Processor -- start 2024-09-19T08:47:47.177 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.178+0000 7f5ca1f2e640 1 -- start start 2024-09-19T08:47:47.178 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.178+0000 7f5ca1f2e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5c9c158fd0 0x7f5c9c0b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.178+0000 7f5ca1f2e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c9c0b4360 0x7f5c9c06bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.178+0000 7f5ca1f2e640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c9c0ccc60 con 0x7f5c9c06c1d0 2024-09-19T08:47:47.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.178+0000 7f5ca1f2e640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f5c9c1593b0 con 0x7f5c9c158fd0 2024-09-19T08:47:47.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.178+0000 7f5ca1f2e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5c9c06c600 con 0x7f5c9c0b4360 2024-09-19T08:47:47.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.178+0000 7f5c9b7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5c9c158fd0 0x7f5c9c0b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.178+0000 7f5c9b7fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f5c9c158fd0 0x7f5c9c0b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37450/0 (socket says 172.21.15.98:37450) 2024-09-19T08:47:47.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.178+0000 7f5c9b7fe640 1 -- 172.21.15.98:0/2280381302 learned_addr learned my addr 172.21.15.98:0/2280381302 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:47.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.178+0000 7f5c9b7fe640 1 -- 172.21.15.98:0/2280381302 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c9c0b4360 msgr2=0x7f5c9c06bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:47.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.178+0000 7f5c9affd640 1 --2- 172.21.15.98:0/2280381302 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c9c0b4360 0x7f5c9c06bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.178+0000 7f5c9b7fe640 1 --2- 172.21.15.98:0/2280381302 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c9c0b4360 0x7f5c9c06bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.178+0000 7f5c9b7fe640 1 -- 172.21.15.98:0/2280381302 >> v1:172.21.15.139:6789/0 conn(0x7f5c9c06c1d0 legacy=0x7f5c9c15efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:47.179 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.178+0000 7f5c9b7fe640 1 -- 172.21.15.98:0/2280381302 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c9c0ccf40 con 0x7f5c9c158fd0 2024-09-19T08:47:47.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.182+0000 7f5c9b7fe640 1 --2- 172.21.15.98:0/2280381302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c9c158fd0 0x7f5c9c0b3e20 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c009870 tx=0x7f5c8c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c6e438c8477a71 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:47.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.182+0000 7f5c9a7fc640 1 -- 172.21.15.98:0/2280381302 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c0ab620 con 0x7f5c9c158fd0 2024-09-19T08:47:47.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.182+0000 7f5c9a7fc640 1 -- 172.21.15.98:0/2280381302 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c8c0b0070 con 0x7f5c9c158fd0 2024-09-19T08:47:47.181 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.182+0000 7f5c9a7fc640 1 -- 172.21.15.98:0/2280381302 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c0ab9e0 con 0x7f5c9c158fd0 2024-09-19T08:47:47.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.182+0000 7f5ca1f2e640 1 -- 172.21.15.98:0/2280381302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c9c158fd0 msgr2=0x7f5c9c0b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:47.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.182+0000 7f5ca1f2e640 1 --2- 172.21.15.98:0/2280381302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c9c158fd0 0x7f5c9c0b3e20 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c009870 tx=0x7f5c8c01c460 comp rx=0 tx=0).stop 2024-09-19T08:47:47.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.182+0000 7f5ca1f2e640 1 -- 172.21.15.98:0/2280381302 shutdown_connections 2024-09-19T08:47:47.182 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.182+0000 7f5ca1f2e640 1 --2- 172.21.15.98:0/2280381302 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c9c0b4360 0x7f5c9c06bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.182+0000 7f5ca1f2e640 1 --2- 172.21.15.98:0/2280381302 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c9c158fd0 0x7f5c9c0b3e20 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.182+0000 7f5ca1f2e640 1 -- 172.21.15.98:0/2280381302 >> 172.21.15.98:0/2280381302 conn(0x7f5c9c0b2010 msgr2=0x7f5c9c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:47.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.182+0000 7f5ca1f2e640 1 -- 172.21.15.98:0/2280381302 shutdown_connections 2024-09-19T08:47:47.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.182+0000 7f5ca1f2e640 1 -- 172.21.15.98:0/2280381302 wait complete. 2024-09-19T08:47:47.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.182+0000 7f5ca1f2e640 1 Processor -- start 2024-09-19T08:47:47.183 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.182+0000 7f5ca1f2e640 1 -- start start 2024-09-19T08:47:47.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.186+0000 7f5ca1f2e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c9c06c1d0 0x7f5c9c150e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.186+0000 7f5ca1f2e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c9c0b4360 0x7f5c9c1515d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.186+0000 7f5ca1f2e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c9c158fd0 0x7f5c9c151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.186+0000 7f5ca1f2e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c15c920 con 0x7f5c9c0b4360 2024-09-19T08:47:47.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.186+0000 7f5ca1f2e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c15c7a0 con 0x7f5c9c158fd0 2024-09-19T08:47:47.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.186+0000 7f5ca1f2e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f5c9c166100 con 0x7f5c9c06c1d0 2024-09-19T08:47:47.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.186+0000 7f5c9affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c9c0b4360 0x7f5c9c1515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.185 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.186+0000 7f5c9affd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c9c0b4360 0x7f5c9c1515d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37462/0 (socket says 172.21.15.98:37462) 2024-09-19T08:47:47.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.186+0000 7f5c9affd640 1 -- 172.21.15.98:0/4014046699 learned_addr learned my addr 172.21.15.98:0/4014046699 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:47.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.186+0000 7f5c9b7fe640 1 --2- 172.21.15.98:0/4014046699 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c9c06c1d0 0x7f5c9c150e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.186+0000 7f5c9bfff640 1 --2- 172.21.15.98:0/4014046699 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c9c158fd0 0x7f5c9c151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.186+0000 7f5c9affd640 1 -- 172.21.15.98:0/4014046699 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c9c06c1d0 msgr2=0x7f5c9c150e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:47.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.186+0000 7f5c9affd640 1 --2- 172.21.15.98:0/4014046699 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c9c06c1d0 0x7f5c9c150e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.186+0000 7f5c9affd640 1 -- 172.21.15.98:0/4014046699 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c9c158fd0 msgr2=0x7f5c9c151b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:47.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.186+0000 7f5c9affd640 1 --2- 172.21.15.98:0/4014046699 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c9c158fd0 0x7f5c9c151b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.186+0000 7f5c9affd640 1 -- 172.21.15.98:0/4014046699 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c9c156d90 con 0x7f5c9c0b4360 2024-09-19T08:47:47.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.186+0000 7f5c9affd640 1 --2- 172.21.15.98:0/4014046699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c9c0b4360 0x7f5c9c1515d0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f5c9000c7f0 tx=0x7f5c9000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:47.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.186+0000 7f5c83fff640 1 -- 172.21.15.98:0/4014046699 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c90015070 con 0x7f5c9c0b4360 2024-09-19T08:47:47.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.186+0000 7f5c83fff640 1 -- 172.21.15.98:0/4014046699 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c90004240 con 0x7f5c9c0b4360 2024-09-19T08:47:47.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.186+0000 7f5ca1f2e640 1 -- 172.21.15.98:0/4014046699 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c9c157070 con 0x7f5c9c0b4360 2024-09-19T08:47:47.186 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.186+0000 7f5c83fff640 1 -- 172.21.15.98:0/4014046699 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c90003620 con 0x7f5c9c0b4360 2024-09-19T08:47:47.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.186+0000 7f5ca1f2e640 1 -- 172.21.15.98:0/4014046699 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c9c157630 con 0x7f5c9c0b4360 2024-09-19T08:47:47.187 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.186+0000 7f5ca1f2e640 1 -- 172.21.15.98:0/4014046699 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c9c157830 con 0x7f5c9c0b4360 2024-09-19T08:47:47.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.186+0000 7f5c83fff640 1 -- 172.21.15.98:0/4014046699 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f5c900039d0 con 0x7f5c9c0b4360 2024-09-19T08:47:47.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.186+0000 7f5c83fff640 1 --2- 172.21.15.98:0/4014046699 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5c88068400 0x7f5c8806a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.186+0000 7f5c83fff640 1 -- 172.21.15.98:0/4014046699 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7f5c90084a20 con 0x7f5c9c0b4360 2024-09-19T08:47:47.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.186+0000 7f5c9b7fe640 1 --2- 172.21.15.98:0/4014046699 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5c88068400 0x7f5c8806a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.188 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.190+0000 7f5c9b7fe640 1 --2- 172.21.15.98:0/4014046699 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5c88068400 0x7f5c8806a8c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c0af000 tx=0x7f5c8c0a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:47.191 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.190+0000 7f5c83fff640 1 -- 172.21.15.98:0/4014046699 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f5c90056cc0 con 0x7f5c9c0b4360 2024-09-19T08:47:47.352 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.354+0000 7f5ca1f2e640 1 -- 172.21.15.98:0/4014046699 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f5c9c06bc90 con 0x7f5c88068400 2024-09-19T08:47:47.353 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.354+0000 7f5c83fff640 1 -- 172.21.15.98:0/4014046699 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5c9c06bc90 con 0x7f5c88068400 2024-09-19T08:47:47.353 INFO:teuthology.orchestra.run.smithi098.stderr:instructed osd(s) 0 to deep-scrub 2024-09-19T08:47:47.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.354+0000 7f5ca1f2e640 1 -- 172.21.15.98:0/4014046699 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5c88068400 msgr2=0x7f5c8806a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:47.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.354+0000 7f5ca1f2e640 1 --2- 172.21.15.98:0/4014046699 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5c88068400 0x7f5c8806a8c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c0af000 tx=0x7f5c8c0a7000 comp rx=0 tx=0).stop 2024-09-19T08:47:47.355 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.354+0000 7f5ca1f2e640 1 -- 172.21.15.98:0/4014046699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c9c0b4360 msgr2=0x7f5c9c1515d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:47.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.354+0000 7f5ca1f2e640 1 --2- 172.21.15.98:0/4014046699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c9c0b4360 0x7f5c9c1515d0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f5c9000c7f0 tx=0x7f5c9000ccb0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.354+0000 7f5ca1f2e640 1 -- 172.21.15.98:0/4014046699 shutdown_connections 2024-09-19T08:47:47.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.354+0000 7f5ca1f2e640 1 --2- 172.21.15.98:0/4014046699 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f5c88068400 0x7f5c8806a8c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.354+0000 7f5ca1f2e640 1 --2- 172.21.15.98:0/4014046699 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5c9c158fd0 0x7f5c9c151b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.354+0000 7f5ca1f2e640 1 --2- 172.21.15.98:0/4014046699 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f5c9c0b4360 0x7f5c9c1515d0 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.354+0000 7f5ca1f2e640 1 --2- 172.21.15.98:0/4014046699 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f5c9c06c1d0 0x7f5c9c150e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.354+0000 7f5ca1f2e640 1 -- 172.21.15.98:0/4014046699 >> 172.21.15.98:0/4014046699 conn(0x7f5c9c0b2010 msgr2=0x7f5c9c165670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:47.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.354+0000 7f5ca1f2e640 1 -- 172.21.15.98:0/4014046699 shutdown_connections 2024-09-19T08:47:47.356 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.354+0000 7f5ca1f2e640 1 -- 172.21.15.98:0/4014046699 wait complete. 2024-09-19T08:47:47.372 INFO:tasks.ceph:Scrubbing osd.1 2024-09-19T08:47:47.372 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-19T08:47:47.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.490+0000 7f32cfe48640 1 Processor -- start 2024-09-19T08:47:47.488 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.490+0000 7f32cfe48640 1 -- start start 2024-09-19T08:47:47.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.490+0000 7f32cfe48640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f32c80b7770 0x7f32c80b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.490+0000 7f32cfe48640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32c80cfd20 con 0x7f32c80d3a60 2024-09-19T08:47:47.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.490+0000 7f32cfe48640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32c80d29c0 con 0x7f32c815e720 2024-09-19T08:47:47.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.490+0000 7f32cfe48640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f32c80d2ca0 con 0x7f32c80b7770 2024-09-19T08:47:47.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.490+0000 7f32ce3e5640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f32c80d3a60 0x7f32c81623b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:58870/0 (socket says 172.21.15.98:58870) 2024-09-19T08:47:47.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.490+0000 7f32ce3e5640 1 -- 172.21.15.98:0/3048176426 learned_addr learned my addr 172.21.15.98:0/3048176426 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:47.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.490+0000 7f32cd3e3640 1 --2- 172.21.15.98:0/3048176426 >> v2:172.21.15.139:3300/0 conn(0x7f32c80b7770 0x7f32c80b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.490+0000 7f32ccbe2640 1 -- 172.21.15.98:0/3048176426 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 492234861 0 0) 0x7f32c80cfd20 con 0x7f32c80d3a60 2024-09-19T08:47:47.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.490+0000 7f32ccbe2640 1 -- 172.21.15.98:0/3048176426 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32b4003530 con 0x7f32c80d3a60 2024-09-19T08:47:47.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.490+0000 7f32ccbe2640 1 -- 172.21.15.98:0/3048176426 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1655970738 0 0) 0x7f32b4003530 con 0x7f32c80d3a60 2024-09-19T08:47:47.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.490+0000 7f32ccbe2640 1 -- 172.21.15.98:0/3048176426 >> v2:172.21.15.139:3300/0 conn(0x7f32c80b7770 msgr2=0x7f32c80b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:47.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.490+0000 7f32ccbe2640 1 --2- 172.21.15.98:0/3048176426 >> v2:172.21.15.139:3300/0 conn(0x7f32c80b7770 0x7f32c80b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.490+0000 7f32ccbe2640 1 -- 172.21.15.98:0/3048176426 >> v1:172.21.15.139:6789/0 conn(0x7f32c815e720 legacy=0x7f32c80cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:47.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.490+0000 7f32ccbe2640 1 -- 172.21.15.98:0/3048176426 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c80d2f80 con 0x7f32c80d3a60 2024-09-19T08:47:47.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.490+0000 7f32ccbe2640 1 -- 172.21.15.98:0/3048176426 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7f32c0001a00 con 0x7f32c80d3a60 2024-09-19T08:47:47.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.490+0000 7f32ccbe2640 1 -- 172.21.15.98:0/3048176426 >> v1:172.21.15.98:6789/0 conn(0x7f32c80d3a60 legacy=0x7f32c81623b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:47.492 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.490+0000 7f32ccbe2640 1 --2- 172.21.15.98:0/3048176426 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32b40039c0 0x7f32b4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.490+0000 7f32ccbe2640 1 --2- 172.21.15.98:0/3048176426 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32b4004810 0x7f32b4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.490+0000 7f32ccbe2640 1 --2- 172.21.15.98:0/3048176426 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32b40051d0 0x7f32b400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.490+0000 7f32ccbe2640 1 -- 172.21.15.98:0/3048176426 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f32c80d2ca0 con 0x7f32b40051d0 2024-09-19T08:47:47.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.490+0000 7f32ccbe2640 1 -- 172.21.15.98:0/3048176426 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f32b400c070 con 0x7f32b4004810 2024-09-19T08:47:47.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.490+0000 7f32ccbe2640 1 -- 172.21.15.98:0/3048176426 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f32b400c270 con 0x7f32b40039c0 2024-09-19T08:47:47.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.490+0000 7f32ce3e5640 1 --2- 172.21.15.98:0/3048176426 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32b40051d0 0x7f32b400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.490+0000 7f32cdbe4640 1 --2- 172.21.15.98:0/3048176426 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32b40039c0 0x7f32b4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.490+0000 7f32ce3e5640 1 -- 172.21.15.98:0/3048176426 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32b40039c0 msgr2=0x7f32b4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:47.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.490+0000 7f32cd3e3640 1 --2- 172.21.15.98:0/3048176426 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32b4004810 0x7f32b4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.490+0000 7f32ce3e5640 1 --2- 172.21.15.98:0/3048176426 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32b40039c0 0x7f32b4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.490+0000 7f32ce3e5640 1 -- 172.21.15.98:0/3048176426 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32b4004810 msgr2=0x7f32b4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:47.493 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.490+0000 7f32ce3e5640 1 --2- 172.21.15.98:0/3048176426 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32b4004810 0x7f32b4004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.490+0000 7f32ce3e5640 1 -- 172.21.15.98:0/3048176426 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b4003530 con 0x7f32b40051d0 2024-09-19T08:47:47.494 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.490+0000 7f32cdbe4640 1 --2- 172.21.15.98:0/3048176426 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32b40039c0 0x7f32b4003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:47.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.494+0000 7f32ce3e5640 1 -- 172.21.15.98:0/3048176426 reap_dead start 2024-09-19T08:47:47.496 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.494+0000 7f32ce3e5640 1 --2- 172.21.15.98:0/3048176426 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32b40051d0 0x7f32b400b9d0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f32c000a850 tx=0x7f32c001d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad9f853fd14f03c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:47.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.494+0000 7f32ccbe2640 1 -- 172.21.15.98:0/3048176426 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c000a4a0 con 0x7f32b40051d0 2024-09-19T08:47:47.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.494+0000 7f32ccbe2640 1 -- 172.21.15.98:0/3048176426 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32c00ad650 con 0x7f32b40051d0 2024-09-19T08:47:47.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.494+0000 7f32ccbe2640 1 -- 172.21.15.98:0/3048176426 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c00ada40 con 0x7f32b40051d0 2024-09-19T08:47:47.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.494+0000 7f32cfe48640 1 -- 172.21.15.98:0/3048176426 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32b40051d0 msgr2=0x7f32b400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:47.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.494+0000 7f32cfe48640 1 --2- 172.21.15.98:0/3048176426 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32b40051d0 0x7f32b400b9d0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f32c000a850 tx=0x7f32c001d3a0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.494+0000 7f32cfe48640 1 -- 172.21.15.98:0/3048176426 shutdown_connections 2024-09-19T08:47:47.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.494+0000 7f32cfe48640 1 --2- 172.21.15.98:0/3048176426 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32b40051d0 0x7f32b400b9d0 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.494+0000 7f32cfe48640 1 -- 172.21.15.98:0/3048176426 >> 172.21.15.98:0/3048176426 conn(0x7f32c80d4640 msgr2=0x7f32c80d4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:47.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.494+0000 7f32cfe48640 1 -- 172.21.15.98:0/3048176426 shutdown_connections 2024-09-19T08:47:47.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.494+0000 7f32cfe48640 1 -- 172.21.15.98:0/3048176426 wait complete. 2024-09-19T08:47:47.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.494+0000 7f32cfe48640 1 Processor -- start 2024-09-19T08:47:47.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.494+0000 7f32cfe48640 1 -- start start 2024-09-19T08:47:47.497 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.494+0000 7f32cfe48640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32b40051d0 0x7f32c8160490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.494+0000 7f32cfe48640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32c81567e0 0x7f32c8156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.494+0000 7f32cfe48640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32c80c9450 0x7f32c80c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.494+0000 7f32cfe48640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f32c81630a0 con 0x7f32b40051d0 2024-09-19T08:47:47.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.494+0000 7f32cfe48640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f32c80d3540 con 0x7f32c80c9450 2024-09-19T08:47:47.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.494+0000 7f32cfe48640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f32c8163220 con 0x7f32c81567e0 2024-09-19T08:47:47.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.494+0000 7f32cdbe4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32b40051d0 0x7f32c8160490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.494+0000 7f32cdbe4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32b40051d0 0x7f32c8160490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:37480/0 (socket says 172.21.15.98:37480) 2024-09-19T08:47:47.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.494+0000 7f32cdbe4640 1 -- 172.21.15.98:0/2594630316 learned_addr learned my addr 172.21.15.98:0/2594630316 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:47.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.494+0000 7f32ce3e5640 1 --2- 172.21.15.98:0/2594630316 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32c80c9450 0x7f32c80c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.494+0000 7f32cdbe4640 1 -- 172.21.15.98:0/2594630316 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32c81567e0 msgr2=0x7f32c8156c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:47.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.494+0000 7f32cd3e3640 1 --2- 172.21.15.98:0/2594630316 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32c81567e0 0x7f32c8156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.498+0000 7f32cdbe4640 1 --2- 172.21.15.98:0/2594630316 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f32c81567e0 0x7f32c8156c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.498+0000 7f32cdbe4640 1 -- 172.21.15.98:0/2594630316 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32c80c9450 msgr2=0x7f32c80c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:47.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.498+0000 7f32cdbe4640 1 --2- 172.21.15.98:0/2594630316 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f32c80c9450 0x7f32c80c98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.498+0000 7f32cdbe4640 1 -- 172.21.15.98:0/2594630316 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c81571a0 con 0x7f32b40051d0 2024-09-19T08:47:47.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.498+0000 7f32cdbe4640 1 --2- 172.21.15.98:0/2594630316 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32b40051d0 0x7f32c8160490 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f32c400d570 tx=0x7f32c400da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:47.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.498+0000 7f32be7fc640 1 -- 172.21.15.98:0/2594630316 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c4005480 con 0x7f32b40051d0 2024-09-19T08:47:47.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.498+0000 7f32be7fc640 1 -- 172.21.15.98:0/2594630316 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32c4017070 con 0x7f32b40051d0 2024-09-19T08:47:47.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.498+0000 7f32be7fc640 1 -- 172.21.15.98:0/2594630316 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c4003840 con 0x7f32b40051d0 2024-09-19T08:47:47.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.498+0000 7f32cfe48640 1 -- 172.21.15.98:0/2594630316 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32c80c8690 con 0x7f32b40051d0 2024-09-19T08:47:47.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.498+0000 7f32cfe48640 1 -- 172.21.15.98:0/2594630316 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32c80c8c50 con 0x7f32b40051d0 2024-09-19T08:47:47.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.498+0000 7f32cfe48640 1 -- 172.21.15.98:0/2594630316 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f32c80c8440 con 0x7f32b40051d0 2024-09-19T08:47:47.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.498+0000 7f32be7fc640 1 -- 172.21.15.98:0/2594630316 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f32c401c460 con 0x7f32b40051d0 2024-09-19T08:47:47.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.498+0000 7f32be7fc640 1 --2- 172.21.15.98:0/2594630316 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f32ac068400 0x7f32ac06a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.498+0000 7f32be7fc640 1 -- 172.21.15.98:0/2594630316 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7f32c4084e20 con 0x7f32b40051d0 2024-09-19T08:47:47.500 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.498+0000 7f32be7fc640 1 --2- 172.21.15.98:0/2594630316 >> [v2:172.21.15.98:6817/1538482309,v1:172.21.15.98:6820/1538482309] conn(0x7f32ac06e2c0 0x7f32ac0706e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.498+0000 7f32be7fc640 1 -- 172.21.15.98:0/2594630316 --> [v2:172.21.15.98:6817/1538482309,v1:172.21.15.98:6820/1538482309] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f32c401fe50 con 0x7f32ac06e2c0 2024-09-19T08:47:47.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.498+0000 7f32be7fc640 1 -- 172.21.15.98:0/2594630316 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=173) ==== 24+0+0 (secure 0 0 0) 0x7f32c4087a70 con 0x7f32b40051d0 2024-09-19T08:47:47.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.498+0000 7f32cd3e3640 1 --2- 172.21.15.98:0/2594630316 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f32ac068400 0x7f32ac06a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.498+0000 7f32ce3e5640 1 --2- 172.21.15.98:0/2594630316 >> [v2:172.21.15.98:6817/1538482309,v1:172.21.15.98:6820/1538482309] conn(0x7f32ac06e2c0 0x7f32ac0706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.498+0000 7f32ce3e5640 1 --2- 172.21.15.98:0/2594630316 >> [v2:172.21.15.98:6817/1538482309,v1:172.21.15.98:6820/1538482309] conn(0x7f32ac06e2c0 0x7f32ac0706e0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:47.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.498+0000 7f32cd3e3640 1 --2- 172.21.15.98:0/2594630316 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f32ac068400 0x7f32ac06a8c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f32c8161900 tx=0x7f32b8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:47.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.498+0000 7f32be7fc640 1 -- 172.21.15.98:0/2594630316 <== osd.1 v2:172.21.15.98:6817/1538482309 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f32c401fe50 con 0x7f32ac06e2c0 2024-09-19T08:47:47.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.510+0000 7f32cfe48640 1 -- 172.21.15.98:0/2594630316 --> [v2:172.21.15.98:6817/1538482309,v1:172.21.15.98:6820/1538482309] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f32c815eb00 con 0x7f32ac06e2c0 2024-09-19T08:47:47.511 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.514+0000 7f32be7fc640 1 -- 172.21.15.98:0/2594630316 <== osd.1 v2:172.21.15.98:6817/1538482309 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f32c815eb00 con 0x7f32ac06e2c0 2024-09-19T08:47:47.512 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:47:47.512 INFO:teuthology.orchestra.run.smithi098.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T08:47:47.512 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:47:47.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.514+0000 7f32cfe48640 1 -- 172.21.15.98:0/2594630316 >> [v2:172.21.15.98:6817/1538482309,v1:172.21.15.98:6820/1538482309] conn(0x7f32ac06e2c0 msgr2=0x7f32ac0706e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:47.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.514+0000 7f32cfe48640 1 --2- 172.21.15.98:0/2594630316 >> [v2:172.21.15.98:6817/1538482309,v1:172.21.15.98:6820/1538482309] conn(0x7f32ac06e2c0 0x7f32ac0706e0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.514+0000 7f32cfe48640 1 -- 172.21.15.98:0/2594630316 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f32ac068400 msgr2=0x7f32ac06a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:47.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.514+0000 7f32cfe48640 1 --2- 172.21.15.98:0/2594630316 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f32ac068400 0x7f32ac06a8c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f32c8161900 tx=0x7f32b8008040 comp rx=0 tx=0).stop 2024-09-19T08:47:47.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.514+0000 7f32cfe48640 1 -- 172.21.15.98:0/2594630316 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32b40051d0 msgr2=0x7f32c8160490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:47.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.514+0000 7f32cfe48640 1 --2- 172.21.15.98:0/2594630316 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f32b40051d0 0x7f32c8160490 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f32c400d570 tx=0x7f32c400da30 comp rx=0 tx=0).stop 2024-09-19T08:47:47.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.514+0000 7f32ce3e5640 1 -- 172.21.15.98:0/2594630316 reap_dead start 2024-09-19T08:47:47.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.514+0000 7f32cfe48640 1 -- 172.21.15.98:0/2594630316 shutdown_connections 2024-09-19T08:47:47.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.514+0000 7f32cfe48640 1 -- 172.21.15.98:0/2594630316 >> 172.21.15.98:0/2594630316 conn(0x7f32c80d4640 msgr2=0x7f32c80b59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:47.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.514+0000 7f32cfe48640 1 -- 172.21.15.98:0/2594630316 shutdown_connections 2024-09-19T08:47:47.513 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.514+0000 7f32cfe48640 1 -- 172.21.15.98:0/2594630316 wait complete. 2024-09-19T08:47:47.526 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-19T08:47:47.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.638+0000 7fec9c828640 1 Processor -- start 2024-09-19T08:47:47.637 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.638+0000 7fec9c828640 1 -- start start 2024-09-19T08:47:47.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.638+0000 7fec9c828640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fec980b0280 0x7fec9806d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.638 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.638+0000 7fec9c828640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fec98074c70 0x7fec980b6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.638+0000 7fec9c828640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec980b6d90 0x7fec980d1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.638+0000 7fec9c828640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fec980b7200 con 0x7fec98074c70 2024-09-19T08:47:47.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.638+0000 7fec9c828640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fec98160350 con 0x7fec980b0280 2024-09-19T08:47:47.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.638+0000 7fec9c828640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fec98160630 con 0x7fec980b6d90 2024-09-19T08:47:47.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.638+0000 7fec95d9b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fec98074c70 0x7fec980b6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.638+0000 7fec95d9b640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fec98074c70 0x7fec980b6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36574/0 (socket says 172.21.15.98:36574) 2024-09-19T08:47:47.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.638+0000 7fec95d9b640 1 -- 172.21.15.98:0/2850180191 learned_addr learned my addr 172.21.15.98:0/2850180191 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:47.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.638+0000 7fec96d9d640 1 --2- 172.21.15.98:0/2850180191 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec980b6d90 0x7fec980d1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.638+0000 7fec9659c640 1 --2- 172.21.15.98:0/2850180191 >> v2:172.21.15.139:3300/0 conn(0x7fec980b0280 0x7fec9806d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.638+0000 7fec95d9b640 1 -- 172.21.15.98:0/2850180191 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec980b6d90 msgr2=0x7fec980d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:47.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.638+0000 7fec95d9b640 1 --2- 172.21.15.98:0/2850180191 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec980b6d90 0x7fec980d1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.638+0000 7fec95d9b640 1 -- 172.21.15.98:0/2850180191 >> v2:172.21.15.139:3300/0 conn(0x7fec980b0280 msgr2=0x7fec9806d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:47.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.638+0000 7fec95d9b640 1 --2- 172.21.15.98:0/2850180191 >> v2:172.21.15.139:3300/0 conn(0x7fec980b0280 0x7fec9806d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.638+0000 7fec95d9b640 1 -- 172.21.15.98:0/2850180191 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec98160910 con 0x7fec98074c70 2024-09-19T08:47:47.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.638+0000 7fec9659c640 1 --2- 172.21.15.98:0/2850180191 >> v2:172.21.15.139:3300/0 conn(0x7fec980b0280 0x7fec9806d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:47.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.642+0000 7fec95d9b640 1 --2- 172.21.15.98:0/2850180191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec98074c70 0x7fec980b6850 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fec98069910 tx=0x7fec8801c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00d5e5ebb2b6b2f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:47.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.642+0000 7fec9559a640 1 -- 172.21.15.98:0/2850180191 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec880b0070 con 0x7fec98074c70 2024-09-19T08:47:47.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.642+0000 7fec9559a640 1 -- 172.21.15.98:0/2850180191 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec880ab600 con 0x7fec98074c70 2024-09-19T08:47:47.640 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.642+0000 7fec9559a640 1 -- 172.21.15.98:0/2850180191 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec880ab9d0 con 0x7fec98074c70 2024-09-19T08:47:47.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.642+0000 7fec9c828640 1 -- 172.21.15.98:0/2850180191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec98074c70 msgr2=0x7fec980b6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:47.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.642+0000 7fec9c828640 1 --2- 172.21.15.98:0/2850180191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec98074c70 0x7fec980b6850 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fec98069910 tx=0x7fec8801c1d0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.642+0000 7fec9c828640 1 -- 172.21.15.98:0/2850180191 shutdown_connections 2024-09-19T08:47:47.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.642+0000 7fec9c828640 1 --2- 172.21.15.98:0/2850180191 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec98074c70 0x7fec980b6850 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.642+0000 7fec9c828640 1 --2- 172.21.15.98:0/2850180191 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec980b6d90 0x7fec980d1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.642+0000 7fec9c828640 1 --2- 172.21.15.98:0/2850180191 >> v2:172.21.15.139:3300/0 conn(0x7fec980b0280 0x7fec9806d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.642+0000 7fec9c828640 1 -- 172.21.15.98:0/2850180191 >> 172.21.15.98:0/2850180191 conn(0x7fec98073170 msgr2=0x7fec98073580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:47.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.642+0000 7fec9c828640 1 -- 172.21.15.98:0/2850180191 shutdown_connections 2024-09-19T08:47:47.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.642+0000 7fec9c828640 1 -- 172.21.15.98:0/2850180191 wait complete. 2024-09-19T08:47:47.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.642+0000 7fec9c828640 1 Processor -- start 2024-09-19T08:47:47.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.642+0000 7fec9c828640 1 -- start start 2024-09-19T08:47:47.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.642+0000 7fec9c828640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec98074c70 0x7fec98158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.642+0000 7fec9c828640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec980b0280 0x7fec9815c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.642+0000 7fec9c828640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec980b6d90 0x7fec9809a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.642+0000 7fec9c828640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fec980ae4c0 con 0x7fec98074c70 2024-09-19T08:47:47.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.642+0000 7fec9c828640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fec980ae340 con 0x7fec980b0280 2024-09-19T08:47:47.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.642+0000 7fec9c828640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fec980ae640 con 0x7fec980b6d90 2024-09-19T08:47:47.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.642+0000 7fec9659c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec98074c70 0x7fec98158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.642+0000 7fec9659c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec98074c70 0x7fec98158f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36586/0 (socket says 172.21.15.98:36586) 2024-09-19T08:47:47.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.642+0000 7fec9659c640 1 -- 172.21.15.98:0/3050338045 learned_addr learned my addr 172.21.15.98:0/3050338045 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:47.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.646+0000 7fec95d9b640 1 --2- 172.21.15.98:0/3050338045 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec980b0280 0x7fec9815c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.646+0000 7fec96d9d640 1 --2- 172.21.15.98:0/3050338045 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec980b6d90 0x7fec9809a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.646+0000 7fec9659c640 1 -- 172.21.15.98:0/3050338045 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec980b6d90 msgr2=0x7fec9809a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:47.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.646+0000 7fec9659c640 1 --2- 172.21.15.98:0/3050338045 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec980b6d90 0x7fec9809a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.646+0000 7fec9659c640 1 -- 172.21.15.98:0/3050338045 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec980b0280 msgr2=0x7fec9815c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:47.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.646+0000 7fec9659c640 1 --2- 172.21.15.98:0/3050338045 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec980b0280 0x7fec9815c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.646+0000 7fec9659c640 1 -- 172.21.15.98:0/3050338045 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec981581d0 con 0x7fec98074c70 2024-09-19T08:47:47.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.646+0000 7fec9659c640 1 --2- 172.21.15.98:0/3050338045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec98074c70 0x7fec98158f50 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00c7c0 tx=0x7fec8c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:47.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.646+0000 7fec86ffd640 1 -- 172.21.15.98:0/3050338045 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c015070 con 0x7fec98074c70 2024-09-19T08:47:47.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.646+0000 7fec86ffd640 1 -- 172.21.15.98:0/3050338045 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec8c007a30 con 0x7fec98074c70 2024-09-19T08:47:47.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.646+0000 7fec9c828640 1 -- 172.21.15.98:0/3050338045 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec9809ad40 con 0x7fec98074c70 2024-09-19T08:47:47.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.646+0000 7fec86ffd640 1 -- 172.21.15.98:0/3050338045 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c007e00 con 0x7fec98074c70 2024-09-19T08:47:47.646 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.646+0000 7fec9c828640 1 -- 172.21.15.98:0/3050338045 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec98158750 con 0x7fec98074c70 2024-09-19T08:47:47.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.646+0000 7fec9c828640 1 -- 172.21.15.98:0/3050338045 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec980b6850 con 0x7fec98074c70 2024-09-19T08:47:47.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.646+0000 7fec86ffd640 1 -- 172.21.15.98:0/3050338045 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7fec8c018070 con 0x7fec98074c70 2024-09-19T08:47:47.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.646+0000 7fec86ffd640 1 --2- 172.21.15.98:0/3050338045 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fec78068400 0x7fec7806a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.646+0000 7fec86ffd640 1 -- 172.21.15.98:0/3050338045 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7fec8c084730 con 0x7fec98074c70 2024-09-19T08:47:47.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.646+0000 7fec95d9b640 1 --2- 172.21.15.98:0/3050338045 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fec78068400 0x7fec7806a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.646+0000 7fec95d9b640 1 --2- 172.21.15.98:0/3050338045 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fec78068400 0x7fec7806a8c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fec880ad000 tx=0x7fec88008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:47.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.650+0000 7fec86ffd640 1 -- 172.21.15.98:0/3050338045 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fec8c056ad0 con 0x7fec98074c70 2024-09-19T08:47:47.798 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.798+0000 7fec9c828640 1 -- 172.21.15.98:0/3050338045 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fec980b5750 con 0x7fec78068400 2024-09-19T08:47:47.799 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.798+0000 7fec86ffd640 1 -- 172.21.15.98:0/3050338045 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fec980b5750 con 0x7fec78068400 2024-09-19T08:47:47.799 INFO:teuthology.orchestra.run.smithi098.stderr:instructed osd(s) 1 to deep-scrub 2024-09-19T08:47:47.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.802+0000 7fec9c828640 1 -- 172.21.15.98:0/3050338045 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fec78068400 msgr2=0x7fec7806a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:47.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.802+0000 7fec9c828640 1 --2- 172.21.15.98:0/3050338045 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fec78068400 0x7fec7806a8c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fec880ad000 tx=0x7fec88008040 comp rx=0 tx=0).stop 2024-09-19T08:47:47.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.802+0000 7fec9c828640 1 -- 172.21.15.98:0/3050338045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec98074c70 msgr2=0x7fec98158f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:47.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.802+0000 7fec9c828640 1 --2- 172.21.15.98:0/3050338045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec98074c70 0x7fec98158f50 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00c7c0 tx=0x7fec8c00cc80 comp rx=0 tx=0).stop 2024-09-19T08:47:47.801 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.802+0000 7fec9c828640 1 -- 172.21.15.98:0/3050338045 shutdown_connections 2024-09-19T08:47:47.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.802+0000 7fec9c828640 1 --2- 172.21.15.98:0/3050338045 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fec78068400 0x7fec7806a8c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.802+0000 7fec9c828640 1 --2- 172.21.15.98:0/3050338045 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fec980b6d90 0x7fec9809a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.802+0000 7fec9c828640 1 --2- 172.21.15.98:0/3050338045 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fec980b0280 0x7fec9815c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.802+0000 7fec9c828640 1 --2- 172.21.15.98:0/3050338045 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fec98074c70 0x7fec98158f50 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.802+0000 7fec9c828640 1 -- 172.21.15.98:0/3050338045 >> 172.21.15.98:0/3050338045 conn(0x7fec98073170 msgr2=0x7fec9815ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:47.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.802+0000 7fec9c828640 1 -- 172.21.15.98:0/3050338045 shutdown_connections 2024-09-19T08:47:47.802 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.802+0000 7fec9c828640 1 -- 172.21.15.98:0/3050338045 wait complete. 2024-09-19T08:47:47.816 INFO:tasks.ceph:Scrubbing osd.2 2024-09-19T08:47:47.816 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-19T08:47:47.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.934+0000 7facd93aa640 1 Processor -- start 2024-09-19T08:47:47.932 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.934+0000 7facd93aa640 1 -- start start 2024-09-19T08:47:47.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.934+0000 7facd93aa640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7facd40cf700 0x7facd415ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.934+0000 7facd93aa640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7facd406d4c0 0x7facd406d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.934+0000 7facd93aa640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facd41608c0 con 0x7facd415c290 2024-09-19T08:47:47.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.934+0000 7facd93aa640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7facd406de70 con 0x7facd406d4c0 2024-09-19T08:47:47.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.934+0000 7facd93aa640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7facd4160e80 con 0x7facd40cf700 2024-09-19T08:47:47.935 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.934+0000 7facd2ffd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7facd415c290 0x7facd40cd9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:44482/0 (socket says 172.21.15.98:44482) 2024-09-19T08:47:47.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.934+0000 7facd2ffd640 1 -- 172.21.15.98:0/2781575942 learned_addr learned my addr 172.21.15.98:0/2781575942 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:47.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.934+0000 7facd37fe640 1 --2- 172.21.15.98:0/2781575942 >> v2:172.21.15.139:3300/0 conn(0x7facd406d4c0 0x7facd406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.934+0000 7facd27fc640 1 --2- 172.21.15.98:0/2781575942 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7facd40cf700 0x7facd415ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.934+0000 7facd1ffb640 1 -- 172.21.15.98:0/2781575942 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2194345555 0 0) 0x7facd41608c0 con 0x7facd415c290 2024-09-19T08:47:47.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.934+0000 7facd1ffb640 1 -- 172.21.15.98:0/2781575942 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7facbc003530 con 0x7facd415c290 2024-09-19T08:47:47.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.934+0000 7facd1ffb640 1 -- 172.21.15.98:0/2781575942 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 615015786 0 0) 0x7facbc003530 con 0x7facd415c290 2024-09-19T08:47:47.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.934+0000 7facd1ffb640 1 -- 172.21.15.98:0/2781575942 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7facd40cf700 msgr2=0x7facd415ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:47.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.934+0000 7facd1ffb640 1 --2- 172.21.15.98:0/2781575942 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7facd40cf700 0x7facd415ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.934+0000 7facd1ffb640 1 -- 172.21.15.98:0/2781575942 >> v2:172.21.15.139:3300/0 conn(0x7facd406d4c0 msgr2=0x7facd406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:47.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.934+0000 7facd1ffb640 1 --2- 172.21.15.98:0/2781575942 >> v2:172.21.15.139:3300/0 conn(0x7facd406d4c0 0x7facd406d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.934+0000 7facd1ffb640 1 -- 172.21.15.98:0/2781575942 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd4160ba0 con 0x7facd415c290 2024-09-19T08:47:47.936 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.934+0000 7facd1ffb640 1 -- 172.21.15.98:0/2781575942 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7facc4001680 con 0x7facd415c290 2024-09-19T08:47:47.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.934+0000 7facd1ffb640 1 -- 172.21.15.98:0/2781575942 >> v1:172.21.15.98:6789/0 conn(0x7facd415c290 legacy=0x7facd40cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:47.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.934+0000 7facd1ffb640 1 --2- 172.21.15.98:0/2781575942 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facbc0039c0 0x7facbc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.934+0000 7facd1ffb640 1 --2- 172.21.15.98:0/2781575942 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facbc004810 0x7facbc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.934+0000 7facd1ffb640 1 --2- 172.21.15.98:0/2781575942 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7facd40cf700 0x7facbc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.934+0000 7facd1ffb640 1 -- 172.21.15.98:0/2781575942 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7facd406de70 con 0x7facbc0039c0 2024-09-19T08:47:47.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.934+0000 7facd1ffb640 1 -- 172.21.15.98:0/2781575942 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7facbc00b9d0 con 0x7facbc004810 2024-09-19T08:47:47.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.934+0000 7facd1ffb640 1 -- 172.21.15.98:0/2781575942 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7facbc00bb50 con 0x7facd40cf700 2024-09-19T08:47:47.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.934+0000 7facd2ffd640 1 --2- 172.21.15.98:0/2781575942 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facbc0039c0 0x7facbc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.934+0000 7facd27fc640 1 --2- 172.21.15.98:0/2781575942 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facbc004810 0x7facbc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.934+0000 7facd2ffd640 1 -- 172.21.15.98:0/2781575942 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7facd40cf700 msgr2=0x7facbc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:47.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.934+0000 7facd37fe640 1 --2- 172.21.15.98:0/2781575942 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7facd40cf700 0x7facbc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.934+0000 7facd2ffd640 1 --2- 172.21.15.98:0/2781575942 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7facd40cf700 0x7facbc0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.937 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.934+0000 7facd2ffd640 1 -- 172.21.15.98:0/2781575942 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facbc004810 msgr2=0x7facbc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:47.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.934+0000 7facd2ffd640 1 --2- 172.21.15.98:0/2781575942 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facbc004810 0x7facbc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.934+0000 7facd2ffd640 1 -- 172.21.15.98:0/2781575942 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facbc003530 con 0x7facbc0039c0 2024-09-19T08:47:47.938 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.934+0000 7facd27fc640 1 --2- 172.21.15.98:0/2781575942 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facbc004810 0x7facbc004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:47.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.938+0000 7facd2ffd640 1 --2- 172.21.15.98:0/2781575942 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facbc0039c0 0x7facbc003de0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7facc400c240 tx=0x7facc401cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c8d05eb4acd52a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:47.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.938+0000 7facd1ffb640 1 -- 172.21.15.98:0/2781575942 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc4007d40 con 0x7facbc0039c0 2024-09-19T08:47:47.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.938+0000 7facd1ffb640 1 -- 172.21.15.98:0/2781575942 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facc40a9020 con 0x7facbc0039c0 2024-09-19T08:47:47.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.938+0000 7facd1ffb640 1 -- 172.21.15.98:0/2781575942 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc4005b00 con 0x7facbc0039c0 2024-09-19T08:47:47.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.938+0000 7facd93aa640 1 -- 172.21.15.98:0/2781575942 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facbc0039c0 msgr2=0x7facbc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:47.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.938+0000 7facd93aa640 1 --2- 172.21.15.98:0/2781575942 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facbc0039c0 0x7facbc003de0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7facc400c240 tx=0x7facc401cbe0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.938+0000 7facd37fe640 1 -- 172.21.15.98:0/2781575942 reap_dead start 2024-09-19T08:47:47.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.938+0000 7facd93aa640 1 -- 172.21.15.98:0/2781575942 shutdown_connections 2024-09-19T08:47:47.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.938+0000 7facd93aa640 1 -- 172.21.15.98:0/2781575942 >> 172.21.15.98:0/2781575942 conn(0x7facd406c090 msgr2=0x7facd406c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:47.941 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.938+0000 7facd93aa640 1 -- 172.21.15.98:0/2781575942 shutdown_connections 2024-09-19T08:47:47.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.938+0000 7facd93aa640 1 -- 172.21.15.98:0/2781575942 wait complete. 2024-09-19T08:47:47.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.938+0000 7facd93aa640 1 Processor -- start 2024-09-19T08:47:47.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.938+0000 7facd93aa640 1 -- start start 2024-09-19T08:47:47.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.938+0000 7facd93aa640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facbc0039c0 0x7facd414bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.938+0000 7facd93aa640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7facd4072270 0x7facd40ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.938+0000 7facd93aa640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facd40726f0 0x7facd40aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.938+0000 7facd93aa640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7facd40b4fc0 con 0x7facd40726f0 2024-09-19T08:47:47.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.938+0000 7facd93aa640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7facd40b4e40 con 0x7facbc0039c0 2024-09-19T08:47:47.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.938+0000 7facd93aa640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7facd40b5140 con 0x7facd4072270 2024-09-19T08:47:47.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.938+0000 7facd37fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facd40726f0 0x7facd40aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.942+0000 7facd37fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facd40726f0 0x7facd40aedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36590/0 (socket says 172.21.15.98:36590) 2024-09-19T08:47:47.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.942+0000 7facd37fe640 1 -- 172.21.15.98:0/711313430 learned_addr learned my addr 172.21.15.98:0/711313430 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:47.942 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.942+0000 7facd27fc640 1 --2- 172.21.15.98:0/711313430 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7facd4072270 0x7facd40ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.942+0000 7facd2ffd640 1 --2- 172.21.15.98:0/711313430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facbc0039c0 0x7facd414bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.942+0000 7facd37fe640 1 -- 172.21.15.98:0/711313430 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7facd4072270 msgr2=0x7facd40ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:47.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.942+0000 7facd37fe640 1 --2- 172.21.15.98:0/711313430 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7facd4072270 0x7facd40ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.942+0000 7facd37fe640 1 -- 172.21.15.98:0/711313430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facbc0039c0 msgr2=0x7facd414bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:47.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.942+0000 7facd37fe640 1 --2- 172.21.15.98:0/711313430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7facbc0039c0 0x7facd414bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.942+0000 7facd37fe640 1 -- 172.21.15.98:0/711313430 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd414c4b0 con 0x7facd40726f0 2024-09-19T08:47:47.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.942+0000 7facd37fe640 1 --2- 172.21.15.98:0/711313430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facd40726f0 0x7facd40aedf0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7facd406f4c0 tx=0x7faccc00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:47.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.942+0000 7facbb7fe640 1 -- 172.21.15.98:0/711313430 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faccc01c070 con 0x7facd40726f0 2024-09-19T08:47:47.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.942+0000 7facbb7fe640 1 -- 172.21.15.98:0/711313430 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faccc017400 con 0x7facd40726f0 2024-09-19T08:47:47.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.942+0000 7facbb7fe640 1 -- 172.21.15.98:0/711313430 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faccc0177f0 con 0x7facd40726f0 2024-09-19T08:47:47.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.942+0000 7facd93aa640 1 -- 172.21.15.98:0/711313430 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facd4072cc0 con 0x7facd40726f0 2024-09-19T08:47:47.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.942+0000 7facd93aa640 1 -- 172.21.15.98:0/711313430 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facd41f7590 con 0x7facd40726f0 2024-09-19T08:47:47.943 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.942+0000 7facd93aa640 1 -- 172.21.15.98:0/711313430 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7facd40b5530 con 0x7facd40726f0 2024-09-19T08:47:47.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.942+0000 7facbb7fe640 1 -- 172.21.15.98:0/711313430 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7faccc01f070 con 0x7facd40726f0 2024-09-19T08:47:47.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.942+0000 7facbb7fe640 1 --2- 172.21.15.98:0/711313430 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7facc0068400 0x7facc006a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.942+0000 7facbb7fe640 1 -- 172.21.15.98:0/711313430 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7faccc0845c0 con 0x7facd40726f0 2024-09-19T08:47:47.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.942+0000 7facbb7fe640 1 --2- 172.21.15.98:0/711313430 >> [v2:172.21.15.98:6808/2466005517,v1:172.21.15.98:6809/2466005517] conn(0x7facc006e2c0 0x7facc00706e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:47.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.942+0000 7facbb7fe640 1 -- 172.21.15.98:0/711313430 --> [v2:172.21.15.98:6808/2466005517,v1:172.21.15.98:6809/2466005517] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7facc0070db0 con 0x7facc006e2c0 2024-09-19T08:47:47.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.942+0000 7facbb7fe640 1 -- 172.21.15.98:0/711313430 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=173) ==== 24+0+0 (secure 0 0 0) 0x7faccc08c500 con 0x7facd40726f0 2024-09-19T08:47:47.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.942+0000 7facd2ffd640 1 --2- 172.21.15.98:0/711313430 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7facc0068400 0x7facc006a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.942+0000 7facd27fc640 1 --2- 172.21.15.98:0/711313430 >> [v2:172.21.15.98:6808/2466005517,v1:172.21.15.98:6809/2466005517] conn(0x7facc006e2c0 0x7facc00706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:47.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.942+0000 7facd27fc640 1 --2- 172.21.15.98:0/711313430 >> [v2:172.21.15.98:6808/2466005517,v1:172.21.15.98:6809/2466005517] conn(0x7facc006e2c0 0x7facc00706e0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:47.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.942+0000 7facd2ffd640 1 --2- 172.21.15.98:0/711313430 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7facc0068400 0x7facc006a8c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7facc40aa000 tx=0x7facd40cd480 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:47.945 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.942+0000 7facbb7fe640 1 -- 172.21.15.98:0/711313430 <== osd.2 v2:172.21.15.98:6808/2466005517 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7facc0070db0 con 0x7facc006e2c0 2024-09-19T08:47:47.955 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.958+0000 7facd93aa640 1 -- 172.21.15.98:0/711313430 --> [v2:172.21.15.98:6808/2466005517,v1:172.21.15.98:6809/2466005517] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7facd4157c50 con 0x7facc006e2c0 2024-09-19T08:47:47.956 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.958+0000 7facbb7fe640 1 -- 172.21.15.98:0/711313430 <== osd.2 v2:172.21.15.98:6808/2466005517 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7facd4157c50 con 0x7facc006e2c0 2024-09-19T08:47:47.956 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:47:47.956 INFO:teuthology.orchestra.run.smithi098.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T08:47:47.956 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:47:47.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.958+0000 7facd93aa640 1 -- 172.21.15.98:0/711313430 >> [v2:172.21.15.98:6808/2466005517,v1:172.21.15.98:6809/2466005517] conn(0x7facc006e2c0 msgr2=0x7facc00706e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:47.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.958+0000 7facd93aa640 1 --2- 172.21.15.98:0/711313430 >> [v2:172.21.15.98:6808/2466005517,v1:172.21.15.98:6809/2466005517] conn(0x7facc006e2c0 0x7facc00706e0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.958+0000 7facd93aa640 1 -- 172.21.15.98:0/711313430 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7facc0068400 msgr2=0x7facc006a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:47.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.958+0000 7facd93aa640 1 --2- 172.21.15.98:0/711313430 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7facc0068400 0x7facc006a8c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7facc40aa000 tx=0x7facd40cd480 comp rx=0 tx=0).stop 2024-09-19T08:47:47.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.958+0000 7facd93aa640 1 -- 172.21.15.98:0/711313430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facd40726f0 msgr2=0x7facd40aedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:47.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.958+0000 7facd93aa640 1 --2- 172.21.15.98:0/711313430 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7facd40726f0 0x7facd40aedf0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7facd406f4c0 tx=0x7faccc00f4a0 comp rx=0 tx=0).stop 2024-09-19T08:47:47.957 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.958+0000 7facd37fe640 1 -- 172.21.15.98:0/711313430 reap_dead start 2024-09-19T08:47:47.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.958+0000 7facd93aa640 1 -- 172.21.15.98:0/711313430 shutdown_connections 2024-09-19T08:47:47.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.958+0000 7facd93aa640 1 -- 172.21.15.98:0/711313430 >> 172.21.15.98:0/711313430 conn(0x7facd406c090 msgr2=0x7facd415b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:47.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.958+0000 7facd93aa640 1 -- 172.21.15.98:0/711313430 shutdown_connections 2024-09-19T08:47:47.958 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:47.958+0000 7facd93aa640 1 -- 172.21.15.98:0/711313430 wait complete. 2024-09-19T08:47:47.970 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-19T08:47:48.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.082+0000 7fc625817640 1 Processor -- start 2024-09-19T08:47:48.080 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.082+0000 7fc625817640 1 -- start start 2024-09-19T08:47:48.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.082+0000 7fc625817640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc62015c550 0x7fc62006bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:48.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.082+0000 7fc625817640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc62006d310 con 0x7fc6200cfad0 2024-09-19T08:47:48.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.082+0000 7fc625817640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc62006d5f0 con 0x7fc62006c1d0 2024-09-19T08:47:48.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.082+0000 7fc625817640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fc62006c640 con 0x7fc62015c550 2024-09-19T08:47:48.081 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.082+0000 7fc61f7fe640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7fc6200cfad0 0x7fc6200723a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:44496/0 (socket says 172.21.15.98:44496) 2024-09-19T08:47:48.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.082+0000 7fc61f7fe640 1 -- 172.21.15.98:0/611156207 learned_addr learned my addr 172.21.15.98:0/611156207 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:48.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.082+0000 7fc61effd640 1 --2- 172.21.15.98:0/611156207 >> v2:172.21.15.98:3300/0 conn(0x7fc62015c550 0x7fc62006bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:48.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.082+0000 7fc61dffb640 1 -- 172.21.15.98:0/611156207 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 806261395 0 0) 0x7fc62006d310 con 0x7fc6200cfad0 2024-09-19T08:47:48.082 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.082+0000 7fc61dffb640 1 -- 172.21.15.98:0/611156207 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc608003530 con 0x7fc6200cfad0 2024-09-19T08:47:48.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.082+0000 7fc61effd640 1 -- 172.21.15.98:0/611156207 >> v1:172.21.15.139:6789/0 conn(0x7fc62006c1d0 legacy=0x7fc6200cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:48.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.082+0000 7fc61effd640 1 -- 172.21.15.98:0/611156207 >> v1:172.21.15.98:6789/0 conn(0x7fc6200cfad0 legacy=0x7fc6200723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:48.083 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.082+0000 7fc61effd640 1 -- 172.21.15.98:0/611156207 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc62006d8d0 con 0x7fc62015c550 2024-09-19T08:47:48.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.086+0000 7fc61effd640 1 --2- 172.21.15.98:0/611156207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc62015c550 0x7fc62006bc90 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fc6140097c0 tx=0x7fc61401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9159b69dec64044f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:48.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.086+0000 7fc61dffb640 1 -- 172.21.15.98:0/611156207 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6140b0070 con 0x7fc62015c550 2024-09-19T08:47:48.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.086+0000 7fc61dffb640 1 -- 172.21.15.98:0/611156207 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6140ab600 con 0x7fc62015c550 2024-09-19T08:47:48.084 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.086+0000 7fc61dffb640 1 -- 172.21.15.98:0/611156207 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6140ab9d0 con 0x7fc62015c550 2024-09-19T08:47:48.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.086+0000 7fc625817640 1 -- 172.21.15.98:0/611156207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc62015c550 msgr2=0x7fc62006bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:48.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.086+0000 7fc625817640 1 --2- 172.21.15.98:0/611156207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc62015c550 0x7fc62006bc90 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fc6140097c0 tx=0x7fc61401c4a0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.086+0000 7fc625817640 1 -- 172.21.15.98:0/611156207 shutdown_connections 2024-09-19T08:47:48.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.086+0000 7fc625817640 1 --2- 172.21.15.98:0/611156207 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc62015c550 0x7fc62006bc90 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.086+0000 7fc625817640 1 -- 172.21.15.98:0/611156207 >> 172.21.15.98:0/611156207 conn(0x7fc6200b2010 msgr2=0x7fc6200b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:48.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.086+0000 7fc625817640 1 -- 172.21.15.98:0/611156207 shutdown_connections 2024-09-19T08:47:48.085 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.086+0000 7fc625817640 1 -- 172.21.15.98:0/611156207 wait complete. 2024-09-19T08:47:48.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.086+0000 7fc625817640 1 Processor -- start 2024-09-19T08:47:48.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.086+0000 7fc625817640 1 -- start start 2024-09-19T08:47:48.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.086+0000 7fc625817640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc62006c1d0 0x7fc62015a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:48.087 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.086+0000 7fc625817640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6200cfad0 0x7fc620159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:48.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.086+0000 7fc625817640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc62015c550 0x7fc620149220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:48.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.086+0000 7fc625817640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc6200736d0 con 0x7fc62006c1d0 2024-09-19T08:47:48.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.086+0000 7fc625817640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc620073550 con 0x7fc6200cfad0 2024-09-19T08:47:48.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.086+0000 7fc625817640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc62016a240 con 0x7fc62015c550 2024-09-19T08:47:48.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.086+0000 7fc61effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc62006c1d0 0x7fc62015a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:48.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.086+0000 7fc61effd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc62006c1d0 0x7fc62015a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36602/0 (socket says 172.21.15.98:36602) 2024-09-19T08:47:48.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.086+0000 7fc61effd640 1 -- 172.21.15.98:0/360986613 learned_addr learned my addr 172.21.15.98:0/360986613 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:48.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.086+0000 7fc61effd640 1 -- 172.21.15.98:0/360986613 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc62015c550 msgr2=0x7fc620149220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:48.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.086+0000 7fc61effd640 1 --2- 172.21.15.98:0/360986613 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc62015c550 0x7fc620149220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.086+0000 7fc61effd640 1 -- 172.21.15.98:0/360986613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6200cfad0 msgr2=0x7fc620159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:48.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.086+0000 7fc61effd640 1 --2- 172.21.15.98:0/360986613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6200cfad0 0x7fc620159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.086+0000 7fc61effd640 1 -- 172.21.15.98:0/360986613 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc620159e90 con 0x7fc62006c1d0 2024-09-19T08:47:48.088 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.086+0000 7fc61effd640 1 --2- 172.21.15.98:0/360986613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc62006c1d0 0x7fc62015a770 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fc6140ae040 tx=0x7fc61401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:48.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.086+0000 7fc607fff640 1 -- 172.21.15.98:0/360986613 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6140b0050 con 0x7fc62006c1d0 2024-09-19T08:47:48.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.086+0000 7fc607fff640 1 -- 172.21.15.98:0/360986613 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6140bc070 con 0x7fc62006c1d0 2024-09-19T08:47:48.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.086+0000 7fc625817640 1 -- 172.21.15.98:0/360986613 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6201588b0 con 0x7fc62006c1d0 2024-09-19T08:47:48.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.086+0000 7fc607fff640 1 -- 172.21.15.98:0/360986613 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6140b5870 con 0x7fc62006c1d0 2024-09-19T08:47:48.089 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.086+0000 7fc625817640 1 -- 172.21.15.98:0/360986613 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc620158e70 con 0x7fc62006c1d0 2024-09-19T08:47:48.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.090+0000 7fc625817640 1 -- 172.21.15.98:0/360986613 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc62006bc90 con 0x7fc62006c1d0 2024-09-19T08:47:48.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.090+0000 7fc607fff640 1 -- 172.21.15.98:0/360986613 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7fc6140b5a10 con 0x7fc62006c1d0 2024-09-19T08:47:48.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.090+0000 7fc607fff640 1 --2- 172.21.15.98:0/360986613 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc60c068430 0x7fc60c06a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:48.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.090+0000 7fc607fff640 1 -- 172.21.15.98:0/360986613 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7fc61411ece0 con 0x7fc62006c1d0 2024-09-19T08:47:48.090 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.090+0000 7fc61e7fc640 1 --2- 172.21.15.98:0/360986613 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc60c068430 0x7fc60c06a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:48.091 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.090+0000 7fc61e7fc640 1 --2- 172.21.15.98:0/360986613 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc60c068430 0x7fc60c06a8f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc610000f70 tx=0x7fc6100079e0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:48.093 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.094+0000 7fc607fff640 1 -- 172.21.15.98:0/360986613 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc614016090 con 0x7fc62006c1d0 2024-09-19T08:47:48.256 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.258+0000 7fc625817640 1 -- 172.21.15.98:0/360986613 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fc6201585a0 con 0x7fc60c068430 2024-09-19T08:47:48.257 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.258+0000 7fc607fff640 1 -- 172.21.15.98:0/360986613 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc6201585a0 con 0x7fc60c068430 2024-09-19T08:47:48.258 INFO:teuthology.orchestra.run.smithi098.stderr:instructed osd(s) 2 to deep-scrub 2024-09-19T08:47:48.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.258+0000 7fc625817640 1 -- 172.21.15.98:0/360986613 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc60c068430 msgr2=0x7fc60c06a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:48.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.258+0000 7fc625817640 1 --2- 172.21.15.98:0/360986613 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc60c068430 0x7fc60c06a8f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc610000f70 tx=0x7fc6100079e0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.258+0000 7fc625817640 1 -- 172.21.15.98:0/360986613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc62006c1d0 msgr2=0x7fc62015a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:48.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.258+0000 7fc625817640 1 --2- 172.21.15.98:0/360986613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc62006c1d0 0x7fc62015a770 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fc6140ae040 tx=0x7fc61401ace0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.258+0000 7fc625817640 1 -- 172.21.15.98:0/360986613 shutdown_connections 2024-09-19T08:47:48.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.258+0000 7fc625817640 1 --2- 172.21.15.98:0/360986613 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc60c068430 0x7fc60c06a8f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.258+0000 7fc625817640 1 --2- 172.21.15.98:0/360986613 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc62015c550 0x7fc620149220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.258+0000 7fc625817640 1 --2- 172.21.15.98:0/360986613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6200cfad0 0x7fc620159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.258+0000 7fc625817640 1 --2- 172.21.15.98:0/360986613 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc62006c1d0 0x7fc62015a770 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.258+0000 7fc625817640 1 -- 172.21.15.98:0/360986613 >> 172.21.15.98:0/360986613 conn(0x7fc6200b2010 msgr2=0x7fc6200afa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:48.260 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.262+0000 7fc625817640 1 -- 172.21.15.98:0/360986613 shutdown_connections 2024-09-19T08:47:48.261 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.262+0000 7fc625817640 1 -- 172.21.15.98:0/360986613 wait complete. 2024-09-19T08:47:48.276 INFO:tasks.ceph:Scrubbing osd.3 2024-09-19T08:47:48.277 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-19T08:47:48.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.398+0000 7f3f68c6c640 1 Processor -- start 2024-09-19T08:47:48.399 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.398+0000 7f3f68c6c640 1 -- start start 2024-09-19T08:47:48.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.402+0000 7f3f68c6c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f6415c290 0x7f3f640cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:48.400 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.402+0000 7f3f68c6c640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3f640cf700 0x7f3f6415ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:48.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.402+0000 7f3f68c6c640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3f6406d4c0 0x7f3f6406d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:48.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.402+0000 7f3f68c6c640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f3f6406de70 con 0x7f3f640cf700 2024-09-19T08:47:48.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.402+0000 7f3f68c6c640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f3f64160e80 con 0x7f3f6406d4c0 2024-09-19T08:47:48.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.402+0000 7f3f68c6c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3f640cd510 con 0x7f3f6415c290 2024-09-19T08:47:48.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.402+0000 7f3f5bfff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3f640cf700 0x7f3f6415ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:48.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.402+0000 7f3f5bfff640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f3f640cf700 0x7f3f6415ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36618/0 (socket says 172.21.15.98:36618) 2024-09-19T08:47:48.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.402+0000 7f3f5bfff640 1 -- 172.21.15.98:0/361016761 learned_addr learned my addr 172.21.15.98:0/361016761 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:48.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.402+0000 7f3f62d9d640 1 --2- 172.21.15.98:0/361016761 >> v2:172.21.15.139:3300/0 conn(0x7f3f6406d4c0 0x7f3f6406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:48.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.402+0000 7f3f6259c640 1 --2- 172.21.15.98:0/361016761 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f6415c290 0x7f3f640cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:48.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.402+0000 7f3f5bfff640 1 -- 172.21.15.98:0/361016761 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f6415c290 msgr2=0x7f3f640cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:48.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.402+0000 7f3f5bfff640 1 --2- 172.21.15.98:0/361016761 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f6415c290 0x7f3f640cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.402+0000 7f3f5bfff640 1 -- 172.21.15.98:0/361016761 >> v2:172.21.15.139:3300/0 conn(0x7f3f6406d4c0 msgr2=0x7f3f6406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:48.401 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.402+0000 7f3f5bfff640 1 --2- 172.21.15.98:0/361016761 >> v2:172.21.15.139:3300/0 conn(0x7f3f6406d4c0 0x7f3f6406d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.402+0000 7f3f5bfff640 1 -- 172.21.15.98:0/361016761 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f641608c0 con 0x7f3f640cf700 2024-09-19T08:47:48.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.402+0000 7f3f5bfff640 1 --2- 172.21.15.98:0/361016761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f640cf700 0x7f3f6415ff50 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f3f640b3b60 tx=0x7f3f4c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=390146caa1991574 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:48.402 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.402+0000 7f3f61d9b640 1 -- 172.21.15.98:0/361016761 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c0b1070 con 0x7f3f640cf700 2024-09-19T08:47:48.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.402+0000 7f3f61d9b640 1 -- 172.21.15.98:0/361016761 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f4c0ac630 con 0x7f3f640cf700 2024-09-19T08:47:48.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.402+0000 7f3f61d9b640 1 -- 172.21.15.98:0/361016761 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c0a8050 con 0x7f3f640cf700 2024-09-19T08:47:48.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.402+0000 7f3f68c6c640 1 -- 172.21.15.98:0/361016761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f640cf700 msgr2=0x7f3f6415ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:48.403 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.402+0000 7f3f68c6c640 1 --2- 172.21.15.98:0/361016761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f640cf700 0x7f3f6415ff50 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f3f640b3b60 tx=0x7f3f4c01c520 comp rx=0 tx=0).stop 2024-09-19T08:47:48.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.402+0000 7f3f68c6c640 1 -- 172.21.15.98:0/361016761 shutdown_connections 2024-09-19T08:47:48.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.402+0000 7f3f68c6c640 1 --2- 172.21.15.98:0/361016761 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f640cf700 0x7f3f6415ff50 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.402+0000 7f3f68c6c640 1 --2- 172.21.15.98:0/361016761 >> v2:172.21.15.139:3300/0 conn(0x7f3f6406d4c0 0x7f3f6406d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.402+0000 7f3f68c6c640 1 --2- 172.21.15.98:0/361016761 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f6415c290 0x7f3f640cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.402+0000 7f3f68c6c640 1 -- 172.21.15.98:0/361016761 >> 172.21.15.98:0/361016761 conn(0x7f3f6406c090 msgr2=0x7f3f6406c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:48.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.402+0000 7f3f68c6c640 1 -- 172.21.15.98:0/361016761 shutdown_connections 2024-09-19T08:47:48.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.402+0000 7f3f68c6c640 1 -- 172.21.15.98:0/361016761 wait complete. 2024-09-19T08:47:48.404 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.406+0000 7f3f68c6c640 1 Processor -- start 2024-09-19T08:47:48.405 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.406+0000 7f3f68c6c640 1 -- start start 2024-09-19T08:47:48.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.406+0000 7f3f68c6c640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f6406d4c0 0x7f3f64164650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:48.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.406+0000 7f3f68c6c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f640cf700 0x7f3f6415a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:48.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.406+0000 7f3f68c6c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f6415c290 0x7f3f64145df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:48.406 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.406+0000 7f3f68c6c640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6415c7f0 con 0x7f3f640cf700 2024-09-19T08:47:48.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.406+0000 7f3f68c6c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6415c670 con 0x7f3f6415c290 2024-09-19T08:47:48.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.406+0000 7f3f68c6c640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f3f64160490 con 0x7f3f6406d4c0 2024-09-19T08:47:48.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.406+0000 7f3f5bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f640cf700 0x7f3f6415a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:48.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.406+0000 7f3f5bfff640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f640cf700 0x7f3f6415a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36630/0 (socket says 172.21.15.98:36630) 2024-09-19T08:47:48.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.406+0000 7f3f5bfff640 1 -- 172.21.15.98:0/2652541084 learned_addr learned my addr 172.21.15.98:0/2652541084 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:48.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.406+0000 7f3f5bfff640 1 -- 172.21.15.98:0/2652541084 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f6406d4c0 msgr2=0x7f3f64164650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:48.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.406+0000 7f3f5bfff640 1 --2- 172.21.15.98:0/2652541084 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3f6406d4c0 0x7f3f64164650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.406+0000 7f3f5bfff640 1 -- 172.21.15.98:0/2652541084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f6415c290 msgr2=0x7f3f64145df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:48.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.406+0000 7f3f5bfff640 1 --2- 172.21.15.98:0/2652541084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3f6415c290 0x7f3f64145df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.407 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.406+0000 7f3f5bfff640 1 -- 172.21.15.98:0/2652541084 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f641489f0 con 0x7f3f640cf700 2024-09-19T08:47:48.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.406+0000 7f3f5bfff640 1 --2- 172.21.15.98:0/2652541084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f640cf700 0x7f3f6415a9e0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c01abd0 tx=0x7f3f4c0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:48.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.406+0000 7f3f5affd640 1 -- 172.21.15.98:0/2652541084 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c0bc070 con 0x7f3f640cf700 2024-09-19T08:47:48.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.406+0000 7f3f5affd640 1 -- 172.21.15.98:0/2652541084 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f4c0ab5e0 con 0x7f3f640cf700 2024-09-19T08:47:48.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.406+0000 7f3f5affd640 1 -- 172.21.15.98:0/2652541084 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c0b1070 con 0x7f3f640cf700 2024-09-19T08:47:48.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.406+0000 7f3f68c6c640 1 -- 172.21.15.98:0/2652541084 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f64148cd0 con 0x7f3f640cf700 2024-09-19T08:47:48.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.406+0000 7f3f68c6c640 1 -- 172.21.15.98:0/2652541084 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f6415a1e0 con 0x7f3f640cf700 2024-09-19T08:47:48.408 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.406+0000 7f3f68c6c640 1 -- 172.21.15.98:0/2652541084 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3f640b5400 con 0x7f3f640cf700 2024-09-19T08:47:48.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.406+0000 7f3f5affd640 1 -- 172.21.15.98:0/2652541084 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f3f4c0bf070 con 0x7f3f640cf700 2024-09-19T08:47:48.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.410+0000 7f3f5affd640 1 --2- 172.21.15.98:0/2652541084 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3f44068400 0x7f3f4406a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:48.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.410+0000 7f3f5affd640 1 -- 172.21.15.98:0/2652541084 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7f3f4c11e270 con 0x7f3f640cf700 2024-09-19T08:47:48.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.410+0000 7f3f5affd640 1 --2- 172.21.15.98:0/2652541084 >> [v2:172.21.15.98:6800/3643795173,v1:172.21.15.98:6801/3643795173] conn(0x7f3f4406e2c0 0x7f3f440706e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:48.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.410+0000 7f3f5affd640 1 -- 172.21.15.98:0/2652541084 --> [v2:172.21.15.98:6800/3643795173,v1:172.21.15.98:6801/3643795173] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3f4c0c4ec0 con 0x7f3f4406e2c0 2024-09-19T08:47:48.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.410+0000 7f3f5affd640 1 -- 172.21.15.98:0/2652541084 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=173) ==== 24+0+0 (secure 0 0 0) 0x7f3f4c1209a0 con 0x7f3f640cf700 2024-09-19T08:47:48.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.410+0000 7f3f62d9d640 1 --2- 172.21.15.98:0/2652541084 >> [v2:172.21.15.98:6800/3643795173,v1:172.21.15.98:6801/3643795173] conn(0x7f3f4406e2c0 0x7f3f440706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:48.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.410+0000 7f3f6259c640 1 --2- 172.21.15.98:0/2652541084 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3f44068400 0x7f3f4406a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:48.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.410+0000 7f3f62d9d640 1 --2- 172.21.15.98:0/2652541084 >> [v2:172.21.15.98:6800/3643795173,v1:172.21.15.98:6801/3643795173] conn(0x7f3f4406e2c0 0x7f3f440706e0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:48.410 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.410+0000 7f3f6259c640 1 --2- 172.21.15.98:0/2652541084 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3f44068400 0x7f3f4406a8c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3f5400a810 tx=0x7f3f54008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:48.411 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.410+0000 7f3f5affd640 1 -- 172.21.15.98:0/2652541084 <== osd.3 v2:172.21.15.98:6800/3643795173 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3f4c0c4ec0 con 0x7f3f4406e2c0 2024-09-19T08:47:48.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.422+0000 7f3f68c6c640 1 -- 172.21.15.98:0/2652541084 --> [v2:172.21.15.98:6800/3643795173,v1:172.21.15.98:6801/3643795173] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3f6406daa0 con 0x7f3f4406e2c0 2024-09-19T08:47:48.421 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.422+0000 7f3f5affd640 1 -- 172.21.15.98:0/2652541084 <== osd.3 v2:172.21.15.98:6800/3643795173 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f3f6406daa0 con 0x7f3f4406e2c0 2024-09-19T08:47:48.422 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:47:48.422 INFO:teuthology.orchestra.run.smithi098.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T08:47:48.422 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:47:48.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.422+0000 7f3f68c6c640 1 -- 172.21.15.98:0/2652541084 >> [v2:172.21.15.98:6800/3643795173,v1:172.21.15.98:6801/3643795173] conn(0x7f3f4406e2c0 msgr2=0x7f3f440706e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:48.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.422+0000 7f3f68c6c640 1 --2- 172.21.15.98:0/2652541084 >> [v2:172.21.15.98:6800/3643795173,v1:172.21.15.98:6801/3643795173] conn(0x7f3f4406e2c0 0x7f3f440706e0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.422+0000 7f3f68c6c640 1 -- 172.21.15.98:0/2652541084 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3f44068400 msgr2=0x7f3f4406a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:48.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.422+0000 7f3f68c6c640 1 --2- 172.21.15.98:0/2652541084 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f3f44068400 0x7f3f4406a8c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3f5400a810 tx=0x7f3f54008040 comp rx=0 tx=0).stop 2024-09-19T08:47:48.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.422+0000 7f3f68c6c640 1 -- 172.21.15.98:0/2652541084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f640cf700 msgr2=0x7f3f6415a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:48.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.422+0000 7f3f68c6c640 1 --2- 172.21.15.98:0/2652541084 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3f640cf700 0x7f3f6415a9e0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c01abd0 tx=0x7f3f4c0192c0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.422+0000 7f3f62d9d640 1 -- 172.21.15.98:0/2652541084 reap_dead start 2024-09-19T08:47:48.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.422+0000 7f3f68c6c640 1 -- 172.21.15.98:0/2652541084 shutdown_connections 2024-09-19T08:47:48.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.422+0000 7f3f68c6c640 1 -- 172.21.15.98:0/2652541084 >> 172.21.15.98:0/2652541084 conn(0x7f3f6406c090 msgr2=0x7f3f640b6050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:48.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.422+0000 7f3f68c6c640 1 -- 172.21.15.98:0/2652541084 shutdown_connections 2024-09-19T08:47:48.423 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.422+0000 7f3f68c6c640 1 -- 172.21.15.98:0/2652541084 wait complete. 2024-09-19T08:47:48.437 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-19T08:47:48.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.538+0000 7fe4e9ad6640 1 Processor -- start 2024-09-19T08:47:48.537 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.538+0000 7fe4e9ad6640 1 -- start start 2024-09-19T08:47:48.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.538+0000 7fe4e9ad6640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe4e4158fd0 0x7fe4e40b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:48.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.538+0000 7fe4e9ad6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe4e406c3a0 0x7fe4e40ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:48.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.538+0000 7fe4e9ad6640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4e40ccc60 con 0x7fe4e40b4360 2024-09-19T08:47:48.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.538+0000 7fe4e9ad6640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fe4e41593b0 con 0x7fe4e4158fd0 2024-09-19T08:47:48.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.538+0000 7fe4e9ad6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe4e40ccf40 con 0x7fe4e406c3a0 2024-09-19T08:47:48.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.538+0000 7fe4e37fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe4e4158fd0 0x7fe4e40b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:48.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.538+0000 7fe4e37fe640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fe4e4158fd0 0x7fe4e40b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36632/0 (socket says 172.21.15.98:36632) 2024-09-19T08:47:48.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.538+0000 7fe4e37fe640 1 -- 172.21.15.98:0/4081964778 learned_addr learned my addr 172.21.15.98:0/4081964778 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:48.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.538+0000 7fe4e3fff640 1 --2- 172.21.15.98:0/4081964778 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe4e406c3a0 0x7fe4e40ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:48.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.538+0000 7fe4e37fe640 1 -- 172.21.15.98:0/4081964778 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe4e406c3a0 msgr2=0x7fe4e40ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:48.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.538+0000 7fe4e37fe640 1 --2- 172.21.15.98:0/4081964778 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe4e406c3a0 0x7fe4e40ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.538+0000 7fe4e37fe640 1 -- 172.21.15.98:0/4081964778 >> v1:172.21.15.139:6789/0 conn(0x7fe4e40b4360 legacy=0x7fe4e406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:48.539 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.538+0000 7fe4e37fe640 1 -- 172.21.15.98:0/4081964778 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e40cd220 con 0x7fe4e4158fd0 2024-09-19T08:47:48.540 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.542+0000 7fe4e37fe640 1 --2- 172.21.15.98:0/4081964778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4e4158fd0 0x7fe4e40b3e20 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009870 tx=0x7fe4d401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8290316706f84e3e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:48.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.542+0000 7fe4e2ffd640 1 -- 172.21.15.98:0/4081964778 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d40ab620 con 0x7fe4e4158fd0 2024-09-19T08:47:48.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.542+0000 7fe4e2ffd640 1 -- 172.21.15.98:0/4081964778 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4d40b0070 con 0x7fe4e4158fd0 2024-09-19T08:47:48.541 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.542+0000 7fe4e2ffd640 1 -- 172.21.15.98:0/4081964778 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d40ab9e0 con 0x7fe4e4158fd0 2024-09-19T08:47:48.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.542+0000 7fe4e9ad6640 1 -- 172.21.15.98:0/4081964778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4e4158fd0 msgr2=0x7fe4e40b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:48.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.542+0000 7fe4e9ad6640 1 --2- 172.21.15.98:0/4081964778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4e4158fd0 0x7fe4e40b3e20 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009870 tx=0x7fe4d401c460 comp rx=0 tx=0).stop 2024-09-19T08:47:48.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.542+0000 7fe4e9ad6640 1 -- 172.21.15.98:0/4081964778 shutdown_connections 2024-09-19T08:47:48.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.542+0000 7fe4e9ad6640 1 --2- 172.21.15.98:0/4081964778 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe4e406c3a0 0x7fe4e40ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.542+0000 7fe4e9ad6640 1 --2- 172.21.15.98:0/4081964778 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4e4158fd0 0x7fe4e40b3e20 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.542+0000 7fe4e9ad6640 1 -- 172.21.15.98:0/4081964778 >> 172.21.15.98:0/4081964778 conn(0x7fe4e40b2010 msgr2=0x7fe4e40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:48.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.542+0000 7fe4e9ad6640 1 -- 172.21.15.98:0/4081964778 shutdown_connections 2024-09-19T08:47:48.542 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.542+0000 7fe4e9ad6640 1 -- 172.21.15.98:0/4081964778 wait complete. 2024-09-19T08:47:48.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.542+0000 7fe4e9ad6640 1 Processor -- start 2024-09-19T08:47:48.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.542+0000 7fe4e9ad6640 1 -- start start 2024-09-19T08:47:48.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.542+0000 7fe4e9ad6640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe4e406c3a0 0x7fe4e40b0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:48.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.542+0000 7fe4e9ad6640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4e40b4360 0x7fe4e406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:48.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.542+0000 7fe4e9ad6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4e4158fd0 0x7fe4e40713d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:48.544 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.542+0000 7fe4e9ad6640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e40cf710 con 0x7fe4e40b4360 2024-09-19T08:47:48.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.542+0000 7fe4e9ad6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e40cf590 con 0x7fe4e4158fd0 2024-09-19T08:47:48.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.542+0000 7fe4e9ad6640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fe4e40cf890 con 0x7fe4e406c3a0 2024-09-19T08:47:48.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.542+0000 7fe4daffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4e40b4360 0x7fe4e406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:48.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.542+0000 7fe4daffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4e40b4360 0x7fe4e406d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36638/0 (socket says 172.21.15.98:36638) 2024-09-19T08:47:48.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.542+0000 7fe4daffd640 1 -- 172.21.15.98:0/2678010879 learned_addr learned my addr 172.21.15.98:0/2678010879 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:48.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.546+0000 7fe4e3fff640 1 --2- 172.21.15.98:0/2678010879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4e4158fd0 0x7fe4e40713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:48.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.546+0000 7fe4e37fe640 1 --2- 172.21.15.98:0/2678010879 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe4e406c3a0 0x7fe4e40b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:48.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.546+0000 7fe4daffd640 1 -- 172.21.15.98:0/2678010879 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe4e406c3a0 msgr2=0x7fe4e40b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:48.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.546+0000 7fe4daffd640 1 --2- 172.21.15.98:0/2678010879 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe4e406c3a0 0x7fe4e40b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.546+0000 7fe4daffd640 1 -- 172.21.15.98:0/2678010879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4e4158fd0 msgr2=0x7fe4e40713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:48.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.546+0000 7fe4daffd640 1 --2- 172.21.15.98:0/2678010879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4e4158fd0 0x7fe4e40713d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.546+0000 7fe4daffd640 1 -- 172.21.15.98:0/2678010879 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e41625e0 con 0x7fe4e40b4360 2024-09-19T08:47:48.545 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.546+0000 7fe4e3fff640 1 --2- 172.21.15.98:0/2678010879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4e4158fd0 0x7fe4e40713d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:48.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.546+0000 7fe4daffd640 1 --2- 172.21.15.98:0/2678010879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4e40b4360 0x7fe4e406d3d0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00c450 tx=0x7fe4cc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:48.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.546+0000 7fe4e0ff9640 1 -- 172.21.15.98:0/2678010879 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc012830 con 0x7fe4e40b4360 2024-09-19T08:47:48.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.546+0000 7fe4e0ff9640 1 -- 172.21.15.98:0/2678010879 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4cc017070 con 0x7fe4e40b4360 2024-09-19T08:47:48.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.546+0000 7fe4e9ad6640 1 -- 172.21.15.98:0/2678010879 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4e40b0790 con 0x7fe4e40b4360 2024-09-19T08:47:48.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.546+0000 7fe4e0ff9640 1 -- 172.21.15.98:0/2678010879 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc012c30 con 0x7fe4e40b4360 2024-09-19T08:47:48.546 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.546+0000 7fe4e9ad6640 1 -- 172.21.15.98:0/2678010879 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4e40b1030 con 0x7fe4e40b4360 2024-09-19T08:47:48.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.546+0000 7fe4e9ad6640 1 -- 172.21.15.98:0/2678010879 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4e40b33b0 con 0x7fe4e40b4360 2024-09-19T08:47:48.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.546+0000 7fe4e0ff9640 1 -- 172.21.15.98:0/2678010879 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7fe4cc01c460 con 0x7fe4e40b4360 2024-09-19T08:47:48.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.546+0000 7fe4e0ff9640 1 --2- 172.21.15.98:0/2678010879 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe4c4068400 0x7fe4c406a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:48.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.546+0000 7fe4e0ff9640 1 -- 172.21.15.98:0/2678010879 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7fe4cc084360 con 0x7fe4e40b4360 2024-09-19T08:47:48.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.546+0000 7fe4e37fe640 1 --2- 172.21.15.98:0/2678010879 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe4c4068400 0x7fe4c406a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:48.547 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.546+0000 7fe4e37fe640 1 --2- 172.21.15.98:0/2678010879 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe4c4068400 0x7fe4c406a8c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40af000 tx=0x7fe4d4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:48.550 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.550+0000 7fe4e0ff9640 1 -- 172.21.15.98:0/2678010879 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe4e41625e0 con 0x7fe4e40b4360 2024-09-19T08:47:48.706 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.706+0000 7fe4e9ad6640 1 -- 172.21.15.98:0/2678010879 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fe4e406be30 con 0x7fe4c4068400 2024-09-19T08:47:48.707 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.706+0000 7fe4e0ff9640 1 -- 172.21.15.98:0/2678010879 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe4e406be30 con 0x7fe4c4068400 2024-09-19T08:47:48.707 INFO:teuthology.orchestra.run.smithi098.stderr:instructed osd(s) 3 to deep-scrub 2024-09-19T08:47:48.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.710+0000 7fe4e9ad6640 1 -- 172.21.15.98:0/2678010879 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe4c4068400 msgr2=0x7fe4c406a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:48.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.710+0000 7fe4e9ad6640 1 --2- 172.21.15.98:0/2678010879 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe4c4068400 0x7fe4c406a8c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40af000 tx=0x7fe4d4008040 comp rx=0 tx=0).stop 2024-09-19T08:47:48.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.710+0000 7fe4e9ad6640 1 -- 172.21.15.98:0/2678010879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4e40b4360 msgr2=0x7fe4e406d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:48.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.710+0000 7fe4e9ad6640 1 --2- 172.21.15.98:0/2678010879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4e40b4360 0x7fe4e406d3d0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00c450 tx=0x7fe4cc00c910 comp rx=0 tx=0).stop 2024-09-19T08:47:48.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.710+0000 7fe4e9ad6640 1 -- 172.21.15.98:0/2678010879 shutdown_connections 2024-09-19T08:47:48.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.710+0000 7fe4e9ad6640 1 --2- 172.21.15.98:0/2678010879 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fe4c4068400 0x7fe4c406a8c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.710+0000 7fe4e9ad6640 1 --2- 172.21.15.98:0/2678010879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe4e4158fd0 0x7fe4e40713d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.710+0000 7fe4e9ad6640 1 --2- 172.21.15.98:0/2678010879 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fe4e40b4360 0x7fe4e406d3d0 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.710+0000 7fe4e9ad6640 1 --2- 172.21.15.98:0/2678010879 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fe4e406c3a0 0x7fe4e40b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.710+0000 7fe4e9ad6640 1 -- 172.21.15.98:0/2678010879 >> 172.21.15.98:0/2678010879 conn(0x7fe4e40b2010 msgr2=0x7fe4e41651b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:48.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.710+0000 7fe4e9ad6640 1 -- 172.21.15.98:0/2678010879 shutdown_connections 2024-09-19T08:47:48.709 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.710+0000 7fe4e9ad6640 1 -- 172.21.15.98:0/2678010879 wait complete. 2024-09-19T08:47:48.724 INFO:tasks.ceph:Scrubbing osd.4 2024-09-19T08:47:48.724 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-19T08:47:48.799 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:47:48.802 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:47:48.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.846+0000 7f19e39eb640 1 Processor -- start 2024-09-19T08:47:48.845 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.846+0000 7f19e39eb640 1 -- start start 2024-09-19T08:47:48.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.846+0000 7f19e39eb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19dc15e720 0x7f19dc0cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:48.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.846+0000 7f19e39eb640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f19dc0cfb50 0x7f19dc0b7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:48.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.846+0000 7f19e39eb640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19dc0d29c0 con 0x7f19dc0b7cb0 2024-09-19T08:47:48.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.846+0000 7f19e39eb640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f19dc0d2ca0 con 0x7f19dc0cfb50 2024-09-19T08:47:48.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.846+0000 7f19e39eb640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f19dc0d2f80 con 0x7f19dc15e720 2024-09-19T08:47:48.846 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.846+0000 7f19e0f86640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f19dc0cfb50 0x7f19dc0b7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:48.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.846+0000 7f19e1f88640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f19dc0b7cb0 0x7f19dc0d3a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:44506/0 (socket says 172.21.15.98:44506) 2024-09-19T08:47:48.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.846+0000 7f19e1f88640 1 -- 172.21.15.98:0/1958811642 learned_addr learned my addr 172.21.15.98:0/1958811642 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:48.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.846+0000 7f19e1787640 1 --2- 172.21.15.98:0/1958811642 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19dc15e720 0x7f19dc0cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:48.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.846+0000 7f19e0f86640 1 -- 172.21.15.98:0/1958811642 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19dc15e720 msgr2=0x7f19dc0cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:48.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.846+0000 7f19e0f86640 1 --2- 172.21.15.98:0/1958811642 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19dc15e720 0x7f19dc0cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.846+0000 7f19e0f86640 1 -- 172.21.15.98:0/1958811642 >> v1:172.21.15.98:6789/0 conn(0x7f19dc0b7cb0 legacy=0x7f19dc0d3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:48.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.846+0000 7f19d3fff640 1 -- 172.21.15.98:0/1958811642 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1504705684 0 0) 0x7f19dc0d29c0 con 0x7f19dc0b7cb0 2024-09-19T08:47:48.847 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.846+0000 7f19e0f86640 1 -- 172.21.15.98:0/1958811642 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19dc0d3260 con 0x7f19dc0cfb50 2024-09-19T08:47:48.848 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.850+0000 7f19e0f86640 1 --2- 172.21.15.98:0/1958811642 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19dc0cfb50 0x7f19dc0b7770 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0097c0 tx=0x7f19cc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04b533ba160d76f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:48.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.850+0000 7f19d3fff640 1 -- 172.21.15.98:0/1958811642 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19cc0b1070 con 0x7f19dc0cfb50 2024-09-19T08:47:48.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.850+0000 7f19d3fff640 1 -- 172.21.15.98:0/1958811642 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19cc0ac410 con 0x7f19dc0cfb50 2024-09-19T08:47:48.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.850+0000 7f19d3fff640 1 -- 172.21.15.98:0/1958811642 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19cc0ac7e0 con 0x7f19dc0cfb50 2024-09-19T08:47:48.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.850+0000 7f19e39eb640 1 -- 172.21.15.98:0/1958811642 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19dc0cfb50 msgr2=0x7f19dc0b7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:48.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.850+0000 7f19e39eb640 1 --2- 172.21.15.98:0/1958811642 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19dc0cfb50 0x7f19dc0b7770 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0097c0 tx=0x7f19cc01c4a0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.850+0000 7f19e39eb640 1 -- 172.21.15.98:0/1958811642 shutdown_connections 2024-09-19T08:47:48.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.850+0000 7f19e39eb640 1 --2- 172.21.15.98:0/1958811642 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19dc0cfb50 0x7f19dc0b7770 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.851 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.850+0000 7f19e39eb640 1 --2- 172.21.15.98:0/1958811642 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19dc15e720 0x7f19dc0cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.850+0000 7f19e39eb640 1 -- 172.21.15.98:0/1958811642 >> 172.21.15.98:0/1958811642 conn(0x7f19dc0d4640 msgr2=0x7f19dc0d4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:48.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.850+0000 7f19e39eb640 1 -- 172.21.15.98:0/1958811642 shutdown_connections 2024-09-19T08:47:48.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.850+0000 7f19e39eb640 1 -- 172.21.15.98:0/1958811642 wait complete. 2024-09-19T08:47:48.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.854+0000 7f19e39eb640 1 Processor -- start 2024-09-19T08:47:48.852 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.854+0000 7f19e39eb640 1 -- start start 2024-09-19T08:47:48.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.854+0000 7f19e39eb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19dc0b7cb0 0x7f19dc15dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:48.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.854+0000 7f19e39eb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19dc0cfb50 0x7f19dc0c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:48.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.854+0000 7f19e39eb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19dc15e720 0x7f19dc0b64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:48.857 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.854+0000 7f19e39eb640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc0d4170 con 0x7f19dc0cfb50 2024-09-19T08:47:48.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.854+0000 7f19e39eb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f19dc15ed10 con 0x7f19dc0b7cb0 2024-09-19T08:47:48.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.854+0000 7f19e39eb640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f19dc0d42f0 con 0x7f19dc15e720 2024-09-19T08:47:48.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.854+0000 7f19e0f86640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19dc0cfb50 0x7f19dc0c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:48.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.854+0000 7f19e0f86640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19dc0cfb50 0x7f19dc0c8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36656/0 (socket says 172.21.15.98:36656) 2024-09-19T08:47:48.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.854+0000 7f19e0f86640 1 -- 172.21.15.98:0/2923973871 learned_addr learned my addr 172.21.15.98:0/2923973871 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:48.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.854+0000 7f19e1787640 1 --2- 172.21.15.98:0/2923973871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19dc0b7cb0 0x7f19dc15dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:48.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.854+0000 7f19e1f88640 1 --2- 172.21.15.98:0/2923973871 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19dc15e720 0x7f19dc0b64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:48.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.854+0000 7f19e0f86640 1 -- 172.21.15.98:0/2923973871 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19dc15e720 msgr2=0x7f19dc0b64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:48.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.854+0000 7f19e0f86640 1 --2- 172.21.15.98:0/2923973871 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19dc15e720 0x7f19dc0b64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.854+0000 7f19e0f86640 1 -- 172.21.15.98:0/2923973871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19dc0b7cb0 msgr2=0x7f19dc15dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:48.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.854+0000 7f19e0f86640 1 --2- 172.21.15.98:0/2923973871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19dc0b7cb0 0x7f19dc15dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.854+0000 7f19e0f86640 1 -- 172.21.15.98:0/2923973871 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19dc0c89c0 con 0x7f19dc0cfb50 2024-09-19T08:47:48.858 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.854+0000 7f19e1787640 1 --2- 172.21.15.98:0/2923973871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f19dc0b7cb0 0x7f19dc15dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:48.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.854+0000 7f19e1f88640 1 --2- 172.21.15.98:0/2923973871 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f19dc15e720 0x7f19dc0b64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:48.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.854+0000 7f19e0f86640 1 --2- 172.21.15.98:0/2923973871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19dc0cfb50 0x7f19dc0c8170 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0af040 tx=0x7f19cc019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:48.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.854+0000 7f19d1ffb640 1 -- 172.21.15.98:0/2923973871 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19cc0b1040 con 0x7f19dc0cfb50 2024-09-19T08:47:48.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.854+0000 7f19d1ffb640 1 -- 172.21.15.98:0/2923973871 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19cc0ba070 con 0x7f19dc0cfb50 2024-09-19T08:47:48.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.854+0000 7f19d1ffb640 1 -- 172.21.15.98:0/2923973871 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19cc0b5690 con 0x7f19dc0cfb50 2024-09-19T08:47:48.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.854+0000 7f19e39eb640 1 -- 172.21.15.98:0/2923973871 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19dc0c8ca0 con 0x7f19dc0cfb50 2024-09-19T08:47:48.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.854+0000 7f19e39eb640 1 -- 172.21.15.98:0/2923973871 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19dc0ca950 con 0x7f19dc0cfb50 2024-09-19T08:47:48.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.854+0000 7f19e39eb640 1 -- 172.21.15.98:0/2923973871 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f19dc0c8ea0 con 0x7f19dc0cfb50 2024-09-19T08:47:48.859 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.854+0000 7f19d1ffb640 1 -- 172.21.15.98:0/2923973871 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f19cc0b5830 con 0x7f19dc0cfb50 2024-09-19T08:47:48.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.858+0000 7f19d1ffb640 1 --2- 172.21.15.98:0/2923973871 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f19c0068400 0x7f19c006a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:48.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.858+0000 7f19d1ffb640 1 -- 172.21.15.98:0/2923973871 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7f19cc11ee50 con 0x7f19dc0cfb50 2024-09-19T08:47:48.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.858+0000 7f19d1ffb640 1 --2- 172.21.15.98:0/2923973871 >> [v2:172.21.15.139:6824/1412125138,v1:172.21.15.139:6825/1412125138] conn(0x7f19c006e2c0 0x7f19c00706e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:48.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.858+0000 7f19d1ffb640 1 -- 172.21.15.98:0/2923973871 --> [v2:172.21.15.139:6824/1412125138,v1:172.21.15.139:6825/1412125138] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f19cc0ac410 con 0x7f19c006e2c0 2024-09-19T08:47:48.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.858+0000 7f19d1ffb640 1 -- 172.21.15.98:0/2923973871 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=173) ==== 24+0+0 (secure 0 0 0) 0x7f19cc1215a0 con 0x7f19dc0cfb50 2024-09-19T08:47:48.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.858+0000 7f19e1787640 1 --2- 172.21.15.98:0/2923973871 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f19c0068400 0x7f19c006a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:48.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.858+0000 7f19e1f88640 1 --2- 172.21.15.98:0/2923973871 >> [v2:172.21.15.139:6824/1412125138,v1:172.21.15.139:6825/1412125138] conn(0x7f19c006e2c0 0x7f19c00706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:48.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.858+0000 7f19e1f88640 1 --2- 172.21.15.98:0/2923973871 >> [v2:172.21.15.139:6824/1412125138,v1:172.21.15.139:6825/1412125138] conn(0x7f19c006e2c0 0x7f19c00706e0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:48.860 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.858+0000 7f19e1787640 1 --2- 172.21.15.98:0/2923973871 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f19c0068400 0x7f19c006a8c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009840 tx=0x7f19d8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:48.861 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.858+0000 7f19d1ffb640 1 -- 172.21.15.98:0/2923973871 <== osd.4 v2:172.21.15.139:6824/1412125138 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f19cc0ac410 con 0x7f19c006e2c0 2024-09-19T08:47:48.870 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.870+0000 7f19e39eb640 1 -- 172.21.15.98:0/2923973871 --> [v2:172.21.15.139:6824/1412125138,v1:172.21.15.139:6825/1412125138] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f19dc0b7770 con 0x7f19c006e2c0 2024-09-19T08:47:48.871 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.870+0000 7f19d1ffb640 1 -- 172.21.15.98:0/2923973871 <== osd.4 v2:172.21.15.139:6824/1412125138 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f19dc0b7770 con 0x7f19c006e2c0 2024-09-19T08:47:48.871 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:47:48.871 INFO:teuthology.orchestra.run.smithi098.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T08:47:48.872 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:47:48.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.874+0000 7f19e39eb640 1 -- 172.21.15.98:0/2923973871 >> [v2:172.21.15.139:6824/1412125138,v1:172.21.15.139:6825/1412125138] conn(0x7f19c006e2c0 msgr2=0x7f19c00706e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:48.872 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.874+0000 7f19e39eb640 1 --2- 172.21.15.98:0/2923973871 >> [v2:172.21.15.139:6824/1412125138,v1:172.21.15.139:6825/1412125138] conn(0x7f19c006e2c0 0x7f19c00706e0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.874+0000 7f19e39eb640 1 -- 172.21.15.98:0/2923973871 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f19c0068400 msgr2=0x7f19c006a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:48.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.874+0000 7f19e39eb640 1 --2- 172.21.15.98:0/2923973871 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f19c0068400 0x7f19c006a8c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009840 tx=0x7f19d8008040 comp rx=0 tx=0).stop 2024-09-19T08:47:48.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.874+0000 7f19e39eb640 1 -- 172.21.15.98:0/2923973871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19dc0cfb50 msgr2=0x7f19dc0c8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:48.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.874+0000 7f19e39eb640 1 --2- 172.21.15.98:0/2923973871 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f19dc0cfb50 0x7f19dc0c8170 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0af040 tx=0x7f19cc019240 comp rx=0 tx=0).stop 2024-09-19T08:47:48.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.874+0000 7f19e1f88640 1 -- 172.21.15.98:0/2923973871 reap_dead start 2024-09-19T08:47:48.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.874+0000 7f19e39eb640 1 -- 172.21.15.98:0/2923973871 shutdown_connections 2024-09-19T08:47:48.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.874+0000 7f19e39eb640 1 -- 172.21.15.98:0/2923973871 >> 172.21.15.98:0/2923973871 conn(0x7f19dc0d4640 msgr2=0x7f19dc158800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:48.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.874+0000 7f19e39eb640 1 -- 172.21.15.98:0/2923973871 shutdown_connections 2024-09-19T08:47:48.873 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.874+0000 7f19e39eb640 1 -- 172.21.15.98:0/2923973871 wait complete. 2024-09-19T08:47:48.887 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-19T08:47:48.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.994+0000 7f1877f1a640 1 Processor -- start 2024-09-19T08:47:48.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.994+0000 7f1877f1a640 1 -- start start 2024-09-19T08:47:48.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.994+0000 7f1877f1a640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f18700732a0 0x7f187015d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:48.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.994+0000 7f1877f1a640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f18700d0140 0x7f18700ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:48.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.994+0000 7f1877f1a640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f187006eb00 con 0x7f18700b0000 2024-09-19T08:47:48.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.994+0000 7f1877f1a640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f187006fe40 con 0x7f18700d0140 2024-09-19T08:47:48.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.994+0000 7f1877f1a640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f187006eda0 con 0x7f18700732a0 2024-09-19T08:47:48.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.994+0000 7f18764b7640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f18700d0140 0x7f18700ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:48.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.994+0000 7f1875cb6640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f18700b0000 0x7f1870072b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:44512/0 (socket says 172.21.15.98:44512) 2024-09-19T08:47:48.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.994+0000 7f1875cb6640 1 -- 172.21.15.98:0/3649146661 learned_addr learned my addr 172.21.15.98:0/3649146661 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:48.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.994+0000 7f18764b7640 1 -- 172.21.15.98:0/3649146661 >> v2:172.21.15.139:3300/0 conn(0x7f18700732a0 msgr2=0x7f187015d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:48.994 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.994+0000 7f18764b7640 1 --2- 172.21.15.98:0/3649146661 >> v2:172.21.15.139:3300/0 conn(0x7f18700732a0 0x7f187015d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.994+0000 7f18764b7640 1 -- 172.21.15.98:0/3649146661 >> v1:172.21.15.98:6789/0 conn(0x7f18700b0000 legacy=0x7f1870072b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:48.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.994+0000 7f18764b7640 1 -- 172.21.15.98:0/3649146661 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187006f080 con 0x7f18700d0140 2024-09-19T08:47:48.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.994+0000 7f1874cb4640 1 -- 172.21.15.98:0/3649146661 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 547066183 0 0) 0x7f187006eb00 con 0x7f18700b0000 2024-09-19T08:47:48.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f18764b7640 1 --2- 172.21.15.98:0/3649146661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18700d0140 0x7f18700ce550 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f1870068600 tx=0x7f186801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edef15cb9f473585 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:48.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f1874cb4640 1 -- 172.21.15.98:0/3649146661 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18680b2070 con 0x7f18700d0140 2024-09-19T08:47:48.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f1874cb4640 1 -- 172.21.15.98:0/3649146661 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18680a58e0 con 0x7f18700d0140 2024-09-19T08:47:48.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f1874cb4640 1 -- 172.21.15.98:0/3649146661 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18680a5cb0 con 0x7f18700d0140 2024-09-19T08:47:48.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f1877f1a640 1 -- 172.21.15.98:0/3649146661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18700d0140 msgr2=0x7f18700ce550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:48.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f1877f1a640 1 --2- 172.21.15.98:0/3649146661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18700d0140 0x7f18700ce550 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f1870068600 tx=0x7f186801c930 comp rx=0 tx=0).stop 2024-09-19T08:47:48.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f1877f1a640 1 -- 172.21.15.98:0/3649146661 shutdown_connections 2024-09-19T08:47:48.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f1877f1a640 1 --2- 172.21.15.98:0/3649146661 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18700d0140 0x7f18700ce550 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f1877f1a640 1 --2- 172.21.15.98:0/3649146661 >> v2:172.21.15.139:3300/0 conn(0x7f18700732a0 0x7f187015d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:48.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f1877f1a640 1 -- 172.21.15.98:0/3649146661 >> 172.21.15.98:0/3649146661 conn(0x7f18700d0980 msgr2=0x7f18700d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:48.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f1877f1a640 1 -- 172.21.15.98:0/3649146661 shutdown_connections 2024-09-19T08:47:48.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f1877f1a640 1 -- 172.21.15.98:0/3649146661 wait complete. 2024-09-19T08:47:48.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f1877f1a640 1 Processor -- start 2024-09-19T08:47:48.998 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f1877f1a640 1 -- start start 2024-09-19T08:47:49.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f1877f1a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18700732a0 0x7f1870151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:49.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f1877f1a640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18700b0000 0x7f1870153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:49.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f1877f1a640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18700d0140 0x7f1870153e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:49.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f1877f1a640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f18700cf420 con 0x7f18700b0000 2024-09-19T08:47:49.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f1877f1a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f18700cf2a0 con 0x7f18700732a0 2024-09-19T08:47:49.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f1877f1a640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f18700cf5a0 con 0x7f18700d0140 2024-09-19T08:47:49.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f18754b5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18700b0000 0x7f1870153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:49.000 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f18754b5640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18700b0000 0x7f1870153940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36684/0 (socket says 172.21.15.98:36684) 2024-09-19T08:47:49.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f18754b5640 1 -- 172.21.15.98:0/495367993 learned_addr learned my addr 172.21.15.98:0/495367993 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:49.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f1875cb6640 1 --2- 172.21.15.98:0/495367993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18700732a0 0x7f1870151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:49.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f18764b7640 1 --2- 172.21.15.98:0/495367993 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18700d0140 0x7f1870153e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:49.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f18754b5640 1 -- 172.21.15.98:0/495367993 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18700d0140 msgr2=0x7f1870153e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:49.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f18754b5640 1 --2- 172.21.15.98:0/495367993 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18700d0140 0x7f1870153e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f18754b5640 1 -- 172.21.15.98:0/495367993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18700732a0 msgr2=0x7f1870151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:49.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f18754b5640 1 --2- 172.21.15.98:0/495367993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18700732a0 0x7f1870151010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f18754b5640 1 -- 172.21.15.98:0/495367993 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18700b3960 con 0x7f18700b0000 2024-09-19T08:47:49.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f18764b7640 1 --2- 172.21.15.98:0/495367993 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18700d0140 0x7f1870153e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:49.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f18754b5640 1 --2- 172.21.15.98:0/495367993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18700b0000 0x7f1870153940 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f186c00b580 tx=0x7f186c00ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:49.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f18667fc640 1 -- 172.21.15.98:0/495367993 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186c014070 con 0x7f18700b0000 2024-09-19T08:47:49.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f18667fc640 1 -- 172.21.15.98:0/495367993 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f186c010890 con 0x7f18700b0000 2024-09-19T08:47:49.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f1877f1a640 1 -- 172.21.15.98:0/495367993 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18700b3c40 con 0x7f18700b0000 2024-09-19T08:47:49.001 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f18667fc640 1 -- 172.21.15.98:0/495367993 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186c010c80 con 0x7f18700b0000 2024-09-19T08:47:49.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:48.998+0000 7f1877f1a640 1 -- 172.21.15.98:0/495367993 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18700b4200 con 0x7f18700b0000 2024-09-19T08:47:49.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.002+0000 7f1877f1a640 1 -- 172.21.15.98:0/495367993 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f187015d250 con 0x7f18700b0000 2024-09-19T08:47:49.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.002+0000 7f18667fc640 1 -- 172.21.15.98:0/495367993 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f186c017070 con 0x7f18700b0000 2024-09-19T08:47:49.002 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.002+0000 7f18667fc640 1 --2- 172.21.15.98:0/495367993 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1854068400 0x7f185406a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:49.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.002+0000 7f18667fc640 1 -- 172.21.15.98:0/495367993 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7f186c084680 con 0x7f18700b0000 2024-09-19T08:47:49.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.002+0000 7f1875cb6640 1 --2- 172.21.15.98:0/495367993 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1854068400 0x7f185406a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:49.003 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.002+0000 7f1875cb6640 1 --2- 172.21.15.98:0/495367993 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1854068400 0x7f185406a8c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1860000f70 tx=0x7f1860007b60 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:49.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.006+0000 7f18667fc640 1 -- 172.21.15.98:0/495367993 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f186c08a030 con 0x7f18700b0000 2024-09-19T08:47:49.153 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.154+0000 7f1877f1a640 1 -- 172.21.15.98:0/495367993 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f18700008e0 con 0x7f1854068400 2024-09-19T08:47:49.154 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.154+0000 7f18667fc640 1 -- 172.21.15.98:0/495367993 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f18700008e0 con 0x7f1854068400 2024-09-19T08:47:49.154 INFO:teuthology.orchestra.run.smithi098.stderr:instructed osd(s) 4 to deep-scrub 2024-09-19T08:47:49.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.158+0000 7f1877f1a640 1 -- 172.21.15.98:0/495367993 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1854068400 msgr2=0x7f185406a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:49.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.158+0000 7f1877f1a640 1 --2- 172.21.15.98:0/495367993 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1854068400 0x7f185406a8c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1860000f70 tx=0x7f1860007b60 comp rx=0 tx=0).stop 2024-09-19T08:47:49.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.158+0000 7f1877f1a640 1 -- 172.21.15.98:0/495367993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18700b0000 msgr2=0x7f1870153940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:49.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.158+0000 7f1877f1a640 1 --2- 172.21.15.98:0/495367993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18700b0000 0x7f1870153940 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f186c00b580 tx=0x7f186c00ba40 comp rx=0 tx=0).stop 2024-09-19T08:47:49.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.158+0000 7f1877f1a640 1 -- 172.21.15.98:0/495367993 shutdown_connections 2024-09-19T08:47:49.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.158+0000 7f1877f1a640 1 --2- 172.21.15.98:0/495367993 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f1854068400 0x7f185406a8c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.158+0000 7f1877f1a640 1 --2- 172.21.15.98:0/495367993 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f18700d0140 0x7f1870153e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.158+0000 7f1877f1a640 1 --2- 172.21.15.98:0/495367993 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f18700b0000 0x7f1870153940 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.158+0000 7f1877f1a640 1 --2- 172.21.15.98:0/495367993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f18700732a0 0x7f1870151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.157 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.158+0000 7f1877f1a640 1 -- 172.21.15.98:0/495367993 >> 172.21.15.98:0/495367993 conn(0x7f18700d0980 msgr2=0x7f1870067a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:49.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.158+0000 7f1877f1a640 1 -- 172.21.15.98:0/495367993 shutdown_connections 2024-09-19T08:47:49.158 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.158+0000 7f1877f1a640 1 -- 172.21.15.98:0/495367993 wait complete. 2024-09-19T08:47:49.171 INFO:tasks.ceph:Scrubbing osd.5 2024-09-19T08:47:49.172 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-19T08:47:49.284 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.286+0000 7fc63dae9640 1 Processor -- start 2024-09-19T08:47:49.285 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.286+0000 7fc63dae9640 1 -- start start 2024-09-19T08:47:49.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.286+0000 7fc63dae9640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fc63815c5e0 0x7fc63806f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:49.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.286+0000 7fc63dae9640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc638073490 0x7fc63806f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:49.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.286+0000 7fc63dae9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc6380d1490 0x7fc638070790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:49.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.286+0000 7fc63dae9640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fc63815c9c0 con 0x7fc638073490 2024-09-19T08:47:49.286 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.286+0000 7fc63dae9640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fc6380b6350 con 0x7fc63815c5e0 2024-09-19T08:47:49.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.286+0000 7fc63dae9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc6380b5d40 con 0x7fc6380d1490 2024-09-19T08:47:49.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.286+0000 7fc636ffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc638073490 0x7fc63806f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:49.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.286+0000 7fc636ffd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fc638073490 0x7fc63806f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36698/0 (socket says 172.21.15.98:36698) 2024-09-19T08:47:49.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.286+0000 7fc636ffd640 1 -- 172.21.15.98:0/3178365042 learned_addr learned my addr 172.21.15.98:0/3178365042 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:49.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.286+0000 7fc637fff640 1 --2- 172.21.15.98:0/3178365042 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc6380d1490 0x7fc638070790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:49.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.286+0000 7fc6377fe640 1 --2- 172.21.15.98:0/3178365042 >> v2:172.21.15.139:3300/0 conn(0x7fc63815c5e0 0x7fc63806f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:49.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.286+0000 7fc636ffd640 1 -- 172.21.15.98:0/3178365042 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc6380d1490 msgr2=0x7fc638070790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:49.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.286+0000 7fc636ffd640 1 --2- 172.21.15.98:0/3178365042 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc6380d1490 0x7fc638070790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.286+0000 7fc636ffd640 1 -- 172.21.15.98:0/3178365042 >> v2:172.21.15.139:3300/0 conn(0x7fc63815c5e0 msgr2=0x7fc63806f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:49.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.286+0000 7fc636ffd640 1 --2- 172.21.15.98:0/3178365042 >> v2:172.21.15.139:3300/0 conn(0x7fc63815c5e0 0x7fc63806f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.286+0000 7fc636ffd640 1 -- 172.21.15.98:0/3178365042 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6380739c0 con 0x7fc638073490 2024-09-19T08:47:49.287 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.286+0000 7fc6377fe640 1 --2- 172.21.15.98:0/3178365042 >> v2:172.21.15.139:3300/0 conn(0x7fc63815c5e0 0x7fc63806f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:49.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.290+0000 7fc636ffd640 1 --2- 172.21.15.98:0/3178365042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc638073490 0x7fc63806f8c0 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fc638072950 tx=0x7fc62c01c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf4d3429ce42ef3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:49.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.290+0000 7fc6367fc640 1 -- 172.21.15.98:0/3178365042 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c0b0070 con 0x7fc638073490 2024-09-19T08:47:49.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.290+0000 7fc6367fc640 1 -- 172.21.15.98:0/3178365042 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc62c0ab600 con 0x7fc638073490 2024-09-19T08:47:49.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.290+0000 7fc6367fc640 1 -- 172.21.15.98:0/3178365042 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c0ab9d0 con 0x7fc638073490 2024-09-19T08:47:49.289 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.290+0000 7fc63dae9640 1 -- 172.21.15.98:0/3178365042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc638073490 msgr2=0x7fc63806f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:49.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.290+0000 7fc63dae9640 1 --2- 172.21.15.98:0/3178365042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc638073490 0x7fc63806f8c0 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fc638072950 tx=0x7fc62c01c1d0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.290+0000 7fc63dae9640 1 -- 172.21.15.98:0/3178365042 shutdown_connections 2024-09-19T08:47:49.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.290+0000 7fc63dae9640 1 --2- 172.21.15.98:0/3178365042 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc638073490 0x7fc63806f8c0 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.290+0000 7fc63dae9640 1 --2- 172.21.15.98:0/3178365042 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc6380d1490 0x7fc638070790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.290+0000 7fc63dae9640 1 --2- 172.21.15.98:0/3178365042 >> v2:172.21.15.139:3300/0 conn(0x7fc63815c5e0 0x7fc63806f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.290+0000 7fc63dae9640 1 -- 172.21.15.98:0/3178365042 >> 172.21.15.98:0/3178365042 conn(0x7fc6380cedf0 msgr2=0x7fc6380cf200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:49.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.290+0000 7fc63dae9640 1 -- 172.21.15.98:0/3178365042 shutdown_connections 2024-09-19T08:47:49.290 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.290+0000 7fc63dae9640 1 -- 172.21.15.98:0/3178365042 wait complete. 2024-09-19T08:47:49.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.290+0000 7fc63dae9640 1 Processor -- start 2024-09-19T08:47:49.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.290+0000 7fc63dae9640 1 -- start start 2024-09-19T08:47:49.292 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.290+0000 7fc63dae9640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc638073490 0x7fc63814b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:49.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.290+0000 7fc63dae9640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc6380d1490 0x7fc63815e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:49.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.290+0000 7fc63dae9640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc63815c5e0 0x7fc63814c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:49.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.290+0000 7fc63dae9640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fc6380b67c0 con 0x7fc638073490 2024-09-19T08:47:49.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.290+0000 7fc63dae9640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc638074310 con 0x7fc63815c5e0 2024-09-19T08:47:49.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.290+0000 7fc63dae9640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fc6380b6940 con 0x7fc6380d1490 2024-09-19T08:47:49.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.290+0000 7fc6377fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc638073490 0x7fc63814b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:49.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.290+0000 7fc6377fe640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc638073490 0x7fc63814b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36704/0 (socket says 172.21.15.98:36704) 2024-09-19T08:47:49.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.290+0000 7fc6377fe640 1 -- 172.21.15.98:0/3321247648 learned_addr learned my addr 172.21.15.98:0/3321247648 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:49.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.290+0000 7fc637fff640 1 --2- 172.21.15.98:0/3321247648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc63815c5e0 0x7fc63814c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:49.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.290+0000 7fc6377fe640 1 -- 172.21.15.98:0/3321247648 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc6380d1490 msgr2=0x7fc63815e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:49.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.290+0000 7fc636ffd640 1 --2- 172.21.15.98:0/3321247648 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc6380d1490 0x7fc63815e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:49.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.290+0000 7fc6377fe640 1 --2- 172.21.15.98:0/3321247648 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fc6380d1490 0x7fc63815e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.293 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.290+0000 7fc6377fe640 1 -- 172.21.15.98:0/3321247648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc63815c5e0 msgr2=0x7fc63814c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:49.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.290+0000 7fc6377fe640 1 --2- 172.21.15.98:0/3321247648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc63815c5e0 0x7fc63814c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.290+0000 7fc6377fe640 1 -- 172.21.15.98:0/3321247648 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6380d1b70 con 0x7fc638073490 2024-09-19T08:47:49.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.290+0000 7fc637fff640 1 --2- 172.21.15.98:0/3321247648 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc63815c5e0 0x7fc63814c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:49.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.290+0000 7fc6377fe640 1 --2- 172.21.15.98:0/3321247648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc638073490 0x7fc63814b780 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7fc63000c7f0 tx=0x7fc63000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:49.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.294+0000 7fc61ffff640 1 -- 172.21.15.98:0/3321247648 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc630015070 con 0x7fc638073490 2024-09-19T08:47:49.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.294+0000 7fc61ffff640 1 -- 172.21.15.98:0/3321247648 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6300077c0 con 0x7fc638073490 2024-09-19T08:47:49.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.294+0000 7fc63dae9640 1 -- 172.21.15.98:0/3321247648 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc63815e620 con 0x7fc638073490 2024-09-19T08:47:49.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.294+0000 7fc61ffff640 1 -- 172.21.15.98:0/3321247648 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc630007bb0 con 0x7fc638073490 2024-09-19T08:47:49.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.294+0000 7fc63dae9640 1 -- 172.21.15.98:0/3321247648 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc63814be60 con 0x7fc638073490 2024-09-19T08:47:49.294 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.294+0000 7fc63dae9640 1 -- 172.21.15.98:0/3321247648 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc638070790 con 0x7fc638073490 2024-09-19T08:47:49.295 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.294+0000 7fc61ffff640 1 -- 172.21.15.98:0/3321247648 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7fc630018070 con 0x7fc638073490 2024-09-19T08:47:49.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.294+0000 7fc61ffff640 1 --2- 172.21.15.98:0/3321247648 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc624068400 0x7fc62406a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:49.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.294+0000 7fc61ffff640 1 -- 172.21.15.98:0/3321247648 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7fc630084ee0 con 0x7fc638073490 2024-09-19T08:47:49.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.294+0000 7fc61ffff640 1 --2- 172.21.15.98:0/3321247648 >> [v2:172.21.15.139:6808/520689888,v1:172.21.15.139:6809/520689888] conn(0x7fc62406e2c0 0x7fc6240706e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:49.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.294+0000 7fc61ffff640 1 -- 172.21.15.98:0/3321247648 --> [v2:172.21.15.139:6808/520689888,v1:172.21.15.139:6809/520689888] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc63001de50 con 0x7fc62406e2c0 2024-09-19T08:47:49.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.294+0000 7fc61ffff640 1 -- 172.21.15.98:0/3321247648 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=173) ==== 24+0+0 (secure 0 0 0) 0x7fc6300877b0 con 0x7fc638073490 2024-09-19T08:47:49.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.294+0000 7fc636ffd640 1 --2- 172.21.15.98:0/3321247648 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc624068400 0x7fc62406a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:49.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.294+0000 7fc637fff640 1 --2- 172.21.15.98:0/3321247648 >> [v2:172.21.15.139:6808/520689888,v1:172.21.15.139:6809/520689888] conn(0x7fc62406e2c0 0x7fc6240706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:49.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.294+0000 7fc636ffd640 1 --2- 172.21.15.98:0/3321247648 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc624068400 0x7fc62406a8c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc62c0ad000 tx=0x7fc62c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:49.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.294+0000 7fc637fff640 1 --2- 172.21.15.98:0/3321247648 >> [v2:172.21.15.139:6808/520689888,v1:172.21.15.139:6809/520689888] conn(0x7fc62406e2c0 0x7fc6240706e0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:49.296 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.298+0000 7fc61ffff640 1 -- 172.21.15.98:0/3321247648 <== osd.5 v2:172.21.15.139:6808/520689888 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc63001de50 con 0x7fc62406e2c0 2024-09-19T08:47:49.308 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.310+0000 7fc63dae9640 1 -- 172.21.15.98:0/3321247648 --> [v2:172.21.15.139:6808/520689888,v1:172.21.15.139:6809/520689888] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc638153480 con 0x7fc62406e2c0 2024-09-19T08:47:49.309 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.310+0000 7fc61ffff640 1 -- 172.21.15.98:0/3321247648 <== osd.5 v2:172.21.15.139:6808/520689888 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc638153480 con 0x7fc62406e2c0 2024-09-19T08:47:49.309 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:47:49.309 INFO:teuthology.orchestra.run.smithi098.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T08:47:49.310 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:47:49.310 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.310+0000 7fc63dae9640 1 -- 172.21.15.98:0/3321247648 >> [v2:172.21.15.139:6808/520689888,v1:172.21.15.139:6809/520689888] conn(0x7fc62406e2c0 msgr2=0x7fc6240706e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:49.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.310+0000 7fc63dae9640 1 --2- 172.21.15.98:0/3321247648 >> [v2:172.21.15.139:6808/520689888,v1:172.21.15.139:6809/520689888] conn(0x7fc62406e2c0 0x7fc6240706e0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.310+0000 7fc63dae9640 1 -- 172.21.15.98:0/3321247648 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc624068400 msgr2=0x7fc62406a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:49.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.310+0000 7fc63dae9640 1 --2- 172.21.15.98:0/3321247648 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fc624068400 0x7fc62406a8c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc62c0ad000 tx=0x7fc62c008040 comp rx=0 tx=0).stop 2024-09-19T08:47:49.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.310+0000 7fc63dae9640 1 -- 172.21.15.98:0/3321247648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc638073490 msgr2=0x7fc63814b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:49.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.310+0000 7fc63dae9640 1 --2- 172.21.15.98:0/3321247648 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fc638073490 0x7fc63814b780 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7fc63000c7f0 tx=0x7fc63000ccb0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.310+0000 7fc637fff640 1 -- 172.21.15.98:0/3321247648 reap_dead start 2024-09-19T08:47:49.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.310+0000 7fc63dae9640 1 -- 172.21.15.98:0/3321247648 shutdown_connections 2024-09-19T08:47:49.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.310+0000 7fc63dae9640 1 -- 172.21.15.98:0/3321247648 >> 172.21.15.98:0/3321247648 conn(0x7fc6380cedf0 msgr2=0x7fc638156720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:49.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.310+0000 7fc63dae9640 1 -- 172.21.15.98:0/3321247648 shutdown_connections 2024-09-19T08:47:49.311 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.310+0000 7fc63dae9640 1 -- 172.21.15.98:0/3321247648 wait complete. 2024-09-19T08:47:49.324 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-19T08:47:49.426 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.426+0000 7f6527522640 1 Processor -- start 2024-09-19T08:47:49.427 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.426+0000 7f6527522640 1 -- start start 2024-09-19T08:47:49.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.426+0000 7f6527522640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f652015c220 0x7f65200733f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:49.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.426+0000 7f6527522640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6520073930 0x7f65200d0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:49.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.426+0000 7f6527522640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65200d1530 0x7f65200b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:49.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.426+0000 7f6527522640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f65200d1990 con 0x7f6520073930 2024-09-19T08:47:49.428 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.426+0000 7f6527522640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f65200740b0 con 0x7f652015c220 2024-09-19T08:47:49.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.426+0000 7f6527522640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f6520073dd0 con 0x7f65200d1530 2024-09-19T08:47:49.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.430+0000 7f6524abd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6520073930 0x7f65200d0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:49.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.430+0000 7f6524abd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f6520073930 0x7f65200d0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36710/0 (socket says 172.21.15.98:36710) 2024-09-19T08:47:49.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.430+0000 7f6524abd640 1 -- 172.21.15.98:0/3740427289 learned_addr learned my addr 172.21.15.98:0/3740427289 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:49.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.430+0000 7f65252be640 1 --2- 172.21.15.98:0/3740427289 >> v2:172.21.15.139:3300/0 conn(0x7f652015c220 0x7f65200733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:49.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.430+0000 7f6524abd640 1 -- 172.21.15.98:0/3740427289 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65200d1530 msgr2=0x7f65200b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:49.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.430+0000 7f6525abf640 1 --2- 172.21.15.98:0/3740427289 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65200d1530 0x7f65200b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:49.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.430+0000 7f6524abd640 1 --2- 172.21.15.98:0/3740427289 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65200d1530 0x7f65200b6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.430+0000 7f6524abd640 1 -- 172.21.15.98:0/3740427289 >> v2:172.21.15.139:3300/0 conn(0x7f652015c220 msgr2=0x7f65200733f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:49.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.430+0000 7f6524abd640 1 --2- 172.21.15.98:0/3740427289 >> v2:172.21.15.139:3300/0 conn(0x7f652015c220 0x7f65200733f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.430+0000 7f6524abd640 1 -- 172.21.15.98:0/3740427289 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65200b6460 con 0x7f6520073930 2024-09-19T08:47:49.429 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.430+0000 7f65252be640 1 --2- 172.21.15.98:0/3740427289 >> v2:172.21.15.139:3300/0 conn(0x7f652015c220 0x7f65200733f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:49.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.430+0000 7f6524abd640 1 --2- 172.21.15.98:0/3740427289 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6520073930 0x7f65200d0ff0 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f6520070350 tx=0x7f651401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=328be2cc317eaa1d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:49.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.430+0000 7f651ffff640 1 -- 172.21.15.98:0/3740427289 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65140aa610 con 0x7f6520073930 2024-09-19T08:47:49.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.430+0000 7f651ffff640 1 -- 172.21.15.98:0/3740427289 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65140af070 con 0x7f6520073930 2024-09-19T08:47:49.430 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.430+0000 7f651ffff640 1 -- 172.21.15.98:0/3740427289 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65140aa9d0 con 0x7f6520073930 2024-09-19T08:47:49.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.430+0000 7f6527522640 1 -- 172.21.15.98:0/3740427289 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6520073930 msgr2=0x7f65200d0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:49.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.430+0000 7f6527522640 1 --2- 172.21.15.98:0/3740427289 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6520073930 0x7f65200d0ff0 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f6520070350 tx=0x7f651401c4f0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.430+0000 7f6527522640 1 -- 172.21.15.98:0/3740427289 shutdown_connections 2024-09-19T08:47:49.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.430+0000 7f6527522640 1 --2- 172.21.15.98:0/3740427289 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6520073930 0x7f65200d0ff0 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.430+0000 7f6527522640 1 --2- 172.21.15.98:0/3740427289 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f65200d1530 0x7f65200b6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.431 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.430+0000 7f6527522640 1 --2- 172.21.15.98:0/3740427289 >> v2:172.21.15.139:3300/0 conn(0x7f652015c220 0x7f65200733f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.430+0000 7f6527522640 1 -- 172.21.15.98:0/3740427289 >> 172.21.15.98:0/3740427289 conn(0x7f65200b40a0 msgr2=0x7f65200b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:49.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.430+0000 7f6527522640 1 -- 172.21.15.98:0/3740427289 shutdown_connections 2024-09-19T08:47:49.432 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.430+0000 7f6527522640 1 -- 172.21.15.98:0/3740427289 wait complete. 2024-09-19T08:47:49.433 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.434+0000 7f6527522640 1 Processor -- start 2024-09-19T08:47:49.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.434+0000 7f6527522640 1 -- start start 2024-09-19T08:47:49.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.434+0000 7f6527522640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6520073930 0x7f652014c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:49.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.434+0000 7f6527522640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65200d1530 0x7f652015d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:49.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.434+0000 7f6527522640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f652015c220 0x7f6520158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:49.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.434+0000 7f6527522640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f65200d0db0 con 0x7f6520073930 2024-09-19T08:47:49.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.434+0000 7f6527522640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f65200d0c30 con 0x7f65200d1530 2024-09-19T08:47:49.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.434+0000 7f6527522640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f652015ce10 con 0x7f652015c220 2024-09-19T08:47:49.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.434+0000 7f65252be640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6520073930 0x7f652014c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:49.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.434+0000 7f65252be640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6520073930 0x7f652014c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36724/0 (socket says 172.21.15.98:36724) 2024-09-19T08:47:49.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.434+0000 7f65252be640 1 -- 172.21.15.98:0/1658391739 learned_addr learned my addr 172.21.15.98:0/1658391739 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:49.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.434+0000 7f6524abd640 1 --2- 172.21.15.98:0/1658391739 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65200d1530 0x7f652015d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:49.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.434+0000 7f65252be640 1 -- 172.21.15.98:0/1658391739 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f652015c220 msgr2=0x7f6520158810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:49.434 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.434+0000 7f65252be640 1 --2- 172.21.15.98:0/1658391739 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f652015c220 0x7f6520158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.434+0000 7f65252be640 1 -- 172.21.15.98:0/1658391739 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65200d1530 msgr2=0x7f652015d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:49.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.434+0000 7f65252be640 1 --2- 172.21.15.98:0/1658391739 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65200d1530 0x7f652015d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.434+0000 7f65252be640 1 -- 172.21.15.98:0/1658391739 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f652015e1c0 con 0x7f6520073930 2024-09-19T08:47:49.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.434+0000 7f65252be640 1 --2- 172.21.15.98:0/1658391739 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6520073930 0x7f652014c850 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f651800c7f0 tx=0x7f651800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:49.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.434+0000 7f651dffb640 1 -- 172.21.15.98:0/1658391739 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6518015070 con 0x7f6520073930 2024-09-19T08:47:49.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.434+0000 7f651dffb640 1 -- 172.21.15.98:0/1658391739 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6518007820 con 0x7f6520073930 2024-09-19T08:47:49.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.434+0000 7f6527522640 1 -- 172.21.15.98:0/1658391739 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f652015e4a0 con 0x7f6520073930 2024-09-19T08:47:49.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.434+0000 7f651dffb640 1 -- 172.21.15.98:0/1658391739 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6518007c10 con 0x7f6520073930 2024-09-19T08:47:49.435 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.434+0000 7f6527522640 1 -- 172.21.15.98:0/1658391739 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f652015ea60 con 0x7f6520073930 2024-09-19T08:47:49.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.434+0000 7f6527522640 1 -- 172.21.15.98:0/1658391739 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65200b6f40 con 0x7f6520073930 2024-09-19T08:47:49.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.434+0000 7f651dffb640 1 -- 172.21.15.98:0/1658391739 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f6518018070 con 0x7f6520073930 2024-09-19T08:47:49.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.434+0000 7f651dffb640 1 --2- 172.21.15.98:0/1658391739 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f650c068400 0x7f650c06a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:49.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.434+0000 7f651dffb640 1 -- 172.21.15.98:0/1658391739 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7f6518084970 con 0x7f6520073930 2024-09-19T08:47:49.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.438+0000 7f6524abd640 1 --2- 172.21.15.98:0/1658391739 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f650c068400 0x7f650c06a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:49.436 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.438+0000 7f6524abd640 1 --2- 172.21.15.98:0/1658391739 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f650c068400 0x7f650c06a8c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f65140ae000 tx=0x7f6514008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:49.439 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.442+0000 7f651dffb640 1 -- 172.21.15.98:0/1658391739 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6518056d10 con 0x7f6520073930 2024-09-19T08:47:49.588 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.590+0000 7f6527522640 1 -- 172.21.15.98:0/1658391739 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f65200d2320 con 0x7f650c068400 2024-09-19T08:47:49.589 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.590+0000 7f651dffb640 1 -- 172.21.15.98:0/1658391739 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f65200d2320 con 0x7f650c068400 2024-09-19T08:47:49.589 INFO:teuthology.orchestra.run.smithi098.stderr:instructed osd(s) 5 to deep-scrub 2024-09-19T08:47:49.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.590+0000 7f6527522640 1 -- 172.21.15.98:0/1658391739 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f650c068400 msgr2=0x7f650c06a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:49.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.590+0000 7f6527522640 1 --2- 172.21.15.98:0/1658391739 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f650c068400 0x7f650c06a8c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f65140ae000 tx=0x7f6514008040 comp rx=0 tx=0).stop 2024-09-19T08:47:49.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.590+0000 7f6527522640 1 -- 172.21.15.98:0/1658391739 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6520073930 msgr2=0x7f652014c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:49.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.590+0000 7f6527522640 1 --2- 172.21.15.98:0/1658391739 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6520073930 0x7f652014c850 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f651800c7f0 tx=0x7f651800ccb0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.590+0000 7f6527522640 1 -- 172.21.15.98:0/1658391739 shutdown_connections 2024-09-19T08:47:49.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.590+0000 7f6527522640 1 --2- 172.21.15.98:0/1658391739 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f650c068400 0x7f650c06a8c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.590+0000 7f6527522640 1 --2- 172.21.15.98:0/1658391739 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f652015c220 0x7f6520158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.590+0000 7f6527522640 1 --2- 172.21.15.98:0/1658391739 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f65200d1530 0x7f652015d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.592 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.594+0000 7f6527522640 1 --2- 172.21.15.98:0/1658391739 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f6520073930 0x7f652014c850 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.594+0000 7f6527522640 1 -- 172.21.15.98:0/1658391739 >> 172.21.15.98:0/1658391739 conn(0x7f65200b40a0 msgr2=0x7f652015aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:49.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.594+0000 7f6527522640 1 -- 172.21.15.98:0/1658391739 shutdown_connections 2024-09-19T08:47:49.593 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.594+0000 7f6527522640 1 -- 172.21.15.98:0/1658391739 wait complete. 2024-09-19T08:47:49.607 INFO:tasks.ceph:Scrubbing osd.6 2024-09-19T08:47:49.607 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-19T08:47:49.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.734+0000 7ff977bcd640 1 Processor -- start 2024-09-19T08:47:49.733 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.734+0000 7ff977bcd640 1 -- start start 2024-09-19T08:47:49.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.734+0000 7ff977bcd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9700ae140 0x7ff9700b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:49.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.734+0000 7ff977bcd640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9700b6060 con 0x7ff9700b02f0 2024-09-19T08:47:49.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.734+0000 7ff977bcd640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9700b2600 con 0x7ff970073290 2024-09-19T08:47:49.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.734+0000 7ff977bcd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff9700b28e0 con 0x7ff9700ae140 2024-09-19T08:47:49.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.734+0000 7ff975168640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7ff9700b02f0 0x7ff9700b0840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:44516/0 (socket says 172.21.15.98:44516) 2024-09-19T08:47:49.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.734+0000 7ff975168640 1 -- 172.21.15.98:0/3801493254 learned_addr learned my addr 172.21.15.98:0/3801493254 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:49.734 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.734+0000 7ff97616a640 1 --2- 172.21.15.98:0/3801493254 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9700ae140 0x7ff9700b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:49.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.734+0000 7ff974967640 1 -- 172.21.15.98:0/3801493254 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3487594722 0 0) 0x7ff9700b6060 con 0x7ff9700b02f0 2024-09-19T08:47:49.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.734+0000 7ff974967640 1 -- 172.21.15.98:0/3801493254 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff95c003530 con 0x7ff9700b02f0 2024-09-19T08:47:49.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.734+0000 7ff974967640 1 -- 172.21.15.98:0/3801493254 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1275797563 0 0) 0x7ff95c003530 con 0x7ff9700b02f0 2024-09-19T08:47:49.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.734+0000 7ff974967640 1 -- 172.21.15.98:0/3801493254 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9700ae140 msgr2=0x7ff9700b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:49.736 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.734+0000 7ff974967640 1 --2- 172.21.15.98:0/3801493254 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9700ae140 0x7ff9700b6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.734+0000 7ff974967640 1 -- 172.21.15.98:0/3801493254 >> v1:172.21.15.139:6789/0 conn(0x7ff970073290 legacy=0x7ff970073670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:49.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.734+0000 7ff974967640 1 -- 172.21.15.98:0/3801493254 --> v1:172.21.15.98:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9700b6300 con 0x7ff9700b02f0 2024-09-19T08:47:49.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.734+0000 7ff974967640 1 -- 172.21.15.98:0/3801493254 <== mon.0 v1:172.21.15.98:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 883640744 0 0) 0x7ff96c001a00 con 0x7ff9700b02f0 2024-09-19T08:47:49.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.734+0000 7ff974967640 1 -- 172.21.15.98:0/3801493254 >> v1:172.21.15.98:6789/0 conn(0x7ff9700b02f0 legacy=0x7ff9700b0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:49.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.734+0000 7ff974967640 1 --2- 172.21.15.98:0/3801493254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff95c0039f0 0x7ff95c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:49.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.734+0000 7ff974967640 1 --2- 172.21.15.98:0/3801493254 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9700ae140 0x7ff95c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:49.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.734+0000 7ff974967640 1 --2- 172.21.15.98:0/3801493254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff95c004d50 0x7ff95c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:49.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.734+0000 7ff974967640 1 -- 172.21.15.98:0/3801493254 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff9700b28e0 con 0x7ff95c0039f0 2024-09-19T08:47:49.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.734+0000 7ff974967640 1 -- 172.21.15.98:0/3801493254 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c00b9d0 con 0x7ff95c004d50 2024-09-19T08:47:49.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.734+0000 7ff974967640 1 -- 172.21.15.98:0/3801493254 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff95c00bbd0 con 0x7ff9700ae140 2024-09-19T08:47:49.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.734+0000 7ff975969640 1 --2- 172.21.15.98:0/3801493254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff95c0039f0 0x7ff95c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:49.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.734+0000 7ff97616a640 1 --2- 172.21.15.98:0/3801493254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff95c004d50 0x7ff95c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:49.737 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.734+0000 7ff975969640 1 -- 172.21.15.98:0/3801493254 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9700ae140 msgr2=0x7ff95c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:49.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.734+0000 7ff975168640 1 --2- 172.21.15.98:0/3801493254 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9700ae140 0x7ff95c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:49.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.734+0000 7ff975969640 1 --2- 172.21.15.98:0/3801493254 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9700ae140 0x7ff95c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.734+0000 7ff975969640 1 -- 172.21.15.98:0/3801493254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff95c004d50 msgr2=0x7ff95c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:49.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.734+0000 7ff975969640 1 --2- 172.21.15.98:0/3801493254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff95c004d50 0x7ff95c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.734+0000 7ff975969640 1 -- 172.21.15.98:0/3801493254 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff95c003530 con 0x7ff95c0039f0 2024-09-19T08:47:49.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.734+0000 7ff97616a640 1 --2- 172.21.15.98:0/3801493254 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff95c004d50 0x7ff95c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:49.738 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.734+0000 7ff975168640 1 --2- 172.21.15.98:0/3801493254 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9700ae140 0x7ff95c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:49.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.738+0000 7ff975969640 1 --2- 172.21.15.98:0/3801493254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff95c0039f0 0x7ff95c003dd0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7ff964001860 tx=0x7ff96401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35dd5777d5c2026 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:49.739 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.738+0000 7ff974967640 1 -- 172.21.15.98:0/3801493254 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9640b0070 con 0x7ff95c0039f0 2024-09-19T08:47:49.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.738+0000 7ff974967640 1 -- 172.21.15.98:0/3801493254 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9640ab600 con 0x7ff95c0039f0 2024-09-19T08:47:49.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.738+0000 7ff974967640 1 -- 172.21.15.98:0/3801493254 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff964001460 con 0x7ff95c0039f0 2024-09-19T08:47:49.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.738+0000 7ff977bcd640 1 -- 172.21.15.98:0/3801493254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff95c0039f0 msgr2=0x7ff95c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:49.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.738+0000 7ff977bcd640 1 --2- 172.21.15.98:0/3801493254 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff95c0039f0 0x7ff95c003dd0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7ff964001860 tx=0x7ff96401c460 comp rx=0 tx=0).stop 2024-09-19T08:47:49.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.738+0000 7ff97616a640 1 -- 172.21.15.98:0/3801493254 reap_dead start 2024-09-19T08:47:49.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.738+0000 7ff977bcd640 1 -- 172.21.15.98:0/3801493254 shutdown_connections 2024-09-19T08:47:49.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.738+0000 7ff977bcd640 1 -- 172.21.15.98:0/3801493254 >> 172.21.15.98:0/3801493254 conn(0x7ff9700b2e60 msgr2=0x7ff9700b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:49.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.738+0000 7ff977bcd640 1 -- 172.21.15.98:0/3801493254 shutdown_connections 2024-09-19T08:47:49.740 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.738+0000 7ff977bcd640 1 -- 172.21.15.98:0/3801493254 wait complete. 2024-09-19T08:47:49.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.738+0000 7ff977bcd640 1 Processor -- start 2024-09-19T08:47:49.741 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.738+0000 7ff977bcd640 1 -- start start 2024-09-19T08:47:49.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.742+0000 7ff977bcd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff97009a2e0 0x7ff9700c7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:49.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.742+0000 7ff977bcd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff97009a6c0 0x7ff97009b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:49.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.742+0000 7ff977bcd640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9700c51f0 0x7ff97009e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:49.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.742+0000 7ff977bcd640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7ff970166f90 con 0x7ff97009a6c0 2024-09-19T08:47:49.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.742+0000 7ff977bcd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff970166e10 con 0x7ff97009a2e0 2024-09-19T08:47:49.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.742+0000 7ff977bcd640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7ff970167110 con 0x7ff9700c51f0 2024-09-19T08:47:49.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.742+0000 7ff975168640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff97009a6c0 0x7ff97009b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:49.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.742+0000 7ff975168640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff97009a6c0 0x7ff97009b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36740/0 (socket says 172.21.15.98:36740) 2024-09-19T08:47:49.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.742+0000 7ff975168640 1 -- 172.21.15.98:0/1597941130 learned_addr learned my addr 172.21.15.98:0/1597941130 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:49.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.742+0000 7ff975969640 1 --2- 172.21.15.98:0/1597941130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff97009a2e0 0x7ff9700c7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:49.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.742+0000 7ff97616a640 1 --2- 172.21.15.98:0/1597941130 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9700c51f0 0x7ff97009e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:49.743 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.742+0000 7ff975168640 1 -- 172.21.15.98:0/1597941130 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9700c51f0 msgr2=0x7ff97009e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:49.744 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.742+0000 7ff975168640 1 --2- 172.21.15.98:0/1597941130 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7ff9700c51f0 0x7ff97009e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.744 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.742+0000 7ff975168640 1 -- 172.21.15.98:0/1597941130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff97009a2e0 msgr2=0x7ff9700c7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:49.744 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.742+0000 7ff975168640 1 --2- 172.21.15.98:0/1597941130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff97009a2e0 0x7ff9700c7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.744 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.742+0000 7ff975168640 1 -- 172.21.15.98:0/1597941130 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff97009d8b0 con 0x7ff97009a6c0 2024-09-19T08:47:49.744 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.742+0000 7ff975969640 1 --2- 172.21.15.98:0/1597941130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff97009a2e0 0x7ff9700c7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T08:47:49.744 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.742+0000 7ff975168640 1 --2- 172.21.15.98:0/1597941130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff97009a6c0 0x7ff97009b2c0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7ff96c0021c0 tx=0x7ff96c00b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:49.744 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.742+0000 7ff9627fc640 1 -- 172.21.15.98:0/1597941130 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c019070 con 0x7ff97009a6c0 2024-09-19T08:47:49.744 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.742+0000 7ff9627fc640 1 -- 172.21.15.98:0/1597941130 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff96c0072d0 con 0x7ff97009a6c0 2024-09-19T08:47:49.744 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.742+0000 7ff977bcd640 1 -- 172.21.15.98:0/1597941130 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff97009db90 con 0x7ff97009a6c0 2024-09-19T08:47:49.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.742+0000 7ff9627fc640 1 -- 172.21.15.98:0/1597941130 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c0076c0 con 0x7ff97009a6c0 2024-09-19T08:47:49.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.742+0000 7ff977bcd640 1 -- 172.21.15.98:0/1597941130 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff97009e150 con 0x7ff97009a6c0 2024-09-19T08:47:49.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.742+0000 7ff977bcd640 1 -- 172.21.15.98:0/1597941130 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff970073670 con 0x7ff97009a6c0 2024-09-19T08:47:49.745 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.742+0000 7ff9627fc640 1 -- 172.21.15.98:0/1597941130 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7ff96c01c070 con 0x7ff97009a6c0 2024-09-19T08:47:49.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.742+0000 7ff9627fc640 1 --2- 172.21.15.98:0/1597941130 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff954068400 0x7ff95406a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:49.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.742+0000 7ff9627fc640 1 -- 172.21.15.98:0/1597941130 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7ff96c085560 con 0x7ff97009a6c0 2024-09-19T08:47:49.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.742+0000 7ff9627fc640 1 --2- 172.21.15.98:0/1597941130 >> [v2:172.21.15.139:6800/2917689006,v1:172.21.15.139:6801/2917689006] conn(0x7ff95406e2c0 0x7ff9540706e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:49.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.742+0000 7ff9627fc640 1 -- 172.21.15.98:0/1597941130 --> [v2:172.21.15.139:6800/2917689006,v1:172.21.15.139:6801/2917689006] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff96c014e50 con 0x7ff95406e2c0 2024-09-19T08:47:49.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.742+0000 7ff9627fc640 1 -- 172.21.15.98:0/1597941130 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=173) ==== 24+0+0 (secure 0 0 0) 0x7ff96c088240 con 0x7ff97009a6c0 2024-09-19T08:47:49.746 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.746+0000 7ff975969640 1 --2- 172.21.15.98:0/1597941130 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff954068400 0x7ff95406a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:49.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.746+0000 7ff97616a640 1 --2- 172.21.15.98:0/1597941130 >> [v2:172.21.15.139:6800/2917689006,v1:172.21.15.139:6801/2917689006] conn(0x7ff95406e2c0 0x7ff9540706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:49.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.746+0000 7ff97616a640 1 --2- 172.21.15.98:0/1597941130 >> [v2:172.21.15.139:6800/2917689006,v1:172.21.15.139:6801/2917689006] conn(0x7ff95406e2c0 0x7ff9540706e0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:49.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.746+0000 7ff975969640 1 --2- 172.21.15.98:0/1597941130 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff954068400 0x7ff95406a8c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff9640ad000 tx=0x7ff964005040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:49.747 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.746+0000 7ff9627fc640 1 -- 172.21.15.98:0/1597941130 <== osd.6 v2:172.21.15.139:6800/2917689006 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff96c014e50 con 0x7ff95406e2c0 2024-09-19T08:47:49.759 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.758+0000 7ff977bcd640 1 -- 172.21.15.98:0/1597941130 --> [v2:172.21.15.139:6800/2917689006,v1:172.21.15.139:6801/2917689006] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff9700c8120 con 0x7ff95406e2c0 2024-09-19T08:47:49.760 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.762+0000 7ff9627fc640 1 -- 172.21.15.98:0/1597941130 <== osd.6 v2:172.21.15.139:6800/2917689006 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff9700c8120 con 0x7ff95406e2c0 2024-09-19T08:47:49.760 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:47:49.760 INFO:teuthology.orchestra.run.smithi098.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T08:47:49.760 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:47:49.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.762+0000 7ff977bcd640 1 -- 172.21.15.98:0/1597941130 >> [v2:172.21.15.139:6800/2917689006,v1:172.21.15.139:6801/2917689006] conn(0x7ff95406e2c0 msgr2=0x7ff9540706e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:49.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.762+0000 7ff977bcd640 1 --2- 172.21.15.98:0/1597941130 >> [v2:172.21.15.139:6800/2917689006,v1:172.21.15.139:6801/2917689006] conn(0x7ff95406e2c0 0x7ff9540706e0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.761 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.762+0000 7ff977bcd640 1 -- 172.21.15.98:0/1597941130 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff954068400 msgr2=0x7ff95406a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:49.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.762+0000 7ff977bcd640 1 --2- 172.21.15.98:0/1597941130 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7ff954068400 0x7ff95406a8c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff9640ad000 tx=0x7ff964005040 comp rx=0 tx=0).stop 2024-09-19T08:47:49.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.762+0000 7ff977bcd640 1 -- 172.21.15.98:0/1597941130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff97009a6c0 msgr2=0x7ff97009b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:49.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.762+0000 7ff977bcd640 1 --2- 172.21.15.98:0/1597941130 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7ff97009a6c0 0x7ff97009b2c0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7ff96c0021c0 tx=0x7ff96c00b560 comp rx=0 tx=0).stop 2024-09-19T08:47:49.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.762+0000 7ff97616a640 1 -- 172.21.15.98:0/1597941130 reap_dead start 2024-09-19T08:47:49.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.762+0000 7ff977bcd640 1 -- 172.21.15.98:0/1597941130 shutdown_connections 2024-09-19T08:47:49.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.762+0000 7ff977bcd640 1 -- 172.21.15.98:0/1597941130 >> 172.21.15.98:0/1597941130 conn(0x7ff9700b2e60 msgr2=0x7ff970073880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:49.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.762+0000 7ff977bcd640 1 -- 172.21.15.98:0/1597941130 shutdown_connections 2024-09-19T08:47:49.762 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.762+0000 7ff977bcd640 1 -- 172.21.15.98:0/1597941130 wait complete. 2024-09-19T08:47:49.776 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-19T08:47:49.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.890+0000 7f37968a8640 1 Processor -- start 2024-09-19T08:47:49.890 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.890+0000 7f37968a8640 1 -- start start 2024-09-19T08:47:49.891 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.890+0000 7f37968a8640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f379006d880 0x7f37900b3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:49.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.890+0000 7f37968a8640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f3790141110 0x7f37900b3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:49.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.890+0000 7f37968a8640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f379015ea10 con 0x7f3790140d30 2024-09-19T08:47:49.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.890+0000 7f37968a8640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f379015e7c0 con 0x7f379006d880 2024-09-19T08:47:49.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.890+0000 7f37968a8640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f379015ecf0 con 0x7f3790141110 2024-09-19T08:47:49.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.890+0000 7f378ffff640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f3790140d30 0x7f379006d170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:44528/0 (socket says 172.21.15.98:44528) 2024-09-19T08:47:49.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.890+0000 7f378ffff640 1 -- 172.21.15.98:0/2154386321 learned_addr learned my addr 172.21.15.98:0/2154386321 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:49.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.890+0000 7f378f7fe640 1 --2- 172.21.15.98:0/2154386321 >> v2:172.21.15.98:3300/0 conn(0x7f379006d880 0x7f37900b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:49.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.894+0000 7f3794e45640 1 --2- 172.21.15.98:0/2154386321 >> v2:172.21.15.139:3300/0 conn(0x7f3790141110 0x7f37900b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:49.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.894+0000 7f378effd640 1 -- 172.21.15.98:0/2154386321 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 536341501 0 0) 0x7f379015ea10 con 0x7f3790140d30 2024-09-19T08:47:49.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.894+0000 7f378f7fe640 1 -- 172.21.15.98:0/2154386321 >> v2:172.21.15.139:3300/0 conn(0x7f3790141110 msgr2=0x7f37900b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:49.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.894+0000 7f378f7fe640 1 --2- 172.21.15.98:0/2154386321 >> v2:172.21.15.139:3300/0 conn(0x7f3790141110 0x7f37900b3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.892 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.894+0000 7f378f7fe640 1 -- 172.21.15.98:0/2154386321 >> v1:172.21.15.98:6789/0 conn(0x7f3790140d30 legacy=0x7f379006d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:49.893 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.894+0000 7f378f7fe640 1 -- 172.21.15.98:0/2154386321 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f379015efd0 con 0x7f379006d880 2024-09-19T08:47:49.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.894+0000 7f378f7fe640 1 --2- 172.21.15.98:0/2154386321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f379006d880 0x7f37900b3310 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f379006cd40 tx=0x7f378401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf594ad96147f34 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:49.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.894+0000 7f378effd640 1 -- 172.21.15.98:0/2154386321 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37840aa600 con 0x7f379006d880 2024-09-19T08:47:49.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.894+0000 7f378effd640 1 -- 172.21.15.98:0/2154386321 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37840af070 con 0x7f379006d880 2024-09-19T08:47:49.894 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.894+0000 7f378effd640 1 -- 172.21.15.98:0/2154386321 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37840aa9c0 con 0x7f379006d880 2024-09-19T08:47:49.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.894+0000 7f37968a8640 1 -- 172.21.15.98:0/2154386321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f379006d880 msgr2=0x7f37900b3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:49.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.894+0000 7f37968a8640 1 --2- 172.21.15.98:0/2154386321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f379006d880 0x7f37900b3310 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f379006cd40 tx=0x7f378401c480 comp rx=0 tx=0).stop 2024-09-19T08:47:49.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.894+0000 7f37968a8640 1 -- 172.21.15.98:0/2154386321 shutdown_connections 2024-09-19T08:47:49.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.894+0000 7f37968a8640 1 --2- 172.21.15.98:0/2154386321 >> v2:172.21.15.139:3300/0 conn(0x7f3790141110 0x7f37900b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.894+0000 7f37968a8640 1 --2- 172.21.15.98:0/2154386321 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f379006d880 0x7f37900b3310 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.894+0000 7f37968a8640 1 -- 172.21.15.98:0/2154386321 >> 172.21.15.98:0/2154386321 conn(0x7f37900b2010 msgr2=0x7f37900b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:49.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.894+0000 7f37968a8640 1 -- 172.21.15.98:0/2154386321 shutdown_connections 2024-09-19T08:47:49.895 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.894+0000 7f37968a8640 1 -- 172.21.15.98:0/2154386321 wait complete. 2024-09-19T08:47:49.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.898+0000 7f37968a8640 1 Processor -- start 2024-09-19T08:47:49.896 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.898+0000 7f37968a8640 1 -- start start 2024-09-19T08:47:49.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.898+0000 7f37968a8640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f379006d880 0x7f3790140200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:49.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.898+0000 7f37968a8640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3790140d30 0x7f37901f56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:49.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.898+0000 7f37968a8640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3790141110 0x7f37900ce440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:49.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.898+0000 7f37968a8640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f379015bdd0 con 0x7f3790141110 2024-09-19T08:47:49.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.898+0000 7f37968a8640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f379015bc50 con 0x7f379006d880 2024-09-19T08:47:49.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.898+0000 7f37968a8640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f379015bf50 con 0x7f3790140d30 2024-09-19T08:47:49.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.898+0000 7f3794e45640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3790141110 0x7f37900ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:49.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.898+0000 7f3794e45640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3790141110 0x7f37900ce440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36748/0 (socket says 172.21.15.98:36748) 2024-09-19T08:47:49.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.898+0000 7f3794e45640 1 -- 172.21.15.98:0/1456306043 learned_addr learned my addr 172.21.15.98:0/1456306043 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:49.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.898+0000 7f378ffff640 1 --2- 172.21.15.98:0/1456306043 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f379006d880 0x7f3790140200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:49.898 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.898+0000 7f378f7fe640 1 --2- 172.21.15.98:0/1456306043 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3790140d30 0x7f37901f56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:49.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.898+0000 7f3794e45640 1 -- 172.21.15.98:0/1456306043 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3790140d30 msgr2=0x7f37901f56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:49.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.898+0000 7f3794e45640 1 --2- 172.21.15.98:0/1456306043 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3790140d30 0x7f37901f56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.898+0000 7f3794e45640 1 -- 172.21.15.98:0/1456306043 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f379006d880 msgr2=0x7f3790140200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:49.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.898+0000 7f3794e45640 1 --2- 172.21.15.98:0/1456306043 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f379006d880 0x7f3790140200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:49.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.898+0000 7f3794e45640 1 -- 172.21.15.98:0/1456306043 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37901f5c20 con 0x7f3790141110 2024-09-19T08:47:49.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.898+0000 7f3794e45640 1 --2- 172.21.15.98:0/1456306043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3790141110 0x7f37900ce440 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f3788007e70 tx=0x7f378800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:49.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.898+0000 7f378cff9640 1 -- 172.21.15.98:0/1456306043 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788014620 con 0x7f3790141110 2024-09-19T08:47:49.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.898+0000 7f378cff9640 1 -- 172.21.15.98:0/1456306043 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3788019070 con 0x7f3790141110 2024-09-19T08:47:49.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.898+0000 7f37968a8640 1 -- 172.21.15.98:0/1456306043 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37900ae280 con 0x7f3790141110 2024-09-19T08:47:49.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.898+0000 7f378cff9640 1 -- 172.21.15.98:0/1456306043 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788014a20 con 0x7f3790141110 2024-09-19T08:47:49.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.898+0000 7f37968a8640 1 -- 172.21.15.98:0/1456306043 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37900ae840 con 0x7f3790141110 2024-09-19T08:47:49.899 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.898+0000 7f37968a8640 1 -- 172.21.15.98:0/1456306043 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37900cf7c0 con 0x7f3790141110 2024-09-19T08:47:49.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.898+0000 7f378cff9640 1 -- 172.21.15.98:0/1456306043 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f3788014bc0 con 0x7f3790141110 2024-09-19T08:47:49.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.898+0000 7f378cff9640 1 --2- 172.21.15.98:0/1456306043 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f377c068400 0x7f377c06a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:49.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.898+0000 7f378cff9640 1 -- 172.21.15.98:0/1456306043 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7f3788084ea0 con 0x7f3790141110 2024-09-19T08:47:49.900 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.902+0000 7f378ffff640 1 --2- 172.21.15.98:0/1456306043 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f377c068400 0x7f377c06a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:49.901 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.902+0000 7f378ffff640 1 --2- 172.21.15.98:0/1456306043 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f377c068400 0x7f377c06a8c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3780000f70 tx=0x7f3780007b60 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:49.903 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:49.906+0000 7f378cff9640 1 -- 172.21.15.98:0/1456306043 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f37901f5c20 con 0x7f3790141110 2024-09-19T08:47:50.062 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.062+0000 7f37968a8640 1 -- 172.21.15.98:0/1456306043 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f379006d0f0 con 0x7f377c068400 2024-09-19T08:47:50.063 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.062+0000 7f378cff9640 1 -- 172.21.15.98:0/1456306043 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f379006d0f0 con 0x7f377c068400 2024-09-19T08:47:50.063 INFO:teuthology.orchestra.run.smithi098.stderr:instructed osd(s) 6 to deep-scrub 2024-09-19T08:47:50.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.066+0000 7f37968a8640 1 -- 172.21.15.98:0/1456306043 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f377c068400 msgr2=0x7f377c06a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:50.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.066+0000 7f37968a8640 1 --2- 172.21.15.98:0/1456306043 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f377c068400 0x7f377c06a8c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3780000f70 tx=0x7f3780007b60 comp rx=0 tx=0).stop 2024-09-19T08:47:50.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.066+0000 7f37968a8640 1 -- 172.21.15.98:0/1456306043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3790141110 msgr2=0x7f37900ce440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:50.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.066+0000 7f37968a8640 1 --2- 172.21.15.98:0/1456306043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3790141110 0x7f37900ce440 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f3788007e70 tx=0x7f378800e4b0 comp rx=0 tx=0).stop 2024-09-19T08:47:50.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.066+0000 7f37968a8640 1 -- 172.21.15.98:0/1456306043 shutdown_connections 2024-09-19T08:47:50.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.066+0000 7f37968a8640 1 --2- 172.21.15.98:0/1456306043 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f377c068400 0x7f377c06a8c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:50.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.066+0000 7f37968a8640 1 --2- 172.21.15.98:0/1456306043 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f3790141110 0x7f37900ce440 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:50.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.066+0000 7f37968a8640 1 --2- 172.21.15.98:0/1456306043 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f3790140d30 0x7f37901f56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:50.066 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.066+0000 7f37968a8640 1 --2- 172.21.15.98:0/1456306043 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f379006d880 0x7f3790140200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:50.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.066+0000 7f37968a8640 1 -- 172.21.15.98:0/1456306043 >> 172.21.15.98:0/1456306043 conn(0x7f37900b2010 msgr2=0x7f379015a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:50.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.066+0000 7f37968a8640 1 -- 172.21.15.98:0/1456306043 shutdown_connections 2024-09-19T08:47:50.067 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.066+0000 7f37968a8640 1 -- 172.21.15.98:0/1456306043 wait complete. 2024-09-19T08:47:50.081 INFO:tasks.ceph:Scrubbing osd.7 2024-09-19T08:47:50.081 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-19T08:47:50.205 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.206+0000 7f89a046e640 1 Processor -- start 2024-09-19T08:47:50.206 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.206+0000 7f89a046e640 1 -- start start 2024-09-19T08:47:50.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.206+0000 7f89a046e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f899815c290 0x7f89980cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:50.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.206+0000 7f89a046e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f89980cf700 0x7f899815ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:50.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.206+0000 7f89a046e640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89981608c0 con 0x7f899806d4c0 2024-09-19T08:47:50.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.206+0000 7f89a046e640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f8998160e80 con 0x7f89980cf700 2024-09-19T08:47:50.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.206+0000 7f89a046e640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f89980cd510 con 0x7f899815c290 2024-09-19T08:47:50.207 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.206+0000 7f899ea0b640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f899806d4c0 0x7f899806d8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:44538/0 (socket says 172.21.15.98:44538) 2024-09-19T08:47:50.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.206+0000 7f899ea0b640 1 -- 172.21.15.98:0/3948078148 learned_addr learned my addr 172.21.15.98:0/3948078148 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:50.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.206+0000 7f899da09640 1 --2- 172.21.15.98:0/3948078148 >> v2:172.21.15.98:3300/0 conn(0x7f89980cf700 0x7f899815ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:50.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.206+0000 7f899e20a640 1 --2- 172.21.15.98:0/3948078148 >> v2:172.21.15.139:3300/0 conn(0x7f899815c290 0x7f89980cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:50.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.206+0000 7f899d208640 1 -- 172.21.15.98:0/3948078148 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3088104867 0 0) 0x7f89981608c0 con 0x7f899806d4c0 2024-09-19T08:47:50.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.206+0000 7f899d208640 1 -- 172.21.15.98:0/3948078148 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8984003530 con 0x7f899806d4c0 2024-09-19T08:47:50.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.206+0000 7f899da09640 1 -- 172.21.15.98:0/3948078148 >> v2:172.21.15.139:3300/0 conn(0x7f899815c290 msgr2=0x7f89980cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:50.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.206+0000 7f899da09640 1 --2- 172.21.15.98:0/3948078148 >> v2:172.21.15.139:3300/0 conn(0x7f899815c290 0x7f89980cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:50.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.206+0000 7f899da09640 1 -- 172.21.15.98:0/3948078148 >> v1:172.21.15.98:6789/0 conn(0x7f899806d4c0 legacy=0x7f899806d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:50.208 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.206+0000 7f899da09640 1 -- 172.21.15.98:0/3948078148 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8998160ba0 con 0x7f89980cf700 2024-09-19T08:47:50.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.210+0000 7f899da09640 1 --2- 172.21.15.98:0/3948078148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89980cf700 0x7f899815ff50 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f89980742e0 tx=0x7f898801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78717d59e3f50dce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:50.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.210+0000 7f899d208640 1 -- 172.21.15.98:0/3948078148 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89880b0070 con 0x7f89980cf700 2024-09-19T08:47:50.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.210+0000 7f899d208640 1 -- 172.21.15.98:0/3948078148 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89880ab600 con 0x7f89980cf700 2024-09-19T08:47:50.209 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.210+0000 7f899d208640 1 -- 172.21.15.98:0/3948078148 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89880ab9b0 con 0x7f89980cf700 2024-09-19T08:47:50.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.210+0000 7f89a046e640 1 -- 172.21.15.98:0/3948078148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89980cf700 msgr2=0x7f899815ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:50.210 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.210+0000 7f89a046e640 1 --2- 172.21.15.98:0/3948078148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89980cf700 0x7f899815ff50 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f89980742e0 tx=0x7f898801c3f0 comp rx=0 tx=0).stop 2024-09-19T08:47:50.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.210+0000 7f89a046e640 1 -- 172.21.15.98:0/3948078148 shutdown_connections 2024-09-19T08:47:50.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.210+0000 7f89a046e640 1 --2- 172.21.15.98:0/3948078148 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89980cf700 0x7f899815ff50 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:50.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.210+0000 7f89a046e640 1 --2- 172.21.15.98:0/3948078148 >> v2:172.21.15.139:3300/0 conn(0x7f899815c290 0x7f89980cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:50.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.210+0000 7f89a046e640 1 -- 172.21.15.98:0/3948078148 >> 172.21.15.98:0/3948078148 conn(0x7f899806c090 msgr2=0x7f899806c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:50.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.210+0000 7f89a046e640 1 -- 172.21.15.98:0/3948078148 shutdown_connections 2024-09-19T08:47:50.211 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.210+0000 7f89a046e640 1 -- 172.21.15.98:0/3948078148 wait complete. 2024-09-19T08:47:50.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.210+0000 7f89a046e640 1 Processor -- start 2024-09-19T08:47:50.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.210+0000 7f89a046e640 1 -- start start 2024-09-19T08:47:50.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.214+0000 7f89a046e640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f899806d4c0 0x7f8998165c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:50.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.214+0000 7f89a046e640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89980cf700 0x7f89981faa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:50.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.214+0000 7f89a046e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f899815c290 0x7f899814adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:50.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.214+0000 7f89a046e640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8998160610 con 0x7f89980cf700 2024-09-19T08:47:50.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.214+0000 7f89a046e640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8998160490 con 0x7f899815c290 2024-09-19T08:47:50.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.214+0000 7f89a046e640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f89980b5400 con 0x7f899806d4c0 2024-09-19T08:47:50.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.214+0000 7f899da09640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89980cf700 0x7f89981faa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:50.214 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.214+0000 7f899da09640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89980cf700 0x7f89981faa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36760/0 (socket says 172.21.15.98:36760) 2024-09-19T08:47:50.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.214+0000 7f899da09640 1 -- 172.21.15.98:0/3777534408 learned_addr learned my addr 172.21.15.98:0/3777534408 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:50.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.214+0000 7f899ea0b640 1 --2- 172.21.15.98:0/3777534408 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f899815c290 0x7f899814adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:50.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.214+0000 7f899e20a640 1 --2- 172.21.15.98:0/3777534408 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f899806d4c0 0x7f8998165c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:50.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.214+0000 7f899da09640 1 -- 172.21.15.98:0/3777534408 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f899806d4c0 msgr2=0x7f8998165c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:50.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.214+0000 7f899da09640 1 --2- 172.21.15.98:0/3777534408 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f899806d4c0 0x7f8998165c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:50.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.214+0000 7f899da09640 1 -- 172.21.15.98:0/3777534408 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f899815c290 msgr2=0x7f899814adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:50.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.214+0000 7f899da09640 1 --2- 172.21.15.98:0/3777534408 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f899815c290 0x7f899814adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:50.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.214+0000 7f899da09640 1 -- 172.21.15.98:0/3777534408 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89981fb100 con 0x7f89980cf700 2024-09-19T08:47:50.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.214+0000 7f899ea0b640 1 --2- 172.21.15.98:0/3777534408 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f899815c290 0x7f899814adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:47:50.215 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.214+0000 7f899da09640 1 --2- 172.21.15.98:0/3777534408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89980cf700 0x7f89981faa70 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f8988000c10 tx=0x7f8988019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:50.216 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.214+0000 7f898effd640 1 -- 172.21.15.98:0/3777534408 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89880ba070 con 0x7f89980cf700 2024-09-19T08:47:50.216 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.214+0000 7f898effd640 1 -- 172.21.15.98:0/3777534408 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89880ab9b0 con 0x7f89980cf700 2024-09-19T08:47:50.216 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.214+0000 7f89a046e640 1 -- 172.21.15.98:0/3777534408 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89981fb3e0 con 0x7f89980cf700 2024-09-19T08:47:50.216 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.214+0000 7f898effd640 1 -- 172.21.15.98:0/3777534408 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89880b0040 con 0x7f89980cf700 2024-09-19T08:47:50.216 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.214+0000 7f89a046e640 1 -- 172.21.15.98:0/3777534408 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8998164630 con 0x7f89980cf700 2024-09-19T08:47:50.217 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.214+0000 7f89a046e640 1 -- 172.21.15.98:0/3777534408 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f89981fb900 con 0x7f89980cf700 2024-09-19T08:47:50.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.214+0000 7f898effd640 1 -- 172.21.15.98:0/3777534408 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f89880bd070 con 0x7f89980cf700 2024-09-19T08:47:50.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.214+0000 7f898effd640 1 --2- 172.21.15.98:0/3777534408 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f897c068400 0x7f897c06a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:50.219 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.214+0000 7f898effd640 1 -- 172.21.15.98:0/3777534408 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7f898811ded0 con 0x7f89980cf700 2024-09-19T08:47:50.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.214+0000 7f898effd640 1 --2- 172.21.15.98:0/3777534408 >> [v2:172.21.15.139:6816/688014474,v1:172.21.15.139:6817/688014474] conn(0x7f897c06e2c0 0x7f897c0706e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:50.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.214+0000 7f898effd640 1 -- 172.21.15.98:0/3777534408 --> [v2:172.21.15.139:6816/688014474,v1:172.21.15.139:6817/688014474] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f897c070db0 con 0x7f897c06e2c0 2024-09-19T08:47:50.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.214+0000 7f898effd640 1 -- 172.21.15.98:0/3777534408 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_get_version_reply(handle=1 version=173) ==== 24+0+0 (secure 0 0 0) 0x7f89881207a0 con 0x7f89980cf700 2024-09-19T08:47:50.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.214+0000 7f899e20a640 1 --2- 172.21.15.98:0/3777534408 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f897c068400 0x7f897c06a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:50.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.214+0000 7f899ea0b640 1 --2- 172.21.15.98:0/3777534408 >> [v2:172.21.15.139:6816/688014474,v1:172.21.15.139:6817/688014474] conn(0x7f897c06e2c0 0x7f897c0706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:50.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.218+0000 7f899e20a640 1 --2- 172.21.15.98:0/3777534408 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f897c068400 0x7f897c06a8c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8994009990 tx=0x7f8994008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:50.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.218+0000 7f899ea0b640 1 --2- 172.21.15.98:0/3777534408 >> [v2:172.21.15.139:6816/688014474,v1:172.21.15.139:6817/688014474] conn(0x7f897c06e2c0 0x7f897c0706e0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:50.220 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.218+0000 7f898effd640 1 -- 172.21.15.98:0/3777534408 <== osd.7 v2:172.21.15.139:6816/688014474 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f897c070db0 con 0x7f897c06e2c0 2024-09-19T08:47:50.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.230+0000 7f89a046e640 1 -- 172.21.15.98:0/3777534408 --> [v2:172.21.15.139:6816/688014474,v1:172.21.15.139:6817/688014474] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f899815ff50 con 0x7f897c06e2c0 2024-09-19T08:47:50.231 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.234+0000 7f898effd640 1 -- 172.21.15.98:0/3777534408 <== osd.7 v2:172.21.15.139:6816/688014474 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f899815ff50 con 0x7f897c06e2c0 2024-09-19T08:47:50.232 INFO:teuthology.orchestra.run.smithi098.stdout:{ 2024-09-19T08:47:50.232 INFO:teuthology.orchestra.run.smithi098.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T08:47:50.232 INFO:teuthology.orchestra.run.smithi098.stdout:} 2024-09-19T08:47:50.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.234+0000 7f89a046e640 1 -- 172.21.15.98:0/3777534408 >> [v2:172.21.15.139:6816/688014474,v1:172.21.15.139:6817/688014474] conn(0x7f897c06e2c0 msgr2=0x7f897c0706e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:50.233 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.234+0000 7f89a046e640 1 --2- 172.21.15.98:0/3777534408 >> [v2:172.21.15.139:6816/688014474,v1:172.21.15.139:6817/688014474] conn(0x7f897c06e2c0 0x7f897c0706e0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:50.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.234+0000 7f89a046e640 1 -- 172.21.15.98:0/3777534408 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f897c068400 msgr2=0x7f897c06a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:50.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.234+0000 7f89a046e640 1 --2- 172.21.15.98:0/3777534408 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f897c068400 0x7f897c06a8c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8994009990 tx=0x7f8994008040 comp rx=0 tx=0).stop 2024-09-19T08:47:50.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.234+0000 7f89a046e640 1 -- 172.21.15.98:0/3777534408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89980cf700 msgr2=0x7f89981faa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:50.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.234+0000 7f89a046e640 1 --2- 172.21.15.98:0/3777534408 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f89980cf700 0x7f89981faa70 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f8988000c10 tx=0x7f8988019190 comp rx=0 tx=0).stop 2024-09-19T08:47:50.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.234+0000 7f899ea0b640 1 -- 172.21.15.98:0/3777534408 reap_dead start 2024-09-19T08:47:50.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.234+0000 7f89a046e640 1 -- 172.21.15.98:0/3777534408 shutdown_connections 2024-09-19T08:47:50.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.234+0000 7f89a046e640 1 -- 172.21.15.98:0/3777534408 >> 172.21.15.98:0/3777534408 conn(0x7f899806c090 msgr2=0x7f899806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:50.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.234+0000 7f89a046e640 1 -- 172.21.15.98:0/3777534408 shutdown_connections 2024-09-19T08:47:50.234 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.234+0000 7f89a046e640 1 -- 172.21.15.98:0/3777534408 wait complete. 2024-09-19T08:47:50.248 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-19T08:47:50.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.362+0000 7f8fbb2e3640 1 Processor -- start 2024-09-19T08:47:50.362 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.362+0000 7f8fbb2e3640 1 -- start start 2024-09-19T08:47:50.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.362+0000 7f8fbb2e3640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8fb406bc90 0x7f8fb406c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:50.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.362+0000 7f8fbb2e3640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fb40ce840 0x7f8fb40cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:50.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.362+0000 7f8fbb2e3640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fb40ccc60 con 0x7f8fb4158fd0 2024-09-19T08:47:50.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.362+0000 7f8fbb2e3640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7f8fb41593b0 con 0x7f8fb406bc90 2024-09-19T08:47:50.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.362+0000 7f8fbb2e3640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8fb40ccf40 con 0x7f8fb40ce840 2024-09-19T08:47:50.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.362+0000 7f8fb9880640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fb40ce840 0x7f8fb40cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:50.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.362+0000 7f8fb887e640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7f8fb406bc90 0x7f8fb406c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:50.363 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.362+0000 7f8fb9880640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fb40ce840 0x7f8fb40cec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3301/0 says I am v2:172.21.15.98:44596/0 (socket says 172.21.15.98:44596) 2024-09-19T08:47:50.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.362+0000 7f8fb9880640 1 -- 172.21.15.98:0/457418174 learned_addr learned my addr 172.21.15.98:0/457418174 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:50.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.366+0000 7f8fb9880640 1 -- 172.21.15.98:0/457418174 >> v2:172.21.15.139:3300/0 conn(0x7f8fb406bc90 msgr2=0x7f8fb406c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:50.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.366+0000 7f8fb9880640 1 --2- 172.21.15.98:0/457418174 >> v2:172.21.15.139:3300/0 conn(0x7f8fb406bc90 0x7f8fb406c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:50.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.366+0000 7f8fb9880640 1 -- 172.21.15.98:0/457418174 >> v1:172.21.15.139:6789/0 conn(0x7f8fb4158fd0 legacy=0x7f8fb40b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:50.364 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.366+0000 7f8fb9880640 1 -- 172.21.15.98:0/457418174 --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb40cd220 con 0x7f8fb40ce840 2024-09-19T08:47:50.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.366+0000 7f8fb9880640 1 --2- 172.21.15.98:0/457418174 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fb40ce840 0x7f8fb40cec70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4009870 tx=0x7f8fa401c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3163189bfc3da70d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:50.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.366+0000 7f8fb3fff640 1 -- 172.21.15.98:0/457418174 <== mon.2 v2:172.21.15.139:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa40b1070 con 0x7f8fb40ce840 2024-09-19T08:47:50.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.366+0000 7f8fb3fff640 1 -- 172.21.15.98:0/457418174 <== mon.2 v2:172.21.15.139:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fa40ac620 con 0x7f8fb40ce840 2024-09-19T08:47:50.366 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.366+0000 7f8fb3fff640 1 -- 172.21.15.98:0/457418174 <== mon.2 v2:172.21.15.139:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa40ac9f0 con 0x7f8fb40ce840 2024-09-19T08:47:50.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.366+0000 7f8fbb2e3640 1 -- 172.21.15.98:0/457418174 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fb40ce840 msgr2=0x7f8fb40cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:50.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.366+0000 7f8fbb2e3640 1 --2- 172.21.15.98:0/457418174 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fb40ce840 0x7f8fb40cec70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4009870 tx=0x7f8fa401c4d0 comp rx=0 tx=0).stop 2024-09-19T08:47:50.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.366+0000 7f8fbb2e3640 1 -- 172.21.15.98:0/457418174 shutdown_connections 2024-09-19T08:47:50.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.366+0000 7f8fbb2e3640 1 --2- 172.21.15.98:0/457418174 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fb40ce840 0x7f8fb40cec70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:50.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.366+0000 7f8fbb2e3640 1 --2- 172.21.15.98:0/457418174 >> v2:172.21.15.139:3300/0 conn(0x7f8fb406bc90 0x7f8fb406c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:50.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.366+0000 7f8fbb2e3640 1 -- 172.21.15.98:0/457418174 >> 172.21.15.98:0/457418174 conn(0x7f8fb40b2010 msgr2=0x7f8fb40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:50.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.366+0000 7f8fbb2e3640 1 -- 172.21.15.98:0/457418174 shutdown_connections 2024-09-19T08:47:50.367 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.366+0000 7f8fbb2e3640 1 -- 172.21.15.98:0/457418174 wait complete. 2024-09-19T08:47:50.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.370+0000 7f8fbb2e3640 1 Processor -- start 2024-09-19T08:47:50.368 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.370+0000 7f8fbb2e3640 1 -- start start 2024-09-19T08:47:50.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.370+0000 7f8fbb2e3640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8fb406bc90 0x7f8fb4163360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:50.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.370+0000 7f8fbb2e3640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fb40ce840 0x7f8fb41638a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:50.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.370+0000 7f8fbb2e3640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fb4158fd0 0x7f8fb4143990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:50.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.370+0000 7f8fbb2e3640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb415c920 con 0x7f8fb406bc90 2024-09-19T08:47:50.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.370+0000 7f8fbb2e3640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb415c7a0 con 0x7f8fb4158fd0 2024-09-19T08:47:50.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.370+0000 7f8fbb2e3640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f8fb4168110 con 0x7f8fb40ce840 2024-09-19T08:47:50.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.370+0000 7f8fb907f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8fb406bc90 0x7f8fb4163360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:50.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.370+0000 7f8fb907f640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8fb406bc90 0x7f8fb4163360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36764/0 (socket says 172.21.15.98:36764) 2024-09-19T08:47:50.370 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.370+0000 7f8fb907f640 1 -- 172.21.15.98:0/2072551846 learned_addr learned my addr 172.21.15.98:0/2072551846 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:50.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.370+0000 7f8fb887e640 1 --2- 172.21.15.98:0/2072551846 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fb40ce840 0x7f8fb41638a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:50.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.370+0000 7f8fb9880640 1 --2- 172.21.15.98:0/2072551846 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fb4158fd0 0x7f8fb4143990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:50.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.370+0000 7f8fb907f640 1 -- 172.21.15.98:0/2072551846 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fb40ce840 msgr2=0x7f8fb41638a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:50.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.370+0000 7f8fb907f640 1 --2- 172.21.15.98:0/2072551846 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fb40ce840 0x7f8fb41638a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:50.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.370+0000 7f8fb907f640 1 -- 172.21.15.98:0/2072551846 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fb4158fd0 msgr2=0x7f8fb4143990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:50.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.370+0000 7f8fb907f640 1 --2- 172.21.15.98:0/2072551846 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fb4158fd0 0x7f8fb4143990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:50.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.370+0000 7f8fb907f640 1 -- 172.21.15.98:0/2072551846 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb415f080 con 0x7f8fb406bc90 2024-09-19T08:47:50.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.370+0000 7f8fb907f640 1 --2- 172.21.15.98:0/2072551846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8fb406bc90 0x7f8fb4163360 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00c730 tx=0x7f8fac00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:50.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.370+0000 7f8fb1ffb640 1 -- 172.21.15.98:0/2072551846 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac016020 con 0x7f8fb406bc90 2024-09-19T08:47:50.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.370+0000 7f8fb1ffb640 1 -- 172.21.15.98:0/2072551846 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fac006d60 con 0x7f8fb406bc90 2024-09-19T08:47:50.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.370+0000 7f8fb1ffb640 1 -- 172.21.15.98:0/2072551846 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac003bc0 con 0x7f8fb406bc90 2024-09-19T08:47:50.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.370+0000 7f8fbb2e3640 1 -- 172.21.15.98:0/2072551846 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb415f360 con 0x7f8fb406bc90 2024-09-19T08:47:50.371 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.370+0000 7f8fbb2e3640 1 -- 172.21.15.98:0/2072551846 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb41561e0 con 0x7f8fb406bc90 2024-09-19T08:47:50.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.370+0000 7f8fbb2e3640 1 -- 172.21.15.98:0/2072551846 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fb406c120 con 0x7f8fb406bc90 2024-09-19T08:47:50.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.370+0000 7f8fb1ffb640 1 -- 172.21.15.98:0/2072551846 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f8fac00a850 con 0x7f8fb406bc90 2024-09-19T08:47:50.372 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.374+0000 7f8fb1ffb640 1 --2- 172.21.15.98:0/2072551846 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8fa0068400 0x7f8fa006a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:50.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.374+0000 7f8fb1ffb640 1 -- 172.21.15.98:0/2072551846 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7f8fac084500 con 0x7f8fb406bc90 2024-09-19T08:47:50.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.374+0000 7f8fb887e640 1 --2- 172.21.15.98:0/2072551846 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8fa0068400 0x7f8fa006a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:50.373 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.374+0000 7f8fb887e640 1 --2- 172.21.15.98:0/2072551846 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8fa0068400 0x7f8fa006a8c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8fb41655b0 tx=0x7f8fa8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:50.376 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.378+0000 7f8fb1ffb640 1 -- 172.21.15.98:0/2072551846 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f8fac020020 con 0x7f8fb406bc90 2024-09-19T08:47:50.534 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.534+0000 7f8fbb2e3640 1 -- 172.21.15.98:0/2072551846 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f8fb415b980 con 0x7f8fa0068400 2024-09-19T08:47:50.535 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.534+0000 7f8fb1ffb640 1 -- 172.21.15.98:0/2072551846 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8fb415b980 con 0x7f8fa0068400 2024-09-19T08:47:50.535 INFO:teuthology.orchestra.run.smithi098.stderr:instructed osd(s) 7 to deep-scrub 2024-09-19T08:47:50.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.538+0000 7f8fbb2e3640 1 -- 172.21.15.98:0/2072551846 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8fa0068400 msgr2=0x7f8fa006a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:50.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.538+0000 7f8fbb2e3640 1 --2- 172.21.15.98:0/2072551846 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8fa0068400 0x7f8fa006a8c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8fb41655b0 tx=0x7f8fa8008040 comp rx=0 tx=0).stop 2024-09-19T08:47:50.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.538+0000 7f8fbb2e3640 1 -- 172.21.15.98:0/2072551846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8fb406bc90 msgr2=0x7f8fb4163360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:50.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.538+0000 7f8fbb2e3640 1 --2- 172.21.15.98:0/2072551846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8fb406bc90 0x7f8fb4163360 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00c730 tx=0x7f8fac00cbf0 comp rx=0 tx=0).stop 2024-09-19T08:47:50.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.538+0000 7f8fbb2e3640 1 -- 172.21.15.98:0/2072551846 shutdown_connections 2024-09-19T08:47:50.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.538+0000 7f8fbb2e3640 1 --2- 172.21.15.98:0/2072551846 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f8fa0068400 0x7f8fa006a8c0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:50.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.538+0000 7f8fbb2e3640 1 --2- 172.21.15.98:0/2072551846 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8fb4158fd0 0x7f8fb4143990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:50.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.538+0000 7f8fbb2e3640 1 --2- 172.21.15.98:0/2072551846 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f8fb40ce840 0x7f8fb41638a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:50.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.538+0000 7f8fbb2e3640 1 --2- 172.21.15.98:0/2072551846 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f8fb406bc90 0x7f8fb4163360 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:50.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.538+0000 7f8fbb2e3640 1 -- 172.21.15.98:0/2072551846 >> 172.21.15.98:0/2072551846 conn(0x7f8fb40b2010 msgr2=0x7f8fb40b3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:50.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.538+0000 7f8fbb2e3640 1 -- 172.21.15.98:0/2072551846 shutdown_connections 2024-09-19T08:47:50.538 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.538+0000 7f8fbb2e3640 1 -- 172.21.15.98:0/2072551846 wait complete. 2024-09-19T08:47:50.554 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:47:50.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.678+0000 7f53c5581640 1 Processor -- start 2024-09-19T08:47:50.676 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.678+0000 7f53c5581640 1 -- start start 2024-09-19T08:47:50.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.678+0000 7f53c5581640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f53c00cef60 0x7f53c0071520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:50.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.678+0000 7f53c5581640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53c0071a60 0x7f53c00cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:50.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.678+0000 7f53c5581640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53c00cffe0 con 0x7f53c015dee0 2024-09-19T08:47:50.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.678+0000 7f53c5581640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7f53c00cf460 con 0x7f53c00cef60 2024-09-19T08:47:50.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.678+0000 7f53c5581640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f53c00cf740 con 0x7f53c0071a60 2024-09-19T08:47:50.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.678+0000 7f53be7fc640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7f53c00cef60 0x7f53c0071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:50.678 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.678+0000 7f53beffd640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7f53c015dee0 0x7f53c00afaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:44546/0 (socket says 172.21.15.98:44546) 2024-09-19T08:47:50.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.678+0000 7f53beffd640 1 -- 172.21.15.98:0/987610106 learned_addr learned my addr 172.21.15.98:0/987610106 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:47:50.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.678+0000 7f53bf7fe640 1 --2- 172.21.15.98:0/987610106 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53c0071a60 0x7f53c00cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:50.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.678+0000 7f53be7fc640 1 -- 172.21.15.98:0/987610106 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53c0071a60 msgr2=0x7f53c00cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:47:50.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.678+0000 7f53be7fc640 1 --2- 172.21.15.98:0/987610106 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53c0071a60 0x7f53c00cb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:50.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.678+0000 7f53be7fc640 1 -- 172.21.15.98:0/987610106 >> v1:172.21.15.98:6789/0 conn(0x7f53c015dee0 legacy=0x7f53c00afaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:50.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.678+0000 7f53bdffb640 1 -- 172.21.15.98:0/987610106 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 795228353 0 0) 0x7f53c00cffe0 con 0x7f53c015dee0 2024-09-19T08:47:50.679 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.678+0000 7f53be7fc640 1 -- 172.21.15.98:0/987610106 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c00cfa20 con 0x7f53c00cef60 2024-09-19T08:47:50.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.682+0000 7f53be7fc640 1 --2- 172.21.15.98:0/987610106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53c00cef60 0x7f53c0071520 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f53a80097c0 tx=0x7f53a801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=485f6cdd404bdfb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:50.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.682+0000 7f53bdffb640 1 -- 172.21.15.98:0/987610106 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a80b0070 con 0x7f53c00cef60 2024-09-19T08:47:50.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.682+0000 7f53bdffb640 1 -- 172.21.15.98:0/987610106 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53a80ab600 con 0x7f53c00cef60 2024-09-19T08:47:50.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.682+0000 7f53bdffb640 1 -- 172.21.15.98:0/987610106 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a80ab9d0 con 0x7f53c00cef60 2024-09-19T08:47:50.681 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.682+0000 7f53c5581640 1 -- 172.21.15.98:0/987610106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53c00cef60 msgr2=0x7f53c0071520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:50.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.682+0000 7f53c5581640 1 --2- 172.21.15.98:0/987610106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53c00cef60 0x7f53c0071520 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f53a80097c0 tx=0x7f53a801c4a0 comp rx=0 tx=0).stop 2024-09-19T08:47:50.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.682+0000 7f53c5581640 1 -- 172.21.15.98:0/987610106 shutdown_connections 2024-09-19T08:47:50.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.682+0000 7f53c5581640 1 --2- 172.21.15.98:0/987610106 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53c0071a60 0x7f53c00cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:50.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.682+0000 7f53c5581640 1 --2- 172.21.15.98:0/987610106 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53c00cef60 0x7f53c0071520 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:50.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.682+0000 7f53c5581640 1 -- 172.21.15.98:0/987610106 >> 172.21.15.98:0/987610106 conn(0x7f53c00d0980 msgr2=0x7f53c00d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:50.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.682+0000 7f53c5581640 1 -- 172.21.15.98:0/987610106 shutdown_connections 2024-09-19T08:47:50.682 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.682+0000 7f53c5581640 1 -- 172.21.15.98:0/987610106 wait complete. 2024-09-19T08:47:50.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.682+0000 7f53c5581640 1 Processor -- start 2024-09-19T08:47:50.684 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.682+0000 7f53c5581640 1 -- start start 2024-09-19T08:47:50.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.682+0000 7f53c5581640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53c0071a60 0x7f53c0157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:50.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.682+0000 7f53c5581640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53c015dee0 0x7f53c00b6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:50.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.682+0000 7f53c5581640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53c0157570 0x7f53c01479d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:50.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.682+0000 7f53c5581640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7f53c00cc510 con 0x7f53c0071a60 2024-09-19T08:47:50.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.682+0000 7f53c5581640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f53c00cc390 con 0x7f53c0157570 2024-09-19T08:47:50.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.682+0000 7f53c5581640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7f53c00cc690 con 0x7f53c015dee0 2024-09-19T08:47:50.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.682+0000 7f53beffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53c0071a60 0x7f53c0157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:50.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.682+0000 7f53beffd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53c0071a60 0x7f53c0157a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:36786/0 (socket says 172.21.15.98:36786) 2024-09-19T08:47:50.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.682+0000 7f53beffd640 1 -- 172.21.15.98:0/1125993824 learned_addr learned my addr 172.21.15.98:0/1125993824 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:47:50.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.682+0000 7f53be7fc640 1 --2- 172.21.15.98:0/1125993824 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53c015dee0 0x7f53c00b6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:50.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.682+0000 7f53beffd640 1 -- 172.21.15.98:0/1125993824 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53c015dee0 msgr2=0x7f53c00b6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:50.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.682+0000 7f53bf7fe640 1 --2- 172.21.15.98:0/1125993824 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53c0157570 0x7f53c01479d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:50.685 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.682+0000 7f53beffd640 1 --2- 172.21.15.98:0/1125993824 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53c015dee0 0x7f53c00b6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:50.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.682+0000 7f53beffd640 1 -- 172.21.15.98:0/1125993824 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53c0157570 msgr2=0x7f53c01479d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:50.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.682+0000 7f53beffd640 1 --2- 172.21.15.98:0/1125993824 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53c0157570 0x7f53c01479d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:50.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.682+0000 7f53beffd640 1 -- 172.21.15.98:0/1125993824 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c0159ec0 con 0x7f53c0071a60 2024-09-19T08:47:50.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.686+0000 7f53beffd640 1 --2- 172.21.15.98:0/1125993824 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53c0071a60 0x7f53c0157a50 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f53b800c890 tx=0x7f53b800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:50.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.686+0000 7f53b77fe640 1 -- 172.21.15.98:0/1125993824 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b8018070 con 0x7f53c0071a60 2024-09-19T08:47:50.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.686+0000 7f53b77fe640 1 -- 172.21.15.98:0/1125993824 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53b800a640 con 0x7f53c0071a60 2024-09-19T08:47:50.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.686+0000 7f53c5581640 1 -- 172.21.15.98:0/1125993824 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53c015a160 con 0x7f53c0071a60 2024-09-19T08:47:50.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.686+0000 7f53b77fe640 1 -- 172.21.15.98:0/1125993824 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b800aa30 con 0x7f53c0071a60 2024-09-19T08:47:50.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.686+0000 7f53c5581640 1 -- 172.21.15.98:0/1125993824 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53c015a720 con 0x7f53c0071a60 2024-09-19T08:47:50.686 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.686+0000 7f53c5581640 1 -- 172.21.15.98:0/1125993824 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53c0071520 con 0x7f53c0071a60 2024-09-19T08:47:50.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.686+0000 7f53b77fe640 1 -- 172.21.15.98:0/1125993824 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7f53b801b070 con 0x7f53c0071a60 2024-09-19T08:47:50.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.686+0000 7f53b77fe640 1 --2- 172.21.15.98:0/1125993824 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f53a0068400 0x7f53a006a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:47:50.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.686+0000 7f53b77fe640 1 -- 172.21.15.98:0/1125993824 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7f53b8083eb0 con 0x7f53c0071a60 2024-09-19T08:47:50.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.686+0000 7f53be7fc640 1 --2- 172.21.15.98:0/1125993824 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f53a0068400 0x7f53a006a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:47:50.687 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.686+0000 7f53be7fc640 1 --2- 172.21.15.98:0/1125993824 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f53a0068400 0x7f53a006a8c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f53a80ad000 tx=0x7f53a8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:47:50.692 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.690+0000 7f53b77fe640 1 -- 172.21.15.98:0/1125993824 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f53b8056250 con 0x7f53c0071a60 2024-09-19T08:47:50.837 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.838+0000 7f53c5581640 1 -- 172.21.15.98:0/1125993824 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f53c00afa40 con 0x7f53a0068400 2024-09-19T08:47:50.839 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.838+0000 7f53b77fe640 1 -- 172.21.15.98:0/1125993824 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13475 (secure 0 0 0) 0x7f53c00afa40 con 0x7f53a0068400 2024-09-19T08:47:50.839 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:47:50.839 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:47:50.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.842+0000 7f53c5581640 1 -- 172.21.15.98:0/1125993824 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f53a0068400 msgr2=0x7f53a006a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:50.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.842+0000 7f53c5581640 1 --2- 172.21.15.98:0/1125993824 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f53a0068400 0x7f53a006a8c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f53a80ad000 tx=0x7f53a8008040 comp rx=0 tx=0).stop 2024-09-19T08:47:50.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.842+0000 7f53c5581640 1 -- 172.21.15.98:0/1125993824 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53c0071a60 msgr2=0x7f53c0157a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:47:50.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.842+0000 7f53c5581640 1 --2- 172.21.15.98:0/1125993824 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53c0071a60 0x7f53c0157a50 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f53b800c890 tx=0x7f53b800cd50 comp rx=0 tx=0).stop 2024-09-19T08:47:50.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.842+0000 7f53c5581640 1 -- 172.21.15.98:0/1125993824 shutdown_connections 2024-09-19T08:47:50.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.842+0000 7f53c5581640 1 --2- 172.21.15.98:0/1125993824 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7f53a0068400 0x7f53a006a8c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:50.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.842+0000 7f53c5581640 1 --2- 172.21.15.98:0/1125993824 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53c0157570 0x7f53c01479d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:50.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.842+0000 7f53c5581640 1 --2- 172.21.15.98:0/1125993824 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7f53c015dee0 0x7f53c00b6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:50.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.842+0000 7f53c5581640 1 --2- 172.21.15.98:0/1125993824 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7f53c0071a60 0x7f53c0157a50 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:47:50.841 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.842+0000 7f53c5581640 1 -- 172.21.15.98:0/1125993824 >> 172.21.15.98:0/1125993824 conn(0x7f53c00d0980 msgr2=0x7f53c006e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:47:50.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.842+0000 7f53c5581640 1 -- 172.21.15.98:0/1125993824 shutdown_connections 2024-09-19T08:47:50.842 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:47:50.842+0000 7f53c5581640 1 -- 172.21.15.98:0/1125993824 wait complete. 2024-09-19T08:47:50.856 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":249,"stamp":"2024-09-19T08:47:48.969738+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":276048,"kb_used_data":18216,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754698672,"statfs":{"total":773094113280,"available":772811440128,"internally_reserved":0,"allocated":18653184,"data_stored":8929416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-53351,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-3,"num_read_kb":-3,"num_write":-77,"num_write_kb":-82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002451"},"pg_stats":[{"pgid":"1.0","version":"71'128","reported_seq":484,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-19T08:47:45.001376+0000","last_change":"2024-09-19T08:38:09.797498+0000","last_active":"2024-09-19T08:47:45.001376+0000","last_peered":"2024-09-19T08:47:45.001376+0000","last_clean":"2024-09-19T08:47:45.001376+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:47:45.001376+0000","last_undegraded":"2024-09-19T08:47:45.001376+0000","last_fullsized":"2024-09-19T08:47:45.001376+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T08:38:08.696389+0000","last_clean_scrub_stamp":"2024-09-19T08:38:08.696389+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:17:08.919761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509559,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33900,"kb_used_data":2712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337940,"statfs":{"total":96636764160,"available":96602050560,"internally_reserved":0,"allocated":2777088,"data_stored":1558953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574968,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33320,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338520,"statfs":{"total":96636764160,"available":96602644480,"internally_reserved":0,"allocated":2183168,"data_stored":968585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509561,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33320,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338520,"statfs":{"total":96636764160,"available":96602644480,"internally_reserved":0,"allocated":2183168,"data_stored":968585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509561,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33324,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338516,"statfs":{"total":96636764160,"available":96602640384,"internally_reserved":0,"allocated":2183168,"data_stored":968585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542263,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37480,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334360,"statfs":{"total":96636764160,"available":96598384640,"internally_reserved":0,"allocated":2183168,"data_stored":968585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33320,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338520,"statfs":{"total":96636764160,"available":96602644480,"internally_reserved":0,"allocated":2183168,"data_stored":968585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509560,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33320,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338520,"statfs":{"total":96636764160,"available":96602644480,"internally_reserved":0,"allocated":2183168,"data_stored":968585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509558,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38064,"kb_used_data":2712,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333776,"statfs":{"total":96636764160,"available":96597786624,"internally_reserved":0,"allocated":2777088,"data_stored":1558953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:47:50.861 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-19T08:38:08.696389+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=8, tm_min=38, tm_sec=8, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=8, tm_min=47, tm_sec=45, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T08:47:50.861 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-19T08:48:10.862 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T08:48:10.992 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.993+0000 7fed790b4640 1 Processor -- start 2024-09-19T08:48:10.993 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.993+0000 7fed790b4640 1 -- start start 2024-09-19T08:48:10.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.993+0000 7fed790b4640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fed7406bc90 0x7fed7406c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:10.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.993+0000 7fed790b4640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fed740ce840 0x7fed740cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:10.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.993+0000 7fed790b4640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed740ccc60 con 0x7fed74158fd0 2024-09-19T08:48:10.995 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.993+0000 7fed790b4640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fed741593b0 con 0x7fed740ce840 2024-09-19T08:48:10.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.993+0000 7fed790b4640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fed740ccf40 con 0x7fed7406bc90 2024-09-19T08:48:10.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.993+0000 7fed7359e640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fed740ce840 0x7fed740cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:10.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.993+0000 7fed72d9d640 1 --1- >> v1:172.21.15.139:6789/0 conn(0x7fed74158fd0 0x7fed740b3e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.139:6789/0 says I am v1:172.21.15.98:58628/0 (socket says 172.21.15.98:58628) 2024-09-19T08:48:10.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.993+0000 7fed72d9d640 1 -- 172.21.15.98:0/421414520 learned_addr learned my addr 172.21.15.98:0/421414520 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:48:10.996 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.993+0000 7fed7259c640 1 --2- 172.21.15.98:0/421414520 >> v2:172.21.15.139:3300/0 conn(0x7fed7406bc90 0x7fed7406c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:10.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.993+0000 7fed7359e640 1 -- 172.21.15.98:0/421414520 >> v2:172.21.15.139:3300/0 conn(0x7fed7406bc90 msgr2=0x7fed7406c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:10.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.993+0000 7fed7359e640 1 --2- 172.21.15.98:0/421414520 >> v2:172.21.15.139:3300/0 conn(0x7fed7406bc90 0x7fed7406c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:10.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.993+0000 7fed7359e640 1 -- 172.21.15.98:0/421414520 >> v1:172.21.15.139:6789/0 conn(0x7fed74158fd0 legacy=0x7fed740b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:10.997 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.993+0000 7fed7359e640 1 -- 172.21.15.98:0/421414520 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed740cd220 con 0x7fed740ce840 2024-09-19T08:48:11.004 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.997+0000 7fed7359e640 1 --2- 172.21.15.98:0/421414520 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed740ce840 0x7fed740cec70 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fed740cdef0 tx=0x7fed6c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c88e1dc33252c193 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:11.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.997+0000 7fed71d9b640 1 -- 172.21.15.98:0/421414520 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c0b0070 con 0x7fed740ce840 2024-09-19T08:48:11.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.997+0000 7fed71d9b640 1 -- 172.21.15.98:0/421414520 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed6c0ab600 con 0x7fed740ce840 2024-09-19T08:48:11.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.997+0000 7fed71d9b640 1 -- 172.21.15.98:0/421414520 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6c0ab9d0 con 0x7fed740ce840 2024-09-19T08:48:11.005 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.997+0000 7fed790b4640 1 -- 172.21.15.98:0/421414520 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed740ce840 msgr2=0x7fed740cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:11.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.997+0000 7fed790b4640 1 --2- 172.21.15.98:0/421414520 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed740ce840 0x7fed740cec70 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fed740cdef0 tx=0x7fed6c01c3f0 comp rx=0 tx=0).stop 2024-09-19T08:48:11.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.997+0000 7fed790b4640 1 -- 172.21.15.98:0/421414520 shutdown_connections 2024-09-19T08:48:11.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.997+0000 7fed790b4640 1 --2- 172.21.15.98:0/421414520 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed740ce840 0x7fed740cec70 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:11.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.997+0000 7fed790b4640 1 --2- 172.21.15.98:0/421414520 >> v2:172.21.15.139:3300/0 conn(0x7fed7406bc90 0x7fed7406c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:11.006 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.997+0000 7fed790b4640 1 -- 172.21.15.98:0/421414520 >> 172.21.15.98:0/421414520 conn(0x7fed740b2010 msgr2=0x7fed740b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:11.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.997+0000 7fed790b4640 1 -- 172.21.15.98:0/421414520 shutdown_connections 2024-09-19T08:48:11.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.997+0000 7fed790b4640 1 -- 172.21.15.98:0/421414520 wait complete. 2024-09-19T08:48:11.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.997+0000 7fed790b4640 1 Processor -- start 2024-09-19T08:48:11.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.997+0000 7fed790b4640 1 -- start start 2024-09-19T08:48:11.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.997+0000 7fed790b4640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed7406bc90 0x7fed741fb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:11.007 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.997+0000 7fed790b4640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed740ce840 0x7fed7415f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:11.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.997+0000 7fed790b4640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed74158fd0 0x7fed74164740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:11.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.997+0000 7fed790b4640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fed7415c920 con 0x7fed740ce840 2024-09-19T08:48:11.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.997+0000 7fed790b4640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fed7415c7a0 con 0x7fed74158fd0 2024-09-19T08:48:11.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.997+0000 7fed790b4640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fed74168110 con 0x7fed7406bc90 2024-09-19T08:48:11.008 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.997+0000 7fed7259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed740ce840 0x7fed7415f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:11.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.997+0000 7fed7259c640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed740ce840 0x7fed7415f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49042/0 (socket says 172.21.15.98:49042) 2024-09-19T08:48:11.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.997+0000 7fed7259c640 1 -- 172.21.15.98:0/2876331200 learned_addr learned my addr 172.21.15.98:0/2876331200 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:48:11.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.997+0000 7fed72d9d640 1 --2- 172.21.15.98:0/2876331200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed7406bc90 0x7fed741fb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:11.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.997+0000 7fed7359e640 1 --2- 172.21.15.98:0/2876331200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed74158fd0 0x7fed74164740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:11.009 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.997+0000 7fed7259c640 1 -- 172.21.15.98:0/2876331200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed7406bc90 msgr2=0x7fed741fb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:11.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.997+0000 7fed7259c640 1 --2- 172.21.15.98:0/2876331200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed7406bc90 0x7fed741fb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:11.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.997+0000 7fed7259c640 1 -- 172.21.15.98:0/2876331200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed74158fd0 msgr2=0x7fed74164740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:11.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.997+0000 7fed7259c640 1 --2- 172.21.15.98:0/2876331200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed74158fd0 0x7fed74164740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:11.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.997+0000 7fed7259c640 1 -- 172.21.15.98:0/2876331200 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed7415fd70 con 0x7fed740ce840 2024-09-19T08:48:11.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.997+0000 7fed7259c640 1 --2- 172.21.15.98:0/2876331200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed740ce840 0x7fed7415f830 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7fed5c007e60 tx=0x7fed5c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:11.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.997+0000 7fed6b7fe640 1 -- 172.21.15.98:0/2876331200 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5c014620 con 0x7fed740ce840 2024-09-19T08:48:11.010 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.997+0000 7fed6b7fe640 1 -- 172.21.15.98:0/2876331200 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed5c019070 con 0x7fed740ce840 2024-09-19T08:48:11.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.997+0000 7fed790b4640 1 -- 172.21.15.98:0/2876331200 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed740aa6f0 con 0x7fed740ce840 2024-09-19T08:48:11.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.997+0000 7fed6b7fe640 1 -- 172.21.15.98:0/2876331200 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5c014a20 con 0x7fed740ce840 2024-09-19T08:48:11.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:10.997+0000 7fed790b4640 1 -- 172.21.15.98:0/2876331200 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed7415afb0 con 0x7fed740ce840 2024-09-19T08:48:11.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.001+0000 7fed790b4640 1 -- 172.21.15.98:0/2876331200 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed7415ada0 con 0x7fed740ce840 2024-09-19T08:48:11.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.001+0000 7fed6b7fe640 1 -- 172.21.15.98:0/2876331200 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7fed5c014bc0 con 0x7fed740ce840 2024-09-19T08:48:11.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.001+0000 7fed6b7fe640 1 --2- 172.21.15.98:0/2876331200 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fed54068400 0x7fed5406a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:11.011 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.001+0000 7fed6b7fe640 1 -- 172.21.15.98:0/2876331200 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7fed5c085e30 con 0x7fed740ce840 2024-09-19T08:48:11.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.001+0000 7fed72d9d640 1 --2- 172.21.15.98:0/2876331200 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fed54068400 0x7fed5406a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:11.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.001+0000 7fed72d9d640 1 --2- 172.21.15.98:0/2876331200 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fed54068400 0x7fed5406a8c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fed64000f70 tx=0x7fed64009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:11.012 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.005+0000 7fed6b7fe640 1 -- 172.21.15.98:0/2876331200 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fed5c058400 con 0x7fed740ce840 2024-09-19T08:48:11.165 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.165+0000 7fed790b4640 1 -- 172.21.15.98:0/2876331200 --> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fed740aa4a0 con 0x7fed54068400 2024-09-19T08:48:11.167 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.165+0000 7fed6b7fe640 1 -- 172.21.15.98:0/2876331200 <== mgr.4106 v2:172.21.15.139:6836/2000984919 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13379 (secure 0 0 0) 0x7fed740aa4a0 con 0x7fed54068400 2024-09-19T08:48:11.167 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:48:11.168 INFO:teuthology.orchestra.run.smithi098.stderr:dumped all 2024-09-19T08:48:11.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.169+0000 7fed790b4640 1 -- 172.21.15.98:0/2876331200 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fed54068400 msgr2=0x7fed5406a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:11.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.169+0000 7fed790b4640 1 --2- 172.21.15.98:0/2876331200 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fed54068400 0x7fed5406a8c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fed64000f70 tx=0x7fed64009040 comp rx=0 tx=0).stop 2024-09-19T08:48:11.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.169+0000 7fed790b4640 1 -- 172.21.15.98:0/2876331200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed740ce840 msgr2=0x7fed7415f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:11.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.169+0000 7fed790b4640 1 --2- 172.21.15.98:0/2876331200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed740ce840 0x7fed7415f830 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7fed5c007e60 tx=0x7fed5c00e4b0 comp rx=0 tx=0).stop 2024-09-19T08:48:11.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.169+0000 7fed790b4640 1 -- 172.21.15.98:0/2876331200 shutdown_connections 2024-09-19T08:48:11.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.169+0000 7fed790b4640 1 --2- 172.21.15.98:0/2876331200 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fed54068400 0x7fed5406a8c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:11.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.169+0000 7fed790b4640 1 --2- 172.21.15.98:0/2876331200 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fed74158fd0 0x7fed74164740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:11.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.169+0000 7fed790b4640 1 --2- 172.21.15.98:0/2876331200 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fed740ce840 0x7fed7415f830 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:11.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.169+0000 7fed790b4640 1 --2- 172.21.15.98:0/2876331200 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fed7406bc90 0x7fed741fb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:11.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.169+0000 7fed790b4640 1 -- 172.21.15.98:0/2876331200 >> 172.21.15.98:0/2876331200 conn(0x7fed740b2010 msgr2=0x7fed74156f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:11.170 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.169+0000 7fed790b4640 1 -- 172.21.15.98:0/2876331200 shutdown_connections 2024-09-19T08:48:11.171 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.169+0000 7fed790b4640 1 -- 172.21.15.98:0/2876331200 wait complete. 2024-09-19T08:48:11.186 INFO:teuthology.orchestra.run.smithi098.stdout:{"pg_ready":true,"pg_map":{"version":260,"stamp":"2024-09-19T08:48:10.975483+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":276036,"kb_used_data":18216,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754698684,"statfs":{"total":773094113280,"available":772811452416,"internally_reserved":0,"allocated":18653184,"data_stored":8929416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":263835184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003093"},"pg_stats":[{"pgid":"1.0","version":"173'130","reported_seq":495,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-19T08:47:51.134128+0000","last_change":"2024-09-19T08:47:51.134200+0000","last_active":"2024-09-19T08:47:51.134128+0000","last_peered":"2024-09-19T08:47:51.134128+0000","last_clean":"2024-09-19T08:47:51.134128+0000","last_became_active":"2024-09-19T08:38:09.795249+0000","last_became_peered":"2024-09-19T08:38:09.795249+0000","last_unstale":"2024-09-19T08:47:51.134128+0000","last_undegraded":"2024-09-19T08:47:51.134128+0000","last_fullsized":"2024-09-19T08:47:51.134128+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'130","last_scrub_stamp":"2024-09-19T08:47:51.133917+0000","last_deep_scrub":"173'130","last_deep_scrub_stamp":"2024-09-19T08:47:51.133917+0000","last_clean_scrub_stamp":"2024-09-19T08:47:51.133917+0000","objects_scrubbed":2,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T20:07:08.020794+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509563,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33896,"kb_used_data":2712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337944,"statfs":{"total":96636764160,"available":96602054656,"internally_reserved":0,"allocated":2777088,"data_stored":1558953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574972,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33320,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338520,"statfs":{"total":96636764160,"available":96602644480,"internally_reserved":0,"allocated":2183168,"data_stored":968585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509565,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33320,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338520,"statfs":{"total":96636764160,"available":96602644480,"internally_reserved":0,"allocated":2183168,"data_stored":968585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509565,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33324,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338516,"statfs":{"total":96636764160,"available":96602640384,"internally_reserved":0,"allocated":2183168,"data_stored":968585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542267,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37480,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334360,"statfs":{"total":96636764160,"available":96598384640,"internally_reserved":0,"allocated":2183168,"data_stored":968585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509562,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33320,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338520,"statfs":{"total":96636764160,"available":96602644480,"internally_reserved":0,"allocated":2183168,"data_stored":968585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509564,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33320,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338520,"statfs":{"total":96636764160,"available":96602644480,"internally_reserved":0,"allocated":2183168,"data_stored":968585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509562,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38056,"kb_used_data":2712,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333784,"statfs":{"total":96636764160,"available":96597794816,"internally_reserved":0,"allocated":2777088,"data_stored":1558953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T08:48:11.187 DEBUG:teuthology.orchestra.run.smithi098:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-19T08:48:11.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.313+0000 7fbaea940640 1 Processor -- start 2024-09-19T08:48:11.312 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.313+0000 7fbaea940640 1 -- start start 2024-09-19T08:48:11.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.313+0000 7fbaea940640 1 --2- >> v2:172.21.15.139:3300/0 conn(0x7fbae406c3a0 0x7fbae40713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:11.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.313+0000 7fbaea940640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbae40718f0 0x7fbae40b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:11.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.313+0000 7fbaea940640 1 -- --> v1:172.21.15.139:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbae4073530 con 0x7fbae4145b70 2024-09-19T08:48:11.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.313+0000 7fbaea940640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7fbae4073810 con 0x7fbae40718f0 2024-09-19T08:48:11.313 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.313+0000 7fbaea940640 1 -- --> v2:172.21.15.139:3300/0 -- mon_getmap magic: 0 -- 0x7fbae4073af0 con 0x7fbae406c3a0 2024-09-19T08:48:11.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.313+0000 7fbae8edd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbae40718f0 0x7fbae40b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:11.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.313+0000 7fbae8edd640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7fbae40718f0 0x7fbae40b3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49052/0 (socket says 172.21.15.98:49052) 2024-09-19T08:48:11.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.313+0000 7fbae8edd640 1 -- 172.21.15.98:0/3891373308 learned_addr learned my addr 172.21.15.98:0/3891373308 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:48:11.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.313+0000 7fbae37fe640 1 --2- 172.21.15.98:0/3891373308 >> v2:172.21.15.139:3300/0 conn(0x7fbae406c3a0 0x7fbae40713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:11.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.313+0000 7fbae8edd640 1 -- 172.21.15.98:0/3891373308 >> v2:172.21.15.139:3300/0 conn(0x7fbae406c3a0 msgr2=0x7fbae40713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:11.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.313+0000 7fbae8edd640 1 --2- 172.21.15.98:0/3891373308 >> v2:172.21.15.139:3300/0 conn(0x7fbae406c3a0 0x7fbae40713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:11.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.313+0000 7fbae8edd640 1 -- 172.21.15.98:0/3891373308 >> v1:172.21.15.139:6789/0 conn(0x7fbae4145b70 legacy=0x7fbae406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:11.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.313+0000 7fbae8edd640 1 -- 172.21.15.98:0/3891373308 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae4073dd0 con 0x7fbae40718f0 2024-09-19T08:48:11.314 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.313+0000 7fbae37fe640 1 --2- 172.21.15.98:0/3891373308 >> v2:172.21.15.139:3300/0 conn(0x7fbae406c3a0 0x7fbae40713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:48:11.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.317+0000 7fbae8edd640 1 --2- 172.21.15.98:0/3891373308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbae40718f0 0x7fbae40b3410 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fbae4074420 tx=0x7fbadc01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa0fcdf73efc3a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:11.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.317+0000 7fbae2ffd640 1 -- 172.21.15.98:0/3891373308 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbadc0a5620 con 0x7fbae40718f0 2024-09-19T08:48:11.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.317+0000 7fbae2ffd640 1 -- 172.21.15.98:0/3891373308 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbadc0a57c0 con 0x7fbae40718f0 2024-09-19T08:48:11.316 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.317+0000 7fbae2ffd640 1 -- 172.21.15.98:0/3891373308 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbadc0a5b90 con 0x7fbae40718f0 2024-09-19T08:48:11.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.317+0000 7fbaea940640 1 -- 172.21.15.98:0/3891373308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbae40718f0 msgr2=0x7fbae40b3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:11.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.317+0000 7fbaea940640 1 --2- 172.21.15.98:0/3891373308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbae40718f0 0x7fbae40b3410 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fbae4074420 tx=0x7fbadc01c960 comp rx=0 tx=0).stop 2024-09-19T08:48:11.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.317+0000 7fbaea940640 1 -- 172.21.15.98:0/3891373308 shutdown_connections 2024-09-19T08:48:11.317 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.317+0000 7fbaea940640 1 --2- 172.21.15.98:0/3891373308 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbae40718f0 0x7fbae40b3410 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:11.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.317+0000 7fbaea940640 1 --2- 172.21.15.98:0/3891373308 >> v2:172.21.15.139:3300/0 conn(0x7fbae406c3a0 0x7fbae40713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:11.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.317+0000 7fbaea940640 1 -- 172.21.15.98:0/3891373308 >> 172.21.15.98:0/3891373308 conn(0x7fbae40b2010 msgr2=0x7fbae40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:11.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.317+0000 7fbaea940640 1 -- 172.21.15.98:0/3891373308 shutdown_connections 2024-09-19T08:48:11.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.317+0000 7fbaea940640 1 -- 172.21.15.98:0/3891373308 wait complete. 2024-09-19T08:48:11.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.317+0000 7fbaea940640 1 Processor -- start 2024-09-19T08:48:11.318 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.317+0000 7fbaea940640 1 -- start start 2024-09-19T08:48:11.320 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.317+0000 7fbaea940640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbae406c3a0 0x7fbae40ce870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:11.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.317+0000 7fbaea940640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbae40718f0 0x7fbae4144550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:11.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.317+0000 7fbaea940640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbae4145b70 0x7fbae4144a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:11.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.317+0000 7fbaea940640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7fbae40b44f0 con 0x7fbae4145b70 2024-09-19T08:48:11.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.317+0000 7fbaea940640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbae40b4370 con 0x7fbae406c3a0 2024-09-19T08:48:11.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.317+0000 7fbaea940640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7fbae40b4670 con 0x7fbae40718f0 2024-09-19T08:48:11.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.317+0000 7fbae8edd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbae4145b70 0x7fbae4144a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:11.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.317+0000 7fbae8edd640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbae4145b70 0x7fbae4144a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49064/0 (socket says 172.21.15.98:49064) 2024-09-19T08:48:11.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.317+0000 7fbae8edd640 1 -- 172.21.15.98:0/1113123457 learned_addr learned my addr 172.21.15.98:0/1113123457 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:48:11.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.321+0000 7fbae37fe640 1 --2- 172.21.15.98:0/1113123457 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbae40718f0 0x7fbae4144550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:11.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.321+0000 7fbae3fff640 1 --2- 172.21.15.98:0/1113123457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbae406c3a0 0x7fbae40ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:11.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.321+0000 7fbae8edd640 1 -- 172.21.15.98:0/1113123457 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbae40718f0 msgr2=0x7fbae4144550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:11.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.321+0000 7fbae8edd640 1 --2- 172.21.15.98:0/1113123457 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbae40718f0 0x7fbae4144550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:11.321 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.321+0000 7fbae8edd640 1 -- 172.21.15.98:0/1113123457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbae406c3a0 msgr2=0x7fbae40ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:11.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.321+0000 7fbae8edd640 1 --2- 172.21.15.98:0/1113123457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbae406c3a0 0x7fbae40ce870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:11.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.321+0000 7fbae8edd640 1 -- 172.21.15.98:0/1113123457 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae40cd7d0 con 0x7fbae4145b70 2024-09-19T08:48:11.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.321+0000 7fbae3fff640 1 --2- 172.21.15.98:0/1113123457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbae406c3a0 0x7fbae40ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T08:48:11.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.321+0000 7fbae8edd640 1 --2- 172.21.15.98:0/1113123457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbae4145b70 0x7fbae4144a90 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fbadc0af040 tx=0x7fbadc019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:11.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.321+0000 7fbae0ff9640 1 -- 172.21.15.98:0/1113123457 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbadc0bc020 con 0x7fbae4145b70 2024-09-19T08:48:11.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.321+0000 7fbae0ff9640 1 -- 172.21.15.98:0/1113123457 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbadc0b8670 con 0x7fbae4145b70 2024-09-19T08:48:11.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.321+0000 7fbaea940640 1 -- 172.21.15.98:0/1113123457 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbae40cdab0 con 0x7fbae4145b70 2024-09-19T08:48:11.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.321+0000 7fbae0ff9640 1 -- 172.21.15.98:0/1113123457 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbadc0b8a40 con 0x7fbae4145b70 2024-09-19T08:48:11.322 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.321+0000 7fbaea940640 1 -- 172.21.15.98:0/1113123457 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbae40ce070 con 0x7fbae4145b70 2024-09-19T08:48:11.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.321+0000 7fbaea940640 1 -- 172.21.15.98:0/1113123457 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbae40713b0 con 0x7fbae4145b70 2024-09-19T08:48:11.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.321+0000 7fbae0ff9640 1 -- 172.21.15.98:0/1113123457 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7fbadc0bf020 con 0x7fbae4145b70 2024-09-19T08:48:11.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.321+0000 7fbae0ff9640 1 --2- 172.21.15.98:0/1113123457 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbac8068430 0x7fbac806a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:11.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.321+0000 7fbae0ff9640 1 -- 172.21.15.98:0/1113123457 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7fbadc11f6e0 con 0x7fbae4145b70 2024-09-19T08:48:11.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.321+0000 7fbae3fff640 1 --2- 172.21.15.98:0/1113123457 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbac8068430 0x7fbac806a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:11.323 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.321+0000 7fbae3fff640 1 --2- 172.21.15.98:0/1113123457 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbac8068430 0x7fbac806a8f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbae4141d30 tx=0x7fbad4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:11.327 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.325+0000 7fbae0ff9640 1 -- 172.21.15.98:0/1113123457 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fbadc0c6030 con 0x7fbae4145b70 2024-09-19T08:48:11.491 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.489+0000 7fbaea940640 1 -- 172.21.15.98:0/1113123457 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fbae40b1bc0 con 0x7fbae4145b70 2024-09-19T08:48:11.498 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.497+0000 7fbae0ff9640 1 -- 172.21.15.98:0/1113123457 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v32)=0 v32) ==== 123+0+0 (secure 0 0 0) 0x7fbadc0165e0 con 0x7fbae4145b70 2024-09-19T08:48:11.499 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.497+0000 7fbae0ff9640 1 -- 172.21.15.98:0/1113123457 <== mon.0 v2:172.21.15.98:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fbadc127080 con 0x7fbae4145b70 2024-09-19T08:48:11.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.501+0000 7fbaea940640 1 -- 172.21.15.98:0/1113123457 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbac8068430 msgr2=0x7fbac806a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:11.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.501+0000 7fbaea940640 1 --2- 172.21.15.98:0/1113123457 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbac8068430 0x7fbac806a8f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbae4141d30 tx=0x7fbad4009040 comp rx=0 tx=0).stop 2024-09-19T08:48:11.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.501+0000 7fbaea940640 1 -- 172.21.15.98:0/1113123457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbae4145b70 msgr2=0x7fbae4144a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:11.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.501+0000 7fbaea940640 1 --2- 172.21.15.98:0/1113123457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbae4145b70 0x7fbae4144a90 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fbadc0af040 tx=0x7fbadc019700 comp rx=0 tx=0).stop 2024-09-19T08:48:11.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.501+0000 7fbaea940640 1 -- 172.21.15.98:0/1113123457 shutdown_connections 2024-09-19T08:48:11.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.501+0000 7fbaea940640 1 --2- 172.21.15.98:0/1113123457 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7fbac8068430 0x7fbac806a8f0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:11.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.501+0000 7fbaea940640 1 --2- 172.21.15.98:0/1113123457 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7fbae4145b70 0x7fbae4144a90 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:11.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.501+0000 7fbaea940640 1 --2- 172.21.15.98:0/1113123457 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7fbae40718f0 0x7fbae4144550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:11.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.501+0000 7fbaea940640 1 --2- 172.21.15.98:0/1113123457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbae406c3a0 0x7fbae40ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:11.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.501+0000 7fbaea940640 1 -- 172.21.15.98:0/1113123457 >> 172.21.15.98:0/1113123457 conn(0x7fbae40b2010 msgr2=0x7fbae40cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:11.501 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.501+0000 7fbaea940640 1 -- 172.21.15.98:0/1113123457 shutdown_connections 2024-09-19T08:48:11.502 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.501+0000 7fbaea940640 1 -- 172.21.15.98:0/1113123457 wait complete. 2024-09-19T08:48:11.518 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T08:48:11.518 DEBUG:teuthology.orchestra.run.smithi098:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T08:48:11.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.637+0000 7feb3bfeb640 1 Processor -- start 2024-09-19T08:48:11.639 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.637+0000 7feb3bfeb640 1 -- start start 2024-09-19T08:48:11.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.641+0000 7feb3bfeb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb3406c3a0 0x7feb340cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:11.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.641+0000 7feb3bfeb640 1 --2- >> v2:172.21.15.98:3300/0 conn(0x7feb340cf900 0x7feb340723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:11.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.641+0000 7feb3bfeb640 1 -- --> v1:172.21.15.98:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb3406d310 con 0x7feb3415c550 2024-09-19T08:48:11.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.641+0000 7feb3bfeb640 1 -- --> v2:172.21.15.98:3300/0 -- mon_getmap magic: 0 -- 0x7feb3406d5f0 con 0x7feb340cf900 2024-09-19T08:48:11.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.641+0000 7feb3bfeb640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7feb3406d8d0 con 0x7feb3406c3a0 2024-09-19T08:48:11.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.641+0000 7feb39d87640 1 --1- >> v1:172.21.15.98:6789/0 conn(0x7feb3415c550 0x7feb3406bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.98:6789/0 says I am v1:172.21.15.98:32960/0 (socket says 172.21.15.98:32960) 2024-09-19T08:48:11.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.641+0000 7feb39d87640 1 -- 172.21.15.98:0/3587387337 learned_addr learned my addr 172.21.15.98:0/3587387337 (peer_addr_for_me v1:172.21.15.98:0/0) 2024-09-19T08:48:11.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.641+0000 7feb3a588640 1 --2- 172.21.15.98:0/3587387337 >> v2:172.21.15.98:3300/0 conn(0x7feb340cf900 0x7feb340723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:11.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.641+0000 7feb39586640 1 --2- 172.21.15.98:0/3587387337 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb3406c3a0 0x7feb340cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:11.641 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.641+0000 7feb38d85640 1 -- 172.21.15.98:0/3587387337 <== mon.0 v1:172.21.15.98:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1900670942 0 0) 0x7feb3406d310 con 0x7feb3415c550 2024-09-19T08:48:11.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.641+0000 7feb38d85640 1 -- 172.21.15.98:0/3587387337 --> v1:172.21.15.98:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb20003530 con 0x7feb3415c550 2024-09-19T08:48:11.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.641+0000 7feb3a588640 1 -- 172.21.15.98:0/3587387337 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb3406c3a0 msgr2=0x7feb340cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T08:48:11.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.641+0000 7feb3a588640 1 --2- 172.21.15.98:0/3587387337 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb3406c3a0 0x7feb340cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:11.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.641+0000 7feb3a588640 1 -- 172.21.15.98:0/3587387337 >> v1:172.21.15.98:6789/0 conn(0x7feb3415c550 legacy=0x7feb3406bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:11.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.641+0000 7feb38d85640 1 -- 172.21.15.98:0/3587387337 <== mon.0 v1:172.21.15.98:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 499737232 0 0) 0x7feb20003530 con 0x7feb3415c550 2024-09-19T08:48:11.642 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.641+0000 7feb3a588640 1 -- 172.21.15.98:0/3587387337 --> v2:172.21.15.98:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb3406dbb0 con 0x7feb340cf900 2024-09-19T08:48:11.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.641+0000 7feb3a588640 1 --2- 172.21.15.98:0/3587387337 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb340cf900 0x7feb340723a0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7feb2c009d90 tx=0x7feb2c01c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52b632ef24f75cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:11.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.641+0000 7feb38d85640 1 -- 172.21.15.98:0/3587387337 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c0b2070 con 0x7feb340cf900 2024-09-19T08:48:11.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.641+0000 7feb38d85640 1 -- 172.21.15.98:0/3587387337 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7feb2c0a55a0 con 0x7feb340cf900 2024-09-19T08:48:11.643 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.641+0000 7feb38d85640 1 -- 172.21.15.98:0/3587387337 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb30001680 con 0x7feb340cf900 2024-09-19T08:48:11.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.645+0000 7feb3bfeb640 1 -- 172.21.15.98:0/3587387337 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb340cf900 msgr2=0x7feb340723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:11.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.645+0000 7feb3bfeb640 1 --2- 172.21.15.98:0/3587387337 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb340cf900 0x7feb340723a0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7feb2c009d90 tx=0x7feb2c01c890 comp rx=0 tx=0).stop 2024-09-19T08:48:11.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.645+0000 7feb3bfeb640 1 -- 172.21.15.98:0/3587387337 shutdown_connections 2024-09-19T08:48:11.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.645+0000 7feb3bfeb640 1 --2- 172.21.15.98:0/3587387337 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb340cf900 0x7feb340723a0 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:11.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.645+0000 7feb3bfeb640 1 --2- 172.21.15.98:0/3587387337 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb3406c3a0 0x7feb340cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:11.644 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.645+0000 7feb3bfeb640 1 -- 172.21.15.98:0/3587387337 >> 172.21.15.98:0/3587387337 conn(0x7feb340b2010 msgr2=0x7feb340b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:11.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.645+0000 7feb3bfeb640 1 -- 172.21.15.98:0/3587387337 shutdown_connections 2024-09-19T08:48:11.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.645+0000 7feb3bfeb640 1 -- 172.21.15.98:0/3587387337 wait complete. 2024-09-19T08:48:11.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.645+0000 7feb3bfeb640 1 Processor -- start 2024-09-19T08:48:11.645 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.645+0000 7feb3bfeb640 1 -- start start 2024-09-19T08:48:11.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.645+0000 7feb3bfeb640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb3406c3a0 0x7feb3414be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:11.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.645+0000 7feb3bfeb640 1 --2- >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb340cf900 0x7feb34159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:11.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.645+0000 7feb3bfeb640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb3415c550 0x7feb341557b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:11.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.645+0000 7feb3bfeb640 1 -- --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_getmap magic: 0 -- 0x7feb34073480 con 0x7feb3415c550 2024-09-19T08:48:11.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.645+0000 7feb3bfeb640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feb34073300 con 0x7feb3406c3a0 2024-09-19T08:48:11.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.645+0000 7feb3bfeb640 1 -- --> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] -- mon_getmap magic: 0 -- 0x7feb34073600 con 0x7feb340cf900 2024-09-19T08:48:11.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.645+0000 7feb3a588640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb3415c550 0x7feb341557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:11.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.645+0000 7feb3a588640 1 --2- >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb3415c550 0x7feb341557b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.98:3300/0 says I am v2:172.21.15.98:49088/0 (socket says 172.21.15.98:49088) 2024-09-19T08:48:11.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.645+0000 7feb3a588640 1 -- 172.21.15.98:0/2744886568 learned_addr learned my addr 172.21.15.98:0/2744886568 (peer_addr_for_me v2:172.21.15.98:0/0) 2024-09-19T08:48:11.647 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.645+0000 7feb39d87640 1 --2- 172.21.15.98:0/2744886568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb3406c3a0 0x7feb3414be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:11.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.645+0000 7feb39586640 1 --2- 172.21.15.98:0/2744886568 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb340cf900 0x7feb34159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:11.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.645+0000 7feb3a588640 1 -- 172.21.15.98:0/2744886568 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb340cf900 msgr2=0x7feb34159290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:11.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.645+0000 7feb3a588640 1 --2- 172.21.15.98:0/2744886568 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb340cf900 0x7feb34159290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:11.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.645+0000 7feb3a588640 1 -- 172.21.15.98:0/2744886568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb3406c3a0 msgr2=0x7feb3414be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:11.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.645+0000 7feb3a588640 1 --2- 172.21.15.98:0/2744886568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb3406c3a0 0x7feb3414be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:11.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.645+0000 7feb3a588640 1 -- 172.21.15.98:0/2744886568 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb34073ef0 con 0x7feb3415c550 2024-09-19T08:48:11.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.645+0000 7feb3a588640 1 --2- 172.21.15.98:0/2744886568 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb3415c550 0x7feb341557b0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0b0040 tx=0x7feb2c019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:11.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.645+0000 7feb2a7fc640 1 -- 172.21.15.98:0/2744886568 <== mon.0 v2:172.21.15.98:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c0b2050 con 0x7feb3415c550 2024-09-19T08:48:11.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.645+0000 7feb2a7fc640 1 -- 172.21.15.98:0/2744886568 <== mon.0 v2:172.21.15.98:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7feb2c0be070 con 0x7feb3415c550 2024-09-19T08:48:11.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.645+0000 7feb3bfeb640 1 -- 172.21.15.98:0/2744886568 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb34159b10 con 0x7feb3415c550 2024-09-19T08:48:11.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.645+0000 7feb2a7fc640 1 -- 172.21.15.98:0/2744886568 <== mon.0 v2:172.21.15.98:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c009450 con 0x7feb3415c550 2024-09-19T08:48:11.648 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.645+0000 7feb3bfeb640 1 -- 172.21.15.98:0/2744886568 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb3415a0d0 con 0x7feb3415c550 2024-09-19T08:48:11.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.649+0000 7feb3bfeb640 1 -- 172.21.15.98:0/2744886568 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb340cccd0 con 0x7feb3415c550 2024-09-19T08:48:11.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.649+0000 7feb2a7fc640 1 -- 172.21.15.98:0/2744886568 <== mon.0 v2:172.21.15.98:3300/0 4 ==== mgrmap(e 27) ==== 88465+0+0 (secure 0 0 0) 0x7feb2c0ada70 con 0x7feb3415c550 2024-09-19T08:48:11.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.649+0000 7feb2a7fc640 1 --2- 172.21.15.98:0/2744886568 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7feb18068430 0x7feb1806a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T08:48:11.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.649+0000 7feb2a7fc640 1 -- 172.21.15.98:0/2744886568 <== mon.0 v2:172.21.15.98:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7feb2c11f310 con 0x7feb3415c550 2024-09-19T08:48:11.649 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.649+0000 7feb39d87640 1 --2- 172.21.15.98:0/2744886568 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7feb18068430 0x7feb1806a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T08:48:11.650 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.649+0000 7feb39d87640 1 --2- 172.21.15.98:0/2744886568 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7feb18068430 0x7feb1806a8f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7feb300011f0 tx=0x7feb30009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T08:48:11.653 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.653+0000 7feb2a7fc640 1 -- 172.21.15.98:0/2744886568 <== mon.0 v2:172.21.15.98:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7feb2c016480 con 0x7feb3415c550 2024-09-19T08:48:11.849 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.849+0000 7feb3bfeb640 1 -- 172.21.15.98:0/2744886568 --> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb340cc770 con 0x7feb3415c550 2024-09-19T08:48:11.850 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.849+0000 7feb2a7fc640 1 -- 172.21.15.98:0/2744886568 <== mon.0 v2:172.21.15.98:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+3919 (secure 0 0 0) 0x7feb2c0f1990 con 0x7feb3415c550 2024-09-19T08:48:11.850 INFO:teuthology.orchestra.run.smithi098.stdout: 2024-09-19T08:48:11.850 INFO:teuthology.orchestra.run.smithi098.stdout:{"epoch":103,"btime":"2024-09-19T08:47:40:809347+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6392,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6834/1027012777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":1027012777},{"type":"v1","addr":"172.21.15.139:6834","nonce":1027012777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7339,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/3216462399","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6833","nonce":3216462399},{"type":"v1","addr":"172.21.15.139:6835","nonce":3216462399}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":9174,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6833/1629579852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6832","nonce":1629579852},{"type":"v1","addr":"172.21.15.98:6833","nonce":1629579852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":9507,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.98:6835/2236878363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.98:6834","nonce":2236878363},{"type":"v1","addr":"172.21.15.98:6835","nonce":2236878363}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103}],"filesystems":[]} 2024-09-19T08:48:11.850 INFO:teuthology.orchestra.run.smithi098.stderr:dumped fsmap epoch 103 2024-09-19T08:48:11.853 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.853+0000 7feb3bfeb640 1 -- 172.21.15.98:0/2744886568 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7feb18068430 msgr2=0x7feb1806a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:11.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.853+0000 7feb3bfeb640 1 --2- 172.21.15.98:0/2744886568 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7feb18068430 0x7feb1806a8f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7feb300011f0 tx=0x7feb30009040 comp rx=0 tx=0).stop 2024-09-19T08:48:11.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.853+0000 7feb3bfeb640 1 -- 172.21.15.98:0/2744886568 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb3415c550 msgr2=0x7feb341557b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T08:48:11.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.853+0000 7feb3bfeb640 1 --2- 172.21.15.98:0/2744886568 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb3415c550 0x7feb341557b0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0b0040 tx=0x7feb2c019630 comp rx=0 tx=0).stop 2024-09-19T08:48:11.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.853+0000 7feb3bfeb640 1 -- 172.21.15.98:0/2744886568 shutdown_connections 2024-09-19T08:48:11.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.853+0000 7feb3bfeb640 1 --2- 172.21.15.98:0/2744886568 >> [v2:172.21.15.139:6836/2000984919,v1:172.21.15.139:6837/2000984919] conn(0x7feb18068430 0x7feb1806a8f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:11.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.853+0000 7feb3bfeb640 1 --2- 172.21.15.98:0/2744886568 >> [v2:172.21.15.98:3300/0,v1:172.21.15.98:6789/0] conn(0x7feb3415c550 0x7feb341557b0 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:11.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.853+0000 7feb3bfeb640 1 --2- 172.21.15.98:0/2744886568 >> [v2:172.21.15.139:3301/0,v1:172.21.15.139:6790/0] conn(0x7feb340cf900 0x7feb34159290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:11.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.853+0000 7feb3bfeb640 1 --2- 172.21.15.98:0/2744886568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb3406c3a0 0x7feb3414be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T08:48:11.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.853+0000 7feb3bfeb640 1 -- 172.21.15.98:0/2744886568 >> 172.21.15.98:0/2744886568 conn(0x7feb340b2010 msgr2=0x7feb34157840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T08:48:11.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.853+0000 7feb3bfeb640 1 -- 172.21.15.98:0/2744886568 shutdown_connections 2024-09-19T08:48:11.854 INFO:teuthology.orchestra.run.smithi098.stderr:2024-09-19T08:48:11.853+0000 7feb3bfeb640 1 -- 172.21.15.98:0/2744886568 wait complete. 2024-09-19T08:48:11.869 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-19T08:48:11.870 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-19T08:48:11.870 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-19T08:48:11.870 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:48:11.881 INFO:tasks.ceph.mds.a:Stopped 2024-09-19T08:48:11.881 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-19T08:48:11.881 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:48:11.934 INFO:tasks.ceph.mds.c:Stopped 2024-09-19T08:48:11.934 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-19T08:48:11.934 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:48:11.950 INFO:tasks.ceph.mds.b:Stopped 2024-09-19T08:48:11.950 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-19T08:48:11.950 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:48:12.016 INFO:tasks.ceph.mds.d:Stopped 2024-09-19T08:48:12.016 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-19T08:48:12.016 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-19T08:48:12.016 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:48:12.074 INFO:tasks.ceph.osd.0:Stopped 2024-09-19T08:48:12.075 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-19T08:48:12.075 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:48:12.174 INFO:tasks.ceph.osd.1:Stopped 2024-09-19T08:48:12.174 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-19T08:48:12.174 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:48:12.268 INFO:tasks.ceph.osd.2:Stopped 2024-09-19T08:48:12.268 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-19T08:48:12.268 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:48:13.640 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-19T08:48:18.370 INFO:tasks.ceph.osd.3:Stopped 2024-09-19T08:48:18.371 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-19T08:48:18.371 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:48:18.426 INFO:tasks.ceph.osd.4:Stopped 2024-09-19T08:48:18.426 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-19T08:48:18.426 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:48:18.494 INFO:tasks.ceph.osd.5:Stopped 2024-09-19T08:48:18.494 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-19T08:48:18.495 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:48:18.585 INFO:tasks.ceph.osd.6:Stopped 2024-09-19T08:48:18.585 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-19T08:48:18.585 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:48:18.678 INFO:tasks.ceph.osd.7:Stopped 2024-09-19T08:48:18.678 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-19T08:48:18.679 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-19T08:48:18.679 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:48:18.710 INFO:tasks.ceph.mgr.y:Stopped 2024-09-19T08:48:18.710 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-19T08:48:18.710 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:48:18.742 INFO:tasks.ceph.mgr.x:Stopped 2024-09-19T08:48:18.742 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-19T08:48:18.742 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-19T08:48:18.742 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:48:18.769 INFO:tasks.ceph.mon.a:Stopped 2024-09-19T08:48:18.769 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-19T08:48:18.769 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:48:18.798 INFO:tasks.ceph.mon.b:Stopped 2024-09-19T08:48:18.798 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-19T08:48:18.798 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T08:48:18.856 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-09-19T08:48:18.857 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2024-09-19T08:48:18.857 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-09-19T08:48:18.857 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-09-19T08:48:18.857 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-19T08:48:18.857 INFO:tasks.ceph.mon.c:Stopped 2024-09-19T08:48:18.857 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-19T08:48:18.857 DEBUG:teuthology.orchestra.run.smithi098:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-09-19T08:48:18.879 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:48:18.879 DEBUG:teuthology.orchestra.run.smithi098:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-19T08:48:18.900 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:48:18.922 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:48:19.247 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:48:19.247 DEBUG:teuthology.orchestra.run.smithi098:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-19T08:48:19.478 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:48:19.478 DEBUG:teuthology.orchestra.run.smithi098:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-19T08:48:19.668 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:48:19.668 DEBUG:teuthology.orchestra.run.smithi098:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-19T08:48:19.852 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:48:19.852 DEBUG:teuthology.orchestra.run.smithi139:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-19T08:48:20.104 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:48:20.105 DEBUG:teuthology.orchestra.run.smithi139:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-19T08:48:20.317 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:48:20.317 DEBUG:teuthology.orchestra.run.smithi139:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-19T08:48:20.524 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:48:20.525 DEBUG:teuthology.orchestra.run.smithi139:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-19T08:48:20.717 INFO:tasks.ceph:Archiving mon data... 2024-09-19T08:48:20.719 DEBUG:teuthology.misc:Transferring archived files from smithi098:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911853/data/mon.a.tgz 2024-09-19T08:48:20.719 DEBUG:teuthology.orchestra.run.smithi098:> mktemp 2024-09-19T08:48:20.725 INFO:teuthology.orchestra.run.smithi098.stdout:/tmp/tmp.QvprEUq6Av 2024-09-19T08:48:20.726 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.QvprEUq6Av 2024-09-19T08:48:20.966 DEBUG:teuthology.orchestra.run.smithi098:> sudo chmod 0666 /tmp/tmp.QvprEUq6Av 2024-09-19T08:48:21.027 DEBUG:teuthology.orchestra.remote:smithi098:/tmp/tmp.QvprEUq6Av is 1MB 2024-09-19T08:48:21.098 DEBUG:teuthology.orchestra.run.smithi098:> rm -fr /tmp/tmp.QvprEUq6Av 2024-09-19T08:48:21.104 DEBUG:teuthology.misc:Transferring archived files from smithi139:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911853/data/mon.b.tgz 2024-09-19T08:48:21.104 DEBUG:teuthology.orchestra.run.smithi139:> mktemp 2024-09-19T08:48:21.110 INFO:teuthology.orchestra.run.smithi139.stdout:/tmp/tmp.ihM7Jfh5KP 2024-09-19T08:48:21.110 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.ihM7Jfh5KP 2024-09-19T08:48:21.363 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 0666 /tmp/tmp.ihM7Jfh5KP 2024-09-19T08:48:21.426 DEBUG:teuthology.orchestra.remote:smithi139:/tmp/tmp.ihM7Jfh5KP is 1MB 2024-09-19T08:48:21.496 DEBUG:teuthology.orchestra.run.smithi139:> rm -fr /tmp/tmp.ihM7Jfh5KP 2024-09-19T08:48:21.501 DEBUG:teuthology.misc:Transferring archived files from smithi139:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911853/data/mon.c.tgz 2024-09-19T08:48:21.501 DEBUG:teuthology.orchestra.run.smithi139:> mktemp 2024-09-19T08:48:21.549 INFO:teuthology.orchestra.run.smithi139.stdout:/tmp/tmp.UIYpl9hvQa 2024-09-19T08:48:21.550 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.UIYpl9hvQa 2024-09-19T08:48:21.797 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 0666 /tmp/tmp.UIYpl9hvQa 2024-09-19T08:48:21.858 DEBUG:teuthology.orchestra.remote:smithi139:/tmp/tmp.UIYpl9hvQa is 1MB 2024-09-19T08:48:21.932 DEBUG:teuthology.orchestra.run.smithi139:> rm -fr /tmp/tmp.UIYpl9hvQa 2024-09-19T08:48:21.938 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-19T08:48:21.938 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-19T08:48:21.940 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-19T08:48:22.004 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-19T08:48:22.046 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-19T08:48:22.046 INFO:tasks.ceph:Archiving crash dumps... 2024-09-19T08:48:22.048 DEBUG:teuthology.misc:Transferring archived files from smithi098:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911853/remote/smithi098/crash 2024-09-19T08:48:22.048 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-19T08:48:22.062 DEBUG:teuthology.misc:Transferring archived files from smithi139:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911853/remote/smithi139/crash 2024-09-19T08:48:22.063 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-19T08:48:22.102 INFO:tasks.ceph:Shutting down logrotate 2024-09-19T08:48:22.102 DEBUG:teuthology.orchestra.run.smithi098:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:48:22.109 DEBUG:teuthology.orchestra.run.smithi139:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T08:48:22.307 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-19T08:48:22.320 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-19T08:48:22.334 INFO:tasks.ceph:Compressing logs... 2024-09-19T08:48:22.334 DEBUG:teuthology.orchestra.run.smithi098:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T08:48:22.365 DEBUG:teuthology.orchestra.run.smithi139:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T08:48:22.389 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35457.log 2024-09-19T08:48:22.389 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47905.log 2024-09-19T08:48:22.391 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.35457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38377.log 2024-09-19T08:48:22.391 INFO:teuthology.orchestra.run.smithi098.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35457.log.gz 2024-09-19T08:48:22.391 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55537.log 2024-09-19T08:48:22.391 INFO:teuthology.orchestra.run.smithi098.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47905.log.gz 2024-09-19T08:48:22.391 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.38377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38280.log 2024-09-19T08:48:22.392 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.55537.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38377.log.gz 2024-09-19T08:48:22.392 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47745.log 2024-09-19T08:48:22.395 INFO:teuthology.orchestra.run.smithi098.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55537.log.gz 2024-09-19T08:48:22.395 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.38280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43268.log 2024-09-19T08:48:22.395 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47745.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38280.log.gz 2024-09-19T08:48:22.395 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35395.log 2024-09-19T08:48:22.395 INFO:teuthology.orchestra.run.smithi098.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47745.log.gz 2024-09-19T08:48:22.395 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52209.log 2024-09-19T08:48:22.395 INFO:teuthology.orchestra.run.smithi098.stderr: 89.0%/var/log/ceph/ceph-client.admin.35395.log: -- replaced with /var/log/ceph/ceph-client.admin.43268.log.gz 2024-09-19T08:48:22.395 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51043.log 2024-09-19T08:48:22.396 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44663.log 2024-09-19T08:48:22.396 INFO:teuthology.orchestra.run.smithi098.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35395.log.gz 2024-09-19T08:48:22.396 INFO:teuthology.orchestra.run.smithi098.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52209.log.gz 2024-09-19T08:48:22.396 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40123.log 2024-09-19T08:48:22.396 INFO:teuthology.orchestra.run.smithi098.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51043.log.gz 2024-09-19T08:48:22.396 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47111.log 2024-09-19T08:48:22.396 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.40123.log: -- /var/log/ceph/ceph-client.admin.54398.log 2024-09-19T08:48:22.396 INFO:teuthology.orchestra.run.smithi098.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44663.log.gz 2024-09-19T08:48:22.396 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49563.log 2024-09-19T08:48:22.396 INFO:teuthology.orchestra.run.smithi098.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40123.log.gz 2024-09-19T08:48:22.396 INFO:teuthology.orchestra.run.smithi098.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47111.log.gz 2024-09-19T08:48:22.397 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.54398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35601.log 2024-09-19T08:48:22.397 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35574.log 2024-09-19T08:48:22.417 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29168.log 2024-09-19T08:48:22.417 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.1.35574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29096.log 2024-09-19T08:48:22.417 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29168.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-19T08:48:22.417 INFO:teuthology.orchestra.run.smithi139.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.29168.log.gz 2024-09-19T08:48:22.417 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34958.log 2024-09-19T08:48:22.417 INFO:teuthology.orchestra.run.smithi139.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.1.35574.log.gz 2024-09-19T08:48:22.417 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29096.log: /var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-19T08:48:22.417 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.1.34958.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-19T08:48:22.417 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.29096.log.gz 2024-09-19T08:48:22.417 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-osd.6.log: 88.5% -- replaced with /var/log/ceph/ceph-client.1.34958.log.gz 2024-09-19T08:48:22.418 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29144.log 2024-09-19T08:48:22.418 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29048.log 2024-09-19T08:48:22.418 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35255.log 2024-09-19T08:48:22.418 INFO:teuthology.orchestra.run.smithi139.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29144.log.gz 2024-09-19T08:48:22.418 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35408.log 2024-09-19T08:48:22.418 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29048.log: /var/log/ceph/ceph-client.1.35255.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-19T08:48:22.418 INFO:teuthology.orchestra.run.smithi139.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.29048.log.gz 2024-09-19T08:48:22.418 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.1.35408.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-19T08:48:22.418 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-mgr.x.log: 88.6% -- replaced with /var/log/ceph/ceph-client.1.35255.log.gz 2024-09-19T08:48:22.418 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.35077.log 2024-09-19T08:48:22.418 INFO:teuthology.orchestra.run.smithi139.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.1.35408.log.gz 2024-09-19T08:48:22.418 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29072.log 2024-09-19T08:48:22.418 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.3.35077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.36162.log 2024-09-19T08:48:22.418 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29072.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-19T08:48:22.419 INFO:teuthology.orchestra.run.smithi139.stderr: 88.2% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.29072.log.gz 2024-09-19T08:48:22.419 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.3.35077.log.gz 2024-09-19T08:48:22.419 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.1.36162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.37158.log 2024-09-19T08:48:22.419 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.36007.log 2024-09-19T08:48:22.419 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.1.37158.log: 92.0% -- replaced with /var/log/ceph/ceph-client.1.36162.log.gz 2024-09-19T08:48:22.419 INFO:teuthology.orchestra.run.smithi139.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.1.37158.log.gz 2024-09-19T08:48:22.419 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.35728.log 2024-09-19T08:48:22.419 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29000.log 2024-09-19T08:48:22.419 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.1.36007.log: /var/log/ceph/ceph-client.1.35728.log: 92.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-19T08:48:22.419 INFO:teuthology.orchestra.run.smithi139.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.1.36007.log.gz 2024-09-19T08:48:22.419 INFO:teuthology.orchestra.run.smithi139.stderr: 89.5%gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-19T08:48:22.419 INFO:teuthology.orchestra.run.smithi139.stderr: -- replaced with /var/log/ceph/ceph-client.1.35728.log.gz 2024-09-19T08:48:22.419 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29000.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-19T08:48:22.419 INFO:teuthology.orchestra.run.smithi139.stderr: 87.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.29000.log.gz -5 --verbose -- /var/log/ceph/ceph-client.1.36968.log 2024-09-19T08:48:22.420 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-09-19T08:48:22.420 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-19T08:48:22.420 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.1.36968.log: 88.9% -- replaced with /var/log/ceph/ceph-client.1.36968.log.gz 2024-09-19T08:48:22.420 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.36386.log 2024-09-19T08:48:22.422 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-mds.b.log: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-19T08:48:22.422 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-19T08:48:22.429 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.1.36386.log: 89.2% -- replaced with /var/log/ceph/ceph-client.1.36386.log.gz 2024-09-19T08:48:22.430 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-19T08:48:22.434 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.36592.log 2024-09-19T08:48:22.440 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-osd.4.log: 90.8% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-19T08:48:22.440 INFO:teuthology.orchestra.run.smithi139.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-19T08:48:22.449 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29024.log 2024-09-19T08:48:22.450 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.1.36592.log: 89.6% -- replaced with /var/log/ceph/ceph-client.1.36592.log.gz 2024-09-19T08:48:22.450 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.36777.log 2024-09-19T08:48:22.450 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29024.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.29024.log.gz 2024-09-19T08:48:22.450 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29120.log 2024-09-19T08:48:22.450 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.1.36777.log: 89.6% -- replaced with /var/log/ceph/ceph-client.1.36777.log.gz 2024-09-19T08:48:22.450 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/ceph-client.admin.29120.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.29120.log.gz 2024-09-19T08:48:22.468 INFO:teuthology.orchestra.run.smithi098.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54398.log.gz 2024-09-19T08:48:22.469 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50523.log 2024-09-19T08:48:22.469 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49563.log.gz 2024-09-19T08:48:22.469 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55433.log 2024-09-19T08:48:22.469 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.35601.log: /var/log/ceph/ceph-client.admin.50523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40227.log 2024-09-19T08:48:22.469 INFO:teuthology.orchestra.run.smithi098.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35601.log.gz 2024-09-19T08:48:22.469 INFO:teuthology.orchestra.run.smithi098.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50523.log.gz 2024-09-19T08:48:22.469 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.55433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51295.log 2024-09-19T08:48:22.469 INFO:teuthology.orchestra.run.smithi098.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55433.log.gz 2024-09-19T08:48:22.469 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.40227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49589.log 2024-09-19T08:48:22.469 INFO:teuthology.orchestra.run.smithi098.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40227.log.gz 2024-09-19T08:48:22.469 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38663.log 2024-09-19T08:48:22.469 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51295.log: gzip/var/log/ceph/ceph-client.admin.49589.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44754.log 2024-09-19T08:48:22.470 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51295.log.gz 2024-09-19T08:48:22.470 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.38663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43500.log 2024-09-19T08:48:22.470 INFO:teuthology.orchestra.run.smithi098.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38663.log.gz 2024-09-19T08:48:22.470 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44754.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.49589.log.gz 2024-09-19T08:48:22.470 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37655.log 2024-09-19T08:48:22.470 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44754.log.gz 2024-09-19T08:48:22.470 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39311.log 2024-09-19T08:48:22.470 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.37655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55459.log 2024-09-19T08:48:22.470 INFO:teuthology.orchestra.run.smithi098.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43500.log.gz 2024-09-19T08:48:22.470 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.39311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41363.log 2024-09-19T08:48:22.470 INFO:teuthology.orchestra.run.smithi098.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37655.log.gz 2024-09-19T08:48:22.470 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39311.log.gz 2024-09-19T08:48:22.470 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.55459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51475.log 2024-09-19T08:48:22.470 INFO:teuthology.orchestra.run.smithi098.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55459.log.gz 2024-09-19T08:48:22.470 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.41363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55799.log 2024-09-19T08:48:22.471 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54476.log 2024-09-19T08:48:22.471 INFO:teuthology.orchestra.run.smithi098.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41363.log.gz 2024-09-19T08:48:22.471 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.55799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53700.log 2024-09-19T08:48:22.471 INFO:teuthology.orchestra.run.smithi098.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51475.log.gz 2024-09-19T08:48:22.471 INFO:teuthology.orchestra.run.smithi098.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55799.log.gz 2024-09-19T08:48:22.471 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.54476.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-19T08:48:22.471 INFO:teuthology.orchestra.run.smithi098.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54476.log.gz 2024-09-19T08:48:22.471 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.53700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43324.log 2024-09-19T08:48:22.471 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41770.log 2024-09-19T08:48:22.471 INFO:teuthology.orchestra.run.smithi098.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53700.log.gz 2024-09-19T08:48:22.471 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50403.log 2024-09-19T08:48:22.471 INFO:teuthology.orchestra.run.smithi098.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43324.log.gz 2024-09-19T08:48:22.471 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.41770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49329.log 2024-09-19T08:48:22.471 INFO:teuthology.orchestra.run.smithi098.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41770.log.gz 2024-09-19T08:48:22.472 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43009.log 2024-09-19T08:48:22.472 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49329.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50403.log.gz 2024-09-19T08:48:22.472 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54292.log 2024-09-19T08:48:22.472 INFO:teuthology.orchestra.run.smithi098.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49329.log.gz 2024-09-19T08:48:22.472 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36221.log 2024-09-19T08:48:22.472 INFO:teuthology.orchestra.run.smithi098.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43009.log.gz 2024-09-19T08:48:22.472 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34871.log 2024-09-19T08:48:22.472 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.54292.log: /var/log/ceph/ceph-client.admin.36221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35653.log 2024-09-19T08:48:22.472 INFO:teuthology.orchestra.run.smithi098.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54292.log.gz 2024-09-19T08:48:22.472 INFO:teuthology.orchestra.run.smithi098.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.36221.log.gz 2024-09-19T08:48:22.472 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.34871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39125.log 2024-09-19T08:48:22.472 INFO:teuthology.orchestra.run.smithi098.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34871.log.gz 2024-09-19T08:48:22.472 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40052.log 2024-09-19T08:48:22.472 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.35653.log: /var/log/ceph/ceph-client.admin.39125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35731.log 2024-09-19T08:48:22.473 INFO:teuthology.orchestra.run.smithi098.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.35653.log.gz 2024-09-19T08:48:22.473 INFO:teuthology.orchestra.run.smithi098.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39125.log.gz 2024-09-19T08:48:22.473 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.40052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50078.log 2024-09-19T08:48:22.473 INFO:teuthology.orchestra.run.smithi098.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40052.log.gz 2024-09-19T08:48:22.473 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5/var/log/ceph/ceph-client.admin.35731.log: --verbose -- /var/log/ceph/ceph-client.admin.52398.log 2024-09-19T08:48:22.473 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39527.log 2024-09-19T08:48:22.473 INFO:teuthology.orchestra.run.smithi098.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35731.log.gz 2024-09-19T08:48:22.473 INFO:teuthology.orchestra.run.smithi098.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50078.log.gz 2024-09-19T08:48:22.473 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36787.log 2024-09-19T08:48:22.473 INFO:teuthology.orchestra.run.smithi098.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52398.log.gz 2024-09-19T08:48:22.473 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38098.log 2024-09-19T08:48:22.473 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.39527.log: gzip/var/log/ceph/ceph-client.admin.36787.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.42745.log 2024-09-19T08:48:22.473 INFO:teuthology.orchestra.run.smithi098.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39527.log.gz 2024-09-19T08:48:22.473 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.38098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51896.log 2024-09-19T08:48:22.474 INFO:teuthology.orchestra.run.smithi098.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36787.log.gz 2024-09-19T08:48:22.474 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42745.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38098.log.gz 2024-09-19T08:48:22.474 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54858.log 2024-09-19T08:48:22.474 INFO:teuthology.orchestra.run.smithi098.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42745.log.gz 2024-09-19T08:48:22.474 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35203.log 2024-09-19T08:48:22.474 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51896.log: /var/log/ceph/ceph-client.admin.54858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55825.log 2024-09-19T08:48:22.474 INFO:teuthology.orchestra.run.smithi098.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54858.log.gz 2024-09-19T08:48:22.474 INFO:teuthology.orchestra.run.smithi098.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51896.log.gz 2024-09-19T08:48:22.474 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.35203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35208.log 2024-09-19T08:48:22.474 INFO:teuthology.orchestra.run.smithi098.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35203.log.gz 2024-09-19T08:48:22.474 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.55825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44730.log 2024-09-19T08:48:22.474 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.35208.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55825.log.gz 2024-09-19T08:48:22.474 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49511.log 2024-09-19T08:48:22.474 INFO:teuthology.orchestra.run.smithi098.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35208.log.gz 2024-09-19T08:48:22.475 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49247.log 2024-09-19T08:48:22.475 INFO:teuthology.orchestra.run.smithi098.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.44730.log.gz 2024-09-19T08:48:22.475 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51425.log 2024-09-19T08:48:22.475 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49511.log.gz 2024-09-19T08:48:22.475 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36605.log 2024-09-19T08:48:22.475 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49247.log: /var/log/ceph/ceph-client.admin.51425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42455.log 2024-09-19T08:48:22.475 INFO:teuthology.orchestra.run.smithi098.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49247.log.gz 2024-09-19T08:48:22.475 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50952.log 2024-09-19T08:48:22.475 INFO:teuthology.orchestra.run.smithi098.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.51425.log.gz 2024-09-19T08:48:22.475 INFO:teuthology.orchestra.run.smithi098.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36605.log.gz 2024-09-19T08:48:22.475 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42455.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-19T08:48:22.475 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50952.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42455.log.gz 2024-09-19T08:48:22.475 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50731.log 2024-09-19T08:48:22.475 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-osd.0.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50952.log.gz 2024-09-19T08:48:22.476 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38455.log 2024-09-19T08:48:22.476 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54979.log 2024-09-19T08:48:22.476 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50731.log: /var/log/ceph/ceph-client.admin.38455.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50731.log.gz 2024-09-19T08:48:22.476 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49195.log 2024-09-19T08:48:22.476 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38455.log.gz 2024-09-19T08:48:22.476 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50353.log 2024-09-19T08:48:22.476 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.54979.log: /var/log/ceph/ceph-client.admin.49195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53525.log 2024-09-19T08:48:22.476 INFO:teuthology.orchestra.run.smithi098.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54979.log.gz 2024-09-19T08:48:22.476 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42719.log 2024-09-19T08:48:22.476 INFO:teuthology.orchestra.run.smithi098.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49195.log.gz 2024-09-19T08:48:22.476 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50353.log.gz 2024-09-19T08:48:22.476 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53883.log 2024-09-19T08:48:22.476 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.53525.log: /var/log/ceph/ceph-client.admin.42719.log: gzip -5 --verbose -- 89.0% /var/log/ceph/ceph-client.admin.42311.log -- replaced with /var/log/ceph/ceph-client.admin.53525.log.gz 2024-09-19T08:48:22.476 INFO:teuthology.orchestra.run.smithi098.stderr: 2024-09-19T08:48:22.477 INFO:teuthology.orchestra.run.smithi098.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42719.log.gz 2024-09-19T08:48:22.477 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.53883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47931.log 2024-09-19T08:48:22.477 INFO:teuthology.orchestra.run.smithi098.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53883.log.gz 2024-09-19T08:48:22.477 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47602.log 2024-09-19T08:48:22.477 INFO:teuthology.orchestra.run.smithi098.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42311.log.gz 2024-09-19T08:48:22.477 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47085.log 2024-09-19T08:48:22.477 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53804.log 2024-09-19T08:48:22.477 INFO:teuthology.orchestra.run.smithi098.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47931.log.gz 2024-09-19T08:48:22.477 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47602.log.gz 2024-09-19T08:48:22.477 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35783.log 2024-09-19T08:48:22.477 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.53804.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47085.log.gz 2024-09-19T08:48:22.477 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51161.log 2024-09-19T08:48:22.477 INFO:teuthology.orchestra.run.smithi098.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53804.log.gz 2024-09-19T08:48:22.477 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49097.log 2024-09-19T08:48:22.477 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.35783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41928.log 2024-09-19T08:48:22.478 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51161.log: /var/log/ceph/ceph-client.admin.49097.log: 89.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.51161.log.gz -5 2024-09-19T08:48:22.478 INFO:teuthology.orchestra.run.smithi098.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50679.log 2024-09-19T08:48:22.478 INFO:teuthology.orchestra.run.smithi098.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49097.log.gz 2024-09-19T08:48:22.478 INFO:teuthology.orchestra.run.smithi098.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.35783.log.gz 2024-09-19T08:48:22.478 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.41928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38176.log 2024-09-19T08:48:22.478 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50679.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41928.log.gz 2024-09-19T08:48:22.478 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54372.log 2024-09-19T08:48:22.478 INFO:teuthology.orchestra.run.smithi098.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50679.log.gz 2024-09-19T08:48:22.478 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.38176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42051.log 2024-09-19T08:48:22.478 INFO:teuthology.orchestra.run.smithi098.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38176.log.gzgzip 2024-09-19T08:48:22.478 INFO:teuthology.orchestra.run.smithi098.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.51835.log 2024-09-19T08:48:22.478 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.54372.log: /var/log/ceph/ceph-client.admin.42051.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54372.log.gz 2024-09-19T08:48:22.478 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34475.log 2024-09-19T08:48:22.478 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42051.log.gz 2024-09-19T08:48:22.479 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53499.log 2024-09-19T08:48:22.479 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51835.log: /var/log/ceph/ceph-client.admin.34475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46804.log 2024-09-19T08:48:22.479 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.53499.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34475.log.gz 2024-09-19T08:48:22.479 INFO:teuthology.orchestra.run.smithi098.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51835.log.gz 2024-09-19T08:48:22.479 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55721.log 2024-09-19T08:48:22.479 INFO:teuthology.orchestra.run.smithi098.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53499.log.gz 2024-09-19T08:48:22.479 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38038.log 2024-09-19T08:48:22.479 INFO:teuthology.orchestra.run.smithi098.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46804.log.gz 2024-09-19T08:48:22.479 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52008.log 2024-09-19T08:48:22.479 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.55721.log: /var/log/ceph/ceph-client.admin.38038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48394.log 2024-09-19T08:48:22.479 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55721.log.gz 2024-09-19T08:48:22.479 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52008.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38038.log.gz 2024-09-19T08:48:22.479 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44867.log 2024-09-19T08:48:22.479 INFO:teuthology.orchestra.run.smithi098.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52008.log.gz 2024-09-19T08:48:22.480 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36496.log 2024-09-19T08:48:22.480 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48394.log: /var/log/ceph/ceph-client.admin.44867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51213.log 2024-09-19T08:48:22.480 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36496.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44867.log.gz 2024-09-19T08:48:22.480 INFO:teuthology.orchestra.run.smithi098.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48394.log.gz 2024-09-19T08:48:22.480 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53577.log 2024-09-19T08:48:22.480 INFO:teuthology.orchestra.run.smithi098.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36496.log.gz 2024-09-19T08:48:22.480 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55258.log 2024-09-19T08:48:22.480 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51213.log: /var/log/ceph/ceph-client.admin.53577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51638.log 2024-09-19T08:48:22.480 INFO:teuthology.orchestra.run.smithi098.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51213.log.gz 2024-09-19T08:48:22.480 INFO:teuthology.orchestra.run.smithi098.stderr: 89.2%/var/log/ceph/ceph-client.admin.55258.log: -- replaced with /var/log/ceph/ceph-client.admin.53577.log.gz 2024-09-19T08:48:22.480 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43415.log 2024-09-19T08:48:22.480 INFO:teuthology.orchestra.run.smithi098.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55258.log.gz 2024-09-19T08:48:22.480 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55232.log 2024-09-19T08:48:22.480 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51638.log: /var/log/ceph/ceph-client.admin.43415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38833.log 2024-09-19T08:48:22.481 INFO:teuthology.orchestra.run.smithi098.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51638.log.gz 2024-09-19T08:48:22.481 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.55232.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43415.log.gz 2024-09-19T08:48:22.481 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34350.log 2024-09-19T08:48:22.481 INFO:teuthology.orchestra.run.smithi098.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.55232.log.gz 2024-09-19T08:48:22.481 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.38833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53059.log 2024-09-19T08:48:22.481 INFO:teuthology.orchestra.run.smithi098.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38833.log.gz 2024-09-19T08:48:22.481 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55851.log 2024-09-19T08:48:22.481 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.34350.log: /var/log/ceph/ceph-client.admin.53059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36962.log 2024-09-19T08:48:22.481 INFO:teuthology.orchestra.run.smithi098.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34350.log.gz 2024-09-19T08:48:22.481 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.55851.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53059.log.gz 2024-09-19T08:48:22.481 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50053.log 2024-09-19T08:48:22.481 INFO:teuthology.orchestra.run.smithi098.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55851.log.gz 2024-09-19T08:48:22.481 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45049.log 2024-09-19T08:48:22.481 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36962.log: /var/log/ceph/ceph-client.admin.50053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48908.log 2024-09-19T08:48:22.481 INFO:teuthology.orchestra.run.smithi098.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36962.log.gz 2024-09-19T08:48:22.482 INFO:teuthology.orchestra.run.smithi098.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50053.log.gz 2024-09-19T08:48:22.482 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39687.log 2024-09-19T08:48:22.482 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52576.log 2024-09-19T08:48:22.482 INFO:teuthology.orchestra.run.smithi098.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45049.log.gz 2024-09-19T08:48:22.482 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48908.log: /var/log/ceph/ceph-client.admin.39687.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48908.log.gz 2024-09-19T08:48:22.482 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40022.log 2024-09-19T08:48:22.482 INFO:teuthology.orchestra.run.smithi098.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39687.log.gz 2024-09-19T08:48:22.482 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47983.log 2024-09-19T08:48:22.482 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52576.log: /var/log/ceph/ceph-client.admin.40022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49903.log 2024-09-19T08:48:22.482 INFO:teuthology.orchestra.run.smithi098.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52576.log.gz 2024-09-19T08:48:22.482 INFO:teuthology.orchestra.run.smithi098.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40022.log.gz 2024-09-19T08:48:22.482 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52639.log 2024-09-19T08:48:22.482 INFO:teuthology.orchestra.run.smithi098.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47983.log.gz 2024-09-19T08:48:22.482 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41822.log 2024-09-19T08:48:22.483 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49903.log: /var/log/ceph/ceph-client.admin.52639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35627.log 2024-09-19T08:48:22.483 INFO:teuthology.orchestra.run.smithi098.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49903.log.gz 2024-09-19T08:48:22.483 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.41822.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52639.log.gz 2024-09-19T08:48:22.483 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40480.log 2024-09-19T08:48:22.483 INFO:teuthology.orchestra.run.smithi098.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41822.log.gz 2024-09-19T08:48:22.483 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.35627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42797.log 2024-09-19T08:48:22.483 INFO:teuthology.orchestra.run.smithi098.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35627.log.gz 2024-09-19T08:48:22.483 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55485.log 2024-09-19T08:48:22.483 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.40480.log: /var/log/ceph/ceph-client.admin.42797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36187.log 2024-09-19T08:48:22.483 INFO:teuthology.orchestra.run.smithi098.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40480.log.gz 2024-09-19T08:48:22.483 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.55485.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42797.log.gz 2024-09-19T08:48:22.484 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28919.log 2024-09-19T08:48:22.484 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55485.log.gz 2024-09-19T08:48:22.484 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44373.log 2024-09-19T08:48:22.484 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36187.log: /var/log/ceph/ceph-client.admin.28919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50003.log 2024-09-19T08:48:22.484 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36187.log.gz 2024-09-19T08:48:22.484 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44373.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.28919.log.gz 2024-09-19T08:48:22.484 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39929.log 2024-09-19T08:48:22.484 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48009.log 2024-09-19T08:48:22.484 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50003.log: /var/log/ceph/ceph-client.admin.39929.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.44373.log.gz 2024-09-19T08:48:22.484 INFO:teuthology.orchestra.run.smithi098.stderr: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.50003.log.gz -5 2024-09-19T08:48:22.484 INFO:teuthology.orchestra.run.smithi098.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50178.log 2024-09-19T08:48:22.484 INFO:teuthology.orchestra.run.smithi098.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39929.log.gz 2024-09-19T08:48:22.484 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38507.log 2024-09-19T08:48:22.485 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48009.log: /var/log/ceph/ceph-client.admin.50178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41900.log 2024-09-19T08:48:22.485 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50178.log.gz 2024-09-19T08:48:22.485 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.38507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42693.log 2024-09-19T08:48:22.485 INFO:teuthology.orchestra.run.smithi098.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.48009.log.gz 2024-09-19T08:48:22.485 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33386.log 2024-09-19T08:48:22.485 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.41900.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38507.log.gz 2024-09-19T08:48:22.485 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42337.log 2024-09-19T08:48:22.485 INFO:teuthology.orchestra.run.smithi098.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41900.log.gz 2024-09-19T08:48:22.485 INFO:teuthology.orchestra.run.smithi098.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42693.log.gz 2024-09-19T08:48:22.485 INFO:teuthology.orchestra.run.smithi098.stderr:gzip/var/log/ceph/ceph-client.admin.33386.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.40350.log 2024-09-19T08:48:22.485 INFO:teuthology.orchestra.run.smithi098.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33386.log.gz 2024-09-19T08:48:22.485 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38403.log 2024-09-19T08:48:22.485 INFO:teuthology.orchestra.run.smithi098.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42337.log.gz 2024-09-19T08:48:22.485 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38572.log 2024-09-19T08:48:22.485 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.40350.log: /var/log/ceph/ceph-client.admin.38403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53195.log 2024-09-19T08:48:22.486 INFO:teuthology.orchestra.run.smithi098.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40350.log.gz 2024-09-19T08:48:22.486 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.38572.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38403.log.gz 2024-09-19T08:48:22.486 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36161.log 2024-09-19T08:48:22.486 INFO:teuthology.orchestra.run.smithi098.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38572.log.gz 2024-09-19T08:48:22.486 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.53195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55902.log 2024-09-19T08:48:22.486 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36161.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53195.log.gz 2024-09-19T08:48:22.486 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54688.log 2024-09-19T08:48:22.486 INFO:teuthology.orchestra.run.smithi098.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36161.log.gz 2024-09-19T08:48:22.486 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.55902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44295.log 2024-09-19T08:48:22.486 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55902.log.gz 2024-09-19T08:48:22.486 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.54688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42188.log 2024-09-19T08:48:22.486 INFO:teuthology.orchestra.run.smithi098.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.54688.log.gz 2024-09-19T08:48:22.486 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46749.log 2024-09-19T08:48:22.486 INFO:teuthology.orchestra.run.smithi098.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44295.log.gz 2024-09-19T08:48:22.487 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43605.log 2024-09-19T08:48:22.487 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42188.log: /var/log/ceph/ceph-client.admin.46749.log: gzip 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42188.log.gz 2024-09-19T08:48:22.487 INFO:teuthology.orchestra.run.smithi098.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.39337.log 2024-09-19T08:48:22.487 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43605.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46749.log.gz 2024-09-19T08:48:22.487 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36411.log 2024-09-19T08:48:22.487 INFO:teuthology.orchestra.run.smithi098.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43605.log.gz 2024-09-19T08:48:22.487 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.39337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52261.log 2024-09-19T08:48:22.487 INFO:teuthology.orchestra.run.smithi098.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39337.log.gz 2024-09-19T08:48:22.487 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47879.log 2024-09-19T08:48:22.487 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-19T08:48:22.487 INFO:teuthology.orchestra.run.smithi098.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36411.log.gz 2024-09-19T08:48:22.487 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52261.log: /var/log/ceph/ceph-client.admin.47879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55928.log 2024-09-19T08:48:22.487 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52261.log.gz 2024-09-19T08:48:22.487 INFO:teuthology.orchestra.run.smithi098.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47879.log.gz 2024-09-19T08:48:22.488 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52877.log 2024-09-19T08:48:22.488 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.55928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40201.log 2024-09-19T08:48:22.488 INFO:teuthology.orchestra.run.smithi098.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55928.log.gz 2024-09-19T08:48:22.488 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51612.log 2024-09-19T08:48:22.488 INFO:teuthology.orchestra.run.smithi098.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52877.log.gz 2024-09-19T08:48:22.488 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44217.log 2024-09-19T08:48:22.488 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.40201.log: /var/log/ceph/ceph-client.admin.51612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34611.log 2024-09-19T08:48:22.488 INFO:teuthology.orchestra.run.smithi098.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40201.log.gz 2024-09-19T08:48:22.488 INFO:teuthology.orchestra.run.smithi098.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51612.log.gz 2024-09-19T08:48:22.488 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37767.log 2024-09-19T08:48:22.488 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55109.log 2024-09-19T08:48:22.488 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.34611.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44217.log.gz 2024-09-19T08:48:22.489 INFO:teuthology.orchestra.run.smithi098.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34611.log.gz 2024-09-19T08:48:22.489 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.37767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44347.log 2024-09-19T08:48:22.489 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45435.log 2024-09-19T08:48:22.489 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9%/var/log/ceph/ceph-client.admin.55109.log: -- replaced with /var/log/ceph/ceph-client.admin.37767.log.gz 2024-09-19T08:48:22.489 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44347.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.55109.log.gz 2024-09-19T08:48:22.489 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41715.log 2024-09-19T08:48:22.489 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39899.log 2024-09-19T08:48:22.489 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45435.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44347.log.gz 2024-09-19T08:48:22.489 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.41715.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45435.log.gz 2024-09-19T08:48:22.489 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37236.log 2024-09-19T08:48:22.489 INFO:teuthology.orchestra.run.smithi098.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41715.log.gz 2024-09-19T08:48:22.489 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40149.log 2024-09-19T08:48:22.489 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.39899.log: /var/log/ceph/ceph-client.admin.37236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38689.log 2024-09-19T08:48:22.490 INFO:teuthology.orchestra.run.smithi098.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39899.log.gz 2024-09-19T08:48:22.490 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37236.log.gz 2024-09-19T08:48:22.490 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.40149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44838.log 2024-09-19T08:48:22.490 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39257.log 2024-09-19T08:48:22.490 INFO:teuthology.orchestra.run.smithi098.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40149.log.gz/var/log/ceph/ceph-client.admin.38689.log: 2024-09-19T08:48:22.490 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44838.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38689.log.gz 2024-09-19T08:48:22.490 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46027.log 2024-09-19T08:48:22.490 INFO:teuthology.orchestra.run.smithi098.stderr:gzip 88.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.44838.log.gz 2024-09-19T08:48:22.490 INFO:teuthology.orchestra.run.smithi098.stderr: --verbose -- /var/log/ceph/ceph-client.admin.40253.log 2024-09-19T08:48:22.490 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.0.39257.log: /var/log/ceph/ceph-client.0.46027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34819.log 2024-09-19T08:48:22.490 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.40253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39099.log 2024-09-19T08:48:22.490 INFO:teuthology.orchestra.run.smithi098.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.0.39257.log.gz 2024-09-19T08:48:22.490 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40253.log.gz 2024-09-19T08:48:22.490 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.34819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47710.log 2024-09-19T08:48:22.491 INFO:teuthology.orchestra.run.smithi098.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34819.log.gz 2024-09-19T08:48:22.491 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34949.log 2024-09-19T08:48:22.491 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.39099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55878.log 2024-09-19T08:48:22.491 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47710.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39099.log.gz 2024-09-19T08:48:22.491 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.34949.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38202.log 2024-09-19T08:48:22.491 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47710.log.gz 2024-09-19T08:48:22.491 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.55878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52496.log 88.8% 2024-09-19T08:48:22.491 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34949.log.gz 2024-09-19T08:48:22.491 INFO:teuthology.orchestra.run.smithi098.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55878.log.gz 2024-09-19T08:48:22.491 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.38202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53413.log 2024-09-19T08:48:22.491 INFO:teuthology.orchestra.run.smithi098.stderr: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36016.log 2024-09-19T08:48:22.491 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52496.log: -- replaced with /var/log/ceph/ceph-client.admin.38202.log.gz 2024-09-19T08:48:22.491 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.53413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37605.log 2024-09-19T08:48:22.491 INFO:teuthology.orchestra.run.smithi098.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52496.log.gz 2024-09-19T08:48:22.492 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53378.log 2024-09-19T08:48:22.492 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53413.log.gz 2024-09-19T08:48:22.492 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36016.log: /var/log/ceph/ceph-client.admin.37605.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36016.log.gz 2024-09-19T08:48:22.492 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36442.log 2024-09-19T08:48:22.492 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.53378.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.37605.log.gz 2024-09-19T08:48:22.492 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52747.log 2024-09-19T08:48:22.492 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53378.log.gz 2024-09-19T08:48:22.492 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47241.log 2024-09-19T08:48:22.492 INFO:teuthology.orchestra.run.smithi098.stderr: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39557.log 2024-09-19T08:48:22.492 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52747.log: -- replaced with /var/log/ceph/ceph-client.admin.36442.log.gz 2024-09-19T08:48:22.492 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52235.log 2024-09-19T08:48:22.492 INFO:teuthology.orchestra.run.smithi098.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52747.log.gz 2024-09-19T08:48:22.492 INFO:teuthology.orchestra.run.smithi098.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47241.log.gz 2024-09-19T08:48:22.492 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.39557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51809.log 2024-09-19T08:48:22.493 INFO:teuthology.orchestra.run.smithi098.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39557.log.gz 2024-09-19T08:48:22.493 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46108.log 2024-09-19T08:48:22.493 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51809.log: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50303.log 2024-09-19T08:48:22.493 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52235.log.gz 2024-09-19T08:48:22.493 INFO:teuthology.orchestra.run.smithi098.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51809.log.gz 2024-09-19T08:48:22.493 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41517.log 2024-09-19T08:48:22.493 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50303.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46108.log.gz 2024-09-19T08:48:22.493 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35420.log 2024-09-19T08:48:22.493 INFO:teuthology.orchestra.run.smithi098.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.50303.log.gz 2024-09-19T08:48:22.493 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.41517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49775.log 2024-09-19T08:48:22.493 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40995.log 2024-09-19T08:48:22.493 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.35420.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41517.log.gz 2024-09-19T08:48:22.493 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49775.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35420.log.gz 2024-09-19T08:48:22.493 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-19T08:48:22.494 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52604.log 2024-09-19T08:48:22.494 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49775.log.gz/var/log/ceph/ceph-client.admin.40995.log: 2024-09-19T08:48:22.494 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54502.log 2024-09-19T08:48:22.494 INFO:teuthology.orchestra.run.smithi098.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40995.log.gz 2024-09-19T08:48:22.494 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52604.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52604.log.gz 2024-09-19T08:48:22.494 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51269.log 2024-09-19T08:48:22.494 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.54502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37318.log 2024-09-19T08:48:22.494 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51269.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54502.log.gz 2024-09-19T08:48:22.494 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55005.log 2024-09-19T08:48:22.494 INFO:teuthology.orchestra.run.smithi098.stderr: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48882.log 2024-09-19T08:48:22.494 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51269.log.gz 2024-09-19T08:48:22.494 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.37318.log: /var/log/ceph/ceph-client.admin.55005.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37318.log.gz 2024-09-19T08:48:22.494 INFO:teuthology.orchestra.run.smithi098.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55005.log.gz 2024-09-19T08:48:22.495 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44072.log 2024-09-19T08:48:22.495 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37953.log 2024-09-19T08:48:22.495 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44072.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48882.log.gz 2024-09-19T08:48:22.495 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51927.log 2024-09-19T08:48:22.495 INFO:teuthology.orchestra.run.smithi098.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34793.log 2024-09-19T08:48:22.495 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44072.log.gz 2024-09-19T08:48:22.495 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.37953.log: /var/log/ceph/ceph-client.admin.51927.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37953.log.gz 2024-09-19T08:48:22.495 INFO:teuthology.orchestra.run.smithi098.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51927.log.gz 2024-09-19T08:48:22.495 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38889.log 2024-09-19T08:48:22.495 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.34793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55031.log 2024-09-19T08:48:22.495 INFO:teuthology.orchestra.run.smithi098.stderr: 87.4%/var/log/ceph/ceph-client.admin.38889.log: -- replaced with /var/log/ceph/ceph-client.admin.34793.log.gz 2024-09-19T08:48:22.495 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34531.log 2024-09-19T08:48:22.495 INFO:teuthology.orchestra.run.smithi098.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38889.log.gz 2024-09-19T08:48:22.495 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.55031.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55031.log.gz 2024-09-19T08:48:22.533 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37793.log 2024-09-19T08:48:22.533 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.34531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44807.log 2024-09-19T08:48:22.533 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.37793.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34531.log.gz 2024-09-19T08:48:22.534 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41649.log 2024-09-19T08:48:22.534 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51697.log 2024-09-19T08:48:22.534 INFO:teuthology.orchestra.run.smithi098.stderr: 89.0%/var/log/ceph/ceph-client.admin.44807.log: -- replaced with /var/log/ceph/ceph-client.admin.37793.log.gz 2024-09-19T08:48:22.534 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.41649.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44807.log.gz 2024-09-19T08:48:22.534 INFO:teuthology.orchestra.run.smithi098.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41649.log.gz 2024-09-19T08:48:22.534 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41684.log 2024-09-19T08:48:22.534 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35705.log 2024-09-19T08:48:22.534 INFO:teuthology.orchestra.run.smithi098.stderr: 87.8%/var/log/ceph/ceph-client.admin.41684.log: -- replaced with /var/log/ceph/ceph-client.admin.51697.log.gz 2024-09-19T08:48:22.534 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47059.log 2024-09-19T08:48:22.534 INFO:teuthology.orchestra.run.smithi098.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41684.log.gz 2024-09-19T08:48:22.534 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.35705.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35705.log.gz 2024-09-19T08:48:22.534 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39500.log 2024-09-19T08:48:22.534 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49695.log 2024-09-19T08:48:22.534 INFO:teuthology.orchestra.run.smithi098.stderr: 88.7%/var/log/ceph/ceph-client.admin.39500.log: -- replaced with /var/log/ceph/ceph-client.admin.47059.log.gz 2024-09-19T08:48:22.535 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41744.log 2024-09-19T08:48:22.535 INFO:teuthology.orchestra.run.smithi098.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39500.log.gz 2024-09-19T08:48:22.535 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.0.49695.log: 90.5% -- replaced with /var/log/ceph/ceph-client.0.49695.log.gz 2024-09-19T08:48:22.535 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41283.log 2024-09-19T08:48:22.535 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54803.log 2024-09-19T08:48:22.535 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.41744.log: /var/log/ceph/ceph-client.0.41283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48835.log 2024-09-19T08:48:22.535 INFO:teuthology.orchestra.run.smithi098.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41744.log.gz 2024-09-19T08:48:22.535 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.54803.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54803.log.gz 2024-09-19T08:48:22.535 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47189.log 2024-09-19T08:48:22.535 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44423.log 2024-09-19T08:48:22.535 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48835.log: /var/log/ceph/ceph-client.admin.47189.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47189.log.gz 2024-09-19T08:48:22.535 INFO:teuthology.orchestra.run.smithi098.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48835.log.gz 2024-09-19T08:48:22.535 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41573.log 2024-09-19T08:48:22.536 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45023.log 2024-09-19T08:48:22.536 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44423.log: /var/log/ceph/ceph-client.admin.41573.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44423.log.gz 2024-09-19T08:48:22.536 INFO:teuthology.orchestra.run.smithi098.stderr: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47470.log 2024-09-19T08:48:22.536 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41573.log.gz 2024-09-19T08:48:22.536 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45023.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45023.log.gz 2024-09-19T08:48:22.536 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53917.log 2024-09-19T08:48:22.536 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39073.log 2024-09-19T08:48:22.536 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47470.log: /var/log/ceph/ceph-client.admin.53917.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47470.log.gz 2024-09-19T08:48:22.536 INFO:teuthology.orchestra.run.smithi098.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53917.log.gz 2024-09-19T08:48:22.536 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42563.log 2024-09-19T08:48:22.536 INFO:teuthology.orchestra.run.smithi098.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.0.41283.log.gz 2024-09-19T08:48:22.536 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43579.log 2024-09-19T08:48:22.536 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.39073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34767.log 2024-09-19T08:48:22.536 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42563.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39073.log.gz 2024-09-19T08:48:22.537 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43579.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42563.log.gz 2024-09-19T08:48:22.537 INFO:teuthology.orchestra.run.smithi098.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43579.log.gz 2024-09-19T08:48:22.537 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50468.log 2024-09-19T08:48:22.537 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.34767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37288.log 2024-09-19T08:48:22.537 INFO:teuthology.orchestra.run.smithi098.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34767.log.gz/var/log/ceph/ceph-client.admin.50468.log: 2024-09-19T08:48:22.537 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40532.log 2024-09-19T08:48:22.537 INFO:teuthology.orchestra.run.smithi098.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50468.log.gz 2024-09-19T08:48:22.537 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.37288.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37288.log.gz 2024-09-19T08:48:22.537 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52799.log 2024-09-19T08:48:22.537 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.40532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41073.log 2024-09-19T08:48:22.537 INFO:teuthology.orchestra.run.smithi098.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40532.log.gz/var/log/ceph/ceph-client.admin.52799.log: 2024-09-19T08:48:22.537 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42364.log 2024-09-19T08:48:22.537 INFO:teuthology.orchestra.run.smithi098.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52799.log.gz 2024-09-19T08:48:22.537 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.41073.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41073.log.gz 2024-09-19T08:48:22.538 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50128.log 2024-09-19T08:48:22.538 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41177.log 2024-09-19T08:48:22.538 INFO:teuthology.orchestra.run.smithi098.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42364.log.gz/var/log/ceph/ceph-client.admin.50128.log: 2024-09-19T08:48:22.538 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43526.log 2024-09-19T08:48:22.538 INFO:teuthology.orchestra.run.smithi098.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.50128.log.gz 2024-09-19T08:48:22.538 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.41177.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.41177.log.gz 2024-09-19T08:48:22.538 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44479.log 2024-09-19T08:48:22.538 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35444.log 2024-09-19T08:48:22.538 INFO:teuthology.orchestra.run.smithi098.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43526.log.gz/var/log/ceph/ceph-client.0.44479.log: 2024-09-19T08:48:22.538 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41125.log 2024-09-19T08:48:22.538 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.35444.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35444.log.gz 2024-09-19T08:48:22.538 INFO:teuthology.orchestra.run.smithi098.stderr:gzip 90.5% -- replaced with /var/log/ceph/ceph-client.0.44479.log.gz -5 2024-09-19T08:48:22.538 INFO:teuthology.orchestra.run.smithi098.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50920.log 2024-09-19T08:48:22.538 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.41125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36247.log 2024-09-19T08:48:22.539 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50920.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41125.log.gz 2024-09-19T08:48:22.539 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44321.log 2024-09-19T08:48:22.539 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48757.log 2024-09-19T08:48:22.539 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50920.log.gz 2024-09-19T08:48:22.539 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36247.log: /var/log/ceph/ceph-client.admin.44321.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36247.log.gz 2024-09-19T08:48:22.539 INFO:teuthology.orchestra.run.smithi098.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44321.log.gz 2024-09-19T08:48:22.539 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41547.log 2024-09-19T08:48:22.539 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43088.log 2024-09-19T08:48:22.539 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.41547.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48757.log.gz 2024-09-19T08:48:22.539 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50757.log 2024-09-19T08:48:22.539 INFO:teuthology.orchestra.run.smithi098.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41547.log.gz 2024-09-19T08:48:22.539 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43088.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43088.log.gz 2024-09-19T08:48:22.539 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49801.log 2024-09-19T08:48:22.539 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40175.log 2024-09-19T08:48:22.540 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49801.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50757.log.gz 2024-09-19T08:48:22.540 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53752.log 2024-09-19T08:48:22.540 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44161.log 2024-09-19T08:48:22.540 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9%/var/log/ceph/ceph-client.admin.40175.log: -- replaced with /var/log/ceph/ceph-client.admin.49801.log.gz 2024-09-19T08:48:22.540 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.53752.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40175.log.gz 2024-09-19T08:48:22.540 INFO:teuthology.orchestra.run.smithi098.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53752.log.gz 2024-09-19T08:48:22.540 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52665.log 2024-09-19T08:48:22.540 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55773.log 2024-09-19T08:48:22.540 INFO:teuthology.orchestra.run.smithi098.stderr: 88.2%/var/log/ceph/ceph-client.admin.52665.log: -- replaced with /var/log/ceph/ceph-client.admin.44161.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49827.log 2024-09-19T08:48:22.540 INFO:teuthology.orchestra.run.smithi098.stderr: 2024-09-19T08:48:22.540 INFO:teuthology.orchestra.run.smithi098.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52665.log.gz 2024-09-19T08:48:22.540 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.55773.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55773.log.gz 2024-09-19T08:48:22.540 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47366.log 2024-09-19T08:48:22.540 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44971.log 2024-09-19T08:48:22.541 INFO:teuthology.orchestra.run.smithi098.stderr: 89.0%/var/log/ceph/ceph-client.admin.47366.log: -- replaced with /var/log/ceph/ceph-client.admin.49827.log.gzgzip 2024-09-19T08:48:22.541 INFO:teuthology.orchestra.run.smithi098.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45739.log 2024-09-19T08:48:22.541 INFO:teuthology.orchestra.run.smithi098.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47366.log.gz 2024-09-19T08:48:22.541 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44971.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44971.log.gz 2024-09-19T08:48:22.541 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43115.log 2024-09-19T08:48:22.541 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52339.log 2024-09-19T08:48:22.541 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43115.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48624.log 2024-09-19T08:48:22.541 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45739.log.gz 2024-09-19T08:48:22.541 INFO:teuthology.orchestra.run.smithi098.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43115.log.gz 2024-09-19T08:48:22.541 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52339.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52339.log.gz 2024-09-19T08:48:22.541 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36813.log 2024-09-19T08:48:22.541 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53169.log 2024-09-19T08:48:22.541 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37396.log 2024-09-19T08:48:22.541 INFO:teuthology.orchestra.run.smithi098.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48624.log.gz 2024-09-19T08:48:22.542 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.53169.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36813.log.gz 2024-09-19T08:48:22.542 INFO:teuthology.orchestra.run.smithi098.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.53169.log.gz 2024-09-19T08:48:22.542 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50497.log 2024-09-19T08:48:22.542 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.37396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35210.log 2024-09-19T08:48:22.542 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50497.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37396.log.gz 2024-09-19T08:48:22.542 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45843.log 2024-09-19T08:48:22.542 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42823.log 2024-09-19T08:48:22.542 INFO:teuthology.orchestra.run.smithi098.stderr: 88.5%/var/log/ceph/ceph-client.admin.35210.log: -- replaced with /var/log/ceph/ceph-client.admin.50497.log.gz 2024-09-19T08:48:22.542 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45843.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35210.log.gz 2024-09-19T08:48:22.542 INFO:teuthology.orchestra.run.smithi098.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45843.log.gz 2024-09-19T08:48:22.542 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40454.log 2024-09-19T08:48:22.542 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53473.log 2024-09-19T08:48:22.542 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.40454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45791.log 2024-09-19T08:48:22.542 INFO:teuthology.orchestra.run.smithi098.stderr: 91.2% 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40454.log.gz 2024-09-19T08:48:22.543 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42823.log.gz 2024-09-19T08:48:22.543 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.53473.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53473.log.gz 2024-09-19T08:48:22.543 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29136.log 2024-09-19T08:48:22.543 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51347.log 2024-09-19T08:48:22.543 INFO:teuthology.orchestra.run.smithi098.stderr: 88.8%/var/log/ceph/ceph-client.admin.29136.log: -- replaced with /var/log/ceph/ceph-client.admin.45791.log.gz 2024-09-19T08:48:22.543 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42025.log 2024-09-19T08:48:22.543 INFO:teuthology.orchestra.run.smithi098.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.29136.log.gz 2024-09-19T08:48:22.543 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51347.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51347.log.gz 2024-09-19T08:48:22.543 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45553.log 2024-09-19T08:48:22.543 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43729.log 2024-09-19T08:48:22.543 INFO:teuthology.orchestra.run.smithi098.stderr: 88.8%/var/log/ceph/ceph-client.admin.45553.log: -- replaced with /var/log/ceph/ceph-client.admin.42025.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43676.log 2024-09-19T08:48:22.543 INFO:teuthology.orchestra.run.smithi098.stderr: 2024-09-19T08:48:22.543 INFO:teuthology.orchestra.run.smithi098.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45553.log.gz 2024-09-19T08:48:22.543 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43729.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43729.log.gz 2024-09-19T08:48:22.544 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40715.log 2024-09-19T08:48:22.544 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38228.log 2024-09-19T08:48:22.544 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.40715.log: 93.3% -- replaced with /var/log/ceph/ceph-client.0.46027.log.gz 2024-09-19T08:48:22.544 INFO:teuthology.orchestra.run.smithi098.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43676.log.gz 2024-09-19T08:48:22.544 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54950.log 2024-09-19T08:48:22.544 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40715.log.gz 2024-09-19T08:48:22.544 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.38228.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38228.log.gz 2024-09-19T08:48:22.544 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52957.log 2024-09-19T08:48:22.544 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.54950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44016.log 2024-09-19T08:48:22.544 INFO:teuthology.orchestra.run.smithi098.stderr: 89.4%/var/log/ceph/ceph-client.admin.52957.log: -- replaced with /var/log/ceph/ceph-client.admin.54950.log.gz 2024-09-19T08:48:22.544 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51098.log 2024-09-19T08:48:22.544 INFO:teuthology.orchestra.run.smithi098.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52957.log.gz 2024-09-19T08:48:22.545 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40809.log 2024-09-19T08:48:22.545 INFO:teuthology.orchestra.run.smithi098.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44016.log.gz 2024-09-19T08:48:22.545 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52851.log 2024-09-19T08:48:22.545 INFO:teuthology.orchestra.run.smithi098.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51098.log.gz 2024-09-19T08:48:22.545 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.40809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43703.log 2024-09-19T08:48:22.545 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52851.log: gzip 89.1% -5 -- replaced with /var/log/ceph/ceph-client.admin.40809.log.gz --verbose 2024-09-19T08:48:22.545 INFO:teuthology.orchestra.run.smithi098.stderr: -- /var/log/ceph/ceph-client.admin.46723.log 2024-09-19T08:48:22.545 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43703.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52851.log.gz 2024-09-19T08:48:22.545 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39625.log 2024-09-19T08:48:22.545 INFO:teuthology.orchestra.run.smithi098.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43703.log.gz 2024-09-19T08:48:22.545 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48934.log 2024-09-19T08:48:22.545 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46723.log: /var/log/ceph/ceph-client.admin.39625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55641.log 2024-09-19T08:48:22.545 INFO:teuthology.orchestra.run.smithi098.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46723.log.gz 2024-09-19T08:48:22.545 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6%/var/log/ceph/ceph-client.admin.48934.log: -- replaced with /var/log/ceph/ceph-client.admin.39625.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38941.log 2024-09-19T08:48:22.546 INFO:teuthology.orchestra.run.smithi098.stderr: 2024-09-19T08:48:22.546 INFO:teuthology.orchestra.run.smithi098.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48934.log.gz 2024-09-19T08:48:22.546 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.55641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37262.log 2024-09-19T08:48:22.546 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51399.log 2024-09-19T08:48:22.546 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.38941.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55641.log.gz 2024-09-19T08:48:22.546 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.37262.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38941.log.gz 2024-09-19T08:48:22.546 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40402.log 2024-09-19T08:48:22.546 INFO:teuthology.orchestra.run.smithi098.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37262.log.gz 2024-09-19T08:48:22.546 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38967.log 2024-09-19T08:48:22.546 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51399.log: /var/log/ceph/ceph-client.admin.40402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37553.log 2024-09-19T08:48:22.546 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.38967.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40402.log.gz 2024-09-19T08:48:22.546 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38254.log 2024-09-19T08:48:22.546 INFO:teuthology.orchestra.run.smithi098.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51399.log.gz 2024-09-19T08:48:22.546 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.37553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35003.log 2024-09-19T08:48:22.547 INFO:teuthology.orchestra.run.smithi098.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38967.log.gz 2024-09-19T08:48:22.547 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.38254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47682.log 2024-09-19T08:48:22.547 INFO:teuthology.orchestra.run.smithi098.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37553.log.gz 2024-09-19T08:48:22.547 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.35003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50887.log 2024-09-19T08:48:22.547 INFO:teuthology.orchestra.run.smithi098.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38254.log.gz 2024-09-19T08:48:22.547 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47682.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47654.log 2024-09-19T08:48:22.547 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35003.log.gz 2024-09-19T08:48:22.547 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50887.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47682.log.gz 2024-09-19T08:48:22.547 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43035.log 2024-09-19T08:48:22.547 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- 88.3% /var/log/ceph/ceph-client.admin.37475.log 2024-09-19T08:48:22.547 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50887.log.gz 2024-09-19T08:48:22.547 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47654.log: /var/log/ceph/ceph-client.admin.43035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50028.log 2024-09-19T08:48:22.547 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47654.log.gz 2024-09-19T08:48:22.547 INFO:teuthology.orchestra.run.smithi098.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43035.log.gz 2024-09-19T08:48:22.548 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52773.log 2024-09-19T08:48:22.548 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.37475.log: /var/log/ceph/ceph-client.admin.50028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50809.log 2024-09-19T08:48:22.548 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52773.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50028.log.gz 2024-09-19T08:48:22.548 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37501.log 2024-09-19T08:48:22.548 INFO:teuthology.orchestra.run.smithi098.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37475.log.gz 2024-09-19T08:48:22.548 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50809.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52773.log.gz 2024-09-19T08:48:22.548 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51532.log 2024-09-19T08:48:22.548 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50809.log.gz 2024-09-19T08:48:22.548 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34988.log 2024-09-19T08:48:22.548 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.37501.log: /var/log/ceph/ceph-client.0.51532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34663.log 2024-09-19T08:48:22.548 INFO:teuthology.orchestra.run.smithi098.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37501.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36936.log 2024-09-19T08:48:22.548 INFO:teuthology.orchestra.run.smithi098.stderr: 2024-09-19T08:48:22.548 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.34988.log: /var/log/ceph/ceph-client.admin.34663.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34988.log.gz 2024-09-19T08:48:22.549 INFO:teuthology.orchestra.run.smithi098.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34663.log.gz 2024-09-19T08:48:22.549 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-19T08:48:22.549 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44611.log 2024-09-19T08:48:22.549 INFO:teuthology.orchestra.run.smithi098.stderr: 89.7%/var/log/ceph/ceph-osd.3.log: -- replaced with /var/log/ceph/ceph-client.admin.36936.log.gz 2024-09-19T08:48:22.549 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41021.log 2024-09-19T08:48:22.549 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44611.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44611.log.gz 2024-09-19T08:48:22.549 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46356.log 2024-09-19T08:48:22.549 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40861.log 2024-09-19T08:48:22.549 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.41021.log: /var/log/ceph/ceph-client.admin.46356.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41021.log.gz 2024-09-19T08:48:22.549 INFO:teuthology.orchestra.run.smithi098.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46356.log.gz 2024-09-19T08:48:22.549 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41874.log 2024-09-19T08:48:22.549 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52365.log 2024-09-19T08:48:22.549 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.40861.log: /var/log/ceph/ceph-client.admin.41874.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40861.log.gz 2024-09-19T08:48:22.549 INFO:teuthology.orchestra.run.smithi098.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41874.log.gz 2024-09-19T08:48:22.550 INFO:teuthology.orchestra.run.smithi098.stderr:gzip 92.5% -- replaced with /var/log/ceph/ceph-client.0.51532.log.gz -5 2024-09-19T08:48:22.550 INFO:teuthology.orchestra.run.smithi098.stderr: --verbose -- /var/log/ceph/ceph-client.admin.51243.log 2024-09-19T08:48:22.550 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47536.log 2024-09-19T08:48:22.550 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40279.log 2024-09-19T08:48:22.550 INFO:teuthology.orchestra.run.smithi098.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52365.log.gz/var/log/ceph/ceph-client.admin.47536.log: 2024-09-19T08:48:22.550 INFO:teuthology.orchestra.run.smithi098.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51243.log.gz 2024-09-19T08:48:22.550 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47536.log.gz 2024-09-19T08:48:22.550 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38915.log 2024-09-19T08:48:22.550 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34689.log 2024-09-19T08:48:22.550 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.40279.log: /var/log/ceph/ceph-client.admin.38915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34999.log 2024-09-19T08:48:22.563 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.34689.log: 88.8% 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40279.log.gz 2024-09-19T08:48:22.563 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38915.log.gz 2024-09-19T08:48:22.563 INFO:teuthology.orchestra.run.smithi098.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34689.log.gz 2024-09-19T08:48:22.563 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41796.log 2024-09-19T08:48:22.563 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49953.log 2024-09-19T08:48:22.563 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.34999.log: /var/log/ceph/ceph-client.admin.41796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36631.log 2024-09-19T08:48:22.563 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49953.log: 88.2% 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34999.log.gz 2024-09-19T08:48:22.564 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41796.log.gz 2024-09-19T08:48:22.564 INFO:teuthology.orchestra.run.smithi098.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49953.log.gz 2024-09-19T08:48:22.564 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47957.log 2024-09-19T08:48:22.564 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47827.log 2024-09-19T08:48:22.564 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36631.log: /var/log/ceph/ceph-client.admin.47957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48059.log 2024-09-19T08:48:22.564 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47827.log: 87.9% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36631.log.gz 2024-09-19T08:48:22.564 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47957.log.gz 2024-09-19T08:48:22.564 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47827.log.gz 2024-09-19T08:48:22.564 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49038.log 2024-09-19T08:48:22.564 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45490.log 2024-09-19T08:48:22.564 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48059.log: /var/log/ceph/ceph-client.admin.49038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48504.log 2024-09-19T08:48:22.564 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45490.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49038.log.gz 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48059.log.gz 2024-09-19T08:48:22.564 INFO:teuthology.orchestra.run.smithi098.stderr: 2024-09-19T08:48:22.565 INFO:teuthology.orchestra.run.smithi098.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45490.log.gz 2024-09-19T08:48:22.565 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55615.log 2024-09-19T08:48:22.565 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47853.log 2024-09-19T08:48:22.565 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48504.log: /var/log/ceph/ceph-client.admin.55615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52086.log 2024-09-19T08:48:22.565 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47853.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48504.log.gz 2024-09-19T08:48:22.565 INFO:teuthology.orchestra.run.smithi098.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55615.log.gz 2024-09-19T08:48:22.565 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53831.log 2024-09-19T08:48:22.565 INFO:teuthology.orchestra.run.smithi098.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47853.log.gz 2024-09-19T08:48:22.565 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54634.log 2024-09-19T08:48:22.565 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52086.log: /var/log/ceph/ceph-client.admin.53831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50437.log 2024-09-19T08:48:22.565 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.0.54634.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53831.log.gz 2024-09-19T08:48:22.565 INFO:teuthology.orchestra.run.smithi098.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52086.log.gz 2024-09-19T08:48:22.565 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51321.log 2024-09-19T08:48:22.566 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41973.log 2024-09-19T08:48:22.566 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50437.log: /var/log/ceph/ceph-client.admin.51321.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50437.log.gz 2024-09-19T08:48:22.566 INFO:teuthology.orchestra.run.smithi098.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51321.log.gz 2024-09-19T08:48:22.566 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40663.log 2024-09-19T08:48:22.566 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53009.log 2024-09-19T08:48:22.566 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.41973.log: /var/log/ceph/ceph-client.admin.40663.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41973.log.gz 2024-09-19T08:48:22.566 INFO:teuthology.orchestra.run.smithi098.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40663.log.gz 2024-09-19T08:48:22.566 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55161.log 2024-09-19T08:48:22.566 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44559.log 2024-09-19T08:48:22.566 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.53009.log: /var/log/ceph/ceph-client.admin.55161.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55161.log.gz 2024-09-19T08:48:22.566 INFO:teuthology.orchestra.run.smithi098.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.53009.log.gz 2024-09-19T08:48:22.566 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38124.log 2024-09-19T08:48:22.566 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43898.log 2024-09-19T08:48:22.566 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44559.log: /var/log/ceph/ceph-client.admin.38124.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44559.log.gz 2024-09-19T08:48:22.567 INFO:teuthology.orchestra.run.smithi098.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.0.54634.log.gz 2024-09-19T08:48:22.567 INFO:teuthology.orchestra.run.smithi098.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38124.log.gz 2024-09-19T08:48:22.567 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53444.log 2024-09-19T08:48:22.567 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43441.log 2024-09-19T08:48:22.567 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43898.log: /var/log/ceph/ceph-client.admin.53444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38006.log 2024-09-19T08:48:22.567 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43441.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43898.log.gz 2024-09-19T08:48:22.567 INFO:teuthology.orchestra.run.smithi098.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53444.log.gz 2024-09-19T08:48:22.567 INFO:teuthology.orchestra.run.smithi098.stderr: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51723.log 2024-09-19T08:48:22.567 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43441.log.gz 2024-09-19T08:48:22.567 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44637.log 2024-09-19T08:48:22.567 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.38006.log: /var/log/ceph/ceph-client.admin.51723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53115.log 2024-09-19T08:48:22.567 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44637.log: 88.4% 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38006.log.gz 2024-09-19T08:48:22.567 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51723.log.gz 2024-09-19T08:48:22.567 INFO:teuthology.orchestra.run.smithi098.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.44637.log.gz 2024-09-19T08:48:22.568 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43298.log 2024-09-19T08:48:22.568 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38807.log 2024-09-19T08:48:22.568 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.0.53115.log: /var/log/ceph/ceph-client.admin.43298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54240.log 2024-09-19T08:48:22.568 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.38807.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43298.log.gz 2024-09-19T08:48:22.568 INFO:teuthology.orchestra.run.smithi098.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38807.log.gz 2024-09-19T08:48:22.568 INFO:teuthology.orchestra.run.smithi098.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-19T08:48:22.568 INFO:teuthology.orchestra.run.smithi098.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.0.53115.log.gz 2024-09-19T08:48:22.568 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48199.log 2024-09-19T08:48:22.568 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.54240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55589.log 2024-09-19T08:48:22.568 INFO:teuthology.orchestra.run.smithi098.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54240.log.gz 2024-09-19T08:48:22.568 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46489.log 2024-09-19T08:48:22.568 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.55589.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48199.log.gz 2024-09-19T08:48:22.569 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45661.log 2024-09-19T08:48:22.569 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46489.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55589.log.gz 2024-09-19T08:48:22.579 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37067.log 2024-09-19T08:48:22.579 INFO:teuthology.orchestra.run.smithi098.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46489.log.gz 2024-09-19T08:48:22.579 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49749.log 2024-09-19T08:48:22.579 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.37067.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45661.log.gz 2024-09-19T08:48:22.579 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45278.log 2024-09-19T08:48:22.579 INFO:teuthology.orchestra.run.smithi098.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37067.log.gz 2024-09-19T08:48:22.579 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46278.log 2024-09-19T08:48:22.579 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45278.log: gzip -5 89.3% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.49749.log.gz 2024-09-19T08:48:22.579 INFO:teuthology.orchestra.run.smithi098.stderr: /var/log/ceph/ceph-client.admin.49878.log 2024-09-19T08:48:22.579 INFO:teuthology.orchestra.run.smithi098.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.45278.log.gz 2024-09-19T08:48:22.579 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41491.log 2024-09-19T08:48:22.579 INFO:teuthology.orchestra.run.smithi098.stderr:gzip 88.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.46278.log.gz 2024-09-19T08:48:22.579 INFO:teuthology.orchestra.run.smithi098.stderr: --verbose -- /var/log/ceph/ceph-client.admin.54450.log 2024-09-19T08:48:22.580 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49878.log: /var/log/ceph/ceph-client.admin.41491.log: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47392.log 2024-09-19T08:48:22.580 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49878.log.gz 2024-09-19T08:48:22.580 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.54450.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41491.log.gzgzip 2024-09-19T08:48:22.580 INFO:teuthology.orchestra.run.smithi098.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42155.log 2024-09-19T08:48:22.580 INFO:teuthology.orchestra.run.smithi098.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54450.log.gz 2024-09-19T08:48:22.580 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35391.log 2024-09-19T08:48:22.580 INFO:teuthology.orchestra.run.smithi098.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47392.log.gz 2024-09-19T08:48:22.580 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40428.log 2024-09-19T08:48:22.580 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42155.log: /var/log/ceph/ceph-client.admin.35391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46330.log 2024-09-19T08:48:22.580 INFO:teuthology.orchestra.run.smithi098.stderr: 88.8% 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35391.log.gz 2024-09-19T08:48:22.580 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42155.log.gz 2024-09-19T08:48:22.580 INFO:teuthology.orchestra.run.smithi098.stderr:gzip/var/log/ceph/ceph-client.admin.40428.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.48731.log 2024-09-19T08:48:22.580 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46226.log 2024-09-19T08:48:22.580 INFO:teuthology.orchestra.run.smithi098.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40428.log.gz 2024-09-19T08:48:22.581 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48731.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46330.log.gz 2024-09-19T08:48:22.581 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43467.log 2024-09-19T08:48:22.581 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46226.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-19T08:48:22.581 INFO:teuthology.orchestra.run.smithi098.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48731.log.gz 2024-09-19T08:48:22.581 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40969.log 2024-09-19T08:48:22.581 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46226.log.gz 2024-09-19T08:48:22.581 INFO:teuthology.orchestra.run.smithi098.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43467.log.gz 2024-09-19T08:48:22.581 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50103.log 2024-09-19T08:48:22.581 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.40969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48420.log 2024-09-19T08:48:22.581 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50103.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40969.log.gz 2024-09-19T08:48:22.581 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40597.log 2024-09-19T08:48:22.581 INFO:teuthology.orchestra.run.smithi098.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.50103.log.gz 2024-09-19T08:48:22.581 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51586.log 2024-09-19T08:48:22.581 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48420.log: /var/log/ceph/ceph-client.admin.40597.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48420.log.gz 2024-09-19T08:48:22.582 INFO:teuthology.orchestra.run.smithi098.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40597.log.gz 2024-09-19T08:48:22.582 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49012.log 2024-09-19T08:48:22.582 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34559.log 2024-09-19T08:48:22.582 INFO:teuthology.orchestra.run.smithi098.stderr: 89.4%/var/log/ceph/ceph-client.admin.49012.log: -- replaced with /var/log/ceph/ceph-client.admin.51586.log.gz 2024-09-19T08:48:22.582 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34585.log 2024-09-19T08:48:22.582 INFO:teuthology.orchestra.run.smithi098.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49012.log.gz 2024-09-19T08:48:22.582 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.34559.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34559.log.gz 2024-09-19T08:48:22.582 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47163.log 2024-09-19T08:48:22.582 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42507.log 2024-09-19T08:48:22.582 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.34585.log: /var/log/ceph/ceph-client.admin.47163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37579.log 2024-09-19T08:48:22.582 INFO:teuthology.orchestra.run.smithi098.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34585.log.gz 2024-09-19T08:48:22.582 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6%/var/log/ceph/ceph-client.admin.42507.log: -- replaced with /var/log/ceph/ceph-client.admin.47163.log.gz 2024-09-19T08:48:22.582 INFO:teuthology.orchestra.run.smithi098.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42507.log.gz 2024-09-19T08:48:22.582 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49537.log 2024-09-19T08:48:22.583 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.37579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45224.log 2024-09-19T08:48:22.583 INFO:teuthology.orchestra.run.smithi098.stderr: 88.4%/var/log/ceph/ceph-client.admin.49537.log: -- replaced with /var/log/ceph/ceph-client.admin.37579.log.gz 2024-09-19T08:48:22.583 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37711.log 2024-09-19T08:48:22.583 INFO:teuthology.orchestra.run.smithi098.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49537.log.gz 2024-09-19T08:48:22.583 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45224.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45224.log.gz 2024-09-19T08:48:22.583 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37819.log 2024-09-19T08:48:22.583 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50378.log 2024-09-19T08:48:22.583 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.0.37711.log: /var/log/ceph/ceph-client.admin.37819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44191.log 2024-09-19T08:48:22.583 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50378.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37819.log.gz 2024-09-19T08:48:22.583 INFO:teuthology.orchestra.run.smithi098.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50378.log.gz 2024-09-19T08:48:22.590 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50203.log 2024-09-19T08:48:22.590 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50253.log 2024-09-19T08:48:22.590 INFO:teuthology.orchestra.run.smithi098.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44191.log.gz 2024-09-19T08:48:22.590 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50203.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.50203.log.gz 2024-09-19T08:48:22.590 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34923.log 2024-09-19T08:48:22.590 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53551.log 2024-09-19T08:48:22.590 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50253.log: /var/log/ceph/ceph-client.admin.34923.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50253.log.gz 2024-09-19T08:48:22.591 INFO:teuthology.orchestra.run.smithi098.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39443.log 2024-09-19T08:48:22.591 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34923.log.gz 2024-09-19T08:48:22.591 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.53551.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53551.log.gz 2024-09-19T08:48:22.591 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37145.log 2024-09-19T08:48:22.591 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45895.log 2024-09-19T08:48:22.591 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.39443.log: /var/log/ceph/ceph-client.admin.37145.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39443.log.gz 2024-09-19T08:48:22.591 INFO:teuthology.orchestra.run.smithi098.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37145.log.gz 2024-09-19T08:48:22.591 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34637.log 2024-09-19T08:48:22.591 INFO:teuthology.orchestra.run.smithi098.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.0.37711.log.gz 2024-09-19T08:48:22.591 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45895.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45895.log.gz 2024-09-19T08:48:22.591 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39747.log 2024-09-19T08:48:22.591 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.34637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51956.log 2024-09-19T08:48:22.591 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.39747.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34637.log.gz 2024-09-19T08:48:22.591 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46406.log 2024-09-19T08:48:22.592 INFO:teuthology.orchestra.run.smithi098.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39747.log.gz 2024-09-19T08:48:22.592 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54266.log 2024-09-19T08:48:22.592 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51956.log: /var/log/ceph/ceph-client.admin.46406.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51956.log.gz 2024-09-19T08:48:22.592 INFO:teuthology.orchestra.run.smithi098.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46406.log.gz 2024-09-19T08:48:22.592 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51373.log 2024-09-19T08:48:22.592 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43964.log 2024-09-19T08:48:22.592 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.54266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48342.log 2024-09-19T08:48:22.592 INFO:teuthology.orchestra.run.smithi098.stderr: 89.2%/var/log/ceph/ceph-client.admin.51373.log: -- replaced with /var/log/ceph/ceph-client.admin.54266.log.gz 2024-09-19T08:48:22.592 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43964.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51373.log.gz 2024-09-19T08:48:22.592 INFO:teuthology.orchestra.run.smithi098.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43964.log.gz 2024-09-19T08:48:22.592 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51780.log 2024-09-19T08:48:22.592 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48316.log 2024-09-19T08:48:22.592 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51780.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48342.log.gz 2024-09-19T08:48:22.593 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49459.log 2024-09-19T08:48:22.593 INFO:teuthology.orchestra.run.smithi098.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55135.log 2024-09-19T08:48:22.593 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51780.log.gz 2024-09-19T08:48:22.601 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48316.log: /var/log/ceph/ceph-client.admin.49459.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48316.log.gz 2024-09-19T08:48:22.601 INFO:teuthology.orchestra.run.smithi098.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49459.log.gz 2024-09-19T08:48:22.601 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55669.log 2024-09-19T08:48:22.601 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38637.log 2024-09-19T08:48:22.601 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.55135.log: /var/log/ceph/ceph-client.admin.55669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55695.log 2024-09-19T08:48:22.601 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.38637.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55135.log.gz 2024-09-19T08:48:22.601 INFO:teuthology.orchestra.run.smithi098.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55669.log.gz 2024-09-19T08:48:22.602 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50601.log 2024-09-19T08:48:22.602 INFO:teuthology.orchestra.run.smithi098.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38637.log.gz 2024-09-19T08:48:22.602 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54766.log 2024-09-19T08:48:22.602 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.55695.log: /var/log/ceph/ceph-client.admin.50601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42615.log 2024-09-19T08:48:22.602 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.54766.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50601.log.gz 2024-09-19T08:48:22.602 INFO:teuthology.orchestra.run.smithi098.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55695.log.gz 2024-09-19T08:48:22.602 INFO:teuthology.orchestra.run.smithi098.stderr: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53655.log 2024-09-19T08:48:22.602 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54766.log.gz 2024-09-19T08:48:22.602 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42481.log 2024-09-19T08:48:22.602 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42615.log: /var/log/ceph/ceph-client.admin.53655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44243.log 2024-09-19T08:48:22.602 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42481.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42615.log.gz 2024-09-19T08:48:22.602 INFO:teuthology.orchestra.run.smithi098.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53655.log.gz 2024-09-19T08:48:22.603 INFO:teuthology.orchestra.run.smithi098.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42481.log.gz 2024-09-19T08:48:22.603 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50835.log 2024-09-19T08:48:22.603 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46882.log 2024-09-19T08:48:22.603 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44243.log: /var/log/ceph/ceph-client.admin.50835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35206.log 2024-09-19T08:48:22.603 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46882.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44243.log.gz 2024-09-19T08:48:22.603 INFO:teuthology.orchestra.run.smithi098.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50835.log.gz 2024-09-19T08:48:22.603 INFO:teuthology.orchestra.run.smithi098.stderr: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43148.log 2024-09-19T08:48:22.603 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46882.log.gz 2024-09-19T08:48:22.603 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55407.log 2024-09-19T08:48:22.603 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.35206.log: /var/log/ceph/ceph-client.admin.43148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48809.log 2024-09-19T08:48:22.603 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.55407.log: 88.3% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35206.log.gz 2024-09-19T08:48:22.603 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43148.log.gz 2024-09-19T08:48:22.603 INFO:teuthology.orchestra.run.smithi098.stderr: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40082.log 2024-09-19T08:48:22.603 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55407.log.gz 2024-09-19T08:48:22.604 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48252.log 2024-09-19T08:48:22.604 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48809.log: /var/log/ceph/ceph-client.admin.40082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51126.log 2024-09-19T08:48:22.604 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48252.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48809.log.gz 2024-09-19T08:48:22.604 INFO:teuthology.orchestra.run.smithi098.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40082.log.gz 2024-09-19T08:48:22.604 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49303.log 2024-09-19T08:48:22.604 INFO:teuthology.orchestra.run.smithi098.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48252.log.gz 2024-09-19T08:48:22.604 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38993.log 2024-09-19T08:48:22.604 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51126.log: /var/log/ceph/ceph-client.admin.49303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39201.log 2024-09-19T08:48:22.604 INFO:teuthology.orchestra.run.smithi098.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51126.log.gz 2024-09-19T08:48:22.604 INFO:teuthology.orchestra.run.smithi098.stderr: 88.6%/var/log/ceph/ceph-client.admin.38993.log: -- replaced with /var/log/ceph/ceph-client.admin.49303.log.gz 2024-09-19T08:48:22.604 INFO:teuthology.orchestra.run.smithi098.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38993.log.gz 2024-09-19T08:48:22.604 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43061.log 2024-09-19T08:48:22.611 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.39201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52825.log 2024-09-19T08:48:22.611 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43061.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52430.log 2024-09-19T08:48:22.611 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39201.log.gz 2024-09-19T08:48:22.611 INFO:teuthology.orchestra.run.smithi098.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43061.log.gz 2024-09-19T08:48:22.611 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52825.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52825.log.gz 2024-09-19T08:48:22.611 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34715.log 2024-09-19T08:48:22.611 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.36111.log 2024-09-19T08:48:22.611 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37882.log 2024-09-19T08:48:22.612 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.34715.log: /var/log/ceph/ceph-client.2.36111.log: 89.2% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52430.log.gz 2024-09-19T08:48:22.612 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34715.log.gz 2024-09-19T08:48:22.612 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50783.log 2024-09-19T08:48:22.612 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44533.log 2024-09-19T08:48:22.612 INFO:teuthology.orchestra.run.smithi098.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.2.36111.log.gz 2024-09-19T08:48:22.612 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.37882.log: /var/log/ceph/ceph-client.admin.50783.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37882.log.gz 2024-09-19T08:48:22.612 INFO:teuthology.orchestra.run.smithi098.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50783.log.gz 2024-09-19T08:48:22.612 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39416.log 2024-09-19T08:48:22.612 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38718.log 2024-09-19T08:48:22.612 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44533.log: /var/log/ceph/ceph-client.admin.39416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42771.log 2024-09-19T08:48:22.612 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.38718.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44533.log.gz 2024-09-19T08:48:22.612 INFO:teuthology.orchestra.run.smithi098.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39416.log.gz 2024-09-19T08:48:22.612 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-19T08:48:22.613 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38718.log.gz 2024-09-19T08:48:22.613 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36325.log 2024-09-19T08:48:22.613 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42771.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36553.log 2024-09-19T08:48:22.613 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36325.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42771.log.gz 2024-09-19T08:48:22.613 INFO:teuthology.orchestra.run.smithi098.stderr: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46545.log 2024-09-19T08:48:22.613 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36325.log.gz 2024-09-19T08:48:22.613 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36553.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36553.log.gz 2024-09-19T08:48:22.613 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45817.log 2024-09-19T08:48:22.613 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43650.log 2024-09-19T08:48:22.613 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46545.log: 86.7%/var/log/ceph/ceph-client.admin.45817.log: -- replaced with /var/log/ceph/ceph.log.gz 2024-09-19T08:48:22.613 INFO:teuthology.orchestra.run.smithi098.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46545.log.gz 2024-09-19T08:48:22.613 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45817.log.gz 2024-09-19T08:48:22.613 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43781.log 2024-09-19T08:48:22.619 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37344.log 2024-09-19T08:48:22.619 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34383.log 2024-09-19T08:48:22.620 INFO:teuthology.orchestra.run.smithi098.stderr: 88.1% 88.0%/var/log/ceph/ceph-client.admin.37344.log: -- replaced with /var/log/ceph/ceph-client.admin.43781.log.gz 2024-09-19T08:48:22.620 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43650.log.gz 2024-09-19T08:48:22.620 INFO:teuthology.orchestra.run.smithi098.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37344.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35838.log 2024-09-19T08:48:22.620 INFO:teuthology.orchestra.run.smithi098.stderr: 2024-09-19T08:48:22.620 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54014.log 2024-09-19T08:48:22.620 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.34383.log: /var/log/ceph/ceph-client.admin.35838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43755.log 2024-09-19T08:48:22.620 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.54014.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34383.log.gz 2024-09-19T08:48:22.620 INFO:teuthology.orchestra.run.smithi098.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35838.log.gz 2024-09-19T08:48:22.620 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42103.log 2024-09-19T08:48:22.620 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54014.log.gz 2024-09-19T08:48:22.620 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43755.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43755.log.gz 2024-09-19T08:48:22.620 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34991.log 2024-09-19T08:48:22.620 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50153.log 2024-09-19T08:48:22.621 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42103.log: /var/log/ceph/ceph-client.admin.34991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52138.log 2024-09-19T08:48:22.621 INFO:teuthology.orchestra.run.smithi098.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34991.log.gz 2024-09-19T08:48:22.621 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50153.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42103.log.gz 2024-09-19T08:48:22.621 INFO:teuthology.orchestra.run.smithi098.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50153.log.gz 2024-09-19T08:48:22.621 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35451.log 2024-09-19T08:48:22.621 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49381.log 2024-09-19T08:48:22.621 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.35451.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52138.log.gz 2024-09-19T08:48:22.621 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40772.log 2024-09-19T08:48:22.621 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35451.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37041.log 2024-09-19T08:48:22.621 INFO:teuthology.orchestra.run.smithi098.stderr: 2024-09-19T08:48:22.621 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49381.log: /var/log/ceph/ceph-client.admin.40772.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49381.log.gz 2024-09-19T08:48:22.621 INFO:teuthology.orchestra.run.smithi098.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40772.log.gz 2024-09-19T08:48:22.621 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47444.log 2024-09-19T08:48:22.622 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38351.log 2024-09-19T08:48:22.622 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.37041.log: /var/log/ceph/ceph-client.admin.47444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54424.log 2024-09-19T08:48:22.627 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.38351.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37041.log.gz 2024-09-19T08:48:22.627 INFO:teuthology.orchestra.run.smithi098.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47444.log.gz 2024-09-19T08:48:22.627 INFO:teuthology.orchestra.run.smithi098.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41337.log 2024-09-19T08:48:22.627 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38351.log.gz 2024-09-19T08:48:22.627 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48226.log 2024-09-19T08:48:22.627 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.54424.log: /var/log/ceph/ceph-client.admin.41337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54318.log 2024-09-19T08:48:22.628 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48226.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41337.log.gz 2024-09-19T08:48:22.628 INFO:teuthology.orchestra.run.smithi098.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54424.log.gz 2024-09-19T08:48:22.628 INFO:teuthology.orchestra.run.smithi098.stderr: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55327.log 2024-09-19T08:48:22.628 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48226.log.gz 2024-09-19T08:48:22.628 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45250.log 2024-09-19T08:48:22.628 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.54318.log: /var/log/ceph/ceph-client.admin.55327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46937.log 2024-09-19T08:48:22.628 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45250.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54318.log.gz 2024-09-19T08:48:22.628 INFO:teuthology.orchestra.run.smithi098.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55327.log.gz 2024-09-19T08:48:22.628 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46911.log 2024-09-19T08:48:22.628 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45250.log.gz 2024-09-19T08:48:22.628 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47286.log 2024-09-19T08:48:22.628 INFO:teuthology.orchestra.run.smithi098.stderr: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38429.log 2024-09-19T08:48:22.628 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46937.log.gz 2024-09-19T08:48:22.629 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46911.log: /var/log/ceph/ceph-client.admin.47286.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46911.log.gz 2024-09-19T08:48:22.629 INFO:teuthology.orchestra.run.smithi098.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47286.log.gz 2024-09-19T08:48:22.629 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38325.log 2024-09-19T08:48:22.629 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.38429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52183.log 2024-09-19T08:48:22.629 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9%/var/log/ceph/ceph-client.admin.38325.log: -- replaced with /var/log/ceph/ceph-client.admin.38429.log.gz 2024-09-19T08:48:22.629 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40506.log 2024-09-19T08:48:22.629 INFO:teuthology.orchestra.run.smithi098.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38325.log.gz 2024-09-19T08:48:22.629 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52183.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52183.log.gz 2024-09-19T08:48:22.629 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55355.log 2024-09-19T08:48:22.629 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.40506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47312.log 2024-09-19T08:48:22.629 INFO:teuthology.orchestra.run.smithi098.stderr: 88.2%/var/log/ceph/ceph-client.admin.55355.log: -- replaced with /var/log/ceph/ceph-client.admin.40506.log.gz 2024-09-19T08:48:22.629 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49485.log 2024-09-19T08:48:22.635 INFO:teuthology.orchestra.run.smithi098.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55355.log.gz 2024-09-19T08:48:22.635 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47312.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47312.log.gz 2024-09-19T08:48:22.635 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45409.log 2024-09-19T08:48:22.635 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37924.log 2024-09-19T08:48:22.635 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49485.log: /var/log/ceph/ceph-client.admin.45409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36709.log 2024-09-19T08:48:22.636 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.37924.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49485.log.gz 2024-09-19T08:48:22.636 INFO:teuthology.orchestra.run.smithi098.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45409.log.gz 2024-09-19T08:48:22.636 INFO:teuthology.orchestra.run.smithi098.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37924.log.gz 2024-09-19T08:48:22.636 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46432.log 2024-09-19T08:48:22.636 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39045.log 2024-09-19T08:48:22.636 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36709.log: /var/log/ceph/ceph-client.admin.46432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44702.log 2024-09-19T08:48:22.636 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.39045.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36709.log.gz 2024-09-19T08:48:22.636 INFO:teuthology.orchestra.run.smithi098.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46432.log.gz 2024-09-19T08:48:22.636 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose 87.6% -- -- replaced with /var/log/ceph/ceph-client.admin.39045.log.gz /var/log/ceph/ceph-client.admin.36351.log 2024-09-19T08:48:22.636 INFO:teuthology.orchestra.run.smithi098.stderr: 2024-09-19T08:48:22.636 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40744.log 2024-09-19T08:48:22.636 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46830.log 2024-09-19T08:48:22.636 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36351.log: /var/log/ceph/ceph-client.admin.44702.log: /var/log/ceph/ceph-client.admin.40744.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.44702.log.gz 2024-09-19T08:48:22.637 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36351.log.gz 2024-09-19T08:48:22.637 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40744.log.gz 2024-09-19T08:48:22.637 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46775.log 2024-09-19T08:48:22.637 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48705.log 2024-09-19T08:48:22.637 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46830.log: /var/log/ceph/ceph-client.admin.46775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50627.log 2024-09-19T08:48:22.637 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48705.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46830.log.gz 2024-09-19T08:48:22.637 INFO:teuthology.orchestra.run.smithi098.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46775.log.gz 2024-09-19T08:48:22.637 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48705.log.gz 2024-09-19T08:48:22.637 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49064.log 2024-09-19T08:48:22.637 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45765.log 2024-09-19T08:48:22.637 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50627.log: gzip/var/log/ceph/ceph-client.admin.49064.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.39717.log 2024-09-19T08:48:22.637 INFO:teuthology.orchestra.run.smithi098.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50627.log.gz/var/log/ceph/ceph-client.admin.45765.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49064.log.gz 2024-09-19T08:48:22.637 INFO:teuthology.orchestra.run.smithi098.stderr: 2024-09-19T08:48:22.637 INFO:teuthology.orchestra.run.smithi098.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45765.log.gz 2024-09-19T08:48:22.645 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44945.log 2024-09-19T08:48:22.645 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.39717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54095.log 2024-09-19T08:48:22.645 INFO:teuthology.orchestra.run.smithi098.stderr: 88.2%/var/log/ceph/ceph-client.admin.44945.log: -- replaced with /var/log/ceph/ceph-client.admin.39717.log.gz 2024-09-19T08:48:22.645 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46134.log 2024-09-19T08:48:22.645 INFO:teuthology.orchestra.run.smithi098.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44945.log.gz 2024-09-19T08:48:22.645 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.54095.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54095.log.gz 2024-09-19T08:48:22.645 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44269.log 2024-09-19T08:48:22.645 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47418.log 2024-09-19T08:48:22.645 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46134.log: /var/log/ceph/ceph-client.admin.44269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38069.log 2024-09-19T08:48:22.645 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47418.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46134.log.gz 2024-09-19T08:48:22.645 INFO:teuthology.orchestra.run.smithi098.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44269.log.gz 2024-09-19T08:48:22.646 INFO:teuthology.orchestra.run.smithi098.stderr: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48986.log 2024-09-19T08:48:22.646 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47418.log.gz 2024-09-19T08:48:22.646 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36657.log 2024-09-19T08:48:22.646 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.38069.log: /var/log/ceph/ceph-client.admin.48986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36579.log 2024-09-19T08:48:22.646 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36657.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38069.log.gz 2024-09-19T08:48:22.646 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48986.log.gz 2024-09-19T08:48:22.646 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53221.log 2024-09-19T08:48:22.646 INFO:teuthology.orchestra.run.smithi098.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36657.log.gz 2024-09-19T08:48:22.646 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34998.log 2024-09-19T08:48:22.646 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36579.log: /var/log/ceph/ceph-client.admin.53221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53857.log 2024-09-19T08:48:22.646 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.34998.log: 88.2% 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36579.log.gz 2024-09-19T08:48:22.646 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53221.log.gz 2024-09-19T08:48:22.646 INFO:teuthology.orchestra.run.smithi098.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34998.log.gz 2024-09-19T08:48:22.646 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34845.log 2024-09-19T08:48:22.647 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37527.log 2024-09-19T08:48:22.647 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.53857.log: /var/log/ceph/ceph-client.admin.34845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43174.log 2024-09-19T08:48:22.647 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.37527.log: 87.5% 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53857.log.gz 2024-09-19T08:48:22.647 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34845.log.gz 2024-09-19T08:48:22.647 INFO:teuthology.orchestra.run.smithi098.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37527.log.gz 2024-09-19T08:48:22.647 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46515.log 2024-09-19T08:48:22.647 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55511.log 2024-09-19T08:48:22.647 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43174.log: /var/log/ceph/ceph-client.admin.46515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51017.log 2024-09-19T08:48:22.647 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.55511.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43174.log.gz 2024-09-19T08:48:22.647 INFO:teuthology.orchestra.run.smithi098.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46515.log.gz 2024-09-19T08:48:22.647 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose 89.2% -- -- replaced with /var/log/ceph/ceph-client.admin.55511.log.gz /var/log/ceph/ceph-client.admin.46304.log 2024-09-19T08:48:22.647 INFO:teuthology.orchestra.run.smithi098.stderr: 2024-09-19T08:48:22.653 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40943.log 2024-09-19T08:48:22.653 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52931.log 2024-09-19T08:48:22.654 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46304.log: /var/log/ceph/ceph-client.admin.40943.log: 88.1% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51017.log.gz 2024-09-19T08:48:22.654 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46304.log.gz 2024-09-19T08:48:22.654 INFO:teuthology.orchestra.run.smithi098.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40943.log.gz 2024-09-19T08:48:22.654 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47030.log 2024-09-19T08:48:22.654 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37093.log 2024-09-19T08:48:22.654 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52931.log: /var/log/ceph/ceph-client.admin.47030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40376.log 2024-09-19T08:48:22.654 INFO:teuthology.orchestra.run.smithi098.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52931.log.gz 2024-09-19T08:48:22.654 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.37093.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47030.log.gz 2024-09-19T08:48:22.654 INFO:teuthology.orchestra.run.smithi098.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37093.log.gz 2024-09-19T08:48:22.654 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37370.log 2024-09-19T08:48:22.654 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.40376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50575.log 2024-09-19T08:48:22.654 INFO:teuthology.orchestra.run.smithi098.stderr: 88.6%/var/log/ceph/ceph-client.admin.37370.log: -- replaced with /var/log/ceph/ceph-client.admin.40376.log.gz 2024-09-19T08:48:22.654 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48783.log 2024-09-19T08:48:22.654 INFO:teuthology.orchestra.run.smithi098.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37370.log.gz 2024-09-19T08:48:22.655 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50575.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50575.log.gz 2024-09-19T08:48:22.655 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34897.log 2024-09-19T08:48:22.655 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35679.log 2024-09-19T08:48:22.655 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48783.log: /var/log/ceph/ceph-client.admin.34897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45971.log 2024-09-19T08:48:22.655 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.35679.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48783.log.gz 2024-09-19T08:48:22.655 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34897.log.gz 2024-09-19T08:48:22.655 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45609.log 2024-09-19T08:48:22.655 INFO:teuthology.orchestra.run.smithi098.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35679.log.gz 2024-09-19T08:48:22.655 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54123.log 2024-09-19T08:48:22.655 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54528.log 2024-09-19T08:48:22.655 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45971.log: /var/log/ceph/ceph-client.admin.54123.log: 88.7% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45971.log.gz 2024-09-19T08:48:22.655 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45609.log.gz 2024-09-19T08:48:22.655 INFO:teuthology.orchestra.run.smithi098.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54123.log.gz 2024-09-19T08:48:22.655 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35966.log 2024-09-19T08:48:22.656 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46252.log 2024-09-19T08:48:22.656 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.54528.log: /var/log/ceph/ceph-client.0.35966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52691.log 2024-09-19T08:48:22.661 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46252.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46252.log.gz 2024-09-19T08:48:22.662 INFO:teuthology.orchestra.run.smithi098.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.54528.log.gz 2024-09-19T08:48:22.662 INFO:teuthology.orchestra.run.smithi098.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.0.35966.log.gz 2024-09-19T08:48:22.662 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52983.log 2024-09-19T08:48:22.662 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37119.log 2024-09-19T08:48:22.662 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47504.log 2024-09-19T08:48:22.662 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52983.log: 88.7%/var/log/ceph/ceph-client.admin.37119.log: -- replaced with /var/log/ceph/ceph-client.admin.52983.log.gz 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52691.log.gz 2024-09-19T08:48:22.662 INFO:teuthology.orchestra.run.smithi098.stderr: 2024-09-19T08:48:22.662 INFO:teuthology.orchestra.run.smithi098.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37119.log.gz 2024-09-19T08:48:22.662 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35191.log 2024-09-19T08:48:22.662 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41848.log 2024-09-19T08:48:22.662 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47504.log: /var/log/ceph/ceph-client.admin.35191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42220.log 2024-09-19T08:48:22.662 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.41848.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35191.log.gz 2024-09-19T08:48:22.662 INFO:teuthology.orchestra.run.smithi098.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47504.log.gz 2024-09-19T08:48:22.662 INFO:teuthology.orchestra.run.smithi098.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41848.log.gz 2024-09-19T08:48:22.663 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44135.log 2024-09-19T08:48:22.663 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48653.log 2024-09-19T08:48:22.663 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49853.log 2024-09-19T08:48:22.663 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44135.log: /var/log/ceph/ceph-client.admin.48653.log: 88.5% 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42220.log.gz 2024-09-19T08:48:22.663 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44135.log.gz 2024-09-19T08:48:22.663 INFO:teuthology.orchestra.run.smithi098.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48653.log.gz 2024-09-19T08:48:22.663 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39474.log 2024-09-19T08:48:22.663 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52313.log 2024-09-19T08:48:22.663 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50549.log 2024-09-19T08:48:22.663 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.39474.log: /var/log/ceph/ceph-client.admin.52313.log: 85.7% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39474.log.gz -- replaced with /var/log/ceph/ceph-client.admin.49853.log.gz 2024-09-19T08:48:22.663 INFO:teuthology.orchestra.run.smithi098.stderr: 2024-09-19T08:48:22.663 INFO:teuthology.orchestra.run.smithi098.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52313.log.gz 2024-09-19T08:48:22.663 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43200.log 2024-09-19T08:48:22.663 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39960.log 2024-09-19T08:48:22.664 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50549.log: /var/log/ceph/ceph-client.admin.43200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36761.log 2024-09-19T08:48:22.664 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.39960.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.50549.log.gz 2024-09-19T08:48:22.664 INFO:teuthology.orchestra.run.smithi098.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43200.log.gz 2024-09-19T08:48:22.664 INFO:teuthology.orchestra.run.smithi098.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39960.log.gz 2024-09-19T08:48:22.664 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54740.log 2024-09-19T08:48:22.667 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47137.log 2024-09-19T08:48:22.667 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41999.log 2024-09-19T08:48:22.668 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.54740.log: /var/log/ceph/ceph-client.admin.47137.log: 88.5% 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36761.log.gz 2024-09-19T08:48:22.668 INFO:teuthology.orchestra.run.smithi098.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54740.log.gz 2024-09-19T08:48:22.668 INFO:teuthology.orchestra.run.smithi098.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47137.log.gz 2024-09-19T08:48:22.668 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51864.log 2024-09-19T08:48:22.668 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35757.log 2024-09-19T08:48:22.668 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.41999.log: /var/log/ceph/ceph-client.admin.51864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45518.log 2024-09-19T08:48:22.668 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.35757.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41999.log.gz 2024-09-19T08:48:22.668 INFO:teuthology.orchestra.run.smithi098.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51864.log.gz 2024-09-19T08:48:22.668 INFO:teuthology.orchestra.run.smithi098.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35757.log.gz 2024-09-19T08:48:22.668 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-19T08:48:22.668 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48532.log 2024-09-19T08:48:22.668 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45518.log: /var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-19T08:48:22.669 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48532.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45518.log.gz 2024-09-19T08:48:22.669 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54158.log 2024-09-19T08:48:22.669 INFO:teuthology.orchestra.run.smithi098.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48532.log.gz 2024-09-19T08:48:22.670 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-mds.c.log: 89.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-19T08:48:22.670 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29028.log 2024-09-19T08:48:22.670 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.54158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50861.log 2024-09-19T08:48:22.670 INFO:teuthology.orchestra.run.smithi098.stderr: 89.3%/var/log/ceph/ceph-client.admin.29028.log: -- replaced with /var/log/ceph/ceph-client.admin.54158.log.gz 2024-09-19T08:48:22.671 INFO:teuthology.orchestra.run.smithi098.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.29028.log.gz 2024-09-19T08:48:22.671 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39839.log 2024-09-19T08:48:22.671 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38540.log 2024-09-19T08:48:22.672 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50861.log: /var/log/ceph/ceph-client.admin.39839.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50861.log.gz 2024-09-19T08:48:22.672 INFO:teuthology.orchestra.run.smithi098.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39839.log.gz 2024-09-19T08:48:22.672 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45172.log 2024-09-19T08:48:22.673 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.38540.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38540.log.gz 2024-09-19T08:48:22.673 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-19T08:48:22.674 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36839.log 2024-09-19T08:48:22.674 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45172.log: /var/log/ceph/ceph-mds.a.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45172.log.gz 2024-09-19T08:48:22.674 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48558.log 2024-09-19T08:48:22.675 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36839.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36839.log.gz 2024-09-19T08:48:22.675 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36871.log 2024-09-19T08:48:22.676 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48558.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48558.log.gz 2024-09-19T08:48:22.676 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43230.log 2024-09-19T08:48:22.677 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36871.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36871.log.gz 2024-09-19T08:48:22.677 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49129.log 2024-09-19T08:48:22.679 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43230.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43230.log.gz 2024-09-19T08:48:22.679 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42285.log 2024-09-19T08:48:22.680 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49129.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49129.log.gz 2024-09-19T08:48:22.680 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37423.log 2024-09-19T08:48:22.681 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42285.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42285.log.gz 2024-09-19T08:48:22.681 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53726.log 2024-09-19T08:48:22.682 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.37423.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37423.log.gz 2024-09-19T08:48:22.682 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43553.log 2024-09-19T08:48:22.683 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.53726.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53726.log.gz 2024-09-19T08:48:22.684 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37173.log 2024-09-19T08:48:22.685 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43553.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43553.log.gz 2024-09-19T08:48:22.685 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46963.log 2024-09-19T08:48:22.686 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.37173.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37173.log.gz 2024-09-19T08:48:22.686 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42537.log 2024-09-19T08:48:22.687 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46963.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46963.log.gz 2024-09-19T08:48:22.688 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48118.log 2024-09-19T08:48:22.688 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42537.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42537.log.gz 2024-09-19T08:48:22.689 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54344.log 2024-09-19T08:48:22.700 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.0.48118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45635.log 2024-09-19T08:48:22.700 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.54344.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54344.log.gz 2024-09-19T08:48:22.706 INFO:teuthology.orchestra.run.smithi098.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.0.48118.log.gz 2024-09-19T08:48:22.707 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52112.log 2024-09-19T08:48:22.708 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45635.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45635.log.gz 2024-09-19T08:48:22.709 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39151.log 2024-09-19T08:48:22.710 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52112.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52112.log.gz 2024-09-19T08:48:22.710 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.24034.log 2024-09-19T08:48:22.711 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.39151.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.39151.log.gz 2024-09-19T08:48:22.712 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42420.log 2024-09-19T08:48:22.712 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph.tmp-client.admin.24034.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.24034.log.gz 2024-09-19T08:48:22.713 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42589.log 2024-09-19T08:48:22.713 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42420.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42420.log.gz 2024-09-19T08:48:22.714 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54040.log 2024-09-19T08:48:22.715 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42589.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42589.log.gz 2024-09-19T08:48:22.715 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45120.log 2024-09-19T08:48:22.716 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.54040.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54040.log.gz 2024-09-19T08:48:22.716 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54184.log 2024-09-19T08:48:22.717 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45120.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45120.log.gz 2024-09-19T08:48:22.718 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54829.log 2024-09-19T08:48:22.719 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.54184.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54184.log.gz 2024-09-19T08:48:22.719 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53352.log 2024-09-19T08:48:22.721 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.54829.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54829.log.gz 2024-09-19T08:48:22.721 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44919.log 2024-09-19T08:48:22.722 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.53352.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53352.log.gz 2024-09-19T08:48:22.722 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41099.log 2024-09-19T08:48:22.722 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44919.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44919.log.gz 2024-09-19T08:48:22.723 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39869.log 2024-09-19T08:48:22.724 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.41099.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41099.log.gz 2024-09-19T08:48:22.724 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40917.log 2024-09-19T08:48:22.726 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.39869.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39869.log.gz 2024-09-19T08:48:22.726 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45869.log 2024-09-19T08:48:22.726 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.40917.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40917.log.gz 2024-09-19T08:48:22.727 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37449.log 2024-09-19T08:48:22.728 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45869.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45869.log.gz 2024-09-19T08:48:22.728 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42392.log 2024-09-19T08:48:22.729 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.37449.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37449.log.gz 2024-09-19T08:48:22.729 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44997.log 2024-09-19T08:48:22.730 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42392.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42392.log.gz 2024-09-19T08:48:22.731 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49407.log 2024-09-19T08:48:22.732 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44997.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44997.log.gz 2024-09-19T08:48:22.732 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39777.log 2024-09-19T08:48:22.733 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49407.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49407.log.gz 2024-09-19T08:48:22.733 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44044.log 2024-09-19T08:48:22.734 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.39777.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39777.log.gz 2024-09-19T08:48:22.735 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48368.log 2024-09-19T08:48:22.735 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44044.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44044.log.gz 2024-09-19T08:48:22.736 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48452.log 2024-09-19T08:48:22.737 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48368.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48368.log.gz 2024-09-19T08:48:22.737 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36384.log 2024-09-19T08:48:22.738 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48452.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48452.log.gz 2024-09-19T08:48:22.738 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39595.log 2024-09-19T08:48:22.739 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36384.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36384.log.gz 2024-09-19T08:48:22.740 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51664.log 2024-09-19T08:48:22.741 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.39595.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39595.log.gz 2024-09-19T08:48:22.741 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47628.log 2024-09-19T08:48:22.742 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51664.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51664.log.gz 2024-09-19T08:48:22.742 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42983.log 2024-09-19T08:48:22.743 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47628.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47628.log.gz 2024-09-19T08:48:22.743 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45583.log 2024-09-19T08:48:22.744 INFO:teuthology.orchestra.run.smithi139.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-19T08:48:22.745 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42983.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42983.log.gz 2024-09-19T08:48:22.745 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54919.log 2024-09-19T08:48:22.746 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45583.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45583.log.gz 2024-09-19T08:48:22.746 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55057.log 2024-09-19T08:48:22.747 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.54919.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54919.log.gz 2024-09-19T08:48:22.747 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45094.log 2024-09-19T08:48:22.749 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.55057.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55057.log.gz 2024-09-19T08:48:22.749 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43807.log 2024-09-19T08:48:22.751 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45094.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45094.log.gz 2024-09-19T08:48:22.751 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37845.log 2024-09-19T08:48:22.752 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43807.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43807.log.gz 2024-09-19T08:48:22.752 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46999.log 2024-09-19T08:48:22.752 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.37845.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37845.log.gz 2024-09-19T08:48:22.753 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55747.log 2024-09-19T08:48:22.754 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46999.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46999.log.gz 2024-09-19T08:48:22.754 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46081.log 2024-09-19T08:48:22.755 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.55747.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55747.log.gz 2024-09-19T08:48:22.755 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34995.log 2024-09-19T08:48:22.756 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46081.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46081.log.gz 2024-09-19T08:48:22.756 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35211.log 2024-09-19T08:48:22.757 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.34995.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34995.log.gz 2024-09-19T08:48:22.758 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41450.log 2024-09-19T08:48:22.760 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.35211.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35211.log.gz 2024-09-19T08:48:22.760 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46200.log 2024-09-19T08:48:22.760 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.41450.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41450.log.gz 2024-09-19T08:48:22.761 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55563.log 2024-09-19T08:48:22.761 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46200.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46200.log.gz 2024-09-19T08:48:22.761 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45344.log 2024-09-19T08:48:22.762 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.55563.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55563.log.gz 2024-09-19T08:48:22.763 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48478.log 2024-09-19T08:48:22.764 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45344.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45344.log.gz 2024-09-19T08:48:22.764 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39655.log 2024-09-19T08:48:22.766 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48478.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48478.log.gz 2024-09-19T08:48:22.766 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36468.log 2024-09-19T08:48:22.766 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.39655.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39655.log.gz 2024-09-19T08:48:22.767 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34741.log 2024-09-19T08:48:22.768 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36468.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36468.log.gz 2024-09-19T08:48:22.768 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36735.log 2024-09-19T08:48:22.769 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.34741.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34741.log.gz 2024-09-19T08:48:22.769 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40835.log 2024-09-19T08:48:22.770 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36735.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36735.log.gz 2024-09-19T08:48:22.770 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48960.log 2024-09-19T08:48:22.771 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.40835.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40835.log.gz 2024-09-19T08:48:22.772 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36523.log 2024-09-19T08:48:22.773 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48960.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48960.log.gz 2024-09-19T08:48:22.773 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38781.log 2024-09-19T08:48:22.774 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36523.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36523.log.gz 2024-09-19T08:48:22.774 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52034.log 2024-09-19T08:48:22.775 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.38781.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38781.log.gz 2024-09-19T08:48:22.776 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54066.log 2024-09-19T08:48:22.777 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52034.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52034.log.gz 2024-09-19T08:48:22.778 INFO:teuthology.orchestra.run.smithi139.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-19T08:48:22.780 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49928.log 2024-09-19T08:48:22.780 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.54066.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54066.log.gz 2024-09-19T08:48:22.780 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39019.log 2024-09-19T08:48:22.781 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49928.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49928.log.gz 2024-09-19T08:48:22.781 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55206.log 2024-09-19T08:48:22.782 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.39019.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39019.log.gz 2024-09-19T08:48:22.782 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45146.log 2024-09-19T08:48:22.782 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.55206.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55206.log.gz 2024-09-19T08:48:22.782 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42641.log 2024-09-19T08:48:22.783 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45146.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45146.log.gz 2024-09-19T08:48:22.783 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39807.log 2024-09-19T08:48:22.784 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42641.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42641.log.gz 2024-09-19T08:48:22.784 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48173.log 2024-09-19T08:48:22.791 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.39807.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39807.log.gz 2024-09-19T08:48:22.791 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44109.log 2024-09-19T08:48:22.791 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48173.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48173.log.gz 2024-09-19T08:48:22.791 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46856.log 2024-09-19T08:48:22.791 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44109.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44109.log.gz 2024-09-19T08:48:22.791 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47340.log 2024-09-19T08:48:22.791 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46856.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46856.log.gz 2024-09-19T08:48:22.791 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36299.log 2024-09-19T08:48:22.791 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47340.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47340.log.gz 2024-09-19T08:48:22.791 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46458.log 2024-09-19T08:48:22.793 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36299.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36299.log.gz 2024-09-19T08:48:22.793 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46160.log 2024-09-19T08:48:22.793 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46458.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46458.log.gz 2024-09-19T08:48:22.793 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53320.log 2024-09-19T08:48:22.794 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.46160.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46160.log.gz 2024-09-19T08:48:22.794 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47797.log 2024-09-19T08:48:22.795 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.53320.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53320.log.gz 2024-09-19T08:48:22.795 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47215.log 2024-09-19T08:48:22.796 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47797.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47797.log.gz 2024-09-19T08:48:22.797 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50705.log 2024-09-19T08:48:22.798 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47215.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47215.log.gz 2024-09-19T08:48:22.798 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41047.log 2024-09-19T08:48:22.799 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50705.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50705.log.gz 2024-09-19T08:48:22.799 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49355.log 2024-09-19T08:48:22.800 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.41047.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41047.log.gz 2024-09-19T08:48:22.801 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41599.log 2024-09-19T08:48:22.802 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49355.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49355.log.gz 2024-09-19T08:48:22.802 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53949.log 2024-09-19T08:48:22.803 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.41599.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41599.log.gz 2024-09-19T08:48:22.803 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50228.log 2024-09-19T08:48:22.804 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.53949.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53949.log.gz 2024-09-19T08:48:22.804 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45461.log 2024-09-19T08:48:22.805 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50228.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50228.log.gz 2024-09-19T08:48:22.806 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36683.log 2024-09-19T08:48:22.807 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45461.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45461.log.gz 2024-09-19T08:48:22.807 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42929.log 2024-09-19T08:48:22.808 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36683.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36683.log.gz 2024-09-19T08:48:22.808 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48679.log 2024-09-19T08:48:22.810 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.0.42929.log: 89.4% -- replaced with /var/log/ceph/ceph-client.0.42929.log.gz 2024-09-19T08:48:22.810 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54887.log 2024-09-19T08:48:22.811 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48679.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48679.log.gz 2024-09-19T08:48:22.811 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50328.log 2024-09-19T08:48:22.812 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.54887.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54887.log.gz 2024-09-19T08:48:22.812 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44893.log 2024-09-19T08:48:22.813 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50328.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.50328.log.gz 2024-09-19T08:48:22.813 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51069.log 2024-09-19T08:48:22.815 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44893.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44893.log.gz 2024-09-19T08:48:22.815 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34449.log 2024-09-19T08:48:22.816 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51069.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51069.log.gz 2024-09-19T08:48:22.816 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51749.log 2024-09-19T08:48:22.817 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.34449.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34449.log.gz 2024-09-19T08:48:22.817 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39363.log 2024-09-19T08:48:22.819 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51749.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51749.log.gz 2024-09-19T08:48:22.819 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40565.log 2024-09-19T08:48:22.820 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.39363.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39363.log.gz 2024-09-19T08:48:22.820 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36273.log 2024-09-19T08:48:22.821 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.40565.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40565.log.gz 2024-09-19T08:48:22.822 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36988.log 2024-09-19T08:48:22.822 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36273.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36273.log.gz 2024-09-19T08:48:22.822 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40689.log 2024-09-19T08:48:22.823 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.36988.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36988.log.gz 2024-09-19T08:48:22.824 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54578.log 2024-09-19T08:48:22.824 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.40689.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40689.log.gz 2024-09-19T08:48:22.825 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41389.log 2024-09-19T08:48:22.826 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.54578.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54578.log.gz 2024-09-19T08:48:22.826 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45713.log 2024-09-19T08:48:22.827 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.41389.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41389.log.gz 2024-09-19T08:48:22.827 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41227.log 2024-09-19T08:48:22.828 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45713.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45713.log.gz 2024-09-19T08:48:22.829 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35001.log 2024-09-19T08:48:22.829 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.41227.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41227.log.gz 2024-09-19T08:48:22.830 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39990.log 2024-09-19T08:48:22.831 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.35001.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35001.log.gz 2024-09-19T08:48:22.831 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40305.log 2024-09-19T08:48:22.832 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.39990.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39990.log.gz 2024-09-19T08:48:22.832 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41151.log 2024-09-19T08:48:22.833 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.40305.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40305.log.gz 2024-09-19T08:48:22.834 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55381.log 2024-09-19T08:48:22.834 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.41151.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41151.log.gz 2024-09-19T08:48:22.835 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43350.log 2024-09-19T08:48:22.836 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.55381.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55381.log.gz 2024-09-19T08:48:22.836 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42129.log 2024-09-19T08:48:22.837 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43350.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43350.log.gz 2024-09-19T08:48:22.837 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49639.log 2024-09-19T08:48:22.838 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42129.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42129.log.gz 2024-09-19T08:48:22.838 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50278.log 2024-09-19T08:48:22.840 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49639.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49639.log.gz 2024-09-19T08:48:22.840 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42077.log 2024-09-19T08:48:22.841 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50278.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50278.log.gz 2024-09-19T08:48:22.841 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47771.log 2024-09-19T08:48:22.842 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42077.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42077.log.gz 2024-09-19T08:48:22.843 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55083.log 2024-09-19T08:48:22.843 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.47771.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47771.log.gz 2024-09-19T08:48:22.844 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52522.log 2024-09-19T08:48:22.845 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.55083.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55083.log.gz 2024-09-19T08:48:22.845 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54210.log 2024-09-19T08:48:22.846 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52522.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52522.log.gz 2024-09-19T08:48:22.846 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43990.log 2024-09-19T08:48:22.847 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.54210.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54210.log.gz 2024-09-19T08:48:22.847 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38863.log 2024-09-19T08:48:22.848 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43990.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.43990.log.gz 2024-09-19T08:48:22.849 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40891.log 2024-09-19T08:48:22.851 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.38863.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38863.log.gz 2024-09-19T08:48:22.851 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45312.log 2024-09-19T08:48:22.851 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.40891.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40891.log.gz 2024-09-19T08:48:22.852 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37201.log 2024-09-19T08:48:22.852 INFO:teuthology.orchestra.run.smithi139.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-19T08:48:22.853 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45312.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45312.log.gz 2024-09-19T08:48:22.853 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53629.log 2024-09-19T08:48:22.854 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.37201.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37201.log.gz 2024-09-19T08:48:22.854 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42667.log 2024-09-19T08:48:22.855 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.53629.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53629.log.gz 2024-09-19T08:48:22.855 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35209.log 2024-09-19T08:48:22.856 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42667.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42667.log.gz 2024-09-19T08:48:22.856 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52287.log 2024-09-19T08:48:22.857 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.35209.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35209.log.gz 2024-09-19T08:48:22.857 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38746.log 2024-09-19T08:48:22.858 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52287.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52287.log.gz 2024-09-19T08:48:22.859 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49978.log 2024-09-19T08:48:22.860 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.38746.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38746.log.gz 2024-09-19T08:48:22.860 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45687.log 2024-09-19T08:48:22.861 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49978.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49978.log.gz 2024-09-19T08:48:22.861 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52548.log 2024-09-19T08:48:22.862 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45687.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45687.log.gz 2024-09-19T08:48:22.863 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48593.log 2024-09-19T08:48:22.863 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52548.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52548.log.gz 2024-09-19T08:48:22.864 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45198.log 2024-09-19T08:48:22.865 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.48593.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48593.log.gz 2024-09-19T08:48:22.865 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38481.log 2024-09-19T08:48:22.866 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45198.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45198.log.gz 2024-09-19T08:48:22.866 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44585.log 2024-09-19T08:48:22.867 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.38481.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38481.log.gz 2024-09-19T08:48:22.867 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52721.log 2024-09-19T08:48:22.868 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.44585.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44585.log.gz 2024-09-19T08:48:22.869 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35447.log 2024-09-19T08:48:22.869 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52721.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52721.log.gz 2024-09-19T08:48:22.870 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37015.log 2024-09-19T08:48:22.871 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.35447.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35447.log.gz 2024-09-19T08:48:22.871 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52060.log 2024-09-19T08:48:22.872 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.37015.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37015.log.gz 2024-09-19T08:48:22.873 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49433.log 2024-09-19T08:48:22.873 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52060.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52060.log.gz 2024-09-19T08:48:22.874 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49221.log 2024-09-19T08:48:22.875 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49433.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49433.log.gz 2024-09-19T08:48:22.875 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54714.log 2024-09-19T08:48:22.876 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49221.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49221.log.gz 2024-09-19T08:48:22.876 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42873.log 2024-09-19T08:48:22.877 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.54714.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54714.log.gz 2024-09-19T08:48:22.877 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53778.log 2024-09-19T08:48:22.878 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.42873.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42873.log.gz 2024-09-19T08:48:22.879 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51982.log 2024-09-19T08:48:22.880 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.53778.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53778.log.gz 2024-09-19T08:48:22.880 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35002.log 2024-09-19T08:48:22.881 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51982.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51982.log.gz 2024-09-19T08:48:22.881 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53247.log 2024-09-19T08:48:22.882 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.35002.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35002.log.gz 2024-09-19T08:48:22.882 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55284.log 2024-09-19T08:48:22.883 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.53247.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53247.log.gz 2024-09-19T08:48:22.884 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43833.log 2024-09-19T08:48:22.884 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.55284.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55284.log.gz 2024-09-19T08:48:22.885 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50653.log 2024-09-19T08:48:22.886 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43833.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43833.log.gz 2024-09-19T08:48:22.886 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45921.log 2024-09-19T08:48:22.887 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.50653.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50653.log.gz 2024-09-19T08:48:22.887 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51187.log 2024-09-19T08:48:22.889 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.45921.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.45921.log.gz 2024-09-19T08:48:22.889 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41415.log 2024-09-19T08:48:22.890 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.51187.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51187.log.gz 2024-09-19T08:48:22.890 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38150.log 2024-09-19T08:48:22.891 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.41415.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41415.log.gz 2024-09-19T08:48:22.892 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53603.log 2024-09-19T08:48:22.893 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.38150.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38150.log.gz 2024-09-19T08:48:22.893 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43866.log 2024-09-19T08:48:22.894 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.53603.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53603.log.gz 2024-09-19T08:48:22.894 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34501.log 2024-09-19T08:48:22.895 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.43866.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43866.log.gz 2024-09-19T08:48:22.895 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35459.log 2024-09-19T08:48:22.896 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.34501.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34501.log.gz 2024-09-19T08:48:22.897 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49275.log 2024-09-19T08:48:22.897 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.35459.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35459.log.gz 2024-09-19T08:48:22.898 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52903.log 2024-09-19T08:48:22.899 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.49275.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49275.log.gz 2024-09-19T08:48:22.899 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53294.log 2024-09-19T08:48:22.900 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.52903.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52903.log.gz 2024-09-19T08:48:22.900 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35201.log 2024-09-19T08:48:22.901 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.53294.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53294.log.gz 2024-09-19T08:48:22.901 INFO:teuthology.orchestra.run.smithi098.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39389.log 2024-09-19T08:48:22.902 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.35201.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35201.log.gz 2024-09-19T08:48:22.904 INFO:teuthology.orchestra.run.smithi098.stderr:/var/log/ceph/ceph-client.admin.39389.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39389.log.gz 2024-09-19T08:48:23.054 INFO:teuthology.orchestra.run.smithi139.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-19T08:48:23.089 INFO:teuthology.orchestra.run.smithi098.stderr: 93.0% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-19T08:48:23.133 INFO:teuthology.orchestra.run.smithi139.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-19T08:48:23.167 INFO:teuthology.orchestra.run.smithi139.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-19T08:48:23.243 INFO:teuthology.orchestra.run.smithi139.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-19T08:48:23.244 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-09-19T08:48:23.244 INFO:teuthology.orchestra.run.smithi139.stderr:real 0m0.859s 2024-09-19T08:48:23.244 INFO:teuthology.orchestra.run.smithi139.stderr:user 0m4.032s 2024-09-19T08:48:23.244 INFO:teuthology.orchestra.run.smithi139.stderr:sys 0m0.192s 2024-09-19T08:48:23.292 INFO:teuthology.orchestra.run.smithi098.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-19T08:48:23.327 INFO:teuthology.orchestra.run.smithi098.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-19T08:48:23.353 INFO:teuthology.orchestra.run.smithi098.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-19T08:48:23.416 INFO:teuthology.orchestra.run.smithi098.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-19T08:48:23.427 INFO:teuthology.orchestra.run.smithi098.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-19T08:48:23.562 INFO:teuthology.orchestra.run.smithi098.stderr: 93.6% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-19T08:48:23.564 INFO:teuthology.orchestra.run.smithi098.stderr: 2024-09-19T08:48:23.564 INFO:teuthology.orchestra.run.smithi098.stderr:real 0m1.197s 2024-09-19T08:48:23.564 INFO:teuthology.orchestra.run.smithi098.stderr:user 0m6.456s 2024-09-19T08:48:23.564 INFO:teuthology.orchestra.run.smithi098.stderr:sys 0m0.383s 2024-09-19T08:48:23.565 INFO:tasks.ceph:Archiving logs... 2024-09-19T08:48:23.565 DEBUG:teuthology.misc:Transferring archived files from smithi098:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911853/remote/smithi098/log 2024-09-19T08:48:23.566 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T08:48:23.863 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-09-19T08:48:23.863 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2024-09-19T08:48:23.863 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-09-19T08:48:23.863 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-09-19T08:48:24.766 DEBUG:teuthology.misc:Transferring archived files from smithi139:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911853/remote/smithi139/log 2024-09-19T08:48:24.767 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T08:48:25.232 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-19T08:48:25.248 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-19T08:48:25.248 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T08:48:25.251 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T08:48:25.317 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-19T08:48:25.327 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-19T08:48:25.327 DEBUG:teuthology.orchestra.run.smithi098:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T08:48:25.330 DEBUG:teuthology.orchestra.run.smithi139:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T08:48:25.340 INFO:teuthology.orchestra.run.smithi098.stdout: remote refid st t when poll reach delay offset jitter 2024-09-19T08:48:25.340 INFO:teuthology.orchestra.run.smithi098.stdout:============================================================================== 2024-09-19T08:48:25.340 INFO:teuthology.orchestra.run.smithi098.stdout:*hv01.front.sepi 67.205.162.81 3 u 37 64 377 0.087 -3.464 0.338 2024-09-19T08:48:25.340 INFO:teuthology.orchestra.run.smithi098.stdout:+hv02.front.sepi 63.231.80.2 3 u 37 64 377 0.087 +2.221 0.657 2024-09-19T08:48:25.341 INFO:teuthology.orchestra.run.smithi098.stdout:+hv03.front.sepi 74.6.168.72 3 u 35 64 377 0.062 -3.973 0.363 2024-09-19T08:48:25.341 INFO:teuthology.orchestra.run.smithi098.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-09-19T08:48:25.342 INFO:teuthology.orchestra.run.smithi139.stdout: remote refid st t when poll reach delay offset jitter 2024-09-19T08:48:25.342 INFO:teuthology.orchestra.run.smithi139.stdout:============================================================================== 2024-09-19T08:48:25.342 INFO:teuthology.orchestra.run.smithi139.stdout:+hv01.front.sepi 67.205.162.81 3 u 30 64 377 0.082 -7.110 3.401 2024-09-19T08:48:25.342 INFO:teuthology.orchestra.run.smithi139.stdout:+hv02.front.sepi 63.231.80.2 3 u 30 64 377 0.085 +2.260 2.342 2024-09-19T08:48:25.343 INFO:teuthology.orchestra.run.smithi139.stdout:*hv03.front.sepi 74.6.168.72 3 u 28 64 377 0.087 -2.642 2.947 2024-09-19T08:48:25.343 INFO:teuthology.orchestra.run.smithi139.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-09-19T08:48:25.343 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-19T08:48:25.359 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-19T08:48:25.360 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-19T08:48:25.372 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-19T08:48:25.383 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-19T08:48:25.395 INFO:teuthology.task.internal:Duration was 1280.617811 seconds 2024-09-19T08:48:25.395 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-19T08:48:25.406 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-19T08:48:25.406 DEBUG:teuthology.orchestra.run.smithi098:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T08:48:25.408 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T08:48:25.446 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-19T08:48:25.447 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi098.front.sepia.ceph.com 2024-09-19T08:48:25.447 DEBUG:teuthology.orchestra.run.smithi098:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T08:48:25.497 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi139.front.sepia.ceph.com 2024-09-19T08:48:25.497 DEBUG:teuthology.orchestra.run.smithi139:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T08:48:25.508 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-19T08:48:25.508 DEBUG:teuthology.orchestra.run.smithi098:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T08:48:25.541 DEBUG:teuthology.orchestra.run.smithi139:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T08:48:25.575 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-19T08:48:25.576 DEBUG:teuthology.orchestra.run.smithi098:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T08:48:25.617 DEBUG:teuthology.orchestra.run.smithi139:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T08:48:25.695 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-19T08:48:25.705 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-19T08:48:25.705 DEBUG:teuthology.orchestra.run.smithi098:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T08:48:25.715 DEBUG:teuthology.orchestra.run.smithi139:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T08:48:25.729 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-19T08:48:25.739 DEBUG:teuthology.orchestra.run.smithi098:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T08:48:25.761 DEBUG:teuthology.orchestra.run.smithi139:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T08:48:25.768 INFO:teuthology.orchestra.run.smithi098.stdout:kernel.core_pattern = core 2024-09-19T08:48:25.783 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern = core 2024-09-19T08:48:25.796 DEBUG:teuthology.orchestra.run.smithi098:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T08:48:25.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:48:25.827 DEBUG:teuthology.orchestra.run.smithi139:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T08:48:25.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T08:48:25.841 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-19T08:48:25.854 INFO:teuthology.task.internal:Transferring archived files... 2024-09-19T08:48:25.855 DEBUG:teuthology.misc:Transferring archived files from smithi098:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911853/remote/smithi098 2024-09-19T08:48:25.856 DEBUG:teuthology.orchestra.run.smithi098:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T08:48:25.893 DEBUG:teuthology.misc:Transferring archived files from smithi139:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911853/remote/smithi139 2024-09-19T08:48:25.894 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T08:48:25.914 INFO:teuthology.task.internal:Removing archive directory... 2024-09-19T08:48:25.914 DEBUG:teuthology.orchestra.run.smithi098:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T08:48:25.933 DEBUG:teuthology.orchestra.run.smithi139:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T08:48:25.957 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-19T08:48:25.968 INFO:teuthology.task.internal:Not uploading archives. 2024-09-19T08:48:25.968 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-19T08:48:25.979 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-19T08:48:25.979 DEBUG:teuthology.orchestra.run.smithi098:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T08:48:25.981 DEBUG:teuthology.orchestra.run.smithi139:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T08:48:25.986 INFO:teuthology.orchestra.run.smithi098.stdout: 658637 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 08:48 /home/ubuntu/cephtest 2024-09-19T08:48:26.005 INFO:teuthology.orchestra.run.smithi139.stdout: 658633 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 08:48 /home/ubuntu/cephtest 2024-09-19T08:48:26.007 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-19T08:48:26.018 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-19T08:48:26.043 INFO:teuthology.run:Summary data: description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}} duration: 1280.6178114414215 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-19T08:48:26.043 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T08:48:26.135 INFO:teuthology.run:pass